From nobody Tue Apr 7 19:54:44 2026 Received: from SJ2PR03CU001.outbound.protection.outlook.com (mail-westusazon11012031.outbound.protection.outlook.com [52.101.43.31]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7A0A7358375; Wed, 11 Mar 2026 20:35:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.43.31 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773261335; cv=fail; b=XUF4asfE7CWOgmtWCJcGao3iWaJ/Cq77C66qoqMGEls8zkOsiVZL7Z4Bz9KN/Qt1DCspC8mNMR5JQigODtYcAImXhBs5mA/1zfBQOoQ2irJODxX/ZU7D3rQDyMo/BQiwEE/j+tA9mJcscZOKABFCJSTMlIrGJmFLotBNS+rbcis= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773261335; c=relaxed/simple; bh=3nq8XY2OV3X098s0CUAit0TepHy7Ig8vVAkhyLZlfY0=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=HekPTtBK1hRjgaa/2c0vX9ZtgiIfSRopowNK4mK86KCwdmsbSmnCpaws6KsnNrB+kaQSkeY3AzD/MQ49Sn3uCzBp7Fww8PtI35HDt0075bq3OKXetYx6WgM/qjhJe56wtUV7zU3Q736yGGw73za0z8rK2NbY0WAePaYY0ByWwyE= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=eV1HxA6L; arc=fail smtp.client-ip=52.101.43.31 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="eV1HxA6L" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=oEzCxlVo1YxCz9QAlDNjCzpHqHkbff3+VIH7kK66qsvejs8VQrY2/hmqeDo6oPFpUnT3zJYb+UETNEMzRoDltMBX06t2QYtb51eEV9HOjet/GaNtN9xsBPJEnBNeGrcylFa0VvQSX2kpD9cuBNBny7XOQkpGURq64D40KzqYfDSfJxCjnCVLGb38phReoGeKVB/FBnWmr5JGTsuAfBEsZyxLqpaFPxA/42WooNN4oWkU6HqNs0lb/AdcXmrJv7qtQ+uYhM9ZHooHv/ZNW7bGe7Y5PT3JYoBTxQwzP6OYfCepLWpVMJk1qe6Yrkqs3G/LdxFMJzdgaXTTzKTgJG4HaQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=/Osg+XKa6H5aPMgxAt/rEU3OitwvLW1Z0Dy4kp30Qvg=; b=X+KqgbO/x8PRtde2h/yALeCUUG5vgoDxtCTnySSILAxNwZFHSCDWWniNu7WCXjXghP1H/P5BfKXPIp4zzUA39ZuNEQYKhDTCsSgYHY+FFiJETUWQsBF96SZU9q6J91arEwHag46+wAWVjHMlRJ5vSD3q0/08J/B3pNpvd6IKRVgJclh3rVEjqJjse/LHRgvX7mHqtk8V7XhQqKRr529mR1IWLta5G4MwVV9vVDOaLG6coYXAjU8SxBmZkOTb/2Vzl90LWuHCWskLu3OMe5TBSA8lk2AA726mRgN7rD65/pp74jqEmNEBP6Ba4pnM5KZ/B/z9lwPe70a46D1BAaiR9A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=/Osg+XKa6H5aPMgxAt/rEU3OitwvLW1Z0Dy4kp30Qvg=; b=eV1HxA6LopMio6it4W2YR5G/7m19anL/LGCTFcwQdoCP9jH4iA4RcC2EmNvfQqiH8aTF3TA1fwuWqlxv+sM3Yp2MDSLepfBoTjqzcJbrYxqOe+BB8eitfCgkq7iV+/s3Z40RRg3ajC3AvPnTzIJo4g5sRyOzRwWEhP32jyv1BSQT6xK/arB9wqhGjcm1+3cHzPoEtCowb7PJnGvh3qCf9U3Zgr3wxRUTPPA2zBxLgLuEwcplxtViQZrsEuVI32DOTxLV3zIDjSnq/484iY8d9EQ4NprbK1aLo1fWD1daJKcT7vNNT7VsQkhAVaR3xbgA37tEIHD/CZpNmjS9tBI5Zw== Received: from MN0PR03CA0001.namprd03.prod.outlook.com (2603:10b6:208:52f::28) by CH1PPFD8936FA16.namprd12.prod.outlook.com (2603:10b6:61f:fc00::624) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9723.3; Wed, 11 Mar 2026 20:35:29 +0000 Received: from BL6PEPF00022574.namprd02.prod.outlook.com (2603:10b6:208:52f:cafe::78) by MN0PR03CA0001.outlook.office365.com (2603:10b6:208:52f::28) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9678.26 via Frontend Transport; Wed, 11 Mar 2026 20:35:21 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by BL6PEPF00022574.mail.protection.outlook.com (10.167.249.42) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9654.16 via Frontend Transport; Wed, 11 Mar 2026 20:35:29 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Wed, 11 Mar 2026 13:35:05 -0700 Received: from rnnvmail201.nvidia.com (10.129.68.8) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Wed, 11 Mar 2026 13:35:04 -0700 Received: from nvidia-4028GR-scsim.nvidia.com (10.127.8.11) by mail.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20 via Frontend Transport; Wed, 11 Mar 2026 13:34:57 -0700 From: To: , , , , , , , , , , , , , , , , , CC: , , , , , , , Subject: [PATCH 01/20] cxl: Introduce cxl_get_hdm_reg_info() Date: Thu, 12 Mar 2026 02:04:21 +0530 Message-ID: <20260311203440.752648-2-mhonap@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20260311203440.752648-1-mhonap@nvidia.com> References: <20260311203440.752648-1-mhonap@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL6PEPF00022574:EE_|CH1PPFD8936FA16:EE_ X-MS-Office365-Filtering-Correlation-Id: 1fcae79f-0c6a-4c28-58c3-08de7fadbc31 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|82310400026|7416014|376014|36860700016|921020|56012099003|22082099003|18002099003; X-Microsoft-Antispam-Message-Info: s2gV3E4BhalTfvv37DPfSLKpf6Wo2e95ojmWnb/xRE5etLRm7z4HgRDdTQRr7QOO1/yyQuab+aaru5oYX4Z+is1tZj5qpQdw1ueu6Gj41K6SpZEJ5+IDClMV6+dpoN4maUWESD7QQRmy7RKWGSNVdRRQoNsuxeqmepqpx6wonHm+TU8bLrjES9EOAVNcAIxELcVHjjf3f8R5e5HwTsAsNGcpReS4cAAzJYcXf3ntr5Y0R6DHkWCWTRlDfv2jJD5j+v0F59oOe4hg0n0V93zZ0tpk3XvCwOY2qD694+sGYyhn8odlKyo4lYwlXWQlcYT3klYRr457NXZ61Xs7QTPYuAtCVwAJYtScb3pfPeuJ/xQROD3nIMxWr1POu9FJPjnmj52Wca3Wjn7iiC2ZS6XK+h6A1Ic0I8/ArsQ319Asmi7UzzhqE0acvBO53fIUwzQk2Ypd3fAYeUKY9g+IPQu2Y72zKPDCJ0lhvltUyKIkTfg83NEJRNPvc5rMtrHstxyQvMfSUD4UAXn9Ky293jkLy64Wp+4jUjQY6rO4NoqIz2z/vS3QLy+xCOsTQ2RUYjn7hJv3xXy4TFLRZOviMckCRbNsNKPAF3Sfz4ox0yPE3K5WB9DJZjjoXti5QJpD8VIOvhBkD4am8kNBcmy6u6RKw5ALFg8eEc04i2qhyJr/TE24np/14t/tox7wGnjSFfRemXSDu3Pt+dfun7lmsBXfYnlElB+XBbdKox/N3JlbNerG9cJ2eaJLKLsGkVICxxMfVjyhN9eXmGbw0H45fQ8QCorL2lyhas5vJXSgiKpCamKfNa6Qg4gBR/Brj6bjJ5B7 X-Forefront-Antispam-Report: CIP:216.228.117.160;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge1.nvidia.com;CAT:NONE;SFS:(13230040)(1800799024)(82310400026)(7416014)(376014)(36860700016)(921020)(56012099003)(22082099003)(18002099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: ZSjH68ZT1uBYukep+B0+tXW1gm0yhAbzAtiMfgSRQ6n4QWfaZ49VvhRKmPxaQSbcFXcJhOYvU40H75OaTStnC1bKIWHTb+2g2OofKtmtxxOHuA3AGvrLeMtiSQwaK51vNc/B8eb3dll+OIpMiRzihcRA6PnZWq0BkcxDHU6fNOqbEox7f3gkOMdNoh7c12vmcrkWK5Mknqq6Bjtfjma2JgSb/Ou+iAVdEoGgobfT7dKTCHEo6j+e4N5R6LC2KkOdDWmEJq2JuOWB+5ydBy7hiVhsf/IIkwbyYQ/J9+MhGh2O8yT5Cg+dDvy53CKdLOHkwT0WLx7cobQZVLDukhASVQiYdWHRCsNJNdP+w3VdHeLzm9KhmnYWQ0xdCxN14JggmhYnyx6vJG4ndtfoaq1Uvxlls64iFxVLmUXF2oljyZ4wa3lrHiNdChr3kVBK47J/ X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Mar 2026 20:35:29.4664 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 1fcae79f-0c6a-4c28-58c3-08de7fadbc31 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BL6PEPF00022574.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH1PPFD8936FA16 Content-Type: text/plain; charset="utf-8" From: Manish Honap CXL core has the information of what CXL register groups a device has. When initializing the device, the CXL core probes the register groups and saves the information. The probing sequence is quite complicated. vfio-cxl requires the HDM register information to emulate the HDM decoder registers. Introduce cxl_get_hdm_reg_info() for vfio-cxl to leverage the HDM register information in the CXL core. Thus, it doesn't need to implement its own probing sequence. Co-developed-by: Zhi Wang Signed-off-by: Zhi Wang Signed-off-by: Manish Honap --- drivers/cxl/core/pci.c | 45 ++++++++++++++++++++++++++++++++++++++++++ include/cxl/cxl.h | 3 +++ 2 files changed, 48 insertions(+) diff --git a/drivers/cxl/core/pci.c b/drivers/cxl/core/pci.c index ba2d393c540a..52ed0b4f5e78 100644 --- a/drivers/cxl/core/pci.c +++ b/drivers/cxl/core/pci.c @@ -449,6 +449,51 @@ int cxl_hdm_decode_init(struct cxl_dev_state *cxlds, s= truct cxl_hdm *cxlhdm, } EXPORT_SYMBOL_NS_GPL(cxl_hdm_decode_init, "CXL"); =20 +/** + * cxl_get_hdm_reg_info - Get HDM decoder register block location and count + * @cxlds: CXL device state (must have component regs enumerated) + * @count: Output: number of HDM decoders (from DVSEC cap). Only set when + * the device has a valid HDM decoder capability. + * @offset: Output: byte offset of the HDM decoder register block within t= he + * component register BAR. Only set when valid. + * @size: Output: size in bytes of the HDM decoder register block. Only set + * when valid. + * + * Reads the CXL component register map and DVSEC capability to return the + * Host Managed Device Memory (HDM) decoder register block offset and size, + * and the number of HDM decoders. This function requires cxlds->cxl_dvsec + * to be non-zero. + * + * Return: 0 on success. A negative errno is returned when config read + * failure or when the decoder registers are not valid. + */ +int cxl_get_hdm_reg_info(struct cxl_dev_state *cxlds, u32 *count, + resource_size_t *offset, resource_size_t *size) +{ + struct pci_dev *pdev =3D to_pci_dev(cxlds->dev); + struct cxl_component_reg_map *map =3D + &cxlds->reg_map.component_map; + int d =3D cxlds->cxl_dvsec; + u16 cap; + int rc; + + /* HDM decoder registers not implemented */ + if (!map->hdm_decoder.valid || !d) + return -ENODEV; + + rc =3D pci_read_config_word(pdev, + d + PCI_DVSEC_CXL_CAP, &cap); + if (rc) + return rc; + + *count =3D FIELD_GET(PCI_DVSEC_CXL_HDM_COUNT, cap); + *offset =3D map->hdm_decoder.offset; + *size =3D map->hdm_decoder.size; + + return 0; +} +EXPORT_SYMBOL_NS_GPL(cxl_get_hdm_reg_info, "CXL"); + #define CXL_DOE_TABLE_ACCESS_REQ_CODE 0x000000ff #define CXL_DOE_TABLE_ACCESS_REQ_CODE_READ 0 #define CXL_DOE_TABLE_ACCESS_TABLE_TYPE 0x0000ff00 diff --git a/include/cxl/cxl.h b/include/cxl/cxl.h index 50acbd13bcf8..8456177b523e 100644 --- a/include/cxl/cxl.h +++ b/include/cxl/cxl.h @@ -284,4 +284,7 @@ int cxl_dpa_free(struct cxl_endpoint_decoder *cxled); struct cxl_region *cxl_create_region(struct cxl_root_decoder *cxlrd, struct cxl_endpoint_decoder **cxled, int ways); + +int cxl_get_hdm_reg_info(struct cxl_dev_state *cxlds, u32 *count, + resource_size_t *offset, resource_size_t *size); #endif /* __CXL_CXL_H__ */ --=20 2.25.1 From nobody Tue Apr 7 19:54:44 2026 Received: from DM5PR21CU001.outbound.protection.outlook.com (mail-centralusazon11011057.outbound.protection.outlook.com [52.101.62.57]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2A3F935DA79; Wed, 11 Mar 2026 20:35:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.62.57 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773261336; cv=fail; b=ef04Pho14+bCOXFs1WKDqQx4gEMPjaRwjpKFz8BVhguiyP+i72RiHQdmVDFr2x1wfy37fmjNgPam/vk6cQssXNtZB7pxGY8v8gq5TVZR61H281RIJKc/bHJBwGfjFAwQ5mBgLIQl7LPNTuzXf0iOvQAjYYaF02jTD7fl9iUtspk= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773261336; c=relaxed/simple; bh=kQe9KEg/5yL4UbO0NYsWjqCszD7Pd/aYg2mQXn4j/BQ=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=LNotnXusUMLypLwfc1oKAmVneaop56EsbpVDOgcABgJ2o1daH8wTbv85kLco+36VS3gxnszB1VJz1ZDF8/H/Z6g2IKU0JBm5yt4IdLU62fXj5wpGO/u0dEmv0BcJ+fImzm2Qcjwl+NHB35wkVyoF4bUhsHRq+RvA1eef7NRG1tw= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=JxggK+qs; arc=fail smtp.client-ip=52.101.62.57 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="JxggK+qs" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=JRZujm+M/tLSEmdeF8rXRaHzJMSkiDM72TGIeQcnMbRXzWkrPoEYleUSdW/6iuVxDxbSvDY8NTLRPPZ7cLaza8CNbiyMz1/aVWZVFFWSBjD+ddQ5bsAKMGUp1y23b1EvQ7v5ZFBut726vsb8fxzYSRW+qrkxosL36KosimZGz2rov9fziE2+ZXuri/1W7HVzOhNv3Iq+W4Gqgf/y3CionS63ySk+zNIjOjvgS3JM2Pffrz7EIO12RLCSL1IkJpzQ4ITNaLCsZNjqS8StLnPwhZ/Fh97VHPAczUEVF50lKiuc3xEgSHxbT5cC/gvE4v+b0CvAtdjUuYQWXa7xp5E5+Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=uGtVFsa8gK7uoYQyFpx7itmrfUYgqdSq+/1HvufG8m8=; b=H9UwhRV9/Bp78F1XDAVu3bSjxc2zpp3SBxHu/OzjprZxyce+8t3QE4KAMGuud2xYY2A4wOT3N0D6Qz9WMo604TvHMScBkDf7hXAkbHEDDUqw1P4RLhzhsy3yGV4IPWWE98Sj2eFekf9pgE3CAQ+zgTfn0DQbQlu9ZwIl0ZtJGavPHxZ01Dq61wEqi3kbDcKcqrSVqh0Y1PuPHwsgJv33Av4p1G9dinZG9Rkdb3PxMr21J7YJLALHSPeGt0BEtyEIbdm6hSCjOQvWumyoUGFF4YSyNb/SIjSbjvzk9wubjxG7PI8GXyzR2wvTUfw3+PdaZO9gVkXah/iRWRK22+6r+A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=uGtVFsa8gK7uoYQyFpx7itmrfUYgqdSq+/1HvufG8m8=; b=JxggK+qsgaJ6p/3vljmlUNNvWU305qrjjzjWjE57/Z/OUEbmLYZ5wgpDOAJmznXz+DjBinUoIdxj3uJVIc6t2Yv5usoInN4oQVDXLousZKJkuYLDwGPmaeQa30a6697hElE6bdbMYaptaMTY7rbrNuv82WX3IOzp7HEV7WGWWzQi9Gw0TMnYdnpmn1FrED/1xUD7kpOFiF/jW4W69hqigHOHFMjjc6Wax/Nv24cubDQsVO2lBUkmZDu8VFmUAAaCXEeywauy8vrr37NMWwKEuhMdOQjMpptSIE2EhSUPbruy/Le/heQCYTQXbI/YBof3hqThfG2XswRTQxjEj01LUQ== Received: from DS7PR05CA0056.namprd05.prod.outlook.com (2603:10b6:8:2f::17) by DS0PR12MB8272.namprd12.prod.outlook.com (2603:10b6:8:fc::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9723.4; Wed, 11 Mar 2026 20:35:31 +0000 Received: from DS2PEPF00003445.namprd04.prod.outlook.com (2603:10b6:8:2f:cafe::33) by DS7PR05CA0056.outlook.office365.com (2603:10b6:8:2f::17) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9678.24 via Frontend Transport; Wed, 11 Mar 2026 20:35:31 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by DS2PEPF00003445.mail.protection.outlook.com (10.167.17.72) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9678.18 via Frontend Transport; Wed, 11 Mar 2026 20:35:31 +0000 Received: from rnnvmail203.nvidia.com (10.129.68.9) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Wed, 11 Mar 2026 13:35:12 -0700 Received: from rnnvmail201.nvidia.com (10.129.68.8) by rnnvmail203.nvidia.com (10.129.68.9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Wed, 11 Mar 2026 13:35:12 -0700 Received: from nvidia-4028GR-scsim.nvidia.com (10.127.8.11) by mail.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20 via Frontend Transport; Wed, 11 Mar 2026 13:35:05 -0700 From: To: , , , , , , , , , , , , , , , , , CC: , , , , , , , Subject: [PATCH 02/20] cxl: Expose cxl subsystem specific functions for vfio Date: Thu, 12 Mar 2026 02:04:22 +0530 Message-ID: <20260311203440.752648-3-mhonap@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20260311203440.752648-1-mhonap@nvidia.com> References: <20260311203440.752648-1-mhonap@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS2PEPF00003445:EE_|DS0PR12MB8272:EE_ X-MS-Office365-Filtering-Correlation-Id: 7c79f25a-707a-49b7-cd81-08de7fadbd71 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|82310400026|376014|36860700016|7416014|921020|22082099003|56012099003|18002099003; X-Microsoft-Antispam-Message-Info: od3DbQD6rXBt7SgZwdHeXGgvDvmwv8pjto+awTF+Fa/CbxtBaWnbTi1A71aiH5jyLjVpfZlktaa2kMkEYCgpBfgolEmqpN2OMuKYx2tShy8wgkQXZ78r0NABLwtd5cT0cJpbbsmtTgdkKsJDcA7lbJX1wY75bVOvmO/rzQEUhI6aUFJZSDLzPaMWDQwPuCl5L+oLlNKwkq+0KEjqbJgtYoRe30966EjD9+8DMoXy2grq99pGpATdWa3sunGvYfgZYrl47TKX6xQQm6/73RDRHJJ/ezbPumEw/zM+gyu18AUbLLUmrKg2hBKVIRl/sQGY2ZsOX1ZTrEdE3pFYayulDWTVfknaNEpyyBSTM788oKp9Ryh1sk5fhjMshoRurTLHHAjuuFoIL968llfNgw5hO2oU/bzQWMsKnuh3fDenvVTtP5iET2eRcJXeiQJHDm8oLAz4yrtqxuAMe4XEeZl6TqcolTKCZ5hdvw8qddyynI5tP3pwMor/D1LP+/aYq7RlIQjyA8aFTBNCcrFIfj4Rn1XsTxQZhw/H978rt5Xags53UoXV2pK+zbHPPNMtt79uKAdiRLzZ6XStjJ/b6PxklLtC87AD/GheoqK1YmvmseIHzBGm2EGr2W1FgAbz8yiHEqUgBDbYyoZKl5GLpWMQo2hoXTzHGZcXq7XjjP9q8utGJPVeJMdJGinGYFdFM9paEdsEDrmolSvRp2tesxGGLjG2E4P77Q7kvjpwel05OIBu+HDn0kTx5VRHujahtdvXQXjvm1i1FjvDPLqoQmSqVHSdD0ZuPIZLLkQtRw0An12paaZqpUckluyGxygnVZGx X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(1800799024)(82310400026)(376014)(36860700016)(7416014)(921020)(22082099003)(56012099003)(18002099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: o/Lj2iioPbZNCv/AGQOF3DG16Z/G33ri0nNGyf/+hamba9Jx8BSgMPlW9mkHxXDW3sg7dX/rMiWtxcNq2h8+Pwz6xp/MOPVAce609k1vwZ+htrmUxPPrh0AbVipEKyPQtKPtVesj6lvn4I7NXgvJ2M13Wwc5nqcTLaBn8FrdHOQ+Wjh5uqxbWdNOf71Nxs8axK2fxn9buCvmtJddc9XFLMW/Q7/0nuCg2ZPRch44XE9k6nXAtJpaNKuE+g9+k97vX1KHU2WZbjse2czK/t1GQClLjuNba1rha9yoqOwxp0j4ZqyGtP3YfIZpLwirATY5sMMqgZM+C0Ronv2a0eguTFa+qaBHob01/lFA/OFe/4Wl2NBmTHav8cEAjf5haPrjLs0EHKAoFwlnhr/LjMSPtMc4ryZ3FSEpPO7EH/XFESzubMLqujWWkd3BkBXKq0VU X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Mar 2026 20:35:31.6072 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 7c79f25a-707a-49b7-cd81-08de7fadbd71 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DS2PEPF00003445.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR12MB8272 Content-Type: text/plain; charset="utf-8" From: Manish Honap Below functions from CXL subsystem will be required in vfio-cxl for supporting the type-2 device passthrough: cxl_find_regblock - To find component registers cxl_probe_component_regs - Probe HDM/RAS capabilities Make these functions available via declaring them from include header instead of subsystem-specific header. Signed-off-by: Manish Honap --- drivers/cxl/cxl.h | 4 ---- include/cxl/cxl.h | 7 +++++++ 2 files changed, 7 insertions(+), 4 deletions(-) diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index 2b1f7d687a0e..10ddab3949ee 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -198,8 +198,6 @@ static inline int ways_to_eiw(unsigned int ways, u8 *ei= w) #define CXLDEV_MBOX_BG_CMD_COMMAND_VENDOR_MASK GENMASK_ULL(63, 48) #define CXLDEV_MBOX_PAYLOAD_OFFSET 0x20 =20 -void cxl_probe_component_regs(struct device *dev, void __iomem *base, - struct cxl_component_reg_map *map); void cxl_probe_device_regs(struct device *dev, void __iomem *base, struct cxl_device_reg_map *map); int cxl_map_device_regs(const struct cxl_register_map *map, @@ -211,8 +209,6 @@ enum cxl_regloc_type; int cxl_count_regblock(struct pci_dev *pdev, enum cxl_regloc_type type); int cxl_find_regblock_instance(struct pci_dev *pdev, enum cxl_regloc_type = type, struct cxl_register_map *map, unsigned int index); -int cxl_find_regblock(struct pci_dev *pdev, enum cxl_regloc_type type, - struct cxl_register_map *map); int cxl_setup_regs(struct cxl_register_map *map); struct cxl_dport; int cxl_dport_map_rcd_linkcap(struct pci_dev *pdev, struct cxl_dport *dpor= t); diff --git a/include/cxl/cxl.h b/include/cxl/cxl.h index 8456177b523e..610711e861d4 100644 --- a/include/cxl/cxl.h +++ b/include/cxl/cxl.h @@ -287,4 +287,11 @@ struct cxl_region *cxl_create_region(struct cxl_root_d= ecoder *cxlrd, =20 int cxl_get_hdm_reg_info(struct cxl_dev_state *cxlds, u32 *count, resource_size_t *offset, resource_size_t *size); +struct pci_dev; +enum cxl_regloc_type; +int cxl_find_regblock(struct pci_dev *pdev, enum cxl_regloc_type type, + struct cxl_register_map *map); +void cxl_probe_component_regs(struct device *dev, void __iomem *base, + struct cxl_component_reg_map *map); + #endif /* __CXL_CXL_H__ */ --=20 2.25.1 From nobody Tue Apr 7 19:54:44 2026 Received: from MW6PR02CU001.outbound.protection.outlook.com (mail-westus2azon11012069.outbound.protection.outlook.com [52.101.48.69]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 37BBF359A7B; Wed, 11 Mar 2026 20:35:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.48.69 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773261354; cv=fail; b=syveazfpG6//aibicxGWcT2n2VTNPYw5Ww3qX+ar92deP9jYUqiCDSt8dcKh9hgehrQ++6FxA6nJCI1o5Y9aboTcIK3MZx8zInGnZrWQa0QQ/tDOTP2+z3tQsmxbqbXslFn1YFLcvxjKqR0oogsp54m1uyiBKd0Gtuumi9dnYx8= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773261354; c=relaxed/simple; bh=94+aV+edETR7Q+XmNp5Ed5W9sSqdkR5LlBL+zCIJXXo=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=YsG+kuHHY0Al4ZsxDIGg61zz4BAGrs3qgTyGMPm4bpJF4VoSv295D7NMRGDEUnqz1pOzPth/JuBjbPTTu1eLw5256j94I2cTTffbwLI+rVHvLyYdZDAGrUM/VTxra2xM3OPRGgpBnf9l/ydZqyI6voiiG0q4a2PLcWHp2KLo/7U= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=R0SwD7rh; arc=fail smtp.client-ip=52.101.48.69 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="R0SwD7rh" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=BFOsVco+FbWdCXEIVjM6a7zU/miRMJ5AzuivdUG1mkGSgLT8oq/7VkikyVPkB5eowHFKQb5o1IbLbv18E5chyFyK4pzLfVEEmk4qwKUiSY6oVinQTN4eFOKfPuv7eXQbciAg1dSZIaPwYGxYFn4RT1tM6mLgdL2VlM1fYr6j/1mQMDuOI7fOLH2FqBbnhP0KkuCKy6urPmRde02gGBFBy3f5hBVVSWdggNwtKduC2kv786+56tqz3kW1TbOwDI0fORM/db8Dyo8dAn2vnIitDPMzJn81VYpXYgg1t4jTzMwk/ZyMPBCleAGeVgMvB+wwzdpWAeM7xeqbv0ZiPQzYrQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=l8QHt4U1h+QCVajNxwjjHy2sS8GW7Gsrn0vEHBbQNTM=; b=yX1E3nmjl8AYWNl0rQSXGDER5JVDuLSCBIR/Q1bPj+GtCzLZhdgIMtQmJMa9FXTS1XzUCvomgwIAFmZZe09F2rgvdq13ton3Q+uovd6rw+TWnCSbNMupLQng7T2GL4TINypTsEk1BXDlc19xgM26TXnY+sDVaUl6P4A702U0X3ZQaj6H87F4rFbSfFPNt1bGzD+KfUJX0RGMdZFvFaUoXVW8QW16Plu4/3q2dRKlkpEeM856vIWusb/Jz12KnF9V/odzGzsm1iwcA8aWeu8/aJQzk1rVHExHAwVe6eAZRDCwlmScoZ+N71qxCR8UyqHYw3uqElJz5B6pcQ6PnTZx3w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=l8QHt4U1h+QCVajNxwjjHy2sS8GW7Gsrn0vEHBbQNTM=; b=R0SwD7rh6R3WcOiNJbeF83b+xnnTvrwjik5W2UxDp088hoaXjjTNh3uUu7Kv5IowLjezsCzsRxAsYMrbemDNk13mfqY94Xv9kp4igmNYHro/GQYyYSeqZ1jIdOLy9vCjI3rpoZPQXTv5IaIau4IAN5MNWhYBkd/AwgITO5WmxVRl8ZKh2clC2zO/0Y996Fye/JLufDHjPXLxsdEJrK6japu9lFGqB8LEE0OsZkW5fIMToh1nFvnGSeMCn0ZC1j+g1l2reWBpWcvrcpERFn4S9ZTMWB7W9XJJeZSjhPLxUN3tKYI3aWo2ZTNknrBy2CDGqahHP+ck3sTXND/aQBM2eQ== Received: from SN7PR04CA0079.namprd04.prod.outlook.com (2603:10b6:806:121::24) by SJ2PR12MB8739.namprd12.prod.outlook.com (2603:10b6:a03:549::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9678.18; Wed, 11 Mar 2026 20:35:44 +0000 Received: from SN1PEPF0002636A.namprd02.prod.outlook.com (2603:10b6:806:121:cafe::40) by SN7PR04CA0079.outlook.office365.com (2603:10b6:806:121::24) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9678.26 via Frontend Transport; Wed, 11 Mar 2026 20:35:45 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by SN1PEPF0002636A.mail.protection.outlook.com (10.167.241.135) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9678.18 via Frontend Transport; Wed, 11 Mar 2026 20:35:44 +0000 Received: from rnnvmail203.nvidia.com (10.129.68.9) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Wed, 11 Mar 2026 13:35:20 -0700 Received: from rnnvmail201.nvidia.com (10.129.68.8) by rnnvmail203.nvidia.com (10.129.68.9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Wed, 11 Mar 2026 13:35:19 -0700 Received: from nvidia-4028GR-scsim.nvidia.com (10.127.8.11) by mail.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20 via Frontend Transport; Wed, 11 Mar 2026 13:35:12 -0700 From: To: , , , , , , , , , , , , , , , , , CC: , , , , , , , Subject: [PATCH 03/20] cxl: Move CXL spec defines to public header Date: Thu, 12 Mar 2026 02:04:23 +0530 Message-ID: <20260311203440.752648-4-mhonap@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20260311203440.752648-1-mhonap@nvidia.com> References: <20260311203440.752648-1-mhonap@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SN1PEPF0002636A:EE_|SJ2PR12MB8739:EE_ X-MS-Office365-Filtering-Correlation-Id: d7f071d7-c21c-4745-8a49-08de7fadc533 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|7416014|82310400026|376014|36860700016|56012099003|22082099003|18002099003|921020; X-Microsoft-Antispam-Message-Info: 0OuxRZwhcf3hGLhvXp23BA/6vF+x37iqAC1XKRh1z6HcDGHOW5WYVoHkM/dWMU6z8R+07KpVI3iT4E3CNiczj9wHbNnCVC7rCwWyOurQfLP80VTt2x300yoVd3/gbFEoww1wv4NZMJ2nSTVvtDQ9ljM/GFHzMWYuo/Qt5UBY3UdJmhK/tJ/PNNPZKAVvGR8lbEruzU5I2B3lWmTLQEsFKqP+VdLMucs9NPMsixZmbVJxZ5kni9joO/0vudrCAJpav5t9srB0+mMRp1HZIPZyZhB89TAyLhnBDUMicgtJhxgLXLiHAOBbuwMrx3372PxogscnEd/d5h2dRnPF5d9zI+KKt4zBDiIFFAC5gGSg/Qwz5Adta77j8pkYWo0rkAbZVSgEkhYJVQqSJedgZjrSOPageRYKmZGDDg+BbOKHAdjeY3CB2cdjtzISaN4rMzJOevvx1VRcpmCWnFagqUieQ61bm4umH0DpYuBMmAKh7BfchgELX07Y1lZLTqLu4ASB5+9fyJvfcbNS7iMqS1lJNwegba1QXVOa5qzmEMEK6h8J/0fb1Iu/YEnWSUDYDDgvNit9uFXxNuudeji5i9xHZMut50MsthAGPqZ1ylvSLSyOn/e+/N0irAAyFHvqsvLWv3A+34RXIau6pbb49VxTvG955t7/iUmICfEL7n2ZGww5OHOhigXTdgywRByY7SsZ8XZic+H4PU3MTvKnXqyzlfETBqLK6DUNKv53z4e53qU9vpeziPy3dRm24iQyq3s36n0Uya3E1V+gWvcwwBlK9gL6CUtJ05BjDGc3LadTZxT9qmLWg6yq35jhUASjLJR3 X-Forefront-Antispam-Report: CIP:216.228.117.160;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge1.nvidia.com;CAT:NONE;SFS:(13230040)(1800799024)(7416014)(82310400026)(376014)(36860700016)(56012099003)(22082099003)(18002099003)(921020);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: Wxr0rLIxIfMweAepsI0S7AKRRPP0yVDjzYI1wo/oStlI84sayJHmU0Y+5E+TfrvM3CQlP1VBuW9GTZKl8ZtlbFkocAU/VxYgSUzBlH9/iaP0JuGfI/N5I2oLTFxvntxBIS8iv1NCXXZhqrxeYsqumtzhpiIA9hOSKAQCX/KpsTmSO6UWtfyGiuUmgKyVjvm2GymL0AxSE+JA+9I/XN5oge8TpaKHKPn4hAscFFLs6pPaNEgpbv1cJKIaLj0+u6t57d3jsofvYoxrohSEuIwMHMQUXdKG9xT4Nm2nz1COgRVGhC3kCu5rvCZxz9Mml2o9S0fvHMZYxQ0ip3CuoPzWSKmoeaRyiOUOja0IYrZLIjXwg1MELKlLY4nMgjvx8ZYHfvGGwXQoizZllKrj3mkPFZvgq5kbGEyuYFig39GiXnKju25iCgvHYU1swkedt3oR X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Mar 2026 20:35:44.6282 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: d7f071d7-c21c-4745-8a49-08de7fadc533 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: SN1PEPF0002636A.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ2PR12MB8739 Content-Type: text/plain; charset="utf-8" From: Manish Honap HDM decoder capability structure and component reg block size needs to be used by VFIO subsystem. Move the macros from private CXL header to public one. Signed-off-by: Manish Honap --- drivers/cxl/cxl.h | 30 ------------------------------ include/cxl/cxl.h | 30 ++++++++++++++++++++++++++++++ 2 files changed, 30 insertions(+), 30 deletions(-) diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index 10ddab3949ee..7146059e0dae 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -24,9 +24,6 @@ extern const struct nvdimm_security_ops *cxl_security_ops; * (port-driver, region-driver, nvdimm object-drivers... etc). */ =20 -/* CXL 2.0 8.2.4 CXL Component Register Layout and Definition */ -#define CXL_COMPONENT_REG_BLOCK_SIZE SZ_64K - /* CXL 2.0 8.2.5 CXL.cache and CXL.mem Registers*/ #define CXL_CM_OFFSET 0x1000 #define CXL_CM_CAP_HDR_OFFSET 0x0 @@ -39,33 +36,6 @@ extern const struct nvdimm_security_ops *cxl_security_op= s; #define CXL_CM_CAP_HDR_ARRAY_SIZE_MASK GENMASK(31, 24) #define CXL_CM_CAP_PTR_MASK GENMASK(31, 20) =20 -/* HDM decoders CXL 2.0 8.2.5.12 CXL HDM Decoder Capability Structure */ -#define CXL_HDM_DECODER_CAP_OFFSET 0x0 -#define CXL_HDM_DECODER_COUNT_MASK GENMASK(3, 0) -#define CXL_HDM_DECODER_TARGET_COUNT_MASK GENMASK(7, 4) -#define CXL_HDM_DECODER_INTERLEAVE_11_8 BIT(8) -#define CXL_HDM_DECODER_INTERLEAVE_14_12 BIT(9) -#define CXL_HDM_DECODER_INTERLEAVE_3_6_12_WAY BIT(11) -#define CXL_HDM_DECODER_INTERLEAVE_16_WAY BIT(12) -#define CXL_HDM_DECODER_CTRL_OFFSET 0x4 -#define CXL_HDM_DECODER_ENABLE BIT(1) -#define CXL_HDM_DECODER0_BASE_LOW_OFFSET(i) (0x20 * (i) + 0x10) -#define CXL_HDM_DECODER0_BASE_HIGH_OFFSET(i) (0x20 * (i) + 0x14) -#define CXL_HDM_DECODER0_SIZE_LOW_OFFSET(i) (0x20 * (i) + 0x18) -#define CXL_HDM_DECODER0_SIZE_HIGH_OFFSET(i) (0x20 * (i) + 0x1c) -#define CXL_HDM_DECODER0_CTRL_OFFSET(i) (0x20 * (i) + 0x20) -#define CXL_HDM_DECODER0_CTRL_IG_MASK GENMASK(3, 0) -#define CXL_HDM_DECODER0_CTRL_IW_MASK GENMASK(7, 4) -#define CXL_HDM_DECODER0_CTRL_LOCK BIT(8) -#define CXL_HDM_DECODER0_CTRL_COMMIT BIT(9) -#define CXL_HDM_DECODER0_CTRL_COMMITTED BIT(10) -#define CXL_HDM_DECODER0_CTRL_COMMIT_ERROR BIT(11) -#define CXL_HDM_DECODER0_CTRL_HOSTONLY BIT(12) -#define CXL_HDM_DECODER0_TL_LOW(i) (0x20 * (i) + 0x24) -#define CXL_HDM_DECODER0_TL_HIGH(i) (0x20 * (i) + 0x28) -#define CXL_HDM_DECODER0_SKIP_LOW(i) CXL_HDM_DECODER0_TL_LOW(i) -#define CXL_HDM_DECODER0_SKIP_HIGH(i) CXL_HDM_DECODER0_TL_HIGH(i) - /* HDM decoder control register constants CXL 3.0 8.2.5.19.7 */ #define CXL_DECODER_MIN_GRANULARITY 256 #define CXL_DECODER_MAX_ENCODED_IG 6 diff --git a/include/cxl/cxl.h b/include/cxl/cxl.h index 610711e861d4..27c006fa53c3 100644 --- a/include/cxl/cxl.h +++ b/include/cxl/cxl.h @@ -75,6 +75,36 @@ struct cxl_regs { #define CXL_CM_CAP_CAP_ID_HDM 0x5 #define CXL_CM_CAP_CAP_HDM_VERSION 1 =20 +/* CXL 2.0 8.2.4 CXL Component Register Layout and Definition */ +#define CXL_COMPONENT_REG_BLOCK_SIZE SZ_64K + +/* HDM decoders CXL 2.0 8.2.5.12 CXL HDM Decoder Capability Structure */ +#define CXL_HDM_DECODER_CAP_OFFSET 0x0 +#define CXL_HDM_DECODER_COUNT_MASK GENMASK(3, 0) +#define CXL_HDM_DECODER_TARGET_COUNT_MASK GENMASK(7, 4) +#define CXL_HDM_DECODER_INTERLEAVE_11_8 BIT(8) +#define CXL_HDM_DECODER_INTERLEAVE_14_12 BIT(9) +#define CXL_HDM_DECODER_INTERLEAVE_3_6_12_WAY BIT(11) +#define CXL_HDM_DECODER_INTERLEAVE_16_WAY BIT(12) +#define CXL_HDM_DECODER_CTRL_OFFSET 0x4 +#define CXL_HDM_DECODER_ENABLE BIT(1) +#define CXL_HDM_DECODER0_BASE_LOW_OFFSET(i) (0x20 * (i) + 0x10) +#define CXL_HDM_DECODER0_BASE_HIGH_OFFSET(i) (0x20 * (i) + 0x14) +#define CXL_HDM_DECODER0_SIZE_LOW_OFFSET(i) (0x20 * (i) + 0x18) +#define CXL_HDM_DECODER0_SIZE_HIGH_OFFSET(i) (0x20 * (i) + 0x1c) +#define CXL_HDM_DECODER0_CTRL_OFFSET(i) (0x20 * (i) + 0x20) +#define CXL_HDM_DECODER0_CTRL_IG_MASK GENMASK(3, 0) +#define CXL_HDM_DECODER0_CTRL_IW_MASK GENMASK(7, 4) +#define CXL_HDM_DECODER0_CTRL_LOCK BIT(8) +#define CXL_HDM_DECODER0_CTRL_COMMIT BIT(9) +#define CXL_HDM_DECODER0_CTRL_COMMITTED BIT(10) +#define CXL_HDM_DECODER0_CTRL_COMMIT_ERROR BIT(11) +#define CXL_HDM_DECODER0_CTRL_HOSTONLY BIT(12) +#define CXL_HDM_DECODER0_TL_LOW(i) (0x20 * (i) + 0x24) +#define CXL_HDM_DECODER0_TL_HIGH(i) (0x20 * (i) + 0x28) +#define CXL_HDM_DECODER0_SKIP_LOW(i) CXL_HDM_DECODER0_TL_LOW(i) +#define CXL_HDM_DECODER0_SKIP_HIGH(i) CXL_HDM_DECODER0_TL_HIGH(i) + struct cxl_reg_map { bool valid; int id; --=20 2.25.1 From nobody Tue Apr 7 19:54:44 2026 Received: from SN4PR0501CU005.outbound.protection.outlook.com (mail-southcentralusazon11011007.outbound.protection.outlook.com [40.93.194.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5AA6B388E44; Wed, 11 Mar 2026 20:35:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.93.194.7 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773261360; cv=fail; b=YZRxexGXmQzjti5Y6cczfq0j1/vHASiX45Y38qNpJ37HymLP5wYA6PrNZLJs/ozxFNuRMFRN4Tszj/OtqMBNiy56c5/2boEQl5ljO9CpNP3AWkfsX046Eu87br4L1962WUP3KzyxzK3QpsAicUzmMypBw2j+U9YrBAjIf08JLYI= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773261360; c=relaxed/simple; bh=oE5uFv5dAMUI1Vu6fnn9d12u+eQet13FwDEda9yalak=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=V3whr2cL6Uk/94+gyER6oGbp9jUz7Q8vgRoab2583uIG2iYEnxUodNInAX1OlC4RAI33DK/K0jQDwqyYRcVsyf4+ZJ4vatXSVqj0isGuKDlrhNArUWV57jN5W9GaplIK5vvHwvxuGfZPXa9eAK71JVx78vOgeytl+SnZw2BSPGY= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=DKvsAf/i; arc=fail smtp.client-ip=40.93.194.7 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="DKvsAf/i" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=PzW6oPQrM4vnBrpv/L9zn64i3WsNCCE9ka3aFr+HJ6RG8rOopXB9y+/kF8/oLJ2cfzthPUWbyjiT9XFT4mbAytmtLtaeHH9+aok3Vejg6UfbS9YbwXggeUDxP/BaDJ7rRaC4t7/TwEj9FtzfTy2U+yzjYx8Od0t+CqzKfbZY48mEB5D1pddMj3tcVq+Z7kyHlmMDRM2Ziyn4sBCmezWdHx3WMhFCR6vKTvYb8LjU0XV+bodZIOUPIG3yILSLkdQQzdv375NDnfUKknr2S+j/EkiepCl3tfMDvOYXE1lwRVh54r50VxV8FRRVqhQyEgVTWn0o5p4XCbMtnHZSBI7ZCQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=FQ3fYVpj9CKY4z2r+rWveKB4c2mwG689sxGowIrzm58=; b=gl8BUTeQHFYSt+A2dUw4ajVXMHVkGoHuNy8x6yF/1ehMzoJwUAxI1xz1ul/f6tF+mv6bIdYFV685dT/PxLaOnu4KZmsqRe8QkJg2YTElUXQ+ZMecRcNLW0JSDZkIRy2epYZ4bj+j1E2NLM5m+yAkfhfVghzypmVYp4UhSq/4s0/2ZHap1YF3Tl46feZJDCF2g5GwfWoPu5WRQrrv9I887rkVfaKNt8EV/ajCAO2PUiMc26gYTwDNoGHVaLQqqcoQPXmSTDDj7hXy12l4jn9gc2cnAvg3aoTmcfdS+Y410XzSETTQiaeu9CvcZKAoyZtojoaG0xGiD+Afc1Bk+eJXdQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=FQ3fYVpj9CKY4z2r+rWveKB4c2mwG689sxGowIrzm58=; b=DKvsAf/iUUjCWQVXqOHGgrk7WGW79vOdEZNuhDF6VJTFQ3b9WZOxV/s9JzO22/ekADkFmCRBbagCxvcUdrWFmUAjihP2gKF4W+KWYFg/V/3BBOAqz7dHREXs6E9u4+qQ82QbsjOxTVCNOkNKQBVdmARehEbaYnbGYdyAknHds1qlltbkpHPLedqkGvyxam886WZj42BexA0AHy8sq01UFE1rIQssK0p1MIKg76m2j3jYTzilJEP/H256DMCXyzdav5Acmj59DkmKVFmK/8hcL3dZ/RAW+k8mzgT+FlnxcuU0PBtfgJrqojrfxlSB1Okhza6eJd+2fgn/SB5l2iNt3w== Received: from SN7PR04CA0088.namprd04.prod.outlook.com (2603:10b6:806:121::33) by PH7PR12MB6833.namprd12.prod.outlook.com (2603:10b6:510:1af::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9700.11; Wed, 11 Mar 2026 20:35:53 +0000 Received: from SN1PEPF0002636A.namprd02.prod.outlook.com (2603:10b6:806:121:cafe::29) by SN7PR04CA0088.outlook.office365.com (2603:10b6:806:121::33) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9678.26 via Frontend Transport; Wed, 11 Mar 2026 20:35:29 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by SN1PEPF0002636A.mail.protection.outlook.com (10.167.241.135) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9678.18 via Frontend Transport; Wed, 11 Mar 2026 20:35:53 +0000 Received: from rnnvmail203.nvidia.com (10.129.68.9) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Wed, 11 Mar 2026 13:35:27 -0700 Received: from rnnvmail201.nvidia.com (10.129.68.8) by rnnvmail203.nvidia.com (10.129.68.9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Wed, 11 Mar 2026 13:35:27 -0700 Received: from nvidia-4028GR-scsim.nvidia.com (10.127.8.11) by mail.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20 via Frontend Transport; Wed, 11 Mar 2026 13:35:20 -0700 From: To: , , , , , , , , , , , , , , , , , CC: , , , , , , , Subject: [PATCH 04/20] cxl: Media ready check refactoring Date: Thu, 12 Mar 2026 02:04:24 +0530 Message-ID: <20260311203440.752648-5-mhonap@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20260311203440.752648-1-mhonap@nvidia.com> References: <20260311203440.752648-1-mhonap@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SN1PEPF0002636A:EE_|PH7PR12MB6833:EE_ X-MS-Office365-Filtering-Correlation-Id: c9355dd6-e280-42fd-6a0a-08de7fadca3d X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|82310400026|1800799024|376014|36860700016|56012099003|22082099003|18002099003|921020; X-Microsoft-Antispam-Message-Info: of2LLsAGqtUtV455hk31HJSxc6GR3j8DPwHHpciuPbDGtD5qIbPJ6f+IDQ4QTRRqnXStJBFhOat5ZNHR44Vkd5ZSzH8DA/+RfZQ1P4fzCFwjobHiDu6+HYatrGIjIRvmxsj9XE9o8Zfn47BHFs6pN42UbP6upO+N2Ii4vFI0AlR+ngK516igJiHzYK2wu9B7A+Fo/VkEvGjIdCGnlQc67YJTSi2/mbSIs4KDfRIgXJBIk3Jzvdd6debBNHy/CrisSNEhezZdRF3MLGiMF50mJ2D11bxtR+g3E+QHRyQlMFHDA4/3ED4d+vV6KGoB7AlYKrtKYIAGXCW4X6edvoLwEfDl4ZK/vwC/6n7r5ymAJz7PJD/wPEESrdRxPNLDMDqZIq2ModkZZULau8Ajp/QUR8P9WUiXL9CxVhyTSAC99r3eK3Hgwq7le4sm2rmvwJGV9Z0cYBX+Mz0yNPPQe06c9xq+WWcEvCppnG3zhluGZAwHarYupBKL8KMViK7FrzqcyBwkb0FKMn4pLdRcrO1gQWRDdlL8gLnubUkEcGXoDM8fRXgZ2t0y9Qeb1cP7CvIgLqEjz7P1WhKgHe8ZUrEaDWJKC9aTWdNYnqc55GOzmRolqVEXrajimxY/ZvBPStv296SURK2bMRCd/xdnXpKswPJ7RPzyqUGbTFRNdhtGYMDjNN6i99cD4iXp96GRWfoAIUFds3tYGzAuiarFvycOZorKOVlwAY+oZwSnQGQABxWsHqFq+LIDVQBrOlRplyZbaNC/g7KM4/S011AwFuG+XA6wxHCF+kpW1DhMyO/MvRrcZ3PFqWN2pTKgFcGZ9YlK X-Forefront-Antispam-Report: CIP:216.228.117.160;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge1.nvidia.com;CAT:NONE;SFS:(13230040)(7416014)(82310400026)(1800799024)(376014)(36860700016)(56012099003)(22082099003)(18002099003)(921020);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 8rFt1jVIKUMug/URqPdiWWHoSeF48rVcbRui2UGrElaZa2yARToGyQKU51KS46FzARerK2oYsz01mo0Lrm9MnQZKZWxx8R/d8BczhjA/JqiMEsd3z/rPdqI7D33tfPPktL5ccvIQDgkZykrX1hH6LDxBvfKyPLTF2v1hqBpp+cDrELs/cw8g/6X2wQ1XM46DW5j2rcvQufL0fbsxYWD8FvrWEnobzdBO3S+eputtivhdt3QHgvWZYIJKCtfEPxWfSyg3g71J9rFbrgdmoNmUJ+7P8JKQAbWR2+Kltc18xOWjIZWnhVR0Xds6txBJZs9qLeE1hZjdv3ApyPJ0E1pp+ha1AutbS3Qi7dqK/B2ShYmJAc1lNMj8aAMN+oR9wCRHQob8e2DUgWDKrohLnRhsRl/dfSuyieFmK9cVymANZiv49k+wQsaYSjZM1yWAcNDv X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Mar 2026 20:35:53.0929 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: c9355dd6-e280-42fd-6a0a-08de7fadca3d X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: SN1PEPF0002636A.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB6833 Content-Type: text/plain; charset="utf-8" From: Manish Honap Before accessing CXL device memory after reset/power-on, the driver must ensure media is ready. Not every CXL device implements the CXL Memory Device register group (many Type-2 devices do not). cxl_await_media_ready() reads cxlds->regs.memdev; calling it on a Type-2 without that block can result in kernel panic. This commit refactors the HDM range based check in a new function which can be safely used for type-2 and type-3 devices. cxl_await_media_ready still uses the same format of checking the HDM range and memory device register status. Co-developed-by: Zhi Wang Signed-off-by: Zhi Wang Signed-off-by: Manish Honap Reviewed-by: Dave Jiang --- drivers/cxl/core/pci.c | 35 ++++++++++++++++++++++++++++++----- include/cxl/cxl.h | 1 + 2 files changed, 31 insertions(+), 5 deletions(-) diff --git a/drivers/cxl/core/pci.c b/drivers/cxl/core/pci.c index 52ed0b4f5e78..2b7e4d73a6dd 100644 --- a/drivers/cxl/core/pci.c +++ b/drivers/cxl/core/pci.c @@ -142,16 +142,24 @@ static int cxl_dvsec_mem_range_active(struct cxl_dev_= state *cxlds, int id) return 0; } =20 -/* - * Wait up to @media_ready_timeout for the device to report memory - * active. +/** + * cxl_await_range_active - Wait for all HDM DVSEC memory ranges to be act= ive + * @cxlds: CXL device state (DVSEC and HDM count must be valid) + * + * For each HDM decoder range reported in the CXL DVSEC capability, waits = for + * the range to report MEM INFO VALID (up to 1s per range), then MEM ACTIVE + * (up to media_ready_timeout seconds per range, default 60s). Used by + * cxl_await_media_ready() and by callers that only need range readiness + * without checking the memory device status register. + * + * Return: 0 if all ranges become valid and active, -ETIMEDOUT if a timeout + * occurs, or a negative errno from config read on failure. */ -int cxl_await_media_ready(struct cxl_dev_state *cxlds) +int cxl_await_range_active(struct cxl_dev_state *cxlds) { struct pci_dev *pdev =3D to_pci_dev(cxlds->dev); int d =3D cxlds->cxl_dvsec; int rc, i, hdm_count; - u64 md_status; u16 cap; =20 rc =3D pci_read_config_word(pdev, @@ -172,6 +180,23 @@ int cxl_await_media_ready(struct cxl_dev_state *cxlds) return rc; } =20 + return 0; +} +EXPORT_SYMBOL_NS_GPL(cxl_await_range_active, "CXL"); + +/* + * Wait up to @media_ready_timeout for the device to report memory + * active. + */ +int cxl_await_media_ready(struct cxl_dev_state *cxlds) +{ + u64 md_status; + int rc; + + rc =3D cxl_await_range_active(cxlds); + if (rc) + return rc; + md_status =3D readq(cxlds->regs.memdev + CXLMDEV_STATUS_OFFSET); if (!CXLMDEV_READY(md_status)) return -EIO; diff --git a/include/cxl/cxl.h b/include/cxl/cxl.h index 27c006fa53c3..684603799fb1 100644 --- a/include/cxl/cxl.h +++ b/include/cxl/cxl.h @@ -323,5 +323,6 @@ int cxl_find_regblock(struct pci_dev *pdev, enum cxl_re= gloc_type type, struct cxl_register_map *map); void cxl_probe_component_regs(struct device *dev, void __iomem *base, struct cxl_component_reg_map *map); +int cxl_await_range_active(struct cxl_dev_state *cxlds); =20 #endif /* __CXL_CXL_H__ */ --=20 2.25.1 From nobody Tue Apr 7 19:54:44 2026 Received: from PH8PR06CU001.outbound.protection.outlook.com (mail-westus3azon11012059.outbound.protection.outlook.com [40.107.209.59]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 58D90355F43; Wed, 11 Mar 2026 20:36:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.209.59 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773261361; cv=fail; b=Xi6X+W/wqHYxoyxmcsesxUOtADFA60uXt9DsZfSOBLdXS6EdF82HHspvFUJKUbX70VI1YSiNoXTWoL6QQDfhbnePO96ISkqg+r4DyKjEPl7XNEk9Xt/DLsjcsy5nDeHXCK5uqqN8LibWiMiTS97bSkPn1x4EdkgHAOWMLxWeWd0= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773261361; c=relaxed/simple; bh=HxSbwzsS+X/miJJ8u/3G3pPmjGgO/gbY1dWJHDlczPk=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=GZLHIT/IWYR8ZG83vgNYmEFLcCCQjWzvYv5iyQmZw/GNqoBo9ljhNKCpnXLCk2LD4w3O15rjdLTeOwwy6JXvrEiUl4NcFW1fIFzQKQ1avb9F2Wm/7Z+IUZKtobjUoN13Lvu9tUf2hJKChIjy2m+Lf4LwXy35Ajmuv3Z/RMNg6Ms= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=GEKcEjfo; arc=fail smtp.client-ip=40.107.209.59 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="GEKcEjfo" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=CXzz7JIJQWLBXkMjXDg7ORDByKpTqSjxoE907+9jHYkRDj+ZjJ9GMz7jrCBMvDL+9O/QGwSJoSjj1II2XOStyaaVYFCiweBCE7K/NqXFfSIua6vxWPLAwtCdDoHQDO0b6DBqs/r0rlfuTPhP0EWr7SWQ4wUnlogCTBl7KTnfS/7BWvuyNgs+NT4Y5wxgCk1tStNnVMOgxSCM/urF6MY26cIT2ED/qql/ESH50Q2PuoNLfJxgdVXsOeMJNuyxkwoGXbD/jzKhBTmuO/R8XNetSNoPO1tXqBV9BcpipfKxBLb1dMVCGlaaxWUnUrSi/EjF9rgDRR8SqnWGnQXQI+CkpA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=tbdeXCY44RBIWU4kVOTrNmxhnBxBnF0SwtZoIX401i0=; b=HlWJTD1i9KmuH+wZH/hZ1OyXmvJDppxzgdkiU+O3AD86krfHR2niLBXKseAH/gvo9RJ0zC4LcuJaulm6DEd+ZsRkxdbHlTaUmM+B+lvaVRIUXHIIe56ZNpXhLmJuywTY/uKAgqzmSqP8DYp9tM+3mNqcpHWSVGeUhn32lQfQ9foYNrkA5F5elSunD8NQUH7Wh//hxbpXLhI6OanOJlTCP0m26feio8QnnMpCPDm8iZ8iO7IOVlK6rv7JBwl5YRM68Q5dYpCDB7OELj6v5e0c3ceXtzjjplsWSafsrxPgpqqY+PET1IbgAVGD5Q+J6LItZTcTAXxVp6wbVbC69F4hCA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=tbdeXCY44RBIWU4kVOTrNmxhnBxBnF0SwtZoIX401i0=; b=GEKcEjfoNEkXinTVTKAOU5KdoE1U33enxvFctonU1KDsR5oRa90/kh+xjKj0gHYA/UmYgcqzbXsuAOIVuuVo0eGtMrKMMIwKnoH+3MtpiToK5ckGU20GQZZR94RoZ23HLul5UiaLkvx2J80LNTWASebd+0lYXB8nnsSjTvRI8coKfpMxPQ9fOGhx0XKNa5+yWIpHIT4r2H6G0mzJKbdsHJVx50FgRcVMd0Pz38afAwYakc583OHe309mfCFB4y1DBB4gNBxoq02o0BNDFnRqOdwagBNS528w71exRYUoq12SugR0IHz1oTSGHGjMsYEC3OefgwQP+vbueB+tfb0DsA== Received: from DS7PR05CA0041.namprd05.prod.outlook.com (2603:10b6:8:2f::8) by SJ0PR12MB7033.namprd12.prod.outlook.com (2603:10b6:a03:448::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9700.11; Wed, 11 Mar 2026 20:35:53 +0000 Received: from DS2PEPF00003445.namprd04.prod.outlook.com (2603:10b6:8:2f:cafe::34) by DS7PR05CA0041.outlook.office365.com (2603:10b6:8:2f::8) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9678.25 via Frontend Transport; Wed, 11 Mar 2026 20:35:53 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by DS2PEPF00003445.mail.protection.outlook.com (10.167.17.72) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9678.18 via Frontend Transport; Wed, 11 Mar 2026 20:35:53 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Wed, 11 Mar 2026 13:35:35 -0700 Received: from rnnvmail201.nvidia.com (10.129.68.8) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Wed, 11 Mar 2026 13:35:34 -0700 Received: from nvidia-4028GR-scsim.nvidia.com (10.127.8.11) by mail.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20 via Frontend Transport; Wed, 11 Mar 2026 13:35:27 -0700 From: To: , , , , , , , , , , , , , , , , , CC: , , , , , , , Subject: [PATCH 05/20] cxl: Expose BAR index and offset from register map Date: Thu, 12 Mar 2026 02:04:25 +0530 Message-ID: <20260311203440.752648-6-mhonap@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20260311203440.752648-1-mhonap@nvidia.com> References: <20260311203440.752648-1-mhonap@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS2PEPF00003445:EE_|SJ0PR12MB7033:EE_ X-MS-Office365-Filtering-Correlation-Id: d86f396a-5f94-41b7-b20e-08de7fadca6b X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|7416014|376014|82310400026|36860700016|56012099003|18002099003|22082099003|921020; X-Microsoft-Antispam-Message-Info: S2nFb+M/pIqcdTLqu6O5qQPrE8vxHZvu8G5QQydGiVH+GdFbZezlU5vkVksAt41i3q2Tj2zP0LmueU0/FpzdYl1L2FKKZCs4lOT4yU9f3FgMPRGbYqoVI5IyfOGl70bUxDwVWFmjx/8vhcc+V8v9JUR+WrNbnYDivA++jRrDE2xwSoQfbMBSVPaa2Z+hmSqwr0Y71tDzd5zfRulXG8D2xcBAFchdo/tGG2aBd+MWWK55QLUmNWU5OzQ1AHlzko86CoVuorSH21axNgqhR4u+MhDONs02FQVWUxOy1ytbbyucU/KJwtVE6mIYwTLcQ2hLZmqFX6dbhhe5sfechcsQcMjEgR+kS9YR0mM2n4MQeUTpLz5vqTfmfOMBfgjkJzg0bFZN0N6dgmH9ZiWgVO/FuxOtKH1EloHSdSWm0fQ1rLI7jdhpha1FQiPWEpg+sTN0WdnqjoV7qOwWPHbpel8Z+xtyqn5Bp4l00VpVJY4sgXRlQci08Jt5lm6zQ9ROzPiEu31C9CPNaRszDnrj5LwCRLAmjQ9cfvj6KBwX3pVeR/vu2lcpQ1xlBulbsgXGDUeqkvPjoyVbJsroeKHFjhAFVtmUoxTJz8Rs0vvIqS+ISHcVzcbmYshhKbhXU42GjjNFoLuhkdmi2lBGmWbO4DaMPnVXJK4zJddLIEY1T85YM44K7I/xBp7bB2NbQKIHYJ41Zuzc/iejoj9tyLZjYzq8VmZXgS3v1MdF3Us2AedzZLf8uvgVws/ieRaVzp5KxwTtIDv4PFEYAJEVsUr+xLvXTM8KuxO1AS5IDoVkgP6GNUsyFGUCcFNvSl8GkxeJLqgc X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(1800799024)(7416014)(376014)(82310400026)(36860700016)(56012099003)(18002099003)(22082099003)(921020);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: waqJCIESFoBgLxR8x1ZhrcqWoKBFxHgFhFrVYJLBT/eTMEDH3Aca7T7TMYB8qLjICEXztEs01ewiRowL8z0PFVs0UE8DkV8vvxHkPsk1VuPr7xj2IxNTCFlu8Hv34anLS1E6XOzLLX9+xjpVIY6xLvLBi/hYSMTJC082KUx/xk6g4FgOvSwQ2wkp+UG1oaqKUpOgDKhqkIAe8LizDtAcqxTUZ3AkVb9rMNp4ZFVZHz7flGB7oJRx0Kt73I/MSPuzU/1zS+KEdEolPgtZQdYCWU6ypA9YYdETUqmtXEr/1EzhISW2hoSCRXjh/g7chs+5w+7QieYa69ofNXo4a0ltogPdb69Y8GD3vu8AcADiqcciHKKEYca/DW9V358mhvusqftf074fifQjwk1JokPC2u/4GoAkdV0+k/kkInp0xxOIgLLW3FOlDIUCaU/mYqlp X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Mar 2026 20:35:53.3925 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: d86f396a-5f94-41b7-b20e-08de7fadca6b X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DS2PEPF00003445.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR12MB7033 Content-Type: text/plain; charset="utf-8" From: Manish Honap The Register Locator DVSEC (CXL 2.0 8.1.9) describes register blocks by BAR index (BIR) and offset within the BAR. CXL core currently only stores the resolved HPA (resource + offset) in struct cxl_register_map, so callers that need to use pci_iomap() or report the BAR to userspace must reverse-engineer the BAR from the HPA. Add bar_index and bar_offset to struct cxl_register_map and fill them in cxl_decode_regblock() when the regblock is BAR-backed (BIR 0-5). Add cxl_regblock_get_bar_info() so callers (e.g. vfio-cxl) can get BAR index and offset directly and use pci_iomap() instead of ioremap(HPA). Signed-off-by: Manish Honap --- drivers/cxl/core/regs.c | 29 +++++++++++++++++++++++++++++ include/cxl/cxl.h | 11 +++++++++++ 2 files changed, 40 insertions(+) diff --git a/drivers/cxl/core/regs.c b/drivers/cxl/core/regs.c index 20c2d9fbcfe7..720eb6eb5a45 100644 --- a/drivers/cxl/core/regs.c +++ b/drivers/cxl/core/regs.c @@ -287,9 +287,37 @@ static bool cxl_decode_regblock(struct pci_dev *pdev, = u32 reg_lo, u32 reg_hi, map->reg_type =3D reg_type; map->resource =3D pci_resource_start(pdev, bar) + offset; map->max_size =3D pci_resource_len(pdev, bar) - offset; + map->bar_index =3D (bar >=3D 0 && bar < PCI_STD_NUM_BARS) ? (u8)bar : 0xF= F; + map->bar_offset =3D offset; return true; } =20 +/** + * cxl_regblock_get_bar_info() - Get BAR index and offset for a BAR-backed= regblock + * @map: Register map from cxl_find_regblock() or cxl_find_regblock_instan= ce() + * @bar_index: Output BAR index (0-5). Optional, may be NULL. + * @bar_offset: Output offset within the BAR. Optional, may be NULL. + * + * When the register block was found via the Register Locator DVSEC and + * lives in a PCI BAR (BIR 0-5), this returns the BAR index and the offset + * within that BAR. Callers can use pci_iomap(pdev, bar_index, size) and + * base + bar_offset instead of ioremap(map->resource). + * + * Return: 0 if the regblock is BAR-backed (bar_index <=3D 5), -EINVAL oth= erwise. + */ +int cxl_regblock_get_bar_info(const struct cxl_register_map *map, u8 *bar_= index, + resource_size_t *bar_offset) +{ + if (!map || map->bar_index > PCI_STD_NUM_BARS - 1) + return -EINVAL; + if (bar_index) + *bar_index =3D map->bar_index; + if (bar_offset) + *bar_offset =3D map->bar_offset; + return 0; +} +EXPORT_SYMBOL_NS_GPL(cxl_regblock_get_bar_info, "CXL"); + /* * __cxl_find_regblock_instance() - Locate a register block or count insta= nces by type / index * Use CXL_INSTANCES_COUNT for @index if counting instances. @@ -308,6 +336,7 @@ static int __cxl_find_regblock_instance(struct pci_dev = *pdev, enum cxl_regloc_ty =20 *map =3D (struct cxl_register_map) { .host =3D &pdev->dev, + .bar_index =3D 0xFF, .resource =3D CXL_RESOURCE_NONE, }; =20 diff --git a/include/cxl/cxl.h b/include/cxl/cxl.h index 684603799fb1..08e327a929ba 100644 --- a/include/cxl/cxl.h +++ b/include/cxl/cxl.h @@ -134,9 +134,16 @@ struct cxl_pmu_reg_map { * @resource: physical resource base of the register block * @max_size: maximum mapping size to perform register search * @reg_type: see enum cxl_regloc_type + * @bar_index: PCI BAR index (0-5) when regblock is BAR-backed; 0xFF other= wise + * @bar_offset: offset within the BAR; only valid when bar_index <=3D 5 * @component_map: cxl_reg_map for component registers * @device_map: cxl_reg_maps for device registers * @pmu_map: cxl_reg_maps for CXL Performance Monitoring Units + * + * When the register block is described by the Register Locator DVSEC with + * a BAR Indicator (BIR 0-5), bar_index and bar_offset are set so callers = can + * use pci_iomap(pdev, bar_index, size) and base + bar_offset instead of + * ioremap(resource). */ struct cxl_register_map { struct device *host; @@ -144,6 +151,8 @@ struct cxl_register_map { resource_size_t resource; resource_size_t max_size; u8 reg_type; + u8 bar_index; + resource_size_t bar_offset; union { struct cxl_component_reg_map component_map; struct cxl_device_reg_map device_map; @@ -319,6 +328,8 @@ int cxl_get_hdm_reg_info(struct cxl_dev_state *cxlds, u= 32 *count, resource_size_t *offset, resource_size_t *size); struct pci_dev; enum cxl_regloc_type; +int cxl_regblock_get_bar_info(const struct cxl_register_map *map, u8 *bar_= index, + resource_size_t *bar_offset); int cxl_find_regblock(struct pci_dev *pdev, enum cxl_regloc_type type, struct cxl_register_map *map); void cxl_probe_component_regs(struct device *dev, void __iomem *base, --=20 2.25.1 From nobody Tue Apr 7 19:54:44 2026 Received: from BL0PR03CU003.outbound.protection.outlook.com (mail-eastusazon11012022.outbound.protection.outlook.com [52.101.53.22]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E3D92355F43; Wed, 11 Mar 2026 20:36:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.53.22 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773261369; cv=fail; b=j0q8nreLroHRQOC7KqilTsFyW6Bw53yEXMsgmN6t2aCdCnXIwESmHL9WPOgtobdiG3jWKQIsblInBWyAtQYsJ2xMFkOJfMgJi4xFqG6fF3jphSgpR+eqdlPlniRAM51yRLwETBz0UCpD9+4n9cxOw5qxgodyUqySrilPCGTPX5M= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773261369; c=relaxed/simple; bh=ctBDLWfOWm2RcIL8GaTlfROzgFHPnzaKMVs6bmLZlME=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=kAyKWNPbrKTENdUEo+6JHBgtOnGW87pF2faM0krZlu+FIPJWkFwJBA0dOz2xFSy0xVyQfqHmn4L+J9QOnvGbnxbb/re20UcYoNeYtrIcv7DxCCrVED8ltjRv5WrsVG52XZCgK6iniQ5+skNrq3WXy6OEfLY9Ys1Ujtdpy2k9v4k= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=ET2fUr/k; arc=fail smtp.client-ip=52.101.53.22 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="ET2fUr/k" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=dotYrIyDhgaWhwlsFLv1VV5XOxYVJ7yErl8vml18VuyXAGlL2pbRUu6fLEzcC161IRH8DQDcC4zAOiPvqrUnnZ98Vemn33EqGjfLRaQezjJDbHs8XpivsmTATEPTelx8qNuIQ7uAwaPwfQ8FZBC4RSqiNAg39wdgShs0rVei2Bf1zBX7RGmdIW/9Arfy8fjYiNjXnMTaLGjXgDhzUZGDi4Y67bZ+WaWkK7D+eUb/dNqTaoUv9a29E9dgboaEihMoQx0swZtvxLDM3s4iNrrFpqybahrHlnCZahQwYfYgil1ofWIxkGvFAv4mRaTR3nZw8zBipnrVLnYV2I1LspstEw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ydQysJ2Zj+jEf6kK9NnkteLIFMK2ZfeAWy7xZP4bOYg=; b=K9eavVSY7J45ny8N+7RY1+jGCFDlkPYrTRDlq6ey/8l8fKIHClCIGqe0Cv6GjuL+2BCXaii7qRxapj8Byu+AaXuZ896x/9dMP3wHRxrZW27swPoT8VTPIZF0b3h3S3gAsm0NUYbhBpL6ATBqKvC0sGAJQyru1zGNpypk0V7uCumZ4IjfNtEubqhim6iVDtMbpnDgR0WDp4cuZajFE3YJnYVMsMMJqDdA8KQTTW3ESbT/ilWUCVWN/WlNPJAAXJPFLMYuN35lJWMLJkVTWvIQZSBIu+TSAAMzh9uVqFqm2ePXXfaFEnZSzIENp++J5vD819+hG++pBYtIQxBzUc7Mvg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ydQysJ2Zj+jEf6kK9NnkteLIFMK2ZfeAWy7xZP4bOYg=; b=ET2fUr/kywv/QSFjNJwrwjVKS9t+OX1BeJJfzhUDI0xoyOcOaPc/PdcWhBTVnoeVTtmfjC/dlHqvLHm0hNgTmKobpSaQ91XNP/Er4aqQ3ZSO4SYs8DrvR+Uny9g0xqwp/JaG7ZUOLzxP9KzGI5unP/m5pxS64Fp4NJ8EZyxu3I/cPQGPVnn2gVviEB7I20sidp+nG2+ulXFQw9FLShhPG+rO4Tt8PUnVolET0EYOUSY9IwhyooYi50ZBHqBQCjuWEI7Ds8rDIqKwlk9LoSOitM14vRQxX9sSVbxaoy+nuK+r8f+i/80O+hrIeN373gyshBsJ8XZUVklafb17jCFgSg== Received: from SA9PR13CA0118.namprd13.prod.outlook.com (2603:10b6:806:24::33) by MN2PR12MB4287.namprd12.prod.outlook.com (2603:10b6:208:1dd::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9700.11; Wed, 11 Mar 2026 20:36:03 +0000 Received: from SN1PEPF0002636B.namprd02.prod.outlook.com (2603:10b6:806:24:cafe::55) by SA9PR13CA0118.outlook.office365.com (2603:10b6:806:24::33) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9678.24 via Frontend Transport; Wed, 11 Mar 2026 20:36:03 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by SN1PEPF0002636B.mail.protection.outlook.com (10.167.241.136) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9678.18 via Frontend Transport; Wed, 11 Mar 2026 20:36:03 +0000 Received: from rnnvmail203.nvidia.com (10.129.68.9) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Wed, 11 Mar 2026 13:35:42 -0700 Received: from rnnvmail201.nvidia.com (10.129.68.8) by rnnvmail203.nvidia.com (10.129.68.9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Wed, 11 Mar 2026 13:35:42 -0700 Received: from nvidia-4028GR-scsim.nvidia.com (10.127.8.11) by mail.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20 via Frontend Transport; Wed, 11 Mar 2026 13:35:35 -0700 From: To: , , , , , , , , , , , , , , , , , CC: , , , , , , , Subject: [PATCH 06/20] vfio/cxl: Add UAPI for CXL Type-2 device passthrough Date: Thu, 12 Mar 2026 02:04:26 +0530 Message-ID: <20260311203440.752648-7-mhonap@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20260311203440.752648-1-mhonap@nvidia.com> References: <20260311203440.752648-1-mhonap@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SN1PEPF0002636B:EE_|MN2PR12MB4287:EE_ X-MS-Office365-Filtering-Correlation-Id: dbad2e26-93ad-4868-e88a-08de7fadd04b X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|82310400026|36860700016|7416014|376014|921020|22082099003|18002099003|56012099003; X-Microsoft-Antispam-Message-Info: cDgW3Ym7c6uuNOQL8Ul/wsrWWGJmT3EcPeIfaUYSCqkNKmgCWwTiJyDR+XLnBoA1Y5A1flUurqq74y/sUTG/AymOp0Z8bTh8ksbAM1LaaCW6X/c1KLtJtoCW6QWoSOd46BwnIUMMpRKuDcGtHzf0VmUClBhtqwYe7PpPT5jqh7jRFdVc8eX9H3ECABzOo7YYcKBaAQ2p8Wa6l6LSeGBGyRKJkkSPEbKvAsSiIT97humICfnGX5A5bugUvpMpd3IAVvDiS6Bd1T8KyT5ORdg3+kzzldvRA+ZfpUIn1ltW5knGrAwS7vu7NYTAtCB4gIQwwwrT6hiN98l+RCWkcu/5LNSjAv3eeYgjJznKFGnbGhfO7t3pVGO+Z3VfrW0v59JLRljZehGuNZ573++dpWf4lNFILncLQT6TURwA7XRb6H6RWQm/myVbWT/9IgQ8pevWGTa7KINPHo/P2Hov/tMDHBJGEqdmrX1o4zmIQknx6Nv5TEYzGdU8orCBzFKozzpA1lEkBOwZHltX9eNSvjaHTClQihmWqyXNoy3Yq12h2gKQgBLkD5PvM6zRnRKBhgGMzO8W3+n14XZ+xWV1xWP7lkHPOv9wtMJQQTyGijxCvUdAyCF6gCI82+9/tU5nb9db+vHunrV3lZmm96kykK4V1cX9q2LBotVIlFn3JwrsXEBb2+VlSJI7L4noMqDWK9/ZhzR8nnQJSc4D62oDMF429VgP7kYnqA8SIs2ZsKUseMuIF7kXGYk28YRJEppR6dnlr5Q0DyEB7QPF5N+MgUdkNkazpTo65wb+LtBlKKSzTCzZLONv33l9jJj4fD2ZJAR8 X-Forefront-Antispam-Report: CIP:216.228.117.160;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge1.nvidia.com;CAT:NONE;SFS:(13230040)(1800799024)(82310400026)(36860700016)(7416014)(376014)(921020)(22082099003)(18002099003)(56012099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: H7XC88JtCFj6GQoFtpjm9BQfQrE8UCHPwPybe26QrrrU6W++EbpfyByVc6EpjVhfRVrHnZGdbf0jsXa53NNGwe5XbJKv+aoHeg36nxNYdyxUVJHOTtPfWd1y4t+qbcdEgLb5ufJ3wIaBQN1vOEct3C6bpB/6FB0gpQlztLjNOTftmMAup+aJGgHLZxG8/xTz+RXYsiH+lssdhXOIceSzjN+txit+uH3fuR7YRFH/echzaE1K2868sRMD1nALxy4oaYruKkxH5aA15Num1duMQkVc91JlvI0EmGMzn/HHq6NvzJ11Ftie7PWeiLmUJ8YDGTZk1E3LoHiJBWKEd4jb154hDqcV522HOzyio6ng3ADlSbWjaJsmxbQdmLmxQtFLwFLKmXM+NTvurc4quhAZMmnHvn1Grvj0IgogRpbfTz0uQEjKFVNS6jXLQ5ka0/3B X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Mar 2026 20:36:03.2415 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: dbad2e26-93ad-4868-e88a-08de7fadd04b X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: SN1PEPF0002636B.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4287 Content-Type: text/plain; charset="utf-8" From: Manish Honap CXL capabilities include: - hdm_count: Number of HDM decoders available - capacity: Total device memory (DPA) - flags: COMMITTED, PRECOMMITTED This UAPI enables VMMs like QEMU to passthrough CXL Type-2 devices (GPUs, accelerators) with coherent memory to VMs. Also added user-kernel API definitions for CXL Type-2 device passthrough. Document how VFIO_DEVICE_FLAGS_CXL relates to VFIO_DEVICE_FLAGS_PCI and VFIO_DEVICE_FLAGS_CAPS, and add field and flag descriptions for the CXL capability. Signed-off-by: Manish Honap --- include/uapi/linux/vfio.h | 52 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 52 insertions(+) diff --git a/include/uapi/linux/vfio.h b/include/uapi/linux/vfio.h index ac2329f24141..7ec0f96cc2d9 100644 --- a/include/uapi/linux/vfio.h +++ b/include/uapi/linux/vfio.h @@ -215,6 +215,13 @@ struct vfio_device_info { #define VFIO_DEVICE_FLAGS_FSL_MC (1 << 6) /* vfio-fsl-mc device */ #define VFIO_DEVICE_FLAGS_CAPS (1 << 7) /* Info supports caps */ #define VFIO_DEVICE_FLAGS_CDX (1 << 8) /* vfio-cdx device */ +/* + * CXL Type-2 device (memory coherent; e.g. GPU, accelerator). When set, + * VFIO_DEVICE_FLAGS_PCI is also set (same device is a PCI device). The + * capability chain (VFIO_DEVICE_FLAGS_CAPS) contains VFIO_DEVICE_INFO_CAP= _CXL + * describing HDM decoders, DPA size, and CXL-specific options. + */ +#define VFIO_DEVICE_FLAGS_CXL (1 << 9) /* Device supports CXL */ __u32 num_regions; /* Max region index + 1 */ __u32 num_irqs; /* Max IRQ index + 1 */ __u32 cap_offset; /* Offset within info struct of first cap */ @@ -257,6 +264,39 @@ struct vfio_device_info_cap_pci_atomic_comp { __u32 reserved; }; =20 +/* + * VFIO_DEVICE_INFO_CAP_CXL - CXL Type-2 device capability + * + * Present in the device info capability chain when VFIO_DEVICE_FLAGS_CXL + * is set. Describes Host Managed Device Memory (HDM) layout and CXL + * memory options so that userspace (e.g. QEMU) can expose the CXL region + * and component registers correctly to the guest. + */ +#define VFIO_DEVICE_INFO_CAP_CXL 6 +struct vfio_device_info_cap_cxl { + struct vfio_info_cap_header header; + __u8 hdm_count; /* Number of HDM decoders */ + __u8 hdm_regs_bar_index; /* PCI BAR containing HDM registers */ + __u16 pad; + __u32 flags; +/* Decoder was committed by host firmware/BIOS */ +#define VFIO_CXL_CAP_COMMITTED (1 << 0) +/* + * Memory was pre-committed (firmware-programmed); VMM need not allocate + * from CXL pool + */ +#define VFIO_CXL_CAP_PRECOMMITTED (1 << 1) + __u64 hdm_regs_size; /* Size in bytes of HDM register block */ + __u64 hdm_regs_offset; /* Byte offset within the BAR to the HDM decoder b= lock */ + __u64 dpa_size; /* Device Physical Address (DPA) size in bytes */ + /* + * Region indices for the two CXL VFIO device regions. + * Avoids forcing userspace to scan all regions by type/subtype. + */ + __u32 dpa_region_index; /* VFIO_REGION_SUBTYPE_CXL */ + __u32 comp_regs_region_index; /* VFIO_REGION_SUBTYPE_CXL_COMP_REGS */ +}; + /** * VFIO_DEVICE_GET_REGION_INFO - _IOWR(VFIO_TYPE, VFIO_BASE + 8, * struct vfio_region_info) @@ -370,6 +410,18 @@ struct vfio_region_info_cap_type { */ #define VFIO_REGION_SUBTYPE_IBM_NVLINK2_ATSD (1) =20 +/* 1e98 vendor PCI sub-types (CXL Consortium) */ +/* + * CXL memory region. Use with region type + * (PCI_VENDOR_ID_CXL | VFIO_REGION_TYPE_PCI_VENDOR_TYPE). + * DPA memory region (fault+zap mmap) + */ +#define VFIO_REGION_SUBTYPE_CXL (1) +/* + * HDM decoder register emulation region (read/write only, no mmap). + */ +#define VFIO_REGION_SUBTYPE_CXL_COMP_REGS (2) + /* sub-types for VFIO_REGION_TYPE_GFX */ #define VFIO_REGION_SUBTYPE_GFX_EDID (1) =20 --=20 2.25.1 From nobody Tue Apr 7 19:54:44 2026 Received: from PH8PR06CU001.outbound.protection.outlook.com (mail-westus3azon11012007.outbound.protection.outlook.com [40.107.209.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E7ED235DA79; Wed, 11 Mar 2026 20:36:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.209.7 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773261377; cv=fail; b=LC3vXo6Sevz+/avSd4+WUqwUkozKgc6FYOzrXKZQjJ6pxD5QwDbeWvCXFQIWnk5XHJ7dRGM/JfdqszzUc3JAmbKWbn81L+l4frQpIJXs1rznoj1pzrjp56go4Oe9MDmWDV0DTq+BY7JuNJpgVr6x33Ae5G+SD5d0PDWi2SwTr7I= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773261377; c=relaxed/simple; bh=5UCi5vtHgPBGAyrMhMcHPC2C7g1O+sN9XFRK64tNCyo=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=CpRxzAKaEM5z81aBgZWgzADQJHrGXuFypXxrnzNnn+0ZS0vpl2esFCUdjq2AJWsR51NISN6tnu0lhxqpIAYi6BSnVg08d0FEjNjRfOrNocAwBTVpfVOa22XGrbZ6DkI6plCz7XGcvds8VOgGCSBykrwEz7DpMm7PYGxEW0xZOYw= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=FQtW3Crn; arc=fail smtp.client-ip=40.107.209.7 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="FQtW3Crn" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=s8qESA5J2eTEFOwHoRdF2hUURHpiEeICbvVoxAGYfYkIKDXuZBv0f7bVThA//2lnrKPbDI1hRjYWEfeechQLVd7UEJc8NleM0MA0FKV91Gm3LKQrnU5bpk17VML3jkiWe7GoMb5xwxTNS+IJXQ2CNidSXVj7w/JTzA3wiCzHDkdB8LRGUDg6FbBCFsS/V7r1R2vRUGwmRv6gDqC4n7dQbUbnrB+MYA/gxKa/fBKO2Y65D+SIVWmugxBC+E1ffvfUHVsZsj7YNj4pgh6ZRqjGWtQHyxBSDtnwZNZuxwRnosnMrevnrTo4BXYBIaTqJ+ntiAR4ZC0bspeYL0+/1Y/MTQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=gKQeIbw+oZ1unfS+DimFSNMeVNcHqzhkZRuUmTi/HfM=; b=auD0XjyONGofECoVAQnh8jfYxlIE0mE6/GzYkRbmFHS3+b2oSKA9FBCs1tPC8MLQLOm4L6LezCTSwrblQ3impahdHZwHr9HnjhLf9A5wYdXrMScddcl3hKNPf6u2n25+20Swu0ijxioPzOIB2Ri2gpH9gzB2ugiQiLFgbEalDOIq/HYf6DC7gGEdoRXROfJteZp/3TDiHdh7BmuHPLxlJH2SybTi6UBKM98f7V+zOajbxQlWuRwOrF3RiHe/K281KSt4/s7cdJHij9eZOyhNjOmdsbqXSLBly7h67xOZ7Bpur8icLdQUewtVqi0Z6tW9OFwc8q85Nm7VU5+8phazzA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=gKQeIbw+oZ1unfS+DimFSNMeVNcHqzhkZRuUmTi/HfM=; b=FQtW3Crn3GjUsc7zYHXwY4GOQqXs61ZEC0KQsIkGN1Zg9F8flkdJowdrIlAQ8yK1udtdtbUonSu6xGuMExgW5b2dLYMJRQMzvwQHE64uKKvBtm7ALqYFJA+0KlGvrZVoHnV3JH575eo9eUlYBWFM89KjcgbS/cQwJNy/Ert2RQOj0d/FKltz52YYnne0tI8RxDxcgTkbbuSley36hSZ/rPzoeZeQl+lIy78HHrnj7OOLsDUfCuk7dBDnORhA5xW+ZNIrRxb0+JoHbe8H+MXT8oZTIiaPHpRrLyJQUzSUq5/VNzOEA5M/zp8i2TMq/nJYqBNRz0ExEDYsU+wcgrovVw== Received: from CH5P222CA0015.NAMP222.PROD.OUTLOOK.COM (2603:10b6:610:1ee::13) by SA5PPF590085732.namprd12.prod.outlook.com (2603:10b6:80f:fc04::8ca) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9700.11; Wed, 11 Mar 2026 20:36:08 +0000 Received: from DS2PEPF00003444.namprd04.prod.outlook.com (2603:10b6:610:1ee:cafe::91) by CH5P222CA0015.outlook.office365.com (2603:10b6:610:1ee::13) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9678.27 via Frontend Transport; Wed, 11 Mar 2026 20:36:11 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by DS2PEPF00003444.mail.protection.outlook.com (10.167.17.71) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9678.18 via Frontend Transport; Wed, 11 Mar 2026 20:36:07 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Wed, 11 Mar 2026 13:35:50 -0700 Received: from rnnvmail201.nvidia.com (10.129.68.8) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Wed, 11 Mar 2026 13:35:49 -0700 Received: from nvidia-4028GR-scsim.nvidia.com (10.127.8.11) by mail.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20 via Frontend Transport; Wed, 11 Mar 2026 13:35:42 -0700 From: To: , , , , , , , , , , , , , , , , , CC: , , , , , , , Subject: [PATCH 07/20] vfio/pci: Add CXL state to vfio_pci_core_device Date: Thu, 12 Mar 2026 02:04:27 +0530 Message-ID: <20260311203440.752648-8-mhonap@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20260311203440.752648-1-mhonap@nvidia.com> References: <20260311203440.752648-1-mhonap@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS2PEPF00003444:EE_|SA5PPF590085732:EE_ X-MS-Office365-Filtering-Correlation-Id: 77f2a5cd-2268-419e-28e2-08de7fadd31f X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|36860700016|82310400026|1800799024|7416014|376014|921020|22082099003|56012099003|18002099003; X-Microsoft-Antispam-Message-Info: Xg6wkPVu6kYaz6Bt2TGhCZtW4U+lPaXQMBn29WwJ6/OtNnrOSgC2rSbkjVDrIYrIHE48v6JRulv1dwAG89aX61Bwaim8WLwMY7m0bPooIT6Z8LkcOhdv6V/9Zjzq1vu4Pe7S4FOeBpcfFY2rO2I8SDaYSfZb782pEXfpEhgoz/GS7owlxTSJQxEs6Zcx/YMGBip/SwyYkRk23hRicatO+2IVutP5joQqwj1rTPHYnZuOTkdYf8+2zAPuEjG4AlU97PJfuQSPoTeEiGLgPIM2KQ1oAkeJuLwpiPVnD8mqIjTXLXh9dGroVqrlABhZuSBdUVlSHzNAq9uuy1uo4v3R+BcH4CgmQC3He/yEJLoj8vnAejprjZuqQ5AeVP6dzjJRRwNdefkLIIUnkTiHEbj7h9OQt2MJwmrUFR0kbyXp5LvFnrEGfW4mQ34yaQ9fAVuagznpIg6QbCs2fAYQ2pHj+/wuggaMEbs0Z2Qc/i2L0Fe35XxBKs/50EncxbPhNu7KwLyM5OkYGj34FtdWa87fKD8ZdC9FcL7MpVUg4kwd9OhoKRIR8akivHfdL08j1l0z7r/KHIqXFro3feM2ScuLYeoaPBTD+yiDI92QK3Aqvah3jw/sJ56K1NSS6FmQ8/05hrpJLyKfK6Uzn7qSNrODcy+fWwOUI8DpHzfw6d/tCEUKnaK2Zhb55H5p16eAiTvRJiVsm0WiQcigwbiUXZDjW0cAiy4amxK/6deonMqlMDNQbTdNU4h/OFxmjA0AxnIsLDkeKL2kwj959YnG3o+8dAQphEg5oaFMJfawPTLSNKx0kWV9SbGcembJzzMH2fHu X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(36860700016)(82310400026)(1800799024)(7416014)(376014)(921020)(22082099003)(56012099003)(18002099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: oo7Wi4FlQYQeJAsSCRHqs9liv2uD4GEolmttz1H/L2Vwd5BKKLco4ZgrgBq+O2KMqUj3FZPt2YHyFy2X4F7SR1BxyxmxHXPSfHz7X0cUuCynbHi7A3axjoiy2cYJ22O11CFRjzpF0qnAzZUzCvlUJMcs0vf7K73/tUrKDH+p1EaA+0Kg0GqjhC+tQ7VHe0Bo5smiGlsQFRKTWcoumVYEBrZ8gOh/0OZAh28fedgu/iDuLhLzz3ZobzbIdWgOzBuXD0qUEeTdAQwOXXwK7Vn5Q7uLJqc/lDDDsQfhDi3g0WwOKuwVuUuovSq+Onq+vq1n3MnZ/e1LqtJq2A0FjevjTt+/a1RbRK+9uRxomXltczx7O7m75RPxteTsQFmuTeSUfEnAkHvqo6SL9A7Db95PvswZAM2E2q/t4Micyz8uIA/JTx5g79zvUXblz9v21DBV X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Mar 2026 20:36:07.9780 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 77f2a5cd-2268-419e-28e2-08de7fadd31f X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DS2PEPF00003444.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA5PPF590085732 Content-Type: text/plain; charset="utf-8" From: Manish Honap Add CXL-specific state to vfio_pci_core_device structure to support CXL Type-2 device passthrough. The new vfio_pci_cxl_state structure embeds CXL core objects: - struct cxl_dev_state: CXL device state (from CXL core) - struct cxl_memdev: CXL memory device - struct cxl_region: CXL region object - Root and endpoint decoders Key design point: The CXL state pointer is NULL for non-CXL devices, allowing vfio-pci-core to handle both CXL and standard PCI devices with minimal overhead. This will follow the approach where vfio-pci-core itself gains CXL awareness, rather than requiring a separate variant driver. Signed-off-by: Manish Honap --- drivers/vfio/pci/cxl/vfio_cxl_priv.h | 29 ++++++++++++++++++++++++++++ include/linux/vfio_pci_core.h | 3 +++ 2 files changed, 32 insertions(+) create mode 100644 drivers/vfio/pci/cxl/vfio_cxl_priv.h diff --git a/drivers/vfio/pci/cxl/vfio_cxl_priv.h b/drivers/vfio/pci/cxl/vf= io_cxl_priv.h new file mode 100644 index 000000000000..818a83a3809d --- /dev/null +++ b/drivers/vfio/pci/cxl/vfio_cxl_priv.h @@ -0,0 +1,29 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Common infrastructure for CXL Type-2 device variant drivers + * + * Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved + */ + +#ifndef __LINUX_VFIO_CXL_PRIV_H +#define __LINUX_VFIO_CXL_PRIV_H + +#include +#include + +/* CXL device state embedded in vfio_pci_core_device */ +struct vfio_pci_cxl_state { + struct cxl_dev_state cxlds; + struct cxl_memdev *cxlmd; + struct cxl_root_decoder *cxlrd; + struct cxl_endpoint_decoder *cxled; + resource_size_t hdm_reg_offset; + size_t hdm_reg_size; + resource_size_t comp_reg_offset; + size_t comp_reg_size; + u32 hdm_count; + u16 dvsec; + u8 comp_reg_bar; +}; + +#endif /* __LINUX_VFIO_CXL_PRIV_H */ diff --git a/include/linux/vfio_pci_core.h b/include/linux/vfio_pci_core.h index 1ac86896875c..cd8ed98a82a3 100644 --- a/include/linux/vfio_pci_core.h +++ b/include/linux/vfio_pci_core.h @@ -30,6 +30,8 @@ struct vfio_pci_region; struct p2pdma_provider; struct dma_buf_phys_vec; struct dma_buf_attachment; +struct vfio_pci_cxl_state; + =20 struct vfio_pci_eventfd { struct eventfd_ctx *ctx; @@ -138,6 +140,7 @@ struct vfio_pci_core_device { struct mutex ioeventfds_lock; struct list_head ioeventfds_list; struct vfio_pci_vf_token *vf_token; + struct vfio_pci_cxl_state *cxl; struct list_head sriov_pfs_item; struct vfio_pci_core_device *sriov_pf_core_dev; struct notifier_block nb; --=20 2.25.1 From nobody Tue Apr 7 19:54:44 2026 Received: from SN4PR0501CU005.outbound.protection.outlook.com (mail-southcentralusazon11011048.outbound.protection.outlook.com [40.93.194.48]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 431BE355F43; Wed, 11 Mar 2026 20:36:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.93.194.48 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773261385; cv=fail; b=h3sVTcef1qHtFxx0+79UQILPyJMJDYA15w8X6xgVIKbbH5C3Ci9k7BeimryBtHNoWwTGWsyMnh20VjPR5izzS06CZuZTy29H5Zlxilr8/3CUX5X7Aen0sT/nca1k+YpjNWFtfzFi+vS85kJOgHZaNJBP7OQeqPKFTJXBnGc4/IY= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773261385; c=relaxed/simple; bh=L/1hLxehiOmD5kg10I4WYspKbrBzK3/WkWKm7YXMLrg=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=klvTrn2ZYMH4kHlcD7m899JVPd4bBJfEAdviFyJXAMgLRac5Tj01vu1zhtpwJP+Sp8oVAUgTypXK1W9YR2towp9GcpXr6fUjBqsLvCPidDsckrx8j0HrSvs7v7MFWrtC8FxzMCiINSnUaegRjEk/Av8nuieVT/57NIEzOvSrc84= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=A4eIuhx8; arc=fail smtp.client-ip=40.93.194.48 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="A4eIuhx8" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=OV+d4T2ohNJjJ5iZ0Q7SR9yXxkLaR239PdJ+XwbqoG6QDC/X8hvidXRsicBD1iHV0E5iZhybM2pmXkDN3TkDb0FO5JDg5I8rJMF/Xix7XFn9CKPH1dWBVeiPYQ1lEPREVY05K8YgHkgF4L2dajGNkfeS/Y9Qazrx38aVRymPEBw8m8/ExvNeGlenJeKCemVXXeACfXJMXOIJcLJtFYPaB2xAd1rlFwUX1cQF0RjviqGK+m61c3wSfiiYnmhsXoFqwXw/wzbcx72Nc8kMcr56LI+00a8WAEKml/CJIuOnTM5vWh0V1tbHAmM7ii2XjDQgKqVAnfZISR8n8QMu/7IKnw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=XOGLVrR/t/2x6MxDg6EgsYuPheuqVj0oFy1iZQjpB/g=; b=VMpwJOjQt4VnNqc0vzrvHQFPubhPb76+gYzRSgQHGHMW5FQbKKLMFbnmR54Ak9jDkLk2XG31XdHUIyCq+81xBST3QYvEFTKdoxBsaYdxWVuI4BGRFd8vYAqUGAbDRupE1ix9jPxxASznXAF1n7TnIhiTRK4rpKEsdrMbLRE+VB2LJ+r2wp4X4kqcrLb7V9890cBIQNe0Xj4FZfrG3GCmCt6NZmUgrww5cHD3tTQ5PdYGL6rdBfWB8jqjy45EEMGWjVU0k9fqKc6olqKq5JNCqa2rck/4PlvAXv1ZoUCbbBjN6v6jMumbAJLINnJafHswKy3l3F3ZAlGctT2fCC/GYA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=XOGLVrR/t/2x6MxDg6EgsYuPheuqVj0oFy1iZQjpB/g=; b=A4eIuhx804J3cRaJI/RlG/D2Rg3Shwv7J7pDxUny07l4vQwB3K8ucr+Jegjy+lk8IMuB8rjStKWBZQAGKruT7atwQj1ZJZFpZxXeSdmASTDxGtNCwi1NWke7xZpzrucBYhDi3Yxew9DN63S/7f3fatWFvkIYwQCYVFqHb5SXWaTap5yK+EbrJmK8wil6YsgZrrU1vZh9dg+VRpDuYQ8dvdwlaNBZR/UUB3auVpqK9bgmf2vDdOKWDFXa18M7wpCSwxJGbUvChSr6t3XShgvZl74NXwv6SO0LJGVUmYwwV9S76ad58+Ipabjy81nhvISBf+vJtfEM3Ftx15TNy8Sbhw== Received: from CH0PR03CA0084.namprd03.prod.outlook.com (2603:10b6:610:cc::29) by LV8PR12MB9262.namprd12.prod.outlook.com (2603:10b6:408:1e7::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9700.11; Wed, 11 Mar 2026 20:36:15 +0000 Received: from DS2PEPF00003446.namprd04.prod.outlook.com (2603:10b6:610:cc:cafe::a) by CH0PR03CA0084.outlook.office365.com (2603:10b6:610:cc::29) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9678.27 via Frontend Transport; Wed, 11 Mar 2026 20:35:59 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by DS2PEPF00003446.mail.protection.outlook.com (10.167.17.73) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9678.18 via Frontend Transport; Wed, 11 Mar 2026 20:36:15 +0000 Received: from rnnvmail202.nvidia.com (10.129.68.7) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Wed, 11 Mar 2026 13:35:57 -0700 Received: from rnnvmail201.nvidia.com (10.129.68.8) by rnnvmail202.nvidia.com (10.129.68.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Wed, 11 Mar 2026 13:35:57 -0700 Received: from nvidia-4028GR-scsim.nvidia.com (10.127.8.11) by mail.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20 via Frontend Transport; Wed, 11 Mar 2026 13:35:50 -0700 From: To: , , , , , , , , , , , , , , , , , CC: , , , , , , , Subject: [PATCH 08/20] vfio/pci: Add vfio-cxl Kconfig and build infrastructure Date: Thu, 12 Mar 2026 02:04:28 +0530 Message-ID: <20260311203440.752648-9-mhonap@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20260311203440.752648-1-mhonap@nvidia.com> References: <20260311203440.752648-1-mhonap@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS2PEPF00003446:EE_|LV8PR12MB9262:EE_ X-MS-Office365-Filtering-Correlation-Id: 1f442780-f628-4adb-98c6-08de7fadd770 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|36860700016|1800799024|376014|82310400026|7416014|921020|22082099003|18002099003|56012099003; X-Microsoft-Antispam-Message-Info: SaTkl30cKR/QBVjt4ZZnnLf9GYbsgqueejCy9oBDNjghyevxVPPcoFky9r4ia9RmE4JxbSSezFIoNtqQRG/hVQ+XdGOXksBhK8HsJn7kpksJjeKuSkxplUKHxsPsAi7OIewRfD9AWXisWi+hc+ghctM/ire6qoYmjUIAaNphfcv8GFHKBuPAeSlTl/Oasw1ejxvr1PueLbFvDdmxkm+w9KEevDNs25QLoAWUnkP29X1ODDkvNadUs1W6s6pIhUb4rs4NcXNxWtmg7tQZONOzVkwv4BYWMu1uVOwdZXgSKWNvO1+4b0XvScqXT+tpXXZ76qu1XyjpZX5TFg32T0xxC2h3Djhlm1FruogL1C3++a+H4SwnaEBB7ExJp9ud2xKi8bo4esyd5dHGYN+3FJHy+P47ndJAM7UvQNGKzuUMsey3n9CrOI20L5hcR7VF31VCkMKhelZrboXZzDk4PgzYMo3vV0oiNb+QX2usoVTlU6brmSgelKME0A/0589GPYwT8Vy94ponFj+++iwfIJaWL6Q2+V5EGgl1Wm0l8G+sVeVIev3L+lcIJ9+YB66eFDUriACxk2b6WdsalgOlMdhFDsrGZX6zfKxtYPlk7GTUN2NN33DTWl2+L2DgZZX0i6C8ityN9aPg+p3hTm2x0lLE+CnY2pMfwn4km4KyQpWevs67FThRgMu5g0GPCWRznSR8iZ5TfgFqVnMi/ilrYwMDrC4sHH84mFP5IZmwvGVVhRO/pj6dROyd0ZYnmDDIMtavBT19bnyuuJpu5GTbrmKJ0iunoubu8aeLie3tKX0bra4g5W3w7+85za2Hg/Jf1HMw X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(36860700016)(1800799024)(376014)(82310400026)(7416014)(921020)(22082099003)(18002099003)(56012099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: BQZrukkaxcaFFO4pn+Zm+SBnEKIIUJ3XgM0Pt+cNTZxv4J/wihHZa4gGfj7Cf7ngbcVbLN/wHDPU22/GVy50dxXegHhAMZeCJ5ORkKHovlzFDBDsjC11zX6mjqF+AQRQFu/j9tHRB5gngveEJiFdj0DMYlJ4BtX9RCD+kBJv4a2/KkptXdxhzFtxeXWxj8eHjAMBYwrRomDR5mcvxPw7p4PUWXYQBtJi6VLVYI4kXHGivBpSw8Y6hBJ8VI4qHCW6ghtj+IH7VpelB8KbUxCFfKCCH77dK2V+OUSt/4FFP4gAIsMmLHc8Qyqy4a1LK8avIsyEqd4GwITJeVYcp8CptwzegplBOPcUwPpjok9/alu+LmzVy3F84KH+bgvOxCjW/ovXDlY7RLyl1Ou98Du0fE5ehqBdxMYAizXglP+EAufppagbbjo7YHLckAvkBvS7 X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Mar 2026 20:36:15.2266 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 1f442780-f628-4adb-98c6-08de7fadd770 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DS2PEPF00003446.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: LV8PR12MB9262 Content-Type: text/plain; charset="utf-8" From: Manish Honap Introduce the Kconfig option CONFIG_VFIO_CXL_CORE and the necessary build rules to compile CXL Type-2 passthrough support into the vfio-pci-core module. The new option depends on VFIO_PCI_CORE, CXL_BUS and CXL_MEM. Wire up the detection and cleanup entry-point stubs in vfio_pci_core_register_device() and vfio_pci_core_unregister_device() so that subsequent patches can fill in the CXL-specific logic without touching the vfio-pci-core flow again. The vfio_cxl_core.c file added here is an empty skeleton; the actual CXL detection and initialisation code is introduced in the following patch to keep this build-system patch reviewable on its own. Signed-off-by: Manish Honap --- drivers/vfio/pci/Kconfig | 2 ++ drivers/vfio/pci/Makefile | 1 + drivers/vfio/pci/cxl/Kconfig | 7 ++++++ drivers/vfio/pci/cxl/vfio_cxl_core.c | 35 ++++++++++++++++++++++++++++ drivers/vfio/pci/vfio_pci_core.c | 4 ++++ drivers/vfio/pci/vfio_pci_priv.h | 14 +++++++++++ 6 files changed, 63 insertions(+) create mode 100644 drivers/vfio/pci/cxl/Kconfig create mode 100644 drivers/vfio/pci/cxl/vfio_cxl_core.c diff --git a/drivers/vfio/pci/Kconfig b/drivers/vfio/pci/Kconfig index 1e82b44bda1a..b981a7c164ca 100644 --- a/drivers/vfio/pci/Kconfig +++ b/drivers/vfio/pci/Kconfig @@ -68,6 +68,8 @@ source "drivers/vfio/pci/virtio/Kconfig" =20 source "drivers/vfio/pci/nvgrace-gpu/Kconfig" =20 +source "drivers/vfio/pci/cxl/Kconfig" + source "drivers/vfio/pci/qat/Kconfig" =20 source "drivers/vfio/pci/xe/Kconfig" diff --git a/drivers/vfio/pci/Makefile b/drivers/vfio/pci/Makefile index e0a0757dd1d2..ecb0eacbc089 100644 --- a/drivers/vfio/pci/Makefile +++ b/drivers/vfio/pci/Makefile @@ -1,6 +1,7 @@ # SPDX-License-Identifier: GPL-2.0-only =20 vfio-pci-core-y :=3D vfio_pci_core.o vfio_pci_intrs.o vfio_pci_rdwr.o vfio= _pci_config.o +vfio-pci-core-$(CONFIG_VFIO_CXL_CORE) +=3D cxl/vfio_cxl_core.o vfio-pci-core-$(CONFIG_VFIO_PCI_ZDEV_KVM) +=3D vfio_pci_zdev.o vfio-pci-core-$(CONFIG_VFIO_PCI_DMABUF) +=3D vfio_pci_dmabuf.o obj-$(CONFIG_VFIO_PCI_CORE) +=3D vfio-pci-core.o diff --git a/drivers/vfio/pci/cxl/Kconfig b/drivers/vfio/pci/cxl/Kconfig new file mode 100644 index 000000000000..41d60dc0de2d --- /dev/null +++ b/drivers/vfio/pci/cxl/Kconfig @@ -0,0 +1,7 @@ +config VFIO_CXL_CORE + bool "VFIO CXL core" + depends on VFIO_PCI_CORE && CXL_BUS && CXL_MEM + help + Core library for VFIO CXL Type-2 device support (enlightened path). + When enabled, vfio-pci-core can detect and manage CXL Type-2 devices + without a separate variant driver. diff --git a/drivers/vfio/pci/cxl/vfio_cxl_core.c b/drivers/vfio/pci/cxl/vf= io_cxl_core.c new file mode 100644 index 000000000000..7698d94e16be --- /dev/null +++ b/drivers/vfio/pci/cxl/vfio_cxl_core.c @@ -0,0 +1,35 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * VFIO CXL Core - Common infrastructure for CXL Type-2 device variant dri= vers + * + * Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved + * + * This module provides common functionality for VFIO variant drivers that + * support CXL Type-2 devices (cache-coherent accelerators with attached m= emory). + */ + +#include +#include +#include +#include + +#include "../vfio_pci_priv.h" +#include "vfio_cxl_priv.h" + +MODULE_IMPORT_NS("CXL"); + +/** + * vfio_pci_cxl_detect_and_init - Detect and initialize CXL Type-2 device + * @vdev: VFIO PCI device + * + * Called from vfio_pci_core_register_device(). Detects CXL DVSEC capabili= ty + * and initializes CXL features. On failure vdev->cxl remains NULL and the + * device operates as a standard PCI device. + */ +void vfio_pci_cxl_detect_and_init(struct vfio_pci_core_device *vdev) +{ +} + +void vfio_pci_cxl_cleanup(struct vfio_pci_core_device *vdev) +{ +} diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_c= ore.c index 3a11e6f450f7..b7364178e23d 100644 --- a/drivers/vfio/pci/vfio_pci_core.c +++ b/drivers/vfio/pci/vfio_pci_core.c @@ -2181,6 +2181,8 @@ int vfio_pci_core_register_device(struct vfio_pci_cor= e_device *vdev) if (ret) goto out_vf; =20 + vfio_pci_cxl_detect_and_init(vdev); + vfio_pci_probe_power_state(vdev); =20 /* @@ -2224,6 +2226,8 @@ void vfio_pci_core_unregister_device(struct vfio_pci_= core_device *vdev) vfio_pci_vf_uninit(vdev); vfio_pci_vga_uninit(vdev); =20 + vfio_pci_cxl_cleanup(vdev); + if (!disable_idle_d3) pm_runtime_get_noresume(&vdev->pdev->dev); =20 diff --git a/drivers/vfio/pci/vfio_pci_priv.h b/drivers/vfio/pci/vfio_pci_p= riv.h index 27ac280f00b9..d7df5538dcde 100644 --- a/drivers/vfio/pci/vfio_pci_priv.h +++ b/drivers/vfio/pci/vfio_pci_priv.h @@ -133,4 +133,18 @@ static inline void vfio_pci_dma_buf_move(struct vfio_p= ci_core_device *vdev, } #endif =20 +#if IS_ENABLED(CONFIG_VFIO_CXL_CORE) + +void vfio_pci_cxl_detect_and_init(struct vfio_pci_core_device *vdev); +void vfio_pci_cxl_cleanup(struct vfio_pci_core_device *vdev); + +#else + +static inline void +vfio_pci_cxl_detect_and_init(struct vfio_pci_core_device *vdev) { } +static inline void +vfio_pci_cxl_cleanup(struct vfio_pci_core_device *vdev) { } + +#endif /* CONFIG_VFIO_CXL_CORE */ + #endif --=20 2.25.1 From nobody Tue Apr 7 19:54:44 2026 Received: from MW6PR02CU001.outbound.protection.outlook.com (mail-westus2azon11012015.outbound.protection.outlook.com [52.101.48.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9594937C921; Wed, 11 Mar 2026 20:36:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.48.15 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773261390; cv=fail; b=rng+1odIOpuDu1e6L1WK3DpQ0AIfoWXo1r68VZLd58anny/7AsUMxg7ET7xrAiLORL5DFlx7ZKE02VaVLnEuXprcQ/cv0fX81sLHu8fmV02YlqWBUNFmYq11EJB/Ogyp+4SogtF7ndYoEhDW69bYE/sby5nqPH1eFppg0HUJxmU= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773261390; c=relaxed/simple; bh=cM4QtJwwBlDpMqFmy32r5EfWXUgBUB4hEzDPByOyz1Y=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=s3zBIuapH8trZhfLxUQ57B1yiPbCWrq0WJ4GyeYnxuE+MLPhSPbzz/eq9B7Fa7T1blUh8qa1u2o/6fH1+/jGpyR2VuUVBF27zRVIi9uXW6Ub46IUKvfWvVA6vlE8Qap3T7u/HkbvJ9NkeYSekRnNJLKMv8KGmjKRezgO77BKTBs= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=Z7VdMxJ5; arc=fail smtp.client-ip=52.101.48.15 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="Z7VdMxJ5" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=gXgiEItWWtOdS88aDPhQz0rOBh6iAUu6C95D9c+/HJopm010Bm1qzuAWmrqxweAgY9bDUfXxuevbWAypfSeEuS7DJg/4W5vK2gIu/l59ElQB11MbpzslRnZjInGyul2zOOth/8MmCHrSciy+Ij56UT5e4r3UmQpnpMC7LIXl85xWgvUw1mgJwZTwuhZamS9T5IxbYHl73QmkNdiDyLD9STlVNEfNahq3S5FcnY41p7PHAfilO5JNwTg8oD0+P4Gff01VIPY8mUftyZh8mxdNGrk3td60pAVipeO+C+a7OghyARmE4CU0zqsuxvQlxI8B3L9hM1rQXgywOiMTOI0/ag== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=AeANyQ+Rx1pFztcnXjoT47J7YdYtjJUvNDPSt0EJbIc=; b=fX074Xk83IioT3mPz566V+/lPlfO/aod+EjuXxoQiUdPYLQHExVmZ4p6A4vwNuPLKOrQTimVtbe/O9Ic6F5bkv/E8lDUp4zGtwLNGwOkPh2KVznW9urJFMT2hEgzlxCZRFJ+zZ2uE/wJ5Q1yr3+1/mJvPIj+go6Gc3is+bQyFUcMZB5zhc6hGdoAwpIc00+wLpGf3XgCszAaUoXxOjZvzRjFRR7UwECncfEcDoX4Mozhnj3IXvE6R7MvkPvIh+UkVYSLpFKbgmQs7ib9vAfvZbmDuQnFRjyORrrDhWhNegrO0GGnYZmk0qbPhpdeb4WR/kk6+CBNJ3rcXNh5HxPGfw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=AeANyQ+Rx1pFztcnXjoT47J7YdYtjJUvNDPSt0EJbIc=; b=Z7VdMxJ5EXkxc3y0cTCwIBcdYLu3ELnpCiWEdbzkOUAXtyQ5/BVIF/G5Z5zwUb+j4WtvNeFrPzN4mIRJiLv8fRn0wAtS2g8UZs+ikz+NyN9CnNKaBbzWaTZG87zm9tvz6XYyEC2ZDncdkbTfV6RGDs92l0eqgP1O3j11LI46INPIBHoZOvbAswhUmv/4rJXZwrIctrfmkqeNEQ4mathcHaJ0Z1qSGvs1I8P314RUyqPWfV7CTgzOvRB+Y4qxi+5OFYthMmo4T6EQ74pPKCWaJGBQ1seDzQ3nQDihj1UstME55T6GOGNr2o5LLUPXpvzi5JcniOVnCFgCgRpWXigPSQ== Received: from CH0PR03CA0084.namprd03.prod.outlook.com (2603:10b6:610:cc::29) by LV9PR12MB9807.namprd12.prod.outlook.com (2603:10b6:408:2eb::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9700.11; Wed, 11 Mar 2026 20:36:24 +0000 Received: from DS2PEPF00003446.namprd04.prod.outlook.com (2603:10b6:610:cc:cafe::15) by CH0PR03CA0084.outlook.office365.com (2603:10b6:610:cc::29) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9678.27 via Frontend Transport; Wed, 11 Mar 2026 20:36:08 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by DS2PEPF00003446.mail.protection.outlook.com (10.167.17.73) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9678.18 via Frontend Transport; Wed, 11 Mar 2026 20:36:24 +0000 Received: from rnnvmail202.nvidia.com (10.129.68.7) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Wed, 11 Mar 2026 13:36:05 -0700 Received: from rnnvmail201.nvidia.com (10.129.68.8) by rnnvmail202.nvidia.com (10.129.68.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Wed, 11 Mar 2026 13:36:04 -0700 Received: from nvidia-4028GR-scsim.nvidia.com (10.127.8.11) by mail.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20 via Frontend Transport; Wed, 11 Mar 2026 13:35:58 -0700 From: To: , , , , , , , , , , , , , , , , , CC: , , , , , , , Subject: [PATCH 09/20] vfio/cxl: Implement CXL device detection and HDM register probing Date: Thu, 12 Mar 2026 02:04:29 +0530 Message-ID: <20260311203440.752648-10-mhonap@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20260311203440.752648-1-mhonap@nvidia.com> References: <20260311203440.752648-1-mhonap@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS2PEPF00003446:EE_|LV9PR12MB9807:EE_ X-MS-Office365-Filtering-Correlation-Id: 7ae02261-cdb5-4829-cc92-08de7faddcb9 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|36860700016|376014|7416014|1800799024|56012099003|18002099003|22082099003|921020; X-Microsoft-Antispam-Message-Info: xwVNvWmd5H7OWJrflJZx8tXvSWxhJo5b8h3nl1L2MA7dTWDYWN46Mb3UZanqmEyXnQ2k8cGoc223W5SBa40KmseqjRAqZpQOdyMbk3e9I1pWBj46iZKFvDjgi1HLC0T4WaMfNEEJW4zEtjwi1ryWHIfa+z8Q9E0ZjFC9bScX5ViHxCHURkhjRp742fMlBgLIzc7GD97vUeZLgmqg1SesAx/dlSQvknrGRRJ2/QzOlTpmonXirSKADrCbml845om26xJEcIGJdrMCMdadzXP/wo0bvLplXSO0NDNumrozWZrjz6bzFCqSMt5wce90ZD8arN8Jd00WlyUZKf1/KHYs5zURcs+CqkHwKR9eJPVO3DHGyImsDPM6c57F9gDbvMhTK8J/bFhn1zaKFBjWs7YHKXfwXjhjjr/26XEokxoIX5C+GlAZJnFrEkc0FA9rCGdITHBgPRU30HtQ0esMMUCFsh2WiLYPgpD5+k1rZ9L9LAFQLpRBeRUrrtzXk6Ak/eVk3+4YSElJ42kRzq/DOpShzgHQEfofFESQ7HbM1/xd5tDggYs595dwErC5dH4TQIH5dLatbe6+TRFHS9P7elJVoeVOj5J0vzsCPJ3ByC9ZSZCGTbiFP0ecw51PVRJN1sWZpiMSm36MjnG4kX7nwdYuwY2Q076JgWMZfj56mKiJqyuD9zfgfB+VuJSWqQ9Y0Dz21r7ZqhuKdo3AZVzsya9997rJXjxIi4ms7p+ZF4a/CHUoBVJ8w5qCic4wc725IQSeoBTsL72vz7GjT9zgMxJfYnTs6lOCyO48L8PLplwHno9dhuJqzVuqQX9WlyqQzFNN X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(82310400026)(36860700016)(376014)(7416014)(1800799024)(56012099003)(18002099003)(22082099003)(921020);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: FadSzgG0rNH9mTRQ/Or1Ej0ASQAEelrWOzyaQE7T4iJ5s7AtPBBZR2rW7HapLnl+hakB+juLQw+XZ04VjiHh2rxdc8ftvW280VCKp9GtmdsRaC7jm0Pcaz6DlJNWtjgbUu7KQZjkLGgMvK86s+F0IHs2jAWahcNOzOCGd1OrLboD+0igOpYjXmDvOFJ8azak5bhHcYKJhMeRvNePkMInneYhnsM1FUVkZu0YRS2B1EyzzzGVKbgqVWW0Jj2wV29bHOdfgKySgl2W2M5I/KnZ5xXCilcp7ImuJBDdialbNkukx34FTG/HZ9Y4JQerI8wYPXHDw2+ez6jEChXtYDJkyBzxUZTHusbGzWHi6Cus3NE2yO+AtWvEfIBaCnN+BOAz20oPz2mLe3Y4y7kHg/2UaC4cQO0CIvgdNDSwA9XfO9XCD0USciOwHIF1sFMkcvjq X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Mar 2026 20:36:24.0897 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 7ae02261-cdb5-4829-cc92-08de7faddcb9 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DS2PEPF00003446.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: LV9PR12MB9807 Content-Type: text/plain; charset="utf-8" From: Manish Honap Implement the core CXL Type-2 device detection and component register probing logic in vfio_pci_cxl_detect_and_init(). Three private helpers are introduced: vfio_cxl_create_device_state() allocates the per-device vfio_pci_cxl_state structure using devm_cxl_dev_state_create() so that lifetime is tied to the PCI device binding. vfio_cxl_find_bar() locates the PCI BAR that contains a given HPA range, returning the BAR index and offset within it. vfio_cxl_setup_regs() uses the CXL core helpers cxl_find_regblock() and cxl_probe_component_regs() to enumerate the HDM decoder register block, then records its BAR index, offset and size in the CXL state. vfio_pci_cxl_detect_and_init() orchestrates detection: 1. Check for CXL DVSEC via pcie_is_cxl() + pci_find_dvsec_capability(). 2. Allocate CXL device state. 3. Temporarily call pci_enable_device_mem() for ioremap, then disable. 4. Probe component registers to find the HDM decoder block. On any failure vdev->cxl is devm_kfree'd so that device falls back to plain PCI mode transparently. Signed-off-by: Manish Honap --- drivers/vfio/pci/cxl/vfio_cxl_core.c | 151 +++++++++++++++++++++++++++ drivers/vfio/pci/cxl/vfio_cxl_priv.h | 8 ++ 2 files changed, 159 insertions(+) diff --git a/drivers/vfio/pci/cxl/vfio_cxl_core.c b/drivers/vfio/pci/cxl/vf= io_cxl_core.c index 7698d94e16be..2da6da1c0605 100644 --- a/drivers/vfio/pci/cxl/vfio_cxl_core.c +++ b/drivers/vfio/pci/cxl/vfio_cxl_core.c @@ -18,6 +18,114 @@ =20 MODULE_IMPORT_NS("CXL"); =20 +static int vfio_cxl_create_device_state(struct vfio_pci_core_device *vdev, + u16 dvsec) +{ + struct pci_dev *pdev =3D vdev->pdev; + struct device *dev =3D &pdev->dev; + struct vfio_pci_cxl_state *cxl; + bool cxl_mem_capable, is_cxl_type3; + u16 cap_word; + + /* + * The devm allocation for the CXL state remains for the entire time + * the PCI device is bound to vfio-pci. From successful CXL init + * in probe until the device is released on unbind. + * No extra explicit free is needed; devm handles it when + * pdev->dev is released. + */ + vdev->cxl =3D devm_cxl_dev_state_create(dev, + CXL_DEVTYPE_DEVMEM, + pdev->dev.id, dvsec, + struct vfio_pci_cxl_state, + cxlds, false); + if (!vdev->cxl) + return -ENOMEM; + + cxl =3D vdev->cxl; + cxl->dvsec =3D dvsec; + + pci_read_config_word(pdev, dvsec + CXL_DVSEC_CAPABILITY_OFFSET, + &cap_word); + + cxl_mem_capable =3D !!(cap_word & CXL_DVSEC_MEM_CAPABLE); + is_cxl_type3 =3D ((pdev->class >> 8) =3D=3D PCI_CLASS_MEMORY_CXL); + + /* + * Type 2 =3D CXL memory capable but NOT Type 3 (e.g. accelerator/GPU) + * Unsupported for non cxl type-2 class of devices. + */ + if (!(cxl_mem_capable && !is_cxl_type3)) { + devm_kfree(&pdev->dev, vdev->cxl); + vdev->cxl =3D NULL; + return -ENODEV; + } + + return 0; +} + +static int vfio_cxl_setup_regs(struct vfio_pci_core_device *vdev) +{ + struct vfio_pci_cxl_state *cxl =3D vdev->cxl; + struct cxl_register_map *map =3D &cxl->cxlds.reg_map; + resource_size_t offset, bar_offset, size; + struct pci_dev *pdev =3D vdev->pdev; + void __iomem *base; + u32 count; + int ret; + u8 bar; + + if (WARN_ON_ONCE(!pci_is_enabled(pdev))) + return -EINVAL; + + /* Find component register block via Register Locator DVSEC */ + ret =3D cxl_find_regblock(pdev, CXL_REGLOC_RBI_COMPONENT, map); + if (ret) + return ret; + + /* Temporarily map the register block */ + base =3D ioremap(map->resource, map->max_size); + if (!base) + return -ENOMEM; + + /* Probe component register capabilities */ + cxl_probe_component_regs(&pdev->dev, base, &map->component_map); + + /* Unmap immediately */ + iounmap(base); + + /* Check if HDM decoder was found */ + if (!map->component_map.hdm_decoder.valid) + return -ENODEV; + + pci_dbg(pdev, + "vfio_cxl: HDM decoder at offset=3D0x%lx, size=3D0x%lx\n", + map->component_map.hdm_decoder.offset, + map->component_map.hdm_decoder.size); + + /* Get HDM register info */ + ret =3D cxl_get_hdm_reg_info(&cxl->cxlds, &count, &offset, &size); + if (ret) + return ret; + + if (!count || !size) + return -ENODEV; + + cxl->hdm_count =3D count; + cxl->hdm_reg_offset =3D offset; + cxl->hdm_reg_size =3D size; + + ret =3D cxl_regblock_get_bar_info(map, &bar, &bar_offset); + if (ret) + return ret; + + cxl->comp_reg_bar =3D bar; + cxl->comp_reg_offset =3D bar_offset; + cxl->comp_reg_size =3D CXL_COMPONENT_REG_BLOCK_SIZE; + + return 0; +} + /** * vfio_pci_cxl_detect_and_init - Detect and initialize CXL Type-2 device * @vdev: VFIO PCI device @@ -28,8 +136,51 @@ MODULE_IMPORT_NS("CXL"); */ void vfio_pci_cxl_detect_and_init(struct vfio_pci_core_device *vdev) { + struct pci_dev *pdev =3D vdev->pdev; + struct vfio_pci_cxl_state *cxl; + u16 dvsec; + int ret; + + if (!pcie_is_cxl(pdev)) + return; + + dvsec =3D pci_find_dvsec_capability(pdev, + PCI_VENDOR_ID_CXL, + PCI_DVSEC_CXL_DEVICE); + if (!dvsec) + return; + + ret =3D vfio_cxl_create_device_state(vdev, dvsec); + if (ret) + return; + + cxl =3D vdev->cxl; + + /* + * Required for ioremap of the component register block and + * calls to cxl_probe_component_regs(). + */ + ret =3D pci_enable_device_mem(pdev); + if (ret) + goto failed; + + ret =3D vfio_cxl_setup_regs(vdev); + if (ret) { + pci_disable_device(pdev); + goto failed; + } + + pci_disable_device(pdev); + + return; + +failed: + devm_kfree(&pdev->dev, vdev->cxl); + vdev->cxl =3D NULL; } =20 void vfio_pci_cxl_cleanup(struct vfio_pci_core_device *vdev) { + if (!vdev->cxl) + return; } diff --git a/drivers/vfio/pci/cxl/vfio_cxl_priv.h b/drivers/vfio/pci/cxl/vf= io_cxl_priv.h index 818a83a3809d..57fed39a80da 100644 --- a/drivers/vfio/pci/cxl/vfio_cxl_priv.h +++ b/drivers/vfio/pci/cxl/vfio_cxl_priv.h @@ -26,4 +26,12 @@ struct vfio_pci_cxl_state { u8 comp_reg_bar; }; =20 +/* + * CXL DVSEC for CXL Devices - register offsets within the DVSEC + * (CXL 2.0+ 8.1.3). + * Offsets are relative to the DVSEC capability base (cxl->dvsec). + */ +#define CXL_DVSEC_CAPABILITY_OFFSET 0xa +#define CXL_DVSEC_MEM_CAPABLE BIT(2) + #endif /* __LINUX_VFIO_CXL_PRIV_H */ --=20 2.25.1 From nobody Tue Apr 7 19:54:44 2026 Received: from BN8PR05CU002.outbound.protection.outlook.com (mail-eastus2azon11011015.outbound.protection.outlook.com [52.101.57.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 182C53750B1; Wed, 11 Mar 2026 20:36:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.57.15 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773261402; cv=fail; b=q1FtFvIDXpn4gSTM0bROTpyWtYUJnuqKwv1WRUPJYsyA4+PLVi15ljvEJ1OkQeKyNB4tr9RrmafJe3md/vzsjTqja1Za8Q58LRP+71+Q43A+J7x70t5YFXqRIJYrIvXZELUWsrEbG/LXyokSYjZJGkPlvU9cnYEhAJbVxj5u3Rg= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773261402; c=relaxed/simple; bh=wZuR0RUizxRtyEKK7ZCBPCczWbP4V7xIeNxSGPUnC+c=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=M8V8tnOzC1Rz59DqY550G9WVLN339OJmhqJ4Nc1Vs+eCfiS6FEGNgPYVH8JfG+4ff98pQreADCR74F3sT3pEIj8yWWT/Tl0/jhhX0vpKAUc13ku8v2GLf81xyYKCeyI6NRsaWQXsof2cuL8V3Cl4UHxqJ7e9bpEqHelFf+QpBbE= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=mRoCor6t; arc=fail smtp.client-ip=52.101.57.15 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="mRoCor6t" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=l91y3/FWUOA6BIGv5v4npPTei2XWxDmDHgSZH3Nveci2c6QfXnmR+5+osHXBnYyG7lVC/GBeTAJLYW28bWwqgK7e1HjjeGBeX92egPrw37494ddaq9p8N7kkjFho2oTSXesrNVrhf1YKD2wKj4dt1IoE/RaG96Y/+ubrrLkXWuZvIBX5EAwaRSldJ1g2bW3tABt1imSAWL7eRttWBK6ZduQ544ZOpirQ/R8saSXzGhR1VQfkyPRPTaiehwj3kco7sNvY/Pr0dvmutwsIhiDMsRtTvIrFRSPE3vlg4cAk7BelZHPEfHSAUGCcXaqlVDoqBSpNS9FzX9zXsD1tr9owOA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=61te41hgIyp+QkI4n2sQJY5AIJUP6l5F48EAoO5c6NI=; b=TGhBFYmUoRDVll0Po/81IaP5OrzmxhzK8U6+YzhRkfEn0agMSWiO2OX+C8kDw2iX7pa4CNfdDfffipaIV2h+/q8PySEa4NYxfvyH7Ge/jPuHPlBqMJBx5mbl/Q6dTDlSlQcVmiEa/212cixhEUf+Pu/pmVgnbXxdGlELNMvHtfe3VbcLbk65535lrsM+OESb4Geh745sFOq5k0ZVnv6xd3sFY9n5A/BHUV+Tu1O5N8vtg6ebyruUjKyqjMh4hcC6LhkdyiIslTw+N0v+t22GokXQw9OAUvz3ZuITiFTDnfxiCRVU4lzQ6BE3qXPcPGUsfzcaa8/tS/5vCHLCftlzZA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=61te41hgIyp+QkI4n2sQJY5AIJUP6l5F48EAoO5c6NI=; b=mRoCor6tYlvY3wDVZd/jpkuqUwF0nCVh5ETIZDUyJuMwQoAqeVsj0W1B5366AGIPZBBlyGW4T6U/WoLV417NRv8kskqi9zJDNYR5Vi1uUmB5UkDeZtXTLYN/97m9Ja3yY0hqgT657Wzy3YDxBpwum2+okLKyvpLPf+ib8wrs9QC3iWJup8160ldIfrVZFKSK/zCp4Pa9Tj/BpLzN3/RwUA8Cvo4vjW0EyUKDy1xh/8+4fUmMS6UbsFNaQgwQAX9yI+13iPSw8FcO43CQwqeGpkO+bOZocBvfISGJmStGwKxk+jnrU489O5hs6aBjJsfElpoyXSRNTPovPkyVJiw+dQ== Received: from DM6PR05CA0040.namprd05.prod.outlook.com (2603:10b6:5:335::9) by CH3PR12MB8993.namprd12.prod.outlook.com (2603:10b6:610:17b::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9700.11; Wed, 11 Mar 2026 20:36:34 +0000 Received: from DS2PEPF00003448.namprd04.prod.outlook.com (2603:10b6:5:335:cafe::f4) by DM6PR05CA0040.outlook.office365.com (2603:10b6:5:335::9) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9700.14 via Frontend Transport; Wed, 11 Mar 2026 20:36:34 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by DS2PEPF00003448.mail.protection.outlook.com (10.167.17.75) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9678.18 via Frontend Transport; Wed, 11 Mar 2026 20:36:34 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Wed, 11 Mar 2026 13:36:12 -0700 Received: from rnnvmail201.nvidia.com (10.129.68.8) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Wed, 11 Mar 2026 13:36:12 -0700 Received: from nvidia-4028GR-scsim.nvidia.com (10.127.8.11) by mail.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20 via Frontend Transport; Wed, 11 Mar 2026 13:36:05 -0700 From: To: , , , , , , , , , , , , , , , , , CC: , , , , , , , Subject: [PATCH 10/20] vfio/cxl: CXL region management Date: Thu, 12 Mar 2026 02:04:30 +0530 Message-ID: <20260311203440.752648-11-mhonap@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20260311203440.752648-1-mhonap@nvidia.com> References: <20260311203440.752648-1-mhonap@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS2PEPF00003448:EE_|CH3PR12MB8993:EE_ X-MS-Office365-Filtering-Correlation-Id: 64561604-17cd-4fa8-0312-08de7fade2bd X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|82310400026|376014|36860700016|7416014|921020|18002099003|56012099003|22082099003; X-Microsoft-Antispam-Message-Info: rNNaxqcLeqgQeT+CRVT8Vovu5IlV87+fk9bv/XAY9KlSL5as3stsHr3J9svYnDM11JFoKRVT50mMIxUPrDO9/6NG4ZkdowPtMLCg/ppJNRswzxBMlWdsZeYQFWiedihN0M1zJlFN7DWm5TQ/mJLvwN1mQbNU4MkdVh7dK/4WlJc7azysfxqgGEXJOE9a8WagPFWAugZ1f1VwZLyuZOuOvaBntyPb1Sigo44sLCTPsJHM4Zw7e416ci3VDToUVssHXbM4HHdAD59xxN96BQurDJDBjp0a+JFT+WCEt1bcfPWW9yNAimrk/RVarmVA28hnRAwrdqqXMFTu0eOItCOsmHjyEJPksZYLOLij0s6k3+ChavgVFXkJVmpYMz90xKiEs6Xb14YauhKOh+qVbTnidDXipYuKg9DjiVw9E5zZ5KI6LyBgYE55ZX/c2t7mfMimzYNCLjvrKzsez8JWrer5JyWsln71H9JoY/vfuCTMoMCAn6v3I5wmy4zjKQU/FW8da6nDojptO8E56YsPdjSX9sNuGm6nNFH65BUkIgv1INqvqMuWW8njCHvS2OH3VrOat8tRgpfX61zHhWFUO3mN/Q9DTGXfbtOyrS2NnAusww1hVB9WGPFLvmtXQJ9GYpszcm4lOMs5QlPvc162g4qvBB/V6Xso8dGbGpvtNfwqRlaAUi7fgVTy7RQhA/Xbq31y0NY+wGIKkd+5JZMpjzl50NZMm7mB3RK+0Fsh3d9qhO96/ljdH0HADp4JfxSHlmZFpHSHeYFzgOvrnlHpyibJhbdkJ+3klonQmxJTgKszHoHWt7fwuONCtEhkgt2aVmum X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(1800799024)(82310400026)(376014)(36860700016)(7416014)(921020)(18002099003)(56012099003)(22082099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: Ule39oWfQzANv2v05uIOogdT+cAHyPqwb156r75RpG2prjdO1NMxOGyOzOS9ffkQ/vEtVScuuMcZQFeidE4XiWZzBHi/CHXstMBA5yRmvmnKyWfCmey35zxHrjxtH0llPj3HV1D7Gfg4+/rtHVoi8k8qUSfPeusCqrfatWGwKz7GGXZ1x+vAatNt7YlnSfozEqKhepJC+r6WFLJINFO8EtAOCPYneItVvuwxiR469O2BuotYkj7/hsa3KJg7bLwt/4Ay22MDq+WZ7YZvpblcRi7+5F6m7WJT4izHTn2rR7aT1lJnNjsYrxFoiloKRJw3nP2k57N2qYkC4/IW+zy5C3Wbb2xt/GMN7dbE45dLwpzl6QNaxT2venHubNidhb+YXD48+v1GtudkYgN6UIAu/jRD1vdIhG6oQSMO+k1OncA2fas93ujqOTty1rnYsdAy X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Mar 2026 20:36:34.1767 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 64561604-17cd-4fa8-0312-08de7fade2bd X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DS2PEPF00003448.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH3PR12MB8993 Content-Type: text/plain; charset="utf-8" From: Manish Honap Add CXL region management for future guest access. Region Management makes use of APIs provided by CXL_CORE as below: CREATE_REGION flow: 1. Validate request (size, decoder availability) 2. Allocate HPA via cxl_get_hpa_freespace() 3. Allocate DPA via cxl_request_dpa() 4. Create region via cxl_create_region() - commits HDM decoder! 5. Get HPA range via cxl_get_region_range() DESTROY_REGION flow: 1. Detach decoder via cxl_decoder_detach() 2. Free DPA via cxl_dpa_free() 3. Release root decoder via cxl_put_root_decoder() Signed-off-by: Manish Honap --- drivers/vfio/pci/cxl/vfio_cxl_core.c | 118 ++++++++++++++++++++++++++- drivers/vfio/pci/cxl/vfio_cxl_priv.h | 5 ++ drivers/vfio/pci/vfio_pci_priv.h | 8 ++ 3 files changed, 130 insertions(+), 1 deletion(-) diff --git a/drivers/vfio/pci/cxl/vfio_cxl_core.c b/drivers/vfio/pci/cxl/vf= io_cxl_core.c index 2da6da1c0605..9c71f592e74e 100644 --- a/drivers/vfio/pci/cxl/vfio_cxl_core.c +++ b/drivers/vfio/pci/cxl/vfio_cxl_core.c @@ -126,6 +126,112 @@ static int vfio_cxl_setup_regs(struct vfio_pci_core_d= evice *vdev) return 0; } =20 +int vfio_cxl_create_cxl_region(struct vfio_pci_core_device *vdev, resource= _size_t size) +{ + struct vfio_pci_cxl_state *cxl =3D vdev->cxl; + resource_size_t max_size; + int ret; + + if (cxl->precommitted) + return 0; + + cxl->cxlrd =3D cxl_get_hpa_freespace(cxl->cxlmd, 1, + CXL_DECODER_F_RAM | + CXL_DECODER_F_TYPE2, + &max_size); + if (IS_ERR(cxl->cxlrd)) + return PTR_ERR(cxl->cxlrd); + + /* Insufficient HPA space */ + if (max_size < size) { + cxl_put_root_decoder(cxl->cxlrd); + cxl->cxlrd =3D NULL; + return -ENOSPC; + } + + cxl->cxled =3D cxl_request_dpa(cxl->cxlmd, CXL_PARTMODE_RAM, size); + if (IS_ERR(cxl->cxled)) { + ret =3D PTR_ERR(cxl->cxled); + goto err_free_hpa; + } + + cxl->region =3D cxl_create_region(cxl->cxlrd, &cxl->cxled, 1); + if (IS_ERR(cxl->region)) { + ret =3D PTR_ERR(cxl->region); + goto err_free_dpa; + } + + return 0; + +err_free_dpa: + cxl_dpa_free(cxl->cxled); +err_free_hpa: + if (cxl->cxlrd) + cxl_put_root_decoder(cxl->cxlrd); + + return ret; +} + +void vfio_cxl_destroy_cxl_region(struct vfio_pci_core_device *vdev) +{ + struct vfio_pci_cxl_state *cxl =3D vdev->cxl; + + if (!cxl->region) + return; + + cxl_unregister_region(cxl->region); + cxl->region =3D NULL; + + if (cxl->precommitted) + return; + + cxl_dpa_free(cxl->cxled); + cxl_put_root_decoder(cxl->cxlrd); +} + +static int vfio_cxl_create_region_helper(struct vfio_pci_core_device *vdev, + resource_size_t capacity) +{ + struct vfio_pci_cxl_state *cxl =3D vdev->cxl; + struct pci_dev *pdev =3D vdev->pdev; + int ret; + + if (cxl->precommitted) { + cxl->cxled =3D cxl_get_committed_decoder(cxl->cxlmd, + &cxl->region); + if (IS_ERR(cxl->cxled)) + return PTR_ERR(cxl->cxled); + } else { + ret =3D vfio_cxl_create_cxl_region(vdev, capacity); + if (ret) + return ret; + } + + if (cxl->region) { + struct range range; + + ret =3D cxl_get_region_range(cxl->region, &range); + if (ret) + goto failed; + + cxl->region_hpa =3D range.start; + cxl->region_size =3D range_len(&range); + + pci_dbg(pdev, "Precommitted decoder: HPA 0x%llx size %lu MB\n", + cxl->region_hpa, cxl->region_size >> 20); + } else { + pci_err(pdev, "Failed to create CXL region\n"); + ret =3D -ENODEV; + goto failed; + } + + return 0; + +failed: + vfio_cxl_destroy_cxl_region(vdev); + return ret; +} + /** * vfio_pci_cxl_detect_and_init - Detect and initialize CXL Type-2 device * @vdev: VFIO PCI device @@ -172,6 +278,12 @@ void vfio_pci_cxl_detect_and_init(struct vfio_pci_core= _device *vdev) =20 pci_disable_device(pdev); =20 + ret =3D vfio_cxl_create_region_helper(vdev, SZ_256M); + if (ret) + goto failed; + + cxl->precommitted =3D true; + return; =20 failed: @@ -181,6 +293,10 @@ void vfio_pci_cxl_detect_and_init(struct vfio_pci_core= _device *vdev) =20 void vfio_pci_cxl_cleanup(struct vfio_pci_core_device *vdev) { - if (!vdev->cxl) + struct vfio_pci_cxl_state *cxl =3D vdev->cxl; + + if (!cxl || !cxl->region) return; + + vfio_cxl_destroy_cxl_region(vdev); } diff --git a/drivers/vfio/pci/cxl/vfio_cxl_priv.h b/drivers/vfio/pci/cxl/vf= io_cxl_priv.h index 57fed39a80da..985680842a13 100644 --- a/drivers/vfio/pci/cxl/vfio_cxl_priv.h +++ b/drivers/vfio/pci/cxl/vfio_cxl_priv.h @@ -17,6 +17,10 @@ struct vfio_pci_cxl_state { struct cxl_memdev *cxlmd; struct cxl_root_decoder *cxlrd; struct cxl_endpoint_decoder *cxled; + struct cxl_region *region; + resource_size_t region_hpa; + size_t region_size; + void __iomem *region_vaddr; resource_size_t hdm_reg_offset; size_t hdm_reg_size; resource_size_t comp_reg_offset; @@ -24,6 +28,7 @@ struct vfio_pci_cxl_state { u32 hdm_count; u16 dvsec; u8 comp_reg_bar; + bool precommitted; }; =20 /* diff --git a/drivers/vfio/pci/vfio_pci_priv.h b/drivers/vfio/pci/vfio_pci_p= riv.h index d7df5538dcde..818d99f098bf 100644 --- a/drivers/vfio/pci/vfio_pci_priv.h +++ b/drivers/vfio/pci/vfio_pci_priv.h @@ -137,6 +137,9 @@ static inline void vfio_pci_dma_buf_move(struct vfio_pc= i_core_device *vdev, =20 void vfio_pci_cxl_detect_and_init(struct vfio_pci_core_device *vdev); void vfio_pci_cxl_cleanup(struct vfio_pci_core_device *vdev); +int vfio_cxl_create_cxl_region(struct vfio_pci_core_device *vdev, + resource_size_t size); +void vfio_cxl_destroy_cxl_region(struct vfio_pci_core_device *vdev); =20 #else =20 @@ -144,6 +147,11 @@ static inline void vfio_pci_cxl_detect_and_init(struct vfio_pci_core_device *vdev) { } static inline void vfio_pci_cxl_cleanup(struct vfio_pci_core_device *vdev) { } +static inline int vfio_cxl_create_cxl_region(struct vfio_pci_core_device *= vdev, + resource_size_t size) +{ return 0; } +static inline void +vfio_cxl_destroy_cxl_region(struct vfio_pci_core_device *vdev) { } =20 #endif /* CONFIG_VFIO_CXL_CORE */ =20 --=20 2.25.1 From nobody Tue Apr 7 19:54:44 2026 Received: from BYAPR05CU005.outbound.protection.outlook.com (mail-westusazon11010061.outbound.protection.outlook.com [52.101.85.61]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7080C38D681; Wed, 11 Mar 2026 20:36:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.85.61 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773261407; cv=fail; b=jRJMmQ+WiCiBONlviODiea1xDJFbuPqH/iZJTFNUE8q4rLkro/8rEcjZ6VxigbJeeuI9+R+Ttz/GVefp8P3boESzEDLh0hcUM4GaYPD16dHIo9HTFotSurWZeERrTs7iI5nMMncVPTwh49tEOKOFEYP4MFws0opLKLhZx5Cz+RM= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773261407; c=relaxed/simple; bh=ZhB5QafXudlZCeB3PpFEyBe8e11HpLuVTdsbpem5eYo=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=GtOur55POFPk8ImdaitdhLFhGp1y6lTclkP0gRVSqfC/HnftaIGvjrDRbPkvitkkDvZ5AwcFM/5dn74OP/VYJfM6/CNqNqK+hiXIkJVdz1JHlHPgd10NZxdyafvkJH00brKkT4+H+Td4unybOUqCLpyaWaerZBFQwZmP9HVnH6Q= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=JlD8ak9C; arc=fail smtp.client-ip=52.101.85.61 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="JlD8ak9C" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=ZCN60P9oCnzcxPSYDnPLqU1dMiUHWLPg5VDNSIHlTtBmxeEG355R1+4rL/D7AErzra0O+8OuDMqc7BCPPXLQS1WdK2oYTNMUzi42cHaIFI02xaDJ8N1F43FfAiMJ1gWwlO7poBXISNFtyaQ6vWXdH3TNGszpLAcg+51Qlf+p2tmLjL0LfNtqoXNjleMWglKicZNLs59yekxl32ZWyeH6A4oE8xdKAaFoPSD6tOg+3KpaKdI6p6rubNdQB7/kwDeLWRdaFbXhRhwlo6SalEHGJSC0tZi70p0OeRWh7jmkPpGqAN+koBClYZuIU2QmBdtcXfIjrF9+UsiPPuX/Kr7apw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=OY33U5hux11whPm0liBGFXpZ7h571LDlZxYV/wE/KZE=; b=DIM6H0UiU/RGwI1DRM2nKWQ0HRDza5Hs/3FtSI5aFfn/jLpFZNYj1K1O4UZiUhox1QSu6Filb9NEnmzQnC1RW7lYt4vY6Pe18SFt2dZatdwt/TR+n8r5OYWMvmqnR1MB/W4wS+M+dx+yCGMFe6tJ7kN71Kv7HxuLyDqKiH838gzfG/Wv5oCEej9HGnK5ZZJBYPw9oG/PiZcHnIKPFmRUjzX/qAIPKXFfZv63/sjUfX3w5dF5Ro9PuWfi5t4D6AMB/nHjpPRGeShX3blC/69sLfay829vflncUwrDF43AT+VINVIrjVr26sFoYWVjorRIAm304o3EYtdy1YQtLcDDYg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=OY33U5hux11whPm0liBGFXpZ7h571LDlZxYV/wE/KZE=; b=JlD8ak9CqIVsODoJFJ1aOt/4DVrJXr8w/1KgYZWY15Swp/Y/bgXDQ8VjHdGb67i5ErsFMImRBFn5kZOiIDFrq/SweYasGWmyve+xaYhF2fwChn2Z3wZr2kLVhv+gq0T+iQqxl8BcMR2YT8mHi7aVAsQzTcrx4/SQxkwyZwPrCPm7vhDGTiBLKhs5oVv20ALhvhX7nVAJID48CyunlB06TDsSKPIDy/uPO37kHRE0yJAF8O9wrz3TzXcrhR9hsdiSZxfgkQP0YzEr1/pkObMLHgQlUBdBYlYjp6c2HvR+1Kt++dYtvsj/1u2cIP4bxAxnGvAgv203vFOtiK7ELpCmWQ== Received: from SA9PR10CA0023.namprd10.prod.outlook.com (2603:10b6:806:a7::28) by DS0PR12MB8197.namprd12.prod.outlook.com (2603:10b6:8:f1::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9700.11; Wed, 11 Mar 2026 20:36:37 +0000 Received: from SN1PEPF00026368.namprd02.prod.outlook.com (2603:10b6:806:a7:cafe::c7) by SA9PR10CA0023.outlook.office365.com (2603:10b6:806:a7::28) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9678.27 via Frontend Transport; Wed, 11 Mar 2026 20:36:23 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by SN1PEPF00026368.mail.protection.outlook.com (10.167.241.133) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9678.18 via Frontend Transport; Wed, 11 Mar 2026 20:36:37 +0000 Received: from rnnvmail202.nvidia.com (10.129.68.7) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Wed, 11 Mar 2026 13:36:20 -0700 Received: from rnnvmail201.nvidia.com (10.129.68.8) by rnnvmail202.nvidia.com (10.129.68.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Wed, 11 Mar 2026 13:36:19 -0700 Received: from nvidia-4028GR-scsim.nvidia.com (10.127.8.11) by mail.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20 via Frontend Transport; Wed, 11 Mar 2026 13:36:12 -0700 From: To: , , , , , , , , , , , , , , , , , CC: , , , , , , , Subject: [PATCH 11/20] vfio/cxl: Expose DPA memory region to userspace with fault+zap mmap Date: Thu, 12 Mar 2026 02:04:31 +0530 Message-ID: <20260311203440.752648-12-mhonap@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20260311203440.752648-1-mhonap@nvidia.com> References: <20260311203440.752648-1-mhonap@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SN1PEPF00026368:EE_|DS0PR12MB8197:EE_ X-MS-Office365-Filtering-Correlation-Id: 90a4dcb9-2c03-44b8-3da0-08de7fade4b3 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|36860700016|376014|7416014|82310400026|921020|22082099003|18002099003|56012099003; X-Microsoft-Antispam-Message-Info: sCcky8QZyrREkU74jlkWEApcZqqAnba0p/eiuS1zhGahOhuKmMXmL/BKiom38Vtt1HyS/eavkzHF8pBEebAT8grB6nRoHSMAW4OuzR85MG2l8Jfk5n7A+yqrf8/6SxujNfgjmwG7jSMrc0VICiW0Pts/QkAndT+LKygEx0Yzp74Oa4+8T4FiO8e2qcvHOqecXvuXIKeN861jXM8LgV9qcNb71ce8emlpsxqqNYIW2w48NI/inMzfULz8PYvqbJ9bGe+h4cVaoR8yACIg5xALCgwHu9a55m+i6G9BE40/RNjU5kW4AXdZlIXHArzQUM7NqIE2hIr1rq4wTNgsUYK0sYrh+VTJnxVLkZ/tdANmJ9RvO2ld4jW4+wgrHxEW+iHf/pl0++qB0dl35JHF0feCHWBu397lVzYHomhqLMASH4t9Aq1sbp2eMXAVgnRW43erWQ7BID+8YE8MoLUHD1Q6evrGwdjS1DPA+LdiUu0Hr3H/REnYR0HKf26/8RUl7s5UQ785d3Q6nfz1i5IaNhhHSRLjKj10EEHbqnOvFzwUW+GZ6A3IyrtYax1fnY5nTbKGC/StJ6rPe0TVglTQk6DQKViSmXAozpPpcPjLJg8q2toWmk1kgEVasw1JPyZ67AhZuXedwqyzkGnYYHnCgn7iXwtIgB21p6MQO18ummih8Dq9pzAWWd/y1AIG8mQ1MJNdB5HJgnwCvD8LIyGs+pCemZulhXTd+5XAN+fhWx9XxkHJCq4L5QYHFERfr6MJpHFNzY7zb0+Zf3t2Scn45fgyK2VHQC79023UH7RLWX4q6r6+1DMC8k14+MenaEd7MDus X-Forefront-Antispam-Report: CIP:216.228.117.160;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge1.nvidia.com;CAT:NONE;SFS:(13230040)(1800799024)(36860700016)(376014)(7416014)(82310400026)(921020)(22082099003)(18002099003)(56012099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: dR+MAjD+LHa/BrKRwz8c7fB/F1wPPCElSPCDjX5FvI94RL9rYX1ApUaKFjnajXOtQr7vd4EcNdBhts4ke+9CmjMv7HHGrFYhBvTcpTjdTjG19LMMANeFp3/7uO+D+GBtF3PI+ZhYN/GZ4wNqlMvWBcfgBWIpKEiu7xBFu5nWyqSP5KBtuO5bcW1CNIDXhRNBMMNNdhqBRKVTLVV2wPjRwqEyfwmlMjJAx12hau84TLH+SR4Q5hJrZqckqYPgQldrn3LPXDym9DkWL8gqOs+LYQjDoKAZT7nLP8ev/pGI2FT2ja9pitMRNzNIAGOCV65B3CTu6piZ/42oa07UWG97VV1dYOZdQ64TGsj9fp5SyrhsUj7jnYiVbttt47YHGvLvAZMWHH1POfn4XPjD4lYsjMipv/So0c5VOXdoVJq8pfIczNypwebTi8K3Z2t+q+Yo X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Mar 2026 20:36:37.4764 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 90a4dcb9-2c03-44b8-3da0-08de7fade4b3 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: SN1PEPF00026368.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR12MB8197 Content-Type: text/plain; charset="utf-8" From: Manish Honap To directly access the device memory, a CXL region is required. For the userspace (e.g. QEMU) to access the CXL region, the region is required to be exposed via VFIO interfaces. Introduce a new VFIO device region and region ops to expose the created CXL region. Introduce a new sub region type for userspace to identify a CXL region. CXL region lifecycle: - The CXL memory region is registered with VFIO layer during vfio_pci_open_device - mmap() establishes the VMA with vm_ops but inserts no PTEs - Each guest page fault calls vfio_cxl_region_page_fault() which inserts a single PFN under the memory_lock read side - On device reset, vfio_cxl_zap_region_locked() sets region_active=3Dfalse and calls unmap_mapping_range() to invalidate all DPA PTEs atomically while holding memory_lock for writing - Faults racing with reset see region_active=3D=3Dfalse and return VM_FAULT_SIGBUS - vfio_cxl_reactivate_region() restores region_active after successful hardware reset Also integrate the zap/reactivate calls into vfio_pci_ioctl_reset() so that FLR correctly invalidates DPA mappings and restores them on success. Co-developed-by: Zhi Wang Signed-off-by: Zhi Wang Signed-off-by: Manish Honap --- drivers/vfio/pci/cxl/vfio_cxl_core.c | 222 +++++++++++++++++++++++++++ drivers/vfio/pci/cxl/vfio_cxl_priv.h | 2 + drivers/vfio/pci/vfio_pci.c | 9 ++ drivers/vfio/pci/vfio_pci_core.c | 11 ++ drivers/vfio/pci/vfio_pci_priv.h | 13 ++ 5 files changed, 257 insertions(+) diff --git a/drivers/vfio/pci/cxl/vfio_cxl_core.c b/drivers/vfio/pci/cxl/vf= io_cxl_core.c index 9c71f592e74e..03846bd11c8a 100644 --- a/drivers/vfio/pci/cxl/vfio_cxl_core.c +++ b/drivers/vfio/pci/cxl/vfio_cxl_core.c @@ -44,6 +44,7 @@ static int vfio_cxl_create_device_state(struct vfio_pci_c= ore_device *vdev, =20 cxl =3D vdev->cxl; cxl->dvsec =3D dvsec; + cxl->dpa_region_idx =3D -1; =20 pci_read_config_word(pdev, dvsec + CXL_DVSEC_CAPABILITY_OFFSET, &cap_word); @@ -300,3 +301,224 @@ void vfio_pci_cxl_cleanup(struct vfio_pci_core_device= *vdev) =20 vfio_cxl_destroy_cxl_region(vdev); } + +/* + * Fault handler for the DPA region VMA. Called under mm->mmap_lock read + * side by the fault path. We take memory_lock read side here to exclude + * the write-side held by vfio_cxl_zap_region_locked() during reset. + */ +static vm_fault_t vfio_cxl_region_page_fault(struct vm_fault *vmf) +{ + struct vm_area_struct *vma =3D vmf->vma; + struct vfio_pci_core_device *vdev =3D vma->vm_private_data; + struct vfio_pci_cxl_state *cxl =3D vdev->cxl; + unsigned long pfn; + + guard(rwsem_read)(&vdev->memory_lock); + + if (!READ_ONCE(cxl->region_active)) + return VM_FAULT_SIGBUS; + + pfn =3D PHYS_PFN(cxl->region_hpa) + + ((vmf->address - vma->vm_start) >> PAGE_SHIFT); + + /* + * Scrub the page via the kernel ioremap_cache mapping before inserting + * the user PFN. Prevent the stale device data from leaking across VFIO + * device open/close boundaries. + */ + memset_io((u8 __iomem *)cxl->region_vaddr + + ((pfn - PHYS_PFN(cxl->region_hpa)) << PAGE_SHIFT), + 0, PAGE_SIZE); + + return vmf_insert_pfn(vma, vmf->address, pfn); +} + +static const struct vm_operations_struct vfio_cxl_region_vm_ops =3D { + .fault =3D vfio_cxl_region_page_fault, +}; + +static int vfio_cxl_region_mmap(struct vfio_pci_core_device *vdev, + struct vfio_pci_region *region, + struct vm_area_struct *vma) +{ + struct vfio_pci_cxl_state *cxl =3D vdev->cxl; + unsigned long req_len; + + if (!(region->flags & VFIO_REGION_INFO_FLAG_MMAP)) + return -EINVAL; + + if (check_sub_overflow(vma->vm_end, vma->vm_start, &req_len)) + return -EOVERFLOW; + + if (req_len > cxl->region_size) + return -EINVAL; + + /* + * Do not insert PTEs here (no remap_pfn_range). PTEs are inserted + * lazily on first fault via vfio_cxl_region_page_fault(). This + * allows vfio_cxl_zap_region_locked() to safely invalidate them + * during device reset without any userspace cooperation. + * Leave vm_page_prot at its default. + */ + + vm_flags_set(vma, VM_IO | VM_PFNMAP | VM_DONTEXPAND | VM_DONTDUMP); + vma->vm_private_data =3D vdev; + vma->vm_ops =3D &vfio_cxl_region_vm_ops; + + return 0; +} + +/* + * vfio_cxl_zap_region_locked - Invalidate all DPA region PTEs. + * + * Must be called with vdev->memory_lock held for writing. Sets + * region_active=3Dfalse before zapping so any fault racing with zap sees + * the inactive state and returns VM_FAULT_SIGBUS rather than inserting + * a stale PFN. + */ +void vfio_cxl_zap_region_locked(struct vfio_pci_core_device *vdev) +{ + struct vfio_device *core_vdev =3D &vdev->vdev; + struct vfio_pci_cxl_state *cxl =3D vdev->cxl; + + lockdep_assert_held_write(&vdev->memory_lock); + + if (!cxl || cxl->dpa_region_idx < 0) + return; + + WRITE_ONCE(cxl->region_active, false); + unmap_mapping_range(core_vdev->inode->i_mapping, + VFIO_PCI_INDEX_TO_OFFSET(VFIO_PCI_NUM_REGIONS + + cxl->dpa_region_idx), + cxl->region_size, true); +} + +/* + * vfio_cxl_reactivate_region - Re-enable DPA region after successful rese= t. + * + * Must be called with vdev->memory_lock held for writing. Re-reads the + * HDM decoder state from hardware (FLR cleared it) and sets region_active + * so that subsequent faults can re-insert PFNs without a new mmap. + */ +void vfio_cxl_reactivate_region(struct vfio_pci_core_device *vdev) +{ + struct vfio_pci_cxl_state *cxl =3D vdev->cxl; + + lockdep_assert_held_write(&vdev->memory_lock); + + if (!cxl) + return; +} + +static ssize_t vfio_cxl_region_rw(struct vfio_pci_core_device *core_dev, + char __user *buf, size_t count, loff_t *ppos, + bool iswrite) +{ + unsigned int i =3D VFIO_PCI_OFFSET_TO_INDEX(*ppos) - VFIO_PCI_NUM_REGIONS; + struct vfio_pci_cxl_state *cxl =3D core_dev->region[i].data; + loff_t pos =3D *ppos & VFIO_PCI_OFFSET_MASK; + + guard(rwsem_read)(&core_dev->memory_lock); + + if (!READ_ONCE(cxl->region_active)) + return -EIO; + + if (!count) + return 0; + + return vfio_pci_core_do_io_rw(core_dev, false, + cxl->region_vaddr, + (char __user *)buf, pos, count, + 0, 0, iswrite, VFIO_PCI_IO_WIDTH_8); +} + +static void vfio_cxl_region_release(struct vfio_pci_core_device *vdev, + struct vfio_pci_region *region) +{ + struct vfio_pci_cxl_state *cxl =3D region->data; + + if (cxl->region_vaddr) { + iounmap(cxl->region_vaddr); + cxl->region_vaddr =3D NULL; + } +} + +static const struct vfio_pci_regops vfio_cxl_regops =3D { + .rw =3D vfio_cxl_region_rw, + .mmap =3D vfio_cxl_region_mmap, + .release =3D vfio_cxl_region_release, +}; + +int vfio_cxl_register_cxl_region(struct vfio_pci_core_device *vdev) +{ + struct vfio_pci_cxl_state *cxl =3D vdev->cxl; + u32 flags; + int ret; + + if (!cxl) + return -ENODEV; + + if (!cxl->region || cxl->region_vaddr) + return -ENODEV; + + cxl->region_vaddr =3D ioremap_cache(cxl->region_hpa, cxl->region_size); + if (!cxl->region_vaddr) + return -ENOMEM; + + flags =3D VFIO_REGION_INFO_FLAG_READ | + VFIO_REGION_INFO_FLAG_WRITE | + VFIO_REGION_INFO_FLAG_MMAP; + + ret =3D vfio_pci_core_register_dev_region(vdev, + PCI_VENDOR_ID_CXL | + VFIO_REGION_TYPE_PCI_VENDOR_TYPE, + VFIO_REGION_SUBTYPE_CXL, + &vfio_cxl_regops, + cxl->region_size, flags, + cxl); + if (ret) { + iounmap(cxl->region_vaddr); + cxl->region_vaddr =3D NULL; + return ret; + } + + /* + * Cache the vdev->region[] index before activating the region. + * vfio_pci_core_register_dev_region() placed the new entry at + * vdev->region[num_regions - 1] and incremented num_regions. + * vfio_cxl_zap_region_locked() uses this to avoid scanning + * vdev->region[] on every FLR. + */ + cxl->dpa_region_idx =3D vdev->num_regions - 1; + WRITE_ONCE(cxl->region_active, true); + + return 0; +} +EXPORT_SYMBOL_GPL(vfio_cxl_register_cxl_region); + +/** + * vfio_cxl_unregister_cxl_region - Undo vfio_cxl_register_cxl_region() + * @vdev: VFIO PCI device + * + * Marks the DPA region inactive so any racing fault returns VM_FAULT_SIGB= US + * and resets dpa_region_idx. Does NOT call release() or touch num_region= s; + * vfio_pci_core_disable() will call the idempotent release() callback as + * normal during device close. + * + * Does NOT touch CXL subsystem state (cxl->region, cxl->cxled, cxl->cxlrd= ). + * The caller must call vfio_cxl_destroy_cxl_region() separately to release + * those objects. + */ +void vfio_cxl_unregister_cxl_region(struct vfio_pci_core_device *vdev) +{ + struct vfio_pci_cxl_state *cxl =3D vdev->cxl; + + if (!cxl || cxl->dpa_region_idx < 0) + return; + + WRITE_ONCE(cxl->region_active, false); + + cxl->dpa_region_idx =3D -1; +} +EXPORT_SYMBOL_GPL(vfio_cxl_unregister_cxl_region); diff --git a/drivers/vfio/pci/cxl/vfio_cxl_priv.h b/drivers/vfio/pci/cxl/vf= io_cxl_priv.h index 985680842a13..b870926bfb19 100644 --- a/drivers/vfio/pci/cxl/vfio_cxl_priv.h +++ b/drivers/vfio/pci/cxl/vfio_cxl_priv.h @@ -26,9 +26,11 @@ struct vfio_pci_cxl_state { resource_size_t comp_reg_offset; size_t comp_reg_size; u32 hdm_count; + int dpa_region_idx; u16 dvsec; u8 comp_reg_bar; bool precommitted; + bool region_active; }; =20 /* diff --git a/drivers/vfio/pci/vfio_pci.c b/drivers/vfio/pci/vfio_pci.c index 0c771064c0b8..d3138badeaa6 100644 --- a/drivers/vfio/pci/vfio_pci.c +++ b/drivers/vfio/pci/vfio_pci.c @@ -120,6 +120,15 @@ static int vfio_pci_open_device(struct vfio_device *co= re_vdev) } } =20 + if (vdev->cxl) { + ret =3D vfio_cxl_register_cxl_region(vdev); + if (ret) { + pci_warn(pdev, "Failed to setup CXL region\n"); + vfio_pci_core_disable(vdev); + return ret; + } + } + vfio_pci_core_finish_enable(vdev); =20 return 0; diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_c= ore.c index b7364178e23d..48e0274c19aa 100644 --- a/drivers/vfio/pci/vfio_pci_core.c +++ b/drivers/vfio/pci/vfio_pci_core.c @@ -1223,6 +1223,9 @@ static int vfio_pci_ioctl_reset(struct vfio_pci_core_= device *vdev, =20 vfio_pci_zap_and_down_write_memory_lock(vdev); =20 + /* Zap CXL DPA region PTEs before hardware reset clears HDM state */ + vfio_cxl_zap_region_locked(vdev); + /* * This function can be invoked while the power state is non-D0. If * pci_try_reset_function() has been called while the power state is @@ -1236,6 +1239,14 @@ static int vfio_pci_ioctl_reset(struct vfio_pci_core= _device *vdev, =20 vfio_pci_dma_buf_move(vdev, true); ret =3D pci_try_reset_function(vdev->pdev); + + /* + * Re-enable DPA region if reset succeeded; fault handler will + * re-insert PFNs on next access without requiring a new mmap. + */ + if (!ret) + vfio_cxl_reactivate_region(vdev); + if (__vfio_pci_memory_enabled(vdev)) vfio_pci_dma_buf_move(vdev, false); up_write(&vdev->memory_lock); diff --git a/drivers/vfio/pci/vfio_pci_priv.h b/drivers/vfio/pci/vfio_pci_p= riv.h index 818d99f098bf..441b4a47637a 100644 --- a/drivers/vfio/pci/vfio_pci_priv.h +++ b/drivers/vfio/pci/vfio_pci_priv.h @@ -140,6 +140,10 @@ void vfio_pci_cxl_cleanup(struct vfio_pci_core_device = *vdev); int vfio_cxl_create_cxl_region(struct vfio_pci_core_device *vdev, resource_size_t size); void vfio_cxl_destroy_cxl_region(struct vfio_pci_core_device *vdev); +int vfio_cxl_register_cxl_region(struct vfio_pci_core_device *vdev); +void vfio_cxl_unregister_cxl_region(struct vfio_pci_core_device *vdev); +void vfio_cxl_zap_region_locked(struct vfio_pci_core_device *vdev); +void vfio_cxl_reactivate_region(struct vfio_pci_core_device *vdev); =20 #else =20 @@ -152,6 +156,15 @@ static inline int vfio_cxl_create_cxl_region(struct vf= io_pci_core_device *vdev, { return 0; } static inline void vfio_cxl_destroy_cxl_region(struct vfio_pci_core_device *vdev) { } +static inline int +vfio_cxl_register_cxl_region(struct vfio_pci_core_device *vdev) +{ return 0; } +static inline void +vfio_cxl_unregister_cxl_region(struct vfio_pci_core_device *vdev) { } +static inline void +vfio_cxl_zap_region_locked(struct vfio_pci_core_device *vdev) { } +static inline void +vfio_cxl_reactivate_region(struct vfio_pci_core_device *vdev) { } =20 #endif /* CONFIG_VFIO_CXL_CORE */ =20 --=20 2.25.1 From nobody Tue Apr 7 19:54:44 2026 Received: from DM5PR21CU001.outbound.protection.outlook.com (mail-centralusazon11011003.outbound.protection.outlook.com [52.101.62.3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4A40C37755B; Wed, 11 Mar 2026 20:36:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.62.3 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773261408; cv=fail; b=Bbo+/sVEjewHDCa9IVA8ne/fj6Q4StIlHnAZmEiQvoKYftm07jmw4kpK8eTaLn3jKgRQO6JsJ4rQti62oV0lcBizjqnKnmMS/j8QgfIdfqWFcnhIXLRllgREkxHbdnEB5sA4X6slGJvuSBf9YUebzBOlE6xK28gsPbAlZn8lAik= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773261408; c=relaxed/simple; bh=bAZ262Ai/3xi8gXmym2r6a/kWOtOBfp0wTylWE08yxQ=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=lcousrIsBe/NvlpRzmeG5qS6nGzYpTldVieB7P6aOz1hjQy5USZ+nPugBrAfojpisILVWdC+2tcLZPy8TmGFn5gVobNKphnvGoVqmskRBltFfpZ1lwE928Jk+sA9pcaOZbWVXnjqgwDiXMpt7xplenO6A9c0nox88J9DMpFIITs= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=mUm5oWPJ; arc=fail smtp.client-ip=52.101.62.3 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="mUm5oWPJ" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=pYcDWpVrH2nf3rqbS/THq/xf9vKVYOb7Ik5pse4/cuT5SAKLlTkj1EWdfgZg45SudztMztsjc8UUwTC+wpsrLfVzoikVebuYhL1nMQINUciWIrcLnYSk/tkOxkQsQaniAgkmCaCIgiYdQqa3ZROZ1i2C4d3yhvHSUT4bcO1hysoAsCTqtca/CY21FA/0TiRjNaUdEokvr/SHqaLcEbPpN2sYsnZnCNoIjt6l0NS9vsWAiQtU1par6zio5RHUEJrkY4ue8sfmCYuwR+vlck50A8hCCSpal8sf/HwMxWeb3OriJOqjHIswdVShCPKmOw7u4WCRm2zAKg4ddZ5A8TPVPA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=KUzrkFU2XIVR1SnEtkDqb1nFZsMXxteX6hBdRzywfFU=; b=Vl+hWpzpOtcSgGtre4XdWrdRkNXHeZe5/nDvCLjdJUjosDJ4mhg9vmcMSYL3qfazrcYHVjPPsH6KRdGq3qmEuPCoBdrxpzRok3DiBciA+UND2VArpSmUyuSSP8xg2FgFZmYF/KhfeiVIJ9XTg5G73koPrnNIIQFi9AuiQfMDpkir2Xmd6eYKuKDAxvrgaDPvaX7X5cgnhriG7hpcJMCyBQQvkNUFuh+FodVIxicBZU9D+2TQi5kMZ1sb4rmVlwOZ0Y3gu6tTlDVOwOVX2kQ9E0BW9ZJ7Hk3DAldANymtLGIYu7I8FIPZNRmia4zUtI9xJRyX34I26Y/1JtTIH6Mz4w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=KUzrkFU2XIVR1SnEtkDqb1nFZsMXxteX6hBdRzywfFU=; b=mUm5oWPJ1B3ZUYxJMmvFij1493SKhIpEmqzNm9qaMJTGg1A1JPvfrfbmfH08BQWS1GhI1UANulTixjV9BduDrY363RfdJwMBDzc3ClO4DPxtAo9T5kEx9TWW3XL3AGS6hyfhA8IJ60gUnNl+fEia4OA8kKZMXhO6sHV2uiJ4vdOLGdWR1bKzVGN+BZZLnM0NlW+6O3vEpXdlL1Pqj1JL9H/32iwN4TF/7AGdjR2tuRyE2LpiNHTjfXltUvAm/iXu+fbGWPnJkXDUq4gtiZ03wNBlUUgZx5LVBnV9UCUZhFXFplzE8Z5XkbfXG4eg9Loq9m9I60Felr+hBnjiSAPdgQ== Received: from SA9PR10CA0005.namprd10.prod.outlook.com (2603:10b6:806:a7::10) by IA0PR12MB8253.namprd12.prod.outlook.com (2603:10b6:208:402::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9723.4; Wed, 11 Mar 2026 20:36:42 +0000 Received: from SN1PEPF00026368.namprd02.prod.outlook.com (2603:10b6:806:a7:cafe::af) by SA9PR10CA0005.outlook.office365.com (2603:10b6:806:a7::10) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9678.25 via Frontend Transport; Wed, 11 Mar 2026 20:36:38 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by SN1PEPF00026368.mail.protection.outlook.com (10.167.241.133) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9678.18 via Frontend Transport; Wed, 11 Mar 2026 20:36:42 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Wed, 11 Mar 2026 13:36:28 -0700 Received: from rnnvmail201.nvidia.com (10.129.68.8) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Wed, 11 Mar 2026 13:36:27 -0700 Received: from nvidia-4028GR-scsim.nvidia.com (10.127.8.11) by mail.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20 via Frontend Transport; Wed, 11 Mar 2026 13:36:20 -0700 From: To: , , , , , , , , , , , , , , , , , CC: , , , , , , , Subject: [PATCH 12/20] vfio/pci: Export config access helpers Date: Thu, 12 Mar 2026 02:04:32 +0530 Message-ID: <20260311203440.752648-13-mhonap@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20260311203440.752648-1-mhonap@nvidia.com> References: <20260311203440.752648-1-mhonap@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SN1PEPF00026368:EE_|IA0PR12MB8253:EE_ X-MS-Office365-Filtering-Correlation-Id: fc134fb5-e3b1-4236-c37e-08de7fade7a9 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|36860700016|376014|7416014|82310400026|921020|18002099003|22082099003|56012099003; X-Microsoft-Antispam-Message-Info: LU94g1nloX2L2G076ZAyq2IfB1SHLyCI8CLeaKeDX0Oci6p1S8DTzmJAkrdhvWcMkydzl8nVbBEthSzpD+AYeo63kRnCFsT/sbJyA4zXUgrSx7HlxdCtwlkD7mwyJTEAmmWo6Egr90PULlurhUrW70s+EhNUxiUM3Hji+o9ktqhgV6AQD716pI7AEkXofPGqEQZH2F3GhfoF1J8xouwd0NogR7oOO+CCps47RICDC0C3hv9kST73S062LY3MnpQ1ylUXT0TzbPYKlKntlMwL0EWNjaYGTsE2uyVCrvtCLJgNQw+QOAX4l/UJG2z/K/cllKK2Guw5TkYJvWYxsmLCKhpYhEirXyNQQct4PFUAxsTajHKFoW8XGXe14aOOwoim8ZKDLBMlKXf+2lRdHae2imVQhv/X3EsC1SPgDKFClDfVRTd4YUFPWDVLKcltl7II92sm34ZmggzwlF+1+6wxu9+ueEfjMeATKAaKZ9End2ZftGCbR4zFFBLmJoHfsPEPOhsQaPn2dSscmfLADwV6i1nbnd+/ZHyB/2DEdaCwJPSQSCRmpMY1Lp8vaWzA1V55xMm2lggsi8FHQHTNbZczz4CnDRg/kNS1aWcT59XFhSmrBWrlOIXk0cVAfwKwB5URNQY0HX90CgXBJRFzE2t7nWO6m4r+eA/kaGw9VdpzIU78u7Rj3E5DwKwyc+d1WFZmD4ZdsL+fpp3MnOFbofzOSnXDZ67Bf2ErgjJ3VlwPJ6m0s3ebgGRpAp4SeQQb8DiO4yvdMOMhwlpPnLx4Ht7dZC9bsjhZuUg8lVvnvp+FYQ8apL5wHPAsHTSI63SLxNwi X-Forefront-Antispam-Report: CIP:216.228.117.160;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge1.nvidia.com;CAT:NONE;SFS:(13230040)(1800799024)(36860700016)(376014)(7416014)(82310400026)(921020)(18002099003)(22082099003)(56012099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: y5+LSycvh1iHKsS2QugseQB5E7hNuwtBorN1bH/iC6ApUYcukICuR7IabWHTLS7Dhm2+s0Llrg+dPT1aDv9YfFHUhZuwGJhf31svsreAybbe9Da7N/bqntwqQb4Q0xRdPq6+QSeYs4UvndHSnBv1kEaUrXqP7VkPWzJ+bGzkmajg2VUKRkzhzkiTr+XLFvV5mTAS5CL9jBoxx7CuFphZptp8nxedU0gNZZ3ZIvy9CzEfj64f+C8SkdDGGfoC0+7aHpSrPsKP20lkhW1vU+kZmfPfDlQv8R36PqIteYvbxd+rXViRVZ4iqU3g38faRDqLGFuONncfSCiF6pAvBmWr7z4nwrWBn5xyucmmJyFtar3XdSgKFLQF8bHVbOtQ6hlUqeIVkdml4baxzLnKcsYBCSar4f9k+2l59bFi6dpxErN5xwiCQrHz2yd5bMJloSVm X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Mar 2026 20:36:42.4447 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: fc134fb5-e3b1-4236-c37e-08de7fade7a9 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: SN1PEPF00026368.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA0PR12MB8253 Content-Type: text/plain; charset="utf-8" From: Manish Honap Promote vfio_raw_config_write() and vfio_raw_config_read() to non-static so that the CXL DVSEC write handler in the next patch can call them. Signed-off-by: Manish Honap --- drivers/vfio/pci/vfio_pci_config.c | 12 ++++++------ drivers/vfio/pci/vfio_pci_priv.h | 8 ++++++++ 2 files changed, 14 insertions(+), 6 deletions(-) diff --git a/drivers/vfio/pci/vfio_pci_config.c b/drivers/vfio/pci/vfio_pci= _config.c index dc4e510e6e1b..79aaf270adb2 100644 --- a/drivers/vfio/pci/vfio_pci_config.c +++ b/drivers/vfio/pci/vfio_pci_config.c @@ -270,9 +270,9 @@ static int vfio_direct_config_read(struct vfio_pci_core= _device *vdev, int pos, } =20 /* Raw access skips any kind of virtualization */ -static int vfio_raw_config_write(struct vfio_pci_core_device *vdev, int po= s, - int count, struct perm_bits *perm, - int offset, __le32 val) +int vfio_raw_config_write(struct vfio_pci_core_device *vdev, int pos, + int count, struct perm_bits *perm, + int offset, __le32 val) { int ret; =20 @@ -283,9 +283,9 @@ static int vfio_raw_config_write(struct vfio_pci_core_d= evice *vdev, int pos, return count; } =20 -static int vfio_raw_config_read(struct vfio_pci_core_device *vdev, int pos, - int count, struct perm_bits *perm, - int offset, __le32 *val) +int vfio_raw_config_read(struct vfio_pci_core_device *vdev, int pos, + int count, struct perm_bits *perm, + int offset, __le32 *val) { int ret; =20 diff --git a/drivers/vfio/pci/vfio_pci_priv.h b/drivers/vfio/pci/vfio_pci_p= riv.h index 441b4a47637a..8f440f9eaa0c 100644 --- a/drivers/vfio/pci/vfio_pci_priv.h +++ b/drivers/vfio/pci/vfio_pci_priv.h @@ -37,6 +37,14 @@ int vfio_pci_set_irqs_ioctl(struct vfio_pci_core_device = *vdev, uint32_t flags, ssize_t vfio_pci_config_rw(struct vfio_pci_core_device *vdev, char __user = *buf, size_t count, loff_t *ppos, bool iswrite); =20 +int vfio_raw_config_write(struct vfio_pci_core_device *vdev, int pos, + int count, struct perm_bits *perm, + int offset, __le32 val); + +int vfio_raw_config_read(struct vfio_pci_core_device *vdev, int pos, + int count, struct perm_bits *perm, + int offset, __le32 *val); + ssize_t vfio_pci_bar_rw(struct vfio_pci_core_device *vdev, char __user *bu= f, size_t count, loff_t *ppos, bool iswrite); =20 --=20 2.25.1 From nobody Tue Apr 7 19:54:44 2026 Received: from CY3PR05CU001.outbound.protection.outlook.com (mail-westcentralusazon11013021.outbound.protection.outlook.com [40.93.201.21]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 64464358375; Wed, 11 Mar 2026 20:37:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.93.201.21 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773261430; cv=fail; b=vC7PexnryOpdQS4ZNWzK+J452Hj1t7AQLHs9YiJMuMHEw17YwA9Rrr24ofEVndiqbP0w1aY/mRH3YhpO95u8qXtwu4YtgAwuIZPEeRUN8VARCYooEvQQEArNDdRI/GNqN2bqyT+C83417BHyaz9DB4Uu2WHlJjEwu7I1JjBJAqk= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773261430; c=relaxed/simple; bh=d6awq4FrYp29jE0aBOg4ejwXLt+ZOfEZww5CnFbaJiw=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=imBHVwwA8EyGD5/YqhO7TExpuktHFh8kVJXhDNCuJdZHBgjYRe5dKGDFM9bViIOyjB6RUt41A3D0LxWgbdU3m1lm6/GTZGir9kaREVdgfMVqKksasLKe/o8FveeSzJLCoZ0MIvTZ3eVvplOliEHyDAXyPEsY2Ron/jmcOpqyVgw= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=NbbQvm42; arc=fail smtp.client-ip=40.93.201.21 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="NbbQvm42" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=gz2BYg2mx3Po8/9JDsB8IcZJULSY4hIuKuiwRR502UU1hEiv9eEOpLmMq/3Uo0h2UAVVUmdk/ucxKwZDqkOYqA0tqLRXb1vxZkIwAg3LtICNEVo6u+3APnni192LKaQbIEswZf9IQblBVapYR6hmQOuSJzegKDOG//A8DAYGJOf4bIUZTpEJdllkQ1IS9oSc1m6L7kVIenlkevhp5gt9K4RbEqzi/JUGbhH3yL8uIxWv5f7SG3bA0WvwEKIfyvF2EoTbdz0QSGV0i+q8Sb2QrTcZ06q6QGPMqWZkBZIeejmcsiNbA2y2szMZ9hTzQXnZGpZ+6d1q61qvgLdwQr3Kdg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=akcAMhoMZxxw2lGC9whZFZUZwOUaIteB0kR2pbyFqII=; b=H97SiPc9bWjxbUtnrb8/Q//RYqonCfhrtBydCqIWDKgqHxYYBe6Q8pQfXl/o2PO5xnF4yrRHT8pbpf/DMFwRIvFKx0GAGJhuSRV3NriBHM8aY7n18ENIblOThdPJ4WXweizEhgk0hkFCHQOWA4aRFnaPhZ6q4vtDt1cUeSiHwkk8EJHJsSRuJRtRHNoHBew/xcBy45XT4lUvPaySOog8G0ut+9iz6xSCPeyfbPn93ofFD9dZWYoYgHyCnqw1/2SZuswcbKQsgTruKEpTSjeGhjdoyyq/idDlAK+fq7h+A8ID2zSC4aURX1vV5y/wlNd6PsHa00WWKGiDKJ8o+Hye1Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=akcAMhoMZxxw2lGC9whZFZUZwOUaIteB0kR2pbyFqII=; b=NbbQvm42q82Rwet4bfs+bpDFfeMdxXyLq2dgBOjgrzMKpeetW6FDJvldEa7Cm0R6IGWBEavJf51tpbufBh+dWcZt/q8zEA1wAR7CEV2l99JXasLmMEgBYZnTCJfS8CSIubGoUaU1lCuQik95tyVO2A7vukIhGpEdhW3EUx8ECnVgLVo8ofsAN/KbqY5QkZcoldmiju1eQG+LR0H7b/Edrz2axVmLcXbcXXIx43MPokb/YUGZDt/iKErjsE3ftYdfXO2kvsL8TqvWefmncsR1kD5kW0/HyAq/kqU+R2xaJhRUshEIbSLT4brrGGXQPCx3FPL8Ic3BngcaUEnAS63Nyg== Received: from CH0PR03CA0069.namprd03.prod.outlook.com (2603:10b6:610:cc::14) by DM4PR12MB5747.namprd12.prod.outlook.com (2603:10b6:8:5e::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9723.4; Wed, 11 Mar 2026 20:36:54 +0000 Received: from DS2PEPF00003446.namprd04.prod.outlook.com (2603:10b6:610:cc:cafe::32) by CH0PR03CA0069.outlook.office365.com (2603:10b6:610:cc::14) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9678.26 via Frontend Transport; Wed, 11 Mar 2026 20:36:47 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by DS2PEPF00003446.mail.protection.outlook.com (10.167.17.73) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9678.18 via Frontend Transport; Wed, 11 Mar 2026 20:36:54 +0000 Received: from rnnvmail202.nvidia.com (10.129.68.7) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Wed, 11 Mar 2026 13:36:35 -0700 Received: from rnnvmail201.nvidia.com (10.129.68.8) by rnnvmail202.nvidia.com (10.129.68.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Wed, 11 Mar 2026 13:36:35 -0700 Received: from nvidia-4028GR-scsim.nvidia.com (10.127.8.11) by mail.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20 via Frontend Transport; Wed, 11 Mar 2026 13:36:28 -0700 From: To: , , , , , , , , , , , , , , , , , CC: , , , , , , , Subject: [PATCH 13/20] vfio/cxl: Introduce HDM decoder register emulation framework Date: Thu, 12 Mar 2026 02:04:33 +0530 Message-ID: <20260311203440.752648-14-mhonap@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20260311203440.752648-1-mhonap@nvidia.com> References: <20260311203440.752648-1-mhonap@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS2PEPF00003446:EE_|DM4PR12MB5747:EE_ X-MS-Office365-Filtering-Correlation-Id: 2681cbf5-b8fc-4782-565a-08de7fadeecc X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|82310400026|36860700016|376014|7416014|921020|22082099003|18002099003|56012099003; X-Microsoft-Antispam-Message-Info: KIQEIEixh2x4Gp0yKyfxnB0l0+2QTQVMF9lKJ2fzQu0AlUVx8iIKLYhykzweLqO/PkSsilN3WpJ4lXuLYj3Nri9xT1WzXwbYdcCb7o4UVZq6/9CmEt0i2BfLfaABQD56WhhQM34xOEBsSzDgYtA6UEYIF7j9mLokc9b+qzdCemzgvoQ0GkZyzA+lOpIDzYM7BVAnigkTKfVTXRflEzJ1LOh8AQWQ0TKGZajvIApSjtlXbVwvWMsW1U6rDnQxT45CyzutPeXoWcF/CGZ0raVYz8GG8OpSboC9+XIeqH2lLF6myNVrkkxbVOd/FpW5NXY3cGJrZdXeuODNyNC4xjPkiPs2/pWykFtmCHFF81rEt1GDAQqJcc2Yn4LnlLpzcOSNuS16T+lQaetgqihEkhKEMPAVXubajte+RuJ6lTuOvRnvBpFTBbojshFS8e1i0nDTVVUJroCZlrhq309NEOcpb8eY6UYNKrsNksJ9N6ICpfo6OHkebeSvXVuxQeCWWGiw/M6gmkGIRDUa+2vD/e7qadMXo0MQjjgim7n0FFDbtLyEP3R8mLCkoIAo69LGgW67QDbyf7dYam9gKBYX2Zp0ZRwKJKPRqx+9j8Y2cMBDjsb4bG83fVb8QM8fvyJucQ44XK30IdWcecImNTMJluPy/fbH0vR0zj0N2jS30DPtwXPG5x9gR7ebIO/CYR/z7xc6SbOYZIgZfqFkFdDxk1r64J+3hwoqGu4WnjSuuEpqUWDEIUy+ua4MtFo2Lwt4y3mzJ0DRA64h421VKdctatRsogX9VAb+vJoj3lEYK+TD7wbonOXxp8pgDJP5GX+Raq1L X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(1800799024)(82310400026)(36860700016)(376014)(7416014)(921020)(22082099003)(18002099003)(56012099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 8IDjYs3rtYomQ8Z+D5ELZYPHBbeSXKEIZG5BUXcW096QhI0n4nMVLXGM34iRuSwtE8YngN7xvH6H0IZmc0uq2bu8VwEIHR5mRPK79V1D103kEmYRZWHXE+KMU7D4EwIsFSe5qssw5NFTHad0l9YD6Rll8R1W2C6CYU3R8PHjjI+Pgq1CdBiJ0H8SUrobKJnyMMJSUYnxpkJHoIJcM/xabg5xv8TV5avM0OVmSdhiuJHc7/4uX+dF4V4SwzMXK3+kYj+b0lqnwTE1ZHsT7xWMdaqMdr4tucw2fBJ4PzrRAK0SEvqi6F2wnXJmCGaJ1DxQ8LM8Nzf5V8k8R1QNued9nOeJPnR8paSxyJBn2ws69TvwpXxNP/O8xAZeYF+6eeK72zupXTqUaAnaRId6271O8cnrU7Cr/X5lGlbERsVVvkktsWEgzhH+9VBhCTyrin1I X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Mar 2026 20:36:54.4149 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 2681cbf5-b8fc-4782-565a-08de7fadeecc X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DS2PEPF00003446.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB5747 From: Manish Honap Introduce an emulation framework to handle CXL MMIO register emulation for CXL devices passed through to a VM. A single compact __le32 array (comp_reg_virt) covers only the HDM decoder register block (hdm_reg_size bytes, typically 256-512 bytes). A new VFIO device region VFIO_REGION_SUBTYPE_CXL_COMP_REGS exposes this array to userspace (QEMU) as a read-write region: - Reads return the emulated state (comp_reg_virt[]) - Writes go through the HDM register write handlers and are forwarded to hardware where appropriate QEMU attaches a notify_change callback to this region. When the COMMIT bit is written in a decoder CTRL register the callback reads the BASE_LO/HI from the same region fd (emulated state) and maps the DPA MemoryRegion at the correct GPA in system_memory. Co-developed-by: Zhi Wang Signed-off-by: Zhi Wang Signed-off-by: Manish Honap --- drivers/vfio/pci/Makefile | 2 +- drivers/vfio/pci/cxl/vfio_cxl_core.c | 36 ++- drivers/vfio/pci/cxl/vfio_cxl_emu.c | 366 +++++++++++++++++++++++++++ drivers/vfio/pci/cxl/vfio_cxl_priv.h | 41 +++ drivers/vfio/pci/vfio_pci_priv.h | 7 + 5 files changed, 450 insertions(+), 2 deletions(-) create mode 100644 drivers/vfio/pci/cxl/vfio_cxl_emu.c diff --git a/drivers/vfio/pci/Makefile b/drivers/vfio/pci/Makefile index ecb0eacbc089..bef916495eae 100644 --- a/drivers/vfio/pci/Makefile +++ b/drivers/vfio/pci/Makefile @@ -1,7 +1,7 @@ # SPDX-License-Identifier: GPL-2.0-only =20 vfio-pci-core-y :=3D vfio_pci_core.o vfio_pci_intrs.o vfio_pci_rdwr.o vfio= _pci_config.o -vfio-pci-core-$(CONFIG_VFIO_CXL_CORE) +=3D cxl/vfio_cxl_core.o +vfio-pci-core-$(CONFIG_VFIO_CXL_CORE) +=3D cxl/vfio_cxl_core.o cxl/vfio_cx= l_emu.o vfio-pci-core-$(CONFIG_VFIO_PCI_ZDEV_KVM) +=3D vfio_pci_zdev.o vfio-pci-core-$(CONFIG_VFIO_PCI_DMABUF) +=3D vfio_pci_dmabuf.o obj-$(CONFIG_VFIO_PCI_CORE) +=3D vfio-pci-core.o diff --git a/drivers/vfio/pci/cxl/vfio_cxl_core.c b/drivers/vfio/pci/cxl/vf= io_cxl_core.c index 03846bd11c8a..d2401871489d 100644 --- a/drivers/vfio/pci/cxl/vfio_cxl_core.c +++ b/drivers/vfio/pci/cxl/vfio_cxl_core.c @@ -45,6 +45,7 @@ static int vfio_cxl_create_device_state(struct vfio_pci_c= ore_device *vdev, cxl =3D vdev->cxl; cxl->dvsec =3D dvsec; cxl->dpa_region_idx =3D -1; + cxl->comp_reg_region_idx =3D -1; =20 pci_read_config_word(pdev, dvsec + CXL_DVSEC_CAPABILITY_OFFSET, &cap_word); @@ -124,6 +125,10 @@ static int vfio_cxl_setup_regs(struct vfio_pci_core_de= vice *vdev) cxl->comp_reg_offset =3D bar_offset; cxl->comp_reg_size =3D CXL_COMPONENT_REG_BLOCK_SIZE; =20 + ret =3D vfio_cxl_setup_virt_regs(vdev); + if (ret) + return ret; + return 0; } =20 @@ -281,12 +286,14 @@ void vfio_pci_cxl_detect_and_init(struct vfio_pci_cor= e_device *vdev) =20 ret =3D vfio_cxl_create_region_helper(vdev, SZ_256M); if (ret) - goto failed; + goto regs_failed; =20 cxl->precommitted =3D true; =20 return; =20 +regs_failed: + vfio_cxl_clean_virt_regs(vdev); failed: devm_kfree(&pdev->dev, vdev->cxl); vdev->cxl =3D NULL; @@ -299,6 +306,7 @@ void vfio_pci_cxl_cleanup(struct vfio_pci_core_device *= vdev) if (!cxl || !cxl->region) return; =20 + vfio_cxl_clean_virt_regs(vdev); vfio_cxl_destroy_cxl_region(vdev); } =20 @@ -409,6 +417,32 @@ void vfio_cxl_reactivate_region(struct vfio_pci_core_d= evice *vdev) =20 if (!cxl) return; + + /* + * Re-initialise the emulated HDM comp_reg_virt[] from hardware. + * After FLR the decoder registers read as zero; mirror that in + * the emulated state so QEMU sees a clean slate. + */ + vfio_cxl_reinit_comp_regs(vdev); + + /* + * Only re-enable the DPA mmap if the hardware has actually + * re-committed decoder 0 after FLR. Read the COMMITTED bit from the + * freshly-re-snapshotted comp_reg_virt[] so we check the post-FLR + * hardware state, not stale pre-reset state. + * + * If COMMITTED is 0 (slow firmware re-commit path), leave + * region_active=3Dfalse. Guest faults will return VM_FAULT_SIGBUS + * until the decoder is re-committed and the region is re-enabled. + */ + if (cxl->precommitted && cxl->comp_reg_virt) { + u32 ctrl =3D le32_to_cpu(cxl->comp_reg_virt[ + CXL_HDM_DECODER0_CTRL_OFFSET(0) / + CXL_REG_SIZE_DWORD]); + + if (ctrl & CXL_HDM_DECODER_CTRL_COMMITTED_BIT) + WRITE_ONCE(cxl->region_active, true); + } } =20 static ssize_t vfio_cxl_region_rw(struct vfio_pci_core_device *core_dev, diff --git a/drivers/vfio/pci/cxl/vfio_cxl_emu.c b/drivers/vfio/pci/cxl/vfi= o_cxl_emu.c new file mode 100644 index 000000000000..d5603c80fe51 --- /dev/null +++ b/drivers/vfio/pci/cxl/vfio_cxl_emu.c @@ -0,0 +1,366 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved + */ + +#include +#include + +#include "../vfio_pci_priv.h" +#include "vfio_cxl_priv.h" + +/* + * comp_reg_virt[] layout: + * Index 0..N correspond to 32-bit registers at byte offset 0..hdm_reg_s= ize-4 + * within the HDM decoder capability block. + * + * Register layout within the HDM block (CXL spec 8.2.5.19): + * 0x00: HDM Decoder Capability + * 0x04: HDM Decoder Global Control + * 0x08: HDM Decoder Global Status + * 0x0c: (reserved) + * For each decoder N (N=3D0..hdm_count-1), at base 0x10 + N*0x20: + * +0x00: BASE_LO + * +0x04: BASE_HI + * +0x08: SIZE_LO + * +0x0c: SIZE_HI + * +0x10: CTRL + * +0x14: TARGET_LIST_LO + * +0x18: TARGET_LIST_HI + * +0x1c: (reserved) + */ + +static inline __le32 *hdm_reg_ptr(struct vfio_pci_cxl_state *cxl, u32 off) +{ + /* + * off is byte offset within the HDM block; comp_reg_virt is indexed + * as an array of __le32. + */ + return &cxl->comp_reg_virt[off / sizeof(__le32)]; +} + +static ssize_t virt_hdm_rev_reg_write(struct vfio_pci_core_device *vdev, + const __le32 *val32, u64 offset, u64 size) +{ + /* Discard writes on reserved registers. */ + return size; +} + +static ssize_t hdm_decoder_n_lo_write(struct vfio_pci_core_device *vdev, + const __le32 *val32, u64 offset, u64 size) +{ + u32 new_val =3D le32_to_cpu(*val32); + + if (WARN_ON_ONCE(size !=3D CXL_REG_SIZE_DWORD)) + return -EINVAL; + + /* Bit [27:0] are reserved. */ + new_val &=3D ~CXL_HDM_DECODER_BASE_LO_RESERVED_MASK; + + *hdm_reg_ptr(vdev->cxl, offset) =3D cpu_to_le32(new_val); + + return size; +} + +static ssize_t hdm_decoder_global_ctrl_write(struct vfio_pci_core_device *= vdev, + const __le32 *val32, u64 offset, u64 size) +{ + u32 hdm_decoder_global_cap; + u32 new_val =3D le32_to_cpu(*val32); + + if (WARN_ON_ONCE(size !=3D CXL_REG_SIZE_DWORD)) + return -EINVAL; + + /* Bit [31:2] are reserved. */ + new_val &=3D ~CXL_HDM_DECODER_GLOBAL_CTRL_RESERVED_MASK; + + /* Poison On Decode Error Enable bit is 0 and RO if not support. */ + hdm_decoder_global_cap =3D le32_to_cpu(*hdm_reg_ptr(vdev->cxl, 0)); + if (!(hdm_decoder_global_cap & CXL_HDM_CAP_POISON_ON_DECODE_ERR_BIT)) + new_val &=3D ~CXL_HDM_DECODER_GLOBAL_CTRL_POISON_EN_BIT; + + *hdm_reg_ptr(vdev->cxl, offset) =3D cpu_to_le32(new_val); + + return size; +} + +/* + * hdm_decoder_n_ctrl_write - Write handler for HDM decoder CTRL register. + * + * The COMMIT bit (bit 9) is the key: setting it requests the hardware to + * lock the decoder. The emulated COMMITTED bit (bit 10) mirrors COMMIT + * immediately to allow QEMU's notify_change to detect the transition and + * map/unmap the DPA MemoryRegion in the guest address space. + * + * Note: the actual hardware HDM decoder programming (writing the real + * BASE/SIZE with host physical addresses) happens in the QEMU notify_chan= ge + * callback BEFORE this write reaches the hardware. This ordering is + * correct because vfio_region_write() calls notify_change() first. + */ +static ssize_t hdm_decoder_n_ctrl_write(struct vfio_pci_core_device *vdev, + const __le32 *val32, u64 offset, u64 size) +{ + u32 hdm_decoder_global_cap; + u32 ro_mask =3D CXL_HDM_DECODER_CTRL_RO_BITS_MASK; + u32 rev_mask =3D CXL_HDM_DECODER_CTRL_RESERVED_MASK; + u32 new_val =3D le32_to_cpu(*val32); + u32 cur_val; + + if (WARN_ON_ONCE(size !=3D CXL_REG_SIZE_DWORD)) + return -EINVAL; + + cur_val =3D le32_to_cpu(*hdm_reg_ptr(vdev->cxl, offset)); + if (cur_val & CXL_HDM_DECODER_CTRL_COMMIT_LOCK_BIT) + return size; + + hdm_decoder_global_cap =3D le32_to_cpu(*hdm_reg_ptr(vdev->cxl, 0)); + ro_mask |=3D CXL_HDM_DECODER_CTRL_DEVICE_BITS_RO; + rev_mask |=3D CXL_HDM_DECODER_CTRL_DEVICE_RESERVED; + if (!(hdm_decoder_global_cap & CXL_HDM_CAP_UIO_SUPPORTED_BIT)) + rev_mask |=3D CXL_HDM_DECODER_CTRL_UIO_RESERVED; + + new_val &=3D ~rev_mask; + cur_val &=3D ro_mask; + new_val =3D (new_val & ~ro_mask) | cur_val; + + /* + * Mirror COMMIT =E2=86=92 COMMITTED immediately in the emulated state. + * QEMU's notify_change (called before this write reaches hardware) + * reads COMMITTED from the region fd to detect commit transitions. + */ + if (new_val & CXL_HDM_DECODER_CTRL_COMMIT_BIT) + new_val |=3D CXL_HDM_DECODER_CTRL_COMMITTED_BIT; + else + new_val &=3D ~CXL_HDM_DECODER_CTRL_COMMITTED_BIT; + + *hdm_reg_ptr(vdev->cxl, offset) =3D cpu_to_le32(new_val); + + return size; +} + +/* + * Dispatch table for COMP_REGS region writes. Indexed by byte offset with= in + * the HDM decoder block. Returns the appropriate write handler. + * + * Layout: + * 0x00 HDM Decoder Capability (RO) + * 0x04 HDM Global Control (RW with reserved masking) + * 0x08 HDM Global Status (RO) + * 0x0c (reserved) (ignored) + * Per decoder N, base =3D 0x10 + N*0x20: + * base+0x00 BASE_LO (RW, [27:0] reserved) + * base+0x04 BASE_HI (RW) + * base+0x08 SIZE_LO (RW, [27:0] reserved) + * base+0x0c SIZE_HI (RW) + * base+0x10 CTRL (RW, complex rules) + * base+0x14 TARGET_LIST_LO (ignored for Type-2) + * base+0x18 TARGET_LIST_HI (ignored for Type-2) + * base+0x1c (reserved) (ignored) + */ +static ssize_t comp_regs_dispatch_write(struct vfio_pci_core_device *vdev, + u32 off, const __le32 *val32, u32 size) +{ + struct vfio_pci_cxl_state *cxl =3D vdev->cxl; + u32 dec_base, dec_off; + + /* HDM Decoder Capability (0x00): RO */ + if (off =3D=3D 0x00) + return size; + + /* HDM Global Control (0x04) */ + if (off =3D=3D CXL_HDM_DECODER_GLOBAL_CTRL_OFFSET) + return hdm_decoder_global_ctrl_write(vdev, val32, off, size); + + /* HDM Global Status (0x08): RO */ + if (off =3D=3D 0x08) + return size; + + /* Per-decoder registers start at 0x10, stride 0x20 */ + if (off < CXL_HDM_DECODER_FIRST_BLOCK_OFFSET) + return size; /* reserved gap */ + + dec_base =3D CXL_HDM_DECODER_FIRST_BLOCK_OFFSET; + dec_off =3D (off - dec_base) % CXL_HDM_DECODER_BLOCK_STRIDE; + + switch (dec_off) { + case CXL_HDM_DECODER_N_BASE_LOW_OFFSET: /* BASE_LO */ + case CXL_HDM_DECODER_N_SIZE_LOW_OFFSET: /* SIZE_LO */ + return hdm_decoder_n_lo_write(vdev, val32, off, size); + case CXL_HDM_DECODER_N_BASE_HIGH_OFFSET: /* BASE_HI */ + case CXL_HDM_DECODER_N_SIZE_HIGH_OFFSET: /* SIZE_HI */ + /* Full 32-bit write, no reserved bits */ + *hdm_reg_ptr(cxl, off) =3D *val32; + return size; + case CXL_HDM_DECODER_N_CTRL_OFFSET: /* CTRL */ + return hdm_decoder_n_ctrl_write(vdev, val32, off, size); + case CXL_HDM_DECODER_N_TARGET_LIST_LOW_OFFSET: + case CXL_HDM_DECODER_N_TARGET_LIST_HIGH_OFFSET: + case CXL_HDM_DECODER_N_REV_OFFSET: + return virt_hdm_rev_reg_write(vdev, val32, off, size); + default: + return size; + } +} + +/* + * vfio_cxl_comp_regs_rw - regops rw handler for VFIO_REGION_SUBTYPE_CXL_C= OMP_REGS. + * + * Reads return the emulated HDM state (comp_reg_virt[]). + * Writes go through comp_regs_dispatch_write() for bit-field enforcement. + * Only 4-byte aligned 4-byte accesses are supported (hardware requirement= ). + */ +static ssize_t vfio_cxl_comp_regs_rw(struct vfio_pci_core_device *vdev, + char __user *buf, size_t count, + loff_t *ppos, bool iswrite) +{ + struct vfio_pci_cxl_state *cxl =3D vdev->cxl; + loff_t pos =3D *ppos & VFIO_PCI_OFFSET_MASK; + size_t done =3D 0; + + if (!count) + return 0; + + /* Clamp to region size */ + if (pos >=3D cxl->hdm_reg_size) + return -EINVAL; + count =3D min(count, (size_t)(cxl->hdm_reg_size - pos)); + + while (done < count) { + u32 sz =3D min_t(u32, CXL_REG_SIZE_DWORD, count - done); + u32 off =3D pos + done; + __le32 v; + + /* Enforce 4-byte alignment */ + if (sz < CXL_REG_SIZE_DWORD || (off & 0x3)) + return done ? (ssize_t)done : -EINVAL; + + if (iswrite) { + if (copy_from_user(&v, buf + done, sizeof(v))) + return done ? (ssize_t)done : -EFAULT; + comp_regs_dispatch_write(vdev, off, &v, sizeof(v)); + } else { + v =3D *hdm_reg_ptr(cxl, off); + if (copy_to_user(buf + done, &v, sizeof(v))) + return done ? (ssize_t)done : -EFAULT; + } + done +=3D sizeof(v); + } + + *ppos +=3D done; + return done; +} + +static void vfio_cxl_comp_regs_release(struct vfio_pci_core_device *vdev, + struct vfio_pci_region *region) +{ + /* comp_reg_virt is freed in vfio_cxl_clean_virt_regs(), not here. */ +} + +static const struct vfio_pci_regops vfio_cxl_comp_regs_ops =3D { + .rw =3D vfio_cxl_comp_regs_rw, + .release =3D vfio_cxl_comp_regs_release, +}; + +/* + * vfio_cxl_setup_virt_regs - Allocate emulated HDM register state. + * + * Allocates comp_reg_virt as a compact __le32 array covering only + * hdm_reg_size bytes of HDM decoder registers. The initial values + * are read from hardware via the BAR ioremap established by the caller. + * + * DVSEC state is accessed via vdev->vconfig (see the following patch). + */ +int vfio_cxl_setup_virt_regs(struct vfio_pci_core_device *vdev) +{ + struct vfio_pci_cxl_state *cxl =3D vdev->cxl; + size_t nregs; + + if (WARN_ON(!cxl->hdm_reg_size)) + return -EINVAL; + + if (pci_resource_len(vdev->pdev, cxl->comp_reg_bar) < + cxl->comp_reg_offset + cxl->hdm_reg_offset + cxl->hdm_reg_size) + return -ENODEV; + + nregs =3D cxl->hdm_reg_size / sizeof(__le32); + cxl->comp_reg_virt =3D kcalloc(nregs, sizeof(__le32), GFP_KERNEL); + if (!cxl->comp_reg_virt) + return -ENOMEM; + + /* Establish persistent mapping; kept alive until vfio_cxl_clean_virt_reg= s(). */ + cxl->hdm_iobase =3D ioremap(pci_resource_start(vdev->pdev, cxl->comp_reg_= bar) + + cxl->comp_reg_offset + cxl->hdm_reg_offset, + cxl->hdm_reg_size); + if (!cxl->hdm_iobase) { + kfree(cxl->comp_reg_virt); + cxl->comp_reg_virt =3D NULL; + return -ENOMEM; + } + + return 0; +} + +/* + * Called with memory_lock write side held (from vfio_cxl_reactivate_regio= n). + * Uses the pre-established hdm_iobase, no ioremap() under the lock, + * which would deadlock on PREEMPT_RT where ioremap() can sleep. + */ +void vfio_cxl_reinit_comp_regs(struct vfio_pci_core_device *vdev) +{ + struct vfio_pci_cxl_state *cxl =3D vdev->cxl; + size_t i, nregs; + + if (!cxl || !cxl->comp_reg_virt || !cxl->hdm_iobase) + return; + + nregs =3D cxl->hdm_reg_size / sizeof(__le32); + + for (i =3D 0; i < nregs; i++) + cxl->comp_reg_virt[i] =3D + cpu_to_le32(readl(cxl->hdm_iobase + i * sizeof(__le32))); +} + +void vfio_cxl_clean_virt_regs(struct vfio_pci_core_device *vdev) +{ + struct vfio_pci_cxl_state *cxl =3D vdev->cxl; + + if (cxl->hdm_iobase) { + iounmap(cxl->hdm_iobase); + cxl->hdm_iobase =3D NULL; + } + kfree(cxl->comp_reg_virt); + cxl->comp_reg_virt =3D NULL; +} + +/* + * vfio_cxl_register_comp_regs_region - Register the COMP_REGS device regi= on. + * + * Exposes the emulated HDM decoder register state as a VFIO device region + * with type VFIO_REGION_SUBTYPE_CXL_COMP_REGS. QEMU attaches a + * notify_change callback to this region to intercept HDM COMMIT writes + * and map the DPA MemoryRegion at the appropriate GPA. + * + * The region is read+write only (no mmap) to ensure all accesses pass + * through comp_regs_dispatch_write() for proper bit-field enforcement. + */ +int vfio_cxl_register_comp_regs_region(struct vfio_pci_core_device *vdev) +{ + struct vfio_pci_cxl_state *cxl =3D vdev->cxl; + u32 flags =3D VFIO_REGION_INFO_FLAG_READ | VFIO_REGION_INFO_FLAG_WRITE; + int ret; + + if (!cxl || !cxl->comp_reg_virt) + return -ENODEV; + + ret =3D vfio_pci_core_register_dev_region(vdev, + PCI_VENDOR_ID_CXL | + VFIO_REGION_TYPE_PCI_VENDOR_TYPE, + VFIO_REGION_SUBTYPE_CXL_COMP_REGS, + &vfio_cxl_comp_regs_ops, + cxl->hdm_reg_size, flags, cxl); + if (!ret) + cxl->comp_reg_region_idx =3D vdev->num_regions - 1; + + return ret; +} +EXPORT_SYMBOL_GPL(vfio_cxl_register_comp_regs_region); diff --git a/drivers/vfio/pci/cxl/vfio_cxl_priv.h b/drivers/vfio/pci/cxl/vf= io_cxl_priv.h index b870926bfb19..4f2637874e9d 100644 --- a/drivers/vfio/pci/cxl/vfio_cxl_priv.h +++ b/drivers/vfio/pci/cxl/vfio_cxl_priv.h @@ -25,14 +25,51 @@ struct vfio_pci_cxl_state { size_t hdm_reg_size; resource_size_t comp_reg_offset; size_t comp_reg_size; + __le32 *comp_reg_virt; + void __iomem *hdm_iobase; u32 hdm_count; int dpa_region_idx; + int comp_reg_region_idx; u16 dvsec; u8 comp_reg_bar; bool precommitted; bool region_active; }; =20 +/* Register access sizes */ +#define CXL_REG_SIZE_WORD 2 +#define CXL_REG_SIZE_DWORD 4 + +/* HDM Decoder - register offsets (CXL 2.0 8.2.5.19) */ +#define CXL_HDM_DECODER_GLOBAL_CTRL_OFFSET 0x4 +#define CXL_HDM_DECODER_FIRST_BLOCK_OFFSET 0x10 +#define CXL_HDM_DECODER_BLOCK_STRIDE 0x20 +#define CXL_HDM_DECODER_N_BASE_LOW_OFFSET 0x0 +#define CXL_HDM_DECODER_N_BASE_HIGH_OFFSET 0x4 +#define CXL_HDM_DECODER_N_SIZE_LOW_OFFSET 0x8 +#define CXL_HDM_DECODER_N_SIZE_HIGH_OFFSET 0xc +#define CXL_HDM_DECODER_N_CTRL_OFFSET 0x10 +#define CXL_HDM_DECODER_N_TARGET_LIST_LOW_OFFSET 0x14 +#define CXL_HDM_DECODER_N_TARGET_LIST_HIGH_OFFSET 0x18 +#define CXL_HDM_DECODER_N_REV_OFFSET 0x1c + +/* HDM Decoder Global Capability / Control - bit definitions */ +#define CXL_HDM_CAP_POISON_ON_DECODE_ERR_BIT BIT(10) +#define CXL_HDM_CAP_UIO_SUPPORTED_BIT BIT(13) + +/* HDM Decoder N Control */ +#define CXL_HDM_DECODER_CTRL_COMMIT_LOCK_BIT BIT(8) +#define CXL_HDM_DECODER_CTRL_COMMIT_BIT BIT(9) +#define CXL_HDM_DECODER_CTRL_COMMITTED_BIT BIT(10) +#define CXL_HDM_DECODER_CTRL_RO_BITS_MASK (BIT(10) | BIT(11)) +#define CXL_HDM_DECODER_CTRL_RESERVED_MASK (BIT(15) | GENMASK(31, 28)) +#define CXL_HDM_DECODER_CTRL_DEVICE_BITS_RO BIT(12) +#define CXL_HDM_DECODER_CTRL_DEVICE_RESERVED (GENMASK(19, 16) | GENMASK(= 23, 20)) +#define CXL_HDM_DECODER_CTRL_UIO_RESERVED (BIT(14) | GENMASK(27, 24)) +#define CXL_HDM_DECODER_BASE_LO_RESERVED_MASK GENMASK(27, 0) +#define CXL_HDM_DECODER_GLOBAL_CTRL_RESERVED_MASK GENMASK(31, 2) +#define CXL_HDM_DECODER_GLOBAL_CTRL_POISON_EN_BIT BIT(0) + /* * CXL DVSEC for CXL Devices - register offsets within the DVSEC * (CXL 2.0+ 8.1.3). @@ -41,4 +78,8 @@ struct vfio_pci_cxl_state { #define CXL_DVSEC_CAPABILITY_OFFSET 0xa #define CXL_DVSEC_MEM_CAPABLE BIT(2) =20 +int vfio_cxl_setup_virt_regs(struct vfio_pci_core_device *vdev); +void vfio_cxl_clean_virt_regs(struct vfio_pci_core_device *vdev); +void vfio_cxl_reinit_comp_regs(struct vfio_pci_core_device *vdev); + #endif /* __LINUX_VFIO_CXL_PRIV_H */ diff --git a/drivers/vfio/pci/vfio_pci_priv.h b/drivers/vfio/pci/vfio_pci_p= riv.h index 8f440f9eaa0c..f8db9a05c033 100644 --- a/drivers/vfio/pci/vfio_pci_priv.h +++ b/drivers/vfio/pci/vfio_pci_priv.h @@ -152,6 +152,8 @@ int vfio_cxl_register_cxl_region(struct vfio_pci_core_d= evice *vdev); void vfio_cxl_unregister_cxl_region(struct vfio_pci_core_device *vdev); void vfio_cxl_zap_region_locked(struct vfio_pci_core_device *vdev); void vfio_cxl_reactivate_region(struct vfio_pci_core_device *vdev); +int vfio_cxl_register_comp_regs_region(struct vfio_pci_core_device *vdev); +void vfio_cxl_reinit_comp_regs(struct vfio_pci_core_device *vdev); =20 #else =20 @@ -173,6 +175,11 @@ static inline void vfio_cxl_zap_region_locked(struct vfio_pci_core_device *vdev) { } static inline void vfio_cxl_reactivate_region(struct vfio_pci_core_device *vdev) { } +static inline int +vfio_cxl_register_comp_regs_region(struct vfio_pci_core_device *vdev) +{ return 0; } +static inline void +vfio_cxl_reinit_comp_regs(struct vfio_pci_core_device *vdev) { } =20 #endif /* CONFIG_VFIO_CXL_CORE */ =20 --=20 2.25.1 From nobody Tue Apr 7 19:54:44 2026 Received: from PH7PR06CU001.outbound.protection.outlook.com (mail-westus3azon11010065.outbound.protection.outlook.com [52.101.201.65]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EEBC938B128; Wed, 11 Mar 2026 20:37:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.201.65 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773261436; cv=fail; b=JziYxw/xYOesr73mcokx+Wk7B9JjQsv9Hh3+5TfwS1JnG5BibCUtaNuFKcu89o+ufIXOnl7KcuHjIUHgaaPVUDJaqsLJHlsSfrafxHt4SLpoM3zjAFOygpGjtqWZR/Z40c88rzbORiHlLhfUUXyHOw0DDeQs9GsXy+ARtSnjly8= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773261436; c=relaxed/simple; bh=e4n10ohjFp6RXqcpVVlyAItBgYZ71DyFYfVTlwZtef8=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=fWhJsDNvZKlS/GJfKxQEGbT7rb/G7k+Yr2KRnLmN0NM8sIwAKVmXJZKNkFcCue90GBBodndY6QOD8TEFYsBey/a3biXlAYnh6gD+T1gk/QetKbWc3aDEfH+MhZ0J5u/AXPm/B/6UyL8jigYKE32rSIuQS3xCBu1VVu3OdcR5FDA= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=qUW/meIW; arc=fail smtp.client-ip=52.101.201.65 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="qUW/meIW" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=RaO+xXgh2IwBh+xMoDMa8gIWVJ29VAZumCVVPC0lGl0rwuCERlzQyxwfVvNa5PxoFSlZI8XNQNbaaAiHHNa+iVwyFY+51+twyI+RNWJ8Wf/loC/td1+sp8Y753k1WGgJH99k7ZGf08cQa74qkexZMO5cg7TSALkR5m7h9t0Q6pzAno8fwP3zzoHTSNXMOHzZnvf4LeoA0csXsARwF7SpDuOGzR48xWVYeCROlpeeQdvRLj+fxM7vA5SFwLf4wu+pW2SIARtwGg/CBbfkNRZ5Hj7jklXKmhcE0ULeo7JhQ7QTsGf/oektFj43Bdiy19tjY0Oalb76Y//dDfZiX9eh3A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=5kcutj/1qDb0w1eYvBzYmjC7sE8B17aXoQwdEUaGrQQ=; b=MhOF0r+vY6ZG96EHtrHGY7p22cdOog8gazolXNFHUZCrYuMaCt1EgUZH9SA5ey5JaN8VE6jYyidXv5+kIrafOylwYOAo0OrGEUnVbRHh2wp7fQ5PCIX0F2VvlSlqinMU10qfZIeFCD5vVcoNI2lgL4gv8Gpogb39kUyW+gl/n0hLOfJK77E3jrYOM448nNJBDoR905efVmU0kn1Y6bjG2BnxVux0HrT22OtLiM88KJet8A29vO5nNr+KiRR2SX/qole2qD22b2gMgd64EwlgY4OE1AOgLzU76k9mN7ncdecee4p3FSEIStS2vTKIUEBn9/lfxWEBJv8uoSPXMXTTZQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=5kcutj/1qDb0w1eYvBzYmjC7sE8B17aXoQwdEUaGrQQ=; b=qUW/meIWJZlS5IL6QKMgwhOJzc+eqXTY2ceds+ipsztXZvmJsgUvbeq6dsuv0FbLq1m8ph/cYseZhrfoU/biz9uOhRicTCEK2dqa3Oft6RG8Fyg4kggkme9U0M2HiG/otfUW02IPqKvcIguFs+EO1NxlHN5lRad1u5cubfXMZYK2ZKeXcf1aFPhziRiesLoocxXyVYx3oICxyHDB7FshKhGmgb9yutUVhixBgEwUIDFckrzt8e02VUvEmJ9FrYbhTNhjzAi3mPMyz9NfGI13hEtr1jzlKBZ2xZI7UabwbT05MWw1unOXs+VmghAJkODFy5tuh9e/CoTEm2S6+0SZig== Received: from DM6PR05CA0051.namprd05.prod.outlook.com (2603:10b6:5:335::20) by DM6PR12MB4450.namprd12.prod.outlook.com (2603:10b6:5:28e::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9723.3; Wed, 11 Mar 2026 20:37:07 +0000 Received: from DS2PEPF00003448.namprd04.prod.outlook.com (2603:10b6:5:335:cafe::5) by DM6PR05CA0051.outlook.office365.com (2603:10b6:5:335::20) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9700.15 via Frontend Transport; Wed, 11 Mar 2026 20:37:07 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by DS2PEPF00003448.mail.protection.outlook.com (10.167.17.75) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9678.18 via Frontend Transport; Wed, 11 Mar 2026 20:37:07 +0000 Received: from rnnvmail204.nvidia.com (10.129.68.6) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Wed, 11 Mar 2026 13:36:43 -0700 Received: from rnnvmail201.nvidia.com (10.129.68.8) by rnnvmail204.nvidia.com (10.129.68.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Wed, 11 Mar 2026 13:36:42 -0700 Received: from nvidia-4028GR-scsim.nvidia.com (10.127.8.11) by mail.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20 via Frontend Transport; Wed, 11 Mar 2026 13:36:35 -0700 From: To: , , , , , , , , , , , , , , , , , CC: , , , , , , , Subject: [PATCH 14/20] vfio/cxl: Check media readiness and create CXL memdev Date: Thu, 12 Mar 2026 02:04:34 +0530 Message-ID: <20260311203440.752648-15-mhonap@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20260311203440.752648-1-mhonap@nvidia.com> References: <20260311203440.752648-1-mhonap@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS2PEPF00003448:EE_|DM6PR12MB4450:EE_ X-MS-Office365-Filtering-Correlation-Id: 125e3338-b950-41a2-e0ab-08de7fadf697 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|36860700016|82310400026|7416014|376014|1800799024|56012099003|18002099003|22082099003|921020; X-Microsoft-Antispam-Message-Info: 97cSGeU55p2mzWykshNUfv/OfA87fUyHtau2MB2Cp3fGvq69r1ZRy9Rgm6El6b/xOSMVmdEDotZ6tQ6KYZbXndWVqT34I5hA86VBS5WPjAq7slPKf8XTPecmr1GDs8u4vzc5ntL23GDQbYxMhTy9Cs3EVQiT+/Q5DYtotWhAvhacxfpyp9RkXWLnJ4HExlRidCfkaNWCbr/Q2ZC34ZV69Ju2TMtC6Sw4XZ7NE/6vuHYJP4rxo3MCz3/5VkCzAyhMmJhiNa395Z4Lhstxwast9fTQDDJuZ4cpl+tplQNF4qxxKqvTBy31WCqicH9f5xEC+FooCpyFZzwc8k6BJGjsqiXw4PppE36pcdQmOBL8AyL8NvRP27bvHee9uWafC8G39gVrypwGxXEhMEGU+Jt4q0yAsCLPiu0r6YHwI1FS8cLPrLXS2TpYpD4n7clpv0nncvuOXeRZZsl+4GMCLToF/3VpUWDthNwctJsw/+kj9p3W7IzKh3ECBakUeftqhqmIJ7NFzBUC4StsON5jc8q7qToBo1xsOD9q5QccSj68olzONIclqVRxbh7Lm2efASZOm9Y9BgBmri7Fl/vQvtkQn6FWgwn15xAWDh6ZZEQpS6VNcOAg5a1hQHge4gFTpGE3POsesOOIpmYz3yj1bU9tAVs+JE9USeH3nVAejO22qt5Hre6PZgMD8Xh4vbUGZ1CY+z5/fVsmsb3sHAxMcLOAURNXYRZWMRnlQKBxR6WEmDoe1is3ROpGGy32DXMWBE3OHDmPV+lpBAQmoHOnUE+e/Xmqm04letSi4hNbnXJmgppNbqA8OhGLTPwq4EmRIQiR X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(36860700016)(82310400026)(7416014)(376014)(1800799024)(56012099003)(18002099003)(22082099003)(921020);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: mQO3AdbRK8eGzd90D9W1ZdJpQfdJXkLOJFzzNRNDoE25KV2hEinXRHwbD52isuT+aDbDzm1AXFT6G3SYy133pjz+JCXl18Ey6RgnqxDnXaEph+2v9qzc/BH8L62y+CeGRBJ6V9WsASm5d6ISKsphaquCAIUpgD9RORitX68/OtOcOjhV2OYrt3yBvNacbVh5dQoIQlG2Fkg2JRt8tMx/Mo3+b0/yhiRo60iijdcooSi9AUvjXrgYBnDJO9qnFSzaoShTomN1Y1dnVe+YfiZclbMlMLAto0SCd+09RlioAOJ6Zk7we9CIOI4F5N8H1gm9ZgI7HDQ5wX/cPeQ58kugHx+vthzi/GJkfu3xtkqeQ+hUOrUGUygXSDNVyYirNsJZ71QfZ4iR5t9hMxVPyhgZHEaGSaqZ7rL7uVN0HL4HzlKlyIUUg31cQ7mcN+xhe0u/ X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Mar 2026 20:37:07.5032 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 125e3338-b950-41a2-e0ab-08de7fadf697 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DS2PEPF00003448.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB4450 Content-Type: text/plain; charset="utf-8" From: Manish Honap Check media readiness at probe time and create a CXL memdev for region management. Media/range-active check is performed at probe time to keep the vfio-is-advertised-as-cxl behavior consistent. A pre-committed HDM decoder already implies media is active, so set media_ready directly instead of calling the potentially blocking cxl_await_range_active(). For memdev creation we need to determine capacity before calling devm_cxl_add_memdev(). Read the committed decoder size directly from HDM decoder hardware registers; the CXL core will see the same values when it enumerates decoders inside add_memdev. For firmware uncommitted decoders, handling will be added in a later commit. Signed-off-by: Manish Honap --- drivers/vfio/pci/cxl/vfio_cxl_core.c | 67 +++++++++++++++++++++++++++- drivers/vfio/pci/cxl/vfio_cxl_emu.c | 48 ++++++++++++++++++++ drivers/vfio/pci/cxl/vfio_cxl_priv.h | 2 + 3 files changed, 115 insertions(+), 2 deletions(-) diff --git a/drivers/vfio/pci/cxl/vfio_cxl_core.c b/drivers/vfio/pci/cxl/vf= io_cxl_core.c index d2401871489d..15b6c0d75d9e 100644 --- a/drivers/vfio/pci/cxl/vfio_cxl_core.c +++ b/drivers/vfio/pci/cxl/vfio_cxl_core.c @@ -132,6 +132,37 @@ static int vfio_cxl_setup_regs(struct vfio_pci_core_de= vice *vdev) return 0; } =20 +static int vfio_cxl_create_memdev(struct vfio_pci_core_device *vdev, + resource_size_t capacity) +{ + struct vfio_pci_cxl_state *cxl =3D vdev->cxl; + struct pci_dev *pdev =3D vdev->pdev; + int ret; + + ret =3D cxl_set_capacity(&cxl->cxlds, capacity); + if (ret) { + pci_err(pdev, "Failed to set capacity: %d\n", ret); + return ret; + } + + pci_dbg(pdev, "Device capacity: %llu MB (from %s)\n", + capacity >> 20, + cxl->precommitted ? "committed decoder" : "sysfs"); + pci_dbg(pdev, + "vfio_cxl: creating memdev: capacity=3D0x%llx bytes (%llu MiB)\n", + (unsigned long long)capacity, + (unsigned long long)(capacity >> 20)); + + cxl->cxlmd =3D devm_cxl_add_memdev(&cxl->cxlds, NULL); + if (IS_ERR(cxl->cxlmd)) { + pci_err(pdev, "Failed to add CXL memdev: %ld\n", + PTR_ERR(cxl->cxlmd)); + return PTR_ERR(cxl->cxlmd); + } + + return 0; +} + int vfio_cxl_create_cxl_region(struct vfio_pci_core_device *vdev, resource= _size_t size) { struct vfio_pci_cxl_state *cxl =3D vdev->cxl; @@ -250,6 +281,7 @@ void vfio_pci_cxl_detect_and_init(struct vfio_pci_core_= device *vdev) { struct pci_dev *pdev =3D vdev->pdev; struct vfio_pci_cxl_state *cxl; + resource_size_t capacity =3D 0; u16 dvsec; int ret; =20 @@ -282,13 +314,44 @@ void vfio_pci_cxl_detect_and_init(struct vfio_pci_cor= e_device *vdev) goto failed; } =20 + cxl->cxlds.media_ready =3D !cxl_await_range_active(&cxl->cxlds); + if (!cxl->cxlds.media_ready) { + pci_disable_device(pdev); + pci_err(pdev, "CXL media not ready\n"); + goto regs_failed; + } + + /* + * Take the single authoritative HDM decoder snapshot now that + * MEM_ACTIVE is confirmed and BAR memory is still enabled. Using + * readl() per-dword ensures correct MMIO serialisation and captures + * the final firmware-written values for all fields including SIZE_HIGH, + * which firmware commits to the BAR at MEM_ACTIVE time. + */ + vfio_cxl_reinit_comp_regs(vdev); + pci_disable_device(pdev); =20 - ret =3D vfio_cxl_create_region_helper(vdev, SZ_256M); - if (ret) + capacity =3D vfio_cxl_read_committed_decoder_size(vdev); + if (capacity =3D=3D 0) { + /* + * TODO: Add handling for devices which do not have + * firmware pre-committed decoders + */ + pci_info(pdev, "Uncommitted region size must be configured via sysfs bef= ore bind\n"); goto regs_failed; + } =20 cxl->precommitted =3D true; + cxl->dpa_size =3D capacity; + + ret =3D vfio_cxl_create_memdev(vdev, capacity); + if (ret) + goto regs_failed; + + ret =3D vfio_cxl_create_region_helper(vdev, capacity); + if (ret) + goto regs_failed; =20 return; =20 diff --git a/drivers/vfio/pci/cxl/vfio_cxl_emu.c b/drivers/vfio/pci/cxl/vfi= o_cxl_emu.c index d5603c80fe51..178a42267642 100644 --- a/drivers/vfio/pci/cxl/vfio_cxl_emu.c +++ b/drivers/vfio/pci/cxl/vfio_cxl_emu.c @@ -300,6 +300,54 @@ int vfio_cxl_setup_virt_regs(struct vfio_pci_core_devi= ce *vdev) return 0; } =20 +/* + * vfio_cxl_read_committed_decoder_size - Extract committed DPA capacity f= rom + * comp_reg_virt[]. + * + * Called from probe context after vfio_cxl_reinit_comp_regs() has taken t= he + * post-MEM_ACTIVE readl() snapshot and patched SIZE_HIGH/SIZE_LOW from DV= SEC. + * comp_reg_virt[] is already correct at this point; no hardware access ne= eded. + * + * Returns the committed DPA capacity in bytes, or 0 if the decoder is not + * committed. + */ +resource_size_t +vfio_cxl_read_committed_decoder_size(struct vfio_pci_core_device *vdev) +{ + struct vfio_pci_cxl_state *cxl =3D vdev->cxl; + struct pci_dev *pdev =3D vdev->pdev; + resource_size_t capacity; + u32 ctrl, sz_hi, sz_lo; + + if (WARN_ON(!cxl || !cxl->comp_reg_virt)) + return 0; + + ctrl =3D le32_to_cpu(cxl->comp_reg_virt[CXL_HDM_DECODER0_CTRL_OFFSET(0) / + CXL_REG_SIZE_DWORD]); + sz_hi =3D le32_to_cpu(cxl->comp_reg_virt[CXL_HDM_DECODER0_SIZE_HIGH_OFFSE= T(0) / + CXL_REG_SIZE_DWORD]); + sz_lo =3D le32_to_cpu(cxl->comp_reg_virt[CXL_HDM_DECODER0_SIZE_LOW_OFFSET= (0) / + CXL_REG_SIZE_DWORD]); + + if (!(ctrl & CXL_HDM_DECODER0_CTRL_COMMITTED)) { + pci_dbg(pdev, + "vfio_cxl: decoder0 not committed: ctrl=3D0x%08x\n", + ctrl); + return 0; + } + + capacity =3D ((resource_size_t)sz_hi << 32) | (sz_lo & GENMASK(31, 28)); + + pci_dbg(pdev, + "vfio_cxl: decoder0 committed: sz_hi=3D0x%08x sz_lo=3D0x%08x " + "capacity=3D0x%llx (%llu MiB)\n", + sz_hi, sz_lo, + (unsigned long long)capacity, + (unsigned long long)(capacity >> 20)); + + return capacity; +} + /* * Called with memory_lock write side held (from vfio_cxl_reactivate_regio= n). * Uses the pre-established hdm_iobase, no ioremap() under the lock, diff --git a/drivers/vfio/pci/cxl/vfio_cxl_priv.h b/drivers/vfio/pci/cxl/vf= io_cxl_priv.h index 4f2637874e9d..3ef8d923a7e8 100644 --- a/drivers/vfio/pci/cxl/vfio_cxl_priv.h +++ b/drivers/vfio/pci/cxl/vfio_cxl_priv.h @@ -26,6 +26,7 @@ struct vfio_pci_cxl_state { resource_size_t comp_reg_offset; size_t comp_reg_size; __le32 *comp_reg_virt; + size_t dpa_size; void __iomem *hdm_iobase; u32 hdm_count; int dpa_region_idx; @@ -81,5 +82,6 @@ struct vfio_pci_cxl_state { int vfio_cxl_setup_virt_regs(struct vfio_pci_core_device *vdev); void vfio_cxl_clean_virt_regs(struct vfio_pci_core_device *vdev); void vfio_cxl_reinit_comp_regs(struct vfio_pci_core_device *vdev); +resource_size_t vfio_cxl_read_committed_decoder_size(struct vfio_pci_core_= device *vdev); =20 #endif /* __LINUX_VFIO_CXL_PRIV_H */ --=20 2.25.1 From nobody Tue Apr 7 19:54:44 2026 Received: from PH8PR06CU001.outbound.protection.outlook.com (mail-westus3azon11012050.outbound.protection.outlook.com [40.107.209.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 24E50379EF4; Wed, 11 Mar 2026 20:37:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.209.50 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773261447; cv=fail; b=mLOOyRPIeULWlQTVrpj9aTW2iCbJzTJjHYnJAa7GrefUosTQm7XNRdok97CTrZW7Ux5Z7pgcvwrpA+YfrhGYSQ3J+/vpa9Zr6ZYbnmlU9UCCipC4nXurhIw5TXMm/YSKz0TVTC1Lu4a70KQ7YjbFV/tqQeShStppfMb3uTfX+oY= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773261447; c=relaxed/simple; bh=J6ENd4AJwiq8yKkAh6G1+AE/lkOfphk1SXYwun3TWdA=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=hNwtEgx7e3FLT/+TjEJsNJBds/qnK/aZrJwpM7CnLDlN7gfnSPz5nDubbpRncb6TYVFA7xi4uWUCzjVxeoPm/BlYRBBP7bjUJQYoSdGfbctkU40vvSc/h2BDeB/4BDmog/nkMPxQqv1fzIp5GwrW2fzyJZQ21OyJe5oFri3k9VE= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=mfzOKdgY; arc=fail smtp.client-ip=40.107.209.50 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="mfzOKdgY" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=lpR+x0Y2VbL+hKAx7S9eagheWzNBJIHRCgjf2FETNsW1XSqVSa/znlc74GJ7WQ6pEfGcfDG6ZYlV2nWOiuJh5vaunWgdFJbpkHcJX3wQe1XaE22+gCnnaBdwdi16KTgSHOryQIXVhXbZ2hWmcnpyMk1sJX9wyPS6MRtNXyQakkJETYN/SRZ2bf5C06LOIyyxY4E+NLZYg4GfkgPgln7cd4PFpEjnE3Gso3/nG5ROcEO7oXb8odJxNsgvF+F4jqbArn22Tz7Ki8yN/gm/5STu2rQUTzEdgbSGQiBnJ82nv+lhqhf9EomOBp2RtmddCNu0t5sP5PDdf4NDM4MJ4kCoPA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=urbTYH4gE5UhYNBEZ3RXc1XeUyOnqto0+ax5DkPJf0o=; b=pc5PzaPjEioHagBaBMRX00UH3Y/bYt4sDd4sD+vWbbCF6XSmsp/KMeoktN9KUS0P6zaVQLDBQ2L2VyhYYVi5f/f+wvBux8HETwhma+jhr7/hcd+RSPiGB6xHyqN5YoAmSVRVFXV9pJWDUHElijU6rxg7CMi1+I3EQAnHTZY+2DYCkcwlG9rYoUyliqAvvAKvsqHvWTeZ0lKcOAxbY8lBcLVoKTOdLQU6n4FhpX0ctNeNdIRNnZEqrMwTEjv5c6qDmRqZw8x/JHuv+2O9Pp7byhOeJg6hh5dB1W2Sjs+3fMAcJ35Qqjnw2nhLSUsjL1aydRH5QmOyQuYck8MkHbGZ9w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=urbTYH4gE5UhYNBEZ3RXc1XeUyOnqto0+ax5DkPJf0o=; b=mfzOKdgYfc7kTqVjv/n/B0YIqwYmx4FxQuGkl38+DhA8s4A9z9HQt6dJpjJx1sJNbFD8NNxUIRcdmcLcW9zXj6S4JpeSW512Ni8aeZu9piMALX5p9913r9qEAzYABkE+Ke2uKmxEmvf1df1R/tDctTnTJuIhOJIO+2qbmYS8pWhhz67OAgCcvTt5p4ew/2tZFGfzStvYRU2GFV++lFGkBCZlUezOxxXoMQiiiV5Wgtjv23uE9YcHf23tzz64Z8wrCj45C/biXRL+pQBM2+Kzr1YzWYVY0n9P/oJKf1sEkAYJO/IsF2NkD8xAtaV+6Uwwx9pC8qRU0sqeHRbMvHU64w== Received: from DS7PR05CA0041.namprd05.prod.outlook.com (2603:10b6:8:2f::8) by SJ0PR12MB6736.namprd12.prod.outlook.com (2603:10b6:a03:47a::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9700.11; Wed, 11 Mar 2026 20:37:13 +0000 Received: from DS2PEPF00003445.namprd04.prod.outlook.com (2603:10b6:8:2f:cafe::1c) by DS7PR05CA0041.outlook.office365.com (2603:10b6:8:2f::8) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9678.25 via Frontend Transport; Wed, 11 Mar 2026 20:37:13 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by DS2PEPF00003445.mail.protection.outlook.com (10.167.17.72) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9678.18 via Frontend Transport; Wed, 11 Mar 2026 20:37:13 +0000 Received: from rnnvmail204.nvidia.com (10.129.68.6) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Wed, 11 Mar 2026 13:36:50 -0700 Received: from rnnvmail201.nvidia.com (10.129.68.8) by rnnvmail204.nvidia.com (10.129.68.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Wed, 11 Mar 2026 13:36:50 -0700 Received: from nvidia-4028GR-scsim.nvidia.com (10.127.8.11) by mail.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20 via Frontend Transport; Wed, 11 Mar 2026 13:36:43 -0700 From: To: , , , , , , , , , , , , , , , , , CC: , , , , , , , Subject: [PATCH 15/20] vfio/cxl: Introduce CXL DVSEC configuration space emulation Date: Thu, 12 Mar 2026 02:04:35 +0530 Message-ID: <20260311203440.752648-16-mhonap@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20260311203440.752648-1-mhonap@nvidia.com> References: <20260311203440.752648-1-mhonap@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS2PEPF00003445:EE_|SJ0PR12MB6736:EE_ X-MS-Office365-Filtering-Correlation-Id: b9ba0643-891d-49d1-f735-08de7fadfa1d X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|36860700016|376014|7416014|82310400026|921020|13003099007|22082099003|18002099003|56012099003; X-Microsoft-Antispam-Message-Info: wklbeJNh01UqW8R3fYjo1ChQaLbI2au0KimVKRQUQTJLImt0zJLGXXpGcn6jJ0Yu3mQzh3hjeAgy6yN5q0Nj0Pic0BcmKIbnR4XEP8ExmfFWsh++UUTx6Cc08/OdwwxRGsPA4jDlaGZ8oSWhcF5z7D9OYmOXusqj9tDGJHaejupzfeXCcN5F4Tmtjn0eE/vTZ2o/k7G7wX3yHiRs1pIhb0XoQ3HOPSN2T6nixNod2+SWpAsLn7XevTz0zgwxOy6UeoLXYTRzpjnAadXTPR8QaG3MKbb75MhU8t1Hk8zBZYoNhxrUpThvFNMhnuuK3apDIM8hJBZcwhoxr2X+aisd6kf8r5PIcEtrtzzsjPeHBzpE0nKsJOFXFMMIH+uK4WG+RMIlO+fUikbpJo66RA9/oKXITezltZbF7NZapSIODK+nzsfpm8DFSeDl2Qw1YglenPxzN4mALWk3H76sA7Cqj4vFqg6JaeYWj3WQsqzmAb+tzoky8/SIKVoQKxK23zRiFvundN0bQz1E9EYykUI/MRXJ6iY0RJ3EIk322egCAOkRN4wjJ497Q7owY7mYYb26ois0x/NScSL1WvGrrYYi5L9TnQVmyO53MmLfheC4nq2+xDxGdPM9uv679lijsEnbsows23AFLrsLLz5tUrGMv0S55CTS2KV+48BcBEcAA3kQPEiElZwlMkys03zwXiF47jeKAsXrpEBF//bFkf3vd8rZICCZOKk+ieAsFUxtOjCyVKsiwShAyLB/inN5rv/OVV0TgNvlyhoREBpHUzRgTKbEo/p+gd+7n/dKm+aeMF9ymjX9vUTnErbIPEs7HVdb X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(1800799024)(36860700016)(376014)(7416014)(82310400026)(921020)(13003099007)(22082099003)(18002099003)(56012099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 7pRZJV22lLm1HWqqmGxBsJs04MNBFgK5brS9wRS+Zi4PNy0M4IKKkDHCvl9YCsW82rcvU4CLFqRJO5ZUtNMtbm9UEr11AFeXFvtvGwkP3AS/ch9R1u5fAeS4CQNPGhGjgyNAB7I0EwJEWS0ka2Q9U6mYbBeC7rA1Z8AJzege5ZrJ/3EmQpNnouxF6tYNCMwVzaoFnsIk+VxurF8Loyh7CnWnS56fxEArfIYl6kT+5hbJ8lgIPhwhIhzuarxdNlxMl+vnM6J2KARsG1gs3+I5QQC4hgrVKoNizVhtsYkBD1IxhAuSjyATbBijC+NLnWrTGOZ7i//Fw5InQ7Hyf3N6jiGzCJGiC38DQvxq45p5ebA9SOfrqrKcaVXKS+7NqYceLGypxjYevQAavHGHUulb8XRGkH+EW/SB2AyOytUyOtvx0Npc266nMX/kctZ6WYgV X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Mar 2026 20:37:13.4062 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: b9ba0643-891d-49d1-f735-08de7fadfa1d X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DS2PEPF00003445.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR12MB6736 Content-Type: text/plain; charset="utf-8" From: Manish Honap CXL devices have CXL DVSEC registers in the configuration space. Many of them affect the behaviors of the devices, e.g. enabling CXL.io/CXL.mem/CXL.cache. However, these configurations are owned by the host and a virtualization policy should be applied when handling the access from the guest. Introduce the emulation of CXL configuration space to handle the access of the virtual CXL configuration space from the guest. vfio-pci-core already allocates vdev->vconfig as the authoritative virtual config space shadow. Directly use vdev->vconfig: - DVSEC reads return data from vdev->vconfig (already populated by vfio_config_init() via vfio_ecap_init()) - DVSEC writes go through new CXL-aware write handlers that update vdev->vconfig in place - The writable DVSEC registers are marked virtual in vdev->pci_config_map Signed-off-by: Zhi Wang Signed-off-by: Manish Honap --- drivers/vfio/pci/Makefile | 2 +- drivers/vfio/pci/cxl/vfio_cxl_config.c | 304 +++++++++++++++++++++++++ drivers/vfio/pci/cxl/vfio_cxl_core.c | 4 + drivers/vfio/pci/cxl/vfio_cxl_priv.h | 38 +++- drivers/vfio/pci/vfio_pci.c | 14 ++ drivers/vfio/pci/vfio_pci_config.c | 46 +++- drivers/vfio/pci/vfio_pci_priv.h | 3 + include/linux/vfio_pci_core.h | 8 +- 8 files changed, 415 insertions(+), 4 deletions(-) create mode 100644 drivers/vfio/pci/cxl/vfio_cxl_config.c diff --git a/drivers/vfio/pci/Makefile b/drivers/vfio/pci/Makefile index bef916495eae..7c86b7845e8f 100644 --- a/drivers/vfio/pci/Makefile +++ b/drivers/vfio/pci/Makefile @@ -1,7 +1,7 @@ # SPDX-License-Identifier: GPL-2.0-only =20 vfio-pci-core-y :=3D vfio_pci_core.o vfio_pci_intrs.o vfio_pci_rdwr.o vfio= _pci_config.o -vfio-pci-core-$(CONFIG_VFIO_CXL_CORE) +=3D cxl/vfio_cxl_core.o cxl/vfio_cx= l_emu.o +vfio-pci-core-$(CONFIG_VFIO_CXL_CORE) +=3D cxl/vfio_cxl_core.o cxl/vfio_cx= l_emu.o cxl/vfio_cxl_config.o vfio-pci-core-$(CONFIG_VFIO_PCI_ZDEV_KVM) +=3D vfio_pci_zdev.o vfio-pci-core-$(CONFIG_VFIO_PCI_DMABUF) +=3D vfio_pci_dmabuf.o obj-$(CONFIG_VFIO_PCI_CORE) +=3D vfio-pci-core.o diff --git a/drivers/vfio/pci/cxl/vfio_cxl_config.c b/drivers/vfio/pci/cxl/= vfio_cxl_config.c new file mode 100644 index 000000000000..a9560661345c --- /dev/null +++ b/drivers/vfio/pci/cxl/vfio_cxl_config.c @@ -0,0 +1,304 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * CXL DVSEC configuration space emulation for vfio-pci. + * + * Integrates into the existing vfio-pci-core ecap_perms[] framework using + * vdev->vconfig as the sole shadow buffer for DVSEC registers. + * + * Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved + */ + +#include +#include + +#include "../vfio_pci_priv.h" +#include "vfio_cxl_priv.h" + +/* Helpers to access vdev->vconfig at a DVSEC-relative offset */ +static inline u16 dvsec_virt_read16(struct vfio_pci_core_device *vdev, + u16 off) +{ + return le16_to_cpu(*(u16 *)(vdev->vconfig + + vdev->cxl->dvsec + off)); +} + +static inline void dvsec_virt_write16(struct vfio_pci_core_device *vdev, + u16 off, u16 val) +{ + *(u16 *)(vdev->vconfig + vdev->cxl->dvsec + off) =3D cpu_to_le16(val); +} + +static inline u32 dvsec_virt_read32(struct vfio_pci_core_device *vdev, + u16 off) +{ + return le32_to_cpu(*(u32 *)(vdev->vconfig + + vdev->cxl->dvsec + off)); +} + +static inline void dvsec_virt_write32(struct vfio_pci_core_device *vdev, + u16 off, u32 val) +{ + *(u32 *)(vdev->vconfig + vdev->cxl->dvsec + off) =3D cpu_to_le32(val); +} + +/* Individual DVSEC register write handlers */ + +static void cxl_control_write(struct vfio_pci_core_device *vdev, + u16 abs_off, u16 new_val) +{ + u16 lock =3D dvsec_virt_read16(vdev, CXL_DVSEC_LOCK_OFFSET); + u16 cap3 =3D dvsec_virt_read16(vdev, CXL_DVSEC_CAPABILITY3_OFFSET); + u16 rev_mask =3D CXL_CTRL_RESERVED_MASK; + + if (lock & CXL_CTRL_LOCK_BIT) + return; /* register is locked after first write */ + + if (!(cap3 & CXL_CAP3_P2P_BIT)) + rev_mask |=3D CXL_CTRL_P2P_REV_MASK; + + new_val &=3D ~rev_mask; + new_val |=3D CXL_CTRL_CXL_IO_ENABLE_BIT; /* CXL.io always enabled */ + + dvsec_virt_write16(vdev, CXL_DVSEC_CONTROL_OFFSET, new_val); +} + +static void cxl_status_write(struct vfio_pci_core_device *vdev, + u16 abs_off, u16 new_val) +{ + u16 cur_val =3D dvsec_virt_read16(vdev, CXL_DVSEC_STATUS_OFFSET); + + new_val &=3D ~CXL_STATUS_RESERVED_MASK; + + /* RW1C: writing a 1 clears the bit; writing 0 leaves it unchanged */ + if (new_val & CXL_STATUS_RW1C_BIT) + new_val &=3D ~CXL_STATUS_RW1C_BIT; + else + new_val =3D (new_val & ~CXL_STATUS_RW1C_BIT) | + (cur_val & CXL_STATUS_RW1C_BIT); + + dvsec_virt_write16(vdev, CXL_DVSEC_STATUS_OFFSET, new_val); +} + +static void cxl_control2_write(struct vfio_pci_core_device *vdev, + u16 abs_off, u16 new_val) +{ + struct pci_dev *pdev =3D vdev->pdev; + u16 cap2 =3D dvsec_virt_read16(vdev, CXL_DVSEC_CAPABILITY2_OFFSET); + u16 cap3 =3D dvsec_virt_read16(vdev, CXL_DVSEC_CAPABILITY3_OFFSET); + u16 rev_mask =3D CXL_CTRL2_RESERVED_MASK; + u16 hw_bits =3D CXL_CTRL2_HW_BITS_MASK; + bool initiate_cxl_reset =3D new_val & CXL_CTRL2_INITIATE_CXL_RESET_BIT; + + if (!(cap3 & CXL_CAP3_VOLATILE_HDM_BIT)) + rev_mask |=3D CXL_CTRL2_VOLATILE_HDM_REV_MASK; + if (!(cap2 & CXL_CAP2_MODIFIED_COMPLETION_BIT)) + rev_mask |=3D CXL_CTRL2_MODIFIED_COMP_REV_MASK; + + new_val &=3D ~rev_mask; + + /* Bits that go directly to hardware */ + hw_bits &=3D new_val; + + dvsec_virt_write16(vdev, CXL_DVSEC_CONTROL2_OFFSET, new_val); + + if (hw_bits) + pci_write_config_word(pdev, abs_off, hw_bits); + + if (initiate_cxl_reset) { + /* TODO: invoke CXL protocol reset via cxl subsystem */ + dev_warn(&pdev->dev, "vfio-cxl: CXL reset requested but not yet supporte= d\n"); + } +} + +static void cxl_status2_write(struct vfio_pci_core_device *vdev, + u16 abs_off, u16 new_val) +{ + u16 cap3 =3D dvsec_virt_read16(vdev, CXL_DVSEC_CAPABILITY3_OFFSET); + + /* RW1CS: write 1 to clear, but only if the capability is supported */ + if ((cap3 & CXL_CAP3_VOLATILE_HDM_BIT) && + (new_val & CXL_STATUS2_RW1CS_BIT)) + pci_write_config_word(vdev->pdev, abs_off, + CXL_STATUS2_RW1CS_BIT); + /* STATUS2 is not mirrored in vconfig - reads go to hardware */ +} + +static void cxl_lock_write(struct vfio_pci_core_device *vdev, + u16 abs_off, u16 new_val) +{ + u16 cur_val =3D dvsec_virt_read16(vdev, CXL_DVSEC_LOCK_OFFSET); + + /* Once the LOCK bit is set it can only be cleared by conventional reset = */ + if (cur_val & CXL_CTRL_LOCK_BIT) + return; + + new_val &=3D ~CXL_LOCK_RESERVED_MASK; + dvsec_virt_write16(vdev, CXL_DVSEC_LOCK_OFFSET, new_val); +} + +static void cxl_range_base_lo_write(struct vfio_pci_core_device *vdev, + u16 dvsec_off, u32 new_val) +{ + new_val &=3D ~CXL_BASE_LO_RESERVED_MASK; + dvsec_virt_write32(vdev, dvsec_off, new_val); +} + +/* + * vfio_cxl_dvsec_readfn - per-device DVSEC read handler. + * + * Called via vfio_pci_dvsec_dispatch_read() for devices that have registe= red + * a dvsec_readfn. Returns shadow vconfig values for virtualized DVSEC + * registers (CONTROL, STATUS, CONTROL2, LOCK) so that userspace reads ref= lect + * the emulated state rather than the raw hardware value. All other DVSEC + * bytes are passed through to hardware via vfio_raw_config_read(). + */ +static int vfio_cxl_dvsec_readfn(struct vfio_pci_core_device *vdev, + int pos, int count, + struct perm_bits *perm, + int offset, __le32 *val) +{ + struct vfio_pci_cxl_state *cxl =3D vdev->cxl; + u16 dvsec_off; + + if (!cxl || (u16)pos < cxl->dvsec || + (u16)pos >=3D cxl->dvsec + cxl->dvsec_length) + return vfio_raw_config_read(vdev, pos, count, perm, offset, val); + + dvsec_off =3D (u16)pos - cxl->dvsec; + + switch (dvsec_off) { + case CXL_DVSEC_CONTROL_OFFSET: + case CXL_DVSEC_STATUS_OFFSET: + case CXL_DVSEC_CONTROL2_OFFSET: + case CXL_DVSEC_LOCK_OFFSET: + /* Return shadow vconfig value for virtualized registers */ + memcpy(val, vdev->vconfig + pos, count); + return count; + default: + return vfio_raw_config_read(vdev, pos, count, + perm, offset, val); + } +} + +/* + * vfio_cxl_dvsec_writefn - ecap_perms write handler for PCI_EXT_CAP_ID_DV= SEC. + * + * Installed once into ecap_perms[PCI_EXT_CAP_ID_DVSEC].writefn by + * vfio_pci_init_perm_bits() when CONFIG_VFIO_CXL_CORE=3Dy. Applies to ev= ery + * device opened under vfio-pci; the vdev->cxl NULL check distinguishes CXL + * devices from non-CXL devices that happen to expose a DVSEC capability. + * + * @pos: absolute byte position in config space + * @offset: byte offset within the capability structure + */ +static int vfio_cxl_dvsec_writefn(struct vfio_pci_core_device *vdev, + int pos, int count, + struct perm_bits *perm, + int offset, __le32 val) +{ + struct vfio_pci_cxl_state *cxl =3D vdev->cxl; + u16 abs_off =3D (u16)pos; + u16 dvsec_off; + u16 wval16; + u32 wval32; + + if (!cxl || (u16)pos < cxl->dvsec || + (u16)pos >=3D cxl->dvsec + cxl->dvsec_length) + return vfio_raw_config_write(vdev, pos, count, perm, + offset, val); + + pci_dbg(vdev->pdev, + "vfio_cxl: DVSEC write: abs=3D0x%04x dvsec_off=3D0x%04x " + "count=3D%d raw_val=3D0x%08x\n", + abs_off, abs_off - cxl->dvsec, count, le32_to_cpu(val)); + + dvsec_off =3D abs_off - cxl->dvsec; + + /* Route to the appropriate per-register handler */ + switch (dvsec_off) { + case CXL_DVSEC_CONTROL_OFFSET: + wval16 =3D (u16)le32_to_cpu(val); + cxl_control_write(vdev, abs_off, wval16); + break; + case CXL_DVSEC_STATUS_OFFSET: + wval16 =3D (u16)le32_to_cpu(val); + cxl_status_write(vdev, abs_off, wval16); + break; + case CXL_DVSEC_CONTROL2_OFFSET: + wval16 =3D (u16)le32_to_cpu(val); + cxl_control2_write(vdev, abs_off, wval16); + break; + case CXL_DVSEC_STATUS2_OFFSET: + wval16 =3D (u16)le32_to_cpu(val); + cxl_status2_write(vdev, abs_off, wval16); + break; + case CXL_DVSEC_LOCK_OFFSET: + wval16 =3D (u16)le32_to_cpu(val); + cxl_lock_write(vdev, abs_off, wval16); + break; + case CXL_DVSEC_RANGE1_BASE_HIGH_OFFSET: + case CXL_DVSEC_RANGE2_BASE_HIGH_OFFSET: + wval32 =3D le32_to_cpu(val); + dvsec_virt_write32(vdev, dvsec_off, wval32); + break; + case CXL_DVSEC_RANGE1_BASE_LOW_OFFSET: + case CXL_DVSEC_RANGE2_BASE_LOW_OFFSET: + wval32 =3D le32_to_cpu(val); + cxl_range_base_lo_write(vdev, dvsec_off, wval32); + break; + default: + /* RO registers: header, capability, range sizes - discard */ + break; + } + + return count; +} + +/* + * vfio_cxl_setup_dvsec_perms - Install per-device CXL DVSEC read/write ho= oks. + * + * Called once per device open after vfio_config_init() has seeded vdev->v= config + * from hardware. Registers vfio_cxl_dvsec_readfn and vfio_cxl_dvsec_writ= efn + * as the per-device DVSEC handlers. The global dispatch functions instal= led + * in ecap_perms[PCI_EXT_CAP_ID_DVSEC] at module init call these per-device + * hooks so that pci_config_map bytes remain PCI_EXT_CAP_ID_DVSEC througho= ut. + * + * vfio_cxl_dvsec_readfn: returns vconfig shadow for CONTROL/STATUS/CONTRO= L2/ + * LOCK; passes all other DVSEC bytes through to hardware. + * vfio_cxl_dvsec_writefn: enforces per-register semantics (RW1C, forced + * IO_ENABLE, reserved-bit masking) and stores results in vconfig. + * + * Also forces CXL.io IO_ENABLE in the CONTROL vconfig shadow so the initi= al + * read returns 1 even before the first write. + */ +void vfio_cxl_setup_dvsec_perms(struct vfio_pci_core_device *vdev) +{ + u16 ctrl =3D dvsec_virt_read16(vdev, CXL_DVSEC_CONTROL_OFFSET); + + /* + * Register per-device DVSEC read/write handlers. The global + * ecap_perms[PCI_EXT_CAP_ID_DVSEC] dispatchers will call them. + * + * vfio_cxl_dvsec_readfn returns vconfig shadow values for the + * virtualized registers (CONTROL, STATUS, CONTROL2, LOCK) so that + * reads reflect emulated state rather than raw hardware. + * + * vfio_cxl_dvsec_writefn enforces per-register semantics (RW1C, + * forced IO_ENABLE, reserved-bit masking) and stores results in + * vconfig. Because ecap_perms[DVSEC].writefn dispatches to this + * handler, the pci_config_map bytes remain as PCI_EXT_CAP_ID_DVSEC + * _ no PCI_CAP_ID_INVALID_VIRT marking is needed or wanted. + */ + vdev->dvsec_readfn =3D vfio_cxl_dvsec_readfn; + vdev->dvsec_writefn =3D vfio_cxl_dvsec_writefn; + + /* + * vconfig is seeded from hardware at open time. Force IO_ENABLE set + * in the CONTROL shadow so the initial read returns 1 even if the + * hardware reset value has it cleared. Subsequent writes are handled + * by cxl_control_write() which also forces this bit. + */ + ctrl |=3D CXL_CTRL_CXL_IO_ENABLE_BIT; + dvsec_virt_write16(vdev, CXL_DVSEC_CONTROL_OFFSET, ctrl); +} +EXPORT_SYMBOL_GPL(vfio_cxl_setup_dvsec_perms); diff --git a/drivers/vfio/pci/cxl/vfio_cxl_core.c b/drivers/vfio/pci/cxl/vf= io_cxl_core.c index 15b6c0d75d9e..e18e992800f6 100644 --- a/drivers/vfio/pci/cxl/vfio_cxl_core.c +++ b/drivers/vfio/pci/cxl/vfio_cxl_core.c @@ -26,6 +26,7 @@ static int vfio_cxl_create_device_state(struct vfio_pci_c= ore_device *vdev, struct vfio_pci_cxl_state *cxl; bool cxl_mem_capable, is_cxl_type3; u16 cap_word; + u32 hdr1; =20 /* * The devm allocation for the CXL state remains for the entire time @@ -47,6 +48,9 @@ static int vfio_cxl_create_device_state(struct vfio_pci_c= ore_device *vdev, cxl->dpa_region_idx =3D -1; cxl->comp_reg_region_idx =3D -1; =20 + pci_read_config_dword(pdev, dvsec + PCI_DVSEC_HEADER1, &hdr1); + cxl->dvsec_length =3D PCI_DVSEC_HEADER1_LEN(hdr1); + pci_read_config_word(pdev, dvsec + CXL_DVSEC_CAPABILITY_OFFSET, &cap_word); =20 diff --git a/drivers/vfio/pci/cxl/vfio_cxl_priv.h b/drivers/vfio/pci/cxl/vf= io_cxl_priv.h index 3ef8d923a7e8..158fe4e67f98 100644 --- a/drivers/vfio/pci/cxl/vfio_cxl_priv.h +++ b/drivers/vfio/pci/cxl/vfio_cxl_priv.h @@ -31,6 +31,7 @@ struct vfio_pci_cxl_state { u32 hdm_count; int dpa_region_idx; int comp_reg_region_idx; + size_t dvsec_length; u16 dvsec; u8 comp_reg_bar; bool precommitted; @@ -76,9 +77,44 @@ struct vfio_pci_cxl_state { * (CXL 2.0+ 8.1.3). * Offsets are relative to the DVSEC capability base (cxl->dvsec). */ -#define CXL_DVSEC_CAPABILITY_OFFSET 0xa +#define CXL_DVSEC_CAPABILITY_OFFSET 0xa +#define CXL_DVSEC_CONTROL_OFFSET 0xc +#define CXL_DVSEC_STATUS_OFFSET 0xe +#define CXL_DVSEC_CONTROL2_OFFSET 0x10 +#define CXL_DVSEC_STATUS2_OFFSET 0x12 +#define CXL_DVSEC_LOCK_OFFSET 0x14 +#define CXL_DVSEC_CAPABILITY2_OFFSET 0x16 +#define CXL_DVSEC_RANGE1_SIZE_HIGH_OFFSET 0x18 +#define CXL_DVSEC_RANGE1_SIZE_LOW_OFFSET 0x1c +#define CXL_DVSEC_RANGE1_BASE_HIGH_OFFSET 0x20 +#define CXL_DVSEC_RANGE1_BASE_LOW_OFFSET 0x24 +#define CXL_DVSEC_RANGE2_SIZE_HIGH_OFFSET 0x28 +#define CXL_DVSEC_RANGE2_SIZE_LOW_OFFSET 0x2c +#define CXL_DVSEC_RANGE2_BASE_HIGH_OFFSET 0x30 +#define CXL_DVSEC_RANGE2_BASE_LOW_OFFSET 0x34 +#define CXL_DVSEC_CAPABILITY3_OFFSET 0x38 + #define CXL_DVSEC_MEM_CAPABLE BIT(2) =20 +/* CXL Control / Status / Lock - bit definitions */ +#define CXL_CTRL_LOCK_BIT BIT(0) +#define CXL_CTRL_CXL_IO_ENABLE_BIT BIT(1) +#define CXL_CTRL2_INITIATE_CXL_RESET_BIT BIT(2) +#define CXL_CAP3_VOLATILE_HDM_BIT BIT(3) +#define CXL_STATUS2_RW1CS_BIT BIT(3) +#define CXL_CAP3_P2P_BIT BIT(4) +#define CXL_CAP2_MODIFIED_COMPLETION_BIT BIT(6) +#define CXL_STATUS_RW1C_BIT BIT(14) +#define CXL_CTRL_RESERVED_MASK (BIT(13) | BIT(15)) +#define CXL_CTRL_P2P_REV_MASK BIT(12) +#define CXL_STATUS_RESERVED_MASK (GENMASK(13, 0) | BIT(15)) +#define CXL_CTRL2_RESERVED_MASK GENMASK(15, 6) +#define CXL_CTRL2_HW_BITS_MASK (BIT(0) | BIT(1) | BIT(3)) +#define CXL_CTRL2_VOLATILE_HDM_REV_MASK BIT(4) +#define CXL_CTRL2_MODIFIED_COMP_REV_MASK BIT(5) +#define CXL_LOCK_RESERVED_MASK GENMASK(15, 1) +#define CXL_BASE_LO_RESERVED_MASK GENMASK(27, 0) + int vfio_cxl_setup_virt_regs(struct vfio_pci_core_device *vdev); void vfio_cxl_clean_virt_regs(struct vfio_pci_core_device *vdev); void vfio_cxl_reinit_comp_regs(struct vfio_pci_core_device *vdev); diff --git a/drivers/vfio/pci/vfio_pci.c b/drivers/vfio/pci/vfio_pci.c index d3138badeaa6..22cf9ea831f9 100644 --- a/drivers/vfio/pci/vfio_pci.c +++ b/drivers/vfio/pci/vfio_pci.c @@ -121,12 +121,26 @@ static int vfio_pci_open_device(struct vfio_device *c= ore_vdev) } =20 if (vdev->cxl) { + /* + * pci_config_map and vconfig are valid now (allocated by + * vfio_config_init() inside vfio_pci_core_enable() above). + */ + vfio_cxl_setup_dvsec_perms(vdev); + ret =3D vfio_cxl_register_cxl_region(vdev); if (ret) { pci_warn(pdev, "Failed to setup CXL region\n"); vfio_pci_core_disable(vdev); return ret; } + + ret =3D vfio_cxl_register_comp_regs_region(vdev); + if (ret) { + pci_warn(pdev, "Failed to register COMP_REGS region\n"); + vfio_cxl_unregister_cxl_region(vdev); + vfio_pci_core_disable(vdev); + return ret; + } } =20 vfio_pci_core_finish_enable(vdev); diff --git a/drivers/vfio/pci/vfio_pci_config.c b/drivers/vfio/pci/vfio_pci= _config.c index 79aaf270adb2..90e2c25381d6 100644 --- a/drivers/vfio/pci/vfio_pci_config.c +++ b/drivers/vfio/pci/vfio_pci_config.c @@ -1085,6 +1085,49 @@ static int __init init_pci_ext_cap_pwr_perm(struct p= erm_bits *perm) return 0; } =20 +/* + * vfio_pci_dvsec_dispatch_read - per-device DVSEC read dispatcher. + * + * Installed as ecap_perms[PCI_EXT_CAP_ID_DVSEC].readfn at module init. + * Calls vdev->dvsec_readfn when a shadow-read handler has been registered + * (e.g. by vfio_cxl_setup_dvsec_perms() for CXL Type-2 devices), otherwise + * falls through to vfio_raw_config_read for hardware pass-through. + * + * This indirection allows per-device DVSEC reads from vconfig shadow + * without touching the global ecap_perms[] table. + */ +static int vfio_pci_dvsec_dispatch_read(struct vfio_pci_core_device *vdev, + int pos, int count, + struct perm_bits *perm, + int offset, __le32 *val) +{ + if (vdev->dvsec_readfn) + return vdev->dvsec_readfn(vdev, pos, count, perm, offset, val); + return vfio_raw_config_read(vdev, pos, count, perm, offset, val); +} + +/* + * vfio_pci_dvsec_dispatch_write - per-device DVSEC write dispatcher. + * + * Installed as ecap_perms[PCI_EXT_CAP_ID_DVSEC].writefn at module init. + * Calls vdev->dvsec_writefn when a handler has been registered for this + * device (e.g. by vfio_cxl_setup_dvsec_perms() for CXL Type-2 devices), + * otherwise falls through to vfio_raw_config_write so that non-CXL + * devices with a DVSEC capability continue to pass writes to hardware. + * + * This indirection allows per-device DVSEC handlers to be registered + * without touching the global ecap_perms[] table. + */ +static int vfio_pci_dvsec_dispatch_write(struct vfio_pci_core_device *vdev, + int pos, int count, + struct perm_bits *perm, + int offset, __le32 val) +{ + if (vdev->dvsec_writefn) + return vdev->dvsec_writefn(vdev, pos, count, perm, offset, val); + return vfio_raw_config_write(vdev, pos, count, perm, offset, val); +} + /* * Initialize the shared permission tables */ @@ -1121,7 +1164,8 @@ int __init vfio_pci_init_perm_bits(void) ret |=3D init_pci_ext_cap_err_perm(&ecap_perms[PCI_EXT_CAP_ID_ERR]); ret |=3D init_pci_ext_cap_pwr_perm(&ecap_perms[PCI_EXT_CAP_ID_PWR]); ecap_perms[PCI_EXT_CAP_ID_VNDR].writefn =3D vfio_raw_config_write; - ecap_perms[PCI_EXT_CAP_ID_DVSEC].writefn =3D vfio_raw_config_write; + ecap_perms[PCI_EXT_CAP_ID_DVSEC].readfn =3D vfio_pci_dvsec_dispatch_read; + ecap_perms[PCI_EXT_CAP_ID_DVSEC].writefn =3D vfio_pci_dvsec_dispatch_writ= e; =20 if (ret) vfio_pci_uninit_perm_bits(); diff --git a/drivers/vfio/pci/vfio_pci_priv.h b/drivers/vfio/pci/vfio_pci_p= riv.h index f8db9a05c033..d778107fa908 100644 --- a/drivers/vfio/pci/vfio_pci_priv.h +++ b/drivers/vfio/pci/vfio_pci_priv.h @@ -154,6 +154,7 @@ void vfio_cxl_zap_region_locked(struct vfio_pci_core_de= vice *vdev); void vfio_cxl_reactivate_region(struct vfio_pci_core_device *vdev); int vfio_cxl_register_comp_regs_region(struct vfio_pci_core_device *vdev); void vfio_cxl_reinit_comp_regs(struct vfio_pci_core_device *vdev); +void vfio_cxl_setup_dvsec_perms(struct vfio_pci_core_device *vdev); =20 #else =20 @@ -180,6 +181,8 @@ vfio_cxl_register_comp_regs_region(struct vfio_pci_core= _device *vdev) { return 0; } static inline void vfio_cxl_reinit_comp_regs(struct vfio_pci_core_device *vdev) { } +static inline void +vfio_cxl_setup_dvsec_perms(struct vfio_pci_core_device *vdev) { } =20 #endif /* CONFIG_VFIO_CXL_CORE */ =20 diff --git a/include/linux/vfio_pci_core.h b/include/linux/vfio_pci_core.h index cd8ed98a82a3..aa159d0c8da7 100644 --- a/include/linux/vfio_pci_core.h +++ b/include/linux/vfio_pci_core.h @@ -31,7 +31,7 @@ struct p2pdma_provider; struct dma_buf_phys_vec; struct dma_buf_attachment; struct vfio_pci_cxl_state; - +struct perm_bits; =20 struct vfio_pci_eventfd { struct eventfd_ctx *ctx; @@ -141,6 +141,12 @@ struct vfio_pci_core_device { struct list_head ioeventfds_list; struct vfio_pci_vf_token *vf_token; struct vfio_pci_cxl_state *cxl; + int (*dvsec_readfn)(struct vfio_pci_core_device *vdev, int pos, + int count, struct perm_bits *perm, + int offset, __le32 *val); + int (*dvsec_writefn)(struct vfio_pci_core_device *vdev, int pos, + int count, struct perm_bits *perm, + int offset, __le32 val); struct list_head sriov_pfs_item; struct vfio_pci_core_device *sriov_pf_core_dev; struct notifier_block nb; --=20 2.25.1 From nobody Tue Apr 7 19:54:44 2026 Received: from SA9PR02CU001.outbound.protection.outlook.com (mail-southcentralusazon11013063.outbound.protection.outlook.com [40.93.196.63]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B848538D6B1; Wed, 11 Mar 2026 20:37:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.93.196.63 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773261450; cv=fail; b=C4xyK0jFXNqQJaf2GzM9P3p8F5mVES72+Ytmp7yAbckJsJ9/ysLuQT2THUgRJyvi32986J0BeqsQ/nuH4y2QMoOJVi2GhG72u8XHdjF8MPu4s02H+psUunJG7dFZ+C0vim7WFKKbuaOBmkZfRwq5ANTLqNM1VSCyjK17pQq7Bi4= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773261450; c=relaxed/simple; bh=rXGMc6Sr/cIZ9IBFx7m2ApfDX5EVV+BVU0aQyT4qpBs=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=qHTW1SeHTu38O4++bY7sv/gOlSZ+9VPNRbTACgS9rl6ZI1YPtjQTQrcqy8j2BKkUTmZUdzeShCxHGiX13wHwkurG98zavgX8QFsgFaHaG7hhAxB/II1R3ehkf5AcnqgM/+S/07ghZowg5Uzs5gPhpv96Mc+DCCTrKTYk4molo20= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=m8xTTkhQ; arc=fail smtp.client-ip=40.93.196.63 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="m8xTTkhQ" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=jRZ9z9Q3LD995HteLyQHu0yz7SRHh8AwamjmrrDnoQTHVIwanjQwCn9jRiyOeLwqSjrAzvAmJKNN9sZu8i+Xyh1Si8WT3UBFF92q0SiZsa88ju3OKk4khT6KPkSw/VNg3+rmVjI7c7/l3w1xq5FZB6ctivuEVwiiZfhVuc2KFyzZYlby2FRhiifDy14PJ1zDlGKQ2fXkffCyYf0Mj7CCNKE+ff7lf/W6DHgG1rSmq9NfbMRpaD2OmEHizyEclCmzU+Bbbfd5szkQRtc08bI7pGNonaNAnTNzx9SeyWxnCNlq7+BCVuDEwmo9gfBDNfM6Pi6RC0jDoVN1hln/I+iRKQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=z7lJoS4Cw9QYqXlHAy9v6/zR2Ngn7a6lj9xyG318lo4=; b=aC88qw1goVnZrNXGIB9HPCHCdxO9bHcW1WFPcneTkIaw5ArhHJsUfwrOubMBmYAmui2CMfeY+o8q7ob2SyfYmOGKMZ0JOmzhZDGE4WMapwB3eEHxa5qr9w5+dtC+f4k2WYzOY3WFht0bwqJAeaV4UomMjxQvPqG4WBDJs88EEm3Zv9IDox3+S1VO20wGS9RDCjhOKTSRVAnrR/AjdYcmWvTqHr0/Pef7nHMH750DbwWOTrtT9Z4NtH901VwH/qx4CIjqZL+TNCvTO324+8xCO4MWYc2HhDo3luXpWUrhCo7bO+CnJ5MNqPkRdwEr0y+s/ZDuZolAIawRr4zAkeEtdA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=z7lJoS4Cw9QYqXlHAy9v6/zR2Ngn7a6lj9xyG318lo4=; b=m8xTTkhQvD3I0lNqurGn7l4wdjqBrKRxhRtfsk/1usbZrCNXMFgWe98NsqfRZklD2UzkQQyUgCT5bI7+38xvX/hpTZrgnW5JCzaX6d7KsqqyYTkELsMV3g9kVpGK1B6LE+j290e16oD+IvIKgvw7AwxWjlH1FV3Q2x00WEdgU56f74twKBKWnifa/Ev7/VdIlUo+WZxImEUqZSDLw59JJAh8QadYfycUc/oii1LQINjmnandpmLZisMpK2bU5c4QssEMuUBf73z6CCqCJ0qlVSdwUuBx/wOTbZAOJHUja+3RRCv6HA7FvxCDr7gju3PLX0lrRu9DgdTRedYk9X+gig== Received: from CH2PR08CA0029.namprd08.prod.outlook.com (2603:10b6:610:5a::39) by IA0PR12MB8304.namprd12.prod.outlook.com (2603:10b6:208:3dc::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9700.11; Wed, 11 Mar 2026 20:37:23 +0000 Received: from DS2PEPF00003443.namprd04.prod.outlook.com (2603:10b6:610:5a:cafe::5f) by CH2PR08CA0029.outlook.office365.com (2603:10b6:610:5a::39) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9678.25 via Frontend Transport; Wed, 11 Mar 2026 20:37:03 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by DS2PEPF00003443.mail.protection.outlook.com (10.167.17.70) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9678.18 via Frontend Transport; Wed, 11 Mar 2026 20:37:22 +0000 Received: from rnnvmail203.nvidia.com (10.129.68.9) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Wed, 11 Mar 2026 13:36:58 -0700 Received: from rnnvmail201.nvidia.com (10.129.68.8) by rnnvmail203.nvidia.com (10.129.68.9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Wed, 11 Mar 2026 13:36:57 -0700 Received: from nvidia-4028GR-scsim.nvidia.com (10.127.8.11) by mail.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20 via Frontend Transport; Wed, 11 Mar 2026 13:36:50 -0700 From: To: , , , , , , , , , , , , , , , , , CC: , , , , , , , Subject: [PATCH 16/20] vfio/pci: Expose CXL device and region info via VFIO ioctl Date: Thu, 12 Mar 2026 02:04:36 +0530 Message-ID: <20260311203440.752648-17-mhonap@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20260311203440.752648-1-mhonap@nvidia.com> References: <20260311203440.752648-1-mhonap@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS2PEPF00003443:EE_|IA0PR12MB8304:EE_ X-MS-Office365-Filtering-Correlation-Id: 210ccfc9-7ad6-43c2-e28c-08de7fadffbe X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|36860700016|376014|7416014|82310400026|921020|22082099003|18002099003|56012099003; X-Microsoft-Antispam-Message-Info: 87x/eIrfjvFXIkNszftOmB1qX//eGlXbEI3Iz1gZlmlbtXAayT/iu95y/pRukLiMgI8tricwdp3LYbZPmKvxqjEPqDJxc9yh66gcZZi0kNuTu6Z+5aXZZK1unheiV45MfFBjm3fO1S0h7ffFJb1isFmuC3KZU0qnASsxnUWzODBLM8mGXWT5MyF9pScq7zlA04djE8KXV0m76qLoy3yw0S7K/y2Fr9V/JzT+1RlWnynjVgvvCnycy7yA7PtwSm3l+vaJo0nirrFQ5bUjLe73J3C6v8O2ceTRhZPUlmaZsk07Fk+iIeCCQkjaQ/SPdWl6VFFXjrqd6x4f/UkDZYgKp1JzAC6nMHLQS5Io3f40xgPj2Xqn7lxePfuwQjJ+zra7UBDXngF2UfQEtyR2BfAwnbPs7NoAqv3qKO2LGk9/NMcSS8Vwduj9nz6U6tGOn+O8kL2H0lbk4GHV/KVJWz3BS/jhwC60r+JPFYdI9MNV8Jl6A0OOHFiV9ppHx6RlNlRg4/QVsdnAr0o32Om4G0ZgAVpACOYp0pgeqj/xTrG0YY5LkToBRjmWH93lyMI6IMGJIJdZRcSqeckeu/QVm1qeR1OOC7ZrvNR8YAjjNkDm5ZxBgRj9TvHJsaAqTne1J9Qx31u8t8XZuNrlua75gShbQEIWvMitufcS2KH1MwiCjuSd9vpyyB8s8moALcV6+AmkbC/80chR9nUv1TTx6M+4X9iM+dNWSGpCJ8e/3hJX/K1BL3yCz5OtQq4/GU/9sF71wKyH90g9CT/wqGCIv5e/3pQC8Yd9uTMyWaG22g2hdbk57VdxNn6ZVZ8GBnN7FZjW X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(1800799024)(36860700016)(376014)(7416014)(82310400026)(921020)(22082099003)(18002099003)(56012099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: XNFNniGyr5ns0p6AMM8vbHqebOLuOLQwS4PfzX6JP64IeRXk5fnQh3sklw+bt149v2Ne/ObGeeevbAnFIoLmiyMmSj5TDcXmcrENBzaisunPqGRKRRI4CN7TlIpDZ8zpwSkPlRv+iarrkNkltKqecPyM81pqgpYKi4GH3Q2W/OEwh34Uj7ZypUhg1P3J1bRULyuo3Y03oikbUOVzEy8yGxgSiWjQnhqw7BBFSD0Hc41Boc7Mn2U01/zS0loCLhphlxuZpUTV1TMl7iYhyi9lVEyH1ey45J5hGN6blG63Z9CkYNAFkFI8unCwoGdXKCijb0poGom3yGtMxLSo21Z2o/MPNTtIm6xomAbgW64TRQMQFjCb6S2Y13FnOqVB+DsHc1pTZ5XXQj9bognsVOKaV6FYduJbM/bt/wDmFf9Gb6wnjMAmTkDOAcG3wIOpxCKm X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Mar 2026 20:37:22.8419 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 210ccfc9-7ad6-43c2-e28c-08de7fadffbe X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DS2PEPF00003443.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA0PR12MB8304 Content-Type: text/plain; charset="utf-8" From: Manish Honap Expose CXL device capability information through the VFIO device info ioctl and hide the CXL component BAR from direct userspace access via the standard region info path. Add vfio_cxl_get_info() which fills a VFIO_DEVICE_INFO_CAP_CXL capability structure with HDM register location, DPA size, commit flags, and the region indices of the two CXL VFIO device regions (DPA and COMP_REGS) so userspace does not need to scan all regions. Add vfio_cxl_get_region_info() which intercepts BAR queries for the component register BAR and returns size=3D0 to hide it, directing userspace to use VFIO_REGION_SUBTYPE_CXL_COMP_REGS instead. Hook both helpers into vfio_pci_ioctl_get_info() and vfio_pci_ioctl_get_region_info() in vfio_pci_core.c. The CXL component register BAR contains the HDM decoder MMIO registers. Userspace must use the VFIO_REGION_SUBTYPE_CXL_COMP_REGS emulated region instead of directly mapping or reading/writing this BAR, to ensure that all accesses go through the emulation layer for correct bit-field enforcement. Reject mmap(), barmap setup, and BAR r/w for the CXL component BAR index in vfio_pci_core_mmap(), vfio_pci_core_setup_barmap(), and vfio_pci_bar_rw() respectively. Signed-off-by: Zhi Wang Signed-off-by: Manish Honap --- drivers/vfio/pci/cxl/vfio_cxl_core.c | 84 ++++++++++++++++++++++++++++ drivers/vfio/pci/vfio_pci_core.c | 16 ++++++ drivers/vfio/pci/vfio_pci_priv.h | 19 +++++++ drivers/vfio/pci/vfio_pci_rdwr.c | 8 +++ 4 files changed, 127 insertions(+) diff --git a/drivers/vfio/pci/cxl/vfio_cxl_core.c b/drivers/vfio/pci/cxl/vf= io_cxl_core.c index e18e992800f6..bda11f99746f 100644 --- a/drivers/vfio/pci/cxl/vfio_cxl_core.c +++ b/drivers/vfio/pci/cxl/vfio_cxl_core.c @@ -18,6 +18,90 @@ =20 MODULE_IMPORT_NS("CXL"); =20 +u8 vfio_cxl_get_component_reg_bar(struct vfio_pci_core_device *vdev) +{ + return vdev->cxl->comp_reg_bar; +} + +int vfio_cxl_get_region_info(struct vfio_pci_core_device *vdev, + struct vfio_region_info *info, + struct vfio_info_cap *caps) +{ + unsigned long minsz =3D offsetofend(struct vfio_region_info, offset); + struct vfio_pci_cxl_state *cxl =3D vdev->cxl; + + if (!cxl) + return -ENOTTY; + + if (!info) + return -ENOTTY; + + if (info->argsz < minsz) + return -EINVAL; + + if (info->index !=3D cxl->comp_reg_bar) + return -ENOTTY; + + /* + * Hide the component BAR for CXL. Report size 0 so userspace + * uses only the VFIO_REGION_SUBTYPE_CXL_COMP_REGS device region + * for BAR MMIO (HDM) emulation. + */ + info->argsz =3D sizeof(*info); + info->offset =3D VFIO_PCI_INDEX_TO_OFFSET(info->index); + info->size =3D 0; + info->flags =3D 0; + info->cap_offset =3D 0; + + return 0; +} + +int vfio_cxl_get_info(struct vfio_pci_core_device *vdev, + struct vfio_info_cap *caps) +{ + struct vfio_pci_cxl_state *cxl =3D vdev->cxl; + struct vfio_device_info_cap_cxl cxl_cap =3D {0}; + + if (!cxl) + return 0; + + /* + * Region indices are set at open time after + * vfio_pci_core_register_dev_region() succeeds. If either is still + * -1, the device is not yet fully initialised; return EAGAIN so + * userspace knows to retry rather than receiving 0xFFFFFFFF. + */ + if (cxl->dpa_region_idx < 0 || cxl->comp_reg_region_idx < 0) + return -EAGAIN; + + /* Fill in from CXL device structure */ + cxl_cap.header.id =3D VFIO_DEVICE_INFO_CAP_CXL; + cxl_cap.header.version =3D 1; + cxl_cap.hdm_count =3D cxl->hdm_count; + cxl_cap.hdm_regs_offset =3D cxl->comp_reg_offset + cxl->hdm_reg_offset; + cxl_cap.hdm_regs_size =3D cxl->hdm_reg_size; + cxl_cap.hdm_regs_bar_index =3D cxl->comp_reg_bar; + cxl_cap.dpa_size =3D cxl->dpa_size; + + if (cxl->precommitted) { + cxl_cap.flags |=3D VFIO_CXL_CAP_COMMITTED | + VFIO_CXL_CAP_PRECOMMITTED; + } + + /* + * Populate absolute VFIO region indices so userspace can query th= em + * directly with VFIO_DEVICE_GET_REGION_INFO. Custom device regio= ns + * live at VFIO_PCI_NUM_REGIONS + local_idx (see vfio_pci_core.c:9= 99). + * dpa_region_idx / comp_reg_region_idx are 0-based local indices,= so + * add VFIO_PCI_NUM_REGIONS to get the index VFIO_DEVICE_GET_REGIO= N_INFO + * expects. + */ + cxl_cap.dpa_region_index =3D VFIO_PCI_NUM_REGIONS + cxl->dpa_regio= n_idx; + cxl_cap.comp_regs_region_index =3D VFIO_PCI_NUM_REGIONS + cxl->com= p_reg_region_idx; + + return vfio_info_add_capability(caps, &cxl_cap.header, sizeof(cxl_cap)); +} + static int vfio_cxl_create_device_state(struct vfio_pci_core_device *vdev, u16 dvsec) { diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_c= ore.c index 48e0274c19aa..5352e7810fed 100644 --- a/drivers/vfio/pci/vfio_pci_core.c +++ b/drivers/vfio/pci/vfio_pci_core.c @@ -989,6 +989,13 @@ static int vfio_pci_ioctl_get_info(struct vfio_pci_cor= e_device *vdev, if (vdev->reset_works) info.flags |=3D VFIO_DEVICE_FLAGS_RESET; =20 + if (vdev->cxl) { + ret =3D vfio_cxl_get_info(vdev, &caps); + if (ret) + return ret; + info.flags |=3D VFIO_DEVICE_FLAGS_CXL; + } + info.num_regions =3D VFIO_PCI_NUM_REGIONS + vdev->num_regions; info.num_irqs =3D VFIO_PCI_NUM_IRQS; =20 @@ -1034,6 +1041,12 @@ int vfio_pci_ioctl_get_region_info(struct vfio_devic= e *core_vdev, struct pci_dev *pdev =3D vdev->pdev; int i, ret; =20 + if (vdev->cxl) { + ret =3D vfio_cxl_get_region_info(vdev, info, caps); + if (ret !=3D -ENOTTY) + return ret; + } + switch (info->index) { case VFIO_PCI_CONFIG_REGION_INDEX: info->offset =3D VFIO_PCI_INDEX_TO_OFFSET(info->index); @@ -1756,6 +1769,9 @@ int vfio_pci_core_mmap(struct vfio_device *core_vdev,= struct vm_area_struct *vma } if (index >=3D VFIO_PCI_ROM_REGION_INDEX) return -EINVAL; + /* Reject mmap of CXL component BAR; use COMP_REGS region only. */ + if (vdev->cxl && index =3D=3D vfio_cxl_get_component_reg_bar(vdev)) + return -EINVAL; if (!vdev->bar_mmap_supported[index]) return -EINVAL; =20 diff --git a/drivers/vfio/pci/vfio_pci_priv.h b/drivers/vfio/pci/vfio_pci_p= riv.h index d778107fa908..c1befe7d028d 100644 --- a/drivers/vfio/pci/vfio_pci_priv.h +++ b/drivers/vfio/pci/vfio_pci_priv.h @@ -156,6 +156,13 @@ int vfio_cxl_register_comp_regs_region(struct vfio_pc= i_core_device *vdev); void vfio_cxl_reinit_comp_regs(struct vfio_pci_core_device *vdev); void vfio_cxl_setup_dvsec_perms(struct vfio_pci_core_device *vdev); =20 +int vfio_cxl_get_info(struct vfio_pci_core_device *vdev, + struct vfio_info_cap *caps); +int vfio_cxl_get_region_info(struct vfio_pci_core_device *vdev, + struct vfio_region_info *info, + struct vfio_info_cap *caps); +u8 vfio_cxl_get_component_reg_bar(struct vfio_pci_core_device *vdev); + #else =20 static inline void @@ -183,6 +190,18 @@ static inline void vfio_cxl_reinit_comp_regs(struct vfio_pci_core_device *vdev) { } static inline void vfio_cxl_setup_dvsec_perms(struct vfio_pci_core_device *vdev) { } +static inline int +vfio_cxl_get_info(struct vfio_pci_core_device *vdev, + struct vfio_info_cap *caps) +{ return -ENOTTY; } +static inline int +vfio_cxl_get_region_info(struct vfio_pci_core_device *vdev, + struct vfio_region_info *info, + struct vfio_info_cap *caps) +{ return -ENOTTY; } +static inline u8 +vfio_cxl_get_component_reg_bar(struct vfio_pci_core_device *vdev) +{ return U8_MAX; } =20 #endif /* CONFIG_VFIO_CXL_CORE */ =20 diff --git a/drivers/vfio/pci/vfio_pci_rdwr.c b/drivers/vfio/pci/vfio_pci_r= dwr.c index b38627b35c35..4f1f4882265a 100644 --- a/drivers/vfio/pci/vfio_pci_rdwr.c +++ b/drivers/vfio/pci/vfio_pci_rdwr.c @@ -207,6 +207,10 @@ int vfio_pci_core_setup_barmap(struct vfio_pci_core_de= vice *vdev, int bar) if (vdev->barmap[bar]) return 0; =20 + /* Do not map the CXL component BAR; use COMP_REGS region only. */ + if (vdev->cxl && bar =3D=3D vfio_cxl_get_component_reg_bar(vdev)) + return -EINVAL; + ret =3D pci_request_selected_regions(pdev, 1 << bar, "vfio"); if (ret) return ret; @@ -236,6 +240,10 @@ ssize_t vfio_pci_bar_rw(struct vfio_pci_core_device *v= dev, char __user *buf, ssize_t done; enum vfio_pci_io_width max_width =3D VFIO_PCI_IO_WIDTH_8; =20 + /* Reject BAR r/w for CXL component BAR; use COMP_REGS region only. */ + if (vdev->cxl && bar =3D=3D vfio_cxl_get_component_reg_bar(vdev)) + return -EINVAL; + if (pci_resource_start(pdev, bar)) end =3D pci_resource_len(pdev, bar); else if (bar =3D=3D PCI_ROM_RESOURCE && pdev->rom && pdev->romlen) --=20 2.25.1 From nobody Tue Apr 7 19:54:44 2026 Received: from SJ2PR03CU001.outbound.protection.outlook.com (mail-westusazon11012067.outbound.protection.outlook.com [52.101.43.67]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 86EC1367F4D; Wed, 11 Mar 2026 20:37:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.43.67 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773261458; cv=fail; b=OjSC6NRz7J3huMR37BWJwp8y+nL30bMkPdI+EdhWujseRmozdi6DzTWGKthlVCNOpHAENZiF0IgzWC8q0fGhRRLdobTf9wQIMOHDmFbDDewtNZE/GlboE4sw5dS0x0lDccPx4mWKKV8cxvlp2BL/dxa7U3ntstMnys5O13SBNXI= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773261458; c=relaxed/simple; bh=5SU0ahdaQkyXW5T5VEukJYi0M/85eh5BwodpNnwZMUA=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=dZ3MuGmAceusyqOoPQQfui0uZeWXL5qQHQBzL8fTMHbFWTpk4IErIhJCaawUpi7PzbSQPeVW3JvK0HaEkUd4a9vK5SKZmHSYQvetLibW04+tgD/glOw/VOnTAbqduGd2DRK4F4ko/O5tYV/ccZXpyk5UTJpdMiStdyS1CEVVv24= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=iJdVkZKL; arc=fail smtp.client-ip=52.101.43.67 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="iJdVkZKL" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=CF9nwdaS0+yK7YdCZikvzfFie4pz872gMV3Apmx/pVlSEtKfLn4QIfo/0FKz5wZl+bTZZQGOclTZJA5z6YjQLvTVbJaY7mK1Jtkm4v9yaThklhRcNHZNKyA2LiR+AdUzG2GKpt1ZUegvc+JrI/P78Bm+OBy0z+U4VrT9zR3qCIiTMBn6NJpXCIddKeSLZqA5eZq64x99asFa6Q/X3Q0uq7ZDdyY74tpp0vn9ln6o61N2WEIC4ACuA4RzlQIlsjpfPT8W4YrVW5dCRkbV/IfdTRZqwAgCcWEaJiPx0cW7gWXroZZJ0eG/cy5uI6sIXw+e3nrILaHO+AeoksreJWZ+8Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=CSFoev65w4Fm1q7D3J977J1dWOrpHXUhrfvR4c6AZQE=; b=qsKiOs/+khQcmcpF86tHf7P6sWHkICb2BiQfyOTp5o9ewBCJNN99k9wg83d3MUqpQLgwTeqe9AOpzeXqyYlDaPh4MtIqACo6MWBiK/meshl3YH1rw/cqUnjb9GPT0CdxF+Xh7CDGliPgUGeReCvFm+Jkmntcb0bpavUQCh7TvHuczAaAwOnNpF29TydR2uW2nVONw+MwLzn8OyBvyrdbxVH4gRkxMm8ToF7jZX7A53IqcuwRG0RTg4b/v9nA1BYnh0XvmTnGGWF3cNdRJgFzlZ1e1nPcYHllXTecJMiFo3P+X4ZdO42ncuHFb/WP4enY/tlkNNwNUnaNLjHisRvYtw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=CSFoev65w4Fm1q7D3J977J1dWOrpHXUhrfvR4c6AZQE=; b=iJdVkZKLJ4l8Gb30/b03EzuUkM0x4VRa2J8q9AwkYehdAFxKAqp7yGt1lD7gDfiJ3ZUZ9MBhFR5vWwTzz3WPYMITesO7l0CRQhC0gOklOOyXiY6G4HZ2Hf98+J/r4awYA4QYoGrcmhWMDu3mr9YgH7iDonXlqhxL45qJZOfwQ6vXDjrw9K/8AC+RJfVuD0NnNM4N2d4vPU1PS6249vMmiJn2lJ7kJpTQgDNVNt1VF2ks58H8eLec8hZoBZfg5FT4ERCW6wc8BAsZiWZItfPylOs4wVDUjcry6ylBWm046qUXMK7oYBIyUtWwgH82CuUidOk4raXFsnT3PLU+64q3oA== Received: from CH5P222CA0003.NAMP222.PROD.OUTLOOK.COM (2603:10b6:610:1ee::24) by MN2PR12MB4422.namprd12.prod.outlook.com (2603:10b6:208:265::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9700.11; Wed, 11 Mar 2026 20:37:29 +0000 Received: from DS2PEPF00003444.namprd04.prod.outlook.com (2603:10b6:610:1ee:cafe::3b) by CH5P222CA0003.outlook.office365.com (2603:10b6:610:1ee::24) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9678.27 via Frontend Transport; Wed, 11 Mar 2026 20:37:30 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by DS2PEPF00003444.mail.protection.outlook.com (10.167.17.71) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9678.18 via Frontend Transport; Wed, 11 Mar 2026 20:37:28 +0000 Received: from rnnvmail203.nvidia.com (10.129.68.9) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Wed, 11 Mar 2026 13:37:05 -0700 Received: from rnnvmail201.nvidia.com (10.129.68.8) by rnnvmail203.nvidia.com (10.129.68.9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Wed, 11 Mar 2026 13:37:05 -0700 Received: from nvidia-4028GR-scsim.nvidia.com (10.127.8.11) by mail.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20 via Frontend Transport; Wed, 11 Mar 2026 13:36:58 -0700 From: To: , , , , , , , , , , , , , , , , , CC: , , , , , , , Subject: [PATCH 17/20] vfio/cxl: Provide opt-out for CXL feature Date: Thu, 12 Mar 2026 02:04:37 +0530 Message-ID: <20260311203440.752648-18-mhonap@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20260311203440.752648-1-mhonap@nvidia.com> References: <20260311203440.752648-1-mhonap@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS2PEPF00003444:EE_|MN2PR12MB4422:EE_ X-MS-Office365-Filtering-Correlation-Id: 175a8db4-bc00-441d-90e3-08de7fae033a X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|82310400026|1800799024|36860700016|3122999024|921020|18002099003|56012099003|22082099003; X-Microsoft-Antispam-Message-Info: j5Sspkrt+fmHgt5tdkL2aIi9OwrHWk+2Ktr9avJSmr7lqhssjyZ5cd3PRjG35HmJ6+2IW1r6MfprXVczeZhiwBel8YQ9bGrm94YibZts45sq29sxlYJ10f/2kMF6CSjGRhB0MOIHYbZ7Xfx56WeDZrwHDYWI1atkDkqQ8bIGmyOMsdWprsQ/9QbSBk6fc0e45RQykPfdZ1ApPazlJ0wopHus/qcKnGqXjnuz7IUaV21G0iohyF+zsLBYEkW4tOQFqRdxbldlFy+Kwr53boxd/+b11iXXn8ruQG0JvGBJYaoVs/4m7v4DyLcUmgo/kKEtELfzJjR5QqLPQ3+AN/3fvMZlFBcMytdCC+v+D5vRjNm08ummTImc5ZeGvU1DCFMgkWA+/njldDc/cia/05PU5mvk1bMVuVoUxp0T6a5BzaISf4QDzjB1fwo1NW9rwdBAraPofACEWWSib3Zqe3AzD7BWCNXzukDaHD4rG/ckLXUmw5wbOg7a7Nl5mCvm4P2nQ+iE1ZyFQ76WuGNE7vMPhYN5eKBUkTcmnIFoQGipyHKfMQ+rWWgntcDrWni9OtiS+EL9RD0rv/bzkcYFf+CwS13XkX2Xifhg1FLE85f/yfoAvSnoqB96MgKRRw2u3Hqd6C0j/pZI0eBjRqzd9SyI40G3mJJGkqhLWgbawyg494s6Nj3Fq6y1/ywBlVUAU4JJqjdUC5Qa0DA2z9P+27WjJED0meGT8sdWzlAGfX+34RLYjRsN17su4d/3VwvBx/Nm42c37ZtJjkCrgN4I0rVhVBBeqBJIF6ENz9ya7NP9Y1F/EJlSzZaQV19K+oiMv/UW X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(7416014)(376014)(82310400026)(1800799024)(36860700016)(3122999024)(921020)(18002099003)(56012099003)(22082099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 2XThmRr4xZ3iXR63/l7TuLR9+Ov/YzEOqpbHRnekFrzbuAgyRuX4bWZTj4wuZiiuP5nh8nDlGW3jvMH04jesCMsMGz0ElTmTHgnytMSmy8QNTQI+42OGZdO/xwEeHwcs/ZKagQf14dNBLqyXHhmJfQGS3VwTqUkDw4Bn+m5o+b13UeGxkreFF4HH0iPeqIlCWyDmvWA9gcIFyXO4Owq3J7qJp8bp8YRwNcEUeLEx6qcv67lSN103zM5Q1sNB06WhUdkq3cPnTtKiEcZMJL0B44FiV2r1gP8GlwPofNSqxBocZR/yGNASZmQfTM3OldqGavnqyaWej9OxAzze4TjQ3HDhKnP1qgzI7LGQCT0h/jqA+LLS8lfT6RX5UNNPHcyOfV3291WbN9NAXW0aaLhowsuoQuIApfw0C8eokaxFCKEPrHqh1etsSAKA2VF4FVOB X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Mar 2026 20:37:28.6876 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 175a8db4-bc00-441d-90e3-08de7fae033a X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DS2PEPF00003444.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4422 Content-Type: text/plain; charset="utf-8" From: Manish Honap This commit provides an opt-out mechanism to disable the CXL support from vfio module. The opt-out is provided both build time and module load time. Build time option CONFIG_VFIO_CXL_CORE is used to enable/disable CXL support in vfio-pci module. For runtime disabling the CXL support, use the module parameter disable_cxl. This is a per-device opt-out on the core device set by the driver before registration. Signed-off-by: Manish Honap --- drivers/vfio/pci/cxl/vfio_cxl_core.c | 4 ++++ drivers/vfio/pci/vfio_pci.c | 9 +++++++++ include/linux/vfio_pci_core.h | 1 + 3 files changed, 14 insertions(+) diff --git a/drivers/vfio/pci/cxl/vfio_cxl_core.c b/drivers/vfio/pci/cxl/vf= io_cxl_core.c index bda11f99746f..8b42ac05a110 100644 --- a/drivers/vfio/pci/cxl/vfio_cxl_core.c +++ b/drivers/vfio/pci/cxl/vfio_cxl_core.c @@ -373,6 +373,10 @@ void vfio_pci_cxl_detect_and_init(struct vfio_pci_core= _device *vdev) u16 dvsec; int ret; =20 + /* Honor the user opt-out decision */ + if (vdev->disable_cxl) + return; + if (!pcie_is_cxl(pdev)) return; =20 diff --git a/drivers/vfio/pci/vfio_pci.c b/drivers/vfio/pci/vfio_pci.c index 22cf9ea831f9..a6b0fb882b9f 100644 --- a/drivers/vfio/pci/vfio_pci.c +++ b/drivers/vfio/pci/vfio_pci.c @@ -60,6 +60,12 @@ static bool disable_denylist; module_param(disable_denylist, bool, 0444); MODULE_PARM_DESC(disable_denylist, "Disable use of device denylist. Disabl= ing the denylist allows binding to devices with known errata that may lead = to exploitable stability or security issues when accessed by untrusted user= s."); =20 +#if IS_ENABLED(CONFIG_VFIO_CXL_CORE) +static bool disable_cxl; +module_param(disable_cxl, bool, 0444); +MODULE_PARM_DESC(disable_cxl, "Disable CXL Type-2 extensions for all devic= es bound to vfio-pci. Variant drivers may instead set vdev->disable_cxl in = their probe for per-device control without needing this parameter."); +#endif + static bool vfio_pci_dev_in_denylist(struct pci_dev *pdev) { switch (pdev->vendor) { @@ -189,6 +195,9 @@ static int vfio_pci_probe(struct pci_dev *pdev, const s= truct pci_device_id *id) return PTR_ERR(vdev); =20 dev_set_drvdata(&pdev->dev, vdev); +#if IS_ENABLED(CONFIG_VFIO_CXL_CORE) + vdev->disable_cxl =3D disable_cxl; +#endif vdev->pci_ops =3D &vfio_pci_dev_ops; ret =3D vfio_pci_core_register_device(vdev); if (ret) diff --git a/include/linux/vfio_pci_core.h b/include/linux/vfio_pci_core.h index aa159d0c8da7..48dc69df52fa 100644 --- a/include/linux/vfio_pci_core.h +++ b/include/linux/vfio_pci_core.h @@ -130,6 +130,7 @@ struct vfio_pci_core_device { bool needs_pm_restore:1; bool pm_intx_masked:1; bool pm_runtime_engaged:1; + bool disable_cxl:1; struct pci_saved_state *pci_saved_state; struct pci_saved_state *pm_save; int ioeventfds_nr; --=20 2.25.1 From nobody Tue Apr 7 19:54:44 2026 Received: from PH7PR06CU001.outbound.protection.outlook.com (mail-westus3azon11010053.outbound.protection.outlook.com [52.101.201.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6130D35F17A; Wed, 11 Mar 2026 20:37:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.201.53 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773261470; cv=fail; b=bOJKuxV9oNJyamUINjHl5Mir23BjwjNbk0Obk2ydcdrF4jztXTUdKCHp0SpkYLe2A8kA6e6Bm+HmPcQzYxjAB1y1EZ7p5C0EyF0zAbGOIrY7u0/qTkfMp+QCJaLSocUcmjNWWJvQNs92FqkoiI7ygFprxNuTGKektxFW0mGCJJo= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773261470; c=relaxed/simple; bh=jAOgi7NXzXg+K7q7UlANCejfSoXtckeNHyrHwwRpAzg=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=ZA7PeIo3fWxZnZA8NEzV2XG59xHKRg84kAm108T6yIp9q+PSGYViivEyFOvY4jKKtLAQhF5h6FZSJ9OnMx1QBmxr7roaVdwRsT2k1QHq25AKI+FW9Ap78Wn+2yn3TTCi6RN87mQ0HHXWgmjXbNIcdO3X8s2bs+F8+7md2+FpfqQ= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=QIQPwvV6; arc=fail smtp.client-ip=52.101.201.53 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="QIQPwvV6" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=k/P2JOmnu7IxuM9ytjvAtUwMqDoGKDYFgedb3iqbX546CcnxLtXutEfSw3hAG8gKSj0L9Xrsg5jDl5mP9NkGMVJpBTFeFQCQyCeHAg4JTDU7s8YAu+NeUjatCpHwytnW8bzVJMDoRIELmQNIpPfJXfD0JVhv8wQRA7Gn0bHCk0BtoRseK8OreHwHyW24HXknoFdyqQtNbycAH++xlZhj6uQfH2mtWtQXNE9qdtECBJK0E3UXGXreNxNZdsJLK1iu83bdSl40cPwab8zsH+nWGNH/Cop++WOjPGpnMP1oZS33EjUABj23LpDKQId4PzuhTRnNKNPZFvrBYwvAqMcX/w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=TdwMp85kIhAH4ATymjHOGXtZ7ezqg8fQvFQHCWZetGQ=; b=u8dd0AREPTkgYLhrvWsrMU/KCopf71DOM6eMUhWxEoPRS5vwj8+RuhXB9I5uILuzKdpkv/QEm994dqbGMAPgpLhHREwVHUh8Pz7U299PDC2TZYLL1fS0eNt6o/2m6Hxr0iEbUhzSWAb/Jv6xRKxeFV2JcBnj19nv77EvK6o6EN9dd+ZM/bIxiTS8aj8bGCm7eGJeQlK9j6qDn9JrxlFyPyB0wdAd9UObVIcaGMX+VplcoG4o75ScJ8f+DNGo3ttZqeSpunwvKE5/iKBZ8BLOSYX9I8NKNBPFrjwyPMw3RTq45C0QhQ2l6G+wZi2y3lmBxAoSHzcJWfY/BvkvRkSelQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=TdwMp85kIhAH4ATymjHOGXtZ7ezqg8fQvFQHCWZetGQ=; b=QIQPwvV6hsFbkNWFu3o7m8xIsHygVmU81/ZtVBfxeZyS2689mfqJGW1vF8bDtto11t8kBoUWnZPXsGpKLCe1WVOUHkYTmDjm/RbpkMBeCkBwfVmtGurPZL+SCLN8yJJ+7XolO9AIzL63T1+Gs1qGRs0jCJVdowFhx7rlDzXsP7OWiUGtAyVd553x7SdDeQd0bS94gS4eYtaxVwJHh51YNkwRNe+bj3F6HoisYdWrJOvg8JEVuQ9x2nRyvi3CtJnPH3WxjddfrZSaDEcvqe5D+5M+JJiYnSl2pQSbI+ErhXprd8RhiHWMgkr/cDtHFX2NsmTnHLzdcm71eXHNPopFKg== Received: from DS7P222CA0007.NAMP222.PROD.OUTLOOK.COM (2603:10b6:8:2e::24) by DM4PR12MB7575.namprd12.prod.outlook.com (2603:10b6:8:10d::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9700.11; Wed, 11 Mar 2026 20:37:36 +0000 Received: from DS2PEPF00003447.namprd04.prod.outlook.com (2603:10b6:8:2e:cafe::db) by DS7P222CA0007.outlook.office365.com (2603:10b6:8:2e::24) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9678.26 via Frontend Transport; Wed, 11 Mar 2026 20:37:28 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by DS2PEPF00003447.mail.protection.outlook.com (10.167.17.74) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9678.18 via Frontend Transport; Wed, 11 Mar 2026 20:37:36 +0000 Received: from rnnvmail203.nvidia.com (10.129.68.9) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Wed, 11 Mar 2026 13:37:13 -0700 Received: from rnnvmail201.nvidia.com (10.129.68.8) by rnnvmail203.nvidia.com (10.129.68.9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Wed, 11 Mar 2026 13:37:12 -0700 Received: from nvidia-4028GR-scsim.nvidia.com (10.127.8.11) by mail.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20 via Frontend Transport; Wed, 11 Mar 2026 13:37:05 -0700 From: To: , , , , , , , , , , , , , , , , , CC: , , , , , , , Subject: [PATCH 18/20] docs: vfio-pci: Document CXL Type-2 device passthrough Date: Thu, 12 Mar 2026 02:04:38 +0530 Message-ID: <20260311203440.752648-19-mhonap@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20260311203440.752648-1-mhonap@nvidia.com> References: <20260311203440.752648-1-mhonap@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS2PEPF00003447:EE_|DM4PR12MB7575:EE_ X-MS-Office365-Filtering-Correlation-Id: 1fd12706-ae02-4271-22d9-08de7fae079e X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|36860700016|376014|82310400026|7416014|921020|18002099003|56012099003|22082099003; X-Microsoft-Antispam-Message-Info: Yn35CoIrF3paHwnYOgcNS2oiJucu0Bt/S5Yu2uapqFsyajVmzN8V0dSVYjHELBILfmOHW3wW0i75iKNu3A0lH6FvHvZLNrs3+JWc5c1kBywa30b1iMDL557p+L7MJ2YgMkTKu+JusKm+wK67CdfZ5ftTG24DXc70K6DlYAs/TdsDVozRjTJDuh2BV0G3RhJ+bbLHBcF3igzcuHrV9wAafsSKbzRa+k6xIgU3r0TRSbtH7aFY58EGNs6xgYlHW1CLv0k3HMtGMNQgdl7N5oInvk0B01IHCzVkBQMzhSrLl8qeD2EVIXnrGLnM00iVj21IcugZ/81pVGhf+9cIiFEWtK1HvkEmMDclMSA+2VavM/ufhzGtZgSnQtKoaKTFXFJmO4iZ5hPcYldDJq0Ogxh65/EqM3So9PUvc+GJhfxImiQbQ8gkdwoU5NUII5mcp5xq1ZabXWBA/wgIcLnY7IlTOK66HCZruzrFPGro0CXFAKE06rDAR9Rt6GKs53k1uGjJvSimIbRXoWtF7ZO20SnYDlu0TLDwxq5M5ZhQDcRRjMvvLIf9ngt0vD+fwfXfKoBSXWqFXAsC+SU7h3Nl2XVd94DlbAqyb1zMKzDby8u6GR5zmDtSdfGnTRpAJZQ+63nGUCGHVyf40OwRo7uU6JDSbf8lg9lOCtdm5KATh1sj3TcwFB2t6Rpgskpvu+hCGlNc0eCbCJPyAZmNoDdzo8G71qnxXdj92qaHP3CiTrLtyGt4GWaDDYij3H3fFS0W+fCpUHDhO2AkyNmpv+/AYvDF6epqbD2GIzyGDyCsPfWXsBqHhp0upGqvcAQjnCukQIdZ X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(1800799024)(36860700016)(376014)(82310400026)(7416014)(921020)(18002099003)(56012099003)(22082099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: Tbc64jHe+pasdO/zOi7grkqNzKV3dXHSyy/tmFyQoH79FcvzT/8M+oWkQm2AzdiGSazSoPiHscwWV8wqHnD4kclGQx+Ym3SiUsAQpWTuXQ1ID0PI6v5OeHH3XwbXux/y88llMjtXKIemLf7E7viIXDqGF9UfkdwwhE72ZutuOLI5/AP5PAoqAvUCk3PnD3wcaU6W2OQuw+G0+kxLuqNU735T7t9CFz1PYiWqxi0YqlHANg7LlamAmnnaSuvILR3gKpYb3EefxTvXLoFAixOe64ddx9dfjR/CSH7yp+BJ2waHVBX3Z1vp02SpmhmpPjyLjFL41842Mo/sIJR9ASlvxPLuY+Knnu42GSqL3aMi+Uo8W9eK+KYY4uzhlbHHGO9GVP25QaHozCqhTDINjcIznQU8/xSmnzQhR1Z+fLjatDgYCpjCX5Wyff4T9uxHGqwK X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Mar 2026 20:37:36.0357 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 1fd12706-ae02-4271-22d9-08de7fae079e X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DS2PEPF00003447.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB7575 From: Manish Honap Add a driver-api document describing the architecture, interfaces, and operational constraints of CXL Type-2 device passthrough via vfio-pci-core. CXL Type-2 devices (cache-coherent accelerators such as GPUs with attached device memory) present unique passthrough requirements not covered by the existing vfio-pci documentation: - The host kernel retains ownership of the HDM decoder hardware through the CXL subsystem, so the guest cannot program decoders directly. - Two additional VFIO device regions expose the emulated HDM register state (COMP_REGS) and the DPA memory window (DPA region) to userspace. - DVSEC configuration space writes are intercepted and virtualized so that the guest cannot alter host-owned CXL.io / CXL.mem enable bits. - Device reset (FLR) is coordinated through vfio_pci_ioctl_reset(): all DPA PTEs are zapped before the reset and restored afterward. Signed-off-by: Manish Honap --- Documentation/driver-api/index.rst | 1 + Documentation/driver-api/vfio-pci-cxl.rst | 216 ++++++++++++++++++++++ 2 files changed, 217 insertions(+) create mode 100644 Documentation/driver-api/vfio-pci-cxl.rst diff --git a/Documentation/driver-api/index.rst b/Documentation/driver-api/= index.rst index 1833e6a0687e..7ec661846f6b 100644 --- a/Documentation/driver-api/index.rst +++ b/Documentation/driver-api/index.rst @@ -47,6 +47,7 @@ of interest to most developers working on device drivers. vfio-mediated-device vfio vfio-pci-device-specific-driver-acceptance + vfio-pci-cxl =20 Bus-level documentation =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D diff --git a/Documentation/driver-api/vfio-pci-cxl.rst b/Documentation/driv= er-api/vfio-pci-cxl.rst new file mode 100644 index 000000000000..f2cbe2fdb036 --- /dev/null +++ b/Documentation/driver-api/vfio-pci-cxl.rst @@ -0,0 +1,216 @@ +.. SPDX-License-Identifier: GPL-2.0 + +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D +VFIO PCI CXL Type-2 Device Passthrough +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D + +Overview +-------- + +CXL (Compute Express Link) Type-2 devices are cache-coherent PCIe accelera= tors +and GPUs that attach their own volatile memory (Device Physical Address sp= ace, +or DPA) to the host memory fabric via the CXL protocol. Examples include +GPU/accelerator cards that expose coherent device memory to the host. + +When such a device is passthroughed to a virtual machine using ``vfio-pci`= `, +the kernel CXL subsystem must remain in control of the Host-managed Device +Memory (HDM) decoders that map the device's DPA into the host physical add= ress +(HPA) space. A VMM such as QEMU cannot program HDM decoders directly; ins= tead +it uses a set of VFIO-specific regions and UAPI extensions described here. + +This support is compiled in when ``CONFIG_VFIO_CXL_CORE=3Dy``. It can be +disabled at module load time for all devices bound to ``vfio-pci`` with:: + + modprobe vfio-pci disable_cxl=3D1 + +Variant drivers can disable CXL extensions for individual devices by setti= ng +``vdev->disable_cxl =3D true`` in their probe function before registration. + +Device Detection +---------------- + +CXL Type-2 detection happens automatically when ``vfio-pci`` registers a +device that has: + +1. A CXL Device DVSEC capability (PCIe DVSEC Vendor ID 0x1E98, ID 0x0000). +2. Bit 2 (Mem_Capable) set in the CXL Capability register within that DVSE= C. +3. A PCI class code that is **not** ``0x050210`` (CXL Type-3 memory device= ). +4. An HDM Decoder block discoverable via the Register Locator DVSEC. +5. A pre-committed HDM decoder (BIOS/firmware programmed) with non-zero si= ze. + +On successful detection ``VFIO_DEVICE_FLAGS_CXL`` is set in +``vfio_device_info.flags`` alongside ``VFIO_DEVICE_FLAGS_PCI``. + +UAPI Extensions +--------------- + +VFIO_DEVICE_GET_INFO Capability: VFIO_DEVICE_INFO_CAP_CXL +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +When ``VFIO_DEVICE_FLAGS_CXL`` is set the device info capability chain +contains a ``vfio_device_info_cap_cxl`` structure (cap ID 6):: + + struct vfio_device_info_cap_cxl { + struct vfio_info_cap_header header; /* id=3D6, version=3D1 */ + __u8 hdm_count; /* number of HDM decoders */ + __u8 hdm_regs_bar_index; /* PCI BAR containing component register= s */ + __u16 pad; + __u32 flags; /* VFIO_CXL_CAP_* flags */ + __u64 hdm_regs_size; /* size in bytes of the HDM decoder bloc= k */ + __u64 hdm_regs_offset; /* byte offset within the BAR to HDM blo= ck */ + __u64 dpa_size; /* total DPA size in bytes */ + __u32 dpa_region_index; /* index of the DPA device region */ + __u32 comp_regs_region_index; /* index of the COMP_REGS device reg= ion */ + }; + +Flags: + +``VFIO_CXL_CAP_COMMITTED`` (bit 0) + The HDM decoder was committed by the kernel CXL subsystem. + +``VFIO_CXL_CAP_PRECOMMITTED`` (bit 1) + The HDM decoder was pre-committed by host firmware/BIOS. The VMM does + not need to allocate CXL HPA space; the mapping is already live. + +VFIO Regions +~~~~~~~~~~~~~ + +A CXL Type-2 device exposes two additional device regions beyond the stand= ard +PCI BAR regions. Their indices are reported in ``dpa_region_index`` and +``comp_regs_region_index`` in the capability structure. + +**DPA Region** (subtype ``VFIO_REGION_SUBTYPE_CXL``) + Flags: ``VFIO_REGION_INFO_FLAG_READ | VFIO_REGION_INFO_FLAG_WRITE | + VFIO_REGION_INFO_FLAG_MMAP`` + + Represents the device's DPA memory mapped at the kernel-assigned HPA. + The VMM should map this region with mmap() to expose device memory to = the + guest. Page faults are handled lazily; the kernel inserts PFNs on fir= st + access rather than at mmap() time. During FLR/reset all PTEs are + invalidated and the region becomes inaccessible until the reset comple= tes. + + Read and write access via the region file descriptor is also supported= and + routes through a kernel-managed virtual address established with + ``ioremap_cache()``. + +**COMP_REGS Region** (subtype ``VFIO_REGION_SUBTYPE_CXL_COMP_REGS``) + Flags: ``VFIO_REGION_INFO_FLAG_READ | VFIO_REGION_INFO_FLAG_WRITE`` + (no mmap). + + An emulated, read/write-only region exposing the HDM decoder registers. + The kernel shadows the hardware HDM register state and enforces all + bit-field rules (reserved bits, read-only bits, commit semantics) on e= very + write. Only 32-bit aligned, 32-bit wide accesses are permitted, match= ing + the hardware requirement. + + The VMM uses this region to read and write HDM decoder BASE, SIZE, and + CTRL registers. Setting the COMMIT bit (bit 9) in a CTRL register cau= ses + the kernel to immediately set the COMMITTED bit (bit 10) in the emulat= ed + shadow state, allowing the VMM to detect the transition via a + ``notify_change`` callback. + + The component register BAR itself (``hdm_regs_bar_index``) is hidden: + ``VFIO_DEVICE_GET_REGION_INFO`` for that BAR index returns ``size =3D = 0``. + All HDM access must go through the COMP_REGS region. + +Region Type Identifiers:: + + /* type =3D PCI_VENDOR_ID_CXL | VFIO_REGION_TYPE_PCI_VENDOR_TYPE (0x80= 001e98) */ + #define VFIO_REGION_SUBTYPE_CXL 1 /* DPA memory region */ + #define VFIO_REGION_SUBTYPE_CXL_COMP_REGS 2 /* HDM register region */ + +DVSEC Configuration Space Emulation +------------------------------------- + +When ``CONFIG_VFIO_CXL_CORE=3Dy`` the kernel installs a CXL-aware write ha= ndler +for the ``PCI_EXT_CAP_ID_DVSEC`` (0x23) extended capability entry in the v= fio-pci +configuration space permission table. This handler runs for every device +opened under ``vfio-pci``; for non-CXL devices it falls through to the +hardware write path unchanged. + +For CXL devices, writes to the following DVSEC registers are intercepted a= nd +emulated in ``vdev->vconfig`` (the per-device shadow configuration space): + ++--------------------+--------+-------------------------------------------+ +| Register | Offset | Emulation | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D= =3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ +| CXL Control | 0x0c | RWL semantics; IO_Enable forced to 1; | +| | | locked after Lock register bit 0 is set. | ++--------------------+--------+-------------------------------------------+ +| CXL Status | 0x0e | Bit 14 (Viral_Status) is RW1CS. | ++--------------------+--------+-------------------------------------------+ +| CXL Control2 | 0x10 | Bits 0, 3 forwarded to hardware; bits | +| | | 1 and 2 trigger subsystem actions. | ++--------------------+--------+-------------------------------------------+ +| CXL Status2 | 0x12 | Bit 3 (RW1CS) forwarded to hardware when | +| | | Capability3 bit 3 is set. | ++--------------------+--------+-------------------------------------------+ +| CXL Lock | 0x14 | RWO; once set, Control becomes read-only | +| | | until conventional reset. | ++--------------------+--------+-------------------------------------------+ +| Range Base High/Lo | varies | Stored in vconfig; Base Low [27:0] | +| | | reserved bits cleared. | ++--------------------+--------+-------------------------------------------+ + +Reads of these registers return the emulated vconfig values. Read-only +registers (Capability, Size registers, range Size High/Low) are also served +from vconfig, which was seeded from hardware at device open time. + +FLR and Reset Behaviour +----------------------- + +During Function Level Reset (FLR): + +1. ``vfio_cxl_zap_region_locked()`` is called under the write side of + ``memory_lock``. It sets ``region_active =3D false`` and calls + ``unmap_mapping_range()`` to invalidate all DPA region PTEs. + +2. Any concurrent page fault or ``read()``/``write()`` on the DPA region + sees ``region_active =3D false`` and returns ``VM_FAULT_SIGBUS`` or ``-= EIO`` + respectively. + +3. After reset completes, ``vfio_cxl_reactivate_region()`` re-reads the HDM + decoder state from hardware into ``comp_reg_virt[]`` (it will typically + be all-zeros after FLR) and sets ``region_active =3D true`` only if the + COMMITTED bit is set in the freshly re-snapshotted hardware state for + pre-committed decoders. The VMM may re-fault into the DPA region witho= ut + issuing a new ``mmap()`` call. Each newly faulted page is scrubbed via + ``memset_io()`` before the PFN is inserted. + +VMM Integration Notes +--------------------- + +A VMM integrating CXL Type-2 passthrough should: + +1. Issue ``VFIO_DEVICE_GET_INFO`` and check ``VFIO_DEVICE_FLAGS_CXL``. +2. Walk the capability chain to find ``VFIO_DEVICE_INFO_CAP_CXL`` (id =3D = 6). +3. Record ``dpa_region_index``, ``comp_regs_region_index``, ``dpa_size``, + ``hdm_count``, ``hdm_regs_offset``, and ``hdm_regs_size``. +4. Map the DPA region (``dpa_region_index``) with mmap() to a guest physic= al + address. The region supports ``PROT_READ | PROT_WRITE``. +5. Open the COMP_REGS region (``comp_regs_region_index``) and attach a + ``notify_change`` callback to detect COMMIT transitions. When bit 10 + (COMMITTED) transitions from 0 to 1 in a CTRL register read, the VMM + should expose the corresponding DPA range to the guest and map the + relevant slice of the DPA mmap. +6. For pre-committed devices (``VFIO_CXL_CAP_PRECOMMITTED`` set) the entire + DPA is already mapped and the VMM need not wait for a guest COMMIT. +7. Program the guest CXL DVSEC registers (via VFIO config space write) to + reflect the guest's view. The kernel emulates all register semantics + including the CONFIG_LOCK one-shot latch. + +Kernel Configuration +-------------------- + +``CONFIG_VFIO_CXL_CORE`` (bool) + Enable CXL Type-2 passthrough support in ``vfio-pci-core``. + Depends on ``CONFIG_VFIO_PCI_CORE``, ``CONFIG_CXL_BUS``, and + ``CONFIG_CXL_MEM``. + +References +---------- + +* CXL Specification 3.1, =C2=A78.1.3 =E2=80=94 DVSEC for CXL Devices +* CXL Specification 3.1, =C2=A78.2.4.20 =E2=80=94 CXL HDM Decoder Capabili= ty Structure +* ``include/uapi/linux/vfio.h`` =E2=80=94 ``VFIO_DEVICE_INFO_CAP_CXL``, + ``VFIO_REGION_SUBTYPE_CXL``, ``VFIO_REGION_SUBTYPE_CXL_COMP_REGS`` --=20 2.25.1 From nobody Tue Apr 7 19:54:44 2026 Received: from SJ2PR03CU001.outbound.protection.outlook.com (mail-westusazon11012008.outbound.protection.outlook.com [52.101.43.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 110F538D6B0; Wed, 11 Mar 2026 20:37:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.43.8 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773261480; cv=fail; b=N9b2vwQEa+tLhqQz4OYTKXvFp1uOwAGo19VlxVd8qgG/nUtJEQYD4tC5n4FdM5mvY+ZXpvG8wsZyhwXxdsi0bWiYB3uktOTdZnQTKFZJHiovrUhjDBoOBAIKRzYZx23DFg51Y6HuriIQqXyRc+bJGAdg6sZ9NDzFWseuojHNR4s= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773261480; c=relaxed/simple; bh=guBRS2ZsjiJAl7AK2gWeYk8dUZJpQ34BrgRdFG9UpVA=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=fvYBQSk5t5Wnbc6mCSgMl4hWbhB/mQpXLr5Ugv5vSxfYCNpwwLQtY/KNXsGsUfLHAe0sCN/vjQGa90f2kum5dXIO5kjpJ/vonEWfX3g2Akhhq/lp3OrN03WI5D4BNVec0E6QvrLbd6TZR0CcILAJNc5m9l287MF7A/Uj9YsrxNE= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=ZYCuFKR5; arc=fail smtp.client-ip=52.101.43.8 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="ZYCuFKR5" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=dd6jovRHFW1wIvBAfL7mtQhuJ+iy4OGcwg0oRsqSh6c+vLoAVMzHm99AKIXO/untcutTX76kT14WcbzZ5nYArWvAuR1NW/ZqQTdZkWjBuHadVp112cLlSPvjCaKLD5ssEYCHWnd/9zc2rzIO/JskXAHy6m6mUFRQCIBGCfVXdX9Tq6SyZ1HZ10gZEU3GaxmSCvh/8NZ/gTUcFInEqh2f5e3unRXcWcDYR6KOm7keccZecN3krsOhswF934pV7qmXV7NPN14sVSJIuAo7r2RQWtdllu6fPjsbiDF/w3rYJucnnrVRUHaTHrHMGnl+eeN1/6pNtWTr/mPBJ2NR6ume5w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=kqpGaqbF/BaKxTQGXtl+B6zoRGtPrGFkzWUwrx+J6+0=; b=lVDKMPmWT9xU1Te+CDoJn03PW4an/CdEGER8GEVMJMy+/MKfCntzKG5yeQTI5fj3YMVOedvchpyHDPwcIc60gGcmd6+cbmJ7YcpvI+R+D55bsBaJGltQzVoLljHdP67wmLl0SB6fzX70HwyPjrJi4RHGmni4mH2F4KVf0dDAhjPmeniIc2Wa9Yiif7rCu0TahQhiqcAtMSYFB7BQJYIHmFAcp/s/EoDqSt3aTZPIjuh0ShhAGVfQb2qfwmL2K50uv/7DKf0Lw+Dh0Zm2ofEIJq9guiD+1yfAS4366B3e66JWVgwrObht725b81y5bp3z2MblxiprkIQDxrmyu1jr0A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=kqpGaqbF/BaKxTQGXtl+B6zoRGtPrGFkzWUwrx+J6+0=; b=ZYCuFKR5+5IYnZTOOLv8QfBtr9wDxF4f2+pd6/FWDUwT7ehHl/vSCwVavxXRArwU702dqS3cZmjgcMOtUxjXfimgzwpqQ2E4/NEF/DiJ/FOU54X3QRS68AGhWxgjzrmtfeNXHfSHz0LlKh/ov+5m12Oef7dS1UUQkg2DAD6uoQ5zn6/1Zf4+ysPHRVUPGSt3Lk7nCaAjlcGg+8wgASbvEmxhC8yyzRcFyxb3eR5rHmobMaXe4BLoVYyeKn47KKHgFzK0bYL8gdwBy67Ih32TRgbUdU5YP8vdL+Ao/oCPruDu8u9aITC4/4P8/XBZq9wjbjGwt6DHxwYog4Pqaf5fkw== Received: from SA9P223CA0026.NAMP223.PROD.OUTLOOK.COM (2603:10b6:806:26::31) by DM4PR12MB6448.namprd12.prod.outlook.com (2603:10b6:8:8a::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9700.11; Wed, 11 Mar 2026 20:37:45 +0000 Received: from SN1PEPF0002636E.namprd02.prod.outlook.com (2603:10b6:806:26:cafe::f8) by SA9P223CA0026.outlook.office365.com (2603:10b6:806:26::31) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9678.25 via Frontend Transport; Wed, 11 Mar 2026 20:37:19 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by SN1PEPF0002636E.mail.protection.outlook.com (10.167.241.139) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9678.18 via Frontend Transport; Wed, 11 Mar 2026 20:37:42 +0000 Received: from rnnvmail205.nvidia.com (10.129.68.10) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Wed, 11 Mar 2026 13:37:20 -0700 Received: from rnnvmail201.nvidia.com (10.129.68.8) by rnnvmail205.nvidia.com (10.129.68.10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Wed, 11 Mar 2026 13:37:20 -0700 Received: from nvidia-4028GR-scsim.nvidia.com (10.127.8.11) by mail.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20 via Frontend Transport; Wed, 11 Mar 2026 13:37:13 -0700 From: To: , , , , , , , , , , , , , , , , , CC: , , , , , , , Subject: [PATCH 19/20] selftests/vfio: Add CXL Type-2 passthrough tests Date: Thu, 12 Mar 2026 02:04:39 +0530 Message-ID: <20260311203440.752648-20-mhonap@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20260311203440.752648-1-mhonap@nvidia.com> References: <20260311203440.752648-1-mhonap@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SN1PEPF0002636E:EE_|DM4PR12MB6448:EE_ X-MS-Office365-Filtering-Correlation-Id: b91e4ca5-90ca-4925-669b-08de7fae0bb6 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|82310400026|1800799024|7416014|36860700016|921020|56012099003|18002099003|22082099003; X-Microsoft-Antispam-Message-Info: topZN424MdhzcZ6BDKbFxz6fyR/ndd8+so0VLDkuZ6vkuoJrqoNBpoL0aS/CEQ0TCp7cQqxeFd6z0T90t+IaRKc5q8f41bT9SHI2y3FYMRZdxN8FJNAspufOh09Goz0k/7StyusQ6iZuQvwvXrNozMnbCKkmBKL4Qfbc+rSn6NLKzLYYXScqr6QTMHjhzwujYIlfCTaBJEk1S8pACWU3eGqTabLovGi/3+0e+D+VjnqLQZeR3G5pwmjG1XwfzLXr9nQNNidUwt8+2tko5whjzw5CQ5dmtn5TA/zL+uBH7qXjUqA0KhgVA7LK+sJWLc6ML82uYzw3EhUMXPTGb5oY0e3c8P1juY4WYt/GFGVJo7PfpF1cRGMpYxiueWdveGCsobt8hTiMCnuVICCKUTHGNj/cqSkbluUz+ElB+05700OcfCP10twHJwkHpTCdaWwJniJJ9GwUNi3BTSEj+s6X0U20MqG1Zw1y3nHIY1QPfxDrl7QyAWeeu1n/F6q/XOstGzlM0PUIBQrDhwNan3KL71SCJdaTaCQ83cTsghadNAzXaMsc6JjjjRqD4d57gAYi4BmfyTtGXDWXtNj6B92wj74P13tTaxtKN+nrEouVsPbTsRrVgddIa1Sc63oQhl6UH8YG9s2WUMQIvp+cioYFV43y2tdspjxm7xx1056j8zuGBgtARbHmdxR6VXZSCyMB96XrtlNQpTn2XJ9QKQ8yQkgYU2JxN7NcNQbnbpZhwDhTzCsoIbi0YWs/mTFctmNkI1nJZ4oKztrtexUUNOQwDLrIixYVf5jXg30KbUIRZhSJXzQ6XRH5qM0BqCoy3CTe X-Forefront-Antispam-Report: CIP:216.228.117.160;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge1.nvidia.com;CAT:NONE;SFS:(13230040)(376014)(82310400026)(1800799024)(7416014)(36860700016)(921020)(56012099003)(18002099003)(22082099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: dE49Hg30Q2a7/TDSPJmH7MoaikvTz2KHA0WpupIHbOCSv17HTWqkPKYCW7w9myCzmVe3Fnk6laBgM7jP9NVCxzUjsHtofSextIncqI197I1blU3e9u0ylV6CpzTGyiCYv16UyMn+YkSMmEVJKS+0PaIZg3YzAN/11Up0WYYTO7jVW8Q8ZuiXsyIeV8ivv5iMC9udI+OTIdUJDuajtA6rh7i/IspBlybA+ZYgdJs20IA4jZnImbgRJU1erpW3ZshMRfjCJH9FVYvE/cCaQvZIub8qxCG4TAq12A6cz604Bn3Y52Gm52GaeIzc+aD1MxLP8riKJr3grFBSLsk4JpzCvBSb5phheOXgBUsIKmG/ru5bSpFq7cGfaiVMGhwmu7Dccq5ySw83fwEf4WX4oSaUJHjt3Ngz1K3Z4pORcpyhWAqTzfNIsj/6klBs2Q8Nlr+k X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Mar 2026 20:37:42.9254 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: b91e4ca5-90ca-4925-669b-08de7fae0bb6 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: SN1PEPF0002636E.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB6448 From: Manish Honap Add a selftest suite exercising the CXL Type-2 device passthrough interfaces introduced by the vfio-cxl patch series. The tests are designed to run on a host with a CXL Type-2 device bound to vfio-pci and CONFIG_VFIO_CXL_CORE=3Dy. They verify the kernel ABI without requiring a running QEMU instance or a guest OS. Test cases: cxl_test_device_info: Open the VFIO device. Call VFIO_DEVICE_GET_INFO and verify: - VFIO_DEVICE_FLAGS_CXL is set in info.flags - VFIO_DEVICE_FLAGS_PCI is also set (CXL implies PCI) - info.num_regions > VFIO_PCI_NUM_REGIONS (CXL adds extra regions) Walk the capability chain and locate VFIO_DEVICE_INFO_CAP_CXL. Verify: - hdm_count >=3D 1 - hdm_regs_bar_index <=3D 5 - hdm_regs_size >=3D 0x20 * hdm_count (minimum: one decoder slot) - dpa_size > 0 (pre-committed decoder present) - dpa_region_index and comp_regs_region_index within bounds cxl_test_component_bar_hidden: Query VFIO_DEVICE_GET_REGION_INFO for the BAR index reported by hdm_regs_bar_index. Verify info.size =3D=3D 0, confirming the host has hidden the component BAR from direct userspace access. cxl_test_comp_regs_region: Query VFIO_DEVICE_GET_REGION_INFO for comp_regs_region_index. Verify: - flags has READ and WRITE set, mmap NOT set - size =3D=3D hdm_regs_size Open the region fd and read 4 bytes at offset 0 (HDM Decoder Cap). Verify the read succeeds and returns a non-zero value. Attempt a misaligned read (3-byte or offset 1) _ verify EINVAL. Attempt a 4-byte write to offset 0 (RO register) _ verify it silently succeeds (write to RO discards without error per design). Write a known pattern to HDM Decoder 0 BASE_LO (offset 0x10) and read back _ verify the written value (with reserved bits cleared) is returned. cxl_test_dpa_region: Query VFIO_DEVICE_GET_REGION_INFO for dpa_region_index. Verify: - flags has READ, WRITE, MMAP set - size =3D=3D dpa_size (consistent with device info) mmap() the full DPA region (MAP_SHARED). Verify mmap() succeeds. Write a test pattern to offset 0 of the mapping (triggers first page fault / PFN insertion). Read back and verify the value. munmap() the region. Verify no crash. cxl_test_bar_mmap_rejected: Attempt to mmap() the component BAR directly via the standard VFIO_PCI_BAR*_REGION_INDEX path. Verify EINVAL is returned. cxl_test_bar_read_rejected: Attempt to read() the component BAR region fd. Verify EINVAL. cxl_test_disable_cxl_param: (Requires root + module reload capability) Reload vfio-pci with disable_cxl=3D1. Rebind the device. Call VFIO_DEVICE_GET_INFO and verify VFIO_DEVICE_FLAGS_CXL is NOT set and num_regions =3D=3D VFIO_PCI_NUM_REGIONS (no extra CXL regions). Reload vfio-pci without the parameter and rebind to restore state. Signed-off-by: Manish Honap --- tools/testing/selftests/vfio/Makefile | 1 + .../selftests/vfio/vfio_cxl_type2_test.c | 816 ++++++++++++++++++ 2 files changed, 817 insertions(+) create mode 100644 tools/testing/selftests/vfio/vfio_cxl_type2_test.c diff --git a/tools/testing/selftests/vfio/Makefile b/tools/testing/selftest= s/vfio/Makefile index 3c796ca99a50..2cac98302609 100644 --- a/tools/testing/selftests/vfio/Makefile +++ b/tools/testing/selftests/vfio/Makefile @@ -4,6 +4,7 @@ TEST_GEN_PROGS +=3D vfio_iommufd_setup_test TEST_GEN_PROGS +=3D vfio_pci_device_test TEST_GEN_PROGS +=3D vfio_pci_device_init_perf_test TEST_GEN_PROGS +=3D vfio_pci_driver_test +TEST_GEN_PROGS +=3D vfio_cxl_type2_test =20 TEST_FILES +=3D scripts/cleanup.sh TEST_FILES +=3D scripts/lib.sh diff --git a/tools/testing/selftests/vfio/vfio_cxl_type2_test.c b/tools/tes= ting/selftests/vfio/vfio_cxl_type2_test.c new file mode 100644 index 000000000000..44df62378749 --- /dev/null +++ b/tools/testing/selftests/vfio/vfio_cxl_type2_test.c @@ -0,0 +1,816 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * vfio_cxl_type2_test - selftests for CXL Type-2 device passthrough via v= fio-pci + * + * Tests the UAPI and emulation layer introduced by CONFIG_VFIO_CXL_CORE: + * - VFIO_DEVICE_INFO_CAP_CXL capability detection and field validation + * - Component BAR hiding (size=3D0 response for hdm_regs_bar_index) + * - DPA region presence, size, and mmap + * - COMP_REGS region presence, size, read/write semantics + * - HDM decoder emulation: reserved-bit masking, COMMIT=E2=86=92COMMITT= ED transition + * - DVSEC configuration space emulation: Control, Status, Lock, Control2 + * + * Usage: + * ./vfio_cxl_type2_test + * or set the environment variable VFIO_SELFTESTS_BDF before running. + * + * The device must be a CXL Type-2 device (e.g. a GPU with coherent memory) + * with a pre-committed HDM decoder. The test is skipped automatically on + * non-CXL devices. + * + * Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved + */ + +#include +#include +#include +#include +#include + +#include +#include + +#include +#include +#include + +#include + +#include "kselftest_harness.h" + +/* Userspace equivalents of kernel helpers not available in user headers */ +#ifndef BIT +#define BIT(n) (1u << (n)) +#endif +#ifndef GENMASK +#define GENMASK(h, l) (((~0u) >> (31 - (h))) & ((~0u) << (l))) +#endif +#define VFIO_PCI_INDEX_TO_OFFSET(idx) ((uint64_t)(idx) << 40) + +static const char *device_bdf; + +/* ------------------------------------------------------------------ */ +/* CXL UAPI constants (mirrors include/uapi/linux/vfio.h) */ +/* ------------------------------------------------------------------ */ + +#define VFIO_DEVICE_FLAGS_CXL (1 << 9) + +#define VFIO_DEVICE_INFO_CAP_CXL 6 + +#define VFIO_CXL_CAP_COMMITTED (1 << 0) +#define VFIO_CXL_CAP_PRECOMMITTED (1 << 1) + +#define PCI_VENDOR_ID_CXL 0x1e98 +#define VFIO_REGION_TYPE_PCI_VENDOR_TYPE (1 << 31) +#ifndef VFIO_REGION_SUBTYPE_CXL +#define VFIO_REGION_SUBTYPE_CXL 1 +#endif +#ifndef VFIO_REGION_SUBTYPE_CXL_COMP_REGS +#define VFIO_REGION_SUBTYPE_CXL_COMP_REGS 2 +#endif + +/* + * HDM Decoder register layout within the component register block. + * Offsets relative to the start of the HDM decoder capability block + * (i.e. relative to the start of the COMP_REGS region). + */ +#define HDM_CAP_OFFSET 0x00 +#define HDM_GLOBAL_CTRL_OFFSET 0x04 +#define HDM_GLOBAL_STATUS_OFFSET 0x08 +#define HDM_DECODER_FIRST_OFFSET 0x10 +#define HDM_DECODER_STRIDE 0x20 +#define HDM_DECODER_BASE_LO 0x00 +#define HDM_DECODER_BASE_HI 0x04 +#define HDM_DECODER_SIZE_LO 0x08 +#define HDM_DECODER_SIZE_HI 0x0c +#define HDM_DECODER_CTRL 0x10 + +#define HDM_CTRL_COMMIT BIT(9) +#define HDM_CTRL_COMMITTED BIT(10) +#define HDM_CTRL_RESERVED_MASK (BIT(15) | GENMASK(31, 28)) +#define HDM_BASE_LO_RESERVED_MASK GENMASK(27, 0) +#define HDM_GLOBAL_CTRL_RESERVED_MASK GENMASK(31, 2) + +/* + * CXL DVSEC register offsets relative to the DVSEC capability base. + * =C2=A78.1.3 of CXL 3.1 specification. + */ +#define CXL_DVSEC_CONTROL_OFFSET 0x0c +#define CXL_DVSEC_STATUS_OFFSET 0x0e +#define CXL_DVSEC_CONTROL2_OFFSET 0x10 +#define CXL_DVSEC_LOCK_OFFSET 0x14 + +#define CXL_CTRL_IO_ENABLE BIT(1) +#define CXL_STATUS_RW1C_BIT BIT(14) +#define CXL_LOCK_BIT BIT(0) +#define CXL_LOCK_RESERVED_MASK GENMASK(15, 1) + +/* ------------------------------------------------------------------ */ +/* Helpers */ +/* ------------------------------------------------------------------ */ + +/* + * Walk the vfio_device_info capability chain embedded in @buf. + * Returns a pointer to the capability with the given @id, or NULL. + */ +static const struct vfio_info_cap_header * +find_device_cap(const void *buf, size_t bufsz, uint16_t id) +{ + const struct vfio_device_info *info =3D buf; + const struct vfio_info_cap_header *cap; + + if (!(info->flags & VFIO_DEVICE_FLAGS_CAPS) || !info->cap_offset) + return NULL; + + cap =3D (const struct vfio_info_cap_header *) + ((const char *)buf + info->cap_offset); + + while (true) { + if (cap->id =3D=3D id) + return cap; + if (!cap->next) + return NULL; + cap =3D (const struct vfio_info_cap_header *) + ((const char *)buf + cap->next); + if ((const char *)cap + sizeof(*cap) > (const char *)buf + bufsz) + return NULL; + } +} + +/* + * Read a 32-bit value from the COMP_REGS region at @offset (HDM-relative). + */ +static uint32_t comp_regs_read32(struct vfio_pci_device *dev, + uint32_t region_idx, uint64_t offset) +{ + uint32_t val; + loff_t pos =3D (loff_t)VFIO_PCI_INDEX_TO_OFFSET(region_idx) + offset; + ssize_t r; + + r =3D pread(dev->fd, &val, sizeof(val), pos); + if (r !=3D sizeof(val)) + return ~0u; + return val; +} + +/* + * Write a 32-bit value to the COMP_REGS region at @offset. + */ +static void comp_regs_write32(struct vfio_pci_device *dev, + uint32_t region_idx, uint64_t offset, + uint32_t val) +{ + loff_t pos =3D (loff_t)VFIO_PCI_INDEX_TO_OFFSET(region_idx) + offset; + pwrite(dev->fd, &val, sizeof(val), pos); +} + +/* + * Find the CXL DVSEC capability base in config space. + * Walks the extended capability list (starting at 0x100). + * Returns the config-space offset of the DVSEC header, or 0. + */ +#define PCI_DVSEC_VENDOR_ID_CXL 0x1e98 +#define PCI_DVSEC_ID_CXL_DEVICE 0x0000 +#define PCI_EXT_CAP_ID_DVSEC 0x23 + +static uint16_t find_cxl_dvsec(struct vfio_pci_device *dev) +{ + uint16_t pos =3D PCI_CFG_SPACE_SIZE; /* 0x100 */ + int iter =3D 0; + + while (pos && iter++ < 64) { + uint32_t hdr =3D vfio_pci_config_readl(dev, pos); + uint32_t hdr1, hdr2; + uint16_t cap_id =3D hdr & 0xffff; + uint16_t next =3D (hdr >> 20) & 0xffc; + + if (cap_id =3D=3D PCI_EXT_CAP_ID_DVSEC) { + hdr1 =3D vfio_pci_config_readl(dev, pos + 4); + hdr2 =3D vfio_pci_config_readl(dev, pos + 8); + /* + * PCIe DVSEC Header 1 layout (Table 9-16): + * Bits [15: 0] =3D DVSEC Vendor ID + * Bits [19:16] =3D DVSEC Revision + * Bits [31:20] =3D DVSEC Length + * DVSEC Header 2 layout: + * Bits [15: 0] =3D DVSEC ID + */ + if ((hdr1 & 0xffff) =3D=3D PCI_DVSEC_VENDOR_ID_CXL && + (hdr2 & 0xffff) =3D=3D PCI_DVSEC_ID_CXL_DEVICE) + return pos; + } + pos =3D next; + } + return 0; +} + + +/* ------------------------------------------------------------------ */ +/* Fixture */ +/* ------------------------------------------------------------------ */ + +FIXTURE(cxl_type2) { + struct iommu *iommu; + struct vfio_pci_device *dev; + + /* Filled in during FIXTURE_SETUP from the CXL cap */ + struct vfio_device_info_cap_cxl cxl_cap; + uint16_t dvsec_base; + + /* DPA mmap pointer (may be NULL if test skips mmap sub-tests) */ + void *dpa_mmap; + size_t dpa_mmap_size; +}; + +FIXTURE_SETUP(cxl_type2) +{ + uint8_t infobuf[512] =3D {}; + struct vfio_device_info *info =3D (void *)infobuf; + const struct vfio_device_info_cap_cxl *cap; + + self->iommu =3D iommu_init(default_iommu_mode); + self->dev =3D vfio_pci_device_init(device_bdf, self->iommu); + + /* Query device info with space for capability chain */ + info->argsz =3D sizeof(infobuf); + ASSERT_EQ(0, ioctl(self->dev->fd, VFIO_DEVICE_GET_INFO, info)); + + if (!(info->flags & VFIO_DEVICE_FLAGS_CXL)) { + printf("Device %s is not a CXL Type-2 device =E2=80=94 skipping\n", + device_bdf); + SKIP(return, "not a CXL Type-2 device"); + } + + cap =3D (const struct vfio_device_info_cap_cxl *) + find_device_cap(infobuf, sizeof(infobuf), + VFIO_DEVICE_INFO_CAP_CXL); + ASSERT_NE(NULL, cap); + memcpy(&self->cxl_cap, cap, sizeof(*cap)); + + self->dvsec_base =3D find_cxl_dvsec(self->dev); + self->dpa_mmap =3D MAP_FAILED; + self->dpa_mmap_size =3D 0; +} + +FIXTURE_TEARDOWN(cxl_type2) +{ + if (self->dpa_mmap !=3D MAP_FAILED && self->dpa_mmap_size) + munmap(self->dpa_mmap, self->dpa_mmap_size); + vfio_pci_device_cleanup(self->dev); + iommu_cleanup(self->iommu); +} + +/* ------------------------------------------------------------------ */ +/* Tests: VFIO_DEVICE_GET_INFO */ +/* ------------------------------------------------------------------ */ + +/* + * CXL and PCI flags must both be set; CAPS must be set since we have a ca= p. + */ +TEST_F(cxl_type2, device_flags) +{ + uint8_t infobuf[512] =3D {}; + struct vfio_device_info *info =3D (void *)infobuf; + + info->argsz =3D sizeof(infobuf); + ASSERT_EQ(0, ioctl(self->dev->fd, VFIO_DEVICE_GET_INFO, info)); + + ASSERT_TRUE(info->flags & VFIO_DEVICE_FLAGS_CXL); + ASSERT_TRUE(info->flags & VFIO_DEVICE_FLAGS_PCI); + ASSERT_TRUE(info->flags & VFIO_DEVICE_FLAGS_CAPS); + + printf("device flags: 0x%x num_regions: %u\n", + info->flags, info->num_regions); +} + +/* + * The CXL capability must report sane HDM and DPA values. + */ +TEST_F(cxl_type2, cxl_cap_fields) +{ + const struct vfio_device_info_cap_cxl *c =3D &self->cxl_cap; + + ASSERT_EQ(VFIO_DEVICE_INFO_CAP_CXL, c->header.id); + ASSERT_EQ(1, c->header.version); + + /* Must have at least one HDM decoder */ + ASSERT_GT(c->hdm_count, 0); + + /* DPA must be non-zero */ + ASSERT_GT(c->dpa_size, 0ULL); + + /* HDM region size must be non-zero and 4-byte aligned */ + ASSERT_GT(c->hdm_regs_size, 0ULL); + ASSERT_EQ(0ULL, c->hdm_regs_size % 4); + + /* Region indices must not be ~0U (sentinel for "not found") */ + ASSERT_NE(~0U, c->dpa_region_index); + ASSERT_NE(~0U, c->comp_regs_region_index); + + /* The two regions must be distinct */ + ASSERT_NE(c->dpa_region_index, c->comp_regs_region_index); + + /* For a pre-committed device both flags must be set */ + if (c->flags & VFIO_CXL_CAP_PRECOMMITTED) + ASSERT_TRUE(c->flags & VFIO_CXL_CAP_COMMITTED); + + printf("hdm_count=3D%u dpa_size=3D0x%llx hdm_regs_size=3D0x%llx " + "dpa_idx=3D%u comp_regs_idx=3D%u flags=3D0x%x\n", + c->hdm_count, (unsigned long long)c->dpa_size, + (unsigned long long)c->hdm_regs_size, + c->dpa_region_index, c->comp_regs_region_index, c->flags); +} + +/* ------------------------------------------------------------------ */ +/* Tests: VFIO_DEVICE_GET_REGION_INFO */ +/* ------------------------------------------------------------------ */ + +/* + * The component register BAR must be hidden: size=3D0 and no flags. + */ +TEST_F(cxl_type2, component_bar_hidden) +{ + struct vfio_region_info reg =3D { .argsz =3D sizeof(reg) }; + + reg.index =3D self->cxl_cap.hdm_regs_bar_index; + ASSERT_EQ(0, ioctl(self->dev->fd, VFIO_DEVICE_GET_REGION_INFO, ®)); + + ASSERT_EQ(0ULL, reg.size); + ASSERT_EQ(0U, reg.flags); + + printf("component BAR %u: size=3D%llu flags=3D0x%x (hidden as expected)\n= ", + self->cxl_cap.hdm_regs_bar_index, + (unsigned long long)reg.size, reg.flags); +} + +/* + * DPA region must be readable, writable, and mmappable. + * Its size must match dpa_size from the CXL capability. + */ +TEST_F(cxl_type2, dpa_region_info) +{ + struct vfio_region_info reg =3D { .argsz =3D sizeof(reg) }; + + reg.index =3D self->cxl_cap.dpa_region_index; + ASSERT_EQ(0, ioctl(self->dev->fd, VFIO_DEVICE_GET_REGION_INFO, ®)); + + ASSERT_EQ(self->cxl_cap.dpa_size, reg.size); + ASSERT_TRUE(reg.flags & VFIO_REGION_INFO_FLAG_READ); + ASSERT_TRUE(reg.flags & VFIO_REGION_INFO_FLAG_WRITE); + ASSERT_TRUE(reg.flags & VFIO_REGION_INFO_FLAG_MMAP); + + printf("DPA region: size=3D0x%llx offset=3D0x%llx flags=3D0x%x\n", + (unsigned long long)reg.size, + (unsigned long long)reg.offset, reg.flags); +} + +/* + * COMP_REGS region must be readable and writable but not mmappable. + * Its size must match hdm_regs_size from the CXL capability. + */ +TEST_F(cxl_type2, comp_regs_region_info) +{ + struct vfio_region_info reg =3D { .argsz =3D sizeof(reg) }; + + reg.index =3D self->cxl_cap.comp_regs_region_index; + ASSERT_EQ(0, ioctl(self->dev->fd, VFIO_DEVICE_GET_REGION_INFO, ®)); + + ASSERT_EQ(self->cxl_cap.hdm_regs_size, reg.size); + ASSERT_TRUE(reg.flags & VFIO_REGION_INFO_FLAG_READ); + ASSERT_TRUE(reg.flags & VFIO_REGION_INFO_FLAG_WRITE); + ASSERT_FALSE(reg.flags & VFIO_REGION_INFO_FLAG_MMAP); + + printf("COMP_REGS region: size=3D0x%llx offset=3D0x%llx flags=3D0x%x\n", + (unsigned long long)reg.size, + (unsigned long long)reg.offset, reg.flags); +} + +/* ------------------------------------------------------------------ */ +/* Tests: DPA region mmap */ +/* ------------------------------------------------------------------ */ + +/* + * mmap() the DPA region and verify the first page can be read. + * The region uses lazy fault insertion so the first access triggers the + * vfio_cxl_region_page_fault path. + */ +TEST_F(cxl_type2, dpa_mmap_fault) +{ + struct vfio_region_info reg =3D { .argsz =3D sizeof(reg) }; + size_t map_size; + void *ptr; + volatile uint8_t *p; + uint8_t val; + + reg.index =3D self->cxl_cap.dpa_region_index; + ASSERT_EQ(0, ioctl(self->dev->fd, VFIO_DEVICE_GET_REGION_INFO, ®)); + + /* Map just the first 2MB or the full region, whichever is smaller */ + map_size =3D (size_t)reg.size < (size_t)(2 * SZ_1M) + ? (size_t)reg.size : (size_t)(2 * SZ_1M); + + ptr =3D mmap(NULL, map_size, PROT_READ | PROT_WRITE, + MAP_SHARED, self->dev->fd, (off_t)reg.offset); + ASSERT_NE(MAP_FAILED, ptr); + + self->dpa_mmap =3D ptr; + self->dpa_mmap_size =3D map_size; + + /* First access =E2=80=94 triggers the page fault handler */ + p =3D (volatile uint8_t *)ptr; + val =3D *p; + (void)val; + + printf("DPA mmap: ptr=3D%p size=3D0x%zx first byte=3D0x%02x\n", + ptr, map_size, (unsigned)val); + + /* Write a pattern and read it back */ + *p =3D 0xab; + ASSERT_EQ(0xab, *p); +} + +/* + * mmap() beyond dpa_size must fail with EINVAL. + */ +TEST_F(cxl_type2, dpa_mmap_overflow) +{ + struct vfio_region_info reg =3D { .argsz =3D sizeof(reg) }; + void *ptr; + + reg.index =3D self->cxl_cap.dpa_region_index; + ASSERT_EQ(0, ioctl(self->dev->fd, VFIO_DEVICE_GET_REGION_INFO, ®)); + + ptr =3D mmap(NULL, (size_t)reg.size + SZ_4K, PROT_READ, + MAP_SHARED, self->dev->fd, (off_t)reg.offset); + ASSERT_EQ(MAP_FAILED, ptr); + + printf("mmap beyond dpa_size correctly failed (errno=3D%d)\n", errno); +} + +/* + * mmap() of the COMP_REGS region (no MMAP flag) must fail. + */ +TEST_F(cxl_type2, comp_regs_no_mmap) +{ + struct vfio_region_info reg =3D { .argsz =3D sizeof(reg) }; + void *ptr; + + reg.index =3D self->cxl_cap.comp_regs_region_index; + ASSERT_EQ(0, ioctl(self->dev->fd, VFIO_DEVICE_GET_REGION_INFO, ®)); + + ptr =3D mmap(NULL, (size_t)reg.size, PROT_READ, + MAP_SHARED, self->dev->fd, (off_t)reg.offset); + ASSERT_EQ(MAP_FAILED, ptr); + + printf("mmap of COMP_REGS correctly failed (errno=3D%d)\n", errno); +} + +/* ------------------------------------------------------------------ */ +/* Tests: COMP_REGS region (HDM decoder emulation) */ +/* ------------------------------------------------------------------ */ + +/* + * Reading HDM Capability (offset 0x00) must return a non-zero value + * consistent with at least one decoder being present. + * Bits [3:0] encode the HDM decoder count. + */ +TEST_F(cxl_type2, hdm_cap_read) +{ + uint32_t cap; + uint32_t idx =3D self->cxl_cap.comp_regs_region_index; + + cap =3D comp_regs_read32(self->dev, idx, HDM_CAP_OFFSET); + ASSERT_NE(~0u, cap); + + /* Lower 4 bits encode the number of decoders */ + ASSERT_GE((int)(cap & 0xf), (int)self->cxl_cap.hdm_count - 1); + + printf("HDM Capability register: 0x%08x decoder_count_field=3D%u\n", + cap, cap & 0xf); +} + +/* + * Writing the HDM Capability register (RO) must be silently discarded. + */ +TEST_F(cxl_type2, hdm_cap_ro) +{ + uint32_t idx =3D self->cxl_cap.comp_regs_region_index; + uint32_t before, after; + + before =3D comp_regs_read32(self->dev, idx, HDM_CAP_OFFSET); + comp_regs_write32(self->dev, idx, HDM_CAP_OFFSET, 0xdeadbeef); + after =3D comp_regs_read32(self->dev, idx, HDM_CAP_OFFSET); + + ASSERT_EQ(before, after); + printf("HDM Capability: write discarded (before=3D0x%08x after=3D0x%08x)\= n", + before, after); +} + +/* + * Writing Global Control (offset 0x04) with reserved bits set must + * result in those bits being cleared in the stored value. + * HDM Decoder Enable (bit 1) is a genuine RW bit. + */ +TEST_F(cxl_type2, hdm_global_ctrl_reserved_bits) +{ + uint32_t idx =3D self->cxl_cap.comp_regs_region_index; + uint32_t readback; + + /* Write all-ones =E2=80=94 reserved bits GENMASK(31,2) must be cleared */ + comp_regs_write32(self->dev, idx, HDM_GLOBAL_CTRL_OFFSET, 0xffffffff); + readback =3D comp_regs_read32(self->dev, idx, HDM_GLOBAL_CTRL_OFFSET); + + ASSERT_EQ(0u, readback & HDM_GLOBAL_CTRL_RESERVED_MASK); + printf("HDM Global Control reserved bits cleared: 0x%08x\n", readback); + + /* Restore to 0 */ + comp_regs_write32(self->dev, idx, HDM_GLOBAL_CTRL_OFFSET, 0); +} + +/* + * Writing decoder N BASE_LO with reserved bits [27:0] set must + * result in those bits being cleared. [31:28] are significant. + */ +TEST_F(cxl_type2, hdm_decoder_base_lo_reserved) +{ + uint32_t idx =3D self->cxl_cap.comp_regs_region_index; + uint64_t ctrl_off =3D HDM_DECODER_FIRST_OFFSET + HDM_DECODER_BASE_LO; + uint32_t readback; + + /* Writing 0xffffffff: only [31:28] should survive */ + comp_regs_write32(self->dev, idx, ctrl_off, 0xffffffff); + readback =3D comp_regs_read32(self->dev, idx, ctrl_off); + + ASSERT_EQ(0u, readback & HDM_BASE_LO_RESERVED_MASK); + ASSERT_EQ(0xf0000000u, readback); + + printf("HDM decoder 0 BASE_LO: reserved bits cleared -> 0x%08x\n", + readback); + + /* Clean up */ + comp_regs_write32(self->dev, idx, ctrl_off, 0); +} + +/* + * Unaligned (sub-dword) access to the COMP_REGS region must fail with EIN= VAL. + */ +TEST_F(cxl_type2, comp_regs_unaligned_access) +{ + struct vfio_region_info reg =3D { .argsz =3D sizeof(reg) }; + uint8_t byte_val =3D 0; + loff_t pos; + ssize_t r; + + reg.index =3D self->cxl_cap.comp_regs_region_index; + ASSERT_EQ(0, ioctl(self->dev->fd, VFIO_DEVICE_GET_REGION_INFO, ®)); + + /* Attempt 1-byte read at offset 1 (unaligned) */ + pos =3D (loff_t)reg.offset + 1; + r =3D pread(self->dev->fd, &byte_val, 1, pos); + ASSERT_EQ(-1, (int)r); + ASSERT_EQ(EINVAL, errno); + + printf("Unaligned COMP_REGS access correctly failed (errno=3D%d)\n", + errno); +} + +/* + * Writing HDM decoder N CTRL with COMMIT=3D1 must cause COMMITTED to + * be set immediately in the emulated shadow state (no hardware wait). + * This models the QEMU notify_change path. + */ +TEST_F(cxl_type2, hdm_ctrl_commit_to_committed) +{ + uint32_t idx =3D self->cxl_cap.comp_regs_region_index; + /* + * Use decoder 0; write BASE/SIZE first so the decoder is in a + * plausible state before committing. Use 256MB alignment + * (bit 28 =3D 1 in SIZE_LO) to satisfy the reserved-bit rule. + */ + uint64_t base_lo_off =3D HDM_DECODER_FIRST_OFFSET + HDM_DECODER_BASE_LO; + uint64_t base_hi_off =3D HDM_DECODER_FIRST_OFFSET + HDM_DECODER_BASE_HI; + uint64_t size_lo_off =3D HDM_DECODER_FIRST_OFFSET + HDM_DECODER_SIZE_LO; + uint64_t size_hi_off =3D HDM_DECODER_FIRST_OFFSET + HDM_DECODER_SIZE_HI; + uint64_t ctrl_off =3D HDM_DECODER_FIRST_OFFSET + HDM_DECODER_CTRL; + uint32_t ctrl_readback; + + /* Skip if COMMITTED is already set (pre-committed decoder) */ + ctrl_readback =3D comp_regs_read32(self->dev, idx, ctrl_off); + if (ctrl_readback & HDM_CTRL_COMMITTED) { + printf("Decoder 0 already committed (ctrl=3D0x%08x); " + "skipping COMMIT test\n", ctrl_readback); + SKIP(return, "decoder already committed"); + } + + /* Set BASE and SIZE to plausible 256MB-aligned values */ + comp_regs_write32(self->dev, idx, base_lo_off, 0x10000000); /* 256MB boun= dary */ + comp_regs_write32(self->dev, idx, base_hi_off, 0); + comp_regs_write32(self->dev, idx, size_lo_off, 0x10000000); /* 256MB */ + comp_regs_write32(self->dev, idx, size_hi_off, 0); + + /* Write COMMIT=3D1 */ + comp_regs_write32(self->dev, idx, ctrl_off, HDM_CTRL_COMMIT); + + /* COMMITTED must be set immediately in the shadow */ + ctrl_readback =3D comp_regs_read32(self->dev, idx, ctrl_off); + ASSERT_TRUE(ctrl_readback & HDM_CTRL_COMMITTED); + + printf("HDM decoder 0 CTRL after COMMIT=3D1: 0x%08x (COMMITTED set)\n", + ctrl_readback); + + /* Writing COMMIT=3D0 must clear COMMITTED */ + comp_regs_write32(self->dev, idx, ctrl_off, 0); + ctrl_readback =3D comp_regs_read32(self->dev, idx, ctrl_off); + ASSERT_FALSE(ctrl_readback & HDM_CTRL_COMMITTED); + + printf("HDM decoder 0 CTRL after COMMIT=3D0: 0x%08x (COMMITTED cleared)\n= ", + ctrl_readback); +} + +/* + * Writing CTRL reserved bits must result in them being cleared. + */ +TEST_F(cxl_type2, hdm_ctrl_reserved_bits) +{ + uint32_t idx =3D self->cxl_cap.comp_regs_region_index; + uint64_t ctrl_off =3D HDM_DECODER_FIRST_OFFSET + HDM_DECODER_CTRL; + uint32_t ctrl_before, ctrl_after; + + ctrl_before =3D comp_regs_read32(self->dev, idx, ctrl_off); + + /* + * Write all-ones; reserved bits (BIT(15) and GENMASK(31,28)) must + * be cleared in the readback. Skip if COMMIT_LOCK is already set. + */ + if (ctrl_before & BIT(8)) { + printf("Decoder 0 COMMIT_LOCK set; skipping reserved-bit test\n"); + SKIP(return, "COMMIT_LOCK set"); + } + + comp_regs_write32(self->dev, idx, ctrl_off, 0xffffffff); + ctrl_after =3D comp_regs_read32(self->dev, idx, ctrl_off); + + ASSERT_EQ(0u, ctrl_after & HDM_CTRL_RESERVED_MASK); + + printf("HDM CTRL reserved bits cleared: before=3D0x%08x after=3D0x%08x\n", + ctrl_before, ctrl_after); + + /* Restore */ + comp_regs_write32(self->dev, idx, ctrl_off, ctrl_before); +} + +/* ------------------------------------------------------------------ */ +/* Tests: DVSEC configuration space emulation */ +/* ------------------------------------------------------------------ */ + +/* + * CXL Control (DVSEC offset 0x0c): IO_Enable (bit 1) must always read 1. + */ +TEST_F(cxl_type2, dvsec_control_io_enable_ro) +{ + uint16_t dvsec =3D self->dvsec_base; + uint16_t ctrl; + + if (!dvsec) + SKIP(return, "CXL DVSEC not found in config space"); + + /* Read current value */ + ctrl =3D vfio_pci_config_readw(self->dev, dvsec + CXL_DVSEC_CONTROL_OFFSE= T); + ASSERT_TRUE(ctrl & CXL_CTRL_IO_ENABLE); + + /* Write with IO_Enable cleared =E2=80=94 it must stay set */ + vfio_pci_config_writew(self->dev, dvsec + CXL_DVSEC_CONTROL_OFFSET, + ctrl & ~CXL_CTRL_IO_ENABLE); + ctrl =3D vfio_pci_config_readw(self->dev, dvsec + CXL_DVSEC_CONTROL_OFFSE= T); + ASSERT_TRUE(ctrl & CXL_CTRL_IO_ENABLE); + + printf("DVSEC CXL Control: IO_Enable always 1 (ctrl=3D0x%04x)\n", ctrl); +} + +/* + * CXL Status (DVSEC offset 0x0e): Bit 14 (Viral_Status) is RW1CS =E2=80= =94 + * writing 1 clears it; writing 0 leaves it unchanged. + */ +TEST_F(cxl_type2, dvsec_status_rw1cs) +{ + uint16_t dvsec =3D self->dvsec_base; + uint16_t status_before, status_after; + + if (!dvsec) + SKIP(return, "CXL DVSEC not found in config space"); + + status_before =3D vfio_pci_config_readw(self->dev, + dvsec + CXL_DVSEC_STATUS_OFFSET); + printf("DVSEC CXL Status before: 0x%04x\n", status_before); + + /* Write 0 to RW1C bit =E2=80=94 value must not change */ + vfio_pci_config_writew(self->dev, dvsec + CXL_DVSEC_STATUS_OFFSET, + status_before & ~CXL_STATUS_RW1C_BIT); + status_after =3D vfio_pci_config_readw(self->dev, + dvsec + CXL_DVSEC_STATUS_OFFSET); + ASSERT_EQ(status_before & CXL_STATUS_RW1C_BIT, + status_after & CXL_STATUS_RW1C_BIT); + + /* Write 1 to RW1C bit =E2=80=94 bit must be cleared */ + vfio_pci_config_writew(self->dev, dvsec + CXL_DVSEC_STATUS_OFFSET, + CXL_STATUS_RW1C_BIT); + status_after =3D vfio_pci_config_readw(self->dev, + dvsec + CXL_DVSEC_STATUS_OFFSET); + ASSERT_FALSE(status_after & CXL_STATUS_RW1C_BIT); + + printf("DVSEC CXL Status RW1C cleared: 0x%04x -> 0x%04x\n", + status_before, status_after); +} + +/* + * CXL Lock (DVSEC offset 0x14): + * - Reserved bits GENMASK(15,1) must be cleared. + * - Once locked, CXL Control writes must be discarded. + */ +TEST_F(cxl_type2, dvsec_lock_semantics) +{ + uint16_t dvsec =3D self->dvsec_base; + uint16_t lock_val, ctrl_before, ctrl_after; + + if (!dvsec) + SKIP(return, "CXL DVSEC not found in config space"); + + lock_val =3D vfio_pci_config_readw(self->dev, + dvsec + CXL_DVSEC_LOCK_OFFSET); + if (lock_val & CXL_LOCK_BIT) { + printf("CXL Lock already set; skipping lock semantics test\n"); + SKIP(return, "CONFIG_LOCK already set"); + } + + /* Write reserved bits =E2=80=94 they must be cleared on readback */ + vfio_pci_config_writew(self->dev, dvsec + CXL_DVSEC_LOCK_OFFSET, + CXL_LOCK_RESERVED_MASK); + lock_val =3D vfio_pci_config_readw(self->dev, + dvsec + CXL_DVSEC_LOCK_OFFSET); + ASSERT_EQ(0u, lock_val & CXL_LOCK_RESERVED_MASK); + ASSERT_FALSE(lock_val & CXL_LOCK_BIT); + + printf("Lock reserved bits cleared correctly\n"); + + /* + * Save the current Control value, then set Lock. + * Any subsequent write to Control must be discarded. + */ + ctrl_before =3D vfio_pci_config_readw(self->dev, + dvsec + CXL_DVSEC_CONTROL_OFFSET); + + vfio_pci_config_writew(self->dev, dvsec + CXL_DVSEC_LOCK_OFFSET, + CXL_LOCK_BIT); + lock_val =3D vfio_pci_config_readw(self->dev, + dvsec + CXL_DVSEC_LOCK_OFFSET); + ASSERT_TRUE(lock_val & CXL_LOCK_BIT); + printf("CXL Lock set (lock=3D0x%04x)\n", lock_val); + + /* Attempt to modify Control after locking =E2=80=94 must be discarded */ + vfio_pci_config_writew(self->dev, dvsec + CXL_DVSEC_CONTROL_OFFSET, + ctrl_before ^ 0x0001); /* flip Cache_Enable */ + ctrl_after =3D vfio_pci_config_readw(self->dev, + dvsec + CXL_DVSEC_CONTROL_OFFSET); + ASSERT_EQ(ctrl_before, ctrl_after); + + printf("CXL Control write after lock discarded: " + "before=3D0x%04x after=3D0x%04x\n", ctrl_before, ctrl_after); +} + +/* + * CXL Control reserved bits (13, 15) must be cleared on write. + */ +TEST_F(cxl_type2, dvsec_control_reserved_bits) +{ + uint16_t dvsec =3D self->dvsec_base; + uint16_t lock_val, ctrl_after; + + if (!dvsec) + SKIP(return, "CXL DVSEC not found in config space"); + + lock_val =3D vfio_pci_config_readw(self->dev, + dvsec + CXL_DVSEC_LOCK_OFFSET); + if (lock_val & CXL_LOCK_BIT) + SKIP(return, "CONFIG_LOCK already set; cannot test Control"); + + vfio_pci_config_writew(self->dev, dvsec + CXL_DVSEC_CONTROL_OFFSET, + 0xffff); + ctrl_after =3D vfio_pci_config_readw(self->dev, + dvsec + CXL_DVSEC_CONTROL_OFFSET); + + /* Bits 13 and 15 must be cleared */ + ASSERT_FALSE(ctrl_after & BIT(13)); + ASSERT_FALSE(ctrl_after & BIT(15)); + + printf("DVSEC Control reserved bits cleared: 0x%04x\n", ctrl_after); +} + +/* ------------------------------------------------------------------ */ +/* main */ +/* ------------------------------------------------------------------ */ + +int main(int argc, char *argv[]) +{ + device_bdf =3D vfio_selftests_get_bdf(&argc, argv); + return test_harness_run(argc, argv); +} --=20 2.25.1 From nobody Tue Apr 7 19:54:44 2026 Received: from CY7PR03CU001.outbound.protection.outlook.com (mail-westcentralusazon11010050.outbound.protection.outlook.com [40.93.198.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7869A3750B1; Wed, 11 Mar 2026 20:37:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.93.198.50 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773261475; cv=fail; b=da8Yp8aNcfjqYnzUM7nmMqBBMEHu4vSxkEyUluWtF1jGZSQ2WGzTgcHniLrZUlncvEjseJMZRck1nmMFz5osxl3ByUFkjogatVt0DztJ/1RgAAaGCGGZi3ejqnJdxehs4CCDecVr+HFHP+WsoLQUHLT0/OEVwL2Odly024BdakM= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773261475; c=relaxed/simple; bh=NTolRLeI1yLv4hGrM89eY3l7ZJM2oN6dds8QlrhFJq8=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=edfVQcQMPy3Ttz+XuV5WQqd9tPgKX+vr2CqKf0bn55KpoIMjg6lkvqheRYAQljARsjsSo6/tC+2iM1IY2w8FeDc86VOpNjm8Mpqgqq8PwqGlucR6hpepypwLokd2JhzLrBPu9POp9nukEDjS15xxgMjJy8vB2hULYorFYhtOGEw= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=qbafMvPn; arc=fail smtp.client-ip=40.93.198.50 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="qbafMvPn" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=h+E+3k6CbOU+bP5neFvkBZH6a5zsLwpeMwm3WNGGUyyiZsey1IBZhogS0LwgOvH+0vL9V0bTghT1Y3qMGbYAxQsnw+Dbmg1llQ+5ghXmDzynxhzxoN016rQevHiYvNS3f2x3+u1+UcM1W2HA0RJ6GANOCHSAtd6vxhTNt2EzVEKs0eKqCkj+ZEooC/Jv6C9DpnUDLxqt+rdaQawccGwXrFifQoWLYXWqfg2BzpCFkxi2ezVYGG3n5A/s7m+QGXThZm9tOKJFn0SPMHGiCRX5YhiVTitwg4OjGd3aIijrZ3+L+/KshBdDtkH107/r9ImKFYNF9o3EUU94f6Bh/b08QA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=U6JrNAE9RbgweA7MhIoYm3P7ydsxCvg1wrUeUvUmNpM=; b=O6zoJ5GsorgczVOPUBzc9mBjxxHQaHTwhteHkn1AgeDny5GkCI/2PsnQCnHrKSlcWK0SnDgDMAUSUXEd7/L1M4ZxqjpDvmtkl9wqUVY2P5C0QQ+dyzhSPXmwn1BdmQMlqAVw5q1j+Ojm/kl6NhfvKUfCQ6kXrgy1DO5wkgNpB6J3q1BMZmj/O+X9HIhLMSEhSJ+0qnS7V2Y/SCnJ09Dyoa+QnWD+7oHihwu8YhIaneXxoecQc2WEPToYmWhzF80LCQCgP8Y01KPLSNGKO2QkQHqSrhlip010cyFOcKsIQ6a+b6y37DEqbcCxYtgr7h3Ce1nzVXTVNut0mTb8xaVEtw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=U6JrNAE9RbgweA7MhIoYm3P7ydsxCvg1wrUeUvUmNpM=; b=qbafMvPnzCCgZrzP5fDBjvOAksydiP8u4QPqi/Tx1NXDWGdiUiEYQdxAQGA/iZZ0fTwcwSs/gKzT7Znge0/HnAd+Fd4IcZsBccYKxBnRjFLrcpCymDqpLbqnlVesCKx/OQ095vkWSouqP4m2xcSU8Lk+ek5WZkznhVluPdQFgQN7RfZN3iVFihhnPCxf/n4gyLVrweqKQ2mJjF7MKeVg6SBpVGlr2AbJ3XtJPwObwVzOoh5WFp3Kx4x0UhUiNvaby9gNviO1i4cniLke+y4w3EfaSrQzRuE09vkZzZanIlEnaaui1qICxdlX5BJwvZyRJNrwToI51qH5YwXr9+kLxg== Received: from DM6PR05CA0038.namprd05.prod.outlook.com (2603:10b6:5:335::7) by DS7PR12MB5863.namprd12.prod.outlook.com (2603:10b6:8:7a::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9700.11; Wed, 11 Mar 2026 20:37:49 +0000 Received: from DS2PEPF00003448.namprd04.prod.outlook.com (2603:10b6:5:335:cafe::2c) by DM6PR05CA0038.outlook.office365.com (2603:10b6:5:335::7) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9678.25 via Frontend Transport; Wed, 11 Mar 2026 20:37:49 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by DS2PEPF00003448.mail.protection.outlook.com (10.167.17.75) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9678.18 via Frontend Transport; Wed, 11 Mar 2026 20:37:49 +0000 Received: from rnnvmail203.nvidia.com (10.129.68.9) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Wed, 11 Mar 2026 13:37:28 -0700 Received: from rnnvmail201.nvidia.com (10.129.68.8) by rnnvmail203.nvidia.com (10.129.68.9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Wed, 11 Mar 2026 13:37:27 -0700 Received: from nvidia-4028GR-scsim.nvidia.com (10.127.8.11) by mail.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20 via Frontend Transport; Wed, 11 Mar 2026 13:37:21 -0700 From: To: , , , , , , , , , , , , , , , , , CC: , , , , , , , Subject: [PATCH 20/20] selftests/vfio: Fix VLA initialisation in vfio_pci_irq_set() Date: Thu, 12 Mar 2026 02:04:40 +0530 Message-ID: <20260311203440.752648-21-mhonap@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20260311203440.752648-1-mhonap@nvidia.com> References: <20260311203440.752648-1-mhonap@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS2PEPF00003448:EE_|DS7PR12MB5863:EE_ X-MS-Office365-Filtering-Correlation-Id: db26646c-4b3c-4e7e-ad69-08de7fae0fa6 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|1800799024|376014|82310400026|36860700016|921020|22082099003|18002099003|56012099003; X-Microsoft-Antispam-Message-Info: 3etJ6HkC5hotjvN1byoPsWPng9DPn/Z/trUj91skOo8SQ0JKx4w6JLBVueg2fjOiDAtdRQ3kpFtvORa77g8FRSHdmNV53GPsZjanG57VbxCpU0dTGrTXLOo65QxDjIXoCx3fDPUwsCsjwjQXjv5F/9h/UOY97RnsNbbTmAtT59PD3z4sLe8Vs2kyE4s4lr2BqUykG894LeLDba8Ls9aVhmrmcSUlh1RjF3wJe1BC/Q8U8ZeFGCxJSPCd31sSE+PfSNEvLhBlBo//z8xdFuASPoQ+EXKsvni5Fui9e1J+Ni8RL+XOzeIwQiaeua+9bF0H4UVtLPq4EP8wDXE6GUPu5cQAH9WMIW6AsO5lFm+6zNlO/LsOW5+NOLaeTOQbzoAgWu5xxH/cvlK6h8bXA6Npywi2PhE/k7xCWEaEftQa3/LGJG4lvEWwjXLslaNuPkcK2kxK7aUXk6P1TANODJUdOViNoEl0ELxLIr5FseuHpEP5znfT4c+H0SMY/SJhlwQ830qtAirSmKcv4bQ4YAkz+Q2FFGGEGsbswAcIQq5nr7uZgYs9YghGD0Jn6T8Tu44IzK8uP8kYGiFkai5cFrIwbe9HKw3+HFim5+gq9JgSKXQgbbuYQRY/8akueFZO0da+mRc9ZhFdyRk8DRTWCohjH/M497SA9pTmAjpQhkANgsDWDHQJL25B95zNm5VrgzAbmCmoy9Z9LgSmDViTz3hBusfMwdU8CXA81iTcHmlsLJ4RPksBgclRH24LeA1sW+Be95GL8dF51m2NVSwCz7fV9ATeDW7Jz3eCuMjmLyvEXAChM1cYjXsQPxVlXP2AFCoA X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(7416014)(1800799024)(376014)(82310400026)(36860700016)(921020)(22082099003)(18002099003)(56012099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: OE7fDMRQVIrV5Ad1HDp6EsnGGhC1Q63LXbF4tbgjBGCM1yYu7pIuDgSnqy7YrOmoVgdHQFcmrsyE9D9IjhR1jN12aGmB7HqFYNRRtYXzWArR/G65T+LiGGp0rM8dQjuFLylqo90tldvzCh2siHp0FcN+xoMH2kAj0KIZs7Osg1I38m7shMhEuOdZC+Z3Rn88fPSTumeUotGwYewRx/2AwQIN7KPbbiRiimFHi8OQ2aA5sCfpgmwaIqeJdw2AxioratoGmEkAa7SAqur7vDr1YrLADdX8npzb93/1ye12lGnRWRx4P7ydBrf2XTWx7hM/kVIior45SGxchpk+VGmIuBLsYAC7taIiLxWU1c53j3AwXweCijLSGSk7A21JuXJ4DKNv+CHz2qlvlCN2ldXGz7/5UUwjcI0N65PnhFM+aCZKMOrYdNI3JvJIixgyQm6L X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Mar 2026 20:37:49.5373 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: db26646c-4b3c-4e7e-ad69-08de7fae0fa6 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DS2PEPF00003448.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR12MB5863 Content-Type: text/plain; charset="utf-8" From: Manish Honap C does not permit initialiser expressions on variable-length arrays. vfio_pci_irq_set() declared u8 buf[sizeof(struct vfio_irq_set) + sizeof(int) * count] =3D {}; where count is a function parameter, making buf a VLA. GCC rejects this with "variable-sized object may not be initialized". Replace the initialiser with an explicit memset() immediately after the declaration. Fixes: 19faf6fd969c2 ("vfio: selftests: Add a helper library for VFIO selft= ests") Signed-off-by: Manish Honap --- tools/testing/selftests/vfio/lib/vfio_pci_device.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/tools/testing/selftests/vfio/lib/vfio_pci_device.c b/tools/tes= ting/selftests/vfio/lib/vfio_pci_device.c index fac4c0ecadef..3258e814f450 100644 --- a/tools/testing/selftests/vfio/lib/vfio_pci_device.c +++ b/tools/testing/selftests/vfio/lib/vfio_pci_device.c @@ -26,8 +26,10 @@ static void vfio_pci_irq_set(struct vfio_pci_device *device, u32 index, u32 vector, u32 count, int *fds) { - u8 buf[sizeof(struct vfio_irq_set) + sizeof(int) * count] =3D {}; + u8 buf[sizeof(struct vfio_irq_set) + sizeof(int) * count]; struct vfio_irq_set *irq =3D (void *)&buf; + + memset(buf, 0, sizeof(buf)); int *irq_fds =3D (void *)&irq->data; =20 irq->argsz =3D sizeof(buf); --=20 2.25.1