From nobody Sat Feb 7 12:25:21 2026 Received: from PH7PR06CU001.outbound.protection.outlook.com (mail-westus3azon11010065.outbound.protection.outlook.com [52.101.201.65]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2029827815D; Tue, 9 Dec 2025 16:51:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.201.65 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1765299090; cv=fail; b=FNmCESsUdtRdBMXoJeij5fBC7gObzShqNGDJMjOAi8E5Vt/IcYGjfvYYQQDpBAUF7l4S0FJEjzarSnO63/cz4JWzvhgW9Usyh0iZ9Jg5d+wDKCozOCR2z99Db3wpy59FYsxKcQ5tBTB0zorOz/hz3nUOBDB9h72mNB6IrhbCVGg= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1765299090; c=relaxed/simple; bh=BSv2jL6FThmkCnFMNsussK9NCtrQZwxpl5QMypDAfR0=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=gOH5X19KD4ETEljFG8IBWjwIj69mtR6xvuVmmc1twFPsMsIPjF396mDGv7nJcLd6iyExD7clLR/IA9yVrF5BPJD3U41OMCiaeD9dnUROpgeJ1ZpajgRmJHwGRoYIgQnNSEOnIRWIOEUe2NhDbdlI0BQ9bA6NDSqwH8woKof8SzI= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=UsB9SiP/; arc=fail smtp.client-ip=52.101.201.65 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="UsB9SiP/" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=fAiqsLGrgEUrRT19qmHIMftiHlq9h2QXs6nCYvfFaYax5DjwJIr1P9AEXyypEz2vl+xMt/f3ZvplJK4w1Pj1v0wUAMV6OzOVRCgbLKAGHCHxVLX6DFTI6x8oCClwb4LQfCapJ5y5yUzSYxdPF3aM/LX706w2DiA7zqetd1WrhM0YxV+IAS09/GW5Hen3qJEHIDEFUNRQM+ar4zoTMsFsKkeKBgOcPZuAoTbwGv8TWdxhc9TjMT7h4DHWwbjBDlBoOWWq0HA2nDtD8nfLqfIiR9MANltiSX2WF7gBjBv47V6XWh0TLQa7i9i6XQ95XI4koLddOkJHiPyTZm2ZmcEh6w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=vk4lSlXl49u7nTsi+pmSg/faOO08TRbUKKX+vWGY9Is=; b=oHfReoDNJJUUwMVFXMOjDc6LImlkYI+rSALseTmDdUmTcXcgxcNAkZyKAzflFR4rIMgl4Mca5t12cWr1LFJ2/3kZsSpX7DYpk+UM2Uo8nGwG7fjKQQdXNLsEquk6vSQPvcBctFsIpwGgiNLOOaBFrmcdHg/vIKzpjxIjWL5Ums4YkJ0wSrmKeY2PGyrYZHNQIjf7bzGWAL9/nVa4S9uS6aUzR7l16+efxZYXuV4X0vZRVisMF//rPwgscLYrpg7Kh90dmadqQyHfNZssdc9eXRVq6cB11Y2aUgYXb6GNtYZO3mwsTwUuPCkwvr1miQ1X0wQcoK2j+/NnHPVZ4wxd/g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=vk4lSlXl49u7nTsi+pmSg/faOO08TRbUKKX+vWGY9Is=; b=UsB9SiP/tQ99v7ivgNauO9wE5VSfC/WuVxkFmM77wXci6gR5KOnXyMB/23I2ce45eaAwEMH17ju3FNc1s/RxnXby7qw6twycVBiutaWLPl2oIC4UYa7Yap5Eb0vaV8xG4QCNKnVbJ08fzs3m2EcTiVk74+raCZ7m/fWjJ20AvbN7BAmN462y4W+vKNp7wIIn0YRXwKzZbNaZPtw1YP+WAwkOMjQ0povJOjRRED8ICfff8aso4LFVWisJamd8TJTB0zRI6nhKaQIZVOtahW7i9bUXLF8ZyyXL+i2D+Lb6mfXWbqQ7sWc5zAHtZlDz2yuh1ti7mSA9dpYdnySx62Gbow== Received: from SJ0PR03CA0246.namprd03.prod.outlook.com (2603:10b6:a03:3a0::11) by MW6PR12MB8835.namprd12.prod.outlook.com (2603:10b6:303:240::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9388.14; Tue, 9 Dec 2025 16:51:18 +0000 Received: from SJ1PEPF00002311.namprd03.prod.outlook.com (2603:10b6:a03:3a0:cafe::45) by SJ0PR03CA0246.outlook.office365.com (2603:10b6:a03:3a0::11) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9388.14 via Frontend Transport; Tue, 9 Dec 2025 16:50:37 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by SJ1PEPF00002311.mail.protection.outlook.com (10.167.242.165) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9388.8 via Frontend Transport; Tue, 9 Dec 2025 16:51:18 +0000 Received: from rnnvmail202.nvidia.com (10.129.68.7) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 9 Dec 2025 08:50:57 -0800 Received: from rnnvmail204.nvidia.com (10.129.68.6) by rnnvmail202.nvidia.com (10.129.68.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 9 Dec 2025 08:50:56 -0800 Received: from nvidia-4028GR-scsim.nvidia.com (10.127.8.11) by mail.nvidia.com (10.129.68.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20 via Frontend Transport; Tue, 9 Dec 2025 08:50:49 -0800 From: To: , , , , , , , , , , , , , , , , , CC: , , , , , , , , , "Li Ming" Subject: [RFC v2 01/15] cxl: factor out cxl_await_range_active() and cxl_media_ready() Date: Tue, 9 Dec 2025 22:20:05 +0530 Message-ID: <20251209165019.2643142-2-mhonap@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20251209165019.2643142-1-mhonap@nvidia.com> References: <20251209165019.2643142-1-mhonap@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ1PEPF00002311:EE_|MW6PR12MB8835:EE_ X-MS-Office365-Filtering-Correlation-Id: 3270ddf5-e6c2-4273-783c-08de37432cd9 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|36860700013|82310400026|1800799024|7416014|376014|921020; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?zPjue+UyZ/+MR3ts9D/fxyr+twyqgIRhGD9myG7PNbcClqTL6FlzW/ovGfPb?= =?us-ascii?Q?orUauXNN9BgHh+r/DJmS/FbBxVrOGjcQJ7ErnqZmbvM53pzbV/Veu1iFSiK6?= =?us-ascii?Q?yo3BTtHJbJl9IMlGcXeQI76PI7BO1HNL+GX4oRwSCUw/+pewjUsieFX0eXwX?= =?us-ascii?Q?2BRp1RcvCRho4Ws9Svthbd4lgmxDOqbsIIvjOVqQ0Lfe/Dl6kYxKJ+lDoAWw?= =?us-ascii?Q?PAGv1L0F1WuZl0DeP236s9abFpovBQ1HErwUT9zhoS/8OXI+Da3gmzAc0Mwt?= =?us-ascii?Q?QITUBUdBwW1nPohBd2C56z3/KK1z/bTF2kSfPd9nYd6p+8LbZ/foelJ1JMPR?= =?us-ascii?Q?t+yNrEbZqte6odaWKiT4PanJLLkJ1wrurOWXAb+7NVSo4yOAtzZDflI/0SOI?= =?us-ascii?Q?8ndeJMofRoD/lXqYVjwESfNKTP54JcjledDFQmci4px8tTBAhKEwQGB885wp?= =?us-ascii?Q?6R2jwejcdb8uTDblwc29H+8zWjdrVifhxg//KjoIHZw2/o5KCQfVem+Nkfvb?= =?us-ascii?Q?kU3OZZo2AL/YsSRwj9lYh5gZHnGEmPKoy0Zv5vN1PNKCaSY3Gh3mGS6SRw9A?= =?us-ascii?Q?Oelpktuf/kLdktkhq9T2Nied4B3IKGsgUPEmrbpVRkifzHSQ81dYmZu95/ES?= =?us-ascii?Q?kHrGKu05uEjT7G1OZ3w+fQ/zTrwDfaFPhA1wMc9hKPS9clt4+1aKBsTtaQNJ?= =?us-ascii?Q?+LTjrpJpp7KOKhrErywQUmmxtJwRzgD5Ufrhgvp5r3bfuSansAUFqyudBG6g?= =?us-ascii?Q?ZRkoxcq6h1Ossmm92+Iz395nHC7aMKVemIAIvZR4d/jdFxak+xrw18MJs8SG?= =?us-ascii?Q?TFwPXqsS8P2r/cndF5Z0gzGCjB1I+SJFg+n2i7+pY9q9dRfTEEbZqgtKCirJ?= =?us-ascii?Q?IlNTpjZF0t8kn0MwA8VBen0jaGfkk7tphG+6qiDAAiyJSOVI45YFVg5wJejM?= =?us-ascii?Q?v3id/xRTzGHlnTPi5YZzvyv1ANzg04wMZXu4Dvd04Zg7AU+8ttqXclhMSg6S?= =?us-ascii?Q?OWLvTbo6jg8DQhJ3RrLMBKRSWgtCO7OgOL3OyfgCjcyv4Q6u9r5SJb+UZv0c?= =?us-ascii?Q?/dSPswOhZdAoPg9edFMlSsYI0iM9LDXVX4wjwDOZUFCAK6IKssphJC/bftvM?= =?us-ascii?Q?CLIJWHmaHOBIrIw910wd5lKUyU43tHy29YHPboHBYmBBtifrYf5FpEkELxnA?= =?us-ascii?Q?mA1GGb2BTf+HZCPHrlYYW249LTxuc+4FZg0lyk82wsALi0skbBNYGEIcRodb?= =?us-ascii?Q?h9F+x639y7S3zNuaacsWPcpej6KeriCqIOoHx4uRGvoWHA4jc3dgCtZhWe58?= =?us-ascii?Q?umq47Fy3mZxXK9c6RzJ677HX/pZ/LluP1wGBirH9H4v27k2hbZxkaxgS3456?= =?us-ascii?Q?C11I2p3srUVbEE7HlL6nIr6nGLylgWUzAVRaWLi8Av990yf3CCfMgMAGj592?= =?us-ascii?Q?0rROXGaJq4CqHP1SLMQeW73VzmhGOVt6zBgKBeytduZf7h/PjFpvvewD18oX?= =?us-ascii?Q?vvTysTsce3qPavUGD1Jhf3b6Nbgrb1TG3qTJI469/6JedareffF3tNpgWcIU?= =?us-ascii?Q?hFRxeWFJlAZzgU3EcQcZd8evueOzum/oGOEqrUG5?= X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(36860700013)(82310400026)(1800799024)(7416014)(376014)(921020);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Dec 2025 16:51:18.6665 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 3270ddf5-e6c2-4273-783c-08de37432cd9 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: SJ1PEPF00002311.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW6PR12MB8835 Content-Type: text/plain; charset="utf-8" From: Zhi Wang Before accessing the CXL device memory after reset/power-on, the driver needs to ensure the device memory media is ready. However, not every CXL device implements the CXL memory device register groups. E.g. a CXL type-2 device. Thus calling cxl_await_media_ready() on these device will lead to a kernel panic. This problem was found when testing the emulated CXL type-2 device without a CXL memory device register. [ 97.662720] BUG: kernel NULL pointer dereference, address: 0000000000000= 000 [ 97.663963] #PF: supervisor read access in kernel mode [ 97.664860] #PF: error_code(0x0000) - not-present page [ 97.665753] PGD 0 P4D 0 [ 97.666198] Oops: Oops: 0000 [#1] PREEMPT SMP NOPTI [ 97.667053] CPU: 8 UID: 0 PID: 7340 Comm: qemu-system-x86 Tainted: G = E 6.11.0-rc2+ #52 [ 97.668656] Tainted: [E]=3DUNSIGNED_MODULE [ 97.669340] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel= -1.16.3-0-ga6ed6b701f0a-prebuilt.qemu.org 04/01/2014 [ 97.671243] RIP: 0010:cxl_await_media_ready+0x1ac/0x1d0 [ 97.672157] Code: e9 03 ff ff ff 0f b7 1d d6 80 31 01 48 8b 7d b8 89 da = 48 c7 c6 60 52 c6 b0 e8 00 46 f6 ff e9 27 ff ff ff 49 8b 86 a0 00 00 00 <48= > 8b 00 83 e0 0c 48 83 f8 04 0f 94 c0 0f b6 c0 8d 44 80 fb e9 0c [ 97.675391] RSP: 0018:ffffb5bac7627c20 EFLAGS: 00010246 [ 97.676298] RAX: 0000000000000000 RBX: 000000000000003c RCX: 00000000000= 00000 [ 97.677527] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 00000000000= 00000 [ 97.678733] RBP: ffffb5bac7627c70 R08: 0000000000000000 R09: 00000000000= 00000 [ 97.679951] R10: 0000000000000000 R11: 0000000000000000 R12: 00000000000= 00000 [ 97.681144] R13: ffff9ef9028a8000 R14: ffff9ef90c1d1a28 R15: 00000000000= 00000 [ 97.682370] FS: 00007386aa4f3d40(0000) GS:ffff9efa77200000(0000) knlGS:= 0000000000000000 [ 97.683721] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 97.684703] CR2: 0000000000000000 CR3: 0000000169a14003 CR4: 00000000007= 70ef0 [ 97.685909] PKRU: 55555554 [ 97.686397] Call Trace: [ 97.686819] [ 97.687243] ? show_regs+0x6c/0x80 [ 97.687840] ? __die+0x24/0x80 [ 97.688391] ? page_fault_oops+0x155/0x570 [ 97.689090] ? srso_alias_return_thunk+0x5/0xfbef5 [ 97.689973] ? srso_alias_return_thunk+0x5/0xfbef5 [ 97.690848] ? __vunmap_range_noflush+0x420/0x4e0 [ 97.691700] ? do_user_addr_fault+0x4b2/0x870 [ 97.692606] ? srso_alias_return_thunk+0x5/0xfbef5 [ 97.693502] ? exc_page_fault+0x82/0x1b0 [ 97.694200] ? asm_exc_page_fault+0x27/0x30 [ 97.694975] ? cxl_await_media_ready+0x1ac/0x1d0 [ 97.695816] vfio_cxl_core_enable+0x386/0x800 [vfio_cxl_core] [ 97.696829] ? srso_alias_return_thunk+0x5/0xfbef5 [ 97.697685] cxl_open_device+0xa6/0xd0 [cxl_accel_vfio_pci] [ 97.698673] vfio_df_open+0xcb/0xf0 [ 97.699313] vfio_group_fops_unl_ioctl+0x294/0x720 [ 97.700149] ? srso_alias_return_thunk+0x5/0xfbef5 [ 97.701011] ? srso_alias_return_thunk+0x5/0xfbef5 [ 97.701858] __x64_sys_ioctl+0xa3/0xf0 [ 97.702536] x64_sys_call+0x11ad/0x25f0 [ 97.703214] do_syscall_64+0x7e/0x170 [ 97.703878] ? srso_alias_return_thunk+0x5/0xfbef5 [ 97.704726] ? do_syscall_64+0x8a/0x170 [ 97.705425] ? srso_alias_return_thunk+0x5/0xfbef5 [ 97.706282] ? kvm_device_ioctl+0xae/0x130 [kvm] [ 97.707135] ? srso_alias_return_thunk+0x5/0xfbef5 [ 97.708001] ? srso_alias_return_thunk+0x5/0xfbef5 [ 97.708853] ? syscall_exit_to_user_mode+0x4e/0x250 [ 97.709724] ? srso_alias_return_thunk+0x5/0xfbef5 [ 97.710609] ? do_syscall_64+0x8a/0x170 [ 97.711300] ? srso_alias_return_thunk+0x5/0xfbef5 [ 97.712132] ? exc_page_fault+0x93/0x1b0 [ 97.712839] entry_SYSCALL_64_after_hwframe+0x76/0x7e [ 97.713735] RIP: 0033:0x7386ab124ded [ 97.714382] Code: 04 25 28 00 00 00 48 89 45 c8 31 c0 48 8d 45 10 c7 45 = b0 10 00 00 00 48 89 45 b8 48 8d 45 d0 48 89 45 c0 b8 10 00 00 00 0f 05 <89= > c2 3d 00 f0 ff ff 77 1a 48 8b 45 c8 64 48 2b 04 25 28 00 00 00 [ 97.717664] RSP: 002b:00007ffcda2a6480 EFLAGS: 00000246 ORIG_RAX: 000000= 0000000010 [ 97.718965] RAX: ffffffffffffffda RBX: 00006293226d9f20 RCX: 00007386ab1= 24ded [ 97.720222] RDX: 00006293226db730 RSI: 0000000000003b6a RDI: 00000000000= 00009 [ 97.721522] RBP: 00007ffcda2a64d0 R08: 00006293214e9010 R09: 00000000000= 00007 [ 97.722858] R10: 00006293226db730 R11: 0000000000000246 R12: 00006293226= e0880 [ 97.724193] R13: 00006293226db730 R14: 00007ffcda2a7740 R15: 00006293226= d94f0 [ 97.725491] [ 97.725883] Modules linked in: cxl_accel_vfio_pci(E) vfio_cxl_core(E) vf= io_pci_core(E) snd_seq_dummy(E) snd_hrtimer(E) snd_seq(E) snd_seq_device(E)= snd_timer(E) snd(E) soundcore(E) qrtr(E) intel_rapl_msr(E) intel_rapl_comm= on(E) kvm_amd(E) ccp(E) binfmt_misc(E) kvm(E) crct10dif_pclmul(E) crc32_pcl= mul(E) polyval_clmulni(E) polyval_generic(E) ghash_clmulni_intel(E) sha256_= ssse3(E) sha1_ssse3(E) aesni_intel(E) i2c_i801(E) crypto_simd(E) cryptd(E) = i2c_smbus(E) lpc_ich(E) joydev(E) input_leds(E) mac_hid(E) serio_raw(E) msr= (E) parport_pc(E) ppdev(E) lp(E) parport(E) efi_pstore(E) dmi_sysfs(E) qemu= _fw_cfg(E) autofs4(E) bochs(E) e1000e(E) drm_vram_helper(E) psmouse(E) drm_= ttm_helper(E) ahci(E) ttm(E) libahci(E) [ 97.736690] CR2: 0000000000000000 [ 97.737285] ---[ end trace 0000000000000000 ]--- Factor out cxl_await_range_active() and cxl_media_ready(). Type-3 device should call both for ensuring media ready while type-2 device should only call cxl_await_range_active(). Cc: Dave Jiang Cc: Li Ming Suggested-by: Dan Williams Reviewed-by: Li Ming Signed-off-by: Zhi Wang Signed-off-by: Manish Honap --- drivers/cxl/core/pci.c | 18 +++++++++++------- drivers/cxl/core/pci_drv.c | 3 +-- drivers/cxl/cxlmem.h | 3 ++- include/cxl/cxl.h | 1 + tools/testing/cxl/Kbuild | 3 ++- tools/testing/cxl/test/mock.c | 21 ++++++++++++++++++--- 6 files changed, 35 insertions(+), 14 deletions(-) diff --git a/drivers/cxl/core/pci.c b/drivers/cxl/core/pci.c index 90a0763e72c4..a0cda2a8fdba 100644 --- a/drivers/cxl/core/pci.c +++ b/drivers/cxl/core/pci.c @@ -225,12 +225,11 @@ static int cxl_dvsec_mem_range_active(struct cxl_dev_= state *cxlds, int id) * Wait up to @media_ready_timeout for the device to report memory * active. */ -int cxl_await_media_ready(struct cxl_dev_state *cxlds) +int cxl_await_range_active(struct cxl_dev_state *cxlds) { struct pci_dev *pdev =3D to_pci_dev(cxlds->dev); int d =3D cxlds->cxl_dvsec; int rc, i, hdm_count; - u64 md_status; u16 cap; =20 rc =3D pci_read_config_word(pdev, @@ -251,13 +250,18 @@ int cxl_await_media_ready(struct cxl_dev_state *cxlds) return rc; } =20 - md_status =3D readq(cxlds->regs.memdev + CXLMDEV_STATUS_OFFSET); - if (!CXLMDEV_READY(md_status)) - return -EIO; - return 0; } -EXPORT_SYMBOL_NS_GPL(cxl_await_media_ready, "CXL"); +EXPORT_SYMBOL_NS_GPL(cxl_await_range_active, "CXL"); + +int cxl_media_ready(struct cxl_dev_state *cxlds) +{ + u64 md_status; + + md_status =3D readq(cxlds->regs.memdev + CXLMDEV_STATUS_OFFSET); + return CXLMDEV_READY(md_status) ? 0 : -EIO; +} +EXPORT_SYMBOL_NS_GPL(cxl_media_ready, "CXL"); =20 static int cxl_set_mem_enable(struct cxl_dev_state *cxlds, u16 val) { diff --git a/drivers/cxl/core/pci_drv.c b/drivers/cxl/core/pci_drv.c index 4c767e2471b8..6e519b197f0d 100644 --- a/drivers/cxl/core/pci_drv.c +++ b/drivers/cxl/core/pci_drv.c @@ -899,8 +899,7 @@ static int cxl_pci_probe(struct pci_dev *pdev, const st= ruct pci_device_id *id) if (rc) return rc; =20 - rc =3D cxl_await_media_ready(cxlds); - if (rc =3D=3D 0) + if (!cxl_await_range_active(cxlds) && !cxl_media_ready(cxlds)) cxlds->media_ready =3D true; else dev_warn(&pdev->dev, "Media not active (%d)\n", rc); diff --git a/drivers/cxl/cxlmem.h b/drivers/cxl/cxlmem.h index 918784edd23c..62ace404d681 100644 --- a/drivers/cxl/cxlmem.h +++ b/drivers/cxl/cxlmem.h @@ -767,7 +767,8 @@ enum { int cxl_internal_send_cmd(struct cxl_mailbox *cxl_mbox, struct cxl_mbox_cmd *cmd); int cxl_dev_state_identify(struct cxl_memdev_state *mds); -int cxl_await_media_ready(struct cxl_dev_state *cxlds); +int cxl_await_range_active(struct cxl_dev_state *cxlds); +int cxl_media_ready(struct cxl_dev_state *cxlds); int cxl_enumerate_cmds(struct cxl_memdev_state *mds); int cxl_mem_dpa_fetch(struct cxl_memdev_state *mds, struct cxl_dpa_info *i= nfo); struct cxl_memdev_state *cxl_memdev_state_create(struct device *dev, u64 s= erial, diff --git a/include/cxl/cxl.h b/include/cxl/cxl.h index e5d1e5a20e06..f18194b9e3e2 100644 --- a/include/cxl/cxl.h +++ b/include/cxl/cxl.h @@ -262,6 +262,7 @@ int cxl_map_component_regs(const struct cxl_register_ma= p *map, struct cxl_component_regs *regs, unsigned long map_mask); int cxl_set_capacity(struct cxl_dev_state *cxlds, u64 capacity); +int cxl_await_range_active(struct cxl_dev_state *cxlds); struct cxl_memdev *devm_cxl_add_memdev(struct device *host, struct cxl_dev_state *cxlds, const struct cxl_memdev_ops *ops); diff --git a/tools/testing/cxl/Kbuild b/tools/testing/cxl/Kbuild index d422c81cefa3..4b05b21083ad 100644 --- a/tools/testing/cxl/Kbuild +++ b/tools/testing/cxl/Kbuild @@ -5,7 +5,8 @@ ldflags-y +=3D --wrap=3Dacpi_evaluate_integer ldflags-y +=3D --wrap=3Dacpi_pci_find_root ldflags-y +=3D --wrap=3Dnvdimm_bus_register ldflags-y +=3D --wrap=3Ddevm_cxl_port_enumerate_dports -ldflags-y +=3D --wrap=3Dcxl_await_media_ready +ldflags-y +=3D --wrap=3Dcxl_await_range_active +ldflags-y +=3D --wrap=3Dcxl_media_ready ldflags-y +=3D --wrap=3Ddevm_cxl_add_rch_dport ldflags-y +=3D --wrap=3Dcxl_endpoint_parse_cdat ldflags-y +=3D --wrap=3Dcxl_dport_init_ras_reporting diff --git a/tools/testing/cxl/test/mock.c b/tools/testing/cxl/test/mock.c index 92fd5c69bef3..4f1f65e50e87 100644 --- a/tools/testing/cxl/test/mock.c +++ b/tools/testing/cxl/test/mock.c @@ -187,7 +187,7 @@ int __wrap_devm_cxl_port_enumerate_dports(struct cxl_po= rt *port) } EXPORT_SYMBOL_NS_GPL(__wrap_devm_cxl_port_enumerate_dports, "CXL"); =20 -int __wrap_cxl_await_media_ready(struct cxl_dev_state *cxlds) +int __wrap_cxl_await_range_active(struct cxl_dev_state *cxlds) { int rc, index; struct cxl_mock_ops *ops =3D get_cxl_mock_ops(&index); @@ -195,12 +195,27 @@ int __wrap_cxl_await_media_ready(struct cxl_dev_state= *cxlds) if (ops && ops->is_mock_dev(cxlds->dev)) rc =3D 0; else - rc =3D cxl_await_media_ready(cxlds); + rc =3D cxl_await_range_active(cxlds); put_cxl_mock_ops(index); =20 return rc; } -EXPORT_SYMBOL_NS_GPL(__wrap_cxl_await_media_ready, "CXL"); +EXPORT_SYMBOL_NS_GPL(__wrap_cxl_await_range_active, "CXL"); + +int __wrap_cxl_media_ready(struct cxl_dev_state *cxlds) +{ + int rc, index; + struct cxl_mock_ops *ops =3D get_cxl_mock_ops(&index); + + if (ops && ops->is_mock_dev(cxlds->dev)) + rc =3D 0; + else + rc =3D cxl_media_ready(cxlds); + put_cxl_mock_ops(index); + + return rc; +} +EXPORT_SYMBOL_NS_GPL(__wrap_cxl_media_ready, "CXL"); =20 struct cxl_dport *__wrap_devm_cxl_add_rch_dport(struct cxl_port *port, struct device *dport_dev, --=20 2.25.1 From nobody Sat Feb 7 12:25:21 2026 Received: from CY7PR03CU001.outbound.protection.outlook.com (mail-westcentralusazon11010023.outbound.protection.outlook.com [40.93.198.23]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D71763002A9; Tue, 9 Dec 2025 16:51:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.93.198.23 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1765299096; cv=fail; b=emn3E5AIn+Kda+bjnv1kvNcSqmwqedN9xWEZyOTCMCXngb45f12UbTOQXjAaUP6nRBii3eYRrDP4XIg7SCImAtv9D/18L2VdoiqNHflyDsMkocK2gaYveeQMjsvWwHkXHYL0Ps7iQM1s0AlgOPEwbjwpxYwgVsYXIxgTmfUZLME= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1765299096; c=relaxed/simple; bh=HmZk1Sa1C2FunV/jPRVZirmqdx0gQWw89s+PWkewmTM=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=iZS5wrVOBGBhr76B4xwPlCqbHvDrytimA9cPfBQL3Ld4SsaymCosBPDzneKwFeARcpc7/feX0iuKRINExyXurBCdnRgokBAy+BrMcX6VP6lnF+2J6qQRWykAkuVlkpeKVTS+BHbz7VEfh0gb4kF9xvoV+yVo3JNcS1whGF369QE= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=eXJCWKex; arc=fail smtp.client-ip=40.93.198.23 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="eXJCWKex" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=pjOGXRT+PQPXS/nX6MhV9x63+2oXOp36yUSHmO2Q28IK2yzeH1fxIarftHq0I8syb/l7C5Z4donRS2835XQQSTLD7Tr9L/09k5/nQB/4O+9xePfPHEPgdCc4YLJoE7DtWYnSZvX7izx021R76kfQa3jewisrhxHkhSh2VDCYr8fr1ELj4u1Z40bYJf3Q+88ZOm0b6X/IMjSMC2NLWVdSTQojTvqX7e4dRr4umHZhUA9KPYE4C7MeNtqhXurvqFQdHl/WkvFVc5bAmkigsXrLH0JT7Fw+9LilP5r+s68Wgm+kRz2tYWz+0L9x64gYWdqGbT6/xqTMQiaecPMYRwMUmw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=HyUU4Oc5ayVI+tqmPNu9i1i6Ktq53SoLcTNWapWyrvs=; b=pqpkiGVebP2RP2jgQcd3TO6BLVQWcI7PngqoYFJTsKOCfWMd9t44QcljMb6aJ6Vr1CHcce0jTp9q71B2VOk9Ac9BXiR73TPAUDWucMfSrOpVwMPqTTb/FNTdNnhWHuOgreTFDe0rN4SrHZODHWHFeW7h2EPV3+LkzGVq12JAATQNU3HAeoQ+oivgFJttD1R7oKp/z/hNtOR2juUqu2tGGLvj37+msKCgKGR6GRnXKiEcS/tE/fe+0mG6qW3myN4ZV+kqwqEoxBDruRmTjba1F18Qy6joQ79DnKvdbZg7Ncn9gACw08bXMI24tjufp91b5qWOmzdCLD7DDdm45IWQcw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=HyUU4Oc5ayVI+tqmPNu9i1i6Ktq53SoLcTNWapWyrvs=; b=eXJCWKextIdGOWUx9s3qTDlS3FiH3UII5qoyoWM8L9+ZMvjhseVBxPZEEw111qjI9r2dUmXjysMdKdwhLrnQfErFi/f70JYonsMHoy+OZnG0ie0t17rnm4sl1wx31ntp2n/baJEnCR9HX3FsHNVoDgXVZ8lch5OwiLAeug8yyw+zjNb7PGpI/zDgTdmKrewzBIxcL5wiWfkyhyFJ7Mcq15TRavy/KNN2r1V8Geupfcx1PFM/FSvcKKiO8EKkRkCODGq7dyo9g5zG3XS5PPCAJv1N/57vCeGcfbIYbkhGub7y/XEQN2JcH8Ka8IgQj9MFnFmyGk8cpHpqM4DrzirWsQ== Received: from BN9PR03CA0451.namprd03.prod.outlook.com (2603:10b6:408:139::6) by MW4PR12MB7213.namprd12.prod.outlook.com (2603:10b6:303:22a::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9388.14; Tue, 9 Dec 2025 16:51:28 +0000 Received: from BN2PEPF0000449F.namprd02.prod.outlook.com (2603:10b6:408:139:cafe::d4) by BN9PR03CA0451.outlook.office365.com (2603:10b6:408:139::6) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9388.14 via Frontend Transport; Tue, 9 Dec 2025 16:51:22 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by BN2PEPF0000449F.mail.protection.outlook.com (10.167.243.150) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9412.4 via Frontend Transport; Tue, 9 Dec 2025 16:51:28 +0000 Received: from rnnvmail205.nvidia.com (10.129.68.10) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 9 Dec 2025 08:51:04 -0800 Received: from rnnvmail204.nvidia.com (10.129.68.6) by rnnvmail205.nvidia.com (10.129.68.10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 9 Dec 2025 08:51:04 -0800 Received: from nvidia-4028GR-scsim.nvidia.com (10.127.8.11) by mail.nvidia.com (10.129.68.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20 via Frontend Transport; Tue, 9 Dec 2025 08:50:57 -0800 From: To: , , , , , , , , , , , , , , , , , CC: , , , , , , , , Subject: [RFC v2 02/15] cxl: introduce cxl_get_hdm_reg_info() Date: Tue, 9 Dec 2025 22:20:06 +0530 Message-ID: <20251209165019.2643142-3-mhonap@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20251209165019.2643142-1-mhonap@nvidia.com> References: <20251209165019.2643142-1-mhonap@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN2PEPF0000449F:EE_|MW4PR12MB7213:EE_ X-MS-Office365-Filtering-Correlation-Id: 54652a73-8650-4190-8458-08de374332a3 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|1800799024|376014|7416014|36860700013|921020; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?IX0KlodpSQ7o65COLyqWY3lXMbuR0m1fXd+DQXD6ezVjv3Ki6QzU0JvbLq6W?= =?us-ascii?Q?I3KUzuYF27h/hzrg8EC4tSqFZsU6EHIO06aVvBQe6R5KnypstXwf7PEC4ODO?= =?us-ascii?Q?/Tm7+EjsWmdzJYYP4Dhkm2fKy5/TTkKo75ycntqfkT/J/v/BDqmA1Z/mU1zR?= =?us-ascii?Q?bkR9QveNV3GgQY7UG8vRdnXVDAFo0xhDCw8QZEF1UrvChh6iSIgTfaDwlxP7?= =?us-ascii?Q?N4CaTG4XOI3lCWYbgeD+xd4J74gzGYsRW1GSCUd2gefOHIYEgSzXRJQFubq1?= =?us-ascii?Q?cDR4ebfBaLq459lVgl9foa852d2RZKeetSEKfPr/HgRX1bsbTF//vfFK0N0A?= =?us-ascii?Q?QzoX3x4Y2F+OAmAFIOkooINauyICx1JVvFFVTeHvCw448DOLVR+CgbfbtElu?= =?us-ascii?Q?P8hi15PXsyd673UAxgQ/b8I7pwKgLqzSiBVgNiABZEQaQRkCMxyTW1pecA9M?= =?us-ascii?Q?pHAdvsBt40PzBAwuP8+nheucbr88vvSsRe+2+oFqDXXePRAeeV7TM2lTJ6HU?= =?us-ascii?Q?1cQNPLZLddWgVqYAl6w9ONn8aYbGYU5tyw1iG696L/NsDcCjIHUWjIMEZqRt?= =?us-ascii?Q?B+TovwmLcBmqLPAKP4rFtXseNClemqsqhs1H873HR+Jiu989cIqlkm5LAPwD?= =?us-ascii?Q?XFrIgwAjuqoMvraLgzoM9S47Qa6d/l9f6JFstZrLx0uWVDukEL4uc6DQunje?= =?us-ascii?Q?691CnQLAltC0HvHxPsIEve7dKNRR7bbPNKL6R/mJC+YCIqHixbv4HMBU1Kz/?= =?us-ascii?Q?T7V/zPIz+GYdTlDjdjQ6jxJxLJLWt3nq0XcZ1Gz+Gi5Sw08d5nNt8J6xQLbq?= =?us-ascii?Q?4xvygc5OTJQy/pEE5BCk7rE6nx7316gtBybuoxKWXl9NQoLsWWpBAOrggQgw?= =?us-ascii?Q?tIyVg/b7+v2H0/oigWd8Gg0BiSyJm3xDcnQVn3k/j6Jnb3xfy/xI22jyQz9v?= =?us-ascii?Q?A42qHjbqwOKXHIMtgPCCIF78y24U873Z/jxprgiorYCCpcJPzwbADH4BJsjQ?= =?us-ascii?Q?mGTx/rpRMW1i+QvSvgbLSsghhUOH4Ydz+pAAWnEKomerwLy/AWGUZMBg5QcV?= =?us-ascii?Q?62a7uJ0c8WrLIXYhukJQCz/6Gt9lOYdYB/iIELVLaPOHbrS6+rMUI5TPsc9l?= =?us-ascii?Q?UzAtWsojRKlKcQMmMfQVx2xC9B/nusGWbkLLAXErwvDObuOG+Z9zeXJwif8a?= =?us-ascii?Q?n3s1jOVpTPjJP6OqpKMs+SXODG5ku2W5VdacVSYuOewCnO0iYoFoEu6Hsm65?= =?us-ascii?Q?TvgXKBbEvVJBG2XRMa8Kv9LJ/LP3SS4e05HLJw7O7JceoNmLdNnkopcnhgbe?= =?us-ascii?Q?bvfCViAzP3c+TPtkeiJg1stdJ/IJM2dEKWNiXbJq/iJHggPJVvr+k5V85jUH?= =?us-ascii?Q?hMg/6jaI734j5Lm4H1JRart5hLNNcluPqwMYYX1jGFmMOU9eV6AQxekJhNF9?= =?us-ascii?Q?K8jnL3Vv3IOWtGxZ2L67er3267m/d1qS1lVwnLVP7cMs2mhBkufF1QuApRXw?= =?us-ascii?Q?N9Y0/sEkYQIOyWLsMNkE281NU2z3WsXxReZ0051vUIvaLKB5emm/PxKSOZNI?= =?us-ascii?Q?Be15gmzVA8bs6xiIDb5HsCr8mwFE9w+SFoOS+UeP?= X-Forefront-Antispam-Report: CIP:216.228.117.160;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge1.nvidia.com;CAT:NONE;SFS:(13230040)(82310400026)(1800799024)(376014)(7416014)(36860700013)(921020);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Dec 2025 16:51:28.3246 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 54652a73-8650-4190-8458-08de374332a3 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN2PEPF0000449F.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW4PR12MB7213 Content-Type: text/plain; charset="utf-8" From: Zhi Wang CXL core has the information of what CXL register groups a device has. When initializing the device, the CXL core probes the register groups and saves the information. The probing sequence is quite complicated. vfio-cxl requires the HDM register information to emualte the HDM decoder registers. Introduce cxl_get_hdm_reg_info() for vfio-cxl to leverage the HDM register information in the CXL core. Thus, it doesn't need to implement its own probing sequence. Signed-off-by: Zhi Wang Signed-off-by: Manish Honap --- drivers/cxl/core/pci.c | 28 ++++++++++++++++++++++++++++ include/cxl/cxl.h | 4 ++++ 2 files changed, 32 insertions(+) diff --git a/drivers/cxl/core/pci.c b/drivers/cxl/core/pci.c index a0cda2a8fdba..f998096050cf 100644 --- a/drivers/cxl/core/pci.c +++ b/drivers/cxl/core/pci.c @@ -532,6 +532,34 @@ int cxl_hdm_decode_init(struct cxl_dev_state *cxlds, s= truct cxl_hdm *cxlhdm, } EXPORT_SYMBOL_NS_GPL(cxl_hdm_decode_init, "CXL"); =20 +int cxl_get_hdm_reg_info(struct cxl_dev_state *cxlds, u64 *count, u64 *off= set, + u64 *size) +{ + struct cxl_component_reg_map *map =3D + &cxlds->reg_map.component_map; + struct pci_dev *pdev =3D to_pci_dev(cxlds->dev); + int d =3D cxlds->cxl_dvsec; + u16 cap; + int rc; + + if (!map->hdm_decoder.valid) { + *count =3D *offset =3D *size =3D 0; + return 0; + } + + *offset =3D map->hdm_decoder.offset; + *size =3D map->hdm_decoder.size; + + rc =3D pci_read_config_word(pdev, + d + PCI_DVSEC_CXL_CAP_OFFSET, &cap); + if (rc) + return rc; + + *count =3D FIELD_GET(PCI_DVSEC_CXL_HDM_COUNT_MASK, cap); + return 0; +} +EXPORT_SYMBOL_NS_GPL(cxl_get_hdm_reg_info, "CXL"); + #define CXL_DOE_TABLE_ACCESS_REQ_CODE 0x000000ff #define CXL_DOE_TABLE_ACCESS_REQ_CODE_READ 0 #define CXL_DOE_TABLE_ACCESS_TABLE_TYPE 0x0000ff00 diff --git a/include/cxl/cxl.h b/include/cxl/cxl.h index f18194b9e3e2..d84405afc72e 100644 --- a/include/cxl/cxl.h +++ b/include/cxl/cxl.h @@ -289,4 +289,8 @@ int cxl_decoder_detach(struct cxl_region *cxlr, enum cxl_detach_mode mode); struct range; int cxl_get_region_range(struct cxl_region *region, struct range *range); + +int cxl_get_hdm_reg_info(struct cxl_dev_state *cxlds, u64 *count, u64 *off= set, + u64 *size); + #endif /* __CXL_CXL_H__ */ --=20 2.25.1 From nobody Sat Feb 7 12:25:21 2026 Received: from BL2PR02CU003.outbound.protection.outlook.com (mail-eastusazon11011041.outbound.protection.outlook.com [52.101.52.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A6AD827815D; Tue, 9 Dec 2025 16:51:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.52.41 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1765299103; cv=fail; b=pfapqOGQiMPg8w5mySKm3L5Sh1GOrlvu/YJNb6463eMnKvJ4vzK4HtEeUwkxeacN3rD9oANiATIpI9wNaKjBLMgFmSLWm7umgCYONCc7Q1roaihf/RKI7wvBoMQnAJMAiQqx6b/RyjgdzQyEZKXiI5JL6VvbOuXhzhP53uVuPkk= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1765299103; c=relaxed/simple; bh=UXIvMa0drT7HaL2qWubLLuMRZyspCFAGLd9wowG5U+k=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=QkVsEtH4e+4Pu3BykpwepwnYUVqpJlztwMtC6ASpPQEdimIOJGCR0H7V0uVEMgTLRzDtPDieNHaR9v9vRigMsfXYXBzwmoxnCXnwxBgUEke+6ibmfRpcKypG6meH9Dupi+ECBf1Sf5eGYC0pZ1DEUzujTNuz/mSeEkhuXm4oFlU= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=pWt4GTeM; arc=fail smtp.client-ip=52.101.52.41 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="pWt4GTeM" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=URYB77LgiuLoW/I4feri+gg9Vd+buTD0ekJ/ih118NNpB2+SQ3FHmW5GEHZF8t2PRgQuujhPjcS83wuR40ZIGuhw5blrd30wD4y6CNca/0nf1QogYE/Ba5fdA0pNTY8Gt7EirXLKKFmFwoJwam18aFaq8IJ7osatPby+0Z4KtOTtBgJ3nHXUQckgsdI6dc7VNuGqKh4sMXiShTFIbill8q5uEn7oO12eMoirGkvJ9gK+qL0+ZPdwbmld8qS5eP57orVK9ccknu7d9irEujaT4PLBgq+Vp6dG/KBSl1NhAucGe8w/USZ2w/w2OZM1FP+NATQU321SJyIkcjcZ/bTJAw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=CbiHONk6rsIkLaUqeO1AhigTqTnAQrZFa43wA14OV1Y=; b=ls2ZC9XmG8B3tPuZPGwmX1CFX/uw4qWpAXumo2Tjl+1g3G+VeAGMQwKvEQXJejBVxhpxkPJlhtb+Q5+dU5L5KdImBAsD7NCe9mXhQLQqo6FC1rQEeQaViELLx2XwIk16+xbdwRwEvMes59OJR9ibyHfEb0WoP3jtmmR41OCPfvTewgMowvz72R+Rxo/VMsWA0iKCnqmHWhRc/8JuakfzcCFIunTW6BHaPoPEQ4Rgl7Wdi3TSsuYZ4wbLmwIWlqY3BPHpGuBCz469muhn8BLjs2VLyN5F9oIBdPxMpvLxPH11F/kRFhkxTIyywUQHNGyaMXtJYp1HXnmUbV8zqLFv9g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=CbiHONk6rsIkLaUqeO1AhigTqTnAQrZFa43wA14OV1Y=; b=pWt4GTeMHuKozvAS3cUmbONsRzqYxS8o6waGALbkJ18Fu/vVhzGlDuTG09NYkDx1IVdSjpzChbvNZJpwiSOYXik78FgrouDjDMztIKl+O05ySFSybPAa7lnO+/bynEOSsqaJbWUW5jLwdLsEA0aYmQ8BFJ0s7PCorZwcszR5mC3dl6suh71O/Fu9J7E0aCKia4M5fQkyjKambjRF5KEoloLCdptUP6Q1loiFX0K8AoDtnLbErBAeD/tPbKAO9FzK0aDbAswlB0K8or2xyyTzu9ymxfm92qShtMHIMpQc0fb9CoKShQw+Pc7TzebGwqryW3IzuS6OP4WloVOhqlX57g== Received: from SJ0PR13CA0198.namprd13.prod.outlook.com (2603:10b6:a03:2c3::23) by DM6PR12MB4433.namprd12.prod.outlook.com (2603:10b6:5:2a1::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9412.7; Tue, 9 Dec 2025 16:51:35 +0000 Received: from SJ1PEPF00002317.namprd03.prod.outlook.com (2603:10b6:a03:2c3:cafe::a6) by SJ0PR13CA0198.outlook.office365.com (2603:10b6:a03:2c3::23) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9412.6 via Frontend Transport; Tue, 9 Dec 2025 16:51:08 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by SJ1PEPF00002317.mail.protection.outlook.com (10.167.242.171) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9388.8 via Frontend Transport; Tue, 9 Dec 2025 16:51:34 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 9 Dec 2025 08:51:12 -0800 Received: from rnnvmail204.nvidia.com (10.129.68.6) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 9 Dec 2025 08:51:11 -0800 Received: from nvidia-4028GR-scsim.nvidia.com (10.127.8.11) by mail.nvidia.com (10.129.68.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20 via Frontend Transport; Tue, 9 Dec 2025 08:51:04 -0800 From: To: , , , , , , , , , , , , , , , , , CC: , , , , , , , , Subject: [RFC v2 03/15] cxl: introduce cxl_find_comp_reglock_offset() Date: Tue, 9 Dec 2025 22:20:07 +0530 Message-ID: <20251209165019.2643142-4-mhonap@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20251209165019.2643142-1-mhonap@nvidia.com> References: <20251209165019.2643142-1-mhonap@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ1PEPF00002317:EE_|DM6PR12MB4433:EE_ X-MS-Office365-Filtering-Correlation-Id: bb8f11f1-2d9b-442b-9823-08de37433694 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|7416014|1800799024|36860700013|82310400026|921020; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?XvZS2iIGp/9mc5akGjw+MVa03Gnqp3Ee21ww9M1xdOaG0FVA/YdE4FLC0gH6?= =?us-ascii?Q?EAQkhelLgoIJnUCQQ6QHMUADPZdrgDA8AHFwMzY3j7iPS4eRmUhEF1JipfAz?= =?us-ascii?Q?yheYKAwNPWQs/Iq1MpVD4Gyr9FJhq8MrITbHqX18XukaOKsvGA7+Kq66CnRO?= =?us-ascii?Q?gTm2JlJOeV3k+D9gqJrhHhODj2ax/Jnu6OtB7crhvmXbJAwmTTzpeoQk0zDE?= =?us-ascii?Q?Qtt0r8FBxuH5Rr2MDQoxycQWBgiLOH4hVZaPetW298y8+y6dWhW8McinAcUV?= =?us-ascii?Q?kYRiCqB2bbs5J5MH5O/5blMdq1oY9t+USEX5lpqNIv5tOldaXXzYV3+TokuG?= =?us-ascii?Q?6l0HCsUlDaoMK5KaFbdv5CCMIQZPDu07R5+UZ7OJaZyI6xmwdJiMPDIgtP7k?= =?us-ascii?Q?cuO37AOl8HLHlBntXeS/XPrt9lb84D6x7r882S7Ibizfl3nBJsSpKI+8lc+1?= =?us-ascii?Q?tw66HgDUStrPczrEQsCEO7RtzxDLqLh4FRQiYg9Yqol/QPdfeDoKOET1lED8?= =?us-ascii?Q?89KsxTyrNROR0tfIgbLbEjMBrCpyuE+L4/xbme/0VmEJnm7YqW5vVXLq4Xuz?= =?us-ascii?Q?wf5WdPJ/F0nCeJGS1wgvXtiiM4hT+UyvJdFZDgmbjSn8/IeTEAk2WrfvjTrS?= =?us-ascii?Q?bKApxsBPzEIoPKW1f2SoVyMBUXZ1tOwjUZqeD7RbsA/wbgowvJu5oUbl9Hdi?= =?us-ascii?Q?SOngSySSO06vF3y68hr7UN21rucVGyPCFRMoPTRPtJTqcSQWA7uWKyUXewJr?= =?us-ascii?Q?5K+8tR4uP6mPpbre83GbGBRHOzRII+76/UC2qCV8OStsAKJdvgpk41gESOu5?= =?us-ascii?Q?zP0kSUEFjMkv6G1PAqVFDWd+hzHWyo8KSW4MOYtx2rFu67U+lnr8Cigh8m95?= =?us-ascii?Q?qlbBpIgKRuu8UJDY05fTNo5a4LiOPeNwZ3+5R97h9bnhRTJ/+j3ZMKNxZ5p3?= =?us-ascii?Q?YfW/74QKVVz1RKlX6O9nB6+oo/mrTG7rFuTvJH7x92lvft7bzr93DbNTelD5?= =?us-ascii?Q?lwlNd7aeZ/eQ5hYh7Htwfv2KsKLK7WIWSsCI/2cVyumkvfyr7WgoXQ6ttE7O?= =?us-ascii?Q?57UKBoQ65hQNmw00ejIqwez2hdGs6XuUGXHrkDrfo7Oi9wyR7trXphQLa7fE?= =?us-ascii?Q?6feJFzW+3On/y/uxRWeBN/y/BoAiA3nz6+kqol9WOwrmlOpUMH1FZSG1FuZH?= =?us-ascii?Q?hF9LTimY9FR4s4FArE9CGEmnuwOlDVZg++SLIkTA/4RU0oOw2heZHsdxMs/T?= =?us-ascii?Q?79BtSM6xLKtiOLWZVyUndPpw9a9Ixh9RggDsfce6fYyzUzIRgWkjG2bjhBrp?= =?us-ascii?Q?4AKa579D2MUatH5yQwjHGVlZxD+EOqXuT5/iXCx+Be1iKIs9Eve9nD8s3M0G?= =?us-ascii?Q?EhsRUvBWJRAvZSrHzYu3qwyroEkIpx50Gd6kSZvjmD436ZGWeCoemfSoFM/0?= =?us-ascii?Q?ToHw62ZMxKY+NprfX+V5lT05fWWKCYf7uSc1i/X2IXFaG6IxGPNLbO9JXtcy?= =?us-ascii?Q?AJAtH2GKlNFe5PxsjN9rmlBlcvaEMPXu4VCfdH7kMzWKP82wy7OeH8f8iNzH?= =?us-ascii?Q?A+g3peysew1vj1k5Hpz9dqBgNi+O9QY+joNPGZ/t?= X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(376014)(7416014)(1800799024)(36860700013)(82310400026)(921020);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Dec 2025 16:51:34.9857 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: bb8f11f1-2d9b-442b-9823-08de37433694 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: SJ1PEPF00002317.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB4433 Content-Type: text/plain; charset="utf-8" From: Zhi Wang CXL core has the information of what CXL register groups a device has.When initializing the device, the CXL core probes the register groups and saves the information. The probing sequence is quite complicated. vfio-cxl needs to handle the CXL MMIO BAR specially. E.g. emulate the HDM decoder register inside the component registers. Thus it requires to know the offset of the CXL component register to locate the PCI BAR where the component register sits. Introduce cxl_find_comp_regblock_offset() for vfio-cxl to leverage the register information in the CXL core. Thus, it doesn't need to implement its own probing sequence. Signed-off-by: Zhi Wang Signed-off-by: Manish Honap --- drivers/cxl/core/regs.c | 22 ++++++++++++++++++++++ include/cxl/cxl.h | 2 ++ 2 files changed, 24 insertions(+) diff --git a/drivers/cxl/core/regs.c b/drivers/cxl/core/regs.c index dcf444f1fe48..c5f31627fa20 100644 --- a/drivers/cxl/core/regs.c +++ b/drivers/cxl/core/regs.c @@ -345,6 +345,28 @@ static int __cxl_find_regblock_instance(struct pci_dev= *pdev, enum cxl_regloc_ty return -ENODEV; } =20 +/** + * cxl_find_comp_regblock_offset() - Locate the offset of component + * register blocks + * @pdev: The CXL PCI device to enumerate. + * @offset: Enumeration output, clobbered on error + * + * Return: 0 if register block enumerated, negative error code otherwise + */ +int cxl_find_comp_regblock_offset(struct pci_dev *pdev, u64 *offset) +{ + struct cxl_register_map map; + int ret; + + ret =3D cxl_find_regblock(pdev, CXL_REGLOC_RBI_COMPONENT, &map); + if (ret) + return ret; + + *offset =3D map.resource; + return 0; +} +EXPORT_SYMBOL_NS_GPL(cxl_find_comp_regblock_offset, "CXL"); + /** * cxl_find_regblock_instance() - Locate a register block by type / index * @pdev: The CXL PCI device to enumerate. diff --git a/include/cxl/cxl.h b/include/cxl/cxl.h index d84405afc72e..28a39bfd74bc 100644 --- a/include/cxl/cxl.h +++ b/include/cxl/cxl.h @@ -7,6 +7,7 @@ =20 #include #include +#include #include #include =20 @@ -292,5 +293,6 @@ int cxl_get_region_range(struct cxl_region *region, str= uct range *range); =20 int cxl_get_hdm_reg_info(struct cxl_dev_state *cxlds, u64 *count, u64 *off= set, u64 *size); +int cxl_find_comp_regblock_offset(struct pci_dev *pdev, u64 *offset); =20 #endif /* __CXL_CXL_H__ */ --=20 2.25.1 From nobody Sat Feb 7 12:25:21 2026 Received: from SA9PR02CU001.outbound.protection.outlook.com (mail-southcentralusazon11013027.outbound.protection.outlook.com [40.93.196.27]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 099F23002CD; Tue, 9 Dec 2025 16:51:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.93.196.27 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1765299113; cv=fail; b=FwvbgukaCssjVqvPemvHaKwdyjvu5ZK2ScYtJ8FsE23zeXIaO5PmFxhqjTtalJ5AyaTXXZD7xi6EHGP9zOHKrqrSWovnLlLAwUGZe6j2Hfdoe6wXKtPqS4GDLzxAiW0PUiqwuNpYjaScbYN92PtCG/2bDO8caUCuy4GnX4X6q64= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1765299113; c=relaxed/simple; bh=Y3PrEnHSlhxFT4e527ZQT7yFRNRnaoRfLoa40OiIwTA=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=Qs3f233NTaryRYscmDGkT0w2CvNNKBKrLvXHVTMVIQ5SzamxY8fz6nS6HgEAXAfam0cONG30XAT2TNs20T0WE7ymi+bF9AmQXbMBczgpgjcxH7zHOvJIfcehlwqkNYOfyX4jXs0dDteiAiwlNahtOl5eKMNYtANPsN3hGUb9sj4= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=B9Kt3f0z; arc=fail smtp.client-ip=40.93.196.27 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="B9Kt3f0z" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=psc67IfrTf+hd17bKRY0xKVdZiEEUF2Ag5sx9m2UubiHs04cliA2o8gpTkrX7FiiKseF2ck7wZS2lWiyss/FZA5biBVyifsemFkxIK01Kk9j7Gi3gQrZx7Dbz10FWdf2MX0HaCqM5yX0MoYGsyQJ4ZjOQuJkHL90+Zw2nYA5A2hhVdooLJnglfYar/dzJ+EzwAquQCMS0ReFOB/aaIRqgdMtDKDspybzCDdyNQwqUhzgvZ0qryUaTs4DTNFe/phTlEAeGki4I390ulJjXnru4x7lwh3Y7eOqepzfAAewkYYXoFpj0l0d0PeJl1VIfR5iFTHGDI2FpK91xIC+nUZfaQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=WwaKGfrFnrXibcbAKe4iATpt0x/YPgJCHKsTDP7mAsI=; b=MJf9TXkTR/hSx364i+uZ6u5e30v0uOJHkTweJHA1JMKQJ9mR8A3xniz+w9xbLxq9Ws9yJZLUokxkIGd2ordY47cCRmk900rlYpytk78T2OD3Hne5tKpVTs9zI0Xu/I4OjzA9F5fq7xwuTfe0eqzV7IeAmPWDOpeZp7ntLHrsBFwP4BaZ4YZhgseM0EqxSVS1d6qFK/KPoqK5BMY70qML54fnqhzVZjhrFLaUvcnyC8M4rL5v6iAhuxIfWBTiEQ0UoB/VFgS4UB97fzlqQYDE6Ld67jDQNLlea6bvfUUz+d7mGzcnqfdsvB4Q6lGSx3URPiqZ+Zr4sk0ax0ENhbYgjA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=WwaKGfrFnrXibcbAKe4iATpt0x/YPgJCHKsTDP7mAsI=; b=B9Kt3f0zfe51umX7D4+zxMy9T2ExRUkGEf8ZxdtBfdBG6WfVYaOKJkyBntBxImB6Tr1qbi9ZTfvkP+DKuHKWRpatCLNGjoqR5YbUohDRUy2jwvY0tWYL7zRekiRUbNi9VsbbMIbVP+O2AAqLtIxUXJKgHQxh1W0VHhoaxf5sHGb9Apwn99Ykqv0cji1nY5HYezRMwR78d5bh60TbVrp3JQWbpYJTAaaqli6iBwBNrtU5ZIizyLIuxxWXWmoyBq99WufbXbwuaxBFhaeCl8IbwJroGj4aU31oBEHJmFPNwFjpRvQYeZoiDCdXP1dK2jhbrT9xZ/Whi3WdvW9ZNMMGfA== Received: from SJ0PR13CA0171.namprd13.prod.outlook.com (2603:10b6:a03:2c7::26) by CH3PR12MB9077.namprd12.prod.outlook.com (2603:10b6:610:1a2::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9388.14; Tue, 9 Dec 2025 16:51:45 +0000 Received: from SJ1PEPF00002313.namprd03.prod.outlook.com (2603:10b6:a03:2c7:cafe::94) by SJ0PR13CA0171.outlook.office365.com (2603:10b6:a03:2c7::26) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9412.6 via Frontend Transport; Tue, 9 Dec 2025 16:51:43 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by SJ1PEPF00002313.mail.protection.outlook.com (10.167.242.167) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9388.8 via Frontend Transport; Tue, 9 Dec 2025 16:51:43 +0000 Received: from rnnvmail203.nvidia.com (10.129.68.9) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 9 Dec 2025 08:51:20 -0800 Received: from rnnvmail204.nvidia.com (10.129.68.6) by rnnvmail203.nvidia.com (10.129.68.9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 9 Dec 2025 08:51:19 -0800 Received: from nvidia-4028GR-scsim.nvidia.com (10.127.8.11) by mail.nvidia.com (10.129.68.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20 via Frontend Transport; Tue, 9 Dec 2025 08:51:12 -0800 From: To: , , , , , , , , , , , , , , , , , CC: , , , , , , , , Subject: [RFC v2 04/15] cxl: introduce devm_cxl_del_memdev() Date: Tue, 9 Dec 2025 22:20:08 +0530 Message-ID: <20251209165019.2643142-5-mhonap@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20251209165019.2643142-1-mhonap@nvidia.com> References: <20251209165019.2643142-1-mhonap@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ1PEPF00002313:EE_|CH3PR12MB9077:EE_ X-MS-Office365-Filtering-Correlation-Id: cc690d65-8a4e-47b8-cb90-08de37433bb6 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|36860700013|82310400026|7416014|1800799024|921020; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?nFWWPRNLGMfYNLK2P7lwm8TqnFkxIjWlxtISO7PqW25nnpYlgxdsD26i7vjk?= =?us-ascii?Q?RLZbR7cT1xxROR5HPo+Gh2mmuJfdyEZOGRLPWAzTvvn7T0ke5A9Ctk744wdn?= =?us-ascii?Q?uANmke3VhOjVH01qmTJ2t5xmhHScT6GSk50aguwWZplSDPAMdbqDN6oqpdHo?= =?us-ascii?Q?/bfmSBfqQEj2c2cLWPPnqE/vwhWI+XgZzJqddNd9ddVpK5Xt5dTCyEs0qBke?= =?us-ascii?Q?rJZcyM/JwB68jK4qsu3FHLzjWaRnX/yaqi+R0s112hJMHsFXaLSH8awfVjM5?= =?us-ascii?Q?Nw3F59qZzac54HmpGvaDD9K6QRoeUoU9mJRJOWDK84yTeUE0EyR5tHn4V0X5?= =?us-ascii?Q?/NaVfA01z8cECpfae4hEqk56lSH7OsJyHMHIpPKz6KU+ZuYDfhw30fyF1wCc?= =?us-ascii?Q?MS57DDnV4J9OYngaVicKxjf6U8dUV0GHjNlIicEA2mSlvLe6k5v0m3m5mHi1?= =?us-ascii?Q?8iModg3fyRX3HrA31dxiZww6V2Lrtt4wUX0g1I0Jg8Uja6UmiC0CZxyoBRvj?= =?us-ascii?Q?9RfwGwn8E29kwUoFj2+4pVvJ/o28a7nTZvtnOmW5jUts+cKiuEA2kHCMM/h2?= =?us-ascii?Q?kVsJQM4tTjGrCVwRWXCle6dxcDESm1yfcfOn7a4AzsztW0rjkvVv63b/CTNY?= =?us-ascii?Q?3CYEc/BTHl1Z6hNYfyMaeRtlenWjhwZwPQZ9jUAbnP1ivN+xKCiybRdlAtMp?= =?us-ascii?Q?MLo27d5V8iHTVTeKMytMejhYFmYUOxswmKWT9MqKYHMyDjMOV+MvI8NQTJN5?= =?us-ascii?Q?UhvT+TXBqnzEd4jrmjpy1GnW8vylgl2zHVZypYjT23YPXkrVzkh451S2fQg9?= =?us-ascii?Q?chdsD7A6mNooKCNNcUhvYQ3XRYYR5D8jLGT581rLY/xK7WuTeAYPveDFnu8B?= =?us-ascii?Q?D/Vl/3wJyTAbQr2na9mReAwRjO5jttL6+FsYUX/sziWJtKIu3mgwYin9km7E?= =?us-ascii?Q?VdyCTCDxoSkfRvplr01y1vAdk6NsTx0YGKaR5U9VOk2SPBk1CdMRk/nxxm1K?= =?us-ascii?Q?HsEhCRbc3Nnlaj7eoNef6A8YnteOj5xf5vtfTt+gW46we8nJNYnM0CjIsNpY?= =?us-ascii?Q?TFafbwkeupd+/c6n0uWF46DJm3hhWJIe8PF6QAZaZHwuA7fnT+o/iYPEisuf?= =?us-ascii?Q?R5WkV24qjFrW2RHW2kit9/3aXaylkv+yb6OPUtg0pi5PMbEWj+uHopFizlDL?= =?us-ascii?Q?hcGjSr4EFTg5ziUqw8Q963nb7o+O8L3AFsIpxd84xF96LGtYNcUlStJddNvd?= =?us-ascii?Q?Ke7hOvCAXSM7jALIvckFzIwDh6pYZD2kgU2fi589IEQt37yVvsoSoKd6KBFo?= =?us-ascii?Q?M1ASGmExXpegwrdgLX4hsJERrBYnJ0Dc9R5+rL1VijQuJXJC45gzwIf6ryyA?= =?us-ascii?Q?z3m6h1GqblQ6ZeVxcW1+03xo4mpr9O4w7bUSPIlHS6YGfmpouOv4kKQQj1BF?= =?us-ascii?Q?M7U0HPD8UIl6WXq7kPNDTUyzw7wwOJkH3mqXIcsHegGrECz02M9LFKTlI3vt?= =?us-ascii?Q?VkmI+v6yk9AiyCV0h5Kw9SjzHaxqXT+hTzFt5L1HmJ2laS8LpM4RyTuVoeqj?= =?us-ascii?Q?fD4OiNeUp0QSXR1/t3ArSOTTruYmGmeVfI4Lv5Fh?= X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(376014)(36860700013)(82310400026)(7416014)(1800799024)(921020);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Dec 2025 16:51:43.5973 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: cc690d65-8a4e-47b8-cb90-08de37433bb6 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: SJ1PEPF00002313.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH3PR12MB9077 Content-Type: text/plain; charset="utf-8" From: Zhi Wang The teardown path of kernel CXL core heavily leverages the device resource manager. Thus, the lifecycle of many created resources are tied to the refcount of parent object and the resourced are freed when the parent object is freed. However, this creates a gap when an external caller wants to swept the resource but keep the parent object for a re-initialization sequence. E.g. in vfio-cxl. Introduce the devm_cxl_del_memdev() for an external caller to destroy the CXL memdev. Signed-off-by: Zhi Wang Signed-off-by: Manish Honap --- drivers/cxl/core/memdev.c | 6 ++++++ include/cxl/cxl.h | 1 + 2 files changed, 7 insertions(+) diff --git a/drivers/cxl/core/memdev.c b/drivers/cxl/core/memdev.c index 9de2ecb2abdc..d281843fb2f4 100644 --- a/drivers/cxl/core/memdev.c +++ b/drivers/cxl/core/memdev.c @@ -775,6 +775,12 @@ int devm_cxl_memdev_add_or_reset(struct device *host, = struct cxl_memdev *cxlmd) } EXPORT_SYMBOL_NS_GPL(devm_cxl_memdev_add_or_reset, "CXL"); =20 +void devm_cxl_del_memdev(struct device *host, struct cxl_memdev *cxlmd) +{ + devm_release_action(host, cxl_memdev_unregister, cxlmd); +} +EXPORT_SYMBOL_NS_GPL(devm_cxl_del_memdev, "CXL"); + static long __cxl_memdev_ioctl(struct cxl_memdev *cxlmd, unsigned int cmd, unsigned long arg) { diff --git a/include/cxl/cxl.h b/include/cxl/cxl.h index 28a39bfd74bc..e3bf8cf0b6d6 100644 --- a/include/cxl/cxl.h +++ b/include/cxl/cxl.h @@ -267,6 +267,7 @@ int cxl_await_range_active(struct cxl_dev_state *cxlds); struct cxl_memdev *devm_cxl_add_memdev(struct device *host, struct cxl_dev_state *cxlds, const struct cxl_memdev_ops *ops); +void devm_cxl_del_memdev(struct device *host, struct cxl_memdev *cxlmd); struct cxl_port; struct cxl_root_decoder *cxl_get_hpa_freespace(struct cxl_memdev *cxlmd, int interleave_ways, --=20 2.25.1 From nobody Sat Feb 7 12:25:21 2026 Received: from CY7PR03CU001.outbound.protection.outlook.com (mail-westcentralusazon11010022.outbound.protection.outlook.com [40.93.198.22]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A9B71329E69; Tue, 9 Dec 2025 16:51:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.93.198.22 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1765299121; cv=fail; b=Xuq0vygKAStUngzwbkn+2dzyKLvORCpMEbbohmfyoFBtItsB4T2qXIfKHYLPdESIA7rciJ99RDsMvthxANXw1dY9Uv3Byf23UdppmqAn0KnY6w8e0X/JYO9QHh7loyqbTlXYMdlHS3Og9SHLXAqhiTtdRqL3XhSR5jnyuOnOMco= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1765299121; c=relaxed/simple; bh=VX6dmpYaV/DtdieE4IzthBL4+VVnTlZqXOW3zqYP8C0=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=fKn9GKpRusSRZzuWrZFo9CuQqIsB6EqrUCm2qgKDbEQM/utvLMSyKlhOOh98xem/vheCev1OU6kbf0K2CNw6PADeCZ4wbbFNYNfO2MckroBj5m/nrxFucERhGgnTlHdUyQ2sQLXjskgURPWD5lWayvuoapbYpDF3mE83Vk5WGWo= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=texrwsyS; arc=fail smtp.client-ip=40.93.198.22 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="texrwsyS" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=MIvbvze9L97WSUdNJZJhiwEOmlVvyDci8B8s8j9VTm0GX5MiHNdlcdS2IemB6wHvnJXXO7PtCVzSPLVSX/l762Vl8zfBh2BJ+q8TNIkCX6PI2eNRGkoy+JfMy5X3fLX0oNFcQ82S6SnTriem1QIPeaL/6xFuyoJ9ICSAtbkgKtLiC3/adjF0xc2jJ06TuvjbGEgaSN6WccmSpk+4B1hasFP2xV8CaHc2X28K9kVgHHTqYwzB9pGxNYus/zAojm510Iql7+SD8gg7ozICgBcBcVbdCI2m9IE+NjyWokYnB+2+nlFRBks0Qp56ob7x7KHQ5jG6AZsfl+NtijMpflYw3A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Ii7KlQy8rAuBxrgwZtcN2qWS7O//IfeJd4bXPLj2Zkw=; b=wccvD4uBz42M48pWf+i69oENXZdJPWfxQqu/HgQoJwaivfkz2STp87BW74EkzSEgvShCazZN8EXDXOup9dwxbnu1o7gtPvyAhGKH4I5u2Dg0N9xEORiCcH3ZzbFV2QVac0Ttd3B2Grg3byQ3xPFUJWWr3qtaWUlu75GkkS6x96p7/gKTwLkH4726K4DeL1Z23UH5KjAUDQfivtxnslY2E50h9OcJuOs/TIyQcib8ZI9sP0zaGW+ORAfvPg59+Snv7wxf552Pgxc96BDCNAW9bM6ueQflM0Yw7nti5gECpP9LCx0K73VCtp8MQPt5w6/kf+OBvbJq5dFFR1qcw3ftjA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Ii7KlQy8rAuBxrgwZtcN2qWS7O//IfeJd4bXPLj2Zkw=; b=texrwsySW2fQ9sg5588YXGpLiupNqHEyYh91NMshMgMCFOdDUbgz/NVelBAySg2+4MuHAxiARHyPauel3BVLVKOUYy3pb8VYqIHr82kPB6ZWbmE7pKmstjkFNCEiBKhwgFvNYpnQqg0JrmMrDB58vurOylOI3PuBSFzxcTbM9EuIQdEie2xQNB+GhbuP12DN+15Wdfz+wPeRcBw9989lpZtUVQVqAE7QmOWZxKLYY9a8n+UaRZZVOPAh6tFOjcjAy2iOvAsHa3BQSxzKEGIlSipdPAPQHrUMIRErZGg6PVOAC175VOjXMEsHsaqKw8eHDxYq/EBzx3Jc74OyphzxjA== Received: from BN1PR12CA0030.namprd12.prod.outlook.com (2603:10b6:408:e1::35) by PH7PR12MB7210.namprd12.prod.outlook.com (2603:10b6:510:205::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9412.6; Tue, 9 Dec 2025 16:51:53 +0000 Received: from BN2PEPF0000449E.namprd02.prod.outlook.com (2603:10b6:408:e1:cafe::44) by BN1PR12CA0030.outlook.office365.com (2603:10b6:408:e1::35) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9412.7 via Frontend Transport; Tue, 9 Dec 2025 16:51:51 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by BN2PEPF0000449E.mail.protection.outlook.com (10.167.243.149) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9412.4 via Frontend Transport; Tue, 9 Dec 2025 16:51:51 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 9 Dec 2025 08:51:27 -0800 Received: from rnnvmail204.nvidia.com (10.129.68.6) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 9 Dec 2025 08:51:27 -0800 Received: from nvidia-4028GR-scsim.nvidia.com (10.127.8.11) by mail.nvidia.com (10.129.68.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20 via Frontend Transport; Tue, 9 Dec 2025 08:51:20 -0800 From: To: , , , , , , , , , , , , , , , , , CC: , , , , , , , , Subject: [RFC v2 05/15] cxl: introduce cxl_get_committed_regions() Date: Tue, 9 Dec 2025 22:20:09 +0530 Message-ID: <20251209165019.2643142-6-mhonap@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20251209165019.2643142-1-mhonap@nvidia.com> References: <20251209165019.2643142-1-mhonap@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN2PEPF0000449E:EE_|PH7PR12MB7210:EE_ X-MS-Office365-Filtering-Correlation-Id: 03ec27fd-7b0b-4e3b-c84b-08de37434059 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|82310400026|1800799024|36860700013|921020; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?WXFVm46XK9wl7zNHQfcbd+tDOPV44MVggNFcaDEpwDP99wGJusEJRd0VrDwt?= =?us-ascii?Q?Ui3thD9YsKH8PjN/ZfA1iAgpgE4U7j+I+7+HHCecKhLfbF8vluCNqOxPtfni?= =?us-ascii?Q?banr42wsAZb2G9q/IagjGrCr+H0pJwKSZH7IyESiy7D050za7fEZ/4Ge1ijW?= =?us-ascii?Q?5gZhJKDp9NaTFJnfdxQ7nWCk9t067D4mNwuBiJmgvBj3c4ZVwOte42513Fyv?= =?us-ascii?Q?h9lqy1i7aFaCBJ9qcgUPpiD0E8bs1nBFmkEs8FHZPJj6/jYYw3Xvs4tzpmp5?= =?us-ascii?Q?XC20DmIlAzGBbUeWa0aJqB4PNpnjQ/fFpX5faUITfG3YpYX6YkayJgzknllL?= =?us-ascii?Q?y240ucf8t0heLKK31kibpaVcruUDaQrF3whXPFxW2Qx2Pa5b7OQtnUR39kEW?= =?us-ascii?Q?l79Cg55vXL8leXdFJ7MV6YwUOI4/IQMSoP0vrfCS8zJXqfCh7f48p1KcJta8?= =?us-ascii?Q?JMm8/rO8XYpGb/ia4y4c5EmcRq95pmK79Uf7Uyvrhwe88XIx0raWrZvf2nE5?= =?us-ascii?Q?SzYp/TQ8KRIvVC3yiJHc6ulTsg0aN+xE+VIH8nJ/n3M/Ti+bb3hG37nMqEUz?= =?us-ascii?Q?CQq5GolKaW1HGEOmGS1f9LSQ85uL2Ao25O7Uq+wi93fGOPYi8LGilU1I4GDt?= =?us-ascii?Q?N8b7FjkK3O50IL1D7O26ECieFL/h35I2fGENIJ6AAx7nH4535Y9PAi57JzBM?= =?us-ascii?Q?zGabEiqCc78wiCMgnTRG10V6UphUoq/Ww6ujgfxrAADhhYDkVF++bDvfO+ss?= =?us-ascii?Q?2A9jnurYKrq37fJZzMJr2U2d2qaYopNtFggwiPbvueSePOan2jVdTzYmN2qU?= =?us-ascii?Q?GPftJokBLC20X7tHRGnXPthRDszWtrA1p3wS+gkLohd38p/LQM7W/VYTKGX7?= =?us-ascii?Q?Xrs2IXCaaS85vFU1Hn6AjWsbf1UWFkjBU0grflviLnvhY+nmg5RMuXGRLTWn?= =?us-ascii?Q?qAp3K3IPfK5LyTaFgpIXJNNnQLtkJ+htCAen540b6sGiY59RJAufsuc1K5BQ?= =?us-ascii?Q?D4Ue09EVyTKQWowaUDqMGxBzat7dwx37SYlC1A7ATB9hJkSptmBf/IB8Q+G4?= =?us-ascii?Q?93rQrESGyd96fvI5VGyhwVxdNrVwcQu52UokGdV2r48E/TYoIylEm1lBv7Ha?= =?us-ascii?Q?hcZ4lS3d3XcE05W1cE42JC+Y02hrxZAof4E+MEtEyxOgTkhSkj0/Iej3RqSL?= =?us-ascii?Q?1NAIX8RWTOJk6el50V2ybwlAm/L3iyIFOgOzTTe7r6/htH1MTUWaGMdVrcc2?= =?us-ascii?Q?WXsmCpFp1qiiYzb/Ugc4jgvNVDawfkGBliKkgtbGC5TD9JEl0AKFjrCIpsiN?= =?us-ascii?Q?zNresc9jz5TS0PQUw3KN4mkmHbAgb1DiQLMcT2whNH9r+7aBN+vDgA3v7p+L?= =?us-ascii?Q?Lwxknj35OvgJwD3tl5HpqOWfpKRitiyaRZZglDV6/9MSV2CLT+cOg1qSMoO/?= =?us-ascii?Q?k61w86RurkSNIOu/rcm0QQUv19QbZPrXiKIiiatISInNLq/MrVx81YBjqrP3?= =?us-ascii?Q?Pf1mbchTj3iAWx5yNNgvHMMTZuozViW+Lee+nbcMzjLLVgn+tR2Gs2iwKgvS?= =?us-ascii?Q?I9khjpfXhRkx/4BDNdg6TV/30D7JlalEs4GU3036?= X-Forefront-Antispam-Report: CIP:216.228.117.160;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge1.nvidia.com;CAT:NONE;SFS:(13230040)(7416014)(376014)(82310400026)(1800799024)(36860700013)(921020);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Dec 2025 16:51:51.3296 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 03ec27fd-7b0b-4e3b-c84b-08de37434059 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN2PEPF0000449E.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB7210 Content-Type: text/plain; charset="utf-8" From: Zhi Wang The kernel CXL core can discover the configured and committed CXL regions from BIOS or firmware, respect its configuration and create the related kernel CXL core data structures without configuring and committing the CXL region. However, those information are kept within the kernel CXL core. A type-2 device can have the same usage and a type-2 driver would like to know about it before creating the CXL regions. Introduce cxl_get_committed_regions() for a type-2 driver to discover the committed regions. Signed-off-by: Zhi Wang Signed-off-by: Manish Honap --- drivers/cxl/core/region.c | 73 +++++++++++++++++++++++++++++++++++++++ include/cxl/cxl.h | 1 + 2 files changed, 74 insertions(+) diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c index e89a98780e76..6c368b4641f1 100644 --- a/drivers/cxl/core/region.c +++ b/drivers/cxl/core/region.c @@ -2785,6 +2785,79 @@ int cxl_get_region_range(struct cxl_region *region, = struct range *range) } EXPORT_SYMBOL_NS_GPL(cxl_get_region_range, "CXL"); =20 +struct match_region_info { + struct cxl_memdev *cxlmd; + struct cxl_region **cxlrs; + int nr_regions; +}; + +static int match_region_by_device(struct device *match, void *data) +{ + struct match_region_info *info =3D data; + struct cxl_endpoint_decoder *cxled; + struct cxl_memdev *cxlmd; + struct cxl_region_params *p; + struct cxl_region *cxlr; + int i; + + if (!is_cxl_region(match)) + return 0; + + lockdep_assert_held(&cxl_rwsem.region); + cxlr =3D to_cxl_region(match); + p =3D &cxlr->params; + + if (p->state !=3D CXL_CONFIG_COMMIT) + return 0; + + for (i =3D 0; i < p->nr_targets; i++) { + void *cxlrs; + + cxled =3D p->targets[i]; + cxlmd =3D cxled_to_memdev(cxled); + + if (info->cxlmd !=3D cxlmd) + continue; + + cxlrs =3D krealloc(info->cxlrs, sizeof(cxlr) * (info->nr_regions + 1), + GFP_KERNEL); + if (!cxlrs) { + kfree(info->cxlrs); + return -ENOMEM; + } + info->cxlrs =3D cxlrs; + + info->cxlrs[info->nr_regions++] =3D cxlr; + } + + return 0; +} + +int cxl_get_committed_regions(struct cxl_memdev *cxlmd, struct cxl_region = ***cxlrs, int *num) +{ + struct match_region_info info =3D {0}; + int ret =3D 0; + + ret =3D down_write_killable(&cxl_rwsem.region); + if (ret) + return ret; + + info.cxlmd =3D cxlmd; + + ret =3D bus_for_each_dev(&cxl_bus_type, NULL, &info, match_region_by_devi= ce); + if (ret) { + kfree(info.cxlrs); + } else { + *cxlrs =3D info.cxlrs; + *num =3D info.nr_regions; + } + + up_write(&cxl_rwsem.region); + + return ret; +} +EXPORT_SYMBOL_NS_GPL(cxl_get_committed_regions, "CXL"); + static ssize_t __create_region_show(struct cxl_root_decoder *cxlrd, char *= buf) { return sysfs_emit(buf, "region%u\n", atomic_read(&cxlrd->region_id)); diff --git a/include/cxl/cxl.h b/include/cxl/cxl.h index e3bf8cf0b6d6..0a1f245557f4 100644 --- a/include/cxl/cxl.h +++ b/include/cxl/cxl.h @@ -295,5 +295,6 @@ int cxl_get_region_range(struct cxl_region *region, str= uct range *range); int cxl_get_hdm_reg_info(struct cxl_dev_state *cxlds, u64 *count, u64 *off= set, u64 *size); int cxl_find_comp_regblock_offset(struct pci_dev *pdev, u64 *offset); +int cxl_get_committed_regions(struct cxl_memdev *cxlmd, struct cxl_region = ***cxlrs, int *num); =20 #endif /* __CXL_CXL_H__ */ --=20 2.25.1 From nobody Sat Feb 7 12:25:21 2026 Received: from CH1PR05CU001.outbound.protection.outlook.com (mail-northcentralusazon11010060.outbound.protection.outlook.com [52.101.193.60]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C949C32A3E1; Tue, 9 Dec 2025 16:52:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.193.60 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1765299126; cv=fail; b=kUwK0qn2SUy8SM5v2bYZfLJ4og4IbK8VDPxs/o9EAvlLCGamAeJjF8WmfsSDyJIgJySe2gm8ggWsATU81aQZHn2+8oI9TuTB3dLtTDKcjfuqvWIEaBS0X9cMTe8T8td1ws8ps4Pn2d7MV9JVEJXFDYeE9OwPDdWZiSKB50UgDuo= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1765299126; c=relaxed/simple; bh=951ka7Sk4LrVp47oo15Y6iLyfZBLsHOdeqKZOVra0/4=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=QsRQNz19l4dVVoXTUfXEroTUlguHuKT8RwP4ItdDingVG3/filyh+GTnxVShshVjPNYrREESSyZMKzzhxB//ds4RuWI6Np/zg1EamRIPyDY23u7DZW323zpB0IWuKTBi63TXt4zS4Fi3cVjwSui3hNHoHR/WdKxAQaywTTtYd1Y= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=k1tIBDyT; arc=fail smtp.client-ip=52.101.193.60 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="k1tIBDyT" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=Rt5BObmMPt7aIzYiur8ybUt59ID8kOCBKQBZVofXMkFTP5SGnwA+flk5GgnY+dXRNiXM+c20x7BrcaXvWdlbZ+Pn16ppCa9pNfi9fdfBFZVUsGcxEkE/WI/0eVPF53QnlHDM2u8mMHxfHYViMzOo/CkTfA+D02WNznZ361QHrfe5r4IHLIYqZRlJvS+HTDH7C0AiABibfmDInwW87g+2oE3A73SfzOMAhaPlQ81rBmVcHJg3NeH+WFXtTO+/TbpDMXKRfAZjUQ1BhBSTux6pJYSiOLuGw+1JWp5JYeohsCbAyNsVkmUiHbFcKbcEwdjIl0Sf7h3MXOYUC9Dw31w56A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=j2i2s/RJrKAMrWW2J6u99rhQrB0igXe2skrHZz/+78s=; b=Nno83CUY00cXuSPunNj+QGlhjgpT/vC7SQAbssCE/nDzHTj/3AHFFsvN1W1a4IPLfTrBD0zWor5azLY8nrAnS7kwIMFbGpHfS3xNuP7Xs5lZ3jthSYnd+Pl+STGFQGvXpyznXspJFr2lM9FIXt/TZ0w5p1uBYoHTzmliAgpedhTV+AaoodpuppPjPmCHXSNwoXSxsOyDw0Ki+GLHcQz0eNCLt5Tw5YbDI4eYH1FCq7tteSyfPHAJr8AZFUg2YfA9esqyKLsJQjBuy3sb7x7WKMtAKHNHaOHcNKdcNbm6N5VxCrHQK0Zwy2cmxO1mqLqOUF91z0hTOEUoARojzrdcIQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=j2i2s/RJrKAMrWW2J6u99rhQrB0igXe2skrHZz/+78s=; b=k1tIBDyTlOL3kqHNk6KHvk6jIufMDYuhD8bl7gX/Or5Iwo4EoQ40ynD0i/kVBywmUjnhEyeXl36BF7Yh03ZBCSYXpVhDwptL15SSwfWAkV2CqTpvKyPWf+NWi5Y2gImSdfgfbC5MnjJYAA7e0C1h3G306w0DzXuSYCR0yqFNOZlAgNp9AWsxrxnV48YTe5U4yJsxO/rZRlaDW3ksuu6GaKgGZKF212eGjAskCW3ShwyYc/62a793qw4US6U4A8ppoLU7ViQ7vM2iigELzx8O3I8EuSoeJMbRXQ7EYORVD2/Cn5NJWgI+fx2SSUcuB4WHVazT9AQGUM9aBvU6N24StQ== Received: from SJ0PR03CA0365.namprd03.prod.outlook.com (2603:10b6:a03:3a1::10) by MW6PR12MB7085.namprd12.prod.outlook.com (2603:10b6:303:238::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9412.6; Tue, 9 Dec 2025 16:51:58 +0000 Received: from SJ1PEPF00002315.namprd03.prod.outlook.com (2603:10b6:a03:3a1:cafe::a2) by SJ0PR03CA0365.outlook.office365.com (2603:10b6:a03:3a1::10) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9388.14 via Frontend Transport; Tue, 9 Dec 2025 16:51:57 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by SJ1PEPF00002315.mail.protection.outlook.com (10.167.242.169) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9388.8 via Frontend Transport; Tue, 9 Dec 2025 16:51:58 +0000 Received: from rnnvmail202.nvidia.com (10.129.68.7) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 9 Dec 2025 08:51:35 -0800 Received: from rnnvmail204.nvidia.com (10.129.68.6) by rnnvmail202.nvidia.com (10.129.68.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 9 Dec 2025 08:51:34 -0800 Received: from nvidia-4028GR-scsim.nvidia.com (10.127.8.11) by mail.nvidia.com (10.129.68.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20 via Frontend Transport; Tue, 9 Dec 2025 08:51:27 -0800 From: To: , , , , , , , , , , , , , , , , , CC: , , , , , , , , Subject: [RFC v2 06/15] vfio/cxl: introduce vfio-cxl core preludes Date: Tue, 9 Dec 2025 22:20:10 +0530 Message-ID: <20251209165019.2643142-7-mhonap@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20251209165019.2643142-1-mhonap@nvidia.com> References: <20251209165019.2643142-1-mhonap@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ1PEPF00002315:EE_|MW6PR12MB7085:EE_ X-MS-Office365-Filtering-Correlation-Id: 39568ebc-c8ef-4b81-b160-08de374344a1 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|36860700013|7416014|376014|82310400026|921020; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?BcHHhdRWmCzEejDiqxO6R9cZTHM84D1ykzDY3MBU4amQD7vV2C9Y1xU/XcRI?= =?us-ascii?Q?fYQ9y7eTV8Cz5EEowUrBrJOPz+LP9vTJVEX3L4jFJZy88XuDGyRBPQYHDVPr?= =?us-ascii?Q?0yxQ4DAgbF0YXOkuFyZEHQ35v2MCu/v+RpoDYKbJ8neFHbHWssSPAy1Ttslt?= =?us-ascii?Q?GiY2FSziejAHEnEfXDGfcX5s+Mv1bK0vzsopYVUpKjUt2JtH7HNquTACcmz/?= =?us-ascii?Q?cszLJxe1OeKeDmY3mWHv7IzDRaQCyxX1MKDklp/zHHmtKsvvJPT/mmRLSoWv?= =?us-ascii?Q?ayh+mpiv+RRmKbBA1txvZY2yWPrPZ20Qm/m0V0VplIYyhZvKOW/wr9y+43ZV?= =?us-ascii?Q?cdIO39oH/7UfihqKPE5ndH0QUuL19QJ/n7co6SHgxLldKpx1E+vbk3hRAz91?= =?us-ascii?Q?N0vdJrvEeoZcT0CMO9DS+TlAmgaM/Aqv9LMl4Qr/kcOMXzf+DtfJxOdCuYGj?= =?us-ascii?Q?6l7clsPK0f9QWHGlLqZgPpY9OZOu7MC3tsCOB6zabpCJWoaF6xImNDZL4w6B?= =?us-ascii?Q?6XpdWK7qD9grXnsf3eGrGhVQAGfkB0aAAN9xHZMDhZDnOMHAYS38Q4gWf58m?= =?us-ascii?Q?dOO98P19OwqSiDu8dKYWglXodg25sPuKDifW3sQf426de/itbAkRx8Blk1Uy?= =?us-ascii?Q?EBZ76tdWSQDi8p5uA7hnKLR4989Wqra/E6IuB6dxZua2ZwRVvM/NQ7m8JOYt?= =?us-ascii?Q?IXv7kscQJHvOJpe6NNUkAh50TvJlpnaZSDtjf1KPuE7KQuHHAlF2M6iTy3zI?= =?us-ascii?Q?rc8lCvGL+LU5FN1AdHCYwAfLs3EcxLs/Qhm6EJwwH/CmgfkubDxFqrmmpH2/?= =?us-ascii?Q?/UshLZYHxMyquELoW1oz5XScbyvDIfXw2a66ZBKxIWpRe0Fq2gzCQa73vqrq?= =?us-ascii?Q?H81kedkuBWuXTuM8vLISvG4PSiQeyXqfZ8tl82dhxLXBUw6oxow5a6KJxCwJ?= =?us-ascii?Q?odjfm+6mIfbrmbDgPUbGT14i8swEpc7ewMpb1Fot/cyu84AdkfEXvRF48+IP?= =?us-ascii?Q?cJq+JJk10JAVKPBWnp6+WqWBP38lZ80+Rbxp5QPk19XWUl4g0DiUPNtlmhJX?= =?us-ascii?Q?AkWrH31JiQpZKIi+PSaJ0ZSzREwD4Hy5Sc+IFLR8W8IbNUdRDmQEgESBP2qi?= =?us-ascii?Q?wknzxDTeZ3ZcwNW4aBTOpye+PseR/lD/cclRRR917dvCVxPzoEgTU61qIWH7?= =?us-ascii?Q?cXSQ6Nn21WH8QMqWiNkv3+AlBA31cLSFpMw8/wmohl02E1apgrevenJNvntz?= =?us-ascii?Q?f1n67pamkgzpspj6LXdHx85Pxx2XTxzQvCRYRxXXvfth/oR9d+AnjoponYOn?= =?us-ascii?Q?cFqGawu8t6uH/cvahVwvWM0mLZ6Lf9baGngk5F7qet6G96n4hxAMZ7PZGR9e?= =?us-ascii?Q?vTEpKvjy0mxgMbegqa9f/tiPzWnt11Z9D/Zm86Qs72jpRQp8dAM/dbt39HII?= =?us-ascii?Q?NPZZr639qGJyeRSJPTJ4t6JlzdMc53p6oHhUcpaqiyrguMtKC26vZl60dZrc?= =?us-ascii?Q?0zWn/Q7DsGuc/A8qJgzSovRCKlC3kklcUn+XZxPrinC41qufGdMu/Tdzjixc?= =?us-ascii?Q?T3Y7lP+Cn7lJWrATPgPRiajRllXU9QjgRtCXJz97?= X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(1800799024)(36860700013)(7416014)(376014)(82310400026)(921020);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Dec 2025 16:51:58.5914 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 39568ebc-c8ef-4b81-b160-08de374344a1 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: SJ1PEPF00002315.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW6PR12MB7085 Content-Type: text/plain; charset="utf-8" From: Manish Honap In VFIO, common functions that used by VFIO variant drivers are managed in a set of "core" functions. E.g. the vfio-pci-core provides the common functions used by VFIO variant drviers to support PCI device passhthrough. Although the CXL type-2 device has a PCI-compatible interface for device configuration and programming, they still needs special handlings when initialize the device: - Probing the CXL DVSECs in the configuration. - Probing the CXL register groups implemented by the device. - Configuring the CXL device state required by the kernel CXL core. - Create the CXL region. - Special handlings of the CXL MMIO BAR. Introduce vfio-cxl core preludes to hold all the common functions used by VFIO variant drivers to support CXL device passthrough. Co-developed-by: Zhi Wang Signed-off-by: Zhi Wang Signed-off-by: Manish Honap --- drivers/vfio/pci/Kconfig | 10 ++ drivers/vfio/pci/Makefile | 3 + drivers/vfio/pci/vfio_cxl_core.c | 238 +++++++++++++++++++++++++++++++ include/linux/vfio_pci_core.h | 50 +++++++ 4 files changed, 301 insertions(+) create mode 100644 drivers/vfio/pci/vfio_cxl_core.c diff --git a/drivers/vfio/pci/Kconfig b/drivers/vfio/pci/Kconfig index 2b0172f54665..2f441d118f1c 100644 --- a/drivers/vfio/pci/Kconfig +++ b/drivers/vfio/pci/Kconfig @@ -7,6 +7,16 @@ config VFIO_PCI_CORE select VFIO_VIRQFD select IRQ_BYPASS_MANAGER =20 +config VFIO_CXL_CORE + tristate "VFIO CXL core" + select VFIO_PCI_CORE + depends on CXL_BUS + help + Support for the generic PCI VFIO-CXL bus driver which can + connect CXL devices to the VFIO framework. + + If you don't know what to do here, say N. + config VFIO_PCI_INTX def_bool y if !S390 depends on VFIO_PCI_CORE diff --git a/drivers/vfio/pci/Makefile b/drivers/vfio/pci/Makefile index cf00c0a7e55c..b51221b94b0b 100644 --- a/drivers/vfio/pci/Makefile +++ b/drivers/vfio/pci/Makefile @@ -8,6 +8,9 @@ vfio-pci-y :=3D vfio_pci.o vfio-pci-$(CONFIG_VFIO_PCI_IGD) +=3D vfio_pci_igd.o obj-$(CONFIG_VFIO_PCI) +=3D vfio-pci.o =20 +vfio-cxl-core-y :=3D vfio_cxl_core.o +obj-$(CONFIG_VFIO_CXL_CORE) +=3D vfio-cxl-core.o + obj-$(CONFIG_MLX5_VFIO_PCI) +=3D mlx5/ =20 obj-$(CONFIG_HISI_ACC_VFIO_PCI) +=3D hisilicon/ diff --git a/drivers/vfio/pci/vfio_cxl_core.c b/drivers/vfio/pci/vfio_cxl_c= ore.c new file mode 100644 index 000000000000..cf53720c0cb7 --- /dev/null +++ b/drivers/vfio/pci/vfio_cxl_core.c @@ -0,0 +1,238 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2025, NVIDIA CORPORATION & AFFILIATES. All rights reserved + */ + +#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "vfio_pci_priv.h" + +#define DRIVER_AUTHOR "Zhi Wang " +#define DRIVER_DESC "core driver for VFIO based CXL devices" + +/* Standard CXL-type 2 driver initialization sequence */ +static int enable_cxl(struct vfio_cxl_core_device *cxl, u16 dvsec, + struct vfio_cxl_dev_info *info) +{ + struct vfio_pci_core_device *pci =3D &cxl->pci_core; + struct vfio_cxl *cxl_core =3D cxl->cxl_core; + struct pci_dev *pdev =3D pci->pdev; + u64 offset, size, count; + int ret; + + ret =3D cxl_pci_setup_regs(pdev, CXL_REGLOC_RBI_COMPONENT, + &cxl_core->cxlds.reg_map); + if (ret) { + pci_err(pdev, "VFIO-CXL: CXL component registers not found\n"); + return ret; + } + + ret =3D cxl_get_hdm_reg_info(&cxl_core->cxlds, &count, &offset, &size); + if (ret) + return ret; + + if (WARN_ON(!count || !size)) + return -ENODEV; + + cxl->hdm_count =3D count; + cxl->hdm_reg_offset =3D offset; + cxl->hdm_reg_size =3D size; + + if (!info->no_media_ready) { + ret =3D cxl_await_range_active(&cxl_core->cxlds); + if (ret) + return -ENODEV; + + cxl_core->cxlds.media_ready =3D true; + } else { + /* Some devices don't have media ready support. E.g. AMD SFC. */ + cxl_core->cxlds.media_ready =3D true; + } + + if (cxl_set_capacity(&cxl_core->cxlds, SZ_256M)) { + pci_err(pdev, "dpa capacity setup failed\n"); + return -ENODEV; + } + + cxl_core->cxlmd =3D devm_cxl_add_memdev(&pdev->dev, + &cxl_core->cxlds, NULL); + if (IS_ERR(cxl_core->cxlmd)) + return PTR_ERR(cxl_core->cxlmd); + + cxl_core->region.noncached =3D info->noncached_region; + + return 0; +} + +static void disable_cxl(struct vfio_cxl_core_device *cxl) +{ + struct vfio_cxl *cxl_core =3D cxl->cxl_core; + + WARN_ON(cxl_core->region.region); + + if (!cxl->hdm_count) + return; + + if (cxl_core->cxled) { + cxl_decoder_detach(NULL, cxl_core->cxled, 0, DETACH_INVALIDATE); + cxl_dpa_free(cxl_core->cxled); + } + + if (cxl_core->cxlrd) + cxl_put_root_decoder(cxl_core->cxlrd); +} + +int vfio_cxl_core_enable(struct vfio_cxl_core_device *cxl, + struct vfio_cxl_dev_info *info) +{ + struct vfio_pci_core_device *pci =3D &cxl->pci_core; + struct pci_dev *pdev =3D pci->pdev; + struct vfio_cxl *cxl_core =3D cxl->cxl_core; + u16 dvsec; + int ret; + + dvsec =3D pci_find_dvsec_capability(pdev, PCI_VENDOR_ID_CXL, + PCI_DVSEC_CXL_DEVICE); + if (!dvsec) + return -ENODEV; + + cxl_core =3D devm_cxl_dev_state_create(&pdev->dev, CXL_DEVTYPE_DEVMEM, + pdev->dev.id, dvsec, struct vfio_cxl, + cxlds, false); + if (!cxl_core) { + pci_err(pdev, "VFIO-CXL: CXL state creation failed"); + return -ENOMEM; + } + + ret =3D vfio_pci_core_enable(pci); + if (ret) + return ret; + + ret =3D enable_cxl(cxl, dvsec, info); + if (ret) + goto err; + + return 0; + +err: + vfio_pci_core_disable(pci); + return ret; +} +EXPORT_SYMBOL_GPL(vfio_cxl_core_enable); + +void vfio_cxl_core_finish_enable(struct vfio_cxl_core_device *cxl) +{ + struct vfio_pci_core_device *pci =3D &cxl->pci_core; + + vfio_pci_core_finish_enable(pci); +} +EXPORT_SYMBOL_GPL(vfio_cxl_core_finish_enable); + +static void disable_device(struct vfio_cxl_core_device *cxl) +{ + disable_cxl(cxl); +} + +void vfio_cxl_core_disable(struct vfio_cxl_core_device *cxl) +{ + disable_device(cxl); + vfio_pci_core_disable(&cxl->pci_core); +} +EXPORT_SYMBOL_GPL(vfio_cxl_core_disable); + +void vfio_cxl_core_close_device(struct vfio_device *vdev) +{ + struct vfio_pci_core_device *pci =3D + container_of(vdev, struct vfio_pci_core_device, vdev); + struct vfio_cxl_core_device *cxl =3D vfio_pci_core_to_cxl(pci); + + disable_device(cxl); + vfio_pci_core_close_device(vdev); +} +EXPORT_SYMBOL_GPL(vfio_cxl_core_close_device); + +static int get_hpa_and_request_dpa(struct vfio_cxl_core_device *cxl, u64 s= ize) +{ + u64 max; + struct vfio_cxl *cxl_core =3D cxl->cxl_core; + + cxl_core->cxlrd =3D cxl_get_hpa_freespace(cxl_core->cxlmd, 1, + CXL_DECODER_F_RAM | + CXL_DECODER_F_TYPE2, + &max); + if (IS_ERR(cxl_core->cxlrd)) + return PTR_ERR(cxl_core->cxlrd); + + if (max < size) + return -ENOSPC; + + cxl_core->cxled =3D cxl_request_dpa(cxl_core->cxlmd, CXL_PARTMODE_RAM, si= ze); + if (IS_ERR(cxl_core->cxled)) + return PTR_ERR(cxl_core->cxled); + + return 0; +} + +int vfio_cxl_core_create_cxl_region(struct vfio_cxl_core_device *cxl, u64 = size) +{ + struct cxl_region *region; + struct range range; + int ret; + struct vfio_cxl *cxl_core =3D cxl->cxl_core; + + if (WARN_ON(cxl_core->region.region)) + return -EEXIST; + + ret =3D get_hpa_and_request_dpa(cxl, size); + if (ret) + return ret; + + region =3D cxl_create_region(cxl_core->cxlrd, &cxl_core->cxled, true); + if (IS_ERR(region)) { + ret =3D PTR_ERR(region); + cxl_dpa_free(cxl_core->cxled); + return ret; + } + + cxl_get_region_range(region, &range); + + cxl_core->region.addr =3D range.start; + cxl_core->region.size =3D size; + cxl_core->region.region =3D region; + return 0; +} +EXPORT_SYMBOL_GPL(vfio_cxl_core_create_cxl_region); + +void vfio_cxl_core_destroy_cxl_region(struct vfio_cxl_core_device *cxl) +{ + struct vfio_cxl *cxl_core =3D cxl->cxl_core; + + if (!cxl_core->region.region) + return; + + cxl_decoder_detach(NULL, cxl_core->cxled, 0, DETACH_INVALIDATE); + cxl_put_root_decoder(cxl_core->cxlrd); + cxl_dpa_free(cxl_core->cxled); + cxl_core->region.region =3D NULL; +} +EXPORT_SYMBOL_GPL(vfio_cxl_core_destroy_cxl_region); + +MODULE_LICENSE("GPL"); +MODULE_AUTHOR(DRIVER_AUTHOR); +MODULE_DESCRIPTION(DRIVER_DESC); +MODULE_IMPORT_NS("CXL"); +MODULE_SOFTDEP("pre: cxl_core cxl_port cxl_acpi cxl-mem"); diff --git a/include/linux/vfio_pci_core.h b/include/linux/vfio_pci_core.h index f541044e42a2..a343b91d2580 100644 --- a/include/linux/vfio_pci_core.h +++ b/include/linux/vfio_pci_core.h @@ -15,6 +15,8 @@ #include #include #include +#include +#include =20 #ifndef VFIO_PCI_CORE_H #define VFIO_PCI_CORE_H @@ -96,6 +98,40 @@ struct vfio_pci_core_device { struct rw_semaphore memory_lock; }; =20 +struct vfio_cxl_region { + struct cxl_region *region; + u64 size; + u64 addr; + bool noncached; +}; + +struct vfio_cxl { + struct cxl_dev_state cxlds; + struct cxl_memdev *cxlmd; + struct cxl_root_decoder *cxlrd; + struct cxl_port *endpoint; + struct cxl_endpoint_decoder *cxled; + + struct vfio_cxl_region region; +}; + +struct vfio_cxl_core_device { + struct vfio_pci_core_device pci_core; + struct vfio_cxl *cxl_core; + + u32 hdm_count; + u64 hdm_reg_offset; + u64 hdm_reg_size; +}; + +struct vfio_cxl_dev_info { + unsigned long *dev_caps; + struct resource dpa_res; + struct resource ram_res; + bool no_media_ready; + bool noncached_region; +}; + /* Will be exported for vfio pci drivers usage */ int vfio_pci_core_register_dev_region(struct vfio_pci_core_device *vdev, unsigned int type, unsigned int subtype, @@ -161,4 +197,18 @@ VFIO_IOREAD_DECLARATION(32) VFIO_IOREAD_DECLARATION(64) #endif =20 +static inline struct vfio_cxl_core_device * +vfio_pci_core_to_cxl(struct vfio_pci_core_device *pci) +{ + return container_of(pci, struct vfio_cxl_core_device, pci_core); +} + +int vfio_cxl_core_enable(struct vfio_cxl_core_device *cxl, + struct vfio_cxl_dev_info *info); +void vfio_cxl_core_finish_enable(struct vfio_cxl_core_device *cxl); +void vfio_cxl_core_disable(struct vfio_cxl_core_device *cxl); +void vfio_cxl_core_close_device(struct vfio_device *vdev); +int vfio_cxl_core_create_cxl_region(struct vfio_cxl_core_device *cxl, u64 = size); +void vfio_cxl_core_destroy_cxl_region(struct vfio_cxl_core_device *cxl); + #endif /* VFIO_PCI_CORE_H */ --=20 2.25.1 From nobody Sat Feb 7 12:25:21 2026 Received: from CH1PR05CU001.outbound.protection.outlook.com (mail-northcentralusazon11010048.outbound.protection.outlook.com [52.101.193.48]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B2BF9329E71; Tue, 9 Dec 2025 16:52:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.193.48 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1765299133; cv=fail; b=AojuyA+AbBFYGjYNAvAHcuzP6BOSwm9OfiuH2ZeWLF0roHNFGMSofRx8At3Ea1d/tskcStl44a+M+J1vUMbird9sse12Uli/vZMxn9SQdFU9CyiRhIDJwYgjjVWmfJNL1HXJwAB138anujOXG1ddGyQsnPkrgRaBpe9ix0o4xgw= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1765299133; c=relaxed/simple; bh=JDa3FakmOyI6yLONqrbhqzMDEy5mw6f2KcVVmJOJKow=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=sz0wZvI0GZpztbX+O30RPLl7UMJnLShLeqmNwcqxFpxwdRHibZUAwhpNyfvZ/DUC+tCLIU5V1ksXhyZZcW0s7S0JnxoIUi75FmX5tXmBpo9H0xTC2cKrcV9MWx1TSqOTopdnLMVdzLayuesWs5cFyftoi1JW4y3YXgs1C7r7zpY= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=k9LLCQ/W; arc=fail smtp.client-ip=52.101.193.48 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="k9LLCQ/W" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=r4z7siPEh7TWCG5ME76PKavrCx3vKik3Uk6a5msB2S8K0Phz7Ndw0ymVqF2usGfZiPgwOkcewstPrGFBSNn4pIcd7Xul4Y2Fy1HOf0R8v4ZF3MHadXXYLEBXlUzNGdsP71n0U+qEG+2pOBGayrHmOfNsubNCuqMkRsziIE7n/Ci01quXCzcasKlUyfdImdMbBBJeD7s+oUBzNm/Tv2+7ibpnZOKLWznUKvXUh9X4kXRkgF8diu4kNUsBfGSeO+5cKYm8e5K7+kUg4v6eT/zDQMiaDOn0wKr/CLCHjhL0ESyTLfNGMokCUPTxl/KoH8GCmP/s0AIq/ZJF2KXDAfqygA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=y/snwP4BuGyz2eSwZtk+MooEvLbQHXKvsUdpfLw5Tw4=; b=Fexi7KAR5AQbbtMO2woiR3ArZYj0FuCmAUmr25U6t0n58+6i+171EYCI869m5qPBW03QDdQT18IyseSjUhC6sCoOqUYOWxky2+ptJLZwTLsXnkK08uA6Y42z7hag5nEpYwtJ6db6smbvQ+2dzRTv9/R/YHkHbyu5h2IDJ0WPOBdIfORIwPyNioxjmAmIxqOYGP1S8Wwk+Yoxa269WRYmW8b3VJzKtvVsbYirXTkPY2jMLA/3/pl+tTMh6zwmF4be/DIdpdo8IJ1ApEYwHt3f2gFlOyrl2Uvyxlb1BaWe7vvw5r7vJjpUTFeyrmubNlb1qDhZv4YTzhAPqurYYiluug== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=y/snwP4BuGyz2eSwZtk+MooEvLbQHXKvsUdpfLw5Tw4=; b=k9LLCQ/WfBjU+EShHeM5ecFNTvID14OMb24RJVWxACkWLGrDPgPE98eOJCKTLdwVgCFnxhB1Clhe4P7JTl5e3Dxo6Y9Ye3nJ20bUCn9qmljalT5NbRhq1GZNTEfMJUYibbsKmb77F5HLBrcxF+UOM5enlad+CRwzvOV7O/MwAtpgL/JbqjeMxg4/mXK/c4ul+yaYtOZg6mOnp6R9qq29lK3g7mvnM+fEZaXmPwVkLSgvInJsruNAzpXxAiwTRZXLSGBeWeYJv3daOCCQ5jflHqSAUjewtwUYeYg5tkFCsZEbOeZXqu+SPg1lxmW46JCooWxT8WSFi7Ow2+UCxBj0XQ== Received: from BN9P221CA0027.NAMP221.PROD.OUTLOOK.COM (2603:10b6:408:10a::21) by PH8PR12MB7277.namprd12.prod.outlook.com (2603:10b6:510:223::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9388.14; Tue, 9 Dec 2025 16:52:07 +0000 Received: from BN2PEPF000044A4.namprd02.prod.outlook.com (2603:10b6:408:10a:cafe::48) by BN9P221CA0027.outlook.office365.com (2603:10b6:408:10a::21) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9388.14 via Frontend Transport; Tue, 9 Dec 2025 16:51:41 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by BN2PEPF000044A4.mail.protection.outlook.com (10.167.243.155) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9412.4 via Frontend Transport; Tue, 9 Dec 2025 16:52:07 +0000 Received: from rnnvmail205.nvidia.com (10.129.68.10) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 9 Dec 2025 08:51:42 -0800 Received: from rnnvmail204.nvidia.com (10.129.68.6) by rnnvmail205.nvidia.com (10.129.68.10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 9 Dec 2025 08:51:42 -0800 Received: from nvidia-4028GR-scsim.nvidia.com (10.127.8.11) by mail.nvidia.com (10.129.68.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20 via Frontend Transport; Tue, 9 Dec 2025 08:51:35 -0800 From: To: , , , , , , , , , , , , , , , , , CC: , , , , , , , , Subject: [RFC v2 07/15] vfio/cxl: expose CXL region to the userspace via a new VFIO device region Date: Tue, 9 Dec 2025 22:20:11 +0530 Message-ID: <20251209165019.2643142-8-mhonap@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20251209165019.2643142-1-mhonap@nvidia.com> References: <20251209165019.2643142-1-mhonap@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN2PEPF000044A4:EE_|PH8PR12MB7277:EE_ X-MS-Office365-Filtering-Correlation-Id: 2222d2a3-973f-437c-ba96-08de374349e5 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|1800799024|36860700013|376014|7416014|921020; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?DrFh6usQcfqN/jdveIKqzd0USd1CGbN/2gAJHi1N/rVQ+TJ3UUFGOzUfHcWK?= =?us-ascii?Q?nFsY/wzNDMK3XenCaTvAfoSree9vzfQ27E2BuZnZFwD/MKXZYI0mY4Fu5P/w?= =?us-ascii?Q?JtWupjOUoY0KN52DnPyCk1/4y8rBeGG+NREDEkokB5DiEfKWrCCA/thT6OIT?= =?us-ascii?Q?1KOx+IH2pqA0d0RTKm0IQcfDbUx/zalK9Jr+KrBurtiB7MqGnOToR1R3w57s?= =?us-ascii?Q?QJg56n5Kaj4uuqToo+gRVB8JMtUHNnrOklte91LtbQG1H89S0KORtMaiw3cv?= =?us-ascii?Q?1+XsQ5SuyBehWLK/wiQXO/KOXvrwjkm+m7BtBvrn3iHyAgy3cInW9VHtZc4W?= =?us-ascii?Q?UXmh0UNs8EOi4ALClXwnqXBFIZ4688tptZInxh1hjXMlGeuBOV7hLeaqeHT6?= =?us-ascii?Q?XAYDDhiqXGqLu/ufRrSBPSVYmMDkXnpLgSad8HOFa5Pgh4eRdwEwn8ag9IBG?= =?us-ascii?Q?+uvf/NX+GrOf+VlYqq+s8CH/k5rAsN7jPwoXdrMJYAGjiTdoZ7ZCQrYHBU61?= =?us-ascii?Q?yDhTSOvoMlYZID3qbY7IPWa24LSH4SPa035tcJNshC3vLoRpaHJHme4HTFPz?= =?us-ascii?Q?V18ueF2ZmN9BHiuB3VqFHyxHJesU2BcbTDJmJRRO9O5Fd9vz2p+VKkJVjS2P?= =?us-ascii?Q?4t0zQYgeNG+GaJPpUwohACzDu6Yr9ut3j/A2lXZH6HS9EyOyA+s1gqytaZBb?= =?us-ascii?Q?daPLiyocnN4Fuv2ftnajYfixQOttGJGUniE7N/Kw8fkX5K1qYiiDksmGa3aj?= =?us-ascii?Q?URCewVQD5awQ5MEORD6K6XKod9ywtpHyguXGvbszhh85sHM9WMBX2BtuFISP?= =?us-ascii?Q?0Bb7F8HjwQC1XEQKtQP94hL8H749w/kwnyf+R/PIUZi6mJCEKCzHM6TRX+y3?= =?us-ascii?Q?KfJ5yXG5A32sNUxI+CMRYaXX3TPKo74ksDbXOtjLJZXlOwyLHuOKpD+2l2Km?= =?us-ascii?Q?MMxxPIsF2u0EdNRAzKDEltj2HPP5mlHn3IZN0C3FXz3kTrxL4y3Gp7WCkMzO?= =?us-ascii?Q?uWfBIaUQ07ZU4d1kcV5fJYVtcCEnk+P60IsVh59pj2/Q8oov9Zbl/WchGQd5?= =?us-ascii?Q?wEWI/bw6Hf2hlmlSVLLXXZcCo9gC1+Jt8yqjkajChfr6E2S+YFNJzUGsG8FR?= =?us-ascii?Q?KDbYg4A4zc1KqFONU0a+hNbVzbvcaBge//bCuOZOF67Uvv0L1cRRug6qwrbF?= =?us-ascii?Q?bCF4tu+4K++gb84cbEpqdAVk3yrPbNe34gTFIQBmwkcnQ7IEwoNZYVq+Kbc9?= =?us-ascii?Q?fbth+s1OM0F11cXfKijSZ+ihFiWtVkyDSxndDHOxh0UDENY15gCeKDmu/qh2?= =?us-ascii?Q?gCf88IgzqO7WQWLngoR4IwLS+veCISJJbKEP+MVAzzkzzEYzzeKFRAvH6Jzq?= =?us-ascii?Q?M/L2w7B00Q/qvO8iZWRpJm1NojS4Dwalm4ZDLRSS3AxFinh4i/VFkRifzTbD?= =?us-ascii?Q?vOMQkcqCdA16pyYRLdMpGSfOwSLlwRPvkWvML3YzkIwmtnK7cJASvfir43II?= =?us-ascii?Q?cc10siPl+JI9HH03II0gO8Ny6c4/1817LCsm3CoXWcm6jYCI1Dlf+uglmGJr?= =?us-ascii?Q?SQYGTVb2H5uJboPEXth6540uu1esG7XfnSh7KCWR?= X-Forefront-Antispam-Report: CIP:216.228.117.160;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge1.nvidia.com;CAT:NONE;SFS:(13230040)(82310400026)(1800799024)(36860700013)(376014)(7416014)(921020);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Dec 2025 16:52:07.3514 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 2222d2a3-973f-437c-ba96-08de374349e5 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN2PEPF000044A4.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH8PR12MB7277 Content-Type: text/plain; charset="utf-8" From: Manish Honap To directly access the device memory, a CXL region is required. Creating a CXL region requires to configure HDM decoders on the path to map the access of HPA level by level and evetually hit the DPA in the CXL topology. For the userspace, e.g. QEMU, to access the CXL region, the region is required to be exposed via VFIO interfaces. Introduce a new VFIO device region and region ops to expose the created CXL region when initialize the device in the vfio-cxl-core. Introduce a new sub region type for the userspace to identify a CXL region. Co-developed-by: Zhi Wang Signed-off-by: Zhi Wang Signed-off-by: Manish Honap --- drivers/vfio/pci/vfio_cxl_core.c | 122 +++++++++++++++++++++++++++++++ drivers/vfio/pci/vfio_pci_core.c | 3 +- include/linux/vfio_pci_core.h | 5 ++ include/uapi/linux/vfio.h | 4 + 4 files changed, 133 insertions(+), 1 deletion(-) diff --git a/drivers/vfio/pci/vfio_cxl_core.c b/drivers/vfio/pci/vfio_cxl_c= ore.c index cf53720c0cb7..35d95de47fa8 100644 --- a/drivers/vfio/pci/vfio_cxl_core.c +++ b/drivers/vfio/pci/vfio_cxl_core.c @@ -231,6 +231,128 @@ void vfio_cxl_core_destroy_cxl_region(struct vfio_cxl= _core_device *cxl) } EXPORT_SYMBOL_GPL(vfio_cxl_core_destroy_cxl_region); =20 +static int vfio_cxl_region_mmap(struct vfio_pci_core_device *pci, + struct vfio_pci_region *region, + struct vm_area_struct *vma) +{ + struct vfio_cxl_region *cxl_region =3D region->data; + u64 req_len, pgoff, req_start, end; + int ret; + + if (!(region->flags & VFIO_REGION_INFO_FLAG_MMAP)) + return -EINVAL; + + if (!(region->flags & VFIO_REGION_INFO_FLAG_READ) && + (vma->vm_flags & VM_READ)) + return -EPERM; + + if (!(region->flags & VFIO_REGION_INFO_FLAG_WRITE) && + (vma->vm_flags & VM_WRITE)) + return -EPERM; + + pgoff =3D vma->vm_pgoff & + ((1U << (VFIO_PCI_OFFSET_SHIFT - PAGE_SHIFT)) - 1); + + if (check_sub_overflow(vma->vm_end, vma->vm_start, &req_len) || + check_add_overflow(PHYS_PFN(cxl_region->addr), pgoff, &req_start) || + check_add_overflow(PFN_PHYS(pgoff), req_len, &end)) + return -EOVERFLOW; + + if (end > cxl_region->size) + return -EINVAL; + + if (cxl_region->noncached) + vma->vm_page_prot =3D pgprot_noncached(vma->vm_page_prot); + vma->vm_page_prot =3D pgprot_decrypted(vma->vm_page_prot); + + vm_flags_set(vma, VM_ALLOW_ANY_UNCACHED | VM_IO | VM_PFNMAP | + VM_DONTEXPAND | VM_DONTDUMP); + + ret =3D remap_pfn_range(vma, vma->vm_start, req_start, + req_len, vma->vm_page_prot); + if (ret) + return ret; + + vma->vm_pgoff =3D req_start; + + return 0; +} + +static ssize_t vfio_cxl_region_rw(struct vfio_pci_core_device *core_dev, + char __user *buf, size_t count, loff_t *ppos, + bool iswrite) +{ + unsigned int i =3D VFIO_PCI_OFFSET_TO_INDEX(*ppos) - VFIO_PCI_NUM_REGIONS; + struct vfio_cxl_region *cxl_region =3D core_dev->region[i].data; + loff_t pos =3D *ppos & VFIO_PCI_OFFSET_MASK; + + if (!count) + return 0; + + return vfio_pci_core_do_io_rw(core_dev, false, + cxl_region->vaddr, + (char __user *)buf, pos, count, + 0, 0, iswrite); +} + +static void vfio_cxl_region_release(struct vfio_pci_core_device *vdev, + struct vfio_pci_region *region) +{ +} + +static const struct vfio_pci_regops vfio_cxl_regops =3D { + .rw =3D vfio_cxl_region_rw, + .mmap =3D vfio_cxl_region_mmap, + .release =3D vfio_cxl_region_release, +}; + +int vfio_cxl_core_register_cxl_region(struct vfio_cxl_core_device *cxl) +{ + struct vfio_pci_core_device *pci =3D &cxl->pci_core; + struct vfio_cxl *cxl_core =3D cxl->cxl_core; + u32 flags; + int ret; + + if (WARN_ON(!cxl_core->region.region || cxl_core->region.vaddr)) + return -EEXIST; + + cxl_core->region.vaddr =3D ioremap(cxl_core->region.addr, cxl_core->regio= n.size); + if (!cxl_core->region.addr) + return -EFAULT; + + flags =3D VFIO_REGION_INFO_FLAG_READ | + VFIO_REGION_INFO_FLAG_WRITE | + VFIO_REGION_INFO_FLAG_MMAP; + + ret =3D vfio_pci_core_register_dev_region(pci, + PCI_VENDOR_ID_CXL | + VFIO_REGION_TYPE_PCI_VENDOR_TYPE, + VFIO_REGION_SUBTYPE_CXL, + &vfio_cxl_regops, + cxl_core->region.size, flags, + &cxl_core->region); + if (ret) { + iounmap(cxl_core->region.vaddr); + cxl_core->region.vaddr =3D NULL; + return ret; + } + + return 0; +} +EXPORT_SYMBOL_GPL(vfio_cxl_core_register_cxl_region); + +void vfio_cxl_core_unregister_cxl_region(struct vfio_cxl_core_device *cxl) +{ + struct vfio_cxl *cxl_core =3D cxl->cxl_core; + + if (WARN_ON(!cxl_core->region.region || !cxl_core->region.vaddr)) + return; + + iounmap(cxl_core->region.vaddr); + cxl_core->region.vaddr =3D NULL; +} +EXPORT_SYMBOL_GPL(vfio_cxl_core_unregister_cxl_region); + MODULE_LICENSE("GPL"); MODULE_AUTHOR(DRIVER_AUTHOR); MODULE_DESCRIPTION(DRIVER_DESC); diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_c= ore.c index 7dcf5439dedc..c0695b5db66d 100644 --- a/drivers/vfio/pci/vfio_pci_core.c +++ b/drivers/vfio/pci/vfio_pci_core.c @@ -1698,12 +1698,13 @@ static vm_fault_t vfio_pci_mmap_page_fault(struct v= m_fault *vmf) return vfio_pci_mmap_huge_fault(vmf, 0); } =20 -static const struct vm_operations_struct vfio_pci_mmap_ops =3D { +const struct vm_operations_struct vfio_pci_mmap_ops =3D { .fault =3D vfio_pci_mmap_page_fault, #ifdef CONFIG_ARCH_SUPPORTS_HUGE_PFNMAP .huge_fault =3D vfio_pci_mmap_huge_fault, #endif }; +EXPORT_SYMBOL_GPL(vfio_pci_mmap_ops); =20 int vfio_pci_core_mmap(struct vfio_device *core_vdev, struct vm_area_struc= t *vma) { diff --git a/include/linux/vfio_pci_core.h b/include/linux/vfio_pci_core.h index a343b91d2580..3474835f5d65 100644 --- a/include/linux/vfio_pci_core.h +++ b/include/linux/vfio_pci_core.h @@ -102,6 +102,7 @@ struct vfio_cxl_region { struct cxl_region *region; u64 size; u64 addr; + void *vaddr; bool noncached; }; =20 @@ -203,6 +204,8 @@ vfio_pci_core_to_cxl(struct vfio_pci_core_device *pci) return container_of(pci, struct vfio_cxl_core_device, pci_core); } =20 +extern const struct vm_operations_struct vfio_pci_mmap_ops; + int vfio_cxl_core_enable(struct vfio_cxl_core_device *cxl, struct vfio_cxl_dev_info *info); void vfio_cxl_core_finish_enable(struct vfio_cxl_core_device *cxl); @@ -210,5 +213,7 @@ void vfio_cxl_core_disable(struct vfio_cxl_core_device = *cxl); void vfio_cxl_core_close_device(struct vfio_device *vdev); int vfio_cxl_core_create_cxl_region(struct vfio_cxl_core_device *cxl, u64 = size); void vfio_cxl_core_destroy_cxl_region(struct vfio_cxl_core_device *cxl); +int vfio_cxl_core_register_cxl_region(struct vfio_cxl_core_device *cxl); +void vfio_cxl_core_unregister_cxl_region(struct vfio_cxl_core_device *cxl); =20 #endif /* VFIO_PCI_CORE_H */ diff --git a/include/uapi/linux/vfio.h b/include/uapi/linux/vfio.h index 75100bf009ba..95be987d2ed5 100644 --- a/include/uapi/linux/vfio.h +++ b/include/uapi/linux/vfio.h @@ -372,6 +372,10 @@ struct vfio_region_info_cap_type { /* sub-types for VFIO_REGION_TYPE_GFX */ #define VFIO_REGION_SUBTYPE_GFX_EDID (1) =20 +/* 1e98 vendor PCI sub-types */ +/* sub-type for VFIO CXL region */ +#define VFIO_REGION_SUBTYPE_CXL (1) + /** * struct vfio_region_gfx_edid - EDID region layout. * --=20 2.25.1 From nobody Sat Feb 7 12:25:21 2026 Received: from PH7PR06CU001.outbound.protection.outlook.com (mail-westus3azon11010029.outbound.protection.outlook.com [52.101.201.29]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3968732AAA9; Tue, 9 Dec 2025 16:52:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.201.29 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1765299142; cv=fail; b=lDDKU2Hgx9Y8jEKwgLOkJLIJzrDiQxypLuI2KOURBkEX+c2+NNxFsmihg2jUX22JBrSHvZyrLmrMK0m9FgOn1NzYqzfnV1vuG/5ITeaSV8fk5tGdrshuumAbxaj9HaRIlkbkP3A4sfLYnw0KJ4vwh4A1EkNnH05B2fPNC/k8pPs= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1765299142; c=relaxed/simple; bh=V01VIXclixPpIeGfeixd8RCXc5VjsPNTX2zIGtE9Br0=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=twincC+886pHvoVZA1k9Si6jYjOajNUJINzbVBkq/L9o5V90yPa0GwchCrNA1QFmjoSYHsAV/9Z4TJyacJ7wlx7nR5KvuuWaowz3WSI1/6MD22yzZEYVk5sjqqIy4YR4O1NaASLUrnXsXLOLnC1F+QOkJ0zonzuli9hxsaEdmQ4= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=ZPM/3mJh; arc=fail smtp.client-ip=52.101.201.29 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="ZPM/3mJh" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=jHsI1W5JAQX2sWk9e3/ZTzWFXehvU9UTJhAObURMB8GAoc03VvsjEYhNMwTQXE34AKzIK2HTPxbJMIaxtpjIrYrtXjNjXfhuhAUn1nLfyDGfyxpBu1BXke2qONUdxLP6gL0mhAjeKBRTxPECdirOdzpZ720iV5FsfePsCByiT5YjBDAiFoaTSw3MHensVhodqzvp2gg6Rqs/YjYkJ/occt9YR8pmFTFoVETekQqUf6J9r8VQVTsK9ypMD/M3BNFN8vRMHdooUw8O5dnORWyOfvcYJUpVMmUxkvBaaaDnLALj+7N0zKlukO6VePl8PEX8jKOyMNhJnPtfHGZ8g/ZSlA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=V+9gIC2Cm69KDeJdD63DRf5hoLwQO/+SRZhfxBse9gI=; b=FWoJ/+k9xopPuafNDQE+omiEBniYmUGqFnBrY/RxAJ4kV1gRWZzpr/GnTHeDkaBryUZlqbSYsDdsRGVw1fRIahGaKRyuPisABAYKvNvD2LDG9ct80i7HVyyfPR+Qq4M22oJt+IpNv29yrkGHOEiQw6bpeWlDhlMZvrJQ5QkGtcNrMC/HHaXAYOBz52w8a0QHoZcg/qFLHmAs5IyANtzvKp4obHE1kSXSi1BqWpmmadcZP/9TkwpngRXzRre3rIexu7mFc8aDU6ZUoNBAU+jSci+Eeth0InB8oi0vOq0ow8f6h+AEI+dCZSwsON3En0H9OJdfHaj9nOb4sI+wubW4vA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=V+9gIC2Cm69KDeJdD63DRf5hoLwQO/+SRZhfxBse9gI=; b=ZPM/3mJh+q7ZrNRQts7+UlI863Z3IG0oAKSJWZlay7Fm9ovVUSE02fdJB4CeYpAHNIc8s2CLA0U1tLZSj1avjmiELABWLwFFz8/lTmSJaOfID/bfXvmV6KU6O3uzC6fupzvRCNcP3qdOdCbp+7UlpvgiZ6V1CMLoqUreIlEtcI6arCiGv9wC5txIvuE69k0QeTEp72/xLajKmuX1Y91dmtyCbbWbU31XRs3AMI1Hd75IK/O6e7+G2db0Yu56guBUqSaiQNhPhykNx90tX3SLKG9+A6fbvVhMMWTADw8GjcxkllUdy5obGo7k49bS5lZDctIQZdgGwKdtn9awBi0VgQ== Received: from BN0PR08CA0016.namprd08.prod.outlook.com (2603:10b6:408:142::35) by CYYPR12MB8922.namprd12.prod.outlook.com (2603:10b6:930:b8::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9412.7; Tue, 9 Dec 2025 16:52:15 +0000 Received: from BN2PEPF0000449D.namprd02.prod.outlook.com (2603:10b6:408:142:cafe::33) by BN0PR08CA0016.outlook.office365.com (2603:10b6:408:142::35) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9412.7 via Frontend Transport; Tue, 9 Dec 2025 16:52:15 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by BN2PEPF0000449D.mail.protection.outlook.com (10.167.243.148) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9388.8 via Frontend Transport; Tue, 9 Dec 2025 16:52:15 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 9 Dec 2025 08:51:50 -0800 Received: from rnnvmail204.nvidia.com (10.129.68.6) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 9 Dec 2025 08:51:49 -0800 Received: from nvidia-4028GR-scsim.nvidia.com (10.127.8.11) by mail.nvidia.com (10.129.68.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20 via Frontend Transport; Tue, 9 Dec 2025 08:51:42 -0800 From: To: , , , , , , , , , , , , , , , , , CC: , , , , , , , , Subject: [RFC v2 08/15] vfio/cxl: discover precommitted CXL region Date: Tue, 9 Dec 2025 22:20:12 +0530 Message-ID: <20251209165019.2643142-9-mhonap@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20251209165019.2643142-1-mhonap@nvidia.com> References: <20251209165019.2643142-1-mhonap@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN2PEPF0000449D:EE_|CYYPR12MB8922:EE_ X-MS-Office365-Filtering-Correlation-Id: a66f9e66-b26d-46b4-8963-08de37434ea0 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|7416014|1800799024|36860700013|82310400026|921020; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?6Kwu0DCdfZaHk2xKVmIetWPTQQm4W/HdgtU6nf9L+yjkp9HKIGmIiZM7TnDe?= =?us-ascii?Q?XFfW0k3u/p7hnKmmDBOStHW6aGBEZZ7wHRaSlM+OLlp4ouWpAHYcgY3MladO?= =?us-ascii?Q?+4tAU3bXWh6mEZcyhPu/EZQyYNOfPMHfUtUGz/vMZApivlvh9Qj3ye1dLYiY?= =?us-ascii?Q?4hkp7RG917MRk9R7YenEljPlphd+5HJe1hnFj7luxTv0v/L0IXdw909K4q4L?= =?us-ascii?Q?K0BfKt1m0eqq+jwgIYk145Wl4rgsGUQtiaXQYJlJ9vGTpPneQ64X3bmtxAd/?= =?us-ascii?Q?5qRSMDgb++yJ9zSlaYorQro067JcuCbMtyJWX5YO0IR0rU50Xwzjj/lrv4Xh?= =?us-ascii?Q?blznRiDrPM3tovu/9a5Iu0bRukwpzIDz+FISjvUo4UnhpiZrQJ9oSaPTcsS1?= =?us-ascii?Q?t2Y9Csy3f3kg45dM35BKrTVvGxS7+OkDsc8l5Kt9oZYGqPDnaEAqBnp0ZQeW?= =?us-ascii?Q?hGH3PDJUY6kTWsYbmNIWhayaY9ZR5M5hN5DEZYxiWyI41ubu0W/L9HDoCc3N?= =?us-ascii?Q?NsUY2zQTHJjG3g13S4b3a+tqsOnwsxlFFovmpFg7Uj9DEO8zUjbaFgr7ygpA?= =?us-ascii?Q?0zlgny9H+jqSwzIvbjyDIFlkpX3vytAeJiyHyH6MYs+c6OKgAuh3ZlA4eM7z?= =?us-ascii?Q?WqvmQQ7RZgZ55+TXVwVg0iWdnnr8cK2wJEmX6Y83Lx4NMo2d1hUEpNQDa3kp?= =?us-ascii?Q?/3Pj+tTD+tEfOxP8CvDbSAHIbTzBZfPXuvC9+ksE5p7txuUI1nXRQfjRbPcJ?= =?us-ascii?Q?wTt8hqV8ZmxWMornmttpt1OJ/YC6WMGvxZY74GxfgqtJb0J1FOgHDOt9mVmD?= =?us-ascii?Q?7AoptR6UjjZ+oz5FfXJO3xCt53wzWC5wWT64FdIm9Ky6LtD/3tgsIlb6UPGr?= =?us-ascii?Q?dJJjn69qgGsvwfYG53L6dcGsFkdC59j1nyBZ2JEnJ7ofNlfncH6rON19SaRo?= =?us-ascii?Q?gErF9sHq0F3i2VXFvE7/0EojnZWKhIRkkoAHbwwttvxyGkusijx2uo6mEIzj?= =?us-ascii?Q?aS8dC0dPdyTCVtK/ukDnyOrXAIO+hqs3N/Fgw+bdvUgP6C9+/R96b7Xmgdyz?= =?us-ascii?Q?26jFiOgTUJAppUkpmN38BV07xFZB1cZmqPQDla3CDn739c6Od/c1bWmJWKYZ?= =?us-ascii?Q?Gq+bzNajUrglK02GoZviYwJ3m6o0tkSaClVTeTIKKPi1b+a7paPaCWcopKeB?= =?us-ascii?Q?YJPXpbSfk/rEMQGwuvdSWzkn0CRFaJZU9iGbmbd6r3Dl9fx+qguY8/nVBbyi?= =?us-ascii?Q?6+Axuw2ecMD9IrJCB+agqiLA1zbntIeGx+zf9NMsOOfCaajnfVwahLnNulOa?= =?us-ascii?Q?T80gv5FAfYze3VQQlVxwaaGcNegLSoKwW/uLJwc+e480dEhNvGHwmKBsq8yb?= =?us-ascii?Q?WoxUHrVV/IgP+ZExKaVYu3TCYi6THYB6rQNv/kkzjL1W6N1XXI9z0mLUUvzU?= =?us-ascii?Q?mmd6IXM08m9NGzUKngYRjRVM7rft+DpqGtMzfh238kwzl9Rn6K3rzDxOBZVL?= =?us-ascii?Q?ndUp1JhXUldM3538bluBjiFc8LQQ9NNenG9MCFjjGoq4kjklj5xC6CRuqQqY?= =?us-ascii?Q?g7ORGoFeSMOi4bwYPcC6TWwYmo7quLTkI2IrTGlq?= X-Forefront-Antispam-Report: CIP:216.228.117.160;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge1.nvidia.com;CAT:NONE;SFS:(13230040)(376014)(7416014)(1800799024)(36860700013)(82310400026)(921020);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Dec 2025 16:52:15.2810 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: a66f9e66-b26d-46b4-8963-08de37434ea0 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN2PEPF0000449D.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CYYPR12MB8922 Content-Type: text/plain; charset="utf-8" From: Zhi Wang A type-2 device can have precommitted CXL region that is configured by BIOS. Before letting a VFIO CXL variant driver create a new CXL region, the VFIO CXL core first needs to discover the precommited CXL region. Discover the precommited CXL region when enabling CXL devices. Signed-off-by: Zhi Wang Signed-off-by: Manish Honap --- drivers/vfio/pci/vfio_cxl_core.c | 29 +++++++++++++++++++++++++++-- include/linux/vfio_pci_core.h | 1 + 2 files changed, 28 insertions(+), 2 deletions(-) diff --git a/drivers/vfio/pci/vfio_cxl_core.c b/drivers/vfio/pci/vfio_cxl_c= ore.c index 35d95de47fa8..099d35866a39 100644 --- a/drivers/vfio/pci/vfio_cxl_core.c +++ b/drivers/vfio/pci/vfio_cxl_core.c @@ -82,11 +82,16 @@ static void disable_cxl(struct vfio_cxl_core_device *cx= l) { struct vfio_cxl *cxl_core =3D cxl->cxl_core; =20 - WARN_ON(cxl_core->region.region); - if (!cxl->hdm_count) return; =20 + if (cxl_core->region.precommitted) { + kfree(cxl_core->region.region); + cxl_core->region.region =3D NULL; + } + + WARN_ON(cxl_core->region.region); + if (cxl_core->cxled) { cxl_decoder_detach(NULL, cxl_core->cxled, 0, DETACH_INVALIDATE); cxl_dpa_free(cxl_core->cxled); @@ -96,6 +101,24 @@ static void disable_cxl(struct vfio_cxl_core_device *cx= l) cxl_put_root_decoder(cxl_core->cxlrd); } =20 +static void discover_precommitted_region(struct vfio_cxl_core_device *cxl) +{ + struct cxl_region **cxlrs =3D NULL; + struct vfio_cxl *cxl_core =3D cxl->cxl_core; + int num, ret; + + ret =3D cxl_get_committed_regions(cxl_core->cxlmd, &cxlrs, &num); + if (ret || !cxlrs) { + kfree(cxlrs); + return; + } + + WARN_ON(num > 1); + + cxl_core->region.region =3D cxlrs[0]; + cxl_core->region.precommitted =3D true; +} + int vfio_cxl_core_enable(struct vfio_cxl_core_device *cxl, struct vfio_cxl_dev_info *info) { @@ -126,6 +149,8 @@ int vfio_cxl_core_enable(struct vfio_cxl_core_device *c= xl, if (ret) goto err; =20 + discover_precommitted_region(cxl); + return 0; =20 err: diff --git a/include/linux/vfio_pci_core.h b/include/linux/vfio_pci_core.h index 3474835f5d65..7237fcaecbb6 100644 --- a/include/linux/vfio_pci_core.h +++ b/include/linux/vfio_pci_core.h @@ -104,6 +104,7 @@ struct vfio_cxl_region { u64 addr; void *vaddr; bool noncached; + bool precommitted; }; =20 struct vfio_cxl { --=20 2.25.1 From nobody Sat Feb 7 12:25:21 2026 Received: from CH4PR04CU002.outbound.protection.outlook.com (mail-northcentralusazon11013006.outbound.protection.outlook.com [40.107.201.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7990732ABC5; Tue, 9 Dec 2025 16:52:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.201.6 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1765299152; cv=fail; b=PYIHrsQHakRcRCb/lvYYJZp1dFFQT+K6lNeRRfZdDaaqUa+PGE4JMaODeGtRKvM1JX/02VlLqJlx3lbtFTgDpjbZ/oYiEDxApoIyazrzonm9ifmTTjzd3/HK5v3C/NCzLftWTVzfCu2aGu5Tpi5l0xFwYd/wo+DSu4zjSJ9pg8E= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1765299152; c=relaxed/simple; bh=UeyPhrM/WF8/9yddaW0mNMJ87bCij1eS154BdW4Knhc=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=f6Cn+SdzbeFCmZbohd3gdft/3YGAdX5j/+xE+9RvrZwCc+I+vX1GeikKtJoKXOMdyjm/akjerFfhlPtBvZrfSGTzBVWq7wnQ1X9jcQERUzNWOlsOTlRnj6/SVgbghKbTKQxHmx/xTGV4gWPNjwYbOHj9MLTGK53mgKD0XK1cOgY= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=J2Y/dg/g; arc=fail smtp.client-ip=40.107.201.6 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="J2Y/dg/g" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=Qvp5cR6PbXi9p/NirNNdmCBNsgEM4bbcwDdArRH+4UNpBbVbn28J2PhtSIhEg0Swe5NYBlZmAVnR/hqpja08xUnNwmQ5dFS2BcVqpccfGcyRUIDjNfqJq+PZB0Ld5eTeBEV26eBOxp7Q4+VmIXbY/fnO8UR5g33sGCg7pVb99g1Uud1bjELaMdr4hUpmsTM7kaHozLdbq/J/KaYFIY9f+irLA2uEDwPoEfttBwJF24zamhmS3DGZdyHLck2ifFJCdmbKp3X0Nu0UpplR67LCmr7/dN9kvgT/oBgEDU+tG8vxRTIs+exDM1N7cuDtDMsYZ0Qyrl6DHPPiPeOzKu6u4g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=dDNiUao1tvoiy9Me5dv49zj7AHCAFB8aSpXfsP2yHXY=; b=UFfbNLlkO3Bw+Gw5BU9D7Gi7PA8uPPmWkBPgGhhCXN97wJ3JI86uVxlNtmdCVciMNIT0g9WOINEBb9cUSB0h8IWFnVglAziMXAO7dkQghdEfTJnf3S3bC8myaxJubIK1Q1wSJC55WRVSZqREbN+yBL3aFrREGW331ln7jVB6xEn2la54mJyeIBdB50X/qpPbadRiyosjsjj7EEjoTGVjCLCkqU2IvtM/nfpH+P+YrBOi+7bDrZ5gZIJeNj5i2N8FKvFdsjze74OZGCkC9foJlyUy9NEXEkJsEhZFGfqq6/SqBzoRgU6VZ+s4SBfl0sInyVKp83sxzurC3DUTLFaLkQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=dDNiUao1tvoiy9Me5dv49zj7AHCAFB8aSpXfsP2yHXY=; b=J2Y/dg/g+Qwjk5dysv3Pvg+66CuK80i9wTE9JomjyVbJSDo+GjPM6hSdsR3qmdl38tc4ctuwzD7n3jJJUFnL8WWUdbPg5gg6Pwtub0feGr/5VZzoGp1f5esn+sIRGkhXFUT7EzwCvgaIKnG91Z4UJYmXjhVI36LAXZZGDOFbvEpL193pn4s8WUWPXq8XG+12z7BKAUoRbvdkx9UnSgjDSUEYgjClXV0qhmblpQ8HQDoVIyENZqmDPkU24jDitjapNnwuvR+ecAMkQn7ADV0s6zFIUGA9u+LFFR2p9J4m8LufieX21rti/zpTEKc7aIFf3ByKITfFrXfV/j7Eyf8x7w== Received: from BY5PR03CA0019.namprd03.prod.outlook.com (2603:10b6:a03:1e0::29) by IA1PR12MB6306.namprd12.prod.outlook.com (2603:10b6:208:3e6::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9388.14; Tue, 9 Dec 2025 16:52:25 +0000 Received: from SJ1PEPF00002312.namprd03.prod.outlook.com (2603:10b6:a03:1e0:cafe::4f) by BY5PR03CA0019.outlook.office365.com (2603:10b6:a03:1e0::29) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9388.14 via Frontend Transport; Tue, 9 Dec 2025 16:52:23 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by SJ1PEPF00002312.mail.protection.outlook.com (10.167.242.166) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9388.8 via Frontend Transport; Tue, 9 Dec 2025 16:52:24 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 9 Dec 2025 08:51:57 -0800 Received: from rnnvmail204.nvidia.com (10.129.68.6) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 9 Dec 2025 08:51:57 -0800 Received: from nvidia-4028GR-scsim.nvidia.com (10.127.8.11) by mail.nvidia.com (10.129.68.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20 via Frontend Transport; Tue, 9 Dec 2025 08:51:50 -0800 From: To: , , , , , , , , , , , , , , , , , CC: , , , , , , , , Subject: [RFC v2 09/15] vfio/cxl: introduce vfio_cxl_core_{read, write}() Date: Tue, 9 Dec 2025 22:20:13 +0530 Message-ID: <20251209165019.2643142-10-mhonap@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20251209165019.2643142-1-mhonap@nvidia.com> References: <20251209165019.2643142-1-mhonap@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ1PEPF00002312:EE_|IA1PR12MB6306:EE_ X-MS-Office365-Filtering-Correlation-Id: 62efd164-6433-407f-3f3b-08de37435441 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|36860700013|82310400026|376014|7416014|1800799024|921020; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?tPU5Xd8IzMSnBAtzl5xkTn61hxAEfzlHPRmPhEKvCx/NoikzBnXdTlcCN9r8?= =?us-ascii?Q?ABgmxbSyXxqe9Y6Tab/5cUzw4LFm8MasdfdvmBGrQkbfVWZd4CWRfrzXLZne?= =?us-ascii?Q?sueXIsUmWlQjbBt9/cyExMFDdQV2tG8IVT6vbL2POgetQZwn5SByqJ4O7w8y?= =?us-ascii?Q?eyQWasKpPpzKBdzRwsmkJknWGR8oCcfIYuusWVHwcl37xsA3ByHSlOeqIhKg?= =?us-ascii?Q?ruOSymlNIWYy1JZV9R8gjHdK/X3XlnxpQbMK/AE4nZdowU+FZUKwyHHHRKzZ?= =?us-ascii?Q?ffVRiVLgoaJ9yhAKP3StoXJzKPeGR1KL9nXmGLWIU/p9E11kGKZd1zMNYQVv?= =?us-ascii?Q?lPcpDZh8t/V+G2R1P9u1YwwqbH4BV+u41XtiVz/iGc8b8OHLzvc3nzNRFhqj?= =?us-ascii?Q?m0jZC4ZMnfUDNMnFVmpDLFw3LLJ7XlRpXuCC2DULfzQIvJduCor9Rq831FZc?= =?us-ascii?Q?gZNlZRHoJa8UiTN5taXaUzcIjIHPEvqvpq6ikRXyW7WLIqAFx4JtRyoVMoAb?= =?us-ascii?Q?fT7cd4Ua2v3PKdvh8ieVF7H+2RUp6Z0zvRDQc6lk0+nMoJPPUmy80lZBKA99?= =?us-ascii?Q?6HhLqBoQ+Qw4ZCCvcMnD4cd/76lG+7PMErYh0xVrZi7OxZ2jUckEYgxr3keS?= =?us-ascii?Q?AXgr1g+qstVG5u+tZy+AnpaV5GXHlgGibJDZjdxoMj826EYoc29hr0E2sbmW?= =?us-ascii?Q?pwEYbEEau6k42P418e/pQJcR6LRUWJiAVD2xK8CzlIHV13PXeYyV4pRiC3VZ?= =?us-ascii?Q?egHuZ16goQJY/CjxwZ7UukH8mH8lTG/CIiEZY0CqGVklB+a6ipkZRBMRI2gk?= =?us-ascii?Q?QA5TGbp3WQXTkJAC2qoNLOQb34w+Hv3bow2NfYxOdBIZbt1d+Ub7N467vLlb?= =?us-ascii?Q?brUeK3QbBZGnGcLiF0FyqPNwihhMhGVLIFfJwSsjpC3VHNpYaMGbdEQOyfNn?= =?us-ascii?Q?ZdKebOGlu4mfDtl9B1p2AMlZrcKWFsZUs7BM4zTox+dIOenA9QUF3B5iqGtM?= =?us-ascii?Q?cJEMPPz+a1PehWfApVlb6CvrzuRCnkwTuL93ql6B/9/rDo+AEOLoX68O+5Cz?= =?us-ascii?Q?J2dOBZGJyvI8IGecbrEvgTodhaFfp/gMigaYZ6qha9EEqaou2SBLXYWpfaAV?= =?us-ascii?Q?kiCAys18ZmCpbIf8izdjJTsRTEcNqoJoG8Vwbysg/0d4L/wXghPL912GL9e7?= =?us-ascii?Q?V6+f7mSMSSZcBb1KX67cFbbCxHmw31hJ1t9DsqtIXo09ctWrYo4Qieew/Ja8?= =?us-ascii?Q?rrHCRPqTD/aqQ0hjwEIGWBcItquGeshCFX7wJHoJ52v4TZjUvauDAmLqT/nS?= =?us-ascii?Q?pesgZvXvVTZ4xgjonZak7BqF3fWGtC9jgU12/JdyD57Azo2a6xc7EoCzxnBs?= =?us-ascii?Q?f8/JUgvG+x1q8mC9LUL2RqNeoyYnzX3Oa7ycTz8bb9J73iBd0+WKOJaOpzIN?= =?us-ascii?Q?N0+cJ+w3v7YSJR1Uc5OeMGiBJ3eK96/TjUL7fSPUW6H0jYe4TojULHlyADs9?= =?us-ascii?Q?i6+gik9y0RQftw8v1al9vgzd0Xn42NbjrutqDqcDVAzS+c18wK+8l9gzT97E?= =?us-ascii?Q?E7xpuNAcYueDIAOiag7mSUiPeT5Fh9PWQgjZuVZY?= X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(36860700013)(82310400026)(376014)(7416014)(1800799024)(921020);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Dec 2025 16:52:24.7805 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 62efd164-6433-407f-3f3b-08de37435441 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: SJ1PEPF00002312.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB6306 Content-Type: text/plain; charset="utf-8" From: Zhi Wang The read/write callbacks in vfio_device_ops is for accessing the device when mmap is not support. It is also used for VFIO variant driver to emulate the device registers. CXL spec illusrates the standard programming interface, part of them are MMIO registers sit in a PCI BAR. Some of them are emulated when passing the CXL type-2 device to the VM. E.g. HDM decoder registers are emulated. Introduce vfio_cxl_core_{read, write}() in the vfio-cxl-core to prepare for emulating the CXL MMIO registers in the PCI BAR. Signed-off-by: Zhi Wang Signed-off-by: Manish Honap --- drivers/vfio/pci/vfio_cxl_core.c | 20 ++++++++++++++++++++ drivers/vfio/pci/vfio_pci_core.c | 5 +++-- include/linux/vfio_pci_core.h | 6 ++++++ 3 files changed, 29 insertions(+), 2 deletions(-) diff --git a/drivers/vfio/pci/vfio_cxl_core.c b/drivers/vfio/pci/vfio_cxl_c= ore.c index 099d35866a39..460f1ee910af 100644 --- a/drivers/vfio/pci/vfio_cxl_core.c +++ b/drivers/vfio/pci/vfio_cxl_core.c @@ -378,6 +378,26 @@ void vfio_cxl_core_unregister_cxl_region(struct vfio_c= xl_core_device *cxl) } EXPORT_SYMBOL_GPL(vfio_cxl_core_unregister_cxl_region); =20 +ssize_t vfio_cxl_core_read(struct vfio_device *core_vdev, char __user *buf, + size_t count, loff_t *ppos) +{ + struct vfio_pci_core_device *vdev =3D + container_of(core_vdev, struct vfio_pci_core_device, vdev); + + return vfio_pci_rw(vdev, buf, count, ppos, false); +} +EXPORT_SYMBOL_GPL(vfio_cxl_core_read); + +ssize_t vfio_cxl_core_write(struct vfio_device *core_vdev, const char __us= er *buf, + size_t count, loff_t *ppos) +{ + struct vfio_pci_core_device *vdev =3D + container_of(core_vdev, struct vfio_pci_core_device, vdev); + + return vfio_pci_rw(vdev, (char __user *)buf, count, ppos, true); +} +EXPORT_SYMBOL_GPL(vfio_cxl_core_write); + MODULE_LICENSE("GPL"); MODULE_AUTHOR(DRIVER_AUTHOR); MODULE_DESCRIPTION(DRIVER_DESC); diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_c= ore.c index c0695b5db66d..502880e927fc 100644 --- a/drivers/vfio/pci/vfio_pci_core.c +++ b/drivers/vfio/pci/vfio_pci_core.c @@ -1520,8 +1520,8 @@ int vfio_pci_core_ioctl_feature(struct vfio_device *d= evice, u32 flags, } EXPORT_SYMBOL_GPL(vfio_pci_core_ioctl_feature); =20 -static ssize_t vfio_pci_rw(struct vfio_pci_core_device *vdev, char __user = *buf, - size_t count, loff_t *ppos, bool iswrite) +ssize_t vfio_pci_rw(struct vfio_pci_core_device *vdev, char __user *buf, + size_t count, loff_t *ppos, bool iswrite) { unsigned int index =3D VFIO_PCI_OFFSET_TO_INDEX(*ppos); int ret; @@ -1566,6 +1566,7 @@ static ssize_t vfio_pci_rw(struct vfio_pci_core_devic= e *vdev, char __user *buf, pm_runtime_put(&vdev->pdev->dev); return ret; } +EXPORT_SYMBOL_GPL(vfio_pci_rw); =20 ssize_t vfio_pci_core_read(struct vfio_device *core_vdev, char __user *buf, size_t count, loff_t *ppos) diff --git a/include/linux/vfio_pci_core.h b/include/linux/vfio_pci_core.h index 7237fcaecbb6..a6885b48f26f 100644 --- a/include/linux/vfio_pci_core.h +++ b/include/linux/vfio_pci_core.h @@ -153,6 +153,8 @@ long vfio_pci_core_ioctl(struct vfio_device *core_vdev,= unsigned int cmd, unsigned long arg); int vfio_pci_core_ioctl_feature(struct vfio_device *device, u32 flags, void __user *arg, size_t argsz); +ssize_t vfio_pci_rw(struct vfio_pci_core_device *vdev, char __user *buf, + size_t count, loff_t *ppos, bool iswrite); ssize_t vfio_pci_core_read(struct vfio_device *core_vdev, char __user *buf, size_t count, loff_t *ppos); ssize_t vfio_pci_core_write(struct vfio_device *core_vdev, const char __us= er *buf, @@ -216,5 +218,9 @@ int vfio_cxl_core_create_cxl_region(struct vfio_cxl_cor= e_device *cxl, u64 size); void vfio_cxl_core_destroy_cxl_region(struct vfio_cxl_core_device *cxl); int vfio_cxl_core_register_cxl_region(struct vfio_cxl_core_device *cxl); void vfio_cxl_core_unregister_cxl_region(struct vfio_cxl_core_device *cxl); +ssize_t vfio_cxl_core_read(struct vfio_device *core_vdev, char __user *buf, + size_t count, loff_t *ppos); +ssize_t vfio_cxl_core_write(struct vfio_device *core_vdev, const char __us= er *buf, + size_t count, loff_t *ppos); =20 #endif /* VFIO_PCI_CORE_H */ --=20 2.25.1 From nobody Sat Feb 7 12:25:21 2026 Received: from CO1PR03CU002.outbound.protection.outlook.com (mail-westus2azon11010029.outbound.protection.outlook.com [52.101.46.29]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E8ABA32BF44; Tue, 9 Dec 2025 16:52:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.46.29 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1765299164; cv=fail; b=WI5iotdH/J+mKVGGM54BgmSZ6JcG1ZutTrpgFsxriCUNuFrxzNQONERL4NWJaf5o9WvbWq+vpAWJQT4DkHjS9ww6yYqBNyQ9DUO9XjyFEP4ZkVzhBLYA/yao4Dl5/tS8FO5xkozYsVaKj3HLdFWigjCPnp0YTnWhwmar6trQqdE= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1765299164; c=relaxed/simple; bh=0g2KKSMYLp/0w+cM3CWahM9nY9LmLSKwljkV0K4IJqM=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=gaCTZDKTFr8iAVrOj4Vn4Jt+c8KbtvQ4pH6+oIPqEq400azVpR2WuJ4T1zxtsCvKYwmtg050Hl1A7m3CsMPWwmVDHaS6GkZMdBfwPUXnsC+RxFfmNDrqWOv1W4KRagDzK5xjP1Ox8l2RFDNVS8GN3Xzf887G9TvbsSu6HFv0ryM= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=Gy629Slz; arc=fail smtp.client-ip=52.101.46.29 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="Gy629Slz" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=o8IjZKYSpE63VkZDBv71io6E/mF1yl//+LHmktZDlwMH0e3w/XVxEybO4n/nN+pspETmkvk9JtechXEfDkkIWWlJWASrVY1yBEJ2d0XjvjXHhMam1Tideeze05khjchFhz7afxtjXjQJ7ESuxCIPxY80pbDeDbZZRn8iQxrnVusvm590a2Yu5mKu5ZKBRg+Zf8WMQPtJpbfQC5UWg5TrpMu8WLfLmX/cMJXcsofVqoqLOU1iHq5PeSW8QocrMTN06BZ79WCH3PsBgfUZvWj7CJpOvCokaidNJsRmw+CR7Y5qa5+voxe9ewnbkVPH8kqpoGhmIuSYkz+Oc2kPEb97iw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=lls0t5ubKBx4HUwjc8FgLmA/tTMMnUI74JJVYfZ7nB4=; b=AWApEB07EdKg1o/PRtSc5tdddoLNdCU2Cq479F9AJdHE8xbKiusfCkD8DqWapeNJ6C3sce3yRI2usPAYvzEf0s8ZaaZPysoU54gelVAXRnJrO++asY3SKfUmLjG5P66iGwoQCCd9P+cd6nA4bQYjide1/sxMy7PzGr/CC1u+hpbDPqjs9qccpeS5B7eKP2ldposdjN0OUKYyPpUWFQUhnPoNky5TVgokPoFAifksy3jm0+i4qM+gu9p0NuX0dGJqV2d62KdiHXBQooYHD5S2yFbZmvTJmAp00Q3VB9tSNwKpZJ4lnMYJwSDEeLjZRKtDYPmbauphfkeeokxuHkQE/A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=lls0t5ubKBx4HUwjc8FgLmA/tTMMnUI74JJVYfZ7nB4=; b=Gy629SlzlyJwfuEb9RzfcaTiMxIO6trKW8wg3F21dqLh2C7Z0ruPzWNis2y+m/qFrOKUtbhsq7FeGzKW6xa13LunE9Iw+xkb58vkX6XI3msszf27KxKHJ9b8FeYDJ5o4Sxy9bjaBHnIkkgPo2bb6dPLk4TwBRErMeEpzIkZaeY9uVVa4bx4LNh/UkJGFGO9f/sbCk+9y8rfRHkuFmDX9hv4kCZ2eeYbW/5SpiolWAKnayo/w8FWs7AK1boCa46u9VHyY8Unx7/egHej3H7ACVRxrSmYfXhOMA8XvVtGcsI0aXmTMlZTRz6I96SHeRnX/b+JIqDg93kCiVUjhwbQbTQ== Received: from SJ0PR13CA0181.namprd13.prod.outlook.com (2603:10b6:a03:2c3::6) by DS2PR12MB9662.namprd12.prod.outlook.com (2603:10b6:8:27d::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9412.7; Tue, 9 Dec 2025 16:52:32 +0000 Received: from SJ1PEPF00002317.namprd03.prod.outlook.com (2603:10b6:a03:2c3:cafe::a8) by SJ0PR13CA0181.outlook.office365.com (2603:10b6:a03:2c3::6) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9412.7 via Frontend Transport; Tue, 9 Dec 2025 16:52:38 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by SJ1PEPF00002317.mail.protection.outlook.com (10.167.242.171) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9388.8 via Frontend Transport; Tue, 9 Dec 2025 16:52:32 +0000 Received: from rnnvmail205.nvidia.com (10.129.68.10) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 9 Dec 2025 08:52:05 -0800 Received: from rnnvmail204.nvidia.com (10.129.68.6) by rnnvmail205.nvidia.com (10.129.68.10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 9 Dec 2025 08:52:04 -0800 Received: from nvidia-4028GR-scsim.nvidia.com (10.127.8.11) by mail.nvidia.com (10.129.68.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20 via Frontend Transport; Tue, 9 Dec 2025 08:51:57 -0800 From: To: , , , , , , , , , , , , , , , , , CC: , , , , , , , , Subject: [RFC v2 10/15] vfio/cxl: introduce the register emulation framework Date: Tue, 9 Dec 2025 22:20:14 +0530 Message-ID: <20251209165019.2643142-11-mhonap@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20251209165019.2643142-1-mhonap@nvidia.com> References: <20251209165019.2643142-1-mhonap@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ1PEPF00002317:EE_|DS2PR12MB9662:EE_ X-MS-Office365-Filtering-Correlation-Id: 2927f263-d89f-4954-7616-08de374358be X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|1800799024|36860700013|376014|7416014|921020; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?uZ4rI/IFAWUFkyb4rYgyxmD4I3D4lecOE7H609iSBZsu4asGlOIUjIS8mpfu?= =?us-ascii?Q?YNGc8MblVjsQAMpLNtTC5U5LhRxOpqAe7OmqC4YHLo2xMYy2z8xRyzx0at9D?= =?us-ascii?Q?nR1VINPnoZRPQGizkf7ZScFTSRxMt57IovJTMVfWcoALcTf/XJCnEfC5ORLX?= =?us-ascii?Q?0pUhx7nBnRhWnAZxWC2XJIG21Z+gZywrL/laWEuJTZtM6sZw/gJYHjLR5SHh?= =?us-ascii?Q?cU0VJLenXddielD1ovKVpO1JchiegjItjIfdvL1E8YlKL3USiAvPwsnEwDah?= =?us-ascii?Q?vW5OH8Xj7qKkjKQCCfmJjy2TQJ+6+ewtgTrXDkI0hiTgQXnPAw/ynJIWuvC1?= =?us-ascii?Q?XoSZon0d8MZiR3JKZb+TwgqlEmMm+eie21kBMHRC6u/BqJRemcAYXvHGFmUd?= =?us-ascii?Q?FS4fVFSOrwfxzi9Z8jN0a9baTKYZM1e2Sw6XwerfFGJI+HlwlDVdM8Iz1ZGW?= =?us-ascii?Q?iQ+P8aPlqr8BiIs46+rh8+FqZp7glZwKHNzszHSbjIfF5VrbOAH0fEbtzTXX?= =?us-ascii?Q?ruV4Xc6lacrzDN3sNqARo1J6ZEHyDPAtohnPP+KLMLfpzuXPAE2sSwQuEOAR?= =?us-ascii?Q?op0VHR3Emv520WCXLHiaj0mP9NHee3Gm1q83WMT8JgWJxYP+3dD4EwfVhyZN?= =?us-ascii?Q?3QGrKUDEDMMPOQGUFk674b1U1IHtCuFXclkNfXdLDL9/ftUC6WKnM7yMO1aq?= =?us-ascii?Q?TfccRIRSISRlVEu67kel0lxwIWNVuz9ewwx5/9Z1/qQx51hUb5qnBxuMz60T?= =?us-ascii?Q?6vJtYAboYnzBSYNoZVmyx3QaEZ89f2/ajhWjTqoEvhsmzFfwHbprNlRha4Ds?= =?us-ascii?Q?lxu3/eHJT2j+mb7AWJvZkPdL2mBuQC+t3o2GXmM57jKRtS3JsrjbkUf2EL3S?= =?us-ascii?Q?4369newFNK7ZQDaSnq1omE/AClxKRBa7RS0Czs2My6O7ADrIBBncUnSFx2I8?= =?us-ascii?Q?4CeVLFI5SHQptWSeewK7P0tnqRY/0yZy0ps8VcbIpLiO/dks+xj+1MKOsX9j?= =?us-ascii?Q?HFfhY+anrqKJkz3movX7MXm+BP3dbaxZB67yNlk7TFW3j9ubgXv49HeZSwcd?= =?us-ascii?Q?1ge4lOI/MMd65pvT9aVBridxyqUy6mFdJ/B+I4Xr+zJcy6iIB9zf9iCyPmlC?= =?us-ascii?Q?rpcQTKfCLojr5Y1WyUwKbdbc4yr9yUlsN4MGfielxb0GVnlRgxACvVErjU8c?= =?us-ascii?Q?71Jp1A3ggJKLiuQctFLXEVhSyI7Sy6k9ysko3LJE4WUh985X6WqP1pR29jG6?= =?us-ascii?Q?Z8fYbmvxI63WDzGvwUaQ7XUm7sz/ZPNRqEIAYJ3b96PlPbhehPSn46PLdyea?= =?us-ascii?Q?g2SI1kIVFyY8sYC2ZEMG6/mlrfyrnqs/HR5qrg0ednjFq0Zv6nSJ5RWgvpgv?= =?us-ascii?Q?CIo6JKVJA7isDeuIKAHfP/0BsRORPiAJyJU0rMIFFz7XQaQHrmAj5HZOnQ3s?= =?us-ascii?Q?vDIF5USnBM3d8kkQeTgAAjIkDmofe5dM8hX/ADNVREoZLBqUxsTIpSuhYkVQ?= =?us-ascii?Q?ZbWvp0hvhBujJRMHg9tg6CLJlM89aku8euNVn3egXylKLmtIu2Pn4j3f9Fvb?= =?us-ascii?Q?uEXmjwIq9uMkvCzfmvl7pJX7nD0T0Pj7uoqwg+p7?= X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(82310400026)(1800799024)(36860700013)(376014)(7416014)(921020);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Dec 2025 16:52:32.3135 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 2927f263-d89f-4954-7616-08de374358be X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: SJ1PEPF00002317.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS2PR12MB9662 Content-Type: text/plain; charset="utf-8" From: Manish Honap CXL devices have the CXL DVSEC registers in its configuration space and CXL MMIO registers in its CXL MMIO BAR. When passing a CXL type-2 device to a VM, the CXL DVSEC registers in the configuration space and the HDM registers in the CXL MMIO BAR should be emulated. Introduce an emulation framework to handle the CXL device configuration emulation and the CXL MMIO registers emulation. Co-developed-by: Zhi Wang Signed-off-by: Zhi Wang Signed-off-by: Manish Honap --- drivers/vfio/pci/Makefile | 2 +- drivers/vfio/pci/vfio_cxl_core.c | 281 +++++++++++++++++++++++++- drivers/vfio/pci/vfio_cxl_core_emu.c | 184 +++++++++++++++++ drivers/vfio/pci/vfio_cxl_core_priv.h | 17 ++ include/linux/vfio_pci_core.h | 29 +++ 5 files changed, 509 insertions(+), 4 deletions(-) create mode 100644 drivers/vfio/pci/vfio_cxl_core_emu.c create mode 100644 drivers/vfio/pci/vfio_cxl_core_priv.h diff --git a/drivers/vfio/pci/Makefile b/drivers/vfio/pci/Makefile index b51221b94b0b..452b7387f9fb 100644 --- a/drivers/vfio/pci/Makefile +++ b/drivers/vfio/pci/Makefile @@ -8,7 +8,7 @@ vfio-pci-y :=3D vfio_pci.o vfio-pci-$(CONFIG_VFIO_PCI_IGD) +=3D vfio_pci_igd.o obj-$(CONFIG_VFIO_PCI) +=3D vfio-pci.o =20 -vfio-cxl-core-y :=3D vfio_cxl_core.o +vfio-cxl-core-y :=3D vfio_cxl_core.o vfio_cxl_core_emu.o obj-$(CONFIG_VFIO_CXL_CORE) +=3D vfio-cxl-core.o =20 obj-$(CONFIG_MLX5_VFIO_PCI) +=3D mlx5/ diff --git a/drivers/vfio/pci/vfio_cxl_core.c b/drivers/vfio/pci/vfio_cxl_c= ore.c index 460f1ee910af..cb75e9f668a7 100644 --- a/drivers/vfio/pci/vfio_cxl_core.c +++ b/drivers/vfio/pci/vfio_cxl_core.c @@ -20,6 +20,7 @@ #include =20 #include "vfio_pci_priv.h" +#include "vfio_cxl_core_priv.h" =20 #define DRIVER_AUTHOR "Zhi Wang " #define DRIVER_DESC "core driver for VFIO based CXL devices" @@ -119,6 +120,119 @@ static void discover_precommitted_region(struct vfio_= cxl_core_device *cxl) cxl_core->region.precommitted =3D true; } =20 +static int find_bar(struct pci_dev *pdev, u64 *offset, int *bar, u64 size) +{ + u64 start, end, flags; + int index, i; + + for (i =3D 0; i < PCI_STD_NUM_BARS; i++) { + index =3D i + PCI_STD_RESOURCES; + flags =3D pci_resource_flags(pdev, index); + + start =3D pci_resource_start(pdev, index); + end =3D pci_resource_end(pdev, index); + + if (*offset >=3D start && *offset + size - 1 <=3D end) + break; + + if (flags & IORESOURCE_MEM_64) + i++; + } + + if (i =3D=3D PCI_STD_NUM_BARS) + return -ENODEV; + + *offset =3D *offset - start; + *bar =3D index; + + return 0; +} + +static int find_comp_regs(struct vfio_cxl_core_device *cxl) +{ + struct vfio_pci_core_device *pci =3D &cxl->pci_core; + struct pci_dev *pdev =3D pci->pdev; + u64 offset; + int ret, bar; + + ret =3D cxl_find_comp_regblock_offset(pdev, &offset); + if (ret) + return ret; + + ret =3D find_bar(pdev, &offset, &bar, SZ_64K); + if (ret) + return ret; + + cxl->comp_reg_bar =3D bar; + cxl->comp_reg_offset =3D offset; + cxl->comp_reg_size =3D SZ_64K; + return 0; +} + +static void clean_virt_regs(struct vfio_cxl_core_device *cxl) +{ + kvfree(cxl->comp_reg_virt); + kvfree(cxl->config_virt); +} + +static void reset_virt_regs(struct vfio_cxl_core_device *cxl) +{ + memcpy(cxl->config_virt, cxl->initial_config_virt, cxl->config_size); + memcpy(cxl->comp_reg_virt, cxl->initial_comp_reg_virt, cxl->comp_reg_size= ); +} + +static int setup_virt_regs(struct vfio_cxl_core_device *cxl) +{ + struct vfio_pci_core_device *pci =3D &cxl->pci_core; + struct pci_dev *pdev =3D pci->pdev; + u64 offset =3D cxl->comp_reg_offset; + int bar =3D cxl->comp_reg_bar; + u64 size =3D cxl->comp_reg_size; + void *regs; + unsigned int i; + + regs =3D kvzalloc(size * 2, GFP_KERNEL); + if (!regs) + return -ENOMEM; + + cxl->comp_reg_virt =3D regs; + cxl->initial_comp_reg_virt =3D regs + size; + + regs =3D ioremap(pci_resource_start(pdev, bar) + offset, size); + if (!regs) { + kvfree(cxl->comp_reg_virt); + return -EFAULT; + } + + for (i =3D 0; i < size; i +=3D 4) + *(u32 *)(cxl->initial_comp_reg_virt + i) =3D + cpu_to_le32(readl(regs + i)); + + iounmap(regs); + + regs =3D kvzalloc(pdev->cfg_size * 2, GFP_KERNEL); + if (!regs) { + kvfree(cxl->comp_reg_virt); + return -ENOMEM; + } + + cxl->config_virt =3D regs; + cxl->initial_config_virt =3D regs + pdev->cfg_size; + cxl->config_size =3D pdev->cfg_size; + + regs =3D cxl->initial_config_virt + cxl->dvsec; + + for (i =3D 0; i < 0x40; i +=3D 4) { + u32 val; + + pci_read_config_dword(pdev, cxl->dvsec + i, &val); + *(u32 *)(regs + i) =3D cpu_to_le32(val); + } + + reset_virt_regs(cxl); + return 0; +} + int vfio_cxl_core_enable(struct vfio_cxl_core_device *cxl, struct vfio_cxl_dev_info *info) { @@ -133,6 +247,8 @@ int vfio_cxl_core_enable(struct vfio_cxl_core_device *c= xl, if (!dvsec) return -ENODEV; =20 + cxl->dvsec =3D dvsec; + cxl_core =3D devm_cxl_dev_state_create(&pdev->dev, CXL_DEVTYPE_DEVMEM, pdev->dev.id, dvsec, struct vfio_cxl, cxlds, false); @@ -141,20 +257,37 @@ int vfio_cxl_core_enable(struct vfio_cxl_core_device = *cxl, return -ENOMEM; } =20 - ret =3D vfio_pci_core_enable(pci); + ret =3D find_comp_regs(cxl); + if (ret) + return -ENODEV; + + ret =3D setup_virt_regs(cxl); if (ret) return ret; =20 + ret =3D vfio_pci_core_enable(pci); + if (ret) + goto err_pci_core_enable; + ret =3D enable_cxl(cxl, dvsec, info); if (ret) - goto err; + goto err_enable_cxl; + + ret =3D vfio_cxl_core_setup_register_emulation(cxl); + if (ret) + goto err_register_emulation; =20 discover_precommitted_region(cxl); =20 return 0; =20 -err: +err_register_emulation: + disable_cxl(cxl); +err_pci_core_enable: + clean_virt_regs(cxl); +err_enable_cxl: vfio_pci_core_disable(pci); + return ret; } EXPORT_SYMBOL_GPL(vfio_cxl_core_enable); @@ -169,7 +302,9 @@ EXPORT_SYMBOL_GPL(vfio_cxl_core_finish_enable); =20 static void disable_device(struct vfio_cxl_core_device *cxl) { + vfio_cxl_core_clean_register_emulation(cxl); disable_cxl(cxl); + clean_virt_regs(cxl); } =20 void vfio_cxl_core_disable(struct vfio_cxl_core_device *cxl) @@ -383,6 +518,20 @@ ssize_t vfio_cxl_core_read(struct vfio_device *core_vd= ev, char __user *buf, { struct vfio_pci_core_device *vdev =3D container_of(core_vdev, struct vfio_pci_core_device, vdev); + struct vfio_cxl_core_device *cxl =3D + container_of(vdev, struct vfio_cxl_core_device, pci_core); + unsigned int index =3D VFIO_PCI_OFFSET_TO_INDEX(*ppos); + + if (!count) + return 0; + + if (index =3D=3D VFIO_PCI_CONFIG_REGION_INDEX) + return vfio_cxl_core_config_rw(core_vdev, buf, count, ppos, + false); + + if (index =3D=3D cxl->comp_reg_bar) + return vfio_cxl_core_mmio_bar_rw(core_vdev, buf, count, ppos, + false); =20 return vfio_pci_rw(vdev, buf, count, ppos, false); } @@ -393,11 +542,137 @@ ssize_t vfio_cxl_core_write(struct vfio_device *core= _vdev, const char __user *bu { struct vfio_pci_core_device *vdev =3D container_of(core_vdev, struct vfio_pci_core_device, vdev); + struct vfio_cxl_core_device *cxl =3D + container_of(vdev, struct vfio_cxl_core_device, pci_core); + unsigned int index =3D VFIO_PCI_OFFSET_TO_INDEX(*ppos); + + if (!count) + return 0; + + if (index =3D=3D VFIO_PCI_CONFIG_REGION_INDEX) + return vfio_cxl_core_config_rw(core_vdev, (char __user *)buf, + count, ppos, true); + + if (index =3D=3D cxl->comp_reg_bar) + return vfio_cxl_core_mmio_bar_rw(core_vdev, (char __user *)buf, + count, ppos, true); =20 return vfio_pci_rw(vdev, (char __user *)buf, count, ppos, true); } EXPORT_SYMBOL_GPL(vfio_cxl_core_write); =20 +static int comp_reg_bar_get_region_info(struct vfio_pci_core_device *pci, + void __user *uarg) +{ + struct vfio_cxl_core_device *cxl =3D + container_of(pci, struct vfio_cxl_core_device, pci_core); + struct pci_dev *pdev =3D pci->pdev; + unsigned long minsz =3D offsetofend(struct vfio_region_info, offset); + struct vfio_info_cap caps =3D { .buf =3D NULL, .size =3D 0 }; + struct vfio_region_info_cap_sparse_mmap *sparse; + struct vfio_region_info info; + u64 start, end, len; + u32 size; + int ret; + + if (copy_from_user(&info, uarg, minsz)) + return -EFAULT; + + if (info.argsz < minsz) + return -EINVAL; + + start =3D pci_resource_start(pdev, cxl->comp_reg_bar); + end =3D pci_resource_end(pdev, cxl->comp_reg_bar) - start; + len =3D pci_resource_len(pdev, cxl->comp_reg_bar); + + if (!cxl->comp_reg_offset || + cxl->comp_reg_offset + cxl->comp_reg_size =3D=3D end) { + size =3D struct_size(sparse, areas, 1); + + sparse =3D kzalloc(size, GFP_KERNEL); + if (!sparse) + return -ENOMEM; + + sparse->nr_areas =3D 1; + sparse->areas[0].offset =3D cxl->comp_reg_offset ? 0 : cxl->comp_reg_siz= e; + sparse->areas[0].size =3D len - cxl->comp_reg_size; + } else { + size =3D struct_size(sparse, areas, 2); + + sparse =3D kzalloc(size, GFP_KERNEL); + if (!sparse) + return -ENOMEM; + + sparse->nr_areas =3D 2; + + sparse->areas[0].offset =3D 0; + sparse->areas[0].size =3D cxl->comp_reg_offset; + + sparse->areas[1].offset =3D sparse->areas[0].size + cxl->comp_reg_size; + sparse->areas[1].size =3D len - sparse->areas[0].size - cxl->comp_reg_si= ze; + } + + sparse->header.id =3D VFIO_REGION_INFO_CAP_SPARSE_MMAP; + sparse->header.version =3D 1; + + ret =3D vfio_info_add_capability(&caps, &sparse->header, size); + kfree(sparse); + if (ret) + return ret; + + info.offset =3D VFIO_PCI_INDEX_TO_OFFSET(info.index); + info.size =3D len; + info.flags =3D VFIO_REGION_INFO_FLAG_READ | + VFIO_REGION_INFO_FLAG_WRITE | + VFIO_REGION_INFO_FLAG_MMAP; + + if (caps.size) { + info.flags |=3D VFIO_REGION_INFO_FLAG_CAPS; + if (info.argsz < sizeof(info) + caps.size) { + info.argsz =3D sizeof(info) + caps.size; + info.cap_offset =3D 0; + } else { + vfio_info_cap_shift(&caps, sizeof(info)); + if (copy_to_user(uarg + sizeof(info), caps.buf, + caps.size)) { + kfree(caps.buf); + return -EFAULT; + } + info.cap_offset =3D sizeof(info); + } + kfree(caps.buf); + } + + return copy_to_user(uarg, &info, minsz) ? -EFAULT : 0; +} + +long vfio_cxl_core_ioctl(struct vfio_device *core_vdev, unsigned int cmd, + unsigned long arg) +{ + struct vfio_pci_core_device *pci =3D + container_of(core_vdev, struct vfio_pci_core_device, vdev); + struct vfio_cxl_core_device *cxl =3D + container_of(pci, struct vfio_cxl_core_device, pci_core); + void __user *uarg =3D (void __user *)arg; + + if (cmd =3D=3D VFIO_DEVICE_GET_REGION_INFO) { + struct vfio_region_info info; + unsigned long minsz =3D offsetofend(struct vfio_region_info, offset); + + if (copy_from_user(&info, (void *)arg, minsz)) + return -EFAULT; + + if (info.argsz < minsz) + return -EINVAL; + + if (info.index =3D=3D cxl->comp_reg_bar) + return comp_reg_bar_get_region_info(pci, uarg); + } + + return vfio_pci_core_ioctl(core_vdev, cmd, arg); +} +EXPORT_SYMBOL_GPL(vfio_cxl_core_ioctl); + MODULE_LICENSE("GPL"); MODULE_AUTHOR(DRIVER_AUTHOR); MODULE_DESCRIPTION(DRIVER_DESC); diff --git a/drivers/vfio/pci/vfio_cxl_core_emu.c b/drivers/vfio/pci/vfio_c= xl_core_emu.c new file mode 100644 index 000000000000..a0674bacecd7 --- /dev/null +++ b/drivers/vfio/pci/vfio_cxl_core_emu.c @@ -0,0 +1,184 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2025, NVIDIA CORPORATION & AFFILIATES. All rights reserved + */ + +#include "vfio_cxl_core_priv.h" + +void vfio_cxl_core_clean_register_emulation(struct vfio_cxl_core_device *c= xl) +{ + struct list_head *pos, *n; + + list_for_each_safe(pos, n, &cxl->config_regblocks_head) + kfree(list_entry(pos, struct vfio_emulated_regblock, list)); + list_for_each_safe(pos, n, &cxl->mmio_regblocks_head) + kfree(list_entry(pos, struct vfio_emulated_regblock, list)); +} + +int vfio_cxl_core_setup_register_emulation(struct vfio_cxl_core_device *cx= l) +{ + INIT_LIST_HEAD(&cxl->config_regblocks_head); + INIT_LIST_HEAD(&cxl->mmio_regblocks_head); + + return 0; +} + +static struct vfio_emulated_regblock * +find_regblock(struct list_head *head, u64 offset, u64 size) +{ + struct vfio_emulated_regblock *block; + struct list_head *pos; + + list_for_each(pos, head) { + block =3D list_entry(pos, struct vfio_emulated_regblock, list); + + if (block->range.start =3D=3D ALIGN_DOWN(offset, + range_len(&block->range))) + return block; + } + return NULL; +} + +static ssize_t emulate_read(struct list_head *head, struct vfio_device *vd= ev, + char __user *buf, size_t count, loff_t *ppos) +{ + struct vfio_pci_core_device *pci =3D + container_of(vdev, struct vfio_pci_core_device, vdev); + struct vfio_cxl_core_device *cxl =3D + container_of(pci, struct vfio_cxl_core_device, pci_core); + struct vfio_emulated_regblock *block; + u64 pos =3D *ppos & VFIO_PCI_OFFSET_MASK; + ssize_t ret; + u32 v; + + block =3D find_regblock(head, pos, count); + if (!block || !block->read) + return vfio_pci_rw(pci, buf, count, ppos, false); + + if (WARN_ON_ONCE(!IS_ALIGNED(pos, range_len(&block->range)))) + return -EINVAL; + + if (count > range_len(&block->range)) + count =3D range_len(&block->range); + + ret =3D block->read(cxl, &v, pos, count); + if (ret < 0) + return ret; + + if (copy_to_user(buf, &v, count)) + return -EFAULT; + + return count; +} + +static ssize_t emulate_write(struct list_head *head, struct vfio_device *v= dev, + char __user *buf, size_t count, loff_t *ppos) +{ + struct vfio_pci_core_device *pci =3D + container_of(vdev, struct vfio_pci_core_device, vdev); + struct vfio_cxl_core_device *cxl =3D + container_of(pci, struct vfio_cxl_core_device, pci_core); + struct vfio_emulated_regblock *block; + u64 pos =3D *ppos & VFIO_PCI_OFFSET_MASK; + ssize_t ret; + u32 v; + + block =3D find_regblock(head, pos, count); + if (!block || !block->write) + return vfio_pci_rw(pci, buf, count, ppos, true); + + if (WARN_ON_ONCE(!IS_ALIGNED(pos, range_len(&block->range)))) + return -EINVAL; + + if (count > range_len(&block->range)) + count =3D range_len(&block->range); + + if (copy_from_user(&v, buf, count)) + return -EFAULT; + + ret =3D block->write(cxl, &v, pos, count); + if (ret < 0) + return ret; + + return count; +} + +ssize_t vfio_cxl_core_config_rw(struct vfio_device *vdev, char __user *buf, + size_t count, loff_t *ppos, bool write) +{ + struct vfio_pci_core_device *pci =3D + container_of(vdev, struct vfio_pci_core_device, vdev); + struct vfio_cxl_core_device *cxl =3D + container_of(pci, struct vfio_cxl_core_device, pci_core); + size_t done =3D 0; + ssize_t ret =3D 0; + loff_t tmp, pos =3D *ppos; + + while (count) { + tmp =3D pos; + + if (count >=3D 4 && IS_ALIGNED(pos, 4)) + ret =3D 4; + else if (count >=3D 2 && IS_ALIGNED(pos, 2)) + ret =3D 2; + else + ret =3D 1; + + if (write) + ret =3D emulate_write(&cxl->config_regblocks_head, + vdev, buf, ret, &tmp); + else + ret =3D emulate_read(&cxl->config_regblocks_head, + vdev, buf, ret, &tmp); + if (ret < 0) + return ret; + + count -=3D ret; + done +=3D ret; + buf +=3D ret; + pos +=3D ret; + } + + *ppos +=3D done; + return done; +} + +ssize_t vfio_cxl_core_mmio_bar_rw(struct vfio_device *vdev, char __user *b= uf, + size_t count, loff_t *ppos, bool write) +{ + struct vfio_pci_core_device *pci =3D + container_of(vdev, struct vfio_pci_core_device, vdev); + struct vfio_cxl_core_device *cxl =3D + container_of(pci, struct vfio_cxl_core_device, pci_core); + size_t done =3D 0; + ssize_t ret =3D 0; + loff_t tmp, pos =3D *ppos; + + while (count) { + tmp =3D pos; + + if (count >=3D 4 && IS_ALIGNED(pos, 4)) + ret =3D 4; + else if (count >=3D 2 && IS_ALIGNED(pos, 2)) + ret =3D 2; + else + ret =3D 1; + + if (write) + ret =3D emulate_write(&cxl->mmio_regblocks_head, + vdev, buf, ret, &tmp); + else + ret =3D emulate_read(&cxl->mmio_regblocks_head, + vdev, buf, ret, &tmp); + if (ret < 0) + return ret; + + count -=3D ret; + done +=3D ret; + buf +=3D ret; + pos +=3D ret; + } + + *ppos +=3D done; + return done; +} diff --git a/drivers/vfio/pci/vfio_cxl_core_priv.h b/drivers/vfio/pci/vfio_= cxl_core_priv.h new file mode 100644 index 000000000000..b5d96e3872d2 --- /dev/null +++ b/drivers/vfio/pci/vfio_cxl_core_priv.h @@ -0,0 +1,17 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +#ifndef VFIO_CXL_CORE_PRIV_H +#define VFIO_CXL_CORE_PRIV_H + +#include + +#include "vfio_pci_priv.h" + +void vfio_cxl_core_clean_register_emulation(struct vfio_cxl_core_device *c= xl); +int vfio_cxl_core_setup_register_emulation(struct vfio_cxl_core_device *cx= l); + +ssize_t vfio_cxl_core_config_rw(struct vfio_device *vdev, char __user *buf, + size_t count, loff_t *ppos, bool write); +ssize_t vfio_cxl_core_mmio_bar_rw(struct vfio_device *vdev, char __user *b= uf, + size_t count, loff_t *ppos, bool write); + +#endif diff --git a/include/linux/vfio_pci_core.h b/include/linux/vfio_pci_core.h index a6885b48f26f..12ded67c7db7 100644 --- a/include/linux/vfio_pci_core.h +++ b/include/linux/vfio_pci_core.h @@ -117,13 +117,40 @@ struct vfio_cxl { struct vfio_cxl_region region; }; =20 +struct vfio_cxl_core_device; + +struct vfio_emulated_regblock { + struct range range; + ssize_t (*read)(struct vfio_cxl_core_device *cxl, void *buf, + u64 offset, u64 size); + ssize_t (*write)(struct vfio_cxl_core_device *cxl, void *buf, + u64 offset, u64 size); + struct list_head list; +}; + struct vfio_cxl_core_device { struct vfio_pci_core_device pci_core; struct vfio_cxl *cxl_core; =20 + struct list_head config_regblocks_head; + struct list_head mmio_regblocks_head; + + void *initial_comp_reg_virt; + void *comp_reg_virt; + u64 comp_reg_size; + + void *initial_config_virt; + void *config_virt; + u64 config_size; + + u16 dvsec; + u32 hdm_count; u64 hdm_reg_offset; u64 hdm_reg_size; + + int comp_reg_bar; + u64 comp_reg_offset; }; =20 struct vfio_cxl_dev_info { @@ -222,5 +249,7 @@ ssize_t vfio_cxl_core_read(struct vfio_device *core_vde= v, char __user *buf, size_t count, loff_t *ppos); ssize_t vfio_cxl_core_write(struct vfio_device *core_vdev, const char __us= er *buf, size_t count, loff_t *ppos); +long vfio_cxl_core_ioctl(struct vfio_device *core_vdev, unsigned int cmd, + unsigned long arg); =20 #endif /* VFIO_PCI_CORE_H */ --=20 2.25.1 From nobody Sat Feb 7 12:25:21 2026 Received: from BYAPR05CU005.outbound.protection.outlook.com (mail-westusazon11010004.outbound.protection.outlook.com [52.101.85.4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A723532E6BA; Tue, 9 Dec 2025 16:52:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.85.4 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1765299169; cv=fail; b=HLkVaKAz35RE5niOMVKUkwNoyv0hgHko5RyGldX9VhpALGUVhqaQS0TXGwSGcnMSQz8Fc7QFvPPe9HotjN36K6USU5Tpj6K0eyzY1yndkGcWxKmJutPZuWT4+zNErc07B55Xse8oeEkv8D29ejBAjZ4o/eBm5Dypw1NydbggOH8= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1765299169; c=relaxed/simple; bh=GZJDxchfh8V+pp1t4IlfH5gr/mdGN3pa2yrvPSNny8I=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=vDhwqS9K51c3XpAVzFhZhAaNIrOAwVrk9n51a2qv1O7RJx3TlkG9vNAKIMpYQCuW5+Fx7pEG2oY8xjcvdfTvhmaCo3l2CT3vO/KyE9z91etVeXuXGrEpayjEo8YJf5JSzafzvtszPi+LdfvJ+0fweiNsJcovGMBU7sDcN6zHLkI= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=lgtHKsYV; arc=fail smtp.client-ip=52.101.85.4 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="lgtHKsYV" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=QJmtTqKU2OgnYjdIJxPrbFmlLeJyCzvFoCJKttY3ThifshegXd/D9IUZJoyspOllbNeOVrok/fXa3AHoWRMum6g1dLJuMRTZC6I51dsSDwRGKGeEcJHdo6U9PRygAyKec9pvAd2ZHCAwsBPaurbnbnvCG8ZsJ8qEpZsJ44wuGLeEyK3PIP8C61fqskN0QB0HRvpSVhbv1e7oumA4tGc2017XFNjmGH2lJ/VLSYAd7tLGwM9P6M3YRkXXIojkQF0AQS8DXsD2dKRD4UftnnAUZE1VfTQuTc5ekDPYrlZS175jYgi518pSkkOACs7OH5Rj6jsdNFh6559Nw7M6P1RaQw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=hxiZUvRANKziqWZ+A3QECR4oaAv5m6hQCCpRwpF3WIo=; b=mpdjQOzQ107Ic8K4H0WmHnBH8ZMxsPSJBCJs1pcl0L0rG/mAP4IpOegAYZm34BCOTuGAsgz2is2B6KjiSoJb0f1fZ9rBxyFFNmpq3nycUf7Ov8S7aQHdebbFeG/YwnZBs1+8J9vjBtDa/cLQXxwD2rpQSv3VDMQ5xPZfpvodTEeInJuaROOVHWk8+xZFhKSfvO5m+rZmURFRN2qet4QRR5QKJ/L48O8y3WH9lWCLHxC03QSO905UVEs8hEd1/k1e9pkA3gphHS9WhSzZNSYx1EVWrbOJ3n5hHtBbpCbfS9afxvOv9G6WPE1yNzwrnMu5uOLphA7/xnbd+yJwWyJOdA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=hxiZUvRANKziqWZ+A3QECR4oaAv5m6hQCCpRwpF3WIo=; b=lgtHKsYVp7MLz+8MIU+PT1yQjvWom4EIuRMXKZNJqYIJqz+io6Bm6oMSna8IE7YcI59+Z+73WK3sWdbWamD5P0zVT7b0o1L9Cfjmv9hK6pYGTQvlCH1XBZ52g00Tfqzmbla1Kw3ciZEopy8enRgPxYFCeCTa43Z7eD+wDLGy8rDXxol/1d8VIZ3LOLzkT4b3wNUAprq1uCT6jQE67aBdrrSInm4KQFd6Jef1c6XCTL2RePLMWn3hEc8WXAvvNrqjToH7UPvZe1vwlQlALU9A1eS05yrJrJbWljIECKXdIqe8zNLlHXY306kPBBIfu2slvaYwm+8cNkWUttQ1S5truQ== Received: from SJ0PR03CA0371.namprd03.prod.outlook.com (2603:10b6:a03:3a1::16) by IA1PR12MB6164.namprd12.prod.outlook.com (2603:10b6:208:3e8::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9388.9; Tue, 9 Dec 2025 16:52:36 +0000 Received: from SJ1PEPF00002315.namprd03.prod.outlook.com (2603:10b6:a03:3a1:cafe::cc) by SJ0PR03CA0371.outlook.office365.com (2603:10b6:a03:3a1::16) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9388.14 via Frontend Transport; Tue, 9 Dec 2025 16:52:03 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by SJ1PEPF00002315.mail.protection.outlook.com (10.167.242.169) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9388.8 via Frontend Transport; Tue, 9 Dec 2025 16:52:36 +0000 Received: from rnnvmail205.nvidia.com (10.129.68.10) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 9 Dec 2025 08:52:13 -0800 Received: from rnnvmail204.nvidia.com (10.129.68.6) by rnnvmail205.nvidia.com (10.129.68.10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 9 Dec 2025 08:52:12 -0800 Received: from nvidia-4028GR-scsim.nvidia.com (10.127.8.11) by mail.nvidia.com (10.129.68.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20 via Frontend Transport; Tue, 9 Dec 2025 08:52:05 -0800 From: To: , , , , , , , , , , , , , , , , , CC: , , , , , , , , Subject: [RFC v2 11/15] vfio/cxl: introduce the emulation of HDM registers Date: Tue, 9 Dec 2025 22:20:15 +0530 Message-ID: <20251209165019.2643142-12-mhonap@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20251209165019.2643142-1-mhonap@nvidia.com> References: <20251209165019.2643142-1-mhonap@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ1PEPF00002315:EE_|IA1PR12MB6164:EE_ X-MS-Office365-Filtering-Correlation-Id: d0af40f3-15bf-4d15-4c0c-08de37435b44 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|36860700013|376014|1800799024|82310400026|921020; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?/Yr8RehwA9DG38P4GHxKKgHSsaOKlJ3fF6NiQmSl/RvRE771xjvkWhhddPQb?= =?us-ascii?Q?5vH0JyafDLHxpqZIdEeH7tydxD1Am0CM/jcGBXCUWMNJ0j9t3hRNiEeX76/6?= =?us-ascii?Q?mBsPDzgimJFgVZ5YgO2aEy7E0B/nRwt6CmLJjfq9V8dEb4Pp+R3DRkFQ/g26?= =?us-ascii?Q?awIrMBmTW9hzNwPvDRr5OJlLfSt+InTKVBmEDS2k9YUrt2AaA1qG9/4anhHn?= =?us-ascii?Q?3N6aVtpi2FqcQJSrkzJ6r0G/ysHgUjmuaAYeK4GXBbMLhydqudCF0KfmZleb?= =?us-ascii?Q?spX9xvn05Tn2G4lTKGaj7ByhPBWO78xZYwX3rBIzcO7k/wM1lTXaQNMNLzU5?= =?us-ascii?Q?ywiScncxgIW+pVL4azxY64Sa7Tx5NpLmWf50PNuCsgAbiKKqU3RzOJ6IW4bA?= =?us-ascii?Q?T8ztr2YwsX5HAqWRm8fWsKNAcyN4wi4FhTqL9MNBX7MtSXxv6HIvrqWCCueh?= =?us-ascii?Q?SVfUoN22BccU3RyPZuZsHvcM+zKLBERxWLfN4wacrox3jnmb2Fesp7TlJOQf?= =?us-ascii?Q?ZB4VE5At8paEPpL9RPZ0qevlGSiDAToM1NkMiMEetde/Z/wF0ywIZSEYI2Cs?= =?us-ascii?Q?/Hc6xoDxPCEjBffDnFTyjE/5NZb1ffASXuCVbw6BoUknKrKHjAnlhwxFHmE7?= =?us-ascii?Q?MVb+WjJAZC37AMMMN4kJgOLc53sS5h3hkVFpb60WtwUBXi6o74Ignr3T8aOB?= =?us-ascii?Q?1M+A0lQgtBoQ0TPSPdl8OvzRfoflt2t6v+/hzHs3RwrDqQ7Gnsszxa5VfPuW?= =?us-ascii?Q?QBzAnQW62+AI7NbimKTt3q9iHrFUVSGTOCeEdbL3AE5IQimMA7/32HiXZ4AJ?= =?us-ascii?Q?KAkwAXA/QM0gJSVqQcjYRA5VFka1lYiaz0BGaVMn6TBw6C+2mK47e5KPeUPj?= =?us-ascii?Q?5LhXkQUZG9HogyPdAy3VkTSNkenJ/1KY00Fse0pSQcGV6pfCAafcGr8E75P9?= =?us-ascii?Q?4RD0v6+OgE058whG2O56AMW1BG1AMFgS3gyFYY/3v3f1DnKt8b1tkCTmMM3x?= =?us-ascii?Q?dKSqRxQb2hy1M1f8znk5fTT7j1xRqX4YcLYx0H6Rl2VbDtI7gZOLeyCt//2I?= =?us-ascii?Q?4R/8ko/5sdBWuUSUbS/SpcWY7ygu3ni3QiTJi7ng68cBPeO8z3ipD/KL044o?= =?us-ascii?Q?/SjCalQOI/W0PLxyHe8UCells0xaW6sCvFiWpJ8Hs7fOkfRdhJOItKCJQgsL?= =?us-ascii?Q?MLvYlE4q0/xHrVPg6Q8zLlsLa1Nzd7eOZ+vqiWfJHT/i9PyABGn49UDHFuo0?= =?us-ascii?Q?RP0FFc8uMifsYk/iW+XzbygAtXO9ZvYixtMcSdBYzLBukWQJ1YT9DP9ky9Py?= =?us-ascii?Q?/uPafMsAJFfKd4L4S716qcnu2XRdpcimCoN3hhPabjJSfY4Lr36Ch25+4u4q?= =?us-ascii?Q?0/LpqZnmwcldo1VNpCqif0jV+QXs2PXeLg7Qit9hZUSZSvqF8AjnUO4BYJU0?= =?us-ascii?Q?vehvXr9Sq2ezOSdPvTIpKzu2aKnlHirg3a+uQguSlQzXylV1g2TrG7IsDGT7?= =?us-ascii?Q?UKjgtR5zjXGELz8DWyymD/pLPz/pwpMeiQqmGNLi28uKwI28wToLyu/IUtQl?= =?us-ascii?Q?az76A7VERVTW8jsVHBdtsvXoPXvbFl3qn4n0WxXI?= X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(7416014)(36860700013)(376014)(1800799024)(82310400026)(921020);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Dec 2025 16:52:36.5455 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: d0af40f3-15bf-4d15-4c0c-08de37435b44 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: SJ1PEPF00002315.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB6164 Content-Type: text/plain; charset="utf-8" From: Manish Honap CXL devices have HDM registers in its CXL MMIO bar. Many HDM registers requires a PA and they are owned by the host in virtualization. Thus, the HDM registers needs to be emulated accordingly so that the guest kernel CXL core can configure the virtual HDM decoders. Intorduce the emulation of HDM registers that emulates the HDM decoders. Co-developed-by: Zhi Wang Signed-off-by: Zhi Wang Signed-off-by: Manish Honap --- drivers/vfio/pci/vfio_cxl_core.c | 7 +- drivers/vfio/pci/vfio_cxl_core_emu.c | 242 +++++++++++++++++++++++++++ include/linux/vfio_pci_core.h | 2 + 3 files changed, 248 insertions(+), 3 deletions(-) diff --git a/drivers/vfio/pci/vfio_cxl_core.c b/drivers/vfio/pci/vfio_cxl_c= ore.c index cb75e9f668a7..c0bdf55997da 100644 --- a/drivers/vfio/pci/vfio_cxl_core.c +++ b/drivers/vfio/pci/vfio_cxl_core.c @@ -247,8 +247,6 @@ int vfio_cxl_core_enable(struct vfio_cxl_core_device *c= xl, if (!dvsec) return -ENODEV; =20 - cxl->dvsec =3D dvsec; - cxl_core =3D devm_cxl_dev_state_create(&pdev->dev, CXL_DEVTYPE_DEVMEM, pdev->dev.id, dvsec, struct vfio_cxl, cxlds, false); @@ -257,9 +255,12 @@ int vfio_cxl_core_enable(struct vfio_cxl_core_device *= cxl, return -ENOMEM; } =20 + cxl->dvsec =3D dvsec; + cxl->cxl_core =3D cxl_core; + ret =3D find_comp_regs(cxl); if (ret) - return -ENODEV; + return ret; =20 ret =3D setup_virt_regs(cxl); if (ret) diff --git a/drivers/vfio/pci/vfio_cxl_core_emu.c b/drivers/vfio/pci/vfio_c= xl_core_emu.c index a0674bacecd7..6711ff8975ef 100644 --- a/drivers/vfio/pci/vfio_cxl_core_emu.c +++ b/drivers/vfio/pci/vfio_cxl_core_emu.c @@ -5,6 +5,239 @@ =20 #include "vfio_cxl_core_priv.h" =20 +typedef ssize_t reg_handler_t(struct vfio_cxl_core_device *cxl, void *buf, + u64 offset, u64 size); + +static struct vfio_emulated_regblock * +new_reg_block(struct vfio_cxl_core_device *cxl, u64 offset, u64 size, + reg_handler_t *read, reg_handler_t *write) +{ + struct vfio_emulated_regblock *block; + + block =3D kzalloc(sizeof(*block), GFP_KERNEL); + if (!block) + return ERR_PTR(-ENOMEM); + + block->range.start =3D offset; + block->range.end =3D offset + size - 1; + block->read =3D read; + block->write =3D write; + + INIT_LIST_HEAD(&block->list); + + return block; +} + +static int new_mmio_block(struct vfio_cxl_core_device *cxl, u64 offset, u6= 4 size, + reg_handler_t *read, reg_handler_t *write) +{ + struct vfio_emulated_regblock *block; + + block =3D new_reg_block(cxl, offset, size, read, write); + if (IS_ERR(block)) + return PTR_ERR(block); + + list_add_tail(&block->list, &cxl->mmio_regblocks_head); + return 0; +} + +static u64 hdm_reg_base(struct vfio_cxl_core_device *cxl) +{ + return cxl->comp_reg_offset + cxl->hdm_reg_offset; +} + +static u64 to_hdm_reg_offset(struct vfio_cxl_core_device *cxl, u64 offset) +{ + return offset - hdm_reg_base(cxl); +} + +static void *hdm_reg_virt(struct vfio_cxl_core_device *cxl, u64 hdm_reg_of= fset) +{ + return cxl->comp_reg_virt + cxl->hdm_reg_offset + hdm_reg_offset; +} + +static ssize_t virt_hdm_reg_read(struct vfio_cxl_core_device *cxl, void *b= uf, + u64 offset, u64 size) +{ + offset =3D to_hdm_reg_offset(cxl, offset); + memcpy(buf, hdm_reg_virt(cxl, offset), size); + + return size; +} + +static ssize_t virt_hdm_reg_write(struct vfio_cxl_core_device *cxl, void *= buf, + u64 offset, u64 size) +{ + offset =3D to_hdm_reg_offset(cxl, offset); + memcpy(hdm_reg_virt(cxl, offset), buf, size); + + return size; +} + +static ssize_t virt_hdm_rev_reg_write(struct vfio_cxl_core_device *cxl, + void *buf, u64 offset, u64 size) +{ + /* Discard writes on reserved registers. */ + return size; +} + +static ssize_t hdm_decoder_n_lo_write(struct vfio_cxl_core_device *cxl, + void *buf, u64 offset, u64 size) +{ + u32 new_val =3D le32_to_cpu(*(u32 *)buf); + + if (WARN_ON_ONCE(size !=3D 4)) + return -EINVAL; + + /* Bit [27:0] are reserved. */ + new_val &=3D ~GENMASK(27, 0); + + new_val =3D cpu_to_le32(new_val); + offset =3D to_hdm_reg_offset(cxl, offset); + memcpy(hdm_reg_virt(cxl, offset), &new_val, size); + return size; +} + +static ssize_t hdm_decoder_global_ctrl_write(struct vfio_cxl_core_device *= cxl, + void *buf, u64 offset, u64 size) +{ + u32 hdm_decoder_global_cap; + u32 new_val =3D le32_to_cpu(*(u32 *)buf); + + if (WARN_ON_ONCE(size !=3D 4)) + return -EINVAL; + + /* Bit [31:2] are reserved. */ + new_val &=3D ~GENMASK(31, 2); + + /* Poison On Decode Error Enable bit is 0 and RO if not support. */ + hdm_decoder_global_cap =3D le32_to_cpu(*(u32 *)hdm_reg_virt(cxl, 0)); + if (!(hdm_decoder_global_cap & BIT(10))) + new_val &=3D ~BIT(0); + + new_val =3D cpu_to_le32(new_val); + offset =3D to_hdm_reg_offset(cxl, offset); + memcpy(hdm_reg_virt(cxl, offset), &new_val, size); + return size; +} + +static ssize_t hdm_decoder_n_ctrl_write(struct vfio_cxl_core_device *cxl, + void *buf, u64 offset, u64 size) +{ + u32 hdm_decoder_global_cap; + u32 ro_mask, rev_mask; + u32 new_val =3D le32_to_cpu(*(u32 *)buf); + u32 cur_val; + + if (WARN_ON_ONCE(size !=3D 4)) + return -EINVAL; + + offset =3D to_hdm_reg_offset(cxl, offset); + cur_val =3D le32_to_cpu(*(u32 *)hdm_reg_virt(cxl, offset)); + + /* Lock on commit */ + if (cur_val & BIT(8)) + return size; + + hdm_decoder_global_cap =3D le32_to_cpu(*(u32 *)hdm_reg_virt(cxl, 0)); + + /* RO and reserved bits in the spec */ + ro_mask =3D BIT(10) | BIT(11); + rev_mask =3D BIT(15) | GENMASK(31, 28); + + /* bits are not valid for devices */ + ro_mask |=3D BIT(12); + rev_mask |=3D GENMASK(19, 16) | GENMASK(23, 20); + + /* bits are reserved when UIO is not supported */ + if (!(hdm_decoder_global_cap & BIT(13))) + rev_mask |=3D BIT(14) | GENMASK(27, 24); + + /* clear reserved bits */ + new_val &=3D ~rev_mask; + + /* keep the RO bits */ + cur_val &=3D ro_mask; + new_val &=3D ~ro_mask; + new_val |=3D cur_val; + + /* emulate HDM decoder commit/de-commit */ + if (new_val & BIT(9)) + new_val |=3D BIT(10); + else + new_val &=3D ~BIT(10); + + new_val =3D cpu_to_le32(new_val); + memcpy(hdm_reg_virt(cxl, offset), &new_val, size); + return size; +} + +static int setup_mmio_emulation(struct vfio_cxl_core_device *cxl) +{ + u64 offset, base; + int ret; + + base =3D hdm_reg_base(cxl); + +#define ALLOC_BLOCK(offset, size, read, write) do { \ + ret =3D new_mmio_block(cxl, offset, size, read, write); \ + if (ret) \ + return ret; \ + } while (0) + + ALLOC_BLOCK(base + 0x4, 4, + virt_hdm_reg_read, + hdm_decoder_global_ctrl_write); + + offset =3D base + 0x10; + while (offset < base + cxl->hdm_reg_size) { + /* HDM N BASE LOW */ + ALLOC_BLOCK(offset, 4, + virt_hdm_reg_read, + hdm_decoder_n_lo_write); + + /* HDM N BASE HIGH */ + ALLOC_BLOCK(offset + 0x4, 4, + virt_hdm_reg_read, + virt_hdm_reg_write); + + /* HDM N SIZE LOW */ + ALLOC_BLOCK(offset + 0x8, 4, + virt_hdm_reg_read, + hdm_decoder_n_lo_write); + + /* HDM N SIZE HIGH */ + ALLOC_BLOCK(offset + 0xc, 4, + virt_hdm_reg_read, + virt_hdm_reg_write); + + /* HDM N CONTROL */ + ALLOC_BLOCK(offset + 0x10, 4, + virt_hdm_reg_read, + hdm_decoder_n_ctrl_write); + + /* HDM N TARGET LIST LOW */ + ALLOC_BLOCK(offset + 0x14, 0x4, + virt_hdm_reg_read, + virt_hdm_rev_reg_write); + + /* HDM N TARGET LIST HIGH */ + ALLOC_BLOCK(offset + 0x18, 0x4, + virt_hdm_reg_read, + virt_hdm_rev_reg_write); + + /* HDM N REV */ + ALLOC_BLOCK(offset + 0x1c, 0x4, + virt_hdm_reg_read, + virt_hdm_rev_reg_write); + + offset +=3D 0x20; + } + +#undef ALLOC_BLOCK + return 0; +} + void vfio_cxl_core_clean_register_emulation(struct vfio_cxl_core_device *c= xl) { struct list_head *pos, *n; @@ -17,10 +250,19 @@ void vfio_cxl_core_clean_register_emulation(struct vfi= o_cxl_core_device *cxl) =20 int vfio_cxl_core_setup_register_emulation(struct vfio_cxl_core_device *cx= l) { + int ret; + INIT_LIST_HEAD(&cxl->config_regblocks_head); INIT_LIST_HEAD(&cxl->mmio_regblocks_head); =20 + ret =3D setup_mmio_emulation(cxl); + if (ret) + goto err; + return 0; +err: + vfio_cxl_core_clean_register_emulation(cxl); + return ret; } =20 static struct vfio_emulated_regblock * diff --git a/include/linux/vfio_pci_core.h b/include/linux/vfio_pci_core.h index 12ded67c7db7..31fd28626846 100644 --- a/include/linux/vfio_pci_core.h +++ b/include/linux/vfio_pci_core.h @@ -251,5 +251,7 @@ ssize_t vfio_cxl_core_write(struct vfio_device *core_vd= ev, const char __user *bu size_t count, loff_t *ppos); long vfio_cxl_core_ioctl(struct vfio_device *core_vdev, unsigned int cmd, unsigned long arg); +int vfio_cxl_core_setup_register_emulation(struct vfio_cxl_core_device *cx= l); +void vfio_cxl_core_clean_register_emulation(struct vfio_cxl_core_device *c= xl); =20 #endif /* VFIO_PCI_CORE_H */ --=20 2.25.1 From nobody Sat Feb 7 12:25:21 2026 Received: from CY3PR05CU001.outbound.protection.outlook.com (mail-westcentralusazon11013017.outbound.protection.outlook.com [40.93.201.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 42EA432E754; Tue, 9 Dec 2025 16:52:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.93.201.17 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1765299176; cv=fail; b=Nwfv9nrqFR6aMhT+b8TsKljMKWQ/HS8JEesJSBjozavn4ciJFqjbR6eto1SNNWqVmcXzmJAh7AaWQTiWLx3BWFBJ0Ze6z8UwUdDrxN6LgcRP1GxQ+yRPHxCxLH3SxngZ0Zmsk4/96Bnacpi5rVDUosXdjTwpv1tQCTZKuLeqjQQ= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1765299176; c=relaxed/simple; bh=LOwsheOKKVVZ7acqCJgghSjBj7GHKmpFtZb4yIm6gYQ=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=Cvsx6xBc3giWyxrpjlWC5MKEsRKAaeG8KPhP8in3ml3E76jlrr4ZegFWsew+Cyj1DTfFIjjpAkK5oO+wJBR/etoLfvHaMgkSHJ6FSPJZkPQl9aZBL/gt7GBbZcICMHRC1cGm2o6PLZp0qqML4MZcXaGUCI/KKAy/YavI43z9zV8= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=XRIrYXaC; arc=fail smtp.client-ip=40.93.201.17 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="XRIrYXaC" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=vQL40svWU2gFZDNoxPSv8ukXwze4ySBBNa4mkQB4pXXeZSSaMRzadBOonDvdfXf1g0LnTjJ4y7izPLj/sHoZnzUhVgM/FAjwZmBroASvnMakx64aXsroXUnMuCgdNW/z6WxSM29YFYX8hWJuMenIHf42yB96KJK2TssctJXUmgE45yMcH/wKkXAKA9C5s4UZ8hp8KBCGoULLRwEY+TEyoLjZBNRqogVT4kIXf74gIa9b9jwwbU5PI0ucc1fCxS1kJp8jZpSjCUwoYirSD353WAVcGJ76ADQ0o8D6klkuLyNKTtfr5Ugy8+4LdydLmQp/81B1KTNcRaK24Bj2JwlmZw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=4qsLY0NgXi1tm8IptIeayfybd52oa2lHS+sDJQlUZXM=; b=m9bvhDp9vpcWKFbznXXLtRTWAV3qY17CTaLWiUvZEYOk8afc4ZW5tmsAhuh83RZy0P0/1VY/DrkOd6tKNY55+Cgp1O4/9oc21XWByFYk+n4t3oLIr5Kep6Yb2q9Nn5xuNHyAwd/hbTO6agGy2pnHVbp90MI5aD8RW7XMqSrXWHsVRAVumutivSkr9CkX8XSxqn0niAUbnAcTFi8pAJ0mCoCenrzF2cfGa7YuhIeaDo8Wbmmd1HPov7DawfB0FMINoWflcDW03V1LYEP+A7kffFcMc/JvSvBosQhWZBuwrgITcJJfYarmSnXAufeY08w83DxFTuqhhgNn5SZ82iMyuQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=4qsLY0NgXi1tm8IptIeayfybd52oa2lHS+sDJQlUZXM=; b=XRIrYXaCei+qxkDivcGNvZXpcB7dSpSM8r9UcXFVAWf/ClXm5cQirbcnMIniLj8w0Tle+K87rOj/Afrv/GZ9H+/RCDJdlqThuFk9OCc+y902WR9N9/WVTNx9b7UwTLMQgRcyD4XCcjCFdRunIa4prY9cTq/5/FSEBQOPlILUxp5ZVD0WLX+wZV/MuTol2oIjZoCllm+d7KzYUYurhw0qDw7aUFwdy9mtPAFuViatIdjHIP7WWW6G8Vw3jq5VVh46JRSpuwL4xBFTs6qvVIApRLPcRcoyoqx3lizox/DEbJlbZzi86Cp9jqNGSvxzzb2Ns8Y2dSx11CHM41PwX6u+VA== Received: from SJ0PR03CA0248.namprd03.prod.outlook.com (2603:10b6:a03:3a0::13) by IA0PPF0A63E7557.namprd12.prod.outlook.com (2603:10b6:20f:fc04::bc6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9388.9; Tue, 9 Dec 2025 16:52:43 +0000 Received: from SJ1PEPF00002311.namprd03.prod.outlook.com (2603:10b6:a03:3a0:cafe::45) by SJ0PR03CA0248.outlook.office365.com (2603:10b6:a03:3a0::13) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9388.14 via Frontend Transport; Tue, 9 Dec 2025 16:52:37 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by SJ1PEPF00002311.mail.protection.outlook.com (10.167.242.165) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9388.8 via Frontend Transport; Tue, 9 Dec 2025 16:52:43 +0000 Received: from rnnvmail204.nvidia.com (10.129.68.6) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 9 Dec 2025 08:52:20 -0800 Received: from rnnvmail204.nvidia.com (10.129.68.6) by rnnvmail204.nvidia.com (10.129.68.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 9 Dec 2025 08:52:20 -0800 Received: from nvidia-4028GR-scsim.nvidia.com (10.127.8.11) by mail.nvidia.com (10.129.68.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20 via Frontend Transport; Tue, 9 Dec 2025 08:52:13 -0800 From: To: , , , , , , , , , , , , , , , , , CC: , , , , , , , , Subject: [RFC v2 12/15] vfio/cxl: introduce the emulation of CXL configuration space Date: Tue, 9 Dec 2025 22:20:16 +0530 Message-ID: <20251209165019.2643142-13-mhonap@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20251209165019.2643142-1-mhonap@nvidia.com> References: <20251209165019.2643142-1-mhonap@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ1PEPF00002311:EE_|IA0PPF0A63E7557:EE_ X-MS-Office365-Filtering-Correlation-Id: 32a9971a-ef1a-4651-18a3-08de37435f28 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|36860700013|376014|82310400026|7416014|1800799024|13003099007|921020; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?hLk15DCFxAxdQIx8WOOYP9h/aoK/Wxkep7dL0uPMLGjKB3aC7H1wHedwBdRi?= =?us-ascii?Q?WoQikgM6efLRxyZ+gMtcc7CiJqDBeLcHcdwUBhjcWfb7iVixO7hNhTPsnVca?= =?us-ascii?Q?DkN0k5VXjF7M7+25zdCHVz5dc9J2y7kVqPUGS+o3EnXAN/jEctsbz0S4ZRTt?= =?us-ascii?Q?A+s+BRofNcHl2zNBw+SHhv+FU4mOjScRs6WWpD/Bt8Icy10ZLWVD+YA3BknF?= =?us-ascii?Q?6CkaxGI+WYKNXFyMcZq/JXpq+YcKjETlUq8d0+cTVed/r+zxCvCGNQkXOSJP?= =?us-ascii?Q?1IfAP3MBpAgT2FiWMoRDYcZrtIuDGE6GWKPyo/MZYGq2p9EDvdltWFZTo7NM?= =?us-ascii?Q?UwU0WZp986GaxtOLnL4+r21DKJ4UprlzJKa/7IRlg5X5jWHve9x0S6tb9/wp?= =?us-ascii?Q?RXxl46FA1d18bdq3BU0tJ2Faxdc3EQRfzfJRLM9PE48UMZi5faUiHUti2/4l?= =?us-ascii?Q?nPNTgf4KUY76TuYXIbD8Hv9GHQdK+kCOrbFJPz4S1P8+niA+IR884AjkBo4G?= =?us-ascii?Q?PRCHlLS4/1u0tSBJUZU5TAC4WEyj0RSe238Dp0Yepa/NOk4etQamOUBukdgX?= =?us-ascii?Q?Qk+QReFXKT6b4A/s3/aznaTjdbvkg4FcZ6jZeBC2aFPH1hHtfKCqV1O39ekU?= =?us-ascii?Q?qi9BCbN+1K13aHoSj25NNVajwS0jD+isZs1UzhoLWok+o8cZ++bGoHqLrUy8?= =?us-ascii?Q?PJhsKJ/pyjPqnOYYN6L7qaBpNUNlLhlCgnd2AWKmEVCkQTnR6PH+AOFT6MTA?= =?us-ascii?Q?0FjCP+VpR/xt8JSnY7tSiIN58gxerqvm8xVQJSqA+IAekz4cygsuAeq0F1F5?= =?us-ascii?Q?I4iRlGKkxHH1ckEVKGKpFsaJEzkqOlniyJfzXCUR5xs+Cy8P9W3Q1Zi5A8iy?= =?us-ascii?Q?l4OzCMwdDgzNsADmOL8dQl3E3GDnm8sBUGJdupAxo4g29lS5z5+olOi7/MCx?= =?us-ascii?Q?W7SkuerM5/HlhPA61wMMlCvwAKYZRik2w+9ADZNq86bWoa4Shf88ZNyTdL72?= =?us-ascii?Q?IneRQW0SvLjKTpqlKg7cuDPMTvUKE2IjdUBPJITXUyz3QFBrcoonZynXm45y?= =?us-ascii?Q?l3Wo1U0v/F2p0O07iFz/GCcb09I8b9MwYorqodmjgSHmIOGCD/qKYqs+ZWCu?= =?us-ascii?Q?JAdYgM48WU0Fb7Y27YyI8/JmCw9oopXiXUMx7lBPCA7OVc00nbbom2qoNFf+?= =?us-ascii?Q?vgV0NOppTRppOnZOL7jRozusbWTD7D/aksMwpnVI0/TIomZGnMCTCbO4GF0y?= =?us-ascii?Q?wVxgMbxkAfkf+nyklmHmtsR0xyNoSsYwElZ8JEmQS5IslMLomngoLLOwElrr?= =?us-ascii?Q?QHTi/dX9H1rfne6BQq0MBTRcFxVOflmd8DMQfWJqckGKQWcjgTSmwV8pcrGj?= =?us-ascii?Q?2c/E3w8jBq86/xHX0LlVkUOrHPEGAP8df1ftIc3vzSkZfMyjmqLsnxFF8sZq?= =?us-ascii?Q?eOSywjmmccSCQMIsPURCGTpVqpXoZ5xk89QTsYBEcz1WBNWXcicCrgiCFolI?= =?us-ascii?Q?vg4z/hCHt3J4FinFxEm+M4uJoO/Sw1KbyhCeQmYk9hWlaZhUY4CpgLwXAE6a?= =?us-ascii?Q?8JYqr7dNol3UVs04pRCrhN1bYGHAHhZhgFvG5PC4?= X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(36860700013)(376014)(82310400026)(7416014)(1800799024)(13003099007)(921020);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Dec 2025 16:52:43.1202 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 32a9971a-ef1a-4651-18a3-08de37435f28 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: SJ1PEPF00002311.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA0PPF0A63E7557 Content-Type: text/plain; charset="utf-8" From: Zhi Wang CXL devices have CXL DVSEC registers in the configuration space. Many of them affects the behaviors of the devices. E.g. enabling CXL.io/CXL.mem/CXL.cache. However, these configuration are owned by the host and a virtualization policy should be applied when handling the access from the guest. Introduce the emulation of CXL configuration space to handle the access of the virtual CXL configuration space from the guest. Signed-off-by: Zhi Wang Signed-off-by: Manish Honap --- drivers/vfio/pci/vfio_cxl_core_emu.c | 340 ++++++++++++++++++++++++++- drivers/vfio/pci/vfio_pci_config.c | 10 +- include/linux/vfio_pci_core.h | 4 + 3 files changed, 346 insertions(+), 8 deletions(-) diff --git a/drivers/vfio/pci/vfio_cxl_core_emu.c b/drivers/vfio/pci/vfio_c= xl_core_emu.c index 6711ff8975ef..8037737838ba 100644 --- a/drivers/vfio/pci/vfio_cxl_core_emu.c +++ b/drivers/vfio/pci/vfio_cxl_core_emu.c @@ -28,6 +28,334 @@ new_reg_block(struct vfio_cxl_core_device *cxl, u64 off= set, u64 size, return block; } =20 +static int new_config_block(struct vfio_cxl_core_device *cxl, u64 offset, + u64 size, reg_handler_t *read, reg_handler_t *write) +{ + struct vfio_emulated_regblock *block; + + block =3D new_reg_block(cxl, offset, size, read, write); + if (IS_ERR(block)) + return PTR_ERR(block); + + list_add_tail(&block->list, &cxl->config_regblocks_head); + return 0; +} + +static ssize_t virt_config_reg_read(struct vfio_cxl_core_device *cxl, void= *buf, + u64 offset, u64 size) +{ + memcpy(buf, cxl->config_virt + offset, size); + return size; +} + +static ssize_t virt_config_reg_write(struct vfio_cxl_core_device *cxl, voi= d *buf, + u64 offset, u64 size) +{ + memcpy(cxl->config_virt + offset, buf, size); + return size; +} + +static ssize_t hw_config_reg_read(struct vfio_cxl_core_device *cxl, void *= buf, + u64 offset, u64 size) +{ + return vfio_user_config_read(cxl->pci_core.pdev, offset, buf, size); +} + +static ssize_t hw_config_reg_write(struct vfio_cxl_core_device *cxl, void = *buf, + u64 offset, u64 size) +{ + __le32 write_val =3D *(__le32 *)buf; + + return vfio_user_config_write(cxl->pci_core.pdev, offset, write_val, size= ); +} + +static ssize_t cxl_control_write(struct vfio_cxl_core_device *cxl, void *b= uf, + u64 offset, u64 size) +{ + u16 lock =3D le16_to_cpu(*(u16 *)(cxl->config_virt + cxl->dvsec + 0x14)); + u16 cap3 =3D le16_to_cpu(*(u16 *)(cxl->config_virt + cxl->dvsec + 0x38)); + u16 new_val =3D le16_to_cpu(*(u16 *)buf); + u16 rev_mask; + + if (WARN_ON_ONCE(size !=3D 2)) + return -EINVAL; + + /* register is locked */ + if (lock & BIT(0)) + return size; + + /* handle reserved bits in the spec */ + rev_mask =3D BIT(13) | BIT(15); + + /* no direct p2p cap */ + if (!(cap3 & BIT(4))) + rev_mask |=3D BIT(12); + + new_val &=3D ~rev_mask; + + /* CXL.io is always enabled. */ + new_val |=3D BIT(1); + + memcpy(cxl->config_virt + offset, &new_val, size); + return size; +} + +static ssize_t cxl_status_write(struct vfio_cxl_core_device *cxl, void *bu= f, + u64 offset, u64 size) +{ + u16 cur_val =3D le16_to_cpu(*(u16 *)(cxl->config_virt + offset)); + u16 new_val =3D le16_to_cpu(*(u16 *)buf); + u16 rev_mask =3D GENMASK(13, 0) | BIT(15); + + if (WARN_ON_ONCE(size !=3D 2)) + return -EINVAL; + + /* handle reserved bits in the spec */ + new_val &=3D ~rev_mask; + + /* emulate RW1C bit */ + if (new_val & BIT(14)) { + new_val &=3D ~BIT(14); + } else { + new_val &=3D ~BIT(14); + new_val |=3D cur_val & BIT(14); + } + + new_val =3D cpu_to_le16(new_val); + memcpy(cxl->config_virt + offset, &new_val, size); + return size; +} + +static ssize_t cxl_control_2_write(struct vfio_cxl_core_device *cxl, void = *buf, + u64 offset, u64 size) +{ + struct pci_dev *pdev =3D cxl->pci_core.pdev; + u16 cap2 =3D le16_to_cpu(*(u16 *)(cxl->config_virt + cxl->dvsec + 0x16)); + u16 cap3 =3D le16_to_cpu(*(u16 *)(cxl->config_virt + cxl->dvsec + 0x38)); + u16 new_val =3D le16_to_cpu(*(u16 *)buf); + u16 rev_mask =3D GENMASK(15, 6) | BIT(1) | BIT(2); + u16 hw_bits =3D BIT(0) | BIT(1) | BIT(3); + bool initiate_cxl_reset =3D new_val & BIT(2); + + if (WARN_ON_ONCE(size !=3D 2)) + return -EINVAL; + + /* no desired volatile HDM state after host reset */ + if (!(cap3 & BIT(2))) + rev_mask |=3D BIT(4); + + /* no modified completion enable */ + if (!(cap2 & BIT(6))) + rev_mask |=3D BIT(5); + + /* handle reserved bits in the spec */ + new_val &=3D ~rev_mask; + + /* bits go to the HW */ + hw_bits &=3D new_val; + + /* update the virt regs */ + new_val =3D cpu_to_le16(new_val); + memcpy(cxl->config_virt + offset, &new_val, size); + + if (hw_bits) + pci_write_config_word(pdev, offset, cpu_to_le16(hw_bits)); + + if (initiate_cxl_reset) { + /* TODO: call linux CXL reset */ + } + return size; +} + +static ssize_t cxl_status_2_write(struct vfio_cxl_core_device *cxl, void *= buf, + u64 offset, u64 size) +{ + struct pci_dev *pdev =3D cxl->pci_core.pdev; + u16 cap3 =3D le16_to_cpu(*(u16 *)(cxl->config_virt + cxl->dvsec + 0x38)); + u16 new_val =3D le16_to_cpu(*(u16 *)buf); + + if (WARN_ON_ONCE(size !=3D 2)) + return -EINVAL; + + /* write RW1CS if supports */ + if ((cap3 & BIT(2)) && (new_val & BIT(3))) + pci_write_config_word(pdev, offset, BIT(3)); + + /* No need to update the virt regs, CXL status reads from the HW */ + return size; +} + +static ssize_t cxl_lock_write(struct vfio_cxl_core_device *cxl, void *buf, + u64 offset, u64 size) +{ + u16 cur_val =3D le16_to_cpu(*(u16 *)(cxl->config_virt + offset)); + u16 new_val =3D le16_to_cpu(*(u16 *)buf); + u16 rev_mask =3D GENMASK(15, 1); + + if (WARN_ON_ONCE(size !=3D 2)) + return -EINVAL; + + /* LOCK is not allowed to be cleared unless conventional reset. */ + if (cur_val & BIT(0)) + return size; + + /* handle reserved bits in the spec */ + new_val &=3D ~rev_mask; + + new_val =3D cpu_to_le16(new_val); + memcpy(cxl->config_virt + offset, &new_val, size); + return size; +} + +static ssize_t cxl_base_lo_write(struct vfio_cxl_core_device *cxl, void *b= uf, + u64 offset, u64 size) +{ + u32 new_val =3D le32_to_cpu(*(u32 *)buf); + u32 rev_mask =3D GENMASK(27, 0); + + if (WARN_ON_ONCE(size !=3D 4)) + return -EINVAL; + + /* handle reserved bits in the spec */ + new_val &=3D ~rev_mask; + + new_val =3D cpu_to_le32(new_val); + memcpy(cxl->config_virt + offset, &new_val, size); + return size; +} + +static ssize_t virt_config_reg_ro_write(struct vfio_cxl_core_device *cxl, = void *buf, + u64 offset, u64 size) +{ + return size; +} + +static int setup_config_emulation(struct vfio_cxl_core_device *cxl) +{ + u16 offset; + int ret; + +#define ALLOC_BLOCK(offset, size, read, write) do { \ + ret =3D new_config_block(cxl, offset, size, read, write); \ + if (ret) \ + return ret; \ + } while (0) + + ALLOC_BLOCK(cxl->dvsec, 4, + virt_config_reg_read, + virt_config_reg_ro_write); + + ALLOC_BLOCK(cxl->dvsec + 0x4, 4, + virt_config_reg_read, + virt_config_reg_ro_write); + + ALLOC_BLOCK(cxl->dvsec + 0x8, 2, + virt_config_reg_read, + virt_config_reg_ro_write); + + /* CXL CAPABILITY */ + ALLOC_BLOCK(cxl->dvsec + 0xa, 2, + virt_config_reg_read, + virt_config_reg_ro_write); + + /* CXL CONTROL */ + ALLOC_BLOCK(cxl->dvsec + 0xc, 2, + virt_config_reg_read, + cxl_control_write); + + /* CXL STATUS */ + ALLOC_BLOCK(cxl->dvsec + 0xe, 2, + virt_config_reg_read, + cxl_status_write); + + /* CXL CONTROL 2 */ + ALLOC_BLOCK(cxl->dvsec + 0x10, 2, + virt_config_reg_read, + cxl_control_2_write); + + /* CXL STATUS 2 */ + ALLOC_BLOCK(cxl->dvsec + 0x12, 2, + hw_config_reg_read, + cxl_status_2_write); + + /* CXL LOCK */ + ALLOC_BLOCK(cxl->dvsec + 0x14, 2, + virt_config_reg_read, + cxl_lock_write); + + /* CXL CAPABILITY 2 */ + ALLOC_BLOCK(cxl->dvsec + 0x16, 2, + virt_config_reg_read, + virt_config_reg_ro_write); + + /* CXL RANGE 1 SIZE HIGH & LOW */ + ALLOC_BLOCK(cxl->dvsec + 0x18, 4, + virt_config_reg_read, + virt_config_reg_ro_write); + + ALLOC_BLOCK(cxl->dvsec + 0x1c, 4, + virt_config_reg_read, + virt_config_reg_ro_write); + + /* CXL RANG BASE 1 HIGH */ + ALLOC_BLOCK(cxl->dvsec + 0x20, 4, + virt_config_reg_read, + virt_config_reg_write); + + /* CXL RANG BASE 1 LOW */ + ALLOC_BLOCK(cxl->dvsec + 0x24, 4, + virt_config_reg_read, + cxl_base_lo_write); + + /* CXL RANGE 2 SIZE HIGH & LOW */ + ALLOC_BLOCK(cxl->dvsec + 0x28, 4, + virt_config_reg_read, + virt_config_reg_ro_write); + + ALLOC_BLOCK(cxl->dvsec + 0x2c, 4, + virt_config_reg_read, + virt_config_reg_ro_write); + + /* CXL RANG BASE 2 HIGH */ + ALLOC_BLOCK(cxl->dvsec + 0x30, 4, + virt_config_reg_read, + virt_config_reg_write); + + /* CXL RANG BASE 2 LOW */ + ALLOC_BLOCK(cxl->dvsec + 0x34, 4, + virt_config_reg_read, + cxl_base_lo_write); + + /* CXL CAPABILITY 3 */ + ALLOC_BLOCK(cxl->dvsec + 0x38, 2, + virt_config_reg_read, + virt_config_reg_ro_write); + + while ((offset =3D pci_find_next_ext_capability(cxl->pci_core.pdev, + offset, + PCI_EXT_CAP_ID_DOE))) { + ALLOC_BLOCK(offset + PCI_DOE_CTRL, 4, + hw_config_reg_read, + hw_config_reg_write); + + ALLOC_BLOCK(offset + PCI_DOE_STATUS, 4, + hw_config_reg_read, + hw_config_reg_write); + + ALLOC_BLOCK(offset + PCI_DOE_WRITE, 4, + hw_config_reg_read, + hw_config_reg_write); + + ALLOC_BLOCK(offset + PCI_DOE_READ, 4, + hw_config_reg_read, + hw_config_reg_write); + } + +#undef ALLOC_BLOCK + + return 0; +} + static int new_mmio_block(struct vfio_cxl_core_device *cxl, u64 offset, u6= 4 size, reg_handler_t *read, reg_handler_t *write) { @@ -179,10 +507,10 @@ static int setup_mmio_emulation(struct vfio_cxl_core_= device *cxl) =20 base =3D hdm_reg_base(cxl); =20 -#define ALLOC_BLOCK(offset, size, read, write) do { \ - ret =3D new_mmio_block(cxl, offset, size, read, write); \ - if (ret) \ - return ret; \ +#define ALLOC_BLOCK(offset, size, read, write) do { \ + ret =3D new_mmio_block(cxl, offset, size, read, write); \ + if (ret) \ + return ret; \ } while (0) =20 ALLOC_BLOCK(base + 0x4, 4, @@ -255,6 +583,10 @@ int vfio_cxl_core_setup_register_emulation(struct vfio= _cxl_core_device *cxl) INIT_LIST_HEAD(&cxl->config_regblocks_head); INIT_LIST_HEAD(&cxl->mmio_regblocks_head); =20 + ret =3D setup_config_emulation(cxl); + if (ret) + goto err; + ret =3D setup_mmio_emulation(cxl); if (ret) goto err; diff --git a/drivers/vfio/pci/vfio_pci_config.c b/drivers/vfio/pci/vfio_pci= _config.c index 8f02f236b5b4..4847d09e58b4 100644 --- a/drivers/vfio/pci/vfio_pci_config.c +++ b/drivers/vfio/pci/vfio_pci_config.c @@ -120,8 +120,8 @@ struct perm_bits { #define NO_WRITE 0 #define ALL_WRITE 0xFFFFFFFFU =20 -static int vfio_user_config_read(struct pci_dev *pdev, int offset, - __le32 *val, int count) +int vfio_user_config_read(struct pci_dev *pdev, int offset, + __le32 *val, int count) { int ret =3D -EINVAL; u32 tmp_val =3D 0; @@ -150,9 +150,10 @@ static int vfio_user_config_read(struct pci_dev *pdev,= int offset, =20 return ret; } +EXPORT_SYMBOL_GPL(vfio_user_config_read); =20 -static int vfio_user_config_write(struct pci_dev *pdev, int offset, - __le32 val, int count) +int vfio_user_config_write(struct pci_dev *pdev, int offset, + __le32 val, int count) { int ret =3D -EINVAL; u32 tmp_val =3D le32_to_cpu(val); @@ -171,6 +172,7 @@ static int vfio_user_config_write(struct pci_dev *pdev,= int offset, =20 return ret; } +EXPORT_SYMBOL_GPL(vfio_user_config_write); =20 static int vfio_default_config_read(struct vfio_pci_core_device *vdev, int= pos, int count, struct perm_bits *perm, diff --git a/include/linux/vfio_pci_core.h b/include/linux/vfio_pci_core.h index 31fd28626846..8293910e0a96 100644 --- a/include/linux/vfio_pci_core.h +++ b/include/linux/vfio_pci_core.h @@ -201,6 +201,10 @@ ssize_t vfio_pci_core_do_io_rw(struct vfio_pci_core_de= vice *vdev, bool test_mem, void __iomem *io, char __user *buf, loff_t off, size_t count, size_t x_start, size_t x_end, bool iswrite); +int vfio_user_config_read(struct pci_dev *pdev, int offset, + __le32 *val, int count); +int vfio_user_config_write(struct pci_dev *pdev, int offset, + __le32 val, int count); bool vfio_pci_core_range_intersect_range(loff_t buf_start, size_t buf_cnt, loff_t reg_start, size_t reg_cnt, loff_t *buf_offset, --=20 2.25.1 From nobody Sat Feb 7 12:25:21 2026 Received: from CO1PR03CU002.outbound.protection.outlook.com (mail-westus2azon11010061.outbound.protection.outlook.com [52.101.46.61]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B7522329E71; Tue, 9 Dec 2025 16:53:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.46.61 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1765299189; cv=fail; b=Gzsa+LyQySbl118Gvu/ho5P6drIC1aoH2oBboET1erX5LhMzFpREhxJW73fFIEsEW1r27lN4BhatxZ/LA0YtQuti615qIl90Thf9KogAqeQPLo/oMr1l4sZdKkpZtox6e3KHO7j7KL2eD/3jVrqEGMYK5ReA8YesaR283lv1aIQ= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1765299189; c=relaxed/simple; bh=gEEgQIOd9mT5oUTGlU3t3r/uaZai0pOuZ5Mm/9+SoJ4=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=eToi3pE5Rn/oxzZhSgP1m/4aRH0Ctahuc3VMYNjzqrjn5E2QjvENe8aQnGnvc+n05zuIclrzNxM4x2It8I7I6JyEzYUEhxW1pCFmoSq/wCGdCg4kvu6C6Pq0W0khspNk1ebQITDaggU/buGBQy7/tf0ymN9NgEmfeweul1Ra/9c= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=Jy7aucls; arc=fail smtp.client-ip=52.101.46.61 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="Jy7aucls" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=sx54D8Pa+dZ8TmIv4bgc0awgGTFiXt0w3wsBavpL3LgJ4YPDagi2OGTeLdL3bKesFq/CYzFmVh9+KBrRw6+3mxLNzkoTqm9syOBTYaoclrY7bTpJms0RtD90xuMbeOI4my+SdmoV9JSn8XGvYd8BPSLR6kgTzsda7nukwAlow7BxTFWy0J/xp48InI0P7yqRJzvb8PlAXl/eiVhoQmS5FHkh1aWE15Rwa/iQrB4TStV5idKwwyl0yB/JKA2xWafInCkta+sksOZt7e9jAL/Dq0a1FkxPqCvgT1YXk/WQcbCU7+hSqke2mw687PJyYkUEDZh2sSVk5mJVLr3JdQ7d7A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=kvYlUtczBtqzbs2dwfStWNHxVT4etEGFP5Sf8DJBnJg=; b=qjHZZOTzpi5yqVpEBzTaWO76k4qeaxE5Y9+lf6wJBZmvDvbIm10uXQezYyzDMgE0b3M/hpTj2qjcyBsMgiBsXYgIDHDI0AKL8/v6pm6JO+O/Lrs+rF1DqBMUp7Dyq8Yvl09PdDLqmAHWdf8mbtMyW5zDnbOsSYnJMKQImhwnAp2OAXSDUOJypZvXDu8UOp8+S5Ike0BemUlJvf22U/QAaOurgcV08eaE0TmWbQt+ZA5e0hKNAh/5mLA3RoG7Lu0ooCetei23J6qB/CyOlwkdv3Cpta5fIm+BwoXwgioEGwt6/roJujPtpuqSaTfYS+Hy00PU+JIjECSX6aklNHAwAg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=kvYlUtczBtqzbs2dwfStWNHxVT4etEGFP5Sf8DJBnJg=; b=Jy7auclsb5tfZ5DOj/q8gmU8Vhdr+vlWec313PiDpENjIUiqpJv39ZNS86l1rV0YR24YW9VRc+8n/b9JaKy6zpDZNi6iZgllxlGWqWizI5J04jYMpBhDi7G538WhAi8r3ouc9HQPup1NKiOxDwOfcMQYjlTQeYGImX/Vv5t7XR1lzupdC9wNueiI+BB66Giz9OkTjETX+L3grF9aU3HS6O7RBxAh5zo8UYf4/kaPf3SI0YrI2MNoiLgVOHx52U9fRC8XaYvAfUbakcdQjY6Up8Ut1y+vkF5X1lCXDoOc4EuXHVyS53nafmu8Cm9jIv6AwkQBLqwi52SoItbAD9Kkwg== Received: from BY5PR03CA0012.namprd03.prod.outlook.com (2603:10b6:a03:1e0::22) by CH2PR12MB4166.namprd12.prod.outlook.com (2603:10b6:610:78::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9412.6; Tue, 9 Dec 2025 16:52:52 +0000 Received: from SJ1PEPF00002312.namprd03.prod.outlook.com (2603:10b6:a03:1e0:cafe::b8) by BY5PR03CA0012.outlook.office365.com (2603:10b6:a03:1e0::22) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9388.14 via Frontend Transport; Tue, 9 Dec 2025 16:52:51 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by SJ1PEPF00002312.mail.protection.outlook.com (10.167.242.166) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9388.8 via Frontend Transport; Tue, 9 Dec 2025 16:52:51 +0000 Received: from rnnvmail205.nvidia.com (10.129.68.10) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 9 Dec 2025 08:52:28 -0800 Received: from rnnvmail204.nvidia.com (10.129.68.6) by rnnvmail205.nvidia.com (10.129.68.10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 9 Dec 2025 08:52:27 -0800 Received: from nvidia-4028GR-scsim.nvidia.com (10.127.8.11) by mail.nvidia.com (10.129.68.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20 via Frontend Transport; Tue, 9 Dec 2025 08:52:20 -0800 From: To: , , , , , , , , , , , , , , , , , CC: , , , , , , , , Subject: [RFC v2 13/15] vfio/pci: introduce CXL device awareness Date: Tue, 9 Dec 2025 22:20:17 +0530 Message-ID: <20251209165019.2643142-14-mhonap@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20251209165019.2643142-1-mhonap@nvidia.com> References: <20251209165019.2643142-1-mhonap@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ1PEPF00002312:EE_|CH2PR12MB4166:EE_ X-MS-Office365-Filtering-Correlation-Id: c2c78d13-2cf5-4309-9aea-08de37436405 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|36860700013|1800799024|376014|7416014|82310400026|921020; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?02MgoSwtP1wFlIadi7GmXhkwGuAiNUdQQ3Mn2WrTydLaGuF0d8mUAYk2W0O6?= =?us-ascii?Q?x92scuGZc2SmOQCvr5lWEUC6FYwfVwE4AZO6vpfJWokOJyx1onXBbgxqj7Sw?= =?us-ascii?Q?IVLQDC0KFgdLJ2aoN9kj10nmU8ZdpdP36frqa1piP8XcL3w7KOXtDBa/voUI?= =?us-ascii?Q?8t6ErFiglMDMz4Rs2mI+Hdacj1NWII9RljLUMnFVe6rdwEYA/+kdi3SyiOVm?= =?us-ascii?Q?OUBzKfJ0uRgdavkBhGpZiy1NaHWFYNdBY+vuidnLpjE7PIUExuFQA9/afPNe?= =?us-ascii?Q?/rI0gotuzxXzbkmq2b5JNrn0AX+/cCtmBPFM+RMr/sA9n+/AzLD1DkDmtMZ+?= =?us-ascii?Q?x0IE9oUYXgVOpKutZpe7n0r68qSmRH2Yt14dbko98M+v5TdLPgGWHHsXCsqq?= =?us-ascii?Q?QiCGaYQhlA6DK4hzNKiQCZM2I/Ei2JaxMScHMndm7zxcmBliB6xMOM3myi9d?= =?us-ascii?Q?omksgnxE/Ejy8S3HB+anDt1EFz4c9D6jVQ/Te1hc+gdViTL/qybtc//NqiUZ?= =?us-ascii?Q?07YtVKyuTi2YDfEHqtdtb9s9At8I5o30dp5kD8zv8Ndr3VLhRhhSVyU7BKB/?= =?us-ascii?Q?1nXR8FjyKL9pVp5BDi8VYY5HsSZLheV+Sa4bENtWB/+0id6EQM5oRo0lm/na?= =?us-ascii?Q?Pn2GGT6aAhG6TsTPFA2Xa3XNOAqHP7k5ywyG/OFbCxa5wyPFWlEcxPHwnxTd?= =?us-ascii?Q?L7AYOPUj2VTGXRsw2MeesJ67XjqSSSVNre+QRkHuc1xTs1s3BuftnjW4ZlBE?= =?us-ascii?Q?Bm0djXcEOOK37LZghx0AlatoGlLVf28a8bXmlaTvoGaCl7pSecpKhtbg2sh3?= =?us-ascii?Q?sU8XQC+qMwWOrYelBo986N+CBPKyj568gj4fi5txYpAe5Zp1xm2I2Ki9LgBI?= =?us-ascii?Q?8JAKrJ87iCCEL68Zn2jm0cYofNavKJOIaw+hmri45t1mKI117dh1/9wN7V8G?= =?us-ascii?Q?yCT2cU3KPdkGgMtrHQlBOHBEHqmKGvfyW8IDaIyN6Tb/3hzUCzKB5yNjBcd1?= =?us-ascii?Q?CJHq+Qg5IGExwc2S7I/WiMQW1RkXL3JhZzsdW2wJM01Aw4naLpEqFwq83m5i?= =?us-ascii?Q?Sk4vCWWVB2oL6hyRkzb2zFLPicl5rc5iMu+at6mXTUwivEGu2mwD8TigrYyI?= =?us-ascii?Q?ifrWMirxCKfMKC4omiDNldqH9eWpwUZwV8zve3AgofN5GUyt5wMGgXphZG9g?= =?us-ascii?Q?EqK5Yn8X1xWYmSl+SgxAsFCywdrOOhddkTGD5iomvbJDcdA3hwuH6rRWbpAV?= =?us-ascii?Q?z30z745zQTHkrK7ZyL/XnoBJwf7HW5RZWIUAbGVDeRI7g5lj/iGEgeQr2y/1?= =?us-ascii?Q?iucZXzowqi44EZ8WV3eT9aoNeJJakLHvlIfUsM4IhK+orltIZlkzdtnDhpTV?= =?us-ascii?Q?iJ2cUhtiXE6rqJvLe2EsVSkWxbjOKrw5enp4rBWF/3RvNXDZftGQDBhpEEp1?= =?us-ascii?Q?Gg2FURmZwrWHP4GkINKdCl8Po0QJhVOwv8hZf2Z00O9Wu1v0jQQALwMgGOH4?= =?us-ascii?Q?ZVcML/AQMtRiVjEvFYxDi1cbUUhMIBfX3q8+JZ2Lz4WV9YoX/X0riaZulooG?= =?us-ascii?Q?F0ZUqcEz6RukFczxy1zXIdSCFOQ3HA1Z8GKbJgYT?= X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(36860700013)(1800799024)(376014)(7416014)(82310400026)(921020);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Dec 2025 16:52:51.2078 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: c2c78d13-2cf5-4309-9aea-08de37436405 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: SJ1PEPF00002312.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH2PR12MB4166 Content-Type: text/plain; charset="utf-8" From: Zhi Wang CXL device programming interfaces are built upon PCI interfaces. Thus the vfio-pci-core can be leveraged to handle a CXL device. However, CXL device also has difference with PCI devicce: - No INTX support, only MSI/MSIX is supported. - Reset is done via CXL reset. FLR only reset CXL.io. Introduce the CXL device awareness to the vfio-pci-core. Expose a new VFIO device flags to the userspace to identify the VFIO device is a CXL device. Disable INTX support in the vfio-pci-core. Disable FLR reset for the CXL device as the kernel CXL core hasn't support CXL reset yet. Disable mmap support on the CXL MMIO BAR in vfio-pci-core. Signed-off-by: Zhi Wang Signed-off-by: Manish Honap --- drivers/vfio/pci/vfio_cxl_core.c | 18 +++++++++++++++++ drivers/vfio/pci/vfio_pci_core.c | 33 ++++++++++++++++++++++++++++---- drivers/vfio/pci/vfio_pci_rdwr.c | 11 ++++++++--- include/linux/vfio_pci_core.h | 3 +++ include/uapi/linux/vfio.h | 10 ++++++++++ 5 files changed, 68 insertions(+), 7 deletions(-) diff --git a/drivers/vfio/pci/vfio_cxl_core.c b/drivers/vfio/pci/vfio_cxl_c= ore.c index c0bdf55997da..84e4f42d97de 100644 --- a/drivers/vfio/pci/vfio_cxl_core.c +++ b/drivers/vfio/pci/vfio_cxl_core.c @@ -25,6 +25,19 @@ #define DRIVER_AUTHOR "Zhi Wang " #define DRIVER_DESC "core driver for VFIO based CXL devices" =20 +static void init_cxl_cap(struct vfio_cxl_core_device *cxl) +{ + struct vfio_pci_core_device *pci =3D &cxl->pci_core; + struct vfio_device_info_cap_cxl *cap =3D &pci->cxl_cap; + + cap->header.id =3D VFIO_DEVICE_INFO_CAP_CXL; + cap->header.version =3D 1; + cap->hdm_count =3D cxl->hdm_count; + cap->hdm_reg_offset =3D cxl->comp_reg_offset + cxl->hdm_reg_offset; + cap->hdm_reg_size =3D cxl->hdm_reg_size; + cap->hdm_reg_bar_index =3D cxl->comp_reg_bar; +} + /* Standard CXL-type 2 driver initialization sequence */ static int enable_cxl(struct vfio_cxl_core_device *cxl, u16 dvsec, struct vfio_cxl_dev_info *info) @@ -74,6 +87,8 @@ static int enable_cxl(struct vfio_cxl_core_device *cxl, u= 16 dvsec, if (IS_ERR(cxl_core->cxlmd)) return PTR_ERR(cxl_core->cxlmd); =20 + init_cxl_cap(cxl); + cxl_core->region.noncached =3D info->noncached_region; =20 return 0; @@ -266,6 +281,9 @@ int vfio_cxl_core_enable(struct vfio_cxl_core_device *c= xl, if (ret) return ret; =20 + pci->is_cxl =3D true; + pci->comp_reg_bar =3D cxl->comp_reg_bar; + ret =3D vfio_pci_core_enable(pci); if (ret) goto err_pci_core_enable; diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_c= ore.c index 502880e927fc..5f8334748841 100644 --- a/drivers/vfio/pci/vfio_pci_core.c +++ b/drivers/vfio/pci/vfio_pci_core.c @@ -483,7 +483,12 @@ int vfio_pci_core_enable(struct vfio_pci_core_device *= vdev) goto out_power; =20 /* If reset fails because of the device lock, fail this path entirely */ - ret =3D pci_try_reset_function(pdev); + if (!vdev->is_cxl) + ret =3D pci_try_reset_function(pdev); + else + /* TODO: CXL reset support is on-going. */ + ret =3D -ENODEV; + if (ret =3D=3D -EAGAIN) goto out_disable_device; =20 @@ -618,8 +623,12 @@ void vfio_pci_core_disable(struct vfio_pci_core_device= *vdev) if (!vdev->barmap[bar]) continue; pci_iounmap(pdev, vdev->barmap[bar]); - pci_release_selected_regions(pdev, 1 << bar); vdev->barmap[bar] =3D NULL; + + if (vdev->is_cxl && i =3D=3D vdev->comp_reg_bar) + continue; + + pci_release_selected_regions(pdev, 1 << bar); } =20 list_for_each_entry_safe(dummy_res, tmp, @@ -960,6 +969,15 @@ static int vfio_pci_ioctl_get_info(struct vfio_pci_cor= e_device *vdev, if (vdev->reset_works) info.flags |=3D VFIO_DEVICE_FLAGS_RESET; =20 + if (vdev->is_cxl) { + ret =3D vfio_info_add_capability(&caps, &vdev->cxl_cap.header, + sizeof(vdev->cxl_cap)); + if (ret) + return ret; + + info.flags |=3D VFIO_DEVICE_FLAGS_CXL; + } + info.num_regions =3D VFIO_PCI_NUM_REGIONS + vdev->num_regions; info.num_irqs =3D VFIO_PCI_NUM_IRQS; =20 @@ -1752,14 +1770,21 @@ int vfio_pci_core_mmap(struct vfio_device *core_vde= v, struct vm_area_struct *vma * we need to request the region and the barmap tracks that. */ if (!vdev->barmap[index]) { + int bars; + + if (vdev->is_cxl && vdev->comp_reg_bar =3D=3D index) + bars =3D 0; + else + bars =3D 1 << index; + ret =3D pci_request_selected_regions(pdev, - 1 << index, "vfio-pci"); + bars, "vfio-pci"); if (ret) return ret; =20 vdev->barmap[index] =3D pci_iomap(pdev, index, 0); if (!vdev->barmap[index]) { - pci_release_selected_regions(pdev, 1 << index); + pci_release_selected_regions(pdev, bars); return -ENOMEM; } } diff --git a/drivers/vfio/pci/vfio_pci_rdwr.c b/drivers/vfio/pci/vfio_pci_r= dwr.c index 6192788c8ba3..057cd0c69f2a 100644 --- a/drivers/vfio/pci/vfio_pci_rdwr.c +++ b/drivers/vfio/pci/vfio_pci_rdwr.c @@ -201,19 +201,24 @@ EXPORT_SYMBOL_GPL(vfio_pci_core_do_io_rw); int vfio_pci_core_setup_barmap(struct vfio_pci_core_device *vdev, int bar) { struct pci_dev *pdev =3D vdev->pdev; - int ret; + int bars, ret; void __iomem *io; =20 if (vdev->barmap[bar]) return 0; =20 - ret =3D pci_request_selected_regions(pdev, 1 << bar, "vfio"); + if (vdev->is_cxl && vdev->comp_reg_bar =3D=3D bar) + bars =3D 0; + else + bars =3D 1 << bar; + + ret =3D pci_request_selected_regions(pdev, bars, "vfio"); if (ret) return ret; =20 io =3D pci_iomap(pdev, bar, 0); if (!io) { - pci_release_selected_regions(pdev, 1 << bar); + pci_release_selected_regions(pdev, bars); return -ENOMEM; } =20 diff --git a/include/linux/vfio_pci_core.h b/include/linux/vfio_pci_core.h index 8293910e0a96..0a354c7788b3 100644 --- a/include/linux/vfio_pci_core.h +++ b/include/linux/vfio_pci_core.h @@ -82,6 +82,9 @@ struct vfio_pci_core_device { bool needs_pm_restore:1; bool pm_intx_masked:1; bool pm_runtime_engaged:1; + bool is_cxl:1; + int comp_reg_bar; + struct vfio_device_info_cap_cxl cxl_cap; struct pci_saved_state *pci_saved_state; struct pci_saved_state *pm_save; int ioeventfds_nr; diff --git a/include/uapi/linux/vfio.h b/include/uapi/linux/vfio.h index 95be987d2ed5..0a9968cd6601 100644 --- a/include/uapi/linux/vfio.h +++ b/include/uapi/linux/vfio.h @@ -214,6 +214,7 @@ struct vfio_device_info { #define VFIO_DEVICE_FLAGS_FSL_MC (1 << 6) /* vfio-fsl-mc device */ #define VFIO_DEVICE_FLAGS_CAPS (1 << 7) /* Info supports caps */ #define VFIO_DEVICE_FLAGS_CDX (1 << 8) /* vfio-cdx device */ +#define VFIO_DEVICE_FLAGS_CXL (1 << 9) /* Device supports CXL */ __u32 num_regions; /* Max region index + 1 */ __u32 num_irqs; /* Max IRQ index + 1 */ __u32 cap_offset; /* Offset within info struct of first cap */ @@ -256,6 +257,15 @@ struct vfio_device_info_cap_pci_atomic_comp { __u32 reserved; }; =20 +#define VFIO_DEVICE_INFO_CAP_CXL 6 +struct vfio_device_info_cap_cxl { + struct vfio_info_cap_header header; + __u8 hdm_count; + __u8 hdm_reg_bar_index; + __u64 hdm_reg_size; + __u64 hdm_reg_offset; +}; + /** * VFIO_DEVICE_GET_REGION_INFO - _IOWR(VFIO_TYPE, VFIO_BASE + 8, * struct vfio_region_info) --=20 2.25.1 From nobody Sat Feb 7 12:25:21 2026 Received: from CH5PR02CU005.outbound.protection.outlook.com (mail-northcentralusazon11012008.outbound.protection.outlook.com [40.107.200.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BE26732AAAD; Tue, 9 Dec 2025 16:53:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.200.8 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1765299193; cv=fail; b=cJwGF8CmUT9/GOpeU9w6VGMqelpigv5Gdlj9kF9tqau0x7jWmTjcKd4/XPHDKb3lN3yJMjslhK/gORgwiCwUlhBssiRTwb15GOAaC4dA/4fo73MDHd2TA/rb5Tir96l8O5xyFArueDz+NhbVEuZ/+XFsR5ablomJo41enM9wwZw= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1765299193; c=relaxed/simple; bh=wPgvua/53TuIIz46ZN4ARi8jhA4VaiO+Re1mLw/zlq0=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=QQPg8nAFV2fU7UzRQo2Q3HEPy9+dQ41KJAxaq9W5h4VdIiBY+4f+y/LYunMHJeIMi5fyu7p7QiionJI4dZEOxDJ30TLHUDIeRhPnQIs61905D3x4SFw9/aXmSfsxWuImC6J69FnAed41w6M2Hgy87Yg94S4sBNInk6Qz+Y9Cwvw= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=qqPwZrrc; arc=fail smtp.client-ip=40.107.200.8 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="qqPwZrrc" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=ievxCybQJrg3lOQdaedW4y7P7bs6Tq22zWr12ra4StbbuevrEYDe1RIoB6Bt3b5ipNZfhN/Ay6DJpyZoa9GfXMplk+7AQMgzJH2Jnkp2rWmk0f4QPXt0HZelBjngt3Syyt7NdNubu6fgmSFwI2uSGheV4MD2u5V9Oid9YnnLQqVcfwKo+VNtfU4ijgMBtxc5rooFa2oRIDISv1+bGNBeDTCg4SnmwU1V6L9kp/0peX+NaTD8E17gaEoDNEFu7RO/tBJScmi4W2CIvBRY2IEHEtqseoxTJUr2Ysl550397S8NUxSFvGp32QKu9bNbwGUtXOHe3dc48HEyD/QMAqxgLA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=NefEiDMnn+AYgYqg3KznnZhXKJ03HqLn7MNFz1hKNjA=; b=ajxj9ff+o/+RMuPX9zPQcOY7LQSN7wupIi6jB5RTwEEJ8UCveu4hs7nWvoWDYNu2UtQhDx2ZQO+WVA93fXpwiM9cX3uMLry9A8EHRC0CVTqn95M3Z1rXbNU3eglEw9RyHsQcfHeF+vzXMDP5C17fd6vCMWM2pa2yQik1FU5mAr2GjR2iY0zgiam2ZEnTKHBpZV+DrR1TSUrafB3ldK/V5L0tNy8iyE9l08c405uKZXlg2rJ9kSneRoGjgm00m4B/2bHPIRTud9AB0IPlzc+ZwRXjc1+nMNhu9fTV0OEJNPs7yIJxneTY1r7SWx7HlhYw2dtE2EReMr4X6tMbDB6xhg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=NefEiDMnn+AYgYqg3KznnZhXKJ03HqLn7MNFz1hKNjA=; b=qqPwZrrcJsOJQjMsawD41ljmM7j7KMwyemHFuCtS2iZOzyvuzWJtdR2cG5Hik2+de26Hn7gQDct39AcqzKSdIik4BiHwIQflkaE1xveEioZ2Hbs3A2I+33VKg51gzolLpuUKita7G6b9nTunJXe/nWEcPQHXMUbBbN/2RXOlvO+EBqIBbVGUVdesMGM3MI8DXqC4IY9rOkl4+fR7xQ8a3HKJTYkRYi8sEUsL3X6ST8N3oEhWD0Cf/s8TQMU2KMG5U0ociVrSWWPHbcsBdagBIy+/ncPlqaAFLeBO4FkctoFreAzg3RraRjwk6L05SLMw+JLGEDSHoOZIU4zKHv364w== Received: from SJ0PR03CA0264.namprd03.prod.outlook.com (2603:10b6:a03:3a0::29) by SJ2PR12MB8883.namprd12.prod.outlook.com (2603:10b6:a03:538::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9388.14; Tue, 9 Dec 2025 16:53:01 +0000 Received: from SJ1PEPF00002316.namprd03.prod.outlook.com (2603:10b6:a03:3a0:cafe::f5) by SJ0PR03CA0264.outlook.office365.com (2603:10b6:a03:3a0::29) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9388.14 via Frontend Transport; Tue, 9 Dec 2025 16:52:25 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by SJ1PEPF00002316.mail.protection.outlook.com (10.167.242.170) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9388.8 via Frontend Transport; Tue, 9 Dec 2025 16:52:58 +0000 Received: from rnnvmail203.nvidia.com (10.129.68.9) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 9 Dec 2025 08:52:35 -0800 Received: from rnnvmail204.nvidia.com (10.129.68.6) by rnnvmail203.nvidia.com (10.129.68.9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 9 Dec 2025 08:52:35 -0800 Received: from nvidia-4028GR-scsim.nvidia.com (10.127.8.11) by mail.nvidia.com (10.129.68.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20 via Frontend Transport; Tue, 9 Dec 2025 08:52:28 -0800 From: To: , , , , , , , , , , , , , , , , , CC: , , , , , , , , Subject: [RFC v2 14/15] vfio/cxl: VFIO variant driver for QEMU CXL accel device Date: Tue, 9 Dec 2025 22:20:18 +0530 Message-ID: <20251209165019.2643142-15-mhonap@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20251209165019.2643142-1-mhonap@nvidia.com> References: <20251209165019.2643142-1-mhonap@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ1PEPF00002316:EE_|SJ2PR12MB8883:EE_ X-MS-Office365-Filtering-Correlation-Id: f5c4d4bd-326d-4d57-7ebe-08de3743687c X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|7416014|1800799024|376014|36860700013|921020; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?ptSsO52z3LXpC162LgwFm/dHA93CN4QLZplYyReGW26UiA2JcYFZi/o1jRIk?= =?us-ascii?Q?d8pmQOcvZyx3/O8qVLyZzD2HP2KFbfMi8z03lvlQyodD8Fsk4lXKTbkoIxVQ?= =?us-ascii?Q?4fsgMW2ApD+yAuctllxXTr81ccfyqHsTpEXV4bwLnMxLtJcaavkIipRJxhpr?= =?us-ascii?Q?q9bBbebIaJpsHWn7W1WiVUuXzuT2wfLpAmeZH9LReefQF+/zNQKpg8cOgPq6?= =?us-ascii?Q?3+Gi6pBXIpae520PRnt6jxsG6owvTi8EX13B0ZEBYyOiTQE3EgSVnLTlQMfY?= =?us-ascii?Q?pqOaj3nghl058Xg4fwjyg53XKEgeyPcBo1qAr5Mq+nzy/vBQShnir9oe7etM?= =?us-ascii?Q?9aW2qwgxEgiCOlW8nA6GnsHSMmSLoQdKfmoH4vjUMwUpQpzey2ghFj9+1tui?= =?us-ascii?Q?Xuy1uOHJnXLAAD6p08+YSWC/cnGRQg2rnuCYPnHR8exwfrP2kQWFR4f0TVoj?= =?us-ascii?Q?YsAa0a1NouwbTdMdnSSl6858VVN3bl8nlnVRMAYkVDnI5wth+Y73mZdUo0ZK?= =?us-ascii?Q?p626ygMimix8WwUPtZXFqX7LLUUW+LltpGK+GlmIyVAyz0aT6bcGqFuOdqbQ?= =?us-ascii?Q?ONImQFaSA8UllmWTcMmKuZAS1TSmjyQbUFUV2O5W9IPp6T0fdU27iEjJ6VcJ?= =?us-ascii?Q?RPPERDp9MDGyMpwrdYDfkAkL3XwmfetDfJ2Iy3y3X2Ur841w1ZFexOSD4tH/?= =?us-ascii?Q?IlzPaR9YYnVRK5fCkUKVH+yoledk+yw1JFdCbshIQSCeRfeYYoeBtDHD3g09?= =?us-ascii?Q?dlW+vE2Cd4/1zVtnH8fS1gEuZ0g0LZ3bPecW9GUOZKZNxTNKdM6TU8ofHbFM?= =?us-ascii?Q?BxQn/ylR+w54sUJ/bMb4wEURRbwqFywBPjj3fRUgt0rsfdHeuV8KQxqxb1WX?= =?us-ascii?Q?Hggowbem/1XUeGfwUOlHJdAC4jsCtkl246dJXJCB/Siui80ms8w8Jl9GQPLX?= =?us-ascii?Q?1VzRFkdFp51d1YaDHTUgr9M8A/EFdrIvwIOnUbfkzjjesRpNMkVT34VkiHDb?= =?us-ascii?Q?SBTFEhc9DCTYzNvdWJUdJp10UMErrUxAH7bvGBVaiL5210jgTTtBHLy6ShfS?= =?us-ascii?Q?U4dmtsVspS99cmpUYHeOqT+iETE86CGQUKMwrBaChiPgPh6mHWA/7I9Pkaua?= =?us-ascii?Q?p0P/xk2/E/lTvQ5rnt3RZxCHAWeYZAkudNE6UEkl3IT7ZHvwcjYLwSV2vzpH?= =?us-ascii?Q?SmD2dv8QHMfsHtXyQVli0RObHn4XXGVBwair86IQbm5D5O1RGuRXiLuR8DEH?= =?us-ascii?Q?yqo5fCEiDWfj8JivSfuWAlbW/Va+YTv3mTrtH6NJHflEeIXwASt5JVMSt6kF?= =?us-ascii?Q?8rfIGcerWc9SaLm5j/9xXUEC9KZRbUfglzwc9vJ5fnxGxN9/06RRBch2oeZr?= =?us-ascii?Q?J5A6jeIbqL4yEZXeXCBUZ5FiJXKRVQ3PIY1K5iR+VUj0YZljec+0hhlWS5rE?= =?us-ascii?Q?E95KGkx7sKgHTyQH7GwVrnD9XAccyxfFi+nbYN1dh8P2XyP+TFtEXkNO5lQw?= =?us-ascii?Q?XUZkcRHVMOdp52kSQu+zBLwgfVAeyfmWnLsv+ePwL3+3H+uyAcE4I7n+/ib4?= =?us-ascii?Q?+mbBfSoRG3AZjNtZtFUQ22sCZnd9Mw4d3ohZjNFK?= X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(82310400026)(7416014)(1800799024)(376014)(36860700013)(921020);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Dec 2025 16:52:58.7247 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: f5c4d4bd-326d-4d57-7ebe-08de3743687c X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: SJ1PEPF00002316.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ2PR12MB8883 Content-Type: text/plain; charset="utf-8" From: Manish Honap To demonstrate the VFIO CXL core, a VFIO variant driver for QEMU CXL accel device is introduced, so that people to test can try the patches. This patch is not meant to be merged. Co-developed-by: Zhi Wang Signed-off-by: Zhi Wang Signed-off-by: Manish Honap --- drivers/vfio/pci/Kconfig | 2 + drivers/vfio/pci/Makefile | 2 + drivers/vfio/pci/cxl-accel/Kconfig | 9 ++ drivers/vfio/pci/cxl-accel/Makefile | 4 + drivers/vfio/pci/cxl-accel/main.c | 143 ++++++++++++++++++++++++++++ 5 files changed, 160 insertions(+) create mode 100644 drivers/vfio/pci/cxl-accel/Kconfig create mode 100644 drivers/vfio/pci/cxl-accel/Makefile create mode 100644 drivers/vfio/pci/cxl-accel/main.c diff --git a/drivers/vfio/pci/Kconfig b/drivers/vfio/pci/Kconfig index 2f441d118f1c..441ded7ea035 100644 --- a/drivers/vfio/pci/Kconfig +++ b/drivers/vfio/pci/Kconfig @@ -77,4 +77,6 @@ source "drivers/vfio/pci/nvgrace-gpu/Kconfig" =20 source "drivers/vfio/pci/qat/Kconfig" =20 +source "drivers/vfio/pci/cxl-accel/Kconfig" + endmenu diff --git a/drivers/vfio/pci/Makefile b/drivers/vfio/pci/Makefile index 452b7387f9fb..1b81d75b8ef7 100644 --- a/drivers/vfio/pci/Makefile +++ b/drivers/vfio/pci/Makefile @@ -22,3 +22,5 @@ obj-$(CONFIG_VIRTIO_VFIO_PCI) +=3D virtio/ obj-$(CONFIG_NVGRACE_GPU_VFIO_PCI) +=3D nvgrace-gpu/ =20 obj-$(CONFIG_QAT_VFIO_PCI) +=3D qat/ + +obj-$(CONFIG_CXL_ACCEL_VFIO_PCI) +=3D cxl-accel/ diff --git a/drivers/vfio/pci/cxl-accel/Kconfig b/drivers/vfio/pci/cxl-acce= l/Kconfig new file mode 100644 index 000000000000..9a8884ded049 --- /dev/null +++ b/drivers/vfio/pci/cxl-accel/Kconfig @@ -0,0 +1,9 @@ +# SPDX-License-Identifier: GPL-2.0-only +config CXL_ACCEL_VFIO_PCI + tristate "VFIO support for the QEMU CXL accel device" + select VFIO_CXL_CORE + help + VFIO support for the CXL devices is needed for assigning the CXL + devices to userspace using KVM/qemu/etc. + + If you don't know what to do here, say N. diff --git a/drivers/vfio/pci/cxl-accel/Makefile b/drivers/vfio/pci/cxl-acc= el/Makefile new file mode 100644 index 000000000000..8d0e076f405f --- /dev/null +++ b/drivers/vfio/pci/cxl-accel/Makefile @@ -0,0 +1,4 @@ +# SPDX-License-Identifier: GPL-2.0-only + +obj-$(CONFIG_CXL_ACCEL_VFIO_PCI) +=3D cxl-accel-vfio-pci.o +cxl-accel-vfio-pci-y :=3D main.o diff --git a/drivers/vfio/pci/cxl-accel/main.c b/drivers/vfio/pci/cxl-accel= /main.c new file mode 100644 index 000000000000..3e5001ed5e2a --- /dev/null +++ b/drivers/vfio/pci/cxl-accel/main.c @@ -0,0 +1,143 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2025, NVIDIA CORPORATION & AFFILIATES. All rights reserved + */ + +#include +#include + +struct cxl_device { + struct vfio_pci_core_device core_device; +}; + +static int cxl_open_device(struct vfio_device *vdev) +{ + struct vfio_cxl_core_device *cxl =3D + container_of(vdev, struct vfio_cxl_core_device, pci_core.vdev); + struct vfio_cxl *cxl_core =3D cxl->cxl_core; + struct vfio_cxl_dev_info info =3D {0}; + int ret; + + /* Driver reports the device DPA and RAM size */ + info.dpa_res =3D DEFINE_RES_MEM(0, SZ_256M); + info.ram_res =3D DEFINE_RES_MEM_NAMED(0, SZ_256M, "ram"); + + /* Initialize the CXL device and enable the vfio-pci-core */ + ret =3D vfio_cxl_core_enable(cxl, &info); + if (ret) + return ret; + + vfio_cxl_core_finish_enable(cxl); + + cxl_core =3D cxl->cxl_core; + + /* No precommitted region, create one. */ + if (!cxl_core->region.region) { + /* + * Driver can choose to create cxl region at a certain time + * E.g. at driver initialization or later + */ + ret =3D vfio_cxl_core_create_cxl_region(cxl, SZ_256M); + if (ret) + goto fail_create_cxl_region; + } + + ret =3D vfio_cxl_core_register_cxl_region(cxl); + if (ret) + goto fail_register_cxl_region; + + return 0; + +fail_register_cxl_region: + if (cxl_core->region.region) + vfio_cxl_core_destroy_cxl_region(cxl); +fail_create_cxl_region: + vfio_cxl_core_disable(cxl); + return ret; +} + +static void cxl_close_device(struct vfio_device *vdev) +{ + struct vfio_cxl_core_device *cxl =3D + container_of(vdev, struct vfio_cxl_core_device, pci_core.vdev); + + vfio_cxl_core_unregister_cxl_region(cxl); + vfio_cxl_core_destroy_cxl_region(cxl); + vfio_cxl_core_close_device(vdev); +} + +static const struct vfio_device_ops cxl_core_ops =3D { + .name =3D "cxl-vfio-pci", + .init =3D vfio_pci_core_init_dev, + .release =3D vfio_pci_core_release_dev, + .open_device =3D cxl_open_device, + .close_device =3D cxl_close_device, + .ioctl =3D vfio_cxl_core_ioctl, + .device_feature =3D vfio_pci_core_ioctl_feature, + .read =3D vfio_cxl_core_read, + .write =3D vfio_cxl_core_write, + .mmap =3D vfio_pci_core_mmap, + .request =3D vfio_pci_core_request, + .match =3D vfio_pci_core_match, + .match_token_uuid =3D vfio_pci_core_match_token_uuid, + .bind_iommufd =3D vfio_iommufd_physical_bind, + .unbind_iommufd =3D vfio_iommufd_physical_unbind, + .attach_ioas =3D vfio_iommufd_physical_attach_ioas, + .detach_ioas =3D vfio_iommufd_physical_detach_ioas, +}; + +static int cxl_probe(struct pci_dev *pdev, + const struct pci_device_id *id) +{ + const struct vfio_device_ops *ops =3D &cxl_core_ops; + struct vfio_cxl_core_device *cxl_device; + int ret; + + cxl_device =3D vfio_alloc_device(vfio_cxl_core_device, pci_core.vdev, + &pdev->dev, ops); + if (IS_ERR(cxl_device)) + return PTR_ERR(cxl_device); + + dev_set_drvdata(&pdev->dev, &cxl_device->pci_core); + + ret =3D vfio_pci_core_register_device(&cxl_device->pci_core); + if (ret) + goto out_put_vdev; + + return ret; + +out_put_vdev: + vfio_put_device(&cxl_device->pci_core.vdev); + return ret; +} + +static void cxl_remove(struct pci_dev *pdev) +{ + struct vfio_pci_core_device *core_device =3D dev_get_drvdata(&pdev->dev); + + vfio_pci_core_unregister_device(core_device); + vfio_put_device(&core_device->vdev); +} + +static const struct pci_device_id cxl_vfio_pci_table[] =3D { + { PCI_DRIVER_OVERRIDE_DEVICE_VFIO(PCI_VENDOR_ID_INTEL, 0xd94) }, + {} +}; + +MODULE_DEVICE_TABLE(pci, cxl_vfio_pci_table); + +static struct pci_driver cxl_vfio_pci_driver =3D { + .name =3D KBUILD_MODNAME, + .id_table =3D cxl_vfio_pci_table, + .probe =3D cxl_probe, + .remove =3D cxl_remove, + .err_handler =3D &vfio_pci_core_err_handlers, + .driver_managed_dma =3D true, +}; + +module_pci_driver(cxl_vfio_pci_driver); + +MODULE_LICENSE("GPL"); +MODULE_AUTHOR("Zhi Wang "); +MODULE_DESCRIPTION("VFIO variant driver for QEMU CXL accel device"); +MODULE_IMPORT_NS("CXL"); --=20 2.25.1 From nobody Sat Feb 7 12:25:21 2026 Received: from MW6PR02CU001.outbound.protection.outlook.com (mail-westus2azon11012007.outbound.protection.outlook.com [52.101.48.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 47B6332FA2D; Tue, 9 Dec 2025 16:53:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.48.7 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1765299198; cv=fail; b=Ej221SF50yBVqR1i/sZLnjrU0FvqhqF1MEAiUqrvpK3muvEywn0grIaB+fJ2lDUDokmZmt0fpHh1lgtvpJDgysqWHDfi9O5vq3qm6uXEm/nyfYDvIwD9MK6S+tirNCZYBZf0LpbdC3K2qT5qMhJyA551UJKAHVJ0Nu/4f27tCps= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1765299198; c=relaxed/simple; bh=q7TT0WiKBIg/2HCITOZmfCyRYiZfFHh2clZZ/nSETGA=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=IxyQQqVWe08EEOlz1dnIMD2GYntV7L6mVt8pJJ2I4f3FDSIuC3JiRDsfCj0hJAC33I5vBxJVshc7AjU+/A9X2I5hJCzN9rAesIeb69escnoGSNM8ZQqOrxBYgRS9crGenM4HlawMcxDjbvLWPrZiO1dZbUPttfLNW3lYMe1N6hw= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=R/Vd3IZY; arc=fail smtp.client-ip=52.101.48.7 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="R/Vd3IZY" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=j38lMJnuQC3S2PSiDTQCsQ+1/OmkCTXT2Ouiu4qS2jUfvcfp/jYPaJy2kAl6wgvUeOKdbikOA7X/6iT6E7usK1A1T86yWmcuP4pHwvnvlNARq2jEKsIQoX6FIcgxsCB+eZ08HhcNTRUP8JdM8iRJTsaBu1J5orWbW4z+zWqCQr5b0geBlEY+KDLlRJZcrAXu4PB3W8IKJdonYsogPQjzEQ1glsXFPwDDjP1feTZ/u2MRSMOjM01x/8i/bXb44O2b8uMwjVIQbvehKUk10p4oHcqu7TNQmYZVUVzFOJKXZBicZinvZE0+G2RIJYN1BOKc99ZdYjLhqMFiw68I12/y0w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=H+wxjLkkEC1MkIDazcv7DRsiT2Cba9T35qc2AOF+czg=; b=scScSdt4TNvoqcssDVnkhvWz1zIOgQF+MSBmZxpwH3MyFSabXS9Pzs3P/Dpng44SiEgWUeI2vf5WV6do6oWLA3bnMz4aBa6k3f18PX4uthoEALBIN6RZ36kksGG37PhL75xd6E6+40vsgd9DVIrqobJ5IlMDnGexHCvDLfL8qs9MDARkACJlIxfuo3gbaS1Wd2CJg0AIiGj7eMBKcwE9cMPbBvL3ouhC3oQEVbxU+SFyzJc2V+ONEGBFg0BlnxZhoqYsKvUCzfqPUzzmudQhtUsT+nbq6ozoI1uqZBGzms5dKTMQWX9jhn1ykty1ZZIKnj3D3DF2S5dgpgVBuTidAQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=H+wxjLkkEC1MkIDazcv7DRsiT2Cba9T35qc2AOF+czg=; b=R/Vd3IZYgLuQzUmToqwlfJ3A2nhqXCEuW6AOgfkMLvjaHJFB643Qk2kF/EcB9zA03kay8IkaJDLCbq/fW5K7tZV8jY6m6h1xXm2/eaACIqUrmkvo3fVb1cZ+rnDrt7HVklyYIdwXnA2UZUhUQEhl1oJQEOgZZY+xzT9iWNP/N3ufpMK55GgrUhltF3pxHgPThKYT7hElfz4ub3qCh0P+nvcUOhVvo0QLYqqp07312wPoZvbSKBJOwtOjyD/KCXbzvAHtQklx5SJ4hsrWI+GscfOOhrjXsj4b4wleYV5YGgMxg87TPKXHEajQN7M3mZVP66hD4I82eWsh0WePUpfmYg== Received: from BN0PR03CA0035.namprd03.prod.outlook.com (2603:10b6:408:e7::10) by LV2PR12MB5990.namprd12.prod.outlook.com (2603:10b6:408:170::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9412.6; Tue, 9 Dec 2025 16:53:08 +0000 Received: from BN2PEPF000044A3.namprd02.prod.outlook.com (2603:10b6:408:e7:cafe::a9) by BN0PR03CA0035.outlook.office365.com (2603:10b6:408:e7::10) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9388.14 via Frontend Transport; Tue, 9 Dec 2025 16:52:42 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by BN2PEPF000044A3.mail.protection.outlook.com (10.167.243.154) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9412.4 via Frontend Transport; Tue, 9 Dec 2025 16:53:08 +0000 Received: from rnnvmail203.nvidia.com (10.129.68.9) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 9 Dec 2025 08:52:43 -0800 Received: from rnnvmail204.nvidia.com (10.129.68.6) by rnnvmail203.nvidia.com (10.129.68.9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 9 Dec 2025 08:52:42 -0800 Received: from nvidia-4028GR-scsim.nvidia.com (10.127.8.11) by mail.nvidia.com (10.129.68.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20 via Frontend Transport; Tue, 9 Dec 2025 08:52:35 -0800 From: To: , , , , , , , , , , , , , , , , , CC: , , , , , , , , Subject: [RFC v2 15/15] cxl/mem: Fix NULL pointer deference in memory device paths Date: Tue, 9 Dec 2025 22:20:19 +0530 Message-ID: <20251209165019.2643142-16-mhonap@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20251209165019.2643142-1-mhonap@nvidia.com> References: <20251209165019.2643142-1-mhonap@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN2PEPF000044A3:EE_|LV2PR12MB5990:EE_ X-MS-Office365-Filtering-Correlation-Id: 059a33e9-2a73-4d89-9669-08de37436e5b X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|82310400026|1800799024|36860700013|921020; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?8L58uRmshPkYNMbSBo/QAWeaG/ZmGTY/yfUeHswM9nBMaVAigFvCS2X9dd0Z?= =?us-ascii?Q?SgyHxmHgJkjJWhrKF9k6vFdGHqP4e0V1ywYyQ+B96n1d9MS4SSkSxxy8TmPn?= =?us-ascii?Q?aYRRaF+bBk8ORAim7kKWXq57HgA3ZucYqEDurmnow8Q0EsFfE9V+jmznF7/m?= =?us-ascii?Q?4cUnwcRZWi+buWgqg8WpMHLFNAJwaQ2Fr81iYLftUXDTBM6tM6ch/Tmx6eqK?= =?us-ascii?Q?ixyorP7Yj2nB/ElnozGOtJ8hSwXrQWWAOqsQvrULetH9J4iDh8adO7xEDixo?= =?us-ascii?Q?Gc4hA5aMcPCcP7VsdGB6T3FMjnHYL5odcu/Cs2E6eajmeSLfxq4Dxjt9iKmc?= =?us-ascii?Q?4F763btCgShflk72OXYpZXjAGwxzXbl8X4NBzKOQyMg95gGqLsRRo3FsTOPL?= =?us-ascii?Q?lIdJ6nYS6kvijE9oYWtdPAa9mn8WW/s4POvt/9js8AZwYbuaa6ERTxKmdV5G?= =?us-ascii?Q?jFOkM/7H/D1a0NdwDlsTW4IS0SrgTdWpTmPtzRSvl0DLXgjBauNshoh5WFqa?= =?us-ascii?Q?+JgW55+lAflnwlm0Ade9GmeWv9i7U8DGxGIeTdAucaIWdl6Ry5yHKiVs3F5s?= =?us-ascii?Q?qUxVO5imfa724qtK3ev9cB2mrVKmm4eUOlfE8tt00Qc0wq2fdIYLEOktbIuT?= =?us-ascii?Q?hvyoU6iJ/dckdf2zLC3pfaIFe60pbOhrQ7JjPM/kLhY0EwFIo9xPUTtMvH29?= =?us-ascii?Q?YWytzr59dOZUY3IHPYx3AeBkrmgJ9bMlhY6KjTI7O1kPpx4dUk8At0D0dhRN?= =?us-ascii?Q?nIW/C3TGUdpx0REVA5s4KOVngNaaZAoydnPwD2Ghd6HSlIlj7bz77yU8STTD?= =?us-ascii?Q?RuiCikKKXsSKZB0lQ+NrZaKZGO967eG6upwtx0pOahq8dwfgGuc4GNw5+ES9?= =?us-ascii?Q?RA2i7HOZjYwDmzS860YqnHVWkH4BMSLrO1aWvf0+CQRjEiVRHtchIL43uBoJ?= =?us-ascii?Q?66pqXMdeBZofZh71tHIPwxEMrvq+iO7k9AQtGDMiQ+j1CuhYlduBDIrHxRA2?= =?us-ascii?Q?VaaG5rGiAJ5PE21W/k2QhKE7GcWt3xokg85mVa+AelEKcZydbi0qDyuHANoC?= =?us-ascii?Q?QvmCYckqqkzDC/6/iWdRbH20V6qrKBF1uFCtKNh66a/mwMblTerGFjwI8bIS?= =?us-ascii?Q?7OFz7aKXMnUuqjPTLSYsdpdnEPd21FLzrIOUaFARjt8pRzOa8BVi6WxtKGvD?= =?us-ascii?Q?bomRRoaUOgU/JOCnp1B0wQu3DrEz+U5r/8jUfoKjZH1HaTgyXlRrwVRJUTon?= =?us-ascii?Q?pBcpQTgGp2l3QwSLCjC9Ro9+vqGVEfQjyO+28IV9uBt9SRKSQ8rfc5KbyjOm?= =?us-ascii?Q?LWOTzyipFhFAA5H1fAPMdNu0hVmCRC/CfTIsPpcbDLS1+ztL0mo9ulZ6ml8v?= =?us-ascii?Q?outDt+NdSx6NkzybIDzkAL3jHLHJ+kmo9M+C0ca+jU39XPBtZVItEHNtSZBN?= =?us-ascii?Q?CKQ12IH6eDKDlm5+bUpH3n/edfni6GbuVfNe7PsGgGZPTHmhJTGBy+kISorc?= =?us-ascii?Q?C2+smYzUt6/3Qr+4hnMRTZuOA1lOk6GstCTy5LhnfWmMSEhmtyJOqgUXSd/A?= =?us-ascii?Q?LCi3D0ZMVYDYRrUjlv8BWeuDVuQqSv5E0p1bz0rd?= X-Forefront-Antispam-Report: CIP:216.228.117.160;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge1.nvidia.com;CAT:NONE;SFS:(13230040)(7416014)(376014)(82310400026)(1800799024)(36860700013)(921020);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Dec 2025 16:53:08.5187 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 059a33e9-2a73-4d89-9669-08de37436e5b X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN2PEPF000044A3.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: LV2PR12MB5990 Content-Type: text/plain; charset="utf-8" From: Manish Honap Add NULL pointer validation in CXL memory device code paths that can be triggered during error scenarios and device cleanup operations. Two crash scenarios have been identified during VFIO-CXL testing: 1. __cxlmd_free() can be called with a NULL cxlmd pointer during error handling paths in device probe/remove sequences. This leads to a NULL pointer dereference when accessing cxlmd->cxlds. 2. cxl_memdev_has_poison_cmd() can receive a cxlmd where the conversion to cxl_memdev_state via to_cxl_memdev_state() returns NULL. This occurs when the device state hasn't been fully initialized yet, causing a crash when test_bit() attempts to access mds->poison.enabled_cmds. Fix by adding defensive NULL checks: - In __cxlmd_free(), return early if cxlmd is NULL to avoid dereferencing an invalid pointer - In cxl_memdev_has_poison_cmd(), validate mds before accessing the poison.enabled_cmds bitmap Signed-off-by: Manish Honap --- drivers/cxl/core/memdev.c | 2 +- drivers/cxl/mem.c | 3 +++ 2 files changed, 4 insertions(+), 1 deletion(-) diff --git a/drivers/cxl/core/memdev.c b/drivers/cxl/core/memdev.c index d281843fb2f4..eb694203a259 100644 --- a/drivers/cxl/core/memdev.c +++ b/drivers/cxl/core/memdev.c @@ -207,7 +207,7 @@ bool cxl_memdev_has_poison_cmd(struct cxl_memdev *cxlmd, { struct cxl_memdev_state *mds =3D to_cxl_memdev_state(cxlmd->cxlds); =20 - return test_bit(cmd, mds->poison.enabled_cmds); + return (mds) ? test_bit(cmd, mds->poison.enabled_cmds) : false; } =20 static int cxl_get_poison_by_memdev(struct cxl_memdev *cxlmd) diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c index d91d08d25bc4..d5a942ba97b2 100644 --- a/drivers/cxl/mem.c +++ b/drivers/cxl/mem.c @@ -188,6 +188,9 @@ static int cxl_mem_probe(struct device *dev) =20 static void __cxlmd_free(struct cxl_memdev *cxlmd) { + if (!cxlmd) + return; + cxlmd->cxlds->cxlmd =3D NULL; put_device(&cxlmd->dev); kfree(cxlmd); --=20 2.25.1