From nobody Wed Apr 1 22:20:18 2026 Received: from DM1PR04CU001.outbound.protection.outlook.com (mail-centralusazon11010021.outbound.protection.outlook.com [52.101.61.21]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BCAAE3D2FE6; Wed, 1 Apr 2026 14:41:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.61.21 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775054510; cv=fail; b=pg1tgAFjuKnL/BseqAI9QqVIkMKE/NzHQKeYD7lYihRUVnOkz5Jo0Ueqt/TFUPWYAOa6qPvQL08GnjvBIPKXatm47XBQVSXQ7YEFU2llbnMq8/6guSdpSrWNR9FwZmXV5lklPpdsTSH2tcrwUQr4JY8whRkJckiAJBFcWqq8dS0= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775054510; c=relaxed/simple; bh=mVMg95XwTw/+yFfJagVJQ+a/t5fkvoeblIEu5hR5Gqk=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=TBW4npl/fn0gGJzGeqj70VRxALsMyA50jK744l63Ts0SaBYbFiUL+wwCG3QPlY9SCuk5phZucgOGpSyw95NZ7H3Ys2VvT1O2XEZh7T8sDjqqrIFIs79xK0ZiGaVR7Yye9+tfRabO3RVV1NKHEWhu+HKaPr73Zq7lxOL/EQvv/6U= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=CGm5TarK; arc=fail smtp.client-ip=52.101.61.21 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="CGm5TarK" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=pa52cpiC9M7tyMIg5bzT0qCTXSJqTrUJHDfvPnY50BsgFymiz2q8TeUPIOD0QVUa5En8J7xanZ8mKQlr4VK1JPx3Omesgt4TftHm+paScYooejEqJIswEznhEB5cR5UVhCqb/RLRO+BveIwuVbGF7KewZY82wvxCY4ofLFMu0pJUB4rD27BTjkksUZgFPhru99w28v6Z2Xc3ggJ6I8Mm6CSIc3fUGebjzXUWJLAetyLfBpWvwhL3B8E7n1ReufxHpQU46NvHopIZ5QekVN39sp7mVXGwXNcwe7iKHf+cc9Tf8n+972wv2WEjyAeflqoYnlxonMuT49iY4j9bHRd5hQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=JYMDfyTVFoYoqhwWe9W8q3jn0cIi/6OHJy/usPMEbo4=; b=oh5nDPw6PYFlWCb4xjfvXRFojaRnD7JTssYBQjLiF6e+o4akE/L7HmrHM7xlU0lwE0NNbVQBZLWScU37wbdXDvQQlPmIeFLmmSlPVytwHopXuMHBQ7VBCjJ3x5qwH5uwE5owbaoeTrm4aTSjXjpzdQmzS02NTKgxL8FdmIp3SgvmzPmebE1A/j8MHnxLfm4654VcpVwJP6geL8f9mBgDRbAKwWvTH9/a3K7MdNwRVLzfgngB/bK1whYEwLjtwANlC5mhRdBQQPex2fJii2L7A2RQq3J/N/K0JRsnrVeyB4iOUuHRDvyhPrc61i7maFYz7Zk0MDPqWrqKBi57HZupcA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=JYMDfyTVFoYoqhwWe9W8q3jn0cIi/6OHJy/usPMEbo4=; b=CGm5TarK/NsVmARBEtuOFQBoD7+1KTzfL6Ok9uhauIzKqTsoZNL/6JcANa10vEtwsgK29g82Stn7ZSRhxsbveKyktkklXA+B/r8JLGAlB2MYLUUhDPFa+lvCoB5GoK3fIE5aG7YjuIqfr6lsYXkP9nDqU9tdLkdikn0nPUbUvgHruwND4HUw9J+wuc2RcYCAPclgYC2Jp7Y1RJpsWU+JAHVtx/iL/CwYxmZCbas/uWY80ii37pyelup6jafIwmiMhyfVn2JPv5zeYLShr7cvqzX0bIiAGXEKTO8V0dcI+bF5OQZkmyGEXhhJeTMi0nGTLWlwWxFaLJo7p5NZcpaHxQ== Received: from IA4P221CA0008.NAMP221.PROD.OUTLOOK.COM (2603:10b6:208:559::6) by MW3PR12MB4347.namprd12.prod.outlook.com (2603:10b6:303:2e::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9769.17; Wed, 1 Apr 2026 14:41:42 +0000 Received: from BN1PEPF00005FFD.namprd05.prod.outlook.com (2603:10b6:208:559:cafe::75) by IA4P221CA0008.outlook.office365.com (2603:10b6:208:559::6) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9745.31 via Frontend Transport; Wed, 1 Apr 2026 14:41:46 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by BN1PEPF00005FFD.mail.protection.outlook.com (10.167.243.229) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9769.17 via Frontend Transport; Wed, 1 Apr 2026 14:41:41 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Wed, 1 Apr 2026 07:41:14 -0700 Received: from nvidia-4028GR-scsim.nvidia.com (10.126.231.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Wed, 1 Apr 2026 07:41:07 -0700 From: To: , , , , , , , , , , , , , , , CC: , , , , , , , , , Subject: [PATCH v2 13/20] vfio/cxl: CXL region management support Date: Wed, 1 Apr 2026 20:09:10 +0530 Message-ID: <20260401143917.108413-14-mhonap@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20260401143917.108413-1-mhonap@nvidia.com> References: <20260401143917.108413-1-mhonap@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: rnnvmail202.nvidia.com (10.129.68.7) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN1PEPF00005FFD:EE_|MW3PR12MB4347:EE_ X-MS-Office365-Filtering-Correlation-Id: f3438c3e-8cf7-44b1-9d0e-08de8ffcc9eb X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|82310400026|36860700016|1800799024|921020|18002099003|22082099003|56012099003; X-Microsoft-Antispam-Message-Info: zZ2+GoEyl65487yVP83WU/5ez+qYy6WGcTaQQ8EQAd4RSPZFFmEIZ9NJdjh3zjs1gUOCvnJrvXjCQLlnwhDPtavFwQFu6WG/tgKNL/R1brff7Tlr+NOMT84XFO5TSfV7cF5+u7IXFZHTPaJvzXMHFfiy40r3gtqf07vHevrfgqqap+3QbODEu0hoYo/WAwy5Zf+Eixv+z31moQFr7gQtksyOVX5FEjFGMSVxirhm7odO/Oz8iu4gmWhOwkbT4OTtf8DJc6UbV+TbrWJGGHUWy5To4OGOuMtmAd52efX6Y2DsV5Duy/kV1vX58FlOfVpVxusT3AOBw1P0IlykuLbveFsC9X79/vappia9wpqpyTU0Qh5DlfpTRlrZqh5REXBW05pHU9TDrHU3LtF38ql5F0jRxzeCWkSks0CZngUpt9izcqRNIqOol7FFqME/K0LwaPhSgcP5g11ZnbCjcxfR+4wdSR3u8XIxEPlAQzVPkYNR+UcEfFvmxqi884qYwF/fh4eYhKMJjlvFXtU8VBgtQDqjc9vPX9A3Y1nTUY19Y8VZqbGKRC8yESv7hc5V2FreZ1LKLRu1iu+cYaerGySvQWdGqd4n2D39G3enzRRpqDkdbNEhvhcqLFTGm2O1haJB/7hxQYJmMs/bdh49+CeSBfJeJCreJpXvUCAoc08tSnSpwJ4VY5mZQRpcCWpAJnF4svZRU0KuMLjqGR2JajVjafCr+QoVV8Epr9uy0kA2PBITLI1+KkfVPGurdZ6iqfJybNAumUZyBIvvsMj5XNNVPDjy7ELPbg2LDaMxlZqeZEJHrj2xtmB8xzNyPywpK8I/ X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(7416014)(376014)(82310400026)(36860700016)(1800799024)(921020)(18002099003)(22082099003)(56012099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: F8DxtPOJvxH3t6bzgbMe4iggXMnqKvMmRm6NQm2Eqw13GTY3Wq2gq7kkDV4rSyKEEa0qDinbwAHtIeWK/9ZZbL1y9iS7D2YIZ65vTeE7QPnDTy4iEJsUlil2UedWoh6ivdbozKmWPpIYNINhpb03nV/X9WyXDM8/YES48iDFaaWqeWUk/D7T5HuiJ92/HV7Vl2JkwcHtrTGB0olkzqvHMKu5+7XMeDrL9/wN9G1+sZ1bJufby/lVFFL044GPTNMNDgwQL2CBxAGE+aBemxwYJCWtZOrJ+hBl1U0GeGSnnHAuhR/0DQf7yw6Kf1p6T+jdAi8s7rnUI9HlSm8VbwNW26aYzDlrheVJ0h00Mdm68LCmK4FQKDoUvH+/r1gfefzv96yJ0lDvbE/OxrNlisZhKekEs26B95CENhDhkSfQX3mMW98eV+qg3tCC6GABAnXQ X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Apr 2026 14:41:41.1482 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: f3438c3e-8cf7-44b1-9d0e-08de8ffcc9eb X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN1PEPF00005FFD.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW3PR12MB4347 Content-Type: text/plain; charset="utf-8" From: Manish Honap Region Management makes use of APIs provided by CXL_CORE as below: CREATE_REGION flow: 1. Validate request (size, decoder availability) 2. Allocate HPA via cxl_get_hpa_freespace() 3. Allocate DPA via cxl_request_dpa() 4. Create region via cxl_create_region() - commits HDM decoder 5. Get HPA range via cxl_get_region_range() DESTROY_REGION flow: 1. Detach decoder via cxl_decoder_detach() 2. Free DPA via cxl_dpa_free() 3. Release root decoder via cxl_put_root_decoder() Use DEFINE_FREE scope helpers so error paths unwind cleanly. Signed-off-by: Manish Honap --- drivers/vfio/pci/cxl/vfio_cxl_core.c | 119 +++++++++++++++++++++++++++ drivers/vfio/pci/cxl/vfio_cxl_priv.h | 8 ++ 2 files changed, 127 insertions(+) diff --git a/drivers/vfio/pci/cxl/vfio_cxl_core.c b/drivers/vfio/pci/cxl/vf= io_cxl_core.c index 02755265d530..30b365b91903 100644 --- a/drivers/vfio/pci/cxl/vfio_cxl_core.c +++ b/drivers/vfio/pci/cxl/vfio_cxl_core.c @@ -21,6 +21,13 @@ #include "../vfio_pci_priv.h" #include "vfio_cxl_priv.h" =20 +/* + * Scope-based cleanup wrappers for the CXL resource APIs + */ +DEFINE_FREE(cxl_put_root_decoder, struct cxl_root_decoder *, if (!IS_ERR_O= R_NULL(_T)) cxl_put_root_decoder(_T)) +DEFINE_FREE(cxl_dpa_free, struct cxl_endpoint_decoder *, if (!IS_ERR_OR_NU= LL(_T)) cxl_dpa_free(_T)) +DEFINE_FREE(cxl_unregister_region, struct cxl_region *, if (!IS_ERR_OR_NUL= L(_T)) cxl_unregister_region(_T)) + /* * vfio_cxl_create_device_state - Allocate and validate CXL device state * @@ -165,6 +172,112 @@ static int vfio_cxl_setup_regs(struct vfio_pci_core_d= evice *vdev, return ret; } =20 +int vfio_cxl_create_cxl_region(struct vfio_pci_cxl_state *cxl, + resource_size_t size) +{ + resource_size_t max_size; + + WARN_ON(cxl->precommitted); + + struct cxl_root_decoder *cxlrd __free(cxl_put_root_decoder) =3D + cxl_get_hpa_freespace(cxl->cxlmd, 1, + CXL_DECODER_F_RAM | CXL_DECODER_F_TYPE2, + &max_size); + if (IS_ERR(cxlrd)) + return PTR_ERR(cxlrd); + + /* Insufficient HPA space; cxlrd freed automatically by __free() */ + if (max_size < size) + return -ENOSPC; + + struct cxl_endpoint_decoder *cxled __free(cxl_dpa_free) =3D + cxl_request_dpa(cxl->cxlmd, CXL_PARTMODE_RAM, size); + if (IS_ERR(cxled)) + return PTR_ERR(cxled); + + struct cxl_region *region __free(cxl_unregister_region) =3D + cxl_create_region(cxlrd, &cxled, 1); + if (IS_ERR(region)) + return PTR_ERR(region); + + /* All operations succeeded; transfer ownership to cxl state */ + cxl->cxlrd =3D no_free_ptr(cxlrd); + cxl->cxled =3D no_free_ptr(cxled); + cxl->region =3D no_free_ptr(region); + + return 0; +} + +void vfio_cxl_destroy_cxl_region(struct vfio_pci_cxl_state *cxl) +{ + if (!cxl->region) + return; + + cxl_unregister_region(cxl->region); + cxl->region =3D NULL; + + if (!cxl->precommitted) { + cxl_dpa_free(cxl->cxled); + cxl_put_root_decoder(cxl->cxlrd); + } + + cxl->cxled =3D NULL; + cxl->cxlrd =3D NULL; +} + +static int vfio_cxl_create_region_helper(struct vfio_pci_core_device *vdev, + struct vfio_pci_cxl_state *cxl, + resource_size_t capacity) +{ + struct pci_dev *pdev =3D vdev->pdev; + struct range range; + int ret; + + if (cxl->precommitted) { + struct cxl_endpoint_decoder *cxled; + struct cxl_region *region; + + cxled =3D cxl_get_committed_decoder(cxl->cxlmd, ®ion); + if (IS_ERR(cxled)) + return PTR_ERR(cxled); + cxl->cxled =3D cxled; + cxl->region =3D region; + } else { + ret =3D vfio_cxl_create_cxl_region(cxl, capacity); + if (ret) + return ret; + } + + if (!cxl->region) { + pci_err(pdev, "Failed to create CXL region\n"); + ret =3D -ENODEV; + goto failed; + } + + ret =3D cxl_get_region_range(cxl->region, &range); + if (ret) + goto failed; + + cxl->region_hpa =3D range.start; + cxl->region_size =3D range_len(&range); + + pci_dbg(pdev, "CXL region: HPA 0x%llx size %lu MB\n", + cxl->region_hpa, cxl->region_size >> 20); + + return 0; + +failed: + if (cxl->region) { + cxl_unregister_region(cxl->region); + cxl->region =3D NULL; + } + + cxl->cxled =3D NULL; + cxl->cxlrd =3D NULL; + + return ret; +} + static int vfio_cxl_create_memdev(struct vfio_pci_cxl_state *cxl, resource_size_t capacity) { @@ -279,6 +392,7 @@ void vfio_pci_cxl_detect_and_init(struct vfio_pci_core_= device *vdev) goto regs_failed; } =20 + cxl->precommitted =3D true; cxl->dpa_size =3D capacity; =20 pci_dbg(pdev, "Device capacity: %llu MB\n", capacity >> 20); @@ -289,6 +403,10 @@ void vfio_pci_cxl_detect_and_init(struct vfio_pci_core= _device *vdev) goto regs_failed; } =20 + ret =3D vfio_cxl_create_region_helper(vdev, cxl, capacity); + if (ret) + goto regs_failed; + /* * Register probing succeeded. Assign vdev->cxl now so that * all subsequent helpers can access state via vdev->cxl. @@ -314,6 +432,7 @@ void vfio_pci_cxl_cleanup(struct vfio_pci_core_device *= vdev) return; =20 vfio_cxl_clean_virt_regs(cxl); + vfio_cxl_destroy_cxl_region(cxl); } =20 MODULE_IMPORT_NS("CXL"); diff --git a/drivers/vfio/pci/cxl/vfio_cxl_priv.h b/drivers/vfio/pci/cxl/vf= io_cxl_priv.h index 6359ad260bde..72a0d7d7e183 100644 --- a/drivers/vfio/pci/cxl/vfio_cxl_priv.h +++ b/drivers/vfio/pci/cxl/vfio_cxl_priv.h @@ -17,6 +17,10 @@ struct vfio_pci_cxl_state { struct cxl_memdev *cxlmd; struct cxl_root_decoder *cxlrd; struct cxl_endpoint_decoder *cxled; + struct cxl_region *region; + resource_size_t region_hpa; + size_t region_size; + void *region_vaddr; resource_size_t hdm_reg_offset; size_t hdm_reg_size; resource_size_t comp_reg_offset; @@ -28,6 +32,7 @@ struct vfio_pci_cxl_state { u8 hdm_count; u8 comp_reg_bar; bool cache_capable; + bool precommitted; }; =20 /* Register access sizes */ @@ -87,5 +92,8 @@ void vfio_cxl_reinit_comp_regs(struct vfio_pci_cxl_state = *cxl); resource_size_t vfio_cxl_read_committed_decoder_size(struct vfio_pci_core_device *vdev, struct vfio_pci_cxl_state *cxl); +int vfio_cxl_create_cxl_region(struct vfio_pci_cxl_state *cxl, + resource_size_t size); +void vfio_cxl_destroy_cxl_region(struct vfio_pci_cxl_state *cxl); =20 #endif /* __LINUX_VFIO_CXL_PRIV_H */ --=20 2.25.1