From nobody Fri Aug 28 00:12:35 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nvidia.com); dmarc=pass(p=reject dis=none) header.from=nvidia.com ARC-Seal: i=2; a=rsa-sha256; t=1786626565; cv=pass; d=zohomail.com; s=zohoarc; b=bIn2izYQhmqAQ1pBgvAHBEcHB/c+1NjbRC5BjfG8XqGkeXYQunsKZ2NRwYHV/vd8xblBFJZmWVInm7Vgv02XumQfuejP8ywsGxpVJXWoGmZTnfG00Da7vzPbWPtia6+8YBam0LUFsmQb031m2/6/4iSPyEE8iY0/V0rMYmLHmAw= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1786626565; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=qZZtbEB7F5TZsDCRBUKxcst8CtHvtk1q2iU4Jd8gXak=; b=DLfVs8zaKyMazmfBqU3aFx7/XiN+mSb8LxqqDbBT4YNtkQdixudYR8eitPhxWEwAP9ZRJ5QI2k2G8YjmjUnBZDgTCiCmXNDu5xVKlt5MJfsRMaR2Da1wrwRt/4r+0L+PBy4WF4ielvbjCe+Ni8CFn4chuCyuPfPaTDFVwb/CI8Y= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nvidia.com); dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1786626565383874.9511180384678; Thu, 13 Aug 2026 06:09:25 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wuV9r-0006K8-PI; Thu, 13 Aug 2026 09:07:43 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wuV9d-0006Ji-De; Thu, 13 Aug 2026 09:07:30 -0400 Received: from mail-eastus2azlp170100001.outbound.protection.outlook.com ([2a01:111:f403:c110::1] helo=BN1PR04CU002.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wuV9a-0005F3-B8; Thu, 13 Aug 2026 09:07:28 -0400 Received: from IA1PR12MB6332.namprd12.prod.outlook.com (2603:10b6:208:3e2::13) by PH0PR12MB8149.namprd12.prod.outlook.com (2603:10b6:510:297::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.315.15; Thu, 13 Aug 2026 13:07:18 +0000 Received: from MW4PR03CA0285.namprd03.prod.outlook.com (2603:10b6:303:b5::20) by IA1PR12MB6332.namprd12.prod.outlook.com (2603:10b6:208:3e2::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.315.14; Thu, 13 Aug 2026 13:07:13 +0000 Received: from CO1PEPF00012E82.namprd03.prod.outlook.com (2603:10b6:303:b5:cafe::95) by MW4PR03CA0285.outlook.office365.com (2603:10b6:303:b5::20) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.315.15 via Frontend Transport; Thu, 13 Aug 2026 13:07:13 +0000 Received: from mail.nvidia.com (216.228.117.160) by CO1PEPF00012E82.mail.protection.outlook.com (10.167.249.57) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.339.3 via Frontend Transport; Thu, 13 Aug 2026 13:07:12 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.46; Thu, 13 Aug 2026 06:06:50 -0700 Received: from nvidia-4028GR-scsim.nvidia.com (10.126.230.37) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Thu, 13 Aug 2026 06:06:43 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=VovlAFgDaoyUgFbeVznnxh31MfpnHyNjdG4gbigVUcGNDFJXjrrDDkiZ0BxcbMDX/ithfDV17iUJe1fJd1rGYMd8kCxM0KiWkRdgRwOng0T4gSDYT9rNpn8qfXT2wul/K2GK7HI+CaOQvO7VZ4JmrlfmOcx3XDvZPDg3DpvY2P6O/XvMGJh3Zpi4J7ukL6aUFo770mK1ZzyrOIyh2sawXLYDam8M9wUfXTJ7XjBgB7P5zNKUblCRkCcRhPYlpjQF/Qw9DtRAdtf2C5UZjKD5wLe40V+ZAqiewTa0imJbXG/8/5/RF5Wn8Si4kpyh8ELb1dfX3gpSP4ZHgDIEbmCsNQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=qZZtbEB7F5TZsDCRBUKxcst8CtHvtk1q2iU4Jd8gXak=; b=NqU5Oah/nWKntr0ZdB0JNAqoxQlOViFJkVhfHCcC1FjdFu2SxR1ADxUToikA2MiTs7+IOeJTgIjCUnd8kxQgJguZBzTgRLzB/LlE5hlwL5XVbPTnfWXvIapvTHb1aTOfbUNpYRBWb2AyeTIo9zS2v9Wl8Wacf+OCG+3a66LYP3eor4DZM1bLcf6vDdExr+z+/R0jrIitxRrxN8pjx4yHMCPkWnIi9uWvLI9VOCZ0pH4DpgasloDg4Hq6JMVpq6bPCcKqk5EZSfFtyN7jVI/hIBpsC+28BGIdO4hdHDzhvN0Ek/k1qnlNmm3hG94NfiVyuECFls/UFqkEOHGQeEOh2w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=shazbot.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=qZZtbEB7F5TZsDCRBUKxcst8CtHvtk1q2iU4Jd8gXak=; b=h3Sd4jccGRUhuHPbnDAMm4Fuj4/a8N/IOyuLBiyhrdAGSUeWHErmDsFPMGcwkLSo3BKrMZbd1gOOjYMQxpwty8C5z1XPobZSyMiSTnfCbQ43AEyN1RmoM4wYDRppFjDQqRJsxb5dFRHAY2euO5X3UXth0KN5SDM0kE0tLJtO9jQbkf9PeqHubsKo1mXZU6soQoTGMZ4vO6knykJghsZl1AqzE2E7stI5gDbEH9WbmaJO6H46yzHyaZ3QPelB0rhDVbcXDUAuz8P6PKw5NHAvNTeY2EiAxsjeSv/oBfEdAQKFwLcDzVo/jwcWo7aOq4t98KamDz2I8b+bI2HPbC/ruQ== X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C From: To: , , , , , , , , , , , , , , , CC: , , , , , Subject: [PATCH 01/10] linux-headers: Update vfio.h for CXL Type-2 passthrough Date: Thu, 13 Aug 2026 18:36:14 +0530 Message-ID: <20260813130623.2499506-2-mhonap@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20260813130623.2499506-1-mhonap@nvidia.com> References: <20260813130623.2499506-1-mhonap@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.126.230.37] X-ClientProxiedBy: rnnvmail203.nvidia.com (10.129.68.9) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1PEPF00012E82:EE_|IA1PR12MB6332:EE_|PH0PR12MB8149:EE_ X-MS-Office365-Filtering-Correlation-Id: 500c7a61-168e-43fa-d316-08def93bca94 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|23010399003|82310400026|36860700016|1800799024|376014|7416014|56012099006|18002099003|22082099003|10067099003|11063799006|921020; X-Microsoft-Antispam-Message-Info: UNLj35yTS21LKBunvRble1Ygx+kSFxerEf1kyB9KWC4gyKYjmf6dBWiZ0ZT7RcpFno7XbavW4cvkSXYSAFCoMcZjTR+zZXJpF387IimkHsmEyorozVLw8Wbw2OjvLxrk2O82NwT6ts52EJSQiUtCZUdeNUFIqQHKR+QuZgafOH/Jz91UeMIskmw2Mjfpi5EMGQYD4YFUQg8xQkTlZTgTTG4kAdJueYmJQiTi7zy+zEA4G/ba1Ngsg9s7U96aGhbQHfEPuhPU0nC96N4zTxvFI4lV1ydyOQPH8dkSjGnzLcSxngyTHV7/qmUHxxk6bfpLIW7LrdS/IQCkUofrF/nvpmtaE62TnzRj6OYyW2FQtRvVFSlLd+zjugzFgmwpOcEFVinNAJjB4/dz2iApj0Xq34lRWviXPxgY2vE7a6wA+5cGqD0D4n2yKXYsCd9ETGewi7cv8He0NWiUYDoA7P3R8nvtrQ3QixvcgixdQ+GbEdZ9crZYEDZHsDI2iHZu5tJ+4Q1uMYmhg9T0A01ntodP+Qanli3+Ko7P9f67upTzHYNEsq4456YFNDzFoax+0cyGXDvqBHRqG8J0dqrha5NB1fMkf7sWEzTA8A0MMTLC8p8vBDC6rdB3bP9uQcxMKvzsLPww8mXVR1McZ43vx6JZvF+7b8jlMsDJyxVtCYpAjNr6pDsPISZgUe+4ljoqhs9F1svWy5RYuLclUG1ONKH3/wzeWjPq/tEa9txt+0vfkS8WkJouwetxdNX75n/ip+v7 X-Forefront-Antispam-Report: CIP:216.228.117.160; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge1.nvidia.com; CAT:NONE; SFS:(13230040)(23010399003)(82310400026)(36860700016)(1800799024)(376014)(7416014)(56012099006)(18002099003)(22082099003)(10067099003)(11063799006)(921020); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 6/+WaJcDfltzyFs/u1XwyDu4eNR8qp7OSQJpM9Cg0t53bMDVimHSjSB++zpAoN/GyCfp8ytdpgfBKsx8T62sGhT7RZSkRvyqSkUcxJCDGWztAFJc8O5tNRR7wCXZTvaoLyiPGClJkAeXxCwbY0fK3QgucSC3u5cRJLdTTYr2RuXuCdbj+nZvmkfycwBocA5Hb/R437oAzUyoETNV2kzKllatuCM/5ZFFnDOd9uShsU6WAXRTIJV7zlLOs2eOGZR8pK6m3JjDvY7rxzKPrfq+EL7MLRzW71j3/+cTy/Kq6IXAWVCgEH+Di3AT31Yw9bTqeu53FTn+SBy2eGvklKnX7fVgEs0eRrqelBKu1TJ+TjOWt3ewPnup38FfbxgEHb/Zhddnvhd8/1kKCeLhmx6vQexz9bTDt55qB+ohZjoyQF2G+QvK4y04tLp68DWGqjqW X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Aug 2026 13:07:12.8951 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 500c7a61-168e-43fa-d316-08def93bca94 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.160]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-CrossTenant-AuthSource: CO1PEPF00012E82.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR12MB8149 Received-SPF: permerror client-ip=2a01:111:f403:c110::1; envelope-from=mhonap@nvidia.com; helo=BN1PR04CU002.outbound.protection.outlook.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.759, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FORGED_SPF_HELO=1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_PASS=-0.001, SPF_NONE=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @Nvidia.com) X-ZM-MESSAGEID: 1786626566998158500 Content-Type: text/plain; charset="utf-8" From: Manish Honap Sync the vfio.h additions from the kernel vfio-cxl series: the VFIO_DEVICE_FLAGS_CXL device flag, the VFIO_REGION_TYPE_CXL region type with the HDM memory and component-register sub-types, and the component-register geometry capability that reports the BAR and offset of the trapped HDM decoder block. The capability ID is provisional until the kernel series lands, at which point this becomes a scripted linux-headers update. Signed-off-by: Manish Honap --- linux-headers/linux/vfio.h | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/linux-headers/linux/vfio.h b/linux-headers/linux/vfio.h index f3282b8e86..f3e6c94b00 100644 --- a/linux-headers/linux/vfio.h +++ b/linux-headers/linux/vfio.h @@ -215,6 +215,7 @@ struct vfio_device_info { #define VFIO_DEVICE_FLAGS_FSL_MC (1 << 6) /* vfio-fsl-mc device */ #define VFIO_DEVICE_FLAGS_CAPS (1 << 7) /* Info supports caps */ #define VFIO_DEVICE_FLAGS_CDX (1 << 8) /* vfio-cdx device */ +#define VFIO_DEVICE_FLAGS_CXL (1 << 9) /* vfio-cxl device */ __u32 num_regions; /* Max region index + 1 */ __u32 num_irqs; /* Max IRQ index + 1 */ __u32 cap_offset; /* Offset within info struct of first cap */ @@ -345,6 +346,7 @@ struct vfio_region_info_cap_type { #define VFIO_REGION_TYPE_GFX (1) #define VFIO_REGION_TYPE_CCW (2) #define VFIO_REGION_TYPE_MIGRATION_DEPRECATED (3) +#define VFIO_REGION_TYPE_CXL (4) =20 /* sub-types for VFIO_REGION_TYPE_PCI_* */ =20 @@ -373,6 +375,12 @@ struct vfio_region_info_cap_type { /* sub-types for VFIO_REGION_TYPE_GFX */ #define VFIO_REGION_SUBTYPE_GFX_EDID (1) =20 +/* sub-types for VFIO_REGION_TYPE_CXL */ +/* CXL.mem HDM memory window of a Type-2 device, mmap-able */ +#define VFIO_REGION_SUBTYPE_CXL_MEM (1) +/* CXL HDM decoder registers, trapped so the guest programs a GPA it never= owns */ +#define VFIO_REGION_SUBTYPE_CXL_COMP_REGS (2) + /** * struct vfio_region_gfx_edid - EDID region layout. * @@ -497,6 +505,20 @@ struct vfio_region_info_cap_nvlink2_lnkspd { __u32 __pad; }; =20 +/* + * Geometry of a CXL Type-2 device's HDM decoder registers, so a VMM can p= lace + * the trapped component register window where the guest expects it. The c= ap ID + * is provisional pending an upstream allocation. + */ +#define VFIO_REGION_INFO_CAP_CXL_COMP_REGS 6 + +struct vfio_region_info_cap_cxl_comp_regs { + struct vfio_info_cap_header header; + __u32 bar; + __u32 __resv; + __aligned_u64 offset; +}; + /** * VFIO_DEVICE_GET_IRQ_INFO - _IOWR(VFIO_TYPE, VFIO_BASE + 9, * struct vfio_irq_info) --=20 2.25.1 From nobody Fri Aug 28 00:12:35 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nvidia.com); dmarc=pass(p=reject dis=none) header.from=nvidia.com ARC-Seal: i=2; a=rsa-sha256; t=1786626510; cv=pass; d=zohomail.com; s=zohoarc; b=Gue+Z3nKaI1syOCHEkHAhy9ehUovly7vLoDpXFzp1WEKD3YtPrsjYEUk55U+Gd8qCH9HzDVdxTIZh7TTGNfkSt/ZUaBYCmOfkb/XRLnYVu6B6wemSNDFRmQCBCf9qYcyba8AdZ6nrvJ4rbtaq9vYU1/UbrBdn8vaiUI5TYMqAKo= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1786626510; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=dkas+geyN3YOn8nI1EGbea1yVInp1OTKw3lunlAlQZY=; b=juYK6QE56qtY6Abqb4wfto1lZafPXPW1wxHiyU2y7fzsR7NumDiwZTeb72eDVOnbxQg/J83RqufDbUcO6YEZNH8ndavMpmGG4PI3G2MEprhOq/ZcEr4l+7D/aIGZFIR0l2iOPjUBb2yJYKrFb18wPv+i4KM3wPm9K09507asWb4= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nvidia.com); dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1786626510748511.5445309426058; Thu, 13 Aug 2026 06:08:30 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wuV9z-0006Ml-5k; Thu, 13 Aug 2026 09:07:51 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wuV9k-0006Jx-Qp; Thu, 13 Aug 2026 09:07:38 -0400 Received: from mail-westcentralusazlp170100005.outbound.protection.outlook.com ([2a01:111:f403:c112::5] helo=CY7PR03CU001.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wuV9j-0005GY-6Q; Thu, 13 Aug 2026 09:07:36 -0400 Received: from MN0PR12MB6319.namprd12.prod.outlook.com (2603:10b6:208:3c0::11) by PH8PR12MB6676.namprd12.prod.outlook.com (2603:10b6:510:1c3::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.315.11; Thu, 13 Aug 2026 13:07:23 +0000 Received: from MW4PR03CA0287.namprd03.prod.outlook.com (2603:10b6:303:b5::22) by MN0PR12MB6319.namprd12.prod.outlook.com (2603:10b6:208:3c0::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.315.14; Thu, 13 Aug 2026 13:07:19 +0000 Received: from CO1PEPF00012E82.namprd03.prod.outlook.com (2603:10b6:303:b5:cafe::73) by MW4PR03CA0287.outlook.office365.com (2603:10b6:303:b5::22) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.315.14 via Frontend Transport; Thu, 13 Aug 2026 13:07:18 +0000 Received: from mail.nvidia.com (216.228.117.160) by CO1PEPF00012E82.mail.protection.outlook.com (10.167.249.57) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.339.3 via Frontend Transport; Thu, 13 Aug 2026 13:07:17 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.46; Thu, 13 Aug 2026 06:06:56 -0700 Received: from nvidia-4028GR-scsim.nvidia.com (10.126.230.37) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Thu, 13 Aug 2026 06:06:50 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=B9XHMkJMIYTHFBer/K/25NHZNKST2OzxTXnBSjRw+MuaKZvaH6z7nyIVup86Da+1Z+FRqnuWO4WlnGKXB2J9Zewiv7A5GFqBnuycz0Nr4fU5waX9WdhZGP+wS8t78FQXFN51SvxJBxV7nQhkoZ15oFByPr31tuaNOncK/rbTFOCjwEXkXdC0E86T+1/YPc1mBEp+zi6Wxe6UUwmVtqdIWq7H43VfSTd1HXhh3LelwsnVTUkNOMwe8MuF1MzH1+uFdV93oofjs5s/m1gbK8RV+zryPYdBJbzXBmRHh3AmEmBJkxMAVtU6ATHg9iZwdPfbBJygEUjypiWs0EzObzfPWQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=dkas+geyN3YOn8nI1EGbea1yVInp1OTKw3lunlAlQZY=; b=a2wvrABRm64Uew4+4haa6X3mQXyOkXqYtm4am9Rqn3bOs2d+EfqdN61zdUKRU8bmgOGlKaT7yUNLBw3h7hv+3FS8JJg18q0bLYjHMcycDEh8s+TsirrQ9gVAPLr/WTt8XLHeb0KnvhqTTL7bP4GiVVIB6d2XYmL0vlP0+t2COM5UG2ocfT0vDuneBofABMIiPINzXeJ1uRz6ahRZ3Fl31uyu0cwHNXpM3/habxxSQdopSz5UQbxanuuBeJz4QiOdzyQr2VbdC2JNifQI4ilfzv2ZTBfkohzeV7YtQuPuATBJhDwpm8ETTYbCQnAuHyiSrFdV3kBN0nodI2jnH0drCw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=shazbot.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=dkas+geyN3YOn8nI1EGbea1yVInp1OTKw3lunlAlQZY=; b=gLOOXX7uV4ejgBzhYfkB5pT3yIiDNPBCKkwATV12LtRJgBo/CxL5Yn1qWNDkqYvOnVcZMiLorIy3I4FLOq2VR7Ct9LdnwObxAjq+OW/7DqMlm7uDEqOCHohZc7cZQV1pk4mcJOTZvAu1GeqzXFtffvgT1K1SCXD1GVAUjixdt0NzBKwkVZwERtzoWNkuCANecd+MqduHOeR/QFq792WfK8YDLWuJPf/gYkBOAlDiPOM9HOWqhmXaaWOguakz1KaWm9Cw1YqoZFuXfe0tI7PEIfs0IAI0MFEe+7Q56aZngTiHlxHPjx3a99gWctEN158gH+/XifhwwI119RdeGSWGpw== X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C From: To: , , , , , , , , , , , , , , , CC: , , , , , Subject: [PATCH 02/10] hw/vfio/region: Add vfio_region_setup_with_ops() Date: Thu, 13 Aug 2026 18:36:15 +0530 Message-ID: <20260813130623.2499506-3-mhonap@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20260813130623.2499506-1-mhonap@nvidia.com> References: <20260813130623.2499506-1-mhonap@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.126.230.37] X-ClientProxiedBy: rnnvmail203.nvidia.com (10.129.68.9) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1PEPF00012E82:EE_|MN0PR12MB6319:EE_|PH8PR12MB6676:EE_ X-MS-Office365-Filtering-Correlation-Id: 68942098-1615-4c16-7271-08def93bcd99 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|82310400026|376014|7416014|23010399003|1800799024|36860700016|18002099003|22082099003|11063799006|56012099006|10067099003|921020; X-Microsoft-Antispam-Message-Info: bdmSIa/2fl+EGwQncSqQwtNtDYgQAtKcsdBzQyxc45H329gsbK+yaNfST6lFWnbJ/+P42Lpf+uHbSkBWY77+zfVDnRVbiUAzhejTQuuuaGqZIK/SqBuaNTLDB97DzFufP5q44twIghRHBxr+LH/3ilw7zLsb4Z1yhD+qPUg9Z0ovwqOGYu6L5ipdSAgkrEI2O/SFQbmpUeIMfRaEPHp0MJieSConzPGdvQWW+Dg2AYPI26SjpBRvnw7zDWl+tA3ooVpOFft1OXlk7V+KQEWUwGhoNNuy53gaNGKW5omcuy9bn5K7u+SW684WEYKFBeDbpm2fdgYaUjXaaQl9XMwQAeG+d7TBB9Nal0N5zO/vOUrAKV7s6+H+NoYueOIl5HCcT+tVFAOBiLeaWx4/lQ8ecaxp6TIsdQH2DOAVX1OcehzN+HqppjyLRoQbVhgWTToun57LtFpS+eqDX+V3MVcvyZeQCex88+h3U3xmoJyigNEGnYjfEn4YAYAV7WwfiKh9bIBfBCSmQu+4BhBO8Mdp0BAlHTjf7XsUnxttuO0BpauMqSTAKCJ75B0N1/y56FO3+BhlgoBIeT3CRYg2dHJgnfNbea5nYWTEbqvMgQoevFbxZ5KMTr29TXCQWiNEFIa88RBfQpcQn3lffNEs32yrQu+J8P+td0x3D3gGJlrSYEqesZb9nuRu0WpX9QN/8Sqk8jua02wkf6yL1CkQ7S78LhuMFsEBDZRxrB3KJGbHzbZg92BXwWSXdV/15H24nWFi X-Forefront-Antispam-Report: CIP:216.228.117.160; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge1.nvidia.com; CAT:NONE; SFS:(13230040)(82310400026)(376014)(7416014)(23010399003)(1800799024)(36860700016)(18002099003)(22082099003)(11063799006)(56012099006)(10067099003)(921020); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: yojOo1AExh8+8kQlSgsWK7hRmPSonX9zLSW7InQkACfeyIVBYW5UpjemlMmJQlK3diXrZXUPlWXVK5rE4KGmKTEfPi6KgHYFJQ4u7fJzAsIDYsDVJSZzbEBDC10SYJOdl31G+0cbDLYkLxVhK8awejJfvE8bFVxVcdBSH0aN8Ay1qJpwCTmXd0wnpHOuW28nNJqqLvkuaMwo1loGgFj44NET0JK43wXsMKPkqtK79AxbeN3LZDba7A6fHrcOuap9MthtaR3eAnts1PbYzGNsy08xWMmo+PsX76jDa7fv+lWxv8abQWA5Pw5j+kYP94LVYcnQW5jFwHv2XhNTYPT15910D5uyebehIjliAct47XvvwqJ+g9eRiwgxAjIGrKWHS8xRSW9BEGBt9GUOshZbx6JIr4PxClBxhLUpristku67pZ3yFMdA9UciZFMnqI4O X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Aug 2026 13:07:17.9823 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 68942098-1615-4c16-7271-08def93bcd99 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.160]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-CrossTenant-AuthSource: CO1PEPF00012E82.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH8PR12MB6676 Received-SPF: permerror client-ip=2a01:111:f403:c112::5; envelope-from=mhonap@nvidia.com; helo=CY7PR03CU001.outbound.protection.outlook.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.759, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FORGED_SPF_HELO=1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_PASS=-0.001, SPF_NONE=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @Nvidia.com) X-ZM-MESSAGEID: 1786626513615158500 Content-Type: text/plain; charset="utf-8" From: Manish Honap A CXL Type-2 device traps its HDM decoder block, so that region needs custom MemoryRegionOps rather than the pass-through default. Split the setup body out and let a caller supply the ops; NULL keeps the existing behaviour. Signed-off-by: Manish Honap --- hw/vfio/region.c | 27 ++++++++++++++++++++++++--- hw/vfio/vfio-region.h | 3 +++ 2 files changed, 27 insertions(+), 3 deletions(-) diff --git a/hw/vfio/region.c b/hw/vfio/region.c index 54ad11a6c8..21b54978b5 100644 --- a/hw/vfio/region.c +++ b/hw/vfio/region.c @@ -228,8 +228,9 @@ static int vfio_setup_region_sparse_mmaps(VFIORegion *r= egion, return 0; } =20 -int vfio_region_setup(Object *obj, VFIODevice *vbasedev, VFIORegion *regio= n, - int index, const char *name, Error **errp) +static int vfio_region_do_setup(Object *obj, VFIODevice *vbasedev, + VFIORegion *region, int index, const char = *name, + const MemoryRegionOps *ops, Error **errp) { struct vfio_region_info *info =3D NULL; int ret; @@ -249,7 +250,7 @@ int vfio_region_setup(Object *obj, VFIODevice *vbasedev= , VFIORegion *region, =20 if (region->size) { region->mem =3D g_new0(MemoryRegion, 1); - memory_region_init_io(region->mem, obj, &vfio_region_ops, + memory_region_init_io(region->mem, obj, ops, region, name, region->size); =20 if (!vbasedev->no_mmap && @@ -273,6 +274,26 @@ int vfio_region_setup(Object *obj, VFIODevice *vbasede= v, VFIORegion *region, return 0; } =20 +int vfio_region_setup(Object *obj, VFIODevice *vbasedev, VFIORegion *regio= n, + int index, const char *name, Error **errp) +{ + return vfio_region_do_setup(obj, vbasedev, region, index, name, + &vfio_region_ops, errp); +} + +/* + * Like vfio_region_setup() but traps the region through @ops instead of t= he + * default pass-through, so a caller can intercept accesses (the CXL HDM + * decoder block). A NULL @ops keeps the default. + */ +int vfio_region_setup_with_ops(Object *obj, VFIODevice *vbasedev, + VFIORegion *region, int index, const char *= name, + const MemoryRegionOps *ops, Error **errp) +{ + return vfio_region_do_setup(obj, vbasedev, region, index, name, + ops ? ops : &vfio_region_ops, errp); +} + static void vfio_subregion_unmap(VFIORegion *region, int index) { trace_vfio_region_unmap(memory_region_name(®ion->mmaps[index].mem), diff --git a/hw/vfio/vfio-region.h b/hw/vfio/vfio-region.h index 58b236f113..8d5699013a 100644 --- a/hw/vfio/vfio-region.h +++ b/hw/vfio/vfio-region.h @@ -39,6 +39,9 @@ uint64_t vfio_region_read(void *opaque, hwaddr addr, unsigned size); int vfio_region_setup(Object *obj, VFIODevice *vbasedev, VFIORegion *regio= n, int index, const char *name, Error **errp); +int vfio_region_setup_with_ops(Object *obj, VFIODevice *vbasedev, + VFIORegion *region, int index, const char *= name, + const MemoryRegionOps *ops, Error **errp); int vfio_region_mmap(VFIORegion *region); void vfio_region_mmaps_set_enabled(VFIORegion *region, bool enabled); void vfio_region_exit(VFIORegion *region); --=20 2.25.1 From nobody Fri Aug 28 00:12:35 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nvidia.com); dmarc=pass(p=reject dis=none) header.from=nvidia.com ARC-Seal: i=2; a=rsa-sha256; t=1786626597; cv=pass; d=zohomail.com; s=zohoarc; b=i1L3Hq2QgFWNAaj9lMcZJIN2b8oWTV9nIMFl84vaQSGbq18G4be9Kb8ktQI1aZuOMUrPWGmfXg6qObg99iPYDgE6PJ+DCxnwIg4X0sE+rOzBNObC5hrXCBMwTiAG4TzQ4HfmRjGKTsvjSnzw0Mcu2WWWc65L5AhZdWfS4t49Pt4= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1786626597; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=No3TEfslhxb/ptd5SDrD580aDCFbvvsVzptvRIcEYGg=; b=EBg/fsof87TKNUZR9urFIJjRDKJFZ7rlPeFKMWA82JsnZPlgNU3SZ/C8+Hxyiv8OLVS2n2pgVwvc7I1xKkbKWl1w5aYVtKEN3ExC0jy2N15a+cCNwOFWdj8gxby6mmTT2IHT9KPznX5lpuUyvg8AQLhBGbDdAi3EuyD0C9/MwkQ= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nvidia.com); dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1786626597528849.8925452512142; Thu, 13 Aug 2026 06:09:57 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wuVA2-0006OI-VE; Thu, 13 Aug 2026 09:07:54 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wuV9s-0006Ki-11; Thu, 13 Aug 2026 09:07:45 -0400 Received: from mail-westus3azlp170110003.outbound.protection.outlook.com ([2a01:111:f403:c107::3] helo=PH0PR06CU001.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wuV9p-0005HS-0A; Thu, 13 Aug 2026 09:07:43 -0400 Received: from IA0PR12MB8326.namprd12.prod.outlook.com (2603:10b6:208:40d::7) by MW6PR12MB8834.namprd12.prod.outlook.com (2603:10b6:303:23c::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.315.13; Thu, 13 Aug 2026 13:07:29 +0000 Received: from MN2PR20CA0066.namprd20.prod.outlook.com (2603:10b6:208:235::35) by IA0PR12MB8326.namprd12.prod.outlook.com (2603:10b6:208:40d::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.315.14; Thu, 13 Aug 2026 13:07:27 +0000 Received: from BN2PEPF000044A2.namprd02.prod.outlook.com (2603:10b6:208:235:cafe::5d) by MN2PR20CA0066.outlook.office365.com (2603:10b6:208:235::35) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.315.14 via Frontend Transport; Thu, 13 Aug 2026 13:07:27 +0000 Received: from mail.nvidia.com (216.228.117.161) by BN2PEPF000044A2.mail.protection.outlook.com (10.167.243.153) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.339.3 via Frontend Transport; Thu, 13 Aug 2026 13:07:26 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.46; Thu, 13 Aug 2026 06:07:02 -0700 Received: from nvidia-4028GR-scsim.nvidia.com (10.126.230.37) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Thu, 13 Aug 2026 06:06:56 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=aG11FkBFD3azR0cb2rlMErSR9ABoNKp3QIn0oFG8j4MSqTSyJUdmhl8WSVcAEXjME2Lw8dAnV8/0MJL2YCKpA2re4rsmxSNF1ecSr26RysMafz55K5HRDTNfPt8XgwP1GAXEgpHMgvYpDkiNMvBRLBd6qNeNn8ayMhCyi1i3YQmHVGYVac8HSYgCFsiOB7qzeUlaKWh/lkpKCw7ePa59plWAJ76+xW3m3iFzOHKrh+DpFGQ2wmEWoRCpMciAh/EDkBmHRwE1H5VFJfttHDFs5km302Nt9Z4sMPxG2a8zCAExBhDNeCBD/iQi9HSk/tQfPPw7Mv/D/JSchSEQJ3Khow== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=No3TEfslhxb/ptd5SDrD580aDCFbvvsVzptvRIcEYGg=; b=s7GZRm1n5blUxx7S519IG5WgkMo5gwU9TuLBlcg3eAfxLmCzRLqx1wTL3DsAEj8VP24LBdwlvak4+yj51jhyPNWRFpkbPZZk81iKMEqPKkleJs4ZJMWwjcA6+bpH3BDA2f5/S/AI+icKV3R1ecKAQAjPpiW9SvH+vREXHyzPuMKPJ/e/k2wv5LCTUdFaeFR964rZsgxZSmntdZ1yPiBPLve4mlkYdpqTAkJ7OeGUFPCpOhIOsv6zGyCHNvxeDOCmvotOpWE4Mt4qAjrwrKnDy3uSKbTuYLX3noGIZK0ZWFBdNf2XqQjzUb0Gafw8BcWz5Mci2lx6Jbgop0sH6cUDzA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=shazbot.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=No3TEfslhxb/ptd5SDrD580aDCFbvvsVzptvRIcEYGg=; b=GguvhXsyHfHBDyLVT12V5RSgWEThD1Vs7pmeM9ajWnmuadqOUb4asJT4gjoAnqur+svla3zYsDWA7DOiMzjWVDin2mAqS8CeJZL2tE0wbO6zDPm36FUOV0Ukzs74TE91ea8zGYYUP1Rj8B64hN+jB4U+SX8Uu5QzmY8Ik6lT2X0e8cnNm9JXM2L9j4137mmIijTmwfxnEszVK3DKrXAaTF9ap9Dqvvddy6RUNbcHOcQ5k50+cueWgKukkX6u1x0b3kyLbaEpIWs75zlCaE8nTrgo1shp27oNH+KMKGrJNqPRWIGrE1+qHs7752TJsC84GT4WeqjIZfhs8J4YderN1w== X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C From: To: , , , , , , , , , , , , , , , CC: , , , , , Subject: [PATCH 03/10] hw/vfio/pci: Detect a CXL Type-2 device and read its geometry Date: Thu, 13 Aug 2026 18:36:16 +0530 Message-ID: <20260813130623.2499506-4-mhonap@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20260813130623.2499506-1-mhonap@nvidia.com> References: <20260813130623.2499506-1-mhonap@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.126.230.37] X-ClientProxiedBy: rnnvmail203.nvidia.com (10.129.68.9) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN2PEPF000044A2:EE_|IA0PR12MB8326:EE_|MW6PR12MB8834:EE_ X-MS-Office365-Filtering-Correlation-Id: f868ba7f-6bb4-4f22-0db4-08def93bd2fd X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|7416014|376014|82310400026|23010399003|1800799024|36860700016|11063799006|56012099006|10067099003|6133799003|921020|18002099003|22082099003; X-Microsoft-Antispam-Message-Info: yC+svBWuowOCJeMiQc2Bxpp0aGVtkBVZjGV3KZBMeHLKWeAt8Q6jDntA0mMuIvcGSlD5E7OjDYNGeabBTwQV1fALgWd1U07EV8KEy11cfrEP19B6G2fvYO/0/Gn+h9q1tsdNIaURGfqpjNwNczi7YHVTXvfuMwYSxIS8/lijYFq/m8uRmWoBa5qycBShyjqgy/ntTbEoJr0HSH30FTOgzSmVdz3s8s3JC1c2qEbFxOxIH1K0KW1hj5qFmDf0i0fu/R6m+HqFVaLo/uMo7btPzZLVhKxY5aN99+TNStWVI46G3kR7Fm7Cwzu5sFST1FO1JBVq8nFmWaYqp4lhocTsvxFqdj66gVHoLtzMwH+usspiR/hRFBwYOttcr1IgXF4t/zssk6qGeWuN27OKO71x+PUELHWvMp+tQvsmJgkiQWlE4wqWRaz3nPX/1LZtdhRIC7zJNrH8FcUkioLei+RtENo4nBffsxjqy3xi3jf5M3KvWbSBDmkugxEa/UJC1zzbPnCULMyR6WFFko5VXotG2tfwqxs+mT/xc3Cs3S9xO8K7Cs4I3AHG59krjEHSMgDq6oEVtaBpUB8X21HT+SgVGNEpPvXO8CBTVHN3Pfsms/5BjIpBCRBP/5oQ9NpzBLX0Wv0CQeGnu2LDPnU4Rq8pzaE7sNLssRAu/YUvYjjoFnPgG9IgiU3kcymUPe4/i1tPayx9/Zd++sOJGyn8WJVDS/7PvMe9Nnp/46q7g65R9wyi/5ScKhU60u8sKN8Dw+mG X-Forefront-Antispam-Report: CIP:216.228.117.161; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge2.nvidia.com; CAT:NONE; SFS:(13230040)(7416014)(376014)(82310400026)(23010399003)(1800799024)(36860700016)(11063799006)(56012099006)(10067099003)(6133799003)(921020)(18002099003)(22082099003); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: /yimBYkeBsoRHRusXQlQL/Q5//+24yEZ2tGZw97mIb0BmAWNHl2ZYY9QyWdr46WhjXJHFlYB4uS9uIT+mpfznWbbh2An7+nyJdVB5R493Evr7VfVnE0caMk3jyP82GHVMtQZ9w+Kmnjuc63opxkYIagOOvichjb9DbhEAZGtr+/AAtQ4fkdEdXMzpB2ExW1mQXu91sA2KeaL45LXXIioQpbgQkOsBuF23icIlzE2g1WudufIfQqLj7S9PuDu0bQ3eZ/f31iXcA3vKs93q0gKA7uSM7+vUdDjitTD4DCNInZXo0Mi4Pu7FV/XKQ9PFYAJuH0g8H6bT8IMbjXrNJ/2iycRMpa2p4FacGyBcLJ00AbHo8lvzgpN1dClIQjHrEyC2O1/61ktZ+jteX4ZBfCt6MzCZf2YWNUot0o+wbXR/0olEEr239rCFZuSiuo+I4UJ X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Aug 2026 13:07:26.9592 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: f868ba7f-6bb4-4f22-0db4-08def93bd2fd X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.161]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-CrossTenant-AuthSource: BN2PEPF000044A2.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW6PR12MB8834 Received-SPF: permerror client-ip=2a01:111:f403:c107::3; envelope-from=mhonap@nvidia.com; helo=PH0PR06CU001.outbound.protection.outlook.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.759, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_NONE=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @Nvidia.com) X-ZM-MESSAGEID: 1786626599248158500 Content-Type: text/plain; charset="utf-8" From: Manish Honap The kernel marks a passed-through CXL Type-2 device with a device flag and exposes two regions: the HDM memory (host physical) and the trapped HDM decoder register block. Read the flag, locate both regions by type, and take the component BAR and block offset from the geometry capability. Realize only records this; later patches build the guest mapping on top. Signed-off-by: Manish Honap --- hw/vfio/pci.c | 68 +++++++++++++++++++++++++++++++++++++++++++++++++++ hw/vfio/pci.h | 15 ++++++++++++ 2 files changed, 83 insertions(+) diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c index 428ab2f069..157786df72 100644 --- a/hw/vfio/pci.c +++ b/hw/vfio/pci.c @@ -3570,6 +3570,60 @@ bool vfio_pci_interrupt_setup(VFIOPCIDevice *vdev, E= rror **errp) return true; } =20 +/* + * Learn the CXL geometry the kernel reports: the HPA-backed HDM memory re= gion + * and the trapped HDM decoder block (which BAR carries it and at what off= set). + * A non-CXL device leaves cxl.enabled false and takes no CXL paths. + */ +static bool vfio_cxl_setup(VFIOPCIDevice *vdev, Error **errp) +{ + VFIODevice *vbasedev =3D &vdev->vbasedev; + VFIOCXL *cxl =3D &vdev->cxl; + struct vfio_region_info *mem_info =3D NULL, *comp_info =3D NULL; + struct vfio_region_info_cap_cxl_comp_regs *cap; + struct vfio_info_cap_header *hdr; + + if (!(vbasedev->flags & VFIO_DEVICE_FLAGS_CXL)) { + return true; + } + + if (vfio_device_get_region_info_type(vbasedev, VFIO_REGION_TYPE_CXL, + VFIO_REGION_SUBTYPE_CXL_MEM, + &mem_info)) { + error_setg(errp, "vfio-cxl: %s: CXL memory region not found", + vbasedev->name); + return false; + } + + if (vfio_device_get_region_info_type(vbasedev, VFIO_REGION_TYPE_CXL, + VFIO_REGION_SUBTYPE_CXL_COMP_REGS, + &comp_info)) { + error_setg(errp, + "vfio-cxl: %s: CXL component-register region not found", + vbasedev->name); + return false; + } + + hdr =3D vfio_get_region_info_cap(comp_info, + VFIO_REGION_INFO_CAP_CXL_COMP_REGS); + if (!hdr) { + error_setg(errp, + "vfio-cxl: %s: component-register geometry not reported= ", + vbasedev->name); + return false; + } + cap =3D container_of(hdr, struct vfio_region_info_cap_cxl_comp_regs, h= eader); + + cxl->mem_region_index =3D mem_info->index; + cxl->comp_regs_region_index =3D comp_info->index; + cxl->dpa_size =3D mem_info->size; + cxl->comp_bar =3D cap->bar; + cxl->hdm_offset =3D cap->offset; + cxl->enabled =3D true; + + return true; +} + static void vfio_pci_realize(PCIDevice *pdev, Error **errp) { ERRP_GUARD(); @@ -3700,6 +3754,20 @@ static void vfio_pci_realize(PCIDevice *pdev, Error = **errp) } } =20 + if (!vfio_cxl_setup(vdev, errp)) { + /* + * vfio_migration_realize() above installed a migration blocker in= auto + * mode (generic vfio-pci exposes no migration ops). out_deregiste= r does + * not remove it, so a rejected CXL setup would leave VM migration + * blocked until QEMU restarts. Drop it here, under the same + * failover-pair condition used to install it. + */ + if (!pdev->failover_pair_id) { + vfio_migration_exit(vbasedev); + } + goto out_deregister; + } + vfio_pci_register_err_notifier(vdev); vfio_pci_register_req_notifier(vdev); vfio_setup_resetfn_quirk(vdev); diff --git a/hw/vfio/pci.h b/hw/vfio/pci.h index c9ab949870..7fdd695704 100644 --- a/hw/vfio/pci.h +++ b/hw/vfio/pci.h @@ -122,10 +122,25 @@ typedef struct VFIOMSIXInfo { =20 OBJECT_DECLARE_SIMPLE_TYPE(VFIOPCIDevice, VFIO_PCI_DEVICE) =20 +/* + * State for a CXL Type-2 device. The kernel owns the host physical placem= ent + * of the device memory; QEMU only maps it at the guest physical address t= he + * guest commits into its endpoint HDM decoder. + */ +typedef struct VFIOCXL { + bool enabled; + uint32_t mem_region_index; /* HPA-backed HDM memory VFIO region = */ + uint32_t comp_regs_region_index; /* trapped HDM decoder register block= */ + uint32_t comp_bar; /* component BAR carrying that block = */ + uint64_t hdm_offset; /* block offset within the component = BAR */ + uint64_t dpa_size; /* size of the HDM memory region */ +} VFIOCXL; + struct VFIOPCIDevice { PCIDevice parent_obj; =20 VFIODevice vbasedev; + VFIOCXL cxl; VFIOINTx intx; unsigned int config_size; uint8_t *emulated_config_bits; /* QEMU emulated bits, little-endian */ --=20 2.25.1 From nobody Fri Aug 28 00:12:35 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nvidia.com); dmarc=pass(p=reject dis=none) header.from=nvidia.com ARC-Seal: i=2; a=rsa-sha256; t=1786626565; cv=pass; d=zohomail.com; s=zohoarc; b=GNaFCZ4HG+N/LGaSsm5gubmm/t75cyiUER5Ql87wg/sxrHSuDlGwvNd2mONQJYglxE9m5RkIHmNSlAthExOowFEC/SnV9w64Hz5NqxErWDz4oqXJkaT+cC5vxZGSCr7T1Jw0YtenB+7IUOVL3dLBKGoituFqNpD0ezxY1e5zK5g= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1786626565; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=4I/ClDKmMU8jHsRzY5L/WmC6q5lyry5DadmczCJuW+Y=; b=kLMwXtG4CkT67ksHCicV87hW7nNLBKTn0SBq+foc3pKPD5ahMihzbCThLXKWCPaBQz7rcKmlItqbISJeqy9AlF5M1xMAjE/ZlmXOuUn0EeTeAXAkHD0CCz9z6bupuNFU5J8LzSMN++FxD/kBZQj67HPX3DHxxivqoQqKSKE5uS4= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nvidia.com); dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1786626565245728.6740905618663; Thu, 13 Aug 2026 06:09:25 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wuVA2-0006OE-S5; Thu, 13 Aug 2026 09:07:54 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wuV9u-0006Ks-F4; Thu, 13 Aug 2026 09:07:47 -0400 Received: from mail-southcentralusazlp170110003.outbound.protection.outlook.com ([2a01:111:f403:c10d::3] helo=SN4PR0501CU005.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wuV9s-0005Hc-RX; Thu, 13 Aug 2026 09:07:46 -0400 Received: from MW6PR12MB7087.namprd12.prod.outlook.com (2603:10b6:303:238::14) by PH7PR12MB9175.namprd12.prod.outlook.com (2603:10b6:510:2e6::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.315.14; Thu, 13 Aug 2026 13:07:34 +0000 Received: from BN9PR03CA0621.namprd03.prod.outlook.com (2603:10b6:408:106::26) by MW6PR12MB7087.namprd12.prod.outlook.com (2603:10b6:303:238::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.315.15; Thu, 13 Aug 2026 13:07:33 +0000 Received: from BN2PEPF000044A0.namprd02.prod.outlook.com (2603:10b6:408:106:cafe::39) by BN9PR03CA0621.outlook.office365.com (2603:10b6:408:106::26) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.315.14 via Frontend Transport; Thu, 13 Aug 2026 13:07:33 +0000 Received: from mail.nvidia.com (216.228.117.161) by BN2PEPF000044A0.mail.protection.outlook.com (10.167.243.151) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.339.3 via Frontend Transport; Thu, 13 Aug 2026 13:07:32 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.46; Thu, 13 Aug 2026 06:07:08 -0700 Received: from nvidia-4028GR-scsim.nvidia.com (10.126.230.37) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Thu, 13 Aug 2026 06:07:02 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=Vg6UK8fUO3BGj3c5q5jslObeMAgFWntfQkKZhnX11gsDajsB7ZIZiZWZDZiW6o7Gy11+as7uxxJqpLFHlqFtFpk5AH29/debLLQL1IcqLmr4M0s9Nj0FWsB6+ufqabixtk9fLHi9k/B4MSuc1AI+LLgThl+WtPGvVP9yroMzsYqsrwaaUzZ35WgJUGYlr7jNgtoDCdkVJ6S+sOz1RZu/x7l9ExvCzDcuRmvGcEFQ2BwPtvH64CwMlsqLnXoERQZ39mHsD7uOlZIjZSALeepT47gmeb8d6NRP+rfeVQycmpWbmLVdCYGzx3Juz5rhoRBG/vOaVJ/ayqgDHHPJ2prWLA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=4I/ClDKmMU8jHsRzY5L/WmC6q5lyry5DadmczCJuW+Y=; b=K2pXbAkQssCcuiXmWti2jn6q6LC3xNb20cZg2uLwZR5Nkbju1LnrIW4cf7oxz6akxrXoXV9PgUL7vOZlBReHC9HCUslu9H5AJvxFuprg8wL5Hi6PQ9G3FHQZru/PI9f0JAEAL/YglOtmib8cSSov1CJBxRxiDXUZVa3j/KFbh7AEDmhGwwF/CDl37kCwVY5ThHpS1o6KBweC4leVMn41f7czAGDZqp5zSqq12d1H0JA3LmpJY3GI3j6zgIMoATvugETGzNV7r/k9ubQvVb9Cw+TaOBbOw0o7KCvV1Z+/c9lzY69P8NZZPFD11Lwuoy9tTESu3mzLalNuuI7HIMwIlQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=shazbot.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=4I/ClDKmMU8jHsRzY5L/WmC6q5lyry5DadmczCJuW+Y=; b=lKJhKQo0gTGw/NKOoz0DPZs/zaq9iruN3pMVzF9WMwsII8E+v0Ckf7rU5v+GlFfPNH9omWl2r21HjaSAAIiOUP7IhnZbuPLQFYh4GP86vHQm1ZyUL5T4xKfk/HO/aVweTplCpgM94/xd5bfD7eNPxjRdcE/cDg4hQBAHYihJf75COCp10UF3n7jNNdsLLGgz5s2/xHP0cIoyPRXuWRwCY3ADVBKztyNtpEoLd92ger2fiozMMrk7WhProTsWnICbrhAigsJP4EqP8Jq95UKacSuNSArsIeuhmY3qMAtgqEykb7wWo5axVb3ExBvwes/eZ7XsbGYHfv5X4pDIMFy6iQ== X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C From: To: , , , , , , , , , , , , , , , CC: , , , , , Subject: [PATCH 04/10] hw/vfio/pci: Enforce the passthrough topology for a CXL device Date: Thu, 13 Aug 2026 18:36:17 +0530 Message-ID: <20260813130623.2499506-5-mhonap@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20260813130623.2499506-1-mhonap@nvidia.com> References: <20260813130623.2499506-1-mhonap@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.126.230.37] X-ClientProxiedBy: rnnvmail203.nvidia.com (10.129.68.9) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN2PEPF000044A0:EE_|MW6PR12MB7087:EE_|PH7PR12MB9175:EE_ X-MS-Office365-Filtering-Correlation-Id: caf7d344-5450-4848-bc7c-08def93bd672 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|376014|7416014|1800799024|82310400026|23010399003|36860700016|18002099003|22082099003|56012099006|11063799006|10067099003|921020; X-Microsoft-Antispam-Message-Info: T7Sihn1kDBzg4gSEdiL5T3mMTpB4UojJ4cnA6alZr+6b4lti7nGjs1o98bAdjASGZ/RttMBcrikyzj/1qfC4aMhGsg61nnDyozA6CT3PSPPMUXaTrp7Z9Po1HJuthZZh1q9FOhPiITv2IECfoWGcSEY4RkgD3kF57zUtZEZUGFknI/8qtETFejUUUm6UFXjcRT54tOkd7NbNHTl0HMNNeonHj+MVgDX2wKdCiug8ioi09i6M8U6KqW03Wb1y9UjCjL+6z3fUIE+zuB58wdbrCpxjsFZDGdyYlPH/XcKeYPX4jPAKKjfNT0XVQ5jOgcvff3QD2XPXisHv6L8MLsOML7ltjhGfY23fr/v0YUzqhzEbKewQyoO7TKj4AXdTN1gwBcdAqLQKHYtK7K9LCRzRY0SHSJp6LMxkGaWbsUHHFz/Q05hypVNxdJPLah72Mz4A0YvgP2L3Qy5n3MrHQbHe787zEWuQQitEScEe0SN6AtxnnEUjaoG99w+Jg7U7QuCfX6ARQhI4hFRqz+hM2wFU7pNLV//+CjNzxb2EHm5F3gV3rKp/jcGq1ERwycqn4bPPXVJhCBzqv3aKg6bGpi8OS15RQjOFgouk5YbxjS81pdbNKEy8lcL53Lt8PIVkYAOvPTlRkD+4Ht3uCMzo66iiBJMRn1d/YGwJIxWW8POFc/KpA4zUAhFZCkNNCYa8fhCRqH+MwU4o0Zns9JaQkmb3MCkrND2rmZiE4Rx4Z/A1/LzEW6UN/wy619w4taxFX/SK X-Forefront-Antispam-Report: CIP:216.228.117.161; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge2.nvidia.com; CAT:NONE; SFS:(13230040)(376014)(7416014)(1800799024)(82310400026)(23010399003)(36860700016)(18002099003)(22082099003)(56012099006)(11063799006)(10067099003)(921020); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: E4izKhoUi8WCpJfvENeTOUUAaGoGpFrMBPC8P+oFITwOmWr0+5gOXue8jPO//MtQBxzVKNv6RtCRNnj3G0Fa0/bRbuoMJdZe/SdDYQo8jD0YMI6rD0/MAknDY/yDpBA7FcuOIJITvnLRlAvQOlLlu36f/IcmTSJCL74hKrMwMvtpAucI2hGLuYCkJYK6dSnEwk0zu0+Q0/EbDs2pYJ08P4YIl+XM9/ydN368VwBoP7zCQfDmZRdWZZztYTbHlM+gSf9Y/7i9XgC+woOr+iO6MpFv07PGE/2iWYrp8mkGbsoGGqvY06SmydVbsQ0aqXZj2ceDPhaZMKLkhFHnxDt411z4xHSmptbrl0nuzwH2aEUeVh8/9uEg5N/3iun+Nn2RrF//4BlsbI62aP3tko7eWaxtmqECPbRr3sU7ROyYWObqmpoBOXJqwt/YTXec5VW8 X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Aug 2026 13:07:32.7516 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: caf7d344-5450-4848-bc7c-08def93bd672 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.161]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-CrossTenant-AuthSource: BN2PEPF000044A0.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB9175 Received-SPF: permerror client-ip=2a01:111:f403:c10d::3; envelope-from=mhonap@nvidia.com; helo=SN4PR0501CU005.outbound.protection.outlook.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.759, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_NONE=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @Nvidia.com) X-ZM-MESSAGEID: 1786626567004158500 Content-Type: text/plain; charset="utf-8" From: Manish Honap Only the guest's endpoint HDM decoder is programmed by the guest; the decoders above it are covered only while the pxb-cxl host bridge stays in passthrough mode. Reject a switch in the path, or a host bridge taken out of passthrough, at realize before any mapping exists. The topology walk lives in one helper so switch support later relaxes it in one place. Signed-off-by: Manish Honap --- hw/vfio/pci.c | 84 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 84 insertions(+) diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c index 157786df72..1f21da9513 100644 --- a/hw/vfio/pci.c +++ b/hw/vfio/pci.c @@ -27,6 +27,7 @@ #include "hw/pci/msi.h" #include "hw/pci/msix.h" #include "hw/pci/pci_bridge.h" +#include "hw/cxl/cxl.h" #include "hw/core/qdev-properties.h" #include "hw/core/qdev-properties-system.h" #include "hw/vfio/vfio-cpr.h" @@ -3570,6 +3571,84 @@ bool vfio_pci_interrupt_setup(VFIOPCIDevice *vdev, E= rror **errp) return true; } =20 +/* + * Walk the endpoint's ancestor bridges up to its pxb-cxl host bridge. A C= XL + * switch in the path is rejected: the guest would then have to program sw= itch + * decoders, which the passthrough model does not cover. This is the single + * place the supported-topology assumption lives, so switch support later + * relaxes it here. + */ +static PXBCXLDev *vfio_cxl_find_pxb(VFIOPCIDevice *vdev, Error **errp) +{ + PCIBus *bus =3D pci_get_bus(&vdev->parent_obj); + + /* + * The endpoint must sit directly on a CXL root port's downstream bus.= A + * cxl-rp carries QEMU_PCIE_CAP_CXL, so it builds a TYPE_CXL_BUS downs= tream + * bus; a generic pcie-root-port builds a TYPE_PCIE_BUS yet still coun= ts as + * the single downstream port checked at bind, so without this the gue= st + * could realize a Type-2 device with no CXL root port above it: bound= in + * appearance but unusable. The bus type is the right test here, not + * pci_bus_is_cxl(): the PCI_BUS_CXL flag is set only on the pxb-cxl r= oot + * bus (where the root ports sit), not on the root port's downstream b= us. + */ + if (!object_dynamic_cast(OBJECT(bus), TYPE_CXL_BUS)) { + error_setg(errp, + "vfio-cxl: %s: endpoint is not on a CXL root port (cxl-= rp)", + vdev->vbasedev.name); + return NULL; + } + + for (; bus; ) { + PCIDevice *bridge =3D bus->parent_dev; + + if (!bridge) { + break; + } + if (object_dynamic_cast(OBJECT(bridge), TYPE_CXL_USP) || + object_dynamic_cast(OBJECT(bridge), TYPE_CXL_DSP)) { + error_setg(errp, + "vfio-cxl: %s: switch-attached topology not support= ed", + vdev->vbasedev.name); + return NULL; + } + if (object_dynamic_cast(OBJECT(bridge), TYPE_PXB_CXL_DEV)) { + return PXB_CXL_DEV(bridge); + } + if (pci_bus_is_root(bus)) { + break; + } + bus =3D pci_get_bus(bridge); + } + + error_setg(errp, "vfio-cxl: %s: not attached below a pxb-cxl host brid= ge", + vdev->vbasedev.name); + return NULL; +} + +/* + * Option I enforcement: only the guest's endpoint decoder is programmed, = so + * the host bridge must stay in HDM passthrough mode (decoders above the + * endpoint are then trivially correct). Reject anything else at realize, = before + * a mapping is built. cxl_get_hb_passthrough() reflects reset state and i= s not + * settled yet here, so gate on the static hdm_for_passthrough property. + */ +static bool vfio_cxl_check_topology(VFIOPCIDevice *vdev, Error **errp) +{ + PXBCXLDev *pxb =3D vfio_cxl_find_pxb(vdev, errp); + + if (!pxb) { + return false; + } + if (pxb->hdm_for_passthrough) { + error_setg(errp, + "vfio-cxl: %s: the pxb-cxl must be in HDM passthrough m= ode " + "(do not set hdm_for_passthrough)", vdev->vbasedev.name= ); + return false; + } + return true; +} + /* * Learn the CXL geometry the kernel reports: the HPA-backed HDM memory re= gion * and the trapped HDM decoder block (which BAR carries it and at what off= set). @@ -3619,6 +3698,11 @@ static bool vfio_cxl_setup(VFIOPCIDevice *vdev, Erro= r **errp) cxl->dpa_size =3D mem_info->size; cxl->comp_bar =3D cap->bar; cxl->hdm_offset =3D cap->offset; + + if (!vfio_cxl_check_topology(vdev, errp)) { + return false; + } + cxl->enabled =3D true; =20 return true; --=20 2.25.1 From nobody Fri Aug 28 00:12:35 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nvidia.com); dmarc=pass(p=reject dis=none) header.from=nvidia.com ARC-Seal: i=2; a=rsa-sha256; t=1786626510; cv=pass; d=zohomail.com; s=zohoarc; b=eAc0IgfSPdHlSNAvqp3/lLN0BEjxKtYZGOiHQnMsPpb2vTRrqiflwq82qjcZ14VWS96X9PNiwMkwcp8VGTFE1H9grVDlv8Qex1dGKH9twnOFlO28ZFla8OuW5uxBIzRtW2lGSiZq4YgjyxLhScb6PVTrRMNrYzcek3kvAf0TFNE= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1786626510; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=c4kHlaLSgY+ovoGxXkIJhhxHLfLtaf44DCI70YLh/NQ=; b=d3XdEwTYprqAZhjCEImkF0o3/7ZDgCcgP2yN1OYV584gdKSBK3ZVa3Z+x8OAc4lvCFwcSST7ddwzQwjpwyzxkR8LQ3IYFtH6cHMd2qH7BBL5aE8OsEEuM9Jr2JnyWQzXts9na4B6W9hfmZmpfN8Zms6mwTdFv1ttlw5XDmXOY14= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nvidia.com); dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1786626510594772.1969420643869; Thu, 13 Aug 2026 06:08:30 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wuVA9-0006Pd-BJ; Thu, 13 Aug 2026 09:08:01 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wuVA1-0006Nh-Ah; Thu, 13 Aug 2026 09:07:54 -0400 Received: from mail-westus2azlp170100005.outbound.protection.outlook.com ([2a01:111:f403:c005::5] helo=CO1PR03CU002.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wuV9y-0005Jy-HZ; Thu, 13 Aug 2026 09:07:52 -0400 Received: from MN2PR12MB4486.namprd12.prod.outlook.com (2603:10b6:208:263::23) by DS7PR12MB8369.namprd12.prod.outlook.com (2603:10b6:8:eb::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.315.15; Thu, 13 Aug 2026 13:07:37 +0000 Received: from BN1PR14CA0028.namprd14.prod.outlook.com (2603:10b6:408:e3::33) by MN2PR12MB4486.namprd12.prod.outlook.com (2603:10b6:208:263::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.315.13; Thu, 13 Aug 2026 13:07:36 +0000 Received: from BN2PEPF000044A1.namprd02.prod.outlook.com (2603:10b6:408:e3:cafe::66) by BN1PR14CA0028.outlook.office365.com (2603:10b6:408:e3::33) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.315.15 via Frontend Transport; Thu, 13 Aug 2026 13:07:36 +0000 Received: from mail.nvidia.com (216.228.117.161) by BN2PEPF000044A1.mail.protection.outlook.com (10.167.243.152) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.339.3 via Frontend Transport; Thu, 13 Aug 2026 13:07:35 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.46; Thu, 13 Aug 2026 06:07:15 -0700 Received: from nvidia-4028GR-scsim.nvidia.com (10.126.230.37) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Thu, 13 Aug 2026 06:07:08 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=i/cdNEUlsjr9703haJOLB0nLScHzcc5UysOpHcRo1Zbvt2rIfTS2XPCFjASzzn9hokyKfFIfia8H2Vhl3j76FnE7MznqVATRoVQ9E2ORXr8OQoeflPyTjdpInguN2KoTHLTPo2i5ADREeUXH12+mmvWYLTrFhOExGCbnIwUr7VFjlxHl4eFJRvfgdzNxvYihYRhUwTQbHMUsntThsPc+kklCdLr7AONbz5N+KaEOcoK3z9PPVZ80CqgpTVzX+KjnZ7wAVq23Sl2YJx/dc0TbcFkUfxCBxp6u78ykpe+ZUzAMALqJJwbkUizXTIVf+23AAjV3IFeP7Hl1qOOKwr6p2w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=c4kHlaLSgY+ovoGxXkIJhhxHLfLtaf44DCI70YLh/NQ=; b=Xg38FFR9u2cqHGkCuk51kWFy8IX8JkRQrKyzwA2PokmmDZWWN4nfIxSKZ5CH/3r9y+KweqGMESkeBNpfLq1/8/NYDTbptk5E4uFUfcPcNMmFloMAz80WfWFdb/KGaQxA/KG/7mrjd5XNsZfeCnpruVpwfwyw+PhdUKqDiYr2xOw2nndY7BosXPN9PNL0N7A+JMULTZnVO+YOu8sfL5Vc/irV4acUIgYZHhsXan16uVXJ22XUzjuav8R8eUjFK1JioDGVChg4hTerutmR/tOF9tIL5XEtiuk1XSBgib0kWZKYTasWKpNJlM47+ELpHTBheaobJSy8XKl3X055IceoIQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=shazbot.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=c4kHlaLSgY+ovoGxXkIJhhxHLfLtaf44DCI70YLh/NQ=; b=YYoHKuhj8s7iCZN+Noa+gtN8EL6X3ymsRYQw1LA1PKqSROvZYKpQYnMzVtFueS5/RWQ85YXb1lLdQw4LcBRmlTM9pnfOre8fgwuNaMeDVyPoacZazjdYvIUOAgHFWiXcJ1zz6GKwTpHZ5TUmdhF+AhACAhjRnpD4TaoLruiscUNF26g0wajlxs5S5wbrzNbkc35M0yGkd1yItrn6Sn6zVXgQrzvTannLURY4sLI+lP34+cam4d3kKOPeMoux4GSxN2YCu5aHK99czqm/50yuyHkgIfgM+3Z2b1jO1pKQeIA6qBlzFByJVNvB2gotXr0W/8g4vhoaBZZhy7leZG5Rmw== X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C From: To: , , , , , , , , , , , , , , , CC: , , , , , Subject: [PATCH 05/10] hw/vfio/pci: Back the CXL memory with a RAM-device region Date: Thu, 13 Aug 2026 18:36:18 +0530 Message-ID: <20260813130623.2499506-6-mhonap@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20260813130623.2499506-1-mhonap@nvidia.com> References: <20260813130623.2499506-1-mhonap@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.126.230.37] X-ClientProxiedBy: rnnvmail203.nvidia.com (10.129.68.9) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN2PEPF000044A1:EE_|MN2PR12MB4486:EE_|DS7PR12MB8369:EE_ X-MS-Office365-Filtering-Correlation-Id: 4049fde2-80c7-4d0e-63a1-08def93bd85f X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|376014|7416014|36860700016|1800799024|23010399003|82310400026|11063799006|10067099003|56012099006|6133799003|18002099003|22082099003|921020; X-Microsoft-Antispam-Message-Info: kYqOgeyRvZwU/g5zbzc02l6zIeBEdLpUSqL/0ZI+sWGBOkpWuMTASLQhYimNma8HTa91xKi+DnaviOnA8WQ0h48rChyx/tWH1kMmbDtJpjofoTaFdV4J5KgYp/ok8XymqbFLuYSS2M1Rl6FMS1B/D30Nj0rE8aucvDWX8i0aA/SmOH3JK9vdn455K8BK/NU/OHOo10DoDyKn60s6xqWgvxCBgWGZRYyaP35MIHex3QcBDvczH2BHN9RHwlym60PTjMQKQ4OD60CP6yxDF+HbXAop2gmPUd3npJWMyEcynTfpsDy7Va4DanxEAc/s9YjY7XoxzSd8NkRLHj7MVAMsgMBF5R3mAoXaTFD8P9UUoPmo0o3fQxW/4OjNPFHjN3CGMk7+LR2umSb8DMB3y8IB/Fda4idZhvdCTprjhFSVNFWbgtOkA2NTaXHSQ/9VyhhzEDDY0csQBBB1C/+AWaQL3CqsU4dNrHMKPffj0pFNc5Ox7F6Qv2C6DCOZ0LT30tLvUdnZKHMpDflrKrIkEdhSTTWECSppZ4cxssXjwlXpHfQFqz7nc56S9mng4ONOMB3t9MVTboa0U96xpuTHaRSbEMBFBfJfVrn/5p9+eWk26xs9w2izfYMfoovdTyQe/EgW36ugyNIQT8knrp70o//zJPcIRclelajhwzqGXxzS2Rk8ECnNGInfthH9Pjmoqe/TLcNP1lt2nYLGxb1XK+JdwZgzF/8cJ4tKW561+iCMT9CrIPP+Gws4wCGV6hi4yDz5 X-Forefront-Antispam-Report: CIP:216.228.117.161; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge2.nvidia.com; CAT:NONE; SFS:(13230040)(376014)(7416014)(36860700016)(1800799024)(23010399003)(82310400026)(11063799006)(10067099003)(56012099006)(6133799003)(18002099003)(22082099003)(921020); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: GQOAPJu75V0wWqrDJDMvg1StsDgrOSpds4e5b0y23/NFCRdw9/B+aeOqWZWjLWYnpJ+qwY1q9O5yFNxgLJozgvNRgqC/ZxjlMTinN5cGRo0ZUIO35u9Nq6mOkP2FEgCGFFuuOGCpdBGvv53M0gpTXIjdHC6XAvca46fYPfe1HKwpnira9cAH30ZFFERSO99unTm5D6hm4HkxSPto36wpj++7BqqV0+EFZ3nTY1DlenDvPxfwdx5XemdAKhgGGGum/naw7wBNRQ0EAEn6QYnni1InoUpcmoCZqpl46dV6qQyra9zw5t/INesTDHiT2bk4mE9df9pXl4tKhVOuhvedaKa+9ixagmdljjgAhCKMtiWe1iRoPR5IYgyYJVkXnHdfXQRXOuCAT+achAVS/jPkPVmV6ZSEZXHOIv0Vhp6VSwL+4C74ZnL7M07EZusOFWJg X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Aug 2026 13:07:35.9867 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 4049fde2-80c7-4d0e-63a1-08def93bd85f X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.161]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-CrossTenant-AuthSource: BN2PEPF000044A1.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR12MB8369 Received-SPF: permerror client-ip=2a01:111:f403:c005::5; envelope-from=mhonap@nvidia.com; helo=CO1PR03CU002.outbound.protection.outlook.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.759, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FORGED_SPF_HELO=1, SPF_HELO_PASS=-0.001, SPF_NONE=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @Nvidia.com) X-ZM-MESSAGEID: 1786626514750158501 Content-Type: text/plain; charset="utf-8" From: Manish Honap mmap the kernel's HDM memory region so its host physical pages back a RAM-device MemoryRegion. It stays out of the guest address space until the guest commits its decoder and QEMU learns the GPA to place it at. Signed-off-by: Manish Honap --- hw/vfio/pci.c | 45 +++++++++++++++++++++++++++++++++++++++++++++ hw/vfio/pci.h | 1 + 2 files changed, 46 insertions(+) diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c index 1f21da9513..066333d52d 100644 --- a/hw/vfio/pci.c +++ b/hw/vfio/pci.c @@ -3221,8 +3221,11 @@ bool vfio_pci_populate_device(VFIOPCIDevice *vdev, E= rror **errp) return true; } =20 +static void vfio_cxl_teardown(VFIOPCIDevice *vdev); + void vfio_pci_put_device(VFIOPCIDevice *vdev) { + vfio_cxl_teardown(vdev); vfio_display_finalize(vdev); vfio_bars_finalize(vdev); vfio_cpr_pci_unregister_device(vdev); @@ -3666,6 +3669,19 @@ static bool vfio_cxl_setup(VFIOPCIDevice *vdev, Erro= r **errp) return true; } =20 + /* + * The HDM memory is exposed only as an mmap-backed RAM-device region;= the + * kernel rejects fd read/write on it. With x-no-mmap the region would= fall + * back to that always-failing fd path, so every guest access to the H= DM + * range would fault. Reject it here rather than start an unusable VM. + */ + if (vbasedev->no_mmap) { + error_setg(errp, "vfio-cxl: %s: x-no-mmap is not supported for a C= XL " + "device; its HDM memory has no fd read/write fallback", + vbasedev->name); + return false; + } + if (vfio_device_get_region_info_type(vbasedev, VFIO_REGION_TYPE_CXL, VFIO_REGION_SUBTYPE_CXL_MEM, &mem_info)) { @@ -3703,11 +3719,40 @@ static bool vfio_cxl_setup(VFIOPCIDevice *vdev, Err= or **errp) return false; } =20 + /* + * The HDM memory is host physical. mmap it now; it is added to the gu= est + * address space only once the guest commits its endpoint decoder. + */ + if (vfio_region_setup(OBJECT(vdev), vbasedev, &cxl->mem_region, + cxl->mem_region_index, "cxl-mem", errp)) { + return false; + } + if (vfio_region_mmap(&cxl->mem_region)) { + error_setg(errp, "vfio-cxl: %s: failed to mmap the HDM memory regi= on", + vbasedev->name); + vfio_region_exit(&cxl->mem_region); + vfio_region_finalize(&cxl->mem_region); + return false; + } + cxl->enabled =3D true; =20 return true; } =20 +static void vfio_cxl_teardown(VFIOPCIDevice *vdev) +{ + VFIOCXL *cxl =3D &vdev->cxl; + + if (!cxl->enabled) { + return; + } + if (cxl->mem_region.mem) { + vfio_region_exit(&cxl->mem_region); + vfio_region_finalize(&cxl->mem_region); + } +} + static void vfio_pci_realize(PCIDevice *pdev, Error **errp) { ERRP_GUARD(); diff --git a/hw/vfio/pci.h b/hw/vfio/pci.h index 7fdd695704..06d15e807d 100644 --- a/hw/vfio/pci.h +++ b/hw/vfio/pci.h @@ -134,6 +134,7 @@ typedef struct VFIOCXL { uint32_t comp_bar; /* component BAR carrying that block = */ uint64_t hdm_offset; /* block offset within the component = BAR */ uint64_t dpa_size; /* size of the HDM memory region */ + VFIORegion mem_region; /* HDM memory, mapped at committed GP= A */ } VFIOCXL; =20 struct VFIOPCIDevice { --=20 2.25.1 From nobody Fri Aug 28 00:12:35 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nvidia.com); dmarc=pass(p=reject dis=none) header.from=nvidia.com ARC-Seal: i=2; a=rsa-sha256; t=1786626536; cv=pass; d=zohomail.com; s=zohoarc; b=FrjX9ykGJTIpo8ykCdAFFhebMVbtqHPtlsHwmGSdoFCkXoPVldbq2T/an7nVqQ0FU2HBIA3oUkDOTIvU3D0PfZTpOIduqtr328qKvJwaTZBAT+qZPZQ30e7fjPUbIv86S8oH8zzvPi4IyWboh243oXu5jXHmVcPvEdETi/DWXA8= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1786626536; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=DC+x8/TcY0zFvTkZExyU1AAEl7ZGuZ9LXF6GjqTkB98=; b=UWfTpjlDdYwy2Ja9wpXxIV3Huo9PBvzKpRRGuxd4YhcIDQthEdoklctv5yJJYPOwLOcnAMEOlG0bEAmwrcKosb63A+ONneZuX3JDimGXOcsfzc3+4MZyNhGtnFqSZK7TORAaSj2OVvUTAwL2kPEPw42L7FlCMHcUB7Xbq71eJXw= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nvidia.com); dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1786626536512940.0235465412534; Thu, 13 Aug 2026 06:08:56 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wuVAP-0006QS-P7; Thu, 13 Aug 2026 09:08:19 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wuVAG-0006QA-QZ; Thu, 13 Aug 2026 09:08:10 -0400 Received: from mail-southcentralusazlp170130001.outbound.protection.outlook.com ([2a01:111:f403:c10c::1] helo=SA9PR02CU001.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wuVAD-0005OG-PT; Thu, 13 Aug 2026 09:08:08 -0400 Received: from DS4PR12MB9587.namprd12.prod.outlook.com (2603:10b6:8:282::15) by CY8PR12MB7514.namprd12.prod.outlook.com (2603:10b6:930:92::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.315.11; Thu, 13 Aug 2026 13:07:52 +0000 Received: from BN9PR03CA0615.namprd03.prod.outlook.com (2603:10b6:408:106::20) by DS4PR12MB9587.namprd12.prod.outlook.com (2603:10b6:8:282::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.315.15; Thu, 13 Aug 2026 13:07:51 +0000 Received: from BN2PEPF000044A0.namprd02.prod.outlook.com (2603:10b6:408:106:cafe::9f) by BN9PR03CA0615.outlook.office365.com (2603:10b6:408:106::20) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.315.15 via Frontend Transport; Thu, 13 Aug 2026 13:07:50 +0000 Received: from mail.nvidia.com (216.228.117.161) by BN2PEPF000044A0.mail.protection.outlook.com (10.167.243.151) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.339.3 via Frontend Transport; Thu, 13 Aug 2026 13:07:50 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.46; Thu, 13 Aug 2026 06:07:28 -0700 Received: from nvidia-4028GR-scsim.nvidia.com (10.126.230.37) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Thu, 13 Aug 2026 06:07:15 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=VZ3zz8COpSaXbDADo20o1GOSwzg3sXC/UzkwiygpIR+N2jn6fYfY2bPHoPQiITX0OIwy8GcTITLEK731ebA8ZSiy1/HLyyLWHYK6IbYOTw+1wBiXoxfsDazOxF3Sthk+bg3CqZ21lmJDwzXiZNaN6AH/MhVAN5py6f8Ib4sWh/3tOqOhlvPv+R5lvgClp4mkU/CSJ1ogVJ/1phy62hkKJTWWnN1OxVWd+AeZO8Funv+LYOddYfGcJm3Xaq3ZYrkZuswO2ty2WtmQkY6uapBbku76qHQH+S3p+2SCvqfaLoXemdHJU2fhZ4/fhFIpVJC+U9mxXXb+sQb1udD8ZcnWDQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=DC+x8/TcY0zFvTkZExyU1AAEl7ZGuZ9LXF6GjqTkB98=; b=RjqawORGcJlqyXj6QCXC4o0BkkSbR265a/rSy4oZ0G9LdIpSvG621Ffqd1Bo035rdCVEXhwIIxi3sLPClsLRBhMaKt1ts4L2bQN3520Dubn7OcogIz8t3arD/wnxeBKZ4b8qUhxeliosBGP17QybXL9n6itjrRnaDTJMRvOgUlLqpA0DCXFRA6YZ0klmi5rzKHvUgXSJeFA3L+EaDtepqbQXzZL73dS/zVdGZQj0k7vjmUhEDvdm5qC5Hj0ITQEPjnlycrR5Mgajjdn7FyyWjcFKN6HO6yoalN5cxqS/Uky1JLwur5dgjgJcSY3ymjl0dq3Cb7Lfw/Ihq/QA8JDkyw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=shazbot.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=DC+x8/TcY0zFvTkZExyU1AAEl7ZGuZ9LXF6GjqTkB98=; b=flG9rTnRMV5DzdhwAHHrqpFZjsZE/pZQh5yRB7H6FuZ8ynw1eaU+ozaCJMEEzMIXnUewW+/iVORAYoUUn14CVVkPjMRuXibw0HRvgQ91h21BBwtNp21+W9cLZH40pxU2vUrcesdCMlnz2Ax+Rv8jpwFZzZWB7+hsqzwd+P5k+Tt7Pag/GGOURnfc4EtDK9d29hpTgABVdty/ax0l4GTIvFuKKmQGeM4e/d5VmXlI8qdizB03+E9OljHvngR5WEljKv6HF68NICrygoXQ4PIrQZ7HkugyYwOtuYA/u1VUe0NydM8rSwYJhhkfnsnoDpyyXq3+yrgrtutK6tfd1gMxgQ== X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C From: To: , , , , , , , , , , , , , , , CC: , , , , , Subject: [PATCH 06/10] hw/vfio/pci: Bind a CXL device to its fixed memory window Date: Thu, 13 Aug 2026 18:36:19 +0530 Message-ID: <20260813130623.2499506-7-mhonap@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20260813130623.2499506-1-mhonap@nvidia.com> References: <20260813130623.2499506-1-mhonap@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.126.230.37] X-ClientProxiedBy: rnnvmail203.nvidia.com (10.129.68.9) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN2PEPF000044A0:EE_|DS4PR12MB9587:EE_|CY8PR12MB7514:EE_ X-MS-Office365-Filtering-Correlation-Id: ea3e3e00-516e-482b-dfd8-08def93be126 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|7416014|376014|36860700016|82310400026|23010399003|1800799024|56012099006|10067099003|6133799003|22082099003|18002099003|11063799006|921020; X-Microsoft-Antispam-Message-Info: LZ6Gqbdk2UPg71Q0C09MMe8NOdFyzJV6irUaBnrZMXoLCHTJETwjJkrMnI6jxNb0K5pvJN/SYh7cjvZ/aTfKqRepg3M3piptUHBbmiq7tWj5HSbRRVg1PwPIfHutSpODL9suxR3L1HyftNRfyAZwCkkCBis4zGO2dFg2vCeLAxoztHyW2KqHNY4b1bxtV8Cqv4GQfFrYXyxhx5rLIHeLKDH7Yn3n3sEa5ZqwoXyQcVN/edKs1ZujkCygw+1dyONzVNL5qakVQzjsnZFLa2UTF8jRjckMMxu9PxceH53f5GO6ZDXDQEnH0wKb9aAzkyYWhZxUbxryXOCLdd8TyBuYibrIEQmY9kzCYR7sKZ4cDxg55KVjimJ6TTy12C5moQ3I/mdfGTAaadSg4vtH68ZB5XrHfipMI1Xs5ov5tnIriPT4ylvJlT+CU3wyRY+Mc3l4Ndj6YRdW1B9A+vkqxcpUWN8UXhVpV6rEJ/Cqh2jfWbCu8/3Wtjxq746JqqViV1YvRCvkBWKGgiL+lkiXA9FUjtnnxnWW4rqVTCSwUsZET+Ly1NfGr+hbl2l4qlogQJizToZRTfJbSctE40X3b+ysxy+dWGauLmPBlv+lxFUw4339wSMxUYONxkxTNgAtSiIlLg7+Xztww6i0SEQ+jRCR6Lf9rMtLv6PmPDlEY6mnDIiJ8ME3PbBl2pdrRnGGTXYi9zM35AL0iAun/340anD/Al1T2ISFkpr+kE+S+ZC+0LZUEh2ZbGPqYcS2msF551u/ X-Forefront-Antispam-Report: CIP:216.228.117.161; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge2.nvidia.com; CAT:NONE; SFS:(13230040)(7416014)(376014)(36860700016)(82310400026)(23010399003)(1800799024)(56012099006)(10067099003)(6133799003)(22082099003)(18002099003)(11063799006)(921020); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: GdVCOAO2En2MhQFe83ZkzOQ3Ebh3H6Q+lKif2S0FQf8PYgyk7tsZSJ9RTJoFCDR2YQEUGszl76YaXVuG5Oc8Zkga0PJAB1KxMRMFRxxJMvSqHj39cZQgaLkJh6aV6l2l34QZdGoyFrYjsLK3388i8Rv1SaKpcFiC2ZuOjAZdlMtzAjk70zIpNa+CP+F4duwWuN0eLOuEc4CKwxIFiXCqBCNuq3Iq69ouHFQRnM9sNOTx86dP//c6iG8SWbi6c1vIxrldwV7FB45qkrNHSLyFdw7/Tm6vmpeXSCEUYwxJtj8NRY54X8g8DXvPJn/68s6EROlo+OJnF4B4I1pg75gR7jDVMb0luZtDputmIyT1Po+xdw70GuFE8fSOHHK4I+nxs3TNlGtkLiqZJ6cw+x7qdq8/nl2dNgEvdfg0OoEIlGWhdT1g66Tq8LG9cKpqp1JS X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Aug 2026 13:07:50.7221 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: ea3e3e00-516e-482b-dfd8-08def93be126 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.161]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-CrossTenant-AuthSource: BN2PEPF000044A0.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR12MB7514 Received-SPF: permerror client-ip=2a01:111:f403:c10c::1; envelope-from=mhonap@nvidia.com; helo=SA9PR02CU001.outbound.protection.outlook.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.759, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FORGED_SPF_HELO=1, SPF_HELO_PASS=-0.001, SPF_NONE=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @Nvidia.com) X-ZM-MESSAGEID: 1786626539052158500 Content-Type: text/plain; charset="utf-8" From: Manish Honap The guest programs a GPA inside a CXL fixed memory window and QEMU maps the HDM memory there, so the device must sit under exactly one single-target CFMWS. Reject any ambiguous, interleaved, unsized, or missing window rather than guess. Match windows by target name, since the resolved target pointers are only filled in by a machine-done notifier that may run after this one. The CFMWS windows are placed at machine-init-done, so a cold-plugged device can only be validated from that notifier, where a bad topology is fatal to startup. Factor the check into a helper that reports through errp so a later change can also validate a hotplugged device from realize. The endpoint's HDM size is device information QEMU already reads from the kernel region, not a value the caller should have to guess. Sizing the window from the device is not yet possible because it is placed in the guest PA map before this device is realized, so treat the device geometry as the source of truth for the check: reject a window too small to hold the endpoint and name the exact size to set, and warn on an oversized window, since the padding becomes guest CEDT that migration has to preserve. Signed-off-by: Manish Honap --- hw/pci-bridge/pci_expander_bridge_stubs.c | 6 + hw/vfio/pci.c | 190 ++++++++++++++++++++++ hw/vfio/pci.h | 3 + 3 files changed, 199 insertions(+) diff --git a/hw/pci-bridge/pci_expander_bridge_stubs.c b/hw/pci-bridge/pci_= expander_bridge_stubs.c index b35180311f..c44ad7fab6 100644 --- a/hw/pci-bridge/pci_expander_bridge_stubs.c +++ b/hw/pci-bridge/pci_expander_bridge_stubs.c @@ -10,5 +10,11 @@ #include "hw/pci/pci_bus.h" #include "hw/pci-bridge/pci_expander_bridge.h" #include "hw/cxl/cxl.h" +#include "hw/cxl/cxl_component.h" =20 void pxb_cxl_hook_up_registers(CXLState *state, PCIBus *bus, Error **errp)= {}; + +bool cxl_get_hb_passthrough(PCIHostState *hb) +{ + return false; +} diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c index 066333d52d..bf6f36d514 100644 --- a/hw/vfio/pci.c +++ b/hw/vfio/pci.c @@ -27,7 +27,12 @@ #include "hw/pci/msi.h" #include "hw/pci/msix.h" #include "hw/pci/pci_bridge.h" +#include "hw/pci/pci_host.h" +#include "hw/pci/pcie_port.h" #include "hw/cxl/cxl.h" +#include "hw/cxl/cxl_host.h" +#include "hw/cxl/cxl_component.h" +#include "system/system.h" #include "hw/core/qdev-properties.h" #include "hw/core/qdev-properties-system.h" #include "hw/vfio/vfio-cpr.h" @@ -3652,6 +3657,187 @@ static bool vfio_cxl_check_topology(VFIOPCIDevice *= vdev, Error **errp) return true; } =20 +/* + * Count the CXL fixed memory windows that target this device's pxb-cxl and + * report the matched window's base, size and target count. Match on the t= arget + * names: cxl_fmws_link_targets() resolves target_hbs[] only at machine_do= ne, + * which may run after this, so the resolved pointers can still be NULL he= re. + */ +static int vfio_cxl_match_fmws(PXBCXLDev *pxb, hwaddr *base, uint64_t *siz= e, + int *nwindows, int *ntargets) +{ + GSList *list =3D cxl_fmws_get_all_sorted(); + GSList *iter; + int matches =3D 0; + + *nwindows =3D g_slist_length(list); + *base =3D 0; + *size =3D 0; + *ntargets =3D 0; + + for (iter =3D list; iter; iter =3D iter->next) { + CXLFixedWindow *fw =3D CXL_FMW(iter->data); + int i; + + for (i =3D 0; i < fw->num_targets; i++) { + bool ambiguous =3D false; + Object *t =3D object_resolve_path_type(fw->targets[i], + TYPE_PXB_CXL_DEV, &ambigu= ous); + + if (t && !ambiguous && PXB_CXL_DEV(t) =3D=3D pxb) { + *base =3D fw->base; + *size =3D fw->size; + *ntargets =3D fw->num_targets; + matches++; + break; + } + } + } + g_slist_free(list); + return matches; +} + +/* + * Fix the bounds of the device's memory window once the topology has sett= led. + * Exactly one single-target CFMWS, with an assigned base and room for the= HDM + * memory, is required; anything else is a misconfiguration the guest cann= ot + * recover from, so reject it rather than guess a window. QEMU presents th= is + * window base as the decoder base to the guest, so the guest works purely= in + * GPA; the host physical placement stays with the kernel. + * + * Returns false and sets errp on a bad topology so the caller can either = fail a + * hotplug device_add or abort a cold boot. + */ +static bool vfio_cxl_do_bind_fmws(VFIOPCIDevice *vdev, Error **errp) +{ + VFIOCXL *cxl =3D &vdev->cxl; + const char *name =3D vdev->vbasedev.name; + int nwindows =3D 0, ntargets =3D 0, matches; + hwaddr base =3D 0; + uint64_t size =3D 0, need; + PXBCXLDev *pxb; + + pxb =3D vfio_cxl_find_pxb(vdev, errp); + if (!pxb) { + return false; + } + /* + * The pxb-cxl enters HDM passthrough only with a single root port and= no + * emulated decoders, but cxl->passthrough is set by pxb_cxl_dev_reset= (), + * which runs at the system reset after machine init done, so it is not + * readable from a machine-init-done notifier yet. Check the same topo= logy + * the reset keys on; hdm_for_passthrough was already rejected at real= ize by + * vfio_cxl_check_topology(). + */ + if (pcie_count_ds_ports(PCI_HOST_BRIDGE(pxb->cxl_host_bridge)->bus) != =3D 1) { + error_setg(errp, + "vfio-cxl: %s: pxb-cxl not in HDM passthrough mode " + "(use a single cxl-rp)", name); + return false; + } + + /* + * A PCIe root-port link is point to point, but QEMU only warns when a + * second endpoint is cold-plugged at a nonzero slot below the port. B= oth + * endpoints would match the same single-target CFMWS and map their HDM + * memory at the same base, aliasing one over the other. Require exact= ly one + * endpoint below the root port. + */ + { + PCIBus *ep_bus =3D pci_get_bus(&vdev->parent_obj); + int slot, fn, nendpoints =3D 0; + + for (slot =3D 0; slot < PCI_SLOT_MAX; slot++) { + for (fn =3D 0; fn < PCI_FUNC_MAX; fn++) { + if (ep_bus->devices[PCI_DEVFN(slot, fn)]) { + nendpoints++; + break; + } + } + } + if (nendpoints !=3D 1) { + error_setg(errp, + "vfio-cxl: %s: %d devices below the cxl-rp; a passe= d " + "through CXL endpoint must be alone below its root = port", + name, nendpoints); + return false; + } + } + + matches =3D vfio_cxl_match_fmws(pxb, &base, &size, &nwindows, &ntarget= s); + if (matches =3D=3D 1 && ntargets =3D=3D 1) { + if (!base) { + error_setg(errp, + "vfio-cxl: %s: matched CFMWS has no base; reduce it= s " + "size or grow the guest PA space", name); + return false; + } + /* + * QEMU reads the endpoint's HDM size from the kernel region, so t= he + * window size is really device information, not a value to make t= he + * caller guess. Sizing the CFMWS from the device the way firmware= does + * is not possible here: the window is placed in the guest PA map = by + * cxl_fmws_set_memmap() before this device is realized, so its si= ze is + * fixed before dpa_size is known. Until a core change can size the + * window from the device, take the device geometry as the source = of + * truth: reject a window too small to hold the endpoint, and warn= when + * it is larger than needed, since the padding becomes guest CEDT = that + * migration has to preserve. + */ + need =3D ROUND_UP(cxl->dpa_size, 256 * MiB); + if (size < need) { + error_setg(errp, + "vfio-cxl: %s: CFMWS size 0x%" PRIx64 " cannot hold= the " + "endpoint; set the cxl-fmw size to 0x%" PRIx64, + name, size, need); + return false; + } + if (size > need) { + warn_report("vfio-cxl: %s: CFMWS size 0x%" PRIx64 " exceeds th= e " + "endpoint's 0x%" PRIx64 "; set the cxl-fmw size to= 0x%" + PRIx64 " to keep the guest memory map stable acros= s " + "migration", name, size, cxl->dpa_size, need); + } + cxl->fmws_base =3D base; + cxl->fmws_size =3D size; + return true; + } + + if (matches =3D=3D 1) { + error_setg(errp, + "vfio-cxl: %s: its CFMWS interleaves %d targets; use a " + "single-target window", name, ntargets); + } else if (matches > 1) { + error_setg(errp, + "vfio-cxl: %s: pxb-cxl is targeted by %d CFMWS; use one= ", + name, matches); + } else { + error_setg(errp, + "vfio-cxl: %s: no CFMWS targets this device (%d present= )", + name, nwindows); + } + return false; +} + +/* + * Cold-plug path: the CFMWS windows are placed at machine init done, so t= he + * binding can only be validated from this notifier. The VM has not run ye= t, so + * a configuration error is fatal to startup. A hotplugged device is valid= ated + * in realize instead (see vfio_cxl_setup), where the failure fails device= _add + * without taking down the running VM. + */ +static void vfio_cxl_bind_fmws(Notifier *n, void *data) +{ + VFIOCXL *cxl =3D container_of(n, VFIOCXL, machine_done); + VFIOPCIDevice *vdev =3D container_of(cxl, VFIOPCIDevice, cxl); + Error *err =3D NULL; + + if (!vfio_cxl_do_bind_fmws(vdev, &err)) { + error_report_err(err); + exit(1); + } +} + /* * Learn the CXL geometry the kernel reports: the HPA-backed HDM memory re= gion * and the trapped HDM decoder block (which BAR carries it and at what off= set). @@ -3735,6 +3921,9 @@ static bool vfio_cxl_setup(VFIOPCIDevice *vdev, Error= **errp) return false; } =20 + cxl->machine_done.notify =3D vfio_cxl_bind_fmws; + qemu_add_machine_init_done_notifier(&cxl->machine_done); + cxl->enabled =3D true; =20 return true; @@ -3747,6 +3936,7 @@ static void vfio_cxl_teardown(VFIOPCIDevice *vdev) if (!cxl->enabled) { return; } + qemu_remove_machine_init_done_notifier(&cxl->machine_done); if (cxl->mem_region.mem) { vfio_region_exit(&cxl->mem_region); vfio_region_finalize(&cxl->mem_region); diff --git a/hw/vfio/pci.h b/hw/vfio/pci.h index 06d15e807d..6f184520bf 100644 --- a/hw/vfio/pci.h +++ b/hw/vfio/pci.h @@ -135,6 +135,9 @@ typedef struct VFIOCXL { uint64_t hdm_offset; /* block offset within the component = BAR */ uint64_t dpa_size; /* size of the HDM memory region */ VFIORegion mem_region; /* HDM memory, mapped at committed GP= A */ + Notifier machine_done; /* CFMWS validated at machine_done */ + hwaddr fmws_base; /* base of the memory window */ + uint64_t fmws_size; /* size of that window */ } VFIOCXL; =20 struct VFIOPCIDevice { --=20 2.25.1 From nobody Fri Aug 28 00:12:35 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nvidia.com); dmarc=pass(p=reject dis=none) header.from=nvidia.com ARC-Seal: i=2; a=rsa-sha256; t=1786626539; cv=pass; d=zohomail.com; s=zohoarc; b=fAmd2CSmzxO9+hEqrJiriEeNMmjBBKN3VKZ4ar7cXiczcY6/RcVMbUxGkBAkrEzR0MuOc8ummQ92sUQlh7Yo/3DJzv1kQADyTrQM511AunzSIe1ekhzkUW7Dy0HDuaaBu14em3cxH2BkgS0fELj10XIXctt1Y/zfL7vepuGGfAE= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1786626539; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=3oe7bwLG7Hmc6CmtwNCZPDQP3gsR+F8rNeoLXuBB59Q=; b=jykTTL7lQpFoapmaE3C5i30B3q5Bp7uMW3IsoGLrdeke8GadUwTyGciPwuqpuYi1rXzzcuvs67SmhEZLjeTcuu66wFlJJzYAwOybYIPKgA1DCkPuVi7u2XWMiIfnxxX+ojLnp0CHKomUXhBp5QdTwbmmkc79HRwnww09RV/GOj4= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nvidia.com); dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 178662653954418.598722449573074; Thu, 13 Aug 2026 06:08:59 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wuVAb-0006YA-Q4; Thu, 13 Aug 2026 09:08:29 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wuVAa-0006TF-1Q; Thu, 13 Aug 2026 09:08:28 -0400 Received: from mail-westus2azlp170120002.outbound.protection.outlook.com ([2a01:111:f403:c007::2] helo=MW6PR02CU001.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wuVAT-0005RY-WA; Thu, 13 Aug 2026 09:08:27 -0400 Received: from SJ0PR12MB6735.namprd12.prod.outlook.com (2603:10b6:a03:479::8) by PH7PR12MB7427.namprd12.prod.outlook.com (2603:10b6:510:202::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.315.14; Thu, 13 Aug 2026 13:08:08 +0000 Received: from SJ0PR13CA0115.namprd13.prod.outlook.com (2603:10b6:a03:2c5::30) by SJ0PR12MB6735.namprd12.prod.outlook.com (2603:10b6:a03:479::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.315.13; Thu, 13 Aug 2026 13:07:56 +0000 Received: from CO1PEPF00012E7F.namprd03.prod.outlook.com (2603:10b6:a03:2c5:cafe::1b) by SJ0PR13CA0115.outlook.office365.com (2603:10b6:a03:2c5::30) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.339.3 via Frontend Transport; Thu, 13 Aug 2026 13:07:56 +0000 Received: from mail.nvidia.com (216.228.117.160) by CO1PEPF00012E7F.mail.protection.outlook.com (10.167.249.54) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.339.3 via Frontend Transport; Thu, 13 Aug 2026 13:07:55 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.46; Thu, 13 Aug 2026 06:07:30 -0700 Received: from nvidia-4028GR-scsim.nvidia.com (10.126.230.37) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Thu, 13 Aug 2026 06:07:21 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=nbQQpN+cQPBNwmk/0aSryqlNOCHl1BFkQlSP9Los4mC9PxNyZvUnwaqZDAFodZQAGKqMTqj8hyw5uTvQMg7xDqThNaBLY+zH0NGhAfYLrDyWYJvVrkh9emUwVBqNikgLS/bVpUGXvvf3xniS179/t9AkS9N8Sr+I+i6bnOLTXXejU61cv1s1k3P1v3dew4pFEDjjfEnzhjsXrLw0p6WXC2ysalnQBJZiAr//N2UYzmvhykTDvkWhujMuCvWlsQLNFMVRZjQe0Ljy9qwWjrkaMiOwK5UHHQPOlsU+g74DwegnA7CWz2MtORYKcGXMtwdCNktKpiNiHeqIOVdFR8oGhA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=3oe7bwLG7Hmc6CmtwNCZPDQP3gsR+F8rNeoLXuBB59Q=; b=tG+aWKnS3tfwXRpmTv8BwhJLxmGjB9/NLrJD9sm6nVuPg3LpFmkitZuaIwvZdzxdWuMRzUZ+YiHPF6DMSS6jOVFU2w34PH1dMrqNVQiiV4+xTPUnhrcLSVQ2hWHwf5SbBBCz70D+mYgWTL0IiBzjc/XhFsUPI2coJOaZKnLsLnKPxJ8tEXz/kXpiRb2QKd2YRg3h18dEeZQB80X/mALHJADAhliVzdIpqzuEeEYP7Wdy/1WZoJFBkAomGIjdbIORlyGAp62bUXYIIKHrvxzpmrEPDVxlBsOK0ZYNRb3diOvmvl6c0n5gfV8xy445hSYEpRK2GZRbylJ32KLR+2Dlsw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=shazbot.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=3oe7bwLG7Hmc6CmtwNCZPDQP3gsR+F8rNeoLXuBB59Q=; b=fpWJSEvBiKVCNsLaJ7+xVqaDMIYScWxKYWlKMA6M7wympQZnFMiggCr44w4RK9vIqF5EDLGrMHOs/teUX9KBYi6ZNUV8vIDWHirCcYexmoUt4tDPHD7Qo/O3G848NcDFkIlnrsvgsYWuIcwgmrc1KPTDwSr6qfK+eAxf/oCFuiqxyo+PY9Vu4LKZUMn8V9UD9Dx4z0uSDoNmVbUbvtJhdPoLVInkWlTg3B9bpSciywBsItdr+IWlt8Sxezuk/x3eVmVJ/CcPNtMRXzx/7ymZ1JFy9KwUpp6Z/SMU5tPnvPX4IQV0fg5mDNTaKdrLfYR6xtueoApvfP8MMvy/Ypnvkg== X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C From: To: , , , , , , , , , , , , , , , CC: , , , , , Subject: [PATCH 07/10] hw/vfio/pci: Map the CXL memory on the guest decoder commit Date: Thu, 13 Aug 2026 18:36:20 +0530 Message-ID: <20260813130623.2499506-8-mhonap@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20260813130623.2499506-1-mhonap@nvidia.com> References: <20260813130623.2499506-1-mhonap@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.126.230.37] X-ClientProxiedBy: rnnvmail203.nvidia.com (10.129.68.9) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1PEPF00012E7F:EE_|SJ0PR12MB6735:EE_|PH7PR12MB7427:EE_ X-MS-Office365-Filtering-Correlation-Id: 661adbe3-0852-4767-21a6-08def93be439 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|36860700016|376014|7416014|82310400026|1800799024|23010399003|18002099003|22082099003|56012099006|11063799006|6133799003|10067099003|921020; X-Microsoft-Antispam-Message-Info: O1hPUerq9o8DX+6GBQ1eQWFJoF0dEZMz6cSSPa1AuPoHSUK6nHKCmEauyiveCAji8tj+/J1YXPWemWCUHi1ObP2jbxFICjvK+jcB/H/j/wbLC19R2XI9yrx5xVoovJCz4XYVPTbMjuZy0oKbCQicIf3M+f63nAAVMekEj7iLERpD0HQkIYig1VLfjJFX8lMdt/ym2qIXLtB/ETo2ju/nq7UUj+BBrYP80GtbIL/sbRXwlnM5qSK0KOR2Gqk0jxjiwfkrvM2qCrbbtTGXcBJ9bmxBo0E42As/xrf4lz7iWo3rinE37R+Xff7kaL0dbNP7ktPMKbhGr76fZkuTyAUis3yWtifgcEd/4+g+sxxJedGeJxSskONyBGkZkNS+rnnTvDWdFVU+9RAjiHbq92aJIuqkkx5kHItXBEdCTSn0AsFCxXBYKorJTFWOVlQ2xiBgqn5uxpt7aWEp5GxRT/3qWTyGDuGApRCCTOcS9fQLnM0i2sjpSq4K6+7CW5z+gVe1SWlj+qhTzsQ8xfX1zhI1pRvb7toWthdCXTevvX7vUp9SE2xXeCF49jgl7fby6eoeUbkgpfj9yUm/WNDXJcVIbHhSeR5UxKJCyI0ceDJ2gBCxmnFbTKtvuPrgJxOYd6SMtsc7BYaAYQletEdHOzv4L4ke+2Z8zf2RUMcwXUG7eUIFJe48aUpqV0WrVZ+Zl9oTRYm/HgDFupgHTLYAllId9A6XKB0PuzVgV7TB76IUXJ8p6FHUV9qivkfFL1r8F3II X-Forefront-Antispam-Report: CIP:216.228.117.160; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge1.nvidia.com; CAT:NONE; SFS:(13230040)(36860700016)(376014)(7416014)(82310400026)(1800799024)(23010399003)(18002099003)(22082099003)(56012099006)(11063799006)(6133799003)(10067099003)(921020); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: JVtgLUvarCeucQBR8Z0wFQGATKnMr5xReHfCpKsmpvF+mbFtJHc1Lka0vd5IEZyTGfW77XD9B8ytsf/IN1rbBnnQ+hVn2R0m2xqH2xHb7owzOSl11XkXEEeZ3EPbW6rsc01WKIe4f0o812GpyyZSvBuSK+RAYuTTNluwMgwOX2qDdEVIqramZpDRNvkKxHrScIApcBt/Gds2qGRZ8PAghJT60OPFSRJBBlHOrKceqlPVLkZbsZczZ0VCfJ5Rjg/F/rkUi3T+YPwO+0Ujmi5EOM3wwrliHilAOmX/he27JB12vtJdNAsj1QTiH8EPgr0NVycCztiwoc5gZDpeLOt6oEca+juvRRI0DdRQV8tdIPFnAg1cHUELyVQFYE7WwQvk0wEwVmhtPXN1dQfwVxb+U2N2Kqg6GTDR6YAltZw1snmt6R9SM7D9DllS6T15fkdU X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Aug 2026 13:07:55.9089 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 661adbe3-0852-4767-21a6-08def93be439 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.160]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-CrossTenant-AuthSource: CO1PEPF00012E7F.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB7427 Received-SPF: permerror client-ip=2a01:111:f403:c007::2; envelope-from=mhonap@nvidia.com; helo=MW6PR02CU001.outbound.protection.outlook.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.759, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_NONE=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @Nvidia.com) X-ZM-MESSAGEID: 1786626540992158501 Content-Type: text/plain; charset="utf-8" From: Manish Honap Overlay the trapped HDM decoder block over the component BAR and forward its accesses to the kernel, which runs the lock-on-commit FSM. This hardware presents a firmware-fixed decoder, so QEMU does not honor the base the guest programmed: when a decoder is committed, QEMU maps the HDM memory at the base of the device's CFMWS window and presents that same base back to the guest through the trapped decoder read, so the guest works purely in GPA and never sees the host physical base. A decommit removes the mapping. Overlap the CFMWS window MemoryRegion the CXL host bridge maps at that base, at a higher priority, so precedence is defined by priority rather than subregion insertion order. The trapped component region and its err: cleanup exist here, so this is also where a hotplugged device is bound: validate the CFMWS from realize and report through errp, so a bad device_add fails cleanly instead of aborting the running VM. A cold-plugged device is still validated from the machine-init-done notifier. Signed-off-by: Manish Honap --- hw/vfio/pci.c | 371 ++++++++++++++++++++++++++++++++++++++++++++++++-- hw/vfio/pci.h | 5 + 2 files changed, 368 insertions(+), 8 deletions(-) diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c index bf6f36d514..dac3011739 100644 --- a/hw/vfio/pci.c +++ b/hw/vfio/pci.c @@ -1453,6 +1453,39 @@ uint32_t vfio_pci_read_config(PCIDevice *pdev, uint3= 2_t addr, int len) return val; } =20 +static void vfio_cxl_decoder_changed(VFIOPCIDevice *vdev); +static void vfio_cxl_unmap_mem(VFIOPCIDevice *vdev); + +/* + * Return true if this config write sets a Function Level Reset bit the ke= rnel + * acts on: the PCIe Device Control FLR bit or the Advanced Features FLR b= it. + * FLR bits are write-1-to-trigger and self-clearing, so inspect the writt= en + * value rather than the post-write config. + */ +static bool vfio_cxl_flr_write(VFIOPCIDevice *vdev, uint32_t addr, + uint32_t val, int len) +{ + PCIDevice *pdev =3D &vdev->parent_obj; + uint32_t off; + + if (pdev->exp.exp_cap) { + /* PCI_EXP_DEVCTL_BCR_FLR (bit 15) is the high byte of Device Ctrl= . */ + off =3D pdev->exp.exp_cap + PCI_EXP_DEVCTL + 1; + if (off >=3D addr && off < addr + len && + ((val >> (8 * (off - addr))) & (PCI_EXP_DEVCTL_BCR_FLR >> 8)))= { + return true; + } + } + if (vdev->cxl.af_offset) { + off =3D vdev->cxl.af_offset + PCI_AF_CTRL; + if (off >=3D addr && off < addr + len && + ((val >> (8 * (off - addr))) & PCI_AF_CTRL_FLR)) { + return true; + } + } + return false; +} + void vfio_pci_write_config(PCIDevice *pdev, uint32_t addr, uint32_t val, int len) { @@ -1522,9 +1555,74 @@ void vfio_pci_write_config(PCIDevice *pdev, vfio_sub_page_bar_update_mapping(pdev, bar); } } + + /* + * A firmware-committed CXL decoder is already committed at boot, = so no + * guest control write triggers the HDM mapping. Map it once the g= uest + * enables memory decoding, so the region enters the guest address= space + * and the IOAS while the device is live. On a clear of memory dec= oding, + * withdraw the overlay: the kernel revokes the HDM PTEs on the sa= me + * write, so a retained memslot would fault a guest access during = the + * disabled interval onto a zapped VMA and stop the VM; the enable= path + * re-maps it. + */ + if (vdev->cxl.enabled && + range_covers_byte(addr, len, PCI_COMMAND)) { + if (pci_get_word(pdev->config + PCI_COMMAND) & PCI_COMMAND_MEM= ORY) { + vfio_cxl_decoder_changed(vdev); + } else { + vfio_cxl_unmap_mem(vdev); + } + } } else { /* Write everything to QEMU to keep emulated bits correct */ pci_default_write_config(pdev, addr, val, len); + + /* + * A guest CXL reset is a write to the CXL Device DVSEC ctrl2 regi= ster, + * forwarded to the kernel above, which re-commits this firmware-f= ixed + * decoder. If the guest decommitted the decoder before the reset,= which + * unmapped the HDM memory, that re-commit is not otherwise visibl= e to + * QEMU, so rescan the decoder here to restore the mapping. Gated = on + * memory decoding still being enabled, like the enable path above. + */ + if (vdev->cxl.enabled && vdev->cxl.dvsec_offset && + ranges_overlap(addr, len, + vdev->cxl.dvsec_offset + + offsetof(CXLDVSECDevice, ctrl2), + sizeof_field(CXLDVSECDevice, ctrl2)) && + (pci_get_word(pdev->config + PCI_COMMAND) & PCI_COMMAND_MEMORY= )) { + vfio_cxl_decoder_changed(vdev); + } + + /* + * A NoSoftRst- device the guest cycles D3hot->D0 has its physical + * decoder restored by the kernel on resume, which re-commits this + * firmware-fixed decoder just like a reset. That re-commit is not + * otherwise visible to QEMU, so on the transition back to D0 resc= an + * the decoder to restore a mapping the guest dropped before it + * suspended. Gated on memory decoding, like the paths above. + */ + if (vdev->cxl.enabled && pdev->pm_cap && + range_covers_byte(addr, len, pdev->pm_cap + PCI_PM_CTRL) && + (pci_get_word(pdev->config + pdev->pm_cap + PCI_PM_CTRL) & + PCI_PM_CTRL_STATE_MASK) =3D=3D 0 && + (pci_get_word(pdev->config + PCI_COMMAND) & PCI_COMMAND_MEMORY= )) { + vfio_cxl_decoder_changed(vdev); + } + + /* + * A Function Level Reset the guest triggers through the PCIe Devi= ce + * Control or Advanced Features FLR bit is forwarded to the kernel, + * which re-commits this firmware-fixed decoder just like a CXL re= set. + * That re-commit is not otherwise visible to QEMU, so rescan the + * decoder to restore a mapping the guest dropped before the FLR. = Gated + * on memory decoding, like the paths above. + */ + if (vdev->cxl.enabled && vfio_cxl_flr_write(vdev, addr, val, len) = && + (pci_get_word(pdev->config + PCI_COMMAND) & PCI_COMMAND_MEMORY= )) { + vfio_cxl_decoder_changed(vdev); + } } } =20 @@ -3838,6 +3936,219 @@ static void vfio_cxl_bind_fmws(Notifier *n, void *d= ata) } } =20 +/* + * HDM decoder registers, relative to the trapped decoder block. The block= holds + * the HDM Decoder Capability register followed by one register set per de= coder, + * each VFIO_CXL_HDM_DECODER_STRIDE apart. Index the sets by decoder so + * multi-decoder support is a loop bound rather than a rewrite; this gener= ation + * commits a single decoder. + */ +#define VFIO_CXL_HDM_CAP 0x00 +#define VFIO_CXL_HDM_DECODER_STRIDE 0x20 +#define VFIO_CXL_HDM_DECODER_BASE_LOW(n) \ + (0x10 + (n) * VFIO_CXL_HDM_DECODER_STRIDE) +#define VFIO_CXL_HDM_DECODER_BASE_HIGH(n) \ + (0x14 + (n) * VFIO_CXL_HDM_DECODER_STRIDE) +#define VFIO_CXL_HDM_DECODER_CTRL(n) \ + (0x20 + (n) * VFIO_CXL_HDM_DECODER_STRIDE) +#define VFIO_CXL_HDM_CTRL_COMMITTED (1 << 10) +#define VFIO_CXL_HDM_BASE_LOW_MASK 0xf0000000U + +static void vfio_cxl_unmap_mem(VFIOPCIDevice *vdev) +{ + VFIOCXL *cxl =3D &vdev->cxl; + + if (!cxl->dpa_mapped) { + return; + } + memory_region_del_subregion(get_system_memory(), cxl->mem_region.mem); + cxl->dpa_mapped =3D false; + cxl->mapped_base =3D 0; +} + +/* + * The HDM Decoder Capability register encodes the decoder count in its low + * nibble (0 -> 1, 1 -> 2, 2 -> 4, 3 -> 6, 4 -> 8 ...). Read it from the t= rapped + * block so the commit handler walks every decoder rather than assuming de= coder + * 0. An unreadable or unknown encoding is treated as a single decoder. + */ +static unsigned vfio_cxl_decoder_count(VFIOPCIDevice *vdev) +{ + off_t off =3D vdev->cxl.comp_regs_region.fd_offset; + uint32_t cap =3D 0; + + if (pread(vdev->vbasedev.fd, &cap, 4, off + VFIO_CXL_HDM_CAP) !=3D 4) { + return 1; + } + switch (le32_to_cpu(cap) & 0xf) { + case 0: return 1; + case 1: return 2; + case 2: return 4; + case 3: return 6; + case 4: return 8; + default: return 1; + } +} + +/* + * The guest committed or tore down an endpoint decoder. Walk each decoder= in + * the trapped HDM block, not just decoder 0. QEMU does not honor the base= the + * guest programmed: the host decoder is firmware-fixed, so QEMU maps the = HDM + * memory at the base of the device's CFMWS window and presents that same = base + * back to the guest (see vfio_cxl_comp_regs_read). The guest works purely= in + * GPA and never sees the host physical base the kernel shadow holds; the = shadow + * base and size are not consulted here. + * + * This generation commits a single, non-interleaved decoder (the kernel + * refuses to bind otherwise) and exposes one HDM memory region, so the wa= lk + * stops at the first committed decoder. Keeping it indexed makes multi-de= coder + * support a kernel-gate relaxation rather than a QEMU rewrite. + */ +static void vfio_cxl_decoder_changed(VFIOPCIDevice *vdev) +{ + VFIOCXL *cxl =3D &vdev->cxl; + VFIODevice *vbasedev =3D &vdev->vbasedev; + off_t off =3D cxl->comp_regs_region.fd_offset; + unsigned n, count =3D vfio_cxl_decoder_count(vdev); + + for (n =3D 0; n < count; n++) { + uint32_t ctrl =3D 0; + + if (pread(vbasedev->fd, &ctrl, 4, + off + VFIO_CXL_HDM_DECODER_CTRL(n)) !=3D 4) { + return; + } + if (!(le32_to_cpu(ctrl) & VFIO_CXL_HDM_CTRL_COMMITTED)) { + continue; + } + + /* + * Map the HDM memory at the guest physical base of the device's C= FMWS + * window, which is also the base QEMU presents to the guest for t= his + * decoder (see vfio_cxl_comp_regs_read). The guest never sees the= host + * physical base the kernel shadow holds. + */ + if (cxl->dpa_mapped && cxl->mapped_base =3D=3D cxl->fmws_base) { + return; + } + /* + * Overlap the CFMWS window MemoryRegion the CXL host bridge alrea= dy + * maps at this base, at a higher priority, so precedence is defin= ed by + * priority rather than subregion insertion order. + */ + memory_region_transaction_begin(); + vfio_cxl_unmap_mem(vdev); + memory_region_add_subregion_overlap(get_system_memory(), + cxl->fmws_base, + cxl->mem_region.mem, 1); + memory_region_transaction_commit(); + cxl->mapped_base =3D cxl->fmws_base; + cxl->dpa_mapped =3D true; + return; + } + + /* No committed decoder in the block: tear down any existing mapping. = */ + vfio_cxl_unmap_mem(vdev); +} + +static uint64_t vfio_cxl_comp_regs_read(void *opaque, hwaddr addr, + unsigned size) +{ + VFIORegion *region =3D opaque; + VFIODevice *vbasedev =3D region->vbasedev; + VFIOPCIDevice *vdev =3D + container_of(region, VFIOPCIDevice, cxl.comp_regs_region); + VFIOCXL *cxl =3D &vdev->cxl; + uint32_t val =3D 0xffffffff; + + if (pread(vbasedev->fd, &val, size, region->fd_offset + addr) !=3D siz= e) { + error_report("vfio-cxl: %s: HDM decoder read at 0x%" HWADDR_PRIx + " failed", vbasedev->name, addr); + } + val =3D le32_to_cpu(val); + + /* + * The kernel shadow holds the host physical base for a firmware-commi= tted + * decoder. Never expose that to the guest: present the guest physical= base + * QEMU maps the HDM memory at (the device's CFMWS window). Any decode= r's + * base registers are virtualized; ctrl, size and the capability heade= r pass + * through. The base registers repeat every VFIO_CXL_HDM_DECODER_STRID= E. + */ + if (addr >=3D VFIO_CXL_HDM_DECODER_BASE_LOW(0) && + (addr - VFIO_CXL_HDM_DECODER_BASE_LOW(0)) % + VFIO_CXL_HDM_DECODER_STRIDE =3D=3D 0) { + val =3D (val & ~VFIO_CXL_HDM_BASE_LOW_MASK) | + ((uint32_t)cxl->fmws_base & VFIO_CXL_HDM_BASE_LOW_MASK); + } else if (addr >=3D VFIO_CXL_HDM_DECODER_BASE_HIGH(0) && + (addr - VFIO_CXL_HDM_DECODER_BASE_HIGH(0)) % + VFIO_CXL_HDM_DECODER_STRIDE =3D=3D 0) { + val =3D (uint32_t)(cxl->fmws_base >> 32); + } + + return val; +} + +static void vfio_cxl_comp_regs_write(void *opaque, hwaddr addr, uint64_t d= ata, + unsigned size) +{ + VFIORegion *region =3D opaque; + VFIODevice *vbasedev =3D region->vbasedev; + VFIOPCIDevice *vdev =3D + container_of(region, VFIOPCIDevice, cxl.comp_regs_region); + uint32_t val =3D cpu_to_le32((uint32_t)data); + + if (pwrite(vbasedev->fd, &val, size, region->fd_offset + addr) !=3D si= ze) { + error_report("vfio-cxl: %s: HDM decoder write at 0x%" HWADDR_PRIx + " failed", vbasedev->name, addr); + return; + } + + /* + * The kernel runs the lock-on-commit FSM in the write above, so the + * committed state is settled by now; a control write on any decoder c= an + * change the mapping. + */ + if (addr >=3D VFIO_CXL_HDM_DECODER_CTRL(0) && + (addr - VFIO_CXL_HDM_DECODER_CTRL(0)) % + VFIO_CXL_HDM_DECODER_STRIDE =3D=3D 0) { + vfio_cxl_decoder_changed(vdev); + } +} + +static const MemoryRegionOps vfio_cxl_comp_regs_ops =3D { + .read =3D vfio_cxl_comp_regs_read, + .write =3D vfio_cxl_comp_regs_write, + .endianness =3D DEVICE_LITTLE_ENDIAN, + .valid =3D { .min_access_size =3D 4, .max_access_size =3D 4 }, + .impl =3D { .min_access_size =3D 4, .max_access_size =3D 4 }, +}; + +/* + * Locate the CXL Device DVSEC (CXL r3.1 8.1.3) in config space. The guest + * triggers a CXL reset by writing its ctrl2 register; QEMU rescans the de= coder + * after that write so the HDM mapping is restored (see vfio_pci_write_con= fig). + * Returns the DVSEC config offset, or 0 if the device does not expose it. + */ +static uint16_t vfio_cxl_find_device_dvsec(PCIDevice *pdev) +{ + uint16_t offset; + + for (offset =3D PCI_CONFIG_SPACE_SIZE; offset; + offset =3D PCI_EXT_CAP_NEXT(pci_get_long(pdev->config + offset)))= { + uint32_t hdr =3D pci_get_long(pdev->config + offset); + + if (PCI_EXT_CAP_ID(hdr) =3D=3D PCI_EXT_CAP_ID_DVSEC && + (pci_get_long(pdev->config + offset + PCI_DVSEC_HEADER1) & 0xf= fff) + =3D=3D CXL_VENDOR_ID && + pci_get_word(pdev->config + offset + PCI_DVSEC_HEADER2) + =3D=3D PCIE_CXL_DEVICE_DVSEC) { + return offset; + } + } + + return 0; +} + /* * Learn the CXL geometry the kernel reports: the HPA-backed HDM memory re= gion * and the trapped HDM decoder block (which BAR carries it and at what off= set). @@ -3900,6 +4211,8 @@ static bool vfio_cxl_setup(VFIOPCIDevice *vdev, Error= **errp) cxl->dpa_size =3D mem_info->size; cxl->comp_bar =3D cap->bar; cxl->hdm_offset =3D cap->offset; + cxl->dvsec_offset =3D vfio_cxl_find_device_dvsec(&vdev->parent_obj); + cxl->af_offset =3D pci_find_capability(&vdev->parent_obj, PCI_CAP_ID_A= F); =20 if (!vfio_cxl_check_topology(vdev, errp)) { return false; @@ -3916,31 +4229,73 @@ static bool vfio_cxl_setup(VFIOPCIDevice *vdev, Err= or **errp) if (vfio_region_mmap(&cxl->mem_region)) { error_setg(errp, "vfio-cxl: %s: failed to mmap the HDM memory regi= on", vbasedev->name); - vfio_region_exit(&cxl->mem_region); - vfio_region_finalize(&cxl->mem_region); - return false; + goto err; } =20 - cxl->machine_done.notify =3D vfio_cxl_bind_fmws; - qemu_add_machine_init_done_notifier(&cxl->machine_done); + /* + * Trap the HDM decoder block: overlay it, priority 1, over the direct= ly + * mapped component BAR, so guest decoder accesses reach the kernel FS= M and + * the commit becomes visible to QEMU. + */ + if (!vdev->bars[cxl->comp_bar].mr) { + error_setg(errp, "vfio-cxl: %s: component BAR %u is not present", + vbasedev->name, cxl->comp_bar); + goto err; + } + if (vfio_region_setup_with_ops(OBJECT(vdev), vbasedev, + &cxl->comp_regs_region, + cxl->comp_regs_region_index, "cxl-comp-= regs", + &vfio_cxl_comp_regs_ops, errp)) { + goto err; + } + memory_region_add_subregion_overlap(vdev->bars[cxl->comp_bar].mr, + cxl->hdm_offset, + cxl->comp_regs_region.mem, 1); + + if (DEVICE(vdev)->hotplugged) { + /* + * The machine is already up, so the CFMWS windows are placed and = the + * binding can be validated now. Report a failure through errp so = a bad + * device_add fails cleanly instead of aborting the running VM. + */ + if (!vfio_cxl_do_bind_fmws(vdev, errp)) { + goto err; + } + } else { + cxl->machine_done.notify =3D vfio_cxl_bind_fmws; + qemu_add_machine_init_done_notifier(&cxl->machine_done); + } =20 cxl->enabled =3D true; =20 return true; + +err: + vfio_cxl_teardown(vdev); + return false; } =20 static void vfio_cxl_teardown(VFIOPCIDevice *vdev) { VFIOCXL *cxl =3D &vdev->cxl; =20 - if (!cxl->enabled) { - return; + if (cxl->comp_regs_region.mem) { + if (vdev->bars[cxl->comp_bar].mr) { + memory_region_del_subregion(vdev->bars[cxl->comp_bar].mr, + cxl->comp_regs_region.mem); + } + vfio_region_exit(&cxl->comp_regs_region); + vfio_region_finalize(&cxl->comp_regs_region); } - qemu_remove_machine_init_done_notifier(&cxl->machine_done); + vfio_cxl_unmap_mem(vdev); if (cxl->mem_region.mem) { vfio_region_exit(&cxl->mem_region); vfio_region_finalize(&cxl->mem_region); } + if (cxl->machine_done.notify) { + qemu_remove_machine_init_done_notifier(&cxl->machine_done); + cxl->machine_done.notify =3D NULL; + } } =20 static void vfio_pci_realize(PCIDevice *pdev, Error **errp) diff --git a/hw/vfio/pci.h b/hw/vfio/pci.h index 6f184520bf..c6c07a2185 100644 --- a/hw/vfio/pci.h +++ b/hw/vfio/pci.h @@ -133,11 +133,16 @@ typedef struct VFIOCXL { uint32_t comp_regs_region_index; /* trapped HDM decoder register block= */ uint32_t comp_bar; /* component BAR carrying that block = */ uint64_t hdm_offset; /* block offset within the component = BAR */ + uint16_t dvsec_offset; /* CXL Device DVSEC offset, 0 if none= */ + uint16_t af_offset; /* PCI Advanced Features cap, 0 if no= ne */ uint64_t dpa_size; /* size of the HDM memory region */ VFIORegion mem_region; /* HDM memory, mapped at committed GP= A */ Notifier machine_done; /* CFMWS validated at machine_done */ hwaddr fmws_base; /* base of the memory window */ uint64_t fmws_size; /* size of that window */ + VFIORegion comp_regs_region; /* trapped HDM decoder block */ + hwaddr mapped_base; /* GPA the HDM memory is mapped at */ + bool dpa_mapped; /* HDM memory currently in system mem= ory */ } VFIOCXL; =20 struct VFIOPCIDevice { --=20 2.25.1 From nobody Fri Aug 28 00:12:35 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nvidia.com); dmarc=pass(p=reject dis=none) header.from=nvidia.com ARC-Seal: i=2; a=rsa-sha256; t=1786626539; cv=pass; d=zohomail.com; s=zohoarc; b=FvhtBbYGGP6rkPgD2o3gDBZQILx76j4gpzgKSNYAVtBddIQegl85o66exzO5MGrf2fgMaTYFh/LCctSxWXx2Dld5D+HbEEWtqsOZnY4sdlf8Rzf2ohThyxhADRoMkQ2F7DOxbPOMrx0fyUVXYwuwnPYO+VIPXlD/G5+IsJTzvZM= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1786626539; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=2J2YZMAYXfJ/rDsdKVkHuV+TkuLcuxaUM7xiofvGgf8=; b=ln16mtxmqt/yOokR/1oJQXNWNx+xug27W1Lj90zHCsxtzl9LPbEx7in34Q4LgroBry2AQGUA4u//JfMYjovDLIJjDL0fO5dbqQfk0g4EAxnldTUucZoSYAP38EEfjh1w0efZTHBaB5dDqu53Ow4e+sAWBb/sbgObQYkzDN9xGJs= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nvidia.com); dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1786626539167733.8559599451669; Thu, 13 Aug 2026 06:08:59 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wuVAU-0006Rq-Hw; Thu, 13 Aug 2026 09:08:22 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wuVAO-0006QW-5S; Thu, 13 Aug 2026 09:08:17 -0400 Received: from mail-northcentralusazlp170100001.outbound.protection.outlook.com ([2a01:111:f403:c105::1] helo=CH1PR05CU001.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wuVAK-0005QS-Tg; Thu, 13 Aug 2026 09:08:14 -0400 Received: from SJ0PR12MB6784.namprd12.prod.outlook.com (2603:10b6:a03:44f::20) by SJ2PR12MB8689.namprd12.prod.outlook.com (2603:10b6:a03:53d::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.315.15; Thu, 13 Aug 2026 13:08:03 +0000 Received: from SJ0PR03CA0077.namprd03.prod.outlook.com (2603:10b6:a03:331::22) by SJ0PR12MB6784.namprd12.prod.outlook.com (2603:10b6:a03:44f::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.315.12; Thu, 13 Aug 2026 13:08:01 +0000 Received: from CO1PEPF00012E7D.namprd03.prod.outlook.com (2603:10b6:a03:331:cafe::a2) by SJ0PR03CA0077.outlook.office365.com (2603:10b6:a03:331::22) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.315.12 via Frontend Transport; Thu, 13 Aug 2026 13:07:58 +0000 Received: from mail.nvidia.com (216.228.117.160) by CO1PEPF00012E7D.mail.protection.outlook.com (10.167.249.52) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.339.3 via Frontend Transport; Thu, 13 Aug 2026 13:07:58 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.46; Thu, 13 Aug 2026 06:07:33 -0700 Received: from nvidia-4028GR-scsim.nvidia.com (10.126.230.37) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Thu, 13 Aug 2026 06:07:27 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=bQ49K/pwSjLXXKrrhrAhl0Z8JpI14qqPNQbftGfsSErsM1y5EuwYQFyYpGDhKa68DGYElxPLD5oQ/1gAXDPzqV44Cg/r2XSzuY8EZ0WDla155C5FGkP05Quu5fTvoV8I5Z3PrnQ0cJFTqBN9CShMm1JTS2S/k0eiuqvX4Ck0BqqngHTsEyhoRt77PqQVMZ7LdWFxJQ6ma6K2YleXgTmNbDn23KXZVP9C8Cn1w30nKgNb4wYdtfg3knPbjeEf4U8E9vduN2FDY8MS/0+VXAi94uHeF4C+B6zE467sPCDjZdaXX+8a+qxmcl0JfD2ui3hkWwmGJG9oY5Rt9VZFszF6KQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=2J2YZMAYXfJ/rDsdKVkHuV+TkuLcuxaUM7xiofvGgf8=; b=c8LaHL61EijcVp6ZDqMylG/goTGc/bD1tbEwMdNjEcP5BLRoIDPcM4W2f4x3UuEYvU+n2C5kU1gVfGJoBgCbiKa0EgVy++2aJ6iDFwb3UHy6TYq0f6y3ZkHgcXEBcGp8ogmolMT46siplpkj6+gipSzi8m/XGPceirt/aPUWS3c+iajXPvlzINeuQ4DDsRivwAYXx3HBLitDVzRX47jVzZ5WNkpF8DYfK5gj1Omye8+yN3gWAkzYutnXgM95D4KoBz9b7doLOKb71t2iD60a35+Wa/OWJoYTd+RlB8e33HJxHFqnBGBDpuMTeJ5tlLgW/UwPODqH+yuJ+uwG7gYHYg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=shazbot.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=2J2YZMAYXfJ/rDsdKVkHuV+TkuLcuxaUM7xiofvGgf8=; b=pj8U5xHj16fjdaNIR653hFgDaB9NJJKEQCOsnzLFtk5GPV6RHKDLAExr7sfzYJ29XSsvX/R4ikISCjMukeC+Oxai+uxu7r1YMQD6xYoUqc4uJQ+xvigk56eMrbXrHV3CzF35rOhnJup9G61O07Da0xecCv/0HavilxkVaMs/JbpgWbG0Wu/09xO4HP9SufkTkbqJjB1Y1TH1MA6jqp9jOreNvFBbnr+R3DqVAaE9jpu0/xNDkozbWdWDsus1tmV/zqPuI5s2Z4mIhuWpdDceJXsOV/PsWfc3lv14vPy/n2E3ra0rEU5JwQaeGK74p/GRstUUMda9olV8bdpKwMUA5g== X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C From: To: , , , , , , , , , , , , , , , CC: , , , , , Subject: [PATCH 08/10] docs/cxl: Document CXL Type-2 device passthrough Date: Thu, 13 Aug 2026 18:36:21 +0530 Message-ID: <20260813130623.2499506-9-mhonap@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20260813130623.2499506-1-mhonap@nvidia.com> References: <20260813130623.2499506-1-mhonap@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.126.230.37] X-ClientProxiedBy: rnnvmail203.nvidia.com (10.129.68.9) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1PEPF00012E7D:EE_|SJ0PR12MB6784:EE_|SJ2PR12MB8689:EE_ X-MS-Office365-Filtering-Correlation-Id: d689f1b9-ca49-41a9-0e30-08def93be5cf X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|376014|7416014|82310400026|23010399003|1800799024|36860700016|921020|6133799003|22082099003|18002099003|56012099006|3023799007|11063799006|10067099003; X-Microsoft-Antispam-Message-Info: CBJ2DZbWoOn5v75hb0vc1zjfX1oBfj78rU32FSJArWRX9tx6NVKH9ODys0CUKXkDIbHemFkCxOyl3N44zXeMoPHZr5x6JhOc9gRP0bdyjo5JqyhqLvthoQOu/UD5SN9pAEvpf5GGXb1mY2NEQdpg3kt1BCyRxMjSiaRhMrndMOZW7okwC7DIjcZYfYtk9fY17AK39JknopZTsdsKGSSaqPABcCG/ajRPjFp0YswhfRB+Ab8gHQzH71yJ8hYMyeZKEUoScgETF10142w6srZibrZ5KQL7bT0+KNhiVLavIVp6G3TOQUtfLXQFC2U8WmnAK/2MmMZFoc+3U90gp4mQPp/VJLrXvIQBo2LsCYsfjeJ1FboQnQvv5kENDW3jgXTILQEG8HotWCMrZlo3vt7Mm1kbMGtn7LNdtgFXqsJFbUODlsx8oyOSI4o1FvRQS4l2bRwzOYQmJyNfLegnQXowfi1lmQGZY3NyEZcynwpYLbS5Zo8/oqrGTB3YS8gH0XCHIUQueMKWKJ02xfI/OsSGhCDb0PK8dof20iWqXxSVp4YCO9/TpUpk/n8bIhTHzv2Xhmzbzca7BPOkjQ2hwK85u2qsBB0kxaO/uxVzOlFciX80725KL7KUMwjep8vtOAAk/J1dObh6w8PkBbVyzftwVNDZ99duiCLLOYk7haLY321hBZMHVZlMa/v7UYGyc8sboZEc/M3Wme+K3H40/sFdtYdN6dFX0EIZjPd/x2ntYlOXfx1HlSC6GCiP62ZxMQIR X-Forefront-Antispam-Report: CIP:216.228.117.160; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge1.nvidia.com; CAT:NONE; SFS:(13230040)(376014)(7416014)(82310400026)(23010399003)(1800799024)(36860700016)(921020)(6133799003)(22082099003)(18002099003)(56012099006)(3023799007)(11063799006)(10067099003); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: Cbtdta/uqyp/WwgqA5RrguDAqIjQgmouT0N64IQImVjoJGLOppJF73Ud+BciIQQTuMR1y5MxztlVDl9w+NT++BY44y6wx9eJDw3FCwarUkAKuh6eB4l86YAJFkQugOrXyu4Y5GP2L4euDxKgbe6gBda++oNUQNBOrjMGCmzlrASAnqIhKp1H9zdEr8vkzjGdFuWJYeEO5pTTHF8XRVc7p81MRWqU9FkxES4GSOJsSCHMURddmU3pqmy3eOtFKyxQuKJubxLvaX2KHEceCZ6CBHajyxfUgKipoC13aGHMlLsPZXPhPekU2SUF+cNDdq9LBM1kyqJVKH3Wv64XBS8RmO13x24wIlWTY+dRO4JmF0/8uJYdV7KPqt+YuZxY/h8+iS89whjI9ckOIcPAl7GJP2O5rofO/aITq0/TAk6JnQmYWDPdLauy+eUHq7sL38Oh X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Aug 2026 13:07:58.5849 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: d689f1b9-ca49-41a9-0e30-08def93be5cf X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.160]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-CrossTenant-AuthSource: CO1PEPF00012E7D.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ2PR12MB8689 Received-SPF: permerror client-ip=2a01:111:f403:c105::1; envelope-from=mhonap@nvidia.com; helo=CH1PR05CU001.outbound.protection.outlook.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.759, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FORGED_SPF_HELO=1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_PASS=-0.001, SPF_NONE=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @Nvidia.com) X-ZM-MESSAGEID: 1786626540958158500 Content-Type: text/plain; charset="utf-8" From: Manish Honap Describe the supported single-endpoint passthrough topology, the command line, and the kernel dependency, and note that a guest CXL reset is handled by the host kernel rather than QEMU. Signed-off-by: Manish Honap --- docs/system/devices/cxl.rst | 46 +++++++++++++++++++++++++++++++++++++ 1 file changed, 46 insertions(+) diff --git a/docs/system/devices/cxl.rst b/docs/system/devices/cxl.rst index 9d0771cdfd..33a9fb169b 100644 --- a/docs/system/devices/cxl.rst +++ b/docs/system/devices/cxl.rst @@ -418,6 +418,52 @@ Volatile Memory device:: -device cxl-type3,bus=3Droot_port13,volatile-memdev=3Dvmem0,id=3Dcxl-vme= m0 \ -M cxl-fmw.0.targets.0=3Dcxl.1,cxl-fmw.0.size=3D4G =20 +Type 2 device passthrough +------------------------- + +A CXL Type 2 device (an accelerator with host-managed device memory) can be +assigned to a guest with vfio-pci, so the guest reaches the device memory +through its own CXL stack. This pairs with the kernel vfio-cxl series: the +host kernel fixes the device memory at a host physical range before the gu= est +sees the device, and QEMU maps that range at the guest physical address the +guest programs into its endpoint HDM decoder. The guest chooses only the G= PA; +the host physical placement is never reprogrammed by the guest. + +The device memory reaches the guest as a CXL fixed memory window (``cxl-fm= w``), +advertised through CEDT, exactly like a Type 3 window; there is no separate +device memory-map slot. Only the endpoint decoder is programmed, by the gu= est, +so the host bridge must stay in HDM passthrough mode: a single ``cxl-rp`` = under +the ``pxb-cxl`` and no ``hdm_for_passthrough``. Switch-attached and interl= eaved +topologies are rejected. + +Example command line:: + + -machine q35,cxl=3Don + -device pxb-cxl,bus_nr=3D12,bus=3Dpcie.0,id=3Dcxl.1 + -device cxl-rp,port=3D0,bus=3Dcxl.1,id=3Drp0,chassis=3D0,slot=3D2 + -device vfio-pci,host=3D,bus=3Drp0,id=3Dcxl-ep0 + -M cxl-fmw.0.targets.0=3Dcxl.1,cxl-fmw.0.size=3D + +The window must be a single-target ``cxl-fmw`` that targets the device's +``pxb-cxl`` and is at least the size of the device memory. The guest trigg= ers a +CXL reset by writing the CXL Device DVSEC; the host kernel runs that seque= nce, +so QEMU has no reset handling of its own. + +On arm64, pass ``accel=3Don`` to the ``arm-smmuv3`` when passing a Type 2 = device +through. The accelerated SMMUv3 describes the device MSI doorbell to the g= uest +through an IORT Reserved Memory Range (RMR) node, which reserves a fixed g= uest +IOVA, so OSPM must preserve the firmware PCI resource assignments rather t= han +re-enumerate them. QEMU requests that through PCI Firmware ``_DSM`` functi= on 5 +(preserve firmware PCI configuration), which on arm64 is emitted for the C= XL +host bridge only when the machine requests preserved configuration, that i= s, +the accelerated SMMUv3 path. This ``_DSM`` is not about the CXL decoder: +vfio-pci keeps the host BAR fixed, and QEMU's trapped component-register b= lock +is a subregion of the guest BAR MemoryRegion, so it follows any guest-visi= ble +BAR relocation on its own. x86 does not use the accelerated SMMU, so it do= es +not advertise or implement preserve-configuration function 5; the CXL host +bridge still emits the ``_DSM`` method, but its function 0 returns an empty +support mask. + Deprecations ------------ =20 --=20 2.25.1 From nobody Fri Aug 28 00:12:35 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nvidia.com); dmarc=pass(p=reject dis=none) header.from=nvidia.com ARC-Seal: i=2; a=rsa-sha256; t=1786626655; cv=pass; d=zohomail.com; s=zohoarc; b=OZvcOkV12NHNzjGzVVjO18b1oJCt5bTH8SpuBGkbEoFS1b5C5tdabJDCaDUG8Ftg8LbX6d5WlqBIHv9iMWlcUaeakHBMy3bRcvZ6JPXE2pHvLU52FQTBOB25b42+Wjc0SPYEvR7FzYEjV4o9wELNiy4p+wpiRs+cIa6bwH4ciw8= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1786626655; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=w2z+jBRT+Mb/Xc1Xaxj7RiosNs44MfVTACU9UAfKXYY=; b=dWzSKDnpRyE+BzIWLqnNX5v4wPV5XhCzhDma5FR/8QmD34oXvMIIj7GZxvzcMi46MBoCrWkQwO2uJX1js/K125EQ69wktxxpXdLMVWg7laSzhvgJ73XLlcY7CmFj4C8SxjTQHtpQ4brj8el0/jgGhyJlOhbZqZ57vpRbnj+p038= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nvidia.com); dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 17866266555929.688723392316206; Thu, 13 Aug 2026 06:10:55 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wuVAd-0006b3-Kn; Thu, 13 Aug 2026 09:08:31 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wuVAY-0006St-4d; Thu, 13 Aug 2026 09:08:26 -0400 Received: from mail-westus2azlp170100005.outbound.protection.outlook.com ([2a01:111:f403:c005::5] helo=CO1PR03CU002.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wuVAU-0005RF-8f; Thu, 13 Aug 2026 09:08:24 -0400 Received: from IA0PR12MB8349.namprd12.prod.outlook.com (2603:10b6:208:407::17) by BN5PR12MB9510.namprd12.prod.outlook.com (2603:10b6:408:2ac::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.315.14; Thu, 13 Aug 2026 13:08:11 +0000 Received: from SJ0PR13CA0095.namprd13.prod.outlook.com (2603:10b6:a03:2c5::10) by IA0PR12MB8349.namprd12.prod.outlook.com (2603:10b6:208:407::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.315.15; Thu, 13 Aug 2026 13:08:08 +0000 Received: from CO1PEPF00012E7F.namprd03.prod.outlook.com (2603:10b6:a03:2c5:cafe::7d) by SJ0PR13CA0095.outlook.office365.com (2603:10b6:a03:2c5::10) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.339.3 via Frontend Transport; Thu, 13 Aug 2026 13:08:08 +0000 Received: from mail.nvidia.com (216.228.117.160) by CO1PEPF00012E7F.mail.protection.outlook.com (10.167.249.54) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.339.3 via Frontend Transport; Thu, 13 Aug 2026 13:08:08 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.46; Thu, 13 Aug 2026 06:07:40 -0700 Received: from nvidia-4028GR-scsim.nvidia.com (10.126.230.37) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Thu, 13 Aug 2026 06:07:34 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=wLjLiQuEkEoHpEzH8FfOslekZg3cWRmhxxaZxN2Q3WMFRYEyNzNoW1eMZYwj0zcw0BIOoL0C4WwEmpnOyfYSXVRvWre5eh2+R6VDg5EK2SMd4Xfgr5ywUvp0xbpCiOxGGsxMdUjLzaQHZenmPn7WWDczRfqDD8DTXlwIQMwusZfcwpLkBL5a/qxp5V4DULdAUAuT9oq8RPbWmaMcNVlzP+0/crEiROsCJBYBqhaLLgTvw/eYaX8vQt7kxmLp/ym6WE3dmZ+QEMNHDREFOJAnCxXHlLllZPGKrhr83rCYTp5txlmbwou0yIH4Ejc0UYudee3cp77SbgbOo1eVlDWKhA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=w2z+jBRT+Mb/Xc1Xaxj7RiosNs44MfVTACU9UAfKXYY=; b=aB3nepKRTC3K7hyeyEkEm/uf2ySr4PGzVrMrMmEzsBU4KHpIAUNEdtep3fWwkv+PXB0hfwi2zpEHGV1RKogjQeQF+Rbnp4agC/bUyoqO6m4SAQVMv+4xS85aqCYTWG+218teOqqA4/6W0Krx1FF2szW+NIC+OC/tVVzQ2j29H0XykB8ULK/FqBA4XRqbuVHNPKTRz/0nhXhI4QYkufmmco1qm5avgzUypbZFErT/Rmgbrt82vRWBqnmWaH5t/JKCCqeo/C375RQmTHZMugw8RZYc8njipVaDSMJLTzQWX0HDm49sYmL8Y2i8h6KNDVow4KfDZ/R+q0ajlgXRtMhFOQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=shazbot.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=w2z+jBRT+Mb/Xc1Xaxj7RiosNs44MfVTACU9UAfKXYY=; b=o/O/kYMv98xWBZxcgCoPUxweLNmJJahPs00+k2lZDtz74mn74nlHThmmzduoIbKrRu5DzC/RhoVVYi9ElQkFf36SrkV9TfK9zpmfL2iVgGEQrN5z+A8w8oVIRYVpWtVj7aSaF5w0L6L0XV25dSGX/3Wp8jhhxw2WuFW4YhvoEPwKdWCKzqDk4sJlbS3MTUYLS6a0T9WKiAc6Nd7MG5bFBv1589QOBfmUi2B/c3/12vVsYwoGWO2Hmt2vSMSpyESfMCnigIB37jjCTEgbi8bSI5XxyZJYqqp2EhZIBAx7KpQX07oi/eI/wC6DAsvQuq10EVfUXBbejQ7eaesEipAs6A== X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C From: To: , , , , , , , , , , , , , , , CC: , , , , , Subject: [PATCH 09/10] hw/arm/smmu-common: Allow pxb-cxl as an SMMUv3 primary bus Date: Thu, 13 Aug 2026 18:36:22 +0530 Message-ID: <20260813130623.2499506-10-mhonap@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20260813130623.2499506-1-mhonap@nvidia.com> References: <20260813130623.2499506-1-mhonap@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.126.230.37] X-ClientProxiedBy: rnnvmail203.nvidia.com (10.129.68.9) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1PEPF00012E7F:EE_|IA0PR12MB8349:EE_|BN5PR12MB9510:EE_ X-MS-Office365-Filtering-Correlation-Id: b1fd7ff3-8bd2-4257-38e9-08def93beb8d X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|82310400026|7416014|376014|1800799024|23010399003|36860700016|11063799006|56012099006|10067099003|22082099003|18002099003|921020; X-Microsoft-Antispam-Message-Info: W4a3NJp3s1F9y4+hvwu8LLJGl3EhuXkCouDh5OiCiDtdcRnQyf+BA4X9ypKmrIzPEA8XyWN2KExuAV1fDCygM8cjbW8mdx2m+HDE9BT8Y2DHYztJUSTUFS22vXKLIxkrj+vm+S3om1TsirHtmn/ZumfNZBKwwI8uba37cBPuyZOUaeBCnA8xx70la1oIHeyM/FJErF4h0cRyUUBVoMzhsu87PpDcUdONtRcz5zgxKwxbXokuNUN+5MPFE6KyyDYvnpw2/BNcnzFA8vdB8JugObWkerv8Sv8OLT5Py1zw6NOaE/4PeKPg0AF1eAeH6dXdBu5FctpSvEQWRzqcvlvDus6I9YsLI0rJIGDj5oDw2+VH5cX+6LawHPaMxTIStkFKSqHwS3fsIydk3OHLvFVPmN2B+bhMkIxybBqFzHVq/4sdCklT2uGe8MjixF8JZeZevyH4j/2ge95NHO3kLZtqZcyhx72hrsjZPhdgwWRHiheD41bLx+e+fjHGMTt4gqdayzHC6wgOiJhywAAIBNb8vl8mtWlBRqI8fC5+x/YmqJHxS06/aEG/w3Gy+KHkhn3SKfZIt2XDwjz9l3ll8tiBgYN2BhjAOvI+G5/V1BORVnI+Ypdc0+q74yQfyiqwFfq6Sza362XfPfuhFKBCyLNpl4TBQ+dyhIuU+aGmnp3Jb0p9WxYE54JYnZCgkn485W3RAARdTo1CrddG3V+AW6Zl/DHqXjO5TJe9bqaMLTqglI19556XiWcYX/crxGUyOAT2 X-Forefront-Antispam-Report: CIP:216.228.117.160; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge1.nvidia.com; CAT:NONE; SFS:(13230040)(82310400026)(7416014)(376014)(1800799024)(23010399003)(36860700016)(11063799006)(56012099006)(10067099003)(22082099003)(18002099003)(921020); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 1zRN9/4qeKDKOYEV3+zN4TwuaGitjBc6XDQpvXH60KVdAsjZV92hrt6HO40i9eFYzonizYPaUoQu6ZYP37vFMoHn+Xcbgii7v5Tf30EaqdPJ6r0RVcMX4SxvoFQR8kNMmmZKCYNWas6N+D1OmbgPpMwZGOvzsuoi3Y2U/4XsHu8y3J5uH3Tgi10Ws1INDJqSRJvELx2nSR42lB1m548ydCXDFr9b1ZzonsL51qtnKECKGA3k0HOPx9z8kBB+9MppvdQh5hW1b5MhGUq67oqMYvmkIv/VJfMxuced4G2xsD/ZpsawvGBfDYS+omwgtGzs2nM/rG0WTBiFoDFNDlUyW9UzYW6ylOTpeVfojicHTt3UBNsRBriz8SC6bBcNLmKiOX9AEy79djt5+A3A0kujCs+dpU8ioIt7aLx4NthoLTtOh8Vww3CeowNJcwGNOnum X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Aug 2026 13:08:08.1916 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: b1fd7ff3-8bd2-4257-38e9-08def93beb8d X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.160]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-CrossTenant-AuthSource: CO1PEPF00012E7F.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN5PR12MB9510 Received-SPF: permerror client-ip=2a01:111:f403:c005::5; envelope-from=mhonap@nvidia.com; helo=CO1PR03CU002.outbound.protection.outlook.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.759, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FORGED_SPF_HELO=1, SPF_HELO_PASS=-0.001, SPF_NONE=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @Nvidia.com) X-ZM-MESSAGEID: 1786626657764158500 Content-Type: text/plain; charset="utf-8" From: Manish Honap The SMMUv3 primary-bus check only accepted pxb-pcie as a valid extra root complex, so a CXL device behind a pxb-cxl could not reach the IOMMU: attaching an arm-smmuv3 to a pxb-cxl bus failed at realize with "SMMU should be attached to a default PCIe root complex (pcie.0) or a pxb-pcie based root complex". A pxb-cxl uses the same PCIe-compatible bus as a pxb-pcie, so accept it too. A CXL Type-2 device passed through with vfio-pci sits behind a pxb-cxl and needs SMMU translation for its DMA and ATS just like a pxb-pcie device. Signed-off-by: Manish Honap --- hw/arm/smmu-common.c | 19 ++++++++++--------- 1 file changed, 10 insertions(+), 9 deletions(-) diff --git a/hw/arm/smmu-common.c b/hw/arm/smmu-common.c index 8e40ba603d..42fb098ab2 100644 --- a/hw/arm/smmu-common.c +++ b/hw/arm/smmu-common.c @@ -963,19 +963,20 @@ static void smmu_base_realize(DeviceState *dev, Error= **errp) s->iommu_ops =3D &smmu_ops; } /* - * We only allow default PCIe Root Complex(pcie.0) or pxb-pcie based e= xtra - * root complexes to be associated with SMMU. + * We only allow the default PCIe root complex (pcie.0) or pxb-pcie / + * pxb-cxl based extra root complexes to be associated with SMMU. */ if (pci_bus_is_express(pci_bus) && pci_bus_is_root(pci_bus) && object_dynamic_cast(OBJECT(pci_bus)->parent, TYPE_PCI_HOST_BRIDGE)= ) { /* - * This condition matches either the default pcie.0, pxb-pcie, or - * pxb-cxl. For both pxb-pcie and pxb-cxl, parent_dev will be set. - * Currently, we don't allow pxb-cxl as it requires further - * verification. Therefore, make sure this is indeed pxb-pcie. + * pcie.0 has no parent_dev; pxb-pcie and pxb-cxl do. Accept both = bus + * types explicitly so other root complexes are still rejected. A + * pxb-cxl uses the same PCIe-compatible bus, so a CXL device behi= nd it + * needs the SMMU just like a pxb-pcie one. */ if (pci_bus->parent_dev) { - if (!object_dynamic_cast(OBJECT(pci_bus), TYPE_PXB_PCIE_BUS)) { + if (!object_dynamic_cast(OBJECT(pci_bus), TYPE_PXB_PCIE_BUS) && + !object_dynamic_cast(OBJECT(pci_bus), TYPE_PXB_CXL_BUS)) { goto out_err; } } @@ -990,8 +991,8 @@ static void smmu_base_realize(DeviceState *dev, Error *= *errp) return; } out_err: - error_setg(errp, "SMMU should be attached to a default PCIe root compl= ex" - "(pcie.0) or a pxb-pcie based root complex"); + error_setg(errp, "SMMU should be attached to a default PCIe root compl= ex " + "(pcie.0), a pxb-pcie, or a pxb-cxl based root complex"); } =20 /* --=20 2.25.1 From nobody Fri Aug 28 00:12:35 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nvidia.com); dmarc=pass(p=reject dis=none) header.from=nvidia.com ARC-Seal: i=2; a=rsa-sha256; t=1786626582; cv=pass; d=zohomail.com; s=zohoarc; b=DDNmdch/3g4Wv2/A57Ha7LPv70zyjbVV5Kp4qysn2vwp7+IkB2LZI7FWwDwjuAnhzoxmAlfyM7RKmcVoRN3TqzVIVA8PKQJ/noyS/hKtqvHnpjnqc7GxxayFhOvnCJ0Yz2RApZxuE3Y2pNe7lnLJX8YnDLCUXxR+pJMlyg0xQ4M= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1786626582; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=9FvIKcb+MYAelhMutav1XB/AV9ekizN7pGUNHpWVnXg=; b=QhJuLOw0VV3ZvtWTDiMnhQjeG2lS5nG9fuaA/oG2B6GfBd4VNWlud3Msu+hZaJXNwa9u6NCNUYxrgtb8fCQ8qRlHU5ebH1rU5prZk9BVN0GzoaMd7SUF8A81VdE72CH3NN7PxaT0ZfdFupm9Pk5btwxaU2Xcw6kZb2FWjdk1YG4= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nvidia.com); dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1786626582760152.11586150222; Thu, 13 Aug 2026 06:09:42 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wuVAb-0006Y4-Nx; Thu, 13 Aug 2026 09:08:29 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wuVAY-0006Ss-4W; Thu, 13 Aug 2026 09:08:26 -0400 Received: from mail-westus3azlp170120001.outbound.protection.outlook.com ([2a01:111:f403:c107::1] helo=PH8PR06CU001.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wuVAT-0005Rl-VN; Thu, 13 Aug 2026 09:08:25 -0400 Received: from SA3PR12MB7858.namprd12.prod.outlook.com (2603:10b6:806:306::17) by IA1PR12MB9467.namprd12.prod.outlook.com (2603:10b6:208:594::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.292.25; Thu, 13 Aug 2026 13:08:13 +0000 Received: from BN1PR14CA0016.namprd14.prod.outlook.com (2603:10b6:408:e3::21) by SA3PR12MB7858.namprd12.prod.outlook.com (2603:10b6:806:306::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.315.14; Thu, 13 Aug 2026 13:08:12 +0000 Received: from BN2PEPF000044A1.namprd02.prod.outlook.com (2603:10b6:408:e3:cafe::5d) by BN1PR14CA0016.outlook.office365.com (2603:10b6:408:e3::21) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.315.14 via Frontend Transport; Thu, 13 Aug 2026 13:08:12 +0000 Received: from mail.nvidia.com (216.228.117.161) by BN2PEPF000044A1.mail.protection.outlook.com (10.167.243.152) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.339.3 via Frontend Transport; Thu, 13 Aug 2026 13:08:11 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.46; Thu, 13 Aug 2026 06:07:46 -0700 Received: from nvidia-4028GR-scsim.nvidia.com (10.126.230.37) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Thu, 13 Aug 2026 06:07:40 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=DVFw0+HaeQYPOVoccC70f7kK3SAcKfm2Bf3BChVb9X9oZEDgjAyW1hKcu30k4gM7YftZTBzZuKj6y0qbyQgh9O7DfdofBCX0+LDxR0UKjW+F0LDmNIV1nU5fQhZZwyaH9NUDUezR8MNo5Na7IVLwlG4q2jhiybILgYssdOTl3rf6j4ZVA3CtON424YdTpdfoNDwSMDEzgEU6LU3O7Zm+gImYpqGj9z/yUXwNUfxG7eYzB6w1SgNrkP+/v9IsfaAh0veFNf1RnRSuNXsTB2HbXoEemSTGqXGhi73787RyUKRDykiiKhzz/gs/KcQ2bqeR2pPFWeS5SEYxpCwUnDvmQg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=9FvIKcb+MYAelhMutav1XB/AV9ekizN7pGUNHpWVnXg=; b=Q18UN8GB5Wh/UmRlVivxHYGDUIoSehL9uOqM/M3A6vogAVV6GaWO/oYNLBBZyywyhBU0ePi/OW25Deu0CBHfZEUu2ihH7yosUsipRCOtzwXjEPupDxZKYOYl1SGsFWHW+XsGbYZCt25AM/dUtf3BOE0LwUENJPDL7upp8YN0ZOhOvCmnCptt6ChMNPeoA6o8YvK1NNFK+E2aXRGTDsepNCMMwwjEkCy/nYFeDP4wpU0RloJQm9CBAUtSAfFFehr5jxLfdcTOAPkffCTCVx3bas4hlYYANGmc99zTe5iPB9UUlXQ9NTwr3de6vCsy7AwPTF/LwDXe9SO0Wz596QEl2Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=shazbot.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=9FvIKcb+MYAelhMutav1XB/AV9ekizN7pGUNHpWVnXg=; b=BM3x4B3kMkwqZfTP3RJpYW7NT7KoJfvvsmjgVoEmhBPvpF7tCulPFFVUJPP30Ar6bi8wDauDN+uTJCU5fV844ZU5RWZhO4GbNv2M5xZk5AVKTvTDL1o/XQCNPhBWnQoNoa55wEr8gRKbFQbypXj4FgCCefzp4fQ1QOSJLlX1xLGnVvhwpfNvXD6QD8mhykfexk9rwAdriyLPDtah4Q4EIdKTFVzIfYArb9e1R5xV7Gp/daas0+j16M6Feff0QB/nl4g7ScajIvFZSTQwvZa/aFJ4cwSx382QzFxP8PVVX2kAkUBoyoFeXt27qKfb9m60qt75Lr03eDZlBrdU9WtShw== X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C From: To: , , , , , , , , , , , , , , , CC: , , , , , Subject: [PATCH 10/10] hw/pci-host: Emit a _DSM on pxb-cxl to preserve firmware PCI config Date: Thu, 13 Aug 2026 18:36:23 +0530 Message-ID: <20260813130623.2499506-11-mhonap@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20260813130623.2499506-1-mhonap@nvidia.com> References: <20260813130623.2499506-1-mhonap@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.126.230.37] X-ClientProxiedBy: rnnvmail203.nvidia.com (10.129.68.9) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN2PEPF000044A1:EE_|SA3PR12MB7858:EE_|IA1PR12MB9467:EE_ X-MS-Office365-Filtering-Correlation-Id: 8c2f990e-0828-41e0-5642-08def93bedc8 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|1800799024|82310400026|23010399003|376014|36860700016|7416014|56012099006|10067099003|11063799006|6133799003|18002099003|3023799007|921020|22082099003; X-Microsoft-Antispam-Message-Info: yxuz5JTjVoYWVOyrs0CiNxiifJafZHpujpyj5rzR8+mfH3nQ8U3upZoLJ3zhr+XP7Zdo493pM4RN8XmJMuCDT5/Yr2uxfaJ7pvc+UqD9MXlt95mAAr50OM/ESAfIDdadHYXmOhd/FrLs0rX94CAl3emvPcMIb1wpGJC133Jao9fj2b9wx0vbhS3su6BgvaOu+Rt1+0mgbvm7TLraZEwm2Lwu8IXgGf7tg8SN2nY8IHiBT6XkPJnkhz+uMdgj5rmVx40VyW9Hl4DAHL1EoPt+tDuZ0H5I2Wa+ssziI9JCs9MQHxDF/q0Nucwk0FymggKWqjx1ZAvf9MZJUvj9khD+9m0wjopbsxeCAr/LsMQXbnMvzzOLKp9AJzIXlvTVTMwrRY6YaLodiR5Gu80AvYx//7fJkG+FWsBoj8K3ETqXfi8AHrLyK6C6OB5wmjwXIADi2km3OsVJGdKaEP+OqibvHtW7CfVGxTSiHXfm5ZXj0VdzTJUDeOkXrljwZqnY1g9WvL7ZKtLdit/8uoqgQSh9a9UFRWXnnQLBLsT0N7LGbK1jWVNR+KLT407KnJ9qjOmy/jlp2/GRTaZcB6pGBf6wFFkqFPW5eWK8Vzju0dHzMFSAxHa1SQfvkKXUt7BMy3phQ4m9+i4v3vzQDa70qeR/it7nCrEK56XLrP5ZOlxUHmOQ3NPdut0cTTc79EsriTjSfJNaEky0JkJYkcgRqTuZ60yWBvgrTRpmoISyg2nf3n0P8zAKB/xa0MHF36YV/Lq9 X-Forefront-Antispam-Report: CIP:216.228.117.161; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge2.nvidia.com; CAT:NONE; SFS:(13230040)(1800799024)(82310400026)(23010399003)(376014)(36860700016)(7416014)(56012099006)(10067099003)(11063799006)(6133799003)(18002099003)(3023799007)(921020)(22082099003); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: oCkDStG2NSwdCAn4bQ6lviYB5M4qlj/M9eD+++Depuh5jvF95NFYVnduVySFhq2qaZ1Tr2FcbdQy0Rq458Gh8ni0QKmnu1thbIZd6NC+TB8u3QPopskpF6OKn/9hEPrDfW6NFTwsbEb005jqXnnz+xhlinl6cFBLr5VqDy5G99RKTE18ReSsoUoCg0JH41GU0zsHK+qE9ajpDYaOO9+YPpVpLwq8dI/46o//EfIGcDu/5xRZKseqnW1WPUImIcubBS0IMGkpF/etPSAb/scMgfMnQ2tfft4lRf12cs8oRzT1LP0bTTAFR9vN06cgfopR1XW1j2mNc/fFrtbSpG+Yte0Ktv/aVcU5pv8cp6jI2gVf4vtxp2w8Vw2P9xx25ENXyXHr+3GCpm1zSg/j+AufWZJtXrjT/Qrqgwq2DNmN9AbijkQaCpVurv+xpZjiVh2O X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Aug 2026 13:08:11.9557 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 8c2f990e-0828-41e0-5642-08def93bedc8 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.161]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-CrossTenant-AuthSource: BN2PEPF000044A1.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB9467 Received-SPF: permerror client-ip=2a01:111:f403:c107::1; envelope-from=mhonap@nvidia.com; helo=PH8PR06CU001.outbound.protection.outlook.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.759, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_NONE=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @Nvidia.com) X-ZM-MESSAGEID: 1786626583161158500 Content-Type: text/plain; charset="utf-8" From: Manish Honap A pxb-cxl host bridge had no PCI host-bridge _DSM method, so it could not emit function 5 (preserve firmware PCI configuration) even when the machine asks OSPM to keep the firmware resource assignments. That preservation is required by the accelerated SMMUv3 (accel=3Don), which describes MSI 1:1 mappings through IORT RMR nodes. An RMR reserves a fixed IOVA range, so OSPM must not re-enumerate and reassign the PCI resources underneath it. virt sets pci_preserve_config for that case and expects every host bridge to emit the _DSM; the pxb-cxl bridge was the one that did not, so a CXL topology under an accelerated SMMU lost the guarantee. This is not about the component register BAR moving. vfio-pci marks every BAR dword ALL_VIRT, so a guest BAR write updates the virtual config only and cannot move the host resource, and QEMU's trapped comp-regs region is a subregion of the BAR MemoryRegion, so it follows any guest-visible relocation while the kernel's physical decoder mapping stays fixed. The CXL.mem window is delivered through CEDT/CFMWS and is likewise unaffected by PCI resource assignment. Wire preserve_config through GPEXConfig into the CXL host bridge OSC path so pxb-cxl bridges emit the _DSM function 5. Rename build_cxl_osc_method() to acpi_dsdt_add_cxl_host_bridge_methods() to match the pxb-pcie analogue acpi_dsdt_add_host_bridge_methods(), since it now appends both the _OSC and the _DSM. x86 passes false as it does not use the accelerated SMMU. Signed-off-by: Manish Honap --- hw/acpi/Kconfig | 1 + hw/acpi/cxl-stub.c | 2 +- hw/acpi/cxl.c | 4 +++- hw/acpi/pci.c | 40 +++++++++++++++++++++++++++++++++++++++ hw/i386/acpi-build.c | 2 +- hw/pci-host/gpex-acpi.c | 42 ++--------------------------------------- include/hw/acpi/cxl.h | 2 +- include/hw/acpi/pci.h | 1 + 8 files changed, 50 insertions(+), 44 deletions(-) diff --git a/hw/acpi/Kconfig b/hw/acpi/Kconfig index daabbe6cd1..9490b75c94 100644 --- a/hw/acpi/Kconfig +++ b/hw/acpi/Kconfig @@ -88,3 +88,4 @@ config ACPI_ERST config ACPI_CXL bool depends on ACPI + select ACPI_PCI diff --git a/hw/acpi/cxl-stub.c b/hw/acpi/cxl-stub.c index 15bc21076b..d7c6731975 100644 --- a/hw/acpi/cxl-stub.c +++ b/hw/acpi/cxl-stub.c @@ -6,7 +6,7 @@ #include "hw/acpi/aml-build.h" #include "hw/acpi/cxl.h" =20 -void build_cxl_osc_method(Aml *dev) +void acpi_dsdt_add_cxl_host_bridge_methods(Aml *dev, bool preserve_config) { g_assert_not_reached(); } diff --git a/hw/acpi/cxl.c b/hw/acpi/cxl.c index f92f7fa3d5..7607ba500f 100644 --- a/hw/acpi/cxl.c +++ b/hw/acpi/cxl.c @@ -23,6 +23,7 @@ #include "hw/pci/pci_host.h" #include "hw/cxl/cxl.h" #include "hw/cxl/cxl_host.h" +#include "hw/acpi/pci.h" #include "hw/mem/memory-device.h" #include "hw/acpi/acpi.h" #include "hw/acpi/aml-build.h" @@ -320,11 +321,12 @@ static Aml *__build_cxl_osc_method(void) return method; } =20 -void build_cxl_osc_method(Aml *dev) +void acpi_dsdt_add_cxl_host_bridge_methods(Aml *dev, bool preserve_config) { aml_append(dev, aml_name_decl("SUPP", aml_int(0))); aml_append(dev, aml_name_decl("CTRL", aml_int(0))); aml_append(dev, aml_name_decl("SUPC", aml_int(0))); aml_append(dev, aml_name_decl("CTRC", aml_int(0))); aml_append(dev, __build_cxl_osc_method()); + aml_append(dev, build_pci_host_bridge_dsm_method(preserve_config)); } diff --git a/hw/acpi/pci.c b/hw/acpi/pci.c index 8c7ed10479..1a20f8469b 100644 --- a/hw/acpi/pci.c +++ b/hw/acpi/pci.c @@ -351,3 +351,43 @@ Aml *build_pci_host_bridge_osc_method(bool enable_nati= ve_pcie_hotplug) aml_append(method, aml_return(aml_arg(3))); return method; } + +Aml *build_pci_host_bridge_dsm_method(bool preserve_config) +{ + Aml *method =3D aml_method("_DSM", 4, AML_NOTSERIALIZED); + Aml *UUID, *ifctx, *ifctx1, *buf; + uint8_t byte_list[1] =3D {0}; + + /* + * PCI Firmware Specification 3.0 + * 4.6.1. _DSM for PCI Express Slot Information + * The UUID in _DSM in this context is + * {E5C937D0-3553-4D7A-9117-EA4D19C3434D} + */ + UUID =3D aml_touuid("E5C937D0-3553-4D7A-9117-EA4D19C3434D"); + ifctx =3D aml_if(aml_equal(aml_arg(0), UUID)); + ifctx1 =3D aml_if(aml_equal(aml_arg(2), aml_int(0))); + if (preserve_config) { + /* support functions other than 0, specifically function 5 */ + byte_list[0] =3D 0x21; + } + buf =3D aml_buffer(1, byte_list); + aml_append(ifctx1, aml_return(buf)); + aml_append(ifctx, ifctx1); + if (preserve_config) { + Aml *ifctx2 =3D aml_if(aml_equal(aml_arg(2), aml_int(5))); + /* + * 0 - The operating system must not ignore the PCI configuration = that + * firmware has done at boot time. + */ + aml_append(ifctx2, aml_return(aml_int(0))); + aml_append(ifctx, ifctx2); + } + + aml_append(method, ifctx); + + byte_list[0] =3D 0; + buf =3D aml_buffer(1, byte_list); + aml_append(method, aml_return(buf)); + return method; +} diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c index 8837b69687..a24bff8c88 100644 --- a/hw/i386/acpi-build.c +++ b/hw/i386/acpi-build.c @@ -1019,7 +1019,7 @@ build_dsdt(GArray *table_data, BIOSLinker *linker, aml_append(aml_pkg, aml_eisaid("PNP0A08")); aml_append(aml_pkg, aml_eisaid("PNP0A03")); aml_append(dev, aml_name_decl("_CID", aml_pkg)); - build_cxl_osc_method(dev); + acpi_dsdt_add_cxl_host_bridge_methods(dev, false); } else if (pci_bus_is_express(bus)) { aml_append(dev, aml_name_decl("_HID", aml_eisaid("PNP0A08"= ))); aml_append(dev, aml_name_decl("_CID", aml_eisaid("PNP0A03"= ))); diff --git a/hw/pci-host/gpex-acpi.c b/hw/pci-host/gpex-acpi.c index d9820f9b41..014f647c7f 100644 --- a/hw/pci-host/gpex-acpi.c +++ b/hw/pci-host/gpex-acpi.c @@ -51,45 +51,6 @@ static void acpi_dsdt_add_pci_route_table(Aml *dev, uint= 32_t irq, } } =20 -static Aml *build_pci_host_bridge_dsm_method(bool preserve_config) -{ - Aml *method =3D aml_method("_DSM", 4, AML_NOTSERIALIZED); - Aml *UUID, *ifctx, *ifctx1, *buf; - uint8_t byte_list[1] =3D {0}; - - /* PCI Firmware Specification 3.0 - * 4.6.1. _DSM for PCI Express Slot Information - * The UUID in _DSM in this context is - * {E5C937D0-3553-4D7A-9117-EA4D19C3434D} - */ - UUID =3D aml_touuid("E5C937D0-3553-4D7A-9117-EA4D19C3434D"); - ifctx =3D aml_if(aml_equal(aml_arg(0), UUID)); - ifctx1 =3D aml_if(aml_equal(aml_arg(2), aml_int(0))); - if (preserve_config) { - /* support functions other than 0, specifically function 5 */ - byte_list[0] =3D 0x21; - } - buf =3D aml_buffer(1, byte_list); - aml_append(ifctx1, aml_return(buf)); - aml_append(ifctx, ifctx1); - if (preserve_config) { - Aml *ifctx2 =3D aml_if(aml_equal(aml_arg(2), aml_int(5))); - /* - * 0 - The operating system must not ignore the PCI configuration = that - * firmware has done at boot time. - */ - aml_append(ifctx2, aml_return(aml_int(0))); - aml_append(ifctx, ifctx2); - } - - aml_append(method, ifctx); - - byte_list[0] =3D 0; - buf =3D aml_buffer(1, byte_list); - aml_append(method, aml_return(buf)); - return method; -} - static void acpi_dsdt_add_host_bridge_methods(Aml *dev, bool enable_native_pcie_hotp= lug, bool preserve_config) @@ -164,7 +125,8 @@ void acpi_dsdt_add_gpex(Aml *scope, struct GPEXConfig *= cfg) aml_append(dev, aml_name_decl("_CRS", crs)); =20 if (is_cxl) { - build_cxl_osc_method(dev); + acpi_dsdt_add_cxl_host_bridge_methods(dev, + cfg->preserve_config= ); } else { /* pxb bridges do not have ACPI PCI Hot-plug enabled */ acpi_dsdt_add_host_bridge_methods(dev, true, diff --git a/include/hw/acpi/cxl.h b/include/hw/acpi/cxl.h index 8f22c71530..6fe6c9c58d 100644 --- a/include/hw/acpi/cxl.h +++ b/include/hw/acpi/cxl.h @@ -24,7 +24,7 @@ void cxl_build_cedt(GArray *table_offsets, GArray *table_data, BIOSLinker *linker, const char *oem_id, const char *oem_table_id, CXLState *cxl_state); -void build_cxl_osc_method(Aml *dev); +void acpi_dsdt_add_cxl_host_bridge_methods(Aml *dev, bool preserve_config); void build_cxl_dsm_method(Aml *dev); =20 #endif diff --git a/include/hw/acpi/pci.h b/include/hw/acpi/pci.h index 20b672575f..c7de33f8cf 100644 --- a/include/hw/acpi/pci.h +++ b/include/hw/acpi/pci.h @@ -42,6 +42,7 @@ void build_pci_bridge_aml(AcpiDevAmlIf *adev, Aml *scope); void build_srat_generic_affinity_structures(GArray *table_data); =20 Aml *build_pci_host_bridge_osc_method(bool enable_native_pcie_hotplug); +Aml *build_pci_host_bridge_dsm_method(bool preserve_config); Aml *build_pci_bridge_edsm(void); =20 #endif --=20 2.25.1