From nobody Thu Dec 18 05:05:03 2025 Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2043.outbound.protection.outlook.com [40.107.244.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 566FF1A2547; Tue, 3 Jun 2025 17:22:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.244.43 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1748971382; cv=fail; b=hnEetwp2O/yZtnhNtnnSO0FTRVlEUG3/IIR5AcpvoQg0/mBEyrm8ktQPVJSSi9EXhMrb2Q1dZY9oPAzcE18vRujhhSRY+6qhIyBdbb2cJ8inkhmTEnb6GxUJgh323Oo4+u1D7MnwrASZjgQImYsgHDV0OpGcMomxT2Q/xuhKmXs= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1748971382; c=relaxed/simple; bh=vRFb4Kf1zyQHFZm3yG9NJtXxnUThjBe0qm9rETopzig=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=iL339shzEkuCKOOVWXJ3ZGW36GKmSmE/Q52Pq8NMbSYoDU1TYmX2NuYmcV1a61CU7JkKBy549Llk89OgY18dwbq4+GbXmcXEB9EuLBoTXAQf36mts/tKJcRvMOO0iWqGuXHSTkXLT2/cnu3PARihqpOC/mIFbB8T7BYs97ehny8= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=Cn5GVnbm; arc=fail smtp.client-ip=40.107.244.43 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="Cn5GVnbm" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=Vom5G7KsqSkZ1Dy0U8gWSH3LhoFKYUK6hA9lRDQ3lEiUERKPBlZmyHJyFKaxA2cjuSunMiLatb44j+OU5lVpenZYBnl1JVuODbbiut7O0erGNzhViBbOz9MtvFwu8/ALLWbpYG4OvXztYJ+uS9vEWWB/sgAsdd48Yh50FSHHxW2obDZN9HnPi+r85odyY/WGuBBybDerYB2KimRVgaalLHIZyRlV2ikvas7PJDb3sPQC3yLb8pEC94tBG2MYhVUmJWye1Dn+Ux7CQ19NVsR+PXl1lD+L9dkwT+lWnzUKO0QkeAENeSaYC/SZaAWfqsyYu6x1hI15GmQCVpBG/K0Vyw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=QAH8vnSytbnu8wbW8RRM891DW4O6CXLGVKT9aqu9n8U=; b=WWRWW9AQuLebB6JUEmRUKRMZeKNn7CnNtVs1KD4Vdr2cYY6BDSs3/XUXn+gQfiYR1KR49k1WPnlmBBFQw+jTF1ciiqakTzh+f9qM1uzZHlDjB5xl/Px076XnDlIzwg9KY3fOyIH6PdI4je62CKupSXP9Apti7gjq4vK3/1Gf5XeOuJEu7eBNS9deWSdDCX0WNW9RxFtT/LiEysRKTsotYRoT2griv6UHI1QCxcGXlNt6OZz5KXlEl0qwhfNJoXXbQ0yaQO764yGjFM9WMd7x45Ghk5y/qez5pDGl+6vszC4KTzoxkZ9Q89TmJf7yxKySWqAa9Zt0ONAHYFerLAAxkA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=stgolabs.net smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=QAH8vnSytbnu8wbW8RRM891DW4O6CXLGVKT9aqu9n8U=; b=Cn5GVnbmjch/fM1DIG0Tn+Kn7DPqeQVdhfRCCPb1nBQmJYIlp+V+2fiUTzOEGNjy1VDnbzj+uYGyVA3ur80Zt8+yZvKU12lPdjPSR318iUjb5hJRpdDOUpyhjq+MK8Wb11xR1R+qbIsV2mFaGdy+iL73tjlBAaGmTy702/BHGyA= Received: from MN2PR06CA0026.namprd06.prod.outlook.com (2603:10b6:208:23d::31) by IA1PR12MB6020.namprd12.prod.outlook.com (2603:10b6:208:3d4::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8792.34; Tue, 3 Jun 2025 17:22:57 +0000 Received: from BL6PEPF00020E60.namprd04.prod.outlook.com (2603:10b6:208:23d:cafe::a9) by MN2PR06CA0026.outlook.office365.com (2603:10b6:208:23d::31) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8769.18 via Frontend Transport; Tue, 3 Jun 2025 17:22:57 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by BL6PEPF00020E60.mail.protection.outlook.com (10.167.249.21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.8792.29 via Frontend Transport; Tue, 3 Jun 2025 17:22:57 +0000 Received: from ethanolx7ea3host.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Tue, 3 Jun 2025 12:22:55 -0500 From: Terry Bowman To: , , , , , , , , , , , , , , , , , , , , , , , , , , Subject: [PATCH v9 01/16] PCI/CXL: Add pcie_is_cxl() Date: Tue, 3 Jun 2025 12:22:24 -0500 Message-ID: <20250603172239.159260-2-terry.bowman@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250603172239.159260-1-terry.bowman@amd.com> References: <20250603172239.159260-1-terry.bowman@amd.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL6PEPF00020E60:EE_|IA1PR12MB6020:EE_ X-MS-Office365-Filtering-Correlation-Id: dc989922-ceee-40c7-0da8-08dda2c3483b X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|36860700013|376014|7416014|1800799024|921020; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?94ZE79Unk92VWG38Q4BOhiGAykZ4R6wUH/X8CJqxTv5E0+1nN7RVpcQ8xgBm?= =?us-ascii?Q?/FG7djZ+5cjU1hHR15Rj5esZT0TDkurTFBOspoMPC1GpUfLJusIexoBI9FvG?= =?us-ascii?Q?rws8lnU3lwR1MTtepqAs4pBInrWJYBDyafUtQi/mGQElz+85KY67XsnGDojp?= =?us-ascii?Q?CFSjxJiXbnwTIMTNjHSvq/5EseCj1HpxsK0C8c0kwhqnAkMMMcQOe6MSXi+0?= =?us-ascii?Q?qJuGqZMi5dZ6WQESbkcokrzumHZ/8jFWnFiBvMhTjgVXbExYKtafbibWcOtI?= =?us-ascii?Q?t2V38BEN57zlJXRh3WcGtdwsSaZQND/o2aDYOxIOthP9DNoTkWTrjT1mxKZE?= =?us-ascii?Q?bIvcw5giOo36hygtY/Vjn4ZPbQFxOCX+nh0XrnUmqLnykmBr4QNfVMq1PrJq?= =?us-ascii?Q?9p9xqeKuTSLm7KQchlT77uUO1apKPhE4edq9sbFCFzDkzg99Sw3uxNMK9Nib?= =?us-ascii?Q?p2tJF6K508DAkzEfhuNNWr0ddn4bD8KntGxtXtOVrShBXrRM82pzyHYS5wD1?= =?us-ascii?Q?Y1wEeTVpwaYUL82yBqXcR2rG+jv8Gl4yWq7ZW2Ka6LfpB5fn4ftp6pJZ4oHM?= =?us-ascii?Q?JFUrUPma7PhW70CsT+OGgn2EXtPWBLZg66/ZsdDfeEK7gvLZAvmc5M090D+x?= =?us-ascii?Q?JjU2q7IF204uy6XsVBlzAQVAhUQ9iPBCnRiz06p+16iHgfN7LVZLHOUrJH32?= =?us-ascii?Q?YzSoIXAOh6GCwMMrsPhvliGawvR64dvCMI8NLS0vCWW32IMWp+P+aAoXS84m?= =?us-ascii?Q?A3RTGzyrawjmnwsmzPG15lTmmcUOnHM9ELMDrZphz+h0FOzV49pN8r2WyvFM?= =?us-ascii?Q?5Ak774rVB4s+ywhbmphTosUZUKMWxm5WQVF9pbppF5HhAjbIGZ8wNsC4L5pI?= =?us-ascii?Q?xvbZtZeGZbtKjyq6fQgq5Onf7fWxuzMYhPA2KEIjyp7QCk74Oof288wgHjII?= =?us-ascii?Q?u5fRCD/9mZ0pYlABIqO71Eg03iYClwHhBEPUDTO3fs3gflcHocjkpvEQ0lvf?= =?us-ascii?Q?KY9nvDzvzZmWEqtlc0Mn+f1f5PJzqUE7NAIUXXVay4kO13VGySEDFdTcZwJq?= =?us-ascii?Q?rkgho8xd3pWRVmgsAfAUCUfKQTy6yW25Iy9SbIsh2esUIyUQXxyLngxqO9DR?= =?us-ascii?Q?JZ6ltktdIEfZhcuOR57+jO9Ys5izDJcvYbdAbTozZxu1sfYW/V/Bt+S9+J8Y?= =?us-ascii?Q?U3YfesGgnkej81GKhzeN6YidFm2FWGnbSXe7PDmeuVSuAbxnUkw9s3N9O1eT?= =?us-ascii?Q?VmFADDGDFKw4MlTKMh2ncRVBusj/ZrDpxSub4/Ank4rPaXhCsPRgg98MgQbr?= =?us-ascii?Q?tOx8Enw4XLNcpTc2NuKtxqgt34LhRSqw3ynRT0LpT1Do+pIGk/93Pgfjsuce?= =?us-ascii?Q?3KosYW6gSRXhAoZDDA9zdkAungwpy2yKPjIri5XNYxKSS/syrM5gLePwGr68?= =?us-ascii?Q?0eUKnlymBjbP6NvzCkNrukXOoE5TuR0hxCK5BuwhfGktwhi83QPsQpFZvYjJ?= =?us-ascii?Q?ZKtZe6L/WR1JL5ANBSq6G69mpOqGvHk0mns5cBCrLvC0BvLJ9So7GB4xow?= =?us-ascii?Q?=3D=3D?= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(82310400026)(36860700013)(376014)(7416014)(1800799024)(921020);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Jun 2025 17:22:57.0180 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: dc989922-ceee-40c7-0da8-08dda2c3483b X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BL6PEPF00020E60.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB6020 Content-Type: text/plain; charset="utf-8" CXL and AER drivers need the ability to identify CXL devices. Add set_pcie_cxl() with logic checking for CXL Flexbus DVSEC presence. The CXL Flexbus DVSEC presence is used because it is required for all the CXL PCIe devices.[1] Add boolean 'struct pci_dev::is_cxl' with the purpose to cache the CXL Flexbus presence. Add function pcie_is_cxl() to return 'struct pci_dev::is_cxl'. [1] CXL 3.1 Spec, 8.1.1 PCIe Designated Vendor-Specific Extended Capability (DVSEC) ID Assignment, Table 8-2 Signed-off-by: Terry Bowman Reviewed-by: Ira Weiny Reviewed-by: Dave Jiang Reviewed-by: Kuppuswamy Sathyanarayanan --- drivers/pci/probe.c | 10 ++++++++++ include/linux/pci.h | 6 ++++++ include/uapi/linux/pci_regs.h | 8 +++++++- 3 files changed, 23 insertions(+), 1 deletion(-) diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c index 364fa2a514f8..aa29b4b98ad1 100644 --- a/drivers/pci/probe.c +++ b/drivers/pci/probe.c @@ -1691,6 +1691,14 @@ static void set_pcie_thunderbolt(struct pci_dev *dev) dev->is_thunderbolt =3D 1; } =20 +static void set_pcie_cxl(struct pci_dev *dev) +{ + u16 dvsec =3D pci_find_dvsec_capability(dev, PCI_VENDOR_ID_CXL, + PCI_DVSEC_CXL_FLEXBUS); + if (dvsec) + dev->is_cxl =3D 1; +} + static void set_pcie_untrusted(struct pci_dev *dev) { struct pci_dev *parent =3D pci_upstream_bridge(dev); @@ -2021,6 +2029,8 @@ int pci_setup_device(struct pci_dev *dev) /* Need to have dev->cfg_size ready */ set_pcie_thunderbolt(dev); =20 + set_pcie_cxl(dev); + set_pcie_untrusted(dev); =20 if (pci_is_pcie(dev)) diff --git a/include/linux/pci.h b/include/linux/pci.h index 51e2bd6405cd..bff3009f9ff0 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -455,6 +455,7 @@ struct pci_dev { unsigned int is_hotplug_bridge:1; unsigned int shpc_managed:1; /* SHPC owned by shpchp */ unsigned int is_thunderbolt:1; /* Thunderbolt controller */ + unsigned int is_cxl:1; /* Compute Express Link (CXL) */ /* * Devices marked being untrusted are the ones that can potentially * execute DMA attacks and similar. They are typically connected @@ -746,6 +747,11 @@ static inline bool pci_is_vga(struct pci_dev *pdev) return false; } =20 +static inline bool pcie_is_cxl(struct pci_dev *pci_dev) +{ + return pci_dev->is_cxl; +} + #define for_each_pci_bridge(dev, bus) \ list_for_each_entry(dev, &bus->devices, bus_list) \ if (!pci_is_bridge(dev)) {} else diff --git a/include/uapi/linux/pci_regs.h b/include/uapi/linux/pci_regs.h index ba326710f9c8..c50ffa75d5fc 100644 --- a/include/uapi/linux/pci_regs.h +++ b/include/uapi/linux/pci_regs.h @@ -1215,9 +1215,15 @@ /* Deprecated old name, replaced with PCI_DOE_DATA_OBJECT_DISC_RSP_3_TYPE = */ #define PCI_DOE_DATA_OBJECT_DISC_RSP_3_PROTOCOL PCI_DOE_DATA_OBJECT_DISC_= RSP_3_TYPE =20 -/* Compute Express Link (CXL r3.1, sec 8.1.5) */ +/* Compute Express Link (CXL r3.2, sec 8.1) + * + * Note that CXL DVSEC id 3 and 7 to be ignored when the CXL link state + * is "disconnected" (CXL r3.2, sec 9.12.3). Re-enumerate these + * registers on downstream link-up events. + */ #define PCI_DVSEC_CXL_PORT 3 #define PCI_DVSEC_CXL_PORT_CTL 0x0c #define PCI_DVSEC_CXL_PORT_CTL_UNMASK_SBR 0x00000001 +#define PCI_DVSEC_CXL_FLEXBUS 7 =20 #endif /* LINUX_PCI_REGS_H */ --=20 2.34.1 From nobody Thu Dec 18 05:05:03 2025 Received: from NAM04-DM6-obe.outbound.protection.outlook.com (mail-dm6nam04on2045.outbound.protection.outlook.com [40.107.102.45]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 181D11C54AF; Tue, 3 Jun 2025 17:23:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.102.45 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1748971394; cv=fail; b=h7iJGk+fkQy6S0YrjFwX2t1mwnGXB5foXrij8bk/LDpK0N4ycf4Bhnijsshz+1fK18tHLjSqXHYRWJ+0W5YAQAsOy8+zi6wPE8VtVXkNLgsx/dLm3IvLKr8URwAwiOt+WNjC2/GbYQo6Euhbf39S8e7E3F7pIVThJFyVkjXC4sA= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1748971394; c=relaxed/simple; bh=d6+JA9gP4iBccKopeBo3MvMuzo1MxxKtftZaq54P2hQ=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=WsI7G6JvvJ5fX1yCTS1jcDiQGQ0lMuXCdFBNKirS555knjUuL8XgD2CCvbU6TDIzznXKdnavRGWzeZdiOG8umVsQ4m90fTbXT59/tEde9fgcYG3BWz+sHubtrmj+59/1K/VnqamRP2NFtpOKqI/cIfGJKLi4MCwTKX6F9yT/sus= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=TfbVeFVE; arc=fail smtp.client-ip=40.107.102.45 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="TfbVeFVE" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=Qs/xerfVDqcgQZrbzl7tM33gy6ak0se4WlRyUqDwivKd/SKIwVtG+DBuuvYHdTRONcLTIbEOxbBOhafcKxe/bZ37hQRn8GlYNLLZWF766FaSWzqrcjDkgFiYRJJegqEiLh05gweLfDR3gWv64alNkjdDd6+dBqZGsqxf4HvwSGVgf83wY0frJ27uTHX23tUYmWN/b5yZnSIwPVaDrHHRGT62zthmoGNq8TOvtgCGyCf+rxhsgeiGOwMUUXdWb4FvEpZvIk2FrVYcWskcc+1md7EmCCdTHZ3tEfjFUvjmw3adpODPv4KVYkNjPhOn9yACdxviehExUeEo6Io0wVxCzg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=3ix7M1tI8ZzrHjOlmVpgoBCzCH6Xog06En+h667QNQ4=; b=u6/seE/QdITwcbOtL+PsSvRiH0WB94T4zgpvYmOK7f+n1syuZgT047gNo6Xd/rw3oQDkkA7xcVct956KJ9yA3d8b/iqNhgIP9sc2wn47HfYkvK6WIpreVlBw8SJSDsO707NxSPgeZRWz+jjh57RPtR34R+GhQJFLae3FElZk/PvAVoiYC2Zb1y2cdA+tSar9BuqrpxDtWhlhXcy7N+rWhgPdUxvIkNs8tEPJcdRH+sxeFa8+hIeV7AWb+3OSfqCwwIRII6buJ/W9jyLk1ac4S43Qkq20P2HQee5DD2S+lhb8kTgiOjdjeMo1knlplY1gSsef0XQV9TNP6YfxtHMt9g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=stgolabs.net smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=3ix7M1tI8ZzrHjOlmVpgoBCzCH6Xog06En+h667QNQ4=; b=TfbVeFVE57L9/LMN58rcDC2zvctbomAH/spA24eYtf6hhsCV59L4GBW8BKM/5ZKFkXXpLdBRFch8KOcPT0x8BX3MOtfiu+rY0IEggQYNm7B/5lBJwkW3JYf8q53IZQVuCenGmpBUQq9j6tLfwWCbByD2+I4xucKJw1V+Ew4G/f8= Received: from BL0PR02CA0123.namprd02.prod.outlook.com (2603:10b6:208:35::28) by SN7PR12MB7130.namprd12.prod.outlook.com (2603:10b6:806:2a2::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8792.34; Tue, 3 Jun 2025 17:23:09 +0000 Received: from BL6PEPF00020E66.namprd04.prod.outlook.com (2603:10b6:208:35:cafe::95) by BL0PR02CA0123.outlook.office365.com (2603:10b6:208:35::28) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8769.18 via Frontend Transport; Tue, 3 Jun 2025 17:23:08 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by BL6PEPF00020E66.mail.protection.outlook.com (10.167.249.27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.8792.29 via Frontend Transport; Tue, 3 Jun 2025 17:23:08 +0000 Received: from ethanolx7ea3host.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Tue, 3 Jun 2025 12:23:07 -0500 From: Terry Bowman To: , , , , , , , , , , , , , , , , , , , , , , , , , , Subject: [PATCH v9 02/16] PCI/AER: Report CXL or PCIe bus error type in trace logging Date: Tue, 3 Jun 2025 12:22:25 -0500 Message-ID: <20250603172239.159260-3-terry.bowman@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250603172239.159260-1-terry.bowman@amd.com> References: <20250603172239.159260-1-terry.bowman@amd.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL6PEPF00020E66:EE_|SN7PR12MB7130:EE_ X-MS-Office365-Filtering-Correlation-Id: 1333cdbb-f5fe-435c-3832-08dda2c34f01 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|1800799024|36860700013|82310400026|7416014|921020; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?rN4+4GNktO+COEt80wyGanpWORdleXjYeJmVZjWsnhj+JOaPPP8iu5lfJbqh?= =?us-ascii?Q?mN2pH6/Mh1h1Bd4yVwJtZdQwvRcSXJjhDFY6Yv8JKV6m7kza/nvpJSivh48n?= =?us-ascii?Q?wEf8dS65MHylQHFXQjh7ujYazMgDZ9iQChBPKCxvsAIoo5jEE2yXhmDknRjI?= =?us-ascii?Q?WgAJ/WN8OpMoRbvK3UuDurif59Xfb7v+bYtLdMXTXI9ABPRGqfUSJ8puWyxP?= =?us-ascii?Q?Pr+owLLaGoVmptxm3MbseBVCk4AKv40l3kdjlZWCP8DgwUiFc5q2wbQrh+q8?= =?us-ascii?Q?hQPgtKc6GXXqYjy1wUJ+PR4MO0zte1vLw/oqahQ8JiI7bP4RwV4Pd+arX5sL?= =?us-ascii?Q?4sZfaCUoMEyWhPYRIsH2FyjVgoNGHrwBgFDc4HA/WrWp4FJwDTRW4FG8apjN?= =?us-ascii?Q?lyuRNesR0ht9ykyS+TJtJVjlePWlHb3IYA6iagjQNxmuNzJWiXr7NEDAbwvb?= =?us-ascii?Q?bHs/WOWxt0iXaF8/M1VUyGJkAkycexV346wRRP8wXOOFRFl2r80KR9jzBWFY?= =?us-ascii?Q?AxjPRhIT0ABSGHcgIS6WJmctD20Yn7XNWUN2EwxxKldRq35MypjQCmUKGuVe?= =?us-ascii?Q?V3FVJMCiv3jKgrKcUHAI+KS0tMopm4g72qH7e3hsFxkTvwZCTZbxH4ZgBxVE?= =?us-ascii?Q?EL/NEba6s8hWYGdk1XoiyGKUlSPaigBvYwoTI49es8PmpH9kjUPTmkvVXC0w?= =?us-ascii?Q?kz8gY37Wss9jUUeaiNaHRLtjagP/cY4KK3V9E8z+oq0IHR6tvgdA3NBkIVA4?= =?us-ascii?Q?H3J2eLDtq6zXPP/LQATYEopJDaZkEQsJaT3/BX2RiOmJxj6GGFLDxDd/XSvE?= =?us-ascii?Q?/O6D5KQveK3GuJ27gNsy3Ece+HL3P6UsHUjWxlLA+t84zCVwh5/BFx2TEhRP?= =?us-ascii?Q?wtBsxNl5BYNJpd7ehuyFbySpyYe812UrGC/LO/OM6+4eP8PA1ZdoQ/wy2/Yk?= =?us-ascii?Q?LRfVW2+W/TlDiuMNgTA/jIBPMsKPoIHpab5S7iuLGyx5tkndnhY9Oy+rvQjn?= =?us-ascii?Q?2zlR2PtQ34kdbe9Lx7MmlVMGxmGqlTTFn4s5e4vyCqw6dzHD+jWG5wDXJ0An?= =?us-ascii?Q?tZeTUrtVvYY0Yxq8B2d0A1Nkvrv9tbApTk3u6j5WDyi1SEdNgSlxYsPJ2Fi2?= =?us-ascii?Q?PTj2EhccDWn5LeOeMgcmWMg1vqgJAH7GEPmL0vXkM4lzW/8uNPj+iU8bk+Ht?= =?us-ascii?Q?ylK+UJrRVygjkL6N3WL0MWE2FKjmQ+12a9y/Zo6vwtfVudV2oDRq770Ugbw1?= =?us-ascii?Q?WfExmMupzTknGFAftrcfFd2rnQA0E8V2qv0R/pW0PBbPsKsFhLOCe2xTeuEX?= =?us-ascii?Q?OTmTND7E3JRKuq2ag2clOf7WFREOWGcrEI4dJqNPGtTzoAIQ3qKMyKxrqt6T?= =?us-ascii?Q?fh+tfAF6M1x7qHRuErqMS6LN7AfiqwjKop7zC4gRHqMcXcimBSJFdHBSEyZ6?= =?us-ascii?Q?T69V1Zx5VTH3aEKl7ZqlADmaE+56eGoctS4CE5Xk+rp9AsPllNC5znVjsSqS?= =?us-ascii?Q?1ZeXlcLN438JA1DpURVd6tH3gG30guzKFoE/B3x0jGZTjGDSEe3JR8DWJw?= =?us-ascii?Q?=3D=3D?= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(376014)(1800799024)(36860700013)(82310400026)(7416014)(921020);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Jun 2025 17:23:08.3815 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 1333cdbb-f5fe-435c-3832-08dda2c34f01 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BL6PEPF00020E66.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN7PR12MB7130 Content-Type: text/plain; charset="utf-8" The AER service driver and aer_event tracing currently log 'PCIe Bus Type' for all errors. Update the driver and aer_event tracing to log 'CXL Bus Type' for CXL device errors. This requires the AER can identify and distinguish between PCIe errors and CXL errors. Introduce boolean 'is_cxl' to 'struct aer_err_info'. Add assignment in aer_get_device_error_info() and pci_print_aer(). Update the aer_event trace routine to accept a bus type string parameter. Signed-off-by: Terry Bowman Reviewed-by: Ira Weiny Reviewed-by: Dave Jiang --- drivers/pci/pci.h | 6 ++++++ drivers/pci/pcie/aer.c | 18 ++++++++++++------ include/ras/ras_event.h | 9 ++++++--- 3 files changed, 24 insertions(+), 9 deletions(-) diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h index b81e99cd4b62..d6296500b004 100644 --- a/drivers/pci/pci.h +++ b/drivers/pci/pci.h @@ -588,6 +588,7 @@ static inline bool pci_dev_test_and_set_removed(struct = pci_dev *dev) struct aer_err_info { struct pci_dev *dev[AER_MAX_MULTI_ERR_DEVICES]; int error_dev_num; + bool is_cxl; =20 unsigned int id:16; =20 @@ -604,6 +605,11 @@ struct aer_err_info { struct pcie_tlp_log tlp; /* TLP Header */ }; =20 +static inline const char *aer_err_bus(struct aer_err_info *info) +{ + return info->is_cxl ? "CXL" : "PCIe"; +} + int aer_get_device_error_info(struct pci_dev *dev, struct aer_err_info *in= fo); void aer_print_error(struct pci_dev *dev, struct aer_err_info *info); =20 diff --git a/drivers/pci/pcie/aer.c b/drivers/pci/pcie/aer.c index a1cf8c7ef628..adb4b1123b9b 100644 --- a/drivers/pci/pcie/aer.c +++ b/drivers/pci/pcie/aer.c @@ -698,13 +698,14 @@ static void __aer_print_error(struct pci_dev *dev, =20 void aer_print_error(struct pci_dev *dev, struct aer_err_info *info) { + const char *bus_type =3D aer_err_bus(info); int layer, agent; int id =3D pci_dev_id(dev); const char *level; =20 if (!info->status) { - pci_err(dev, "PCIe Bus Error: severity=3D%s, type=3DInaccessible, (Unreg= istered Agent ID)\n", - aer_error_severity_string[info->severity]); + pci_err(dev, "%s Bus Error: severity=3D%s, type=3DInaccessible, (Unregis= tered Agent ID)\n", + bus_type, aer_error_severity_string[info->severity]); goto out; } =20 @@ -713,8 +714,8 @@ void aer_print_error(struct pci_dev *dev, struct aer_er= r_info *info) =20 level =3D (info->severity =3D=3D AER_CORRECTABLE) ? KERN_WARNING : KERN_E= RR; =20 - aer_printk(level, dev, "PCIe Bus Error: severity=3D%s, type=3D%s, (%s)\n", - aer_error_severity_string[info->severity], + aer_printk(level, dev, "%s Bus Error: severity=3D%s, type=3D%s, (%s)\n", + bus_type, aer_error_severity_string[info->severity], aer_error_layer[layer], aer_agent_string[agent]); =20 aer_printk(level, dev, " device [%04x:%04x] error status/mask=3D%08x/%08= x\n", @@ -729,7 +730,7 @@ void aer_print_error(struct pci_dev *dev, struct aer_er= r_info *info) if (info->id && info->error_dev_num > 1 && info->id =3D=3D id) pci_err(dev, " Error of this Agent is reported first\n"); =20 - trace_aer_event(dev_name(&dev->dev), (info->status & ~info->mask), + trace_aer_event(dev_name(&dev->dev), bus_type, (info->status & ~info->mas= k), info->severity, info->tlp_header_valid, &info->tlp); } =20 @@ -763,6 +764,7 @@ EXPORT_SYMBOL_GPL(cper_severity_to_aer); void pci_print_aer(struct pci_dev *dev, int aer_severity, struct aer_capability_regs *aer) { + const char *bus_type; int layer, agent, tlp_header_valid =3D 0; u32 status, mask; struct aer_err_info info; @@ -784,6 +786,9 @@ void pci_print_aer(struct pci_dev *dev, int aer_severit= y, info.status =3D status; info.mask =3D mask; info.first_error =3D PCI_ERR_CAP_FEP(aer->cap_control); + info.is_cxl =3D pcie_is_cxl(dev); + + bus_type =3D aer_err_bus(&info); =20 pci_err(dev, "aer_status: 0x%08x, aer_mask: 0x%08x\n", status, mask); __aer_print_error(dev, &info); @@ -797,7 +802,7 @@ void pci_print_aer(struct pci_dev *dev, int aer_severit= y, if (tlp_header_valid) pcie_print_tlp_log(dev, &aer->header_log, dev_fmt(" ")); =20 - trace_aer_event(dev_name(&dev->dev), (status & ~mask), + trace_aer_event(dev_name(&dev->dev), bus_type, (status & ~mask), aer_severity, tlp_header_valid, &aer->header_log); } EXPORT_SYMBOL_NS_GPL(pci_print_aer, "CXL"); @@ -1215,6 +1220,7 @@ int aer_get_device_error_info(struct pci_dev *dev, st= ruct aer_err_info *info) /* Must reset in this function */ info->status =3D 0; info->tlp_header_valid =3D 0; + info->is_cxl =3D pcie_is_cxl(dev); =20 /* The device might not support AER */ if (!aer) diff --git a/include/ras/ras_event.h b/include/ras/ras_event.h index 14c9f943d53f..080829d59c36 100644 --- a/include/ras/ras_event.h +++ b/include/ras/ras_event.h @@ -297,15 +297,17 @@ TRACE_EVENT(non_standard_event, =20 TRACE_EVENT(aer_event, TP_PROTO(const char *dev_name, + const char *bus_type, const u32 status, const u8 severity, const u8 tlp_header_valid, struct pcie_tlp_log *tlp), =20 - TP_ARGS(dev_name, status, severity, tlp_header_valid, tlp), + TP_ARGS(dev_name, bus_type, status, severity, tlp_header_valid, tlp), =20 TP_STRUCT__entry( __string( dev_name, dev_name ) + __string( bus_type, bus_type ) __field( u32, status ) __field( u8, severity ) __field( u8, tlp_header_valid) @@ -314,6 +316,7 @@ TRACE_EVENT(aer_event, =20 TP_fast_assign( __assign_str(dev_name); + __assign_str(bus_type); __entry->status =3D status; __entry->severity =3D severity; __entry->tlp_header_valid =3D tlp_header_valid; @@ -325,8 +328,8 @@ TRACE_EVENT(aer_event, } ), =20 - TP_printk("%s PCIe Bus Error: severity=3D%s, %s, TLP Header=3D%s\n", - __get_str(dev_name), + TP_printk("%s %s Bus Error: severity=3D%s, %s, TLP Header=3D%s\n", + __get_str(dev_name), __get_str(bus_type), __entry->severity =3D=3D AER_CORRECTABLE ? "Corrected" : __entry->severity =3D=3D AER_FATAL ? "Fatal" : "Uncorrected, non-fatal", --=20 2.34.1 From nobody Thu Dec 18 05:05:03 2025 Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2058.outbound.protection.outlook.com [40.107.92.58]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 333071D5173; Tue, 3 Jun 2025 17:23:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.92.58 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1748971407; cv=fail; b=FnKs3lm0yvvdcr8LAz6PxgczdOv4u6nh9rEb/Sj2soM4eGVxPtuOvyUDaj3temxl8S3f8NQ2X3Kg0wbNw/xv8WJjIoqmlZ74kii+Yux1fFsL0aVBkjDqk4F+unFdSo99+pKh1hD9+CQUey0EGt7CadyFcdInL3Nu7t8CiVshV6M= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1748971407; c=relaxed/simple; bh=tUNA0DREN4WBXkwsWhP1tX9Pi7IYZRjmuO1zin9vMbQ=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=IPbRlxF6zwuFzCRyARVLaqo58n6oP3cnljhzK1baacnpDQCozYUOxhoZkihZSHdY3f+w4MB1ik6fD3H7WpKPpip694UGDCcPGtEHEp1tVTrYwJlSxb41FOiBH19h74sIKfzoI09D79DEWv2dsXOi+e8YxsW13oQn/umK71wZ+JU= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=3mmazKF9; arc=fail smtp.client-ip=40.107.92.58 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="3mmazKF9" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=TCl6jXfZQlrhvuxsXQ+JjzwkDPdWuwk34Luejbnkb9qzKmm+v4NLD3VG0M8B3sfcNHZWXxkqb8gL7MIPfJB9mLF0+aL1papFxqCXkHPU3OteXMzZMO0YET1hHJoXjU2lHMAPAUHVsMQDrR9NYRo5gof79U7Lqo1lbRTM1pIsdYMS/Pt+1aO3j3S/+nNP0tYWIk8tWPFsByjiRERdmDtpmPWfSPPzJyNGDFtw1cH5thWlxI18hduFQqN6RAVDUQCrhJHZk89zDHyVI+C3GvcZs2+MPBfWuhLP/OJjE1XaI7hVrkABHfWnZJMAxiqlEs/lOt/tTrJncuOwVZx3ZCWIew== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=iQa4cM1S/anb8AtJnt0hyjzf46Ann7bQOLhvozVzTAA=; b=P5vtfa6v/CzX9/Uu/194WIgFbwqo1A8lZxLZRHGIX6BQq8W/K6RKqHNN0RBLkbVvxQx39rIK2iQdhVmPaGnaqJ47TqKy0wsaxTs3gGOnhVlqWGXgZPXJ5rnDRJ+8fP2KEkZXztavErjTItEdNAUYSEbkF3beM5B1eaTQqJXscCd2s5pljiCxGnTmvGt759hARrA3ly3ijr2rZrPDLINeekqHgCDw8mL9cac+ZvHnnTWFzeg9R7qxIeNYwv0S0JAXgYigIIfK/45fFoAt8Pr/JzL0UYhzzDkc9EnPhZ8Du+pVPuV+AkDmafa00julx6HwGVzTUl5ovBiYxbkJIGBhcw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=stgolabs.net smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=iQa4cM1S/anb8AtJnt0hyjzf46Ann7bQOLhvozVzTAA=; b=3mmazKF9MOKppqFT6bU5HCM5gfHvcXIBDniu7Yc/vp73WL712yf7ocXxDOBHng8QFv8gtkraSXvLz1X+kbbcJDoJGmVdUvWE7zbVt1XniEk7FYh8RkfqEopcv8AVPm71JYqHX3A0v51IgfQtVVJqa6AiXxzKPeSniAbLwFzteVw= Received: from BL0PR02CA0122.namprd02.prod.outlook.com (2603:10b6:208:35::27) by DS5PPF5C5D42165.namprd12.prod.outlook.com (2603:10b6:f:fc00::64f) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8699.21; Tue, 3 Jun 2025 17:23:20 +0000 Received: from BL6PEPF00020E66.namprd04.prod.outlook.com (2603:10b6:208:35:cafe::1d) by BL0PR02CA0122.outlook.office365.com (2603:10b6:208:35::27) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8769.18 via Frontend Transport; Tue, 3 Jun 2025 17:23:19 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by BL6PEPF00020E66.mail.protection.outlook.com (10.167.249.27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.8792.29 via Frontend Transport; Tue, 3 Jun 2025 17:23:19 +0000 Received: from ethanolx7ea3host.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Tue, 3 Jun 2025 12:23:18 -0500 From: Terry Bowman To: , , , , , , , , , , , , , , , , , , , , , , , , , , Subject: [PATCH v9 03/16] CXL/AER: Introduce kfifo for forwarding CXL errors Date: Tue, 3 Jun 2025 12:22:26 -0500 Message-ID: <20250603172239.159260-4-terry.bowman@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250603172239.159260-1-terry.bowman@amd.com> References: <20250603172239.159260-1-terry.bowman@amd.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL6PEPF00020E66:EE_|DS5PPF5C5D42165:EE_ X-MS-Office365-Filtering-Correlation-Id: e1f586c5-7c81-4630-4a2f-08dda2c355bd X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|1800799024|36860700013|7416014|376014|921020; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?4/AF/cJ6sH3MI8Ab1WW0/9cH3BEeIOrV7qYuDGIFd2sF2IWr6wyHwRGRMTFr?= =?us-ascii?Q?Gk2JJXIdOdZu9WrhzU50kH20XADnHWJBH3qWLaII4dBDLWODynniSoX4gD7E?= =?us-ascii?Q?o6lLcVDdYGq+BlSXhJokQurM2wvSv1v/BQOVtVwTaW5xx/s00uT7ta+YhroY?= =?us-ascii?Q?wmiUhk1kcbskc8DG95sm2iETcgqLm+RtrJbDfAUiH6vrIjv7lpNolx6ez0GW?= =?us-ascii?Q?K1CCuMldZEy/n4PhmAVJD0mhfnTYN/Rd+2oIiw1pAHWGvsBriPN91sOsooeW?= =?us-ascii?Q?IvXROFFe9nfEEuskZcIfcjLvb4QThyU9JmEswuQApBrMrmdE9I5KvPzX7Bb7?= =?us-ascii?Q?lJ/2xrpleTEd7fxhSyaGwhueLIaTrBNm/BaDTa3Lm5v9KbBNoyYpCLtxz1ob?= =?us-ascii?Q?rFNMc9DQSfFxPWnGHWQg4UFbMLxK8qwm/TS6ZgnkVTyEgki9l+3KgvRaCQrB?= =?us-ascii?Q?nM4XT55iPrequiZLWJDGtY+tisNzESxXrM6EojATiGLwuYCyhULwwwnWuUZn?= =?us-ascii?Q?lXsQqkMG4mISEkoJP/iwrQZgqKkb8Gvy6OPHhR/64wz7JhMnwxijS0D7cuZu?= =?us-ascii?Q?CDW8LqfBkNNxgyImdeb29RDKC/Pu5H+2RHg61BfZFUnol7LUKUVQmT/oaL77?= =?us-ascii?Q?p18rVV1Zk2ujMHU1rF2nYwHMKypl0QlcW2AYxoTPb74IiFNp0XJp1nzAlFvT?= =?us-ascii?Q?8c587PR9Za3h7wFTF+swPe8DhQezGBoZQGdVgf8GlLVrvd4OqKtQ+UxouGl8?= =?us-ascii?Q?ofaYExPOZr/KxIvkDdAJCNDGcelj/wgBECT1q1V95AGOR57INRoG4N6CfGQo?= =?us-ascii?Q?mqSuLkbK5D3qOdJ88RNIwRIMglp/Y6Z7q67rHbo6M/wDGMiXVI8scBlB7oMu?= =?us-ascii?Q?wLHwvbquthOH1n41qpmyCVljads2f2KpTvF+guNExXZvFjUM4xFlW8lGOa17?= =?us-ascii?Q?Wk1GJvxOvsEktIHosb+CZbnS5tSMA4YT35A7QsNWvvmoZ5YYEDDMGoYpNJjU?= =?us-ascii?Q?Gb9vAu0X3mk0cDq3Q56riuxFO91SJ9xjo9wwyJFbB2hB/xD+Nx8iUiYDs8Lv?= =?us-ascii?Q?X17FDUCM+KpnTV8SACaEnwd7NeOO8X/zwfjPzdpPuq3ucHuUQwRpjl37fj+C?= =?us-ascii?Q?HW4gemH3c8kEvfzewmoXmLf0zavcXFVx77tPYAAdzoBUgIBbj7JS+oiKKODf?= =?us-ascii?Q?E2Fvge9DNL3eug4stTjlRQooggavbrSadYJpZIZUQVVO4LpS+rT/PR9kVRzj?= =?us-ascii?Q?1Fa1e6E5G+Lku63WivQC6UEU+XwvtAYE9vfCy4smFXvicWvJeWE6gaTXe9yw?= =?us-ascii?Q?6j5fOLYFU8hHiagdDERmeR5iD+uqjQbf67KM32Oau2t2Wq/WN9A3kpeUGnw3?= =?us-ascii?Q?dd0YEJ1+4ezh+YMus9NZ2xhpEGYW1VGTmK7zfyngS6W7XvXD+5HCLS9l6lA7?= =?us-ascii?Q?XsuGVNaTTzGKROa6VTGJCv+4SNOalyXymmuA700SHQG6mNYR4icJBdwCrzxi?= =?us-ascii?Q?vg1bvtDuazQhL8h1/IrTSjKKivhFr7HWh2j1JJOCgDZsu7V6GZ7oeKsq1Q?= =?us-ascii?Q?=3D=3D?= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(82310400026)(1800799024)(36860700013)(7416014)(376014)(921020);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Jun 2025 17:23:19.6819 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: e1f586c5-7c81-4630-4a2f-08dda2c355bd X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BL6PEPF00020E66.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS5PPF5C5D42165 Content-Type: text/plain; charset="utf-8" CXL error handling will soon be moved from the AER driver into the CXL driver. This requires a notification mechanism for the AER driver to share the AER interrupt with the CXL driver. The notification will be used as an indication for the CXL drivers to handle and log the CXL RAS errors. Add a kfifo work queue to be used by the AER driver and CXL driver. The AER driver will be the sole kfifo producer adding work and the cxl_core will be the sole kfifo consumer removing work. Add the boilerplate kfifo support. Add CXL work queue handler registration functions in the AER driver. Export the functions allowing CXL driver to access. Implement registration functions for the CXL driver to assign or clear the work handler function. Introduce function cxl_create_prot_err_info() and 'struct cxl_prot_err_info= '. Implement cxl_create_prot_err_info() to populate a 'struct cxl_prot_err_inf= o' instance with the AER severity and the erring device's PCI SBDF. The SBDF details will be used to rediscover the erring device after the CXL driver dequeues the kfifo work. The device rediscovery will be introduced along with the CXL handling in future patches. Signed-off-by: Terry Bowman --- drivers/cxl/core/ras.c | 31 +++++++++- drivers/cxl/cxlpci.h | 1 + drivers/pci/pcie/aer.c | 132 ++++++++++++++++++++++++++++------------- include/linux/aer.h | 36 +++++++++++ 4 files changed, 157 insertions(+), 43 deletions(-) diff --git a/drivers/cxl/core/ras.c b/drivers/cxl/core/ras.c index 485a831695c7..d35525e79e04 100644 --- a/drivers/cxl/core/ras.c +++ b/drivers/cxl/core/ras.c @@ -5,6 +5,7 @@ #include #include #include +#include #include "trace.h" =20 static void cxl_cper_trace_corr_port_prot_err(struct pci_dev *pdev, @@ -107,13 +108,41 @@ static void cxl_cper_prot_err_work_fn(struct work_str= uct *work) } static DECLARE_WORK(cxl_cper_prot_err_work, cxl_cper_prot_err_work_fn); =20 +#ifdef CONFIG_PCIEAER_CXL + +static void cxl_prot_err_work_fn(struct work_struct *work) +{ +} + +#else +static void cxl_prot_err_work_fn(struct work_struct *work) { } +#endif /* CONFIG_PCIEAER_CXL */ + +static struct work_struct cxl_prot_err_work; +static DECLARE_WORK(cxl_prot_err_work, cxl_prot_err_work_fn); + int cxl_ras_init(void) { - return cxl_cper_register_prot_err_work(&cxl_cper_prot_err_work); + int rc; + + rc =3D cxl_cper_register_prot_err_work(&cxl_cper_prot_err_work); + if (rc) + pr_err("Failed to register CPER AER kfifo (%x)", rc); + + rc =3D cxl_register_prot_err_work(&cxl_prot_err_work); + if (rc) { + pr_err("Failed to register native AER kfifo (%x)", rc); + return rc; + } + + return 0; } =20 void cxl_ras_exit(void) { cxl_cper_unregister_prot_err_work(&cxl_cper_prot_err_work); cancel_work_sync(&cxl_cper_prot_err_work); + + cxl_unregister_prot_err_work(); + cancel_work_sync(&cxl_prot_err_work); } diff --git a/drivers/cxl/cxlpci.h b/drivers/cxl/cxlpci.h index 54e219b0049e..6f1396ef7b77 100644 --- a/drivers/cxl/cxlpci.h +++ b/drivers/cxl/cxlpci.h @@ -4,6 +4,7 @@ #define __CXL_PCI_H__ #include #include "cxl.h" +#include "linux/aer.h" =20 #define CXL_MEMORY_PROGIF 0x10 =20 diff --git a/drivers/pci/pcie/aer.c b/drivers/pci/pcie/aer.c index adb4b1123b9b..5350fa5be784 100644 --- a/drivers/pci/pcie/aer.c +++ b/drivers/pci/pcie/aer.c @@ -114,6 +114,14 @@ struct aer_stats { static int pcie_aer_disable; static pci_ers_result_t aer_root_reset(struct pci_dev *dev); =20 +#if defined(CONFIG_PCIEAER_CXL) +#define CXL_ERROR_SOURCES_MAX 128 +static DEFINE_KFIFO(cxl_prot_err_fifo, struct cxl_prot_err_work_data, + CXL_ERROR_SOURCES_MAX); +static DEFINE_SPINLOCK(cxl_prot_err_fifo_lock); +struct work_struct *cxl_prot_err_work; +#endif + void pci_no_aer(void) { pcie_aer_disable =3D 1; @@ -1004,45 +1012,17 @@ static bool is_internal_error(struct aer_err_info *= info) return info->status & PCI_ERR_UNC_INTN; } =20 -static int cxl_rch_handle_error_iter(struct pci_dev *dev, void *data) +static bool is_cxl_error(struct pci_dev *pdev, struct aer_err_info *info) { - struct aer_err_info *info =3D (struct aer_err_info *)data; - const struct pci_error_handlers *err_handler; + if (!info || !info->is_cxl) + return false; =20 - if (!is_cxl_mem_dev(dev) || !cxl_error_is_native(dev)) - return 0; + /* Only CXL Endpoints are currently supported */ + if ((pci_pcie_type(pdev) !=3D PCI_EXP_TYPE_ENDPOINT) && + (pci_pcie_type(pdev) !=3D PCI_EXP_TYPE_RC_EC)) + return false; =20 - /* Protect dev->driver */ - device_lock(&dev->dev); - - err_handler =3D dev->driver ? dev->driver->err_handler : NULL; - if (!err_handler) - goto out; - - if (info->severity =3D=3D AER_CORRECTABLE) { - if (err_handler->cor_error_detected) - err_handler->cor_error_detected(dev); - } else if (err_handler->error_detected) { - if (info->severity =3D=3D AER_NONFATAL) - err_handler->error_detected(dev, pci_channel_io_normal); - else if (info->severity =3D=3D AER_FATAL) - err_handler->error_detected(dev, pci_channel_io_frozen); - } -out: - device_unlock(&dev->dev); - return 0; -} - -static void cxl_rch_handle_error(struct pci_dev *dev, struct aer_err_info = *info) -{ - /* - * Internal errors of an RCEC indicate an AER error in an - * RCH's downstream port. Check and handle them in the CXL.mem - * device driver. - */ - if (pci_pcie_type(dev) =3D=3D PCI_EXP_TYPE_RC_EC && - is_internal_error(info)) - pcie_walk_rcec(dev, cxl_rch_handle_error_iter, info); + return is_internal_error(info); } =20 static int handles_cxl_error_iter(struct pci_dev *dev, void *data) @@ -1056,13 +1036,17 @@ static int handles_cxl_error_iter(struct pci_dev *d= ev, void *data) return *handles_cxl; } =20 -static bool handles_cxl_errors(struct pci_dev *rcec) +static bool handles_cxl_errors(struct pci_dev *dev) { bool handles_cxl =3D false; =20 - if (pci_pcie_type(rcec) =3D=3D PCI_EXP_TYPE_RC_EC && - pcie_aer_is_native(rcec)) - pcie_walk_rcec(rcec, handles_cxl_error_iter, &handles_cxl); + if (!pcie_aer_is_native(dev)) + return false; + + if (pci_pcie_type(dev) =3D=3D PCI_EXP_TYPE_RC_EC) + pcie_walk_rcec(dev, handles_cxl_error_iter, &handles_cxl); + else + handles_cxl =3D pcie_is_cxl(dev); =20 return handles_cxl; } @@ -1076,10 +1060,46 @@ static void cxl_rch_enable_rcec(struct pci_dev *rce= c) pci_info(rcec, "CXL: Internal errors unmasked"); } =20 +static int cxl_create_prot_error_info(struct pci_dev *pdev, + struct aer_err_info *aer_err_info, + struct cxl_prot_error_info *cxl_err_info) +{ + cxl_err_info->severity =3D aer_err_info->severity; + + cxl_err_info->function =3D PCI_FUNC(pdev->devfn); + cxl_err_info->device =3D PCI_SLOT(pdev->devfn); + cxl_err_info->bus =3D pdev->bus->number; + cxl_err_info->segment =3D pci_domain_nr(pdev->bus); + + return 0; +} + +static void forward_cxl_error(struct pci_dev *pdev, struct aer_err_info *a= er_err_info) +{ + struct cxl_prot_err_work_data wd; + struct cxl_prot_error_info *cxl_err_info =3D &wd.err_info; + + cxl_create_prot_error_info(pdev, aer_err_info, cxl_err_info); + + if (!kfifo_put(&cxl_prot_err_fifo, wd)) { + dev_err_ratelimited(&pdev->dev, "CXL kfifo overflow\n"); + return; + } + + schedule_work(cxl_prot_err_work); +} + #else static inline void cxl_rch_enable_rcec(struct pci_dev *dev) { } static inline void cxl_rch_handle_error(struct pci_dev *dev, struct aer_err_info *info) { } +static inline void forward_cxl_error(struct pci_dev *dev, + struct aer_err_info *info) { } +static inline bool handles_cxl_errors(struct pci_dev *dev) +{ + return false; +} +static bool is_cxl_error(struct pci_dev *pdev, struct aer_err_info *info) = { return 0; }; #endif =20 /** @@ -1117,8 +1137,11 @@ static void pci_aer_handle_error(struct pci_dev *dev= , struct aer_err_info *info) =20 static void handle_error_source(struct pci_dev *dev, struct aer_err_info *= info) { - cxl_rch_handle_error(dev, info); - pci_aer_handle_error(dev, info); + if (is_cxl_error(dev, info)) + forward_cxl_error(dev, info); + else + pci_aer_handle_error(dev, info); + pci_dev_put(dev); } =20 @@ -1582,6 +1605,31 @@ static pci_ers_result_t aer_root_reset(struct pci_de= v *dev) return rc ? PCI_ERS_RESULT_DISCONNECT : PCI_ERS_RESULT_RECOVERED; } =20 +#if defined(CONFIG_PCIEAER_CXL) + +int cxl_register_prot_err_work(struct work_struct *work) +{ + guard(spinlock)(&cxl_prot_err_fifo_lock); + cxl_prot_err_work =3D work; + return 0; +} +EXPORT_SYMBOL_NS_GPL(cxl_register_prot_err_work, "CXL"); + +int cxl_unregister_prot_err_work(void) +{ + guard(spinlock)(&cxl_prot_err_fifo_lock); + cxl_prot_err_work =3D NULL; + return 0; +} +EXPORT_SYMBOL_NS_GPL(cxl_unregister_prot_err_work, "CXL"); + +int cxl_prot_err_kfifo_get(struct cxl_prot_err_work_data *wd) +{ + return kfifo_get(&cxl_prot_err_fifo, wd); +} +EXPORT_SYMBOL_NS_GPL(cxl_prot_err_kfifo_get, "CXL"); +#endif + static struct pcie_port_service_driver aerdriver =3D { .name =3D "aer", .port_type =3D PCIE_ANY_PORT, diff --git a/include/linux/aer.h b/include/linux/aer.h index 02940be66324..550407240ab5 100644 --- a/include/linux/aer.h +++ b/include/linux/aer.h @@ -10,6 +10,7 @@ =20 #include #include +#include =20 #define AER_NONFATAL 0 #define AER_FATAL 1 @@ -53,6 +54,27 @@ struct aer_capability_regs { u16 uncor_err_source; }; =20 +/** + * struct cxl_prot_err_info - Error information used in CXL error handling + * @severity: AER severity + * @function: Device's PCI function + * @device: Device's PCI device + * @bus: Device's PCI bus + * @segment: Device's PCI segment + */ +struct cxl_prot_error_info { + int severity; + + u8 function; + u8 device; + u8 bus; + u16 segment; +}; + +struct cxl_prot_err_work_data { + struct cxl_prot_error_info err_info; +}; + #if defined(CONFIG_PCIEAER) int pci_aer_clear_nonfatal_status(struct pci_dev *dev); int pcie_aer_is_native(struct pci_dev *dev); @@ -64,6 +86,20 @@ static inline int pci_aer_clear_nonfatal_status(struct p= ci_dev *dev) static inline int pcie_aer_is_native(struct pci_dev *dev) { return 0; } #endif =20 +#if defined(CONFIG_PCIEAER_CXL) +int cxl_register_prot_err_work(struct work_struct *work); +int cxl_unregister_prot_err_work(void); +int cxl_prot_err_kfifo_get(struct cxl_prot_err_work_data *wd); +#else +static inline int +cxl_register_prot_err_work(struct work_struct *work) +{ + return 0; +} +static inline int cxl_unregister_prot_err_work(void) { return 0; } +static inline int cxl_prot_err_kfifo_get(struct cxl_prot_err_work_data *wd= ) { return 0; } +#endif + void pci_print_aer(struct pci_dev *dev, int aer_severity, struct aer_capability_regs *aer); int cper_severity_to_aer(int cper_severity); --=20 2.34.1 From nobody Thu Dec 18 05:05:03 2025 Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2063.outbound.protection.outlook.com [40.107.244.63]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3300C1EA7E4; Tue, 3 Jun 2025 17:23:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.244.63 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1748971416; cv=fail; b=EhBWiDk9/7XmyLN6QmFeAMrjDQKWM3OyPQIwUPsl/Qem2XolOKeL29DalZPuU2QKh7g/2lVqH0aZZBlD9yXkMjBfTUXP8Mj0V3F7F1eGzhjKKY1KU1uyh1aBrBLUiWX1Ko/+zoYfFTVGluNx7kAUe6QornkNK4EeNxOj66YriYw= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1748971416; c=relaxed/simple; bh=BbOJhPo8UiRk859cfsZ5Zql8qC4V3F2LX23PzbPTq1Q=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=EkgQS8qECrFvd7H+VSfd6uQbCnabHkxnTS35zaSLu+elOSFbNcgESKp/f9+7vlLXSJoAI2IoZdnP+znT1nrZ/yI3Ve7zGp8PaBBI104RXAsuip5IKDMA1toNCIsZGyIKFjEXMg7xQfEu2yBSDIxdir+Eq4gH3GObieo6/lcGTBk= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=cVRflQl4; arc=fail smtp.client-ip=40.107.244.63 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="cVRflQl4" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=M8E9LJsgyb+qJmFmFZtdGE6HQbX1sqmSEmTgLV6GBxjqeHiSebWYIBuGr9iH77BGmJ+DTu3AzEws19VEaUv+1WKNFtcHQNYC5y39eNSz1JCuHyew4wqjWQEkfJurtmD+v4xrfLmBlZ/HWc6nFxxYxlayt9Z1YoG4X2W7r0cw6FmRE4s7pdAWrY6W5n6c/krszamTKZck4N+VvOqT+YFw3Mb0qKgZU0CDAXM3Qc1L5BzkbSz+RTaHCfxlUMiI0Ya+i66XpYvF0O3tBTXNBLmbT2fOBNGE1uZ4klJgb5P6EtecLUx14nJnK4YEEiS0voEAGZuogXyGTaZT0tM7OuaKMQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=SfDBpbTDENEZJ15zunUKTL+CAv5SbxUSYFLDeIKLjvc=; b=uumJFsvVk/48IqBmfK+6jrh67o6KHu9Jy9LHNdHd5ZHuhMArQNkl6jWRY0a2LBApy+4VX/WeWTbBoZXM/fq3CgVxU6C8zjWXK7Ocgl+qsVdCY3Bp7KhkKXmvFSxsCW00ols//82D9DP9y7MaSHD4h0EQA5LkBbTlhxwE4VVzb4vbKT1qTpgBG3vignGm2eqG3KL5nHT0JUt3acpnyi9Vkt2mcOph7rdVL41yB6zPR0hzDpY1jwWBucpmAgmRgszbLjQmUIPntnzDqwIPIfjLKq7FuqtWaccIC0LhXkFzJSPCi9CEU65+sVA2krCW9hRs8FEA8IU+alrQ6HohqwnIIg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=stgolabs.net smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=SfDBpbTDENEZJ15zunUKTL+CAv5SbxUSYFLDeIKLjvc=; b=cVRflQl4abCl0+hCyH3n9V9MnHbKcqVrJDYtBAy45x5HmhgeQdB2U8dbI8aVtMXCVpk1fngHpPMm9hxcF/E7ROQTNGesJRCvhjacCl39aU9WK8l2DIf0/jiI4QumSVb0fh7HdpdBgUwuvfVoVZAr8BuyDiMoFSxqJttDT1az+gA= Received: from BN9PR03CA0185.namprd03.prod.outlook.com (2603:10b6:408:f9::10) by PH7PR12MB8180.namprd12.prod.outlook.com (2603:10b6:510:2b6::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8769.36; Tue, 3 Jun 2025 17:23:31 +0000 Received: from BL6PEPF00020E62.namprd04.prod.outlook.com (2603:10b6:408:f9:cafe::7e) by BN9PR03CA0185.outlook.office365.com (2603:10b6:408:f9::10) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8792.34 via Frontend Transport; Tue, 3 Jun 2025 17:23:31 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by BL6PEPF00020E62.mail.protection.outlook.com (10.167.249.23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.8792.29 via Frontend Transport; Tue, 3 Jun 2025 17:23:31 +0000 Received: from ethanolx7ea3host.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Tue, 3 Jun 2025 12:23:29 -0500 From: Terry Bowman To: , , , , , , , , , , , , , , , , , , , , , , , , , , Subject: [PATCH v9 04/16] PCI/AER: Dequeue forwarded CXL error Date: Tue, 3 Jun 2025 12:22:27 -0500 Message-ID: <20250603172239.159260-5-terry.bowman@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250603172239.159260-1-terry.bowman@amd.com> References: <20250603172239.159260-1-terry.bowman@amd.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL6PEPF00020E62:EE_|PH7PR12MB8180:EE_ X-MS-Office365-Filtering-Correlation-Id: 77c86d00-540e-49b0-086d-08dda2c35c81 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|36860700013|1800799024|82310400026|7416014|921020; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?+DSGivDffNzk2a+ZohR3WWWe4FEndUXiEAGkh+LMFDkae2Iqwl4wkPMFZcQF?= =?us-ascii?Q?ff5rzhXD8p3hWlrju71vIwVe+N7b/UknZX9AVdWsx1WGIKn3PZ0gThuO+pxA?= =?us-ascii?Q?gA73qhThDVNHz4VI2beXJ9W3DQoWIDziwnTmVCiYZtBT26TCBK681FakVzcJ?= =?us-ascii?Q?f4Nn2l0k+IfLD4/vt01PSrqFYptZnZVWm9ongPP5LmbacPS1r2RnMt7jztre?= =?us-ascii?Q?tKmArGKKH5otYjKIWWoi0IFcrJ4/ajWhBvn23lVmFtyMTGlrYSsSU7aESvDU?= =?us-ascii?Q?+x1JnNRPNscHuwIQ5tnpdTXRpzGFviUZcc8plSWV9pPMTI6LCHiHHE0KBwS6?= =?us-ascii?Q?IjKbTCepf2HlqEaChsM9ucJcufbZQ74doG153QUz6TJmOHro7Coza1kCB0/1?= =?us-ascii?Q?ZXuc+6n9WmBzImak/V7ed9I8lIBpu/KoLZFhMqE0OHYfsyTWx+qS4o74Vjam?= =?us-ascii?Q?+MmvExOx82hiKkmq11hcAhN4/Lf7YxRsR4hsA1rcKkR7E5hfK0HvWR0lYDyg?= =?us-ascii?Q?Jo9Z7rw6CmOUVdMHjkbepo8cfTlt5FZSleeEe/zPeNQ12l/LVNEXLfkffVCJ?= =?us-ascii?Q?WYfLgpNCHdQsKmjOx/LzUUrLBVlenWaMWbMrhy2w11Vk9u2Fgfj6FBglFiLH?= =?us-ascii?Q?qb9Yh18u6Hn0IOz+dmT0Dq5nWcbLQNY09AU6dkHTCoAmmzc4tcNizB84S8Qq?= =?us-ascii?Q?hNvZcy89bikQz54CLnv5i35S20T296PRYa9spp42S9Lkxaa8rjJus4i2BvDI?= =?us-ascii?Q?ETkqJtR5XEUggNtf3CChwQNPhiEqq3AV6+55eTyN2Eukyr1cycx5x05oheMV?= =?us-ascii?Q?AKsvjB05qJ/1pSFPKsIgdKww9i7rAkE/D/+xK/RJGcNzNeNtrIcYEBxx8a2v?= =?us-ascii?Q?rg2x4PMkiSlnKdh4r7gOBSX63ufGS5/WB0UObce661K5UfzgUD0+UamOSUhU?= =?us-ascii?Q?98RU44PBM//WK7/ghenWJKrUMJDJahUneiNKTQWcukZUeA65yTAatmkvX9o2?= =?us-ascii?Q?nQfTDzE0KrS0C6Ld59Y+ZqsqXfNVcUUazP5AT1tBwrom/JqcfwjFORJd9Sgh?= =?us-ascii?Q?cusMijLR/G5q+hWPWanRBXbvPzG/2Pp4muHx0fUECyPMAtxelORKkxUQdaDD?= =?us-ascii?Q?xK8CTrlRkuy17XG+9Xh1MtU28l1wbVJM+Z60B+8AfDbirmte9FQfER3sF+6z?= =?us-ascii?Q?j0qi42gYc6Mx1vGWsxHSGTiPBq+Nm3EkgkSH6C+10MSvVY20AaVOufLdxE+W?= =?us-ascii?Q?pBDdniahJ548KgK3IADyg+qtjSBxG1Gs0bIIHDsY/7Zc5xa5bh7zLzsnTWd0?= =?us-ascii?Q?pkj5vN4McdO6w6HpKkydfoMiHuDlPKoKTpY+81yBR57SWgs1Q+6V2yupsJ1o?= =?us-ascii?Q?PJx4p3DOyBR1tr5eyo/ROqkZvdFiH5auosSk/p9byF7nONFi5jI8EHaR1Q5X?= =?us-ascii?Q?I8L5mvf5PzH4meYIJUSCALNWRk3yE5Vtp/oVgSicC1S7Jili7BA3X9nVOOt1?= =?us-ascii?Q?HIrCLyJrMkhfAYWvMCYvP8bnL1BNA/+hUX3vAR+irPB7GraHmiR/cGfmGA?= =?us-ascii?Q?=3D=3D?= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(376014)(36860700013)(1800799024)(82310400026)(7416014)(921020);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Jun 2025 17:23:31.0311 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 77c86d00-540e-49b0-086d-08dda2c35c81 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BL6PEPF00020E62.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB8180 Content-Type: text/plain; charset="utf-8" The AER driver is now designed to forward CXL protocol errors to the CXL driver. Update the CXL driver with functionality to dequeue the forwarded CXL error from the kfifo. Also, update the CXL driver to begin the protocol error handling processing using the work received from the FIFO. Introduce function cxl_prot_err_work_fn() to dequeue work forwarded by the AER service driver. This will begin the CXL protocol error processing with a call to cxl_handle_prot_error(). Update cxl/core/ras.c by adding cxl_rch_handle_error_iter() that was previously in the AER driver. Introduce sbdf_to_pci() to take the SBDF values from 'struct cxl_prot_error= _info' and use in discovering the erring PCI device. Make scope based reference increments/decrements for the discovered PCI device and the associated CXL device. Implement cxl_handle_prot_error() to differentiate between Restricted CXL Host (RCH) protocol errors and CXL virtual host (VH) protocol errors. RCH errors will be processed with a call to walk the associated Root Complex Event Collector's (RCEC) secondary bus looking for the Root Complex Integrated Endpoint (RCiEP) to handle the RCH error. Export pcie_walk_rcec() so the CXL driver can walk the RCEC's downstream bus, searching for the RCiEP. VH correctable error (CE) processing will call the CXL CE handler. VH uncorrectable errors (UCE) will call cxl_do_recovery(), implemented as a stub for now and to be updated in future patch. Export pci_aer_clean_fatal_= status() and pci_clean_device_status() used to clean up AER status after handling. Signed-off-by: Terry Bowman Reviewed-by: Kuppuswamy Sathyanarayanan --- drivers/cxl/core/ras.c | 92 +++++++++++++++++++++++++++++++++++++++++ drivers/pci/pci.c | 1 + drivers/pci/pci.h | 8 ---- drivers/pci/pcie/aer.c | 1 + drivers/pci/pcie/rcec.c | 1 + include/linux/aer.h | 2 + include/linux/pci.h | 10 +++++ 7 files changed, 107 insertions(+), 8 deletions(-) diff --git a/drivers/cxl/core/ras.c b/drivers/cxl/core/ras.c index d35525e79e04..9ed5c682e128 100644 --- a/drivers/cxl/core/ras.c +++ b/drivers/cxl/core/ras.c @@ -110,8 +110,100 @@ static DECLARE_WORK(cxl_cper_prot_err_work, cxl_cper_= prot_err_work_fn); =20 #ifdef CONFIG_PCIEAER_CXL =20 +static void cxl_do_recovery(struct pci_dev *pdev) +{ +} + +static int cxl_rch_handle_error_iter(struct pci_dev *pdev, void *data) +{ + struct cxl_prot_error_info *err_info =3D data; + struct pci_dev *pdev_ref __free(pci_dev_put) =3D pci_dev_get(pdev); + struct cxl_dev_state *cxlds; + + /* + * The capability, status, and control fields in Device 0, + * Function 0 DVSEC control the CXL functionality of the + * entire device (CXL 3.0, 8.1.3). + */ + if (pdev->devfn !=3D PCI_DEVFN(0, 0)) + return 0; + + /* + * CXL Memory Devices must have the 502h class code set (CXL + * 3.0, 8.1.12.1). + */ + if ((pdev->class >> 8) !=3D PCI_CLASS_MEMORY_CXL) + return 0; + + if (!is_cxl_memdev(&pdev->dev) || !pdev->dev.driver) + return 0; + + cxlds =3D pci_get_drvdata(pdev); + struct device *dev __free(put_device) =3D get_device(&cxlds->cxlmd->dev); + + if (err_info->severity =3D=3D AER_CORRECTABLE) + cxl_cor_error_detected(pdev); + else + cxl_do_recovery(pdev); + + return 1; +} + +static struct pci_dev *sbdf_to_pci(struct cxl_prot_error_info *err_info) +{ + unsigned int devfn =3D PCI_DEVFN(err_info->device, + err_info->function); + struct pci_dev *pdev __free(pci_dev_put) =3D + pci_get_domain_bus_and_slot(err_info->segment, + err_info->bus, + devfn); + return pdev; +} + +static void cxl_handle_prot_error(struct cxl_prot_error_info *err_info) +{ + struct pci_dev *pdev __free(pci_dev_put) =3D pci_dev_get(sbdf_to_pci(err_= info)); + + if (!pdev) { + pr_err("Failed to find the CXL device\n"); + return; + } + + /* + * Internal errors of an RCEC indicate an AER error in an + * RCH's downstream port. Check and handle them in the CXL.mem + * device driver. + */ + if (pci_pcie_type(pdev) =3D=3D PCI_EXP_TYPE_RC_EC) + return pcie_walk_rcec(pdev, cxl_rch_handle_error_iter, err_info); + + if (err_info->severity =3D=3D AER_CORRECTABLE) { + int aer =3D pdev->aer_cap; + struct cxl_dev_state *cxlds =3D pci_get_drvdata(pdev); + struct device *dev __free(put_device) =3D get_device(&cxlds->cxlmd->dev); + + if (aer) + pci_clear_and_set_config_dword(pdev, + aer + PCI_ERR_COR_STATUS, + 0, PCI_ERR_COR_INTERNAL); + + cxl_cor_error_detected(pdev); + + pcie_clear_device_status(pdev); + } else { + cxl_do_recovery(pdev); + } +} + static void cxl_prot_err_work_fn(struct work_struct *work) { + struct cxl_prot_err_work_data wd; + + while (cxl_prot_err_kfifo_get(&wd)) { + struct cxl_prot_error_info *err_info =3D &wd.err_info; + + cxl_handle_prot_error(err_info); + } } =20 #else diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c index e77d5b53c0ce..524ac32b744a 100644 --- a/drivers/pci/pci.c +++ b/drivers/pci/pci.c @@ -2328,6 +2328,7 @@ void pcie_clear_device_status(struct pci_dev *dev) pcie_capability_read_word(dev, PCI_EXP_DEVSTA, &sta); pcie_capability_write_word(dev, PCI_EXP_DEVSTA, sta); } +EXPORT_SYMBOL_NS_GPL(pcie_clear_device_status, "CXL"); #endif =20 /** diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h index d6296500b004..3c54a5ed803e 100644 --- a/drivers/pci/pci.h +++ b/drivers/pci/pci.h @@ -649,16 +649,10 @@ static inline bool pci_dpc_recovered(struct pci_dev *= pdev) { return false; } void pci_rcec_init(struct pci_dev *dev); void pci_rcec_exit(struct pci_dev *dev); void pcie_link_rcec(struct pci_dev *rcec); -void pcie_walk_rcec(struct pci_dev *rcec, - int (*cb)(struct pci_dev *, void *), - void *userdata); #else static inline void pci_rcec_init(struct pci_dev *dev) { } static inline void pci_rcec_exit(struct pci_dev *dev) { } static inline void pcie_link_rcec(struct pci_dev *rcec) { } -static inline void pcie_walk_rcec(struct pci_dev *rcec, - int (*cb)(struct pci_dev *, void *), - void *userdata) { } #endif =20 #ifdef CONFIG_PCI_ATS @@ -967,7 +961,6 @@ void pci_no_aer(void); void pci_aer_init(struct pci_dev *dev); void pci_aer_exit(struct pci_dev *dev); extern const struct attribute_group aer_stats_attr_group; -void pci_aer_clear_fatal_status(struct pci_dev *dev); int pci_aer_clear_status(struct pci_dev *dev); int pci_aer_raw_clear_status(struct pci_dev *dev); void pci_save_aer_state(struct pci_dev *dev); @@ -976,7 +969,6 @@ void pci_restore_aer_state(struct pci_dev *dev); static inline void pci_no_aer(void) { } static inline void pci_aer_init(struct pci_dev *d) { } static inline void pci_aer_exit(struct pci_dev *d) { } -static inline void pci_aer_clear_fatal_status(struct pci_dev *dev) { } static inline int pci_aer_clear_status(struct pci_dev *dev) { return -EINV= AL; } static inline int pci_aer_raw_clear_status(struct pci_dev *dev) { return -= EINVAL; } static inline void pci_save_aer_state(struct pci_dev *dev) { } diff --git a/drivers/pci/pcie/aer.c b/drivers/pci/pcie/aer.c index 5350fa5be784..6e88331c6303 100644 --- a/drivers/pci/pcie/aer.c +++ b/drivers/pci/pcie/aer.c @@ -290,6 +290,7 @@ void pci_aer_clear_fatal_status(struct pci_dev *dev) if (status) pci_write_config_dword(dev, aer + PCI_ERR_UNCOR_STATUS, status); } +EXPORT_SYMBOL_GPL(pci_aer_clear_fatal_status); =20 /** * pci_aer_raw_clear_status - Clear AER error registers. diff --git a/drivers/pci/pcie/rcec.c b/drivers/pci/pcie/rcec.c index d0bcd141ac9c..fb6cf6449a1d 100644 --- a/drivers/pci/pcie/rcec.c +++ b/drivers/pci/pcie/rcec.c @@ -145,6 +145,7 @@ void pcie_walk_rcec(struct pci_dev *rcec, int (*cb)(str= uct pci_dev *, void *), =20 walk_rcec(walk_rcec_helper, &rcec_data); } +EXPORT_SYMBOL_NS_GPL(pcie_walk_rcec, "CXL"); =20 void pci_rcec_init(struct pci_dev *dev) { diff --git a/include/linux/aer.h b/include/linux/aer.h index 550407240ab5..c9a18eca16f8 100644 --- a/include/linux/aer.h +++ b/include/linux/aer.h @@ -77,12 +77,14 @@ struct cxl_prot_err_work_data { =20 #if defined(CONFIG_PCIEAER) int pci_aer_clear_nonfatal_status(struct pci_dev *dev); +void pci_aer_clear_fatal_status(struct pci_dev *dev); int pcie_aer_is_native(struct pci_dev *dev); #else static inline int pci_aer_clear_nonfatal_status(struct pci_dev *dev) { return -EINVAL; } +static inline void pci_aer_clear_fatal_status(struct pci_dev *dev) { } static inline int pcie_aer_is_native(struct pci_dev *dev) { return 0; } #endif =20 diff --git a/include/linux/pci.h b/include/linux/pci.h index bff3009f9ff0..cd53715d53f3 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -1806,6 +1806,9 @@ extern bool pcie_ports_native; =20 int pcie_set_target_speed(struct pci_dev *port, enum pci_bus_speed speed_r= eq, bool use_lt); +void pcie_walk_rcec(struct pci_dev *rcec, + int (*cb)(struct pci_dev *, void *), + void *userdata); #else #define pcie_ports_disabled true #define pcie_ports_native false @@ -1816,8 +1819,15 @@ static inline int pcie_set_target_speed(struct pci_d= ev *port, { return -EOPNOTSUPP; } + +static inline void pcie_walk_rcec(struct pci_dev *rcec, + int (*cb)(struct pci_dev *, void *), + void *userdata) { } + #endif =20 +void pcie_clear_device_status(struct pci_dev *dev); + #define PCIE_LINK_STATE_L0S (BIT(0) | BIT(1)) /* Upstr/dwnstr L0s */ #define PCIE_LINK_STATE_L1 BIT(2) /* L1 state */ #define PCIE_LINK_STATE_L1_1 BIT(3) /* ASPM L1.1 state */ --=20 2.34.1 From nobody Thu Dec 18 05:05:03 2025 Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2085.outbound.protection.outlook.com [40.107.237.85]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 73AE21CDFD5; Tue, 3 Jun 2025 17:23:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.237.85 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1748971427; cv=fail; b=Gf6E96vNTt+hq1d3KF75PS74wG+BmalNLo6PwgFYO4Wvx9avplJaMO+D5LHOcPHNkVNYTSMJ20DY8zXxq1GmHqkp/bZ7bD4DdcVY1X4y7kqCeUs1j7Lab8x4e32JD6pR1NplogXNHtGlyLGYmMxopUHXmVd6g9BVPCgTo+z/YEE= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1748971427; c=relaxed/simple; bh=xmV8wJjWXtIrI2QGFBw5dZ8hgvYP84pgHkfFZKWq/Xs=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=QqhF8EtAWMA3i3w99Y6y+JL2AHwJA1SofU5TlyW7iAGwYWuL+TZR2GINPwwSfIK4M264BcY7+09j61jj9pYgQDkjlWudk2fXkNQITdP23cnwZSzaGF3nHcOM6m1p4OhCQo5PqUs7pi2XHb+msE8cAY2ocYVOMO8TwHtf3QPACdA= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=tLeWP4B+; arc=fail smtp.client-ip=40.107.237.85 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="tLeWP4B+" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=Co64x9H0D9q8pP5xWT9exYioYcWy4npHvazRd+c33t3qUwnJOXd3Uc4axoi7OZVYwL2jMrkYzeRA34KEXl2iGcseYz5N+xzkMza/BMGkc75CB/GkOHOdds3hevsHLqyO8441buKwljUvWIGeSLrHcw+T4C0GmDj9ZlC0c9iCE297h9ZgX+7w0wV/mho8Ld2TnyEfBeiPlZxEYYGjyIjF+zyeJaqS4hn/qmeRh+QmKdW+KgHuvDjMRKKXFlqfCfPnH6W2j7idTwmMVxoCqDQwk/o/SMpQR98dF7Yz0CpX/2SjqgkmR03LKQhX7wQ2MqxOPA67eikXGf8W7AlVEX7/lg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=TeL6V0VSnRJB1hZdh9WgHXOhAUGLbV2d2vJFMQngWlI=; b=jiUiL+4OTDNcIp3Gxwv03EY3GugJfqxISM8wELAQTcIhLPkk3bQUAl2d7teogCR0oRY8Iq8hfNRkkvvuiezrCfazEwkCDA+WCWNPyjNJIY1OVKwywb2Elkda14UwzgjNnmBW2J8h6x9y1vZJUr82l8RkksKCyUJivzxZ8Pi4f8chImq4E9K/tS4CZXgFIs+wox/AQGQ9T+EY13azjt9YE8MHm/VoPRt0aPmRI5vLD2Svlywvc5z6U0ZBwc3SVRJsYFFCGejFWJc6v7hh8u4cvZTkcrOiUNZ7i7674hWo9vJM9cKeBUuDxLkPN7/8tlnuAAsTTv83Z9ZeFY/w5m614Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=stgolabs.net smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=TeL6V0VSnRJB1hZdh9WgHXOhAUGLbV2d2vJFMQngWlI=; b=tLeWP4B+BqYao8UFIIYJTGdxcLnTesva8toojwSb1S08CAFLXo/zcjP11QaGv8K1gfEn2+LzX6iVHymFngwQOFoH1v7+XVo+HEf2tvvPYK/PNjSw82uD9raxsyOZrsrE8CQjomzlFgbhdy+qCgPvbuYdgzn3zR12j89hi2r3gEc= Received: from BL1PR13CA0200.namprd13.prod.outlook.com (2603:10b6:208:2be::25) by DM4PR12MB8557.namprd12.prod.outlook.com (2603:10b6:8:18b::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8792.34; Tue, 3 Jun 2025 17:23:42 +0000 Received: from BL6PEPF00020E61.namprd04.prod.outlook.com (2603:10b6:208:2be:cafe::8e) by BL1PR13CA0200.outlook.office365.com (2603:10b6:208:2be::25) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8792.19 via Frontend Transport; Tue, 3 Jun 2025 17:23:42 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by BL6PEPF00020E61.mail.protection.outlook.com (10.167.249.22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.8792.29 via Frontend Transport; Tue, 3 Jun 2025 17:23:42 +0000 Received: from ethanolx7ea3host.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Tue, 3 Jun 2025 12:23:41 -0500 From: Terry Bowman To: , , , , , , , , , , , , , , , , , , , , , , , , , , Subject: [PATCH v9 05/16] CXL/PCI: Introduce CXL uncorrectable protocol error recovery Date: Tue, 3 Jun 2025 12:22:28 -0500 Message-ID: <20250603172239.159260-6-terry.bowman@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250603172239.159260-1-terry.bowman@amd.com> References: <20250603172239.159260-1-terry.bowman@amd.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL6PEPF00020E61:EE_|DM4PR12MB8557:EE_ X-MS-Office365-Filtering-Correlation-Id: 88f43e6c-4a08-4ed7-2729-08dda2c36335 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|36860700013|82310400026|1800799024|7416014|376014|921020; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?LFRHsB1ZKqPVvSrFkf7Qt3PSKe8p/TzFGaiSN7kdvMeW77/j/GGnbO/iS7lG?= =?us-ascii?Q?Dzr46n5Je2bapeoePWYgwXNWTS3g+aQnI10XyvjKqfmJTeK2JQbWGYkWSESM?= =?us-ascii?Q?1DzRWs327DgNdbeS1II3U/0Gu6+hv5jB1ygkxLBH85e3BSgp4lIcXVmBTQ8z?= =?us-ascii?Q?8ZuGPWZ92IwVdYQaF3aADlIEVXENrVx4Tf8HIpiQ/B8QDpSBhfkJiohNLzRo?= =?us-ascii?Q?i8m5lfNYzsZTl40WB24HyJFjrzKeOiuz3zyDw88QnU8vBd/9BldpwREIPzn6?= =?us-ascii?Q?6PfolHdq5NKQHGdjIXr+MH441OXCbWzVd6/wLtz4Im3RDMHMZyWBwoQbddGa?= =?us-ascii?Q?cnwWRmHs6dHfEABhIX3UaAYsauQwAAmJOWI75RYfkohd3mVa5kaPJn3VQrmy?= =?us-ascii?Q?2NfymlpjhEXgQTdczKCOqY88D8TCXbRG/Jc5WOwXSqf+Ayy2lv5Lhj0ydiVC?= =?us-ascii?Q?yjRxxp8/WxLDUnMJ0S1YzA6cYomuFOSumESVLABZMam+ITu86hQX9KeDI/qs?= =?us-ascii?Q?xG6WTqoWT/csYC38ZO+71ux2EuVP5xu1mqAudJSJSMI7xanE8fW6BEa4jqee?= =?us-ascii?Q?WeHp3aTtP20Mu+U4rmLqvHBxzflK6QMrLCE0PUue/T0xH9XtXtiODDJMolAC?= =?us-ascii?Q?QsGzKE8Pdiol/FiBJVxuqTb9IeTL+Dqs2AXgWHrqe3Xoel2aqSowzNYsLtfQ?= =?us-ascii?Q?vvkGaXRRz+FvD9jE4Z1mIVFnASzjHXVBWUpPDkFpCnLcWMuw3jAxRRiGun4F?= =?us-ascii?Q?d+P89ttn5EvqC8EGTT079PB0Bs7uaA4puoEzHtixJdbNDdCPcltrgBXvjh1w?= =?us-ascii?Q?aT5vzuMWSC12wd+YLeNzj4XeUobIry7pqfFG6W9X1I+dKpfTwmPBy2Iq5gtf?= =?us-ascii?Q?ddHiDZ+LWXjqVlRAi8q0Me8mUWzOj1WjdAoTVtu9lfoYhQS0ikj00HgL1cYV?= =?us-ascii?Q?OoOnwa7CIcdoH7P6wTY6Djhdn8rieDYWT9wZvZSrLfM7J+DToAn9U+88T4UB?= =?us-ascii?Q?0B9Ve0KcRnFp6t2yORZfnK7oj5eexuITtuy33VF3z1Vnntq2LYlFB77mFuQs?= =?us-ascii?Q?okOOOk9BSuv+yB1Rhn6JbTYSV7UWNK2q4H6N9Jr/g0fsSkhri7UA+N1veKJ4?= =?us-ascii?Q?YB1ytKgI9d+nphSS1IHMIqC86gjJ1gT18WLrP5XMW2VTaDqk0VTjKCUkG5kY?= =?us-ascii?Q?57UMPL1L1u6lVoQNlEr+lVVXMI2VcZX8Jrljv4Kg0FoKtCmTNljb9a8m8ld5?= =?us-ascii?Q?7ZVVkQnfxu0pZPs58Df5jQ3VXxQaOsg0c9G622w2JRzf19p+yslT/SQqFf32?= =?us-ascii?Q?QIxSETMLTbWYsb0GTMc1lHgVouxZWhRZkLyP4gJ+2a3KPGnVvxwmoGa82GYC?= =?us-ascii?Q?+v5fbl26Ed2uLS9oirrQyRyqk72PN+pgTQYBRjyY6n5Z+fVZdlWvdxEW6qmr?= =?us-ascii?Q?vNXZ5kZmKM6EU0FWJttVDbd9FBC+jaVrYGsKoO2sVbeRjcnIwdRq9cjcR+sc?= =?us-ascii?Q?ZpfmXUAv3ZfIaRANOrMsm5Q3jwAy2Q2GLNsb5hDmXd2clzRZICzMrIWRRg?= =?us-ascii?Q?=3D=3D?= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(36860700013)(82310400026)(1800799024)(7416014)(376014)(921020);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Jun 2025 17:23:42.2783 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 88f43e6c-4a08-4ed7-2729-08dda2c36335 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BL6PEPF00020E61.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB8557 Content-Type: text/plain; charset="utf-8" Create cxl_do_recovery() to provide uncorrectable protocol error (UCE) handling. Follow similar design as found in PCIe error driver, pcie_do_recovery(). One difference is cxl_do_recovery() will treat all UCEs as fatal with a kernel panic. This is to prevent corruption on CXL memory. Copy the PCI error driver's merge_result() and rename as cxl_merge_result(). Introduce PCI_ERS_RESULT_PANIC and add support in the cxl_merge_result() routine. Copy pci_walk_bridge() to cxl_walk_bridge(). Make a change to walk the first device in all cases. Copy the PCI error driver's report_error_detected() to cxl_report_error_det= ected(). Note, only CXL Endpoints are currently supported. Add locking for PCI device as done in PCI's report_error_detected(). Add reference counting for the CXL device responsible for cleanup of the CXL RAS. This is necessary to prevent the RAS registers from disappearing before logging is completed. Call panic() to halt the system in the case of uncorrectable errors (UCE) in cxl_do_recovery(). Export pci_aer_clear_fatal_status() for CXL to use if a UCE is not found. In this case the AER status must be cleared and uses pci_aer_clear_fatal_status(). Signed-off-by: Terry Bowman --- drivers/cxl/core/ras.c | 79 ++++++++++++++++++++++++++++++++++++++++++ include/linux/pci.h | 3 ++ 2 files changed, 82 insertions(+) diff --git a/drivers/cxl/core/ras.c b/drivers/cxl/core/ras.c index 9ed5c682e128..715f7221ea3a 100644 --- a/drivers/cxl/core/ras.c +++ b/drivers/cxl/core/ras.c @@ -110,8 +110,87 @@ static DECLARE_WORK(cxl_cper_prot_err_work, cxl_cper_p= rot_err_work_fn); =20 #ifdef CONFIG_PCIEAER_CXL =20 +static pci_ers_result_t cxl_merge_result(enum pci_ers_result orig, + enum pci_ers_result new) +{ + if (new =3D=3D PCI_ERS_RESULT_PANIC) + return PCI_ERS_RESULT_PANIC; + + if (new =3D=3D PCI_ERS_RESULT_NO_AER_DRIVER) + return PCI_ERS_RESULT_NO_AER_DRIVER; + + if (new =3D=3D PCI_ERS_RESULT_NONE) + return orig; + + switch (orig) { + case PCI_ERS_RESULT_CAN_RECOVER: + case PCI_ERS_RESULT_RECOVERED: + orig =3D new; + break; + case PCI_ERS_RESULT_DISCONNECT: + if (new =3D=3D PCI_ERS_RESULT_NEED_RESET) + orig =3D PCI_ERS_RESULT_NEED_RESET; + break; + default: + break; + } + + return orig; +} + +static int cxl_report_error_detected(struct pci_dev *pdev, void *data) +{ + pci_ers_result_t vote, *result =3D data; + struct cxl_dev_state *cxlds; + + if ((pci_pcie_type(pdev) !=3D PCI_EXP_TYPE_ENDPOINT) && + (pci_pcie_type(pdev) !=3D PCI_EXP_TYPE_RC_END)) + return 0; + + cxlds =3D pci_get_drvdata(pdev); + struct device *dev __free(put_device) =3D get_device(&cxlds->cxlmd->dev); + + device_lock(&pdev->dev); + vote =3D cxl_error_detected(pdev, pci_channel_io_frozen); + *result =3D cxl_merge_result(*result, vote); + device_unlock(&pdev->dev); + + return 0; +} + +static void cxl_walk_bridge(struct pci_dev *bridge, + int (*cb)(struct pci_dev *, void *), + void *userdata) +{ + if (cb(bridge, userdata)) + return; + + if (bridge->subordinate) + pci_walk_bus(bridge->subordinate, cb, userdata); +} + static void cxl_do_recovery(struct pci_dev *pdev) { + struct pci_host_bridge *host =3D pci_find_host_bridge(pdev->bus); + pci_ers_result_t status =3D PCI_ERS_RESULT_CAN_RECOVER; + + cxl_walk_bridge(pdev, cxl_report_error_detected, &status); + if (status =3D=3D PCI_ERS_RESULT_PANIC) + panic("CXL cachemem error."); + + /* + * If we have native control of AER, clear error status in the device + * that detected the error. If the platform retained control of AER, + * it is responsible for clearing this status. In that case, the + * signaling device may not even be visible to the OS. + */ + if (host->native_aer) { + pcie_clear_device_status(pdev); + pci_aer_clear_nonfatal_status(pdev); + pci_aer_clear_fatal_status(pdev); + } + + pci_info(pdev, "CXL uncorrectable error.\n"); } =20 static int cxl_rch_handle_error_iter(struct pci_dev *pdev, void *data) diff --git a/include/linux/pci.h b/include/linux/pci.h index cd53715d53f3..b0e7545162de 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -870,6 +870,9 @@ enum pci_ers_result { =20 /* No AER capabilities registered for the driver */ PCI_ERS_RESULT_NO_AER_DRIVER =3D (__force pci_ers_result_t) 6, + + /* System is unstable, panic */ + PCI_ERS_RESULT_PANIC =3D (__force pci_ers_result_t) 7, }; =20 /* PCI bus error event callbacks */ --=20 2.34.1 From nobody Thu Dec 18 05:05:03 2025 Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2085.outbound.protection.outlook.com [40.107.237.85]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5F23F1B4257; Tue, 3 Jun 2025 17:23:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.237.85 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1748971438; cv=fail; b=UWzhW+V6By9hlfoJBWrjJIgjzGXsYmYCz18WEEasrf1jFsO/ZXxSG2qsMiaKI8frt2wgCQ40UDdHyh6AYgtePIocQb/98Wg/oFrIPJjVz5Kx9p+QCY6v4k6pg0tQlqVF7THvzxxpsUAUxSRxZrhmbDls+CV1VPDaFSMaV8l8gYk= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1748971438; c=relaxed/simple; bh=4sWd0sMGsq9kevAC9Nw+JNtj6zs9ANYjC/48o4Hxtuo=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=nrs8DpxsmE+TJe9LdcSdnwt0G3ULAasCH6x7QZuyjO60AKjj35YHupp3MhrC4v6hZp0jqVWyLtYPVHvMDrqYc9iAwufIy98gNoTrJNo4DMTQ1TyS6PIfTe/Uim0ZBo20oWCha3f2UuNKTz3wpRIS5kh/koIsBY85eIQtFQkVTtM= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=IFXGR1Ri; arc=fail smtp.client-ip=40.107.237.85 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="IFXGR1Ri" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=GhmeqFkfu2Oze24N8wIE0tiZMiMxmvY1/acSPxMFbRrmMzKhEniq/xks0jo1sBr+BhvJoljrikI2TFY4xkqy/OBl7peI4Q1JzMY26Vv1xPnxduw1iPlAAZ0bAubXeUr/pUl9bscinEck0mw9tinxm3tO9fo2Cbw9RD0WEZwy8su/ElkOofQg/IrDC1gULlQKTJVIvVwyDTOBTlXli/0nnl8PCMn+gvDjpEjY73M4BJ+T7owSA7Xvfom76uQCIxmiYrRYYzdZ7cvIgncziRQ3l5AhO/biIDvht6GYxSelLqZJ+ZVKovMEqQXFIvz7N9Npb+bRE6hiLqJ5mlTjmAoSLw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=GBUcnkR7LMC1p1rZBDikoNyRdbgodK+3yeh6mzx1ebA=; b=DPq+kNbASrnMVO4Itum9n652JlBPWwfm43H90M5kF8C8dWZvf+SoRJJ4sCkqdKb8UpWcKwx2H1sxeYJvy33DYA6oInP0E44jAO7nlIYLDy1mlNzBzu4dfkxytiTQyQ1R+cjo77ZJavaEbn/3smY0hEFIJEYEeUzYbSIZBoSg7qmApVUD4kCTOsyDPlNMg1nHGa+Tx316J7MUsEd1YxPsnQURoY9/e66LDBW/2B6ziWjBzGCcjD5W+1wk4IUwmlLyLUazzICpc/KyH1nJB4m8HrOQx4hgZo7Y3SZkSljmZjqWoyAHkNGMZ3hew6STzKIpSWAzFycSYTrHbDNYSXcHbQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=stgolabs.net smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=GBUcnkR7LMC1p1rZBDikoNyRdbgodK+3yeh6mzx1ebA=; b=IFXGR1RiP1bKalFVzYs6F10r8gfPpnCyhVcLEpYP+/MKraUEf72NrQ8NBzFuuUqAfE3OiuEP6pEu+57GVPk0ZNkyPtVdTXBn3zRkvRWfRHNfFNvbRYMDuelLF2DIwrEG4VS9PHCcbhcjtYS3FpzTlNJHTa1cn3CHjkC/h6UGuXc= Received: from BLAPR03CA0092.namprd03.prod.outlook.com (2603:10b6:208:32a::7) by BL1PR12MB5970.namprd12.prod.outlook.com (2603:10b6:208:399::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8769.37; Tue, 3 Jun 2025 17:23:53 +0000 Received: from BL6PEPF00020E63.namprd04.prod.outlook.com (2603:10b6:208:32a:cafe::8c) by BLAPR03CA0092.outlook.office365.com (2603:10b6:208:32a::7) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8769.20 via Frontend Transport; Tue, 3 Jun 2025 17:23:53 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by BL6PEPF00020E63.mail.protection.outlook.com (10.167.249.24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.8792.29 via Frontend Transport; Tue, 3 Jun 2025 17:23:53 +0000 Received: from ethanolx7ea3host.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Tue, 3 Jun 2025 12:23:52 -0500 From: Terry Bowman To: , , , , , , , , , , , , , , , , , , , , , , , , , , Subject: [PATCH v9 06/16] cxl/pci: Move RAS initialization to cxl_port driver Date: Tue, 3 Jun 2025 12:22:29 -0500 Message-ID: <20250603172239.159260-7-terry.bowman@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250603172239.159260-1-terry.bowman@amd.com> References: <20250603172239.159260-1-terry.bowman@amd.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL6PEPF00020E63:EE_|BL1PR12MB5970:EE_ X-MS-Office365-Filtering-Correlation-Id: 7f25760b-71bd-4a46-6fc9-08dda2c369ef X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|36860700013|7416014|376014|1800799024|921020; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?lulRIWxG7XuHpu92J0wqUe+KsanNHsQtOL7Q5r9vB1QCo9pyGfccdUm8lRc/?= =?us-ascii?Q?WSzQuwyvT7clrzJ7Qqjpu7S1rxQXP1SOBZZTV9m6akoBlUzFN5UqQMJwg2mH?= =?us-ascii?Q?edJ5GQ7ybaAZ0Y/2eds88RXpYV7VZW3n1ZjQbhwYRXjpv1qjmEWBDxYRMRf3?= =?us-ascii?Q?n+cZFxiAhk4L8Iyuwy+DOSoSelpSUOKhNvwrEM+9ZCEkYycxe22NY4Y624v5?= =?us-ascii?Q?IGdQ9BSDg71k6aBglaQqtmJ7VAhw4A8FXOxIPVSqMophOG+iZzylLNUVbiJa?= =?us-ascii?Q?e8M2wWR1GXzNIhrD2gHEp6Ez5qZRr6mbtljyYKeLS8HIpNBf9q2h5MyaALVQ?= =?us-ascii?Q?ZrhL9WWQBTyc1bLIyGRnroxBbEWNWqcyveDhMIIqWLudUDnL6rCjQsD2WSfY?= =?us-ascii?Q?o+Bb+d+mx84Eg0xMgQ0+YUSNj6p8X6cKTcO6sheIfDURE6jgIDOCYrRCpJoX?= =?us-ascii?Q?LkEj5P9bZMYDyWKJ5LPFOGDvDnb1ZU8ebnDH3KVzFQYqEhTMumymHQRWtfrr?= =?us-ascii?Q?Au20k+pJb2Wmu38vBWFp4eEcVLgzY5criUEtqMaW3VD6iSCam2AZCdeYkgze?= =?us-ascii?Q?4pob82ze2Dj3ZNE6Zsxyb3vAXMjWrGapAyZKFUQyamcu5xqE5Ect1v0dy5JX?= =?us-ascii?Q?FWnt6PjxdcWwH5B1aORHzeQXPxiTMfc+2GgCJj+xpcHYsu2XpC4aMr27rS64?= =?us-ascii?Q?TaXo9qcadILt0Nvx0D2C7Y3SWaVd6ETkmFsjefWW4l6yGLrm3ryeImJ5/k4P?= =?us-ascii?Q?/iUb2SUT8HHh4ZU0ZLCPvucL8/71Y0uAGpktkp3jc7yRo7d7/0zGZBYMVh4x?= =?us-ascii?Q?dQbnJX5roQnwFvtil8J43w0ouqAoqfPwYEaLIEu7NBjBWvWw/pSlmnkfd2Td?= =?us-ascii?Q?BGv8cgVmIwxHvlYRpgMGEH7zus6mnp4eJ/dJ2KVzJoIuQjPn43KWvm4LD0Vj?= =?us-ascii?Q?Bem4D/UEpWZ982WZzO1ilK0kJPW92pJbXF4Xdegb8XoGx+ImwnCcR0TeXEFP?= =?us-ascii?Q?xWAeiglUqnD36y+bRrv4bMCLkrxxKnVqlKfi+VR0/hSJHwqlqmhRzZPP1Cfp?= =?us-ascii?Q?FOcl/fBnbW7bTRT/Xe6tcktIn/YmUPrfvyk0knxny2YRMMKfcbUVaNd8rYFP?= =?us-ascii?Q?+FhMPhX3lxe6HWrAPciiCydST/NMu24yyuVuOxlyLhYCHjJy5JqDJcKjz2Gb?= =?us-ascii?Q?j/YVXMJA+jU41m5IVsYjH23wAAFfcVvxXuvtGJV3tk1L53bSdQhMq/Yd6crz?= =?us-ascii?Q?zW05KxeG2ndcoUvYWntJo4B0QoRu0nJNUw2UDAtZZ5gvOJkOb6+3leUBnHNl?= =?us-ascii?Q?hurbhIhndt1Wx2thsjQZncSCsk2RzqTyS8LtSKDEvSyZbuskziP2MbyFozT4?= =?us-ascii?Q?E2bnMXXB7+ztNppdP9CnpA5evV067xBfE6QY9GnulQNvQ/tfj4no4i8LZ412?= =?us-ascii?Q?w3JbdxUR8e+kCi29nn5H4f07TfpN2el4B6ObV+oy1MWXqJwtEBIhcijM6Rpp?= =?us-ascii?Q?SnP9IYTZwqZT1F56YGYQERDRjZgDgIhOQwL7A2v9OlHZKzXPjkdoTKIvgA?= =?us-ascii?Q?=3D=3D?= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(82310400026)(36860700013)(7416014)(376014)(1800799024)(921020);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Jun 2025 17:23:53.5650 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 7f25760b-71bd-4a46-6fc9-08dda2c369ef X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BL6PEPF00020E63.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL1PR12MB5970 Content-Type: text/plain; charset="utf-8" The cxl_port driver is intended to manage CXL Endpoint Ports and CXL Switch Ports. Move existing RAS initialization to the cxl_port driver. Restricted CXL Host (RCH) Downstream Port RAS initialization currently resides in cxl/core/pci.c. The PCI source file is not otherwise associated with CXL Port management. Additional CXL Port RAS initialization will be added in future patches to support a CXL Port device's CXL errors. Signed-off-by: Terry Bowman --- drivers/cxl/core/pci.c | 73 -------------------------------------- drivers/cxl/core/regs.c | 2 ++ drivers/cxl/cxl.h | 6 ++++ drivers/cxl/port.c | 78 +++++++++++++++++++++++++++++++++++++++++ 4 files changed, 86 insertions(+), 73 deletions(-) diff --git a/drivers/cxl/core/pci.c b/drivers/cxl/core/pci.c index b50551601c2e..317cd0a91ffe 100644 --- a/drivers/cxl/core/pci.c +++ b/drivers/cxl/core/pci.c @@ -748,79 +748,6 @@ static bool cxl_handle_endpoint_ras(struct cxl_dev_sta= te *cxlds) =20 #ifdef CONFIG_PCIEAER_CXL =20 -static void cxl_dport_map_rch_aer(struct cxl_dport *dport) -{ - resource_size_t aer_phys; - struct device *host; - u16 aer_cap; - - aer_cap =3D cxl_rcrb_to_aer(dport->dport_dev, dport->rcrb.base); - if (aer_cap) { - host =3D dport->reg_map.host; - aer_phys =3D aer_cap + dport->rcrb.base; - dport->regs.dport_aer =3D devm_cxl_iomap_block(host, aer_phys, - sizeof(struct aer_capability_regs)); - } -} - -static void cxl_dport_map_ras(struct cxl_dport *dport) -{ - struct cxl_register_map *map =3D &dport->reg_map; - struct device *dev =3D dport->dport_dev; - - if (!map->component_map.ras.valid) - dev_dbg(dev, "RAS registers not found\n"); - else if (cxl_map_component_regs(map, &dport->regs.component, - BIT(CXL_CM_CAP_CAP_ID_RAS))) - dev_dbg(dev, "Failed to map RAS capability.\n"); -} - -static void cxl_disable_rch_root_ints(struct cxl_dport *dport) -{ - void __iomem *aer_base =3D dport->regs.dport_aer; - u32 aer_cmd_mask, aer_cmd; - - if (!aer_base) - return; - - /* - * Disable RCH root port command interrupts. - * CXL 3.0 12.2.1.1 - RCH Downstream Port-detected Errors - * - * This sequence may not be necessary. CXL spec states disabling - * the root cmd register's interrupts is required. But, PCI spec - * shows these are disabled by default on reset. - */ - aer_cmd_mask =3D (PCI_ERR_ROOT_CMD_COR_EN | - PCI_ERR_ROOT_CMD_NONFATAL_EN | - PCI_ERR_ROOT_CMD_FATAL_EN); - aer_cmd =3D readl(aer_base + PCI_ERR_ROOT_COMMAND); - aer_cmd &=3D ~aer_cmd_mask; - writel(aer_cmd, aer_base + PCI_ERR_ROOT_COMMAND); -} - -/** - * cxl_dport_init_ras_reporting - Setup CXL RAS report on this dport - * @dport: the cxl_dport that needs to be initialized - * @host: host device for devm operations - */ -void cxl_dport_init_ras_reporting(struct cxl_dport *dport, struct device *= host) -{ - dport->reg_map.host =3D host; - cxl_dport_map_ras(dport); - - if (dport->rch) { - struct pci_host_bridge *host_bridge =3D to_pci_host_bridge(dport->dport_= dev); - - if (!host_bridge->native_aer) - return; - - cxl_dport_map_rch_aer(dport); - cxl_disable_rch_root_ints(dport); - } -} -EXPORT_SYMBOL_NS_GPL(cxl_dport_init_ras_reporting, "CXL"); - static void cxl_handle_rdport_cor_ras(struct cxl_dev_state *cxlds, struct cxl_dport *dport) { diff --git a/drivers/cxl/core/regs.c b/drivers/cxl/core/regs.c index 5ca7b0eed568..b8e767a9571c 100644 --- a/drivers/cxl/core/regs.c +++ b/drivers/cxl/core/regs.c @@ -199,6 +199,7 @@ void __iomem *devm_cxl_iomap_block(struct device *dev, = resource_size_t addr, =20 return ret_val; } +EXPORT_SYMBOL_NS_GPL(devm_cxl_iomap_block, "CXL"); =20 int cxl_map_component_regs(const struct cxl_register_map *map, struct cxl_component_regs *regs, @@ -517,6 +518,7 @@ u16 cxl_rcrb_to_aer(struct device *dev, resource_size_t= rcrb) =20 return offset; } +EXPORT_SYMBOL_NS_GPL(cxl_rcrb_to_aer, "CXL"); =20 static resource_size_t cxl_rcrb_to_linkcap(struct device *dev, struct cxl_= dport *dport) { diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index ba08b77b65da..0dc43bfba76a 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -313,6 +313,12 @@ int cxl_setup_regs(struct cxl_register_map *map); struct cxl_dport; resource_size_t cxl_rcd_component_reg_phys(struct device *dev, struct cxl_dport *dport); + +u16 cxl_rcrb_to_aer(struct device *dev, resource_size_t rcrb); + +void __iomem *devm_cxl_iomap_block(struct device *dev, resource_size_t add= r, + resource_size_t length); + int cxl_dport_map_rcd_linkcap(struct pci_dev *pdev, struct cxl_dport *dpor= t); =20 #define CXL_RESOURCE_NONE ((resource_size_t) -1) diff --git a/drivers/cxl/port.c b/drivers/cxl/port.c index fe4b593331da..7b61f09347a5 100644 --- a/drivers/cxl/port.c +++ b/drivers/cxl/port.c @@ -6,6 +6,7 @@ =20 #include "cxlmem.h" #include "cxlpci.h" +#include "cxl.h" =20 /** * DOC: cxl port @@ -57,6 +58,83 @@ static int discover_region(struct device *dev, void *unu= sed) return 0; } =20 +#ifdef CONFIG_PCIEAER_CXL + +static void cxl_dport_map_rch_aer(struct cxl_dport *dport) +{ + resource_size_t aer_phys; + struct device *host; + u16 aer_cap; + + aer_cap =3D cxl_rcrb_to_aer(dport->dport_dev, dport->rcrb.base); + if (aer_cap) { + host =3D dport->reg_map.host; + aer_phys =3D aer_cap + dport->rcrb.base; + dport->regs.dport_aer =3D devm_cxl_iomap_block(host, aer_phys, + sizeof(struct aer_capability_regs)); + } +} + +static void cxl_dport_map_ras(struct cxl_dport *dport) +{ + struct cxl_register_map *map =3D &dport->reg_map; + struct device *dev =3D dport->dport_dev; + + if (!map->component_map.ras.valid) + dev_dbg(dev, "RAS registers not found\n"); + else if (cxl_map_component_regs(map, &dport->regs.component, + BIT(CXL_CM_CAP_CAP_ID_RAS))) + dev_dbg(dev, "Failed to map RAS capability.\n"); +} + +static void cxl_disable_rch_root_ints(struct cxl_dport *dport) +{ + void __iomem *aer_base =3D dport->regs.dport_aer; + u32 aer_cmd_mask, aer_cmd; + + if (!aer_base) + return; + + /* + * Disable RCH root port command interrupts. + * CXL 3.0 12.2.1.1 - RCH Downstream Port-detected Errors + * + * This sequence may not be necessary. CXL spec states disabling + * the root cmd register's interrupts is required. But, PCI spec + * shows these are disabled by default on reset. + */ + aer_cmd_mask =3D (PCI_ERR_ROOT_CMD_COR_EN | + PCI_ERR_ROOT_CMD_NONFATAL_EN | + PCI_ERR_ROOT_CMD_FATAL_EN); + aer_cmd =3D readl(aer_base + PCI_ERR_ROOT_COMMAND); + aer_cmd &=3D ~aer_cmd_mask; + writel(aer_cmd, aer_base + PCI_ERR_ROOT_COMMAND); +} + +/** + * cxl_dport_init_ras_reporting - Setup CXL RAS report on this dport + * @dport: the cxl_dport that needs to be initialized + * @host: host device for devm operations + */ +void cxl_dport_init_ras_reporting(struct cxl_dport *dport, struct device *= host) +{ + dport->reg_map.host =3D host; + cxl_dport_map_ras(dport); + + if (dport->rch) { + struct pci_host_bridge *host_bridge =3D to_pci_host_bridge(dport->dport_= dev); + + if (!host_bridge->native_aer) + return; + + cxl_dport_map_rch_aer(dport); + cxl_disable_rch_root_ints(dport); + } +} +EXPORT_SYMBOL_NS_GPL(cxl_dport_init_ras_reporting, "CXL"); + +#endif /* CONFIG_PCIEAER_CXL */ + static int cxl_switch_port_probe(struct cxl_port *port) { struct cxl_hdm *cxlhdm; --=20 2.34.1 From nobody Thu Dec 18 05:05:03 2025 Received: from NAM02-BN1-obe.outbound.protection.outlook.com (mail-bn1nam02on2051.outbound.protection.outlook.com [40.107.212.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B747E23A98D; Tue, 3 Jun 2025 17:24:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.212.51 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1748971451; cv=fail; b=b/07AXGutmkVNu+qH/gkd+eZRNGFAsazhTVfHZuaBUtHpayw/9j3FuOEICzr4v6UySHXxgjPZDt8AjVvJjxZHsETuutQZhdyMuld3rXNf1QqYq5trvxtbSY3zRJyOu1++k9/P3Lvvzz/pyjF9peCb3nrpuOF0Xy3YScNw1qv/Wc= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1748971451; c=relaxed/simple; bh=DTi8BN2RcdSPDVnQjsXHYJh56+yfWKOisZO3GoIvyP8=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=nU4n/2dEcGPwRNtPTE4dtIJFEKmMm3bay5tUvgZxKJhbbs4u7Jm3R66cNNj9d3+7c72lfn0Y8/ZVOpbqyLBVMfA6YtBNVL8c9l8e2FHsrbCLoCUcTqvS3twl8qTE4XNSijZOKnzigXHcWtX3b0XEf1E5ZHIMnVcfR4Xcqw4thzQ= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=SjRltPtJ; arc=fail smtp.client-ip=40.107.212.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="SjRltPtJ" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=WyiYgEEAEYU2ZYysEnCWBJWvO2GfR88r1CkxP0ddt7C+1AIHHA6nsI09Q//no7U5/KpjsPeRH97d22nM24EMOYbbILTJ+SRCFR0Dnm5rhDCyPv3kFQi98qsClWyf1tlCizQHFEXjVqwBU1qOVdx8v0nn8fySiNym7yqMuqhltRRZ/ZYd3vq9MOkNTRL0SJeFevQMHfwQwsi/XxU3LmIkMBcH9jVEFj1i0WOKcDLxuwhTWFkod0R6tHln5I/NOl9aqNX7ksYz6H1Wuy95n2wxl3t2z1OvbCs9qdsB8EonAhOozE6Y5YuJzwWvHZ5dSnWXAO0CHGPQfC7XfmFihGqUUw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=s0dNDMGZ66tEG/1Zn+kDI5ynK/ToIm9FmzANk8jJ8NM=; b=vkQF6wEpMBRhsozn1FPdCUfEb8mBR4YfaXRitgqKAZFXl8kxQYW47i4etlAiauf4oBQGtNm3xsVsLF9B7undJ+3PaZ5ogv5V9wjg2HaQRFO3FpTLpewD+UAMeiU63xvBhOH+uISlpYWs7mssG8gfigqBogbQ4Ym0lxd/pZWAmjgAQNV5HYXqOXJdIUnHI1HOAHLvJmqpehXqzZxCgM6HP/6/CXqUN7MNgTcoJGYo/T60pWWOBuQq3EKfHEJ103vDeP45dDIY2uaHxapDiGaGiMhOp0F+rpYN5NM+qYAL1c1QjEdwdPQ1O8dm1cfik6VNVqCsJwkfADP9DrjGMUXJpA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=stgolabs.net smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=s0dNDMGZ66tEG/1Zn+kDI5ynK/ToIm9FmzANk8jJ8NM=; b=SjRltPtJCg7VlblkSGHPMANEIjKOEwV+Pi9y8XYl8S9N8rcHmw2Go/wUAptbHorEVhu6KlawtZeqpDCUZdofo9dqDe+XkZ2g/EOtBRToycLoRqjwAKn+3rOnkWqc5DWmjKMCUz5p1OcOwuazYbsNtUqOfp0BoWg/I2YirAfHLx0= Received: from MN0PR04CA0019.namprd04.prod.outlook.com (2603:10b6:208:52d::33) by DS7PR12MB6166.namprd12.prod.outlook.com (2603:10b6:8:99::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8769.31; Tue, 3 Jun 2025 17:24:05 +0000 Received: from BL6PEPF00020E64.namprd04.prod.outlook.com (2603:10b6:208:52d:cafe::da) by MN0PR04CA0019.outlook.office365.com (2603:10b6:208:52d::33) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8792.19 via Frontend Transport; Tue, 3 Jun 2025 17:24:05 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by BL6PEPF00020E64.mail.protection.outlook.com (10.167.249.25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.8792.29 via Frontend Transport; Tue, 3 Jun 2025 17:24:05 +0000 Received: from ethanolx7ea3host.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Tue, 3 Jun 2025 12:24:03 -0500 From: Terry Bowman To: , , , , , , , , , , , , , , , , , , , , , , , , , , Subject: [PATCH v9 07/16] cxl/pci: Map CXL Endpoint Port and CXL Switch Port RAS registers Date: Tue, 3 Jun 2025 12:22:30 -0500 Message-ID: <20250603172239.159260-8-terry.bowman@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250603172239.159260-1-terry.bowman@amd.com> References: <20250603172239.159260-1-terry.bowman@amd.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL6PEPF00020E64:EE_|DS7PR12MB6166:EE_ X-MS-Office365-Filtering-Correlation-Id: c87e5898-cf3e-4455-6a87-08dda2c370d0 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|1800799024|376014|7416014|36860700013|921020; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?ySDII6nQoUPJe1NKnFruc3R3nXNbxpkUIkGP55l8gKcvbxjhAGTa58KVRkxX?= =?us-ascii?Q?cXeSrT9qWAOu55hH2U4sOZih2Sk03LwvGEGP7vhILuH1H8dX6VY6xKgvMV1K?= =?us-ascii?Q?o+4MChL6BH2ZyLqK8APyq4WmgaZvGnJcmZXzCTADTMb9tvIB2kZmKSzNP9kb?= =?us-ascii?Q?VVQve8HPxXMEx+RLVx/9+CbYZJ8Q54hWrsfJpiHo8AUtskv7hS+aX8es/q6q?= =?us-ascii?Q?ziw1A+Cxrj9Yu7sB7PpVeguolLX0H9BWTyCxWXmsPvtQzoQito6WfDvSm6kG?= =?us-ascii?Q?OvjHSn+g97RNY6lCn4Q9As4ql7b8xCbTVojiWYOqE+3us7VjzGpyQTLNQ/9Z?= =?us-ascii?Q?XIAGZII/+mfcjdooAODAbZdjlGHzgIuu2QffOfL9CH7NUbhnow9d+sbh3H7T?= =?us-ascii?Q?DU5AYSR8vnM3no1JPshC+doBj7u4L8QBZ+JEWPJ3BBl880pgHvr6v+lH4Kj9?= =?us-ascii?Q?BJ9tlnXBJv/2zWfTL2U7m+Vgi5FTIanXnRBMyUAibEkxN3EazGDbqfYi+aHC?= =?us-ascii?Q?3i7rIjrFk7sAe5xaTddWvH2CtB2LrRbUspwIDgBJvp71csNXZEhQbqWHYagP?= =?us-ascii?Q?U4jkaaZcAPSxVza2Tp6XZ4kq8C/n6Q5goU4t3lVpWnioW95Nhwnf3Cx3UGmp?= =?us-ascii?Q?m94GO7tvDXlJVXTdcNLMoQHF4l1XF0qRx6bxoT650i3pXOaQxalbJNTLwy+9?= =?us-ascii?Q?/hBTKWJOvoC1VP4/CJLcdEXGM6GMxNG1YSu6+pzZx+bPfrLpol6QIfIFg3nP?= =?us-ascii?Q?vtGeRgMV9Jj+6s0ZIAzMK6s9hWoyG+NWCgH5JyEhMMVM08kGFju9S/eE9/nX?= =?us-ascii?Q?VDpkKjir0cnJ1bGNK/KrTT5p/3Y2dyqHKhmmasSxxDSR5OX7ung4gv0RE9Vj?= =?us-ascii?Q?nyQXyWRNfkJJLndswaHhJ3xKNOIYfgE+pX67ojcldQzJfIAlk/sx9dwKXDPf?= =?us-ascii?Q?EXu/KQNE4w5yZ9y19QzdZqj0wn0NX5Sji0VSJozT3pr40/hQMi+wdOPogIfo?= =?us-ascii?Q?fJuz2tExZb42d7k/w4fLv8sfqYAXIzabm5H2XT636L4HIg2LVcOmRtO9NTPe?= =?us-ascii?Q?GE7rzfjc7sdrTohrYhJsXtuMwpqvpiBXoQv9HksL84P18LmiqQ+c5MAYte7t?= =?us-ascii?Q?5QP1FZpdGgEffMz8agKi9tVVlD2b11B2AB5gmuvK8KkZs+JlBvcn3PM/yI7w?= =?us-ascii?Q?n6C79X0JpCHnRpa45DhdmK6hG8x+KZizlF18KJTtalyfC+DNv4lbvzRXaIku?= =?us-ascii?Q?iY+UofRAtxDTyvFX/iDN47oeKB3rL+o4nH/A4Y+adMAuCZ0McXHVRmxbrcE9?= =?us-ascii?Q?yVgltfrEAZGosihiZHIgIAk87iOpw/bn+0j61dorcFTafbN4BbM7gzfPXYfE?= =?us-ascii?Q?5Pvd2OtPLsfiCKNl/2Zvtb4/w0P8s2FHhkhY7JwZ7VCLAMWbr5eYyTOfe99J?= =?us-ascii?Q?f5eQJGpGsVRz7rT9vKVj8WZSqnhL1b+XAfJurmbGmWM0Tz1EoR0fIHLW8e9u?= =?us-ascii?Q?BMqtFD7s6jCXRgWkx6Sa8hQd2IHzzaYbSh6tFm6slDL7dTCQjX2xHk08VQ?= =?us-ascii?Q?=3D=3D?= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(82310400026)(1800799024)(376014)(7416014)(36860700013)(921020);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Jun 2025 17:24:05.1044 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: c87e5898-cf3e-4455-6a87-08dda2c370d0 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BL6PEPF00020E64.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR12MB6166 Content-Type: text/plain; charset="utf-8" CXL Endpoint (EP) Ports may include Root Ports (RP) or Downstream Switch Ports (DSP). CXL RPs and DSPs contain RAS registers that require memory mapping to enable RAS logging. This initialization is currently missing and must be added for CXL RPs and DSPs. Update cxl_dport_init_ras_reporting() to support RP and DSP RAS mapping. Add alongside the existing Restricted CXL Host Downstream Port RAS mapping. Update cxl_endpoint_port_probe() to invoke cxl_dport_init_ras_reporting(). This will initiate the RAS mapping for CXL RPs and DSPs when each CXL EP is created and added to the EP port. Signed-off-by: Terry Bowman --- drivers/cxl/cxl.h | 2 ++ drivers/cxl/mem.c | 3 ++- drivers/cxl/port.c | 55 +++++++++++++++++++++++++++++++++++++++++++++- 3 files changed, 58 insertions(+), 2 deletions(-) diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index 0dc43bfba76a..73be66ef36a2 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -590,6 +590,7 @@ struct cxl_dax_region { * @parent_dport: dport that points to this port in the parent * @decoder_ida: allocator for decoder ids * @reg_map: component and ras register mapping parameters + * @uport_regs: mapped component registers * @nr_dports: number of entries in @dports * @hdm_end: track last allocated HDM decoder instance for allocation orde= ring * @commit_end: cursor to track highest committed decoder for commit order= ing @@ -610,6 +611,7 @@ struct cxl_port { struct cxl_dport *parent_dport; struct ida decoder_ida; struct cxl_register_map reg_map; + struct cxl_component_regs uport_regs; int nr_dports; int hdm_end; int commit_end; diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c index 9675243bd05b..29dc4a624b15 100644 --- a/drivers/cxl/mem.c +++ b/drivers/cxl/mem.c @@ -166,7 +166,8 @@ static int cxl_mem_probe(struct device *dev) else endpoint_parent =3D &parent_port->dev; =20 - cxl_dport_init_ras_reporting(dport, dev); + if (dport->rch) + cxl_dport_init_ras_reporting(dport, dev); =20 scoped_guard(device, endpoint_parent) { if (!endpoint_parent->driver) { diff --git a/drivers/cxl/port.c b/drivers/cxl/port.c index 7b61f09347a5..0f7c4010ba58 100644 --- a/drivers/cxl/port.c +++ b/drivers/cxl/port.c @@ -111,6 +111,17 @@ static void cxl_disable_rch_root_ints(struct cxl_dport= *dport) writel(aer_cmd, aer_base + PCI_ERR_ROOT_COMMAND); } =20 +static void cxl_uport_init_ras_reporting(struct cxl_port *port, + struct device *host) +{ + struct cxl_register_map *map =3D &port->reg_map; + + map->host =3D host; + if (cxl_map_component_regs(map, &port->uport_regs, + BIT(CXL_CM_CAP_CAP_ID_RAS))) + dev_dbg(&port->dev, "Failed to map RAS capability\n"); +} + /** * cxl_dport_init_ras_reporting - Setup CXL RAS report on this dport * @dport: the cxl_dport that needs to be initialized @@ -119,7 +130,6 @@ static void cxl_disable_rch_root_ints(struct cxl_dport = *dport) void cxl_dport_init_ras_reporting(struct cxl_dport *dport, struct device *= host) { dport->reg_map.host =3D host; - cxl_dport_map_ras(dport); =20 if (dport->rch) { struct pci_host_bridge *host_bridge =3D to_pci_host_bridge(dport->dport_= dev); @@ -127,12 +137,51 @@ void cxl_dport_init_ras_reporting(struct cxl_dport *d= port, struct device *host) if (!host_bridge->native_aer) return; =20 + cxl_dport_map_ras(dport); cxl_dport_map_rch_aer(dport); cxl_disable_rch_root_ints(dport); + return; } + + if (cxl_map_component_regs(&dport->reg_map, &dport->regs.component, + BIT(CXL_CM_CAP_CAP_ID_RAS))) + dev_dbg(dport->dport_dev, "Failed to map RAS capability\n"); + } EXPORT_SYMBOL_NS_GPL(cxl_dport_init_ras_reporting, "CXL"); =20 +static void cxl_switch_port_init_ras(struct cxl_port *port) +{ + struct device *dev __free(put_device) =3D get_device(&port->dev); + + if (is_cxl_root(to_cxl_port(port->dev.parent))) + return; + + /* Check for parent DSP */ + if (port->parent_dport) + cxl_dport_init_ras_reporting(port->parent_dport, dev); + + cxl_uport_init_ras_reporting(port, dev); +} + +static void cxl_endpoint_port_init_ras(struct cxl_port *port) +{ + struct cxl_dport *dport; + struct cxl_memdev *cxlmd =3D to_cxl_memdev(port->uport_dev); + struct cxl_port *parent_port __free(put_cxl_port) =3D + cxl_mem_find_port(cxlmd, &dport); + + if (!dport || !dev_is_pci(dport->dport_dev)) { + dev_err(&port->dev, "CXL port topology not found\n"); + return; + } + + cxl_dport_init_ras_reporting(dport, &cxlmd->dev); +} + +#else +static void cxl_endpoint_port_init_ras(struct cxl_port *port) { } +static void cxl_switch_port_init_ras(struct cxl_port *port) { } #endif /* CONFIG_PCIEAER_CXL */ =20 static int cxl_switch_port_probe(struct cxl_port *port) @@ -149,6 +198,8 @@ static int cxl_switch_port_probe(struct cxl_port *port) =20 cxl_switch_parse_cdat(port); =20 + cxl_switch_port_init_ras(port); + cxlhdm =3D devm_cxl_setup_hdm(port, NULL); if (!IS_ERR(cxlhdm)) return devm_cxl_enumerate_decoders(cxlhdm, NULL); @@ -203,6 +254,8 @@ static int cxl_endpoint_port_probe(struct cxl_port *por= t) if (rc) return rc; =20 + cxl_endpoint_port_init_ras(port); + /* * Now that all endpoint decoders are successfully enumerated, try to * assemble regions from committed decoders --=20 2.34.1 From nobody Thu Dec 18 05:05:03 2025 Received: from NAM02-BN1-obe.outbound.protection.outlook.com (mail-bn1nam02on2081.outbound.protection.outlook.com [40.107.212.81]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4001F23C4F8; Tue, 3 Jun 2025 17:24:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.212.81 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1748971462; cv=fail; b=cFiqJiQTXFah9/PYibslf3TwdEH7Zxo7tG02s1vVyss/xo2zhPALWpROCPhO7v3530rUhjX8o5X2sH8citVUIv1shG6Jz9Opt8ur2PeISpTNUv0X3OKJhEMR6krRQhoe9TFuF+glhqDL5mEzXIZ6v80gP8/1HKysS9lOvpvj0JA= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1748971462; c=relaxed/simple; bh=/KQwqy1ZbTlKRDzl95Q38jBjpEK60aDJRRdBlcrEsCk=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=n5QRt70FfZgXKCgm/Ih78ieTtYxrffhrhE+Pnw6UBs6p9rhRi4QaqmZUTz887cnW5aFZW1S7xcGa7C+xBJtYkAGZNzM1nezWhLsemiMgtProJv+Q8hDUNvAtmW4SjlAAo+BbI52xZXgoNW85aQ7s/ht2m4T+uTsZvTQlb5HYk2E= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=ZvC9iLl1; arc=fail smtp.client-ip=40.107.212.81 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="ZvC9iLl1" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=gN1a0xKbUjWc6Yx2pJBOOno82qKtsx18pa38sznvPrGaP2u9LX2USTii6oVRdIoGNjYXMy4cdiL74/Jt51oquZxSgz/SQHmbEMOof1j/a/+e7P9u2wSpH77qmPrI9nMO4BIQovfEs/zsP0WI55kLCLZa0jG+BIcnInyueDH+xKWAM72m4sR8PQrShJke1nl14kot614q9UrIFlOpsDbuPCOp1Fhf5w56GrNaUmJHc38AKTFZZ9M85YN7xTyb5LeJZMBTZc+AJD0bwf76ACnSbfsfQq2oR4JDee2Af9dwyMCK09ZNUaVRkCZsCSn2RdqzWPPj7S07ES9aNC2D5DCKDQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=KPnKW/xp5c2W2wdCMrUt016efRwffIhKGY1Qh8CsN8g=; b=JJeYE9UjaRpV3uau9IUHNM82RIrtXeT/kRvEpTqGGS71ob1CLeDnUeCbzl3Bx3Xc/IOCqaSZROu2DjAc/FIRzejrXqKN2FMXRWrUQUUEMdZ1ccuPAaS6dgc2YyF72Gdg5Q2yP1+PtH4OeyMkKGhQH2hzGWRv0bboG8VVytTcz2IsbGrTVlfDmjscp0GSbVnhhqOSVa5+TUH18hPde0CM6lX+x4XURFubM4PrvAyPOhuWGPaS12MP3NDCOdp1qw5Xfvg1SQh1zJukAlZBuNmkzAagceGI2PCwF/HRPcENNX7ogZV6AAVHquDUCUB1JaKWitqEGB7UrZy1G7HDPiMXEw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=stgolabs.net smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=KPnKW/xp5c2W2wdCMrUt016efRwffIhKGY1Qh8CsN8g=; b=ZvC9iLl1/pWe+tpVoAi2vHIDi6ekB1+lOppzWcRCt2q5S65NEBEQZsTXwUM/h2Vk9NtD/Vv4kQcPGzNZ7aVZYhqJsotzqStospI9sOsZ+oUVclsX68iNEIzelFcZbsJEhgbIQsUdNMS57XFbyUIXY3sdjdGHuiKMKAYFstA/7k0= Received: from MN0PR04CA0022.namprd04.prod.outlook.com (2603:10b6:208:52d::21) by SJ2PR12MB9116.namprd12.prod.outlook.com (2603:10b6:a03:557::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8792.33; Tue, 3 Jun 2025 17:24:16 +0000 Received: from BL6PEPF00020E64.namprd04.prod.outlook.com (2603:10b6:208:52d:cafe::e3) by MN0PR04CA0022.outlook.office365.com (2603:10b6:208:52d::21) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8792.19 via Frontend Transport; Tue, 3 Jun 2025 17:24:16 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by BL6PEPF00020E64.mail.protection.outlook.com (10.167.249.25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.8792.29 via Frontend Transport; Tue, 3 Jun 2025 17:24:16 +0000 Received: from ethanolx7ea3host.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Tue, 3 Jun 2025 12:24:15 -0500 From: Terry Bowman To: , , , , , , , , , , , , , , , , , , , , , , , , , , Subject: [PATCH v9 08/16] cxl/pci: Update RAS handler interfaces to also support CXL Ports Date: Tue, 3 Jun 2025 12:22:31 -0500 Message-ID: <20250603172239.159260-9-terry.bowman@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250603172239.159260-1-terry.bowman@amd.com> References: <20250603172239.159260-1-terry.bowman@amd.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL6PEPF00020E64:EE_|SJ2PR12MB9116:EE_ X-MS-Office365-Filtering-Correlation-Id: 8fc3653a-4cbc-4821-1db2-08dda2c3775e X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|82310400026|1800799024|376014|36860700013|921020; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?kLy6cOkoFhpqFyad3q8sDkDeP7UbUrg2VWNonVUJkKq3fjRZ2VCGA5EKOBV4?= =?us-ascii?Q?Dg7miMTmhEh7YkK92eGppjZwJlxbIGB5SxztMBmox77o0KuCBodTI6m/apjT?= =?us-ascii?Q?q5lBHeM88M2vgB9UlegFa+TCzWeNL/axVCv7IfcCP4SxMkp1xauoIlZwBD2l?= =?us-ascii?Q?CI/BbD0uxlizFKV8W5cZrADKdYSnhpKL8hpOVebK3Xhl9uBGs/xKBL6m3ILf?= =?us-ascii?Q?phOro73tmpOatNzlSlUSNEnvY4gOjimmhr3GrwVGYMfhsbmc7X7hLSWd3kc0?= =?us-ascii?Q?BpD/vBmDsdtAqlWzpViOiUTNnziJSQPt5AvELl/QxerXMiHBI8/Cq4OfBud5?= =?us-ascii?Q?1z8pfR1YHBNsGO/VVuSgTa9Q4OekQ/m+xbV3ApTvkhxQau/ZEUKKl/dNQiQc?= =?us-ascii?Q?InNsg/CeLjPkB1pyjBbnWmW62bkkbvoESNMTn48Qwrn6nKpQdcVQKDWR1ucN?= =?us-ascii?Q?tCnu5WCsPK+DdLvuMDsfMLqrWUftPxkcyegym+tRJ0GeYNuYpL2CCoZqr6ao?= =?us-ascii?Q?HoaiRO8OENDyUmXDEqoaOvCglTIGpw9td3KUClj7O01CL9RmCaO4H9Em6f6b?= =?us-ascii?Q?3Va3TMc/LzODvG9RNVv8vFNwNPNEyRE17w+1GgbpTxJLQF/q1uLWd5P/n8Qt?= =?us-ascii?Q?oJKxWPSpksYrp+sGijO79mNE3yoga5BRzP63NriEVid+LUnB46cZ5g4X9XXI?= =?us-ascii?Q?Ijb6XOWQbUgQvpV4E7+Ks+O3qlkkN0o2vCzIn5Yojwv3VgewOg2F9e5Hs9yL?= =?us-ascii?Q?4tSrQMcXbPGPdIEWCmtslnrCOEpSJJ0hnltte3cOTUOXiyFLkn1lRrJ9x2TZ?= =?us-ascii?Q?jtYuE0dSrs+ZZPWnHyIRUXyBqrIfC9gGO7nqB+vX0Hz2mufvF+bSSQymdrml?= =?us-ascii?Q?bt0Gyl35cQrqCWTAokbUSqgcfQotiXaVyNp2aK6hAMc/SWjcvE2QD0G3p91U?= =?us-ascii?Q?qWvO7DCTPvj4OaHcKqRlr/s+KiqvJvrzHWGqkEwsLeGCvMMtVd6AYeBLRdBI?= =?us-ascii?Q?t7QSZSnY8TCTB8hn/rmM1FvWntQqg2teM/rYVQ5CCFYNp5Eu2YToaPL0+ruG?= =?us-ascii?Q?daU0QDUf1/UBT91ygwD70jK4Ao/nRZFb+D9QDwE32crzOHj2bv+kbrfoVJgJ?= =?us-ascii?Q?SQSDxig79SpMsItpvdXRu31I/v22GJZ89hc8QAyVsVL8xlCz+YVHUNgyeesa?= =?us-ascii?Q?beuPd53/Kg7GBUYidDqSknF2hQTMeD4Xaoerp4AGb9CGC2YlgyHAwmr6RWRO?= =?us-ascii?Q?f3Ki6oQBn7IlLzQt5kScXZAJfWONEZD1GZ5x3UGPjqigB+h77hgJbPqTcki5?= =?us-ascii?Q?lFRdA1j0SDAtBmSIipanFpInJWSTW6D2/xku3wlU/YwXu1w9z2hi7kGmQLxq?= =?us-ascii?Q?7gr/8jzmeSF+3c9eeaRf1BWGw3ZSUVA5BnuPy8gqZSFUwAWLWYcnVY4wUcJL?= =?us-ascii?Q?XXf+vNQyEQADRdmzquhKqAJnPU432jLsLOiGZh3MDUmcDWjUEabXTClhhvLW?= =?us-ascii?Q?juQElx/+ldMXiad6sRlWedlxAWg64Lv9yuWsOOgV4VicfFvRd8QGXbbyJg?= =?us-ascii?Q?=3D=3D?= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(7416014)(82310400026)(1800799024)(376014)(36860700013)(921020);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Jun 2025 17:24:16.0976 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 8fc3653a-4cbc-4821-1db2-08dda2c3775e X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BL6PEPF00020E64.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ2PR12MB9116 Content-Type: text/plain; charset="utf-8" CXL PCIe Port Protocol Error handling support will be added to the CXL drivers in the future. In preparation, rename the existing interfaces to support handling all CXL PCIe Port Protocol Errors. The driver's RAS support functions currently rely on a 'struct cxl_dev_state' type parameter, which is not available for CXL Port devices. However, since the same CXL RAS capability structure is needed across most CXL components and devices, a common handling approach should be adopted. To accommodate this, update the __cxl_handle_cor_ras() and __cxl_handle_ras() functions to use a `struct device` instead of `struct cxl_dev_state`. No functional changes are introduced. [1] CXL 3.1 Spec, 8.2.4 CXL.cache and CXL.mem Registers Signed-off-by: Terry Bowman Reviewed-by: Alejandro Lucero Reviewed-by: Ira Weiny Reviewed-by: Gregory Price Reviewed-by: Dave Jiang Reviewed-by: Jonathan Cameron Reviewed-by: Kuppuswamy Sathyanarayanan --- drivers/cxl/core/pci.c | 17 ++++++++--------- 1 file changed, 8 insertions(+), 9 deletions(-) diff --git a/drivers/cxl/core/pci.c b/drivers/cxl/core/pci.c index 317cd0a91ffe..78735da7e63d 100644 --- a/drivers/cxl/core/pci.c +++ b/drivers/cxl/core/pci.c @@ -664,7 +664,7 @@ void read_cdat_data(struct cxl_port *port) } EXPORT_SYMBOL_NS_GPL(read_cdat_data, "CXL"); =20 -static void __cxl_handle_cor_ras(struct cxl_dev_state *cxlds, +static void __cxl_handle_cor_ras(struct device *dev, void __iomem *ras_base) { void __iomem *addr; @@ -677,13 +677,13 @@ static void __cxl_handle_cor_ras(struct cxl_dev_state= *cxlds, status =3D readl(addr); if (status & CXL_RAS_CORRECTABLE_STATUS_MASK) { writel(status & CXL_RAS_CORRECTABLE_STATUS_MASK, addr); - trace_cxl_aer_correctable_error(cxlds->cxlmd, status); + trace_cxl_aer_correctable_error(to_cxl_memdev(dev), status); } } =20 static void cxl_handle_endpoint_cor_ras(struct cxl_dev_state *cxlds) { - return __cxl_handle_cor_ras(cxlds, cxlds->regs.ras); + return __cxl_handle_cor_ras(&cxlds->cxlmd->dev, cxlds->regs.ras); } =20 /* CXL spec rev3.0 8.2.4.16.1 */ @@ -707,8 +707,7 @@ static void header_log_copy(void __iomem *ras_base, u32= *log) * Log the state of the RAS status registers and prepare them to log the * next error status. Return 1 if reset needed. */ -static bool __cxl_handle_ras(struct cxl_dev_state *cxlds, - void __iomem *ras_base) +static bool __cxl_handle_ras(struct device *dev, void __iomem *ras_base) { u32 hl[CXL_HEADERLOG_SIZE_U32]; void __iomem *addr; @@ -735,7 +734,7 @@ static bool __cxl_handle_ras(struct cxl_dev_state *cxld= s, } =20 header_log_copy(ras_base, hl); - trace_cxl_aer_uncorrectable_error(cxlds->cxlmd, status, fe, hl); + trace_cxl_aer_uncorrectable_error(to_cxl_memdev(dev), status, fe, hl); writel(status & CXL_RAS_UNCORRECTABLE_STATUS_MASK, addr); =20 return true; @@ -743,7 +742,7 @@ static bool __cxl_handle_ras(struct cxl_dev_state *cxld= s, =20 static bool cxl_handle_endpoint_ras(struct cxl_dev_state *cxlds) { - return __cxl_handle_ras(cxlds, cxlds->regs.ras); + return __cxl_handle_ras(&cxlds->cxlmd->dev, cxlds->regs.ras); } =20 #ifdef CONFIG_PCIEAER_CXL @@ -751,13 +750,13 @@ static bool cxl_handle_endpoint_ras(struct cxl_dev_st= ate *cxlds) static void cxl_handle_rdport_cor_ras(struct cxl_dev_state *cxlds, struct cxl_dport *dport) { - return __cxl_handle_cor_ras(cxlds, dport->regs.ras); + return __cxl_handle_cor_ras(&cxlds->cxlmd->dev, dport->regs.ras); } =20 static bool cxl_handle_rdport_ras(struct cxl_dev_state *cxlds, struct cxl_dport *dport) { - return __cxl_handle_ras(cxlds, dport->regs.ras); + return __cxl_handle_ras(&cxlds->cxlmd->dev, dport->regs.ras); } =20 /* --=20 2.34.1 From nobody Thu Dec 18 05:05:03 2025 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2076.outbound.protection.outlook.com [40.107.220.76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1E3FF1F4C8C; Tue, 3 Jun 2025 17:24:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.220.76 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1748971497; cv=fail; b=Cv9POotVz0WpHIrSohkE1EAyRjLXBdEcDHF3/WeuORJBv2jpnDt/Z6+zkUix063ppHnF4vLlV02KddQyDcIS3jTw624GQbizM2lfbTjk4jBSUwVaAy+vlAJVxAl6Ui4flcLOZMz7ZgTK03vJOh0pqc+os+YQwgrMV/lXLIqEiXU= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1748971497; c=relaxed/simple; bh=5Ob5lDqm2IBdhcRJVuC8Iq47QenJgA02VTeY3HwSup8=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=O+ccpHu66PRWD3wxQ+bwnVnPdE+BE08Dki43Ekt8aies0F41MRQGeYDsFtUeo3TZF49kv0CGC4PCgyiHO7ZxibGxKlvjUOR+YS6REVekrf0hDB/4YzkjJd6rcODlkkWgz+iVOB9a4OGYbfSkrfns+Afxhx1kxu/0ehxqRO6asJM= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=37BAwVJr; arc=fail smtp.client-ip=40.107.220.76 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="37BAwVJr" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=AIKV2zyyhBO119o8PfJDtXSVVWi+HoZVxoigttQBsPyKTmX079jFIw/a1fIp43A7cn7lsK8tI5JY3VREEQB8VNT95Btz7wkrqKuNP2hlGwu3GvyZz/iLxsGwaATJkCxlZyCTgyIRlYiezocSEoYDi0yjpvyGtH0ev10w/3jGs3LSzCUxBUiyKf/L7mfLZ5KEaf/Sq/A+xqJdXE3QvidHHykBkJRHAiRnCaUUeVnMfXNs9biPAKqj85809mmnoWC0JrMabq1L8gJO3833joKw9NukSalc081ewWxSuxvIhdPo6OGdko8MuNmRffnki9+JVIlY2L8YdBqpeHY/VfRa0w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=PJWq8kxUMgTDAG0DNXx57XXasyowSKzMwqiPFuHACeo=; b=QlUUeoOxgykCrix2UOQjBMho7BOM7uFUnk5328g7RFfAqo0ubGyhaJJWUzcAmtEzKe49cj0Ors3wCQ6IXKopINjz5CnP8CpG6BGM5hqq2sxVohTNae1nFIkYRtNaDQG8JwpoXLP4JyXQ3esGSTmeyJz5skDMdmho6Mf9VFjpRHaesIcF7uxAfhgrN8r5TAAH+ITdxEhxrhE2vY7A3hKIFpaW4uclaGSLDfAyqGQbOQhkc3sIGb++NNgfRhMD3kcXOssLbdcjiZJdFc2x3KbdEly67oh9PzpVEryXpBUTrYH8rI82ID7qxLk51JdqoxtW+cDaBvYiJUg/Q91LK1HrOA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=stgolabs.net smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=PJWq8kxUMgTDAG0DNXx57XXasyowSKzMwqiPFuHACeo=; b=37BAwVJrOjCvq6TT5my055jsHs33mlAl0DMzcW1OrLJBtOrgAHfL2+YSr58pU31f/s0O/UWHwG6KEyuV/diufistG17XLK0q1QQloUGv9z4Eq8RmQqGRTyLmLZDIoENFkA8hZHW7jmXfasNWcmV3CpWcwQzVEIqec1hi1TXOvMY= Received: from BL1PR13CA0200.namprd13.prod.outlook.com (2603:10b6:208:2be::25) by SJ5PPF0C60B25BF.namprd12.prod.outlook.com (2603:10b6:a0f:fc02::98a) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8655.33; Tue, 3 Jun 2025 17:24:52 +0000 Received: from BL6PEPF00020E61.namprd04.prod.outlook.com (2603:10b6:208:2be:cafe::39) by BL1PR13CA0200.outlook.office365.com (2603:10b6:208:2be::25) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8792.19 via Frontend Transport; Tue, 3 Jun 2025 17:24:51 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by BL6PEPF00020E61.mail.protection.outlook.com (10.167.249.22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.8792.29 via Frontend Transport; Tue, 3 Jun 2025 17:24:50 +0000 Received: from ethanolx7ea3host.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Tue, 3 Jun 2025 12:24:26 -0500 From: Terry Bowman To: , , , , , , , , , , , , , , , , , , , , , , , , , , Subject: [PATCH v9 09/16] cxl/pci: Log message if RAS registers are unmapped Date: Tue, 3 Jun 2025 12:22:32 -0500 Message-ID: <20250603172239.159260-10-terry.bowman@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250603172239.159260-1-terry.bowman@amd.com> References: <20250603172239.159260-1-terry.bowman@amd.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL6PEPF00020E61:EE_|SJ5PPF0C60B25BF:EE_ X-MS-Office365-Filtering-Correlation-Id: e7b1a1ee-5318-4467-0ddb-08dda2c38c28 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|36860700013|1800799024|7416014|376014|82310400026|921020; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?rYJavWLNnubiaPB4gxQPxGu1eLp3+/t3K1LEKuWR73wI/RGbQET963rn2T1b?= =?us-ascii?Q?FTtwk+88Hm99rbWOomjVp593mBJCJH68dWozcXsYivbeXwwXmU45K/fnRjH0?= =?us-ascii?Q?A41rrkSmpLOyU6A1xa26UorxO9yIf2y+jzgoWELNUy/kyJfgbiM9JTaFtEKH?= =?us-ascii?Q?IBqWpoysqrE9FB4jyk139W+C5axegMzUgHHdhphGuM3X4lbmRVPkHpYbjx2R?= =?us-ascii?Q?jmTnkcOQuGRvfqUwflEbzZhU1lSaNCYOqiDfCogusStf1sZrnurGusTK5/LP?= =?us-ascii?Q?Yx3WRwyJghuY1xVoLb6hw4atMDheuvpCZ55c9lT97Rq9lfx47dUvykqZcq1p?= =?us-ascii?Q?WyY7RXcSYJdk4vZxB+M6BbtvuQnxnTTqCIcCGpwKTxenx4rTV4pXpcgvIZjL?= =?us-ascii?Q?oHCeeI6WlmRBqiLbAgcXdEdwrm/0IrGVkEqAF0A0KVu3ifFDiG1360B9uhWp?= =?us-ascii?Q?rXN/r3p0PCvQtBtTejLvoEySr4ObOvaEt+m/dy4UORCTesqE0gxOyzGNiaYN?= =?us-ascii?Q?yLVKO711Sa5/yV2YjbVJXFU6D1WBP8iqjE8lJ4zLZFQGX7aPR+vsNKVUFWeo?= =?us-ascii?Q?xrH4W5NDC1PuWz8ywVy051tq+58mzPdEcUA9B08drWzyfAn5yJKmq7tKT7t5?= =?us-ascii?Q?hd+tGYnR/Yu1DtsClVfC2L64GlyzkoUfMgJDIj9d5HWmPqcc5eFXLq0U9YPk?= =?us-ascii?Q?00Qqwh2ekk0U/aRT5/o/1v9+PRSIEFOp3FJSWgmMNWmYR/CNTXUQQRtGmc6P?= =?us-ascii?Q?1c/IKyq5fMqkLiA2kxiixgQV4CBpZTGtVLH6zSoEq2eRBI5NjQpKZzmkp+m9?= =?us-ascii?Q?Dj1fQvOIJPjNVcukYW/PhCuLz5pfBsE6Rq0Wl5w28vuD62IDN5Lk/nmIkIH8?= =?us-ascii?Q?mpkCKwbh7/YtTsZtIDB4e2yJy2TttXwNEUqLjJqHoHRiTkTOgJIwb8oW2LxF?= =?us-ascii?Q?R9FriL/IWMKAZDG8j4DkvVlTVDnfDUEjsx5FgGpUkubr3kPh0CkXauWZlVo2?= =?us-ascii?Q?QAVBFv09K+OHGjhL6QM3hiyd7tn76Mf7oV2no2TTUQeOudOJLMJ6jBoacbtN?= =?us-ascii?Q?udl8fgyZqpR10Zx0ofjhitFEwAacKcwrmbP7bMgvIy5/bw86G/Bh9LN3oy9l?= =?us-ascii?Q?nx14nNG658QiGxSbzOFcWH8vsGVajBkmnPNrqZRTXbuPtehY/krIYLumwbCW?= =?us-ascii?Q?Do9PJSqnDiIXNy+1ybRGHmGVgFSlly7mQuSYfxMq18ZEPULVUooAUtHjp7Vx?= =?us-ascii?Q?NG4+clli8qjdFmkXmTGfStkmEWo8gDF7cl2LfdZFjM71rQjODi4qdtz8htVZ?= =?us-ascii?Q?wwExulbIHr3RfTssT3GSl3PZ0NtXeWd1cORjz11VrTa36ykKUEknM3nEGPRQ?= =?us-ascii?Q?VJd/4ymcEO28/WhKIGj4fsU0xcXJH18A53x7DPEbUiLOxEgFaxSMFBLBMAUD?= =?us-ascii?Q?6KnEyLXL97uGLOl1Gw5tTSrynd/mcJEvH4jHzkkAUHUkFWjO+nUlPJn3oJPK?= =?us-ascii?Q?Axclv2JxHgFW94gwuHrDufPB8YYP7YF37im0lbMD6yUJu1vAvnuV38eV6Q?= =?us-ascii?Q?=3D=3D?= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(36860700013)(1800799024)(7416014)(376014)(82310400026)(921020);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Jun 2025 17:24:50.9794 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: e7b1a1ee-5318-4467-0ddb-08dda2c38c28 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BL6PEPF00020E61.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ5PPF0C60B25BF Content-Type: text/plain; charset="utf-8" The CXL RAS handlers do not currently log if the RAS registers are unmapped. This is needed in order to help debug CXL error handling. Update the CXL driver to log a warning message if the RAS register block is unmapped during RAS error handling. Signed-off-by: Terry Bowman Reviewed-by: Jonathan Cameron Reviewed-by: Dave Jiang Reviewed-by: Kuppuswamy Sathyanarayanan --- drivers/cxl/core/pci.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/drivers/cxl/core/pci.c b/drivers/cxl/core/pci.c index 78735da7e63d..186a5a20b951 100644 --- a/drivers/cxl/core/pci.c +++ b/drivers/cxl/core/pci.c @@ -670,8 +670,10 @@ static void __cxl_handle_cor_ras(struct device *dev, void __iomem *addr; u32 status; =20 - if (!ras_base) + if (!ras_base) { + dev_warn_once(dev, "CXL RAS register block is not mapped"); return; + } =20 addr =3D ras_base + CXL_RAS_CORRECTABLE_STATUS_OFFSET; status =3D readl(addr); @@ -714,8 +716,10 @@ static bool __cxl_handle_ras(struct device *dev, void = __iomem *ras_base) u32 status; u32 fe; =20 - if (!ras_base) + if (!ras_base) { + dev_warn_once(dev, "CXL RAS register block is not mapped"); return false; + } =20 addr =3D ras_base + CXL_RAS_UNCORRECTABLE_STATUS_OFFSET; status =3D readl(addr); --=20 2.34.1 From nobody Thu Dec 18 05:05:03 2025 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2079.outbound.protection.outlook.com [40.107.220.79]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3A73F1F4C90; Tue, 3 Jun 2025 17:24:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.220.79 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1748971498; cv=fail; b=dniCi0vKpyEGwWO9Zr2gx5GaAiTxzAQMsffgpfgur9g3RKrFlPvA54a/vHRkouvxqxnBoHE9RXoouXYKG6kKN88I1EBO8l5xMVQZv8jCi4RsP4DqlW7JRd3ftdotf2OkxLaADtIEGak4skYsQzvtvf/JiXm7+0lJlbtxUvo3LgE= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1748971498; c=relaxed/simple; bh=ayIunaeR8cJ+q39qam/q/ZX7ONSyQ2dPdtyBcAmXUHk=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=pcPTJE1jL7g1gyrVc+pC47JQWh9UC9ilArq3vF38Kn2OxsWWmp+HP4zaCVZl66L+A1S9FRMBlgjHhk6+pvNKU77P2C+C4B6yfWGjOhLEZh/aRUGoYsD9rcNkogCsR+PulS9J+HxFYJBdpie7FiACoqYt8nAv+t9vWrsO6UOT99E= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=iW7m24ty; arc=fail smtp.client-ip=40.107.220.79 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="iW7m24ty" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=rc2ZRELvE1k+zqBgydfEjXz4wozMciGV9N8ctNEBEcScD3+GR7sVRCdDYQ76aMR5FJZvnYhvYlDMN/zy626VCU3iRwuhh2yzo962FIbS+agl0wRysMLnWwNk8kp/alMZhN3YtrdSmx4Lu3ZLDMFzAckFpUglNp1kP+H5wBzOxxZKf9BVnqOKQvGXb3nLWg5397/2hXw17RSfF4OO6RRPJ2h/MgWUbPX9EBrhWwq+XoHSQJm9hm864wlyiAEy7soPhSmiYX7PihwfdGBKOdj7Z+xq4wPsZwTQxkqrVfLPabmxjNwNa/LxURzAV9RjIlyPRkwRFRTroWRNtxGzPoM1ew== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=FLqsJ2oyeS47XC63eje+FcyHAWxPKEQRahaEIuB4aio=; b=ccvpalApn+R/W01Vo1B++EaPM1bKoy6rg8VmYn3qBsJHJn8k3WkJvjUEoP1p2xgzOo0VVOFkyC2AZIdh2sk7Prlg4w+Q4z5XqQPHMoat5KBGN//S2S50ESLZU69AaJBPh0tF3BXsM3gLbNtRU1VZ6SBCSxc6wvENYNBNluXdMz78r4HP5nwI4pOPzkOeaLd/m5UvZS3eNKFhvN+A/yav449A7opjdP3D+LwivFSAYTchHsVDm9VtgtcHOIRtvTJQMTsYJS5BXEiZN0/PrMgjNOBJ4KwvNFiir2GfXzh9sYD++9r78TNYiQSmkbeCZAihBrs372Y6Fqhs7yBTgrnnWQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=stgolabs.net smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=FLqsJ2oyeS47XC63eje+FcyHAWxPKEQRahaEIuB4aio=; b=iW7m24tyB/aCfqTQj58hMziisDzWvqDBS5JVEjzMK6PwdGTpUFJ/FZq5vK7MhRdhlgMCnu2auC6/gG609iE6eXFL+BCjBX9sXJpI/XArPwN0yGkGAqfKDpue5nGqr8lhNN4ELGXhzfmOuZjzwSG/cVzUD7HnMgmtpHtMUyngZYw= Received: from BL1PR13CA0196.namprd13.prod.outlook.com (2603:10b6:208:2be::21) by CY5PR12MB6155.namprd12.prod.outlook.com (2603:10b6:930:25::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8769.36; Tue, 3 Jun 2025 17:24:53 +0000 Received: from BL6PEPF00020E61.namprd04.prod.outlook.com (2603:10b6:208:2be:cafe::6) by BL1PR13CA0196.outlook.office365.com (2603:10b6:208:2be::21) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8769.18 via Frontend Transport; Tue, 3 Jun 2025 17:24:53 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by BL6PEPF00020E61.mail.protection.outlook.com (10.167.249.22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.8792.29 via Frontend Transport; Tue, 3 Jun 2025 17:24:53 +0000 Received: from ethanolx7ea3host.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Tue, 3 Jun 2025 12:24:37 -0500 From: Terry Bowman To: , , , , , , , , , , , , , , , , , , , , , , , , , , Subject: [PATCH v9 10/16] cxl/pci: Unify CXL trace logging for CXL Endpoints and CXL Ports Date: Tue, 3 Jun 2025 12:22:33 -0500 Message-ID: <20250603172239.159260-11-terry.bowman@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250603172239.159260-1-terry.bowman@amd.com> References: <20250603172239.159260-1-terry.bowman@amd.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL6PEPF00020E61:EE_|CY5PR12MB6155:EE_ X-MS-Office365-Filtering-Correlation-Id: 34dc4ff4-9088-47e0-2fb6-08dda2c38dc3 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|82310400026|36860700013|7416014|376014|921020; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?S38c3NZX8FwHLk+AsxIXnkXF1NNTX09ulk7gFPQCwes7YhLS5+lfULdoE0cT?= =?us-ascii?Q?60EnTeg1XKOEkEbmrrvq0ZX2c/KSJqyp2v/ru6K/7wAjtLPmCdxAVQDXAzG7?= =?us-ascii?Q?nxYzKS5TnGnRSOYXMG7eecDH3X2+NKzRDfmacG3AteenSXSj/H/qRcvwwEHd?= =?us-ascii?Q?JoXUBM/+nW8FXkpuZsgj9MIt9fxxvdiI4Uf7CKSmNETS6moSUZRBvQjgLzJj?= =?us-ascii?Q?sgsYrfm0rduu3s6h8ZruAM6Mlkp7xX3xBzek6fd+fjMOMOVG/76qchtRZUgZ?= =?us-ascii?Q?z5O4xmhcygB30urEN6VLm+kVWrbIwmnqwHOA1T11QZ+UXt7l4ryt1J9SuYFy?= =?us-ascii?Q?f1ZyvbV+NVR7d//ohTm47+Sp8Y8EGI1TL32JeQskeQizGzSt5+eXNYU9L/MF?= =?us-ascii?Q?BNka6U0vGo9N5nqVodbttTGd8xWlaX895BlLwU5jcLpYzFleL7IxnTeZu0Vd?= =?us-ascii?Q?ZQy5TcF7jP8udPWjUIuCnlTuxbNwb8IgJrVj7Z7Xcv7nwjtuuo1jjre6c+Yr?= =?us-ascii?Q?wdOFglhJi3Tl7KT8qYyLY0GS0o6paiohyupvLjgx7zid8S1Ys1CUkaHhQTqN?= =?us-ascii?Q?nXFEPHt4PDfsq/UO6H4AvbX/UiA8Ynux1glOl1YEcoO9pnWBj48ARs5YXFnr?= =?us-ascii?Q?LbYMZDmYm4eHIBPynlxrxmGTtoeQSRq7eu/EOdZRuHU+u0S3rcy4nZCBFM6z?= =?us-ascii?Q?ekmA9tn3mNFYhT5cM3PS/QfKBOrNUY7DmvRCiam0+LRoTnRHbzNCeXuZuy6V?= =?us-ascii?Q?j1UTb6YILUK2Nh75FPRVUm/m093D/ZR3heWRtudYxPFibqDChgHLzn2D9bDA?= =?us-ascii?Q?g7uAdMMCMUKCitifiBK+xHlv/kznH2xIqTlG6jCJdDve6dXPJVsduM/tpcTT?= =?us-ascii?Q?1WGuuCdxSqyp31jQa51KdM1ztYiccwWORJ8DR9BtSKZiNyz0Wh0BsJ90Rpsk?= =?us-ascii?Q?ZMqGWbJYXEDA0kUZD2aXGS/gjgEZw8mqrkJRT+QpoO4EgTcGm0bkmoefV/uX?= =?us-ascii?Q?okT3N6xseq5Vo6X7bUCyrjMoCZ8D9k1i41Xmzfgb2OxheGhFOphAktMpjYja?= =?us-ascii?Q?4oflwmgs02FaB3mXiuDMZDBj2ttS++TGdJfNNBrRKf5k3c63KWhbrUBZh0ri?= =?us-ascii?Q?ZyQAnXcGArEWmmIVgfzCRIhVUbysNqPKYXegakJw4SqXn2cjZ93Wh7tuj+7j?= =?us-ascii?Q?/poiZjhN6eVVfYXSw7Uw9cngr3T5CUQkPYV0ZaZZEPW9xxAmo4svUFmkM6At?= =?us-ascii?Q?CwtBNA7keEBBERhHvrjvOW6ABLqjOhDzEh4NPJG9UBClMyw8+ZDAeXgjx2Xa?= =?us-ascii?Q?MLTKu917ZgWcEYM5+4Nk93xQPNAJ3EColeiLDTMrj52+WO+n/g8/G39D9VTd?= =?us-ascii?Q?y2oswLE1BVUXGSapgZpBiiyqTIlDSZdw7eHluZcavh7j1n+a9EkufgWElYge?= =?us-ascii?Q?6hX+Ke0P5GY3alrBkTrcUvx1t98Te3jifnNy4Qi33pAnK3fLXZ+0vzLrlf7r?= =?us-ascii?Q?KFmywaJcccFm6ar6uY3K4JxVDr4UmuX/JnnpsAy7lLhTsAU9U16aFZEY7A?= =?us-ascii?Q?=3D=3D?= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(1800799024)(82310400026)(36860700013)(7416014)(376014)(921020);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Jun 2025 17:24:53.6728 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 34dc4ff4-9088-47e0-2fb6-08dda2c38dc3 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BL6PEPF00020E61.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY5PR12MB6155 Content-Type: text/plain; charset="utf-8" CXL currently has separate trace routines for CXL Port errors and CXL Endpoint errors. This is inconvenient for the user because they must enable 2 sets of trace routines. Make updates to the trace logging such that a single trace routine logs both CXL Endpoint and CXL Port protocol errors. Rename the 'host' field from the CXL Endpoint trace to 'parent' in the unified trace routines. 'host' does not correctly apply to CXL Port devices. Parent is more general and applies to CXL Port devices and CXL Endpoints. Add serial number parameter to the trace logging. This is used for EPs and 0 is provided for CXL port devices without a serial number. Below is output of correctable and uncorrectable protocol error logging. CXL Root Port and CXL Endpoint examples are included below. Root Port: cxl_aer_correctable_error: device=3D0000:0c:00.0 parent=3Dpci0000:0c serial= : 0 status=3D'CRC Threshold Hit' cxl_aer_uncorrectable_error: device=3D0000:0c:00.0 parent=3Dpci0000:0c seri= al: 0 status: 'Cache Byte Enable Parity Error' first_error: 'Cache Byte Ena= ble Parity Error' Endpoint: cxl_aer_correctable_error: device=3Dmem3 parent=3D0000:0f:00.0 serial=3D0 s= tatus=3D'CRC Threshold Hit' cxl_aer_uncorrectable_error: device=3Dmem3 parent=3D0000:0f:00.0 serial: 0 = status: 'Cache Byte Enable Parity Error' first_error: 'Cache Byte Enable Pa= rity Error' Signed-off-by: Terry Bowman Reviewed-by: Kuppuswamy Sathyanarayanan --- drivers/cxl/core/pci.c | 18 +++++---- drivers/cxl/core/ras.c | 14 ++++--- drivers/cxl/core/trace.h | 84 +++++++++------------------------------- 3 files changed, 37 insertions(+), 79 deletions(-) diff --git a/drivers/cxl/core/pci.c b/drivers/cxl/core/pci.c index 186a5a20b951..0f4c07fd64a5 100644 --- a/drivers/cxl/core/pci.c +++ b/drivers/cxl/core/pci.c @@ -664,7 +664,7 @@ void read_cdat_data(struct cxl_port *port) } EXPORT_SYMBOL_NS_GPL(read_cdat_data, "CXL"); =20 -static void __cxl_handle_cor_ras(struct device *dev, +static void __cxl_handle_cor_ras(struct device *dev, u64 serial, void __iomem *ras_base) { void __iomem *addr; @@ -679,13 +679,13 @@ static void __cxl_handle_cor_ras(struct device *dev, status =3D readl(addr); if (status & CXL_RAS_CORRECTABLE_STATUS_MASK) { writel(status & CXL_RAS_CORRECTABLE_STATUS_MASK, addr); - trace_cxl_aer_correctable_error(to_cxl_memdev(dev), status); + trace_cxl_aer_correctable_error(dev, serial, status); } } =20 static void cxl_handle_endpoint_cor_ras(struct cxl_dev_state *cxlds) { - return __cxl_handle_cor_ras(&cxlds->cxlmd->dev, cxlds->regs.ras); + return __cxl_handle_cor_ras(&cxlds->cxlmd->dev, cxlds->serial, cxlds->reg= s.ras); } =20 /* CXL spec rev3.0 8.2.4.16.1 */ @@ -709,7 +709,8 @@ static void header_log_copy(void __iomem *ras_base, u32= *log) * Log the state of the RAS status registers and prepare them to log the * next error status. Return 1 if reset needed. */ -static bool __cxl_handle_ras(struct device *dev, void __iomem *ras_base) +static bool __cxl_handle_ras(struct device *dev, u64 serial, + void __iomem *ras_base) { u32 hl[CXL_HEADERLOG_SIZE_U32]; void __iomem *addr; @@ -738,7 +739,7 @@ static bool __cxl_handle_ras(struct device *dev, void _= _iomem *ras_base) } =20 header_log_copy(ras_base, hl); - trace_cxl_aer_uncorrectable_error(to_cxl_memdev(dev), status, fe, hl); + trace_cxl_aer_uncorrectable_error(dev, serial, status, fe, hl); writel(status & CXL_RAS_UNCORRECTABLE_STATUS_MASK, addr); =20 return true; @@ -746,7 +747,8 @@ static bool __cxl_handle_ras(struct device *dev, void _= _iomem *ras_base) =20 static bool cxl_handle_endpoint_ras(struct cxl_dev_state *cxlds) { - return __cxl_handle_ras(&cxlds->cxlmd->dev, cxlds->regs.ras); + + return __cxl_handle_ras(&cxlds->cxlmd->dev, cxlds->serial, cxlds->regs.ra= s); } =20 #ifdef CONFIG_PCIEAER_CXL @@ -754,13 +756,13 @@ static bool cxl_handle_endpoint_ras(struct cxl_dev_st= ate *cxlds) static void cxl_handle_rdport_cor_ras(struct cxl_dev_state *cxlds, struct cxl_dport *dport) { - return __cxl_handle_cor_ras(&cxlds->cxlmd->dev, dport->regs.ras); + return __cxl_handle_cor_ras(&cxlds->cxlmd->dev, cxlds->serial, dport->reg= s.ras); } =20 static bool cxl_handle_rdport_ras(struct cxl_dev_state *cxlds, struct cxl_dport *dport) { - return __cxl_handle_ras(&cxlds->cxlmd->dev, dport->regs.ras); + return __cxl_handle_ras(&cxlds->cxlmd->dev, cxlds->serial, dport->regs.ra= s); } =20 /* diff --git a/drivers/cxl/core/ras.c b/drivers/cxl/core/ras.c index 715f7221ea3a..0ef8c2068c0c 100644 --- a/drivers/cxl/core/ras.c +++ b/drivers/cxl/core/ras.c @@ -13,7 +13,7 @@ static void cxl_cper_trace_corr_port_prot_err(struct pci_= dev *pdev, { u32 status =3D ras_cap.cor_status & ~ras_cap.cor_mask; =20 - trace_cxl_port_aer_correctable_error(&pdev->dev, status); + trace_cxl_aer_correctable_error(&pdev->dev, 0, status); } =20 static void cxl_cper_trace_uncorr_port_prot_err(struct pci_dev *pdev, @@ -28,8 +28,8 @@ static void cxl_cper_trace_uncorr_port_prot_err(struct pc= i_dev *pdev, else fe =3D status; =20 - trace_cxl_port_aer_uncorrectable_error(&pdev->dev, status, fe, - ras_cap.header_log); + trace_cxl_aer_uncorrectable_error(&pdev->dev, 0, status, fe, + ras_cap.header_log); } =20 static void cxl_cper_trace_corr_prot_err(struct pci_dev *pdev, @@ -42,7 +42,8 @@ static void cxl_cper_trace_corr_prot_err(struct pci_dev *= pdev, if (!cxlds) return; =20 - trace_cxl_aer_correctable_error(cxlds->cxlmd, status); + trace_cxl_aer_correctable_error(&cxlds->cxlmd->dev, cxlds->serial, + status); } =20 static void cxl_cper_trace_uncorr_prot_err(struct pci_dev *pdev, @@ -62,8 +63,9 @@ static void cxl_cper_trace_uncorr_prot_err(struct pci_dev= *pdev, else fe =3D status; =20 - trace_cxl_aer_uncorrectable_error(cxlds->cxlmd, status, fe, - ras_cap.header_log); + trace_cxl_aer_uncorrectable_error(&cxlds->cxlmd->dev, + cxlds->serial, status, + fe, ras_cap.header_log); } =20 static void cxl_cper_handle_prot_err(struct cxl_cper_prot_err_work_data *d= ata) diff --git a/drivers/cxl/core/trace.h b/drivers/cxl/core/trace.h index 25ebfbc1616c..8c91b0f3d165 100644 --- a/drivers/cxl/core/trace.h +++ b/drivers/cxl/core/trace.h @@ -48,49 +48,22 @@ { CXL_RAS_UC_IDE_RX_ERR, "IDE Rx Error" } \ ) =20 -TRACE_EVENT(cxl_port_aer_uncorrectable_error, - TP_PROTO(struct device *dev, u32 status, u32 fe, u32 *hl), - TP_ARGS(dev, status, fe, hl), - TP_STRUCT__entry( - __string(device, dev_name(dev)) - __string(host, dev_name(dev->parent)) - __field(u32, status) - __field(u32, first_error) - __array(u32, header_log, CXL_HEADERLOG_SIZE_U32) - ), - TP_fast_assign( - __assign_str(device); - __assign_str(host); - __entry->status =3D status; - __entry->first_error =3D fe; - /* - * Embed the 512B headerlog data for user app retrieval and - * parsing, but no need to print this in the trace buffer. - */ - memcpy(__entry->header_log, hl, CXL_HEADERLOG_SIZE); - ), - TP_printk("device=3D%s host=3D%s status: '%s' first_error: '%s'", - __get_str(device), __get_str(host), - show_uc_errs(__entry->status), - show_uc_errs(__entry->first_error) - ) -); - TRACE_EVENT(cxl_aer_uncorrectable_error, - TP_PROTO(const struct cxl_memdev *cxlmd, u32 status, u32 fe, u32 *hl), - TP_ARGS(cxlmd, status, fe, hl), + TP_PROTO(struct device *dev, u64 serial, u32 status, u32 fe, + u32 *hl), + TP_ARGS(dev, serial, status, fe, hl), TP_STRUCT__entry( - __string(memdev, dev_name(&cxlmd->dev)) - __string(host, dev_name(cxlmd->dev.parent)) + __string(name, dev_name(dev)) + __string(parent, dev_name(dev->parent)) __field(u64, serial) __field(u32, status) __field(u32, first_error) __array(u32, header_log, CXL_HEADERLOG_SIZE_U32) ), TP_fast_assign( - __assign_str(memdev); - __assign_str(host); - __entry->serial =3D cxlmd->cxlds->serial; + __assign_str(name); + __assign_str(parent); + __entry->serial =3D serial; __entry->status =3D status; __entry->first_error =3D fe; /* @@ -99,8 +72,8 @@ TRACE_EVENT(cxl_aer_uncorrectable_error, */ memcpy(__entry->header_log, hl, CXL_HEADERLOG_SIZE); ), - TP_printk("memdev=3D%s host=3D%s serial=3D%lld: status: '%s' first_error:= '%s'", - __get_str(memdev), __get_str(host), __entry->serial, + TP_printk("device=3D%s parent=3D%s serial=3D%lld status=3D'%s' first_erro= r=3D'%s'", + __get_str(name), __get_str(parent), __entry->serial, show_uc_errs(__entry->status), show_uc_errs(__entry->first_error) ) @@ -124,42 +97,23 @@ TRACE_EVENT(cxl_aer_uncorrectable_error, { CXL_RAS_CE_PHYS_LAYER_ERR, "Received Error From Physical Layer" } \ ) =20 -TRACE_EVENT(cxl_port_aer_correctable_error, - TP_PROTO(struct device *dev, u32 status), - TP_ARGS(dev, status), - TP_STRUCT__entry( - __string(device, dev_name(dev)) - __string(host, dev_name(dev->parent)) - __field(u32, status) - ), - TP_fast_assign( - __assign_str(device); - __assign_str(host); - __entry->status =3D status; - ), - TP_printk("device=3D%s host=3D%s status=3D'%s'", - __get_str(device), __get_str(host), - show_ce_errs(__entry->status) - ) -); - TRACE_EVENT(cxl_aer_correctable_error, - TP_PROTO(const struct cxl_memdev *cxlmd, u32 status), - TP_ARGS(cxlmd, status), + TP_PROTO(struct device *dev, u64 serial, u32 status), + TP_ARGS(dev, serial, status), TP_STRUCT__entry( - __string(memdev, dev_name(&cxlmd->dev)) - __string(host, dev_name(cxlmd->dev.parent)) + __string(name, dev_name(dev)) + __string(parent, dev_name(dev->parent)) __field(u64, serial) __field(u32, status) ), TP_fast_assign( - __assign_str(memdev); - __assign_str(host); - __entry->serial =3D cxlmd->cxlds->serial; + __assign_str(name); + __assign_str(parent); + __entry->serial =3D serial; __entry->status =3D status; ), - TP_printk("memdev=3D%s host=3D%s serial=3D%lld: status: '%s'", - __get_str(memdev), __get_str(host), __entry->serial, + TP_printk("device=3D%s parent=3D%s serial=3D%lld status=3D'%s'", + __get_str(name), __get_str(parent), __entry->serial, show_ce_errs(__entry->status) ) ); --=20 2.34.1 From nobody Thu Dec 18 05:05:03 2025 Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2056.outbound.protection.outlook.com [40.107.243.56]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3EDD924166D; Tue, 3 Jun 2025 17:24:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.243.56 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1748971500; cv=fail; b=YP0P55UM4ocLJymAVpTNoFkO6ArhT+wYESWePDk3FU+ERnplXZmrSPfNeQY0tDmOc/fteWZDwEgAkJRlX7w7WhbOtW/Ff3ZovCkyIjoAkhxdL8oQ/gs8PihSdP5eZ/DDeP9o7W+d7E3ltrC1CtL4bM5zf6G7jwUWtRoWu/G7Iek= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1748971500; c=relaxed/simple; bh=wKYT2RX2WS2rMYQ8Nxy6U8dUrXd+1T0ePtqEyUHEK6s=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=jmhcTqSq6VFay8uiBegUyClglotkdUB3G4bO9NOluP9qPHRy5asimtTClhQPlEj99FFXqutfVtHgRfdSAjez0oz/RNcdYwJ6DDHmfnQ6p/fvgFN6EjU0bIIGosWkZNnGmDHNvFXTFrlAzdffePCuz7bgGI6Jnyo0USk8JVyzyrE= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=fux0+Ewv; arc=fail smtp.client-ip=40.107.243.56 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="fux0+Ewv" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=GzlqzGsG8kiao/+kLeXHZ92WhW6NhaO1rgybOsSMNIRMyEUb3Z9ButrGqKeMy+I5+zIBzOFWcfUKPLyYWZiDCs0DjgXPOLQGsdKp67qppBZUUaJEDSTz424P5g5WeHj5Gi1rUBaxtsD9qsfkU/E63AUGjesrSeMxB8P1vLb2MmcnQJno3eQoktwne46JBaXxonYrkD/V7s4KfXQqxckAN6advwOYnb5deoTSWrBC9nYUgKR8HRnllCyKGe3wEL9uuyB53a6M5MHMMeNesCb6Q4zHAQb5TpIs/Hd9CzdPG9ybxZubTJQi+gVZWQfmtKI3TaY6a+zirYFNU9UXuoMPfg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Scya8gfCWFz1+mMbTZwepfFxg5bxLhLwPryvZ5GuzOY=; b=uOmig5UbWc8GiP6yYKydF2+7GnTJJzANGsRzcB6YcLmaeidSwvqKhHN+BNzNu1pg/UJbY4QcETd7kiSFhw+0MVttiDgc1DdQeSBJWvSOvsNZbCuWbOJfJx1xMcURvhaTB0+3o/YfmJPs8pdQT3NVHOpuXwpnjtSub9xGyHlAIwfyrw9v3GRTfAreBxphdspdeU6w03Ew1I8SQ7qGk/XvkEdGvKQn5MHrGPjOaPVWLt2qSzWvPHvi9cXE2Eug5csyvaCaGgC0fPWmSoGq0CXzNniO0DGVxBEMyt7CJ1Qwsq2RjMRVkwVgT9JPZDCKC8thEbk1mPdhGXLtJWxK4LrWqA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=stgolabs.net smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Scya8gfCWFz1+mMbTZwepfFxg5bxLhLwPryvZ5GuzOY=; b=fux0+EwvE9CPJF5Fgz22lPDH6MIClIamhTdIpy4xCFFMMB+tjMbtgq8gvUH8pL/GjF7a9tVJE2tvbkKsYrpMjJvCfbEoc/Lyd2DY2MUZowFAbPFgAR4hxXkQ4CIjGbZYYMlYVxTg3zdTDiGoMvP/B525zVGkOJmklGrQPh/idFE= Received: from BL1PR13CA0204.namprd13.prod.outlook.com (2603:10b6:208:2be::29) by MN0PR12MB5882.namprd12.prod.outlook.com (2603:10b6:208:37a::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8769.24; Tue, 3 Jun 2025 17:24:55 +0000 Received: from BL6PEPF00020E61.namprd04.prod.outlook.com (2603:10b6:208:2be:cafe::ae) by BL1PR13CA0204.outlook.office365.com (2603:10b6:208:2be::29) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8769.18 via Frontend Transport; Tue, 3 Jun 2025 17:24:54 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by BL6PEPF00020E61.mail.protection.outlook.com (10.167.249.22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.8792.29 via Frontend Transport; Tue, 3 Jun 2025 17:24:54 +0000 Received: from ethanolx7ea3host.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Tue, 3 Jun 2025 12:24:48 -0500 From: Terry Bowman To: , , , , , , , , , , , , , , , , , , , , , , , , , , Subject: [PATCH v9 11/16] cxl/pci: Update __cxl_handle_cor_ras() to return early if no RAS errors Date: Tue, 3 Jun 2025 12:22:34 -0500 Message-ID: <20250603172239.159260-12-terry.bowman@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250603172239.159260-1-terry.bowman@amd.com> References: <20250603172239.159260-1-terry.bowman@amd.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL6PEPF00020E61:EE_|MN0PR12MB5882:EE_ X-MS-Office365-Filtering-Correlation-Id: a84b1f4c-de6c-4c8f-a65b-08dda2c38e09 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|1800799024|36860700013|82310400026|7416014|921020; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?LTaujmTU1/JfAvY7Lsr+xyr5m1/xL7uQbRKnpuhmLOJTyLYgbkGHeudG4hgp?= =?us-ascii?Q?LWEUOU7n7yrE0kJQEHeAEIfoQ2O8PeqZzsDMRf1v5ps8BoS0ZGl9p4DC6CSh?= =?us-ascii?Q?hIBu8uxUH5/ocFXwU2jlCx/hBsjkyLqL6Ubl3Z66O+TrQmvT5n0vdTCtBcVZ?= =?us-ascii?Q?FfQfh+GQu/BPSphp+UQlcWmOcAwzWEVm8DfiaF58jklWQX5KNDr98YrpuDmD?= =?us-ascii?Q?m+NMRsZvCnbYPyCQlGdoiLYrrEkyeDiYKFPCy3g8QnrJRuINs/EPY/UXOBsp?= =?us-ascii?Q?kdr2ZWTGu+fMQBTFKXRyYgn1R9q8ejWadEEgvMZ5ZmctiUmGheDsQPe5Qr5Y?= =?us-ascii?Q?TB5NrZQb071mDc4Y+Sx8mj3Frr9d9+FFzKSTsgFpaT88WG2qfeQ1CPrN1dVW?= =?us-ascii?Q?xdR9pgX7idgULLFrbo4WLXFYPc3gCyDrKgGp+QFmxvClEoL3e3YJ06+c7K5W?= =?us-ascii?Q?mfs+sTelaGHS548xrsX8qMhFgchiWSZFXI+OKdcRlqU4uBiWZztYFaL/XW4k?= =?us-ascii?Q?gMXn7YNxMwWWOuQHNYF3Rv+XWE5YKhuLh+ev1JkQ7dLZ2wxV2SQtRpeuFHg3?= =?us-ascii?Q?HubSrzJg11KxzikedRVnuRNwpTEs5fcNrbYeja+pqbtZzEA7Mg3nAK/lGAKY?= =?us-ascii?Q?/jeiFaMJ0i8KipwKhW51W8MEtMGvgN6Zx1BAFqydKsk9ZsXVZETOYCS22WG4?= =?us-ascii?Q?kUnRRxrriVziw/75WnrThDHB/HjAH+pyx3KZVAnIn31q3eQIiTV6ElG75FrS?= =?us-ascii?Q?Nn7ACmHPW78aoHSCRxyJT10Hy0910sWjDYdA2BclFlJlYAwdn48h8dWivrcb?= =?us-ascii?Q?XyTiF4yjo52JTOgfufnC12mlPqVCxOVlHZ/3E08cs0243xpxLs4Pvy5sXPt1?= =?us-ascii?Q?B4Bnc6VRI9gKowaQ7MY36HtOUfqxImXxGyaPZx445CDHnZMjZN8F4b9RGoDB?= =?us-ascii?Q?EPljLN1+/YDRkgn0Il0sBtd3B/PBlBQU7rZLejSgCxGAs3dMNVP7+L3QDAuj?= =?us-ascii?Q?9D7yHF6r+iNHo1C2cZ8Swtkvsr7cTSqszJHzEA4Ats+1t9ErzkR3dHQsLQy0?= =?us-ascii?Q?dj5U5VbJ7XBnqndzUAWW/7Vy5V1wrxfxKcX2eZ/bm/6m3UuTpWSJ3txzclxj?= =?us-ascii?Q?0TRbeuCTiTByvxwJKEfrou2S+NUcti++jwjt/SHda52johO7QkFHx3eZMnPl?= =?us-ascii?Q?XWlIJWlffPvCAmzG0HgXWi8XjRyPWZwjXFkrbbAycsPHoDjNTg1YiXEoTdFQ?= =?us-ascii?Q?AOV1aPvAld/DrHI9v1JVj67GCE4GjveUIr27yoPmtYvvwB1KRJq2iMB+em3D?= =?us-ascii?Q?2MT93+j2h87Va+buu9eIJS3d3t5FCK+AEccKMu0e6LSJqvnVU0miJ6MF04E2?= =?us-ascii?Q?ZL02G8IAvkAuR2fZu08TuhxTStwRz3kYNiLEtxJ5MlkaMzZhCS7QUA2vEXYr?= =?us-ascii?Q?pfCVtyPxuUmJGs8l1fCLk62/cvHqKxTq6411KqapXAGme2FEOcTtMByzo6uQ?= =?us-ascii?Q?pTlmSMIPhFw/u1EIryHmAdfxQoVoaiZbwLbKJoOFxo8EJH2DQgInOcQh2w?= =?us-ascii?Q?=3D=3D?= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(376014)(1800799024)(36860700013)(82310400026)(7416014)(921020);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Jun 2025 17:24:54.1356 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: a84b1f4c-de6c-4c8f-a65b-08dda2c38e09 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BL6PEPF00020E61.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN0PR12MB5882 Content-Type: text/plain; charset="utf-8" __cxl_handle_cor_ras() is missing logic to leave the function early in the case there is no RAS error. Update __cxl_handle_cor_ras() to exit early in the case there is no RAS errors detected after applying the mask. Signed-off-by: Terry Bowman Reviewed-by: Dave Jiang Reviewed-by: Kuppuswamy Sathyanarayanan --- drivers/cxl/core/pci.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/drivers/cxl/core/pci.c b/drivers/cxl/core/pci.c index 0f4c07fd64a5..f5f87c2c3fd5 100644 --- a/drivers/cxl/core/pci.c +++ b/drivers/cxl/core/pci.c @@ -677,10 +677,11 @@ static void __cxl_handle_cor_ras(struct device *dev, = u64 serial, =20 addr =3D ras_base + CXL_RAS_CORRECTABLE_STATUS_OFFSET; status =3D readl(addr); - if (status & CXL_RAS_CORRECTABLE_STATUS_MASK) { - writel(status & CXL_RAS_CORRECTABLE_STATUS_MASK, addr); - trace_cxl_aer_correctable_error(dev, serial, status); - } + if (!(status & CXL_RAS_CORRECTABLE_STATUS_MASK)) + return; + writel(status & CXL_RAS_CORRECTABLE_STATUS_MASK, addr); + + trace_cxl_aer_correctable_error(dev, serial, status); } =20 static void cxl_handle_endpoint_cor_ras(struct cxl_dev_state *cxlds) --=20 2.34.1 From nobody Thu Dec 18 05:05:03 2025 Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on2045.outbound.protection.outlook.com [40.107.223.45]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E09351DAC92; Tue, 3 Jun 2025 17:25:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.223.45 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1748971514; cv=fail; b=kxjpewqD856ZsK3bxcSYm2NsG8L/1mMzQolfm8Yxo88gL+koAdGmc9MJbK7akyvqAuWdKXBvGurZ+Jzx4hdanvhoOUvbdhnj/mS+T02nk4G61MGHr4jcSc26KgUgWsXLfO5UR5h/Z75fboePBfIKFFjvfhDAjYZB/vhewhdhHe8= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1748971514; c=relaxed/simple; bh=xUNIZuFW5Ede/pkil/BhQOrBSMqZ71aURRRtugY4si0=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=oGf7u5U/jdff/XE3kJBTpYomfKOdPslgBbvMe5T6JPDKXODB/c9Wlw2psf1K1nUmtlAu38rs2Y3To4TG3OqTIxzzPYLRqCChOWjRONHLM31GG6dhQHOQul5CzvxtEHDeNugnCfqMlHb1hEBGoGoX3DFDpUdLi5QfSbGoDvhNT2Y= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=srxYVaDp; arc=fail smtp.client-ip=40.107.223.45 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="srxYVaDp" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=XepsLUytwUgBc1ZvGTx1DkDArTIh63/Nzn5+ZNqPoLDvfO/cEVHOntjtgVzPvuha4YPwV+p3aoqSoOIh9rw4yBRjt1Q+RCIxpBMaGxkwTvACIL6kJJqDRNnZ0ADKfR4ERneCZmvU4vkRVx7cnS+EN0ZoMDXeNJx7pJpsfxyHLR5PbP3A1MxNmlAOcVOS5D0bIF1bR5QuFc7PP2+dU/QTDuxXYnYOuS/lVPxzIEslNOxSRaaGxJHCkvgIyvjXTJ8kzpr8khKLbDAzpsoR6t1OinT/MHD0JhV2I/5vu/JHhKr/PSBGR51UIN8FSkT5cWrQCG8vGij8RkGjQDnMZFDixQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Xt5W+h15YIgGglewz+nVkk/Adz2pZqkQUUikLhstBYU=; b=VSHtiRBM0GsZG/gkwinG905BrecaLLLcl0Y5rDVIEay7It8Aa/XjrXDrtUL3/ktWtDuVFPlutOSQvF5ZrBPrhgV+19AHhT7s/bAvrhTyRDLzcQrgtnQgwI3T4+AgWw9jFiNXheUtjYtfYKg6PLUJMoM9bLCC+8+VLXGTscXahoXhlFRyJ/ZloqZWEl/zVRWhJjcK1Q152jUMrEEkYFPflv0vqsOf6PJJ3/+1aaaElWOWLEDDec1KZH4SqVT5KHhvwG5en4R50Fuf34M0bMRPE2TIQf64QBR+jpxgIb2f78DSUbz0NIspoADvqzhMsTFeb//3MC0XEaZLzEiMMiXF1A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=stgolabs.net smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Xt5W+h15YIgGglewz+nVkk/Adz2pZqkQUUikLhstBYU=; b=srxYVaDpiGLg4yI/JWPsIgegbvHShvxaRmxAom4Rb6HGUrNOkFwSPiiZEtxGaB3+2f4E7zCPlDs2eMVjv7DE+PZ8sEDvDQWQIeUW+ZYHRfW/mFwasdxR0eHv4NCxFQ1H0TXyiJehODm42KX8aNdIxBBx0JRL+2ia1oVVnAQc0/Y= Received: from MN2PR11CA0025.namprd11.prod.outlook.com (2603:10b6:208:23b::30) by IA1PR12MB8239.namprd12.prod.outlook.com (2603:10b6:208:3f7::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8769.26; Tue, 3 Jun 2025 17:25:10 +0000 Received: from BL6PEPF00020E65.namprd04.prod.outlook.com (2603:10b6:208:23b:cafe::9a) by MN2PR11CA0025.outlook.office365.com (2603:10b6:208:23b::30) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8769.21 via Frontend Transport; Tue, 3 Jun 2025 17:25:10 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by BL6PEPF00020E65.mail.protection.outlook.com (10.167.249.26) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.8792.29 via Frontend Transport; Tue, 3 Jun 2025 17:25:10 +0000 Received: from ethanolx7ea3host.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Tue, 3 Jun 2025 12:25:00 -0500 From: Terry Bowman To: , , , , , , , , , , , , , , , , , , , , , , , , , , Subject: [PATCH v9 12/16] cxl/pci: Introduce CXL Endpoint protocol error handlers Date: Tue, 3 Jun 2025 12:22:35 -0500 Message-ID: <20250603172239.159260-13-terry.bowman@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250603172239.159260-1-terry.bowman@amd.com> References: <20250603172239.159260-1-terry.bowman@amd.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL6PEPF00020E65:EE_|IA1PR12MB8239:EE_ X-MS-Office365-Filtering-Correlation-Id: be798828-f6d0-478c-2c2e-08dda2c39797 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|1800799024|376014|82310400026|36860700013|921020; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?zSLlfy8n4FZ13v4w39hdCVhGIfwYhyEWhc/iafS3nTX3tQHlsYPux21hMFC5?= =?us-ascii?Q?iTAXQK5E08L8WklZiqRvw8AL4qE79rp4oAg4OyrhATa+YgAwSHvvwX+4tAq0?= =?us-ascii?Q?0r5n8fdKJ++cfEhYDHj3uNqOenaUtXBtsE7XlcTzPmdhlFmizDA+O3dNQv9Z?= =?us-ascii?Q?hbkCfX5nDVVF4yqVvJhfkFKBH/iG1ZY8dw/4V64fTj8955juo8hRsHTnLcjm?= =?us-ascii?Q?8w/cusE7adu6kXmSOvKtcvOIVVRI+NvSHBSqK0h+53tRZfuClMWkzM3Ojf5A?= =?us-ascii?Q?mP3GoEkOhnBtsbQo2+ZzB4eLlA/4JNI/e0pc+VUShEFczZAc78epOju/r3sC?= =?us-ascii?Q?CwmYJ6tIYoAVWZ4Nn3gZGIewaBkX8AfekRwJEZf+NyD3yNx6qjMoazte3xo/?= =?us-ascii?Q?cQZcF3xP8UaZdql8UefGdehqgb/raSjQJaQ91nN/5ZQG+ecfeQ2unIAih+pY?= =?us-ascii?Q?bWg3ApMGX/bThrLD33D9xJM9732khlI+waCAttDvFh6AYzn8lv4ejrsE9B6G?= =?us-ascii?Q?2c49YfvBk2fEZgJhZVrmi6IwFPXJEdVimxWjuzFdt24bBSyUAg4v4/A8R2l3?= =?us-ascii?Q?7DcI+CmhyPMYB2gvh/qByND1nl36ZtS4IzX/JbT199mJjqJkKFPdREhIgv2S?= =?us-ascii?Q?DTix1NdPh39L+czUG9vwl/HL6ESy6QRneiUytYo9xtCNSQRENWpg7ojzLiE8?= =?us-ascii?Q?Huu/muNiUrBvoN+SjsbcFFhuo/sBSzTdfv9SlsPgNSlikoxQNPdzjLkqolAr?= =?us-ascii?Q?W5TK33sLUoKeTFng5/9C4hTDvvUxGkJuS4HLbYCLWhWd0S/GL8XtdIgvZ86M?= =?us-ascii?Q?1JlJ1HI7/ApugPb+44MeACedXe2mNNyWAHE9C2UPyVYN5Eedf92LjEthdY/d?= =?us-ascii?Q?nkEGlXfR2noje+kPAfskjCaPvtZAeFcKkAIXDNF73JWSx5np/COY/W3I5CdA?= =?us-ascii?Q?PSigJx8MBFkBqNToeWX4vwISbLUzz6axTWr/PlcOMQI7zwZGYdjnjo0tXl8P?= =?us-ascii?Q?3hZ6tO74zIRP9oKvFyPyygtqfZAe6p+/V+TbAUTT7aXyBOfKr0YPqqVIFciw?= =?us-ascii?Q?M7iylDZa751zhk5UOFkfEx0MzehxHjfvb/yvDTiE+DwB2fo6994UJ1Ieq1de?= =?us-ascii?Q?8DMWqnkteXu2bjJWOezshlhE2h5j5L7DyKWZSnWVSo+Gaq0Fhj40SkwieBp8?= =?us-ascii?Q?SIXV4LL1yzEBwELzmfB93z+n7K8Fjuhy6V6eyAwkLnxko54Nq1v5BsRnclYY?= =?us-ascii?Q?6KeKAdEzwpaPVxJKS6S8htrqxdIHdDmq3tS3ZvxUfpU69yt3fKPu60hl7Cdh?= =?us-ascii?Q?rS9sbGZjHuFk/71zEE2W78E/u/BUF6VLqWIp7dy0RjPYvhINlebBjzfWv8wF?= =?us-ascii?Q?0upvAccwD9O81mnOFOflJLvqvEt3XFRj7ofL0ajFvRxkUYVAsoCLhhrpfy87?= =?us-ascii?Q?a6ClE+lDP/o0FLqBV1vGacsLW9N/1cLwkLk4aIMTajsB7GoE25qDsixeWt90?= =?us-ascii?Q?DYxt0oDFYBCr0l96w/Crh2z3FWtU2ljn3sS27Hapf98Je2eAaZZqDE1b7w?= =?us-ascii?Q?=3D=3D?= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(7416014)(1800799024)(376014)(82310400026)(36860700013)(921020);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Jun 2025 17:25:10.1440 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: be798828-f6d0-478c-2c2e-08dda2c39797 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BL6PEPF00020E65.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB8239 Content-Type: text/plain; charset="utf-8" CXL Endpoint protocol errors are currently handled using PCI error handlers. The CXL Endpoint requires CXL specific handling in the case of uncorrectable error handling not provided by the PCI handlers. Add CXL specific handlers for CXL Endpoints. Rename the existing cxl_error_handlers to be pci_error_handlers to more correctly indicate the error type and follow naming consistency. Keep the existing PCI Endpoint handlers. PCI handlers can be called if the CXL device is not trained for alternate protocol (CXL). Update the CXL Endpoint PCI handlers to call the CXL handler. If the CXL uncorrectable handler returns PCI_ERS_RESULT_PANIC then the PCI handler invokes panic(). Signed-off-by: Terry Bowman Reviewed-by: Kuppuswamy Sathyanarayanan --- drivers/cxl/core/pci.c | 80 ++++++++++++++++++++++-------------------- drivers/cxl/core/ras.c | 10 +++--- drivers/cxl/cxl.h | 4 +++ drivers/cxl/cxlpci.h | 6 ++-- drivers/cxl/pci.c | 8 ++--- 5 files changed, 58 insertions(+), 50 deletions(-) diff --git a/drivers/cxl/core/pci.c b/drivers/cxl/core/pci.c index f5f87c2c3fd5..e094ef518e0a 100644 --- a/drivers/cxl/core/pci.c +++ b/drivers/cxl/core/pci.c @@ -710,8 +710,8 @@ static void header_log_copy(void __iomem *ras_base, u32= *log) * Log the state of the RAS status registers and prepare them to log the * next error status. Return 1 if reset needed. */ -static bool __cxl_handle_ras(struct device *dev, u64 serial, - void __iomem *ras_base) +static pci_ers_result_t __cxl_handle_ras(struct device *dev, u64 serial, + void __iomem *ras_base) { u32 hl[CXL_HEADERLOG_SIZE_U32]; void __iomem *addr; @@ -720,13 +720,13 @@ static bool __cxl_handle_ras(struct device *dev, u64 = serial, =20 if (!ras_base) { dev_warn_once(dev, "CXL RAS register block is not mapped"); - return false; + return PCI_ERS_RESULT_NONE; } =20 addr =3D ras_base + CXL_RAS_UNCORRECTABLE_STATUS_OFFSET; status =3D readl(addr); if (!(status & CXL_RAS_UNCORRECTABLE_STATUS_MASK)) - return false; + return PCI_ERS_RESULT_NONE; =20 /* If multiple errors, log header points to first error from ctrl reg */ if (hweight32(status) > 1) { @@ -743,12 +743,11 @@ static bool __cxl_handle_ras(struct device *dev, u64 = serial, trace_cxl_aer_uncorrectable_error(dev, serial, status, fe, hl); writel(status & CXL_RAS_UNCORRECTABLE_STATUS_MASK, addr); =20 - return true; + return PCI_ERS_RESULT_PANIC; } =20 static bool cxl_handle_endpoint_ras(struct cxl_dev_state *cxlds) { - return __cxl_handle_ras(&cxlds->cxlmd->dev, cxlds->serial, cxlds->regs.ra= s); } =20 @@ -844,14 +843,15 @@ static void cxl_handle_rdport_errors(struct cxl_dev_s= tate *cxlds) static void cxl_handle_rdport_errors(struct cxl_dev_state *cxlds) { } #endif =20 -void cxl_cor_error_detected(struct pci_dev *pdev) +void cxl_cor_error_detected(struct device *dev) { + struct pci_dev *pdev =3D to_pci_dev(dev); struct cxl_dev_state *cxlds =3D pci_get_drvdata(pdev); - struct device *dev =3D &cxlds->cxlmd->dev; + struct device *cxlmd_dev =3D &cxlds->cxlmd->dev; =20 - scoped_guard(device, dev) { - if (!dev->driver) { - dev_warn(&pdev->dev, + scoped_guard(device, cxlmd_dev) { + if (!cxlmd_dev->driver) { + dev_warn(dev, "%s: memdev disabled, abort error handling\n", dev_name(dev)); return; @@ -865,20 +865,28 @@ void cxl_cor_error_detected(struct pci_dev *pdev) } EXPORT_SYMBOL_NS_GPL(cxl_cor_error_detected, "CXL"); =20 -pci_ers_result_t cxl_error_detected(struct pci_dev *pdev, - pci_channel_state_t state) +void pci_cor_error_detected(struct pci_dev *pdev) { struct cxl_dev_state *cxlds =3D pci_get_drvdata(pdev); - struct cxl_memdev *cxlmd =3D cxlds->cxlmd; - struct device *dev =3D &cxlmd->dev; - bool ue; + struct device *cxlmd_dev __free(put_device) =3D get_device(&cxlds->cxlmd-= >dev); =20 - scoped_guard(device, dev) { + cxl_cor_error_detected(&pdev->dev); +} +EXPORT_SYMBOL_NS_GPL(pci_cor_error_detected, "CXL"); + +pci_ers_result_t cxl_error_detected(struct device *dev) +{ + struct pci_dev *pdev =3D to_pci_dev(dev); + struct cxl_dev_state *cxlds =3D pci_get_drvdata(pdev); + struct device *cxlmd_dev =3D &cxlds->cxlmd->dev; + pci_ers_result_t ue; + + scoped_guard(device, cxlmd_dev) { if (!dev->driver) { dev_warn(&pdev->dev, "%s: memdev disabled, abort error handling\n", dev_name(dev)); - return PCI_ERS_RESULT_DISCONNECT; + return PCI_ERS_RESULT_PANIC; } =20 if (cxlds->rcd) @@ -892,29 +900,25 @@ pci_ers_result_t cxl_error_detected(struct pci_dev *p= dev, ue =3D cxl_handle_endpoint_ras(cxlds); } =20 - - switch (state) { - case pci_channel_io_normal: - if (ue) { - device_release_driver(dev); - return PCI_ERS_RESULT_NEED_RESET; - } - return PCI_ERS_RESULT_CAN_RECOVER; - case pci_channel_io_frozen: - dev_warn(&pdev->dev, - "%s: frozen state error detected, disable CXL.mem\n", - dev_name(dev)); - device_release_driver(dev); - return PCI_ERS_RESULT_NEED_RESET; - case pci_channel_io_perm_failure: - dev_warn(&pdev->dev, - "failure state error detected, request disconnect\n"); - return PCI_ERS_RESULT_DISCONNECT; - } - return PCI_ERS_RESULT_NEED_RESET; + return ue; } EXPORT_SYMBOL_NS_GPL(cxl_error_detected, "CXL"); =20 +pci_ers_result_t pci_error_detected(struct pci_dev *pdev, + pci_channel_state_t error) +{ + struct cxl_dev_state *cxlds =3D pci_get_drvdata(pdev); + struct device *cxlmd_dev __free(put_device) =3D &cxlds->cxlmd->dev; + pci_ers_result_t rc; + + rc =3D cxl_error_detected(&pdev->dev); + if (rc =3D=3D PCI_ERS_RESULT_PANIC) + panic("CXL cachemem error."); + + return rc; +} +EXPORT_SYMBOL_NS_GPL(pci_error_detected, "CXL"); + static int cxl_flit_size(struct pci_dev *pdev) { if (cxl_pci_flit_256(pdev)) diff --git a/drivers/cxl/core/ras.c b/drivers/cxl/core/ras.c index 0ef8c2068c0c..664f532cc838 100644 --- a/drivers/cxl/core/ras.c +++ b/drivers/cxl/core/ras.c @@ -153,7 +153,7 @@ static int cxl_report_error_detected(struct pci_dev *pd= ev, void *data) struct device *dev __free(put_device) =3D get_device(&cxlds->cxlmd->dev); =20 device_lock(&pdev->dev); - vote =3D cxl_error_detected(pdev, pci_channel_io_frozen); + vote =3D cxl_error_detected(&pdev->dev); *result =3D cxl_merge_result(*result, vote); device_unlock(&pdev->dev); =20 @@ -223,7 +223,7 @@ static int cxl_rch_handle_error_iter(struct pci_dev *pd= ev, void *data) struct device *dev __free(put_device) =3D get_device(&cxlds->cxlmd->dev); =20 if (err_info->severity =3D=3D AER_CORRECTABLE) - cxl_cor_error_detected(pdev); + cxl_cor_error_detected(dev); else cxl_do_recovery(pdev); =20 @@ -244,6 +244,8 @@ static struct pci_dev *sbdf_to_pci(struct cxl_prot_erro= r_info *err_info) static void cxl_handle_prot_error(struct cxl_prot_error_info *err_info) { struct pci_dev *pdev __free(pci_dev_put) =3D pci_dev_get(sbdf_to_pci(err_= info)); + struct cxl_dev_state *cxlds =3D pci_get_drvdata(pdev); + struct device *cxlmd_dev __free(put_device) =3D get_device(&cxlds->cxlmd-= >dev); =20 if (!pdev) { pr_err("Failed to find the CXL device\n"); @@ -260,15 +262,13 @@ static void cxl_handle_prot_error(struct cxl_prot_err= or_info *err_info) =20 if (err_info->severity =3D=3D AER_CORRECTABLE) { int aer =3D pdev->aer_cap; - struct cxl_dev_state *cxlds =3D pci_get_drvdata(pdev); - struct device *dev __free(put_device) =3D get_device(&cxlds->cxlmd->dev); =20 if (aer) pci_clear_and_set_config_dword(pdev, aer + PCI_ERR_COR_STATUS, 0, PCI_ERR_COR_INTERNAL); =20 - cxl_cor_error_detected(pdev); + cxl_cor_error_detected(&pdev->dev); =20 pcie_clear_device_status(pdev); } else { diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index 73be66ef36a2..6fd9a42eb304 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -11,6 +11,7 @@ #include #include #include +#include =20 extern const struct nvdimm_security_ops *cxl_security_ops; =20 @@ -797,6 +798,9 @@ static inline int cxl_root_decoder_autoremove(struct de= vice *host, } int cxl_endpoint_autoremove(struct cxl_memdev *cxlmd, struct cxl_port *end= point); =20 +void cxl_cor_error_detected(struct device *dev); +pci_ers_result_t cxl_error_detected(struct device *dev); + /** * struct cxl_endpoint_dvsec_info - Cached DVSEC info * @mem_enabled: cached value of mem_enabled in the DVSEC at init time diff --git a/drivers/cxl/cxlpci.h b/drivers/cxl/cxlpci.h index 6f1396ef7b77..a572c57c6c63 100644 --- a/drivers/cxl/cxlpci.h +++ b/drivers/cxl/cxlpci.h @@ -133,7 +133,7 @@ struct cxl_dev_state; int cxl_hdm_decode_init(struct cxl_dev_state *cxlds, struct cxl_hdm *cxlhd= m, struct cxl_endpoint_dvsec_info *info); void read_cdat_data(struct cxl_port *port); -void cxl_cor_error_detected(struct pci_dev *pdev); -pci_ers_result_t cxl_error_detected(struct pci_dev *pdev, - pci_channel_state_t state); +void pci_cor_error_detected(struct pci_dev *pdev); +pci_ers_result_t pci_error_detected(struct pci_dev *pdev, + pci_channel_state_t error); #endif /* __CXL_PCI_H__ */ diff --git a/drivers/cxl/pci.c b/drivers/cxl/pci.c index 785aa2af5eaa..2b948e94bc97 100644 --- a/drivers/cxl/pci.c +++ b/drivers/cxl/pci.c @@ -1112,11 +1112,11 @@ static void cxl_reset_done(struct pci_dev *pdev) } } =20 -static const struct pci_error_handlers cxl_error_handlers =3D { - .error_detected =3D cxl_error_detected, +static const struct pci_error_handlers pci_error_handlers =3D { + .error_detected =3D pci_error_detected, .slot_reset =3D cxl_slot_reset, .resume =3D cxl_error_resume, - .cor_error_detected =3D cxl_cor_error_detected, + .cor_error_detected =3D pci_cor_error_detected, .reset_done =3D cxl_reset_done, }; =20 @@ -1124,7 +1124,7 @@ static struct pci_driver cxl_pci_driver =3D { .name =3D KBUILD_MODNAME, .id_table =3D cxl_mem_pci_tbl, .probe =3D cxl_pci_probe, - .err_handler =3D &cxl_error_handlers, + .err_handler =3D &pci_error_handlers, .dev_groups =3D cxl_rcd_groups, .driver =3D { .probe_type =3D PROBE_PREFER_ASYNCHRONOUS, --=20 2.34.1 From nobody Thu Dec 18 05:05:03 2025 Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2086.outbound.protection.outlook.com [40.107.92.86]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 528BE24291C; Tue, 3 Jun 2025 17:25:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.92.86 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1748971521; cv=fail; b=eteO/SFEa+gVebY+qMPem2kCutmv18tx5EGV4RoZb/JKCFtr9Yv8t7aJboaUEvQTujr4Ufxit60/M53fUy83IH5ojRkDI3uJjU5nBpgj2oT6Vn7SqWMYE13arf28vStF5CKpCOWnbpm3nhJKEbPBYyZsxSPyZBUvLicFtX5DFUg= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1748971521; c=relaxed/simple; bh=3d628hLQu69+xLbkVnQiNnTERMTtqWwRhXWVdvcZAAQ=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=cMtNT8OtAC20NkQKHU3+FzbRJh9vj+FxY7HfS8BxpzpW51Scv1pkJtZFkGuWgb1hZyZoRyUp/skwekuGtem1FSa0lJnYKZA8QbgneHdVTvCn0nZrOja7XEP92Q3x7nsXthPX9x0pJv/MPl3sMa02Y460GTdvCn9jIh59blMdNfc= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=K1vn3f43; arc=fail smtp.client-ip=40.107.92.86 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="K1vn3f43" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=sPHmdf/MNoiRvsGJEXz+PRi96YxUYLaKeq8W8y3ZurZqhcrb3DYjzGXRiwWlJllz4j701yd4wF9C3gzRpNmBm+y1mL0CdDW/JmR2BV/DnioY/thycelxXFNlJzE0GJf6a4jX4d5Goo+zfUo+kuEmARG9aeeHWIRzUFCJcVDIaXImPPeTtY8PQKu/Q1eJnw6kQF09Y8e8MegvCA+sRciNn4cDYqHnU4HWjx+P5NWRoCaIfoTS2bpFoNOw4mXB2TzREd/2zvhQ9tZUTuk6R4ss0XfAI49j2aYTpCVCMnxXEr/7fmeC6Go4LBE6NJZPhgLaWKoOEeVQgyjIaWku0tzWHg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=NPD80tG7FgwYOB9ot4RNZJ/WPPDkfSKnQwJEYybNJeQ=; b=oWIdIUobZAtQgSKBKkFv8yj/akNVOuwV5neAjSLyYI4Uqij8DPo8x0kBPPkNWXpzbqBbrmOem0pmLa0pkvnuH2dtPUHwS7P4QPuWG0nqJzmVjXBggwA6txBkPWrI5K8Ng5i+cfDXE5eufjScGcYb0oLkjgjLqGDqBOstXPV83LfsGrrcMcQfiskQyfggL+50WW0vz5Dl23LEIgr/36VygLZwRr/3Y9FKwVh4DGKNmEwnK64/1mANJVfcf+nTfVj3q1emVMkNEGZc7Th0g7qdkh4QI2WwqWcRBhbuQvtJbkJ4q+0wPWtYI+3IAbDXfbg2Y4Qzqkg8OURiSXhcIZAyyg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=stgolabs.net smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=NPD80tG7FgwYOB9ot4RNZJ/WPPDkfSKnQwJEYybNJeQ=; b=K1vn3f43sutZwuxxB+oYIuZ3bgce88Ij+wbQMyO/hcNCioksFEksRgGOrd/kOQrYLHmUuIu7TisSMhkGJwtUrt6yMTPGy8Sf9rQCKAjYO7dwI+7CCJRcTpXSeAJUnsjge159itjgWQT/2XCBELPqwwK5qfAOApCVK3GdCOfVzqs= Received: from BL1PR13CA0234.namprd13.prod.outlook.com (2603:10b6:208:2bf::29) by CH1PPFC8B3B7859.namprd12.prod.outlook.com (2603:10b6:61f:fc00::622) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8678.27; Tue, 3 Jun 2025 17:25:13 +0000 Received: from BN2PEPF000044AB.namprd04.prod.outlook.com (2603:10b6:208:2bf:cafe::98) by BL1PR13CA0234.outlook.office365.com (2603:10b6:208:2bf::29) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8792.21 via Frontend Transport; Tue, 3 Jun 2025 17:25:13 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by BN2PEPF000044AB.mail.protection.outlook.com (10.167.243.106) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.8792.29 via Frontend Transport; Tue, 3 Jun 2025 17:25:12 +0000 Received: from ethanolx7ea3host.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Tue, 3 Jun 2025 12:25:11 -0500 From: Terry Bowman To: , , , , , , , , , , , , , , , , , , , , , , , , , , Subject: [PATCH v9 13/16] cxl/pci: Introduce CXL Port protocol error handlers Date: Tue, 3 Jun 2025 12:22:36 -0500 Message-ID: <20250603172239.159260-14-terry.bowman@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250603172239.159260-1-terry.bowman@amd.com> References: <20250603172239.159260-1-terry.bowman@amd.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN2PEPF000044AB:EE_|CH1PPFC8B3B7859:EE_ X-MS-Office365-Filtering-Correlation-Id: 25d6df14-bbac-4111-89af-08dda2c3993c X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|36860700013|82310400026|7416014|376014|921020; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?OaCOOJ7+/aoL6+Yry/RmS/Fig5n0kbVTkVYzg4HX9pj3TJWTwxbnxHjWXEOL?= =?us-ascii?Q?RjSgIFktk1M7ltmp32RP2WOvfB1QiDA9xISW7U23wgz3wXvu5R3F40iwfZI+?= =?us-ascii?Q?sQw0nNY7yPlH3VpWrPl7eJ/V43UOxl+dg5gQIY3r00nA1hzTe0d7VbDbIkfQ?= =?us-ascii?Q?FUe3m2/hKApqjuKGSvCvL61z0mk0OUPqTdwjpcfJdse4KRSTtSVyN1U7nZkj?= =?us-ascii?Q?Cq61o63IiKkVMU2wW0iAIYHjVWekJ+afzN0DMEE+Jvpoa5E+7xnDE0P2m1yr?= =?us-ascii?Q?sHnWqEfe24FCjiM5rqSmyufo0tVPJsU9HNQtIgvSCFlOvY0UZjLRuQONO/pm?= =?us-ascii?Q?E4urAsmctjiYWXLh6oIQMFs0qf92OTq6Csk8OKFBAf8O1Vbayd3rJJGK/CFz?= =?us-ascii?Q?uyNucRcs+H6rWoDqEbSTXHUUHdv9yp07PredaOyBYUcYsNjOmdcJkIq1oXYM?= =?us-ascii?Q?LYXzkCczwo3ttwpAaKFPGiupf/UuD00dUqZ4c3uOsbAhIzI0xZPkwmfpuPm3?= =?us-ascii?Q?pVxh9w6UFWsVNXpPVOCUMPx9GykmHosoZOpjopupOi0VkFdHb/iwKwzrgHGU?= =?us-ascii?Q?Tl1CuZh+BLgvnGD44lnUdAIzY/IfSZloKkq+nSi4fuXy25s+AJJ3MrWChM8Z?= =?us-ascii?Q?CQDX1R7SyPqtsbrUs6QrawSsoFHpS1jCXW/YlkvzQD1Pm4/xBUu6ZSixJn0x?= =?us-ascii?Q?qIOpkdwDWv0DRsxJdXL+tUcwiZ5too1EcmwcCEiKF/37YxrXlpQYrHruzcpo?= =?us-ascii?Q?v1nARDK66t0UYF9JquZ43qdwvHiiQJvbCKA7LrkX+AWmXyvYOh9lpQdfKxeU?= =?us-ascii?Q?mP6BBUaHkLz1O9Lc8h9vxQUR0SyOXdZSgQRlx8tJjnb4BK7l6JAcErKC4c4z?= =?us-ascii?Q?FDWc1MPfOtpPqGfQ97LVOfvtdPtx49ASCQv6GhKMYct49BNJEwrH8ACN7ZB8?= =?us-ascii?Q?58w/AuH2urvddDDEzaiiY+h0LjrofMNXjOZMfXjrCSwkT62EbeZbgX1eYc/E?= =?us-ascii?Q?iQTB1tpBhnumdsrjGpKkdpvmlZaEYGSIr7+4kNovjw5BJPdJRZsccBPJmFlU?= =?us-ascii?Q?0Za+LQQUoIopmldUPz11LqsSDa4iyiGXVZh91VjE1ganj0PCo4Ush7A2aoTv?= =?us-ascii?Q?4KqpanYOSinLG/vndcSDumnXjENS1P5Fr5UXxAGKYGd2VFin/tfhqXpeQGAY?= =?us-ascii?Q?rd0iYOb83ScTGzBwyPktaB4GucAZw9NDJ37vCB1D1c6Q2wcNTB6TLUCMu+fB?= =?us-ascii?Q?NqjqMWNMQCecJHmSCmgiVSFfxB5Chu6hSVijgmuqLuayw5SrSjMkcsbTYgFO?= =?us-ascii?Q?C5EKRc8caXPP5bhtQDyYerrAyfTe4N79QI7RMV6Uvz6qtb/bZlxrhqx6gzGD?= =?us-ascii?Q?vmQ/IphPZ1ww4k0vG8MP0zvIM+E/3aLdDPmx3gAtmiYU8qI2DByyYgcn8UT9?= =?us-ascii?Q?wIFu0PEBMesMjZ/7v2IHw90VVUoQT9atJkXa1cmyAcjUpsC5lOQN0j9ti+Dt?= =?us-ascii?Q?wkDuXJF7aiuDaQRg/dUfxJLNWNKdPZZh0kFnYppIGMfok2w1iDP/Vgagug?= =?us-ascii?Q?=3D=3D?= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(1800799024)(36860700013)(82310400026)(7416014)(376014)(921020);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Jun 2025 17:25:12.9236 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 25d6df14-bbac-4111-89af-08dda2c3993c X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BN2PEPF000044AB.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH1PPFC8B3B7859 Content-Type: text/plain; charset="utf-8" Introduce CXL error handlers for CXL Port devices. Add functions cxl_port_cor_error_detected() and cxl_port_error_detected(). These will serve as the handlers for all CXL Port devices. Introduce cxl_get_ras_base() to provide the RAS base address needed by the handlers. Update cxl_handle_prot_error() to call the CXL Port or CXL Endpoint handler depending on which CXL device reports the error. Implement pci_get_ras_base() to return the cached RAS register address of a CXL Root Port, CXL Downstream Port, or CXL Upstream Port. Update the AER driver's is_cxl_error() to remove the filter PCI type check because CXL Port devices are now supported. Signed-off-by: Terry Bowman Reviewed-by: Jonathan Cameron Reviewed-by: Kuppuswamy Sathyanarayanan --- drivers/cxl/core/core.h | 2 + drivers/cxl/core/pci.c | 61 ++++++++++++++++++++++++++ drivers/cxl/core/port.c | 4 +- drivers/cxl/core/ras.c | 96 ++++++++++++++++++++++++++++++++++++----- drivers/cxl/cxl.h | 5 +++ drivers/pci/pcie/aer.c | 5 --- 6 files changed, 155 insertions(+), 18 deletions(-) diff --git a/drivers/cxl/core/core.h b/drivers/cxl/core/core.h index c73f39d14dd7..23d15eef01d2 100644 --- a/drivers/cxl/core/core.h +++ b/drivers/cxl/core/core.h @@ -122,6 +122,8 @@ void cxl_ras_exit(void); int cxl_gpf_port_setup(struct cxl_dport *dport); int cxl_acpi_get_extended_linear_cache_size(struct resource *backing_res, int nid, resource_size_t *size); +struct cxl_port *find_cxl_port(struct device *dport_dev, + struct cxl_dport **dport); =20 #ifdef CONFIG_CXL_FEATURES size_t cxl_get_feature(struct cxl_mailbox *cxl_mbox, const uuid_t *feat_uu= id, diff --git a/drivers/cxl/core/pci.c b/drivers/cxl/core/pci.c index e094ef518e0a..b6836825e8df 100644 --- a/drivers/cxl/core/pci.c +++ b/drivers/cxl/core/pci.c @@ -753,6 +753,67 @@ static bool cxl_handle_endpoint_ras(struct cxl_dev_sta= te *cxlds) =20 #ifdef CONFIG_PCIEAER_CXL =20 +static void __iomem *cxl_get_ras_base(struct device *dev) +{ + struct pci_dev *pdev =3D to_pci_dev(dev); + void __iomem *ras_base; + + switch (pci_pcie_type(pdev)) { + case PCI_EXP_TYPE_ROOT_PORT: + case PCI_EXP_TYPE_DOWNSTREAM: + { + struct cxl_dport *dport =3D NULL; + struct cxl_port *port __free(put_cxl_port) =3D find_cxl_port(&pdev->dev,= &dport); + + if (!dport || !dport->dport_dev) { + pci_err(pdev, "Failed to find the CXL device"); + return NULL; + } + + ras_base =3D dport ? dport->regs.ras : NULL; + break; + } + case PCI_EXP_TYPE_UPSTREAM: + { + struct cxl_port *port; + struct device *dev __free(put_device) =3D bus_find_device(&cxl_bus_type,= NULL, + &pdev->dev, match_uport); + + if (!dev || !is_cxl_port(dev)) { + pci_err(pdev, "Failed to find the CXL device"); + return NULL; + } + + port =3D to_cxl_port(dev); + ras_base =3D port ? port->uport_regs.ras : NULL; + break; + } + default: + { + pci_warn_once(pdev, "Error: Unsupported device type (%X)", pci_pcie_type= (pdev)); + return NULL; + } + } + + return ras_base; +} + +void cxl_port_cor_error_detected(struct device *dev) +{ + void __iomem *ras_base =3D cxl_get_ras_base(dev); + + __cxl_handle_cor_ras(dev, 0, ras_base); +} +EXPORT_SYMBOL_NS_GPL(cxl_port_cor_error_detected, "CXL"); + +pci_ers_result_t cxl_port_error_detected(struct device *dev) +{ + void __iomem *ras_base =3D cxl_get_ras_base(dev); + + return __cxl_handle_ras(dev, 0, ras_base); +} +EXPORT_SYMBOL_NS_GPL(cxl_port_error_detected, "CXL"); + static void cxl_handle_rdport_cor_ras(struct cxl_dev_state *cxlds, struct cxl_dport *dport) { diff --git a/drivers/cxl/core/port.c b/drivers/cxl/core/port.c index eb46c6764d20..07b9bb0f601f 100644 --- a/drivers/cxl/core/port.c +++ b/drivers/cxl/core/port.c @@ -1341,8 +1341,8 @@ static struct cxl_port *__find_cxl_port(struct cxl_fi= nd_port_ctx *ctx) return NULL; } =20 -static struct cxl_port *find_cxl_port(struct device *dport_dev, - struct cxl_dport **dport) +struct cxl_port *find_cxl_port(struct device *dport_dev, + struct cxl_dport **dport) { struct cxl_find_port_ctx ctx =3D { .dport_dev =3D dport_dev, diff --git a/drivers/cxl/core/ras.c b/drivers/cxl/core/ras.c index 664f532cc838..6093e70ece37 100644 --- a/drivers/cxl/core/ras.c +++ b/drivers/cxl/core/ras.c @@ -140,20 +140,85 @@ static pci_ers_result_t cxl_merge_result(enum pci_ers= _result orig, return orig; } =20 -static int cxl_report_error_detected(struct pci_dev *pdev, void *data) +int match_uport(struct device *dev, const void *data) { - pci_ers_result_t vote, *result =3D data; - struct cxl_dev_state *cxlds; + const struct device *uport_dev =3D data; + struct cxl_port *port; =20 - if ((pci_pcie_type(pdev) !=3D PCI_EXP_TYPE_ENDPOINT) && - (pci_pcie_type(pdev) !=3D PCI_EXP_TYPE_RC_END)) + if (!is_cxl_port(dev)) return 0; =20 - cxlds =3D pci_get_drvdata(pdev); - struct device *dev __free(put_device) =3D get_device(&cxlds->cxlmd->dev); + port =3D to_cxl_port(dev); + + return port->uport_dev =3D=3D uport_dev; +} +EXPORT_SYMBOL_NS_GPL(match_uport, "CXL"); + +/* Return 'struct device*' responsible for freeing pdev's CXL resources */ +static struct device *get_pci_cxl_host_dev(struct pci_dev *pdev) +{ + struct device *host_dev; + + switch (pci_pcie_type(pdev)) { + case PCI_EXP_TYPE_ROOT_PORT: + case PCI_EXP_TYPE_DOWNSTREAM: + { + struct cxl_dport *dport =3D NULL; + struct cxl_port *port =3D find_cxl_port(&pdev->dev, &dport); + + if (!dport || !dport->dport_dev) + return NULL; + + host_dev =3D &port->dev; + break; + } + case PCI_EXP_TYPE_UPSTREAM: + { + struct cxl_port *port; + struct device *cxl_dev =3D bus_find_device(&cxl_bus_type, NULL, + &pdev->dev, match_uport); + + if (!cxl_dev || !is_cxl_port(cxl_dev)) + return NULL; + + port =3D to_cxl_port(cxl_dev); + host_dev =3D &port->dev; + break; + } + case PCI_EXP_TYPE_ENDPOINT: + case PCI_EXP_TYPE_RC_END: + { + struct cxl_dev_state *cxlds =3D pci_get_drvdata(pdev); + + if (!cxlds) + return NULL; + + host_dev =3D get_device(&cxlds->cxlmd->dev); + break; + } + default: + { + pci_warn_once(pdev, "Error: Unsupported device type (%X)", pci_pcie_type= (pdev)); + return NULL; + } + } + + return host_dev; +} + +static int cxl_report_error_detected(struct pci_dev *pdev, void *data) +{ + struct device *dev =3D &pdev->dev; + struct device *host_dev __free(put_device) =3D get_pci_cxl_host_dev(pdev); + pci_ers_result_t vote, *result =3D data; =20 device_lock(&pdev->dev); - vote =3D cxl_error_detected(&pdev->dev); + if ((pci_pcie_type(pdev) =3D=3D PCI_EXP_TYPE_ENDPOINT) || + (pci_pcie_type(pdev) =3D=3D PCI_EXP_TYPE_RC_END)) { + vote =3D cxl_error_detected(dev); + } else { + vote =3D cxl_port_error_detected(dev); + } *result =3D cxl_merge_result(*result, vote); device_unlock(&pdev->dev); =20 @@ -244,14 +309,18 @@ static struct pci_dev *sbdf_to_pci(struct cxl_prot_er= ror_info *err_info) static void cxl_handle_prot_error(struct cxl_prot_error_info *err_info) { struct pci_dev *pdev __free(pci_dev_put) =3D pci_dev_get(sbdf_to_pci(err_= info)); - struct cxl_dev_state *cxlds =3D pci_get_drvdata(pdev); - struct device *cxlmd_dev __free(put_device) =3D get_device(&cxlds->cxlmd-= >dev); =20 if (!pdev) { pr_err("Failed to find the CXL device\n"); return; } =20 + struct device *host_dev __free(put_device) =3D get_pci_cxl_host_dev(pdev); + if (!host_dev) { + pr_err("Failed to find the CXL device's host\n"); + return; + } + /* * Internal errors of an RCEC indicate an AER error in an * RCH's downstream port. Check and handle them in the CXL.mem @@ -261,6 +330,7 @@ static void cxl_handle_prot_error(struct cxl_prot_error= _info *err_info) return pcie_walk_rcec(pdev, cxl_rch_handle_error_iter, err_info); =20 if (err_info->severity =3D=3D AER_CORRECTABLE) { + struct device *dev =3D &pdev->dev; int aer =3D pdev->aer_cap; =20 if (aer) @@ -268,7 +338,11 @@ static void cxl_handle_prot_error(struct cxl_prot_erro= r_info *err_info) aer + PCI_ERR_COR_STATUS, 0, PCI_ERR_COR_INTERNAL); =20 - cxl_cor_error_detected(&pdev->dev); + if ((pci_pcie_type(pdev) =3D=3D PCI_EXP_TYPE_ENDPOINT) || + (pci_pcie_type(pdev) =3D=3D PCI_EXP_TYPE_RC_END)) + cxl_cor_error_detected(dev); + else + cxl_port_cor_error_detected(dev); =20 pcie_clear_device_status(pdev); } else { diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index 6fd9a42eb304..2c1c00466a25 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -801,6 +801,9 @@ int cxl_endpoint_autoremove(struct cxl_memdev *cxlmd, s= truct cxl_port *endpoint) void cxl_cor_error_detected(struct device *dev); pci_ers_result_t cxl_error_detected(struct device *dev); =20 +void cxl_port_cor_error_detected(struct device *dev); +pci_ers_result_t cxl_port_error_detected(struct device *dev); + /** * struct cxl_endpoint_dvsec_info - Cached DVSEC info * @mem_enabled: cached value of mem_enabled in the DVSEC at init time @@ -915,6 +918,8 @@ void cxl_coordinates_combine(struct access_coordinate *= out, =20 bool cxl_endpoint_decoder_reset_detected(struct cxl_port *port); =20 +int match_uport(struct device *dev, const void *data); + /* * Unit test builds overrides this to __weak, find the 'strong' version * of these symbols in tools/testing/cxl/. diff --git a/drivers/pci/pcie/aer.c b/drivers/pci/pcie/aer.c index 6e88331c6303..5efe5a718960 100644 --- a/drivers/pci/pcie/aer.c +++ b/drivers/pci/pcie/aer.c @@ -1018,11 +1018,6 @@ static bool is_cxl_error(struct pci_dev *pdev, struc= t aer_err_info *info) if (!info || !info->is_cxl) return false; =20 - /* Only CXL Endpoints are currently supported */ - if ((pci_pcie_type(pdev) !=3D PCI_EXP_TYPE_ENDPOINT) && - (pci_pcie_type(pdev) !=3D PCI_EXP_TYPE_RC_EC)) - return false; - return is_internal_error(info); } =20 --=20 2.34.1 From nobody Thu Dec 18 05:05:03 2025 Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on2086.outbound.protection.outlook.com [40.107.223.86]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5553F1E833C; Tue, 3 Jun 2025 17:25:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.223.86 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1748971530; cv=fail; b=JVLHY8K1BC7MboiyM2rwOGOWpDrS9rY5rDYpIohbcGGDTjlvs+LqeWjfW5dU9SasuEtWk30v3B2oEHSxVWS6YggoM3PpEApJ4bCjILYRSAnCyU8+v+yxOD1JB0flaLGzDM3vYlS8wtemHfYEKJ6CrVsy6JWrWcGE2uNJw/fJSbo= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1748971530; c=relaxed/simple; bh=8yCGrYGcmLH9vVi42NF+bZdrjlrvoeLmtbTIncjlxkc=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=I1Kvi4XGiahNFwqiDh+eKyJmZ4sTlbprzH7Z9FH8J8mJCieQgcAVcyh8krabQnJfldTjyWHx4ylMzfetf/u2GnLGXm0f2RuuoQJYkuSNLwWkIokjnjNYLoXtvr9ixZrWmEYfYv9rMQxC7uunKoDQ5lEt6CDae+TFTKKVr86Y3/o= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=dLYs0v5n; arc=fail smtp.client-ip=40.107.223.86 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="dLYs0v5n" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=LMz5zJAdm+MUJP03dJlcyHUmiv5q/2wkK4Sb8yAA8OvdKnWK9h4FL/3tvtobEnW4IdX8FygXoHXODEi6vaMf59FmBNLOIkPIgPBqQ0bprrnUMysdMC4rYEsmz23ul17ayplO1qdVkuUfMcPcgux8RKjrcrWUbg72j02zmB+UMAF91Sp9PsWWmrT1h1MDZTQNufPv4ASMle24ygNm0+ah+/ssOWdpGxfv4D1OgyOfJysW+JnbaUIWyrPNufeSJYDKaPQrvOTLnbjvGZuaOfoIrYQoi0PxB1A7Cp3Woa6J1zZ3sT2LEjqX8V393JsxK+BkGClFMWGMq9sM/82mGzjogg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=WDu+6RLQEiKG04plK7SaifLx1c5fAM+24Bll3AveRvQ=; b=ndk2peRhWzEnWv5aUFVg8jS8jpiCQrFoNzow1FBJZkhUwLEokP4teN9hDWm5L/zcz5ZuHIm8ofxdlvI8CVqT1Zn7LwDsdAUSXtWU1eZXbgp499wg8kp8olekeU2Je/LGTcZveVDalW9IeQ3fAR9z6ZjMz8do8IfgxhNwzsNfCJVS9nc/xOxbr38XSC95dhi8o81/kF2J4PZSxkeBArc/Zmf6r4lU+FEAXs8gHi3jUtCyUlanirJspnuUHj5naYb/0+TY/XcyqoHY+W+mr1w9wUzYBxYIrl1sYAxBokJ97as1kFx1lypU7jLps6Ix11iGhqvI4YhQhot/CKQHfG9q/Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=stgolabs.net smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=WDu+6RLQEiKG04plK7SaifLx1c5fAM+24Bll3AveRvQ=; b=dLYs0v5nfYd5L7UIfBCRY0i2s8mN1IUonbigo1OiFWnx4GXtZTd7P/exKrndjmRziaX9GC/BKgNW0G5OlrbcdNcsF7ykgzBWO36hubRatiPHFgjMh33NP1AfU/MCcCjJN7TJ1FovKMi/xxlXBBzxpwI3VCmBz84ZadbdA4Sm7wc= Received: from BLAPR03CA0114.namprd03.prod.outlook.com (2603:10b6:208:32a::29) by DS2PR12MB9688.namprd12.prod.outlook.com (2603:10b6:8:27d::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8792.34; Tue, 3 Jun 2025 17:25:26 +0000 Received: from BL6PEPF00020E63.namprd04.prod.outlook.com (2603:10b6:208:32a:cafe::c6) by BLAPR03CA0114.outlook.office365.com (2603:10b6:208:32a::29) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8769.19 via Frontend Transport; Tue, 3 Jun 2025 17:25:26 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by BL6PEPF00020E63.mail.protection.outlook.com (10.167.249.24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.8792.29 via Frontend Transport; Tue, 3 Jun 2025 17:25:26 +0000 Received: from ethanolx7ea3host.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Tue, 3 Jun 2025 12:25:22 -0500 From: Terry Bowman To: , , , , , , , , , , , , , , , , , , , , , , , , , , Subject: [PATCH v9 14/16] cxl/pci: Remove unnecessary CXL Endpoint handling helper functions Date: Tue, 3 Jun 2025 12:22:37 -0500 Message-ID: <20250603172239.159260-15-terry.bowman@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250603172239.159260-1-terry.bowman@amd.com> References: <20250603172239.159260-1-terry.bowman@amd.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL6PEPF00020E63:EE_|DS2PR12MB9688:EE_ X-MS-Office365-Filtering-Correlation-Id: c8772dd3-b6d3-46a8-c9ef-08dda2c3a127 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|36860700013|82310400026|1800799024|7416014|376014|921020; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?ZvflKEs11pwaB85MHzCTXCLAbAE7fZXRGn3ksfPSm32iaBewytF0mDPpeCrI?= =?us-ascii?Q?wNaWubE4+9bm2JakopFyRtB08ovKCiFnqhK+U0KrlnDGA0TBRF5nfflkF0Rv?= =?us-ascii?Q?yaniLlxMOhDGLu2a81Po/MXYM9f+rNEDomY3z/uI3EVWTXBQ27MOJf7GAum5?= =?us-ascii?Q?bc+wVjVqv3EEpRsVO6lKd+7+OTbXW50irIiWADeEG0qYvdcuF4Dpo5odV3UB?= =?us-ascii?Q?ZL8nOQmiK0RhJqmTwdocTO1pYOhGTdqOCQ1w58SX3sEtcBZzmts3LwEKOVxd?= =?us-ascii?Q?sMGXbnoAJ1bwj6N7nDF2UzEl8uqsLVrn0G9qymEhAm9IiHU/s8Y1XbCx1kVX?= =?us-ascii?Q?HqTKpLn+erjH3oczjF4p+RtIYbCJKLGNX+12nfA3OeAvnt+2rwGRXDyA7/db?= =?us-ascii?Q?jiLLVfCq0mpvk7jQWr72cPsvjrFvkNk3pLSpU8h/7G4Qc4MDhQIAp4oCesWA?= =?us-ascii?Q?TcXoi1TH+08OQmpBYt9y5iS4LBURH4WvKkPqikjPDIyG7ogaIDcm/D4XinEz?= =?us-ascii?Q?B3vk0+tn6QWQyza0cOpHVIKYHQkBQ8VRMJWD7Ef9rVotkfT9ifw1Zs16GOFK?= =?us-ascii?Q?t3GcJJW7XiN6RfAy5Jv9+eV8wUCszD3JI0mQCIco1tuOkaI1O24HBFYL3Y2I?= =?us-ascii?Q?qIwAYDhKCAiWDzGaBsR2uhowBnvLOXXLE/Jyjy2Ue4NGuwB3lGSZj0aKzwmc?= =?us-ascii?Q?enhwDeQJlM9fcpUQNm5boJ1BIo7HO8JAqy2DHrcjnZL+cMnZTAX3PbzVtXrH?= =?us-ascii?Q?ptjIJTJs8vXBsDTu2mMOao2GgyVge9k09P3QeE1/mjNxIQvvC7h9ftrCDHMO?= =?us-ascii?Q?SLyTAfBy5ReYkU1BMhmgY3EkGkKc5w0DgisYesi3ZueAfHcefa9ojYxYOjBv?= =?us-ascii?Q?/F0/YGwUDLtVGBNIPhZ90CHeD4rJy1Ktudc99nbQaSLUusSUy5uBA+AG6oXW?= =?us-ascii?Q?h5bj2KPa4vCNfhS1dDFBIYxIhHvTXTxB95u8Ge2MP8sFALPFEc/Q5/ogRfsN?= =?us-ascii?Q?hkmnSIj7IgYmNEW0mFKlQR1Rdx47DXWdRndnZC80aUQOLqFK9zpA3ryptN2W?= =?us-ascii?Q?QKRtGMP9on4IYAqE+uNMdp0blgqsRm4N8c1t2YrwOPu0/ZVGwUswrcTHwysi?= =?us-ascii?Q?KBHdG1JFmf2ipQ7GDF+ugBaC3a5OdxeV2nQvfHBrlCPzasrh/IAQ3qZby15/?= =?us-ascii?Q?ngoCwEppSUbuR1SSidsfLelcaFAErKXaP9JIERQWh6r4lPU/njKcdNDWaR1L?= =?us-ascii?Q?+xbikAuXGRpuQsW/jGm7blySLVJaSpOQZ4vm86bdp7t5gi2+TGDJb7jZN4cC?= =?us-ascii?Q?1InLUdu+Zkens6pVlCZPqee6aNEQ02CK02lAX8idOD6l8hVwzzWcI4s1wtdU?= =?us-ascii?Q?jcCch/HIh543zJAw21l01q1VyeDvj6hH1c2Gl5WX2CFjKi5LgsLU3iyhqDlJ?= =?us-ascii?Q?udl5+ESfLN+Qq843cYXI5iR9qg4Kvn5QksAtUMlpkntiCeTGRGEn1RyHK9NO?= =?us-ascii?Q?JgBGryCATfI675ATQBOePl0Y5olgjkBLYuaQCRtA+HawUxu+u16F8zUo0Q?= =?us-ascii?Q?=3D=3D?= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(36860700013)(82310400026)(1800799024)(7416014)(376014)(921020);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Jun 2025 17:25:26.2097 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: c8772dd3-b6d3-46a8-c9ef-08dda2c3a127 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BL6PEPF00020E63.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS2PR12MB9688 Content-Type: text/plain; charset="utf-8" The CXL driver's cxl_handle_endpoint_cor_ras()/cxl_handle_endpoint_ras() are unnecessary helper functions used only for Endpoints. Remove these functions as they are not common for all CXL devices and do not provide value for EP handling. Rename __cxl_handle_ras to cxl_handle_ras() and __cxl_handle_cor_ras() to cxl_handle_cor_ras(). Signed-off-by: Terry Bowman Reviewed-by: Jonathan Cameron Reviewed-by: Kuppuswamy Sathyanarayanan --- drivers/cxl/core/pci.c | 32 ++++++++++++-------------------- 1 file changed, 12 insertions(+), 20 deletions(-) diff --git a/drivers/cxl/core/pci.c b/drivers/cxl/core/pci.c index b6836825e8df..b36a58607041 100644 --- a/drivers/cxl/core/pci.c +++ b/drivers/cxl/core/pci.c @@ -664,8 +664,8 @@ void read_cdat_data(struct cxl_port *port) } EXPORT_SYMBOL_NS_GPL(read_cdat_data, "CXL"); =20 -static void __cxl_handle_cor_ras(struct device *dev, u64 serial, - void __iomem *ras_base) +static void cxl_handle_cor_ras(struct device *dev, u64 serial, + void __iomem *ras_base) { void __iomem *addr; u32 status; @@ -684,11 +684,6 @@ static void __cxl_handle_cor_ras(struct device *dev, u= 64 serial, trace_cxl_aer_correctable_error(dev, serial, status); } =20 -static void cxl_handle_endpoint_cor_ras(struct cxl_dev_state *cxlds) -{ - return __cxl_handle_cor_ras(&cxlds->cxlmd->dev, cxlds->serial, cxlds->reg= s.ras); -} - /* CXL spec rev3.0 8.2.4.16.1 */ static void header_log_copy(void __iomem *ras_base, u32 *log) { @@ -710,8 +705,8 @@ static void header_log_copy(void __iomem *ras_base, u32= *log) * Log the state of the RAS status registers and prepare them to log the * next error status. Return 1 if reset needed. */ -static pci_ers_result_t __cxl_handle_ras(struct device *dev, u64 serial, - void __iomem *ras_base) +static pci_ers_result_t cxl_handle_ras(struct device *dev, u64 serial, + void __iomem *ras_base) { u32 hl[CXL_HEADERLOG_SIZE_U32]; void __iomem *addr; @@ -746,11 +741,6 @@ static pci_ers_result_t __cxl_handle_ras(struct device= *dev, u64 serial, return PCI_ERS_RESULT_PANIC; } =20 -static bool cxl_handle_endpoint_ras(struct cxl_dev_state *cxlds) -{ - return __cxl_handle_ras(&cxlds->cxlmd->dev, cxlds->serial, cxlds->regs.ra= s); -} - #ifdef CONFIG_PCIEAER_CXL =20 static void __iomem *cxl_get_ras_base(struct device *dev) @@ -802,7 +792,7 @@ void cxl_port_cor_error_detected(struct device *dev) { void __iomem *ras_base =3D cxl_get_ras_base(dev); =20 - __cxl_handle_cor_ras(dev, 0, ras_base); + cxl_handle_cor_ras(dev, 0, ras_base); } EXPORT_SYMBOL_NS_GPL(cxl_port_cor_error_detected, "CXL"); =20 @@ -810,20 +800,20 @@ pci_ers_result_t cxl_port_error_detected(struct devic= e *dev) { void __iomem *ras_base =3D cxl_get_ras_base(dev); =20 - return __cxl_handle_ras(dev, 0, ras_base); + return cxl_handle_ras(dev, 0, ras_base); } EXPORT_SYMBOL_NS_GPL(cxl_port_error_detected, "CXL"); =20 static void cxl_handle_rdport_cor_ras(struct cxl_dev_state *cxlds, struct cxl_dport *dport) { - return __cxl_handle_cor_ras(&cxlds->cxlmd->dev, cxlds->serial, dport->reg= s.ras); + return cxl_handle_cor_ras(&cxlds->cxlmd->dev, cxlds->serial, dport->regs.= ras); } =20 static bool cxl_handle_rdport_ras(struct cxl_dev_state *cxlds, struct cxl_dport *dport) { - return __cxl_handle_ras(&cxlds->cxlmd->dev, cxlds->serial, dport->regs.ra= s); + return cxl_handle_ras(&cxlds->cxlmd->dev, cxlds->serial, dport->regs.ras); } =20 /* @@ -921,7 +911,8 @@ void cxl_cor_error_detected(struct device *dev) if (cxlds->rcd) cxl_handle_rdport_errors(cxlds); =20 - cxl_handle_endpoint_cor_ras(cxlds); + cxl_handle_cor_ras(&cxlds->cxlmd->dev, cxlds->serial, + cxlds->regs.ras); } } EXPORT_SYMBOL_NS_GPL(cxl_cor_error_detected, "CXL"); @@ -958,7 +949,8 @@ pci_ers_result_t cxl_error_detected(struct device *dev) * chance the situation is recoverable dump the status of the RAS * capability registers and bounce the active state of the memdev. */ - ue =3D cxl_handle_endpoint_ras(cxlds); + ue =3D cxl_handle_ras(&cxlds->cxlmd->dev, cxlds->serial, + cxlds->regs.ras); } =20 return ue; --=20 2.34.1 From nobody Thu Dec 18 05:05:03 2025 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2073.outbound.protection.outlook.com [40.107.94.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CA5DC1D7E54; Tue, 3 Jun 2025 17:25:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.94.73 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1748971545; cv=fail; b=l9sYGeQBYOg1Yz+oMIyLQk0zOpyTDfa4iuVZY3g7wPzgFn+a62c5zGn+PcKRFbM4/jzY+r+lrfBmK0ZvIXM8iL3JBDbwOtaWb7QZ+9ls/qcXdpGFlMFYoQ+SBO35j1hXZEifr0cDCr/SkpLKqjm8MP7tEAvfRMcgj1NNpi42AEA= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1748971545; c=relaxed/simple; bh=LPden8JADMhQgRRuZMkuW0Uh/k9Q0fRp985UEiUtOQQ=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=MQWgM9wvK51rNXEXxCTr295ejKeA/0GBQU9dcFP1z4PwmpOMA4pPPjTZBBcFl6Uo6TzW3hR1eW2BCZXnVMGiKr7p6Yhzlf90JfYE4POs8RjQ6Qk0rvTBOA8yfU5sGco8MrfPMTF1GSnRyz2jHGTEr5hT+nRS8VgzylAMFdY/kms= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=X/UR5vvg; arc=fail smtp.client-ip=40.107.94.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="X/UR5vvg" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=E6k2PhJOOQ60fAJhYx4DyqmvaM8SwPxogOSzB8BLkHrPr6almtzA8FrfOdtLPZLDO7cFxlr43q36r7wqP0Aau1kupJXmT6JmCZSra8KJfwKU5yiHyZFkOh6+jSTtLV1chibkT5JfqGrb97P4prLIhQCHCOId/fTeH4HafvixGxEoRzXCE5o2dYFCqmVXh1t2puS6t+WOOZvHRTmkAKuB1dPnrKhgjrI7c4KU9dy2N+BYGfdE+SC+s2vFcbt8s/h2cdWxCTancBi7p+pFQuwP5Waj57Wdn6127Q6U/8UlvRKH2sIIjydWrJ6QHx0a2PHF2P6EYQ05C5Iq4UjlWr9UaQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=oNMgicyjTK7CCJPhBNHC/eIeexxphbwly0hlkdN1lIE=; b=e6YYm6yr5V2dgJGCmVlwKEOpxgtF9qW094b2e8WUicDgSUGY4bvh6rxCEe4Fq8ZRPKwASIgpHq1vVI1GQzEjtyqTrqDI6Ilcirs2XT78xZgaAZzxc4uMPPDiUx6hJmE8SOrYJBy6p8+g3valJWOHXZAF9n9t8t8w4xkY6aj/Qtejn2Fk1SoDk6BBP0qam0QN7HTQc2eMzBlm2kTziZdiF6eRZMNGGg9QbsxXFJJXBNx+dASoGR9mNpwS9N1MoXEwxSr/2kzNalLLhiCvVDZJbqw71DXPR5zm/cBVjGY7vhZ9A2WOPeNOnyWnJDOFWBL+siiES/Xyg5hE9ZCMXGVd0g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=stgolabs.net smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=oNMgicyjTK7CCJPhBNHC/eIeexxphbwly0hlkdN1lIE=; b=X/UR5vvgpaObKS6ji0dw8wBCBeWRT3RxGKUlteP+GEEuqKAnoIDdaR18So5inWpYfDFdcvWvsZQq3VmZS9DttIUqpZkUZyix3P0Lw8+Je2EFVUwfulyyUSAvEzFeS2KCdw2CAqv6CZCWLq9vWaO+Eg6VGU0S2lRuWZ2pQmVZtFk= Received: from BN9P223CA0012.NAMP223.PROD.OUTLOOK.COM (2603:10b6:408:10b::17) by DM4PR12MB6040.namprd12.prod.outlook.com (2603:10b6:8:af::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8792.34; Tue, 3 Jun 2025 17:25:36 +0000 Received: from BL6PEPF00020E5F.namprd04.prod.outlook.com (2603:10b6:408:10b:cafe::a8) by BN9P223CA0012.outlook.office365.com (2603:10b6:408:10b::17) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8769.24 via Frontend Transport; Tue, 3 Jun 2025 17:25:36 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by BL6PEPF00020E5F.mail.protection.outlook.com (10.167.249.20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.8792.29 via Frontend Transport; Tue, 3 Jun 2025 17:25:36 +0000 Received: from ethanolx7ea3host.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Tue, 3 Jun 2025 12:25:34 -0500 From: Terry Bowman To: , , , , , , , , , , , , , , , , , , , , , , , , , , Subject: [PATCH v9 15/16] CXL/PCI: Enable CXL protocol errors during CXL Port probe Date: Tue, 3 Jun 2025 12:22:38 -0500 Message-ID: <20250603172239.159260-16-terry.bowman@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250603172239.159260-1-terry.bowman@amd.com> References: <20250603172239.159260-1-terry.bowman@amd.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL6PEPF00020E5F:EE_|DM4PR12MB6040:EE_ X-MS-Office365-Filtering-Correlation-Id: 28e3248e-3ba9-443f-fea0-08dda2c3a71e X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|36860700013|1800799024|82310400026|376014|7416014|921020; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?J81qCrD0gyOrmhqiYVbbMsvy1B+RWhOe7q4cR7+PFjTYV28lclGIF1Q7cQlw?= =?us-ascii?Q?VzgVKtjVTdl2tuYgp/FMiWNLrQ6JWgsZF4l9+bq8TrB0iiaUARiR2BoPUcPq?= =?us-ascii?Q?eBLprEkYgPIOzMIVW8YYlFfb+diM/ZrrlXE+9twIbrVLm/aL+TQt7CKGnh4X?= =?us-ascii?Q?XXZOWSCPwOFsDjVIZN7nc6+QEqdSyEoqGCOIJf3kfk3/CZfMA73/Iyh0Ca19?= =?us-ascii?Q?goQqkUq237TbPwsH8cJsKLZF1ltGDah1IwiU24+P/DHhbqpHfjKuSe83IexM?= =?us-ascii?Q?5lsh03YKKcqpk7Jp+1iMqHX9gqWQ8rwnplTiXy8BIQm6SFp9R0cwamX0aRFB?= =?us-ascii?Q?ou0zp3wkH1UnJPb/owjAIoXWWA/BQG3o1NSpm7eo9EJnDreDrgFvJMww/qT5?= =?us-ascii?Q?W7Qmp3iQVmu1uCwDbryCs+oEwfnQJnkVnKKhpwWYk07GQHErH8PYNE6czA7i?= =?us-ascii?Q?FKyNGgepN0ZzhdmO0AfVAf2AgMI/TYB4YkuAEUHmPfRkbvFu5wtKS2YPlJwh?= =?us-ascii?Q?izb4Ufzw7tJHOvPXBjraEMH6NIiyYVqX48mlsFOAGdD2TrEbMrk4DjPRSMwn?= =?us-ascii?Q?fQPge617eXGBqYl8AY9rYINROi3eDff8XUu7pJgDc16XeZreLimqDwfISMiq?= =?us-ascii?Q?RzSIyaUISi7/tJENjenmEsPS08gbCz/Dfr5GbdPTIvwE9UpXcF5gw7i9KyPe?= =?us-ascii?Q?Le/HxulVwSWJLa+bC0qef9HBntuZAe4cqGJjYcBSAcotMD/mHCuNuavBCMoO?= =?us-ascii?Q?mGSuD1eijsFEXRBXHbYsIaB44dzyllTMLLg6OvBbx9E4eYI1mqUUM3odtXpQ?= =?us-ascii?Q?lOjjP8grlQMNmjU8nYx6Q/M8SDhgq09nvDpwKNTlkXdK9NKOGoEHx+AkvIZq?= =?us-ascii?Q?w8JSBxCKslu0dKIJmDwSFPyuvY+ZnRl8pSeox0/75IBN4Gp3CHfJ4+rZpQPg?= =?us-ascii?Q?UtxwuKAwWeq+NcDMBWx5E0wlT+93yCuvADSpm/s2UvJM+yLwTJfncPAYRHXo?= =?us-ascii?Q?TEsGWhkXag2Wu2FQB3/G9Bm6OBY83C/lhth2IWpTfQ0eLDrD6rMN+wshVJV7?= =?us-ascii?Q?qb/AJ4fykkXMKfiGMYBrl3GF5EruSHVwYUIu6dB7SedCSpOuTL3il5kSpkvv?= =?us-ascii?Q?UkJqnBP0Rs8XGsPy5SvwnC8+fzOp/A4pSRRXxEI3NE5JHaLP00U2Y1qw2f5T?= =?us-ascii?Q?KRoILtgZQbibT5WVV0QrsA8UOaMfsvTtcWgMkdZZd5UPp0XtQoiEAVb4xlmI?= =?us-ascii?Q?HLl1yJFQC7I6Sr8OXG5scpePDyvp0bL/9nRWbuozkcq/aJXtzT5aNCoqyM/j?= =?us-ascii?Q?Ocs1FJYSCyqL8qfg5BE8jq6SOJMQDcwE+5Xz27wE8xCmd74DR1tzPmt70BBs?= =?us-ascii?Q?pORXwurgRvwfO/6M960LIM7TCaKv13ZjtmS+AA+bh4hduvWwG1Lpoz3qKROs?= =?us-ascii?Q?4l3LZKsZHl0i8yrptfWE/ZKEvQnOPBxdQB8HfUK/CIcQnCYWWwTiggZL4YSu?= =?us-ascii?Q?1z92R3EV4F430pYtrzahQs3W9petjDiGMGJjtEW+GJk8KAlS+nCidNeJeQ?= =?us-ascii?Q?=3D=3D?= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(36860700013)(1800799024)(82310400026)(376014)(7416014)(921020);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Jun 2025 17:25:36.2174 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 28e3248e-3ba9-443f-fea0-08dda2c3a71e X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BL6PEPF00020E5F.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB6040 Content-Type: text/plain; charset="utf-8" CXL protocol errors are not enabled for all CXL devices after boot. These must be enabled inorder to process CXL protocol errors. Export the AER service driver's pci_aer_unmask_internal_errors(). Introduce cxl_unmask_prot_interrupts() to call pci_aer_unmask_internal_erro= rs(). pci_aer_unmask_internal_errors() expects the pdev->aer_cap is initialized. But, dev->aer_cap is not initialized for CXL Upstream Switch Ports and CXL Downstream Switch Ports. Initialize the dev->aer_cap if necessary. Enable A= ER correctable internal errors and uncorrectable internal errors for all CXL devices. Signed-off-by: Terry Bowman Reviewed-by: Jonathan Cameron Reviewed-by: Kuppuswamy Sathyanarayanan --- drivers/cxl/port.c | 29 +++++++++++++++++++++++++++-- drivers/pci/pcie/aer.c | 3 ++- include/linux/aer.h | 1 + 3 files changed, 30 insertions(+), 3 deletions(-) diff --git a/drivers/cxl/port.c b/drivers/cxl/port.c index 0f7c4010ba58..3687848ae772 100644 --- a/drivers/cxl/port.c +++ b/drivers/cxl/port.c @@ -3,6 +3,7 @@ #include #include #include +#include =20 #include "cxlmem.h" #include "cxlpci.h" @@ -60,6 +61,21 @@ static int discover_region(struct device *dev, void *unu= sed) =20 #ifdef CONFIG_PCIEAER_CXL =20 +static void cxl_unmask_prot_interrupts(struct device *dev) +{ + struct pci_dev *pdev __free(pci_dev_put) =3D + pci_dev_get(to_pci_dev(dev)); + + if (!pdev->aer_cap) { + pdev->aer_cap =3D pci_find_ext_capability(pdev, + PCI_EXT_CAP_ID_ERR); + if (!pdev->aer_cap) + return; + } + + pci_aer_unmask_internal_errors(pdev); +} + static void cxl_dport_map_rch_aer(struct cxl_dport *dport) { resource_size_t aer_phys; @@ -118,8 +134,12 @@ static void cxl_uport_init_ras_reporting(struct cxl_po= rt *port, =20 map->host =3D host; if (cxl_map_component_regs(map, &port->uport_regs, - BIT(CXL_CM_CAP_CAP_ID_RAS))) + BIT(CXL_CM_CAP_CAP_ID_RAS))) { dev_dbg(&port->dev, "Failed to map RAS capability\n"); + return; + } + + cxl_unmask_prot_interrupts(port->uport_dev); } =20 /** @@ -144,9 +164,12 @@ void cxl_dport_init_ras_reporting(struct cxl_dport *dp= ort, struct device *host) } =20 if (cxl_map_component_regs(&dport->reg_map, &dport->regs.component, - BIT(CXL_CM_CAP_CAP_ID_RAS))) + BIT(CXL_CM_CAP_CAP_ID_RAS))) { dev_dbg(dport->dport_dev, "Failed to map RAS capability\n"); + return; + } =20 + cxl_unmask_prot_interrupts(dport->dport_dev); } EXPORT_SYMBOL_NS_GPL(cxl_dport_init_ras_reporting, "CXL"); =20 @@ -177,6 +200,8 @@ static void cxl_endpoint_port_init_ras(struct cxl_port = *port) } =20 cxl_dport_init_ras_reporting(dport, &cxlmd->dev); + + cxl_unmask_prot_interrupts(cxlmd->cxlds->dev); } =20 #else diff --git a/drivers/pci/pcie/aer.c b/drivers/pci/pcie/aer.c index 5efe5a718960..2d202ad1453a 100644 --- a/drivers/pci/pcie/aer.c +++ b/drivers/pci/pcie/aer.c @@ -964,7 +964,7 @@ static bool find_source_device(struct pci_dev *parent, * Note: AER must be enabled and supported by the device which must be * checked in advance, e.g. with pcie_aer_is_native(). */ -static void pci_aer_unmask_internal_errors(struct pci_dev *dev) +void pci_aer_unmask_internal_errors(struct pci_dev *dev) { int aer =3D dev->aer_cap; u32 mask; @@ -977,6 +977,7 @@ static void pci_aer_unmask_internal_errors(struct pci_d= ev *dev) mask &=3D ~PCI_ERR_COR_INTERNAL; pci_write_config_dword(dev, aer + PCI_ERR_COR_MASK, mask); } +EXPORT_SYMBOL_NS_GPL(pci_aer_unmask_internal_errors, "CXL"); =20 static bool is_cxl_mem_dev(struct pci_dev *dev) { diff --git a/include/linux/aer.h b/include/linux/aer.h index c9a18eca16f8..74600e75705f 100644 --- a/include/linux/aer.h +++ b/include/linux/aer.h @@ -107,5 +107,6 @@ void pci_print_aer(struct pci_dev *dev, int aer_severit= y, int cper_severity_to_aer(int cper_severity); void aer_recover_queue(int domain, unsigned int bus, unsigned int devfn, int severity, struct aer_capability_regs *aer_regs); +void pci_aer_unmask_internal_errors(struct pci_dev *dev); #endif //_AER_H_ =20 --=20 2.34.1 From nobody Thu Dec 18 05:05:03 2025 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2064.outbound.protection.outlook.com [40.107.93.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 846E423C8BE; Tue, 3 Jun 2025 17:25:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.93.64 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1748971552; cv=fail; b=l12rbJAuLpJm4yVTkdACsfKjyu3tdwIdmsmMt/Gll31L87Ryf++ztyaxWBYtkxeJwucd2jcNGSpuwtLiJySNwh0mvHfUqDQ1SQJuZjUs0jN6iqJ3irdgIWEKHEsxYYgT5kN//bIZBUEoKd8YmWfU1PVNB5LqPiF3txXgB/kV+To= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1748971552; c=relaxed/simple; bh=gRUkO5oviJDwNNl9/+IAHfExVxAKXOZ6b6rGmTwm174=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=rDv1WIuNkTifWZC/XaMJa8x5dYIAW+7yyicYy1ay7w8YRlFcnUkrzGXGu8Ug0JvibbIY+5Shdkl40Q6rBT6vr8jpD1uukXkCCZgtEGAkVXqkrBoZiwheheTh3OZIq3zmtzJ+gUa++wafuRIw9KfsTlOE5A/4ZvxVsChnpHggn04= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=jtC7K3Ge; arc=fail smtp.client-ip=40.107.93.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="jtC7K3Ge" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=kZNDZJZgRCNcvKGl7sRdVruN/uv7J/qTjxNCU8z0B+b4031TyI0aHIPuSB7Hdg9gLkuowkfleRBkZCPn5bgt5qLismumcorrYs7Im1BmAGmHEwgphVziTx7GHNAgo6Bv+6cwnZOo62CvxWucbDUm59mr78qmzz1hs+I1gLbRUSZDRsMf7FOZ+q/z6qfoPj4SUsCGrVSuVPLhBexj3vpPTxylMILvUUn9m4/WSuxVHWbfqDP8cFZsIS5SC1aZMGrLV9xXH1RnMtm2qtqrEEJmn3QSJWe7dEVzeeHUc9t37KitwiP5PSxH/Eu1zDDfySPqVj76nDm2ASFTEbcckMuibQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=2Sp1cLMGjpVBT+T+z3A/NkgYK82ptv1cbgcPyB0pQ/8=; b=SgDeT8Dc4P5V2YcpMz7p6jnbpsVNCJRG/R0J2NHRhfgNM/E5FWnHuvOkUu/My3mlti/FeYUPW7Z7qmJAzV6eFLEXFBgCQzNsJEuIjL2TFqUylt7I//7cVFjPv8BODPsMet7DulWRRgZpcNI7d3lYBW2ocvqfeSXweZxoe9SZMh6CETSst230fP4Ki3XNdEAphRw2nT0OFnHTc6KniJmyO1s/Tuwry1RToLfp+dO3DChD+5+N4ZUAxuHVdYRQ3FIsCu9v3cmGitH3hI6MvGNwXDn+b7j2gCfksTuH8M0X5SxOn0b24ECCndxNeTdk1GBuKXU1MCoMsn+tXLxDSFgmmA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=stgolabs.net smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=2Sp1cLMGjpVBT+T+z3A/NkgYK82ptv1cbgcPyB0pQ/8=; b=jtC7K3GetCtEOTxeL8dT7TCFcEqvo+P2wlIFKWaGfFjc52kMF2X68eQfpynEQaDUIz7QJLmOzzqYuYNgEmsy23Qe5cb61dzEhJoNH5sDY8tdeXF8QAZkzX8H/TXAUj7Ryozoc1ecW4TR2zjHVujqtbV+03NSKmmOULU+sMUy4pQ= Received: from BL1PR13CA0208.namprd13.prod.outlook.com (2603:10b6:208:2be::33) by CY8PR12MB7315.namprd12.prod.outlook.com (2603:10b6:930:51::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8769.30; Tue, 3 Jun 2025 17:25:46 +0000 Received: from BL6PEPF00020E61.namprd04.prod.outlook.com (2603:10b6:208:2be:cafe::a4) by BL1PR13CA0208.outlook.office365.com (2603:10b6:208:2be::33) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8769.17 via Frontend Transport; Tue, 3 Jun 2025 17:25:46 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by BL6PEPF00020E61.mail.protection.outlook.com (10.167.249.22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.8792.29 via Frontend Transport; Tue, 3 Jun 2025 17:25:46 +0000 Received: from ethanolx7ea3host.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Tue, 3 Jun 2025 12:25:45 -0500 From: Terry Bowman To: , , , , , , , , , , , , , , , , , , , , , , , , , , Subject: [PATCH v9 16/16] CXL/PCI: Disable CXL protocol error interrupts during CXL Port cleanup Date: Tue, 3 Jun 2025 12:22:39 -0500 Message-ID: <20250603172239.159260-17-terry.bowman@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250603172239.159260-1-terry.bowman@amd.com> References: <20250603172239.159260-1-terry.bowman@amd.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL6PEPF00020E61:EE_|CY8PR12MB7315:EE_ X-MS-Office365-Filtering-Correlation-Id: 881aff2a-ced9-4a04-aa76-08dda2c3ad48 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|376014|7416014|36860700013|1800799024|921020; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?7ClvlW14cBoAES73kOp3L8RPwqPz//tOqQzvlchh7DSj9EeSCLd36cK6Vw+D?= =?us-ascii?Q?Pvmhf/4VVKRQvQuPHAMqxrVDwf03VFiqjLg6c7W+UzH2RtVAa9SdbW9nUgc0?= =?us-ascii?Q?q77rf6m99sestHu/wLIBxmZWt2WCCREmjWPE+sK7GsBESCEAIHD/yvq5+d1p?= =?us-ascii?Q?ysIVlCglkPHNIIPjBPplgzbxHkWa6mpNJXSyvv7aciADje4oo3smwFEOKvC+?= =?us-ascii?Q?5tAmIFgtoLzH+VgZ4o8Xl+W7zM4BBp1s8IxWGZ7/+dF9PyjhhdueEfTqNg4z?= =?us-ascii?Q?LZ69TAJYBWNW4PIlkRn5plvG4CffGp5ZtLlCYRiXly73Z+XYe26Vvkd1i3B2?= =?us-ascii?Q?qjzGezFq5rC5W59NOHWCuMMTKtDnUye3kxtT7FFvFw94XK0uKNoOZI+VVtPo?= =?us-ascii?Q?ea7C0o+TvZ4rzpu55MvKJgeEPJDhjGld++guOZ9LQBUZZ66BX5n+rTdcAtzG?= =?us-ascii?Q?V5NnDmy+0+glK74fe+ZZMostNnm70U881MkAblSmDDs712GcXC0tHZ1M73i3?= =?us-ascii?Q?Ojh4Us4oPju3UuUs5qmMHkxj0bpQtGZo6GbhPKfJCpCr1nwne+K/WSQiXUSQ?= =?us-ascii?Q?P79f7OxsA1hOflHv6f1QJtvEbPzcBYi7EG3AI1NlQZ5iSRVolF08oe++Jv/G?= =?us-ascii?Q?+dYHh8iR50rA/KQs673+ajTZfvdLjUpp3DM+z0ZQ4j7PjVh0wp4U0CXqbrHo?= =?us-ascii?Q?kzfibZCp756OucMBF+8NP4CiUUlg6XLZyOsZB8imwUM1Ezp/cOiIjr4SFNlY?= =?us-ascii?Q?qnyT8eWIRjw6dpXC/9tSr9mEVJXVDkrh4egQPduvhA3vTHigbMwJgV5Gs164?= =?us-ascii?Q?aNb33WN7PYvIvtlCjgTTUqhLw6woNmk8xBzr+lgVkzOn02BoAg8VC0JZO+09?= =?us-ascii?Q?SPpPMiP18Y26Xlq117gBO4c1zV5WXQ2fLA0/M7iDEdrYjx3UUEsR3MfI2qO8?= =?us-ascii?Q?jYIhdX7Vn8YQUJ80fNFSODnrhAzZH71qP75pyJjvl2U2s4tOnt544+nYtb2D?= =?us-ascii?Q?ZSqUPzY0Nm3Rgb6XGqIfjLgD+CHUmGfTehvj12CiiXlzmUFl3yE1BkMAAXhu?= =?us-ascii?Q?cxE0GsvfOkpM+8u8XfwmsnXKzUtkZ/ylN7UbSEs+9lDrFqK9vryPbDbnGjlZ?= =?us-ascii?Q?Lh6Da43e+V82h5osJPH26Ur5WMbO06pY/Z9lMuDZ//ljtg7jlI+dQ7EzOyY+?= =?us-ascii?Q?HUP88d66Ftl7GVQ8FPmFh0kPyuiJa7txp2hWh+vp0FZM2u3hYK30Dj3zioM+?= =?us-ascii?Q?Nb7quY1NIBvThlezhyuBIygSyxeqXuPN6cL7Y8kwyWyYo0hWKd8QbTUckY1J?= =?us-ascii?Q?P/UquwDAgTAMvJThJWm14v1T7EFkVmaOiUAmJvN00Yeo0NZ06tQq7gPB2VBi?= =?us-ascii?Q?uQnNK802s19vJCFSncT3TKhv/U2DjK2qPfnUCMPMXB77Mua9cnvjlhJ/UiAc?= =?us-ascii?Q?EJ44EhHN/uAFuxD4ePAd88CwzLlh/9Jm7dO/7nceKZsBBIXpL9bXTgq+ObO8?= =?us-ascii?Q?Pho/ANfkrd8YXMSvleSBOheCjYVe2hExzqdR6GbnZZ7ZzpXZyIMWe1+1Ew?= =?us-ascii?Q?=3D=3D?= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(82310400026)(376014)(7416014)(36860700013)(1800799024)(921020);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Jun 2025 17:25:46.5567 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 881aff2a-ced9-4a04-aa76-08dda2c3ad48 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BL6PEPF00020E61.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR12MB7315 Content-Type: text/plain; charset="utf-8" During CXL device cleanup the CXL PCIe Port device interrupts remain enabled. This potentially allows unnecessary interrupt processing on behalf of the CXL errors while the device is destroyed. Disable CXL protocol errors by setting the CXL devices' AER mask register. Introduce pci_aer_mask_internal_errors() similar to pci_aer_unmask_internal= _errors(). Introduce cxl_mask_prot_interrupts() to call pci_aer_mask_internal_errors(). Add calls to cxl_mask_prot_interrupts() within CXL Port teardown for CXL Root Ports, CXL Downstream Switch Ports, CXL Upstream Switch Ports, and CXL Endpoints. Follow the same "bottom-up" approach used during CXL Port teardown. Implement cxl_mask_prot_interrupts() in a header file to avoid introducing Kconfig ifdefs in cxl/core/port.c. Signed-off-by: Terry Bowman Reviewed-by: Dave Jiang Reviewed-by: Jonathan Cameron Reviewed-by: Kuppuswamy Sathyanarayanan --- drivers/cxl/core/port.c | 6 ++++++ drivers/cxl/cxl.h | 8 ++++++++ drivers/pci/pcie/aer.c | 21 +++++++++++++++++++++ include/linux/aer.h | 1 + 4 files changed, 36 insertions(+) diff --git a/drivers/cxl/core/port.c b/drivers/cxl/core/port.c index 07b9bb0f601f..6aaaad002a7f 100644 --- a/drivers/cxl/core/port.c +++ b/drivers/cxl/core/port.c @@ -1433,6 +1433,9 @@ EXPORT_SYMBOL_NS_GPL(cxl_endpoint_autoremove, "CXL"); */ static void delete_switch_port(struct cxl_port *port) { + cxl_mask_prot_interrupts(port->uport_dev); + cxl_mask_prot_interrupts(port->parent_dport->dport_dev); + devm_release_action(port->dev.parent, cxl_unlink_parent_dport, port); devm_release_action(port->dev.parent, cxl_unlink_uport, port); devm_release_action(port->dev.parent, unregister_port, port); @@ -1446,6 +1449,7 @@ static void reap_dports(struct cxl_port *port) device_lock_assert(&port->dev); =20 xa_for_each(&port->dports, index, dport) { + cxl_mask_prot_interrupts(dport->dport_dev); devm_release_action(&port->dev, cxl_dport_unlink, dport); devm_release_action(&port->dev, cxl_dport_remove, dport); devm_kfree(&port->dev, dport); @@ -1476,6 +1480,8 @@ static void cxl_detach_ep(void *data) { struct cxl_memdev *cxlmd =3D data; =20 + cxl_mask_prot_interrupts(cxlmd->cxlds->dev); + for (int i =3D cxlmd->depth - 1; i >=3D 1; i--) { struct cxl_port *port, *parent_port; struct detach_ctx ctx =3D { diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index 2c1c00466a25..2753db3d473e 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -12,6 +12,7 @@ #include #include #include +#include =20 extern const struct nvdimm_security_ops *cxl_security_ops; =20 @@ -771,9 +772,16 @@ struct cxl_dport *devm_cxl_add_rch_dport(struct cxl_po= rt *port, #ifdef CONFIG_PCIEAER_CXL void cxl_setup_parent_dport(struct device *host, struct cxl_dport *dport); void cxl_dport_init_ras_reporting(struct cxl_dport *dport, struct device *= host); +static inline void cxl_mask_prot_interrupts(struct device *dev) +{ + struct pci_dev *pdev __free(pci_dev_put) =3D pci_dev_get(to_pci_dev(dev)); + + pci_aer_mask_internal_errors(pdev); +} #else static inline void cxl_dport_init_ras_reporting(struct cxl_dport *dport, struct device *host) { } +static inline void cxl_mask_prot_interrupts(struct device *dev) { } #endif =20 struct cxl_decoder *to_cxl_decoder(struct device *dev); diff --git a/drivers/pci/pcie/aer.c b/drivers/pci/pcie/aer.c index 2d202ad1453a..69230cf87d79 100644 --- a/drivers/pci/pcie/aer.c +++ b/drivers/pci/pcie/aer.c @@ -979,6 +979,27 @@ void pci_aer_unmask_internal_errors(struct pci_dev *de= v) } EXPORT_SYMBOL_NS_GPL(pci_aer_unmask_internal_errors, "CXL"); =20 +/** + * pci_aer_mask_internal_errors - mask internal errors + * @dev: pointer to the pcie_dev data structure + * + * Masks internal errors in the Uncorrectable and Correctable Error + * Mask registers. + * + * Note: AER must be enabled and supported by the device which must be + * checked in advance, e.g. with pcie_aer_is_native(). + */ +void pci_aer_mask_internal_errors(struct pci_dev *dev) +{ + int aer =3D dev->aer_cap; + + pci_clear_and_set_config_dword(dev, aer + PCI_ERR_UNCOR_MASK, + 0, PCI_ERR_UNC_INTN); + pci_clear_and_set_config_dword(dev, aer + PCI_ERR_COR_MASK, + 0, PCI_ERR_COR_INTERNAL); +} +EXPORT_SYMBOL_NS_GPL(pci_aer_mask_internal_errors, "CXL"); + static bool is_cxl_mem_dev(struct pci_dev *dev) { /* diff --git a/include/linux/aer.h b/include/linux/aer.h index 74600e75705f..41167ad3797a 100644 --- a/include/linux/aer.h +++ b/include/linux/aer.h @@ -108,5 +108,6 @@ int cper_severity_to_aer(int cper_severity); void aer_recover_queue(int domain, unsigned int bus, unsigned int devfn, int severity, struct aer_capability_regs *aer_regs); void pci_aer_unmask_internal_errors(struct pci_dev *dev); +void pci_aer_mask_internal_errors(struct pci_dev *dev); #endif //_AER_H_ =20 --=20 2.34.1