From nobody Mon Apr 6 21:31:00 2026 Received: from SJ2PR03CU001.outbound.protection.outlook.com (mail-westusazon11012062.outbound.protection.outlook.com [52.101.43.62]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D332F19AD8B; Tue, 17 Mar 2026 19:16:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.43.62 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773774996; cv=fail; b=kS7iJSVaf53aZXYvuJ09Gh5ZK0sIXoPX6eOSDn5USvYVPQIEyCXcgvsWMuLG5Ppqbn7tFVYzavbSvqi3trX/pWLCsoUqY+PiTJv71ZCdXMfNNyHm5X+Cgkq92dsFSNY8rroyle2uxurO8rwql5TzehAXPJgh0h1qALCjk6PrUrE= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773774996; c=relaxed/simple; bh=XBrN7lG6xJGay0JwkA0maAxk3vmXTdtXfRbMOyhF5qg=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=P4G/WLCekyAGjsVm5eYTUBa/UUgSn8u1zr7Z9Q6KiIp+E3aZh+X3npXkNYSwKv2FegAyCidc6lR0t1eCqMfz/ktQOU40eep4s/YtWiBbjFo+IYAb58bF0ubpdc/pFM3BTAPyD+X8Q31w24Zn42D0dACJAB1rDJujugM+psBZXcA= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=SThumL/s; arc=fail smtp.client-ip=52.101.43.62 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="SThumL/s" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=f+IZriv45OT2HK5OKZFRb4kFmcAk9Yj3UCwqO31xG39G/+4nATnYRB7jKRzhk82bppwAGV+C+an+7+W5r/ZsqJzZOkxoCKucUOEQUdFXw9eVnvUH0Z4BBltn7GjgiPYDygO78GE6/FF6hsfeABhlhC8wpt8py5wBqm74IpBa4zY4IuX936ts4yx4ZEmbASC7kluUB7XjtmJlfqA2Kizyci6Ka2VkHEblCBypZ/F+x03qxMwIYRm86ZlgO2ssbBs0XWCAroHkkV/RBzGYHZw0qcGhWcjZb+zD8fuzjnw61pg2AX1Rxm7Y6LDT//eXc8foLkmKSlfHXcievGPUxGcHZw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=lP2sQGIIcHZGt2MelZ+hkUOiEKCYZtquuB0qBQV5X58=; b=Gpmdc3E/ngTikgpnLXDxQ6iOIvASvu9c/ZBOSZqNa7SSxicukImUf86bnHnfuFUNqudRPQQ/bZm4Dy7xHCGahDMGfeUC9qHO8HClw9TcZY3TKOSIw5BpqxUvJYUXZSIqBDpjq02ydbGSsHuR5ADDVb1G+oKh3HEN9iF7avIGXNNGU0y5BZpEZHoJCuZ235PQ2WOECBd8aeJHzdNNXtvcgDckzim7G2v6ig4n4xZA2V30MQZSS9dL98vzSDfSx+tzoWtl5G30nNYQZ4kz/jtlDK/AWnx9i0IhkzHuJGAgYL+/wXunbx3J11dds7eXDji5R90VbN3hIR9Lf9AATP/HZA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=lP2sQGIIcHZGt2MelZ+hkUOiEKCYZtquuB0qBQV5X58=; b=SThumL/sbk4XYPiriQD5MgFLbtN+mAKwCH67PPWLCQ0QAgvXgnjyAl13nQxCEAQ11pe8RZtuNV7oXqWoNGXtXYQnMUb3vlivegFVOpRLBeng+5MXgHcLimLGKXWzwVt2aCIoZHRJJP6R+H6XMbPkdoUeQx0dqg2izlCD+FlDWJkBmiHmtLkTr0U8djpnoop6hsjXIXxHbFlonMGVv/kEQ/wh8K6QhIRZ1xSMhPHRKtWor3Mauwrcs9kiO7HTrojGbKVBoRFaYhKO/zLksNYPVi6whhfUDk4feRPGFS2aPVhH+MnurV98p8NEg3xwgu4d5+JNk176RfhsP6D8DqSsPQ== Received: from PH0PR07CA0115.namprd07.prod.outlook.com (2603:10b6:510:4::30) by IA1PR12MB6018.namprd12.prod.outlook.com (2603:10b6:208:3d6::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9723.19; Tue, 17 Mar 2026 19:16:27 +0000 Received: from MWH0EPF000A6735.namprd04.prod.outlook.com (2603:10b6:510:4:cafe::9a) by PH0PR07CA0115.outlook.office365.com (2603:10b6:510:4::30) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9700.25 via Frontend Transport; Tue, 17 Mar 2026 19:15:57 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by MWH0EPF000A6735.mail.protection.outlook.com (10.167.249.27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9700.17 via Frontend Transport; Tue, 17 Mar 2026 19:16:27 +0000 Received: from rnnvmail203.nvidia.com (10.129.68.9) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 17 Mar 2026 12:16:03 -0700 Received: from rnnvmail204.nvidia.com (10.129.68.6) by rnnvmail203.nvidia.com (10.129.68.9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 17 Mar 2026 12:16:03 -0700 Received: from Asurada-Nvidia.nvidia.com (10.127.8.9) by mail.nvidia.com (10.129.68.6) with Microsoft SMTP Server id 15.2.2562.20 via Frontend Transport; Tue, 17 Mar 2026 12:16:02 -0700 From: Nicolin Chen To: , , , , CC: , , , , , , , , , , , Subject: [PATCH v2 1/7] iommu: Do not call pci_dev_reset_iommu_done() unless reset succeeds Date: Tue, 17 Mar 2026 12:15:34 -0700 Message-ID: <21fa71d59c8ab787c0f3d8cf3e9fc725330fd0d5.1773774441.git.nicolinc@nvidia.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MWH0EPF000A6735:EE_|IA1PR12MB6018:EE_ X-MS-Office365-Filtering-Correlation-Id: 18ce1c58-e08d-43eb-3f9b-08de8459b016 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|82310400026|7416014|36860700016|376014|18002099003|56012099003|22082099003; X-Microsoft-Antispam-Message-Info: oqsrSOtyGGWTAR5HOoGinwA+rlOsYrvqCHS3jRRTdeeo8mM14qu9tApLm+4Tsz4s0bY/viZ6oPbm61+NRAo9jtycYB8cMDHHebZVWniIu/+cs605AJA/LCx6HQ4vVWCIMTN7/va2gNW698diQlqc7wjvs6fleToORZ9v3wECbFOiQ0N4Tsk1Z9x71PtKfXkLjaGHokCmk/1aggf6xZjY5j9vdmlWIV18sovGCa6VpH7GG9WCttJOtUTI34D4X2N5EyQiyCdnmkbUiDvxe/moDhgjz6JtzlXHMfGJsaMZVcnm7+L/G7Y4ftlbk9xtTinYAumYyUqyzsdX4p5qtKXw3FsmqLF6BgbNlYl9qp42YkpNo/ZB0aNQWTZhIGCju2M55A5dbFW6igd8Ek9oo38uOP6Sim3JTeomNCouFuh0N+9ZbyFjQr/tXUan2CbJ/0iuNEwl5ybSAyK/QVTBRbKMzPWcx08HQUvcox9iMiD4MIKZbClffjqeziUuxVhFIFErHE0Zgp13hM8YvZOy9PdCEOQeCwIlL0Ue0RqWFjuJJDOMQanLFHE2oZWU7nRNC6NKboGlYpnaxjBVEKPS8ZnW4gmu07W1ldVCNoEa9PL/RFcSS306Bv/W7+fWTtOFhOwvuRl9fceHY5XHmApOKPdUP9hdoyoKxrMWfARphGYA5zn+7b07RYIwp0pWkXgKhOKmAJFDLL2c2y779ggHZXOHgJ66m1IyayW1nXRRwPM4xo9scPbS9XR6Hdb1JvBWfhXFpsog/moyufzw3DK0SEri0w== X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(1800799024)(82310400026)(7416014)(36860700016)(376014)(18002099003)(56012099003)(22082099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: qKFLFf26oqUaGKPLrAdCwlLCI04clzqIqHkGutWbukBAqI7SBHV/Ikl4t1uJeDXtaDNhYpon3hSpDzp2yUQ0Puoq/zjtc/f/wH+bqk02mmNepCfOGmiwK4eWAIiBiC6qJu6P8Ad0SvwHWtA1EMAnTgdeasryoWeWCHGiPGA/cIyYzf+UPjW5oXiSiW9zpqGHIcHbaXtbQ/QAckHbHq/39Q5RsgvjhC5Z1QRj1vF1vA+Zi0wLWrlST2T7n5LnPZfR6wYnJ//hprC8AHPd1R8Kjd0AXRNnIETV+f9jr8jfIZBuDymH9gUFThcjlGlbYtGXWvpwJHMSibsoqaB9Y0iPheD4nd4BSymNudutACE3t4y/C1Uw252Sd7fFvH5cZfol9Fo0oBwsIp2X+lG4nT7ltfy/Iw859J8I/Rgo7B+CGzkEBQvoxXF1173an2e/pgJS X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Mar 2026 19:16:27.4026 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 18ce1c58-e08d-43eb-3f9b-08de8459b016 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: MWH0EPF000A6735.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB6018 Content-Type: text/plain; charset="utf-8" IOMMU drivers handle ATC cache maintenance. They may encounter ATC-related errors (e.g., ATC invalidation request timeout), indicating that ATC cache might have stale entries that can corrupt the memory. In this case, IOMMU driver has no choice but to block the device's ATS function and wait for a device recovery. The pci_dev_reset_iommu_done() called at the end of a reset function could serve as a reliable signal to the IOMMU subsystem that the physical device cache is completely clean. However, the function is called unconditionally even if the reset operation had actually failed, which would re-attach the faulty device back to a normal translation domain. And this will leave the system highly exposed, creating vulnerabilities for data corruption: IOMMU blocks RID/ATS pci_reset_function(): pci_dev_reset_iommu_prepare(); // Block RID/ATS __reset(); // Failed (ATC is still stale) pci_dev_reset_iommu_done(); // Unblock RID/ATS (ah-ha) The simplest fix is to use pci_dev_reset_iommu_done() only on a successful reset: IOMMU blocks RID/ATS pci_reset_function(): pci_dev_reset_iommu_prepare(); // Block RID/ATS if (!__reset()) pci_dev_reset_iommu_done(); // Unblock RID/ATS else // Keep the device blocked by IOMMU However, this breaks the symmetric requirement of these reset APIs so that we have to allow a re-entry to pass a second reset attempt: IOMMU blocks RID/ATS pci_reset_function(): pci_dev_reset_iommu_prepare(); // Block RID/ATS __reset(); // Failed (ATC is still stale) // Keep the device blocked by IOMMU Second reset: pci_reset_function(): pci_dev_reset_iommu_prepare(); // Re-entry (!) Update the function kdocs and all the existing callers to only unblock ATS when the reset succeeds. Drop the WARN_ON in pci_dev_reset_iommu_prepare() to allow re-entries. Signed-off-by: Nicolin Chen --- drivers/iommu/iommu.c | 16 +++++++++----- drivers/pci/pci-acpi.c | 11 +++++++++- drivers/pci/pci.c | 50 +++++++++++++++++++++++++++++++++++++----- drivers/pci/quirks.c | 11 +++++++++- 4 files changed, 75 insertions(+), 13 deletions(-) diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c index 35db517809540..40a15c9360bd1 100644 --- a/drivers/iommu/iommu.c +++ b/drivers/iommu/iommu.c @@ -3938,8 +3938,10 @@ EXPORT_SYMBOL_NS_GPL(iommu_replace_group_handle, "IO= MMUFD_INTERNAL"); * IOMMU activity while leaving the group->domain pointer intact. Later wh= en the * reset is finished, pci_dev_reset_iommu_done() can restore everything. * - * Caller must use pci_dev_reset_iommu_prepare() with pci_dev_reset_iommu_= done() - * before/after the core-level reset routine, to unset the resetting_domai= n. + * Caller must use pci_dev_reset_iommu_done() after a successful PCI-level= reset + * to unset the resetting_domain. If the reset fails, caller can choose to= keep + * the device in the resetting_domain to protect system memory using IOMMU= from + * any bad ATS. * * Return: 0 on success or negative error code if the preparation failed. * @@ -3961,9 +3963,9 @@ int pci_dev_reset_iommu_prepare(struct pci_dev *pdev) =20 guard(mutex)(&group->mutex); =20 - /* Re-entry is not allowed */ - if (WARN_ON(group->resetting_domain)) - return -EBUSY; + /* Already prepared */ + if (group->resetting_domain) + return 0; =20 ret =3D __iommu_group_alloc_blocking_domain(group); if (ret) @@ -4001,7 +4003,9 @@ EXPORT_SYMBOL_GPL(pci_dev_reset_iommu_prepare); * re-attaching all RID/PASID of the device's back to the domains retained= in * the core-level structure. * - * Caller must pair it with a successful pci_dev_reset_iommu_prepare(). + * This is a pairing function for pci_dev_reset_iommu_prepare(). Caller sh= ould + * use it on a successful PCI-level reset. Otherwise, it's suggested for c= aller + * to keep the device in the resetting_domain to protect system memory. * * Note that, although unlikely, there is a risk that re-attaching domains= might * fail due to some unexpected happening like OOM. diff --git a/drivers/pci/pci-acpi.c b/drivers/pci/pci-acpi.c index 4d0f2cb6c695b..f1a918938242c 100644 --- a/drivers/pci/pci-acpi.c +++ b/drivers/pci/pci-acpi.c @@ -16,6 +16,7 @@ #include #include #include +#include #include #include #include @@ -977,7 +978,15 @@ int pci_dev_acpi_reset(struct pci_dev *dev, bool probe) ret =3D -ENOTTY; } =20 - pci_dev_reset_iommu_done(dev); + /* + * The reset might be invoked to recover a serious error. E.g. when the + * ATC failed to invalidate its stale entries, which can result in data + * corruption. Thus, do not unblock ATS until a successful reset. + */ + if (!ret || !pci_ats_supported(dev)) + pci_dev_reset_iommu_done(dev); + else + pci_warn(dev, "Reset failed. Blocking ATS to protect memory\n"); return ret; } =20 diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c index 8479c2e1f74f1..80c5cf6eeebdc 100644 --- a/drivers/pci/pci.c +++ b/drivers/pci/pci.c @@ -4358,7 +4358,15 @@ int pcie_flr(struct pci_dev *dev) =20 ret =3D pci_dev_wait(dev, "FLR", PCIE_RESET_READY_POLL_MS); done: - pci_dev_reset_iommu_done(dev); + /* + * The reset might be invoked to recover a serious error. E.g. when the + * ATC failed to invalidate its stale entries, which can result in data + * corruption. Thus, do not unblock ATS until a successful reset. + */ + if (!ret || !pci_ats_supported(dev)) + pci_dev_reset_iommu_done(dev); + else + pci_warn(dev, "Reset failed. Blocking ATS to protect memory\n"); return ret; } EXPORT_SYMBOL_GPL(pcie_flr); @@ -4436,7 +4444,15 @@ static int pci_af_flr(struct pci_dev *dev, bool prob= e) =20 ret =3D pci_dev_wait(dev, "AF_FLR", PCIE_RESET_READY_POLL_MS); done: - pci_dev_reset_iommu_done(dev); + /* + * The reset might be invoked to recover a serious error. E.g. when the + * ATC failed to invalidate its stale entries, which can result in data + * corruption. Thus, do not unblock ATS until a successful reset. + */ + if (!ret || !pci_ats_supported(dev)) + pci_dev_reset_iommu_done(dev); + else + pci_warn(dev, "Reset failed. Blocking ATS to protect memory\n"); return ret; } =20 @@ -4490,7 +4506,15 @@ static int pci_pm_reset(struct pci_dev *dev, bool pr= obe) pci_dev_d3_sleep(dev); =20 ret =3D pci_dev_wait(dev, "PM D3hot->D0", PCIE_RESET_READY_POLL_MS); - pci_dev_reset_iommu_done(dev); + /* + * The reset might be invoked to recover a serious error. E.g. when the + * ATC failed to invalidate its stale entries, which can result in data + * corruption. Thus, do not unblock ATS until a successful reset. + */ + if (!ret || !pci_ats_supported(dev)) + pci_dev_reset_iommu_done(dev); + else + pci_warn(dev, "Reset failed. Blocking ATS to protect memory\n"); return ret; } =20 @@ -4933,7 +4957,15 @@ static int pci_reset_bus_function(struct pci_dev *de= v, bool probe) =20 rc =3D pci_parent_bus_reset(dev, probe); done: - pci_dev_reset_iommu_done(dev); + /* + * The reset might be invoked to recover a serious error. E.g. when the + * ATC failed to invalidate its stale entries, which can result in data + * corruption. Thus, do not unblock ATS until a successful reset. + */ + if (!rc || !pci_ats_supported(dev)) + pci_dev_reset_iommu_done(dev); + else + pci_warn(dev, "Reset failed. Blocking ATS to protect memory\n"); return rc; } =20 @@ -4978,7 +5010,15 @@ static int cxl_reset_bus_function(struct pci_dev *de= v, bool probe) pci_write_config_word(bridge, dvsec + PCI_DVSEC_CXL_PORT_CTL, reg); =20 - pci_dev_reset_iommu_done(dev); + /* + * The reset might be invoked to recover a serious error. E.g. when the + * ATC failed to invalidate its stale entries, which can result in data + * corruption. Thus, do not unblock ATS until a successful reset. + */ + if (!rc || !pci_ats_supported(dev)) + pci_dev_reset_iommu_done(dev); + else + pci_warn(dev, "Reset failed. Blocking ATS to protect memory\n"); return rc; } =20 diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c index 48946cca4be72..d9a03a7772916 100644 --- a/drivers/pci/quirks.c +++ b/drivers/pci/quirks.c @@ -30,6 +30,7 @@ #include #include #include +#include #include #include #include @@ -4269,7 +4270,15 @@ static int __pci_dev_specific_reset(struct pci_dev *= dev, bool probe, } =20 ret =3D i->reset(dev, probe); - pci_dev_reset_iommu_done(dev); + /* + * The reset might be invoked to recover a serious error. E.g. when the + * ATC failed to invalidate its stale entries, which can result in data + * corruption. Thus, do not unblock ATS until a successful reset. + */ + if (!ret || !pci_ats_supported(dev)) + pci_dev_reset_iommu_done(dev); + else + pci_warn(dev, "Reset failed. Blocking ATS to protect memory\n"); return ret; } =20 --=20 2.43.0 From nobody Mon Apr 6 21:31:00 2026 Received: from BYAPR05CU005.outbound.protection.outlook.com (mail-westusazon11010071.outbound.protection.outlook.com [52.101.85.71]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1A4C31C862D; Tue, 17 Mar 2026 19:16:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.85.71 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773774995; cv=fail; b=tJ1D6RKMofr5w0OX6LQHUuoBCyp4IzfMSXVR1MiQQtFac90nzxvFa0ccd/GK50PpTbKbyGNF+EOdb/7Os3WrhO0UimeIf2O13Z2F3/HYV3VwCr4HaThDoqnZiW9tGWSqa8ESe3MRGOqokqaHwvKdE4jhWUE86i1vtzbFfGqLhDM= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773774995; c=relaxed/simple; bh=UIOLCMzc6cPyRyxEBiWNi+YiRQPnIlO9U8QIYxioULY=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=KsOFsXMQmsnsMbyTgFctaLSspgkcL9amQreLApIEIX8aCyQN3+tWoHcvozl0VSJ64LEuuXtWoUSjwWPbXHNYFKGvNJxH+KMoqRGh/vBw1sUsMvMxSIn36IvFDKTKqm80p7HkNUoT55aOhU6+aU5W505teexzPT7/JNzvK6dq6Wg= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=N9OMAmGO; arc=fail smtp.client-ip=52.101.85.71 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="N9OMAmGO" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=HQInAeNMO2izWNSNzlIE9xdEjAbQxDGpjxJaceqcv95dVPWB8LPPgWw0Mil3qM2lmuzihS/8DbPnCK48Sgo2Tdha8G3VYqczwPOPlAeAhjUbtVAk4/T0TYvZT+BbMZ0CFrT77JHTY82dnJLZDwXDFTU6X9WLajy0ZWI65zJfyvCX3fy79WUKGtqicVzbVAnBkvJ9kTVIRuqaLE84ksPPI1p65F3UA7YWaqLYBV3ytYDsSY2knXWSm4zG9WrN6u8OOwVSWhndB2YsBg5OO2dwGBsHTdNfaX8ZguplpH87CMUXGjkyBZB29YmErnpQIYCCFDSOq+c/GcFqLgh8zE2M+Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=qKviejPpLxFR0KEBnmrbDk2Huubm20TRVcytIyTnug4=; b=ZapXMjaPkrihJJNhnj/Mpye3dFZmI7FytOIGgue9g8xCZefQRNhqWeZS67cpD7uKSlVDzO+q/SE3jGpZL+i9NNHU4T1YN7ZOtoynT10GLfPrR0zD7HeBKUlVl1BfeZyEpt6Q3dMWn9oGJdn4ZvxVXhA9oFZhh6N+fTqfF+Wyc7F3CzFfWg7oMWPiRY+hJOESJeCq67nhEoGY06jR4/2zK3i3D7ynPZKjz3jbnO0INW52FzrfKLRt/Ss+VyFEKiyUfkwAuKDICO6oemfBwe5Di8hOJCEQyYcNLYSG6SfNaKSepwVdivrLkhIAp+SoHGfffig5+8shWcK2vYPeVnlIng== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=qKviejPpLxFR0KEBnmrbDk2Huubm20TRVcytIyTnug4=; b=N9OMAmGOwp9Iq3owIYfTcGtkwD+wb3GxUTeIBzvAnc3USfzNn4863aQ2qbdl9FR7Q2rVnXjE9quRTi8DESHzopJaezlKBK8t6XEF5cxQPTILiEY+kve16iwJ1e150EUCr1vuyzqGKS9ySH4bzfEJXAdB2W7dZL+a2XJ1GPNXqLc7ndN8xd2NLzesZdTKwES5TZrzqEAJ5DnHFZQzgNFh3XfOvGPh32oesBsKFJO8d9dQVG8MnqctVObFuyqzdFe9L0erXHDS4F1gAVUk5Vyo27ChYkewcENsQM8m4DmqzOCjbwmmceb96muUGs5dJz2rC+6yfEtEpALOVKLSvGuODQ== Received: from SJ0PR03CA0367.namprd03.prod.outlook.com (2603:10b6:a03:3a1::12) by SJ1PR12MB6171.namprd12.prod.outlook.com (2603:10b6:a03:45a::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9723.17; Tue, 17 Mar 2026 19:16:28 +0000 Received: from MWH0EPF000A6730.namprd04.prod.outlook.com (2603:10b6:a03:3a1:cafe::12) by SJ0PR03CA0367.outlook.office365.com (2603:10b6:a03:3a1::12) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9700.24 via Frontend Transport; Tue, 17 Mar 2026 19:16:07 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by MWH0EPF000A6730.mail.protection.outlook.com (10.167.249.22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9700.17 via Frontend Transport; Tue, 17 Mar 2026 19:16:28 +0000 Received: from rnnvmail202.nvidia.com (10.129.68.7) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 17 Mar 2026 12:16:05 -0700 Received: from rnnvmail204.nvidia.com (10.129.68.6) by rnnvmail202.nvidia.com (10.129.68.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 17 Mar 2026 12:16:04 -0700 Received: from Asurada-Nvidia.nvidia.com (10.127.8.9) by mail.nvidia.com (10.129.68.6) with Microsoft SMTP Server id 15.2.2562.20 via Frontend Transport; Tue, 17 Mar 2026 12:16:03 -0700 From: Nicolin Chen To: , , , , CC: , , , , , , , , , , , Subject: [PATCH v2 2/7] iommu: Add reset_device_done callback for hardware fault recovery Date: Tue, 17 Mar 2026 12:15:35 -0700 Message-ID: <3750a106b4ab4235df842fa2b9defbc8226ebbef.1773774441.git.nicolinc@nvidia.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MWH0EPF000A6730:EE_|SJ1PR12MB6171:EE_ X-MS-Office365-Filtering-Correlation-Id: af1c5e22-c0dd-48c1-8f46-08de8459b07e X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|376014|7416014|36860700016|82310400026|22082099003|18002099003|56012099003; X-Microsoft-Antispam-Message-Info: ancfFLjTPmgjC1s9iQ97kYHzUFQvewbPLWlQW2+FhZkIOnaR+Pvs9foo6V+VlCWB3d0rX+m/Fz23hKuHq9T7OtFfrSBwmqbrLFA34b2k718qU2WrQko7bpLmYqvNlEMUB3HBFZXPqECEesEWbXOEnR1T1kSBsQknCFsgBlSXo2qJNYdQiNuHAwY+Frxmq+8ECVceJfWBF6WRhorNQI3WfS4dZdA1H8ZGxroFvTBFKbilTM9iAX7bXSB6q5NZzqjTo3qTEs8/dHK0cG9SaGDL0XQrmMf9JqePR8SIG2j8blUH1Z8+YETDPcW60GxKdnpPvuCEWzFdVodbJmzlNfZCrLHfGJn9IDGIyxdaSfdmGl2PsmhpnXNVIebc72YyfNYGvN8t1pidsyCI2uMBJnArUgs3g6ova2t+NRk84/H1FpNRaNzt3J2sZr1GlB5Sq+lEAplRIZpLbDfL5c7Q150htFuddP3PjUqfG7XCzN4gbCwxG/hOxFwgkKxHJy9qVSfIwW6EBKNxkRmBZxYb7x3RY5IhEssCKzNizvECxRl5dJ73mRlSgQ1QJYvL4iaSem91CzBcPeLxcJZk65Wv3yhP2DQsV0LC/B3Vpc2M2tTFh0LhVE4WQXVnw3uGe7Up8F+1Mx11miCF01Wi3JzLqQQgoiJS9epEmp1B11Vzhj/NtfkRtLgIw7L5u9zdzt3G1ogQhNFbj+zdiM0ra50hCy6QfcFiavXXZwdYta7MSvb6Ikv2L3ng/snVMv5nBp/LhgEns0XJtj3A4wuLLDh/uHxjkA== X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(1800799024)(376014)(7416014)(36860700016)(82310400026)(22082099003)(18002099003)(56012099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 5ZzJoOb6Oscgahx1SVS2nG1wMArNRkEMwgVtxP55Zph0dsicSBCpldhuz8fGikorriKk/89nPplw1ht5oeyTzVrKmD5XEhlC/pSpX7ueTqsKcKGlvZ6TzWSpY3RttJCmRbFV4nWm1XFVwNO7uUzA9F+g3UAMOFM3wCsjK94K3Qhajkqof3KsyoJu69nBEefnhsbeXKgLSyzRXQ0+q2PmKD7uy6kWyaJ1tHBQWp9Cufs9hwo0S66z5dxQX5AqGtqbNjOLxboa9MK+h2T9ln0tQ17QE8dDU4JzoiTCwY/tb+K0IoA90CFtWewXXdxGPUbJx77SnB6GFxYJ6YTmSP8tN7nTF+Yszi25aH6YJXn/Zb//3YlZfe+N0VCV/ACGsnXQRmJ7WzKDT6xwmQepjVauIEMZOgE3/CzK9vIy4uqIjmHwubMV2qyENO/x7DpZj6i1 X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Mar 2026 19:16:28.0190 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: af1c5e22-c0dd-48c1-8f46-08de8459b07e X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: MWH0EPF000A6730.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ1PR12MB6171 Content-Type: text/plain; charset="utf-8" When an IOMMU hardware detects an error due to a faulty device (e.g. an ATS invalidation timeout), IOMMU drivers may quarantine the device by disabling specific hardware features or dropping translation capabilities. To recover from these states, the IOMMU driver needs a reliable signal that the underlying physical hardware has been cleanly reset (e.g., via PCIe AER or a sysfs Function Level Reset) so as to lift the quarantine. Introduce a reset_device_done callback in struct iommu_ops. Trigger it from the existing pci_dev_reset_iommu_done() path to notify the underlying IOMMU driver that the device's internal state has been sanitized. Signed-off-by: Nicolin Chen --- include/linux/iommu.h | 2 ++ drivers/iommu/iommu.c | 12 ++++++++++++ 2 files changed, 14 insertions(+) diff --git a/include/linux/iommu.h b/include/linux/iommu.h index 54b8b48c762e8..9ba12b2164724 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -626,6 +626,7 @@ __iommu_copy_struct_to_user(const struct iommu_user_dat= a *dst_data, * @release_device: Remove device from iommu driver handling * @probe_finalize: Do final setup work after the device is added to an IO= MMU * group and attached to the groups domain + * @reset_device_done: Notify the driver about the completion of a device = reset * @device_group: find iommu group for a particular device * @get_resv_regions: Request list of reserved regions for a device * @of_xlate: add OF master IDs to iommu grouping @@ -683,6 +684,7 @@ struct iommu_ops { struct iommu_device *(*probe_device)(struct device *dev); void (*release_device)(struct device *dev); void (*probe_finalize)(struct device *dev); + void (*reset_device_done)(struct device *dev); struct iommu_group *(*device_group)(struct device *dev); =20 /* Request/Free a list of reserved regions for a device */ diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c index 40a15c9360bd1..fcd2902d9e8db 100644 --- a/drivers/iommu/iommu.c +++ b/drivers/iommu/iommu.c @@ -4013,11 +4013,13 @@ EXPORT_SYMBOL_GPL(pci_dev_reset_iommu_prepare); void pci_dev_reset_iommu_done(struct pci_dev *pdev) { struct iommu_group *group =3D pdev->dev.iommu_group; + const struct iommu_ops *ops; unsigned long pasid; void *entry; =20 if (!pci_ats_supported(pdev) || !dev_has_iommu(&pdev->dev)) return; + ops =3D dev_iommu_ops(&pdev->dev); =20 guard(mutex)(&group->mutex); =20 @@ -4029,6 +4031,16 @@ void pci_dev_reset_iommu_done(struct pci_dev *pdev) if (WARN_ON(!group->blocking_domain)) return; =20 + /* + * A PCI device might have been in an error state, so the IOMMU driver + * had to quarantine the device by disabling specific hardware feature + * or dropping translation capability. Here notify the IOMMU driver as + * a reliable signal that the faulty PCI device has been cleanly reset + * so now it can lift its quarantine and restore full functionality. + */ + if (ops && ops->reset_device_done) + ops->reset_device_done(&pdev->dev); + /* Re-attach RID domain back to group->domain */ if (group->domain !=3D group->blocking_domain) { WARN_ON(__iommu_attach_device(group->domain, &pdev->dev, --=20 2.43.0 From nobody Mon Apr 6 21:31:00 2026 Received: from SA9PR02CU001.outbound.protection.outlook.com (mail-southcentralusazon11013049.outbound.protection.outlook.com [40.93.196.49]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 59CCD341AD6; Tue, 17 Mar 2026 19:16:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.93.196.49 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773774998; cv=fail; b=igomYSKMx7R7oV1ifqM3yDeeucGZcsUTt6FYpNBBOXt+QxSR+xI4N+2d0Csx5ZHJT9Z0bQ8fxYIudb0iET4kzhzgBjn46Aci0SKm2azR3+G9kPaiYrNFF5SgGyqx8Y624pK6x8mf5qMjc8lyvFgVmVMMzrDSJ9VSup6aqaLEsoM= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773774998; c=relaxed/simple; bh=02YX6YOHamDb3srsopNGwxlsjI+NQKye6lLacWuuQcs=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=NUNolQKzCoChNvK2LqiOLiG7rK01W5o/inXRWNwURfgex+GJa66CfptOif9s+0Cx2MGW+6fwAmBX2z4Wi2mp7w/DpaK+oXlLr/H88Bmmu7zMqjJfcXaJqSZSFbCO3l+D/UzlUIn+BmFr/IiZUUChe8KNiUCWpE4QuTWbd4liPE8= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=hzXi+aA7; arc=fail smtp.client-ip=40.93.196.49 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="hzXi+aA7" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=F4NtSXyP4VbrPNig4Q9FUm90QOy0ImEAO6hrlD1F0FkYxH0gBO6o14Jh1TjR+slWCXmxYvsWOGVGL05gjEPH0Mgx1qHC8QAX3HhMpWzvWIxe1c0Mh8NbHZSNoF//r1umMdGr9BWysFaBVD3Dc9trHLbkTVWjNSkYTWYEObofnED+pHg2aMM1u9kNQ2IFvoii/LQqRnM16of5iWMUtCyvFMRJgCFkjBsda8HjGgqOpu54+pIxHjd4aZZ3SIkz+0j840kd89CLCUFbVtACgix5cHJmlOJbhQLI1ngu/zk8QdceI2+s5MS5VJv5MfMP+nGrtnFxcM0OHcdW+ZReR+6MMw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=aegLmu1eE3TBW+yxrEgVOqRgd37SSAQVOx05m4K1Jmg=; b=xe8cj/gYSy45Llzuhj7gLPiNO/6VbeisJX3w8zxbph9u7Bgqqg+PcFWxGHqEsUw2n9QKCsMWpPPNOrzURyB300dEikby2LeH03jzs7wxbNuFbJxeFk1azPfYmQq0tghemEaZOAbyIP9ZsoziZeWE8/vVlPeToORLScQdfUD6w8wLOqPAB+Z6ZeGKUAPYib7I/ID461XpD8B1LexMPQ/SuOdxMtdOPVDhu2nVqMP5KlPLNqGaGvAlsyHz2exD94BtmKxLLYID6sctOCP8mN4L0SHBGjKkkx3ztK+I9y8HYl1UtECxLNqHVKmalRR99bz0dVIU2XnWcv842R8Lxmw9sA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=aegLmu1eE3TBW+yxrEgVOqRgd37SSAQVOx05m4K1Jmg=; b=hzXi+aA7GG2n9IG2bC4bd+cdr31xYHjQn3GONG00SdAp2mU/aQJLcykDf2xCjQ/5QR72iFgUOtECtW0AvYBYoVaF2aXX6nhWGnpW3L2yoz51MJfgdPy47Qztk/SI1PnsSEDDB4AewStW64TWsKMiPni6drK+/xwtWdgO4hXCeLF21eNUN5hLXn6YJcDsC+tX7Igbd0dDf1vbXgJ+5OiUyrTpPCQOqOIC6NDOZL5ODBGvU946wscClLt6UavRTXqmENI+bT9mp7Uio45wJg6LIYuvYaZZCsKu9BI3UyNv4EashNUciOTHrWsTMT95ZrcXVomab1Es/DVJKqcZ7EIxhQ== Received: from BN9PR03CA0858.namprd03.prod.outlook.com (2603:10b6:408:13d::23) by BL1PR12MB5969.namprd12.prod.outlook.com (2603:10b6:208:398::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9723.17; Tue, 17 Mar 2026 19:16:29 +0000 Received: from BN1PEPF00004689.namprd05.prod.outlook.com (2603:10b6:408:13d:cafe::5b) by BN9PR03CA0858.outlook.office365.com (2603:10b6:408:13d::23) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9700.27 via Frontend Transport; Tue, 17 Mar 2026 19:16:25 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by BN1PEPF00004689.mail.protection.outlook.com (10.167.243.134) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9700.17 via Frontend Transport; Tue, 17 Mar 2026 19:16:29 +0000 Received: from rnnvmail203.nvidia.com (10.129.68.9) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 17 Mar 2026 12:16:06 -0700 Received: from rnnvmail204.nvidia.com (10.129.68.6) by rnnvmail203.nvidia.com (10.129.68.9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 17 Mar 2026 12:16:05 -0700 Received: from Asurada-Nvidia.nvidia.com (10.127.8.9) by mail.nvidia.com (10.129.68.6) with Microsoft SMTP Server id 15.2.2562.20 via Frontend Transport; Tue, 17 Mar 2026 12:16:04 -0700 From: Nicolin Chen To: , , , , CC: , , , , , , , , , , , Subject: [PATCH v2 3/7] iommu: Add iommu_report_device_broken() to quarantine a broken device Date: Tue, 17 Mar 2026 12:15:36 -0700 Message-ID: X-Mailer: git-send-email 2.43.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN1PEPF00004689:EE_|BL1PR12MB5969:EE_ X-MS-Office365-Filtering-Correlation-Id: 9ca3b226-705d-4bc9-2251-08de8459b15f X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|376014|7416014|36860700016|82310400026|56012099003|22082099003|18002099003; X-Microsoft-Antispam-Message-Info: YB4xbCU4NQuiMeY8Yg3+bPTyqu9NNPHo83HX83sUW9MeWzmeg7fUDVQOnBcu0d9iHaNYFg0OS/bBYmjReHMKu4+ZGBnp/y8zEIZyxZ6FzLU59E2EmfXYXGhHgHLk8+kXESltj0HQ+wBVaSHO5yrScBwYZ6ZpM6uBI7/q4PYq439zXMayt0ri7pO1dEmYSpAnYzAtswyv677Vdz/P/UiSe6GaUZHkHI8QYYvh9zIdALj9iEgVPCf1/J9QvAE8QDPQKDR4qw47YUo6iXupuq/CJjLELMO8vhQYg66wdDAk2mRGwmzVY4hgB+wfS15mw985mm0nnVpAjzjLYlGBwGoKz9GCpryskfQTUHB59tRkdDY0BMeWdnFcNBz/2/2Flr7xvU8D1kINS+FXG6ERHwk/4Cb7lblGLdDaLGclqxWmZa4Fh0QymqgaDqmNvyj6pG4qFatRxaNOfk4DpUmE7AQovnxEnMhNhL+Bc0AjnEV/gI+Ub1DbH4kqYN7AnDC101OI7rIjlTk7/3sdhDVe9x+xch1VmwxvaptppXPv1deGduy/favtiRqjIcDL6Eu52hW22MrcFVIFjTWtJVkcZuu4SbOfM4f/V/gAIl/gCDKoPTGNWOeHRAc6GmaYFqI3FYYvPqNtxS8MNxa+7IXHnVP9WbFgVAoNWLKeur/r3xYc9FJ2cuI8PINIp7wk2E8CM1m0Tjry0EZHvrz+VP5Kw7+kAC3iQhrXm89+T5hC1iw/G4DgTpSTpDo9VtC8QKD6uda3dvKPH20WH9lecM8QMBcOnA== X-Forefront-Antispam-Report: CIP:216.228.117.160;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge1.nvidia.com;CAT:NONE;SFS:(13230040)(1800799024)(376014)(7416014)(36860700016)(82310400026)(56012099003)(22082099003)(18002099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: xfkCZd4B+bKLGz5sG3sBWj9EFmuwoY1AFjjIR6ATlMESFHv8VNCJ/fprj18L+iuWWBiaIS/PMHkWxINkS3f1NVvteV15QxVu9MC1JxKJAqnhqKppXgXOxUmPZKDXupuX9AQX5HX74tDYkoKLChTcvPC7+WGlDxx08eELsoNuAP2ow0PG3eECWFhxGCsYRUdlIvtoXV02GSMZjJH+/R7BoCqtvatrHZezYWCVrX7EIj/RXmiSGHLcobfQh/COLGjSdbmmYlOlo4Bv17Xd2YNoG0AM6mDRErJAMGNf76o/+4+gewypIP7m39v9uUGJz6WCORPPYDUvPa4PURIKMJw29sSLesxdS3DxgwFEg14Luzn4YeH+LfWDV/n4E2NnzWSO3QB4ZjqT82sfL8onotZXk5+d65zMOK4tX04k5gbcSUNac6wOJEzMHwyC5MaOekMp X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Mar 2026 19:16:29.4103 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 9ca3b226-705d-4bc9-2251-08de8459b15f X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN1PEPF00004689.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL1PR12MB5969 Content-Type: text/plain; charset="utf-8" When an IOMMU hardware detects an error due to a faulty device (e.g. an ATS invalidation timeout), IOMMU drivers may quarantine the device by disabling specific hardware features or dropping translation capabilities. However, the core-level states of the faulty device are out of sync, as the device can be still attached to a translation domain or even potentially be moved to a new domain that might overwrite the driver-level quarantine. Given that such an error can be likely an ISR, introduce a broken_work per iommu_group, and add a helper function to allow driver to report the broken device, so as to completely quarantine it in the core. Use the existing pci_dev_reset_iommu_prepare() function to shift the device to its resetting_domain/blocking_domain. A later pci_dev_reset_iommu_done() call will clear it and move it out of the quarantine. Signed-off-by: Nicolin Chen --- include/linux/iommu.h | 2 ++ drivers/iommu/iommu.c | 59 +++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 61 insertions(+) diff --git a/include/linux/iommu.h b/include/linux/iommu.h index 9ba12b2164724..9b5f94e566ff9 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -891,6 +891,8 @@ static inline struct iommu_device *__iommu_get_iommu_de= v(struct device *dev) #define iommu_get_iommu_dev(dev, type, member) \ container_of(__iommu_get_iommu_dev(dev), type, member) =20 +void iommu_report_device_broken(struct device *dev); + static inline void iommu_iotlb_gather_init(struct iommu_iotlb_gather *gath= er) { *gather =3D (struct iommu_iotlb_gather) { diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c index fcd2902d9e8db..2f297f689a3a3 100644 --- a/drivers/iommu/iommu.c +++ b/drivers/iommu/iommu.c @@ -55,6 +55,8 @@ struct iommu_group { struct list_head devices; struct xarray pasid_array; struct mutex mutex; + struct work_struct broken_work; + bool requires_reset; void *iommu_data; void (*iommu_data_release)(void *iommu_data); char *name; @@ -146,6 +148,7 @@ static struct group_device *iommu_group_alloc_device(st= ruct iommu_group *group, struct device *dev); static void __iommu_group_free_device(struct iommu_group *group, struct group_device *grp_dev); +static void iommu_group_broken_worker(struct work_struct *work); static void iommu_domain_init(struct iommu_domain *domain, unsigned int ty= pe, const struct iommu_ops *ops); =20 @@ -1057,6 +1060,7 @@ struct iommu_group *iommu_group_alloc(void) if (!group) return ERR_PTR(-ENOMEM); =20 + INIT_WORK(&group->broken_work, iommu_group_broken_worker); group->kobj.kset =3D iommu_group_kset; mutex_init(&group->mutex); INIT_LIST_HEAD(&group->devices); @@ -4031,6 +4035,7 @@ void pci_dev_reset_iommu_done(struct pci_dev *pdev) if (WARN_ON(!group->blocking_domain)) return; =20 + WRITE_ONCE(group->requires_reset, false); /* * A PCI device might have been in an error state, so the IOMMU driver * had to quarantine the device by disabling specific hardware feature @@ -4062,6 +4067,60 @@ void pci_dev_reset_iommu_done(struct pci_dev *pdev) } EXPORT_SYMBOL_GPL(pci_dev_reset_iommu_done); =20 +static void iommu_group_broken_worker(struct work_struct *work) +{ + struct iommu_group *group =3D + container_of(work, struct iommu_group, broken_work); + struct pci_dev *pdev =3D NULL; + struct device *dev; + + scoped_guard(mutex, &group->mutex) { + /* Do not block the device again if it has been recovered */ + if (!READ_ONCE(group->requires_reset)) + goto out_put; + if (list_is_singular(&group->devices)) { + /* Note: only support group with a single device */ + dev =3D iommu_group_first_dev(group); + if (dev_is_pci(dev)) { + pdev =3D to_pci_dev(dev); + pci_dev_get(pdev); + } + } + } + + if (pdev) { + /* + * Quarantine the device completely. This will be cleared upon + * a pci_dev_reset_iommu_done() call indicating the recovery. + */ + pci_dev_lock(pdev); + pci_dev_reset_iommu_prepare(pdev); + pci_dev_unlock(pdev); + pci_dev_put(pdev); + } +out_put: + iommu_group_put(group); +} + +void iommu_report_device_broken(struct device *dev) +{ + struct iommu_group *group =3D iommu_group_get(dev); + + if (!group) + return; + + if (READ_ONCE(group->requires_reset)) { + iommu_group_put(group); + return; + } + WRITE_ONCE(group->requires_reset, true); + + /* Put the group now or later in iommu_group_broken_worker() */ + if (!schedule_work(&group->broken_work)) + iommu_group_put(group); +} +EXPORT_SYMBOL_GPL(iommu_report_device_broken); + #if IS_ENABLED(CONFIG_IRQ_MSI_IOMMU) /** * iommu_dma_prepare_msi() - Map the MSI page in the IOMMU domain --=20 2.43.0 From nobody Mon Apr 6 21:31:00 2026 Received: from DM1PR04CU001.outbound.protection.outlook.com (mail-centralusazon11010000.outbound.protection.outlook.com [52.101.61.0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 64FBE341AD6; Tue, 17 Mar 2026 19:16:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.61.0 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773775009; cv=fail; b=dcTgywH2/MwaJtECLW5K0miyhS3/S0L5CODCbgjJA3O9ViFv34mfhQq0MawhizXdV23dqOieIhEuPsA67SdfoXLoKnqErIBLgFGlcc1OTjvX73oypTUv/B6CvbuoIVQlHsUduwtfPrOoUWgYdIU4+Unmzqsv5J3OtSua9Coylww= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773775009; c=relaxed/simple; bh=5xTXbAQHezaMvBlLQfEnAp8fdtMX3W/rMQEDLIQQJfk=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=qVzFlZRipT2NQCLYti8IzJxG1U6k3SZfOYjI934vYio0LOsHOwqu7d22sMLZpLWhC9PaEVQFI2vFcwHBEO8V/JSHedAyLwmpFth3qb7KkEiRWqP5DJu4s36aQSHzInbBIfYCrO/0fTEp4WKntdPhGl1dA3Aa7KskTsFkc9ZvUwI= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=fyn+6Ccd; arc=fail smtp.client-ip=52.101.61.0 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="fyn+6Ccd" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=S0GJ+T5gbbQYGs1vATVqHF2Pr9V4JPUn3Ve+G52T4oakJqj0wVQl7oQuTfHDuKgTv2T9vEhm6adAxND6MxdKoFqdd1oW/WShnjCiEXTPzV9tECAFFSxdCFMavqcHoOHQJlBjOuooxZkjih+pUvIgalfYMm3QrACJ8iGm8OupFmTwO8ULK1cbD4V7RYKtfxLdc5BglkdUob9xAs350pMHBN2PkXOrZcbauLMa0srfYlHpLUXP1PpuKUQl7lEA/fzNaylLnaGfaJeSmexxVJa45DDM+/nsPUdUrkzuHgW9zv7f6e9kOzVzQ1woh4dD+G1EuJXC+AneXaDinyY4RgUBmQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=nPjrGkj7efyR279r2rpgxoG4IjG4PFOTy0LL+66LtLc=; b=r0M0ce8JGdzx31cOlp7hnUO4zvkPb5L9IL8vFeOLUjfQIggKhKCb8sjrrr5GMCq74BH07PsgOshNUwCAzxxU0AXNMsB/VCp3iVzsomBQa8ChOyb7aEzUstdswb8nXsfNEIDQyiK6chVvEl8i36qFSi6tdNcl3WPT2+h7z0YikuVh5CmSk0y4GNad1xOg79RyDhnAiwphMASPTMcSyouIueTBy0GhqJUKOIYQu1sJJRhvreUIbVmXNyO99mwxnkY5m3KPp1INI8Zy85Nyx7a/VNMdjYwYNktPYP1tXk5skDlze3rliQgEhBIMGg7sStlmX3sx4d0FSC52SiqgE0/Tng== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=nPjrGkj7efyR279r2rpgxoG4IjG4PFOTy0LL+66LtLc=; b=fyn+6Ccds1g0CV4N1yZhSUpqy72nC/1T9KiOSsVKfT72mNTzdS2dUCVkbbeudXFb1VrEuWqUAjXroRJnpkbRxDWYnGkyjBaNK2WAKEEu3Rq7wa5FeEyZ6Zj05jsPIvWxcJHZ8ZxTTcZgoNR/dZn1fk2rBn6cvbnPRr7yXd3chVauP8Vc1/HLRKFhSAiTFOLatHsbJ0COXYI52aBehSMqExU/FtpBgiRh4I9g/hl2QO6XjmfcrY8rcNlzf72ns2IiPp8NQkfWVpDraS6UY0p2V3ge/r9V1SFUNrm4KWC35dEKJxr1KWrKBL+SfPCXe+8HIoqVBhfMivaQARsuiuhAZg== Received: from PH0PR07CA0113.namprd07.prod.outlook.com (2603:10b6:510:4::28) by CH1PR12MB9623.namprd12.prod.outlook.com (2603:10b6:610:2b3::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9723.13; Tue, 17 Mar 2026 19:16:35 +0000 Received: from MWH0EPF000A6735.namprd04.prod.outlook.com (2603:10b6:510:4:cafe::fb) by PH0PR07CA0113.outlook.office365.com (2603:10b6:510:4::28) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9700.27 via Frontend Transport; Tue, 17 Mar 2026 19:16:15 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by MWH0EPF000A6735.mail.protection.outlook.com (10.167.249.27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9700.17 via Frontend Transport; Tue, 17 Mar 2026 19:16:32 +0000 Received: from rnnvmail203.nvidia.com (10.129.68.9) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 17 Mar 2026 12:16:07 -0700 Received: from rnnvmail204.nvidia.com (10.129.68.6) by rnnvmail203.nvidia.com (10.129.68.9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 17 Mar 2026 12:16:07 -0700 Received: from Asurada-Nvidia.nvidia.com (10.127.8.9) by mail.nvidia.com (10.129.68.6) with Microsoft SMTP Server id 15.2.2562.20 via Frontend Transport; Tue, 17 Mar 2026 12:16:06 -0700 From: Nicolin Chen To: , , , , CC: , , , , , , , , , , , Subject: [PATCH v2 4/7] iommu/arm-smmu-v3: Mark ATC invalidate timeouts via lockless bitmap Date: Tue, 17 Mar 2026 12:15:37 -0700 Message-ID: <0c5525367cc67ccc84a675544d1d9f8462704065.1773774441.git.nicolinc@nvidia.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MWH0EPF000A6735:EE_|CH1PR12MB9623:EE_ X-MS-Office365-Filtering-Correlation-Id: ecb046d3-b377-4472-311a-08de8459b2de X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|36860700016|82310400026|1800799024|376014|7416014|22082099003|18002099003|56012099003; X-Microsoft-Antispam-Message-Info: uP6nGgnc57C3fb9SAhKg2LhE8JUvc4ji/RaIKKe3LniFTNvO8uBKPjoyEBqrrKCPb1zg73d6PGJ1b3ECaG7zM2kJCPRIMByRtXPytkTMFKogYzbOpApWJ2US2w7LKrKoCQeEM/cqtqXuMSVDU0hBloKnq7Qyv9TOQDidOntQCNC0QioLvKpeL1QsGftHlp24QiXnyX1UYCt6qNZwgHR3TvylygMpbGA7Msx0/nC91WW6wGYiUSj4XzAFMuKubWm4GLVLN1l/HuxryqEEJV5ayzj2TMzBMbItR2JobRl9kTM1aBFWHFLrdaPxU6256kep/etge7b7O2ujQjNX90mYLIdOHa8vDEmRuws7ObCm3/uMFiPas/FaLQ9HLOzWKfXzFgQtj7b3+o4jyLmLDRh3qLUCDXCrsiV+sV4wUqXLuSAVrQoCTGEE/8o2Z+QGbnGlw/roUkjJpoaYRnWYLOVIZy4N5Ob3PSNobRpdmckCtU0Db8zz+a6l6Ku18WWTpWpgKcRXlWFjClBA6GDUK1PXQLPkJOYmu9DP5CYao5vZQVlYtWiESy4KNAeCsg26ri2qtvjJsEOlE5vTvScTtqETT6jL5x+XuYx9lSthfsImzldLjt19G7p8GHM6ZRgNNFT+hbvr7SZYuaNk2D6rJuEPQjODOP246PX7PuC8oytQTcVGuIS4wMOGg9mLifu3Q0/Ys1mQV4li9D9DbZLexcw688rwGJaGWFFNb6g9pmLITGRHVjo+5ydXG02ugzAjZnyAKpK+WO/3K6UMo8SxC4c/2Q== X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(36860700016)(82310400026)(1800799024)(376014)(7416014)(22082099003)(18002099003)(56012099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 3bA+vhqeLIa6vferb1y3s5kaWZH5BMYk35cy/s7jRQdmFiHqO2g/FT3MDXcE7MG/ykAbWKksSY3KqbiR2Mi/igkh9thIOc5qwRMRGs2stKecUpVEU/7XzpKJKbE+YtSM3+2i1A3YUpBU3BSviWYUVhC9h3Ma+kHBtF32tEP8lss0877ooC6bA+rLDbYFGafVFGhz33Wf52ImUxFejybY/CEVHKcTOEly1f0nIUdRD0YkamsxTK1mbm9iwpdMyqbIver/pdxvyq0WlwnvBt3+dAuN6jF+rELs/jkyqD7NJw8+ipLcBlAuBGDrC+Fpy/yY4JPYCZqWeONyd2Lpf0yDSfbLBRDQ+Co+h2As/X7xQPxX+oH+BzpaGMIt0XoQMXZy1uexalKqbIqHPVEPccz0/9r2RxgFbLr3iw4oR7HttOJUY65aPHo2/xed1Ize678p X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Mar 2026 19:16:32.0053 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: ecb046d3-b377-4472-311a-08de8459b2de X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: MWH0EPF000A6735.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH1PR12MB9623 Content-Type: text/plain; charset="utf-8" An ATC invalidation timeout is a fatal error. While the SMMUv3 hardware is aware of the timeout via a GERROR interrupt, the driver thread issuing the commands lacks a direct mechanism to verify whether its specific batch was the cause or not, as polling the CMD_SYNC status doesn't natively return a failure code, making it very difficult to coordinate per-device recovery. Introduce an atc_sync_timeouts bitmap in the cmdq structure to bridge this gap. When the ISR detects an ATC timeout, set the bit corresponding to the physical CMDQ index of the faulting CMD_SYNC command. On the issuer side, after polling completes (or times out), test and clear its dedicated bit. If set, override any generic timeout, return -ETIMEDOUT to trigger device quarantine. Signed-off-by: Nicolin Chen --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 1 + drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 20 +++++++++++++++++++- 2 files changed, 20 insertions(+), 1 deletion(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/ar= m/arm-smmu-v3/arm-smmu-v3.h index 36de2b0b2ebe6..3eb12a34b086a 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -633,6 +633,7 @@ struct arm_smmu_cmdq { atomic_long_t *valid_map; atomic_t owner_prod; atomic_t lock; + unsigned long *atc_sync_timeouts; bool (*supports_cmd)(struct arm_smmu_cmdq_ent *ent); }; =20 diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/ar= m/arm-smmu-v3/arm-smmu-v3.c index 01030ffd2fe23..9c8972ebc94f9 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -445,7 +445,10 @@ void __arm_smmu_cmdq_skip_err(struct arm_smmu_device *= smmu, * at the CMD_SYNC. Attempt to complete other pending commands * by repeating the CMD_SYNC, though we might well end up back * here since the ATC invalidation may still be pending. + * + * Mark the faulty batch in the bitmap for the issuer to match. */ + set_bit(Q_IDX(&q->llq, cons), cmdq->atc_sync_timeouts); return; case CMDQ_ERR_CERROR_ILL_IDX: default: @@ -895,9 +898,19 @@ int arm_smmu_cmdq_issue_cmdlist(struct arm_smmu_device= *smmu, =20 /* 5. If we are inserting a CMD_SYNC, we must wait for it to complete */ if (sync) { + u32 sync_prod; + llq.prod =3D queue_inc_prod_n(&llq, n); + sync_prod =3D llq.prod; + ret =3D arm_smmu_cmdq_poll_until_sync(smmu, cmdq, &llq); - if (ret) { + if (test_and_clear_bit(Q_IDX(&llq, sync_prod), + cmdq->atc_sync_timeouts)) { + dev_err_ratelimited(smmu->dev, + "CMD_SYNC for ATC_INV timeout at prod=3D0x%08x\n", + sync_prod); + ret =3D -ETIMEDOUT; + } else if (ret) { dev_err_ratelimited(smmu->dev, "CMD_SYNC timeout at 0x%08x [hwprod 0x%08x, hwcons 0x%08x]\n", llq.prod, @@ -4458,6 +4471,11 @@ int arm_smmu_cmdq_init(struct arm_smmu_device *smmu, if (!cmdq->valid_map) return -ENOMEM; =20 + cmdq->atc_sync_timeouts =3D + devm_bitmap_zalloc(smmu->dev, nents, GFP_KERNEL); + if (!cmdq->atc_sync_timeouts) + return -ENOMEM; + return 0; } =20 --=20 2.43.0 From nobody Mon Apr 6 21:31:00 2026 Received: from DM5PR21CU001.outbound.protection.outlook.com (mail-centralusazon11011001.outbound.protection.outlook.com [52.101.62.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5434333E37D; Tue, 17 Mar 2026 19:16:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.62.1 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773775001; cv=fail; b=oxtV1Mnwk3XFL8veqVpzMzcqI/CvxafqCiqQPL1goIyzi8jPw9ytUWUVvtGlNar9p+ge3/UNuiI3UPmQh7jruCqMn+dDxWSOaENHBaKAAUcL4x8gZyL1Cx72ZSqJiKqIAG+sIxQtfbmV1jbjISqWIrHFey/TmBCgul3VsAMgjgA= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773775001; c=relaxed/simple; bh=8IdppG3r7uOoEfFHkHQKgtLcnrlI8FjDafZvkgte9PQ=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=S2HisPSZhsEIZInlUip68AXH7IG54fY/W/bBVPMJtK1ZRHX/y+K4sgso57SynbEK2Sd4FElO/UEsNekLGVLsqejb+eoi/kkuKDIWlH2j1oGbQGOYz+18GQRC3s7wWjkR/s7K0eRe1XHT6vqyDxBBOtsh7x0z/X45yozHZld2Onc= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=rH1GUDw8; arc=fail smtp.client-ip=52.101.62.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="rH1GUDw8" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=JZcxTEafvMCMaMOv9SxNEaEgSn8nfB0ezSnVNbeuyPy7WiH3pcCouXDzULn7V67ufbBGNQCYNOpWBBirgdrcyXiG4nL95qlkrExzxf+RjOj0ZLr/uvLuVh9W+s6aYqjFHJ0xCJB1w6eNkPaqAEYgJ1swm87dz1+WVpxnFGFOXr0eUQDMrPFfxGyXo6OzUiIPMgh9Jb+IcZmTHQOCyKkg6Hx1IiQzHIYR6Htq+AO1TXtC+MeJ3A1sEPbNyCRseu/tidk33HCpmGpY64b+i8bwJevST136nbI/iY1eg0lj1ebYWvHgd6Pc4KCj2qCIBvH3fwBryWGEgeevk7AQsW4EjA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Cs7IkxuxaIp1J9S0YBd/xcF7iWQbDGAPWJc01CZ+7To=; b=CkuQb5nFtfPqhzkvGNtOmeFv+jD3SnTcRAjC4bLPKmAqEJwdGAWaUTGnIRc9wi0iG20Cq4OYGWM9+5XvVqCGu93r/5PJyIO1GF8Tnm61b3Zzw/ryJutXrYoCDIoUSC2NANXB9/lS7oLmHHwIjgRV+9mOSDr45YDJIYCPw/086KYEYWfdsQEI2iVutG+swOLoC06q6Htnwh7rEnfD8MZJ4LM7iw73N1Y+iSfIHVU1T5LH9o1slp/Sb8uMLcBP/JL85ZWFIpKN2AOjviiF/6ktKfEBGr9dxszHNx0qlSPJk+MDN5Vp4ac76T+kNyALauWd6C/8akV5CuDM88DjqE/rqA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Cs7IkxuxaIp1J9S0YBd/xcF7iWQbDGAPWJc01CZ+7To=; b=rH1GUDw8UtgYft8vHuLtV4aKv3fKOz6reo49BVQ+lYZlN3ikeZZo5TKzG52v05+ELWiX6NT3mXIexzPYrHMmTWT5SdxhWb6s0tBwWyYIdJsRouvCzFTeCOmgnOneVWhKwsYAukeXmM0MIFKkGAn8lGyiXWPCTzunRTgvtxTTyfSZ3SUnOle1Hp3Zx2CBzuzU9v50J4Se5JupOZjvzeyzB88wpoEKNmOGGHRqwtrSgEGM4LS+/SQErBUzW87l2j1z9H/plaL0nG9wIwOQI8OLMLBrxqMcIJoJxiIBATnT7hSmeKMu8AQzKh44akNEY5lk4cR8N3s2GIo8xnhBEWhN6w== Received: from SJ0PR03CA0122.namprd03.prod.outlook.com (2603:10b6:a03:33c::7) by SA1PR12MB9547.namprd12.prod.outlook.com (2603:10b6:806:45a::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9723.19; Tue, 17 Mar 2026 19:16:34 +0000 Received: from MWH0EPF000A672E.namprd04.prod.outlook.com (2603:10b6:a03:33c:cafe::6d) by SJ0PR03CA0122.outlook.office365.com (2603:10b6:a03:33c::7) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9700.25 via Frontend Transport; Tue, 17 Mar 2026 19:16:14 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by MWH0EPF000A672E.mail.protection.outlook.com (10.167.249.20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9700.17 via Frontend Transport; Tue, 17 Mar 2026 19:16:34 +0000 Received: from rnnvmail203.nvidia.com (10.129.68.9) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 17 Mar 2026 12:16:08 -0700 Received: from rnnvmail204.nvidia.com (10.129.68.6) by rnnvmail203.nvidia.com (10.129.68.9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 17 Mar 2026 12:16:08 -0700 Received: from Asurada-Nvidia.nvidia.com (10.127.8.9) by mail.nvidia.com (10.129.68.6) with Microsoft SMTP Server id 15.2.2562.20 via Frontend Transport; Tue, 17 Mar 2026 12:16:07 -0700 From: Nicolin Chen To: , , , , CC: , , , , , , , , , , , Subject: [PATCH v2 5/7] iommu/arm-smmu-v3: Replace smmu with master in arm_smmu_inv Date: Tue, 17 Mar 2026 12:15:38 -0700 Message-ID: <5ea483632238b353cf491904cb0e81f51a9f8327.1773774441.git.nicolinc@nvidia.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MWH0EPF000A672E:EE_|SA1PR12MB9547:EE_ X-MS-Office365-Filtering-Correlation-Id: 762dd030-2f12-4f99-2d0a-08de8459b443 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|376014|7416014|36860700016|82310400026|56012099003|22082099003|18002099003; X-Microsoft-Antispam-Message-Info: t/BCP2Rn3NNpjqnu/G+Byort8X6Guwjn5kh82lgACodDfZsE3Y2ND1pc7WBY1HaxUwLFSu0QFCCAA+iw1LcuuRymz6kB8W3Uc0/A1S3vXpTV55cViKU1F0VjEjd40N17jomgyjOuM6aZrrKTM0rXv1Md0wyGFUQXQKwx4Fjp9EtSZzDFILYb0wLMHTIQtAjhfT4c/SOSEIqMhA9X54ooMx4PIK8qaalObnxpN1wRaGlwJxtrrhrXHz9SH494cbzXoNaBsMl5eS9U2D0GaBqn6u6GInV9yO140FH2YpO33NUHdTHzaE3rtu3sZAL2g5OulAX6eEexjgR4WN27+zZwtQao4hLjRHyKSanYdjlhtmUWm6NfVoMyAC8wKgFc6nHbJT+DbDKUVJ1wPosRptS044+YRN4VdevHMkbrB/QzmgxPVqMGWLZxeVQd898RnR1DCesXmxoIi6lz1rNM/53cCHWZ+76GbjAJ/eSh2Exiig8iBa659zyRN2/GZSmMGDhaC4zFdfI4EblLTjLEC34EOPO/h2mnkN8HF7JQL9pz+4t+VuTynWSSgJ9vJ1JDqhtvB0526teiRsxqE+znFtdfbNXS6yOI5+phqDjvbRyBJhJA4zIjgnnRmKAvUSbyyh8kZLcqN6WtrclxOHkvHCxi7usPDKIh43ORurCyf+2mv8RI98HpacvYC6sNRcsybT1bnt8sY8sxh3ePjv2Pxwt6JZob+cS1eFmiqcaVrogMh//H2z2ThaN8rFjP5noKnQ4PpI6W4tz8+IClGUj22ebvrQ== X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(1800799024)(376014)(7416014)(36860700016)(82310400026)(56012099003)(22082099003)(18002099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: +mEUP2ZhP09u7Ldj5TirQPMomcYPIFXpR7XN/BrGa9ljGA2MElfvhCYM/u9LIEZVTUXRal11ewEcQPfR0gyV3XwbEP83pxkdUOcSJgicXMULDPHSVvhgcrQC/ci3UJkfv1OHBser4Wjmbf+/tiBsxMYH4c90jntpgsZ2OCdzpT+9/YWoIOIAwxg0ZzuNxq2YHtYN+f69Sv6QkpBGeQLBrZzqWDY4jyutqr6SKNTLRpA8Sx1UlaaoIhZ6CLO8MKrj7WSUYxv6zAwxHckoqq+a96ntUTXgQXy0Q5LpD8M5Df8O902r4ydODvd4vE1/SRQmkAgwIYjvjpd1t/aKwkZtKpMntka3V6UOI2RHodkh6W09HSO4IU/1SoTiQroU4QUmUcIwwEhGjTttpFRUj0DhIbtnhtmqwAdIuPaB96DDroa/RsuvcKEfev4fYxNTi9n/ X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Mar 2026 19:16:34.3362 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 762dd030-2f12-4f99-2d0a-08de8459b443 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: MWH0EPF000A672E.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR12MB9547 Content-Type: text/plain; charset="utf-8" Storing master allows to backtrack the master pointer from an invalidation entry, which will be useful when handling ATC invalidation time outs. Signed-off-by: Nicolin Chen --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 2 +- .../iommu/arm/arm-smmu-v3/arm-smmu-v3-test.c | 34 +++++++++++-------- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 25 ++++++++------ 3 files changed, 34 insertions(+), 27 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/ar= m/arm-smmu-v3/arm-smmu-v3.h index 3eb12a34b086a..cb83ea1f3407f 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -662,7 +662,7 @@ enum arm_smmu_inv_type { }; =20 struct arm_smmu_inv { - struct arm_smmu_device *smmu; + struct arm_smmu_master *master; u8 type; u8 size_opcode; u8 nsize_opcode; diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-test.c b/drivers/iom= mu/arm/arm-smmu-v3/arm-smmu-v3-test.c index add671363c828..ef0c0bfe44206 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-test.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-test.c @@ -653,39 +653,43 @@ static void arm_smmu_v3_invs_test_verify(struct kunit= *test, } } =20 +static struct arm_smmu_master invs_master =3D { + .smmu =3D &smmu, +}; + static struct arm_smmu_invs invs1 =3D { .num_invs =3D 3, - .inv =3D { { .type =3D INV_TYPE_S2_VMID, .id =3D 1, }, - { .type =3D INV_TYPE_S2_VMID_S1_CLEAR, .id =3D 1, }, - { .type =3D INV_TYPE_ATS, .id =3D 3, }, }, + .inv =3D { { .master =3D &invs_master, .type =3D INV_TYPE_S2_VMID, .id = =3D 1, }, + { .master =3D &invs_master, .type =3D INV_TYPE_S2_VMID_S1_CLEAR, .id = =3D 1, }, + { .master =3D &invs_master, .type =3D INV_TYPE_ATS, .id =3D 3, }, }, }; =20 static struct arm_smmu_invs invs2 =3D { .num_invs =3D 3, - .inv =3D { { .type =3D INV_TYPE_S2_VMID, .id =3D 1, }, /* duplicated */ - { .type =3D INV_TYPE_ATS, .id =3D 4, }, - { .type =3D INV_TYPE_ATS, .id =3D 5, }, }, + .inv =3D { { .master =3D &invs_master, .type =3D INV_TYPE_S2_VMID, .id = =3D 1, }, /* dup */ + { .master =3D &invs_master, .type =3D INV_TYPE_ATS, .id =3D 4, }, + { .master =3D &invs_master, .type =3D INV_TYPE_ATS, .id =3D 5, }, }, }; =20 static struct arm_smmu_invs invs3 =3D { .num_invs =3D 3, - .inv =3D { { .type =3D INV_TYPE_S2_VMID, .id =3D 1, }, /* duplicated */ - { .type =3D INV_TYPE_ATS, .id =3D 5, }, /* recover a trash */ - { .type =3D INV_TYPE_ATS, .id =3D 6, }, }, + .inv =3D { { .master =3D &invs_master, .type =3D INV_TYPE_S2_VMID, .id = =3D 1, }, /* dup */ + { .master =3D &invs_master, .type =3D INV_TYPE_ATS, .id =3D 5, }, /* re= cover a trash */ + { .master =3D &invs_master, .type =3D INV_TYPE_ATS, .id =3D 6, }, }, }; =20 static struct arm_smmu_invs invs4 =3D { .num_invs =3D 3, - .inv =3D { { .type =3D INV_TYPE_ATS, .id =3D 10, .ssid =3D 1 }, - { .type =3D INV_TYPE_ATS, .id =3D 10, .ssid =3D 3 }, - { .type =3D INV_TYPE_ATS, .id =3D 12, .ssid =3D 1 }, }, + .inv =3D { { .master =3D &invs_master, .type =3D INV_TYPE_ATS, .id =3D 10= , .ssid =3D 1 }, + { .master =3D &invs_master, .type =3D INV_TYPE_ATS, .id =3D 10, .ssid = =3D 3 }, + { .master =3D &invs_master, .type =3D INV_TYPE_ATS, .id =3D 12, .ssid = =3D 1 }, }, }; =20 static struct arm_smmu_invs invs5 =3D { .num_invs =3D 3, - .inv =3D { { .type =3D INV_TYPE_ATS, .id =3D 10, .ssid =3D 2 }, - { .type =3D INV_TYPE_ATS, .id =3D 10, .ssid =3D 3 }, /* duplicate */ - { .type =3D INV_TYPE_ATS, .id =3D 12, .ssid =3D 2 }, }, + .inv =3D { { .master =3D &invs_master, .type =3D INV_TYPE_ATS, .id =3D 10= , .ssid =3D 2 }, + { .master =3D &invs_master, .type =3D INV_TYPE_ATS, .id =3D 10, .ssid = =3D 3 }, /* dup */ + { .master =3D &invs_master, .type =3D INV_TYPE_ATS, .id =3D 12, .ssid = =3D 2 }, }, }; =20 static void arm_smmu_v3_invs_test(struct kunit *test) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/ar= m/arm-smmu-v3/arm-smmu-v3.c index 9c8972ebc94f9..aa42fe39d66b6 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -1071,8 +1071,9 @@ arm_smmu_invs_iter_next(struct arm_smmu_invs *invs, s= ize_t next, size_t *idx) static int arm_smmu_inv_cmp(const struct arm_smmu_inv *inv_l, const struct arm_smmu_inv *inv_r) { - if (inv_l->smmu !=3D inv_r->smmu) - return cmp_int((uintptr_t)inv_l->smmu, (uintptr_t)inv_r->smmu); + if (inv_l->master->smmu !=3D inv_r->master->smmu) + return cmp_int((uintptr_t)inv_l->master->smmu, + (uintptr_t)inv_r->master->smmu); if (inv_l->type !=3D inv_r->type) return cmp_int(inv_l->type, inv_r->type); if (inv_l->id !=3D inv_r->id) @@ -2629,22 +2630,22 @@ static void arm_smmu_inv_to_cmdq_batch(struct arm_s= mmu_inv *inv, unsigned long iova, size_t size, unsigned int granule) { - if (arm_smmu_inv_size_too_big(inv->smmu, size, granule)) { + if (arm_smmu_inv_size_too_big(inv->master->smmu, size, granule)) { cmd->opcode =3D inv->nsize_opcode; - arm_smmu_cmdq_batch_add(inv->smmu, cmds, cmd); + arm_smmu_cmdq_batch_add(inv->master->smmu, cmds, cmd); return; } =20 cmd->opcode =3D inv->size_opcode; - arm_smmu_cmdq_batch_add_range(inv->smmu, cmds, cmd, iova, size, granule, - inv->pgsize); + arm_smmu_cmdq_batch_add_range(inv->master->smmu, cmds, cmd, iova, size, + granule, inv->pgsize); } =20 static inline bool arm_smmu_invs_end_batch(struct arm_smmu_inv *cur, struct arm_smmu_inv *next) { /* Changing smmu means changing command queue */ - if (cur->smmu !=3D next->smmu) + if (cur->master->smmu !=3D next->master->smmu) return true; /* The batch for S2 TLBI must be done before nested S1 ASIDs */ if (cur->type !=3D INV_TYPE_S2_VMID_S1_CLEAR && @@ -2671,7 +2672,7 @@ static void __arm_smmu_domain_inv_range(struct arm_sm= mu_invs *invs, if (READ_ONCE(cur->users)) break; while (cur !=3D end) { - struct arm_smmu_device *smmu =3D cur->smmu; + struct arm_smmu_device *smmu =3D cur->master->smmu; struct arm_smmu_cmdq_ent cmd =3D { /* * Pick size_opcode to run arm_smmu_get_cmdq(). This can @@ -2700,7 +2701,8 @@ static void __arm_smmu_domain_inv_range(struct arm_sm= mu_invs *invs, break; case INV_TYPE_S2_VMID_S1_CLEAR: /* CMDQ_OP_TLBI_S12_VMALL already flushed S1 entries */ - if (arm_smmu_inv_size_too_big(cur->smmu, size, granule)) + if (arm_smmu_inv_size_too_big(cur->master->smmu, size, + granule)) continue; cmd.tlbi.vmid =3D cur->id; arm_smmu_cmdq_batch_add(smmu, &cmds, &cmd); @@ -3225,7 +3227,7 @@ arm_smmu_master_build_inv(struct arm_smmu_master *mas= ter, { struct arm_smmu_invs *build_invs =3D master->build_invs; struct arm_smmu_inv *cur, inv =3D { - .smmu =3D master->smmu, + .master =3D master, .type =3D type, .id =3D id, .pgsize =3D pgsize, @@ -3261,6 +3263,7 @@ arm_smmu_master_build_inv(struct arm_smmu_master *mas= ter, case INV_TYPE_ATS: case INV_TYPE_ATS_FULL: cur->size_opcode =3D cur->nsize_opcode =3D CMDQ_OP_ATC_INV; + cur->master =3D master; cur->ssid =3D ssid; break; } @@ -3457,7 +3460,7 @@ static void arm_smmu_inv_flush_iotlb_tag(struct arm_s= mmu_inv *inv) } =20 cmd.opcode =3D inv->nsize_opcode; - arm_smmu_cmdq_issue_cmd_with_sync(inv->smmu, &cmd); + arm_smmu_cmdq_issue_cmd_with_sync(inv->master->smmu, &cmd); } =20 /* Should be installed after arm_smmu_install_ste_for_dev() */ --=20 2.43.0 From nobody Mon Apr 6 21:31:00 2026 Received: from CH1PR05CU001.outbound.protection.outlook.com (mail-northcentralusazon11010059.outbound.protection.outlook.com [52.101.193.59]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 56F0F334C0D; Tue, 17 Mar 2026 19:16:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.193.59 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773774996; cv=fail; b=As9Am/bGhR48r3RxBFUQyLaJddsw2S9rBVGtjmNFNDeG54un8XRymR3EC/KUDVBIthW6/ugDNRFq7h+Vqwp7v7v9sKVHRNrPNPkMY2NpB7KBWYET61xaN/ct8G/Vc01f1jPL0zoFQICF1+9Tuk/1kB5ctNg5StyB8o+uMNwMuuc= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773774996; c=relaxed/simple; bh=0bmxXQ/7d9HXzfUT++4jjCbYVdXHo2nWvDAuww2WazU=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=K5+wfFaMrMmlCTLidhJElgXNUooKDNRezP4458w+OK0A7iECyA/SOuMR3YMz8OfEM2tuUNdMflMU7y7cmWL8UQIp3i+mNK9HtsDFUtYa4mhtenc8Dt8FRhBm8smIelvhNnMzANGpmZ3g+GeArSPIAFVrA8/hRtgyRql6f7S73DQ= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=TdM5dYo0; arc=fail smtp.client-ip=52.101.193.59 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="TdM5dYo0" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=Of241KZWZE6JwuIXd/duEt+mFeRTEuxqMlHMg4GIy3Yw6JwZ0W/4Fj7qmLT9+2wZnFvsKbtfJn8BphvrwGo+BpK/YNCWDRmjkPlJnUUSBrxDEEh/vjmqZhuCCFvkaOoYr+8zym7HSw1OcZX8nSt9147gw+fmk/b07PjGnB69n1IGJqsD/bMiPlxmLy/YOv/YiP3Jzgku79YZuaErMXrYL5NJryCh9do58+wozOqGG4VmlPI6TYbRriMTIsyqvgXA0gmv/t2/f0GPnODWF+0gLIzjK1s/AQiR9FPFYUZWE3YIEAtZ3jNBuo22/zMTaEzy/+TkVSg2FJ71CZLYzvuU6Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=pzBvpWy7T+f0H4mu1AFA1JABq2Xve8FPaI37vGoXRLY=; b=jnHtx1qC03sfG2GlLcISxtV20PgmixHEhAAyWBWdfQ1o6Jr8MABNrYSwo+M/1PY6Q3Y1eDs4BLPGZQby2W9D7iARvUH9m+T5Ew+TvojDGnI3d+ggW8KotwBF4We2mGmo3gFZFdJO/bLl/pWdbM8TjJA7+O8wuSsNgoytWXqzvZNILNb8gvRdXZzB139sG5faJyeAJz9moD3Rk3hWe721pESIM/1tsgW5Y2yHqso7RWmQ+OFUflc0UpsF/sqGmGHslfid1xaousPe/Fydi5lPAcuHt9p72QMsHna0H8g6NdwfH52YEDTcRvyRqHlIp08OjwfgVl/C+AlfSp7p9qRUYA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=pzBvpWy7T+f0H4mu1AFA1JABq2Xve8FPaI37vGoXRLY=; b=TdM5dYo0S3TRBcPE1Q2UAFSoCpP8YvRh8rQ0sqHTc0Lz2kbJ5DQEOuE0YLWMoUra45RpvyaTBnWQbnvshxvqT1lfQy9FROf/jpOMTUd/4qJldOwXANuvCRabEBh4uAhADZ/Ov/2mUY6MepaIbDhE8+U4Vmgjh9l7piyIr7VXE/YujsudXnMHaFkyNtA213agnCSfr6yp9aB7zQVsh/kOTyy3DkVR5GOB0Eg1la0h4E9/lc+mX5g+WSYxfuIwvgA5GEPhA9rFjqcz7Ut+9JtaIuEbmMdlfw6PTICfMQSQlYmnYpmOAgPjT5DiO9iTOf8ZPl3SgBB2Jov9rBHFPoxaEg== Received: from BN0PR03CA0017.namprd03.prod.outlook.com (2603:10b6:408:e6::22) by IA0PR12MB8088.namprd12.prod.outlook.com (2603:10b6:208:409::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9723.19; Tue, 17 Mar 2026 19:16:32 +0000 Received: from BN1PEPF0000468C.namprd05.prod.outlook.com (2603:10b6:408:e6:cafe::ed) by BN0PR03CA0017.outlook.office365.com (2603:10b6:408:e6::22) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9700.24 via Frontend Transport; Tue, 17 Mar 2026 19:16:09 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by BN1PEPF0000468C.mail.protection.outlook.com (10.167.243.137) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9700.17 via Frontend Transport; Tue, 17 Mar 2026 19:16:32 +0000 Received: from rnnvmail203.nvidia.com (10.129.68.9) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 17 Mar 2026 12:16:10 -0700 Received: from rnnvmail204.nvidia.com (10.129.68.6) by rnnvmail203.nvidia.com (10.129.68.9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 17 Mar 2026 12:16:09 -0700 Received: from Asurada-Nvidia.nvidia.com (10.127.8.9) by mail.nvidia.com (10.129.68.6) with Microsoft SMTP Server id 15.2.2562.20 via Frontend Transport; Tue, 17 Mar 2026 12:16:08 -0700 From: Nicolin Chen To: , , , , CC: , , , , , , , , , , , Subject: [PATCH v2 6/7] iommu/arm-smmu-v3: Introduce master->ats_broken flag Date: Tue, 17 Mar 2026 12:15:39 -0700 Message-ID: X-Mailer: git-send-email 2.43.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN1PEPF0000468C:EE_|IA0PR12MB8088:EE_ X-MS-Office365-Filtering-Correlation-Id: d1dabc28-55bf-44d9-2068-08de8459b30d X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|36860700016|7416014|82310400026|376014|1800799024|22082099003|56012099003|18002099003; X-Microsoft-Antispam-Message-Info: tyCkBRYT6OZLrMZhHncpJFnbQZBHR4Nqrf3jxNOZ9fTf7rewDGLpnhsVsCwQuRTcP5Ln3uBikPZoxpCZi7hbbZ5MTuuAycRG3pRFqGvKGTbxcIxf0RNyo+oFxjUcZaR2W1/V0MzyaRykKijwhTzWl4lWOtxuR4s5Wf+J8Rswl1nxNo0mOl6dj6vvJsPRjX4l0DWUMTdSYXrY3In8t01yW+9QV+Ag1AVxYRRhUX45PZjx85OSrl86NoEGmJjbqw81kVuGxbC01EaM2pdEvzt5L+SNpE7RkaMvPaLAxA0OXKU+8FaeHs9hwwwxju5ckbuZRjAG7NBaMg0aA38Gjm24fm7O13OsxCfZfC52qJoJHGiC/gxbM0h7Zlw5rRkzMHpyAJT39Lewe71RfPRAOFa03U7Ad70Xxh2w1xqfXhThKd8jLaBqZb7uL2uFlrqNNHDUPvQ8XUt+BW2jeWbbGE4AHxm/dNz0xlRLex12Jel6InEZOEvEF+gyK/9j9R749hdA8RmNLGjTxzrbZ3qqQ09belUo3sBRrIh9YSOLQOwBStrnzm7Prw2mPtKJ2r7tu+z8iCoRow2Nvpm/vkgU3fH/p/EPTC03EBi3n986DdJ4BSqnEWJsEz7Um9H23RMV0RQjKs9F4zhKDeSVmXxzzq/8se/FG+nEYQKahqjfFv7BUQdOUmWdmyZAAZmpFpXkuMd1YemWLcyOOlRaslCRaij8qN0+pJGME3maZmTl27hhvUrdacrlCSgG47+veFxVkfESLkUt8RDbUwAe0G7TG0HFGA== X-Forefront-Antispam-Report: CIP:216.228.117.160;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge1.nvidia.com;CAT:NONE;SFS:(13230040)(36860700016)(7416014)(82310400026)(376014)(1800799024)(22082099003)(56012099003)(18002099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: Zkm07m14Zg29K4BBz0uYGlOD4d7exesvoOjpDXQdc0U1iEZduZMTViadGGeX61BHops9x6rLqaYffBSLDcLvmhj1e3ICHUzDVs1EATo1bjYaEznWEWr3GH8VYdFlt1H70vu134Qqc653aJ90qjurmvMLuc2+1C65Um2iy0heB1H2+J05M5KDeFP7JRhpfKo4COk67o9hQynSnQ8JGB8oRcz57B1cV9NyDVNS0i1arWZvMladJsEt/Z01oENeNc93KMy52yRqONrjI1FAGPaqhsJ2zoUzcwVC+0j1RTZDBhxo4Te1rO8I1gvbtRlbVCMIEcnHDXby0vUtF23YKAPl+NP+tGhQXVSe/merNxh4+X3Fe/HJ7JcF1yflpDivspl+ZuzmV03NfJe2u2neQgWPmoSGmu6dAyL3SotIveYCF1ZKZFCp2BOFuOKsqrEDNSCS X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Mar 2026 19:16:32.2306 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: d1dabc28-55bf-44d9-2068-08de8459b30d X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN1PEPF0000468C.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA0PR12MB8088 Content-Type: text/plain; charset="utf-8" The flag will be set when IOMMU cannot trust device's ATS function. E.g., when ATC invalidation request to the device times out. Once it is set, unsupport the ATS feature to prevent data corruption, and skip further ATC invalidation commands to avoid new timeouts. Unset the flag when the device finishes a reset for recovery. Signed-off-by: Nicolin Chen --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 1 + drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 28 +++++++++++++++++++++ 2 files changed, 29 insertions(+) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/ar= m/arm-smmu-v3/arm-smmu-v3.h index cb83ea1f3407f..0a0a88bb60e65 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -941,6 +941,7 @@ struct arm_smmu_master { /* Locked by the iommu core using the group mutex */ struct arm_smmu_ctx_desc_cfg cd_table; unsigned int num_streams; + bool ats_broken; bool ats_enabled : 1; bool ste_ats_enabled : 1; bool stall_enabled; diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/ar= m/arm-smmu-v3/arm-smmu-v3.c index aa42fe39d66b6..366d812668011 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -2502,6 +2502,10 @@ static int arm_smmu_atc_inv_master(struct arm_smmu_m= aster *master, struct arm_smmu_cmdq_ent cmd; struct arm_smmu_cmdq_batch cmds; =20 + /* Do not issue ATC_INV that will definitely time out */ + if (READ_ONCE(master->ats_broken)) + return 0; + arm_smmu_atc_inv_to_cmd(ssid, 0, 0, &cmd); =20 arm_smmu_cmdq_batch_init(master->smmu, &cmds, &cmd); @@ -2708,11 +2712,17 @@ static void __arm_smmu_domain_inv_range(struct arm_= smmu_invs *invs, arm_smmu_cmdq_batch_add(smmu, &cmds, &cmd); break; case INV_TYPE_ATS: + /* Do not issue ATC_INV that will definitely time out */ + if (READ_ONCE(cur->master->ats_broken)) + continue; arm_smmu_atc_inv_to_cmd(cur->ssid, iova, size, &cmd); cmd.atc.sid =3D cur->id; arm_smmu_cmdq_batch_add(smmu, &cmds, &cmd); break; case INV_TYPE_ATS_FULL: + /* Do not issue ATC_INV that will definitely time out */ + if (READ_ONCE(cur->master->ats_broken)) + continue; arm_smmu_atc_inv_to_cmd(IOMMU_NO_PASID, 0, 0, &cmd); cmd.atc.sid =3D cur->id; arm_smmu_cmdq_batch_add(smmu, &cmds, &cmd); @@ -3048,6 +3058,15 @@ void arm_smmu_install_ste_for_dev(struct arm_smmu_ma= ster *master, } } =20 +static void arm_smmu_reset_device_done(struct device *dev) +{ + struct arm_smmu_master *master =3D dev_iommu_priv_get(dev); + + if (WARN_ON(!master)) + return; + WRITE_ONCE(master->ats_broken, false); +} + static bool arm_smmu_ats_supported(struct arm_smmu_master *master) { struct device *dev =3D master->dev; @@ -3060,6 +3079,14 @@ static bool arm_smmu_ats_supported(struct arm_smmu_m= aster *master) if (!(fwspec->flags & IOMMU_FWSPEC_PCI_RC_ATS)) return false; =20 + /* + * Reject any new ATS request because ATC invalidation was timed out. + * The PCI device should go through a recovery (reset) and notify the + * SMMUv3 driver via a reset_device_done callback. + */ + if (READ_ONCE(master->ats_broken)) + return false; + return dev_is_pci(dev) && pci_ats_supported(to_pci_dev(dev)); } =20 @@ -4392,6 +4419,7 @@ static const struct iommu_ops arm_smmu_ops =3D { .domain_alloc_paging_flags =3D arm_smmu_domain_alloc_paging_flags, .probe_device =3D arm_smmu_probe_device, .release_device =3D arm_smmu_release_device, + .reset_device_done =3D arm_smmu_reset_device_done, .device_group =3D arm_smmu_device_group, .of_xlate =3D arm_smmu_of_xlate, .get_resv_regions =3D arm_smmu_get_resv_regions, --=20 2.43.0 From nobody Mon Apr 6 21:31:00 2026 Received: from BN1PR04CU002.outbound.protection.outlook.com (mail-eastus2azon11010059.outbound.protection.outlook.com [52.101.56.59]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6C7DD33FE12; Tue, 17 Mar 2026 19:16:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.56.59 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773775003; cv=fail; b=hP2E6ycvxoxzkRXmGnymh6RstPeLJkzoh8CPxjwOxgS1UWqP5Gv6lmD5yry9Foe07qUaQTzf69NUD7B7UfQ/wyDqbOzArKvZ706cCB2dOwItjenx4YAAcB7FanXpaQDsAXPOd11gxnzxhTV1MtYUbwcrG+gcZU0JTmg6IgymwH0= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773775003; c=relaxed/simple; bh=auMTp9G2OzurJGsLGt5pqpS9fCw0gu3TYFDdtf4Zsrs=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=KIu2k70gixECRWHNSlW8eSwC5FlC4ErEl5h8TNRRDAPGBiZC9mpGp68pcDom7wgaKviaaTIzxz/PQOe47g6cGxklyI3QlCMC9azvD484T8zOOTxk2ZETqD4pqgFms9DxF4xukb6nKz0NjPJ2DBcM0uQ6GksbX2a815yZHvmpImE= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=oMytbCuX; arc=fail smtp.client-ip=52.101.56.59 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="oMytbCuX" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=aurWpxOxT/ScEQS/5t+Gq3aSVmZ88bxNr1LNZp1muDfMN8QUFXIeyIe71l9579SRPgif30WKwN+LVIk+PjRxJi/Lmhmsp2lkLAQ2UtmO1KA7uqCJgfBIDiRrnpj0rznEpH53t63HHNKShO1YM715zik0PzECFPPpE/989flQ2HMVpt0CNs2TZxo+1Qp2idvDImCyzUjHGk1ytCsyiVfEBjjuhxsCY14tJ1Ws8T42AdFazXn2t6Hv3rGnnJSHyNhUPMsmTEqoamzrE2vPx0zKi1a6MtHogF5Fr5U4a4v49azUD/pKq52Ev5inYo2ujz2C7b1oYEts/27IypQFRiDV+A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=KVWk3sfJaF2Ksf1YdGaLHlvw0naTKX8iYWmZ1jhpkRA=; b=ApS7FX7M4XAOVo+nqdabl5lAU0R6Y58BWpxJE3jlpOfYexvG1voRwIe4ujNX8stwnPROZFJxVM0OsYYW5Ui30ADtesX9e177znoFgPk6SIWDFQC1ay92zS3j/tHl2RWMbTAn73VnrLDZQrlhS78ZVyax/Kn8leaDHFn32rpMi7uVn0K70tKBZFf0kvMxI+tK3vqgIH38maKH+3cLk7sktIi4ziLT0PMb1nSUpRsHqA5Z/4PfoFG5XH2BbC5J9DBSo3v10c0vuxXw0gJgHccSoHe/3JQA0fVuMwqvQxsBlzy32RQOym8cYYIO/95PXWp912VpcDo9+5Lu8K/Nt6mSCA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=KVWk3sfJaF2Ksf1YdGaLHlvw0naTKX8iYWmZ1jhpkRA=; b=oMytbCuX787STTXrLUuoABSFR0TN3a8wxYgDRu2eA4HlRO7BYADEZS71CRyYKZ0KrxHf+pOu6lSYAbTcE3OYTCCHX93unoy91cr41+DzQTDXV+ar3DtgBXoRXYCyXEC8QU6XssXVkk6Yfs0915wWvaA+2R2gNZtxx/dCH2HVC5m7OBvIiSvuJ+DheIcFeXRgYocDobIYsQ0wEct66FweXbMXekf9+ATiiEiEayFujdLs7AUB8OidQzxhZGSbCDg5F7QVcR1SN8D24oJmjpY/zcrMSQTrispbEb1pZofMrZ429x8wLKohZ97OOEqbk7RJmT4SswLgy1RTdzCxpVa2Lw== Received: from BN9PR03CA0191.namprd03.prod.outlook.com (2603:10b6:408:f9::16) by BL1PR12MB5778.namprd12.prod.outlook.com (2603:10b6:208:391::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9723.16; Tue, 17 Mar 2026 19:16:34 +0000 Received: from BN1PEPF0000468D.namprd05.prod.outlook.com (2603:10b6:408:f9:cafe::47) by BN9PR03CA0191.outlook.office365.com (2603:10b6:408:f9::16) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9700.24 via Frontend Transport; Tue, 17 Mar 2026 19:16:21 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by BN1PEPF0000468D.mail.protection.outlook.com (10.167.243.138) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9700.17 via Frontend Transport; Tue, 17 Mar 2026 19:16:33 +0000 Received: from rnnvmail203.nvidia.com (10.129.68.9) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 17 Mar 2026 12:16:11 -0700 Received: from rnnvmail204.nvidia.com (10.129.68.6) by rnnvmail203.nvidia.com (10.129.68.9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 17 Mar 2026 12:16:10 -0700 Received: from Asurada-Nvidia.nvidia.com (10.127.8.9) by mail.nvidia.com (10.129.68.6) with Microsoft SMTP Server id 15.2.2562.20 via Frontend Transport; Tue, 17 Mar 2026 12:16:10 -0700 From: Nicolin Chen To: , , , , CC: , , , , , , , , , , , Subject: [PATCH v2 7/7] iommu/arm-smmu-v3: Block ATS upon an ATC invalidation timeout Date: Tue, 17 Mar 2026 12:15:40 -0700 Message-ID: <7e21e14faddeb0e3af692356f4fefbae2dfbebda.1773774441.git.nicolinc@nvidia.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN1PEPF0000468D:EE_|BL1PR12MB5778:EE_ X-MS-Office365-Filtering-Correlation-Id: 66f86d5f-db41-4f1f-e4b2-08de8459b38a X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|36860700016|376014|7416014|1800799024|56012099003|22082099003|18002099003; X-Microsoft-Antispam-Message-Info: MrCzGpvo5qoVsQY1dK2yha9kaGnizShQu9UWvH24RnxRUBMQT/aCVERPKEnJF0EDX1yWYF8/+CI1bU1JmQQ/xe7AI3kpwSPy46fJjSEPjLJnIWUG56Q+8TYeAKenxPMc/ZLlHpNvwg0hG22/vfC+jnhT0PgKyHw2DVVQmuHMkYvZeUwSW9jnjJXRm5ue0v84EEg66fCdr17MCAGsgGW8rPzCxyxe8zEd3Q5tlGmAE/Tdph6VLvuGnRpRQY6b69EMwScRcNDL7GBy5Pk5C8ctZ8V5xHCkyd0SdW6MSrZBQ3Hm0bi6bkxp65Lr5BoVzibidiOPM6MoJzjwsEKlE1JPLysZeD3SO1vo+HPAOPYHXwx6cXV0QmuHOt9EYihfZzHgy4UMHyX8+368z4SOwnVEBGccJEUoNJVfPXW3UMWg25oL6S+SY3/98CaXGVsix7txBtB/MSFUyPetNS7aHk/DWk6+5bzbVfitmJSLAsgVYQwD+OrPdhQ9m1xsHXSimL6JgRd2mRfmMUU8cRdnvrDUqYik9SyNrculjHSIPnFXu05Sfh/OtvKLVKnod/kAQXDCrw97h2jXIFPvk1Mz7NLaxsZVyb/OF7adNn4RxYbf5GvPxjHyQ2iKm7MM/ZKutE53UCVMx7s77mYwbfF6ajXr10W7LAnMlUqBf3J8+aPlB3MfTv7MfUkX2Dw0dGf5lFDZhvXupsbERYR5dy3xNW5UW/8XOQpdpnWCAlXmUDRx3ZL3GtxZxY89l5/mApCkWv8NgRu6+aeV6Sb+YBeSlwxSRA== X-Forefront-Antispam-Report: CIP:216.228.117.160;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge1.nvidia.com;CAT:NONE;SFS:(13230040)(82310400026)(36860700016)(376014)(7416014)(1800799024)(56012099003)(22082099003)(18002099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: oBn2F3nNq2OWFW3epPw3NaDjBuiEsXggcJzf0S6FB8ONjxvF+bKTUtZoQnx0IyAPT63+AYu64CsJap4WLTYsFdpK/j964XjzHqdxGfCTZ0ztC8bFxwbpE5SnfX8cQH02DRPRg9TOAnoHM2zPxjszZyWV43PVIJSwCvpncfjOuev3cSM4Y7yTsK/VnmjratbYHuKsBWZSuUCzPIL9zTO9rFj9mm+VgM7s0y6CtU5iX+tCJgiRlguJonkSn/BP0Njk6AmfxGT0pVRajSkpIZKptlBUmCvKO2BtwMY+uFlPBC0J95TtBFBAm3zvUOEc3H13dXhotcTUtrWQFUhUYBR2KgTf+2Hwyyqakrxje7znywZQQa1N2/+nSGHGLdEsXSccUuwQIF6R9s0BDckV+QgSMOWY+jaiQfHwg7Nv8I/ER4tytuTZjvdgWVVT7HA/XLe3 X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Mar 2026 19:16:33.0499 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 66f86d5f-db41-4f1f-e4b2-08de8459b38a X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN1PEPF0000468D.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL1PR12MB5778 Content-Type: text/plain; charset="utf-8" Currently, when GERROR_CMDQ_ERR occurs, the arm_smmu_cmdq_skip_err() won't do anything for the CMDQ_ERR_CERROR_ATC_INV_IDX. When a device wasn't responsive to an ATC invalidation request, this often results in constant CMDQ errors: unexpected global error reported (0x00000001), this could be serious CMDQ error (cons 0x0302bb84): ATC invalidate timeout unexpected global error reported (0x00000001), this could be serious CMDQ error (cons 0x0302bb88): ATC invalidate timeout unexpected global error reported (0x00000001), this could be serious CMDQ error (cons 0x0302bb8c): ATC invalidate timeout ... An ATC invalidation timeout indicates that the device failed to respond to a protocol-critical coherency request, which means that device's internal ATS state is desynchronized from the SMMU. Furthermore, ignoring the timeout leaves the system in an unsafe state, as the device cache may retain stale ATC entries for memory pages that the OS has already reclaimed and reassigned. This might lead to data corruption. Isolate the device that is confirmed to be unresponsive by a surgical STE update to unset its EATS bit so as to reject any further ATS transaction, which could corrupt the memory. Also, set the master->ats_broken flag that is revertible after the device completes a reset. This flag avoids further ATS requests and invalidations from happening. Finally, report this broken device to the IOMMU core to isolate the device in the core level too. For batched ATC_INV commands, SMMU hardware only reports a timeout at the CMD_SYNC, which could follow the batch issued for multiple devices. So, it isn't straightforward to identify which command in a batch resulted in the timeout. Fortunately, the invs array has a sorted list of ATC entries. So, the issued batch must be sorted as well. This makes it possible to bisect the batch to retry the command per Stream ID and identify the master. Signed-off-by: Nicolin Chen --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 92 ++++++++++++++++++++- 1 file changed, 89 insertions(+), 3 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/ar= m/arm-smmu-v3/arm-smmu-v3.c index 366d812668011..418ee2f40d96d 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -107,6 +107,8 @@ static const char * const event_class_str[] =3D { [3] =3D "Reserved", }; =20 +static struct arm_smmu_ste * +arm_smmu_get_step_for_sid(struct arm_smmu_device *smmu, u32 sid); static int arm_smmu_alloc_cd_tables(struct arm_smmu_master *master); =20 static void parse_driver_options(struct arm_smmu_device *smmu) @@ -2495,10 +2497,43 @@ arm_smmu_atc_inv_to_cmd(int ssid, unsigned long iov= a, size_t size, cmd->atc.size =3D log2_span; } =20 +static void arm_smmu_disable_eats_for_sid(struct arm_smmu_device *smmu, + struct arm_smmu_cmdq *cmdq, u32 sid) +{ + struct arm_smmu_cmdq_ent ent =3D { + .opcode =3D CMDQ_OP_CFGI_STE, + .cfgi =3D { + .sid =3D sid, + .leaf =3D true, + }, + }; + struct arm_smmu_ste *step; + u64 cmd[CMDQ_ENT_DWORDS]; + + step =3D arm_smmu_get_step_for_sid(smmu, sid); + WRITE_ONCE(step->data[1], + READ_ONCE(step->data[1]) & cpu_to_le64(~STRTAB_STE_1_EATS)); + + arm_smmu_cmdq_build_cmd(cmd, &ent); + arm_smmu_cmdq_issue_cmdlist(smmu, cmdq, cmd, 1, true); +} + +static void arm_smmu_master_disable_ats(struct arm_smmu_master *master, + struct arm_smmu_cmdq *cmdq) +{ + int i; + + for (i =3D 0; i < master->num_streams; i++) + arm_smmu_disable_eats_for_sid(master->smmu, cmdq, + master->streams[i].id); + WRITE_ONCE(master->ats_broken, true); + iommu_report_device_broken(master->dev); +} + static int arm_smmu_atc_inv_master(struct arm_smmu_master *master, ioasid_t ssid) { - int i; + int i, ret; struct arm_smmu_cmdq_ent cmd; struct arm_smmu_cmdq_batch cmds; =20 @@ -2514,7 +2549,10 @@ static int arm_smmu_atc_inv_master(struct arm_smmu_m= aster *master, arm_smmu_cmdq_batch_add(master->smmu, &cmds, &cmd); } =20 - return arm_smmu_cmdq_batch_submit(master->smmu, &cmds); + ret =3D arm_smmu_cmdq_batch_submit(master->smmu, &cmds); + if (ret) + arm_smmu_master_disable_ats(master, cmds.cmdq); + return ret; } =20 /* IO_PGTABLE API */ @@ -2661,6 +2699,53 @@ static inline bool arm_smmu_invs_end_batch(struct ar= m_smmu_inv *cur, return false; } =20 +static void arm_smmu_invs_disable_ats(struct arm_smmu_invs *invs, + struct arm_smmu_cmdq *cmdq, + struct arm_smmu_device *smmu, u32 sid) +{ + struct arm_smmu_inv *cur; + size_t i; + + arm_smmu_invs_for_each_entry(invs, i, cur) { + if (cur->master->smmu =3D=3D smmu && arm_smmu_inv_is_ats(cur) && + cur->id =3D=3D sid) { + arm_smmu_master_disable_ats(cur->master, cmdq); + break; + } + } +} + +static void arm_smmu_cmdq_batch_retry(struct arm_smmu_device *smmu, + struct arm_smmu_invs *invs, + struct arm_smmu_cmdq_batch *cmds) +{ + u64 atc[CMDQ_ENT_DWORDS] =3D {0}; + int i; + + /* Only a timed out ATC_INV command needs a retry */ + if (!invs->has_ats) + return; + + for (i =3D 0; i < cmds->num * CMDQ_ENT_DWORDS; i +=3D CMDQ_ENT_DWORDS) { + struct arm_smmu_cmdq *cmdq =3D cmds->cmdq; + u32 sid; + + /* Only need to retry ATC invalidations */ + if (FIELD_GET(CMDQ_0_OP, cmds->cmds[i]) !=3D CMDQ_OP_ATC_INV) + continue; + + /* Only need to retry with one ATC_INV per Stream ID (device) */ + sid =3D FIELD_GET(CMDQ_ATC_0_SID, cmds->cmds[i]); + if (atc[0] && sid =3D=3D FIELD_GET(CMDQ_ATC_0_SID, atc[0])) + continue; + + atc[0] =3D cmds->cmds[i]; + atc[1] =3D cmds->cmds[i + 1]; + if (arm_smmu_cmdq_issue_cmdlist(smmu, cmdq, atc, 1, true)) + arm_smmu_invs_disable_ats(invs, cmdq, smmu, sid); + } +} + static void __arm_smmu_domain_inv_range(struct arm_smmu_invs *invs, unsigned long iova, size_t size, unsigned int granule, bool leaf) @@ -2739,7 +2824,8 @@ static void __arm_smmu_domain_inv_range(struct arm_sm= mu_invs *invs, =20 if (cmds.num && (next =3D=3D end || arm_smmu_invs_end_batch(cur, next))) { - arm_smmu_cmdq_batch_submit(smmu, &cmds); + if (arm_smmu_cmdq_batch_submit(smmu, &cmds)) + arm_smmu_cmdq_batch_retry(smmu, invs, &cmds); cmds.num =3D 0; } cur =3D next; --=20 2.43.0