From nobody Wed Oct 8 11:37:21 2025 Received: from NAM04-DM6-obe.outbound.protection.outlook.com (mail-dm6nam04on2084.outbound.protection.outlook.com [40.107.102.84]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 97A6F224225; Sat, 28 Jun 2025 07:43:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.102.84 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1751096592; cv=fail; b=SIg5z/37vwfEOpKxG/basbQ2wgKfNR6//owMS25hobORhBu2kl8+JY0R9oF9kohF6oJydpr1Bzo4L8GhGoY/uTz0WkT7eXhHcp8B6OEPowoi2CQsJO6w2ar4vtKRrM0eYnfFGoDjL0jTX2d5FrKf72s8b/Zk7n6AyTqWz73/dOg= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1751096592; c=relaxed/simple; bh=gjeK/1OK3tnOyVVm7rUr3RX5chBgzldzSoC0mifsLYA=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=QzU0C89p+2tbwWdjv9VgsDf2psI4oHl4I51OHoRwaHNR2vbQApItNKXRrOaac4feMzqdJDkrj6c5NEUtXcEW+8zjHfGkJCKb8uWvrwLORBeChlnZiGMQVn1E2i8WsVk8r8bWQR4KvZgPngtYHYAR5vY2wx8IaTDcvcZpQNjtnt4= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=czH7ouRd; arc=fail smtp.client-ip=40.107.102.84 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="czH7ouRd" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=idLw0FyWMlu8tmaf/Sfhsy72RiJmZ2QUzXCz8nQkNXo2Q+vm6D9Nwnb18NAg4jqQvq6taG+Kob40647cZjIOe+aGxNbjM1iBJmnL8pN9ehoGxbFTOECBAp75K4J2XlKibQSXbfoakbvp7cij0Pt+jx1j2MewWvOpeMme0tvt0im/Pk+6mPn5AwLOgKnTRYZjwNzG8k//85GwDUGjj6zwQCEWrXcDJVRpFgewNIOQXUn2VFeci6FbmM4x1hxDHW6q1M/TaV2RuOm/W4mNmTkGHz7t/3E1oG4u4KkE+WuMf+TDHE8qpqkgp7DI4UrmKx4NQChnLtURq9n5aAJGUl8WZw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=q2BEfv/wEVhKAiiZV0RGewMzpgD3BK+7XOCguJwzTX0=; b=Oj/THE6jxtD2m3ds32/vZ/lH6k63WNI8R9Wh2ANqV7sStxIr1uUpIrQoKAbYy3f6y5A2D6LtnTMzaN/MzkseuoaiuHU6DykCnY3fHr3Gxf9wAlR70MKhIIQTOyG8oH/23jWNzrFnB4ubYSaVsgYZpfvIeKPkYzxu9SIs9U/1HOcNQTrI1eSJKEV1TlyTanrldHtpK8OSyZb2Nb8MEFNOEaFAs1xF2ZbbXmHIbARKg6Ta4RYgLQ2vorPskHfBCPagHQFfEsula2TZS1V4cgYC29UHgF2hscCOOPKx3m4ESetk+HzYt1sPmSpKhqpj7s+kflk4rin6BSY2VZU7rvy0Uw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=temperror (sender ip is 216.228.117.160) smtp.rcpttodomain=8bytes.org smtp.mailfrom=nvidia.com; dmarc=temperror action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=q2BEfv/wEVhKAiiZV0RGewMzpgD3BK+7XOCguJwzTX0=; b=czH7ouRdnsa3aX742WxBWswhETx//jG+R1PHGZwYHoYbJg6czblGDk/PLNZd0kdkxWMUV0JW4SVhnHjlaeBnWdsRywCpA65xhN7H3xjNofM+xARGH5xlDwq3YO6OTQ9jwvev0XHvQ4HiKO3pMmGVXNaGDLu6o8PefCjIM7Qqbp0uF4NjyA9XFA+GKYudaiXLCjkeakCELjJ/xeigkt4SCsx+4EGpq532uILgzE2zzcFmJDEIB0v+C4g13p0diiRA0P7I9peT94AErMZU8ZTQ3IZrsY4j0jOWOy8Ojemne5NH+QTXWVtUJv7WvAbETfPCuu7ss1xPB0btwQ8mLNqxQQ== Received: from PH7PR10CA0022.namprd10.prod.outlook.com (2603:10b6:510:23d::6) by SA1PR12MB6774.namprd12.prod.outlook.com (2603:10b6:806:259::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8835.30; Sat, 28 Jun 2025 07:43:02 +0000 Received: from SN1PEPF000252A0.namprd05.prod.outlook.com (2603:10b6:510:23d:cafe::6a) by PH7PR10CA0022.outlook.office365.com (2603:10b6:510:23d::6) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8880.25 via Frontend Transport; Sat, 28 Jun 2025 07:43:02 +0000 X-MS-Exchange-Authentication-Results: spf=temperror (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=temperror action=none header.from=nvidia.com; Received-SPF: TempError (protection.outlook.com: error in processing during lookup of nvidia.com: DNS Timeout) Received: from mail.nvidia.com (216.228.117.160) by SN1PEPF000252A0.mail.protection.outlook.com (10.167.242.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8880.14 via Frontend Transport; Sat, 28 Jun 2025 07:43:01 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Sat, 28 Jun 2025 00:42:48 -0700 Received: from rnnvmail205.nvidia.com (10.129.68.10) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.14; Sat, 28 Jun 2025 00:42:48 -0700 Received: from Asurada-Nvidia.nvidia.com (10.127.8.13) by mail.nvidia.com (10.129.68.10) with Microsoft SMTP Server id 15.2.1544.14 via Frontend Transport; Sat, 28 Jun 2025 00:42:47 -0700 From: Nicolin Chen To: , , , , , , CC: , , , , , , , , Subject: [PATCH RFC v2 1/4] iommu: Lock group->mutex in iommu_deferred_attach Date: Sat, 28 Jun 2025 00:42:39 -0700 Message-ID: <98fc799d0cad52d6886ed1136e84a654b0065820.1751096303.git.nicolinc@nvidia.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: AnonymousSubmission X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SN1PEPF000252A0:EE_|SA1PR12MB6774:EE_ X-MS-Office365-Filtering-Correlation-Id: c174b96e-fe4e-433a-b0a4-08ddb6176901 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|36860700013|82310400026|376014|7416014; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?203TTe5ElWzM/w2zuE+ieaQxTdZtVljievnU8csIKi7jGFVjyy4tkq1zb2OQ?= =?us-ascii?Q?qsLigeepSzF4qr23K760TucWx+N51U1YK+rv1YCuFqlJ9CYzGInOQlijh534?= =?us-ascii?Q?TxlMxJex1HD+sgYS7+wgVv24P1ZsT0YS69WDjYsE3Gi+2WaojqgV5CBRs8o4?= =?us-ascii?Q?a6M5fIyG3pGvYdz1i6oag8/AQhYRBOz8XO86WpKScY8IClQ3/CHwtzPMUIyi?= =?us-ascii?Q?/0dMT/SCgS6KtsMRmJehYSMkU0/Ug8LzE6tUppxOBY3qQTS1bldLbkJqz7tn?= =?us-ascii?Q?AVvtegYdUbT2Lw3GHq+kXhIw3Jvurhttmk/Q7OMilfFMBf4jcHMlYI75eZSl?= =?us-ascii?Q?8SfjSqOlqQ/pa84AWaezosDhc0oVQRRLZuWuSJhGwFFKnbwaQsvKJVcm9H1Y?= =?us-ascii?Q?blUyBwwjjuq9MXGEKgREhitc1FWG3l/ALIwBetOGl/ufEY1SzpQ1oMnfFv/6?= =?us-ascii?Q?Oz2JrsWQdWGWyiobXTeavNMuKxQPJhr5WisWk5m/IdfRYS2uG20Pz0b8yJXK?= =?us-ascii?Q?1XV+agFNPJkaxyC9zAPdn3yEj0XjbxGeihHO8z9ykub7tPsr10C+ebaJ5AyK?= =?us-ascii?Q?yl0q9/YVA/o0XkXmHEX+y0tbywdWOJdzAelmz9TjZjrJSie7cCz4rGDgoHWA?= =?us-ascii?Q?xzaKdFgtFfyY3wpf00OQXP0rWgc/RFfWKWWE/GCIUBbwjpui0hYtqfj8CSoF?= =?us-ascii?Q?mV2CBjPCCoGX0XldsHohLzWks+c3IlF7BHcwowCZuW/jhhzkm0LnhaRq/qL6?= =?us-ascii?Q?y1GXpO2bYi1Zbp9AtLhcDZG+N4MLhhexYTZZR8ILWpmGPd3zk7QsvNuHcjgc?= =?us-ascii?Q?MifbNVM7LOThZZS38a0xuFf25VaBuXIsH2QtNfUmmMaPkAIYefGZxjr9T+Wu?= =?us-ascii?Q?Ot6B7RyjpATH0Kas+2ESqJVjLdJNIsF7kI9iHRvUDQg9tFQVh9UPiXS65pmq?= =?us-ascii?Q?s9ablcIViUo77WAUOxN2nr2kU8+PAkidvPzHxUmidorfIgbVvrGoqjju5o8k?= =?us-ascii?Q?eVHrOP2y+Ige7Cv1QHlp+OO+ga5ezVhgrLRfCq8/p4YkY2/nw6TVuCEp6naK?= =?us-ascii?Q?pCSZtbnStXDbXjHKrp9AvuqSloLHcCXyoRua7ITR/1rIwAjOAiF5T10Ipkfv?= =?us-ascii?Q?x/fFFRlbUaSNW4sdrlJyMGV0t8PWfkix4kc/sI0JO3Mf0tn+qnB+lxabAvgB?= =?us-ascii?Q?U7YDLov5qq8APCiimw68pSDM4XmoaotBFUe72qXz0cJffQpnC63waNlWnErM?= =?us-ascii?Q?UQDZHIJoH8pwRQDR6jNIG+hKyqtft7KhKgeMPvzWNB3OFtTv2w1p9A12xd3f?= =?us-ascii?Q?F478+bCYnUkICuZg4XG1Wqlq8192Q7zKoUHupAp7mVdxiCT7N9IohN+z3kwi?= =?us-ascii?Q?r9gpDi5xUjOMl+rCekDI39WFaKnCIV/oy/LIohavpk+VcYhp56iXtofd6wFo?= =?us-ascii?Q?/QHBAca0Quvw2+0xX63UbbphHdFz37eqbOT7Gag92hoV0tAugzpRbkzcDxX+?= =?us-ascii?Q?ri7pRwEyLAg+DW8AQrYyXoWzmkMAgPAIFvGb?= X-Forefront-Antispam-Report: CIP:216.228.117.160;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge1.nvidia.com;CAT:NONE;SFS:(13230040)(1800799024)(36860700013)(82310400026)(376014)(7416014);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Jun 2025 07:43:01.7198 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: c174b96e-fe4e-433a-b0a4-08ddb6176901 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: SN1PEPF000252A0.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR12MB6774 Content-Type: text/plain; charset="utf-8" The iommu_deferred_attach() is a runtime asynchronous function called by iommu-dma function, which will race against other attach functions if it accesses something in the dev->iommu_group. Grab the lock to protect it like others who call __iommu_attach_device() as it will need to access dev->iommu_group. Signed-off-by: Nicolin Chen Reviewed-by: Jason Gunthorpe --- drivers/iommu/iommu.c | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c index a4b606c591da..08ff7efa8925 100644 --- a/drivers/iommu/iommu.c +++ b/drivers/iommu/iommu.c @@ -2151,10 +2151,14 @@ EXPORT_SYMBOL_GPL(iommu_attach_device); =20 int iommu_deferred_attach(struct device *dev, struct iommu_domain *domain) { - if (dev->iommu && dev->iommu->attach_deferred) - return __iommu_attach_device(domain, dev); + struct iommu_group *group =3D dev->iommu_group; + int ret =3D 0; =20 - return 0; + mutex_lock(&group->mutex); + if (dev->iommu && dev->iommu->attach_deferred) + ret =3D __iommu_attach_device(domain, dev); + mutex_unlock(&group->mutex); + return ret; } =20 void iommu_detach_device(struct iommu_domain *domain, struct device *dev) --=20 2.43.0 From nobody Wed Oct 8 11:37:21 2025 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2072.outbound.protection.outlook.com [40.107.236.72]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A94FD15A856; Sat, 28 Jun 2025 07:43:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.236.72 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1751096587; cv=fail; b=sTL1LwY6wDIic0ST376zIC+QSfkzO6/4tFXB/wqKrDCLpsGeE2VDCrZak0gcOvzQ88qdqFp9/IDeAqZmPhz1BHWQCwtBl85j5114HOZxQDIMb9KxR8QWNCtvOz49OusC/qiRRnyYv/D4/uL+7NwV8fkSpxnD56FuUCNX27uq0YE= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1751096587; c=relaxed/simple; bh=/xTnbqaUw46VygF4vXMLrYkMZzqEISELVVriNCwKgug=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=YJrnMwkf1fvDRZJHc+vu+AYzmvyRhrYB3fLTf/MwLfG2aNW7CQbd+iY5qwJsTkLECVpFMrQAFXGzZNMNdeV6rrPjs3X34uYT5oe9+l0VR2tQ2/Z1wnX7fn/wEz8zHrm+1F27+f7N9wvb9Zn8AYZsRcmAiw5ZjoyoUZlR+Cp2oGU= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=UhDZefan; arc=fail smtp.client-ip=40.107.236.72 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="UhDZefan" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=T2Ezvu1uHaBJewAClZikSBFjiU55YHSrdqBthTZJkB02Q4DLdQsjaOZIFbHekpzzy9YDZB8xC1l6QjVUHK33mNmleY/EiM4pJT/ecYiR254zaXRzgpIGBcdcfrWbfjeUEP/pRB3L+ZkylslTHEd59k8TvcPMAFqRS1/YVqPvgKRApg20b8jcDjzl46yEnJeR8o+G8iOztu+XsGljLlT1aJ6Lc1SIMXuE2b31ETX0ElP//r3qQH6V2NR1XGXKSs67WDq0OdiAPTm+aMab/z8ump823mJQjXhYBiUaOV+3AuI1tYGugncwSx+r8Zxj8SCLNdAyAOX1Q0Ii0ZRce8ShtA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=DvttU3IH2zMTh/oVRO9vhdifVWmqUaPXXbK/9IZtB6c=; b=E5ArKMrRJAHLAW1XxGZsovz/PO1iLDpoK9k33UWgYwIc4wQtID2tMuIl52+ac9F2wC66cKrjalN/MWPT2L23f6CrJAsKBH5Y8Ud+YcPoEqyAZmvoIY2n+tdmgxf474/lkAP68dDebcAMr8ug0Ux+xSSO5r54o/2cLgzDatI+J4PrglB187RpxIlgezGG1xjAWijeOPWHgvUHRKGRcbB+ZBcspywPXQ20LxGpN0wj3o87naxU46Jj2VzV6jZ+yGyXdGsIIAP3WfDR7biwBbjC3e0PlB56t2auWLA/1WfYTTHsiF6IDEuZ3AeX7Sx25LLg2pI6TBgln0WjN/rxodZTPw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=8bytes.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=DvttU3IH2zMTh/oVRO9vhdifVWmqUaPXXbK/9IZtB6c=; b=UhDZefanmC658f7T1qcdO7sAiHuRwBIqxXJqKruKW3sMdRExzwhLCkvSJRB0TC61b1r+BPtnVYY/pQcGBa3Q2kLQWJhgGtwQ28Mtx1WtcXpMz7aHUg+Kk1JU09aGhImvRuMAYpxzCbt55GJFrcsmyiBfrbt5TRfvH8FfLPuTcQYpBEkNIeQWJ9cQgON34MaaSZi4LK+Z4Zg9sc+GF8Qr/AkoQbmyJOVBFMjyA/lcRCHmabFX6gKC4KBx2hVDtvx6ETkDJ9VJX0Waef8uWjid4sFMgtsI8lWIURj+A5f/HzvQA9aJ6rSbSJcJba5BfWphhwJEnJJPqwIxnHzjiNXEhg== Received: from DS7PR06CA0032.namprd06.prod.outlook.com (2603:10b6:8:54::17) by DM4PR12MB6399.namprd12.prod.outlook.com (2603:10b6:8:b7::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8835.28; Sat, 28 Jun 2025 07:43:01 +0000 Received: from DS3PEPF000099D5.namprd04.prod.outlook.com (2603:10b6:8:54:cafe::5) by DS7PR06CA0032.outlook.office365.com (2603:10b6:8:54::17) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8880.21 via Frontend Transport; Sat, 28 Jun 2025 07:43:01 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by DS3PEPF000099D5.mail.protection.outlook.com (10.167.17.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8880.14 via Frontend Transport; Sat, 28 Jun 2025 07:43:01 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Sat, 28 Jun 2025 00:42:50 -0700 Received: from rnnvmail205.nvidia.com (10.129.68.10) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.14; Sat, 28 Jun 2025 00:42:49 -0700 Received: from Asurada-Nvidia.nvidia.com (10.127.8.13) by mail.nvidia.com (10.129.68.10) with Microsoft SMTP Server id 15.2.1544.14 via Frontend Transport; Sat, 28 Jun 2025 00:42:48 -0700 From: Nicolin Chen To: , , , , , , CC: , , , , , , , , Subject: [PATCH RFC v2 2/4] iommu: Pass in gdev to __iommu_device_set_domain Date: Sat, 28 Jun 2025 00:42:40 -0700 Message-ID: <46a1c0e7d07722c542084be9611f2fdf770c3a1d.1751096303.git.nicolinc@nvidia.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: AnonymousSubmission X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS3PEPF000099D5:EE_|DM4PR12MB6399:EE_ X-MS-Office365-Filtering-Correlation-Id: 111e2dc0-5c3c-4115-07c2-08ddb61768ef X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|1800799024|82310400026|7416014|36860700013; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?DH/aq1Ag14s7rRoos+kjWRQWPtxd0DoQ46J92Ot+vQUlUW8qCJ/oIl3cFUYP?= =?us-ascii?Q?1RJdVg9zrBadpQVj3GNHVKwbMreIpZ4kFRRhESk8zfejgjcGnJtQFAQyUlmo?= =?us-ascii?Q?ZlzTlsESQF0MJNEbIl40bbg0BrVRHxyrODeVo/qJUyH34hEiAuM1Uzt7R4ot?= =?us-ascii?Q?AV8FtR4QmfX5F1ts3jUB90FzExFDNUnbAGpbCBRyrgBjZ5T7Ug6u5wsY5RlD?= =?us-ascii?Q?2fbP3twzA7VJjnHEH5HoakRNGcTxh3K7QqE6BC8N3Xww7SjME1lhrTYlwbd6?= =?us-ascii?Q?jcK6G29XfT0j43rfXwSf5lCk2OSXL6q+OSgAxNOjOjsvgOWvdFZUh6Y3QdvE?= =?us-ascii?Q?u7R93HjnBRK8O0aiJBdGCtRGgUuQl9YIbmZH+d/OSzxx0TsOSrzrLNwS9GwR?= =?us-ascii?Q?Z7alvQGVP0yVofNdScGrs4xQN8f/HTjVNNNUDs3FelMv0YxgrFOPuDu2KkNK?= =?us-ascii?Q?MlUOjN8wnlPdV+GSqf0lJdnjUIGf0MKX6CGTYsGAPdU/kPwE5f0lcg0MaKIk?= =?us-ascii?Q?nQDT+KytM93/qZ985rhZ54c159Zadg5yYjneW2y5pwodN73+zthyS0C7PRQK?= =?us-ascii?Q?UUTfgBrDVW9fupuxc8BCjRE10ZQYdL0rjvStm+i27OY5cfxZdZp2yGE6jUpR?= =?us-ascii?Q?V8dAMjy4eTzYhoW0g7d5Gbv7Ghms0wIPtXD/XZFpsVU8smQWpDt/jjn/rYSx?= =?us-ascii?Q?89Avuw17qAvoeV5nUNkjcrHoHuvtTiIjOPiMmJYPOcBS7EMXsAmVZU+fwDMZ?= =?us-ascii?Q?/yoRnCrsJ2tdXoCL7tKNDO+o8qLsq6eR9ZgxNL3kJBWZ2nGBI2XQ+oRMo18f?= =?us-ascii?Q?XRKgXa9hiQqkPbO9BAPWzIp1L+7SX3ek6aHOYbWRGXROuakTgyEDDYjgmy//?= =?us-ascii?Q?2hQX3YkM5GYED8pMfb9N5PY1UV6s50IsxZyoTcs5DfV5/0Ksy44qqfiWsrJa?= =?us-ascii?Q?iDttFtQXJ4nz/ClZgHfBA3SoARPPvC6eXCAN4gCMGpku+AUukSEfmQQFAKQq?= =?us-ascii?Q?kOGqKJ7qa44kU/pNR2O1z+7EUJeaRql1vRy+tlxq4E+DWjCvCSL1NN8Qjhqt?= =?us-ascii?Q?LzjOxq9v5KCxJDkUAWZaFd+aAbWUXJ8Vj8gitBR/nw1RE9Oo4f7s5K2pbLqa?= =?us-ascii?Q?XKA6s6Y43yHMilOUqGgu20+FGX4XJHkPla1bkTEl4BT2U54LpGrOki0HgoLe?= =?us-ascii?Q?U0CA4ZhuSQVMc8HGWZAut9AgehT8I7pEeicMzgFp656V9KeRT8NkaIyP2gta?= =?us-ascii?Q?hUbBdBIikXyjclXg70stZdaMU7LA49Zt9DUGcue4k/gx4/GeiFULXQDAwgIR?= =?us-ascii?Q?G9fRMoHVnVtj7Rd5GuxAx2ADll+KhVn+9abQLagdssVYdt4gdJ8JhOPOjVBH?= =?us-ascii?Q?yV1vGc5zAn+Xj9Jyxw//lqsoJNQwFMeeSzLLdmZz5e/iQ4jpOFG6spax+VO3?= =?us-ascii?Q?+0/qaeusaL9cFlLhKsFlgKot4Ktb9imnE6zH0v1BoOViEF2KnZizXmnX5YZE?= =?us-ascii?Q?lUKSsruTKSuxEwraEHOFa0MqUfy6IUDtf9QW?= X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(376014)(1800799024)(82310400026)(7416014)(36860700013);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Jun 2025 07:43:01.6031 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 111e2dc0-5c3c-4115-07c2-08ddb61768ef X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DS3PEPF000099D5.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB6399 Content-Type: text/plain; charset="utf-8" This will need to check a per gdev property, since the dev pointer cannot store any private iommu flag for the iommu code to use. Thus, pass in the gdev pointer instead. Signed-off-by: Nicolin Chen Reviewed-by: Jason Gunthorpe --- drivers/iommu/iommu.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c index 08ff7efa8925..bd3deedcd2de 100644 --- a/drivers/iommu/iommu.c +++ b/drivers/iommu/iommu.c @@ -112,7 +112,7 @@ enum { }; =20 static int __iommu_device_set_domain(struct iommu_group *group, - struct device *dev, + struct group_device *gdev, struct iommu_domain *new_domain, unsigned int flags); static int __iommu_group_set_domain_internal(struct iommu_group *group, @@ -602,7 +602,7 @@ static int __iommu_probe_device(struct device *dev, str= uct list_head *group_list if (group->default_domain) iommu_create_device_direct_mappings(group->default_domain, dev); if (group->domain) { - ret =3D __iommu_device_set_domain(group, dev, group->domain, 0); + ret =3D __iommu_device_set_domain(group, gdev, group->domain, 0); if (ret) goto err_remove_gdev; } else if (!group->default_domain && !group_list) { @@ -2267,10 +2267,11 @@ int iommu_attach_group(struct iommu_domain *domain,= struct iommu_group *group) EXPORT_SYMBOL_GPL(iommu_attach_group); =20 static int __iommu_device_set_domain(struct iommu_group *group, - struct device *dev, + struct group_device *gdev, struct iommu_domain *new_domain, unsigned int flags) { + struct device *dev =3D gdev->dev; int ret; =20 /* @@ -2350,8 +2351,7 @@ static int __iommu_group_set_domain_internal(struct i= ommu_group *group, */ result =3D 0; for_each_group_device(group, gdev) { - ret =3D __iommu_device_set_domain(group, gdev->dev, new_domain, - flags); + ret =3D __iommu_device_set_domain(group, gdev, new_domain, flags); if (ret) { result =3D ret; /* @@ -2383,7 +2383,7 @@ static int __iommu_group_set_domain_internal(struct i= ommu_group *group, */ if (group->domain) WARN_ON(__iommu_device_set_domain( - group, gdev->dev, group->domain, + group, gdev, group->domain, IOMMU_SET_DOMAIN_MUST_SUCCEED)); if (gdev =3D=3D last_gdev) break; --=20 2.43.0 From nobody Wed Oct 8 11:37:21 2025 Received: from NAM04-BN8-obe.outbound.protection.outlook.com (mail-bn8nam04on2084.outbound.protection.outlook.com [40.107.100.84]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B4504223702; Sat, 28 Jun 2025 07:43:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.100.84 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1751096591; cv=fail; b=bhTn3X/vW2VKGT1/zI1myeUNTSHuKUyneWnU/N9D+tWDlVPHTR5Oo+8ImTPcXl35BbFjP5pfmcwRGQ9jGHzhHFQGwmg+eQTA8cpKBxNMi9+bTcoTYmukq8kwI6/yXHuui7at76bdyLUrzHeeJ+EF0cgCjcOEShpOV0G4xKCXYgY= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1751096591; c=relaxed/simple; bh=y0Lb2eeJETGd7adfYgujEN4XdEDwdQn7CxLYL5cTchY=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=WGJHpUOS/NqZE2/rNKY3BOvtnzEKQslS/4LlF3bQL5cauPOlLhg0MkBKbAfdZJ7bC3XxqpWnpqdkwzSOQkNq4KO2qRwQlk0U4MBU73JsCnpSmZ6EZlv2yqda4epJBfmdSrnCNlRRwd4KLZWysBEq/Qgb1yggSGfZtQ4g7pCb7ek= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=ffaoo6QW; arc=fail smtp.client-ip=40.107.100.84 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="ffaoo6QW" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=qNySbmo69Z55vQ8qValWh1Rzz52Ge+cQ3D2lK6WK9T9ZWQerL2PXeeWNfCO8pz7KJ5gTeZrCmT+MZidjVwcXSeNO1SyeTqoFToPx3chS7g7/tlD5H8peAx7HdKazbYtPgzM3c6ubMZvjlpc4s/rQOK7ekYXdXtTa9xX1EumB9mXnRJPHE2/nctm7B/sUb2QSf4Eb4qJ9s5ddL7LplJQyuHmXPTTN5fjJpxWIW5eA3SD6zHEysXhVntBawSFM6iE164Qa2tlrrJF2BapnO2ar7rVH4O50+yLGNBX3HzWsj5KZmeLOITcRj6ssnkb7LZCucox9aCPD+qnyNynsP1EN+A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=q3he9eaivuTJOJxA9cvv0atC+gKGLtza2iC7DsHC2T4=; b=G/PVWab3HgKKcI5QmN1Gtp3QbsJNVpkxm6yOmPr/qeR8WU9k7S3cwXg+w1iD1rE/duBiA9SOp3mz3ZgEEW0R1ednxzioTmNF23tLvqhpteX02qFsLhmdawj/uAQnTNBUrDAp3hPy4s3VCx7b38hxA2n54EPA9bJwhVYOPKnJotmzxw3E2STV7w2A/i0lnRuTesBrnFQSIJmfryaHMTJAIH/+FsaN0eZ2R0zSto6fZVXzn2jbwYs0Em6j3fF+F82LhoFNdmKaVOdBQq0og3kA2lhJid4icent8RsbscXn0EXBAibKJfzV96/WK+cRUFVs85pzY3J2PKabTYiglm43fA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=8bytes.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=q3he9eaivuTJOJxA9cvv0atC+gKGLtza2iC7DsHC2T4=; b=ffaoo6QW9tmAzTRt2EJPGE4lvk6GDJPrvPZNmvP82xLbL3ZlJy/LdAiyzKjXZGlAf7wUxSXUZSq9ZLmj5fy8Ay2e4rVb+0NbJVaBwC0IJlCvp55eh+Z47wd2195WwDsrAVRvZNIrtoSzFhU1JSg38k6AeS9PmQ+bcugugBYgwjWdOpWeKOCtm9voNOaSTpfbAhrbST19ZRC+G9VddUDWal6sWhQ2kkqdPyg+Uzr40AkgrfrNMk2b4bJ2bHJEzSSqMq0RtG6VYehx10GiVO6L10xvAxgbTkzbUBI6Ekfsyxfj63i9Lc5XyBQDOwc7XY4fE7gYSdZjhNtlDKrwnu5buw== Received: from SA0PR11CA0169.namprd11.prod.outlook.com (2603:10b6:806:1bb::24) by DS5PPF8002542C7.namprd12.prod.outlook.com (2603:10b6:f:fc00::657) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8678.34; Sat, 28 Jun 2025 07:43:05 +0000 Received: from SN1PEPF000252A2.namprd05.prod.outlook.com (2603:10b6:806:1bb:cafe::5f) by SA0PR11CA0169.outlook.office365.com (2603:10b6:806:1bb::24) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8880.23 via Frontend Transport; Sat, 28 Jun 2025 07:43:04 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by SN1PEPF000252A2.mail.protection.outlook.com (10.167.242.9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8880.14 via Frontend Transport; Sat, 28 Jun 2025 07:43:04 +0000 Received: from rnnvmail202.nvidia.com (10.129.68.7) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Sat, 28 Jun 2025 00:42:53 -0700 Received: from rnnvmail205.nvidia.com (10.129.68.10) by rnnvmail202.nvidia.com (10.129.68.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.14; Sat, 28 Jun 2025 00:42:51 -0700 Received: from Asurada-Nvidia.nvidia.com (10.127.8.13) by mail.nvidia.com (10.129.68.10) with Microsoft SMTP Server id 15.2.1544.14 via Frontend Transport; Sat, 28 Jun 2025 00:42:50 -0700 From: Nicolin Chen To: , , , , , , CC: , , , , , , , , Subject: [PATCH RFC v2 3/4] iommu: Introduce iommu_dev_reset_prepare() and iommu_dev_reset_done() Date: Sat, 28 Jun 2025 00:42:41 -0700 Message-ID: <9042270b6c2d15a53e66d22d29b87c1c59e60669.1751096303.git.nicolinc@nvidia.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: AnonymousSubmission X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SN1PEPF000252A2:EE_|DS5PPF8002542C7:EE_ X-MS-Office365-Filtering-Correlation-Id: cb056ebc-f992-48ee-86a2-08ddb6176acf X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|36860700013|1800799024|376014|7416014; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?NPWzo7OesS5LCSTksMr+0noteUq7OGWgxYRh4wf2WnAOMYovsZe0mTVSk+G9?= =?us-ascii?Q?QaCA21Ydk9EkUpv10EtJihKK2bx3ZoY03xNcIj2jkBnCWTWHV9Xy/vJljjLA?= =?us-ascii?Q?+3XR9s3cvdtX+QZNwzx4XcglcUNxCB2m6mTUB9sYpMZjurkO8KBD1xKjBJpA?= =?us-ascii?Q?I6Mh6ryGKCPkLGBfsfW5BBkpiMY8SMJUrA+08SOJjIQFGdYCTjAdHdngbOBa?= =?us-ascii?Q?xUbPOrGQqP4++2d0d/bHJ5Q89/5desDnrj5/8a3yfnc1bfReAZvt0wGFxC0g?= =?us-ascii?Q?sCDZEvvgZZT2SetuzhLcMV9vlyMXXfHc/wUZpoP2aru1FPxj7rMsia9NZrNV?= =?us-ascii?Q?kR/ZegGrIhscdyiGGloorTZ50+uojJsNt/tGCnYmu+OBuokyAIcNzfLhhvzK?= =?us-ascii?Q?uKkiy2+U17s7BAL9W3SQDBDwiY3NRm4zpZ7ISlBM83sIrHuldl0M0dLScvXx?= =?us-ascii?Q?YvvEFTypvePWGe6UuF+RlYxt+p0QMCn4gBXdyrnb0LsJLfPsSy92xJCdifCX?= =?us-ascii?Q?nMQOWw1VV3g6Ef9V/REd3ZiV+6SfypUlQt5VbZ0ZXiclhpOGnuAcZYtb0V4U?= =?us-ascii?Q?m7Q74054ZlBIDeTry8FLLdCZWG1ZAIxGnuCYBcFLVHy/lSHXUTkdP/23NZq5?= =?us-ascii?Q?lbtSGtCQwNZXsy2UaGZYqjUXnArMQPceT+TPUKJs4kzNLzhKAop0XcN4DHSu?= =?us-ascii?Q?RKMJdPJA/T0qdgWiSC10GxH/d0/4dg/WGxcUWhh/W7mkn2BkFbRxPHVnSuPi?= =?us-ascii?Q?Ve+JSXrdIUFchWppknyVt8JwGgnm4JefswVL7L59DAVeEEMWiAJPRq2CNBg0?= =?us-ascii?Q?BxsyPqBDFdEdpC+ffj64W5hsz3O6XA1Q2rG7183Mtm5yl7o7kvy0LI0GsUd4?= =?us-ascii?Q?D7g5VTPfH9n4dI5YKbGLrAAO+ARmlGBQWFEnHRlzCOszG9CXSs51KoiculKg?= =?us-ascii?Q?cVK2i+QbWP1hbsIDMgnxbZ8VE0b0snoJCX8M1OQXRq5KBl3M1itnpcgr3ZNP?= =?us-ascii?Q?ToTgGseqw/jOEVDDraDvW6oXgsS+RPalBZzzgmpDikdUpmwcF7ssEJStEu/j?= =?us-ascii?Q?sxooj7ENNrz+mkRjoMENa7t6bZ6tYUBmn+JiY25kxDLu3QrO7Pr2eAmvwWna?= =?us-ascii?Q?lgsDpQCcU1gX6fCzuYp62SrrCGR3DIUOXGFuNiMFhpHMCGTT5XKL11WpdaWr?= =?us-ascii?Q?dEUETQfiRNADbtLWpo4b6SfvBugXVhiO6arGhWrBvBox9b9BZ/S1Oef5p4BO?= =?us-ascii?Q?cJqMBfRVexhZQZQeTMgraKPgx8SIUjv8Qc/N1rUyHl4jPPJEkW5fPrFzKEj1?= =?us-ascii?Q?iHmdV4MRW+2wx8S0mmSUZll/CtaEYkqPU3lWaiCBr13RGhIIU4vst3vHssRK?= =?us-ascii?Q?vORCCfT3t5Jv4hg6R+XowSMea7XfKMGCJhi7rKR26PscfCvQ+zDKybrwF2Gh?= =?us-ascii?Q?C7FoMDTjUC1RA3kphVx9fqoIrOjBzTMWw8DgIFXDk3VnfNuvSLT3zhCb/S00?= =?us-ascii?Q?w1KSpXdhuLpXylasTsGRV8GZnZ8HcEn77U/x?= X-Forefront-Antispam-Report: CIP:216.228.117.160;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge1.nvidia.com;CAT:NONE;SFS:(13230040)(82310400026)(36860700013)(1800799024)(376014)(7416014);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Jun 2025 07:43:04.7526 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: cb056ebc-f992-48ee-86a2-08ddb6176acf X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: SN1PEPF000252A2.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS5PPF8002542C7 Content-Type: text/plain; charset="utf-8" PCIe permits a device to ignore ATS invalidation TLPs, while processing a reset. This creates a problem visible to the OS where an ATS invalidation command will time out: e.g. an SVA domain will have no coordination with a reset event and can racily issue ATS invalidations to a resetting device. The OS should do something to mitigate this as we do not want production systems to be reporting critical ATS failures, especially in a hypervisor environment. Broadly, OS could arrange to ignore the timeouts, block page table mutations to prevent invalidations, or disable and block ATS. The PCIe spec in sec 10.3.1 IMPLEMENTATION NOTE recommends to disable and block ATS before initiating a Function Level Reset. It also mentions that other reset methods could have the same vulnerability as well. Provide a callback from the PCI subsystem that will enclose the reset and have the iommu core temporarily change all the attached domain to BLOCKED. After attaching a BLOCKED domain, IOMMU drivers should fence any incoming ATS queries, synchronously stop issuing new ATS invalidations, and wait for all ATS invalidations to complete. This can avoid any ATS invaliation timeouts. However, if there is a domain attachment/replacement happening during an ongoing reset, the ATS might be re-enabled between the two function calls. Introduce a new pending_reset flag in group_device to defer an attachment during a reset, allowing iommu core to cache the target domains in the SW level but bypassing the driver. The iommu_dev_reset_done() will re-attach these soft-attached domains via __iommu_attach_device/set_group_pasid(). Notes: - This only works for IOMMU drivers that implemented ops->blocked_domain correctly with pci_disable_ats(). - This only works for IOMMU drivers that will not issue ATS invalidation requests to the device, after it's docked at ops->blocked_domain. Driver should fix itself to align with the aforementioned notes. Signed-off-by: Nicolin Chen --- include/linux/iommu.h | 12 ++++ drivers/iommu/iommu.c | 158 ++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 170 insertions(+) diff --git a/include/linux/iommu.h b/include/linux/iommu.h index 156732807994..a17161b8625a 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -1123,6 +1123,9 @@ void dev_iommu_priv_set(struct device *dev, void *pri= v); extern struct mutex iommu_probe_device_lock; int iommu_probe_device(struct device *dev); =20 +int iommu_dev_reset_prepare(struct device *dev); +void iommu_dev_reset_done(struct device *dev); + int iommu_device_use_default_domain(struct device *dev); void iommu_device_unuse_default_domain(struct device *dev); =20 @@ -1407,6 +1410,15 @@ static inline int iommu_fwspec_add_ids(struct device= *dev, u32 *ids, return -ENODEV; } =20 +static inline int iommu_dev_reset_prepare(struct device *dev) +{ + return 0; +} + +static inline void iommu_dev_reset_done(struct device *dev) +{ +} + static inline struct iommu_fwspec *dev_iommu_fwspec_get(struct device *dev) { return NULL; diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c index bd3deedcd2de..14bfeaa9ac29 100644 --- a/drivers/iommu/iommu.c +++ b/drivers/iommu/iommu.c @@ -71,12 +71,29 @@ struct group_device { struct list_head list; struct device *dev; char *name; + bool pending_reset : 1; }; =20 /* Iterate over each struct group_device in a struct iommu_group */ #define for_each_group_device(group, pos) \ list_for_each_entry(pos, &(group)->devices, list) =20 +/* Callers must hold the dev->iommu_group->mutex */ +static struct group_device *device_to_group_device(struct device *dev) +{ + struct iommu_group *group =3D dev->iommu_group; + struct group_device *gdev; + + lockdep_assert_held(&group->mutex); + + /* gdev must be in the list */ + for_each_group_device(group, gdev) { + if (gdev->dev =3D=3D dev) + break; + } + return gdev; +} + struct iommu_group_attribute { struct attribute attr; ssize_t (*show)(struct iommu_group *group, char *buf); @@ -2155,8 +2172,17 @@ int iommu_deferred_attach(struct device *dev, struct= iommu_domain *domain) int ret =3D 0; =20 mutex_lock(&group->mutex); + + /* + * There is a racy attach while the device is resetting. Defer it until + * the iommu_dev_reset_done() that attaches the device to group->domain. + */ + if (device_to_group_device(dev)->pending_reset) + goto unlock; + if (dev->iommu && dev->iommu->attach_deferred) ret =3D __iommu_attach_device(domain, dev); +unlock: mutex_unlock(&group->mutex); return ret; } @@ -2295,6 +2321,13 @@ static int __iommu_device_set_domain(struct iommu_gr= oup *group, dev->iommu->attach_deferred =3D 0; } =20 + /* + * There is a racy attach while the device is resetting. Defer it until + * the iommu_dev_reset_done() that attaches the device to group->domain. + */ + if (gdev->pending_reset) + return 0; + ret =3D __iommu_attach_device(new_domain, dev); if (ret) { /* @@ -3378,6 +3411,13 @@ static int __iommu_set_group_pasid(struct iommu_doma= in *domain, int ret; =20 for_each_group_device(group, device) { + /* + * There is a racy attach while the device is resetting. Defer + * it until the iommu_dev_reset_done() that attaches the device + * to group->domain. + */ + if (device->pending_reset) + continue; if (device->dev->iommu->max_pasids > 0) { ret =3D domain->ops->set_dev_pasid(domain, device->dev, pasid, old); @@ -3799,6 +3839,124 @@ int iommu_replace_group_handle(struct iommu_group *= group, } EXPORT_SYMBOL_NS_GPL(iommu_replace_group_handle, "IOMMUFD_INTERNAL"); =20 +/* + * Caller must use iommu_dev_reset_prepare() and iommu_dev_reset_done() to= gether + * before/after the core-level reset routine, to unclear the pending_reset= flag + * and to put the iommu_group reference. + * + * These two functions are designed to be used by PCI reset functions that= would + * not invoke any racy iommu_release_device() since PCI sysfs node gets re= moved + * before it notifies with a BUS_NOTIFY_REMOVED_DEVICE. When using them in= other + * case, callers must ensure there will be no racy iommu_release_device() = call, + * which otherwise would UAF the dev->iommu_group pointer. + */ +int iommu_dev_reset_prepare(struct device *dev) +{ + const struct iommu_ops *ops; + struct iommu_group *group; + unsigned long pasid; + void *entry; + int ret =3D 0; + + if (!dev_has_iommu(dev)) + return 0; + + if (dev->iommu->require_direct) { + dev_warn( + dev, + "Firmware has requested this device have a 1:1 IOMMU mapping, rejecting= configuring the device without a 1:1 mapping. Contact your platform vendor= .\n"); + return -EINVAL; + } + + /* group will be put in iommu_dev_reset_done() */ + group =3D iommu_group_get(dev); + + /* Caller ensures no racy iommu_release_device(), so this won't UAF */ + mutex_lock(&group->mutex); + + ops =3D dev_iommu_ops(dev); + if (!ops->blocked_domain) { + dev_warn(dev, + "IOMMU driver doesn't support IOMMU_DOMAIN_BLOCKED\n"); + ret =3D -EOPNOTSUPP; + goto unlock; + } + + device_to_group_device(dev)->pending_reset =3D true; + + /* Device is already attached to the blocked_domain. Nothing to do */ + if (group->domain->type =3D=3D IOMMU_DOMAIN_BLOCKED) + goto unlock; + + /* Dock RID domain to blocked_domain while retaining group->domain */ + ret =3D __iommu_attach_device(ops->blocked_domain, dev); + if (ret) + goto unlock; + + /* Dock PASID domains to blocked_domain while retaining pasid_array */ + xa_lock(&group->pasid_array); + xa_for_each_start(&group->pasid_array, pasid, entry, 1) + iommu_remove_dev_pasid(dev, pasid, + pasid_array_entry_to_domain(entry)); + xa_unlock(&group->pasid_array); + +unlock: + mutex_unlock(&group->mutex); + if (ret) + iommu_group_put(group); + return ret; +} +EXPORT_SYMBOL_GPL(iommu_dev_reset_prepare); + +/* + * Pair with a previous iommu_dev_reset_prepare() that was successfully re= turned + * + * Note that, although unlikely, there is a risk that re-attaching domains= might + * fail due to some unexpected happening like OOM. + */ +void iommu_dev_reset_done(struct device *dev) +{ + struct iommu_group *group =3D dev->iommu_group; + const struct iommu_ops *ops; + struct group_device *gdev; + unsigned long pasid; + void *entry; + + if (!dev_has_iommu(dev)) + return; + + mutex_lock(&group->mutex); + + gdev =3D device_to_group_device(dev); + + ops =3D dev_iommu_ops(dev); + /* iommu_dev_reset_prepare() was not successfully called */ + if (WARN_ON(!ops->blocked_domain || !gdev->pending_reset)) { + mutex_unlock(&group->mutex); + return; + } + + if (group->domain->type =3D=3D IOMMU_DOMAIN_BLOCKED) + goto done; + + /* Shift RID domain back to group->domain */ + WARN_ON(__iommu_attach_device(group->domain, dev)); + + /* Shift PASID domains back to domains retained in pasid_array */ + xa_lock(&group->pasid_array); + xa_for_each_start(&group->pasid_array, pasid, entry, 1) + WARN_ON(__iommu_set_group_pasid( + pasid_array_entry_to_domain(entry), group, pasid, + ops->blocked_domain)); + xa_unlock(&group->pasid_array); + +done: + gdev->pending_reset =3D false; + mutex_unlock(&group->mutex); + iommu_group_put(group); +} +EXPORT_SYMBOL_GPL(iommu_dev_reset_done); + #if IS_ENABLED(CONFIG_IRQ_MSI_IOMMU) /** * iommu_dma_prepare_msi() - Map the MSI page in the IOMMU domain --=20 2.43.0 From nobody Wed Oct 8 11:37:21 2025 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2065.outbound.protection.outlook.com [40.107.220.65]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EFA3122424E; Sat, 28 Jun 2025 07:43:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.220.65 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1751096594; cv=fail; b=qMhxxrWr2/+H96B1wNFQFFfFYhpbemZjCpozwWSu4YzFoy8hGcGMP6K1/hxV1wTdhoeci2iHKdgOefvk5fRdQRPlJy3pAbaQlmICPE9hCNdBpSwYqd49wZv2Cw6vngfZF9XMopo6jDUg37+Xzs/yhdaV5B2NZydf0lFmhpxrOAo= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1751096594; c=relaxed/simple; bh=DSUJWXZVAgg+0M4G2a7NSxbWRxWTNKCUe8W0PqXzOzY=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=DRZOB7DtTIX/b8P9dyLyjUIB5Gl1VCnrNFzDkevst5/xwnFs1s5zd8XlmeJCT3R15v91fsZizP15sVsH2bIkFdcIRXbwQdGajebxSaSIlGKs5mYZAt4jUIOF4rnr7ryJ12SOua1FOabeYzqEzx7IW+oJ9RBSp0LuuAH4B7Ikm/A= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=LhVmLBR3; arc=fail smtp.client-ip=40.107.220.65 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="LhVmLBR3" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=R0+kohvDRaRIxPymFiDUTGmfPnO7kWmwb/Q9DiryRNuwFdQnlx6FjLNVy4wY6roriOgZ3Y9fTmErjLIiPMO5LC33E3vsMz78TdbplD6jQwiQSPo/8Q7VXXYzNdMcRaltgeGi2ZgBvjmOK/7ZWsy1WHxJWMSJxKQeKypSOen3wJv7eRq3YXvHRy0DP6rQAT/+exG2PL/sAimVXVB2JlccO4/ad6vDrLqBswuAwBvb9X4b/tjTD8K5chwd1Z0ceYDTrPM9/wHQ7gSVSpGNKSyjhs+YJDG5z+LMJIYZKvftbxfEp3X1fvI1xbx+EdrlNYkISPndrIeI8uAHzs31zIIaBA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=W/jwPQRi4RYqn+THHKz9zgiZld7K8iajcbiMrsdXnmo=; b=bttyn1mbvq8dB18lolihFJsOxhpX4hcfY6u4OMQ3AnpLguBY2oFgQ+Q7VCiG/s48MaXxBwKv+mu4l0PlU8KC9oQdgVz6sXHOLInfMKTaaoHguXgfsAJKoHgvRpTNBudmWWFAEx5K2ldiAogFxL45YIkt1386c++A0OzhnYlXXkooqWZg0Rt4CZt4F269tFNWbidQj69lbFJc5AmxqBlON0jSZX+rXGwm6doDPfn3iVRdP316PGfWCwYh8kU0hMbWhPLcTlYFU+tkmRLC0XrWOmVJqWjN2HO2ysfF+y8GIy12uxTMdHtiIKNNlGKEeaI6cYJq6B/0SclWuRyJh/VTTQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=8bytes.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=W/jwPQRi4RYqn+THHKz9zgiZld7K8iajcbiMrsdXnmo=; b=LhVmLBR3dS0i5bWc1ssU0naJo/RTU7L6f0BqUWWfSVUzdYXNmrqAllzdYSm8lAZ52CO+odg+uauvq/Gkrf4X0nrfvN7N2CqMZngBJZ8VpkiO2AlM/j+BVV4X9QwYsWXEHvQe6ZA2HPbTvs5RgjDHpOxjqgv4osCAc2HBUiE6YyfC96BhZbsZ6Bi4FN5EvUy7R5G4t5UEF9BiTiPAULHidxhKDNMZEjHV+fdX9D3ZQJ/C7v0oYErNEEAEGlNDWp1WtXLX4InbYVaLYRt/LeWNRDfpB8meJu7iwPXAHD4lLlFlD54e2xGefWmtNJMV3uBPa+c7Y3z4Y/QTO7QKdsctTw== Received: from SA9PR13CA0063.namprd13.prod.outlook.com (2603:10b6:806:23::8) by MW4PR12MB7263.namprd12.prod.outlook.com (2603:10b6:303:226::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8880.22; Sat, 28 Jun 2025 07:43:07 +0000 Received: from SN1PEPF000252A3.namprd05.prod.outlook.com (2603:10b6:806:23:cafe::e9) by SA9PR13CA0063.outlook.office365.com (2603:10b6:806:23::8) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8901.7 via Frontend Transport; Sat, 28 Jun 2025 07:43:06 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by SN1PEPF000252A3.mail.protection.outlook.com (10.167.242.10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8880.14 via Frontend Transport; Sat, 28 Jun 2025 07:43:06 +0000 Received: from rnnvmail204.nvidia.com (10.129.68.6) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Sat, 28 Jun 2025 00:42:53 -0700 Received: from rnnvmail205.nvidia.com (10.129.68.10) by rnnvmail204.nvidia.com (10.129.68.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.14; Sat, 28 Jun 2025 00:42:53 -0700 Received: from Asurada-Nvidia.nvidia.com (10.127.8.13) by mail.nvidia.com (10.129.68.10) with Microsoft SMTP Server id 15.2.1544.14 via Frontend Transport; Sat, 28 Jun 2025 00:42:52 -0700 From: Nicolin Chen To: , , , , , , CC: , , , , , , , , Subject: [PATCH RFC v2 4/4] pci: Suspend iommu function prior to resetting a device Date: Sat, 28 Jun 2025 00:42:42 -0700 Message-ID: <7889db2790263640c6e9bb98956c3a3d55b87ee6.1751096303.git.nicolinc@nvidia.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: AnonymousSubmission X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SN1PEPF000252A3:EE_|MW4PR12MB7263:EE_ X-MS-Office365-Filtering-Correlation-Id: c910519f-fe3e-409c-dceb-08ddb6176bde X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|82310400026|36860700013|376014|7416014; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?HJ8OIdtPdaAWJ1ho3LhFz4SrMh5HgtsgFzvxnxY2YOlT4Pc/tr4c3ZqarIqm?= =?us-ascii?Q?pSf3W3TmHFLRVJ+MOwV+TJbDCidCPSihZVXvm9pBMoC4Z5PNUtJESj12xvu4?= =?us-ascii?Q?rlwpB/lPtMKQGn12aRzC+hVFe+peyMI/d4GCvDxmeSlyOmSghRAgxJgAT3Cy?= =?us-ascii?Q?Kb+YDeshlGNfZDvAeQnwwcpvF11zwoRMMznGbY6z++FP5dmB0+WgkeMd/R/U?= =?us-ascii?Q?B6BTAArOachcUP+fqNW3kToPWIdII2yKkWJnWZJTtVzb69kcr+9A2RF1vgNl?= =?us-ascii?Q?PA9ULG+CpYP1HeNrLfgZVSQkIalADDfT38RRyQWZ6wjAdBf7qRzjO4AnCkwB?= =?us-ascii?Q?NhA93oXNkEvKod5x/3OcENs5P96jNGIijg4d0z4olTKFwtS4bB+832Hspiha?= =?us-ascii?Q?8xQFwlBE2nfOxHea7DwNJC4UFEFhfBkd9dqgPfmvo2Xd81H9ld2SmUxGskFB?= =?us-ascii?Q?z5psXrtt4vAKnqL3r972wZY8jRc/KFDVs4KI0HBy3V8m/qGT/MzcJauxq81m?= =?us-ascii?Q?13zwloBB+ugII1l7rXt3fpvlDz8owENWvAlaZ6Kv6qC0tzeELnPa7dkUXe1Q?= =?us-ascii?Q?Hmint8QYO/B9WexT6cBPOJ7FGNFzZbRuzXpL+Wh+1LKOhcBY6k7lPidWOumq?= =?us-ascii?Q?kZgGtkLuxaneOUTuEPE6Bkn90z7ntEgIN6wyNJkHKCf0stcMN+lNTHOg2OSg?= =?us-ascii?Q?R7vYWqF4Ehlql4fIHj/LJaULXCxfuGYcTleubWYgObhuUu1P5aSiBrtT1w7x?= =?us-ascii?Q?JM7Saf6XJ2Uye0IR6Vglc9FnfQHl+5KNHlQ9dH6+mMMZB6XEeeUm4vX6DTSx?= =?us-ascii?Q?dUAk+9L341BAJrWtEDexVbImG0S/1vMrCn0O4COVPwsKXuSRafOTcbPvcAI4?= =?us-ascii?Q?9ettNIgaBwCbyWjI7Zyi0XEDIhNJmU5LdCwLjNFT+wUWrLlkN1jBBuaepgYK?= =?us-ascii?Q?R/NdWkcCOBPVBsmBDZb7bnv6O2Oc5b6QC32bQ6+qFP6gUxKTntJU23vUAVhs?= =?us-ascii?Q?pNv7yi0mAO4b6fKikHmuCBwOHeIKgP8BUdgNCjNFwm26aChVqdnAvnOEpJ0T?= =?us-ascii?Q?8F7pmPkVfwmhTHE1kjSW0JhBtT6OWKDJJi7Cv6D0drzpP4aexTbmS+fEo2Qs?= =?us-ascii?Q?QllNf+Y/8Jz4ZaV00WVH/SzSk8evld7nUvohtN8M4Igk4t8Re7PT6MdZa3y3?= =?us-ascii?Q?tCJQcf5vvsPdS1Ou1dCcNgHuoxElHjyNsY5zKp3qYo2/GKeuF6hckTrIvaLb?= =?us-ascii?Q?YvRWYoJSaJPvbZjSvcBhr7Y5lk4th4bnDbjgPpfbVNSQdkuCzZXs7AW1+QLX?= =?us-ascii?Q?96ZcjO7OUC2Q0xyS9iGK8fK/h0EraEkq9qBW5sggSKhCNPSYnxnGwyl0HI4Y?= =?us-ascii?Q?xLTnIGeXvt31DDKqy0lf2rawYFd36a5V/8I2xM4wlvTXLhVA/2G3XzrulKm7?= =?us-ascii?Q?kyc0TjQG4aqffH+9M5Ohpio6/4CzmQn80Wb9X1tIEB+bUZs1p0c0dEJ/fWl+?= =?us-ascii?Q?Af3hYUNmP9myBpiQV5tCBdJvItWOiRCA0n61?= X-Forefront-Antispam-Report: CIP:216.228.117.160;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge1.nvidia.com;CAT:NONE;SFS:(13230040)(1800799024)(82310400026)(36860700013)(376014)(7416014);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Jun 2025 07:43:06.5277 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: c910519f-fe3e-409c-dceb-08ddb6176bde X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: SN1PEPF000252A3.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW4PR12MB7263 Content-Type: text/plain; charset="utf-8" PCIe permits a device to ignore ATS invalidation TLPs, while processing a reset. This creates a problem visible to the OS where an ATS invalidation command will time out: e.g. an SVA domain will have no coordination with a reset event and can racily issue ATS invalidations to a resetting device. The PCIe spec in sec 10.3.1 IMPLEMENTATION NOTE recommends to disable and block ATS before initiating a Function Level Reset. It also mentions that other reset methods could have the same vulnerability as well. Now iommu_dev_reset_prepare/done() helpers are introduced for this matter. Use them in all the existing reset functions, which will attach the device to an IOMMU_DOMAIN_BLOCKED during a reset, so as to allow IOMMU driver to: - invoke pci_disable_ats() and pci_enable_ats() respectively - wait for all ATS invalidations to complete - stop issuing new ATS invalidations - fence any incoming ATS queries Add a warning if ATS isn't disabled, in which case IOMMU driver should fix itself to disable ATS following the design in iommu_dev_reset_prepare(). Signed-off-by: Nicolin Chen --- drivers/pci/pci-acpi.c | 21 ++++++++++- drivers/pci/pci.c | 84 +++++++++++++++++++++++++++++++++++++++--- drivers/pci/quirks.c | 27 +++++++++++++- 3 files changed, 124 insertions(+), 8 deletions(-) diff --git a/drivers/pci/pci-acpi.c b/drivers/pci/pci-acpi.c index b78e0e417324..727957f193ca 100644 --- a/drivers/pci/pci-acpi.c +++ b/drivers/pci/pci-acpi.c @@ -9,6 +9,7 @@ =20 #include #include +#include #include #include #include @@ -974,6 +975,7 @@ void pci_set_acpi_fwnode(struct pci_dev *dev) int pci_dev_acpi_reset(struct pci_dev *dev, bool probe) { acpi_handle handle =3D ACPI_HANDLE(&dev->dev); + int ret =3D 0; =20 if (!handle || !acpi_has_method(handle, "_RST")) return -ENOTTY; @@ -981,12 +983,27 @@ int pci_dev_acpi_reset(struct pci_dev *dev, bool prob= e) if (probe) return 0; =20 + /* + * Per PCIe r6.3, sec 10.3.1 IMPLEMENTATION NOTE, software disables ATS + * before initiating a reset. Notify the iommu driver that enabled ATS. + */ + ret =3D iommu_dev_reset_prepare(&dev->dev); + if (ret) { + pci_err(dev, "failed to stop IOMMU\n"); + return ret; + } + + /* Something wrong with the iommu driver that failed to disable ATS */ + if (dev->ats_enabled) + pci_err(dev, "failed to stop ATS. ATS invalidation may time out\n"); + if (ACPI_FAILURE(acpi_evaluate_object(handle, "_RST", NULL, NULL))) { pci_warn(dev, "ACPI _RST failed\n"); - return -ENOTTY; + ret =3D -ENOTTY; } =20 - return 0; + iommu_dev_reset_done(&dev->dev); + return ret; } =20 bool acpi_pci_power_manageable(struct pci_dev *dev) diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c index e9448d55113b..ddb7a10ef500 100644 --- a/drivers/pci/pci.c +++ b/drivers/pci/pci.c @@ -13,6 +13,7 @@ #include #include #include +#include #include #include #include @@ -4518,13 +4519,30 @@ EXPORT_SYMBOL(pci_wait_for_pending_transaction); */ int pcie_flr(struct pci_dev *dev) { + int ret =3D 0; + if (!pci_wait_for_pending_transaction(dev)) pci_err(dev, "timed out waiting for pending transaction; performing func= tion level reset anyway\n"); =20 + /* + * Per PCIe r6.3, sec 10.3.1 IMPLEMENTATION NOTE, software disables ATS + * before initiating a reset. Notify the iommu driver that enabled ATS. + * Have to call it after waiting for pending DMA transaction. + */ + ret =3D iommu_dev_reset_prepare(&dev->dev); + if (ret) { + pci_err(dev, "failed to stop IOMMU\n"); + return ret; + } + + /* Something wrong with the iommu driver that failed to disable ATS */ + if (dev->ats_enabled) + pci_err(dev, "failed to stop ATS. ATS invalidation may time out\n"); + pcie_capability_set_word(dev, PCI_EXP_DEVCTL, PCI_EXP_DEVCTL_BCR_FLR); =20 if (dev->imm_ready) - return 0; + goto done; =20 /* * Per PCIe r4.0, sec 6.6.2, a device must complete an FLR within @@ -4533,7 +4551,11 @@ int pcie_flr(struct pci_dev *dev) */ msleep(100); =20 - return pci_dev_wait(dev, "FLR", PCIE_RESET_READY_POLL_MS); + ret =3D pci_dev_wait(dev, "FLR", PCIE_RESET_READY_POLL_MS); + +done: + iommu_dev_reset_done(&dev->dev); + return ret; } EXPORT_SYMBOL_GPL(pcie_flr); =20 @@ -4561,6 +4583,7 @@ EXPORT_SYMBOL_GPL(pcie_reset_flr); =20 static int pci_af_flr(struct pci_dev *dev, bool probe) { + int ret =3D 0; int pos; u8 cap; =20 @@ -4587,10 +4610,25 @@ static int pci_af_flr(struct pci_dev *dev, bool pro= be) PCI_AF_STATUS_TP << 8)) pci_err(dev, "timed out waiting for pending transaction; performing AF f= unction level reset anyway\n"); =20 + /* + * Per PCIe r6.3, sec 10.3.1 IMPLEMENTATION NOTE, software disables ATS + * before initiating a reset. Notify the iommu driver that enabled ATS. + * Have to call it after waiting for pending DMA transaction. + */ + ret =3D iommu_dev_reset_prepare(&dev->dev); + if (ret) { + pci_err(dev, "failed to stop IOMMU\n"); + return ret; + } + + /* Something wrong with the iommu driver that failed to disable ATS */ + if (dev->ats_enabled) + pci_err(dev, "failed to stop ATS. ATS invalidation may time out\n"); + pci_write_config_byte(dev, pos + PCI_AF_CTRL, PCI_AF_CTRL_FLR); =20 if (dev->imm_ready) - return 0; + goto done; =20 /* * Per Advanced Capabilities for Conventional PCI ECN, 13 April 2006, @@ -4600,7 +4638,11 @@ static int pci_af_flr(struct pci_dev *dev, bool prob= e) */ msleep(100); =20 - return pci_dev_wait(dev, "AF_FLR", PCIE_RESET_READY_POLL_MS); + ret =3D pci_dev_wait(dev, "AF_FLR", PCIE_RESET_READY_POLL_MS); + +done: + iommu_dev_reset_done(&dev->dev); + return ret; } =20 /** @@ -4621,6 +4663,7 @@ static int pci_af_flr(struct pci_dev *dev, bool probe) static int pci_pm_reset(struct pci_dev *dev, bool probe) { u16 csr; + int ret; =20 if (!dev->pm_cap || dev->dev_flags & PCI_DEV_FLAGS_NO_PM_RESET) return -ENOTTY; @@ -4635,6 +4678,20 @@ static int pci_pm_reset(struct pci_dev *dev, bool pr= obe) if (dev->current_state !=3D PCI_D0) return -EINVAL; =20 + /* + * Per PCIe r6.3, sec 10.3.1 IMPLEMENTATION NOTE, software disables ATS + * before initiating a reset. Notify the iommu driver that enabled ATS. + */ + ret =3D iommu_dev_reset_prepare(&dev->dev); + if (ret) { + pci_err(dev, "failed to stop IOMMU\n"); + return ret; + } + + /* Something wrong with the iommu driver that failed to disable ATS */ + if (dev->ats_enabled) + pci_err(dev, "failed to stop ATS. ATS invalidation may time out\n"); + csr &=3D ~PCI_PM_CTRL_STATE_MASK; csr |=3D PCI_D3hot; pci_write_config_word(dev, dev->pm_cap + PCI_PM_CTRL, csr); @@ -4645,7 +4702,9 @@ static int pci_pm_reset(struct pci_dev *dev, bool pro= be) pci_write_config_word(dev, dev->pm_cap + PCI_PM_CTRL, csr); pci_dev_d3_sleep(dev); =20 - return pci_dev_wait(dev, "PM D3hot->D0", PCIE_RESET_READY_POLL_MS); + ret =3D pci_dev_wait(dev, "PM D3hot->D0", PCIE_RESET_READY_POLL_MS); + iommu_dev_reset_done(&dev->dev); + return ret; } =20 /** @@ -5100,6 +5159,20 @@ static int cxl_reset_bus_function(struct pci_dev *de= v, bool probe) if (rc) return -ENOTTY; =20 + /* + * Per PCIe r6.3, sec 10.3.1 IMPLEMENTATION NOTE, software disables ATS + * before initiating a reset. Notify the iommu driver that enabled ATS. + */ + rc =3D iommu_dev_reset_prepare(&dev->dev); + if (rc) { + pci_err(dev, "failed to stop IOMMU\n"); + return rc; + } + + /* Something wrong with the iommu driver that failed to disable ATS */ + if (dev->ats_enabled) + pci_err(dev, "failed to stop ATS. ATS invalidation may time out\n"); + if (reg & PCI_DVSEC_CXL_PORT_CTL_UNMASK_SBR) { val =3D reg; } else { @@ -5114,6 +5187,7 @@ static int cxl_reset_bus_function(struct pci_dev *dev= , bool probe) pci_write_config_word(bridge, dvsec + PCI_DVSEC_CXL_PORT_CTL, reg); =20 + iommu_dev_reset_done(&dev->dev); return rc; } =20 diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c index d7f4ee634263..7a66c01392d9 100644 --- a/drivers/pci/quirks.c +++ b/drivers/pci/quirks.c @@ -21,6 +21,7 @@ #include #include /* isa_dma_bridge_buggy */ #include +#include #include #include #include @@ -4223,6 +4224,30 @@ static const struct pci_dev_reset_methods pci_dev_re= set_methods[] =3D { { 0 } }; =20 +static int __pci_dev_specific_reset(struct pci_dev *dev, bool probe, + const struct pci_dev_reset_methods *i) +{ + int ret; + + /* + * Per PCIe r6.3, sec 10.3.1 IMPLEMENTATION NOTE, software disables ATS + * before initiating a reset. Notify the iommu driver that enabled ATS. + */ + ret =3D iommu_dev_reset_prepare(&dev->dev); + if (ret) { + pci_err(dev, "failed to stop IOMMU\n"); + return ret; + } + + /* Something wrong with the iommu driver that failed to disable ATS */ + if (dev->ats_enabled) + pci_err(dev, "failed to stop ATS. ATS invalidation may time out\n"); + + ret =3D i->reset(dev, probe); + iommu_dev_reset_done(&dev->dev); + return ret; +} + /* * These device-specific reset methods are here rather than in a driver * because when a host assigns a device to a guest VM, the host may need @@ -4237,7 +4262,7 @@ int pci_dev_specific_reset(struct pci_dev *dev, bool = probe) i->vendor =3D=3D (u16)PCI_ANY_ID) && (i->device =3D=3D dev->device || i->device =3D=3D (u16)PCI_ANY_ID)) - return i->reset(dev, probe); + return __pci_dev_specific_reset(dev, probe, i); } =20 return -ENOTTY; --=20 2.43.0