From nobody Sat Sep 26 13:08:18 2026 Received: from SN4PR0501CU005.outbound.protection.outlook.com (mail-southcentralusazon11011041.outbound.protection.outlook.com [40.93.194.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F1002479899; Tue, 1 Sep 2026 09:33:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.93.194.41 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788255233; cv=fail; b=ttSkXCsgLd9uPPX+z65nF+eKv1q1LbgN2p91G1xChJ2E6pIB/TscfDPwxOxKm5nQubpXvEKv+GckcNCFR6cnXrs59lbXPJs8uJHvgTpAemROeV4Dy4pX4GaiN65XcwR2VitHjjL5uWIGah/eBi+MQjQAqaDbE9bQVS2d1cIqyDU= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788255233; c=relaxed/simple; bh=oKfis5IYjBpNMMKH/3X08d15gbhbFzJbIFMTe1Nfnp8=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=P0NmyhqnhAd8rnDlRRyC+UA7yV3ZHRfwiNfDdlMIoRxjtGk8CXSTkwMDcccqcrTnFjGIDw8cfv723m32RtmnHtBhDE14gS1fFL/OOt9HmZDdY1T/F8pzauOBa1MqppXdy5YP1M3kpiwxK7PCzogInC9pDzZgRb3gylcexnkTxi0= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=Sy9yFWKn; arc=fail smtp.client-ip=40.93.194.41 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="Sy9yFWKn" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=vcAYOqqTDy9DOAuNuYsCxEDli1p+0rnIgQDt4Iqus/woUBTdqvedA7U+HkRV2Bx3QWSKVLseVVpoSFhQshQfbtqWTJdDeXTYJDfFdBKuVKsny6Thy2DUP6Jb76N9Gqe0hO2JYcIqSVs0NFxKrxYWFFcLSE1wxEZoaxYXwX9acWFLKqdPXVzr9OIgWr8wRdAzfX2UUTpgmhqZhbqmkot3ouU5PGX0U20SOlGKCnob7El09kuVEirF0i8vwoyA+AZB8tIum5ovTuUoBRMCMon8M81Pyc5rupOq9pz+MwpasVadO9tu0PM3R5Qx8UAFC4cNPIOSIgS0RsvIOPFVWrJyjA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=YsVHDVIQ7iJptVDDyKc8hVRL/XSx8/O8tcF+kGZBAqY=; b=vaM61EfzCKBYluXsIcspFo0+ozBpodVMRgZ3qMYB+ZvZe/0tnWESXAgzZC0SrvBEP11/LC90+DxcZgLECjE8MXK8E35NcRU1TtmhoS4f8xefKtOXv+wDEZ0xCVG5hnSSgmu6pFCdbAmRY+cpu4xGdXDvhHwoK299eTtETDCy1DGFAvGiR/xO/3WrvsaycMwYrC9jeP4iZMGjcwGGsTBtM79PBdi3Qiz6WIQK/FqmnEywtGGKXk4PUv6d+/HCMSahFajzvGPOURv/nbGUjjNk617oGKGGbrO9WyBwLigwVqr7E8TE6DKTzgODhJp+8z8wJCvhkw2IOCRoOfz6T6VXvQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=YsVHDVIQ7iJptVDDyKc8hVRL/XSx8/O8tcF+kGZBAqY=; b=Sy9yFWKngMD2RrhKybaAHzZ9pxNIBB/35+PMJ8aa8sM52Nj9wodC09vGFPi2H3twnapm5TdHXT7odu3o7MrEqhACZVan5zWeDyrnLg0f4DE/1Jy0MoZA1QgMCHi8kM1GYv/qEzYq0ENXHyiLzm17rQ1J3oTViQEj1Lb+gLAY5PdvdeYq62U9yD74fyD8UUHFmhxAcWhADCAYloYhtnC0cdrR4FfuC3Eg7bxiOqjNQuJwEcX4vUYY3Iu7m2NTZDj76vqDUKEOZa1XfwJn+FcHqfRCJrKQCXd0tUvpmpflcvr/aK/XAlaRh0EftFzOfA6XkkKy79OFI98FnMXGJ2uq0A== Received: from BN9PR03CA0219.namprd03.prod.outlook.com (2603:10b6:408:f8::14) by PH7PR12MB6934.namprd12.prod.outlook.com (2603:10b6:510:1b8::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.360.12; Tue, 1 Sep 2026 09:33:35 +0000 Received: from BL02EPF00021F6D.namprd02.prod.outlook.com (2603:10b6:408:f8:cafe::28) by BN9PR03CA0219.outlook.office365.com (2603:10b6:408:f8::14) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.360.13 via Frontend Transport; Tue, 1 Sep 2026 09:33:35 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by BL02EPF00021F6D.mail.protection.outlook.com (10.167.249.9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.382.8 via Frontend Transport; Tue, 1 Sep 2026 09:33:34 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.46; Tue, 1 Sep 2026 02:33:16 -0700 Received: from NV-2Y5XW94.nvidia.com (10.126.230.37) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.46; Tue, 1 Sep 2026 02:33:13 -0700 From: Shameer Kolothum To: , , CC: , , , , , , , , Subject: [RFC PATCH 01/19] vfio/pci: Add PCI error recovery support state Date: Tue, 1 Sep 2026 10:31:59 +0100 Message-ID: <20260901093217.8539-2-skolothumtho@nvidia.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260901093217.8539-1-skolothumtho@nvidia.com> References: <20260901093217.8539-1-skolothumtho@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: rnnvmail203.nvidia.com (10.129.68.9) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL02EPF00021F6D:EE_|PH7PR12MB6934:EE_ X-MS-Office365-Filtering-Correlation-Id: ec0a3515-53e5-4782-4820-08df080c184b X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|23010399003|376014|36860700016|1800799024|6133799003|10067099003|56012099006|11063799006|5023799004|18002099003|22082099003; X-Microsoft-Antispam-Message-Info: DO/qjpHXVsXSDHkuCkrO5nr2d/0QRGDL/bFFh4WAJQpj4LrHjGlfwgCaBG48I0XhD/4bZGX7PrkawTBYpAK4tHsPC4ACvjwpZ8yc0V7vf0Zffq2c+Mio2R8jc19sMshAMVQM0KvGr2dltrXePb1/A9OMi+Yn0pN0KUTG/TD2rx5gzh90PAVebZ6huof/ybZ93NcsXONdS4qJATyq2Bb/fbyPdon/PbpSK6aQnHMezGVCNI0rs/WahLuNBW4/ppo7ZjGxAgYD4OuNF+1hT1SFMIHxNbJfiJAJ/m4+Hrc+x+mC5wZnsAz41y7IqQQmPntiXXs6Y4xIn1x91XSgcefPgOdqm5abS1NxHTmbJsnNxr/8j53jwqII+ot3K0Cvgn2REPUsO/4cFwHG4Bx7/KNDPLBk/Y1uI+x7OyiAMt8h3bizyaHS2kdWxj9bCVJEWZXevGa8yN+75K+t8YnJeAapBqY+72qzlLpqiwQAulDuWwKvS+YInmroETlHMrTFCWWRHYPx2+/jneh8ANHQ+AgQCji26bjAuLFZuClAeaeNi5xsccEXMr/zgirKnyQq3V1M0mL6oXHPsdoae0lQWi9iypA7c+dAKarN0in8Q69Nt1reot2vk6e2R4h/6nm8CGIDchZA1K2eOBlAz/PteQAW9heeNIn2WreqadDKidCBuoktbGv8jzrn2NU0YWKnWTOWwMcExHb8vEDilSBvqErFkQ== X-Forefront-Antispam-Report: CIP:216.228.117.160;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge1.nvidia.com;CAT:NONE;SFS:(13230040)(82310400026)(23010399003)(376014)(36860700016)(1800799024)(6133799003)(10067099003)(56012099006)(11063799006)(5023799004)(18002099003)(22082099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: qJGVDH1McRTLZeupKBSsqbBffnBf8M1wuWhcCNWPsgxB3n73R3EZtIylASDPxxOJxVRvdWDg8mB/Hfe8aTAeHp2xY2UuxKeeqzZYlH1aQTlLqyf9AseE/GT5UJeuHPmKT57/jKBlur2Ob9WWXGdd5Vo4Ft1DMRYlS8L/lbJIPYQ0OkTyZMbGmJODvfG/HESoGjTEJq1PKB4Ng8P+hxDK0wMwNv7bz3zfmTlVc/QHeitGtokvMtkiyJx76pUwPyKlDUfgGo/xpsiWOmvouF2i/zreKc4eVsLrRpqKJH49t0J8X486lfoO5WGAHkEEig2ZnURV/TpFd2BQSI/U0FuaHDLsMXQqEEulMLz2nc8pqXgkuq0GJwKnzPrPcEmLiZUQd7nBshRvIC6x2CCqDda5SizVsGgcyAoobXn5UKNtMRPoUn2AZ+DmDlOb/EuXkAcM X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Sep 2026 09:33:34.8197 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: ec0a3515-53e5-4782-4820-08df080c184b X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BL02EPF00021F6D.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB6934 Content-Type: text/plain; charset="utf-8" Add the state, the lock and the wait queue that the rest of the series needs. Nothing uses any of it yet. The lifecycle and the access guards come next, the recovery callbacks later. The flags word records what happened in one event. IN_PROGRESS while recovery is running, FROZEN if the channel was frozen, RESET if the host reset the device, and FAILED if it did not recover. These are internal. A later patch reports the same set to userspace along with a sequence number, so an event can be told from the one before it. Where both locks are held, recovery_lock goes outside memory_lock. Recovery has to shut out new device access and wait for whatever is already running before it can take memory_lock and revoke the BAR mappings. The other way round deadlocks. An access path sits on memory_lock waiting for recovery_lock, while recovery sits on recovery_lock waiting for memory_lock. So nothing may take recovery_lock when it already holds memory_lock. Only the generic vfio-pci driver advertises support, so variant drivers carry on as they do now. Userspace activation is a separate switch, so even on generic vfio-pci nothing changes until userspace asks for it. Signed-off-by: Shameer Kolothum --- include/linux/vfio_pci_core.h | 59 ++++++++++++++++++++++++++++++++ drivers/vfio/pci/vfio_pci.c | 1 + drivers/vfio/pci/vfio_pci_core.c | 2 ++ 3 files changed, 62 insertions(+) diff --git a/include/linux/vfio_pci_core.h b/include/linux/vfio_pci_core.h index 9a1674c152aa..42a77ed6b93c 100644 --- a/include/linux/vfio_pci_core.h +++ b/include/linux/vfio_pci_core.h @@ -95,6 +95,11 @@ static inline int vfio_pci_core_get_dmabuf_phys( } #endif =20 +#define VFIO_PCI_RECOVERY_IN_PROGRESS BIT(0) +#define VFIO_PCI_RECOVERY_FROZEN BIT(1) +#define VFIO_PCI_RECOVERY_RESET BIT(2) +#define VFIO_PCI_RECOVERY_FAILED BIT(3) + struct vfio_pci_core_device { struct vfio_device vdev; struct pci_dev *pdev; @@ -129,6 +134,7 @@ struct vfio_pci_core_device { bool disable_idle_d3:1; bool nointxmask:1; bool disable_vga:1; + bool pci_recovery_supported:1; /* Flags modified at runtime - dedicated storage unit */ bool needs_reset; bool pm_intx_masked; @@ -147,7 +153,60 @@ struct vfio_pci_core_device { struct list_head sriov_pfs_item; struct vfio_pci_core_device *sriov_pf_core_dev; struct notifier_block nb; + /* + * Serializes host PCI error recovery with device access and the + * open/close lifecycle. recovery_lock nests outside memory_lock. + */ + struct rw_semaphore recovery_lock; struct rw_semaphore memory_lock; + /* + * PCI error recovery state, written under recovery_lock held for + * writing except where noted. + * + * Some readers cannot take recovery_lock. An interrupt handler cannot + * sleep. The ioeventfd write runs on a workqueue which is flushed with + * the lock held, so it would block behind a queued writer. A + * wait-queue condition cannot take it either. Some checks would refuse + * work against ordinary device traffic if they took it. Work deferred + * past the guard has none to take, since it reaches pci_bus_sem. + * + * Those readers use READ_ONCE() on pci_recovery_flags, + * pci_recovery_enabled, pci_recovery_access_blocked and + * pci_recovery_device_open instead. All of them fail safe. A stale + * read costs an extra refusal or retry, never an unguarded access. + * + * recovery_lock does not exclude those readers, so publish each field + * with a single store of its final value rather than clearing and + * then setting. That keeps the states a reader can observe to ones + * which are meaningful on their own. + * + * device_open and access_blocked mean different things. device_open + * says the device is open and its per-open state, vconfig included, + * is allocated. access_blocked says a recovery or reset is blocking + * access right now. + * + * access_blocked is only ever set while device_open is set. Nothing + * sets it without testing device_open first under recovery_lock, and + * close clears access_blocked before it clears device_open, so a + * block never outlives the open which created it. A reader which + * finds access_blocked set can rely on the per-open state being + * there with it. + */ + u32 pci_recovery_flags; + u64 pci_recovery_sequence; + /* PCI_COMMAND value saved before recovery quiesces the device. */ + u16 pci_recovery_command; + /* Userspace enabled recovery for this device open. */ + bool pci_recovery_enabled; + /* pci_recovery_command contains a restorable value. */ + bool pci_recovery_command_valid; + /* A recovery or reset transaction is blocking physical access. */ + bool pci_recovery_access_blocked; + /* Device initialization completed and close teardown has not started. */ + bool pci_recovery_device_open; + /* May be set while recovery_lock is held for reading during ROM unmap. */ + bool pci_recovery_rom_disable; + wait_queue_head_t pci_recovery_wait; struct list_head dmabufs; }; =20 diff --git a/drivers/vfio/pci/vfio_pci.c b/drivers/vfio/pci/vfio_pci.c index 830369ff878d..46544dbe70d0 100644 --- a/drivers/vfio/pci/vfio_pci.c +++ b/drivers/vfio/pci/vfio_pci.c @@ -139,6 +139,7 @@ static int vfio_pci_init_dev(struct vfio_device *core_v= dev) */ vdev->nointxmask =3D nointxmask; vdev->disable_idle_d3 =3D disable_idle_d3; + vdev->pci_recovery_supported =3D true; #ifdef CONFIG_VFIO_PCI_VGA vdev->disable_vga =3D disable_vga; #endif diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_c= ore.c index 6757054e9d87..e0be5ddf7039 100644 --- a/drivers/vfio/pci/vfio_pci_core.c +++ b/drivers/vfio/pci/vfio_pci_core.c @@ -2196,7 +2196,9 @@ int vfio_pci_core_init_dev(struct vfio_device *core_v= dev) if (ret && ret !=3D -EOPNOTSUPP) return ret; INIT_LIST_HEAD(&vdev->dmabufs); + init_rwsem(&vdev->recovery_lock); init_rwsem(&vdev->memory_lock); + init_waitqueue_head(&vdev->pci_recovery_wait); xa_init(&vdev->ctx); =20 return 0; --=20 2.43.0 From nobody Sat Sep 26 13:08:18 2026 Received: from SN4PR2101CU001.outbound.protection.outlook.com (mail-southcentralusazon11012012.outbound.protection.outlook.com [40.93.195.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5346E476066; Tue, 1 Sep 2026 09:33:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.93.195.12 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788255237; cv=fail; b=CtvQDGLDzWLP359YyoIGMkxCFv58XWsePKHAkwGfR4qTS/72kGFMb2vV10N6TXg6QYwy6NZ+NJFYjj7OsQQJkwbLF7tKeKQPB3baYExqLKCwHhogVSMb5DfKpJfs1yZ84s3exKDnpMCV4djAzumUVh6miqez4oGBmKRHwv7jWoI= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788255237; c=relaxed/simple; bh=kaZXVrrCP2U9Ug8O9yMNnVLRKWzQf/F6fTgHO2aMYes=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=RW37JxrRa1I1HUQb9r9/yx/Um4Q1kaFJR+BWuAilh9Yj/gNWKb88o+92SbhK9p7PxhKF4BZopwBMWlRvtI312L9fmfN/AvinN3qrkUCFHvbgJruSrNe0op7pnwhU4eQWpVdvSB1H7hiBFyuLV/GQJ6uZu0KaHhrnqm5/cfwW5co= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=t1kokl6R; arc=fail smtp.client-ip=40.93.195.12 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="t1kokl6R" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=nwyEQ0Aw3mXW7qllkd7a2WLnk/SRgDoVrByspaXEXoDlhn8FkVti0L0zj1yNpD9EYdpfFvV2ruDion8bavr7iCA6JB9gaMvbtav0ay/C0LgpCij7sFDECDAYwYPgYcDzacqOyUf6/eoTGzxZ3ZLKy2cgXCeX5BQB3Ji6ES6SRDwAv6NIxkNxLb2I6hCLEQwfdkfnKJSafCCciUEHkRO5CCPPkA7dmTqn8bQ0JbjONFgK/xFs7t6lETj+3uxE+WMcfdsCi31r3qG1vv5kb/OZkDlXI2bp3upo6+7bSHrWZgKKSj5VtMqTvi/ExZ9j+j8oraOWZ+SbqARl0UZWAMZbEw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=eTcGDI1SipXcb5nQctScIiUvyi7LV3lMLuxyR56YF0w=; b=TpunAM4Uq5qZ2E6OpgDtGQX5we5Hvi+fHNwlrSGOVmoex7/WZm0P3WFFauZT3ZfKeSBakMKIGkQEW0AGHRlLTHPdmZe8Ge26m6akm4ME9lgjDaTzXO2BP/ps6aps/Fp2OUZHKJJJwquHv7NVDCsDmvhYLttljNCe/3kKwWleq1Man1b7azLfeQw6YvaXDXxGlsXf5i1dvV2g/hLApjgFvKus7RGflErYdabrG00nZuK14aezdJ8ZjA5PwI+meoxdhwqK9ySLyA6zwsLMHSVNf6Sxy9ZHCeTR9sm1x5DKtVd5CPJOC25YKBVuMTYAxUsS+aWnYB0G544SvvQ5HDeKvA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=eTcGDI1SipXcb5nQctScIiUvyi7LV3lMLuxyR56YF0w=; b=t1kokl6R0cXTiw9FAb4jkRdXPmjHdrr4L+txwHDj/4yp/csAcTF23VIGVodSFexcBhWpA872zB3cDQkCqGprTiYEgkUQEfTK+IaakiulPq+8ahydsM7znUYeuwr19CIKdh9FQCjcXC0OLNpU3QkbXEHQKLZqgSWWLZGHDqKgx4rCYs+PQaaA2j4/XHZfupn2OmVaCUrHiVV6NQfWHN33kPPAcVeWLi7nRtRB1nS4JR5HoNErcPhhIBalydjqraqffr1KoZAD7YZNupDFspMRe46Btwpt6lPtKRUoBMG91LMz5/Ob4gYAwY7Eu5Zjt/twBK18OZ8xE5cqldhi7eAWbA== Received: from BN0PR04CA0126.namprd04.prod.outlook.com (2603:10b6:408:ed::11) by SN7PR12MB6983.namprd12.prod.outlook.com (2603:10b6:806:261::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.315.17; Tue, 1 Sep 2026 09:33:39 +0000 Received: from BL02EPF00021F69.namprd02.prod.outlook.com (2603:10b6:408:ed:cafe::58) by BN0PR04CA0126.outlook.office365.com (2603:10b6:408:ed::11) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.360.13 via Frontend Transport; Tue, 1 Sep 2026 09:33:39 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by BL02EPF00021F69.mail.protection.outlook.com (10.167.249.5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.382.8 via Frontend Transport; Tue, 1 Sep 2026 09:33:39 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.46; Tue, 1 Sep 2026 02:33:21 -0700 Received: from NV-2Y5XW94.nvidia.com (10.126.230.37) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.46; Tue, 1 Sep 2026 02:33:18 -0700 From: Shameer Kolothum To: , , CC: , , , , , , , , Subject: [RFC PATCH 02/19] vfio/pci: Serialize generic device lifetime with recovery Date: Tue, 1 Sep 2026 10:32:00 +0100 Message-ID: <20260901093217.8539-3-skolothumtho@nvidia.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260901093217.8539-1-skolothumtho@nvidia.com> References: <20260901093217.8539-1-skolothumtho@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: rnnvmail203.nvidia.com (10.129.68.9) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL02EPF00021F69:EE_|SN7PR12MB6983:EE_ X-MS-Office365-Filtering-Correlation-Id: 9f964b94-df0d-44e0-20b7-08df080c1b09 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|1800799024|36860700016|23010399003|82310400026|11063799006|10067099003|5023799004|6133799003|22082099003|18002099003|56012099006; X-Microsoft-Antispam-Message-Info: WOc7WAzZNIGEeXvHUBXocUmqvXdDZKYiiOLaPMeI0qWy/ZaQ3mbImHLJDiQ8yyr+a8t8jQupo6WB/gMGVrXGE5ODgwLQ5baydvlyA407rz31X6hi2yhm2tCVekXeV0UBKqBYL5LqvkhUcBqLNnosHC78vNuVCDluz6Sut+JOaZYO6bczvNRzIlP5QfJLWc4v2s4BHIYnlKeW2JcGsG2k7wYS1TtnPKObjwl+vq37LR0ubL59Xnik5r4LGUSieintzuTf5eV8J6ZMHs92DBR53ws9Ubs3gTzdBK16H1OcCmA43DyPNR6vQByJjPfFvUAK7mVPdsTAuCZz3h+PJ5E3ulafjmF5GbzxQ47LILObFvlbUCV2uw1Hdpw9E8/xuvsj86alskI5UTNQsxcmyvRFjRqNx5axUXPO1SaV4cRaSaTPVOA2Kc19AbIo2oQ5qb37O9FpOTfwZgxahMQ+IKN8DuPCguNkwqKrendUozWQvXcNuPy3dXoOkie7XCy2uNNThAXuyzVuuFMp/VMDem8aB+Z6OIiT4pGYaWjLUWFILXuJkJ6E+1s2w+D+o6K5rBEZWMTrpZLhcEwSGJpb01j2wB6eVSyagcl5Wmt4AJkZWCAxmTnVkBlBZTH4rwGWqgI/eDq9niRJ+HcYJ/5dP9Q/J7647Kcxp0LbItmy0xum6VDxN17XFi3PP29wC4ceQQTbcnkWF6YsWFcekMBu6jyhpg== X-Forefront-Antispam-Report: CIP:216.228.117.160;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge1.nvidia.com;CAT:NONE;SFS:(13230040)(376014)(1800799024)(36860700016)(23010399003)(82310400026)(11063799006)(10067099003)(5023799004)(6133799003)(22082099003)(18002099003)(56012099006);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 8v/Gi/DFb/+7mNVYOiwIxEQXttehG0BOq6ADh6jbBuFuao9WEe2Ai+raAJa0A0eZQQyH8ed99S/k6NKJaxUk/0KSWdFiSdesUFkICT3PJZIwxu7KjUstBhr66Ccg9LTIAilpgdSE5Xh2d6mRrn33QV7fqiswYEdy/84H+UwAXOaPsdDGkI3fco/FZFtvVJyil3p/rDsTFd9lchYeyoUaOV11KahtjzHc9JsBMSodw1IC7mAwn1kLZ6jKeitZrZg4NZI6xrfzpppDTjvpp+itq4Gtm69p6B+AJEQf3YvmQSZaQLFbeFjKfqPgmvRqIH6yNs0g1TqX1mYJcI9kVjmfhtOeAdZPiycp5eKLBaz8nZEZDn1R8P+RuwvPX+2aS2p0aYjwfowO/tFtIejVfeVau3rXJYmTWuO0L3zxOv/+o1aG3ScpMLV/TVcf9m6UdKuP X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Sep 2026 09:33:39.3665 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 9f964b94-df0d-44e0-20b7-08df080c1b09 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BL02EPF00021F69.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN7PR12MB6983 Content-Type: text/plain; charset="utf-8" vfio_pci_core_disable() frees vconfig while holding only the vfio device_set mutex. The PCI error callbacks never take that one. They run under the PCI device_lock instead, and vfio's close path does not hold that. So a callback still running when close starts can walk into state which is being freed. Publish a device_open flag under recovery_lock. enable() clears it before it touches the device, finish_enable() sets it once vfio_config_init() has allocated vconfig, and prepare_close() clears it again before the teardown frees vconfig. All three take recovery_lock for writing, so a callback either gets there first and close waits for it, or it finds the flag clear and does nothing. The access guards added later test the same flag. recovery_lock is not held across vfio_pci_core_disable(). A later patch has error_detected() take it from under pci_bus_sem, and disable() gets to pci_reset_bus(), which takes pci_bus_sem the other way round. access_blocked is only ever set while device_open is set. Nothing sets it without testing device_open first, and close clears access_blocked before it clears device_open. If close left it set, nothing could clear it afterwards. The transaction which set it cannot clear it once device_open is gone, and every path which refuses work on a blocked device would go on refusing. Clear it before device_open so a lock-free reader never sees it set on a device which is closed. open() now refuses a disconnected device with -ENODEV. That is new. Signed-off-by: Shameer Kolothum --- drivers/vfio/pci/vfio_pci_core.c | 69 +++++++++++++++++++++++++++++++- 1 file changed, 68 insertions(+), 1 deletion(-) diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_c= ore.c index e0be5ddf7039..8de586e4bb73 100644 --- a/drivers/vfio/pci/vfio_pci_core.c +++ b/drivers/vfio/pci/vfio_pci_core.c @@ -591,10 +591,23 @@ static const struct dev_pm_ops vfio_pci_core_pm_ops = =3D { int vfio_pci_core_enable(struct vfio_pci_core_device *vdev) { struct pci_dev *pdev =3D vdev->pdev; + bool supported =3D vdev->pci_recovery_supported; int ret; u16 cmd; u8 msix_pos; =20 + if (supported) { + down_write(&vdev->recovery_lock); + if (pci_dev_is_disconnected(pdev)) { + up_write(&vdev->recovery_lock); + return -ENODEV; + } + + vdev->pci_recovery_command_valid =3D false; + WRITE_ONCE(vdev->pci_recovery_device_open, false); + up_write(&vdev->recovery_lock); + } + if (!vdev->disable_idle_d3) { ret =3D pm_runtime_resume_and_get(&pdev->dev); if (ret < 0) @@ -815,7 +828,40 @@ void vfio_pci_core_disable(struct vfio_pci_core_device= *vdev) } EXPORT_SYMBOL_GPL(vfio_pci_core_disable); =20 -void vfio_pci_core_close_device(struct vfio_device *core_vdev) +static void vfio_pci_core_prepare_close(struct vfio_pci_core_device *vdev) +{ + if (!vdev->pci_recovery_supported) + return; + + down_write(&vdev->recovery_lock); + WRITE_ONCE(vdev->pci_recovery_enabled, false); + vdev->pci_recovery_command_valid =3D false; + /* + * Clear access_blocked before device_open, so a lock-free reader + * never sees it set on a device which is no longer open. A + * transaction which is still running cannot clear it once + * device_open is gone, and paths which refuse work on a blocked + * device would then refuse it for good. + */ + WRITE_ONCE(vdev->pci_recovery_access_blocked, false); + WRITE_ONCE(vdev->pci_recovery_device_open, false); + WRITE_ONCE(vdev->pci_recovery_flags, 0); + + /* + * Publish the closing state and drop recovery_lock before any + * teardown. Recovery is disabled and its state cleared, so + * slot_reset() and resume() become no-ops and a later + * error_detected() only follows the legacy notification path. + * Holding the lock across vfio_pci_core_disable() protects nothing + * and inverts the lock order. disable() reaches pci_reset_bus(), + * which takes pci_bus_sem, while error_detected() takes + * recovery_lock from under pci_bus_sem. + */ + up_write(&vdev->recovery_lock); + wake_up_all(&vdev->pci_recovery_wait); +} + +static void vfio_pci_core_finish_close(struct vfio_device *core_vdev) { struct vfio_pci_core_device *vdev =3D container_of(core_vdev, struct vfio_pci_core_device, vdev); @@ -838,6 +884,15 @@ void vfio_pci_core_close_device(struct vfio_device *co= re_vdev) vfio_pci_eventfd_replace_locked(vdev, &vdev->req_trigger, NULL); mutex_unlock(&vdev->igate); } + +void vfio_pci_core_close_device(struct vfio_device *core_vdev) +{ + struct vfio_pci_core_device *vdev =3D + container_of(core_vdev, struct vfio_pci_core_device, vdev); + + vfio_pci_core_prepare_close(vdev); + vfio_pci_core_finish_close(core_vdev); +} EXPORT_SYMBOL_GPL(vfio_pci_core_close_device); =20 void vfio_pci_core_finish_enable(struct vfio_pci_core_device *vdev) @@ -852,6 +907,18 @@ void vfio_pci_core_finish_enable(struct vfio_pci_core_= device *vdev) vdev->sriov_pf_core_dev->vf_token->users++; mutex_unlock(&vdev->sriov_pf_core_dev->vf_token->lock); } + + if (vdev->pci_recovery_supported) { + down_write(&vdev->recovery_lock); + WRITE_ONCE(vdev->pci_recovery_flags, 0); + vdev->pci_recovery_sequence =3D 0; + WRITE_ONCE(vdev->pci_recovery_enabled, false); + /* Close clears this too. Start unblocked either way. */ + WRITE_ONCE(vdev->pci_recovery_access_blocked, false); + WRITE_ONCE(vdev->pci_recovery_device_open, true); + WRITE_ONCE(vdev->pci_recovery_rom_disable, false); + up_write(&vdev->recovery_lock); + } } EXPORT_SYMBOL_GPL(vfio_pci_core_finish_enable); =20 --=20 2.43.0 From nobody Sat Sep 26 13:08:18 2026 Received: from CH4PR04CU002.outbound.protection.outlook.com (mail-northcentralusazon11013064.outbound.protection.outlook.com [40.107.201.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F34644746AA; Tue, 1 Sep 2026 09:33:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.201.64 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788255238; cv=fail; b=ZU9dC8flw4B7B6kuJSyS54sGEX2am04oAlWriwB/TVo+cHwMmHQHu2JCV0jGi5G1ECtXTmn80x0pfsJTrQ/ASmw7Zpek+KGglTkIUPXiWhWg5d7WBZQw95YnavFEQK4gsnaIEFi7r/qakrkAwBMGaHNArOczWn6SLMg7yEECfRM= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788255238; c=relaxed/simple; bh=CdrSjnyMgvAr/PlF3O7bgT6Y9Qp4zRGbCD4vw0jgSng=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=K72R3UBcz4gom+smmQJFpDrwa939MHL7s2ghgqRMFM1j5tNAqiUyZTSY0DP1B7uOs21VF/Sdu2pdWO0GLeZiCEvQiNczBZdf4/s1CI+4y1lTvkJbSjwoweIGjM2URyfzJDNrVfkOmgB0MrXaA1SPF0bDnP5MThbjdXBQqoaN3jg= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=OgAvu2BT; arc=fail smtp.client-ip=40.107.201.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="OgAvu2BT" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=agnKNxCC6EHTlcDr5IN14sEZ5o2z+GlLFi3lpk7vFRbibWPxbu8+KRMaWO4QWG2SYUClPCh6r1Q945hAdFiFZDc2KR2kGpXwElmGZSuvTvykScXtMj2FB1Nmu+VPK4EQRSZHTX+o/rNrj6TXlhXffX3A2icyxCyn77kbX8o+lUD61Qqf5QijBb62U23CNjyX669oQ9W2MQRtqM4qp1FT+G5VyQplDrpCY1FpgnTxu6GLSHWdzlk4414wvKD3UVAFEiyrw/uLt0Oa4cDONXJVcbCXSDt4q6hP43kbUltiP58jjzwcOj5+atcCcOcFklWcBCqICrW+dgfuM9Tvf/S2zA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ADrrE5khSKKQ1v9E8xOPt8dxuiFErbph9vBrdBu5y+A=; b=JYcODw2xKx9pkxUoUtzzGOTDPY4NjUU3Kt93i2oPOVeCgdan38Qvrq+n4bmElRuTQAveapnqADnZoatwJv0S2v2hcig4oA5Q/nuhjRaYBpcmsFPWAqNNsUAxceCDiAIaFYQ8wUJpqTbcVirt1LfLIwHhT1UGTYei1p6HJwJZp2MXTZEgJb7BFsJr7AWdCHceg+ioDuoVeTIBDgPQ2Jbc3RoAfHrHT77cwONOJoONrl5Qh+5ogM68p8An4GRVfaNEmNde6GGzGfXlMRDW+mZgVfMHyGeMt6qhPPLFxekxNNsAcYF26bjRcX08u82cqGDNkd5v43AOlav3KcQrnQcuYA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ADrrE5khSKKQ1v9E8xOPt8dxuiFErbph9vBrdBu5y+A=; b=OgAvu2BTMayngCv0gkPNGSzFXLDpRMQMdTdguswkarm0ofZYnjMyAS9dBQKqPzfAXn8c56/U6deqstGBwyqaU+IwY/H3WmX7vApl8LxuIovMEJMDLMhE9sJRFpRIliPbKC70+aGVKT5xQ+a7zYj69plkqhcJW9qpl2HmGsRdrpq2XtP5KEmf2+23QLfa7bm0IIrTijac16gNGr50Ej4uA1RuWbryWuCpa07A/Egr5dN7bP2gaaOyuiBIDKtEEs9Ta01vHo6vu5NCIyVhtVwK1bQGRa9p/mWqdCvduji+VBOLNeXKE0OIimxyB1CbsWD7l8XnJ6wjO81PRC/QrtAjkw== Received: from CH0P220CA0006.NAMP220.PROD.OUTLOOK.COM (2603:10b6:610:ef::27) by CY8PR12MB7562.namprd12.prod.outlook.com (2603:10b6:930:95::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.360.13; Tue, 1 Sep 2026 09:33:39 +0000 Received: from CH2PEPF00000146.namprd02.prod.outlook.com (2603:10b6:610:ef:cafe::24) by CH0P220CA0006.outlook.office365.com (2603:10b6:610:ef::27) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.382.10 via Frontend Transport; Tue, 1 Sep 2026 09:33:39 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by CH2PEPF00000146.mail.protection.outlook.com (10.167.244.103) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.382.8 via Frontend Transport; Tue, 1 Sep 2026 09:33:39 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.46; Tue, 1 Sep 2026 02:33:24 -0700 Received: from NV-2Y5XW94.nvidia.com (10.126.230.37) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.46; Tue, 1 Sep 2026 02:33:21 -0700 From: Shameer Kolothum To: , , CC: , , , , , , , , Subject: [RFC PATCH 03/19] vfio/pci: Add PCI recovery access guards Date: Tue, 1 Sep 2026 10:32:01 +0100 Message-ID: <20260901093217.8539-4-skolothumtho@nvidia.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260901093217.8539-1-skolothumtho@nvidia.com> References: <20260901093217.8539-1-skolothumtho@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: rnnvmail203.nvidia.com (10.129.68.9) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PEPF00000146:EE_|CY8PR12MB7562:EE_ X-MS-Office365-Filtering-Correlation-Id: d5f13bc5-481b-4b22-5920-08df080c1af2 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|23010399003|36860700016|82310400026|376014|1800799024|56012099006|10067099003|11063799006|5023799004|22082099003|18002099003; X-Microsoft-Antispam-Message-Info: SPnbmy/5Wd9HSIVNZ3c6Cqa3quljYpNwNd02Jqin734bFld/KNIW03cMo03xSumViIIKwM6/sxX6M2us5amvZREWHYw4XFu7tMSPsJTE8Cly5g6VdeO5YYbOigqvZL84rE56z6h7Npmuxgg5O1Fu2kph6oXQSqplhxY8Nvwgc8IvMlcO7qFf/g2T1jQQBSHKLDu8cC6ZZPNpeEJVgyl4hCMnenRmu2Ff7UaIyN3FyMPAQwQCxjYiLw6sFu+mu8wcifrPmage47MfrccAIkGa3UP8LGyLu/09U9s3MzJOnFquKO4wkFcdLRINzv//v241B+M7yQlQGN/DhGPuR9zBNdGjkiH9hAGMNNXLfLUlGLilXGyk7LZzkB2hFIF1iMzBIgMp0w2hqyYqVbSsaWEMivYXlEriH7qnTrXcWFmnvKFJQMaSbQDYfM62K8z7GBh3NrtCM3UltujChtko4ALe2g/4pQguRPOBZPFKomiYhTckRfIvJ406IVtsXge2JBRPsLPsHXOkUBgLK4sbKwiNi7FwSKxMpQ9k3u0MDcHLyeOxJ4lHy09ImfA09NzTuDse+URKYhfuYRsspXkFbz84uehAqfg5bPgG3xvsbjtytGe9vhb0onS/RfwgYYXTWfgGucZ1TG+Idi70akLDjz6DfWShdmPsy+P3MZ9lrbKoFDeFZs5XnAWdAtoEQNr82DOt7HXev06pxUpdJBfEB5mwCw== X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(23010399003)(36860700016)(82310400026)(376014)(1800799024)(56012099006)(10067099003)(11063799006)(5023799004)(22082099003)(18002099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: G54U9nfPVxeF+28tBrnnQeqKPqV45yF3ZiXLdcxN6fNfBwBtq6s5Cq+Tydw3C2Re9xz8mlyn2d23rms3nms7ZDuEjtn6Sq1jL7fzlK/7g4r4jf1XSj1nVdxhQoaTeiSLM4ox9O+I/vjAE3Fo0QKEVDcuhD05jrRrJvObWP3MDu2CxGr3iJd/kUrC7kNyZOnqWnNwjZHwNByMUsxUA3Mv21Sbj/rgJGGmI75woJsQ6UAb9+E9edUmFeQ0Fm/1nHVtRxAxUR7qmgyx35dsF5Ue6UEitpgZGN34PVW7RM8WMPEYnDp5Np3OJ7gaNd2AogKZJIySDWqlXdq4O4I4SMrQaoaY2Ly2tM8kuiq1wK3VSo0Rk7NEhfT21Zq6K/JnSx/fZ+g3e8Ec/SeLr9PWFtXc03BnRVyoWEvCR/tqWtj2EEBf6g4HylVWlsRP/KkZ/tqT X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Sep 2026 09:33:39.3122 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: d5f13bc5-481b-4b22-5920-08df080c1af2 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CH2PEPF00000146.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR12MB7562 Content-Type: text/plain; charset="utf-8" Add a pair of helpers to wrap each operation which touches the device. access_begin() takes recovery_lock for reading and refuses if the device is not open, or if recovery is blocking access. The callers come in later patches. access_end() drops the lock without looking at the recovery state, so only call it after access_begin() returned 0. On failure the lock is already gone. Both helpers key off pci_recovery_supported, which is fixed for the lifetime of the device, so the pair stays balanced. The lock is taken even when userspace has not enabled recovery. Enabling takes recovery_lock for writing, which waits for anything already in flight. Without that, an operation which started before enable could still be touching the device when the first error arrives, and there would be nothing to wait on. access_blocked is checked either way. Nothing sets it yet, so nothing which works today gets rejected. The cost is one rwsem acquire per guarded access on devices that never turn recovery on. For BAR traffic that is once per width-sized access, alongside the memory_lock read already taken there. Signed-off-by: Shameer Kolothum --- drivers/vfio/pci/vfio_pci_priv.h | 3 +++ drivers/vfio/pci/vfio_pci_core.c | 21 +++++++++++++++++++++ 2 files changed, 24 insertions(+) diff --git a/drivers/vfio/pci/vfio_pci_priv.h b/drivers/vfio/pci/vfio_pci_p= riv.h index 4e7162234a2e..6daf51669d05 100644 --- a/drivers/vfio/pci/vfio_pci_priv.h +++ b/drivers/vfio/pci/vfio_pci_priv.h @@ -73,6 +73,9 @@ u16 vfio_pci_memory_lock_and_enable(struct vfio_pci_core_= device *vdev); void vfio_pci_memory_unlock_and_restore(struct vfio_pci_core_device *vdev, u16 cmd); =20 +int vfio_pci_core_access_begin(struct vfio_pci_core_device *vdev); +void vfio_pci_core_access_end(struct vfio_pci_core_device *vdev); + #ifdef CONFIG_VFIO_PCI_IGD bool vfio_pci_is_intel_display(struct pci_dev *pdev); int vfio_pci_igd_init(struct vfio_pci_core_device *vdev); diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_c= ore.c index 8de586e4bb73..4194d44d6530 100644 --- a/drivers/vfio/pci/vfio_pci_core.c +++ b/drivers/vfio/pci/vfio_pci_core.c @@ -1747,6 +1747,27 @@ static ssize_t vfio_pci_rw(struct vfio_pci_core_devi= ce *vdev, char __user *buf, return ret; } =20 +int vfio_pci_core_access_begin(struct vfio_pci_core_device *vdev) +{ + if (!vdev->pci_recovery_supported) + return 0; + + down_read(&vdev->recovery_lock); + if (unlikely(!vdev->pci_recovery_device_open || + vdev->pci_recovery_access_blocked)) { + up_read(&vdev->recovery_lock); + return -EIO; + } + + return 0; +} + +void vfio_pci_core_access_end(struct vfio_pci_core_device *vdev) +{ + if (vdev->pci_recovery_supported) + up_read(&vdev->recovery_lock); +} + ssize_t vfio_pci_core_read(struct vfio_device *core_vdev, char __user *buf, size_t count, loff_t *ppos) { --=20 2.43.0 From nobody Sat Sep 26 13:08:18 2026 Received: from PH0PR06CU001.outbound.protection.outlook.com (mail-westus3azon11011017.outbound.protection.outlook.com [40.107.208.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id ED28A479876; Tue, 1 Sep 2026 09:33:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.208.17 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788255242; cv=fail; b=Uo98wS7sS/4Gx7OLQqMyC4cOnNwFDx5+MdtUXX16pa2OlEGH9xUJzYFOnbjwkMC8vLbP6wTc3Uug8QLfNpfi9ZZEbCBZdcQ2AC5Y0oRutgilbfFO1ga6n7Td57VGvySxZayGgG/Ati+siq0bnrPdEAgxFMvsrdMsmoQBXD+PFGo= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788255242; c=relaxed/simple; bh=Iw4gEiXk5lRr2qefpjCONKqU6fA/t+be5+41zix5pvQ=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=mpKGFJOb1HI/WP1NlZFTMsF2bZvUOFf8mY8zvpHSQyAMXpFbtr2BJKxW6G74WvpGzHYzfnr2s+JRBh2ro6UShjj9fEqRmIMzBD01GkXMYCkTZmAaBjsVqcw32DMCgFDdkNt+oE9senricsTg+miK5W83NLM7U+7lMA11JtrU+Wc= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=XaDHWJIn; arc=fail smtp.client-ip=40.107.208.17 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="XaDHWJIn" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=KCHvjzF74TtUcnS/OSclkyk8C8MjnCwHgiqhGcb0Qll0QxCVTWy6boXEoAcPFdKRednoah6CoTvNMPTlg8QB40y5ZbAiHS41OT4mg0zYjT4uTOKYT477fBz98bF9npBDkseAl8IrlszrsayoNEZ5IRRcpOUniWnAmPP5arPTVKJZLHoiATwRYJbMybXs38dhbQtv4sJAmOIzjV/A50AEuY1dUoJzxjgOF0h/Cm9ykZR/nRnQ5qetLiwsjF/q/vHYvcM9RXkXlPU5O68UaXmlQLOVu4T9I/fafvxnSvJjWlXYo9p1htffwjbqNtM17HyCSU3N9cDq9oCRjqSCXHhOqg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=JhaNg42w3D8r+kjVMls62vmV4oSyLHZtbFn3wn45MN8=; b=Xy/N3bpEwRQ7ix6U4no4RXcMOrMWhMCGaH0kVrdXsKEEAJ7mSvQyr3tyVVI01RscfbWjWxE4+k6IWBDqXfmllICV0zUvGP6ih05rk7cwoZ+9Yy4I/a4EmCfZ4/AdVl3XvYwPTzwIA/LRslcw+/92dkXMVdoHyhLHkKs63FnHd+ETyR8h59IgGl1EaHdM1wPoC+ngzd3Gq4Vt5HN6xJDJu1z7v3Vu68OejvBasc6cG1pegMqAs7wjiR2kGQKm+BmFoOl107QBitdllLEoVSUJkEJts4WWGPTd1hx1XZZaR9LJFrSKqCRiPUWlZU42yjwnK7oBCYRBVeU0bCaTsFsFwg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=JhaNg42w3D8r+kjVMls62vmV4oSyLHZtbFn3wn45MN8=; b=XaDHWJInkjudh0TiWNbK7RmSpU+gMjGpT72cn5DQ1JPK5ZG/k/slmrtPsaD7Wx4eCxPSDYjpWhT/vih0rfqiMQifXVRb42hAafW2FObIFGdQR6O9ElaLxPMwtFSxxwVRIZNCBcEmYGWvuGe8X+r47ZPg59U2jZojdT85xl3h+0/myIs1DFhSKDJhaFbv3+XUcnwYiTxX95TI4J5iqcaK5/SMKSNY+x+ux5I8YxLsXNZ49ywN4OBnUiUkbN2lrXxrTCqSxnC+m6e7Q76lPqiIgoxAOVKQUJak9hXfo6umdLZVI1CMCSldW40SNiDSw0iCJSSwCdkUN5LsVIdtrNgJqg== Received: from CH0P220CA0023.NAMP220.PROD.OUTLOOK.COM (2603:10b6:610:ef::25) by IA0PR12MB8326.namprd12.prod.outlook.com (2603:10b6:208:40d::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.360.13; Tue, 1 Sep 2026 09:33:43 +0000 Received: from CH2PEPF00000146.namprd02.prod.outlook.com (2603:10b6:610:ef:cafe::a5) by CH0P220CA0023.outlook.office365.com (2603:10b6:610:ef::25) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.360.13 via Frontend Transport; Tue, 1 Sep 2026 09:33:43 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by CH2PEPF00000146.mail.protection.outlook.com (10.167.244.103) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.382.8 via Frontend Transport; Tue, 1 Sep 2026 09:33:43 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.46; Tue, 1 Sep 2026 02:33:28 -0700 Received: from NV-2Y5XW94.nvidia.com (10.126.230.37) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.46; Tue, 1 Sep 2026 02:33:25 -0700 From: Shameer Kolothum To: , , CC: , , , , , , , , Subject: [RFC PATCH 04/19] vfio/pci: Serialize function reset with recovery Date: Tue, 1 Sep 2026 10:32:02 +0100 Message-ID: <20260901093217.8539-5-skolothumtho@nvidia.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260901093217.8539-1-skolothumtho@nvidia.com> References: <20260901093217.8539-1-skolothumtho@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: rnnvmail203.nvidia.com (10.129.68.9) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PEPF00000146:EE_|IA0PR12MB8326:EE_ X-MS-Office365-Filtering-Correlation-Id: 74c63cee-cc91-4b1c-da07-08df080c1d28 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|23010399003|36860700016|82310400026|376014|1800799024|6133799003|56012099006|10067099003|11063799006|5023799004|22082099003|18002099003; X-Microsoft-Antispam-Message-Info: E7Sl2qdOC8Xst5m+ialE7bMcXs8ns5dvElUL/Idz0cxSNphVyV4o1w5u7ZODowGAs1+0uTzivizA2iez+vobebTyfxYGma/7k20rnQ50qF4/+Np+dO4C1ZiSdI6GVWm/srpnmjrpf14HlKChGAb0VRETc6rh1jeYFtK3G8ycoN8PIrrXKZdS2ec1j3WEnwvkkNGT03rXAoFN92tJEEBOze7lI8QozBffYwkW0n4ebDhje7dz+FlfKq/eZkWwZKd7ZzuLSjOwC0jRkw72Qu9/CWdg/PRyaZ/2dwvHmqZsTNAFURqWJGG0pYwtYG4HPsZD8D3Vr7iBSYnUUQKSDdZyrKOk2Pp7mUtEiZVEaITr0YQwnEbza7I2+RNu8v1rd26dgS75sht/0QP8redK6HlAXFlTfRoKtLLzi7GfzMW0Ycebf3knE7dzB1qgCiLGO/Kg5pEPmVCpaa5n/JbMUv+d+1craq2C8YQwd0fZk/zw4xAQ6OqK/jHqqBJ6T3+Xv+rzxZoJ5p0qZRJidJ1BMOI5jwcM46I24L+nULzfeYjjf+Bs/GVt8rhpW7NHf/OkOZQsbEq+o/chBU1Cm7Tyv2mNC8OnsOY6CePwI9lrEpmGEHKZvRCczepRv/j81PiiesEI/6yc3xlrlhIRaVj0UgpEs0Wbc8v7RYIabe/Wh7/kZiOtHuOs1Bebc2mwYZj7pnQ5t0QDTAwPI3pex7wyYCp/JQ== X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(23010399003)(36860700016)(82310400026)(376014)(1800799024)(6133799003)(56012099006)(10067099003)(11063799006)(5023799004)(22082099003)(18002099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: a1ZbyUklArfNGIy1Sg1mIQNV3HYLljoBuoSkGLOQTvL47Oet2GSTTzp2nFnFfGA4sxVPZpVbEjIV2DyQd49D4lYZAHNxIw6DQYPPV7ROzEbeWahNykOB4rzpFDPKRz1te8/j4KCBuJpwYXioUK4JtDNPNQtzZHl8GePTC2p2GePvWe2bPevb+FNIfotF6QE6n72/PShcIUebWaphK+ut497PCXuRh539rawUCiZwGngMET4htQw/4VVzmkUrwvGrwTOjX50fSME1tvQY+hsgFBDWoBa768AerXlRPuP/PkEiQyn+Z6CbF0uSQH6iL4WSN3d7mySjo7Qtahs00n827/OA14yai/E/4Ifbup0ejeE+SQ8+HFo3gaBzfCoAkTK3iDeTsjsB+ObeIxoACkMxl/9jfBlG7MBETQwcGrbUOmeDJD5XevvceiLL5Ky0woyN X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Sep 2026 09:33:43.0233 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 74c63cee-cc91-4b1c-da07-08df080c1d28 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CH2PEPF00000146.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA0PR12MB8326 Content-Type: text/plain; charset="utf-8" Add a function reset helper and use it for VFIO_DEVICE_RESET. A later patch routes the guest triggered config space FLR through it as well. That path never did the power state transition, so make it optional. With recovery enabled, take recovery_lock for writing, refuse the reset with -EBUSY if access is already blocked, otherwise block access and drop the lock again before revoking mappings or running the reset. recovery_lock cannot be held across the reset because a reset method can take pci_bus_sem, and the PCI error callbacks take recovery_lock from under it. Dropping it is safe in both directions. The error callbacks hold recovery_lock for their whole body, so one already running has finished before the reset starts. One which arrives while the lock is down runs its own event, and the PCI core calls it with the device lock held, which pci_try_reset_function() also takes, so it cannot overlap the reset itself. Only unblock access at the end for a reset which is still the one blocking it. An event which started meanwhile owns the state from then on, and resume() is what ends it. With recovery not enabled, leave access_blocked alone. Two concurrent resets still serialize on memory_lock, same as today. Setting the flag for a device which never opted in would turn a working VFIO_DEVICE_RESET into -EBUSY. Access stays blocked until the reset is done and memory state is back, and the wait queue is woken once it clears. A later patch adds the BAR fault path, which waits there rather than failing the fault while a reset is in flight. Assisted-by: Claude:claude-opus-5 Signed-off-by: Shameer Kolothum --- drivers/vfio/pci/vfio_pci_priv.h | 3 ++ drivers/vfio/pci/vfio_pci_core.c | 85 +++++++++++++++++++++++++++++--- 2 files changed, 82 insertions(+), 6 deletions(-) diff --git a/drivers/vfio/pci/vfio_pci_priv.h b/drivers/vfio/pci/vfio_pci_p= riv.h index 6daf51669d05..8a7f9fe22386 100644 --- a/drivers/vfio/pci/vfio_pci_priv.h +++ b/drivers/vfio/pci/vfio_pci_priv.h @@ -41,6 +41,9 @@ ssize_t vfio_pci_config_rw_single(struct vfio_pci_core_de= vice *vdev, char __user *buf, size_t count, loff_t *ppos, bool iswrite); =20 +int vfio_pci_try_reset_function(struct vfio_pci_core_device *vdev, + bool reset_power_state); + ssize_t vfio_pci_bar_rw(struct vfio_pci_core_device *vdev, char __user *bu= f, size_t count, loff_t *ppos, bool iswrite); =20 diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_c= ore.c index 4194d44d6530..3645daa8891f 100644 --- a/drivers/vfio/pci/vfio_pci_core.c +++ b/drivers/vfio/pci/vfio_pci_core.c @@ -1379,14 +1379,53 @@ static int vfio_pci_ioctl_set_irqs(struct vfio_pci_= core_device *vdev, return ret; } =20 -static int vfio_pci_ioctl_reset(struct vfio_pci_core_device *vdev, - void __user *arg) +int vfio_pci_try_reset_function(struct vfio_pci_core_device *vdev, + bool reset_power_state) { + struct pci_dev *pdev =3D vdev->pdev; + bool enabled =3D false; + bool supported =3D vdev->pci_recovery_supported; int ret; =20 - if (!vdev->reset_works) - return -EINVAL; + /* + * Claim the device against recovery before resetting it. The PCI + * error callbacks hold recovery_lock for their whole body, so taking + * it for writing here waits for one already running, and + * access_blocked keeps a later one away while the lock is dropped. + */ + if (supported) { + down_write(&vdev->recovery_lock); + if (!vdev->pci_recovery_device_open) { + ret =3D -ENODEV; + goto out_recovery; + } =20 + enabled =3D vdev->pci_recovery_enabled; + + /* + * Only claim access_blocked when recovery is enabled. + * error_detected() returns early for a device which has not + * enabled it, so there is nothing to exclude, and claiming it + * anyway would fail the second of two concurrent + * VFIO_DEVICE_RESET calls with -EBUSY. + */ + if (enabled) { + if (vdev->pci_recovery_access_blocked) { + ret =3D -EBUSY; + goto out_recovery; + } + WRITE_ONCE(vdev->pci_recovery_access_blocked, true); + } + up_write(&vdev->recovery_lock); + } + + /* + * On a device which supports recovery, taking recovery_lock for + * writing above waited for anything already past its access check, + * and if recovery is enabled access_blocked keeps new ones out. Do + * not hold recovery_lock while taking memory_lock or running a reset + * method, since a reset can take pci_bus_sem. + */ vfio_pci_zap_and_down_write_memory_lock(vdev); =20 /* @@ -1398,15 +1437,49 @@ static int vfio_pci_ioctl_reset(struct vfio_pci_cor= e_device *vdev, * reset without restoring the original state (saved locally in * 'vdev->pm_save'). */ - vfio_pci_set_power_state(vdev, PCI_D0); + if (reset_power_state) + vfio_pci_set_power_state(vdev, PCI_D0); =20 vfio_pci_dma_buf_move(vdev, true); - ret =3D pci_try_reset_function(vdev->pdev); + ret =3D pci_try_reset_function(pdev); if (__vfio_pci_memory_enabled(vdev)) vfio_pci_dma_buf_move(vdev, false); up_write(&vdev->memory_lock); =20 + if (enabled) { + down_write(&vdev->recovery_lock); + /* + * An error callback can have started an event while the lock + * was down. Leave the state to it. Only unblock access for a + * reset which is still the one holding it. + */ + if (vdev->pci_recovery_device_open && + !(vdev->pci_recovery_flags & (VFIO_PCI_RECOVERY_IN_PROGRESS | + VFIO_PCI_RECOVERY_FAILED))) + WRITE_ONCE(vdev->pci_recovery_access_blocked, false); + up_write(&vdev->recovery_lock); + /* + * Access is blocked for the length of the reset, so anything + * waiting for it to clear has to be woken here. A later patch + * adds the BAR fault path which waits on this. + */ + wake_up_all(&vdev->pci_recovery_wait); + } + return ret; + +out_recovery: + up_write(&vdev->recovery_lock); + return ret; +} + +static int vfio_pci_ioctl_reset(struct vfio_pci_core_device *vdev, + void __user *arg) +{ + if (!vdev->reset_works) + return -EINVAL; + + return vfio_pci_try_reset_function(vdev, true); } =20 static int vfio_pci_ioctl_get_pci_hot_reset_info( --=20 2.43.0 From nobody Sat Sep 26 13:08:18 2026 Received: from PH0PR06CU001.outbound.protection.outlook.com (mail-westus3azon11011062.outbound.protection.outlook.com [40.107.208.62]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BCE93476CF6; Tue, 1 Sep 2026 09:35:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.208.62 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788255320; cv=fail; b=Tu23jzuoEq0xSCc9+yQ3vQdrH0BtDj48F/CFObFuMhHiIhsc14wATfkTuzXDOQXV6Hr+nD1hpJwNoqtmb9ZFYGvjUmwvknOJfKpDks8JXjh/RrxNorudQV7njNvvhy1A8JuAQG7WssNgCQtQrrN5n6xDDP2zSvGqm9keX2UnNBA= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788255320; c=relaxed/simple; bh=Z0dl3DDxuON5u1Too572rkZuF5bkl8wwRDPwM9siBgM=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=l4OtihAdtmqTrBYa2fI3GlfPFouiN4CrTdCd/vvrYuPbuef3DuwEfYAvD53cLYJfgcOcWw2ywP4cqZEjCDhOPnvit+9Rihwn+1LglbAMH6VOyZcFynJx8ynO93zX33PQYSbIsytNs4g5uJtL0IV4emTYDjlK8/ICMUATMlgddh4= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=p8zbi891; arc=fail smtp.client-ip=40.107.208.62 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="p8zbi891" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=pPv+i5BZUHMxcFhyScnWavwRojZpIkDXac+nlBqooL7dL++H0wBq3mziqSMlySa9ppy+JaDXz/FRogQjvLMmP+VcRDyofjGpmFXYfskMYW8LRHWtKJ8bgJ1p6qpSKLrivo8+HmZN2qFplWz74/109m4Q9dNIKNg9srp+NTRzwoa5Re+8uV32nBdrhHKZLxWUjkNZsBsVTdOdhfVRzNUtTRWNbZOe2MHBS4C+jfLlAyRHHpv/6a5DVF7kncFIg8HwYdc9UMVyQj8MKC5LrvetmNdAGgytY+g9LGtgff7+LrQ30gGSC2L8/ZpochTWqUWfKfuXrXUvGFTgusF6BayF4g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=pEdwHkz6dN3IQa4vjkGr/lYpvq9DNJ3kkpniFi0X5B8=; b=hLhqVTWY18pIbXaFYisvrhKRLY4jODz+lw8uhkXw7lKQSJ4LwoeYj2Ua9nNK3P0a+zwba470T2ITGZ4EPCUyuLsitcYP6Zkdr58qrDee8lnO4HFrpYKxzvHIWkEZgEp7tlIjfQFj0S+EQj+uKfRWISCqXGcH//O554i0B4qku7oIM/hxjn6MLBfnLhMREulnWdsU2sEPfu2ULeGvnjkqAZeCSYrMJ0uHfEyIjdr8KkGotqRnOzI+XpLDVgjt24ThjotvO6AiSxO6YktkjOBWhONi3cQiUjPx2hj22CLMJkg0jv6z3Q8q0zlYcJY384f79uwuNq28Wtp1fQJxnyB5QA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=pEdwHkz6dN3IQa4vjkGr/lYpvq9DNJ3kkpniFi0X5B8=; b=p8zbi891pS9i/PxWhiyumHnL0kAIpkxXNuguy0CUtrmPAlLFVizDl1toMxuDYpBib4DOzs2Kr0fF6xtNU25/U5Moy3aDl2JkvEO2ppaijHykGdlSAvOcJaz+UANrPYw7nTiqLJv5FhzUU/YiLjKLLCSaETVgYfispPG47nqDT57mtqZ4u/urUVrqsGoDZt2Z+I05Wfi1y1mgaRmUJT08IrAUpJGCqEs9NIVE0FlN3wNTQtHwO02ATLrK+7IMPu/gWXDsq/Z4QoSur20k4EOGbmMAALDUQb0X/00thICTangtjTHn0KEO3bzoBQICjDWp+Dvr36WV5bZuMohH0OSHYw== Received: from BN0PR04CA0187.namprd04.prod.outlook.com (2603:10b6:408:e9::12) by DM4PR12MB7598.namprd12.prod.outlook.com (2603:10b6:8:10a::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.360.13; Tue, 1 Sep 2026 09:33:50 +0000 Received: from BL02EPF00021F6E.namprd02.prod.outlook.com (2603:10b6:408:e9:cafe::90) by BN0PR04CA0187.outlook.office365.com (2603:10b6:408:e9::12) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.360.13 via Frontend Transport; Tue, 1 Sep 2026 09:33:50 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by BL02EPF00021F6E.mail.protection.outlook.com (10.167.249.10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.382.8 via Frontend Transport; Tue, 1 Sep 2026 09:33:50 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.46; Tue, 1 Sep 2026 02:33:32 -0700 Received: from NV-2Y5XW94.nvidia.com (10.126.230.37) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.46; Tue, 1 Sep 2026 02:33:29 -0700 From: Shameer Kolothum To: , , CC: , , , , , , , , Subject: [RFC PATCH 05/19] vfio/pci: Serialize config access with recovery Date: Tue, 1 Sep 2026 10:32:03 +0100 Message-ID: <20260901093217.8539-6-skolothumtho@nvidia.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260901093217.8539-1-skolothumtho@nvidia.com> References: <20260901093217.8539-1-skolothumtho@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: rnnvmail203.nvidia.com (10.129.68.9) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL02EPF00021F6E:EE_|DM4PR12MB7598:EE_ X-MS-Office365-Filtering-Correlation-Id: fa934fc9-6f14-484b-5929-08df080c2188 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|1800799024|36860700016|23010399003|376014|22082099003|18002099003|56012099006|5023799004|11063799006|10067099003; X-Microsoft-Antispam-Message-Info: BfndhIeUOauJk3FgMLoW0IW1G/EzvE2fPvW8UeKTivsslCGkp/BCtFlQsiinPuZr1/cJHWyIf25lkCZLHYXW21pgvl6K7cQsiCC598n8P4qiz0whfkkESBI4sHy7TxQ3yc6/1KNrW3YMwhpufv/rXa7NuN5+0h3TByS/wok8zz10Ul3xlvHuJrtshDod2Wbwms4IrgHsRsUyiE+Iu4/dQp0gjNU2Srko1lnzlV40zYy6IMWtrPLzLoh552Tz27IWB5f5+plmmdXrMGnc5rolRC1X4a3s1mXyfVkO5NF+cTdAAu0UF9cJkBU2WwgjivqT4rQtiF8SKY8eQVq2i8YDXO9iBDrDG985iC2EZvJl4MibblfwFUVoJOtm1xtfZnx+Hb5OEj/TRf2YjR1BPaPnNsWqnjBO4mnIH3ixODBs5sMR0sF8WDmkT5HBO2OuLd/gYUdP1PW3fpEB90OkQBIpMPGhNNT75zp8FzBYGFhBSSBmbVdkwUEKHXDshE7jRn0h6LHCPQXLG9S6n/ufq4yZPxDVXsw1ohadAxgnZrKvL8yM04ThwTxn1Ptg7um72Wp4oyRi+cXiH+4JA+yIqH98FRxgXCitz0IGTkQ/B4QgLpHdlNW8jpa23DH2rpo8b3ng53MJafL1kJ+cmZ9MXTkW/7SYywAX2qur6JdYAzhyGzOoNrFEAncYlKhn/rrxrtwbm7eWzL+mkDIaIqtoGGvKNA== X-Forefront-Antispam-Report: CIP:216.228.117.160;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge1.nvidia.com;CAT:NONE;SFS:(13230040)(82310400026)(1800799024)(36860700016)(23010399003)(376014)(22082099003)(18002099003)(56012099006)(5023799004)(11063799006)(10067099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: P9TfxyqI1Iza1JL+vMlHOFfzMW4tTVcYNGDXAo5GvXHgAZehfS1T7iBOWMwC08ZVThTSotoZf1Gs4Ddbedj0FS3CG5J3q9KWafLjQ6yV4wJ7olHiW9dzdQT8V1XbGBRkBg6h4nu/7DwJsIwml2njo7mnc09VyAE/K7hZ2BjuK8Jd+T9kYeoEVGOxqTtZuGdbccsFXDOqKBsrLCL7mY55onU914niBIOJlqt51uqoEKNMXdNFw/pKCdmFkQnvkh7E8nrJd1ewC5vK0qmTImawgIG1kN8pG/J6po+smfNgDKZnCSv8p3P84FKTp+3cP/GO9YZJAOiBnHQtNag1Opw08vCF0B2NNRCWJvo/TbJFQds0rDTtFSQey3EAWUgPNC2d9o306VBDADxa83WMQfyT7I1LSGMcvWtt1N6w/XJcsaISRffBgkqC0ePA+xo9GV6B X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Sep 2026 09:33:50.3290 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: fa934fc9-6f14-484b-5929-08df080c2188 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BL02EPF00021F6E.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB7598 Content-Type: text/plain; charset="utf-8" Hold recovery_lock for reading across each config space operation, so recovery can shut out new ones and wait for whatever is already running. The user copies stay outside the lock, since a copy can fault. Take the lock in the dispatcher rather than around the individual hardware accessors. That means once recovery blocks access every config read fails with -EIO, even a read served entirely from vconfig which never touches the device. Userspace which wants to know what is going on reads the device feature instead. That one stays available during an event. The PCIe and AF capability writes no longer reset the device themselves, and the power management write no longer moves it to D0 itself. They record what was asked for and the dispatcher does it after dropping recovery_lock. Both take pci_bus_sem, which AER already holds when it calls into the driver, so doing either inside the lock would be the wrong order. A reset method reaches it directly, and a D0 transition reaches it through pci_set_full_power_state() calling pcie_aspm_pm_state_change(). The lower power states take neither, so those still run in the writefn. The writefn declaration says so. Both stay best effort, as the guest requested FLR always was. The result is not reported back through the config write. With recovery enabled they are dropped while a recovery or reset is already in flight, since that leaves the device in D0 and reset anyway. The reset helper tests the recovery state for itself. The power up does not, so the dispatcher tests it before that one. Assisted-by: Claude:claude-opus-5 Signed-off-by: Shameer Kolothum --- drivers/vfio/pci/vfio_pci_config.c | 147 ++++++++++++++++++++--------- 1 file changed, 102 insertions(+), 45 deletions(-) diff --git a/drivers/vfio/pci/vfio_pci_config.c b/drivers/vfio/pci/vfio_pci= _config.c index 9914f3ac69ae..3365100acf21 100644 --- a/drivers/vfio/pci/vfio_pci_config.c +++ b/drivers/vfio/pci/vfio_pci_config.c @@ -99,6 +99,12 @@ static const u16 pci_ext_cap_length[PCI_EXT_CAP_ID_MAX += 1] =3D { [PCI_EXT_CAP_ID_DVSEC] =3D 0xFF, }; =20 +/* What a config write asked for which has to wait for the access guard. */ +struct vfio_pci_config_deferred { + bool flr; /* a function-level reset */ + bool power_up; /* a transition to D0 */ +}; + /* * Read/Write Permission Bits - one bit for each bit in capability * Any field can be read if it exists, but what is read depends on @@ -111,8 +117,17 @@ struct perm_bits { u8 *write; /* writeable bits */ int (*readfn)(struct vfio_pci_core_device *vdev, int pos, int count, struct perm_bits *perm, int offset, __le32 *val); + /* + * @deferred records work the write asked for which a writefn must not + * do itself. Both a reset method and a transition to D0 acquire + * pci_bus_sem, which AER already holds when it enters the driver, so + * doing either here would invert the lock order against recovery_lock. + * The dispatcher does them after dropping recovery_lock. Callers zero + * it, and a writefn only sets a field on a success return. + */ int (*writefn)(struct vfio_pci_core_device *vdev, int pos, int count, - struct perm_bits *perm, int offset, __le32 val); + struct perm_bits *perm, int offset, __le32 val, + struct vfio_pci_config_deferred *deferred); }; =20 #define NO_VIRT 0 @@ -200,7 +215,8 @@ static int vfio_default_config_read(struct vfio_pci_cor= e_device *vdev, int pos, =20 static int vfio_default_config_write(struct vfio_pci_core_device *vdev, in= t pos, int count, struct perm_bits *perm, - int offset, __le32 val) + int offset, __le32 val, + struct vfio_pci_config_deferred *deferred) { __le32 virt =3D 0, write =3D 0; =20 @@ -272,7 +288,8 @@ static int vfio_direct_config_read(struct vfio_pci_core= _device *vdev, int pos, /* Raw access skips any kind of virtualization */ static int vfio_raw_config_write(struct vfio_pci_core_device *vdev, int po= s, int count, struct perm_bits *perm, - int offset, __le32 val) + int offset, __le32 val, + struct vfio_pci_config_deferred *deferred) { int ret; =20 @@ -299,7 +316,8 @@ static int vfio_raw_config_read(struct vfio_pci_core_de= vice *vdev, int pos, /* Virt access uses only virtualization */ static int vfio_virt_config_write(struct vfio_pci_core_device *vdev, int p= os, int count, struct perm_bits *perm, - int offset, __le32 val) + int offset, __le32 val, + struct vfio_pci_config_deferred *deferred) { memcpy(vdev->vconfig + pos, &val, count); return count; @@ -563,7 +581,8 @@ static bool vfio_need_bar_restore(struct vfio_pci_core_= device *vdev) =20 static int vfio_basic_config_write(struct vfio_pci_core_device *vdev, int = pos, int count, struct perm_bits *perm, - int offset, __le32 val) + int offset, __le32 val, + struct vfio_pci_config_deferred *deferred) { struct pci_dev *pdev =3D vdev->pdev; __le16 *virt_cmd; @@ -613,7 +632,8 @@ static int vfio_basic_config_write(struct vfio_pci_core= _device *vdev, int pos, vfio_bar_restore(vdev); } =20 - count =3D vfio_default_config_write(vdev, pos, count, perm, offset, val); + count =3D vfio_default_config_write(vdev, pos, count, perm, offset, val, + deferred); if (count < 0) { if (offset =3D=3D PCI_COMMAND) up_write(&vdev->memory_lock); @@ -727,9 +747,11 @@ static void vfio_lock_and_set_power_state(struct vfio_= pci_core_device *vdev, =20 static int vfio_pm_config_write(struct vfio_pci_core_device *vdev, int pos, int count, struct perm_bits *perm, - int offset, __le32 val) + int offset, __le32 val, + struct vfio_pci_config_deferred *deferred) { - count =3D vfio_default_config_write(vdev, pos, count, perm, offset, val); + count =3D vfio_default_config_write(vdev, pos, count, perm, offset, val, + deferred); if (count < 0) return count; =20 @@ -738,8 +760,15 @@ static int vfio_pm_config_write(struct vfio_pci_core_d= evice *vdev, int pos, =20 switch (le32_to_cpu(val) & PCI_PM_CTRL_STATE_MASK) { case 0: - state =3D PCI_D0; - break; + /* + * Going to D0 reaches pci_set_full_power_state(), + * which takes pci_bus_sem through + * pcie_aspm_pm_state_change(). Leave it to the + * dispatcher. The lower states do not, so they run + * here. + */ + deferred->power_up =3D true; + return count; case 1: state =3D PCI_D1; break; @@ -799,7 +828,8 @@ static int __init init_pci_cap_pm_perm(struct perm_bits= *perm) =20 static int vfio_vpd_config_write(struct vfio_pci_core_device *vdev, int po= s, int count, struct perm_bits *perm, - int offset, __le32 val) + int offset, __le32 val, + struct vfio_pci_config_deferred *deferred) { struct pci_dev *pdev =3D vdev->pdev; __le16 *paddr =3D (__le16 *)(vdev->vconfig + pos - offset + PCI_VPD_ADDR); @@ -812,7 +842,8 @@ static int vfio_vpd_config_write(struct vfio_pci_core_d= evice *vdev, int pos, * of PCI_VPD_ADDR, then the PCI_VPD_ADDR_F bit is written and we * have work to do. */ - count =3D vfio_default_config_write(vdev, pos, count, perm, offset, val); + count =3D vfio_default_config_write(vdev, pos, count, perm, offset, val, + deferred); if (count < 0 || offset > PCI_VPD_ADDR + 1 || offset + count <=3D PCI_VPD_ADDR + 1) return count; @@ -881,21 +912,24 @@ static int __init init_pci_cap_pcix_perm(struct perm_= bits *perm) =20 static int vfio_exp_config_write(struct vfio_pci_core_device *vdev, int po= s, int count, struct perm_bits *perm, - int offset, __le32 val) + int offset, __le32 val, + struct vfio_pci_config_deferred *deferred) { __le16 *ctrl =3D (__le16 *)(vdev->vconfig + pos - offset + PCI_EXP_DEVCTL); int readrq =3D le16_to_cpu(*ctrl) & PCI_EXP_DEVCTL_READRQ; =20 - count =3D vfio_default_config_write(vdev, pos, count, perm, offset, val); + count =3D vfio_default_config_write(vdev, pos, count, perm, offset, val, + deferred); if (count < 0) return count; =20 /* * The FLR bit is virtualized, if set and the device supports PCIe - * FLR, issue a reset_function. Regardless, clear the bit, the spec - * requires it to be always read as zero. NB, reset_function might - * not use a PCIe FLR, we don't have that level of granularity. + * FLR, request a function reset once recovery_lock has been + * released. Regardless, clear the bit, the spec requires it to be + * always read as zero. NB, reset_function might not use a PCIe FLR, + * we don't have that level of granularity. */ if (*ctrl & cpu_to_le16(PCI_EXP_DEVCTL_BCR_FLR)) { u32 cap; @@ -907,14 +941,8 @@ static int vfio_exp_config_write(struct vfio_pci_core_= device *vdev, int pos, pos - offset + PCI_EXP_DEVCAP, &cap); =20 - if (!ret && (cap & PCI_EXP_DEVCAP_FLR)) { - vfio_pci_zap_and_down_write_memory_lock(vdev); - vfio_pci_dma_buf_move(vdev, true); - pci_try_reset_function(vdev->pdev); - if (__vfio_pci_memory_enabled(vdev)) - vfio_pci_dma_buf_move(vdev, false); - up_write(&vdev->memory_lock); - } + if (!ret && (cap & PCI_EXP_DEVCAP_FLR)) + deferred->flr =3D true; } =20 /* @@ -968,19 +996,22 @@ static int __init init_pci_cap_exp_perm(struct perm_b= its *perm) =20 static int vfio_af_config_write(struct vfio_pci_core_device *vdev, int pos, int count, struct perm_bits *perm, - int offset, __le32 val) + int offset, __le32 val, + struct vfio_pci_config_deferred *deferred) { u8 *ctrl =3D vdev->vconfig + pos - offset + PCI_AF_CTRL; =20 - count =3D vfio_default_config_write(vdev, pos, count, perm, offset, val); + count =3D vfio_default_config_write(vdev, pos, count, perm, offset, val, + deferred); if (count < 0) return count; =20 /* * The FLR bit is virtualized, if set and the device supports AF - * FLR, issue a reset_function. Regardless, clear the bit, the spec - * requires it to be always read as zero. NB, reset_function might - * not use an AF FLR, we don't have that level of granularity. + * FLR, request a function reset once recovery_lock has been + * released. Regardless, clear the bit, the spec requires it to be + * always read as zero. NB, reset_function might not use an AF FLR, + * we don't have that level of granularity. */ if (*ctrl & PCI_AF_CTRL_FLR) { u8 cap; @@ -992,14 +1023,8 @@ static int vfio_af_config_write(struct vfio_pci_core_= device *vdev, int pos, pos - offset + PCI_AF_CAP, &cap); =20 - if (!ret && (cap & PCI_AF_CAP_FLR) && (cap & PCI_AF_CAP_TP)) { - vfio_pci_zap_and_down_write_memory_lock(vdev); - vfio_pci_dma_buf_move(vdev, true); - pci_try_reset_function(vdev->pdev); - if (__vfio_pci_memory_enabled(vdev)) - vfio_pci_dma_buf_move(vdev, false); - up_write(&vdev->memory_lock); - } + if (!ret && (cap & PCI_AF_CAP_FLR) && (cap & PCI_AF_CAP_TP)) + deferred->flr =3D true; } =20 return count; @@ -1168,9 +1193,11 @@ static int vfio_msi_config_read(struct vfio_pci_core= _device *vdev, int pos, =20 static int vfio_msi_config_write(struct vfio_pci_core_device *vdev, int po= s, int count, struct perm_bits *perm, - int offset, __le32 val) + int offset, __le32 val, + struct vfio_pci_config_deferred *deferred) { - count =3D vfio_default_config_write(vdev, pos, count, perm, offset, val); + count =3D vfio_default_config_write(vdev, pos, count, perm, offset, val, + deferred); if (count < 0) return count; =20 @@ -1889,6 +1916,8 @@ ssize_t vfio_pci_config_rw_single(struct vfio_pci_cor= e_device *vdev, struct perm_bits *perm; __le32 val =3D 0; int cap_start =3D 0, offset; + int access_ret; + struct vfio_pci_config_deferred deferred =3D {}; u8 cap_id; ssize_t ret; =20 @@ -1957,14 +1986,42 @@ ssize_t vfio_pci_config_rw_single(struct vfio_pci_c= ore_device *vdev, if (copy_from_user(&val, buf, count)) return -EFAULT; =20 - ret =3D perm->writefn(vdev, *ppos, count, perm, offset, val); + access_ret =3D vfio_pci_core_access_begin(vdev); + if (access_ret) + return access_ret; + ret =3D perm->writefn(vdev, *ppos, count, perm, offset, val, + &deferred); + vfio_pci_core_access_end(vdev); + if (ret < 0) + return ret; + /* + * Both of these take pci_bus_sem, so run them with the access + * guard dropped. The reset re-checks the recovery state for + * itself. The power up does not, so check it here. + * + * Both are best effort, as the guest-requested FLR has always + * been. The result is not reported back through the config + * write. Without recovery enabled the only failure is -EAGAIN + * from device lock contention, exactly as before. With it they + * are dropped while a recovery or reset transaction is in + * flight, which leaves the device in D0 and reset anyway. + */ + if (deferred.power_up && + !(vdev->pci_recovery_supported && + READ_ONCE(vdev->pci_recovery_access_blocked))) + vfio_lock_and_set_power_state(vdev, PCI_D0); + if (deferred.flr) + vfio_pci_try_reset_function(vdev, false); } else { - if (perm->readfn) { + access_ret =3D vfio_pci_core_access_begin(vdev); + if (access_ret) + return access_ret; + if (perm->readfn) ret =3D perm->readfn(vdev, *ppos, count, perm, offset, &val); - if (ret < 0) - return ret; - } + vfio_pci_core_access_end(vdev); + if (ret < 0) + return ret; =20 if (copy_to_user(buf, &val, count)) return -EFAULT; --=20 2.43.0 From nobody Sat Sep 26 13:08:18 2026 Received: from BYAPR05CU005.outbound.protection.outlook.com (mail-westusazon11010033.outbound.protection.outlook.com [52.101.85.33]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2E00D47ACE9; Tue, 1 Sep 2026 09:34:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.85.33 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788255244; cv=fail; b=F8HDcuQfPhTfw/+RGmYerF/+xiX7ib8wOogZGKgpG+FOI28rdihEKGYvEMpWt7SmxzyV4fnRAokXtQ724JU2o77+2LJTJOqUK3PFPkMo/+T5HFz+q+WCh6uaux42f8eMH+xBWCmYCrvGZOn6nI5sIrwxO666Vkip+P9DXw5s2as= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788255244; c=relaxed/simple; bh=Fcd2St04MedTpPfLOqj+akdHwJ/JdWyLbXl2ahjPKDM=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=aFG/GPc9CV+FSBc8B+50ofYVOH9gaj8ogRFkaFFcXK6/ib42Cx4ZJIa8cAV4opEaaMbcmAJFJ91mIFcL0B9XQGvIUO4yfpY8sVR0HGJorOoN6EZNHppKyLhzstlMOUITKhUvr7P1MjmIgBoyXGaHruJ4F4Alu95UyjixrT44lzw= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=dfPJeB2q; arc=fail smtp.client-ip=52.101.85.33 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="dfPJeB2q" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=azvBqgORumC/0W63Ca+iWsppkz6joJ+DpVxhhPbY9yQESbT6buDjX19LbXqWfwbDZpBTOMzOnkPTfpvrbN2M7H9/QKw1XTZ3K2HvUjJosEnKAwlyTEcSvYFwGTMU5nK0fuH/2Q5QGLcXazHlC0v96zmsZFlbtgSEmyXnb5/Z0j555NKSJRQ/AFyclfFpOfL5n9hckNFL7jFJaemucPZG6/QqkZA7nxf0W1mYYmw4xD8dmgaGLGBjFft7ZyVOZSvjLMMY0ybIjn2YR8LZ8z4sj15kWvuJFJG3Ru4KMcLUUZ5XXU05tXgX8XHhwmSjdl6SfJ3jy+Yrvh+SkEUUg0E6Ow== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Lq7zjeRolvU5bJGzirYTyvOwGz2HoIXIjjGAtlBnHxU=; b=JsS1YRiHo88qK+vS6GK16umzrHzFowrl8JVkgXxa+XOA1h+pBol95nHXROV4kd+02Gl3QGw6jmbaHAgxA0oPizWl1y9ZACsf77NmuVY3LoCJKaMq/26TFTO0+AW894p2TbJ7EKHgWWwbVuAUjMK0fphL3vyyLgAoK1HKjeYtuJlaaGXfWJnLqFzH8H3lKsDRHBvUvL895X2fsyIZRwwiJm88AmuUD/k3S6Y/wK2wDJofpCh8mJ8T+9nSrMNQYdd7FkHmmRHx9ZNWAMGI3YIzlY+bIVpuOpVELdLtgDQ8u5Fo4Nh9thRXXoU5CornH1c03V9gQBQrUNt5loY3c6qMVg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Lq7zjeRolvU5bJGzirYTyvOwGz2HoIXIjjGAtlBnHxU=; b=dfPJeB2qMOCLYBHx1JjW/mF5bwuURNDC6Z4lmKHxLuosFLaprQV7asDoLd4UVcQMaU+hgJZ4R446KzNbkKpt9S+jc6KeE84lgQQiSx3JiGyXxFtxbndAyRIrmPlF7S5BX04kjlY6sOionuwC+Hkn/r9pks5QjkNgIAOAwzwSVTiqRqhiWWTso3RpQ77eQlUlkUbMTRXJ9FWyG4gk4v/Ma2AGaG9hNFn1ueVkLmJFDgXLaxoNCxpnScTxY2rh+yt+Ea1crr0yrgw/Iotzngx4nwxVlyOEgq5SAC6JziA40fSfj7IuQ8K0OzX+Qm8Y5mjrjD127sAXlu/MlGDqjLX9Qg== Received: from CH0PR13CA0047.namprd13.prod.outlook.com (2603:10b6:610:b2::22) by BL1PR12MB5754.namprd12.prod.outlook.com (2603:10b6:208:391::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.360.12; Tue, 1 Sep 2026 09:33:51 +0000 Received: from CH2PEPF00000144.namprd02.prod.outlook.com (2603:10b6:610:b2:cafe::6f) by CH0PR13CA0047.outlook.office365.com (2603:10b6:610:b2::22) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.382.10 via Frontend Transport; Tue, 1 Sep 2026 09:33:51 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by CH2PEPF00000144.mail.protection.outlook.com (10.167.244.101) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.382.8 via Frontend Transport; Tue, 1 Sep 2026 09:33:51 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.46; Tue, 1 Sep 2026 02:33:36 -0700 Received: from NV-2Y5XW94.nvidia.com (10.126.230.37) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.46; Tue, 1 Sep 2026 02:33:33 -0700 From: Shameer Kolothum To: , , CC: , , , , , , , , Subject: [RFC PATCH 06/19] vfio/pci: Serialize ioeventfd writes with recovery Date: Tue, 1 Sep 2026 10:32:04 +0100 Message-ID: <20260901093217.8539-7-skolothumtho@nvidia.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260901093217.8539-1-skolothumtho@nvidia.com> References: <20260901093217.8539-1-skolothumtho@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: rnnvmail203.nvidia.com (10.129.68.9) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PEPF00000144:EE_|BL1PR12MB5754:EE_ X-MS-Office365-Filtering-Correlation-Id: e99b1f56-16bc-49ea-72b1-08df080c2200 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|1800799024|376014|23010399003|36860700016|10067099003|56012099006|11063799006|18002099003|22082099003; X-Microsoft-Antispam-Message-Info: Hp0e7//UX0AdPvz6/WLNVrnx4pnptcrI9469IGOAOTsh4nl6ilWn34LGxwgWLbv3WKJclj1kQypafq/ONdHiXFGUMLoqUakFICNE1aSye9ssssYOGZe8nkz6HqQ5GHi/mqFxI8z43x39Hlmtfnc0reBVwSkv0zoST4DpMvSl/yXGlKq59PhloHndPjSBdcG0djOFYqEPJkgi/xApyksnS0xYHg2qh3AULabqp8W6++r/tg0Cdl9WKxKQRKIngbot3bef0/17OHgmtXzvE6ppHJqVXsEGNRe81n89tbbvT89mL00Iig12oQ6y/I2e8bM2KkpMaf6UzxhDei/iZVB93Cp83qIJFKY4/YBiGz4PupYcvjgrdie4zuhktk2PKh0n7W8kdstJGNbq0obd+C75G3LS5RJ5xx5Eb64CDReWl9/HdkufwuZalUPeqA9CCjUmMC6cW66Rjox64wcGHB2f6hWTcPB/r7RGDgyUMKt7lHi19HpoZPJQoRxK0O29qiy6k5mCQwby3De78B31fyvzBGye/TZxgTEdm+ax5aEtyRwMMvkx2kP3q0JRZyGRzDJuui3gAHeGVpg7tygIwWJ+X+axmupZ2Dddb7PhJrdoJpRMzl2X07/hy/j/AqN4QQfT4p/d8DndtOyPGP4Wg1tNgz4FUE+YMrGcaX2xhG7kAfjYcORpaR4oqYVVnBpfphkAugZT2ba9UlMckwjNgl/fFQ== X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(82310400026)(1800799024)(376014)(23010399003)(36860700016)(10067099003)(56012099006)(11063799006)(18002099003)(22082099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: B8H61ERdqEoYtvt1218XKstH3P8Hdlg9aC0Ofv/ZrywimPKN77Kd4k/Qbxah6DyRDs5S5D2I8IdilVUsWa0xhr34/mDVHGvqVTF5EIq8rRMpDi6LABcImgMeD8/BKmmEL/D+gJ0dMfIWDlMu2CTrDj365vg+83c9q2UAYf/PEvzf6sdhXohlyuvz+N4qeUWxTX4iqo04shSuobbqed5j9Z/AR7oWTAJBRlkNqO71AHLuHkhXWRMvkldLBPum4/hpM3Q2TrjqKBeHMGBpsK2V6GMVQkXUPxNW4MClMdVnR+i7RxJ7gerUxQtCBGmpSBTaWEHYMsWCjZxVcpopYOJSd77Rl0Q5STGAWU7G+zRQUkmhQykl0URDhEfd4hBjkY6wx/7BjVeRuS7lTysLL6IydygP5zJHACM+wb/iYf/4MCO2ICKebzuGKeXeN1RbvUEP X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Sep 2026 09:33:51.1408 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: e99b1f56-16bc-49ea-72b1-08df080c2200 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CH2PEPF00000144.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL1PR12MB5754 Content-Type: text/plain; charset="utf-8" Share one write helper between the ioeventfd fast path and the threaded one. It takes memory_lock, checks the recovery state, then writes. The fast path runs from the virqfd wakeup with a spinlock held, so it trylocks and hands off to the thread if the lock is busy. The thread can block. The ioeventfd write path must not take recovery_lock at all. It is reached through flush_work() from the virqfd cleanup workqueue, and VFIO_DEVICE_SET_IRQS later calls vfio_virqfd_disable(), which does that flush while holding recovery_lock for reading. If the write then blocked on recovery_lock behind a queued AER writer, all three would be stuck. The flush waits for the write, the write waits for the AER writer, and the AER writer waits for the reader driving the flush. So the recovery state is read lock-free, and the write goes through the raw vfio_iowrite*() accessors rather than vfio_pci_core_iowrite*(). A later patch makes those take recovery_lock, which is what this path has to stay clear of. memory_lock is still what drains a write already under way, and the order is what makes it safe. The lock is taken before the flag is read, so a write which saw the flag clear is already holding the read side, and recovery waits for it when it takes memory_lock for writing. A write to an I/O port BAR takes no memory_lock and is not drained. Those are best effort. So is a write which arrives just after a reset has finished, since VFIO_DEVICE_RESET releases memory_lock before it retakes recovery_lock to unblock access. The check is skipped for drivers which do not advertise support. pci_recovery_device_open is only ever set when the recovery machinery is live, so testing it unconditionally would drop every ioeventfd write for every other vfio-pci-core driver. Assisted-by: Claude:claude-opus-5 Signed-off-by: Shameer Kolothum --- drivers/vfio/pci/vfio_pci_rdwr.c | 91 +++++++++++++++++++++----------- 1 file changed, 60 insertions(+), 31 deletions(-) diff --git a/drivers/vfio/pci/vfio_pci_rdwr.c b/drivers/vfio/pci/vfio_pci_r= dwr.c index 7f14dd46de17..20362e2f0166 100644 --- a/drivers/vfio/pci/vfio_pci_rdwr.c +++ b/drivers/vfio/pci/vfio_pci_rdwr.c @@ -349,56 +349,85 @@ ssize_t vfio_pci_vga_rw(struct vfio_pci_core_device *= vdev, char __user *buf, } #endif =20 -static void vfio_pci_ioeventfd_do_write(struct vfio_pci_ioeventfd *ioevent= fd, - bool test_mem) +static int vfio_pci_ioeventfd_do_write(struct vfio_pci_ioeventfd *ioeventf= d, + bool trylock) { + struct vfio_pci_core_device *vdev =3D ioeventfd->vdev; + + if (ioeventfd->test_mem) { + if (trylock) { + if (!down_read_trylock(&vdev->memory_lock)) + return 1; /* Lock contended, use thread */ + } else { + down_read(&vdev->memory_lock); + } + } + + /* + * Read the recovery state lock-free rather than under recovery_lock. + * This path runs from the virqfd cleanup workqueue, which is flushed + * from paths that take recovery_lock for reading, so blocking on it + * here would deadlock behind a queued writer. + * + * For a memory BAR, a blocked device still waits for a write already + * under way, through memory_lock. The lock is taken above before the + * flag is read, so a write which saw the flag clear is already + * holding the read side, and the blocker waits for it when it takes + * memory_lock for writing. An I/O port BAR takes no memory_lock, so + * a write which saw the flag clear can still land afterwards. Port + * writes are best effort here. + * + * A write can also be dropped for a short while after a reset has + * finished, since VFIO_DEVICE_RESET releases memory_lock before it + * retakes recovery_lock to unblock access. Closing that would mean + * taking recovery_lock inside memory_lock, which is the wrong way + * round. + * + * pci_recovery_device_open records that the recovery machinery is + * live, so it is only ever set for drivers which advertise support. + * Testing it unconditionally would drop every write for every other + * driver. + * + * The raw vfio_iowrite*() accessors below are used for the same + * reason. This path must not take recovery_lock. + */ + if (vdev->pci_recovery_supported && + (!READ_ONCE(vdev->pci_recovery_device_open) || + READ_ONCE(vdev->pci_recovery_access_blocked))) + goto out_memory; + + if (ioeventfd->test_mem && !__vfio_pci_memory_enabled(vdev)) + goto out_memory; + switch (ioeventfd->count) { case 1: - vfio_pci_core_iowrite8(ioeventfd->vdev, test_mem, - ioeventfd->data, ioeventfd->addr); + vfio_iowrite8(ioeventfd->data, ioeventfd->addr); break; case 2: - vfio_pci_core_iowrite16(ioeventfd->vdev, test_mem, - ioeventfd->data, ioeventfd->addr); + vfio_iowrite16(ioeventfd->data, ioeventfd->addr); break; case 4: - vfio_pci_core_iowrite32(ioeventfd->vdev, test_mem, - ioeventfd->data, ioeventfd->addr); + vfio_iowrite32(ioeventfd->data, ioeventfd->addr); break; case 8: - vfio_pci_core_iowrite64(ioeventfd->vdev, test_mem, - ioeventfd->data, ioeventfd->addr); + vfio_iowrite64(ioeventfd->data, ioeventfd->addr); break; } -} - -static int vfio_pci_ioeventfd_handler(void *opaque, void *unused) -{ - struct vfio_pci_ioeventfd *ioeventfd =3D opaque; - struct vfio_pci_core_device *vdev =3D ioeventfd->vdev; - - if (ioeventfd->test_mem) { - if (!down_read_trylock(&vdev->memory_lock)) - return 1; /* Lock contended, use thread */ - if (!__vfio_pci_memory_enabled(vdev)) { - up_read(&vdev->memory_lock); - return 0; - } - } - - vfio_pci_ioeventfd_do_write(ioeventfd, false); =20 +out_memory: if (ioeventfd->test_mem) up_read(&vdev->memory_lock); - return 0; } =20 -static void vfio_pci_ioeventfd_thread(void *opaque, void *unused) +static int vfio_pci_ioeventfd_handler(void *opaque, void *unused) { - struct vfio_pci_ioeventfd *ioeventfd =3D opaque; + return vfio_pci_ioeventfd_do_write(opaque, true); +} =20 - vfio_pci_ioeventfd_do_write(ioeventfd, ioeventfd->test_mem); +static void vfio_pci_ioeventfd_thread(void *opaque, void *unused) +{ + vfio_pci_ioeventfd_do_write(opaque, false); } =20 int vfio_pci_ioeventfd(struct vfio_pci_core_device *vdev, loff_t offset, --=20 2.43.0 From nobody Sat Sep 26 13:08:18 2026 Received: from CH5PR02CU005.outbound.protection.outlook.com (mail-northcentralusazon11012052.outbound.protection.outlook.com [40.107.200.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 708C7477E20; Tue, 1 Sep 2026 09:34:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.200.52 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788255255; cv=fail; b=UPebMVDWMwII6eykMzwm+m5TKESjXX5SpL9Wf8SaT7XPyy9WQU1y2MX6Q3IMiXe59OmVUWpyZkrOuJvmkjU6GuLjXdrUH6JeEbK6iKWsVsDI5NE/w2vRz6ifGynYy19henKzySRFJTfrKUqEmzwf9PL9m96MfzGiJtdRDConXN8= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788255255; c=relaxed/simple; bh=By0QsfUtpSurdQxUMLis1iXJ5ebYafw/x0bA+xUAPMo=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=h5Z0UqHcquqkU9Is+a9mLXV/V2F3lNF6BjU24zDeKUDLmezfIHohpDjw94atB9VHVCXeIlPntJWe6JHAGp7c9NanO7ysySNkbiuRupjugMi3vHpHJf66a2t09MZuBLGenxzyVVG0Hla7KAm9/idHpaxcWEfewRzKCLbupM6ZDQs= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=KXr0CnPK; arc=fail smtp.client-ip=40.107.200.52 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="KXr0CnPK" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=avRQZkNL65Zgjvkot+P4uvZuEW+scC/TPEszu+OVQeWVeCqrodRgi5u1f6tsVYg1660iCzDle62US5fU7c/DBA5d5Ih2BpkQIJ4mAIa9Hpen4aa50VgRe/GJlx/xZnWsmOSOg6p+DFqqBWwsgOiHNoF58BoRKQuEcz4ivpMKkhoHwk+W2BxLLcY5a7u5DYsz/IJQm0baeClhFwq9xEphzhzdJksfxo4Ob6z3sqD7m+6tYEHa+R+4IXKfsgVr5qxH6laQuHkRfmtr/Psk1tRGavqx7dEd9zYmAyLwpq7O+ewbWciHfmjqFpqLLb97LTxe/AUGxhYfUdA1uo1VoLRqpw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=YSZ7bqoz9YWBc1j0WC3EsEqKtG1xlz7Hj91FGbI4dzU=; b=duFcsR+FLqg/F3H40V8n08Nd7Ed8izHT1ODQtJT5Wfua+/cP18MvuBuYnKJNqpPGCFDfL6DtXOjGPai44C4nuMwq542O97jKdcI/jK+hDGRH06IAWtLWxooqrF4ZFRqzKGdJXZfgl8r9bUlQ73Paemy1U4iZrED+hLBtco3S/wA1mrYKyVsTK463D2z9xn0Ph68yA1V3NvN5JsZh73jGoYTKaFoZaaP0XZ/YMLkDu4mEH6z+V8WVVRK+jRqdD3qZBKCmHW9p3KSyEjrPRNNyKXmED+H5Z8a5fR49Sf2QGFdLrX2M7ng6iZrDKtCp0ZUl3zC9ZZbm144SpsDG7UG5ig== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=YSZ7bqoz9YWBc1j0WC3EsEqKtG1xlz7Hj91FGbI4dzU=; b=KXr0CnPKPFm9KP1wLlTfhK9CQBpm7d5z6EuFdgc3lG+tQju5ft7zflOKWCNFWpili7G1zoujlDboRBeTW4urPK9KSn6f+XcK05KmMLqqdk/pEVw7K98I5xzU++70fNQdUxAiyja9VPS2Vz6sIAwcXuai2DumybIUd0zeLP2CsquZQvQAv6juvZZh5koT8QNxzvKMvlghiieRwmw7kgtTj2Av9x3kfI/fRyCX+82FvSlt0lB49Bzkk+kCoI8vS7w1AJx7i2UO9kBsfpMl6GSfHDf4SDQW5W+i7t6O7rOB0PrHd4n61j2r1lQUqM0MwWMMWP7eafmfwo/x0mr50aaefQ== Received: from CH0PR04CA0104.namprd04.prod.outlook.com (2603:10b6:610:75::19) by SA1PR12MB8721.namprd12.prod.outlook.com (2603:10b6:806:38d::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.360.13; Tue, 1 Sep 2026 09:33:55 +0000 Received: from CH2PEPF00000147.namprd02.prod.outlook.com (2603:10b6:610:75:cafe::11) by CH0PR04CA0104.outlook.office365.com (2603:10b6:610:75::19) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.382.10 via Frontend Transport; Tue, 1 Sep 2026 09:33:54 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by CH2PEPF00000147.mail.protection.outlook.com (10.167.244.104) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.382.8 via Frontend Transport; Tue, 1 Sep 2026 09:33:54 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.46; Tue, 1 Sep 2026 02:33:39 -0700 Received: from NV-2Y5XW94.nvidia.com (10.126.230.37) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.46; Tue, 1 Sep 2026 02:33:36 -0700 From: Shameer Kolothum To: , , CC: , , , , , , , , Subject: [RFC PATCH 07/19] vfio/pci: Retry BAR faults after temporary recovery Date: Tue, 1 Sep 2026 10:32:05 +0100 Message-ID: <20260901093217.8539-8-skolothumtho@nvidia.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260901093217.8539-1-skolothumtho@nvidia.com> References: <20260901093217.8539-1-skolothumtho@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: rnnvmail203.nvidia.com (10.129.68.9) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PEPF00000147:EE_|SA1PR12MB8721:EE_ X-MS-Office365-Filtering-Correlation-Id: bf2b1b02-bd07-4780-e18f-08df080c2424 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|376014|23010399003|36860700016|82310400026|10067099003|56012099006|11063799006|18002099003|22082099003; X-Microsoft-Antispam-Message-Info: AMyUaDAtrz1ftMPXMZqyt3IQ6YX/KrN3rrthjaxofLuYO1zFRPT7rBWoEa8rD5g1tOTEdUjwSYapFJxXlPJ1QLNaobdOmZE0/jn6sBovehvbdTe8BVKh11Et7724N4CJFOpwibORRC7cH/5cBcV2um32zrlhAwK1xNioeXfVhT5Lkyf7yD8HLYKCwsMCwa+/6DGXOCKQhoIVFbPzECbkgVdYYkBiTV0Txs3CuckH4vW8HPOAiR0XqmUF7kbXcM64Q/cxqaXgUGGAIy4UotNRkxEKm68vSPVeBAjR69CvCUKXg4tay0l8y2FVkNWk+pgVmJT133cjU7rRLv9g4kzSoogdAHfIuqS3garl6tF7VuZ5a5KEy4ldI/RMVYxvS9xhNltgCmk2pf5ncKOpEsh9RgQUEyJZiLaUeluYrupwf8DfUixBYV6EtBXHQs/nz8ngmu6eKU+a17GIISecFmUzMDgFjksqhdOvzI29fLAzJ14F1yeNUoJDEH7dwqTvGcvZs7eYgTkRY22SU4aGarP/LhuibNQ+BSl/r42U5D5vJ4o7tkKZq1V2SjXLTRuxEosT5nicFUq6km5tUKIxdnXFhFG+RMMZlUmcctM32sNnQRS4anokdF5f1bWPyiS1dt+qexhdv3ZhlaocJf+IqkTxb/qYICsK5b0xsrjqErciBgdE8fS6D0UjPuLHHVCuF8BfUZDapqpr0f2E3+xHf8G6oA== X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(1800799024)(376014)(23010399003)(36860700016)(82310400026)(10067099003)(56012099006)(11063799006)(18002099003)(22082099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: hMSSfGpDqmYWdOPoiga7qZYY48yeglhjFtHmSHXQ9uR2uYIapeEoZLZdogZ993w3QaqUKerZ+LjVLfPyIieabV+d2KC+EnBckUWh33DNCataPC6NKTl9S3q5rD85l03wlDquonktSLdC2bBHuRxKArIE5uoGrAuP6jcpLNCvfMGhALmo8WUFshQHNegSD3xDw1js//H6N6QvOH15iadmsADatxHHKrrp6uwtRKmg4PwBBCJGNcTu64efREJbOumWVOW+dtH4PbrGs+5kUNNRDfd0Q2NZTkbynLQFRUmEWkiCYuGyVglNF3++R/nTyMAzBI0lkOOwpBwKuPbKeIAQnlXTW+vriJCviysSSgtK8tab47US+iKNfL6wCFiZwvwPPUFyKND5sLuhRFw24wGQY935CcGqBfkScfNoa428VkawqPlQJFO918K5GfEAMNuJ X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Sep 2026 09:33:54.7344 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: bf2b1b02-bd07-4780-e18f-08df080c2424 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CH2PEPF00000147.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR12MB8721 Content-Type: text/plain; charset="utf-8" A guest can fault on a mapped BAR while host recovery is running. Do not insert the PFN in that case. The device is not usable until recovery is finished. Wait whenever access is blocked, not only while a recovery transaction is in progress. A function reset blocks access without starting one, and error_detected() blocks it before it publishes the flags, so a fault in either window would otherwise fail for good. Only a closed device, or one which has failed for good, ends the fault, which is what VFIO_PCI_RECOVERY_FAILED records. On the first attempt the fault lock can be dropped, so drop it, wait for recovery, and return VM_FAULT_RETRY to bring the fault back later. The wait is killable. Take a reference on the device registration before dropping the lock, because the wait outlives the lock and the device could go away. This is the FAULT_FLAG_ALLOW_RETRY set and FAULT_FLAG_TRIED clear case. Once the lock is dropped the VMA may be gone, so return without touching it. The caller returns early too and skips its debug print, which reads both the VMA and the device. When the fault lock cannot be dropped, because the caller did not allow a retry or this fault has already used one, wait with it held. Look at the state once more after that wait and return SIGBUS if recovery is still not done, rather than wait again with the lock held. That fails a fault which might still have recovered, but the window is narrow. The wait condition is read without recovery_lock, so it only says when to look again. Every path which unblocks access wakes the queue, and the decision itself is taken under the lock on the next look. It is not a deadlock. Recovery revokes mappings with unmap_mapping_range(), which does not take mmap_lock. The wait is killable. SIGBUS is also what a closed device, or one which has failed for good, returns. The order matters. The fault takes memory_lock before it releases recovery_lock, so from the check until the PFN is in it always holds at least one of the two. Recovery needs both, so it cannot finish revoking while a fault is part way through. If the fault let go of recovery_lock before taking memory_lock, recovery could slip into that window and revoke everything, and the fault would then map a PFN for a device which was already revoked. Assisted-by: Claude:claude-opus-5 Signed-off-by: Shameer Kolothum --- drivers/vfio/pci/vfio_pci_core.c | 128 ++++++++++++++++++++++++++++++- 1 file changed, 126 insertions(+), 2 deletions(-) diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_c= ore.c index 3645daa8891f..d46448662e84 100644 --- a/drivers/vfio/pci/vfio_pci_core.c +++ b/drivers/vfio/pci/vfio_pci_core.c @@ -1937,6 +1937,127 @@ vm_fault_t vfio_pci_vmf_insert_pfn(struct vfio_pci_= core_device *vdev, } EXPORT_SYMBOL_GPL(vfio_pci_vmf_insert_pfn); =20 +/* + * Whether a fault which found access blocked is worth retrying. Read + * without recovery_lock, so it is only a hint about when to look again. + * vfio_pci_fault_trylock_once() takes the lock and decides. Read the flags + * once so the two tests below see the same value. Every writer which can + * make this true wakes pci_recovery_wait. + */ +static bool vfio_pci_recovery_done(struct vfio_pci_core_device *vdev) +{ + u32 flags =3D READ_ONCE(vdev->pci_recovery_flags); + + if (!READ_ONCE(vdev->pci_recovery_device_open)) + return true; + if (flags & VFIO_PCI_RECOVERY_IN_PROGRESS) + return false; + if (flags & VFIO_PCI_RECOVERY_FAILED) + return true; + return !READ_ONCE(vdev->pci_recovery_access_blocked); +} + +static int vfio_pci_wait_for_recovery(struct vfio_pci_core_device *vdev) +{ + return wait_event_killable(vdev->pci_recovery_wait, + vfio_pci_recovery_done(vdev)); +} + +/* What one look at the recovery state says the fault should do. */ +enum vfio_pci_fault_action { + VFIO_PCI_FAULT_PROCEED, /* returns with memory_lock held */ + VFIO_PCI_FAULT_WAIT, /* recovery is running, may still recover */ + VFIO_PCI_FAULT_FAIL, /* closed, or failed for good */ +}; + +static enum vfio_pci_fault_action +vfio_pci_fault_trylock_once(struct vfio_pci_core_device *vdev) +{ + enum vfio_pci_fault_action action; + + down_read(&vdev->recovery_lock); + if (!vdev->pci_recovery_device_open || + (vdev->pci_recovery_flags & VFIO_PCI_RECOVERY_FAILED)) { + action =3D VFIO_PCI_FAULT_FAIL; + } else if (vdev->pci_recovery_access_blocked) { + /* + * Blocked for a reason which still ends: a recovery which has + * not failed, or a function reset. Test FAILED above rather + * than IN_PROGRESS here, so a fault does not fail for good + * while a reset is running, or in the window where + * error_detected() has blocked access but not yet published + * the flags. + */ + action =3D VFIO_PCI_FAULT_WAIT; + } else { + down_read(&vdev->memory_lock); + action =3D VFIO_PCI_FAULT_PROCEED; + } + up_read(&vdev->recovery_lock); + + return action; +} + +/* + * Return true with memory_lock held for a fault that may proceed. Otherwi= se + * return false with @ret set to the result the fault handler should retur= n. + */ +static bool vfio_pci_core_fault_trylock(struct vfio_pci_core_device *vdev, + struct vm_fault *vmf, + vm_fault_t *ret) +{ + if (!vdev->pci_recovery_supported) { + down_read(&vdev->memory_lock); + return true; + } + + switch (vfio_pci_fault_trylock_once(vdev)) { + case VFIO_PCI_FAULT_PROCEED: + return true; + case VFIO_PCI_FAULT_FAIL: + *ret =3D VM_FAULT_SIGBUS; + return false; + case VFIO_PCI_FAULT_WAIT: + break; + } + + if (fault_flag_allow_retry_first(vmf->flags)) { + if (vmf->flags & FAULT_FLAG_RETRY_NOWAIT) { + *ret =3D VM_FAULT_RETRY; + return false; + } + + if (!vfio_device_try_get_registration(&vdev->vdev)) { + *ret =3D VM_FAULT_SIGBUS; + return false; + } + + release_fault_lock(vmf); + vfio_pci_wait_for_recovery(vdev); + vfio_device_put_registration(&vdev->vdev); + *ret =3D VM_FAULT_RETRY; + return false; + } + + /* + * The fault lock cannot be dropped here: either the caller did not + * allow a retry, or this fault has already used one. So wait with + * it held. It is not a deadlock. Recovery revokes mappings through + * unmap_mapping_range(), which never takes mmap_lock. The wait is + * killable. + */ + if (vfio_pci_wait_for_recovery(vdev)) { + *ret =3D VM_FAULT_NOPAGE; + return false; + } + + if (vfio_pci_fault_trylock_once(vdev) =3D=3D VFIO_PCI_FAULT_PROCEED) + return true; + + *ret =3D VM_FAULT_SIGBUS; + return false; +} + static vm_fault_t vfio_pci_mmap_huge_fault(struct vm_fault *vmf, unsigned int order) { @@ -1948,8 +2069,11 @@ static vm_fault_t vfio_pci_mmap_huge_fault(struct vm= _fault *vmf, vm_fault_t ret =3D VM_FAULT_FALLBACK; =20 if (is_aligned_for_order(vma, addr, pfn, order)) { - scoped_guard(rwsem_read, &vdev->memory_lock) - ret =3D vfio_pci_vmf_insert_pfn(vdev, vmf, pfn, order); + if (!vfio_pci_core_fault_trylock(vdev, vmf, &ret)) + return ret; + + ret =3D vfio_pci_vmf_insert_pfn(vdev, vmf, pfn, order); + up_read(&vdev->memory_lock); } =20 dev_dbg_ratelimited(&vdev->pdev->dev, --=20 2.43.0 From nobody Sat Sep 26 13:08:18 2026 Received: from DM1PR04CU001.outbound.protection.outlook.com (mail-centralusazon11010071.outbound.protection.outlook.com [52.101.61.71]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3C8D036B926; Tue, 1 Sep 2026 09:34:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.61.71 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788255258; cv=fail; b=IVc4wCt1eDzoQB6NrOxTWli6RaH7UqE4FKEL9fd/UBGnVQmIJrxH4QwcopY2mvhmRXSV80zSlF3iuqXkW/AITaZhFdbDbDs7A7Gi+lSdyz5CfAZjyDEz+OOq6+kLuPkbqaqMmUs1zKVdflEvHgy3rCtj9rE9kmeCW7XoI1c6krQ= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788255258; c=relaxed/simple; bh=8ivqZZgdMuIQ9V3KKVBrZiAt6qhAD1TqSkvMyRdUbU4=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=GAsCmsId4OEBhdWzw268gLvlfY9BOd60kGRoMsEmzerrR9RPK37WUEc3vOQ1+Qr2A+3Knbsb/acfrpdxKTqDXcYlrs8JLBYfloUr8vF3U8U81Izk3yd4VvG4rEqFnUtMFIf/P+M3wmLNWygXij40M6PIWZ3HdSqJjaqV6H0Wx8A= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=nv1McUTS; arc=fail smtp.client-ip=52.101.61.71 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="nv1McUTS" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=rGDXLMZKfyIGGfV2/xHvkMLmaM/GLg8uv3IHLvrgOGv6G+WUTUGq1hiRfzcJt6JSIuGCToND5JxZVX61QgbKJbVtkrgkjyQh7W9nitH2isMqFwhgO91YOeiitKvkFNVbCN/ESvv0eT/1SdgM5xQNP4bsd0gvLRQc9Yqe3VeFjz7LPdZ6XpdsNomfyco9IFDHAfG0nUs1KX0RmZ79gxnqYsWILa8mhAkByaPukhMhCHjOelEej7/RgMCS2iBJWnBmWeL7K5vDWeJrUhd/vaDqAE3Uv4Bc5q4oP7P3jbWLHQ/35gMgnTIV5krM9/e8Avq22vaoUx7bUkFCyQTL7B99tw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=D0mF3TibbH4dPDdtyb79PKJtHSTe49D+Js5wVI8oRnI=; b=yeoMEZ+/FzE47uFTdinsJdZ5TRtAhOfQduRuLrwePELVdu/pk/aZmLxcF9X8rmhg7u2cEv7YF5yh3pXOEXDVwunXdHDT40iNQSqSSbQ1aBjJL16sWzmpvBH+vMUFbtwPEGjZc4cfT0NKzVO9wvkpsVMzlPsJ669EYbKCqykZP3MW7oPxROQM3UeVPRnKmdUB99fCIhP+N9S/NWk9njXpLx2lDLo3zLHi3s/L/1LKVOhS5TWPBIJ6SMREZoxn2do6b/FdEL732IbxCg4rpe1xr5GKfAfU/6NklddtqbK/RwM92uhrxjIBXEyGARHcXguijbrQlncUQPzfGLvN+jqVNQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=D0mF3TibbH4dPDdtyb79PKJtHSTe49D+Js5wVI8oRnI=; b=nv1McUTSQWeStbl0NfMDZhr/ZOWxFKKbb/X0tByVIYyfpPKTZaGfLXqY1zbwhnMhZWQStChnSRDa4JHMG/ma36UG+iCyCGsYk+iW1FNqpY2h6p0J4AfiIq4bbYDjOegioEkriWIlFSzAYACWxXoFUv+A6IYuBdPr2/8D986TFatW3B49jM0Fb5X5Rgmexz9SSrhcJ+juByKqnvZQqRn3lrmU/MnKy/ynOV++k5kkK7pmv/2ir6iUlid2wx1vykKF/12h70/8e24LBbAqQ+BgrYqIdgFghkFyKuGBhLD3eLAn1lMvhp+sh30wEjU2ihsi0tPRbOLHf6cfToCkUGu0ug== Received: from CH0P221CA0020.NAMP221.PROD.OUTLOOK.COM (2603:10b6:610:11c::21) by PHXPR12MB999231.namprd12.prod.outlook.com (2603:10b6:510:3ce::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.360.13; Tue, 1 Sep 2026 09:33:59 +0000 Received: from CH2PEPF0000014A.namprd02.prod.outlook.com (2603:10b6:610:11c:cafe::8e) by CH0P221CA0020.outlook.office365.com (2603:10b6:610:11c::21) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.382.10 via Frontend Transport; Tue, 1 Sep 2026 09:33:59 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by CH2PEPF0000014A.mail.protection.outlook.com (10.167.244.107) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.382.8 via Frontend Transport; Tue, 1 Sep 2026 09:33:59 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.46; Tue, 1 Sep 2026 02:33:43 -0700 Received: from NV-2Y5XW94.nvidia.com (10.126.230.37) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.46; Tue, 1 Sep 2026 02:33:40 -0700 From: Shameer Kolothum To: , , CC: , , , , , , , , Subject: [RFC PATCH 08/19] vfio/pci: Serialize BAR and ROM access with recovery Date: Tue, 1 Sep 2026 10:32:06 +0100 Message-ID: <20260901093217.8539-9-skolothumtho@nvidia.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260901093217.8539-1-skolothumtho@nvidia.com> References: <20260901093217.8539-1-skolothumtho@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: rnnvmail203.nvidia.com (10.129.68.9) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PEPF0000014A:EE_|PHXPR12MB999231:EE_ X-MS-Office365-Filtering-Correlation-Id: 1f8a0117-30e7-42f8-199a-08df080c2701 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|36860700016|82310400026|23010399003|1800799024|376014|10067099003|11063799006|56012099006|18002099003|22082099003; X-Microsoft-Antispam-Message-Info: /a9U0Y2bZWCg9Zs9Xd4zmCWsFLfL84rKnvGYEV7axBSdOikL9DueObzqAwymw7pSHBKBYfl82JGEp3z98f3OY7kjfXR2e8hpCH9ucU4+q52cHvf1jAWjKOZofd2Cci9UhpYY/Ebrf38DsWuZPlbIZXGW3x6h+qocI7zPFwxhspJA/uqkJRLYu2StzixUIDnTQnthURYGtQducv1zCYexeGBMTfeTw+jcgQQgbxGQJhoR+YZh/Qr4vBNjCpn+A+Hp2SFcVCciYPViRteAq3aKrZNqtl7+i2Crtslt8gDeesLq/jI02RrXMBkqtW+dsQ77psYzjabx63a8pJGc05I6V3tS+9zisX5NdCSWSp4BvKAQ1wTFc7T82EzS44c0hJrSUZfWa8MK1KduCev+IvGn+kTUdDmMiZtUNqnmxfnELJNCH6NYA+L0uh5yJTOG6jNd7OzHBnHmNpqUhZLIZuUYv8mBk/XZ9ljxq+8i7mgFTLPCNVgXi4+RDKx9EJ832nZ+XqqbRkzGuuQkNwVtq4Gy+m9dGE7Ze1vILCMqWRgPxIBpvNYMeMASE46uHTVxRFIvGiI/NDpo6jOtajETeLBp8cCC2gu9QZXIIaGjfLcakUYR27hPej6yhzIaOieUnWAMLYMdeQSWAPqveZM+ouolddeTCReI8hTjfBvQ31i+s5pMXBzVrsYXXY0MyUCffIxWKaB5cxgyS133rAulDPNI7A== X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(36860700016)(82310400026)(23010399003)(1800799024)(376014)(10067099003)(11063799006)(56012099006)(18002099003)(22082099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: WPvYe+5QljtGjBDBNyfl/StKjXqlndxi18iPf+twPUbGu7kvo1Psexug5rP6KnN6fWC+YZuA1zFBryC3qx4+d+dio9Lbq5SxSowxx8dBr8BchUbqq4+TdkfygDZ57yJHR5D2GGRWvn0bMOKpiJpUwnrDJgDfvtrn8BlK2Wi8WsCK6wt0N1CsBZJwLLho7GcS3fLTLow8Lu2mLy9jtIZwp9H5cAzteNepsrFcmNejtuEPtvbmbLRxNahJNN4uIj2aGi4DM22V75ODmJ4ct0pTPOkAnC6xRB2U4Cd6PGmFTHuPVSJox6txq5ukiGwCwG1UowaUcGDVgd8Ed5RV8wU+Bv6tnypMG0tl1rv53yKCFRwTdLYnWoNoTnfE08wtzDzT6GVOAi4taf3CWTxmse+BgNisOqni7pl5Cu6MVS1I6bcpefjOxjJPa/rMlxcxcP2u X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Sep 2026 09:33:59.5273 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 1f8a0117-30e7-42f8-199a-08df080c2701 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CH2PEPF0000014A.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PHXPR12MB999231 Content-Type: text/plain; charset="utf-8" Hold recovery_lock for reading around trapped BAR reads and writes, and around the ROM mapping, so they do not run while host recovery has access blocked. The lock is taken inside the width-specific I/O helpers, one access at a time, rather than across the whole transfer. copy_to_user() and copy_from_user() run in the callers of those helpers and so stay outside it. A user buffer can fault, and with userfaultfd the fault is serviced by userspace, so holding recovery_lock across the copy would let a user stall error_detected() for as long as it likes. VFIO_DEVICE_GET_REGION_INFO probes the ROM the same way, enabling memory decode and mapping it to see whether the contents are valid, so guard that too. Mapping and unmapping the ROM both write config space: pci_map_rom() enables decode, and assigns the resource first if it has none, and pci_unmap_rom() disables it again. If recovery has blocked access, do the iounmap and record the disable in pci_recovery_rom_disable instead. recovery_lock is held across the decision and the record so recovery cannot complete in between. Do the recorded disable from vfio_pci_try_reset_function() once the reset has finished, and from the resume() handler a later patch adds. Both are points where whatever blocked access has ended. A closed device is skipped, since close puts the device back through reset and config restore without holding recovery_lock. Assisted-by: Claude:claude-opus-5 Signed-off-by: Shameer Kolothum --- drivers/vfio/pci/vfio_pci_core.c | 32 +++++++++++++++++++- drivers/vfio/pci/vfio_pci_rdwr.c | 51 +++++++++++++++++++++++++++++++- 2 files changed, 81 insertions(+), 2 deletions(-) diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_c= ore.c index d46448662e84..0b1b2398dc88 100644 --- a/drivers/vfio/pci/vfio_pci_core.c +++ b/drivers/vfio/pci/vfio_pci_core.c @@ -1244,6 +1244,9 @@ int vfio_pci_ioctl_get_region_info(struct vfio_device= *core_vdev, * Check ROM content is valid. Need to enable memory * decode for ROM access in pci_map_rom(). */ + ret =3D vfio_pci_core_access_begin(vdev); + if (ret) + return ret; cmd =3D vfio_pci_memory_lock_and_enable(vdev); io =3D pci_map_rom(pdev, &size); if (io) { @@ -1254,6 +1257,7 @@ int vfio_pci_ioctl_get_region_info(struct vfio_device= *core_vdev, pci_unmap_rom(pdev, io); } vfio_pci_memory_unlock_and_restore(vdev, cmd); + vfio_pci_core_access_end(vdev); } else if (pdev->rom && pdev->romlen) { info->flags =3D VFIO_REGION_INFO_FLAG_READ; /* Report BAR size as power of two. */ @@ -1379,6 +1383,30 @@ static int vfio_pci_ioctl_set_irqs(struct vfio_pci_c= ore_device *vdev, return ret; } =20 +/* + * Complete a ROM unmap which could not disable decode through config spac= e. + * Call once whatever blocked access has finished. A closed device is skip= ped. + * It runs without recovery_lock, and close puts the device back through r= eset + * and config restore. + * + * The IORESOURCE_ROM_ENABLE test is what pci_unmap_rom() would have done. + * A ROM which firmware left enabled is not ours to turn off. + */ +static void vfio_pci_recovery_rom_disable(struct vfio_pci_core_device *vde= v) +{ + struct pci_dev *pdev =3D vdev->pdev; + + lockdep_assert_held_write(&vdev->recovery_lock); + + if (!vdev->pci_recovery_device_open || + !READ_ONCE(vdev->pci_recovery_rom_disable)) + return; + + if (!(pdev->resource[PCI_ROM_RESOURCE].flags & IORESOURCE_ROM_ENABLE)) + pci_disable_rom(pdev); + WRITE_ONCE(vdev->pci_recovery_rom_disable, false); +} + int vfio_pci_try_reset_function(struct vfio_pci_core_device *vdev, bool reset_power_state) { @@ -1455,8 +1483,10 @@ int vfio_pci_try_reset_function(struct vfio_pci_core= _device *vdev, */ if (vdev->pci_recovery_device_open && !(vdev->pci_recovery_flags & (VFIO_PCI_RECOVERY_IN_PROGRESS | - VFIO_PCI_RECOVERY_FAILED))) + VFIO_PCI_RECOVERY_FAILED))) { + vfio_pci_recovery_rom_disable(vdev); WRITE_ONCE(vdev->pci_recovery_access_blocked, false); + } up_write(&vdev->recovery_lock); /* * Access is blocked for the length of the reset, so anything diff --git a/drivers/vfio/pci/vfio_pci_rdwr.c b/drivers/vfio/pci/vfio_pci_r= dwr.c index 20362e2f0166..86fadc999962 100644 --- a/drivers/vfio/pci/vfio_pci_rdwr.c +++ b/drivers/vfio/pci/vfio_pci_rdwr.c @@ -42,10 +42,17 @@ int vfio_pci_core_iowrite##size(struct vfio_pci_core_device *vdev, \ bool test_mem, u##size val, void __iomem *io) \ { \ + int ret; \ + \ + ret =3D vfio_pci_core_access_begin(vdev); \ + if (ret) \ + return ret; \ + \ if (test_mem) { \ down_read(&vdev->memory_lock); \ if (!__vfio_pci_memory_enabled(vdev)) { \ up_read(&vdev->memory_lock); \ + vfio_pci_core_access_end(vdev); \ return -EIO; \ } \ } \ @@ -54,6 +61,7 @@ int vfio_pci_core_iowrite##size(struct vfio_pci_core_devi= ce *vdev, \ \ if (test_mem) \ up_read(&vdev->memory_lock); \ + vfio_pci_core_access_end(vdev); \ \ return 0; \ } \ @@ -68,10 +76,17 @@ VFIO_IOWRITE(64) int vfio_pci_core_ioread##size(struct vfio_pci_core_device *vdev, \ bool test_mem, u##size *val, void __iomem *io) \ { \ + int ret; \ + \ + ret =3D vfio_pci_core_access_begin(vdev); \ + if (ret) \ + return ret; \ + \ if (test_mem) { \ down_read(&vdev->memory_lock); \ if (!__vfio_pci_memory_enabled(vdev)) { \ up_read(&vdev->memory_lock); \ + vfio_pci_core_access_end(vdev); \ return -EIO; \ } \ } \ @@ -80,6 +95,7 @@ int vfio_pci_core_ioread##size(struct vfio_pci_core_devic= e *vdev, \ \ if (test_mem) \ up_read(&vdev->memory_lock); \ + vfio_pci_core_access_end(vdev); \ \ return 0; \ } \ @@ -198,12 +214,41 @@ ssize_t vfio_pci_core_do_io_rw(struct vfio_pci_core_d= evice *vdev, bool test_mem, } EXPORT_SYMBOL_GPL(vfio_pci_core_do_io_rw); =20 +/* + * Undo pci_map_rom(). The iounmap is always safe, but pci_disable_rom() i= s a + * config space write. If recovery has blocked access, do the iounmap now = and + * record the disable, for whichever of resume() or the reset tail unblocks + * access again. recovery_lock spans the decision and the record so recove= ry + * cannot complete in between. + */ +static void vfio_pci_unmap_rom(struct vfio_pci_core_device *vdev, + void __iomem *io) +{ + struct pci_dev *pdev =3D vdev->pdev; + + if (!vdev->pci_recovery_supported) { + pci_unmap_rom(pdev, io); + return; + } + + down_read(&vdev->recovery_lock); + if (vdev->pci_recovery_device_open && + !vdev->pci_recovery_access_blocked) { + pci_unmap_rom(pdev, io); + } else { + iounmap(io); + WRITE_ONCE(vdev->pci_recovery_rom_disable, true); + } + up_read(&vdev->recovery_lock); +} + ssize_t vfio_pci_bar_rw(struct vfio_pci_core_device *vdev, char __user *bu= f, size_t count, loff_t *ppos, bool iswrite) { struct pci_dev *pdev =3D vdev->pdev; loff_t pos =3D *ppos & VFIO_PCI_OFFSET_MASK; int bar =3D VFIO_PCI_OFFSET_TO_INDEX(*ppos); + int ret; size_t x_start =3D 0, x_end =3D 0; resource_size_t end; void __iomem *io; @@ -230,7 +275,11 @@ ssize_t vfio_pci_bar_rw(struct vfio_pci_core_device *v= dev, char __user *buf, * filling large ROM BARs much faster. */ if (pci_resource_start(pdev, bar)) { + ret =3D vfio_pci_core_access_begin(vdev); + if (ret) + return ret; io =3D pci_map_rom(pdev, &x_start); + vfio_pci_core_access_end(vdev); } else { io =3D ioremap(pdev->rom, pdev->romlen); x_start =3D pdev->romlen; @@ -269,7 +318,7 @@ ssize_t vfio_pci_bar_rw(struct vfio_pci_core_device *vd= ev, char __user *buf, =20 if (bar =3D=3D PCI_ROM_RESOURCE) { if (pci_resource_start(pdev, bar)) - pci_unmap_rom(pdev, io); + vfio_pci_unmap_rom(vdev, io); else iounmap(io); } --=20 2.43.0 From nobody Sat Sep 26 13:08:18 2026 Received: from PH0PR06CU001.outbound.protection.outlook.com (mail-westus3azon11011053.outbound.protection.outlook.com [40.107.208.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CB7C547A870; Tue, 1 Sep 2026 09:34:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.208.53 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788255273; cv=fail; b=P39FWiV6+WDZNXep1/9KfMvDzaSF+a8KPY81Xn2fLWgRKc/o7uCqhXBcL1kL23QA05iIXWNCEpmNvL0cZg02f2ZGiQ08LuG7cdo1ZYwFjmG22Ua08vwIuQnMFQL1FJv2wck6b9oOXt48+XMq4nA8X577dKB8annTC3BsQeiyFNs= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788255273; c=relaxed/simple; bh=LLSltA3mrYj9j1inrFwzTsGzs0szn7iFp2JCaGX7BD0=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=OAX3JXGVbwkV6GdbVaYWGlaOdcUUBP/zV/81D8J2jjIl46V+k0wRD0j/eij2COj6zo25Ovwlju+5HU3IA+JpyOA0VHjSZGOspwjZlvnynPJlhIV/VGXv1s4Bj7JcsWYNn+m2YS2rPfVenW9FXmyocGAMKILIAZ7a3qb1NyVLSKk= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=T4zfCtxf; arc=fail smtp.client-ip=40.107.208.53 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="T4zfCtxf" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=IHFDvETAV4KaoGi6CMDU9MoAvOd99oSrfT5z+eRjOdfLpeNLWdcE0itBQg0IYBJk8SSPoSiknCNttw8tarWkE1Kz62tKZoaZsC4RBW6BpZ/kdGV4eojiP3g5MuHbRYzPtmXCdtAW2WeC++P8p9GNu/9O7HEixx4iaopTPg7ZxJOSrKDVHruTmG7bcJm+XaAGlqJi17iBEluctMvlW/4Ynknoh1YKlP7N9kHNdx12EA3WedgqLHMP5DHYHyzOh/xmI8NNolDVU8KyMs1hJ2E9B7xgd+RhItDM0g2C2xwX5JSWawmpe99YOILBpWs+gWUlXYI9ECUrvMTdG+vAj509mg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ljUXET5aIR959352MZTAkYp1UriNAa0g2vt67TAKO2E=; b=B/+2NVIdPd/Ss88b11sPSW9rmwI4sIAr/wGEtSFKRanIfbNTVXWyEbGe4lVe2djdf/2/3XwwWdso9snWf17IBF59GF9Ko9uFLu0YuT+JnU1/G+ur92qNm+5nDdazusZCHEHcTc6QLby1vIRaPeV8Yz2GSzJpIgTZ8brv65u3qPJv04oCcqR+7l1tBzkUx1X6uAk2g8Gf5DFDBAC97dwJw0QdDwdHw3u7sZS9BTQ7FsGkmdNg1eTwiFqUf9+3jHx2Mv7sOcmxxO8CK8TlffqpktaGssWWB+GU3MYVd4tK2hHa0ak7Q0fhtj6TAYoumUCrF3a6alBRLVC/ayphFSfs0A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ljUXET5aIR959352MZTAkYp1UriNAa0g2vt67TAKO2E=; b=T4zfCtxf70wZEvl2kremgBDaS+Rv5sztUPMqjkpNdtx9UG3wxwLFVAQmx5nidPHHVxx51ue8uGkhBBkIs1oGtKHsQDwDLcghlUqCG1w1bEMr1jgIqgOsVrKRaUDpOe+opHck3OHGQfokNZjAz/zYtXcJ7BSFDyIX+ZQd46j9D1nIDmFu2pXtSdzVXxTiwT1ndx4B8ct/q4SQamhzsGXOTMHiTV5XIe1lqaahnT8GZLMQ8xviNsdtqtC4WZNH+FzqBgZJgUC5bT3evi9sDMqpoZUkz6jREnUnJBFklQzZAYX3zuRz1HZiAI679Hn+ZXtnk0F4rTzPCsLRjkV+PSx0Ig== Received: from CH0P220CA0028.NAMP220.PROD.OUTLOOK.COM (2603:10b6:610:ef::7) by CY5PR12MB6202.namprd12.prod.outlook.com (2603:10b6:930:25::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.382.10; Tue, 1 Sep 2026 09:34:06 +0000 Received: from CH2PEPF00000146.namprd02.prod.outlook.com (2603:10b6:610:ef:cafe::44) by CH0P220CA0028.outlook.office365.com (2603:10b6:610:ef::7) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.360.13 via Frontend Transport; Tue, 1 Sep 2026 09:34:06 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by CH2PEPF00000146.mail.protection.outlook.com (10.167.244.103) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.382.8 via Frontend Transport; Tue, 1 Sep 2026 09:34:05 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.46; Tue, 1 Sep 2026 02:33:47 -0700 Received: from NV-2Y5XW94.nvidia.com (10.126.230.37) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.46; Tue, 1 Sep 2026 02:33:44 -0700 From: Shameer Kolothum To: , , CC: , , , , , , , , Subject: [RFC PATCH 09/19] vfio/pci: Serialize interrupt operations with recovery Date: Tue, 1 Sep 2026 10:32:07 +0100 Message-ID: <20260901093217.8539-10-skolothumtho@nvidia.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260901093217.8539-1-skolothumtho@nvidia.com> References: <20260901093217.8539-1-skolothumtho@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: rnnvmail203.nvidia.com (10.129.68.9) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PEPF00000146:EE_|CY5PR12MB6202:EE_ X-MS-Office365-Filtering-Correlation-Id: 81c28ef2-4fbe-489d-844b-08df080c2aca X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|82310400026|23010399003|376014|36860700016|6133799003|22082099003|18002099003|5023799004|56012099006|10067099003|11063799006; X-Microsoft-Antispam-Message-Info: HbQJHp6fKNo2ZeG1KiFKjyaQu2IWgf5LcyGKrxcbO9mXgroQgwRs7+cpQZc9oMknbrUgUAwsBMynpfIQruuqijv7THhBgns5iIrlJsslL1+PuO8716rE7C6XIkwh6IZbmEyy2owdaU+tnivX3oVNYfhB6niHFTz87zfJvywXp3z4zqdx4rW8aRuRR9AL7Je7wy0jsDwIhDRQpshenmJ5h7eNuV0aM33s2QLLLfqeIVYA7bt5Tyl5jXznfWjSUFOxBNovh7xL5PFZEffpiwYbZDwKfDKpfTvcwKap4j+uDCzj+eIU0gvpJj35tXoU+dCOnz7WdU+Ux1gCBHUEmW9OeG8qaxUNqwS4z5+4MUv2qo+GxuhyEZJm8t3DLSSGR/YcfkOsrqfWAzC2VK6m9b/QsWNZppDRKz/DObAwnjgAgfIgAGMKf+z/128/A771KUCN9TWEZ4FMOOc8l8MmJo6DL+rDr34eTFBebsWFhmDMjAZeGF8JbBnT9z6iFa1Ht9Srwj0i9HKPA4nWVirykk5m4TE6nXda5Cit442fp3Q8ULwtUQ1+/5mvzabMP3Ia39QA7On/rL0NL0b+C4tPMeicU5mxNt87MYedQBKoIIygphKyQvKgH13ppDtSwE6xmeklyRWwl2O5ENAtr6MQ1gw/v8q4jnv59eFiBHeqNtEeA1cnV5/kvJvteXnejxNycbIt9oLHXFeNto4+JIqy68gZGw== X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(1800799024)(82310400026)(23010399003)(376014)(36860700016)(6133799003)(22082099003)(18002099003)(5023799004)(56012099006)(10067099003)(11063799006);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: BiYiVsXrubkednvnugmG9ZvwsbkXzW5o9feS4UeTamX+++3eqUcMKXDYPVcLWIxLcBa3k1GODq4vp90PbWy6q+zFBJdGZb1xep1DAPikCetra5Fe6+wlpcvibTrqOw+wECu/zrFcPC0524G13myXFNSPADXZzQS78R8eQv5vOdqTwpFYkyaMrkEeUz7Yrg8+9Vv5z6/yTFtvvU4Zwy+MO9hNsI3vj3/dmHHWqGQ6T9zFv38gGUkIySIwZHIaUf7Kqpv3Ka6m86C6ONDXfuvtj6OMFdd7r13lLXC2+8RRlsfneCLN4Q4f8C797oL4UZ8tsnyTn3zPJ14T84JcoviNq0wLMhSDGauYTXR99SfI4R/iyYnEHd1UTch0l8TnCeTt78FkCwFeaznFP+1AM/DPatLLkJT8KfduyekL7NqT860GEHLXDTuifnSlj9/Y1UVd X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Sep 2026 09:34:05.8924 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 81c28ef2-4fbe-489d-844b-08df080c2aca X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CH2PEPF00000146.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY5PR12MB6202 Content-Type: text/plain; charset="utf-8" Hold recovery_lock for reading around INTx, MSI and MSI-X capability queries and configuration changes. ERR and REQ are software-only indexes and stay available while recovery blocks device access. INTx is covered by the same test even though its count comes from the virtual config space, so that one rule applies to every index which can reach hardware. The test is on the index alone, so a blocked device also refuses the few requests on those indexes which would not have touched it: signalling an eventfd for test purposes, and adding or removing the virqfd behind INTx masking. Both return -EIO until access is unblocked, which for a non-fatal error is the time the host takes to log it. Reading the flags or the count of a request is not enough to tell whether it reaches the device, and refusing a few extra requests for the length of an error event is cheaper than getting that classification wrong. Copy the IRQ payload from userspace before taking recovery_lock. The copy can fault, and with userfaultfd the fault is serviced by userspace, so holding the lock across it would let a user stall error_detected() for as long as it likes. The count read and the interrupt operation each take the lock for themselves. Assisted-by: Claude:claude-opus-5 Signed-off-by: Shameer Kolothum --- drivers/vfio/pci/vfio_pci_core.c | 55 ++++++++++++++++++++++++++++++++ 1 file changed, 55 insertions(+) diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_c= ore.c index 0b1b2398dc88..876ff51d6987 100644 --- a/drivers/vfio/pci/vfio_pci_core.c +++ b/drivers/vfio/pci/vfio_pci_core.c @@ -1313,11 +1313,29 @@ int vfio_pci_ioctl_get_region_info(struct vfio_devi= ce *core_vdev, } EXPORT_SYMBOL_GPL(vfio_pci_ioctl_get_region_info); =20 +/* + * Which IRQ indexes can reach the device. ERR and REQ are software only. + * An index added later gets no access guard until it is listed here. + */ +static bool vfio_pci_irq_index_is_device(u32 index) +{ + switch (index) { + case VFIO_PCI_INTX_IRQ_INDEX: + case VFIO_PCI_MSI_IRQ_INDEX: + case VFIO_PCI_MSIX_IRQ_INDEX: + return true; + default: + return false; + } +} + static int vfio_pci_ioctl_get_irq_info(struct vfio_pci_core_device *vdev, struct vfio_irq_info __user *arg) { unsigned long minsz =3D offsetofend(struct vfio_irq_info, count); struct vfio_irq_info info; + bool device_irq; + int ret; =20 if (copy_from_user(&info, arg, minsz)) return -EFAULT; @@ -1336,7 +1354,15 @@ static int vfio_pci_ioctl_get_irq_info(struct vfio_p= ci_core_device *vdev, =20 info.flags =3D VFIO_IRQ_INFO_EVENTFD; =20 + device_irq =3D vfio_pci_irq_index_is_device(info.index); + if (device_irq) { + ret =3D vfio_pci_core_access_begin(vdev); + if (ret) + return ret; + } info.count =3D vfio_pci_get_irq_count(vdev, info.index); + if (device_irq) + vfio_pci_core_access_end(vdev); =20 if (info.index =3D=3D VFIO_PCI_INTX_IRQ_INDEX) info.flags |=3D @@ -1353,13 +1379,23 @@ static int vfio_pci_ioctl_set_irqs(struct vfio_pci_= core_device *vdev, unsigned long minsz =3D offsetofend(struct vfio_irq_set, count); struct vfio_irq_set hdr; u8 *data =3D NULL; + bool device_irq; int max, ret =3D 0; size_t data_size =3D 0; =20 if (copy_from_user(&hdr, arg, minsz)) return -EFAULT; =20 + device_irq =3D vfio_pci_irq_index_is_device(hdr.index); + if (device_irq) { + ret =3D vfio_pci_core_access_begin(vdev); + if (ret) + return ret; + } max =3D vfio_pci_get_irq_count(vdev, hdr.index); + /* Dropped for the user copy below, which can fault under userfaultfd. */ + if (device_irq) + vfio_pci_core_access_end(vdev); =20 ret =3D vfio_set_irqs_validate_and_prepare(&hdr, max, VFIO_PCI_NUM_IRQS, &data_size); @@ -1372,12 +1408,31 @@ static int vfio_pci_ioctl_set_irqs(struct vfio_pci_= core_device *vdev, return PTR_ERR(data); } =20 + /* + * Interrupt teardown reaches vfio_virqfd_disable(), which flushes the + * global virqfd cleanup workqueue, so recovery_lock is held here for + * as long as work queued by any vfio device takes. Shutdown work waits + * for its inject worker, and an ioeventfd inject takes that device's + * memory_lock, so the wait can last as long as a reset there. That is + * only a wait. Nothing on that workqueue takes recovery_lock, which is + * why the ioeventfd write path reads the recovery state without it. A + * callback there which used the vfio_pci_core_iowrite*() accessors + * would break that and deadlock against a queued writer. + */ + if (device_irq) { + ret =3D vfio_pci_core_access_begin(vdev); + if (ret) + goto out_free; + } mutex_lock(&vdev->igate); =20 ret =3D vfio_pci_set_irqs_ioctl(vdev, hdr.flags, hdr.index, hdr.start, hdr.count, data); =20 mutex_unlock(&vdev->igate); + if (device_irq) + vfio_pci_core_access_end(vdev); +out_free: kfree(data); =20 return ret; --=20 2.43.0 From nobody Sat Sep 26 13:08:18 2026 Received: from CH4PR04CU002.outbound.protection.outlook.com (mail-northcentralusazon11013062.outbound.protection.outlook.com [40.107.201.62]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 61450470EA8; Tue, 1 Sep 2026 09:34:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.201.62 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788255269; cv=fail; b=UmhirH6GwLgLsESER0SdECTEqqQyLxfk/TmkbtcWVefJ9Lo88tGdHB3y2Hfdt0g8hcjCaSB0JNY4r/soj0WmJGI2gTpSYhUou8oMrr384krdtjMduEcegHAIdEDGVFupq9+T+PhW3AB+n9GlmacK0UZkB4k9ODxnxvIJWmIo+8Q= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788255269; c=relaxed/simple; bh=Fym25nMiS4L9H6deI4pf6dmoUXP7D1xZZLTX4z0jKS0=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=uRHwwONpRihW1WaXZQT4K/DjJh3ae/bqv1zQleo/5aulqi7YiJvjjpPx7cwWhT30WNkYRU0Oc5gHexNoIedW8VOCYkIwQ/y+KZY7wccpqYSK0XIEZMcAkqXl1h08iyLrv/0dbI3rdUS2AMNBNFGI2Y1iVAIjQJASMNWD9PIICGI= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=NGxZrLiA; arc=fail smtp.client-ip=40.107.201.62 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="NGxZrLiA" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=hckE+BMuDIq/HbMHXfmvhPIrnVQ7ULX89ihK+VUK21nWAVcGCnxo1TF5hiFQPXHW5ECFtuIujfNlqNu/QaE1ZS4a26QMJ+FZT8nTGP1H44NkvEe5FijViHHNtwjKH1POAN4zS5Quk1AhvAW1pZf3fx132ruc7WWVDjivSPqro9z33JnxLv/QNjzvXey/gzRVsGbyCGzs4UnBKIBb4OUqYCQ4tyLaojKwismkPXA7At2q2RYmDoBjd7EQqM6hw3DqBxThyXsTcQ18ts0DxlcmVZeGpQ0wXszB8FnTF3YGQjYTd6UbX/jr8lf2+42Fz/f3NkFkrqvGrnz0EQpzmi8YtA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=CtSHOhkW2Qnii4iyhAaBDsc/xs0eHQyhWcLvveBNd0Y=; b=BXurCIKO+Mtob6oDURDzncoACZaYWb9crx08FC/vWsXkrXtMZpChxXQNyXYtn+9EhhLBg4Dub12m2B67pzyeivSeoFyfTByIyCeCMUZpVytwLeQdkngOS9nPJqoQkv1f66eLCzA87gA9EQ5HkxG0btcYme+Fa310CEHKSM+09hdHwm2wtQG1FvYQ4OXgj0BTriUHx7MQwyRiY9ql2z1NQeQl3EZb4ByOTzesjYH5HFqQRr3azNxce/vcETVm6BJMh+QauvDT3xiXQUk24HKjmnRRhHGo7kfOBbJ+qDWxUFvzhuiCRuo3Ff/EVbywFLO1q16VjptKYvNjx2jkukLwzQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=CtSHOhkW2Qnii4iyhAaBDsc/xs0eHQyhWcLvveBNd0Y=; b=NGxZrLiAcIffPnD5b0qcK3MpFEEPK7aIgay/65LjhH3DDXvQigOsq6ML5B1eh1MJamEn0YzL4nlT9Ylqp2O2AKQQ2vjmq8GE0NRxYG6FLkSVfR/sAS43kupxGZw8FHRBFzbGcQB6V22uSxbqCqpKat0HmpK0vC0EEqUtWXmGKGEvSbC5HHbjtZychFXNFoU8eTYQNMTb7rojueQd2I5U/sgnz3kYQF/nXdplDasdvxAhyIfHwv5D80yYGuGDe7JtJx8L8NtN/YXko9Eywn0+5i4H5qHRpdYN6gZ3rn1hH0sjW+I9VTmaLmfL4ABu6wwv6fy1YtEK+JbpdDJ53832Gg== Received: from CH2PR16CA0028.namprd16.prod.outlook.com (2603:10b6:610:50::38) by DS0PR12MB8788.namprd12.prod.outlook.com (2603:10b6:8:14f::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.360.13; Tue, 1 Sep 2026 09:34:10 +0000 Received: from CH2PEPF00000148.namprd02.prod.outlook.com (2603:10b6:610:50:cafe::2d) by CH2PR16CA0028.outlook.office365.com (2603:10b6:610:50::38) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.360.13 via Frontend Transport; Tue, 1 Sep 2026 09:34:09 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by CH2PEPF00000148.mail.protection.outlook.com (10.167.244.105) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.382.8 via Frontend Transport; Tue, 1 Sep 2026 09:34:09 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.46; Tue, 1 Sep 2026 02:33:50 -0700 Received: from NV-2Y5XW94.nvidia.com (10.126.230.37) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.46; Tue, 1 Sep 2026 02:33:48 -0700 From: Shameer Kolothum To: , , CC: , , , , , , , , Subject: [RFC PATCH 10/19] vfio/pci: Serialize hot reset with recovery Date: Tue, 1 Sep 2026 10:32:08 +0100 Message-ID: <20260901093217.8539-11-skolothumtho@nvidia.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260901093217.8539-1-skolothumtho@nvidia.com> References: <20260901093217.8539-1-skolothumtho@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: rnnvmail203.nvidia.com (10.129.68.9) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PEPF00000148:EE_|DS0PR12MB8788:EE_ X-MS-Office365-Filtering-Correlation-Id: 4ec9f7c3-d9f1-4ab0-a22f-08df080c2d26 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|36860700016|23010399003|82310400026|376014|1800799024|56012099006|10067099003|5023799004|11063799006|22082099003|18002099003; X-Microsoft-Antispam-Message-Info: orbt+P0kHBPFF7rfQprQEURb3/JgIqtuDTr6UUXvDmbCXYK1+wBFWrMbCSSTM0N4Iv6oXutFKD/PJ00PhcGmr0Itenv++egiOIlIAHdB3fyIFLpX4a25dSkMDMMHQe2r/Rn1VxAdRq8vnB/ze1XJfhZHY84hn6ecafo8hidVt+bkpIOAedsdN/8QVqWG1GmKuRX48w+DAmBlpr/KB6CzfxZxdrN9RumCf9msqVxesiS9PSye5u5Lp2zdc0SkNVvckpv5shYZ8uVl8EfOA9RGAqJOLPV7K4ZWFGmrdCkQGBVk9zW41rFw+svmp3mY1ptYtp6WCYPDUoJIMOgiMw9fztdNXa8ovQhAr1BLTUiRfVvQ2wAq1yig8BgWWuveO80Rx9HpzifsAwg1rklHIHGWXfsmMHASdMmybsIliV3IKROzFnELUXTvu4Z1mYBPw6girnxxbZIXPIdUVX7bSypCzbGZTMDuFN/IBmIljzm+IsOD+9NjzX+HCw9pJJJclQ1qLAGeA6008xfJXZamTjAcpyhbGAmsEikHkSVIHkHXtx45uEZVpzbpsOGcjVkuBYJ1tpKxPTmqWNsBKUnvhFJ5wI0E8JJWyW1BHzuhMsIxAXPIVUcEbrOb45xv4k3aEtUc3YDjJCL8J0wq7jQX1QBJ5ZPSLg2BnlFG5NureN1Kn+tLyF6usMB4nw3Bq1BF1dbEaS3/d0seupCbqe3a5S8F4g== X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(36860700016)(23010399003)(82310400026)(376014)(1800799024)(56012099006)(10067099003)(5023799004)(11063799006)(22082099003)(18002099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: IHMazd3qewQKMo7+xkLEjfpOeA03XMQ8Xw18c+99oRd7tz5GjcA8yB3e6FnmU7HxPbnDuJgXJk2SLiv8bxGaLDvq5zUiIi7R59hX3d06C00hQ2yhVpkAVnqddRsbbDoTM3wi35DBPKu7l+7vAP/W3i3hUgJ0bR+tY6JlLSFXCjpCLcU2REiSH1otDTbhXYxR48lXTrf0btMH5rhq2nd5Ejy2XwnQDyyLKUwj6Bq1XyNxuIZOAojp4RAuZig8pSdmkRVDIBBL8dGxvnLd1hn4aoquaDSdMHkQXwVjM49+zWmPqxEpt6b/ZZRs2LliQXwYKNPg5DEIxW791yN//fMImymnKdqO13ANV9zYO3j+BA03mSakppEr1LHGg3ZS13oj2EjLMgkgs8pT1ffERnu6gKLz1l2qvmP1qaLBRK0gWu45d4EzBQhAe29k4a+t8iiv X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Sep 2026 09:34:09.8434 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 4ec9f7c3-d9f1-4ab0-a22f-08df080c2d26 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CH2PEPF00000148.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR12MB8788 Content-Type: text/plain; charset="utf-8" Refuse a user hot reset while a recovery transaction blocks access to any device in the set. Between the AER callbacks memory_lock is not held, so access_blocked is the only thing marking the device as unavailable. A device which has failed for good is let through. Nothing is running on its behalf, and a hot reset covers the whole set, so refusing there would stop a healthy sibling from being reset because an unrelated device errored. The failed device stays blocked either way, since the reset does not clear access_blocked once FAILED is set. Do not hold recovery_lock while resetting. pci_reset_bus() reaches pci_bridge_wait_for_secondary_bus(), which takes pci_bus_sem, and AER already holds pci_bus_sem when it enters the driver and takes recovery_lock. Holding recovery_lock across the reset would invert that order for every device in the set. Assisted-by: Claude:claude-opus-5 Signed-off-by: Shameer Kolothum --- drivers/vfio/pci/vfio_pci_core.c | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_c= ore.c index 876ff51d6987..bd3d79d28f27 100644 --- a/drivers/vfio/pci/vfio_pci_core.c +++ b/drivers/vfio/pci/vfio_pci_core.c @@ -2937,6 +2937,27 @@ static int vfio_pci_dev_set_hot_reset(struct vfio_de= vice_set *dev_set, break; } =20 + /* + * Between the AER callbacks memory_lock is not held, so refuse + * the reset on access_blocked as well. Read it rather than + * take recovery_lock, which would have to be released before + * pci_reset_bus() anyway since that reaches pci_bus_sem. + * + * Let a device which has failed for good through. Nothing is + * running on its behalf, and this reset covers the whole set, + * so refusing there would stop a healthy sibling from being + * reset because an unrelated device errored. The failed one + * stays blocked, since the reset does not unblock a device + * with FAILED set. + */ + if (vdev->pci_recovery_supported && + READ_ONCE(vdev->pci_recovery_access_blocked) && + !(READ_ONCE(vdev->pci_recovery_flags) & + VFIO_PCI_RECOVERY_FAILED)) { + ret =3D -EBUSY; + break; + } + /* * Take the memory write lock for each device and zap BAR * mappings to prevent the user accessing the device while in --=20 2.43.0 From nobody Sat Sep 26 13:08:18 2026 Received: from BL2PR02CU003.outbound.protection.outlook.com (mail-eastusazon11011039.outbound.protection.outlook.com [52.101.52.39]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6ADA547DFA9; Tue, 1 Sep 2026 09:34:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.52.39 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788255274; cv=fail; b=jTfeYZzcqLy7fh+p6ZJMEn6BGAHbPTWOUSHCGTarQiU17T11x2DmCSooqYlEbs/qqN5jLRh1xMWSytY+oJwDA12hAlek3YeVK7XHRNeAIdr6xtsOMt95XGSQ4OgE852DmLU79AN7vc8VVFvGGjdA2fZds1xnIj0pTLN29WZltfk= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788255274; c=relaxed/simple; bh=Ehn62oPGFdYlZiFCt7Po+Ep+1T4oBtjL3w4xuf1/8Sg=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=lanKiCD9slLJe69yBeKV0TTHXRVRJg6ciNgfON4wig3x+OWsjtut6x3ANN/bthkj8qaVUYQ44S3lTI26EnzWJ7aHXYWx0Nkot6aHvzVXZCEoQeN8ouzNJYvIyk7JiGWlIV57XYPiaiWRNz7up9l1vVsmn/oWvTdYXLz4QXemzcE= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=IzV2viWT; arc=fail smtp.client-ip=52.101.52.39 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="IzV2viWT" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=jj+pn8+0REyG/1lB/OrSKDZEvgSO5gup6DaQJUaWmnTmio3YwUf0Iqj3evizlg88z1LL9+KKeTJ3kMf8IcqbzbJ/65YbcZmle8gRIhid2iGC53tJI5WsBKlx6TYMXLvnSEibiZIZJR+6hsi/gVDWQXpdLtuPR2LDuuRmXZgJ37bs8eJMEUqhJ61b0direLRwuRj8A2uHHkKOZM0/Kcfq8lWBNBfFGC3QkFLERNJvl6+bPNzzA+t1VsuFy09oBz7gFwtbi3bF/bPW9/mXb1gi1MngCPrgMVfgnFdKwCWX7lbZUSKe7PLvFh5Bqv9UJcR5CsUYigtc6o5IrhRwyCTQFA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Ph6bobwgGnyUGJ3SMPMALaWaZHE0wJfF3YR+05aLV70=; b=gvAEveAMF17Zs/Aq3ukAo8SIxbTsesMO48DRc/CSEeHmpzsliJ2HtRCZCw+n+UGGCaAUCA2AoPe9/OZKmSbfmnA9Iu6DPfcPnFsl4b06yD6TGFPhuJFaCOV3Nw7g1dd6BBjawjXoC/D0XuVa8IBW45BYNb28AGeod0z4i2uLLeDXPuh73/G7zvPOMPGxYBUo3cAxwPd6b7revpKloiBilqj5Lh92RojuTFgKU4odQ2YUgRGqpb10sVpzD70B8zQh/Urw+sdshJideXOF6HxfS+8U/pqX2k5ymTp+qXsFd0QwF14j43/0V0kYkOK2YiCOHYvoyHCNY26NAS+tcTQqkA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Ph6bobwgGnyUGJ3SMPMALaWaZHE0wJfF3YR+05aLV70=; b=IzV2viWTKU7lA66uHKYbplncseVwdBTYCipUOEfSa1N/ZMwJ5AxZSl+tbyaK2GDRZqyOQzanommEHWEUzqBdYJGM2ghOEv7vtd0zKzqBoGtWkSC1cBTD3C2wONUyAbgT/EWEsNPq78CNJNtxkIerjBUcC5t00kB5oCZ//vKaueJwlzhDEsTnUnKHS5p7N+j8UolAzbYNZ0sW9oWErRLXXPxEZhCnbt+bcK4PHLMcmxzX/ee5KRhYG1zaXAzqBKrpIWDWIjU3FxWOaV4frbkEvmcJjfscJ4+kkpSg8ZHq1/jhtwaZrG2MRqDL2fnyyOVpakkAKs3b8gGfrHttappQCg== Received: from CH0P221CA0004.NAMP221.PROD.OUTLOOK.COM (2603:10b6:610:11c::14) by PH7PR12MB6953.namprd12.prod.outlook.com (2603:10b6:510:1ac::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.360.7; Tue, 1 Sep 2026 09:34:13 +0000 Received: from CH2PEPF0000014A.namprd02.prod.outlook.com (2603:10b6:610:11c:cafe::72) by CH0P221CA0004.outlook.office365.com (2603:10b6:610:11c::14) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.382.10 via Frontend Transport; Tue, 1 Sep 2026 09:34:12 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by CH2PEPF0000014A.mail.protection.outlook.com (10.167.244.107) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.382.8 via Frontend Transport; Tue, 1 Sep 2026 09:34:12 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.46; Tue, 1 Sep 2026 02:33:54 -0700 Received: from NV-2Y5XW94.nvidia.com (10.126.230.37) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.46; Tue, 1 Sep 2026 02:33:51 -0700 From: Shameer Kolothum To: , , CC: , , , , , , , , Subject: [RFC PATCH 11/19] vfio/pci: Serialize runtime PM with recovery Date: Tue, 1 Sep 2026 10:32:09 +0100 Message-ID: <20260901093217.8539-12-skolothumtho@nvidia.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260901093217.8539-1-skolothumtho@nvidia.com> References: <20260901093217.8539-1-skolothumtho@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: rnnvmail203.nvidia.com (10.129.68.9) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PEPF0000014A:EE_|PH7PR12MB6953:EE_ X-MS-Office365-Filtering-Correlation-Id: 624247d2-456e-40bb-1e7c-08df080c2ee7 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|36860700016|82310400026|23010399003|376014|1800799024|10067099003|56012099006|11063799006|18002099003|22082099003; X-Microsoft-Antispam-Message-Info: hKa8r4ypx7CDtipS69K/RVcsdvw1h5faagM5oFUn9TrCh2m6eWkHqYehb9oXQR3MEuG3ez3nnKLAqouLn6Cqs12BnyykdomCGSG9y87kHWESUg2OQuNIlSj8wcW8EPP/fVlb5e8yCDXv9iE8HLBRXHgotwpzWWci2GvZKq13f5Aj9t9QSi5kNc+azfItr/iyJYZOZEY2+RI/2/7Se2V1j7nS3A7ONFHqhbnnEPJ2mhrSH5ZPd4kR/1mzCfQ6olhQ/BzZgzXfSfw1in7bMwDeqUDygfxj3Jp+F9SohSjLobVV4dMgFbaMHj5rGh74mP/uNOqCc08M5j1UCoSulgsxwNNb3X2A+3zNgWGxLLh0fTvPNKFumDdERs/ocJpvEA2wDqkY9ewg9OO6K1aBhdwMwR3ROQTS/xxooPxksh1MAGLQC/Sgd99pQiaPMa+VkzFjQOVTaVAWYgyUA5VIL/PueCid9WZs9gaswcWvyB8azjmCvAVVv91miON+o+ogyHYIEFLfz+jZ+5YwAY5Ksar0u8fB2YMlZmYMG0wVk+7ppHtam+h5pblafQH4ayzWpph7atphh46ztsFD0HEA1J0WVo4CpveeMnfde8Dclj1CZz+tlS3tnqgy+UAK8hu8JSNxiaN+RLMrGPnUvQTLf7LvgiDBwNylF44qitWVULvqgGqeC4ygEyhQ8ErdcUdU8DQux1J6WPKqph/JOE7/6DnJXg== X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(36860700016)(82310400026)(23010399003)(376014)(1800799024)(10067099003)(56012099006)(11063799006)(18002099003)(22082099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: RnLlEA2TDt07JE7/HNjcalH9w9YiID4IYzw+GGA24cxkrxfhpShy0iSKbodSGpqL/EWorErMYzFbnsZghGngbQP740XEsVDorvqHlRbcaGJyNb+ufZfMppj0vTti8iSnkDuCITyi/nUSjJt1ejGyRufYnSPV87mBLyojMOUYwWGXc0OeXQWS1R2AsdRMsLLJsQcdKTTdt/cRns6YvtZIEE9EKu/AWPvXhaUQ87PMbfh0yW1/LCpLYoL9yF0l6un/SRHsxJvIxEIfnwsNh8TdruB4WRLRyYX//+h4Y1tygyYQ6n4wEEceh0GV1dEP2cIkD/9cnR1B29DCR3jMw9E8lpHzweJUEN/9Hly0y+5mjBzxSTm8KqybY88TRJM1e06/NfzBjg4bLGtCGAsL+bd5t3Q0iLLorqo9+PfLRc1nZ9hUuZZdzH4AocIqHxVNS/I8 X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Sep 2026 09:34:12.7881 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 624247d2-456e-40bb-1e7c-08df080c2ee7 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CH2PEPF0000014A.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB6953 Content-Type: text/plain; charset="utf-8" Hold recovery_lock for reading around low-power entry and exit. Entry zaps the BAR mappings and revokes the DMA-BUF exports under memory_lock, and exit restores the exports. Taking recovery_lock first keeps the same order the AER callbacks use. Neither wakes the device. Entry only decrements the runtime PM usage count, and the suspend which follows runs when the vfio core drops its own reference after the ioctl returns, outside the lock. Exit takes a reference without resuming. So neither reaches pci_bus_sem while recovery_lock is held. Check the recovery state before the runtime resume in the region read and write path, but do not hold recovery_lock across it. A resume takes pci_bus_sem, through pcie_aspm_pm_state_change() and, from D3cold, through pci_bridge_wait_for_secondary_bus(), and the error callbacks take recovery_lock from under it. The check is best effort. It avoids waking a device whose access is already blocked, and the region access which follows takes recovery_lock for itself. A recovery which starts after the check is not excluded, and does not need to be. pcie_do_recovery() runtime resumes every device under the bridge and holds the reference until it finishes, so a resume which runs alongside it does no more than take a reference of its own. Assisted-by: Claude:claude-opus-5 Signed-off-by: Shameer Kolothum --- drivers/vfio/pci/vfio_pci_core.c | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+) diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_c= ore.c index bd3d79d28f27..95884e713a4b 100644 --- a/drivers/vfio/pci/vfio_pci_core.c +++ b/drivers/vfio/pci/vfio_pci_core.c @@ -372,15 +372,21 @@ int vfio_pci_set_power_state(struct vfio_pci_core_dev= ice *vdev, pci_power_t stat static int vfio_pci_runtime_pm_entry(struct vfio_pci_core_device *vdev, struct eventfd_ctx *efdctx) { + int ret; + /* * The vdev power related flags are protected with 'memory_lock' * semaphore. */ + ret =3D vfio_pci_core_access_begin(vdev); + if (ret) + return ret; vfio_pci_zap_and_down_write_memory_lock(vdev); vfio_pci_dma_buf_move(vdev, true); =20 if (vdev->pm_runtime_engaged) { up_write(&vdev->memory_lock); + vfio_pci_core_access_end(vdev); return -EINVAL; } =20 @@ -388,6 +394,7 @@ static int vfio_pci_runtime_pm_entry(struct vfio_pci_co= re_device *vdev, vdev->pm_wake_eventfd_ctx =3D efdctx; pm_runtime_put_noidle(&vdev->pdev->dev); up_write(&vdev->memory_lock); + vfio_pci_core_access_end(vdev); =20 return 0; } @@ -483,7 +490,11 @@ static int vfio_pci_core_pm_exit(struct vfio_pci_core_= device *vdev, u32 flags, * already signaled the eventfd and exited low power mode itself. * pm_runtime_engaged protects the redundant call here. */ + ret =3D vfio_pci_core_access_begin(vdev); + if (ret) + return ret; vfio_pci_runtime_pm_exit(vdev); + vfio_pci_core_access_end(vdev); return 0; } =20 @@ -1867,6 +1878,24 @@ static ssize_t vfio_pci_rw(struct vfio_pci_core_devi= ce *vdev, char __user *buf, if (index >=3D VFIO_PCI_NUM_REGIONS + vdev->num_regions) return -EINVAL; =20 + ret =3D vfio_pci_core_access_begin(vdev); + if (ret) + return ret; + vfio_pci_core_access_end(vdev); + + /* + * Resume with the guard dropped. A resume takes pci_bus_sem, through + * pcie_aspm_pm_state_change() and, from D3cold, through + * pci_bridge_wait_for_secondary_bus(). The error callbacks take + * recovery_lock from under pci_bus_sem, so holding it here would + * invert the order. + * + * The check above only avoids waking a device whose access is already + * blocked. A recovery which starts in between is not excluded, and + * does not need to be. pcie_do_recovery() has already resumed every + * device under the bridge and holds the reference until it finishes. + * The region access below takes the guard for itself. + */ ret =3D pm_runtime_resume_and_get(&vdev->pdev->dev); if (ret) { pci_info_ratelimited(vdev->pdev, "runtime resume failed %d\n", --=20 2.43.0 From nobody Sat Sep 26 13:08:18 2026 Received: from CH1PR05CU001.outbound.protection.outlook.com (mail-northcentralusazon11010070.outbound.protection.outlook.com [52.101.193.70]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EFDF347B423; Tue, 1 Sep 2026 09:34:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.193.70 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788255275; cv=fail; b=vBY6AGsSA1wynMBiqLv5GIpVu+hoQO/1Cy0KhjXgoaSEIqKHmaIvpeEJaO5tW9n6iqePf8NQ1/xiKLLEq8cf3lKWljfi3UYI1SkMTWs2xgefpgScoh1BIlEDvDktaLFyENIR4LZBrnTJxW4FYIkdRUiWmsabhidSklQO7DueyRs= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788255275; c=relaxed/simple; bh=+4M+cqaBshtoDg7+6SccNvipDSq9Y5Jbk94BQxPWAj4=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=vAgB9DvECAUYqiyqEEIvVC0sAZO7lupl78GdMCfW8/gL9CxmDdXTxGcmmWmseFRKSKtTiClvJLlpCrjugTUCH8Rr+0MXQ0oHagRL0EWBGut8eJVhsgvynmT3bu+DzM9ZRNho6TYBabJBRI9Imha1dFLRz//1bN7sgx9NcO5gJ9Q= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=S+Ae3PGQ; arc=fail smtp.client-ip=52.101.193.70 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="S+Ae3PGQ" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=By0+vOqzNDf1I1O9sc85HzTiFDtxtbRd7FfG4HkJnzJiXY83wyi1QZQ8ODIu2pNmR30tU0jvbyosNkv96eqo/Ln6gOUtdJVmuR+mmGk36MmMjNBAXdu2O93oGPj5g9q5pHPHqntW/0oOaO7Nyb4P2vLqIQuDxxQAwiR/motzuiHEF05M0Xslrdu2n6j1uVu6xC13yMzO+WpUbBSLsJnoKlxQKlIiAPd4FmXzE1fWg6CvrPmTUoQE2HAhRMgcye+5COC05e12vEbGm3k2Gw0AntCSxNs5bO8FUAzaW/ALm5a7/M1+tppr1MGVd3ZlvF+z8ArmtCY+MqCk8WotI7ZTPA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=/En5j1Cx6pO5WXnKyFfWs3D9RfkR6Nnr0FP1/zV7oz4=; b=bPMEmGZcQ4aKh9HRnxZeIkliUAEbIRfKBzAVRtMmJq3YGe1wG4Ahg4vOTwZmFrBgtpOOxribzrH4B5DUBt6YPrztV+cn10qHTeHjP92WkdAba6Pc+S/2LWYpvmxSeN5rq3RpEY8c0Xh/k4Tk/R+CIH33VcnJksRPr1P5CjSYgjUdv5IS6diiLWSJllwQc1ZH4Fj114M6ZUlwLsdgionWk38qUYHVPyO4iVF91iLOSvZEK4wmcjzH98T+c8MzrEmN4aSb2PayxMvImWCwE+06707pD29j0QFiinTisCQjBd9bsqcZw25r2cjO0mr98gOFsxf3l/mnO4HBiphbAxQfCQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=/En5j1Cx6pO5WXnKyFfWs3D9RfkR6Nnr0FP1/zV7oz4=; b=S+Ae3PGQi6qTSXaYCOLJSRjIiEaZrIonfvlLPcCUjR8lAV0g5A6H8v/dEP8NBQlK1bzlcj1/2Mj+wacMfVGeceh6eS6KOjrontvId9njfvWnhSo3jDR8rotI54plaL4Sn/zWsW5gP4Zbuvf17NB8MCq35g4qr5IqlUsGtGxI1voJDArcvh3ejMP2m+dFGIvzOUO8OLYhAukpnAayyDOlfkC7eyN0iJfsaqBis672I0Jv5Xtoi+6mFXrhxbgMPW6JIY7rhoFxawo24h9+kDJrYV8ui1LKK6w1xQj3xXeaJ8J4kTTeP3dOS+Rh4NBLGe32Wxs1NZbHnIiqvQadVBVKug== Received: from BN0PR04CA0081.namprd04.prod.outlook.com (2603:10b6:408:ea::26) by IA0PR12MB7553.namprd12.prod.outlook.com (2603:10b6:208:43f::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.360.10; Tue, 1 Sep 2026 09:34:16 +0000 Received: from BL02EPF00021F6A.namprd02.prod.outlook.com (2603:10b6:408:ea:cafe::a7) by BN0PR04CA0081.outlook.office365.com (2603:10b6:408:ea::26) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.360.13 via Frontend Transport; Tue, 1 Sep 2026 09:34:16 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by BL02EPF00021F6A.mail.protection.outlook.com (10.167.249.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.382.8 via Frontend Transport; Tue, 1 Sep 2026 09:34:15 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.46; Tue, 1 Sep 2026 02:33:58 -0700 Received: from NV-2Y5XW94.nvidia.com (10.126.230.37) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.46; Tue, 1 Sep 2026 02:33:55 -0700 From: Shameer Kolothum To: , , CC: , , , , , , , , Subject: [RFC PATCH 12/19] vfio/pci: Serialize physical device information queries with recovery Date: Tue, 1 Sep 2026 10:32:10 +0100 Message-ID: <20260901093217.8539-13-skolothumtho@nvidia.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260901093217.8539-1-skolothumtho@nvidia.com> References: <20260901093217.8539-1-skolothumtho@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: rnnvmail203.nvidia.com (10.129.68.9) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL02EPF00021F6A:EE_|IA0PR12MB7553:EE_ X-MS-Office365-Filtering-Correlation-Id: 5b6d9d36-a143-44ae-36e2-08df080c30ca X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|36860700016|376014|82310400026|23010399003|10067099003|56012099006|11063799006|18002099003|22082099003; X-Microsoft-Antispam-Message-Info: OBKHAXvGtIiXx5rVMdbAbrw4bjeKiPr076y4xkFfMlkJwhq2XUShzbMW1ddF1Y/UY7qp2qG4kuHRckDcgr8t1y4NEZZZm1Cpz6o4CH/fRLChDf7uTS6bk90UQkclX7S9SGgb/V+mfgNIrJign5Ff0SGE8WRIeSxCLF3cl31WY7E8nLe49u448alz1C/A6YVh+DYRlZZGJ29wsmmoa4d/Hpz7WFtF0lnT8bUZl/t6OmUjH8ryCyIMGV3pHOgFtrA6H8SwLU5U75ZlMQihnEoWH/BAi1vsCrNTnuAiG7pOdO9GyXiFJzs5cqZ8Dke6hCkmsnuGvW05w4psuYdnbfKoSkYJoS5RMTuYWCsfS58ijDJ5E2aFSNwHz3EWFjdKBRh68Q47waf0YGlz99l2x/99TosbgrLEtOwMw1znWlDRukapCY+B2VNRmZAeWzROtNtdSFioLL8/i2hnc46ufPNiXqIPfNtansxk/zETjP9q0DRA2HlxOVBXhwFzmqphubAxfWbm7YW2UyBvVILDTd7uMQVTC8YuTn/OCMrTa9r0LdDLY4t9MkKCMGCpfIAHv5/fbGjh2F+jHQ/xxaKnUSJUh0Luo+LSCWXq1lPUTLd6dkMUgRIaYhZ+gXx01a4stdPR7ecvr65HcTj/1KH7WWqmQnvmyzZf1Ozx+smce1jbMLRnXP4p7WXNY/Gy/fre+ziuaUuNZouDhDfPIseTStnyYA== X-Forefront-Antispam-Report: CIP:216.228.117.160;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge1.nvidia.com;CAT:NONE;SFS:(13230040)(1800799024)(36860700016)(376014)(82310400026)(23010399003)(10067099003)(56012099006)(11063799006)(18002099003)(22082099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: v+yLINiczsiOF6uinXvC8H9CvXXdTtSPKuRMy97PqkAAmfun769TO42fdN5flKzkhNDhHeqKQQnb9WMokwoOwm81bmhr4y5GvaElpdqVtRuKFFdXt1l9+5gqy0PmZQurm6R0hE78dS6dvs2YBs70KJ7/WFRK//Iii84WRT8W+O5+XBR+i+ch7bgissILX5NHQ2S3RDSebZNkB/6Dqr0RaWLQTmlF0zzUxmaW73u+IEhnjakFpArnzY9lLyyuQXLS+obCV3Jka8CTeyLNXdnT1QXTBe3xpVMW8Px2dezKRmA9LstN/VWzo9yOuZQantXvEP5BbaCIdQaUThHd94r8HPMspNwSSMUzfH1AdaL3AEk7+8SHhWwwHPc9gG+SQD/qBX3ilSE5cBDb9719DPZXmDzLFloWi04XLav7tDUx86VnNtFosBRgpKARfYCYOlXM X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Sep 2026 09:34:15.9285 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 5b6d9d36-a143-44ae-36e2-08df080c30ca X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BL02EPF00021F6A.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA0PR12MB7553 Content-Type: text/plain; charset="utf-8" Hold recovery_lock for reading around AtomicOps capability discovery, which reads config space to work out what the device supports. The rest of what VFIO_DEVICE_GET_INFO and VFIO_DEVICE_GET_REGION_INFO report is cached or software only, and remains available without taking recovery_lock. Assisted-by: Claude:claude-opus-5 Signed-off-by: Shameer Kolothum --- drivers/vfio/pci/vfio_pci_core.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_c= ore.c index 95884e713a4b..4447967413e7 100644 --- a/drivers/vfio/pci/vfio_pci_core.c +++ b/drivers/vfio/pci/vfio_pci_core.c @@ -1179,7 +1179,13 @@ static int vfio_pci_ioctl_get_info(struct vfio_pci_c= ore_device *vdev, return ret; } =20 + ret =3D vfio_pci_core_access_begin(vdev); + if (ret) { + kfree(caps.buf); + return ret; + } ret =3D vfio_pci_info_atomic_cap(vdev, &caps); + vfio_pci_core_access_end(vdev); if (ret && ret !=3D -ENODEV) { pci_warn(vdev->pdev, "Failed to setup AtomicOps info capability\n"); --=20 2.43.0 From nobody Sat Sep 26 13:08:18 2026 Received: from CY3PR05CU001.outbound.protection.outlook.com (mail-westcentralusazon11013035.outbound.protection.outlook.com [40.93.201.35]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9A5DB47A880; Tue, 1 Sep 2026 09:34:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.93.201.35 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788255292; cv=fail; b=bmyZZyzqFgyCtuwAn/4rMdwb2GBHBBFpr+lMwfottdapVvv+ZMrNfuPce9/ASJtKmwFbxPPsX59mrwl8JWrNN8ZXy2uDZCe/56Y9urv0SMFwKpE8DibTRuIHHnposuGONM/zyv8SKH4zrm5NxTyFYw5zkC24LTZoJUIT1rgarzo= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788255292; c=relaxed/simple; bh=6/0+rIF18R2hHwHIk3wR7v1uy6TyvcTkvPikNA/fLE4=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=Q9+ZDdKDwcD3bqaHZ+nYdsjAGE3bVfRN7dSZ+oXqMPIkXNT8qGv8JI/DtpvywVkByj8Ez1y1b8N/zJvxa3YGWtcyYKL6e1TX4+zWO5E6GT6X6Q6DtmfJuzcjVvGeL79DA6170hqu+2ezBpLr3sMBv19gchMXRoqcBeudy7Mpp8Q= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=oEjQZGJ7; arc=fail smtp.client-ip=40.93.201.35 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="oEjQZGJ7" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=uijsJ0nYeNUEd424/lkq9Qon64pJate8LC3RhyVQaWEhxMJyOvF8Lvj5wEppNF+ZF8xIiWd+SjjHJzHUSi1qPCgWxQOn57BauHjnHKJkvO2wd699yNxW9g2/U8KFzJUtbHsneTtvyqWGaAvLwW6BbeHXsRsyxKOfP7BtTyHsQtmCP2XQo+4AWU5HTwdAHsm2FHzD07TeWsGMOOG1yArp3xROg+Fg4aofqDDHy4HsB1R/xlfHoECFfyB3ZOVaXW+g0oBCJ5MTWOOzdQFlrHRiqT1PlSJsM+QnJUoJ+Dmt+KmfzVMeJJO+OHarbhePMktGLy/nvdpvaKGDxbO5Z8gaIQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ds/ULF2lyfwVZgxd1u+g9e/hq6Ves5ljXltO51Craio=; b=yO1V3lXbC+iKpVuB9M9DXxj8943kLIt3TpshnJvAvXmpaDihx4VzIamhiquR66pOK+AS47cYf0PMaHE3NdN3q4f2m2+inIK0DTfNSqItzYEO3biQCTo5HSXWkvTYjMF9Zd9HsDEDwjcunXZcnkyFBgzEafEDeA/ZgJqHY6VCg9frIrDRrsr9pkl6m1TkfZYaHG1wBoMmFjIoTz980WPURbyyNyLyGqoIbsF/g3wK/uux92OcLlxF2dxjUtOm7aG1mfM49ljHhlde+cVGfMwWJt98FNgNiUen2y0xNsWmTg9a2RlsFVT38YMVdkMrV1ujnc5bJ54hNWAdHbnaqCv+Lg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ds/ULF2lyfwVZgxd1u+g9e/hq6Ves5ljXltO51Craio=; b=oEjQZGJ7ywPTApJniEFr3ewmm6YUFS9nBhIIyBuT9xpcM8M6V3Mn3vNP6Homnt1+87NOC9544zsEu7sJ3pw8hWnNEKWfQe5/YUftpPXu+bgGr5GGKvoxPI6PkftJKPSvvNsKYlHdQo0bmDyBJ2Pmgv8YPxlF3BTXLDz+oYG1ZoW9KjaEAiZwU27C8t5pxpXzD2IOZoHqVIRRN4EwrxizrTTr38YzE3z2dGUrSckzdtcPTQtg1KK7Uf3GN5ZyC1VjZzhjvG+dDy3xu83rusLPXMWaPMXftzmpnS1g92NKOjnbzSnWL/UUG/6eCMB5zb809qnDCBk6H2kdLvLbqFz5hg== Received: from CH0P220CA0014.NAMP220.PROD.OUTLOOK.COM (2603:10b6:610:ef::29) by SN7PR12MB7129.namprd12.prod.outlook.com (2603:10b6:806:2a1::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.360.13; Tue, 1 Sep 2026 09:34:22 +0000 Received: from CH2PEPF00000146.namprd02.prod.outlook.com (2603:10b6:610:ef:cafe::21) by CH0P220CA0014.outlook.office365.com (2603:10b6:610:ef::29) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.360.13 via Frontend Transport; Tue, 1 Sep 2026 09:34:22 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by CH2PEPF00000146.mail.protection.outlook.com (10.167.244.103) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.382.8 via Frontend Transport; Tue, 1 Sep 2026 09:34:22 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.46; Tue, 1 Sep 2026 02:34:01 -0700 Received: from NV-2Y5XW94.nvidia.com (10.126.230.37) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.46; Tue, 1 Sep 2026 02:33:59 -0700 From: Shameer Kolothum To: , , CC: , , , , , , , , Subject: [RFC PATCH 13/19] vfio/pci: Serialize DMA-BUF export with recovery Date: Tue, 1 Sep 2026 10:32:11 +0100 Message-ID: <20260901093217.8539-14-skolothumtho@nvidia.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260901093217.8539-1-skolothumtho@nvidia.com> References: <20260901093217.8539-1-skolothumtho@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: rnnvmail203.nvidia.com (10.129.68.9) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PEPF00000146:EE_|SN7PR12MB7129:EE_ X-MS-Office365-Filtering-Correlation-Id: 72599f74-b2ee-4487-9afd-08df080c34ab X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|376014|23010399003|82310400026|36860700016|56012099006|10067099003|11063799006|18002099003|22082099003; X-Microsoft-Antispam-Message-Info: I5qF/7eM1O9LeEWeATwNXYEhl7jqWBKsI9irXjybUogA91EtPtzkP130UbqhQiikPKBefM99ZXCrFCX/6QciNO3CdiSRygdjNIucbbi897Px6QjCGXR0kMdYtny+scvZrMaEv5DINz1+Z9qTUSA0NbxCOf/l5lmrhniZgxZWjh5yq9ONBt5gB2TBAKihoLMBAlaND6JrJ/N4Z39S87qEwVBEjLyUFjGvLXAmImnj7khPhR6Q+SdLHJaigs+BBzPzKfrq345dhf5X1tumBHqsu/FV9bkYBjOIg+GGa4ibzl6ZonwGzVQJcBF+zCwgoEPwsJ7EN1S2Vz/vJI12Yha/H4HOWS9r9CVj/eW4z6Wc6Qv90AUHpnrZE7sVAYN+wsYOhnpEIGf536klHU3BJWewIaJnozg8ChGsUPZHeqbMTbPzP70iLkx4aM8e50vnrtUtlGIj70ijoQeH15o430irdXR7ZIG8/uEWQLZZYduzxN/o3A0S1xqA7OQsM56NhBZ1KemPSAZ0f62HC4eB9pklZagcvheDlKv4tRXuQlTXd/Mw2FzmK2+K5ffFDZ0NXgQ2Lovlj/FS+sYjOBOm7ijpBJQ3eS0PuOaM3g8EidigA7KfPF5GCJbyTCp2WEHDOvLVihU2dn9HK6zPyWGt24eAiPfWhDZ7OzxEQdPxqhiSji17RVDHwLmTcInemuEqX+FwrG1/yaIaW4K4I+eXzMDqSQ== X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(1800799024)(376014)(23010399003)(82310400026)(36860700016)(56012099006)(10067099003)(11063799006)(18002099003)(22082099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: eTEKkIHSZPN9R2kAKD94WYsoQln6zlnTaPXD1sEgOyUM1vmj7fUP9YlU4hcTFQxbIY1uhi7tBnDRmG0C5KXNLQ+zMXOIcxL++yuK7Pc2N4WUUMmyNkk39tq6ixCmQGE3+WQ5vcd18sD1cgTpxWqxQacRURox7yYi32eKO2FyeN2DFoYGWxCFNS8qm6cY3L1nLldZrmRZH6PWYvj2Z5CR5E+ocB8dmOtUWEIMac4Ye73ALm78JeLG6tpDzhh3WeRgXqRxt88rY8xROilS1yHjeBcC64hVLrVJpNjbJaHFKqBr/yk0qMLSK23E6BVYc3BIM7K3I/4p1AZgHzbqaZaPpkMcpm4If806Ox1BPLvM/n3Y84GnTpsj8/acYvwm0kreL5G9gLDdnzf/yjiyzxFZrWmvzN2cVVI6q9m2C5bXESUpoINFAV++jJecEVQUL3a9 X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Sep 2026 09:34:22.4653 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 72599f74-b2ee-4487-9afd-08df080c34ab X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CH2PEPF00000146.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN7PR12MB7129 Content-Type: text/plain; charset="utf-8" Move the BAR iomap check under recovery_lock and hold the lock for reading from there until the export is on vdev->dmabufs, so recovery cannot invalidate the mapping in between. An export attempted while access is blocked is refused by vfio_pci_core_access_begin() like any other device access. Assisted-by: Claude:claude-opus-5 Signed-off-by: Shameer Kolothum --- drivers/vfio/pci/vfio_pci_dmabuf.c | 26 ++++++++++++++++++-------- 1 file changed, 18 insertions(+), 8 deletions(-) diff --git a/drivers/vfio/pci/vfio_pci_dmabuf.c b/drivers/vfio/pci/vfio_pci= _dmabuf.c index c16f460c01d6..a54d199a72c9 100644 --- a/drivers/vfio/pci/vfio_pci_dmabuf.c +++ b/drivers/vfio/pci/vfio_pci_dmabuf.c @@ -243,12 +243,8 @@ int vfio_pci_core_feature_dma_buf(struct vfio_pci_core= _device *vdev, u32 flags, if (!get_dma_buf.nr_ranges || get_dma_buf.flags) return -EINVAL; =20 - /* - * For PCI the region_index is the BAR number like everything - * else. Check that PCI resources have been claimed for it. - */ - if (get_dma_buf.region_index >=3D VFIO_PCI_ROM_REGION_INDEX || - IS_ERR(vfio_pci_core_get_iomap(vdev, get_dma_buf.region_index))) + /* For PCI the region_index is the BAR number like everything else. */ + if (get_dma_buf.region_index >=3D VFIO_PCI_ROM_REGION_INDEX) return -ENODEV; =20 dma_ranges =3D memdup_array_user(&arg->dma_ranges, get_dma_buf.nr_ranges, @@ -274,19 +270,30 @@ int vfio_pci_core_feature_dma_buf(struct vfio_pci_cor= e_device *vdev, u32 flags, priv->vdev =3D vdev; priv->nr_ranges =3D get_dma_buf.nr_ranges; priv->size =3D length; + + ret =3D vfio_pci_core_access_begin(vdev); + if (ret) + goto err_free_phys; + + /* Check that PCI resources have been claimed for the BAR. */ + if (IS_ERR(vfio_pci_core_get_iomap(vdev, get_dma_buf.region_index))) { + ret =3D -ENODEV; + goto err_access; + } + ret =3D vdev->pci_ops->get_dmabuf_phys(vdev, &priv->provider, get_dma_buf.region_index, priv->phys_vec, dma_ranges, priv->nr_ranges); if (ret) - goto err_free_phys; + goto err_access; =20 kfree(dma_ranges); dma_ranges =3D NULL; =20 if (!vfio_device_try_get_registration(&vdev->vdev)) { ret =3D -ENODEV; - goto err_free_phys; + goto err_access; } =20 exp_info.ops =3D &vfio_pci_dmabuf_ops; @@ -311,6 +318,7 @@ int vfio_pci_core_feature_dma_buf(struct vfio_pci_core_= device *vdev, u32 flags, list_add_tail(&priv->dmabufs_elm, &vdev->dmabufs); dma_resv_unlock(priv->dmabuf->resv); up_write(&vdev->memory_lock); + vfio_pci_core_access_end(vdev); =20 /* * dma_buf_fd() consumes the reference, when the file closes the dmabuf @@ -324,6 +332,8 @@ int vfio_pci_core_feature_dma_buf(struct vfio_pci_core_= device *vdev, u32 flags, =20 err_dev_put: vfio_device_put_registration(&vdev->vdev); +err_access: + vfio_pci_core_access_end(vdev); err_free_phys: kfree(priv->phys_vec); err_free_priv: --=20 2.43.0 From nobody Sat Sep 26 13:08:18 2026 Received: from SN4PR0501CU005.outbound.protection.outlook.com (mail-southcentralusazon11011054.outbound.protection.outlook.com [40.93.194.54]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7D30147A885; Tue, 1 Sep 2026 09:34:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.93.194.54 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788255284; cv=fail; b=DRkgCaDRGsiYYv9aowutHP/GmR0QcoOh47fxVl2XdNcQzoybnm6LqpR6G9A7sMCYs4fX3OXYG2H+qLG+C9hthzAMKKJwEMi58T3mcdw6XVx2ojnC0pDKv0Za/SGsvlylvj8ybXUZ/T/bLHUWP2R+R0eL9YY+khKuqohfWcl7OMA= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788255284; c=relaxed/simple; bh=GbsXmPd0px9Y8I9MQRSlikrFCnbR0XaEzx5UEj7RNo8=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=K8HQlD3tX/fO8duLANy8wuMkPC36RGyuT8D1fYb+fWanDnIZ6XJEnXZtlnwIDeOTSAgWMLg3kqut8KnzH9MMNEdJZf5rjkhEbUKDRvB8hFKEmclU4BKmqs83/3852X3dc8m8UwN5PzEyg3zfAzxzY/xjDNjcAcNAxSP9RhJFm7Q= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=ENr3CK+r; arc=fail smtp.client-ip=40.93.194.54 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="ENr3CK+r" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=Fwkse8kVBMHK0jTVcF7PFb+pIxo5qmLOmgZwuU1w5dJUw8nxagGQVhEwf2enW7xEOx24HAFYErX2IYxuwMUsDwjK3I9AGLfJQx8FCyqV/kTiNC8d0TeTZaPJtauSFe7ezzXxk4SbeWIlXYKscnyi0NV3bqBZ7HQt3eHta2TwIOR+ETp+P4Xbd+M5iWhBznz/RJiu9kH+3nvj5irrqyB3Lagi5ebO0JffRS6xT+5Vi8w9rpJhzlLbCU3w7ra5OkHn9gvIhP4tvSS/AQy7KBFFADO0Zre5X64VukQeF0nO+dKS9oz6IO4IZOHT+aHduOH7nUlLB92AC5apzueMpyd9Gg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=lQYvJq7MalZjQc+3FUaQ6CA7OyX2KvLfbvStvaMWosE=; b=kyQ4QcgOXnC/+wojk6cv8jrAA2Ht5oKTM6e32Afqm9vJKwJDlsyagJRjIW8vJ0BFTpKp6DbY1q7j4WgoGtNCiFl8UjbCDomRkDa5xOLNLb8WSpuOT+e9jXCq2vxkRbMELBgpqI9FMwKIMa0l2WoQcV1TZcWWS7HOPZzSMNGcy0Z9FSs4HAW29phgwibRX5MvdXx4gJNHOFAMLJ/YcjQeUfzs14UEf5k9DSHx8D4paYmn23p7e8li7nkgUXKNEn3ZI2o6aNMWXQwGgFeaMCoQilbLZQYWmbU7sGlFAf/2zsJobbAymI+uN5nSQUd5jNARhwRm8A9OuunMh09lSCeNzQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=lQYvJq7MalZjQc+3FUaQ6CA7OyX2KvLfbvStvaMWosE=; b=ENr3CK+rFoyAXNUdAdqtCCL6RjMjIz3N44GcYGzlN0Rhc5Xh/A6DYL7ubCgUeXUYgQW+FJbuQoJ+y9k/1HwiqM6NpjoF7cq9vvFY+C9W/uzSIk2fEWCFR262Z2bmJbcP28tYae5ygE8qm/usrisHw78DWL1xp5JdPS84e4jo2UZTIbwbWeqa33ibkM/CDVAbvHAiWW4hVmOU3+dbFMoW1infZMoVEMTjATeSXibbs6DgdX0acuRzFHAu+nylUG3Rs2uzVAZencJfISVUar0AuR8PqgF+HftzZ4j99FXd4D4GpjlMV6WU6jSnWkdGiv6SwAmlAJI3Un6Rvp7lvTncmw== Received: from BN0PR04CA0196.namprd04.prod.outlook.com (2603:10b6:408:e9::21) by SN7PR12MB6930.namprd12.prod.outlook.com (2603:10b6:806:262::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.360.13; Tue, 1 Sep 2026 09:34:20 +0000 Received: from BL02EPF00021F6E.namprd02.prod.outlook.com (2603:10b6:408:e9:cafe::19) by BN0PR04CA0196.outlook.office365.com (2603:10b6:408:e9::21) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.360.13 via Frontend Transport; Tue, 1 Sep 2026 09:34:20 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by BL02EPF00021F6E.mail.protection.outlook.com (10.167.249.10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.382.8 via Frontend Transport; Tue, 1 Sep 2026 09:34:20 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.46; Tue, 1 Sep 2026 02:34:05 -0700 Received: from NV-2Y5XW94.nvidia.com (10.126.230.37) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.46; Tue, 1 Sep 2026 02:34:02 -0700 From: Shameer Kolothum To: , , CC: , , , , , , , , Subject: [RFC PATCH 14/19] vfio/pci: Add generic PCI error slot reset handling Date: Tue, 1 Sep 2026 10:32:12 +0100 Message-ID: <20260901093217.8539-15-skolothumtho@nvidia.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260901093217.8539-1-skolothumtho@nvidia.com> References: <20260901093217.8539-1-skolothumtho@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: rnnvmail203.nvidia.com (10.129.68.9) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL02EPF00021F6E:EE_|SN7PR12MB6930:EE_ X-MS-Office365-Filtering-Correlation-Id: 8c7784cd-a6ce-4af9-bc4a-08df080c3376 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|36860700016|1800799024|376014|23010399003|6133799003|10067099003|56012099006|11063799006|5023799004|22082099003|18002099003; X-Microsoft-Antispam-Message-Info: zmKHKeEkSojKzPSUZcv9IxIbLMD2pH58S76+s4xxdGwadzCDXGk/OZaznc+l9AUdcbRiFXOmi0KobajfroRr8vZhpSlV4OqbNU4C0tq9OQIoYApA9Dfi4LF+/sPET+ZrSkjdjXw3hOhphoON48L13/NgL6TXl40U8K3O0Qw9s9FQUiDEgmFtPCCaagkSplBVJPeWLvESCzVYjIZyUr1NAaF6TCPLOyv81dLYi7XV/QVHfZJKoZoJb/RGW+dEIfd72XL6AMJUwhkM63UZ1/niaGvtb8MOReF2ju14YNB1KUVTXbJWnCN5x12PE9v/TagMpjavEcKxIz7fKW7YpqVSkW/Z6DOht5OnDGUCDK6ovnTgpzg7UFXeGDbWavpD1pTb50vhBzHUJ9NDSSK41BRJAGWIcKRcPF8HsLKLYUIdm6KVq2xQlg61wsbMSmnpVvtkNocf0J1baxvFtWemWjAoh/X4riuyGhM5YuHB89MIl74ywA2dIWvKOb/oH92QqO34wwwzMO5EyX7LMs0NU/hYC36VFfnqRMqhEceSO2PswfjMPacCu7UQldfC517thFjSUbJ5T8NQcsiJUMj3Og3Lh251jkD7F5r3EIjBZuAic24BlQn0c2IY4XRPLtXyqwrQuSwHzfTE+AzKPnKON5aBKb1K7UhB93V2maN9EqicGWpx3nIOeD4hKhxghxNDvvxNWZvZsasSaTevNtbx1GxaQg== X-Forefront-Antispam-Report: CIP:216.228.117.160;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge1.nvidia.com;CAT:NONE;SFS:(13230040)(82310400026)(36860700016)(1800799024)(376014)(23010399003)(6133799003)(10067099003)(56012099006)(11063799006)(5023799004)(22082099003)(18002099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: LRZDzdmWDjTTGDZKz7i2yDvFFDqiPwXZ+v2KLuBZToOw29oMa2rb0gH2SUzWZaV4koWbqCve93FqXUbp7ypM/CFUIPfSblCTj0zzS40b3hbFQyoVs2HaLFz/yaMKASiHP66mGnX4zRWSWno2rNmdqi1yoENgmHBOMKCuJESjlJ0SIUZqWS+2ziNKhDCb36gHs7BE8kIBIMG3noWqA/4TJYMzBvwDBlpnzT9U54DtUd1yMI+SIHHr1WWHxCOMtLgHvIXhe6xw70bq2Jh/VO/9gyJWZj3NIr40BSlKXKR33aWS4wzapNcIrMwjuh0Xcj472Y3bpBY04DAu+SCpqQ0TaL0k/2ShFPrEUGvAwcFX4XdBc97fLPpFeZarMK2AercxbOq3/y5z8komaiTBxHg/tjCmx2sCzEihcLmY6YsGndzOfQMs0mI5LekF7Eyhok4G X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Sep 2026 09:34:20.4078 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 8c7784cd-a6ce-4af9-bc4a-08df080c3376 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BL02EPF00021F6E.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN7PR12MB6930 Content-Type: text/plain; charset="utf-8" Add a slot_reset() handler for vfio-pci-core. Restore the saved PCI state first. The host resets the link without restoring config space, so on entry the BARs read as zero, and tearing down MSI-X before the restore would write through a stale table address the device no longer decodes. Then tear down the stale interrupt configuration, holding recovery_lock across it. vfio_pci_core_disable() runs the same interrupt teardown when the device is closed and takes no igate, relying on there being no other user by then. vfio_msi_set_vector_signal() frees the per-vector context with no atomicity between the lookup and the erase, so two callers which both find it free the irq, the name and the eventfd context twice. vfio_pci_core_prepare_close() takes recovery_lock for writing before it, so holding it here keeps close out. The interrupt teardown ends in a flush of the virqfd cleanup workqueue, which is shared by every vfio device in the system. So this holds recovery_lock while waiting for other devices' work to finish. That cannot deadlock, because none of that work ever asks for recovery_lock. It can be slow. The flush waits for an ioeventfd write to complete, and that write waits for its own device's memory_lock, which a reset on that device holds. So the wait here can last as long as a reset somewhere else, and anything waiting on this device's recovery_lock waits with it. If the teardown fails, record it as FAILED and return PCI_ERS_RESULT_NONE. Returning DISCONNECT would fail every device under the bridge for a problem which is local to this one. The handler does nothing until a later patch starts a recovery transaction. Assisted-by: Claude:claude-opus-5 Signed-off-by: Shameer Kolothum --- drivers/vfio/pci/vfio_pci_core.c | 72 ++++++++++++++++++++++++++++++++ 1 file changed, 72 insertions(+) diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_c= ore.c index 4447967413e7..b3ad7ed261e1 100644 --- a/drivers/vfio/pci/vfio_pci_core.c +++ b/drivers/vfio/pci/vfio_pci_core.c @@ -2756,6 +2756,77 @@ pci_ers_result_t vfio_pci_core_aer_err_detected(stru= ct pci_dev *pdev, } EXPORT_SYMBOL_GPL(vfio_pci_core_aer_err_detected); =20 +static pci_ers_result_t vfio_pci_core_aer_slot_reset(struct pci_dev *pdev) +{ + struct vfio_pci_core_device *vdev =3D dev_get_drvdata(&pdev->dev); + pci_ers_result_t result =3D PCI_ERS_RESULT_RECOVERED; + int ret =3D 0; + + down_write(&vdev->recovery_lock); + if (!(vdev->pci_recovery_flags & VFIO_PCI_RECOVERY_IN_PROGRESS) || + !vdev->pci_recovery_device_open) { + up_write(&vdev->recovery_lock); + return PCI_ERS_RESULT_NONE; + } + + /* + * Restore first. aer_root_reset() resets the link with + * PCI_RESET_NO_RESTORE, so on entry the BARs read as zero. Tearing + * down MSI-X before this would have pci_msix_shutdown() write through + * the stale table mapping to an address the device no longer decodes. + */ + pci_restore_state(pdev); + + /* + * Hold recovery_lock across the interrupt teardown. + * vfio_pci_core_disable() runs the same teardown on close without + * taking igate, and running the per-vector teardown twice frees the + * irq, the name and the eventfd context twice. + * vfio_pci_core_prepare_close() takes recovery_lock for writing + * before it, so holding it here keeps the two apart. + */ + mutex_lock(&vdev->igate); + if (vdev->irq_type < VFIO_PCI_NUM_IRQS) + ret =3D vfio_pci_set_irqs_ioctl(vdev, + VFIO_IRQ_SET_DATA_NONE | + VFIO_IRQ_SET_ACTION_TRIGGER, + vdev->irq_type, 0, 0, NULL); + mutex_unlock(&vdev->igate); + + if (ret) { + WRITE_ONCE(vdev->pci_recovery_flags, + (vdev->pci_recovery_flags | + VFIO_PCI_RECOVERY_FAILED) & + ~VFIO_PCI_RECOVERY_IN_PROGRESS); + vdev->pci_recovery_command_valid =3D false; + /* + * Vote NONE, not DISCONNECT. A DISCONNECT anywhere in the + * domain makes the core skip resume() for every device under + * the bridge and report permanent failure for all of them. + * Our interrupt teardown failing says nothing about the + * others, so record it locally and leave the domain verdict + * alone. + */ + result =3D PCI_ERS_RESULT_NONE; + } else { + WRITE_ONCE(vdev->pci_recovery_flags, + vdev->pci_recovery_flags | + VFIO_PCI_RECOVERY_RESET); + } + + up_write(&vdev->recovery_lock); + /* + * Whoever clears IN_PROGRESS owes the wake. resume() will not do it, + * since it bails once IN_PROGRESS is clear, and the core skips it + * altogether if the domain verdict is not RECOVERED. On success the + * transaction carries on and resume() wakes. + */ + if (ret) + wake_up_all(&vdev->pci_recovery_wait); + + return result; +} + int vfio_pci_core_sriov_configure(struct vfio_pci_core_device *vdev, int nr_virtfn) { @@ -2828,6 +2899,7 @@ EXPORT_SYMBOL_GPL(vfio_pci_core_sriov_configure); =20 const struct pci_error_handlers vfio_pci_core_err_handlers =3D { .error_detected =3D vfio_pci_core_aer_err_detected, + .slot_reset =3D vfio_pci_core_aer_slot_reset, }; EXPORT_SYMBOL_GPL(vfio_pci_core_err_handlers); =20 --=20 2.43.0 From nobody Sat Sep 26 13:08:18 2026 Received: from BN1PR04CU002.outbound.protection.outlook.com (mail-eastus2azon11010001.outbound.protection.outlook.com [52.101.56.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B9BA6377553; Tue, 1 Sep 2026 09:34:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.56.1 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788255287; cv=fail; b=HZZXG/pGpoTE3MKg7nWtbI4X4Ihf0IXCKuWYKhpbJGQlAydZszyLSD/Txm3c3gy8zwXzqAvmjmuuZZT4rC+qU4TIPavRpo8rpBYqzb9j71bDPvOM47WIAsJlqd+DACpdmnlfGswoiq8O91BGXryfqzk0RxhH7B+dFpFrqNaApsg= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788255287; c=relaxed/simple; bh=RT0Gc85JUb/6wT1vm+vaOpL4a8dvcUODNT6FxY2KINo=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=EbABncGz8iOcpzsfuuZtw04Tw2JU1BRBBbNk0JCFGqLsW35kBKd7qXkrYfXYidzPNbyTb2nQ4g9z7d8BmHr0+YM+O6bpk1DIPOLPsNZKGX117rzxABDsPiBERr2Y5FwNsvctkg3+deWwyHvMjUa5VkRFRs7e6N5UXcaQwEQ4myY= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=c+iSk81v; arc=fail smtp.client-ip=52.101.56.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="c+iSk81v" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=UuO9atuDBAAu8QtixR9Z2d68p/2T07c+OQZHrjTClh9x6UE4Vvux3upiN8FO3YiLDgyAnxolSeD+MxERWKAZumixmrdUVnmFL7VECynPQksDZVECJEvTpPfF/+dItuhbyWo/vgqdz4ay3JtVPZqMfCN017yUmyper68GlKBJcHLN5ne5BxICrLsyGLmuE3f8uwUS8kG7SHZcuzHfJaYFFklCutlCQnwjN9h+PlQpQYpZmEy61LFaJ+PokrF3UL3QEGp5HHldUgfJ8gPkXXP3apGFrjTDxdAQz7x7ghfw5wuV326bvD38jYN23owenr8/Z83IhD239/r4/cRlPR70uA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=STyc5hfCh9lq32weljCIPCl8r2ZXaM3EAl7Wxg/bxD4=; b=xY8biZpMUblGhdEB36UJTrXp39TIPiBxkbPz7aNh5h2OQMRcsSr/VAxLVFxf8+zSYzvgD9IiYCRcVR+AVeOUK2X+/vfgGia2pIwbLibfB5TtCZu7GSgFK7sLbAU2sQ5zH6Qd/njs3SEbK9++3D+bhPafbJE+yCP40wtEithII6uvTHhp6A6V9fmBH5z3xb5mQZGkim4sLhwqJYacXkTuokVIFtKx0zfB1rqRq3i+bK5rcuOgGXjs/JKLca9n2q57/GLlr2+db7vM+C/HWt9rtJiHOwv5DE9iGFSuyBHX5qV7MnOm1GHa+IFEoJiTsHw/t8BITgrQ4rKeqLiU1lPvOA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=STyc5hfCh9lq32weljCIPCl8r2ZXaM3EAl7Wxg/bxD4=; b=c+iSk81v3TppVxE/2WNqm0sunueE1ziSFu57PXSwPBEll/FceS+PPYXtrYF0xxiX9GJOvkRyhAO2KzSdEn0bWK5UVJqo/5tjPdOxLchEizZM+aGfnj8jC21uQtfbNNpcc+uWhQHJQWq32Q+biJ9sD1OTwbsBRHktCaDxg8DRYwckbMBmw+s2oNI6EOMcgJP8qPVd3/qftbJcRUP3Fq1o2rG+Tc7bZy85YR7j6maSwwF/l77twk2YKiWbmAplePylHUUdqgBpj4EOUQ3Ts2t9VYoJb4moZdjseqFiFtAYQy/N0fcxOLdEuuTrXnY/ESVJWPfS7jG6WxrqZ8AWxl5Zkg== Received: from BN9PR03CA0228.namprd03.prod.outlook.com (2603:10b6:408:f8::23) by IA0PR12MB8648.namprd12.prod.outlook.com (2603:10b6:208:486::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.360.13; Tue, 1 Sep 2026 09:34:27 +0000 Received: from BL02EPF00021F6D.namprd02.prod.outlook.com (2603:10b6:408:f8:cafe::28) by BN9PR03CA0228.outlook.office365.com (2603:10b6:408:f8::23) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.360.13 via Frontend Transport; Tue, 1 Sep 2026 09:34:27 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by BL02EPF00021F6D.mail.protection.outlook.com (10.167.249.9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.382.8 via Frontend Transport; Tue, 1 Sep 2026 09:34:26 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.46; Tue, 1 Sep 2026 02:34:09 -0700 Received: from NV-2Y5XW94.nvidia.com (10.126.230.37) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.46; Tue, 1 Sep 2026 02:34:06 -0700 From: Shameer Kolothum To: , , CC: , , , , , , , , Subject: [RFC PATCH 15/19] vfio/pci: Add INTx helpers for PCI recovery Date: Tue, 1 Sep 2026 10:32:13 +0100 Message-ID: <20260901093217.8539-16-skolothumtho@nvidia.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260901093217.8539-1-skolothumtho@nvidia.com> References: <20260901093217.8539-1-skolothumtho@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: rnnvmail203.nvidia.com (10.129.68.9) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL02EPF00021F6D:EE_|IA0PR12MB8648:EE_ X-MS-Office365-Filtering-Correlation-Id: 5b5a9988-0ade-4e63-6974-08df080c375b X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|376014|1800799024|23010399003|36860700016|22082099003|18002099003|56012099006|5023799004|6133799003|11063799006|10067099003; X-Microsoft-Antispam-Message-Info: 012P70SioJlY3MRHc8725nWOBQwDOO0dlGRcUJca7TpE0j2/LdJ9eSMESSRYQu/grMqHlULDXcmevEFQpJb44inEnGv4duMCZGhmHfqJffnDYCijN5GgA01F4eRmC3GzrSD7n5jNXZhfRwkDlHk+YlwP6Xpyj/ZfgqVTPv1ryra5afEhZdhZwaqsHohZ4s7hQNrFgGjmfkV+osHGgFMOW5EhgVOd7v3SV1iFo0qNzhpUDxxRt66YLNMIbh7qxp4m/8jOtO9ozUER3/qlzeZkSf/zoARMIPzZRdbsULvNz0LvIaYHNSt7LROALEmOb2JTh5tJa5Tp/AweUjnyPtue3e3uCtDmMGG7VY4Inqz1KjfHntmLBv4adjv8qFf+L+VgVmvVIcegIoQnixHrZyQ1V86n9GfpTebOn6kOGVidx3oh7pXDs3Z12RsihHvi6YfVBIoWzTuP4h8bOJoV/HYvsJyDjIZVzd3Lz/5uK3G5CPkNw9UuhMLxf2FS5k9sTpS5Cn3odYF1UxEc25H358nNPIHGFbUwXGt9Nf1B4wFRcoqIWgtulBuOqBT3TrHQ8T61pCAzxfIpnczvIH6mzn6fhaNj6wBRjRgpTF8yCMa2KggLrS4jn/omB5+2AMRZLbmrQJjrW9cQRa368so6N6/SWG4u+QzpSuSj4yxgMEc3afNgDpqMzA01sKRw9a2CPyZSyJ3HyiakLU22gzNxks26cg== X-Forefront-Antispam-Report: CIP:216.228.117.160;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge1.nvidia.com;CAT:NONE;SFS:(13230040)(82310400026)(376014)(1800799024)(23010399003)(36860700016)(22082099003)(18002099003)(56012099006)(5023799004)(6133799003)(11063799006)(10067099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: YvToXHLGtDEHdzODkJtU3aiwQ7yCGyIktQwcED+xiI7VibLYZqK4GWMwpj3md0ssUGDkd0WGP855PTiiouUtZUq5Y2OxCoPQSiF0FnwMDvDnxuOBY9wzxp8yglKVXet15y3GVd3sd9U7O0QYhSgErUQ+A8PRTtVlo8JWBOYjBgJUAXTZdBXgrKgfWlZqye/ja4NigCo8pQYcqOLPA/0Uly/Bm5pw3AKOMu7yG+dR5UBXSEvBbLpDEjfH+vky6SWAF8vM0ctxrH98jRTCG20BOJ7/pJn7Q73iFyRhxsLVHsjzBHyE4rS/QvfzoJLzwNW0UIIO34wMzzOIs21OctHsJliILf92q6wINQXOECzuoOonKJhlx3IRikR8x9nhZZprOn7dTt/Y7BQBqCe/+Vb7IMpmDk7U+zzwsMQg8UFr4dzYqSAL5N8Urck1rZjIlmUj X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Sep 2026 09:34:26.9472 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 5b5a9988-0ade-4e63-6974-08df080c375b X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BL02EPF00021F6D.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA0PR12MB8648 Content-Type: text/plain; charset="utf-8" Add the helpers the recovery callbacks need to keep INTx in step with an error event, and the two per-context flags they record it in. Nothing calls them yet. vfio_pci_intx_recovery_start() masks the line when an event begins. Nothing has fired at that point, and pci_check_and_mask_intx() only writes DisINTx when the status register says an interrupt is pending, so it would find nothing to do and leave the line enabled. Use pci_intx() instead, which masks whatever the device is doing. __vfio_pci_intx_mask() already does this for the same reason. vfio_pci_intx_recovery_finish() replays what the event masked, and any unmask which arrived while it ran, once the event ends. An interrupt which was masked and delivered while access was blocked is not replayed. The user was told about it, and unmasks it as it would outside recovery. vfio_pci_intx_recovery_command() reconciles INTX_DISABLE with the command word error_detected() saves. error_detected() sets that bit when it writes the quiesced command word, without recording it as a mask, so the saved word and ctx->masked can disagree about it. The helper keeps the bit as the INTx state has it, for resume() to use when it restores the word. Restoring the saved bit instead would unmask a line the handler still believes is masked, and a shared pci_2_3 line would storm until note_interrupt() disabled it for every device on it. Assisted-by: Claude:claude-opus-5 Signed-off-by: Shameer Kolothum --- drivers/vfio/pci/vfio_pci_priv.h | 4 ++ drivers/vfio/pci/vfio_pci_intrs.c | 116 ++++++++++++++++++++++++++++++ 2 files changed, 120 insertions(+) diff --git a/drivers/vfio/pci/vfio_pci_priv.h b/drivers/vfio/pci/vfio_pci_p= riv.h index 8a7f9fe22386..5598e472da4b 100644 --- a/drivers/vfio/pci/vfio_pci_priv.h +++ b/drivers/vfio/pci/vfio_pci_priv.h @@ -25,6 +25,10 @@ struct vfio_pci_ioeventfd { =20 bool vfio_pci_intx_mask(struct vfio_pci_core_device *vdev); void vfio_pci_intx_unmask(struct vfio_pci_core_device *vdev); +void vfio_pci_intx_recovery_start(struct vfio_pci_core_device *vdev); +void vfio_pci_intx_recovery_finish(struct vfio_pci_core_device *vdev); +u16 vfio_pci_intx_recovery_command(struct vfio_pci_core_device *vdev, + u16 command); =20 int vfio_pci_eventfd_replace_locked(struct vfio_pci_core_device *vdev, struct vfio_pci_eventfd __rcu **peventfd, diff --git a/drivers/vfio/pci/vfio_pci_intrs.c b/drivers/vfio/pci/vfio_pci_= intrs.c index 64f80f64ff57..c4a075b5bb2e 100644 --- a/drivers/vfio/pci/vfio_pci_intrs.c +++ b/drivers/vfio/pci/vfio_pci_intrs.c @@ -29,6 +29,8 @@ struct vfio_pci_irq_ctx { struct virqfd *mask; char *name; bool masked; + bool recovery_masked; + bool unmask_pending; struct irq_bypass_producer producer; }; =20 @@ -220,6 +222,40 @@ void vfio_pci_intx_unmask(struct vfio_pci_core_device = *vdev) mutex_unlock(&vdev->igate); } =20 +/* + * Mask INTx because recovery has blocked device access. Returns true if t= his + * call did the masking, which means recovery is the one which must unmask. + * + * Nothing is normally asserted when a recovery starts, and + * pci_check_and_mask_intx() only writes DisINTx when the status register = says + * an interrupt is pending, so it would leave the line alone. pci_intx() m= asks + * whatever the device is doing, as __vfio_pci_intx_mask() already does fo= r the + * same reason. + * + * Masking a pci_2_3 device goes through config space. If the error left + * config space unreadable the write has no effect and the line stays + * asserted, which is no worse than not trying. For a non-fatal error conf= ig + * space still works, and this is what keeps a shared line from storming w= hile + * access is blocked. + */ +static bool vfio_pci_intx_mask_for_recovery(struct vfio_pci_core_device *v= dev, + struct vfio_pci_irq_ctx *ctx) +{ + lockdep_assert_held(&vdev->irqlock); + + if (ctx->masked) + return false; + + if (!vdev->pci_2_3) + disable_irq_nosync(vdev->pdev->irq); + else + pci_intx(vdev->pdev, 0); + + ctx->masked =3D true; + ctx->recovery_masked =3D true; + return true; +} + static irqreturn_t vfio_intx_handler(int irq, void *dev_id) { struct vfio_pci_irq_ctx *ctx =3D dev_id; @@ -247,6 +283,86 @@ static irqreturn_t vfio_intx_handler(int irq, void *de= v_id) return ret; } =20 +void vfio_pci_intx_recovery_start(struct vfio_pci_core_device *vdev) +{ + struct vfio_pci_irq_ctx *ctx; + unsigned long flags; + + lockdep_assert_held_write(&vdev->recovery_lock); + + spin_lock_irqsave(&vdev->irqlock, flags); + if (!is_intx(vdev)) + goto out_unlock; + + ctx =3D vfio_irq_ctx_get(vdev, 0); + if (WARN_ON_ONCE(!ctx)) + goto out_unlock; + + vfio_pci_intx_mask_for_recovery(vdev, ctx); + +out_unlock: + spin_unlock_irqrestore(&vdev->irqlock, flags); +} + +/* + * Replay the masking recovery did, and any unmask which arrived while it = was + * blocked. Call this only after access_blocked has been cleared, or the + * replayed unmask is swallowed and recorded as pending again with nothing + * left to replay it. + */ +/* + * The command word saved before the quiesce can have INTX_DISABLE clear, = but + * the INTx handler may have masked the line since. Keep the bit as the IN= Tx + * state has it, so hardware and ctx->masked agree until + * vfio_pci_intx_recovery_finish() replays. Restoring the saved bit instead + * would unmask a line the handler still believes is masked, and a shared + * pci_2_3 line would then storm until note_interrupt() disables it. + */ +u16 vfio_pci_intx_recovery_command(struct vfio_pci_core_device *vdev, + u16 command) +{ + struct vfio_pci_irq_ctx *ctx; + + lockdep_assert_held(&vdev->irqlock); + + if (!is_intx(vdev)) + return command; + + ctx =3D vfio_irq_ctx_get(vdev, 0); + if (ctx && ctx->masked) + command |=3D PCI_COMMAND_INTX_DISABLE; + + return command; +} + +void vfio_pci_intx_recovery_finish(struct vfio_pci_core_device *vdev) +{ + struct vfio_pci_irq_ctx *ctx; + unsigned long flags; + bool replay =3D false; + + lockdep_assert_held_write(&vdev->recovery_lock); + + mutex_lock(&vdev->igate); + spin_lock_irqsave(&vdev->irqlock, flags); + if (!is_intx(vdev)) + goto out_unlock; + + ctx =3D vfio_irq_ctx_get(vdev, 0); + if (WARN_ON_ONCE(!ctx)) + goto out_unlock; + + replay =3D ctx->recovery_masked || ctx->unmask_pending; + ctx->recovery_masked =3D false; + ctx->unmask_pending =3D false; + +out_unlock: + spin_unlock_irqrestore(&vdev->irqlock, flags); + if (replay) + __vfio_pci_intx_unmask(vdev); + mutex_unlock(&vdev->igate); +} + static int vfio_intx_enable(struct vfio_pci_core_device *vdev, struct eventfd_ctx *trigger) { --=20 2.43.0 From nobody Sat Sep 26 13:08:18 2026 Received: from BN8PR05CU002.outbound.protection.outlook.com (mail-eastus2azon11011043.outbound.protection.outlook.com [52.101.57.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A868D3955EF; Tue, 1 Sep 2026 09:34:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.57.43 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788255291; cv=fail; b=JxI4c8pJLhKz755P5wyPAgWwQyU0xzY70CapTpyA9K47j9BlBe/x+vvudztEDd1nEUp3dYOdXCsX7ibhGtPxJMy4cVe9koj3WuvyvN6B3B/1H2WYYViatFa0kI1bBWH+kWkOtiJ7JOhohZYbki1y6Ksj91MyP/kLYF/fp4+eSs0= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788255291; c=relaxed/simple; bh=lP9kP7DCdKujEUq/92dquV6AQq1zF82/IU9NMClYwf0=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=NLQZnSv9beB9AL51NywrxyngvzRH3XpQb28+WdNGXCxUyetx0El3BmlfI40JUf7I8XzJx+fQ8Dp1idiraT7297vLNJ6Yv0+VTWUVnMa+dqiH+MPw4PN88rSP0nDe/n6xVGVovp81e50m4PrC5wJGBPXlmnLOeTNwVTv3UFmtHig= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=Jn0TSUOS; arc=fail smtp.client-ip=52.101.57.43 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="Jn0TSUOS" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=LuyzUD9zImcPK9PYz7yHZDylLCfgnOsrQ4GGlthnqlqgPt3liSZgV3pKVkgQKkqHedyvhkFwdxCmRcN7qZGGdB0b3+OTludX0YQUUObj6bNCoZ0ciDSe0yVtmBscYj8WwcOSTFkzugJKidzForRQ7ccbGKCpseqSzcfqC5dSdOOK8ajQiEPqjedEzjaa8bvjNQ5TfOta/c3HVpNvi8hsrOxE/fbkq0JD/2f5sEmH7lltI/GJd/ztDLlLRtm7NtyqyKkFozJHwMA3rbEHKM9a9G2/izVDSvaCQ7+oeMI0++td/n0bnbvCriKiCOVwcZnzkddLWYUrdF1MTfHOapoJTA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Cd0srwENxB6+E1FwtIKPOG831nCxmcLFbgoM+joZ+O0=; b=WjyTR4YMiTd3M4UzLuChuMomv0U4hYOMrYVuG3Y479jVPrHOH6apxmjzMemk10essQ1cgFi2TMB9jY1DZmqmzJ7u9FQCsbpq3OnzBFKWp++DCO6yQIFNDDJ+Ot2XIqagIqeZ2lyO1gwqvTS9wrCUGiKh3lVu+lRnZkjk0k8IFNCd5Q4eXLM9tfAe4SrKuKPHLr/ZmSq7MzK2LBAi7GN+JgPfVLQrNRdRk2z/7pBbreFSE/yDDLsRsDQJkBmsJy36t+wGHZ17Gpo45/PL/doN7hRdJ9sIcu5M1ZWNc+Z6zFoKMGmLDC85iUHUPBZ/Og9x4e+USgRaxufOP20X5di3HA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Cd0srwENxB6+E1FwtIKPOG831nCxmcLFbgoM+joZ+O0=; b=Jn0TSUOSK6Gxd+u8HiJbVKIy6Q8enRIql8czs12rpBv91IqEIsX3O9Xwe3Icr0O/h8NAxoe4+hPank2APi6lhRl1svmnznpdUeUCSMbR8704N4F71kQXt2hHe85iKlMxo/L/Ux0UNEr+5ZdoPBjR2jIiZPHLQMu2h/1TXCzC69UC1DDoItDne6ihOhHCMaU6dli3FJC/sGJQDThvWXVfI4paOjM1cUIPxIyqT/fm0HTeQvQF3MZTFm8MUuBnOP/hTWQIuDBZz9PZQdI4Vr+TbxCacUq50kfwckW+njSJHN9KYuNqbMeiJPkT8pI65eTFppshVuJULVklQDnr8/pVyQ== Received: from CH2PR16CA0018.namprd16.prod.outlook.com (2603:10b6:610:50::28) by CH3PR12MB9098.namprd12.prod.outlook.com (2603:10b6:610:19e::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.360.12; Tue, 1 Sep 2026 09:34:31 +0000 Received: from CH2PEPF00000148.namprd02.prod.outlook.com (2603:10b6:610:50:cafe::53) by CH2PR16CA0018.outlook.office365.com (2603:10b6:610:50::28) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.360.13 via Frontend Transport; Tue, 1 Sep 2026 09:34:31 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by CH2PEPF00000148.mail.protection.outlook.com (10.167.244.105) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.382.8 via Frontend Transport; Tue, 1 Sep 2026 09:34:31 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.46; Tue, 1 Sep 2026 02:34:13 -0700 Received: from NV-2Y5XW94.nvidia.com (10.126.230.37) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.46; Tue, 1 Sep 2026 02:34:10 -0700 From: Shameer Kolothum To: , , CC: , , , , , , , , Subject: [RFC PATCH 16/19] vfio/pci: Quiesce INTx during PCI recovery Date: Tue, 1 Sep 2026 10:32:14 +0100 Message-ID: <20260901093217.8539-17-skolothumtho@nvidia.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260901093217.8539-1-skolothumtho@nvidia.com> References: <20260901093217.8539-1-skolothumtho@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: rnnvmail203.nvidia.com (10.129.68.9) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PEPF00000148:EE_|CH3PR12MB9098:EE_ X-MS-Office365-Filtering-Correlation-Id: cb0693c4-41d1-4ce9-56e4-08df080c39e8 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|23010399003|36860700016|82310400026|1800799024|6133799003|5023799004|11063799006|10067099003|56012099006|22082099003|18002099003; X-Microsoft-Antispam-Message-Info: aO5W8mSZMLmXO3yUVXEHRUfz2WUuJ4PERqtuc9jcqGrQqLdIzv9x7EBdSYnj3SiYri/lKVuYy4Q2BCU8IFCgUD9JfAuIfWUbH9Dky1icETNRXdNpt6GrX743PjLi/r8Pzbs2CRyBMGXT9namKgTJqGrpuK1s6H8iRlooFf+YsgXhlByoS6wZBIYwGc8E7YA8qYjI3F0jTdTix7zbG/RoT4HoEcy5+cm1EoOHp1LNS/m8ZZNKeTV34eWIJLblgMXnCRhghjbsjaVfoXfqpxgjksOCVc6B99CxDv3jaaSAYEri0yRNvuTl9A4PO8RSOrimDKhnWqPmkjvDO2P1Ijv07NeMbZA199Noep9FyqPkzP8nAcQ82X1FkbNyiDz/N69odbhpqnpXlLz9/aBUg9QGo1qu4ZTt7sncHeu137HTJs5AD48TM/Wpn4Vf9mPjJ1iWHEgVhUh/Qu3OxkNcq6Qq8OcQY+G9jUz6nMd8X/8l7M8xEIWMhnxDkDDoWd38AUz7hOZB3RA5AOdEY8rueIOeWn/pd5ZVVVOasUf82VqG013CzYdx8o84IhClNbOboe0EYbWmBuwp78sAxzJL40mXYSxjf+5axPaK+piBI5Hc8WGZjg+YLG0Hl7Vh9t4pByQmM6rVR97kZPS0bIcP6vHuHfMk6YLpzaOjOy2ZhjSKDJYXLngQpFKrZHl/BRh5lxqUwW5t9/yiKgoqFDGkYtEcbQ== X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(376014)(23010399003)(36860700016)(82310400026)(1800799024)(6133799003)(5023799004)(11063799006)(10067099003)(56012099006)(22082099003)(18002099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: bTKRxJ9jkYbRtiGkKma1YD+x0GoE9SJAQGwSqUCn28ylwlLgHeY4O17mvH2iEqoN7enaDuecDS8vtRIsaJGeCqFj4Xt8K+IzZLiZi+NJRMHD2Z46CqV3SwxkL3cCCbD1TojvPFNo7tERDugbMruysUbEgHzclXxyWWa3qlExO3teUwKlVnoZ4psWDxTM4qkQc02wH5FG+FEb0fCEwwcIf5X855R2Ve4rCrkXXQku1vPGz2HjL8WFuXGmsOCjkgHNV9+mTXWSArPq9pjIgdzOYNPZFdwQ6LJHsYVbeqPkWyktAEIsA2RIvGfvGGLPWlcuCfoLT5C8fXSUFuHtmaMY/IygqCPQ+9kAKhoceqv6gW6eJHTtc23IAT4Vx7ufh2muoaPjWMsscEnYu+8vhN7emUratASgvMU/pVTx+ocyRhV3tlHwsPOGYBULjRvNAloP X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Sep 2026 09:34:31.2565 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: cb0693c4-41d1-4ce9-56e4-08df080c39e8 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CH2PEPF00000148.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH3PR12MB9098 Content-Type: text/plain; charset="utf-8" Mask INTx when recovery blocks device access. A PCI 2.3 device shares its line, so leaving a level interrupt asserted and returning IRQ_NONE would storm until note_interrupt() disables the line for every device on it. Mask through pci_check_and_mask_intx() and return IRQ_HANDLED instead. A device without per-function masking has the line to itself and is quiesced through genirq. That is a different job from the masking recovery does for itself, so vfio_pci_intx_mask_for_recovery() gains a @quiesce argument to tell them apart. The handler knows an interrupt has fired but not whether this device is the one asserting the shared line, which is what pci_check_and_mask_intx() reports, and a false return means the interrupt belongs to another device. It is also the user who unmasks afterwards, having been sent the eventfd, so the handler does not record the mask as one recovery owes. Record unmask requests received during recovery instead of losing them, so vfio_pci_intx_recovery_finish() can replay them once the event ends. The unmask handler checks for a blocked device before the existing INTx test, so a blocked device is not touched through pci_intx(). It only dereferences the interrupt context once that test has passed. The DisINTx emulation calls the unmask path whatever irq_type is set to, and the context is NULL when INTx is not in use. Replay from the reset path as well. The unmask eventfd reaches vfio_pci_intx_unmask_handler() through virqfd, which takes no recovery_lock and so cannot be refused, and VFIO_DEVICE_RESET blocks access for its duration. Without a replay there the line stays masked and the guest waits for an interrupt which cannot arrive, since it only unmasks again after receiving one. Assisted-by: Claude:claude-opus-5 Signed-off-by: Shameer Kolothum --- drivers/vfio/pci/vfio_pci_core.c | 1 + drivers/vfio/pci/vfio_pci_intrs.c | 61 ++++++++++++++++++++++++++----- 2 files changed, 53 insertions(+), 9 deletions(-) diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_c= ore.c index b3ad7ed261e1..658cccecab12 100644 --- a/drivers/vfio/pci/vfio_pci_core.c +++ b/drivers/vfio/pci/vfio_pci_core.c @@ -1558,6 +1558,7 @@ int vfio_pci_try_reset_function(struct vfio_pci_core_= device *vdev, VFIO_PCI_RECOVERY_FAILED))) { vfio_pci_recovery_rom_disable(vdev); WRITE_ONCE(vdev->pci_recovery_access_blocked, false); + vfio_pci_intx_recovery_finish(vdev); } up_write(&vdev->recovery_lock); /* diff --git a/drivers/vfio/pci/vfio_pci_intrs.c b/drivers/vfio/pci/vfio_pci_= intrs.c index c4a075b5bb2e..1981a64b6e18 100644 --- a/drivers/vfio/pci/vfio_pci_intrs.c +++ b/drivers/vfio/pci/vfio_pci_intrs.c @@ -51,6 +51,15 @@ static bool is_irq_none(struct vfio_pci_core_device *vde= v) vdev->irq_type =3D=3D VFIO_PCI_MSIX_IRQ_INDEX); } =20 +static bool vfio_pci_recovery_blocks_irq(struct vfio_pci_core_device *vdev) +{ + if (!vdev->pci_recovery_supported) + return false; + + return READ_ONCE(vdev->pci_recovery_enabled) && + READ_ONCE(vdev->pci_recovery_access_blocked); +} + static struct vfio_pci_irq_ctx *vfio_irq_ctx_get(struct vfio_pci_core_device *vde= v, unsigned long index) @@ -173,6 +182,16 @@ static int vfio_pci_intx_unmask_handler(void *opaque, = void *data) int ret =3D 0; =20 spin_lock_irqsave(&vdev->irqlock, flags); + /* + * Check for a blocked device before the INTx test below, so a blocked + * device is not touched through pci_intx(). @ctx is only valid when + * INTx is in use, so record the request only then. + */ + if (unlikely(vfio_pci_recovery_blocks_irq(vdev))) { + if (is_intx(vdev)) + ctx->unmask_pending =3D true; + goto out_unlock; + } =20 /* * Unmasking comes from ioctl or config, so again, have the @@ -184,6 +203,9 @@ static int vfio_pci_intx_unmask_handler(void *opaque, v= oid *data) goto out_unlock; } =20 + ctx->unmask_pending =3D false; + ctx->recovery_masked =3D false; + if (ctx->masked && !vdev->virq_disabled) { /* * A pending interrupt here would immediately trigger, @@ -224,13 +246,16 @@ void vfio_pci_intx_unmask(struct vfio_pci_core_device= *vdev) =20 /* * Mask INTx because recovery has blocked device access. Returns true if t= his - * call did the masking, which means recovery is the one which must unmask. + * call did the masking. * - * Nothing is normally asserted when a recovery starts, and + * Set @quiesce when recovery is masking the line itself rather than maski= ng + * one delivered interrupt. Nothing is normally asserted at that point, and * pci_check_and_mask_intx() only writes DisINTx when the status register = says - * an interrupt is pending, so it would leave the line alone. pci_intx() m= asks - * whatever the device is doing, as __vfio_pci_intx_mask() already does fo= r the - * same reason. + * an interrupt is pending, so it would leave the line alone. The interrupt + * handler wants that test, since a false return there means the interrupt + * belongs to another device on a shared line. @quiesce also records that + * recovery is the one which must unmask. A masked interrupt which was + * delivered is the user's to unmask, exactly as outside recovery. * * Masking a pci_2_3 device goes through config space. If the error left * config space unreadable the write has no effect and the line stays @@ -239,7 +264,8 @@ void vfio_pci_intx_unmask(struct vfio_pci_core_device *= vdev) * access is blocked. */ static bool vfio_pci_intx_mask_for_recovery(struct vfio_pci_core_device *v= dev, - struct vfio_pci_irq_ctx *ctx) + struct vfio_pci_irq_ctx *ctx, + bool quiesce) { lockdep_assert_held(&vdev->irqlock); =20 @@ -248,11 +274,14 @@ static bool vfio_pci_intx_mask_for_recovery(struct vf= io_pci_core_device *vdev, =20 if (!vdev->pci_2_3) disable_irq_nosync(vdev->pdev->irq); - else + else if (quiesce) pci_intx(vdev->pdev, 0); + else if (!pci_check_and_mask_intx(vdev->pdev)) + return false; =20 ctx->masked =3D true; - ctx->recovery_masked =3D true; + if (quiesce) + ctx->recovery_masked =3D true; return true; } =20 @@ -264,6 +293,19 @@ static irqreturn_t vfio_intx_handler(int irq, void *de= v_id) int ret =3D IRQ_NONE; =20 spin_lock_irqsave(&vdev->irqlock, flags); + if (unlikely(vfio_pci_recovery_blocks_irq(vdev))) { + /* + * Mask rather than return IRQ_NONE with the line still + * asserted. For a shared pci_2_3 line an unhandled level + * interrupt storms until note_interrupt() disables the line + * for every device on it, not just this one. + */ + if (vfio_pci_intx_mask_for_recovery(vdev, ctx, false)) + ret =3D IRQ_HANDLED; + else if (ctx->masked && !vdev->pci_2_3) + ret =3D IRQ_HANDLED; + goto out_unlock; + } =20 if (!vdev->pci_2_3) { disable_irq_nosync(vdev->pdev->irq); @@ -275,6 +317,7 @@ static irqreturn_t vfio_intx_handler(int irq, void *dev= _id) ret =3D IRQ_HANDLED; } =20 +out_unlock: spin_unlock_irqrestore(&vdev->irqlock, flags); =20 if (ret =3D=3D IRQ_HANDLED) @@ -298,7 +341,7 @@ void vfio_pci_intx_recovery_start(struct vfio_pci_core_= device *vdev) if (WARN_ON_ONCE(!ctx)) goto out_unlock; =20 - vfio_pci_intx_mask_for_recovery(vdev, ctx); + vfio_pci_intx_mask_for_recovery(vdev, ctx, true); =20 out_unlock: spin_unlock_irqrestore(&vdev->irqlock, flags); --=20 2.43.0 From nobody Sat Sep 26 13:08:18 2026 Received: from SA9PR02CU001.outbound.protection.outlook.com (mail-southcentralusazon11013037.outbound.protection.outlook.com [40.93.196.37]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 48EE447798B; Tue, 1 Sep 2026 09:34:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.93.196.37 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788255300; cv=fail; b=OKu9xpNuh5HNctCwrNrcyHDjSivQfDHtcfsjOS3sqZ7PibKWPj7LX3NstQKMuznyqXjFkd1OOY2OdeqImTmb2nPtWOafzE+5BqWzLB6ueegcEryHbdAONlwA7k2sm4Vd3CXl7J7RXiP1d0PccrRKFyhW9T6iRdqDmd0b14MsNU0= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788255300; c=relaxed/simple; bh=4jsXGs+J2sCiGycsIrxOIo3v+w4WmYuwD/LU5IBgkrI=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=In/qcbp+2YAqr1dR/U4WlHFjAQu41flKOZ07I3rFrGJkGCgQrD8Cc/Gax+7qy+ozkqaMlhAL5mIgbjttmPHCK/JH5uPpDTyLXcOJ9hdo5foc++ELNTyMP1BJgOQgePx44vSBn9BPBioPdOcGh7qyAVTZFuhgpnOnwr7qFzW0v3A= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=KWV1aYHD; arc=fail smtp.client-ip=40.93.196.37 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="KWV1aYHD" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=hGFVeBvi06zugAyzINHbJW+bVEA8CEIgoJLPLDzA4nne4oQ/6xTYyrvrc0KuHp0ISkQjBjtnsTCxqJnjHLycTH073E+ehJlkpiN/+W/Kh6IJ+qJVYkzl8RddU5LVv26iVjvLVf3ldjxUJfhKlJIL+xmC0tYjYVxlV75H28w0GdmyAMknTDUagCXqc00ACxVw3xjbZ+N6cJr8IcWpZCq9WOA1LVW65AopsvOn3eC02h/CT4o35R9bYAUObuHnQkAy2jDpDNdWhgT4UDwTL8KTYGDvNVIqP0tDKqLr8IR/Wp7bxarOzuwRh8hO/pand5pU0LXT18EB9NiETvIlIMUeiA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=3bL/NEcWNk17n+4zOtixjgiWBmQVQWkTOJJ47AnFrEk=; b=NGbGSufncm0EAyvHfzT+3ANF2IMKjnErhos5A+rTkxiX76CLZOqFGZA/SarXAbxvzfx8jsg2M/W7iczsr3+EWWnH2jmYJBpcmFkOE4zoozVJUOh6rnw1pm5zG6WGJOZRkM2doy8KQ7N8fCm/WivroFjJqxRX/mhdSqRi+VHT9XSNy+XMWs4XB4fDlGcqwwFdF6Cvyvw3+D3PJokvD1MFRXwkS5pwGBIO7UXzf+rz8ej4aC1y01igvnsO1cGUZGUQApwdQXJdSepV4g7GkillG0BG1CnLhIBjezHKH8r1m8Yo2wXRi5GYUOdDL/KRKDybcUXLSnjg6Uh83+cR1JhbVQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=3bL/NEcWNk17n+4zOtixjgiWBmQVQWkTOJJ47AnFrEk=; b=KWV1aYHDkIY5OrSj+mbawAoixBhMUEVbDANsbRVvlsBIGqz2O1hshA3VBiD6b6Hv6fERhwLqpmgQI3VtZmlLEpQevqYUsiQ2qQRHl/xHvgJX5/VSGdHCCC1t58WKEVYdGgUSTU6PzrFCTjwjGlQH/uYLCOyVdodxA4DDtH95zXtWRWpEpwINO/mBJxfcUcSRmWp6CJevMoCPWxSsT6fKDApaPcJC2aqi5vWR4ZTjc4WptPT6zsYIt7ujupHn42Ioz2lPzeO+GHMfmBdF7OB/muE6f4NGmxVeiPYJzhysq5ow/AWzZDhCNgc4ybVhN4UMrMs5GXpIkNPJhfLxcPtRrg== Received: from CH2PR16CA0002.namprd16.prod.outlook.com (2603:10b6:610:50::12) by DS7PR12MB6215.namprd12.prod.outlook.com (2603:10b6:8:95::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.360.13; Tue, 1 Sep 2026 09:34:36 +0000 Received: from CH2PEPF00000148.namprd02.prod.outlook.com (2603:10b6:610:50:cafe::7a) by CH2PR16CA0002.outlook.office365.com (2603:10b6:610:50::12) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.382.10 via Frontend Transport; Tue, 1 Sep 2026 09:34:36 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by CH2PEPF00000148.mail.protection.outlook.com (10.167.244.105) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.382.8 via Frontend Transport; Tue, 1 Sep 2026 09:34:36 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.46; Tue, 1 Sep 2026 02:34:19 -0700 Received: from NV-2Y5XW94.nvidia.com (10.126.230.37) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.46; Tue, 1 Sep 2026 02:34:16 -0700 From: Shameer Kolothum To: , , CC: , , , , , , , , Subject: [RFC PATCH 17/19] vfio/pci: Add generic PCI error resume handling Date: Tue, 1 Sep 2026 10:32:15 +0100 Message-ID: <20260901093217.8539-18-skolothumtho@nvidia.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260901093217.8539-1-skolothumtho@nvidia.com> References: <20260901093217.8539-1-skolothumtho@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: rnnvmail203.nvidia.com (10.129.68.9) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PEPF00000148:EE_|DS7PR12MB6215:EE_ X-MS-Office365-Filtering-Correlation-Id: d5ddda40-c5a5-481d-4812-08df080c3cdf X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|376014|82310400026|36860700016|23010399003|10067099003|56012099006|11063799006|22082099003|18002099003; X-Microsoft-Antispam-Message-Info: Vougr2qrhI14Ie2DIFIaLI1N7Lelq0jHlZGLQA5LJ3zYzWGVicuzjwzaw88nBa7ngQqPalrbTckrGtMcjbZMpGZan5jgq2DCa6q4B68VGFCEs54aNZp/Mgvy/e2XCyiO6eNes8Ei48+oqCiH4278Sf1WMAvt13wU/j7qQci50bHAowp7Nhby67k+Iobc85L39WjZ+rM6gOAyBkcGchWomj3744V0tnWovi6ED9ppFcRFx/X+STgHZJi722NWdY4qNpUOXagkCcixftOjpTYsvrOvUtkpsM1F/rcJtnducx3WfcYzrvo6JRRbDLOPgiOQjww2rWLrTdXugjdfeQwsP+LiwnNFYdNa9/USRzvHoG6LAWLnGMo9lncTn7XHvB5Z8mXvqrlpR8aKbB4QcKAjo/ptUe0EwNwoV3fbR0fTBXxqH+T4HPvNnPciIsJbgcD6XJ4Xo9y6wBdSzbsgClynHt0DCYy9LiW/vS1f/A4VwCLghskyrWL0QkS6XOfKz6CwlyjW+QPY2OKVmfcfWr/3QJr4UHosMe/qmC/xowQ6l5aZl2B2dYkWwPxmMdzT1RX9wgGXqNQYBvA4kRTCgZEvEyT0dGLXtqB2nIaLAOUUTo4G3HH/OQ5GN1c3g4zZGBoYUci0FIrydYjJoCWBSJEQgkWPEGBnsaYNOmjY3/1RH/sYxQR0GNJs/BlVqYhFXFAPyIpJZeNkeZckXb54PAWtKA== X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(1800799024)(376014)(82310400026)(36860700016)(23010399003)(10067099003)(56012099006)(11063799006)(22082099003)(18002099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: Y9Ybdfs5TOyx3N/oWZp629wQBlZMKGHIrYxSB/kLSohCrn5i0LkLFRvxE9zrsBo1NphRBxA8gAkeDz87p9bbDrcNyux/QeG0DBEdD/HTsEICICPwTqiT5wk40W39SfmmRrSAXvqUl5sPUIPlDiRUYt0vxKeJmUU+RsC8T6PdZsQbIvXsBMXCvq6QLUr5AWL08MJ8l9PjbP5Qqk7RUlpH9dJPXZMDRuF5bolniLeRlTg2dLeIEaFXKNghoi5v21Nc7BOJWrzL3Q9uHufAHhUwL2zH9y8EJVyIUE4rggwKOsAD2o5YYOqyvK4U8hBCiKVFxc48/LDVtSkwIX35b9Qc5eo8MFz4p/3Mx2Prnc9VzaSLe8RWjw65nWQXEPhGZ71I5cvDcFvO987Un48w1GsCVWnZCIvDTTfCVFd0Mgf7E+aMvk87Js+DBqwRVdupokGY X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Sep 2026 09:34:36.2302 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: d5ddda40-c5a5-481d-4812-08df080c3cdf X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CH2PEPF00000148.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR12MB6215 Content-Type: text/plain; charset="utf-8" Add a resume() handler for vfio-pci-core. It ends the recovery transaction and makes the device usable again. INTX_DISABLE is taken from the INTx state rather than from the saved word. The handler can have masked the line after the word was saved, and restoring the saved bit would unmask a line it still believes is masked. The replay is what unmasks it. Restore the command word saved by error_detected(), unless the device was reset, in which case slot_reset() already restored the whole config space and the saved value is stale. Then unblock access, un-revoke exported DMA-BUFs if the guest still has memory decode enabled, clear IN_PROGRESS and replay the INTx state recorded during the event. If the command write fails, keep access blocked and publish FAILED. IN_PROGRESS is cleared and FAILED set in a single store, so a lock-free reader never sees the intermediate state, which would read as a successful completion. Pay any deferred ROM decode disable first, before the in-progress check. A failed transaction has already cleared that flag, and the disable would be lost. The helper skips a closed device on its own. Until a later patch starts a recovery transaction, only the deferred ROM disable runs here. The rest returns early. Signed-off-by: Shameer Kolothum --- drivers/vfio/pci/vfio_pci_core.c | 68 ++++++++++++++++++++++++++++++++ 1 file changed, 68 insertions(+) diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_c= ore.c index 658cccecab12..eed0430c32ee 100644 --- a/drivers/vfio/pci/vfio_pci_core.c +++ b/drivers/vfio/pci/vfio_pci_core.c @@ -2828,6 +2828,73 @@ static pci_ers_result_t vfio_pci_core_aer_slot_reset= (struct pci_dev *pdev) return result; } =20 +static void vfio_pci_core_aer_resume(struct pci_dev *pdev) +{ + struct vfio_pci_core_device *vdev =3D dev_get_drvdata(&pdev->dev); + unsigned long irq_flags; + u32 flags; + int ret =3D 0; + + down_write(&vdev->recovery_lock); + + /* + * Pay any deferred ROM disable before the in-progress check below, + * which a failed transaction has already cleared, or it would be + * lost. + */ + vfio_pci_recovery_rom_disable(vdev); + + if (!(vdev->pci_recovery_flags & VFIO_PCI_RECOVERY_IN_PROGRESS)) + goto out_unlock; + + if (!vdev->pci_recovery_device_open) { + vdev->pci_recovery_command_valid =3D false; + WRITE_ONCE(vdev->pci_recovery_flags, + vdev->pci_recovery_flags & + ~VFIO_PCI_RECOVERY_IN_PROGRESS); + goto out_unlock; + } + + down_write(&vdev->memory_lock); + /* + * Restore the command word and clear access_blocked under irqlock. + * The INTx handler writes the same register through + * pci_check_and_mask_intx(), so it must not interleave with the + * restore, and it must not see access blocked cleared while the + * temporary command value is still installed. + * + * INTX_DISABLE comes from the INTx state rather than from the saved + * word, which can be older than the last mask. The replay below is + * what unmasks the line. + */ + spin_lock_irqsave(&vdev->irqlock, irq_flags); + if (!(vdev->pci_recovery_flags & VFIO_PCI_RECOVERY_RESET) && + vdev->pci_recovery_command_valid) { + u16 cmd =3D vdev->pci_recovery_command; + + cmd =3D vfio_pci_intx_recovery_command(vdev, cmd); + ret =3D pci_write_config_word(pdev, PCI_COMMAND, cmd); + } + if (!ret) + WRITE_ONCE(vdev->pci_recovery_access_blocked, false); + spin_unlock_irqrestore(&vdev->irqlock, irq_flags); + if (!ret && __vfio_pci_memory_enabled(vdev)) + vfio_pci_dma_buf_move(vdev, false); + up_write(&vdev->memory_lock); + + vdev->pci_recovery_command_valid =3D false; + flags =3D vdev->pci_recovery_flags & ~VFIO_PCI_RECOVERY_IN_PROGRESS; + if (ret) + flags |=3D VFIO_PCI_RECOVERY_FAILED; + WRITE_ONCE(vdev->pci_recovery_flags, flags); + if (!ret) + vfio_pci_intx_recovery_finish(vdev); + +out_unlock: + up_write(&vdev->recovery_lock); + wake_up_all(&vdev->pci_recovery_wait); +} + int vfio_pci_core_sriov_configure(struct vfio_pci_core_device *vdev, int nr_virtfn) { @@ -2901,6 +2968,7 @@ EXPORT_SYMBOL_GPL(vfio_pci_core_sriov_configure); const struct pci_error_handlers vfio_pci_core_err_handlers =3D { .error_detected =3D vfio_pci_core_aer_err_detected, .slot_reset =3D vfio_pci_core_aer_slot_reset, + .resume =3D vfio_pci_core_aer_resume, }; EXPORT_SYMBOL_GPL(vfio_pci_core_err_handlers); =20 --=20 2.43.0 From nobody Sat Sep 26 13:08:18 2026 Received: from CO1PR03CU002.outbound.protection.outlook.com (mail-westus2azon11010028.outbound.protection.outlook.com [52.101.46.28]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 44E3736B926; Tue, 1 Sep 2026 09:34:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.46.28 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788255316; cv=fail; b=Zoepe/A5xzuvrBXZGq+oTuL6/b+qruKoMIc8cZjQjGvtIytbudEhd1Bigg/xBrkpv/B71H0oFUUU17QF6ocOBSNFtDt6+Fl3h3vS6QcyZIxSW/O/QG3sJDIzz1hVvhFfRYAN3mloTqK+cj+38FOdfROQT4ybLMNNBqXwGK/nMiY= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788255316; c=relaxed/simple; bh=cdTjF/dLO5TBEbaoYbpK+4WBjzOGDJaXlZLeezYJBC0=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=CuAaEyyr/hpRmwVxqILGpPybRsuhbxkkqgPJnHZz8xEEjJjDghFaYwe3Hk1Of7ExN+DqawvjGXLXdTJtkeJjFnu8Pokenpr+F8TEdsaYwd2VW4+gJBqWstCYqBKmI61LD+SkZpBs7dkM04vNfZLQhLKYQuGUdyTrse6QBIl5pgo= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=HqPJ0+n8; arc=fail smtp.client-ip=52.101.46.28 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="HqPJ0+n8" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=G/PKQK9UyVXeU3q2qzRMCEZJSAWw7JDyZKw+S9Fs+P1GbGRp7vI2zSFj1oHe4S9RouJLSzqb4/JwzHsnXjIjCZcH+GlgmGPG1zyfP3QWHi6MGWYG+kuErhVggQnuz80ijgJZ6eg8puE7lQy+WoO7q992tzNMUkvLb/YruS/kgYdTeLTjluqI+ER+uUpxZXrtOWMOUfRnE5fnw9swDYZo5vB8ZxJ1MbYgIRC07wqZLeJE+OxyyM6zwPHxPtehLG7EJlMrRO+fHVlcx1jnit1sPKclRx3sRMI4G8BylhSQA+ewc7Vr1z8oJ0L0xQqYXY3p6hTZxQawfGd5kbHz0yCKmw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Lo3lIu0Bg8ETd8dcbT7huGOHg8uGmD/rKWn6YJrzki8=; b=bTHZqqk90thRODhRRXeCqZnMfJZZ2hLYZ5x7r5rrOhvppPrP6HDZG5RcQDdpBWucucvKmUMq80+KQksig7f01Tr5SHDuEKuFFG49biLyphwK5g6OUnYbDacPPLzLx0Wb+jL0mIzopPDX23hJhXnAYI7xx9WWXX+xWF2KZjukn5/O9omAsDowiCsCOIGRreJdIjKygUq2kaTa26JuQCz3kBotvpWbAmu8bGAGKdDq7HnhIkFqwoZMwbpIpjwqosIIhIEkaiuTjSymtHwcEwVhYnMmF4H0S2OCHbT4/oaYRvgZwzjJbbvVKSI32IHuAYLL5E/PlG7UKtdrdH7+BT+94Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Lo3lIu0Bg8ETd8dcbT7huGOHg8uGmD/rKWn6YJrzki8=; b=HqPJ0+n8epgbNZzvpY2XlWzfWjcqqMcGn2tgaxy2q31BZX6CndC1LJOmbqhb55DtjmEJQvr7pZuWYBxJiVT1eMBDAZUlsPqTnmlwcWHfc2KNpMixZw9fIEEGOK8s1wVqtL2V24adrRIYTxWr8fqFU+7ipjXlwbhwteeedYUJoHrIjErE7Rk5QSU71Zt+/l4a25mEk7nYrj7wag3EknmmPMUHzMBv0aJdWjSnTLPnkF2cEZKlxPStmeMRqzk6FQWt8LsNhn3OBS4xrXXHRtPUb5zIICRuQWGmGSWy7RoVJEtnMxJ2CLmhJZYltc/z6gqJ2g3/jgLUifPm3NkMuEN57Q== Received: from CH2PR14CA0022.namprd14.prod.outlook.com (2603:10b6:610:60::32) by BY5PR12MB4049.namprd12.prod.outlook.com (2603:10b6:a03:201::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.360.13; Tue, 1 Sep 2026 09:34:42 +0000 Received: from CH2PEPF00000143.namprd02.prod.outlook.com (2603:10b6:610:60:cafe::28) by CH2PR14CA0022.outlook.office365.com (2603:10b6:610:60::32) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.382.10 via Frontend Transport; Tue, 1 Sep 2026 09:34:41 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by CH2PEPF00000143.mail.protection.outlook.com (10.167.244.100) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.382.8 via Frontend Transport; Tue, 1 Sep 2026 09:34:41 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.46; Tue, 1 Sep 2026 02:34:24 -0700 Received: from NV-2Y5XW94.nvidia.com (10.126.230.37) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.46; Tue, 1 Sep 2026 02:34:21 -0700 From: Shameer Kolothum To: , , CC: , , , , , , , , Subject: [RFC PATCH 18/19] vfio/pci: Coordinate generic device access with host recovery Date: Tue, 1 Sep 2026 10:32:16 +0100 Message-ID: <20260901093217.8539-19-skolothumtho@nvidia.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260901093217.8539-1-skolothumtho@nvidia.com> References: <20260901093217.8539-1-skolothumtho@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: rnnvmail203.nvidia.com (10.129.68.9) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PEPF00000143:EE_|BY5PR12MB4049:EE_ X-MS-Office365-Filtering-Correlation-Id: f5fbcbaa-19ba-42ea-ef6b-08df080c402e X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|82310400026|36860700016|23010399003|376014|6133799003|22082099003|18002099003|11063799006|56012099006|5023799004|10067099003; X-Microsoft-Antispam-Message-Info: HLTOKpQMD+xMqSxpCOnVUE3XbxerICdyStZSByeHVRbv8beu/yF+5VltqjlwLplg1HR261gel22/cORrVvaZYww+6+uJF23qk6BHEzpuXQel8X64qAs1tJK/S+TIu+yo1V4RVUhrQNghq3igNLY1yLxeRPvIjB89Bj9hDA4todsfgjybwULbHG/NBqXRqp56O3PWraqYvjLsoVWN6ovLv0t2yF5AM911gYVGvn5RexqntCvbgo0JifG69MWnhsj1V2hxxmgbtq9s0iN3ixs8MUKyAxVO6aMvQhjOg4Q5179BZynR5GgGhTr96/Ovt5b2P/YilWAIjo/Stj6TKv1VRXYeCXHHvvWEWj71Cea7mbvWS7G9i4Fpy0oGjlonFvpmMyOyvrpEQ0DkfN9o3LN8BfmQg6vUoE1EDMjK4wF2zSiKZYJvDsEPAMMD1/TrMlHUJPz0ki7GG5EBi79iJ+EcQFeqDIYmGV1qIa+0WuIkhlP4prFWiv10ceiha+wRqzTB2kvaba8zrmpPS/XlXJWjPztwQTXpmfy+UUjQBftNnv1l0KiGhU5N/sL4XcmuHybjWDZUxHYFr8IcT2ivKUtOKrybN7ir8vMZs5QZh2ZJD1rnYhmh60AMqhzb7yJ/CDA5VrYOlSKsYSlqOi8DON0itXCaBsdh4uDrdOpv7KvOdWYacMPU/rqybUy9SVs+JNSSt+a5dIYB57NOQ1AMw4e+ug== X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(1800799024)(82310400026)(36860700016)(23010399003)(376014)(6133799003)(22082099003)(18002099003)(11063799006)(56012099006)(5023799004)(10067099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: mvbljsNGIz5drrL33bU5mn0wwkIPzl3Ueot91gBmgXEo/TqEnKUX0AhB4ATuhZq1qdqBz0H5FXBiVJ0ox635bwzgh9zOl6+37Z8C7KpzzStqqotVahkpze7NQfjlmQA1r6wAOgHRsXUQAo/blXIaLRToC6hxmvY90q0o97mhNOBX7y30Iw7l8eng1yxI307emcgk0wHwamNYLGklnQRYs2kOrqS/9MvvFnRvyAv6YGqM5EIQpdn+1nTWGO956hEpdSO7S/MOIg8LUvnfrtEAX3ziNso2ve8b/Srja7nIXe4Ilpbhyzh3cbPf/NLqh04KVgiFs+XmKXfzXBTHDFV1PbitapuBIw2ECmOw48OhtorrhVCHRNYosLLRxNDElp7NWd7IiOMthYOhCZOOzF6N4SesVckcglSg7Lwen+W+lviWLAxikkCNY8LraWdERmU8 X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Sep 2026 09:34:41.7708 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: f5fbcbaa-19ba-42ea-ef6b-08df080c402e X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CH2PEPF00000143.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR12MB4049 Content-Type: text/plain; charset="utf-8" Wire up error_detected() for generic vfio-pci devices whose user has enabled recovery. Devices which have not opted in, and variant drivers, keep the existing signal-only behaviour. Block new device access and drain what is already running, then revoke BAR mappings, revoke exported DMA-BUFs and stop bus mastering, so nothing touches the device while the host recovers it. A non-fatal error gets the same treatment as a frozen one. The host has not finished deciding what the error was, and can still escalate to a reset, so the device is not the user's again until resume() says so. Do not trust a command word which reads as all ones. A device which has stopped responding still returns success, and writing that value back would set every command bit while saving it would restore them at the end. Treat it as a config access failure instead. Quiesce INTx first. For a device with per-function masking, also save PCI_COMMAND and write it back with INTX_DISABLE set and bus mastering cleared, under irqlock so an interrupt handler cannot interleave. Publish the state in one store. IN_PROGRESS and CHANNEL_FROZEN go out together so a lock-free reader cannot see an event which is in progress but not yet marked frozen. FAILED stays set until the device is closed and reopened. A frozen channel votes NEED_RESET. A config access failure of our own votes NONE, which leaves the rest of the recovery domain alone. Only a permanent channel failure reported to us votes DISCONNECT. If a ROM unmap raced the blocked interval, its config write is left for resume() to complete. A second event which arrives before resume() has finished the first joins the transaction already running. It keeps the sequence number, the command word saved before the device was quiesced, and any reset a slot_reset() in between recorded. Starting again would save the quiesced command word and restore a device with bus mastering off, and would drop the record of a reset the host had already performed. An event which arrives while a VFIO_DEVICE_RESET has access blocked runs as usual. The PCI core calls this with the device lock held, which pci_try_reset_function() also takes, so the two cannot overlap the reset itself, and the reset leaves the state alone once this has claimed it. Suppressing the event instead would lose a permanent failure or a bus reset the host went on to perform, which is the state userspace most needs. Signed-off-by: Shameer Kolothum --- drivers/vfio/pci/vfio_pci_core.c | 166 ++++++++++++++++++++++++++++++- 1 file changed, 165 insertions(+), 1 deletion(-) diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_c= ore.c index eed0430c32ee..2d757d6a5fe1 100644 --- a/drivers/vfio/pci/vfio_pci_core.c +++ b/drivers/vfio/pci/vfio_pci_core.c @@ -2746,14 +2746,178 @@ pci_ers_result_t vfio_pci_core_aer_err_detected(st= ruct pci_dev *pdev, { struct vfio_pci_core_device *vdev =3D dev_get_drvdata(&pdev->dev); struct vfio_pci_eventfd *eventfd; + pci_ers_result_t result =3D PCI_ERS_RESULT_CAN_RECOVER; + unsigned long irq_flags; + bool terminal =3D false; + bool nested; + u32 flags; + int ret; + + if (!vdev->pci_recovery_supported || + !READ_ONCE(vdev->pci_recovery_enabled)) + goto out; + + down_write(&vdev->recovery_lock); + if (!vdev->pci_recovery_enabled) + goto out_unlock; + + /* + * A failed device remains blocked until close and a new open have + * reinitialized it. A later bridge event cannot make the saved VFIO + * state valid again. + */ + if (vdev->pci_recovery_flags & VFIO_PCI_RECOVERY_FAILED) { + result =3D PCI_ERS_RESULT_NONE; + goto out_unlock; + } + + if (!vdev->pci_recovery_device_open) { + result =3D PCI_ERS_RESULT_NONE; + /* + * PCI core rebroadcasts permanent failure when subtree + * recovery fails. Complete an event which started before + * close so a later open is not permanently stuck on + * IN_PROGRESS. + */ + if (state =3D=3D pci_channel_io_perm_failure && + (vdev->pci_recovery_flags & + VFIO_PCI_RECOVERY_IN_PROGRESS)) { + WRITE_ONCE(vdev->pci_recovery_flags, + (vdev->pci_recovery_flags | + VFIO_PCI_RECOVERY_FAILED) & + ~VFIO_PCI_RECOVERY_IN_PROGRESS); + vdev->pci_recovery_command_valid =3D false; + terminal =3D true; + } + goto out_unlock; + } + + WRITE_ONCE(vdev->pci_recovery_access_blocked, true); + /* + * A second event before resume() has finished the first joins the + * transaction already running rather than starting one. Keep its + * sequence number, the command word it saved before the device was + * quiesced, and any reset a slot_reset() in between recorded. Reading + * the command word again here would save the quiesced value, and + * restoring that leaves the device with bus mastering off. + */ + nested =3D vdev->pci_recovery_flags & VFIO_PCI_RECOVERY_IN_PROGRESS; + if (!nested) + vdev->pci_recovery_command_valid =3D false; + vfio_pci_intx_recovery_start(vdev); + /* + * INTx hardirq and virqfd callbacks cannot take recovery_lock. + * For devices with per-function INTx masking, mask INTx while holding + * irqlock so a callback which passed its blocked-state check is drained + * before the temporary command value is installed. Devices without + * per-function masking were quiesced above through genirq. + */ + spin_lock_irqsave(&vdev->irqlock, irq_flags); + ret =3D 0; + if (state =3D=3D pci_channel_io_normal && vdev->pci_2_3 && !nested) { + u16 command; =20 + ret =3D pci_read_config_word(pdev, PCI_COMMAND, + &vdev->pci_recovery_command); + /* + * A read from a device which has stopped responding succeeds + * and returns all ones. Writing that back would set every + * command bit, and saving it would restore them at the end. + */ + if (!ret && PCI_POSSIBLE_ERROR(vdev->pci_recovery_command)) + ret =3D -EIO; + if (!ret) { + command =3D (vdev->pci_recovery_command & + ~PCI_COMMAND_MASTER) | + PCI_COMMAND_INTX_DISABLE; + ret =3D pci_write_config_word(pdev, PCI_COMMAND, command); + } + if (!ret) + vdev->pci_recovery_command_valid =3D true; + } + spin_unlock_irqrestore(&vdev->irqlock, irq_flags); + vfio_pci_zap_and_down_write_memory_lock(vdev); + vfio_pci_dma_buf_move(vdev, true); + + /* + * Allocate a sequence for a new transaction, and drop the flags the + * previous one left behind for userspace to read. A nested event adds + * to the flags already there. Each path below publishes the result in + * one store, so a lock-free reader never observes a cleared state that + * looks like successful completion. + */ + flags =3D vdev->pci_recovery_flags; + if (!nested) { + if (++vdev->pci_recovery_sequence =3D=3D 0) + vdev->pci_recovery_sequence++; + flags =3D 0; + } + + if (state =3D=3D pci_channel_io_perm_failure) { + WRITE_ONCE(vdev->pci_recovery_flags, + (flags | VFIO_PCI_RECOVERY_FAILED) & + ~VFIO_PCI_RECOVERY_IN_PROGRESS); + vdev->pci_recovery_command_valid =3D false; + result =3D PCI_ERS_RESULT_DISCONNECT; + terminal =3D true; + goto out_memory; + } + + if (state =3D=3D pci_channel_io_frozen) { + WRITE_ONCE(vdev->pci_recovery_flags, + flags | VFIO_PCI_RECOVERY_IN_PROGRESS | + VFIO_PCI_RECOVERY_FROZEN); + result =3D PCI_ERS_RESULT_NEED_RESET; + goto out_memory; + } + + WRITE_ONCE(vdev->pci_recovery_flags, + flags | VFIO_PCI_RECOVERY_IN_PROGRESS); + if (ret) + goto out_failed; + if (vdev->pci_2_3 || nested) + goto out_memory; + + ret =3D pci_read_config_word(pdev, PCI_COMMAND, + &vdev->pci_recovery_command); + if (ret) + goto out_failed; + + if (PCI_POSSIBLE_ERROR(vdev->pci_recovery_command)) { + ret =3D -EIO; + goto out_failed; + } + + ret =3D pci_write_config_word(pdev, PCI_COMMAND, + vdev->pci_recovery_command & + ~PCI_COMMAND_MASTER); + if (ret) + goto out_failed; + + vdev->pci_recovery_command_valid =3D true; + goto out_memory; + +out_failed: + WRITE_ONCE(vdev->pci_recovery_flags, + (vdev->pci_recovery_flags | VFIO_PCI_RECOVERY_FAILED) & + ~VFIO_PCI_RECOVERY_IN_PROGRESS); + result =3D PCI_ERS_RESULT_NONE; + terminal =3D true; +out_memory: + up_write(&vdev->memory_lock); +out_unlock: + up_write(&vdev->recovery_lock); + if (terminal) + wake_up_all(&vdev->pci_recovery_wait); + +out: rcu_read_lock(); eventfd =3D rcu_dereference(vdev->err_trigger); if (eventfd) eventfd_signal(eventfd->ctx); rcu_read_unlock(); =20 - return PCI_ERS_RESULT_CAN_RECOVER; + return result; } EXPORT_SYMBOL_GPL(vfio_pci_core_aer_err_detected); =20 --=20 2.43.0 From nobody Sat Sep 26 13:08:18 2026 Received: from BYAPR05CU005.outbound.protection.outlook.com (mail-westusazon11010041.outbound.protection.outlook.com [52.101.85.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8BA02480DFF; Tue, 1 Sep 2026 09:34:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.85.41 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788255309; cv=fail; b=CFRQ3Axr4SDwS/YvqduWyPBhNuvQV1nRwpuZqXVjY00aei4EiQaejaKM7uxwnJZ4eaJZDFbttYxFsn4LlntB2GKQveP/q6slHGbnmTDNb2kRu89J3cTq3C9MX+SYxe6M9p2Ijn5JuyJdhnChCb5rkDMt2v3ynAUWxxMgvIE3RXM= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788255309; c=relaxed/simple; bh=FKvjy1rkAMcuzC3haH78swvWaCzTUK37/iHh3yhJVKU=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=X4wCHluo0BCy0BeYyvU339qhMuI7/YKBpfoIyZz4r4RfNiKgEdIQ0KydkJ93v2Z+j+jinkTZ0At/oiwfnALT/G8T5SEeG5UzcaZWcLthDCI3U4wZv/6YTkRbt9T+ibMzgFzwkNg2iH6kryxv3fv08GKYynhcV+ZniP4s4LGJMQw= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=hNvfI8Aq; arc=fail smtp.client-ip=52.101.85.41 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="hNvfI8Aq" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=r3ncKX2HU87asxeMWgBWdOBkEKMSBO0SsKtA124+Qyw72pHlD7CqC9Odk/zL7VTXHESVPT0vmERIzwyWDIAzjSHc099bkd2bHEiKuiFogZCIbd7GC+sh4EA9tXeaO9uujvLHrMOjSczMJk1NPtIrDLDYHh6c1IWW5i6mfv727lkxOgTtIn0R7LU/66fd/Q8wL4VpHuaQTcXMuIyxdzTh0LSr3WqM3ySOtIHl32OxdO+d8ztd/GuVek4zkHTPcd9XT7F6+8Hydil7JctHvpsZrKBOCDoNpOhdui0v3qeHk1VbpUKmwEeumAerby4jIXW10oUAQNqRVB5CbpMpOEFP9w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=njwtbRLpn5JR/bgVUvsRTV8LDJL0nncm0P3WMKYkRYY=; b=Kxr1jnFCy+VIygChJb1fftrkZqyQwcGVN2XBq8l/H4aec9eXsQaQT5bUDDXQ90R0FQr+QBmL4mQ0Y/1AR3mTw8TxFImUDWSQS3BRi4FeNkK/Al2Yxp0gfYShtYZy269PicKJmp9P+3ypvQQRGwU/jWFZQeIaHufXzGd3kZutLTddI0JezwMH4kWxt2skMjGOj1oovtX9SvkSgywB/tyTgV0SJYI4I2durcKTcmljrh19mTKewpAdRiyzrUBHULZGCDqqKgItJx9CDAnwv/9jF0z+uw3qvO0RyIU5jI0sohkmFwUnUQ5SRGS5gYSBffOKEUuSdgwfYWopvrCBQdB+sg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=njwtbRLpn5JR/bgVUvsRTV8LDJL0nncm0P3WMKYkRYY=; b=hNvfI8Aqr8aTN048nR+y2n8biM3u3WPyqsxO3timWQdQKejBhYQFE/5effo2/0cStUUSxopWmvZOBmvv++sTBVyxLmeWXV7SPdy7srW8etDwr8ezKDTPNVq871lTddOoPxfHaTd7dIVrrnaCWetW4++fx/mWTFdM7hVeSXJmT2SLWVNWSSzD/t3GzRcZknxbrMO9k+ALQ3P+XUalilp9wLKh5G652sPLm3EVdUxyp0ct7yRYlMCtcz6NLVIqaTo2gdmsSj1NSZwoxy/lFrhJDCg8b3VW4TlShQahw+pFHuqysjqimgnSop31g7hJaDGWLFfHOXAAL2RejC8LD/NRoA== Received: from CH2PR14CA0012.namprd14.prod.outlook.com (2603:10b6:610:60::22) by CY8PR12MB7434.namprd12.prod.outlook.com (2603:10b6:930:52::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.360.11; Tue, 1 Sep 2026 09:34:45 +0000 Received: from CH2PEPF00000143.namprd02.prod.outlook.com (2603:10b6:610:60:cafe::4f) by CH2PR14CA0012.outlook.office365.com (2603:10b6:610:60::22) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.382.10 via Frontend Transport; Tue, 1 Sep 2026 09:34:45 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by CH2PEPF00000143.mail.protection.outlook.com (10.167.244.100) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.382.8 via Frontend Transport; Tue, 1 Sep 2026 09:34:45 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.46; Tue, 1 Sep 2026 02:34:28 -0700 Received: from NV-2Y5XW94.nvidia.com (10.126.230.37) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.46; Tue, 1 Sep 2026 02:34:25 -0700 From: Shameer Kolothum To: , , CC: , , , , , , , , Subject: [RFC PATCH 19/19] vfio/pci: Expose and enable host PCI error recovery Date: Tue, 1 Sep 2026 10:32:17 +0100 Message-ID: <20260901093217.8539-20-skolothumtho@nvidia.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260901093217.8539-1-skolothumtho@nvidia.com> References: <20260901093217.8539-1-skolothumtho@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: rnnvmail203.nvidia.com (10.129.68.9) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PEPF00000143:EE_|CY8PR12MB7434:EE_ X-MS-Office365-Filtering-Correlation-Id: a1638f25-658e-4da7-5f36-08df080c423e X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|23010399003|36860700016|82310400026|376014|1800799024|6133799003|56012099006|10067099003|5023799004|11063799006|3023799007|22082099003|18002099003; X-Microsoft-Antispam-Message-Info: V41/9uHQkAkE+Dqr8oxqk+KP0e90dGPkCUNPlDbypC/+Kg4pz13vvVeT3Otx/QF5ny/FbwSkZ0qlww6SCbL0NwMFrguzpKkOjxKHtd+Z5YDk5Jqj6zPJljrd8JKePF9bU3J+KqsX4yaQMzWYccVeS93FzzqSBPQrG6Py4lMDtk/RJVj5Z0pMDOvP3nwsmuYytyvi9+w8nEHD4FpBD1LMyukX2HCHMJpeJBiyfduixdoMqP0iPOcxubZcBQPZWuay9qhrnwG1Gp0D98HQE659U3h+9nGqp5DXZh9f5fKjsSPW3yCJOKy5CyQ2sa7YL0SAQ9tL3RJLZGpPzj4xaBB7TF89LUcYi72zfdp9azLjL4U1TMH3XkESI9BxYtgLuEh8AtpIQ3smoY8ePSlLMFgyYOfWGq52mTskj+BEKGpxPbz7ORf5siev8v6hdmZFjSwg+xV3l6j+CeTL/QutKHr1xsmNBEEzkEaRZULifjbH0tjspfRgz/2NIa0aO4j6dnn9rpYMn5asc/Vo1Hyh0G3CvGtRvQhwv3YSuNN2bq1pNeWXL+U0v/vq3aBz0LNro0AugTFMkccs+WeiVCUQyQMl4HuuKDC48D1SurEVTlUCXorK9IQIpZOq9fuq0vgTTxbTRCFtlHpFBbgOQyjzmuQb7GwbXRsFtwerIwKs4gbhSS+YD7BSimK1bSzvq6R/yRu4LaoE4HxlxSRWevFwTt/wew== X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(23010399003)(36860700016)(82310400026)(376014)(1800799024)(6133799003)(56012099006)(10067099003)(5023799004)(11063799006)(3023799007)(22082099003)(18002099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: Va0jmcEXq1tA2mqSI35gZrQwC8byxphvKxvQ0qilSjoigvh2YIh8GXMKu5kc7moIzpoTc1ctiltJEi8/lh+h7Urjw2PjZVeBXLfBCqrH/H3yGImIN6p3oIEN0QXiLtDyi6J8vcLHTiUzLcSspPDaM8GCMMu00ZxPQkJSpQZrLFAL/UIc0c1MjqhDIA2IqCyyL6rZxbMguOknvZvjDjkJOGiJIrMUp6kcLwobdnG1+Tn7dN+svBD1mZHqgZ9E5aBREGJT09fmu3+LoyGQ/alDmDl4NEk/TwVgbvV9nYGRtGzHU7hK5Cwt9kLzb6Q3wmqvupjoHaGlzwG1q0Og+9yOxV7fzO2MjRp5vbuzUnIB8nw7yxCJNl8iJz0n18JPxekt6QP0Xj0jvel0dBc97bGR7fR8o8u9CzDgO9V5sSFUKfJasVqqn94ULWbZXe5Jbi+x X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Sep 2026 09:34:45.2640 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: a1638f25-658e-4da7-5f36-08df080c423e X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CH2PEPF00000143.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR12MB7434 Content-Type: text/plain; charset="utf-8" Add a VFIO device feature that reports whether host PCI recovery is enabled, whether recovery is in progress, whether the channel was frozen, whether the host reset the device, and whether recovery failed. A sequence number lets userspace distinguish events. Installing a dedicated recovery eventfd enables recovery. The eventfd is additional to VFIO_PCI_ERR_IRQ_INDEX, which keeps reporting errors as it does today either way. Recovery can be disabled only when not in progress and before terminal failure. Reject both enable and disable while device access is blocked so feature changes cannot race lifecycle teardown or an explicit reset. Clear the sequence number and the status bits whichever way the feature is being changed, so a sequence number always describes an event the eventfd holding it was notified of. A user which replaces the eventfd would otherwise read status for an event it never heard about. Variant drivers return -ENOTTY as they do not advertise recovery support. Signed-off-by: Shameer Kolothum --- include/linux/vfio_pci_core.h | 1 + include/uapi/linux/vfio.h | 69 +++++++++++++++ drivers/vfio/pci/vfio_pci_core.c | 142 +++++++++++++++++++++++++++++-- 3 files changed, 206 insertions(+), 6 deletions(-) diff --git a/include/linux/vfio_pci_core.h b/include/linux/vfio_pci_core.h index 42a77ed6b93c..fe42089be3fc 100644 --- a/include/linux/vfio_pci_core.h +++ b/include/linux/vfio_pci_core.h @@ -145,6 +145,7 @@ struct vfio_pci_core_device { int ioeventfds_nr; struct vfio_pci_eventfd __rcu *err_trigger; struct vfio_pci_eventfd __rcu *req_trigger; + struct vfio_pci_eventfd __rcu *pci_recovery_trigger; struct eventfd_ctx *pm_wake_eventfd_ctx; struct list_head dummy_resources_list; struct mutex ioeventfds_lock; diff --git a/include/uapi/linux/vfio.h b/include/uapi/linux/vfio.h index e41437fa17ad..ce0cda2dcbbb 100644 --- a/include/uapi/linux/vfio.h +++ b/include/uapi/linux/vfio.h @@ -1555,6 +1555,75 @@ struct vfio_device_feature_zpci_err { =20 #define VFIO_DEVICE_FEATURE_ZPCI_ERROR 13 =20 +/* + * Report host PCI error recovery state for this device. + * + * The sequence number is incremented at the start of each event and remai= ns + * unchanged for its subsequent state changes. Userspace can therefore + * distinguish a new event from completion of the current one and detect + * coalesced notifications. It restarts from zero each time recovery is + * enabled, so it is only meaningful within one enabled period. + * + * ENABLED reports that userspace has enabled recovery. + * CHANNEL_FROZEN records that recovery started with the PCI channel froze= n. + * DEVICE_RESET records that the host reset the device. FAILED records that + * recovery did not complete successfully. Event status bits remain set af= ter + * IN_PROGRESS is cleared. A new event supersedes status from a previous + * successful event. FAILED is terminal for the current device open and + * remains set until the device is closed and reopened. + * + * Status bits may also be set while IN_PROGRESS is still set, describing = the + * event so far. Act on them once IN_PROGRESS is clear. Device access is + * refused with -EIO until then. + * + * When DEVICE_RESET is reported the host reset the device, which tears do= wn + * the interrupt configuration the user had established. INTx, MSI and MSI= -X + * must be re-armed with VFIO_DEVICE_SET_IRQS before interrupts resume. + * + * VFIO_DEVICE_FEATURE_GET returns the current state and -1 in eventfd. GE= T is + * never refused, including while recovery blocks device access, so that + * userspace can read this state during an event. It can wait for a recove= ry + * callback which is already running. + * + * IN_PROGRESS is not guaranteed to be observable. A recovery which needs = no + * device reset can complete within microseconds of the notification, befo= re + * userspace is scheduled, so a GET which follows the eventfd may already = see + * IN_PROGRESS clear. Userspace must treat a notification as "an event + * occurred" and read the sequence number and the status bits to learn what + * happened. It must not wait for IN_PROGRESS to appear set. + * + * VFIO_DEVICE_FEATURE_SET with a valid eventfd enables recovery + * and installs the eventfd as a notification for recovery start and termi= nal + * completion. SET with eventfd -1 disables recovery when none + * is in progress and the latest event has not failed. SET returns -EBUSY = when + * any of those restrictions prevents the requested transition, including = while + * an explicit VFIO_DEVICE_RESET blocks device access, and -ENODEV if devi= ce + * close has begun. flags and sequence must be zero for SET. + * + * This eventfd is separate from VFIO_PCI_ERR_IRQ_INDEX and additional to = it. + * VFIO_PCI_ERR_IRQ_INDEX keeps reporting errors as it does today whether = or + * not this feature is enabled, so a user of both receives two notificatio= ns + * for one event. + * + * Enabling recovery does not recover an event which is already being hand= led + * for this device. Such an event was declined before it started, so it + * completes without notification and without status, even though the host= may + * reset the device as part of it. Enable recovery before errors occur rat= her + * than in response to one. + */ +struct vfio_device_pci_error_recovery { + __u32 flags; +#define VFIO_PCI_ERROR_RECOVERY_IN_PROGRESS (1U << 0) +#define VFIO_PCI_ERROR_RECOVERY_CHANNEL_FROZEN (1U << 1) +#define VFIO_PCI_ERROR_RECOVERY_DEVICE_RESET (1U << 2) +#define VFIO_PCI_ERROR_RECOVERY_FAILED (1U << 3) +#define VFIO_PCI_ERROR_RECOVERY_ENABLED (1U << 4) + __s32 eventfd; + __aligned_u64 sequence; +}; + +#define VFIO_DEVICE_FEATURE_PCI_ERROR_RECOVERY 14 + /* -------- API for Type1 VFIO IOMMU -------- */ =20 /** diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_c= ore.c index 2d757d6a5fe1..c1ea3c868fc5 100644 --- a/drivers/vfio/pci/vfio_pci_core.c +++ b/drivers/vfio/pci/vfio_pci_core.c @@ -893,6 +893,10 @@ static void vfio_pci_core_finish_close(struct vfio_dev= ice *core_vdev) mutex_lock(&vdev->igate); vfio_pci_eventfd_replace_locked(vdev, &vdev->err_trigger, NULL); vfio_pci_eventfd_replace_locked(vdev, &vdev->req_trigger, NULL); + if (vdev->pci_recovery_supported) + vfio_pci_eventfd_replace_locked(vdev, + &vdev->pci_recovery_trigger, + NULL); mutex_unlock(&vdev->igate); } =20 @@ -1850,6 +1854,106 @@ static int vfio_pci_core_feature_token(struct vfio_= pci_core_device *vdev, return 0; } =20 +static int +vfio_pci_core_feature_error_recovery(struct vfio_pci_core_device *vdev, u3= 2 flags, + struct vfio_device_pci_error_recovery __user *arg, + size_t argsz) +{ + struct vfio_device_pci_error_recovery state =3D { .eventfd =3D -1 }; + struct eventfd_ctx *ctx =3D NULL; + bool enable; + int ret; + + if (!vdev->pci_recovery_supported) + return -ENOTTY; + + ret =3D vfio_check_feature(flags, argsz, + VFIO_DEVICE_FEATURE_GET | + VFIO_DEVICE_FEATURE_SET, sizeof(state)); + if (ret !=3D 1) + return ret; + + if (flags & VFIO_DEVICE_FEATURE_GET) { + down_read(&vdev->recovery_lock); + if (vdev->pci_recovery_enabled) + state.flags |=3D VFIO_PCI_ERROR_RECOVERY_ENABLED; + if (vdev->pci_recovery_flags & VFIO_PCI_RECOVERY_IN_PROGRESS) + state.flags |=3D VFIO_PCI_ERROR_RECOVERY_IN_PROGRESS; + if (vdev->pci_recovery_flags & VFIO_PCI_RECOVERY_FROZEN) + state.flags |=3D + VFIO_PCI_ERROR_RECOVERY_CHANNEL_FROZEN; + if (vdev->pci_recovery_flags & VFIO_PCI_RECOVERY_RESET) + state.flags |=3D VFIO_PCI_ERROR_RECOVERY_DEVICE_RESET; + if (vdev->pci_recovery_flags & VFIO_PCI_RECOVERY_FAILED) + state.flags |=3D VFIO_PCI_ERROR_RECOVERY_FAILED; + state.sequence =3D vdev->pci_recovery_sequence; + up_read(&vdev->recovery_lock); + + if (copy_to_user(arg, &state, sizeof(state))) + return -EFAULT; + return 0; + } + + if (copy_from_user(&state, arg, sizeof(state))) + return -EFAULT; + if (state.flags || state.sequence || state.eventfd < -1) + return -EINVAL; + + enable =3D state.eventfd >=3D 0; + if (enable) { + ctx =3D eventfd_ctx_fdget(state.eventfd); + if (IS_ERR(ctx)) + return PTR_ERR(ctx); + } + + down_write(&vdev->recovery_lock); + if (!vdev->pci_recovery_device_open) { + ret =3D -ENODEV; + goto out_unlock; + } + if (vdev->pci_recovery_access_blocked) { + ret =3D -EBUSY; + goto out_unlock; + } + + if (!enable && + (vdev->pci_recovery_flags & + (VFIO_PCI_RECOVERY_IN_PROGRESS | VFIO_PCI_RECOVERY_FAILED))) { + ret =3D -EBUSY; + goto out_unlock; + } + + mutex_lock(&vdev->igate); + ret =3D vfio_pci_eventfd_replace_locked(vdev, + &vdev->pci_recovery_trigger, + ctx); + mutex_unlock(&vdev->igate); + if (ret) + goto out_unlock; + + WRITE_ONCE(vdev->pci_recovery_enabled, enable); + /* + * Start each enabled period from a clear state, so a sequence number + * and the status bits beside it always describe an event this + * eventfd was notified of. Nothing is in flight to lose. A + * transaction holds access_blocked, which failed this call with + * -EBUSY above. + * + * access_blocked itself is not cleared here, so userspace can never + * disable its way out of a block. + */ + WRITE_ONCE(vdev->pci_recovery_flags, 0); + vdev->pci_recovery_sequence =3D 0; + vdev->pci_recovery_command_valid =3D false; + +out_unlock: + up_write(&vdev->recovery_lock); + if (ret && ctx) + eventfd_ctx_put(ctx); + + return ret; +} + int vfio_pci_core_ioctl_feature(struct vfio_device *device, u32 flags, void __user *arg, size_t argsz) { @@ -1870,6 +1974,9 @@ int vfio_pci_core_ioctl_feature(struct vfio_device *d= evice, u32 flags, return vfio_pci_core_feature_dma_buf(vdev, flags, arg, argsz); case VFIO_DEVICE_FEATURE_ZPCI_ERROR: return vfio_pci_zdev_feature_err(device, flags, arg, argsz); + + case VFIO_DEVICE_FEATURE_PCI_ERROR_RECOVERY: + return vfio_pci_core_feature_error_recovery(vdev, flags, arg, argsz); default: return -ENOTTY; } @@ -2741,6 +2848,18 @@ void vfio_pci_core_unregister_device(struct vfio_pci= _core_device *vdev) } EXPORT_SYMBOL_GPL(vfio_pci_core_unregister_device); =20 +static void +vfio_pci_signal_recovery_event(struct vfio_pci_core_device *vdev) +{ + struct vfio_pci_eventfd *eventfd; + + rcu_read_lock(); + eventfd =3D rcu_dereference(vdev->pci_recovery_trigger); + if (eventfd) + eventfd_signal(eventfd->ctx); + rcu_read_unlock(); +} + pci_ers_result_t vfio_pci_core_aer_err_detected(struct pci_dev *pdev, pci_channel_state_t state) { @@ -2748,6 +2867,7 @@ pci_ers_result_t vfio_pci_core_aer_err_detected(struc= t pci_dev *pdev, struct vfio_pci_eventfd *eventfd; pci_ers_result_t result =3D PCI_ERS_RESULT_CAN_RECOVER; unsigned long irq_flags; + bool notify_recovery =3D false; bool terminal =3D false; bool nested; u32 flags; @@ -2792,6 +2912,7 @@ pci_ers_result_t vfio_pci_core_aer_err_detected(struc= t pci_dev *pdev, goto out_unlock; } =20 + notify_recovery =3D true; WRITE_ONCE(vdev->pci_recovery_access_blocked, true); /* * A second event before resume() has finished the first joins the @@ -2916,6 +3037,8 @@ pci_ers_result_t vfio_pci_core_aer_err_detected(struc= t pci_dev *pdev, if (eventfd) eventfd_signal(eventfd->ctx); rcu_read_unlock(); + if (notify_recovery) + vfio_pci_signal_recovery_event(vdev); =20 return result; } @@ -2981,13 +3104,15 @@ static pci_ers_result_t vfio_pci_core_aer_slot_rese= t(struct pci_dev *pdev) =20 up_write(&vdev->recovery_lock); /* - * Whoever clears IN_PROGRESS owes the wake. resume() will not do it, - * since it bails once IN_PROGRESS is clear, and the core skips it - * altogether if the domain verdict is not RECOVERED. On success the - * transaction carries on and resume() wakes. + * Whoever clears IN_PROGRESS owes the wake and the event. resume() + * will not do it, since it bails once IN_PROGRESS is clear, and the + * core skips it altogether if the domain verdict is not RECOVERED. + * On success the transaction carries on and resume() does both. */ - if (ret) + if (ret) { wake_up_all(&vdev->pci_recovery_wait); + vfio_pci_signal_recovery_event(vdev); + } =20 return result; } @@ -2996,6 +3121,7 @@ static void vfio_pci_core_aer_resume(struct pci_dev *= pdev) { struct vfio_pci_core_device *vdev =3D dev_get_drvdata(&pdev->dev); unsigned long irq_flags; + bool notify_recovery =3D false; u32 flags; int ret =3D 0; =20 @@ -3011,6 +3137,7 @@ static void vfio_pci_core_aer_resume(struct pci_dev *= pdev) if (!(vdev->pci_recovery_flags & VFIO_PCI_RECOVERY_IN_PROGRESS)) goto out_unlock; =20 + notify_recovery =3D true; if (!vdev->pci_recovery_device_open) { vdev->pci_recovery_command_valid =3D false; WRITE_ONCE(vdev->pci_recovery_flags, @@ -3056,7 +3183,10 @@ static void vfio_pci_core_aer_resume(struct pci_dev = *pdev) =20 out_unlock: up_write(&vdev->recovery_lock); - wake_up_all(&vdev->pci_recovery_wait); + if (notify_recovery) { + wake_up_all(&vdev->pci_recovery_wait); + vfio_pci_signal_recovery_event(vdev); + } } =20 int vfio_pci_core_sriov_configure(struct vfio_pci_core_device *vdev, --=20 2.43.0