From nobody Sat Feb 7 15:12:39 2026 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5EE7D35CB7F for ; Thu, 22 Jan 2026 01:51:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.13 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769046688; cv=none; b=pGn/Y6WQICSwEHQPY6VDivwOC1QQqClXXVaba1Z0NLKqprugHhSkiD1r1HWGr5ooYYJjeD9yQHJk30kY0qAwVDQe2YAlB6Azo2a7k/FohTc9HeROlBxTJTnWU7GK2cMoJ1Imy5oIzMXr6iFZfAOv44VPQ7zcIjU7muEAu2Q34Sk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769046688; c=relaxed/simple; bh=ccjGZDmIa9lrD/sPBBC6A+WacPVxmul231gi/NFW/LY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=a3rfHBftllaZRC8hBdDZDtFpdS689a+KuJmExj/sYInuwvx20uy/WxT7ePlY1CwqfTEpJ0FVp97Bf3Cliy+avfbRNFVcVnoJgVQoUS2LhzqZ5oUCyUDlV507E5D+KWxzhDRZSpbxNoL4Z92UfDXEy0+0MCUZRX6F9sRX5xOVCWY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=pass smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=FUL94RNh; arc=none smtp.client-ip=198.175.65.13 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="FUL94RNh" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1769046686; x=1800582686; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=ccjGZDmIa9lrD/sPBBC6A+WacPVxmul231gi/NFW/LY=; b=FUL94RNhepJnHrtCU6I5Cva8k2blb/YK4ltcF3Ff7/lf3JXOszsx7Iug E2ek7WFXfPdbNlZD2/7cgrwJ5CwYebXtq91n3wITpGhAm94BogXXfV68G CfFBbwrJzVvaxxPCR00Wm+3XBJuz/o/IgtYXyFimIR9TFd5RQxAC1dJe6 J1bFvD5HpGLtnLD2KBaMkrSk2vnUXZbE0m5ipkk9hjYnpohSC65ydvF/p AAq/vLrEfU+41lxzYALSJ5xMLIZqsXtl/T7mIHu3DJKffMvCzZEbqG/Tr N2TBlP5M5HkW1oL4QzRsykHnA7ZU9LNsFhnJ3pjJPxHVgjF0LK3d3Mx/W Q==; X-CSE-ConnectionGUID: 3zfbalAyRnuj77Pf0j41/Q== X-CSE-MsgGUID: YTflKPdyQMiKPaUj8igVvg== X-IronPort-AV: E=McAfee;i="6800,10657,11678"; a="81393034" X-IronPort-AV: E=Sophos;i="6.21,244,1763452800"; d="scan'208";a="81393034" Received: from fmviesa004.fm.intel.com ([10.60.135.144]) by orvoesa105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Jan 2026 17:51:26 -0800 X-CSE-ConnectionGUID: ItYt5QW+QYmnjtRzsRFEXg== X-CSE-MsgGUID: YzeNCHDDRjS+LHj1DbZQag== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.21,244,1763452800"; d="scan'208";a="211455624" Received: from allen-box.sh.intel.com ([10.239.159.52]) by fmviesa004.fm.intel.com with ESMTP; 21 Jan 2026 17:51:24 -0800 From: Lu Baolu To: Joerg Roedel Cc: Yi Liu , Dmytro Maluka , Jinhui Guo , iommu@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH 1/7] iommu/vt-d: Skip dev-iotlb flush for inaccessible PCIe device without scalable mode Date: Thu, 22 Jan 2026 09:48:50 +0800 Message-ID: <20260122014856.2457052-2-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260122014856.2457052-1-baolu.lu@linux.intel.com> References: <20260122014856.2457052-1-baolu.lu@linux.intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable From: Jinhui Guo PCIe endpoints with ATS enabled and passed through to userspace (e.g., QEMU, DPDK) can hard-lock the host when their link drops, either by surprise removal or by a link fault. Commit 4fc82cd907ac ("iommu/vt-d: Don't issue ATS Invalidation request when device is disconnected") adds pci_dev_is_disconnected() to devtlb_invalidation_with_pasid() so ATS invalidation is skipped only when the device is being safely removed, but it applies only when Intel IOMMU scalable mode is enabled. With scalable mode disabled or unsupported, a system hard-lock occurs when a PCIe endpoint's link drops because the Intel IOMMU waits indefinitely for an ATS invalidation that cannot complete. Call Trace: qi_submit_sync qi_flush_dev_iotlb __context_flush_dev_iotlb.part.0 domain_context_clear_one_cb pci_for_each_dma_alias device_block_translation blocking_domain_attach_dev iommu_deinit_device __iommu_group_remove_device iommu_release_device iommu_bus_notifier blocking_notifier_call_chain bus_notify device_del pci_remove_bus_device pci_stop_and_remove_bus_device pciehp_unconfigure_device pciehp_disable_slot pciehp_handle_presence_or_link_change pciehp_ist Commit 81e921fd3216 ("iommu/vt-d: Fix NULL domain on device release") adds intel_pasid_teardown_sm_context() to intel_iommu_release_device(), which calls qi_flush_dev_iotlb() and can also hard-lock the system when a PCIe endpoint's link drops. Call Trace: qi_submit_sync qi_flush_dev_iotlb __context_flush_dev_iotlb.part.0 intel_context_flush_no_pasid device_pasid_table_teardown pci_pasid_table_teardown pci_for_each_dma_alias intel_pasid_teardown_sm_context intel_iommu_release_device iommu_deinit_device __iommu_group_remove_device iommu_release_device iommu_bus_notifier blocking_notifier_call_chain bus_notify device_del pci_remove_bus_device pci_stop_and_remove_bus_device pciehp_unconfigure_device pciehp_disable_slot pciehp_handle_presence_or_link_change pciehp_ist Sometimes the endpoint loses connection without a link-down event (e.g., due to a link fault); killing the process (virsh destroy) then hard-locks the host. Call Trace: qi_submit_sync qi_flush_dev_iotlb __context_flush_dev_iotlb.part.0 domain_context_clear_one_cb pci_for_each_dma_alias device_block_translation blocking_domain_attach_dev __iommu_attach_device __iommu_device_set_domain __iommu_group_set_domain_internal iommu_detach_group vfio_iommu_type1_detach_group vfio_group_detach_container vfio_group_fops_release __fput pci_dev_is_disconnected() only covers safe-removal paths; pci_device_is_present() tests accessibility by reading vendor/device IDs and internally calls pci_dev_is_disconnected(). On a ConnectX-5 (8 GT/s, x2) this costs ~70 =C2=B5s. Since __context_flush_dev_iotlb() is only called on {attach,release}_dev paths (not hot), add pci_device_is_present() there to skip inaccessible devices and avoid the hard-lock. Fixes: 37764b952e1b ("iommu/vt-d: Global devTLB flush when present context = entry changed") Fixes: 81e921fd3216 ("iommu/vt-d: Fix NULL domain on device release") Cc: stable@vger.kernel.org Signed-off-by: Jinhui Guo Link: https://lore.kernel.org/r/20251211035946.2071-2-guojinhui.liam@byteda= nce.com Signed-off-by: Lu Baolu --- drivers/iommu/intel/pasid.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/drivers/iommu/intel/pasid.c b/drivers/iommu/intel/pasid.c index 3e2255057079..3f6d78180d79 100644 --- a/drivers/iommu/intel/pasid.c +++ b/drivers/iommu/intel/pasid.c @@ -1102,6 +1102,14 @@ static void __context_flush_dev_iotlb(struct device_= domain_info *info) if (!info->ats_enabled) return; =20 + /* + * Skip dev-IOTLB flush for inaccessible PCIe devices to prevent the + * Intel IOMMU from waiting indefinitely for an ATS invalidation that + * cannot complete. + */ + if (!pci_device_is_present(to_pci_dev(info->dev))) + return; + qi_flush_dev_iotlb(info->iommu, PCI_DEVID(info->bus, info->devfn), info->pfsid, info->ats_qdep, 0, MAX_AGAW_PFN_WIDTH); =20 --=20 2.43.0 From nobody Sat Feb 7 15:12:39 2026 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4FA04352FBC for ; Thu, 22 Jan 2026 01:51:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.13 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769046690; cv=none; b=dVSZw18UW95/VaP1eMWxbdPnVA6f8oEssIahMrIEZmbtCY2BIbzf/ziRkEzrw5/Hf6KsvAHQrN5YvFEUjIVh9oCAsZtI6DHb4F20DCZEx41+ykKyh9ix4zKGZlTmmETrH8T0mDalcFdT+SsCRog6nocb4bCR4oTadkyXa5T5mXE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769046690; c=relaxed/simple; bh=o8WhUb2cIgqs47EaaOIo01hqCpRDU6Atj+vOy0GL3BM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=Typ6lkTjPEMJ3C+WDIdbjs0AKcU0jo/bAzi6DMWMSB5oteH7ZrEDRyXgYKcmm2GMGEn5ohcFUTl0dUaTn/Klds6HGRfw/hp/ogcfcY+jXaeBdrZeLXRUpKFQV9PlixIABI/l71xangBQVh3uU/QMp4tUF8W8aoEcNrWjsNKd+Qk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=pass smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=NnXQOqAo; arc=none smtp.client-ip=198.175.65.13 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="NnXQOqAo" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1769046688; x=1800582688; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=o8WhUb2cIgqs47EaaOIo01hqCpRDU6Atj+vOy0GL3BM=; b=NnXQOqAoWw7oRkUtU7MS14dDpxvLWhvl3JCtQBQje920hLSjYxUiRYZD yHSmWHfOmTEX/AOpB2eQwLwEx/K/UzC88zuHUtoSkK6FTq76p1kJctvZJ 1DApRiwTN5EA7fodAp71fOB6ypSsu9KkzIrDaHsYcp4gX5zdovPz7I8Zo /tASuO+LB5WpBHowskEQLED7kI/3e70hOfR2NHvmbtBsVqYUiICj04BKk dkWHlzkVdTeGRHsdqLXyK5VLCxvTkXqC6YURb/Wu4+ip4krZ/zLbFwrh2 iZSmBFgUNwaKlC3bfduFiBeR4QwiQom/vAaZm79IkrcSi2S7QeMmfv3KY w==; X-CSE-ConnectionGUID: TUZgodKbQeG8XCtxaF9JyA== X-CSE-MsgGUID: T0aVKdE+RSuyCnTSaQ7yAA== X-IronPort-AV: E=McAfee;i="6800,10657,11678"; a="81393039" X-IronPort-AV: E=Sophos;i="6.21,244,1763452800"; d="scan'208";a="81393039" Received: from fmviesa004.fm.intel.com ([10.60.135.144]) by orvoesa105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Jan 2026 17:51:28 -0800 X-CSE-ConnectionGUID: QNETBFCOTPqqEIAqbWZd6g== X-CSE-MsgGUID: WrbwaRmkSUKKv3lTrwu7vQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.21,244,1763452800"; d="scan'208";a="211455628" Received: from allen-box.sh.intel.com ([10.239.159.52]) by fmviesa004.fm.intel.com with ESMTP; 21 Jan 2026 17:51:26 -0800 From: Lu Baolu To: Joerg Roedel Cc: Yi Liu , Dmytro Maluka , Jinhui Guo , iommu@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH 2/7] iommu/vt-d: Flush dev-IOTLB only when PCIe device is accessible in scalable mode Date: Thu, 22 Jan 2026 09:48:51 +0800 Message-ID: <20260122014856.2457052-3-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260122014856.2457052-1-baolu.lu@linux.intel.com> References: <20260122014856.2457052-1-baolu.lu@linux.intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable From: Jinhui Guo Commit 4fc82cd907ac ("iommu/vt-d: Don't issue ATS Invalidation request when device is disconnected") relies on pci_dev_is_disconnected() to skip ATS invalidation for safely-removed devices, but it does not cover link-down caused by faults, which can still hard-lock the system. For example, if a VM fails to connect to the PCIe device, "virsh destroy" is executed to release resources and isolate the fault, but a hard-lockup occurs while releasing the group fd. Call Trace: qi_submit_sync qi_flush_dev_iotlb intel_pasid_tear_down_entry device_block_translation blocking_domain_attach_dev __iommu_attach_device __iommu_device_set_domain __iommu_group_set_domain_internal iommu_detach_group vfio_iommu_type1_detach_group vfio_group_detach_container vfio_group_fops_release __fput Although pci_device_is_present() is slower than pci_dev_is_disconnected(), it still takes only ~70 =C2=B5s on a ConnectX-5 (8 GT/s, x2) and becomes even faster as PCIe speed and width increase. Besides, devtlb_invalidation_with_pasid() is called only in the paths below, which are far less frequent than memory map/unmap. 1. mm-struct release 2. {attach,release}_dev 3. set/remove PASID 4. dirty-tracking setup The gain in system stability far outweighs the negligible cost of using pci_device_is_present() instead of pci_dev_is_disconnected() to decide when to skip ATS invalidation, especially under GDR high-load conditions. Fixes: 4fc82cd907ac ("iommu/vt-d: Don't issue ATS Invalidation request when= device is disconnected") Cc: stable@vger.kernel.org Signed-off-by: Jinhui Guo Link: https://lore.kernel.org/r/20251211035946.2071-3-guojinhui.liam@byteda= nce.com Signed-off-by: Lu Baolu --- drivers/iommu/intel/pasid.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/iommu/intel/pasid.c b/drivers/iommu/intel/pasid.c index 3f6d78180d79..99692f88b883 100644 --- a/drivers/iommu/intel/pasid.c +++ b/drivers/iommu/intel/pasid.c @@ -218,7 +218,7 @@ devtlb_invalidation_with_pasid(struct intel_iommu *iomm= u, if (!info || !info->ats_enabled) return; =20 - if (pci_dev_is_disconnected(to_pci_dev(dev))) + if (!pci_device_is_present(to_pci_dev(dev))) return; =20 sid =3D PCI_DEVID(info->bus, info->devfn); --=20 2.43.0 From nobody Sat Feb 7 15:12:39 2026 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D2DE635F8A4 for ; Thu, 22 Jan 2026 01:51:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.13 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769046691; cv=none; b=Xd76Onf3O2BOeY1FNiiHJKFiQcsB0ptDUeHw7LsSAEINl1Y7jvJEfry/tAgTUnPNsMSu0w2AALbKqsBKzzUxvklHzmjV6UrUYuPU92VGSTRe25ozg8Lr+LhGIx/vABZRdVBE4GCosR7ONJJV5XX2fWFxT/nqQEKXM95P+XOSQRI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769046691; c=relaxed/simple; bh=SsGMZADROTEU6smSqhYfvF4X4CchLTXShywLJIVw950=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=dva3d82ETx+eqD5S91g9BIQyj7Dv9uZTGCzFoRGmpFDSP1ObXW3v/yg+G48mbEfsaWKgn6M17uydSJWijSMvoQIykIYz1BWjKr5R5X21jT80fOT04KhtPkfVZUzHYjHPLp/GdDIlDUWNgQs7T4P3kb0zbHjIeJyRRTlHQmzU7Ng= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=pass smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=gA0nt51s; arc=none smtp.client-ip=198.175.65.13 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="gA0nt51s" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1769046690; x=1800582690; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=SsGMZADROTEU6smSqhYfvF4X4CchLTXShywLJIVw950=; b=gA0nt51si+C88pNbu/+yHLZUU8HPRrFJlAXMFO2ar24CfuG+UsEwYYEw Z8Afj+OyToZDaPREJ9t1iuxlnulnihMvnbU+i1O0Xswz1mjTRDq+gMrqS Dh3hnDUHo3SdUA6iZFFKGfSw2XvyCZEqgBPAHecUnyAMJnYISEWYWRbHJ sTSSw+cas7+vTe0toyfXcYOJb6tby1O6t5FyOXYMVBDLPUghrKpxcbndg I6OWZbW6neLBmsZ1HM6y+2qlP5Ov2gDQ1x9k8CKXXnFuktfNR2jDrHqhQ HKt5bkA1L+keueGnpc1kCC0Fr/fT1z7w1TBr2EBOwK48KTPG6lQWBQoU6 w==; X-CSE-ConnectionGUID: lv1FTZonQ9C4D/0oTKLCFg== X-CSE-MsgGUID: Du2wycpMQMOipry2cFEqWg== X-IronPort-AV: E=McAfee;i="6800,10657,11678"; a="81393044" X-IronPort-AV: E=Sophos;i="6.21,244,1763452800"; d="scan'208";a="81393044" Received: from fmviesa004.fm.intel.com ([10.60.135.144]) by orvoesa105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Jan 2026 17:51:29 -0800 X-CSE-ConnectionGUID: PI0C6/2bTDarPwRZ2ahLBg== X-CSE-MsgGUID: 2H/VUMfNTYW8ugnJS+pLnA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.21,244,1763452800"; d="scan'208";a="211455636" Received: from allen-box.sh.intel.com ([10.239.159.52]) by fmviesa004.fm.intel.com with ESMTP; 21 Jan 2026 17:51:27 -0800 From: Lu Baolu To: Joerg Roedel Cc: Yi Liu , Dmytro Maluka , Jinhui Guo , iommu@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH 3/7] iommu/vt-d: Flush cache for PASID table before using it Date: Thu, 22 Jan 2026 09:48:52 +0800 Message-ID: <20260122014856.2457052-4-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260122014856.2457052-1-baolu.lu@linux.intel.com> References: <20260122014856.2457052-1-baolu.lu@linux.intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Dmytro Maluka When writing the address of a freshly allocated zero-initialized PASID table to a PASID directory entry, do that after the CPU cache flush for this PASID table, not before it, to avoid the time window when this PASID table may be already used by non-coherent IOMMU hardware while its contents in RAM is still some random old data, not zero-initialized. Fixes: 194b3348bdbb ("iommu/vt-d: Fix PASID directory pointer coherency") Signed-off-by: Dmytro Maluka Reviewed-by: Kevin Tian Link: https://lore.kernel.org/r/20251221123508.37495-1-dmaluka@chromium.org Signed-off-by: Lu Baolu --- drivers/iommu/intel/pasid.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/drivers/iommu/intel/pasid.c b/drivers/iommu/intel/pasid.c index 99692f88b883..6379b211f12b 100644 --- a/drivers/iommu/intel/pasid.c +++ b/drivers/iommu/intel/pasid.c @@ -153,6 +153,9 @@ static struct pasid_entry *intel_pasid_get_entry(struct= device *dev, u32 pasid) if (!entries) return NULL; =20 + if (!ecap_coherent(info->iommu->ecap)) + clflush_cache_range(entries, VTD_PAGE_SIZE); + /* * The pasid directory table entry won't be freed after * allocation. No worry about the race with free and @@ -165,10 +168,8 @@ static struct pasid_entry *intel_pasid_get_entry(struc= t device *dev, u32 pasid) iommu_free_pages(entries); goto retry; } - if (!ecap_coherent(info->iommu->ecap)) { - clflush_cache_range(entries, VTD_PAGE_SIZE); + if (!ecap_coherent(info->iommu->ecap)) clflush_cache_range(&dir[dir_index].val, sizeof(*dir)); - } } =20 return &entries[index]; --=20 2.43.0 From nobody Sat Feb 7 15:12:39 2026 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 58FA036826B for ; Thu, 22 Jan 2026 01:51:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.13 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769046693; cv=none; b=kytf+O3GAdx4aAFjwfT+ZKhFsp+5U9roXPB3287hdNnTfa3vjdKo+o08yTG298eY/eDNEPIWDy0LiQjkegxUIBerXQ2KK/+fUD/HareFoJNs5RKEkTbT++sd1JTYvK0bpFoeRCQHwsb/K7j1eBnzp+5r5koGcbrYGQGUzSr50ec= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769046693; c=relaxed/simple; bh=Kszgw8EuzZbIrSzlqXr9hREhZUa7uq18+lEd0ySC9CI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=PKpJU/Q+PfMvi7iQTixNvs3FTGSKm+raAgMWkJYrp0SKkMi/BXHAXkib7z8ixxYZg+vwyh1YNRc+3AUPQ8CLH8puKZkKYBkkjeoSc5wNqhg0233XIZR4OpYoq8hehQZx5gz12H0eUsamGUyTTvgMU+ppihhCMZQYpZroYAxrJe8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=pass smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=Z4aunM4S; arc=none smtp.client-ip=198.175.65.13 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="Z4aunM4S" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1769046691; x=1800582691; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Kszgw8EuzZbIrSzlqXr9hREhZUa7uq18+lEd0ySC9CI=; b=Z4aunM4SZwk6E3Bt4Po22+Vxjy5Mf0n/QnVFis9z8fkwVK2mt85T+5A0 xDGm2Hw344xnLAhtA7Mcz6H1ET9fq36jjOSlFL+Qrul5yYaTc5C4ljAlt Xs7BPQTq+qGzc61Hyw4k5wxr2ZR1qVMx1rJjs9SKzCBifTEA5zcG6zHAk k107YT8fAWyjHORu2GWOj8XyuBDvHKH2yvpL4ITAuF/5TkT1DUjASKPxB uAxLfa956dGhJyS8XTXyGZrhF5sANDQ7o+ForHY//S73yhPGsrGyd7vGg 6R5j7avxUffGpqgGvDcBR+aw1FXjuwgpvvkMuDgLp3Xr2652EH2EmpFRT Q==; X-CSE-ConnectionGUID: Nt9OFsZ8RR2Xyccb0knPTA== X-CSE-MsgGUID: wALLU8zXTTa10twPq6UOSA== X-IronPort-AV: E=McAfee;i="6800,10657,11678"; a="81393049" X-IronPort-AV: E=Sophos;i="6.21,244,1763452800"; d="scan'208";a="81393049" Received: from fmviesa004.fm.intel.com ([10.60.135.144]) by orvoesa105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Jan 2026 17:51:31 -0800 X-CSE-ConnectionGUID: HhFqm0wOTR6NKE8F4RgaNg== X-CSE-MsgGUID: gKGEWUkyTxqVfhJt5pdwVQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.21,244,1763452800"; d="scan'208";a="211455640" Received: from allen-box.sh.intel.com ([10.239.159.52]) by fmviesa004.fm.intel.com with ESMTP; 21 Jan 2026 17:51:29 -0800 From: Lu Baolu To: Joerg Roedel Cc: Yi Liu , Dmytro Maluka , Jinhui Guo , iommu@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH 4/7] iommu/vt-d: Flush piotlb for SVM and Nested domain Date: Thu, 22 Jan 2026 09:48:53 +0800 Message-ID: <20260122014856.2457052-5-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260122014856.2457052-1-baolu.lu@linux.intel.com> References: <20260122014856.2457052-1-baolu.lu@linux.intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Yi Liu Besides the paging domains that use FS, SVM and Nested domains need to use piotlb invalidation descriptor as well. Fixes: b33125296b50 ("iommu/vt-d: Create unique domain ops for each stage") Cc: stable@vger.kernel.org Signed-off-by: Yi Liu Reviewed-by: Kevin Tian Link: https://lore.kernel.org/r/20251223065824.6164-1-yi.l.liu@intel.com Signed-off-by: Lu Baolu --- drivers/iommu/intel/cache.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/drivers/iommu/intel/cache.c b/drivers/iommu/intel/cache.c index 265e7290256b..385ae5cfb30d 100644 --- a/drivers/iommu/intel/cache.c +++ b/drivers/iommu/intel/cache.c @@ -363,6 +363,13 @@ static void qi_batch_add_pasid_dev_iotlb(struct intel_= iommu *iommu, u16 sid, u16 qi_batch_increment_index(iommu, batch); } =20 +static bool intel_domain_use_piotlb(struct dmar_domain *domain) +{ + return domain->domain.type =3D=3D IOMMU_DOMAIN_SVA || + domain->domain.type =3D=3D IOMMU_DOMAIN_NESTED || + intel_domain_is_fs_paging(domain); +} + static void cache_tag_flush_iotlb(struct dmar_domain *domain, struct cache= _tag *tag, unsigned long addr, unsigned long pages, unsigned long mask, int ih) @@ -370,7 +377,7 @@ static void cache_tag_flush_iotlb(struct dmar_domain *d= omain, struct cache_tag * struct intel_iommu *iommu =3D tag->iommu; u64 type =3D DMA_TLB_PSI_FLUSH; =20 - if (intel_domain_is_fs_paging(domain)) { + if (intel_domain_use_piotlb(domain)) { qi_batch_add_piotlb(iommu, tag->domain_id, tag->pasid, addr, pages, ih, domain->qi_batch); return; --=20 2.43.0 From nobody Sat Feb 7 15:12:39 2026 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 17E8D36997D for ; Thu, 22 Jan 2026 01:51:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.13 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769046695; cv=none; b=thf6aZNNqOwpYhAQh/ZrY+CLKfE/azSjkZdprddyInJsWXQKRFbAyXhKhkNuch6+JGKwoIaCYn1CP7/YEohYKXi3DHXDf0kHHZ8WGMlVCtyAzShPSulk5NYqBDGVYnMsMbeqVW3inOX1sZ13c+6wv2S95OUUjydgPGDrVn+o80I= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769046695; c=relaxed/simple; bh=f/2d1fGzDtMdu8PeYLJCG0hCit54tFui1Ge4r9D7pqk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ipFf4CpPBitSaVG9jmwUttknbLROKyuNmdML9/P8fFuQZUCR9BXmGKHxC+ojyBc3n5tOnHj3kwfYqTGSEcDrWRjJflj4xcB5Mpwph0aQI2tGWPQx9KZs/tSjah8oYiYbEshURxoa0FC2uYPn5inL0C1ywfW7fJNzbdBRjLPgGw8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=pass smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=FP0YExDJ; arc=none smtp.client-ip=198.175.65.13 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="FP0YExDJ" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1769046693; x=1800582693; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=f/2d1fGzDtMdu8PeYLJCG0hCit54tFui1Ge4r9D7pqk=; b=FP0YExDJh5Xhq8CQrZ2PyTetopTWQpwGyiK23EysySTLvMdZYeaC9kTq 8oRqvQWnkXkgjpbgSIKHDF7/pe8ATIGvpyMfpEyi3+bh2qKpu42au0EsC d49HwRLnc4MkYS9EIzLtszjJIhvCAim4t5agYZnc0jZXy9dip+qHlw/sW K+E0wkuFOZPPd9UOMXclh5k80SycXP6gffrOYOlmsvJTBBvNQBNlst4Gd NcJ5nLrZLDuf/wbYTaERwx9/LS2yFgSkbbdzOVFhJwGuaD6/uSlsffMO8 JccJDIvJZ8K2MDwiTjux1883sQwVtVjRxcaOgYsp0Nf3UmwFtzInskhnh g==; X-CSE-ConnectionGUID: NNpPARWxSDyXHXdTnn+5zA== X-CSE-MsgGUID: gAy9D+QOToWHoEB4o8UTBg== X-IronPort-AV: E=McAfee;i="6800,10657,11678"; a="81393054" X-IronPort-AV: E=Sophos;i="6.21,244,1763452800"; d="scan'208";a="81393054" Received: from fmviesa004.fm.intel.com ([10.60.135.144]) by orvoesa105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Jan 2026 17:51:33 -0800 X-CSE-ConnectionGUID: Ffgv696ITwWg+LI3yc/Dkg== X-CSE-MsgGUID: DoyKMvuNRFiE07FNxp+dEQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.21,244,1763452800"; d="scan'208";a="211455644" Received: from allen-box.sh.intel.com ([10.239.159.52]) by fmviesa004.fm.intel.com with ESMTP; 21 Jan 2026 17:51:31 -0800 From: Lu Baolu To: Joerg Roedel Cc: Yi Liu , Dmytro Maluka , Jinhui Guo , iommu@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH 5/7] iommu/vt-d: Clear Present bit before tearing down PASID entry Date: Thu, 22 Jan 2026 09:48:54 +0800 Message-ID: <20260122014856.2457052-6-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260122014856.2457052-1-baolu.lu@linux.intel.com> References: <20260122014856.2457052-1-baolu.lu@linux.intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The Intel VT-d Scalable Mode PASID table entry consists of 512 bits (64 bytes). When tearing down an entry, the current implementation zeros the entire 64-byte structure immediately using multiple 64-bit writes. Since the IOMMU hardware may fetch these 64 bytes using multiple internal transactions (e.g., four 128-bit bursts), updating or zeroing the entire entry while it is active (P=3D1) risks a "torn" read. If a hardware fetch occurs simultaneously with the CPU zeroing the entry, the hardware could observe an inconsistent state, leading to unpredictable behavior or spurious faults. Follow the "Guidance to Software for Invalidations" in the VT-d spec (Section 6.5.3.3) by implementing the recommended ownership handshake: 1. Clear only the 'Present' (P) bit of the PASID entry. 2. Use a dma_wmb() to ensure the cleared bit is visible to hardware before proceeding. 3. Execute the required invalidation sequence (PASID cache, IOTLB, and Device-TLB flush) to ensure the hardware has released all cached references. 4. Only after the flushes are complete, zero out the remaining fields of the PASID entry. Also, add a dma_wmb() in pasid_set_present() to ensure that all other fields of the PASID entry are visible to the hardware before the Present bit is set. Fixes: 0bbeb01a4faf ("iommu/vt-d: Manage scalalble mode PASID tables") Signed-off-by: Lu Baolu Reviewed-by: Dmytro Maluka Reviewed-by: Samiullah Khawaja Reviewed-by: Kevin Tian Link: https://lore.kernel.org/r/20260120061816.2132558-2-baolu.lu@linux.int= el.com --- drivers/iommu/intel/pasid.h | 14 ++++++++++++++ drivers/iommu/intel/pasid.c | 6 +++++- 2 files changed, 19 insertions(+), 1 deletion(-) diff --git a/drivers/iommu/intel/pasid.h b/drivers/iommu/intel/pasid.h index b4c85242dc79..0b303bd0b0c1 100644 --- a/drivers/iommu/intel/pasid.h +++ b/drivers/iommu/intel/pasid.h @@ -234,9 +234,23 @@ static inline void pasid_set_wpe(struct pasid_entry *p= e) */ static inline void pasid_set_present(struct pasid_entry *pe) { + dma_wmb(); pasid_set_bits(&pe->val[0], 1 << 0, 1); } =20 +/* + * Clear the Present (P) bit (bit 0) of a scalable-mode PASID table entry. + * This initiates the transition of the entry's ownership from hardware + * to software. The caller is responsible for fulfilling the invalidation + * handshake recommended by the VT-d spec, Section 6.5.3.3 (Guidance to + * Software for Invalidations). + */ +static inline void pasid_clear_present(struct pasid_entry *pe) +{ + pasid_set_bits(&pe->val[0], 1 << 0, 0); + dma_wmb(); +} + /* * Setup Page Walk Snoop bit (Bit 87) of a scalable mode PASID * entry. diff --git a/drivers/iommu/intel/pasid.c b/drivers/iommu/intel/pasid.c index 6379b211f12b..07e056b24605 100644 --- a/drivers/iommu/intel/pasid.c +++ b/drivers/iommu/intel/pasid.c @@ -273,7 +273,7 @@ void intel_pasid_tear_down_entry(struct intel_iommu *io= mmu, struct device *dev, =20 did =3D pasid_get_domain_id(pte); pgtt =3D pasid_pte_get_pgtt(pte); - intel_pasid_clear_entry(dev, pasid, fault_ignore); + pasid_clear_present(pte); spin_unlock(&iommu->lock); =20 if (!ecap_coherent(iommu->ecap)) @@ -287,6 +287,10 @@ void intel_pasid_tear_down_entry(struct intel_iommu *i= ommu, struct device *dev, iommu->flush.flush_iotlb(iommu, did, 0, 0, DMA_TLB_DSI_FLUSH); =20 devtlb_invalidation_with_pasid(iommu, dev, pasid); + intel_pasid_clear_entry(dev, pasid, fault_ignore); + if (!ecap_coherent(iommu->ecap)) + clflush_cache_range(pte, sizeof(*pte)); + if (!fault_ignore) intel_iommu_drain_pasid_prq(dev, pasid); } --=20 2.43.0 From nobody Sat Feb 7 15:12:39 2026 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D245C36AB4A for ; Thu, 22 Jan 2026 01:51:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.13 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769046697; cv=none; b=p+A3nSV0vzUiL9Styx5xSPG4BNAmDsVRk4Slk8Q///yW1wsbvzPpMjAQ8alilfUM+3Xk2/rShMHLrtHVKIS0kk2ve6jEL1k1v2OvM/hcqZK4hPvj8f3pOa/W1MftJtlNQl8lVNchlGN8ockdaD3BzNbKbDaPxAzzUTqAxiZRWrQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769046697; c=relaxed/simple; bh=feve70lOzZ7CiXg4b8hjR+SRMNJSTVjOQ4VpPQj1shc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=o3uDoRUtXlnZEG5eizteJMnZ/fyC9CPqrlXbXQCipGgz1AVMMvjjjXD/mnouhTPeb8p6GM7AQkxgviOmCVXGVttTIFVPab0l91nWHRs4jVB233wmpQhyyXNLNxKr7Wo5sS1dL5QFFTUDj7D6+ybjcKWLIklgtLDGvGmMKL9S1Qo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=pass smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=U/jbfwir; arc=none smtp.client-ip=198.175.65.13 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="U/jbfwir" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1769046695; x=1800582695; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=feve70lOzZ7CiXg4b8hjR+SRMNJSTVjOQ4VpPQj1shc=; b=U/jbfwirS0bXPUlFigGdLBbLVU8C84RgbPA1GuJwgDvgm1dHqTUExn/x AJiQE1bEZtB1eYTzny4cA1SlJRA3sEsEAnnLyEV24Tt4KhZacfmSlHzLe zRUMtwIcOwGfU8YZEaJCc6DlS+Y1r6SRQkPqeeJ3uIbTbGDBYRpf/lp+0 oUd6bepE8XLW6SlBifOf7B6uz5HKXWFNjqkZWZ/p0BZ+VU4Ae8R9dsojm JYDMfftUYlnFHWWFjXXWxvJ1Um+vhcrTTF2vTOm8ojSsP03/emTsRcAW5 8dRhOwUDYkgP4qfYY+0NTIRGJB/5p1NKe5PxNWjDwcxPjzno68+MG3nnw Q==; X-CSE-ConnectionGUID: zrRKl7aGTj+6pvfkUTSziw== X-CSE-MsgGUID: MClakdw1S22sN2PcunVI9Q== X-IronPort-AV: E=McAfee;i="6800,10657,11678"; a="81393060" X-IronPort-AV: E=Sophos;i="6.21,244,1763452800"; d="scan'208";a="81393060" Received: from fmviesa004.fm.intel.com ([10.60.135.144]) by orvoesa105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Jan 2026 17:51:34 -0800 X-CSE-ConnectionGUID: /4ejGBJ+R5aSTbk409CQDg== X-CSE-MsgGUID: 8YO/urzPRCKW4h3CARzhtg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.21,244,1763452800"; d="scan'208";a="211455651" Received: from allen-box.sh.intel.com ([10.239.159.52]) by fmviesa004.fm.intel.com with ESMTP; 21 Jan 2026 17:51:33 -0800 From: Lu Baolu To: Joerg Roedel Cc: Yi Liu , Dmytro Maluka , Jinhui Guo , iommu@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH 6/7] iommu/vt-d: Clear Present bit before tearing down context entry Date: Thu, 22 Jan 2026 09:48:55 +0800 Message-ID: <20260122014856.2457052-7-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260122014856.2457052-1-baolu.lu@linux.intel.com> References: <20260122014856.2457052-1-baolu.lu@linux.intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable When tearing down a context entry, the current implementation zeros the entire 128-bit entry using multiple 64-bit writes. This creates a window where the hardware can fetch a "torn" entry =E2=80=94 where some fields are already zeroed while the 'Present' bit is still set =E2=80=94 leading to unpredictable behavior or spurious faults. While x86 provides strong write ordering, the compiler may reorder writes to the two 64-bit halves of the context entry. Even without compiler reordering, the hardware fetch is not guaranteed to be atomic with respect to multiple CPU writes. Align with the "Guidance to Software for Invalidations" in the VT-d spec (Section 6.5.3.3) by implementing the recommended ownership handshake: 1. Clear only the 'Present' (P) bit of the context entry first to signal the transition of ownership from hardware to software. 2. Use dma_wmb() to ensure the cleared bit is visible to the IOMMU. 3. Perform the required cache and context-cache invalidation to ensure hardware no longer has cached references to the entry. 4. Fully zero out the entry only after the invalidation is complete. Also, add a dma_wmb() to context_set_present() to ensure the entry is fully initialized before the 'Present' bit becomes visible. Fixes: ba39592764ed2 ("Intel IOMMU: Intel IOMMU driver") Reported-by: Dmytro Maluka Closes: https://lore.kernel.org/all/aTG7gc7I5wExai3S@google.com/ Signed-off-by: Lu Baolu Reviewed-by: Dmytro Maluka Reviewed-by: Samiullah Khawaja Reviewed-by: Kevin Tian Link: https://lore.kernel.org/r/20260120061816.2132558-3-baolu.lu@linux.int= el.com --- drivers/iommu/intel/iommu.h | 21 ++++++++++++++++++++- drivers/iommu/intel/iommu.c | 4 +++- drivers/iommu/intel/pasid.c | 5 ++++- 3 files changed, 27 insertions(+), 3 deletions(-) diff --git a/drivers/iommu/intel/iommu.h b/drivers/iommu/intel/iommu.h index 25c5e22096d4..599913fb65d5 100644 --- a/drivers/iommu/intel/iommu.h +++ b/drivers/iommu/intel/iommu.h @@ -900,7 +900,26 @@ static inline int pfn_level_offset(u64 pfn, int level) =20 static inline void context_set_present(struct context_entry *context) { - context->lo |=3D 1; + u64 val; + + dma_wmb(); + val =3D READ_ONCE(context->lo) | 1; + WRITE_ONCE(context->lo, val); +} + +/* + * Clear the Present (P) bit (bit 0) of a context table entry. This initia= tes + * the transition of the entry's ownership from hardware to software. The + * caller is responsible for fulfilling the invalidation handshake recomme= nded + * by the VT-d spec, Section 6.5.3.3 (Guidance to Software for Invalidatio= ns). + */ +static inline void context_clear_present(struct context_entry *context) +{ + u64 val; + + val =3D READ_ONCE(context->lo) & GENMASK_ULL(63, 1); + WRITE_ONCE(context->lo, val); + dma_wmb(); } =20 static inline void context_set_fault_enable(struct context_entry *context) diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c index 134302fbcd92..c66cc51f9e51 100644 --- a/drivers/iommu/intel/iommu.c +++ b/drivers/iommu/intel/iommu.c @@ -1240,10 +1240,12 @@ static void domain_context_clear_one(struct device_= domain_info *info, u8 bus, u8 } =20 did =3D context_domain_id(context); - context_clear_entry(context); + context_clear_present(context); __iommu_flush_cache(iommu, context, sizeof(*context)); spin_unlock(&iommu->lock); intel_context_flush_no_pasid(info, context, did); + context_clear_entry(context); + __iommu_flush_cache(iommu, context, sizeof(*context)); } =20 int __domain_setup_first_level(struct intel_iommu *iommu, struct device *d= ev, diff --git a/drivers/iommu/intel/pasid.c b/drivers/iommu/intel/pasid.c index 07e056b24605..f5dfa9b9eb3e 100644 --- a/drivers/iommu/intel/pasid.c +++ b/drivers/iommu/intel/pasid.c @@ -1024,7 +1024,7 @@ static int device_pasid_table_setup(struct device *de= v, u8 bus, u8 devfn) } =20 if (context_copied(iommu, bus, devfn)) { - context_clear_entry(context); + context_clear_present(context); __iommu_flush_cache(iommu, context, sizeof(*context)); =20 /* @@ -1044,6 +1044,9 @@ static int device_pasid_table_setup(struct device *de= v, u8 bus, u8 devfn) iommu->flush.flush_iotlb(iommu, 0, 0, 0, DMA_TLB_GLOBAL_FLUSH); devtlb_invalidation_with_pasid(iommu, dev, IOMMU_NO_PASID); =20 + context_clear_entry(context); + __iommu_flush_cache(iommu, context, sizeof(*context)); + /* * At this point, the device is supposed to finish reset at * its driver probe stage, so no in-flight DMA will exist, --=20 2.43.0 From nobody Sat Feb 7 15:12:39 2026 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 761B636C0D3 for ; Thu, 22 Jan 2026 01:51:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.13 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769046699; cv=none; b=X6UDvkTTmQxEaRQU2WhJcKLXVgDShG7L3VUfB0sR5NmoCdJf5AJ2zH3RR97aCZMSC0ookMIFvJj9Pm6ajxJD+CgU3+lh5Q851zhZQvCP3AglLG2iu7ffsMkWPyKSny/8lynTebsq+rT6kmnYVyndmWEW3XJhN4yLVnsxY8TyiCQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769046699; c=relaxed/simple; bh=s+S44jNzOKhUOYjjyD2ifmKsf29R+/KnlNbTGp/yirQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=BPSN1SioTB8In2TSfMySpcC3FZ7jQOEAId3imvqwbxAIS0BuFfGLWpf2LRJwBq5yUhUK8O3Fh8vp1naUlcD8NKsudPAFNUzOPQW7IGpTkihvBcQXQg9m25n61LNxelprNgnflpW0GeHhKOwrbWu0xwXYe2wKDkZ/ZsxIdxHjxIo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=pass smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=CNpwpzJT; arc=none smtp.client-ip=198.175.65.13 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="CNpwpzJT" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1769046696; x=1800582696; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=s+S44jNzOKhUOYjjyD2ifmKsf29R+/KnlNbTGp/yirQ=; b=CNpwpzJTHXyocOqiv8hAOE136oPxaJDfDfqcoGj9xEUFnd/ot8zUAYQU vnXADagYJlWBS6nyVxwEcS+Pg6Cq+yHkXH/p+/TtJDOPyxWAXrigOI5YB k2grfLu+LWwb/TQA0oFXJf9ta8TzZpnfXfyFpeHtBp0W7eMtsMzQuJz5g Byz7ytD/KJz81ovoq2CJpPw9GkrWU42n1I5KCmViu1lQLbuYCxAbNtEqk DRjsWCm3b0WL0iB20070eX6uT38cYo86lVAh1OYLF4SFtZsoSyUfaHx/J zRDIk6UyeWxpkfQ7E8EEagm/EU8vh4RCAMh3MqVtzuTiM+Y6l4bQFL2xV g==; X-CSE-ConnectionGUID: DaYyh/PvTRagzVRWRl7mXA== X-CSE-MsgGUID: lTeDMG7USb69xGUXSrmPyQ== X-IronPort-AV: E=McAfee;i="6800,10657,11678"; a="81393065" X-IronPort-AV: E=Sophos;i="6.21,244,1763452800"; d="scan'208";a="81393065" Received: from fmviesa004.fm.intel.com ([10.60.135.144]) by orvoesa105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Jan 2026 17:51:36 -0800 X-CSE-ConnectionGUID: KDVmAAEVRuSEl6htPBEikw== X-CSE-MsgGUID: Cbq0e/yUR26Ro2JSv5Jmww== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.21,244,1763452800"; d="scan'208";a="211455655" Received: from allen-box.sh.intel.com ([10.239.159.52]) by fmviesa004.fm.intel.com with ESMTP; 21 Jan 2026 17:51:34 -0800 From: Lu Baolu To: Joerg Roedel Cc: Yi Liu , Dmytro Maluka , Jinhui Guo , iommu@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH 7/7] iommu/vt-d: Fix race condition during PASID entry replacement Date: Thu, 22 Jan 2026 09:48:56 +0800 Message-ID: <20260122014856.2457052-8-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260122014856.2457052-1-baolu.lu@linux.intel.com> References: <20260122014856.2457052-1-baolu.lu@linux.intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable The Intel VT-d PASID table entry is 512 bits (64 bytes). When replacing an active PASID entry (e.g., during domain replacement), the current implementation calculates a new entry on the stack and copies it to the table using a single structure assignment. struct pasid_entry *pte, new_pte; pte =3D intel_pasid_get_entry(dev, pasid); pasid_pte_config_first_level(iommu, &new_pte, ...); *pte =3D new_pte; Because the hardware may fetch the 512-bit PASID entry in multiple 128-bit chunks, updating the entire entry while it is active (Present bit set) risks a "torn" read. In this scenario, the IOMMU hardware could observe an inconsistent state =E2=80=94 partially new data and partia= lly old data =E2=80=94 leading to unpredictable behavior or spurious faults. Fix this by removing the unsafe "replace" helpers and following the "clear-then-update" flow, which ensures the Present bit is cleared and the required invalidation handshake is completed before the new configuration is applied. Fixes: 7543ee63e811 ("iommu/vt-d: Add pasid replace helpers") Signed-off-by: Lu Baolu Reviewed-by: Samiullah Khawaja Reviewed-by: Kevin Tian Link: https://lore.kernel.org/r/20260120061816.2132558-4-baolu.lu@linux.int= el.com --- drivers/iommu/intel/pasid.h | 14 --- drivers/iommu/intel/iommu.c | 29 +++--- drivers/iommu/intel/nested.c | 9 +- drivers/iommu/intel/pasid.c | 184 ----------------------------------- 4 files changed, 16 insertions(+), 220 deletions(-) diff --git a/drivers/iommu/intel/pasid.h b/drivers/iommu/intel/pasid.h index 0b303bd0b0c1..c3c8c907983e 100644 --- a/drivers/iommu/intel/pasid.h +++ b/drivers/iommu/intel/pasid.h @@ -316,20 +316,6 @@ int intel_pasid_setup_pass_through(struct intel_iommu = *iommu, struct device *dev, u32 pasid); int intel_pasid_setup_nested(struct intel_iommu *iommu, struct device *dev, u32 pasid, struct dmar_domain *domain); -int intel_pasid_replace_first_level(struct intel_iommu *iommu, - struct device *dev, phys_addr_t fsptptr, - u32 pasid, u16 did, u16 old_did, int flags); -int intel_pasid_replace_second_level(struct intel_iommu *iommu, - struct dmar_domain *domain, - struct device *dev, u16 old_did, - u32 pasid); -int intel_pasid_replace_pass_through(struct intel_iommu *iommu, - struct device *dev, u16 old_did, - u32 pasid); -int intel_pasid_replace_nested(struct intel_iommu *iommu, - struct device *dev, u32 pasid, - u16 old_did, struct dmar_domain *domain); - void intel_pasid_tear_down_entry(struct intel_iommu *iommu, struct device *dev, u32 pasid, bool fault_ignore); diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c index c66cc51f9e51..705828b06e32 100644 --- a/drivers/iommu/intel/iommu.c +++ b/drivers/iommu/intel/iommu.c @@ -1252,12 +1252,10 @@ int __domain_setup_first_level(struct intel_iommu *= iommu, struct device *dev, ioasid_t pasid, u16 did, phys_addr_t fsptptr, int flags, struct iommu_domain *old) { - if (!old) - return intel_pasid_setup_first_level(iommu, dev, fsptptr, pasid, - did, flags); - return intel_pasid_replace_first_level(iommu, dev, fsptptr, pasid, did, - iommu_domain_did(old, iommu), - flags); + if (old) + intel_pasid_tear_down_entry(iommu, dev, pasid, false); + + return intel_pasid_setup_first_level(iommu, dev, fsptptr, pasid, did, fla= gs); } =20 static int domain_setup_second_level(struct intel_iommu *iommu, @@ -1265,23 +1263,20 @@ static int domain_setup_second_level(struct intel_i= ommu *iommu, struct device *dev, ioasid_t pasid, struct iommu_domain *old) { - if (!old) - return intel_pasid_setup_second_level(iommu, domain, - dev, pasid); - return intel_pasid_replace_second_level(iommu, domain, dev, - iommu_domain_did(old, iommu), - pasid); + if (old) + intel_pasid_tear_down_entry(iommu, dev, pasid, false); + + return intel_pasid_setup_second_level(iommu, domain, dev, pasid); } =20 static int domain_setup_passthrough(struct intel_iommu *iommu, struct device *dev, ioasid_t pasid, struct iommu_domain *old) { - if (!old) - return intel_pasid_setup_pass_through(iommu, dev, pasid); - return intel_pasid_replace_pass_through(iommu, dev, - iommu_domain_did(old, iommu), - pasid); + if (old) + intel_pasid_tear_down_entry(iommu, dev, pasid, false); + + return intel_pasid_setup_pass_through(iommu, dev, pasid); } =20 static int domain_setup_first_level(struct intel_iommu *iommu, diff --git a/drivers/iommu/intel/nested.c b/drivers/iommu/intel/nested.c index a3fb8c193ca6..e9a440e9c960 100644 --- a/drivers/iommu/intel/nested.c +++ b/drivers/iommu/intel/nested.c @@ -136,11 +136,10 @@ static int domain_setup_nested(struct intel_iommu *io= mmu, struct device *dev, ioasid_t pasid, struct iommu_domain *old) { - if (!old) - return intel_pasid_setup_nested(iommu, dev, pasid, domain); - return intel_pasid_replace_nested(iommu, dev, pasid, - iommu_domain_did(old, iommu), - domain); + if (old) + intel_pasid_tear_down_entry(iommu, dev, pasid, false); + + return intel_pasid_setup_nested(iommu, dev, pasid, domain); } =20 static int intel_nested_set_dev_pasid(struct iommu_domain *domain, diff --git a/drivers/iommu/intel/pasid.c b/drivers/iommu/intel/pasid.c index f5dfa9b9eb3e..b63a71904cfb 100644 --- a/drivers/iommu/intel/pasid.c +++ b/drivers/iommu/intel/pasid.c @@ -417,50 +417,6 @@ int intel_pasid_setup_first_level(struct intel_iommu *= iommu, struct device *dev, return 0; } =20 -int intel_pasid_replace_first_level(struct intel_iommu *iommu, - struct device *dev, phys_addr_t fsptptr, - u32 pasid, u16 did, u16 old_did, - int flags) -{ - struct pasid_entry *pte, new_pte; - - if (!ecap_flts(iommu->ecap)) { - pr_err("No first level translation support on %s\n", - iommu->name); - return -EINVAL; - } - - if ((flags & PASID_FLAG_FL5LP) && !cap_fl5lp_support(iommu->cap)) { - pr_err("No 5-level paging support for first-level on %s\n", - iommu->name); - return -EINVAL; - } - - pasid_pte_config_first_level(iommu, &new_pte, fsptptr, did, flags); - - spin_lock(&iommu->lock); - pte =3D intel_pasid_get_entry(dev, pasid); - if (!pte) { - spin_unlock(&iommu->lock); - return -ENODEV; - } - - if (!pasid_pte_is_present(pte)) { - spin_unlock(&iommu->lock); - return -EINVAL; - } - - WARN_ON(old_did !=3D pasid_get_domain_id(pte)); - - *pte =3D new_pte; - spin_unlock(&iommu->lock); - - intel_pasid_flush_present(iommu, dev, pasid, old_did, pte); - intel_iommu_drain_pasid_prq(dev, pasid); - - return 0; -} - /* * Set up the scalable mode pasid entry for second only translation type. */ @@ -527,51 +483,6 @@ int intel_pasid_setup_second_level(struct intel_iommu = *iommu, return 0; } =20 -int intel_pasid_replace_second_level(struct intel_iommu *iommu, - struct dmar_domain *domain, - struct device *dev, u16 old_did, - u32 pasid) -{ - struct pasid_entry *pte, new_pte; - u16 did; - - /* - * If hardware advertises no support for second level - * translation, return directly. - */ - if (!ecap_slts(iommu->ecap)) { - pr_err("No second level translation support on %s\n", - iommu->name); - return -EINVAL; - } - - did =3D domain_id_iommu(domain, iommu); - - pasid_pte_config_second_level(iommu, &new_pte, domain, did); - - spin_lock(&iommu->lock); - pte =3D intel_pasid_get_entry(dev, pasid); - if (!pte) { - spin_unlock(&iommu->lock); - return -ENODEV; - } - - if (!pasid_pte_is_present(pte)) { - spin_unlock(&iommu->lock); - return -EINVAL; - } - - WARN_ON(old_did !=3D pasid_get_domain_id(pte)); - - *pte =3D new_pte; - spin_unlock(&iommu->lock); - - intel_pasid_flush_present(iommu, dev, pasid, old_did, pte); - intel_iommu_drain_pasid_prq(dev, pasid); - - return 0; -} - /* * Set up dirty tracking on a second only or nested translation type. */ @@ -684,38 +595,6 @@ int intel_pasid_setup_pass_through(struct intel_iommu = *iommu, return 0; } =20 -int intel_pasid_replace_pass_through(struct intel_iommu *iommu, - struct device *dev, u16 old_did, - u32 pasid) -{ - struct pasid_entry *pte, new_pte; - u16 did =3D FLPT_DEFAULT_DID; - - pasid_pte_config_pass_through(iommu, &new_pte, did); - - spin_lock(&iommu->lock); - pte =3D intel_pasid_get_entry(dev, pasid); - if (!pte) { - spin_unlock(&iommu->lock); - return -ENODEV; - } - - if (!pasid_pte_is_present(pte)) { - spin_unlock(&iommu->lock); - return -EINVAL; - } - - WARN_ON(old_did !=3D pasid_get_domain_id(pte)); - - *pte =3D new_pte; - spin_unlock(&iommu->lock); - - intel_pasid_flush_present(iommu, dev, pasid, old_did, pte); - intel_iommu_drain_pasid_prq(dev, pasid); - - return 0; -} - /* * Set the page snoop control for a pasid entry which has been set up. */ @@ -849,69 +728,6 @@ int intel_pasid_setup_nested(struct intel_iommu *iommu= , struct device *dev, return 0; } =20 -int intel_pasid_replace_nested(struct intel_iommu *iommu, - struct device *dev, u32 pasid, - u16 old_did, struct dmar_domain *domain) -{ - struct iommu_hwpt_vtd_s1 *s1_cfg =3D &domain->s1_cfg; - struct dmar_domain *s2_domain =3D domain->s2_domain; - u16 did =3D domain_id_iommu(domain, iommu); - struct pasid_entry *pte, new_pte; - - /* Address width should match the address width supported by hardware */ - switch (s1_cfg->addr_width) { - case ADDR_WIDTH_4LEVEL: - break; - case ADDR_WIDTH_5LEVEL: - if (!cap_fl5lp_support(iommu->cap)) { - dev_err_ratelimited(dev, - "5-level paging not supported\n"); - return -EINVAL; - } - break; - default: - dev_err_ratelimited(dev, "Invalid stage-1 address width %d\n", - s1_cfg->addr_width); - return -EINVAL; - } - - if ((s1_cfg->flags & IOMMU_VTD_S1_SRE) && !ecap_srs(iommu->ecap)) { - pr_err_ratelimited("No supervisor request support on %s\n", - iommu->name); - return -EINVAL; - } - - if ((s1_cfg->flags & IOMMU_VTD_S1_EAFE) && !ecap_eafs(iommu->ecap)) { - pr_err_ratelimited("No extended access flag support on %s\n", - iommu->name); - return -EINVAL; - } - - pasid_pte_config_nestd(iommu, &new_pte, s1_cfg, s2_domain, did); - - spin_lock(&iommu->lock); - pte =3D intel_pasid_get_entry(dev, pasid); - if (!pte) { - spin_unlock(&iommu->lock); - return -ENODEV; - } - - if (!pasid_pte_is_present(pte)) { - spin_unlock(&iommu->lock); - return -EINVAL; - } - - WARN_ON(old_did !=3D pasid_get_domain_id(pte)); - - *pte =3D new_pte; - spin_unlock(&iommu->lock); - - intel_pasid_flush_present(iommu, dev, pasid, old_did, pte); - intel_iommu_drain_pasid_prq(dev, pasid); - - return 0; -} - /* * Interfaces to setup or teardown a pasid table to the scalable-mode * context table entry: --=20 2.43.0