From nobody Fri Sep 12 00:58:13 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C7569EB64DD for ; Wed, 9 Aug 2023 12:50:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232195AbjHIMua (ORCPT ); Wed, 9 Aug 2023 08:50:30 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41502 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229737AbjHIMu1 (ORCPT ); Wed, 9 Aug 2023 08:50:27 -0400 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.65]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C549D1BCF for ; Wed, 9 Aug 2023 05:50:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1691585426; x=1723121426; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=b1l7wbez1lIPaZoNpt+IbSB8KZkWFJlDzJWmzqAT8dw=; b=HHCE4uLt5/xDvUR2JT0WAAlKxuwcighiJMifwNJXnmqaEUcRFVAKkX7M zqC3uaCH7QeRjLgcq0h5eyhmpeRos8G4lyxqqkN1cQZCdZ3b4taVCBWRJ 9MqfredKgdbHCvIVvQiVSDN+YcPu0UbkmuqPFDSF84RxnkBPZr2/6a/jG hNDsnfqzqmXNPYlyZuccVYLS14bIv2R7W/HY/hpUJjOVxkavwozlzdyJG qcbt8hwNL7HX0L1rUxUYF2VuBB6mqQz0p0Bk9o/Z9C7RH/A3agSbwtC9O ntSRT0AD3SEySW2bRDCnwLtXdQCnp5lmFn4MVjUOlb6oXSK7BqCTW24Zx w==; X-IronPort-AV: E=McAfee;i="6600,9927,10795"; a="374821670" X-IronPort-AV: E=Sophos;i="6.01,159,1684825200"; d="scan'208";a="374821670" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Aug 2023 05:50:26 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10795"; a="855521652" X-IronPort-AV: E=Sophos;i="6.01,159,1684825200"; d="scan'208";a="855521652" Received: from allen-box.sh.intel.com ([10.239.159.127]) by orsmga004.jf.intel.com with ESMTP; 09 Aug 2023 05:50:24 -0700 From: Lu Baolu To: Joerg Roedel Cc: YueHaibing , Yanfei Xu , Jacob Pan , iommu@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH 01/13] iommu: Generalize PASID 0 for normal DMA w/o PASID Date: Wed, 9 Aug 2023 20:47:54 +0800 Message-Id: <20230809124806.45516-2-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230809124806.45516-1-baolu.lu@linux.intel.com> References: <20230809124806.45516-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Jacob Pan PCIe Process address space ID (PASID) is used to tag DMA traffic, it provides finer grained isolation than requester ID (RID). For each device/RID, 0 is a special PASID for the normal DMA (no PASID). This is universal across all architectures that supports PASID, therefore warranted to be reserved globally and declared in the common header. Consequently, we can avoid the conflict between different PASID use cases in the generic code. e.g. SVA and DMA API with PASIDs. This paved away for device drivers to choose global PASID policy while continue doing normal DMA. Noting that VT-d could support none-zero RID/NO_PASID, but currently not used. Reviewed-by: Lu Baolu Reviewed-by: Kevin Tian Reviewed-by: Jean-Philippe Brucker Reviewed-by: Jason Gunthorpe Signed-off-by: Jacob Pan Link: https://lore.kernel.org/r/20230802212427.1497170-2-jacob.jun.pan@linu= x.intel.com Signed-off-by: Lu Baolu --- include/linux/iommu.h | 1 + drivers/iommu/intel/pasid.h | 2 -- .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 2 +- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 16 ++++++------- drivers/iommu/intel/iommu.c | 24 +++++++++---------- drivers/iommu/intel/pasid.c | 2 +- 6 files changed, 23 insertions(+), 24 deletions(-) diff --git a/include/linux/iommu.h b/include/linux/iommu.h index f1e18e81fca7..e71a4153041d 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -196,6 +196,7 @@ enum iommu_dev_features { IOMMU_DEV_FEAT_IOPF, }; =20 +#define IOMMU_NO_PASID (0U) /* Reserved for DMA w/o PASID */ #define IOMMU_PASID_INVALID (-1U) typedef unsigned int ioasid_t; =20 diff --git a/drivers/iommu/intel/pasid.h b/drivers/iommu/intel/pasid.h index d6b7d21244b1..4e9e68c3c388 100644 --- a/drivers/iommu/intel/pasid.h +++ b/drivers/iommu/intel/pasid.h @@ -10,8 +10,6 @@ #ifndef __INTEL_PASID_H #define __INTEL_PASID_H =20 -#define PASID_RID2PASID 0x0 -#define PASID_MIN 0x1 #define PASID_MAX 0x100000 #define PASID_PTE_MASK 0x3F #define PASID_PTE_PRESENT 1 diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iomm= u/arm/arm-smmu-v3/arm-smmu-v3-sva.c index a5a63b1c947e..5e6b39881c04 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -80,7 +80,7 @@ arm_smmu_share_asid(struct mm_struct *mm, u16 asid) * be some overlap between use of both ASIDs, until we invalidate the * TLB. */ - arm_smmu_write_ctx_desc(smmu_domain, 0, cd); + arm_smmu_write_ctx_desc(smmu_domain, IOMMU_NO_PASID, cd); =20 /* Invalidate TLB entries previously associated with that context */ arm_smmu_tlb_inv_asid(smmu, asid); diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/ar= m/arm-smmu-v3/arm-smmu-v3.c index 9b0dc3505601..ee70687f060b 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -1059,7 +1059,7 @@ int arm_smmu_write_ctx_desc(struct arm_smmu_domain *s= mmu_domain, int ssid, /* * This function handles the following cases: * - * (1) Install primary CD, for normal DMA traffic (SSID =3D 0). + * (1) Install primary CD, for normal DMA traffic (SSID =3D IOMMU_NO_PASI= D =3D 0). * (2) Install a secondary CD, for SID+SSID traffic. * (3) Update ASID of a CD. Atomically write the first 64 bits of the * CD, then invalidate the old entry and mappings. @@ -1607,7 +1607,7 @@ static void arm_smmu_handle_ppr(struct arm_smmu_devic= e *smmu, u64 *evt) =20 sid =3D FIELD_GET(PRIQ_0_SID, evt[0]); ssv =3D FIELD_GET(PRIQ_0_SSID_V, evt[0]); - ssid =3D ssv ? FIELD_GET(PRIQ_0_SSID, evt[0]) : 0; + ssid =3D ssv ? FIELD_GET(PRIQ_0_SSID, evt[0]) : IOMMU_NO_PASID; last =3D FIELD_GET(PRIQ_0_PRG_LAST, evt[0]); grpid =3D FIELD_GET(PRIQ_1_PRG_IDX, evt[1]); =20 @@ -1748,7 +1748,7 @@ arm_smmu_atc_inv_to_cmd(int ssid, unsigned long iova,= size_t size, */ *cmd =3D (struct arm_smmu_cmdq_ent) { .opcode =3D CMDQ_OP_ATC_INV, - .substream_valid =3D !!ssid, + .substream_valid =3D (ssid !=3D IOMMU_NO_PASID), .atc.ssid =3D ssid, }; =20 @@ -1795,7 +1795,7 @@ static int arm_smmu_atc_inv_master(struct arm_smmu_ma= ster *master) struct arm_smmu_cmdq_ent cmd; struct arm_smmu_cmdq_batch cmds; =20 - arm_smmu_atc_inv_to_cmd(0, 0, 0, &cmd); + arm_smmu_atc_inv_to_cmd(IOMMU_NO_PASID, 0, 0, &cmd); =20 cmds.num =3D 0; for (i =3D 0; i < master->num_streams; i++) { @@ -1875,7 +1875,7 @@ static void arm_smmu_tlb_inv_context(void *cookie) cmd.tlbi.vmid =3D smmu_domain->s2_cfg.vmid; arm_smmu_cmdq_issue_cmd_with_sync(smmu, &cmd); } - arm_smmu_atc_inv_domain(smmu_domain, 0, 0, 0); + arm_smmu_atc_inv_domain(smmu_domain, IOMMU_NO_PASID, 0, 0); } =20 static void __arm_smmu_tlb_inv_range(struct arm_smmu_cmdq_ent *cmd, @@ -1968,7 +1968,7 @@ static void arm_smmu_tlb_inv_range_domain(unsigned lo= ng iova, size_t size, * Unfortunately, this can't be leaf-only since we may have * zapped an entire table. */ - arm_smmu_atc_inv_domain(smmu_domain, 0, iova, size); + arm_smmu_atc_inv_domain(smmu_domain, IOMMU_NO_PASID, iova, size); } =20 void arm_smmu_tlb_inv_range_asid(unsigned long iova, size_t size, int asid, @@ -2142,7 +2142,7 @@ static int arm_smmu_domain_finalise_s1(struct arm_smm= u_domain *smmu_domain, * the master has been added to the devices list for this domain. * This isn't an issue because the STE hasn't been installed yet. */ - ret =3D arm_smmu_write_ctx_desc(smmu_domain, 0, &cfg->cd); + ret =3D arm_smmu_write_ctx_desc(smmu_domain, IOMMU_NO_PASID, &cfg->cd); if (ret) goto out_free_cd_tables; =20 @@ -2328,7 +2328,7 @@ static void arm_smmu_enable_ats(struct arm_smmu_maste= r *master) pdev =3D to_pci_dev(master->dev); =20 atomic_inc(&smmu_domain->nr_ats_masters); - arm_smmu_atc_inv_domain(smmu_domain, 0, 0, 0); + arm_smmu_atc_inv_domain(smmu_domain, IOMMU_NO_PASID, 0, 0); if (pci_enable_ats(pdev, stu)) dev_err(master->dev, "Failed to enable ATS (STU %zu)\n", stu); } diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c index d5ca2387e65c..89013a2913af 100644 --- a/drivers/iommu/intel/iommu.c +++ b/drivers/iommu/intel/iommu.c @@ -877,7 +877,7 @@ void dmar_fault_dump_ptes(struct intel_iommu *iommu, u1= 6 source_id, } /* For request-without-pasid, get the pasid from context entry */ if (intel_iommu_sm && pasid =3D=3D IOMMU_PASID_INVALID) - pasid =3D PASID_RID2PASID; + pasid =3D IOMMU_NO_PASID; =20 dir_index =3D pasid >> PASID_PDE_SHIFT; pde =3D &dir[dir_index]; @@ -1449,7 +1449,7 @@ static void __iommu_flush_dev_iotlb(struct device_dom= ain_info *info, qdep =3D info->ats_qdep; qi_flush_dev_iotlb(info->iommu, sid, info->pfsid, qdep, addr, mask); - quirk_extra_dev_tlb_flush(info, addr, mask, PASID_RID2PASID, qdep); + quirk_extra_dev_tlb_flush(info, addr, mask, IOMMU_NO_PASID, qdep); } =20 static void iommu_flush_dev_iotlb(struct dmar_domain *domain, @@ -1484,7 +1484,7 @@ static void iommu_flush_iotlb_psi(struct intel_iommu = *iommu, ih =3D 1 << 6; =20 if (domain->use_first_level) { - qi_flush_piotlb(iommu, did, PASID_RID2PASID, addr, pages, ih); + qi_flush_piotlb(iommu, did, IOMMU_NO_PASID, addr, pages, ih); } else { unsigned long bitmask =3D aligned_pages - 1; =20 @@ -1554,7 +1554,7 @@ static void intel_flush_iotlb_all(struct iommu_domain= *domain) u16 did =3D domain_id_iommu(dmar_domain, iommu); =20 if (dmar_domain->use_first_level) - qi_flush_piotlb(iommu, did, PASID_RID2PASID, 0, -1, 0); + qi_flush_piotlb(iommu, did, IOMMU_NO_PASID, 0, -1, 0); else iommu->flush.flush_iotlb(iommu, did, 0, 0, DMA_TLB_DSI_FLUSH); @@ -1940,7 +1940,7 @@ static int domain_context_mapping_one(struct dmar_dom= ain *domain, context_pdts(pds); =20 /* Setup the RID_PASID field: */ - context_set_sm_rid2pasid(context, PASID_RID2PASID); + context_set_sm_rid2pasid(context, IOMMU_NO_PASID); =20 /* * Setup the Device-TLB enable bit and Page request @@ -2420,13 +2420,13 @@ static int dmar_domain_attach_device(struct dmar_do= main *domain, /* Setup the PASID entry for requests without PASID: */ if (hw_pass_through && domain_type_is_si(domain)) ret =3D intel_pasid_setup_pass_through(iommu, domain, - dev, PASID_RID2PASID); + dev, IOMMU_NO_PASID); else if (domain->use_first_level) ret =3D domain_setup_first_level(iommu, domain, dev, - PASID_RID2PASID); + IOMMU_NO_PASID); else ret =3D intel_pasid_setup_second_level(iommu, domain, - dev, PASID_RID2PASID); + dev, IOMMU_NO_PASID); if (ret) { dev_err(dev, "Setup RID2PASID failed\n"); device_block_translation(dev); @@ -3961,7 +3961,7 @@ static void dmar_remove_one_dev_info(struct device *d= ev) if (!dev_is_real_dma_subdevice(info->dev)) { if (dev_is_pci(info->dev) && sm_supported(iommu)) intel_pasid_tear_down_entry(iommu, info->dev, - PASID_RID2PASID, false); + IOMMU_NO_PASID, false); =20 iommu_disable_pci_caps(info); domain_context_clear(info); @@ -3990,7 +3990,7 @@ static void device_block_translation(struct device *d= ev) if (!dev_is_real_dma_subdevice(dev)) { if (sm_supported(iommu)) intel_pasid_tear_down_entry(iommu, dev, - PASID_RID2PASID, false); + IOMMU_NO_PASID, false); else domain_context_clear(info); } @@ -4324,7 +4324,7 @@ static void domain_set_force_snooping(struct dmar_dom= ain *domain) =20 list_for_each_entry(info, &domain->devices, link) intel_pasid_setup_page_snoop_control(info->iommu, info->dev, - PASID_RID2PASID); + IOMMU_NO_PASID); } =20 static bool intel_iommu_enforce_cache_coherency(struct iommu_domain *domai= n) @@ -4980,7 +4980,7 @@ void quirk_extra_dev_tlb_flush(struct device_domain_i= nfo *info, return; =20 sid =3D PCI_DEVID(info->bus, info->devfn); - if (pasid =3D=3D PASID_RID2PASID) { + if (pasid =3D=3D IOMMU_NO_PASID) { qi_flush_dev_iotlb(info->iommu, sid, info->pfsid, qdep, address, mask); } else { diff --git a/drivers/iommu/intel/pasid.c b/drivers/iommu/intel/pasid.c index c5d479770e12..23dca3bc319d 100644 --- a/drivers/iommu/intel/pasid.c +++ b/drivers/iommu/intel/pasid.c @@ -438,7 +438,7 @@ devtlb_invalidation_with_pasid(struct intel_iommu *iomm= u, * SVA usage, device could do DMA with multiple PASIDs. It is more * efficient to flush devTLB specific to the PASID. */ - if (pasid =3D=3D PASID_RID2PASID) + if (pasid =3D=3D IOMMU_NO_PASID) qi_flush_dev_iotlb(iommu, sid, pfsid, qdep, 0, 64 - VTD_PAGE_SHIFT); else qi_flush_dev_iotlb_pasid(iommu, sid, pfsid, pasid, qdep, 0, 64 - VTD_PAG= E_SHIFT); --=20 2.34.1 From nobody Fri Sep 12 00:58:13 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BF665C04A6A for ; Wed, 9 Aug 2023 12:50:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232458AbjHIMuc (ORCPT ); Wed, 9 Aug 2023 08:50:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41516 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231980AbjHIMu3 (ORCPT ); Wed, 9 Aug 2023 08:50:29 -0400 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.65]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 56D0F1FD4 for ; Wed, 9 Aug 2023 05:50:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1691585428; x=1723121428; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=fbUepnffesza+o/B0zOxYfWkMXam385Ymd7dnEQ5QDA=; b=YqXBT57n9WTe/U16OjF49JNgDafUUQWL7gfjkUIKREu/S5zST3Hom/B/ gh4XPGxDr91020AQhDu1i8rY6OoorQuUq56DpNGOt8xpq8MZIGz+vHljF Bnofyzo0hLSEPX9e5KNjKkvHnVcIg2+2B/KcHdSPSqTzR4O5Ii1afySP7 0xsTHKryddD6gjT4ShWcTJ7PXEToyfoqgDu45Nswv6W0fi4HXmjmAOSuJ j0HpX55H0hXru+ptbZaInlVx9OOiLPF5c6Bo0UCAZAopGiryVPDfdOwUS WqFp+E4aBXJLEgGWY9boOHN4GIUSi/IESD4qstO3rsRHNrrzUOwTolR/A Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10795"; a="374821676" X-IronPort-AV: E=Sophos;i="6.01,159,1684825200"; d="scan'208";a="374821676" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Aug 2023 05:50:28 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10795"; a="855521675" X-IronPort-AV: E=Sophos;i="6.01,159,1684825200"; d="scan'208";a="855521675" Received: from allen-box.sh.intel.com ([10.239.159.127]) by orsmga004.jf.intel.com with ESMTP; 09 Aug 2023 05:50:26 -0700 From: Lu Baolu To: Joerg Roedel Cc: YueHaibing , Yanfei Xu , Jacob Pan , iommu@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH 02/13] iommu: Move global PASID allocation from SVA to core Date: Wed, 9 Aug 2023 20:47:55 +0800 Message-Id: <20230809124806.45516-3-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230809124806.45516-1-baolu.lu@linux.intel.com> References: <20230809124806.45516-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Jacob Pan Intel ENQCMD requires a single PASID to be shared between multiple devices, as the PASID is stored in a single MSR register per-process and userspace can use only that one PASID. This means that the PASID allocation for any ENQCMD using device driver must always come from a shared global pool, regardless of what kind of domain the PASID will be used with. Split the code for the global PASID allocator into iommu_alloc/free_global_pasid() so that drivers can attach non-SVA domains to PASIDs as well. This patch moves global PASID allocation APIs from SVA to IOMMU APIs. Reserved PASIDs, currently only RID_PASID, are excluded from the global PASID allocation. It is expected that device drivers will use the allocated PASIDs to attach to appropriate IOMMU domains for use. Reviewed-by: Lu Baolu Reviewed-by: Kevin Tian Reviewed-by: Jason Gunthorpe Signed-off-by: Jacob Pan Link: https://lore.kernel.org/r/20230802212427.1497170-3-jacob.jun.pan@linu= x.intel.com Signed-off-by: Lu Baolu --- include/linux/iommu.h | 10 ++++++++++ drivers/iommu/iommu-sva.c | 29 ++++++++++------------------- drivers/iommu/iommu.c | 28 ++++++++++++++++++++++++++++ 3 files changed, 48 insertions(+), 19 deletions(-) diff --git a/include/linux/iommu.h b/include/linux/iommu.h index e71a4153041d..4cb584b08b81 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -197,6 +197,7 @@ enum iommu_dev_features { }; =20 #define IOMMU_NO_PASID (0U) /* Reserved for DMA w/o PASID */ +#define IOMMU_FIRST_GLOBAL_PASID (1U) /*starting range for allocation */ #define IOMMU_PASID_INVALID (-1U) typedef unsigned int ioasid_t; =20 @@ -733,6 +734,8 @@ void iommu_detach_device_pasid(struct iommu_domain *dom= ain, struct iommu_domain * iommu_get_domain_for_dev_pasid(struct device *dev, ioasid_t pasid, unsigned int type); +ioasid_t iommu_alloc_global_pasid(struct device *dev); +void iommu_free_global_pasid(ioasid_t pasid); #else /* CONFIG_IOMMU_API */ =20 struct iommu_ops {}; @@ -1094,6 +1097,13 @@ iommu_get_domain_for_dev_pasid(struct device *dev, i= oasid_t pasid, { return NULL; } + +static inline ioasid_t iommu_alloc_global_pasid(struct device *dev) +{ + return IOMMU_PASID_INVALID; +} + +static inline void iommu_free_global_pasid(ioasid_t pasid) {} #endif /* CONFIG_IOMMU_API */ =20 /** diff --git a/drivers/iommu/iommu-sva.c b/drivers/iommu/iommu-sva.c index 05c0fb2acbc4..b78671a8a914 100644 --- a/drivers/iommu/iommu-sva.c +++ b/drivers/iommu/iommu-sva.c @@ -10,34 +10,30 @@ #include "iommu-sva.h" =20 static DEFINE_MUTEX(iommu_sva_lock); -static DEFINE_IDA(iommu_global_pasid_ida); =20 /* Allocate a PASID for the mm within range (inclusive) */ -static int iommu_sva_alloc_pasid(struct mm_struct *mm, ioasid_t min, ioasi= d_t max) +static int iommu_sva_alloc_pasid(struct mm_struct *mm, struct device *dev) { + ioasid_t pasid; int ret =3D 0; =20 - if (min =3D=3D IOMMU_PASID_INVALID || - max =3D=3D IOMMU_PASID_INVALID || - min =3D=3D 0 || max < min) - return -EINVAL; - if (!arch_pgtable_dma_compat(mm)) return -EBUSY; =20 mutex_lock(&iommu_sva_lock); /* Is a PASID already associated with this mm? */ if (mm_valid_pasid(mm)) { - if (mm->pasid < min || mm->pasid > max) + if (mm->pasid >=3D dev->iommu->max_pasids) ret =3D -EOVERFLOW; goto out; } =20 - ret =3D ida_alloc_range(&iommu_global_pasid_ida, min, max, GFP_KERNEL); - if (ret < 0) + pasid =3D iommu_alloc_global_pasid(dev); + if (pasid =3D=3D IOMMU_PASID_INVALID) { + ret =3D -ENOSPC; goto out; - - mm->pasid =3D ret; + } + mm->pasid =3D pasid; ret =3D 0; out: mutex_unlock(&iommu_sva_lock); @@ -64,15 +60,10 @@ struct iommu_sva *iommu_sva_bind_device(struct device *= dev, struct mm_struct *mm { struct iommu_domain *domain; struct iommu_sva *handle; - ioasid_t max_pasids; int ret; =20 - max_pasids =3D dev->iommu->max_pasids; - if (!max_pasids) - return ERR_PTR(-EOPNOTSUPP); - /* Allocate mm->pasid if necessary. */ - ret =3D iommu_sva_alloc_pasid(mm, 1, max_pasids - 1); + ret =3D iommu_sva_alloc_pasid(mm, dev); if (ret) return ERR_PTR(ret); =20 @@ -217,5 +208,5 @@ void mm_pasid_drop(struct mm_struct *mm) if (likely(!mm_valid_pasid(mm))) return; =20 - ida_free(&iommu_global_pasid_ida, mm->pasid); + iommu_free_global_pasid(mm->pasid); } diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c index 25d7327e8013..738fee1a24f7 100644 --- a/drivers/iommu/iommu.c +++ b/drivers/iommu/iommu.c @@ -39,6 +39,7 @@ =20 static struct kset *iommu_group_kset; static DEFINE_IDA(iommu_group_ida); +static DEFINE_IDA(iommu_global_pasid_ida); =20 static unsigned int iommu_def_domain_type __read_mostly; static bool iommu_dma_strict __read_mostly =3D IS_ENABLED(CONFIG_IOMMU_DEF= AULT_DMA_STRICT); @@ -3453,3 +3454,30 @@ struct iommu_domain *iommu_sva_domain_alloc(struct d= evice *dev, =20 return domain; } + +ioasid_t iommu_alloc_global_pasid(struct device *dev) +{ + int ret; + + /* max_pasids =3D=3D 0 means that the device does not support PASID */ + if (!dev->iommu->max_pasids) + return IOMMU_PASID_INVALID; + + /* + * max_pasids is set up by vendor driver based on number of PASID bits + * supported but the IDA allocation is inclusive. + */ + ret =3D ida_alloc_range(&iommu_global_pasid_ida, IOMMU_FIRST_GLOBAL_PASID, + dev->iommu->max_pasids - 1, GFP_KERNEL); + return ret < 0 ? IOMMU_PASID_INVALID : ret; +} +EXPORT_SYMBOL_GPL(iommu_alloc_global_pasid); + +void iommu_free_global_pasid(ioasid_t pasid) +{ + if (WARN_ON(pasid =3D=3D IOMMU_PASID_INVALID)) + return; + + ida_free(&iommu_global_pasid_ida, pasid); +} +EXPORT_SYMBOL_GPL(iommu_free_global_pasid); --=20 2.34.1 From nobody Fri Sep 12 00:58:13 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0776AEB64DD for ; Wed, 9 Aug 2023 12:50:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232605AbjHIMuf (ORCPT ); Wed, 9 Aug 2023 08:50:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52970 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232341AbjHIMua (ORCPT ); Wed, 9 Aug 2023 08:50:30 -0400 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.65]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1C7281FCC for ; Wed, 9 Aug 2023 05:50:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1691585430; x=1723121430; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Rp/t14IpXr23atoHWSRr0mV1PUySUm2ThGD6KeIHYNg=; b=aOU5dluLB49aDHzt1doTDqbuW6HejTiq+iElgfQnlgsQc19+SRvMQvJw 6rMg9vNnGm4RKV70ElfuM+Ptao1WbpOlhtwbB1xWaC+ktzEgR/A71sPEJ I/PYN2mmOpFDvMkntVEMcKTHxCsiIN/BbMmbDDAx5jP5138dQ1uP4AkeM e45JZ4+AQ6HTjS5xoGHL9gXfBEeMqBIkJerMrksgATtLoiU5hS+FUn/zn rHCtc65p8HuBx3KZ4F4D0EhIRdquHrwRl1HIM5wNCMviydREp/EUb+Iwc eRcCEnhwzPhuWApTzHbSVB+mHGpbk/tR/Lgr5QOLsj6AFzfDCCCNj10z9 A==; X-IronPort-AV: E=McAfee;i="6600,9927,10795"; a="374821687" X-IronPort-AV: E=Sophos;i="6.01,159,1684825200"; d="scan'208";a="374821687" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Aug 2023 05:50:29 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10795"; a="855521688" X-IronPort-AV: E=Sophos;i="6.01,159,1684825200"; d="scan'208";a="855521688" Received: from allen-box.sh.intel.com ([10.239.159.127]) by orsmga004.jf.intel.com with ESMTP; 09 Aug 2023 05:50:28 -0700 From: Lu Baolu To: Joerg Roedel Cc: YueHaibing , Yanfei Xu , Jacob Pan , iommu@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH 03/13] iommu/vt-d: Add domain_flush_pasid_iotlb() Date: Wed, 9 Aug 2023 20:47:56 +0800 Message-Id: <20230809124806.45516-4-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230809124806.45516-1-baolu.lu@linux.intel.com> References: <20230809124806.45516-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The VT-d spec requires to use PASID-based-IOTLB invalidation descriptor to invalidate IOTLB and the paging-structure caches for a first-stage page table. Add a generic helper to do this. RID2PASID is used if the domain has been attached to a physical device, otherwise real PASIDs that the domain has been attached to will be used. The 'real' PASID attachment is handled in the subsequent change. Signed-off-by: Lu Baolu Signed-off-by: Jacob Pan Reviewed-by: Kevin Tian Link: https://lore.kernel.org/r/20230802212427.1497170-4-jacob.jun.pan@linu= x.intel.com --- drivers/iommu/intel/iommu.c | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c index 89013a2913af..bb8316fec1aa 100644 --- a/drivers/iommu/intel/iommu.c +++ b/drivers/iommu/intel/iommu.c @@ -1467,6 +1467,18 @@ static void iommu_flush_dev_iotlb(struct dmar_domain= *domain, spin_unlock_irqrestore(&domain->lock, flags); } =20 +static void domain_flush_pasid_iotlb(struct intel_iommu *iommu, + struct dmar_domain *domain, u64 addr, + unsigned long npages, bool ih) +{ + u16 did =3D domain_id_iommu(domain, iommu); + unsigned long flags; + + spin_lock_irqsave(&domain->lock, flags); + qi_flush_piotlb(iommu, did, IOMMU_NO_PASID, addr, npages, ih); + spin_unlock_irqrestore(&domain->lock, flags); +} + static void iommu_flush_iotlb_psi(struct intel_iommu *iommu, struct dmar_domain *domain, unsigned long pfn, unsigned int pages, @@ -1484,7 +1496,7 @@ static void iommu_flush_iotlb_psi(struct intel_iommu = *iommu, ih =3D 1 << 6; =20 if (domain->use_first_level) { - qi_flush_piotlb(iommu, did, IOMMU_NO_PASID, addr, pages, ih); + domain_flush_pasid_iotlb(iommu, domain, addr, pages, ih); } else { unsigned long bitmask =3D aligned_pages - 1; =20 @@ -1554,7 +1566,7 @@ static void intel_flush_iotlb_all(struct iommu_domain= *domain) u16 did =3D domain_id_iommu(dmar_domain, iommu); =20 if (dmar_domain->use_first_level) - qi_flush_piotlb(iommu, did, IOMMU_NO_PASID, 0, -1, 0); + domain_flush_pasid_iotlb(iommu, dmar_domain, 0, -1, 0); else iommu->flush.flush_iotlb(iommu, did, 0, 0, DMA_TLB_DSI_FLUSH); --=20 2.34.1 From nobody Fri Sep 12 00:58:13 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 78AA3EB64DD for ; Wed, 9 Aug 2023 12:50:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232664AbjHIMui (ORCPT ); Wed, 9 Aug 2023 08:50:38 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53018 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232596AbjHIMuc (ORCPT ); Wed, 9 Aug 2023 08:50:32 -0400 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.65]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0BB271FD4 for ; Wed, 9 Aug 2023 05:50:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1691585432; x=1723121432; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=DYxnh/kP9R8JBvFnqsi/X/T1RmcQV+dkPJCY2p+HqgM=; b=IY1PsioQTbd2OM5Pk/Pbong5lEIn5vBxaOoMJ0LTUI7EG851z+JNiptk 9XP3NkP0bRoPcRY/FBNTmKPQBKDJcG5xyJu/sDc9vP8tCYUUTEVS/27ke vGhfVEZkaeVi6g/aC27CMHyy4vl6cQE3OtEWC2zuQt/QIRPdBJog73hPm hHKwrlUV0TUl1lWIUkKIu9dY1SQVRJ9K7hnWbPqdUtrMfvqljzcFx+y9C URIS1yFlc8Y/8/F5XXiy+L7tq/gsJNH9+INenEHu7Uxqx5P8mSt4Cqbas +2ALMJ0MBSB+kmBLqndeTN+mb71raffW59JCDEMP1TA0P9RavvNzgK4mG Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10795"; a="374821699" X-IronPort-AV: E=Sophos;i="6.01,159,1684825200"; d="scan'208";a="374821699" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Aug 2023 05:50:31 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10795"; a="855521711" X-IronPort-AV: E=Sophos;i="6.01,159,1684825200"; d="scan'208";a="855521711" Received: from allen-box.sh.intel.com ([10.239.159.127]) by orsmga004.jf.intel.com with ESMTP; 09 Aug 2023 05:50:30 -0700 From: Lu Baolu To: Joerg Roedel Cc: YueHaibing , Yanfei Xu , Jacob Pan , iommu@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH 04/13] iommu/vt-d: Remove pasid_mutex Date: Wed, 9 Aug 2023 20:47:57 +0800 Message-Id: <20230809124806.45516-5-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230809124806.45516-1-baolu.lu@linux.intel.com> References: <20230809124806.45516-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The pasid_mutex was used to protect the paths of set/remove_dev_pasid(). It's duplicate with iommu_sva_lock. Remove it to avoid duplicate code. Signed-off-by: Lu Baolu Signed-off-by: Jacob Pan Reviewed-by: Kevin Tian Link: https://lore.kernel.org/r/20230802212427.1497170-5-jacob.jun.pan@linu= x.intel.com --- drivers/iommu/intel/svm.c | 45 +++++---------------------------------- 1 file changed, 5 insertions(+), 40 deletions(-) diff --git a/drivers/iommu/intel/svm.c b/drivers/iommu/intel/svm.c index e95b339e9cdc..2a82864e9d57 100644 --- a/drivers/iommu/intel/svm.c +++ b/drivers/iommu/intel/svm.c @@ -259,8 +259,6 @@ static const struct mmu_notifier_ops intel_mmuops =3D { .invalidate_range =3D intel_invalidate_range, }; =20 -static DEFINE_MUTEX(pasid_mutex); - static int pasid_to_svm_sdev(struct device *dev, unsigned int pasid, struct intel_svm **rsvm, struct intel_svm_dev **rsdev) @@ -268,10 +266,6 @@ static int pasid_to_svm_sdev(struct device *dev, unsig= ned int pasid, struct intel_svm_dev *sdev =3D NULL; struct intel_svm *svm; =20 - /* The caller should hold the pasid_mutex lock */ - if (WARN_ON(!mutex_is_locked(&pasid_mutex))) - return -EINVAL; - if (pasid =3D=3D IOMMU_PASID_INVALID || pasid >=3D PASID_MAX) return -EINVAL; =20 @@ -371,22 +365,19 @@ static int intel_svm_bind_mm(struct intel_iommu *iomm= u, struct device *dev, return ret; } =20 -/* Caller must hold pasid_mutex */ -static int intel_svm_unbind_mm(struct device *dev, u32 pasid) +void intel_svm_remove_dev_pasid(struct device *dev, u32 pasid) { struct intel_svm_dev *sdev; struct intel_iommu *iommu; struct intel_svm *svm; struct mm_struct *mm; - int ret =3D -EINVAL; =20 iommu =3D device_to_iommu(dev, NULL, NULL); if (!iommu) - goto out; + return; =20 - ret =3D pasid_to_svm_sdev(dev, pasid, &svm, &sdev); - if (ret) - goto out; + if (pasid_to_svm_sdev(dev, pasid, &svm, &sdev)) + return; mm =3D svm->mm; =20 if (sdev) { @@ -418,8 +409,6 @@ static int intel_svm_unbind_mm(struct device *dev, u32 = pasid) kfree(svm); } } -out: - return ret; } =20 /* Page request queue descriptor */ @@ -520,19 +509,7 @@ static void intel_svm_drain_prq(struct device *dev, u3= 2 pasid) goto prq_retry; } =20 - /* - * A work in IO page fault workqueue may try to lock pasid_mutex now. - * Holding pasid_mutex while waiting in iopf_queue_flush_dev() for - * all works in the workqueue to finish may cause deadlock. - * - * It's unnecessary to hold pasid_mutex in iopf_queue_flush_dev(). - * Unlock it to allow the works to be handled while waiting for - * them to finish. - */ - lockdep_assert_held(&pasid_mutex); - mutex_unlock(&pasid_mutex); iopf_queue_flush_dev(dev); - mutex_lock(&pasid_mutex); =20 /* * Perform steps described in VT-d spec CH7.10 to drain page @@ -827,26 +804,14 @@ int intel_svm_page_response(struct device *dev, return ret; } =20 -void intel_svm_remove_dev_pasid(struct device *dev, ioasid_t pasid) -{ - mutex_lock(&pasid_mutex); - intel_svm_unbind_mm(dev, pasid); - mutex_unlock(&pasid_mutex); -} - static int intel_svm_set_dev_pasid(struct iommu_domain *domain, struct device *dev, ioasid_t pasid) { struct device_domain_info *info =3D dev_iommu_priv_get(dev); struct intel_iommu *iommu =3D info->iommu; struct mm_struct *mm =3D domain->mm; - int ret; =20 - mutex_lock(&pasid_mutex); - ret =3D intel_svm_bind_mm(iommu, dev, mm); - mutex_unlock(&pasid_mutex); - - return ret; + return intel_svm_bind_mm(iommu, dev, mm); } =20 static void intel_svm_domain_free(struct iommu_domain *domain) --=20 2.34.1 From nobody Fri Sep 12 00:58:13 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8F6AEC0015E for ; Wed, 9 Aug 2023 12:50:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232635AbjHIMul (ORCPT ); Wed, 9 Aug 2023 08:50:41 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53062 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232623AbjHIMuf (ORCPT ); Wed, 9 Aug 2023 08:50:35 -0400 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.65]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DC67E2108 for ; Wed, 9 Aug 2023 05:50:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1691585433; x=1723121433; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Q97WzXCK6i2hC49d5NozMEhqvYe2vprBvZi0/iS7G6U=; b=Uj/+5GH/YULMRrUtnvRpJex4o3zv8oE45FgDtvDdKwYkuaydV+4ygLFj Le7R1QMg70D9XX1ZIk/VmY/Od1tw7XES4mMHXm6Vi2NefeAPx9zPlfaE7 d2CIXgfpZXCklvmq8NGPcdbvxRDvijPuMpzLlX4ZM904HBPimORtUL8vj F1cBIEodd1rVKWyussDvNTRQ4tIGr0dlpi+9V9JVXI1ywU3LWCftFx7ph myrrNjFrFisk/plAb6j2dqFCflmzUNdsX3pEoW/SEwExDDcjJTg4ukhQe hygpJC7tQ83IPLRMCsyTrNl28C0ZkRyddpoYYcHbzBrPk5Ee1QUuhH7U2 Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10795"; a="374821709" X-IronPort-AV: E=Sophos;i="6.01,159,1684825200"; d="scan'208";a="374821709" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Aug 2023 05:50:33 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10795"; a="855521729" X-IronPort-AV: E=Sophos;i="6.01,159,1684825200"; d="scan'208";a="855521729" Received: from allen-box.sh.intel.com ([10.239.159.127]) by orsmga004.jf.intel.com with ESMTP; 09 Aug 2023 05:50:32 -0700 From: Lu Baolu To: Joerg Roedel Cc: YueHaibing , Yanfei Xu , Jacob Pan , iommu@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH 05/13] iommu/vt-d: Make prq draining code generic Date: Wed, 9 Aug 2023 20:47:58 +0800 Message-Id: <20230809124806.45516-6-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230809124806.45516-1-baolu.lu@linux.intel.com> References: <20230809124806.45516-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Currently draining page requests and responses for a pasid is part of SVA implementation. This is because the driver only supports attaching an SVA domain to a device pasid. As we are about to support attaching other types of domains to a device pasid, the prq draining code becomes generic. Signed-off-by: Lu Baolu Signed-off-by: Jacob Pan Reviewed-by: Kevin Tian Link: https://lore.kernel.org/r/20230802212427.1497170-6-jacob.jun.pan@linu= x.intel.com --- drivers/iommu/intel/iommu.h | 2 ++ drivers/iommu/intel/iommu.c | 30 +++++++++++++++++++----------- drivers/iommu/intel/svm.c | 17 ++--------------- 3 files changed, 23 insertions(+), 26 deletions(-) diff --git a/drivers/iommu/intel/iommu.h b/drivers/iommu/intel/iommu.h index 1c5e1d88862b..6d94a29f5d52 100644 --- a/drivers/iommu/intel/iommu.h +++ b/drivers/iommu/intel/iommu.h @@ -844,6 +844,7 @@ int intel_svm_page_response(struct device *dev, struct = iommu_fault_event *evt, struct iommu_page_response *msg); struct iommu_domain *intel_svm_domain_alloc(void); void intel_svm_remove_dev_pasid(struct device *dev, ioasid_t pasid); +void intel_drain_pasid_prq(struct device *dev, u32 pasid); =20 struct intel_svm_dev { struct list_head list; @@ -862,6 +863,7 @@ struct intel_svm { }; #else static inline void intel_svm_check(struct intel_iommu *iommu) {} +static inline void intel_drain_pasid_prq(struct device *dev, u32 pasid) {} static inline struct iommu_domain *intel_svm_domain_alloc(void) { return NULL; diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c index bb8316fec1aa..bdde38a5e43a 100644 --- a/drivers/iommu/intel/iommu.c +++ b/drivers/iommu/intel/iommu.c @@ -4720,21 +4720,29 @@ static void intel_iommu_remove_dev_pasid(struct dev= ice *dev, ioasid_t pasid) struct intel_iommu *iommu =3D device_to_iommu(dev, NULL, NULL); struct iommu_domain *domain; =20 - /* Domain type specific cleanup: */ domain =3D iommu_get_domain_for_dev_pasid(dev, pasid, 0); - if (domain) { - switch (domain->type) { - case IOMMU_DOMAIN_SVA: - intel_svm_remove_dev_pasid(dev, pasid); - break; - default: - /* should never reach here */ - WARN_ON(1); - break; - } + if (WARN_ON_ONCE(!domain)) + goto out_tear_down; + + /* + * The SVA implementation needs to handle its own stuffs like the mm + * notification. Before consolidating that code into iommu core, let + * the intel sva code handle it. + */ + if (domain->type =3D=3D IOMMU_DOMAIN_SVA) { + intel_svm_remove_dev_pasid(dev, pasid); + goto out_tear_down; } =20 + /* + * Should never reach here until we add support for attaching + * non-SVA domain to a pasid. + */ + WARN_ON(1); + +out_tear_down: intel_pasid_tear_down_entry(iommu, dev, pasid, false); + intel_drain_pasid_prq(dev, pasid); } =20 const struct iommu_ops intel_iommu_ops =3D { diff --git a/drivers/iommu/intel/svm.c b/drivers/iommu/intel/svm.c index 2a82864e9d57..9fbae9af6615 100644 --- a/drivers/iommu/intel/svm.c +++ b/drivers/iommu/intel/svm.c @@ -26,8 +26,6 @@ #include "trace.h" =20 static irqreturn_t prq_event_thread(int irq, void *d); -static void intel_svm_drain_prq(struct device *dev, u32 pasid); -#define to_intel_svm_dev(handle) container_of(handle, struct intel_svm_dev= , sva) =20 static DEFINE_XARRAY_ALLOC(pasid_private_array); static int pasid_private_add(ioasid_t pasid, void *priv) @@ -382,17 +380,6 @@ void intel_svm_remove_dev_pasid(struct device *dev, u3= 2 pasid) =20 if (sdev) { list_del_rcu(&sdev->list); - /* - * Flush the PASID cache and IOTLB for this device. - * Note that we do depend on the hardware *not* using - * the PASID any more. Just as we depend on other - * devices never using PASIDs that they have no right - * to use. We have a *shared* PASID table, because it's - * large and has to be physically contiguous. So it's - * hard to be as defensive as we might like. - */ - intel_pasid_tear_down_entry(iommu, dev, svm->pasid, false); - intel_svm_drain_prq(dev, svm->pasid); kfree_rcu(sdev, rcu); =20 if (list_empty(&svm->devs)) { @@ -449,7 +436,7 @@ static bool is_canonical_address(u64 addr) } =20 /** - * intel_svm_drain_prq - Drain page requests and responses for a pasid + * intel_drain_pasid_prq - Drain page requests and responses for a pasid * @dev: target device * @pasid: pasid for draining * @@ -463,7 +450,7 @@ static bool is_canonical_address(u64 addr) * described in VT-d spec CH7.10 to drain all page requests and page * responses pending in the hardware. */ -static void intel_svm_drain_prq(struct device *dev, u32 pasid) +void intel_drain_pasid_prq(struct device *dev, u32 pasid) { struct device_domain_info *info; struct dmar_domain *domain; --=20 2.34.1 From nobody Fri Sep 12 00:58:13 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EB0CDC0015E for ; Wed, 9 Aug 2023 12:50:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232659AbjHIMuu (ORCPT ); Wed, 9 Aug 2023 08:50:50 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53062 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232642AbjHIMuh (ORCPT ); Wed, 9 Aug 2023 08:50:37 -0400 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.65]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BA27D2113 for ; Wed, 9 Aug 2023 05:50:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1691585435; x=1723121435; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=BbJI+sqyl8iiSSnSlKYkiNQ2OWmfelGb3IWm5XO5axA=; b=gzvVl3Yg/lRvyJBrBIHmiS84O/yF9EKTNmJx2N9qNP5mD25ZDlM0rZ1I MfFQ4FUqAgyvKnH5s41Aq1kig7UT6GbbFo8HdUL3YCnfT5zXihu8YBrLq 8rHTf34l03enm2jyBTkNSLh4DuEdLnSRhGPCtxoNGGETpb5RA1NUfcRRk Ww2v/LEpgpKaws9mPO7WeQKKCjPJAzJyQCTO7h2OmppZ1Y9Q6P+rZlnuM wuCpBTAwuMdUaUtc57gO4YjyZCEGfuAjpoXADbgETvzr1S9GZ5v8jAjOZ aAsrGCO2tqUsbWLq06B948oWI2krIJKBgjRHUmzZ1OkwlTnsghDTvK56G Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10795"; a="374821717" X-IronPort-AV: E=Sophos;i="6.01,159,1684825200"; d="scan'208";a="374821717" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Aug 2023 05:50:35 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10795"; a="855521745" X-IronPort-AV: E=Sophos;i="6.01,159,1684825200"; d="scan'208";a="855521745" Received: from allen-box.sh.intel.com ([10.239.159.127]) by orsmga004.jf.intel.com with ESMTP; 09 Aug 2023 05:50:33 -0700 From: Lu Baolu To: Joerg Roedel Cc: YueHaibing , Yanfei Xu , Jacob Pan , iommu@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH 06/13] iommu/vt-d: Prepare for set_dev_pasid callback Date: Wed, 9 Aug 2023 20:47:59 +0800 Message-Id: <20230809124806.45516-7-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230809124806.45516-1-baolu.lu@linux.intel.com> References: <20230809124806.45516-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The domain_flush_pasid_iotlb() helper function is used to flush the IOTLB entries for a given PASID. Previously, this function assumed that RID2PASID was only used for the first-level DMA translation. However, with the introduction of the set_dev_pasid callback, this assumption is no longer valid. Add a check before using the RID2PASID for PASID invalidation. This check ensures that the domain has been attached to a physical device before using RID2PASID. Signed-off-by: Lu Baolu Signed-off-by: Jacob Pan Reviewed-by: Kevin Tian Link: https://lore.kernel.org/r/20230802212427.1497170-7-jacob.jun.pan@linu= x.intel.com --- drivers/iommu/intel/iommu.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c index bdde38a5e43a..4a41aca6a2ba 100644 --- a/drivers/iommu/intel/iommu.c +++ b/drivers/iommu/intel/iommu.c @@ -1475,7 +1475,8 @@ static void domain_flush_pasid_iotlb(struct intel_iom= mu *iommu, unsigned long flags; =20 spin_lock_irqsave(&domain->lock, flags); - qi_flush_piotlb(iommu, did, IOMMU_NO_PASID, addr, npages, ih); + if (!list_empty(&domain->devices)) + qi_flush_piotlb(iommu, did, IOMMU_NO_PASID, addr, npages, ih); spin_unlock_irqrestore(&domain->lock, flags); } =20 --=20 2.34.1 From nobody Fri Sep 12 00:58:13 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9651AC41513 for ; Wed, 9 Aug 2023 12:50:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232680AbjHIMuw (ORCPT ); Wed, 9 Aug 2023 08:50:52 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53102 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232697AbjHIMuk (ORCPT ); Wed, 9 Aug 2023 08:50:40 -0400 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.65]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C569E212C for ; Wed, 9 Aug 2023 05:50:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1691585437; x=1723121437; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=tdgrp3Maev1ZcS6b/C0V5HVL9R+QcLiORmhmxsQhAQw=; b=aA/KkQjwWxHBTdYXsAmIDFiFpdSahx7+HUoKErgeax7TSTToktFnyeSS tIKFc1fE/LgwzQiJZZ4xUnjkkl1qGIZLSAHSj4L52yPQIru6bqKCSbrAJ r45YJg4ZLImo2xgqsgSnqYOA7TymifxnzmeMFID2xahZRVzOef4/EWotD l7ASV4jNzrU4o3lB2qG6mJlzxOh6FZsecuBDuNsltKKU+igbT5DHB6Y0p 7e4kz+vmRel6QPRsaSrpDVPSx/MIn5kx5d6WbUVO/kWLD4dzlhQ7OQxPl n0YMHTeM7N5qze59t0cFPoQnZoJpv+5c+HfDsH2IcT7HZtfvVLmjmWOMM Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10795"; a="374821724" X-IronPort-AV: E=Sophos;i="6.01,159,1684825200"; d="scan'208";a="374821724" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Aug 2023 05:50:37 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10795"; a="855521762" X-IronPort-AV: E=Sophos;i="6.01,159,1684825200"; d="scan'208";a="855521762" Received: from allen-box.sh.intel.com ([10.239.159.127]) by orsmga004.jf.intel.com with ESMTP; 09 Aug 2023 05:50:35 -0700 From: Lu Baolu To: Joerg Roedel Cc: YueHaibing , Yanfei Xu , Jacob Pan , iommu@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH 07/13] iommu/vt-d: Add set_dev_pasid callback for dma domain Date: Wed, 9 Aug 2023 20:48:00 +0800 Message-Id: <20230809124806.45516-8-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230809124806.45516-1-baolu.lu@linux.intel.com> References: <20230809124806.45516-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" This allows the upper layers to set a domain to a PASID of a device if the PASID feature is supported by the IOMMU hardware. The typical use cases are, for example, kernel DMA with PASID and hardware assisted mediated device drivers. The attaching device and pasid information is tracked in a per-domain list and is used for IOTLB and devTLB invalidation. Signed-off-by: Lu Baolu Signed-off-by: Jacob Pan Reviewed-by: Kevin Tian Link: https://lore.kernel.org/r/20230802212427.1497170-8-jacob.jun.pan@linu= x.intel.com --- drivers/iommu/intel/iommu.h | 7 +++ drivers/iommu/intel/iommu.c | 104 ++++++++++++++++++++++++++++++++++-- 2 files changed, 106 insertions(+), 5 deletions(-) diff --git a/drivers/iommu/intel/iommu.h b/drivers/iommu/intel/iommu.h index 6d94a29f5d52..c18fb699c87a 100644 --- a/drivers/iommu/intel/iommu.h +++ b/drivers/iommu/intel/iommu.h @@ -595,6 +595,7 @@ struct dmar_domain { =20 spinlock_t lock; /* Protect device tracking lists */ struct list_head devices; /* all devices' list */ + struct list_head dev_pasids; /* all attached pasids */ =20 struct dma_pte *pgd; /* virtual address */ int gaw; /* max guest address width */ @@ -717,6 +718,12 @@ struct device_domain_info { struct pasid_table *pasid_table; /* pasid table */ }; =20 +struct dev_pasid_info { + struct list_head link_domain; /* link to domain siblings */ + struct device *dev; + ioasid_t pasid; +}; + static inline void __iommu_flush_cache( struct intel_iommu *iommu, void *addr, int size) { diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c index 4a41aca6a2ba..f939fc2aa66c 100644 --- a/drivers/iommu/intel/iommu.c +++ b/drivers/iommu/intel/iommu.c @@ -1359,6 +1359,7 @@ domain_lookup_dev_info(struct dmar_domain *domain, =20 static void domain_update_iotlb(struct dmar_domain *domain) { + struct dev_pasid_info *dev_pasid; struct device_domain_info *info; bool has_iotlb_device =3D false; unsigned long flags; @@ -1370,6 +1371,14 @@ static void domain_update_iotlb(struct dmar_domain *= domain) break; } } + + list_for_each_entry(dev_pasid, &domain->dev_pasids, link_domain) { + info =3D dev_iommu_priv_get(dev_pasid->dev); + if (info->ats_enabled) { + has_iotlb_device =3D true; + break; + } + } domain->has_iotlb_device =3D has_iotlb_device; spin_unlock_irqrestore(&domain->lock, flags); } @@ -1455,6 +1464,7 @@ static void __iommu_flush_dev_iotlb(struct device_dom= ain_info *info, static void iommu_flush_dev_iotlb(struct dmar_domain *domain, u64 addr, unsigned mask) { + struct dev_pasid_info *dev_pasid; struct device_domain_info *info; unsigned long flags; =20 @@ -1464,6 +1474,19 @@ static void iommu_flush_dev_iotlb(struct dmar_domain= *domain, spin_lock_irqsave(&domain->lock, flags); list_for_each_entry(info, &domain->devices, link) __iommu_flush_dev_iotlb(info, addr, mask); + + list_for_each_entry(dev_pasid, &domain->dev_pasids, link_domain) { + info =3D dev_iommu_priv_get(dev_pasid->dev); + + if (!info->ats_enabled) + continue; + + qi_flush_dev_iotlb_pasid(info->iommu, + PCI_DEVID(info->bus, info->devfn), + info->pfsid, dev_pasid->pasid, + info->ats_qdep, addr, + mask); + } spin_unlock_irqrestore(&domain->lock, flags); } =20 @@ -1472,9 +1495,13 @@ static void domain_flush_pasid_iotlb(struct intel_io= mmu *iommu, unsigned long npages, bool ih) { u16 did =3D domain_id_iommu(domain, iommu); + struct dev_pasid_info *dev_pasid; unsigned long flags; =20 spin_lock_irqsave(&domain->lock, flags); + list_for_each_entry(dev_pasid, &domain->dev_pasids, link_domain) + qi_flush_piotlb(iommu, did, dev_pasid->pasid, addr, npages, ih); + if (!list_empty(&domain->devices)) qi_flush_piotlb(iommu, did, IOMMU_NO_PASID, addr, npages, ih); spin_unlock_irqrestore(&domain->lock, flags); @@ -1739,6 +1766,7 @@ static struct dmar_domain *alloc_domain(unsigned int = type) domain->use_first_level =3D true; domain->has_iotlb_device =3D false; INIT_LIST_HEAD(&domain->devices); + INIT_LIST_HEAD(&domain->dev_pasids); spin_lock_init(&domain->lock); xa_init(&domain->iommu_array); =20 @@ -4719,7 +4747,10 @@ static void intel_iommu_iotlb_sync_map(struct iommu_= domain *domain, static void intel_iommu_remove_dev_pasid(struct device *dev, ioasid_t pasi= d) { struct intel_iommu *iommu =3D device_to_iommu(dev, NULL, NULL); + struct dev_pasid_info *curr, *dev_pasid =3D NULL; + struct dmar_domain *dmar_domain; struct iommu_domain *domain; + unsigned long flags; =20 domain =3D iommu_get_domain_for_dev_pasid(dev, pasid, 0); if (WARN_ON_ONCE(!domain)) @@ -4735,17 +4766,79 @@ static void intel_iommu_remove_dev_pasid(struct dev= ice *dev, ioasid_t pasid) goto out_tear_down; } =20 - /* - * Should never reach here until we add support for attaching - * non-SVA domain to a pasid. - */ - WARN_ON(1); + dmar_domain =3D to_dmar_domain(domain); + spin_lock_irqsave(&dmar_domain->lock, flags); + list_for_each_entry(curr, &dmar_domain->dev_pasids, link_domain) { + if (curr->dev =3D=3D dev && curr->pasid =3D=3D pasid) { + list_del(&curr->link_domain); + dev_pasid =3D curr; + break; + } + } + WARN_ON_ONCE(!dev_pasid); + spin_unlock_irqrestore(&dmar_domain->lock, flags); =20 + domain_detach_iommu(dmar_domain, iommu); + kfree(dev_pasid); out_tear_down: intel_pasid_tear_down_entry(iommu, dev, pasid, false); intel_drain_pasid_prq(dev, pasid); } =20 +static int intel_iommu_set_dev_pasid(struct iommu_domain *domain, + struct device *dev, ioasid_t pasid) +{ + struct device_domain_info *info =3D dev_iommu_priv_get(dev); + struct dmar_domain *dmar_domain =3D to_dmar_domain(domain); + struct intel_iommu *iommu =3D info->iommu; + struct dev_pasid_info *dev_pasid; + unsigned long flags; + int ret; + + if (!pasid_supported(iommu) || dev_is_real_dma_subdevice(dev)) + return -EOPNOTSUPP; + + if (context_copied(iommu, info->bus, info->devfn)) + return -EBUSY; + + ret =3D prepare_domain_attach_device(domain, dev); + if (ret) + return ret; + + dev_pasid =3D kzalloc(sizeof(*dev_pasid), GFP_KERNEL); + if (!dev_pasid) + return -ENOMEM; + + ret =3D domain_attach_iommu(dmar_domain, iommu); + if (ret) + goto out_free; + + if (domain_type_is_si(dmar_domain)) + ret =3D intel_pasid_setup_pass_through(iommu, dmar_domain, + dev, pasid); + else if (dmar_domain->use_first_level) + ret =3D domain_setup_first_level(iommu, dmar_domain, + dev, pasid); + else + ret =3D intel_pasid_setup_second_level(iommu, dmar_domain, + dev, pasid); + if (ret) + goto out_detach_iommu; + + dev_pasid->dev =3D dev; + dev_pasid->pasid =3D pasid; + spin_lock_irqsave(&dmar_domain->lock, flags); + list_add(&dev_pasid->link_domain, &dmar_domain->dev_pasids); + spin_unlock_irqrestore(&dmar_domain->lock, flags); + + return 0; +out_detach_iommu: + domain_detach_iommu(dmar_domain, iommu); +out_free: + kfree(dev_pasid); + return ret; +} + const struct iommu_ops intel_iommu_ops =3D { .capable =3D intel_iommu_capable, .domain_alloc =3D intel_iommu_domain_alloc, @@ -4765,6 +4858,7 @@ const struct iommu_ops intel_iommu_ops =3D { #endif .default_domain_ops =3D &(const struct iommu_domain_ops) { .attach_dev =3D intel_iommu_attach_device, + .set_dev_pasid =3D intel_iommu_set_dev_pasid, .map_pages =3D intel_iommu_map_pages, .unmap_pages =3D intel_iommu_unmap_pages, .iotlb_sync_map =3D intel_iommu_iotlb_sync_map, --=20 2.34.1 From nobody Fri Sep 12 00:58:13 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 11891C0015E for ; Wed, 9 Aug 2023 12:50:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232698AbjHIMu4 (ORCPT ); Wed, 9 Aug 2023 08:50:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53068 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232746AbjHIMup (ORCPT ); Wed, 9 Aug 2023 08:50:45 -0400 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.65]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0198E2689 for ; Wed, 9 Aug 2023 05:50:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1691585439; x=1723121439; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=EAtw3gOGZIsDEHQX/LxoqAudmvfAUHTaK+xebMGGpOU=; b=AN7ksM8e5Y7W1lsZvdJ7l96QYw4+F0bmejfxHj0M19wFxk7DJ/ZuzKeR mSFl1dvkbMA+rreGV0IAsQpfAH4oK8XDPNFWT6ygP/EbeWo+EXgwtKAnr QmI/CmVUsWjUYHt7bHMxS+CDYqyHkYxaDD09B2jXj89MaBSnYqsWWBI3H 9bUTY2My2dmb2Gq54uCivA4saxc9a9UArtV0S/uhdPpLDaoYKixkItM+6 ezAzB21HYGgSnXc9EAODWYUB6RJkTsC00LU9ZO4I1gAha9srOtXQPwaM7 oxTam6F6sJ7UofxIG123NzSefjRCo2TytoHliy+Hz73HIycrMbxI18oG5 w==; X-IronPort-AV: E=McAfee;i="6600,9927,10795"; a="374821738" X-IronPort-AV: E=Sophos;i="6.01,159,1684825200"; d="scan'208";a="374821738" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Aug 2023 05:50:39 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10795"; a="855521786" X-IronPort-AV: E=Sophos;i="6.01,159,1684825200"; d="scan'208";a="855521786" Received: from allen-box.sh.intel.com ([10.239.159.127]) by orsmga004.jf.intel.com with ESMTP; 09 Aug 2023 05:50:37 -0700 From: Lu Baolu To: Joerg Roedel Cc: YueHaibing , Yanfei Xu , Jacob Pan , iommu@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH 08/13] dmaengine/idxd: Re-enable kernel workqueue under DMA API Date: Wed, 9 Aug 2023 20:48:01 +0800 Message-Id: <20230809124806.45516-9-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230809124806.45516-1-baolu.lu@linux.intel.com> References: <20230809124806.45516-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Jacob Pan Kernel workqueues were disabled due to flawed use of kernel VA and SVA API. Now that we have the support for attaching PASID to the device's default domain and the ability to reserve global PASIDs from SVA APIs, we can re-enable the kernel work queues and use them under DMA API. We also use non-privileged access for in-kernel DMA to be consistent with the IOMMU settings. Consequently, interrupt for user privilege is enabled for work completion IRQs. Link: https://lore.kernel.org/linux-iommu/20210511194726.GP1002214@nvidia.c= om/ Tested-by: Tony Zhu Reviewed-by: Dave Jiang Reviewed-by: Fenghua Yu Reviewed-by: Lu Baolu Reviewed-by: Jason Gunthorpe Acked-by: Vinod Koul Signed-off-by: Jacob Pan Link: https://lore.kernel.org/r/20230802212427.1497170-9-jacob.jun.pan@linu= x.intel.com Signed-off-by: Lu Baolu --- drivers/dma/idxd/idxd.h | 9 +++++++ drivers/dma/idxd/device.c | 39 ++++++++++------------------ drivers/dma/idxd/dma.c | 5 ++-- drivers/dma/idxd/init.c | 54 ++++++++++++++++++++++++++++++++++++--- drivers/dma/idxd/sysfs.c | 7 ----- 5 files changed, 76 insertions(+), 38 deletions(-) diff --git a/drivers/dma/idxd/idxd.h b/drivers/dma/idxd/idxd.h index 5428a2e1b1ec..502be9db63f4 100644 --- a/drivers/dma/idxd/idxd.h +++ b/drivers/dma/idxd/idxd.h @@ -473,6 +473,15 @@ static inline struct idxd_device *ie_to_idxd(struct id= xd_irq_entry *ie) return container_of(ie, struct idxd_device, ie); } =20 +static inline void idxd_set_user_intr(struct idxd_device *idxd, bool enabl= e) +{ + union gencfg_reg reg; + + reg.bits =3D ioread32(idxd->reg_base + IDXD_GENCFG_OFFSET); + reg.user_int_en =3D enable; + iowrite32(reg.bits, idxd->reg_base + IDXD_GENCFG_OFFSET); +} + extern struct bus_type dsa_bus_type; =20 extern bool support_enqcmd; diff --git a/drivers/dma/idxd/device.c b/drivers/dma/idxd/device.c index 5abbcc61c528..169b7ade8919 100644 --- a/drivers/dma/idxd/device.c +++ b/drivers/dma/idxd/device.c @@ -299,21 +299,6 @@ void idxd_wqs_unmap_portal(struct idxd_device *idxd) } } =20 -static void __idxd_wq_set_priv_locked(struct idxd_wq *wq, int priv) -{ - struct idxd_device *idxd =3D wq->idxd; - union wqcfg wqcfg; - unsigned int offset; - - offset =3D WQCFG_OFFSET(idxd, wq->id, WQCFG_PRIVL_IDX); - spin_lock(&idxd->dev_lock); - wqcfg.bits[WQCFG_PRIVL_IDX] =3D ioread32(idxd->reg_base + offset); - wqcfg.priv =3D priv; - wq->wqcfg->bits[WQCFG_PRIVL_IDX] =3D wqcfg.bits[WQCFG_PRIVL_IDX]; - iowrite32(wqcfg.bits[WQCFG_PRIVL_IDX], idxd->reg_base + offset); - spin_unlock(&idxd->dev_lock); -} - static void __idxd_wq_set_pasid_locked(struct idxd_wq *wq, int pasid) { struct idxd_device *idxd =3D wq->idxd; @@ -1423,15 +1408,14 @@ int drv_enable_wq(struct idxd_wq *wq) } =20 /* - * In the event that the WQ is configurable for pasid and priv bits. - * For kernel wq, the driver should setup the pasid, pasid_en, and priv b= it. - * However, for non-kernel wq, the driver should only set the pasid_en bi= t for - * shared wq. A dedicated wq that is not 'kernel' type will configure pas= id and + * In the event that the WQ is configurable for pasid, the driver + * should setup the pasid, pasid_en bit. This is true for both kernel + * and user shared workqueues. There is no need to setup priv bit in + * that in-kernel DMA will also do user privileged requests. + * A dedicated wq that is not 'kernel' type will configure pasid and * pasid_en later on so there is no need to setup. */ if (test_bit(IDXD_FLAG_CONFIGURABLE, &idxd->flags)) { - int priv =3D 0; - if (wq_pasid_enabled(wq)) { if (is_idxd_wq_kernel(wq) || wq_shared(wq)) { u32 pasid =3D wq_dedicated(wq) ? idxd->pasid : 0; @@ -1439,10 +1423,6 @@ int drv_enable_wq(struct idxd_wq *wq) __idxd_wq_set_pasid_locked(wq, pasid); } } - - if (is_idxd_wq_kernel(wq)) - priv =3D 1; - __idxd_wq_set_priv_locked(wq, priv); } =20 rc =3D 0; @@ -1550,6 +1530,15 @@ int idxd_device_drv_probe(struct idxd_dev *idxd_dev) if (rc < 0) return -ENXIO; =20 + /* + * System PASID is preserved across device disable/enable cycle, but + * genconfig register content gets cleared during device reset. We + * need to re-enable user interrupts for kernel work queue completion + * IRQ to function. + */ + if (idxd->pasid !=3D IOMMU_PASID_INVALID) + idxd_set_user_intr(idxd, 1); + rc =3D idxd_device_evl_setup(idxd); if (rc < 0) { idxd->cmd_status =3D IDXD_SCMD_DEV_EVL_ERR; diff --git a/drivers/dma/idxd/dma.c b/drivers/dma/idxd/dma.c index eb35ca313684..07623fb0f52f 100644 --- a/drivers/dma/idxd/dma.c +++ b/drivers/dma/idxd/dma.c @@ -75,9 +75,10 @@ static inline void idxd_prep_desc_common(struct idxd_wq = *wq, hw->xfer_size =3D len; /* * For dedicated WQ, this field is ignored and HW will use the WQCFG.priv - * field instead. This field should be set to 1 for kernel descriptors. + * field instead. This field should be set to 0 for kernel descriptors + * since kernel DMA on VT-d supports "user" privilege only. */ - hw->priv =3D 1; + hw->priv =3D 0; hw->completion_addr =3D compl; } =20 diff --git a/drivers/dma/idxd/init.c b/drivers/dma/idxd/init.c index 1aa823974cda..0eb1c827a215 100644 --- a/drivers/dma/idxd/init.c +++ b/drivers/dma/idxd/init.c @@ -550,14 +550,59 @@ static struct idxd_device *idxd_alloc(struct pci_dev = *pdev, struct idxd_driver_d =20 static int idxd_enable_system_pasid(struct idxd_device *idxd) { - return -EOPNOTSUPP; + struct pci_dev *pdev =3D idxd->pdev; + struct device *dev =3D &pdev->dev; + struct iommu_domain *domain; + ioasid_t pasid; + int ret; + + /* + * Attach a global PASID to the DMA domain so that we can use ENQCMDS + * to submit work on buffers mapped by DMA API. + */ + domain =3D iommu_get_domain_for_dev(dev); + if (!domain) + return -EPERM; + + pasid =3D iommu_alloc_global_pasid(dev); + if (pasid =3D=3D IOMMU_PASID_INVALID) + return -ENOSPC; + + /* + * DMA domain is owned by the driver, it should support all valid + * types such as DMA-FQ, identity, etc. + */ + ret =3D iommu_attach_device_pasid(domain, dev, pasid); + if (ret) { + dev_err(dev, "failed to attach device pasid %d, domain type %d", + pasid, domain->type); + iommu_free_global_pasid(pasid); + return ret; + } + + /* Since we set user privilege for kernel DMA, enable completion IRQ */ + idxd_set_user_intr(idxd, 1); + idxd->pasid =3D pasid; + + return ret; } =20 static void idxd_disable_system_pasid(struct idxd_device *idxd) { + struct pci_dev *pdev =3D idxd->pdev; + struct device *dev =3D &pdev->dev; + struct iommu_domain *domain; =20 - iommu_sva_unbind_device(idxd->sva); + domain =3D iommu_get_domain_for_dev(dev); + if (!domain) + return; + + iommu_detach_device_pasid(domain, dev, idxd->pasid); + iommu_free_global_pasid(idxd->pasid); + + idxd_set_user_intr(idxd, 0); idxd->sva =3D NULL; + idxd->pasid =3D IOMMU_PASID_INVALID; } =20 static int idxd_enable_sva(struct pci_dev *pdev) @@ -600,8 +645,9 @@ static int idxd_probe(struct idxd_device *idxd) } else { set_bit(IDXD_FLAG_USER_PASID_ENABLED, &idxd->flags); =20 - if (idxd_enable_system_pasid(idxd)) - dev_warn(dev, "No in-kernel DMA with PASID.\n"); + rc =3D idxd_enable_system_pasid(idxd); + if (rc) + dev_warn(dev, "No in-kernel DMA with PASID. %d\n", rc); else set_bit(IDXD_FLAG_PASID_ENABLED, &idxd->flags); } diff --git a/drivers/dma/idxd/sysfs.c b/drivers/dma/idxd/sysfs.c index 293739ac5596..63f6966c51aa 100644 --- a/drivers/dma/idxd/sysfs.c +++ b/drivers/dma/idxd/sysfs.c @@ -948,13 +948,6 @@ static ssize_t wq_name_store(struct device *dev, if (strlen(buf) > WQ_NAME_SIZE || strlen(buf) =3D=3D 0) return -EINVAL; =20 - /* - * This is temporarily placed here until we have SVM support for - * dmaengine. - */ - if (wq->type =3D=3D IDXD_WQT_KERNEL && device_pasid_enabled(wq->idxd)) - return -EOPNOTSUPP; - input =3D kstrndup(buf, count, GFP_KERNEL); if (!input) return -ENOMEM; --=20 2.34.1 From nobody Fri Sep 12 00:58:13 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 704DBC0015E for ; Wed, 9 Aug 2023 12:51:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232197AbjHIMvD (ORCPT ); Wed, 9 Aug 2023 08:51:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48616 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232627AbjHIMus (ORCPT ); Wed, 9 Aug 2023 08:50:48 -0400 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.65]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7B07D2699 for ; Wed, 9 Aug 2023 05:50:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1691585441; x=1723121441; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=0Ne5/6sMClEOpGghoXh4tELZayLADaZI9iSbfRL/ce4=; b=nNWm4/d/4NwAd2aR9Zn77f43cZCSTzkoZtFmnfq1HJk/pUml2ynMHdUK tsuhusx0BDI3bHZr4BtNqwO8jzJH+8GaqCEr+hpvEaynz271tTSgf4NO2 b5rwiOwLKAX7Sr3sTajDO4b4ziaAVXFCCoAiyvXX6wrLi2caPS2tcMW4M t7TVrgM042WpC5j7YCpasf+Bc5MaLZHAF+Rs9MNXSTbqRzYBgLWI3QinA tcJ1k1BslOkX/+L4+aYZTf8P8jyCukVxlP1TEO6GotVmh39zqqu9/PzeP YtxlQxr2XgtaYuLN+TvJo4FvzG544GJ7KkH901iXU4qEouyDjXHrK7s/r Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10795"; a="374821745" X-IronPort-AV: E=Sophos;i="6.01,159,1684825200"; d="scan'208";a="374821745" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Aug 2023 05:50:41 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10795"; a="855521809" X-IronPort-AV: E=Sophos;i="6.01,159,1684825200"; d="scan'208";a="855521809" Received: from allen-box.sh.intel.com ([10.239.159.127]) by orsmga004.jf.intel.com with ESMTP; 09 Aug 2023 05:50:39 -0700 From: Lu Baolu To: Joerg Roedel Cc: YueHaibing , Yanfei Xu , Jacob Pan , iommu@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH 09/13] iommu: Prevent RESV_DIRECT devices from blocking domains Date: Wed, 9 Aug 2023 20:48:02 +0800 Message-Id: <20230809124806.45516-10-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230809124806.45516-1-baolu.lu@linux.intel.com> References: <20230809124806.45516-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The IOMMU_RESV_DIRECT flag indicates that a memory region must be mapped 1:1 at all times. This means that the region must always be accessible to the device, even if the device is attached to a blocking domain. This is equal to saying that IOMMU_RESV_DIRECT flag prevents devices from being attached to blocking domains. This also implies that devices that implement RESV_DIRECT regions will be prevented from being assigned to user space since taking the DMA ownership immediately switches to a blocking domain. The rule of preventing devices with the IOMMU_RESV_DIRECT regions from being assigned to user space has existed in the Intel IOMMU driver for a long time. Now, this rule is being lifted up to a general core rule, as other architectures like AMD and ARM also have RMRR-like reserved regions. This has been discussed in the community mailing list and refer to below link for more details. Other places using unmanaged domains for kernel DMA must follow the iommu_get_resv_regions() and setup IOMMU_RESV_DIRECT - we do not restrict them in the core code. Cc: Robin Murphy Cc: Alex Williamson Cc: Kevin Tian Signed-off-by: Jason Gunthorpe Link: https://lore.kernel.org/linux-iommu/BN9PR11MB5276E84229B5BD952D78E959= 8C639@BN9PR11MB5276.namprd11.prod.outlook.com Signed-off-by: Lu Baolu Reviewed-by: Jason Gunthorpe Acked-by: Joerg Roedel Link: https://lore.kernel.org/r/20230724060352.113458-2-baolu.lu@linux.inte= l.com --- include/linux/iommu.h | 2 ++ drivers/iommu/iommu.c | 37 +++++++++++++++++++++++++++---------- 2 files changed, 29 insertions(+), 10 deletions(-) diff --git a/include/linux/iommu.h b/include/linux/iommu.h index 4cb584b08b81..9ed139bf111f 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -414,6 +414,7 @@ struct iommu_fault_param { * @max_pasids: number of PASIDs this device can consume * @attach_deferred: the dma domain attachment is deferred * @pci_32bit_workaround: Limit DMA allocations to 32-bit IOVAs + * @require_direct: device requires IOMMU_RESV_DIRECT regions * * TODO: migrate other per device data pointers under iommu_dev_data, e.g. * struct iommu_group *iommu_group; @@ -428,6 +429,7 @@ struct dev_iommu { u32 max_pasids; u32 attach_deferred:1; u32 pci_32bit_workaround:1; + u32 require_direct:1; }; =20 int iommu_device_register(struct iommu_device *iommu, diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c index 738fee1a24f7..49e96a267a2b 100644 --- a/drivers/iommu/iommu.c +++ b/drivers/iommu/iommu.c @@ -1032,14 +1032,12 @@ static int iommu_create_device_direct_mappings(stru= ct iommu_domain *domain, unsigned long pg_size; int ret =3D 0; =20 - if (!iommu_is_dma_domain(domain)) - return 0; - - BUG_ON(!domain->pgsize_bitmap); - - pg_size =3D 1UL << __ffs(domain->pgsize_bitmap); + pg_size =3D domain->pgsize_bitmap ? 1UL << __ffs(domain->pgsize_bitmap) := 0; INIT_LIST_HEAD(&mappings); =20 + if (WARN_ON_ONCE(iommu_is_dma_domain(domain) && !pg_size)) + return -EINVAL; + iommu_get_resv_regions(dev, &mappings); =20 /* We need to consider overlapping regions for different devices */ @@ -1047,13 +1045,17 @@ static int iommu_create_device_direct_mappings(stru= ct iommu_domain *domain, dma_addr_t start, end, addr; size_t map_size =3D 0; =20 + if (entry->type =3D=3D IOMMU_RESV_DIRECT) + dev->iommu->require_direct =3D 1; + + if ((entry->type !=3D IOMMU_RESV_DIRECT && + entry->type !=3D IOMMU_RESV_DIRECT_RELAXABLE) || + !iommu_is_dma_domain(domain)) + continue; + start =3D ALIGN(entry->start, pg_size); end =3D ALIGN(entry->start + entry->length, pg_size); =20 - if (entry->type !=3D IOMMU_RESV_DIRECT && - entry->type !=3D IOMMU_RESV_DIRECT_RELAXABLE) - continue; - for (addr =3D start; addr <=3D end; addr +=3D pg_size) { phys_addr_t phys_addr; =20 @@ -2182,6 +2184,21 @@ static int __iommu_device_set_domain(struct iommu_gr= oup *group, { int ret; =20 + /* + * If the device requires IOMMU_RESV_DIRECT then we cannot allow + * the blocking domain to be attached as it does not contain the + * required 1:1 mapping. This test effectively excludes the device + * being used with iommu_group_claim_dma_owner() which will block + * vfio and iommufd as well. + */ + if (dev->iommu->require_direct && + (new_domain->type =3D=3D IOMMU_DOMAIN_BLOCKED || + new_domain =3D=3D group->blocking_domain)) { + dev_warn(dev, + "Firmware has requested this device have a 1:1 IOMMU mapping, rejectin= g configuring the device without a 1:1 mapping. Contact your platform vendo= r.\n"); + return -EINVAL; + } + if (dev->iommu->attach_deferred) { if (new_domain =3D=3D group->default_domain) return 0; --=20 2.34.1 From nobody Fri Sep 12 00:58:13 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 37551EB64DD for ; Wed, 9 Aug 2023 12:51:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229596AbjHIMvH (ORCPT ); Wed, 9 Aug 2023 08:51:07 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53128 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232724AbjHIMvA (ORCPT ); Wed, 9 Aug 2023 08:51:00 -0400 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.65]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 85B7726A6 for ; Wed, 9 Aug 2023 05:50:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1691585443; x=1723121443; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=WIrVHdlFcP2yuyMe9xdW5lDUhBW7ZaLR8PAhwSLCIyw=; b=JDNNiiM/sYtq1Ty/+Mz1s4KLM/piOCrz++Fz9hcDKffFlj6cboojLiBt ZlNESYw97IuAJCWX/nxSr0YvGrS3tc8KVXny45x1AFLMnD1+A/IQ6PKP+ nyOyK392LQ7DKoFjq9P25Ou1pWOAIBERqGDEySRHVTjzOFVwrC4m/kxT+ i9llQsdoA2eS3BQF9H9BQSajXlTv8a+46KfIDPnu1N65af+bFG2E+D04Y LYyPEW0t4KkFCoavB51MCgfP3A5m1bMZE884WoTGUs4hlvBcL0iQBMR6W W3MtoiGxUeoKFejMjYFzAkzNRANRxMrjcj7Sc9zJva9QUTL+BQyKtjZ2S A==; X-IronPort-AV: E=McAfee;i="6600,9927,10795"; a="374821755" X-IronPort-AV: E=Sophos;i="6.01,159,1684825200"; d="scan'208";a="374821755" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Aug 2023 05:50:43 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10795"; a="855521828" X-IronPort-AV: E=Sophos;i="6.01,159,1684825200"; d="scan'208";a="855521828" Received: from allen-box.sh.intel.com ([10.239.159.127]) by orsmga004.jf.intel.com with ESMTP; 09 Aug 2023 05:50:41 -0700 From: Lu Baolu To: Joerg Roedel Cc: YueHaibing , Yanfei Xu , Jacob Pan , iommu@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH 10/13] iommu/vt-d: Remove rmrr check in domain attaching device path Date: Wed, 9 Aug 2023 20:48:03 +0800 Message-Id: <20230809124806.45516-11-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230809124806.45516-1-baolu.lu@linux.intel.com> References: <20230809124806.45516-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The core code now prevents devices with RMRR regions from being assigned to user space. There is no need to check for this condition in individual drivers. Remove it to avoid duplicate code. Signed-off-by: Lu Baolu Reviewed-by: Jason Gunthorpe Reviewed-by: Kevin Tian Link: https://lore.kernel.org/r/20230724060352.113458-3-baolu.lu@linux.inte= l.com --- drivers/iommu/intel/iommu.c | 58 ------------------------------------- 1 file changed, 58 deletions(-) diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c index f939fc2aa66c..1f89714ca462 100644 --- a/drivers/iommu/intel/iommu.c +++ b/drivers/iommu/intel/iommu.c @@ -2487,30 +2487,6 @@ static int dmar_domain_attach_device(struct dmar_dom= ain *domain, return 0; } =20 -static bool device_has_rmrr(struct device *dev) -{ - struct dmar_rmrr_unit *rmrr; - struct device *tmp; - int i; - - rcu_read_lock(); - for_each_rmrr_units(rmrr) { - /* - * Return TRUE if this RMRR contains the device that - * is passed in. - */ - for_each_active_dev_scope(rmrr->devices, - rmrr->devices_cnt, i, tmp) - if (tmp =3D=3D dev || - is_downstream_to_pci_bridge(dev, tmp)) { - rcu_read_unlock(); - return true; - } - } - rcu_read_unlock(); - return false; -} - /** * device_rmrr_is_relaxable - Test whether the RMRR of this device * is relaxable (ie. is allowed to be not enforced under some conditions) @@ -2540,34 +2516,6 @@ static bool device_rmrr_is_relaxable(struct device *= dev) return false; } =20 -/* - * There are a couple cases where we need to restrict the functionality of - * devices associated with RMRRs. The first is when evaluating a device f= or - * identity mapping because problems exist when devices are moved in and o= ut - * of domains and their respective RMRR information is lost. This means t= hat - * a device with associated RMRRs will never be in a "passthrough" domain. - * The second is use of the device through the IOMMU API. This interface - * expects to have full control of the IOVA space for the device. We cann= ot - * satisfy both the requirement that RMRR access is maintained and have an - * unencumbered IOVA space. We also have no ability to quiesce the device= 's - * use of the RMRR space or even inform the IOMMU API user of the restrict= ion. - * We therefore prevent devices associated with an RMRR from participating= in - * the IOMMU API, which eliminates them from device assignment. - * - * In both cases, devices which have relaxable RMRRs are not concerned by = this - * restriction. See device_rmrr_is_relaxable comment. - */ -static bool device_is_rmrr_locked(struct device *dev) -{ - if (!device_has_rmrr(dev)) - return false; - - if (device_rmrr_is_relaxable(dev)) - return false; - - return true; -} - /* * Return the required default domain type for a specific device. * @@ -4173,12 +4121,6 @@ static int intel_iommu_attach_device(struct iommu_do= main *domain, struct device_domain_info *info =3D dev_iommu_priv_get(dev); int ret; =20 - if (domain->type =3D=3D IOMMU_DOMAIN_UNMANAGED && - device_is_rmrr_locked(dev)) { - dev_warn(dev, "Device is ineligible for IOMMU domain attach due to platf= orm RMRR requirement. Contact your platform vendor.\n"); - return -EPERM; - } - if (info->domain) device_block_translation(dev); =20 --=20 2.34.1 From nobody Fri Sep 12 00:58:13 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 407CFC0015E for ; Wed, 9 Aug 2023 12:51:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232253AbjHIMvU (ORCPT ); Wed, 9 Aug 2023 08:51:20 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48616 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232740AbjHIMvB (ORCPT ); Wed, 9 Aug 2023 08:51:01 -0400 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.65]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 997A72135 for ; Wed, 9 Aug 2023 05:50:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1691585445; x=1723121445; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=gc8uxTUf70zvnrXq1dYbH5QztxWkn/Z6QZCZ9eNeV0U=; b=fxrc00g2xn3P4Z3DfKLqPRsSKOjRqQvnOut2M/rWpMPLHw5bM5NavCyH fRlWTiNnxrpPmx41t3nMSmZqHGJCO9kBPEKziDjWd5Jc4pltn5rwSIFpv UZTNleIMS7vJnPCL+IoOa4J38/VbwvzOSmp+gj4EAteG4DnkHEWXF2sze wbLq6TadB2NZ6FYiQnyn5ScqiLdg61agrpNpGQO6kclZ0OymY3EYb6BIc KulHe1dmrJGm/ENOdNNveR6UfsWqZEDx7lC6SjjoaMJtUPrC0/evu1/Ha /8HxvuWVnw7koWFGaMGON9VLcYFb7tj7mqMQJizrcKN4Zk2JhFgYje9uw Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10795"; a="374821763" X-IronPort-AV: E=Sophos;i="6.01,159,1684825200"; d="scan'208";a="374821763" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Aug 2023 05:50:45 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10795"; a="855521861" X-IronPort-AV: E=Sophos;i="6.01,159,1684825200"; d="scan'208";a="855521861" Received: from allen-box.sh.intel.com ([10.239.159.127]) by orsmga004.jf.intel.com with ESMTP; 09 Aug 2023 05:50:43 -0700 From: Lu Baolu To: Joerg Roedel Cc: YueHaibing , Yanfei Xu , Jacob Pan , iommu@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH 11/13] iommu/vt-d: Fix to flush cache of PASID directory table Date: Wed, 9 Aug 2023 20:48:04 +0800 Message-Id: <20230809124806.45516-12-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230809124806.45516-1-baolu.lu@linux.intel.com> References: <20230809124806.45516-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Yanfei Xu Even the PCI devices don't support pasid capability, PASID table is mandatory for a PCI device in scalable mode. However flushing cache of pasid directory table for these devices are not taken after pasid table is allocated as the "size" of table is zero. Fix it by calculating the size by page order. Found this when reading the code, no real problem encountered for now. Fixes: 194b3348bdbb ("iommu/vt-d: Fix PASID directory pointer coherency") Suggested-by: Lu Baolu Signed-off-by: Yanfei Xu Link: https://lore.kernel.org/r/20230616081045.721873-1-yanfei.xu@intel.com Signed-off-by: Lu Baolu --- drivers/iommu/intel/pasid.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/iommu/intel/pasid.c b/drivers/iommu/intel/pasid.c index 23dca3bc319d..8f92b92f3d2a 100644 --- a/drivers/iommu/intel/pasid.c +++ b/drivers/iommu/intel/pasid.c @@ -129,7 +129,7 @@ int intel_pasid_alloc_table(struct device *dev) info->pasid_table =3D pasid_table; =20 if (!ecap_coherent(info->iommu->ecap)) - clflush_cache_range(pasid_table->table, size); + clflush_cache_range(pasid_table->table, (1 << order) * PAGE_SIZE); =20 return 0; } --=20 2.34.1 From nobody Fri Sep 12 00:58:13 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EFFB6EB64DD for ; Wed, 9 Aug 2023 12:51:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232717AbjHIMvW (ORCPT ); Wed, 9 Aug 2023 08:51:22 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48626 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232762AbjHIMvC (ORCPT ); Wed, 9 Aug 2023 08:51:02 -0400 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.65]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 08EDB2681 for ; Wed, 9 Aug 2023 05:50:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1691585449; x=1723121449; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=eoot58iZi4D+nuF4+mBQY+JTYodvcOohwejKW6VG6nI=; b=KvrDqgpdIfR1HYx6cuX1Dj0rk/olBiTQwYOr8czDBVpHu68P/HGWxgy4 oqL5IgstnWX847W1qabjEezp+yOgJdW4kkBIDxoiOItFBOWiPESzY6SdT xwEbt+v0h8MV4WVMsplBWnmpYkB9K7GZIJGoNaoCqLn93VHyEFJmC8YPx 6AUNDjLFEwBlbtkhYjv9iah3ZupFGo/iePHc4Bw3ROP4mMUeHNWpFh5Y7 b31diWg8WOPkbhyQPP8iWQElu1WA+lJJf/2NHQI4cL7sjt6VzTaqgbFTJ Vb2dpX7O4h221Rmr0dLhm9CXEFCkuydTja4S0DaY32wfJtGGXvmgMOTK/ w==; X-IronPort-AV: E=McAfee;i="6600,9927,10795"; a="374821770" X-IronPort-AV: E=Sophos;i="6.01,159,1684825200"; d="scan'208";a="374821770" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Aug 2023 05:50:47 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10795"; a="855521878" X-IronPort-AV: E=Sophos;i="6.01,159,1684825200"; d="scan'208";a="855521878" Received: from allen-box.sh.intel.com ([10.239.159.127]) by orsmga004.jf.intel.com with ESMTP; 09 Aug 2023 05:50:45 -0700 From: Lu Baolu To: Joerg Roedel Cc: YueHaibing , Yanfei Xu , Jacob Pan , iommu@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH 12/13] iommu/vt-d: Fix to convert mm pfn to dma pfn Date: Wed, 9 Aug 2023 20:48:05 +0800 Message-Id: <20230809124806.45516-13-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230809124806.45516-1-baolu.lu@linux.intel.com> References: <20230809124806.45516-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Yanfei Xu For the case that VT-d page is smaller than mm page, converting dma pfn should be handled in two cases which are for start pfn and for end pfn. Currently the calculation of end dma pfn is incorrect and the result is less than real page frame number which is causing the mapping of iova always misses some page frames. Rename the mm_to_dma_pfn() to mm_to_dma_pfn_start() and add a new helper for converting end dma pfn named mm_to_dma_pfn_end(). Signed-off-by: Yanfei Xu Link: https://lore.kernel.org/r/20230625082046.979742-1-yanfei.xu@intel.com Signed-off-by: Lu Baolu --- drivers/iommu/intel/iommu.c | 22 +++++++++++++--------- 1 file changed, 13 insertions(+), 9 deletions(-) diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c index 1f89714ca462..dd8ff358867d 100644 --- a/drivers/iommu/intel/iommu.c +++ b/drivers/iommu/intel/iommu.c @@ -113,13 +113,17 @@ static inline unsigned long lvl_to_nr_pages(unsigned = int lvl) =20 /* VT-d pages must always be _smaller_ than MM pages. Otherwise things are never going to work. */ -static inline unsigned long mm_to_dma_pfn(unsigned long mm_pfn) +static inline unsigned long mm_to_dma_pfn_start(unsigned long mm_pfn) { return mm_pfn << (PAGE_SHIFT - VTD_PAGE_SHIFT); } +static inline unsigned long mm_to_dma_pfn_end(unsigned long mm_pfn) +{ + return ((mm_pfn + 1) << (PAGE_SHIFT - VTD_PAGE_SHIFT)) - 1; +} static inline unsigned long page_to_dma_pfn(struct page *pg) { - return mm_to_dma_pfn(page_to_pfn(pg)); + return mm_to_dma_pfn_start(page_to_pfn(pg)); } static inline unsigned long virt_to_dma_pfn(void *p) { @@ -2403,8 +2407,8 @@ static int __init si_domain_init(int hw) =20 for_each_mem_pfn_range(i, nid, &start_pfn, &end_pfn, NULL) { ret =3D iommu_domain_identity_map(si_domain, - mm_to_dma_pfn(start_pfn), - mm_to_dma_pfn(end_pfn)); + mm_to_dma_pfn_start(start_pfn), + mm_to_dma_pfn_end(end_pfn)); if (ret) return ret; } @@ -2425,8 +2429,8 @@ static int __init si_domain_init(int hw) continue; =20 ret =3D iommu_domain_identity_map(si_domain, - mm_to_dma_pfn(start >> PAGE_SHIFT), - mm_to_dma_pfn(end >> PAGE_SHIFT)); + mm_to_dma_pfn_start(start >> PAGE_SHIFT), + mm_to_dma_pfn_end(end >> PAGE_SHIFT)); if (ret) return ret; } @@ -3549,8 +3553,8 @@ static int intel_iommu_memory_notifier(struct notifie= r_block *nb, unsigned long val, void *v) { struct memory_notify *mhp =3D v; - unsigned long start_vpfn =3D mm_to_dma_pfn(mhp->start_pfn); - unsigned long last_vpfn =3D mm_to_dma_pfn(mhp->start_pfn + + unsigned long start_vpfn =3D mm_to_dma_pfn_start(mhp->start_pfn); + unsigned long last_vpfn =3D mm_to_dma_pfn_end(mhp->start_pfn + mhp->nr_pages - 1); =20 switch (val) { @@ -4247,7 +4251,7 @@ static void intel_iommu_tlb_sync(struct iommu_domain = *domain, unsigned long i; =20 nrpages =3D aligned_nrpages(gather->start, size); - start_pfn =3D mm_to_dma_pfn(iova_pfn); + start_pfn =3D mm_to_dma_pfn_start(iova_pfn); =20 xa_for_each(&dmar_domain->iommu_array, i, info) iommu_flush_iotlb_psi(info->iommu, dmar_domain, --=20 2.34.1 From nobody Fri Sep 12 00:58:13 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0AE1DEB64DD for ; Wed, 9 Aug 2023 12:51:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232744AbjHIMv0 (ORCPT ); Wed, 9 Aug 2023 08:51:26 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59044 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232769AbjHIMvL (ORCPT ); Wed, 9 Aug 2023 08:51:11 -0400 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.65]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7CD352712 for ; Wed, 9 Aug 2023 05:50:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1691585454; x=1723121454; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=3Ql8GvqSQc3MnpAxdXzxymy2oKa6nPM5ZIi5KTo5aEM=; b=NcMFahvDeq2pHq4IO9pJeljERzE/KWidwC+QfF6DiUx9C6llbnOLZD3Y 1DPtLY54FW169MbWdgyCbmwnBRDwrHYWiCp+H1ABZm9QAnh8XchWJOgpS pW/5qRla6TWH3y/QWaNDs4UcYI6Jw66BAHx7z2B8qIXFs6azn6Rh5HgmN tjUQJkBFAgRMPSuYhswKLMWasDPT2gnW9OIk45vLiy36r/FYq16j9kLh9 Nc37EgZ3K/D9suEDQxZe0QM8dz99J3Y89+VMDzKGHA4GzAs0TU2ZI6Jn3 MoTwRclHBPLbc7jm5inHrbocviFZsJqeVOtA0YPwIH8ADOt/jLs9EwgFa g==; X-IronPort-AV: E=McAfee;i="6600,9927,10795"; a="374821777" X-IronPort-AV: E=Sophos;i="6.01,159,1684825200"; d="scan'208";a="374821777" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Aug 2023 05:50:49 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10795"; a="855521902" X-IronPort-AV: E=Sophos;i="6.01,159,1684825200"; d="scan'208";a="855521902" Received: from allen-box.sh.intel.com ([10.239.159.127]) by orsmga004.jf.intel.com with ESMTP; 09 Aug 2023 05:50:47 -0700 From: Lu Baolu To: Joerg Roedel Cc: YueHaibing , Yanfei Xu , Jacob Pan , iommu@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH 13/13] iommu/vt-d: Remove unused extern declaration dmar_parse_dev_scope() Date: Wed, 9 Aug 2023 20:48:06 +0800 Message-Id: <20230809124806.45516-14-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230809124806.45516-1-baolu.lu@linux.intel.com> References: <20230809124806.45516-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: YueHaibing Since commit 2e4552893038 ("iommu/vt-d: Unify the way to process DMAR device scope array") this is not used anymore, so can remove it. Signed-off-by: YueHaibing Link: https://lore.kernel.org/r/20230802133934.19712-1-yuehaibing@huawei.com Signed-off-by: Lu Baolu --- include/linux/dmar.h | 2 -- 1 file changed, 2 deletions(-) diff --git a/include/linux/dmar.h b/include/linux/dmar.h index 27dbd4c64860..e34b601b71fd 100644 --- a/include/linux/dmar.h +++ b/include/linux/dmar.h @@ -106,8 +106,6 @@ static inline bool dmar_rcu_check(void) extern int dmar_table_init(void); extern int dmar_dev_scope_init(void); extern void dmar_register_bus_notifier(void); -extern int dmar_parse_dev_scope(void *start, void *end, int *cnt, - struct dmar_dev_scope **devices, u16 segment); extern void *dmar_alloc_dev_scope(void *start, void *end, int *cnt); extern void dmar_free_dev_scope(struct dmar_dev_scope **devices, int *cnt); extern int dmar_insert_dev_scope(struct dmar_pci_notify_info *info, --=20 2.34.1