From nobody Thu May 7 20:22:18 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C7799C433EF for ; Thu, 19 May 2022 07:24:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234704AbiESHYe (ORCPT ); Thu, 19 May 2022 03:24:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41972 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234298AbiESHYX (ORCPT ); Thu, 19 May 2022 03:24:23 -0400 Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8AE397E1CB for ; Thu, 19 May 2022 00:24:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1652945062; x=1684481062; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=MTzb5C8RYTuJcmfENi33AzfOFxHnBCRQAHRLFQ/DKwQ=; b=iTBOHEuGWnttQFnJJU1vd8Rwjn6LUKuuzxbTTsK4PCbZ9vj5blVvfBPu 2jURCoP4Q/RXyh2fB97cCpGj9Yk19u4ZNTJ5lxyGUQmiQXz9CdJrni6br HbBr5dXuUmCpGogWdBqc3FKjWcFD84S1aqdnQT0Qom0R0rC7xgmFPlIA7 GEzzezdiRwfaZJr3SikayeaRg3LOjC5+UVdtNoHJ/ygT9mI8CxyBuQVW+ o5DYjLm2/SY2dNaij0u02Pngsyzwyc2H1pwBnksGhyDHslIFWnSVRsUf7 G8wDr/l8zorPRj+0U9S6QQI+G79mVShtLxYRsjMSAJKmUonrJv0tIWP4Y g==; X-IronPort-AV: E=McAfee;i="6400,9594,10351"; a="272195844" X-IronPort-AV: E=Sophos;i="5.91,236,1647327600"; d="scan'208";a="272195844" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 May 2022 00:24:22 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.91,236,1647327600"; d="scan'208";a="714853016" Received: from allen-box.sh.intel.com ([10.239.159.48]) by fmsmga001.fm.intel.com with ESMTP; 19 May 2022 00:24:18 -0700 From: Lu Baolu To: Joerg Roedel , Jason Gunthorpe , Christoph Hellwig , Kevin Tian , Ashok Raj , Will Deacon , Robin Murphy , Jean-Philippe Brucker , Dave Jiang , Vinod Koul Cc: Eric Auger , Liu Yi L , Jacob jun Pan , iommu@lists.linux-foundation.org, linux-kernel@vger.kernel.org, Lu Baolu , Jean-Philippe Brucker Subject: [PATCH v7 01/10] iommu: Add pasids field in struct iommu_device Date: Thu, 19 May 2022 15:20:38 +0800 Message-Id: <20220519072047.2996983-2-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220519072047.2996983-1-baolu.lu@linux.intel.com> References: <20220519072047.2996983-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Use this field to keep the number of supported PASIDs that an IOMMU hardware is able to support. This is a generic attribute of an IOMMU and lifting it into the per-IOMMU device structure makes it possible to allocate a PASID for device without calls into the IOMMU drivers. Any iommu driver which suports PASID related features should set this field before enabling them on the devices. Signed-off-by: Lu Baolu Reviewed-by: Jean-Philippe Brucker --- include/linux/iommu.h | 2 ++ drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 1 + drivers/iommu/intel/dmar.c | 4 ++++ 3 files changed, 7 insertions(+) diff --git a/include/linux/iommu.h b/include/linux/iommu.h index 5e1afe169549..da423e87f248 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -318,12 +318,14 @@ struct iommu_domain_ops { * @list: Used by the iommu-core to keep a list of registered iommus * @ops: iommu-ops for talking to this iommu * @dev: struct device for sysfs handling + * @pasids: number of supported PASIDs */ struct iommu_device { struct list_head list; const struct iommu_ops *ops; struct fwnode_handle *fwnode; struct device *dev; + u32 pasids; }; =20 /** diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/ar= m/arm-smmu-v3/arm-smmu-v3.c index 88817a3376ef..6e2cd082c670 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -3546,6 +3546,7 @@ static int arm_smmu_device_hw_probe(struct arm_smmu_d= evice *smmu) /* SID/SSID sizes */ smmu->ssid_bits =3D FIELD_GET(IDR1_SSIDSIZE, reg); smmu->sid_bits =3D FIELD_GET(IDR1_SIDSIZE, reg); + smmu->iommu.pasids =3D smmu->ssid_bits; =20 /* * If the SMMU supports fewer bits than would fill a single L2 stream diff --git a/drivers/iommu/intel/dmar.c b/drivers/iommu/intel/dmar.c index 4de960834a1b..1c3cf267934d 100644 --- a/drivers/iommu/intel/dmar.c +++ b/drivers/iommu/intel/dmar.c @@ -1126,6 +1126,10 @@ static int alloc_iommu(struct dmar_drhd_unit *drhd) =20 raw_spin_lock_init(&iommu->register_lock); =20 + /* Supports full 20-bit PASID in scalable mode. */ + if (ecap_pasid(iommu->ecap)) + iommu->iommu.pasids =3D 1UL << 20; + /* * This is only for hotplug; at boot time intel_iommu_enabled won't * be set yet. When intel_iommu_init() runs, it registers the units --=20 2.25.1 From nobody Thu May 7 20:22:18 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E412AC433FE for ; Thu, 19 May 2022 07:24:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234730AbiESHYz (ORCPT ); Thu, 19 May 2022 03:24:55 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42050 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234693AbiESHYa (ORCPT ); Thu, 19 May 2022 03:24:30 -0400 Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BFD6A8023D for ; Thu, 19 May 2022 00:24:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1652945066; x=1684481066; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Hxh5bC4XsnO/pdDFOXgRYBbR4ul2RVcQMyWdVYYH/P8=; b=EraxgDUlJ1ZzMklukTB5d0b9B0Nbk2pvcjMCdmU6oeOXaRHieKuZ51kj qYNoowslA2PETYHTj2EpSxB2GHMeTxs/25OKFi+y3zpK2znrbo3rfHtS7 5Imu7POWj1ztWTc1c+jx8/pGha32+9mRn7gxDeKPeA299ts7rkMjmlOaw 8eMHmxZtnUXxk3FLdOcmZnMmZni0Xh2YIHEU7Kbix/h39ySfvC/2sdgPE xKfVMxS/nbb2bqw1TletRVMBfFQqEJ3VcM86Cg5pd6Dl2DQbEUnTzXzGJ ZWRmcbpqja/XnCBQWYHbN8bvAuMoUi0ZK6ixLZKWYxqHoKcO/swJhpC4J A==; X-IronPort-AV: E=McAfee;i="6400,9594,10351"; a="272195865" X-IronPort-AV: E=Sophos;i="5.91,236,1647327600"; d="scan'208";a="272195865" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 May 2022 00:24:26 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.91,236,1647327600"; d="scan'208";a="714853031" Received: from allen-box.sh.intel.com ([10.239.159.48]) by fmsmga001.fm.intel.com with ESMTP; 19 May 2022 00:24:22 -0700 From: Lu Baolu To: Joerg Roedel , Jason Gunthorpe , Christoph Hellwig , Kevin Tian , Ashok Raj , Will Deacon , Robin Murphy , Jean-Philippe Brucker , Dave Jiang , Vinod Koul Cc: Eric Auger , Liu Yi L , Jacob jun Pan , iommu@lists.linux-foundation.org, linux-kernel@vger.kernel.org, Lu Baolu , Jacob Pan Subject: [PATCH v7 02/10] iommu: Remove SVM_FLAG_SUPERVISOR_MODE support Date: Thu, 19 May 2022 15:20:39 +0800 Message-Id: <20220519072047.2996983-3-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220519072047.2996983-1-baolu.lu@linux.intel.com> References: <20220519072047.2996983-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org The current kernel DMA with PASID support is based on the SVA with a flag SVM_FLAG_SUPERVISOR_MODE. The IOMMU driver binds the kernel memory address space to a PASID of the device. The device driver programs the device with kernel virtual address (KVA) for DMA access. There have been security and functional issues with this approach: - The lack of IOTLB synchronization upon kernel page table updates. (vmalloc, module/BPF loading, CONFIG_DEBUG_PAGEALLOC etc.) - Other than slight more protection, using kernel virtual address (KVA) has little advantage over physical address. There are also no use cases yet where DMA engines need kernel virtual addresses for in-kernel DMA. This removes SVM_FLAG_SUPERVISOR_MODE support from the IOMMU interface. The device drivers are suggested to handle kernel DMA with PASID through the kernel DMA APIs. The drvdata parameter in iommu_sva_bind_device() and all callbacks is not needed anymore. Cleanup them as well. Link: https://lore.kernel.org/linux-iommu/20210511194726.GP1002214@nvidia.c= om/ Signed-off-by: Jacob Pan Signed-off-by: Lu Baolu Reviewed-by: Jason Gunthorpe Reviewed-by: Jean-Philippe Brucker Reviewed-by: Kevin Tian --- include/linux/intel-iommu.h | 3 +- include/linux/intel-svm.h | 13 ----- include/linux/iommu.h | 8 +-- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 5 +- drivers/dma/idxd/cdev.c | 2 +- drivers/dma/idxd/init.c | 24 +------- .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 3 +- drivers/iommu/intel/svm.c | 57 +++++-------------- drivers/iommu/iommu.c | 5 +- drivers/misc/uacce/uacce.c | 2 +- 10 files changed, 26 insertions(+), 96 deletions(-) diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h index 4f29139bbfc3..df23300cfa88 100644 --- a/include/linux/intel-iommu.h +++ b/include/linux/intel-iommu.h @@ -739,8 +739,7 @@ struct intel_iommu *device_to_iommu(struct device *dev,= u8 *bus, u8 *devfn); extern void intel_svm_check(struct intel_iommu *iommu); extern int intel_svm_enable_prq(struct intel_iommu *iommu); extern int intel_svm_finish_prq(struct intel_iommu *iommu); -struct iommu_sva *intel_svm_bind(struct device *dev, struct mm_struct *mm, - void *drvdata); +struct iommu_sva *intel_svm_bind(struct device *dev, struct mm_struct *mm); void intel_svm_unbind(struct iommu_sva *handle); u32 intel_svm_get_pasid(struct iommu_sva *handle); int intel_svm_page_response(struct device *dev, struct iommu_fault_event *= evt, diff --git a/include/linux/intel-svm.h b/include/linux/intel-svm.h index 207ef06ba3e1..f9a0d44f6fdb 100644 --- a/include/linux/intel-svm.h +++ b/include/linux/intel-svm.h @@ -13,17 +13,4 @@ #define PRQ_RING_MASK ((0x1000 << PRQ_ORDER) - 0x20) #define PRQ_DEPTH ((0x1000 << PRQ_ORDER) >> 5) =20 -/* - * The SVM_FLAG_SUPERVISOR_MODE flag requests a PASID which can be used on= ly - * for access to kernel addresses. No IOTLB flushes are automatically done - * for kernel mappings; it is valid only for access to the kernel's static - * 1:1 mapping of physical memory =E2=80=94 not to vmalloc or even module = mappings. - * A future API addition may permit the use of such ranges, by means of an - * explicit IOTLB flush call (akin to the DMA API's unmap method). - * - * It is unlikely that we will ever hook into flush_tlb_kernel_range() to - * do such IOTLB flushes automatically. - */ -#define SVM_FLAG_SUPERVISOR_MODE BIT(0) - #endif /* __INTEL_SVM_H__ */ diff --git a/include/linux/iommu.h b/include/linux/iommu.h index da423e87f248..0c358b7c583b 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -243,8 +243,7 @@ struct iommu_ops { int (*dev_enable_feat)(struct device *dev, enum iommu_dev_features f); int (*dev_disable_feat)(struct device *dev, enum iommu_dev_features f); =20 - struct iommu_sva *(*sva_bind)(struct device *dev, struct mm_struct *mm, - void *drvdata); + struct iommu_sva *(*sva_bind)(struct device *dev, struct mm_struct *mm); void (*sva_unbind)(struct iommu_sva *handle); u32 (*sva_get_pasid)(struct iommu_sva *handle); =20 @@ -667,8 +666,7 @@ int iommu_dev_disable_feature(struct device *dev, enum = iommu_dev_features f); bool iommu_dev_feature_enabled(struct device *dev, enum iommu_dev_features= f); =20 struct iommu_sva *iommu_sva_bind_device(struct device *dev, - struct mm_struct *mm, - void *drvdata); + struct mm_struct *mm); void iommu_sva_unbind_device(struct iommu_sva *handle); u32 iommu_sva_get_pasid(struct iommu_sva *handle); =20 @@ -1010,7 +1008,7 @@ iommu_dev_disable_feature(struct device *dev, enum io= mmu_dev_features feat) } =20 static inline struct iommu_sva * -iommu_sva_bind_device(struct device *dev, struct mm_struct *mm, void *drvd= ata) +iommu_sva_bind_device(struct device *dev, struct mm_struct *mm) { return NULL; } diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/ar= m/arm-smmu-v3/arm-smmu-v3.h index cd48590ada30..d2ba86470c42 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -754,8 +754,7 @@ bool arm_smmu_master_sva_enabled(struct arm_smmu_master= *master); int arm_smmu_master_enable_sva(struct arm_smmu_master *master); int arm_smmu_master_disable_sva(struct arm_smmu_master *master); bool arm_smmu_master_iopf_supported(struct arm_smmu_master *master); -struct iommu_sva *arm_smmu_sva_bind(struct device *dev, struct mm_struct *= mm, - void *drvdata); +struct iommu_sva *arm_smmu_sva_bind(struct device *dev, struct mm_struct *= mm); void arm_smmu_sva_unbind(struct iommu_sva *handle); u32 arm_smmu_sva_get_pasid(struct iommu_sva *handle); void arm_smmu_sva_notifier_synchronize(void); @@ -791,7 +790,7 @@ static inline bool arm_smmu_master_iopf_supported(struc= t arm_smmu_master *master } =20 static inline struct iommu_sva * -arm_smmu_sva_bind(struct device *dev, struct mm_struct *mm, void *drvdata) +arm_smmu_sva_bind(struct device *dev, struct mm_struct *mm) { return ERR_PTR(-ENODEV); } diff --git a/drivers/dma/idxd/cdev.c b/drivers/dma/idxd/cdev.c index b9b2b4a4124e..312ec37ebf91 100644 --- a/drivers/dma/idxd/cdev.c +++ b/drivers/dma/idxd/cdev.c @@ -100,7 +100,7 @@ static int idxd_cdev_open(struct inode *inode, struct f= ile *filp) filp->private_data =3D ctx; =20 if (device_pasid_enabled(idxd)) { - sva =3D iommu_sva_bind_device(dev, current->mm, NULL); + sva =3D iommu_sva_bind_device(dev, current->mm); if (IS_ERR(sva)) { rc =3D PTR_ERR(sva); dev_err(dev, "pasid allocation failed: %d\n", rc); diff --git a/drivers/dma/idxd/init.c b/drivers/dma/idxd/init.c index 993a5dcca24f..1fd80c63248a 100644 --- a/drivers/dma/idxd/init.c +++ b/drivers/dma/idxd/init.c @@ -466,29 +466,7 @@ static struct idxd_device *idxd_alloc(struct pci_dev *= pdev, struct idxd_driver_d =20 static int idxd_enable_system_pasid(struct idxd_device *idxd) { - int flags; - unsigned int pasid; - struct iommu_sva *sva; - - flags =3D SVM_FLAG_SUPERVISOR_MODE; - - sva =3D iommu_sva_bind_device(&idxd->pdev->dev, NULL, &flags); - if (IS_ERR(sva)) { - dev_warn(&idxd->pdev->dev, - "iommu sva bind failed: %ld\n", PTR_ERR(sva)); - return PTR_ERR(sva); - } - - pasid =3D iommu_sva_get_pasid(sva); - if (pasid =3D=3D IOMMU_PASID_INVALID) { - iommu_sva_unbind_device(sva); - return -ENODEV; - } - - idxd->sva =3D sva; - idxd->pasid =3D pasid; - dev_dbg(&idxd->pdev->dev, "system pasid: %u\n", pasid); - return 0; + return -EOPNOTSUPP; } =20 static void idxd_disable_system_pasid(struct idxd_device *idxd) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iomm= u/arm/arm-smmu-v3/arm-smmu-v3-sva.c index 1ef7bbb4acf3..f155d406c5d5 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -367,8 +367,7 @@ __arm_smmu_sva_bind(struct device *dev, struct mm_struc= t *mm) return ERR_PTR(ret); } =20 -struct iommu_sva * -arm_smmu_sva_bind(struct device *dev, struct mm_struct *mm, void *drvdata) +struct iommu_sva *arm_smmu_sva_bind(struct device *dev, struct mm_struct *= mm) { struct iommu_sva *handle; struct iommu_domain *domain =3D iommu_get_domain_for_dev(dev); diff --git a/drivers/iommu/intel/svm.c b/drivers/iommu/intel/svm.c index 7ee37d996e15..d04880a291c3 100644 --- a/drivers/iommu/intel/svm.c +++ b/drivers/iommu/intel/svm.c @@ -313,8 +313,7 @@ static int pasid_to_svm_sdev(struct device *dev, unsign= ed int pasid, return 0; } =20 -static int intel_svm_alloc_pasid(struct device *dev, struct mm_struct *mm, - unsigned int flags) +static int intel_svm_alloc_pasid(struct device *dev, struct mm_struct *mm) { ioasid_t max_pasid =3D dev_is_pci(dev) ? pci_max_pasids(to_pci_dev(dev)) : intel_pasid_max_id; @@ -324,8 +323,7 @@ static int intel_svm_alloc_pasid(struct device *dev, st= ruct mm_struct *mm, =20 static struct iommu_sva *intel_svm_bind_mm(struct intel_iommu *iommu, struct device *dev, - struct mm_struct *mm, - unsigned int flags) + struct mm_struct *mm) { struct device_domain_info *info =3D dev_iommu_priv_get(dev); unsigned long iflags, sflags; @@ -341,22 +339,18 @@ static struct iommu_sva *intel_svm_bind_mm(struct int= el_iommu *iommu, =20 svm->pasid =3D mm->pasid; svm->mm =3D mm; - svm->flags =3D flags; INIT_LIST_HEAD_RCU(&svm->devs); =20 - if (!(flags & SVM_FLAG_SUPERVISOR_MODE)) { - svm->notifier.ops =3D &intel_mmuops; - ret =3D mmu_notifier_register(&svm->notifier, mm); - if (ret) { - kfree(svm); - return ERR_PTR(ret); - } + svm->notifier.ops =3D &intel_mmuops; + ret =3D mmu_notifier_register(&svm->notifier, mm); + if (ret) { + kfree(svm); + return ERR_PTR(ret); } =20 ret =3D pasid_private_add(svm->pasid, svm); if (ret) { - if (svm->notifier.ops) - mmu_notifier_unregister(&svm->notifier, mm); + mmu_notifier_unregister(&svm->notifier, mm); kfree(svm); return ERR_PTR(ret); } @@ -391,9 +385,7 @@ static struct iommu_sva *intel_svm_bind_mm(struct intel= _iommu *iommu, } =20 /* Setup the pasid table: */ - sflags =3D (flags & SVM_FLAG_SUPERVISOR_MODE) ? - PASID_FLAG_SUPERVISOR_MODE : 0; - sflags |=3D cpu_feature_enabled(X86_FEATURE_LA57) ? PASID_FLAG_FL5LP : 0; + sflags =3D cpu_feature_enabled(X86_FEATURE_LA57) ? PASID_FLAG_FL5LP : 0; spin_lock_irqsave(&iommu->lock, iflags); ret =3D intel_pasid_setup_first_level(iommu, dev, mm->pgd, mm->pasid, FLPT_DEFAULT_DID, sflags); @@ -410,8 +402,7 @@ static struct iommu_sva *intel_svm_bind_mm(struct intel= _iommu *iommu, kfree(sdev); free_svm: if (list_empty(&svm->devs)) { - if (svm->notifier.ops) - mmu_notifier_unregister(&svm->notifier, mm); + mmu_notifier_unregister(&svm->notifier, mm); pasid_private_remove(mm->pasid); kfree(svm); } @@ -767,7 +758,7 @@ static irqreturn_t prq_event_thread(int irq, void *d) * to unbind the mm while any page faults are outstanding. */ svm =3D pasid_private_find(req->pasid); - if (IS_ERR_OR_NULL(svm) || (svm->flags & SVM_FLAG_SUPERVISOR_MODE)) + if (IS_ERR_OR_NULL(svm)) goto bad_req; } =20 @@ -818,40 +809,20 @@ static irqreturn_t prq_event_thread(int irq, void *d) return IRQ_RETVAL(handled); } =20 -struct iommu_sva *intel_svm_bind(struct device *dev, struct mm_struct *mm,= void *drvdata) +struct iommu_sva *intel_svm_bind(struct device *dev, struct mm_struct *mm) { struct intel_iommu *iommu =3D device_to_iommu(dev, NULL, NULL); - unsigned int flags =3D 0; struct iommu_sva *sva; int ret; =20 - if (drvdata) - flags =3D *(unsigned int *)drvdata; - - if (flags & SVM_FLAG_SUPERVISOR_MODE) { - if (!ecap_srs(iommu->ecap)) { - dev_err(dev, "%s: Supervisor PASID not supported\n", - iommu->name); - return ERR_PTR(-EOPNOTSUPP); - } - - if (mm) { - dev_err(dev, "%s: Supervisor PASID with user provided mm\n", - iommu->name); - return ERR_PTR(-EINVAL); - } - - mm =3D &init_mm; - } - mutex_lock(&pasid_mutex); - ret =3D intel_svm_alloc_pasid(dev, mm, flags); + ret =3D intel_svm_alloc_pasid(dev, mm); if (ret) { mutex_unlock(&pasid_mutex); return ERR_PTR(ret); } =20 - sva =3D intel_svm_bind_mm(iommu, dev, mm, flags); + sva =3D intel_svm_bind_mm(iommu, dev, mm); mutex_unlock(&pasid_mutex); =20 return sva; diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c index 847ad47a2dfd..9955f58bd08c 100644 --- a/drivers/iommu/iommu.c +++ b/drivers/iommu/iommu.c @@ -2768,7 +2768,6 @@ EXPORT_SYMBOL_GPL(iommu_dev_feature_enabled); * iommu_sva_bind_device() - Bind a process address space to a device * @dev: the device * @mm: the mm to bind, caller must hold a reference to it - * @drvdata: opaque data pointer to pass to bind callback * * Create a bond between device and address space, allowing the device to = access * the mm using the returned PASID. If a bond already exists between @devi= ce and @@ -2781,7 +2780,7 @@ EXPORT_SYMBOL_GPL(iommu_dev_feature_enabled); * On error, returns an ERR_PTR value. */ struct iommu_sva * -iommu_sva_bind_device(struct device *dev, struct mm_struct *mm, void *drvd= ata) +iommu_sva_bind_device(struct device *dev, struct mm_struct *mm) { struct iommu_group *group; struct iommu_sva *handle =3D ERR_PTR(-EINVAL); @@ -2806,7 +2805,7 @@ iommu_sva_bind_device(struct device *dev, struct mm_s= truct *mm, void *drvdata) if (iommu_group_device_count(group) !=3D 1) goto out_unlock; =20 - handle =3D ops->sva_bind(dev, mm, drvdata); + handle =3D ops->sva_bind(dev, mm); =20 out_unlock: mutex_unlock(&group->mutex); diff --git a/drivers/misc/uacce/uacce.c b/drivers/misc/uacce/uacce.c index 281c54003edc..3238a867ea51 100644 --- a/drivers/misc/uacce/uacce.c +++ b/drivers/misc/uacce/uacce.c @@ -99,7 +99,7 @@ static int uacce_bind_queue(struct uacce_device *uacce, s= truct uacce_queue *q) if (!(uacce->flags & UACCE_DEV_SVA)) return 0; =20 - handle =3D iommu_sva_bind_device(uacce->parent, current->mm, NULL); + handle =3D iommu_sva_bind_device(uacce->parent, current->mm); if (IS_ERR(handle)) return PTR_ERR(handle); =20 --=20 2.25.1 From nobody Thu May 7 20:22:18 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5F0EAC4332F for ; Thu, 19 May 2022 07:25:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230097AbiESHZo (ORCPT ); Thu, 19 May 2022 03:25:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42340 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234298AbiESHYv (ORCPT ); Thu, 19 May 2022 03:24:51 -0400 Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3451A84A3B for ; Thu, 19 May 2022 00:24:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1652945073; x=1684481073; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=uQByY9B8csm2hhoYwsPpfjb7HNh5A0kJm90DdWqIp6E=; b=IdcW2/b1BELnGrLkh4XfaUn2choCukE0YcwZJrvDbPwUsti3GpVD3vqv jk/R8I1aSaatiZza6aO8V+PmYf5jYg0BLcRH3KuL0+wgwvntfe3u/Dzb1 EtpTS4Fi2OPVegJHWpfq59zaSwNB8OpP8+95W/2JZa7LxGsmQ1DTRSrE3 BAD1Y5994raKSu4tdQTG/M/rjcjO3BVtt9ioRa9olynsQDyG50FU2CB4t 7NSTUKiJfGni9Dbw2uyxAClUCjhrceSPrQ9S/aWNBv6obJEL1/llE2o2A tu5kFzycvE6Bpi/pM71stg5MOpwZxfrlEVktfpbkl+WMaSl2jT+Gcdj6Y w==; X-IronPort-AV: E=McAfee;i="6400,9594,10351"; a="272195892" X-IronPort-AV: E=Sophos;i="5.91,236,1647327600"; d="scan'208";a="272195892" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 May 2022 00:24:30 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.91,236,1647327600"; d="scan'208";a="714853046" Received: from allen-box.sh.intel.com ([10.239.159.48]) by fmsmga001.fm.intel.com with ESMTP; 19 May 2022 00:24:26 -0700 From: Lu Baolu To: Joerg Roedel , Jason Gunthorpe , Christoph Hellwig , Kevin Tian , Ashok Raj , Will Deacon , Robin Murphy , Jean-Philippe Brucker , Dave Jiang , Vinod Koul Cc: Eric Auger , Liu Yi L , Jacob jun Pan , iommu@lists.linux-foundation.org, linux-kernel@vger.kernel.org, Lu Baolu , Jean-Philippe Brucker Subject: [PATCH v7 03/10] iommu/sva: Add iommu_sva_domain support Date: Thu, 19 May 2022 15:20:40 +0800 Message-Id: <20220519072047.2996983-4-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220519072047.2996983-1-baolu.lu@linux.intel.com> References: <20220519072047.2996983-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The iommu_sva_domain represents a hardware pagetable that the IOMMU hardware could use for SVA translation. This adds some infrastructure to support SVA domain in the iommu common layer. It includes: - Add a new struct iommu_sva_domain and new IOMMU_DOMAIN_SVA domain type. - Add a new domain ops pointer in iommu_ops. The IOMMU drivers that support SVA should provide the callbacks. - Add helpers to allocate and free an SVA domain. - Add helpers to set an SVA domain to a device and the reverse operation. Some buses, like PCI, route packets without considering the PASID value. Thus a DMA target address with PASID might be treated as P2P if the address falls into the MMIO BAR of other devices in the group. To make things simple, the attach/detach interfaces only apply to devices belonging to the singleton groups, and the singleton is immutable in fabric i.e. not affected by hotplug. The iommu_set/block_device_pasid() can be used for other purposes, such as kernel DMA with pasid, mediation device, etc. Hence, it is put in the iommu.c. Suggested-by: Jean-Philippe Brucker Suggested-by: Jason Gunthorpe Signed-off-by: Lu Baolu Reviewed-by: Jean-Philippe Brucker --- include/linux/iommu.h | 51 +++++++++++++++++++++++++ drivers/iommu/iommu-sva-lib.h | 15 ++++++++ drivers/iommu/iommu-sva-lib.c | 48 +++++++++++++++++++++++ drivers/iommu/iommu.c | 71 +++++++++++++++++++++++++++++++++++ 4 files changed, 185 insertions(+) diff --git a/include/linux/iommu.h b/include/linux/iommu.h index 0c358b7c583b..e8cf82d46ce1 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -64,6 +64,9 @@ struct iommu_domain_geometry { #define __IOMMU_DOMAIN_PT (1U << 2) /* Domain is identity mapped */ #define __IOMMU_DOMAIN_DMA_FQ (1U << 3) /* DMA-API uses flush queue */ =20 +#define __IOMMU_DOMAIN_SHARED (1U << 4) /* Page table shared from CPU */ +#define __IOMMU_DOMAIN_HOST_VA (1U << 5) /* Host CPU virtual address */ + /* * This are the possible domain-types * @@ -86,6 +89,8 @@ struct iommu_domain_geometry { #define IOMMU_DOMAIN_DMA_FQ (__IOMMU_DOMAIN_PAGING | \ __IOMMU_DOMAIN_DMA_API | \ __IOMMU_DOMAIN_DMA_FQ) +#define IOMMU_DOMAIN_SVA (__IOMMU_DOMAIN_SHARED | \ + __IOMMU_DOMAIN_HOST_VA) =20 struct iommu_domain { unsigned type; @@ -254,6 +259,7 @@ struct iommu_ops { int (*def_domain_type)(struct device *dev); =20 const struct iommu_domain_ops *default_domain_ops; + const struct iommu_domain_ops *sva_domain_ops; unsigned long pgsize_bitmap; struct module *owner; }; @@ -262,6 +268,8 @@ struct iommu_ops { * struct iommu_domain_ops - domain specific operations * @attach_dev: attach an iommu domain to a device * @detach_dev: detach an iommu domain from a device + * @set_dev_pasid: set an iommu domain to a pasid of device + * @block_dev_pasid: block pasid of device from using iommu domain * @map: map a physically contiguous memory region to an iommu domain * @map_pages: map a physically contiguous set of pages of the same size to * an iommu domain. @@ -282,6 +290,10 @@ struct iommu_ops { struct iommu_domain_ops { int (*attach_dev)(struct iommu_domain *domain, struct device *dev); void (*detach_dev)(struct iommu_domain *domain, struct device *dev); + int (*set_dev_pasid)(struct iommu_domain *domain, struct device *dev, + ioasid_t pasid); + void (*block_dev_pasid)(struct iommu_domain *domain, struct device *dev, + ioasid_t pasid); =20 int (*map)(struct iommu_domain *domain, unsigned long iova, phys_addr_t paddr, size_t size, int prot, gfp_t gfp); @@ -677,6 +689,10 @@ int iommu_group_claim_dma_owner(struct iommu_group *gr= oup, void *owner); void iommu_group_release_dma_owner(struct iommu_group *group); bool iommu_group_dma_owner_claimed(struct iommu_group *group); =20 +int iommu_set_device_pasid(struct iommu_domain *domain, struct device *dev, + ioasid_t pasid); +void iommu_block_device_pasid(struct iommu_domain *domain, struct device *= dev, + ioasid_t pasid); #else /* CONFIG_IOMMU_API */ =20 struct iommu_ops {}; @@ -1050,6 +1066,17 @@ static inline bool iommu_group_dma_owner_claimed(str= uct iommu_group *group) { return false; } + +static inline int iommu_set_device_pasid(struct iommu_domain *domain, + struct device *dev, ioasid_t pasid) +{ + return -ENODEV; +} + +static inline void iommu_block_device_pasid(struct iommu_domain *domain, + struct device *dev, ioasid_t pasid) +{ +} #endif /* CONFIG_IOMMU_API */ =20 /** @@ -1075,4 +1102,28 @@ void iommu_debugfs_setup(void); static inline void iommu_debugfs_setup(void) {} #endif =20 +#ifdef CONFIG_IOMMU_SVA +struct iommu_domain * +iommu_sva_alloc_domain(struct bus_type *bus, struct mm_struct *mm); +void iommu_sva_free_domain(struct iommu_domain *domain); +int iommu_sva_set_domain(struct iommu_domain *domain, struct device *dev, + ioasid_t pasid); +#else /* CONFIG_IOMMU_SVA */ +static inline struct iommu_domain * +iommu_sva_alloc_domain(struct bus_type *bus, struct mm_struct *mm) +{ + return ERR_PTR(-EINVAL); +} + +static inline void iommu_sva_free_domain(struct iommu_domain *domain) +{ +} + +static inline int iommu_sva_set_domain(struct iommu_domain *domain, + struct device *dev, ioasid_t pasid) +{ + return -EINVAL; +} +#endif /* CONFIG_IOMMU_SVA */ + #endif /* __LINUX_IOMMU_H */ diff --git a/drivers/iommu/iommu-sva-lib.h b/drivers/iommu/iommu-sva-lib.h index 8909ea1094e3..1be21e6b93ec 100644 --- a/drivers/iommu/iommu-sva-lib.h +++ b/drivers/iommu/iommu-sva-lib.h @@ -7,6 +7,7 @@ =20 #include #include +#include =20 int iommu_sva_alloc_pasid(struct mm_struct *mm, ioasid_t min, ioasid_t max= ); struct mm_struct *iommu_sva_find(ioasid_t pasid); @@ -16,6 +17,20 @@ struct device; struct iommu_fault; struct iopf_queue; =20 +struct iommu_sva_domain { + struct iommu_domain domain; + struct mm_struct *mm; +}; + +#define to_sva_domain(d) container_of_safe(d, struct iommu_sva_domain, dom= ain) + +static inline struct mm_struct *domain_to_mm(struct iommu_domain *domain) +{ + struct iommu_sva_domain *sva_domain =3D to_sva_domain(domain); + + return sva_domain->mm; +} + #ifdef CONFIG_IOMMU_SVA int iommu_queue_iopf(struct iommu_fault *fault, void *cookie); =20 diff --git a/drivers/iommu/iommu-sva-lib.c b/drivers/iommu/iommu-sva-lib.c index 106506143896..210c376f6043 100644 --- a/drivers/iommu/iommu-sva-lib.c +++ b/drivers/iommu/iommu-sva-lib.c @@ -69,3 +69,51 @@ struct mm_struct *iommu_sva_find(ioasid_t pasid) return ioasid_find(&iommu_sva_pasid, pasid, __mmget_not_zero); } EXPORT_SYMBOL_GPL(iommu_sva_find); + +/* + * IOMMU SVA driver-oriented interfaces + */ +struct iommu_domain * +iommu_sva_alloc_domain(struct bus_type *bus, struct mm_struct *mm) +{ + struct iommu_sva_domain *sva_domain; + struct iommu_domain *domain; + + if (!bus->iommu_ops || !bus->iommu_ops->sva_domain_ops) + return ERR_PTR(-ENODEV); + + sva_domain =3D kzalloc(sizeof(*sva_domain), GFP_KERNEL); + if (!sva_domain) + return ERR_PTR(-ENOMEM); + + mmgrab(mm); + sva_domain->mm =3D mm; + + domain =3D &sva_domain->domain; + domain->type =3D IOMMU_DOMAIN_SVA; + domain->ops =3D bus->iommu_ops->sva_domain_ops; + + return domain; +} + +void iommu_sva_free_domain(struct iommu_domain *domain) +{ + struct iommu_sva_domain *sva_domain =3D to_sva_domain(domain); + + mmdrop(sva_domain->mm); + kfree(sva_domain); +} + +int iommu_sva_set_domain(struct iommu_domain *domain, struct device *dev, + ioasid_t pasid) +{ + struct bus_type *bus =3D dev->bus; + + if (!bus || !bus->iommu_ops || !bus->iommu_ops->sva_domain_ops) + return -ENODEV; + + if (domain->ops !=3D bus->iommu_ops->sva_domain_ops) + return -EINVAL; + + return iommu_set_device_pasid(domain, dev, pasid); +} diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c index 9955f58bd08c..789816e4b9d6 100644 --- a/drivers/iommu/iommu.c +++ b/drivers/iommu/iommu.c @@ -38,6 +38,7 @@ struct iommu_group { struct kobject kobj; struct kobject *devices_kobj; struct list_head devices; + struct xarray pasid_array; struct mutex mutex; void *iommu_data; void (*iommu_data_release)(void *iommu_data); @@ -640,6 +641,7 @@ struct iommu_group *iommu_group_alloc(void) mutex_init(&group->mutex); INIT_LIST_HEAD(&group->devices); INIT_LIST_HEAD(&group->entry); + xa_init(&group->pasid_array); =20 ret =3D ida_simple_get(&iommu_group_ida, 0, 0, GFP_KERNEL); if (ret < 0) { @@ -3251,3 +3253,72 @@ bool iommu_group_dma_owner_claimed(struct iommu_grou= p *group) return user; } EXPORT_SYMBOL_GPL(iommu_group_dma_owner_claimed); + +static bool device_group_immutable_singleton(struct device *dev) +{ + struct iommu_group *group =3D iommu_group_get(dev); + int count; + + if (!group) + return false; + + mutex_lock(&group->mutex); + count =3D iommu_group_device_count(group); + mutex_unlock(&group->mutex); + iommu_group_put(group); + + if (count !=3D 1) + return false; + + /* + * The PCI device could be considered to be fully isolated if all + * devices on the path from the device to the host-PCI bridge are + * protected from peer-to-peer DMA by ACS. + */ + if (dev_is_pci(dev)) + return pci_acs_path_enabled(to_pci_dev(dev), NULL, + REQ_ACS_FLAGS); + + return true; +} + +int iommu_set_device_pasid(struct iommu_domain *domain, struct device *dev, + ioasid_t pasid) +{ + struct iommu_group *group; + int ret =3D -EBUSY; + void *curr; + + if (!domain->ops->set_dev_pasid) + return -EOPNOTSUPP; + + if (!device_group_immutable_singleton(dev)) + return -EINVAL; + + group =3D iommu_group_get(dev); + mutex_lock(&group->mutex); + curr =3D xa_cmpxchg(&group->pasid_array, pasid, NULL, domain, GFP_KERNEL); + if (curr) + goto out_unlock; + ret =3D domain->ops->set_dev_pasid(domain, dev, pasid); + if (ret) + xa_erase(&group->pasid_array, pasid); +out_unlock: + mutex_unlock(&group->mutex); + iommu_group_put(group); + + return ret; +} + +void iommu_block_device_pasid(struct iommu_domain *domain, struct device *= dev, + ioasid_t pasid) +{ + struct iommu_group *group =3D iommu_group_get(dev); + + mutex_lock(&group->mutex); + domain->ops->block_dev_pasid(domain, dev, pasid); + xa_erase(&group->pasid_array, pasid); + mutex_unlock(&group->mutex); + + iommu_group_put(group); +} --=20 2.25.1 From nobody Thu May 7 20:22:18 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4E638C433F5 for ; Thu, 19 May 2022 07:25:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231438AbiESHZE (ORCPT ); Thu, 19 May 2022 03:25:04 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42376 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234696AbiESHYv (ORCPT ); Thu, 19 May 2022 03:24:51 -0400 Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BD84087205 for ; Thu, 19 May 2022 00:24:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1652945075; x=1684481075; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=z00PR00Z0bBOcue3nPUlEzoXnGwLMcqBoCnXf6Kl2GI=; b=MabUvEJZQsSnZRAt/DUK1s9aGTdD0+zHOUGl00tMV3r9BjN32Qxa9349 nvAI+eM+x8OKuiNKeV3ap8yokd3mO6fuL42r3Q/Ph74xM6ylUBs61Cp4m hRyiRLSTd/XPcndr69tygVHGp8inqYZaHPhexE1199kCYG6qrlcsy8yHG tOeHqccI/yN1SafqBuxEkF2CCvbGBAjQiGotTG3dZdNRcLke+hZJRLKjW hzsMMTNRgFR7Pfg8fk5Z5OPXrnZ85ODC9e0UwtLhYGwy4ZqkHkRehDa/R XrA9+83uKe4McR6Rplk19M+XXnA3A+YvCTlB3jDUtvxta8HHhaC/gcPHQ w==; X-IronPort-AV: E=McAfee;i="6400,9594,10351"; a="272195908" X-IronPort-AV: E=Sophos;i="5.91,236,1647327600"; d="scan'208";a="272195908" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 May 2022 00:24:34 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.91,236,1647327600"; d="scan'208";a="714853061" Received: from allen-box.sh.intel.com ([10.239.159.48]) by fmsmga001.fm.intel.com with ESMTP; 19 May 2022 00:24:31 -0700 From: Lu Baolu To: Joerg Roedel , Jason Gunthorpe , Christoph Hellwig , Kevin Tian , Ashok Raj , Will Deacon , Robin Murphy , Jean-Philippe Brucker , Dave Jiang , Vinod Koul Cc: Eric Auger , Liu Yi L , Jacob jun Pan , iommu@lists.linux-foundation.org, linux-kernel@vger.kernel.org, Lu Baolu Subject: [PATCH v7 04/10] iommu/vt-d: Add SVA domain support Date: Thu, 19 May 2022 15:20:41 +0800 Message-Id: <20220519072047.2996983-5-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220519072047.2996983-1-baolu.lu@linux.intel.com> References: <20220519072047.2996983-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Add support for domain ops callbacks for an SVA domain. Signed-off-by: Lu Baolu --- include/linux/intel-iommu.h | 4 ++++ drivers/iommu/intel/iommu.c | 4 ++++ drivers/iommu/intel/svm.c | 37 ++++++++++++++++++++++++++++++++----- 3 files changed, 40 insertions(+), 5 deletions(-) diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h index df23300cfa88..5e88eaa245aa 100644 --- a/include/linux/intel-iommu.h +++ b/include/linux/intel-iommu.h @@ -744,6 +744,10 @@ void intel_svm_unbind(struct iommu_sva *handle); u32 intel_svm_get_pasid(struct iommu_sva *handle); int intel_svm_page_response(struct device *dev, struct iommu_fault_event *= evt, struct iommu_page_response *msg); +int intel_svm_attach_dev_pasid(struct iommu_domain *domain, + struct device *dev, ioasid_t pasid); +void intel_svm_detach_dev_pasid(struct iommu_domain *domain, + struct device *dev, ioasid_t pasid); =20 struct intel_svm_dev { struct list_head list; diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c index e56b3a4b6998..2b6a52c87c73 100644 --- a/drivers/iommu/intel/iommu.c +++ b/drivers/iommu/intel/iommu.c @@ -4923,6 +4923,10 @@ const struct iommu_ops intel_iommu_ops =3D { .sva_unbind =3D intel_svm_unbind, .sva_get_pasid =3D intel_svm_get_pasid, .page_response =3D intel_svm_page_response, + .sva_domain_ops =3D &(const struct iommu_domain_ops) { + .set_dev_pasid =3D intel_svm_attach_dev_pasid, + .block_dev_pasid =3D intel_svm_detach_dev_pasid, + }, #endif .default_domain_ops =3D &(const struct iommu_domain_ops) { .attach_dev =3D intel_iommu_attach_device, diff --git a/drivers/iommu/intel/svm.c b/drivers/iommu/intel/svm.c index d04880a291c3..d575792441f3 100644 --- a/drivers/iommu/intel/svm.c +++ b/drivers/iommu/intel/svm.c @@ -323,6 +323,7 @@ static int intel_svm_alloc_pasid(struct device *dev, st= ruct mm_struct *mm) =20 static struct iommu_sva *intel_svm_bind_mm(struct intel_iommu *iommu, struct device *dev, + ioasid_t pasid, struct mm_struct *mm) { struct device_domain_info *info =3D dev_iommu_priv_get(dev); @@ -331,13 +332,13 @@ static struct iommu_sva *intel_svm_bind_mm(struct int= el_iommu *iommu, struct intel_svm *svm; int ret =3D 0; =20 - svm =3D pasid_private_find(mm->pasid); + svm =3D pasid_private_find(pasid); if (!svm) { svm =3D kzalloc(sizeof(*svm), GFP_KERNEL); if (!svm) return ERR_PTR(-ENOMEM); =20 - svm->pasid =3D mm->pasid; + svm->pasid =3D pasid; svm->mm =3D mm; INIT_LIST_HEAD_RCU(&svm->devs); =20 @@ -387,7 +388,7 @@ static struct iommu_sva *intel_svm_bind_mm(struct intel= _iommu *iommu, /* Setup the pasid table: */ sflags =3D cpu_feature_enabled(X86_FEATURE_LA57) ? PASID_FLAG_FL5LP : 0; spin_lock_irqsave(&iommu->lock, iflags); - ret =3D intel_pasid_setup_first_level(iommu, dev, mm->pgd, mm->pasid, + ret =3D intel_pasid_setup_first_level(iommu, dev, mm->pgd, pasid, FLPT_DEFAULT_DID, sflags); spin_unlock_irqrestore(&iommu->lock, iflags); =20 @@ -403,7 +404,7 @@ static struct iommu_sva *intel_svm_bind_mm(struct intel= _iommu *iommu, free_svm: if (list_empty(&svm->devs)) { mmu_notifier_unregister(&svm->notifier, mm); - pasid_private_remove(mm->pasid); + pasid_private_remove(pasid); kfree(svm); } =20 @@ -822,7 +823,7 @@ struct iommu_sva *intel_svm_bind(struct device *dev, st= ruct mm_struct *mm) return ERR_PTR(ret); } =20 - sva =3D intel_svm_bind_mm(iommu, dev, mm); + sva =3D intel_svm_bind_mm(iommu, dev, mm->pasid, mm); mutex_unlock(&pasid_mutex); =20 return sva; @@ -931,3 +932,29 @@ int intel_svm_page_response(struct device *dev, mutex_unlock(&pasid_mutex); return ret; } + +int intel_svm_attach_dev_pasid(struct iommu_domain *domain, + struct device *dev, ioasid_t pasid) +{ + struct device_domain_info *info =3D dev_iommu_priv_get(dev); + struct mm_struct *mm =3D domain_to_mm(domain); + struct intel_iommu *iommu =3D info->iommu; + struct iommu_sva *sva; + int ret =3D 0; + + mutex_lock(&pasid_mutex); + sva =3D intel_svm_bind_mm(iommu, dev, pasid, mm); + if (IS_ERR(sva)) + ret =3D PTR_ERR(sva); + mutex_unlock(&pasid_mutex); + + return ret; +} + +void intel_svm_detach_dev_pasid(struct iommu_domain *domain, + struct device *dev, ioasid_t pasid) +{ + mutex_lock(&pasid_mutex); + intel_svm_unbind_mm(dev, pasid); + mutex_unlock(&pasid_mutex); +} --=20 2.25.1 From nobody Thu May 7 20:22:18 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6D1D6C433EF for ; Thu, 19 May 2022 07:25:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234794AbiESHZi (ORCPT ); Thu, 19 May 2022 03:25:38 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42406 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234709AbiESHYv (ORCPT ); Thu, 19 May 2022 03:24:51 -0400 Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CDDA887210 for ; Thu, 19 May 2022 00:24:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1652945078; x=1684481078; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Plk0plqOguHnq6zJqwBQ6ucbqjFEQFVamoSa6HmPMUw=; b=VAbEy6R9nNsFhHzYS2rEgQG5wZwJQw1Qhuoko+im0jxiDhHMlNYpwwhA WGu4IUrEtq/I8z7Dc/89vRg9q8vCultuAwLjTVWMnqbvIXBegJMQyo1TA kK5Pfix5cOmq9e/bhyHCTFuhTCt0A6V6kW7i0n7yMSVG5FCKIMrik1l3f Z6Zrd3HPlpmqWUXWa41/0g1Rewh2yrArgAERLvOHUxkcdfjs7AAq725+M lZkzL3HepPmR46kKmGVRJv/ohFt3YchXbR4V/kXhnjsITBx9kUdMGbFer rcqOBM5+orSoX9f+YpdW8kNWigv3pf4nrFpwNL6j5yfmcY+JhQxO1iTx6 A==; X-IronPort-AV: E=McAfee;i="6400,9594,10351"; a="272195931" X-IronPort-AV: E=Sophos;i="5.91,236,1647327600"; d="scan'208";a="272195931" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 May 2022 00:24:38 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.91,236,1647327600"; d="scan'208";a="714853079" Received: from allen-box.sh.intel.com ([10.239.159.48]) by fmsmga001.fm.intel.com with ESMTP; 19 May 2022 00:24:34 -0700 From: Lu Baolu To: Joerg Roedel , Jason Gunthorpe , Christoph Hellwig , Kevin Tian , Ashok Raj , Will Deacon , Robin Murphy , Jean-Philippe Brucker , Dave Jiang , Vinod Koul Cc: Eric Auger , Liu Yi L , Jacob jun Pan , iommu@lists.linux-foundation.org, linux-kernel@vger.kernel.org, Lu Baolu Subject: [PATCH v7 05/10] arm-smmu-v3/sva: Add SVA domain support Date: Thu, 19 May 2022 15:20:42 +0800 Message-Id: <20220519072047.2996983-6-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220519072047.2996983-1-baolu.lu@linux.intel.com> References: <20220519072047.2996983-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Add support for domain ops callbacks for an SVA domain. Signed-off-by: Lu Baolu Reviewed-by: Jean-Philippe Brucker --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 4 ++ .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 46 +++++++++++++++++++ drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 6 +++ 3 files changed, 56 insertions(+) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/ar= m/arm-smmu-v3/arm-smmu-v3.h index d2ba86470c42..ec77f6a51ff9 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -758,6 +758,10 @@ struct iommu_sva *arm_smmu_sva_bind(struct device *dev= , struct mm_struct *mm); void arm_smmu_sva_unbind(struct iommu_sva *handle); u32 arm_smmu_sva_get_pasid(struct iommu_sva *handle); void arm_smmu_sva_notifier_synchronize(void); +int arm_smmu_sva_attach_dev_pasid(struct iommu_domain *domain, + struct device *dev, ioasid_t id); +void arm_smmu_sva_detach_dev_pasid(struct iommu_domain *domain, + struct device *dev, ioasid_t id); #else /* CONFIG_ARM_SMMU_V3_SVA */ static inline bool arm_smmu_sva_supported(struct arm_smmu_device *smmu) { diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iomm= u/arm/arm-smmu-v3/arm-smmu-v3-sva.c index f155d406c5d5..6969974ca89e 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -549,3 +549,49 @@ void arm_smmu_sva_notifier_synchronize(void) */ mmu_notifier_synchronize(); } + +int arm_smmu_sva_attach_dev_pasid(struct iommu_domain *domain, + struct device *dev, ioasid_t id) +{ + int ret =3D 0; + struct mm_struct *mm; + struct iommu_sva *handle; + + if (domain->type !=3D IOMMU_DOMAIN_SVA) + return -EINVAL; + + mm =3D domain_to_mm(domain); + if (WARN_ON(!mm)) + return -ENODEV; + + mutex_lock(&sva_lock); + handle =3D __arm_smmu_sva_bind(dev, mm); + if (IS_ERR(handle)) + ret =3D PTR_ERR(handle); + mutex_unlock(&sva_lock); + + return ret; +} + +void arm_smmu_sva_detach_dev_pasid(struct iommu_domain *domain, + struct device *dev, ioasid_t id) +{ + struct arm_smmu_bond *bond =3D NULL, *t; + struct mm_struct *mm =3D domain_to_mm(domain); + struct arm_smmu_master *master =3D dev_iommu_priv_get(dev); + + mutex_lock(&sva_lock); + list_for_each_entry(t, &master->bonds, list) { + if (t->mm =3D=3D mm) { + bond =3D t; + break; + } + } + + if (!WARN_ON(!bond) && refcount_dec_and_test(&bond->refs)) { + list_del(&bond->list); + arm_smmu_mmu_notifier_put(bond->smmu_mn); + kfree(bond); + } + mutex_unlock(&sva_lock); +} diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/ar= m/arm-smmu-v3/arm-smmu-v3.c index 6e2cd082c670..4ad3ca70cf89 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -2858,6 +2858,12 @@ static struct iommu_ops arm_smmu_ops =3D { .page_response =3D arm_smmu_page_response, .pgsize_bitmap =3D -1UL, /* Restricted during device attach */ .owner =3D THIS_MODULE, +#ifdef CONFIG_ARM_SMMU_V3_SVA + .sva_domain_ops =3D &(const struct iommu_domain_ops) { + .set_dev_pasid =3D arm_smmu_sva_attach_dev_pasid, + .block_dev_pasid =3D arm_smmu_sva_detach_dev_pasid, + }, +#endif .default_domain_ops =3D &(const struct iommu_domain_ops) { .attach_dev =3D arm_smmu_attach_dev, .map_pages =3D arm_smmu_map_pages, --=20 2.25.1 From nobody Thu May 7 20:22:18 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 76E2DC433F5 for ; Thu, 19 May 2022 07:25:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234814AbiESHZX (ORCPT ); Thu, 19 May 2022 03:25:23 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42382 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234761AbiESHYx (ORCPT ); Thu, 19 May 2022 03:24:53 -0400 Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BF82687224 for ; Thu, 19 May 2022 00:24:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1652945082; x=1684481082; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=GJH5xTT7hMx/rX/qXAHSoaXpu47CO+TDZwMogQOFMQU=; b=VFnep0XCV110DBCTQgrTQnbDcGWbZhu1rQgPEvO86+vB1VoA3RIqCyeu nPv7DliEubpj+3K6D0hk1hNR0ujlR7fcJ5aA5qT7iKuqcQt/165q5z5z9 xeTqwcmVzWFPUSWGM2HJAM0/S3VwDJ0Q4/CaUJghnYtin5oUcyf9z2P0c cjrQFlNCgEf7vLoi77DosW2x6qj/tFPBquQFouo7avRn7OIwTqbJFDoWr 1oSS2K4jt4kbmS8I9PMUHAFhwBEUjNiG6ClO3tNN6jOqZx44hok+8vvYw Yb19cUiKxbneGr26lfJv62t52DQ3iOAyOgxO0jt61I4+obyVGIS8pWJRg A==; X-IronPort-AV: E=McAfee;i="6400,9594,10351"; a="272195952" X-IronPort-AV: E=Sophos;i="5.91,236,1647327600"; d="scan'208";a="272195952" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 May 2022 00:24:42 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.91,236,1647327600"; d="scan'208";a="714853087" Received: from allen-box.sh.intel.com ([10.239.159.48]) by fmsmga001.fm.intel.com with ESMTP; 19 May 2022 00:24:38 -0700 From: Lu Baolu To: Joerg Roedel , Jason Gunthorpe , Christoph Hellwig , Kevin Tian , Ashok Raj , Will Deacon , Robin Murphy , Jean-Philippe Brucker , Dave Jiang , Vinod Koul Cc: Eric Auger , Liu Yi L , Jacob jun Pan , iommu@lists.linux-foundation.org, linux-kernel@vger.kernel.org, Lu Baolu Subject: [PATCH v7 06/10] iommu/sva: Refactoring iommu_sva_bind/unbind_device() Date: Thu, 19 May 2022 15:20:43 +0800 Message-Id: <20220519072047.2996983-7-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220519072047.2996983-1-baolu.lu@linux.intel.com> References: <20220519072047.2996983-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The existing iommu SVA interfaces are implemented by calling the SVA specific iommu ops provided by the IOMMU drivers. There's no need for any SVA specific ops in iommu_ops vector anymore as we can achieve this through the generic attach/detach_dev_pasid domain ops. This refactors the IOMMU SVA interfaces implementation by using the set/block_pasid_dev ops and align them with the concept of the SVA iommu domain. Put the new SVA code in the sva related file in order to make it self-contained. Signed-off-by: Lu Baolu --- include/linux/iommu.h | 48 ++++++++------ drivers/iommu/iommu-sva-lib.h | 1 + drivers/iommu/iommu-sva-lib.c | 113 ++++++++++++++++++++++++++++++++ drivers/iommu/iommu.c | 119 ++++++++-------------------------- 4 files changed, 170 insertions(+), 111 deletions(-) diff --git a/include/linux/iommu.h b/include/linux/iommu.h index e8cf82d46ce1..d9ac5ebe5bbb 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -635,6 +635,7 @@ struct iommu_fwspec { */ struct iommu_sva { struct device *dev; + refcount_t users; }; =20 int iommu_fwspec_init(struct device *dev, struct fwnode_handle *iommu_fwno= de, @@ -677,11 +678,6 @@ int iommu_dev_enable_feature(struct device *dev, enum = iommu_dev_features f); int iommu_dev_disable_feature(struct device *dev, enum iommu_dev_features = f); bool iommu_dev_feature_enabled(struct device *dev, enum iommu_dev_features= f); =20 -struct iommu_sva *iommu_sva_bind_device(struct device *dev, - struct mm_struct *mm); -void iommu_sva_unbind_device(struct iommu_sva *handle); -u32 iommu_sva_get_pasid(struct iommu_sva *handle); - int iommu_device_use_default_domain(struct device *dev); void iommu_device_unuse_default_domain(struct device *dev); =20 @@ -693,6 +689,8 @@ int iommu_set_device_pasid(struct iommu_domain *domain,= struct device *dev, ioasid_t pasid); void iommu_block_device_pasid(struct iommu_domain *domain, struct device *= dev, ioasid_t pasid); +struct iommu_domain * +iommu_get_domain_for_dev_pasid(struct device *dev, ioasid_t pasid); #else /* CONFIG_IOMMU_API */ =20 struct iommu_ops {}; @@ -1023,21 +1021,6 @@ iommu_dev_disable_feature(struct device *dev, enum i= ommu_dev_features feat) return -ENODEV; } =20 -static inline struct iommu_sva * -iommu_sva_bind_device(struct device *dev, struct mm_struct *mm) -{ - return NULL; -} - -static inline void iommu_sva_unbind_device(struct iommu_sva *handle) -{ -} - -static inline u32 iommu_sva_get_pasid(struct iommu_sva *handle) -{ - return IOMMU_PASID_INVALID; -} - static inline struct iommu_fwspec *dev_iommu_fwspec_get(struct device *dev) { return NULL; @@ -1077,6 +1060,12 @@ static inline void iommu_block_device_pasid(struct i= ommu_domain *domain, struct device *dev, ioasid_t pasid) { } + +static inline struct iommu_domain * +iommu_get_domain_for_dev_pasid(struct device *dev, ioasid_t pasid) +{ + return NULL; +} #endif /* CONFIG_IOMMU_API */ =20 /** @@ -1108,6 +1097,10 @@ iommu_sva_alloc_domain(struct bus_type *bus, struct = mm_struct *mm); void iommu_sva_free_domain(struct iommu_domain *domain); int iommu_sva_set_domain(struct iommu_domain *domain, struct device *dev, ioasid_t pasid); +struct iommu_sva *iommu_sva_bind_device(struct device *dev, + struct mm_struct *mm); +void iommu_sva_unbind_device(struct iommu_sva *handle); +u32 iommu_sva_get_pasid(struct iommu_sva *handle); #else /* CONFIG_IOMMU_SVA */ static inline struct iommu_domain * iommu_sva_alloc_domain(struct bus_type *bus, struct mm_struct *mm) @@ -1124,6 +1117,21 @@ static inline int iommu_sva_set_domain(struct iommu_= domain *domain, { return -EINVAL; } + +static inline struct iommu_sva * +iommu_sva_bind_device(struct device *dev, struct mm_struct *mm) +{ + return NULL; +} + +static inline void iommu_sva_unbind_device(struct iommu_sva *handle) +{ +} + +static inline u32 iommu_sva_get_pasid(struct iommu_sva *handle) +{ + return IOMMU_PASID_INVALID; +} #endif /* CONFIG_IOMMU_SVA */ =20 #endif /* __LINUX_IOMMU_H */ diff --git a/drivers/iommu/iommu-sva-lib.h b/drivers/iommu/iommu-sva-lib.h index 1be21e6b93ec..ebab5a8cb126 100644 --- a/drivers/iommu/iommu-sva-lib.h +++ b/drivers/iommu/iommu-sva-lib.h @@ -20,6 +20,7 @@ struct iopf_queue; struct iommu_sva_domain { struct iommu_domain domain; struct mm_struct *mm; + struct iommu_sva bond; }; =20 #define to_sva_domain(d) container_of_safe(d, struct iommu_sva_domain, dom= ain) diff --git a/drivers/iommu/iommu-sva-lib.c b/drivers/iommu/iommu-sva-lib.c index 210c376f6043..568e0f64edac 100644 --- a/drivers/iommu/iommu-sva-lib.c +++ b/drivers/iommu/iommu-sva-lib.c @@ -4,6 +4,8 @@ */ #include #include +#include +#include =20 #include "iommu-sva-lib.h" =20 @@ -117,3 +119,114 @@ int iommu_sva_set_domain(struct iommu_domain *domain,= struct device *dev, =20 return iommu_set_device_pasid(domain, dev, pasid); } + +/** + * iommu_sva_bind_device() - Bind a process address space to a device + * @dev: the device + * @mm: the mm to bind, caller must hold a reference to mm_users + * + * Create a bond between device and address space, allowing the device to = access + * the mm using the returned PASID. If a bond already exists between @devi= ce and + * @mm, it is returned and an additional reference is taken. Caller must c= all + * iommu_sva_unbind_device() to release each reference. + * + * iommu_dev_enable_feature(dev, IOMMU_DEV_FEAT_SVA) must be called first,= to + * initialize the required SVA features. + * + * On error, returns an ERR_PTR value. + */ +struct iommu_sva *iommu_sva_bind_device(struct device *dev, struct mm_stru= ct *mm) +{ + struct iommu_sva_domain *sva_domain; + struct iommu_domain *domain; + ioasid_t max_pasid =3D 0; + int ret =3D -EINVAL; + + /* Allocate mm->pasid if necessary. */ + if (!dev->iommu->iommu_dev->pasids) + return ERR_PTR(-EOPNOTSUPP); + + if (dev_is_pci(dev)) { + max_pasid =3D pci_max_pasids(to_pci_dev(dev)); + if (max_pasid < 0) + return ERR_PTR(max_pasid); + } else { + ret =3D device_property_read_u32(dev, "pasid-num-bits", + &max_pasid); + if (ret) + return ERR_PTR(ret); + max_pasid =3D (1UL << max_pasid); + } + max_pasid =3D min_t(u32, max_pasid, dev->iommu->iommu_dev->pasids); + ret =3D iommu_sva_alloc_pasid(mm, 1, max_pasid - 1); + if (ret) + return ERR_PTR(ret); + + mutex_lock(&iommu_sva_lock); + /* Search for an existing domain. */ + domain =3D iommu_get_domain_for_dev_pasid(dev, mm->pasid); + if (domain) { + sva_domain =3D to_sva_domain(domain); + refcount_inc(&sva_domain->bond.users); + goto out_success; + } + + /* Allocate a new domain and set it on device pasid. */ + domain =3D iommu_sva_alloc_domain(dev->bus, mm); + if (IS_ERR(domain)) { + ret =3D PTR_ERR(domain); + goto out_unlock; + } + + ret =3D iommu_sva_set_domain(domain, dev, mm->pasid); + if (ret) + goto out_free_domain; + sva_domain =3D to_sva_domain(domain); + sva_domain->bond.dev =3D dev; + refcount_set(&sva_domain->bond.users, 1); + +out_success: + mutex_unlock(&iommu_sva_lock); + return &sva_domain->bond; + +out_free_domain: + iommu_sva_free_domain(domain); +out_unlock: + mutex_unlock(&iommu_sva_lock); + + return ERR_PTR(ret); +} +EXPORT_SYMBOL_GPL(iommu_sva_bind_device); + +/** + * iommu_sva_unbind_device() - Remove a bond created with iommu_sva_bind_d= evice + * @handle: the handle returned by iommu_sva_bind_device() + * + * Put reference to a bond between device and address space. The device sh= ould + * not be issuing any more transaction for this PASID. All outstanding page + * requests for this PASID must have been flushed to the IOMMU. + */ +void iommu_sva_unbind_device(struct iommu_sva *handle) +{ + struct device *dev =3D handle->dev; + struct iommu_sva_domain *sva_domain =3D + container_of(handle, struct iommu_sva_domain, bond); + ioasid_t pasid =3D iommu_sva_get_pasid(handle); + + mutex_lock(&iommu_sva_lock); + if (refcount_dec_and_test(&sva_domain->bond.users)) { + iommu_block_device_pasid(&sva_domain->domain, dev, pasid); + iommu_sva_free_domain(&sva_domain->domain); + } + mutex_unlock(&iommu_sva_lock); +} +EXPORT_SYMBOL_GPL(iommu_sva_unbind_device); + +u32 iommu_sva_get_pasid(struct iommu_sva *handle) +{ + struct iommu_sva_domain *sva_domain =3D + container_of(handle, struct iommu_sva_domain, bond); + + return sva_domain->mm->pasid; +} +EXPORT_SYMBOL_GPL(iommu_sva_get_pasid); diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c index 789816e4b9d6..e49c5a5b8cc1 100644 --- a/drivers/iommu/iommu.c +++ b/drivers/iommu/iommu.c @@ -2766,97 +2766,6 @@ bool iommu_dev_feature_enabled(struct device *dev, e= num iommu_dev_features feat) } EXPORT_SYMBOL_GPL(iommu_dev_feature_enabled); =20 -/** - * iommu_sva_bind_device() - Bind a process address space to a device - * @dev: the device - * @mm: the mm to bind, caller must hold a reference to it - * - * Create a bond between device and address space, allowing the device to = access - * the mm using the returned PASID. If a bond already exists between @devi= ce and - * @mm, it is returned and an additional reference is taken. Caller must c= all - * iommu_sva_unbind_device() to release each reference. - * - * iommu_dev_enable_feature(dev, IOMMU_DEV_FEAT_SVA) must be called first,= to - * initialize the required SVA features. - * - * On error, returns an ERR_PTR value. - */ -struct iommu_sva * -iommu_sva_bind_device(struct device *dev, struct mm_struct *mm) -{ - struct iommu_group *group; - struct iommu_sva *handle =3D ERR_PTR(-EINVAL); - const struct iommu_ops *ops =3D dev_iommu_ops(dev); - - if (!ops->sva_bind) - return ERR_PTR(-ENODEV); - - group =3D iommu_group_get(dev); - if (!group) - return ERR_PTR(-ENODEV); - - /* Ensure device count and domain don't change while we're binding */ - mutex_lock(&group->mutex); - - /* - * To keep things simple, SVA currently doesn't support IOMMU groups - * with more than one device. Existing SVA-capable systems are not - * affected by the problems that required IOMMU groups (lack of ACS - * isolation, device ID aliasing and other hardware issues). - */ - if (iommu_group_device_count(group) !=3D 1) - goto out_unlock; - - handle =3D ops->sva_bind(dev, mm); - -out_unlock: - mutex_unlock(&group->mutex); - iommu_group_put(group); - - return handle; -} -EXPORT_SYMBOL_GPL(iommu_sva_bind_device); - -/** - * iommu_sva_unbind_device() - Remove a bond created with iommu_sva_bind_d= evice - * @handle: the handle returned by iommu_sva_bind_device() - * - * Put reference to a bond between device and address space. The device sh= ould - * not be issuing any more transaction for this PASID. All outstanding page - * requests for this PASID must have been flushed to the IOMMU. - */ -void iommu_sva_unbind_device(struct iommu_sva *handle) -{ - struct iommu_group *group; - struct device *dev =3D handle->dev; - const struct iommu_ops *ops =3D dev_iommu_ops(dev); - - if (!ops->sva_unbind) - return; - - group =3D iommu_group_get(dev); - if (!group) - return; - - mutex_lock(&group->mutex); - ops->sva_unbind(handle); - mutex_unlock(&group->mutex); - - iommu_group_put(group); -} -EXPORT_SYMBOL_GPL(iommu_sva_unbind_device); - -u32 iommu_sva_get_pasid(struct iommu_sva *handle) -{ - const struct iommu_ops *ops =3D dev_iommu_ops(handle->dev); - - if (!ops->sva_get_pasid) - return IOMMU_PASID_INVALID; - - return ops->sva_get_pasid(handle); -} -EXPORT_SYMBOL_GPL(iommu_sva_get_pasid); - /* * Changes the default domain of an iommu group that has *only* one device * @@ -3322,3 +3231,31 @@ void iommu_block_device_pasid(struct iommu_domain *d= omain, struct device *dev, =20 iommu_group_put(group); } + +/* + * This is a variant of iommu_get_domain_for_dev(). It returns the existing + * domain attached to pasid of a device. It's only for internal use of the + * IOMMU subsystem. The caller must take care to avoid any possible + * use-after-free case. + */ +struct iommu_domain * +iommu_get_domain_for_dev_pasid(struct device *dev, ioasid_t pasid) +{ + struct iommu_domain *domain; + struct iommu_group *group; + + if (!pasid_valid(pasid)) + return NULL; + + group =3D iommu_group_get(dev); + if (!group) + return NULL; + /* + * The xarray protects its internal state with RCU. Hence the domain + * obtained is either NULL or fully formed. + */ + domain =3D xa_load(&group->pasid_array, pasid); + iommu_group_put(group); + + return domain; +} --=20 2.25.1 From nobody Thu May 7 20:22:18 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5A947C433F5 for ; Thu, 19 May 2022 07:25:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230311AbiESHZe (ORCPT ); Thu, 19 May 2022 03:25:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42342 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234774AbiESHYx (ORCPT ); Thu, 19 May 2022 03:24:53 -0400 Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BB1A78023D for ; Thu, 19 May 2022 00:24:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1652945086; x=1684481086; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=vOJ+0X+gvpEqwA/UyYFKtBgGzzfy5dr+lfcjrc7KYhc=; b=VOycxL5Hs30+qtZk0p7Cue195Js0Ipx6oBOdmk0YzTfOETxQSQe8Qre3 Rlx99CReWWNKu9uSm3VYSPWjcP00a1hOekNybRFadgPeZngC29ZsfA6hV wXUqxwxOCrXNexcR1sekiXNqy0z2qUFWOk0sFHlBbnWEEHAbnuat7hewK QbGhBprm6MGIwUh/8Z2SJOeDB5AsDtBFfUEE5GYDmcYwv3l1HGOe4F6os bh4hd1L4cPLD+KLBkP7ggNUBgfwWyQo9GCY3cq4GWxB6OkRY5qQ8Vxkjh bo6kAggTIVD52eOypDg/YXkDoVwSCEc+4ltEbbQTw4fRFLB71whoOLBKm Q==; X-IronPort-AV: E=McAfee;i="6400,9594,10351"; a="272195971" X-IronPort-AV: E=Sophos;i="5.91,236,1647327600"; d="scan'208";a="272195971" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 May 2022 00:24:46 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.91,236,1647327600"; d="scan'208";a="714853111" Received: from allen-box.sh.intel.com ([10.239.159.48]) by fmsmga001.fm.intel.com with ESMTP; 19 May 2022 00:24:42 -0700 From: Lu Baolu To: Joerg Roedel , Jason Gunthorpe , Christoph Hellwig , Kevin Tian , Ashok Raj , Will Deacon , Robin Murphy , Jean-Philippe Brucker , Dave Jiang , Vinod Koul Cc: Eric Auger , Liu Yi L , Jacob jun Pan , iommu@lists.linux-foundation.org, linux-kernel@vger.kernel.org, Lu Baolu , Jean-Philippe Brucker Subject: [PATCH v7 07/10] iommu: Remove SVA related callbacks from iommu ops Date: Thu, 19 May 2022 15:20:44 +0800 Message-Id: <20220519072047.2996983-8-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220519072047.2996983-1-baolu.lu@linux.intel.com> References: <20220519072047.2996983-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" These ops'es have been replaced with the dev_attach/detach_pasid domain ops'es. There's no need for them anymore. Remove them to avoid dead code. Signed-off-by: Lu Baolu Reviewed-by: Jean-Philippe Brucker Reviewed-by: Kevin Tian --- include/linux/intel-iommu.h | 3 -- include/linux/iommu.h | 7 --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 16 ------ .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 40 --------------- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 3 -- drivers/iommu/intel/iommu.c | 3 -- drivers/iommu/intel/svm.c | 49 ------------------- 7 files changed, 121 deletions(-) diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h index 5e88eaa245aa..536f229fd274 100644 --- a/include/linux/intel-iommu.h +++ b/include/linux/intel-iommu.h @@ -739,9 +739,6 @@ struct intel_iommu *device_to_iommu(struct device *dev,= u8 *bus, u8 *devfn); extern void intel_svm_check(struct intel_iommu *iommu); extern int intel_svm_enable_prq(struct intel_iommu *iommu); extern int intel_svm_finish_prq(struct intel_iommu *iommu); -struct iommu_sva *intel_svm_bind(struct device *dev, struct mm_struct *mm); -void intel_svm_unbind(struct iommu_sva *handle); -u32 intel_svm_get_pasid(struct iommu_sva *handle); int intel_svm_page_response(struct device *dev, struct iommu_fault_event *= evt, struct iommu_page_response *msg); int intel_svm_attach_dev_pasid(struct iommu_domain *domain, diff --git a/include/linux/iommu.h b/include/linux/iommu.h index d9ac5ebe5bbb..e4ce2fe0e144 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -212,9 +212,6 @@ struct iommu_iotlb_gather { * @dev_has/enable/disable_feat: per device entries to check/enable/disable * iommu specific features. * @dev_feat_enabled: check enabled feature - * @sva_bind: Bind process address space to device - * @sva_unbind: Unbind process address space from device - * @sva_get_pasid: Get PASID associated to a SVA handle * @page_response: handle page request response * @def_domain_type: device default domain type, return value: * - IOMMU_DOMAIN_IDENTITY: must use an identity domain @@ -248,10 +245,6 @@ struct iommu_ops { int (*dev_enable_feat)(struct device *dev, enum iommu_dev_features f); int (*dev_disable_feat)(struct device *dev, enum iommu_dev_features f); =20 - struct iommu_sva *(*sva_bind)(struct device *dev, struct mm_struct *mm); - void (*sva_unbind)(struct iommu_sva *handle); - u32 (*sva_get_pasid)(struct iommu_sva *handle); - int (*page_response)(struct device *dev, struct iommu_fault_event *evt, struct iommu_page_response *msg); diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/ar= m/arm-smmu-v3/arm-smmu-v3.h index ec77f6a51ff9..0f0f5ba26dd5 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -754,9 +754,6 @@ bool arm_smmu_master_sva_enabled(struct arm_smmu_master= *master); int arm_smmu_master_enable_sva(struct arm_smmu_master *master); int arm_smmu_master_disable_sva(struct arm_smmu_master *master); bool arm_smmu_master_iopf_supported(struct arm_smmu_master *master); -struct iommu_sva *arm_smmu_sva_bind(struct device *dev, struct mm_struct *= mm); -void arm_smmu_sva_unbind(struct iommu_sva *handle); -u32 arm_smmu_sva_get_pasid(struct iommu_sva *handle); void arm_smmu_sva_notifier_synchronize(void); int arm_smmu_sva_attach_dev_pasid(struct iommu_domain *domain, struct device *dev, ioasid_t id); @@ -793,19 +790,6 @@ static inline bool arm_smmu_master_iopf_supported(stru= ct arm_smmu_master *master return false; } =20 -static inline struct iommu_sva * -arm_smmu_sva_bind(struct device *dev, struct mm_struct *mm) -{ - return ERR_PTR(-ENODEV); -} - -static inline void arm_smmu_sva_unbind(struct iommu_sva *handle) {} - -static inline u32 arm_smmu_sva_get_pasid(struct iommu_sva *handle) -{ - return IOMMU_PASID_INVALID; -} - static inline void arm_smmu_sva_notifier_synchronize(void) {} #endif /* CONFIG_ARM_SMMU_V3_SVA */ #endif /* _ARM_SMMU_V3_H */ diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iomm= u/arm/arm-smmu-v3/arm-smmu-v3-sva.c index 6969974ca89e..8290d66569f3 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -344,11 +344,6 @@ __arm_smmu_sva_bind(struct device *dev, struct mm_stru= ct *mm) if (!bond) return ERR_PTR(-ENOMEM); =20 - /* Allocate a PASID for this mm if necessary */ - ret =3D iommu_sva_alloc_pasid(mm, 1, (1U << master->ssid_bits) - 1); - if (ret) - goto err_free_bond; - bond->mm =3D mm; bond->sva.dev =3D dev; refcount_set(&bond->refs, 1); @@ -367,41 +362,6 @@ __arm_smmu_sva_bind(struct device *dev, struct mm_stru= ct *mm) return ERR_PTR(ret); } =20 -struct iommu_sva *arm_smmu_sva_bind(struct device *dev, struct mm_struct *= mm) -{ - struct iommu_sva *handle; - struct iommu_domain *domain =3D iommu_get_domain_for_dev(dev); - struct arm_smmu_domain *smmu_domain =3D to_smmu_domain(domain); - - if (smmu_domain->stage !=3D ARM_SMMU_DOMAIN_S1) - return ERR_PTR(-EINVAL); - - mutex_lock(&sva_lock); - handle =3D __arm_smmu_sva_bind(dev, mm); - mutex_unlock(&sva_lock); - return handle; -} - -void arm_smmu_sva_unbind(struct iommu_sva *handle) -{ - struct arm_smmu_bond *bond =3D sva_to_bond(handle); - - mutex_lock(&sva_lock); - if (refcount_dec_and_test(&bond->refs)) { - list_del(&bond->list); - arm_smmu_mmu_notifier_put(bond->smmu_mn); - kfree(bond); - } - mutex_unlock(&sva_lock); -} - -u32 arm_smmu_sva_get_pasid(struct iommu_sva *handle) -{ - struct arm_smmu_bond *bond =3D sva_to_bond(handle); - - return bond->mm->pasid; -} - bool arm_smmu_sva_supported(struct arm_smmu_device *smmu) { unsigned long reg, fld; diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/ar= m/arm-smmu-v3/arm-smmu-v3.c index 4ad3ca70cf89..b74f8964cc13 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -2852,9 +2852,6 @@ static struct iommu_ops arm_smmu_ops =3D { .dev_feat_enabled =3D arm_smmu_dev_feature_enabled, .dev_enable_feat =3D arm_smmu_dev_enable_feature, .dev_disable_feat =3D arm_smmu_dev_disable_feature, - .sva_bind =3D arm_smmu_sva_bind, - .sva_unbind =3D arm_smmu_sva_unbind, - .sva_get_pasid =3D arm_smmu_sva_get_pasid, .page_response =3D arm_smmu_page_response, .pgsize_bitmap =3D -1UL, /* Restricted during device attach */ .owner =3D THIS_MODULE, diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c index 2b6a52c87c73..10e07d59d4c8 100644 --- a/drivers/iommu/intel/iommu.c +++ b/drivers/iommu/intel/iommu.c @@ -4919,9 +4919,6 @@ const struct iommu_ops intel_iommu_ops =3D { .def_domain_type =3D device_def_domain_type, .pgsize_bitmap =3D SZ_4K, #ifdef CONFIG_INTEL_IOMMU_SVM - .sva_bind =3D intel_svm_bind, - .sva_unbind =3D intel_svm_unbind, - .sva_get_pasid =3D intel_svm_get_pasid, .page_response =3D intel_svm_page_response, .sva_domain_ops =3D &(const struct iommu_domain_ops) { .set_dev_pasid =3D intel_svm_attach_dev_pasid, diff --git a/drivers/iommu/intel/svm.c b/drivers/iommu/intel/svm.c index d575792441f3..e412a442d9a4 100644 --- a/drivers/iommu/intel/svm.c +++ b/drivers/iommu/intel/svm.c @@ -313,14 +313,6 @@ static int pasid_to_svm_sdev(struct device *dev, unsig= ned int pasid, return 0; } =20 -static int intel_svm_alloc_pasid(struct device *dev, struct mm_struct *mm) -{ - ioasid_t max_pasid =3D dev_is_pci(dev) ? - pci_max_pasids(to_pci_dev(dev)) : intel_pasid_max_id; - - return iommu_sva_alloc_pasid(mm, PASID_MIN, max_pasid - 1); -} - static struct iommu_sva *intel_svm_bind_mm(struct intel_iommu *iommu, struct device *dev, ioasid_t pasid, @@ -810,47 +802,6 @@ static irqreturn_t prq_event_thread(int irq, void *d) return IRQ_RETVAL(handled); } =20 -struct iommu_sva *intel_svm_bind(struct device *dev, struct mm_struct *mm) -{ - struct intel_iommu *iommu =3D device_to_iommu(dev, NULL, NULL); - struct iommu_sva *sva; - int ret; - - mutex_lock(&pasid_mutex); - ret =3D intel_svm_alloc_pasid(dev, mm); - if (ret) { - mutex_unlock(&pasid_mutex); - return ERR_PTR(ret); - } - - sva =3D intel_svm_bind_mm(iommu, dev, mm->pasid, mm); - mutex_unlock(&pasid_mutex); - - return sva; -} - -void intel_svm_unbind(struct iommu_sva *sva) -{ - struct intel_svm_dev *sdev =3D to_intel_svm_dev(sva); - - mutex_lock(&pasid_mutex); - intel_svm_unbind_mm(sdev->dev, sdev->pasid); - mutex_unlock(&pasid_mutex); -} - -u32 intel_svm_get_pasid(struct iommu_sva *sva) -{ - struct intel_svm_dev *sdev; - u32 pasid; - - mutex_lock(&pasid_mutex); - sdev =3D to_intel_svm_dev(sva); - pasid =3D sdev->pasid; - mutex_unlock(&pasid_mutex); - - return pasid; -} - int intel_svm_page_response(struct device *dev, struct iommu_fault_event *evt, struct iommu_page_response *msg) --=20 2.25.1 From nobody Thu May 7 20:22:18 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4DC1DC4332F for ; Thu, 19 May 2022 07:25:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234786AbiESHZQ (ORCPT ); Thu, 19 May 2022 03:25:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42438 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234779AbiESHYx (ORCPT ); Thu, 19 May 2022 03:24:53 -0400 Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A4BBA8020E for ; Thu, 19 May 2022 00:24:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1652945090; x=1684481090; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=YeJzuFoZ/UdMGyllJXeJZ4l5ck5D6EfYJagEliigKTg=; b=TFXEFeZ7CP/KYqvF7TTapsG0gbxd/R7v9gWoCQDcw3EbM/uEgFcj0Kjd Q1lCjWBXOapcfdSLkEByuneZ4ab9ADrU/1ZTYFL9ctkgnIMcmqXKmBlww mV6yVml2zrDva/+WuIqeaNlTR+pBpCuFuMAwdNXQxjr2FSm9z9WV7Sxiu xPn7d8maTn8SZT8DNgTiRZzxAE0SJrbfhL8MymXyNVBAHYY/e45g5BGb2 A/uOoajTM+PgTLUa6ehq18UYcFfLzNkK1gdkTYf563l2sy7hPcfW047oZ alRfAtMLG22EUd4dTQDEBnW0v8KP4rtsj27+41InmxSEUD9vdnKJocu3w g==; X-IronPort-AV: E=McAfee;i="6400,9594,10351"; a="272195986" X-IronPort-AV: E=Sophos;i="5.91,236,1647327600"; d="scan'208";a="272195986" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 May 2022 00:24:50 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.91,236,1647327600"; d="scan'208";a="714853152" Received: from allen-box.sh.intel.com ([10.239.159.48]) by fmsmga001.fm.intel.com with ESMTP; 19 May 2022 00:24:46 -0700 From: Lu Baolu To: Joerg Roedel , Jason Gunthorpe , Christoph Hellwig , Kevin Tian , Ashok Raj , Will Deacon , Robin Murphy , Jean-Philippe Brucker , Dave Jiang , Vinod Koul Cc: Eric Auger , Liu Yi L , Jacob jun Pan , iommu@lists.linux-foundation.org, linux-kernel@vger.kernel.org, Lu Baolu , Jean-Philippe Brucker Subject: [PATCH v7 08/10] iommu: Prepare IOMMU domain for IOPF Date: Thu, 19 May 2022 15:20:45 +0800 Message-Id: <20220519072047.2996983-9-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220519072047.2996983-1-baolu.lu@linux.intel.com> References: <20220519072047.2996983-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" This adds some mechanisms around the iommu_domain so that the I/O page fault handling framework could route a page fault to the domain and call the fault handler from it. Add pointers to the page fault handler and its private data in struct iommu_domain. The fault handler will be called with the private data as a parameter once a page fault is routed to the domain. Any kernel component which owns an iommu domain could install handler and its private parameter so that the page fault could be further routed and handled. This also prepares the SVA implementation to be the first consumer of the per-domain page fault handling model. Suggested-by: Jean-Philippe Brucker Signed-off-by: Lu Baolu Reviewed-by: Jean-Philippe Brucker --- include/linux/iommu.h | 3 ++ drivers/iommu/io-pgfault.c | 7 ++++ drivers/iommu/iommu-sva-lib.c | 65 +++++++++++++++++++++++++++++++++++ 3 files changed, 75 insertions(+) diff --git a/include/linux/iommu.h b/include/linux/iommu.h index e4ce2fe0e144..45f274b2640d 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -100,6 +100,9 @@ struct iommu_domain { void *handler_token; struct iommu_domain_geometry geometry; struct iommu_dma_cookie *iova_cookie; + enum iommu_page_response_code (*iopf_handler)(struct iommu_fault *fault, + void *data); + void *fault_data; }; =20 static inline bool iommu_is_dma_domain(struct iommu_domain *domain) diff --git a/drivers/iommu/io-pgfault.c b/drivers/iommu/io-pgfault.c index 1df8c1dcae77..aee9e033012f 100644 --- a/drivers/iommu/io-pgfault.c +++ b/drivers/iommu/io-pgfault.c @@ -181,6 +181,13 @@ static void iopf_handle_group(struct work_struct *work) * request completes, outstanding faults will have been dealt with by the = time * the PASID is freed. * + * Any valid page fault will be eventually routed to an iommu domain and t= he + * page fault handler installed there will get called. The users of this + * handling framework should guarantee that the iommu domain could only be + * freed after the device has stopped generating page faults (or the iommu + * hardware has been set to block the page faults) and the pending page fa= ults + * have been flushed. + * * Return: 0 on success and <0 on error. */ int iommu_queue_iopf(struct iommu_fault *fault, void *cookie) diff --git a/drivers/iommu/iommu-sva-lib.c b/drivers/iommu/iommu-sva-lib.c index 568e0f64edac..317ab8e8c149 100644 --- a/drivers/iommu/iommu-sva-lib.c +++ b/drivers/iommu/iommu-sva-lib.c @@ -72,6 +72,69 @@ struct mm_struct *iommu_sva_find(ioasid_t pasid) } EXPORT_SYMBOL_GPL(iommu_sva_find); =20 +/* + * I/O page fault handler for SVA + * + * Copied from io-pgfault.c with mmget_not_zero() added before + * mmap_read_lock(). + */ +static enum iommu_page_response_code +iommu_sva_handle_iopf(struct iommu_fault *fault, void *data) +{ + vm_fault_t ret; + struct mm_struct *mm; + struct vm_area_struct *vma; + unsigned int access_flags =3D 0; + struct iommu_domain *domain =3D data; + unsigned int fault_flags =3D FAULT_FLAG_REMOTE; + struct iommu_fault_page_request *prm =3D &fault->prm; + enum iommu_page_response_code status =3D IOMMU_PAGE_RESP_INVALID; + + if (!(prm->flags & IOMMU_FAULT_PAGE_REQUEST_PASID_VALID)) + return status; + + mm =3D domain_to_mm(domain); + if (IS_ERR_OR_NULL(mm) || !mmget_not_zero(mm)) + return status; + + mmap_read_lock(mm); + + vma =3D find_extend_vma(mm, prm->addr); + if (!vma) + /* Unmapped area */ + goto out_put_mm; + + if (prm->perm & IOMMU_FAULT_PERM_READ) + access_flags |=3D VM_READ; + + if (prm->perm & IOMMU_FAULT_PERM_WRITE) { + access_flags |=3D VM_WRITE; + fault_flags |=3D FAULT_FLAG_WRITE; + } + + if (prm->perm & IOMMU_FAULT_PERM_EXEC) { + access_flags |=3D VM_EXEC; + fault_flags |=3D FAULT_FLAG_INSTRUCTION; + } + + if (!(prm->perm & IOMMU_FAULT_PERM_PRIV)) + fault_flags |=3D FAULT_FLAG_USER; + + if (access_flags & ~vma->vm_flags) + /* Access fault */ + goto out_put_mm; + + ret =3D handle_mm_fault(vma, prm->addr, fault_flags, NULL); + status =3D ret & VM_FAULT_ERROR ? IOMMU_PAGE_RESP_INVALID : + IOMMU_PAGE_RESP_SUCCESS; + +out_put_mm: + mmap_read_unlock(mm); + mmput(mm); + + return status; +} + /* * IOMMU SVA driver-oriented interfaces */ @@ -94,6 +157,8 @@ iommu_sva_alloc_domain(struct bus_type *bus, struct mm_s= truct *mm) domain =3D &sva_domain->domain; domain->type =3D IOMMU_DOMAIN_SVA; domain->ops =3D bus->iommu_ops->sva_domain_ops; + domain->iopf_handler =3D iommu_sva_handle_iopf; + domain->fault_data =3D domain; =20 return domain; } --=20 2.25.1 From nobody Thu May 7 20:22:18 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id ABD09C433F5 for ; Thu, 19 May 2022 07:25:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234718AbiESHZK (ORCPT ); Thu, 19 May 2022 03:25:10 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42520 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234745AbiESHY4 (ORCPT ); Thu, 19 May 2022 03:24:56 -0400 Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1871784A0D for ; Thu, 19 May 2022 00:24:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1652945095; x=1684481095; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=ZhVJrH8ZIL7/etGs0byavRR7lBKr0I4+Mu5OuvDVjHU=; b=ApF2/zvGq7koW1vMvoJAZ9akghVa1y9UZ+AlwTd5Qx+nNiMtarfnSCIj HDIOieJgK9rtLhscbsCj3L92DApc37pYGj+fwXGkcmR8YV83cs++kPhad /p8MXEOPX/XyoHc+wai1EZ7IMexZByVxAkgd93NECJdabKYUZ+paZYVEo Qr/dnK0INsBM7oHeuZZp21B+O9NmRZ0+9HXVWerCX4M1U8uJf9wSJfS0R iQE/WB83ualMGon6AZxqG4xBweUwPmQb6r9DpDtC9sq6Fn3zAxMV9WO78 gI5DG8Jn4CeiUPt/qxVt+yL4mOuoSlgolM7Y87NtG7UGYI21xpYhfRs15 A==; X-IronPort-AV: E=McAfee;i="6400,9594,10351"; a="272196011" X-IronPort-AV: E=Sophos;i="5.91,236,1647327600"; d="scan'208";a="272196011" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 May 2022 00:24:54 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.91,236,1647327600"; d="scan'208";a="714853169" Received: from allen-box.sh.intel.com ([10.239.159.48]) by fmsmga001.fm.intel.com with ESMTP; 19 May 2022 00:24:50 -0700 From: Lu Baolu To: Joerg Roedel , Jason Gunthorpe , Christoph Hellwig , Kevin Tian , Ashok Raj , Will Deacon , Robin Murphy , Jean-Philippe Brucker , Dave Jiang , Vinod Koul Cc: Eric Auger , Liu Yi L , Jacob jun Pan , iommu@lists.linux-foundation.org, linux-kernel@vger.kernel.org, Lu Baolu , Jean-Philippe Brucker Subject: [PATCH v7 09/10] iommu: Per-domain I/O page fault handling Date: Thu, 19 May 2022 15:20:46 +0800 Message-Id: <20220519072047.2996983-10-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220519072047.2996983-1-baolu.lu@linux.intel.com> References: <20220519072047.2996983-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Tweak the I/O page fault handling framework to route the page faults to the domain and call the page fault handler retrieved from the domain. This makes the I/O page fault handling framework possible to serve more usage scenarios as long as they have an IOMMU domain and install a page fault handler in it. Some unused functions are also removed to avoid dead code. The iommu_get_domain_for_dev_pasid() which retrieves attached domain for a {device, PASID} pair is used. It will be used by the page fault handling framework which knows {device, PASID} reported from the iommu driver. We have a guarantee that the SVA domain doesn't go away during IOPF handling, because unbind() waits for pending faults with iopf_queue_flush_dev() before freeing the domain. Hence, there's no need to synchronize life cycle of the iommu domains between the unbind() and the interrupt threads. Signed-off-by: Lu Baolu Reviewed-by: Jean-Philippe Brucker --- drivers/iommu/io-pgfault.c | 64 +++++--------------------------------- 1 file changed, 7 insertions(+), 57 deletions(-) diff --git a/drivers/iommu/io-pgfault.c b/drivers/iommu/io-pgfault.c index aee9e033012f..4f24ec703479 100644 --- a/drivers/iommu/io-pgfault.c +++ b/drivers/iommu/io-pgfault.c @@ -69,69 +69,18 @@ static int iopf_complete_group(struct device *dev, stru= ct iopf_fault *iopf, return iommu_page_response(dev, &resp); } =20 -static enum iommu_page_response_code -iopf_handle_single(struct iopf_fault *iopf) -{ - vm_fault_t ret; - struct mm_struct *mm; - struct vm_area_struct *vma; - unsigned int access_flags =3D 0; - unsigned int fault_flags =3D FAULT_FLAG_REMOTE; - struct iommu_fault_page_request *prm =3D &iopf->fault.prm; - enum iommu_page_response_code status =3D IOMMU_PAGE_RESP_INVALID; - - if (!(prm->flags & IOMMU_FAULT_PAGE_REQUEST_PASID_VALID)) - return status; - - mm =3D iommu_sva_find(prm->pasid); - if (IS_ERR_OR_NULL(mm)) - return status; - - mmap_read_lock(mm); - - vma =3D find_extend_vma(mm, prm->addr); - if (!vma) - /* Unmapped area */ - goto out_put_mm; - - if (prm->perm & IOMMU_FAULT_PERM_READ) - access_flags |=3D VM_READ; - - if (prm->perm & IOMMU_FAULT_PERM_WRITE) { - access_flags |=3D VM_WRITE; - fault_flags |=3D FAULT_FLAG_WRITE; - } - - if (prm->perm & IOMMU_FAULT_PERM_EXEC) { - access_flags |=3D VM_EXEC; - fault_flags |=3D FAULT_FLAG_INSTRUCTION; - } - - if (!(prm->perm & IOMMU_FAULT_PERM_PRIV)) - fault_flags |=3D FAULT_FLAG_USER; - - if (access_flags & ~vma->vm_flags) - /* Access fault */ - goto out_put_mm; - - ret =3D handle_mm_fault(vma, prm->addr, fault_flags, NULL); - status =3D ret & VM_FAULT_ERROR ? IOMMU_PAGE_RESP_INVALID : - IOMMU_PAGE_RESP_SUCCESS; - -out_put_mm: - mmap_read_unlock(mm); - mmput(mm); - - return status; -} - static void iopf_handle_group(struct work_struct *work) { struct iopf_group *group; + struct iommu_domain *domain; struct iopf_fault *iopf, *next; enum iommu_page_response_code status =3D IOMMU_PAGE_RESP_SUCCESS; =20 group =3D container_of(work, struct iopf_group, work); + domain =3D iommu_get_domain_for_dev_pasid(group->dev, + group->last_fault.fault.prm.pasid); + if (!domain || !domain->iopf_handler) + status =3D IOMMU_PAGE_RESP_INVALID; =20 list_for_each_entry_safe(iopf, next, &group->faults, list) { /* @@ -139,7 +88,8 @@ static void iopf_handle_group(struct work_struct *work) * faults in the group if there is an error. */ if (status =3D=3D IOMMU_PAGE_RESP_SUCCESS) - status =3D iopf_handle_single(iopf); + status =3D domain->iopf_handler(&iopf->fault, + domain->fault_data); =20 if (!(iopf->fault.prm.flags & IOMMU_FAULT_PAGE_REQUEST_LAST_PAGE)) --=20 2.25.1 From nobody Thu May 7 20:22:18 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 260C6C433EF for ; Thu, 19 May 2022 07:25:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234863AbiESHZx (ORCPT ); Thu, 19 May 2022 03:25:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42572 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234693AbiESHY7 (ORCPT ); Thu, 19 May 2022 03:24:59 -0400 Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 85A018020E for ; Thu, 19 May 2022 00:24:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1652945098; x=1684481098; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=3VCJyDHv+RURhurVF6s727cvipO7vTDi2523jaNX4yg=; b=asAfGBZsQXUgRe5neT5rUS8EKF+2cdf0TbMicqcW61hc/VMhxRsK9Aq8 +JPkewIuDkaC3wyiZg9x0y1xvEiPuom+dLsAuaPgWk06uGgZFie/euhjN eedRYrQWRhTa4Nirb5f8ylykCmFdQNRUe5K5+NHtb5p2sfRNl3v1kNdCn aFtH+9SG5J/M36VnzWHlRmLTqKzlV06xooFhYaAJ9yk0u4isEhmusbSMd +fJdjA9KokVZtekGCjJjCEnQVqWDnqU/skry4Wv5IqQX+2em9KfCXSxQd xgCz5xRXDlzaoUOXF/Zx47qx6X/rDn03If8B6hZ5pf1hekQ+g+JI0+Ugm g==; X-IronPort-AV: E=McAfee;i="6400,9594,10351"; a="272196027" X-IronPort-AV: E=Sophos;i="5.91,236,1647327600"; d="scan'208";a="272196027" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 May 2022 00:24:58 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.91,236,1647327600"; d="scan'208";a="714853181" Received: from allen-box.sh.intel.com ([10.239.159.48]) by fmsmga001.fm.intel.com with ESMTP; 19 May 2022 00:24:54 -0700 From: Lu Baolu To: Joerg Roedel , Jason Gunthorpe , Christoph Hellwig , Kevin Tian , Ashok Raj , Will Deacon , Robin Murphy , Jean-Philippe Brucker , Dave Jiang , Vinod Koul Cc: Eric Auger , Liu Yi L , Jacob jun Pan , iommu@lists.linux-foundation.org, linux-kernel@vger.kernel.org, Lu Baolu , Jean-Philippe Brucker Subject: [PATCH v7 10/10] iommu: Rename iommu-sva-lib.{c,h} Date: Thu, 19 May 2022 15:20:47 +0800 Message-Id: <20220519072047.2996983-11-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220519072047.2996983-1-baolu.lu@linux.intel.com> References: <20220519072047.2996983-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Rename iommu-sva-lib.c[h] to iommu-sva.c[h] as it contains all code for SVA implementation in iommu core. Signed-off-by: Lu Baolu Reviewed-by: Jean-Philippe Brucker --- drivers/iommu/{iommu-sva-lib.h =3D> iommu-sva.h} | 0 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 2 +- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 2 +- drivers/iommu/intel/iommu.c | 2 +- drivers/iommu/intel/svm.c | 2 +- drivers/iommu/io-pgfault.c | 2 +- drivers/iommu/{iommu-sva-lib.c =3D> iommu-sva.c} | 2 +- drivers/iommu/Makefile | 2 +- 8 files changed, 7 insertions(+), 7 deletions(-) rename drivers/iommu/{iommu-sva-lib.h =3D> iommu-sva.h} (100%) rename drivers/iommu/{iommu-sva-lib.c =3D> iommu-sva.c} (99%) diff --git a/drivers/iommu/iommu-sva-lib.h b/drivers/iommu/iommu-sva.h similarity index 100% rename from drivers/iommu/iommu-sva-lib.h rename to drivers/iommu/iommu-sva.h diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iomm= u/arm/arm-smmu-v3/arm-smmu-v3-sva.c index 8290d66569f3..b238b09c4999 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -10,7 +10,7 @@ #include =20 #include "arm-smmu-v3.h" -#include "../../iommu-sva-lib.h" +#include "../../iommu-sva.h" #include "../../io-pgtable-arm.h" =20 struct arm_smmu_mmu_notifier { diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/ar= m/arm-smmu-v3/arm-smmu-v3.c index b74f8964cc13..e1b80b7ac858 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -31,7 +31,7 @@ #include =20 #include "arm-smmu-v3.h" -#include "../../iommu-sva-lib.h" +#include "../../iommu-sva.h" =20 static bool disable_bypass =3D true; module_param(disable_bypass, bool, 0444); diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c index 10e07d59d4c8..2289939725a7 100644 --- a/drivers/iommu/intel/iommu.c +++ b/drivers/iommu/intel/iommu.c @@ -27,7 +27,7 @@ #include =20 #include "../irq_remapping.h" -#include "../iommu-sva-lib.h" +#include "../iommu-sva.h" #include "pasid.h" #include "cap_audit.h" =20 diff --git a/drivers/iommu/intel/svm.c b/drivers/iommu/intel/svm.c index e412a442d9a4..2c45579e5e81 100644 --- a/drivers/iommu/intel/svm.c +++ b/drivers/iommu/intel/svm.c @@ -25,7 +25,7 @@ =20 #include "pasid.h" #include "perf.h" -#include "../iommu-sva-lib.h" +#include "../iommu-sva.h" =20 static irqreturn_t prq_event_thread(int irq, void *d); static void intel_svm_drain_prq(struct device *dev, u32 pasid); diff --git a/drivers/iommu/io-pgfault.c b/drivers/iommu/io-pgfault.c index 4f24ec703479..91b1c6bd01d4 100644 --- a/drivers/iommu/io-pgfault.c +++ b/drivers/iommu/io-pgfault.c @@ -11,7 +11,7 @@ #include #include =20 -#include "iommu-sva-lib.h" +#include "iommu-sva.h" =20 /** * struct iopf_queue - IO Page Fault queue diff --git a/drivers/iommu/iommu-sva-lib.c b/drivers/iommu/iommu-sva.c similarity index 99% rename from drivers/iommu/iommu-sva-lib.c rename to drivers/iommu/iommu-sva.c index 317ab8e8c149..0e4f7c889f54 100644 --- a/drivers/iommu/iommu-sva-lib.c +++ b/drivers/iommu/iommu-sva.c @@ -7,7 +7,7 @@ #include #include =20 -#include "iommu-sva-lib.h" +#include "iommu-sva.h" =20 static DEFINE_MUTEX(iommu_sva_lock); static DECLARE_IOASID_SET(iommu_sva_pasid); diff --git a/drivers/iommu/Makefile b/drivers/iommu/Makefile index 44475a9b3eea..c1763476162b 100644 --- a/drivers/iommu/Makefile +++ b/drivers/iommu/Makefile @@ -27,6 +27,6 @@ obj-$(CONFIG_FSL_PAMU) +=3D fsl_pamu.o fsl_pamu_domain.o obj-$(CONFIG_S390_IOMMU) +=3D s390-iommu.o obj-$(CONFIG_HYPERV_IOMMU) +=3D hyperv-iommu.o obj-$(CONFIG_VIRTIO_IOMMU) +=3D virtio-iommu.o -obj-$(CONFIG_IOMMU_SVA) +=3D iommu-sva-lib.o io-pgfault.o +obj-$(CONFIG_IOMMU_SVA) +=3D iommu-sva.o io-pgfault.o obj-$(CONFIG_SPRD_IOMMU) +=3D sprd-iommu.o obj-$(CONFIG_APPLE_DART) +=3D apple-dart.o --=20 2.25.1