From nobody Thu Apr 2 11:39:36 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8CA96C38A02 for ; Mon, 31 Oct 2022 01:06:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229686AbiJaBGG (ORCPT ); Sun, 30 Oct 2022 21:06:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36332 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229549AbiJaBGE (ORCPT ); Sun, 30 Oct 2022 21:06:04 -0400 Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 97B7D9FDE for ; Sun, 30 Oct 2022 18:06:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1667178363; x=1698714363; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=zwbxaON8XYHMDEYGmgYuDpqK1SsbsgRTeecwhgmM5zs=; b=e5HyvekrGdv+kgq/Yr9xZ009rsVwK2I8vQ0+3CAyDizI2pU+m5qTWVIp 8gX4KTRJ8Ch4uHKgH4cm9Pw8hY8Z2qHBQqDzCQibn4ffqlMpgHAsj9NQm IGMoiIbxfyNt72qxefFyatbjn05up1XW723kacT1a9OXx5G+FG2QdwGz0 szKZkzjXngrh1IKXh+XRhbCpTQ18y2hO954CkLHxlYYjj0FjWoqyN+gIA Gh4hbuTIU8PpfDvWuXotik3Q8M18z9zapdlIba8VHX+Z9EvJERdxVk8TK /fL/FOKgpHDml4UDgxKRuBMMhZ4IvvL+C15QVD5RioCGQBdEG9soCOQJF Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10516"; a="373001657" X-IronPort-AV: E=Sophos;i="5.95,227,1661842800"; d="scan'208";a="373001657" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Oct 2022 18:06:03 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10516"; a="584496121" X-IronPort-AV: E=Sophos;i="5.95,227,1661842800"; d="scan'208";a="584496121" Received: from allen-box.sh.intel.com ([10.239.159.48]) by orsmga003.jf.intel.com with ESMTP; 30 Oct 2022 18:05:58 -0700 From: Lu Baolu To: Joerg Roedel Cc: Jason Gunthorpe , Kevin Tian , Will Deacon , Robin Murphy , Jean-Philippe Brucker , iommu@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [RESEND PATCH v14 01/13] iommu: Add max_pasids field in struct iommu_device Date: Mon, 31 Oct 2022 08:59:05 +0800 Message-Id: <20221031005917.45690-2-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221031005917.45690-1-baolu.lu@linux.intel.com> References: <20221031005917.45690-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Use this field to keep the number of supported PASIDs that an IOMMU hardware is able to support. This is a generic attribute of an IOMMU and lifting it into the per-IOMMU device structure makes it possible to allocate a PASID for device without calls into the IOMMU drivers. Any iommu driver that supports PASID related features should set this field before enabling them on the devices. In the Intel IOMMU driver, intel_iommu_sm is moved to CONFIG_INTEL_IOMMU enclave so that the pasid_supported() helper could be used in dmar.c without compilation errors. Signed-off-by: Lu Baolu Reviewed-by: Jean-Philippe Brucker Reviewed-by: Kevin Tian Reviewed-by: Jason Gunthorpe Reviewed-by: Yi Liu Tested-by: Zhangfei Gao Tested-by: Tony Zhu --- include/linux/iommu.h | 2 ++ drivers/iommu/intel/iommu.h | 4 ++-- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 1 + drivers/iommu/intel/dmar.c | 7 +++++++ 4 files changed, 12 insertions(+), 2 deletions(-) diff --git a/include/linux/iommu.h b/include/linux/iommu.h index 3c9da1f8979e..e3af4f46e6e0 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -322,12 +322,14 @@ struct iommu_domain_ops { * @list: Used by the iommu-core to keep a list of registered iommus * @ops: iommu-ops for talking to this iommu * @dev: struct device for sysfs handling + * @max_pasids: number of supported PASIDs */ struct iommu_device { struct list_head list; const struct iommu_ops *ops; struct fwnode_handle *fwnode; struct device *dev; + u32 max_pasids; }; =20 /** diff --git a/drivers/iommu/intel/iommu.h b/drivers/iommu/intel/iommu.h index 92023dff9513..cce0598f4109 100644 --- a/drivers/iommu/intel/iommu.h +++ b/drivers/iommu/intel/iommu.h @@ -480,8 +480,6 @@ enum { #define VTD_FLAG_IRQ_REMAP_PRE_ENABLED (1 << 1) #define VTD_FLAG_SVM_CAPABLE (1 << 2) =20 -extern int intel_iommu_sm; - #define sm_supported(iommu) (intel_iommu_sm && ecap_smts((iommu)->ecap)) #define pasid_supported(iommu) (sm_supported(iommu) && \ ecap_pasid((iommu)->ecap)) @@ -795,6 +793,7 @@ struct context_entry *iommu_context_addr(struct intel_i= ommu *iommu, u8 bus, extern const struct iommu_ops intel_iommu_ops; =20 #ifdef CONFIG_INTEL_IOMMU +extern int intel_iommu_sm; extern int iommu_calculate_agaw(struct intel_iommu *iommu); extern int iommu_calculate_max_sagaw(struct intel_iommu *iommu); extern int dmar_disabled; @@ -810,6 +809,7 @@ static inline int iommu_calculate_max_sagaw(struct inte= l_iommu *iommu) } #define dmar_disabled (1) #define intel_iommu_enabled (0) +#define intel_iommu_sm (0) #endif =20 static inline const char *decode_prq_descriptor(char *str, size_t size, diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/ar= m/arm-smmu-v3/arm-smmu-v3.c index 6d5df91c5c46..21cb13da122c 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -3543,6 +3543,7 @@ static int arm_smmu_device_hw_probe(struct arm_smmu_d= evice *smmu) /* SID/SSID sizes */ smmu->ssid_bits =3D FIELD_GET(IDR1_SSIDSIZE, reg); smmu->sid_bits =3D FIELD_GET(IDR1_SIDSIZE, reg); + smmu->iommu.max_pasids =3D 1UL << smmu->ssid_bits; =20 /* * If the SMMU supports fewer bits than would fill a single L2 stream diff --git a/drivers/iommu/intel/dmar.c b/drivers/iommu/intel/dmar.c index 5a8f780e7ffd..3528058d253e 100644 --- a/drivers/iommu/intel/dmar.c +++ b/drivers/iommu/intel/dmar.c @@ -1104,6 +1104,13 @@ static int alloc_iommu(struct dmar_drhd_unit *drhd) =20 raw_spin_lock_init(&iommu->register_lock); =20 + /* + * A value of N in PSS field of eCap register indicates hardware + * supports PASID field of N+1 bits. + */ + if (pasid_supported(iommu)) + iommu->iommu.max_pasids =3D 2UL << ecap_pss(iommu->ecap); + /* * This is only for hotplug; at boot time intel_iommu_enabled won't * be set yet. When intel_iommu_init() runs, it registers the units --=20 2.34.1 From nobody Thu Apr 2 11:39:36 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 49869FA3742 for ; Mon, 31 Oct 2022 01:06:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229721AbiJaBGI (ORCPT ); Sun, 30 Oct 2022 21:06:08 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36338 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229637AbiJaBGF (ORCPT ); Sun, 30 Oct 2022 21:06:05 -0400 Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 41DED9FF2 for ; Sun, 30 Oct 2022 18:06:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1667178364; x=1698714364; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=uEGja7y75UUPzrjK6H3IN4daa8pbnGT3JK5PG289ku4=; b=PjN/ZZek0Eiw7ZTPC7V1wgLIElfvP44tkfDNJ2Ha+CYitT2KwqaSp/1D gIOY8218JtufLNIOeTVVH9wp/Yn4rPTS0dMZkTQ4XTj3rIOyOWQzWO3RS yAnEGgzWXwvsaCwc2fRmnV1afKnSjk95O6HWhSuVC1J4nJeFvxFgZObCj tIilq0vKWHglGnU06wKvbOiUV/Q3whNH80nmKn/s9/fQSMnQvWkSYLRRT VCjgNfUT6U6EeFrKA40oB9ZNIMk0vHJ2NJzvGH4lKorkySPBaBFQ9SONB eiWEEiQaEnZtzvTJwP21ZwHj3PDCjJNEToT5BMDgTx2zYzTis9WBJLZSs Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10516"; a="373001659" X-IronPort-AV: E=Sophos;i="5.95,227,1661842800"; d="scan'208";a="373001659" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Oct 2022 18:06:03 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10516"; a="584496136" X-IronPort-AV: E=Sophos;i="5.95,227,1661842800"; d="scan'208";a="584496136" Received: from allen-box.sh.intel.com ([10.239.159.48]) by orsmga003.jf.intel.com with ESMTP; 30 Oct 2022 18:06:01 -0700 From: Lu Baolu To: Joerg Roedel Cc: Jason Gunthorpe , Kevin Tian , Will Deacon , Robin Murphy , Jean-Philippe Brucker , iommu@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [RESEND PATCH v14 02/13] iommu: Add max_pasids field in struct dev_iommu Date: Mon, 31 Oct 2022 08:59:06 +0800 Message-Id: <20221031005917.45690-3-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221031005917.45690-1-baolu.lu@linux.intel.com> References: <20221031005917.45690-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Use this field to save the number of PASIDs that a device is able to consume. It is a generic attribute of a device and lifting it into the per-device dev_iommu struct could help to avoid the boilerplate code in various IOMMU drivers. Signed-off-by: Lu Baolu Reviewed-by: Kevin Tian Reviewed-by: Jason Gunthorpe Reviewed-by: Yi Liu Tested-by: Zhangfei Gao Tested-by: Tony Zhu --- include/linux/iommu.h | 2 ++ drivers/iommu/iommu.c | 20 ++++++++++++++++++++ 2 files changed, 22 insertions(+) diff --git a/include/linux/iommu.h b/include/linux/iommu.h index e3af4f46e6e0..ac3f6c6dcc6d 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -368,6 +368,7 @@ struct iommu_fault_param { * @fwspec: IOMMU fwspec data * @iommu_dev: IOMMU device this device is linked to * @priv: IOMMU Driver private data + * @max_pasids: number of PASIDs this device can consume * * TODO: migrate other per device data pointers under iommu_dev_data, e.g. * struct iommu_group *iommu_group; @@ -379,6 +380,7 @@ struct dev_iommu { struct iommu_fwspec *fwspec; struct iommu_device *iommu_dev; void *priv; + u32 max_pasids; }; =20 int iommu_device_register(struct iommu_device *iommu, diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c index 65a3b3d886dc..297ac79bc21c 100644 --- a/drivers/iommu/iommu.c +++ b/drivers/iommu/iommu.c @@ -21,6 +21,7 @@ #include #include #include +#include #include #include #include @@ -278,6 +279,24 @@ static void dev_iommu_free(struct device *dev) kfree(param); } =20 +static u32 dev_iommu_get_max_pasids(struct device *dev) +{ + u32 max_pasids =3D 0, bits =3D 0; + int ret; + + if (dev_is_pci(dev)) { + ret =3D pci_max_pasids(to_pci_dev(dev)); + if (ret > 0) + max_pasids =3D ret; + } else { + ret =3D device_property_read_u32(dev, "pasid-num-bits", &bits); + if (!ret) + max_pasids =3D 1UL << bits; + } + + return min_t(u32, max_pasids, dev->iommu->iommu_dev->max_pasids); +} + static int __iommu_probe_device(struct device *dev, struct list_head *grou= p_list) { const struct iommu_ops *ops =3D dev->bus->iommu_ops; @@ -303,6 +322,7 @@ static int __iommu_probe_device(struct device *dev, str= uct list_head *group_list } =20 dev->iommu->iommu_dev =3D iommu_dev; + dev->iommu->max_pasids =3D dev_iommu_get_max_pasids(dev); =20 group =3D iommu_group_get_for_dev(dev); if (IS_ERR(group)) { --=20 2.34.1 From nobody Thu Apr 2 11:39:36 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 34173FA373D for ; Mon, 31 Oct 2022 01:06:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229776AbiJaBGS (ORCPT ); Sun, 30 Oct 2022 21:06:18 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36354 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229696AbiJaBGI (ORCPT ); Sun, 30 Oct 2022 21:06:08 -0400 Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D40409FDC for ; Sun, 30 Oct 2022 18:06:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1667178366; x=1698714366; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=yUhy7+fHq9NwxbO3ARISSF+PormUfUcThyIRGtESztE=; b=B/7tEksZVtdc+9yRcBrkC8MvTUba7nctZy+8wTkDgKWnCuqim47nrIgq 5ZNeCvAIldzOgV0Z+dJrzuXVAA1VywtsJ8yljIWDLyxoTm33AMELpGWia /TBDppXJ97gDrPBUljDFYXQy4wBPb38sMpDudpSl4sDoo2vz7p3/St7L9 n2ulMlDrwcI74x8A3FDRqSrkA2HstNobzuejHYD6m5h7wWHXwXujkmPbh DFk2aImoUzJtWDdx/EnxrFerRoQuTdCX4wn9h4ZwSwRfkB1ePbbAG2s9G edGt4SKBirj5uwRB/nUyCkS/FTPIYsPG1uvkxHsy1xcPTRn43hvPLrQlj g==; X-IronPort-AV: E=McAfee;i="6500,9779,10516"; a="373001663" X-IronPort-AV: E=Sophos;i="5.95,227,1661842800"; d="scan'208";a="373001663" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Oct 2022 18:06:06 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10516"; a="584496150" X-IronPort-AV: E=Sophos;i="5.95,227,1661842800"; d="scan'208";a="584496150" Received: from allen-box.sh.intel.com ([10.239.159.48]) by orsmga003.jf.intel.com with ESMTP; 30 Oct 2022 18:06:04 -0700 From: Lu Baolu To: Joerg Roedel Cc: Jason Gunthorpe , Kevin Tian , Will Deacon , Robin Murphy , Jean-Philippe Brucker , iommu@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [RESEND PATCH v14 03/13] iommu: Remove SVM_FLAG_SUPERVISOR_MODE support Date: Mon, 31 Oct 2022 08:59:07 +0800 Message-Id: <20221031005917.45690-4-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221031005917.45690-1-baolu.lu@linux.intel.com> References: <20221031005917.45690-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org The current kernel DMA with PASID support is based on the SVA with a flag SVM_FLAG_SUPERVISOR_MODE. The IOMMU driver binds the kernel memory address space to a PASID of the device. The device driver programs the device with kernel virtual address (KVA) for DMA access. There have been security and functional issues with this approach: - The lack of IOTLB synchronization upon kernel page table updates. (vmalloc, module/BPF loading, CONFIG_DEBUG_PAGEALLOC etc.) - Other than slight more protection, using kernel virtual address (KVA) has little advantage over physical address. There are also no use cases yet where DMA engines need kernel virtual addresses for in-kernel DMA. This removes SVM_FLAG_SUPERVISOR_MODE support from the IOMMU interface. The device drivers are suggested to handle kernel DMA with PASID through the kernel DMA APIs. The drvdata parameter in iommu_sva_bind_device() and all callbacks is not needed anymore. Cleanup them as well. Link: https://lore.kernel.org/linux-iommu/20210511194726.GP1002214@nvidia.c= om/ Signed-off-by: Jacob Pan Signed-off-by: Lu Baolu Reviewed-by: Jason Gunthorpe Reviewed-by: Jean-Philippe Brucker Reviewed-by: Kevin Tian Reviewed-by: Fenghua Yu Tested-by: Zhangfei Gao Tested-by: Tony Zhu --- include/linux/intel-svm.h | 13 ----- include/linux/iommu.h | 8 +-- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 5 +- drivers/iommu/intel/iommu.h | 3 +- drivers/dma/idxd/cdev.c | 3 +- drivers/dma/idxd/init.c | 25 +-------- .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 3 +- drivers/iommu/intel/svm.c | 55 +++++-------------- drivers/iommu/iommu.c | 5 +- drivers/misc/uacce/uacce.c | 2 +- 10 files changed, 25 insertions(+), 97 deletions(-) diff --git a/include/linux/intel-svm.h b/include/linux/intel-svm.h index 207ef06ba3e1..f9a0d44f6fdb 100644 --- a/include/linux/intel-svm.h +++ b/include/linux/intel-svm.h @@ -13,17 +13,4 @@ #define PRQ_RING_MASK ((0x1000 << PRQ_ORDER) - 0x20) #define PRQ_DEPTH ((0x1000 << PRQ_ORDER) >> 5) =20 -/* - * The SVM_FLAG_SUPERVISOR_MODE flag requests a PASID which can be used on= ly - * for access to kernel addresses. No IOTLB flushes are automatically done - * for kernel mappings; it is valid only for access to the kernel's static - * 1:1 mapping of physical memory =E2=80=94 not to vmalloc or even module = mappings. - * A future API addition may permit the use of such ranges, by means of an - * explicit IOTLB flush call (akin to the DMA API's unmap method). - * - * It is unlikely that we will ever hook into flush_tlb_kernel_range() to - * do such IOTLB flushes automatically. - */ -#define SVM_FLAG_SUPERVISOR_MODE BIT(0) - #endif /* __INTEL_SVM_H__ */ diff --git a/include/linux/iommu.h b/include/linux/iommu.h index ac3f6c6dcc6d..72bb0531aa76 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -247,8 +247,7 @@ struct iommu_ops { int (*dev_enable_feat)(struct device *dev, enum iommu_dev_features f); int (*dev_disable_feat)(struct device *dev, enum iommu_dev_features f); =20 - struct iommu_sva *(*sva_bind)(struct device *dev, struct mm_struct *mm, - void *drvdata); + struct iommu_sva *(*sva_bind)(struct device *dev, struct mm_struct *mm); void (*sva_unbind)(struct iommu_sva *handle); u32 (*sva_get_pasid)(struct iommu_sva *handle); =20 @@ -668,8 +667,7 @@ int iommu_dev_enable_feature(struct device *dev, enum i= ommu_dev_features f); int iommu_dev_disable_feature(struct device *dev, enum iommu_dev_features = f); =20 struct iommu_sva *iommu_sva_bind_device(struct device *dev, - struct mm_struct *mm, - void *drvdata); + struct mm_struct *mm); void iommu_sva_unbind_device(struct iommu_sva *handle); u32 iommu_sva_get_pasid(struct iommu_sva *handle); =20 @@ -1000,7 +998,7 @@ iommu_dev_disable_feature(struct device *dev, enum iom= mu_dev_features feat) } =20 static inline struct iommu_sva * -iommu_sva_bind_device(struct device *dev, struct mm_struct *mm, void *drvd= ata) +iommu_sva_bind_device(struct device *dev, struct mm_struct *mm) { return NULL; } diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/ar= m/arm-smmu-v3/arm-smmu-v3.h index cd48590ada30..d2ba86470c42 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -754,8 +754,7 @@ bool arm_smmu_master_sva_enabled(struct arm_smmu_master= *master); int arm_smmu_master_enable_sva(struct arm_smmu_master *master); int arm_smmu_master_disable_sva(struct arm_smmu_master *master); bool arm_smmu_master_iopf_supported(struct arm_smmu_master *master); -struct iommu_sva *arm_smmu_sva_bind(struct device *dev, struct mm_struct *= mm, - void *drvdata); +struct iommu_sva *arm_smmu_sva_bind(struct device *dev, struct mm_struct *= mm); void arm_smmu_sva_unbind(struct iommu_sva *handle); u32 arm_smmu_sva_get_pasid(struct iommu_sva *handle); void arm_smmu_sva_notifier_synchronize(void); @@ -791,7 +790,7 @@ static inline bool arm_smmu_master_iopf_supported(struc= t arm_smmu_master *master } =20 static inline struct iommu_sva * -arm_smmu_sva_bind(struct device *dev, struct mm_struct *mm, void *drvdata) +arm_smmu_sva_bind(struct device *dev, struct mm_struct *mm) { return ERR_PTR(-ENODEV); } diff --git a/drivers/iommu/intel/iommu.h b/drivers/iommu/intel/iommu.h index cce0598f4109..33e5bcaf2a6c 100644 --- a/drivers/iommu/intel/iommu.h +++ b/drivers/iommu/intel/iommu.h @@ -748,8 +748,7 @@ struct intel_iommu *device_to_iommu(struct device *dev,= u8 *bus, u8 *devfn); extern void intel_svm_check(struct intel_iommu *iommu); extern int intel_svm_enable_prq(struct intel_iommu *iommu); extern int intel_svm_finish_prq(struct intel_iommu *iommu); -struct iommu_sva *intel_svm_bind(struct device *dev, struct mm_struct *mm, - void *drvdata); +struct iommu_sva *intel_svm_bind(struct device *dev, struct mm_struct *mm); void intel_svm_unbind(struct iommu_sva *handle); u32 intel_svm_get_pasid(struct iommu_sva *handle); int intel_svm_page_response(struct device *dev, struct iommu_fault_event *= evt, diff --git a/drivers/dma/idxd/cdev.c b/drivers/dma/idxd/cdev.c index c2808fd081d6..66720001ba1c 100644 --- a/drivers/dma/idxd/cdev.c +++ b/drivers/dma/idxd/cdev.c @@ -6,7 +6,6 @@ #include #include #include -#include #include #include #include @@ -100,7 +99,7 @@ static int idxd_cdev_open(struct inode *inode, struct fi= le *filp) filp->private_data =3D ctx; =20 if (device_user_pasid_enabled(idxd)) { - sva =3D iommu_sva_bind_device(dev, current->mm, NULL); + sva =3D iommu_sva_bind_device(dev, current->mm); if (IS_ERR(sva)) { rc =3D PTR_ERR(sva); dev_err(dev, "pasid allocation failed: %d\n", rc); diff --git a/drivers/dma/idxd/init.c b/drivers/dma/idxd/init.c index 2b18d512cbfc..2c0fcfdc75c7 100644 --- a/drivers/dma/idxd/init.c +++ b/drivers/dma/idxd/init.c @@ -14,7 +14,6 @@ #include #include #include -#include #include #include #include @@ -502,29 +501,7 @@ static struct idxd_device *idxd_alloc(struct pci_dev *= pdev, struct idxd_driver_d =20 static int idxd_enable_system_pasid(struct idxd_device *idxd) { - int flags; - unsigned int pasid; - struct iommu_sva *sva; - - flags =3D SVM_FLAG_SUPERVISOR_MODE; - - sva =3D iommu_sva_bind_device(&idxd->pdev->dev, NULL, &flags); - if (IS_ERR(sva)) { - dev_warn(&idxd->pdev->dev, - "iommu sva bind failed: %ld\n", PTR_ERR(sva)); - return PTR_ERR(sva); - } - - pasid =3D iommu_sva_get_pasid(sva); - if (pasid =3D=3D IOMMU_PASID_INVALID) { - iommu_sva_unbind_device(sva); - return -ENODEV; - } - - idxd->sva =3D sva; - idxd->pasid =3D pasid; - dev_dbg(&idxd->pdev->dev, "system pasid: %u\n", pasid); - return 0; + return -EOPNOTSUPP; } =20 static void idxd_disable_system_pasid(struct idxd_device *idxd) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iomm= u/arm/arm-smmu-v3/arm-smmu-v3-sva.c index 5968a568aae2..8fcf0df4bd0e 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -367,8 +367,7 @@ __arm_smmu_sva_bind(struct device *dev, struct mm_struc= t *mm) return ERR_PTR(ret); } =20 -struct iommu_sva * -arm_smmu_sva_bind(struct device *dev, struct mm_struct *mm, void *drvdata) +struct iommu_sva *arm_smmu_sva_bind(struct device *dev, struct mm_struct *= mm) { struct iommu_sva *handle; struct iommu_domain *domain =3D iommu_get_domain_for_dev(dev); diff --git a/drivers/iommu/intel/svm.c b/drivers/iommu/intel/svm.c index 7d08eb034f2d..94bc47b68c93 100644 --- a/drivers/iommu/intel/svm.c +++ b/drivers/iommu/intel/svm.c @@ -296,8 +296,7 @@ static int pasid_to_svm_sdev(struct device *dev, unsign= ed int pasid, return 0; } =20 -static int intel_svm_alloc_pasid(struct device *dev, struct mm_struct *mm, - unsigned int flags) +static int intel_svm_alloc_pasid(struct device *dev, struct mm_struct *mm) { ioasid_t max_pasid =3D dev_is_pci(dev) ? pci_max_pasids(to_pci_dev(dev)) : intel_pasid_max_id; @@ -307,8 +306,7 @@ static int intel_svm_alloc_pasid(struct device *dev, st= ruct mm_struct *mm, =20 static struct iommu_sva *intel_svm_bind_mm(struct intel_iommu *iommu, struct device *dev, - struct mm_struct *mm, - unsigned int flags) + struct mm_struct *mm) { struct device_domain_info *info =3D dev_iommu_priv_get(dev); struct intel_svm_dev *sdev; @@ -324,22 +322,18 @@ static struct iommu_sva *intel_svm_bind_mm(struct int= el_iommu *iommu, =20 svm->pasid =3D mm->pasid; svm->mm =3D mm; - svm->flags =3D flags; INIT_LIST_HEAD_RCU(&svm->devs); =20 - if (!(flags & SVM_FLAG_SUPERVISOR_MODE)) { - svm->notifier.ops =3D &intel_mmuops; - ret =3D mmu_notifier_register(&svm->notifier, mm); - if (ret) { - kfree(svm); - return ERR_PTR(ret); - } + svm->notifier.ops =3D &intel_mmuops; + ret =3D mmu_notifier_register(&svm->notifier, mm); + if (ret) { + kfree(svm); + return ERR_PTR(ret); } =20 ret =3D pasid_private_add(svm->pasid, svm); if (ret) { - if (svm->notifier.ops) - mmu_notifier_unregister(&svm->notifier, mm); + mmu_notifier_unregister(&svm->notifier, mm); kfree(svm); return ERR_PTR(ret); } @@ -374,9 +368,7 @@ static struct iommu_sva *intel_svm_bind_mm(struct intel= _iommu *iommu, } =20 /* Setup the pasid table: */ - sflags =3D (flags & SVM_FLAG_SUPERVISOR_MODE) ? - PASID_FLAG_SUPERVISOR_MODE : 0; - sflags |=3D cpu_feature_enabled(X86_FEATURE_LA57) ? PASID_FLAG_FL5LP : 0; + sflags =3D cpu_feature_enabled(X86_FEATURE_LA57) ? PASID_FLAG_FL5LP : 0; ret =3D intel_pasid_setup_first_level(iommu, dev, mm->pgd, mm->pasid, FLPT_DEFAULT_DID, sflags); if (ret) @@ -390,8 +382,7 @@ static struct iommu_sva *intel_svm_bind_mm(struct intel= _iommu *iommu, kfree(sdev); free_svm: if (list_empty(&svm->devs)) { - if (svm->notifier.ops) - mmu_notifier_unregister(&svm->notifier, mm); + mmu_notifier_unregister(&svm->notifier, mm); pasid_private_remove(mm->pasid); kfree(svm); } @@ -780,40 +771,20 @@ static irqreturn_t prq_event_thread(int irq, void *d) return IRQ_RETVAL(handled); } =20 -struct iommu_sva *intel_svm_bind(struct device *dev, struct mm_struct *mm,= void *drvdata) +struct iommu_sva *intel_svm_bind(struct device *dev, struct mm_struct *mm) { struct intel_iommu *iommu =3D device_to_iommu(dev, NULL, NULL); - unsigned int flags =3D 0; struct iommu_sva *sva; int ret; =20 - if (drvdata) - flags =3D *(unsigned int *)drvdata; - - if (flags & SVM_FLAG_SUPERVISOR_MODE) { - if (!ecap_srs(iommu->ecap)) { - dev_err(dev, "%s: Supervisor PASID not supported\n", - iommu->name); - return ERR_PTR(-EOPNOTSUPP); - } - - if (mm) { - dev_err(dev, "%s: Supervisor PASID with user provided mm\n", - iommu->name); - return ERR_PTR(-EINVAL); - } - - mm =3D &init_mm; - } - mutex_lock(&pasid_mutex); - ret =3D intel_svm_alloc_pasid(dev, mm, flags); + ret =3D intel_svm_alloc_pasid(dev, mm); if (ret) { mutex_unlock(&pasid_mutex); return ERR_PTR(ret); } =20 - sva =3D intel_svm_bind_mm(iommu, dev, mm, flags); + sva =3D intel_svm_bind_mm(iommu, dev, mm); mutex_unlock(&pasid_mutex); =20 return sva; diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c index 297ac79bc21c..a94ec648c88b 100644 --- a/drivers/iommu/iommu.c +++ b/drivers/iommu/iommu.c @@ -2750,7 +2750,6 @@ EXPORT_SYMBOL_GPL(iommu_dev_disable_feature); * iommu_sva_bind_device() - Bind a process address space to a device * @dev: the device * @mm: the mm to bind, caller must hold a reference to it - * @drvdata: opaque data pointer to pass to bind callback * * Create a bond between device and address space, allowing the device to = access * the mm using the returned PASID. If a bond already exists between @devi= ce and @@ -2763,7 +2762,7 @@ EXPORT_SYMBOL_GPL(iommu_dev_disable_feature); * On error, returns an ERR_PTR value. */ struct iommu_sva * -iommu_sva_bind_device(struct device *dev, struct mm_struct *mm, void *drvd= ata) +iommu_sva_bind_device(struct device *dev, struct mm_struct *mm) { struct iommu_group *group; struct iommu_sva *handle =3D ERR_PTR(-EINVAL); @@ -2788,7 +2787,7 @@ iommu_sva_bind_device(struct device *dev, struct mm_s= truct *mm, void *drvdata) if (iommu_group_device_count(group) !=3D 1) goto out_unlock; =20 - handle =3D ops->sva_bind(dev, mm, drvdata); + handle =3D ops->sva_bind(dev, mm); =20 out_unlock: mutex_unlock(&group->mutex); diff --git a/drivers/misc/uacce/uacce.c b/drivers/misc/uacce/uacce.c index b70a013139c7..905eff1f840e 100644 --- a/drivers/misc/uacce/uacce.c +++ b/drivers/misc/uacce/uacce.c @@ -108,7 +108,7 @@ static int uacce_bind_queue(struct uacce_device *uacce,= struct uacce_queue *q) if (!(uacce->flags & UACCE_DEV_SVA)) return 0; =20 - handle =3D iommu_sva_bind_device(uacce->parent, current->mm, NULL); + handle =3D iommu_sva_bind_device(uacce->parent, current->mm); if (IS_ERR(handle)) return PTR_ERR(handle); =20 --=20 2.34.1 From nobody Thu Apr 2 11:39:36 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 215B4FA3742 for ; Mon, 31 Oct 2022 01:06:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229556AbiJaBGV (ORCPT ); Sun, 30 Oct 2022 21:06:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36566 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229728AbiJaBGP (ORCPT ); Sun, 30 Oct 2022 21:06:15 -0400 Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C25B8A1A2 for ; Sun, 30 Oct 2022 18:06:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1667178368; x=1698714368; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=EMDXCRdwqlGlnOwAqWbXNb8UnPKBXi5/6JNHQV9oHYo=; b=Rc4hZU6pbFJ/ujbgG31gzzi9SDobqf527RhzyyRZaCWOEYaRQ7PCzcs4 TJMq1VVoK5OlLckouXVGsgwbhuJAgEbHZHYByl5zWgJ2f3UbazMFIx7Ub K0RQfpuR5o3qgHQq5aq521ne84egwAMPQTqqitcy/23AWHwrl9K3VW/AI uC4F6aLdQVbZqpMFR7sb6wKdAEobefF35/PhDYdtE5N9SCT0fMAzn7DHL vWcx4R7KbiFg0jKnVlA2J9dcAdX4ju5mCnyW2S6KDAE9JUCcrhHG72nIM fXgX25uyaZDKFZLu9A9sIy1EHfoYbVQGruakZwPPQsnhLxzx9wMJRVhUF g==; X-IronPort-AV: E=McAfee;i="6500,9779,10516"; a="373001668" X-IronPort-AV: E=Sophos;i="5.95,227,1661842800"; d="scan'208";a="373001668" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Oct 2022 18:06:08 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10516"; a="584496166" X-IronPort-AV: E=Sophos;i="5.95,227,1661842800"; d="scan'208";a="584496166" Received: from allen-box.sh.intel.com ([10.239.159.48]) by orsmga003.jf.intel.com with ESMTP; 30 Oct 2022 18:06:06 -0700 From: Lu Baolu To: Joerg Roedel Cc: Jason Gunthorpe , Kevin Tian , Will Deacon , Robin Murphy , Jean-Philippe Brucker , iommu@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [RESEND PATCH v14 04/13] PCI: Enable PASID only when ACS RR & UF enabled on upstream path Date: Mon, 31 Oct 2022 08:59:08 +0800 Message-Id: <20221031005917.45690-5-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221031005917.45690-1-baolu.lu@linux.intel.com> References: <20221031005917.45690-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The Requester ID/Process Address Space ID (PASID) combination identifies an address space distinct from the PCI bus address space, e.g., an address space defined by an IOMMU. But the PCIe fabric routes Memory Requests based on the TLP address, ignoring any PASID (PCIe r6.0, sec 2.2.10.4), so a TLP with PASID that SHOULD go upstream to the IOMMU may instead be routed as a P2P Request if its address falls in a bridge window. To ensure that all Memory Requests with PASID are routed upstream, only enable PASID if ACS P2P Request Redirect and Upstream Forwarding are enabled for the path leading to the device. Suggested-by: Jason Gunthorpe Suggested-by: Kevin Tian Signed-off-by: Lu Baolu Acked-by: Bjorn Helgaas Reviewed-by: Jason Gunthorpe Tested-by: Tony Zhu --- drivers/pci/ats.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/drivers/pci/ats.c b/drivers/pci/ats.c index c967ad6e2626..f9cc2e10b676 100644 --- a/drivers/pci/ats.c +++ b/drivers/pci/ats.c @@ -382,6 +382,9 @@ int pci_enable_pasid(struct pci_dev *pdev, int features) if (!pasid) return -EINVAL; =20 + if (!pci_acs_path_enabled(pdev, NULL, PCI_ACS_RR | PCI_ACS_UF)) + return -EINVAL; + pci_read_config_word(pdev, pasid + PCI_PASID_CAP, &supported); supported &=3D PCI_PASID_CAP_EXEC | PCI_PASID_CAP_PRIV; =20 --=20 2.34.1 From nobody Thu Apr 2 11:39:36 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 12119C38A02 for ; Mon, 31 Oct 2022 01:06:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229826AbiJaBGY (ORCPT ); Sun, 30 Oct 2022 21:06:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36572 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229772AbiJaBGS (ORCPT ); Sun, 30 Oct 2022 21:06:18 -0400 Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3D4A9AE71 for ; Sun, 30 Oct 2022 18:06:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1667178371; x=1698714371; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=9KEVQgp8MCwDeUciNf14XJVl7Lj4A7PPEtyBWdQMqwE=; b=ho1zwoHZYAFYj0hNlsBRSo8NHF2pXLq5yVEBahxb73e559H1i0nKfm2G CXAVjjP+JFvxGkODRaCXmpHCCZk4Yt/YCcfRQt7vDRrrMSFaduAZGIuoo ux3r9cTPv6AmnLHYubVcdM2v40Kn5IO8D1UvRr7IrKaJKBTF46wrmBSSR SmbEomNzSpYKrp70c4JHd/t69F57IQNFyd1wICydMVH3KcKT4NUWWix5Z /2NCbXeS52pWfVcb0WxKa3zVESwvBcHdh9xll4fMJAHBP3l6OZFXnVulx uALdwS1BljrIRMNC30vKiCmiFteBaoopw2/djBs6nHZEaKDnaCoJdEP5a Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10516"; a="373001674" X-IronPort-AV: E=Sophos;i="5.95,227,1661842800"; d="scan'208";a="373001674" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Oct 2022 18:06:11 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10516"; a="584496185" X-IronPort-AV: E=Sophos;i="5.95,227,1661842800"; d="scan'208";a="584496185" Received: from allen-box.sh.intel.com ([10.239.159.48]) by orsmga003.jf.intel.com with ESMTP; 30 Oct 2022 18:06:08 -0700 From: Lu Baolu To: Joerg Roedel Cc: Jason Gunthorpe , Kevin Tian , Will Deacon , Robin Murphy , Jean-Philippe Brucker , iommu@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [RESEND PATCH v14 05/13] iommu: Add attach/detach_dev_pasid iommu interfaces Date: Mon, 31 Oct 2022 08:59:09 +0800 Message-Id: <20221031005917.45690-6-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221031005917.45690-1-baolu.lu@linux.intel.com> References: <20221031005917.45690-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Attaching an IOMMU domain to a PASID of a device is a generic operation for modern IOMMU drivers which support PASID-granular DMA address translation. Currently visible usage scenarios include (but not limited): - SVA (Shared Virtual Address) - kernel DMA with PASID - hardware-assist mediated device This adds the set_dev_pasid domain ops for setting the domain onto a PASID of a device and remove_dev_pasid iommu ops for removing any setup on a PASID of device. This also adds interfaces for device drivers to attach/detach/retrieve a domain for a PASID of a device. If multiple devices share a single group, it's fine as long the fabric always routes every TLP marked with a PASID to the host bridge and only the host bridge. For example, ACS achieves this universally and has been checked when pci_enable_pasid() is called. As we can't reliably tell the source apart in a group, all the devices in a group have to be considered as the same source, and mapped to the same PASID table. The DMA ownership is about the whole device (more precisely, iommu group), including the RID and PASIDs. When the ownership is converted, the pasid array must be empty. This also adds necessary checks in the DMA ownership interfaces. Signed-off-by: Lu Baolu Reviewed-by: Jean-Philippe Brucker Reviewed-by: Kevin Tian Reviewed-by: Yi Liu Reviewed-by: Jason Gunthorpe Tested-by: Zhangfei Gao Tested-by: Tony Zhu --- include/linux/iommu.h | 32 ++++++++++ drivers/iommu/iommu.c | 141 ++++++++++++++++++++++++++++++++++++++++-- 2 files changed, 169 insertions(+), 4 deletions(-) diff --git a/include/linux/iommu.h b/include/linux/iommu.h index 72bb0531aa76..5d2b78ac5416 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -223,6 +223,9 @@ struct iommu_iotlb_gather { * - IOMMU_DOMAIN_DMA: must use a dma domain * - 0: use the default setting * @default_domain_ops: the default ops for domains + * @remove_dev_pasid: Remove any translation configurations of a specific + * pasid, so that any DMA transactions with this pasid + * will be blocked by the hardware. * @pgsize_bitmap: bitmap of all possible supported page sizes * @owner: Driver module providing these ops */ @@ -256,6 +259,7 @@ struct iommu_ops { struct iommu_page_response *msg); =20 int (*def_domain_type)(struct device *dev); + void (*remove_dev_pasid)(struct device *dev, ioasid_t pasid); =20 const struct iommu_domain_ops *default_domain_ops; unsigned long pgsize_bitmap; @@ -266,6 +270,7 @@ struct iommu_ops { * struct iommu_domain_ops - domain specific operations * @attach_dev: attach an iommu domain to a device * @detach_dev: detach an iommu domain from a device + * @set_dev_pasid: set an iommu domain to a pasid of device * @map: map a physically contiguous memory region to an iommu domain * @map_pages: map a physically contiguous set of pages of the same size to * an iommu domain. @@ -286,6 +291,8 @@ struct iommu_ops { struct iommu_domain_ops { int (*attach_dev)(struct iommu_domain *domain, struct device *dev); void (*detach_dev)(struct iommu_domain *domain, struct device *dev); + int (*set_dev_pasid)(struct iommu_domain *domain, struct device *dev, + ioasid_t pasid); =20 int (*map)(struct iommu_domain *domain, unsigned long iova, phys_addr_t paddr, size_t size, int prot, gfp_t gfp); @@ -678,6 +685,13 @@ int iommu_group_claim_dma_owner(struct iommu_group *gr= oup, void *owner); void iommu_group_release_dma_owner(struct iommu_group *group); bool iommu_group_dma_owner_claimed(struct iommu_group *group); =20 +int iommu_attach_device_pasid(struct iommu_domain *domain, + struct device *dev, ioasid_t pasid); +void iommu_detach_device_pasid(struct iommu_domain *domain, + struct device *dev, ioasid_t pasid); +struct iommu_domain * +iommu_get_domain_for_dev_pasid(struct device *dev, ioasid_t pasid, + unsigned int type); #else /* CONFIG_IOMMU_API */ =20 struct iommu_ops {}; @@ -1040,6 +1054,24 @@ static inline bool iommu_group_dma_owner_claimed(str= uct iommu_group *group) { return false; } + +static inline int iommu_attach_device_pasid(struct iommu_domain *domain, + struct device *dev, ioasid_t pasid) +{ + return -ENODEV; +} + +static inline void iommu_detach_device_pasid(struct iommu_domain *domain, + struct device *dev, ioasid_t pasid) +{ +} + +static inline struct iommu_domain * +iommu_get_domain_for_dev_pasid(struct device *dev, ioasid_t pasid, + unsigned int type) +{ + return NULL; +} #endif /* CONFIG_IOMMU_API */ =20 /** diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c index a94ec648c88b..bf22992beb98 100644 --- a/drivers/iommu/iommu.c +++ b/drivers/iommu/iommu.c @@ -43,6 +43,7 @@ struct iommu_group { struct kobject kobj; struct kobject *devices_kobj; struct list_head devices; + struct xarray pasid_array; struct mutex mutex; void *iommu_data; void (*iommu_data_release)(void *iommu_data); @@ -723,6 +724,7 @@ struct iommu_group *iommu_group_alloc(void) mutex_init(&group->mutex); INIT_LIST_HEAD(&group->devices); INIT_LIST_HEAD(&group->entry); + xa_init(&group->pasid_array); =20 ret =3D ida_alloc(&iommu_group_ida, GFP_KERNEL); if (ret < 0) { @@ -3106,7 +3108,8 @@ int iommu_device_use_default_domain(struct device *de= v) =20 mutex_lock(&group->mutex); if (group->owner_cnt) { - if (group->owner || !iommu_is_default_domain(group)) { + if (group->owner || !iommu_is_default_domain(group) || + !xa_empty(&group->pasid_array)) { ret =3D -EBUSY; goto unlock_out; } @@ -3137,7 +3140,7 @@ void iommu_device_unuse_default_domain(struct device = *dev) return; =20 mutex_lock(&group->mutex); - if (!WARN_ON(!group->owner_cnt)) + if (!WARN_ON(!group->owner_cnt || !xa_empty(&group->pasid_array))) group->owner_cnt--; =20 mutex_unlock(&group->mutex); @@ -3185,7 +3188,8 @@ int iommu_group_claim_dma_owner(struct iommu_group *g= roup, void *owner) ret =3D -EPERM; goto unlock_out; } else { - if (group->domain && group->domain !=3D group->default_domain) { + if ((group->domain && group->domain !=3D group->default_domain) || + !xa_empty(&group->pasid_array)) { ret =3D -EBUSY; goto unlock_out; } @@ -3219,7 +3223,8 @@ void iommu_group_release_dma_owner(struct iommu_group= *group) int ret; =20 mutex_lock(&group->mutex); - if (WARN_ON(!group->owner_cnt || !group->owner)) + if (WARN_ON(!group->owner_cnt || !group->owner || + !xa_empty(&group->pasid_array))) goto unlock_out; =20 group->owner_cnt =3D 0; @@ -3250,3 +3255,131 @@ bool iommu_group_dma_owner_claimed(struct iommu_gro= up *group) return user; } EXPORT_SYMBOL_GPL(iommu_group_dma_owner_claimed); + +static int __iommu_set_group_pasid(struct iommu_domain *domain, + struct iommu_group *group, ioasid_t pasid) +{ + struct group_device *device; + int ret =3D 0; + + list_for_each_entry(device, &group->devices, list) { + ret =3D domain->ops->set_dev_pasid(domain, device->dev, pasid); + if (ret) + break; + } + + return ret; +} + +static void __iommu_remove_group_pasid(struct iommu_group *group, + ioasid_t pasid) +{ + struct group_device *device; + const struct iommu_ops *ops; + + list_for_each_entry(device, &group->devices, list) { + ops =3D dev_iommu_ops(device->dev); + ops->remove_dev_pasid(device->dev, pasid); + } +} + +/* + * iommu_attach_device_pasid() - Attach a domain to pasid of device + * @domain: the iommu domain. + * @dev: the attached device. + * @pasid: the pasid of the device. + * + * Return: 0 on success, or an error. + */ +int iommu_attach_device_pasid(struct iommu_domain *domain, + struct device *dev, ioasid_t pasid) +{ + struct iommu_group *group; + void *curr; + int ret; + + if (!domain->ops->set_dev_pasid) + return -EOPNOTSUPP; + + group =3D iommu_group_get(dev); + if (!group) + return -ENODEV; + + mutex_lock(&group->mutex); + curr =3D xa_cmpxchg(&group->pasid_array, pasid, NULL, domain, GFP_KERNEL); + if (curr) { + ret =3D xa_err(curr) ? : -EBUSY; + goto out_unlock; + } + + ret =3D __iommu_set_group_pasid(domain, group, pasid); + if (ret) { + __iommu_remove_group_pasid(group, pasid); + xa_erase(&group->pasid_array, pasid); + } +out_unlock: + mutex_unlock(&group->mutex); + iommu_group_put(group); + + return ret; +} +EXPORT_SYMBOL_GPL(iommu_attach_device_pasid); + +/* + * iommu_detach_device_pasid() - Detach the domain from pasid of device + * @domain: the iommu domain. + * @dev: the attached device. + * @pasid: the pasid of the device. + * + * The @domain must have been attached to @pasid of the @dev with + * iommu_attach_device_pasid(). + */ +void iommu_detach_device_pasid(struct iommu_domain *domain, struct device = *dev, + ioasid_t pasid) +{ + struct iommu_group *group =3D iommu_group_get(dev); + + mutex_lock(&group->mutex); + __iommu_remove_group_pasid(group, pasid); + WARN_ON(xa_erase(&group->pasid_array, pasid) !=3D domain); + mutex_unlock(&group->mutex); + + iommu_group_put(group); +} +EXPORT_SYMBOL_GPL(iommu_detach_device_pasid); + +/* + * iommu_get_domain_for_dev_pasid() - Retrieve domain for @pasid of @dev + * @dev: the queried device + * @pasid: the pasid of the device + * @type: matched domain type, 0 for any match + * + * This is a variant of iommu_get_domain_for_dev(). It returns the existing + * domain attached to pasid of a device. Callers must hold a lock around t= his + * function, and both iommu_attach/detach_dev_pasid() whenever a domain of + * type is being manipulated. This API does not internally resolve races w= ith + * attach/detach. + * + * Return: attached domain on success, NULL otherwise. + */ +struct iommu_domain *iommu_get_domain_for_dev_pasid(struct device *dev, + ioasid_t pasid, + unsigned int type) +{ + struct iommu_domain *domain; + struct iommu_group *group; + + group =3D iommu_group_get(dev); + if (!group) + return NULL; + + xa_lock(&group->pasid_array); + domain =3D xa_load(&group->pasid_array, pasid); + if (type && domain && domain->type !=3D type) + domain =3D ERR_PTR(-EBUSY); + xa_unlock(&group->pasid_array); + iommu_group_put(group); + + return domain; +} +EXPORT_SYMBOL_GPL(iommu_get_domain_for_dev_pasid); --=20 2.34.1 From nobody Thu Apr 2 11:39:36 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A7B23FA373D for ; Mon, 31 Oct 2022 01:06:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229752AbiJaBGi (ORCPT ); Sun, 30 Oct 2022 21:06:38 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36690 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229785AbiJaBGS (ORCPT ); Sun, 30 Oct 2022 21:06:18 -0400 Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 865CBB844 for ; Sun, 30 Oct 2022 18:06:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1667178373; x=1698714373; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=nTBCdGKexYCHUJx9Vu/JLrB95BP+arp/FT6C/BQTnXY=; b=OrJ1Mekje2oNvkPNtjUKGs8xDaIQKKeuHc6R6/4yzNuSZfDTTovHceVH /VPuae0fJbSLFnI3HdBhPyzi3USAdr52T6VZ5QwsYeMKNkR0AFUDz42/8 sZxHozqOvoQzBHDDJEZu0fHlKr38MjMlh+7t8E8rqU2WGtAVvppDMtBX4 qTNzMQjXN3HcEmQOdLu/t1Kgs9hFLZHqPRH1WjHBlsnvzsUKgfFy5m4rR gw8o8cUJ3AUJzsgihdi//LGXIfEuP3Kc/Q/uXSXj0DjznkaFnoQ4EVZGM qg2ehlBAokpfOq3An45yKmlPOjhQoDbmxDW6FyBOEuZTNnL+hHU3zTMQA Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10516"; a="373001682" X-IronPort-AV: E=Sophos;i="5.95,227,1661842800"; d="scan'208";a="373001682" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Oct 2022 18:06:13 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10516"; a="584496203" X-IronPort-AV: E=Sophos;i="5.95,227,1661842800"; d="scan'208";a="584496203" Received: from allen-box.sh.intel.com ([10.239.159.48]) by orsmga003.jf.intel.com with ESMTP; 30 Oct 2022 18:06:11 -0700 From: Lu Baolu To: Joerg Roedel Cc: Jason Gunthorpe , Kevin Tian , Will Deacon , Robin Murphy , Jean-Philippe Brucker , iommu@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [RESEND PATCH v14 06/13] iommu: Add IOMMU SVA domain support Date: Mon, 31 Oct 2022 08:59:10 +0800 Message-Id: <20221031005917.45690-7-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221031005917.45690-1-baolu.lu@linux.intel.com> References: <20221031005917.45690-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The SVA iommu_domain represents a hardware pagetable that the IOMMU hardware could use for SVA translation. This adds some infrastructures to support SVA domain in the iommu core. It includes: - Extend the iommu_domain to support a new IOMMU_DOMAIN_SVA domain type. The IOMMU drivers that support allocation of the SVA domain should provide its own SVA domain specific iommu_domain_ops. - Add a helper to allocate an SVA domain. The iommu_domain_free() is still used to free an SVA domain. The report_iommu_fault() should be replaced by the new iommu_report_device_fault(). Leave the existing fault handler with the existing users and the newly added SVA members excludes it. Suggested-by: Jean-Philippe Brucker Suggested-by: Jason Gunthorpe Signed-off-by: Lu Baolu Reviewed-by: Jean-Philippe Brucker Reviewed-by: Kevin Tian Reviewed-by: Jason Gunthorpe Reviewed-by: Yi Liu Tested-by: Zhangfei Gao Tested-by: Tony Zhu --- include/linux/iommu.h | 25 +++++++++++++++++++++++-- drivers/iommu/iommu.c | 20 ++++++++++++++++++++ 2 files changed, 43 insertions(+), 2 deletions(-) diff --git a/include/linux/iommu.h b/include/linux/iommu.h index 5d2b78ac5416..776baa375967 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -64,6 +64,8 @@ struct iommu_domain_geometry { #define __IOMMU_DOMAIN_PT (1U << 2) /* Domain is identity mapped */ #define __IOMMU_DOMAIN_DMA_FQ (1U << 3) /* DMA-API uses flush queue */ =20 +#define __IOMMU_DOMAIN_SVA (1U << 4) /* Shared process address space */ + /* * This are the possible domain-types * @@ -77,6 +79,8 @@ struct iommu_domain_geometry { * certain optimizations for these domains * IOMMU_DOMAIN_DMA_FQ - As above, but definitely using batched TLB * invalidation. + * IOMMU_DOMAIN_SVA - DMA addresses are shared process addresses + * represented by mm_struct's. */ #define IOMMU_DOMAIN_BLOCKED (0U) #define IOMMU_DOMAIN_IDENTITY (__IOMMU_DOMAIN_PT) @@ -86,15 +90,24 @@ struct iommu_domain_geometry { #define IOMMU_DOMAIN_DMA_FQ (__IOMMU_DOMAIN_PAGING | \ __IOMMU_DOMAIN_DMA_API | \ __IOMMU_DOMAIN_DMA_FQ) +#define IOMMU_DOMAIN_SVA (__IOMMU_DOMAIN_SVA) =20 struct iommu_domain { unsigned type; const struct iommu_domain_ops *ops; unsigned long pgsize_bitmap; /* Bitmap of page sizes in use */ - iommu_fault_handler_t handler; - void *handler_token; struct iommu_domain_geometry geometry; struct iommu_dma_cookie *iova_cookie; + union { + struct { + iommu_fault_handler_t handler; + void *handler_token; + }; + struct { /* IOMMU_DOMAIN_SVA */ + struct mm_struct *mm; + int users; + }; + }; }; =20 static inline bool iommu_is_dma_domain(struct iommu_domain *domain) @@ -685,6 +698,8 @@ int iommu_group_claim_dma_owner(struct iommu_group *gro= up, void *owner); void iommu_group_release_dma_owner(struct iommu_group *group); bool iommu_group_dma_owner_claimed(struct iommu_group *group); =20 +struct iommu_domain *iommu_sva_domain_alloc(struct device *dev, + struct mm_struct *mm); int iommu_attach_device_pasid(struct iommu_domain *domain, struct device *dev, ioasid_t pasid); void iommu_detach_device_pasid(struct iommu_domain *domain, @@ -1055,6 +1070,12 @@ static inline bool iommu_group_dma_owner_claimed(str= uct iommu_group *group) return false; } =20 +static inline struct iommu_domain * +iommu_sva_domain_alloc(struct device *dev, struct mm_struct *mm) +{ + return NULL; +} + static inline int iommu_attach_device_pasid(struct iommu_domain *domain, struct device *dev, ioasid_t pasid) { diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c index bf22992beb98..6a1cd2018e30 100644 --- a/drivers/iommu/iommu.c +++ b/drivers/iommu/iommu.c @@ -29,6 +29,7 @@ #include #include #include +#include =20 #include "dma-iommu.h" =20 @@ -1934,6 +1935,8 @@ EXPORT_SYMBOL_GPL(iommu_domain_alloc); =20 void iommu_domain_free(struct iommu_domain *domain) { + if (domain->type =3D=3D IOMMU_DOMAIN_SVA) + mmdrop(domain->mm); iommu_put_dma_cookie(domain); domain->ops->free(domain); } @@ -3383,3 +3386,20 @@ struct iommu_domain *iommu_get_domain_for_dev_pasid(= struct device *dev, return domain; } EXPORT_SYMBOL_GPL(iommu_get_domain_for_dev_pasid); + +struct iommu_domain *iommu_sva_domain_alloc(struct device *dev, + struct mm_struct *mm) +{ + const struct iommu_ops *ops =3D dev_iommu_ops(dev); + struct iommu_domain *domain; + + domain =3D ops->domain_alloc(IOMMU_DOMAIN_SVA); + if (!domain) + return NULL; + + domain->type =3D IOMMU_DOMAIN_SVA; + mmgrab(mm); + domain->mm =3D mm; + + return domain; +} --=20 2.34.1 From nobody Thu Apr 2 11:39:36 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0FFA7FA373D for ; Mon, 31 Oct 2022 01:06:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229770AbiJaBGo (ORCPT ); Sun, 30 Oct 2022 21:06:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36894 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229827AbiJaBGZ (ORCPT ); Sun, 30 Oct 2022 21:06:25 -0400 Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2BF46BC38 for ; Sun, 30 Oct 2022 18:06:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1667178376; x=1698714376; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=H2yp9akidaoFr+h4pIHRp2BZ1P9liVQ3w2A6JZy31yM=; b=Kj3+bh92j0YoIht8XZTNnyaUbbK41PU+zle4YyepZ05TVF7dX8mTvT3N WVyli/qzy2N87SGn0V5CnIx1obGssIzCTEsT7GwVKlYoEwsjJJJSoZHCn NzF0fyF9ncTvGg+V5WVxOuW0QqmsU5Yqy1CFgDSLCa5mjGdAnOV3roTvv CYlX8AuU5ezkDhIh6qggZnIAlpYPc9D0AhQ8pxf0N2dmj9b/qk7MWlt8n 6bSiYQEt0WaGlkMUillWV/6ZtnbrSV+BhxkK40ZTKfh49Z0eC3KvRcze9 uGGFrEGM5cfYrZYF5pB6Tce8rYfEu4j7/qnSs79M0vEOH6Bes2IE5crya g==; X-IronPort-AV: E=McAfee;i="6500,9779,10516"; a="373001688" X-IronPort-AV: E=Sophos;i="5.95,227,1661842800"; d="scan'208";a="373001688" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Oct 2022 18:06:15 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10516"; a="584496218" X-IronPort-AV: E=Sophos;i="5.95,227,1661842800"; d="scan'208";a="584496218" Received: from allen-box.sh.intel.com ([10.239.159.48]) by orsmga003.jf.intel.com with ESMTP; 30 Oct 2022 18:06:13 -0700 From: Lu Baolu To: Joerg Roedel Cc: Jason Gunthorpe , Kevin Tian , Will Deacon , Robin Murphy , Jean-Philippe Brucker , iommu@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [RESEND PATCH v14 07/13] iommu/vt-d: Add SVA domain support Date: Mon, 31 Oct 2022 08:59:11 +0800 Message-Id: <20221031005917.45690-8-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221031005917.45690-1-baolu.lu@linux.intel.com> References: <20221031005917.45690-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Add support for SVA domain allocation and provide an SVA-specific iommu_domain_ops. This implementation is based on the existing SVA code. Possible cleanup and refactoring are left for incremental changes later. The VT-d driver will also need to support setting a DMA domain to a PASID of device. Current SVA implementation uses different data structures to track the domain and device PASID relationship. That's the reason why we need to check the domain type in remove_dev_pasid callback. Eventually we'll consolidate the data structures and remove the need of domain type check. Signed-off-by: Lu Baolu Reviewed-by: Kevin Tian Reviewed-by: Jason Gunthorpe Reviewed-by: Yi Liu Tested-by: Tony Zhu --- drivers/iommu/intel/iommu.h | 10 ++++++++ drivers/iommu/intel/iommu.c | 25 ++++++++++++++++++++ drivers/iommu/intel/svm.c | 47 +++++++++++++++++++++++++++++++++++++ 3 files changed, 82 insertions(+) diff --git a/drivers/iommu/intel/iommu.h b/drivers/iommu/intel/iommu.h index 33e5bcaf2a6c..252fa344f88a 100644 --- a/drivers/iommu/intel/iommu.h +++ b/drivers/iommu/intel/iommu.h @@ -753,6 +753,8 @@ void intel_svm_unbind(struct iommu_sva *handle); u32 intel_svm_get_pasid(struct iommu_sva *handle); int intel_svm_page_response(struct device *dev, struct iommu_fault_event *= evt, struct iommu_page_response *msg); +struct iommu_domain *intel_svm_domain_alloc(void); +void intel_svm_remove_dev_pasid(struct device *dev, ioasid_t pasid); =20 struct intel_svm_dev { struct list_head list; @@ -777,6 +779,14 @@ struct intel_svm { }; #else static inline void intel_svm_check(struct intel_iommu *iommu) {} +static inline struct iommu_domain *intel_svm_domain_alloc(void) +{ + return NULL; +} + +static inline void intel_svm_remove_dev_pasid(struct device *dev, ioasid_t= pasid) +{ +} #endif =20 #ifdef CONFIG_INTEL_IOMMU_DEBUGFS diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c index 48cdcd0a5cf3..7b67e431dd36 100644 --- a/drivers/iommu/intel/iommu.c +++ b/drivers/iommu/intel/iommu.c @@ -4169,6 +4169,8 @@ static struct iommu_domain *intel_iommu_domain_alloc(= unsigned type) return domain; case IOMMU_DOMAIN_IDENTITY: return &si_domain->domain; + case IOMMU_DOMAIN_SVA: + return intel_svm_domain_alloc(); default: return NULL; } @@ -4712,6 +4714,28 @@ static void intel_iommu_iotlb_sync_map(struct iommu_= domain *domain, __mapping_notify_one(info->iommu, dmar_domain, pfn, pages); } =20 +static void intel_iommu_remove_dev_pasid(struct device *dev, ioasid_t pasi= d) +{ + struct intel_iommu *iommu =3D device_to_iommu(dev, NULL, NULL); + struct iommu_domain *domain; + + /* Domain type specific cleanup: */ + domain =3D iommu_get_domain_for_dev_pasid(dev, pasid, 0); + if (domain) { + switch (domain->type) { + case IOMMU_DOMAIN_SVA: + intel_svm_remove_dev_pasid(dev, pasid); + break; + default: + /* should never reach here */ + WARN_ON(1); + break; + } + } + + intel_pasid_tear_down_entry(iommu, dev, pasid, false); +} + const struct iommu_ops intel_iommu_ops =3D { .capable =3D intel_iommu_capable, .domain_alloc =3D intel_iommu_domain_alloc, @@ -4724,6 +4748,7 @@ const struct iommu_ops intel_iommu_ops =3D { .dev_disable_feat =3D intel_iommu_dev_disable_feat, .is_attach_deferred =3D intel_iommu_is_attach_deferred, .def_domain_type =3D device_def_domain_type, + .remove_dev_pasid =3D intel_iommu_remove_dev_pasid, .pgsize_bitmap =3D SZ_4K, #ifdef CONFIG_INTEL_IOMMU_SVM .sva_bind =3D intel_svm_bind, diff --git a/drivers/iommu/intel/svm.c b/drivers/iommu/intel/svm.c index 94bc47b68c93..86c8ea0d9635 100644 --- a/drivers/iommu/intel/svm.c +++ b/drivers/iommu/intel/svm.c @@ -882,3 +882,50 @@ int intel_svm_page_response(struct device *dev, out: return ret; } + +void intel_svm_remove_dev_pasid(struct device *dev, ioasid_t pasid) +{ + mutex_lock(&pasid_mutex); + intel_svm_unbind_mm(dev, pasid); + mutex_unlock(&pasid_mutex); +} + +static int intel_svm_set_dev_pasid(struct iommu_domain *domain, + struct device *dev, ioasid_t pasid) +{ + struct device_domain_info *info =3D dev_iommu_priv_get(dev); + struct intel_iommu *iommu =3D info->iommu; + struct mm_struct *mm =3D domain->mm; + struct iommu_sva *sva; + int ret =3D 0; + + mutex_lock(&pasid_mutex); + sva =3D intel_svm_bind_mm(iommu, dev, mm); + if (IS_ERR(sva)) + ret =3D PTR_ERR(sva); + mutex_unlock(&pasid_mutex); + + return ret; +} + +static void intel_svm_domain_free(struct iommu_domain *domain) +{ + kfree(to_dmar_domain(domain)); +} + +static const struct iommu_domain_ops intel_svm_domain_ops =3D { + .set_dev_pasid =3D intel_svm_set_dev_pasid, + .free =3D intel_svm_domain_free +}; + +struct iommu_domain *intel_svm_domain_alloc(void) +{ + struct dmar_domain *domain; + + domain =3D kzalloc(sizeof(*domain), GFP_KERNEL); + if (!domain) + return NULL; + domain->domain.ops =3D &intel_svm_domain_ops; + + return &domain->domain; +} --=20 2.34.1 From nobody Thu Apr 2 11:39:36 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 36E60FA373D for ; Mon, 31 Oct 2022 01:06:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229839AbiJaBGt (ORCPT ); Sun, 30 Oct 2022 21:06:49 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36692 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229841AbiJaBGg (ORCPT ); Sun, 30 Oct 2022 21:06:36 -0400 Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 60C5FBE14 for ; Sun, 30 Oct 2022 18:06:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1667178378; x=1698714378; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=3kTF9wd8HGVyu3rBeZbbMCuRbqEwRWrHpebS2QDRoIA=; b=Iq7fpgaoFv+VKSIN0Izx7WkbwHlaiEKWDSEeVAclh55pR64EkV4KpF93 2dTbThCO/l49gxlwdjYIXT1jPOjhE3S8ZFD3FB2g/HDdvI3R+dwvSrtzf aJ7q88SiHvqGy9QnFgjbZNslgQDHwRJi+kUbWks93qeoh4TltsX702hkQ cunPkHc+aVgJQb0o2FzDFHzT/ygRVl59GMoFrMU0s6LgT9v0F1USDamYA J2vNLn/XaKJ8mBUqjZL4TVqzl9Sf/YZW9+xsKR3mKe3FcLuOGUtrx1eoD 37qEyR5/AlCrmlwENeEGTcTQxnbEDvDUg3elu71kE7cBqAEBvvrtRqnpg w==; X-IronPort-AV: E=McAfee;i="6500,9779,10516"; a="373001697" X-IronPort-AV: E=Sophos;i="5.95,227,1661842800"; d="scan'208";a="373001697" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Oct 2022 18:06:18 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10516"; a="584496240" X-IronPort-AV: E=Sophos;i="5.95,227,1661842800"; d="scan'208";a="584496240" Received: from allen-box.sh.intel.com ([10.239.159.48]) by orsmga003.jf.intel.com with ESMTP; 30 Oct 2022 18:06:15 -0700 From: Lu Baolu To: Joerg Roedel Cc: Jason Gunthorpe , Kevin Tian , Will Deacon , Robin Murphy , Jean-Philippe Brucker , iommu@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [RESEND PATCH v14 08/13] arm-smmu-v3/sva: Add SVA domain support Date: Mon, 31 Oct 2022 08:59:12 +0800 Message-Id: <20221031005917.45690-9-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221031005917.45690-1-baolu.lu@linux.intel.com> References: <20221031005917.45690-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Add support for SVA domain allocation and provide an SVA-specific iommu_domain_ops. This implementation is based on the existing SVA code. Possible cleanup and refactoring are left for incremental changes later. Signed-off-by: Lu Baolu Reviewed-by: Jean-Philippe Brucker Reviewed-by: Jason Gunthorpe Tested-by: Zhangfei Gao --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 14 +++++ .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 61 +++++++++++++++++++ drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 15 +++++ 3 files changed, 90 insertions(+) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/ar= m/arm-smmu-v3/arm-smmu-v3.h index d2ba86470c42..5aa853e98d38 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -758,6 +758,9 @@ struct iommu_sva *arm_smmu_sva_bind(struct device *dev,= struct mm_struct *mm); void arm_smmu_sva_unbind(struct iommu_sva *handle); u32 arm_smmu_sva_get_pasid(struct iommu_sva *handle); void arm_smmu_sva_notifier_synchronize(void); +struct iommu_domain *arm_smmu_sva_domain_alloc(void); +void arm_smmu_sva_remove_dev_pasid(struct iommu_domain *domain, + struct device *dev, ioasid_t id); #else /* CONFIG_ARM_SMMU_V3_SVA */ static inline bool arm_smmu_sva_supported(struct arm_smmu_device *smmu) { @@ -803,5 +806,16 @@ static inline u32 arm_smmu_sva_get_pasid(struct iommu_= sva *handle) } =20 static inline void arm_smmu_sva_notifier_synchronize(void) {} + +static inline struct iommu_domain *arm_smmu_sva_domain_alloc(void) +{ + return NULL; +} + +static inline void arm_smmu_sva_remove_dev_pasid(struct iommu_domain *doma= in, + struct device *dev, + ioasid_t id) +{ +} #endif /* CONFIG_ARM_SMMU_V3_SVA */ #endif /* _ARM_SMMU_V3_H */ diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iomm= u/arm/arm-smmu-v3/arm-smmu-v3-sva.c index 8fcf0df4bd0e..2d188d12419e 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -549,3 +549,64 @@ void arm_smmu_sva_notifier_synchronize(void) */ mmu_notifier_synchronize(); } + +void arm_smmu_sva_remove_dev_pasid(struct iommu_domain *domain, + struct device *dev, ioasid_t id) +{ + struct mm_struct *mm =3D domain->mm; + struct arm_smmu_bond *bond =3D NULL, *t; + struct arm_smmu_master *master =3D dev_iommu_priv_get(dev); + + mutex_lock(&sva_lock); + list_for_each_entry(t, &master->bonds, list) { + if (t->mm =3D=3D mm) { + bond =3D t; + break; + } + } + + if (!WARN_ON(!bond) && refcount_dec_and_test(&bond->refs)) { + list_del(&bond->list); + arm_smmu_mmu_notifier_put(bond->smmu_mn); + kfree(bond); + } + mutex_unlock(&sva_lock); +} + +static int arm_smmu_sva_set_dev_pasid(struct iommu_domain *domain, + struct device *dev, ioasid_t id) +{ + int ret =3D 0; + struct iommu_sva *handle; + struct mm_struct *mm =3D domain->mm; + + mutex_lock(&sva_lock); + handle =3D __arm_smmu_sva_bind(dev, mm); + if (IS_ERR(handle)) + ret =3D PTR_ERR(handle); + mutex_unlock(&sva_lock); + + return ret; +} + +static void arm_smmu_sva_domain_free(struct iommu_domain *domain) +{ + kfree(domain); +} + +static const struct iommu_domain_ops arm_smmu_sva_domain_ops =3D { + .set_dev_pasid =3D arm_smmu_sva_set_dev_pasid, + .free =3D arm_smmu_sva_domain_free +}; + +struct iommu_domain *arm_smmu_sva_domain_alloc(void) +{ + struct iommu_domain *domain; + + domain =3D kzalloc(sizeof(*domain), GFP_KERNEL); + if (!domain) + return NULL; + domain->ops =3D &arm_smmu_sva_domain_ops; + + return domain; +} diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/ar= m/arm-smmu-v3/arm-smmu-v3.c index 21cb13da122c..eed2eb8effa3 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -2009,6 +2009,9 @@ static struct iommu_domain *arm_smmu_domain_alloc(uns= igned type) { struct arm_smmu_domain *smmu_domain; =20 + if (type =3D=3D IOMMU_DOMAIN_SVA) + return arm_smmu_sva_domain_alloc(); + if (type !=3D IOMMU_DOMAIN_UNMANAGED && type !=3D IOMMU_DOMAIN_DMA && type !=3D IOMMU_DOMAIN_DMA_FQ && @@ -2838,6 +2841,17 @@ static int arm_smmu_def_domain_type(struct device *d= ev) return 0; } =20 +static void arm_smmu_remove_dev_pasid(struct device *dev, ioasid_t pasid) +{ + struct iommu_domain *domain; + + domain =3D iommu_get_domain_for_dev_pasid(dev, pasid, IOMMU_DOMAIN_SVA); + if (WARN_ON(IS_ERR(domain)) || !domain) + return; + + arm_smmu_sva_remove_dev_pasid(domain, dev, pasid); +} + static struct iommu_ops arm_smmu_ops =3D { .capable =3D arm_smmu_capable, .domain_alloc =3D arm_smmu_domain_alloc, @@ -2846,6 +2860,7 @@ static struct iommu_ops arm_smmu_ops =3D { .device_group =3D arm_smmu_device_group, .of_xlate =3D arm_smmu_of_xlate, .get_resv_regions =3D arm_smmu_get_resv_regions, + .remove_dev_pasid =3D arm_smmu_remove_dev_pasid, .dev_enable_feat =3D arm_smmu_dev_enable_feature, .dev_disable_feat =3D arm_smmu_dev_disable_feature, .sva_bind =3D arm_smmu_sva_bind, --=20 2.34.1 From nobody Thu Apr 2 11:39:36 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8865CC38A02 for ; Mon, 31 Oct 2022 01:07:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229795AbiJaBHA (ORCPT ); Sun, 30 Oct 2022 21:07:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36550 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229868AbiJaBGi (ORCPT ); Sun, 30 Oct 2022 21:06:38 -0400 Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B4857A467 for ; Sun, 30 Oct 2022 18:06:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1667178380; x=1698714380; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=b1d6sViO5fWOIGO0wlM7s87valHVmhNuqdZzNLB22vw=; b=JwNcoo2/az/vCPQEasVIkrduuIRCFHAJj1KC/HsVwFpCHloSpg+8sCe6 vk8YFfFgCSf7612U+6o4uDSEbWtwTjB2Mp44lf7Z5//Q74EKxznUL6kLw j1UpYzCq8NLPYaXwnqPeiMZESJbOHEK7D6iti8p44QTzxNlmhL/eOOYim SfOT7aCp1RcL7MEkY6xdOo9uz4H3AEEljWBiClkXZBBE8Yhizt3DJMSwf srfTOgv13Q+Kb0bQ/H4rIXAWEJamWDcrj5maSGh+VsKgQtngW3puxal7e iWC/KEDRZ1c1WCSEXZ3Zs77DuvCcYI9xdlMlYM5G8hMWNAvwC2Yac/yQm g==; X-IronPort-AV: E=McAfee;i="6500,9779,10516"; a="373001704" X-IronPort-AV: E=Sophos;i="5.95,227,1661842800"; d="scan'208";a="373001704" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Oct 2022 18:06:20 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10516"; a="584496256" X-IronPort-AV: E=Sophos;i="5.95,227,1661842800"; d="scan'208";a="584496256" Received: from allen-box.sh.intel.com ([10.239.159.48]) by orsmga003.jf.intel.com with ESMTP; 30 Oct 2022 18:06:18 -0700 From: Lu Baolu To: Joerg Roedel Cc: Jason Gunthorpe , Kevin Tian , Will Deacon , Robin Murphy , Jean-Philippe Brucker , iommu@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [RESEND PATCH v14 09/13] iommu/sva: Refactoring iommu_sva_bind/unbind_device() Date: Mon, 31 Oct 2022 08:59:13 +0800 Message-Id: <20221031005917.45690-10-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221031005917.45690-1-baolu.lu@linux.intel.com> References: <20221031005917.45690-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The existing iommu SVA interfaces are implemented by calling the SVA specific iommu ops provided by the IOMMU drivers. There's no need for any SVA specific ops in iommu_ops vector anymore as we can achieve this through the generic attach/detach_dev_pasid domain ops. This refactors the IOMMU SVA interfaces implementation by using the iommu_attach/detach_device_pasid interfaces and align them with the concept of the SVA iommu domain. Put the new SVA code in the SVA related file in order to make it self-contained. Signed-off-by: Lu Baolu Reviewed-by: Jean-Philippe Brucker Reviewed-by: Kevin Tian Reviewed-by: Jason Gunthorpe Tested-by: Zhangfei Gao Tested-by: Tony Zhu --- include/linux/iommu.h | 43 +++++++------ drivers/iommu/iommu-sva-lib.c | 111 ++++++++++++++++++++++++++++++++++ drivers/iommu/iommu.c | 91 ---------------------------- 3 files changed, 134 insertions(+), 111 deletions(-) diff --git a/include/linux/iommu.h b/include/linux/iommu.h index 776baa375967..bee5659d07eb 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -645,6 +645,7 @@ struct iommu_fwspec { */ struct iommu_sva { struct device *dev; + struct iommu_domain *domain; }; =20 int iommu_fwspec_init(struct device *dev, struct fwnode_handle *iommu_fwno= de, @@ -686,11 +687,6 @@ void iommu_release_device(struct device *dev); int iommu_dev_enable_feature(struct device *dev, enum iommu_dev_features f= ); int iommu_dev_disable_feature(struct device *dev, enum iommu_dev_features = f); =20 -struct iommu_sva *iommu_sva_bind_device(struct device *dev, - struct mm_struct *mm); -void iommu_sva_unbind_device(struct iommu_sva *handle); -u32 iommu_sva_get_pasid(struct iommu_sva *handle); - int iommu_device_use_default_domain(struct device *dev); void iommu_device_unuse_default_domain(struct device *dev); =20 @@ -1026,21 +1022,6 @@ iommu_dev_disable_feature(struct device *dev, enum i= ommu_dev_features feat) return -ENODEV; } =20 -static inline struct iommu_sva * -iommu_sva_bind_device(struct device *dev, struct mm_struct *mm) -{ - return NULL; -} - -static inline void iommu_sva_unbind_device(struct iommu_sva *handle) -{ -} - -static inline u32 iommu_sva_get_pasid(struct iommu_sva *handle) -{ - return IOMMU_PASID_INVALID; -} - static inline struct iommu_fwspec *dev_iommu_fwspec_get(struct device *dev) { return NULL; @@ -1154,4 +1135,26 @@ static inline void iommu_dma_compose_msi_msg(struct = msi_desc *desc, struct msi_m =20 #endif /* CONFIG_IOMMU_DMA */ =20 +#ifdef CONFIG_IOMMU_SVA +struct iommu_sva *iommu_sva_bind_device(struct device *dev, + struct mm_struct *mm); +void iommu_sva_unbind_device(struct iommu_sva *handle); +u32 iommu_sva_get_pasid(struct iommu_sva *handle); +#else +static inline struct iommu_sva * +iommu_sva_bind_device(struct device *dev, struct mm_struct *mm) +{ + return NULL; +} + +static inline void iommu_sva_unbind_device(struct iommu_sva *handle) +{ +} + +static inline u32 iommu_sva_get_pasid(struct iommu_sva *handle) +{ + return IOMMU_PASID_INVALID; +} +#endif /* CONFIG_IOMMU_SVA */ + #endif /* __LINUX_IOMMU_H */ diff --git a/drivers/iommu/iommu-sva-lib.c b/drivers/iommu/iommu-sva-lib.c index 106506143896..e425573a1787 100644 --- a/drivers/iommu/iommu-sva-lib.c +++ b/drivers/iommu/iommu-sva-lib.c @@ -4,6 +4,7 @@ */ #include #include +#include =20 #include "iommu-sva-lib.h" =20 @@ -69,3 +70,113 @@ struct mm_struct *iommu_sva_find(ioasid_t pasid) return ioasid_find(&iommu_sva_pasid, pasid, __mmget_not_zero); } EXPORT_SYMBOL_GPL(iommu_sva_find); + +/** + * iommu_sva_bind_device() - Bind a process address space to a device + * @dev: the device + * @mm: the mm to bind, caller must hold a reference to mm_users + * + * Create a bond between device and address space, allowing the device to + * access the mm using the PASID returned by iommu_sva_get_pasid(). If a + * bond already exists between @device and @mm, an additional internal + * reference is taken. Caller must call iommu_sva_unbind_device() + * to release each reference. + * + * iommu_dev_enable_feature(dev, IOMMU_DEV_FEAT_SVA) must be called first,= to + * initialize the required SVA features. + * + * On error, returns an ERR_PTR value. + */ +struct iommu_sva *iommu_sva_bind_device(struct device *dev, struct mm_stru= ct *mm) +{ + struct iommu_domain *domain; + struct iommu_sva *handle; + ioasid_t max_pasids; + int ret; + + max_pasids =3D dev->iommu->max_pasids; + if (!max_pasids) + return ERR_PTR(-EOPNOTSUPP); + + /* Allocate mm->pasid if necessary. */ + ret =3D iommu_sva_alloc_pasid(mm, 1, max_pasids - 1); + if (ret) + return ERR_PTR(ret); + + handle =3D kzalloc(sizeof(*handle), GFP_KERNEL); + if (!handle) + return ERR_PTR(-ENOMEM); + + mutex_lock(&iommu_sva_lock); + /* Search for an existing domain. */ + domain =3D iommu_get_domain_for_dev_pasid(dev, mm->pasid, + IOMMU_DOMAIN_SVA); + if (IS_ERR(domain)) { + ret =3D PTR_ERR(domain); + goto out_unlock; + } + + if (domain) { + domain->users++; + goto out; + } + + /* Allocate a new domain and set it on device pasid. */ + domain =3D iommu_sva_domain_alloc(dev, mm); + if (!domain) { + ret =3D -ENOMEM; + goto out_unlock; + } + + ret =3D iommu_attach_device_pasid(domain, dev, mm->pasid); + if (ret) + goto out_free_domain; + domain->users =3D 1; +out: + mutex_unlock(&iommu_sva_lock); + handle->dev =3D dev; + handle->domain =3D domain; + + return handle; + +out_free_domain: + iommu_domain_free(domain); +out_unlock: + mutex_unlock(&iommu_sva_lock); + kfree(handle); + + return ERR_PTR(ret); +} +EXPORT_SYMBOL_GPL(iommu_sva_bind_device); + +/** + * iommu_sva_unbind_device() - Remove a bond created with iommu_sva_bind_d= evice + * @handle: the handle returned by iommu_sva_bind_device() + * + * Put reference to a bond between device and address space. The device sh= ould + * not be issuing any more transaction for this PASID. All outstanding page + * requests for this PASID must have been flushed to the IOMMU. + */ +void iommu_sva_unbind_device(struct iommu_sva *handle) +{ + struct iommu_domain *domain =3D handle->domain; + ioasid_t pasid =3D domain->mm->pasid; + struct device *dev =3D handle->dev; + + mutex_lock(&iommu_sva_lock); + if (--domain->users =3D=3D 0) { + iommu_detach_device_pasid(domain, dev, pasid); + iommu_domain_free(domain); + } + mutex_unlock(&iommu_sva_lock); + kfree(handle); +} +EXPORT_SYMBOL_GPL(iommu_sva_unbind_device); + +u32 iommu_sva_get_pasid(struct iommu_sva *handle) +{ + struct iommu_domain *domain =3D handle->domain; + + return domain->mm->pasid; +} +EXPORT_SYMBOL_GPL(iommu_sva_get_pasid); diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c index 6a1cd2018e30..c9da0a1bb3b8 100644 --- a/drivers/iommu/iommu.c +++ b/drivers/iommu/iommu.c @@ -2751,97 +2751,6 @@ int iommu_dev_disable_feature(struct device *dev, en= um iommu_dev_features feat) } EXPORT_SYMBOL_GPL(iommu_dev_disable_feature); =20 -/** - * iommu_sva_bind_device() - Bind a process address space to a device - * @dev: the device - * @mm: the mm to bind, caller must hold a reference to it - * - * Create a bond between device and address space, allowing the device to = access - * the mm using the returned PASID. If a bond already exists between @devi= ce and - * @mm, it is returned and an additional reference is taken. Caller must c= all - * iommu_sva_unbind_device() to release each reference. - * - * iommu_dev_enable_feature(dev, IOMMU_DEV_FEAT_SVA) must be called first,= to - * initialize the required SVA features. - * - * On error, returns an ERR_PTR value. - */ -struct iommu_sva * -iommu_sva_bind_device(struct device *dev, struct mm_struct *mm) -{ - struct iommu_group *group; - struct iommu_sva *handle =3D ERR_PTR(-EINVAL); - const struct iommu_ops *ops =3D dev_iommu_ops(dev); - - if (!ops->sva_bind) - return ERR_PTR(-ENODEV); - - group =3D iommu_group_get(dev); - if (!group) - return ERR_PTR(-ENODEV); - - /* Ensure device count and domain don't change while we're binding */ - mutex_lock(&group->mutex); - - /* - * To keep things simple, SVA currently doesn't support IOMMU groups - * with more than one device. Existing SVA-capable systems are not - * affected by the problems that required IOMMU groups (lack of ACS - * isolation, device ID aliasing and other hardware issues). - */ - if (iommu_group_device_count(group) !=3D 1) - goto out_unlock; - - handle =3D ops->sva_bind(dev, mm); - -out_unlock: - mutex_unlock(&group->mutex); - iommu_group_put(group); - - return handle; -} -EXPORT_SYMBOL_GPL(iommu_sva_bind_device); - -/** - * iommu_sva_unbind_device() - Remove a bond created with iommu_sva_bind_d= evice - * @handle: the handle returned by iommu_sva_bind_device() - * - * Put reference to a bond between device and address space. The device sh= ould - * not be issuing any more transaction for this PASID. All outstanding page - * requests for this PASID must have been flushed to the IOMMU. - */ -void iommu_sva_unbind_device(struct iommu_sva *handle) -{ - struct iommu_group *group; - struct device *dev =3D handle->dev; - const struct iommu_ops *ops =3D dev_iommu_ops(dev); - - if (!ops->sva_unbind) - return; - - group =3D iommu_group_get(dev); - if (!group) - return; - - mutex_lock(&group->mutex); - ops->sva_unbind(handle); - mutex_unlock(&group->mutex); - - iommu_group_put(group); -} -EXPORT_SYMBOL_GPL(iommu_sva_unbind_device); - -u32 iommu_sva_get_pasid(struct iommu_sva *handle) -{ - const struct iommu_ops *ops =3D dev_iommu_ops(handle->dev); - - if (!ops->sva_get_pasid) - return IOMMU_PASID_INVALID; - - return ops->sva_get_pasid(handle); -} -EXPORT_SYMBOL_GPL(iommu_sva_get_pasid); - /* * Changes the default domain of an iommu group that has *only* one device * --=20 2.34.1 From nobody Thu Apr 2 11:39:36 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 09E69FA373D for ; Mon, 31 Oct 2022 01:07:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229889AbiJaBHE (ORCPT ); Sun, 30 Oct 2022 21:07:04 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37254 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229799AbiJaBGi (ORCPT ); Sun, 30 Oct 2022 21:06:38 -0400 Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 275DEBF41 for ; Sun, 30 Oct 2022 18:06:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1667178383; x=1698714383; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=v2vrO1zHGU6UsNfICjQe4zlLzGSyshd5e+XGSMmwSJo=; b=A+5kqQpPHgrjlZKleEHcgYojxDGqjIdZfIgFzB0u3xyiil4nO1occkqC IumrnCSzdITOOdAxWYHsO1WHjbMgXRpGtttJHg13O7bENcu5ZdtkV+YQS xbcJBMbQ4Z1Xd2fnoZ9TLyiI+wyEqI1jCYBacAiY+M9fFi8kmiDS8w1mq 0Gpmu/8gUdUPUTmwJabRC8p2g2ZIsOuQlSwyu87cAfD2Ec8FUgBclpDp3 QSiSTYkigAV86Yi/Aamycua2UENLIV0i5PwSXOEhOP9ve9qNp4ReLhD8T c3ZdADm6j/qU3wPw/J2QNYg6sDJhZWIO+aeYTouv8Na/MtDgk3QQrIx1n Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10516"; a="373001711" X-IronPort-AV: E=Sophos;i="5.95,227,1661842800"; d="scan'208";a="373001711" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Oct 2022 18:06:22 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10516"; a="584496269" X-IronPort-AV: E=Sophos;i="5.95,227,1661842800"; d="scan'208";a="584496269" Received: from allen-box.sh.intel.com ([10.239.159.48]) by orsmga003.jf.intel.com with ESMTP; 30 Oct 2022 18:06:20 -0700 From: Lu Baolu To: Joerg Roedel Cc: Jason Gunthorpe , Kevin Tian , Will Deacon , Robin Murphy , Jean-Philippe Brucker , iommu@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [RESEND PATCH v14 10/13] iommu: Remove SVA related callbacks from iommu ops Date: Mon, 31 Oct 2022 08:59:14 +0800 Message-Id: <20221031005917.45690-11-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221031005917.45690-1-baolu.lu@linux.intel.com> References: <20221031005917.45690-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" These ops'es have been deprecated. There's no need for them anymore. Remove them to avoid dead code. Signed-off-by: Lu Baolu Reviewed-by: Jean-Philippe Brucker Reviewed-by: Kevin Tian Reviewed-by: Jason Gunthorpe Reviewed-by: Yi Liu Tested-by: Zhangfei Gao Tested-by: Tony Zhu --- include/linux/iommu.h | 7 --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 16 ------ drivers/iommu/intel/iommu.h | 3 -- .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 40 --------------- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 3 -- drivers/iommu/intel/iommu.c | 3 -- drivers/iommu/intel/svm.c | 49 ------------------- 7 files changed, 121 deletions(-) diff --git a/include/linux/iommu.h b/include/linux/iommu.h index bee5659d07eb..c337ef1c97bc 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -227,9 +227,6 @@ struct iommu_iotlb_gather { * driver init to device driver init (default no) * @dev_enable/disable_feat: per device entries to enable/disable * iommu specific features. - * @sva_bind: Bind process address space to device - * @sva_unbind: Unbind process address space from device - * @sva_get_pasid: Get PASID associated to a SVA handle * @page_response: handle page request response * @def_domain_type: device default domain type, return value: * - IOMMU_DOMAIN_IDENTITY: must use an identity domain @@ -263,10 +260,6 @@ struct iommu_ops { int (*dev_enable_feat)(struct device *dev, enum iommu_dev_features f); int (*dev_disable_feat)(struct device *dev, enum iommu_dev_features f); =20 - struct iommu_sva *(*sva_bind)(struct device *dev, struct mm_struct *mm); - void (*sva_unbind)(struct iommu_sva *handle); - u32 (*sva_get_pasid)(struct iommu_sva *handle); - int (*page_response)(struct device *dev, struct iommu_fault_event *evt, struct iommu_page_response *msg); diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/ar= m/arm-smmu-v3/arm-smmu-v3.h index 5aa853e98d38..8d772ea8a583 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -754,9 +754,6 @@ bool arm_smmu_master_sva_enabled(struct arm_smmu_master= *master); int arm_smmu_master_enable_sva(struct arm_smmu_master *master); int arm_smmu_master_disable_sva(struct arm_smmu_master *master); bool arm_smmu_master_iopf_supported(struct arm_smmu_master *master); -struct iommu_sva *arm_smmu_sva_bind(struct device *dev, struct mm_struct *= mm); -void arm_smmu_sva_unbind(struct iommu_sva *handle); -u32 arm_smmu_sva_get_pasid(struct iommu_sva *handle); void arm_smmu_sva_notifier_synchronize(void); struct iommu_domain *arm_smmu_sva_domain_alloc(void); void arm_smmu_sva_remove_dev_pasid(struct iommu_domain *domain, @@ -792,19 +789,6 @@ static inline bool arm_smmu_master_iopf_supported(stru= ct arm_smmu_master *master return false; } =20 -static inline struct iommu_sva * -arm_smmu_sva_bind(struct device *dev, struct mm_struct *mm) -{ - return ERR_PTR(-ENODEV); -} - -static inline void arm_smmu_sva_unbind(struct iommu_sva *handle) {} - -static inline u32 arm_smmu_sva_get_pasid(struct iommu_sva *handle) -{ - return IOMMU_PASID_INVALID; -} - static inline void arm_smmu_sva_notifier_synchronize(void) {} =20 static inline struct iommu_domain *arm_smmu_sva_domain_alloc(void) diff --git a/drivers/iommu/intel/iommu.h b/drivers/iommu/intel/iommu.h index 252fa344f88a..251a609fdce3 100644 --- a/drivers/iommu/intel/iommu.h +++ b/drivers/iommu/intel/iommu.h @@ -748,9 +748,6 @@ struct intel_iommu *device_to_iommu(struct device *dev,= u8 *bus, u8 *devfn); extern void intel_svm_check(struct intel_iommu *iommu); extern int intel_svm_enable_prq(struct intel_iommu *iommu); extern int intel_svm_finish_prq(struct intel_iommu *iommu); -struct iommu_sva *intel_svm_bind(struct device *dev, struct mm_struct *mm); -void intel_svm_unbind(struct iommu_sva *handle); -u32 intel_svm_get_pasid(struct iommu_sva *handle); int intel_svm_page_response(struct device *dev, struct iommu_fault_event *= evt, struct iommu_page_response *msg); struct iommu_domain *intel_svm_domain_alloc(void); diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iomm= u/arm/arm-smmu-v3/arm-smmu-v3-sva.c index 2d188d12419e..9541afbba73c 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -344,11 +344,6 @@ __arm_smmu_sva_bind(struct device *dev, struct mm_stru= ct *mm) if (!bond) return ERR_PTR(-ENOMEM); =20 - /* Allocate a PASID for this mm if necessary */ - ret =3D iommu_sva_alloc_pasid(mm, 1, (1U << master->ssid_bits) - 1); - if (ret) - goto err_free_bond; - bond->mm =3D mm; bond->sva.dev =3D dev; refcount_set(&bond->refs, 1); @@ -367,41 +362,6 @@ __arm_smmu_sva_bind(struct device *dev, struct mm_stru= ct *mm) return ERR_PTR(ret); } =20 -struct iommu_sva *arm_smmu_sva_bind(struct device *dev, struct mm_struct *= mm) -{ - struct iommu_sva *handle; - struct iommu_domain *domain =3D iommu_get_domain_for_dev(dev); - struct arm_smmu_domain *smmu_domain =3D to_smmu_domain(domain); - - if (smmu_domain->stage !=3D ARM_SMMU_DOMAIN_S1) - return ERR_PTR(-EINVAL); - - mutex_lock(&sva_lock); - handle =3D __arm_smmu_sva_bind(dev, mm); - mutex_unlock(&sva_lock); - return handle; -} - -void arm_smmu_sva_unbind(struct iommu_sva *handle) -{ - struct arm_smmu_bond *bond =3D sva_to_bond(handle); - - mutex_lock(&sva_lock); - if (refcount_dec_and_test(&bond->refs)) { - list_del(&bond->list); - arm_smmu_mmu_notifier_put(bond->smmu_mn); - kfree(bond); - } - mutex_unlock(&sva_lock); -} - -u32 arm_smmu_sva_get_pasid(struct iommu_sva *handle) -{ - struct arm_smmu_bond *bond =3D sva_to_bond(handle); - - return bond->mm->pasid; -} - bool arm_smmu_sva_supported(struct arm_smmu_device *smmu) { unsigned long reg, fld; diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/ar= m/arm-smmu-v3/arm-smmu-v3.c index eed2eb8effa3..891e87ea54db 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -2863,9 +2863,6 @@ static struct iommu_ops arm_smmu_ops =3D { .remove_dev_pasid =3D arm_smmu_remove_dev_pasid, .dev_enable_feat =3D arm_smmu_dev_enable_feature, .dev_disable_feat =3D arm_smmu_dev_disable_feature, - .sva_bind =3D arm_smmu_sva_bind, - .sva_unbind =3D arm_smmu_sva_unbind, - .sva_get_pasid =3D arm_smmu_sva_get_pasid, .page_response =3D arm_smmu_page_response, .def_domain_type =3D arm_smmu_def_domain_type, .pgsize_bitmap =3D -1UL, /* Restricted during device attach */ diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c index 7b67e431dd36..5a41b10593b7 100644 --- a/drivers/iommu/intel/iommu.c +++ b/drivers/iommu/intel/iommu.c @@ -4751,9 +4751,6 @@ const struct iommu_ops intel_iommu_ops =3D { .remove_dev_pasid =3D intel_iommu_remove_dev_pasid, .pgsize_bitmap =3D SZ_4K, #ifdef CONFIG_INTEL_IOMMU_SVM - .sva_bind =3D intel_svm_bind, - .sva_unbind =3D intel_svm_unbind, - .sva_get_pasid =3D intel_svm_get_pasid, .page_response =3D intel_svm_page_response, #endif .default_domain_ops =3D &(const struct iommu_domain_ops) { diff --git a/drivers/iommu/intel/svm.c b/drivers/iommu/intel/svm.c index 86c8ea0d9635..fceae9387018 100644 --- a/drivers/iommu/intel/svm.c +++ b/drivers/iommu/intel/svm.c @@ -296,14 +296,6 @@ static int pasid_to_svm_sdev(struct device *dev, unsig= ned int pasid, return 0; } =20 -static int intel_svm_alloc_pasid(struct device *dev, struct mm_struct *mm) -{ - ioasid_t max_pasid =3D dev_is_pci(dev) ? - pci_max_pasids(to_pci_dev(dev)) : intel_pasid_max_id; - - return iommu_sva_alloc_pasid(mm, PASID_MIN, max_pasid - 1); -} - static struct iommu_sva *intel_svm_bind_mm(struct intel_iommu *iommu, struct device *dev, struct mm_struct *mm) @@ -771,47 +763,6 @@ static irqreturn_t prq_event_thread(int irq, void *d) return IRQ_RETVAL(handled); } =20 -struct iommu_sva *intel_svm_bind(struct device *dev, struct mm_struct *mm) -{ - struct intel_iommu *iommu =3D device_to_iommu(dev, NULL, NULL); - struct iommu_sva *sva; - int ret; - - mutex_lock(&pasid_mutex); - ret =3D intel_svm_alloc_pasid(dev, mm); - if (ret) { - mutex_unlock(&pasid_mutex); - return ERR_PTR(ret); - } - - sva =3D intel_svm_bind_mm(iommu, dev, mm); - mutex_unlock(&pasid_mutex); - - return sva; -} - -void intel_svm_unbind(struct iommu_sva *sva) -{ - struct intel_svm_dev *sdev =3D to_intel_svm_dev(sva); - - mutex_lock(&pasid_mutex); - intel_svm_unbind_mm(sdev->dev, sdev->pasid); - mutex_unlock(&pasid_mutex); -} - -u32 intel_svm_get_pasid(struct iommu_sva *sva) -{ - struct intel_svm_dev *sdev; - u32 pasid; - - mutex_lock(&pasid_mutex); - sdev =3D to_intel_svm_dev(sva); - pasid =3D sdev->pasid; - mutex_unlock(&pasid_mutex); - - return pasid; -} - int intel_svm_page_response(struct device *dev, struct iommu_fault_event *evt, struct iommu_page_response *msg) --=20 2.34.1 From nobody Thu Apr 2 11:39:36 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BD28DFA373D for ; Mon, 31 Oct 2022 01:07:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229915AbiJaBHP (ORCPT ); Sun, 30 Oct 2022 21:07:15 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36594 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229906AbiJaBGm (ORCPT ); Sun, 30 Oct 2022 21:06:42 -0400 Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9554DBF68 for ; Sun, 30 Oct 2022 18:06:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1667178385; x=1698714385; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=DhtYeSNqUAaAkHikDPzpgpLKsdlOqzZmLs+oo9yKc0M=; b=iw2rli/b932T20fiQDZy46OEgVEqe/lqnp5SCML59xexMtHZJ9+oKlFR Pw4YN/id/R+Y6K9XMqDRIZHkyu9rx+JOmfOVoipBbZ6d+nNw0GPsr4m3M CvaZYM6JuifG2Z+HXOS6QF+IkHdLFg6Gnk3s04FpRPYkpK2u264lTbQBO ekn3YOzeGpU583VPsWgwLFVU+IkKTuMls2FU3FwSlSQSZKSrGa8VVLoi5 ruMmN5h1e2syIrd/LY27E5oSep2vjFB/yUP0nxesoLfwBLJSSvX6ynnFC AKMi9JfvLG9yowdt7+in9cpv0tMDYNUUdfIgyJgrJciAeRM2d6Get6Q9R w==; X-IronPort-AV: E=McAfee;i="6500,9779,10516"; a="373001716" X-IronPort-AV: E=Sophos;i="5.95,227,1661842800"; d="scan'208";a="373001716" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Oct 2022 18:06:25 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10516"; a="584496279" X-IronPort-AV: E=Sophos;i="5.95,227,1661842800"; d="scan'208";a="584496279" Received: from allen-box.sh.intel.com ([10.239.159.48]) by orsmga003.jf.intel.com with ESMTP; 30 Oct 2022 18:06:23 -0700 From: Lu Baolu To: Joerg Roedel Cc: Jason Gunthorpe , Kevin Tian , Will Deacon , Robin Murphy , Jean-Philippe Brucker , iommu@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [RESEND PATCH v14 11/13] iommu: Prepare IOMMU domain for IOPF Date: Mon, 31 Oct 2022 08:59:15 +0800 Message-Id: <20221031005917.45690-12-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221031005917.45690-1-baolu.lu@linux.intel.com> References: <20221031005917.45690-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" This adds some mechanisms around the iommu_domain so that the I/O page fault handling framework could route a page fault to the domain and call the fault handler from it. Add pointers to the page fault handler and its private data in struct iommu_domain. The fault handler will be called with the private data as a parameter once a page fault is routed to the domain. Any kernel component which owns an iommu domain could install handler and its private parameter so that the page fault could be further routed and handled. This also prepares the SVA implementation to be the first consumer of the per-domain page fault handling model. The I/O page fault handler for SVA is copied to the SVA file with mmget_not_zero() added before mmap_read_lock(). Suggested-by: Jean-Philippe Brucker Signed-off-by: Lu Baolu Reviewed-by: Jean-Philippe Brucker Reviewed-by: Kevin Tian Reviewed-by: Jason Gunthorpe Tested-by: Zhangfei Gao Tested-by: Tony Zhu --- include/linux/iommu.h | 3 ++ drivers/iommu/iommu-sva-lib.h | 8 +++++ drivers/iommu/io-pgfault.c | 7 +++++ drivers/iommu/iommu-sva-lib.c | 58 +++++++++++++++++++++++++++++++++++ drivers/iommu/iommu.c | 4 +++ 5 files changed, 80 insertions(+) diff --git a/include/linux/iommu.h b/include/linux/iommu.h index c337ef1c97bc..7d2648058e43 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -98,6 +98,9 @@ struct iommu_domain { unsigned long pgsize_bitmap; /* Bitmap of page sizes in use */ struct iommu_domain_geometry geometry; struct iommu_dma_cookie *iova_cookie; + enum iommu_page_response_code (*iopf_handler)(struct iommu_fault *fault, + void *data); + void *fault_data; union { struct { iommu_fault_handler_t handler; diff --git a/drivers/iommu/iommu-sva-lib.h b/drivers/iommu/iommu-sva-lib.h index 8909ea1094e3..1b3ace4b5863 100644 --- a/drivers/iommu/iommu-sva-lib.h +++ b/drivers/iommu/iommu-sva-lib.h @@ -26,6 +26,8 @@ int iopf_queue_flush_dev(struct device *dev); struct iopf_queue *iopf_queue_alloc(const char *name); void iopf_queue_free(struct iopf_queue *queue); int iopf_queue_discard_partial(struct iopf_queue *queue); +enum iommu_page_response_code +iommu_sva_handle_iopf(struct iommu_fault *fault, void *data); =20 #else /* CONFIG_IOMMU_SVA */ static inline int iommu_queue_iopf(struct iommu_fault *fault, void *cookie) @@ -63,5 +65,11 @@ static inline int iopf_queue_discard_partial(struct iopf= _queue *queue) { return -ENODEV; } + +static inline enum iommu_page_response_code +iommu_sva_handle_iopf(struct iommu_fault *fault, void *data) +{ + return IOMMU_PAGE_RESP_INVALID; +} #endif /* CONFIG_IOMMU_SVA */ #endif /* _IOMMU_SVA_LIB_H */ diff --git a/drivers/iommu/io-pgfault.c b/drivers/iommu/io-pgfault.c index 1df8c1dcae77..aee9e033012f 100644 --- a/drivers/iommu/io-pgfault.c +++ b/drivers/iommu/io-pgfault.c @@ -181,6 +181,13 @@ static void iopf_handle_group(struct work_struct *work) * request completes, outstanding faults will have been dealt with by the = time * the PASID is freed. * + * Any valid page fault will be eventually routed to an iommu domain and t= he + * page fault handler installed there will get called. The users of this + * handling framework should guarantee that the iommu domain could only be + * freed after the device has stopped generating page faults (or the iommu + * hardware has been set to block the page faults) and the pending page fa= ults + * have been flushed. + * * Return: 0 on success and <0 on error. */ int iommu_queue_iopf(struct iommu_fault *fault, void *cookie) diff --git a/drivers/iommu/iommu-sva-lib.c b/drivers/iommu/iommu-sva-lib.c index e425573a1787..089fd61ff453 100644 --- a/drivers/iommu/iommu-sva-lib.c +++ b/drivers/iommu/iommu-sva-lib.c @@ -180,3 +180,61 @@ u32 iommu_sva_get_pasid(struct iommu_sva *handle) return domain->mm->pasid; } EXPORT_SYMBOL_GPL(iommu_sva_get_pasid); + +/* + * I/O page fault handler for SVA + */ +enum iommu_page_response_code +iommu_sva_handle_iopf(struct iommu_fault *fault, void *data) +{ + vm_fault_t ret; + struct vm_area_struct *vma; + struct mm_struct *mm =3D data; + unsigned int access_flags =3D 0; + unsigned int fault_flags =3D FAULT_FLAG_REMOTE; + struct iommu_fault_page_request *prm =3D &fault->prm; + enum iommu_page_response_code status =3D IOMMU_PAGE_RESP_INVALID; + + if (!(prm->flags & IOMMU_FAULT_PAGE_REQUEST_PASID_VALID)) + return status; + + if (!mmget_not_zero(mm)) + return status; + + mmap_read_lock(mm); + + vma =3D find_extend_vma(mm, prm->addr); + if (!vma) + /* Unmapped area */ + goto out_put_mm; + + if (prm->perm & IOMMU_FAULT_PERM_READ) + access_flags |=3D VM_READ; + + if (prm->perm & IOMMU_FAULT_PERM_WRITE) { + access_flags |=3D VM_WRITE; + fault_flags |=3D FAULT_FLAG_WRITE; + } + + if (prm->perm & IOMMU_FAULT_PERM_EXEC) { + access_flags |=3D VM_EXEC; + fault_flags |=3D FAULT_FLAG_INSTRUCTION; + } + + if (!(prm->perm & IOMMU_FAULT_PERM_PRIV)) + fault_flags |=3D FAULT_FLAG_USER; + + if (access_flags & ~vma->vm_flags) + /* Access fault */ + goto out_put_mm; + + ret =3D handle_mm_fault(vma, prm->addr, fault_flags, NULL); + status =3D ret & VM_FAULT_ERROR ? IOMMU_PAGE_RESP_INVALID : + IOMMU_PAGE_RESP_SUCCESS; + +out_put_mm: + mmap_read_unlock(mm); + mmput(mm); + + return status; +} diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c index c9da0a1bb3b8..9e0fb18e1b34 100644 --- a/drivers/iommu/iommu.c +++ b/drivers/iommu/iommu.c @@ -33,6 +33,8 @@ =20 #include "dma-iommu.h" =20 +#include "iommu-sva-lib.h" + static struct kset *iommu_group_kset; static DEFINE_IDA(iommu_group_ida); =20 @@ -3309,6 +3311,8 @@ struct iommu_domain *iommu_sva_domain_alloc(struct de= vice *dev, domain->type =3D IOMMU_DOMAIN_SVA; mmgrab(mm); domain->mm =3D mm; + domain->iopf_handler =3D iommu_sva_handle_iopf; + domain->fault_data =3D mm; =20 return domain; } --=20 2.34.1 From nobody Thu Apr 2 11:39:36 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 57E6FC38A02 for ; Mon, 31 Oct 2022 01:07:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229956AbiJaBHS (ORCPT ); Sun, 30 Oct 2022 21:07:18 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36976 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229919AbiJaBGp (ORCPT ); Sun, 30 Oct 2022 21:06:45 -0400 Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EC8E9B7DC for ; Sun, 30 Oct 2022 18:06:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1667178387; x=1698714387; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=kqU6scITSfUNs+6xn3Wc3tcykXoMaxuv7CM9PVOBKyk=; b=MFFs+fBTvtuTVRvDZjBbdK2V2S2Sv0HbNyxQ541uQ5l/VW5jMz22QShM b5bQHBe0zV19bPnSu/78x9vZ1qvs7AEI9X9NAksFsXh9iwpFUSSOzkBqA OI9LhAfAXGg5IpHe/8ZUNc+xiQfbMp1nQemLs+o+dFhgtZF2ij+O+neaj 7/Hixby8viCnnBpWQRgcMHHvS9hqqTicMTmeN13V4B8Eq1IyB1DhRQ3Qs BtK6YC4DZLEnZqsTmGFywkHEOAsWyW7nNSAAkdFIuo60VpOICm4mpLrjP TKW0lAt5x+VDRY27bkn9QnQPpGJJ6kLU5fxOgO6mscrH3XvlHMFDFJ4Mx w==; X-IronPort-AV: E=McAfee;i="6500,9779,10516"; a="373001723" X-IronPort-AV: E=Sophos;i="5.95,227,1661842800"; d="scan'208";a="373001723" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Oct 2022 18:06:27 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10516"; a="584496292" X-IronPort-AV: E=Sophos;i="5.95,227,1661842800"; d="scan'208";a="584496292" Received: from allen-box.sh.intel.com ([10.239.159.48]) by orsmga003.jf.intel.com with ESMTP; 30 Oct 2022 18:06:25 -0700 From: Lu Baolu To: Joerg Roedel Cc: Jason Gunthorpe , Kevin Tian , Will Deacon , Robin Murphy , Jean-Philippe Brucker , iommu@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [RESEND PATCH v14 12/13] iommu: Per-domain I/O page fault handling Date: Mon, 31 Oct 2022 08:59:16 +0800 Message-Id: <20221031005917.45690-13-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221031005917.45690-1-baolu.lu@linux.intel.com> References: <20221031005917.45690-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Tweak the I/O page fault handling framework to route the page faults to the domain and call the page fault handler retrieved from the domain. This makes the I/O page fault handling framework possible to serve more usage scenarios as long as they have an IOMMU domain and install a page fault handler in it. Some unused functions are also removed to avoid dead code. The iommu_get_domain_for_dev_pasid() which retrieves attached domain for a {device, PASID} pair is used. It will be used by the page fault handling framework which knows {device, PASID} reported from the iommu driver. We have a guarantee that the SVA domain doesn't go away during IOPF handling, because unbind() won't free the domain until all the pending page requests have been flushed from the pipeline. The drivers either call iopf_queue_flush_dev() explicitly, or in stall case, the device driver is required to flush all DMAs including stalled transactions before calling unbind(). This also renames iopf_handle_group() to iopf_handler() to avoid confusing. Signed-off-by: Lu Baolu Reviewed-by: Jean-Philippe Brucker Reviewed-by: Kevin Tian Reviewed-by: Jason Gunthorpe Tested-by: Zhangfei Gao Tested-by: Tony Zhu --- drivers/iommu/io-pgfault.c | 68 +++++--------------------------------- 1 file changed, 9 insertions(+), 59 deletions(-) diff --git a/drivers/iommu/io-pgfault.c b/drivers/iommu/io-pgfault.c index aee9e033012f..d046d89cec55 100644 --- a/drivers/iommu/io-pgfault.c +++ b/drivers/iommu/io-pgfault.c @@ -69,69 +69,18 @@ static int iopf_complete_group(struct device *dev, stru= ct iopf_fault *iopf, return iommu_page_response(dev, &resp); } =20 -static enum iommu_page_response_code -iopf_handle_single(struct iopf_fault *iopf) -{ - vm_fault_t ret; - struct mm_struct *mm; - struct vm_area_struct *vma; - unsigned int access_flags =3D 0; - unsigned int fault_flags =3D FAULT_FLAG_REMOTE; - struct iommu_fault_page_request *prm =3D &iopf->fault.prm; - enum iommu_page_response_code status =3D IOMMU_PAGE_RESP_INVALID; - - if (!(prm->flags & IOMMU_FAULT_PAGE_REQUEST_PASID_VALID)) - return status; - - mm =3D iommu_sva_find(prm->pasid); - if (IS_ERR_OR_NULL(mm)) - return status; - - mmap_read_lock(mm); - - vma =3D find_extend_vma(mm, prm->addr); - if (!vma) - /* Unmapped area */ - goto out_put_mm; - - if (prm->perm & IOMMU_FAULT_PERM_READ) - access_flags |=3D VM_READ; - - if (prm->perm & IOMMU_FAULT_PERM_WRITE) { - access_flags |=3D VM_WRITE; - fault_flags |=3D FAULT_FLAG_WRITE; - } - - if (prm->perm & IOMMU_FAULT_PERM_EXEC) { - access_flags |=3D VM_EXEC; - fault_flags |=3D FAULT_FLAG_INSTRUCTION; - } - - if (!(prm->perm & IOMMU_FAULT_PERM_PRIV)) - fault_flags |=3D FAULT_FLAG_USER; - - if (access_flags & ~vma->vm_flags) - /* Access fault */ - goto out_put_mm; - - ret =3D handle_mm_fault(vma, prm->addr, fault_flags, NULL); - status =3D ret & VM_FAULT_ERROR ? IOMMU_PAGE_RESP_INVALID : - IOMMU_PAGE_RESP_SUCCESS; - -out_put_mm: - mmap_read_unlock(mm); - mmput(mm); - - return status; -} - -static void iopf_handle_group(struct work_struct *work) +static void iopf_handler(struct work_struct *work) { struct iopf_group *group; + struct iommu_domain *domain; struct iopf_fault *iopf, *next; enum iommu_page_response_code status =3D IOMMU_PAGE_RESP_SUCCESS; =20 group =3D container_of(work, struct iopf_group, work); + domain =3D iommu_get_domain_for_dev_pasid(group->dev, + group->last_fault.fault.prm.pasid, 0); + if (!domain || !domain->iopf_handler) + status =3D IOMMU_PAGE_RESP_INVALID; =20 list_for_each_entry_safe(iopf, next, &group->faults, list) { /* @@ -139,7 +88,8 @@ static void iopf_handle_group(struct work_struct *work) * faults in the group if there is an error. */ if (status =3D=3D IOMMU_PAGE_RESP_SUCCESS) - status =3D iopf_handle_single(iopf); + status =3D domain->iopf_handler(&iopf->fault, + domain->fault_data); =20 if (!(iopf->fault.prm.flags & IOMMU_FAULT_PAGE_REQUEST_LAST_PAGE)) @@ -242,7 +192,7 @@ int iommu_queue_iopf(struct iommu_fault *fault, void *c= ookie) group->last_fault.fault =3D *fault; INIT_LIST_HEAD(&group->faults); list_add(&group->last_fault.list, &group->faults); - INIT_WORK(&group->work, iopf_handle_group); + INIT_WORK(&group->work, iopf_handler); =20 /* See if we have partial faults for this group */ list_for_each_entry_safe(iopf, next, &iopf_param->partial, list) { --=20 2.34.1 From nobody Thu Apr 2 11:39:36 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EE539FA373D for ; Mon, 31 Oct 2022 01:07:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229717AbiJaBH0 (ORCPT ); Sun, 30 Oct 2022 21:07:26 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37216 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229786AbiJaBGw (ORCPT ); Sun, 30 Oct 2022 21:06:52 -0400 Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 49CD9C746 for ; Sun, 30 Oct 2022 18:06:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1667178390; x=1698714390; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=SlITOTwQPXc6NRO9eB+eaUJaWMyo11OeGNe/DYXmsHw=; b=ANSIZGVKiNDlQ122wurzXrY2FIyFVkawvyuB4PhU7p5GFHPf/Om6wAmM WMTeDoOBREDSD67tjcFClV+pYbU1ngLBwVH4GzQN2D2dHcConUDgjZBVh 8+io+4hcXIrug5Y9Kggo126QCS3gQ990ibXROH1IV7NN8jcZ6lKNAjFyF TKrvhALy4UFClYzhWfS1NZ6YL+YjdcT42teHuamMHCn5umlDc/ySRARSb fIl1UPz5FFJ2KOtbOXDkz1XVeFV/lobE+XM9zQcngHXhvg1DUa2YsxDsa h+2vjVgufm+9KmVyrM4M0y1AFrM0EosNB0DwVP5Ler4YjxomU4pOGRZH0 A==; X-IronPort-AV: E=McAfee;i="6500,9779,10516"; a="373001726" X-IronPort-AV: E=Sophos;i="5.95,227,1661842800"; d="scan'208";a="373001726" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Oct 2022 18:06:30 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10516"; a="584496312" X-IronPort-AV: E=Sophos;i="5.95,227,1661842800"; d="scan'208";a="584496312" Received: from allen-box.sh.intel.com ([10.239.159.48]) by orsmga003.jf.intel.com with ESMTP; 30 Oct 2022 18:06:27 -0700 From: Lu Baolu To: Joerg Roedel Cc: Jason Gunthorpe , Kevin Tian , Will Deacon , Robin Murphy , Jean-Philippe Brucker , iommu@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [RESEND PATCH v14 13/13] iommu: Rename iommu-sva-lib.{c,h} Date: Mon, 31 Oct 2022 08:59:17 +0800 Message-Id: <20221031005917.45690-14-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221031005917.45690-1-baolu.lu@linux.intel.com> References: <20221031005917.45690-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Rename iommu-sva-lib.c[h] to iommu-sva.c[h] as it contains all code for SVA implementation in iommu core. Signed-off-by: Lu Baolu Reviewed-by: Jean-Philippe Brucker Reviewed-by: Kevin Tian Reviewed-by: Jason Gunthorpe Tested-by: Zhangfei Gao Tested-by: Tony Zhu --- drivers/iommu/{iommu-sva-lib.h =3D> iommu-sva.h} | 6 +++--- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 2 +- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 2 +- drivers/iommu/intel/iommu.c | 2 +- drivers/iommu/intel/svm.c | 2 +- drivers/iommu/io-pgfault.c | 2 +- drivers/iommu/{iommu-sva-lib.c =3D> iommu-sva.c} | 2 +- drivers/iommu/iommu.c | 2 +- drivers/iommu/Makefile | 2 +- 9 files changed, 11 insertions(+), 11 deletions(-) rename drivers/iommu/{iommu-sva-lib.h =3D> iommu-sva.h} (95%) rename drivers/iommu/{iommu-sva-lib.c =3D> iommu-sva.c} (99%) diff --git a/drivers/iommu/iommu-sva-lib.h b/drivers/iommu/iommu-sva.h similarity index 95% rename from drivers/iommu/iommu-sva-lib.h rename to drivers/iommu/iommu-sva.h index 1b3ace4b5863..7215a761b962 100644 --- a/drivers/iommu/iommu-sva-lib.h +++ b/drivers/iommu/iommu-sva.h @@ -2,8 +2,8 @@ /* * SVA library for IOMMU drivers */ -#ifndef _IOMMU_SVA_LIB_H -#define _IOMMU_SVA_LIB_H +#ifndef _IOMMU_SVA_H +#define _IOMMU_SVA_H =20 #include #include @@ -72,4 +72,4 @@ iommu_sva_handle_iopf(struct iommu_fault *fault, void *da= ta) return IOMMU_PAGE_RESP_INVALID; } #endif /* CONFIG_IOMMU_SVA */ -#endif /* _IOMMU_SVA_LIB_H */ +#endif /* _IOMMU_SVA_H */ diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iomm= u/arm/arm-smmu-v3/arm-smmu-v3-sva.c index 9541afbba73c..a5a63b1c947e 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -10,7 +10,7 @@ #include =20 #include "arm-smmu-v3.h" -#include "../../iommu-sva-lib.h" +#include "../../iommu-sva.h" #include "../../io-pgtable-arm.h" =20 struct arm_smmu_mmu_notifier { diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/ar= m/arm-smmu-v3/arm-smmu-v3.c index 891e87ea54db..94a2e53368af 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -29,7 +29,7 @@ =20 #include "arm-smmu-v3.h" #include "../../dma-iommu.h" -#include "../../iommu-sva-lib.h" +#include "../../iommu-sva.h" =20 static bool disable_bypass =3D true; module_param(disable_bypass, bool, 0444); diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c index 5a41b10593b7..a934a46bb9e6 100644 --- a/drivers/iommu/intel/iommu.c +++ b/drivers/iommu/intel/iommu.c @@ -27,7 +27,7 @@ #include "iommu.h" #include "../dma-iommu.h" #include "../irq_remapping.h" -#include "../iommu-sva-lib.h" +#include "../iommu-sva.h" #include "pasid.h" #include "cap_audit.h" =20 diff --git a/drivers/iommu/intel/svm.c b/drivers/iommu/intel/svm.c index fceae9387018..f32de15da61a 100644 --- a/drivers/iommu/intel/svm.c +++ b/drivers/iommu/intel/svm.c @@ -24,7 +24,7 @@ #include "iommu.h" #include "pasid.h" #include "perf.h" -#include "../iommu-sva-lib.h" +#include "../iommu-sva.h" #include "trace.h" =20 static irqreturn_t prq_event_thread(int irq, void *d); diff --git a/drivers/iommu/io-pgfault.c b/drivers/iommu/io-pgfault.c index d046d89cec55..e5b8b9110c13 100644 --- a/drivers/iommu/io-pgfault.c +++ b/drivers/iommu/io-pgfault.c @@ -11,7 +11,7 @@ #include #include =20 -#include "iommu-sva-lib.h" +#include "iommu-sva.h" =20 /** * struct iopf_queue - IO Page Fault queue diff --git a/drivers/iommu/iommu-sva-lib.c b/drivers/iommu/iommu-sva.c similarity index 99% rename from drivers/iommu/iommu-sva-lib.c rename to drivers/iommu/iommu-sva.c index 089fd61ff453..24bf9b2b58aa 100644 --- a/drivers/iommu/iommu-sva-lib.c +++ b/drivers/iommu/iommu-sva.c @@ -6,7 +6,7 @@ #include #include =20 -#include "iommu-sva-lib.h" +#include "iommu-sva.h" =20 static DEFINE_MUTEX(iommu_sva_lock); static DECLARE_IOASID_SET(iommu_sva_pasid); diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c index 9e0fb18e1b34..c50f68b2b656 100644 --- a/drivers/iommu/iommu.c +++ b/drivers/iommu/iommu.c @@ -33,7 +33,7 @@ =20 #include "dma-iommu.h" =20 -#include "iommu-sva-lib.h" +#include "iommu-sva.h" =20 static struct kset *iommu_group_kset; static DEFINE_IDA(iommu_group_ida); diff --git a/drivers/iommu/Makefile b/drivers/iommu/Makefile index cc9f381013c3..7fbf6a337662 100644 --- a/drivers/iommu/Makefile +++ b/drivers/iommu/Makefile @@ -28,6 +28,6 @@ obj-$(CONFIG_FSL_PAMU) +=3D fsl_pamu.o fsl_pamu_domain.o obj-$(CONFIG_S390_IOMMU) +=3D s390-iommu.o obj-$(CONFIG_HYPERV_IOMMU) +=3D hyperv-iommu.o obj-$(CONFIG_VIRTIO_IOMMU) +=3D virtio-iommu.o -obj-$(CONFIG_IOMMU_SVA) +=3D iommu-sva-lib.o io-pgfault.o +obj-$(CONFIG_IOMMU_SVA) +=3D iommu-sva.o io-pgfault.o obj-$(CONFIG_SPRD_IOMMU) +=3D sprd-iommu.o obj-$(CONFIG_APPLE_DART) +=3D apple-dart.o --=20 2.34.1