From nobody Sun Apr 19 09:04:37 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0A965C433EF for ; Tue, 5 Jul 2022 05:11:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229621AbiGEFLs (ORCPT ); Tue, 5 Jul 2022 01:11:48 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37882 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229615AbiGEFLp (ORCPT ); Tue, 5 Jul 2022 01:11:45 -0400 Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3B65012624 for ; Mon, 4 Jul 2022 22:11:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1656997904; x=1688533904; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Qo+6klAfeW414OJoIRT3UwYdgTWk/xVjBH9Ql67rjGo=; b=dIQWK04MrCJlAvvjQ1v0k82mmYd61A449/3V24fcAqoFsigGmCXQz1/9 YGG4wkiT/NcgqQOlNhjEwoGfP4QbH3OlEBPmVoVXsCebHEyHoNB4avtCT pChWXv+tpEp0hBZWgkH7SeecEn8tRVpik6lY1eisRpb7JzvUmlX+SKgX1 SPmAmB/+GbHQoKTSDOEwJVeteEkSL/EgaMmm5eOqhP+vFPjdw8bXVB+F4 nnDqmCc3IGQPRjtO633Uw8i6ImefOtlDSLCBQQccfjaPXAdn/ljfoshuX JFvvZx247vIjhqPZiak4b3gqvXof5bjQWa9NtiZFIlEYaeTaluhxhAFIE w==; X-IronPort-AV: E=McAfee;i="6400,9594,10398"; a="272029370" X-IronPort-AV: E=Sophos;i="5.92,245,1650956400"; d="scan'208";a="272029370" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Jul 2022 22:11:44 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.92,245,1650956400"; d="scan'208";a="542807360" Received: from allen-box.sh.intel.com ([10.239.159.48]) by orsmga003.jf.intel.com with ESMTP; 04 Jul 2022 22:11:39 -0700 From: Lu Baolu To: Joerg Roedel , Jason Gunthorpe , Christoph Hellwig , Kevin Tian , Ashok Raj , Will Deacon , Robin Murphy , Jean-Philippe Brucker , Dave Jiang , Vinod Koul Cc: Eric Auger , Liu Yi L , Jacob jun Pan , Zhangfei Gao , Zhu Tony , iommu@lists.linux.dev, linux-kernel@vger.kernel.org, Lu Baolu , Jean-Philippe Brucker Subject: [PATCH v10 01/12] iommu: Add max_pasids field in struct iommu_device Date: Tue, 5 Jul 2022 13:06:59 +0800 Message-Id: <20220705050710.2887204-2-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220705050710.2887204-1-baolu.lu@linux.intel.com> References: <20220705050710.2887204-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Use this field to keep the number of supported PASIDs that an IOMMU hardware is able to support. This is a generic attribute of an IOMMU and lifting it into the per-IOMMU device structure makes it possible to allocate a PASID for device without calls into the IOMMU drivers. Any iommu driver that supports PASID related features should set this field before enabling them on the devices. In the Intel IOMMU driver, intel_iommu_sm is moved to CONFIG_INTEL_IOMMU enclave so that the pasid_supported() helper could be used in dmar.c without compilation errors. Signed-off-by: Lu Baolu Reviewed-by: Jean-Philippe Brucker Reviewed-by: Kevin Tian Tested-by: Zhangfei Gao Tested-by: Tony Zhu Reviewed-by: Jason Gunthorpe Reviewed-by: Yi Liu --- include/linux/intel-iommu.h | 3 ++- include/linux/iommu.h | 2 ++ drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 1 + drivers/iommu/intel/dmar.c | 7 +++++++ 4 files changed, 12 insertions(+), 1 deletion(-) diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h index 4f29139bbfc3..e065cbe3c857 100644 --- a/include/linux/intel-iommu.h +++ b/include/linux/intel-iommu.h @@ -479,7 +479,6 @@ enum { #define VTD_FLAG_IRQ_REMAP_PRE_ENABLED (1 << 1) #define VTD_FLAG_SVM_CAPABLE (1 << 2) =20 -extern int intel_iommu_sm; extern spinlock_t device_domain_lock; =20 #define sm_supported(iommu) (intel_iommu_sm && ecap_smts((iommu)->ecap)) @@ -786,6 +785,7 @@ struct context_entry *iommu_context_addr(struct intel_i= ommu *iommu, u8 bus, extern const struct iommu_ops intel_iommu_ops; =20 #ifdef CONFIG_INTEL_IOMMU +extern int intel_iommu_sm; extern int iommu_calculate_agaw(struct intel_iommu *iommu); extern int iommu_calculate_max_sagaw(struct intel_iommu *iommu); extern int dmar_disabled; @@ -802,6 +802,7 @@ static inline int iommu_calculate_max_sagaw(struct inte= l_iommu *iommu) } #define dmar_disabled (1) #define intel_iommu_enabled (0) +#define intel_iommu_sm (0) #endif =20 static inline const char *decode_prq_descriptor(char *str, size_t size, diff --git a/include/linux/iommu.h b/include/linux/iommu.h index 5e1afe169549..03fbb1b71536 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -318,12 +318,14 @@ struct iommu_domain_ops { * @list: Used by the iommu-core to keep a list of registered iommus * @ops: iommu-ops for talking to this iommu * @dev: struct device for sysfs handling + * @max_pasids: number of supported PASIDs */ struct iommu_device { struct list_head list; const struct iommu_ops *ops; struct fwnode_handle *fwnode; struct device *dev; + u32 max_pasids; }; =20 /** diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/ar= m/arm-smmu-v3/arm-smmu-v3.c index 88817a3376ef..ae8ec8df47c1 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -3546,6 +3546,7 @@ static int arm_smmu_device_hw_probe(struct arm_smmu_d= evice *smmu) /* SID/SSID sizes */ smmu->ssid_bits =3D FIELD_GET(IDR1_SSIDSIZE, reg); smmu->sid_bits =3D FIELD_GET(IDR1_SIDSIZE, reg); + smmu->iommu.max_pasids =3D 1UL << smmu->ssid_bits; =20 /* * If the SMMU supports fewer bits than would fill a single L2 stream diff --git a/drivers/iommu/intel/dmar.c b/drivers/iommu/intel/dmar.c index 592c1e1a5d4b..6c338888061a 100644 --- a/drivers/iommu/intel/dmar.c +++ b/drivers/iommu/intel/dmar.c @@ -1123,6 +1123,13 @@ static int alloc_iommu(struct dmar_drhd_unit *drhd) =20 raw_spin_lock_init(&iommu->register_lock); =20 + /* + * A value of N in PSS field of eCap register indicates hardware + * supports PASID field of N+1 bits. + */ + if (pasid_supported(iommu)) + iommu->iommu.max_pasids =3D 2UL << ecap_pss(iommu->ecap); + /* * This is only for hotplug; at boot time intel_iommu_enabled won't * be set yet. When intel_iommu_init() runs, it registers the units --=20 2.25.1 From nobody Sun Apr 19 09:04:37 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E4BDCC433EF for ; Tue, 5 Jul 2022 05:12:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229687AbiGEFMS (ORCPT ); Tue, 5 Jul 2022 01:12:18 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38118 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229543AbiGEFMD (ORCPT ); Tue, 5 Jul 2022 01:12:03 -0400 Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F1F8313D15 for ; Mon, 4 Jul 2022 22:12:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1656997923; x=1688533923; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=AOuJcckSmEJG/2YCRrXp2jo9koUTopyV3uPrKxZRKYg=; b=iknNbGjFXoxfYWTTRDfKAqlPDLCAyoRCJKWRyMPxNQ9AOIpvXK9u6zPe slgsH76964OeI7CJ2ugN+qt5SDYkZGfSuk/+EwM9oYAxZWYasDOuI6kcg TufY5Jns7ql48MShiZn6ZachEOINr95pHSYI73Tb/98FmGLL1EFKnlrAx rvvUAn9n0oa4lqNBLtu0//qsn7iUMz8U65ZtE3IwjQekBh/iGFOgOX6NQ APulH21Pil/TvPeALb43zxYAXn6MHTXhSMn6kJoGJU+jfe6ALxiRPFd1D evDdr6021PhOkbH6H95c4GU9pJhfH3xvn7rbwkT337LeFH42BTu8UL/0G w==; X-IronPort-AV: E=McAfee;i="6400,9594,10398"; a="272029380" X-IronPort-AV: E=Sophos;i="5.92,245,1650956400"; d="scan'208";a="272029380" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Jul 2022 22:11:48 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.92,245,1650956400"; d="scan'208";a="542807370" Received: from allen-box.sh.intel.com ([10.239.159.48]) by orsmga003.jf.intel.com with ESMTP; 04 Jul 2022 22:11:44 -0700 From: Lu Baolu To: Joerg Roedel , Jason Gunthorpe , Christoph Hellwig , Kevin Tian , Ashok Raj , Will Deacon , Robin Murphy , Jean-Philippe Brucker , Dave Jiang , Vinod Koul Cc: Eric Auger , Liu Yi L , Jacob jun Pan , Zhangfei Gao , Zhu Tony , iommu@lists.linux.dev, linux-kernel@vger.kernel.org, Lu Baolu Subject: [PATCH v10 02/12] iommu: Add max_pasids field in struct dev_iommu Date: Tue, 5 Jul 2022 13:07:00 +0800 Message-Id: <20220705050710.2887204-3-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220705050710.2887204-1-baolu.lu@linux.intel.com> References: <20220705050710.2887204-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Use this field to save the number of PASIDs that a device is able to consume. It is a generic attribute of a device and lifting it into the per-device dev_iommu struct could help to avoid the boilerplate code in various IOMMU drivers. Signed-off-by: Lu Baolu Reviewed-by: Kevin Tian Tested-by: Zhangfei Gao Tested-by: Tony Zhu Reviewed-by: Jason Gunthorpe Reviewed-by: Yi Liu --- include/linux/iommu.h | 2 ++ drivers/iommu/iommu.c | 20 ++++++++++++++++++++ 2 files changed, 22 insertions(+) diff --git a/include/linux/iommu.h b/include/linux/iommu.h index 03fbb1b71536..418a1914a041 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -364,6 +364,7 @@ struct iommu_fault_param { * @fwspec: IOMMU fwspec data * @iommu_dev: IOMMU device this device is linked to * @priv: IOMMU Driver private data + * @max_pasids: number of PASIDs this device can consume * * TODO: migrate other per device data pointers under iommu_dev_data, e.g. * struct iommu_group *iommu_group; @@ -375,6 +376,7 @@ struct dev_iommu { struct iommu_fwspec *fwspec; struct iommu_device *iommu_dev; void *priv; + u32 max_pasids; }; =20 int iommu_device_register(struct iommu_device *iommu, diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c index cdc86c39954e..0cb0750f61e8 100644 --- a/drivers/iommu/iommu.c +++ b/drivers/iommu/iommu.c @@ -20,6 +20,7 @@ #include #include #include +#include #include #include #include @@ -218,6 +219,24 @@ static void dev_iommu_free(struct device *dev) kfree(param); } =20 +static u32 dev_iommu_get_max_pasids(struct device *dev) +{ + u32 max_pasids =3D 0, bits =3D 0; + int ret; + + if (dev_is_pci(dev)) { + ret =3D pci_max_pasids(to_pci_dev(dev)); + if (ret > 0) + max_pasids =3D ret; + } else { + ret =3D device_property_read_u32(dev, "pasid-num-bits", &bits); + if (!ret) + max_pasids =3D 1UL << bits; + } + + return min_t(u32, max_pasids, dev->iommu->iommu_dev->max_pasids); +} + static int __iommu_probe_device(struct device *dev, struct list_head *grou= p_list) { const struct iommu_ops *ops =3D dev->bus->iommu_ops; @@ -243,6 +262,7 @@ static int __iommu_probe_device(struct device *dev, str= uct list_head *group_list } =20 dev->iommu->iommu_dev =3D iommu_dev; + dev->iommu->max_pasids =3D dev_iommu_get_max_pasids(dev); =20 group =3D iommu_group_get_for_dev(dev); if (IS_ERR(group)) { --=20 2.25.1 From nobody Sun Apr 19 09:04:37 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 73C17C43334 for ; Tue, 5 Jul 2022 05:12:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229615AbiGEFMB (ORCPT ); Tue, 5 Jul 2022 01:12:01 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38094 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229633AbiGEFL5 (ORCPT ); Tue, 5 Jul 2022 01:11:57 -0400 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 09EB913D0E for ; Mon, 4 Jul 2022 22:11:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1656997914; x=1688533914; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Jrr+bz26rJCsbIQSBUs9mjSQi8pwmQPt2UaZLqjDw9k=; b=TVQQjFF6wr+vP8xy31Mhmymcfi8HKykJmpy2iq+etbSgzyrX6GEsMf7m 97zhUCzdNxoBEmdm3JKDUeChODWbVv21RUML7kqqDkGe0imvrOgeZ6d8W 8Ji1V9bXHFG1O546CzwcQpk7FEccmQOL/CmBUmhesOxkDLdt6WFez0/1p cg8Rdsl8kJkVadeyROkFUbrD+H8NIodK3oXOPc5Dnlgkp1lYr2BevTwiG usxnMG/vQIokVz7rH5S+lawXh7LhdAB+fk5wjh392qTo2MQPmsyZ8PkgV ctYjsXq77JCXDX+eRoh+eVc0GX5+2PcCEh4mubUeFrOcNLOz9T0LbX7rW A==; X-IronPort-AV: E=McAfee;i="6400,9594,10398"; a="263046076" X-IronPort-AV: E=Sophos;i="5.92,245,1650956400"; d="scan'208";a="263046076" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Jul 2022 22:11:53 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.92,245,1650956400"; d="scan'208";a="542807406" Received: from allen-box.sh.intel.com ([10.239.159.48]) by orsmga003.jf.intel.com with ESMTP; 04 Jul 2022 22:11:48 -0700 From: Lu Baolu To: Joerg Roedel , Jason Gunthorpe , Christoph Hellwig , Kevin Tian , Ashok Raj , Will Deacon , Robin Murphy , Jean-Philippe Brucker , Dave Jiang , Vinod Koul Cc: Eric Auger , Liu Yi L , Jacob jun Pan , Zhangfei Gao , Zhu Tony , iommu@lists.linux.dev, linux-kernel@vger.kernel.org, Lu Baolu , Jacob Pan , Jean-Philippe Brucker Subject: [PATCH v10 03/12] iommu: Remove SVM_FLAG_SUPERVISOR_MODE support Date: Tue, 5 Jul 2022 13:07:01 +0800 Message-Id: <20220705050710.2887204-4-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220705050710.2887204-1-baolu.lu@linux.intel.com> References: <20220705050710.2887204-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org The current kernel DMA with PASID support is based on the SVA with a flag SVM_FLAG_SUPERVISOR_MODE. The IOMMU driver binds the kernel memory address space to a PASID of the device. The device driver programs the device with kernel virtual address (KVA) for DMA access. There have been security and functional issues with this approach: - The lack of IOTLB synchronization upon kernel page table updates. (vmalloc, module/BPF loading, CONFIG_DEBUG_PAGEALLOC etc.) - Other than slight more protection, using kernel virtual address (KVA) has little advantage over physical address. There are also no use cases yet where DMA engines need kernel virtual addresses for in-kernel DMA. This removes SVM_FLAG_SUPERVISOR_MODE support from the IOMMU interface. The device drivers are suggested to handle kernel DMA with PASID through the kernel DMA APIs. The drvdata parameter in iommu_sva_bind_device() and all callbacks is not needed anymore. Cleanup them as well. Link: https://lore.kernel.org/linux-iommu/20210511194726.GP1002214@nvidia.c= om/ Signed-off-by: Jacob Pan Signed-off-by: Lu Baolu Reviewed-by: Jason Gunthorpe Reviewed-by: Jean-Philippe Brucker Reviewed-by: Kevin Tian Tested-by: Zhangfei Gao Tested-by: Tony Zhu --- include/linux/intel-iommu.h | 3 +- include/linux/intel-svm.h | 13 ----- include/linux/iommu.h | 8 +-- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 5 +- drivers/dma/idxd/cdev.c | 3 +- drivers/dma/idxd/init.c | 25 +------- .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 3 +- drivers/iommu/intel/svm.c | 57 +++++-------------- drivers/iommu/iommu.c | 5 +- drivers/misc/uacce/uacce.c | 2 +- 10 files changed, 26 insertions(+), 98 deletions(-) diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h index e065cbe3c857..31e3edc0fc7e 100644 --- a/include/linux/intel-iommu.h +++ b/include/linux/intel-iommu.h @@ -738,8 +738,7 @@ struct intel_iommu *device_to_iommu(struct device *dev,= u8 *bus, u8 *devfn); extern void intel_svm_check(struct intel_iommu *iommu); extern int intel_svm_enable_prq(struct intel_iommu *iommu); extern int intel_svm_finish_prq(struct intel_iommu *iommu); -struct iommu_sva *intel_svm_bind(struct device *dev, struct mm_struct *mm, - void *drvdata); +struct iommu_sva *intel_svm_bind(struct device *dev, struct mm_struct *mm); void intel_svm_unbind(struct iommu_sva *handle); u32 intel_svm_get_pasid(struct iommu_sva *handle); int intel_svm_page_response(struct device *dev, struct iommu_fault_event *= evt, diff --git a/include/linux/intel-svm.h b/include/linux/intel-svm.h index 207ef06ba3e1..f9a0d44f6fdb 100644 --- a/include/linux/intel-svm.h +++ b/include/linux/intel-svm.h @@ -13,17 +13,4 @@ #define PRQ_RING_MASK ((0x1000 << PRQ_ORDER) - 0x20) #define PRQ_DEPTH ((0x1000 << PRQ_ORDER) >> 5) =20 -/* - * The SVM_FLAG_SUPERVISOR_MODE flag requests a PASID which can be used on= ly - * for access to kernel addresses. No IOTLB flushes are automatically done - * for kernel mappings; it is valid only for access to the kernel's static - * 1:1 mapping of physical memory =E2=80=94 not to vmalloc or even module = mappings. - * A future API addition may permit the use of such ranges, by means of an - * explicit IOTLB flush call (akin to the DMA API's unmap method). - * - * It is unlikely that we will ever hook into flush_tlb_kernel_range() to - * do such IOTLB flushes automatically. - */ -#define SVM_FLAG_SUPERVISOR_MODE BIT(0) - #endif /* __INTEL_SVM_H__ */ diff --git a/include/linux/iommu.h b/include/linux/iommu.h index 418a1914a041..f41eb2b3c7da 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -243,8 +243,7 @@ struct iommu_ops { int (*dev_enable_feat)(struct device *dev, enum iommu_dev_features f); int (*dev_disable_feat)(struct device *dev, enum iommu_dev_features f); =20 - struct iommu_sva *(*sva_bind)(struct device *dev, struct mm_struct *mm, - void *drvdata); + struct iommu_sva *(*sva_bind)(struct device *dev, struct mm_struct *mm); void (*sva_unbind)(struct iommu_sva *handle); u32 (*sva_get_pasid)(struct iommu_sva *handle); =20 @@ -669,8 +668,7 @@ int iommu_dev_disable_feature(struct device *dev, enum = iommu_dev_features f); bool iommu_dev_feature_enabled(struct device *dev, enum iommu_dev_features= f); =20 struct iommu_sva *iommu_sva_bind_device(struct device *dev, - struct mm_struct *mm, - void *drvdata); + struct mm_struct *mm); void iommu_sva_unbind_device(struct iommu_sva *handle); u32 iommu_sva_get_pasid(struct iommu_sva *handle); =20 @@ -1012,7 +1010,7 @@ iommu_dev_disable_feature(struct device *dev, enum io= mmu_dev_features feat) } =20 static inline struct iommu_sva * -iommu_sva_bind_device(struct device *dev, struct mm_struct *mm, void *drvd= ata) +iommu_sva_bind_device(struct device *dev, struct mm_struct *mm) { return NULL; } diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/ar= m/arm-smmu-v3/arm-smmu-v3.h index cd48590ada30..d2ba86470c42 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -754,8 +754,7 @@ bool arm_smmu_master_sva_enabled(struct arm_smmu_master= *master); int arm_smmu_master_enable_sva(struct arm_smmu_master *master); int arm_smmu_master_disable_sva(struct arm_smmu_master *master); bool arm_smmu_master_iopf_supported(struct arm_smmu_master *master); -struct iommu_sva *arm_smmu_sva_bind(struct device *dev, struct mm_struct *= mm, - void *drvdata); +struct iommu_sva *arm_smmu_sva_bind(struct device *dev, struct mm_struct *= mm); void arm_smmu_sva_unbind(struct iommu_sva *handle); u32 arm_smmu_sva_get_pasid(struct iommu_sva *handle); void arm_smmu_sva_notifier_synchronize(void); @@ -791,7 +790,7 @@ static inline bool arm_smmu_master_iopf_supported(struc= t arm_smmu_master *master } =20 static inline struct iommu_sva * -arm_smmu_sva_bind(struct device *dev, struct mm_struct *mm, void *drvdata) +arm_smmu_sva_bind(struct device *dev, struct mm_struct *mm) { return ERR_PTR(-ENODEV); } diff --git a/drivers/dma/idxd/cdev.c b/drivers/dma/idxd/cdev.c index c2808fd081d6..66720001ba1c 100644 --- a/drivers/dma/idxd/cdev.c +++ b/drivers/dma/idxd/cdev.c @@ -6,7 +6,6 @@ #include #include #include -#include #include #include #include @@ -100,7 +99,7 @@ static int idxd_cdev_open(struct inode *inode, struct fi= le *filp) filp->private_data =3D ctx; =20 if (device_user_pasid_enabled(idxd)) { - sva =3D iommu_sva_bind_device(dev, current->mm, NULL); + sva =3D iommu_sva_bind_device(dev, current->mm); if (IS_ERR(sva)) { rc =3D PTR_ERR(sva); dev_err(dev, "pasid allocation failed: %d\n", rc); diff --git a/drivers/dma/idxd/init.c b/drivers/dma/idxd/init.c index 355fb3ef4cbf..00b437f4f573 100644 --- a/drivers/dma/idxd/init.c +++ b/drivers/dma/idxd/init.c @@ -14,7 +14,6 @@ #include #include #include -#include #include #include #include @@ -466,29 +465,7 @@ static struct idxd_device *idxd_alloc(struct pci_dev *= pdev, struct idxd_driver_d =20 static int idxd_enable_system_pasid(struct idxd_device *idxd) { - int flags; - unsigned int pasid; - struct iommu_sva *sva; - - flags =3D SVM_FLAG_SUPERVISOR_MODE; - - sva =3D iommu_sva_bind_device(&idxd->pdev->dev, NULL, &flags); - if (IS_ERR(sva)) { - dev_warn(&idxd->pdev->dev, - "iommu sva bind failed: %ld\n", PTR_ERR(sva)); - return PTR_ERR(sva); - } - - pasid =3D iommu_sva_get_pasid(sva); - if (pasid =3D=3D IOMMU_PASID_INVALID) { - iommu_sva_unbind_device(sva); - return -ENODEV; - } - - idxd->sva =3D sva; - idxd->pasid =3D pasid; - dev_dbg(&idxd->pdev->dev, "system pasid: %u\n", pasid); - return 0; + return -EOPNOTSUPP; } =20 static void idxd_disable_system_pasid(struct idxd_device *idxd) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iomm= u/arm/arm-smmu-v3/arm-smmu-v3-sva.c index 1ef7bbb4acf3..f155d406c5d5 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -367,8 +367,7 @@ __arm_smmu_sva_bind(struct device *dev, struct mm_struc= t *mm) return ERR_PTR(ret); } =20 -struct iommu_sva * -arm_smmu_sva_bind(struct device *dev, struct mm_struct *mm, void *drvdata) +struct iommu_sva *arm_smmu_sva_bind(struct device *dev, struct mm_struct *= mm) { struct iommu_sva *handle; struct iommu_domain *domain =3D iommu_get_domain_for_dev(dev); diff --git a/drivers/iommu/intel/svm.c b/drivers/iommu/intel/svm.c index 7ee37d996e15..d04880a291c3 100644 --- a/drivers/iommu/intel/svm.c +++ b/drivers/iommu/intel/svm.c @@ -313,8 +313,7 @@ static int pasid_to_svm_sdev(struct device *dev, unsign= ed int pasid, return 0; } =20 -static int intel_svm_alloc_pasid(struct device *dev, struct mm_struct *mm, - unsigned int flags) +static int intel_svm_alloc_pasid(struct device *dev, struct mm_struct *mm) { ioasid_t max_pasid =3D dev_is_pci(dev) ? pci_max_pasids(to_pci_dev(dev)) : intel_pasid_max_id; @@ -324,8 +323,7 @@ static int intel_svm_alloc_pasid(struct device *dev, st= ruct mm_struct *mm, =20 static struct iommu_sva *intel_svm_bind_mm(struct intel_iommu *iommu, struct device *dev, - struct mm_struct *mm, - unsigned int flags) + struct mm_struct *mm) { struct device_domain_info *info =3D dev_iommu_priv_get(dev); unsigned long iflags, sflags; @@ -341,22 +339,18 @@ static struct iommu_sva *intel_svm_bind_mm(struct int= el_iommu *iommu, =20 svm->pasid =3D mm->pasid; svm->mm =3D mm; - svm->flags =3D flags; INIT_LIST_HEAD_RCU(&svm->devs); =20 - if (!(flags & SVM_FLAG_SUPERVISOR_MODE)) { - svm->notifier.ops =3D &intel_mmuops; - ret =3D mmu_notifier_register(&svm->notifier, mm); - if (ret) { - kfree(svm); - return ERR_PTR(ret); - } + svm->notifier.ops =3D &intel_mmuops; + ret =3D mmu_notifier_register(&svm->notifier, mm); + if (ret) { + kfree(svm); + return ERR_PTR(ret); } =20 ret =3D pasid_private_add(svm->pasid, svm); if (ret) { - if (svm->notifier.ops) - mmu_notifier_unregister(&svm->notifier, mm); + mmu_notifier_unregister(&svm->notifier, mm); kfree(svm); return ERR_PTR(ret); } @@ -391,9 +385,7 @@ static struct iommu_sva *intel_svm_bind_mm(struct intel= _iommu *iommu, } =20 /* Setup the pasid table: */ - sflags =3D (flags & SVM_FLAG_SUPERVISOR_MODE) ? - PASID_FLAG_SUPERVISOR_MODE : 0; - sflags |=3D cpu_feature_enabled(X86_FEATURE_LA57) ? PASID_FLAG_FL5LP : 0; + sflags =3D cpu_feature_enabled(X86_FEATURE_LA57) ? PASID_FLAG_FL5LP : 0; spin_lock_irqsave(&iommu->lock, iflags); ret =3D intel_pasid_setup_first_level(iommu, dev, mm->pgd, mm->pasid, FLPT_DEFAULT_DID, sflags); @@ -410,8 +402,7 @@ static struct iommu_sva *intel_svm_bind_mm(struct intel= _iommu *iommu, kfree(sdev); free_svm: if (list_empty(&svm->devs)) { - if (svm->notifier.ops) - mmu_notifier_unregister(&svm->notifier, mm); + mmu_notifier_unregister(&svm->notifier, mm); pasid_private_remove(mm->pasid); kfree(svm); } @@ -767,7 +758,7 @@ static irqreturn_t prq_event_thread(int irq, void *d) * to unbind the mm while any page faults are outstanding. */ svm =3D pasid_private_find(req->pasid); - if (IS_ERR_OR_NULL(svm) || (svm->flags & SVM_FLAG_SUPERVISOR_MODE)) + if (IS_ERR_OR_NULL(svm)) goto bad_req; } =20 @@ -818,40 +809,20 @@ static irqreturn_t prq_event_thread(int irq, void *d) return IRQ_RETVAL(handled); } =20 -struct iommu_sva *intel_svm_bind(struct device *dev, struct mm_struct *mm,= void *drvdata) +struct iommu_sva *intel_svm_bind(struct device *dev, struct mm_struct *mm) { struct intel_iommu *iommu =3D device_to_iommu(dev, NULL, NULL); - unsigned int flags =3D 0; struct iommu_sva *sva; int ret; =20 - if (drvdata) - flags =3D *(unsigned int *)drvdata; - - if (flags & SVM_FLAG_SUPERVISOR_MODE) { - if (!ecap_srs(iommu->ecap)) { - dev_err(dev, "%s: Supervisor PASID not supported\n", - iommu->name); - return ERR_PTR(-EOPNOTSUPP); - } - - if (mm) { - dev_err(dev, "%s: Supervisor PASID with user provided mm\n", - iommu->name); - return ERR_PTR(-EINVAL); - } - - mm =3D &init_mm; - } - mutex_lock(&pasid_mutex); - ret =3D intel_svm_alloc_pasid(dev, mm, flags); + ret =3D intel_svm_alloc_pasid(dev, mm); if (ret) { mutex_unlock(&pasid_mutex); return ERR_PTR(ret); } =20 - sva =3D intel_svm_bind_mm(iommu, dev, mm, flags); + sva =3D intel_svm_bind_mm(iommu, dev, mm); mutex_unlock(&pasid_mutex); =20 return sva; diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c index 0cb0750f61e8..74a0a3ec0907 100644 --- a/drivers/iommu/iommu.c +++ b/drivers/iommu/iommu.c @@ -2788,7 +2788,6 @@ EXPORT_SYMBOL_GPL(iommu_dev_feature_enabled); * iommu_sva_bind_device() - Bind a process address space to a device * @dev: the device * @mm: the mm to bind, caller must hold a reference to it - * @drvdata: opaque data pointer to pass to bind callback * * Create a bond between device and address space, allowing the device to = access * the mm using the returned PASID. If a bond already exists between @devi= ce and @@ -2801,7 +2800,7 @@ EXPORT_SYMBOL_GPL(iommu_dev_feature_enabled); * On error, returns an ERR_PTR value. */ struct iommu_sva * -iommu_sva_bind_device(struct device *dev, struct mm_struct *mm, void *drvd= ata) +iommu_sva_bind_device(struct device *dev, struct mm_struct *mm) { struct iommu_group *group; struct iommu_sva *handle =3D ERR_PTR(-EINVAL); @@ -2826,7 +2825,7 @@ iommu_sva_bind_device(struct device *dev, struct mm_s= truct *mm, void *drvdata) if (iommu_group_device_count(group) !=3D 1) goto out_unlock; =20 - handle =3D ops->sva_bind(dev, mm, drvdata); + handle =3D ops->sva_bind(dev, mm); =20 out_unlock: mutex_unlock(&group->mutex); diff --git a/drivers/misc/uacce/uacce.c b/drivers/misc/uacce/uacce.c index 281c54003edc..3238a867ea51 100644 --- a/drivers/misc/uacce/uacce.c +++ b/drivers/misc/uacce/uacce.c @@ -99,7 +99,7 @@ static int uacce_bind_queue(struct uacce_device *uacce, s= truct uacce_queue *q) if (!(uacce->flags & UACCE_DEV_SVA)) return 0; =20 - handle =3D iommu_sva_bind_device(uacce->parent, current->mm, NULL); + handle =3D iommu_sva_bind_device(uacce->parent, current->mm); if (IS_ERR(handle)) return PTR_ERR(handle); =20 --=20 2.25.1 From nobody Sun Apr 19 09:04:37 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 003B4C433EF for ; Tue, 5 Jul 2022 05:12:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229658AbiGEFME (ORCPT ); Tue, 5 Jul 2022 01:12:04 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38118 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229641AbiGEFL7 (ORCPT ); Tue, 5 Jul 2022 01:11:59 -0400 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 73DB313D16 for ; Mon, 4 Jul 2022 22:11:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1656997918; x=1688533918; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Eh7Ts/jrbu3gu6i73diB/9eZuSBu3uGNqSXuF4Z/1cg=; b=UO/yWV9PD6YZXBi6lCbsf3iU916Nq8DlCpC/LXjY2BOT8O9jjQ7yywQw XGonprDzDjitS/RBej7ckf4X1yIw3PO1lSiPwuiFDvbQm6STZwORknBsd RL4VfQP7aNZB7O8gqh3RsfmPy1WNL3BlM+EqFVKcyAi9PxsBMv8g9dv7y jcXsDMEJmqXhN/IBAlq7SJM44h+0Yn+K/iFbbHsQqN0V1pr36+E5XvLHj wV+Bgpix5Ubx87p0Pytbp1d6XCgxMa9bcHsG9mM6DVFyasZfRwp2cw+2+ zM2F5pANDgjDHiwNkdaO7tnYEFWx4rdISixZTtejAu2WtV+U6GPYD4bCN g==; X-IronPort-AV: E=McAfee;i="6400,9594,10398"; a="263046090" X-IronPort-AV: E=Sophos;i="5.92,245,1650956400"; d="scan'208";a="263046090" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Jul 2022 22:11:58 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.92,245,1650956400"; d="scan'208";a="542807414" Received: from allen-box.sh.intel.com ([10.239.159.48]) by orsmga003.jf.intel.com with ESMTP; 04 Jul 2022 22:11:53 -0700 From: Lu Baolu To: Joerg Roedel , Jason Gunthorpe , Christoph Hellwig , Kevin Tian , Ashok Raj , Will Deacon , Robin Murphy , Jean-Philippe Brucker , Dave Jiang , Vinod Koul Cc: Eric Auger , Liu Yi L , Jacob jun Pan , Zhangfei Gao , Zhu Tony , iommu@lists.linux.dev, linux-kernel@vger.kernel.org, Lu Baolu , Jean-Philippe Brucker Subject: [PATCH v10 04/12] iommu: Add attach/detach_dev_pasid iommu interface Date: Tue, 5 Jul 2022 13:07:02 +0800 Message-Id: <20220705050710.2887204-5-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220705050710.2887204-1-baolu.lu@linux.intel.com> References: <20220705050710.2887204-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Attaching an IOMMU domain to a PASID of a device is a generic operation for modern IOMMU drivers which support PASID-granular DMA address translation. Currently visible usage scenarios include (but not limited): - SVA (Shared Virtual Address) - kernel DMA with PASID - hardware-assist mediated device This adds a pair of domain ops for this purpose and adds the interfaces for device drivers to attach/detach a domain to/from a {device, PASID}. Some buses, like PCI, route packets without considering the PASID value. Thus a DMA target address with PASID might be treated as P2P if the address falls into the MMIO BAR of other devices in the group. To make things simple, these interfaces only apply to devices belonging to the singleton groups, and the singleton is immutable in fabric (i.e. not affected by hotplug). Signed-off-by: Lu Baolu Reviewed-by: Jean-Philippe Brucker Tested-by: Zhangfei Gao Tested-by: Tony Zhu Reviewed-by: Kevin Tian Reviewed-by: Yi Liu --- include/linux/iommu.h | 21 ++++++++++++ drivers/iommu/iommu.c | 75 +++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 96 insertions(+) diff --git a/include/linux/iommu.h b/include/linux/iommu.h index f41eb2b3c7da..f2b5aa7efe43 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -262,6 +262,8 @@ struct iommu_ops { * struct iommu_domain_ops - domain specific operations * @attach_dev: attach an iommu domain to a device * @detach_dev: detach an iommu domain from a device + * @set_dev_pasid: set an iommu domain to a pasid of device + * @block_dev_pasid: block pasid of device from using iommu domain * @map: map a physically contiguous memory region to an iommu domain * @map_pages: map a physically contiguous set of pages of the same size to * an iommu domain. @@ -282,6 +284,10 @@ struct iommu_ops { struct iommu_domain_ops { int (*attach_dev)(struct iommu_domain *domain, struct device *dev); void (*detach_dev)(struct iommu_domain *domain, struct device *dev); + int (*set_dev_pasid)(struct iommu_domain *domain, struct device *dev, + ioasid_t pasid); + void (*block_dev_pasid)(struct iommu_domain *domain, struct device *dev, + ioasid_t pasid); =20 int (*map)(struct iommu_domain *domain, unsigned long iova, phys_addr_t paddr, size_t size, int prot, gfp_t gfp); @@ -679,6 +685,10 @@ int iommu_group_claim_dma_owner(struct iommu_group *gr= oup, void *owner); void iommu_group_release_dma_owner(struct iommu_group *group); bool iommu_group_dma_owner_claimed(struct iommu_group *group); =20 +int iommu_attach_device_pasid(struct iommu_domain *domain, struct device *= dev, + ioasid_t pasid); +void iommu_detach_device_pasid(struct iommu_domain *domain, struct device = *dev, + ioasid_t pasid); #else /* CONFIG_IOMMU_API */ =20 struct iommu_ops {}; @@ -1052,6 +1062,17 @@ static inline bool iommu_group_dma_owner_claimed(str= uct iommu_group *group) { return false; } + +static inline int iommu_attach_device_pasid(struct iommu_domain *domain, + struct device *dev, ioasid_t pasid) +{ + return -ENODEV; +} + +static inline void iommu_detach_device_pasid(struct iommu_domain *domain, + struct device *dev, ioasid_t pasid) +{ +} #endif /* CONFIG_IOMMU_API */ =20 /** diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c index 74a0a3ec0907..be48b09371f4 100644 --- a/drivers/iommu/iommu.c +++ b/drivers/iommu/iommu.c @@ -39,6 +39,7 @@ struct iommu_group { struct kobject kobj; struct kobject *devices_kobj; struct list_head devices; + struct xarray pasid_array; struct mutex mutex; void *iommu_data; void (*iommu_data_release)(void *iommu_data); @@ -660,6 +661,7 @@ struct iommu_group *iommu_group_alloc(void) mutex_init(&group->mutex); INIT_LIST_HEAD(&group->devices); INIT_LIST_HEAD(&group->entry); + xa_init(&group->pasid_array); =20 ret =3D ida_alloc(&iommu_group_ida, GFP_KERNEL); if (ret < 0) { @@ -3271,3 +3273,76 @@ bool iommu_group_dma_owner_claimed(struct iommu_grou= p *group) return user; } EXPORT_SYMBOL_GPL(iommu_group_dma_owner_claimed); + +static bool iommu_group_immutable_singleton(struct iommu_group *group, + struct device *dev) +{ + int count; + + mutex_lock(&group->mutex); + count =3D iommu_group_device_count(group); + mutex_unlock(&group->mutex); + + if (count !=3D 1) + return false; + + /* + * The PCI device could be considered to be fully isolated if all + * devices on the path from the device to the host-PCI bridge are + * protected from peer-to-peer DMA by ACS. + */ + if (dev_is_pci(dev)) + return pci_acs_path_enabled(to_pci_dev(dev), NULL, + REQ_ACS_FLAGS); + + /* + * Otherwise, the device came from DT/ACPI, assume it is static and + * then singleton can know from the device count in the group. + */ + return true; +} + +int iommu_attach_device_pasid(struct iommu_domain *domain, struct device *= dev, + ioasid_t pasid) +{ + struct iommu_group *group; + void *curr; + int ret; + + if (!domain->ops->set_dev_pasid) + return -EOPNOTSUPP; + + group =3D iommu_group_get(dev); + if (!group || !iommu_group_immutable_singleton(group, dev)) { + iommu_group_put(group); + return -EINVAL; + } + + mutex_lock(&group->mutex); + curr =3D xa_cmpxchg(&group->pasid_array, pasid, NULL, domain, GFP_KERNEL); + if (curr) { + ret =3D xa_err(curr) ? : -EBUSY; + goto out_unlock; + } + ret =3D domain->ops->set_dev_pasid(domain, dev, pasid); + if (ret) + xa_erase(&group->pasid_array, pasid); +out_unlock: + mutex_unlock(&group->mutex); + iommu_group_put(group); + + return ret; +} + +void iommu_detach_device_pasid(struct iommu_domain *domain, struct device = *dev, + ioasid_t pasid) +{ + struct iommu_group *group =3D iommu_group_get(dev); + + mutex_lock(&group->mutex); + domain->ops->block_dev_pasid(domain, dev, pasid); + xa_erase(&group->pasid_array, pasid); + mutex_unlock(&group->mutex); + + iommu_group_put(group); +} --=20 2.25.1 From nobody Sun Apr 19 09:04:37 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D1798C433EF for ; Tue, 5 Jul 2022 05:12:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229798AbiGEFMY (ORCPT ); Tue, 5 Jul 2022 01:12:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38152 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229660AbiGEFME (ORCPT ); Tue, 5 Jul 2022 01:12:04 -0400 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7122213D11 for ; Mon, 4 Jul 2022 22:12:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1656997923; x=1688533923; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=fTrtMAAJhQfGMMOU6j85FQf2LBXIcS+qto9gmyXkZP4=; b=CAF63RQQUDOs11Ojrk/PPoyfcOZooAA0Oz7hKj0ce4HIFkqHhWWzWZrB mXcmpRCEC6lSBtN7gOMKTULU0jsw5vo1EUvqTfO6k3Xh12mGMjdvrbq1c kcW6ITQ/ge1kSAe8MoOEx4gCbMwl2AqXmWbq+VgJTOkyDn70awvMWUVhG Tv/IERN93a+r8ktEq65ZvbqFcfwvzxs8Mf017QFm43FjEiERBG0hJylB1 4GV8e4uAIoRuw6y17a/ZWYIDYn2EmOsHLqTTEdYJW/j6ARC9dzhwL9e3e 6f2kLJq1z7G3ynerWAuI6vcD8NX5U3HbSwTVFlOIzXB0hoMjmrYTEfnu/ g==; X-IronPort-AV: E=McAfee;i="6400,9594,10398"; a="263046104" X-IronPort-AV: E=Sophos;i="5.92,245,1650956400"; d="scan'208";a="263046104" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Jul 2022 22:12:03 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.92,245,1650956400"; d="scan'208";a="542807440" Received: from allen-box.sh.intel.com ([10.239.159.48]) by orsmga003.jf.intel.com with ESMTP; 04 Jul 2022 22:11:58 -0700 From: Lu Baolu To: Joerg Roedel , Jason Gunthorpe , Christoph Hellwig , Kevin Tian , Ashok Raj , Will Deacon , Robin Murphy , Jean-Philippe Brucker , Dave Jiang , Vinod Koul Cc: Eric Auger , Liu Yi L , Jacob jun Pan , Zhangfei Gao , Zhu Tony , iommu@lists.linux.dev, linux-kernel@vger.kernel.org, Lu Baolu , Jean-Philippe Brucker Subject: [PATCH v10 05/12] iommu: Add IOMMU SVA domain support Date: Tue, 5 Jul 2022 13:07:03 +0800 Message-Id: <20220705050710.2887204-6-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220705050710.2887204-1-baolu.lu@linux.intel.com> References: <20220705050710.2887204-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The sva iommu_domain represents a hardware pagetable that the IOMMU hardware could use for SVA translation. This adds some infrastructure to support SVA domain in the iommu common layer. It includes: - Extend the iommu_domain to support a new IOMMU_DOMAIN_SVA domain type. The IOMMU drivers that support allocation of the SVA domain should provide its own sva domain specific iommu_domain_ops. - Add a helper to allocate an SVA domain. The iommu_domain_free() is still used to free an SVA domain. The report_iommu_fault() should be replaced by the new iommu_report_device_fault(). Leave the existing fault handler with the existing users and the newly added SVA members excludes it. Suggested-by: Jean-Philippe Brucker Suggested-by: Jason Gunthorpe Signed-off-by: Lu Baolu Reviewed-by: Jean-Philippe Brucker Tested-by: Zhangfei Gao Tested-by: Tony Zhu Reviewed-by: Jason Gunthorpe Reviewed-by: Kevin Tian Reviewed-by: Yi Liu --- include/linux/iommu.h | 24 ++++++++++++++++++++++-- drivers/iommu/iommu.c | 20 ++++++++++++++++++++ 2 files changed, 42 insertions(+), 2 deletions(-) diff --git a/include/linux/iommu.h b/include/linux/iommu.h index f2b5aa7efe43..42f0418dc22c 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -64,6 +64,8 @@ struct iommu_domain_geometry { #define __IOMMU_DOMAIN_PT (1U << 2) /* Domain is identity mapped */ #define __IOMMU_DOMAIN_DMA_FQ (1U << 3) /* DMA-API uses flush queue */ =20 +#define __IOMMU_DOMAIN_SVA (1U << 4) /* Shared process address space */ + /* * This are the possible domain-types * @@ -77,6 +79,8 @@ struct iommu_domain_geometry { * certain optimizations for these domains * IOMMU_DOMAIN_DMA_FQ - As above, but definitely using batched TLB * invalidation. + * IOMMU_DOMAIN_SVA - DMA addresses are shared process address + * spaces represented by mm_struct's. */ #define IOMMU_DOMAIN_BLOCKED (0U) #define IOMMU_DOMAIN_IDENTITY (__IOMMU_DOMAIN_PT) @@ -86,15 +90,23 @@ struct iommu_domain_geometry { #define IOMMU_DOMAIN_DMA_FQ (__IOMMU_DOMAIN_PAGING | \ __IOMMU_DOMAIN_DMA_API | \ __IOMMU_DOMAIN_DMA_FQ) +#define IOMMU_DOMAIN_SVA (__IOMMU_DOMAIN_SVA) =20 struct iommu_domain { unsigned type; const struct iommu_domain_ops *ops; unsigned long pgsize_bitmap; /* Bitmap of page sizes in use */ - iommu_fault_handler_t handler; - void *handler_token; struct iommu_domain_geometry geometry; struct iommu_dma_cookie *iova_cookie; + union { + struct { + iommu_fault_handler_t handler; + void *handler_token; + }; + struct { /* IOMMU_DOMAIN_SVA */ + struct mm_struct *mm; + }; + }; }; =20 static inline bool iommu_is_dma_domain(struct iommu_domain *domain) @@ -685,6 +697,8 @@ int iommu_group_claim_dma_owner(struct iommu_group *gro= up, void *owner); void iommu_group_release_dma_owner(struct iommu_group *group); bool iommu_group_dma_owner_claimed(struct iommu_group *group); =20 +struct iommu_domain *iommu_sva_domain_alloc(struct device *dev, + struct mm_struct *mm); int iommu_attach_device_pasid(struct iommu_domain *domain, struct device *= dev, ioasid_t pasid); void iommu_detach_device_pasid(struct iommu_domain *domain, struct device = *dev, @@ -1063,6 +1077,12 @@ static inline bool iommu_group_dma_owner_claimed(str= uct iommu_group *group) return false; } =20 +static inline struct iommu_domain * +iommu_sva_domain_alloc(struct device *dev, struct mm_struct *mm) +{ + return NULL; +} + static inline int iommu_attach_device_pasid(struct iommu_domain *domain, struct device *dev, ioasid_t pasid) { diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c index be48b09371f4..10479c5e4d23 100644 --- a/drivers/iommu/iommu.c +++ b/drivers/iommu/iommu.c @@ -27,6 +27,7 @@ #include #include #include +#include =20 static struct kset *iommu_group_kset; static DEFINE_IDA(iommu_group_ida); @@ -1957,6 +1958,8 @@ EXPORT_SYMBOL_GPL(iommu_domain_alloc); =20 void iommu_domain_free(struct iommu_domain *domain) { + if (domain->type =3D=3D IOMMU_DOMAIN_SVA) + mmdrop(domain->mm); iommu_put_dma_cookie(domain); domain->ops->free(domain); } @@ -3274,6 +3277,23 @@ bool iommu_group_dma_owner_claimed(struct iommu_grou= p *group) } EXPORT_SYMBOL_GPL(iommu_group_dma_owner_claimed); =20 +struct iommu_domain *iommu_sva_domain_alloc(struct device *dev, + struct mm_struct *mm) +{ + const struct iommu_ops *ops =3D dev_iommu_ops(dev); + struct iommu_domain *domain; + + domain =3D ops->domain_alloc(IOMMU_DOMAIN_SVA); + if (!domain) + return NULL; + + domain->type =3D IOMMU_DOMAIN_SVA; + mmgrab(mm); + domain->mm =3D mm; + + return domain; +} + static bool iommu_group_immutable_singleton(struct iommu_group *group, struct device *dev) { --=20 2.25.1 From nobody Sun Apr 19 09:04:37 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C9615C43334 for ; Tue, 5 Jul 2022 05:12:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229906AbiGEFMe (ORCPT ); Tue, 5 Jul 2022 01:12:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38628 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229746AbiGEFMN (ORCPT ); Tue, 5 Jul 2022 01:12:13 -0400 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7289C13D43 for ; Mon, 4 Jul 2022 22:12:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1656997928; x=1688533928; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=uS+4zAhU3LsHFG2nNclNJmjXA6SLYohmWdVIExa9sM0=; b=fkBh1KilHr/D2Crj1UjNLuIEHiwbaI0UBrJYZ27GxGl1GSnjonzZOsTc 3Tx6Rr1VPhyePf0vLa+i1LrPSlAfSuGUwiTllKvUcUqUeJGdM8PO5BYkI su8dT22LGwozxjbNifMYUbCA4XZyhNxqrkrqEbNZEtlOAjnWYqFLpBPB+ QLnZEp7/9E9veSSn4THkKtzc0YUhU4T/WnvKFDoh+Mr6m0SPm7My4cQ/0 J/HDQ/SblWxRLIdIMojApSn+K7xJ0oS6l1S9s4UKxCuVNkVON0CmuBhXU n0rhekGGpJG6TZjha9lRYKPW/DJIg9Be6cDGwfmSGXtFaAi805X5W1KtD A==; X-IronPort-AV: E=McAfee;i="6400,9594,10398"; a="282011473" X-IronPort-AV: E=Sophos;i="5.92,245,1650956400"; d="scan'208";a="282011473" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Jul 2022 22:12:07 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.92,245,1650956400"; d="scan'208";a="542807494" Received: from allen-box.sh.intel.com ([10.239.159.48]) by orsmga003.jf.intel.com with ESMTP; 04 Jul 2022 22:12:03 -0700 From: Lu Baolu To: Joerg Roedel , Jason Gunthorpe , Christoph Hellwig , Kevin Tian , Ashok Raj , Will Deacon , Robin Murphy , Jean-Philippe Brucker , Dave Jiang , Vinod Koul Cc: Eric Auger , Liu Yi L , Jacob jun Pan , Zhangfei Gao , Zhu Tony , iommu@lists.linux.dev, linux-kernel@vger.kernel.org, Lu Baolu Subject: [PATCH v10 06/12] iommu/vt-d: Add SVA domain support Date: Tue, 5 Jul 2022 13:07:04 +0800 Message-Id: <20220705050710.2887204-7-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220705050710.2887204-1-baolu.lu@linux.intel.com> References: <20220705050710.2887204-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Add support for SVA domain allocation and provide an SVA-specific iommu_domain_ops. Signed-off-by: Lu Baolu Reviewed-by: Kevin Tian Tested-by: Tony Zhu Reviewed-by: Jason Gunthorpe Reviewed-by: Yi Liu --- include/linux/intel-iommu.h | 5 ++++ drivers/iommu/intel/iommu.c | 2 ++ drivers/iommu/intel/svm.c | 49 +++++++++++++++++++++++++++++++++++++ 3 files changed, 56 insertions(+) diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h index 31e3edc0fc7e..9007428a68f1 100644 --- a/include/linux/intel-iommu.h +++ b/include/linux/intel-iommu.h @@ -743,6 +743,7 @@ void intel_svm_unbind(struct iommu_sva *handle); u32 intel_svm_get_pasid(struct iommu_sva *handle); int intel_svm_page_response(struct device *dev, struct iommu_fault_event *= evt, struct iommu_page_response *msg); +struct iommu_domain *intel_svm_domain_alloc(void); =20 struct intel_svm_dev { struct list_head list; @@ -768,6 +769,10 @@ struct intel_svm { }; #else static inline void intel_svm_check(struct intel_iommu *iommu) {} +static inline struct iommu_domain *intel_svm_domain_alloc(void) +{ + return NULL; +} #endif =20 #ifdef CONFIG_INTEL_IOMMU_DEBUGFS diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c index 44016594831d..993a1ce509a8 100644 --- a/drivers/iommu/intel/iommu.c +++ b/drivers/iommu/intel/iommu.c @@ -4298,6 +4298,8 @@ static struct iommu_domain *intel_iommu_domain_alloc(= unsigned type) return domain; case IOMMU_DOMAIN_IDENTITY: return &si_domain->domain; + case IOMMU_DOMAIN_SVA: + return intel_svm_domain_alloc(); default: return NULL; } diff --git a/drivers/iommu/intel/svm.c b/drivers/iommu/intel/svm.c index d04880a291c3..7d4f9d173013 100644 --- a/drivers/iommu/intel/svm.c +++ b/drivers/iommu/intel/svm.c @@ -931,3 +931,52 @@ int intel_svm_page_response(struct device *dev, mutex_unlock(&pasid_mutex); return ret; } + +static int intel_svm_set_dev_pasid(struct iommu_domain *domain, + struct device *dev, ioasid_t pasid) +{ + struct device_domain_info *info =3D dev_iommu_priv_get(dev); + struct intel_iommu *iommu =3D info->iommu; + struct mm_struct *mm =3D domain->mm; + struct iommu_sva *sva; + int ret =3D 0; + + mutex_lock(&pasid_mutex); + sva =3D intel_svm_bind_mm(iommu, dev, mm); + if (IS_ERR(sva)) + ret =3D PTR_ERR(sva); + mutex_unlock(&pasid_mutex); + + return ret; +} + +static void intel_svm_block_dev_pasid(struct iommu_domain *domain, + struct device *dev, ioasid_t pasid) +{ + mutex_lock(&pasid_mutex); + intel_svm_unbind_mm(dev, pasid); + mutex_unlock(&pasid_mutex); +} + +static void intel_svm_domain_free(struct iommu_domain *domain) +{ + kfree(to_dmar_domain(domain)); +} + +static const struct iommu_domain_ops intel_svm_domain_ops =3D { + .set_dev_pasid =3D intel_svm_set_dev_pasid, + .block_dev_pasid =3D intel_svm_block_dev_pasid, + .free =3D intel_svm_domain_free, +}; + +struct iommu_domain *intel_svm_domain_alloc(void) +{ + struct dmar_domain *domain; + + domain =3D kzalloc(sizeof(*domain), GFP_KERNEL); + if (!domain) + return NULL; + domain->domain.ops =3D &intel_svm_domain_ops; + + return &domain->domain; +} --=20 2.25.1 From nobody Sun Apr 19 09:04:37 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0084AC43334 for ; Tue, 5 Jul 2022 05:12:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229932AbiGEFMj (ORCPT ); Tue, 5 Jul 2022 01:12:39 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38426 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229812AbiGEFMa (ORCPT ); Tue, 5 Jul 2022 01:12:30 -0400 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CA8FE13D53 for ; Mon, 4 Jul 2022 22:12:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1656997932; x=1688533932; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=S5whMXXGsszBDylNhVPSI8VZBjv9fDAEEUSTnAkbGgY=; b=gZVp+hdxjYLRGpLtuGCmKGVo2HKa8FFkqoWRsVSwMuDQCSkDF54dbnTF VP7Zt4y6E6DZNf1EhHjfZSoUycm1Nk5Zuwq18CHS/2H09k3bkRvWPfJHN z7vIf2kbHE78rEzTALmhepatKi5jYVc8dDhQ6WevFyEJ2QnD9uQIzilgs WHrIPIOz5yr270IldffyX3aTNkSHXfZq5zTqyZCBK84CSXhTZ29oHVAee 6obFBc/gr7EKlZs8oH3z0CmmnyWu7IaufRPyGBvrq8lG2p5cRtGGBCGiQ Kkmbw2+M9Kf6kZU1GbqLMN/h2ZIqx9ghXtO1iMFGTIoI6/HGd2AYtumcD g==; X-IronPort-AV: E=McAfee;i="6400,9594,10398"; a="282011481" X-IronPort-AV: E=Sophos;i="5.92,245,1650956400"; d="scan'208";a="282011481" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Jul 2022 22:12:12 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.92,245,1650956400"; d="scan'208";a="542807526" Received: from allen-box.sh.intel.com ([10.239.159.48]) by orsmga003.jf.intel.com with ESMTP; 04 Jul 2022 22:12:07 -0700 From: Lu Baolu To: Joerg Roedel , Jason Gunthorpe , Christoph Hellwig , Kevin Tian , Ashok Raj , Will Deacon , Robin Murphy , Jean-Philippe Brucker , Dave Jiang , Vinod Koul Cc: Eric Auger , Liu Yi L , Jacob jun Pan , Zhangfei Gao , Zhu Tony , iommu@lists.linux.dev, linux-kernel@vger.kernel.org, Lu Baolu , Jean-Philippe Brucker Subject: [PATCH v10 07/12] arm-smmu-v3/sva: Add SVA domain support Date: Tue, 5 Jul 2022 13:07:05 +0800 Message-Id: <20220705050710.2887204-8-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220705050710.2887204-1-baolu.lu@linux.intel.com> References: <20220705050710.2887204-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Add support for SVA domain allocation and provide an SVA-specific iommu_domain_ops. Signed-off-by: Lu Baolu Reviewed-by: Jean-Philippe Brucker Tested-by: Zhangfei Gao --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 6 ++ .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 69 +++++++++++++++++++ drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 3 + 3 files changed, 78 insertions(+) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/ar= m/arm-smmu-v3/arm-smmu-v3.h index d2ba86470c42..96399dd3a67a 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -758,6 +758,7 @@ struct iommu_sva *arm_smmu_sva_bind(struct device *dev,= struct mm_struct *mm); void arm_smmu_sva_unbind(struct iommu_sva *handle); u32 arm_smmu_sva_get_pasid(struct iommu_sva *handle); void arm_smmu_sva_notifier_synchronize(void); +struct iommu_domain *arm_smmu_sva_domain_alloc(void); #else /* CONFIG_ARM_SMMU_V3_SVA */ static inline bool arm_smmu_sva_supported(struct arm_smmu_device *smmu) { @@ -803,5 +804,10 @@ static inline u32 arm_smmu_sva_get_pasid(struct iommu_= sva *handle) } =20 static inline void arm_smmu_sva_notifier_synchronize(void) {} + +static inline struct iommu_domain *arm_smmu_sva_domain_alloc(void) +{ + return NULL; +} #endif /* CONFIG_ARM_SMMU_V3_SVA */ #endif /* _ARM_SMMU_V3_H */ diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iomm= u/arm/arm-smmu-v3/arm-smmu-v3-sva.c index f155d406c5d5..fc4555dac5b4 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -549,3 +549,72 @@ void arm_smmu_sva_notifier_synchronize(void) */ mmu_notifier_synchronize(); } + +static int arm_smmu_sva_set_dev_pasid(struct iommu_domain *domain, + struct device *dev, ioasid_t id) +{ + int ret =3D 0; + struct mm_struct *mm; + struct iommu_sva *handle; + + if (domain->type !=3D IOMMU_DOMAIN_SVA) + return -EINVAL; + + mm =3D domain->mm; + if (WARN_ON(!mm)) + return -ENODEV; + + mutex_lock(&sva_lock); + handle =3D __arm_smmu_sva_bind(dev, mm); + if (IS_ERR(handle)) + ret =3D PTR_ERR(handle); + mutex_unlock(&sva_lock); + + return ret; +} + +static void arm_smmu_sva_block_dev_pasid(struct iommu_domain *domain, + struct device *dev, ioasid_t id) +{ + struct mm_struct *mm =3D domain->mm; + struct arm_smmu_bond *bond =3D NULL, *t; + struct arm_smmu_master *master =3D dev_iommu_priv_get(dev); + + mutex_lock(&sva_lock); + list_for_each_entry(t, &master->bonds, list) { + if (t->mm =3D=3D mm) { + bond =3D t; + break; + } + } + + if (!WARN_ON(!bond) && refcount_dec_and_test(&bond->refs)) { + list_del(&bond->list); + arm_smmu_mmu_notifier_put(bond->smmu_mn); + kfree(bond); + } + mutex_unlock(&sva_lock); +} + +static void arm_smmu_sva_domain_free(struct iommu_domain *domain) +{ + kfree(domain); +} + +static const struct iommu_domain_ops arm_smmu_sva_domain_ops =3D { + .set_dev_pasid =3D arm_smmu_sva_set_dev_pasid, + .block_dev_pasid =3D arm_smmu_sva_block_dev_pasid, + .free =3D arm_smmu_sva_domain_free, +}; + +struct iommu_domain *arm_smmu_sva_domain_alloc(void) +{ + struct iommu_domain *domain; + + domain =3D kzalloc(sizeof(*domain), GFP_KERNEL); + if (!domain) + return NULL; + domain->ops =3D &arm_smmu_sva_domain_ops; + + return domain; +} diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/ar= m/arm-smmu-v3/arm-smmu-v3.c index ae8ec8df47c1..a30b252e2f95 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -1999,6 +1999,9 @@ static struct iommu_domain *arm_smmu_domain_alloc(uns= igned type) { struct arm_smmu_domain *smmu_domain; =20 + if (type =3D=3D IOMMU_DOMAIN_SVA) + return arm_smmu_sva_domain_alloc(); + if (type !=3D IOMMU_DOMAIN_UNMANAGED && type !=3D IOMMU_DOMAIN_DMA && type !=3D IOMMU_DOMAIN_DMA_FQ && --=20 2.25.1 From nobody Sun Apr 19 09:04:37 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8B4A0C433EF for ; Tue, 5 Jul 2022 05:12:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229690AbiGEFM4 (ORCPT ); Tue, 5 Jul 2022 01:12:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39138 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229842AbiGEFMc (ORCPT ); Tue, 5 Jul 2022 01:12:32 -0400 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 11D4E13D68 for ; Mon, 4 Jul 2022 22:12:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1656997938; x=1688533938; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=/Wkbh+Rc4TZi3JFWm20vngP/XAx76T8eahnGT1VYNro=; b=YeTaIiQ93bCJ9yctnxEg8uE1kCchFZ+eAx4E3dSfxxxhbBoPN3SuKl+/ OwDNWaQOkUiQS4LFZGHggQ21xXWoOFFKX+eFPczRZy6sa4ZNQqujF67Nw vDOBwy9iYO27E2rWueJez2vrYgQ3xHs0+IQ+lUMWF9VjrymaoqIfvIy3i m6dEbjzbZuPJAJQlqua2s0YkIgJYUBGi+726l6TqgBkJWfu1R7NhOvu4p cw2xJ+cdQI9fGm8VRpHyuONbOQSOp4bQATTt0i7vRUtzsbjhRL7/rmonE YxWhioXH6s3/398bVhJ/vlL5rSlWpG5jO6p+VyNU6wKA4oljPry+PAjAL Q==; X-IronPort-AV: E=McAfee;i="6400,9594,10398"; a="282011493" X-IronPort-AV: E=Sophos;i="5.92,245,1650956400"; d="scan'208";a="282011493" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Jul 2022 22:12:17 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.92,245,1650956400"; d="scan'208";a="542807547" Received: from allen-box.sh.intel.com ([10.239.159.48]) by orsmga003.jf.intel.com with ESMTP; 04 Jul 2022 22:12:12 -0700 From: Lu Baolu To: Joerg Roedel , Jason Gunthorpe , Christoph Hellwig , Kevin Tian , Ashok Raj , Will Deacon , Robin Murphy , Jean-Philippe Brucker , Dave Jiang , Vinod Koul Cc: Eric Auger , Liu Yi L , Jacob jun Pan , Zhangfei Gao , Zhu Tony , iommu@lists.linux.dev, linux-kernel@vger.kernel.org, Lu Baolu Subject: [PATCH v10 08/12] iommu/sva: Refactoring iommu_sva_bind/unbind_device() Date: Tue, 5 Jul 2022 13:07:06 +0800 Message-Id: <20220705050710.2887204-9-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220705050710.2887204-1-baolu.lu@linux.intel.com> References: <20220705050710.2887204-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The existing iommu SVA interfaces are implemented by calling the SVA specific iommu ops provided by the IOMMU drivers. There's no need for any SVA specific ops in iommu_ops vector anymore as we can achieve this through the generic attach/detach_dev_pasid domain ops. This refactors the IOMMU SVA interfaces implementation by using the set/block_pasid_dev ops and align them with the concept of the SVA iommu domain. Put the new SVA code in the sva related file in order to make it self-contained. Signed-off-by: Lu Baolu Tested-by: Zhangfei Gao Tested-by: Tony Zhu Reviewed-by: Jean-Philippe Brucker Reviewed-by: Kevin Tian --- include/linux/iommu.h | 67 +++++++++++-------- drivers/iommu/iommu-sva-lib.c | 98 ++++++++++++++++++++++++++++ drivers/iommu/iommu.c | 119 ++++++++-------------------------- 3 files changed, 165 insertions(+), 119 deletions(-) diff --git a/include/linux/iommu.h b/include/linux/iommu.h index 42f0418dc22c..f59b0ecd3995 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -39,7 +39,6 @@ struct device; struct iommu_domain; struct iommu_domain_ops; struct notifier_block; -struct iommu_sva; struct iommu_fault_event; struct iommu_dma_cookie; =20 @@ -57,6 +56,14 @@ struct iommu_domain_geometry { bool force_aperture; /* DMA only allowed in mappable range? */ }; =20 +/** + * struct iommu_sva - handle to a device-mm bond + */ +struct iommu_sva { + struct device *dev; + refcount_t users; +}; + /* Domain feature flags */ #define __IOMMU_DOMAIN_PAGING (1U << 0) /* Support for iommu_map/unmap */ #define __IOMMU_DOMAIN_DMA_API (1U << 1) /* Domain for use in DMA-API @@ -105,6 +112,7 @@ struct iommu_domain { }; struct { /* IOMMU_DOMAIN_SVA */ struct mm_struct *mm; + struct iommu_sva bond; }; }; }; @@ -638,13 +646,6 @@ struct iommu_fwspec { /* ATS is supported */ #define IOMMU_FWSPEC_PCI_RC_ATS (1 << 0) =20 -/** - * struct iommu_sva - handle to a device-mm bond - */ -struct iommu_sva { - struct device *dev; -}; - int iommu_fwspec_init(struct device *dev, struct fwnode_handle *iommu_fwno= de, const struct iommu_ops *ops); void iommu_fwspec_free(struct device *dev); @@ -685,11 +686,6 @@ int iommu_dev_enable_feature(struct device *dev, enum = iommu_dev_features f); int iommu_dev_disable_feature(struct device *dev, enum iommu_dev_features = f); bool iommu_dev_feature_enabled(struct device *dev, enum iommu_dev_features= f); =20 -struct iommu_sva *iommu_sva_bind_device(struct device *dev, - struct mm_struct *mm); -void iommu_sva_unbind_device(struct iommu_sva *handle); -u32 iommu_sva_get_pasid(struct iommu_sva *handle); - int iommu_device_use_default_domain(struct device *dev); void iommu_device_unuse_default_domain(struct device *dev); =20 @@ -703,6 +699,8 @@ int iommu_attach_device_pasid(struct iommu_domain *doma= in, struct device *dev, ioasid_t pasid); void iommu_detach_device_pasid(struct iommu_domain *domain, struct device = *dev, ioasid_t pasid); +struct iommu_domain * +iommu_get_domain_for_dev_pasid(struct device *dev, ioasid_t pasid); #else /* CONFIG_IOMMU_API */ =20 struct iommu_ops {}; @@ -1033,21 +1031,6 @@ iommu_dev_disable_feature(struct device *dev, enum i= ommu_dev_features feat) return -ENODEV; } =20 -static inline struct iommu_sva * -iommu_sva_bind_device(struct device *dev, struct mm_struct *mm) -{ - return NULL; -} - -static inline void iommu_sva_unbind_device(struct iommu_sva *handle) -{ -} - -static inline u32 iommu_sva_get_pasid(struct iommu_sva *handle) -{ - return IOMMU_PASID_INVALID; -} - static inline struct iommu_fwspec *dev_iommu_fwspec_get(struct device *dev) { return NULL; @@ -1093,6 +1076,12 @@ static inline void iommu_detach_device_pasid(struct = iommu_domain *domain, struct device *dev, ioasid_t pasid) { } + +static inline struct iommu_domain * +iommu_get_domain_for_dev_pasid(struct device *dev, ioasid_t pasid) +{ + return NULL; +} #endif /* CONFIG_IOMMU_API */ =20 /** @@ -1118,4 +1107,26 @@ void iommu_debugfs_setup(void); static inline void iommu_debugfs_setup(void) {} #endif =20 +#ifdef CONFIG_IOMMU_SVA +struct iommu_sva *iommu_sva_bind_device(struct device *dev, + struct mm_struct *mm); +void iommu_sva_unbind_device(struct iommu_sva *handle); +u32 iommu_sva_get_pasid(struct iommu_sva *handle); +#else +static inline struct iommu_sva * +iommu_sva_bind_device(struct device *dev, struct mm_struct *mm) +{ + return NULL; +} + +static inline void iommu_sva_unbind_device(struct iommu_sva *handle) +{ +} + +static inline u32 iommu_sva_get_pasid(struct iommu_sva *handle) +{ + return IOMMU_PASID_INVALID; +} +#endif /* CONFIG_IOMMU_SVA */ + #endif /* __LINUX_IOMMU_H */ diff --git a/drivers/iommu/iommu-sva-lib.c b/drivers/iommu/iommu-sva-lib.c index 106506143896..751366980232 100644 --- a/drivers/iommu/iommu-sva-lib.c +++ b/drivers/iommu/iommu-sva-lib.c @@ -4,6 +4,7 @@ */ #include #include +#include =20 #include "iommu-sva-lib.h" =20 @@ -69,3 +70,100 @@ struct mm_struct *iommu_sva_find(ioasid_t pasid) return ioasid_find(&iommu_sva_pasid, pasid, __mmget_not_zero); } EXPORT_SYMBOL_GPL(iommu_sva_find); + +/** + * iommu_sva_bind_device() - Bind a process address space to a device + * @dev: the device + * @mm: the mm to bind, caller must hold a reference to mm_users + * + * Create a bond between device and address space, allowing the device to = access + * the mm using the returned PASID. If a bond already exists between @devi= ce and + * @mm, it is returned and an additional reference is taken. Caller must c= all + * iommu_sva_unbind_device() to release each reference. + * + * iommu_dev_enable_feature(dev, IOMMU_DEV_FEAT_SVA) must be called first,= to + * initialize the required SVA features. + * + * On error, returns an ERR_PTR value. + */ +struct iommu_sva *iommu_sva_bind_device(struct device *dev, struct mm_stru= ct *mm) +{ + struct iommu_domain *domain; + ioasid_t max_pasids; + int ret =3D -EINVAL; + + max_pasids =3D dev->iommu->max_pasids; + if (!max_pasids) + return ERR_PTR(-EOPNOTSUPP); + + /* Allocate mm->pasid if necessary. */ + ret =3D iommu_sva_alloc_pasid(mm, 1, max_pasids - 1); + if (ret) + return ERR_PTR(ret); + + mutex_lock(&iommu_sva_lock); + /* Search for an existing domain. */ + domain =3D iommu_get_domain_for_dev_pasid(dev, mm->pasid); + if (domain) { + refcount_inc(&domain->bond.users); + goto out_success; + } + + /* Allocate a new domain and set it on device pasid. */ + domain =3D iommu_sva_domain_alloc(dev, mm); + if (!domain) { + ret =3D -ENOMEM; + goto out_unlock; + } + + ret =3D iommu_attach_device_pasid(domain, dev, mm->pasid); + if (ret) + goto out_free_domain; + domain->bond.dev =3D dev; + refcount_set(&domain->bond.users, 1); + +out_success: + mutex_unlock(&iommu_sva_lock); + return &domain->bond; + +out_free_domain: + iommu_domain_free(domain); +out_unlock: + mutex_unlock(&iommu_sva_lock); + + return ERR_PTR(ret); +} +EXPORT_SYMBOL_GPL(iommu_sva_bind_device); + +/** + * iommu_sva_unbind_device() - Remove a bond created with iommu_sva_bind_d= evice + * @handle: the handle returned by iommu_sva_bind_device() + * + * Put reference to a bond between device and address space. The device sh= ould + * not be issuing any more transaction for this PASID. All outstanding page + * requests for this PASID must have been flushed to the IOMMU. + */ +void iommu_sva_unbind_device(struct iommu_sva *handle) +{ + struct device *dev =3D handle->dev; + struct iommu_domain *domain =3D + container_of(handle, struct iommu_domain, bond); + ioasid_t pasid =3D iommu_sva_get_pasid(handle); + + mutex_lock(&iommu_sva_lock); + if (refcount_dec_and_test(&domain->bond.users)) { + iommu_detach_device_pasid(domain, dev, pasid); + iommu_domain_free(domain); + } + mutex_unlock(&iommu_sva_lock); +} +EXPORT_SYMBOL_GPL(iommu_sva_unbind_device); + +u32 iommu_sva_get_pasid(struct iommu_sva *handle) +{ + struct iommu_domain *domain =3D + container_of(handle, struct iommu_domain, bond); + + return domain->mm->pasid; +} +EXPORT_SYMBOL_GPL(iommu_sva_get_pasid); diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c index 10479c5e4d23..e1491eb3c7b6 100644 --- a/drivers/iommu/iommu.c +++ b/drivers/iommu/iommu.c @@ -2789,97 +2789,6 @@ bool iommu_dev_feature_enabled(struct device *dev, e= num iommu_dev_features feat) } EXPORT_SYMBOL_GPL(iommu_dev_feature_enabled); =20 -/** - * iommu_sva_bind_device() - Bind a process address space to a device - * @dev: the device - * @mm: the mm to bind, caller must hold a reference to it - * - * Create a bond between device and address space, allowing the device to = access - * the mm using the returned PASID. If a bond already exists between @devi= ce and - * @mm, it is returned and an additional reference is taken. Caller must c= all - * iommu_sva_unbind_device() to release each reference. - * - * iommu_dev_enable_feature(dev, IOMMU_DEV_FEAT_SVA) must be called first,= to - * initialize the required SVA features. - * - * On error, returns an ERR_PTR value. - */ -struct iommu_sva * -iommu_sva_bind_device(struct device *dev, struct mm_struct *mm) -{ - struct iommu_group *group; - struct iommu_sva *handle =3D ERR_PTR(-EINVAL); - const struct iommu_ops *ops =3D dev_iommu_ops(dev); - - if (!ops->sva_bind) - return ERR_PTR(-ENODEV); - - group =3D iommu_group_get(dev); - if (!group) - return ERR_PTR(-ENODEV); - - /* Ensure device count and domain don't change while we're binding */ - mutex_lock(&group->mutex); - - /* - * To keep things simple, SVA currently doesn't support IOMMU groups - * with more than one device. Existing SVA-capable systems are not - * affected by the problems that required IOMMU groups (lack of ACS - * isolation, device ID aliasing and other hardware issues). - */ - if (iommu_group_device_count(group) !=3D 1) - goto out_unlock; - - handle =3D ops->sva_bind(dev, mm); - -out_unlock: - mutex_unlock(&group->mutex); - iommu_group_put(group); - - return handle; -} -EXPORT_SYMBOL_GPL(iommu_sva_bind_device); - -/** - * iommu_sva_unbind_device() - Remove a bond created with iommu_sva_bind_d= evice - * @handle: the handle returned by iommu_sva_bind_device() - * - * Put reference to a bond between device and address space. The device sh= ould - * not be issuing any more transaction for this PASID. All outstanding page - * requests for this PASID must have been flushed to the IOMMU. - */ -void iommu_sva_unbind_device(struct iommu_sva *handle) -{ - struct iommu_group *group; - struct device *dev =3D handle->dev; - const struct iommu_ops *ops =3D dev_iommu_ops(dev); - - if (!ops->sva_unbind) - return; - - group =3D iommu_group_get(dev); - if (!group) - return; - - mutex_lock(&group->mutex); - ops->sva_unbind(handle); - mutex_unlock(&group->mutex); - - iommu_group_put(group); -} -EXPORT_SYMBOL_GPL(iommu_sva_unbind_device); - -u32 iommu_sva_get_pasid(struct iommu_sva *handle) -{ - const struct iommu_ops *ops =3D dev_iommu_ops(handle->dev); - - if (!ops->sva_get_pasid) - return IOMMU_PASID_INVALID; - - return ops->sva_get_pasid(handle); -} -EXPORT_SYMBOL_GPL(iommu_sva_get_pasid); - /* * Changes the default domain of an iommu group that has *only* one device * @@ -3366,3 +3275,31 @@ void iommu_detach_device_pasid(struct iommu_domain *= domain, struct device *dev, =20 iommu_group_put(group); } + +/* + * This is a variant of iommu_get_domain_for_dev(). It returns the existing + * domain attached to pasid of a device. It's only for internal use of the + * IOMMU subsystem. The caller must take care to avoid any possible + * use-after-free case. + */ +struct iommu_domain * +iommu_get_domain_for_dev_pasid(struct device *dev, ioasid_t pasid) +{ + struct iommu_domain *domain; + struct iommu_group *group; + + if (!pasid_valid(pasid)) + return NULL; + + group =3D iommu_group_get(dev); + if (!group) + return NULL; + /* + * The xarray protects its internal state with RCU. Hence the domain + * obtained is either NULL or fully formed. + */ + domain =3D xa_load(&group->pasid_array, pasid); + iommu_group_put(group); + + return domain; +} --=20 2.25.1 From nobody Sun Apr 19 09:04:37 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4FCAAC433EF for ; Tue, 5 Jul 2022 05:13:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229499AbiGEFNG (ORCPT ); Tue, 5 Jul 2022 01:13:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39202 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229903AbiGEFMe (ORCPT ); Tue, 5 Jul 2022 01:12:34 -0400 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6444913D09 for ; Mon, 4 Jul 2022 22:12:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1656997942; x=1688533942; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=bwPnzzOA7t43ekvh+eMUZgW0xFb6pe1jgcsqtTA2d2E=; b=FcAeiLPVdDdbyWcx37ezHY/yMPNpiwMiPimAjJjJCcNiQOnZleFwToU5 aXU4zfCi5tiVEW/0oZHyMMehMG6vBobLgw3SjSU7I14gNlb7Gmi3XyBq5 yL6CTjWqbmN8zzgmlRYv1vYm0sxvN3ey+oUuNjSwFF/na3cWW4pxtZ8tS F+fFX6nNZL51FQivAyVOFrW4dZHCgrxVz7Dl1VVtyyhTgLUQzk1/TeGQd uQUaGiVPj9oArpDnCnIPu7k+ohwl2di2mkzEvXqbknbG6qjsDhhU5L6Tr wuTpUVKhImF2Q5RPO1ItMTw052iSIvX82zLIKr+fU4zH/ZZ4LmCcvTbbr g==; X-IronPort-AV: E=McAfee;i="6400,9594,10398"; a="282011502" X-IronPort-AV: E=Sophos;i="5.92,245,1650956400"; d="scan'208";a="282011502" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Jul 2022 22:12:22 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.92,245,1650956400"; d="scan'208";a="542807578" Received: from allen-box.sh.intel.com ([10.239.159.48]) by orsmga003.jf.intel.com with ESMTP; 04 Jul 2022 22:12:17 -0700 From: Lu Baolu To: Joerg Roedel , Jason Gunthorpe , Christoph Hellwig , Kevin Tian , Ashok Raj , Will Deacon , Robin Murphy , Jean-Philippe Brucker , Dave Jiang , Vinod Koul Cc: Eric Auger , Liu Yi L , Jacob jun Pan , Zhangfei Gao , Zhu Tony , iommu@lists.linux.dev, linux-kernel@vger.kernel.org, Lu Baolu , Jean-Philippe Brucker Subject: [PATCH v10 09/12] iommu: Remove SVA related callbacks from iommu ops Date: Tue, 5 Jul 2022 13:07:07 +0800 Message-Id: <20220705050710.2887204-10-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220705050710.2887204-1-baolu.lu@linux.intel.com> References: <20220705050710.2887204-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" These ops'es have been replaced with the dev_attach/detach_pasid domain ops'es. There's no need for them anymore. Remove them to avoid dead code. Signed-off-by: Lu Baolu Reviewed-by: Jean-Philippe Brucker Reviewed-by: Kevin Tian Tested-by: Zhangfei Gao Tested-by: Tony Zhu Reviewed-by: Jason Gunthorpe Reviewed-by: Yi Liu --- include/linux/intel-iommu.h | 3 -- include/linux/iommu.h | 7 --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 16 ------ .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 40 --------------- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 3 -- drivers/iommu/intel/iommu.c | 3 -- drivers/iommu/intel/svm.c | 49 ------------------- 7 files changed, 121 deletions(-) diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h index 9007428a68f1..5bd19c95a926 100644 --- a/include/linux/intel-iommu.h +++ b/include/linux/intel-iommu.h @@ -738,9 +738,6 @@ struct intel_iommu *device_to_iommu(struct device *dev,= u8 *bus, u8 *devfn); extern void intel_svm_check(struct intel_iommu *iommu); extern int intel_svm_enable_prq(struct intel_iommu *iommu); extern int intel_svm_finish_prq(struct intel_iommu *iommu); -struct iommu_sva *intel_svm_bind(struct device *dev, struct mm_struct *mm); -void intel_svm_unbind(struct iommu_sva *handle); -u32 intel_svm_get_pasid(struct iommu_sva *handle); int intel_svm_page_response(struct device *dev, struct iommu_fault_event *= evt, struct iommu_page_response *msg); struct iommu_domain *intel_svm_domain_alloc(void); diff --git a/include/linux/iommu.h b/include/linux/iommu.h index f59b0ecd3995..ae0cfca064e6 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -227,9 +227,6 @@ struct iommu_iotlb_gather { * @dev_has/enable/disable_feat: per device entries to check/enable/disable * iommu specific features. * @dev_feat_enabled: check enabled feature - * @sva_bind: Bind process address space to device - * @sva_unbind: Unbind process address space from device - * @sva_get_pasid: Get PASID associated to a SVA handle * @page_response: handle page request response * @def_domain_type: device default domain type, return value: * - IOMMU_DOMAIN_IDENTITY: must use an identity domain @@ -263,10 +260,6 @@ struct iommu_ops { int (*dev_enable_feat)(struct device *dev, enum iommu_dev_features f); int (*dev_disable_feat)(struct device *dev, enum iommu_dev_features f); =20 - struct iommu_sva *(*sva_bind)(struct device *dev, struct mm_struct *mm); - void (*sva_unbind)(struct iommu_sva *handle); - u32 (*sva_get_pasid)(struct iommu_sva *handle); - int (*page_response)(struct device *dev, struct iommu_fault_event *evt, struct iommu_page_response *msg); diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/ar= m/arm-smmu-v3/arm-smmu-v3.h index 96399dd3a67a..15dd4c7e6d3a 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -754,9 +754,6 @@ bool arm_smmu_master_sva_enabled(struct arm_smmu_master= *master); int arm_smmu_master_enable_sva(struct arm_smmu_master *master); int arm_smmu_master_disable_sva(struct arm_smmu_master *master); bool arm_smmu_master_iopf_supported(struct arm_smmu_master *master); -struct iommu_sva *arm_smmu_sva_bind(struct device *dev, struct mm_struct *= mm); -void arm_smmu_sva_unbind(struct iommu_sva *handle); -u32 arm_smmu_sva_get_pasid(struct iommu_sva *handle); void arm_smmu_sva_notifier_synchronize(void); struct iommu_domain *arm_smmu_sva_domain_alloc(void); #else /* CONFIG_ARM_SMMU_V3_SVA */ @@ -790,19 +787,6 @@ static inline bool arm_smmu_master_iopf_supported(stru= ct arm_smmu_master *master return false; } =20 -static inline struct iommu_sva * -arm_smmu_sva_bind(struct device *dev, struct mm_struct *mm) -{ - return ERR_PTR(-ENODEV); -} - -static inline void arm_smmu_sva_unbind(struct iommu_sva *handle) {} - -static inline u32 arm_smmu_sva_get_pasid(struct iommu_sva *handle) -{ - return IOMMU_PASID_INVALID; -} - static inline void arm_smmu_sva_notifier_synchronize(void) {} =20 static inline struct iommu_domain *arm_smmu_sva_domain_alloc(void) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iomm= u/arm/arm-smmu-v3/arm-smmu-v3-sva.c index fc4555dac5b4..e36c689f56c5 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -344,11 +344,6 @@ __arm_smmu_sva_bind(struct device *dev, struct mm_stru= ct *mm) if (!bond) return ERR_PTR(-ENOMEM); =20 - /* Allocate a PASID for this mm if necessary */ - ret =3D iommu_sva_alloc_pasid(mm, 1, (1U << master->ssid_bits) - 1); - if (ret) - goto err_free_bond; - bond->mm =3D mm; bond->sva.dev =3D dev; refcount_set(&bond->refs, 1); @@ -367,41 +362,6 @@ __arm_smmu_sva_bind(struct device *dev, struct mm_stru= ct *mm) return ERR_PTR(ret); } =20 -struct iommu_sva *arm_smmu_sva_bind(struct device *dev, struct mm_struct *= mm) -{ - struct iommu_sva *handle; - struct iommu_domain *domain =3D iommu_get_domain_for_dev(dev); - struct arm_smmu_domain *smmu_domain =3D to_smmu_domain(domain); - - if (smmu_domain->stage !=3D ARM_SMMU_DOMAIN_S1) - return ERR_PTR(-EINVAL); - - mutex_lock(&sva_lock); - handle =3D __arm_smmu_sva_bind(dev, mm); - mutex_unlock(&sva_lock); - return handle; -} - -void arm_smmu_sva_unbind(struct iommu_sva *handle) -{ - struct arm_smmu_bond *bond =3D sva_to_bond(handle); - - mutex_lock(&sva_lock); - if (refcount_dec_and_test(&bond->refs)) { - list_del(&bond->list); - arm_smmu_mmu_notifier_put(bond->smmu_mn); - kfree(bond); - } - mutex_unlock(&sva_lock); -} - -u32 arm_smmu_sva_get_pasid(struct iommu_sva *handle) -{ - struct arm_smmu_bond *bond =3D sva_to_bond(handle); - - return bond->mm->pasid; -} - bool arm_smmu_sva_supported(struct arm_smmu_device *smmu) { unsigned long reg, fld; diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/ar= m/arm-smmu-v3/arm-smmu-v3.c index a30b252e2f95..8b9b78c7a67d 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -2855,9 +2855,6 @@ static struct iommu_ops arm_smmu_ops =3D { .dev_feat_enabled =3D arm_smmu_dev_feature_enabled, .dev_enable_feat =3D arm_smmu_dev_enable_feature, .dev_disable_feat =3D arm_smmu_dev_disable_feature, - .sva_bind =3D arm_smmu_sva_bind, - .sva_unbind =3D arm_smmu_sva_unbind, - .sva_get_pasid =3D arm_smmu_sva_get_pasid, .page_response =3D arm_smmu_page_response, .pgsize_bitmap =3D -1UL, /* Restricted during device attach */ .owner =3D THIS_MODULE, diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c index 993a1ce509a8..37d68eda1889 100644 --- a/drivers/iommu/intel/iommu.c +++ b/drivers/iommu/intel/iommu.c @@ -4921,9 +4921,6 @@ const struct iommu_ops intel_iommu_ops =3D { .def_domain_type =3D device_def_domain_type, .pgsize_bitmap =3D SZ_4K, #ifdef CONFIG_INTEL_IOMMU_SVM - .sva_bind =3D intel_svm_bind, - .sva_unbind =3D intel_svm_unbind, - .sva_get_pasid =3D intel_svm_get_pasid, .page_response =3D intel_svm_page_response, #endif .default_domain_ops =3D &(const struct iommu_domain_ops) { diff --git a/drivers/iommu/intel/svm.c b/drivers/iommu/intel/svm.c index 7d4f9d173013..db55b06cafdf 100644 --- a/drivers/iommu/intel/svm.c +++ b/drivers/iommu/intel/svm.c @@ -313,14 +313,6 @@ static int pasid_to_svm_sdev(struct device *dev, unsig= ned int pasid, return 0; } =20 -static int intel_svm_alloc_pasid(struct device *dev, struct mm_struct *mm) -{ - ioasid_t max_pasid =3D dev_is_pci(dev) ? - pci_max_pasids(to_pci_dev(dev)) : intel_pasid_max_id; - - return iommu_sva_alloc_pasid(mm, PASID_MIN, max_pasid - 1); -} - static struct iommu_sva *intel_svm_bind_mm(struct intel_iommu *iommu, struct device *dev, struct mm_struct *mm) @@ -809,47 +801,6 @@ static irqreturn_t prq_event_thread(int irq, void *d) return IRQ_RETVAL(handled); } =20 -struct iommu_sva *intel_svm_bind(struct device *dev, struct mm_struct *mm) -{ - struct intel_iommu *iommu =3D device_to_iommu(dev, NULL, NULL); - struct iommu_sva *sva; - int ret; - - mutex_lock(&pasid_mutex); - ret =3D intel_svm_alloc_pasid(dev, mm); - if (ret) { - mutex_unlock(&pasid_mutex); - return ERR_PTR(ret); - } - - sva =3D intel_svm_bind_mm(iommu, dev, mm); - mutex_unlock(&pasid_mutex); - - return sva; -} - -void intel_svm_unbind(struct iommu_sva *sva) -{ - struct intel_svm_dev *sdev =3D to_intel_svm_dev(sva); - - mutex_lock(&pasid_mutex); - intel_svm_unbind_mm(sdev->dev, sdev->pasid); - mutex_unlock(&pasid_mutex); -} - -u32 intel_svm_get_pasid(struct iommu_sva *sva) -{ - struct intel_svm_dev *sdev; - u32 pasid; - - mutex_lock(&pasid_mutex); - sdev =3D to_intel_svm_dev(sva); - pasid =3D sdev->pasid; - mutex_unlock(&pasid_mutex); - - return pasid; -} - int intel_svm_page_response(struct device *dev, struct iommu_fault_event *evt, struct iommu_page_response *msg) --=20 2.25.1 From nobody Sun Apr 19 09:04:37 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 13D6DC433EF for ; Tue, 5 Jul 2022 05:13:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229995AbiGEFNO (ORCPT ); Tue, 5 Jul 2022 01:13:14 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38426 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229917AbiGEFMg (ORCPT ); Tue, 5 Jul 2022 01:12:36 -0400 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6BF5C13DDB for ; Mon, 4 Jul 2022 22:12:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1656997947; x=1688533947; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=OWwHhF07L7X9H3VNJvCd6Ts1+Ao5qc0bFPamGJaYSpE=; b=aGVIPmIcUy1+rXyEwzcYTPB9uhhxyzTs9+7XwBguv9sdh3XdqOGBXuPw xe7yZerZEf7ldxvD+tToVBW2auI5PI7yJ8CoXuFwc5XCLeMfpdaTw7yhW p4gQELopN3Tzf4W9yWSdr15I0UMInYCth/Byk4nFVEsy9Krq0EKAvvjvm 64j1HEAKgV1t0ocvMd4rUMhuhl1zCAWqsldBzc9mzFM4RP0RBBQa3QmJI 6bKMd5ASsrLpHn4JlnCDvu5t7JuSjmXRVdeAm5ov6GKJ3oneGw4kJqV4f g6jfVkajgDde8/YyUauCNVpWzniE1bPfLCyTuOLw846wfhG0lW3jem8LA A==; X-IronPort-AV: E=McAfee;i="6400,9594,10398"; a="282011513" X-IronPort-AV: E=Sophos;i="5.92,245,1650956400"; d="scan'208";a="282011513" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Jul 2022 22:12:26 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.92,245,1650956400"; d="scan'208";a="542807605" Received: from allen-box.sh.intel.com ([10.239.159.48]) by orsmga003.jf.intel.com with ESMTP; 04 Jul 2022 22:12:22 -0700 From: Lu Baolu To: Joerg Roedel , Jason Gunthorpe , Christoph Hellwig , Kevin Tian , Ashok Raj , Will Deacon , Robin Murphy , Jean-Philippe Brucker , Dave Jiang , Vinod Koul Cc: Eric Auger , Liu Yi L , Jacob jun Pan , Zhangfei Gao , Zhu Tony , iommu@lists.linux.dev, linux-kernel@vger.kernel.org, Lu Baolu , Jean-Philippe Brucker Subject: [PATCH v10 10/12] iommu: Prepare IOMMU domain for IOPF Date: Tue, 5 Jul 2022 13:07:08 +0800 Message-Id: <20220705050710.2887204-11-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220705050710.2887204-1-baolu.lu@linux.intel.com> References: <20220705050710.2887204-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" This adds some mechanisms around the iommu_domain so that the I/O page fault handling framework could route a page fault to the domain and call the fault handler from it. Add pointers to the page fault handler and its private data in struct iommu_domain. The fault handler will be called with the private data as a parameter once a page fault is routed to the domain. Any kernel component which owns an iommu domain could install handler and its private parameter so that the page fault could be further routed and handled. This also prepares the SVA implementation to be the first consumer of the per-domain page fault handling model. The I/O page fault handler for SVA is copied to the SVA file with mmget_not_zero() added before mmap_read_lock(). Suggested-by: Jean-Philippe Brucker Signed-off-by: Lu Baolu Reviewed-by: Jean-Philippe Brucker Tested-by: Zhangfei Gao Tested-by: Tony Zhu Reviewed-by: Kevin Tian --- include/linux/iommu.h | 3 ++ drivers/iommu/iommu-sva-lib.h | 8 +++++ drivers/iommu/io-pgfault.c | 7 +++++ drivers/iommu/iommu-sva-lib.c | 58 +++++++++++++++++++++++++++++++++++ drivers/iommu/iommu.c | 4 +++ 5 files changed, 80 insertions(+) diff --git a/include/linux/iommu.h b/include/linux/iommu.h index ae0cfca064e6..47610f21d451 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -105,6 +105,9 @@ struct iommu_domain { unsigned long pgsize_bitmap; /* Bitmap of page sizes in use */ struct iommu_domain_geometry geometry; struct iommu_dma_cookie *iova_cookie; + enum iommu_page_response_code (*iopf_handler)(struct iommu_fault *fault, + void *data); + void *fault_data; union { struct { iommu_fault_handler_t handler; diff --git a/drivers/iommu/iommu-sva-lib.h b/drivers/iommu/iommu-sva-lib.h index 8909ea1094e3..1b3ace4b5863 100644 --- a/drivers/iommu/iommu-sva-lib.h +++ b/drivers/iommu/iommu-sva-lib.h @@ -26,6 +26,8 @@ int iopf_queue_flush_dev(struct device *dev); struct iopf_queue *iopf_queue_alloc(const char *name); void iopf_queue_free(struct iopf_queue *queue); int iopf_queue_discard_partial(struct iopf_queue *queue); +enum iommu_page_response_code +iommu_sva_handle_iopf(struct iommu_fault *fault, void *data); =20 #else /* CONFIG_IOMMU_SVA */ static inline int iommu_queue_iopf(struct iommu_fault *fault, void *cookie) @@ -63,5 +65,11 @@ static inline int iopf_queue_discard_partial(struct iopf= _queue *queue) { return -ENODEV; } + +static inline enum iommu_page_response_code +iommu_sva_handle_iopf(struct iommu_fault *fault, void *data) +{ + return IOMMU_PAGE_RESP_INVALID; +} #endif /* CONFIG_IOMMU_SVA */ #endif /* _IOMMU_SVA_LIB_H */ diff --git a/drivers/iommu/io-pgfault.c b/drivers/iommu/io-pgfault.c index 1df8c1dcae77..aee9e033012f 100644 --- a/drivers/iommu/io-pgfault.c +++ b/drivers/iommu/io-pgfault.c @@ -181,6 +181,13 @@ static void iopf_handle_group(struct work_struct *work) * request completes, outstanding faults will have been dealt with by the = time * the PASID is freed. * + * Any valid page fault will be eventually routed to an iommu domain and t= he + * page fault handler installed there will get called. The users of this + * handling framework should guarantee that the iommu domain could only be + * freed after the device has stopped generating page faults (or the iommu + * hardware has been set to block the page faults) and the pending page fa= ults + * have been flushed. + * * Return: 0 on success and <0 on error. */ int iommu_queue_iopf(struct iommu_fault *fault, void *cookie) diff --git a/drivers/iommu/iommu-sva-lib.c b/drivers/iommu/iommu-sva-lib.c index 751366980232..536d34855c74 100644 --- a/drivers/iommu/iommu-sva-lib.c +++ b/drivers/iommu/iommu-sva-lib.c @@ -167,3 +167,61 @@ u32 iommu_sva_get_pasid(struct iommu_sva *handle) return domain->mm->pasid; } EXPORT_SYMBOL_GPL(iommu_sva_get_pasid); + +/* + * I/O page fault handler for SVA + */ +enum iommu_page_response_code +iommu_sva_handle_iopf(struct iommu_fault *fault, void *data) +{ + vm_fault_t ret; + struct vm_area_struct *vma; + struct mm_struct *mm =3D data; + unsigned int access_flags =3D 0; + unsigned int fault_flags =3D FAULT_FLAG_REMOTE; + struct iommu_fault_page_request *prm =3D &fault->prm; + enum iommu_page_response_code status =3D IOMMU_PAGE_RESP_INVALID; + + if (!(prm->flags & IOMMU_FAULT_PAGE_REQUEST_PASID_VALID)) + return status; + + if (IS_ERR_OR_NULL(mm) || !mmget_not_zero(mm)) + return status; + + mmap_read_lock(mm); + + vma =3D find_extend_vma(mm, prm->addr); + if (!vma) + /* Unmapped area */ + goto out_put_mm; + + if (prm->perm & IOMMU_FAULT_PERM_READ) + access_flags |=3D VM_READ; + + if (prm->perm & IOMMU_FAULT_PERM_WRITE) { + access_flags |=3D VM_WRITE; + fault_flags |=3D FAULT_FLAG_WRITE; + } + + if (prm->perm & IOMMU_FAULT_PERM_EXEC) { + access_flags |=3D VM_EXEC; + fault_flags |=3D FAULT_FLAG_INSTRUCTION; + } + + if (!(prm->perm & IOMMU_FAULT_PERM_PRIV)) + fault_flags |=3D FAULT_FLAG_USER; + + if (access_flags & ~vma->vm_flags) + /* Access fault */ + goto out_put_mm; + + ret =3D handle_mm_fault(vma, prm->addr, fault_flags, NULL); + status =3D ret & VM_FAULT_ERROR ? IOMMU_PAGE_RESP_INVALID : + IOMMU_PAGE_RESP_SUCCESS; + +out_put_mm: + mmap_read_unlock(mm); + mmput(mm); + + return status; +} diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c index e1491eb3c7b6..c6e9c8e82771 100644 --- a/drivers/iommu/iommu.c +++ b/drivers/iommu/iommu.c @@ -29,6 +29,8 @@ #include #include =20 +#include "iommu-sva-lib.h" + static struct kset *iommu_group_kset; static DEFINE_IDA(iommu_group_ida); =20 @@ -3199,6 +3201,8 @@ struct iommu_domain *iommu_sva_domain_alloc(struct de= vice *dev, domain->type =3D IOMMU_DOMAIN_SVA; mmgrab(mm); domain->mm =3D mm; + domain->iopf_handler =3D iommu_sva_handle_iopf; + domain->fault_data =3D mm; =20 return domain; } --=20 2.25.1 From nobody Sun Apr 19 09:04:37 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 98BDAC433EF for ; Tue, 5 Jul 2022 05:13:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229711AbiGEFNW (ORCPT ); Tue, 5 Jul 2022 01:13:22 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39100 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229823AbiGEFMu (ORCPT ); Tue, 5 Jul 2022 01:12:50 -0400 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C93E613D25 for ; Mon, 4 Jul 2022 22:12:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1656997951; x=1688533951; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=PfwS+hiqc+eUZB9Ki3ZTXVTOzQ5KVaVKWRARMEXqlhs=; b=M068U6dnFINA2vjClORJAXM14dUrTzoiF14BrvirA2c1ITsXTnH0w0Qu GG05Yu7MnWnBqMmBi9gGxDgASTUqmgxbk7AxzY1jUYgRBBXPvUJJc5kzR QWTmrlEIos65TGr3yxkTAojZgBcCadNhLRTWydB+iZHvcm4zq6Igxpgz7 op4CHwlDbpabCVimA5ksvU/pOoiLFbEYSvTDlTJEoKU8ShOyqjkLMWsrU 6ugqqCkZMBX8v3iG38zc5riBFKNG+j6i4VD2lrVjcc82sLzElQHJZHIsi 5+YPPsezHRtHewilIfJ1SPykaPC1n8+ns4V0xECxBLvLBjEAaa61e3gNs A==; X-IronPort-AV: E=McAfee;i="6400,9594,10398"; a="282011518" X-IronPort-AV: E=Sophos;i="5.92,245,1650956400"; d="scan'208";a="282011518" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Jul 2022 22:12:31 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.92,245,1650956400"; d="scan'208";a="542807626" Received: from allen-box.sh.intel.com ([10.239.159.48]) by orsmga003.jf.intel.com with ESMTP; 04 Jul 2022 22:12:26 -0700 From: Lu Baolu To: Joerg Roedel , Jason Gunthorpe , Christoph Hellwig , Kevin Tian , Ashok Raj , Will Deacon , Robin Murphy , Jean-Philippe Brucker , Dave Jiang , Vinod Koul Cc: Eric Auger , Liu Yi L , Jacob jun Pan , Zhangfei Gao , Zhu Tony , iommu@lists.linux.dev, linux-kernel@vger.kernel.org, Lu Baolu , Jean-Philippe Brucker Subject: [PATCH v10 11/12] iommu: Per-domain I/O page fault handling Date: Tue, 5 Jul 2022 13:07:09 +0800 Message-Id: <20220705050710.2887204-12-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220705050710.2887204-1-baolu.lu@linux.intel.com> References: <20220705050710.2887204-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Tweak the I/O page fault handling framework to route the page faults to the domain and call the page fault handler retrieved from the domain. This makes the I/O page fault handling framework possible to serve more usage scenarios as long as they have an IOMMU domain and install a page fault handler in it. Some unused functions are also removed to avoid dead code. The iommu_get_domain_for_dev_pasid() which retrieves attached domain for a {device, PASID} pair is used. It will be used by the page fault handling framework which knows {device, PASID} reported from the iommu driver. We have a guarantee that the SVA domain doesn't go away during IOPF handling, because unbind() won't free the domain until all the pending page requests have been flushed from the pipeline. The drivers either call iopf_queue_flush_dev() explicitly, or in stall case, the device driver is required to flush all DMAs including stalled transactions before calling unbind(). This also renames iopf_handle_group() to iopf_handler() to avoid confusing. Signed-off-by: Lu Baolu Reviewed-by: Jean-Philippe Brucker Tested-by: Zhangfei Gao Tested-by: Tony Zhu Reviewed-by: Jason Gunthorpe Reviewed-by: Kevin Tian --- drivers/iommu/io-pgfault.c | 68 +++++--------------------------------- 1 file changed, 9 insertions(+), 59 deletions(-) diff --git a/drivers/iommu/io-pgfault.c b/drivers/iommu/io-pgfault.c index aee9e033012f..d1c522f4ab34 100644 --- a/drivers/iommu/io-pgfault.c +++ b/drivers/iommu/io-pgfault.c @@ -69,69 +69,18 @@ static int iopf_complete_group(struct device *dev, stru= ct iopf_fault *iopf, return iommu_page_response(dev, &resp); } =20 -static enum iommu_page_response_code -iopf_handle_single(struct iopf_fault *iopf) -{ - vm_fault_t ret; - struct mm_struct *mm; - struct vm_area_struct *vma; - unsigned int access_flags =3D 0; - unsigned int fault_flags =3D FAULT_FLAG_REMOTE; - struct iommu_fault_page_request *prm =3D &iopf->fault.prm; - enum iommu_page_response_code status =3D IOMMU_PAGE_RESP_INVALID; - - if (!(prm->flags & IOMMU_FAULT_PAGE_REQUEST_PASID_VALID)) - return status; - - mm =3D iommu_sva_find(prm->pasid); - if (IS_ERR_OR_NULL(mm)) - return status; - - mmap_read_lock(mm); - - vma =3D find_extend_vma(mm, prm->addr); - if (!vma) - /* Unmapped area */ - goto out_put_mm; - - if (prm->perm & IOMMU_FAULT_PERM_READ) - access_flags |=3D VM_READ; - - if (prm->perm & IOMMU_FAULT_PERM_WRITE) { - access_flags |=3D VM_WRITE; - fault_flags |=3D FAULT_FLAG_WRITE; - } - - if (prm->perm & IOMMU_FAULT_PERM_EXEC) { - access_flags |=3D VM_EXEC; - fault_flags |=3D FAULT_FLAG_INSTRUCTION; - } - - if (!(prm->perm & IOMMU_FAULT_PERM_PRIV)) - fault_flags |=3D FAULT_FLAG_USER; - - if (access_flags & ~vma->vm_flags) - /* Access fault */ - goto out_put_mm; - - ret =3D handle_mm_fault(vma, prm->addr, fault_flags, NULL); - status =3D ret & VM_FAULT_ERROR ? IOMMU_PAGE_RESP_INVALID : - IOMMU_PAGE_RESP_SUCCESS; - -out_put_mm: - mmap_read_unlock(mm); - mmput(mm); - - return status; -} - -static void iopf_handle_group(struct work_struct *work) +static void iopf_handler(struct work_struct *work) { struct iopf_group *group; + struct iommu_domain *domain; struct iopf_fault *iopf, *next; enum iommu_page_response_code status =3D IOMMU_PAGE_RESP_SUCCESS; =20 group =3D container_of(work, struct iopf_group, work); + domain =3D iommu_get_domain_for_dev_pasid(group->dev, + group->last_fault.fault.prm.pasid); + if (!domain || !domain->iopf_handler) + status =3D IOMMU_PAGE_RESP_INVALID; =20 list_for_each_entry_safe(iopf, next, &group->faults, list) { /* @@ -139,7 +88,8 @@ static void iopf_handle_group(struct work_struct *work) * faults in the group if there is an error. */ if (status =3D=3D IOMMU_PAGE_RESP_SUCCESS) - status =3D iopf_handle_single(iopf); + status =3D domain->iopf_handler(&iopf->fault, + domain->fault_data); =20 if (!(iopf->fault.prm.flags & IOMMU_FAULT_PAGE_REQUEST_LAST_PAGE)) @@ -242,7 +192,7 @@ int iommu_queue_iopf(struct iommu_fault *fault, void *c= ookie) group->last_fault.fault =3D *fault; INIT_LIST_HEAD(&group->faults); list_add(&group->last_fault.list, &group->faults); - INIT_WORK(&group->work, iopf_handle_group); + INIT_WORK(&group->work, iopf_handler); =20 /* See if we have partial faults for this group */ list_for_each_entry_safe(iopf, next, &iopf_param->partial, list) { --=20 2.25.1 From nobody Sun Apr 19 09:04:37 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 73F6BC433EF for ; Tue, 5 Jul 2022 05:13:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229733AbiGEFN2 (ORCPT ); Tue, 5 Jul 2022 01:13:28 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39146 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229848AbiGEFMw (ORCPT ); Tue, 5 Jul 2022 01:12:52 -0400 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8C71B13D3D for ; Mon, 4 Jul 2022 22:12:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1656997956; x=1688533956; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=2PIRrxrWkqooSJD1BFX6QzuVGWWS7p2CUAaeu4XMy5U=; b=E2otksIYBIXE5pIg3/EzXVJAPafK5Le5kDCmqCldgJV6k38R96LejWxi YrR1Y6G2fjSecaDQKz2p5pUT55lrnhEKJx6ZFFEV+EO0+UrL4KBdyO/pM lJ7J/vUofi0hh3od0c1jmKUIyOsqJzTa0LPzBJGTpcZzyomrJmS/9aR61 6hhXRMjGcfC/qf5MalucPDuKlccLBz9PSGbCOG56VwYBE4WbqbnpmRP0l qdCKGxMvlMy0w70U43lLyHlBLYHkeBqVUuuB3XNv/xYO0W0QLRm8y+Xwq SlQp7U4f4qlNzp31+qnvcthcSEsxwcQ6nKvoAhjTjGJdgaqtIbTwWQHSg Q==; X-IronPort-AV: E=McAfee;i="6400,9594,10398"; a="282011531" X-IronPort-AV: E=Sophos;i="5.92,245,1650956400"; d="scan'208";a="282011531" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Jul 2022 22:12:36 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.92,245,1650956400"; d="scan'208";a="542807653" Received: from allen-box.sh.intel.com ([10.239.159.48]) by orsmga003.jf.intel.com with ESMTP; 04 Jul 2022 22:12:31 -0700 From: Lu Baolu To: Joerg Roedel , Jason Gunthorpe , Christoph Hellwig , Kevin Tian , Ashok Raj , Will Deacon , Robin Murphy , Jean-Philippe Brucker , Dave Jiang , Vinod Koul Cc: Eric Auger , Liu Yi L , Jacob jun Pan , Zhangfei Gao , Zhu Tony , iommu@lists.linux.dev, linux-kernel@vger.kernel.org, Lu Baolu , Jean-Philippe Brucker Subject: [PATCH v10 12/12] iommu: Rename iommu-sva-lib.{c,h} Date: Tue, 5 Jul 2022 13:07:10 +0800 Message-Id: <20220705050710.2887204-13-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220705050710.2887204-1-baolu.lu@linux.intel.com> References: <20220705050710.2887204-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Rename iommu-sva-lib.c[h] to iommu-sva.c[h] as it contains all code for SVA implementation in iommu core. Signed-off-by: Lu Baolu Reviewed-by: Jean-Philippe Brucker Reviewed-by: Kevin Tian Tested-by: Zhangfei Gao Tested-by: Tony Zhu Reviewed-by: Jason Gunthorpe --- drivers/iommu/{iommu-sva-lib.h =3D> iommu-sva.h} | 6 +++--- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 2 +- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 2 +- drivers/iommu/intel/iommu.c | 2 +- drivers/iommu/intel/svm.c | 2 +- drivers/iommu/io-pgfault.c | 2 +- drivers/iommu/{iommu-sva-lib.c =3D> iommu-sva.c} | 2 +- drivers/iommu/iommu.c | 2 +- drivers/iommu/Makefile | 2 +- 9 files changed, 11 insertions(+), 11 deletions(-) rename drivers/iommu/{iommu-sva-lib.h =3D> iommu-sva.h} (95%) rename drivers/iommu/{iommu-sva-lib.c =3D> iommu-sva.c} (99%) diff --git a/drivers/iommu/iommu-sva-lib.h b/drivers/iommu/iommu-sva.h similarity index 95% rename from drivers/iommu/iommu-sva-lib.h rename to drivers/iommu/iommu-sva.h index 1b3ace4b5863..7215a761b962 100644 --- a/drivers/iommu/iommu-sva-lib.h +++ b/drivers/iommu/iommu-sva.h @@ -2,8 +2,8 @@ /* * SVA library for IOMMU drivers */ -#ifndef _IOMMU_SVA_LIB_H -#define _IOMMU_SVA_LIB_H +#ifndef _IOMMU_SVA_H +#define _IOMMU_SVA_H =20 #include #include @@ -72,4 +72,4 @@ iommu_sva_handle_iopf(struct iommu_fault *fault, void *da= ta) return IOMMU_PAGE_RESP_INVALID; } #endif /* CONFIG_IOMMU_SVA */ -#endif /* _IOMMU_SVA_LIB_H */ +#endif /* _IOMMU_SVA_H */ diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iomm= u/arm/arm-smmu-v3/arm-smmu-v3-sva.c index e36c689f56c5..b33bc592ccfa 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -10,7 +10,7 @@ #include =20 #include "arm-smmu-v3.h" -#include "../../iommu-sva-lib.h" +#include "../../iommu-sva.h" #include "../../io-pgtable-arm.h" =20 struct arm_smmu_mmu_notifier { diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/ar= m/arm-smmu-v3/arm-smmu-v3.c index 8b9b78c7a67d..79e8991e9181 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -31,7 +31,7 @@ #include =20 #include "arm-smmu-v3.h" -#include "../../iommu-sva-lib.h" +#include "../../iommu-sva.h" =20 static bool disable_bypass =3D true; module_param(disable_bypass, bool, 0444); diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c index 37d68eda1889..d16ab6d1cc05 100644 --- a/drivers/iommu/intel/iommu.c +++ b/drivers/iommu/intel/iommu.c @@ -27,7 +27,7 @@ #include =20 #include "../irq_remapping.h" -#include "../iommu-sva-lib.h" +#include "../iommu-sva.h" #include "pasid.h" #include "cap_audit.h" =20 diff --git a/drivers/iommu/intel/svm.c b/drivers/iommu/intel/svm.c index db55b06cafdf..58656a93b201 100644 --- a/drivers/iommu/intel/svm.c +++ b/drivers/iommu/intel/svm.c @@ -25,7 +25,7 @@ =20 #include "pasid.h" #include "perf.h" -#include "../iommu-sva-lib.h" +#include "../iommu-sva.h" =20 static irqreturn_t prq_event_thread(int irq, void *d); static void intel_svm_drain_prq(struct device *dev, u32 pasid); diff --git a/drivers/iommu/io-pgfault.c b/drivers/iommu/io-pgfault.c index d1c522f4ab34..7a60b123e6b9 100644 --- a/drivers/iommu/io-pgfault.c +++ b/drivers/iommu/io-pgfault.c @@ -11,7 +11,7 @@ #include #include =20 -#include "iommu-sva-lib.h" +#include "iommu-sva.h" =20 /** * struct iopf_queue - IO Page Fault queue diff --git a/drivers/iommu/iommu-sva-lib.c b/drivers/iommu/iommu-sva.c similarity index 99% rename from drivers/iommu/iommu-sva-lib.c rename to drivers/iommu/iommu-sva.c index 536d34855c74..21ffbf1ac39e 100644 --- a/drivers/iommu/iommu-sva-lib.c +++ b/drivers/iommu/iommu-sva.c @@ -6,7 +6,7 @@ #include #include =20 -#include "iommu-sva-lib.h" +#include "iommu-sva.h" =20 static DEFINE_MUTEX(iommu_sva_lock); static DECLARE_IOASID_SET(iommu_sva_pasid); diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c index c6e9c8e82771..8b9350e10d0b 100644 --- a/drivers/iommu/iommu.c +++ b/drivers/iommu/iommu.c @@ -29,7 +29,7 @@ #include #include =20 -#include "iommu-sva-lib.h" +#include "iommu-sva.h" =20 static struct kset *iommu_group_kset; static DEFINE_IDA(iommu_group_ida); diff --git a/drivers/iommu/Makefile b/drivers/iommu/Makefile index 44475a9b3eea..c1763476162b 100644 --- a/drivers/iommu/Makefile +++ b/drivers/iommu/Makefile @@ -27,6 +27,6 @@ obj-$(CONFIG_FSL_PAMU) +=3D fsl_pamu.o fsl_pamu_domain.o obj-$(CONFIG_S390_IOMMU) +=3D s390-iommu.o obj-$(CONFIG_HYPERV_IOMMU) +=3D hyperv-iommu.o obj-$(CONFIG_VIRTIO_IOMMU) +=3D virtio-iommu.o -obj-$(CONFIG_IOMMU_SVA) +=3D iommu-sva-lib.o io-pgfault.o +obj-$(CONFIG_IOMMU_SVA) +=3D iommu-sva.o io-pgfault.o obj-$(CONFIG_SPRD_IOMMU) +=3D sprd-iommu.o obj-$(CONFIG_APPLE_DART) +=3D apple-dart.o --=20 2.25.1