From nobody Fri Jun 19 21:27:52 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6438FC433FE for ; Tue, 29 Mar 2022 05:40:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232511AbiC2FmB (ORCPT ); Tue, 29 Mar 2022 01:42:01 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55334 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231612AbiC2Fl7 (ORCPT ); Tue, 29 Mar 2022 01:41:59 -0400 Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1044131DCB for ; Mon, 28 Mar 2022 22:40:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1648532417; x=1680068417; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=ntEI0llQ2lWEuMkMlCRDDWhOOcAZFhuhyEbyMV63nuo=; b=BJ5HTCW7Yhq+TLJc25r044cFFKrAG+ZbITgQmMNcVlN9iJrXnNGmvPqL lBpKavMQjJbRdzGmbTJXaAnTxGOaDLDIZ7kd8cnDIFwE2ehW44voqw89z vUJEpcgrnFNRPOu7+IkDBOFmhRvlDVouwYzJ7LdmsV4QKtPDWp6ZhSkOn Yqx3IeetDOFw4AmoAnmd7LXkCSxbwvamu39HX3L5eZfGPnJSYRfDQca1D 6hoyAOKroXppHCNFxsnB2WYHC3VYRwAyY+ifw7blec836KulUpwreQ4sM HGarmpaCrm5lCjaRSNmS8O9QD1AUktzspMIsSCW1hvOyYD+aF41XpSVGx Q==; X-IronPort-AV: E=McAfee;i="6200,9189,10300"; a="259136963" X-IronPort-AV: E=Sophos;i="5.90,219,1643702400"; d="scan'208";a="259136963" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Mar 2022 22:40:16 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.90,219,1643702400"; d="scan'208";a="694603573" Received: from allen-box.sh.intel.com ([10.239.159.48]) by fmsmga001.fm.intel.com with ESMTP; 28 Mar 2022 22:40:13 -0700 From: Lu Baolu To: Joerg Roedel , Jason Gunthorpe , Christoph Hellwig , Kevin Tian , Ashok Raj , Will Deacon , Robin Murphy , Jean-Philippe Brucker Cc: Eric Auger , Liu Yi L , Jacob jun Pan , iommu@lists.linux-foundation.org, linux-kernel@vger.kernel.org, Lu Baolu Subject: [PATCH RFC v2 01/11] iommu: Add pasid_bits field in struct dev_iommu Date: Tue, 29 Mar 2022 13:37:50 +0800 Message-Id: <20220329053800.3049561-2-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220329053800.3049561-1-baolu.lu@linux.intel.com> References: <20220329053800.3049561-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Use this field to save the pasid/ssid bits that a device is able to support with its IOMMU hardware. It is a generic attribute of a device and lifting it into the per-device dev_iommu struct makes it possible to allocate a PASID for device without calls into the IOMMU drivers. Any iommu driver which suports PASID related features should set this field before features are enabled on the devices. Signed-off-by: Lu Baolu --- include/linux/iommu.h | 1 + drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 2 ++ drivers/iommu/intel/iommu.c | 5 ++++- 3 files changed, 7 insertions(+), 1 deletion(-) diff --git a/include/linux/iommu.h b/include/linux/iommu.h index 6ef2df258673..36f43af0af53 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -368,6 +368,7 @@ struct dev_iommu { struct iommu_fwspec *fwspec; struct iommu_device *iommu_dev; void *priv; + unsigned int pasid_bits; }; =20 int iommu_device_register(struct iommu_device *iommu, diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/ar= m/arm-smmu-v3/arm-smmu-v3.c index 627a3ed5ee8f..afc63fce6107 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -2681,6 +2681,8 @@ static struct iommu_device *arm_smmu_probe_device(str= uct device *dev) smmu->features & ARM_SMMU_FEAT_STALL_FORCE) master->stall_enabled =3D true; =20 + dev->iommu->pasid_bits =3D master->ssid_bits; + return &smmu->iommu; =20 err_free_master: diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c index 6f7485c44a4b..c1b91bce1530 100644 --- a/drivers/iommu/intel/iommu.c +++ b/drivers/iommu/intel/iommu.c @@ -4587,8 +4587,11 @@ static struct iommu_device *intel_iommu_probe_device= (struct device *dev) if (pasid_supported(iommu)) { int features =3D pci_pasid_features(pdev); =20 - if (features >=3D 0) + if (features >=3D 0) { info->pasid_supported =3D features | 1; + dev->iommu->pasid_bits =3D + fls(pci_max_pasids(pdev)) - 1; + } } =20 if (info->ats_supported && ecap_prs(iommu->ecap) && --=20 2.25.1 From nobody Fri Jun 19 21:27:52 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 539FAC4332F for ; Tue, 29 Mar 2022 05:40:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232532AbiC2FmG (ORCPT ); Tue, 29 Mar 2022 01:42:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55590 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232519AbiC2FmC (ORCPT ); Tue, 29 Mar 2022 01:42:02 -0400 Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6B52531935 for ; Mon, 28 Mar 2022 22:40:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1648532420; x=1680068420; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Hl0RI/UgCJ86U9MWOz/qKbzv7cGWyLDs3VmWUCZbxMU=; b=kBnlTwIM8aKiCnN+nJLuMhtMPhf/vttZSYhRjhNIRJT+6iBNebYfWROh GXMc5fNta7iXoPkN0GhkT3ZIC7KIM1yE85AMRiN5NdTC0S2ophsCBsQyb mzlpnLyuqnYisyal0wIMr7iVP4AfueMyzJpt9Xf1YAidUrZRw0HDBCbIL EuM+K2Zhh4YvWzmtyqRUWolhT21bqMeb4CKI9eMCTGYrKbK4KP2PS4U4h 7QoC101Lvd9obkIzQBDg3JgG3TUCa5gBpN+OH7jA2evYvms9P0gjr26JV LmkOM8HIQ9eRv3+70bFpYcDKAkrnFycKB+5ROYZnxsg5FvPda26lEbviY Q==; X-IronPort-AV: E=McAfee;i="6200,9189,10300"; a="259136969" X-IronPort-AV: E=Sophos;i="5.90,219,1643702400"; d="scan'208";a="259136969" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Mar 2022 22:40:20 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.90,219,1643702400"; d="scan'208";a="694603582" Received: from allen-box.sh.intel.com ([10.239.159.48]) by fmsmga001.fm.intel.com with ESMTP; 28 Mar 2022 22:40:16 -0700 From: Lu Baolu To: Joerg Roedel , Jason Gunthorpe , Christoph Hellwig , Kevin Tian , Ashok Raj , Will Deacon , Robin Murphy , Jean-Philippe Brucker Cc: Eric Auger , Liu Yi L , Jacob jun Pan , iommu@lists.linux-foundation.org, linux-kernel@vger.kernel.org, Lu Baolu Subject: [PATCH RFC v2 02/11] iommu: Add iommu_group_singleton_lockdown() Date: Tue, 29 Mar 2022 13:37:51 +0800 Message-Id: <20220329053800.3049561-3-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220329053800.3049561-1-baolu.lu@linux.intel.com> References: <20220329053800.3049561-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Some of the interfaces in the IOMMU core require that only a single kernel device driver controls the device in the IOMMU group. The existing method is to check the device count in the IOMMU group in the interfaces. This is unreliable because any device added to the IOMMU group later breaks this assumption without notifying the driver using the interface. This adds iommu_group_singleton_lockdown() that checks the requirement and locks down the IOMMU group with only single device driver bound. Signed-off-by: Lu Baolu --- drivers/iommu/iommu.c | 30 ++++++++++++++++++------------ 1 file changed, 18 insertions(+), 12 deletions(-) diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c index 0c42ece25854..9fb8a5b4491e 100644 --- a/drivers/iommu/iommu.c +++ b/drivers/iommu/iommu.c @@ -48,6 +48,7 @@ struct iommu_group { struct list_head entry; unsigned int owner_cnt; void *owner; + bool singleton_lockdown; }; =20 struct group_device { @@ -968,15 +969,16 @@ void iommu_group_remove_device(struct device *dev) } EXPORT_SYMBOL_GPL(iommu_group_remove_device); =20 -static int iommu_group_device_count(struct iommu_group *group) +/* Callers should hold the group->mutex. */ +static bool iommu_group_singleton_lockdown(struct iommu_group *group) { - struct group_device *entry; - int ret =3D 0; - - list_for_each_entry(entry, &group->devices, list) - ret++; + if (group->owner_cnt !=3D 1 || + group->domain !=3D group->default_domain || + group->owner) + return false; + group->singleton_lockdown =3D true; =20 - return ret; + return true; } =20 static int __iommu_group_for_each_dev(struct iommu_group *group, void *dat= a, @@ -1936,7 +1938,7 @@ int iommu_attach_device(struct iommu_domain *domain, = struct device *dev) */ mutex_lock(&group->mutex); ret =3D -EINVAL; - if (iommu_group_device_count(group) !=3D 1) + if (!iommu_group_singleton_lockdown(group)) goto out_unlock; =20 ret =3D __iommu_attach_group(domain, group); @@ -1979,7 +1981,7 @@ void iommu_detach_device(struct iommu_domain *domain,= struct device *dev) return; =20 mutex_lock(&group->mutex); - if (iommu_group_device_count(group) !=3D 1) { + if (!iommu_group_singleton_lockdown(group)) { WARN_ON(1); goto out_unlock; } @@ -2745,7 +2747,7 @@ iommu_sva_bind_device(struct device *dev, struct mm_s= truct *mm, void *drvdata) * affected by the problems that required IOMMU groups (lack of ACS * isolation, device ID aliasing and other hardware issues). */ - if (iommu_group_device_count(group) !=3D 1) + if (!iommu_group_singleton_lockdown(group)) goto out_unlock; =20 handle =3D ops->sva_bind(dev, mm, drvdata); @@ -2842,7 +2844,7 @@ static int iommu_change_dev_def_domain(struct iommu_g= roup *group, * case) that has already acquired some of the device locks and might be * waiting for T1 to release other device locks. */ - if (iommu_group_device_count(group) !=3D 1) { + if (!iommu_group_singleton_lockdown(group)) { dev_err_ratelimited(prev_dev, "Cannot change default domain: Group has m= ore than one device\n"); ret =3D -EINVAL; goto out; @@ -2975,7 +2977,7 @@ static ssize_t iommu_group_store_type(struct iommu_gr= oup *group, * 2. Get struct *dev which is needed to lock device */ mutex_lock(&group->mutex); - if (iommu_group_device_count(group) !=3D 1) { + if (!iommu_group_singleton_lockdown(group)) { mutex_unlock(&group->mutex); pr_err_ratelimited("Cannot change default domain: Group has more than on= e device\n"); return -EINVAL; @@ -3050,6 +3052,7 @@ int iommu_device_use_default_domain(struct device *de= v) mutex_lock(&group->mutex); if (group->owner_cnt) { if (group->domain !=3D group->default_domain || + group->singleton_lockdown || group->owner) { ret =3D -EBUSY; goto unlock_out; @@ -3084,6 +3087,9 @@ void iommu_device_unuse_default_domain(struct device = *dev) if (!WARN_ON(!group->owner_cnt)) group->owner_cnt--; =20 + if (!group->owner_cnt) + group->singleton_lockdown =3D false; + mutex_unlock(&group->mutex); iommu_group_put(group); } --=20 2.25.1 From nobody Fri Jun 19 21:27:52 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0598CC433FE for ; Tue, 29 Mar 2022 05:40:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232535AbiC2FmT (ORCPT ); Tue, 29 Mar 2022 01:42:19 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55916 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232538AbiC2FmJ (ORCPT ); Tue, 29 Mar 2022 01:42:09 -0400 Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8999523D45E for ; Mon, 28 Mar 2022 22:40:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1648532424; x=1680068424; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=AnPaIsnG/p3ivIG6I+bXTOQ6EzNmHvly5wxHWTR6dqE=; b=ViGbliQRiTxx+eDArgRxSvxi0wJQPCVTTxLCTdSgDHOJFNiQ8TQaI10q 52euLV4EviVBtlmJ+RHV+BgDCMczI5M/+vJ+aIzxBoEF6WDeGB8rvc5n7 1CQd3uZpUQOwZxHhI5XwI8REGXi+O3nTFteTFfjtL6m5sxJOB2j+fzi8v KQDag6AinxUMnRaQINd+Tza7Lbp+18LoVTVxQjGIbHL6TVZMPZUZFCtvZ xNIFhG/VV6ZXMw/c/pMhQOyLQtqL1zOyINymGKTNOoOqtWotgBP4mNIqC BjgO0KuslL9GOw+DK+vI0cLFvPlIABIcyVOxGz4fkbqX9QIxxYQlxU4Qr w==; X-IronPort-AV: E=McAfee;i="6200,9189,10300"; a="259136976" X-IronPort-AV: E=Sophos;i="5.90,219,1643702400"; d="scan'208";a="259136976" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Mar 2022 22:40:23 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.90,219,1643702400"; d="scan'208";a="694603589" Received: from allen-box.sh.intel.com ([10.239.159.48]) by fmsmga001.fm.intel.com with ESMTP; 28 Mar 2022 22:40:20 -0700 From: Lu Baolu To: Joerg Roedel , Jason Gunthorpe , Christoph Hellwig , Kevin Tian , Ashok Raj , Will Deacon , Robin Murphy , Jean-Philippe Brucker Cc: Eric Auger , Liu Yi L , Jacob jun Pan , iommu@lists.linux-foundation.org, linux-kernel@vger.kernel.org, Lu Baolu Subject: [PATCH RFC v2 03/11] iommu/sva: Add iommu_domain type for SVA Date: Tue, 29 Mar 2022 13:37:52 +0800 Message-Id: <20220329053800.3049561-4-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220329053800.3049561-1-baolu.lu@linux.intel.com> References: <20220329053800.3049561-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Add a new iommu domain type IOMMU_DOMAIN_SVA to represent an I/O page table which is shared from CPU host VA. Add some helpers to get and put an SVA domain and implement SVA domain life cycle management. Signed-off-by: Lu Baolu --- include/linux/iommu.h | 7 +++ drivers/iommu/iommu-sva-lib.h | 10 ++++ drivers/iommu/iommu-sva-lib.c | 89 +++++++++++++++++++++++++++++++++++ 3 files changed, 106 insertions(+) diff --git a/include/linux/iommu.h b/include/linux/iommu.h index 36f43af0af53..29c4c2edd706 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -42,6 +42,7 @@ struct notifier_block; struct iommu_sva; struct iommu_fault_event; struct iommu_dma_cookie; +struct iommu_sva_cookie; =20 /* iommu fault flags */ #define IOMMU_FAULT_READ 0x0 @@ -64,6 +65,9 @@ struct iommu_domain_geometry { #define __IOMMU_DOMAIN_PT (1U << 2) /* Domain is identity mapped */ #define __IOMMU_DOMAIN_DMA_FQ (1U << 3) /* DMA-API uses flush queue */ =20 +#define __IOMMU_DOMAIN_SHARED (1U << 4) /* Page table shared from CPU */ +#define __IOMMU_DOMAIN_HOST_VA (1U << 5) /* Host CPU virtual address */ + /* * This are the possible domain-types * @@ -86,6 +90,8 @@ struct iommu_domain_geometry { #define IOMMU_DOMAIN_DMA_FQ (__IOMMU_DOMAIN_PAGING | \ __IOMMU_DOMAIN_DMA_API | \ __IOMMU_DOMAIN_DMA_FQ) +#define IOMMU_DOMAIN_SVA (__IOMMU_DOMAIN_SHARED | \ + __IOMMU_DOMAIN_HOST_VA) =20 struct iommu_domain { unsigned type; @@ -95,6 +101,7 @@ struct iommu_domain { void *handler_token; struct iommu_domain_geometry geometry; struct iommu_dma_cookie *iova_cookie; + struct iommu_sva_cookie *sva_cookie; }; =20 static inline bool iommu_is_dma_domain(struct iommu_domain *domain) diff --git a/drivers/iommu/iommu-sva-lib.h b/drivers/iommu/iommu-sva-lib.h index 8909ea1094e3..1a71218b07f5 100644 --- a/drivers/iommu/iommu-sva-lib.h +++ b/drivers/iommu/iommu-sva-lib.h @@ -10,6 +10,7 @@ =20 int iommu_sva_alloc_pasid(struct mm_struct *mm, ioasid_t min, ioasid_t max= ); struct mm_struct *iommu_sva_find(ioasid_t pasid); +struct mm_struct *iommu_sva_domain_mm(struct iommu_domain *domain); =20 /* I/O Page fault */ struct device; @@ -26,6 +27,8 @@ int iopf_queue_flush_dev(struct device *dev); struct iopf_queue *iopf_queue_alloc(const char *name); void iopf_queue_free(struct iopf_queue *queue); int iopf_queue_discard_partial(struct iopf_queue *queue); +bool iommu_sva_domain_get_user(struct iommu_domain *domain); +void iommu_sva_domain_put_user(struct iommu_domain *domain); =20 #else /* CONFIG_IOMMU_SVA */ static inline int iommu_queue_iopf(struct iommu_fault *fault, void *cookie) @@ -63,5 +66,12 @@ static inline int iopf_queue_discard_partial(struct iopf= _queue *queue) { return -ENODEV; } + +static inline bool iommu_sva_domain_get_user(struct iommu_domain *domain) +{ + return false; +} + +static inline void iommu_sva_domain_put_user(struct iommu_domain *domain) = { } #endif /* CONFIG_IOMMU_SVA */ #endif /* _IOMMU_SVA_LIB_H */ diff --git a/drivers/iommu/iommu-sva-lib.c b/drivers/iommu/iommu-sva-lib.c index 106506143896..78820be23f15 100644 --- a/drivers/iommu/iommu-sva-lib.c +++ b/drivers/iommu/iommu-sva-lib.c @@ -3,12 +3,21 @@ * Helpers for IOMMU drivers implementing SVA */ #include +#include +#include #include =20 #include "iommu-sva-lib.h" =20 static DEFINE_MUTEX(iommu_sva_lock); static DECLARE_IOASID_SET(iommu_sva_pasid); +static DEFINE_XARRAY_ALLOC(sva_domain_array); + +struct iommu_sva_cookie { + struct mm_struct *mm; + ioasid_t pasid; + refcount_t users; +}; =20 /** * iommu_sva_alloc_pasid - Allocate a PASID for the mm @@ -69,3 +78,83 @@ struct mm_struct *iommu_sva_find(ioasid_t pasid) return ioasid_find(&iommu_sva_pasid, pasid, __mmget_not_zero); } EXPORT_SYMBOL_GPL(iommu_sva_find); + +static struct iommu_domain * +iommu_sva_alloc_domain(struct device *dev, struct mm_struct *mm) +{ + struct bus_type *bus =3D dev->bus; + struct iommu_sva_cookie *cookie; + struct iommu_domain *domain; + void *curr; + + if (!bus || !bus->iommu_ops) + return NULL; + + cookie =3D kzalloc(sizeof(*cookie), GFP_KERNEL); + if (!cookie) + return NULL; + + domain =3D bus->iommu_ops->domain_alloc(IOMMU_DOMAIN_SVA); + if (!domain) + goto err_domain_alloc; + + cookie->mm =3D mm; + cookie->pasid =3D mm->pasid; + refcount_set(&cookie->users, 1); + domain->type =3D IOMMU_DOMAIN_SVA; + domain->sva_cookie =3D cookie; + curr =3D xa_store(&sva_domain_array, mm->pasid, domain, GFP_KERNEL); + if (xa_err(curr)) + goto err_xa_store; + + return domain; +err_xa_store: + domain->ops->free(domain); +err_domain_alloc: + kfree(cookie); + return NULL; +} + +static void iommu_sva_free_domain(struct iommu_domain *domain) +{ + xa_erase(&sva_domain_array, domain->sva_cookie->pasid); + kfree(domain->sva_cookie); + domain->ops->free(domain); +} + +bool iommu_sva_domain_get_user(struct iommu_domain *domain) +{ + struct iommu_sva_cookie *cookie =3D domain->sva_cookie; + + return refcount_inc_not_zero(&cookie->users); +} + +void iommu_sva_domain_put_user(struct iommu_domain *domain) +{ + struct iommu_sva_cookie *cookie =3D domain->sva_cookie; + + if (refcount_dec_and_test(&cookie->users)) + iommu_sva_free_domain(domain); +} + +static __maybe_unused struct iommu_domain * +iommu_sva_get_domain(struct device *dev, struct mm_struct *mm) +{ + struct iommu_domain *domain; + ioasid_t pasid =3D mm->pasid; + + if (pasid =3D=3D INVALID_IOASID) + return NULL; + + domain =3D xa_load(&sva_domain_array, pasid); + if (!domain) + return iommu_sva_alloc_domain(dev, mm); + iommu_sva_domain_get_user(domain); + + return domain; +} + +struct mm_struct *iommu_sva_domain_mm(struct iommu_domain *domain) +{ + return domain->sva_cookie->mm; +} --=20 2.25.1 From nobody Fri Jun 19 21:27:52 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 17300C4332F for ; Tue, 29 Mar 2022 05:40:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232536AbiC2FmV (ORCPT ); Tue, 29 Mar 2022 01:42:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55866 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232552AbiC2FmN (ORCPT ); Tue, 29 Mar 2022 01:42:13 -0400 Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D91C224A75B for ; Mon, 28 Mar 2022 22:40:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1648532427; x=1680068427; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=WKkLvCTT03BDHMi45GzO4KtUhX/7xVoo1Mlm4MxAfn4=; b=i7ahvYNNqwRBdu/KfJCj73hH/YOB9TZQJJvYQoAEufDzLdnND6GPfyTv TJ3+kmKVkePE35+SZE1tOZ9BeH9rfLch/64pGqMIPCCTsCO0GAf8Lhn7l SjS3qla8AZ0wYqQlSGtDBnn0Z1cmmInEzTO/OaFobBY9ZsoU3fWcwdnC/ f1C0i58mFkm+PeqodcMgUbFyKuDJYHtgSx4xC34PpPlV9SrjOEZHizwSe ayhgGae5iAeP3eF46meEr3TX39oCo0CTgaOZB5F0tSugcJACwRtdIHfxn Kyt2uiunn674tul55sU1K0RmvNxQYsEGETUgZLi+yE5oSl7PCC4VX4Bz1 Q==; X-IronPort-AV: E=McAfee;i="6200,9189,10300"; a="259136987" X-IronPort-AV: E=Sophos;i="5.90,219,1643702400"; d="scan'208";a="259136987" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Mar 2022 22:40:27 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.90,219,1643702400"; d="scan'208";a="694603599" Received: from allen-box.sh.intel.com ([10.239.159.48]) by fmsmga001.fm.intel.com with ESMTP; 28 Mar 2022 22:40:23 -0700 From: Lu Baolu To: Joerg Roedel , Jason Gunthorpe , Christoph Hellwig , Kevin Tian , Ashok Raj , Will Deacon , Robin Murphy , Jean-Philippe Brucker Cc: Eric Auger , Liu Yi L , Jacob jun Pan , iommu@lists.linux-foundation.org, linux-kernel@vger.kernel.org, Lu Baolu Subject: [PATCH RFC v2 04/11] iommu: Add attach/detach_dev_pasid domain ops Date: Tue, 29 Mar 2022 13:37:53 +0800 Message-Id: <20220329053800.3049561-5-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220329053800.3049561-1-baolu.lu@linux.intel.com> References: <20220329053800.3049561-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Attaching an IOMMU domain to a PASID of a device is a generic operation for modern IOMMU drivers which support PASID-granular DMA address translation. Currently visible usage scenarios include (but not limited): - SVA (Shared Virtual Address) - kernel DMA with PASID - hardware-assist mediated device This adds a pair of common domain ops for this purpose and adds some common helpers to attach/detach a domain to/from a {device, PASID} and get/put the domain attached to {device, PASID}. Signed-off-by: Lu Baolu --- include/linux/iommu.h | 36 ++++++++++++++++++ drivers/iommu/iommu.c | 88 +++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 124 insertions(+) diff --git a/include/linux/iommu.h b/include/linux/iommu.h index 29c4c2edd706..a46285488a57 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -269,6 +269,8 @@ struct iommu_ops { * struct iommu_domain_ops - domain specific operations * @attach_dev: attach an iommu domain to a device * @detach_dev: detach an iommu domain from a device + * @attach_dev_pasid: attach an iommu domain to a pasid of device + * @detach_dev_pasid: detach an iommu domain from a pasid of device * @map: map a physically contiguous memory region to an iommu domain * @map_pages: map a physically contiguous set of pages of the same size to * an iommu domain. @@ -286,6 +288,10 @@ struct iommu_ops { struct iommu_domain_ops { int (*attach_dev)(struct iommu_domain *domain, struct device *dev); void (*detach_dev)(struct iommu_domain *domain, struct device *dev); + int (*attach_dev_pasid)(struct iommu_domain *domain, + struct device *dev, ioasid_t id); + void (*detach_dev_pasid)(struct iommu_domain *domain, + struct device *dev, ioasid_t id); =20 int (*map)(struct iommu_domain *domain, unsigned long iova, phys_addr_t paddr, size_t size, int prot, gfp_t gfp); @@ -679,6 +685,14 @@ int iommu_group_claim_dma_owner(struct iommu_group *gr= oup, void *owner); void iommu_group_release_dma_owner(struct iommu_group *group); bool iommu_group_dma_owner_claimed(struct iommu_group *group); =20 +int iommu_attach_device_pasid(struct iommu_domain *domain, + struct device *dev, ioasid_t pasid); +void iommu_detach_device_pasid(struct iommu_domain *domain, + struct device *dev, ioasid_t pasid); +struct iommu_domain * +iommu_get_domain_for_dev_pasid(struct device *dev, ioasid_t pasid); +void iommu_put_domain_for_dev_pasid(struct iommu_domain *domain); + #else /* CONFIG_IOMMU_API */ =20 struct iommu_ops {}; @@ -1047,6 +1061,28 @@ static inline bool iommu_group_dma_owner_claimed(str= uct iommu_group *group) { return false; } + +static inline int iommu_attach_device_pasid(struct iommu_domain *domain, + struct device *dev, ioasid_t pasid) +{ + return -ENODEV; +} + +static inline void iommu_detach_device_pasid(struct iommu_domain *domain, + struct device *dev, ioasid_t pasid) +{ +} + +static inline struct iommu_domain * +iommu_get_domain_for_dev_pasid(struct device *dev, ioasid_t pasid) +{ + return NULL; +} + +static inline void +iommu_put_domain_for_dev_pasid(struct iommu_domain *domain) +{ +} #endif /* CONFIG_IOMMU_API */ =20 /** diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c index 9fb8a5b4491e..8163ad7f6902 100644 --- a/drivers/iommu/iommu.c +++ b/drivers/iommu/iommu.c @@ -27,6 +27,8 @@ #include #include =20 +#include "iommu-sva-lib.h" + static struct kset *iommu_group_kset; static DEFINE_IDA(iommu_group_ida); =20 @@ -38,6 +40,7 @@ struct iommu_group { struct kobject kobj; struct kobject *devices_kobj; struct list_head devices; + struct xarray pasid_array; struct mutex mutex; void *iommu_data; void (*iommu_data_release)(void *iommu_data); @@ -631,6 +634,7 @@ struct iommu_group *iommu_group_alloc(void) mutex_init(&group->mutex); INIT_LIST_HEAD(&group->devices); INIT_LIST_HEAD(&group->entry); + xa_init(&group->pasid_array); =20 ret =3D ida_simple_get(&iommu_group_ida, 0, 0, GFP_KERNEL); if (ret < 0) { @@ -3173,3 +3177,87 @@ bool iommu_group_dma_owner_claimed(struct iommu_grou= p *group) return user; } EXPORT_SYMBOL_GPL(iommu_group_dma_owner_claimed); + +int iommu_attach_device_pasid(struct iommu_domain *domain, + struct device *dev, ioasid_t pasid) +{ + struct iommu_group *group; + int ret =3D -EINVAL; + void *curr; + + if (!domain->ops->attach_dev_pasid) + return -EINVAL; + + group =3D iommu_group_get(dev); + if (!group) + return -ENODEV; + + mutex_lock(&group->mutex); + /* + * To keep things simple, we currently don't support IOMMU groups + * with more than one device. Existing SVA-capable systems are not + * affected by the problems that required IOMMU groups (lack of ACS + * isolation, device ID aliasing and other hardware issues). + */ + if (!iommu_group_singleton_lockdown(group)) + goto out_unlock; + + xa_lock(&group->pasid_array); + curr =3D __xa_cmpxchg(&group->pasid_array, pasid, NULL, + domain, GFP_KERNEL); + xa_unlock(&group->pasid_array); + if (curr) + goto out_unlock; + + ret =3D domain->ops->attach_dev_pasid(domain, dev, pasid); + if (ret) + xa_erase(&group->pasid_array, pasid); + +out_unlock: + mutex_unlock(&group->mutex); + iommu_group_put(group); + + return ret; +} + +void iommu_detach_device_pasid(struct iommu_domain *domain, + struct device *dev, ioasid_t pasid) +{ + struct iommu_group *group; + + group =3D iommu_group_get(dev); + if (WARN_ON(!group)) + return; + + mutex_lock(&group->mutex); + domain->ops->detach_dev_pasid(domain, dev, pasid); + xa_erase(&group->pasid_array, pasid); + mutex_unlock(&group->mutex); + iommu_group_put(group); +} + +struct iommu_domain * +iommu_get_domain_for_dev_pasid(struct device *dev, ioasid_t pasid) +{ + struct iommu_domain *domain; + struct iommu_group *group; + + group =3D iommu_group_get(dev); + if (!group) + return NULL; + + mutex_lock(&group->mutex); + domain =3D xa_load(&group->pasid_array, pasid); + if (domain && domain->type =3D=3D IOMMU_DOMAIN_SVA) + iommu_sva_domain_get_user(domain); + mutex_unlock(&group->mutex); + iommu_group_put(group); + + return domain; +} + +void iommu_put_domain_for_dev_pasid(struct iommu_domain *domain) +{ + if (domain->type =3D=3D IOMMU_DOMAIN_SVA) + iommu_sva_domain_put_user(domain); +} --=20 2.25.1 From nobody Fri Jun 19 21:27:52 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D9706C433FE for ; Tue, 29 Mar 2022 05:40:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232519AbiC2Fm3 (ORCPT ); Tue, 29 Mar 2022 01:42:29 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56232 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232543AbiC2FmP (ORCPT ); Tue, 29 Mar 2022 01:42:15 -0400 Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 49EB123D45E for ; Mon, 28 Mar 2022 22:40:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1648532433; x=1680068433; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=rmTrrsN6o/iiXpbXXW5KgezOuk/1ln46uK9jx/UqwWI=; b=ca3Tq2SJasmqPILKPlagA7Pjsg3O9+Qr8ElOrr4038c6sJlmw+YT/u+V 0nTbswk3XSw2FR+SxWW61sNfOSQzo7NGfO2rLYdrOn9eT0zLZe0pDAVsC f0Q1ULS/rTn0yv1kRzPLOTCd9xpfkcsLTSDR5hC5CabRCw6fsxwlWDQP+ FwhHYIktN4EuvIKGeQJFjGcimq3KqVD7UE2jmwQS1YnmM+zaLY2EyLgDJ dyKKxRowEiF0gVn7uDYyXqHQuwwMShXEGgMXv+ufemgzMci9AMDYfhe5H BTJLqhChu1oEjXCDlosSW3U3NIOTjWQ3rnnSJOjoDV9c6iof02TcXVBGw w==; X-IronPort-AV: E=McAfee;i="6200,9189,10300"; a="259136999" X-IronPort-AV: E=Sophos;i="5.90,219,1643702400"; d="scan'208";a="259136999" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Mar 2022 22:40:31 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.90,219,1643702400"; d="scan'208";a="694603611" Received: from allen-box.sh.intel.com ([10.239.159.48]) by fmsmga001.fm.intel.com with ESMTP; 28 Mar 2022 22:40:27 -0700 From: Lu Baolu To: Joerg Roedel , Jason Gunthorpe , Christoph Hellwig , Kevin Tian , Ashok Raj , Will Deacon , Robin Murphy , Jean-Philippe Brucker Cc: Eric Auger , Liu Yi L , Jacob jun Pan , iommu@lists.linux-foundation.org, linux-kernel@vger.kernel.org, Lu Baolu , Jacob Pan Subject: [PATCH RFC v2 05/11] iommu/vt-d: Remove SVM_FLAG_SUPERVISOR_MODE suport Date: Tue, 29 Mar 2022 13:37:54 +0800 Message-Id: <20220329053800.3049561-6-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220329053800.3049561-1-baolu.lu@linux.intel.com> References: <20220329053800.3049561-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The current in-kernel supervisor PASID support is based on the SVA machinery in SVA lib. The binding between a kernel PASID and kernel mapping has many flaws. Remove SVM_FLAG_SUPERVISOR_MODE support. Link: https://lore.kernel.org/linux-iommu/20210511194726.GP1002214@nvidia.c= om/ Signed-off-by: Jacob Pan Signed-off-by: Lu Baolu --- drivers/iommu/intel/svm.c | 53 +++++++++------------------------------ 1 file changed, 12 insertions(+), 41 deletions(-) diff --git a/drivers/iommu/intel/svm.c b/drivers/iommu/intel/svm.c index d1c42dfae6ca..ee5ecde5b318 100644 --- a/drivers/iommu/intel/svm.c +++ b/drivers/iommu/intel/svm.c @@ -313,8 +313,7 @@ static int pasid_to_svm_sdev(struct device *dev, unsign= ed int pasid, return 0; } =20 -static int intel_svm_alloc_pasid(struct device *dev, struct mm_struct *mm, - unsigned int flags) +static int intel_svm_alloc_pasid(struct device *dev, struct mm_struct *mm) { ioasid_t max_pasid =3D dev_is_pci(dev) ? pci_max_pasids(to_pci_dev(dev)) : intel_pasid_max_id; @@ -324,8 +323,7 @@ static int intel_svm_alloc_pasid(struct device *dev, st= ruct mm_struct *mm, =20 static struct iommu_sva *intel_svm_bind_mm(struct intel_iommu *iommu, struct device *dev, - struct mm_struct *mm, - unsigned int flags) + struct mm_struct *mm) { struct device_domain_info *info =3D dev_iommu_priv_get(dev); unsigned long iflags, sflags; @@ -341,22 +339,18 @@ static struct iommu_sva *intel_svm_bind_mm(struct int= el_iommu *iommu, =20 svm->pasid =3D mm->pasid; svm->mm =3D mm; - svm->flags =3D flags; INIT_LIST_HEAD_RCU(&svm->devs); =20 - if (!(flags & SVM_FLAG_SUPERVISOR_MODE)) { - svm->notifier.ops =3D &intel_mmuops; - ret =3D mmu_notifier_register(&svm->notifier, mm); - if (ret) { - kfree(svm); - return ERR_PTR(ret); - } + svm->notifier.ops =3D &intel_mmuops; + ret =3D mmu_notifier_register(&svm->notifier, mm); + if (ret) { + kfree(svm); + return ERR_PTR(ret); } =20 ret =3D pasid_private_add(svm->pasid, svm); if (ret) { - if (svm->notifier.ops) - mmu_notifier_unregister(&svm->notifier, mm); + mmu_notifier_unregister(&svm->notifier, mm); kfree(svm); return ERR_PTR(ret); } @@ -391,9 +385,7 @@ static struct iommu_sva *intel_svm_bind_mm(struct intel= _iommu *iommu, } =20 /* Setup the pasid table: */ - sflags =3D (flags & SVM_FLAG_SUPERVISOR_MODE) ? - PASID_FLAG_SUPERVISOR_MODE : 0; - sflags |=3D cpu_feature_enabled(X86_FEATURE_LA57) ? PASID_FLAG_FL5LP : 0; + sflags =3D cpu_feature_enabled(X86_FEATURE_LA57) ? PASID_FLAG_FL5LP : 0; sflags |=3D PASID_FLAG_PAGE_SNOOP; spin_lock_irqsave(&iommu->lock, iflags); ret =3D intel_pasid_setup_first_level(iommu, dev, mm->pgd, mm->pasid, @@ -411,8 +403,7 @@ static struct iommu_sva *intel_svm_bind_mm(struct intel= _iommu *iommu, kfree(sdev); free_svm: if (list_empty(&svm->devs)) { - if (svm->notifier.ops) - mmu_notifier_unregister(&svm->notifier, mm); + mmu_notifier_unregister(&svm->notifier, mm); pasid_private_remove(mm->pasid); kfree(svm); } @@ -822,37 +813,17 @@ static irqreturn_t prq_event_thread(int irq, void *d) struct iommu_sva *intel_svm_bind(struct device *dev, struct mm_struct *mm,= void *drvdata) { struct intel_iommu *iommu =3D device_to_iommu(dev, NULL, NULL); - unsigned int flags =3D 0; struct iommu_sva *sva; int ret; =20 - if (drvdata) - flags =3D *(unsigned int *)drvdata; - - if (flags & SVM_FLAG_SUPERVISOR_MODE) { - if (!ecap_srs(iommu->ecap)) { - dev_err(dev, "%s: Supervisor PASID not supported\n", - iommu->name); - return ERR_PTR(-EOPNOTSUPP); - } - - if (mm) { - dev_err(dev, "%s: Supervisor PASID with user provided mm\n", - iommu->name); - return ERR_PTR(-EINVAL); - } - - mm =3D &init_mm; - } - mutex_lock(&pasid_mutex); - ret =3D intel_svm_alloc_pasid(dev, mm, flags); + ret =3D intel_svm_alloc_pasid(dev, mm); if (ret) { mutex_unlock(&pasid_mutex); return ERR_PTR(ret); } =20 - sva =3D intel_svm_bind_mm(iommu, dev, mm, flags); + sva =3D intel_svm_bind_mm(iommu, dev, mm); mutex_unlock(&pasid_mutex); =20 return sva; --=20 2.25.1 From nobody Fri Jun 19 21:27:52 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D7E58C433EF for ; Tue, 29 Mar 2022 05:40:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232555AbiC2FmZ (ORCPT ); Tue, 29 Mar 2022 01:42:25 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56312 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232559AbiC2FmS (ORCPT ); Tue, 29 Mar 2022 01:42:18 -0400 Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id ED40531935 for ; Mon, 28 Mar 2022 22:40:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1648532434; x=1680068434; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=qef1x61kxuBKogixpH2FkpRMgqvDZAiwuSUR/Lcg4M4=; b=KR59Wg0n3rjdyvz738GF03EuycqXnrnUg9LWY7I6ooMGRjZyILqt8Hrp XN2cCCu42d96jP1hVKl1s9JOThsTzGJIvUCLua77hDhsJ60mPT1UYpH5q NoSq8CFuNbaHfzPtqlZ17CjUzmY8jKTTE7rlhwfbeRnHlv5fplHHWtt0n Dx+RqqhdwN+D2fHSRZWWyUczA6n4iyk8nZLuavmKYa6lYOb6qlRgQ9czh b1jU83joIr0zqKbHw6JyAuoCv8KxIw6AKVAxXaupfpVLJj42aihxINHIn MM94ICy6nKjy2I7XpeVmMNH3QdNWMiEwQwcf61vcPkcfQznhlMhCkZzOm g==; X-IronPort-AV: E=McAfee;i="6200,9189,10300"; a="259137006" X-IronPort-AV: E=Sophos;i="5.90,219,1643702400"; d="scan'208";a="259137006" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Mar 2022 22:40:34 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.90,219,1643702400"; d="scan'208";a="694603629" Received: from allen-box.sh.intel.com ([10.239.159.48]) by fmsmga001.fm.intel.com with ESMTP; 28 Mar 2022 22:40:31 -0700 From: Lu Baolu To: Joerg Roedel , Jason Gunthorpe , Christoph Hellwig , Kevin Tian , Ashok Raj , Will Deacon , Robin Murphy , Jean-Philippe Brucker Cc: Eric Auger , Liu Yi L , Jacob jun Pan , iommu@lists.linux-foundation.org, linux-kernel@vger.kernel.org, Lu Baolu Subject: [PATCH RFC v2 06/11] iommu/vt-d: Add SVA domain support Date: Tue, 29 Mar 2022 13:37:55 +0800 Message-Id: <20220329053800.3049561-7-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220329053800.3049561-1-baolu.lu@linux.intel.com> References: <20220329053800.3049561-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Add support for SVA domain allocation and provide an SVA-specific iommu_domain_ops. Signed-off-by: Lu Baolu --- include/linux/intel-iommu.h | 1 + drivers/iommu/intel/iommu.c | 10 ++++++++++ drivers/iommu/intel/svm.c | 37 +++++++++++++++++++++++++++++++++++++ 3 files changed, 48 insertions(+) diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h index 2f9891cb3d00..c14283137fb5 100644 --- a/include/linux/intel-iommu.h +++ b/include/linux/intel-iommu.h @@ -744,6 +744,7 @@ void intel_svm_unbind(struct iommu_sva *handle); u32 intel_svm_get_pasid(struct iommu_sva *handle); int intel_svm_page_response(struct device *dev, struct iommu_fault_event *= evt, struct iommu_page_response *msg); +extern const struct iommu_domain_ops intel_svm_domain_ops; =20 struct intel_svm_dev { struct list_head list; diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c index c1b91bce1530..5eae7cf9bee5 100644 --- a/drivers/iommu/intel/iommu.c +++ b/drivers/iommu/intel/iommu.c @@ -4318,6 +4318,16 @@ static struct iommu_domain *intel_iommu_domain_alloc= (unsigned type) return domain; case IOMMU_DOMAIN_IDENTITY: return &si_domain->domain; +#ifdef CONFIG_INTEL_IOMMU_SVM + case IOMMU_DOMAIN_SVA: + dmar_domain =3D alloc_domain(type); + if (!dmar_domain) + return NULL; + domain =3D &dmar_domain->domain; + domain->ops =3D &intel_svm_domain_ops; + + return domain; +#endif /* CONFIG_INTEL_IOMMU_SVM */ default: return NULL; } diff --git a/drivers/iommu/intel/svm.c b/drivers/iommu/intel/svm.c index ee5ecde5b318..8f59dd641b2d 100644 --- a/drivers/iommu/intel/svm.c +++ b/drivers/iommu/intel/svm.c @@ -932,3 +932,40 @@ int intel_svm_page_response(struct device *dev, mutex_unlock(&pasid_mutex); return ret; } + +static int intel_svm_attach_dev_pasid(struct iommu_domain *domain, + struct device *dev, ioasid_t pasid) +{ + struct device_domain_info *info =3D dev_iommu_priv_get(dev); + struct mm_struct *mm =3D iommu_sva_domain_mm(domain); + struct intel_iommu *iommu =3D info->iommu; + struct iommu_sva *sva; + int ret =3D 0; + + mutex_lock(&pasid_mutex); + sva =3D intel_svm_bind_mm(iommu, dev, mm); + if (IS_ERR(sva)) + ret =3D PTR_ERR(sva); + mutex_unlock(&pasid_mutex); + + return ret; +} + +static void intel_svm_detach_dev_pasid(struct iommu_domain *domain, + struct device *dev, ioasid_t pasid) +{ + mutex_lock(&pasid_mutex); + intel_svm_unbind_mm(dev, pasid); + mutex_unlock(&pasid_mutex); +} + +static void intel_svm_domain_free(struct iommu_domain *domain) +{ + kfree(domain); +} + +const struct iommu_domain_ops intel_svm_domain_ops =3D { + .attach_dev_pasid =3D intel_svm_attach_dev_pasid, + .detach_dev_pasid =3D intel_svm_detach_dev_pasid, + .free =3D intel_svm_domain_free, +}; --=20 2.25.1 From nobody Fri Jun 19 21:27:52 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2C4FCC433EF for ; Tue, 29 Mar 2022 05:40:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232614AbiC2Fmd (ORCPT ); Tue, 29 Mar 2022 01:42:33 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56722 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232573AbiC2FmU (ORCPT ); Tue, 29 Mar 2022 01:42:20 -0400 Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 82DC796826 for ; Mon, 28 Mar 2022 22:40:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1648532438; x=1680068438; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=XEvCo8vc7HyGtJNWT/dzX6msN51mZr6s4VKvbdupNm4=; b=l/sYaOf0N1aDjtzeOVIAjb6mUEc6hd0VMruLw63oV6ZM4zzxToGn5KyI w27ZwK0wAqI1UXG+BRXtIcD2bzbWOTUrXOumE3JXIMgryhLRm89W+8d5Y WG899u/NkYVgnDwkkJK6WPInsbNeBA8HNCioLQWWzX5+HPgvXI03qIjhG IdSt3auk6b0ySuxKvrzSHpN4yqDq9yUJjYTjSrKe+c0uBkavVDJBfliqZ v7+pHY6kBBm/UmHOScbmFXcyEDJlKD+WDB3f8yyBB8Ri525/2GSAyAQ9X 6jMbz+XuvzKdgL41X3sLgFKrv/rqVCcHi9CzmMiSjYrMMKsJjoZu5EGZk Q==; X-IronPort-AV: E=McAfee;i="6200,9189,10300"; a="259137019" X-IronPort-AV: E=Sophos;i="5.90,219,1643702400"; d="scan'208";a="259137019" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Mar 2022 22:40:38 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.90,219,1643702400"; d="scan'208";a="694603644" Received: from allen-box.sh.intel.com ([10.239.159.48]) by fmsmga001.fm.intel.com with ESMTP; 28 Mar 2022 22:40:34 -0700 From: Lu Baolu To: Joerg Roedel , Jason Gunthorpe , Christoph Hellwig , Kevin Tian , Ashok Raj , Will Deacon , Robin Murphy , Jean-Philippe Brucker Cc: Eric Auger , Liu Yi L , Jacob jun Pan , iommu@lists.linux-foundation.org, linux-kernel@vger.kernel.org, Lu Baolu Subject: [PATCH RFC v2 07/11] arm-smmu-v3/sva: Add SVA domain support Date: Tue, 29 Mar 2022 13:37:56 +0800 Message-Id: <20220329053800.3049561-8-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220329053800.3049561-1-baolu.lu@linux.intel.com> References: <20220329053800.3049561-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Add support for SVA domain allocation and provide an SVA-specific iommu_domain_ops. Signed-off-by: Lu Baolu --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 14 +++++++ .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 42 +++++++++++++++++++ drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 21 ++++++++++ 3 files changed, 77 insertions(+) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/ar= m/arm-smmu-v3/arm-smmu-v3.h index cd48590ada30..7631c00fdcbd 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -759,6 +759,10 @@ struct iommu_sva *arm_smmu_sva_bind(struct device *dev= , struct mm_struct *mm, void arm_smmu_sva_unbind(struct iommu_sva *handle); u32 arm_smmu_sva_get_pasid(struct iommu_sva *handle); void arm_smmu_sva_notifier_synchronize(void); +int arm_smmu_sva_attach_dev_pasid(struct iommu_domain *domain, + struct device *dev, ioasid_t id); +void arm_smmu_sva_detach_dev_pasid(struct iommu_domain *domain, + struct device *dev, ioasid_t id); #else /* CONFIG_ARM_SMMU_V3_SVA */ static inline bool arm_smmu_sva_supported(struct arm_smmu_device *smmu) { @@ -804,5 +808,15 @@ static inline u32 arm_smmu_sva_get_pasid(struct iommu_= sva *handle) } =20 static inline void arm_smmu_sva_notifier_synchronize(void) {} + +static inline int arm_smmu_sva_attach_dev_pasid(struct iommu_domain *domai= n, + struct device *dev, ioasid_t id) +{ + return -ENODEV; +} + +static inline void arm_smmu_sva_detach_dev_pasid(struct iommu_domain *doma= in, + struct device *dev, + ioasid_t id) {} #endif /* CONFIG_ARM_SMMU_V3_SVA */ #endif /* _ARM_SMMU_V3_H */ diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iomm= u/arm/arm-smmu-v3/arm-smmu-v3-sva.c index 22ddd05bbdcd..ce229820086d 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -534,3 +534,45 @@ void arm_smmu_sva_notifier_synchronize(void) */ mmu_notifier_synchronize(); } + +int arm_smmu_sva_attach_dev_pasid(struct iommu_domain *domain, + struct device *dev, ioasid_t id) +{ + int ret =3D 0; + struct iommu_sva *handle; + struct mm_struct *mm =3D iommu_sva_domain_mm(domain); + + if (domain->type !=3D IOMMU_DOMAIN_SVA || !mm) + return -EINVAL; + + mutex_lock(&sva_lock); + handle =3D __arm_smmu_sva_bind(dev, mm); + if (IS_ERR(handle)) + ret =3D PTR_ERR(handle); + mutex_unlock(&sva_lock); + + return ret; +} + +void arm_smmu_sva_detach_dev_pasid(struct iommu_domain *domain, + struct device *dev, ioasid_t id) +{ + struct arm_smmu_bond *bond =3D NULL, *t; + struct mm_struct *mm =3D iommu_sva_domain_mm(domain); + struct arm_smmu_master *master =3D dev_iommu_priv_get(dev); + + mutex_lock(&sva_lock); + list_for_each_entry(t, &master->bonds, list) { + if (t->mm =3D=3D mm) { + bond =3D t; + break; + } + } + + if (!WARN_ON(!bond) && refcount_dec_and_test(&bond->refs)) { + list_del(&bond->list); + arm_smmu_mmu_notifier_put(bond->smmu_mn); + kfree(bond); + } + mutex_unlock(&sva_lock); +} diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/ar= m/arm-smmu-v3/arm-smmu-v3.c index afc63fce6107..bd80de0bad98 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -1995,10 +1995,31 @@ static bool arm_smmu_capable(enum iommu_cap cap) } } =20 +static void arm_smmu_sva_domain_free(struct iommu_domain *domain) +{ + kfree(domain); +} + +static const struct iommu_domain_ops arm_smmu_sva_domain_ops =3D { + .attach_dev_pasid =3D arm_smmu_sva_attach_dev_pasid, + .detach_dev_pasid =3D arm_smmu_sva_detach_dev_pasid, + .free =3D arm_smmu_sva_domain_free, +}; + static struct iommu_domain *arm_smmu_domain_alloc(unsigned type) { struct arm_smmu_domain *smmu_domain; =20 + if (type =3D=3D IOMMU_DOMAIN_SVA) { + struct iommu_domain *domain; + + domain =3D kzalloc(sizeof(*domain), GFP_KERNEL); + if (domain) + domain->ops =3D &arm_smmu_sva_domain_ops; + + return domain; + } + if (type !=3D IOMMU_DOMAIN_UNMANAGED && type !=3D IOMMU_DOMAIN_DMA && type !=3D IOMMU_DOMAIN_DMA_FQ && --=20 2.25.1 From nobody Fri Jun 19 21:27:52 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5EFC4C433FE for ; Tue, 29 Mar 2022 05:41:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232583AbiC2Fmk (ORCPT ); Tue, 29 Mar 2022 01:42:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57478 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232582AbiC2Fm2 (ORCPT ); Tue, 29 Mar 2022 01:42:28 -0400 Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9431924A76B for ; Mon, 28 Mar 2022 22:40:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1648532442; x=1680068442; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=6btWeZ5iTNOQlxZgaeVAufZpKRbPQP+0YXeY7Gwq8LE=; b=KyvpPyq40jYU+qWjHw+GjF2zJRoharkU65OVQLnbqK9BWb4N79Y/4BmI YLNTnfs92djxniJpbiBBfFGc30yqE6WCtX9fviwR04F2NdsuD3JYS8Jvt YELcmX8fap65vvaCQAjPhbBcZrCXPb7yzVDcG3xqXvUHFfSlQ1g9hVvao HezDNuxR0abmyY+GHpP2wkQDu6MJEmz+h47uLT7dR7vS+uyljm3bbqPRv YBuJ6lb94jAXWrdQkvNm8ZrCAKbxh8f9Kct1v3oCMcWf5IpoDVOdVpuDc otwPoH5Qmn2/BKzWwa+494L3lcCLO8xpeSAIuxLrxTlsTgnOk445spy4x g==; X-IronPort-AV: E=McAfee;i="6200,9189,10300"; a="259137030" X-IronPort-AV: E=Sophos;i="5.90,219,1643702400"; d="scan'208";a="259137030" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Mar 2022 22:40:42 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.90,219,1643702400"; d="scan'208";a="694603662" Received: from allen-box.sh.intel.com ([10.239.159.48]) by fmsmga001.fm.intel.com with ESMTP; 28 Mar 2022 22:40:38 -0700 From: Lu Baolu To: Joerg Roedel , Jason Gunthorpe , Christoph Hellwig , Kevin Tian , Ashok Raj , Will Deacon , Robin Murphy , Jean-Philippe Brucker Cc: Eric Auger , Liu Yi L , Jacob jun Pan , iommu@lists.linux-foundation.org, linux-kernel@vger.kernel.org, Lu Baolu Subject: [PATCH RFC v2 08/11] iommu/sva: Use attach/detach_pasid_dev in SVA interfaces Date: Tue, 29 Mar 2022 13:37:57 +0800 Message-Id: <20220329053800.3049561-9-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220329053800.3049561-1-baolu.lu@linux.intel.com> References: <20220329053800.3049561-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The existing iommu SVA interfaces are implemented by calling the SVA specific iommu ops provided by the IOMMU drivers. There's no need for any SVA specific ops in iommu_ops vector anymore as we can achieve this through the generic attach/detach_dev_pasid domain ops. This refactors the IOMMU SVA interfaces implementation by using the attach/detach_pasid_dev ops and align them with the concept of the iommu domain. Put the new SVA code in the sva related file in order to make it self-contained. Signed-off-by: Lu Baolu --- include/linux/iommu.h | 51 +++++++++------- drivers/iommu/iommu-sva-lib.c | 110 +++++++++++++++++++++++++++++++++- drivers/iommu/iommu.c | 92 ---------------------------- 3 files changed, 138 insertions(+), 115 deletions(-) diff --git a/include/linux/iommu.h b/include/linux/iommu.h index a46285488a57..11c4d99e122d 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -629,7 +629,12 @@ struct iommu_fwspec { * struct iommu_sva - handle to a device-mm bond */ struct iommu_sva { - struct device *dev; + struct device *dev; + ioasid_t pasid; + struct iommu_domain *domain; + /* Link to sva domain's bonds list */ + struct list_head node; + refcount_t users; }; =20 int iommu_fwspec_init(struct device *dev, struct fwnode_handle *iommu_fwno= de, @@ -672,12 +677,6 @@ int iommu_dev_enable_feature(struct device *dev, enum = iommu_dev_features f); int iommu_dev_disable_feature(struct device *dev, enum iommu_dev_features = f); bool iommu_dev_feature_enabled(struct device *dev, enum iommu_dev_features= f); =20 -struct iommu_sva *iommu_sva_bind_device(struct device *dev, - struct mm_struct *mm, - void *drvdata); -void iommu_sva_unbind_device(struct iommu_sva *handle); -u32 iommu_sva_get_pasid(struct iommu_sva *handle); - int iommu_device_use_default_domain(struct device *dev); void iommu_device_unuse_default_domain(struct device *dev); =20 @@ -1018,21 +1017,6 @@ iommu_dev_disable_feature(struct device *dev, enum i= ommu_dev_features feat) return -ENODEV; } =20 -static inline struct iommu_sva * -iommu_sva_bind_device(struct device *dev, struct mm_struct *mm, void *drvd= ata) -{ - return NULL; -} - -static inline void iommu_sva_unbind_device(struct iommu_sva *handle) -{ -} - -static inline u32 iommu_sva_get_pasid(struct iommu_sva *handle) -{ - return IOMMU_PASID_INVALID; -} - static inline struct iommu_fwspec *dev_iommu_fwspec_get(struct device *dev) { return NULL; @@ -1085,6 +1069,29 @@ iommu_put_domain_for_dev_pasid(struct iommu_domain *= domain) } #endif /* CONFIG_IOMMU_API */ =20 +#ifdef CONFIG_IOMMU_SVA +struct iommu_sva *iommu_sva_bind_device(struct device *dev, + struct mm_struct *mm, + void *drvdata); +void iommu_sva_unbind_device(struct iommu_sva *handle); +u32 iommu_sva_get_pasid(struct iommu_sva *handle); +#else /* CONFIG_IOMMU_SVA */ +static inline struct iommu_sva * +iommu_sva_bind_device(struct device *dev, struct mm_struct *mm, void *drvd= ata) +{ + return NULL; +} + +static inline void iommu_sva_unbind_device(struct iommu_sva *handle) +{ +} + +static inline u32 iommu_sva_get_pasid(struct iommu_sva *handle) +{ + return IOMMU_PASID_INVALID; +} +#endif /* CONFIG_IOMMU_SVA */ + /** * iommu_map_sgtable - Map the given buffer to the IOMMU domain * @domain: The IOMMU domain to perform the mapping diff --git a/drivers/iommu/iommu-sva-lib.c b/drivers/iommu/iommu-sva-lib.c index 78820be23f15..1b45b7d01836 100644 --- a/drivers/iommu/iommu-sva-lib.c +++ b/drivers/iommu/iommu-sva-lib.c @@ -17,6 +17,7 @@ struct iommu_sva_cookie { struct mm_struct *mm; ioasid_t pasid; refcount_t users; + struct list_head bonds; }; =20 /** @@ -101,6 +102,7 @@ iommu_sva_alloc_domain(struct device *dev, struct mm_st= ruct *mm) cookie->mm =3D mm; cookie->pasid =3D mm->pasid; refcount_set(&cookie->users, 1); + INIT_LIST_HEAD(&cookie->bonds); domain->type =3D IOMMU_DOMAIN_SVA; domain->sva_cookie =3D cookie; curr =3D xa_store(&sva_domain_array, mm->pasid, domain, GFP_KERNEL); @@ -118,6 +120,7 @@ iommu_sva_alloc_domain(struct device *dev, struct mm_st= ruct *mm) static void iommu_sva_free_domain(struct iommu_domain *domain) { xa_erase(&sva_domain_array, domain->sva_cookie->pasid); + WARN_ON(!list_empty(&domain->sva_cookie->bonds)); kfree(domain->sva_cookie); domain->ops->free(domain); } @@ -137,7 +140,7 @@ void iommu_sva_domain_put_user(struct iommu_domain *dom= ain) iommu_sva_free_domain(domain); } =20 -static __maybe_unused struct iommu_domain * +static struct iommu_domain * iommu_sva_get_domain(struct device *dev, struct mm_struct *mm) { struct iommu_domain *domain; @@ -158,3 +161,108 @@ struct mm_struct *iommu_sva_domain_mm(struct iommu_do= main *domain) { return domain->sva_cookie->mm; } + +/** + * iommu_sva_bind_device() - Bind a process address space to a device + * @dev: the device + * @mm: the mm to bind, caller must hold a reference to it + * @drvdata: opaque data pointer to pass to bind callback + * + * Create a bond between device and address space, allowing the device to = access + * the mm using the returned PASID. If a bond already exists between @devi= ce and + * @mm, it is returned and an additional reference is taken. Caller must c= all + * iommu_sva_unbind_device() to release each reference. + * + * iommu_dev_enable_feature(dev, IOMMU_DEV_FEAT_SVA) must be called first,= to + * initialize the required SVA features. + * + * On error, returns an ERR_PTR value. + */ +struct iommu_sva * +iommu_sva_bind_device(struct device *dev, struct mm_struct *mm, void *drvd= ata) +{ + int ret =3D -EINVAL; + struct iommu_sva *handle; + struct iommu_domain *domain; + + ret =3D iommu_sva_alloc_pasid(mm, 1, (1U << dev->iommu->pasid_bits) - 1); + if (ret) + return ERR_PTR(ret); + + mutex_lock(&iommu_sva_lock); + domain =3D iommu_sva_get_domain(dev, mm); + if (!domain) { + ret =3D -ENOMEM; + goto out_unlock; + } + + /* Search for an existing bond. */ + list_for_each_entry(handle, &domain->sva_cookie->bonds, node) { + if (handle->dev =3D=3D dev && handle->pasid =3D=3D mm->pasid) { + refcount_inc(&handle->users); + mutex_lock(&iommu_sva_lock); + + return handle; + } + } + + handle =3D kzalloc(sizeof(*handle), GFP_KERNEL); + if (!handle) { + ret =3D -ENOMEM; + goto out_put_domain; + } + + ret =3D iommu_attach_device_pasid(domain, dev, mm->pasid); + if (ret) + goto out_free_handle; + + handle->dev =3D dev; + handle->domain =3D domain; + handle->pasid =3D mm->pasid; + refcount_set(&handle->users, 1); + list_add_tail(&handle->node, &domain->sva_cookie->bonds); + + mutex_unlock(&iommu_sva_lock); + return handle; + +out_free_handle: + kfree(handle); +out_put_domain: + iommu_sva_domain_put_user(domain); +out_unlock: + mutex_unlock(&iommu_sva_lock); + return ERR_PTR(ret); +} +EXPORT_SYMBOL_GPL(iommu_sva_bind_device); + +/** + * iommu_sva_unbind_device() - Remove a bond created with iommu_sva_bind_d= evice + * @handle: the handle returned by iommu_sva_bind_device() + * + * Put reference to a bond between device and address space. The device sh= ould + * not be issuing any more transaction for this PASID. All outstanding page + * requests for this PASID must have been flushed to the IOMMU. + */ +void iommu_sva_unbind_device(struct iommu_sva *handle) +{ + struct device *dev =3D handle->dev; + struct iommu_domain *domain =3D handle->domain; + struct mm_struct *mm =3D iommu_sva_domain_mm(domain); + + mutex_lock(&iommu_sva_lock); + if (refcount_dec_and_test(&handle->users)) { + list_del(&handle->node); + iommu_detach_device_pasid(domain, dev, mm->pasid); + kfree(handle); + } + + iommu_sva_domain_put_user(domain); + mutex_unlock(&iommu_sva_lock); +} +EXPORT_SYMBOL_GPL(iommu_sva_unbind_device); + +u32 iommu_sva_get_pasid(struct iommu_sva *handle) +{ + return handle->pasid; +} +EXPORT_SYMBOL_GPL(iommu_sva_get_pasid); diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c index 8163ad7f6902..6b51ead9d63b 100644 --- a/drivers/iommu/iommu.c +++ b/drivers/iommu/iommu.c @@ -2712,98 +2712,6 @@ bool iommu_dev_feature_enabled(struct device *dev, e= num iommu_dev_features feat) } EXPORT_SYMBOL_GPL(iommu_dev_feature_enabled); =20 -/** - * iommu_sva_bind_device() - Bind a process address space to a device - * @dev: the device - * @mm: the mm to bind, caller must hold a reference to it - * @drvdata: opaque data pointer to pass to bind callback - * - * Create a bond between device and address space, allowing the device to = access - * the mm using the returned PASID. If a bond already exists between @devi= ce and - * @mm, it is returned and an additional reference is taken. Caller must c= all - * iommu_sva_unbind_device() to release each reference. - * - * iommu_dev_enable_feature(dev, IOMMU_DEV_FEAT_SVA) must be called first,= to - * initialize the required SVA features. - * - * On error, returns an ERR_PTR value. - */ -struct iommu_sva * -iommu_sva_bind_device(struct device *dev, struct mm_struct *mm, void *drvd= ata) -{ - struct iommu_group *group; - struct iommu_sva *handle =3D ERR_PTR(-EINVAL); - const struct iommu_ops *ops =3D dev_iommu_ops(dev); - - if (!ops->sva_bind) - return ERR_PTR(-ENODEV); - - group =3D iommu_group_get(dev); - if (!group) - return ERR_PTR(-ENODEV); - - /* Ensure device count and domain don't change while we're binding */ - mutex_lock(&group->mutex); - - /* - * To keep things simple, SVA currently doesn't support IOMMU groups - * with more than one device. Existing SVA-capable systems are not - * affected by the problems that required IOMMU groups (lack of ACS - * isolation, device ID aliasing and other hardware issues). - */ - if (!iommu_group_singleton_lockdown(group)) - goto out_unlock; - - handle =3D ops->sva_bind(dev, mm, drvdata); - -out_unlock: - mutex_unlock(&group->mutex); - iommu_group_put(group); - - return handle; -} -EXPORT_SYMBOL_GPL(iommu_sva_bind_device); - -/** - * iommu_sva_unbind_device() - Remove a bond created with iommu_sva_bind_d= evice - * @handle: the handle returned by iommu_sva_bind_device() - * - * Put reference to a bond between device and address space. The device sh= ould - * not be issuing any more transaction for this PASID. All outstanding page - * requests for this PASID must have been flushed to the IOMMU. - */ -void iommu_sva_unbind_device(struct iommu_sva *handle) -{ - struct iommu_group *group; - struct device *dev =3D handle->dev; - const struct iommu_ops *ops =3D dev_iommu_ops(dev); - - if (!ops->sva_unbind) - return; - - group =3D iommu_group_get(dev); - if (!group) - return; - - mutex_lock(&group->mutex); - ops->sva_unbind(handle); - mutex_unlock(&group->mutex); - - iommu_group_put(group); -} -EXPORT_SYMBOL_GPL(iommu_sva_unbind_device); - -u32 iommu_sva_get_pasid(struct iommu_sva *handle) -{ - const struct iommu_ops *ops =3D dev_iommu_ops(handle->dev); - - if (!ops->sva_get_pasid) - return IOMMU_PASID_INVALID; - - return ops->sva_get_pasid(handle); -} -EXPORT_SYMBOL_GPL(iommu_sva_get_pasid); - /* * Changes the default domain of an iommu group that has *only* one device * --=20 2.25.1 From nobody Fri Jun 19 21:27:52 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 41C4DC433EF for ; Tue, 29 Mar 2022 05:41:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232073AbiC2Fmr (ORCPT ); Tue, 29 Mar 2022 01:42:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57478 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232578AbiC2Fmd (ORCPT ); Tue, 29 Mar 2022 01:42:33 -0400 Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D142E24A8A0 for ; Mon, 28 Mar 2022 22:40:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1648532445; x=1680068445; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=HPRpNxLNho273Kiyb7QbGF1e2R0pwTt9u5XD+VrfeyM=; b=NPS0pQs1gmPvLBH/mJ2ElrkM+Ip7EMGmGefLvp+KzYpdMc5HLA7At9qX /zOTDNJY8hxa9vS5zjwdONvkSwIybfqHOh93gJOo6+MQQAlN6xpB/e/YI setirw5YgvsbVGn/HkAkB4ABoepBNgsOmgH7fqBsYcG3ixQQStOLGRM+P jjwzV5LAVLf0D/wdpNPQKGG/BxZ5kRBxTWROt+kFYS38wkw+h4bK2nBT8 J34qtAUyQIrjDCH7bhS6mknTYZrbyZ1y/XQLl0TuLv+JE0Z00zdouSRbm zTgy2eyESghbl8Om/m+XrEe69jp/tKJ2gPl3Gx6UyNKCyALfn4jn6RMpt w==; X-IronPort-AV: E=McAfee;i="6200,9189,10300"; a="259137043" X-IronPort-AV: E=Sophos;i="5.90,219,1643702400"; d="scan'208";a="259137043" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Mar 2022 22:40:45 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.90,219,1643702400"; d="scan'208";a="694603675" Received: from allen-box.sh.intel.com ([10.239.159.48]) by fmsmga001.fm.intel.com with ESMTP; 28 Mar 2022 22:40:42 -0700 From: Lu Baolu To: Joerg Roedel , Jason Gunthorpe , Christoph Hellwig , Kevin Tian , Ashok Raj , Will Deacon , Robin Murphy , Jean-Philippe Brucker Cc: Eric Auger , Liu Yi L , Jacob jun Pan , iommu@lists.linux-foundation.org, linux-kernel@vger.kernel.org, Lu Baolu Subject: [PATCH RFC v2 09/11] iommu: Remove SVA related callbacks from iommu ops Date: Tue, 29 Mar 2022 13:37:58 +0800 Message-Id: <20220329053800.3049561-10-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220329053800.3049561-1-baolu.lu@linux.intel.com> References: <20220329053800.3049561-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" These ops'es have been replaced with the dev_attach/detach_pasid domain ops'es. There's no need for them anymore. Remove them to avoid dead code. Signed-off-by: Lu Baolu --- include/linux/intel-iommu.h | 4 -- include/linux/iommu.h | 8 --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 17 ------- drivers/iommu/iommu-sva-lib.h | 1 - .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 41 ---------------- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 3 -- drivers/iommu/intel/iommu.c | 3 -- drivers/iommu/intel/svm.c | 49 ------------------- drivers/iommu/iommu-sva-lib.c | 4 +- 9 files changed, 2 insertions(+), 128 deletions(-) diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h index c14283137fb5..7ce12590ce0f 100644 --- a/include/linux/intel-iommu.h +++ b/include/linux/intel-iommu.h @@ -738,10 +738,6 @@ struct intel_iommu *device_to_iommu(struct device *dev= , u8 *bus, u8 *devfn); extern void intel_svm_check(struct intel_iommu *iommu); extern int intel_svm_enable_prq(struct intel_iommu *iommu); extern int intel_svm_finish_prq(struct intel_iommu *iommu); -struct iommu_sva *intel_svm_bind(struct device *dev, struct mm_struct *mm, - void *drvdata); -void intel_svm_unbind(struct iommu_sva *handle); -u32 intel_svm_get_pasid(struct iommu_sva *handle); int intel_svm_page_response(struct device *dev, struct iommu_fault_event *= evt, struct iommu_page_response *msg); extern const struct iommu_domain_ops intel_svm_domain_ops; diff --git a/include/linux/iommu.h b/include/linux/iommu.h index 11c4d99e122d..7e30b88d7bef 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -213,9 +213,6 @@ struct iommu_iotlb_gather { * @dev_has/enable/disable_feat: per device entries to check/enable/disable * iommu specific features. * @dev_feat_enabled: check enabled feature - * @sva_bind: Bind process address space to device - * @sva_unbind: Unbind process address space from device - * @sva_get_pasid: Get PASID associated to a SVA handle * @page_response: handle page request response * @def_domain_type: device default domain type, return value: * - IOMMU_DOMAIN_IDENTITY: must use an identity domain @@ -249,11 +246,6 @@ struct iommu_ops { int (*dev_enable_feat)(struct device *dev, enum iommu_dev_features f); int (*dev_disable_feat)(struct device *dev, enum iommu_dev_features f); =20 - struct iommu_sva *(*sva_bind)(struct device *dev, struct mm_struct *mm, - void *drvdata); - void (*sva_unbind)(struct iommu_sva *handle); - u32 (*sva_get_pasid)(struct iommu_sva *handle); - int (*page_response)(struct device *dev, struct iommu_fault_event *evt, struct iommu_page_response *msg); diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/ar= m/arm-smmu-v3/arm-smmu-v3.h index 7631c00fdcbd..2513309ec0db 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -754,10 +754,6 @@ bool arm_smmu_master_sva_enabled(struct arm_smmu_maste= r *master); int arm_smmu_master_enable_sva(struct arm_smmu_master *master); int arm_smmu_master_disable_sva(struct arm_smmu_master *master); bool arm_smmu_master_iopf_supported(struct arm_smmu_master *master); -struct iommu_sva *arm_smmu_sva_bind(struct device *dev, struct mm_struct *= mm, - void *drvdata); -void arm_smmu_sva_unbind(struct iommu_sva *handle); -u32 arm_smmu_sva_get_pasid(struct iommu_sva *handle); void arm_smmu_sva_notifier_synchronize(void); int arm_smmu_sva_attach_dev_pasid(struct iommu_domain *domain, struct device *dev, ioasid_t id); @@ -794,19 +790,6 @@ static inline bool arm_smmu_master_iopf_supported(stru= ct arm_smmu_master *master return false; } =20 -static inline struct iommu_sva * -arm_smmu_sva_bind(struct device *dev, struct mm_struct *mm, void *drvdata) -{ - return ERR_PTR(-ENODEV); -} - -static inline void arm_smmu_sva_unbind(struct iommu_sva *handle) {} - -static inline u32 arm_smmu_sva_get_pasid(struct iommu_sva *handle) -{ - return IOMMU_PASID_INVALID; -} - static inline void arm_smmu_sva_notifier_synchronize(void) {} =20 static inline int arm_smmu_sva_attach_dev_pasid(struct iommu_domain *domai= n, diff --git a/drivers/iommu/iommu-sva-lib.h b/drivers/iommu/iommu-sva-lib.h index 1a71218b07f5..95d51f748dff 100644 --- a/drivers/iommu/iommu-sva-lib.h +++ b/drivers/iommu/iommu-sva-lib.h @@ -8,7 +8,6 @@ #include #include =20 -int iommu_sva_alloc_pasid(struct mm_struct *mm, ioasid_t min, ioasid_t max= ); struct mm_struct *iommu_sva_find(ioasid_t pasid); struct mm_struct *iommu_sva_domain_mm(struct iommu_domain *domain); =20 diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iomm= u/arm/arm-smmu-v3/arm-smmu-v3-sva.c index ce229820086d..5591321f9e11 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -328,11 +328,6 @@ __arm_smmu_sva_bind(struct device *dev, struct mm_stru= ct *mm) if (!bond) return ERR_PTR(-ENOMEM); =20 - /* Allocate a PASID for this mm if necessary */ - ret =3D iommu_sva_alloc_pasid(mm, 1, (1U << master->ssid_bits) - 1); - if (ret) - goto err_free_bond; - bond->mm =3D mm; bond->sva.dev =3D dev; refcount_set(&bond->refs, 1); @@ -351,42 +346,6 @@ __arm_smmu_sva_bind(struct device *dev, struct mm_stru= ct *mm) return ERR_PTR(ret); } =20 -struct iommu_sva * -arm_smmu_sva_bind(struct device *dev, struct mm_struct *mm, void *drvdata) -{ - struct iommu_sva *handle; - struct iommu_domain *domain =3D iommu_get_domain_for_dev(dev); - struct arm_smmu_domain *smmu_domain =3D to_smmu_domain(domain); - - if (smmu_domain->stage !=3D ARM_SMMU_DOMAIN_S1) - return ERR_PTR(-EINVAL); - - mutex_lock(&sva_lock); - handle =3D __arm_smmu_sva_bind(dev, mm); - mutex_unlock(&sva_lock); - return handle; -} - -void arm_smmu_sva_unbind(struct iommu_sva *handle) -{ - struct arm_smmu_bond *bond =3D sva_to_bond(handle); - - mutex_lock(&sva_lock); - if (refcount_dec_and_test(&bond->refs)) { - list_del(&bond->list); - arm_smmu_mmu_notifier_put(bond->smmu_mn); - kfree(bond); - } - mutex_unlock(&sva_lock); -} - -u32 arm_smmu_sva_get_pasid(struct iommu_sva *handle) -{ - struct arm_smmu_bond *bond =3D sva_to_bond(handle); - - return bond->mm->pasid; -} - bool arm_smmu_sva_supported(struct arm_smmu_device *smmu) { unsigned long reg, fld; diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/ar= m/arm-smmu-v3/arm-smmu-v3.c index bd80de0bad98..543d3ef1c102 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -2875,9 +2875,6 @@ static struct iommu_ops arm_smmu_ops =3D { .dev_feat_enabled =3D arm_smmu_dev_feature_enabled, .dev_enable_feat =3D arm_smmu_dev_enable_feature, .dev_disable_feat =3D arm_smmu_dev_disable_feature, - .sva_bind =3D arm_smmu_sva_bind, - .sva_unbind =3D arm_smmu_sva_unbind, - .sva_get_pasid =3D arm_smmu_sva_get_pasid, .page_response =3D arm_smmu_page_response, .pgsize_bitmap =3D -1UL, /* Restricted during device attach */ .owner =3D THIS_MODULE, diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c index 5eae7cf9bee5..1d6a62512bca 100644 --- a/drivers/iommu/intel/iommu.c +++ b/drivers/iommu/intel/iommu.c @@ -4890,9 +4890,6 @@ const struct iommu_ops intel_iommu_ops =3D { .def_domain_type =3D device_def_domain_type, .pgsize_bitmap =3D SZ_4K, #ifdef CONFIG_INTEL_IOMMU_SVM - .sva_bind =3D intel_svm_bind, - .sva_unbind =3D intel_svm_unbind, - .sva_get_pasid =3D intel_svm_get_pasid, .page_response =3D intel_svm_page_response, #endif .default_domain_ops =3D &(const struct iommu_domain_ops) { diff --git a/drivers/iommu/intel/svm.c b/drivers/iommu/intel/svm.c index 8f59dd641b2d..c76b99b46626 100644 --- a/drivers/iommu/intel/svm.c +++ b/drivers/iommu/intel/svm.c @@ -313,14 +313,6 @@ static int pasid_to_svm_sdev(struct device *dev, unsig= ned int pasid, return 0; } =20 -static int intel_svm_alloc_pasid(struct device *dev, struct mm_struct *mm) -{ - ioasid_t max_pasid =3D dev_is_pci(dev) ? - pci_max_pasids(to_pci_dev(dev)) : intel_pasid_max_id; - - return iommu_sva_alloc_pasid(mm, PASID_MIN, max_pasid - 1); -} - static struct iommu_sva *intel_svm_bind_mm(struct intel_iommu *iommu, struct device *dev, struct mm_struct *mm) @@ -810,47 +802,6 @@ static irqreturn_t prq_event_thread(int irq, void *d) return IRQ_RETVAL(handled); } =20 -struct iommu_sva *intel_svm_bind(struct device *dev, struct mm_struct *mm,= void *drvdata) -{ - struct intel_iommu *iommu =3D device_to_iommu(dev, NULL, NULL); - struct iommu_sva *sva; - int ret; - - mutex_lock(&pasid_mutex); - ret =3D intel_svm_alloc_pasid(dev, mm); - if (ret) { - mutex_unlock(&pasid_mutex); - return ERR_PTR(ret); - } - - sva =3D intel_svm_bind_mm(iommu, dev, mm); - mutex_unlock(&pasid_mutex); - - return sva; -} - -void intel_svm_unbind(struct iommu_sva *sva) -{ - struct intel_svm_dev *sdev =3D to_intel_svm_dev(sva); - - mutex_lock(&pasid_mutex); - intel_svm_unbind_mm(sdev->dev, sdev->pasid); - mutex_unlock(&pasid_mutex); -} - -u32 intel_svm_get_pasid(struct iommu_sva *sva) -{ - struct intel_svm_dev *sdev; - u32 pasid; - - mutex_lock(&pasid_mutex); - sdev =3D to_intel_svm_dev(sva); - pasid =3D sdev->pasid; - mutex_unlock(&pasid_mutex); - - return pasid; -} - int intel_svm_page_response(struct device *dev, struct iommu_fault_event *evt, struct iommu_page_response *msg) diff --git a/drivers/iommu/iommu-sva-lib.c b/drivers/iommu/iommu-sva-lib.c index 1b45b7d01836..96e967e58aa9 100644 --- a/drivers/iommu/iommu-sva-lib.c +++ b/drivers/iommu/iommu-sva-lib.c @@ -32,7 +32,8 @@ struct iommu_sva_cookie { * * Returns 0 on success and < 0 on error. */ -int iommu_sva_alloc_pasid(struct mm_struct *mm, ioasid_t min, ioasid_t max) +static int iommu_sva_alloc_pasid(struct mm_struct *mm, + ioasid_t min, ioasid_t max) { int ret =3D 0; ioasid_t pasid; @@ -58,7 +59,6 @@ int iommu_sva_alloc_pasid(struct mm_struct *mm, ioasid_t = min, ioasid_t max) mutex_unlock(&iommu_sva_lock); return ret; } -EXPORT_SYMBOL_GPL(iommu_sva_alloc_pasid); =20 /* ioasid_find getter() requires a void * argument */ static bool __mmget_not_zero(void *mm) --=20 2.25.1 From nobody Fri Jun 19 21:27:52 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 92B89C433F5 for ; Tue, 29 Mar 2022 05:41:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232600AbiC2Fmy (ORCPT ); Tue, 29 Mar 2022 01:42:54 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57914 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232627AbiC2Fmg (ORCPT ); Tue, 29 Mar 2022 01:42:36 -0400 Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A41E324B0B7 for ; Mon, 28 Mar 2022 22:40:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1648532449; x=1680068449; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=oNG4UW0zzEZch3nv0WWBRbW0DC9J7Xey5ODKkvnd/rY=; b=HAzIcBozggvYMzi3kPYDqqp7k76e+aCM/bMY5OgZAdmgHTedDVVLsv0l JPzDkjFpZATQ4FIDhhhFCf6uvCxpqZVxoPV8rqp+kW3ZhaO7SFtZKBS2c krWhQU89wov+RPLhFK5BNh4a37eHIH2On5QLR47DSHK5Kqcc+XPFfnkxj CohpGwlLLlh1J4w9cSU8ohImDrJ2nFIE9xPYFiwdzhWZE4obTqOCLH0XQ NcleAKBvgsp07MgP7+Bcp7MjeecjYgQx7DA5/Cf6jw77yRKCD9C7KdRlR N8TILagDOef3cHu+ueHEqqk5hWgldolrXe9+L/s/SHSNpPcD8z8leFIKP g==; X-IronPort-AV: E=McAfee;i="6200,9189,10300"; a="259137052" X-IronPort-AV: E=Sophos;i="5.90,219,1643702400"; d="scan'208";a="259137052" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Mar 2022 22:40:49 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.90,219,1643702400"; d="scan'208";a="694603685" Received: from allen-box.sh.intel.com ([10.239.159.48]) by fmsmga001.fm.intel.com with ESMTP; 28 Mar 2022 22:40:45 -0700 From: Lu Baolu To: Joerg Roedel , Jason Gunthorpe , Christoph Hellwig , Kevin Tian , Ashok Raj , Will Deacon , Robin Murphy , Jean-Philippe Brucker Cc: Eric Auger , Liu Yi L , Jacob jun Pan , iommu@lists.linux-foundation.org, linux-kernel@vger.kernel.org, Lu Baolu Subject: [PATCH RFC v2 10/11] iommu: Per-domain I/O page fault handling Date: Tue, 29 Mar 2022 13:37:59 +0800 Message-Id: <20220329053800.3049561-11-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220329053800.3049561-1-baolu.lu@linux.intel.com> References: <20220329053800.3049561-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Tweak the I/O page fault handling framework to route the page faults to the domain and call the page fault handler retrieved from the domain. This makes the I/O page fault handling framework possible to serve more usage scenarios as long as they have an IOMMU domain and install a page fault handler in it. This also refactors the SVA implementation to be the first consumer of the per-domain page fault handling model. Signed-off-by: Lu Baolu --- include/linux/iommu.h | 4 ++ drivers/iommu/iommu-sva-lib.h | 1 - drivers/iommu/io-pgfault.c | 70 +++++++--------------------------- drivers/iommu/iommu-sva-lib.c | 71 +++++++++++++++++++++++++++-------- 4 files changed, 72 insertions(+), 74 deletions(-) diff --git a/include/linux/iommu.h b/include/linux/iommu.h index 7e30b88d7bef..729d602fcba5 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -51,6 +51,8 @@ struct iommu_sva_cookie; typedef int (*iommu_fault_handler_t)(struct iommu_domain *, struct device *, unsigned long, int, void *); typedef int (*iommu_dev_fault_handler_t)(struct iommu_fault *, void *); +typedef enum iommu_page_response_code (*iommu_domain_iopf_handler_t) + (struct iommu_fault *, void *); =20 struct iommu_domain_geometry { dma_addr_t aperture_start; /* First address that can be mapped */ @@ -102,6 +104,8 @@ struct iommu_domain { struct iommu_domain_geometry geometry; struct iommu_dma_cookie *iova_cookie; struct iommu_sva_cookie *sva_cookie; + iommu_domain_iopf_handler_t fault_handler; + void *fault_data; }; =20 static inline bool iommu_is_dma_domain(struct iommu_domain *domain) diff --git a/drivers/iommu/iommu-sva-lib.h b/drivers/iommu/iommu-sva-lib.h index 95d51f748dff..abef83ac1b2d 100644 --- a/drivers/iommu/iommu-sva-lib.h +++ b/drivers/iommu/iommu-sva-lib.h @@ -8,7 +8,6 @@ #include #include =20 -struct mm_struct *iommu_sva_find(ioasid_t pasid); struct mm_struct *iommu_sva_domain_mm(struct iommu_domain *domain); =20 /* I/O Page fault */ diff --git a/drivers/iommu/io-pgfault.c b/drivers/iommu/io-pgfault.c index 1df8c1dcae77..159e4f107fe3 100644 --- a/drivers/iommu/io-pgfault.c +++ b/drivers/iommu/io-pgfault.c @@ -69,62 +69,6 @@ static int iopf_complete_group(struct device *dev, struc= t iopf_fault *iopf, return iommu_page_response(dev, &resp); } =20 -static enum iommu_page_response_code -iopf_handle_single(struct iopf_fault *iopf) -{ - vm_fault_t ret; - struct mm_struct *mm; - struct vm_area_struct *vma; - unsigned int access_flags =3D 0; - unsigned int fault_flags =3D FAULT_FLAG_REMOTE; - struct iommu_fault_page_request *prm =3D &iopf->fault.prm; - enum iommu_page_response_code status =3D IOMMU_PAGE_RESP_INVALID; - - if (!(prm->flags & IOMMU_FAULT_PAGE_REQUEST_PASID_VALID)) - return status; - - mm =3D iommu_sva_find(prm->pasid); - if (IS_ERR_OR_NULL(mm)) - return status; - - mmap_read_lock(mm); - - vma =3D find_extend_vma(mm, prm->addr); - if (!vma) - /* Unmapped area */ - goto out_put_mm; - - if (prm->perm & IOMMU_FAULT_PERM_READ) - access_flags |=3D VM_READ; - - if (prm->perm & IOMMU_FAULT_PERM_WRITE) { - access_flags |=3D VM_WRITE; - fault_flags |=3D FAULT_FLAG_WRITE; - } - - if (prm->perm & IOMMU_FAULT_PERM_EXEC) { - access_flags |=3D VM_EXEC; - fault_flags |=3D FAULT_FLAG_INSTRUCTION; - } - - if (!(prm->perm & IOMMU_FAULT_PERM_PRIV)) - fault_flags |=3D FAULT_FLAG_USER; - - if (access_flags & ~vma->vm_flags) - /* Access fault */ - goto out_put_mm; - - ret =3D handle_mm_fault(vma, prm->addr, fault_flags, NULL); - status =3D ret & VM_FAULT_ERROR ? IOMMU_PAGE_RESP_INVALID : - IOMMU_PAGE_RESP_SUCCESS; - -out_put_mm: - mmap_read_unlock(mm); - mmput(mm); - - return status; -} - static void iopf_handle_group(struct work_struct *work) { struct iopf_group *group; @@ -134,12 +78,24 @@ static void iopf_handle_group(struct work_struct *work) group =3D container_of(work, struct iopf_group, work); =20 list_for_each_entry_safe(iopf, next, &group->faults, list) { + struct iommu_domain *domain; + + domain =3D iommu_get_domain_for_dev_pasid(group->dev, + iopf->fault.prm.pasid); + + if (!domain || !domain->fault_handler) + status =3D IOMMU_PAGE_RESP_INVALID; + /* * For the moment, errors are sticky: don't handle subsequent * faults in the group if there is an error. */ if (status =3D=3D IOMMU_PAGE_RESP_SUCCESS) - status =3D iopf_handle_single(iopf); + status =3D domain->fault_handler(&iopf->fault, + domain->fault_data); + + if (domain) + iommu_put_domain_for_dev_pasid(domain); =20 if (!(iopf->fault.prm.flags & IOMMU_FAULT_PAGE_REQUEST_LAST_PAGE)) diff --git a/drivers/iommu/iommu-sva-lib.c b/drivers/iommu/iommu-sva-lib.c index 96e967e58aa9..a53574f9559a 100644 --- a/drivers/iommu/iommu-sva-lib.c +++ b/drivers/iommu/iommu-sva-lib.c @@ -60,25 +60,62 @@ static int iommu_sva_alloc_pasid(struct mm_struct *mm, return ret; } =20 -/* ioasid_find getter() requires a void * argument */ -static bool __mmget_not_zero(void *mm) +static enum iommu_page_response_code +iommu_sva_handle_iopf(struct iommu_fault *fault, void *data) { - return mmget_not_zero(mm); -} + vm_fault_t ret; + struct mm_struct *mm; + struct vm_area_struct *vma; + unsigned int access_flags =3D 0; + struct iommu_domain *domain =3D data; + unsigned int fault_flags =3D FAULT_FLAG_REMOTE; + struct iommu_fault_page_request *prm =3D &fault->prm; + enum iommu_page_response_code status =3D IOMMU_PAGE_RESP_INVALID; =20 -/** - * iommu_sva_find() - Find mm associated to the given PASID - * @pasid: Process Address Space ID assigned to the mm - * - * On success a reference to the mm is taken, and must be released with mm= put(). - * - * Returns the mm corresponding to this PASID, or an error if not found. - */ -struct mm_struct *iommu_sva_find(ioasid_t pasid) -{ - return ioasid_find(&iommu_sva_pasid, pasid, __mmget_not_zero); + if (!(prm->flags & IOMMU_FAULT_PAGE_REQUEST_PASID_VALID)) + return status; + + mm =3D iommu_sva_domain_mm(domain); + if (IS_ERR_OR_NULL(mm) || !mmget_not_zero(mm)) + return status; + + mmap_read_lock(mm); + + vma =3D find_extend_vma(mm, prm->addr); + if (!vma) + /* Unmapped area */ + goto out_put_mm; + + if (prm->perm & IOMMU_FAULT_PERM_READ) + access_flags |=3D VM_READ; + + if (prm->perm & IOMMU_FAULT_PERM_WRITE) { + access_flags |=3D VM_WRITE; + fault_flags |=3D FAULT_FLAG_WRITE; + } + + if (prm->perm & IOMMU_FAULT_PERM_EXEC) { + access_flags |=3D VM_EXEC; + fault_flags |=3D FAULT_FLAG_INSTRUCTION; + } + + if (!(prm->perm & IOMMU_FAULT_PERM_PRIV)) + fault_flags |=3D FAULT_FLAG_USER; + + if (access_flags & ~vma->vm_flags) + /* Access fault */ + goto out_put_mm; + + ret =3D handle_mm_fault(vma, prm->addr, fault_flags, NULL); + status =3D ret & VM_FAULT_ERROR ? IOMMU_PAGE_RESP_INVALID : + IOMMU_PAGE_RESP_SUCCESS; + +out_put_mm: + mmap_read_unlock(mm); + mmput(mm); + + return status; } -EXPORT_SYMBOL_GPL(iommu_sva_find); =20 static struct iommu_domain * iommu_sva_alloc_domain(struct device *dev, struct mm_struct *mm) @@ -105,6 +142,8 @@ iommu_sva_alloc_domain(struct device *dev, struct mm_st= ruct *mm) INIT_LIST_HEAD(&cookie->bonds); domain->type =3D IOMMU_DOMAIN_SVA; domain->sva_cookie =3D cookie; + domain->fault_handler =3D iommu_sva_handle_iopf; + domain->fault_data =3D domain; curr =3D xa_store(&sva_domain_array, mm->pasid, domain, GFP_KERNEL); if (xa_err(curr)) goto err_xa_store; --=20 2.25.1 From nobody Fri Jun 19 21:27:52 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A0568C433F5 for ; Tue, 29 Mar 2022 05:41:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232648AbiC2Fm7 (ORCPT ); Tue, 29 Mar 2022 01:42:59 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57518 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232568AbiC2Fmk (ORCPT ); Tue, 29 Mar 2022 01:42:40 -0400 Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E211B24A764 for ; Mon, 28 Mar 2022 22:40:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1648532453; x=1680068453; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=JfGjVHRgrpt8QUxiPQxKQ7422++u6wuqZXu+sum80fQ=; b=oHkE9Zw/3DRO8W3iyKM5sO09trgqpJr4KADCKsaDpW/B2QhWfJ5k/jiQ yTp2LkPOS6aPv64iEztr/0a4A7e9E7O6VplgauvyLdyDpLhOdkNcQr7wV uBUE5JjyrWOsNBlJv5pW9GP405LOoMdoJrsf0G6AWuQb+5tYZKl96/P+k 1MkZUSabkcNnrNr/JSsdbL9Ypab3uXQi8ziRSqv/9pYFGrfvmNOKG8lM+ OWoaP2AufS9KCSCeaiBDwl1ugLik4rEr0qmzWjEBZbHOpaeAjrL9o7KW/ bVGalORwajcu7d66mS578TjejYf3eGbxSmxPXY+7NrozcQpqRMT2uJbCy g==; X-IronPort-AV: E=McAfee;i="6200,9189,10300"; a="259137066" X-IronPort-AV: E=Sophos;i="5.90,219,1643702400"; d="scan'208";a="259137066" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Mar 2022 22:40:52 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.90,219,1643702400"; d="scan'208";a="694603691" Received: from allen-box.sh.intel.com ([10.239.159.48]) by fmsmga001.fm.intel.com with ESMTP; 28 Mar 2022 22:40:49 -0700 From: Lu Baolu To: Joerg Roedel , Jason Gunthorpe , Christoph Hellwig , Kevin Tian , Ashok Raj , Will Deacon , Robin Murphy , Jean-Philippe Brucker Cc: Eric Auger , Liu Yi L , Jacob jun Pan , iommu@lists.linux-foundation.org, linux-kernel@vger.kernel.org, Lu Baolu Subject: [PATCH RFC v2 11/11] iommu: Rename iommu-sva-lib.{c,h} Date: Tue, 29 Mar 2022 13:38:00 +0800 Message-Id: <20220329053800.3049561-12-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220329053800.3049561-1-baolu.lu@linux.intel.com> References: <20220329053800.3049561-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Rename iommu-sva-lib.c[h] to iommu-sva.c[h] as it contains all code for SVA implementation in iommu core. Signed-off-by: Lu Baolu --- drivers/iommu/{iommu-sva-lib.h =3D> iommu-sva.h} | 0 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 2 +- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 2 +- drivers/iommu/intel/iommu.c | 2 +- drivers/iommu/intel/svm.c | 2 +- drivers/iommu/io-pgfault.c | 2 +- drivers/iommu/{iommu-sva-lib.c =3D> iommu-sva.c} | 2 +- drivers/iommu/iommu.c | 2 +- drivers/iommu/Makefile | 2 +- 9 files changed, 8 insertions(+), 8 deletions(-) rename drivers/iommu/{iommu-sva-lib.h =3D> iommu-sva.h} (100%) rename drivers/iommu/{iommu-sva-lib.c =3D> iommu-sva.c} (99%) diff --git a/drivers/iommu/iommu-sva-lib.h b/drivers/iommu/iommu-sva.h similarity index 100% rename from drivers/iommu/iommu-sva-lib.h rename to drivers/iommu/iommu-sva.h diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iomm= u/arm/arm-smmu-v3/arm-smmu-v3-sva.c index 5591321f9e11..a13543aa9df4 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -9,7 +9,7 @@ #include =20 #include "arm-smmu-v3.h" -#include "../../iommu-sva-lib.h" +#include "../../iommu-sva.h" #include "../../io-pgtable-arm.h" =20 struct arm_smmu_mmu_notifier { diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/ar= m/arm-smmu-v3/arm-smmu-v3.c index 543d3ef1c102..ca2bd17eec41 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -31,7 +31,7 @@ #include =20 #include "arm-smmu-v3.h" -#include "../../iommu-sva-lib.h" +#include "../../iommu-sva.h" =20 static bool disable_bypass =3D true; module_param(disable_bypass, bool, 0444); diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c index 1d6a62512bca..57a3d54c414b 100644 --- a/drivers/iommu/intel/iommu.c +++ b/drivers/iommu/intel/iommu.c @@ -27,7 +27,7 @@ #include =20 #include "../irq_remapping.h" -#include "../iommu-sva-lib.h" +#include "../iommu-sva.h" #include "pasid.h" #include "cap_audit.h" =20 diff --git a/drivers/iommu/intel/svm.c b/drivers/iommu/intel/svm.c index c76b99b46626..b6b83ca705d5 100644 --- a/drivers/iommu/intel/svm.c +++ b/drivers/iommu/intel/svm.c @@ -25,7 +25,7 @@ =20 #include "pasid.h" #include "perf.h" -#include "../iommu-sva-lib.h" +#include "../iommu-sva.h" =20 static irqreturn_t prq_event_thread(int irq, void *d); static void intel_svm_drain_prq(struct device *dev, u32 pasid); diff --git a/drivers/iommu/io-pgfault.c b/drivers/iommu/io-pgfault.c index 159e4f107fe3..5468993c8c6c 100644 --- a/drivers/iommu/io-pgfault.c +++ b/drivers/iommu/io-pgfault.c @@ -11,7 +11,7 @@ #include #include =20 -#include "iommu-sva-lib.h" +#include "iommu-sva.h" =20 /** * struct iopf_queue - IO Page Fault queue diff --git a/drivers/iommu/iommu-sva-lib.c b/drivers/iommu/iommu-sva.c similarity index 99% rename from drivers/iommu/iommu-sva-lib.c rename to drivers/iommu/iommu-sva.c index a53574f9559a..dbe4b9958e48 100644 --- a/drivers/iommu/iommu-sva-lib.c +++ b/drivers/iommu/iommu-sva.c @@ -7,7 +7,7 @@ #include #include =20 -#include "iommu-sva-lib.h" +#include "iommu-sva.h" =20 static DEFINE_MUTEX(iommu_sva_lock); static DECLARE_IOASID_SET(iommu_sva_pasid); diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c index 6b51ead9d63b..f4a5d6351421 100644 --- a/drivers/iommu/iommu.c +++ b/drivers/iommu/iommu.c @@ -27,7 +27,7 @@ #include #include =20 -#include "iommu-sva-lib.h" +#include "iommu-sva.h" =20 static struct kset *iommu_group_kset; static DEFINE_IDA(iommu_group_ida); diff --git a/drivers/iommu/Makefile b/drivers/iommu/Makefile index 44475a9b3eea..c1763476162b 100644 --- a/drivers/iommu/Makefile +++ b/drivers/iommu/Makefile @@ -27,6 +27,6 @@ obj-$(CONFIG_FSL_PAMU) +=3D fsl_pamu.o fsl_pamu_domain.o obj-$(CONFIG_S390_IOMMU) +=3D s390-iommu.o obj-$(CONFIG_HYPERV_IOMMU) +=3D hyperv-iommu.o obj-$(CONFIG_VIRTIO_IOMMU) +=3D virtio-iommu.o -obj-$(CONFIG_IOMMU_SVA) +=3D iommu-sva-lib.o io-pgfault.o +obj-$(CONFIG_IOMMU_SVA) +=3D iommu-sva.o io-pgfault.o obj-$(CONFIG_SPRD_IOMMU) +=3D sprd-iommu.o obj-$(CONFIG_APPLE_DART) +=3D apple-dart.o --=20 2.25.1