From nobody Fri May 8 06:00:20 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 226CAC433EF for ; Tue, 10 May 2022 06:21:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237081AbiEJGZB (ORCPT ); Tue, 10 May 2022 02:25:01 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33574 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236552AbiEJGYx (ORCPT ); Tue, 10 May 2022 02:24:53 -0400 Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3B5A41CB0E for ; Mon, 9 May 2022 23:20:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1652163655; x=1683699655; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=pRXrCzT/RrwXpbxi4XQ00QCp7eyOS42v/Go3QgjMA9U=; b=iiZnuwv3Ql2UE89SJ/wtXwXU8pjJ5EEJXkqvMhvA5v/A6o1wJxDorhjh KF+VpyH3/vANBorwpcn6o4xoZyG+HqIzuA/ByOplDfBHrJSDY8xrBKzSM pzQXGit4BVttzpAnqwvcTeNoMCJGn58qVMj7cH4NQkdZPPD5HJnJA2Rq8 uWQjqXQBv0axsWgTUf1SddJZwIAXekpbblKuIFPTrBtFpmt/wppacC/9R bNBI0kVedalAvcVw1Pg5pVYZnd5qDB1aXjC7xPPOyLIViLd9+uEAxVhOR ylhbmkPst3zZHPMZ1hNCyu9XLWDm7VFvLvxYX2Ega5I5Ho2H+lH4tkLxR g==; X-IronPort-AV: E=McAfee;i="6400,9594,10342"; a="332312851" X-IronPort-AV: E=Sophos;i="5.91,213,1647327600"; d="scan'208";a="332312851" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 May 2022 23:20:51 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.91,213,1647327600"; d="scan'208";a="552636333" Received: from allen-box.sh.intel.com ([10.239.159.48]) by orsmga002.jf.intel.com with ESMTP; 09 May 2022 23:20:47 -0700 From: Lu Baolu To: Joerg Roedel , Jason Gunthorpe , Christoph Hellwig , Kevin Tian , Ashok Raj , Will Deacon , Robin Murphy , Jean-Philippe Brucker , Dave Jiang , Vinod Koul Cc: Eric Auger , Liu Yi L , Jacob jun Pan , iommu@lists.linux-foundation.org, linux-kernel@vger.kernel.org, Jacob Pan , Lu Baolu Subject: [PATCH v6 01/12] dmaengine: idxd: Separate user and kernel pasid enabling Date: Tue, 10 May 2022 14:17:27 +0800 Message-Id: <20220510061738.2761430-2-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220510061738.2761430-1-baolu.lu@linux.intel.com> References: <20220510061738.2761430-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Dave Jiang The idxd driver always gated the pasid enabling under a single knob and this assumption is incorrect. The pasid used for kernel operation can be independently toggled and has no dependency on the user pasid (and vice versa). Split the two so they are independent "enabled" flags. Cc: Vinod Koul Signed-off-by: Dave Jiang Signed-off-by: Jacob Pan Link: https://lore.kernel.org/linux-iommu/20220315050713.2000518-10-jacob.j= un.pan@linux.intel.com/ Signed-off-by: Lu Baolu --- drivers/dma/idxd/idxd.h | 6 ++++++ drivers/dma/idxd/cdev.c | 4 ++-- drivers/dma/idxd/init.c | 30 ++++++++++++++++++------------ 3 files changed, 26 insertions(+), 14 deletions(-) diff --git a/drivers/dma/idxd/idxd.h b/drivers/dma/idxd/idxd.h index da72eb15f610..ccbefd0be617 100644 --- a/drivers/dma/idxd/idxd.h +++ b/drivers/dma/idxd/idxd.h @@ -239,6 +239,7 @@ enum idxd_device_flag { IDXD_FLAG_CONFIGURABLE =3D 0, IDXD_FLAG_CMD_RUNNING, IDXD_FLAG_PASID_ENABLED, + IDXD_FLAG_USER_PASID_ENABLED, }; =20 struct idxd_dma_dev { @@ -469,6 +470,11 @@ static inline bool device_pasid_enabled(struct idxd_de= vice *idxd) return test_bit(IDXD_FLAG_PASID_ENABLED, &idxd->flags); } =20 +static inline bool device_user_pasid_enabled(struct idxd_device *idxd) +{ + return test_bit(IDXD_FLAG_USER_PASID_ENABLED, &idxd->flags); +} + static inline bool device_swq_supported(struct idxd_device *idxd) { return (support_enqcmd && device_pasid_enabled(idxd)); diff --git a/drivers/dma/idxd/cdev.c b/drivers/dma/idxd/cdev.c index b9b2b4a4124e..7df996deffbe 100644 --- a/drivers/dma/idxd/cdev.c +++ b/drivers/dma/idxd/cdev.c @@ -99,7 +99,7 @@ static int idxd_cdev_open(struct inode *inode, struct fil= e *filp) ctx->wq =3D wq; filp->private_data =3D ctx; =20 - if (device_pasid_enabled(idxd)) { + if (device_user_pasid_enabled(idxd)) { sva =3D iommu_sva_bind_device(dev, current->mm, NULL); if (IS_ERR(sva)) { rc =3D PTR_ERR(sva); @@ -152,7 +152,7 @@ static int idxd_cdev_release(struct inode *node, struct= file *filep) if (wq_shared(wq)) { idxd_device_drain_pasid(idxd, ctx->pasid); } else { - if (device_pasid_enabled(idxd)) { + if (device_user_pasid_enabled(idxd)) { /* The wq disable in the disable pasid function will drain the wq */ rc =3D idxd_wq_disable_pasid(wq); if (rc < 0) diff --git a/drivers/dma/idxd/init.c b/drivers/dma/idxd/init.c index 993a5dcca24f..e1b5d1e4a949 100644 --- a/drivers/dma/idxd/init.c +++ b/drivers/dma/idxd/init.c @@ -513,16 +513,19 @@ static int idxd_probe(struct idxd_device *idxd) =20 if (IS_ENABLED(CONFIG_INTEL_IDXD_SVM) && sva) { rc =3D iommu_dev_enable_feature(dev, IOMMU_DEV_FEAT_SVA); - if (rc =3D=3D 0) { - rc =3D idxd_enable_system_pasid(idxd); - if (rc < 0) { - iommu_dev_disable_feature(dev, IOMMU_DEV_FEAT_SVA); - dev_warn(dev, "Failed to enable PASID. No SVA support: %d\n", rc); - } else { - set_bit(IDXD_FLAG_PASID_ENABLED, &idxd->flags); - } - } else { + if (rc) { + /* + * Do not bail here since legacy DMA is still + * supported, both user and in-kernel DMA with + * PASID rely on SVA feature. + */ dev_warn(dev, "Unable to turn on SVA feature.\n"); + } else { + set_bit(IDXD_FLAG_USER_PASID_ENABLED, &idxd->flags); + if (idxd_enable_system_pasid(idxd)) + dev_warn(dev, "No in-kernel DMA with PASID.\n"); + else + set_bit(IDXD_FLAG_PASID_ENABLED, &idxd->flags); } } else if (!sva) { dev_warn(dev, "User forced SVA off via module param.\n"); @@ -561,7 +564,8 @@ static int idxd_probe(struct idxd_device *idxd) err: if (device_pasid_enabled(idxd)) idxd_disable_system_pasid(idxd); - iommu_dev_disable_feature(dev, IOMMU_DEV_FEAT_SVA); + if (device_user_pasid_enabled(idxd) || device_pasid_enabled(idxd)) + iommu_dev_disable_feature(dev, IOMMU_DEV_FEAT_SVA); return rc; } =20 @@ -574,7 +578,8 @@ static void idxd_cleanup(struct idxd_device *idxd) idxd_cleanup_internals(idxd); if (device_pasid_enabled(idxd)) idxd_disable_system_pasid(idxd); - iommu_dev_disable_feature(dev, IOMMU_DEV_FEAT_SVA); + if (device_user_pasid_enabled(idxd) || device_pasid_enabled(idxd)) + iommu_dev_disable_feature(dev, IOMMU_DEV_FEAT_SVA); } =20 static int idxd_pci_probe(struct pci_dev *pdev, const struct pci_device_id= *id) @@ -691,7 +696,8 @@ static void idxd_remove(struct pci_dev *pdev) free_irq(irq_entry->vector, irq_entry); pci_free_irq_vectors(pdev); pci_iounmap(pdev, idxd->reg_base); - iommu_dev_disable_feature(&pdev->dev, IOMMU_DEV_FEAT_SVA); + if (device_user_pasid_enabled(idxd) || device_pasid_enabled(idxd)) + iommu_dev_disable_feature(&pdev->dev, IOMMU_DEV_FEAT_SVA); pci_disable_device(pdev); destroy_workqueue(idxd->wq); perfmon_pmu_remove(idxd); --=20 2.25.1 From nobody Fri May 8 06:00:20 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9612FC433EF for ; Tue, 10 May 2022 06:21:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237102AbiEJGZJ (ORCPT ); Tue, 10 May 2022 02:25:09 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33592 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232406AbiEJGYx (ORCPT ); Tue, 10 May 2022 02:24:53 -0400 Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E0D541DA59 for ; Mon, 9 May 2022 23:20:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1652163657; x=1683699657; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=l8dfTFBmYxhrsbRwhY44CdbQ8TexdcQQbR+MJY4tLVs=; b=C2hVPz6amInLXG/St7h7Up9FdIbPoJI7Q+ty/rhw/vOveN4vT57uGE9W xDKdebpSuiNVhlcO/bsneZKURbBITCAh5tkZtZKud5Av+dcSuGCH/9nUY 6z5VpXg2Ks/bnfVIzzOodyOC0NxG1DOhY3Lgj+OYJXsvAb9wZwWIjTDuX 3qVLcJEUg6OPaNYfJDbBCv6y+oPklPr9/NOb+TR3/U1PQChtybgIT9Mi2 Tsbfz+xDpoH1R9swRmxamlu7ZNJGtW5dd+1bUAoEHEPklVDFG9azclmet Xav1KBOj1ywrhHl9LmKpgE4zhO+YNxeUtP/S0nWPy1lSKIXdc+7holzFL Q==; X-IronPort-AV: E=McAfee;i="6400,9594,10342"; a="332312866" X-IronPort-AV: E=Sophos;i="5.91,213,1647327600"; d="scan'208";a="332312866" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 May 2022 23:20:55 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.91,213,1647327600"; d="scan'208";a="552636366" Received: from allen-box.sh.intel.com ([10.239.159.48]) by orsmga002.jf.intel.com with ESMTP; 09 May 2022 23:20:51 -0700 From: Lu Baolu To: Joerg Roedel , Jason Gunthorpe , Christoph Hellwig , Kevin Tian , Ashok Raj , Will Deacon , Robin Murphy , Jean-Philippe Brucker , Dave Jiang , Vinod Koul Cc: Eric Auger , Liu Yi L , Jacob jun Pan , iommu@lists.linux-foundation.org, linux-kernel@vger.kernel.org, Lu Baolu , Jean-Philippe Brucker Subject: [PATCH v6 02/12] iommu: Add pasid_bits field in struct dev_iommu Date: Tue, 10 May 2022 14:17:28 +0800 Message-Id: <20220510061738.2761430-3-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220510061738.2761430-1-baolu.lu@linux.intel.com> References: <20220510061738.2761430-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Use this field to save the pasid/ssid bits that a device is able to support with its IOMMU hardware. It is a generic attribute of a device and lifting it into the per-device dev_iommu struct makes it possible to allocate a PASID for device without calls into the IOMMU drivers. Any iommu driver which suports PASID related features should set this field before features are enabled on the devices. For initialization of this field in the VT-d driver, the info->pasid_supported is only set for PCI devices. So the status is that non-PCI SVA hasn't been supported yet. Setting this field only for PCI devices has no functional change. Signed-off-by: Lu Baolu Reviewed-by: Jean-Philippe Brucker --- include/linux/iommu.h | 1 + drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 2 ++ drivers/iommu/intel/iommu.c | 5 ++++- 3 files changed, 7 insertions(+), 1 deletion(-) diff --git a/include/linux/iommu.h b/include/linux/iommu.h index 5e1afe169549..b8ffaf2cb1d0 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -373,6 +373,7 @@ struct dev_iommu { struct iommu_fwspec *fwspec; struct iommu_device *iommu_dev; void *priv; + unsigned int pasid_bits; }; =20 int iommu_device_register(struct iommu_device *iommu, diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/ar= m/arm-smmu-v3/arm-smmu-v3.c index 627a3ed5ee8f..afc63fce6107 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -2681,6 +2681,8 @@ static struct iommu_device *arm_smmu_probe_device(str= uct device *dev) smmu->features & ARM_SMMU_FEAT_STALL_FORCE) master->stall_enabled =3D true; =20 + dev->iommu->pasid_bits =3D master->ssid_bits; + return &smmu->iommu; =20 err_free_master: diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c index 2990f80c5e08..99643f897f26 100644 --- a/drivers/iommu/intel/iommu.c +++ b/drivers/iommu/intel/iommu.c @@ -4624,8 +4624,11 @@ static struct iommu_device *intel_iommu_probe_device= (struct device *dev) if (pasid_supported(iommu)) { int features =3D pci_pasid_features(pdev); =20 - if (features >=3D 0) + if (features >=3D 0) { info->pasid_supported =3D features | 1; + dev->iommu->pasid_bits =3D + fls(pci_max_pasids(pdev)) - 1; + } } =20 if (info->ats_supported && ecap_prs(iommu->ecap) && --=20 2.25.1 From nobody Fri May 8 06:00:20 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 96FF1C433EF for ; Tue, 10 May 2022 06:21:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237082AbiEJGZS (ORCPT ); Tue, 10 May 2022 02:25:18 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33834 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237068AbiEJGY5 (ORCPT ); Tue, 10 May 2022 02:24:57 -0400 Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8AF8F1E3EB for ; Mon, 9 May 2022 23:21:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1652163660; x=1683699660; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=SkuWVMiahK5jbVuor3KDnHkiDiZb/OYULE9ThL5dTD0=; b=S2jPF2XWioNIwanVmFIkC/zW035wusxxVJY05zV86Oge7xuV3ODx88zr ht1sMICv0wH44wWW3dVghc3zeW51LeYGwL6EgGtDaq9erMObR3jNTUik1 mVxMWwYm4MGFChSMssRAXbEnuNHUQDoOb/46gfsEunb4yzZ/oI22b8LPo g73cfIHveztbWFY+zl4CjGrLoaGd8rw1wEZyO1QUCZsiMtDxp0CsH/Kky 1xDvq0m09fs9vcyAnBIWpXIz6W7wI6A84u38zI4RIBorAtqQZhk//72LT RhCA8StSMvMNycbmIlKFLbJf5hzAw6+57g7tgieQsMEiv3iK98LCtZvFu A==; X-IronPort-AV: E=McAfee;i="6400,9594,10342"; a="332312883" X-IronPort-AV: E=Sophos;i="5.91,213,1647327600"; d="scan'208";a="332312883" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 May 2022 23:21:00 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.91,213,1647327600"; d="scan'208";a="552636397" Received: from allen-box.sh.intel.com ([10.239.159.48]) by orsmga002.jf.intel.com with ESMTP; 09 May 2022 23:20:56 -0700 From: Lu Baolu To: Joerg Roedel , Jason Gunthorpe , Christoph Hellwig , Kevin Tian , Ashok Raj , Will Deacon , Robin Murphy , Jean-Philippe Brucker , Dave Jiang , Vinod Koul Cc: Eric Auger , Liu Yi L , Jacob jun Pan , iommu@lists.linux-foundation.org, linux-kernel@vger.kernel.org, Lu Baolu , Jean-Philippe Brucker Subject: [PATCH v6 03/12] iommu: Add attach/detach_dev_pasid domain ops Date: Tue, 10 May 2022 14:17:29 +0800 Message-Id: <20220510061738.2761430-4-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220510061738.2761430-1-baolu.lu@linux.intel.com> References: <20220510061738.2761430-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Attaching an IOMMU domain to a PASID of a device is a generic operation for modern IOMMU drivers which support PASID-granular DMA address translation. Currently visible usage scenarios include (but not limited): - SVA (Shared Virtual Address) - kernel DMA with PASID - hardware-assist mediated device This adds a pair of common domain ops for this purpose and adds helpers to attach/detach a domain to/from a {device, PASID}. Some buses, like PCI, route packets without considering the PASID value. Thus a DMA target address with PASID might be treated as P2P if the address falls into the MMIO BAR of other devices in the group. To make things simple, these interfaces only apply to devices belonging to the singleton groups, and the singleton is immutable in fabric i.e. not affected by hotplug. Signed-off-by: Lu Baolu Reviewed-by: Jean-Philippe Brucker --- include/linux/iommu.h | 21 +++++++++++++ drivers/iommu/iommu.c | 71 +++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 92 insertions(+) diff --git a/include/linux/iommu.h b/include/linux/iommu.h index b8ffaf2cb1d0..ab36244d4e94 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -263,6 +263,8 @@ struct iommu_ops { * struct iommu_domain_ops - domain specific operations * @attach_dev: attach an iommu domain to a device * @detach_dev: detach an iommu domain from a device + * @attach_dev_pasid: attach an iommu domain to a pasid of device + * @detach_dev_pasid: detach an iommu domain from a pasid of device * @map: map a physically contiguous memory region to an iommu domain * @map_pages: map a physically contiguous set of pages of the same size to * an iommu domain. @@ -283,6 +285,10 @@ struct iommu_ops { struct iommu_domain_ops { int (*attach_dev)(struct iommu_domain *domain, struct device *dev); void (*detach_dev)(struct iommu_domain *domain, struct device *dev); + int (*attach_dev_pasid)(struct iommu_domain *domain, + struct device *dev, ioasid_t pasid); + void (*detach_dev_pasid)(struct iommu_domain *domain, + struct device *dev, ioasid_t pasid); =20 int (*map)(struct iommu_domain *domain, unsigned long iova, phys_addr_t paddr, size_t size, int prot, gfp_t gfp); @@ -678,6 +684,10 @@ int iommu_group_claim_dma_owner(struct iommu_group *gr= oup, void *owner); void iommu_group_release_dma_owner(struct iommu_group *group); bool iommu_group_dma_owner_claimed(struct iommu_group *group); =20 +int iommu_attach_device_pasid(struct iommu_domain *domain, + struct device *dev, ioasid_t pasid); +void iommu_detach_device_pasid(struct iommu_domain *domain, + struct device *dev, ioasid_t pasid); #else /* CONFIG_IOMMU_API */ =20 struct iommu_ops {}; @@ -1051,6 +1061,17 @@ static inline bool iommu_group_dma_owner_claimed(str= uct iommu_group *group) { return false; } + +static inline int iommu_attach_device_pasid(struct iommu_domain *domain, + struct device *dev, ioasid_t pasid) +{ + return -ENODEV; +} + +static inline void iommu_detach_device_pasid(struct iommu_domain *domain, + struct device *dev, ioasid_t pasid) +{ +} #endif /* CONFIG_IOMMU_API */ =20 /** diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c index 29906bc16371..16e8db2d86fc 100644 --- a/drivers/iommu/iommu.c +++ b/drivers/iommu/iommu.c @@ -38,6 +38,7 @@ struct iommu_group { struct kobject kobj; struct kobject *devices_kobj; struct list_head devices; + struct xarray pasid_array; struct mutex mutex; void *iommu_data; void (*iommu_data_release)(void *iommu_data); @@ -630,6 +631,7 @@ struct iommu_group *iommu_group_alloc(void) mutex_init(&group->mutex); INIT_LIST_HEAD(&group->devices); INIT_LIST_HEAD(&group->entry); + xa_init(&group->pasid_array); =20 ret =3D ida_simple_get(&iommu_group_ida, 0, 0, GFP_KERNEL); if (ret < 0) { @@ -3190,3 +3192,72 @@ bool iommu_group_dma_owner_claimed(struct iommu_grou= p *group) return user; } EXPORT_SYMBOL_GPL(iommu_group_dma_owner_claimed); + +static bool device_group_immutable_singleton(struct device *dev) +{ + struct iommu_group *group =3D iommu_group_get(dev); + int count; + + if (!group) + return false; + + mutex_lock(&group->mutex); + count =3D iommu_group_device_count(group); + mutex_unlock(&group->mutex); + iommu_group_put(group); + + if (count !=3D 1) + return false; + + /* + * The PCI device could be considered to be fully isolated if all + * devices on the path from the device to the host-PCI bridge are + * protected from peer-to-peer DMA by ACS. + */ + if (dev_is_pci(dev)) + return pci_acs_path_enabled(to_pci_dev(dev), NULL, + REQ_ACS_FLAGS); + + return true; +} + +int iommu_attach_device_pasid(struct iommu_domain *domain, + struct device *dev, ioasid_t pasid) +{ + struct iommu_group *group; + int ret =3D -EINVAL; + void *curr; + + if (!domain->ops->attach_dev_pasid) + return -EOPNOTSUPP; + + if (!device_group_immutable_singleton(dev)) + return -EINVAL; + + group =3D iommu_group_get(dev); + mutex_lock(&group->mutex); + curr =3D xa_cmpxchg(&group->pasid_array, pasid, NULL, domain, GFP_KERNEL); + if (curr) + goto out_unlock; + ret =3D domain->ops->attach_dev_pasid(domain, dev, pasid); + if (ret) + xa_erase(&group->pasid_array, pasid); +out_unlock: + mutex_unlock(&group->mutex); + iommu_group_put(group); + + return ret; +} + +void iommu_detach_device_pasid(struct iommu_domain *domain, + struct device *dev, ioasid_t pasid) +{ + struct iommu_group *group =3D iommu_group_get(dev); + + mutex_lock(&group->mutex); + domain->ops->detach_dev_pasid(domain, dev, pasid); + xa_erase(&group->pasid_array, pasid); + mutex_unlock(&group->mutex); + + iommu_group_put(group); +} --=20 2.25.1 From nobody Fri May 8 06:00:20 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5DAA6C433F5 for ; Tue, 10 May 2022 06:21:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237160AbiEJGZY (ORCPT ); Tue, 10 May 2022 02:25:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34106 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237089AbiEJGZB (ORCPT ); Tue, 10 May 2022 02:25:01 -0400 Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B75CD1F62E for ; Mon, 9 May 2022 23:21:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1652163664; x=1683699664; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=VjhNoKI6MtW1lOMCYZbpbYvPa5VAnC2OCUg7RydhhmI=; b=GzNcFKo69X2N/KIgKsbasrP2rtxT6Ql5GpMAGjt3usMgWGbNdoYGCrDS 0n9QkPHo6JeQzh9+SOlIWRemrKBXeIvndqD+CLMHKf9gqsKTMlyzx8rvY 3giLv0Us9FJcEQXio2J89P5hmLUR8YNpaDCH8uyOOaud8yrETo52nnW8b OqEoiUk+P/xGLK6ti1Z2WkglBmC91nB10wdDhTOzycvlspD07GCqsSwPc 66/8JgFqjEpbCeTT9H2bru3FY1Hm6xKnDD4cBQAKABMOrVKZ0Fx3WJGqo Wu9srJfpUgwSN9IMWp2PDWdDaTes7xjKPq5Gb9KV9fjcnR8J+j2AlrUUl g==; X-IronPort-AV: E=McAfee;i="6400,9594,10342"; a="332312921" X-IronPort-AV: E=Sophos;i="5.91,213,1647327600"; d="scan'208";a="332312921" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 May 2022 23:21:04 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.91,213,1647327600"; d="scan'208";a="552636440" Received: from allen-box.sh.intel.com ([10.239.159.48]) by orsmga002.jf.intel.com with ESMTP; 09 May 2022 23:21:00 -0700 From: Lu Baolu To: Joerg Roedel , Jason Gunthorpe , Christoph Hellwig , Kevin Tian , Ashok Raj , Will Deacon , Robin Murphy , Jean-Philippe Brucker , Dave Jiang , Vinod Koul Cc: Eric Auger , Liu Yi L , Jacob jun Pan , iommu@lists.linux-foundation.org, linux-kernel@vger.kernel.org, Lu Baolu , Jean-Philippe Brucker Subject: [PATCH v6 04/12] iommu/sva: Basic data structures for SVA Date: Tue, 10 May 2022 14:17:30 +0800 Message-Id: <20220510061738.2761430-5-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220510061738.2761430-1-baolu.lu@linux.intel.com> References: <20220510061738.2761430-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Use below data structures for SVA implementation in the IOMMU core: - struct iommu_domain (IOMMU_DOMAIN_SVA type) Represent a hardware pagetable that the IOMMU hardware could use for SVA translation. Multiple iommu domains could be bound with an SVA mm and each grabs a mm_count of the mm in order to make sure mm could only be freed after all domains have been unbound. A new mm field is added to struct iommu_domain and a helper is added to retrieve mm from a domain pointer. - struct iommu_sva (existing) Represent a bond relationship between an SVA ioas and an iommu domain. If a bond already exists, it's reused and a reference is taken. - struct dev_iommu::sva_bonds A pasid-indexed xarray to track the bonds happened on the device. Suggested-by: Jean-Philippe Brucker Signed-off-by: Lu Baolu --- include/linux/iommu.h | 13 +++++++++++++ drivers/iommu/iommu.c | 3 +++ 2 files changed, 16 insertions(+) diff --git a/include/linux/iommu.h b/include/linux/iommu.h index ab36244d4e94..2921e634491e 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -64,6 +64,9 @@ struct iommu_domain_geometry { #define __IOMMU_DOMAIN_PT (1U << 2) /* Domain is identity mapped */ #define __IOMMU_DOMAIN_DMA_FQ (1U << 3) /* DMA-API uses flush queue */ =20 +#define __IOMMU_DOMAIN_SHARED (1U << 4) /* Page table shared from CPU */ +#define __IOMMU_DOMAIN_HOST_VA (1U << 5) /* Host CPU virtual address */ + /* * This are the possible domain-types * @@ -86,6 +89,8 @@ struct iommu_domain_geometry { #define IOMMU_DOMAIN_DMA_FQ (__IOMMU_DOMAIN_PAGING | \ __IOMMU_DOMAIN_DMA_API | \ __IOMMU_DOMAIN_DMA_FQ) +#define IOMMU_DOMAIN_SVA (__IOMMU_DOMAIN_SHARED | \ + __IOMMU_DOMAIN_HOST_VA) =20 struct iommu_domain { unsigned type; @@ -95,6 +100,9 @@ struct iommu_domain { void *handler_token; struct iommu_domain_geometry geometry; struct iommu_dma_cookie *iova_cookie; +#ifdef CONFIG_IOMMU_SVA + struct mm_struct *mm; +#endif }; =20 static inline bool iommu_is_dma_domain(struct iommu_domain *domain) @@ -380,6 +388,9 @@ struct dev_iommu { struct iommu_device *iommu_dev; void *priv; unsigned int pasid_bits; +#ifdef CONFIG_IOMMU_SVA + struct xarray sva_bonds; +#endif }; =20 int iommu_device_register(struct iommu_device *iommu, @@ -629,6 +640,8 @@ struct iommu_fwspec { */ struct iommu_sva { struct device *dev; + struct iommu_domain *domain; + refcount_t users; }; =20 int iommu_fwspec_init(struct device *dev, struct fwnode_handle *iommu_fwno= de, diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c index 16e8db2d86fc..1abff5fc9554 100644 --- a/drivers/iommu/iommu.c +++ b/drivers/iommu/iommu.c @@ -202,6 +202,9 @@ static struct dev_iommu *dev_iommu_get(struct device *d= ev) return NULL; =20 mutex_init(¶m->lock); +#ifdef CONFIG_IOMMU_SVA + xa_init(¶m->sva_bonds); +#endif dev->iommu =3D param; return param; } --=20 2.25.1 From nobody Fri May 8 06:00:20 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0F6DAC433EF for ; Tue, 10 May 2022 06:21:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237208AbiEJGZi (ORCPT ); Tue, 10 May 2022 02:25:38 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34108 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237086AbiEJGZH (ORCPT ); Tue, 10 May 2022 02:25:07 -0400 Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2C161286F9 for ; Mon, 9 May 2022 23:21:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1652163669; x=1683699669; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=9+qpeiFGVw2YCgZsa9Di6+VI7Ti6W93R96q9XJXadnA=; b=lo1eY2FHAicKtKQqgkY0CqgI69+a4Y0RBgpnYuqs1RaPU8dHc1Jke3I/ 0qL91yLCH0sYgbg+8N3dOyGuLms/REcOBFiBDfWJzWWk9VOKe/nM+B2RB Tmc/G/sgPH1BuBSAfXWfS6qG/5YpoKclvxAoL74ll306iihm1dvNEPFee plnIJE2SZ4fL5GSqtuqSOO0eZ6ukskhUQZulhTsumj4O0MTBVNJI59IYs tqarcIm8IM1XxZKoMoQQzwp1GjsJeZuHvNrNzMi64YIufx7f8kQLbsBMw 3zed5DdErsNanbZBxBOERD9qw4HNhZiU0eemXz+Zlj02MuaX+p54sG+vJ Q==; X-IronPort-AV: E=McAfee;i="6400,9594,10342"; a="332312946" X-IronPort-AV: E=Sophos;i="5.91,213,1647327600"; d="scan'208";a="332312946" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 May 2022 23:21:08 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.91,213,1647327600"; d="scan'208";a="552636472" Received: from allen-box.sh.intel.com ([10.239.159.48]) by orsmga002.jf.intel.com with ESMTP; 09 May 2022 23:21:04 -0700 From: Lu Baolu To: Joerg Roedel , Jason Gunthorpe , Christoph Hellwig , Kevin Tian , Ashok Raj , Will Deacon , Robin Murphy , Jean-Philippe Brucker , Dave Jiang , Vinod Koul Cc: Eric Auger , Liu Yi L , Jacob jun Pan , iommu@lists.linux-foundation.org, linux-kernel@vger.kernel.org, Lu Baolu , Jacob Pan Subject: [PATCH v6 05/12] iommu/vt-d: Remove SVM_FLAG_SUPERVISOR_MODE support Date: Tue, 10 May 2022 14:17:31 +0800 Message-Id: <20220510061738.2761430-6-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220510061738.2761430-1-baolu.lu@linux.intel.com> References: <20220510061738.2761430-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The current kernel DMA with PASID support is based on the SVA with a flag SVM_FLAG_SUPERVISOR_MODE. The IOMMU driver binds the kernel memory address space to a PASID of the device. The device driver programs the device with kernel virtual address (KVA) for DMA access. There have been security and functional issues with this approach: - The lack of IOTLB synchronization upon kernel page table updates. (vmalloc, module/BPF loading, CONFIG_DEBUG_PAGEALLOC etc.) - Other than slight more protection, using kernel virtual address (KVA) has little advantage over physical address. There are also no use cases yet where DMA engines need kernel virtual addresses for in-kernel DMA. This removes SVM_FLAG_SUPERVISOR_MODE support in the Intel IOMMU driver. The device driver is suggested to handle kernel DMA with PASID through the kernel DMA APIs. Link: https://lore.kernel.org/linux-iommu/20210511194726.GP1002214@nvidia.c= om/ Signed-off-by: Jacob Pan Signed-off-by: Lu Baolu Reviewed-by: Jason Gunthorpe --- drivers/iommu/intel/svm.c | 53 +++++++++------------------------------ 1 file changed, 12 insertions(+), 41 deletions(-) diff --git a/drivers/iommu/intel/svm.c b/drivers/iommu/intel/svm.c index 7ee37d996e15..574ddddaa33a 100644 --- a/drivers/iommu/intel/svm.c +++ b/drivers/iommu/intel/svm.c @@ -313,8 +313,7 @@ static int pasid_to_svm_sdev(struct device *dev, unsign= ed int pasid, return 0; } =20 -static int intel_svm_alloc_pasid(struct device *dev, struct mm_struct *mm, - unsigned int flags) +static int intel_svm_alloc_pasid(struct device *dev, struct mm_struct *mm) { ioasid_t max_pasid =3D dev_is_pci(dev) ? pci_max_pasids(to_pci_dev(dev)) : intel_pasid_max_id; @@ -324,8 +323,7 @@ static int intel_svm_alloc_pasid(struct device *dev, st= ruct mm_struct *mm, =20 static struct iommu_sva *intel_svm_bind_mm(struct intel_iommu *iommu, struct device *dev, - struct mm_struct *mm, - unsigned int flags) + struct mm_struct *mm) { struct device_domain_info *info =3D dev_iommu_priv_get(dev); unsigned long iflags, sflags; @@ -341,22 +339,18 @@ static struct iommu_sva *intel_svm_bind_mm(struct int= el_iommu *iommu, =20 svm->pasid =3D mm->pasid; svm->mm =3D mm; - svm->flags =3D flags; INIT_LIST_HEAD_RCU(&svm->devs); =20 - if (!(flags & SVM_FLAG_SUPERVISOR_MODE)) { - svm->notifier.ops =3D &intel_mmuops; - ret =3D mmu_notifier_register(&svm->notifier, mm); - if (ret) { - kfree(svm); - return ERR_PTR(ret); - } + svm->notifier.ops =3D &intel_mmuops; + ret =3D mmu_notifier_register(&svm->notifier, mm); + if (ret) { + kfree(svm); + return ERR_PTR(ret); } =20 ret =3D pasid_private_add(svm->pasid, svm); if (ret) { - if (svm->notifier.ops) - mmu_notifier_unregister(&svm->notifier, mm); + mmu_notifier_unregister(&svm->notifier, mm); kfree(svm); return ERR_PTR(ret); } @@ -391,9 +385,7 @@ static struct iommu_sva *intel_svm_bind_mm(struct intel= _iommu *iommu, } =20 /* Setup the pasid table: */ - sflags =3D (flags & SVM_FLAG_SUPERVISOR_MODE) ? - PASID_FLAG_SUPERVISOR_MODE : 0; - sflags |=3D cpu_feature_enabled(X86_FEATURE_LA57) ? PASID_FLAG_FL5LP : 0; + sflags =3D cpu_feature_enabled(X86_FEATURE_LA57) ? PASID_FLAG_FL5LP : 0; spin_lock_irqsave(&iommu->lock, iflags); ret =3D intel_pasid_setup_first_level(iommu, dev, mm->pgd, mm->pasid, FLPT_DEFAULT_DID, sflags); @@ -410,8 +402,7 @@ static struct iommu_sva *intel_svm_bind_mm(struct intel= _iommu *iommu, kfree(sdev); free_svm: if (list_empty(&svm->devs)) { - if (svm->notifier.ops) - mmu_notifier_unregister(&svm->notifier, mm); + mmu_notifier_unregister(&svm->notifier, mm); pasid_private_remove(mm->pasid); kfree(svm); } @@ -821,37 +812,17 @@ static irqreturn_t prq_event_thread(int irq, void *d) struct iommu_sva *intel_svm_bind(struct device *dev, struct mm_struct *mm,= void *drvdata) { struct intel_iommu *iommu =3D device_to_iommu(dev, NULL, NULL); - unsigned int flags =3D 0; struct iommu_sva *sva; int ret; =20 - if (drvdata) - flags =3D *(unsigned int *)drvdata; - - if (flags & SVM_FLAG_SUPERVISOR_MODE) { - if (!ecap_srs(iommu->ecap)) { - dev_err(dev, "%s: Supervisor PASID not supported\n", - iommu->name); - return ERR_PTR(-EOPNOTSUPP); - } - - if (mm) { - dev_err(dev, "%s: Supervisor PASID with user provided mm\n", - iommu->name); - return ERR_PTR(-EINVAL); - } - - mm =3D &init_mm; - } - mutex_lock(&pasid_mutex); - ret =3D intel_svm_alloc_pasid(dev, mm, flags); + ret =3D intel_svm_alloc_pasid(dev, mm); if (ret) { mutex_unlock(&pasid_mutex); return ERR_PTR(ret); } =20 - sva =3D intel_svm_bind_mm(iommu, dev, mm, flags); + sva =3D intel_svm_bind_mm(iommu, dev, mm); mutex_unlock(&pasid_mutex); =20 return sva; --=20 2.25.1 From nobody Fri May 8 06:00:20 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C134BC433F5 for ; Tue, 10 May 2022 06:21:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237177AbiEJGZe (ORCPT ); Tue, 10 May 2022 02:25:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34298 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237106AbiEJGZL (ORCPT ); Tue, 10 May 2022 02:25:11 -0400 Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6B61631900 for ; Mon, 9 May 2022 23:21:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1652163672; x=1683699672; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=2oif/KlXSEUCU88GJ6ATwpartNvHpX2RX3MjlkgL9ng=; b=D40f6HCY4Z/DA7L+DZQcREjTlcsXvO5bJLcyqeL/O+ipb6dU+d5RBDMl enO3IpliwW+WJKZ4WjeAtR0dCPjPS+OmULpN75sDcBHQrOGXW+bZN6tKt MBpRIabtUryqRozcFA4UPa69TF2blOskPrNmHthmcz1bKnzEP+huVqJVO yYnTVhiC4DnTdXJmrCaFn63IBXl8CO54iWj8FwxKVguZoQQZB+2S2QD0Z rBrpfJCOrM5KyIySyaEo6dnjtjxSGo4mCXdbXdHCo+eZx+r/b1Ym+LJE0 xaS0PInA/mln9wbw18HscRtgV2fsbFpJTvmy8nzrRP5gV8FPW+hRIcCsX Q==; X-IronPort-AV: E=McAfee;i="6400,9594,10342"; a="332312960" X-IronPort-AV: E=Sophos;i="5.91,213,1647327600"; d="scan'208";a="332312960" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 May 2022 23:21:12 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.91,213,1647327600"; d="scan'208";a="552636499" Received: from allen-box.sh.intel.com ([10.239.159.48]) by orsmga002.jf.intel.com with ESMTP; 09 May 2022 23:21:08 -0700 From: Lu Baolu To: Joerg Roedel , Jason Gunthorpe , Christoph Hellwig , Kevin Tian , Ashok Raj , Will Deacon , Robin Murphy , Jean-Philippe Brucker , Dave Jiang , Vinod Koul Cc: Eric Auger , Liu Yi L , Jacob jun Pan , iommu@lists.linux-foundation.org, linux-kernel@vger.kernel.org, Lu Baolu Subject: [PATCH v6 06/12] iommu/vt-d: Add SVA domain support Date: Tue, 10 May 2022 14:17:32 +0800 Message-Id: <20220510061738.2761430-7-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220510061738.2761430-1-baolu.lu@linux.intel.com> References: <20220510061738.2761430-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Add support for SVA domain allocation and provide an SVA-specific iommu_domain_ops. Signed-off-by: Lu Baolu --- include/linux/intel-iommu.h | 5 ++++ drivers/iommu/intel/iommu.c | 2 ++ drivers/iommu/intel/svm.c | 48 +++++++++++++++++++++++++++++++++++++ 3 files changed, 55 insertions(+) diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h index 72e5d7900e71..2397c2007cda 100644 --- a/include/linux/intel-iommu.h +++ b/include/linux/intel-iommu.h @@ -744,6 +744,7 @@ void intel_svm_unbind(struct iommu_sva *handle); u32 intel_svm_get_pasid(struct iommu_sva *handle); int intel_svm_page_response(struct device *dev, struct iommu_fault_event *= evt, struct iommu_page_response *msg); +struct iommu_domain *intel_svm_domain_alloc(void); =20 struct intel_svm_dev { struct list_head list; @@ -769,6 +770,10 @@ struct intel_svm { }; #else static inline void intel_svm_check(struct intel_iommu *iommu) {} +static inline struct iommu_domain *intel_svm_domain_alloc(void) +{ + return NULL; +} #endif =20 #ifdef CONFIG_INTEL_IOMMU_DEBUGFS diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c index 99643f897f26..10b1e9dcbd98 100644 --- a/drivers/iommu/intel/iommu.c +++ b/drivers/iommu/intel/iommu.c @@ -4343,6 +4343,8 @@ static struct iommu_domain *intel_iommu_domain_alloc(= unsigned type) return domain; case IOMMU_DOMAIN_IDENTITY: return &si_domain->domain; + case IOMMU_DOMAIN_SVA: + return intel_svm_domain_alloc(); default: return NULL; } diff --git a/drivers/iommu/intel/svm.c b/drivers/iommu/intel/svm.c index 574ddddaa33a..641ab0491ada 100644 --- a/drivers/iommu/intel/svm.c +++ b/drivers/iommu/intel/svm.c @@ -931,3 +931,51 @@ int intel_svm_page_response(struct device *dev, mutex_unlock(&pasid_mutex); return ret; } + +static int intel_svm_attach_dev_pasid(struct iommu_domain *domain, + struct device *dev, ioasid_t pasid) +{ + struct device_domain_info *info =3D dev_iommu_priv_get(dev); + struct intel_iommu *iommu =3D info->iommu; + struct mm_struct *mm =3D domain->mm; + struct iommu_sva *sva; + int ret =3D 0; + + mutex_lock(&pasid_mutex); + sva =3D intel_svm_bind_mm(iommu, dev, mm); + if (IS_ERR(sva)) + ret =3D PTR_ERR(sva); + mutex_unlock(&pasid_mutex); + + return ret; +} + +static void intel_svm_detach_dev_pasid(struct iommu_domain *domain, + struct device *dev, ioasid_t pasid) +{ + mutex_lock(&pasid_mutex); + intel_svm_unbind_mm(dev, pasid); + mutex_unlock(&pasid_mutex); +} + +static void intel_svm_domain_free(struct iommu_domain *domain) +{ + kfree(domain); +} + +static const struct iommu_domain_ops intel_svm_domain_ops =3D { + .attach_dev_pasid =3D intel_svm_attach_dev_pasid, + .detach_dev_pasid =3D intel_svm_detach_dev_pasid, + .free =3D intel_svm_domain_free, +}; + +struct iommu_domain *intel_svm_domain_alloc(void) +{ + struct iommu_domain *domain; + + domain =3D kzalloc(sizeof(*domain), GFP_KERNEL); + if (domain) + domain->ops =3D &intel_svm_domain_ops; + + return domain; +} --=20 2.25.1 From nobody Fri May 8 06:00:20 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 06319C433EF for ; Tue, 10 May 2022 06:24:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237258AbiEJG0A (ORCPT ); Tue, 10 May 2022 02:26:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35770 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232406AbiEJGZX (ORCPT ); Tue, 10 May 2022 02:25:23 -0400 Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2E31846673 for ; Mon, 9 May 2022 23:21:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1652163679; x=1683699679; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=60KTAdzso4Vrtd8ZqxID77V2u8yEj+ii15BiJceQGSM=; b=nc6xityHnxFF+gg+KYhDH0VMMl3db4vo2rDWfwqkJ75BhXt3L+aV3Kzc TKtyz3l4NWpRXnzUBr0IxE0mtMys5L2V1U0uPxPUzi0OfHTr+JjmEknOr x/O1z/67B+e8kspKx2go4QO4w/s+hgfNRT31ytvzuofpxjBFEhdcyrnRM 4Wf0gpSXI94cEMm1RKuaNDcX3bgVcdUfxCoWwbxPNgmvgwJ7ikap0XMvD jKL1Sjxx7KqetHEgH3logRuo5eTPCLADGw1DZDuA00Q0eosdg/miVoXWH YkjaL7uU3ABKXhU8DVjvVVRtO8PAtDgAqcR0Dsja82u3+aJWRKBEELxc5 A==; X-IronPort-AV: E=McAfee;i="6400,9594,10342"; a="332312970" X-IronPort-AV: E=Sophos;i="5.91,213,1647327600"; d="scan'208";a="332312970" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 May 2022 23:21:18 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.91,213,1647327600"; d="scan'208";a="552636530" Received: from allen-box.sh.intel.com ([10.239.159.48]) by orsmga002.jf.intel.com with ESMTP; 09 May 2022 23:21:12 -0700 From: Lu Baolu To: Joerg Roedel , Jason Gunthorpe , Christoph Hellwig , Kevin Tian , Ashok Raj , Will Deacon , Robin Murphy , Jean-Philippe Brucker , Dave Jiang , Vinod Koul Cc: Eric Auger , Liu Yi L , Jacob jun Pan , iommu@lists.linux-foundation.org, linux-kernel@vger.kernel.org, Lu Baolu Subject: [PATCH v6 07/12] arm-smmu-v3/sva: Add SVA domain support Date: Tue, 10 May 2022 14:17:33 +0800 Message-Id: <20220510061738.2761430-8-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220510061738.2761430-1-baolu.lu@linux.intel.com> References: <20220510061738.2761430-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Add support for SVA domain allocation and provide an SVA-specific iommu_domain_ops. Signed-off-by: Lu Baolu --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 6 ++ .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 68 +++++++++++++++++++ drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 3 + 3 files changed, 77 insertions(+) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/ar= m/arm-smmu-v3/arm-smmu-v3.h index cd48590ada30..e077f21e2528 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -759,6 +759,7 @@ struct iommu_sva *arm_smmu_sva_bind(struct device *dev,= struct mm_struct *mm, void arm_smmu_sva_unbind(struct iommu_sva *handle); u32 arm_smmu_sva_get_pasid(struct iommu_sva *handle); void arm_smmu_sva_notifier_synchronize(void); +struct iommu_domain *arm_smmu_sva_domain_alloc(void); #else /* CONFIG_ARM_SMMU_V3_SVA */ static inline bool arm_smmu_sva_supported(struct arm_smmu_device *smmu) { @@ -804,5 +805,10 @@ static inline u32 arm_smmu_sva_get_pasid(struct iommu_= sva *handle) } =20 static inline void arm_smmu_sva_notifier_synchronize(void) {} + +static inline struct iommu_domain *arm_smmu_sva_domain_alloc(void) +{ + return NULL; +} #endif /* CONFIG_ARM_SMMU_V3_SVA */ #endif /* _ARM_SMMU_V3_H */ diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iomm= u/arm/arm-smmu-v3/arm-smmu-v3-sva.c index c623dae1e115..9d176e836d6b 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -541,3 +541,71 @@ void arm_smmu_sva_notifier_synchronize(void) */ mmu_notifier_synchronize(); } + +static int arm_smmu_sva_attach_dev_pasid(struct iommu_domain *domain, + struct device *dev, ioasid_t id) +{ + int ret =3D 0; + struct mm_struct *mm; + struct iommu_sva *handle; + + if (domain->type !=3D IOMMU_DOMAIN_SVA) + return -EINVAL; + + mm =3D domain->mm; + if (WARN_ON(!mm)) + return -ENODEV; + + mutex_lock(&sva_lock); + handle =3D __arm_smmu_sva_bind(dev, mm); + if (IS_ERR(handle)) + ret =3D PTR_ERR(handle); + mutex_unlock(&sva_lock); + + return ret; +} + +static void arm_smmu_sva_detach_dev_pasid(struct iommu_domain *domain, + struct device *dev, ioasid_t id) +{ + struct mm_struct *mm =3D domain->mm; + struct arm_smmu_bond *bond =3D NULL, *t; + struct arm_smmu_master *master =3D dev_iommu_priv_get(dev); + + mutex_lock(&sva_lock); + list_for_each_entry(t, &master->bonds, list) { + if (t->mm =3D=3D mm) { + bond =3D t; + break; + } + } + + if (!WARN_ON(!bond) && refcount_dec_and_test(&bond->refs)) { + list_del(&bond->list); + arm_smmu_mmu_notifier_put(bond->smmu_mn); + kfree(bond); + } + mutex_unlock(&sva_lock); +} + +static void arm_smmu_sva_domain_free(struct iommu_domain *domain) +{ + kfree(domain); +} + +static const struct iommu_domain_ops arm_smmu_sva_domain_ops =3D { + .attach_dev_pasid =3D arm_smmu_sva_attach_dev_pasid, + .detach_dev_pasid =3D arm_smmu_sva_detach_dev_pasid, + .free =3D arm_smmu_sva_domain_free, +}; + +struct iommu_domain *arm_smmu_sva_domain_alloc(void) +{ + struct iommu_domain *domain; + + domain =3D kzalloc(sizeof(*domain), GFP_KERNEL); + if (domain) + domain->ops =3D &arm_smmu_sva_domain_ops; + + return domain; +} diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/ar= m/arm-smmu-v3/arm-smmu-v3.c index afc63fce6107..9daf3de7e539 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -1999,6 +1999,9 @@ static struct iommu_domain *arm_smmu_domain_alloc(uns= igned type) { struct arm_smmu_domain *smmu_domain; =20 + if (type =3D=3D IOMMU_DOMAIN_SVA) + return arm_smmu_sva_domain_alloc(); + if (type !=3D IOMMU_DOMAIN_UNMANAGED && type !=3D IOMMU_DOMAIN_DMA && type !=3D IOMMU_DOMAIN_DMA_FQ && --=20 2.25.1 From nobody Fri May 8 06:00:20 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B308AC433EF for ; Tue, 10 May 2022 06:22:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237127AbiEJG0X (ORCPT ); Tue, 10 May 2022 02:26:23 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35852 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237157AbiEJGZX (ORCPT ); Tue, 10 May 2022 02:25:23 -0400 Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D126449FB9 for ; Mon, 9 May 2022 23:21:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1652163680; x=1683699680; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=gVxfnt6lVpxehCu36GAJabM1VnD1JEkVYjx2ejVXThg=; b=TPKg0eo+PXqFFZcAwijedLPxzJEuhXeoP9dFMOmN4LsZtMLmbDwlHFXk +I6P1q1YNPpwhLLQds16l5l6N1K3LaMx4Uo635CIPA7nHkVEiYH5CmfrY tdWhjeR1M31X0dNIN0MYHQxrOnlgxEG85jDIJ12z/Ivw4KhYgz+UzhEtp ZPFibbnG2xUdP1gFkL5OhDnQHUmDzDNIvuXPckNPYrgeTcjJU7XP9ocVA aQo1uuMK9NRYUZ7sAjwBqPtAIuKJD8HKZVCFX3q4tJLtkRMQ3rvRChvZi 786O3f6TfFpPKLyUGo2kHIfVskFSPhG9YI7EDR0sEXpmzsXO62Z+2JrQs A==; X-IronPort-AV: E=McAfee;i="6400,9594,10342"; a="332312983" X-IronPort-AV: E=Sophos;i="5.91,213,1647327600"; d="scan'208";a="332312983" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 May 2022 23:21:20 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.91,213,1647327600"; d="scan'208";a="552636566" Received: from allen-box.sh.intel.com ([10.239.159.48]) by orsmga002.jf.intel.com with ESMTP; 09 May 2022 23:21:16 -0700 From: Lu Baolu To: Joerg Roedel , Jason Gunthorpe , Christoph Hellwig , Kevin Tian , Ashok Raj , Will Deacon , Robin Murphy , Jean-Philippe Brucker , Dave Jiang , Vinod Koul Cc: Eric Auger , Liu Yi L , Jacob jun Pan , iommu@lists.linux-foundation.org, linux-kernel@vger.kernel.org, Lu Baolu Subject: [PATCH v6 08/12] iommu/sva: Use attach/detach_pasid_dev in SVA interfaces Date: Tue, 10 May 2022 14:17:34 +0800 Message-Id: <20220510061738.2761430-9-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220510061738.2761430-1-baolu.lu@linux.intel.com> References: <20220510061738.2761430-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The existing iommu SVA interfaces are implemented by calling the SVA specific iommu ops provided by the IOMMU drivers. There's no need for any SVA specific ops in iommu_ops vector anymore as we can achieve this through the generic attach/detach_dev_pasid domain ops. This refactors the IOMMU SVA interfaces implementation by using the attach/detach_pasid_dev ops and align them with the concept of the iommu domain. Put the new SVA code in the sva related file in order to make it self-contained. Signed-off-by: Lu Baolu --- include/linux/iommu.h | 44 ++++++----- drivers/iommu/iommu-sva-lib.c | 145 ++++++++++++++++++++++++++++++++++ drivers/iommu/iommu.c | 92 --------------------- 3 files changed, 168 insertions(+), 113 deletions(-) diff --git a/include/linux/iommu.h b/include/linux/iommu.h index 2921e634491e..5a3ef4d58b1f 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -684,12 +684,6 @@ int iommu_dev_enable_feature(struct device *dev, enum = iommu_dev_features f); int iommu_dev_disable_feature(struct device *dev, enum iommu_dev_features = f); bool iommu_dev_feature_enabled(struct device *dev, enum iommu_dev_features= f); =20 -struct iommu_sva *iommu_sva_bind_device(struct device *dev, - struct mm_struct *mm, - void *drvdata); -void iommu_sva_unbind_device(struct iommu_sva *handle); -u32 iommu_sva_get_pasid(struct iommu_sva *handle); - int iommu_device_use_default_domain(struct device *dev); void iommu_device_unuse_default_domain(struct device *dev); =20 @@ -1031,21 +1025,6 @@ iommu_dev_disable_feature(struct device *dev, enum i= ommu_dev_features feat) return -ENODEV; } =20 -static inline struct iommu_sva * -iommu_sva_bind_device(struct device *dev, struct mm_struct *mm, void *drvd= ata) -{ - return NULL; -} - -static inline void iommu_sva_unbind_device(struct iommu_sva *handle) -{ -} - -static inline u32 iommu_sva_get_pasid(struct iommu_sva *handle) -{ - return IOMMU_PASID_INVALID; -} - static inline struct iommu_fwspec *dev_iommu_fwspec_get(struct device *dev) { return NULL; @@ -1087,6 +1066,29 @@ static inline void iommu_detach_device_pasid(struct = iommu_domain *domain, } #endif /* CONFIG_IOMMU_API */ =20 +#ifdef CONFIG_IOMMU_SVA +struct iommu_sva *iommu_sva_bind_device(struct device *dev, + struct mm_struct *mm, + void *drvdata); +void iommu_sva_unbind_device(struct iommu_sva *handle); +u32 iommu_sva_get_pasid(struct iommu_sva *handle); +#else /* CONFIG_IOMMU_SVA */ +static inline struct iommu_sva * +iommu_sva_bind_device(struct device *dev, struct mm_struct *mm, void *drvd= ata) +{ + return NULL; +} + +static inline void iommu_sva_unbind_device(struct iommu_sva *handle) +{ +} + +static inline u32 iommu_sva_get_pasid(struct iommu_sva *handle) +{ + return IOMMU_PASID_INVALID; +} +#endif /* CONFIG_IOMMU_SVA */ + /** * iommu_map_sgtable - Map the given buffer to the IOMMU domain * @domain: The IOMMU domain to perform the mapping diff --git a/drivers/iommu/iommu-sva-lib.c b/drivers/iommu/iommu-sva-lib.c index 106506143896..e7301514f286 100644 --- a/drivers/iommu/iommu-sva-lib.c +++ b/drivers/iommu/iommu-sva-lib.c @@ -3,6 +3,8 @@ * Helpers for IOMMU drivers implementing SVA */ #include +#include +#include #include =20 #include "iommu-sva-lib.h" @@ -69,3 +71,146 @@ struct mm_struct *iommu_sva_find(ioasid_t pasid) return ioasid_find(&iommu_sva_pasid, pasid, __mmget_not_zero); } EXPORT_SYMBOL_GPL(iommu_sva_find); + +/* + * IOMMU SVA driver-oriented interfaces + */ +static struct iommu_domain * +iommu_sva_alloc_domain(struct device *dev, struct mm_struct *mm) +{ + struct bus_type *bus =3D dev->bus; + struct iommu_domain *domain; + + if (!bus || !bus->iommu_ops) + return NULL; + + domain =3D bus->iommu_ops->domain_alloc(IOMMU_DOMAIN_SVA); + if (!domain) + return NULL; + + mmgrab(mm); + domain->mm =3D mm; + domain->type =3D IOMMU_DOMAIN_SVA; + + return domain; +} + +static void iommu_sva_free_domain(struct iommu_domain *domain) +{ + mmdrop(domain->mm); + iommu_domain_free(domain); +} + +/** + * iommu_sva_bind_device() - Bind a process address space to a device + * @dev: the device + * @mm: the mm to bind, caller must hold a reference to mm_users + * @drvdata: opaque data pointer to pass to bind callback + * + * Create a bond between device and address space, allowing the device to = access + * the mm using the returned PASID. If a bond already exists between @devi= ce and + * @mm, it is returned and an additional reference is taken. Caller must c= all + * iommu_sva_unbind_device() to release each reference. + * + * iommu_dev_enable_feature(dev, IOMMU_DEV_FEAT_SVA) must be called first,= to + * initialize the required SVA features. + * + * On error, returns an ERR_PTR value. + */ +struct iommu_sva * +iommu_sva_bind_device(struct device *dev, struct mm_struct *mm, void *drvd= ata) +{ + int ret =3D -EINVAL; + struct iommu_sva *handle; + struct iommu_domain *domain; + + /* + * TODO: Remove the drvdata parameter after kernel PASID support is + * enabled for the idxd driver. + */ + if (drvdata) + return ERR_PTR(-EOPNOTSUPP); + + /* Allocate mm->pasid if necessary. */ + ret =3D iommu_sva_alloc_pasid(mm, 1, (1U << dev->iommu->pasid_bits) - 1); + if (ret) + return ERR_PTR(ret); + + mutex_lock(&iommu_sva_lock); + /* Search for an existing bond. */ + handle =3D xa_load(&dev->iommu->sva_bonds, mm->pasid); + if (handle) { + refcount_inc(&handle->users); + goto out_success; + } + + handle =3D kzalloc(sizeof(*handle), GFP_KERNEL); + if (!handle) { + ret =3D -ENOMEM; + goto out_unlock; + } + + domain =3D iommu_sva_alloc_domain(dev, mm); + if (!domain) { + ret =3D -ENODEV; + goto out_free_bond; + } + + ret =3D iommu_attach_device_pasid(domain, dev, mm->pasid); + if (ret) + goto out_free_domain; + + handle->dev =3D dev; + handle->domain =3D domain; + refcount_set(&handle->users, 1); + ret =3D xa_err(xa_store(&dev->iommu->sva_bonds, mm->pasid, + handle, GFP_KERNEL)); + if (ret) + goto out_detach_domain; + +out_success: + mutex_unlock(&iommu_sva_lock); + return handle; + +out_detach_domain: + iommu_detach_device_pasid(domain, dev, mm->pasid); +out_free_domain: + iommu_sva_free_domain(domain); +out_free_bond: + kfree(handle); +out_unlock: + mutex_unlock(&iommu_sva_lock); + return ERR_PTR(ret); +} +EXPORT_SYMBOL_GPL(iommu_sva_bind_device); + +/** + * iommu_sva_unbind_device() - Remove a bond created with iommu_sva_bind_d= evice + * @handle: the handle returned by iommu_sva_bind_device() + * + * Put reference to a bond between device and address space. The device sh= ould + * not be issuing any more transaction for this PASID. All outstanding page + * requests for this PASID must have been flushed to the IOMMU. + */ +void iommu_sva_unbind_device(struct iommu_sva *handle) +{ + struct device *dev =3D handle->dev; + struct iommu_domain *domain =3D handle->domain; + ioasid_t pasid =3D iommu_sva_get_pasid(handle); + + mutex_lock(&iommu_sva_lock); + if (refcount_dec_and_test(&handle->users)) { + xa_erase(&dev->iommu->sva_bonds, pasid); + iommu_detach_device_pasid(domain, dev, pasid); + iommu_sva_free_domain(domain); + kfree(handle); + } + mutex_unlock(&iommu_sva_lock); +} +EXPORT_SYMBOL_GPL(iommu_sva_unbind_device); + +u32 iommu_sva_get_pasid(struct iommu_sva *handle) +{ + return handle->domain->mm->pasid; +} +EXPORT_SYMBOL_GPL(iommu_sva_get_pasid); diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c index 1abff5fc9554..367d0ecf6e12 100644 --- a/drivers/iommu/iommu.c +++ b/drivers/iommu/iommu.c @@ -2734,98 +2734,6 @@ bool iommu_dev_feature_enabled(struct device *dev, e= num iommu_dev_features feat) } EXPORT_SYMBOL_GPL(iommu_dev_feature_enabled); =20 -/** - * iommu_sva_bind_device() - Bind a process address space to a device - * @dev: the device - * @mm: the mm to bind, caller must hold a reference to it - * @drvdata: opaque data pointer to pass to bind callback - * - * Create a bond between device and address space, allowing the device to = access - * the mm using the returned PASID. If a bond already exists between @devi= ce and - * @mm, it is returned and an additional reference is taken. Caller must c= all - * iommu_sva_unbind_device() to release each reference. - * - * iommu_dev_enable_feature(dev, IOMMU_DEV_FEAT_SVA) must be called first,= to - * initialize the required SVA features. - * - * On error, returns an ERR_PTR value. - */ -struct iommu_sva * -iommu_sva_bind_device(struct device *dev, struct mm_struct *mm, void *drvd= ata) -{ - struct iommu_group *group; - struct iommu_sva *handle =3D ERR_PTR(-EINVAL); - const struct iommu_ops *ops =3D dev_iommu_ops(dev); - - if (!ops->sva_bind) - return ERR_PTR(-ENODEV); - - group =3D iommu_group_get(dev); - if (!group) - return ERR_PTR(-ENODEV); - - /* Ensure device count and domain don't change while we're binding */ - mutex_lock(&group->mutex); - - /* - * To keep things simple, SVA currently doesn't support IOMMU groups - * with more than one device. Existing SVA-capable systems are not - * affected by the problems that required IOMMU groups (lack of ACS - * isolation, device ID aliasing and other hardware issues). - */ - if (iommu_group_device_count(group) !=3D 1) - goto out_unlock; - - handle =3D ops->sva_bind(dev, mm, drvdata); - -out_unlock: - mutex_unlock(&group->mutex); - iommu_group_put(group); - - return handle; -} -EXPORT_SYMBOL_GPL(iommu_sva_bind_device); - -/** - * iommu_sva_unbind_device() - Remove a bond created with iommu_sva_bind_d= evice - * @handle: the handle returned by iommu_sva_bind_device() - * - * Put reference to a bond between device and address space. The device sh= ould - * not be issuing any more transaction for this PASID. All outstanding page - * requests for this PASID must have been flushed to the IOMMU. - */ -void iommu_sva_unbind_device(struct iommu_sva *handle) -{ - struct iommu_group *group; - struct device *dev =3D handle->dev; - const struct iommu_ops *ops =3D dev_iommu_ops(dev); - - if (!ops->sva_unbind) - return; - - group =3D iommu_group_get(dev); - if (!group) - return; - - mutex_lock(&group->mutex); - ops->sva_unbind(handle); - mutex_unlock(&group->mutex); - - iommu_group_put(group); -} -EXPORT_SYMBOL_GPL(iommu_sva_unbind_device); - -u32 iommu_sva_get_pasid(struct iommu_sva *handle) -{ - const struct iommu_ops *ops =3D dev_iommu_ops(handle->dev); - - if (!ops->sva_get_pasid) - return IOMMU_PASID_INVALID; - - return ops->sva_get_pasid(handle); -} -EXPORT_SYMBOL_GPL(iommu_sva_get_pasid); - /* * Changes the default domain of an iommu group that has *only* one device * --=20 2.25.1 From nobody Fri May 8 06:00:20 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DC561C433F5 for ; Tue, 10 May 2022 06:22:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237212AbiEJG0b (ORCPT ); Tue, 10 May 2022 02:26:31 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36890 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237187AbiEJGZh (ORCPT ); Tue, 10 May 2022 02:25:37 -0400 Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9968A4B878 for ; Mon, 9 May 2022 23:21:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1652163685; x=1683699685; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=NaNtWrfv3W647Mx2sWxzXqxtVfiPJBus1GNJDmeNZgg=; b=GDYsgsJk9kF6vgwAmxMfOPu8EsXf/odpbdlrELMzhLFpe41T/Aj8ox1z mYD5f5MLCYWNip05SHxlIrx/MqF1x02LiXpEDbPehkyH0AWhwzJbn7Nmf NGDgQP13P90+6deiOHiOq0VUQqAwGtdW629cM6mtAKMVTA8Ezdy55+OOx mSlT0Wp17jByRioQcFG63j+wUBChu/ljXnxGesFBaGqeUlmj8iWKlWubs lik97CFpEXD7oce4f/NpHqSnVo1RIprPYIAezs4kT7naiiFP4R8jx5Zq7 zCNj6cuenBJ5c/LRLkI3z/5ZvnkY36vsuLVgs6P2mAFphWaOcvk66QTm4 w==; X-IronPort-AV: E=McAfee;i="6400,9594,10342"; a="332313064" X-IronPort-AV: E=Sophos;i="5.91,213,1647327600"; d="scan'208";a="332313064" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 May 2022 23:21:25 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.91,213,1647327600"; d="scan'208";a="552636606" Received: from allen-box.sh.intel.com ([10.239.159.48]) by orsmga002.jf.intel.com with ESMTP; 09 May 2022 23:21:20 -0700 From: Lu Baolu To: Joerg Roedel , Jason Gunthorpe , Christoph Hellwig , Kevin Tian , Ashok Raj , Will Deacon , Robin Murphy , Jean-Philippe Brucker , Dave Jiang , Vinod Koul Cc: Eric Auger , Liu Yi L , Jacob jun Pan , iommu@lists.linux-foundation.org, linux-kernel@vger.kernel.org, Lu Baolu , Jean-Philippe Brucker Subject: [PATCH v6 09/12] iommu: Remove SVA related callbacks from iommu ops Date: Tue, 10 May 2022 14:17:35 +0800 Message-Id: <20220510061738.2761430-10-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220510061738.2761430-1-baolu.lu@linux.intel.com> References: <20220510061738.2761430-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" These ops'es have been replaced with the dev_attach/detach_pasid domain ops'es. There's no need for them anymore. Remove them to avoid dead code. Signed-off-by: Lu Baolu Reviewed-by: Jean-Philippe Brucker --- include/linux/intel-iommu.h | 4 -- include/linux/iommu.h | 8 --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 17 ------- drivers/iommu/iommu-sva-lib.h | 1 - .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 41 ---------------- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 3 -- drivers/iommu/intel/iommu.c | 3 -- drivers/iommu/intel/svm.c | 49 ------------------- drivers/iommu/iommu-sva-lib.c | 4 +- 9 files changed, 2 insertions(+), 128 deletions(-) diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h index 2397c2007cda..4a5cc796f917 100644 --- a/include/linux/intel-iommu.h +++ b/include/linux/intel-iommu.h @@ -738,10 +738,6 @@ struct intel_iommu *device_to_iommu(struct device *dev= , u8 *bus, u8 *devfn); extern void intel_svm_check(struct intel_iommu *iommu); extern int intel_svm_enable_prq(struct intel_iommu *iommu); extern int intel_svm_finish_prq(struct intel_iommu *iommu); -struct iommu_sva *intel_svm_bind(struct device *dev, struct mm_struct *mm, - void *drvdata); -void intel_svm_unbind(struct iommu_sva *handle); -u32 intel_svm_get_pasid(struct iommu_sva *handle); int intel_svm_page_response(struct device *dev, struct iommu_fault_event *= evt, struct iommu_page_response *msg); struct iommu_domain *intel_svm_domain_alloc(void); diff --git a/include/linux/iommu.h b/include/linux/iommu.h index 5a3ef4d58b1f..392b8adc3495 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -215,9 +215,6 @@ struct iommu_iotlb_gather { * @dev_has/enable/disable_feat: per device entries to check/enable/disable * iommu specific features. * @dev_feat_enabled: check enabled feature - * @sva_bind: Bind process address space to device - * @sva_unbind: Unbind process address space from device - * @sva_get_pasid: Get PASID associated to a SVA handle * @page_response: handle page request response * @def_domain_type: device default domain type, return value: * - IOMMU_DOMAIN_IDENTITY: must use an identity domain @@ -251,11 +248,6 @@ struct iommu_ops { int (*dev_enable_feat)(struct device *dev, enum iommu_dev_features f); int (*dev_disable_feat)(struct device *dev, enum iommu_dev_features f); =20 - struct iommu_sva *(*sva_bind)(struct device *dev, struct mm_struct *mm, - void *drvdata); - void (*sva_unbind)(struct iommu_sva *handle); - u32 (*sva_get_pasid)(struct iommu_sva *handle); - int (*page_response)(struct device *dev, struct iommu_fault_event *evt, struct iommu_page_response *msg); diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/ar= m/arm-smmu-v3/arm-smmu-v3.h index e077f21e2528..15dd4c7e6d3a 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -754,10 +754,6 @@ bool arm_smmu_master_sva_enabled(struct arm_smmu_maste= r *master); int arm_smmu_master_enable_sva(struct arm_smmu_master *master); int arm_smmu_master_disable_sva(struct arm_smmu_master *master); bool arm_smmu_master_iopf_supported(struct arm_smmu_master *master); -struct iommu_sva *arm_smmu_sva_bind(struct device *dev, struct mm_struct *= mm, - void *drvdata); -void arm_smmu_sva_unbind(struct iommu_sva *handle); -u32 arm_smmu_sva_get_pasid(struct iommu_sva *handle); void arm_smmu_sva_notifier_synchronize(void); struct iommu_domain *arm_smmu_sva_domain_alloc(void); #else /* CONFIG_ARM_SMMU_V3_SVA */ @@ -791,19 +787,6 @@ static inline bool arm_smmu_master_iopf_supported(stru= ct arm_smmu_master *master return false; } =20 -static inline struct iommu_sva * -arm_smmu_sva_bind(struct device *dev, struct mm_struct *mm, void *drvdata) -{ - return ERR_PTR(-ENODEV); -} - -static inline void arm_smmu_sva_unbind(struct iommu_sva *handle) {} - -static inline u32 arm_smmu_sva_get_pasid(struct iommu_sva *handle) -{ - return IOMMU_PASID_INVALID; -} - static inline void arm_smmu_sva_notifier_synchronize(void) {} =20 static inline struct iommu_domain *arm_smmu_sva_domain_alloc(void) diff --git a/drivers/iommu/iommu-sva-lib.h b/drivers/iommu/iommu-sva-lib.h index 8909ea1094e3..3420654c6e2f 100644 --- a/drivers/iommu/iommu-sva-lib.h +++ b/drivers/iommu/iommu-sva-lib.h @@ -8,7 +8,6 @@ #include #include =20 -int iommu_sva_alloc_pasid(struct mm_struct *mm, ioasid_t min, ioasid_t max= ); struct mm_struct *iommu_sva_find(ioasid_t pasid); =20 /* I/O Page fault */ diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iomm= u/arm/arm-smmu-v3/arm-smmu-v3-sva.c index 9d176e836d6b..56644e553c42 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -335,11 +335,6 @@ __arm_smmu_sva_bind(struct device *dev, struct mm_stru= ct *mm) if (!bond) return ERR_PTR(-ENOMEM); =20 - /* Allocate a PASID for this mm if necessary */ - ret =3D iommu_sva_alloc_pasid(mm, 1, (1U << master->ssid_bits) - 1); - if (ret) - goto err_free_bond; - bond->mm =3D mm; bond->sva.dev =3D dev; refcount_set(&bond->refs, 1); @@ -358,42 +353,6 @@ __arm_smmu_sva_bind(struct device *dev, struct mm_stru= ct *mm) return ERR_PTR(ret); } =20 -struct iommu_sva * -arm_smmu_sva_bind(struct device *dev, struct mm_struct *mm, void *drvdata) -{ - struct iommu_sva *handle; - struct iommu_domain *domain =3D iommu_get_domain_for_dev(dev); - struct arm_smmu_domain *smmu_domain =3D to_smmu_domain(domain); - - if (smmu_domain->stage !=3D ARM_SMMU_DOMAIN_S1) - return ERR_PTR(-EINVAL); - - mutex_lock(&sva_lock); - handle =3D __arm_smmu_sva_bind(dev, mm); - mutex_unlock(&sva_lock); - return handle; -} - -void arm_smmu_sva_unbind(struct iommu_sva *handle) -{ - struct arm_smmu_bond *bond =3D sva_to_bond(handle); - - mutex_lock(&sva_lock); - if (refcount_dec_and_test(&bond->refs)) { - list_del(&bond->list); - arm_smmu_mmu_notifier_put(bond->smmu_mn); - kfree(bond); - } - mutex_unlock(&sva_lock); -} - -u32 arm_smmu_sva_get_pasid(struct iommu_sva *handle) -{ - struct arm_smmu_bond *bond =3D sva_to_bond(handle); - - return bond->mm->pasid; -} - bool arm_smmu_sva_supported(struct arm_smmu_device *smmu) { unsigned long reg, fld; diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/ar= m/arm-smmu-v3/arm-smmu-v3.c index 9daf3de7e539..6a10fa181827 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -2857,9 +2857,6 @@ static struct iommu_ops arm_smmu_ops =3D { .dev_feat_enabled =3D arm_smmu_dev_feature_enabled, .dev_enable_feat =3D arm_smmu_dev_enable_feature, .dev_disable_feat =3D arm_smmu_dev_disable_feature, - .sva_bind =3D arm_smmu_sva_bind, - .sva_unbind =3D arm_smmu_sva_unbind, - .sva_get_pasid =3D arm_smmu_sva_get_pasid, .page_response =3D arm_smmu_page_response, .pgsize_bitmap =3D -1UL, /* Restricted during device attach */ .owner =3D THIS_MODULE, diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c index 10b1e9dcbd98..a5728f743c6d 100644 --- a/drivers/iommu/intel/iommu.c +++ b/drivers/iommu/intel/iommu.c @@ -4919,9 +4919,6 @@ const struct iommu_ops intel_iommu_ops =3D { .def_domain_type =3D device_def_domain_type, .pgsize_bitmap =3D SZ_4K, #ifdef CONFIG_INTEL_IOMMU_SVM - .sva_bind =3D intel_svm_bind, - .sva_unbind =3D intel_svm_unbind, - .sva_get_pasid =3D intel_svm_get_pasid, .page_response =3D intel_svm_page_response, #endif .default_domain_ops =3D &(const struct iommu_domain_ops) { diff --git a/drivers/iommu/intel/svm.c b/drivers/iommu/intel/svm.c index 641ab0491ada..ca83ebc708a8 100644 --- a/drivers/iommu/intel/svm.c +++ b/drivers/iommu/intel/svm.c @@ -313,14 +313,6 @@ static int pasid_to_svm_sdev(struct device *dev, unsig= ned int pasid, return 0; } =20 -static int intel_svm_alloc_pasid(struct device *dev, struct mm_struct *mm) -{ - ioasid_t max_pasid =3D dev_is_pci(dev) ? - pci_max_pasids(to_pci_dev(dev)) : intel_pasid_max_id; - - return iommu_sva_alloc_pasid(mm, PASID_MIN, max_pasid - 1); -} - static struct iommu_sva *intel_svm_bind_mm(struct intel_iommu *iommu, struct device *dev, struct mm_struct *mm) @@ -809,47 +801,6 @@ static irqreturn_t prq_event_thread(int irq, void *d) return IRQ_RETVAL(handled); } =20 -struct iommu_sva *intel_svm_bind(struct device *dev, struct mm_struct *mm,= void *drvdata) -{ - struct intel_iommu *iommu =3D device_to_iommu(dev, NULL, NULL); - struct iommu_sva *sva; - int ret; - - mutex_lock(&pasid_mutex); - ret =3D intel_svm_alloc_pasid(dev, mm); - if (ret) { - mutex_unlock(&pasid_mutex); - return ERR_PTR(ret); - } - - sva =3D intel_svm_bind_mm(iommu, dev, mm); - mutex_unlock(&pasid_mutex); - - return sva; -} - -void intel_svm_unbind(struct iommu_sva *sva) -{ - struct intel_svm_dev *sdev =3D to_intel_svm_dev(sva); - - mutex_lock(&pasid_mutex); - intel_svm_unbind_mm(sdev->dev, sdev->pasid); - mutex_unlock(&pasid_mutex); -} - -u32 intel_svm_get_pasid(struct iommu_sva *sva) -{ - struct intel_svm_dev *sdev; - u32 pasid; - - mutex_lock(&pasid_mutex); - sdev =3D to_intel_svm_dev(sva); - pasid =3D sdev->pasid; - mutex_unlock(&pasid_mutex); - - return pasid; -} - int intel_svm_page_response(struct device *dev, struct iommu_fault_event *evt, struct iommu_page_response *msg) diff --git a/drivers/iommu/iommu-sva-lib.c b/drivers/iommu/iommu-sva-lib.c index e7301514f286..ef6ed87d04ba 100644 --- a/drivers/iommu/iommu-sva-lib.c +++ b/drivers/iommu/iommu-sva-lib.c @@ -24,7 +24,8 @@ static DECLARE_IOASID_SET(iommu_sva_pasid); * * Returns 0 on success and < 0 on error. */ -int iommu_sva_alloc_pasid(struct mm_struct *mm, ioasid_t min, ioasid_t max) +static int iommu_sva_alloc_pasid(struct mm_struct *mm, + ioasid_t min, ioasid_t max) { int ret =3D 0; ioasid_t pasid; @@ -50,7 +51,6 @@ int iommu_sva_alloc_pasid(struct mm_struct *mm, ioasid_t = min, ioasid_t max) mutex_unlock(&iommu_sva_lock); return ret; } -EXPORT_SYMBOL_GPL(iommu_sva_alloc_pasid); =20 /* ioasid_find getter() requires a void * argument */ static bool __mmget_not_zero(void *mm) --=20 2.25.1 From nobody Fri May 8 06:00:20 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3CFCAC433F5 for ; Tue, 10 May 2022 06:22:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237229AbiEJG0k (ORCPT ); Tue, 10 May 2022 02:26:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38312 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237072AbiEJGZt (ORCPT ); Tue, 10 May 2022 02:25:49 -0400 Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 42B2836326 for ; Mon, 9 May 2022 23:21:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1652163689; x=1683699689; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=1H1VZfmh6IMSsHWOo2uEG/ch0molOpL33CJ2YDYCw0g=; b=gIHnK0Xneq1HgY5aFy5fvwGFmZKtWZzVdhtAuuF26WPhbwKjjxY5wf0S AQ1L5cSZi17rY7N7FflBAD3djfIRWnxUatij3HtulUaj3K7yFcJf8Lmsj 8fI/l040LKz519BkT7kK6uh4vVxnr9pk+nYwOxXxFL4KsZZbwATpuOsS8 6LP46s0jLjuyUOeNqIJU39UQI/mtr//hsTJI66RWwrubtk/iPF2sY7di7 /XmSkUjcPljz4bTPUiv99YwtHijPrTp6HHYubGVnUmcAMMPWcPzFj2uGk Ndn6ePE78VGVKJT0G/Am5z0/sqtD3joxpLsDlVglcXqbnI9BB8m93VOf8 w==; X-IronPort-AV: E=McAfee;i="6400,9594,10342"; a="332313098" X-IronPort-AV: E=Sophos;i="5.91,213,1647327600"; d="scan'208";a="332313098" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 May 2022 23:21:28 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.91,213,1647327600"; d="scan'208";a="552636647" Received: from allen-box.sh.intel.com ([10.239.159.48]) by orsmga002.jf.intel.com with ESMTP; 09 May 2022 23:21:24 -0700 From: Lu Baolu To: Joerg Roedel , Jason Gunthorpe , Christoph Hellwig , Kevin Tian , Ashok Raj , Will Deacon , Robin Murphy , Jean-Philippe Brucker , Dave Jiang , Vinod Koul Cc: Eric Auger , Liu Yi L , Jacob jun Pan , iommu@lists.linux-foundation.org, linux-kernel@vger.kernel.org, Lu Baolu , Jean-Philippe Brucker Subject: [PATCH v6 10/12] iommu: Prepare IOMMU domain for IOPF Date: Tue, 10 May 2022 14:17:36 +0800 Message-Id: <20220510061738.2761430-11-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220510061738.2761430-1-baolu.lu@linux.intel.com> References: <20220510061738.2761430-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" This adds some mechanisms around the iommu_domain so that the I/O page fault handling framework could route a page fault to the domain and call the fault handler from it. Add pointers to the page fault handler and its private data in struct iommu_domain. The fault handler will be called with the private data as a parameter once a page fault is routed to the domain. Any kernel component which owns an iommu domain could install handler and its private parameter so that the page fault could be further routed and handled. A new helper iommu_get_domain_for_iopf() which retrieves attached domain for a {device, PASID} pair is added. It will be used by the page fault handling framework which knows {device, PASID} reported from the iommu driver. We have a guarantee that the SVA domain doesn't go away during IOPF handling, because unbind() waits for pending faults with iopf_queue_flush_dev() before freeing the domain. Hence, there's no need to synchronize life cycle of the iommu domains between the unbind() and the interrupt threads. This also prepares the SVA implementation to be the first consumer of the per-domain page fault handling model. Suggested-by: Jean-Philippe Brucker Signed-off-by: Lu Baolu --- include/linux/iommu.h | 12 +++++++ drivers/iommu/io-pgfault.c | 7 ++++ drivers/iommu/iommu-sva-lib.c | 65 +++++++++++++++++++++++++++++++++++ drivers/iommu/iommu.c | 27 +++++++++++++++ 4 files changed, 111 insertions(+) diff --git a/include/linux/iommu.h b/include/linux/iommu.h index 392b8adc3495..9405034e3013 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -103,6 +103,9 @@ struct iommu_domain { #ifdef CONFIG_IOMMU_SVA struct mm_struct *mm; #endif + enum iommu_page_response_code (*iopf_handler)(struct iommu_fault *fault, + void *data); + void *fault_data; }; =20 static inline bool iommu_is_dma_domain(struct iommu_domain *domain) @@ -687,6 +690,9 @@ int iommu_attach_device_pasid(struct iommu_domain *doma= in, struct device *dev, ioasid_t pasid); void iommu_detach_device_pasid(struct iommu_domain *domain, struct device *dev, ioasid_t pasid); +struct iommu_domain * +iommu_get_domain_for_iopf(struct device *dev, ioasid_t pasid); + #else /* CONFIG_IOMMU_API */ =20 struct iommu_ops {}; @@ -1056,6 +1062,12 @@ static inline void iommu_detach_device_pasid(struct = iommu_domain *domain, struct device *dev, ioasid_t pasid) { } + +static inline struct iommu_domain * +iommu_get_domain_for_iopf(struct device *dev, ioasid_t pasid) +{ + return NULL; +} #endif /* CONFIG_IOMMU_API */ =20 #ifdef CONFIG_IOMMU_SVA diff --git a/drivers/iommu/io-pgfault.c b/drivers/iommu/io-pgfault.c index 1df8c1dcae77..aee9e033012f 100644 --- a/drivers/iommu/io-pgfault.c +++ b/drivers/iommu/io-pgfault.c @@ -181,6 +181,13 @@ static void iopf_handle_group(struct work_struct *work) * request completes, outstanding faults will have been dealt with by the = time * the PASID is freed. * + * Any valid page fault will be eventually routed to an iommu domain and t= he + * page fault handler installed there will get called. The users of this + * handling framework should guarantee that the iommu domain could only be + * freed after the device has stopped generating page faults (or the iommu + * hardware has been set to block the page faults) and the pending page fa= ults + * have been flushed. + * * Return: 0 on success and <0 on error. */ int iommu_queue_iopf(struct iommu_fault *fault, void *cookie) diff --git a/drivers/iommu/iommu-sva-lib.c b/drivers/iommu/iommu-sva-lib.c index ef6ed87d04ba..32c836e4a60e 100644 --- a/drivers/iommu/iommu-sva-lib.c +++ b/drivers/iommu/iommu-sva-lib.c @@ -72,6 +72,69 @@ struct mm_struct *iommu_sva_find(ioasid_t pasid) } EXPORT_SYMBOL_GPL(iommu_sva_find); =20 +/* + * I/O page fault handler for SVA + * + * Copied from io-pgfault.c with mmget_not_zero() added before + * mmap_read_lock(). + */ +static enum iommu_page_response_code +iommu_sva_handle_iopf(struct iommu_fault *fault, void *data) +{ + vm_fault_t ret; + struct mm_struct *mm; + struct vm_area_struct *vma; + unsigned int access_flags =3D 0; + struct iommu_domain *domain =3D data; + unsigned int fault_flags =3D FAULT_FLAG_REMOTE; + struct iommu_fault_page_request *prm =3D &fault->prm; + enum iommu_page_response_code status =3D IOMMU_PAGE_RESP_INVALID; + + if (!(prm->flags & IOMMU_FAULT_PAGE_REQUEST_PASID_VALID)) + return status; + + mm =3D domain->mm; + if (IS_ERR_OR_NULL(mm) || !mmget_not_zero(mm)) + return status; + + mmap_read_lock(mm); + + vma =3D find_extend_vma(mm, prm->addr); + if (!vma) + /* Unmapped area */ + goto out_put_mm; + + if (prm->perm & IOMMU_FAULT_PERM_READ) + access_flags |=3D VM_READ; + + if (prm->perm & IOMMU_FAULT_PERM_WRITE) { + access_flags |=3D VM_WRITE; + fault_flags |=3D FAULT_FLAG_WRITE; + } + + if (prm->perm & IOMMU_FAULT_PERM_EXEC) { + access_flags |=3D VM_EXEC; + fault_flags |=3D FAULT_FLAG_INSTRUCTION; + } + + if (!(prm->perm & IOMMU_FAULT_PERM_PRIV)) + fault_flags |=3D FAULT_FLAG_USER; + + if (access_flags & ~vma->vm_flags) + /* Access fault */ + goto out_put_mm; + + ret =3D handle_mm_fault(vma, prm->addr, fault_flags, NULL); + status =3D ret & VM_FAULT_ERROR ? IOMMU_PAGE_RESP_INVALID : + IOMMU_PAGE_RESP_SUCCESS; + +out_put_mm: + mmap_read_unlock(mm); + mmput(mm); + + return status; +} + /* * IOMMU SVA driver-oriented interfaces */ @@ -91,6 +154,8 @@ iommu_sva_alloc_domain(struct device *dev, struct mm_str= uct *mm) mmgrab(mm); domain->mm =3D mm; domain->type =3D IOMMU_DOMAIN_SVA; + domain->iopf_handler =3D iommu_sva_handle_iopf; + domain->fault_data =3D domain; =20 return domain; } diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c index 367d0ecf6e12..00d0262a1fe9 100644 --- a/drivers/iommu/iommu.c +++ b/drivers/iommu/iommu.c @@ -3172,3 +3172,30 @@ void iommu_detach_device_pasid(struct iommu_domain *= domain, =20 iommu_group_put(group); } + +struct iommu_domain *iommu_get_domain_for_iopf(struct device *dev, + ioasid_t pasid) +{ + struct iommu_domain *domain; + struct iommu_group *group; + + if (!pasid_valid(pasid)) + return NULL; + + group =3D iommu_group_get(dev); + if (!group) + return NULL; + + /* + * Safe to fetch outside the group mutex because: + * + * - The xarray protects its internal state with RCU; + * - The domain obtained is either NULL or fully formed; + * - Current IOPF framework requires that all IOPFs should be quieted + * and flushed from the pending list before the domain is freed. + */ + domain =3D xa_load(&group->pasid_array, pasid); + iommu_group_put(group); + + return domain; +} --=20 2.25.1 From nobody Fri May 8 06:00:20 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 36F0FC433F5 for ; Tue, 10 May 2022 06:22:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237252AbiEJG0o (ORCPT ); Tue, 10 May 2022 02:26:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35712 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237231AbiEJGZv (ORCPT ); Tue, 10 May 2022 02:25:51 -0400 Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 426774BFC4 for ; Mon, 9 May 2022 23:21:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1652163694; x=1683699694; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=QroNYLJpUsYM/NBPVJf3TutCJ8ddwM1iHl2fHZWXXoo=; b=eL2sb1YaQXyO1ZrQGtOUV3fPYJSZmD8ON/zFYwXpVn/ZXU+KqS1hIH6c e+BcSLJe/GdFn7E+whRnKDJ5ZiPZKYRG0jnekZmbGukeMCxA/zanbhM9w Tb4nhWirPq8uCeKQCOiJt/jZuTkgSMo4sm6y/yDUVH32Slefe5yL08s15 XRROZS+6LyHBUvQ8VjFOxOzawQVVXPjYQMrYHq37tUAq3l5nYNORFl00K ajNUCBL6yiu/aNkKToy4k0qzQOGV9E5eVkkUdl6CH5yuMaJxPYq6CRGOj SpAIAO2NYvDfnfnxI542cPAorccZPJQjXMToasb81IpQ3jVhyh+uRtPh7 g==; X-IronPort-AV: E=McAfee;i="6400,9594,10342"; a="332313120" X-IronPort-AV: E=Sophos;i="5.91,213,1647327600"; d="scan'208";a="332313120" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 May 2022 23:21:33 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.91,213,1647327600"; d="scan'208";a="552636709" Received: from allen-box.sh.intel.com ([10.239.159.48]) by orsmga002.jf.intel.com with ESMTP; 09 May 2022 23:21:29 -0700 From: Lu Baolu To: Joerg Roedel , Jason Gunthorpe , Christoph Hellwig , Kevin Tian , Ashok Raj , Will Deacon , Robin Murphy , Jean-Philippe Brucker , Dave Jiang , Vinod Koul Cc: Eric Auger , Liu Yi L , Jacob jun Pan , iommu@lists.linux-foundation.org, linux-kernel@vger.kernel.org, Lu Baolu , Jean-Philippe Brucker Subject: [PATCH v6 11/12] iommu: Per-domain I/O page fault handling Date: Tue, 10 May 2022 14:17:37 +0800 Message-Id: <20220510061738.2761430-12-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220510061738.2761430-1-baolu.lu@linux.intel.com> References: <20220510061738.2761430-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Tweak the I/O page fault handling framework to route the page faults to the domain and call the page fault handler retrieved from the domain. This makes the I/O page fault handling framework possible to serve more usage scenarios as long as they have an IOMMU domain and install a page fault handler in it. Some unused functions are also removed to avoid dead code. Signed-off-by: Lu Baolu Reviewed-by: Jean-Philippe Brucker --- drivers/iommu/iommu-sva-lib.h | 2 -- drivers/iommu/io-pgfault.c | 64 ++++------------------------------- drivers/iommu/iommu-sva-lib.c | 20 ----------- 3 files changed, 7 insertions(+), 79 deletions(-) diff --git a/drivers/iommu/iommu-sva-lib.h b/drivers/iommu/iommu-sva-lib.h index 3420654c6e2f..74ce2e76321b 100644 --- a/drivers/iommu/iommu-sva-lib.h +++ b/drivers/iommu/iommu-sva-lib.h @@ -8,8 +8,6 @@ #include #include =20 -struct mm_struct *iommu_sva_find(ioasid_t pasid); - /* I/O Page fault */ struct device; struct iommu_fault; diff --git a/drivers/iommu/io-pgfault.c b/drivers/iommu/io-pgfault.c index aee9e033012f..9efe5259402b 100644 --- a/drivers/iommu/io-pgfault.c +++ b/drivers/iommu/io-pgfault.c @@ -69,69 +69,18 @@ static int iopf_complete_group(struct device *dev, stru= ct iopf_fault *iopf, return iommu_page_response(dev, &resp); } =20 -static enum iommu_page_response_code -iopf_handle_single(struct iopf_fault *iopf) -{ - vm_fault_t ret; - struct mm_struct *mm; - struct vm_area_struct *vma; - unsigned int access_flags =3D 0; - unsigned int fault_flags =3D FAULT_FLAG_REMOTE; - struct iommu_fault_page_request *prm =3D &iopf->fault.prm; - enum iommu_page_response_code status =3D IOMMU_PAGE_RESP_INVALID; - - if (!(prm->flags & IOMMU_FAULT_PAGE_REQUEST_PASID_VALID)) - return status; - - mm =3D iommu_sva_find(prm->pasid); - if (IS_ERR_OR_NULL(mm)) - return status; - - mmap_read_lock(mm); - - vma =3D find_extend_vma(mm, prm->addr); - if (!vma) - /* Unmapped area */ - goto out_put_mm; - - if (prm->perm & IOMMU_FAULT_PERM_READ) - access_flags |=3D VM_READ; - - if (prm->perm & IOMMU_FAULT_PERM_WRITE) { - access_flags |=3D VM_WRITE; - fault_flags |=3D FAULT_FLAG_WRITE; - } - - if (prm->perm & IOMMU_FAULT_PERM_EXEC) { - access_flags |=3D VM_EXEC; - fault_flags |=3D FAULT_FLAG_INSTRUCTION; - } - - if (!(prm->perm & IOMMU_FAULT_PERM_PRIV)) - fault_flags |=3D FAULT_FLAG_USER; - - if (access_flags & ~vma->vm_flags) - /* Access fault */ - goto out_put_mm; - - ret =3D handle_mm_fault(vma, prm->addr, fault_flags, NULL); - status =3D ret & VM_FAULT_ERROR ? IOMMU_PAGE_RESP_INVALID : - IOMMU_PAGE_RESP_SUCCESS; - -out_put_mm: - mmap_read_unlock(mm); - mmput(mm); - - return status; -} - static void iopf_handle_group(struct work_struct *work) { struct iopf_group *group; + struct iommu_domain *domain; struct iopf_fault *iopf, *next; enum iommu_page_response_code status =3D IOMMU_PAGE_RESP_SUCCESS; =20 group =3D container_of(work, struct iopf_group, work); + domain =3D iommu_get_domain_for_iopf(group->dev, + group->last_fault.fault.prm.pasid); + if (!domain || !domain->iopf_handler) + status =3D IOMMU_PAGE_RESP_INVALID; =20 list_for_each_entry_safe(iopf, next, &group->faults, list) { /* @@ -139,7 +88,8 @@ static void iopf_handle_group(struct work_struct *work) * faults in the group if there is an error. */ if (status =3D=3D IOMMU_PAGE_RESP_SUCCESS) - status =3D iopf_handle_single(iopf); + status =3D domain->iopf_handler(&iopf->fault, + domain->fault_data); =20 if (!(iopf->fault.prm.flags & IOMMU_FAULT_PAGE_REQUEST_LAST_PAGE)) diff --git a/drivers/iommu/iommu-sva-lib.c b/drivers/iommu/iommu-sva-lib.c index 32c836e4a60e..ea12504a9e12 100644 --- a/drivers/iommu/iommu-sva-lib.c +++ b/drivers/iommu/iommu-sva-lib.c @@ -52,26 +52,6 @@ static int iommu_sva_alloc_pasid(struct mm_struct *mm, return ret; } =20 -/* ioasid_find getter() requires a void * argument */ -static bool __mmget_not_zero(void *mm) -{ - return mmget_not_zero(mm); -} - -/** - * iommu_sva_find() - Find mm associated to the given PASID - * @pasid: Process Address Space ID assigned to the mm - * - * On success a reference to the mm is taken, and must be released with mm= put(). - * - * Returns the mm corresponding to this PASID, or an error if not found. - */ -struct mm_struct *iommu_sva_find(ioasid_t pasid) -{ - return ioasid_find(&iommu_sva_pasid, pasid, __mmget_not_zero); -} -EXPORT_SYMBOL_GPL(iommu_sva_find); - /* * I/O page fault handler for SVA * --=20 2.25.1 From nobody Fri May 8 06:00:20 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E5B66C433F5 for ; Tue, 10 May 2022 06:22:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237257AbiEJG0w (ORCPT ); Tue, 10 May 2022 02:26:52 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38908 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237123AbiEJGZz (ORCPT ); Tue, 10 May 2022 02:25:55 -0400 Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9F4E32A76AB for ; Mon, 9 May 2022 23:21:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1652163697; x=1683699697; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=VHde7Ed3dTEnDTSMg2+VjB6VeqUvISFvmHCLE7rzT4A=; b=QAoo2dB9BOdhgHZqzdsLrPLGxUqntigxfIxc985xtyVkoQ8NuEZND/TF TJQdcVhCUtwsVicTRxnOZ1nRUC9dIqubxnXv8afOlvVULhnlXjvwAH1zL tH3/bx1rCZGnYgJ1Ak7yXI0g/joubI0jd7lNYoI0pss93nTExKuVKw03P /vHaCkCHXSXODCwk4pTFIHIDKcp+LpP1bCWlzIoKrv+yED7RTO1Ahxy6n TuExtfpLD86FrLN6Fa9IxquV91rSLCr9PA3v2B+Cb5pG8eiphNYtf+DC2 1DMrKT6SjzGXZkdiNpIOqzvxfZmveifJCE9pFv+R8pChwxIx4zAoS5CZ8 w==; X-IronPort-AV: E=McAfee;i="6400,9594,10342"; a="332313129" X-IronPort-AV: E=Sophos;i="5.91,213,1647327600"; d="scan'208";a="332313129" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 May 2022 23:21:37 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.91,213,1647327600"; d="scan'208";a="552636731" Received: from allen-box.sh.intel.com ([10.239.159.48]) by orsmga002.jf.intel.com with ESMTP; 09 May 2022 23:21:33 -0700 From: Lu Baolu To: Joerg Roedel , Jason Gunthorpe , Christoph Hellwig , Kevin Tian , Ashok Raj , Will Deacon , Robin Murphy , Jean-Philippe Brucker , Dave Jiang , Vinod Koul Cc: Eric Auger , Liu Yi L , Jacob jun Pan , iommu@lists.linux-foundation.org, linux-kernel@vger.kernel.org, Lu Baolu , Jean-Philippe Brucker Subject: [PATCH v6 12/12] iommu: Rename iommu-sva-lib.{c,h} Date: Tue, 10 May 2022 14:17:38 +0800 Message-Id: <20220510061738.2761430-13-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220510061738.2761430-1-baolu.lu@linux.intel.com> References: <20220510061738.2761430-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Rename iommu-sva-lib.c[h] to iommu-sva.c[h] as it contains all code for SVA implementation in iommu core. Signed-off-by: Lu Baolu Reviewed-by: Jean-Philippe Brucker --- drivers/iommu/{iommu-sva-lib.h =3D> iommu-sva.h} | 0 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 2 +- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 2 +- drivers/iommu/intel/iommu.c | 2 +- drivers/iommu/intel/svm.c | 2 +- drivers/iommu/io-pgfault.c | 2 +- drivers/iommu/{iommu-sva-lib.c =3D> iommu-sva.c} | 2 +- drivers/iommu/Makefile | 2 +- 8 files changed, 7 insertions(+), 7 deletions(-) rename drivers/iommu/{iommu-sva-lib.h =3D> iommu-sva.h} (100%) rename drivers/iommu/{iommu-sva-lib.c =3D> iommu-sva.c} (99%) diff --git a/drivers/iommu/iommu-sva-lib.h b/drivers/iommu/iommu-sva.h similarity index 100% rename from drivers/iommu/iommu-sva-lib.h rename to drivers/iommu/iommu-sva.h diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iomm= u/arm/arm-smmu-v3/arm-smmu-v3-sva.c index 56644e553c42..265b125d7dc4 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -9,7 +9,7 @@ #include =20 #include "arm-smmu-v3.h" -#include "../../iommu-sva-lib.h" +#include "../../iommu-sva.h" #include "../../io-pgtable-arm.h" =20 struct arm_smmu_mmu_notifier { diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/ar= m/arm-smmu-v3/arm-smmu-v3.c index 6a10fa181827..de3b6fbf8766 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -31,7 +31,7 @@ #include =20 #include "arm-smmu-v3.h" -#include "../../iommu-sva-lib.h" +#include "../../iommu-sva.h" =20 static bool disable_bypass =3D true; module_param(disable_bypass, bool, 0444); diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c index a5728f743c6d..1c2c92b657c7 100644 --- a/drivers/iommu/intel/iommu.c +++ b/drivers/iommu/intel/iommu.c @@ -27,7 +27,7 @@ #include =20 #include "../irq_remapping.h" -#include "../iommu-sva-lib.h" +#include "../iommu-sva.h" #include "pasid.h" #include "cap_audit.h" =20 diff --git a/drivers/iommu/intel/svm.c b/drivers/iommu/intel/svm.c index ca83ebc708a8..44331db060e4 100644 --- a/drivers/iommu/intel/svm.c +++ b/drivers/iommu/intel/svm.c @@ -25,7 +25,7 @@ =20 #include "pasid.h" #include "perf.h" -#include "../iommu-sva-lib.h" +#include "../iommu-sva.h" =20 static irqreturn_t prq_event_thread(int irq, void *d); static void intel_svm_drain_prq(struct device *dev, u32 pasid); diff --git a/drivers/iommu/io-pgfault.c b/drivers/iommu/io-pgfault.c index 9efe5259402b..2a8604013b7e 100644 --- a/drivers/iommu/io-pgfault.c +++ b/drivers/iommu/io-pgfault.c @@ -11,7 +11,7 @@ #include #include =20 -#include "iommu-sva-lib.h" +#include "iommu-sva.h" =20 /** * struct iopf_queue - IO Page Fault queue diff --git a/drivers/iommu/iommu-sva-lib.c b/drivers/iommu/iommu-sva.c similarity index 99% rename from drivers/iommu/iommu-sva-lib.c rename to drivers/iommu/iommu-sva.c index ea12504a9e12..1791ac1e3d34 100644 --- a/drivers/iommu/iommu-sva-lib.c +++ b/drivers/iommu/iommu-sva.c @@ -7,7 +7,7 @@ #include #include =20 -#include "iommu-sva-lib.h" +#include "iommu-sva.h" =20 static DEFINE_MUTEX(iommu_sva_lock); static DECLARE_IOASID_SET(iommu_sva_pasid); diff --git a/drivers/iommu/Makefile b/drivers/iommu/Makefile index 44475a9b3eea..c1763476162b 100644 --- a/drivers/iommu/Makefile +++ b/drivers/iommu/Makefile @@ -27,6 +27,6 @@ obj-$(CONFIG_FSL_PAMU) +=3D fsl_pamu.o fsl_pamu_domain.o obj-$(CONFIG_S390_IOMMU) +=3D s390-iommu.o obj-$(CONFIG_HYPERV_IOMMU) +=3D hyperv-iommu.o obj-$(CONFIG_VIRTIO_IOMMU) +=3D virtio-iommu.o -obj-$(CONFIG_IOMMU_SVA) +=3D iommu-sva-lib.o io-pgfault.o +obj-$(CONFIG_IOMMU_SVA) +=3D iommu-sva.o io-pgfault.o obj-$(CONFIG_SPRD_IOMMU) +=3D sprd-iommu.o obj-$(CONFIG_APPLE_DART) +=3D apple-dart.o --=20 2.25.1