From nobody Mon Jun 22 23:53:34 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2DDE9C433EF for ; Tue, 15 Mar 2022 05:11:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344852AbiCOFMV (ORCPT ); Tue, 15 Mar 2022 01:12:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48032 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1344840AbiCOFMR (ORCPT ); Tue, 15 Mar 2022 01:12:17 -0400 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A93524924A for ; Mon, 14 Mar 2022 22:11:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1647321066; x=1678857066; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=ekgMTzrt4B/E5Ajgcaoa3ieo7uw7tCFXNnm15MECQas=; b=DTQDd3SmHyOG2kwoQNdAOo31txWrmjNssI1YFsMCIIVPwRofpccbAq1T gbdiQq8ucnX5J/VWXG0U7hWyXPxzoQR4oS0uo6CcHTAohgmDDqSH5Y2Ho xJg+rGg29ljCvWhv/rF0bzIW9ou0Z9prPgAQ7gY7OSv7FPzMzrXc2nrZ1 zIKcsvYtp1Hhx9iVzZ1Xxhcl2xJ4iGEx1ac6vLiRfmQ/7oQy73FKLn7Sb ZWKWDUZExWHg4TwbNzWnLuepYP3spvmuKHXlD4vVl1TXVWOoXClDBFHSx ULldq99ggKz6Fj+T+LGuf5nEGX9aeOP6KGQIHAuZSamSheqw7Dyp1rnG1 g==; X-IronPort-AV: E=McAfee;i="6200,9189,10286"; a="236159099" X-IronPort-AV: E=Sophos;i="5.90,182,1643702400"; d="scan'208";a="236159099" Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Mar 2022 22:03:58 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.90,182,1643702400"; d="scan'208";a="580383567" Received: from otc-wp-03.jf.intel.com (HELO jacob-builder.jf.intel.com) ([10.54.39.79]) by orsmga001.jf.intel.com with ESMTP; 14 Mar 2022 22:03:58 -0700 From: Jacob Pan To: iommu@lists.linux-foundation.org, LKML , Joerg Roedel , Jason Gunthorpe , "Christoph Hellwig" , "Lu Baolu" , Jean-Philippe Brucker Cc: Jacob Pan , Raj Ashok , "Kumar, Sanjay K" , Dave Jiang , Tony Luck , "Zanussi, Tom" , Dan Williams , "Tian, Kevin" , Yi Liu Subject: [PATCH v2 1/8] iommu: Assign per device max PASID Date: Mon, 14 Mar 2022 22:07:05 -0700 Message-Id: <20220315050713.2000518-2-jacob.jun.pan@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220315050713.2000518-1-jacob.jun.pan@linux.intel.com> References: <20220315050713.2000518-1-jacob.jun.pan@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Lu Baolu PCIe spec defines Max PASID Width per-device. Since a PASID is only used with IOMMU enabled, this patch introduces a PASID max variable on the per-device IOMMU data. It will be used for limiting PASID allocation in that PASID table is per-device. Signed-off-by: Lu Baolu Signed-off-by: Jacob Pan --- drivers/iommu/intel/iommu.c | 4 +++- include/linux/iommu.h | 13 +++++++++++++ 2 files changed, 16 insertions(+), 1 deletion(-) diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c index 50666d250b36..881f8361eca2 100644 --- a/drivers/iommu/intel/iommu.c +++ b/drivers/iommu/intel/iommu.c @@ -2602,8 +2602,10 @@ static struct dmar_domain *dmar_insert_one_dev_info(= struct intel_iommu *iommu, if (sm_supported(iommu)) { if (pasid_supported(iommu)) { int features =3D pci_pasid_features(pdev); - if (features >=3D 0) + if (features >=3D 0) { info->pasid_supported =3D features | 1; + iommu_set_dev_pasid_max(&pdev->dev, pci_max_pasids(pdev)); + } } =20 if (info->ats_supported && ecap_prs(iommu->ecap) && diff --git a/include/linux/iommu.h b/include/linux/iommu.h index de0c57a567c8..369f05c2a4e2 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -364,6 +364,7 @@ struct iommu_fault_param { * @fwspec: IOMMU fwspec data * @iommu_dev: IOMMU device this device is linked to * @priv: IOMMU Driver private data + * @pasid_max Max PASID value supported by this device * * TODO: migrate other per device data pointers under iommu_dev_data, e.g. * struct iommu_group *iommu_group; @@ -375,8 +376,20 @@ struct dev_iommu { struct iommu_fwspec *fwspec; struct iommu_device *iommu_dev; void *priv; + unsigned int pasid_max; }; =20 +static inline void iommu_set_dev_pasid_max(struct device *dev, + unsigned int max) +{ + struct dev_iommu *param =3D dev->iommu; + + if (WARN_ON(!param)) + return; + + param->pasid_max =3D max; +} + int iommu_device_register(struct iommu_device *iommu, const struct iommu_ops *ops, struct device *hwdev); --=20 2.25.1 From nobody Mon Jun 22 23:53:34 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4CD3FC433EF for ; Tue, 15 Mar 2022 05:11:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344880AbiCOFMe (ORCPT ); Tue, 15 Mar 2022 01:12:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48052 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1344842AbiCOFMT (ORCPT ); Tue, 15 Mar 2022 01:12:19 -0400 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9188A49250 for ; Mon, 14 Mar 2022 22:11:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1647321067; x=1678857067; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Ol0nXzisKyEPOLK+60+mhBiQ8qX4SQP0mQu06nsUuvU=; b=R7klZzYOg5Hmtx0aGtul9HX0RNXHOYSHxpQ25zf/RbQ5xKO5JPTLcrdB 3qWWJWbJKv4950gn/8SC9nSNtC8CppQkMZwOaFg3zYB1anlT6DoDG1EjK 2RRk8lKrBF9UDmwhpkemIKFvGtwVxWLANUL7f81e2MDK1JPeNpTUjNIZZ 6qWx8FiDAxrstu/c3BljOY34/kJhzZtKNJhmJ3SUQ1fIw06y5YsnJrVws go+0I7ayxK+4eXVajnsuxJvNSjeNOaRK63KLZDeqcORDQ4bUUQCtoyZgp T95C71dNDBip9Cs38RYDS96gSiQnXwpfCc4LeYXEkPYPHqjaJzkRWFZdG Q==; X-IronPort-AV: E=McAfee;i="6200,9189,10286"; a="236159100" X-IronPort-AV: E=Sophos;i="5.90,182,1643702400"; d="scan'208";a="236159100" Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Mar 2022 22:03:58 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.90,182,1643702400"; d="scan'208";a="580383571" Received: from otc-wp-03.jf.intel.com (HELO jacob-builder.jf.intel.com) ([10.54.39.79]) by orsmga001.jf.intel.com with ESMTP; 14 Mar 2022 22:03:58 -0700 From: Jacob Pan To: iommu@lists.linux-foundation.org, LKML , Joerg Roedel , Jason Gunthorpe , "Christoph Hellwig" , "Lu Baolu" , Jean-Philippe Brucker Cc: Jacob Pan , Raj Ashok , "Kumar, Sanjay K" , Dave Jiang , Tony Luck , "Zanussi, Tom" , Dan Williams , "Tian, Kevin" , Yi Liu Subject: [PATCH v2 2/8] iommu: Add attach/detach_dev_pasid domain ops Date: Mon, 14 Mar 2022 22:07:06 -0700 Message-Id: <20220315050713.2000518-3-jacob.jun.pan@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220315050713.2000518-1-jacob.jun.pan@linux.intel.com> References: <20220315050713.2000518-1-jacob.jun.pan@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Lu Baolu An IOMMU domain represents an address space which can be attached by devices that perform DMA within a domain. However, for platforms with PASID capability the domain attachment needs be handled at device+PASID level. There can be multiple PASIDs within a device and multiple devices attached to a given domain. This patch introduces a new IOMMU op which support device, PASID, and IOMMU domain attachment. The immediate use case is for PASID capable devices to perform DMA under DMA APIs. Signed-off-by: Lu Baolu Signed-off-by: Jacob Pan --- include/linux/iommu.h | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/include/linux/iommu.h b/include/linux/iommu.h index 369f05c2a4e2..fde5b933dbe3 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -227,6 +227,8 @@ struct iommu_iotlb_gather { * @aux_get_pasid: get the pasid given an aux-domain * @sva_bind: Bind process address space to device * @sva_unbind: Unbind process address space from device + * @attach_dev_pasid: attach an iommu domain to a pasid of device + * @detach_dev_pasid: detach an iommu domain from a pasid of device * @sva_get_pasid: Get PASID associated to a SVA handle * @page_response: handle page request response * @cache_invalidate: invalidate translation caches @@ -296,6 +298,10 @@ struct iommu_ops { struct iommu_sva *(*sva_bind)(struct device *dev, struct mm_struct *mm, void *drvdata); void (*sva_unbind)(struct iommu_sva *handle); + int (*attach_dev_pasid)(struct iommu_domain *domain, + struct device *dev, ioasid_t id); + void (*detach_dev_pasid)(struct iommu_domain *domain, + struct device *dev, ioasid_t id); u32 (*sva_get_pasid)(struct iommu_sva *handle); =20 int (*page_response)(struct device *dev, --=20 2.25.1 From nobody Mon Jun 22 23:53:34 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0F5BEC433EF for ; Tue, 15 Mar 2022 05:11:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344868AbiCOFMa (ORCPT ); Tue, 15 Mar 2022 01:12:30 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48046 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242877AbiCOFMS (ORCPT ); Tue, 15 Mar 2022 01:12:18 -0400 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AF05E4924E for ; Mon, 14 Mar 2022 22:11:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1647321066; x=1678857066; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=9Q9rSf59BKk7qLg8hPcHDXhXEuoUaTeoAQ+DuVJmmyE=; b=TdsFFqB0WvY8wvZ1HM6ZKnQNAGR4yJVEVtl6aNx7fA6T97xoScG6fRpI cynmpHysnTUSq4ZPk/dU8to22lvTCQVUq2bLfNv77nqOM70+dMc6CcT+F 48zly5twi26UQ/pMvhPzdgOAXgCxkyCkfaCWqVCoEGGM/gy5tnJyowQoO YiuaFK5oaouzfI1nH0jOUFL3puRxe4n9BRgyJuR2HtPTggz8C/YETHaNY XcDCVFWw91NuLguajTWhIbb7UjfRbo/rmOpdUeP03UaTONnVbMiajPpCZ lnOE0ZICh6OPdzVy3LTP88lgwAMcXNkKDa7+5GKWRF7pdYkudSv+FgZKB A==; X-IronPort-AV: E=McAfee;i="6200,9189,10286"; a="236159101" X-IronPort-AV: E=Sophos;i="5.90,182,1643702400"; d="scan'208";a="236159101" Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Mar 2022 22:03:58 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.90,182,1643702400"; d="scan'208";a="580383575" Received: from otc-wp-03.jf.intel.com (HELO jacob-builder.jf.intel.com) ([10.54.39.79]) by orsmga001.jf.intel.com with ESMTP; 14 Mar 2022 22:03:58 -0700 From: Jacob Pan To: iommu@lists.linux-foundation.org, LKML , Joerg Roedel , Jason Gunthorpe , "Christoph Hellwig" , "Lu Baolu" , Jean-Philippe Brucker Cc: Jacob Pan , Raj Ashok , "Kumar, Sanjay K" , Dave Jiang , Tony Luck , "Zanussi, Tom" , Dan Williams , "Tian, Kevin" , Yi Liu Subject: [PATCH v2 3/8] iommu/vt-d: Implement device_pasid domain attach ops Date: Mon, 14 Mar 2022 22:07:07 -0700 Message-Id: <20220315050713.2000518-4-jacob.jun.pan@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220315050713.2000518-1-jacob.jun.pan@linux.intel.com> References: <20220315050713.2000518-1-jacob.jun.pan@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" On VT-d platforms with scalable mode enabled, devices issue DMA requests with PASID need to attach to the correct IOMMU domains. The attach operation involves the following: - programming the PASID into device's PASID table - tracking device domain and the PASID relationship - managing IOTLB and device TLB invalidations This patch extends DMAR domain and device domain info with xarrays to track per domain and per device PASIDs. It provides the flexibility to be used beyond DMA API PASID support. Signed-off-by: Lu Baolu Signed-off-by: Jacob Pan --- drivers/iommu/intel/iommu.c | 194 +++++++++++++++++++++++++++++++++++- include/linux/intel-iommu.h | 12 ++- 2 files changed, 202 insertions(+), 4 deletions(-) diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c index 881f8361eca2..9267194eaed3 100644 --- a/drivers/iommu/intel/iommu.c +++ b/drivers/iommu/intel/iommu.c @@ -1622,20 +1622,48 @@ static void __iommu_flush_dev_iotlb(struct device_d= omain_info *info, qdep, addr, mask); } =20 +static void __iommu_flush_dev_piotlb(struct device_domain_info *info, + u64 address, + ioasid_t pasid, unsigned int mask) +{ + u16 sid, qdep; + + if (!info || !info->ats_enabled) + return; + + sid =3D info->bus << 8 | info->devfn; + qdep =3D info->ats_qdep; + qi_flush_dev_iotlb_pasid(info->iommu, sid, info->pfsid, + pasid, qdep, address, mask); +} + static void iommu_flush_dev_iotlb(struct dmar_domain *domain, u64 addr, unsigned mask) { unsigned long flags; struct device_domain_info *info; struct subdev_domain_info *sinfo; + unsigned long pasid; + struct pasid_info *pinfo; =20 if (!domain->has_iotlb_device) return; =20 spin_lock_irqsave(&device_domain_lock, flags); - list_for_each_entry(info, &domain->devices, link) - __iommu_flush_dev_iotlb(info, addr, mask); - + list_for_each_entry(info, &domain->devices, link) { + /* + * We cannot use PASID based devTLB invalidation on RID2PASID + * Device does not understand RID2PASID/0. This is different + * than IOTLB invalidation where RID2PASID is also used for + * tagging. + */ + xa_for_each(&info->pasids, pasid, pinfo) { + if (!pasid) + __iommu_flush_dev_iotlb(info, addr, mask); + else + __iommu_flush_dev_piotlb(info, addr, pasid, mask); + } + } list_for_each_entry(sinfo, &domain->subdevices, link_domain) { info =3D get_domain_info(sinfo->pdev); __iommu_flush_dev_iotlb(info, addr, mask); @@ -1648,6 +1676,8 @@ static void domain_flush_piotlb(struct intel_iommu *i= ommu, u64 addr, unsigned long npages, bool ih) { u16 did =3D domain->iommu_did[iommu->seq_id]; + struct pasid_info *pinfo; + unsigned long pasid; =20 if (domain->default_pasid) qi_flush_piotlb(iommu, did, domain->default_pasid, @@ -1655,6 +1685,21 @@ static void domain_flush_piotlb(struct intel_iommu *= iommu, =20 if (!list_empty(&domain->devices)) qi_flush_piotlb(iommu, did, PASID_RID2PASID, addr, npages, ih); + + if (list_empty(&domain->devices) || xa_empty(&domain->pasids)) + return; + + /* + * Flush IOTLBs for all the PASIDs attached to this domain, RID2PASID + * included. + * TODO: If there are many PASIDs, we may resort to flush with + * domain ID which may have performance benefits due to fewer + * invalidation descriptors. VM exits may be reduced when running on + * vIOMMU. The current use cases utilize no more than 2 PASIDs per + * device, i.e. RID2PASID and a kernel DMA API PASID. + */ + xa_for_each(&domain->pasids, pasid, pinfo) + qi_flush_piotlb(iommu, did, pasid, addr, npages, ih); } =20 static void iommu_flush_iotlb_psi(struct intel_iommu *iommu, @@ -1902,6 +1947,7 @@ static struct dmar_domain *alloc_domain(unsigned int = type) domain->has_iotlb_device =3D false; INIT_LIST_HEAD(&domain->devices); INIT_LIST_HEAD(&domain->subdevices); + xa_init(&domain->pasids); =20 return domain; } @@ -2556,6 +2602,144 @@ static bool dev_is_real_dma_subdevice(struct device= *dev) pci_real_dma_dev(to_pci_dev(dev)) !=3D to_pci_dev(dev); } =20 + +static bool is_device_domain_attached(struct dmar_domain *dmar_domain, + struct device *dev) +{ + struct device_domain_info *info; + + list_for_each_entry(info, &dmar_domain->devices, link) { + if (info->dev =3D=3D dev) + return true; + } + + return false; +} + +static int intel_iommu_attach_dev_pasid(struct iommu_domain *domain, + struct device *dev, ioasid_t pasid) +{ + struct dmar_domain *dmar_domain =3D to_dmar_domain(domain); + struct device_domain_info *info =3D get_domain_info(dev); + struct intel_iommu *iommu =3D info->iommu; + struct pasid_info *pinfo; + unsigned long flags; + int ret =3D 0; + void *entry; + + if (!info) + return -ENODEV; + /* + * Allow attaching kernel PASIDs only after the device is already attached + * with RID2PASID, which is used for legacy DMA. + */ + if (pasid !=3D PASID_RID2PASID && !is_device_domain_attached(dmar_domain,= dev)) { + dev_err(dev, "Device not attached, must attach device before PASID!\n"); + return -ENODEV; + } + + /* + * The same PASID from the same device can only attach to this domain + * once. PASID table is per device. NULL entry is not allowed in the + * device-domain xarray. + */ + entry =3D xa_load(&info->pasids, pasid); + if (entry) { + dev_err(dev, "PASID %d already attached!\n", pasid); + return -EBUSY; + } + + spin_lock_irqsave(&device_domain_lock, flags); + spin_lock(&iommu->lock); + if (hw_pass_through && domain_type_is_si(info->domain)) + ret =3D intel_pasid_setup_pass_through(info->iommu, info->domain, + dev, pasid); + else if (domain_use_first_level(dmar_domain)) + ret =3D domain_setup_first_level(iommu, dmar_domain, dev, pasid); + else + ret =3D intel_pasid_setup_second_level(iommu, dmar_domain, dev, pasid); + spin_unlock(&iommu->lock); + spin_unlock_irqrestore(&device_domain_lock, flags); + if (ret) + return ret; + + xa_lock(&dmar_domain->pasids); + /* + * Each domain could have multiple devices attached with shared or per + * device PASIDs. At the domain level, we keep track of unique PASIDs and + * device user count. + * E.g. If a domain has two devices attached, device A has PASID 0, 1; + * device B has PASID 0, 2. Then the domain would have PASID 0, 1, 2. + */ + entry =3D xa_load(&dmar_domain->pasids, pasid); + if (entry) { + pinfo =3D entry; + } else { + pinfo =3D kzalloc(sizeof(*pinfo), GFP_ATOMIC); + if (!pinfo) + return -ENOMEM; + pinfo->pasid =3D pasid; + /* Store the new PASID info in the per domain array */ + ret =3D xa_err(__xa_store(&dmar_domain->pasids, pasid, pinfo, + GFP_ATOMIC)); + if (ret) + goto xa_store_err; + } + /* Store PASID in per device-domain array, this is for tracking devTLB */ + ret =3D xa_err(xa_store(&info->pasids, pasid, pinfo, GFP_ATOMIC)); + if (ret) + goto xa_store_err; + + atomic_inc(&pinfo->users); + xa_unlock(&dmar_domain->pasids); + + return 0; + +xa_store_err: + xa_unlock(&dmar_domain->pasids); + spin_lock_irqsave(&iommu->lock, flags); + intel_pasid_tear_down_entry(iommu, dev, pasid, false); + spin_unlock_irqrestore(&iommu->lock, flags); + + if (!atomic_read(&pinfo->users)) { + __xa_erase(&dmar_domain->pasids, pasid); + kfree(pinfo); + } + return ret; +} + +static void intel_iommu_detach_dev_pasid(struct iommu_domain *domain, + struct device *dev, ioasid_t pasid) +{ + struct dmar_domain *dmar_domain =3D to_dmar_domain(domain); + struct device_domain_info *info =3D get_domain_info(dev); + struct intel_iommu *iommu =3D info->iommu; + unsigned long flags; + struct pasid_info *pinfo; + + if (WARN_ON(!is_device_domain_attached(dmar_domain, dev))) + return; + + spin_lock_irqsave(&iommu->lock, flags); + intel_pasid_tear_down_entry(iommu, dev, pasid, false); + spin_unlock_irqrestore(&iommu->lock, flags); + + xa_lock(&dmar_domain->pasids); + pinfo =3D xa_load(&dmar_domain->pasids, pasid); + if (!pinfo) { + dev_err(dev, "PASID %d not attached!\n", pasid); + xa_unlock(&dmar_domain->pasids); + return; + } + + xa_erase(&info->pasids, pasid); + if (atomic_dec_and_test(&pinfo->users)) { + __xa_erase(&dmar_domain->pasids, pasid); + kfree(pinfo); + } + xa_unlock(&dmar_domain->pasids); +} + static struct dmar_domain *dmar_insert_one_dev_info(struct intel_iommu *io= mmu, int bus, int devfn, struct device *dev, @@ -2590,6 +2774,7 @@ static struct dmar_domain *dmar_insert_one_dev_info(s= truct intel_iommu *iommu, info->pasid_table =3D NULL; info->auxd_enabled =3D 0; INIT_LIST_HEAD(&info->subdevices); + xa_init(&info->pasids); =20 if (dev && dev_is_pci(dev)) { struct pci_dev *pdev =3D to_pci_dev(info->dev); @@ -5165,6 +5350,7 @@ static void intel_iommu_probe_finalize(struct device = *dev) iommu_setup_dma_ops(dev, 0, U64_MAX); } =20 + static void intel_iommu_get_resv_regions(struct device *device, struct list_head *head) { @@ -5491,6 +5677,8 @@ const struct iommu_ops intel_iommu_ops =3D { .enable_nesting =3D intel_iommu_enable_nesting, .attach_dev =3D intel_iommu_attach_device, .detach_dev =3D intel_iommu_detach_device, + .attach_dev_pasid =3D intel_iommu_attach_dev_pasid, + .detach_dev_pasid =3D intel_iommu_detach_dev_pasid, .aux_attach_dev =3D intel_iommu_aux_attach_device, .aux_detach_dev =3D intel_iommu_aux_detach_device, .aux_get_pasid =3D intel_iommu_aux_get_pasid, diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h index 82955524fad8..3f4c98f170ec 100644 --- a/include/linux/intel-iommu.h +++ b/include/linux/intel-iommu.h @@ -567,7 +567,7 @@ struct dmar_domain { * The default pasid used for non-SVM * traffic on mediated devices. */ - + struct xarray pasids; /* Tracks the PASIDs attached to this domain */ struct iommu_domain domain; /* generic domain data structure for iommu core */ }; @@ -628,6 +628,15 @@ struct subdev_domain_info { int users; /* user count */ }; =20 +struct pasid_info { + struct device_domain_info *info; + ioasid_t pasid; + atomic_t users; /* Count the number of devices attached + * with the PASID + */ + u32 flags; /* permission and other attributes */ +}; + /* PCI domain-device relationship */ struct device_domain_info { struct list_head link; /* link to domain siblings */ @@ -650,6 +659,7 @@ struct device_domain_info { struct intel_iommu *iommu; /* IOMMU used by this device */ struct dmar_domain *domain; /* pointer to domain */ struct pasid_table *pasid_table; /* pasid table */ + struct xarray pasids; /* PASIDs attached to this domain-device */ }; =20 static inline void __iommu_flush_cache( --=20 2.25.1 From nobody Mon Jun 22 23:53:34 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 21337C433EF for ; Tue, 15 Mar 2022 05:11:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344867AbiCOFMk (ORCPT ); Tue, 15 Mar 2022 01:12:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48054 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1344841AbiCOFMT (ORCPT ); Tue, 15 Mar 2022 01:12:19 -0400 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BE36A49253 for ; Mon, 14 Mar 2022 22:11:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1647321067; x=1678857067; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=PtNrYq5SgtOh4S5p2eFwtPOoZIVbpRJnL3OhUjSFkCY=; b=Mo1Lr7yD0yFf6gtoIdn2x0RT+Y94tfmQaHISTDPk+sm2cI7NKYJ5rli3 8IjQi7VPk7vewlIoiRoK8Y9MuUhZpju3hIHNAdWJG6cDUpTbkZckqP7tn cTLF2dCyPfwKXMs8/bNIpddUK9hiZEXdg0edfRyGQWptMpcLx3SZXWlVp q5EK6t6QpD/3fRnEJv5NnXqfLZzoQq/h7wvIALwA7mmDC4swthco1mylK WWxo6JDGVKV8Z42eUuic3TgRZgVlrjgM/0jPg/9rL81m+eVjV6yUM5JPE 5T3I/N6XaySPnvSF4X8PzeI0JXWcDfrypcyhxvqA5/+bS5EkxtqanU+fV Q==; X-IronPort-AV: E=McAfee;i="6200,9189,10286"; a="236159102" X-IronPort-AV: E=Sophos;i="5.90,182,1643702400"; d="scan'208";a="236159102" Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Mar 2022 22:03:59 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.90,182,1643702400"; d="scan'208";a="580383579" Received: from otc-wp-03.jf.intel.com (HELO jacob-builder.jf.intel.com) ([10.54.39.79]) by orsmga001.jf.intel.com with ESMTP; 14 Mar 2022 22:03:58 -0700 From: Jacob Pan To: iommu@lists.linux-foundation.org, LKML , Joerg Roedel , Jason Gunthorpe , "Christoph Hellwig" , "Lu Baolu" , Jean-Philippe Brucker Cc: Jacob Pan , Raj Ashok , "Kumar, Sanjay K" , Dave Jiang , Tony Luck , "Zanussi, Tom" , Dan Williams , "Tian, Kevin" , Yi Liu Subject: [PATCH v2 4/8] iommu/vt-d: Use device_pasid attach op for RID2PASID Date: Mon, 14 Mar 2022 22:07:08 -0700 Message-Id: <20220315050713.2000518-5-jacob.jun.pan@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220315050713.2000518-1-jacob.jun.pan@linux.intel.com> References: <20220315050713.2000518-1-jacob.jun.pan@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" With the availability of a generic device-PASID-domain attachment API, there's no need to special case RID2PASID. Use the API to replace duplicated code. Signed-off-by: Jacob Pan --- drivers/iommu/intel/iommu.c | 18 ++---------------- 1 file changed, 2 insertions(+), 16 deletions(-) diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c index 9267194eaed3..f832b7599d21 100644 --- a/drivers/iommu/intel/iommu.c +++ b/drivers/iommu/intel/iommu.c @@ -1683,9 +1683,6 @@ static void domain_flush_piotlb(struct intel_iommu *i= ommu, qi_flush_piotlb(iommu, did, domain->default_pasid, addr, npages, ih); =20 - if (!list_empty(&domain->devices)) - qi_flush_piotlb(iommu, did, PASID_RID2PASID, addr, npages, ih); - if (list_empty(&domain->devices) || xa_empty(&domain->pasids)) return; =20 @@ -2826,17 +2823,7 @@ static struct dmar_domain *dmar_insert_one_dev_info(= struct intel_iommu *iommu, } =20 /* Setup the PASID entry for requests without PASID: */ - spin_lock_irqsave(&iommu->lock, flags); - if (hw_pass_through && domain_type_is_si(domain)) - ret =3D intel_pasid_setup_pass_through(iommu, domain, - dev, PASID_RID2PASID); - else if (domain_use_first_level(domain)) - ret =3D domain_setup_first_level(iommu, domain, dev, - PASID_RID2PASID); - else - ret =3D intel_pasid_setup_second_level(iommu, domain, - dev, PASID_RID2PASID); - spin_unlock_irqrestore(&iommu->lock, flags); + ret =3D intel_iommu_attach_dev_pasid(&domain->domain, dev, PASID_RID2PAS= ID); if (ret) { dev_err(dev, "Setup RID2PASID failed\n"); dmar_remove_one_dev_info(dev); @@ -4618,8 +4605,7 @@ static void __dmar_remove_one_dev_info(struct device_= domain_info *info) =20 if (info->dev && !dev_is_real_dma_subdevice(info->dev)) { if (dev_is_pci(info->dev) && sm_supported(iommu)) - intel_pasid_tear_down_entry(iommu, info->dev, - PASID_RID2PASID, false); + intel_iommu_detach_dev_pasid(&domain->domain, info->dev, PASID_RID2PASI= D); =20 iommu_disable_dev_iotlb(info); domain_context_clear(info); --=20 2.25.1 From nobody Mon Jun 22 23:53:34 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 762EDC433EF for ; Tue, 15 Mar 2022 05:11:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344900AbiCOFMp (ORCPT ); Tue, 15 Mar 2022 01:12:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48070 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1344843AbiCOFMT (ORCPT ); Tue, 15 Mar 2022 01:12:19 -0400 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7E5C349254 for ; Mon, 14 Mar 2022 22:11:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1647321068; x=1678857068; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Rk+QUCnbRB21mlAbYWkm6ssZeIW/wunY1IQvr0sPRwQ=; b=lyemL0cGDkiWesUiC9p0k5NrPG1zaBe4PG04W+4rIdWbJ7ZZGX3idI/f 7hWLpq2FpABrk3H2/k2s3DIV2CM0qYUgDJzroLvOM29mnjomrRfAwy5sx BZNwJVc/BlMhWWh7tY5LLH1++1DwNfqV1UHUiuZnmpt/Ip4PTWydWwOaf 6l6BGyhOhYBr4pdJGj1c3tT46mGOWEmnnZmeGqer3n2MwDPiokE+V4t0N xaOWKIHFDfNN+VrHGmyk3ShoTOQNq40lVLL2TN9tpzYVo3nxOW72llE95 LNfle3zC/yJflndmVkM0Q6ym/tAvKiVEfMi6e0rTceKMX42lhdqsd3mKC g==; X-IronPort-AV: E=McAfee;i="6200,9189,10286"; a="236159104" X-IronPort-AV: E=Sophos;i="5.90,182,1643702400"; d="scan'208";a="236159104" Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Mar 2022 22:03:59 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.90,182,1643702400"; d="scan'208";a="580383584" Received: from otc-wp-03.jf.intel.com (HELO jacob-builder.jf.intel.com) ([10.54.39.79]) by orsmga001.jf.intel.com with ESMTP; 14 Mar 2022 22:03:59 -0700 From: Jacob Pan To: iommu@lists.linux-foundation.org, LKML , Joerg Roedel , Jason Gunthorpe , "Christoph Hellwig" , "Lu Baolu" , Jean-Philippe Brucker Cc: Jacob Pan , Raj Ashok , "Kumar, Sanjay K" , Dave Jiang , Tony Luck , "Zanussi, Tom" , Dan Williams , "Tian, Kevin" , Yi Liu Subject: [PATCH v2 5/8] iommu: Add PASID support for DMA mapping API users Date: Mon, 14 Mar 2022 22:07:09 -0700 Message-Id: <20220315050713.2000518-6-jacob.jun.pan@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220315050713.2000518-1-jacob.jun.pan@linux.intel.com> References: <20220315050713.2000518-1-jacob.jun.pan@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" DMA mapping API is the de facto standard for in-kernel DMA. It operates on a per device/RID basis which is not PASID-aware. Some modern devices such as Intel Data Streaming Accelerator, PASID is required for certain work submissions. To allow such devices use DMA mapping API, we need the following functionalities: 1. Provide device a way to retrieve a PASID for work submission within the kernel 2. Enable the kernel PASID on the IOMMU for the device 3. Attach the kernel PASID to the device's default DMA domain, let it be IOVA or physical address in case of pass-through. This patch introduces a driver facing API that enables DMA API PASID usage. Once enabled, device drivers can continue to use DMA APIs as is. There is no difference in dma_handle between without PASID and with PASID. Signed-off-by: Jacob Pan --- drivers/iommu/dma-iommu.c | 65 +++++++++++++++++++++++++++++++++++++++ include/linux/dma-iommu.h | 7 +++++ include/linux/iommu.h | 9 ++++++ 3 files changed, 81 insertions(+) diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c index b22034975301..d0ff1a34b1b6 100644 --- a/drivers/iommu/dma-iommu.c +++ b/drivers/iommu/dma-iommu.c @@ -39,6 +39,8 @@ enum iommu_dma_cookie_type { IOMMU_DMA_MSI_COOKIE, }; =20 +static DECLARE_IOASID_SET(iommu_dma_pasid); + struct iommu_dma_cookie { enum iommu_dma_cookie_type type; union { @@ -370,6 +372,69 @@ void iommu_put_dma_cookie(struct iommu_domain *domain) domain->iova_cookie =3D NULL; } =20 +/** + * iommu_enable_pasid_dma --Enable in-kernel DMA request with PASID + * @dev: Device to be enabled + * + * DMA request with PASID will be mapped the same way as the legacy DMA. + * If the device is in pass-through, PASID will also pass-through. If the + * device is in IOVA map, the supervisor PASID will point to the same IOVA + * page table. + * + * @return the kernel PASID to be used for DMA or INVALID_IOASID on failure + */ +int iommu_enable_pasid_dma(struct device *dev, ioasid_t *pasid) +{ + struct iommu_domain *dom; + ioasid_t id, max; + int ret; + + dom =3D iommu_get_domain_for_dev(dev); + if (!dom || !dom->ops || !dom->ops->attach_dev_pasid) + return -ENODEV; + max =3D iommu_get_dev_pasid_max(dev); + if (!max) + return -EINVAL; + + id =3D ioasid_alloc(&iommu_dma_pasid, 1, max, dev); + if (id =3D=3D INVALID_IOASID) + return -ENOMEM; + + ret =3D dom->ops->attach_dev_pasid(dom, dev, id); + if (ret) { + ioasid_put(id); + return ret; + } + *pasid =3D id; + + return ret; +} +EXPORT_SYMBOL(iommu_enable_pasid_dma); + +/** + * iommu_disable_pasid_dma --Disable in-kernel DMA request with PASID + * @dev: Device's PASID DMA to be disabled + * + * It is the device driver's responsibility to ensure no more incoming DMA + * requests with the kernel PASID before calling this function. IOMMU driv= er + * ensures PASID cache, IOTLBs related to the kernel PASID are cleared and + * drained. + * + * @return 0 on success or error code on failure + */ +void iommu_disable_pasid_dma(struct device *dev, ioasid_t pasid) +{ + struct iommu_domain *dom; + + /* TODO: check the given PASID is within the ioasid_set */ + dom =3D iommu_get_domain_for_dev(dev); + if (!dom->ops->detach_dev_pasid) + return; + dom->ops->detach_dev_pasid(dom, dev, pasid); + ioasid_put(pasid); +} +EXPORT_SYMBOL(iommu_disable_pasid_dma); + /** * iommu_dma_get_resv_regions - Reserved region driver helper * @dev: Device from iommu_get_resv_regions() diff --git a/include/linux/dma-iommu.h b/include/linux/dma-iommu.h index 24607dc3c2ac..e6cb9b52a420 100644 --- a/include/linux/dma-iommu.h +++ b/include/linux/dma-iommu.h @@ -18,6 +18,13 @@ int iommu_get_dma_cookie(struct iommu_domain *domain); int iommu_get_msi_cookie(struct iommu_domain *domain, dma_addr_t base); void iommu_put_dma_cookie(struct iommu_domain *domain); =20 +/* + * For devices that can do DMA request with PASID, setup a system PASID. + * Address modes (IOVA, PA) are selected by the platform code. + */ +int iommu_enable_pasid_dma(struct device *dev, ioasid_t *pasid); +void iommu_disable_pasid_dma(struct device *dev, ioasid_t pasid); + /* Setup call for arch DMA mapping code */ void iommu_setup_dma_ops(struct device *dev, u64 dma_base, u64 dma_limit); int iommu_dma_init_fq(struct iommu_domain *domain); diff --git a/include/linux/iommu.h b/include/linux/iommu.h index fde5b933dbe3..fb011722e4f8 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -395,6 +395,15 @@ static inline void iommu_set_dev_pasid_max(struct devi= ce *dev, =20 param->pasid_max =3D max; } +static inline ioasid_t iommu_get_dev_pasid_max(struct device *dev) +{ + struct dev_iommu *param =3D dev->iommu; + + if (WARN_ON(!param)) + return 0; + + return param->pasid_max; +} =20 int iommu_device_register(struct iommu_device *iommu, const struct iommu_ops *ops, --=20 2.25.1 From nobody Mon Jun 22 23:53:34 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E791FC433FE for ; Tue, 15 Mar 2022 05:12:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344988AbiCOFND (ORCPT ); Tue, 15 Mar 2022 01:13:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48082 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1344846AbiCOFMU (ORCPT ); Tue, 15 Mar 2022 01:12:20 -0400 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CF2C149245 for ; Mon, 14 Mar 2022 22:11:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1647321068; x=1678857068; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=QAEt5X446gvrSnh6YavkULr75aEPiIM4uDOUVyXV88A=; b=cD/C8LREAj9JCjoHE0DrsCLT3KXjFqLToZ/uVL110BoSLhJcxYHdMD4w mL00SDKseK7oakqsMgk3aunS68MbI6pYBV5Zn6zYMk00Dt4nrisUT9Rqj Zb87OLy512Chl8echEGHTX/GqoHnDJXeJjtHoVzsGSt3ej/S4za1vumX8 6xyYts5qTCoO+snqUqjFJyjgqE6v/wVETjTbZ592c9jDe+sSgSa0xdW8Q xXoHvNDCi0piJDJd1ytgLxtFgjIbTu3zdik3RHpsMBj6w5hBzoieHwuD7 7m2T4gdDDQYxLzmq11P6Cyb2xQd90pee/k88xtDri2yBp27NlEmJmddDE w==; X-IronPort-AV: E=McAfee;i="6200,9189,10286"; a="236159105" X-IronPort-AV: E=Sophos;i="5.90,182,1643702400"; d="scan'208";a="236159105" Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Mar 2022 22:03:59 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.90,182,1643702400"; d="scan'208";a="580383587" Received: from otc-wp-03.jf.intel.com (HELO jacob-builder.jf.intel.com) ([10.54.39.79]) by orsmga001.jf.intel.com with ESMTP; 14 Mar 2022 22:03:59 -0700 From: Jacob Pan To: iommu@lists.linux-foundation.org, LKML , Joerg Roedel , Jason Gunthorpe , "Christoph Hellwig" , "Lu Baolu" , Jean-Philippe Brucker Cc: Jacob Pan , Raj Ashok , "Kumar, Sanjay K" , Dave Jiang , Tony Luck , "Zanussi, Tom" , Dan Williams , "Tian, Kevin" , Yi Liu Subject: [PATCH v2 6/8] dmaengine: idxd: Use DMA API for in-kernel DMA with PASID Date: Mon, 14 Mar 2022 22:07:10 -0700 Message-Id: <20220315050713.2000518-7-jacob.jun.pan@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220315050713.2000518-1-jacob.jun.pan@linux.intel.com> References: <20220315050713.2000518-1-jacob.jun.pan@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The current in-kernel supervisor PASID support is based on the SVM/SVA machinery in SVA lib. The binding between a kernel PASID and kernel mapping has many flaws. See discussions in the link below. This patch enables in-kernel DMA by switching from SVA lib to the standard DMA mapping APIs. Since both DMA requests with and without PASIDs are mapped identically, there is no change to how DMA APIs are used after the kernel PASID is enabled. Link: https://lore.kernel.org/linux-iommu/20210511194726.GP1002214@nvidia.c= om/ Signed-off-by: Jacob Pan --- drivers/dma/idxd/idxd.h | 1 - drivers/dma/idxd/init.c | 34 +++++++++------------------------- drivers/dma/idxd/sysfs.c | 7 ------- 3 files changed, 9 insertions(+), 33 deletions(-) diff --git a/drivers/dma/idxd/idxd.h b/drivers/dma/idxd/idxd.h index da72eb15f610..a09ab4a6e1c1 100644 --- a/drivers/dma/idxd/idxd.h +++ b/drivers/dma/idxd/idxd.h @@ -276,7 +276,6 @@ struct idxd_device { struct idxd_wq **wqs; struct idxd_engine **engines; =20 - struct iommu_sva *sva; unsigned int pasid; =20 int num_groups; diff --git a/drivers/dma/idxd/init.c b/drivers/dma/idxd/init.c index 08a5f4310188..5d1f8dd4abf6 100644 --- a/drivers/dma/idxd/init.c +++ b/drivers/dma/idxd/init.c @@ -16,6 +16,7 @@ #include #include #include +#include #include #include #include "../dmaengine.h" @@ -466,36 +467,22 @@ static struct idxd_device *idxd_alloc(struct pci_dev = *pdev, struct idxd_driver_d =20 static int idxd_enable_system_pasid(struct idxd_device *idxd) { - int flags; - unsigned int pasid; - struct iommu_sva *sva; + u32 pasid; + int ret; =20 - flags =3D SVM_FLAG_SUPERVISOR_MODE; - - sva =3D iommu_sva_bind_device(&idxd->pdev->dev, NULL, &flags); - if (IS_ERR(sva)) { - dev_warn(&idxd->pdev->dev, - "iommu sva bind failed: %ld\n", PTR_ERR(sva)); - return PTR_ERR(sva); - } - - pasid =3D iommu_sva_get_pasid(sva); - if (pasid =3D=3D IOMMU_PASID_INVALID) { - iommu_sva_unbind_device(sva); - return -ENODEV; + ret =3D iommu_enable_pasid_dma(&idxd->pdev->dev, &pasid); + if (ret) { + dev_err(&idxd->pdev->dev, "No DMA PASID %d\n", ret); + return ret; } - - idxd->sva =3D sva; idxd->pasid =3D pasid; - dev_dbg(&idxd->pdev->dev, "system pasid: %u\n", pasid); + return 0; } =20 static void idxd_disable_system_pasid(struct idxd_device *idxd) { - - iommu_sva_unbind_device(idxd->sva); - idxd->sva =3D NULL; + iommu_disable_pasid_dma(&idxd->pdev->dev, idxd->pasid); } =20 static int idxd_probe(struct idxd_device *idxd) @@ -524,10 +511,7 @@ static int idxd_probe(struct idxd_device *idxd) } else { dev_warn(dev, "Unable to turn on SVA feature.\n"); } - } else if (!sva) { - dev_warn(dev, "User forced SVA off via module param.\n"); } - idxd_read_caps(idxd); idxd_read_table_offsets(idxd); =20 diff --git a/drivers/dma/idxd/sysfs.c b/drivers/dma/idxd/sysfs.c index 7e19ab92b61a..fde6656695ba 100644 --- a/drivers/dma/idxd/sysfs.c +++ b/drivers/dma/idxd/sysfs.c @@ -839,13 +839,6 @@ static ssize_t wq_name_store(struct device *dev, if (strlen(buf) > WQ_NAME_SIZE || strlen(buf) =3D=3D 0) return -EINVAL; =20 - /* - * This is temporarily placed here until we have SVM support for - * dmaengine. - */ - if (wq->type =3D=3D IDXD_WQT_KERNEL && device_pasid_enabled(wq->idxd)) - return -EOPNOTSUPP; - memset(wq->name, 0, WQ_NAME_SIZE + 1); strncpy(wq->name, buf, WQ_NAME_SIZE); strreplace(wq->name, '\n', '\0'); --=20 2.25.1 From nobody Mon Jun 22 23:53:34 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 04455C433EF for ; Tue, 15 Mar 2022 05:12:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1345019AbiCOFNI (ORCPT ); Tue, 15 Mar 2022 01:13:08 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48096 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1344848AbiCOFMU (ORCPT ); Tue, 15 Mar 2022 01:12:20 -0400 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8879749249 for ; Mon, 14 Mar 2022 22:11:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1647321069; x=1678857069; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=6bPP85I/X2JN+a+CF0RYCDiAmljsVzkxdKPwz9WA20I=; b=lC5d9QQvyIDKpkS1oew0pMOQwQl5BS2eJg+Hk6wScABUpV5HWII4G8FH 7+lSf4pWdarjAFGFx5duayzLd8kmuSkjgvTk9yDU08jW5GIrR1YMPpKmD Mc9FvB6PqOHjlrjNuQWLQqjbH4JHgchsGlMuyWQ0ia7TFoj2vLZjkXvfx MBSiufVsCoReLUbR8fz9/69CI3G9jnX63T4wEm/i3VKNsLH/QKZNF9M4Q jNXnwrP2lA5lCJHnaw4T4FSlpXzNUYV9y1BEqepnOH1dwLwM84qBPVPWM 0lC+8JNXxEEqyLj+NnxGebS1umCM1ryOe1Fwut1e6HJeJkCrh+3SDcmtA Q==; X-IronPort-AV: E=McAfee;i="6200,9189,10286"; a="236159107" X-IronPort-AV: E=Sophos;i="5.90,182,1643702400"; d="scan'208";a="236159107" Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Mar 2022 22:03:59 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.90,182,1643702400"; d="scan'208";a="580383590" Received: from otc-wp-03.jf.intel.com (HELO jacob-builder.jf.intel.com) ([10.54.39.79]) by orsmga001.jf.intel.com with ESMTP; 14 Mar 2022 22:03:59 -0700 From: Jacob Pan To: iommu@lists.linux-foundation.org, LKML , Joerg Roedel , Jason Gunthorpe , "Christoph Hellwig" , "Lu Baolu" , Jean-Philippe Brucker Cc: Jacob Pan , Raj Ashok , "Kumar, Sanjay K" , Dave Jiang , Tony Luck , "Zanussi, Tom" , Dan Williams , "Tian, Kevin" , Yi Liu Subject: [PATCH v2 7/8] iommu/vt-d: Delete supervisor/kernel SVA Date: Mon, 14 Mar 2022 22:07:11 -0700 Message-Id: <20220315050713.2000518-8-jacob.jun.pan@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220315050713.2000518-1-jacob.jun.pan@linux.intel.com> References: <20220315050713.2000518-1-jacob.jun.pan@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" In-kernel DMA with PASID should use DMA API now, remove supervisor PASID SVA support. Remove special cases in bind mm and page request service. Signed-off-by: Jacob Pan --- drivers/iommu/intel/svm.c | 42 ++++++++------------------------------- 1 file changed, 8 insertions(+), 34 deletions(-) diff --git a/drivers/iommu/intel/svm.c b/drivers/iommu/intel/svm.c index 2c53689da461..37d6218f173b 100644 --- a/drivers/iommu/intel/svm.c +++ b/drivers/iommu/intel/svm.c @@ -516,11 +516,10 @@ static void intel_svm_free_pasid(struct mm_struct *mm) =20 static struct iommu_sva *intel_svm_bind_mm(struct intel_iommu *iommu, struct device *dev, - struct mm_struct *mm, - unsigned int flags) + struct mm_struct *mm) { struct device_domain_info *info =3D get_domain_info(dev); - unsigned long iflags, sflags; + unsigned long iflags, sflags =3D 0; struct intel_svm_dev *sdev; struct intel_svm *svm; int ret =3D 0; @@ -533,16 +532,13 @@ static struct iommu_sva *intel_svm_bind_mm(struct int= el_iommu *iommu, =20 svm->pasid =3D mm->pasid; svm->mm =3D mm; - svm->flags =3D flags; INIT_LIST_HEAD_RCU(&svm->devs); =20 - if (!(flags & SVM_FLAG_SUPERVISOR_MODE)) { - svm->notifier.ops =3D &intel_mmuops; - ret =3D mmu_notifier_register(&svm->notifier, mm); - if (ret) { - kfree(svm); - return ERR_PTR(ret); - } + svm->notifier.ops =3D &intel_mmuops; + ret =3D mmu_notifier_register(&svm->notifier, mm); + if (ret) { + kfree(svm); + return ERR_PTR(ret); } =20 ret =3D pasid_private_add(svm->pasid, svm); @@ -583,8 +579,6 @@ static struct iommu_sva *intel_svm_bind_mm(struct intel= _iommu *iommu, } =20 /* Setup the pasid table: */ - sflags =3D (flags & SVM_FLAG_SUPERVISOR_MODE) ? - PASID_FLAG_SUPERVISOR_MODE : 0; sflags |=3D cpu_feature_enabled(X86_FEATURE_LA57) ? PASID_FLAG_FL5LP : 0; spin_lock_irqsave(&iommu->lock, iflags); ret =3D intel_pasid_setup_first_level(iommu, dev, mm->pgd, mm->pasid, @@ -957,7 +951,7 @@ static irqreturn_t prq_event_thread(int irq, void *d) * to unbind the mm while any page faults are outstanding. */ svm =3D pasid_private_find(req->pasid); - if (IS_ERR_OR_NULL(svm) || (svm->flags & SVM_FLAG_SUPERVISOR_MODE)) + if (IS_ERR_OR_NULL(svm)) goto bad_req; } =20 @@ -1011,29 +1005,9 @@ static irqreturn_t prq_event_thread(int irq, void *d) struct iommu_sva *intel_svm_bind(struct device *dev, struct mm_struct *mm,= void *drvdata) { struct intel_iommu *iommu =3D device_to_iommu(dev, NULL, NULL); - unsigned int flags =3D 0; struct iommu_sva *sva; int ret; =20 - if (drvdata) - flags =3D *(unsigned int *)drvdata; - - if (flags & SVM_FLAG_SUPERVISOR_MODE) { - if (!ecap_srs(iommu->ecap)) { - dev_err(dev, "%s: Supervisor PASID not supported\n", - iommu->name); - return ERR_PTR(-EOPNOTSUPP); - } - - if (mm) { - dev_err(dev, "%s: Supervisor PASID with user provided mm\n", - iommu->name); - return ERR_PTR(-EINVAL); - } - - mm =3D &init_mm; - } - mutex_lock(&pasid_mutex); ret =3D intel_svm_alloc_pasid(dev, mm, flags); if (ret) { --=20 2.25.1 From nobody Mon Jun 22 23:53:34 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 83281C433F5 for ; Tue, 15 Mar 2022 05:12:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344912AbiCOFMu (ORCPT ); Tue, 15 Mar 2022 01:12:50 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48080 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1344845AbiCOFMU (ORCPT ); Tue, 15 Mar 2022 01:12:20 -0400 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CF1B149240 for ; Mon, 14 Mar 2022 22:11:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1647321068; x=1678857068; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=A38Y0t7PfbeOjDPwxZIh5wdCYYvnUSyCF8g2Op2aXRY=; b=ifXHBuiDQnzZnGwtooP5zzIPgbswZ1vUy1cGg849n2zhEYyPnvGYRQsr LR7xlr0XQzFCdbBTelC/w7HbzoImsx297aTbQfXG2e74wt12zIDrQLE2q bGkC4xPJoWcobgVK/LCuJZwZFDkr8/+RsLCOVKdQnWDTqIMT1Za7OBqLz AjU3HYLm4uUtIvSvXSbCWmNsmi36UB4mR43UtKjNSsr1U/L0ZlFX0VZhL cH4W3iIjy7dcLBREtvQ+A783njVNf+zVQpwAaQIl2/vVDY0SlHGipAVDe gi8GFjTte4M4plhTTHs2+FQdXVZMDWxpGRnZk4+7XaaYALlIlQ25VqMmR g==; X-IronPort-AV: E=McAfee;i="6200,9189,10286"; a="236159108" X-IronPort-AV: E=Sophos;i="5.90,182,1643702400"; d="scan'208";a="236159108" Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Mar 2022 22:04:00 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.90,182,1643702400"; d="scan'208";a="580383593" Received: from otc-wp-03.jf.intel.com (HELO jacob-builder.jf.intel.com) ([10.54.39.79]) by orsmga001.jf.intel.com with ESMTP; 14 Mar 2022 22:03:59 -0700 From: Jacob Pan To: iommu@lists.linux-foundation.org, LKML , Joerg Roedel , Jason Gunthorpe , "Christoph Hellwig" , "Lu Baolu" , Jean-Philippe Brucker Cc: Jacob Pan , Raj Ashok , "Kumar, Sanjay K" , Dave Jiang , Tony Luck , "Zanussi, Tom" , Dan Williams , "Tian, Kevin" , Yi Liu Subject: [PATCH v2 8/8] iommu: Remove unused driver data in sva_bind_device Date: Mon, 14 Mar 2022 22:07:12 -0700 Message-Id: <20220315050713.2000518-9-jacob.jun.pan@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220315050713.2000518-1-jacob.jun.pan@linux.intel.com> References: <20220315050713.2000518-1-jacob.jun.pan@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" No one is using drvdata for sva_bind_device after kernel SVA support is removed from VT-d driver. Remove the drvdata parameter as well. Signed-off-by: Jacob Pan Reviewed-by: Jean-Philippe Brucker --- drivers/dma/idxd/cdev.c | 2 +- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 2 +- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 5 ++--- drivers/iommu/intel/svm.c | 9 ++++----- drivers/iommu/iommu.c | 4 ++-- drivers/misc/uacce/uacce.c | 2 +- include/linux/intel-iommu.h | 3 +-- include/linux/iommu.h | 9 +++------ 8 files changed, 15 insertions(+), 21 deletions(-) diff --git a/drivers/dma/idxd/cdev.c b/drivers/dma/idxd/cdev.c index b9b2b4a4124e..312ec37ebf91 100644 --- a/drivers/dma/idxd/cdev.c +++ b/drivers/dma/idxd/cdev.c @@ -100,7 +100,7 @@ static int idxd_cdev_open(struct inode *inode, struct f= ile *filp) filp->private_data =3D ctx; =20 if (device_pasid_enabled(idxd)) { - sva =3D iommu_sva_bind_device(dev, current->mm, NULL); + sva =3D iommu_sva_bind_device(dev, current->mm); if (IS_ERR(sva)) { rc =3D PTR_ERR(sva); dev_err(dev, "pasid allocation failed: %d\n", rc); diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iomm= u/arm/arm-smmu-v3/arm-smmu-v3-sva.c index a737ba5f727e..eb2f5cb0701a 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -354,7 +354,7 @@ __arm_smmu_sva_bind(struct device *dev, struct mm_struc= t *mm) } =20 struct iommu_sva * -arm_smmu_sva_bind(struct device *dev, struct mm_struct *mm, void *drvdata) +arm_smmu_sva_bind(struct device *dev, struct mm_struct *mm) { struct iommu_sva *handle; struct iommu_domain *domain =3D iommu_get_domain_for_dev(dev); diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/ar= m/arm-smmu-v3/arm-smmu-v3.h index cd48590ada30..d2ba86470c42 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -754,8 +754,7 @@ bool arm_smmu_master_sva_enabled(struct arm_smmu_master= *master); int arm_smmu_master_enable_sva(struct arm_smmu_master *master); int arm_smmu_master_disable_sva(struct arm_smmu_master *master); bool arm_smmu_master_iopf_supported(struct arm_smmu_master *master); -struct iommu_sva *arm_smmu_sva_bind(struct device *dev, struct mm_struct *= mm, - void *drvdata); +struct iommu_sva *arm_smmu_sva_bind(struct device *dev, struct mm_struct *= mm); void arm_smmu_sva_unbind(struct iommu_sva *handle); u32 arm_smmu_sva_get_pasid(struct iommu_sva *handle); void arm_smmu_sva_notifier_synchronize(void); @@ -791,7 +790,7 @@ static inline bool arm_smmu_master_iopf_supported(struc= t arm_smmu_master *master } =20 static inline struct iommu_sva * -arm_smmu_sva_bind(struct device *dev, struct mm_struct *mm, void *drvdata) +arm_smmu_sva_bind(struct device *dev, struct mm_struct *mm) { return ERR_PTR(-ENODEV); } diff --git a/drivers/iommu/intel/svm.c b/drivers/iommu/intel/svm.c index 37d6218f173b..94deb58375f5 100644 --- a/drivers/iommu/intel/svm.c +++ b/drivers/iommu/intel/svm.c @@ -500,8 +500,7 @@ int intel_svm_unbind_gpasid(struct device *dev, u32 pas= id) return ret; } =20 -static int intel_svm_alloc_pasid(struct device *dev, struct mm_struct *mm, - unsigned int flags) +static int intel_svm_alloc_pasid(struct device *dev, struct mm_struct *mm) { ioasid_t max_pasid =3D dev_is_pci(dev) ? pci_max_pasids(to_pci_dev(dev)) : intel_pasid_max_id; @@ -1002,20 +1001,20 @@ static irqreturn_t prq_event_thread(int irq, void *= d) return IRQ_RETVAL(handled); } =20 -struct iommu_sva *intel_svm_bind(struct device *dev, struct mm_struct *mm,= void *drvdata) +struct iommu_sva *intel_svm_bind(struct device *dev, struct mm_struct *mm) { struct intel_iommu *iommu =3D device_to_iommu(dev, NULL, NULL); struct iommu_sva *sva; int ret; =20 mutex_lock(&pasid_mutex); - ret =3D intel_svm_alloc_pasid(dev, mm, flags); + ret =3D intel_svm_alloc_pasid(dev, mm); if (ret) { mutex_unlock(&pasid_mutex); return ERR_PTR(ret); } =20 - sva =3D intel_svm_bind_mm(iommu, dev, mm, flags); + sva =3D intel_svm_bind_mm(iommu, dev, mm); if (IS_ERR_OR_NULL(sva)) intel_svm_free_pasid(mm); mutex_unlock(&pasid_mutex); diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c index 107dcf5938d6..fef34879bc0c 100644 --- a/drivers/iommu/iommu.c +++ b/drivers/iommu/iommu.c @@ -3049,7 +3049,7 @@ EXPORT_SYMBOL_GPL(iommu_aux_get_pasid); * On error, returns an ERR_PTR value. */ struct iommu_sva * -iommu_sva_bind_device(struct device *dev, struct mm_struct *mm, void *drvd= ata) +iommu_sva_bind_device(struct device *dev, struct mm_struct *mm) { struct iommu_group *group; struct iommu_sva *handle =3D ERR_PTR(-EINVAL); @@ -3074,7 +3074,7 @@ iommu_sva_bind_device(struct device *dev, struct mm_s= truct *mm, void *drvdata) if (iommu_group_device_count(group) !=3D 1) goto out_unlock; =20 - handle =3D ops->sva_bind(dev, mm, drvdata); + handle =3D ops->sva_bind(dev, mm); =20 out_unlock: mutex_unlock(&group->mutex); diff --git a/drivers/misc/uacce/uacce.c b/drivers/misc/uacce/uacce.c index 281c54003edc..3238a867ea51 100644 --- a/drivers/misc/uacce/uacce.c +++ b/drivers/misc/uacce/uacce.c @@ -99,7 +99,7 @@ static int uacce_bind_queue(struct uacce_device *uacce, s= truct uacce_queue *q) if (!(uacce->flags & UACCE_DEV_SVA)) return 0; =20 - handle =3D iommu_sva_bind_device(uacce->parent, current->mm, NULL); + handle =3D iommu_sva_bind_device(uacce->parent, current->mm); if (IS_ERR(handle)) return PTR_ERR(handle); =20 diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h index 3f4c98f170ec..9dc855d7479d 100644 --- a/include/linux/intel-iommu.h +++ b/include/linux/intel-iommu.h @@ -777,8 +777,7 @@ extern int intel_svm_finish_prq(struct intel_iommu *iom= mu); int intel_svm_bind_gpasid(struct iommu_domain *domain, struct device *dev, struct iommu_gpasid_bind_data *data); int intel_svm_unbind_gpasid(struct device *dev, u32 pasid); -struct iommu_sva *intel_svm_bind(struct device *dev, struct mm_struct *mm, - void *drvdata); +struct iommu_sva *intel_svm_bind(struct device *dev, struct mm_struct *mm); void intel_svm_unbind(struct iommu_sva *handle); u32 intel_svm_get_pasid(struct iommu_sva *handle); int intel_svm_page_response(struct device *dev, struct iommu_fault_event *= evt, diff --git a/include/linux/iommu.h b/include/linux/iommu.h index fb011722e4f8..b570b37181ad 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -294,9 +294,7 @@ struct iommu_ops { int (*aux_attach_dev)(struct iommu_domain *domain, struct device *dev); void (*aux_detach_dev)(struct iommu_domain *domain, struct device *dev); int (*aux_get_pasid)(struct iommu_domain *domain, struct device *dev); - - struct iommu_sva *(*sva_bind)(struct device *dev, struct mm_struct *mm, - void *drvdata); + struct iommu_sva *(*sva_bind)(struct device *dev, struct mm_struct *mm); void (*sva_unbind)(struct iommu_sva *handle); int (*attach_dev_pasid)(struct iommu_domain *domain, struct device *dev, ioasid_t id); @@ -705,8 +703,7 @@ void iommu_aux_detach_device(struct iommu_domain *domai= n, struct device *dev); int iommu_aux_get_pasid(struct iommu_domain *domain, struct device *dev); =20 struct iommu_sva *iommu_sva_bind_device(struct device *dev, - struct mm_struct *mm, - void *drvdata); + struct mm_struct *mm); void iommu_sva_unbind_device(struct iommu_sva *handle); u32 iommu_sva_get_pasid(struct iommu_sva *handle); =20 @@ -1065,7 +1062,7 @@ iommu_aux_get_pasid(struct iommu_domain *domain, stru= ct device *dev) } =20 static inline struct iommu_sva * -iommu_sva_bind_device(struct device *dev, struct mm_struct *mm, void *drvd= ata) +iommu_sva_bind_device(struct device *dev, struct mm_struct *mm) { return NULL; } --=20 2.25.1 From nobody Mon Jun 22 23:53:34 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9667DC433EF for ; Tue, 15 Mar 2022 05:12:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344907AbiCOFNS (ORCPT ); Tue, 15 Mar 2022 01:13:18 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48116 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1344851AbiCOFMV (ORCPT ); Tue, 15 Mar 2022 01:12:21 -0400 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 350304924A for ; Mon, 14 Mar 2022 22:11:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1647321070; x=1678857070; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=6uSpqGl1pbsEqnfIlo3vVDu6SwoYiBP9Hzf6kLLgWR8=; b=d9CYuwcTPTkqpN8Munh15JCMAam9CC9OQHu+6q9cM9/yQiUNvjQInBeq BpPhFDZvLJ99s0jsXx8atZwYoraF/IrqROn3+Ixd71q/GygZElT6+XzsJ KagRk0viMHoC+LDq4/z26xwyIoWsk6LMJKibFplDUEnbzUhMvo9CZGG3v Q1vwWuvRHrl+OvX5Mxxo2EkLkzB0WRDGvskOceC2FdjFxY3yt9JnVpGhC 5I6EJJOhk/KY7+kC9jFiBX3yxOdgf83fd0np42N1rJkfRqxoYsrCaYbPo qIc92tXKU5jXDFQeRI+CNoaHKH8STIO+tCIxi8bA0A05XQvu2kYRYoEG1 w==; X-IronPort-AV: E=McAfee;i="6200,9189,10286"; a="236159110" X-IronPort-AV: E=Sophos;i="5.90,182,1643702400"; d="scan'208";a="236159110" Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Mar 2022 22:04:00 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.90,182,1643702400"; d="scan'208";a="580383598" Received: from otc-wp-03.jf.intel.com (HELO jacob-builder.jf.intel.com) ([10.54.39.79]) by orsmga001.jf.intel.com with ESMTP; 14 Mar 2022 22:04:00 -0700 From: Jacob Pan To: iommu@lists.linux-foundation.org, LKML , Joerg Roedel , Jason Gunthorpe , "Christoph Hellwig" , "Lu Baolu" , Jean-Philippe Brucker Cc: Jacob Pan , Raj Ashok , "Kumar, Sanjay K" , Dave Jiang , Tony Luck , "Zanussi, Tom" , Dan Williams , "Tian, Kevin" , Yi Liu Subject: [PATCH v2 9/9] dmaengine: idxd: separate user and kernel pasid enabling Date: Mon, 14 Mar 2022 22:07:13 -0700 Message-Id: <20220315050713.2000518-10-jacob.jun.pan@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220315050713.2000518-1-jacob.jun.pan@linux.intel.com> References: <20220315050713.2000518-1-jacob.jun.pan@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Dave Jiang The idxd driver always gated the pasid enabling under a single knob and this assumption is incorrect. The pasid used for kernel operation can be independently toggled and has no dependency on the user pasid (and vice versa). Split the two so they are independent "enabled" flags. Signed-off-by: Dave Jiang Signed-off-by: Jacob Pan --- drivers/dma/idxd/cdev.c | 4 ++-- drivers/dma/idxd/idxd.h | 6 ++++++ drivers/dma/idxd/init.c | 30 ++++++++++++++++++------------ 3 files changed, 26 insertions(+), 14 deletions(-) diff --git a/drivers/dma/idxd/cdev.c b/drivers/dma/idxd/cdev.c index 312ec37ebf91..addaebca7683 100644 --- a/drivers/dma/idxd/cdev.c +++ b/drivers/dma/idxd/cdev.c @@ -99,7 +99,7 @@ static int idxd_cdev_open(struct inode *inode, struct fil= e *filp) ctx->wq =3D wq; filp->private_data =3D ctx; =20 - if (device_pasid_enabled(idxd)) { + if (device_user_pasid_enabled(idxd)) { sva =3D iommu_sva_bind_device(dev, current->mm); if (IS_ERR(sva)) { rc =3D PTR_ERR(sva); @@ -152,7 +152,7 @@ static int idxd_cdev_release(struct inode *node, struct= file *filep) if (wq_shared(wq)) { idxd_device_drain_pasid(idxd, ctx->pasid); } else { - if (device_pasid_enabled(idxd)) { + if (device_user_pasid_enabled(idxd)) { /* The wq disable in the disable pasid function will drain the wq */ rc =3D idxd_wq_disable_pasid(wq); if (rc < 0) diff --git a/drivers/dma/idxd/idxd.h b/drivers/dma/idxd/idxd.h index a09ab4a6e1c1..190b08bd7c08 100644 --- a/drivers/dma/idxd/idxd.h +++ b/drivers/dma/idxd/idxd.h @@ -239,6 +239,7 @@ enum idxd_device_flag { IDXD_FLAG_CONFIGURABLE =3D 0, IDXD_FLAG_CMD_RUNNING, IDXD_FLAG_PASID_ENABLED, + IDXD_FLAG_USER_PASID_ENABLED, }; =20 struct idxd_dma_dev { @@ -468,6 +469,11 @@ static inline bool device_pasid_enabled(struct idxd_de= vice *idxd) return test_bit(IDXD_FLAG_PASID_ENABLED, &idxd->flags); } =20 +static inline bool device_user_pasid_enabled(struct idxd_device *idxd) +{ + return test_bit(IDXD_FLAG_USER_PASID_ENABLED, &idxd->flags); +} + static inline bool device_swq_supported(struct idxd_device *idxd) { return (support_enqcmd && device_pasid_enabled(idxd)); diff --git a/drivers/dma/idxd/init.c b/drivers/dma/idxd/init.c index 5d1f8dd4abf6..981150b7d09b 100644 --- a/drivers/dma/idxd/init.c +++ b/drivers/dma/idxd/init.c @@ -500,16 +500,19 @@ static int idxd_probe(struct idxd_device *idxd) =20 if (IS_ENABLED(CONFIG_INTEL_IDXD_SVM) && sva) { rc =3D iommu_dev_enable_feature(dev, IOMMU_DEV_FEAT_SVA); - if (rc =3D=3D 0) { - rc =3D idxd_enable_system_pasid(idxd); - if (rc < 0) { - iommu_dev_disable_feature(dev, IOMMU_DEV_FEAT_SVA); - dev_warn(dev, "Failed to enable PASID. No SVA support: %d\n", rc); - } else { - set_bit(IDXD_FLAG_PASID_ENABLED, &idxd->flags); - } - } else { + if (rc) { + /* + * Do not bail here since legacy DMA is still + * supported, both user and in-kernel DMA with + * PASID rely on SVA feature. + */ dev_warn(dev, "Unable to turn on SVA feature.\n"); + } else { + set_bit(IDXD_FLAG_USER_PASID_ENABLED, &idxd->flags); + if (idxd_enable_system_pasid(idxd)) + dev_warn(dev, "No in-kernel DMA with PASID.\n"); + else + set_bit(IDXD_FLAG_PASID_ENABLED, &idxd->flags); } } idxd_read_caps(idxd); @@ -545,7 +548,8 @@ static int idxd_probe(struct idxd_device *idxd) err: if (device_pasid_enabled(idxd)) idxd_disable_system_pasid(idxd); - iommu_dev_disable_feature(dev, IOMMU_DEV_FEAT_SVA); + if (device_user_pasid_enabled(idxd) || device_pasid_enabled(idxd)) + iommu_dev_disable_feature(dev, IOMMU_DEV_FEAT_SVA); return rc; } =20 @@ -558,7 +562,8 @@ static void idxd_cleanup(struct idxd_device *idxd) idxd_cleanup_internals(idxd); if (device_pasid_enabled(idxd)) idxd_disable_system_pasid(idxd); - iommu_dev_disable_feature(dev, IOMMU_DEV_FEAT_SVA); + if (device_user_pasid_enabled(idxd) || device_pasid_enabled(idxd)) + iommu_dev_disable_feature(dev, IOMMU_DEV_FEAT_SVA); } =20 static int idxd_pci_probe(struct pci_dev *pdev, const struct pci_device_id= *id) @@ -677,7 +682,8 @@ static void idxd_remove(struct pci_dev *pdev) free_irq(irq_entry->vector, irq_entry); pci_free_irq_vectors(pdev); pci_iounmap(pdev, idxd->reg_base); - iommu_dev_disable_feature(&pdev->dev, IOMMU_DEV_FEAT_SVA); + if (device_user_pasid_enabled(idxd) || device_pasid_enabled(idxd)) + iommu_dev_disable_feature(&pdev->dev, IOMMU_DEV_FEAT_SVA); pci_disable_device(pdev); destroy_workqueue(idxd->wq); perfmon_pmu_remove(idxd); --=20 2.25.1