From nobody Thu Dec 18 09:42:56 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 84E62C04A94 for ; Mon, 14 Aug 2023 01:21:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231775AbjHNBVD (ORCPT ); Sun, 13 Aug 2023 21:21:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40322 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231716AbjHNBUd (ORCPT ); Sun, 13 Aug 2023 21:20:33 -0400 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.65]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1DAF2E6E; Sun, 13 Aug 2023 18:20:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1691976032; x=1723512032; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=1VKEBWyIoCdoQh4ifxDFIHU7H3LnyvQp0GU6ahA/QQE=; b=Qrv0sR5eRASz2+cnddIJc2aipVni2jlP8zyLSrTa2yiEXlIPkSy2f3kq 5hbFOFk4l84ZpIq7ncNPOGwAH6DOSayKba1Dx6PGj2LYfmGmQAPUdsVIm 75khmaGbGL3fkKqb8grqrLWCxYI1zVhcFcRGhm5OaOiJlDxMoX/DkyCFD foyKxGLpTPwOvpmmDFU8pCyal3+EAwvk1AsMi7BDyPzHJvC7RBJi1RNdW lzicgRlKgmLhUI76WDmzm1BWFxHeHIwMaqcStcbaXm2bEoeed+0tdkzBy D8De356fcbE5gime2X5E6yP7GqKvSgSe3EcHShadsDCKiaDf4xrdXclQ5 A==; X-IronPort-AV: E=McAfee;i="6600,9927,10801"; a="375645305" X-IronPort-AV: E=Sophos;i="6.01,171,1684825200"; d="scan'208";a="375645305" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Aug 2023 18:20:31 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10801"; a="726842292" X-IronPort-AV: E=Sophos;i="6.01,171,1684825200"; d="scan'208";a="726842292" Received: from allen-box.sh.intel.com ([10.239.159.127]) by orsmga007.jf.intel.com with ESMTP; 13 Aug 2023 18:20:28 -0700 From: Lu Baolu To: Joerg Roedel , Will Deacon , Robin Murphy , Jason Gunthorpe , Kevin Tian , Jean-Philippe Brucker , Nicolin Chen Cc: Yi Liu , Jacob Pan , iommu@lists.linux.dev, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Lu Baolu Subject: [PATCH v2 1/3] iommu: Make single-device group for PASID explicit Date: Mon, 14 Aug 2023 09:17:57 +0800 Message-Id: <20230814011759.102089-2-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230814011759.102089-1-baolu.lu@linux.intel.com> References: <20230814011759.102089-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The PASID interfaces have always supported only single-device groups. This was first introduced in commit 26b25a2b98e45 ("iommu: Bind process address spaces to devices"), and has been kept consistent in subsequent commits. However, the core code doesn't explicitly check for this requirement after commit 201007ef707a8 ("PCI: Enable PASID only when ACS RR & UF enabled on upstream path"), which made this requirement implicit. Restore the check to make it explicit that the PASID interfaces only support devices belonging to single-device groups. Signed-off-by: Lu Baolu --- drivers/iommu/iommu.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c index 71b9c41f2a9e..f1eba60e573f 100644 --- a/drivers/iommu/iommu.c +++ b/drivers/iommu/iommu.c @@ -3408,6 +3408,11 @@ int iommu_attach_device_pasid(struct iommu_domain *d= omain, return -ENODEV; =20 mutex_lock(&group->mutex); + if (list_count_nodes(&group->devices) !=3D 1) { + ret =3D -EINVAL; + goto out_unlock; + } + curr =3D xa_cmpxchg(&group->pasid_array, pasid, NULL, domain, GFP_KERNEL); if (curr) { ret =3D xa_err(curr) ? : -EBUSY; --=20 2.34.1 From nobody Thu Dec 18 09:42:56 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C496FC04FE0 for ; Mon, 14 Aug 2023 01:21:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231838AbjHNBVG (ORCPT ); Sun, 13 Aug 2023 21:21:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40368 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231724AbjHNBUg (ORCPT ); Sun, 13 Aug 2023 21:20:36 -0400 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.65]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 85B72E75; Sun, 13 Aug 2023 18:20:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1691976035; x=1723512035; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=277c02PzldzlnRnH2jM0ESjUVEL/whZkd2HGaLLh2vY=; b=MrspJpInasTenaA2hCo/inKO4d/avRNFMUOSiPLb82QNmcj33UBG6udA ol5MzsfKHPt7gkChUZwECEzxDDeabyAsIQGPK5hfWMhyIje2JSrqQd3YS snPTKpMxIDR4coxd2KXwE+Ycj/4cpFTd6wsI6ChRQRzT1aHu4h0dThJCz fhR5bvzoQ8HSonD3HYnO/uIcqp28KsHRV0q6ATV/eZLFZ3pewyV9aOv85 zt0gOdJLC8AGdsNDWTWRHoW6aZLIc3NzmR6OIquDqwpuPrLc9nFBhwa69 UNDkssJAMg1gLfxXWuhipbsAOmxepnEGneh933wwH540mU9AJ/j4ttb7k w==; X-IronPort-AV: E=McAfee;i="6600,9927,10801"; a="375645319" X-IronPort-AV: E=Sophos;i="6.01,171,1684825200"; d="scan'208";a="375645319" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Aug 2023 18:20:35 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10801"; a="726842296" X-IronPort-AV: E=Sophos;i="6.01,171,1684825200"; d="scan'208";a="726842296" Received: from allen-box.sh.intel.com ([10.239.159.127]) by orsmga007.jf.intel.com with ESMTP; 13 Aug 2023 18:20:31 -0700 From: Lu Baolu To: Joerg Roedel , Will Deacon , Robin Murphy , Jason Gunthorpe , Kevin Tian , Jean-Philippe Brucker , Nicolin Chen Cc: Yi Liu , Jacob Pan , iommu@lists.linux.dev, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Lu Baolu Subject: [PATCH v2 2/3] iommu: Consolidate pasid dma ownership check Date: Mon, 14 Aug 2023 09:17:58 +0800 Message-Id: <20230814011759.102089-3-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230814011759.102089-1-baolu.lu@linux.intel.com> References: <20230814011759.102089-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" When switching device DMA ownership, it is required that all the device's pasid DMA be disabled. This is done by checking if the pasid array of the group is empty. Consolidate all the open code into a single helper. No intentional functionality change. Signed-off-by: Lu Baolu --- drivers/iommu/iommu.c | 23 ++++++++++++++++++----- 1 file changed, 18 insertions(+), 5 deletions(-) diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c index f1eba60e573f..d4a06a37ce39 100644 --- a/drivers/iommu/iommu.c +++ b/drivers/iommu/iommu.c @@ -3127,6 +3127,19 @@ static bool iommu_is_default_domain(struct iommu_gro= up *group) return false; } =20 +/* + * Assert no PASID DMA when claiming or releasing group's DMA ownership. + * The device pasid interfaces are only for device drivers that have + * claimed the DMA ownership. Return true if no pasid DMA setup, otherwise + * return false with a WARN(). + */ +static bool assert_pasid_dma_ownership(struct iommu_group *group) +{ + lockdep_assert_held(&group->mutex); + + return !WARN_ON(!xa_empty(&group->pasid_array)); +} + /** * iommu_device_use_default_domain() - Device driver wants to handle device * DMA through the kernel DMA API. @@ -3147,7 +3160,7 @@ int iommu_device_use_default_domain(struct device *de= v) mutex_lock(&group->mutex); if (group->owner_cnt) { if (group->owner || !iommu_is_default_domain(group) || - !xa_empty(&group->pasid_array)) { + !assert_pasid_dma_ownership(group)) { ret =3D -EBUSY; goto unlock_out; } @@ -3177,7 +3190,7 @@ void iommu_device_unuse_default_domain(struct device = *dev) return; =20 mutex_lock(&group->mutex); - if (!WARN_ON(!group->owner_cnt || !xa_empty(&group->pasid_array))) + if (!WARN_ON(!group->owner_cnt) && assert_pasid_dma_ownership(group)) group->owner_cnt--; =20 mutex_unlock(&group->mutex); @@ -3211,7 +3224,7 @@ static int __iommu_take_dma_ownership(struct iommu_gr= oup *group, void *owner) int ret; =20 if ((group->domain && group->domain !=3D group->default_domain) || - !xa_empty(&group->pasid_array)) + !assert_pasid_dma_ownership(group)) return -EBUSY; =20 ret =3D __iommu_group_alloc_blocking_domain(group); @@ -3296,8 +3309,8 @@ EXPORT_SYMBOL_GPL(iommu_device_claim_dma_owner); =20 static void __iommu_release_dma_ownership(struct iommu_group *group) { - if (WARN_ON(!group->owner_cnt || !group->owner || - !xa_empty(&group->pasid_array))) + if (WARN_ON(!group->owner_cnt || !group->owner) || + !assert_pasid_dma_ownership(group)) return; =20 group->owner_cnt =3D 0; --=20 2.34.1 From nobody Thu Dec 18 09:42:56 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D3077C04FDF for ; Mon, 14 Aug 2023 01:21:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231992AbjHNBVI (ORCPT ); Sun, 13 Aug 2023 21:21:08 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40384 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231753AbjHNBUm (ORCPT ); Sun, 13 Aug 2023 21:20:42 -0400 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.65]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 046B110E2; Sun, 13 Aug 2023 18:20:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1691976039; x=1723512039; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=xz9IkjBL49Aqu2OLcjsslSruJe0mn1CqaV/hLG6938M=; b=LoDGsb7PBZKgVGyaKwoW3K1U+On+2MI4JF4sOcAxpSRFwGdkJ27oJS/r XzdjnfSloXRtMHdz1IM32uAcyUcizCIN7xZzsJ2saXRj6ceG92KzXhFRq j95uWRD1Y2pnosc1p6cW6hllLMLVLmCo7m56NM2qdwVA1TJeTv6ZoRN3/ 0bGd6bDvRWpE6Wg1Xh1Yqo/bCFrrHULezgypId2gBjtTXpykg5UWkqipM Nf0XZgbWUAtXTITiwGssew6xa22ekyXryB2BqFVThhhtGRev83iwwozC/ bjvU8/UdInKviqnZd4P9hg1+hWBUsJzw6kmDiBrI99JUoz4hDWvmH4DJM w==; X-IronPort-AV: E=McAfee;i="6600,9927,10801"; a="375645336" X-IronPort-AV: E=Sophos;i="6.01,171,1684825200"; d="scan'208";a="375645336" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Aug 2023 18:20:38 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10801"; a="726842300" X-IronPort-AV: E=Sophos;i="6.01,171,1684825200"; d="scan'208";a="726842300" Received: from allen-box.sh.intel.com ([10.239.159.127]) by orsmga007.jf.intel.com with ESMTP; 13 Aug 2023 18:20:35 -0700 From: Lu Baolu To: Joerg Roedel , Will Deacon , Robin Murphy , Jason Gunthorpe , Kevin Tian , Jean-Philippe Brucker , Nicolin Chen Cc: Yi Liu , Jacob Pan , iommu@lists.linux.dev, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Lu Baolu Subject: [PATCH v2 3/3] iommu: Move pasid array from group to device Date: Mon, 14 Aug 2023 09:17:59 +0800 Message-Id: <20230814011759.102089-4-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230814011759.102089-1-baolu.lu@linux.intel.com> References: <20230814011759.102089-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The PASID (Process Address Space ID) feature is a device feature that allows a device driver to manage the PASID value and attach or detach its domain to the pasid. The pasid array, which is used to store the domain of each pasid, is currently stored in iommu group struct, but it would be more natural to move it to the device so that the device drivers don't need to understand the internal iommu group concept. Signed-off-by: Lu Baolu --- include/linux/iommu.h | 2 ++ drivers/iommu/iommu.c | 80 +++++++++++++++---------------------------- 2 files changed, 29 insertions(+), 53 deletions(-) diff --git a/include/linux/iommu.h b/include/linux/iommu.h index 411ce9b998dc..76c960741449 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -412,6 +412,7 @@ struct iommu_fault_param { * @iopf_param: I/O Page Fault queue and data * @fwspec: IOMMU fwspec data * @iommu_dev: IOMMU device this device is linked to + * @pasid_array: pasid-indexed array of domains attached to pasid * @priv: IOMMU Driver private data * @max_pasids: number of PASIDs this device can consume * @attach_deferred: the dma domain attachment is deferred @@ -427,6 +428,7 @@ struct dev_iommu { struct iopf_device_param *iopf_param; struct iommu_fwspec *fwspec; struct iommu_device *iommu_dev; + struct xarray pasid_array; void *priv; u32 max_pasids; u32 attach_deferred:1; diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c index d4a06a37ce39..35cccfb5c24a 100644 --- a/drivers/iommu/iommu.c +++ b/drivers/iommu/iommu.c @@ -51,7 +51,6 @@ struct iommu_group { struct kobject kobj; struct kobject *devices_kobj; struct list_head devices; - struct xarray pasid_array; struct mutex mutex; void *iommu_data; void (*iommu_data_release)(void *iommu_data); @@ -310,6 +309,7 @@ static struct dev_iommu *dev_iommu_get(struct device *d= ev) return NULL; =20 mutex_init(¶m->lock); + xa_init(¶m->pasid_array); dev->iommu =3D param; return param; } @@ -919,7 +919,6 @@ struct iommu_group *iommu_group_alloc(void) mutex_init(&group->mutex); INIT_LIST_HEAD(&group->devices); INIT_LIST_HEAD(&group->entry); - xa_init(&group->pasid_array); =20 ret =3D ida_alloc(&iommu_group_ida, GFP_KERNEL); if (ret < 0) { @@ -3135,9 +3134,15 @@ static bool iommu_is_default_domain(struct iommu_gro= up *group) */ static bool assert_pasid_dma_ownership(struct iommu_group *group) { + struct group_device *device; + lockdep_assert_held(&group->mutex); + for_each_group_device(group, device) { + if (WARN_ON(!xa_empty(&device->dev->iommu->pasid_array))) + return false; + } =20 - return !WARN_ON(!xa_empty(&group->pasid_array)); + return true; } =20 /** @@ -3371,33 +3376,6 @@ bool iommu_group_dma_owner_claimed(struct iommu_grou= p *group) } EXPORT_SYMBOL_GPL(iommu_group_dma_owner_claimed); =20 -static int __iommu_set_group_pasid(struct iommu_domain *domain, - struct iommu_group *group, ioasid_t pasid) -{ - struct group_device *device; - int ret =3D 0; - - for_each_group_device(group, device) { - ret =3D domain->ops->set_dev_pasid(domain, device->dev, pasid); - if (ret) - break; - } - - return ret; -} - -static void __iommu_remove_group_pasid(struct iommu_group *group, - ioasid_t pasid) -{ - struct group_device *device; - const struct iommu_ops *ops; - - for_each_group_device(group, device) { - ops =3D dev_iommu_ops(device->dev); - ops->remove_dev_pasid(device->dev, pasid); - } -} - /* * iommu_attach_device_pasid() - Attach a domain to pasid of device * @domain: the iommu domain. @@ -3411,6 +3389,7 @@ int iommu_attach_device_pasid(struct iommu_domain *do= main, { /* Caller must be a probed driver on dev */ struct iommu_group *group =3D dev->iommu_group; + struct dev_iommu *param =3D dev->iommu; void *curr; int ret; =20 @@ -3422,23 +3401,23 @@ int iommu_attach_device_pasid(struct iommu_domain *= domain, =20 mutex_lock(&group->mutex); if (list_count_nodes(&group->devices) !=3D 1) { - ret =3D -EINVAL; - goto out_unlock; + mutex_unlock(&group->mutex); + return -EINVAL; } + mutex_unlock(&group->mutex); =20 - curr =3D xa_cmpxchg(&group->pasid_array, pasid, NULL, domain, GFP_KERNEL); + mutex_lock(¶m->lock); + curr =3D xa_cmpxchg(¶m->pasid_array, pasid, NULL, domain, GFP_KERNEL); if (curr) { ret =3D xa_err(curr) ? : -EBUSY; goto out_unlock; } =20 - ret =3D __iommu_set_group_pasid(domain, group, pasid); - if (ret) { - __iommu_remove_group_pasid(group, pasid); - xa_erase(&group->pasid_array, pasid); - } + ret =3D domain->ops->set_dev_pasid(domain, dev, pasid); + if (ret) + xa_erase(¶m->pasid_array, pasid); out_unlock: - mutex_unlock(&group->mutex); + mutex_unlock(¶m->lock); return ret; } EXPORT_SYMBOL_GPL(iommu_attach_device_pasid); @@ -3455,13 +3434,13 @@ EXPORT_SYMBOL_GPL(iommu_attach_device_pasid); void iommu_detach_device_pasid(struct iommu_domain *domain, struct device = *dev, ioasid_t pasid) { - /* Caller must be a probed driver on dev */ - struct iommu_group *group =3D dev->iommu_group; + const struct iommu_ops *ops =3D dev_iommu_ops(dev); + struct dev_iommu *param =3D dev->iommu; =20 - mutex_lock(&group->mutex); - __iommu_remove_group_pasid(group, pasid); - WARN_ON(xa_erase(&group->pasid_array, pasid) !=3D domain); - mutex_unlock(&group->mutex); + mutex_lock(¶m->lock); + ops->remove_dev_pasid(dev, pasid); + WARN_ON(xa_erase(¶m->pasid_array, pasid) !=3D domain); + mutex_unlock(¶m->lock); } EXPORT_SYMBOL_GPL(iommu_detach_device_pasid); =20 @@ -3483,18 +3462,13 @@ struct iommu_domain *iommu_get_domain_for_dev_pasid= (struct device *dev, ioasid_t pasid, unsigned int type) { - /* Caller must be a probed driver on dev */ - struct iommu_group *group =3D dev->iommu_group; struct iommu_domain *domain; =20 - if (!group) - return NULL; - - xa_lock(&group->pasid_array); - domain =3D xa_load(&group->pasid_array, pasid); + xa_lock(&dev->iommu->pasid_array); + domain =3D xa_load(&dev->iommu->pasid_array, pasid); if (type && domain && domain->type !=3D type) domain =3D ERR_PTR(-EBUSY); - xa_unlock(&group->pasid_array); + xa_unlock(&dev->iommu->pasid_array); =20 return domain; } --=20 2.34.1