From nobody Sat Apr 11 10:10:32 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8184BC678DB for ; Mon, 6 Mar 2023 02:59:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229646AbjCFC7W (ORCPT ); Sun, 5 Mar 2023 21:59:22 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48934 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229579AbjCFC7S (ORCPT ); Sun, 5 Mar 2023 21:59:18 -0500 Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E479110419 for ; Sun, 5 Mar 2023 18:59:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1678071557; x=1709607557; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=ubDKJPz7LI3eEIbVA/X1FG8UptdiqBziq+folsggKew=; b=loQ2l8wtHeRqOZ1dTE+oFp22bw+gX9/iTOAgkcgjMiaF8pNTaoUYsDel hDTUXE7udma+Tyvvz+QcKxlXPxtEclqVyYkfVPNBqU9FzXcuFYtt10XYn nW+y8keBRUZ5vf1ftEVNACWjspMg1qynw/gVQCu9DYDqAjCaKQcXD1TUL /wSZPYbcbbJrG/K1bAQByK3eniEP3rmDtb6j1EqXnlStfx9Cux/ioQ8Jm 8CyfUr2KrYHKB9tU2pZE4ml7akDFPJH2Hkk2AJhgxrih3V0vDCed5+agm vj+lNzY6fIqKTPvbiGzT+z2QmI2wooaC9/scGtpdWS6RbJgpTHT+FVHBc w==; X-IronPort-AV: E=McAfee;i="6500,9779,10640"; a="363071480" X-IronPort-AV: E=Sophos;i="5.98,236,1673942400"; d="scan'208";a="363071480" Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Mar 2023 18:59:17 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10640"; a="765072469" X-IronPort-AV: E=Sophos;i="5.98,236,1673942400"; d="scan'208";a="765072469" Received: from allen-box.sh.intel.com ([10.239.159.48]) by FMSMGA003.fm.intel.com with ESMTP; 05 Mar 2023 18:59:15 -0800 From: Lu Baolu To: iommu@lists.linux.dev Cc: Joerg Roedel , Jason Gunthorpe , Christoph Hellwig , Kevin Tian , Will Deacon , Robin Murphy , linux-kernel@vger.kernel.org, Lu Baolu Subject: [PATCH v3 1/6] ARM/dma-mapping: Add arm_iommu_release_device() Date: Mon, 6 Mar 2023 10:57:59 +0800 Message-Id: <20230306025804.13912-2-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230306025804.13912-1-baolu.lu@linux.intel.com> References: <20230306025804.13912-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" It is like arm_iommu_detach_device() except it handles the special case of being called under an iommu driver's release operation. In this case the driver must have already detached the device from any attached domain before calling this function. Replace arm_iommu_detach_device() with arm_iommu_release_device() in the release path of the ipmmu-vmsa driver. The bonus is that it also removes a obstacle of arm_iommu_detach_device() re-entering the iommu core during release_device. With this removed, the iommu core code could be simplified a lot. Signed-off-by: Jason Gunthorpe Signed-off-by: Lu Baolu Reviewed-by: Jason Gunthorpe --- arch/arm/include/asm/dma-iommu.h | 1 + arch/arm/mm/dma-mapping.c | 25 +++++++++++++++++++++++++ drivers/iommu/ipmmu-vmsa.c | 15 +++++++++++++-- 3 files changed, 39 insertions(+), 2 deletions(-) diff --git a/arch/arm/include/asm/dma-iommu.h b/arch/arm/include/asm/dma-io= mmu.h index fe9ef6f79e9c..ea7198a17861 100644 --- a/arch/arm/include/asm/dma-iommu.h +++ b/arch/arm/include/asm/dma-iommu.h @@ -31,6 +31,7 @@ void arm_iommu_release_mapping(struct dma_iommu_mapping *= mapping); int arm_iommu_attach_device(struct device *dev, struct dma_iommu_mapping *mapping); void arm_iommu_detach_device(struct device *dev); +void arm_iommu_release_device(struct device *dev); =20 #endif /* __KERNEL__ */ #endif diff --git a/arch/arm/mm/dma-mapping.c b/arch/arm/mm/dma-mapping.c index 8bc01071474a..96fa27f4a164 100644 --- a/arch/arm/mm/dma-mapping.c +++ b/arch/arm/mm/dma-mapping.c @@ -1682,6 +1682,31 @@ int arm_iommu_attach_device(struct device *dev, } EXPORT_SYMBOL_GPL(arm_iommu_attach_device); =20 +/** + * arm_iommu_release_device + * @dev: valid struct device pointer + * + * This is like arm_iommu_detach_device() except it handles the special + * case of being called under an iommu driver's release operation. In this + * case the driver must have already detached the device from any attached + * domain before calling this function. + */ +void arm_iommu_release_device(struct device *dev) +{ + struct dma_iommu_mapping *mapping; + + mapping =3D to_dma_iommu_mapping(dev); + if (!mapping) { + dev_warn(dev, "Not attached\n"); + return; + } + + kref_put(&mapping->kref, release_iommu_mapping); + to_dma_iommu_mapping(dev) =3D NULL; + set_dma_ops(dev, NULL); +} +EXPORT_SYMBOL_GPL(arm_iommu_release_device); + /** * arm_iommu_detach_device * @dev: valid struct device pointer diff --git a/drivers/iommu/ipmmu-vmsa.c b/drivers/iommu/ipmmu-vmsa.c index bdf1a4e5eae0..de9c74cf61a4 100644 --- a/drivers/iommu/ipmmu-vmsa.c +++ b/drivers/iommu/ipmmu-vmsa.c @@ -30,7 +30,7 @@ #define arm_iommu_create_mapping(...) NULL #define arm_iommu_attach_device(...) -ENODEV #define arm_iommu_release_mapping(...) do {} while (0) -#define arm_iommu_detach_device(...) do {} while (0) +#define arm_iommu_release_device(...) do {} while (0) #endif =20 #define IPMMU_CTX_MAX 16U @@ -820,7 +820,18 @@ static void ipmmu_probe_finalize(struct device *dev) =20 static void ipmmu_release_device(struct device *dev) { - arm_iommu_detach_device(dev); + struct iommu_fwspec *fwspec =3D dev_iommu_fwspec_get(dev); + struct ipmmu_vmsa_device *mmu =3D to_ipmmu(dev); + unsigned int i; + + for (i =3D 0; i < fwspec->num_ids; ++i) { + unsigned int utlb =3D fwspec->ids[i]; + + ipmmu_imuctr_write(mmu, utlb, 0); + mmu->utlb_ctx[utlb] =3D IPMMU_CTX_INVALID; + } + + arm_iommu_release_device(dev); } =20 static struct iommu_group *ipmmu_find_group(struct device *dev) --=20 2.34.1 From nobody Sat Apr 11 10:10:32 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 50FB0C678DB for ; Mon, 6 Mar 2023 02:59:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229670AbjCFC70 (ORCPT ); Sun, 5 Mar 2023 21:59:26 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49080 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229600AbjCFC7V (ORCPT ); Sun, 5 Mar 2023 21:59:21 -0500 Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 617C81165B for ; Sun, 5 Mar 2023 18:59:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1678071560; x=1709607560; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=r1yKdk6UOO9yO9Ew/Jiy3oqCAhgo4LhD3eUDU5kx0jM=; b=X72YTi6zFCO9TMth5sEwh5m4Np/LDHk3q9Mq3BvHlJo++2+mKA5p00r4 n0jIXkZn9kEkYC25nhWkVbLirvf1bfK/Bs8D3h4MHsg8r4WCvLdCD+bZe RBHI6HYI2lJ40eUwjfgaF7ATBWl/th3ShkqLbzE9we9+WRKMFbQXslPg8 BK5O8v1/NSyOZ0qqdlnlMq0FTXvR7APccetXZ6li24yl3Xq93Q+V1aDz8 2r5TXP0FxcpLbgTtQajPhAPShU+rqdSpHzSVEJHJKtq9Wv1zmUTskm81v T5M3PYQckj5S6MtTAnDH8Vv0nVDteJ63XGWLDl2aXnpeGSt3mBkVNHKO4 Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10640"; a="363071488" X-IronPort-AV: E=Sophos;i="5.98,236,1673942400"; d="scan'208";a="363071488" Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Mar 2023 18:59:20 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10640"; a="765072474" X-IronPort-AV: E=Sophos;i="5.98,236,1673942400"; d="scan'208";a="765072474" Received: from allen-box.sh.intel.com ([10.239.159.48]) by FMSMGA003.fm.intel.com with ESMTP; 05 Mar 2023 18:59:17 -0800 From: Lu Baolu To: iommu@lists.linux.dev Cc: Joerg Roedel , Jason Gunthorpe , Christoph Hellwig , Kevin Tian , Will Deacon , Robin Murphy , linux-kernel@vger.kernel.org, Lu Baolu Subject: [PATCH v3 2/6] iommu: Split iommu_group_remove_device() into helpers Date: Mon, 6 Mar 2023 10:58:00 +0800 Message-Id: <20230306025804.13912-3-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230306025804.13912-1-baolu.lu@linux.intel.com> References: <20230306025804.13912-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" So that code could be re-used by iommu_release_device() in the subsequent change. No intention for functionality change. Signed-off-by: Jason Gunthorpe Signed-off-by: Lu Baolu Reviewed-by: Jason Gunthorpe --- drivers/iommu/iommu.c | 64 +++++++++++++++++++++++++++++-------------- 1 file changed, 44 insertions(+), 20 deletions(-) diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c index 80211bf08c0d..bd9b293e07a8 100644 --- a/drivers/iommu/iommu.c +++ b/drivers/iommu/iommu.c @@ -465,6 +465,46 @@ int iommu_probe_device(struct device *dev) =20 } =20 +/* + * Remove a device from a group's device list and return the group device + * if successful. + */ +static struct group_device * +__iommu_group_remove_device(struct iommu_group *group, struct device *dev) +{ + struct group_device *device; + + lockdep_assert_held(&group->mutex); + list_for_each_entry(device, &group->devices, list) { + if (device->dev =3D=3D dev) { + list_del(&device->list); + return device; + } + } + + return NULL; +} + +/* + * Release a device from its group and decrements the iommu group reference + * count. + */ +static void __iommu_group_release_device(struct iommu_group *group, + struct group_device *grp_dev) +{ + struct device *dev =3D grp_dev->dev; + + sysfs_remove_link(group->devices_kobj, grp_dev->name); + sysfs_remove_link(&dev->kobj, "iommu_group"); + + trace_remove_device_from_group(group->id, dev); + + kfree(grp_dev->name); + kfree(grp_dev); + dev->iommu_group =3D NULL; + kobject_put(group->devices_kobj); +} + void iommu_release_device(struct device *dev) { const struct iommu_ops *ops; @@ -1080,7 +1120,7 @@ EXPORT_SYMBOL_GPL(iommu_group_add_device); void iommu_group_remove_device(struct device *dev) { struct iommu_group *group =3D dev->iommu_group; - struct group_device *tmp_device, *device =3D NULL; + struct group_device *device; =20 if (!group) return; @@ -1088,27 +1128,11 @@ void iommu_group_remove_device(struct device *dev) dev_info(dev, "Removing from iommu group %d\n", group->id); =20 mutex_lock(&group->mutex); - list_for_each_entry(tmp_device, &group->devices, list) { - if (tmp_device->dev =3D=3D dev) { - device =3D tmp_device; - list_del(&device->list); - break; - } - } + device =3D __iommu_group_remove_device(group, dev); mutex_unlock(&group->mutex); =20 - if (!device) - return; - - sysfs_remove_link(group->devices_kobj, device->name); - sysfs_remove_link(&dev->kobj, "iommu_group"); - - trace_remove_device_from_group(group->id, dev); - - kfree(device->name); - kfree(device); - dev->iommu_group =3D NULL; - kobject_put(group->devices_kobj); + if (device) + __iommu_group_release_device(group, device); } EXPORT_SYMBOL_GPL(iommu_group_remove_device); =20 --=20 2.34.1 From nobody Sat Apr 11 10:10:32 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 53013C61DA4 for ; Mon, 6 Mar 2023 02:59:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229666AbjCFC73 (ORCPT ); Sun, 5 Mar 2023 21:59:29 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49178 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229591AbjCFC7X (ORCPT ); Sun, 5 Mar 2023 21:59:23 -0500 Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D2F871114C for ; Sun, 5 Mar 2023 18:59:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1678071562; x=1709607562; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=23OLppNDX/QUue37RTrJRL+8G3hSt1KtGJZMXHrLeEU=; b=J0+weIw6Mci7I/ufJQBR/7s3vJ+jZYM32NCgzjSl6TN2jE8mvQgZ8Qpj LHh6V6c8K+v1mVXZH8yb8VXofw/V8NKv/7virFc+avbZq50m8g3Q34awR o2xwiZkyaM90FTKipEr5xW33gt0h91yFLXK+GNs2DBsCXunNt9re710ub a+0DY+18bsXsivsI/AZQeLq0/nJWqqCQnpYI8thvO3ngeS0nCvV2gYsC8 xpxC91gFvThvZQyGyLKQeYDss8e8/55lAjIIRNVqbuZd9KQH3N7YQkUzF ya/M/u8qLUwPKPIJ05jKp1YO9jwggI8zJ5RL+wHNYifV3DoJ/2aCMOPbh g==; X-IronPort-AV: E=McAfee;i="6500,9779,10640"; a="363071513" X-IronPort-AV: E=Sophos;i="5.98,236,1673942400"; d="scan'208";a="363071513" Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Mar 2023 18:59:22 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10640"; a="765072478" X-IronPort-AV: E=Sophos;i="5.98,236,1673942400"; d="scan'208";a="765072478" Received: from allen-box.sh.intel.com ([10.239.159.48]) by FMSMGA003.fm.intel.com with ESMTP; 05 Mar 2023 18:59:20 -0800 From: Lu Baolu To: iommu@lists.linux.dev Cc: Joerg Roedel , Jason Gunthorpe , Christoph Hellwig , Kevin Tian , Will Deacon , Robin Murphy , linux-kernel@vger.kernel.org, Lu Baolu Subject: [PATCH v3 3/6] iommu: Same critical region for device release and removal Date: Mon, 6 Mar 2023 10:58:01 +0800 Message-Id: <20230306025804.13912-4-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230306025804.13912-1-baolu.lu@linux.intel.com> References: <20230306025804.13912-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" In a non-driver context, it is crucial to ensure the consistency of a device's iommu ops. Otherwise, it may result in a situation where a device is released but it's iommu ops are still used. Put the ops->release_device and __iommu_group_remove_device() in a some group->mutext critical region, so that, as long as group->mutex is held and the device is in its group's device list, its iommu ops are always consistent. Add check of group ownership if the released device is the last one. Signed-off-by: Jason Gunthorpe Signed-off-by: Lu Baolu Reviewed-by: Jason Gunthorpe --- drivers/iommu/iommu.c | 30 ++++++++++++++++++++++++++++-- 1 file changed, 28 insertions(+), 2 deletions(-) diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c index bd9b293e07a8..0bcd9625090d 100644 --- a/drivers/iommu/iommu.c +++ b/drivers/iommu/iommu.c @@ -507,18 +507,44 @@ static void __iommu_group_release_device(struct iommu= _group *group, =20 void iommu_release_device(struct device *dev) { + struct iommu_group *group =3D dev->iommu_group; + struct group_device *device; const struct iommu_ops *ops; =20 - if (!dev->iommu) + if (!dev->iommu || !group) return; =20 iommu_device_unlink(dev->iommu->iommu_dev, dev); =20 + mutex_lock(&group->mutex); + device =3D __iommu_group_remove_device(group, dev); + + /* + * If the group has become empty then ownership must have been released, + * and the current domain must be set back to NULL or the default + * domain. + */ + if (list_empty(&group->devices)) + WARN_ON(group->owner_cnt || + group->domain !=3D group->default_domain); + + /* + * release_device() must stop using any attached domain on the device. + * If there are still other devices in the group they are not effected + * by this callback. + * + * The IOMMU driver must set the device to either an identity or + * blocking translation and stop using any domain pointer, as it is + * going to be freed. + */ ops =3D dev_iommu_ops(dev); if (ops->release_device) ops->release_device(dev); + mutex_unlock(&group->mutex); + + if (device) + __iommu_group_release_device(group, device); =20 - iommu_group_remove_device(dev); module_put(ops->owner); dev_iommu_free(dev); } --=20 2.34.1 From nobody Sat Apr 11 10:10:32 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9682BC61DA4 for ; Mon, 6 Mar 2023 02:59:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229700AbjCFC7e (ORCPT ); Sun, 5 Mar 2023 21:59:34 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49282 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229676AbjCFC71 (ORCPT ); Sun, 5 Mar 2023 21:59:27 -0500 Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4548A11668 for ; Sun, 5 Mar 2023 18:59:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1678071565; x=1709607565; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=MdWuluD3W8nfCHmXoPEsni78ipVq2xDF2ne8PYskGyM=; b=JFVSLHH5v9MCBzXy40dN2rMYzu/Po924qnmOkJOUJmXpGxL/K/3otyVN EIM7f3bMDZzicpjwbAtb1VJaS8NhA+Op6ZAP5lEh5/TVQuCv/HJRFScYv IM6NK+SYMk9uujPtksR652+MSMZbI/AnaNHELWqAodHE5eeLhD8bD3gha KBqHxtgrtyXhv+OPO6OlA45aqyfE+HBL9M+2Dma9JdzUE845toBd1sp33 23b0VGQ2U+ku+w7nEfIv3+hxWNLJw778Pm0YPHqRh92MZBw8BCI86Xvfj 9uXixdOK3ba0pcnUBBgSu2iToKgtVpZzIOYH18w6L0FEpNKX/7hj6pDaB Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10640"; a="363071524" X-IronPort-AV: E=Sophos;i="5.98,236,1673942400"; d="scan'208";a="363071524" Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Mar 2023 18:59:25 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10640"; a="765072484" X-IronPort-AV: E=Sophos;i="5.98,236,1673942400"; d="scan'208";a="765072484" Received: from allen-box.sh.intel.com ([10.239.159.48]) by FMSMGA003.fm.intel.com with ESMTP; 05 Mar 2023 18:59:22 -0800 From: Lu Baolu To: iommu@lists.linux.dev Cc: Joerg Roedel , Jason Gunthorpe , Christoph Hellwig , Kevin Tian , Will Deacon , Robin Murphy , linux-kernel@vger.kernel.org, Lu Baolu Subject: [PATCH v3 4/6] iommu: Move lock from iommu_change_dev_def_domain() to its caller Date: Mon, 6 Mar 2023 10:58:02 +0800 Message-Id: <20230306025804.13912-5-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230306025804.13912-1-baolu.lu@linux.intel.com> References: <20230306025804.13912-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The intention is to make it possible to put group ownership check and default domain change in a same critical region protected by the group's mutex lock. No intentional functional change. Signed-off-by: Lu Baolu Reviewed-by: Jason Gunthorpe --- drivers/iommu/iommu.c | 29 ++++++++++++++--------------- 1 file changed, 14 insertions(+), 15 deletions(-) diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c index 0bcd9625090d..f8f400548a10 100644 --- a/drivers/iommu/iommu.c +++ b/drivers/iommu/iommu.c @@ -2945,7 +2945,7 @@ static int iommu_change_dev_def_domain(struct iommu_g= roup *group, int ret, dev_def_dom; struct device *dev; =20 - mutex_lock(&group->mutex); + lockdep_assert_held(&group->mutex); =20 if (group->default_domain !=3D group->domain) { dev_err_ratelimited(prev_dev, "Group not assigned to default domain\n"); @@ -3033,28 +3033,15 @@ static int iommu_change_dev_def_domain(struct iommu= _group *group, goto free_new_domain; =20 group->domain =3D group->default_domain; - - /* - * Release the mutex here because ops->probe_finalize() call-back of - * some vendor IOMMU drivers calls arm_iommu_attach_device() which - * in-turn might call back into IOMMU core code, where it tries to take - * group->mutex, resulting in a deadlock. - */ - mutex_unlock(&group->mutex); - - /* Make sure dma_ops is appropriatley set */ - iommu_group_do_probe_finalize(dev, group->default_domain); iommu_domain_free(prev_dom); + return 0; =20 free_new_domain: iommu_domain_free(group->default_domain); group->default_domain =3D prev_dom; group->domain =3D prev_dom; - out: - mutex_unlock(&group->mutex); - return ret; } =20 @@ -3142,7 +3129,19 @@ static ssize_t iommu_group_store_type(struct iommu_g= roup *group, goto out; } =20 + mutex_lock(&group->mutex); ret =3D iommu_change_dev_def_domain(group, dev, req_type); + /* + * Release the mutex here because ops->probe_finalize() call-back of + * some vendor IOMMU drivers calls arm_iommu_attach_device() which + * in-turn might call back into IOMMU core code, where it tries to take + * group->mutex, resulting in a deadlock. + */ + mutex_unlock(&group->mutex); + + /* Make sure dma_ops is appropriatley set */ + if (!ret) + iommu_group_do_probe_finalize(dev, group->default_domain); ret =3D ret ?: count; =20 out: --=20 2.34.1 From nobody Sat Apr 11 10:10:32 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3A297C61DA4 for ; Mon, 6 Mar 2023 02:59:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229688AbjCFC7m (ORCPT ); Sun, 5 Mar 2023 21:59:42 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49658 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229687AbjCFC7a (ORCPT ); Sun, 5 Mar 2023 21:59:30 -0500 Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AD4C31286F for ; Sun, 5 Mar 2023 18:59:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1678071567; x=1709607567; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=e5onQTPUwerLr2g9cgW3cFZ0Fra9mu+whj+aXFchN2A=; b=aIMH4XtB2Vjg52Vbj7Ee5xXusDUd3/4vGa96f+q9JC2HtgbaDszM9bn7 yDnT2hSXX7Ov0AoONFg8hFd4c9FdcL/K31BlnGEyTv/pMXJaqKINFussi fNdul/oNjGA9ZbQRxA4ieKiX4wQ/6UF1lgkcfOLtObynEwBwjJ2p/WcUm PV9MX/1ZfGAI2npdltIsM1JmeMlw61/4SKUdtFLBR6lY8AFv1b2oysd1s hyTUlG1XuwKDUjNuZlVkQ71h/xo3aFupGLxpS+XpjezChmHMVRqE7T9l1 lIPSxu5KCtFb+Y+7PXyXuihs6QNM4Ly60LXTF9FCYvRtkuLZXxRdQJOSB A==; X-IronPort-AV: E=McAfee;i="6500,9779,10640"; a="363071534" X-IronPort-AV: E=Sophos;i="5.98,236,1673942400"; d="scan'208";a="363071534" Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Mar 2023 18:59:27 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10640"; a="765072487" X-IronPort-AV: E=Sophos;i="5.98,236,1673942400"; d="scan'208";a="765072487" Received: from allen-box.sh.intel.com ([10.239.159.48]) by FMSMGA003.fm.intel.com with ESMTP; 05 Mar 2023 18:59:25 -0800 From: Lu Baolu To: iommu@lists.linux.dev Cc: Joerg Roedel , Jason Gunthorpe , Christoph Hellwig , Kevin Tian , Will Deacon , Robin Murphy , linux-kernel@vger.kernel.org, Lu Baolu Subject: [PATCH v3 5/6] iommu: Replace device_lock() with group->mutex Date: Mon, 6 Mar 2023 10:58:03 +0800 Message-Id: <20230306025804.13912-6-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230306025804.13912-1-baolu.lu@linux.intel.com> References: <20230306025804.13912-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" device_lock() was used in iommu_group_store_type() to prevent the devices in an iommu group from being attached by any device driver. On the other hand, in order to avoid lock race between group->mutex and device_lock(), it limited the usage scenario to the singleton groups. We already have the DMA ownership scheme to avoid driver attachment and group->mutex ensures that device ops are always valid, there's no need for device_lock() anymore. Remove device_lock() and the singleton group limitation. Signed-off-by: Lu Baolu Reviewed-by: Jason Gunthorpe --- drivers/iommu/iommu.c | 78 +++++++++---------------------------------- 1 file changed, 16 insertions(+), 62 deletions(-) diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c index f8f400548a10..9fe6d149f281 100644 --- a/drivers/iommu/iommu.c +++ b/drivers/iommu/iommu.c @@ -3011,14 +3011,6 @@ static int iommu_change_dev_def_domain(struct iommu_= group *group, goto out; } =20 - /* We can bring up a flush queue without tearing down the domain */ - if (type =3D=3D IOMMU_DOMAIN_DMA_FQ && prev_dom->type =3D=3D IOMMU_DOMAIN= _DMA) { - ret =3D iommu_dma_init_fq(prev_dom); - if (!ret) - prev_dom->type =3D IOMMU_DOMAIN_DMA_FQ; - goto out; - } - /* Sets group->default_domain to the newly allocated domain */ ret =3D iommu_group_alloc_default_domain(group, dev, type); if (ret) @@ -3051,7 +3043,7 @@ static int iommu_change_dev_def_domain(struct iommu_g= roup *group, * transition. Return failure if this isn't met. * * We need to consider the race between this and the device release path. - * device_lock(dev) is used here to guarantee that the device release path + * group->mutex is used here to guarantee that the device release path * will not be entered at the same time. */ static ssize_t iommu_group_store_type(struct iommu_group *group, @@ -3077,60 +3069,27 @@ static ssize_t iommu_group_store_type(struct iommu_= group *group, else return -EINVAL; =20 - /* - * Lock/Unlock the group mutex here before device lock to - * 1. Make sure that the iommu group has only one device (this is a - * prerequisite for step 2) - * 2. Get struct *dev which is needed to lock device - */ mutex_lock(&group->mutex); - if (iommu_group_device_count(group) !=3D 1) { + /* We can bring up a flush queue without tearing down the domain. */ + if (req_type =3D=3D IOMMU_DOMAIN_DMA_FQ && + group->default_domain->type =3D=3D IOMMU_DOMAIN_DMA) { + ret =3D iommu_dma_init_fq(group->default_domain); + if (!ret) + group->default_domain->type =3D IOMMU_DOMAIN_DMA_FQ; mutex_unlock(&group->mutex); - pr_err_ratelimited("Cannot change default domain: Group has more than on= e device\n"); - return -EINVAL; + + return ret ?: count; } =20 - /* Since group has only one device */ + /* Otherwise, ensure that device exists and no driver is bound. */ dev =3D iommu_group_first_dev(group); - get_device(dev); - - /* - * Don't hold the group mutex because taking group mutex first and then - * the device lock could potentially cause a deadlock as below. Assume - * two threads T1 and T2. T1 is trying to change default domain of an - * iommu group and T2 is trying to hot unplug a device or release [1] VF - * of a PCIe device which is in the same iommu group. T1 takes group - * mutex and before it could take device lock assume T2 has taken device - * lock and is yet to take group mutex. Now, both the threads will be - * waiting for the other thread to release lock. Below, lock order was - * suggested. - * device_lock(dev); - * mutex_lock(&group->mutex); - * iommu_change_dev_def_domain(); - * mutex_unlock(&group->mutex); - * device_unlock(dev); - * - * [1] Typical device release path - * device_lock() from device/driver core code - * -> bus_notifier() - * -> iommu_bus_notifier() - * -> iommu_release_device() - * -> ops->release_device() vendor driver calls back iommu core code - * -> mutex_lock() from iommu core code - */ - mutex_unlock(&group->mutex); - - /* Check if the device in the group still has a driver bound to it */ - device_lock(dev); - if (device_is_bound(dev) && !(req_type =3D=3D IOMMU_DOMAIN_DMA_FQ && - group->default_domain->type =3D=3D IOMMU_DOMAIN_DMA)) { - pr_err_ratelimited("Device is still bound to driver\n"); - ret =3D -EBUSY; - goto out; + if (!dev || group->owner_cnt) { + mutex_unlock(&group->mutex); + return -EPERM; } =20 - mutex_lock(&group->mutex); ret =3D iommu_change_dev_def_domain(group, dev, req_type); + /* * Release the mutex here because ops->probe_finalize() call-back of * some vendor IOMMU drivers calls arm_iommu_attach_device() which @@ -3141,14 +3100,9 @@ static ssize_t iommu_group_store_type(struct iommu_g= roup *group, =20 /* Make sure dma_ops is appropriatley set */ if (!ret) - iommu_group_do_probe_finalize(dev, group->default_domain); - ret =3D ret ?: count; - -out: - device_unlock(dev); - put_device(dev); + __iommu_group_dma_finalize(group); =20 - return ret; + return ret ?: count; } =20 static bool iommu_is_default_domain(struct iommu_group *group) --=20 2.34.1 From nobody Sat Apr 11 10:10:32 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B377DC678DB for ; Mon, 6 Mar 2023 02:59:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229676AbjCFC7w (ORCPT ); Sun, 5 Mar 2023 21:59:52 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50002 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229709AbjCFC7j (ORCPT ); Sun, 5 Mar 2023 21:59:39 -0500 Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3AEE912069 for ; Sun, 5 Mar 2023 18:59:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1678071570; x=1709607570; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Taiw2o2chkLGjTGNeAmtwqU/UhEE9L3oW9AzQ9w2d3I=; b=mAtgoytfVraoykZ6qVCf+TvF0IPuLGw/txjzRvN9GMVa0YVXwtQIM2AB X1XvLgvuS/oEwOgEMgOoruQeV+Duc7D34CTZWZHbPCPSH52d+Mndgt/vt jjcpLXFNK4I9Hf2JVqOpOLqJiJHdTlimGljngkM2EkpK/G8WUNE1u3zHW wbrTT8rRdDQR3NZNSog324A1qMp4j5LH+Q4XBzhEBVpxT+4u7xQcmjh6r qCcHsd5yJ/COq0y+dku6r3isu0SIym40aUAJcyqZnoBtukpDjmyUV/xjX G6vc0rutLwDFTF+RN44LRKWx/T6OpnLmnpHwkr1EL/JrbIjIH2DjaAwSr g==; X-IronPort-AV: E=McAfee;i="6500,9779,10640"; a="363071543" X-IronPort-AV: E=Sophos;i="5.98,236,1673942400"; d="scan'208";a="363071543" Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Mar 2023 18:59:29 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10640"; a="765072490" X-IronPort-AV: E=Sophos;i="5.98,236,1673942400"; d="scan'208";a="765072490" Received: from allen-box.sh.intel.com ([10.239.159.48]) by FMSMGA003.fm.intel.com with ESMTP; 05 Mar 2023 18:59:27 -0800 From: Lu Baolu To: iommu@lists.linux.dev Cc: Joerg Roedel , Jason Gunthorpe , Christoph Hellwig , Kevin Tian , Will Deacon , Robin Murphy , linux-kernel@vger.kernel.org, Lu Baolu Subject: [PATCH v3 6/6] iommu: Cleanup iommu_change_dev_def_domain() Date: Mon, 6 Mar 2023 10:58:04 +0800 Message-Id: <20230306025804.13912-7-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230306025804.13912-1-baolu.lu@linux.intel.com> References: <20230306025804.13912-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" As the singleton group limitation has been removed, cleanup the code in iommu_change_dev_def_domain() accordingly. Documentation is also updated. Signed-off-by: Lu Baolu Reviewed-by: Jason Gunthorpe --- drivers/iommu/iommu.c | 81 +++++-------------- .../ABI/testing/sysfs-kernel-iommu_groups | 1 - 2 files changed, 21 insertions(+), 61 deletions(-) diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c index 9fe6d149f281..735554ccc04b 100644 --- a/drivers/iommu/iommu.c +++ b/drivers/iommu/iommu.c @@ -2924,11 +2924,10 @@ int iommu_dev_disable_feature(struct device *dev, e= num iommu_dev_features feat) EXPORT_SYMBOL_GPL(iommu_dev_disable_feature); =20 /* - * Changes the default domain of an iommu group that has *only* one device + * Changes the default domain of an iommu group * * @group: The group for which the default domain should be changed - * @prev_dev: The device in the group (this is used to make sure that the = device - * hasn't changed after the caller has called this function) + * @dev: The first device in the group * @type: The type of the new default domain that gets associated with the= group * * Returns 0 on success and error code on failure @@ -2939,101 +2938,63 @@ EXPORT_SYMBOL_GPL(iommu_dev_disable_feature); * Please take a closer look if intended to use for other purposes. */ static int iommu_change_dev_def_domain(struct iommu_group *group, - struct device *prev_dev, int type) + struct device *dev, int type) { + struct __group_domain_type gtype =3D {NULL, 0}; struct iommu_domain *prev_dom; - int ret, dev_def_dom; - struct device *dev; + int ret; =20 lockdep_assert_held(&group->mutex); =20 - if (group->default_domain !=3D group->domain) { - dev_err_ratelimited(prev_dev, "Group not assigned to default domain\n"); - ret =3D -EBUSY; - goto out; - } - - /* - * iommu group wasn't locked while acquiring device lock in - * iommu_group_store_type(). So, make sure that the device count hasn't - * changed while acquiring device lock. - * - * Changing default domain of an iommu group with two or more devices - * isn't supported because there could be a potential deadlock. Consider - * the following scenario. T1 is trying to acquire device locks of all - * the devices in the group and before it could acquire all of them, - * there could be another thread T2 (from different sub-system and use - * case) that has already acquired some of the device locks and might be - * waiting for T1 to release other device locks. - */ - if (iommu_group_device_count(group) !=3D 1) { - dev_err_ratelimited(prev_dev, "Cannot change default domain: Group has m= ore than one device\n"); - ret =3D -EINVAL; - goto out; - } - - /* Since group has only one device */ - dev =3D iommu_group_first_dev(group); - - if (prev_dev !=3D dev) { - dev_err_ratelimited(prev_dev, "Cannot change default domain: Device has = been changed\n"); - ret =3D -EBUSY; - goto out; - } - prev_dom =3D group->default_domain; - if (!prev_dom) { - ret =3D -EINVAL; - goto out; - } - - dev_def_dom =3D iommu_get_def_domain_type(dev); + __iommu_group_for_each_dev(group, >ype, + probe_get_default_domain_type); if (!type) { /* * If the user hasn't requested any specific type of domain and * if the device supports both the domains, then default to the * domain the device was booted with */ - type =3D dev_def_dom ? : iommu_def_domain_type; - } else if (dev_def_dom && type !=3D dev_def_dom) { - dev_err_ratelimited(prev_dev, "Device cannot be in %s domain\n", + type =3D gtype.type ? : iommu_def_domain_type; + } else if (gtype.type && type !=3D gtype.type) { + dev_err_ratelimited(dev, "Device cannot be in %s domain\n", iommu_domain_type_str(type)); - ret =3D -EINVAL; - goto out; + return -EINVAL; } =20 /* * Switch to a new domain only if the requested domain type is different * from the existing default domain type */ - if (prev_dom->type =3D=3D type) { - ret =3D 0; - goto out; - } + if (prev_dom->type =3D=3D type) + return 0; + + group->default_domain =3D NULL; + group->domain =3D NULL; =20 /* Sets group->default_domain to the newly allocated domain */ ret =3D iommu_group_alloc_default_domain(group, dev, type); if (ret) - goto out; + goto restore_old_domain; =20 - ret =3D iommu_create_device_direct_mappings(group, dev); + ret =3D iommu_group_create_direct_mappings(group); if (ret) goto free_new_domain; =20 - ret =3D __iommu_attach_device(group->default_domain, dev); + ret =3D __iommu_attach_group(group->default_domain, group); if (ret) goto free_new_domain; =20 - group->domain =3D group->default_domain; iommu_domain_free(prev_dom); =20 return 0; =20 free_new_domain: iommu_domain_free(group->default_domain); +restore_old_domain: group->default_domain =3D prev_dom; group->domain =3D prev_dom; -out: + return ret; } =20 diff --git a/Documentation/ABI/testing/sysfs-kernel-iommu_groups b/Document= ation/ABI/testing/sysfs-kernel-iommu_groups index b15af6a5bc08..a42d4383d999 100644 --- a/Documentation/ABI/testing/sysfs-kernel-iommu_groups +++ b/Documentation/ABI/testing/sysfs-kernel-iommu_groups @@ -53,7 +53,6 @@ Description: /sys/kernel/iommu_groups//type shows= the type of default =20 The default domain type of a group may be modified only when =20 - - The group has only one device. - The device in the group is not bound to any device driver. So, the users must unbind the appropriate driver before changing the default domain type. --=20 2.34.1