From nobody Tue Oct 7 00:22:44 2025 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CB5EB256C6D for ; Wed, 16 Jul 2025 07:12:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.15 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752649968; cv=none; b=b89fpEaUMq7Ls9Rt+TN2bH/X/v009glvolU3TYadcrbrJCCent4McZu//0iD1z098y7WS4lUe5aobBWFnZ22iq496c55F3xpcFWhYOw886ld7CKQDFF/BvGEE3D9YUW3KIOY/0TqfB9tGov3+cv3qgwuerVnCkxGy5tteCUxR64= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752649968; c=relaxed/simple; bh=CsxQqXTtX64Ksavxo4Aj3iPKCc2CvcTVo51rF2K2wO0=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=eswts4SAUfY0RVwZcn/3eJc/tV15wvyAOvslm097q9NJLCAmhjpLxv81w01VqxPV+gyag/kD5wOGkl+Y8H1Nlu0TgivrJVHjVq8rrLrsUyEPxPLozoZb1iL7uIPGIZTVQVEYg46RTXhpaKvXyUnzsYbN+qVHX6KMQNveFfmE2fs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=T65K579R; arc=none smtp.client-ip=198.175.65.15 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="T65K579R" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1752649967; x=1784185967; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=CsxQqXTtX64Ksavxo4Aj3iPKCc2CvcTVo51rF2K2wO0=; b=T65K579Ravq6o+AEMfx1Eq/jyn+8Ax5Y3rFaPEzqlULnDs40bPHrKh04 Fm892cTuEzJLgFbFtkm8Gwkpgprr0Cf0qalkgIw8AhD+HwjlFSzg+ijgx rwcju3y3KNQt8Q3mnlm/BStRShgo0dxUAx3wvVHT5ZYdTwJ1HGB5MJGr8 gLeRYUB/hXIVjg2+xVnL7I1xCEVvy5hMpSIMc1C9YRowZfhkiwOPJMd/K 87DAglJR+OzuwY+W8obQAZ5NbD/e9wxyIhixtrGqWF37VeIxC7DWs8Ogm 6ZNLXXx9tvjLjzUVeKGCEE1G2260ITjN9SYTPmSXdKobTB2VuDkAo8cwS w==; X-CSE-ConnectionGUID: 42vYx6gTSJa6PY3xhfG/DQ== X-CSE-MsgGUID: Ha4GJ4h/QoyWI43tXaw+0Q== X-IronPort-AV: E=McAfee;i="6800,10657,11493"; a="58540784" X-IronPort-AV: E=Sophos;i="6.16,315,1744095600"; d="scan'208";a="58540784" Received: from fmviesa003.fm.intel.com ([10.60.135.143]) by orvoesa107.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Jul 2025 00:12:33 -0700 X-CSE-ConnectionGUID: yuUVXbKtQ9+UQNdIp/TIVg== X-CSE-MsgGUID: 6bmKm5ngTAuzwWCcnfiL1A== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.16,315,1744095600"; d="scan'208";a="161453621" Received: from yilunxu-optiplex-7050.sh.intel.com ([10.239.159.165]) by fmviesa003.fm.intel.com with ESMTP; 16 Jul 2025 00:12:30 -0700 From: Xu Yilun To: jgg@nvidia.com, jgg@ziepe.ca, kevin.tian@intel.com, will@kernel.org, aneesh.kumar@kernel.org Cc: iommu@lists.linux.dev, linux-kernel@vger.kernel.org, joro@8bytes.org, robin.murphy@arm.com, shuah@kernel.org, nicolinc@nvidia.com, aik@amd.com, dan.j.williams@intel.com, baolu.lu@linux.intel.com, yilun.xu@intel.com Subject: [PATCH v6 1/8] iommufd/viommu: Roll back to use iommufd_object_alloc() for vdevice Date: Wed, 16 Jul 2025 15:03:42 +0800 Message-Id: <20250716070349.1807226-2-yilun.xu@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20250716070349.1807226-1-yilun.xu@linux.intel.com> References: <20250716070349.1807226-1-yilun.xu@linux.intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" To solve the vdevice lifecycle issue, future patches make the vdevice allocation protected by lock. That will make _iommufd_object_alloc_ucmd() not applicable for vdevice. Roll back to use _iommufd_object_alloc() for preparation. Reviewed-by: Nicolin Chen Reviewed-by: Kevin Tian Reviewed-by: Jason Gunthorpe Tested-by: Nicolin Chen Signed-off-by: Xu Yilun --- drivers/iommu/iommufd/viommu.c | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/drivers/iommu/iommufd/viommu.c b/drivers/iommu/iommufd/viommu.c index 91339f799916..dcf8a85b9f6e 100644 --- a/drivers/iommu/iommufd/viommu.c +++ b/drivers/iommu/iommufd/viommu.c @@ -167,8 +167,8 @@ int iommufd_vdevice_alloc_ioctl(struct iommufd_ucmd *uc= md) vdev_size =3D viommu->ops->vdevice_size; } =20 - vdev =3D (struct iommufd_vdevice *)_iommufd_object_alloc_ucmd( - ucmd, vdev_size, IOMMUFD_OBJ_VDEVICE); + vdev =3D (struct iommufd_vdevice *)_iommufd_object_alloc( + ucmd->ictx, vdev_size, IOMMUFD_OBJ_VDEVICE); if (IS_ERR(vdev)) { rc =3D PTR_ERR(vdev); goto out_put_idev; @@ -183,18 +183,24 @@ int iommufd_vdevice_alloc_ioctl(struct iommufd_ucmd *= ucmd) curr =3D xa_cmpxchg(&viommu->vdevs, virt_id, NULL, vdev, GFP_KERNEL); if (curr) { rc =3D xa_err(curr) ?: -EEXIST; - goto out_put_idev; + goto out_abort; } =20 if (viommu->ops && viommu->ops->vdevice_init) { rc =3D viommu->ops->vdevice_init(vdev); if (rc) - goto out_put_idev; + goto out_abort; } =20 cmd->out_vdevice_id =3D vdev->obj.id; rc =3D iommufd_ucmd_respond(ucmd, sizeof(*cmd)); + if (rc) + goto out_abort; + iommufd_object_finalize(ucmd->ictx, &vdev->obj); + goto out_put_idev; =20 +out_abort: + iommufd_object_abort_and_destroy(ucmd->ictx, &vdev->obj); out_put_idev: iommufd_put_object(ucmd->ictx, &idev->obj); out_put_viommu: --=20 2.25.1 From nobody Tue Oct 7 00:22:44 2025 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 830B5289E16 for ; Wed, 16 Jul 2025 07:12:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.15 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752649970; cv=none; b=ZqCan5f2tg9xb4v5np6kgSsLueSHUWWIdFt/Mg2nxt7G2W7exQYFp4do16qJ+gsAnjbqeYtGs2sy3M4ckd4Jbw2XqQyc0maL2tSxcARQFE33Pm7OyUzG7Tewmul3M23kXLaSk998qvMluxUc2z1EpQXxRcinjcVq3sIuWPaFVTQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752649970; c=relaxed/simple; bh=M0kCbGdguK2I01sYIudQTuezVBCEY6JDjSwYgkYvvVE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=IBATtXkfO5Dl+rWuEQtWqiWnU/RzLP4o5fWfcfk3JMLCGqFpH/k9lcf2qSWKw2apDbmfJ+UNAKaNsr2RNgdcnNUeGDCMUchFWOi909Doovh7eMfGegsKNYdU/FuJ/i+scLl8hiJXA0xCu34MgjxyaBTIvHiOg4+/lN6ELAaOK3Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=MS7fB0N4; arc=none smtp.client-ip=198.175.65.15 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="MS7fB0N4" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1752649969; x=1784185969; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=M0kCbGdguK2I01sYIudQTuezVBCEY6JDjSwYgkYvvVE=; b=MS7fB0N4UwaHt+drPdy+iosriAkzhE5Uup8mcKQ7RVPaLT2f38az8znd 9AClt5O8z1YUqNyKwz8Q09dBaQB3L1s7ngmmgeEEIER2GnSAlKjVXD1I8 zDtZflGDgxPo4Dvrgs1pHXW8Ki+XyrSav5NL8fFmUxACl/aPiBVaCwndP MIhqchC5DOJEYleJrMb418ivQlCm0fbRCwAF9ogV66Iy1EfZ4YD4HZNz5 828I8iQknj9lJZ1VNhBp1e5ynhNhb+etOLEY8yFJcjV5DNXkW+T1giJqh XRbRnyCvpz2c6+kOarESEu5+D9JFCMaZ15cJWUisooiH29OqfQw0zEcAo Q==; X-CSE-ConnectionGUID: dV/COuAuRT2DaHFylPwo/A== X-CSE-MsgGUID: TZsN+ROrQSau3wS8sZ4k4g== X-IronPort-AV: E=McAfee;i="6800,10657,11493"; a="58540798" X-IronPort-AV: E=Sophos;i="6.16,315,1744095600"; d="scan'208";a="58540798" Received: from fmviesa003.fm.intel.com ([10.60.135.143]) by orvoesa107.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Jul 2025 00:12:37 -0700 X-CSE-ConnectionGUID: j9rzoG2zTP+vUbJ97F1ByA== X-CSE-MsgGUID: vo9QC6qAThyXxKIvxVVXKQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.16,315,1744095600"; d="scan'208";a="161453640" Received: from yilunxu-optiplex-7050.sh.intel.com ([10.239.159.165]) by fmviesa003.fm.intel.com with ESMTP; 16 Jul 2025 00:12:33 -0700 From: Xu Yilun To: jgg@nvidia.com, jgg@ziepe.ca, kevin.tian@intel.com, will@kernel.org, aneesh.kumar@kernel.org Cc: iommu@lists.linux.dev, linux-kernel@vger.kernel.org, joro@8bytes.org, robin.murphy@arm.com, shuah@kernel.org, nicolinc@nvidia.com, aik@amd.com, dan.j.williams@intel.com, baolu.lu@linux.intel.com, yilun.xu@intel.com Subject: [PATCH v6 2/8] iommufd: Add iommufd_object_tombstone_user() helper Date: Wed, 16 Jul 2025 15:03:43 +0800 Message-Id: <20250716070349.1807226-3-yilun.xu@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20250716070349.1807226-1-yilun.xu@linux.intel.com> References: <20250716070349.1807226-1-yilun.xu@linux.intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add the iommufd_object_tombstone_user() helper, which allows the caller to destroy an iommufd object created by userspace. This is useful on some destroy paths when the kernel caller finds the object should have been removed by userspace but is still alive. With this helper, the caller destroys the object but leave the object ID reserved (so called tombstone). The tombstone prevents repurposing the object ID without awareness of the original user. Since this happens for abnormal userspace behavior, for simplicity, the tombstoned object ID would be permanently leaked until iommufd_fops_release(). I.e. the original user gets an error when calling ioctl(IOMMU_DESTROY) on that ID. The first use case would be to ensure the iommufd_vdevice can't outlive the associated iommufd_device. Suggested-by: Jason Gunthorpe Reviewed-by: Lu Baolu Reviewed-by: Nicolin Chen Reviewed-by: Kevin Tian Reviewed-by: Jason Gunthorpe Co-developed-by: Aneesh Kumar K.V (Arm) Signed-off-by: Aneesh Kumar K.V (Arm) Tested-by: Nicolin Chen Signed-off-by: Xu Yilun --- drivers/iommu/iommufd/iommufd_private.h | 23 ++++++++++++++++++++++- drivers/iommu/iommufd/main.c | 24 +++++++++++++++++++++++- 2 files changed, 45 insertions(+), 2 deletions(-) diff --git a/drivers/iommu/iommufd/iommufd_private.h b/drivers/iommu/iommuf= d/iommufd_private.h index cd14163abdd1..149545060029 100644 --- a/drivers/iommu/iommufd/iommufd_private.h +++ b/drivers/iommu/iommufd/iommufd_private.h @@ -202,7 +202,8 @@ void iommufd_object_finalize(struct iommufd_ctx *ictx, struct iommufd_object *obj); =20 enum { - REMOVE_WAIT_SHORTTERM =3D 1, + REMOVE_WAIT_SHORTTERM =3D BIT(0), + REMOVE_OBJ_TOMBSTONE =3D BIT(1), }; int iommufd_object_remove(struct iommufd_ctx *ictx, struct iommufd_object *to_destroy, u32 id, @@ -228,6 +229,26 @@ static inline void iommufd_object_destroy_user(struct = iommufd_ctx *ictx, WARN_ON(ret); } =20 +/* + * Similar to iommufd_object_destroy_user(), except that the object ID is = left + * reserved/tombstoned. + */ +static inline void iommufd_object_tombstone_user(struct iommufd_ctx *ictx, + struct iommufd_object *obj) +{ + int ret; + + ret =3D iommufd_object_remove(ictx, obj, obj->id, + REMOVE_WAIT_SHORTTERM | REMOVE_OBJ_TOMBSTONE); + + /* + * If there is a bug and we couldn't destroy the object then we did put + * back the caller's users refcount and will eventually try to free it + * again during close. + */ + WARN_ON(ret); +} + /* * The HWPT allocated by autodomains is used in possibly many devices and * is automatically destroyed when its refcount reaches zero. diff --git a/drivers/iommu/iommufd/main.c b/drivers/iommu/iommufd/main.c index 69c2195e77ca..71135f0ec72d 100644 --- a/drivers/iommu/iommufd/main.c +++ b/drivers/iommu/iommufd/main.c @@ -225,7 +225,7 @@ int iommufd_object_remove(struct iommufd_ctx *ictx, goto err_xa; } =20 - xas_store(&xas, NULL); + xas_store(&xas, (flags & REMOVE_OBJ_TOMBSTONE) ? XA_ZERO_ENTRY : NULL); if (ictx->vfio_ioas =3D=3D container_of(obj, struct iommufd_ioas, obj)) ictx->vfio_ioas =3D NULL; xa_unlock(&ictx->objects); @@ -311,19 +311,41 @@ static int iommufd_fops_release(struct inode *inode, = struct file *filp) while (!xa_empty(&ictx->objects)) { unsigned int destroyed =3D 0; unsigned long index; + bool empty =3D true; =20 + /* + * We can't use xa_empty() to end the loop as the tombstones + * are stored as XA_ZERO_ENTRY in the xarray. However + * xa_for_each() automatically converts them to NULL and skips + * them causing xa_empty() to be kept false. Thus once + * xa_for_each() finds no further !NULL entries the loop is + * done. + */ xa_for_each(&ictx->objects, index, obj) { + empty =3D false; if (!refcount_dec_if_one(&obj->users)) continue; + destroyed++; xa_erase(&ictx->objects, index); iommufd_object_ops[obj->type].destroy(obj); kfree(obj); } + + if (empty) + break; + /* Bug related to users refcount */ if (WARN_ON(!destroyed)) break; } + + /* + * There may be some tombstones left over from + * iommufd_object_tombstone_user() + */ + xa_destroy(&ictx->objects); + WARN_ON(!xa_empty(&ictx->groups)); =20 mutex_destroy(&ictx->sw_msi_lock); --=20 2.25.1 From nobody Tue Oct 7 00:22:44 2025 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5C8C9289355 for ; Wed, 16 Jul 2025 07:12:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.15 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752649969; cv=none; b=RCzaMjXjPh0SGRn0jA0+BdN5STK7fqkD4jCmSvs4acVTvmjohYjVrg2dQN2ja/8KW9idhoDtCwHKYpV5OxqolFxr0RQy9cchSmLAS07sjgfKlG+u12VHjzdH6hXYcWdQ9eNscx/QvskEEIknOp1BbLsTrrcwwz+hSqbMW4qX53s= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752649969; c=relaxed/simple; bh=EPmB3ER5RAbb7WUUJI2oQq79I7IlOIoDuKOxpyoZWOY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=K0kZZHEEr8HO0g38jdPbvo4z3ITEAk2/Gxft/nY4yA9uOF6JELpLrxJttsxNf5ZOSA5mUboIapYsYuikxYc00Maho8tTr2ug8IMW2nVWvgO3NBV5W6UaVXP1wnxAwX9+PZuhkVHXtMVDcNdYmVLUzwguxd6ks/2vixaVk1g26Ag= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=czzlQeCk; arc=none smtp.client-ip=198.175.65.15 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="czzlQeCk" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1752649968; x=1784185968; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=EPmB3ER5RAbb7WUUJI2oQq79I7IlOIoDuKOxpyoZWOY=; b=czzlQeCkLDzTbax2cBzSz2SBOlGo8X0UKp15Tb+0GCbBBV6EowSpQ9OO dIIS2JTc/mXYRxMPangsOex2phO+JV5ZSH+1/9WXrquz65LxyedAVy5+t G1WD5BQFTeSKcfEoGzaXYrBjt/kUYK5yCsHDQKcOas2gpRHIs10ol+XXb 5pirhTFCgZTtx2eBS9/lYPpsUjMiv3NPJhIAWqZdToT2pZUM8Tgsh49Us X2uAn6VuUP5KFTPPmS8VzVfiuN5c9WszWf6w15pkd8jP77GFlBfoVp4BT 7YC4/zAsNOem9vwiGJcuPFoGYB+6RAuXyGXxSP01wsDMRVcziEe1ofjy8 g==; X-CSE-ConnectionGUID: hhgyIbtMQLCJ/MHU+T7MFg== X-CSE-MsgGUID: CcrV/9EJQJOPz96BeWjWpQ== X-IronPort-AV: E=McAfee;i="6800,10657,11493"; a="58540812" X-IronPort-AV: E=Sophos;i="6.16,315,1744095600"; d="scan'208";a="58540812" Received: from fmviesa003.fm.intel.com ([10.60.135.143]) by orvoesa107.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Jul 2025 00:12:40 -0700 X-CSE-ConnectionGUID: 2KHVZEhMRH+FoRgpVKFGsQ== X-CSE-MsgGUID: xxsexjI7RNiyxt6M6Vt78w== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.16,315,1744095600"; d="scan'208";a="161453643" Received: from yilunxu-optiplex-7050.sh.intel.com ([10.239.159.165]) by fmviesa003.fm.intel.com with ESMTP; 16 Jul 2025 00:12:36 -0700 From: Xu Yilun To: jgg@nvidia.com, jgg@ziepe.ca, kevin.tian@intel.com, will@kernel.org, aneesh.kumar@kernel.org Cc: iommu@lists.linux.dev, linux-kernel@vger.kernel.org, joro@8bytes.org, robin.murphy@arm.com, shuah@kernel.org, nicolinc@nvidia.com, aik@amd.com, dan.j.williams@intel.com, baolu.lu@linux.intel.com, yilun.xu@intel.com Subject: [PATCH v6 3/8] iommufd: Add a pre_destroy() op for objects Date: Wed, 16 Jul 2025 15:03:44 +0800 Message-Id: <20250716070349.1807226-4-yilun.xu@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20250716070349.1807226-1-yilun.xu@linux.intel.com> References: <20250716070349.1807226-1-yilun.xu@linux.intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add a pre_destroy() op which gives objects a chance to clear their short term users references before destruction. This op is intended for external driver created objects (e.g. idev) which does deterministic destruction. In order to manage the lifecycle of interrelated objects as well as the deterministic destruction (e.g. vdev can't outlive idev, and idev destruction can't fail), short term users references are allowed to live out of an ioctl execution. An immediate use case is, vdev holds idev's short term user reference until vdev destruction completes, idev leverages existing wait_shortterm mechanism to ensure it is destroyed after vdev. This extended usage makes the referenced object unable to just wait for its reference gone. It needs to actively trigger the reference removal, as well as prevent new references before wait. Should implement these work in pre_destroy(). Suggested-by: Jason Gunthorpe Reviewed-by: Kevin Tian Reviewed-by: Nicolin Chen Reviewed-by: Jason Gunthorpe Tested-by: Nicolin Chen Signed-off-by: Xu Yilun --- drivers/iommu/iommufd/main.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/drivers/iommu/iommufd/main.c b/drivers/iommu/iommufd/main.c index 71135f0ec72d..53085d24ce4a 100644 --- a/drivers/iommu/iommufd/main.c +++ b/drivers/iommu/iommufd/main.c @@ -23,6 +23,7 @@ #include "iommufd_test.h" =20 struct iommufd_object_ops { + void (*pre_destroy)(struct iommufd_object *obj); void (*destroy)(struct iommufd_object *obj); void (*abort)(struct iommufd_object *obj); }; @@ -160,6 +161,9 @@ static int iommufd_object_dec_wait_shortterm(struct iom= mufd_ctx *ictx, if (refcount_dec_and_test(&to_destroy->shortterm_users)) return 0; =20 + if (iommufd_object_ops[to_destroy->type].pre_destroy) + iommufd_object_ops[to_destroy->type].pre_destroy(to_destroy); + if (wait_event_timeout(ictx->destroy_wait, refcount_read(&to_destroy->shortterm_users) =3D=3D 0, msecs_to_jiffies(60000))) --=20 2.25.1 From nobody Tue Oct 7 00:22:44 2025 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CFAEE28A72F for ; Wed, 16 Jul 2025 07:12:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.15 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752649971; cv=none; b=DkIHODlnFjIBS0wPqZzZPbZ4lDiKZxS6OaVc9SyrbA2WD/PdvTZ3YcoENwQlgXdfZR6RjMFZVu1DrYsvgIhWYmDAkganqHLTgwV6TNbOahRgvOyvIlpwjhBCUXi15vG4UFP3QHDcTgNJKfAOulzezSJuVS7NwxlbfGHWisBa4vk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752649971; c=relaxed/simple; bh=kcR1ngZvWrLOyihkqx2rkHbIyYAELZHiy6H4piDtezk=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=XdNR39qN/ACfHEocJtRBTsRy+oFy5+cQxGXE49BLco7PMMfOfp+tB2tw1iVfgsnCD9eSkoRZUYFS9ebVWdaZJvnNLBbtFqV8glcpTvwXNL/+0DEVSrDn5Grvb3BQDQCBo7AqkcWBpVVnhta92q0HXuSbpAFygsOsbNhZmCgPIRM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=m2uAcpnE; arc=none smtp.client-ip=198.175.65.15 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="m2uAcpnE" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1752649970; x=1784185970; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=kcR1ngZvWrLOyihkqx2rkHbIyYAELZHiy6H4piDtezk=; b=m2uAcpnEJhI+SUt+EqvGrRRQHf6asFavWBy6QyvDHPcFWqIjOWXMpyxu OHR/XBF3fz4VK5qcbzlGTP6LrslHlfPkqQV1LJQUr9go2uzM0dy2G0gv4 ML0LS4dRnwQIqmf/L3SvjV4p5PAyomo0fvzLRliKA6kCu2zlV/AYfFYzg KSy8j7gYb2nAHUBGpjwWpCyQb6YvzW4JBCEnt8hF06mCFsZEsU3tmbHVc XEY08hEkV0nERxD5ANEdpZoNAIPKv5Y/2Kcyr20+/us4fRt0MDSLsrCDR +X3gjc2bYi7PJYkYzVF4I/pC+yXcUczPz39NfOV3+reOeKxEoW8uKoghU Q==; X-CSE-ConnectionGUID: yyS8Dy7MRTG5qslG4UjVQQ== X-CSE-MsgGUID: CgZwbHlIT82jxPMT6JGfIg== X-IronPort-AV: E=McAfee;i="6800,10657,11493"; a="58540829" X-IronPort-AV: E=Sophos;i="6.16,315,1744095600"; d="scan'208";a="58540829" Received: from fmviesa003.fm.intel.com ([10.60.135.143]) by orvoesa107.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Jul 2025 00:12:44 -0700 X-CSE-ConnectionGUID: jOa8olktSdmHQV4xQU2UsA== X-CSE-MsgGUID: MaY+qq4RRxqaEnW6OlunMA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.16,315,1744095600"; d="scan'208";a="161453649" Received: from yilunxu-optiplex-7050.sh.intel.com ([10.239.159.165]) by fmviesa003.fm.intel.com with ESMTP; 16 Jul 2025 00:12:40 -0700 From: Xu Yilun To: jgg@nvidia.com, jgg@ziepe.ca, kevin.tian@intel.com, will@kernel.org, aneesh.kumar@kernel.org Cc: iommu@lists.linux.dev, linux-kernel@vger.kernel.org, joro@8bytes.org, robin.murphy@arm.com, shuah@kernel.org, nicolinc@nvidia.com, aik@amd.com, dan.j.williams@intel.com, baolu.lu@linux.intel.com, yilun.xu@intel.com Subject: [PATCH v6 4/8] iommufd: Destroy vdevice on idevice destroy Date: Wed, 16 Jul 2025 15:03:45 +0800 Message-Id: <20250716070349.1807226-5-yilun.xu@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20250716070349.1807226-1-yilun.xu@linux.intel.com> References: <20250716070349.1807226-1-yilun.xu@linux.intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Destroy iommufd_vdevice (vdev) on iommufd_idevice (idev) destruction so that vdev can't outlive idev. idev represents the physical device bound to iommufd, while the vdev represents the virtual instance of the physical device in the VM. The lifecycle of the vdev should not be longer than idev. This doesn't cause real problem on existing use cases cause vdev doesn't impact the physical device, only provides virtualization information. But to extend vdev for Confidential Computing (CC), there are needs to do secure configuration for the vdev, e.g. TSM Bind/Unbind. These configurations should be rolled back on idev destroy, or the external driver (VFIO) functionality may be impact. The idev is created by external driver so its destruction can't fail. The idev implements pre_destroy() op to actively remove its associated vdev before destroying itself. There are 3 cases on idev pre_destroy(): 1. vdev is already destroyed by userspace. No extra handling needed. 2. vdev is still alive. Use iommufd_object_tombstone_user() to destroy vdev and tombstone the vdev ID. 3. vdev is being destroyed by userspace. The vdev ID is already freed, but vdev destroy handler is not completed. This requires multi-threads syncing - vdev holds idev's short term users reference until vdev destruction completes, idev leverages existing wait_shortterm mechanism for syncing. idev should also block any new reference to it after pre_destroy(), or the following wait shortterm would timeout. Introduce a 'destroying' flag, set it to true on idev pre_destroy(). Any attempt to reference idev should honor this flag under the protection of idev->igroup->lock. Originally-by: Nicolin Chen Suggested-by: Jason Gunthorpe Reviewed-by: Kevin Tian Reviewed-by: Nicolin Chen Reviewed-by: Jason Gunthorpe Co-developed-by: Aneesh Kumar K.V (Arm) Signed-off-by: Aneesh Kumar K.V (Arm) Tested-by: Nicolin Chen Signed-off-by: Xu Yilun --- drivers/iommu/iommufd/device.c | 51 ++++++++++++++++++++++++ drivers/iommu/iommufd/iommufd_private.h | 12 ++++++ drivers/iommu/iommufd/main.c | 2 + drivers/iommu/iommufd/viommu.c | 52 +++++++++++++++++++++++-- include/linux/iommufd.h | 1 + include/uapi/linux/iommufd.h | 5 +++ 6 files changed, 119 insertions(+), 4 deletions(-) diff --git a/drivers/iommu/iommufd/device.c b/drivers/iommu/iommufd/device.c index e2ba21c43ad2..ee6ff4caf398 100644 --- a/drivers/iommu/iommufd/device.c +++ b/drivers/iommu/iommufd/device.c @@ -137,6 +137,57 @@ static struct iommufd_group *iommufd_get_group(struct = iommufd_ctx *ictx, } } =20 +static void iommufd_device_remove_vdev(struct iommufd_device *idev) +{ + struct iommufd_vdevice *vdev; + + mutex_lock(&idev->igroup->lock); + /* prevent new references from vdev */ + idev->destroying =3D true; + /* vdev has been completely destroyed by userspace */ + if (!idev->vdev) + goto out_unlock; + + vdev =3D iommufd_get_vdevice(idev->ictx, idev->vdev->obj.id); + /* + * An ongoing vdev destroy ioctl has removed the vdev from the object + * xarray, but has not finished iommufd_vdevice_destroy() yet as it + * needs the same mutex. We exit the locking then wait on short term + * users for the vdev destruction. + */ + if (IS_ERR(vdev)) + goto out_unlock; + + /* Should never happen */ + if (WARN_ON(vdev !=3D idev->vdev)) { + iommufd_put_object(idev->ictx, &vdev->obj); + goto out_unlock; + } + + /* + * vdev is still alive. Hold a users refcount to prevent racing with + * userspace destruction, then use iommufd_object_tombstone_user() to + * destroy it and leave a tombstone. + */ + refcount_inc(&vdev->obj.users); + iommufd_put_object(idev->ictx, &vdev->obj); + mutex_unlock(&idev->igroup->lock); + iommufd_object_tombstone_user(idev->ictx, &vdev->obj); + return; + +out_unlock: + mutex_unlock(&idev->igroup->lock); +} + +void iommufd_device_pre_destroy(struct iommufd_object *obj) +{ + struct iommufd_device *idev =3D + container_of(obj, struct iommufd_device, obj); + + /* Release the short term users on this */ + iommufd_device_remove_vdev(idev); +} + void iommufd_device_destroy(struct iommufd_object *obj) { struct iommufd_device *idev =3D diff --git a/drivers/iommu/iommufd/iommufd_private.h b/drivers/iommu/iommuf= d/iommufd_private.h index 149545060029..5d6ea5395cfe 100644 --- a/drivers/iommu/iommufd/iommufd_private.h +++ b/drivers/iommu/iommufd/iommufd_private.h @@ -489,6 +489,8 @@ struct iommufd_device { /* always the physical device */ struct device *dev; bool enforce_cache_coherency; + struct iommufd_vdevice *vdev; + bool destroying; }; =20 static inline struct iommufd_device * @@ -499,6 +501,7 @@ iommufd_get_device(struct iommufd_ucmd *ucmd, u32 id) struct iommufd_device, obj); } =20 +void iommufd_device_pre_destroy(struct iommufd_object *obj); void iommufd_device_destroy(struct iommufd_object *obj); int iommufd_get_hw_info(struct iommufd_ucmd *ucmd); =20 @@ -687,9 +690,18 @@ int iommufd_viommu_alloc_ioctl(struct iommufd_ucmd *uc= md); void iommufd_viommu_destroy(struct iommufd_object *obj); int iommufd_vdevice_alloc_ioctl(struct iommufd_ucmd *ucmd); void iommufd_vdevice_destroy(struct iommufd_object *obj); +void iommufd_vdevice_abort(struct iommufd_object *obj); int iommufd_hw_queue_alloc_ioctl(struct iommufd_ucmd *ucmd); void iommufd_hw_queue_destroy(struct iommufd_object *obj); =20 +static inline struct iommufd_vdevice * +iommufd_get_vdevice(struct iommufd_ctx *ictx, u32 id) +{ + return container_of(iommufd_get_object(ictx, id, + IOMMUFD_OBJ_VDEVICE), + struct iommufd_vdevice, obj); +} + #ifdef CONFIG_IOMMUFD_TEST int iommufd_test(struct iommufd_ucmd *ucmd); void iommufd_selftest_destroy(struct iommufd_object *obj); diff --git a/drivers/iommu/iommufd/main.c b/drivers/iommu/iommufd/main.c index 53085d24ce4a..99c1aab3d396 100644 --- a/drivers/iommu/iommufd/main.c +++ b/drivers/iommu/iommufd/main.c @@ -655,6 +655,7 @@ static const struct iommufd_object_ops iommufd_object_o= ps[] =3D { .destroy =3D iommufd_access_destroy_object, }, [IOMMUFD_OBJ_DEVICE] =3D { + .pre_destroy =3D iommufd_device_pre_destroy, .destroy =3D iommufd_device_destroy, }, [IOMMUFD_OBJ_FAULT] =3D { @@ -676,6 +677,7 @@ static const struct iommufd_object_ops iommufd_object_o= ps[] =3D { }, [IOMMUFD_OBJ_VDEVICE] =3D { .destroy =3D iommufd_vdevice_destroy, + .abort =3D iommufd_vdevice_abort, }, [IOMMUFD_OBJ_VEVENTQ] =3D { .destroy =3D iommufd_veventq_destroy, diff --git a/drivers/iommu/iommufd/viommu.c b/drivers/iommu/iommufd/viommu.c index dcf8a85b9f6e..ecbae5091ffe 100644 --- a/drivers/iommu/iommufd/viommu.c +++ b/drivers/iommu/iommufd/viommu.c @@ -110,20 +110,37 @@ int iommufd_viommu_alloc_ioctl(struct iommufd_ucmd *u= cmd) return rc; } =20 -void iommufd_vdevice_destroy(struct iommufd_object *obj) +void iommufd_vdevice_abort(struct iommufd_object *obj) { struct iommufd_vdevice *vdev =3D container_of(obj, struct iommufd_vdevice, obj); struct iommufd_viommu *viommu =3D vdev->viommu; + struct iommufd_device *idev =3D vdev->idev; + + lockdep_assert_held(&idev->igroup->lock); =20 if (vdev->destroy) vdev->destroy(vdev); /* xa_cmpxchg is okay to fail if alloc failed xa_cmpxchg previously */ xa_cmpxchg(&viommu->vdevs, vdev->virt_id, vdev, NULL, GFP_KERNEL); refcount_dec(&viommu->obj.users); + idev->vdev =3D NULL; put_device(vdev->dev); } =20 +void iommufd_vdevice_destroy(struct iommufd_object *obj) +{ + struct iommufd_vdevice *vdev =3D + container_of(obj, struct iommufd_vdevice, obj); + struct iommufd_device *idev =3D vdev->idev; + struct iommufd_ctx *ictx =3D idev->ictx; + + mutex_lock(&idev->igroup->lock); + iommufd_vdevice_abort(obj); + mutex_unlock(&idev->igroup->lock); + iommufd_put_object(ictx, &idev->obj); +} + int iommufd_vdevice_alloc_ioctl(struct iommufd_ucmd *ucmd) { struct iommu_vdevice_alloc *cmd =3D ucmd->cmd; @@ -153,6 +170,17 @@ int iommufd_vdevice_alloc_ioctl(struct iommufd_ucmd *u= cmd) goto out_put_idev; } =20 + mutex_lock(&idev->igroup->lock); + if (idev->destroying) { + rc =3D -ENOENT; + goto out_unlock_igroup; + } + + if (idev->vdev) { + rc =3D -EEXIST; + goto out_unlock_igroup; + } + if (viommu->ops && viommu->ops->vdevice_size) { /* * It is a driver bug for: @@ -171,7 +199,7 @@ int iommufd_vdevice_alloc_ioctl(struct iommufd_ucmd *uc= md) ucmd->ictx, vdev_size, IOMMUFD_OBJ_VDEVICE); if (IS_ERR(vdev)) { rc =3D PTR_ERR(vdev); - goto out_put_idev; + goto out_unlock_igroup; } =20 vdev->virt_id =3D virt_id; @@ -179,6 +207,19 @@ int iommufd_vdevice_alloc_ioctl(struct iommufd_ucmd *u= cmd) get_device(idev->dev); vdev->viommu =3D viommu; refcount_inc(&viommu->obj.users); + /* + * A short term users reference is held on the idev so long as we have + * the pointer. iommufd_device_pre_destroy() will revoke it before the + * idev real destruction. + */ + vdev->idev =3D idev; + + /* + * iommufd_device_destroy() delays until idev->vdev is NULL before + * freeing the idev, which only happens once the vdev is finished + * destruction. + */ + idev->vdev =3D vdev; =20 curr =3D xa_cmpxchg(&viommu->vdevs, virt_id, NULL, vdev, GFP_KERNEL); if (curr) { @@ -197,12 +238,15 @@ int iommufd_vdevice_alloc_ioctl(struct iommufd_ucmd *= ucmd) if (rc) goto out_abort; iommufd_object_finalize(ucmd->ictx, &vdev->obj); - goto out_put_idev; + goto out_unlock_igroup; =20 out_abort: iommufd_object_abort_and_destroy(ucmd->ictx, &vdev->obj); +out_unlock_igroup: + mutex_unlock(&idev->igroup->lock); out_put_idev: - iommufd_put_object(ucmd->ictx, &idev->obj); + if (rc) + iommufd_put_object(ucmd->ictx, &idev->obj); out_put_viommu: iommufd_put_object(ucmd->ictx, &viommu->obj); return rc; diff --git a/include/linux/iommufd.h b/include/linux/iommufd.h index e3a0cd47384d..b88911026bc4 100644 --- a/include/linux/iommufd.h +++ b/include/linux/iommufd.h @@ -108,6 +108,7 @@ struct iommufd_viommu { struct iommufd_vdevice { struct iommufd_object obj; struct iommufd_viommu *viommu; + struct iommufd_device *idev; struct device *dev; =20 /* diff --git a/include/uapi/linux/iommufd.h b/include/uapi/linux/iommufd.h index 554aacf89ea7..c218c89e0e2e 100644 --- a/include/uapi/linux/iommufd.h +++ b/include/uapi/linux/iommufd.h @@ -1070,6 +1070,11 @@ struct iommu_viommu_alloc { * * Allocate a virtual device instance (for a physical device) against a vI= OMMU. * This instance holds the device's information (related to its vIOMMU) in= a VM. + * User should use IOMMU_DESTROY to destroy the virtual device before + * destroying the physical device (by closing vfio_cdev fd). Otherwise the + * virtual device would be forcibly destroyed on physical device destructi= on, + * its vdevice_id would be permanently leaked (unremovable & unreusable) u= ntil + * iommu fd closed. */ struct iommu_vdevice_alloc { __u32 size; --=20 2.25.1 From nobody Tue Oct 7 00:22:44 2025 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0FC5928AAE9 for ; Wed, 16 Jul 2025 07:12:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.15 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752649972; cv=none; b=g8exiICQ8jZT8F/eI24K/opaI5p/9T0I8gy+9JXxX7CNLzj/IQqjz51c/Gu3yAuXc7ASNoHkQB7SwyzjYZX+Jf1zVDfRxAT7diHyO8OJavvhjCfB1BgyKWiHl4PE7HCySBhHtHBmfc2vrt/WL3ra1pwTVFd8mLIy+y/6qQ635+g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752649972; c=relaxed/simple; bh=nJV35tCeC3FV4M9MblNrza0Hf2M8xmHmaqVlITtrpxY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=XVlIoc6blHLOxP7SPspoTMbrJtQDboEpcr3CCVZ34dhGSDWC2hdxawd6PhZ4SOUKiuxWPvzb9Nyi/RP8oCWFVOieq9G2IKMV44g+ok9yasDYY9zoGXGBccaitURDT2A2rKQngwag9zlLKLUD1JhmRsGV0XZ+eEMc2ONsDxS04No= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=MVKKdpb5; arc=none smtp.client-ip=198.175.65.15 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="MVKKdpb5" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1752649970; x=1784185970; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=nJV35tCeC3FV4M9MblNrza0Hf2M8xmHmaqVlITtrpxY=; b=MVKKdpb5ZIoNtQZ8KShAdKqITJSIZ7V4rLy9/kFBgUTbrJcOPiR5qmeW Fw39beAXoGFZsYoGJWVLxg+By4g+9OrT1sT6AkM73zKTms8tZmpQuCUjl eXcBC5ZNIM8R7ga8SOwNb70Dzxq6a8UeRJHAC+15jMIEld3/EebvykaLe oL6g/jIJUHfNjV2AaNmWd7UFq7W80LnqxGGo19O12ezx2kN2kCX4UwY6T lTaRX++IDsd+ycirAV7tfvRyV2rcCiPOhuATntwAabbkomsGu//D4Siy0 omDoA9JaQznUFOH0hnU4/DMboOtAqh2SjLur2HZBviafU+bVy37GhmdY7 Q==; X-CSE-ConnectionGUID: LlChxA7IR8Smf3Q6EE2j9A== X-CSE-MsgGUID: ET0Q7hTcTuC6oioSPpcTqA== X-IronPort-AV: E=McAfee;i="6800,10657,11493"; a="58540846" X-IronPort-AV: E=Sophos;i="6.16,315,1744095600"; d="scan'208";a="58540846" Received: from fmviesa003.fm.intel.com ([10.60.135.143]) by orvoesa107.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Jul 2025 00:12:47 -0700 X-CSE-ConnectionGUID: leKmR03/R0OGVM/cL5VZew== X-CSE-MsgGUID: WXL3r+GJT1K7Yv+lB5dVSQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.16,315,1744095600"; d="scan'208";a="161453654" Received: from yilunxu-optiplex-7050.sh.intel.com ([10.239.159.165]) by fmviesa003.fm.intel.com with ESMTP; 16 Jul 2025 00:12:43 -0700 From: Xu Yilun To: jgg@nvidia.com, jgg@ziepe.ca, kevin.tian@intel.com, will@kernel.org, aneesh.kumar@kernel.org Cc: iommu@lists.linux.dev, linux-kernel@vger.kernel.org, joro@8bytes.org, robin.murphy@arm.com, shuah@kernel.org, nicolinc@nvidia.com, aik@amd.com, dan.j.williams@intel.com, baolu.lu@linux.intel.com, yilun.xu@intel.com Subject: [PATCH v6 5/8] iommufd/vdevice: Remove struct device reference from struct vdevice Date: Wed, 16 Jul 2025 15:03:46 +0800 Message-Id: <20250716070349.1807226-6-yilun.xu@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20250716070349.1807226-1-yilun.xu@linux.intel.com> References: <20250716070349.1807226-1-yilun.xu@linux.intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Remove struct device *dev from struct vdevice. The dev pointer is the Plan B for vdevice to reference the physical device. As now vdev->idev is added without refcounting concern, just use vdev->idev->dev when needed. To avoid exposing struct iommufd_device in the public header, export a iommufd_vdevice_to_device() helper. Reviewed-by: Jason Gunthorpe Reviewed-by: Kevin Tian Reviewed-by: Nicolin Chen Co-developed-by: Nicolin Chen Signed-off-by: Nicolin Chen Tested-by: Nicolin Chen Signed-off-by: Xu Yilun --- drivers/iommu/arm/arm-smmu-v3/tegra241-cmdqv.c | 3 ++- drivers/iommu/iommufd/driver.c | 10 ++++++++-- drivers/iommu/iommufd/viommu.c | 3 --- include/linux/iommufd.h | 8 +++++++- 4 files changed, 17 insertions(+), 7 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/tegra241-cmdqv.c b/drivers/iommu= /arm/arm-smmu-v3/tegra241-cmdqv.c index eb90af5093d8..4c86eacd36b1 100644 --- a/drivers/iommu/arm/arm-smmu-v3/tegra241-cmdqv.c +++ b/drivers/iommu/arm/arm-smmu-v3/tegra241-cmdqv.c @@ -1218,7 +1218,8 @@ static void tegra241_vintf_destroy_vsid(struct iommuf= d_vdevice *vdev) =20 static int tegra241_vintf_init_vsid(struct iommufd_vdevice *vdev) { - struct arm_smmu_master *master =3D dev_iommu_priv_get(vdev->dev); + struct device *dev =3D iommufd_vdevice_to_device(vdev); + struct arm_smmu_master *master =3D dev_iommu_priv_get(dev); struct tegra241_vintf *vintf =3D viommu_to_vintf(vdev->viommu); struct tegra241_vintf_sid *vsid =3D vdev_to_vsid(vdev); struct arm_smmu_stream *stream =3D &master->streams[0]; diff --git a/drivers/iommu/iommufd/driver.c b/drivers/iommu/iommufd/driver.c index e4eae20bcd4e..6f1010da221c 100644 --- a/drivers/iommu/iommufd/driver.c +++ b/drivers/iommu/iommufd/driver.c @@ -83,6 +83,12 @@ void _iommufd_destroy_mmap(struct iommufd_ctx *ictx, } EXPORT_SYMBOL_NS_GPL(_iommufd_destroy_mmap, "IOMMUFD"); =20 +struct device *iommufd_vdevice_to_device(struct iommufd_vdevice *vdev) +{ + return vdev->idev->dev; +} +EXPORT_SYMBOL_NS_GPL(iommufd_vdevice_to_device, "IOMMUFD"); + /* Caller should xa_lock(&viommu->vdevs) to protect the return value */ struct device *iommufd_viommu_find_dev(struct iommufd_viommu *viommu, unsigned long vdev_id) @@ -92,7 +98,7 @@ struct device *iommufd_viommu_find_dev(struct iommufd_vio= mmu *viommu, lockdep_assert_held(&viommu->vdevs.xa_lock); =20 vdev =3D xa_load(&viommu->vdevs, vdev_id); - return vdev ? vdev->dev : NULL; + return vdev ? iommufd_vdevice_to_device(vdev) : NULL; } EXPORT_SYMBOL_NS_GPL(iommufd_viommu_find_dev, "IOMMUFD"); =20 @@ -109,7 +115,7 @@ int iommufd_viommu_get_vdev_id(struct iommufd_viommu *v= iommu, =20 xa_lock(&viommu->vdevs); xa_for_each(&viommu->vdevs, index, vdev) { - if (vdev->dev =3D=3D dev) { + if (iommufd_vdevice_to_device(vdev) =3D=3D dev) { *vdev_id =3D vdev->virt_id; rc =3D 0; break; diff --git a/drivers/iommu/iommufd/viommu.c b/drivers/iommu/iommufd/viommu.c index ecbae5091ffe..6cf0bd5d8f08 100644 --- a/drivers/iommu/iommufd/viommu.c +++ b/drivers/iommu/iommufd/viommu.c @@ -125,7 +125,6 @@ void iommufd_vdevice_abort(struct iommufd_object *obj) xa_cmpxchg(&viommu->vdevs, vdev->virt_id, vdev, NULL, GFP_KERNEL); refcount_dec(&viommu->obj.users); idev->vdev =3D NULL; - put_device(vdev->dev); } =20 void iommufd_vdevice_destroy(struct iommufd_object *obj) @@ -203,8 +202,6 @@ int iommufd_vdevice_alloc_ioctl(struct iommufd_ucmd *uc= md) } =20 vdev->virt_id =3D virt_id; - vdev->dev =3D idev->dev; - get_device(idev->dev); vdev->viommu =3D viommu; refcount_inc(&viommu->obj.users); /* diff --git a/include/linux/iommufd.h b/include/linux/iommufd.h index b88911026bc4..810e4d8ac912 100644 --- a/include/linux/iommufd.h +++ b/include/linux/iommufd.h @@ -109,7 +109,6 @@ struct iommufd_vdevice { struct iommufd_object obj; struct iommufd_viommu *viommu; struct iommufd_device *idev; - struct device *dev; =20 /* * Virtual device ID per vIOMMU, e.g. vSID of ARM SMMUv3, vDeviceID of @@ -261,6 +260,7 @@ int _iommufd_alloc_mmap(struct iommufd_ctx *ictx, struc= t iommufd_object *owner, unsigned long *offset); void _iommufd_destroy_mmap(struct iommufd_ctx *ictx, struct iommufd_object *owner, unsigned long offset); +struct device *iommufd_vdevice_to_device(struct iommufd_vdevice *vdev); struct device *iommufd_viommu_find_dev(struct iommufd_viommu *viommu, unsigned long vdev_id); int iommufd_viommu_get_vdev_id(struct iommufd_viommu *viommu, @@ -295,6 +295,12 @@ static inline void _iommufd_destroy_mmap(struct iommuf= d_ctx *ictx, { } =20 +static inline struct device * +iommufd_vdevice_to_device(struct iommufd_vdevice *vdev) +{ + return NULL; +} + static inline struct device * iommufd_viommu_find_dev(struct iommufd_viommu *viommu, unsigned long vdev_= id) { --=20 2.25.1 From nobody Tue Oct 7 00:22:44 2025 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4F98528BAB9 for ; Wed, 16 Jul 2025 07:12:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.15 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752649973; cv=none; b=Uwo7zFxioeVWMRvXW19+g7eKa2hHN6/NsShATiZKmkM+a6JELxBqcdutu5/I85JJr8qc3AkZIPlQbu5FoiTT6sQOUmcILr+8q02VxvXETGs7zko7BHlDKT97vHlN10mLjlSpmneHHcszQwG+yAgcS+0lwfbyTl+iwdTU2b2H8Ms= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752649973; c=relaxed/simple; bh=tm3o1OKvzbmwZmyZhLkn0o0jzJ+ODPPxlZSk76PSRB0=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=d2xaY0n2VcJJ9QedLMBE73RqqZ0/yo4j5KppSvlLoIIqwaMENKTnmBE300V2OLdFleX0c5gylYJZDJM24QCygoDIoNJrblwRRz/pgkKEJKL+OFtf7cNJO4ldOc6KdsMyqisdzzr6/jkq6JeoJTnUvxigKXHH0pvDcpP3x+ZvwTk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=FjOAA60c; arc=none smtp.client-ip=198.175.65.15 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="FjOAA60c" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1752649971; x=1784185971; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=tm3o1OKvzbmwZmyZhLkn0o0jzJ+ODPPxlZSk76PSRB0=; b=FjOAA60cL8pq+cOwLematfHwwlNw0gXP1EqdJaEv2UC/lvxlmvB3BaaJ +ofvkHMAsL5b7ukdRqH9E1KXMmQAewrEXIViQ/Rv+fDHrF4M8eKmWbRjs mr4sqaey3xV6oZe+g+8Sh55sn/TDSYSuYgy+G/F/PuFG5KP7IxMWvW04g 7hwb2JIbsm/AU9TbFpZXhMBO6ffeG4PkPVsc7bsbzgiwlzSn+DzjVQlav /diGE2BdoQasjtfTTRBvNlqA+JfVIuteHXW9IU5fIGFv+bJUIvrRu2FK6 XnTmcaf6+tBgdpuujv6+RJOHuM6yFYFsK7t0JQ/nvUIN7/GUyoI6aZVF/ w==; X-CSE-ConnectionGUID: LmDYe7WCQtCztOQNypUrQA== X-CSE-MsgGUID: +IF5s1VYS9ePntdrNKlZ6g== X-IronPort-AV: E=McAfee;i="6800,10657,11493"; a="58540864" X-IronPort-AV: E=Sophos;i="6.16,315,1744095600"; d="scan'208";a="58540864" Received: from fmviesa003.fm.intel.com ([10.60.135.143]) by orvoesa107.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Jul 2025 00:12:51 -0700 X-CSE-ConnectionGUID: BQgT1JBATPWHY6+/zRt/jQ== X-CSE-MsgGUID: 8gh4cKRdQLWEkqBbCwX8vQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.16,315,1744095600"; d="scan'208";a="161453660" Received: from yilunxu-optiplex-7050.sh.intel.com ([10.239.159.165]) by fmviesa003.fm.intel.com with ESMTP; 16 Jul 2025 00:12:47 -0700 From: Xu Yilun To: jgg@nvidia.com, jgg@ziepe.ca, kevin.tian@intel.com, will@kernel.org, aneesh.kumar@kernel.org Cc: iommu@lists.linux.dev, linux-kernel@vger.kernel.org, joro@8bytes.org, robin.murphy@arm.com, shuah@kernel.org, nicolinc@nvidia.com, aik@amd.com, dan.j.williams@intel.com, baolu.lu@linux.intel.com, yilun.xu@intel.com Subject: [PATCH v6 6/8] iommufd/selftest: Explicitly skip tests for inapplicable variant Date: Wed, 16 Jul 2025 15:03:47 +0800 Message-Id: <20250716070349.1807226-7-yilun.xu@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20250716070349.1807226-1-yilun.xu@linux.intel.com> References: <20250716070349.1807226-1-yilun.xu@linux.intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" no_viommu is not applicable for some viommu/vdevice tests. Explicitly report the skipping, don't do it silently. Opportunistically adjust the line wrappings after the indentation changes using git clang-format. Only add the prints. No functional change intended. Reviewed-by: Jason Gunthorpe Reviewed-by: Kevin Tian Reviewed-by: Nicolin Chen Tested-by: Nicolin Chen Signed-off-by: Xu Yilun --- tools/testing/selftests/iommu/iommufd.c | 363 ++++++++++++------------ 1 file changed, 176 insertions(+), 187 deletions(-) diff --git a/tools/testing/selftests/iommu/iommufd.c b/tools/testing/selfte= sts/iommu/iommufd.c index d59d48022a24..1b629bedeb1c 100644 --- a/tools/testing/selftests/iommu/iommufd.c +++ b/tools/testing/selftests/iommu/iommufd.c @@ -2779,35 +2779,32 @@ TEST_F(iommufd_viommu, viommu_alloc_nested_iopf) uint32_t fault_fd; uint32_t vdev_id; =20 - if (self->device_id) { - test_ioctl_fault_alloc(&fault_id, &fault_fd); - test_err_hwpt_alloc_iopf( - ENOENT, dev_id, viommu_id, UINT32_MAX, - IOMMU_HWPT_FAULT_ID_VALID, &iopf_hwpt_id, - IOMMU_HWPT_DATA_SELFTEST, &data, sizeof(data)); - test_err_hwpt_alloc_iopf( - EOPNOTSUPP, dev_id, viommu_id, fault_id, - IOMMU_HWPT_FAULT_ID_VALID | (1 << 31), &iopf_hwpt_id, - IOMMU_HWPT_DATA_SELFTEST, &data, sizeof(data)); - test_cmd_hwpt_alloc_iopf( - dev_id, viommu_id, fault_id, IOMMU_HWPT_FAULT_ID_VALID, - &iopf_hwpt_id, IOMMU_HWPT_DATA_SELFTEST, &data, - sizeof(data)); + if (!dev_id) + SKIP(return, "Skipping test for variant no_viommu"); =20 - /* Must allocate vdevice before attaching to a nested hwpt */ - test_err_mock_domain_replace(ENOENT, self->stdev_id, - iopf_hwpt_id); - test_cmd_vdevice_alloc(viommu_id, dev_id, 0x99, &vdev_id); - test_cmd_mock_domain_replace(self->stdev_id, iopf_hwpt_id); - EXPECT_ERRNO(EBUSY, - _test_ioctl_destroy(self->fd, iopf_hwpt_id)); - test_cmd_trigger_iopf(dev_id, fault_fd); + test_ioctl_fault_alloc(&fault_id, &fault_fd); + test_err_hwpt_alloc_iopf(ENOENT, dev_id, viommu_id, UINT32_MAX, + IOMMU_HWPT_FAULT_ID_VALID, &iopf_hwpt_id, + IOMMU_HWPT_DATA_SELFTEST, &data, sizeof(data)); + test_err_hwpt_alloc_iopf(EOPNOTSUPP, dev_id, viommu_id, fault_id, + IOMMU_HWPT_FAULT_ID_VALID | (1 << 31), + &iopf_hwpt_id, IOMMU_HWPT_DATA_SELFTEST, &data, + sizeof(data)); + test_cmd_hwpt_alloc_iopf(dev_id, viommu_id, fault_id, + IOMMU_HWPT_FAULT_ID_VALID, &iopf_hwpt_id, + IOMMU_HWPT_DATA_SELFTEST, &data, sizeof(data)); =20 - test_cmd_mock_domain_replace(self->stdev_id, self->ioas_id); - test_ioctl_destroy(iopf_hwpt_id); - close(fault_fd); - test_ioctl_destroy(fault_id); - } + /* Must allocate vdevice before attaching to a nested hwpt */ + test_err_mock_domain_replace(ENOENT, self->stdev_id, iopf_hwpt_id); + test_cmd_vdevice_alloc(viommu_id, dev_id, 0x99, &vdev_id); + test_cmd_mock_domain_replace(self->stdev_id, iopf_hwpt_id); + EXPECT_ERRNO(EBUSY, _test_ioctl_destroy(self->fd, iopf_hwpt_id)); + test_cmd_trigger_iopf(dev_id, fault_fd); + + test_cmd_mock_domain_replace(self->stdev_id, self->ioas_id); + test_ioctl_destroy(iopf_hwpt_id); + close(fault_fd); + test_ioctl_destroy(fault_id); } =20 TEST_F(iommufd_viommu, viommu_alloc_with_data) @@ -2902,169 +2899,161 @@ TEST_F(iommufd_viommu, vdevice_cache) uint32_t vdev_id =3D 0; uint32_t num_inv; =20 - if (dev_id) { - test_cmd_vdevice_alloc(viommu_id, dev_id, 0x99, &vdev_id); - - test_cmd_dev_check_cache_all(dev_id, - IOMMU_TEST_DEV_CACHE_DEFAULT); - - /* Check data_type by passing zero-length array */ - num_inv =3D 0; - test_cmd_viommu_invalidate(viommu_id, inv_reqs, - sizeof(*inv_reqs), &num_inv); - assert(!num_inv); - - /* Negative test: Invalid data_type */ - num_inv =3D 1; - test_err_viommu_invalidate(EINVAL, viommu_id, inv_reqs, - IOMMU_VIOMMU_INVALIDATE_DATA_SELFTEST_INVALID, - sizeof(*inv_reqs), &num_inv); - assert(!num_inv); - - /* Negative test: structure size sanity */ - num_inv =3D 1; - test_err_viommu_invalidate(EINVAL, viommu_id, inv_reqs, - IOMMU_VIOMMU_INVALIDATE_DATA_SELFTEST, - sizeof(*inv_reqs) + 1, &num_inv); - assert(!num_inv); - - num_inv =3D 1; - test_err_viommu_invalidate(EINVAL, viommu_id, inv_reqs, - IOMMU_VIOMMU_INVALIDATE_DATA_SELFTEST, - 1, &num_inv); - assert(!num_inv); - - /* Negative test: invalid flag is passed */ - num_inv =3D 1; - inv_reqs[0].flags =3D 0xffffffff; - inv_reqs[0].vdev_id =3D 0x99; - test_err_viommu_invalidate(EOPNOTSUPP, viommu_id, inv_reqs, - IOMMU_VIOMMU_INVALIDATE_DATA_SELFTEST, - sizeof(*inv_reqs), &num_inv); - assert(!num_inv); - - /* Negative test: invalid data_uptr when array is not empty */ - num_inv =3D 1; - inv_reqs[0].flags =3D 0; - inv_reqs[0].vdev_id =3D 0x99; - test_err_viommu_invalidate(EINVAL, viommu_id, NULL, - IOMMU_VIOMMU_INVALIDATE_DATA_SELFTEST, - sizeof(*inv_reqs), &num_inv); - assert(!num_inv); - - /* Negative test: invalid entry_len when array is not empty */ - num_inv =3D 1; - inv_reqs[0].flags =3D 0; - inv_reqs[0].vdev_id =3D 0x99; - test_err_viommu_invalidate(EINVAL, viommu_id, inv_reqs, - IOMMU_VIOMMU_INVALIDATE_DATA_SELFTEST, - 0, &num_inv); - assert(!num_inv); - - /* Negative test: invalid cache_id */ - num_inv =3D 1; - inv_reqs[0].flags =3D 0; - inv_reqs[0].vdev_id =3D 0x99; - inv_reqs[0].cache_id =3D MOCK_DEV_CACHE_ID_MAX + 1; - test_err_viommu_invalidate(EINVAL, viommu_id, inv_reqs, - IOMMU_VIOMMU_INVALIDATE_DATA_SELFTEST, - sizeof(*inv_reqs), &num_inv); - assert(!num_inv); + if (!dev_id) + SKIP(return, "Skipping test for variant no_viommu"); =20 - /* Negative test: invalid vdev_id */ - num_inv =3D 1; - inv_reqs[0].flags =3D 0; - inv_reqs[0].vdev_id =3D 0x9; - inv_reqs[0].cache_id =3D 0; - test_err_viommu_invalidate(EINVAL, viommu_id, inv_reqs, - IOMMU_VIOMMU_INVALIDATE_DATA_SELFTEST, - sizeof(*inv_reqs), &num_inv); - assert(!num_inv); + test_cmd_vdevice_alloc(viommu_id, dev_id, 0x99, &vdev_id); + + test_cmd_dev_check_cache_all(dev_id, IOMMU_TEST_DEV_CACHE_DEFAULT); + + /* Check data_type by passing zero-length array */ + num_inv =3D 0; + test_cmd_viommu_invalidate(viommu_id, inv_reqs, sizeof(*inv_reqs), + &num_inv); + assert(!num_inv); + + /* Negative test: Invalid data_type */ + num_inv =3D 1; + test_err_viommu_invalidate(EINVAL, viommu_id, inv_reqs, + IOMMU_VIOMMU_INVALIDATE_DATA_SELFTEST_INVALID, + sizeof(*inv_reqs), &num_inv); + assert(!num_inv); + + /* Negative test: structure size sanity */ + num_inv =3D 1; + test_err_viommu_invalidate(EINVAL, viommu_id, inv_reqs, + IOMMU_VIOMMU_INVALIDATE_DATA_SELFTEST, + sizeof(*inv_reqs) + 1, &num_inv); + assert(!num_inv); + + num_inv =3D 1; + test_err_viommu_invalidate(EINVAL, viommu_id, inv_reqs, + IOMMU_VIOMMU_INVALIDATE_DATA_SELFTEST, 1, + &num_inv); + assert(!num_inv); + + /* Negative test: invalid flag is passed */ + num_inv =3D 1; + inv_reqs[0].flags =3D 0xffffffff; + inv_reqs[0].vdev_id =3D 0x99; + test_err_viommu_invalidate(EOPNOTSUPP, viommu_id, inv_reqs, + IOMMU_VIOMMU_INVALIDATE_DATA_SELFTEST, + sizeof(*inv_reqs), &num_inv); + assert(!num_inv); + + /* Negative test: invalid data_uptr when array is not empty */ + num_inv =3D 1; + inv_reqs[0].flags =3D 0; + inv_reqs[0].vdev_id =3D 0x99; + test_err_viommu_invalidate(EINVAL, viommu_id, NULL, + IOMMU_VIOMMU_INVALIDATE_DATA_SELFTEST, + sizeof(*inv_reqs), &num_inv); + assert(!num_inv); + + /* Negative test: invalid entry_len when array is not empty */ + num_inv =3D 1; + inv_reqs[0].flags =3D 0; + inv_reqs[0].vdev_id =3D 0x99; + test_err_viommu_invalidate(EINVAL, viommu_id, inv_reqs, + IOMMU_VIOMMU_INVALIDATE_DATA_SELFTEST, 0, + &num_inv); + assert(!num_inv); + + /* Negative test: invalid cache_id */ + num_inv =3D 1; + inv_reqs[0].flags =3D 0; + inv_reqs[0].vdev_id =3D 0x99; + inv_reqs[0].cache_id =3D MOCK_DEV_CACHE_ID_MAX + 1; + test_err_viommu_invalidate(EINVAL, viommu_id, inv_reqs, + IOMMU_VIOMMU_INVALIDATE_DATA_SELFTEST, + sizeof(*inv_reqs), &num_inv); + assert(!num_inv); + + /* Negative test: invalid vdev_id */ + num_inv =3D 1; + inv_reqs[0].flags =3D 0; + inv_reqs[0].vdev_id =3D 0x9; + inv_reqs[0].cache_id =3D 0; + test_err_viommu_invalidate(EINVAL, viommu_id, inv_reqs, + IOMMU_VIOMMU_INVALIDATE_DATA_SELFTEST, + sizeof(*inv_reqs), &num_inv); + assert(!num_inv); =20 - /* - * Invalidate the 1st cache entry but fail the 2nd request - * due to invalid flags configuration in the 2nd request. - */ - num_inv =3D 2; - inv_reqs[0].flags =3D 0; - inv_reqs[0].vdev_id =3D 0x99; - inv_reqs[0].cache_id =3D 0; - inv_reqs[1].flags =3D 0xffffffff; - inv_reqs[1].vdev_id =3D 0x99; - inv_reqs[1].cache_id =3D 1; - test_err_viommu_invalidate(EOPNOTSUPP, viommu_id, inv_reqs, - IOMMU_VIOMMU_INVALIDATE_DATA_SELFTEST, - sizeof(*inv_reqs), &num_inv); - assert(num_inv =3D=3D 1); - test_cmd_dev_check_cache(dev_id, 0, 0); - test_cmd_dev_check_cache(dev_id, 1, - IOMMU_TEST_DEV_CACHE_DEFAULT); - test_cmd_dev_check_cache(dev_id, 2, - IOMMU_TEST_DEV_CACHE_DEFAULT); - test_cmd_dev_check_cache(dev_id, 3, - IOMMU_TEST_DEV_CACHE_DEFAULT); - - /* - * Invalidate the 1st cache entry but fail the 2nd request - * due to invalid cache_id configuration in the 2nd request. - */ - num_inv =3D 2; - inv_reqs[0].flags =3D 0; - inv_reqs[0].vdev_id =3D 0x99; - inv_reqs[0].cache_id =3D 0; - inv_reqs[1].flags =3D 0; - inv_reqs[1].vdev_id =3D 0x99; - inv_reqs[1].cache_id =3D MOCK_DEV_CACHE_ID_MAX + 1; - test_err_viommu_invalidate(EINVAL, viommu_id, inv_reqs, - IOMMU_VIOMMU_INVALIDATE_DATA_SELFTEST, - sizeof(*inv_reqs), &num_inv); - assert(num_inv =3D=3D 1); - test_cmd_dev_check_cache(dev_id, 0, 0); - test_cmd_dev_check_cache(dev_id, 1, - IOMMU_TEST_DEV_CACHE_DEFAULT); - test_cmd_dev_check_cache(dev_id, 2, - IOMMU_TEST_DEV_CACHE_DEFAULT); - test_cmd_dev_check_cache(dev_id, 3, - IOMMU_TEST_DEV_CACHE_DEFAULT); - - /* Invalidate the 2nd cache entry and verify */ - num_inv =3D 1; - inv_reqs[0].flags =3D 0; - inv_reqs[0].vdev_id =3D 0x99; - inv_reqs[0].cache_id =3D 1; - test_cmd_viommu_invalidate(viommu_id, inv_reqs, - sizeof(*inv_reqs), &num_inv); - assert(num_inv =3D=3D 1); - test_cmd_dev_check_cache(dev_id, 0, 0); - test_cmd_dev_check_cache(dev_id, 1, 0); - test_cmd_dev_check_cache(dev_id, 2, - IOMMU_TEST_DEV_CACHE_DEFAULT); - test_cmd_dev_check_cache(dev_id, 3, - IOMMU_TEST_DEV_CACHE_DEFAULT); - - /* Invalidate the 3rd and 4th cache entries and verify */ - num_inv =3D 2; - inv_reqs[0].flags =3D 0; - inv_reqs[0].vdev_id =3D 0x99; - inv_reqs[0].cache_id =3D 2; - inv_reqs[1].flags =3D 0; - inv_reqs[1].vdev_id =3D 0x99; - inv_reqs[1].cache_id =3D 3; - test_cmd_viommu_invalidate(viommu_id, inv_reqs, - sizeof(*inv_reqs), &num_inv); - assert(num_inv =3D=3D 2); - test_cmd_dev_check_cache_all(dev_id, 0); + /* + * Invalidate the 1st cache entry but fail the 2nd request + * due to invalid flags configuration in the 2nd request. + */ + num_inv =3D 2; + inv_reqs[0].flags =3D 0; + inv_reqs[0].vdev_id =3D 0x99; + inv_reqs[0].cache_id =3D 0; + inv_reqs[1].flags =3D 0xffffffff; + inv_reqs[1].vdev_id =3D 0x99; + inv_reqs[1].cache_id =3D 1; + test_err_viommu_invalidate(EOPNOTSUPP, viommu_id, inv_reqs, + IOMMU_VIOMMU_INVALIDATE_DATA_SELFTEST, + sizeof(*inv_reqs), &num_inv); + assert(num_inv =3D=3D 1); + test_cmd_dev_check_cache(dev_id, 0, 0); + test_cmd_dev_check_cache(dev_id, 1, IOMMU_TEST_DEV_CACHE_DEFAULT); + test_cmd_dev_check_cache(dev_id, 2, IOMMU_TEST_DEV_CACHE_DEFAULT); + test_cmd_dev_check_cache(dev_id, 3, IOMMU_TEST_DEV_CACHE_DEFAULT); =20 - /* Invalidate all cache entries for nested_dev_id[1] and verify */ - num_inv =3D 1; - inv_reqs[0].vdev_id =3D 0x99; - inv_reqs[0].flags =3D IOMMU_TEST_INVALIDATE_FLAG_ALL; - test_cmd_viommu_invalidate(viommu_id, inv_reqs, - sizeof(*inv_reqs), &num_inv); - assert(num_inv =3D=3D 1); - test_cmd_dev_check_cache_all(dev_id, 0); - test_ioctl_destroy(vdev_id); - } + /* + * Invalidate the 1st cache entry but fail the 2nd request + * due to invalid cache_id configuration in the 2nd request. + */ + num_inv =3D 2; + inv_reqs[0].flags =3D 0; + inv_reqs[0].vdev_id =3D 0x99; + inv_reqs[0].cache_id =3D 0; + inv_reqs[1].flags =3D 0; + inv_reqs[1].vdev_id =3D 0x99; + inv_reqs[1].cache_id =3D MOCK_DEV_CACHE_ID_MAX + 1; + test_err_viommu_invalidate(EINVAL, viommu_id, inv_reqs, + IOMMU_VIOMMU_INVALIDATE_DATA_SELFTEST, + sizeof(*inv_reqs), &num_inv); + assert(num_inv =3D=3D 1); + test_cmd_dev_check_cache(dev_id, 0, 0); + test_cmd_dev_check_cache(dev_id, 1, IOMMU_TEST_DEV_CACHE_DEFAULT); + test_cmd_dev_check_cache(dev_id, 2, IOMMU_TEST_DEV_CACHE_DEFAULT); + test_cmd_dev_check_cache(dev_id, 3, IOMMU_TEST_DEV_CACHE_DEFAULT); + + /* Invalidate the 2nd cache entry and verify */ + num_inv =3D 1; + inv_reqs[0].flags =3D 0; + inv_reqs[0].vdev_id =3D 0x99; + inv_reqs[0].cache_id =3D 1; + test_cmd_viommu_invalidate(viommu_id, inv_reqs, sizeof(*inv_reqs), + &num_inv); + assert(num_inv =3D=3D 1); + test_cmd_dev_check_cache(dev_id, 0, 0); + test_cmd_dev_check_cache(dev_id, 1, 0); + test_cmd_dev_check_cache(dev_id, 2, IOMMU_TEST_DEV_CACHE_DEFAULT); + test_cmd_dev_check_cache(dev_id, 3, IOMMU_TEST_DEV_CACHE_DEFAULT); + + /* Invalidate the 3rd and 4th cache entries and verify */ + num_inv =3D 2; + inv_reqs[0].flags =3D 0; + inv_reqs[0].vdev_id =3D 0x99; + inv_reqs[0].cache_id =3D 2; + inv_reqs[1].flags =3D 0; + inv_reqs[1].vdev_id =3D 0x99; + inv_reqs[1].cache_id =3D 3; + test_cmd_viommu_invalidate(viommu_id, inv_reqs, sizeof(*inv_reqs), + &num_inv); + assert(num_inv =3D=3D 2); + test_cmd_dev_check_cache_all(dev_id, 0); + + /* Invalidate all cache entries for nested_dev_id[1] and verify */ + num_inv =3D 1; + inv_reqs[0].vdev_id =3D 0x99; + inv_reqs[0].flags =3D IOMMU_TEST_INVALIDATE_FLAG_ALL; + test_cmd_viommu_invalidate(viommu_id, inv_reqs, sizeof(*inv_reqs), + &num_inv); + assert(num_inv =3D=3D 1); + test_cmd_dev_check_cache_all(dev_id, 0); + test_ioctl_destroy(vdev_id); } =20 TEST_F(iommufd_viommu, hw_queue) --=20 2.25.1 From nobody Tue Oct 7 00:22:44 2025 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 85F8128E616 for ; Wed, 16 Jul 2025 07:12:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.15 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752649976; cv=none; b=IjzYy9JrjpcM5VtNxTF1vb3cKbGywtnHED5LBP+Ics7KG7NdrcnuSi3LuB/AkJbCd7xUAH6I6Gza0MR1Kv0CDQBYpuMuIFlZQAJ97Aqf6uheoH/FwHIoHzx+hcoCnSdZ69vKW27kxjRbWCnt0aRjgF9gIEfNqzHxLfVYGdPVbyo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752649976; c=relaxed/simple; bh=UFzZVPewCANHosCYHCZyErsVJrZf28BY92qL2etK5SU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=UY1NOh9nK7Lx1/5iB5FNX6QgDbF2d5U/74xPWo86pJ7O9llZHuVn4kcJS5TE3XCNzeiOvxJxkGP39qmvKFItkqi1YwreU44kYsEwRemQYzyNgsXLmNQho/k3K5k1/u6ldRm8fVMG8OnkjUKIVQmTLIyul40Mz7pI3BkN2i2tHXA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=eMzUPb1f; arc=none smtp.client-ip=198.175.65.15 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="eMzUPb1f" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1752649975; x=1784185975; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=UFzZVPewCANHosCYHCZyErsVJrZf28BY92qL2etK5SU=; b=eMzUPb1fI36C5YmzYy57SApkJdb94Hwf3xh6EkvF0AHwLJNHKOD4iKGH TtKbUcnIoXiWT5WtKss07Ak3mF1q/VpMM7MXjFqPOlv4Cc2NmtK+PWjxx rqGv9QlOoujx2g4Dh4QSVdaYj4g5qVlGUky2AG1pKfiHjErjWv6VrHNvj K+DDpjULCTbFREmAtVWwiyjZSIRqIR2Wl9PItTH8uqlRSm9hnKb0AuLpn 06SzYs3xBFZpoIwUn1gh2Tiwhv8qec8AsJu0yA6sseWHNadCtCpfNW25Y u2Lb7WfhVskmsrENX14txbsstSD7kINRte3AAHo6O1R+/F+oh5w8U1tL6 A==; X-CSE-ConnectionGUID: hCN7cREMT52+J84dvnPFhg== X-CSE-MsgGUID: GjdInlKcQBONej6Raorb6A== X-IronPort-AV: E=McAfee;i="6800,10657,11493"; a="58540876" X-IronPort-AV: E=Sophos;i="6.16,315,1744095600"; d="scan'208";a="58540876" Received: from fmviesa003.fm.intel.com ([10.60.135.143]) by orvoesa107.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Jul 2025 00:12:54 -0700 X-CSE-ConnectionGUID: gTGNLvXwQRSjzUzWtGaw9w== X-CSE-MsgGUID: e1+XLYb1Sy60Ejg2tPmNQg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.16,315,1744095600"; d="scan'208";a="161453670" Received: from yilunxu-optiplex-7050.sh.intel.com ([10.239.159.165]) by fmviesa003.fm.intel.com with ESMTP; 16 Jul 2025 00:12:51 -0700 From: Xu Yilun To: jgg@nvidia.com, jgg@ziepe.ca, kevin.tian@intel.com, will@kernel.org, aneesh.kumar@kernel.org Cc: iommu@lists.linux.dev, linux-kernel@vger.kernel.org, joro@8bytes.org, robin.murphy@arm.com, shuah@kernel.org, nicolinc@nvidia.com, aik@amd.com, dan.j.williams@intel.com, baolu.lu@linux.intel.com, yilun.xu@intel.com Subject: [PATCH v6 7/8] iommufd/selftest: Add coverage for vdevice tombstone Date: Wed, 16 Jul 2025 15:03:48 +0800 Message-Id: <20250716070349.1807226-8-yilun.xu@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20250716070349.1807226-1-yilun.xu@linux.intel.com> References: <20250716070349.1807226-1-yilun.xu@linux.intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" This tests the flow to tombstone vdevice when idevice is to be unbound before vdevice destruction. The expected results of the tombstone are: - The vdevice ID can't be reused anymore (not tested in this patch). - Even ioctl(IOMMU_DESTROY) can't free the vdevice ID. - iommufd_fops_release() can still free everything. Reviewed-by: Nicolin Chen Tested-by: Nicolin Chen Signed-off-by: Xu Yilun --- tools/testing/selftests/iommu/iommufd.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/tools/testing/selftests/iommu/iommufd.c b/tools/testing/selfte= sts/iommu/iommufd.c index 1b629bedeb1c..26fe912c62ae 100644 --- a/tools/testing/selftests/iommu/iommufd.c +++ b/tools/testing/selftests/iommu/iommufd.c @@ -3115,6 +3115,20 @@ TEST_F(iommufd_viommu, hw_queue) test_ioctl_ioas_unmap(iova, PAGE_SIZE); } =20 +TEST_F(iommufd_viommu, vdevice_tombstone) +{ + uint32_t viommu_id =3D self->viommu_id; + uint32_t dev_id =3D self->device_id; + uint32_t vdev_id =3D 0; + + if (!dev_id) + SKIP(return, "Skipping test for variant no_viommu"); + + test_cmd_vdevice_alloc(viommu_id, dev_id, 0x99, &vdev_id); + test_ioctl_destroy(self->stdev_id); + EXPECT_ERRNO(ENOENT, _test_ioctl_destroy(self->fd, vdev_id)); +} + FIXTURE(iommufd_device_pasid) { int fd; --=20 2.25.1 From nobody Tue Oct 7 00:22:44 2025 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 302DC290092 for ; Wed, 16 Jul 2025 07:12:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.15 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752649979; cv=none; b=jR/nmXkg0cUXOs/+oYABvAZEytKfv4etE6xVthUzBTqVl3+LQgNvnODMjOO4SYOkqFdQo0SIeAGDSa/570IOsB3ca2Mt0VvD6dv9//44e1QcJBKokNaEMc1hsbfzjye2S3NXKpaYwqf1PjAsq0lvown7A/WQTq2TrVjjfnKLykw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752649979; c=relaxed/simple; bh=YaPJqZ8tPXcl7vKy59vvbjNeLPK3pnoNZGDI4WEGdKc=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=lqsDx4xCt/sUCFHDwrx1EKhWUPimnRGE4N7644Wn6NrUVaep0AhS7GNsNqBZcGVjtTkncAtLoOWnCqLzImblRjStSJ7SI+jCKpDV6LemadZUPNNovvSZpSKUHlWUMjURQ9At0whEmMRUHFVgzLbmuyrMCOCuVCqV8q3bUMW9g/U= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=D1WmrGER; arc=none smtp.client-ip=198.175.65.15 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="D1WmrGER" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1752649978; x=1784185978; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=YaPJqZ8tPXcl7vKy59vvbjNeLPK3pnoNZGDI4WEGdKc=; b=D1WmrGERphOWUqjM0Ge4ekdfsOLsgxssNhZcLSfYh0gTvchtaOSVl/lJ 34UVkyEdknYiBchoKoaQBSMqZMvjFoFFuqRZq+Omisv2Hp7qP0gLx1GgZ KW4G+pOQONHnAhHbN0ZzVMv/C61rbpikBSGRWkCrR85vyc97nM03uMobl Sw8nlH22oIu+pYBxi8vt6SaGHK8PM+bclFv9emy0Wm/XrrGmpDes6FdgQ GCYwieziDzWfBNiJaTW6AFMvfVIfj2lLP2k4nwCBdttd9O1BYZECioVX7 pPSUd1Jv2hvMjNdC1Xd2rKjHrjUU2yjAtdxOkJb0a4dQKBNeC9pGaJ2al A==; X-CSE-ConnectionGUID: LLT03biZTsCzv/cYjdNbvg== X-CSE-MsgGUID: rfWAf10OTpCZkHgVvncTZQ== X-IronPort-AV: E=McAfee;i="6800,10657,11493"; a="58540888" X-IronPort-AV: E=Sophos;i="6.16,315,1744095600"; d="scan'208";a="58540888" Received: from fmviesa003.fm.intel.com ([10.60.135.143]) by orvoesa107.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Jul 2025 00:12:58 -0700 X-CSE-ConnectionGUID: K0PL0QQ0Tli94zpY9+b4LQ== X-CSE-MsgGUID: fM7nAJ+jRNu/RHQPhTnrVQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.16,315,1744095600"; d="scan'208";a="161453675" Received: from yilunxu-optiplex-7050.sh.intel.com ([10.239.159.165]) by fmviesa003.fm.intel.com with ESMTP; 16 Jul 2025 00:12:54 -0700 From: Xu Yilun To: jgg@nvidia.com, jgg@ziepe.ca, kevin.tian@intel.com, will@kernel.org, aneesh.kumar@kernel.org Cc: iommu@lists.linux.dev, linux-kernel@vger.kernel.org, joro@8bytes.org, robin.murphy@arm.com, shuah@kernel.org, nicolinc@nvidia.com, aik@amd.com, dan.j.williams@intel.com, baolu.lu@linux.intel.com, yilun.xu@intel.com Subject: [PATCH v6 8/8] iommufd: Rename some shortterm-related identifiers Date: Wed, 16 Jul 2025 15:03:49 +0800 Message-Id: <20250716070349.1807226-9-yilun.xu@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20250716070349.1807226-1-yilun.xu@linux.intel.com> References: <20250716070349.1807226-1-yilun.xu@linux.intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Rename the shortterm-related identifiers to wait-related. The usage of shortterm_users refcount is now beyond its name. It is also used for references which live longer than an ioctl execution. E.g. vdev holds idev's shortterm_users refcount on vdev allocation, releases it during idev's pre_destroy(). Rename the refcount as wait_cnt, since it is always used to sync the referencing & the destruction of the object by waiting for it to go to zero. List all changed identifiers: iommufd_object::shortterm_users -> iommufd_object::wait_cnt REMOVE_WAIT_SHORTTERM -> REMOVE_WAIT iommufd_object_dec_wait_shortterm() -> iommufd_object_dec_wait() zerod_shortterm -> zerod_wait_cnt No functional change intended. Suggested-by: Kevin Tian Suggested-by: Jason Gunthorpe Reviewed-by: Jason Gunthorpe Reviewed-by: Nicolin Chen Tested-by: Nicolin Chen Signed-off-by: Xu Yilun --- drivers/iommu/iommufd/device.c | 6 ++-- drivers/iommu/iommufd/iommufd_private.h | 18 ++++++------ drivers/iommu/iommufd/main.c | 39 +++++++++++++------------ drivers/iommu/iommufd/viommu.c | 4 +-- include/linux/iommufd.h | 8 ++++- 5 files changed, 41 insertions(+), 34 deletions(-) diff --git a/drivers/iommu/iommufd/device.c b/drivers/iommu/iommufd/device.c index ee6ff4caf398..65fbd098f9e9 100644 --- a/drivers/iommu/iommufd/device.c +++ b/drivers/iommu/iommufd/device.c @@ -152,8 +152,8 @@ static void iommufd_device_remove_vdev(struct iommufd_d= evice *idev) /* * An ongoing vdev destroy ioctl has removed the vdev from the object * xarray, but has not finished iommufd_vdevice_destroy() yet as it - * needs the same mutex. We exit the locking then wait on short term - * users for the vdev destruction. + * needs the same mutex. We exit the locking then wait on wait_cnt + * reference for the vdev destruction. */ if (IS_ERR(vdev)) goto out_unlock; @@ -184,7 +184,7 @@ void iommufd_device_pre_destroy(struct iommufd_object *= obj) struct iommufd_device *idev =3D container_of(obj, struct iommufd_device, obj); =20 - /* Release the short term users on this */ + /* Release the wait_cnt reference on this */ iommufd_device_remove_vdev(idev); } =20 diff --git a/drivers/iommu/iommufd/iommufd_private.h b/drivers/iommu/iommuf= d/iommufd_private.h index 5d6ea5395cfe..0da2a81eedfa 100644 --- a/drivers/iommu/iommufd/iommufd_private.h +++ b/drivers/iommu/iommufd/iommufd_private.h @@ -169,7 +169,7 @@ static inline bool iommufd_lock_obj(struct iommufd_obje= ct *obj) { if (!refcount_inc_not_zero(&obj->users)) return false; - if (!refcount_inc_not_zero(&obj->shortterm_users)) { + if (!refcount_inc_not_zero(&obj->wait_cnt)) { /* * If the caller doesn't already have a ref on obj this must be * called under the xa_lock. Otherwise the caller is holding a @@ -187,11 +187,11 @@ static inline void iommufd_put_object(struct iommufd_= ctx *ictx, struct iommufd_object *obj) { /* - * Users first, then shortterm so that REMOVE_WAIT_SHORTTERM never sees - * a spurious !0 users with a 0 shortterm_users. + * Users first, then wait_cnt so that REMOVE_WAIT never sees a spurious + * !0 users with a 0 wait_cnt. */ refcount_dec(&obj->users); - if (refcount_dec_and_test(&obj->shortterm_users)) + if (refcount_dec_and_test(&obj->wait_cnt)) wake_up_interruptible_all(&ictx->destroy_wait); } =20 @@ -202,7 +202,7 @@ void iommufd_object_finalize(struct iommufd_ctx *ictx, struct iommufd_object *obj); =20 enum { - REMOVE_WAIT_SHORTTERM =3D BIT(0), + REMOVE_WAIT =3D BIT(0), REMOVE_OBJ_TOMBSTONE =3D BIT(1), }; int iommufd_object_remove(struct iommufd_ctx *ictx, @@ -211,15 +211,15 @@ int iommufd_object_remove(struct iommufd_ctx *ictx, =20 /* * The caller holds a users refcount and wants to destroy the object. At t= his - * point the caller has no shortterm_users reference and at least the xarr= ay - * will be holding one. + * point the caller has no wait_cnt reference and at least the xarray will= be + * holding one. */ static inline void iommufd_object_destroy_user(struct iommufd_ctx *ictx, struct iommufd_object *obj) { int ret; =20 - ret =3D iommufd_object_remove(ictx, obj, obj->id, REMOVE_WAIT_SHORTTERM); + ret =3D iommufd_object_remove(ictx, obj, obj->id, REMOVE_WAIT); =20 /* * If there is a bug and we couldn't destroy the object then we did put @@ -239,7 +239,7 @@ static inline void iommufd_object_tombstone_user(struct= iommufd_ctx *ictx, int ret; =20 ret =3D iommufd_object_remove(ictx, obj, obj->id, - REMOVE_WAIT_SHORTTERM | REMOVE_OBJ_TOMBSTONE); + REMOVE_WAIT | REMOVE_OBJ_TOMBSTONE); =20 /* * If there is a bug and we couldn't destroy the object then we did put diff --git a/drivers/iommu/iommufd/main.c b/drivers/iommu/iommufd/main.c index 99c1aab3d396..15af7ced0501 100644 --- a/drivers/iommu/iommufd/main.c +++ b/drivers/iommu/iommufd/main.c @@ -42,7 +42,7 @@ struct iommufd_object *_iommufd_object_alloc(struct iommu= fd_ctx *ictx, return ERR_PTR(-ENOMEM); obj->type =3D type; /* Starts out bias'd by 1 until it is removed from the xarray */ - refcount_set(&obj->shortterm_users, 1); + refcount_set(&obj->wait_cnt, 1); refcount_set(&obj->users, 1); =20 /* @@ -155,22 +155,22 @@ struct iommufd_object *iommufd_get_object(struct iomm= ufd_ctx *ictx, u32 id, return obj; } =20 -static int iommufd_object_dec_wait_shortterm(struct iommufd_ctx *ictx, - struct iommufd_object *to_destroy) +static int iommufd_object_dec_wait(struct iommufd_ctx *ictx, + struct iommufd_object *to_destroy) { - if (refcount_dec_and_test(&to_destroy->shortterm_users)) + if (refcount_dec_and_test(&to_destroy->wait_cnt)) return 0; =20 if (iommufd_object_ops[to_destroy->type].pre_destroy) iommufd_object_ops[to_destroy->type].pre_destroy(to_destroy); =20 if (wait_event_timeout(ictx->destroy_wait, - refcount_read(&to_destroy->shortterm_users) =3D=3D 0, + refcount_read(&to_destroy->wait_cnt) =3D=3D 0, msecs_to_jiffies(60000))) return 0; =20 pr_crit("Time out waiting for iommufd object to become free\n"); - refcount_inc(&to_destroy->shortterm_users); + refcount_inc(&to_destroy->wait_cnt); return -EBUSY; } =20 @@ -184,17 +184,18 @@ int iommufd_object_remove(struct iommufd_ctx *ictx, { struct iommufd_object *obj; XA_STATE(xas, &ictx->objects, id); - bool zerod_shortterm =3D false; + bool zerod_wait_cnt =3D false; int ret; =20 /* - * The purpose of the shortterm_users is to ensure deterministic - * destruction of objects used by external drivers and destroyed by this - * function. Any temporary increment of the refcount must increment - * shortterm_users, such as during ioctl execution. + * The purpose of the wait_cnt is to ensure deterministic destruction + * of objects used by external drivers and destroyed by this function. + * Incrementing this wait_cnt should either be short lived, such as + * during ioctl execution, or be revoked and blocked during + * pre_destroy(), such as vdev holding the idev's refcount. */ - if (flags & REMOVE_WAIT_SHORTTERM) { - ret =3D iommufd_object_dec_wait_shortterm(ictx, to_destroy); + if (flags & REMOVE_WAIT) { + ret =3D iommufd_object_dec_wait(ictx, to_destroy); if (ret) { /* * We have a bug. Put back the callers reference and @@ -203,7 +204,7 @@ int iommufd_object_remove(struct iommufd_ctx *ictx, refcount_dec(&to_destroy->users); return ret; } - zerod_shortterm =3D true; + zerod_wait_cnt =3D true; } =20 xa_lock(&ictx->objects); @@ -235,11 +236,11 @@ int iommufd_object_remove(struct iommufd_ctx *ictx, xa_unlock(&ictx->objects); =20 /* - * Since users is zero any positive users_shortterm must be racing + * Since users is zero any positive wait_cnt must be racing * iommufd_put_object(), or we have a bug. */ - if (!zerod_shortterm) { - ret =3D iommufd_object_dec_wait_shortterm(ictx, obj); + if (!zerod_wait_cnt) { + ret =3D iommufd_object_dec_wait(ictx, obj); if (WARN_ON(ret)) return ret; } @@ -249,9 +250,9 @@ int iommufd_object_remove(struct iommufd_ctx *ictx, return 0; =20 err_xa: - if (zerod_shortterm) { + if (zerod_wait_cnt) { /* Restore the xarray owned reference */ - refcount_set(&obj->shortterm_users, 1); + refcount_set(&obj->wait_cnt, 1); } xa_unlock(&ictx->objects); =20 diff --git a/drivers/iommu/iommufd/viommu.c b/drivers/iommu/iommufd/viommu.c index 6cf0bd5d8f08..2ca5809b238b 100644 --- a/drivers/iommu/iommufd/viommu.c +++ b/drivers/iommu/iommufd/viommu.c @@ -205,8 +205,8 @@ int iommufd_vdevice_alloc_ioctl(struct iommufd_ucmd *uc= md) vdev->viommu =3D viommu; refcount_inc(&viommu->obj.users); /* - * A short term users reference is held on the idev so long as we have - * the pointer. iommufd_device_pre_destroy() will revoke it before the + * A wait_cnt reference is held on the idev so long as we have the + * pointer. iommufd_device_pre_destroy() will revoke it before the * idev real destruction. */ vdev->idev =3D idev; diff --git a/include/linux/iommufd.h b/include/linux/iommufd.h index 810e4d8ac912..6e7efe83bc5d 100644 --- a/include/linux/iommufd.h +++ b/include/linux/iommufd.h @@ -46,7 +46,13 @@ enum iommufd_object_type { =20 /* Base struct for all objects with a userspace ID handle. */ struct iommufd_object { - refcount_t shortterm_users; + /* + * Destroy will sleep and wait for wait_cnt to go to zero. This allows + * concurrent users of the ID to reliably avoid causing a spurious + * destroy failure. Incrementing this count should either be short + * lived or be revoked and blocked during pre_destroy(). + */ + refcount_t wait_cnt; refcount_t users; enum iommufd_object_type type; unsigned int id; --=20 2.25.1