From nobody Tue Oct 7 03:50:36 2025 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E802B26D4DA for ; Tue, 15 Jul 2025 06:41:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.19 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752561689; cv=none; b=NWpNcSHPmDRDhYpl5ZaXwkbCSvRxyaLNwqOSqRXB/+9OTR+BLC6tCvx8ZPJsO9/GEkmksZkBAlsw5fke+nOx2uaX+MxcZLQ94bMbYEfI4enlQBoh1YvtiWA+KmcQYQpSEZeaPnU4q3KqPSddBLes0lIrK9x2Na/tjRgQoxnYqro= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752561689; c=relaxed/simple; bh=dW8N0hXsHJ0xZQ7gyU6p87aQVgeVycVBLBzar7O5u6Y=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Dyby9+H8Mr5COs6BBzAWmTQxQBhOEpUtYRw3lCEQGfIswiCGhUcQc1T66U93P+tQ4nxT6NQUlTznEkIiJl36KTMa9olvpOg9qjQ7s4v+3fRe38NJjt52LBokUcb77sbNW0jbmQzkKhzcSfJgVEtPCUEm5TuzyOQ7pfdoNz+y5jk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=m+aFmf42; arc=none smtp.client-ip=192.198.163.19 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="m+aFmf42" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1752561688; x=1784097688; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=dW8N0hXsHJ0xZQ7gyU6p87aQVgeVycVBLBzar7O5u6Y=; b=m+aFmf42ZjaiL9qMh9vIz9wKzX8W2mvnVZcfvXA9wUms10JhDvmyS9aD Ppz0u5CD+dcNMjpdC8KTzczE5hJcDhZ+u6fFyEZmM46/TNAtEZZy6UnsU loCqh/POZnVhdLwUXhl2WuOJmFJXRteiHCPKXYQ8OB7B9b2nKsE67LgFf CiJ2rKWiXMJpnYtNP4QqtF4ZdqnIi+u0PQ8V81VV3eJfAq/LHRr/la61k 8etwdj+w0ndCS3BWWTBNJ93r6GJoDOxoQZ56iL6Dm8ByIas7WGiRtUMCp SQm+S+OMfyXBut9o2kUOqAdNtf8hfqZNddvo16+3QY1ljYg+uxAfuzplq A==; X-CSE-ConnectionGUID: HE7xRXxHQ729f7Waunjcdg== X-CSE-MsgGUID: xHuByERoSoqaWFaUVjM4ig== X-IronPort-AV: E=McAfee;i="6800,10657,11491"; a="53880825" X-IronPort-AV: E=Sophos;i="6.16,313,1744095600"; d="scan'208";a="53880825" Received: from orviesa008.jf.intel.com ([10.64.159.148]) by fmvoesa113.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jul 2025 23:41:28 -0700 X-CSE-ConnectionGUID: TAZ3xvnWRuyK0ttEvH4aeg== X-CSE-MsgGUID: Z+F9k3SbTg2SYuG/I878FQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.16,313,1744095600"; d="scan'208";a="157648940" Received: from yilunxu-optiplex-7050.sh.intel.com ([10.239.159.165]) by orviesa008.jf.intel.com with ESMTP; 14 Jul 2025 23:41:24 -0700 From: Xu Yilun To: jgg@nvidia.com, jgg@ziepe.ca, kevin.tian@intel.com, will@kernel.org, aneesh.kumar@kernel.org Cc: iommu@lists.linux.dev, linux-kernel@vger.kernel.org, joro@8bytes.org, robin.murphy@arm.com, shuah@kernel.org, nicolinc@nvidia.com, aik@amd.com, dan.j.williams@intel.com, baolu.lu@linux.intel.com, yilun.xu@intel.com Subject: [PATCH v5 1/8] iommufd/viommu: Roll back to use iommufd_object_alloc() for vdevice Date: Tue, 15 Jul 2025 14:32:38 +0800 Message-Id: <20250715063245.1799534-2-yilun.xu@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20250715063245.1799534-1-yilun.xu@linux.intel.com> References: <20250715063245.1799534-1-yilun.xu@linux.intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" To solve the vdevice lifecycle issue, future patches make the vdevice allocation protected by lock. That will make _iommufd_object_alloc_ucmd() not applicable for vdevice. Roll back to use _iommufd_object_alloc() for preparation. Reviewed-by: Nicolin Chen Reviewed-by: Kevin Tian Reviewed-by: Jason Gunthorpe Signed-off-by: Xu Yilun Tested-by: Nicolin Chen --- drivers/iommu/iommufd/viommu.c | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/drivers/iommu/iommufd/viommu.c b/drivers/iommu/iommufd/viommu.c index 91339f799916..dcf8a85b9f6e 100644 --- a/drivers/iommu/iommufd/viommu.c +++ b/drivers/iommu/iommufd/viommu.c @@ -167,8 +167,8 @@ int iommufd_vdevice_alloc_ioctl(struct iommufd_ucmd *uc= md) vdev_size =3D viommu->ops->vdevice_size; } =20 - vdev =3D (struct iommufd_vdevice *)_iommufd_object_alloc_ucmd( - ucmd, vdev_size, IOMMUFD_OBJ_VDEVICE); + vdev =3D (struct iommufd_vdevice *)_iommufd_object_alloc( + ucmd->ictx, vdev_size, IOMMUFD_OBJ_VDEVICE); if (IS_ERR(vdev)) { rc =3D PTR_ERR(vdev); goto out_put_idev; @@ -183,18 +183,24 @@ int iommufd_vdevice_alloc_ioctl(struct iommufd_ucmd *= ucmd) curr =3D xa_cmpxchg(&viommu->vdevs, virt_id, NULL, vdev, GFP_KERNEL); if (curr) { rc =3D xa_err(curr) ?: -EEXIST; - goto out_put_idev; + goto out_abort; } =20 if (viommu->ops && viommu->ops->vdevice_init) { rc =3D viommu->ops->vdevice_init(vdev); if (rc) - goto out_put_idev; + goto out_abort; } =20 cmd->out_vdevice_id =3D vdev->obj.id; rc =3D iommufd_ucmd_respond(ucmd, sizeof(*cmd)); + if (rc) + goto out_abort; + iommufd_object_finalize(ucmd->ictx, &vdev->obj); + goto out_put_idev; =20 +out_abort: + iommufd_object_abort_and_destroy(ucmd->ictx, &vdev->obj); out_put_idev: iommufd_put_object(ucmd->ictx, &idev->obj); out_put_viommu: --=20 2.25.1 From nobody Tue Oct 7 03:50:36 2025 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 46E5926F478 for ; Tue, 15 Jul 2025 06:41:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.19 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752561693; cv=none; b=KBdsf1Ktu7uJiRAvod0BJeOXwcswYpGUqoIk89BsaVvwxwfBBb3g5PIJtunh+IBdRAPH4azO8bg+ZqLLblzyU6OmTZfQqXeUMDgV9ZzYvptv6NNAId7JJDG/2JgqT4gjdL3kwy7zOOCWCQZMq5zdfB659ASXadI9ZOuXVxs+IuA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752561693; c=relaxed/simple; bh=JITLAe7Ss9LRUk8R17DJr/BRXXLBIam+ZkRxFIaW8YI=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=sRoYXlizq/nvq+1KqT3xskGVl3CeovigobEneNq152KVdm4yb/ixsujXqeitXp7YERCpoLbUeYzLWoaI/ZSQbbz0CPrRIeixF4ZOxeJQU/7TeOzbdLYP+MRxRHTPSjkTT3sq1jzLHht66gYVyVjWuxRSgEa0iQSiCVKRoRIMjm0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=R82GccxX; arc=none smtp.client-ip=192.198.163.19 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="R82GccxX" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1752561692; x=1784097692; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=JITLAe7Ss9LRUk8R17DJr/BRXXLBIam+ZkRxFIaW8YI=; b=R82GccxX0s75VB5lflLJJjpJZw1o66skg8UkDLNeVRbhKUTVu3xI2Lua xXtkQ6edUfBDU6vihD8yYM7SoNuHvZYBNLAaTe5HyIARRaZaeTAIE26Mr pz6ZEoYFpxc93i0T2vWuxoFDmDVoiLXtP2th/3WsXBYDQ8JmSkb2gZRMd HPvTMSyg5LsTJf6iAxIV+kjarcBm4jL1cR0a3XzXzR4WoXUA43v+eaGXj 0Hrx6uF3o12RedWMvdojBth3mJo5U3tQIykyRIgHOFZ1+nHCFfAcW9krt ODPTxmSsbzu+MstARAc7bSEX8AYMLYSZLdCwGlyBMywflzSiqb4xkAOKh Q==; X-CSE-ConnectionGUID: vGtpgzTDTYCz6dt9V19c2g== X-CSE-MsgGUID: Hb4uS8RqQRWNGcJZ5d73lg== X-IronPort-AV: E=McAfee;i="6800,10657,11491"; a="53880835" X-IronPort-AV: E=Sophos;i="6.16,313,1744095600"; d="scan'208";a="53880835" Received: from orviesa008.jf.intel.com ([10.64.159.148]) by fmvoesa113.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jul 2025 23:41:32 -0700 X-CSE-ConnectionGUID: ZxAGOtpZSBGNm/sp09Qn8Q== X-CSE-MsgGUID: 9fhg/NNRQ4KbAqJxNCyu2g== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.16,313,1744095600"; d="scan'208";a="157648967" Received: from yilunxu-optiplex-7050.sh.intel.com ([10.239.159.165]) by orviesa008.jf.intel.com with ESMTP; 14 Jul 2025 23:41:28 -0700 From: Xu Yilun To: jgg@nvidia.com, jgg@ziepe.ca, kevin.tian@intel.com, will@kernel.org, aneesh.kumar@kernel.org Cc: iommu@lists.linux.dev, linux-kernel@vger.kernel.org, joro@8bytes.org, robin.murphy@arm.com, shuah@kernel.org, nicolinc@nvidia.com, aik@amd.com, dan.j.williams@intel.com, baolu.lu@linux.intel.com, yilun.xu@intel.com Subject: [PATCH v5 2/8] iommufd: Add iommufd_object_tombstone_user() helper Date: Tue, 15 Jul 2025 14:32:39 +0800 Message-Id: <20250715063245.1799534-3-yilun.xu@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20250715063245.1799534-1-yilun.xu@linux.intel.com> References: <20250715063245.1799534-1-yilun.xu@linux.intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add the iommufd_object_tombstone_user() helper, which allows the caller to destroy an iommufd object created by userspace. This is useful on some destroy paths when the kernel caller finds the object should have been removed by userspace but is still alive. With this helper, the caller destroys the object but leave the object ID reserved (so called tombstone). The tombstone prevents repurposing the object ID without awareness of the original user. Since this happens for abnormal userspace behavior, for simplicity, the tombstoned object ID would be permanently leaked until iommufd_fops_release(). I.e. the original user gets an error when calling ioctl(IOMMU_DESTROY) on that ID. The first use case would be to ensure the iommufd_vdevice can't outlive the associated iommufd_device. Suggested-by: Jason Gunthorpe Reviewed-by: Lu Baolu Reviewed-by: Nicolin Chen Reviewed-by: Kevin Tian Reviewed-by: Jason Gunthorpe Co-developed-by: Aneesh Kumar K.V (Arm) Signed-off-by: Aneesh Kumar K.V (Arm) Signed-off-by: Xu Yilun Tested-by: Nicolin Chen --- drivers/iommu/iommufd/iommufd_private.h | 23 ++++++++++++++++++++++- drivers/iommu/iommufd/main.c | 24 +++++++++++++++++++++++- 2 files changed, 45 insertions(+), 2 deletions(-) diff --git a/drivers/iommu/iommufd/iommufd_private.h b/drivers/iommu/iommuf= d/iommufd_private.h index cd14163abdd1..149545060029 100644 --- a/drivers/iommu/iommufd/iommufd_private.h +++ b/drivers/iommu/iommufd/iommufd_private.h @@ -202,7 +202,8 @@ void iommufd_object_finalize(struct iommufd_ctx *ictx, struct iommufd_object *obj); =20 enum { - REMOVE_WAIT_SHORTTERM =3D 1, + REMOVE_WAIT_SHORTTERM =3D BIT(0), + REMOVE_OBJ_TOMBSTONE =3D BIT(1), }; int iommufd_object_remove(struct iommufd_ctx *ictx, struct iommufd_object *to_destroy, u32 id, @@ -228,6 +229,26 @@ static inline void iommufd_object_destroy_user(struct = iommufd_ctx *ictx, WARN_ON(ret); } =20 +/* + * Similar to iommufd_object_destroy_user(), except that the object ID is = left + * reserved/tombstoned. + */ +static inline void iommufd_object_tombstone_user(struct iommufd_ctx *ictx, + struct iommufd_object *obj) +{ + int ret; + + ret =3D iommufd_object_remove(ictx, obj, obj->id, + REMOVE_WAIT_SHORTTERM | REMOVE_OBJ_TOMBSTONE); + + /* + * If there is a bug and we couldn't destroy the object then we did put + * back the caller's users refcount and will eventually try to free it + * again during close. + */ + WARN_ON(ret); +} + /* * The HWPT allocated by autodomains is used in possibly many devices and * is automatically destroyed when its refcount reaches zero. diff --git a/drivers/iommu/iommufd/main.c b/drivers/iommu/iommufd/main.c index 69c2195e77ca..71135f0ec72d 100644 --- a/drivers/iommu/iommufd/main.c +++ b/drivers/iommu/iommufd/main.c @@ -225,7 +225,7 @@ int iommufd_object_remove(struct iommufd_ctx *ictx, goto err_xa; } =20 - xas_store(&xas, NULL); + xas_store(&xas, (flags & REMOVE_OBJ_TOMBSTONE) ? XA_ZERO_ENTRY : NULL); if (ictx->vfio_ioas =3D=3D container_of(obj, struct iommufd_ioas, obj)) ictx->vfio_ioas =3D NULL; xa_unlock(&ictx->objects); @@ -311,19 +311,41 @@ static int iommufd_fops_release(struct inode *inode, = struct file *filp) while (!xa_empty(&ictx->objects)) { unsigned int destroyed =3D 0; unsigned long index; + bool empty =3D true; =20 + /* + * We can't use xa_empty() to end the loop as the tombstones + * are stored as XA_ZERO_ENTRY in the xarray. However + * xa_for_each() automatically converts them to NULL and skips + * them causing xa_empty() to be kept false. Thus once + * xa_for_each() finds no further !NULL entries the loop is + * done. + */ xa_for_each(&ictx->objects, index, obj) { + empty =3D false; if (!refcount_dec_if_one(&obj->users)) continue; + destroyed++; xa_erase(&ictx->objects, index); iommufd_object_ops[obj->type].destroy(obj); kfree(obj); } + + if (empty) + break; + /* Bug related to users refcount */ if (WARN_ON(!destroyed)) break; } + + /* + * There may be some tombstones left over from + * iommufd_object_tombstone_user() + */ + xa_destroy(&ictx->objects); + WARN_ON(!xa_empty(&ictx->groups)); =20 mutex_destroy(&ictx->sw_msi_lock); --=20 2.25.1 From nobody Tue Oct 7 03:50:36 2025 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 10C9626D4DA for ; Tue, 15 Jul 2025 06:41:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.19 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752561698; cv=none; b=jYA0Po8L1xW08cCSyn+4CWbbKqEO0u/4ON08rEJW55Cvsf5Y0UNG924I5Ata/0E+aLfiwZuE5Mv4KA0a1Mv1cdrid8GWINV4MHjBdmYvp94hdc2QiJsKINYXpc1kVOphgR4lD/6+eUIke/FYhEm8AJNlRqb0A0w1FTPjsU2hF+E= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752561698; c=relaxed/simple; bh=ldERnd6WuJBj0vJBisCoWvYmOAetmnwBakfZRokt2A0=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=hz+e+rc895DqDsRPp1RwKr2Fxl7A8j/p3DnhQrAFPWuYVtz66XskIZQhDdsLTieBeRqCIc2z/h6X2vjM2SuXPiBnJq5vY0Ylo36YQkvMnZwfblme25EmY3CIrA9Mu4ie4FzC61Wm/ixiLf7ufDQRFMbciwuOz5GrzTj+4DaRea4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=P58ZLj/Z; arc=none smtp.client-ip=192.198.163.19 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="P58ZLj/Z" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1752561696; x=1784097696; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=ldERnd6WuJBj0vJBisCoWvYmOAetmnwBakfZRokt2A0=; b=P58ZLj/Z+lhnCE29ny5SCiFao+EJ+/U/Ijq1kymol56CZPiZPHNPqUlY tXFpUxLuHi6gqyT34kv5bb4CTC0eR0KN9Oia9dticQp5yRChnYNncmFQo Jiq2Djz9WOkFs7a71qaVTi9GtO6rrNa0JwC2zxmn+NW+3A+1lVqKlpQOX GgREgqPS2Z05A2LDsjIGt31JInS/uMfNDMpSONGnU63aGxTn8uK8JsUWe ac597/qTWrNWb1faKA3wzqqpEOiuqODamGOTHqYMpKuSPTd4VwwpfJfCY V0pXItL2F786/FEqtr/pvqKeKqjkHfBINL2oXQsjtiY15Yz85EBJ4wAQZ g==; X-CSE-ConnectionGUID: sn8VpsxfSLaU6CX1A/+3zw== X-CSE-MsgGUID: dQxN9k37RM+I48P1cRzzXA== X-IronPort-AV: E=McAfee;i="6800,10657,11491"; a="53880860" X-IronPort-AV: E=Sophos;i="6.16,313,1744095600"; d="scan'208";a="53880860" Received: from orviesa008.jf.intel.com ([10.64.159.148]) by fmvoesa113.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jul 2025 23:41:36 -0700 X-CSE-ConnectionGUID: 0yLybFLMS+2FOL7azW89Pw== X-CSE-MsgGUID: wFNM6iKGSAeqfeftkA4bPA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.16,313,1744095600"; d="scan'208";a="157648991" Received: from yilunxu-optiplex-7050.sh.intel.com ([10.239.159.165]) by orviesa008.jf.intel.com with ESMTP; 14 Jul 2025 23:41:32 -0700 From: Xu Yilun To: jgg@nvidia.com, jgg@ziepe.ca, kevin.tian@intel.com, will@kernel.org, aneesh.kumar@kernel.org Cc: iommu@lists.linux.dev, linux-kernel@vger.kernel.org, joro@8bytes.org, robin.murphy@arm.com, shuah@kernel.org, nicolinc@nvidia.com, aik@amd.com, dan.j.williams@intel.com, baolu.lu@linux.intel.com, yilun.xu@intel.com Subject: [PATCH v5 3/8] iommufd: Add a pre_destroy() op for objects Date: Tue, 15 Jul 2025 14:32:40 +0800 Message-Id: <20250715063245.1799534-4-yilun.xu@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20250715063245.1799534-1-yilun.xu@linux.intel.com> References: <20250715063245.1799534-1-yilun.xu@linux.intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add a pre_destroy() op which gives objects a chance to clear their short term users references before destruction. This op is intended for external driver created objects (e.g. idev) which does deterministic destruction. In order to manage the lifecycle of interrelated objects as well as the deterministic destruction (e.g. vdev can't outlive idev, and idev destruction can't fail), short term users references are allowed to live out of an ioctl execution. An immediate use case is, vdev holds idev's short term user reference until vdev destruction completes, idev leverages existing wait_shortterm mechanism to ensure it is destroyed after vdev. This extended usage makes the referenced object unable to just wait for its reference gone. It needs to actively trigger the reference removal, as well as prevent new references before wait. Should implement these work in pre_destroy(). Suggested-by: Jason Gunthorpe Reviewed-by: Kevin Tian Reviewed-by: Nicolin Chen Signed-off-by: Xu Yilun Reviewed-by: Jason Gunthorpe Tested-by: Nicolin Chen --- drivers/iommu/iommufd/main.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/drivers/iommu/iommufd/main.c b/drivers/iommu/iommufd/main.c index 71135f0ec72d..53085d24ce4a 100644 --- a/drivers/iommu/iommufd/main.c +++ b/drivers/iommu/iommufd/main.c @@ -23,6 +23,7 @@ #include "iommufd_test.h" =20 struct iommufd_object_ops { + void (*pre_destroy)(struct iommufd_object *obj); void (*destroy)(struct iommufd_object *obj); void (*abort)(struct iommufd_object *obj); }; @@ -160,6 +161,9 @@ static int iommufd_object_dec_wait_shortterm(struct iom= mufd_ctx *ictx, if (refcount_dec_and_test(&to_destroy->shortterm_users)) return 0; =20 + if (iommufd_object_ops[to_destroy->type].pre_destroy) + iommufd_object_ops[to_destroy->type].pre_destroy(to_destroy); + if (wait_event_timeout(ictx->destroy_wait, refcount_read(&to_destroy->shortterm_users) =3D=3D 0, msecs_to_jiffies(60000))) --=20 2.25.1 From nobody Tue Oct 7 03:50:36 2025 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 59FE12701C4 for ; Tue, 15 Jul 2025 06:41:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.19 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752561702; cv=none; b=bTlg58LQxxkToqojRKJUrS0ySrlUrCfCahsRyP3MUuW/AsFmJw1U0VOdtHej2N4Cl6+0B71NQpD3P2xlIt4iLS/f+AknIFMmQyxNOlY9gGbmngBGw35RWorRvgi56TXfaJXjO4bPJj8noxOV7b5SMpbSzDx9Jp5c03cfoZhulh8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752561702; c=relaxed/simple; bh=shnm6bgro/rhfz5nWp9YsK4zVE2tz+/luZAIycNTsrw=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=DWRkZ37bjHtUihcReeMI14UPi8dujHCHp/vm8rN9dGBXQjH9J1c/cTI9rwoMUazAvyIrNK3otJFBkOGJm9cilRlNkh6qIqGUghTEWGFw3tZnUNggqHc6afrSBYDGEEaNzYE59n/TxKmXQ5AhPa9RRP069S+q44CHSkmyjPuD8SM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=QwSBjGNt; arc=none smtp.client-ip=192.198.163.19 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="QwSBjGNt" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1752561700; x=1784097700; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=shnm6bgro/rhfz5nWp9YsK4zVE2tz+/luZAIycNTsrw=; b=QwSBjGNtnmYz8NwtdeFpegQFvlbThQZMV/LCULqXgzGQ5fNsKI3OzZuN bm/VqubkgfhYF82ayPj1ORYnxzTg2HmFjNpnVO4bUiEw2ulugf0T0HE/S ZRfvDrqwN2sxCHUmg7lYf/Rftl8swihA7L7vZJ9Uc6qYvER5AXRuGSz4o 8Gonwaoj/86WfeOmlJ0WZshtbBtoVoHGHBrfATKfoXjlFmOxDiQb4sg8n CtTgl5LFZkfhcp0tMD7t7tyRD/gX1dy6JqWScOzNZmA4u2YYxjNbXYhpL Qo6of6xRAtFCNjRP+50ePx67WHqXg1lTc3vUP/OTI47NTm6LwbCnP6G9O w==; X-CSE-ConnectionGUID: 2H2L4F6wQde/MVL/kzqJlQ== X-CSE-MsgGUID: qWY+xTf4Qn+OeV5RpWpmog== X-IronPort-AV: E=McAfee;i="6800,10657,11491"; a="53880870" X-IronPort-AV: E=Sophos;i="6.16,313,1744095600"; d="scan'208";a="53880870" Received: from orviesa008.jf.intel.com ([10.64.159.148]) by fmvoesa113.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jul 2025 23:41:40 -0700 X-CSE-ConnectionGUID: 4IqddqbjTAWWE8IAuLTJ2g== X-CSE-MsgGUID: J9yXILHiQ4e/Z76d5mcSLw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.16,313,1744095600"; d="scan'208";a="157649010" Received: from yilunxu-optiplex-7050.sh.intel.com ([10.239.159.165]) by orviesa008.jf.intel.com with ESMTP; 14 Jul 2025 23:41:36 -0700 From: Xu Yilun To: jgg@nvidia.com, jgg@ziepe.ca, kevin.tian@intel.com, will@kernel.org, aneesh.kumar@kernel.org Cc: iommu@lists.linux.dev, linux-kernel@vger.kernel.org, joro@8bytes.org, robin.murphy@arm.com, shuah@kernel.org, nicolinc@nvidia.com, aik@amd.com, dan.j.williams@intel.com, baolu.lu@linux.intel.com, yilun.xu@intel.com Subject: [PATCH v5 4/8] iommufd: Destroy vdevice on idevice destroy Date: Tue, 15 Jul 2025 14:32:41 +0800 Message-Id: <20250715063245.1799534-5-yilun.xu@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20250715063245.1799534-1-yilun.xu@linux.intel.com> References: <20250715063245.1799534-1-yilun.xu@linux.intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Destroy iommufd_vdevice (vdev) on iommufd_idevice (idev) destruction so that vdev can't outlive idev. idev represents the physical device bound to iommufd, while the vdev represents the virtual instance of the physical device in the VM. The lifecycle of the vdev should not be longer than idev. This doesn't cause real problem on existing use cases cause vdev doesn't impact the physical device, only provides virtualization information. But to extend vdev for Confidential Computing (CC), there are needs to do secure configuration for the vdev, e.g. TSM Bind/Unbind. These configurations should be rolled back on idev destroy, or the external driver (VFIO) functionality may be impact. The idev is created by external driver so its destruction can't fail. The idev implements pre_destroy() op to actively remove its associated vdev before destroying itself. There are 3 cases on idev pre_destroy(): 1. vdev is already destroyed by userspace. No extra handling needed. 2. vdev is still alive. Use iommufd_object_tombstone_user() to destroy vdev and tombstone the vdev ID. 3. vdev is being destroyed by userspace. The vdev ID is already freed, but vdev destroy handler is not completed. This requires multi-threads syncing - vdev holds idev's short term users reference until vdev destruction completes, idev leverages existing wait_shortterm mechanism for syncing. idev should also block any new reference to it after pre_destroy(), or the following wait shortterm would timeout. Introduce a 'destroying' flag, set it to true on idev pre_destroy(). Any attempt to reference idev should honor this flag under the protection of idev->igroup->lock. Originally-by: Nicolin Chen Suggested-by: Jason Gunthorpe Reviewed-by: Kevin Tian Reviewed-by: Nicolin Chen Co-developed-by: Aneesh Kumar K.V (Arm) Signed-off-by: Aneesh Kumar K.V (Arm) Signed-off-by: Xu Yilun Reviewed-by: Jason Gunthorpe Tested-by: Nicolin Chen --- drivers/iommu/iommufd/device.c | 51 ++++++++++++++++++++++++ drivers/iommu/iommufd/iommufd_private.h | 12 ++++++ drivers/iommu/iommufd/main.c | 2 + drivers/iommu/iommufd/viommu.c | 52 +++++++++++++++++++++++-- include/linux/iommufd.h | 1 + include/uapi/linux/iommufd.h | 5 +++ 6 files changed, 119 insertions(+), 4 deletions(-) diff --git a/drivers/iommu/iommufd/device.c b/drivers/iommu/iommufd/device.c index e2ba21c43ad2..ee6ff4caf398 100644 --- a/drivers/iommu/iommufd/device.c +++ b/drivers/iommu/iommufd/device.c @@ -137,6 +137,57 @@ static struct iommufd_group *iommufd_get_group(struct = iommufd_ctx *ictx, } } =20 +static void iommufd_device_remove_vdev(struct iommufd_device *idev) +{ + struct iommufd_vdevice *vdev; + + mutex_lock(&idev->igroup->lock); + /* prevent new references from vdev */ + idev->destroying =3D true; + /* vdev has been completely destroyed by userspace */ + if (!idev->vdev) + goto out_unlock; + + vdev =3D iommufd_get_vdevice(idev->ictx, idev->vdev->obj.id); + /* + * An ongoing vdev destroy ioctl has removed the vdev from the object + * xarray, but has not finished iommufd_vdevice_destroy() yet as it + * needs the same mutex. We exit the locking then wait on short term + * users for the vdev destruction. + */ + if (IS_ERR(vdev)) + goto out_unlock; + + /* Should never happen */ + if (WARN_ON(vdev !=3D idev->vdev)) { + iommufd_put_object(idev->ictx, &vdev->obj); + goto out_unlock; + } + + /* + * vdev is still alive. Hold a users refcount to prevent racing with + * userspace destruction, then use iommufd_object_tombstone_user() to + * destroy it and leave a tombstone. + */ + refcount_inc(&vdev->obj.users); + iommufd_put_object(idev->ictx, &vdev->obj); + mutex_unlock(&idev->igroup->lock); + iommufd_object_tombstone_user(idev->ictx, &vdev->obj); + return; + +out_unlock: + mutex_unlock(&idev->igroup->lock); +} + +void iommufd_device_pre_destroy(struct iommufd_object *obj) +{ + struct iommufd_device *idev =3D + container_of(obj, struct iommufd_device, obj); + + /* Release the short term users on this */ + iommufd_device_remove_vdev(idev); +} + void iommufd_device_destroy(struct iommufd_object *obj) { struct iommufd_device *idev =3D diff --git a/drivers/iommu/iommufd/iommufd_private.h b/drivers/iommu/iommuf= d/iommufd_private.h index 149545060029..5d6ea5395cfe 100644 --- a/drivers/iommu/iommufd/iommufd_private.h +++ b/drivers/iommu/iommufd/iommufd_private.h @@ -489,6 +489,8 @@ struct iommufd_device { /* always the physical device */ struct device *dev; bool enforce_cache_coherency; + struct iommufd_vdevice *vdev; + bool destroying; }; =20 static inline struct iommufd_device * @@ -499,6 +501,7 @@ iommufd_get_device(struct iommufd_ucmd *ucmd, u32 id) struct iommufd_device, obj); } =20 +void iommufd_device_pre_destroy(struct iommufd_object *obj); void iommufd_device_destroy(struct iommufd_object *obj); int iommufd_get_hw_info(struct iommufd_ucmd *ucmd); =20 @@ -687,9 +690,18 @@ int iommufd_viommu_alloc_ioctl(struct iommufd_ucmd *uc= md); void iommufd_viommu_destroy(struct iommufd_object *obj); int iommufd_vdevice_alloc_ioctl(struct iommufd_ucmd *ucmd); void iommufd_vdevice_destroy(struct iommufd_object *obj); +void iommufd_vdevice_abort(struct iommufd_object *obj); int iommufd_hw_queue_alloc_ioctl(struct iommufd_ucmd *ucmd); void iommufd_hw_queue_destroy(struct iommufd_object *obj); =20 +static inline struct iommufd_vdevice * +iommufd_get_vdevice(struct iommufd_ctx *ictx, u32 id) +{ + return container_of(iommufd_get_object(ictx, id, + IOMMUFD_OBJ_VDEVICE), + struct iommufd_vdevice, obj); +} + #ifdef CONFIG_IOMMUFD_TEST int iommufd_test(struct iommufd_ucmd *ucmd); void iommufd_selftest_destroy(struct iommufd_object *obj); diff --git a/drivers/iommu/iommufd/main.c b/drivers/iommu/iommufd/main.c index 53085d24ce4a..99c1aab3d396 100644 --- a/drivers/iommu/iommufd/main.c +++ b/drivers/iommu/iommufd/main.c @@ -655,6 +655,7 @@ static const struct iommufd_object_ops iommufd_object_o= ps[] =3D { .destroy =3D iommufd_access_destroy_object, }, [IOMMUFD_OBJ_DEVICE] =3D { + .pre_destroy =3D iommufd_device_pre_destroy, .destroy =3D iommufd_device_destroy, }, [IOMMUFD_OBJ_FAULT] =3D { @@ -676,6 +677,7 @@ static const struct iommufd_object_ops iommufd_object_o= ps[] =3D { }, [IOMMUFD_OBJ_VDEVICE] =3D { .destroy =3D iommufd_vdevice_destroy, + .abort =3D iommufd_vdevice_abort, }, [IOMMUFD_OBJ_VEVENTQ] =3D { .destroy =3D iommufd_veventq_destroy, diff --git a/drivers/iommu/iommufd/viommu.c b/drivers/iommu/iommufd/viommu.c index dcf8a85b9f6e..ecbae5091ffe 100644 --- a/drivers/iommu/iommufd/viommu.c +++ b/drivers/iommu/iommufd/viommu.c @@ -110,20 +110,37 @@ int iommufd_viommu_alloc_ioctl(struct iommufd_ucmd *u= cmd) return rc; } =20 -void iommufd_vdevice_destroy(struct iommufd_object *obj) +void iommufd_vdevice_abort(struct iommufd_object *obj) { struct iommufd_vdevice *vdev =3D container_of(obj, struct iommufd_vdevice, obj); struct iommufd_viommu *viommu =3D vdev->viommu; + struct iommufd_device *idev =3D vdev->idev; + + lockdep_assert_held(&idev->igroup->lock); =20 if (vdev->destroy) vdev->destroy(vdev); /* xa_cmpxchg is okay to fail if alloc failed xa_cmpxchg previously */ xa_cmpxchg(&viommu->vdevs, vdev->virt_id, vdev, NULL, GFP_KERNEL); refcount_dec(&viommu->obj.users); + idev->vdev =3D NULL; put_device(vdev->dev); } =20 +void iommufd_vdevice_destroy(struct iommufd_object *obj) +{ + struct iommufd_vdevice *vdev =3D + container_of(obj, struct iommufd_vdevice, obj); + struct iommufd_device *idev =3D vdev->idev; + struct iommufd_ctx *ictx =3D idev->ictx; + + mutex_lock(&idev->igroup->lock); + iommufd_vdevice_abort(obj); + mutex_unlock(&idev->igroup->lock); + iommufd_put_object(ictx, &idev->obj); +} + int iommufd_vdevice_alloc_ioctl(struct iommufd_ucmd *ucmd) { struct iommu_vdevice_alloc *cmd =3D ucmd->cmd; @@ -153,6 +170,17 @@ int iommufd_vdevice_alloc_ioctl(struct iommufd_ucmd *u= cmd) goto out_put_idev; } =20 + mutex_lock(&idev->igroup->lock); + if (idev->destroying) { + rc =3D -ENOENT; + goto out_unlock_igroup; + } + + if (idev->vdev) { + rc =3D -EEXIST; + goto out_unlock_igroup; + } + if (viommu->ops && viommu->ops->vdevice_size) { /* * It is a driver bug for: @@ -171,7 +199,7 @@ int iommufd_vdevice_alloc_ioctl(struct iommufd_ucmd *uc= md) ucmd->ictx, vdev_size, IOMMUFD_OBJ_VDEVICE); if (IS_ERR(vdev)) { rc =3D PTR_ERR(vdev); - goto out_put_idev; + goto out_unlock_igroup; } =20 vdev->virt_id =3D virt_id; @@ -179,6 +207,19 @@ int iommufd_vdevice_alloc_ioctl(struct iommufd_ucmd *u= cmd) get_device(idev->dev); vdev->viommu =3D viommu; refcount_inc(&viommu->obj.users); + /* + * A short term users reference is held on the idev so long as we have + * the pointer. iommufd_device_pre_destroy() will revoke it before the + * idev real destruction. + */ + vdev->idev =3D idev; + + /* + * iommufd_device_destroy() delays until idev->vdev is NULL before + * freeing the idev, which only happens once the vdev is finished + * destruction. + */ + idev->vdev =3D vdev; =20 curr =3D xa_cmpxchg(&viommu->vdevs, virt_id, NULL, vdev, GFP_KERNEL); if (curr) { @@ -197,12 +238,15 @@ int iommufd_vdevice_alloc_ioctl(struct iommufd_ucmd *= ucmd) if (rc) goto out_abort; iommufd_object_finalize(ucmd->ictx, &vdev->obj); - goto out_put_idev; + goto out_unlock_igroup; =20 out_abort: iommufd_object_abort_and_destroy(ucmd->ictx, &vdev->obj); +out_unlock_igroup: + mutex_unlock(&idev->igroup->lock); out_put_idev: - iommufd_put_object(ucmd->ictx, &idev->obj); + if (rc) + iommufd_put_object(ucmd->ictx, &idev->obj); out_put_viommu: iommufd_put_object(ucmd->ictx, &viommu->obj); return rc; diff --git a/include/linux/iommufd.h b/include/linux/iommufd.h index e3a0cd47384d..b88911026bc4 100644 --- a/include/linux/iommufd.h +++ b/include/linux/iommufd.h @@ -108,6 +108,7 @@ struct iommufd_viommu { struct iommufd_vdevice { struct iommufd_object obj; struct iommufd_viommu *viommu; + struct iommufd_device *idev; struct device *dev; =20 /* diff --git a/include/uapi/linux/iommufd.h b/include/uapi/linux/iommufd.h index 554aacf89ea7..c218c89e0e2e 100644 --- a/include/uapi/linux/iommufd.h +++ b/include/uapi/linux/iommufd.h @@ -1070,6 +1070,11 @@ struct iommu_viommu_alloc { * * Allocate a virtual device instance (for a physical device) against a vI= OMMU. * This instance holds the device's information (related to its vIOMMU) in= a VM. + * User should use IOMMU_DESTROY to destroy the virtual device before + * destroying the physical device (by closing vfio_cdev fd). Otherwise the + * virtual device would be forcibly destroyed on physical device destructi= on, + * its vdevice_id would be permanently leaked (unremovable & unreusable) u= ntil + * iommu fd closed. */ struct iommu_vdevice_alloc { __u32 size; --=20 2.25.1 From nobody Tue Oct 7 03:50:36 2025 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6EA7F270EB2 for ; Tue, 15 Jul 2025 06:41:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.19 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752561706; cv=none; b=DvpaCaKMR4RlJW0DEWirwNsDlCJZ70B+gnzzO7jyjTxqM9dXFBliHfYf7+99+6rRAsztEl7aVIhvA8d4CukVjahH7rv7YQ+DuPmahzL2sXugWlyjLlMxfLuHfD3PB3vXZGzjA62N38FmbHiqenO3WqJvMYorOXpWdV4pj3LDu40= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752561706; c=relaxed/simple; bh=IuYZ7TDNy1bf7E/m7zyAAPqADAKNCwz0eV+rlgC83/4=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Jyk+m4qV5wMDadVX8VsmUN+fZEfWoYPtK1Az3j1jS5IvqPjchTDlh96zZcK9T0AD7Fb85lGmlAZqZ37tzGYGQu+/NRnOa+j0rxombTQBoSrHo9I+7TX1v9/p3B8L9VY1B3gEztW1s3haoTC+I++IdyY1tZ8iQvdfAzqMOSJPtpk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=W+pETf8s; arc=none smtp.client-ip=192.198.163.19 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="W+pETf8s" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1752561705; x=1784097705; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=IuYZ7TDNy1bf7E/m7zyAAPqADAKNCwz0eV+rlgC83/4=; b=W+pETf8s+pqKUVMM/BUiqBAfPUm6F3Bj/ad7izZuJsCfNqDcb0D0nKDZ tZqeR4Uhoz/2POjzT6hzvztG5B9g4+vO1j/fBaW0gG2xqNodgYOghhsXX yph8yoQC4jfcXe8dxQCtLztJww2+87OCiGP7xbDuXQSNI1oNYqY6/XowO /RRsBGdAotLOu+B1K9XPyajJ95ROX6SssJzRYPZAVC7wv1jnB41xIIuRN IF55o7LoW7U2bBLDWZxPfavckP48FBqPq4QWPPxKrRXqzQ8vXbtbfCE+7 w+lYOvv7CnxJkq/u5NuHuyBi+Z8QTwRKEsyWKsNEIFGn20KqSBoLKMfbp Q==; X-CSE-ConnectionGUID: Vnmhgp72TfWJCCbzSeBAbw== X-CSE-MsgGUID: OdvjD4lKTqWhfj5s7zCdsQ== X-IronPort-AV: E=McAfee;i="6800,10657,11491"; a="53880881" X-IronPort-AV: E=Sophos;i="6.16,313,1744095600"; d="scan'208";a="53880881" Received: from orviesa008.jf.intel.com ([10.64.159.148]) by fmvoesa113.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jul 2025 23:41:44 -0700 X-CSE-ConnectionGUID: Kzi/gXcYRHGPSA/4BHaWmg== X-CSE-MsgGUID: dg9mlxoTRJm8f0phGIyoNQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.16,313,1744095600"; d="scan'208";a="157649026" Received: from yilunxu-optiplex-7050.sh.intel.com ([10.239.159.165]) by orviesa008.jf.intel.com with ESMTP; 14 Jul 2025 23:41:40 -0700 From: Xu Yilun To: jgg@nvidia.com, jgg@ziepe.ca, kevin.tian@intel.com, will@kernel.org, aneesh.kumar@kernel.org Cc: iommu@lists.linux.dev, linux-kernel@vger.kernel.org, joro@8bytes.org, robin.murphy@arm.com, shuah@kernel.org, nicolinc@nvidia.com, aik@amd.com, dan.j.williams@intel.com, baolu.lu@linux.intel.com, yilun.xu@intel.com Subject: [PATCH v5 5/8] iommufd/vdevice: Remove struct device reference from struct vdevice Date: Tue, 15 Jul 2025 14:32:42 +0800 Message-Id: <20250715063245.1799534-6-yilun.xu@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20250715063245.1799534-1-yilun.xu@linux.intel.com> References: <20250715063245.1799534-1-yilun.xu@linux.intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Remove struct device *dev from struct vdevice. The dev pointer is the Plan B for vdevice to reference the physical device. As now vdev->idev is added without refcounting concern, just use vdev->idev->dev when needed. Reviewed-by: Jason Gunthorpe Reviewed-by: Kevin Tian Reviewed-by: Nicolin Chen Signed-off-by: Xu Yilun Tested-by: Nicolin Chen --- drivers/iommu/arm/arm-smmu-v3/tegra241-cmdqv.c | 2 +- drivers/iommu/iommufd/driver.c | 4 ++-- drivers/iommu/iommufd/viommu.c | 3 --- include/linux/iommufd.h | 1 - 4 files changed, 3 insertions(+), 7 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/tegra241-cmdqv.c b/drivers/iommu= /arm/arm-smmu-v3/tegra241-cmdqv.c index eb90af5093d8..8a515987b948 100644 --- a/drivers/iommu/arm/arm-smmu-v3/tegra241-cmdqv.c +++ b/drivers/iommu/arm/arm-smmu-v3/tegra241-cmdqv.c @@ -1218,7 +1218,7 @@ static void tegra241_vintf_destroy_vsid(struct iommuf= d_vdevice *vdev) =20 static int tegra241_vintf_init_vsid(struct iommufd_vdevice *vdev) { - struct arm_smmu_master *master =3D dev_iommu_priv_get(vdev->dev); + struct arm_smmu_master *master =3D dev_iommu_priv_get(vdev->idev->dev); struct tegra241_vintf *vintf =3D viommu_to_vintf(vdev->viommu); struct tegra241_vintf_sid *vsid =3D vdev_to_vsid(vdev); struct arm_smmu_stream *stream =3D &master->streams[0]; diff --git a/drivers/iommu/iommufd/driver.c b/drivers/iommu/iommufd/driver.c index e4eae20bcd4e..df25db6d2eaf 100644 --- a/drivers/iommu/iommufd/driver.c +++ b/drivers/iommu/iommufd/driver.c @@ -92,7 +92,7 @@ struct device *iommufd_viommu_find_dev(struct iommufd_vio= mmu *viommu, lockdep_assert_held(&viommu->vdevs.xa_lock); =20 vdev =3D xa_load(&viommu->vdevs, vdev_id); - return vdev ? vdev->dev : NULL; + return vdev ? vdev->idev->dev : NULL; } EXPORT_SYMBOL_NS_GPL(iommufd_viommu_find_dev, "IOMMUFD"); =20 @@ -109,7 +109,7 @@ int iommufd_viommu_get_vdev_id(struct iommufd_viommu *v= iommu, =20 xa_lock(&viommu->vdevs); xa_for_each(&viommu->vdevs, index, vdev) { - if (vdev->dev =3D=3D dev) { + if (vdev->idev->dev =3D=3D dev) { *vdev_id =3D vdev->virt_id; rc =3D 0; break; diff --git a/drivers/iommu/iommufd/viommu.c b/drivers/iommu/iommufd/viommu.c index ecbae5091ffe..6cf0bd5d8f08 100644 --- a/drivers/iommu/iommufd/viommu.c +++ b/drivers/iommu/iommufd/viommu.c @@ -125,7 +125,6 @@ void iommufd_vdevice_abort(struct iommufd_object *obj) xa_cmpxchg(&viommu->vdevs, vdev->virt_id, vdev, NULL, GFP_KERNEL); refcount_dec(&viommu->obj.users); idev->vdev =3D NULL; - put_device(vdev->dev); } =20 void iommufd_vdevice_destroy(struct iommufd_object *obj) @@ -203,8 +202,6 @@ int iommufd_vdevice_alloc_ioctl(struct iommufd_ucmd *uc= md) } =20 vdev->virt_id =3D virt_id; - vdev->dev =3D idev->dev; - get_device(idev->dev); vdev->viommu =3D viommu; refcount_inc(&viommu->obj.users); /* diff --git a/include/linux/iommufd.h b/include/linux/iommufd.h index b88911026bc4..ecb0c8abd251 100644 --- a/include/linux/iommufd.h +++ b/include/linux/iommufd.h @@ -109,7 +109,6 @@ struct iommufd_vdevice { struct iommufd_object obj; struct iommufd_viommu *viommu; struct iommufd_device *idev; - struct device *dev; =20 /* * Virtual device ID per vIOMMU, e.g. vSID of ARM SMMUv3, vDeviceID of --=20 2.25.1 From nobody Tue Oct 7 03:50:36 2025 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C0FD2271478 for ; Tue, 15 Jul 2025 06:41:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.19 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752561710; cv=none; b=JQ5OiF4V66vQdo9dlhbKlwnPYwe0hdeJgmkH09YVk9foMDHBFcbgnh8PLGi22zkdzxDtny9hyBEK41+QdJqqzoagotb6Qnt+ijaVo5jL9nHqJUPVMvuaQpLGErDdz27afZ142kU/KwFclAIT/MxsGKgXhw65NWZ0rfx708rerKI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752561710; c=relaxed/simple; bh=XQxU2MzFiVEMn0002F0j2LQdy7BvNN5+2QDQyQV5dAQ=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=d/k7W73PDbIC9WYYkTUhrR3CGR5vg/1Z0Nq/xuPFZelNBUO0ZwRUKgXPkk8B2YEzmqhJC2j3/XKDDpO01jlEkYeDGrKDjC7A90bTaPl5kW/kLooBo2rrXFQv+jVymf5aB8zZAZZRq7D//FPdt9sGKpRETK/PuwV0HI9v96Ch9gA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=VzkcZiXw; arc=none smtp.client-ip=192.198.163.19 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="VzkcZiXw" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1752561709; x=1784097709; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=XQxU2MzFiVEMn0002F0j2LQdy7BvNN5+2QDQyQV5dAQ=; b=VzkcZiXwOlrehMHaYfwbuHmN6xYeMeDDaqJRzzTxR43Bfe02uPwfIk6T ltqUY7lZy4MlyuQe7BTbmNkixLL2v1Ea2NKiLoIQzKPQYKREozHr18GY5 GWXoFzx89aLKvoRrQU8kgQ7eQlowWHTFPBYsP3FDhXYc+DkHiwCHaaUMw DOjyVa7o3sOjIs22YsUGM8RZa5yCq8nbcft7wUutUlTbQVtI5K6RovDGD qDndeNIzi4WVhfbSc1W7GL2E+9LYse/t9BTtXjgWAmqENUWAZqsFurzAO 0fx+in8L6D9pT5YBBWo93q0bWj6PDL5qFgydwqHXHKU7s+LgW6s2leGiw g==; X-CSE-ConnectionGUID: EMbHsp9hQ+e6qySD2Nxgxg== X-CSE-MsgGUID: 6gBAb++SQKi8hjrZkXNyQg== X-IronPort-AV: E=McAfee;i="6800,10657,11491"; a="53880893" X-IronPort-AV: E=Sophos;i="6.16,313,1744095600"; d="scan'208";a="53880893" Received: from orviesa008.jf.intel.com ([10.64.159.148]) by fmvoesa113.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jul 2025 23:41:49 -0700 X-CSE-ConnectionGUID: nIpZZPhDQdm4kBqk5/utDA== X-CSE-MsgGUID: tdsCWviJQzKHVoi93y4CSg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.16,313,1744095600"; d="scan'208";a="157649039" Received: from yilunxu-optiplex-7050.sh.intel.com ([10.239.159.165]) by orviesa008.jf.intel.com with ESMTP; 14 Jul 2025 23:41:44 -0700 From: Xu Yilun To: jgg@nvidia.com, jgg@ziepe.ca, kevin.tian@intel.com, will@kernel.org, aneesh.kumar@kernel.org Cc: iommu@lists.linux.dev, linux-kernel@vger.kernel.org, joro@8bytes.org, robin.murphy@arm.com, shuah@kernel.org, nicolinc@nvidia.com, aik@amd.com, dan.j.williams@intel.com, baolu.lu@linux.intel.com, yilun.xu@intel.com Subject: [PATCH v5 6/8] iommufd/selftest: Explicitly skip tests for inapplicable variant Date: Tue, 15 Jul 2025 14:32:43 +0800 Message-Id: <20250715063245.1799534-7-yilun.xu@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20250715063245.1799534-1-yilun.xu@linux.intel.com> References: <20250715063245.1799534-1-yilun.xu@linux.intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" no_viommu is not applicable for some viommu/vdevice tests. Explicitly report the skipping, don't do it silently. Opportunistically adjust the line wrappings after the indentation changes using git clang-format. Only add the prints. No functional change intended. Reviewed-by: Jason Gunthorpe Reviewed-by: Kevin Tian Reviewed-by: Nicolin Chen Signed-off-by: Xu Yilun Tested-by: Nicolin Chen --- tools/testing/selftests/iommu/iommufd.c | 365 ++++++++++++------------ 1 file changed, 178 insertions(+), 187 deletions(-) diff --git a/tools/testing/selftests/iommu/iommufd.c b/tools/testing/selfte= sts/iommu/iommufd.c index d59d48022a24..16ea10ea1dbf 100644 --- a/tools/testing/selftests/iommu/iommufd.c +++ b/tools/testing/selftests/iommu/iommufd.c @@ -2779,35 +2779,34 @@ TEST_F(iommufd_viommu, viommu_alloc_nested_iopf) uint32_t fault_fd; uint32_t vdev_id; =20 - if (self->device_id) { - test_ioctl_fault_alloc(&fault_id, &fault_fd); - test_err_hwpt_alloc_iopf( - ENOENT, dev_id, viommu_id, UINT32_MAX, - IOMMU_HWPT_FAULT_ID_VALID, &iopf_hwpt_id, - IOMMU_HWPT_DATA_SELFTEST, &data, sizeof(data)); - test_err_hwpt_alloc_iopf( - EOPNOTSUPP, dev_id, viommu_id, fault_id, - IOMMU_HWPT_FAULT_ID_VALID | (1 << 31), &iopf_hwpt_id, - IOMMU_HWPT_DATA_SELFTEST, &data, sizeof(data)); - test_cmd_hwpt_alloc_iopf( - dev_id, viommu_id, fault_id, IOMMU_HWPT_FAULT_ID_VALID, - &iopf_hwpt_id, IOMMU_HWPT_DATA_SELFTEST, &data, - sizeof(data)); + if (!dev_id) + SKIP(return, "Skipping test for variant no_viommu"); =20 - /* Must allocate vdevice before attaching to a nested hwpt */ - test_err_mock_domain_replace(ENOENT, self->stdev_id, - iopf_hwpt_id); - test_cmd_vdevice_alloc(viommu_id, dev_id, 0x99, &vdev_id); - test_cmd_mock_domain_replace(self->stdev_id, iopf_hwpt_id); - EXPECT_ERRNO(EBUSY, - _test_ioctl_destroy(self->fd, iopf_hwpt_id)); - test_cmd_trigger_iopf(dev_id, fault_fd); + test_ioctl_fault_alloc(&fault_id, &fault_fd); + test_err_hwpt_alloc_iopf(ENOENT, dev_id, viommu_id, UINT32_MAX, + IOMMU_HWPT_FAULT_ID_VALID, &iopf_hwpt_id, + IOMMU_HWPT_DATA_SELFTEST, &data, + sizeof(data)); + test_err_hwpt_alloc_iopf(EOPNOTSUPP, dev_id, viommu_id, fault_id, + IOMMU_HWPT_FAULT_ID_VALID | (1 << 31), + &iopf_hwpt_id, IOMMU_HWPT_DATA_SELFTEST, &data, + sizeof(data)); + test_cmd_hwpt_alloc_iopf(dev_id, viommu_id, fault_id, + IOMMU_HWPT_FAULT_ID_VALID, &iopf_hwpt_id, + IOMMU_HWPT_DATA_SELFTEST, &data, + sizeof(data)); =20 - test_cmd_mock_domain_replace(self->stdev_id, self->ioas_id); - test_ioctl_destroy(iopf_hwpt_id); - close(fault_fd); - test_ioctl_destroy(fault_id); - } + /* Must allocate vdevice before attaching to a nested hwpt */ + test_err_mock_domain_replace(ENOENT, self->stdev_id, iopf_hwpt_id); + test_cmd_vdevice_alloc(viommu_id, dev_id, 0x99, &vdev_id); + test_cmd_mock_domain_replace(self->stdev_id, iopf_hwpt_id); + EXPECT_ERRNO(EBUSY, _test_ioctl_destroy(self->fd, iopf_hwpt_id)); + test_cmd_trigger_iopf(dev_id, fault_fd); + + test_cmd_mock_domain_replace(self->stdev_id, self->ioas_id); + test_ioctl_destroy(iopf_hwpt_id); + close(fault_fd); + test_ioctl_destroy(fault_id); } =20 TEST_F(iommufd_viommu, viommu_alloc_with_data) @@ -2902,169 +2901,161 @@ TEST_F(iommufd_viommu, vdevice_cache) uint32_t vdev_id =3D 0; uint32_t num_inv; =20 - if (dev_id) { - test_cmd_vdevice_alloc(viommu_id, dev_id, 0x99, &vdev_id); - - test_cmd_dev_check_cache_all(dev_id, - IOMMU_TEST_DEV_CACHE_DEFAULT); - - /* Check data_type by passing zero-length array */ - num_inv =3D 0; - test_cmd_viommu_invalidate(viommu_id, inv_reqs, - sizeof(*inv_reqs), &num_inv); - assert(!num_inv); - - /* Negative test: Invalid data_type */ - num_inv =3D 1; - test_err_viommu_invalidate(EINVAL, viommu_id, inv_reqs, - IOMMU_VIOMMU_INVALIDATE_DATA_SELFTEST_INVALID, - sizeof(*inv_reqs), &num_inv); - assert(!num_inv); - - /* Negative test: structure size sanity */ - num_inv =3D 1; - test_err_viommu_invalidate(EINVAL, viommu_id, inv_reqs, - IOMMU_VIOMMU_INVALIDATE_DATA_SELFTEST, - sizeof(*inv_reqs) + 1, &num_inv); - assert(!num_inv); - - num_inv =3D 1; - test_err_viommu_invalidate(EINVAL, viommu_id, inv_reqs, - IOMMU_VIOMMU_INVALIDATE_DATA_SELFTEST, - 1, &num_inv); - assert(!num_inv); - - /* Negative test: invalid flag is passed */ - num_inv =3D 1; - inv_reqs[0].flags =3D 0xffffffff; - inv_reqs[0].vdev_id =3D 0x99; - test_err_viommu_invalidate(EOPNOTSUPP, viommu_id, inv_reqs, - IOMMU_VIOMMU_INVALIDATE_DATA_SELFTEST, - sizeof(*inv_reqs), &num_inv); - assert(!num_inv); - - /* Negative test: invalid data_uptr when array is not empty */ - num_inv =3D 1; - inv_reqs[0].flags =3D 0; - inv_reqs[0].vdev_id =3D 0x99; - test_err_viommu_invalidate(EINVAL, viommu_id, NULL, - IOMMU_VIOMMU_INVALIDATE_DATA_SELFTEST, - sizeof(*inv_reqs), &num_inv); - assert(!num_inv); - - /* Negative test: invalid entry_len when array is not empty */ - num_inv =3D 1; - inv_reqs[0].flags =3D 0; - inv_reqs[0].vdev_id =3D 0x99; - test_err_viommu_invalidate(EINVAL, viommu_id, inv_reqs, - IOMMU_VIOMMU_INVALIDATE_DATA_SELFTEST, - 0, &num_inv); - assert(!num_inv); - - /* Negative test: invalid cache_id */ - num_inv =3D 1; - inv_reqs[0].flags =3D 0; - inv_reqs[0].vdev_id =3D 0x99; - inv_reqs[0].cache_id =3D MOCK_DEV_CACHE_ID_MAX + 1; - test_err_viommu_invalidate(EINVAL, viommu_id, inv_reqs, - IOMMU_VIOMMU_INVALIDATE_DATA_SELFTEST, - sizeof(*inv_reqs), &num_inv); - assert(!num_inv); + if (!dev_id) + SKIP(return, "Skipping test for variant no_viommu"); =20 - /* Negative test: invalid vdev_id */ - num_inv =3D 1; - inv_reqs[0].flags =3D 0; - inv_reqs[0].vdev_id =3D 0x9; - inv_reqs[0].cache_id =3D 0; - test_err_viommu_invalidate(EINVAL, viommu_id, inv_reqs, - IOMMU_VIOMMU_INVALIDATE_DATA_SELFTEST, - sizeof(*inv_reqs), &num_inv); - assert(!num_inv); + test_cmd_vdevice_alloc(viommu_id, dev_id, 0x99, &vdev_id); + + test_cmd_dev_check_cache_all(dev_id, IOMMU_TEST_DEV_CACHE_DEFAULT); + + /* Check data_type by passing zero-length array */ + num_inv =3D 0; + test_cmd_viommu_invalidate(viommu_id, inv_reqs, sizeof(*inv_reqs), + &num_inv); + assert(!num_inv); + + /* Negative test: Invalid data_type */ + num_inv =3D 1; + test_err_viommu_invalidate(EINVAL, viommu_id, inv_reqs, + IOMMU_VIOMMU_INVALIDATE_DATA_SELFTEST_INVALID, + sizeof(*inv_reqs), &num_inv); + assert(!num_inv); + + /* Negative test: structure size sanity */ + num_inv =3D 1; + test_err_viommu_invalidate(EINVAL, viommu_id, inv_reqs, + IOMMU_VIOMMU_INVALIDATE_DATA_SELFTEST, + sizeof(*inv_reqs) + 1, &num_inv); + assert(!num_inv); + + num_inv =3D 1; + test_err_viommu_invalidate(EINVAL, viommu_id, inv_reqs, + IOMMU_VIOMMU_INVALIDATE_DATA_SELFTEST, 1, + &num_inv); + assert(!num_inv); + + /* Negative test: invalid flag is passed */ + num_inv =3D 1; + inv_reqs[0].flags =3D 0xffffffff; + inv_reqs[0].vdev_id =3D 0x99; + test_err_viommu_invalidate(EOPNOTSUPP, viommu_id, inv_reqs, + IOMMU_VIOMMU_INVALIDATE_DATA_SELFTEST, + sizeof(*inv_reqs), &num_inv); + assert(!num_inv); + + /* Negative test: invalid data_uptr when array is not empty */ + num_inv =3D 1; + inv_reqs[0].flags =3D 0; + inv_reqs[0].vdev_id =3D 0x99; + test_err_viommu_invalidate(EINVAL, viommu_id, NULL, + IOMMU_VIOMMU_INVALIDATE_DATA_SELFTEST, + sizeof(*inv_reqs), &num_inv); + assert(!num_inv); + + /* Negative test: invalid entry_len when array is not empty */ + num_inv =3D 1; + inv_reqs[0].flags =3D 0; + inv_reqs[0].vdev_id =3D 0x99; + test_err_viommu_invalidate(EINVAL, viommu_id, inv_reqs, + IOMMU_VIOMMU_INVALIDATE_DATA_SELFTEST, 0, + &num_inv); + assert(!num_inv); + + /* Negative test: invalid cache_id */ + num_inv =3D 1; + inv_reqs[0].flags =3D 0; + inv_reqs[0].vdev_id =3D 0x99; + inv_reqs[0].cache_id =3D MOCK_DEV_CACHE_ID_MAX + 1; + test_err_viommu_invalidate(EINVAL, viommu_id, inv_reqs, + IOMMU_VIOMMU_INVALIDATE_DATA_SELFTEST, + sizeof(*inv_reqs), &num_inv); + assert(!num_inv); + + /* Negative test: invalid vdev_id */ + num_inv =3D 1; + inv_reqs[0].flags =3D 0; + inv_reqs[0].vdev_id =3D 0x9; + inv_reqs[0].cache_id =3D 0; + test_err_viommu_invalidate(EINVAL, viommu_id, inv_reqs, + IOMMU_VIOMMU_INVALIDATE_DATA_SELFTEST, + sizeof(*inv_reqs), &num_inv); + assert(!num_inv); =20 - /* - * Invalidate the 1st cache entry but fail the 2nd request - * due to invalid flags configuration in the 2nd request. - */ - num_inv =3D 2; - inv_reqs[0].flags =3D 0; - inv_reqs[0].vdev_id =3D 0x99; - inv_reqs[0].cache_id =3D 0; - inv_reqs[1].flags =3D 0xffffffff; - inv_reqs[1].vdev_id =3D 0x99; - inv_reqs[1].cache_id =3D 1; - test_err_viommu_invalidate(EOPNOTSUPP, viommu_id, inv_reqs, - IOMMU_VIOMMU_INVALIDATE_DATA_SELFTEST, - sizeof(*inv_reqs), &num_inv); - assert(num_inv =3D=3D 1); - test_cmd_dev_check_cache(dev_id, 0, 0); - test_cmd_dev_check_cache(dev_id, 1, - IOMMU_TEST_DEV_CACHE_DEFAULT); - test_cmd_dev_check_cache(dev_id, 2, - IOMMU_TEST_DEV_CACHE_DEFAULT); - test_cmd_dev_check_cache(dev_id, 3, - IOMMU_TEST_DEV_CACHE_DEFAULT); - - /* - * Invalidate the 1st cache entry but fail the 2nd request - * due to invalid cache_id configuration in the 2nd request. - */ - num_inv =3D 2; - inv_reqs[0].flags =3D 0; - inv_reqs[0].vdev_id =3D 0x99; - inv_reqs[0].cache_id =3D 0; - inv_reqs[1].flags =3D 0; - inv_reqs[1].vdev_id =3D 0x99; - inv_reqs[1].cache_id =3D MOCK_DEV_CACHE_ID_MAX + 1; - test_err_viommu_invalidate(EINVAL, viommu_id, inv_reqs, - IOMMU_VIOMMU_INVALIDATE_DATA_SELFTEST, - sizeof(*inv_reqs), &num_inv); - assert(num_inv =3D=3D 1); - test_cmd_dev_check_cache(dev_id, 0, 0); - test_cmd_dev_check_cache(dev_id, 1, - IOMMU_TEST_DEV_CACHE_DEFAULT); - test_cmd_dev_check_cache(dev_id, 2, - IOMMU_TEST_DEV_CACHE_DEFAULT); - test_cmd_dev_check_cache(dev_id, 3, - IOMMU_TEST_DEV_CACHE_DEFAULT); - - /* Invalidate the 2nd cache entry and verify */ - num_inv =3D 1; - inv_reqs[0].flags =3D 0; - inv_reqs[0].vdev_id =3D 0x99; - inv_reqs[0].cache_id =3D 1; - test_cmd_viommu_invalidate(viommu_id, inv_reqs, - sizeof(*inv_reqs), &num_inv); - assert(num_inv =3D=3D 1); - test_cmd_dev_check_cache(dev_id, 0, 0); - test_cmd_dev_check_cache(dev_id, 1, 0); - test_cmd_dev_check_cache(dev_id, 2, - IOMMU_TEST_DEV_CACHE_DEFAULT); - test_cmd_dev_check_cache(dev_id, 3, - IOMMU_TEST_DEV_CACHE_DEFAULT); - - /* Invalidate the 3rd and 4th cache entries and verify */ - num_inv =3D 2; - inv_reqs[0].flags =3D 0; - inv_reqs[0].vdev_id =3D 0x99; - inv_reqs[0].cache_id =3D 2; - inv_reqs[1].flags =3D 0; - inv_reqs[1].vdev_id =3D 0x99; - inv_reqs[1].cache_id =3D 3; - test_cmd_viommu_invalidate(viommu_id, inv_reqs, - sizeof(*inv_reqs), &num_inv); - assert(num_inv =3D=3D 2); - test_cmd_dev_check_cache_all(dev_id, 0); + /* + * Invalidate the 1st cache entry but fail the 2nd request + * due to invalid flags configuration in the 2nd request. + */ + num_inv =3D 2; + inv_reqs[0].flags =3D 0; + inv_reqs[0].vdev_id =3D 0x99; + inv_reqs[0].cache_id =3D 0; + inv_reqs[1].flags =3D 0xffffffff; + inv_reqs[1].vdev_id =3D 0x99; + inv_reqs[1].cache_id =3D 1; + test_err_viommu_invalidate(EOPNOTSUPP, viommu_id, inv_reqs, + IOMMU_VIOMMU_INVALIDATE_DATA_SELFTEST, + sizeof(*inv_reqs), &num_inv); + assert(num_inv =3D=3D 1); + test_cmd_dev_check_cache(dev_id, 0, 0); + test_cmd_dev_check_cache(dev_id, 1, IOMMU_TEST_DEV_CACHE_DEFAULT); + test_cmd_dev_check_cache(dev_id, 2, IOMMU_TEST_DEV_CACHE_DEFAULT); + test_cmd_dev_check_cache(dev_id, 3, IOMMU_TEST_DEV_CACHE_DEFAULT); =20 - /* Invalidate all cache entries for nested_dev_id[1] and verify */ - num_inv =3D 1; - inv_reqs[0].vdev_id =3D 0x99; - inv_reqs[0].flags =3D IOMMU_TEST_INVALIDATE_FLAG_ALL; - test_cmd_viommu_invalidate(viommu_id, inv_reqs, - sizeof(*inv_reqs), &num_inv); - assert(num_inv =3D=3D 1); - test_cmd_dev_check_cache_all(dev_id, 0); - test_ioctl_destroy(vdev_id); - } + /* + * Invalidate the 1st cache entry but fail the 2nd request + * due to invalid cache_id configuration in the 2nd request. + */ + num_inv =3D 2; + inv_reqs[0].flags =3D 0; + inv_reqs[0].vdev_id =3D 0x99; + inv_reqs[0].cache_id =3D 0; + inv_reqs[1].flags =3D 0; + inv_reqs[1].vdev_id =3D 0x99; + inv_reqs[1].cache_id =3D MOCK_DEV_CACHE_ID_MAX + 1; + test_err_viommu_invalidate(EINVAL, viommu_id, inv_reqs, + IOMMU_VIOMMU_INVALIDATE_DATA_SELFTEST, + sizeof(*inv_reqs), &num_inv); + assert(num_inv =3D=3D 1); + test_cmd_dev_check_cache(dev_id, 0, 0); + test_cmd_dev_check_cache(dev_id, 1, IOMMU_TEST_DEV_CACHE_DEFAULT); + test_cmd_dev_check_cache(dev_id, 2, IOMMU_TEST_DEV_CACHE_DEFAULT); + test_cmd_dev_check_cache(dev_id, 3, IOMMU_TEST_DEV_CACHE_DEFAULT); + + /* Invalidate the 2nd cache entry and verify */ + num_inv =3D 1; + inv_reqs[0].flags =3D 0; + inv_reqs[0].vdev_id =3D 0x99; + inv_reqs[0].cache_id =3D 1; + test_cmd_viommu_invalidate(viommu_id, inv_reqs, sizeof(*inv_reqs), + &num_inv); + assert(num_inv =3D=3D 1); + test_cmd_dev_check_cache(dev_id, 0, 0); + test_cmd_dev_check_cache(dev_id, 1, 0); + test_cmd_dev_check_cache(dev_id, 2, IOMMU_TEST_DEV_CACHE_DEFAULT); + test_cmd_dev_check_cache(dev_id, 3, IOMMU_TEST_DEV_CACHE_DEFAULT); + + /* Invalidate the 3rd and 4th cache entries and verify */ + num_inv =3D 2; + inv_reqs[0].flags =3D 0; + inv_reqs[0].vdev_id =3D 0x99; + inv_reqs[0].cache_id =3D 2; + inv_reqs[1].flags =3D 0; + inv_reqs[1].vdev_id =3D 0x99; + inv_reqs[1].cache_id =3D 3; + test_cmd_viommu_invalidate(viommu_id, inv_reqs, sizeof(*inv_reqs), + &num_inv); + assert(num_inv =3D=3D 2); + test_cmd_dev_check_cache_all(dev_id, 0); + + /* Invalidate all cache entries for nested_dev_id[1] and verify */ + num_inv =3D 1; + inv_reqs[0].vdev_id =3D 0x99; + inv_reqs[0].flags =3D IOMMU_TEST_INVALIDATE_FLAG_ALL; + test_cmd_viommu_invalidate(viommu_id, inv_reqs, sizeof(*inv_reqs), + &num_inv); + assert(num_inv =3D=3D 1); + test_cmd_dev_check_cache_all(dev_id, 0); + test_ioctl_destroy(vdev_id); } =20 TEST_F(iommufd_viommu, hw_queue) --=20 2.25.1 From nobody Tue Oct 7 03:50:36 2025 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 87248272806 for ; Tue, 15 Jul 2025 06:41:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.19 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752561713; cv=none; b=NrGFDBMVOHMihPTllhsidZW5auTv7bNc9Fk8NkrLncvaa0JqT7IPugfquZvyU4SZ3ki1aUFbvKbYYKb23OYNh0GRBV/byUCg0mm37Ud31KmC1/U0jGbI/lebilDia/nc42Sqmoaz9J7idKrMA0KxElgw+bIFkU7rmcvDoEFXjAs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752561713; c=relaxed/simple; bh=+FnZBthc0QfK6q6BLjuKbUJ1xSApwzKyXwldHpF8ahA=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=UNa9R94OJAs7o2q/+lTZWTdRfT/SI+kIsWVU4Ol25MS8OpBACEi6PFUdp+wFLkB5XdusOD/3QJUDL0Lsrp9uXfXdMtJ5B+XIdgUvM7wzoZGtpX84Sgky0o/Rs972WMWtNck9tFevLV13n5zAaA7pfwJRobdHnjidOYlpzBoCU4I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=jN7IV/Oh; arc=none smtp.client-ip=192.198.163.19 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="jN7IV/Oh" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1752561713; x=1784097713; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=+FnZBthc0QfK6q6BLjuKbUJ1xSApwzKyXwldHpF8ahA=; b=jN7IV/OhuV/U7PkSOc5q9j0KDgcP3f0NDECzrW451t9cbWRfenVD8d6q IoZJGs0jY41m41o5CXH11fTV3/l6MrvIsD/ZIIt2wnD5jugDebXb7oFMI qI2uMQBqpbE3PINT+zZ0O5ymaM0p27kUhyjvKWDbks1cdaAmSFm87jj5A dKe1pqtRZdykjkJOY3aXocv6u7dka/nxiTPsT7j9NTSDA53NSC7L3yPA6 E3xJ9DE8BbP7D8bRQ8S/lCqCZIVvvfJw+SuGJlNCt4Jn7agucWUHlQpwb EEJBHVagL7CRPtrbnas992K1eIxLAPEHOdglQESyLMM3NYj38qRj/gds2 w==; X-CSE-ConnectionGUID: ydfnjaNIQIurOwsNGo9wRA== X-CSE-MsgGUID: 1kXYbicNQ2CLHAtXUqdfng== X-IronPort-AV: E=McAfee;i="6800,10657,11491"; a="53880902" X-IronPort-AV: E=Sophos;i="6.16,313,1744095600"; d="scan'208";a="53880902" Received: from orviesa008.jf.intel.com ([10.64.159.148]) by fmvoesa113.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jul 2025 23:41:52 -0700 X-CSE-ConnectionGUID: IqY9hFhuSxu04MSLDUl62A== X-CSE-MsgGUID: gvCMW5LITt2VXXjo8PAwxA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.16,313,1744095600"; d="scan'208";a="157649045" Received: from yilunxu-optiplex-7050.sh.intel.com ([10.239.159.165]) by orviesa008.jf.intel.com with ESMTP; 14 Jul 2025 23:41:48 -0700 From: Xu Yilun To: jgg@nvidia.com, jgg@ziepe.ca, kevin.tian@intel.com, will@kernel.org, aneesh.kumar@kernel.org Cc: iommu@lists.linux.dev, linux-kernel@vger.kernel.org, joro@8bytes.org, robin.murphy@arm.com, shuah@kernel.org, nicolinc@nvidia.com, aik@amd.com, dan.j.williams@intel.com, baolu.lu@linux.intel.com, yilun.xu@intel.com Subject: [PATCH v5 7/8] iommufd/selftest: Add coverage for vdevice tombstone Date: Tue, 15 Jul 2025 14:32:44 +0800 Message-Id: <20250715063245.1799534-8-yilun.xu@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20250715063245.1799534-1-yilun.xu@linux.intel.com> References: <20250715063245.1799534-1-yilun.xu@linux.intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" This tests the flow to tombstone vdevice when idevice is to be unbound before vdevice destruction. The expected results of the tombstone are: - The vdevice ID can't be reused anymore (not tested in this patch). - Even ioctl(IOMMU_DESTROY) can't free the vdevice ID. - iommufd_fops_release() can still free everything. Signed-off-by: Xu Yilun Reviewed-by: Nicolin Chen Tested-by: Nicolin Chen --- tools/testing/selftests/iommu/iommufd.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/tools/testing/selftests/iommu/iommufd.c b/tools/testing/selfte= sts/iommu/iommufd.c index 16ea10ea1dbf..6084396f7111 100644 --- a/tools/testing/selftests/iommu/iommufd.c +++ b/tools/testing/selftests/iommu/iommufd.c @@ -3117,6 +3117,20 @@ TEST_F(iommufd_viommu, hw_queue) test_ioctl_ioas_unmap(iova, PAGE_SIZE); } =20 +TEST_F(iommufd_viommu, vdevice_tombstone) +{ + uint32_t viommu_id =3D self->viommu_id; + uint32_t dev_id =3D self->device_id; + uint32_t vdev_id =3D 0; + + if (!dev_id) + SKIP(return, "Skipping test for variant no_viommu"); + + test_cmd_vdevice_alloc(viommu_id, dev_id, 0x99, &vdev_id); + test_ioctl_destroy(self->stdev_id); + EXPECT_ERRNO(ENOENT, _test_ioctl_destroy(self->fd, vdev_id)); +} + FIXTURE(iommufd_device_pasid) { int fd; --=20 2.25.1 From nobody Tue Oct 7 03:50:36 2025 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D098F273D63 for ; Tue, 15 Jul 2025 06:41:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.19 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752561719; cv=none; b=sTrHf5Yb3lzDa5FzWi2WgiRbqpMmFKmuGurSbGjPnde2nWTKSBcOFEYQ2zU+uARYSjIUp8liygznYub5qhBjlbF84NSUrs0uaAuQjQEaS9OttGnd7swzkoKTB+YenWzptw0PWoYlr+6/W6eTD9RZYV/+sewrBvOPgi2khXe+mYE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752561719; c=relaxed/simple; bh=epwSE/ikRyIkgIQU9p6ks0NaXeHbxkMa/0QDmh9LA3Y=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=SEqHEacFGeMrh50s1R2z8W8hI5DXIcXr/HTh7rl6KWWGl01t50bFJpdbV6L8BqAz3UgpYnrZGNSmkrG/WBEFxhIQtpH4PPRNW3MNhd0CS+7vLpI9hcI1KxOcAIVnXGtQ5BTRWyauV+rYquWOCxk0quxmAaKyho8Pa4bmCg9nIOM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=mCdA6SSm; arc=none smtp.client-ip=192.198.163.19 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="mCdA6SSm" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1752561717; x=1784097717; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=epwSE/ikRyIkgIQU9p6ks0NaXeHbxkMa/0QDmh9LA3Y=; b=mCdA6SSmnhiXzCLYuFwUxlFMrD9piZHgQUMpIZa9WbUV/O8KR/q0OvZ8 fGV3h7RAQ136imboySXS1tQhnMD5IGn2bfiWg9BwpBdCw4j+JZx25jV8w oXAi9r92tO8HhvfZ7Dkjkcz6/7ZAPj0vGUldciYlYqDXwkXtpW9NM8DSC dCUPyOpPhMYGqthuPY/tmMc59D4teTwZDF6EAfTiv2ReLz6QiXZIBuvbW S6yzbpQBsoUeFY0HhKvLrJjYxG3YsdpL1DJDe6YhPI7gPHb4CqzPpNfDI 04tgFANynjYF05C2DnFnWSYUCBJsrOCif3sSadchk7Ci4PODMrmZa3XHt A==; X-CSE-ConnectionGUID: Rm6iS9z4Tw2MeSXP5Lk4MA== X-CSE-MsgGUID: 9/OsR+pMStiLg4hMKyzdDw== X-IronPort-AV: E=McAfee;i="6800,10657,11491"; a="53880912" X-IronPort-AV: E=Sophos;i="6.16,313,1744095600"; d="scan'208";a="53880912" Received: from orviesa008.jf.intel.com ([10.64.159.148]) by fmvoesa113.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jul 2025 23:41:57 -0700 X-CSE-ConnectionGUID: XwNyccElToORXS0eR0wzJQ== X-CSE-MsgGUID: +vqMGmQrRRelXVVqlOYW8g== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.16,313,1744095600"; d="scan'208";a="157649069" Received: from yilunxu-optiplex-7050.sh.intel.com ([10.239.159.165]) by orviesa008.jf.intel.com with ESMTP; 14 Jul 2025 23:41:52 -0700 From: Xu Yilun To: jgg@nvidia.com, jgg@ziepe.ca, kevin.tian@intel.com, will@kernel.org, aneesh.kumar@kernel.org Cc: iommu@lists.linux.dev, linux-kernel@vger.kernel.org, joro@8bytes.org, robin.murphy@arm.com, shuah@kernel.org, nicolinc@nvidia.com, aik@amd.com, dan.j.williams@intel.com, baolu.lu@linux.intel.com, yilun.xu@intel.com Subject: [PATCH v5 8/8] iommufd: Rename some shortterm-related identifiers Date: Tue, 15 Jul 2025 14:32:45 +0800 Message-Id: <20250715063245.1799534-9-yilun.xu@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20250715063245.1799534-1-yilun.xu@linux.intel.com> References: <20250715063245.1799534-1-yilun.xu@linux.intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Rename the shortterm-related identifiers to wait-related. The usage of shortterm_users refcount is now beyond its name. It is also used for references which live longer than an ioctl execution. E.g. vdev holds idev's shortterm_users refcount on vdev allocation, releases it during idev's pre_destroy(). Rename the refcount as wait_cnt, since it is always used to sync the referencing & the destruction of the object by waiting for it to go to zero. List all changed identifiers: iommufd_object::shortterm_users -> iommufd_object::wait_cnt REMOVE_WAIT_SHORTTERM -> REMOVE_WAIT iommufd_object_dec_wait_shortterm() -> iommufd_object_dec_wait() zerod_shortterm -> zerod_wait_cnt No functional change intended. Suggested-by: Kevin Tian Suggested-by: Jason Gunthorpe Signed-off-by: Xu Yilun Reviewed-by: Jason Gunthorpe Reviewed-by: Nicolin Chen Tested-by: Nicolin Chen --- drivers/iommu/iommufd/device.c | 6 ++-- drivers/iommu/iommufd/iommufd_private.h | 18 ++++++------ drivers/iommu/iommufd/main.c | 39 +++++++++++++------------ drivers/iommu/iommufd/viommu.c | 4 +-- include/linux/iommufd.h | 8 ++++- 5 files changed, 41 insertions(+), 34 deletions(-) diff --git a/drivers/iommu/iommufd/device.c b/drivers/iommu/iommufd/device.c index ee6ff4caf398..65fbd098f9e9 100644 --- a/drivers/iommu/iommufd/device.c +++ b/drivers/iommu/iommufd/device.c @@ -152,8 +152,8 @@ static void iommufd_device_remove_vdev(struct iommufd_d= evice *idev) /* * An ongoing vdev destroy ioctl has removed the vdev from the object * xarray, but has not finished iommufd_vdevice_destroy() yet as it - * needs the same mutex. We exit the locking then wait on short term - * users for the vdev destruction. + * needs the same mutex. We exit the locking then wait on wait_cnt + * reference for the vdev destruction. */ if (IS_ERR(vdev)) goto out_unlock; @@ -184,7 +184,7 @@ void iommufd_device_pre_destroy(struct iommufd_object *= obj) struct iommufd_device *idev =3D container_of(obj, struct iommufd_device, obj); =20 - /* Release the short term users on this */ + /* Release the wait_cnt reference on this */ iommufd_device_remove_vdev(idev); } =20 diff --git a/drivers/iommu/iommufd/iommufd_private.h b/drivers/iommu/iommuf= d/iommufd_private.h index 5d6ea5395cfe..0da2a81eedfa 100644 --- a/drivers/iommu/iommufd/iommufd_private.h +++ b/drivers/iommu/iommufd/iommufd_private.h @@ -169,7 +169,7 @@ static inline bool iommufd_lock_obj(struct iommufd_obje= ct *obj) { if (!refcount_inc_not_zero(&obj->users)) return false; - if (!refcount_inc_not_zero(&obj->shortterm_users)) { + if (!refcount_inc_not_zero(&obj->wait_cnt)) { /* * If the caller doesn't already have a ref on obj this must be * called under the xa_lock. Otherwise the caller is holding a @@ -187,11 +187,11 @@ static inline void iommufd_put_object(struct iommufd_= ctx *ictx, struct iommufd_object *obj) { /* - * Users first, then shortterm so that REMOVE_WAIT_SHORTTERM never sees - * a spurious !0 users with a 0 shortterm_users. + * Users first, then wait_cnt so that REMOVE_WAIT never sees a spurious + * !0 users with a 0 wait_cnt. */ refcount_dec(&obj->users); - if (refcount_dec_and_test(&obj->shortterm_users)) + if (refcount_dec_and_test(&obj->wait_cnt)) wake_up_interruptible_all(&ictx->destroy_wait); } =20 @@ -202,7 +202,7 @@ void iommufd_object_finalize(struct iommufd_ctx *ictx, struct iommufd_object *obj); =20 enum { - REMOVE_WAIT_SHORTTERM =3D BIT(0), + REMOVE_WAIT =3D BIT(0), REMOVE_OBJ_TOMBSTONE =3D BIT(1), }; int iommufd_object_remove(struct iommufd_ctx *ictx, @@ -211,15 +211,15 @@ int iommufd_object_remove(struct iommufd_ctx *ictx, =20 /* * The caller holds a users refcount and wants to destroy the object. At t= his - * point the caller has no shortterm_users reference and at least the xarr= ay - * will be holding one. + * point the caller has no wait_cnt reference and at least the xarray will= be + * holding one. */ static inline void iommufd_object_destroy_user(struct iommufd_ctx *ictx, struct iommufd_object *obj) { int ret; =20 - ret =3D iommufd_object_remove(ictx, obj, obj->id, REMOVE_WAIT_SHORTTERM); + ret =3D iommufd_object_remove(ictx, obj, obj->id, REMOVE_WAIT); =20 /* * If there is a bug and we couldn't destroy the object then we did put @@ -239,7 +239,7 @@ static inline void iommufd_object_tombstone_user(struct= iommufd_ctx *ictx, int ret; =20 ret =3D iommufd_object_remove(ictx, obj, obj->id, - REMOVE_WAIT_SHORTTERM | REMOVE_OBJ_TOMBSTONE); + REMOVE_WAIT | REMOVE_OBJ_TOMBSTONE); =20 /* * If there is a bug and we couldn't destroy the object then we did put diff --git a/drivers/iommu/iommufd/main.c b/drivers/iommu/iommufd/main.c index 99c1aab3d396..15af7ced0501 100644 --- a/drivers/iommu/iommufd/main.c +++ b/drivers/iommu/iommufd/main.c @@ -42,7 +42,7 @@ struct iommufd_object *_iommufd_object_alloc(struct iommu= fd_ctx *ictx, return ERR_PTR(-ENOMEM); obj->type =3D type; /* Starts out bias'd by 1 until it is removed from the xarray */ - refcount_set(&obj->shortterm_users, 1); + refcount_set(&obj->wait_cnt, 1); refcount_set(&obj->users, 1); =20 /* @@ -155,22 +155,22 @@ struct iommufd_object *iommufd_get_object(struct iomm= ufd_ctx *ictx, u32 id, return obj; } =20 -static int iommufd_object_dec_wait_shortterm(struct iommufd_ctx *ictx, - struct iommufd_object *to_destroy) +static int iommufd_object_dec_wait(struct iommufd_ctx *ictx, + struct iommufd_object *to_destroy) { - if (refcount_dec_and_test(&to_destroy->shortterm_users)) + if (refcount_dec_and_test(&to_destroy->wait_cnt)) return 0; =20 if (iommufd_object_ops[to_destroy->type].pre_destroy) iommufd_object_ops[to_destroy->type].pre_destroy(to_destroy); =20 if (wait_event_timeout(ictx->destroy_wait, - refcount_read(&to_destroy->shortterm_users) =3D=3D 0, + refcount_read(&to_destroy->wait_cnt) =3D=3D 0, msecs_to_jiffies(60000))) return 0; =20 pr_crit("Time out waiting for iommufd object to become free\n"); - refcount_inc(&to_destroy->shortterm_users); + refcount_inc(&to_destroy->wait_cnt); return -EBUSY; } =20 @@ -184,17 +184,18 @@ int iommufd_object_remove(struct iommufd_ctx *ictx, { struct iommufd_object *obj; XA_STATE(xas, &ictx->objects, id); - bool zerod_shortterm =3D false; + bool zerod_wait_cnt =3D false; int ret; =20 /* - * The purpose of the shortterm_users is to ensure deterministic - * destruction of objects used by external drivers and destroyed by this - * function. Any temporary increment of the refcount must increment - * shortterm_users, such as during ioctl execution. + * The purpose of the wait_cnt is to ensure deterministic destruction + * of objects used by external drivers and destroyed by this function. + * Incrementing this wait_cnt should either be short lived, such as + * during ioctl execution, or be revoked and blocked during + * pre_destroy(), such as vdev holding the idev's refcount. */ - if (flags & REMOVE_WAIT_SHORTTERM) { - ret =3D iommufd_object_dec_wait_shortterm(ictx, to_destroy); + if (flags & REMOVE_WAIT) { + ret =3D iommufd_object_dec_wait(ictx, to_destroy); if (ret) { /* * We have a bug. Put back the callers reference and @@ -203,7 +204,7 @@ int iommufd_object_remove(struct iommufd_ctx *ictx, refcount_dec(&to_destroy->users); return ret; } - zerod_shortterm =3D true; + zerod_wait_cnt =3D true; } =20 xa_lock(&ictx->objects); @@ -235,11 +236,11 @@ int iommufd_object_remove(struct iommufd_ctx *ictx, xa_unlock(&ictx->objects); =20 /* - * Since users is zero any positive users_shortterm must be racing + * Since users is zero any positive wait_cnt must be racing * iommufd_put_object(), or we have a bug. */ - if (!zerod_shortterm) { - ret =3D iommufd_object_dec_wait_shortterm(ictx, obj); + if (!zerod_wait_cnt) { + ret =3D iommufd_object_dec_wait(ictx, obj); if (WARN_ON(ret)) return ret; } @@ -249,9 +250,9 @@ int iommufd_object_remove(struct iommufd_ctx *ictx, return 0; =20 err_xa: - if (zerod_shortterm) { + if (zerod_wait_cnt) { /* Restore the xarray owned reference */ - refcount_set(&obj->shortterm_users, 1); + refcount_set(&obj->wait_cnt, 1); } xa_unlock(&ictx->objects); =20 diff --git a/drivers/iommu/iommufd/viommu.c b/drivers/iommu/iommufd/viommu.c index 6cf0bd5d8f08..2ca5809b238b 100644 --- a/drivers/iommu/iommufd/viommu.c +++ b/drivers/iommu/iommufd/viommu.c @@ -205,8 +205,8 @@ int iommufd_vdevice_alloc_ioctl(struct iommufd_ucmd *uc= md) vdev->viommu =3D viommu; refcount_inc(&viommu->obj.users); /* - * A short term users reference is held on the idev so long as we have - * the pointer. iommufd_device_pre_destroy() will revoke it before the + * A wait_cnt reference is held on the idev so long as we have the + * pointer. iommufd_device_pre_destroy() will revoke it before the * idev real destruction. */ vdev->idev =3D idev; diff --git a/include/linux/iommufd.h b/include/linux/iommufd.h index ecb0c8abd251..61410a78cbce 100644 --- a/include/linux/iommufd.h +++ b/include/linux/iommufd.h @@ -46,7 +46,13 @@ enum iommufd_object_type { =20 /* Base struct for all objects with a userspace ID handle. */ struct iommufd_object { - refcount_t shortterm_users; + /* + * Destroy will sleep and wait for wait_cnt to go to zero. This allows + * concurrent users of the ID to reliably avoid causing a spurious + * destroy failure. Incrementing this count should either be short + * lived or be revoked and blocked during pre_destroy(). + */ + refcount_t wait_cnt; refcount_t users; enum iommufd_object_type type; unsigned int id; --=20 2.25.1