From nobody Mon Nov 25 00:39:33 2024 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2057.outbound.protection.outlook.com [40.107.93.57]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0CCE3228037; Wed, 30 Oct 2024 21:35:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.93.57 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730324107; cv=fail; b=PvV9E02geUv/D395yzRtK4er86V9KLeqN8cByrFli2LS3dzNKwCz6J9M9XOBRx6/7uuMWqRS2tBQUNKysnTzymXJoQC5Z50TaEupu4meh4+xM2zf6q5FGsySrtfY6tE8ygmPm2dAdNYHWGjki7tVrI45B/rNyZrKmbCxOeHAVp4= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730324107; c=relaxed/simple; bh=7sPB15V/w4wInnTmw0ej6hMvg40uSiSj5vrqQcGoylU=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=eXgDdabno99Z0soear8HNl1J3vCpwBMOdD0JFsx83B8F/ZnURaGRVwZLQFddTIeGWA1XPPiFEoikPC9fN1utMZPB59PJV3wLcOiCWCTabeSFr5vYd0KxCVlLipVhm/grYrhWuzird5LlqPmrSAYI8YceL3B52GvHi20NmtyDpg4= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=tYAutUch; arc=fail smtp.client-ip=40.107.93.57 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="tYAutUch" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=BiYwDhdwmEME0V1zcq5TjwAMKi58h+JkULsyijucYAJm9jqWKnzMQLMRRJhitY/m+I1vseiccXICiD4053O4NpghPYOeQfA1oOAestBssMSA1GZ2uJMcv2qQuAH2ittWbBHeO9spySsOmctoJO18R1S0R3a+6j+xyWp8HBJ/u08WZ4n2Z3Jbu7wSORjD/4aRY2qwK90tX5+sAPDZ72q8M633L5+SwSf2NfkUIHptnK47IjtwhHJMXGWUUacxIP5+UqOl6uNqN3JguHhmwd77o4d3LVW4XlkLpTm3F7QTuklpyJLVZvtML0RZSE6Fzy+o8DqC+2mrRAM1XyOCeWjOOw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=2m6/F+JKpnHyzJ1kxenJ++GEDw1KOY8ocDrGh1wzehw=; b=h2kh1xp6dwYnrEUUsmurTRRrKvyHF/fPm3ofaJV4682CP5uUAXOnS95pSBzlH9SbaBhf9Ij7CGDulUUEnbmbTOa+6kcVWNquyk7k3cwvz9DiUdNquv/clxFNn88KUkyRilkXPQoeETv+PECV+aJuuoGPTjWwIkx1CYRa9hEzqFxXfXYvBYa2sUsa01X/vKvXtv4tdblJecwG/ZFQnOlKfPOy+PTku4TM+dqVp9q7rtmmj3ehsKGvkC7YM8NRZo6Brc+9CEdtYNhwK0w7bqTR2HhOW27PweDQHIf8BT87ANqkzd1Koyt3MxHKKym11z1DA6W+ezXiHNFPpyFAxh9PQQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.118.233) smtp.rcpttodomain=amd.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=2m6/F+JKpnHyzJ1kxenJ++GEDw1KOY8ocDrGh1wzehw=; b=tYAutUchnyfVXMC8bX/wLYDqrYNSV1syxNn+jbSCmBv72Rqx1X7XALJwduBUY8XTPbVn5aDpIxtUxk6s2wepqBGP5qgo237i8qtmVwS+fIIse/l1ejmXnU69w9OuS3L+w8z90U3nN/bxajd/1lTvY1yTuQ9aSU+sGxErhU9vrAMPkBHGlVBxDy2HQg2hyxkGm+Zn7zQokgNwdFa6EzaeZE/bQweoKrzfb1z/KIXAfvJF7lj6bth3YuIiJoOWYQ+B6UEvR4Le6p4Flz5ftfm9QVeErrry+LlNmKqEh6XUcG1P2GN5RLQbHbdCKTK0NtfwwaA/VcM7hd+f0XOheNlk7g== Received: from CH0PR13CA0060.namprd13.prod.outlook.com (2603:10b6:610:b2::35) by DM6PR12MB4282.namprd12.prod.outlook.com (2603:10b6:5:223::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8093.32; Wed, 30 Oct 2024 21:35:01 +0000 Received: from DS3PEPF000099E1.namprd04.prod.outlook.com (2603:10b6:610:b2:cafe::63) by CH0PR13CA0060.outlook.office365.com (2603:10b6:610:b2::35) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8114.8 via Frontend Transport; Wed, 30 Oct 2024 21:35:01 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.118.233) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.118.233 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.118.233; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.118.233) by DS3PEPF000099E1.mail.protection.outlook.com (10.167.17.196) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8114.16 via Frontend Transport; Wed, 30 Oct 2024 21:35:01 +0000 Received: from drhqmail203.nvidia.com (10.126.190.182) by mail.nvidia.com (10.127.129.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Wed, 30 Oct 2024 14:34:57 -0700 Received: from drhqmail202.nvidia.com (10.126.190.181) by drhqmail203.nvidia.com (10.126.190.182) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Wed, 30 Oct 2024 14:34:57 -0700 Received: from Asurada-Nvidia.nvidia.com (10.127.8.12) by mail.nvidia.com (10.126.190.181) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Wed, 30 Oct 2024 14:34:56 -0700 From: Nicolin Chen To: , , CC: , , , , , , , , , , , , , , , , , , , , Subject: [PATCH v6 01/13] iommufd: Move struct iommufd_object to public iommufd header Date: Wed, 30 Oct 2024 14:34:27 -0700 Message-ID: <54a43b0768089d690104530754f499ca05ce0074.1730313237.git.nicolinc@nvidia.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS3PEPF000099E1:EE_|DM6PR12MB4282:EE_ X-MS-Office365-Filtering-Correlation-Id: f9262d02-6e63-4bb1-eceb-08dcf92ab5ac X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|36860700013|7416014|376014|82310400026; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?2ii6SAf6zRhjKtQ0ey4+MPL4/W6hbqbEe23hb71HhOFWAvUWczTmVzvSbA6R?= =?us-ascii?Q?c0cONHr0rXeY8bmOlpcdCvj0VeuraChWnxh0wHNlqkte388kISNV0t4YF8DB?= =?us-ascii?Q?/obuUyQ4as0GPBlWajkO9oOMgha3VxpeFjw53ZxcvtDw1BcSIbKh7WOVHVvz?= =?us-ascii?Q?ZxbUlFJ4p0BhMqZd1vX4SyCp1tIv4mJY8VyyL25bD/BS2EluGsuL5K9YUdLz?= =?us-ascii?Q?JDpE57gDrPgX0RKCdo7IzybssaFCm1WPR2uv/FAu45ciuDpBaBJUBYAn19CN?= =?us-ascii?Q?bU82Ws9tl+FEFS31tG7K5p+wAXYrg9yX7R/YocLihY4WJnofSmDJvEkgtBgO?= =?us-ascii?Q?fAtzsisRgnYIg5IrARb2kW7si+RvfPpt0C6su6VLDvYMWYkZplKc2xOynUGM?= =?us-ascii?Q?uxoLhIwFF/YMj0KJ3neNvjFm3CKdRK86B6yZ3Pocu6R0cVHvRGgxKpckBjq1?= =?us-ascii?Q?Br8l22+1eaTgScEY3IxJ/v66PT2uacSKiSsqRDATern+JuT3g4QTvTgn4uE5?= =?us-ascii?Q?3WjN8cWfoT0xV3Jl7BUmEMo+ATyj5SpSsmJezEG7ea3vIHx3dmLgyTp2ulu2?= =?us-ascii?Q?qaQnyChkiSZZ36J6zhDkqySeXc6d9yXy28ECLcdDyR/TIHKL81Js982WLVjq?= =?us-ascii?Q?GaZ0Jb5UcDakPv0nXbmWbxaXKPZ3WU11sBCl9giKF/9VU2+oek6nudeA2WmT?= =?us-ascii?Q?HeGiX0l+Veu1W7FYo5MZh8C9i1tOpGz11C82DRZX3YmuufiR/i/c+tMEK1M3?= =?us-ascii?Q?3vEYBLGH6fpPnJtgAt6M3/NX16U+WT1hiq6bX3HZKjTHSgkycoybXpT257bN?= =?us-ascii?Q?bTpXEwVzcJ9f/EP2dR+oOmZunS2lb4ZK4vmQCoIowtBJOY11ooMyOFSWOqAs?= =?us-ascii?Q?FU00ISZTyJl0rGDdFll3Wg5zZ0tEbw2fs+zgJe2s7LZIZfNJftR6G2wZEWfX?= =?us-ascii?Q?OVBrigA/OxISPG0OMnzpUcTITnJcZwINxZH5sJRptZczdP6ArCi/s7MuFqos?= =?us-ascii?Q?OyhzLNdyBU1IfX3i/hfcpwi4dpFKFLjUdF/Y6Zg8ROyjsCPPUeFMc6kdSz6o?= =?us-ascii?Q?m+D9bgBM5KWCpycx+SkrKSbBXk5MpUfnA5OgK5UUujV07QcGVqOz2zDNqEjr?= =?us-ascii?Q?mNt/ztvgZC1zBgdY5fopxCWNGvICP3VWhIoFiL9vIAwsWXgRURBqdvILZZZS?= =?us-ascii?Q?+STv4smO28MR+cEnbHIfwTmUS/M1+qlf2C68EJQjKfSxhI7BJQJ/SGgVjc7g?= =?us-ascii?Q?YOPgIMs7US2qQJ8c37MK/wE19VXSRed3e0mMfRqbCGCcJdrOwdSKeRCGO7iX?= =?us-ascii?Q?5QDyHrjColWjcwc/FtMO+uELr4SbmQbcdMC0q3DQAfdsvWmAi2sE1hR5Bbg6?= =?us-ascii?Q?wP14A9OppM5jdLiWuugNmI8rN/cP+Du+xTYCOAFM+tD4dzF5Sw=3D=3D?= X-Forefront-Antispam-Report: CIP:216.228.118.233;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc7edge2.nvidia.com;CAT:NONE;SFS:(13230040)(1800799024)(36860700013)(7416014)(376014)(82310400026);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Oct 2024 21:35:01.0313 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: f9262d02-6e63-4bb1-eceb-08dcf92ab5ac X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.118.233];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DS3PEPF000099E1.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB4282 Content-Type: text/plain; charset="utf-8" Prepare for an embedded structure design for driver-level iommufd_viommu objects: // include/linux/iommufd.h struct iommufd_viommu { struct iommufd_object obj; .... }; // Some IOMMU driver struct iommu_driver_viommu { struct iommufd_viommu core; .... }; It has to expose struct iommufd_object and enum iommufd_object_type from the core-level private header to the public iommufd header. Reviewed-by: Jason Gunthorpe Reviewed-by: Kevin Tian Signed-off-by: Nicolin Chen --- drivers/iommu/iommufd/iommufd_private.h | 25 +------------------------ include/linux/iommufd.h | 24 ++++++++++++++++++++++++ 2 files changed, 25 insertions(+), 24 deletions(-) diff --git a/drivers/iommu/iommufd/iommufd_private.h b/drivers/iommu/iommuf= d/iommufd_private.h index 8f3c21a664bd..94cfcab7e9de 100644 --- a/drivers/iommu/iommufd/iommufd_private.h +++ b/drivers/iommu/iommufd/iommufd_private.h @@ -5,8 +5,8 @@ #define __IOMMUFD_PRIVATE_H =20 #include +#include #include -#include #include #include #include @@ -126,29 +126,6 @@ static inline int iommufd_ucmd_respond(struct iommufd_= ucmd *ucmd, return 0; } =20 -enum iommufd_object_type { - IOMMUFD_OBJ_NONE, - IOMMUFD_OBJ_ANY =3D IOMMUFD_OBJ_NONE, - IOMMUFD_OBJ_DEVICE, - IOMMUFD_OBJ_HWPT_PAGING, - IOMMUFD_OBJ_HWPT_NESTED, - IOMMUFD_OBJ_IOAS, - IOMMUFD_OBJ_ACCESS, - IOMMUFD_OBJ_FAULT, -#ifdef CONFIG_IOMMUFD_TEST - IOMMUFD_OBJ_SELFTEST, -#endif - IOMMUFD_OBJ_MAX, -}; - -/* Base struct for all objects with a userspace ID handle. */ -struct iommufd_object { - refcount_t shortterm_users; - refcount_t users; - enum iommufd_object_type type; - unsigned int id; -}; - static inline bool iommufd_lock_obj(struct iommufd_object *obj) { if (!refcount_inc_not_zero(&obj->users)) diff --git a/include/linux/iommufd.h b/include/linux/iommufd.h index 30f832a60ccb..22948dd03d67 100644 --- a/include/linux/iommufd.h +++ b/include/linux/iommufd.h @@ -8,6 +8,7 @@ =20 #include #include +#include #include =20 struct device; @@ -18,6 +19,29 @@ struct iommufd_ctx; struct iommufd_device; struct page; =20 +enum iommufd_object_type { + IOMMUFD_OBJ_NONE, + IOMMUFD_OBJ_ANY =3D IOMMUFD_OBJ_NONE, + IOMMUFD_OBJ_DEVICE, + IOMMUFD_OBJ_HWPT_PAGING, + IOMMUFD_OBJ_HWPT_NESTED, + IOMMUFD_OBJ_IOAS, + IOMMUFD_OBJ_ACCESS, + IOMMUFD_OBJ_FAULT, +#ifdef CONFIG_IOMMUFD_TEST + IOMMUFD_OBJ_SELFTEST, +#endif + IOMMUFD_OBJ_MAX, +}; + +/* Base struct for all objects with a userspace ID handle. */ +struct iommufd_object { + refcount_t shortterm_users; + refcount_t users; + enum iommufd_object_type type; + unsigned int id; +}; + struct iommufd_device *iommufd_device_bind(struct iommufd_ctx *ictx, struct device *dev, u32 *id); void iommufd_device_unbind(struct iommufd_device *idev); --=20 2.43.0 From nobody Mon Nov 25 00:39:33 2024 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2080.outbound.protection.outlook.com [40.107.93.80]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C31C1228033; Wed, 30 Oct 2024 21:35:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.93.80 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730324106; cv=fail; b=b6lWyKnWhpB2FMUnIX/q29BD1rFbOSmJH8oxk5Ou8sFsg02Z38JrbWzsweoaXirgZCBxtkcO6HFjpX30g9kO3yLrGwubSdW16sDJAcQZ7wPAvADLSxkSQAyyoec5fzI4x7fZrNDc+aG3hrXqIqFEBq6UBGoeV0cf7HFUYIIgKac= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730324106; c=relaxed/simple; bh=GiBpWhwsK75jfmPmZ/LXdC9qTic4i+H4B+GSzktqjQE=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=lGRfxLAFNwUEJnDByXIDkp7Xi7CHtoPZcbgksIQt0p1uH3/XmGsr83yQSHjt47XkUBvNiEq4+AOzDAVPxbV9WfAm/YxjxREqAnoyShFX84SlEgU0H/H4aJGPrQ69GIIfkopnE/vqZzYjDeXRCjnVqnx8IuxoX5xudMYsupQOITE= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=rAtVzuyz; arc=fail smtp.client-ip=40.107.93.80 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="rAtVzuyz" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=LJv6bJUYqyfg2FpLQgt0X0I0dtljm4lqKfa66I5wZbWqa3BB8yoeOf9akYxXzmfoQfKSP11dlzAFmTHedysQDpUpxZ+N/qpWrmAo/nWYEZUYtEhN9NwOAKh1pq7QWt5S93vxTv6nEqtLYq0hGLNtvUnxVXH9AIPz0U6AKOH5pfsOTolzU3KnmDSd+d5N4LtU3PpCHKJV+M2xQDXnZIYlpjSgcTvKGZ+Kw55nwJrBE/03bELjAyTbGa/HwKotEFCqGI30l6gCe2AAzpDWTNzr/AhBr1NbxuW1pDmIQWr252xRMeslZ7M+OuI/bS/fjjkNkV0wjYZ4pbnedZIO9tsYJw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=QCLUkSfs/dIbW6V0NnS/PLqmx9D9uPhl5/eSQ7tnUu8=; b=g9TJ8uJI4ZAnScy3dMQR4wemZosLHTaXTTcmsUfDmq0LMk/pNYyh4IlkpszjjVQTEJIoJEVON6rjSkENMuAoXzRUpWQldrMfEoOUMFQUQbn6TeB64d4TtC59sd8530kCFAm3SHsr47QrRWpaSqYvf28FQBOzJLX4BMxDGLImu6C20LY4mfgRlDpuUCxenckZ0CxbHDX+S2F00GJ5p2jNT4C/JzVx2WKhTGXZ4EIwFD2fklqXxzpJeysWd2KfqzHWyxdMKA8dCdfiOcNpL6lKghKKj9CeaVDdls6Cw+Omet0L6a45oXIwf075RLpcm7b8tnEe4x0yCy81SEMElMrHlw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.118.232) smtp.rcpttodomain=amd.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=QCLUkSfs/dIbW6V0NnS/PLqmx9D9uPhl5/eSQ7tnUu8=; b=rAtVzuyzla8hs8/nKLzqY6gApNVmq5mIJeILo65MUiAbpUVr49jRa/cNKuSiR9/I6R0ZKiWHuzTFmKfZxbhjhjt7bDVYYQE7WB8J754Qc8XhQAkNWOXf0uqTnfPIONfqOycweIbBinD3WE9XUPArafV5ylE2i68m5+7Qi6Hb67ErQYrOlOTdWc65TVGPB9ZLTHYmdVT+mnEhvaszqJ+7nu/1rupPqWwg2zE/IwM8nLdGAp7QrVAX8r4883lY1UcKyzdIEMYFnGlpm5XYJ9vBpPzOCuW+p48amzoOKcYqZ5XEWPsYwMgX06Whb5kBlk57quM7hAw3cF/MQS4M++w5YA== Received: from SN7PR04CA0113.namprd04.prod.outlook.com (2603:10b6:806:122::28) by SJ2PR12MB8980.namprd12.prod.outlook.com (2603:10b6:a03:542::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8093.32; Wed, 30 Oct 2024 21:35:00 +0000 Received: from SN1PEPF0002BA4D.namprd03.prod.outlook.com (2603:10b6:806:122:cafe::85) by SN7PR04CA0113.outlook.office365.com (2603:10b6:806:122::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8093.29 via Frontend Transport; Wed, 30 Oct 2024 21:35:00 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.118.232) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.118.232 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.118.232; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.118.232) by SN1PEPF0002BA4D.mail.protection.outlook.com (10.167.242.70) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8114.16 via Frontend Transport; Wed, 30 Oct 2024 21:34:59 +0000 Received: from drhqmail203.nvidia.com (10.126.190.182) by mail.nvidia.com (10.127.129.5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Wed, 30 Oct 2024 14:34:59 -0700 Received: from drhqmail202.nvidia.com (10.126.190.181) by drhqmail203.nvidia.com (10.126.190.182) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Wed, 30 Oct 2024 14:34:58 -0700 Received: from Asurada-Nvidia.nvidia.com (10.127.8.12) by mail.nvidia.com (10.126.190.181) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Wed, 30 Oct 2024 14:34:57 -0700 From: Nicolin Chen To: , , CC: , , , , , , , , , , , , , , , , , , , , Subject: [PATCH v6 02/13] iommufd: Move _iommufd_object_alloc helper to a sharable file Date: Wed, 30 Oct 2024 14:34:28 -0700 Message-ID: <52abef167409b25bfee559a0c03f2607ce77be3c.1730313237.git.nicolinc@nvidia.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SN1PEPF0002BA4D:EE_|SJ2PR12MB8980:EE_ X-MS-Office365-Filtering-Correlation-Id: 8d9c392c-cd56-4bf8-36e9-08dcf92ab4ff X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|376014|36860700013|7416014|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?xi7vyM8ziiWSknomYqQK4aRZ9PQvII3NEjhu4TqSAbgzvIKee2m1Mz+417Dt?= =?us-ascii?Q?vmF0nRXpZHYW0REDAUEP56L0pQGX6p8bxRD29QtSYd9j3qXn8f/qdA8qoln6?= =?us-ascii?Q?o+BYNBcPhvscuy82UmFfW5aiUdLr9Awahh10V6cZT33oGWzvcYR+6Xv4bx4G?= =?us-ascii?Q?rSFyCY33trNxdtEdnPmT9GQQiafy8VeEyOq3FryaAN8ygh/zFiD/XdRNkIZU?= =?us-ascii?Q?EFkzUvyP5sZRl+5t41WH5HuYQvTQKt2f4IFdMYgrdTc4eDLIVANrWqbBc7mP?= =?us-ascii?Q?uUgQyTYhTXm/TImgSSo78h0QexauN0ZGXiP/SsEZXQpa4tBppXUcJe3dTVhu?= =?us-ascii?Q?4AavYC7XkpmW/qsCMWaFZmCZTNvxzOJLiDqtp1ajfQ9YDKzvqzCvFu5cCphy?= =?us-ascii?Q?Tr2tBVnEQ3U0o30j3pPX6JPKXRzKENOzhrFgDVUl884iCM5yvCAGVhJM157r?= =?us-ascii?Q?mcIHAK/JR3bE8LifwoZ1oazAfsT1xBC6pGJXcGUGTUYf+SFBRo2075uUMAvz?= =?us-ascii?Q?Kcd4o3dMJ4bxKirx3W1wLEdMuCkmm9PKh7nWFpJj7kdSh1cyO+Y0X+j80QA/?= =?us-ascii?Q?+qQLYb7MVtDgtA3YUFmvjjLR5csKsEKmqvnWcmSdu2GCFYDvnaoY9SaDPKYJ?= =?us-ascii?Q?qPQQa0yTJdr21+GV5lhr0K73fbD4kABPJATujZY/DKvwyHJe+k59gQwq0ODK?= =?us-ascii?Q?EeKuZC09sWXZBc0B8wLxaKyxp7o3Kx/rfPJiQonddairHTQJS6O31P7Ev9kE?= =?us-ascii?Q?akKUNdd5POJaS65h4XI1BXL4R+TlHGukoj7nWnyrtjDD2+gjWY6NJzk8/cGe?= =?us-ascii?Q?mjl9sOPRzAUFC5AhWS7O56absx8rm89aoAog3lARm0lbB8J4BN7tgjpZnrwX?= =?us-ascii?Q?g19+QckTizWHz7+62mWLuJv9OyZdLmNOTLWD8JSh8Yo6jsx+Mx8wFluXdeJe?= =?us-ascii?Q?iFVsznrIZouJhTBEi9Oc06ofv01a3qpXJs2axu2TXLzpsEBILxBX23zvUC4F?= =?us-ascii?Q?MjAXb+htLjtgq1/oXwBB5tfuzmxYOqwYy4IJeILwSsrO0zq6wha5v2FVloQ7?= =?us-ascii?Q?3+QhySDg/KmBFT7gYBKN4Yr1pPDej85Uy7AqLaCmDgNakIlt9+uWO9Z84FL0?= =?us-ascii?Q?MRZ3Rzn6s+JdTnjhgdLY3g+p88SAyAnMsWVyHbcd+Rb+uYeb6YEquDZ5b0fP?= =?us-ascii?Q?a4KDqDVLnurm7aI1AaNtQPDVVcEeMxT2k4hZZuEUtVUTE4QJhGTC/3sWRtqx?= =?us-ascii?Q?JtFsL4cihtdDTpF4CitmtUsJZhCm9r73cRVe+hOLKRhWiYH5YmUUb3N5jj+v?= =?us-ascii?Q?vR+ioxoP5Wg4ak9Fb2wsxlb2OSPc1jognqYigpnAN62qbTPfO/Vv5Rs1fnA1?= =?us-ascii?Q?j08u9igsePC5CrD5z72SIM9b9EjoXatjcZ0l0ae+buVwCYBDhA=3D=3D?= X-Forefront-Antispam-Report: CIP:216.228.118.232;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc7edge1.nvidia.com;CAT:NONE;SFS:(13230040)(82310400026)(376014)(36860700013)(7416014)(1800799024);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Oct 2024 21:34:59.9241 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 8d9c392c-cd56-4bf8-36e9-08dcf92ab4ff X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.118.232];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: SN1PEPF0002BA4D.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ2PR12MB8980 Content-Type: text/plain; charset="utf-8" The following patch will add a new vIOMMU allocator that will require this _iommufd_object_alloc to be sharable with IOMMU drivers (and iommufd too). Add a new driver.c file that will be built with CONFIG_IOMMUFD_DRIVER_CORE selected by CONFIG_IOMMUFD, and put the CONFIG_DRIVER under that remaining to be selectable for drivers to build the existing iova_bitmap.c file. Suggested-by: Jason Gunthorpe Signed-off-by: Nicolin Chen Reviewed-by: Jason Gunthorpe --- drivers/iommu/iommufd/Kconfig | 5 ++++ drivers/iommu/iommufd/Makefile | 5 +++- drivers/iommu/iommufd/iommufd_private.h | 4 --- include/linux/iommufd.h | 10 +++++++ drivers/iommu/iommufd/driver.c | 40 +++++++++++++++++++++++++ drivers/iommu/iommufd/main.c | 32 -------------------- 6 files changed, 59 insertions(+), 37 deletions(-) create mode 100644 drivers/iommu/iommufd/driver.c diff --git a/drivers/iommu/iommufd/Kconfig b/drivers/iommu/iommufd/Kconfig index 76656fe0470d..83addbe2ff7b 100644 --- a/drivers/iommu/iommufd/Kconfig +++ b/drivers/iommu/iommufd/Kconfig @@ -1,9 +1,14 @@ # SPDX-License-Identifier: GPL-2.0-only +config IOMMUFD_DRIVER_CORE + tristate + default IOMMUFD_DRIVER || IOMMUFD + config IOMMUFD tristate "IOMMU Userspace API" select INTERVAL_TREE select INTERVAL_TREE_SPAN_ITER select IOMMU_API + select IOMMUFD_DRIVER_CORE default n help Provides /dev/iommu, the user API to control the IOMMU subsystem as diff --git a/drivers/iommu/iommufd/Makefile b/drivers/iommu/iommufd/Makefile index cf4605962bea..266e23d657ff 100644 --- a/drivers/iommu/iommufd/Makefile +++ b/drivers/iommu/iommufd/Makefile @@ -12,4 +12,7 @@ iommufd-y :=3D \ iommufd-$(CONFIG_IOMMUFD_TEST) +=3D selftest.o =20 obj-$(CONFIG_IOMMUFD) +=3D iommufd.o -obj-$(CONFIG_IOMMUFD_DRIVER) +=3D iova_bitmap.o + +iommufd_driver-y :=3D driver.o +iommufd_driver-$(CONFIG_IOMMUFD_DRIVER) +=3D iova_bitmap.o +obj-$(CONFIG_IOMMUFD_DRIVER_CORE) +=3D iommufd_driver.o diff --git a/drivers/iommu/iommufd/iommufd_private.h b/drivers/iommu/iommuf= d/iommufd_private.h index 94cfcab7e9de..be347f726fda 100644 --- a/drivers/iommu/iommufd/iommufd_private.h +++ b/drivers/iommu/iommufd/iommufd_private.h @@ -206,10 +206,6 @@ iommufd_object_put_and_try_destroy(struct iommufd_ctx = *ictx, iommufd_object_remove(ictx, obj, obj->id, 0); } =20 -struct iommufd_object *_iommufd_object_alloc(struct iommufd_ctx *ictx, - size_t size, - enum iommufd_object_type type); - #define __iommufd_object_alloc(ictx, ptr, type, obj) = \ container_of(_iommufd_object_alloc( \ ictx, \ diff --git a/include/linux/iommufd.h b/include/linux/iommufd.h index 22948dd03d67..2f72452fad27 100644 --- a/include/linux/iommufd.h +++ b/include/linux/iommufd.h @@ -94,6 +94,9 @@ int iommufd_access_rw(struct iommufd_access *access, unsi= gned long iova, int iommufd_vfio_compat_ioas_get_id(struct iommufd_ctx *ictx, u32 *out_ioa= s_id); int iommufd_vfio_compat_ioas_create(struct iommufd_ctx *ictx); int iommufd_vfio_compat_set_no_iommu(struct iommufd_ctx *ictx); +struct iommufd_object *_iommufd_object_alloc(struct iommufd_ctx *ictx, + size_t size, + enum iommufd_object_type type); #else /* !CONFIG_IOMMUFD */ static inline struct iommufd_ctx *iommufd_ctx_from_file(struct file *file) { @@ -134,5 +137,12 @@ static inline int iommufd_vfio_compat_set_no_iommu(str= uct iommufd_ctx *ictx) { return -EOPNOTSUPP; } + +static inline struct iommufd_object * +_iommufd_object_alloc(struct iommufd_ctx *ictx, size_t size, + enum iommufd_object_type type) +{ + return ERR_PTR(-EOPNOTSUPP); +} #endif /* CONFIG_IOMMUFD */ #endif diff --git a/drivers/iommu/iommufd/driver.c b/drivers/iommu/iommufd/driver.c new file mode 100644 index 000000000000..2bc47d92a0ab --- /dev/null +++ b/drivers/iommu/iommufd/driver.c @@ -0,0 +1,40 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* Copyright (c) 2024, NVIDIA CORPORATION & AFFILIATES + */ +#include "iommufd_private.h" + +struct iommufd_object *_iommufd_object_alloc(struct iommufd_ctx *ictx, + size_t size, + enum iommufd_object_type type) +{ + struct iommufd_object *obj; + int rc; + + obj =3D kzalloc(size, GFP_KERNEL_ACCOUNT); + if (!obj) + return ERR_PTR(-ENOMEM); + obj->type =3D type; + /* Starts out bias'd by 1 until it is removed from the xarray */ + refcount_set(&obj->shortterm_users, 1); + refcount_set(&obj->users, 1); + + /* + * Reserve an ID in the xarray but do not publish the pointer yet since + * the caller hasn't initialized it yet. Once the pointer is published + * in the xarray and visible to other threads we can't reliably destroy + * it anymore, so the caller must complete all errorable operations + * before calling iommufd_object_finalize(). + */ + rc =3D xa_alloc(&ictx->objects, &obj->id, XA_ZERO_ENTRY, xa_limit_31b, + GFP_KERNEL_ACCOUNT); + if (rc) + goto out_free; + return obj; +out_free: + kfree(obj); + return ERR_PTR(rc); +} +EXPORT_SYMBOL_NS_GPL(_iommufd_object_alloc, IOMMUFD); + +MODULE_DESCRIPTION("iommufd code shared with builtin modules"); +MODULE_LICENSE("GPL"); diff --git a/drivers/iommu/iommufd/main.c b/drivers/iommu/iommufd/main.c index 826a2b2be52f..3c32b440471b 100644 --- a/drivers/iommu/iommufd/main.c +++ b/drivers/iommu/iommufd/main.c @@ -29,38 +29,6 @@ struct iommufd_object_ops { static const struct iommufd_object_ops iommufd_object_ops[]; static struct miscdevice vfio_misc_dev; =20 -struct iommufd_object *_iommufd_object_alloc(struct iommufd_ctx *ictx, - size_t size, - enum iommufd_object_type type) -{ - struct iommufd_object *obj; - int rc; - - obj =3D kzalloc(size, GFP_KERNEL_ACCOUNT); - if (!obj) - return ERR_PTR(-ENOMEM); - obj->type =3D type; - /* Starts out bias'd by 1 until it is removed from the xarray */ - refcount_set(&obj->shortterm_users, 1); - refcount_set(&obj->users, 1); - - /* - * Reserve an ID in the xarray but do not publish the pointer yet since - * the caller hasn't initialized it yet. Once the pointer is published - * in the xarray and visible to other threads we can't reliably destroy - * it anymore, so the caller must complete all errorable operations - * before calling iommufd_object_finalize(). - */ - rc =3D xa_alloc(&ictx->objects, &obj->id, XA_ZERO_ENTRY, - xa_limit_31b, GFP_KERNEL_ACCOUNT); - if (rc) - goto out_free; - return obj; -out_free: - kfree(obj); - return ERR_PTR(rc); -} - /* * Allow concurrent access to the object. * --=20 2.43.0 From nobody Mon Nov 25 00:39:33 2024 Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on2086.outbound.protection.outlook.com [40.107.223.86]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CB66322803E; Wed, 30 Oct 2024 21:35:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.223.86 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730324108; cv=fail; b=GBoPyMv1kp1Gjf7sOJ3TZEGWoRUSj9BF9+wPzI3p0yRmTfXLDzYOP4u0G/ZX2GQdP88do3n3DTFy3wGgsdPe0OTmj/xQ+aF0ahfje7w65UUVZKjqlOx8PMo8VtL1ZWvkQ60n/y6GEKJNbrbq49vlg3UkXpl/VsAnOGDJQrmHoyw= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730324108; c=relaxed/simple; bh=GnlEqUbtCOBugdwWenf0URGYG1gs8cNFbdOk87yROeQ=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=hvhuqngyJO74RMOkVyjgaO4bwSUuzIb3SI8HiDKKZ45+7bZKssrm5KwMr45kX0UBkgFJxTBnWvL3TO6tpIHGodzNCdAZ0t+odFoxW3xdNBmvPSQdkqtAVF7qOBnGr1lUvCau9Xa5Qz3FgWUh7eAbUtDOExPHD9GKISmHNn1WIhc= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=hbsFUHL0; arc=fail smtp.client-ip=40.107.223.86 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="hbsFUHL0" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=GLUelmJn0yxo6yacZvOiZd3+gd4TwwVqXaBXgIlzr15k6O2kF+SPm3BfoKZpVDb/fUV4EQH2Hlb2Eb1UVQXfI/E4wZLw6VTLSTpldR69tegFwbmcQeIgZDs3U6lwZKF+eHhYiHyqVCSzRsT6onnFXuOlo7gpfFkgsoJkEsnsyW2MHsWMfLrVH9kHd8N0c+woa21Xv/rNgL+TY1jDeHLMx++Q6B249lpwSgG+i8Nr4Fmtk7f2g1skhcFoY4J96nfJfsIuUlzMMjW7D2MOtV1cBK60H+1NajspieUkp71glk84dk5DE/zOII6QyeeNWFiTHKfr/mYa7ndNL0gNpC5Lbg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=3hkztPKrOqqa7rC7sObEukfJLKe85KtYYTJxH2Ol71A=; b=IaohbSuSEAtjAQ9f8cWfDYZfff73uQ05gjF2cIch/+In/xDN09gziNmiRmkTyU6VlMl1vxuscdoaLWcvSYU0n+M8hHtL8T+dg/SOHlpAqEbFUMShBpiz/W+pRtArU4AvU1MyX5t3NHu8XCQZ+RHPbAvAlgKLJMALhSIhuGLvyQw414nnA7a9VcMhiY0KCS8/qeZ0YsuyDvLMllCtqtqjiYaCYI704tZIWQAvAMdl83moKI3WSL2og8vGT8mES1E98UvZvaR0wx8kbg4G2jtgiJXiilHtDMvw78RTz/tOyWMJkUZSDF95lZUtL4+6winyyRd9mZdOIfOc7jQLFTkFLQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.118.232) smtp.rcpttodomain=amd.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=3hkztPKrOqqa7rC7sObEukfJLKe85KtYYTJxH2Ol71A=; b=hbsFUHL0CmnlAt/wH6grvu0V3lzxNS7xbyZXVZ3C+qVuEVZS9LBXgcnF7qiHPYI8qhbYnGrJGK2lxUZfVT66IcstSJelv1+o/H/IlvHvXDrVU3IcdiU1cW6wyOc0kxa8H4Pu0REbUvhxydJIEFeUiSghOjP3xq14ZWf5F2MHRGGxzwkLdSCW5BtrmhaTILVY5N5Lk8k6dl8GahWtYgbEhRjDYD1C29XLKM5IO5PKVAqfHQvbiNPhGJ/VZmHOeVnEfmgUQvfTBC8nNNNfU89PY5r5PeEe+ZBnurLtLrAvnOF9YOAnr6vOw5wIO7ncrwkx1FDEiU2vA7O4PaH3+cGH3A== Received: from SN7PR04CA0118.namprd04.prod.outlook.com (2603:10b6:806:122::33) by MN0PR12MB6343.namprd12.prod.outlook.com (2603:10b6:208:3c0::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8093.32; Wed, 30 Oct 2024 21:35:01 +0000 Received: from SN1PEPF0002BA4D.namprd03.prod.outlook.com (2603:10b6:806:122:cafe::6e) by SN7PR04CA0118.outlook.office365.com (2603:10b6:806:122::33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8093.29 via Frontend Transport; Wed, 30 Oct 2024 21:35:01 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.118.232) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.118.232 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.118.232; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.118.232) by SN1PEPF0002BA4D.mail.protection.outlook.com (10.167.242.70) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8114.16 via Frontend Transport; Wed, 30 Oct 2024 21:35:01 +0000 Received: from drhqmail203.nvidia.com (10.126.190.182) by mail.nvidia.com (10.127.129.5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Wed, 30 Oct 2024 14:35:00 -0700 Received: from drhqmail202.nvidia.com (10.126.190.181) by drhqmail203.nvidia.com (10.126.190.182) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Wed, 30 Oct 2024 14:35:00 -0700 Received: from Asurada-Nvidia.nvidia.com (10.127.8.12) by mail.nvidia.com (10.126.190.181) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Wed, 30 Oct 2024 14:34:59 -0700 From: Nicolin Chen To: , , CC: , , , , , , , , , , , , , , , , , , , , Subject: [PATCH v6 03/13] iommufd: Introduce IOMMUFD_OBJ_VIOMMU and its related struct Date: Wed, 30 Oct 2024 14:34:29 -0700 Message-ID: <83a5bc32b62e42def9395f48da961263e18adf77.1730313237.git.nicolinc@nvidia.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SN1PEPF0002BA4D:EE_|MN0PR12MB6343:EE_ X-MS-Office365-Filtering-Correlation-Id: 6cf4e93f-c410-49d7-06c5-08dcf92ab5d1 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|376014|7416014|36860700013|82310400026; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?OeBsya4xnuVmMbIZKuTGm4wSwREQylGu0z9xWMl28nNvP1gFOKiRoJOuUS+9?= =?us-ascii?Q?COmwwIjypyT7Kf070t5sHqaes752Z21vPIIIrDn0boyc4uJP5rs3/zpEUmHQ?= =?us-ascii?Q?kOFl7TiOhE18+tQl6/7oQdukK3wGUxbg0VVL75D1LIxi0frtxeWCwltIitbg?= =?us-ascii?Q?bgHDpV1soYin4QKUJHPGiqXONSxKentGXfze7ilGuRQjfidEhsul6JkJAo8U?= =?us-ascii?Q?2E6jwJfuLkZpIDXbkBTOTDjzoxlX+ID3QV3ia4pEb2RPBNOhhHGfQpgdam4u?= =?us-ascii?Q?Ar2K8KEmSXZvAFzIGOMyQKDM/w3prg1YgRKVV0NAZmlhMAUbJRKXLJ5LzNIs?= =?us-ascii?Q?JVCxp3FPjJP8V77ncdWaF2vLh4ecYK7klcJwMaUJD6WUwDEFrXb6hWhuwsy8?= =?us-ascii?Q?vpV3vkRK6WKRwCQFz1FXfTwCz5Ibe4sKBCEyxA6ZAlWC8K4j0o4YM09wPCS4?= =?us-ascii?Q?wkdEVX6i5PnVO7Eo22+v5hpSHPC5urVzsS6FU61ZuRyvie2drgW7qjYrGiMp?= =?us-ascii?Q?9hqU/XpD4cZO5A2i88AaRfOobzsuxBbZ4sxBg/bUrWbgjupOlchqPng17631?= =?us-ascii?Q?thbkGjOjAMVEp2fdcDgqb7GSPo7Dh9x5mjrzM/NYaEw9ghtrc0AxLpueN1HL?= =?us-ascii?Q?zkS1kXSzuHWX+CP0eubtcWqnGjZu6Ed3TnlCkGIPw5MnkQMaoHOYlJw6tDG4?= =?us-ascii?Q?8hWJz8M+VoD4bcRWc8NTJ2YEtj4weqbtWrVD3m+RjdMejRelh68G0xsvehD6?= =?us-ascii?Q?ATrREkz27oE7CDqr4uylf+Vnu3kmL2MyxAy5iCjEPjfuxjxDGFQO2REsPs1m?= =?us-ascii?Q?vHo4HSIWQmSJH4I/F7WrqqcfHiKUfwNZLOUo1ISy+gVp56ouiMydhWa5qzHq?= =?us-ascii?Q?2N6QIebQUW/V38hHiscYbWEFQqmB6fG7BTRNnH6i/1j4CavRGOJx8Uydw8ou?= =?us-ascii?Q?3FgJq7oCaWOda9YKHCdEoHFHAyApSd8w0NojCj4xyyu8qUPOQJcV+bce80xe?= =?us-ascii?Q?/RPoFgzG5P3Y+5Tptvf8Rx6Z/h8RbcpewmJsjFbQ4qxU2P54yHR48NoJOoJZ?= =?us-ascii?Q?LVWd/Q3t3RJAE1ntL75QMoWKsmcgs/E/JLoEc7t16YOw1Do2KXoUVPUJzb+Z?= =?us-ascii?Q?uiRF20L+INEVRGQDGwYE5UPbd9LLAK+k3otGRYclTRdyp+WDcUszgjUHd6gD?= =?us-ascii?Q?LN3Oe5L+ydHe10NfLjW1u47nM942OkAkQRLPBqr7K4Cj9rXSSHl1IT+Ev0Hv?= =?us-ascii?Q?mPvoRo46wFjVnzXOb8YAIK9oCSOiRKxh1Rmksxl/HzQTQaZ07PE6ORsWX7P9?= =?us-ascii?Q?lGF9+Ec9rcmYPTa29dTQf3VWXvi01SIaAyNfM9c/Afjvnp2OixGnZF86b9wK?= =?us-ascii?Q?82YN20QD1LU1giGMvgT5oCxAqBs0iAwK8ALDtMbI2n5yXIBfIA=3D=3D?= X-Forefront-Antispam-Report: CIP:216.228.118.232;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc7edge1.nvidia.com;CAT:NONE;SFS:(13230040)(1800799024)(376014)(7416014)(36860700013)(82310400026);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Oct 2024 21:35:01.2835 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 6cf4e93f-c410-49d7-06c5-08dcf92ab5d1 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.118.232];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: SN1PEPF0002BA4D.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN0PR12MB6343 Content-Type: text/plain; charset="utf-8" Add a new IOMMUFD_OBJ_VIOMMU with an iommufd_viommu structure to represent a slice of physical IOMMU device passed to or shared with a user space VM. This slice, now a vIOMMU object, is a group of virtualization resources of a physical IOMMU's, such as: - Security namespace for guest owned ID, e.g. guest-controlled cache tags - Non-device-affiliated event reporting, e.g. invalidation queue errors - Access to a sharable nesting parent pagetable across physical IOMMUs - Virtualization of various platforms IDs, e.g. RIDs and others - Delivery of paravirtualized invalidation - Direct assigned invalidation queues - Direct assigned interrupts Add a new viommu_alloc op in iommu_ops, for drivers to allocate their own vIOMMU structures. And this allocation also needs a free(), so add struct iommufd_viommu_ops. To simplify a vIOMMU allocation, provide a iommufd_viommu_alloc() helper. It's suggested that a driver should embed a core-level viommu structure in its driver-level viommu struct and call the iommufd_viommu_alloc() helper, meanwhile the driver can also implement a viommu ops: struct my_driver_viommu { struct iommufd_viommu core; /* driver-owned properties/features */ .... }; static const struct iommufd_viommu_ops my_driver_viommu_ops =3D { .free =3D my_driver_viommu_free, /* future ops for virtualization features */ .... }; static struct iommufd_viommu my_driver_viommu_alloc(...) { struct my_driver_viommu *my_viommu =3D iommufd_viommu_alloc(ictx, my_driver_viommu, core, my_driver_viommu_ops); /* Init my_viommu and related HW feature */ .... return &my_viommu->core; } static struct iommu_domain_ops my_driver_domain_ops =3D { .... .viommu_alloc =3D my_driver_viommu_alloc, }; Suggested-by: Jason Gunthorpe Reviewed-by: Kevin Tian Reviewed-by: Jason Gunthorpe Signed-off-by: Nicolin Chen --- include/linux/iommu.h | 14 ++++++++++++++ include/linux/iommufd.h | 40 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 54 insertions(+) diff --git a/include/linux/iommu.h b/include/linux/iommu.h index bd722f473635..2574fc8abaf2 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -42,6 +42,8 @@ struct notifier_block; struct iommu_sva; struct iommu_dma_cookie; struct iommu_fault_param; +struct iommufd_ctx; +struct iommufd_viommu; =20 #define IOMMU_FAULT_PERM_READ (1 << 0) /* read */ #define IOMMU_FAULT_PERM_WRITE (1 << 1) /* write */ @@ -542,6 +544,14 @@ static inline int __iommu_copy_struct_from_user_array( * @remove_dev_pasid: Remove any translation configurations of a specific * pasid, so that any DMA transactions with this pasid * will be blocked by the hardware. + * @viommu_alloc: Allocate an iommufd_viommu on a physical IOMMU instance = behind + * the @dev, as the set of virtualization resources shared/= passed + * to user space IOMMU instance. And associate it with a ne= sting + * @parent_domain. The @viommu_type must be defined in the = header + * include/uapi/linux/iommufd.h + * It is required to call iommufd_viommu_alloc() helper for + * a bundled allocation of the core and the driver structur= es, + * using the given @ictx pointer. * @pgsize_bitmap: bitmap of all possible supported page sizes * @owner: Driver module providing these ops * @identity_domain: An always available, always attachable identity @@ -591,6 +601,10 @@ struct iommu_ops { void (*remove_dev_pasid)(struct device *dev, ioasid_t pasid, struct iommu_domain *domain); =20 + struct iommufd_viommu *(*viommu_alloc)( + struct device *dev, struct iommu_domain *parent_domain, + struct iommufd_ctx *ictx, unsigned int viommu_type); + const struct iommu_domain_ops *default_domain_ops; unsigned long pgsize_bitmap; struct module *owner; diff --git a/include/linux/iommufd.h b/include/linux/iommufd.h index 2f72452fad27..457e763e8b63 100644 --- a/include/linux/iommufd.h +++ b/include/linux/iommufd.h @@ -17,6 +17,7 @@ struct iommu_group; struct iommufd_access; struct iommufd_ctx; struct iommufd_device; +struct iommufd_viommu_ops; struct page; =20 enum iommufd_object_type { @@ -28,6 +29,7 @@ enum iommufd_object_type { IOMMUFD_OBJ_IOAS, IOMMUFD_OBJ_ACCESS, IOMMUFD_OBJ_FAULT, + IOMMUFD_OBJ_VIOMMU, #ifdef CONFIG_IOMMUFD_TEST IOMMUFD_OBJ_SELFTEST, #endif @@ -78,6 +80,26 @@ void iommufd_access_detach(struct iommufd_access *access= ); =20 void iommufd_ctx_get(struct iommufd_ctx *ictx); =20 +struct iommufd_viommu { + struct iommufd_object obj; + struct iommufd_ctx *ictx; + struct iommu_device *iommu_dev; + struct iommufd_hwpt_paging *hwpt; + + const struct iommufd_viommu_ops *ops; + + unsigned int type; +}; + +/** + * struct iommufd_viommu_ops - vIOMMU specific operations + * @destroy: Clean up all driver-specific parts of an iommufd_viommu. The = memory + * of the vIOMMU will be free-ed by iommufd core after calling t= his op + */ +struct iommufd_viommu_ops { + void (*destroy)(struct iommufd_viommu *viommu); +}; + #if IS_ENABLED(CONFIG_IOMMUFD) struct iommufd_ctx *iommufd_ctx_from_file(struct file *file); struct iommufd_ctx *iommufd_ctx_from_fd(int fd); @@ -145,4 +167,22 @@ _iommufd_object_alloc(struct iommufd_ctx *ictx, size_t= size, return ERR_PTR(-EOPNOTSUPP); } #endif /* CONFIG_IOMMUFD */ + +/* + * Helpers for IOMMU driver to allocate driver structures that will be fre= ed by + * the iommufd core. The free op will be called prior to freeing the memor= y. + */ +#define iommufd_viommu_alloc(ictx, drv_struct, member, viommu_ops) = \ + ({ \ + drv_struct *ret; \ + \ + static_assert(__same_type(struct iommufd_viommu, \ + ((drv_struct *)NULL)->member)); \ + static_assert(offsetof(drv_struct, member.obj) =3D=3D 0); \ + ret =3D (drv_struct *)_iommufd_object_alloc( \ + ictx, sizeof(drv_struct), IOMMUFD_OBJ_VIOMMU); \ + if (!IS_ERR(ret)) \ + ret->member.ops =3D viommu_ops; \ + ret; \ + }) #endif --=20 2.43.0 From nobody Mon Nov 25 00:39:33 2024 Received: from NAM04-BN8-obe.outbound.protection.outlook.com (mail-bn8nam04on2062.outbound.protection.outlook.com [40.107.100.62]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7AC9B2281DA; Wed, 30 Oct 2024 21:35:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.100.62 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730324111; cv=fail; b=lK8NrFoM7sr605iI9CRPzU1tTvf4MjUR0VUcA/ESI9cVkuzOzcVusCTav22mtG2dc0/VOJoMIjPZp4AMYU8XY7HrpEnCRpQgKWco2p+OscJYxTvtD3YtLVEHatbGPvYOSOyEASfhM3ntXu5hJbHApsXgVL6KHj7ELl814KQRv5Q= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730324111; c=relaxed/simple; bh=pHf/gD37lHkraB3+JNnYPKaV+hzpKQjuT+yMlPfyooc=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=FyHSfgJQzxxARPvlLVZc4jkdbCVF2iFDjNnQDt9Pr5A6Clpv2w1/L0g/+IFhU9MKRLEsgedUW2OjDMrHwiiHh9EcSF6v839CP0CqMxxAOIvhNYcyCHIaSOO0ZHwFHUkLcpgrwtr8wgnQxlm7CwGoFPrYCM7rLFOEf6SJmzhpErk= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=ep3AOnd4; arc=fail smtp.client-ip=40.107.100.62 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="ep3AOnd4" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=SbwJuCdN4j7YLhDvbIAZs7SxhMPM6JNkYu+mffr5eVicKwpYYh74+Vns1QeIsnbAGzUSXDvc6AIMvtvZao/RHXz2VgNgXQMihEm1hdMD/WMhbNm4mIgFG3VSUAcdCzCAprOelozGr8yYERdVa2c0ZsJAeZBZ59zGdgGzVbW84xQycm/DW2FNt35nKAfVKq5vH/eoFOIIJFsd6LtXPWhp42ZXkdYUFyku+nRKNpf49QlFelEzHLqxUsiEzkvGWhx6P2C6sjPTIeUlVbgS25tY6nICPcqLxSvaxR2NYWrZvjCgfsexaqJHPX7ff3XV5FS1zJAI3QgTJqbsuCMDNjPFow== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=eiANv90Uqb5h38O3Jt2rHfHoJRszoA8yzSqB2qGaZME=; b=IwaKwW3EagFGIWUZ2VDSQWWoo6QKFCnBc9Ep4AaECeETICpXj8HlLv+4CKfSSTg0E6hcRfgZZ28ikpr2jrYfbUgpi+EFFAqDizFuqeN4zAJO95ZiLnCViG+hn3Umq8qiZOQ6Y+fAY8I8J2YQ3X2H/lDIvO4zan81Ks8ibVHtZLFn94xDCi+DT4hNKKLoCLqrBz7c7Nc0m8q3fbIWsNiKUQRnQpY/CfkFiv8/ZNiVZ5DAzYggvyCYVCHNkfR2BSonA+73RQT/PM5kTpRjPzDOUpVaVGLX2bRChBSccchOS0I3Y8Yf9bIQItL4yB0WuHeQXEcdFWHwvevRAQWDekVYKA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.118.232) smtp.rcpttodomain=amd.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=eiANv90Uqb5h38O3Jt2rHfHoJRszoA8yzSqB2qGaZME=; b=ep3AOnd4eoaUAVHldMQu2aHf+6SGD6PukJ1MRqOEVlrdRK3Y+wDO285WErmuJfQOmH3ijrhytIJ+/yiNGnsi96L+0HahI5LuX9jqjLKWxmGxhUyb1Dcdnnh6Jp4dxzG5Xj50IxKvEy2m0rUPdVlhE4zt5iQwKqASuiNYfhGk06ZMLLikCON9dv4G/yHvUL33vru/GAeXGvvmTjEYDyC8cZIAxoaPzb8R045bgfqKZ6Goleiv+K/l/Fzj2myH7TwTuUyOmgUSmYYbVRkeNGUV7Wn0V+j8q/7gZF/ntmS5f4E5frnLlfDOTkTeHqLfMEkzdVJZ1m+YIOxrBAc7rUIF2g== Received: from SN7PR04CA0101.namprd04.prod.outlook.com (2603:10b6:806:122::16) by DS0PR12MB7827.namprd12.prod.outlook.com (2603:10b6:8:146::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8093.25; Wed, 30 Oct 2024 21:35:03 +0000 Received: from SN1PEPF0002BA4D.namprd03.prod.outlook.com (2603:10b6:806:122:cafe::b) by SN7PR04CA0101.outlook.office365.com (2603:10b6:806:122::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8093.31 via Frontend Transport; Wed, 30 Oct 2024 21:35:02 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.118.232) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.118.232 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.118.232; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.118.232) by SN1PEPF0002BA4D.mail.protection.outlook.com (10.167.242.70) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8114.16 via Frontend Transport; Wed, 30 Oct 2024 21:35:02 +0000 Received: from drhqmail203.nvidia.com (10.126.190.182) by mail.nvidia.com (10.127.129.5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Wed, 30 Oct 2024 14:35:02 -0700 Received: from drhqmail202.nvidia.com (10.126.190.181) by drhqmail203.nvidia.com (10.126.190.182) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Wed, 30 Oct 2024 14:35:01 -0700 Received: from Asurada-Nvidia.nvidia.com (10.127.8.12) by mail.nvidia.com (10.126.190.181) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Wed, 30 Oct 2024 14:35:00 -0700 From: Nicolin Chen To: , , CC: , , , , , , , , , , , , , , , , , , , , Subject: [PATCH v6 04/13] iommufd: Verify object in iommufd_object_finalize/abort() Date: Wed, 30 Oct 2024 14:34:30 -0700 Message-ID: <6c39c8ed60ea015fe2a5ce36b54132507d570ea2.1730313237.git.nicolinc@nvidia.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SN1PEPF0002BA4D:EE_|DS0PR12MB7827:EE_ X-MS-Office365-Filtering-Correlation-Id: 879869e9-2200-4159-e9bc-08dcf92ab6b5 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|376014|7416014|36860700013|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?4NTErLIgs8Jra/3Y6xZVdZM/CNNcEOKHiels8DPurg0t+leHPtVX1PvHiiHT?= =?us-ascii?Q?l89OlH76O1soXYDHLn738rbus1VhaDpi7WoOd3RJxArtiq7rRdBa1zd4h/lC?= =?us-ascii?Q?JiRVh5gADY4v022PBLRFv90G0SOYD+hD+nBYNFg3r11JkLDzRVKInnVGvOIa?= =?us-ascii?Q?7No6O1mtfbBKTgA1bHfg3ik6MiUr8D8UdA04D+RiNp+CFLesL22SPKpoNcqx?= =?us-ascii?Q?NxR3mmrkn7UIDYvQEjkLn7Pje71BDa4EEZQWjK4lpTngdFFH68dDgEL/uclS?= =?us-ascii?Q?RrMy5FJB6nisjMfj5Z6g+RhT4loptb6gMxjdpTeG6eru/3OF+qyxErzsq6Wh?= =?us-ascii?Q?SHVyzskh/oDLfbIlGtnmxNT4RCEU8r3OieaweaiQtKxZDpYs/qiqsmxaWhP6?= =?us-ascii?Q?2ZzSh2qOHkI5yIy+0woeusdZGMpcjt14vtEmiayuWFqqSq2HFSKbAOnRxf1M?= =?us-ascii?Q?ZH8Jm4skVaUttp215XcymenNFeJjrYOZQMoW90TsNwcD9ahnZ/4/VSjygC3X?= =?us-ascii?Q?+qAk011e8fzgfklh1M3mgcmMgwfMNbctRXSY7EGxl5z/B+methZphTJmbZjT?= =?us-ascii?Q?zeukH68EcGDiOsYEIN/LR1iLaSww+YVvD10L/+fXqYtkNmvQXKQCz/TUSL1M?= =?us-ascii?Q?8jxMhx7YQfT6twBmu7M4SM66+PSK3tLQys3Q8Q+rK/VrR+td99O9kM9w0CSW?= =?us-ascii?Q?IcTb1Bi/dUi6Y5ImFP4uHuCks6zutfm9Wv4J6RMwJ1cxBpj6KKF2sRShJb4c?= =?us-ascii?Q?6gwiHSG9P8sbTvH5WZcQATEX1uUB8KOT2ZVE8gu/+tf+VZTTSEf4NS2RQwtc?= =?us-ascii?Q?yoZiPZev/l321VOuK1wZg0sJM0J4ZH6hA4mPSnbmt7xfiK5Bbo1NcJOyhP8i?= =?us-ascii?Q?NYaHxojeOqgafAFXA2AfEY8C2UeMrqpYYgWP88i4mcJVIwdi2RHo7EZyEsRE?= =?us-ascii?Q?96uXAmzCgWdonsSH5t5ijk2tpk7t1mpIPk6BQgn0HDeDT6otyWkYzda1mKfZ?= =?us-ascii?Q?DhBDav3SZu/VUDNZOthVKdkazF2qVnWXCUBph7G05CmmElo4zGxM1dDC20aY?= =?us-ascii?Q?BlszT7ferO+8pF6ThP209OAjGdjtODpeefHn/xM0Inv21wG0UGvVh0n18icQ?= =?us-ascii?Q?RiQGfIT4CFR7UXIwhReVCxLUKibW+9fKBCElH8P5nv1fuPtH2aUmOj/1XCT1?= =?us-ascii?Q?RyYz9pTb5yuAP9L0S+wiGyG8DtZ/ld1oHiJYuANgnCNL18QvFHKjRUS1YJbT?= =?us-ascii?Q?qp1lhNlXoMlqCEoI0FNjuBBCi0rZLca0cO+F1y/rkuVdU9c2pVyRUs2lo63V?= =?us-ascii?Q?DXeTfND9WCykjle2wKHjE802ykWKBXcv769XJ+Dbsxr+57kENR/DWGpvDnMT?= =?us-ascii?Q?w9+nRb1MM3k9mYTTWFEHjOGaNduofkf/MzIyVfMubXvfXxtHxw=3D=3D?= X-Forefront-Antispam-Report: CIP:216.228.118.232;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc7edge1.nvidia.com;CAT:NONE;SFS:(13230040)(82310400026)(376014)(7416014)(36860700013)(1800799024);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Oct 2024 21:35:02.7679 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 879869e9-2200-4159-e9bc-08dcf92ab6b5 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.118.232];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: SN1PEPF0002BA4D.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR12MB7827 Content-Type: text/plain; charset="utf-8" To support driver-allocated vIOMMU objects, it's required for IOMMU driver to call the provided iommufd_viommu_alloc helper to embed the core struct. However, there is no guarantee that every driver will call it and allocate objects properly. Make the iommufd_object_finalize/abort functions more robust to verify if the xarray slot indexed by the input obj->id is having an XA_ZERO_ENTRY, which is the reserved value stored by xa_alloc via iommufd_object_alloc. Suggested-by: Jason Gunthorpe Signed-off-by: Nicolin Chen Reviewed-by: Jason Gunthorpe --- drivers/iommu/iommufd/main.c | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/drivers/iommu/iommufd/main.c b/drivers/iommu/iommufd/main.c index 3c32b440471b..30e6c2af3b45 100644 --- a/drivers/iommu/iommufd/main.c +++ b/drivers/iommu/iommufd/main.c @@ -41,20 +41,26 @@ static struct miscdevice vfio_misc_dev; void iommufd_object_finalize(struct iommufd_ctx *ictx, struct iommufd_object *obj) { + XA_STATE(xas, &ictx->objects, obj->id); void *old; =20 - old =3D xa_store(&ictx->objects, obj->id, obj, GFP_KERNEL); - /* obj->id was returned from xa_alloc() so the xa_store() cannot fail */ - WARN_ON(old); + xa_lock(&ictx->objects); + old =3D xas_store(&xas, obj); + xa_unlock(&ictx->objects); + /* obj->id was returned from xa_alloc() so the xas_store() cannot fail */ + WARN_ON(old !=3D XA_ZERO_ENTRY); } =20 /* Undo _iommufd_object_alloc() if iommufd_object_finalize() was not calle= d */ void iommufd_object_abort(struct iommufd_ctx *ictx, struct iommufd_object = *obj) { + XA_STATE(xas, &ictx->objects, obj->id); void *old; =20 - old =3D xa_erase(&ictx->objects, obj->id); - WARN_ON(old); + xa_lock(&ictx->objects); + old =3D xas_store(&xas, NULL); + xa_unlock(&ictx->objects); + WARN_ON(old !=3D XA_ZERO_ENTRY); kfree(obj); } =20 --=20 2.43.0 From nobody Mon Nov 25 00:39:33 2024 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2073.outbound.protection.outlook.com [40.107.93.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 42796218582; Wed, 30 Oct 2024 21:35:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.93.73 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730324120; cv=fail; b=oMCDwi/LZhOxkN0Bc0f3ZOsifDy1naAqpvb4a4Aw1Ws2g2/kOtkqXuxMvs9TGb4jDF5i3ZKXu22ATWezGAMd9+7g7TCmo3w+L1uWNNxEMkbuyD8KCuHeAs4HGQaurakdVQSPcc+oY3WN5ilKSQk6b7EMiTZoM8sXiJn83GCShAE= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730324120; c=relaxed/simple; bh=sBzU28DAjRKWzySh4SxnP3LzEEHZb6pTH9bJWOP8Ha0=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=rNk8yuGNMaAU3HSdDZvY9MIr1v6L7FgigOzeiuIPebsZOauA9mqebnuKdCPypxesNlB8dt6LAcljhAKFDmLUMqCs2TH1XvOyB4G3C0n9pnznckCnPTY5Z5YufLpiKeufUwaMnvEg/QrDNE2B0CeGotEHojTsHvXazIDFuOjhaFI= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=LKm5PMiR; arc=fail smtp.client-ip=40.107.93.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="LKm5PMiR" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=xzheKyanZ6+GtCyOOU6IN/lqWMqdCiKA2p7IgLg3a1fkTswZB57wH6wuTaPrOKPhR/BMBrui+ZS5zKJApoAVroiOuKLeMh1XCpd2c6MQR8fL9RfoLxo04/zfBMWwmIq6F0BETqCreMR0R/9cDcrnFOFbQHEfNgUFszA1HMGtjQ6AjIYqqje525igagm7Ui74tIJfy2j78tcH27w44nLFvfy2GyfnzHkdKHYeJMrnVJgi+Sl1QdcwqxzunIwrcMvTZvnlnK8SIG8E5dc9085Q7z8zI24zaF1lzxwqoIvtLvDxe7nOccQmjm6JkWilfXaZe/EEC/nsJJktDGNObF0HFA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=15IebqioQLVniPg33q96CYi05OjcWulaE5tJTzOAFek=; b=YkCHEs98ixeVmiOIHEoRZf8+vyw9cviGZ8MAenOdjFXkF1sfJ6Iqmrs/MVR+5uKHqtQd327R1Rrg1eUXjH4jFuBqMLS1QtKtcelhz5YjKKKEZWY1L/U487ZAbFHnJn+nmTEFL+C0AHoxjjeBcd+h2QqZF4NO3MC+ZKO5o/c0IXLXT9Bme08VRp5IkZZ9Ovv+wzRZ7gcdRGahUxjaa9k25Hmo3O4Zzf1i3EWfxJs8eNvkmPtFfcyrVpHuaNZMYehWnbmExlM1H7c32GPjrzfUz7s9crqzrPpV9hhQorOIzIJG5lhqSThFOhje86JzyECtdbqsdb9Dv1wyAbdqGrFX2Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.118.233) smtp.rcpttodomain=amd.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=15IebqioQLVniPg33q96CYi05OjcWulaE5tJTzOAFek=; b=LKm5PMiRVu4K4SMQh+eltzKRzgUdmaXmOWcXmdRGHE6qZCcZWQK55zXTw3KEW4a/aDFoBb8UP6NxUuJMrIqylarV/u9HPr5HWGKo6FhR42NNLxt4TU6qTWWn4vfFAZ2gxNwjyz4P6/BFNLGNZ/fREd9xx/8iD8No5AnQYqjhjF59tkutC91CURAnbt0D00iOwJ2gHKiNxZdgcTHG5OwSFsgKodPzCQAoYPZWl7lrAjkvBtJSXgW/svpGRA2uqoIF0Pa5X44Ab72HAPqqBY+EodxVMyQT05jSvnPeoXUWUjKrKZl+A/8fGT3Ay9kIIUDVkuwN2y20uM6Htz6VKtLLcw== Received: from CH2PR20CA0016.namprd20.prod.outlook.com (2603:10b6:610:58::26) by MN0PR12MB6366.namprd12.prod.outlook.com (2603:10b6:208:3c1::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8093.32; Wed, 30 Oct 2024 21:35:08 +0000 Received: from DS3PEPF000099DE.namprd04.prod.outlook.com (2603:10b6:610:58:cafe::2f) by CH2PR20CA0016.outlook.office365.com (2603:10b6:610:58::26) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8114.20 via Frontend Transport; Wed, 30 Oct 2024 21:35:08 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.118.233) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.118.233 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.118.233; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.118.233) by DS3PEPF000099DE.mail.protection.outlook.com (10.167.17.200) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8114.16 via Frontend Transport; Wed, 30 Oct 2024 21:35:08 +0000 Received: from drhqmail203.nvidia.com (10.126.190.182) by mail.nvidia.com (10.127.129.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Wed, 30 Oct 2024 14:35:03 -0700 Received: from drhqmail202.nvidia.com (10.126.190.181) by drhqmail203.nvidia.com (10.126.190.182) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Wed, 30 Oct 2024 14:35:02 -0700 Received: from Asurada-Nvidia.nvidia.com (10.127.8.12) by mail.nvidia.com (10.126.190.181) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Wed, 30 Oct 2024 14:35:01 -0700 From: Nicolin Chen To: , , CC: , , , , , , , , , , , , , , , , , , , , Subject: [PATCH v6 05/13] iommufd/viommu: Add IOMMU_VIOMMU_ALLOC ioctl Date: Wed, 30 Oct 2024 14:34:31 -0700 Message-ID: <5d2efe229821f5db7a832d9d6be84217289ac3f4.1730313237.git.nicolinc@nvidia.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS3PEPF000099DE:EE_|MN0PR12MB6366:EE_ X-MS-Office365-Filtering-Correlation-Id: c0464dfd-0b3f-4ecc-8baf-08dcf92ab9d8 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|36860700013|7416014|376014|82310400026; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?qEhFxC99pePxK0PWG/n4FFb6hW2BfBgU3bLwQJDghkawIvv6eTowtOV7qG4n?= =?us-ascii?Q?UPnSNz0tbnjkkG0lUpMFNksqDy21bM3fpCCu2NE1r8TL9RLYKson4/GSAcwE?= =?us-ascii?Q?DvUfc+8x4LWhqv0kiXFhYt/mE0LPNEcADLYQFIRdrCiVIXKp1B8rB44T39Vv?= =?us-ascii?Q?z4F1/OQZWyh6EPThueJUlEV6FJ4L5EUMQJpH93jJPKb74OHyoRVR4R/UA+qn?= =?us-ascii?Q?ZtTdQV9SvM/YC1rY8FVA8fiHoFG3EHcwR91YE0d0Zd5AB0hD2TtM9g5MCN1H?= =?us-ascii?Q?YG4uiFH1qZzi/oatGiPihrcwCaVO9h9G3N9RkzkntDUAbFdTUgZZG+WY5vr4?= =?us-ascii?Q?MPREbGM0Np1nRmWcWQtF0w90V3CMZ9JwQ1S2IFE6Vnarb0Atw87OnuoaAm7g?= =?us-ascii?Q?vVWLXNntGnLCpzkwugEjC0edYzN2l1X2Pg95ec8I1sSsHG8VF3NZP7F8GgQj?= =?us-ascii?Q?UCdXMgoSTbBixoBmXOkGPmL0BCdQXBUgv80rMX1ssEPrqv1p+9uHgx6ri6sz?= =?us-ascii?Q?x/3xunBKJjlUwUP+kTkX73qVuhwR3OtQBSHj8dwbRpcVmx6x5+Y/Pwo+eTRZ?= =?us-ascii?Q?olO6lCAV7EApp5c6nKfMlPEn95LrGALVnYFEvEy7tSiCtPVEnTzPV97zgphf?= =?us-ascii?Q?lAxAzVqnxh9AY+ZU0h3vvS2afWgxlCjtTFrt12DYUMXaU4sEKajlFWjlSula?= =?us-ascii?Q?yNyBKBYC+p1oCV+lnvIlbescNCbbk2GpZTSwk7sFSy5JEfbDJ1G7/dKIgtdB?= =?us-ascii?Q?QUuZQcjjlB6YpeaP+OFu2+A/KekSD/T3E9x77vVJG0TfDn1r85XWnAtRz7bq?= =?us-ascii?Q?WZb8SyGp87VNU1yTOdgjPZwUjU5TIfhKRnKky0mqZsmMZQkAkKM2+4R9HHFS?= =?us-ascii?Q?TSu9k8sjSE8NjZjAeHcwE6spoF1DD7m8vD5JYVjdsrJLiFI8GKla9rxGGDRd?= =?us-ascii?Q?wXmoVDGSIqur5uzp8h7/TcaUqN9KlicCBevygZNea8WMa60vMsUzK6G0onRI?= =?us-ascii?Q?lIBCE+HNOQu5ea2moCY7KSGkuYnvmAjsNMMsR1OizmjPuzMDlMms4LWIaweR?= =?us-ascii?Q?FpsPtMqv+ZzyvgBpw/Bdk0RCGV2mP7IbU5uDqSw2LniW7e+/sGibr00C2QR+?= =?us-ascii?Q?3gLtWeu48MmgHVoFvzsnl0O+nRSOsV4CNrsZpUHSkjXNuvuRX85W1iZuhMcK?= =?us-ascii?Q?QQDmjGCzwHZA+UMNJDXIEKwiENCOPYH7rM/SgL1Ily7/RsM9Aj78S//RyO2U?= =?us-ascii?Q?aqf6xHOAvaD7cPDQu1CRlcGA2pYVO2tyPISdcm3WNvHskObO+rwbOlGVcKvG?= =?us-ascii?Q?F01b2PpKIakush/dkR+h1Vkyd72ASv1xMXhNue+6XX2KBSQ7dgI4oV9cQ6eL?= =?us-ascii?Q?0Ox6vyYzNuvEot/IyFu8jikZNvYAZnw86BboKfkr+7ZAnx9Shw=3D=3D?= X-Forefront-Antispam-Report: CIP:216.228.118.233;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc7edge2.nvidia.com;CAT:NONE;SFS:(13230040)(1800799024)(36860700013)(7416014)(376014)(82310400026);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Oct 2024 21:35:08.0436 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: c0464dfd-0b3f-4ecc-8baf-08dcf92ab9d8 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.118.233];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DS3PEPF000099DE.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN0PR12MB6366 Content-Type: text/plain; charset="utf-8" Add a new ioctl for user space to do a vIOMMU allocation. It must be based on a nesting parent HWPT, so take its refcount. IOMMU driver wanting to support vIOMMUs must define its IOMMU_VIOMMU_TYPE_ in the uAPI header and implement a viommu_alloc op in its iommu_ops. Reviewed-by: Jason Gunthorpe Reviewed-by: Kevin Tian Signed-off-by: Nicolin Chen --- drivers/iommu/iommufd/Makefile | 3 +- drivers/iommu/iommufd/iommufd_private.h | 3 + include/uapi/linux/iommufd.h | 40 ++++++++++++ drivers/iommu/iommufd/main.c | 6 ++ drivers/iommu/iommufd/viommu.c | 81 +++++++++++++++++++++++++ 5 files changed, 132 insertions(+), 1 deletion(-) create mode 100644 drivers/iommu/iommufd/viommu.c diff --git a/drivers/iommu/iommufd/Makefile b/drivers/iommu/iommufd/Makefile index 266e23d657ff..8f6f0dd038d0 100644 --- a/drivers/iommu/iommufd/Makefile +++ b/drivers/iommu/iommufd/Makefile @@ -7,7 +7,8 @@ iommufd-y :=3D \ ioas.o \ main.o \ pages.o \ - vfio_compat.o + vfio_compat.o \ + viommu.o =20 iommufd-$(CONFIG_IOMMUFD_TEST) +=3D selftest.o =20 diff --git a/drivers/iommu/iommufd/iommufd_private.h b/drivers/iommu/iommuf= d/iommufd_private.h index be347f726fda..a8104d9d4cef 100644 --- a/drivers/iommu/iommufd/iommufd_private.h +++ b/drivers/iommu/iommufd/iommufd_private.h @@ -506,6 +506,9 @@ static inline int iommufd_hwpt_replace_device(struct io= mmufd_device *idev, return iommu_group_replace_domain(idev->igroup->group, hwpt->domain); } =20 +int iommufd_viommu_alloc_ioctl(struct iommufd_ucmd *ucmd); +void iommufd_viommu_destroy(struct iommufd_object *obj); + #ifdef CONFIG_IOMMUFD_TEST int iommufd_test(struct iommufd_ucmd *ucmd); void iommufd_selftest_destroy(struct iommufd_object *obj); diff --git a/include/uapi/linux/iommufd.h b/include/uapi/linux/iommufd.h index 41b1a01e9293..302844136b02 100644 --- a/include/uapi/linux/iommufd.h +++ b/include/uapi/linux/iommufd.h @@ -52,6 +52,7 @@ enum { IOMMUFD_CMD_HWPT_INVALIDATE =3D 0x8d, IOMMUFD_CMD_FAULT_QUEUE_ALLOC =3D 0x8e, IOMMUFD_CMD_IOAS_MAP_FILE =3D 0x8f, + IOMMUFD_CMD_VIOMMU_ALLOC =3D 0x90, }; =20 /** @@ -822,4 +823,43 @@ struct iommu_fault_alloc { __u32 out_fault_fd; }; #define IOMMU_FAULT_QUEUE_ALLOC _IO(IOMMUFD_TYPE, IOMMUFD_CMD_FAULT_QUEUE_= ALLOC) + +/** + * enum iommu_viommu_type - Virtual IOMMU Type + * @IOMMU_VIOMMU_TYPE_DEFAULT: Reserved for future use + */ +enum iommu_viommu_type { + IOMMU_VIOMMU_TYPE_DEFAULT =3D 0, +}; + +/** + * struct iommu_viommu_alloc - ioctl(IOMMU_VIOMMU_ALLOC) + * @size: sizeof(struct iommu_viommu_alloc) + * @flags: Must be 0 + * @type: Type of the virtual IOMMU. Must be defined in enum iommu_viommu_= type + * @dev_id: The device's physical IOMMU will be used to back the virtual I= OMMU + * @hwpt_id: ID of a nesting parent HWPT to associate to + * @out_viommu_id: Output virtual IOMMU ID for the allocated object + * + * Allocate a virtual IOMMU object, representing the underlying physical I= OMMU's + * virtualization support that is a security-isolated slice of the real IO= MMU HW + * that is unique to a specific VM. Operations global to the IOMMU are con= nected + * to the vIOMMU, such as: + * - Security namespace for guest owned ID, e.g. guest-controlled cache ta= gs + * - Non-device-affiliated event reporting, e.g. invalidation queue errors + * - Access to a sharable nesting parent pagetable across physical IOMMUs + * - Virtualization of various platforms IDs, e.g. RIDs and others + * - Delivery of paravirtualized invalidation + * - Direct assigned invalidation queues + * - Direct assigned interrupts + */ +struct iommu_viommu_alloc { + __u32 size; + __u32 flags; + __u32 type; + __u32 dev_id; + __u32 hwpt_id; + __u32 out_viommu_id; +}; +#define IOMMU_VIOMMU_ALLOC _IO(IOMMUFD_TYPE, IOMMUFD_CMD_VIOMMU_ALLOC) #endif diff --git a/drivers/iommu/iommufd/main.c b/drivers/iommu/iommufd/main.c index 30e6c2af3b45..cc514f9bc3e6 100644 --- a/drivers/iommu/iommufd/main.c +++ b/drivers/iommu/iommufd/main.c @@ -307,6 +307,7 @@ union ucmd_buffer { struct iommu_ioas_unmap unmap; struct iommu_option option; struct iommu_vfio_ioas vfio_ioas; + struct iommu_viommu_alloc viommu; #ifdef CONFIG_IOMMUFD_TEST struct iommu_test_cmd test; #endif @@ -360,6 +361,8 @@ static const struct iommufd_ioctl_op iommufd_ioctl_ops[= ] =3D { val64), IOCTL_OP(IOMMU_VFIO_IOAS, iommufd_vfio_ioas, struct iommu_vfio_ioas, __reserved), + IOCTL_OP(IOMMU_VIOMMU_ALLOC, iommufd_viommu_alloc_ioctl, + struct iommu_viommu_alloc, out_viommu_id), #ifdef CONFIG_IOMMUFD_TEST IOCTL_OP(IOMMU_TEST_CMD, iommufd_test, struct iommu_test_cmd, last), #endif @@ -495,6 +498,9 @@ static const struct iommufd_object_ops iommufd_object_o= ps[] =3D { [IOMMUFD_OBJ_FAULT] =3D { .destroy =3D iommufd_fault_destroy, }, + [IOMMUFD_OBJ_VIOMMU] =3D { + .destroy =3D iommufd_viommu_destroy, + }, #ifdef CONFIG_IOMMUFD_TEST [IOMMUFD_OBJ_SELFTEST] =3D { .destroy =3D iommufd_selftest_destroy, diff --git a/drivers/iommu/iommufd/viommu.c b/drivers/iommu/iommufd/viommu.c new file mode 100644 index 000000000000..888239b78667 --- /dev/null +++ b/drivers/iommu/iommufd/viommu.c @@ -0,0 +1,81 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* Copyright (c) 2024, NVIDIA CORPORATION & AFFILIATES + */ +#include "iommufd_private.h" + +void iommufd_viommu_destroy(struct iommufd_object *obj) +{ + struct iommufd_viommu *viommu =3D + container_of(obj, struct iommufd_viommu, obj); + + if (viommu->ops && viommu->ops->destroy) + viommu->ops->destroy(viommu); + refcount_dec(&viommu->hwpt->common.obj.users); +} + +int iommufd_viommu_alloc_ioctl(struct iommufd_ucmd *ucmd) +{ + struct iommu_viommu_alloc *cmd =3D ucmd->cmd; + struct iommufd_hwpt_paging *hwpt_paging; + struct iommufd_viommu *viommu; + struct iommufd_device *idev; + const struct iommu_ops *ops; + int rc; + + if (cmd->flags || cmd->type =3D=3D IOMMU_VIOMMU_TYPE_DEFAULT) + return -EOPNOTSUPP; + + idev =3D iommufd_get_device(ucmd, cmd->dev_id); + if (IS_ERR(idev)) + return PTR_ERR(idev); + + ops =3D dev_iommu_ops(idev->dev); + if (!ops->viommu_alloc) { + rc =3D -EOPNOTSUPP; + goto out_put_idev; + } + + hwpt_paging =3D iommufd_get_hwpt_paging(ucmd, cmd->hwpt_id); + if (IS_ERR(hwpt_paging)) { + rc =3D PTR_ERR(hwpt_paging); + goto out_put_idev; + } + + if (!hwpt_paging->nest_parent) { + rc =3D -EINVAL; + goto out_put_hwpt; + } + + viommu =3D ops->viommu_alloc(idev->dev, hwpt_paging->common.domain, + ucmd->ictx, cmd->type); + if (IS_ERR(viommu)) { + rc =3D PTR_ERR(viommu); + goto out_put_hwpt; + } + + viommu->type =3D cmd->type; + viommu->ictx =3D ucmd->ictx; + viommu->hwpt =3D hwpt_paging; + refcount_inc(&viommu->hwpt->common.obj.users); + /* + * It is the most likely case that a physical IOMMU is unpluggable. A + * pluggable IOMMU instance (if exists) is responsible for refcounting + * on its own. + */ + viommu->iommu_dev =3D __iommu_get_iommu_dev(idev->dev); + + cmd->out_viommu_id =3D viommu->obj.id; + rc =3D iommufd_ucmd_respond(ucmd, sizeof(*cmd)); + if (rc) + goto out_abort; + iommufd_object_finalize(ucmd->ictx, &viommu->obj); + goto out_put_hwpt; + +out_abort: + iommufd_object_abort_and_destroy(ucmd->ictx, &viommu->obj); +out_put_hwpt: + iommufd_put_object(ucmd->ictx, &hwpt_paging->common.obj); +out_put_idev: + iommufd_put_object(ucmd->ictx, &idev->obj); + return rc; +} --=20 2.43.0 From nobody Mon Nov 25 00:39:33 2024 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2076.outbound.protection.outlook.com [40.107.220.76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 42C9E228B64; Wed, 30 Oct 2024 21:35:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.220.76 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730324120; cv=fail; b=fDxyoojh6WGiotnkH7J2vMmU6LoXsb0aPtRKGcx66/Pmkhy7Z4JpUjyC+mLUyKie10AVxcOieSGITrYtZmy5UeePSbtpHd7pmcwPYpEuRfmA1a3JuA0/P42SyBs2DrmhE+sVkBQxjIS41WqktvvZoAVvunSyQY/Vw8A+55LW9Zc= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730324120; c=relaxed/simple; bh=t97PcZI+7XE90w0tbKnApVt2sZ/ixCrFm9pYuL6ycFU=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=ItqYYN0gl+shkgV5WM3wMVlg7yLlYhRpHUXhVPV0LAQFofdxaM6rbp/iQnbmu8UoCL5DKvNcUbOURHYsFj0UAZoYhNHIZKf114AjzSA0xGmEwGzoNTaxFmT9C9I+r8Vy5p4uVsXWFENLOYAMO7grjsyCglehvbSJAl9puDMRebw= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=Ke20CTNG; arc=fail smtp.client-ip=40.107.220.76 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="Ke20CTNG" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=H8NhMiGsxqfWWbzbV8iUumdt0vvqbpu9jIU8QG2RbmJwjVVNpq/CnUVkIy54IlZP3jAxnjp9rz5Jc9sjkdltCBjLETYsHV7xJCgqaxz60lkYV8479ERyD37hNxAMa5ltgFpqA2VxZf6JPNX9z980RL6FjJ+uOCL/dCDU5lNcJsydn5qZ3llqIqDfnZLh74WF7HbICLoNRhv3bxVurr/MEUFzJ6vg2DzLARcLvmV868BBI+hmpOzCwEX5cy2T/tUrQAvAj8KhCUOP2+O/n3aEVcVXy8uH3FMg//28vj2YgR7eOiO7EwuqZmIPBQE984Di1bd7co0mxz+HO0m+zKRhMw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=L7yhxB3ToBGOWp/P8r6NtWeZuBj1bQjK7yQRPxEllrI=; b=btVzy+jTMXP6RAJ4TYHu2LnxbAqhWUtxrGWogtD6dfYatzoNF/hSPV31crh+UESPE4dk3WK21iOd5qnZtDFB6zHca1RzN0pK3lvIU7QRXWqXLLshDji6qiWSlNtZHq/A62xsTFC58paPw+zYJArnOFKhByq/DClx3jl2vaiaIvClKfSOHOEauaPS6UPkUZfYcqFdZHXPostSi8zHYzoKKIs45JnZTFJpFrrRxSq3qn8mvL7gK1tW+H6AnMsXkSUVknpa6EdWC91OzSYrpm/7kxFNPCjVk/iWnoVlo3d5vnRk7eflEGVbDtOAGbq1aaJ1pGo1BmI1MgDQCLP3aAdNqg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.118.232) smtp.rcpttodomain=amd.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=L7yhxB3ToBGOWp/P8r6NtWeZuBj1bQjK7yQRPxEllrI=; b=Ke20CTNG0RpJU7/g4VeH4RzKP3/QCunT+MF1waNmYPmedQ+03S685H8LtXD33Nswu6cRTdhfxR+KCq2X6dO1/+ZYwN0w/ip0+ip40WguSSnVybYqEU3fd4koBgB3E1+mN3evIgGQZfmQ7B5k/oqro/THX9Qq4XIO/xd3kkkUj8Y0vimt9GJhOFK0S3kLIPg+xHBmZ+EFYTPfyoS31LttA+dh5afSGTNN5n9YsH4UsFUTuSRcDdhDy/wDlrpfQKO44HPa0k7cucaOdNvosZ+5oWa0dqVxGpH7weXmAm9iw89O4vkEpFiZyq4nP4wUcua8CaFzYEAF0oHmEkDVKrb5xA== Received: from SN4PR0501CA0009.namprd05.prod.outlook.com (2603:10b6:803:40::22) by LV8PR12MB9183.namprd12.prod.outlook.com (2603:10b6:408:193::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8093.27; Wed, 30 Oct 2024 21:35:13 +0000 Received: from SN1PEPF0002BA4B.namprd03.prod.outlook.com (2603:10b6:803:40:cafe::2c) by SN4PR0501CA0009.outlook.office365.com (2603:10b6:803:40::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8114.17 via Frontend Transport; Wed, 30 Oct 2024 21:35:13 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.118.232) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.118.232 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.118.232; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.118.232) by SN1PEPF0002BA4B.mail.protection.outlook.com (10.167.242.68) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8114.16 via Frontend Transport; Wed, 30 Oct 2024 21:35:13 +0000 Received: from drhqmail203.nvidia.com (10.126.190.182) by mail.nvidia.com (10.127.129.5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Wed, 30 Oct 2024 14:35:05 -0700 Received: from drhqmail202.nvidia.com (10.126.190.181) by drhqmail203.nvidia.com (10.126.190.182) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Wed, 30 Oct 2024 14:35:04 -0700 Received: from Asurada-Nvidia.nvidia.com (10.127.8.12) by mail.nvidia.com (10.126.190.181) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Wed, 30 Oct 2024 14:35:03 -0700 From: Nicolin Chen To: , , CC: , , , , , , , , , , , , , , , , , , , , Subject: [PATCH v6 06/13] iommufd: Add alloc_domain_nested op to iommufd_viommu_ops Date: Wed, 30 Oct 2024 14:34:32 -0700 Message-ID: <475a4c7bcf8af86ed00259bc29bbfdbb42c66b7f.1730313237.git.nicolinc@nvidia.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SN1PEPF0002BA4B:EE_|LV8PR12MB9183:EE_ X-MS-Office365-Filtering-Correlation-Id: 9af5444b-bd4a-4f25-b790-08dcf92abd1c X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|376014|7416014|82310400026|36860700013; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?0nPwqRTcbnXpnWcuw2qzzF6uMSfCNPhlmpCCThFEc4KxWk5P7yYF3OeXT71C?= =?us-ascii?Q?Girp7bq8zNNskEG6A59ygOc31crnzVjvox9r6Nyto9yxn7xH9mnk26i72Bi/?= =?us-ascii?Q?xX0Vy+JzcJm2i5ecP/J60gPys+dmQGgrH3CuAABQGLYkyi3tOcg8NneXYraJ?= =?us-ascii?Q?d54wt+yer8X3buuWJSgvFQf6vJTGc3fdNmrFE9fPCzTayTObpubYivnoGu6R?= =?us-ascii?Q?z+11pK42YZ3X/JgU/lUv9I1oKQAEQfv2kkben4oHu8giJM4758wtqIq2xzT8?= =?us-ascii?Q?mfPlIcALVjPmVz8lgRJ7nzFn22Z7IM2jppmrcYntgfSaZ/YQOKsTJ3BQolZO?= =?us-ascii?Q?t0F4qXbIqrcnPlR9lw0K3FRuEhHcLMyoXjFUdfQvr5xv3eqqKXDCfWMapQOv?= =?us-ascii?Q?fsqoIDnRBy8CV0J4woFUlS0zDbbAXyL8uphjZFhZIVUG5Yhqp1V0PzZBqKJu?= =?us-ascii?Q?iaAWJIA6q7PYymB+NyKldwoczHJuoo9qqoDN0SZbn0m2Cpg/8Xl48Of9mtU0?= =?us-ascii?Q?5wHyrLCmeYyqbYu9zerVmDajhHOGExH9griA8BnTKJIjSyT25lzQmuQkLBJ8?= =?us-ascii?Q?oopBK6rnSZykOhD/f1+34eYLq3ZAYysixoKqa4uyiX+Pd2Xcs/swsDdhr7Ie?= =?us-ascii?Q?o4hf77XW8KQ6T6MvgrbJDg2N5MG+eGVbXlY21l2G4UKeT37LHUAVCZse+OeP?= =?us-ascii?Q?sTvyZ6BgvNlNLPKzJkxg2H/8OASRRPh8M/R0WhtUBF9RvNQhq6IGbtwni4WM?= =?us-ascii?Q?vjfUsxa+bBgiaP2LzsxOhRikzrbcHyHIFfoQS7YY0w2jN8M0cSZrSC9HMbWd?= =?us-ascii?Q?c14yqPkMN0+fmRvfzuozAVuuIVmC4IOPtIJuErfLB3jcBj/U0KJYG49KIQbk?= =?us-ascii?Q?/7ok/JR8jJKdbUd5FlRmod6CBxqhXxItq2gKGliN/Xxemxc/XggDr+Llpqnb?= =?us-ascii?Q?9JMI82/X8cXtb9Az4rsUh2M5DXkO3lrDoVw+5Z6Z8fAmtugEFT8ck40P/ZeR?= =?us-ascii?Q?o6csX9ZE5Ehwibwcs5o4Hv0EfKw7DAM6+3R/T0qmUsPCPkqnB+HCfM/yHG38?= =?us-ascii?Q?2Xtioknhz3Zj5PK/Us0Qaa3qP6QySO71SwCgKOcPQLFrbHxfd8jb8+7MCP9z?= =?us-ascii?Q?lT7YxZ1un80V428w6vt0uc/i86O69jf1LpueyKHjOutQHsk/yr0+8q1D2gn+?= =?us-ascii?Q?oYpiPdHx4XN5NF2QFxsmmjk9pzWUGSFBRTbMEegHgq394hNpgFJ0g4h9GZkF?= =?us-ascii?Q?VmvuxuvJC3HiUJMlzs55YufjezlrHiXaf7Hvaksxywjw3nocUxpaHBBx/o1p?= =?us-ascii?Q?0TqAappahlXyfiHFQOPUVIOMElT3tDctMKt3Rt2qWEpJVsT3ltZ1jP+xTvUa?= =?us-ascii?Q?a69Ze5FEF3QrWYB1EStqC5goQLQXFxqWFJ5yaKHS+p+Lr9Imbg=3D=3D?= X-Forefront-Antispam-Report: CIP:216.228.118.232;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc7edge1.nvidia.com;CAT:NONE;SFS:(13230040)(1800799024)(376014)(7416014)(82310400026)(36860700013);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Oct 2024 21:35:13.5080 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 9af5444b-bd4a-4f25-b790-08dcf92abd1c X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.118.232];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: SN1PEPF0002BA4B.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: LV8PR12MB9183 Content-Type: text/plain; charset="utf-8" Allow IOMMU driver to use a vIOMMU object that holds a nesting parent hwpt/domain to allocate a nested domain. Suggested-by: Jason Gunthorpe Reviewed-by: Kevin Tian Reviewed-by: Jason Gunthorpe Signed-off-by: Nicolin Chen --- include/linux/iommufd.h | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/include/linux/iommufd.h b/include/linux/iommufd.h index 457e763e8b63..f03c75410938 100644 --- a/include/linux/iommufd.h +++ b/include/linux/iommufd.h @@ -14,6 +14,7 @@ struct device; struct file; struct iommu_group; +struct iommu_user_data; struct iommufd_access; struct iommufd_ctx; struct iommufd_device; @@ -95,9 +96,17 @@ struct iommufd_viommu { * struct iommufd_viommu_ops - vIOMMU specific operations * @destroy: Clean up all driver-specific parts of an iommufd_viommu. The = memory * of the vIOMMU will be free-ed by iommufd core after calling t= his op + * @alloc_domain_nested: Allocate a IOMMU_DOMAIN_NESTED on a vIOMMU that h= olds a + * nesting parent domain (IOMMU_DOMAIN_PAGING). @use= r_data + * must be defined in include/uapi/linux/iommufd.h. + * It must fully initialize the new iommu_domain bef= ore + * returning. Upon failure, ERR_PTR must be returned. */ struct iommufd_viommu_ops { void (*destroy)(struct iommufd_viommu *viommu); + struct iommu_domain *(*alloc_domain_nested)( + struct iommufd_viommu *viommu, u32 flags, + const struct iommu_user_data *user_data); }; =20 #if IS_ENABLED(CONFIG_IOMMUFD) --=20 2.43.0 From nobody Mon Nov 25 00:39:33 2024 Received: from NAM02-BN1-obe.outbound.protection.outlook.com (mail-bn1nam02on2070.outbound.protection.outlook.com [40.107.212.70]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 96FFD218593; Wed, 30 Oct 2024 21:35:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.212.70 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730324123; cv=fail; b=cMGjanUkxnibBQ/eBsghl8kxCGWQgNQ/gD2OW+xGTtKLCp9xfClvYQgHY4sdCMdeet+6vI6JYSnnHtrIbvlmI7xtyhPFl37c55e6lbSsqt2f78xYU64IfQFH66UVnpYWPEvp/EurvYI6lrDN4VLKJu/FhCRr+0mQ2sqmOWMQoP8= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730324123; c=relaxed/simple; bh=cDRLYdzQm1IPPhP1XTJs7cXDmg1Xn1Iosk4EUessKQo=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=BkHKdmcevCVh6z/mxm1GriVlko4TcBUm+gOTaqG8mBjEFXKpAgdjEhW2UAps2F5CxpSXhX70YIkPjJutRk3MfRgN3JwGk7e0tn+O33OF8lCyqAJZReVH8LJ8kej8wjMsureub08P6QQXiRkrhdycPrdVV35P0NXEECB9giZCxjo= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=FYko5vp3; arc=fail smtp.client-ip=40.107.212.70 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="FYko5vp3" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=nYMSpoBJ5ZqProYXHTl0Wxg0W9nHLCC6thXp+9RcfRNLUQam8thz0akogniAWOIaEbRbnblw0D37nssXtw7Z9pFCi3/Ttznes2LFPoIvZyRN9PoyFqabuFnipb1VmeJtV9qYRJ0OtPNS1WojtBMZGaoZXC84Xh2eYb/lUKlvaX7B2XpAgLuATZH9si4h2jiCu11Cleh3Ht2E8YrDpAAGoiZs0jt0M0p8YvBMifBXJd4ZgUn2LyfLD9EkBbZr6xtoamj8lXcg25nyuQC1l2qih2ScJczu2HLpLpkEmOr91QMPTMWV3Yq/H27i6lWroke2M65BliUxDSVu8w38kleiOg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=f17+eqxu4c7DRCvtR6AHha02sBOO9Y9368BrSi4yXRU=; b=CDpPzuPq2JaHFExjHcxoXndLrEiTy0R6uAY50GUrn36QEk/BqLGYlL1fwFySMPN7jbxizGfa5a5tmXQmuqVAhNrG2W1yO8hXEODkv8qRjRaiHaDDR3pGZEQ6OAg5kzUlG3PZDc9lSQ9sJhAAeUyrrQu/PDFUa0pijhXzoKdIwUD8tPXEr59s0CmiY+q0f2V1pPkxJgmXBrqnmvUPd3sW36/EZJcXGYhXzDBlEqon+diPNuYCJS40FC3lUc01EQNxHUwQay82FmJfSCeB8g6ozk4a0VfA7SLa8kupAY60axMGwr6H4sr3eFCDkY3x15PSvmJKdoopyv2ox3t2Zqw+lQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.118.232) smtp.rcpttodomain=amd.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=f17+eqxu4c7DRCvtR6AHha02sBOO9Y9368BrSi4yXRU=; b=FYko5vp3IhKsb/AhjaxU1tCZfY7oKwO4MPIO77d7U6sNXxO22Sg+oAYPjGBO05R9/c5CZ8zAlyGq3DLm3t8ssXdXOuVtvEfG5lWm2hJD0AoJhUuRUu39jrMb3n9peLtYzwmJrK6WOa2Wi6IWR/YUvffiF6Kx2fvezdOBsAeacPB7tdPKYyyIujsD5uyyGIrlrDNydQ4sy7Z5y7ntMpK329LYZ2hLpqEXGoO3vWoVqaqkQQ/S097RyuO8drJB2F35sLO9VG2FCn+S8WVqDBvgDLPbCvGlkGi/nb3HwZ7ApTBvDv8Q+D9t31mDsq6hOH6OZZ9rUlVFpcGGlw7fiKw3wg== Received: from SN4PR0501CA0023.namprd05.prod.outlook.com (2603:10b6:803:40::36) by SN7PR12MB8820.namprd12.prod.outlook.com (2603:10b6:806:341::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8114.18; Wed, 30 Oct 2024 21:35:16 +0000 Received: from SN1PEPF0002BA4B.namprd03.prod.outlook.com (2603:10b6:803:40:cafe::76) by SN4PR0501CA0023.outlook.office365.com (2603:10b6:803:40::36) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8069.15 via Frontend Transport; Wed, 30 Oct 2024 21:35:16 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.118.232) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.118.232 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.118.232; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.118.232) by SN1PEPF0002BA4B.mail.protection.outlook.com (10.167.242.68) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8114.16 via Frontend Transport; Wed, 30 Oct 2024 21:35:16 +0000 Received: from drhqmail203.nvidia.com (10.126.190.182) by mail.nvidia.com (10.127.129.5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Wed, 30 Oct 2024 14:35:06 -0700 Received: from drhqmail202.nvidia.com (10.126.190.181) by drhqmail203.nvidia.com (10.126.190.182) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Wed, 30 Oct 2024 14:35:05 -0700 Received: from Asurada-Nvidia.nvidia.com (10.127.8.12) by mail.nvidia.com (10.126.190.181) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Wed, 30 Oct 2024 14:35:04 -0700 From: Nicolin Chen To: , , CC: , , , , , , , , , , , , , , , , , , , , Subject: [PATCH v6 07/13] iommufd: Allow pt_id to carry viommu_id for IOMMU_HWPT_ALLOC Date: Wed, 30 Oct 2024 14:34:33 -0700 Message-ID: <41800674501340c4503cc3ad79affd19449a9ee1.1730313237.git.nicolinc@nvidia.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SN1PEPF0002BA4B:EE_|SN7PR12MB8820:EE_ X-MS-Office365-Filtering-Correlation-Id: acc5200b-2a04-4f46-b6f9-08dcf92abee1 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|7416014|376014|1800799024|36860700013; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?wWsCt5i8g2XJKoRVLHDJq/hPNUn5MnBzVbjUONCpQbwzkaLszpCPbmPIcwv/?= =?us-ascii?Q?7Sltyl704Y3xMVyXqdGt/+U/hNeGiPryR336LLbnTzx3TZr76CSCL3IKi8if?= =?us-ascii?Q?4JDHArxQBJI3CHuCLHDjcO1lQ8Jj+dHcp+wyUsOF730WGt+x/WNlcTzv1qUV?= =?us-ascii?Q?2tLQr7E7bAuW6RuCRVffsAvWCmt5bCT9F2xwrzjmEoyziBCenND+dBKadbuR?= =?us-ascii?Q?Q8fYLl80nPpcet8x3aMPei8jywvY5BYQG6B1OQzChoNIxJkJYuqtXCbtXhf4?= =?us-ascii?Q?YPpzYT76ygdzG01AiFsL6juQoOuOyaH1Df5VlctBd8z28DYpFnqSfSX8Idrv?= =?us-ascii?Q?ziLmt/HYY9AZaMO2Co+HQge59R5t860oyHB6QVB8F+75MW3bS0c+LUMYjvSW?= =?us-ascii?Q?ClmAjbZq4TMchwMifu2hDsDPgs7GvhxCuICNlAanDaTdzzkWerUB4F6ZnNMY?= =?us-ascii?Q?IIIoOIORtl4dx33IUCg8jvllnhzhqsN3+ZfYmkqppspkocMgwgZJYGvs4QhW?= =?us-ascii?Q?edCN27VedUV/fI+VD+VNObYAUde8ILWDxwEqxqGWCQ3+aO9T1XlUYUY7BjVe?= =?us-ascii?Q?F6mSNt7QuydSQaaQAAZmckb5aKRiQazKBXnDlUKmGzVodf6oll6L7WBE2Dt4?= =?us-ascii?Q?4GjmZU6Bw7uj30pGthmxPlygbggQZe2MsKIAsZWbiGoQL7iit5VFkEw1ecQK?= =?us-ascii?Q?fUpRHPY3hWG3CC30VcMk/Za777JCiB+J2znEbL7Av4pjSPGI62mMNLazIBB0?= =?us-ascii?Q?T5S+HKNCLBkHK2lhWnSYvu6gaT1WkpappadiMcyeKDyS2JCfFae4wckKVaaQ?= =?us-ascii?Q?4s7Umt6LaqZQELJL4H9s+FwrbssyJFQPYFrXjKEivHWfooPmLw1UxBZZVdAx?= =?us-ascii?Q?VLeVCdDYDkn20m47rMwhd+gL/n40XY3dDFRSGxWDmrikX3c93C7UbXaEvb/p?= =?us-ascii?Q?LKGqxqs3W32U3fxshApDy+gBRb+J1aWRK7F8764ix+1cXb0stJgq2OxrBbxV?= =?us-ascii?Q?4c+oKROsBqwawi4xQwpqOVHko+7IckHllt5VgDLiNnM63jgISV+s2Y6c78i/?= =?us-ascii?Q?4y9EmMWpbB8ELzFttVbZYYT+U9EqowveUA2hkg69pKiLvnHGBsAC2T1xjlTT?= =?us-ascii?Q?7cwWfV/WcaNGKco9Uiw1TySGD1w29HTK73DJ8zk+qmkV+O79VXYqBdAdzdYT?= =?us-ascii?Q?9lMPJezfu0XpyWxb3l14kvw5RO+jV12y5bfbaQ5htQVBebSCAFPZFgDk0KkL?= =?us-ascii?Q?xQl/KkAQHyv4zdHYas7L3r5paLXcoe4ACzPdN7+KhnmWCy52c8BjyraOuEQm?= =?us-ascii?Q?Yv7ynGneAi1N0iA0tWb3KuadoLvkOdVY8umE0Ur3AmpwSli6TsGFOW0O7WeS?= =?us-ascii?Q?mOVQFi1OZZnW77ko0X6Cr7GQiDRXJMJSq6h3hWw4B/S6AfmPqA=3D=3D?= X-Forefront-Antispam-Report: CIP:216.228.118.232;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc7edge1.nvidia.com;CAT:NONE;SFS:(13230040)(82310400026)(7416014)(376014)(1800799024)(36860700013);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Oct 2024 21:35:16.4923 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: acc5200b-2a04-4f46-b6f9-08dcf92abee1 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.118.232];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: SN1PEPF0002BA4B.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN7PR12MB8820 Content-Type: text/plain; charset="utf-8" Now a vIOMMU holds a shareable nesting parent HWPT. So, it can act like that nesting parent HWPT to allocate a nested HWPT. Support that in the IOMMU_HWPT_ALLOC ioctl handler, and update its kdoc. Also, add an iommufd_viommu_alloc_hwpt_nested helper to allocate a nested HWPT for a vIOMMU object. Since a vIOMMU object holds the parent hwpt's refcount already, increase the refcount of the vIOMMU only. Reviewed-by: Kevin Tian Signed-off-by: Nicolin Chen --- drivers/iommu/iommufd/iommufd_private.h | 1 + include/uapi/linux/iommufd.h | 14 ++--- drivers/iommu/iommufd/hw_pagetable.c | 72 ++++++++++++++++++++++++- 3 files changed, 80 insertions(+), 7 deletions(-) diff --git a/drivers/iommu/iommufd/iommufd_private.h b/drivers/iommu/iommuf= d/iommufd_private.h index a8104d9d4cef..e8f5ef550cc9 100644 --- a/drivers/iommu/iommufd/iommufd_private.h +++ b/drivers/iommu/iommufd/iommufd_private.h @@ -290,6 +290,7 @@ struct iommufd_hwpt_paging { struct iommufd_hwpt_nested { struct iommufd_hw_pagetable common; struct iommufd_hwpt_paging *parent; + struct iommufd_viommu *viommu; }; =20 static inline bool hwpt_is_paging(struct iommufd_hw_pagetable *hwpt) diff --git a/include/uapi/linux/iommufd.h b/include/uapi/linux/iommufd.h index 302844136b02..a498d4838f9a 100644 --- a/include/uapi/linux/iommufd.h +++ b/include/uapi/linux/iommufd.h @@ -435,7 +435,7 @@ enum iommu_hwpt_data_type { * @size: sizeof(struct iommu_hwpt_alloc) * @flags: Combination of enum iommufd_hwpt_alloc_flags * @dev_id: The device to allocate this HWPT for - * @pt_id: The IOAS or HWPT to connect this HWPT to + * @pt_id: The IOAS or HWPT or vIOMMU to connect this HWPT to * @out_hwpt_id: The ID of the new HWPT * @__reserved: Must be 0 * @data_type: One of enum iommu_hwpt_data_type @@ -454,11 +454,13 @@ enum iommu_hwpt_data_type { * IOMMU_HWPT_DATA_NONE. The HWPT can be allocated as a parent HWPT for a * nesting configuration by passing IOMMU_HWPT_ALLOC_NEST_PARENT via @flag= s. * - * A user-managed nested HWPT will be created from a given parent HWPT via - * @pt_id, in which the parent HWPT must be allocated previously via the - * same ioctl from a given IOAS (@pt_id). In this case, the @data_type - * must be set to a pre-defined type corresponding to an I/O page table - * type supported by the underlying IOMMU hardware. + * A user-managed nested HWPT will be created from a given vIOMMU (wrappin= g a + * parent HWPT) or a parent HWPT via @pt_id, in which the parent HWPT must= be + * allocated previously via the same ioctl from a given IOAS (@pt_id). In = this + * case, the @data_type must be set to a pre-defined type corresponding to= an + * I/O page table type supported by the underlying IOMMU hardware. The dev= ice + * via @dev_id and the vIOMMU via @pt_id must be associated to the same IO= MMU + * instance. * * If the @data_type is set to IOMMU_HWPT_DATA_NONE, @data_len and * @data_uptr should be zero. Otherwise, both @data_len and @data_uptr diff --git a/drivers/iommu/iommufd/hw_pagetable.c b/drivers/iommu/iommufd/h= w_pagetable.c index d06bf6e6c19f..384afa374d25 100644 --- a/drivers/iommu/iommufd/hw_pagetable.c +++ b/drivers/iommu/iommufd/hw_pagetable.c @@ -57,7 +57,10 @@ void iommufd_hwpt_nested_destroy(struct iommufd_object *= obj) container_of(obj, struct iommufd_hwpt_nested, common.obj); =20 __iommufd_hwpt_destroy(&hwpt_nested->common); - refcount_dec(&hwpt_nested->parent->common.obj.users); + if (hwpt_nested->viommu) + refcount_dec(&hwpt_nested->viommu->obj.users); + else + refcount_dec(&hwpt_nested->parent->common.obj.users); } =20 void iommufd_hwpt_nested_abort(struct iommufd_object *obj) @@ -260,6 +263,57 @@ iommufd_hwpt_nested_alloc(struct iommufd_ctx *ictx, return ERR_PTR(rc); } =20 +/** + * iommufd_viommu_alloc_hwpt_nested() - Get a hwpt_nested for a vIOMMU + * @viommu: vIOMMU ojbect to associate the hwpt_nested/domain with + * @user_data: user_data pointer. Must be valid + * + * Allocate a new IOMMU_DOMAIN_NESTED for a vIOMMU and return it as a NEST= ED + * hw_pagetable. + */ +static struct iommufd_hwpt_nested * +iommufd_viommu_alloc_hwpt_nested(struct iommufd_viommu *viommu, u32 flags, + const struct iommu_user_data *user_data) +{ + struct iommufd_hwpt_nested *hwpt_nested; + struct iommufd_hw_pagetable *hwpt; + int rc; + + if ((flags & ~IOMMU_HWPT_FAULT_ID_VALID) || !user_data->len) + return ERR_PTR(-EOPNOTSUPP); + if (!viommu->ops || !viommu->ops->alloc_domain_nested) + return ERR_PTR(-EOPNOTSUPP); + + hwpt_nested =3D __iommufd_object_alloc( + viommu->ictx, hwpt_nested, IOMMUFD_OBJ_HWPT_NESTED, common.obj); + if (IS_ERR(hwpt_nested)) + return ERR_CAST(hwpt_nested); + hwpt =3D &hwpt_nested->common; + + hwpt_nested->viommu =3D viommu; + refcount_inc(&viommu->obj.users); + hwpt_nested->parent =3D viommu->hwpt; + + hwpt->domain =3D + viommu->ops->alloc_domain_nested(viommu, flags, user_data); + if (IS_ERR(hwpt->domain)) { + rc =3D PTR_ERR(hwpt->domain); + hwpt->domain =3D NULL; + goto out_abort; + } + hwpt->domain->owner =3D viommu->iommu_dev->ops; + + if (WARN_ON_ONCE(hwpt->domain->type !=3D IOMMU_DOMAIN_NESTED)) { + rc =3D -EINVAL; + goto out_abort; + } + return hwpt_nested; + +out_abort: + iommufd_object_abort_and_destroy(viommu->ictx, &hwpt->obj); + return ERR_PTR(rc); +} + int iommufd_hwpt_alloc(struct iommufd_ucmd *ucmd) { struct iommu_hwpt_alloc *cmd =3D ucmd->cmd; @@ -316,6 +370,22 @@ int iommufd_hwpt_alloc(struct iommufd_ucmd *ucmd) goto out_unlock; } hwpt =3D &hwpt_nested->common; + } else if (pt_obj->type =3D=3D IOMMUFD_OBJ_VIOMMU) { + struct iommufd_hwpt_nested *hwpt_nested; + struct iommufd_viommu *viommu; + + viommu =3D container_of(pt_obj, struct iommufd_viommu, obj); + if (viommu->iommu_dev !=3D __iommu_get_iommu_dev(idev->dev)) { + rc =3D -EINVAL; + goto out_unlock; + } + hwpt_nested =3D iommufd_viommu_alloc_hwpt_nested( + viommu, cmd->flags, &user_data); + if (IS_ERR(hwpt_nested)) { + rc =3D PTR_ERR(hwpt_nested); + goto out_unlock; + } + hwpt =3D &hwpt_nested->common; } else { rc =3D -EINVAL; goto out_put_pt; --=20 2.43.0 From nobody Mon Nov 25 00:39:33 2024 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2075.outbound.protection.outlook.com [40.107.93.75]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 698E72296DD; Wed, 30 Oct 2024 21:35:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.93.75 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730324135; cv=fail; b=i0ft/uLqVJR8SKbSLA1P+s3V8of9n06nu5SJXi9KvwaaTe6PNvt9p6sUCY358Lxwl8mBKe32LqAt/94uci0P61zQcqvk6s/7O6tPsFHybrXTAvDQxhFHqjpa+/q459l56sqkogb7CxZyHObIYvWsGv1hpRaM5IWdxCh442Sdl+k= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730324135; c=relaxed/simple; bh=ju5HyEZzFfmDYHxRYjKrj8DcYQoamkrIU8iuQIhWIUo=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=VbvktbvBKrkqALbv2AWqJIkEKIm1A2y0tAHVoGFhgTlODUxT5fUcS2TziFU4yvRukgH+VKbOxY1XwV+EouQpLe6ZtaAoKE8sE0G+dO0DDfspOiaAQHp3b4aA0axCE17abNRn6Fh1ifd4JxmBa53bzDIKGTwlVp85wesD0kBe4/A= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=fN/uYz3L; arc=fail smtp.client-ip=40.107.93.75 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="fN/uYz3L" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=wv9dfzaLiUNkpdprP6gr7EYeRxcZ+lRGkv4cgK4m7OE4XPcG5R2q2gcBtUTH2Zj+3PzMwPJ+ID4XSh50FHdpUO2LlYVtrs1/9xk4Bv/jZ8dGpfEJA+uNgM81nIajEIdtgkeFTLBwo0l8dUQJf31PYSkcVXIQRg2J9yV7jiBJsA5iSngDDPdQqeXYzYCZOgJXnGmfwQE1tm200256dUr9wlPDVBOqhzCD3In0xS44CH/D0FqEewkelSYiCGTni4tBEf+gpeAc5vvOh8TGdgPdWw0gOv+IdeSxOG5SeOmovxeIpkBO8xVymmY0ntcot4p5lnZPYg01J25BBy8jXhbo3Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ae3NYYj8Mb/IQHqia9NXSz9x6kihsiTvFqB6gqoxl+Y=; b=YiRGxyniUchf+35Ky9PgDMi6fyeFdOXFSrLRygi9TcWl9bL3ULRhK0t1pLGNamW7oQ6Iju+UYPAsXPVRzKATGkKmIAGETrGfFY7yX6tEu8mE8nruiAv+KNCtLptfmcF5hj4t2qwdBR0OXyiLYyUbZYGUpLbiXY8H2u9hQ0uz3kE3Hix0FoRVPQxY9M3LP4pKTi7g5Y8hSXUloDTi5Lm6pP9QD2U3exvhxh/JR+6Xn4IfUsMYbiBxIS+e6JdRoHh4FBVIBOHEe9SK7OZgmL0+HkALBsjM2duTXLdr1ZRTvR4xsIMZOqL986XC50MXWsUujtM6+XrnzRCEgB+KY9zUrQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.118.232) smtp.rcpttodomain=amd.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ae3NYYj8Mb/IQHqia9NXSz9x6kihsiTvFqB6gqoxl+Y=; b=fN/uYz3L18GiZYSl1IoPjcb77WbgktvFW0/58lt42W1fmPBcxfpUY1u3Xtr6ynO45MbL0Bz3Rmvk/uFgEHQ2Xf2mwAj9M8E8nyAzNzQUSmoXzg0YF9uj9zk/Gors1/anZDKT+LA6OrpUHjDXGhknV3ct6CSXOWzbCjtXaPHwir5xFxErxJqa4gZVhsunv4/L+sMJRE/kWRt4zsSpeIhSvBAYPwez6HQf1H8ziAC9j9hFYKif9Nsio2KRLnnjO1VN0PsKyxUKJYU2jxIGuUwqGuVbsJxJ5QoM7X6jS5PGghktpC6JeWugJ8wVBdDGsh6CdmWSR6iiXzAVq283f04Dpw== Received: from SA9PR13CA0126.namprd13.prod.outlook.com (2603:10b6:806:27::11) by PH7PR12MB6561.namprd12.prod.outlook.com (2603:10b6:510:213::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8093.25; Wed, 30 Oct 2024 21:35:22 +0000 Received: from SN1PEPF0002BA4E.namprd03.prod.outlook.com (2603:10b6:806:27:cafe::2d) by SA9PR13CA0126.outlook.office365.com (2603:10b6:806:27::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8137.7 via Frontend Transport; Wed, 30 Oct 2024 21:35:22 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.118.232) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.118.232 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.118.232; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.118.232) by SN1PEPF0002BA4E.mail.protection.outlook.com (10.167.242.71) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8114.16 via Frontend Transport; Wed, 30 Oct 2024 21:35:21 +0000 Received: from drhqmail203.nvidia.com (10.126.190.182) by mail.nvidia.com (10.127.129.5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Wed, 30 Oct 2024 14:35:07 -0700 Received: from drhqmail202.nvidia.com (10.126.190.181) by drhqmail203.nvidia.com (10.126.190.182) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Wed, 30 Oct 2024 14:35:07 -0700 Received: from Asurada-Nvidia.nvidia.com (10.127.8.12) by mail.nvidia.com (10.126.190.181) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Wed, 30 Oct 2024 14:35:06 -0700 From: Nicolin Chen To: , , CC: , , , , , , , , , , , , , , , , , , , , Subject: [PATCH v6 08/13] iommufd/selftest: Add container_of helpers Date: Wed, 30 Oct 2024 14:34:34 -0700 Message-ID: X-Mailer: git-send-email 2.43.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SN1PEPF0002BA4E:EE_|PH7PR12MB6561:EE_ X-MS-Office365-Filtering-Correlation-Id: 280cd6c8-7338-48c8-7066-08dcf92ac21e X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|1800799024|82310400026|36860700013; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?Bp8OqrSLIedITnY0VY8QtLyDQA77iS1mw6V2mewU8eUOWNriNgabARMQXASn?= =?us-ascii?Q?ojhknmWVG5ZG/mM8sNqTvGg28TJkr0BEmETcjAXdXr+tS7BO4oK0wtGSdGFr?= =?us-ascii?Q?BcDRGz/5E3oURHMm6rQU82k7OKmcRMnqkCHYQFkAI7TI5HvOmVspjTLZs+ue?= =?us-ascii?Q?Ob6OI3l5HJ0oWY0OdaogFT5g4kI6FwCAvpJh77BKqg3UYqEOV5G3gphEh138?= =?us-ascii?Q?3qwtudKJTZmxQq4EXjk20QiuZW8EcG07cukZN0l8wJ4BhEa+u20BmxMDk0iz?= =?us-ascii?Q?dm8+IXN+C5G1xhg5e1kCqa5BDzbcWXo7f4IHqNHZY39ZyP1AXyRSEy+gzgYw?= =?us-ascii?Q?kPhSa8VCK0/yDvFtmkZYeefTcoxpbtYmo1ARNBn+T2YiW2wU73ne+IxbvOIB?= =?us-ascii?Q?X1Ef0tHLQb+OuZifc5Zhhqhgdc666HOwKx+lSIHGlXjwzc4Ip6w2NgT0xQjT?= =?us-ascii?Q?G7t5CfqUan4CiHtOLONQf0MItr/Ephbb35mjXZTdySruOUArflIKTt4vpUW7?= =?us-ascii?Q?/y6J1vru0IxUEIh6t3A3JgIlQDwTiMTRuMeS3eaxjcc3R6HscVrXsqikSSM7?= =?us-ascii?Q?4x0CubqRp2itjDmxcELdtY6diA9N+12JmrrzDNYWpr0BRQoMwhOSWhdL7d/V?= =?us-ascii?Q?iw4ZL19BmFvjFav8Q18RhfALiOSkp5AIfUt9uaFwqXLE5kuJF9Qv0N9+8qWo?= =?us-ascii?Q?2U0uvQ78ztJ6fKOfFJY/ExhxK2w+9vF8G/0jnvK+XiARfWo95nAMIttdnCVs?= =?us-ascii?Q?LpR3LfmDjMA6nV/Lt/K7zuwEvRWmlX7QFAMEK8vmxD/VQJUnVvoDcOruuJNX?= =?us-ascii?Q?j8rDHyrSaKfXCZ7a5QPOHIrjW6K/NLDXnfajl+CiDOUvkW2Umw7zCeBnblUG?= =?us-ascii?Q?x9BCzhzof4N4MvIjZXVtmlYpXSsZ8QNl9VHTrcnPG3q+azbq8VyW6MvZQ5Mf?= =?us-ascii?Q?TVAr2rnpK39ucCs278esumES5k1qmO7hrVfjk3cthAqX6UsUWwEdhXhHu6cP?= =?us-ascii?Q?qtsSGycllMNrAil7/8QowgMoOZ4SRXbxI6E4Tfg3oe3bULSnRDh0y8CigSMA?= =?us-ascii?Q?1VvNRjdjxs74TiuQbnl95gRCsjKv+xApXxMmYvkXySq4zjuKYpz70C+qPwpI?= =?us-ascii?Q?MZ6LNwmjG6BGJ2yyM3qvxTdu9bfzjLyhJ7GInacCHNc8k6CZO8Q5H/uw1JvO?= =?us-ascii?Q?VocoPrJqoesdaFm8IHgre7Z567uBVl59PlKZteenya4hXzHTTnnEOexVPXfb?= =?us-ascii?Q?0CzZO1hiGCv7Gg7XfR59ZjNuXf7bdP7kPB8SSGq/YbrvRY8waLCTUgFtENns?= =?us-ascii?Q?4u3mOP8cPNV99xCeaIZkT+3zzNCRDfqeqOEvFMLW0EZks7UvFLSW1wtOYxoR?= =?us-ascii?Q?+dlUuYicIXtQRikjmj3UX2Vt4pm/l92fOXsPqnW3V/UMe5Ot+Q=3D=3D?= X-Forefront-Antispam-Report: CIP:216.228.118.232;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc7edge1.nvidia.com;CAT:NONE;SFS:(13230040)(7416014)(376014)(1800799024)(82310400026)(36860700013);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Oct 2024 21:35:21.9235 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 280cd6c8-7338-48c8-7066-08dcf92ac21e X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.118.232];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: SN1PEPF0002BA4E.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB6561 Content-Type: text/plain; charset="utf-8" Use these inline helpers to shorten those container_of lines. Note that one of them goes back and forth between iommu_domain and mock_iommu_domain, which isn't necessary. So drop its container_of. Reviewed-by: Kevin Tian Reviewed-by: Jason Gunthorpe Signed-off-by: Nicolin Chen --- drivers/iommu/iommufd/selftest.c | 75 ++++++++++++++++++-------------- 1 file changed, 42 insertions(+), 33 deletions(-) diff --git a/drivers/iommu/iommufd/selftest.c b/drivers/iommu/iommufd/selft= est.c index 540437be168a..322e57ff3605 100644 --- a/drivers/iommu/iommufd/selftest.c +++ b/drivers/iommu/iommufd/selftest.c @@ -126,12 +126,24 @@ struct mock_iommu_domain { struct xarray pfns; }; =20 +static inline struct mock_iommu_domain * +to_mock_domain(struct iommu_domain *domain) +{ + return container_of(domain, struct mock_iommu_domain, domain); +} + struct mock_iommu_domain_nested { struct iommu_domain domain; struct mock_iommu_domain *parent; u32 iotlb[MOCK_NESTED_DOMAIN_IOTLB_NUM]; }; =20 +static inline struct mock_iommu_domain_nested * +to_mock_nested(struct iommu_domain *domain) +{ + return container_of(domain, struct mock_iommu_domain_nested, domain); +} + enum selftest_obj_type { TYPE_IDEV, }; @@ -142,6 +154,11 @@ struct mock_dev { int id; }; =20 +static inline struct mock_dev *to_mock_dev(struct device *dev) +{ + return container_of(dev, struct mock_dev, dev); +} + struct selftest_obj { struct iommufd_object obj; enum selftest_obj_type type; @@ -155,10 +172,15 @@ struct selftest_obj { }; }; =20 +static inline struct selftest_obj *to_selftest_obj(struct iommufd_object *= obj) +{ + return container_of(obj, struct selftest_obj, obj); +} + static int mock_domain_nop_attach(struct iommu_domain *domain, struct device *dev) { - struct mock_dev *mdev =3D container_of(dev, struct mock_dev, dev); + struct mock_dev *mdev =3D to_mock_dev(dev); =20 if (domain->dirty_ops && (mdev->flags & MOCK_FLAGS_DEVICE_NO_DIRTY)) return -EINVAL; @@ -193,8 +215,7 @@ static void *mock_domain_hw_info(struct device *dev, u3= 2 *length, u32 *type) static int mock_domain_set_dirty_tracking(struct iommu_domain *domain, bool enable) { - struct mock_iommu_domain *mock =3D - container_of(domain, struct mock_iommu_domain, domain); + struct mock_iommu_domain *mock =3D to_mock_domain(domain); unsigned long flags =3D mock->flags; =20 if (enable && !domain->dirty_ops) @@ -243,8 +264,7 @@ static int mock_domain_read_and_clear_dirty(struct iomm= u_domain *domain, unsigned long flags, struct iommu_dirty_bitmap *dirty) { - struct mock_iommu_domain *mock =3D - container_of(domain, struct mock_iommu_domain, domain); + struct mock_iommu_domain *mock =3D to_mock_domain(domain); unsigned long end =3D iova + size; void *ent; =20 @@ -281,7 +301,7 @@ static const struct iommu_dirty_ops dirty_ops =3D { =20 static struct iommu_domain *mock_domain_alloc_paging(struct device *dev) { - struct mock_dev *mdev =3D container_of(dev, struct mock_dev, dev); + struct mock_dev *mdev =3D to_mock_dev(dev); struct mock_iommu_domain *mock; =20 mock =3D kzalloc(sizeof(*mock), GFP_KERNEL); @@ -327,7 +347,7 @@ mock_domain_alloc_user(struct device *dev, u32 flags, =20 /* must be mock_domain */ if (!parent) { - struct mock_dev *mdev =3D container_of(dev, struct mock_dev, dev); + struct mock_dev *mdev =3D to_mock_dev(dev); bool has_dirty_flag =3D flags & IOMMU_HWPT_ALLOC_DIRTY_TRACKING; bool no_dirty_ops =3D mdev->flags & MOCK_FLAGS_DEVICE_NO_DIRTY; struct iommu_domain *domain; @@ -341,8 +361,7 @@ mock_domain_alloc_user(struct device *dev, u32 flags, if (!domain) return ERR_PTR(-ENOMEM); if (has_dirty_flag) - container_of(domain, struct mock_iommu_domain, domain) - ->domain.dirty_ops =3D &dirty_ops; + domain->dirty_ops =3D &dirty_ops; return domain; } =20 @@ -352,7 +371,7 @@ mock_domain_alloc_user(struct device *dev, u32 flags, if (!parent || parent->ops !=3D mock_ops.default_domain_ops) return ERR_PTR(-EINVAL); =20 - mock_parent =3D container_of(parent, struct mock_iommu_domain, domain); + mock_parent =3D to_mock_domain(parent); if (!mock_parent) return ERR_PTR(-EINVAL); =20 @@ -366,8 +385,7 @@ mock_domain_alloc_user(struct device *dev, u32 flags, =20 static void mock_domain_free(struct iommu_domain *domain) { - struct mock_iommu_domain *mock =3D - container_of(domain, struct mock_iommu_domain, domain); + struct mock_iommu_domain *mock =3D to_mock_domain(domain); =20 WARN_ON(!xa_empty(&mock->pfns)); kfree(mock); @@ -378,8 +396,7 @@ static int mock_domain_map_pages(struct iommu_domain *d= omain, size_t pgsize, size_t pgcount, int prot, gfp_t gfp, size_t *mapped) { - struct mock_iommu_domain *mock =3D - container_of(domain, struct mock_iommu_domain, domain); + struct mock_iommu_domain *mock =3D to_mock_domain(domain); unsigned long flags =3D MOCK_PFN_START_IOVA; unsigned long start_iova =3D iova; =20 @@ -430,8 +447,7 @@ static size_t mock_domain_unmap_pages(struct iommu_doma= in *domain, size_t pgcount, struct iommu_iotlb_gather *iotlb_gather) { - struct mock_iommu_domain *mock =3D - container_of(domain, struct mock_iommu_domain, domain); + struct mock_iommu_domain *mock =3D to_mock_domain(domain); bool first =3D true; size_t ret =3D 0; void *ent; @@ -479,8 +495,7 @@ static size_t mock_domain_unmap_pages(struct iommu_doma= in *domain, static phys_addr_t mock_domain_iova_to_phys(struct iommu_domain *domain, dma_addr_t iova) { - struct mock_iommu_domain *mock =3D - container_of(domain, struct mock_iommu_domain, domain); + struct mock_iommu_domain *mock =3D to_mock_domain(domain); void *ent; =20 WARN_ON(iova % MOCK_IO_PAGE_SIZE); @@ -491,7 +506,7 @@ static phys_addr_t mock_domain_iova_to_phys(struct iomm= u_domain *domain, =20 static bool mock_domain_capable(struct device *dev, enum iommu_cap cap) { - struct mock_dev *mdev =3D container_of(dev, struct mock_dev, dev); + struct mock_dev *mdev =3D to_mock_dev(dev); =20 switch (cap) { case IOMMU_CAP_CACHE_COHERENCY: @@ -571,18 +586,14 @@ static const struct iommu_ops mock_ops =3D { =20 static void mock_domain_free_nested(struct iommu_domain *domain) { - struct mock_iommu_domain_nested *mock_nested =3D - container_of(domain, struct mock_iommu_domain_nested, domain); - - kfree(mock_nested); + kfree(to_mock_nested(domain)); } =20 static int mock_domain_cache_invalidate_user(struct iommu_domain *domain, struct iommu_user_data_array *array) { - struct mock_iommu_domain_nested *mock_nested =3D - container_of(domain, struct mock_iommu_domain_nested, domain); + struct mock_iommu_domain_nested *mock_nested =3D to_mock_nested(domain); struct iommu_hwpt_invalidate_selftest inv; u32 processed =3D 0; int i =3D 0, j; @@ -657,7 +668,7 @@ get_md_pagetable(struct iommufd_ucmd *ucmd, u32 mockpt_= id, iommufd_put_object(ucmd->ictx, &hwpt->obj); return ERR_PTR(-EINVAL); } - *mock =3D container_of(hwpt->domain, struct mock_iommu_domain, domain); + *mock =3D to_mock_domain(hwpt->domain); return hwpt; } =20 @@ -675,14 +686,13 @@ get_md_pagetable_nested(struct iommufd_ucmd *ucmd, u3= 2 mockpt_id, iommufd_put_object(ucmd->ictx, &hwpt->obj); return ERR_PTR(-EINVAL); } - *mock_nested =3D container_of(hwpt->domain, - struct mock_iommu_domain_nested, domain); + *mock_nested =3D to_mock_nested(hwpt->domain); return hwpt; } =20 static void mock_dev_release(struct device *dev) { - struct mock_dev *mdev =3D container_of(dev, struct mock_dev, dev); + struct mock_dev *mdev =3D to_mock_dev(dev); =20 ida_free(&mock_dev_ida, mdev->id); kfree(mdev); @@ -813,7 +823,7 @@ static int iommufd_test_mock_domain_replace(struct iomm= ufd_ucmd *ucmd, if (IS_ERR(dev_obj)) return PTR_ERR(dev_obj); =20 - sobj =3D container_of(dev_obj, struct selftest_obj, obj); + sobj =3D to_selftest_obj(dev_obj); if (sobj->type !=3D TYPE_IDEV) { rc =3D -EINVAL; goto out_dev_obj; @@ -951,8 +961,7 @@ static int iommufd_test_md_check_iotlb(struct iommufd_u= cmd *ucmd, if (IS_ERR(hwpt)) return PTR_ERR(hwpt); =20 - mock_nested =3D container_of(hwpt->domain, - struct mock_iommu_domain_nested, domain); + mock_nested =3D to_mock_nested(hwpt->domain); =20 if (iotlb_id > MOCK_NESTED_DOMAIN_IOTLB_ID_MAX || mock_nested->iotlb[iotlb_id] !=3D iotlb) @@ -1431,7 +1440,7 @@ static int iommufd_test_trigger_iopf(struct iommufd_u= cmd *ucmd, =20 void iommufd_selftest_destroy(struct iommufd_object *obj) { - struct selftest_obj *sobj =3D container_of(obj, struct selftest_obj, obj); + struct selftest_obj *sobj =3D to_selftest_obj(obj); =20 switch (sobj->type) { case TYPE_IDEV: --=20 2.43.0 From nobody Mon Nov 25 00:39:33 2024 Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2080.outbound.protection.outlook.com [40.107.244.80]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 27AC22296D8; Wed, 30 Oct 2024 21:35:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.244.80 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730324134; cv=fail; b=AVq1yYbvVtg4o9vNkObsN7cMvV3zBCc3IGEOCm4ncQGZEoe/um7J/2yE6djDrz1ObBDIk89HIvr2Eq5jjSJP5vjH4WgiOgE63huUAY3BeNWvv2RTTmac+0Xugu1qmjBOKRPwnbuMpP3qWT8O3Iw1RCqR3VldMoW2SNT6Glg5IuA= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730324134; c=relaxed/simple; bh=gGhsKUAuS3tN+ZVvOMxcUVmYnU4xo6EcIIjj8fa0HYQ=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=dpfSSAQh6by+AOxAW45IuHRv7eS9Z2QRvyX8gJuG3Rn7CYDnfrL3yIjZPkZmvOqbV0bLrrFPRQUDkXifIt/Ox4CmdYND6NRCLzKTI+53gqMLk0Y7JUuktRmSeaBhTOM+scsVIgeLEP2Abnp9waoZxiuZtQ840iVjsFGn8v2KJEg= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=rMWuMLpB; arc=fail smtp.client-ip=40.107.244.80 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="rMWuMLpB" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=CSsQwhmyyAPxgBBMZFe/qAOljT9d1hEE7ERciKGrxjqMT9hsA3WXmae/CwOnbFZyNzfFmCWwm0cYK8kphsjk4mEtIQ3miRN0iW6VjxzBg2SZs1zP+TxH42J4B3yGUxFdtsQG307Fw6VoTibujDKbSNRTv1KIG4ZItf9uuscuBcXknAAuDL7I1ZORvpFwVFolb4ZcM+1uefOKvnZFun0DJ6Xt1/9dLku3RuTgi3nIz4uVQtLVCssP0sIqqa8U6vcIPjoODTMDCFiKoQpfbXYTNgjGmBKgBao41SdLviIc1nsrqhEusro7PdpdAWI4pquUPk3iznG3Wt2ZJe63Uj4s+Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=PUbjJXnib9EiC6H9vV18C3nn1EiLRyVYUNzy8Y9BqNA=; b=EtqxXhbqAmwUl4Rkt5VWumHZkxCEXpbgL2ucZdcyRcEUcM56rAQcmzRFwh9m5H1lpyUF1O1aaXm9xYAMXC57ZR1Nt47HdwQIy3HlDK58FQcFzPyLwM/HSmXHw9f5wdNTar+Gb1qJAvxZs6QW9SVz4cMeIDsKv6FdSCD0urVJ/tAXNw/UHi5kZlX6V4KPT++ksmS9aRpDefzeCD/yjoBcsG+khjhF6JSd5k74DlfA/SmOAfftrTI29u8jwD4S75oQUtRLgTn8+yrlFVGP07318FY51Dt6j/fXyuzwnIe94r1DmJLgu8sfhyF38KQkm38ZfjHz0fuUM6uq6yBHXvTRQg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.118.232) smtp.rcpttodomain=amd.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=PUbjJXnib9EiC6H9vV18C3nn1EiLRyVYUNzy8Y9BqNA=; b=rMWuMLpBYFzZLmYmpN1nfeh6M37A0UlcQw5YnyZEU6ZdUEo0Nc+0qXSUqpWZ9M6iQg9x0CD83TU+6nf+np7DMKteA4sbOZ3hKG6M08qkzYVttOrEc/aNQiCB3it2VPD+4nuwtNE8OP4y1OOl6leasdT3OUbI9u50mjOKF7L3baD90Qq6cQ63L/Q2rapnz/zKOTd1p8+qLuK/TxHtEqiFvXxalnBGpIXLrNBDkiixhuE4GZcqwHOE0yeCQRAFbE0t8/gt1zWfssXkHBVG4g9wGxYEa6LSMITgaHTAm2m8+Vb0jBaAxpBoIMBATi3kzB8pXir59JRxIrs+xWHplpChIQ== Received: from PH0P220CA0012.NAMP220.PROD.OUTLOOK.COM (2603:10b6:510:d3::35) by CY5PR12MB6370.namprd12.prod.outlook.com (2603:10b6:930:20::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8093.27; Wed, 30 Oct 2024 21:35:27 +0000 Received: from SN1PEPF0002BA4C.namprd03.prod.outlook.com (2603:10b6:510:d3:cafe::3d) by PH0P220CA0012.outlook.office365.com (2603:10b6:510:d3::35) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8114.20 via Frontend Transport; Wed, 30 Oct 2024 21:35:27 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.118.232) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.118.232 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.118.232; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.118.232) by SN1PEPF0002BA4C.mail.protection.outlook.com (10.167.242.69) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8114.16 via Frontend Transport; Wed, 30 Oct 2024 21:35:27 +0000 Received: from drhqmail203.nvidia.com (10.126.190.182) by mail.nvidia.com (10.127.129.5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Wed, 30 Oct 2024 14:35:08 -0700 Received: from drhqmail202.nvidia.com (10.126.190.181) by drhqmail203.nvidia.com (10.126.190.182) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Wed, 30 Oct 2024 14:35:08 -0700 Received: from Asurada-Nvidia.nvidia.com (10.127.8.12) by mail.nvidia.com (10.126.190.181) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Wed, 30 Oct 2024 14:35:07 -0700 From: Nicolin Chen To: , , CC: , , , , , , , , , , , , , , , , , , , , Subject: [PATCH v6 09/13] iommufd/selftest: Prepare for mock_viommu_alloc_domain_nested() Date: Wed, 30 Oct 2024 14:34:35 -0700 Message-ID: <5b8362c4b2ee64cc777a5b223661e7202a3fbae9.1730313237.git.nicolinc@nvidia.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SN1PEPF0002BA4C:EE_|CY5PR12MB6370:EE_ X-MS-Office365-Filtering-Correlation-Id: 1f37c0f5-627a-4e21-5ec5-08dcf92ac554 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|376014|7416014|82310400026|36860700013; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?wHlw3G4IZixTiXaet6OLK5t7slf3LFYSH1CeWAGDibmjGdffT+QETs5SllMB?= =?us-ascii?Q?GDXQE1cU/aZFVzDr0dqpfRGaq6gESB7C1LjPbg7Dw17e+YaXJ9HN1T9tXsUX?= =?us-ascii?Q?xBiRxS1MQcDCjCaerPEropOHDLBbV4Y98ELO9CUXqs346JV7o5UCuVBJAN4L?= =?us-ascii?Q?Tjb+wF1j8trwgBMzOHbiOtCanmUZJfQ4DMCJ2+gFg7Y0guV3Os3ANQyOhKnK?= =?us-ascii?Q?nhtX1bUoXEJ3eiV82cPz7SokRSoSVUcfAWjeL1/QBQepdaftIa5+QIKiLxsD?= =?us-ascii?Q?Dg/bDireCrj5BxKU93yFy14hUrbifTvhXYbj1YMUlPzBSpzgzDv0bjlKIm6H?= =?us-ascii?Q?VlVFboK2zAnQSvUkvZ+MIDs7j1IN8VWgwwmxn87OGZ4Wnqt6/W6VBcIwu594?= =?us-ascii?Q?ycDbbzuekeUexbv/s13CxNiLTwmKwzmh+T/JYvtSXKx/JbGoBd7WeEBYt+fC?= =?us-ascii?Q?KSRED7bqqkAvzagHdU1vznKKo6J3GiRCLXc/xujL0qcyrGxgK+IbIyG+ZvhM?= =?us-ascii?Q?OMZRuyf5nuFgga6WSE9TGT6LTo+ZY97Hoi//T4rEPJ87LIOoO0lJDjFTmTj4?= =?us-ascii?Q?duEyJPPLktfDch1Y44k6ERPsQKh0rrRpX79bBqmcg5ip6kQ3ddD07epWTgQ8?= =?us-ascii?Q?KmGBnCe24W/zR+R2IDWIOpbfAP4+U0B68x20uvCUOeM4FBuIALmAOvubqFFv?= =?us-ascii?Q?GyBHUFQGlPJhEOIjT7cLYoD4yOWOGRaVGgPRPqgGa/g8eeoS8KIKTqVc1bud?= =?us-ascii?Q?N8dhNflNaTRgdj3GnEWQLupoCGgv4G0WzKGhEe+BzndRGc4HyR6bTAO+HzRj?= =?us-ascii?Q?opFpIV3DgYkSTlOvnyezz82i0VIIfVEgA/BOONW7QMgVwWUyUpXnN0sEC0Mb?= =?us-ascii?Q?1PDtkM54CPU7olTm678HsH9WV1MhE+5JjafIVVW0FACNMz1JvJCJMjIX/H25?= =?us-ascii?Q?EH+JBRdvXmIL94IL7ZW68rhHsjQHI+mTE0l+fqxYfYYVNXq8XrHfK969q6D+?= =?us-ascii?Q?z0KYtk55L0WfASgobaDt4eNXFLp5hFv/0FP6EyWqm5dF6PntcjaFanx8AI1O?= =?us-ascii?Q?USrmAvhyfLaKieXCPUjBgu2sm7cqiamQlGge3aeRXZ3rT03eXInpuSkhEpSX?= =?us-ascii?Q?ExTlpx/PSPkJqKPTmguJyO9NqFXlIpqZTJYGj4KcFUvjZ/ZNDuP+fjzenCU6?= =?us-ascii?Q?/RvnJkbKnO9AwuVwPX85xvKM+4a7YXLZQrex0R99j4RVT64hc0ymasQtp3jJ?= =?us-ascii?Q?oR/hYh/3YHwGyEzGzcpFiUWv4debERj9FGUOP4ddiHzjEXHlAel98Ev/1GSy?= =?us-ascii?Q?QAotdfNoEEbysaUWr1eyTLxMQvbalHy3BWGdQoDuef4Scfwac0Zf7iwt/tCk?= =?us-ascii?Q?31bYSAWRGTA3tRrKkrZbEiAciJUhAP4oXou+pmZSU2L078d4VQ=3D=3D?= X-Forefront-Antispam-Report: CIP:216.228.118.232;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc7edge1.nvidia.com;CAT:NONE;SFS:(13230040)(1800799024)(376014)(7416014)(82310400026)(36860700013);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Oct 2024 21:35:27.3101 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 1f37c0f5-627a-4e21-5ec5-08dcf92ac554 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.118.232];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: SN1PEPF0002BA4C.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY5PR12MB6370 Content-Type: text/plain; charset="utf-8" A nested domain now can be allocated for a parent domain or for a vIOMMU object. Rework the existing allocators to prepare for the latter case. Reviewed-by: Kevin Tian Reviewed-by: Jason Gunthorpe Signed-off-by: Nicolin Chen --- drivers/iommu/iommufd/selftest.c | 89 ++++++++++++++++++-------------- 1 file changed, 50 insertions(+), 39 deletions(-) diff --git a/drivers/iommu/iommufd/selftest.c b/drivers/iommu/iommufd/selft= est.c index 322e57ff3605..92d753985640 100644 --- a/drivers/iommu/iommufd/selftest.c +++ b/drivers/iommu/iommufd/selftest.c @@ -318,55 +318,39 @@ static struct iommu_domain *mock_domain_alloc_paging(= struct device *dev) return &mock->domain; } =20 -static struct iommu_domain * -__mock_domain_alloc_nested(struct mock_iommu_domain *mock_parent, - const struct iommu_hwpt_selftest *user_cfg) +static struct mock_iommu_domain_nested * +__mock_domain_alloc_nested(const struct iommu_user_data *user_data) { struct mock_iommu_domain_nested *mock_nested; - int i; + struct iommu_hwpt_selftest user_cfg; + int rc, i; + + if (user_data->type !=3D IOMMU_HWPT_DATA_SELFTEST) + return ERR_PTR(-EOPNOTSUPP); + + rc =3D iommu_copy_struct_from_user(&user_cfg, user_data, + IOMMU_HWPT_DATA_SELFTEST, iotlb); + if (rc) + return ERR_PTR(rc); =20 mock_nested =3D kzalloc(sizeof(*mock_nested), GFP_KERNEL); if (!mock_nested) return ERR_PTR(-ENOMEM); - mock_nested->parent =3D mock_parent; mock_nested->domain.ops =3D &domain_nested_ops; mock_nested->domain.type =3D IOMMU_DOMAIN_NESTED; for (i =3D 0; i < MOCK_NESTED_DOMAIN_IOTLB_NUM; i++) - mock_nested->iotlb[i] =3D user_cfg->iotlb; - return &mock_nested->domain; + mock_nested->iotlb[i] =3D user_cfg.iotlb; + return mock_nested; } =20 static struct iommu_domain * -mock_domain_alloc_user(struct device *dev, u32 flags, - struct iommu_domain *parent, - const struct iommu_user_data *user_data) +mock_domain_alloc_nested(struct iommu_domain *parent, u32 flags, + const struct iommu_user_data *user_data) { + struct mock_iommu_domain_nested *mock_nested; struct mock_iommu_domain *mock_parent; - struct iommu_hwpt_selftest user_cfg; - int rc; - - /* must be mock_domain */ - if (!parent) { - struct mock_dev *mdev =3D to_mock_dev(dev); - bool has_dirty_flag =3D flags & IOMMU_HWPT_ALLOC_DIRTY_TRACKING; - bool no_dirty_ops =3D mdev->flags & MOCK_FLAGS_DEVICE_NO_DIRTY; - struct iommu_domain *domain; - - if (flags & (~(IOMMU_HWPT_ALLOC_NEST_PARENT | - IOMMU_HWPT_ALLOC_DIRTY_TRACKING))) - return ERR_PTR(-EOPNOTSUPP); - if (user_data || (has_dirty_flag && no_dirty_ops)) - return ERR_PTR(-EOPNOTSUPP); - domain =3D mock_domain_alloc_paging(dev); - if (!domain) - return ERR_PTR(-ENOMEM); - if (has_dirty_flag) - domain->dirty_ops =3D &dirty_ops; - return domain; - } =20 - /* must be mock_domain_nested */ - if (user_data->type !=3D IOMMU_HWPT_DATA_SELFTEST || flags) + if (flags) return ERR_PTR(-EOPNOTSUPP); if (!parent || parent->ops !=3D mock_ops.default_domain_ops) return ERR_PTR(-EINVAL); @@ -375,12 +359,39 @@ mock_domain_alloc_user(struct device *dev, u32 flags, if (!mock_parent) return ERR_PTR(-EINVAL); =20 - rc =3D iommu_copy_struct_from_user(&user_cfg, user_data, - IOMMU_HWPT_DATA_SELFTEST, iotlb); - if (rc) - return ERR_PTR(rc); + mock_nested =3D __mock_domain_alloc_nested(user_data); + if (IS_ERR(mock_nested)) + return ERR_CAST(mock_nested); + mock_nested->parent =3D mock_parent; + return &mock_nested->domain; +} + +static struct iommu_domain * +mock_domain_alloc_user(struct device *dev, u32 flags, + struct iommu_domain *parent, + const struct iommu_user_data *user_data) +{ + bool has_dirty_flag =3D flags & IOMMU_HWPT_ALLOC_DIRTY_TRACKING; + const u32 PAGING_FLAGS =3D IOMMU_HWPT_ALLOC_DIRTY_TRACKING | + IOMMU_HWPT_ALLOC_NEST_PARENT; + bool no_dirty_ops =3D to_mock_dev(dev)->flags & + MOCK_FLAGS_DEVICE_NO_DIRTY; + struct iommu_domain *domain; + + if (parent) + return mock_domain_alloc_nested(parent, flags, user_data); =20 - return __mock_domain_alloc_nested(mock_parent, &user_cfg); + if (user_data) + return ERR_PTR(-EOPNOTSUPP); + if ((flags & ~PAGING_FLAGS) || (has_dirty_flag && no_dirty_ops)) + return ERR_PTR(-EOPNOTSUPP); + + domain =3D mock_domain_alloc_paging(dev); + if (!domain) + return ERR_PTR(-ENOMEM); + if (has_dirty_flag) + domain->dirty_ops =3D &dirty_ops; + return domain; } =20 static void mock_domain_free(struct iommu_domain *domain) --=20 2.43.0 From nobody Mon Nov 25 00:39:33 2024 Received: from NAM04-MW2-obe.outbound.protection.outlook.com (mail-mw2nam04on2085.outbound.protection.outlook.com [40.107.101.85]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F0FC6229B22; Wed, 30 Oct 2024 21:35:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.101.85 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730324138; cv=fail; b=gKsWn29mfB3YnqzYzkxIP2IdHNq90rU5PSMMGd8WeZ+YdmCGUT/umnOV3OE8/zhK6aTTOH6qeBKmEjV7wcLh9kA0VPS7phoXHNW1UcQPH63cVo1G44EQ0IZYy7WaV/M1KkuILPLCzBOyBbC+n41vyAXuQVq/AkU4tWcZ9WMl8NA= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730324138; c=relaxed/simple; bh=+iScv+JPH5qZd8WJf8hN+2Zy1gN2CgRyr1NqLkHDIe8=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=Lt3qeDqoQNetI/Qk68h164kyEO7KpR3dRRU6QDPOan3lRe/g8y6pzxv+/B3/cim5Xw0PSSkhPCtEF7EHAkzY6GHZ2MtPJ2IZ5oMCwhA5mou5xNosDpjmSX7JGw+hZADhik59JR4NdchlbtUJKyU6XlxdSxvGpfHqSDlOeGUscCs= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=qNDtXNc5; arc=fail smtp.client-ip=40.107.101.85 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="qNDtXNc5" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=QB47W5lLwDpKOSha1lhoCj4wnKRxRfDFRYDA4XzU8TwZg70qpPu1qXgR9lZuwjSKHbmrJprzGRge8YCsgLhM1KR0LlSNdg7YEVlDFlOQsGNCOaSrDnFI/NyTq+FWXtqQYH+j1dOqkjisLH1Yo5zifG9VZ8PcJ3YNpQR4785LWPDVA+bP7u+e35m5r0qf689yYWhvyJwJ2wQppUnZNrYcGJDY+rg4hp+aXpe7CVnnTFg9ncSlcQUx4gdhfNBrEAGcclXeucoY34+SSaITR1UMjX36gNPjUpBdkfGzYmggHqrSzi5N1nOo1Qrvy3mgUYP3vy2h//K6Lh0Z6hPVOHNoLg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=37CS+mIDQfL06j1QvVP3AC2sA04PpVCXgVs/EoL/36Q=; b=qbtWee8QIFpdzZZYCbtbKjAPXmzO040cOvXiKvPc8VNsUBenZIK8fmvNSWfWnDjPo97bmMVl5u465gC2oEpYVhGCb4Z3SN0NRFn9jVDfQSkRzbv/L/wMtONHStS+qykkBlaZxuYdsgvanSytOgdwpAqW8Ccm5iIbKJ2ZRq/T7QJe/XKyZ0I/Q7J40GanWYYmJ6a5IkVqh7zmq0QCp6zDCFA1L4jmQ0NkQLmtRRgpsPD17SFeyqjCjW5Cr6PicLol3hr64e/KETuawESVcZvrItQD1Ge+b/Gn0W0IPQY5LZ+RbSp0kEe7/HeMjSz3XAuh823CJXLeX3cOSZV7Hn0BCg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.118.233) smtp.rcpttodomain=amd.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=37CS+mIDQfL06j1QvVP3AC2sA04PpVCXgVs/EoL/36Q=; b=qNDtXNc5Oet2kfcwNhBSmUkQ229LG+dp92LGI9r7jp/TLP3O2xkafbBLNruta0J3lHJZD0xKeckmUZH3NdtzQOx+ppv6RNJygnDbXxROXnsrC2V0hFoPrhJeSpnsS1RjySZBtaMNY3S/TgI9psqOFqD/qSezHTV5lgG1v7oPaDfL3K9DuAixuYnf+uUqlttdndWpkS7fKj7XULuKdDr3H+F6p7nc4PQoAapidjSPxnBDF36ebZUhqJHcxvg7TamhJHPSZETzrNVdETkodzD0uXwL6V+XzEm1iqeVJyP1XvZ2lKQ2N9Gx5FWOW96QNFas20FjNesdPAwRu2EgOS/uEg== Received: from CH0PR04CA0098.namprd04.prod.outlook.com (2603:10b6:610:75::13) by SN7PR12MB7155.namprd12.prod.outlook.com (2603:10b6:806:2a6::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8093.25; Wed, 30 Oct 2024 21:35:30 +0000 Received: from DS3PEPF000099DD.namprd04.prod.outlook.com (2603:10b6:610:75:cafe::41) by CH0PR04CA0098.outlook.office365.com (2603:10b6:610:75::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8114.20 via Frontend Transport; Wed, 30 Oct 2024 21:35:29 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.118.233) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.118.233 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.118.233; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.118.233) by DS3PEPF000099DD.mail.protection.outlook.com (10.167.17.199) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8114.16 via Frontend Transport; Wed, 30 Oct 2024 21:35:29 +0000 Received: from drhqmail203.nvidia.com (10.126.190.182) by mail.nvidia.com (10.127.129.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Wed, 30 Oct 2024 14:35:10 -0700 Received: from drhqmail202.nvidia.com (10.126.190.181) by drhqmail203.nvidia.com (10.126.190.182) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Wed, 30 Oct 2024 14:35:09 -0700 Received: from Asurada-Nvidia.nvidia.com (10.127.8.12) by mail.nvidia.com (10.126.190.181) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Wed, 30 Oct 2024 14:35:08 -0700 From: Nicolin Chen To: , , CC: , , , , , , , , , , , , , , , , , , , , Subject: [PATCH v6 10/13] iommufd/selftest: Add refcount to mock_iommu_device Date: Wed, 30 Oct 2024 14:34:36 -0700 Message-ID: <4229037558d2c8103ec09674bfa3fd748bf7ba0e.1730313237.git.nicolinc@nvidia.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS3PEPF000099DD:EE_|SN7PR12MB7155:EE_ X-MS-Office365-Filtering-Correlation-Id: 229a4851-be51-4f70-08f0-08dcf92ac6d2 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|36860700013|82310400026|376014|7416014; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?+DSXRsJKO+rh2bqjqbDnmSXBGD/A9UOuAbW0gPtng5In4Z5LIxqryAvjPY4J?= =?us-ascii?Q?2fVYSkAwnYIVeovPfX0uq66BW8oyYCVP0/1TUFVncHTBxXC5/1gJeItA2EOe?= =?us-ascii?Q?zFI03BAdb9pgjxkz9Xq9GmX2z7KfF8B0Wrg2mQnZijlodTmLI17dqgOytjtJ?= =?us-ascii?Q?3l/V6i2MM+Q7iqLhPnhv2Ce9ESvFoqcmzdnyZpA5tSeNDjMyj2TdyCkJ+K+g?= =?us-ascii?Q?Yk10wNeKSMj/Fuo2q2xI/sopS4G9wrEMbDZRtaziOEkEKHDgDaQ9JVLRR50f?= =?us-ascii?Q?vwubjfjYNQdf2fgf0eXa3dd3ZBckucCVweRMX0OL6K+bCOTmc2tKAAIejeMK?= =?us-ascii?Q?uhHfRNhK6TY43TEIWisHYmdxwRVBLTec5enBHCPzJnqpOLTIbkWfCyZrXg0m?= =?us-ascii?Q?qghwmGNoWlACQvj4wbURbhgLuonA8OHeTkhG/R4F0JL80S3wHWycQfjD8Uon?= =?us-ascii?Q?sdeDVYm0URBTfUVwNVGb8ZHcqt0szg+4PcvILisXplDSkJ7MfnZdaBSZcexP?= =?us-ascii?Q?t+MW7J2ERt2lMVm9ASAkgwKXg8gkCJBOD9UHs/BqCzFya2X+DjzvK1mHi/Z+?= =?us-ascii?Q?Dw/I0iB7QHrJQ+pN4lt3XEghK4+wo873A3JeUWzJohNzyRyYDmCK0QmPWzgs?= =?us-ascii?Q?YpIBYT5Qc7cRwBY9hWW5rEIHJcVH99e9yKhwCSS4Ktf8lVSNDAnaW/O3+AmH?= =?us-ascii?Q?OQW7FGuXWhG3+iAQjD3oGoe/XL8Wsjxc8uKhbgX0X8VI8QqNR8Xb/5Zn1tTc?= =?us-ascii?Q?o9w/dWD2V9Jfu++sTpU3QfNuckYdOK+tBO1vMs9O7J6IyddOthak4EtAL0TG?= =?us-ascii?Q?+Rc5WW1yvyjSRnUzBpNo5yCQ+nkuMhX2flOKvcx653Hvtc8S4uxoJSiqyAZX?= =?us-ascii?Q?OwPIpLy2kX7lgxfZ0BKZ7NA2xjHkiwDmBm1PX7K0dqTA/7yEc9+lz/Gn0UZP?= =?us-ascii?Q?rmbNt7FwoM3XhOVEOti5690wmWOkChzxOQoKLb09RvFkWpA8mmk71EAwSsSb?= =?us-ascii?Q?IM/0+w9xpBDxqQ7J0oTJ2Td8b93uGIx6msc05SGsMOBCZBDLsCJlbuMmzu0N?= =?us-ascii?Q?WUvBs6y7P8ZJNri+7KT6waqxB8FK78cUrfp35TjmJDP+kU23TCk/eM1JhKv1?= =?us-ascii?Q?87RPoOXDRQjiJqquryqiuTSMqZzWi35/xjEI07Lh14a7fIGxbpbZxZTcagYA?= =?us-ascii?Q?Q9h5YSKqYzhD73p+msCpogIK4yM9y6Hvz5Kwr8kuB0at9psgL32auAhBv7bZ?= =?us-ascii?Q?Bqb5yKHH5VS8J+1GPo7tEHrtRX2HW6UWSgGK3uFLj8DMza/hCMD/rywEUBQE?= =?us-ascii?Q?zFv3rLDsN10Zykut5mdsrhd2D2n2R0Xn1fYA3wmrmsayO1TGnIa9XDd8Npcb?= =?us-ascii?Q?vAaLKp/7eARthNT9tzE6M0PUYWGoTMiqSbTZOSOg/2BtzRXcCA=3D=3D?= X-Forefront-Antispam-Report: CIP:216.228.118.233;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc7edge2.nvidia.com;CAT:NONE;SFS:(13230040)(1800799024)(36860700013)(82310400026)(376014)(7416014);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Oct 2024 21:35:29.8270 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 229a4851-be51-4f70-08f0-08dcf92ac6d2 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.118.233];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DS3PEPF000099DD.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN7PR12MB7155 Content-Type: text/plain; charset="utf-8" For an iommu_dev that can unplug (so far only this selftest does so), the viommu->iommu_dev pointer has no guarantee of its life cycle after it is copied from the idev->dev->iommu->iommu_dev. Track the user count of the iommu_dev. Postpone the exit routine using a completion, if refcount is unbalanced. The refcount inc/dec will be added in the following patch. Suggested-by: Jason Gunthorpe Reviewed-by: Kevin Tian Reviewed-by: Jason Gunthorpe Signed-off-by: Nicolin Chen --- drivers/iommu/iommufd/selftest.c | 39 +++++++++++++++++++++++++------- 1 file changed, 31 insertions(+), 8 deletions(-) diff --git a/drivers/iommu/iommufd/selftest.c b/drivers/iommu/iommufd/selft= est.c index 92d753985640..4f67a83f667a 100644 --- a/drivers/iommu/iommufd/selftest.c +++ b/drivers/iommu/iommufd/selftest.c @@ -533,14 +533,17 @@ static bool mock_domain_capable(struct device *dev, e= num iommu_cap cap) =20 static struct iopf_queue *mock_iommu_iopf_queue; =20 -static struct iommu_device mock_iommu_device =3D { -}; +static struct mock_iommu_device { + struct iommu_device iommu_dev; + struct completion complete; + refcount_t users; +} mock_iommu; =20 static struct iommu_device *mock_probe_device(struct device *dev) { if (dev->bus !=3D &iommufd_mock_bus_type.bus) return ERR_PTR(-ENODEV); - return &mock_iommu_device; + return &mock_iommu.iommu_dev; } =20 static void mock_domain_page_response(struct device *dev, struct iopf_faul= t *evt, @@ -1556,24 +1559,27 @@ int __init iommufd_test_init(void) if (rc) goto err_platform; =20 - rc =3D iommu_device_sysfs_add(&mock_iommu_device, + rc =3D iommu_device_sysfs_add(&mock_iommu.iommu_dev, &selftest_iommu_dev->dev, NULL, "%s", dev_name(&selftest_iommu_dev->dev)); if (rc) goto err_bus; =20 - rc =3D iommu_device_register_bus(&mock_iommu_device, &mock_ops, + rc =3D iommu_device_register_bus(&mock_iommu.iommu_dev, &mock_ops, &iommufd_mock_bus_type.bus, &iommufd_mock_bus_type.nb); if (rc) goto err_sysfs; =20 + refcount_set(&mock_iommu.users, 1); + init_completion(&mock_iommu.complete); + mock_iommu_iopf_queue =3D iopf_queue_alloc("mock-iopfq"); =20 return 0; =20 err_sysfs: - iommu_device_sysfs_remove(&mock_iommu_device); + iommu_device_sysfs_remove(&mock_iommu.iommu_dev); err_bus: bus_unregister(&iommufd_mock_bus_type.bus); err_platform: @@ -1583,6 +1589,22 @@ int __init iommufd_test_init(void) return rc; } =20 +static void iommufd_test_wait_for_users(void) +{ + if (refcount_dec_and_test(&mock_iommu.users)) + return; + /* + * Time out waiting for iommu device user count to become 0. + * + * Note that this is just making an example here, since the selftest is + * built into the iommufd module, i.e. it only unplugs the iommu device + * when unloading the module. So, it is expected that this WARN_ON will + * not trigger, as long as any iommufd FDs are open. + */ + WARN_ON(!wait_for_completion_timeout(&mock_iommu.complete, + msecs_to_jiffies(10000))); +} + void iommufd_test_exit(void) { if (mock_iommu_iopf_queue) { @@ -1590,8 +1612,9 @@ void iommufd_test_exit(void) mock_iommu_iopf_queue =3D NULL; } =20 - iommu_device_sysfs_remove(&mock_iommu_device); - iommu_device_unregister_bus(&mock_iommu_device, + iommufd_test_wait_for_users(); + iommu_device_sysfs_remove(&mock_iommu.iommu_dev); + iommu_device_unregister_bus(&mock_iommu.iommu_dev, &iommufd_mock_bus_type.bus, &iommufd_mock_bus_type.nb); bus_unregister(&iommufd_mock_bus_type.bus); --=20 2.43.0 From nobody Mon Nov 25 00:39:33 2024 Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2082.outbound.protection.outlook.com [40.107.244.82]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 501A52296DA; Wed, 30 Oct 2024 21:35:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.244.82 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730324134; cv=fail; b=Hc8wDzX4ybxe0soKJnp9we2scVhHZ3Mf7LDQwTowRjJrmE1tjTlzzFIsy3XrxC6SVA++Dw6d3PN9poMQDZh6Lq/mKMtSYp/MPWXqFcQEHzphzXcNiEIog2Erv4qyUHgZoT2NVJxr7F/tbM/NTwtPo3T7VPsgz4Emjdh2zar9woQ= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730324134; c=relaxed/simple; bh=XAnHHsXmfB9omCBKJVEBuBtMUpCwsE2Iax9K1of39lY=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=cnlsfyXVCSRMC6kzVoWLb3GPaNEzUX1AzN74C7PaWi3+xMuCi32LEzkMOrXhoSSsgEBhz+IyA+rOtAuDykG0tDC4jKZn+rqxuWYf53C+A2KzUwr0jew+yO44bIoDRkxoiyt87NMB20F48njsVjJazK1Akh9dgQQcL/Hcd8i7ey0= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=MfFN2WQP; arc=fail smtp.client-ip=40.107.244.82 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="MfFN2WQP" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=UKN/kb+SMl5z/AcdUGKgBebNRSJFEsSH7r3FfKUbjqpCRNMs2KjuKG7+psSTge4JcqmumgBqGc909+UO7KYOoA6lbV2qDMSBISjY2aFV+E/TsrnSaLW4klZauSL8G70NEuXtBZT8vkhJcmwL0t/XFwBto/7lKvpshgFmhST0KNUDGhRs2h466hoydHLtc/526qPDKsDd7XLtfpVDAAjcGBB+9t9NmQhyWxD/mMNfjoA2EAVgEivXwHYtr5UobfmXAa50mZZYZm3GuQFqJXnsvROfKVYoqvYTLaqsLb3pjqtqxIfmeHW5ayN/53kOw3+mHCp87MhPiRuQdAloJTOqyg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=8KzkVgcFO0ftegl7pn6y2UsVmj7zwEHZ8Ed8lJfivy8=; b=fmor5LrRIX5DD5aER5ywVYL3pvM/Jk/VDjroO9IXuz3+4aA8uEM04S+6abuPC0mgjGwbl/KVCcNOzlgM5ACnHLCZFMu0rkUVtbp2/5ymyDT1eT+eFo0ajVlS6Ka8eDZdVF3CU0naD6usRxp3c67kzr3655H41WYWgASuWJbIodU+xcKz2hGR2TFVye9cAWjwGRuzRLAS9sCgQQwGYPECxHE+Kooowe+3LxCgC3J5ul0GTfjCq7k04FsyU1pciBzVriTq8AbakK+sMitKfTIG+TYZBML3GWqwc8Yo6oUdYYdVMfQ0ytA6VR5B5lZqUF2SX3DLocUEJZi96PWKfLs0yQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.118.232) smtp.rcpttodomain=amd.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=8KzkVgcFO0ftegl7pn6y2UsVmj7zwEHZ8Ed8lJfivy8=; b=MfFN2WQPgzuUrhAWjp6F8lc1NlDRwQuDap8Qi7u3jgpXz0NACbmsksSxqLtIAZ9fiMJtI7GpqsEBKeEorHmZHpngn+I6YtyhRT8nkqMZaJTp8/FCQ361s5QLvztL4pTQHqacl60JjJ2NSrqyD8itm8DHLAZACh5JOXoOANfA+ATy7qmWAggXHfYKHPGM/WlMIMmoO5J9UQhekj/HUhc41rChmEqNtOkm12oTAOBh7qW2jEd0CkL0hvNGnsl1l3GKmYTQMLHDRKwXuYWiiEhETqfFie9Ts7X/06ukROfnARXm8F10krz+attjGwJxBWF+vKXhT3q66LfG7ed/aEntkA== Received: from SA9PR13CA0143.namprd13.prod.outlook.com (2603:10b6:806:27::28) by PH7PR12MB6665.namprd12.prod.outlook.com (2603:10b6:510:1a7::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8093.32; Wed, 30 Oct 2024 21:35:29 +0000 Received: from SN1PEPF0002BA4E.namprd03.prod.outlook.com (2603:10b6:806:27:cafe::d3) by SA9PR13CA0143.outlook.office365.com (2603:10b6:806:27::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8114.16 via Frontend Transport; Wed, 30 Oct 2024 21:35:29 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.118.232) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.118.232 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.118.232; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.118.232) by SN1PEPF0002BA4E.mail.protection.outlook.com (10.167.242.71) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8114.16 via Frontend Transport; Wed, 30 Oct 2024 21:35:29 +0000 Received: from drhqmail203.nvidia.com (10.126.190.182) by mail.nvidia.com (10.127.129.5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Wed, 30 Oct 2024 14:35:11 -0700 Received: from drhqmail202.nvidia.com (10.126.190.181) by drhqmail203.nvidia.com (10.126.190.182) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Wed, 30 Oct 2024 14:35:10 -0700 Received: from Asurada-Nvidia.nvidia.com (10.127.8.12) by mail.nvidia.com (10.126.190.181) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Wed, 30 Oct 2024 14:35:10 -0700 From: Nicolin Chen To: , , CC: , , , , , , , , , , , , , , , , , , , , Subject: [PATCH v6 11/13] iommufd/selftest: Add IOMMU_VIOMMU_TYPE_SELFTEST Date: Wed, 30 Oct 2024 14:34:37 -0700 Message-ID: <3f2c6db0a7a5ad0209c61304b6c5348a8d3df6ec.1730313237.git.nicolinc@nvidia.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SN1PEPF0002BA4E:EE_|PH7PR12MB6665:EE_ X-MS-Office365-Filtering-Correlation-Id: a47c069b-b154-4706-1918-08dcf92ac673 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|376014|7416014|36860700013|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?an9fYIR2N6W6XVIBID+5+bJzvkxgs2wXDotrHtooaDbqGx03OYHCGaZZhZw2?= =?us-ascii?Q?rYvpoYHgfvTJHazGWdS9tMzhzejIuWtNhl3ukp3nrTP+NCd2EIDqkQE/dkWL?= =?us-ascii?Q?Y4IlS5ah+Yap6yOxj/lIuPb8eCLjlrKfDFUmHvDBAvWMEiIO0hVl2FrDhUwG?= =?us-ascii?Q?lVbxr+V/6Rr+MEpaL9rEXiAO47Ne1NfC8T4VEpj+I6i5A75m7d3eb2ybGaS9?= =?us-ascii?Q?2H3O8jotwZ4S9/ZUM/oSWGHV/dhmdn+51ffJEw11h4fA2153AjbK0iVrOgvr?= =?us-ascii?Q?4B3pRCckT+QRYdQFqogGadhyY57lsrl2aIIR2ROw37+RfbIfRFr1zHVdCz5N?= =?us-ascii?Q?8aG2dyEC965XYkJpcLvRB1wylp7EFBSj4Woc77/pECdJQ1KSbBctjGUeFFcB?= =?us-ascii?Q?7/FjWJYJAlqqoAFK69CRR5pnl7UrRMx+/WGFGUnPAE3kRf9MGkib9lkjPEG/?= =?us-ascii?Q?/qY90csaAczgwyuQqkKZS2U3yeJcW20z9NoOFsoycyeDF4EDKMliUVhBqByu?= =?us-ascii?Q?kax0h3tJXDNet0CWel4XThXFCbLp6e8BDzv7G8RbtE/2PButoYOS27W/sWtr?= =?us-ascii?Q?l4bdToP1eVA13wzzIfkZFgrg+n1lO26MYLe86QMc2rhNFKBPRQuHIpcwGpoJ?= =?us-ascii?Q?/sjBVfZWIPVLTkfatUbIpVIGEoA47E3fUYq+1iJLYGJjLj/MWiJh8pJTymZO?= =?us-ascii?Q?4JPvnwIS1Gj7kHWTxP1tqnvLKObJ6/tKpBtMzc+T8v/9ySecn9r1riFYW+z6?= =?us-ascii?Q?EQkEj8YvFRVmjWnOn57cTj1Dl7ktSMKALKBSAgdSr18Hd+1A/DsXifIH6b2X?= =?us-ascii?Q?NQaCdMplnzgr6b8wSEi5ctCTUa4BHjiQWnx2uZv+/OCnJmnRQuOoiHDh1Nrl?= =?us-ascii?Q?K6pcNQrpksMv4lZQzeM5gcJoamrksfxAvAK38AO4qvXYkHZGBSgln3F+T5BO?= =?us-ascii?Q?CEBICVtyn9XgC00dy5/NXY4bM3l+Ii+BtVNeebyw3B/q1UqkCz7i+KBpVNFK?= =?us-ascii?Q?BTiXWdBPJc5VxZKX5hf27k02ozQ63z7wMv2D8bEzMOZLu9+C/Ub8Zz44Vl5s?= =?us-ascii?Q?AUJZB5lAfnXOOml2e1pSLyjAIkXr0IUUaApNn57uYdATryOUyDuBBiXso0pO?= =?us-ascii?Q?jsOCN+hEVfhwEQ07N/jPZiwxbqYhv/BLSyOFuDQsYhSHUyppAZCT0boIpccu?= =?us-ascii?Q?tHN+KoRxVSOmcyDLcYa6hFij5h2hMVVQjTqTgRDVDwvO+xqN+Jnhz7Dr3qtR?= =?us-ascii?Q?eFZgkuJ7+EsRZB9X9Dl5BsLoMf0GbF4EFTM0csIwv7sliRCUR7uJ/pFmumGB?= =?us-ascii?Q?XtvkNm5rusJ/MznReFpmQSKtfl8VnNWRe4rU/KQRlcZPrEb1CfF4tGJx8Jab?= =?us-ascii?Q?uX+ohVeMez2uK/TEZAQGrHFY6Lb8ztAVukF+70P1Nth1aUW4ew=3D=3D?= X-Forefront-Antispam-Report: CIP:216.228.118.232;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc7edge1.nvidia.com;CAT:NONE;SFS:(13230040)(82310400026)(376014)(7416014)(36860700013)(1800799024);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Oct 2024 21:35:29.1736 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: a47c069b-b154-4706-1918-08dcf92ac673 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.118.232];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: SN1PEPF0002BA4E.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB6665 Content-Type: text/plain; charset="utf-8" Implement the viommu alloc/free functions to increase/reduce refcount of its dependent mock iommu device. User space can verify this loop via the IOMMU_VIOMMU_TYPE_SELFTEST. Reviewed-by: Kevin Tian Reviewed-by: Jason Gunthorpe Signed-off-by: Nicolin Chen --- drivers/iommu/iommufd/iommufd_test.h | 2 + drivers/iommu/iommufd/selftest.c | 67 ++++++++++++++++++++++++++++ 2 files changed, 69 insertions(+) diff --git a/drivers/iommu/iommufd/iommufd_test.h b/drivers/iommu/iommufd/i= ommufd_test.h index f4bc23a92f9a..edced4ac7cd3 100644 --- a/drivers/iommu/iommufd/iommufd_test.h +++ b/drivers/iommu/iommufd/iommufd_test.h @@ -180,4 +180,6 @@ struct iommu_hwpt_invalidate_selftest { __u32 iotlb_id; }; =20 +#define IOMMU_VIOMMU_TYPE_SELFTEST 0xdeadbeef + #endif diff --git a/drivers/iommu/iommufd/selftest.c b/drivers/iommu/iommufd/selft= est.c index 4f67a83f667a..31c8f78a3a66 100644 --- a/drivers/iommu/iommufd/selftest.c +++ b/drivers/iommu/iommufd/selftest.c @@ -134,6 +134,7 @@ to_mock_domain(struct iommu_domain *domain) =20 struct mock_iommu_domain_nested { struct iommu_domain domain; + struct mock_viommu *mock_viommu; struct mock_iommu_domain *parent; u32 iotlb[MOCK_NESTED_DOMAIN_IOTLB_NUM]; }; @@ -144,6 +145,16 @@ to_mock_nested(struct iommu_domain *domain) return container_of(domain, struct mock_iommu_domain_nested, domain); } =20 +struct mock_viommu { + struct iommufd_viommu core; + struct mock_iommu_domain *s2_parent; +}; + +static inline struct mock_viommu *to_mock_viommu(struct iommufd_viommu *vi= ommu) +{ + return container_of(viommu, struct mock_viommu, core); +} + enum selftest_obj_type { TYPE_IDEV, }; @@ -569,6 +580,61 @@ static int mock_dev_disable_feat(struct device *dev, e= num iommu_dev_features fea return 0; } =20 +static void mock_viommu_destroy(struct iommufd_viommu *viommu) +{ + struct mock_iommu_device *mock_iommu =3D container_of( + viommu->iommu_dev, struct mock_iommu_device, iommu_dev); + + if (refcount_dec_and_test(&mock_iommu->users)) + complete(&mock_iommu->complete); + + /* iommufd core frees mock_viommu and viommu */ +} + +static struct iommu_domain * +mock_viommu_alloc_domain_nested(struct iommufd_viommu *viommu, u32 flags, + const struct iommu_user_data *user_data) +{ + struct mock_viommu *mock_viommu =3D to_mock_viommu(viommu); + struct mock_iommu_domain_nested *mock_nested; + + if (flags & ~IOMMU_HWPT_FAULT_ID_VALID) + return ERR_PTR(-EOPNOTSUPP); + + mock_nested =3D __mock_domain_alloc_nested(user_data); + if (IS_ERR(mock_nested)) + return ERR_CAST(mock_nested); + mock_nested->mock_viommu =3D mock_viommu; + mock_nested->parent =3D mock_viommu->s2_parent; + return &mock_nested->domain; +} + +static struct iommufd_viommu_ops mock_viommu_ops =3D { + .destroy =3D mock_viommu_destroy, + .alloc_domain_nested =3D mock_viommu_alloc_domain_nested, +}; + +static struct iommufd_viommu *mock_viommu_alloc(struct device *dev, + struct iommu_domain *domain, + struct iommufd_ctx *ictx, + unsigned int viommu_type) +{ + struct mock_iommu_device *mock_iommu =3D + iommu_get_iommu_dev(dev, struct mock_iommu_device, iommu_dev); + struct mock_viommu *mock_viommu; + + if (viommu_type !=3D IOMMU_VIOMMU_TYPE_SELFTEST) + return ERR_PTR(-EOPNOTSUPP); + + mock_viommu =3D iommufd_viommu_alloc(ictx, struct mock_viommu, core, + &mock_viommu_ops); + if (IS_ERR(mock_viommu)) + return ERR_CAST(mock_viommu); + + refcount_inc(&mock_iommu->users); + return &mock_viommu->core; +} + static const struct iommu_ops mock_ops =3D { /* * IOMMU_DOMAIN_BLOCKED cannot be returned from def_domain_type() @@ -588,6 +654,7 @@ static const struct iommu_ops mock_ops =3D { .dev_enable_feat =3D mock_dev_enable_feat, .dev_disable_feat =3D mock_dev_disable_feat, .user_pasid_table =3D true, + .viommu_alloc =3D mock_viommu_alloc, .default_domain_ops =3D &(struct iommu_domain_ops){ .free =3D mock_domain_free, --=20 2.43.0 From nobody Mon Nov 25 00:39:33 2024 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2042.outbound.protection.outlook.com [40.107.93.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8F3C52296D8; Wed, 30 Oct 2024 21:35:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.93.42 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730324145; cv=fail; b=caKxqQw4eyxypPIcPU98VmVXsvbXhR+OQOnBcgHqQKY2OTItqdQ2QuFnuuotUIEz+aaNzrE/HxppKfUtSL8Xd/VhSi0+UrYt0YeZHUUdTPvR2vpU8b/rV+0cawcJhw+ox/GH+Wsy+Cib5grybODeLQyBR3JYISRKhgbTVH4mLp8= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730324145; c=relaxed/simple; bh=T1wbI6by0JBYmd//pmoH2ZvxWDittyZsftmpRBU2YWI=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=MfsgO2eL8gSgi+KR6lZ5XdRcT+m3xCF0AK6b9BwVizw0eNABIUZJkS7bQo7cfIb3l/+B9pglN5GZCI4RjwFwnOKjVK7xMdQ2MDNsKNKCwt86+0DW9vhleEmInn6Bm5KibZH1EeHapoQNRptg6lE5bdrh2WRpE8H5dXSWiwS4QQs= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=S1bEz0OJ; arc=fail smtp.client-ip=40.107.93.42 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="S1bEz0OJ" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=EpxAtOXUJG61JIyoCsTqbOkp2r7UgvdolK4AYRbS1O9OVTI40UPldVqAqQMN2jzw2+9EH+gHY3laS1XNYQOuzUbeGw+gOY+7M3+DvOSsQJhgl8f0cGtd5tE5o/CiD/5WE0ds3GkhXcDn9Ihpr5jmMoOvqRWXEPMaIjcJi3lyJCyuh1JuPPbT9UQv6r72eblwmtZhyCklE1i5BKWQ/HMHVWUCt5yhqde3ItEclZHbLUIxsyfjaeawCMEXjkllna1QDkzl5toqJ3a6cDa80sigfiVzgBskYR2iW7OB9DTfAFeMuj3EGicZttYTsf3nsET2BMlgW2YOfmJ2oxEfPLG/jg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=WMSjT/egPflIojDvdvcXmjZeuSPlTWIFCl4jiFCu+gA=; b=dtxO+bGhVm6tTBhqXCkGrJmGNEeeWtnveDZE//qGckk4y0GM16/NyqGFkmASGfGkWQraEKYPOvEqi5Stl/hIfdKz5adKOOI9T3+F4A5YVP6h/wQglGXA1USuqPNF46/1BY5oPoRhS1jTmozz1X8K4NEn/Ym77SXqNemaVhIRe2yKvWPS/84PRNCNr/X9lju47C+GcPZDPI/FQES56IxSjJAI2oiFjFc5VX8XA2sZmMSmKT+YayisI5QMp/tyA7SitoBJ02W8oax8ouIk7s8zGoJYi3niPSjB1z1qGmtwPw82kDt35lzs/rFwV0db0i8VxFJLjC6Kc938Q/HBXWO+wA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.118.233) smtp.rcpttodomain=amd.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=WMSjT/egPflIojDvdvcXmjZeuSPlTWIFCl4jiFCu+gA=; b=S1bEz0OJmuF7alReda3XdQSZeUltDeITF3ctYY8V3LI0a8bzHxO8Xi+Nf3FEcGcll4XDhYSj4pxJG4Cj793Td6NOQS6kEsEAZHGY5gRAmGTetcSVSIDmPmIxRR3WsWBTfhzGBSksiAF58ptcPlBQBT0MHByD84x4XfRy61ySJ0tmOvvAx0tmdgFj8edQOFVpWeJwB3R893Q0GqRaaivlWuoRr8hMPTNgkK6NlRLFbKp9lzhUeDyKW0/jBYfRtNeJAi3OlKc9WcToyQFW7X138MQbnCoSBYvTlulUaW7CJnBcVqjns1eunVeArrfbw/wCIpbklDpXDSRP6fLkDhFfiQ== Received: from CH2PR20CA0003.namprd20.prod.outlook.com (2603:10b6:610:58::13) by SJ2PR12MB8978.namprd12.prod.outlook.com (2603:10b6:a03:545::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8093.32; Wed, 30 Oct 2024 21:35:35 +0000 Received: from DS3PEPF000099DE.namprd04.prod.outlook.com (2603:10b6:610:58:cafe::a8) by CH2PR20CA0003.outlook.office365.com (2603:10b6:610:58::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8114.20 via Frontend Transport; Wed, 30 Oct 2024 21:35:35 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.118.233) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.118.233 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.118.233; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.118.233) by DS3PEPF000099DE.mail.protection.outlook.com (10.167.17.200) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8114.16 via Frontend Transport; Wed, 30 Oct 2024 21:35:35 +0000 Received: from drhqmail203.nvidia.com (10.126.190.182) by mail.nvidia.com (10.127.129.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Wed, 30 Oct 2024 14:35:13 -0700 Received: from drhqmail202.nvidia.com (10.126.190.181) by drhqmail203.nvidia.com (10.126.190.182) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Wed, 30 Oct 2024 14:35:12 -0700 Received: from Asurada-Nvidia.nvidia.com (10.127.8.12) by mail.nvidia.com (10.126.190.181) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Wed, 30 Oct 2024 14:35:11 -0700 From: Nicolin Chen To: , , CC: , , , , , , , , , , , , , , , , , , , , Subject: [PATCH v6 12/13] iommufd/selftest: Add IOMMU_VIOMMU_ALLOC test coverage Date: Wed, 30 Oct 2024 14:34:38 -0700 Message-ID: <8188b47bbee49e071907e1df37a3af269bd840b6.1730313237.git.nicolinc@nvidia.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS3PEPF000099DE:EE_|SJ2PR12MB8978:EE_ X-MS-Office365-Filtering-Correlation-Id: f6bff9da-87e6-4025-3581-08dcf92aca14 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|36860700013|376014|7416014|82310400026; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?zdFZTYXB3PNvbO6JcFx3+ynp+Nip9RMHE3Sxme3bKvZcPZkWOZVnOxTePD8b?= =?us-ascii?Q?nsB2xkDQ9izcLC8cTZSsLWqNHZf1/LtUonFAPKF5wpflP9lWV1ShrcjZgxQA?= =?us-ascii?Q?40CAeKuahBF/61j5dnvXqFlM/VIirx+LBK7LZdwueppFX7lraq6vziUPVFds?= =?us-ascii?Q?4HOMXyxT6N19JqVxoK+72DznqQRu/DfpI1SsypgrCguw3GGTnfhDx6n6Pz0W?= =?us-ascii?Q?ZL8FhmDjHzb6wleNVrJMsJhWVSBwDEP3EbfQ8bYfXNNtZvAPWB73p/6Xfgop?= =?us-ascii?Q?iUkh4W5F6yJgmJNcTc0j2c41V/xSagLsQoxjH92YNpXKMmkoaHypTSSV0LO0?= =?us-ascii?Q?o3pH+feo5zH0pAS6bbSg6yl2kvUBYrBbG49fVxM66x1Bswj4Ypxw9oqFmJAQ?= =?us-ascii?Q?LGil/r8q/8l2r39eODK0EJz05Jsn7/o6KmqzV4erU99aDAz8vfH/r6fZCugi?= =?us-ascii?Q?NVLXhBXXECgyAm84oGeZNrwi8k6ht2rsze32T9qrkS36yyR3JAJe0nzVOKCw?= =?us-ascii?Q?7Fw1Sl9fp44xTpwwxTAog5uUk7Kv11Ro/43CyWpsToGjgCFt1sbOzdpJ+hWb?= =?us-ascii?Q?2O7MyudxvHBX2PnSYxelWYYj1WhXADSQ2UrYUnQ6GMrvmKwp1C8B6VsvdHVi?= =?us-ascii?Q?py+zVaTslbYTqad5wWTzLPUHJBsp2/fu3KHPI1MCoY7XC6QfyaT3fAVi0iid?= =?us-ascii?Q?s72zDDqWol880/+AqDUAQt4IBLFBIa9IIac/VsVt3Ko2WRxfv2z02MhSUhCN?= =?us-ascii?Q?pPUJPgWFd3+c1lBxudqa/toEFa3LSCIowArppf+naOYYhSLWcuib2ZPbiQE7?= =?us-ascii?Q?/Ayy11bXN2/2SeC7DCwIGocsH4rGuMDqdyBBkHHttT3iwI5RWA2GPsZH5eo0?= =?us-ascii?Q?5ApeSz2un52Sgt6XY5CqJoj4eI2wB/82Ywg9JXDiRwiOAUbAuY8S0G6nO9Jz?= =?us-ascii?Q?PpaiSObj/xEY9oRUSy581B0+GVMmKwccCPRtFxdwM8QX92bmJI+VMu4DzqSq?= =?us-ascii?Q?CeHxFt8TUfJ+P8QRugTJ5LjmEA+G92L83KZlIIbvq+R0Jd+Gf6xhcS9kQThc?= =?us-ascii?Q?wwcd/+xSYFruJNnG3fJpF814lZFtRX2OzUIlc0U+DkotaiU6R7UqAHc2OX09?= =?us-ascii?Q?SNznj8au2MtZr+Yj7XLPuNe4unLXrsPd5Nu7sdLHQ3s/Ec+ceawmhX16IPO6?= =?us-ascii?Q?ywT6FfKBlSRq3Umg06yJFs0ZpEAUxrnIHNs6mpun5laO/CIk3kMkUQn/2c5P?= =?us-ascii?Q?HitltVOCR4ogDaYa/uAvsVOqQpwcMinUihlv2LC1T0xw/l/bxXc6YAAJAF2h?= =?us-ascii?Q?ZlrFpuggUKELPHotZQ/knrZnfffvem+zu4N8CKWMkfd6RxSfjtauyqGY3tre?= =?us-ascii?Q?7xeqiEjmT7QWo9Kb7HNLly/Z/5GmGgyh9fEiVjdpyFvUwgOdDA=3D=3D?= X-Forefront-Antispam-Report: CIP:216.228.118.233;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc7edge2.nvidia.com;CAT:NONE;SFS:(13230040)(1800799024)(36860700013)(376014)(7416014)(82310400026);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Oct 2024 21:35:35.2626 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: f6bff9da-87e6-4025-3581-08dcf92aca14 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.118.233];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DS3PEPF000099DE.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ2PR12MB8978 Content-Type: text/plain; charset="utf-8" Add a new iommufd_viommu FIXTURE and setup it up with a vIOMMU object. Any new vIOMMU feature will be added as a TEST_F under that. Reviewed-by: Kevin Tian Signed-off-by: Nicolin Chen --- tools/testing/selftests/iommu/iommufd_utils.h | 28 ++++ tools/testing/selftests/iommu/iommufd.c | 128 ++++++++++++++++++ .../selftests/iommu/iommufd_fail_nth.c | 11 ++ 3 files changed, 167 insertions(+) diff --git a/tools/testing/selftests/iommu/iommufd_utils.h b/tools/testing/= selftests/iommu/iommufd_utils.h index 6a11c26370f3..7dabc261fae2 100644 --- a/tools/testing/selftests/iommu/iommufd_utils.h +++ b/tools/testing/selftests/iommu/iommufd_utils.h @@ -819,3 +819,31 @@ static int _test_cmd_trigger_iopf(int fd, __u32 device= _id, __u32 fault_fd) =20 #define test_cmd_trigger_iopf(device_id, fault_fd) \ ASSERT_EQ(0, _test_cmd_trigger_iopf(self->fd, device_id, fault_fd)) + +static int _test_cmd_viommu_alloc(int fd, __u32 device_id, __u32 hwpt_id, + __u32 type, __u32 flags, __u32 *viommu_id) +{ + struct iommu_viommu_alloc cmd =3D { + .size =3D sizeof(cmd), + .flags =3D flags, + .type =3D type, + .dev_id =3D device_id, + .hwpt_id =3D hwpt_id, + }; + int ret; + + ret =3D ioctl(fd, IOMMU_VIOMMU_ALLOC, &cmd); + if (ret) + return ret; + if (viommu_id) + *viommu_id =3D cmd.out_viommu_id; + return 0; +} + +#define test_cmd_viommu_alloc(device_id, hwpt_id, type, viommu_id) \ + ASSERT_EQ(0, _test_cmd_viommu_alloc(self->fd, device_id, hwpt_id, \ + type, 0, viommu_id)) +#define test_err_viommu_alloc(_errno, device_id, hwpt_id, type, viommu_id)= \ + EXPECT_ERRNO(_errno, \ + _test_cmd_viommu_alloc(self->fd, device_id, hwpt_id, \ + type, 0, viommu_id)) diff --git a/tools/testing/selftests/iommu/iommufd.c b/tools/testing/selfte= sts/iommu/iommufd.c index 88b92bb69756..3142819cc26d 100644 --- a/tools/testing/selftests/iommu/iommufd.c +++ b/tools/testing/selftests/iommu/iommufd.c @@ -133,6 +133,7 @@ TEST_F(iommufd, cmd_length) TEST_LENGTH(iommu_option, IOMMU_OPTION, val64); TEST_LENGTH(iommu_vfio_ioas, IOMMU_VFIO_IOAS, __reserved); TEST_LENGTH(iommu_ioas_map_file, IOMMU_IOAS_MAP_FILE, iova); + TEST_LENGTH(iommu_viommu_alloc, IOMMU_VIOMMU_ALLOC, out_viommu_id); #undef TEST_LENGTH } =20 @@ -2480,4 +2481,131 @@ TEST_F(vfio_compat_mock_domain, huge_map) } } =20 +FIXTURE(iommufd_viommu) +{ + int fd; + uint32_t ioas_id; + uint32_t stdev_id; + uint32_t hwpt_id; + uint32_t nested_hwpt_id; + uint32_t device_id; + uint32_t viommu_id; +}; + +FIXTURE_VARIANT(iommufd_viommu) +{ + unsigned int viommu; +}; + +FIXTURE_SETUP(iommufd_viommu) +{ + self->fd =3D open("/dev/iommu", O_RDWR); + ASSERT_NE(-1, self->fd); + test_ioctl_ioas_alloc(&self->ioas_id); + test_ioctl_set_default_memory_limit(); + + if (variant->viommu) { + struct iommu_hwpt_selftest data =3D { + .iotlb =3D IOMMU_TEST_IOTLB_DEFAULT, + }; + + test_cmd_mock_domain(self->ioas_id, &self->stdev_id, NULL, + &self->device_id); + + /* Negative test -- invalid hwpt */ + test_err_viommu_alloc(ENOENT, self->device_id, self->hwpt_id, + IOMMU_VIOMMU_TYPE_SELFTEST, NULL); + + /* Negative test -- not a nesting parent hwpt */ + test_cmd_hwpt_alloc(self->device_id, self->ioas_id, 0, + &self->hwpt_id); + test_err_viommu_alloc(EINVAL, self->device_id, self->hwpt_id, + IOMMU_VIOMMU_TYPE_SELFTEST, NULL); + test_ioctl_destroy(self->hwpt_id); + + /* Allocate a nesting parent hwpt */ + test_cmd_hwpt_alloc(self->device_id, self->ioas_id, + IOMMU_HWPT_ALLOC_NEST_PARENT, + &self->hwpt_id); + /* Negative test -- unsupported viommu type */ + test_err_viommu_alloc(EOPNOTSUPP, self->device_id, + self->hwpt_id, 0xdead, NULL); + + /* Allocate a vIOMMU taking refcount of the parent hwpt */ + test_cmd_viommu_alloc(self->device_id, self->hwpt_id, + IOMMU_VIOMMU_TYPE_SELFTEST, + &self->viommu_id); + EXPECT_ERRNO(EBUSY, + _test_ioctl_destroy(self->fd, self->hwpt_id)); + + /* Allocate a regular nested hwpt */ + test_cmd_hwpt_alloc_nested(self->device_id, self->viommu_id, 0, + &self->nested_hwpt_id, + IOMMU_HWPT_DATA_SELFTEST, &data, + sizeof(data)); + EXPECT_ERRNO(EBUSY, + _test_ioctl_destroy(self->fd, self->viommu_id)); + } else { + test_err_viommu_alloc(ENOENT, self->device_id, self->hwpt_id, + IOMMU_VIOMMU_TYPE_SELFTEST, NULL); + } +} + +FIXTURE_TEARDOWN(iommufd_viommu) +{ + teardown_iommufd(self->fd, _metadata); +} + +FIXTURE_VARIANT_ADD(iommufd_viommu, no_viommu) +{ + .viommu =3D 0, +}; + +FIXTURE_VARIANT_ADD(iommufd_viommu, mock_viommu) +{ + .viommu =3D 1, +}; + +TEST_F(iommufd_viommu, viommu_auto_destroy) +{ +} + +TEST_F(iommufd_viommu, viommu_alloc_nested_iopf) +{ + struct iommu_hwpt_selftest data =3D { + .iotlb =3D IOMMU_TEST_IOTLB_DEFAULT, + }; + uint32_t viommu_id =3D self->viommu_id; + uint32_t dev_id =3D self->device_id; + uint32_t iopf_hwpt_id; + uint32_t fault_id; + uint32_t fault_fd; + + if (self->device_id) { + test_ioctl_fault_alloc(&fault_id, &fault_fd); + test_err_hwpt_alloc_iopf( + ENOENT, dev_id, viommu_id, UINT32_MAX, + IOMMU_HWPT_FAULT_ID_VALID, &iopf_hwpt_id, + IOMMU_HWPT_DATA_SELFTEST, &data, sizeof(data)); + test_err_hwpt_alloc_iopf( + EOPNOTSUPP, dev_id, viommu_id, fault_id, + IOMMU_HWPT_FAULT_ID_VALID | (1 << 31), &iopf_hwpt_id, + IOMMU_HWPT_DATA_SELFTEST, &data, sizeof(data)); + test_cmd_hwpt_alloc_iopf( + dev_id, viommu_id, fault_id, IOMMU_HWPT_FAULT_ID_VALID, + &iopf_hwpt_id, IOMMU_HWPT_DATA_SELFTEST, &data, + sizeof(data)); + + test_cmd_mock_domain_replace(self->stdev_id, iopf_hwpt_id); + EXPECT_ERRNO(EBUSY, + _test_ioctl_destroy(self->fd, iopf_hwpt_id)); + test_cmd_trigger_iopf(dev_id, fault_fd); + + test_cmd_mock_domain_replace(self->stdev_id, self->ioas_id); + test_ioctl_destroy(iopf_hwpt_id); + close(fault_fd); + test_ioctl_destroy(fault_id); + } +} + TEST_HARNESS_MAIN diff --git a/tools/testing/selftests/iommu/iommufd_fail_nth.c b/tools/testi= ng/selftests/iommu/iommufd_fail_nth.c index 2d7d01638be8..fb618485d7ca 100644 --- a/tools/testing/selftests/iommu/iommufd_fail_nth.c +++ b/tools/testing/selftests/iommu/iommufd_fail_nth.c @@ -621,6 +621,7 @@ TEST_FAIL_NTH(basic_fail_nth, device) uint32_t stdev_id; uint32_t idev_id; uint32_t hwpt_id; + uint32_t viommu_id; __u64 iova; =20 self->fd =3D open("/dev/iommu", O_RDWR); @@ -663,6 +664,16 @@ TEST_FAIL_NTH(basic_fail_nth, device) =20 if (_test_cmd_mock_domain_replace(self->fd, stdev_id, hwpt_id, NULL)) return -1; + + if (_test_cmd_hwpt_alloc(self->fd, idev_id, ioas_id, 0, + IOMMU_HWPT_ALLOC_NEST_PARENT, &hwpt_id, + IOMMU_HWPT_DATA_NONE, 0, 0)) + return -1; + + if (_test_cmd_viommu_alloc(self->fd, idev_id, hwpt_id, + IOMMU_VIOMMU_TYPE_SELFTEST, 0, &viommu_id)) + return -1; + return 0; } =20 --=20 2.43.0 From nobody Mon Nov 25 00:39:33 2024 Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2071.outbound.protection.outlook.com [40.107.243.71]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 52D7D2296F2; Wed, 30 Oct 2024 21:35:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.243.71 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730324138; cv=fail; b=Lva3u6Dl4og7nV/6rNkdHYVnwndVwmCtaxvsJ81dy9jiLIG1gci37bCjqbiYkLV1Tst1vvMDKbwh7IZIcJEZl0zbRFq8pMyV6bymt+zUIm3clNrA1mNDM3AjOf88Cq1+WAwxsGATv6G7vVb/5xyukt6mwb3q1E6XSGRIPmHZgUw= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730324138; c=relaxed/simple; bh=Bc7irgx/6rjjEyhgYWm5e4HG/xlq3Zz6q1tfakfeWrM=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=qfCBRUJwsDmw7oVR6yKJKyhR219UeLpNG6uFh7V6B6E6YMiYreVtJT4Gw2Dn1WXP0c62d26i1X6K++QIsbs/ctlrToQkyBgzh3AEIPepQpRFSEfQa2G39H8KfWXnC09t6ZpNghQAXDTVpv5CoVTaVxMWkn/EekvmLSzsBmE6FAw= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=lvXseqPC; arc=fail smtp.client-ip=40.107.243.71 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="lvXseqPC" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=L1jOOYuKpA8kK8c0NaUeIfsL7KSsSU3BPQ9okaatyC+oco/4E6UkPGFU2uOrKOJ6zuM2aKLACRgU9cH1Zp6dIulOqWofdr50Fkj59gLd/rK/lN2LT66gNQE3oWksZB103CRN/tv6NZg5ceiEkpDQPexWbcBD40aTgrsXnEmVKyWxfQRAT1FFCpN6+6xHFoMmnb5lq7sHRbLP052Ik/P6Io5t1czsI8c5aJrPTW0kN5Fgl3VOmL23ukP6Ou84D3XP3KUGJmn3HKMukZtTh+LElmXfbxn9ewX9xQr8AhzIOMxUl4m1FNrqVJXzA2x9vwHBl5fFVG7Ou7MU+EXTvAczsw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=LEMmgACXamRJHRJ8k8PHyMQpZBwcJXQxMazWsnOea9A=; b=kS2Y+NJVj029x3bxZXOGu7XYis/Bfg+yx49PR4pn1Fb7hEfdzQgJV0NMVuMLYn9ZKfC5vB1Kkb5wDj8bAdRLATHLf+08gbNJ2MGdzPqUxMgs0tRgPGMZlBFUD942YlHQlIs7BBDFMsUsioNO2gkkwDyGVI2jjDZA2zKXTV/pdKqyDoPN4e6SxGthExHF4aAuVeAV6RYbvsogAMD11avW0m82kZPIAvYDYNj/KkYhTofpRRXEcaethoMOCeXbOTAhGYrylaKkCswMMPNNOVhmxWBLjOe7CDcUTvpfpazRUAHiGus9AesIcCy6+yY+YjMLDcipo5Ls+gl8QkkbSqQ0VA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.118.232) smtp.rcpttodomain=amd.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=LEMmgACXamRJHRJ8k8PHyMQpZBwcJXQxMazWsnOea9A=; b=lvXseqPCCRzJQNPQwLe85r9113e/1X+yVTVI1QqI6JS1jvBRLZD8RnmjvgOKCMRc24bwTiwm91k8GFIllMwF4A/OiF3pqxt53C3MOnTYQcV7Knr9NMQvWxYh5GZVohC9dNsE7ejMdT4svJ8shCzWUOTaS1ukf2bzzBweV5mffUnOZdXTgo+TqBVCIOsqfe12/TkCc9ryz4qzkTeVywn5xOtmIgyP74V3l/Sy7Gvhr4CaJ5hDDSJtydt5W4N7CoEMZ260dN0wyT0VVSdg0MtoXZ1ZAzKz68SfhgUXEzOEKbfAjWMbGnqs+pHpdqlsMlr1c1VO9C/Vo7LG2nNWm4kjmQ== Received: from SA9PR13CA0130.namprd13.prod.outlook.com (2603:10b6:806:27::15) by MN2PR12MB4344.namprd12.prod.outlook.com (2603:10b6:208:26e::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8093.25; Wed, 30 Oct 2024 21:35:31 +0000 Received: from SN1PEPF0002BA4E.namprd03.prod.outlook.com (2603:10b6:806:27:cafe::8e) by SA9PR13CA0130.outlook.office365.com (2603:10b6:806:27::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8114.11 via Frontend Transport; Wed, 30 Oct 2024 21:35:31 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.118.232) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.118.232 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.118.232; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.118.232) by SN1PEPF0002BA4E.mail.protection.outlook.com (10.167.242.71) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8114.16 via Frontend Transport; Wed, 30 Oct 2024 21:35:31 +0000 Received: from drhqmail203.nvidia.com (10.126.190.182) by mail.nvidia.com (10.127.129.5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Wed, 30 Oct 2024 14:35:14 -0700 Received: from drhqmail202.nvidia.com (10.126.190.181) by drhqmail203.nvidia.com (10.126.190.182) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Wed, 30 Oct 2024 14:35:13 -0700 Received: from Asurada-Nvidia.nvidia.com (10.127.8.12) by mail.nvidia.com (10.126.190.181) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Wed, 30 Oct 2024 14:35:12 -0700 From: Nicolin Chen To: , , CC: , , , , , , , , , , , , , , , , , , , , Subject: [PATCH v6 13/13] Documentation: userspace-api: iommufd: Update vIOMMU Date: Wed, 30 Oct 2024 14:34:39 -0700 Message-ID: <621bc7a4f727b5563b01985e86969a7c8131c6b2.1730313237.git.nicolinc@nvidia.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SN1PEPF0002BA4E:EE_|MN2PR12MB4344:EE_ X-MS-Office365-Filtering-Correlation-Id: 2d908eaf-d577-429a-4728-08dcf92ac7a4 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|1800799024|7416014|82310400026|36860700013; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?zcQ3ceeC5vcMpgl7z5IQNm0YTu3LB5uPEQ2nmIOdvkFBdIAZPj1YFVwZm21o?= =?us-ascii?Q?YXkO/vcuKxVow9aEkkcSNY+rsXGka2mdAgDiFzKwOTVJ1kKLnODkqUZgO7Yk?= =?us-ascii?Q?5HHMmFrpyDsN8kaSEgaCCqwglW0bUs1/hbJsBjqTGmszY1wVkChWoPqi58L+?= =?us-ascii?Q?rhGX+eD+sF7858asTmphK8EOfbplpj9bwSy64GtHCpa4O3xY8e1f4BPYkWHK?= =?us-ascii?Q?HuO8VrG8e/xwQsaJU0HA2q1251tR5rle1dBi8/YsVEYGyPrxCxcfB33GpF1d?= =?us-ascii?Q?iliWr25+jHZWQB/AD3lAD0J7aI+b6S72hThKv9VmAYwdc8/KjPx3TmSI3oJp?= =?us-ascii?Q?2jse3sEc5jCNVGgusXcVsBCn49v4vBXVYVmh1wStb3rEjRMrJFWNqDPUHhJF?= =?us-ascii?Q?3oY+mkM0PW/vErDZ/YAYHiTpMnj4A1Kk0BoFldyV4S6dmtVzITonG0hOoqp2?= =?us-ascii?Q?ub4BsAWDG9PnSpgEoR6sh+/d16IqRGWd6J3co6su2y8R1Ldr7IoO5dFBbfam?= =?us-ascii?Q?uJVhY779Ptd78ItFVq8bK4frdxvFjIwDbKje8yjAeAYQWQn8dP7YIB93hiwc?= =?us-ascii?Q?7JLGp470Gdb5gJGmbrU/NLDV4pSaUI2hWqiiuK7VEz4EBxT78SGkjLEvcj2z?= =?us-ascii?Q?EYs8mh63dZpannt5wtv2Bqs48pVNeYAs6lhgLLJeUfAImM+DZm/PMjYIUNZg?= =?us-ascii?Q?G7WLvjKunpMfnxnjrohoLGrtli+CsfPhDHnKbb6yHS4myUePf1ShblKVF0LR?= =?us-ascii?Q?UZ1dylQtYuY3BAvIV5fw5QDtMERoAYaoMguD7WwjuN/dUXnQyPVKlSWYN/VZ?= =?us-ascii?Q?vFi16ExJcCjjvWGW37LfcEki3G295aqhLlbTnr6NRpXNjjVwU0HxcwEQjB5x?= =?us-ascii?Q?ZEqs+2nJc+M57E79OgpT6kOTdLCwM0ggxZuWWYT01wnXqlpzovxoh58+5Spq?= =?us-ascii?Q?CM6/C0Abgb1STDKTK+qasZqB7pEk42WAjN58GJ4zAWhx68VD7kiteGenX3AH?= =?us-ascii?Q?otZ06UhzqQvWuRsmOIWWCBnDukNLvrCWd2FJzZZm1TtvgjkdjGp6VsSdPmJ5?= =?us-ascii?Q?8oC2TDbTpGku31qbEuirl/sGy1EtEhcDteuzcVI4UCj4CBRMrhkyrKehJEbB?= =?us-ascii?Q?qWBom0pnIPAQ+POPKXcCDAhJN2qd07IuF1cx+HXENutaOsIv/L9t4VqaVY5b?= =?us-ascii?Q?kZA2D//Av7f7m/yC4aGrZVrGF9ix2drPvHJGzIVsZLc/8S0FmZVS0Ysnqz2R?= =?us-ascii?Q?5UaDIzvKKGJu7GLYIGFzSqIGw4KloMe/Lv4Ac3G9JBEqd70PsxHM7atHIPDN?= =?us-ascii?Q?roe/oUZ6vfUIN5JesaMEyHFbaoVST/GLmWclvHTmr7jtQM3lOD+pL26kOC8s?= =?us-ascii?Q?/cQ0Bj8Ldf0Pjrwxfq1rB2ZpF2SfjF8JxTcQaxA+bpjaDxq8mQ=3D=3D?= X-Forefront-Antispam-Report: CIP:216.228.118.232;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc7edge1.nvidia.com;CAT:NONE;SFS:(13230040)(376014)(1800799024)(7416014)(82310400026)(36860700013);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Oct 2024 21:35:31.1892 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 2d908eaf-d577-429a-4728-08dcf92ac7a4 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.118.232];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: SN1PEPF0002BA4E.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4344 Content-Type: text/plain; charset="utf-8" With the introduction of the new object and its infrastructure, update the doc to reflect that and add a new graph. Reviewed-by: Jason Gunthorpe Reviewed-by: Kevin Tian Signed-off-by: Nicolin Chen --- Documentation/userspace-api/iommufd.rst | 69 ++++++++++++++++++++++++- 1 file changed, 68 insertions(+), 1 deletion(-) diff --git a/Documentation/userspace-api/iommufd.rst b/Documentation/usersp= ace-api/iommufd.rst index 2deba93bf159..a8b7766c2849 100644 --- a/Documentation/userspace-api/iommufd.rst +++ b/Documentation/userspace-api/iommufd.rst @@ -63,6 +63,37 @@ Following IOMMUFD objects are exposed to userspace: space usually has mappings from guest-level I/O virtual addresses to gue= st- level physical addresses. =20 +- IOMMUFD_OBJ_VIOMMU, representing a slice of the physical IOMMU instance, + passed to or shared with a VM. It may be some HW-accelerated virtualizat= ion + features and some SW resources used by the VM. For examples: + * Security namespace for guest owned ID, e.g. guest-controlled cache tags + * Non-device-affiliated event reporting, e.g. invalidation queue errors + * Access to a sharable nesting parent pagetable across physical IOMMUs + * Virtualization of various platforms IDs, e.g. RIDs and others + * Delivery of paravirtualized invalidation + * Direct assigned invalidation queues + * Direct assigned interrupts + Such a vIOMMU object generally has the access to a nesting parent pageta= ble + to support some HW-accelerated virtualization features. So, a vIOMMU obj= ect + must be created given a nesting parent HWPT_PAGING object, and then it w= ould + encapsulate that HWPT_PAGING object. Therefore, a vIOMMU object can be u= sed + to allocate an HWPT_NESTED object in place of the encapsulated HWPT_PAGI= NG. + + .. note:: + + The name "vIOMMU" isn't necessarily identical to a virtualized IOMMU = in a + VM. A VM can have one giant virtualized IOMMU running on a machine ha= ving + multiple physical IOMMUs, in which case the VMM will dispatch the req= uests + or configurations from this single virtualized IOMMU instance to mult= iple + vIOMMU objects created for individual slices of different physical IO= MMUs. + In other words, a vIOMMU object is always a representation of one phy= sical + IOMMU, not necessarily of a virtualized IOMMU. For VMMs that want the= full + virtualization features from physical IOMMUs, it is suggested to buil= d the + same number of virtualized IOMMUs as the number of physical IOMMUs, s= o the + passed-through devices would be connected to their own virtualized IO= MMUs + backed by corresponding vIOMMU objects, in which case a guest OS woul= d do + the "dispatch" naturally instead of VMM trappings. + All user-visible objects are destroyed via the IOMMU_DESTROY uAPI. =20 The diagrams below show relationships between user-visible objects and ker= nel @@ -101,6 +132,28 @@ creating the objects and links:: |------------>|iommu_domain|<----|iommu_domain|<----|device| |____________| |____________| |______| =20 + _______________________________________________________________________ + | iommufd (with vIOMMU) | + | | + | [5] | + | _____________ | + | | | | + | |----------------| vIOMMU | | + | | | | | + | | | | | + | | [1] | | [4] [2] | + | | ______ | | _____________ ________ | + | | | | | [3] | | | | | | + | | | IOAS |<---|(HWPT_PAGING)|<---| HWPT_NESTED |<--| DEVICE | | + | | |______| |_____________| |_____________| |________| | + | | | | | | | + |______|________|______________|__________________|_______________|_____| + | | | | | + ______v_____ | ______v_____ ______v_____ ___v__ + | struct | | PFN | (paging) | | (nested) | |struct| + |iommu_device| |------>|iommu_domain|<----|iommu_domain|<----|device| + |____________| storage|____________| |____________| |______| + 1. IOMMUFD_OBJ_IOAS is created via the IOMMU_IOAS_ALLOC uAPI. An iommufd c= an hold multiple IOAS objects. IOAS is the most generic object and does not expose interfaces that are specific to single IOMMU drivers. All operat= ions @@ -132,7 +185,8 @@ creating the objects and links:: flag is set. =20 4. IOMMUFD_OBJ_HWPT_NESTED can be only manually created via the IOMMU_HWPT= _ALLOC - uAPI, provided an hwpt_id via @pt_id to associate the new HWPT_NESTED o= bject + uAPI, provided an hwpt_id or a viommu_id of a vIOMMU object encapsulati= ng a + nesting parent HWPT_PAGING via @pt_id to associate the new HWPT_NESTED = object to the corresponding HWPT_PAGING object. The associating HWPT_PAGING ob= ject must be a nesting parent manually allocated via the same uAPI previousl= y with an IOMMU_HWPT_ALLOC_NEST_PARENT flag, otherwise the allocation will fai= l. The @@ -149,6 +203,18 @@ creating the objects and links:: created via the same IOMMU_HWPT_ALLOC uAPI. The difference is at the= type of the object passed in via the @pt_id field of struct iommufd_hwpt_= alloc. =20 +5. IOMMUFD_OBJ_VIOMMU can be only manually created via the IOMMU_VIOMMU_AL= LOC + uAPI, provided a dev_id (for the device's physical IOMMU to back the vI= OMMU) + and an hwpt_id (to associate the vIOMMU to a nesting parent HWPT_PAGING= ). The + iommufd core will link the vIOMMU object to the struct iommu_device tha= t the + struct device is behind. And an IOMMU driver can implement a viommu_all= oc op + to allocate its own vIOMMU data structure embedding the core-level stru= cture + iommufd_viommu and some driver-specific data. If necessary, the driver = can + also configure its HW virtualization feature for that vIOMMU (and thus = for + the VM). Successful completion of this operation sets up the linkages b= etween + the vIOMMU object and the HWPT_PAGING, then this vIOMMU object can be u= sed + as a nesting parent object to allocate an HWPT_NESTED object described = above. + A device can only bind to an iommufd due to DMA ownership claim and attach= to at most one IOAS object (no support of PASID yet). =20 @@ -161,6 +227,7 @@ User visible objects are backed by following datastruct= ures: - iommufd_device for IOMMUFD_OBJ_DEVICE. - iommufd_hwpt_paging for IOMMUFD_OBJ_HWPT_PAGING. - iommufd_hwpt_nested for IOMMUFD_OBJ_HWPT_NESTED. +- iommufd_viommu for IOMMUFD_OBJ_VIOMMU. =20 Several terminologies when looking at these datastructures: =20 --=20 2.43.0