From nobody Thu Dec 18 14:25:06 2025 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2061.outbound.protection.outlook.com [40.107.236.61]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8C7C41A5B90 for ; Tue, 25 Mar 2025 04:05:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.236.61 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1742875552; cv=fail; b=XGr5m075oDYhLc3+qdunYYceOZ6i9JBt1baOj/lyTv6qDfG+1AcwWvKfs8LKc+R2cejDSEVfS6tGUJB+xAILE4Bf2t9SdJlxLGCAavyK+T7gHLINjrZF0JPQ17//lOrX2Qt50ud9aaCBblVMX3tqF6BVncnnUAjm24RIFMbnhL4= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1742875552; c=relaxed/simple; bh=KMc3Z9QtcvPaiT4Lb6RMUbyhGMK/tB94EGHDSl9nqNs=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=S9AxmKuiJy5YHTZthGows5UyRWvW2X8n9LRUXORRQTdvXMA/gkrOeLTcK+698T99NaUX08K/Ma4PpqdsnASO8+xIGUeXvPJIGVcp0+SAhpKbcPewsTneBiG0sMcLs+amS2Dk6vFtUSc2h/xb4IOMEPZ3xdyDZgwY0nMHiPoe7EY= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=PdMlJzxf; arc=fail smtp.client-ip=40.107.236.61 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="PdMlJzxf" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=U7smCjdADMsQgxK9whcuf301Pp5Kjqi7xYKmqddE6MyrgX7b2MH0lPWN4MnxXAVvp4YyHqtk1u+Ue/lzSVkOEoQ6Ax4p5NfmtYJ5ssXdgzVIkF8spF9Wa29mbdvE0uX1elDykqb0TB/LJ5C6DAnB9I9AeksnzZUeHYtH2AoeoKRWEBZCa+KP5cEdTS4w8y+IZGOYBTKSFOjp9wCMJSjojdVcClYpR902T9o+y4wAa06Acpf8J89IuIdOpAj9bf19H4OWbpObDUDwjp7wq99fARpXj8ky7ss+fIKG7q/DvetL+C81aaVMYBA5ZXUS4Ol+BTVCCNaOHOoU5Suh3nFy5Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=y7nRM4Cmqibc1ea2+t61wNNfm0QT+iik+n3hUCIfeCo=; b=PSM2enXF9bt1lMF3SFef9Abaw8GVzrbTDmln7D2AYI0/Wd/c+K0VM3jauKb/cf95s4A7USQCbzJ++AStsGDWSTG+63fLDLeT8oZZ1GxBCFY8JhulMyOVl9bSNFeW5OG+qd44bAyBiXsx9oeWaPS3q9tEUPxaA/U7tPLWKDN5e+mhS3yscCZUABSKW/jz7ceFQ2gUO6eTxK76TDDtyGru6Tb0ozyOFDdqig5FfhBcMUW0CJvep7Xudik3xf88tSxVrkDtRoErQh3VCde5Za5/nyAMGFa3cKf49XOeCZX0TjTzB/cwRlogY4E7fV8gmF2RbPs2Vf8Ld/T8Fu1H2nPC6A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=arm.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=y7nRM4Cmqibc1ea2+t61wNNfm0QT+iik+n3hUCIfeCo=; b=PdMlJzxf9157EBwMNkK+f0yyZqaT80kzbk4pcWTlW5MnfoyIiYmqt2O7Vt34cZqupaLbo31ArKZkbRb2HT8zsDulPwRb0HZm15gJUraolDASQlfaqMyzMT3KQ96B/6qMyJEGz88s8rfO4rwi+9FVpnjgmBmdmId74E/XxnH0EHDU82sCZoD2Z5kSrK18qNj5P1szjiMsJP7UU9PgUtdWv2EXHY2ntbI4EFkb1zKOmiJvfyFR1obJkwLFSq/aMJn1jziGvaMqLWfszQW+zUKXJ7kzlUDo2pma/4MLB9ipfII1GcErq0urEFhu+TjO3TNMSmL3MpySiw9AmlPlfNdCwQ== Received: from SJ0PR03CA0219.namprd03.prod.outlook.com (2603:10b6:a03:39f::14) by DS7PR12MB8274.namprd12.prod.outlook.com (2603:10b6:8:da::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8534.42; Tue, 25 Mar 2025 04:05:43 +0000 Received: from CO1PEPF000075EE.namprd03.prod.outlook.com (2603:10b6:a03:39f:cafe::f2) by SJ0PR03CA0219.outlook.office365.com (2603:10b6:a03:39f::14) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8534.42 via Frontend Transport; Tue, 25 Mar 2025 04:05:43 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by CO1PEPF000075EE.mail.protection.outlook.com (10.167.249.37) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8534.20 via Frontend Transport; Tue, 25 Mar 2025 04:05:42 +0000 Received: from rnnvmail202.nvidia.com (10.129.68.7) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Mon, 24 Mar 2025 21:05:27 -0700 Received: from rnnvmail202.nvidia.com (10.129.68.7) by rnnvmail202.nvidia.com (10.129.68.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.14; Mon, 24 Mar 2025 21:05:27 -0700 Received: from Asurada-Nvidia.nvidia.com (10.127.8.13) by mail.nvidia.com (10.129.68.7) with Microsoft SMTP Server id 15.2.1544.14 via Frontend Transport; Mon, 24 Mar 2025 21:05:26 -0700 From: Nicolin Chen To: CC: , , , , , , , , Subject: [PATCH v5 1/3] iommu: Sort out domain user data Date: Mon, 24 Mar 2025 21:05:15 -0700 Message-ID: <1ace9076c95204bbe193ee77499d395f15f44b23.1742871535.git.nicolinc@nvidia.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: AnonymousSubmission X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1PEPF000075EE:EE_|DS7PR12MB8274:EE_ X-MS-Office365-Filtering-Correlation-Id: 924a3587-069a-418e-76cf-08dd6b524fd7 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|36860700013|1800799024|376014|82310400026|7053199007; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?Kt9/gBgKSTj+hmZfjSxiR7t409eYQSmMQ27AhdUOpGPGDnwmqIbeOgr7ZhHs?= =?us-ascii?Q?pzmsiiGwcJmqqRYkLfvYbxC7I8QgXinfq8RiimStt1IAA/71fi/sC0qhEU9E?= =?us-ascii?Q?sAU8yKmtHcaYyKLVS8z4RC0MMIgGg728myoaXWWhg8XjfLeC6bH+Z6tAuWqL?= =?us-ascii?Q?mHpL7ElbNNYtoKiZdVamulAw8k3hPkZEZfDdxV0KNlRQDGzOVvAH68Iot+Cn?= =?us-ascii?Q?QsHJGihQsts9EcjkLWVoI2GqLqZOdX0ReaO85kqP/i5PssCjpSqffGlL5T1l?= =?us-ascii?Q?DcwUKHKfKWOTjkmSbPW4gbe2hllirMAPEU3DGFngIu17i7WM0K21CMQeF2zd?= =?us-ascii?Q?mTPyT+TDJ4Em2pvZ+VjY3jz4nx3SYbeRbqTz1TMvYY/Zod1AhKajGp1oWyrt?= =?us-ascii?Q?Y6yHcxAYL6+aCvhZQKS/4h86oL9OVhvevnV9l7e+EK8DlHr6MIfJxg/iJAT/?= =?us-ascii?Q?YbiRgqK49aorSya1umhTTg0hwjujrKBlYqnXHsW167EjctdfcPBoBtTX1Djv?= =?us-ascii?Q?4YwwrUiZnMLOZSv5aDdAzngG3vYYw78r8XtHPXR4jBnR1o/lS76mKTSoixHk?= =?us-ascii?Q?ViZaIIjPcReotvkqCPWhNuC/gu+z6AL/n/KQRi8AJXnwwwwwUCmacs24EHzR?= =?us-ascii?Q?RrQo/Gje+jeloj/poCz5bNdq7ks8jz7p1uHgCTrjWACA6cw6fPkjK+h3cvr+?= =?us-ascii?Q?u46IPRIicXpPuRTkubPpMJAZ4e0yxkPk4GF28a3pRMm39OYcJl0hAeNZLwwC?= =?us-ascii?Q?aTUSUFHMyzHO8RPU8XsF1eGUBJsIMq7Q65gckvyCxkWZwT2U8njp/doL23TE?= =?us-ascii?Q?n/ZIMLg0AJSRpvXxlnfO0J0yuHNd6ztN4obCpfhoj/4wNIUgMwh+HYTG7EfZ?= =?us-ascii?Q?NDmhAhi6Gzq5V82yME2EEgsOK9DUQoHUwAXOG/kfvP8yaP7aofYwQfSHyz/n?= =?us-ascii?Q?LunQoDlLpAY1WrOWecfnYpb5eMtriUL9MmrsYC0B7bQR03DyjkTMrohSj2j9?= =?us-ascii?Q?0bEXc1nVLXfFiQ/GKeJhOty56CP8xSbRb+bDDQun23iK6csfekOJopV0gqTU?= =?us-ascii?Q?IpfnMw9wkiO24ZuOPMLraFIdZVvaihwuTIUf61/xmB75Ef0w7RSttzPzNqM/?= =?us-ascii?Q?ZZbGMcLdY+8rx/nRnRlFB2++RbtklUi40HLRTSlWiRMrL8/PBrkC3/8vjPRr?= =?us-ascii?Q?IO1T6pnB6aBNKmjFw6DMWGke97VixvE+JcQFd7zCFVxXTcFmtkdl4YQdrBoR?= =?us-ascii?Q?B3vq3vjW03mY0YY7r1pVnPWOlP6/lwQFc8lCZtpG7bjnk+eH6o6YJmSSbJ3q?= =?us-ascii?Q?PUKrp2vzXfEmx7IyleIsk+E/5c399NDVQl8yVjE2EOMMH1vggGgtxxi12dWa?= =?us-ascii?Q?CuTp1DBiDudqQlWhhGFJXkyILQbLYOOdbulbCH0zCFITy4txE7NNE0YRuq2Z?= =?us-ascii?Q?98fbCxJl8P3FqZGw24IP1kVtdTMbGDvwXLx+hfk6QazkFE7xmNbCcpRn0M7L?= =?us-ascii?Q?ksde8jKj8Vur5NU=3D?= X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(36860700013)(1800799024)(376014)(82310400026)(7053199007);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Mar 2025 04:05:42.6551 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 924a3587-069a-418e-76cf-08dd6b524fd7 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1PEPF000075EE.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR12MB8274 Content-Type: text/plain; charset="utf-8" From: Robin Murphy When DMA/MSI cookies were made first-class citizens back in commit 46983fcd67ac ("iommu: Pull IOVA cookie management into the core"), there was no real need to further expose the two different cookie types. However, now that IOMMUFD wants to add a third type of MSI-mapping cookie, we do have a nicely compelling reason to properly dismabiguate things at the domain level beyond just vaguely guessing from the domain type. Meanwhile, we also effectively have another "cookie" in the form of the anonymous union for other user data, which isn't much better in terms of being vague and unenforced. The fact is that all these cookie types are mutually exclusive, in the sense that combining them makes zero sense and/or would be catastrophic (iommu_set_fault_handler() on an SVA domain, anyone?) - the only combination which *might* be reasonable is perhaps a fault handler and an MSI cookie, but nobody's doing that at the moment, so let's rule it out as well for the sake of being clear and robust. To that end, we pull DMA and MSI cookies apart a little more, mostly to clear up the ambiguity at domain teardown, then for clarity (and to save a little space), move them into the union, whose ownership we can then properly describe and enforce entirely unambiguously. [nicolinc: rebase on latest tree; use prefix IOMMU_COOKIE_; merge unions in iommu_domain; add IOMMU_COOKIE_IOMMUFD for iommufd_hwpt] Signed-off-by: Robin Murphy Reviewed-by: Kevin Tian Signed-off-by: Nicolin Chen Reviewed-by: Jason Gunthorpe --- drivers/iommu/dma-iommu.h | 5 + include/linux/iommu.h | 20 ++- drivers/iommu/dma-iommu.c | 194 ++++++++++++++------------- drivers/iommu/iommu-sva.c | 1 + drivers/iommu/iommu.c | 18 ++- drivers/iommu/iommufd/hw_pagetable.c | 3 + 6 files changed, 143 insertions(+), 98 deletions(-) diff --git a/drivers/iommu/dma-iommu.h b/drivers/iommu/dma-iommu.h index c12d63457c76..9cca11806e5d 100644 --- a/drivers/iommu/dma-iommu.h +++ b/drivers/iommu/dma-iommu.h @@ -13,6 +13,7 @@ void iommu_setup_dma_ops(struct device *dev); =20 int iommu_get_dma_cookie(struct iommu_domain *domain); void iommu_put_dma_cookie(struct iommu_domain *domain); +void iommu_put_msi_cookie(struct iommu_domain *domain); =20 int iommu_dma_init_fq(struct iommu_domain *domain); =20 @@ -40,6 +41,10 @@ static inline void iommu_put_dma_cookie(struct iommu_dom= ain *domain) { } =20 +static inline void iommu_put_msi_cookie(struct iommu_domain *domain) +{ +} + static inline void iommu_dma_get_resv_regions(struct device *dev, struct l= ist_head *list) { } diff --git a/include/linux/iommu.h b/include/linux/iommu.h index e93d2e918599..06cc14e9993d 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -41,6 +41,7 @@ struct iommu_dirty_ops; struct notifier_block; struct iommu_sva; struct iommu_dma_cookie; +struct iommu_dma_msi_cookie; struct iommu_fault_param; struct iommufd_ctx; struct iommufd_viommu; @@ -165,6 +166,15 @@ struct iommu_domain_geometry { bool force_aperture; /* DMA only allowed in mappable range? */ }; =20 +enum iommu_domain_cookie_type { + IOMMU_COOKIE_NONE, + IOMMU_COOKIE_DMA_IOVA, + IOMMU_COOKIE_DMA_MSI, + IOMMU_COOKIE_FAULT_HANDLER, + IOMMU_COOKIE_SVA, + IOMMU_COOKIE_IOMMUFD, +}; + /* Domain feature flags */ #define __IOMMU_DOMAIN_PAGING (1U << 0) /* Support for iommu_map/unmap */ #define __IOMMU_DOMAIN_DMA_API (1U << 1) /* Domain for use in DMA-API @@ -211,12 +221,12 @@ struct iommu_domain_geometry { =20 struct iommu_domain { unsigned type; + enum iommu_domain_cookie_type cookie_type; const struct iommu_domain_ops *ops; const struct iommu_dirty_ops *dirty_ops; const struct iommu_ops *owner; /* Whose domain_alloc we came from */ unsigned long pgsize_bitmap; /* Bitmap of page sizes in use */ struct iommu_domain_geometry geometry; - struct iommu_dma_cookie *iova_cookie; int (*iopf_handler)(struct iopf_group *group); =20 #if IS_ENABLED(CONFIG_IRQ_MSI_IOMMU) @@ -224,10 +234,10 @@ struct iommu_domain { phys_addr_t msi_addr); #endif =20 - union { /* Pointer usable by owner of the domain */ - struct iommufd_hw_pagetable *iommufd_hwpt; /* iommufd */ - }; - union { /* Fault handler */ + union { /* cookie */ + struct iommu_dma_cookie *iova_cookie; + struct iommu_dma_msi_cookie *msi_cookie; + struct iommufd_hw_pagetable *iommufd_hwpt; struct { iommu_fault_handler_t handler; void *handler_token; diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c index 94263ed2c564..31a7b4b81656 100644 --- a/drivers/iommu/dma-iommu.c +++ b/drivers/iommu/dma-iommu.c @@ -42,11 +42,6 @@ struct iommu_dma_msi_page { phys_addr_t phys; }; =20 -enum iommu_dma_cookie_type { - IOMMU_DMA_IOVA_COOKIE, - IOMMU_DMA_MSI_COOKIE, -}; - enum iommu_dma_queue_type { IOMMU_DMA_OPTS_PER_CPU_QUEUE, IOMMU_DMA_OPTS_SINGLE_QUEUE, @@ -59,35 +54,31 @@ struct iommu_dma_options { }; =20 struct iommu_dma_cookie { - enum iommu_dma_cookie_type type; + struct iova_domain iovad; + struct list_head msi_page_list; + /* Flush queue */ union { - /* Full allocator for IOMMU_DMA_IOVA_COOKIE */ - struct { - struct iova_domain iovad; - /* Flush queue */ - union { - struct iova_fq *single_fq; - struct iova_fq __percpu *percpu_fq; - }; - /* Number of TLB flushes that have been started */ - atomic64_t fq_flush_start_cnt; - /* Number of TLB flushes that have been finished */ - atomic64_t fq_flush_finish_cnt; - /* Timer to regularily empty the flush queues */ - struct timer_list fq_timer; - /* 1 when timer is active, 0 when not */ - atomic_t fq_timer_on; - }; - /* Trivial linear page allocator for IOMMU_DMA_MSI_COOKIE */ - dma_addr_t msi_iova; + struct iova_fq *single_fq; + struct iova_fq __percpu *percpu_fq; }; - struct list_head msi_page_list; - + /* Number of TLB flushes that have been started */ + atomic64_t fq_flush_start_cnt; + /* Number of TLB flushes that have been finished */ + atomic64_t fq_flush_finish_cnt; + /* Timer to regularily empty the flush queues */ + struct timer_list fq_timer; + /* 1 when timer is active, 0 when not */ + atomic_t fq_timer_on; /* Domain for flush queue callback; NULL if flush queue not in use */ - struct iommu_domain *fq_domain; + struct iommu_domain *fq_domain; /* Options for dma-iommu use */ - struct iommu_dma_options options; - struct mutex mutex; + struct iommu_dma_options options; + struct mutex mutex; +}; + +struct iommu_dma_msi_cookie { + dma_addr_t msi_iova; + struct list_head msi_page_list; }; =20 static DEFINE_STATIC_KEY_FALSE(iommu_deferred_attach_enabled); @@ -369,40 +360,26 @@ int iommu_dma_init_fq(struct iommu_domain *domain) return 0; } =20 -static inline size_t cookie_msi_granule(struct iommu_dma_cookie *cookie) -{ - if (cookie->type =3D=3D IOMMU_DMA_IOVA_COOKIE) - return cookie->iovad.granule; - return PAGE_SIZE; -} - -static struct iommu_dma_cookie *cookie_alloc(enum iommu_dma_cookie_type ty= pe) -{ - struct iommu_dma_cookie *cookie; - - cookie =3D kzalloc(sizeof(*cookie), GFP_KERNEL); - if (cookie) { - INIT_LIST_HEAD(&cookie->msi_page_list); - cookie->type =3D type; - } - return cookie; -} - /** * iommu_get_dma_cookie - Acquire DMA-API resources for a domain * @domain: IOMMU domain to prepare for DMA-API usage */ int iommu_get_dma_cookie(struct iommu_domain *domain) { - if (domain->iova_cookie) + struct iommu_dma_cookie *cookie; + + if (domain->cookie_type !=3D IOMMU_COOKIE_NONE) return -EEXIST; =20 - domain->iova_cookie =3D cookie_alloc(IOMMU_DMA_IOVA_COOKIE); - if (!domain->iova_cookie) + cookie =3D kzalloc(sizeof(*cookie), GFP_KERNEL); + if (!cookie) return -ENOMEM; =20 - mutex_init(&domain->iova_cookie->mutex); + mutex_init(&cookie->mutex); + INIT_LIST_HEAD(&cookie->msi_page_list); iommu_domain_set_sw_msi(domain, iommu_dma_sw_msi); + domain->cookie_type =3D IOMMU_COOKIE_DMA_IOVA; + domain->iova_cookie =3D cookie; return 0; } =20 @@ -420,29 +397,30 @@ int iommu_get_dma_cookie(struct iommu_domain *domain) */ int iommu_get_msi_cookie(struct iommu_domain *domain, dma_addr_t base) { - struct iommu_dma_cookie *cookie; + struct iommu_dma_msi_cookie *cookie; =20 if (domain->type !=3D IOMMU_DOMAIN_UNMANAGED) return -EINVAL; =20 - if (domain->iova_cookie) + if (domain->cookie_type !=3D IOMMU_COOKIE_NONE) return -EEXIST; =20 - cookie =3D cookie_alloc(IOMMU_DMA_MSI_COOKIE); + cookie =3D kzalloc(sizeof(*cookie), GFP_KERNEL); if (!cookie) return -ENOMEM; =20 cookie->msi_iova =3D base; - domain->iova_cookie =3D cookie; + INIT_LIST_HEAD(&cookie->msi_page_list); iommu_domain_set_sw_msi(domain, iommu_dma_sw_msi); + domain->cookie_type =3D IOMMU_COOKIE_DMA_MSI; + domain->msi_cookie =3D cookie; return 0; } EXPORT_SYMBOL(iommu_get_msi_cookie); =20 /** * iommu_put_dma_cookie - Release a domain's DMA mapping resources - * @domain: IOMMU domain previously prepared by iommu_get_dma_cookie() or - * iommu_get_msi_cookie() + * @domain: IOMMU domain previously prepared by iommu_get_dma_cookie() */ void iommu_put_dma_cookie(struct iommu_domain *domain) { @@ -454,20 +432,27 @@ void iommu_put_dma_cookie(struct iommu_domain *domain) return; #endif =20 - if (!cookie) - return; - - if (cookie->type =3D=3D IOMMU_DMA_IOVA_COOKIE && cookie->iovad.granule) { + if (cookie->iovad.granule) { iommu_dma_free_fq(cookie); put_iova_domain(&cookie->iovad); } + list_for_each_entry_safe(msi, tmp, &cookie->msi_page_list, list) + kfree(msi); + kfree(cookie); +} =20 - list_for_each_entry_safe(msi, tmp, &cookie->msi_page_list, list) { - list_del(&msi->list); +/** + * iommu_put_msi_cookie - Release a domain's MSI mapping resources + * @domain: IOMMU domain previously prepared by iommu_get_msi_cookie() + */ +void iommu_put_msi_cookie(struct iommu_domain *domain) +{ + struct iommu_dma_msi_cookie *cookie =3D domain->msi_cookie; + struct iommu_dma_msi_page *msi, *tmp; + + list_for_each_entry_safe(msi, tmp, &cookie->msi_page_list, list) kfree(msi); - } kfree(cookie); - domain->iova_cookie =3D NULL; } =20 /** @@ -687,7 +672,7 @@ static int iommu_dma_init_domain(struct iommu_domain *d= omain, struct device *dev struct iova_domain *iovad; int ret; =20 - if (!cookie || cookie->type !=3D IOMMU_DMA_IOVA_COOKIE) + if (!cookie || domain->cookie_type !=3D IOMMU_COOKIE_DMA_IOVA) return -EINVAL; =20 iovad =3D &cookie->iovad; @@ -777,9 +762,9 @@ static dma_addr_t iommu_dma_alloc_iova(struct iommu_dom= ain *domain, struct iova_domain *iovad =3D &cookie->iovad; unsigned long shift, iova_len, iova; =20 - if (cookie->type =3D=3D IOMMU_DMA_MSI_COOKIE) { - cookie->msi_iova +=3D size; - return cookie->msi_iova - size; + if (domain->cookie_type =3D=3D IOMMU_COOKIE_DMA_MSI) { + domain->msi_cookie->msi_iova +=3D size; + return domain->msi_cookie->msi_iova - size; } =20 shift =3D iova_shift(iovad); @@ -816,16 +801,16 @@ static dma_addr_t iommu_dma_alloc_iova(struct iommu_d= omain *domain, return (dma_addr_t)iova << shift; } =20 -static void iommu_dma_free_iova(struct iommu_dma_cookie *cookie, - dma_addr_t iova, size_t size, struct iommu_iotlb_gather *gather) +static void iommu_dma_free_iova(struct iommu_domain *domain, dma_addr_t io= va, + size_t size, struct iommu_iotlb_gather *gather) { - struct iova_domain *iovad =3D &cookie->iovad; + struct iova_domain *iovad =3D &domain->iova_cookie->iovad; =20 /* The MSI case is only ever cleaning up its most recent allocation */ - if (cookie->type =3D=3D IOMMU_DMA_MSI_COOKIE) - cookie->msi_iova -=3D size; + if (domain->cookie_type =3D=3D IOMMU_COOKIE_DMA_MSI) + domain->msi_cookie->msi_iova -=3D size; else if (gather && gather->queued) - queue_iova(cookie, iova_pfn(iovad, iova), + queue_iova(domain->iova_cookie, iova_pfn(iovad, iova), size >> iova_shift(iovad), &gather->freelist); else @@ -853,7 +838,7 @@ static void __iommu_dma_unmap(struct device *dev, dma_a= ddr_t dma_addr, =20 if (!iotlb_gather.queued) iommu_iotlb_sync(domain, &iotlb_gather); - iommu_dma_free_iova(cookie, dma_addr, size, &iotlb_gather); + iommu_dma_free_iova(domain, dma_addr, size, &iotlb_gather); } =20 static dma_addr_t __iommu_dma_map(struct device *dev, phys_addr_t phys, @@ -881,7 +866,7 @@ static dma_addr_t __iommu_dma_map(struct device *dev, p= hys_addr_t phys, return DMA_MAPPING_ERROR; =20 if (iommu_map(domain, iova, phys - iova_off, size, prot, GFP_ATOMIC)) { - iommu_dma_free_iova(cookie, iova, size, NULL); + iommu_dma_free_iova(domain, iova, size, NULL); return DMA_MAPPING_ERROR; } return iova + iova_off; @@ -1018,7 +1003,7 @@ static struct page **__iommu_dma_alloc_noncontiguous(= struct device *dev, out_free_sg: sg_free_table(sgt); out_free_iova: - iommu_dma_free_iova(cookie, iova, size, NULL); + iommu_dma_free_iova(domain, iova, size, NULL); out_free_pages: __iommu_dma_free_pages(pages, count); return NULL; @@ -1495,7 +1480,7 @@ int iommu_dma_map_sg(struct device *dev, struct scatt= erlist *sg, int nents, return __finalise_sg(dev, sg, nents, iova); =20 out_free_iova: - iommu_dma_free_iova(cookie, iova, iova_len, NULL); + iommu_dma_free_iova(domain, iova, iova_len, NULL); out_restore_sg: __invalidate_sg(sg, nents); out: @@ -1773,17 +1758,47 @@ void iommu_setup_dma_ops(struct device *dev) dev->dma_iommu =3D false; } =20 +static bool has_msi_cookie(const struct iommu_domain *domain) +{ + return domain && (domain->cookie_type =3D=3D IOMMU_COOKIE_DMA_IOVA || + domain->cookie_type =3D=3D IOMMU_COOKIE_DMA_MSI); +} + +static size_t cookie_msi_granule(const struct iommu_domain *domain) +{ + switch (domain->cookie_type) { + case IOMMU_COOKIE_DMA_IOVA: + return domain->iova_cookie->iovad.granule; + case IOMMU_COOKIE_DMA_MSI: + return PAGE_SIZE; + default: + unreachable(); + }; +} + +static struct list_head *cookie_msi_pages(const struct iommu_domain *domai= n) +{ + switch (domain->cookie_type) { + case IOMMU_COOKIE_DMA_IOVA: + return &domain->iova_cookie->msi_page_list; + case IOMMU_COOKIE_DMA_MSI: + return &domain->msi_cookie->msi_page_list; + default: + unreachable(); + }; +} + static struct iommu_dma_msi_page *iommu_dma_get_msi_page(struct device *de= v, phys_addr_t msi_addr, struct iommu_domain *domain) { - struct iommu_dma_cookie *cookie =3D domain->iova_cookie; + struct list_head *msi_page_list =3D cookie_msi_pages(domain); struct iommu_dma_msi_page *msi_page; dma_addr_t iova; int prot =3D IOMMU_WRITE | IOMMU_NOEXEC | IOMMU_MMIO; - size_t size =3D cookie_msi_granule(cookie); + size_t size =3D cookie_msi_granule(domain); =20 msi_addr &=3D ~(phys_addr_t)(size - 1); - list_for_each_entry(msi_page, &cookie->msi_page_list, list) + list_for_each_entry(msi_page, msi_page_list, list) if (msi_page->phys =3D=3D msi_addr) return msi_page; =20 @@ -1801,11 +1816,11 @@ static struct iommu_dma_msi_page *iommu_dma_get_msi= _page(struct device *dev, INIT_LIST_HEAD(&msi_page->list); msi_page->phys =3D msi_addr; msi_page->iova =3D iova; - list_add(&msi_page->list, &cookie->msi_page_list); + list_add(&msi_page->list, msi_page_list); return msi_page; =20 out_free_iova: - iommu_dma_free_iova(cookie, iova, size, NULL); + iommu_dma_free_iova(domain, iova, size, NULL); out_free_page: kfree(msi_page); return NULL; @@ -1817,7 +1832,7 @@ static int iommu_dma_sw_msi(struct iommu_domain *doma= in, struct msi_desc *desc, struct device *dev =3D msi_desc_to_dev(desc); const struct iommu_dma_msi_page *msi_page; =20 - if (!domain->iova_cookie) { + if (!has_msi_cookie(domain)) { msi_desc_set_iommu_msi_iova(desc, 0, 0); return 0; } @@ -1827,9 +1842,8 @@ static int iommu_dma_sw_msi(struct iommu_domain *doma= in, struct msi_desc *desc, if (!msi_page) return -ENOMEM; =20 - msi_desc_set_iommu_msi_iova( - desc, msi_page->iova, - ilog2(cookie_msi_granule(domain->iova_cookie))); + msi_desc_set_iommu_msi_iova(desc, msi_page->iova, + ilog2(cookie_msi_granule(domain))); return 0; } =20 diff --git a/drivers/iommu/iommu-sva.c b/drivers/iommu/iommu-sva.c index 503c5d23c1ea..ab18bc494eef 100644 --- a/drivers/iommu/iommu-sva.c +++ b/drivers/iommu/iommu-sva.c @@ -310,6 +310,7 @@ static struct iommu_domain *iommu_sva_domain_alloc(stru= ct device *dev, } =20 domain->type =3D IOMMU_DOMAIN_SVA; + domain->cookie_type =3D IOMMU_COOKIE_SVA; mmgrab(mm); domain->mm =3D mm; domain->owner =3D ops; diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c index 0ee17893810f..c92e47f333cb 100644 --- a/drivers/iommu/iommu.c +++ b/drivers/iommu/iommu.c @@ -1953,8 +1953,10 @@ void iommu_set_fault_handler(struct iommu_domain *do= main, iommu_fault_handler_t handler, void *token) { - BUG_ON(!domain); + if (WARN_ON(!domain || domain->cookie_type !=3D IOMMU_COOKIE_NONE)) + return; =20 + domain->cookie_type =3D IOMMU_COOKIE_FAULT_HANDLER; domain->handler =3D handler; domain->handler_token =3D token; } @@ -2024,9 +2026,19 @@ EXPORT_SYMBOL_GPL(iommu_paging_domain_alloc_flags); =20 void iommu_domain_free(struct iommu_domain *domain) { - if (domain->type =3D=3D IOMMU_DOMAIN_SVA) + switch (domain->cookie_type) { + case IOMMU_COOKIE_DMA_IOVA: + iommu_put_dma_cookie(domain); + break; + case IOMMU_COOKIE_DMA_MSI: + iommu_put_msi_cookie(domain); + break; + case IOMMU_COOKIE_SVA: mmdrop(domain->mm); - iommu_put_dma_cookie(domain); + break; + default: + break; + } if (domain->ops->free) domain->ops->free(domain); } diff --git a/drivers/iommu/iommufd/hw_pagetable.c b/drivers/iommu/iommufd/h= w_pagetable.c index 9a89f3a28dc5..fded3f07bfa7 100644 --- a/drivers/iommu/iommufd/hw_pagetable.c +++ b/drivers/iommu/iommufd/hw_pagetable.c @@ -160,6 +160,7 @@ iommufd_hwpt_paging_alloc(struct iommufd_ctx *ictx, str= uct iommufd_ioas *ioas, } } hwpt->domain->iommufd_hwpt =3D hwpt; + hwpt->domain->cookie_type =3D IOMMU_COOKIE_IOMMUFD; iommu_domain_set_sw_msi(hwpt->domain, iommufd_sw_msi); =20 /* @@ -257,6 +258,7 @@ iommufd_hwpt_nested_alloc(struct iommufd_ctx *ictx, } hwpt->domain->owner =3D ops; hwpt->domain->iommufd_hwpt =3D hwpt; + hwpt->domain->cookie_type =3D IOMMU_COOKIE_IOMMUFD; iommu_domain_set_sw_msi(hwpt->domain, iommufd_sw_msi); =20 if (WARN_ON_ONCE(hwpt->domain->type !=3D IOMMU_DOMAIN_NESTED)) { @@ -315,6 +317,7 @@ iommufd_viommu_alloc_hwpt_nested(struct iommufd_viommu = *viommu, u32 flags, } hwpt->domain->iommufd_hwpt =3D hwpt; hwpt->domain->owner =3D viommu->iommu_dev->ops; + hwpt->domain->cookie_type =3D IOMMU_COOKIE_IOMMUFD; iommu_domain_set_sw_msi(hwpt->domain, iommufd_sw_msi); =20 if (WARN_ON_ONCE(hwpt->domain->type !=3D IOMMU_DOMAIN_NESTED)) { --=20 2.43.0 From nobody Thu Dec 18 14:25:06 2025 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2065.outbound.protection.outlook.com [40.107.93.65]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 355FB1A2C11 for ; Tue, 25 Mar 2025 04:05:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.93.65 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1742875551; cv=fail; b=p1knFSag9VuKQAhg8rOlwMGjRW8agRz78RBmpMpsJiSqTH2j3AHD4YYEmkBSjwklqgMGgySVio6cbzjA4wGLaBuYVXPUoAIbhyG/+ICgTxHS9nE/wBrYyc7ljc9B0mPGDIIyrVo57KDyveH3mbTYNm3I68+9HzaHdwNIUNfl/v4= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1742875551; c=relaxed/simple; bh=DkLMrCWuFSFnnjXli/zKmzTX6SElhAAIBkiAdzgajW0=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=gWv3Wntgg1baV5VHAOYFImvlXey4ju/WBWgpmgwq8liCzIyQOHoM0YLdN+in7F7NkGmdsXguYNI3gLDM+X+5/EIB/zbhgULk2iRE4TSKNE/LTowRYJQJcJ/ke9YYP2JiEd4brJungW7iJAyDZw2vdb6pyxZNT0Yc8Aq8lOfbRtY= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=I9DRhbpI; arc=fail smtp.client-ip=40.107.93.65 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="I9DRhbpI" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=dsxikHo9mtxo39Omq9JkmROEIPnViT0AyABAO5qlS0l2l2ZwIepCs917EHJodkRKzP0tzizxvm2GQCVKW9phL/kVJvvMjJxGGW/xHEwAWx/5+T4Qe5hkq8mK797ZWlfLSFwkTjzm1+sQI6oIxv47/oibew+PfQ83l3txeOoOYBdnVUerQGmJ8/AdTcsBuEwvYWzsiyuajo/GralBjWSKxeeRVyEVoXFRsgyjvhBYlkfvXFh8vY3rqJZrfNSgre7NLJqZnPr+qIwHc5xX0M10u4EBS5wapMC6wXr0FWjmZ+53D5leV6Dk09vcIkIAR6gGv9TDXDIzkFrD3CajjWjakw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=dFK7YfycJoDXwBGZZtPXrQqKFr58fCidFpvfEIDy18A=; b=W1j42b1YWurFcQ/cKazGKTknbYF4I042LH7Eq8yV3IgRxajn5t+uW/345bfnC9xFC7z3NYh3sBMkUOY05SQdobGmIlfwiJcNoUvi+/boLd07JAvJyX2+35LD1FW2nQGpkj09fyf1GmdSA5UeMbwMnhwwsUbPXS2Ici9oktIEk1pUSSLqwRG6GNLmLcx2C0bMyVZW6bYsomlRimSqvBUeJ+FZpFmDFm/WqMAhwm02PPNjkfmm3T56ICqL4xGP9UHQNlTzzRm8A+zYY7BZkAV7VV8iI+ebIqSubO6oFY3FJNJ9bx4YKYkfIp1G3sanaar1StKATZoeKBC08lIQSjHxWA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=arm.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=dFK7YfycJoDXwBGZZtPXrQqKFr58fCidFpvfEIDy18A=; b=I9DRhbpIQx28kN0suOexsK9JgbMWe98A9JG7xhvaPR9sN26bA9HOuOdcVrwOYEYJiRl5C5NlOAn/yiae6TIzszxiujkxHGVxvWmBKdCCYKsLKe9R7khocZ2MWmBOgIEDjTViBWOC0TA1htJ1EjFss31uSAOvvrT6eJFz4xp87OK68kkmxC/hUipXwrC/J86DCHh5NQwTdfuLr5VZRN5KJNpZQgF5PgoqgfyCE6E4cjkDgt2LQUFPMLM81xsnjRkpCRqUQzz6TnIlzMnoOVjzpI9MHsEw9yd6p5p9zN+0NuWRaHJ55VxqRKHpdwvNhv3TECBKOPd+oaUhisaCMxWNew== Received: from SJ0PR03CA0235.namprd03.prod.outlook.com (2603:10b6:a03:39f::30) by DS2PR12MB9615.namprd12.prod.outlook.com (2603:10b6:8:275::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8534.42; Tue, 25 Mar 2025 04:05:44 +0000 Received: from CO1PEPF000075EE.namprd03.prod.outlook.com (2603:10b6:a03:39f:cafe::79) by SJ0PR03CA0235.outlook.office365.com (2603:10b6:a03:39f::30) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8534.43 via Frontend Transport; Tue, 25 Mar 2025 04:05:44 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by CO1PEPF000075EE.mail.protection.outlook.com (10.167.249.37) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8534.20 via Frontend Transport; Tue, 25 Mar 2025 04:05:43 +0000 Received: from rnnvmail202.nvidia.com (10.129.68.7) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Mon, 24 Mar 2025 21:05:29 -0700 Received: from rnnvmail202.nvidia.com (10.129.68.7) by rnnvmail202.nvidia.com (10.129.68.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.14; Mon, 24 Mar 2025 21:05:28 -0700 Received: from Asurada-Nvidia.nvidia.com (10.127.8.13) by mail.nvidia.com (10.129.68.7) with Microsoft SMTP Server id 15.2.1544.14 via Frontend Transport; Mon, 24 Mar 2025 21:05:27 -0700 From: Nicolin Chen To: CC: , , , , , , , , Subject: [PATCH v5 2/3] iommufd: Move iommufd_sw_msi and related functions to driver.c Date: Mon, 24 Mar 2025 21:05:16 -0700 Message-ID: <374c159592dba7852bee20968f3f66fa0ee8ca93.1742871535.git.nicolinc@nvidia.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: AnonymousSubmission X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1PEPF000075EE:EE_|DS2PR12MB9615:EE_ X-MS-Office365-Filtering-Correlation-Id: 951d558d-86c9-4f0c-f7dc-08dd6b52509a X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|36860700013|1800799024|376014; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?t9DZxOlynoVIhC75qegUEmV5wPFSbpvJF0oWoOj2wpHzzVETyXoJF7SQQ68X?= =?us-ascii?Q?I4EupjQATSp2YTucRO7VI6bs2Rld0Jc1RfOKFTSOIzm/iuCnoq3nJZJL9yHq?= =?us-ascii?Q?oXoR3srBNHS9S88XYd5Y+Noxp6sAhRfBXxRKCMyQXZ9LAMA68TPvDKmKGcU2?= =?us-ascii?Q?DJfyZScyLl/QcjWn5jwCmOyiRiIZtGJCu+GboWUdXKVLyJDxwOKyvLUamBRc?= =?us-ascii?Q?D0iXsy849TieWQTpTyfUwPI/gM7NNsJkHVZwkwBGbeO6JMd0f/wYaroiuYR9?= =?us-ascii?Q?F74WrNJ/13FGcgdo/tVV6Rbvr8a5rGxKPqPfPZhMBN44RYXkdTh2xH6MFK3E?= =?us-ascii?Q?gOIW896IQpJvO4k8ja+r8ecfIKE1MmWyhomTdNXThTwgXiNrES91seAoxgsl?= =?us-ascii?Q?VqnVn5P5rZNZ16v5wuiluiTktFORwn0e7kVrtUa3yQGp9B5uGZ+VX/bmNNcY?= =?us-ascii?Q?4d4G5KYe1aD5vuCXBLY0uPR3R0pr8ZXCCL2knkiM+eAc0fV1MOZJXPbX27L3?= =?us-ascii?Q?jO6TG/OPoL79QCQeFtZNQGlZ8jXvBNTc7vdJQGkksd9WIO9ZrO6CEUHI0SBz?= =?us-ascii?Q?vQj1iFC5DxYpTA/0tJbW92uc4qVslIucO6nfYfE9n21Sf38baX7vFJ/k0ea0?= =?us-ascii?Q?zm+jHJdfyc4tmmCwjAvInFiKjt9gBJeMiM3fQLo5RWfouOJ5lp9tHUMEUK8l?= =?us-ascii?Q?ZGz7HLCcW12yMG5DBHxkZAs0TPxGxGMbi1npWbhQs1cTenKwV2ymHGm5+oHC?= =?us-ascii?Q?9C6i8zWRE6h3k4F+rfB9xZOEnLSpNUzM7AB1L0ChP4cf0xh27TVu2UU09NAi?= =?us-ascii?Q?/RIRxgTEZTru8EDQJjI/ZbhrY/CBKrUvQWM6rsO2qDfP0ozh7H7Gpy+fMTtx?= =?us-ascii?Q?aEnEYwO8DyKn+vHvAJYgv/ctsZQbhR8y1f/udROVukbiMvbinVZBtwskVBJT?= =?us-ascii?Q?TxRpnXeVkY3Xh58pfK5MK8gXkL22uoUyljfaT8vZyHIFItbdl9wwrrvZ0RNQ?= =?us-ascii?Q?/1jxqikIIyAw68rINuRj4EjpxWLbgTJrtrK44Ca6zjhEvP8+2BG9l/OVvMVP?= =?us-ascii?Q?ItAD84AWgiLTRHQJBaG2LdCDbiG2+kTw9kKWr392NL3MgZufpX4oZZeydixS?= =?us-ascii?Q?U8RQsU4Y86Lulv9B0DfG2/P05ZUOlb9AJ1vfLNwqASIl8kJd6bSrGfqpmWjI?= =?us-ascii?Q?bu1+016nCAbFGhk9vp9Eyj1k+7DFfjIzrHap6T008Jw5zqYdV5tFl/yhHDia?= =?us-ascii?Q?La0pfYcU0yO4a2hzuMjeTqx6oiidqhHrx6ENDl9Z+7Rv8ziI13mS7Zx1sncb?= =?us-ascii?Q?wKL+qOOCfPFm4n82TguZpc07EbSgrmbUqy8ZTKPEKjxN03Fl5yQ1looKbBaW?= =?us-ascii?Q?+b9dhfUxIj7UxZzIXwNVzuoqPL/3K8hh+DeHzIEPA5GlfFv7raNqhbxwIliT?= =?us-ascii?Q?Zxz5vT2gz3ZyBvCwkAyqSHM9CFAigqNCJHzEeXhPfuvrk1S35/xKp+3Mc775?= =?us-ascii?Q?O/+CJRlWXa4zJD0=3D?= X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(82310400026)(36860700013)(1800799024)(376014);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Mar 2025 04:05:43.9364 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 951d558d-86c9-4f0c-f7dc-08dd6b52509a X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1PEPF000075EE.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS2PR12MB9615 Content-Type: text/plain; charset="utf-8" To provide the iommufd_sw_msi() to the iommu core that is under a different Kconfig, move it and its related functions to driver.c. Then, stub it into the iommu-priv header. The iommufd_sw_msi_install() continues to be used by iommufd internal, so put it in the private header. Note that iommufd_sw_msi() will be called in the iommu core, replacing the sw_msi function pointer. Given that IOMMU_API is "bool" in Kconfig, change IOMMUFD_DRIVER_CORE to "bool" as well. Since this affects the module size, here is before-n-after size comparison: [Before] text data bss dec hex filename 18797 848 56 19701 4cf5 drivers/iommu/iommufd/device.o 722 44 0 766 2fe drivers/iommu/iommufd/driver.o [After] text data bss dec hex filename 17735 808 56 18599 48a7 drivers/iommu/iommufd/device.o 3020 180 0 3200 c80 drivers/iommu/iommufd/driver.o Signed-off-by: Nicolin Chen Reviewed-by: Kevin Tian Reviewed-by: Jason Gunthorpe --- drivers/iommu/iommufd/Kconfig | 2 +- drivers/iommu/iommu-priv.h | 13 +++ drivers/iommu/iommufd/iommufd_private.h | 7 +- drivers/iommu/iommufd/device.c | 131 ++---------------------- drivers/iommu/iommufd/driver.c | 126 +++++++++++++++++++++++ 5 files changed, 153 insertions(+), 126 deletions(-) diff --git a/drivers/iommu/iommufd/Kconfig b/drivers/iommu/iommufd/Kconfig index 0a07f9449fd9..2beeb4f60ee5 100644 --- a/drivers/iommu/iommufd/Kconfig +++ b/drivers/iommu/iommufd/Kconfig @@ -1,6 +1,6 @@ # SPDX-License-Identifier: GPL-2.0-only config IOMMUFD_DRIVER_CORE - tristate + bool default (IOMMUFD_DRIVER || IOMMUFD) if IOMMUFD!=3Dn =20 config IOMMUFD diff --git a/drivers/iommu/iommu-priv.h b/drivers/iommu/iommu-priv.h index b4508423e13b..c74fff25be78 100644 --- a/drivers/iommu/iommu-priv.h +++ b/drivers/iommu/iommu-priv.h @@ -5,6 +5,7 @@ #define __LINUX_IOMMU_PRIV_H =20 #include +#include =20 static inline const struct iommu_ops *dev_iommu_ops(struct device *dev) { @@ -43,4 +44,16 @@ void iommu_detach_group_handle(struct iommu_domain *doma= in, int iommu_replace_group_handle(struct iommu_group *group, struct iommu_domain *new_domain, struct iommu_attach_handle *handle); + +#if IS_ENABLED(CONFIG_IOMMUFD_DRIVER_CORE) && IS_ENABLED(CONFIG_IRQ_MSI_IO= MMU) +int iommufd_sw_msi(struct iommu_domain *domain, struct msi_desc *desc, + phys_addr_t msi_addr); +#else /* !CONFIG_IOMMUFD_DRIVER_CORE || !CONFIG_IRQ_MSI_IOMMU */ +static inline int iommufd_sw_msi(struct iommu_domain *domain, + struct msi_desc *desc, phys_addr_t msi_addr) +{ + return -EOPNOTSUPP; +} +#endif /* CONFIG_IOMMUFD_DRIVER_CORE && CONFIG_IRQ_MSI_IOMMU */ + #endif /* __LINUX_IOMMU_PRIV_H */ diff --git a/drivers/iommu/iommufd/iommufd_private.h b/drivers/iommu/iommuf= d/iommufd_private.h index 8cda9c4672eb..8c49ca16919a 100644 --- a/drivers/iommu/iommufd/iommufd_private.h +++ b/drivers/iommu/iommufd/iommufd_private.h @@ -32,8 +32,11 @@ struct iommufd_sw_msi_maps { DECLARE_BITMAP(bitmap, 64); }; =20 -int iommufd_sw_msi(struct iommu_domain *domain, struct msi_desc *desc, - phys_addr_t msi_addr); +#ifdef CONFIG_IRQ_MSI_IOMMU +int iommufd_sw_msi_install(struct iommufd_ctx *ictx, + struct iommufd_hwpt_paging *hwpt_paging, + struct iommufd_sw_msi_map *msi_map); +#endif =20 struct iommufd_ctx { struct file *file; diff --git a/drivers/iommu/iommufd/device.c b/drivers/iommu/iommufd/device.c index bd50146e2ad0..d18ea9a61522 100644 --- a/drivers/iommu/iommufd/device.c +++ b/drivers/iommu/iommufd/device.c @@ -5,7 +5,6 @@ #include #include #include -#include =20 #include "../iommu-priv.h" #include "io_pagetable.h" @@ -294,129 +293,7 @@ u32 iommufd_device_to_id(struct iommufd_device *idev) } EXPORT_SYMBOL_NS_GPL(iommufd_device_to_id, "IOMMUFD"); =20 -/* - * Get a iommufd_sw_msi_map for the msi physical address requested by the = irq - * layer. The mapping to IOVA is global to the iommufd file descriptor, ev= ery - * domain that is attached to a device using the same MSI parameters will = use - * the same IOVA. - */ -static __maybe_unused struct iommufd_sw_msi_map * -iommufd_sw_msi_get_map(struct iommufd_ctx *ictx, phys_addr_t msi_addr, - phys_addr_t sw_msi_start) -{ - struct iommufd_sw_msi_map *cur; - unsigned int max_pgoff =3D 0; - - lockdep_assert_held(&ictx->sw_msi_lock); - - list_for_each_entry(cur, &ictx->sw_msi_list, sw_msi_item) { - if (cur->sw_msi_start !=3D sw_msi_start) - continue; - max_pgoff =3D max(max_pgoff, cur->pgoff + 1); - if (cur->msi_addr =3D=3D msi_addr) - return cur; - } - - if (ictx->sw_msi_id >=3D - BITS_PER_BYTE * sizeof_field(struct iommufd_sw_msi_maps, bitmap)) - return ERR_PTR(-EOVERFLOW); - - cur =3D kzalloc(sizeof(*cur), GFP_KERNEL); - if (!cur) - return ERR_PTR(-ENOMEM); - - cur->sw_msi_start =3D sw_msi_start; - cur->msi_addr =3D msi_addr; - cur->pgoff =3D max_pgoff; - cur->id =3D ictx->sw_msi_id++; - list_add_tail(&cur->sw_msi_item, &ictx->sw_msi_list); - return cur; -} - -static int iommufd_sw_msi_install(struct iommufd_ctx *ictx, - struct iommufd_hwpt_paging *hwpt_paging, - struct iommufd_sw_msi_map *msi_map) -{ - unsigned long iova; - - lockdep_assert_held(&ictx->sw_msi_lock); - - iova =3D msi_map->sw_msi_start + msi_map->pgoff * PAGE_SIZE; - if (!test_bit(msi_map->id, hwpt_paging->present_sw_msi.bitmap)) { - int rc; - - rc =3D iommu_map(hwpt_paging->common.domain, iova, - msi_map->msi_addr, PAGE_SIZE, - IOMMU_WRITE | IOMMU_READ | IOMMU_MMIO, - GFP_KERNEL_ACCOUNT); - if (rc) - return rc; - __set_bit(msi_map->id, hwpt_paging->present_sw_msi.bitmap); - } - return 0; -} - -/* - * Called by the irq code if the platform translates the MSI address throu= gh the - * IOMMU. msi_addr is the physical address of the MSI page. iommufd will - * allocate a fd global iova for the physical page that is the same on all - * domains and devices. - */ #ifdef CONFIG_IRQ_MSI_IOMMU -int iommufd_sw_msi(struct iommu_domain *domain, struct msi_desc *desc, - phys_addr_t msi_addr) -{ - struct device *dev =3D msi_desc_to_dev(desc); - struct iommufd_hwpt_paging *hwpt_paging; - struct iommu_attach_handle *raw_handle; - struct iommufd_attach_handle *handle; - struct iommufd_sw_msi_map *msi_map; - struct iommufd_ctx *ictx; - unsigned long iova; - int rc; - - /* - * It is safe to call iommu_attach_handle_get() here because the iommu - * core code invokes this under the group mutex which also prevents any - * change of the attach handle for the duration of this function. - */ - iommu_group_mutex_assert(dev); - - raw_handle =3D - iommu_attach_handle_get(dev->iommu_group, IOMMU_NO_PASID, 0); - if (IS_ERR(raw_handle)) - return 0; - hwpt_paging =3D find_hwpt_paging(domain->iommufd_hwpt); - - handle =3D to_iommufd_handle(raw_handle); - /* No IOMMU_RESV_SW_MSI means no change to the msi_msg */ - if (handle->idev->igroup->sw_msi_start =3D=3D PHYS_ADDR_MAX) - return 0; - - ictx =3D handle->idev->ictx; - guard(mutex)(&ictx->sw_msi_lock); - /* - * The input msi_addr is the exact byte offset of the MSI doorbell, we - * assume the caller has checked that it is contained with a MMIO region - * that is secure to map at PAGE_SIZE. - */ - msi_map =3D iommufd_sw_msi_get_map(handle->idev->ictx, - msi_addr & PAGE_MASK, - handle->idev->igroup->sw_msi_start); - if (IS_ERR(msi_map)) - return PTR_ERR(msi_map); - - rc =3D iommufd_sw_msi_install(ictx, hwpt_paging, msi_map); - if (rc) - return rc; - __set_bit(msi_map->id, handle->idev->igroup->required_sw_msi.bitmap); - - iova =3D msi_map->sw_msi_start + msi_map->pgoff * PAGE_SIZE; - msi_desc_set_iommu_msi_iova(desc, iova, PAGE_SHIFT); - return 0; -} -#endif - static int iommufd_group_setup_msi(struct iommufd_group *igroup, struct iommufd_hwpt_paging *hwpt_paging) { @@ -443,6 +320,14 @@ static int iommufd_group_setup_msi(struct iommufd_grou= p *igroup, } return 0; } +#else +static inline int +iommufd_group_setup_msi(struct iommufd_group *igroup, + struct iommufd_hwpt_paging *hwpt_paging) +{ + return 0; +} +#endif =20 static int iommufd_device_attach_reserved_iova(struct iommufd_device *idev, diff --git a/drivers/iommu/iommufd/driver.c b/drivers/iommu/iommufd/driver.c index 75b365561c16..a08ff0f37fc6 100644 --- a/drivers/iommu/iommufd/driver.c +++ b/drivers/iommu/iommufd/driver.c @@ -121,5 +121,131 @@ int iommufd_viommu_report_event(struct iommufd_viommu= *viommu, } EXPORT_SYMBOL_NS_GPL(iommufd_viommu_report_event, "IOMMUFD"); =20 +#ifdef CONFIG_IRQ_MSI_IOMMU +/* + * Get a iommufd_sw_msi_map for the msi physical address requested by the = irq + * layer. The mapping to IOVA is global to the iommufd file descriptor, ev= ery + * domain that is attached to a device using the same MSI parameters will = use + * the same IOVA. + */ +static struct iommufd_sw_msi_map * +iommufd_sw_msi_get_map(struct iommufd_ctx *ictx, phys_addr_t msi_addr, + phys_addr_t sw_msi_start) +{ + struct iommufd_sw_msi_map *cur; + unsigned int max_pgoff =3D 0; + + lockdep_assert_held(&ictx->sw_msi_lock); + + list_for_each_entry(cur, &ictx->sw_msi_list, sw_msi_item) { + if (cur->sw_msi_start !=3D sw_msi_start) + continue; + max_pgoff =3D max(max_pgoff, cur->pgoff + 1); + if (cur->msi_addr =3D=3D msi_addr) + return cur; + } + + if (ictx->sw_msi_id >=3D + BITS_PER_BYTE * sizeof_field(struct iommufd_sw_msi_maps, bitmap)) + return ERR_PTR(-EOVERFLOW); + + cur =3D kzalloc(sizeof(*cur), GFP_KERNEL); + if (!cur) + return ERR_PTR(-ENOMEM); + + cur->sw_msi_start =3D sw_msi_start; + cur->msi_addr =3D msi_addr; + cur->pgoff =3D max_pgoff; + cur->id =3D ictx->sw_msi_id++; + list_add_tail(&cur->sw_msi_item, &ictx->sw_msi_list); + return cur; +} + +int iommufd_sw_msi_install(struct iommufd_ctx *ictx, + struct iommufd_hwpt_paging *hwpt_paging, + struct iommufd_sw_msi_map *msi_map) +{ + unsigned long iova; + + lockdep_assert_held(&ictx->sw_msi_lock); + + iova =3D msi_map->sw_msi_start + msi_map->pgoff * PAGE_SIZE; + if (!test_bit(msi_map->id, hwpt_paging->present_sw_msi.bitmap)) { + int rc; + + rc =3D iommu_map(hwpt_paging->common.domain, iova, + msi_map->msi_addr, PAGE_SIZE, + IOMMU_WRITE | IOMMU_READ | IOMMU_MMIO, + GFP_KERNEL_ACCOUNT); + if (rc) + return rc; + __set_bit(msi_map->id, hwpt_paging->present_sw_msi.bitmap); + } + return 0; +} +EXPORT_SYMBOL_NS_GPL(iommufd_sw_msi_install, "IOMMUFD_INTERNAL"); + +/* + * Called by the irq code if the platform translates the MSI address throu= gh the + * IOMMU. msi_addr is the physical address of the MSI page. iommufd will + * allocate a fd global iova for the physical page that is the same on all + * domains and devices. + */ +int iommufd_sw_msi(struct iommu_domain *domain, struct msi_desc *desc, + phys_addr_t msi_addr) +{ + struct device *dev =3D msi_desc_to_dev(desc); + struct iommufd_hwpt_paging *hwpt_paging; + struct iommu_attach_handle *raw_handle; + struct iommufd_attach_handle *handle; + struct iommufd_sw_msi_map *msi_map; + struct iommufd_ctx *ictx; + unsigned long iova; + int rc; + + /* + * It is safe to call iommu_attach_handle_get() here because the iommu + * core code invokes this under the group mutex which also prevents any + * change of the attach handle for the duration of this function. + */ + iommu_group_mutex_assert(dev); + + raw_handle =3D + iommu_attach_handle_get(dev->iommu_group, IOMMU_NO_PASID, 0); + if (IS_ERR(raw_handle)) + return 0; + hwpt_paging =3D find_hwpt_paging(domain->iommufd_hwpt); + + handle =3D to_iommufd_handle(raw_handle); + /* No IOMMU_RESV_SW_MSI means no change to the msi_msg */ + if (handle->idev->igroup->sw_msi_start =3D=3D PHYS_ADDR_MAX) + return 0; + + ictx =3D handle->idev->ictx; + guard(mutex)(&ictx->sw_msi_lock); + /* + * The input msi_addr is the exact byte offset of the MSI doorbell, we + * assume the caller has checked that it is contained with a MMIO region + * that is secure to map at PAGE_SIZE. + */ + msi_map =3D iommufd_sw_msi_get_map(handle->idev->ictx, + msi_addr & PAGE_MASK, + handle->idev->igroup->sw_msi_start); + if (IS_ERR(msi_map)) + return PTR_ERR(msi_map); + + rc =3D iommufd_sw_msi_install(ictx, hwpt_paging, msi_map); + if (rc) + return rc; + __set_bit(msi_map->id, handle->idev->igroup->required_sw_msi.bitmap); + + iova =3D msi_map->sw_msi_start + msi_map->pgoff * PAGE_SIZE; + msi_desc_set_iommu_msi_iova(desc, iova, PAGE_SHIFT); + return 0; +} +EXPORT_SYMBOL_NS_GPL(iommufd_sw_msi, "IOMMUFD"); +#endif + MODULE_DESCRIPTION("iommufd code shared with builtin modules"); +MODULE_IMPORT_NS("IOMMUFD_INTERNAL"); MODULE_LICENSE("GPL"); --=20 2.43.0 From nobody Thu Dec 18 14:25:06 2025 Received: from NAM04-BN8-obe.outbound.protection.outlook.com (mail-bn8nam04on2082.outbound.protection.outlook.com [40.107.100.82]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1E6E31A5BA1 for ; Tue, 25 Mar 2025 04:05:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.100.82 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1742875553; cv=fail; b=H4WzS8lps0jtJ96YUAWvTAp0ZSiXuxQgR+B7UkCFrnXk25swvATpx2Eud4QbV0XERRRuGRvw/sRqBpMffZjb12C/V0VmVun+xzG/4Edy8Ful0eU5/AlUun8mIGzykq5vU/YGNFzB0SKXP2PeZEjcVMuwottx5dYRGqI7g4IrCPk= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1742875553; c=relaxed/simple; bh=9+8BLr7fBdGbh132wUKhu2qAQGjGFCEO+H5to65PUKo=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=HCSCD0XwkEAJtGEF4A4FylGqvLtbwRRW75OTje4T8SUFQ9k3Vrhfb6Ej1zWhyT/4RafNN5Dg4IGFruxUgFRAeiCgZcUnQfAw3q/JTdgQT23kZbx+3NI/hAevE58Iynf9cOO9BC4kjZPYdxjC4TJNcB6rXUVAAPdqw10HNx2dAIE= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=ZmHuyQuu; arc=fail smtp.client-ip=40.107.100.82 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="ZmHuyQuu" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=wWxKZjuMqPhvMf8cuGmcE+6++p5fxbhPBw0EQ+GzkT1bAiQg0WSid5ZdkV1hr1aRJ6ByidtsXBMBY0ubnzpbEhq1aMUg9m1o5DA34Qk4IBY/H7xtfqLpiCX3ZKsDqeR+VLpLq3zAuEgYk+Tzm4MYojbadKFa/vQl9WjbRyU5/5MWIi1WkNobRs7mqAu9uKFK23RGt18DlJgQoywP96PNSXTxls6mjgBTpV6CuZaRUlrQOuJVQvHD7nwqyeYTt7ubKPEJj2zdw3XS4tRggoM8IzlOWphc/fb2rM0dDRolYXMPKpxXx8U6Xuvgjq2yTpVqCchyFN1ZD0uenGhHpHsRiw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=G3KYuSK3s6jWb87t5ralnM9ZuGYADSsZaZlcl5uwWdI=; b=ai8sDpnwyvCA2m1oO3qetPTDzIkOZO45J3oR93lmwLhbGOKe3XJ6gnWL8xqFX1ve8sV06pU1/E80VIYCpNxRXIyADyaJ240eRLL2+UVYARdG0x1tVifjLqTMn43gVGSLehQgmi2zO5kMRflVejWRSVm1QEIejo408A4FLtSztZkeFrmI6FJXNo/XDYTq1Bia0n/dXfn+ZlhfgC1D29O5sJRkMmlcNl2Jmi+Ln4EiSyTOnSEXquIdQ/pplrzfayOb9EUUygU7KRSdgR0/Vz69A5cjDkf2slIox9hLtVbn5NrhGq5vC9A/h5eEviyHz6vTUhxdGUXbdFcmcfqrIOQ5fg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=arm.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=G3KYuSK3s6jWb87t5ralnM9ZuGYADSsZaZlcl5uwWdI=; b=ZmHuyQuuGcJkgg+fj2o2Vu9Ofnbd39ArDFj8+SndyPpvG5JS8YdOkmWDzQBHOO9Ow5FWUwQFnwzT49uiItD8lFIZK7+YPTunjgfBgGd4RwAIfvH7IwUygWFkPPxxfqsS+nfQMrohDt7uhGP3KGG0RKU8g67mEZZQeOyXlpd9ZBCuxIa5vLqOrzFJFsH3FVhisz0V29AMQaf239mg41slbNifmeXIgiLLj9z1tVEkHvMQcypeAh5WbJYyjDKo9cpIcq5pvT+cBq/1jc73CGTdM3mPjaaTTL70L3vBdWDZHXMTpHm0lo8JmeQPSq3e9+1ptOsPzOaFKvM62Xj7R9B9ng== Received: from MW3PR05CA0027.namprd05.prod.outlook.com (2603:10b6:303:2b::32) by SJ5PPFE4FC9FAB3.namprd12.prod.outlook.com (2603:10b6:a0f:fc02::9a7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8534.42; Tue, 25 Mar 2025 04:05:46 +0000 Received: from CO1PEPF000075F3.namprd03.prod.outlook.com (2603:10b6:303:2b:cafe::62) by MW3PR05CA0027.outlook.office365.com (2603:10b6:303:2b::32) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8534.42 via Frontend Transport; Tue, 25 Mar 2025 04:05:46 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by CO1PEPF000075F3.mail.protection.outlook.com (10.167.249.42) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8534.20 via Frontend Transport; Tue, 25 Mar 2025 04:05:46 +0000 Received: from rnnvmail202.nvidia.com (10.129.68.7) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Mon, 24 Mar 2025 21:05:30 -0700 Received: from rnnvmail202.nvidia.com (10.129.68.7) by rnnvmail202.nvidia.com (10.129.68.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.14; Mon, 24 Mar 2025 21:05:30 -0700 Received: from Asurada-Nvidia.nvidia.com (10.127.8.13) by mail.nvidia.com (10.129.68.7) with Microsoft SMTP Server id 15.2.1544.14 via Frontend Transport; Mon, 24 Mar 2025 21:05:29 -0700 From: Nicolin Chen To: CC: , , , , , , , , Subject: [PATCH v5 3/3] iommu: Drop sw_msi from iommu_domain Date: Mon, 24 Mar 2025 21:05:17 -0700 Message-ID: <7ded87c871afcbaac665b71354de0a335087bf0f.1742871535.git.nicolinc@nvidia.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: AnonymousSubmission X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1PEPF000075F3:EE_|SJ5PPFE4FC9FAB3:EE_ X-MS-Office365-Filtering-Correlation-Id: 26a091e8-f986-49e4-b1bb-08dd6b5251fa X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|82310400026|376014|36860700013|7053199007; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?UBvk/11Gc7w4ZZZVdcn4Tt3DqKl2FHkMOzH7SB+COjlqPVxvIP54dHYUPXl/?= =?us-ascii?Q?1e7GDVX2SQyNKI3+wGoOeKvHKedVTLOI+A4Z/ck+WmVlwZUVvGDIZ5WahpOR?= =?us-ascii?Q?YusMxO2k8SDI7uR4/D/gNLB/yjvspUerplsTNBT5pZ2Nte9juOU22juToRui?= =?us-ascii?Q?IPyDMatT5zbjLZ2ADvE5LcFerH2AmwV91EPfVsHiVbVWhCLRJVgBCQJW9Umz?= =?us-ascii?Q?IETvLCl6ncFxdVJijO/x22lGWPGgdNdYMvWAtXzJI6qFOlcrzWbLOWcGoPFz?= =?us-ascii?Q?wtmZnojcDyE1+v1SCWylZrMhzlU+oJvxlxqYRy9Wm6W9CrOe2uleJd+FQoaH?= =?us-ascii?Q?uk+kYWNPLCTJRj1C9FmQ8HdPVCWfqjW8QR6rrRvneUnLVzVNO3wAqGFEelX1?= =?us-ascii?Q?bB78stkNlxLui1/DZ/tmfGsFW9dbBt6ag7x5eVx05fqKDV7JT1pRTNx0iYq+?= =?us-ascii?Q?Daxc61Vx/X0+a0/nxTJF3iPDD5yDXOR1fxusd781N9AovudJF1MwVO+KPdWh?= =?us-ascii?Q?LRwiSqhHwklYNvUS3lEGSr/0Uexrx/8SlFCvre6W4m3eiNyHCJmV2EPy41m4?= =?us-ascii?Q?86aZtiUKzha+yoHCgk/4nNh7LemeVikY9XiWPqGkmgkNOa5RWQvi+ASuLRiL?= =?us-ascii?Q?qWtJU46Pg3UiC8iZRdVHi5UFKtbagTpxKl+1t6N0mZ8EiR1kzfmiPMFkz+29?= =?us-ascii?Q?cNkkmE7KJRmNgel2aVY/WAssiR2ZfRF8HhO6S7oR3ktkXD+i13oBEezLoqtC?= =?us-ascii?Q?4CYUigxt9qmYGpsXki1txZ7ZoA9X1fC5VZQWIDTgzlqR060dHkIR4F0OL6tQ?= =?us-ascii?Q?1egVq0fmHGCNLXIjey6g2jFnzIgFlMKeN1oacndLAleMq+gVmsh0YYlKoLv3?= =?us-ascii?Q?m1M/5V/JtsOGHWZLeVNxUyupqjDUNvY4mbSjIeFfH+9TmLt0d5NyuCYzEJnT?= =?us-ascii?Q?xsm3FIm9EsdTW6h30sXW72J1IOrrprlYbsIDsVpGZGaHSkeo+AoRctkkoJxC?= =?us-ascii?Q?+TIbFXDt1CKQXm3704RFhPcbZ7SWvINMZtK9UJWoj1WGyfEnGFvaNuh1NK+5?= =?us-ascii?Q?JM6BRVKVC+MFlVrcAHVj+TbEgF2FX0ZSahp23vTX/cZ7/6KexftphxnKHNmK?= =?us-ascii?Q?FIQtpHnj6rcUuXTTZUr2FFAtwI3lmbMJRsjCPVxnfalyW3jLdL9JX019N2B9?= =?us-ascii?Q?L+JUoHI8uXwSyQu2vk7UbcNGukjTb3XxrrtgdMReRwbS/rIHQGQpJ8SJXdi3?= =?us-ascii?Q?6cc3vieubqo2x173K2Ky8DmeYSYyUGlvhb/8V+XiRK92qCopDPHgIW2yvY05?= =?us-ascii?Q?Nx8zX1JOQ3a9nXXJcE4QowEhH55XP/7Tj+3+eaCKZwRldCZaI4V2DZCPA7fo?= =?us-ascii?Q?hPkSob2Uu8G9Bnjtscwj17Xb5k+4kAqQelFbjIe88qX8Q/vte/9j28ktfL6J?= =?us-ascii?Q?2AwOem8oxqND3+ktD1kP32tBZeSms62bta4Suh1lrmiALYsasUK1Mo831In8?= =?us-ascii?Q?g4RFXTmcMWP9wWI=3D?= X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(1800799024)(82310400026)(376014)(36860700013)(7053199007);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Mar 2025 04:05:46.1790 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 26a091e8-f986-49e4-b1bb-08dd6b5251fa X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1PEPF000075F3.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ5PPFE4FC9FAB3 Content-Type: text/plain; charset="utf-8" There are only two sw_msi implementations in the entire system, thus it's not very necessary to have an sw_msi pointer. Instead, check domain->cookie_type to call the two sw_msi implementations directly from the core code. Suggested-by: Robin Murphy Reviewed-by: Robin Murphy Reviewed-by: Kevin Tian Signed-off-by: Nicolin Chen Reviewed-by: Jason Gunthorpe --- drivers/iommu/dma-iommu.h | 9 +++++++++ include/linux/iommu.h | 15 --------------- drivers/iommu/dma-iommu.c | 14 ++------------ drivers/iommu/iommu.c | 18 ++++++++++++++++-- drivers/iommu/iommufd/hw_pagetable.c | 3 --- 5 files changed, 27 insertions(+), 32 deletions(-) diff --git a/drivers/iommu/dma-iommu.h b/drivers/iommu/dma-iommu.h index 9cca11806e5d..eca201c1f963 100644 --- a/drivers/iommu/dma-iommu.h +++ b/drivers/iommu/dma-iommu.h @@ -19,6 +19,9 @@ int iommu_dma_init_fq(struct iommu_domain *domain); =20 void iommu_dma_get_resv_regions(struct device *dev, struct list_head *list= ); =20 +int iommu_dma_sw_msi(struct iommu_domain *domain, struct msi_desc *desc, + phys_addr_t msi_addr); + extern bool iommu_dma_forcedac; =20 #else /* CONFIG_IOMMU_DMA */ @@ -49,5 +52,11 @@ static inline void iommu_dma_get_resv_regions(struct dev= ice *dev, struct list_he { } =20 +static inline int iommu_dma_sw_msi(struct iommu_domain *domain, + struct msi_desc *desc, phys_addr_t msi_addr) +{ + return -ENODEV; +} + #endif /* CONFIG_IOMMU_DMA */ #endif /* __DMA_IOMMU_H */ diff --git a/include/linux/iommu.h b/include/linux/iommu.h index 06cc14e9993d..e01c855ae8a7 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -229,11 +229,6 @@ struct iommu_domain { struct iommu_domain_geometry geometry; int (*iopf_handler)(struct iopf_group *group); =20 -#if IS_ENABLED(CONFIG_IRQ_MSI_IOMMU) - int (*sw_msi)(struct iommu_domain *domain, struct msi_desc *desc, - phys_addr_t msi_addr); -#endif - union { /* cookie */ struct iommu_dma_cookie *iova_cookie; struct iommu_dma_msi_cookie *msi_cookie; @@ -254,16 +249,6 @@ struct iommu_domain { }; }; =20 -static inline void iommu_domain_set_sw_msi( - struct iommu_domain *domain, - int (*sw_msi)(struct iommu_domain *domain, struct msi_desc *desc, - phys_addr_t msi_addr)) -{ -#if IS_ENABLED(CONFIG_IRQ_MSI_IOMMU) - domain->sw_msi =3D sw_msi; -#endif -} - static inline bool iommu_is_dma_domain(struct iommu_domain *domain) { return domain->type & __IOMMU_DOMAIN_DMA_API; diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c index 31a7b4b81656..2bd9f80a83fe 100644 --- a/drivers/iommu/dma-iommu.c +++ b/drivers/iommu/dma-iommu.c @@ -94,9 +94,6 @@ static int __init iommu_dma_forcedac_setup(char *str) } early_param("iommu.forcedac", iommu_dma_forcedac_setup); =20 -static int iommu_dma_sw_msi(struct iommu_domain *domain, struct msi_desc *= desc, - phys_addr_t msi_addr); - /* Number of entries per flush queue */ #define IOVA_DEFAULT_FQ_SIZE 256 #define IOVA_SINGLE_FQ_SIZE 32768 @@ -377,7 +374,6 @@ int iommu_get_dma_cookie(struct iommu_domain *domain) =20 mutex_init(&cookie->mutex); INIT_LIST_HEAD(&cookie->msi_page_list); - iommu_domain_set_sw_msi(domain, iommu_dma_sw_msi); domain->cookie_type =3D IOMMU_COOKIE_DMA_IOVA; domain->iova_cookie =3D cookie; return 0; @@ -411,7 +407,6 @@ int iommu_get_msi_cookie(struct iommu_domain *domain, d= ma_addr_t base) =20 cookie->msi_iova =3D base; INIT_LIST_HEAD(&cookie->msi_page_list); - iommu_domain_set_sw_msi(domain, iommu_dma_sw_msi); domain->cookie_type =3D IOMMU_COOKIE_DMA_MSI; domain->msi_cookie =3D cookie; return 0; @@ -427,11 +422,6 @@ void iommu_put_dma_cookie(struct iommu_domain *domain) struct iommu_dma_cookie *cookie =3D domain->iova_cookie; struct iommu_dma_msi_page *msi, *tmp; =20 -#if IS_ENABLED(CONFIG_IRQ_MSI_IOMMU) - if (domain->sw_msi !=3D iommu_dma_sw_msi) - return; -#endif - if (cookie->iovad.granule) { iommu_dma_free_fq(cookie); put_iova_domain(&cookie->iovad); @@ -1826,8 +1816,8 @@ static struct iommu_dma_msi_page *iommu_dma_get_msi_p= age(struct device *dev, return NULL; } =20 -static int iommu_dma_sw_msi(struct iommu_domain *domain, struct msi_desc *= desc, - phys_addr_t msi_addr) +int iommu_dma_sw_msi(struct iommu_domain *domain, struct msi_desc *desc, + phys_addr_t msi_addr) { struct device *dev =3D msi_desc_to_dev(desc); const struct iommu_dma_msi_page *msi_page; diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c index c92e47f333cb..d96e6fabb4da 100644 --- a/drivers/iommu/iommu.c +++ b/drivers/iommu/iommu.c @@ -18,6 +18,7 @@ #include #include #include +#include #include #include #include @@ -3650,8 +3651,21 @@ int iommu_dma_prepare_msi(struct msi_desc *desc, phy= s_addr_t msi_addr) return 0; =20 mutex_lock(&group->mutex); - if (group->domain && group->domain->sw_msi) - ret =3D group->domain->sw_msi(group->domain, desc, msi_addr); + /* An IDENTITY domain must pass through */ + if (group->domain && group->domain->type !=3D IOMMU_DOMAIN_IDENTITY) { + switch (group->domain->cookie_type) { + case IOMMU_COOKIE_DMA_MSI: + case IOMMU_COOKIE_DMA_IOVA: + ret =3D iommu_dma_sw_msi(group->domain, desc, msi_addr); + break; + case IOMMU_COOKIE_IOMMUFD: + ret =3D iommufd_sw_msi(group->domain, desc, msi_addr); + break; + default: + ret =3D -EOPNOTSUPP; + break; + } + } mutex_unlock(&group->mutex); return ret; } diff --git a/drivers/iommu/iommufd/hw_pagetable.c b/drivers/iommu/iommufd/h= w_pagetable.c index fded3f07bfa7..8e87ae71e128 100644 --- a/drivers/iommu/iommufd/hw_pagetable.c +++ b/drivers/iommu/iommufd/hw_pagetable.c @@ -161,7 +161,6 @@ iommufd_hwpt_paging_alloc(struct iommufd_ctx *ictx, str= uct iommufd_ioas *ioas, } hwpt->domain->iommufd_hwpt =3D hwpt; hwpt->domain->cookie_type =3D IOMMU_COOKIE_IOMMUFD; - iommu_domain_set_sw_msi(hwpt->domain, iommufd_sw_msi); =20 /* * Set the coherency mode before we do iopt_table_add_domain() as some @@ -259,7 +258,6 @@ iommufd_hwpt_nested_alloc(struct iommufd_ctx *ictx, hwpt->domain->owner =3D ops; hwpt->domain->iommufd_hwpt =3D hwpt; hwpt->domain->cookie_type =3D IOMMU_COOKIE_IOMMUFD; - iommu_domain_set_sw_msi(hwpt->domain, iommufd_sw_msi); =20 if (WARN_ON_ONCE(hwpt->domain->type !=3D IOMMU_DOMAIN_NESTED)) { rc =3D -EINVAL; @@ -318,7 +316,6 @@ iommufd_viommu_alloc_hwpt_nested(struct iommufd_viommu = *viommu, u32 flags, hwpt->domain->iommufd_hwpt =3D hwpt; hwpt->domain->owner =3D viommu->iommu_dev->ops; hwpt->domain->cookie_type =3D IOMMU_COOKIE_IOMMUFD; - iommu_domain_set_sw_msi(hwpt->domain, iommufd_sw_msi); =20 if (WARN_ON_ONCE(hwpt->domain->type !=3D IOMMU_DOMAIN_NESTED)) { rc =3D -EINVAL; --=20 2.43.0