From nobody Sun Dec 22 03:14:30 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.libvirt.org designates 8.43.85.245 as permitted sender) client-ip=8.43.85.245; envelope-from=devel-bounces@lists.libvirt.org; helo=lists.libvirt.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of lists.libvirt.org designates 8.43.85.245 as permitted sender) smtp.mailfrom=devel-bounces@lists.libvirt.org; arc=fail (BodyHash is different from the expected one); dmarc=pass(p=reject dis=none) header.from=lists.libvirt.org Return-Path: Received: from lists.libvirt.org (lists.libvirt.org [8.43.85.245]) by mx.zohomail.com with SMTPS id 1733963147327802.1261396831701; Wed, 11 Dec 2024 16:25:47 -0800 (PST) Received: by lists.libvirt.org (Postfix, from userid 996) id 51FF6136C; Wed, 11 Dec 2024 19:25:46 -0500 (EST) Received: from lists.libvirt.org (localhost [IPv6:::1]) by lists.libvirt.org (Postfix) with ESMTP id 6D6CF1289; Wed, 11 Dec 2024 19:25:01 -0500 (EST) Received: by lists.libvirt.org (Postfix, from userid 996) id 2AF5A12FF; Wed, 11 Dec 2024 19:24:57 -0500 (EST) Received: from NAM04-BN8-obe.outbound.protection.outlook.com (mail-bn8nam04on2075.outbound.protection.outlook.com [40.107.100.75]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.libvirt.org (Postfix) with ESMTPS id 7727A1270 for ; Wed, 11 Dec 2024 19:24:43 -0500 (EST) Received: from SN7PR12MB6838.namprd12.prod.outlook.com (2603:10b6:806:266::18) by MN0PR12MB6103.namprd12.prod.outlook.com (2603:10b6:208:3c9::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8251.16; Thu, 12 Dec 2024 00:24:40 +0000 Received: from SN7PR12MB6838.namprd12.prod.outlook.com ([fe80::529d:478:bc5d:b400]) by SN7PR12MB6838.namprd12.prod.outlook.com ([fe80::529d:478:bc5d:b400%6]) with mapi id 15.20.8251.008; Thu, 12 Dec 2024 00:24:40 +0000 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on lists.libvirt.org X-Spam-Level: X-Spam-Status: No, score=0.2 required=5.0 tests=DKIM_INVALID,DKIM_SIGNED, FORGED_SPF_HELO,MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H2, RCVD_IN_VALIDITY_RPBL_BLOCKED,RCVD_IN_VALIDITY_SAFE_BLOCKED, SPF_HELO_PASS autolearn=no autolearn_force=no version=3.4.4 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=DWIGwr+cWnlWXa89eRC5iilX4gbDAZwSnH/7EMXwr8y776GqV97JL1BduAvHVM4VyqgWgLWM/uqqPl30U2R2esqUyIetMpDLH+3eeNAr+/kHoTAo/qoxm1S51VSHmR5XUuvrIUJ4wjdCnBpMl/sY+9L1R+E6TYzp2PWAdqa8gCNVn8SmWGhnx+TPoVGo2izGHjuZJyNh0Ux8iROSDG9R8eUKz1wWMuKHd5maDUEjVuSExrgDPmNyOs58wYSSrp4v10H0HdyaIjfiNXSOHw14HAdRwqULVaamVe4WDQehkirFxXW3A2Dc3RYAkRRutcVeb98MhW+sVUqZVP/2mqUpEQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=eZ6XtH4eY35sSVQwcHC8vFgqUWzZLfYedDgOYCE8HZ0=; b=iRPnv2nBzmQkPqLgzoHheLJ8qVvtfEVqwGifIFroG62EFqotZvoyWj58WisyxReGZna0cyAhxYAC+Llk9DcVamwEO5nJF++XSJ8t3z6OcZXyM0SRpsNG+fxchLQmbuLY8njkXivxwz30HFQq7ZCLlnOUsArXP4vujTVDNwQ00qHK4YEOzpQclH5Rc4GLp6jH1bV3nC7s+Xy8D9sAQ9SjKMFW37MOk/xJHis+mr+AR2G7JnAi4fxzGUXd+pdQdYvnwtMB8krjPl9/iMS0+JmT0H2suaukUE/h3ImMkz1k5quofCYWUfwu0Ey2KYWhSB3Zy8MAxIHe2o9lJYVASY0hqw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=eZ6XtH4eY35sSVQwcHC8vFgqUWzZLfYedDgOYCE8HZ0=; b=Tjxf+KCNfLReonJyDXAYSyVYxhzwrVbpTC6Jc+q1opdrKaqIuyZ+V5mFSQxq7jpKmmN8Fhb21Y0PQiZsXb55Q3xk/q2J9ItiODRmyGisgHAQLqeuHIH5kbZqFM70xmZFodEGkEjovQayz68BC0UUUU3D7dxMB1bXADoOj9dJpkqgD6IG5yP4c9RrTHGNi+RjoWYx4JP74zmySpXLLRbDl9BFLZTme3qEPULQA9RBfuQRi+fD6n9qo7t2kBZfETaT5hQUGiLGvFT9BcG5egY/CACspc0kR2ItVkijrobBgZoLHJvGCiVXzSZUDQpJa9jN273ATpOrOLrSB2rasJCntg== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; To: devel@lists.libvirt.org Subject: [RFC PATCH 1/5] conf: Add a nestedSmmuv3 IOMMU model Date: Wed, 11 Dec 2024 16:24:19 -0800 Message-Id: <20241212002423.1050892-2-nathanc@nvidia.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241212002423.1050892-1-nathanc@nvidia.com> References: <20241212002423.1050892-1-nathanc@nvidia.com> X-ClientProxiedBy: SJ0PR05CA0208.namprd05.prod.outlook.com (2603:10b6:a03:330::33) To SN7PR12MB6838.namprd12.prod.outlook.com (2603:10b6:806:266::18) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SN7PR12MB6838:EE_|MN0PR12MB6103:EE_ X-MS-Office365-Filtering-Correlation-Id: 092cf2b1-994b-419a-75aa-08dd1a435e73 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|366016|376014; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?pD4OUHagfdv9TuXcVTwyr5qKodwjRevtCB2EU5w0Ynxf72pduPlOwIPzHpNp?= =?us-ascii?Q?ClWjET6wpLFcAv6oGlE+4iTJdGgF5rQcivGJVb8TW6GCZnPCIU2mb95mEM3p?= =?us-ascii?Q?L+wV1vSlXEsJInmdmsQy3+nbO3S7thEQSmPOlDPOFw5rS3NzLv1dgwuPhdWj?= =?us-ascii?Q?M7WzLYzaMMLYLKHC/zzpTKu/C/24Q1ix+VcZDRY6Tcz/d8HqQ/yK5EuCvCTN?= =?us-ascii?Q?vCucaeMIz6US2z/YGMkfnO9JxOeq5tPTGYuB69xRQmXXhDEl6/+lfGr3xgVI?= =?us-ascii?Q?YYHPs7Wv8IHwHzYmv5+toicWsBuFRq6P9V8TXUtVQhQwqAiWaQrfvKkuQfgV?= =?us-ascii?Q?XtPRigi2mlq0QDjwQPTFbPmVek4OeaADD2fjlxtPFL4qiNQx960hbcIFXarx?= =?us-ascii?Q?VE0yFydZh03Ccriia4iAcEYr90Nw+1WEsBBJxeLC5GY6lYQqnERpqZUv6BnP?= =?us-ascii?Q?miMHaB2H12kMdEfuwAoK1Jfp8HPeLNDh/VEVfOFiyBCK9P0MUhRaqD6FyUmZ?= =?us-ascii?Q?cGofshoKWH8OgZXwth4RQM/cBNYotL/W9kh11EUu9LnO7PSz8yTZzpoOJT2g?= =?us-ascii?Q?Rh854LNAR2EW7ZwNs0pc6NaCIk4OrzN5/srrIwwbQUYFcUV131kV90KqGEqZ?= =?us-ascii?Q?8gEGRdQ2/SdQuLdYMeb8DJCXsDStgyQ/LM0e24kysg28iZTRwFFEm9H1KHyz?= =?us-ascii?Q?Su/4DVnCeRmWURePCeksdWg8Siv5XE3WsZlGSrrgO0jdcTOV44MZVkNDhiOq?= =?us-ascii?Q?C+E13wQbUcKn2b5OgHpTRB/SVoOqBamAsDy7fBeOykQ8NOL8EzSFyxuc4zyb?= =?us-ascii?Q?dBo/RkaxvV8/twnU9saD2s1ukTjRn9F5EbN8LbaojDcIc7MyV4mcLMnmu0D6?= =?us-ascii?Q?F/1UzlBMBnpZb4x1S6JSve5SIySZ/vH8t43eawHkN9h9CBWqKF4H4JbXu+3V?= =?us-ascii?Q?PhJl/jc2/6+oa7pv/9hdOQY1PI6wfGD8fNIafz+7NIMqhcrkeuBhCjmCWQrX?= =?us-ascii?Q?dxtyzzfEYxSmdyGG/9NlcX98uZeAnuNZyIk7BSnRxLkUWUZvROXj0d7cniMj?= =?us-ascii?Q?oN5bSQ3wrIYT1iN6SeUsINs4hvTYJF0f/dK4DxlmPdDxhMRBSW/BD/PDp0L9?= =?us-ascii?Q?qYX/Y0fhrMzkDIrCpd56m6/1JL9x15oK3sjMK37Z6WREcxjLnI+ZZB+pzU/K?= =?us-ascii?Q?MrN/kHsNFKynYLxOL6i1JrlcgkCN4lIUDUSkMu77IONMVHSLPJbJwZKZ4Zf0?= =?us-ascii?Q?/14QcKbkgP2GxRX6cVHH9KGJD3/m0ifFSr9IkUSycRzXvDVNOnRg3q2XOUgL?= =?us-ascii?Q?dY9dFdC+0eNIpzh7/CsbjpoO+peDt5lm1w0a3MxnHJgKng=3D=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SN7PR12MB6838.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(366016)(376014);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?JvneOjohy1JjYc/lCcb1qC+x9jc+8F+W6Fj/hmJ4aea1HBIV4xPfiVbkv5Ng?= =?us-ascii?Q?yHrc+oqHS2DCyzhaKFCn39+8I4TglzzC8hwZJumrUgpE1s8koh5/W7wo6itf?= =?us-ascii?Q?iJMOTh4np3TDwOwbRxhFjlGSw0qVIjuXzwVd5OcIzLAyd0FlRZw4OIoMiVu+?= =?us-ascii?Q?JBtpyjQsMu24w7jZlnuWxYkzYm6Ju7BDFlrb0rS8yqZcPQ/RXFzD5iWfFh0L?= =?us-ascii?Q?ladTsexfNci/tKZWiAHqFwnUuuz8H3H8v+gB6FT0sPUNERLRhBS8/awU2NA4?= =?us-ascii?Q?daOmEoxmxzxkSgg9PyAH4Eumzmhp2UeLvyESAqJ8xc/3WMctc53QQPP84o0A?= =?us-ascii?Q?Kss8OrT3t0eD0cqKkREL5z/1BLL5R3XCKYo8B6VZFXHKf5YmGBbaq9v3du1Q?= =?us-ascii?Q?b9KuvqXXfWZbQ6xBNW/6yNq3LOjm8Unp7BpQHY/Ei3sU39vWu4FS0tPx+ici?= =?us-ascii?Q?GoU4DNxH1tkBe7eRPbRarX9tbjS03j1ClXA/79bamt+jBB73RBnsG8AqSfdq?= =?us-ascii?Q?l4E64BgUgOD8Rg1d7Ij5t8ETylaiPz6dyJgtBNO/Qm2YPRo/EB23GE4G2ojf?= =?us-ascii?Q?kcVguyr1n8URIhMIHmDucRgm42Sty27Vg/22N4v6jweKBCIdDpQ0RJs6EdDz?= =?us-ascii?Q?WQeULNJQTydT2iO+U09roK5uMfsgLHt8MtkDS0mXOlaqFXyb0rR45D8qAVPe?= =?us-ascii?Q?d9I1l7OHlr5XnOCSUt0vv0QSI7D+aYU/TDQroLgVIrOFxChMVRzLXlG1JmTA?= =?us-ascii?Q?WqqyWLN3SxoMfUUv3cqdEQ/4xdCyuvVsQ2es4Af0D/IO0bDn0eUPejUPjTlH?= =?us-ascii?Q?U2jzVCH2lJX/gC6K2NsaDhewaDRb1pVm1DXl7GCYulytpbzquTOoNZPAcNSm?= =?us-ascii?Q?VFVK+soQUqMb0ZJoCikuQg2wHtAMk+YO6oSKerRvGwYlULrKraJ6hcPemSs3?= =?us-ascii?Q?Vg4+cb/pnnT1jVlKevJVvfsBsdAHDzyhqJFGCFBAdam1k/y8MQxO1A/T+7ts?= =?us-ascii?Q?SyH0a0IG+BxlJVhSjIdieb4bHTfQZ3fs8j+5ODQH5rgdbGzBO7k04fUZCIh+?= =?us-ascii?Q?/u2xCqXK55FJ9oIeDjHbV9LmwaRveUFU37wOIoSt8QTx+UUy9fdICFTn8TdM?= =?us-ascii?Q?Nix/gHyGbDHULcBt8x/QNNnICrf6Qz6W2y4NPHKkFep4SCcU6OgNoLoHRqi5?= =?us-ascii?Q?Wf3AGS86tbsqbbfEI/pWXTRumEkLkuwZJ81Mmzhma7aXwoZFLbfedzugPpJt?= =?us-ascii?Q?TKWezJBP11xQmP7f3pLa2m1zdQMilwGx8TwGytqXsdLEANN4s7GgTY6hWomR?= =?us-ascii?Q?h7thfe/3lPKz60B6pZlT9DEShqOc0JrJ2BN+aDnfrj8RdcEUwQg2Nx1UnHCW?= =?us-ascii?Q?ot9qrFLn6tz2EfnImVYIiWwqz6sTiHqolY5bkUHyqowgGiPlhDV39ronZckT?= =?us-ascii?Q?Qqk0j2bMEq94Dc3accP1kkgUXYLG3sfLFKHsKVudqEftlhsv6W2dptUGmbUS?= =?us-ascii?Q?4u8I7IDXMBWzVeEOUiALmgcD7DXEKqLe6cQkNaXTrCsWCy2KoewMcStAdY0f?= =?us-ascii?Q?ZWMMxgJ6rgMVROL93BAalVxbYUMk4DP3+jA1SnWC?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 092cf2b1-994b-419a-75aa-08dd1a435e73 X-MS-Exchange-CrossTenant-AuthSource: SN7PR12MB6838.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Dec 2024 00:24:40.7147 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 60womefQRdKBT9yQ0szVk58DfOo2vdheRvSa/k+vDLQgSUtZaWEJIQw8tdhI3xOzvPF2I2lT92cGXSJd9oBvnQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN0PR12MB6103 Content-Transfer-Encoding: quoted-printable Message-ID-Hash: OT2RXQLFAHGSQQ6EBOT775PAIUJYUIXI X-Message-ID-Hash: OT2RXQLFAHGSQQ6EBOT775PAIUJYUIXI X-MailFrom: nathanc@nvidia.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-config-1; header-match-config-2; header-match-config-3; header-match-devel.lists.libvirt.org-0; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; suspicious-header CC: Nicolin Chen , Shameerali Kolothum Thodi , Nathan Chen X-Mailman-Version: 3.2.2 Precedence: list List-Id: Development discussions about the libvirt library & tools Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: From: Nathan Chen via Devel Reply-To: Nathan Chen X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1733963147801116600 Content-Type: text/plain; charset="utf-8" Add support for specifying "nestedSmmuv3" as the IOMMU model. In the follow= ing commits, when the "nestedSmmuv3" IOMMU model is parsed from the VM definiti= on, PXB controllers and "nestedSmmuv3" devices will be auto-added to the VM definition and VFIO devices will be routed to PXB controllers based on their association with host SMMU nodes. The auto-added "nestedSmmuv3" devices will each be attached to a PXB controller so that VFIO devices with unique host SMMU node associations are attached to unique PXB controllers. Signed-off-by: Nathan Chen --- docs/formatdomain.rst | 5 +++-- src/conf/domain_conf.c | 1 + src/conf/domain_conf.h | 1 + src/conf/domain_validate.c | 2 ++ src/conf/schemas/domaincommon.rng | 1 + src/qemu/qemu_command.c | 6 +++++- src/qemu/qemu_domain_address.c | 2 ++ src/qemu/qemu_validate.c | 15 +++++++++++++++ 8 files changed, 30 insertions(+), 3 deletions(-) diff --git a/docs/formatdomain.rst b/docs/formatdomain.rst index 60bee8bd4f..63bb565991 100644 --- a/docs/formatdomain.rst +++ b/docs/formatdomain.rst @@ -8729,8 +8729,9 @@ Example: ... =20 ``model`` - Supported values are ``intel`` (for Q35 guests) ``smmuv3`` - (:since:`since 5.5.0`, for ARM virt guests), and ``virtio`` + Supported values are ``intel`` (for Q35 guests), ``smmuv3`` + (:since:`since 5.5.0`, for ARM virt guests), ``nestedSmmuv3`` + (for ARM virt guests), and ``virtio`` (:since:`since 8.3.0`, for Q35 and ARM virt guests). =20 ``driver`` diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index 4ad8289b89..c1092551e0 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -1349,6 +1349,7 @@ VIR_ENUM_IMPL(virDomainIOMMUModel, "intel", "smmuv3", "virtio", + "nestedSmmuv3", ); =20 VIR_ENUM_IMPL(virDomainVsockModel, diff --git a/src/conf/domain_conf.h b/src/conf/domain_conf.h index a187ab4083..f8ab1b7d2f 100644 --- a/src/conf/domain_conf.h +++ b/src/conf/domain_conf.h @@ -2944,6 +2944,7 @@ typedef enum { VIR_DOMAIN_IOMMU_MODEL_INTEL, VIR_DOMAIN_IOMMU_MODEL_SMMUV3, VIR_DOMAIN_IOMMU_MODEL_VIRTIO, + VIR_DOMAIN_IOMMU_MODEL_NESTED_SMMUV3, =20 VIR_DOMAIN_IOMMU_MODEL_LAST } virDomainIOMMUModel; diff --git a/src/conf/domain_validate.c b/src/conf/domain_validate.c index 1034bb57f5..c8b7b701bf 100644 --- a/src/conf/domain_validate.c +++ b/src/conf/domain_validate.c @@ -3001,6 +3001,7 @@ virDomainIOMMUDefValidate(const virDomainIOMMUDef *io= mmu) { switch (iommu->model) { case VIR_DOMAIN_IOMMU_MODEL_SMMUV3: + case VIR_DOMAIN_IOMMU_MODEL_NESTED_SMMUV3: case VIR_DOMAIN_IOMMU_MODEL_VIRTIO: if (iommu->intremap !=3D VIR_TRISTATE_SWITCH_ABSENT || iommu->caching_mode !=3D VIR_TRISTATE_SWITCH_ABSENT || @@ -3022,6 +3023,7 @@ virDomainIOMMUDefValidate(const virDomainIOMMUDef *io= mmu) =20 switch (iommu->model) { case VIR_DOMAIN_IOMMU_MODEL_SMMUV3: + case VIR_DOMAIN_IOMMU_MODEL_NESTED_SMMUV3: case VIR_DOMAIN_IOMMU_MODEL_INTEL: if (iommu->info.type !=3D VIR_DOMAIN_DEVICE_ADDRESS_TYPE_NONE) { virReportError(VIR_ERR_XML_ERROR, diff --git a/src/conf/schemas/domaincommon.rng b/src/conf/schemas/domaincom= mon.rng index b3fdbf7ffb..de6a1e5c7e 100644 --- a/src/conf/schemas/domaincommon.rng +++ b/src/conf/schemas/domaincommon.rng @@ -6144,6 +6144,7 @@ intel smmuv3 virtio + nestedSmmuv3 diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c index dcb9c4934e..6629addace 100644 --- a/src/qemu/qemu_command.c +++ b/src/qemu/qemu_command.c @@ -6063,6 +6063,9 @@ qemuBuildIOMMUCommandLine(virCommand *cmd, /* There is no -device for SMMUv3, so nothing to be done here */ return 0; =20 + case VIR_DOMAIN_IOMMU_MODEL_NESTED_SMMUV3: + return 0; + case VIR_DOMAIN_IOMMU_MODEL_LAST: default: virReportEnumRangeError(virDomainIOMMUModel, iommu->model); @@ -6885,7 +6888,8 @@ qemuBuildMachineCommandLine(virCommand *cmd, case VIR_DOMAIN_IOMMU_MODEL_VIRTIO: /* These IOMMUs are formatted in qemuBuildIOMMUCommandLine */ break; - + case VIR_DOMAIN_IOMMU_MODEL_NESTED_SMMUV3: + break; case VIR_DOMAIN_IOMMU_MODEL_LAST: default: virReportEnumRangeError(virDomainIOMMUModel, def->iommu->model= ); diff --git a/src/qemu/qemu_domain_address.c b/src/qemu/qemu_domain_address.c index 970ae3949d..51f4bbd6eb 100644 --- a/src/qemu/qemu_domain_address.c +++ b/src/qemu/qemu_domain_address.c @@ -943,6 +943,7 @@ qemuDomainDeviceCalculatePCIConnectFlags(virDomainDevic= eDef *dev, =20 case VIR_DOMAIN_IOMMU_MODEL_INTEL: case VIR_DOMAIN_IOMMU_MODEL_SMMUV3: + case VIR_DOMAIN_IOMMU_MODEL_NESTED_SMMUV3: case VIR_DOMAIN_IOMMU_MODEL_LAST: /* These are not PCI devices */ return 0; @@ -2367,6 +2368,7 @@ qemuDomainAssignDevicePCISlots(virDomainDef *def, =20 case VIR_DOMAIN_IOMMU_MODEL_INTEL: case VIR_DOMAIN_IOMMU_MODEL_SMMUV3: + case VIR_DOMAIN_IOMMU_MODEL_NESTED_SMMUV3: case VIR_DOMAIN_IOMMU_MODEL_LAST: /* These are not PCI devices */ break; diff --git a/src/qemu/qemu_validate.c b/src/qemu/qemu_validate.c index aaa056379e..9f07ffe4a3 100644 --- a/src/qemu/qemu_validate.c +++ b/src/qemu/qemu_validate.c @@ -5104,6 +5104,21 @@ qemuValidateDomainDeviceDefIOMMU(const virDomainIOMM= UDef *iommu, } break; =20 + case VIR_DOMAIN_IOMMU_MODEL_NESTED_SMMUV3: + if (!qemuDomainIsARMVirt(def)) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, + _("IOMMU device: '%1$s' is only supported with = ARM Virt machines"), + virDomainIOMMUModelTypeToString(iommu->model)); + return -1; + } + if (!virQEMUCapsGet(qemuCaps, QEMU_CAPS_MACHINE_VIRT_IOMMU)) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, + _("IOMMU device: '%1$s' is not supported with t= his QEMU binary"), + virDomainIOMMUModelTypeToString(iommu->model)); + return -1; + } + break; + case VIR_DOMAIN_IOMMU_MODEL_LAST: default: virReportEnumRangeError(virDomainIOMMUModel, iommu->model); --=20 2.34.1 From nobody Sun Dec 22 03:14:30 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.libvirt.org designates 8.43.85.245 as permitted sender) client-ip=8.43.85.245; envelope-from=devel-bounces@lists.libvirt.org; helo=lists.libvirt.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of lists.libvirt.org designates 8.43.85.245 as permitted sender) smtp.mailfrom=devel-bounces@lists.libvirt.org; arc=fail (BodyHash is different from the expected one); dmarc=pass(p=reject dis=none) header.from=lists.libvirt.org Return-Path: Received: from lists.libvirt.org (lists.libvirt.org [8.43.85.245]) by mx.zohomail.com with SMTPS id 1733963230409492.7564293667592; Wed, 11 Dec 2024 16:27:10 -0800 (PST) Received: by lists.libvirt.org (Postfix, from userid 996) id 44EEB11C8; Wed, 11 Dec 2024 19:27:09 -0500 (EST) Received: from lists.libvirt.org (localhost [IPv6:::1]) by lists.libvirt.org (Postfix) with ESMTP id 183B7E75; Wed, 11 Dec 2024 19:25:13 -0500 (EST) Received: by lists.libvirt.org (Postfix, from userid 996) id CAB4A11DD; Wed, 11 Dec 2024 19:25:07 -0500 (EST) Received: from NAM02-BN1-obe.outbound.protection.outlook.com (mail-bn1nam02on2057.outbound.protection.outlook.com [40.107.212.57]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.libvirt.org (Postfix) with ESMTPS id DF34EE72 for ; Wed, 11 Dec 2024 19:24:49 -0500 (EST) Received: from SN7PR12MB6838.namprd12.prod.outlook.com (2603:10b6:806:266::18) by CY5PR12MB6322.namprd12.prod.outlook.com (2603:10b6:930:21::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8251.14; Thu, 12 Dec 2024 00:24:44 +0000 Received: from SN7PR12MB6838.namprd12.prod.outlook.com ([fe80::529d:478:bc5d:b400]) by SN7PR12MB6838.namprd12.prod.outlook.com ([fe80::529d:478:bc5d:b400%6]) with mapi id 15.20.8251.008; Thu, 12 Dec 2024 00:24:44 +0000 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on lists.libvirt.org X-Spam-Level: X-Spam-Status: No, score=0.2 required=5.0 tests=DKIM_INVALID,DKIM_SIGNED, FORGED_SPF_HELO,MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H2, RCVD_IN_VALIDITY_RPBL_BLOCKED,RCVD_IN_VALIDITY_SAFE_BLOCKED, SPF_HELO_PASS autolearn=no autolearn_force=no version=3.4.4 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=yzlDiVK6cjL6xwv2C+CtoAJNUy//96Uf2YUgzuu/2MCTxhZ9BDj6XcVti4RMxYppxpWdat3g0rKkfYMNn2foPkRJ01GINCG19/3l37jApIP7abncz37iaNZHLiFdQGEsmL2BWgXrPhCnSuo5I4+7go3o+IsSLsrNBqbJbeTgnW+1j+hEBtlii1m324HWdTT4fYqIamVaD6ETIaSTvjqI/6BZB4HuAjMokiXhA7zJw4fO5Pe5QTsOcLfigQHCFLWFLC6KBp4ob/K/YtXBsW4tkVq2iWDenIriTGveqauKKlH0s/xu1CIwduvk6ZT3H0WmWZ9kYswjt0fJ4b1jJPun3w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=rd9d+D1r6yi5aGOcQ/LMhgr5O9aQ7ya7ojDg9lJWbCI=; b=x6FnppdjY3cpv7BG23jMJyb0XOVjcJAq5/HEcn/z6X9gRasB0+jhRYnagTmiITfOMQTTQ69zdfkUoeawibFmA/rx8ePEEkT2/K/A73RbdP8S5ZLRNB75GZc/qr3Jn2TRm8NrAKDBc3ibtY6VWViWqZqNv2Cst6MUpjdPsQYQKv9/rYojs7vrkSC7RWCjrtxEw8EX9mCycdH33IArZyhFjuVHOkMh/vbL+/WaRXwjY6oG4mRdPdPaZPeK7qkDQGWjQo8w8mdMarlG8hQvjA3JvwWN5NMasZwTQcHDRwbARmr20LJ9cOWbdbr7V4gVUH5TwUhJisD0I7MO1FYBJdzrAA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=rd9d+D1r6yi5aGOcQ/LMhgr5O9aQ7ya7ojDg9lJWbCI=; b=HRbdr/HhyZ2ZH/2nsxxvlSr0KUef7aun6Pa2mQ57VOfzfnqLOUMwhYPIiifnqHRz0Acg6Tl4IKpHjnPBnUqY2ZFr1kjRvC8h/EoCWxRchBlFrdAoLYZdX+ZiTi0u7Zv4C3kNTWxzbeuwCGw6rPBzrbcE/FRtYuEhzFRTvGAeLierT93IGxMB7GmbfnWxyECpu4W8DyOuySlScFGzib2/GDGTkE2Uo/MZzY2fNVDb0/hxik55HGmq/Sh/qlU5asCjmAO/wN3VnxZPyGoQ26dgu5Zkbp0WG12BavHg6sRBgup0WE056AgfzRUlaYTXJ4BH8lym5J+L5lh5erauEaAgYA== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; To: devel@lists.libvirt.org Subject: [RFC PATCH 2/5] qemu: Implement and auto-add a nestedSmmuv3 device type Date: Wed, 11 Dec 2024 16:24:20 -0800 Message-Id: <20241212002423.1050892-3-nathanc@nvidia.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241212002423.1050892-1-nathanc@nvidia.com> References: <20241212002423.1050892-1-nathanc@nvidia.com> X-ClientProxiedBy: SJ0PR05CA0206.namprd05.prod.outlook.com (2603:10b6:a03:330::31) To SN7PR12MB6838.namprd12.prod.outlook.com (2603:10b6:806:266::18) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SN7PR12MB6838:EE_|CY5PR12MB6322:EE_ X-MS-Office365-Filtering-Correlation-Id: 1ad1d511-22ae-4bbc-6911-08dd1a43609e X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|376014|366016; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?xaWPLcTtBeh3ItH4mqdi/ARi7B+Ls89AET4DQyeWdOcSQ1sWzEeHsujWKwMo?= =?us-ascii?Q?eVfBVLZwZ2WXrFYQqdJm4V4EgWUwnkihLjNkc4QoejN0BoSctT/GpduGz3oe?= =?us-ascii?Q?YS59pEg/S6NYxx8ET7FdII+H4zZvVFHELNiNUXno8QRjIXFQ6tJUNlUo3XOD?= =?us-ascii?Q?YwMDBPNLHya/eCZkq9A3G/tJvs3zhWBQgNLtn+CYkPQf/dSpwwZmDPcRETWd?= =?us-ascii?Q?z48nEQ3glGbszkvRjBm93WMtkRbJXxrCHIrfXMGCZDHzX7aQFBfnzeBYoKzc?= =?us-ascii?Q?D5kZAWXoPXRTKKjZDOfOKZ66YWhd3aswhAfwwpA8p1bXnh7eRC/wZf+OaOYp?= =?us-ascii?Q?ysA/Ki2yO3nNXCrbXuXpEvZPo90p41S5wrplGXA5SzCiiKB5keQhS9n2rSUH?= =?us-ascii?Q?4WpFJIwBi2+wwDVUFI3EB54hFSMaM/b6o5T4HnGAkS/35QcUbdu2F9XnBU/j?= =?us-ascii?Q?z2y9EdrcxdM/HAtoQxlG4D3rqqaYgA6/YHgVkzGZyCzUI6nIKhC9GvvzCWkq?= =?us-ascii?Q?D61AT4Lw7mdepfjoxGo5Wv1pi03tGUfj3UTB9atC6EBAXTGhcAJE4UyVJDEh?= =?us-ascii?Q?45uT5Q+leFR0RrEXj8OLspWJbioFg4zMB8RtxKtSM0OLEoDN6MJXvcMfFfmA?= =?us-ascii?Q?aL5QISMZS136LRZyWNyBp2hWAY9pGLweqnWGUKo4ITEaf1XpvOrJ0J25B8VX?= =?us-ascii?Q?b0ETdo+tAul6VTst7ZWV8FBvfhG/5muY+/FBKR3KmN//ypr/rg3CIKcLpifC?= =?us-ascii?Q?BKnQDqsflQCh2qzBCDf97x/dhfhLjgz1VQdhIw4rycJd6bi5VMJLhTQuGnrQ?= =?us-ascii?Q?vMJSt4MBfp7hIfC+5YTwQ9b0/o1jN8jEpCy8on5yNgRY1n4JDbcL31bltj44?= =?us-ascii?Q?nhfTV/g+4eY0v/7cdfet1ADu/dYRy5dTEPAnuw6RzPdzAyhWC1Eq4k/fB5Y6?= =?us-ascii?Q?Hv3G0D3zQ0zdUGO103vU/1DuVs0qHpsmJx+rLB1PYFcL8AP3xTxuMhYdR7QP?= =?us-ascii?Q?dpA1N450hsMa1cKBPkA1vUhLm+bL/ZnqHOc+BdQ0e6mPDGLQhPQ04gDBwx/+?= =?us-ascii?Q?jUR7kFeq0ins1xOGr9VvluVNlXk2E4prxcnzWcunJHy/87KG3pYJZyZEAmuf?= =?us-ascii?Q?iYgNUZUYu95N3ELKLBF80J8dcm1gH9tpSF3NAKeLjfgCUhw/mCXGKhvcFr+S?= =?us-ascii?Q?3IJumx4BMFexa8foU/+S15/W3Q+ODyo3bLV/xNBpceOvlgTTITg6QKxrsqU9?= =?us-ascii?Q?4Zy9Wa+YsCgZkKt5+JdPG9dSSc107m88QiBIVngs9RkNEC6JgXGtw6UQ1ak+?= =?us-ascii?Q?r93aoXywKoey4LtP2dN+J/g65cD7uwntMFhETBoCIrv9xA=3D=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SN7PR12MB6838.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(376014)(366016);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?5ni+fLA+1zXrxrVUWBFUpwaTY6jvxZhLpHZStalMpsEjw1J0meJ7+sV4HOot?= =?us-ascii?Q?q1bAur2ClbxJXTG1obNkBdyISP6lqDOcC2MluR4iDy+3q9HssdW8B5B1+HGI?= =?us-ascii?Q?OaLaIoI4QaioQjNxyteAFDvvWwoKvKgUvAiLKz8LjFSQHfS4MxsrfjbyCG1U?= =?us-ascii?Q?hPJuFgBRjs9sk+h+ndGvYKdY8u4oM0CG5ksHvzPRdWKkFhFapLLX3xDw9i1U?= =?us-ascii?Q?JKf4s6JftvBMfa7frM27HYPbxXTEoUUOFd/e+OiXLywL4loIXnW55PKiDBfM?= =?us-ascii?Q?PN+2oDBOrXSlnzxRPsgqqSf9DzcCvafd0DKlTfi+urFGbpeDC/vOUsBgkDBx?= =?us-ascii?Q?BQw7HexLr74tCCGEdIsh6vBMeTA/zaSB6/XZJxvIDqu+wV5k4CeYyyGLjEI4?= =?us-ascii?Q?0GbV7R5ByDmB9naPLh+o5r0G0BcrWiEY+VgKy07/zd6jJ6VZuHs1ECbW78Ze?= =?us-ascii?Q?4nZjEV2Rk9QKpOSND4DVoX1tnhaB2bTVnhDnNKquT7ZqSj2Y+0DhN0iCuNQs?= =?us-ascii?Q?JdHSbGsnlmrd/C06kQNvpUAKFMqmVUJPotFeooHVto+VhqPIRAhyS4lM2KWc?= =?us-ascii?Q?T1kOM7B/eqVupXp2GKjOO95ojdza4JQenRTQxxTbHcDDJ+8QIIslcq7/PibW?= =?us-ascii?Q?1WCCZBeeke+Vle0cpI9CO1myKuJARTHkOZxtTPtTsQAdWta/doLiqwmgue6n?= =?us-ascii?Q?nFkrcrG4KwOT5H4nv3uIHr57Misar8xTZKCbAsm3uEOD65PLgCEPqE0xcFLU?= =?us-ascii?Q?Hja7+GEHaFp5D0elKOGPX7FuxqcUuX09dzCo+4NP/vOkVcJQXmnR9PzLM48D?= =?us-ascii?Q?nkA9xMMGPmvoML5gCThz9R8nJnnPW5Pevegsd0Yy7my3LgIUsLtah8a+T6Br?= =?us-ascii?Q?gL16pKhKNBrsTcEGcVhSKFBu+IWDzW/MNrqppawrCxzoHFgieBFQCgIgNIp9?= =?us-ascii?Q?DMejX9Ca6wS1ZTr4TmhrFajGCvZV0MrGJB+0MzSj8QhuDVZfGus0Qj5cRy54?= =?us-ascii?Q?OsJyk7ew3IUt8h5uWrm4qiAxrwoj0JCvg9XxIAx5AAsVYQhZdpaBD33LsgJr?= =?us-ascii?Q?Y7vy2KYGWq9AGyx/mWQlr4fJAj5sc5xsspAMFRbFzAD/lHfU2nLudtKynRPl?= =?us-ascii?Q?kvqTHWCXpoMIaoMURITpxNYctFXWOwaWjh0k0fv5ANOaBuHwFYl+c5YZ4CSo?= =?us-ascii?Q?aqZgQfOxtolUrVntQrxE9DQNcTMtRAH487s8xwmYmkcN2rejclLKVNKjYLso?= =?us-ascii?Q?PYOQERtLBUwThacaTxDODwzoLqZ3seHDHB4MgaN+Aj+1ZRs/vwuRdL3yK7Xp?= =?us-ascii?Q?gIAGf7aivrQ8cucOb3yTmwhD0rtiqzjCkgtu1kjcHDap5kEV6M1zeOkaJmS3?= =?us-ascii?Q?5OKHdDbA4J232rGt3O6E+hmnsAyAsN1bgeBl1VSmTTbL5uaFk91kDfQFXI4f?= =?us-ascii?Q?72AUn5HbROxGV8eWovGRi0MrIbxt+fy+Cqzbwby+I3HdyNpK6k4zl9UAVvQH?= =?us-ascii?Q?Jy7bU61h40JbDN4C7uAWp5+FuSyj1kuHrkpLcg1FG3SnMdl8RE5UBa/k7CTd?= =?us-ascii?Q?Yr0sXt973yPfkgRrVLRM6iErhPe38LmAkBwVyvut?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 1ad1d511-22ae-4bbc-6911-08dd1a43609e X-MS-Exchange-CrossTenant-AuthSource: SN7PR12MB6838.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Dec 2024 00:24:44.3995 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: P9o5R9eFMJT0XGb7nns4C3BTPsEX9H0+zYhhedkdMUKn9WoxLjN15PgW7HogaOlEdds/piYW6i7nLA4gIEu9dg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY5PR12MB6322 Content-Transfer-Encoding: quoted-printable Message-ID-Hash: CNG5CPILNYNTJO62IUJH46UCZR4G7WQI X-Message-ID-Hash: CNG5CPILNYNTJO62IUJH46UCZR4G7WQI X-MailFrom: nathanc@nvidia.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-config-1; header-match-config-2; header-match-config-3; header-match-devel.lists.libvirt.org-0; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; suspicious-header CC: Nicolin Chen , Shameerali Kolothum Thodi , Nathan Chen X-Mailman-Version: 3.2.2 Precedence: list List-Id: Development discussions about the libvirt library & tools Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: From: Nathan Chen via Devel Reply-To: Nathan Chen X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1733963232334116600 Content-Type: text/plain; charset="utf-8" Add support for parsing "nestedSmmuv3" devices from the VM definition. When= a "nestedSmmuv3" IOMMU model is parsed from the VM definition, add a "nestedSmmuv3" device to the VM definition for each host SMMU node detected= from sysfs. Specify the associated host SMMU sysfs filename as the "name" attrib= ute for each "nestedSmmuv3" device in the VM definition. Signed-off-by: Nathan Chen --- docs/formatdomain.rst | 20 ++++ src/ch/ch_domain.c | 1 + src/conf/domain_addr.h | 1 + src/conf/domain_conf.c | 186 ++++++++++++++++++++++++++++++ src/conf/domain_conf.h | 12 ++ src/conf/domain_postparse.c | 1 + src/conf/domain_validate.c | 22 ++++ src/conf/schemas/domaincommon.rng | 16 +++ src/conf/virconftypes.h | 2 + src/lxc/lxc_driver.c | 6 + src/qemu/qemu_command.c | 58 ++++++++++ src/qemu/qemu_command.h | 4 + src/qemu/qemu_domain.c | 2 + src/qemu/qemu_domain_address.c | 5 + src/qemu/qemu_driver.c | 3 + src/qemu/qemu_hotplug.c | 5 + src/qemu/qemu_postparse.c | 1 + src/qemu/qemu_validate.c | 1 + src/test/test_driver.c | 4 + tests/schemas/device.rng.in | 1 + 20 files changed, 351 insertions(+) diff --git a/docs/formatdomain.rst b/docs/formatdomain.rst index 63bb565991..739b5b1b89 100644 --- a/docs/formatdomain.rst +++ b/docs/formatdomain.rst @@ -8800,6 +8800,26 @@ The optional ``driver`` element allows to specify vi= rtio options, see ... =20 =20 +NestedSmmuv3 +~~~~~~~~~~~~ +A representation of a host SMMU node. This is used to keep track of which +controller to assign a VFIO device to in the VM if it is associated with a +host SMMU node. This supports having multiple vSMMU nodes in the VM by +attaching devices with different SMMU nodes to different pcie-expander-bus +controllers in the VM. The ``name`` attribute denotes the host SMMU node's +identifier parsed from the host's sysfs. + +:: + + ... + + + smmu3.0x0000000005000000 + + + ... + + Crypto ~~~~~~ =20 diff --git a/src/ch/ch_domain.c b/src/ch/ch_domain.c index bfccabed49..7e7ca1c6ac 100644 --- a/src/ch/ch_domain.c +++ b/src/ch/ch_domain.c @@ -180,6 +180,7 @@ chValidateDomainDeviceDef(const virDomainDeviceDef *dev, case VIR_DOMAIN_DEVICE_IOMMU: case VIR_DOMAIN_DEVICE_AUDIO: case VIR_DOMAIN_DEVICE_CRYPTO: + case VIR_DOMAIN_DEVICE_NESTED_SMMUV3: case VIR_DOMAIN_DEVICE_PSTORE: virReportError(VIR_ERR_CONFIG_UNSUPPORTED, _("Cloud-Hypervisor doesn't support '%1$s' device"), diff --git a/src/conf/domain_addr.h b/src/conf/domain_addr.h index e72fb48847..9781685903 100644 --- a/src/conf/domain_addr.h +++ b/src/conf/domain_addr.h @@ -57,6 +57,7 @@ typedef enum { VIR_PCI_CONNECT_TYPE_PCIE_EXPANDER_BUS =3D 1 << 11, VIR_PCI_CONNECT_TYPE_PCI_BRIDGE =3D 1 << 12, VIR_PCI_CONNECT_TYPE_PCIE_TO_PCI_BRIDGE =3D 1 << 13, + VIR_PCI_CONNECT_TYPE_NESTED_SMMUV3 =3D 1 << 14, } virDomainPCIConnectFlags; =20 /* a combination of all bits that describe the type of connections diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index c1092551e0..24aff1cfbe 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -341,6 +341,7 @@ VIR_ENUM_IMPL(virDomainDevice, "audio", "crypto", "pstore", + "nestedSmmuv3", ); =20 VIR_ENUM_IMPL(virDomainDiskDevice, @@ -3449,6 +3450,19 @@ virDomainHostdevDefNew(void) } =20 =20 +virDomainNestedSmmuv3Def * +virDomainNestedSmmuv3DefNew(void) +{ + virDomainNestedSmmuv3Def *def; + + def =3D g_new0(virDomainNestedSmmuv3Def, 1); + + def->info =3D g_new0(virDomainDeviceInfo, 1); + + return def; +} + + static virDomainTPMDef * virDomainTPMDefNew(virDomainXMLOption *xmlopt) { @@ -3509,6 +3523,18 @@ void virDomainHostdevDefFree(virDomainHostdevDef *de= f) g_free(def); } =20 +void virDomainNestedSmmuv3DefFree(virDomainNestedSmmuv3Def *def) +{ + if (!def) + return; + + g_free(def->name); + + virDomainDeviceInfoFree(def->info); + + g_free(def); +} + void virDomainHubDefFree(virDomainHubDef *def) { if (!def) @@ -3671,6 +3697,9 @@ void virDomainDeviceDefFree(virDomainDeviceDef *def) case VIR_DOMAIN_DEVICE_PSTORE: virDomainPstoreDefFree(def->data.pstore); break; + case VIR_DOMAIN_DEVICE_NESTED_SMMUV3: + virDomainNestedSmmuv3DefFree(def->data.nestedsmmuv3); + break; case VIR_DOMAIN_DEVICE_LAST: case VIR_DOMAIN_DEVICE_NONE: break; @@ -4597,6 +4626,8 @@ virDomainDeviceGetInfo(const virDomainDeviceDef *devi= ce) return &device->data.crypto->info; case VIR_DOMAIN_DEVICE_PSTORE: return &device->data.pstore->info; + case VIR_DOMAIN_DEVICE_NESTED_SMMUV3: + return device->data.nestedsmmuv3->info; =20 /* The following devices do not contain virDomainDeviceInfo */ case VIR_DOMAIN_DEVICE_LEASE: @@ -4705,6 +4736,9 @@ virDomainDeviceSetData(virDomainDeviceDef *device, case VIR_DOMAIN_DEVICE_PSTORE: device->data.pstore =3D devicedata; break; + case VIR_DOMAIN_DEVICE_NESTED_SMMUV3: + device->data.nestedsmmuv3 =3D devicedata; + break; case VIR_DOMAIN_DEVICE_NONE: case VIR_DOMAIN_DEVICE_LAST: break; @@ -4930,6 +4964,13 @@ virDomainDeviceInfoIterateFlags(virDomainDef *def, return rc; } =20 + device.type =3D VIR_DOMAIN_DEVICE_NESTED_SMMUV3; + for (i =3D 0; i < def->nnestedsmmus; i++) { + device.data.nestedsmmuv3 =3D def->nestedsmmus[i]; + if ((rc =3D cb(def, &device, def->nestedsmmus[i]->info, opaque)) != =3D 0) + return rc; + } + /* If the flag below is set, make sure @cb can handle @info being NULL= */ if (iteratorFlags & DOMAIN_DEVICE_ITERATE_MISSING_INFO) { device.type =3D VIR_DOMAIN_DEVICE_GRAPHICS; @@ -4990,6 +5031,7 @@ virDomainDeviceInfoIterateFlags(virDomainDef *def, case VIR_DOMAIN_DEVICE_AUDIO: case VIR_DOMAIN_DEVICE_CRYPTO: case VIR_DOMAIN_DEVICE_PSTORE: + case VIR_DOMAIN_DEVICE_NESTED_SMMUV3: break; } #endif @@ -13365,6 +13407,40 @@ virDomainHostdevDefParseXML(virDomainXMLOption *xm= lopt, } =20 =20 +static virDomainNestedSmmuv3Def * +virDomainNestedSmmuv3DefParseXML(virDomainXMLOption *xmlopt, + xmlNodePtr node, + xmlXPathContextPtr ctxt, + unsigned int flags) +{ + virDomainNestedSmmuv3Def *def; + size_t nameLength; + VIR_XPATH_NODE_AUTORESTORE(ctxt) + + ctxt->node =3D node; + + if (!(def =3D virDomainNestedSmmuv3DefNew())) + goto error; + + nameLength =3D strlen(virXPathString("string(./name)", ctxt)) + 1; + VIR_REALLOC_N(def->name, nameLength); + if (!def->name) + goto error; + virStrcpy(def->name, virXPathString("string(./name)", ctxt), nameLengt= h); + + if (def->info->type =3D=3D VIR_DOMAIN_DEVICE_ADDRESS_TYPE_NONE) { + if (virDomainDeviceInfoParseXML(xmlopt, node, ctxt, def->info, + flags) < 0) + goto error; + } + return def; + + error: + virDomainNestedSmmuv3DefFree(def); + return NULL; +} + + static virDomainRedirdevDef * virDomainRedirdevDefParseXML(virDomainXMLOption *xmlopt, xmlNodePtr node, @@ -14366,6 +14442,12 @@ virDomainDeviceDefParse(const char *xmlStr, return NULL; } break; + case VIR_DOMAIN_DEVICE_NESTED_SMMUV3: + if (!(dev->data.nestedsmmuv3 =3D virDomainNestedSmmuv3DefParseXML(= xmlopt, node, + ct= xt, flags))) { + return NULL; + } + break; case VIR_DOMAIN_DEVICE_NONE: case VIR_DOMAIN_DEVICE_LAST: break; @@ -19445,6 +19527,21 @@ virDomainDefParseXML(xmlXPathContextPtr ctxt, =20 VIR_FREE(nodes); =20 + /* analysis of the nested SMMUs */ + if ((n =3D virXPathNodeSet("./devices/nestedSmmuv3", ctxt, &nodes)) < = 0) + return NULL; + if (n > 0) + VIR_REALLOC_N(def->nestedsmmus, def->nnestedsmmus + n); + for (i =3D 0; i < n; i++) { + virDomainNestedSmmuv3Def *nestedsmmuv3; + nestedsmmuv3 =3D virDomainNestedSmmuv3DefParseXML(xmlopt, nodes[i]= , ctxt, + flags); + if (!nestedsmmuv3) + return NULL; + def->nestedsmmus[def->nnestedsmmus++] =3D nestedsmmuv3; + } + VIR_FREE(nodes); + /* analysis of the host devices */ if ((n =3D virXPathNodeSet("./devices/hostdev", ctxt, &nodes)) < 0) return NULL; @@ -20664,6 +20761,20 @@ virDomainHostdevDefCheckABIStability(virDomainHost= devDef *src, } =20 =20 +static bool +virDomainNestedSmmuv3DefCheckABIStability(virDomainNestedSmmuv3Def *src, + virDomainNestedSmmuv3Def *dst) +{ + if (STRNEQ(src->name, dst->name)) + return false; + + if (!virDomainDeviceInfoCheckABIStability(src->info, dst->info)) + return false; + + return true; +} + + static bool virDomainSmartcardDefCheckABIStability(virDomainSmartcardDef *src, virDomainSmartcardDef *dst) @@ -22001,6 +22112,18 @@ virDomainDefCheckABIStabilityFlags(virDomainDef *s= rc, goto error; } =20 + if (src->nnestedsmmus !=3D dst->nnestedsmmus) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, + _("Target domain nested SMMUs count %1$zu does not = match source %2$zu"), + dst->nnestedsmmus, src->nnestedsmmus); + goto error; + } + + for (i =3D 0; i < src->nnestedsmmus; i++) + if (!virDomainNestedSmmuv3DefCheckABIStability(src->nestedsmmus[i], + dst->nestedsmmus[i])) + goto error; + if ((!src->redirfilter && dst->redirfilter) || (src->redirfilter && !dst->redirfilter)) { virReportError(VIR_ERR_CONFIG_UNSUPPORTED, @@ -22171,6 +22294,7 @@ virDomainDefCheckABIStabilityFlags(virDomainDef *sr= c, case VIR_DOMAIN_DEVICE_AUDIO: case VIR_DOMAIN_DEVICE_CRYPTO: case VIR_DOMAIN_DEVICE_PSTORE: + case VIR_DOMAIN_DEVICE_NESTED_SMMUV3: break; } #endif @@ -22366,6 +22490,36 @@ virDomainDefAddImplicitVideo(virDomainDef *def, vi= rDomainXMLOption *xmlopt) return 0; } =20 +static int +virDomainDefAddImplicitNestedSmmuv3(virDomainDef *def) +{ + // Get the number of host-level SMMUv3 instances + g_autoptr(DIR) dir =3D NULL; + struct dirent *dent; + int num =3D 0; + + virDomainNestedSmmuv3Def* nestedsmmuv3 =3D NULL; + + if (virDirOpen(&dir, "/sys/class/iommu") < 0) + return -1; + + while (virDirRead(dir, &dent, "/sys/class/iommu") > 0) { + if (!(nestedsmmuv3 =3D virDomainNestedSmmuv3DefNew())) + return -1; + if (STRPREFIX(dent->d_name, "smmu3.0x")) { + VIR_REALLOC_N(nestedsmmuv3->name, strlen(dent->d_name) + 1); + virStrcpy(nestedsmmuv3->name, dent->d_name, strlen(dent->d_nam= e) + 1); + VIR_REALLOC_N(def->nestedsmmus, def->nnestedsmmus + 1); + def->nestedsmmus[def->nnestedsmmus++] =3D nestedsmmuv3; + num++; + } + } + if (num =3D=3D 0) + return -1; + + return 0; +} + int virDomainDefAddImplicitDevices(virDomainDef *def, virDomainXMLOption *xmlo= pt) { @@ -22379,6 +22533,13 @@ virDomainDefAddImplicitDevices(virDomainDef *def, = virDomainXMLOption *xmlopt) if (virDomainDefAddImplicitVideo(def, xmlopt) < 0) return -1; =20 + if (def->iommu !=3D NULL && + def->iommu->model =3D=3D VIR_DOMAIN_IOMMU_MODEL_NESTED_SMMUV3 && + def->nnestedsmmus < 1) { + if (virDomainDefAddImplicitNestedSmmuv3(def) < 0) + return -1; + } + return 0; } =20 @@ -26814,6 +26975,24 @@ virDomainHostdevDefFormat(virBuffer *buf, return 0; } =20 +static int +virDomainNestedSmmuv3DefFormat(virBuffer *buf, + virDomainNestedSmmuv3Def *def, + unsigned int flags) +{ + virBufferAddLit(buf, "\n"); + virBufferAdjustIndent(buf, 2); + + virBufferAsprintf(buf, "%s\n", def->name); + + virDomainDeviceInfoFormat(buf, def->info, flags); + + virBufferAdjustIndent(buf, -2); + virBufferAddLit(buf, "\n"); + + return 0; +} + static int virDomainRedirdevDefFormat(virBuffer *buf, virDomainRedirdevDef *def, @@ -28672,6 +28851,12 @@ virDomainDefFormatInternalSetRootName(virDomainDef= *def, for (n =3D 0; n < def->ncryptos; n++) { virDomainCryptoDefFormat(buf, def->cryptos[n], flags); } + + for (n =3D 0; n < def->nnestedsmmus; n++) { + if (virDomainNestedSmmuv3DefFormat(buf, def->nestedsmmus[n], flags= ) < 0) + return -1; + } + if (def->iommu) virDomainIOMMUDefFormat(buf, def->iommu); =20 @@ -28841,6 +29026,7 @@ virDomainDeviceIsUSB(virDomainDeviceDef *dev, case VIR_DOMAIN_DEVICE_AUDIO: case VIR_DOMAIN_DEVICE_CRYPTO: case VIR_DOMAIN_DEVICE_PSTORE: + case VIR_DOMAIN_DEVICE_NESTED_SMMUV3: break; } =20 diff --git a/src/conf/domain_conf.h b/src/conf/domain_conf.h index f8ab1b7d2f..edde9b63d6 100644 --- a/src/conf/domain_conf.h +++ b/src/conf/domain_conf.h @@ -88,6 +88,7 @@ typedef enum { VIR_DOMAIN_DEVICE_AUDIO, VIR_DOMAIN_DEVICE_CRYPTO, VIR_DOMAIN_DEVICE_PSTORE, + VIR_DOMAIN_DEVICE_NESTED_SMMUV3, =20 VIR_DOMAIN_DEVICE_LAST } virDomainDeviceType; @@ -122,6 +123,7 @@ struct _virDomainDeviceDef { virDomainAudioDef *audio; virDomainCryptoDef *crypto; virDomainPstoreDef *pstore; + virDomainNestedSmmuv3Def *nestedsmmuv3; } data; }; =20 @@ -353,6 +355,11 @@ typedef enum { VIR_DOMAIN_STARTUP_POLICY_LAST } virDomainStartupPolicy; =20 +struct _virDomainNestedSmmuv3Def { + char *name; + virDomainDeviceInfo *info; /* Guest address */ +}; + /* basic device for direct passthrough */ struct _virDomainHostdevDef { /* If 'parentnet' is non-NULL it means this host dev was @@ -3197,6 +3204,9 @@ struct _virDomainDef { size_t ntpms; virDomainTPMDef **tpms; =20 + size_t nnestedsmmus; + virDomainNestedSmmuv3Def **nestedsmmus; + /* Only 1 */ virDomainMemballoonDef *memballoon; virDomainNVRAMDef *nvram; @@ -3677,6 +3687,8 @@ virDomainVideoDef *virDomainVideoDefNew(virDomainXMLO= ption *xmlopt); void virDomainVideoDefFree(virDomainVideoDef *def); G_DEFINE_AUTOPTR_CLEANUP_FUNC(virDomainVideoDef, virDomainVideoDefFree); void virDomainVideoDefClear(virDomainVideoDef *def); +virDomainNestedSmmuv3Def *virDomainNestedSmmuv3DefNew(void); +void virDomainNestedSmmuv3DefFree(virDomainNestedSmmuv3Def *def); virDomainHostdevDef *virDomainHostdevDefNew(void); void virDomainHostdevDefFree(virDomainHostdevDef *def); void virDomainHubDefFree(virDomainHubDef *def); diff --git a/src/conf/domain_postparse.c b/src/conf/domain_postparse.c index bf33f29638..b838a9688e 100644 --- a/src/conf/domain_postparse.c +++ b/src/conf/domain_postparse.c @@ -757,6 +757,7 @@ virDomainDeviceDefPostParseCommon(virDomainDeviceDef *d= ev, case VIR_DOMAIN_DEVICE_IOMMU: case VIR_DOMAIN_DEVICE_AUDIO: case VIR_DOMAIN_DEVICE_CRYPTO: + case VIR_DOMAIN_DEVICE_NESTED_SMMUV3: case VIR_DOMAIN_DEVICE_PSTORE: ret =3D 0; break; diff --git a/src/conf/domain_validate.c b/src/conf/domain_validate.c index c8b7b701bf..20d16c8b84 100644 --- a/src/conf/domain_validate.c +++ b/src/conf/domain_validate.c @@ -2310,6 +2310,25 @@ virDomainHostdevDefValidate(const virDomainHostdevDe= f *hostdev) } =20 =20 +static int +virDomainNestedSmmuv3DefValidate(const virDomainNestedSmmuv3Def *nestedsmm= uv3) +{ + if (nestedsmmuv3->info->type !=3D VIR_DOMAIN_DEVICE_ADDRESS_TYPE_NONE = && + nestedsmmuv3->info->type !=3D VIR_DOMAIN_DEVICE_ADDRESS_TYPE_PCI) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("nestedsmmuv3 must use 'pci' address type")); + return -1; + } + + if (nestedsmmuv3->name[0] =3D=3D '\0') { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("nestedsmmuv3 must have an associated sysfs node = name")); + return -1; + } + return 0; +} + + /** * virDomainMemoryGetMappedSize: * @mem: memory device definition @@ -3230,6 +3249,9 @@ virDomainDeviceDefValidateInternal(const virDomainDev= iceDef *dev, case VIR_DOMAIN_DEVICE_PSTORE: return virDomainPstoreDefValidate(dev->data.pstore); =20 + case VIR_DOMAIN_DEVICE_NESTED_SMMUV3: + return virDomainNestedSmmuv3DefValidate(dev->data.nestedsmmuv3); + case VIR_DOMAIN_DEVICE_LEASE: case VIR_DOMAIN_DEVICE_WATCHDOG: case VIR_DOMAIN_DEVICE_HUB: diff --git a/src/conf/schemas/domaincommon.rng b/src/conf/schemas/domaincom= mon.rng index de6a1e5c7e..e39559952e 100644 --- a/src/conf/schemas/domaincommon.rng +++ b/src/conf/schemas/domaincommon.rng @@ -6359,6 +6359,21 @@ =20 + + + + + + + + + + + + + + + @@ -6808,6 +6823,7 @@ + diff --git a/src/conf/virconftypes.h b/src/conf/virconftypes.h index 59be61cea4..b36020d9a5 100644 --- a/src/conf/virconftypes.h +++ b/src/conf/virconftypes.h @@ -118,6 +118,8 @@ typedef struct _virDomainHostdevCaps virDomainHostdevCa= ps; =20 typedef struct _virDomainHostdevDef virDomainHostdevDef; =20 +typedef struct _virDomainNestedSmmuv3Def virDomainNestedSmmuv3Def; + typedef struct _virDomainHostdevSubsys virDomainHostdevSubsys; =20 typedef struct _virDomainHostdevSubsysMediatedDev virDomainHostdevSubsysMe= diatedDev; diff --git a/src/lxc/lxc_driver.c b/src/lxc/lxc_driver.c index 2488940feb..5a7fb9b62d 100644 --- a/src/lxc/lxc_driver.c +++ b/src/lxc/lxc_driver.c @@ -3021,6 +3021,7 @@ lxcDomainAttachDeviceConfig(virDomainDef *vmdef, case VIR_DOMAIN_DEVICE_AUDIO: case VIR_DOMAIN_DEVICE_CRYPTO: case VIR_DOMAIN_DEVICE_PSTORE: + case VIR_DOMAIN_DEVICE_NESTED_SMMUV3: virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", _("persistent attach of device is not supported")); break; @@ -3087,6 +3088,7 @@ lxcDomainUpdateDeviceConfig(virDomainDef *vmdef, case VIR_DOMAIN_DEVICE_AUDIO: case VIR_DOMAIN_DEVICE_CRYPTO: case VIR_DOMAIN_DEVICE_PSTORE: + case VIR_DOMAIN_DEVICE_NESTED_SMMUV3: virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", _("persistent update of device is not supported")); break; @@ -3169,6 +3171,7 @@ lxcDomainDetachDeviceConfig(virDomainDef *vmdef, case VIR_DOMAIN_DEVICE_CRYPTO: case VIR_DOMAIN_DEVICE_AUDIO: case VIR_DOMAIN_DEVICE_PSTORE: + case VIR_DOMAIN_DEVICE_NESTED_SMMUV3: virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", _("persistent detach of device is not supported")); break; @@ -3271,6 +3274,7 @@ lxcDomainAttachDeviceMknodHelper(pid_t pid G_GNUC_UNU= SED, case VIR_DOMAIN_DEVICE_AUDIO: case VIR_DOMAIN_DEVICE_CRYPTO: case VIR_DOMAIN_DEVICE_PSTORE: + case VIR_DOMAIN_DEVICE_NESTED_SMMUV3: virReportError(VIR_ERR_INTERNAL_ERROR, _("Unexpected device type %1$d"), data->def->type); @@ -3947,6 +3951,7 @@ lxcDomainAttachDeviceLive(virLXCDriver *driver, case VIR_DOMAIN_DEVICE_AUDIO: case VIR_DOMAIN_DEVICE_CRYPTO: case VIR_DOMAIN_DEVICE_PSTORE: + case VIR_DOMAIN_DEVICE_NESTED_SMMUV3: virReportError(VIR_ERR_CONFIG_UNSUPPORTED, _("device type '%1$s' cannot be attached"), virDomainDeviceTypeToString(dev->type)); @@ -4365,6 +4370,7 @@ lxcDomainDetachDeviceLive(virLXCDriver *driver, case VIR_DOMAIN_DEVICE_AUDIO: case VIR_DOMAIN_DEVICE_CRYPTO: case VIR_DOMAIN_DEVICE_PSTORE: + case VIR_DOMAIN_DEVICE_NESTED_SMMUV3: virReportError(VIR_ERR_CONFIG_UNSUPPORTED, _("device type '%1$s' cannot be detached"), virDomainDeviceTypeToString(dev->type)); diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c index 6629addace..3186e60a0e 100644 --- a/src/qemu/qemu_command.c +++ b/src/qemu/qemu_command.c @@ -975,6 +975,7 @@ qemuBuildVirtioDevGetConfigDev(const virDomainDeviceDef= *device, case VIR_DOMAIN_DEVICE_IOMMU: case VIR_DOMAIN_DEVICE_AUDIO: case VIR_DOMAIN_DEVICE_PSTORE: + case VIR_DOMAIN_DEVICE_NESTED_SMMUV3: case VIR_DOMAIN_DEVICE_LAST: default: break; @@ -4564,6 +4565,33 @@ qemuBuildVideoCommandLine(virCommand *cmd, } =20 =20 +virJSONValue * +qemuBuildPCINestedSmmuv3DevProps(const virDomainDef *def, + virDomainNestedSmmuv3Def *nestedsmmuv3) +{ + g_autoptr(virJSONValue) props =3D NULL; + g_autofree char *pciaddr =3D NULL; + g_autofree char *bus =3D qemuBuildDeviceAddressPCIGetBus(def, nestedsm= muv3->info); + + if (!bus) + return NULL; + + if (nestedsmmuv3->info->addr.pci.function !=3D 0) + pciaddr =3D g_strdup_printf("0x%x.0x%x", nestedsmmuv3->info->addr.= pci.slot, + nestedsmmuv3->info->addr.pci.function); + else + pciaddr =3D g_strdup_printf("0x%x", nestedsmmuv3->info->addr.pci.s= lot); + + if (virJSONValueObjectAdd(&props, + "s:driver", "arm-smmuv3-nested", + "s:bus", bus, + NULL) < 0) + return NULL; + + return g_steal_pointer(&props); +} + + virJSONValue * qemuBuildPCIHostdevDevProps(const virDomainDef *def, virDomainHostdevDef *dev) @@ -4988,6 +5016,33 @@ qemuBuildHostdevSCSICommandLine(virCommand *cmd, } =20 =20 +static int +qemuBuildNestedSmmuv3CommandLine(virCommand *cmd, + const virDomainDef *def, + virQEMUCaps *qemuCaps) +{ + size_t i; + + for (i =3D 0; i < def->nnestedsmmus; i++) { + g_autoptr(virJSONValue) devprops =3D NULL; + virDomainNestedSmmuv3Def *nestedsmmuv3 =3D def->nestedsmmus[i]; + + if (nestedsmmuv3->info->type =3D=3D VIR_DOMAIN_DEVICE_ADDRESS_TYPE= _UNASSIGNED) + continue; + + if (qemuCommandAddExtDevice(cmd, nestedsmmuv3->info, def, qemuCaps= ) < 0) + return -1; + + if (!(devprops =3D qemuBuildPCINestedSmmuv3DevProps(def, nestedsmm= uv3))) + return -1; + + if (qemuBuildDeviceCommandlineFromJSON(cmd, devprops, def, qemuCap= s) < 0) + return -1; + } + return 0; +} + + static int qemuBuildHostdevCommandLine(virCommand *cmd, const virDomainDef *def, @@ -10558,6 +10613,9 @@ qemuBuildCommandLine(virDomainObj *vm, if (qemuBuildHostdevCommandLine(cmd, def, qemuCaps) < 0) return NULL; =20 + if (qemuBuildNestedSmmuv3CommandLine(cmd, def, qemuCaps) < 0) + return NULL; + if (migrateURI) virCommandAddArgList(cmd, "-incoming", migrateURI, NULL); =20 diff --git a/src/qemu/qemu_command.h b/src/qemu/qemu_command.h index 76c514b5f7..f09dcc514c 100644 --- a/src/qemu/qemu_command.h +++ b/src/qemu/qemu_command.h @@ -157,6 +157,10 @@ qemuBuildThreadContextProps(virJSONValue **tcProps, qemuDomainObjPrivate *priv, virBitmap *nodemask); =20 +virJSONValue * +qemuBuildPCINestedSmmuv3DevProps(const virDomainDef *def, + virDomainNestedSmmuv3Def *nestedsmmuv3); + /* Current, best practice */ virJSONValue * qemuBuildPCIHostdevDevProps(const virDomainDef *def, diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c index 3366346624..2567d620d9 100644 --- a/src/qemu/qemu_domain.c +++ b/src/qemu/qemu_domain.c @@ -8718,6 +8718,7 @@ qemuDomainPrepareChardevSourceOne(virDomainDeviceDef = *dev, case VIR_DOMAIN_DEVICE_VSOCK: case VIR_DOMAIN_DEVICE_AUDIO: case VIR_DOMAIN_DEVICE_CRYPTO: + case VIR_DOMAIN_DEVICE_NESTED_SMMUV3: case VIR_DOMAIN_DEVICE_PSTORE: break; } @@ -10670,6 +10671,7 @@ qemuDomainDeviceBackendChardevForeachOne(virDomainD= eviceDef *dev, case VIR_DOMAIN_DEVICE_VSOCK: case VIR_DOMAIN_DEVICE_AUDIO: case VIR_DOMAIN_DEVICE_CRYPTO: + case VIR_DOMAIN_DEVICE_NESTED_SMMUV3: case VIR_DOMAIN_DEVICE_PSTORE: /* no chardev backend */ break; diff --git a/src/qemu/qemu_domain_address.c b/src/qemu/qemu_domain_address.c index 51f4bbd6eb..31004bfc7e 100644 --- a/src/qemu/qemu_domain_address.c +++ b/src/qemu/qemu_domain_address.c @@ -470,6 +470,7 @@ qemuDomainDeviceSupportZPCI(virDomainDeviceDef *device) case VIR_DOMAIN_DEVICE_VSOCK: case VIR_DOMAIN_DEVICE_AUDIO: case VIR_DOMAIN_DEVICE_CRYPTO: + case VIR_DOMAIN_DEVICE_NESTED_SMMUV3: case VIR_DOMAIN_DEVICE_PSTORE: break; =20 @@ -819,6 +820,10 @@ qemuDomainDeviceCalculatePCIConnectFlags(virDomainDevi= ceDef *dev, return pciFlags; } =20 + case VIR_DOMAIN_DEVICE_NESTED_SMMUV3: { + return VIR_PCI_CONNECT_TYPE_NESTED_SMMUV3; + } + case VIR_DOMAIN_DEVICE_MEMBALLOON: switch (dev->data.memballoon->model) { case VIR_DOMAIN_MEMBALLOON_MODEL_VIRTIO_TRANSITIONAL: diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 09f7edda7d..d9ade7000b 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -6880,6 +6880,7 @@ qemuDomainAttachDeviceConfig(virDomainDef *vmdef, case VIR_DOMAIN_DEVICE_AUDIO: case VIR_DOMAIN_DEVICE_CRYPTO: case VIR_DOMAIN_DEVICE_PSTORE: + case VIR_DOMAIN_DEVICE_NESTED_SMMUV3: case VIR_DOMAIN_DEVICE_LAST: virReportError(VIR_ERR_OPERATION_UNSUPPORTED, _("persistent attach of device '%1$s' is not suppo= rted"), @@ -7099,6 +7100,7 @@ qemuDomainDetachDeviceConfig(virDomainDef *vmdef, case VIR_DOMAIN_DEVICE_AUDIO: case VIR_DOMAIN_DEVICE_CRYPTO: case VIR_DOMAIN_DEVICE_PSTORE: + case VIR_DOMAIN_DEVICE_NESTED_SMMUV3: case VIR_DOMAIN_DEVICE_LAST: virReportError(VIR_ERR_OPERATION_UNSUPPORTED, _("persistent detach of device '%1$s' is not suppor= ted"), @@ -7225,6 +7227,7 @@ qemuDomainUpdateDeviceConfig(virDomainDef *vmdef, case VIR_DOMAIN_DEVICE_AUDIO: case VIR_DOMAIN_DEVICE_CRYPTO: case VIR_DOMAIN_DEVICE_PSTORE: + case VIR_DOMAIN_DEVICE_NESTED_SMMUV3: case VIR_DOMAIN_DEVICE_LAST: virReportError(VIR_ERR_OPERATION_UNSUPPORTED, _("persistent update of device '%1$s' is not suppor= ted"), diff --git a/src/qemu/qemu_hotplug.c b/src/qemu/qemu_hotplug.c index 3c18af6b0c..0a67df8605 100644 --- a/src/qemu/qemu_hotplug.c +++ b/src/qemu/qemu_hotplug.c @@ -3521,6 +3521,7 @@ qemuDomainAttachDeviceLive(virDomainObj *vm, case VIR_DOMAIN_DEVICE_AUDIO: case VIR_DOMAIN_DEVICE_CRYPTO: case VIR_DOMAIN_DEVICE_PSTORE: + case VIR_DOMAIN_DEVICE_NESTED_SMMUV3: case VIR_DOMAIN_DEVICE_LAST: virReportError(VIR_ERR_OPERATION_UNSUPPORTED, _("live attach of device '%1$s' is not supported"), @@ -5420,6 +5421,7 @@ qemuDomainRemoveAuditDevice(virDomainObj *vm, case VIR_DOMAIN_DEVICE_AUDIO: case VIR_DOMAIN_DEVICE_CRYPTO: case VIR_DOMAIN_DEVICE_PSTORE: + case VIR_DOMAIN_DEVICE_NESTED_SMMUV3: case VIR_DOMAIN_DEVICE_LAST: /* libvirt doesn't yet support detaching these devices */ break; @@ -5525,6 +5527,7 @@ qemuDomainRemoveDevice(virQEMUDriver *driver, case VIR_DOMAIN_DEVICE_AUDIO: case VIR_DOMAIN_DEVICE_CRYPTO: case VIR_DOMAIN_DEVICE_PSTORE: + case VIR_DOMAIN_DEVICE_NESTED_SMMUV3: case VIR_DOMAIN_DEVICE_LAST: virReportError(VIR_ERR_OPERATION_UNSUPPORTED, _("don't know how to remove a %1$s device"), @@ -6420,6 +6423,7 @@ qemuDomainDetachDeviceLive(virDomainObj *vm, case VIR_DOMAIN_DEVICE_AUDIO: case VIR_DOMAIN_DEVICE_CRYPTO: case VIR_DOMAIN_DEVICE_PSTORE: + case VIR_DOMAIN_DEVICE_NESTED_SMMUV3: case VIR_DOMAIN_DEVICE_LAST: virReportError(VIR_ERR_OPERATION_UNSUPPORTED, _("live detach of device '%1$s' is not supported"), @@ -7412,6 +7416,7 @@ qemuDomainUpdateDeviceLive(virDomainObj *vm, case VIR_DOMAIN_DEVICE_AUDIO: case VIR_DOMAIN_DEVICE_CRYPTO: case VIR_DOMAIN_DEVICE_PSTORE: + case VIR_DOMAIN_DEVICE_NESTED_SMMUV3: case VIR_DOMAIN_DEVICE_LAST: virReportError(VIR_ERR_CONFIG_UNSUPPORTED, _("live update of device '%1$s' is not supported"), diff --git a/src/qemu/qemu_postparse.c b/src/qemu/qemu_postparse.c index 03b5ef825a..4e8892b838 100644 --- a/src/qemu/qemu_postparse.c +++ b/src/qemu/qemu_postparse.c @@ -897,6 +897,7 @@ qemuDomainDeviceDefPostParse(virDomainDeviceDef *dev, case VIR_DOMAIN_DEVICE_NVRAM: case VIR_DOMAIN_DEVICE_RNG: case VIR_DOMAIN_DEVICE_AUDIO: + case VIR_DOMAIN_DEVICE_NESTED_SMMUV3: case VIR_DOMAIN_DEVICE_CRYPTO: ret =3D 0; break; diff --git a/src/qemu/qemu_validate.c b/src/qemu/qemu_validate.c index 9f07ffe4a3..d84f04903f 100644 --- a/src/qemu/qemu_validate.c +++ b/src/qemu/qemu_validate.c @@ -5470,6 +5470,7 @@ qemuValidateDomainDeviceDef(const virDomainDeviceDef = *dev, case VIR_DOMAIN_DEVICE_PSTORE: return qemuValidateDomainDeviceDefPstore(dev->data.pstore, def, qe= muCaps); =20 + case VIR_DOMAIN_DEVICE_NESTED_SMMUV3: case VIR_DOMAIN_DEVICE_LEASE: case VIR_DOMAIN_DEVICE_PANIC: case VIR_DOMAIN_DEVICE_NONE: diff --git a/src/test/test_driver.c b/src/test/test_driver.c index f1cefb5c50..c63e76baa5 100644 --- a/src/test/test_driver.c +++ b/src/test/test_driver.c @@ -10451,6 +10451,7 @@ testDomainAttachDeviceLive(virDomainObj *vm, case VIR_DOMAIN_DEVICE_AUDIO: case VIR_DOMAIN_DEVICE_CRYPTO: case VIR_DOMAIN_DEVICE_PSTORE: + case VIR_DOMAIN_DEVICE_NESTED_SMMUV3: case VIR_DOMAIN_DEVICE_LAST: virReportError(VIR_ERR_OPERATION_UNSUPPORTED, _("live attach of device '%1$s' is not supported"), @@ -10594,6 +10595,7 @@ testDomainUpdateDevice(virDomainDef *vmdef, case VIR_DOMAIN_DEVICE_AUDIO: case VIR_DOMAIN_DEVICE_CRYPTO: case VIR_DOMAIN_DEVICE_PSTORE: + case VIR_DOMAIN_DEVICE_NESTED_SMMUV3: case VIR_DOMAIN_DEVICE_LAST: virReportError(VIR_ERR_OPERATION_UNSUPPORTED, _("persistent update of device '%1$s' is not suppor= ted"), @@ -10965,6 +10967,7 @@ testDomainRemoveDevice(testDriver *driver, case VIR_DOMAIN_DEVICE_AUDIO: case VIR_DOMAIN_DEVICE_CRYPTO: case VIR_DOMAIN_DEVICE_PSTORE: + case VIR_DOMAIN_DEVICE_NESTED_SMMUV3: case VIR_DOMAIN_DEVICE_LAST: virReportError(VIR_ERR_OPERATION_UNSUPPORTED, _("live detach of device '%1$s' is not supported"), @@ -11036,6 +11039,7 @@ testDomainDetachDeviceLive(testDriver *driver, case VIR_DOMAIN_DEVICE_AUDIO: case VIR_DOMAIN_DEVICE_CRYPTO: case VIR_DOMAIN_DEVICE_PSTORE: + case VIR_DOMAIN_DEVICE_NESTED_SMMUV3: case VIR_DOMAIN_DEVICE_LAST: virReportError(VIR_ERR_OPERATION_UNSUPPORTED, _("live detach of device '%1$s' is not supported"), diff --git a/tests/schemas/device.rng.in b/tests/schemas/device.rng.in index b322b5275e..fc59700574 100644 --- a/tests/schemas/device.rng.in +++ b/tests/schemas/device.rng.in @@ -45,6 +45,7 @@ + =20 --=20 2.34.1 From nobody Sun Dec 22 03:14:30 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.libvirt.org designates 8.43.85.245 as permitted sender) client-ip=8.43.85.245; envelope-from=devel-bounces@lists.libvirt.org; helo=lists.libvirt.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of lists.libvirt.org designates 8.43.85.245 as permitted sender) smtp.mailfrom=devel-bounces@lists.libvirt.org; arc=fail (BodyHash is different from the expected one); dmarc=pass(p=reject dis=none) header.from=lists.libvirt.org Return-Path: Received: from lists.libvirt.org (lists.libvirt.org [8.43.85.245]) by mx.zohomail.com with SMTPS id 1733963280086229.56661111045423; Wed, 11 Dec 2024 16:28:00 -0800 (PST) Received: by lists.libvirt.org (Postfix, from userid 996) id E4D5311B0; Wed, 11 Dec 2024 19:27:58 -0500 (EST) Received: from lists.libvirt.org (localhost [IPv6:::1]) by lists.libvirt.org (Postfix) with ESMTP id 469291225; Wed, 11 Dec 2024 19:25:20 -0500 (EST) Received: by lists.libvirt.org (Postfix, from userid 996) id 235E611E4; Wed, 11 Dec 2024 19:25:14 -0500 (EST) Received: from NAM02-BN1-obe.outbound.protection.outlook.com (mail-bn1nam02on2085.outbound.protection.outlook.com [40.107.212.85]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.libvirt.org (Postfix) with ESMTPS id 772C011C2 for ; Wed, 11 Dec 2024 19:24:56 -0500 (EST) Received: from SN7PR12MB6838.namprd12.prod.outlook.com (2603:10b6:806:266::18) by CY5PR12MB6322.namprd12.prod.outlook.com (2603:10b6:930:21::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8251.14; Thu, 12 Dec 2024 00:24:49 +0000 Received: from SN7PR12MB6838.namprd12.prod.outlook.com ([fe80::529d:478:bc5d:b400]) by SN7PR12MB6838.namprd12.prod.outlook.com ([fe80::529d:478:bc5d:b400%6]) with mapi id 15.20.8251.008; Thu, 12 Dec 2024 00:24:49 +0000 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on lists.libvirt.org X-Spam-Level: X-Spam-Status: No, score=0.2 required=5.0 tests=DKIM_INVALID,DKIM_SIGNED, FORGED_SPF_HELO,MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H2, RCVD_IN_VALIDITY_RPBL_BLOCKED,RCVD_IN_VALIDITY_SAFE_BLOCKED, SPF_HELO_PASS autolearn=no autolearn_force=no version=3.4.4 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=A2Mab4CAvmPwtCDg9n4iOjiPt881av6hRyzN79FRFGgLRHYe29wUxRQ/DyVcM6REm6QUy53XNIZY3xiqa4rx6dL6Nlx34V9vD6UESsHC3txDDXGuNhgcAZsNIPBbL5v+8EljSiF94ZjCzt/+xqzona1RQbLunHkes4BgQUAft5TSbkTSszGZXPtY/52bztZ9aTarVX+9iv4ipAwi4WFc5wlIbG7QahZXj7qtJhiHnuUR4pMGV0E4KNnBl9yaYaQTG7TthRnyYPxv4SlVCbCTsZ5lJD8WNUAzqCRWCwMR5oPpIOsK+ZmSEFLFXiG34oK1LMFLxMORjBJCZEyh2mYxyg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=9PWA0WMEpt+mmdqddHfXlhRVqpXdQTjq0fZ7kER9KNI=; b=p0iMXbiJSFcQxqvZR0GMQSuOIRyCRxPkOQHT93ZVqJN7/yJp/SDw+SvwFjcBLeQ+rC3I5dXCI2YHYAzB3WZLtSrGrnqCkhQwTFiGRo6rG/gZhFzw0+Pw/N5AUMPu0LOc71ynSZ4qw9GgeLasS6+QtQopmtCrgAwaodM7rBoqZ88CjhUlPduX5q/l+vkn0qfgNXVgStGuOp41QhipiCsS21Sz9DuYfZ8zEmBK9I3nKyTVOL4FEljiTaHrqMjwRTuZxLK9luNjZCuvqKIy34tyERsFj2VfS/Iy/tQikJ9CTk0nwqImCerx427fKsgwKIT2nY8Zp2+LjcyETSsqpMdiIA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=9PWA0WMEpt+mmdqddHfXlhRVqpXdQTjq0fZ7kER9KNI=; b=L8lvkiFn/TkWBbQPoGw9MbnDSgZe3QvhCSYBVkYVgfrCV7OUdq6qQvwhxcjKBBeSDy1UqRtqtwK3kbnQ01pusOVBDLD3TJ3DtInSydupDCw3KIywoxZVu73hupVyP46Hc7jVjiJBeL318lExPJJaAFUGh2h7GuB0j1pTEc5mrPSEjE4gDMtoQHVhTeIa5HHK8uWYTyH/xBn1SBHJwAPBVD2/Nv2faXKU2oHQC5SX2vQMgZmBpqzvVjuHsSW1aNsxsrqDhJIG+mgPhK8K2tvYaT7X06bojgFXUL10b+l9X6C6CIec/vBGLjDR4p7DLTu4uNoJzqI/9n4ipf1IdONQ3w== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; To: devel@lists.libvirt.org Subject: [RFC PATCH 3/5] qemu: Create PXBs and auto-assign VFIO devs and nested SMMUs Date: Wed, 11 Dec 2024 16:24:21 -0800 Message-Id: <20241212002423.1050892-4-nathanc@nvidia.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241212002423.1050892-1-nathanc@nvidia.com> References: <20241212002423.1050892-1-nathanc@nvidia.com> X-ClientProxiedBy: BYAPR05CA0028.namprd05.prod.outlook.com (2603:10b6:a03:c0::41) To SN7PR12MB6838.namprd12.prod.outlook.com (2603:10b6:806:266::18) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SN7PR12MB6838:EE_|CY5PR12MB6322:EE_ X-MS-Office365-Filtering-Correlation-Id: 6b14225a-d56c-43be-4a0c-08dd1a4363d9 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|376014|366016; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?tZpa+yx+iEHB029YVpXnnlcdBRBUnDmRCXCsaSpq/IVk5NGAZGddVtjRjZX4?= =?us-ascii?Q?/indzZ/6bjGw6Pb+RjVT+NihKCdsmJJzOkm5GvGiEl4kgUPtHzRtIMB9SK1y?= =?us-ascii?Q?HKppMRA4iRLSBspXL2jOxyh/4LyUqLCGc2UET4kUS98n+UV+s5nv5HfjVGfB?= =?us-ascii?Q?bACIaIV1Yl710DWQALzmJDY8rTRdrRpdd7gJOnx8ow1SOYv2gYK/EqQ3SUqQ?= =?us-ascii?Q?gILaktqEnOrcgqkxTtPPqrYEESIX99uiL1hUal1sKaKwg/hobDTFS9kGv7Lm?= =?us-ascii?Q?fiPlG4tvJrdpBPEAZiLV/ZEL8aOiANiWYYLtcaBlDQK9zLlRflfmtLMtlc+j?= =?us-ascii?Q?R/j+jvjSqMmPpX+/V4g4IkNYGu81V3yxKdnkJKWle08nhE3Ds7JOThnSU7wi?= =?us-ascii?Q?S1FEvk5eWzma4BAA+5kRn3bmjsEzQN7PA6e6jb7ve11F8a81iTo3JXaDIx1B?= =?us-ascii?Q?dMAJ7ZeXNeXi3aijDRaH9a6tWm1Jxv1j782SKnREn8g/1F8E+Yjb4y81YGjx?= =?us-ascii?Q?muDNtilONeZquNRViXauwOWqYir1zVBrUB6ChdPEXF6VMu9kv/HQ9gxrO52P?= =?us-ascii?Q?kRzib9UmBpMx7yAqSiPNV5Q1RhbIHibA5XoIXHTx3VJ6GUBFjHgUM5dD65I7?= =?us-ascii?Q?L/eAHUW6ERgUsRHglGGnrFw9A2FsXu3C5tUMNJTe+bOJ5cwSbuQPNWKQJ6m8?= =?us-ascii?Q?krjFC2QmBs9iLyh1UpF58zGAGSoRaGUB5q3y21F1FKdmHqHCjFfj1lZruc+Z?= =?us-ascii?Q?5kBNcUYFjdowsDU8OQligNptXkOUVI6Mq16s0FOBcdUgNec8qMAAgI7MO8/L?= =?us-ascii?Q?iLWNhjwOd5UuwHL7XcJedgj1ymueL4L83uiQ2nEJ9eRCRvXvIGcIzbyrombV?= =?us-ascii?Q?BGlMtBJGQVYnmlfFgqgWyCMoAMoCpG+7o3fxw6X8k82dL95gSPsjCIF0dHMK?= =?us-ascii?Q?r8suNVVrh7Vk1lbxQT+gbpkpVYXqXYsEzcPG+/C4NMfaTOMJDAyK3TU6es6S?= =?us-ascii?Q?fRShdiQX6TLD75HYLJPEqFfHTrxyhH9HC9ElqKaYnz68Rbq5mRgq2jMXVWAp?= =?us-ascii?Q?zFelgyc0biGxm3ZSEnzKKJv8FRURk6B3r9cug7eRjPikvSgYoT8gjVjySnno?= =?us-ascii?Q?vUXyGMAcK89fAbp0DRXR46W7Ol8dPeVPjW46uuVIkqFtlgOSdc3IliqX4HsV?= =?us-ascii?Q?1EQ8ShXSRqYZq6ufgoym7+R8QYQ3oVKkPDO1Jzamt0XrFfTDS7r72XGGV0VS?= =?us-ascii?Q?MLEWYs4C0H4E/aDlZQiEZ7T5E+U/ytYRzpJ853k2x3xwH6ofgnq5JmOuHzHL?= =?us-ascii?Q?u5qZErh+/fwb9b1e7dP53gfQ3X9qAYaVZ0CjlDIpi15tSQ=3D=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SN7PR12MB6838.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(376014)(366016);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?bxHhiug7MfKYwDz6luKoVhyfhVdW8NLGIdGdvMw28x8AllZfhZqruJhBZj6a?= =?us-ascii?Q?YdGk0ilUvCMzi3wDIqV0fi07DnL91vonW6oFAaDIHIQKUVcksr1xJtPDiIYf?= =?us-ascii?Q?0Tn/r0fCfkoAuQUdz1PVn9IxneUw5YK5a2saUWUnfGf3Il71a/nrVsGoRRi6?= =?us-ascii?Q?oobvTffXweRgXG2ZvmRZwXLot2jr4G/hJ9t57xKWiCM46Me1e1a2Ykyyr3pY?= =?us-ascii?Q?BLdqaASI4jvEcBPATGwXuImTV47KFbuSrpxKO0JigpMv2n3ejRyyTql1xDfm?= =?us-ascii?Q?7REK00ilCrhoYET6r5eWNVIG9hd/+o6zfnkQ18rKTF5WMOQXjgkuSmm5ej0F?= =?us-ascii?Q?86Np6iQCPSF7NenMwu1hS/O/j2pKy4DwkpkPfyjTWJRhIpBX1ayTKGvjycdr?= =?us-ascii?Q?HV99tncHL6Yof3hvRb6aLj4yTAwdrjQGQ1n11AB8lzWHsXCwWdjKaCURC84u?= =?us-ascii?Q?h4YIJyVXxHquxqSyuGsgf1xDmO6NpV4RZtz3GFOVNSOA2NAe24AidRmKBVz0?= =?us-ascii?Q?EBgyWl7B/LvAe1IQcXWT0aWHV1t4tyya/v8uhH48OB6G0OhWyM6BvQcNbMxu?= =?us-ascii?Q?waS7ym1xwd2tz64nxpBhZJgCU4OG1PXu6m2U9YmjKxCmm8EgURoSG6TcIAFd?= =?us-ascii?Q?S/lhEJfLfzO/nqaA+rreI/jUV3yjg0TdCokrqjyLzHfVggauQZwDYnuK5IA2?= =?us-ascii?Q?5iVSw9c0/NqjUOngBjpC5/fjjsVTCUxxzZt1cUXvh13oBkKRvSkHD+k9o3aW?= =?us-ascii?Q?2/LfOwzK60zdybeprDdCeJe15OJxDRFDSzBWA2LnK+MQ2K4bPkOSZQFP83xa?= =?us-ascii?Q?HU4GUTTBPwB9DYeMuNZ1R6bjXdDu+paLiwYd0qrx5AfUD4eoDVFkE1MUhUkV?= =?us-ascii?Q?m1cXhsYSQJaSVLpy79vyXICBUqZKxo7d8Po8+scJSC6o2zz9S+j9AalxBT9D?= =?us-ascii?Q?Loitia/5+JACE9h6n76b6WhovN2+aCymH0/45UdG67bNHp31TwG/4QuGT/m/?= =?us-ascii?Q?xIoe6jFJEaVa8SzkIRiy0AU+u/bi0fpbjieJknP3scCI+S7OSqT8BgvdizJR?= =?us-ascii?Q?vSdIsyWDpTruKLiulhtZt4v61cD8brNYTKhH1BDr3s2engWmV7F87BBL4cNS?= =?us-ascii?Q?Jt7mW+ZJFvvurIXCbGPBbocEu041+xOnaBU8pZaW1GWE63LYbfQs8NY7/Ip4?= =?us-ascii?Q?vRBeL5scs5A4Ko+fHpgdK6QkpnXRJHM920C3r17B+MwQ+Hk2firmuN/SPe6j?= =?us-ascii?Q?0IuXhzWL8gFLCGGf5SrKSaMtRKRDNK/B44+2GqrmvcCNsb+btRgZeTI9LHh8?= =?us-ascii?Q?YGZ1Ir4iyqvRbd6zdjFxblguj82//19HJOVDnrbUOGAC3E3vfCaKZIJaJ1kS?= =?us-ascii?Q?F2ID39c4AaywZR4Ky15IqkhULtRArZvrV9LRgKd887luNlXfWOJW6Iqg8sbh?= =?us-ascii?Q?CUBWAWByMmfZG3KID/X1iMFVX2WJvrKNFSNpOc/UPhdSLZUyZ6UuoeRHmAzY?= =?us-ascii?Q?JkH9xPnlmN6/EAQhLen/IFZA5zpy0R0Y60k80oaS2yNj9laDM9dHdGM73F6O?= =?us-ascii?Q?1WIYD6Ui0VbyKkNKBQIxJINb0BVf+jyU58Z7PaMr?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 6b14225a-d56c-43be-4a0c-08dd1a4363d9 X-MS-Exchange-CrossTenant-AuthSource: SN7PR12MB6838.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Dec 2024 00:24:49.8359 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 33H8c17GUvmk+T4cqGhxO+ORQh3KPvrkFrMeSU3WYiSvBtnYFHLrG8v4CHsEtiieBuyfDG9IgqoTqhNzMLj/3w== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY5PR12MB6322 Content-Transfer-Encoding: quoted-printable Message-ID-Hash: LDEOWD77CVBQWF2QPLN6BPYMWKEBCU7X X-Message-ID-Hash: LDEOWD77CVBQWF2QPLN6BPYMWKEBCU7X X-MailFrom: nathanc@nvidia.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-config-1; header-match-config-2; header-match-config-3; header-match-devel.lists.libvirt.org-0; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; suspicious-header CC: Nicolin Chen , Shameerali Kolothum Thodi , Nathan Chen X-Mailman-Version: 3.2.2 Precedence: list List-Id: Development discussions about the libvirt library & tools Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: From: Nathan Chen via Devel Reply-To: Nathan Chen X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1733963282502116600 Content-Type: text/plain; charset="utf-8" Add a pcie-expander-bus controller to the VM definition for each "nestedSmm= uv3" device that is generated when the "nestedSmmuv3" IOMMU model is parsed from= the VM definition. Assign each "nestedSmmuv3" device to one PXB controller, and route any unmanaged "hostdev" VFIO devices with associated host SMMU nodes = to their corresponding PXB controller based on the "name" attributes of "nestedSmmuv3" devices attached to these PXB controllers. Signed-off-by: Nathan Chen --- src/conf/domain_addr.c | 26 ++++++- src/conf/domain_addr.h | 3 +- src/conf/domain_conf.c | 1 + src/qemu/qemu_domain_address.c | 134 +++++++++++++++++++++++++++++++++ 4 files changed, 162 insertions(+), 2 deletions(-) diff --git a/src/conf/domain_addr.c b/src/conf/domain_addr.c index a53ff6df6c..6d8ca89025 100644 --- a/src/conf/domain_addr.c +++ b/src/conf/domain_addr.c @@ -386,6 +386,8 @@ virDomainPCIAddressFlagsCompatible(virPCIDeviceAddress = *addr, connectStr =3D "pcie-expander-bus"; } else if (devFlags & VIR_PCI_CONNECT_TYPE_PCI_BRIDGE) { connectStr =3D "pci-bridge"; + } else if (devFlags & VIR_PCI_CONNECT_TYPE_NESTED_SMMUV3) { + connectStr =3D "nestedSmmuv3 device"; } else { /* this should never happen. If it does, there is a * bug in the code that sets the flag bits for devices. @@ -565,7 +567,8 @@ virDomainPCIAddressBusSetModel(virDomainPCIAddressBus *= bus, * dmi-to-pci-bridge */ bus->flags =3D (VIR_PCI_CONNECT_TYPE_PCIE_ROOT_PORT | - VIR_PCI_CONNECT_TYPE_DMI_TO_PCI_BRIDGE); + VIR_PCI_CONNECT_TYPE_DMI_TO_PCI_BRIDGE | + VIR_PCI_CONNECT_TYPE_NESTED_SMMUV3); bus->minSlot =3D 0; bus->maxSlot =3D VIR_PCI_ADDRESS_SLOT_LAST; break; @@ -690,6 +693,8 @@ virDomainPCIAddressSetGrow(virDomainPCIAddressSet *addr= s, } else if (flags & (VIR_PCI_CONNECT_TYPE_PCIE_DEVICE | VIR_PCI_CONNECT_TYPE_PCIE_SWITCH_UPSTREAM_PORT)) { model =3D VIR_DOMAIN_CONTROLLER_MODEL_PCIE_ROOT_PORT; + } else if (flags & VIR_PCI_CONNECT_TYPE_NESTED_SMMUV3) { + model =3D VIR_DOMAIN_CONTROLLER_MODEL_PCIE_EXPANDER_BUS; } else { /* The types of devices that we can't auto-add a controller for: * @@ -1030,6 +1035,11 @@ virDomainPCIAddressFindUnusedFunctionOnBus(virDomain= PCIAddressBus *bus, break; } =20 + if (flags =3D=3D VIR_PCI_CONNECT_TYPE_NESTED_SMMUV3) { + *found =3D false; + break; + } + if (flags & VIR_PCI_CONNECT_AGGREGATE_SLOT && bus->slot[searchAddr->slot].aggregate) { /* slot and device are okay with aggregating devices */ @@ -1087,6 +1097,20 @@ virDomainPCIAddressGetNextAddr(virDomainPCIAddressSe= t *addrs, else a.function =3D function; =20 + if (flags =3D=3D VIR_PCI_CONNECT_TYPE_NESTED_SMMUV3) { + if (addrs->dryRun) { + virDomainPCIAddressBus *bus =3D &addrs->buses[addrs->nbuses - = 1]; + /* a is already set to the first new bus */ + a.bus =3D addrs->nbuses; + a.slot =3D bus->minSlot; + if (virDomainPCIAddressSetGrow(addrs, &a, flags) < 0) + return -1; + /* this device will use the first slot of the new bus */ + a.slot =3D addrs->buses[a.bus].minSlot; + goto success; + } + } + /* When looking for a suitable bus for the device, start by being * very strict and ignoring all those where the isolation groups * don't match. This ensures all devices sharing the same isolation diff --git a/src/conf/domain_addr.h b/src/conf/domain_addr.h index 9781685903..2881f2dadb 100644 --- a/src/conf/domain_addr.h +++ b/src/conf/domain_addr.h @@ -72,7 +72,8 @@ typedef enum { VIR_PCI_CONNECT_TYPE_PCI_EXPANDER_BUS | \ VIR_PCI_CONNECT_TYPE_PCIE_EXPANDER_BUS | \ VIR_PCI_CONNECT_TYPE_PCI_BRIDGE | \ - VIR_PCI_CONNECT_TYPE_PCIE_TO_PCI_BRIDGE) + VIR_PCI_CONNECT_TYPE_PCIE_TO_PCI_BRIDGE | \ + VIR_PCI_CONNECT_TYPE_NESTED_SMMUV3) =20 /* combination of all bits that could be used to connect a normal * endpoint device (i.e. excluding the connection possible between an diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index 24aff1cfbe..46f9b9b0cf 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -25,6 +25,7 @@ #include #include #include +#include =20 #include "configmake.h" #include "internal.h" diff --git a/src/qemu/qemu_domain_address.c b/src/qemu/qemu_domain_address.c index 31004bfc7e..dee198a7d2 100644 --- a/src/qemu/qemu_domain_address.c +++ b/src/qemu/qemu_domain_address.c @@ -1627,6 +1627,19 @@ qemuDomainPCIAddressSetCreate(virDomainDef *def, =20 addrs->dryRun =3D dryRun; =20 + /* PXB indices must come before pcie-root-port indices in qemu, + * so add PXB buses to addrs before the pcie-root-ports. */ + + if (addrs->dryRun) { + for (i =3D 0; i < def->nnestedsmmus; i++) { + if (!virDeviceInfoPCIAddressIsWanted(def->nestedsmmus[i]->info= )) + continue; + if (qemuDomainPCIAddressReserveNextAddr(addrs, + def->nestedsmmus[i]->i= nfo) < 0) + return NULL; + } + } + /* pSeries domains support multiple pci-root controllers */ if (qemuDomainIsPSeries(def)) addrs->areMultipleRootsSupported =3D true; @@ -2030,6 +2043,109 @@ qemuDomainValidateDevicePCISlotsChipsets(virDomainD= ef *def, } =20 =20 +static char* +retrieveSysfsDevPath(virPCIDeviceAddress* addr, const char* path) +{ + return g_strdup_printf("/sys/bus/pci/devices/%04x:%02x:%02x.%01x%s", + addr->domain, + addr->bus, + addr->slot, + addr->function, + path ? path : ""); +} + + +static char * +nestedSmmuVfioHostdevFound(virDomainHostdevDef *hostdev, bool dryRun) +{ + char* devPath =3D NULL; + char* devSmmuPath =3D NULL; + char* devVFIOPath =3D NULL; + g_autoptr(DIR) dir =3D NULL; + g_autoptr(DIR) smmuDir =3D NULL; + g_autoptr(DIR) VFIODir =3D NULL; + char* dir_iommu =3D NULL; + char* smmu_node =3D NULL; + devPath =3D retrieveSysfsDevPath(&hostdev->source.subsys.u.pci.addr, "= "); + if (virDirOpenIfExists(&dir, devPath) < 1) + return NULL; + devSmmuPath =3D retrieveSysfsDevPath(&hostdev->source.subsys.u.pci.add= r, "/iommu"); + if (virDirOpenIfExists(&smmuDir, devSmmuPath) < 1) + return NULL; + devVFIOPath =3D retrieveSysfsDevPath(&hostdev->source.subsys.u.pci.add= r, "/vfio-dev"); + if (hostdev->source.subsys.type =3D=3D VIR_DOMAIN_HOSTDEV_SUBSYS_TYPE_= PCI && + (hostdev->info->addr.pci.bus !=3D 0 || dryRun)) { + // We only want to route vfio hostdevs + if (hostdev->managed || + (virDirOpenIfExists(&VFIODir, devVFIOPath) =3D=3D 1)) { + // Get the hostdev's associated SMMU node name + dir_iommu =3D realpath(devSmmuPath, NULL); + if (!dir_iommu) + return NULL; + smmu_node =3D g_path_get_basename(dir_iommu); + if (!smmu_node) + return NULL; + } + } + return smmu_node; +} + + +static virDomainControllerDef * +qemuDomainGetUpstreamCont(virDomainDef *def, + virDomainDeviceInfo *downstreamInfo, + int model) +{ + size_t i; + for (i =3D 0; i < def->ncontrollers; i++) { + if (def->controllers[i]->idx =3D=3D downstreamInfo->addr.pci.bus && + def->controllers[i]->model =3D=3D model) + return def->controllers[i]; + } + return NULL; +} + + +static int +qemuDomainAssignNestedSmmuv3HostdevSlots(virDomainDef *def, + virDomainPCIAddressSet *addrs) +{ + size_t i, j; + char* smmu_node =3D NULL; + virDomainControllerDef *rootPort; + virDomainPCIAddressSet *set =3D NULL; + if (def->iommu !=3D NULL && def->iommu->model =3D=3D VIR_DOMAIN_IOMMU_= MODEL_NESTED_SMMUV3 && + def->nnestedsmmus > 0) { + for (i =3D 0; i < def->nhostdevs; i++) { + if (!(smmu_node =3D nestedSmmuVfioHostdevFound(def->hostdevs[i= ], addrs->dryRun))) + continue; + /* Find a hostdev and nested SMMU pair */ + for (j =3D 0; j < def->nnestedsmmus; j++) { + unsigned int nestedSmmuBus =3D def->nestedsmmus[j]->info->= addr.pci.bus; + virDomainControllerDef *pxb; + if (!STREQLEN(def->nestedsmmus[j]->name, smmu_node, strlen= (smmu_node))) + continue; + /* Get the hostdev's pcie-root-port controller */ + rootPort =3D qemuDomainGetUpstreamCont(def, def->hostdevs[= i]->info, + VIR_DOMAIN_CONTROLLER= _MODEL_PCIE_ROOT_PORT); + /* Skip if already assigned */ + pxb =3D qemuDomainGetUpstreamCont(def, &rootPort->info, + VIR_DOMAIN_CONTROLLER_MODE= L_PCIE_EXPANDER_BUS); + if (pxb) + break; + /* Assign the controller to the next available slot/func on + * the corresponding PXB */ + set =3D virDomainPCIAddressSetAlloc(nestedSmmuBus + 1, + VIR_PCI_ADDRESS_EXTENSIO= N_NONE); + set->buses[nestedSmmuBus] =3D addrs->buses[nestedSmmuBus]; + qemuDomainPCIAddressReserveNextAddr(set, &rootPort->info); + break; + } + } + } + return 0; +} + /* * This assigns static PCI slots to all configured devices. * The ordering here is chosen to match the ordering used @@ -2262,6 +2378,18 @@ qemuDomainAssignDevicePCISlots(virDomainDef *def, return -1; } =20 + /* Nested SMMUs */ + if (!addrs->dryRun) { + for (i =3D 0; i < def->nnestedsmmus; i++) { + if (!virDeviceInfoPCIAddressIsWanted(def->nestedsmmus[i]->info= )) + continue; + + if (qemuDomainPCIAddressReserveNextAddr(addrs, + def->nestedsmmus[i]->i= nfo) < 0) + return -1; + } + } + /* Host PCI devices */ for (i =3D 0; i < def->nhostdevs; i++) { virDomainHostdevSubsys *subsys =3D &def->hostdevs[i]->source.subsy= s; @@ -2286,6 +2414,12 @@ qemuDomainAssignDevicePCISlots(virDomainDef *def, return -1; } =20 + // Route hostdevs to nested SMMUs + if (!addrs->dryRun) { + if (qemuDomainAssignNestedSmmuv3HostdevSlots(def, addrs) < 0) + return -1; + } + /* memballoon. the qemu driver only accepts virtio memballoon devices = */ if (virDomainDefHasMemballoon(def) && virDeviceInfoPCIAddressIsWanted(&def->memballoon->info)) { --=20 2.34.1 From nobody Sun Dec 22 03:14:30 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.libvirt.org designates 8.43.85.245 as permitted sender) client-ip=8.43.85.245; envelope-from=devel-bounces@lists.libvirt.org; helo=lists.libvirt.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of lists.libvirt.org designates 8.43.85.245 as permitted sender) smtp.mailfrom=devel-bounces@lists.libvirt.org; arc=fail (BodyHash is different from the expected one); dmarc=pass(p=reject dis=none) header.from=lists.libvirt.org Return-Path: Received: from lists.libvirt.org (lists.libvirt.org [8.43.85.245]) by mx.zohomail.com with SMTPS id 1733963302789745.2543324661403; Wed, 11 Dec 2024 16:28:22 -0800 (PST) Received: by lists.libvirt.org (Postfix, from userid 996) id B243812DC; Wed, 11 Dec 2024 19:28:21 -0500 (EST) Received: from lists.libvirt.org (localhost [IPv6:::1]) by lists.libvirt.org (Postfix) with ESMTP id 617741273; Wed, 11 Dec 2024 19:25:21 -0500 (EST) Received: by lists.libvirt.org (Postfix, from userid 996) id 8248F1273; Wed, 11 Dec 2024 19:25:15 -0500 (EST) Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2057.outbound.protection.outlook.com [40.107.236.57]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.libvirt.org (Postfix) with ESMTPS id 89C36128C for ; Wed, 11 Dec 2024 19:24:58 -0500 (EST) Received: from SN7PR12MB6838.namprd12.prod.outlook.com (2603:10b6:806:266::18) by PH0PR12MB7791.namprd12.prod.outlook.com (2603:10b6:510:280::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8230.18; Thu, 12 Dec 2024 00:24:54 +0000 Received: from SN7PR12MB6838.namprd12.prod.outlook.com ([fe80::529d:478:bc5d:b400]) by SN7PR12MB6838.namprd12.prod.outlook.com ([fe80::529d:478:bc5d:b400%6]) with mapi id 15.20.8251.008; Thu, 12 Dec 2024 00:24:54 +0000 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on lists.libvirt.org X-Spam-Level: X-Spam-Status: No, score=0.2 required=5.0 tests=DKIM_INVALID,DKIM_SIGNED, FORGED_SPF_HELO,MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H2, RCVD_IN_VALIDITY_RPBL_BLOCKED,RCVD_IN_VALIDITY_SAFE_BLOCKED, SPF_HELO_PASS autolearn=no autolearn_force=no version=3.4.4 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=gbX+z9w4l0nrtYVqlAHFxy75UnQldAymsBIHv2Nd5smbCpm+LHH6a/ZJ1ZmjqAnHxCb2vfFtHOXZvffRl1FjYkSUf45EXB43E0PLDRx3239xC5Vy19d6JOw25mt5mwF8Bq4Kf8o4oWlN6mVefO3Ptvmg/fyu0JqOZjZp+hRRuW3t8sVCGh0H5GUmwV2M5HSn/REyrsQeMmY+LA/4DwuUxShhD1Ff6Vpr/yHAGiBL26ZQ1rKZAxR60hWq2QFlwvH/MtaQXOgwVowTn3s7EbVbHXUZqyBPhdWlyFJOyBij6fH3/04qc6cYk6iRz5IGUjgU2f88bA+O9gEpy5cTNKKl8A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=12DpWZnQeZGu6pGoGhDNjI11hW7+UUSHficQK5tMZiE=; b=S41mqPotB1bNGZ6zQoyBL2QKnlcr/g0q5yzMtWMSXe7vzL2jA32x2J6l+x9pRWOd8SSrPIlsQoj4gny6cpmGPl+1HIvoebJfL24z2Nn2kH1JmXkgHVkqeG5T5h1OsX3EODhbC2iDarWzV3/9w/EnnqtpD9rwVF6Wlqn9Gk4n5dgURii6hbTkocWDPjmCa47A+9MNVjuiKRj1Fx3ka2Ynlg0vp9EFtEAv1Jsh5c1BayWhVswlqmqG/wxZdn7Fs1/rkEVyYOtTBtoHR48VYdnvfsady7Wes7AwRFwdEhLHgdnLsyQ5q4CrTd09a0osdKkYnNDnxikMvjg0zqY0oO2mcA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=12DpWZnQeZGu6pGoGhDNjI11hW7+UUSHficQK5tMZiE=; b=RRG81AuZs9eHjHeHBJIjF0u29e3K2HMbnXKsy2aAODUE45XivmBup26NRElda2rvNRR96udiDrWDtT+rsaCD3lK0YCuNeRr7JFPKzlejmO53xqA0fEo4hVFhKJFL/D6ciEV9iHCDs/Hmr2ex2rFpBQfkHCs7R4I4NFdIA+UJNu7Gy5K24kftpZtcVSSeQTW5l3QkZ7ctAewWlXy4MxvMki6ypr7YDirRjkh7zHQHKbj6bb3ISuw2c8/q2+fzyBfffDOKbsHUmIg7BcWV55RT8++X0UtUd09bgC8Bbl6VG+JJ4tdMpuxPLKeJ8C4jVPN7yV9o0TI8OY0tw8l6QXXcpQ== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; To: devel@lists.libvirt.org Subject: [RFC PATCH 4/5] qemu: Update PXB busNr for nestedSmmuv3 controllers Date: Wed, 11 Dec 2024 16:24:22 -0800 Message-Id: <20241212002423.1050892-5-nathanc@nvidia.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241212002423.1050892-1-nathanc@nvidia.com> References: <20241212002423.1050892-1-nathanc@nvidia.com> X-ClientProxiedBy: BYAPR05CA0023.namprd05.prod.outlook.com (2603:10b6:a03:c0::36) To SN7PR12MB6838.namprd12.prod.outlook.com (2603:10b6:806:266::18) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SN7PR12MB6838:EE_|PH0PR12MB7791:EE_ X-MS-Office365-Filtering-Correlation-Id: 1e4f9cfe-99b8-46da-e885-08dd1a436676 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|1800799024|376014; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?hXZrOrl1Wh2q+1rl6LyQ6jxjRyYPW8MeRT50y0+QB00h5e8it+8djfM8g4b6?= =?us-ascii?Q?0ImkORh3hqo2LJfDDdPzadWY3tCYWl36qL/1/eofc3Dw5sF8CjvCeXNey2XY?= =?us-ascii?Q?OigjgcYgXl2I+YTvPCFLnvlHBOQHx9i7Av3ixjmCxXQEMy5P/4+9jFiHnVvK?= =?us-ascii?Q?9AycET3sZO2UeIKf3Qc3XOlL1iiKMy03AKR6rmMLkGDqm/CtFVXntv9cTFzn?= =?us-ascii?Q?TjOWe7enaw4Q7wdGrnjuqwi6KpNbJ5M4gwxlaow/pWHT5MMnCzpyOrwRRn3d?= =?us-ascii?Q?bQV2dn/U+ohAKwlBPo6NLQollfHHuwxyqV/nrLyKETuGJK24a53KrFP1Evd3?= =?us-ascii?Q?HV1s+v2e0kom8CKAMgeLJWKNH4KRtmjsQ3nzLOxZfoUq07IVn2Qs2832Ta51?= =?us-ascii?Q?TkcX51UQ9bIm54wUnHNOU28nkxZ2NWt87NgXvoSxf/3KFq3+6kaM4AWCg4gC?= =?us-ascii?Q?dLBSUsUObkNwXs/jg1gJmmvT9Kn+nZUXEw8+qdgDikqprmAWe/lmxZA2bjFr?= =?us-ascii?Q?LlgSpjhCTNIIVQ1dbbsL4iC6FvoPy0nS6JiHmTy/i5qON3MobIvNazu+Y1QN?= =?us-ascii?Q?wB44N6S3nqPFn5va2tGNTEByCCOEwQkHzYSds+40tsCVdfdL5pfnP68RIDVV?= =?us-ascii?Q?ENt62u46XPMf4VQsQkXdxx4fNNOYKWrvK2ycqhXQU76GeN/K0gdrEjJ2dIbK?= =?us-ascii?Q?8CLwErsXnGY5ul97QEgKMRl5QDkcEqoGw3lCmnj9I46QWeF+0yPS4ddmrmeJ?= =?us-ascii?Q?XijetGcUQbsgb937evxYAFTYKPIlM5GPHBD1J9LNDkP7cCG7YgoGvtdDmJbp?= =?us-ascii?Q?tXNOGmm5Pc3bamqnK7mSKUPZDD1A2sDeYDfmnc/f206c6GEU7i2BcLL+1+vB?= =?us-ascii?Q?gQuJXxG0L45BI0X4PA12AqS2gOzXTM5NXguwcHRy4Cg2836xmbScCvuU226/?= =?us-ascii?Q?Ierp7jldm9xAv3xo1nSkwMXD/0LayMYyygCh+SNNDlwX3U0ZUNwH+zaKz5DQ?= =?us-ascii?Q?9exXhwBjD5KVyrYCjnwmufSW5Za50x72elxN14JvRGdHzZAB/7h2i7EyyDCh?= =?us-ascii?Q?UgxjUEwF7Qecnf0Ei1zmyUvPK3BDPVyw0d3Qsm9fwFA4cbRDlosOKXfk/SI9?= =?us-ascii?Q?wXYGjpV09EgI2bvHNlcCJX99uI7RCoMYLeTamg0sUs3Eb+93ff8631T42Ze2?= =?us-ascii?Q?vFv4QIVTnqHyfBN3c2Jv38mfhyyvq7Yh6lPZuyBduK6J5zpEAPY5tR+6AEkg?= =?us-ascii?Q?PXZ7oDbz8c7+/TH4q++u+6mJ6JgfREtLKWMGdHJMiyVrG/YS99kDFaWoCf1/?= =?us-ascii?Q?EPjo5hfe3f3/qPeDsZ1heh23oFH/wcI9Sz3WbYS9jcBdpQ=3D=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SN7PR12MB6838.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(1800799024)(376014);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?Cp+iRrh8PO5vHYWdVi0JapHOc3VWP2CX6T0MAMYaqTgiYLXtKfKeFLi0qlek?= =?us-ascii?Q?EObnV2zZD6HylC7NTKHSf0W8Xlo13JQpPLAnhcl8fe4fzeNGoDAYkGN951Yt?= =?us-ascii?Q?Ibq09dKiEaE/tTws6RyrgUQ/CbSvjKxOXHnEGS2PHzUgGVxgiDipbj2U6d3Q?= =?us-ascii?Q?HqM2RvcP16ODDBBD8LV6l2yG9lvtSIlmMIpGjyTMSOUMJPjemuR3kkdcxrGR?= =?us-ascii?Q?aOlbg6lKNd1ZEyhAPbmBPentuw8qhMKlxAFXqUCbIIVOKtiuNPr1I1wVnXyy?= =?us-ascii?Q?x6nj6CYDS3RFmF65kg0F0crbZ3172ivdYkfEjeqeM2wVjwJsvkj/m+ZWRjNm?= =?us-ascii?Q?1ruW+QS16Kg7WEy6TbP1gKu8w9uCWqZmaj5okc58BBkXzx39uIDr8Bzkeprp?= =?us-ascii?Q?XhJ0u6wcqo571WaKTmrdZkgVb2iCxVtdQ7g0YXWbF0EBYnuZyr9PwUEkT1kv?= =?us-ascii?Q?dTH0lnifNjRc+jX8dj0sI/zQSvgGUcsZ9702LHLoBAB7As4M6i1omHOauqGA?= =?us-ascii?Q?Hm7VGvWxG7GBBWtSP6P78CPyq0p43mXMTIpFYVSJUlXGlXnpj1VyLm7qOxKv?= =?us-ascii?Q?dOe8bi5xrn+VNT/4LUYm9vmY8vUccGpELYr3MtLKfSYBZJKN0mkjHP+nah8+?= =?us-ascii?Q?6xp9ALXGY2LiOZE/MjS6QAsyKR+KNdD9blPvnHNTc1GfLocyw4jX0ARg1eO3?= =?us-ascii?Q?atajyTiVAXq1ns7oF3GdJvd5g56nXeW+x/aOyUx3yKeLvYkZanmlSwLahHPv?= =?us-ascii?Q?oBIoxj9qwnvY+DydomjKTo1zUDOlpavUBin8aENLa2Msy2Z+GRGkVOUegUqO?= =?us-ascii?Q?sQqrQg+rd/+7l1C+YM2F3gYHICS+f1drIVJ26OZ11Uz+D17Bi4tExPef9A77?= =?us-ascii?Q?rXC30S1piTf7Orm/ZHE5XRT4S+6DpVCftudVYzS3Ry8tXuhYHSAuKd+pAXij?= =?us-ascii?Q?c3CdB84W3raFR/zgTSoFMFGR2j7QHQX0rR264WXDTEFN0X0mHWONDOC1I27Z?= =?us-ascii?Q?8Cc8My+9O7fgX+iM04olPwFFeqkvRS0feNnocePfK56MvFfTIFB/FWDFkbo3?= =?us-ascii?Q?0yaCJ4OKFEf6lha4H5lvw/QamCeLJqNVdS7KbusmWR4sdkdYTfcLEiJIO6pa?= =?us-ascii?Q?1EM8liXUhk2qxu23JcgsenCKjrRMRqgcNEOS2Cco0aFPtLNn7UEgGDn41aoW?= =?us-ascii?Q?Rlu7iRpb20wuOx4lZl/qJcI1rFtMe+jRqd4bR1tYWpWBp8lUbSwVdOGf7aLo?= =?us-ascii?Q?2GGc9vX9savlQ3uOpjx9h0vNAQVl2mXNEAhSv48R7HCbGGyUsdBsMlmDrN0C?= =?us-ascii?Q?bhFvf7kZxxdxbEeb9oR19PLshO9vzYknTahXVSEd4isU/CF8X7xD3Zpf+iCQ?= =?us-ascii?Q?gwE87crMMUXtZz+zb/3XtkBvE4wK+/YWzZnLdoJIc3dOd/v6FjEY9sHT6qMr?= =?us-ascii?Q?ilhl7IHnt6cKpxU/h3aQOHpB7S57ylW8Z4TwadUukWuBDlvUhVtm1pB+0jRs?= =?us-ascii?Q?rm/nRctggYlw8bOe0KDfxVxXogSSNeL2y6GjO6T7A2JSk79DoFjcPoJ+AmId?= =?us-ascii?Q?+KIEnGwhXJyNeurUa3Wt2aq1Xet3kZwTOc8Wmr2q?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 1e4f9cfe-99b8-46da-e885-08dd1a436676 X-MS-Exchange-CrossTenant-AuthSource: SN7PR12MB6838.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Dec 2024 00:24:54.1644 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: qckdc7tl3JCbcAF3r+IezUtSiPzipJErhF0itgqr8sx5/zLmfmHhYek36fZxRaZqpYIQtSSmFh2sGvt7hHrVrQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR12MB7791 Content-Transfer-Encoding: quoted-printable Message-ID-Hash: FTGPNKMYQKQG42LK7HPBUZDX5TVZPSMB X-Message-ID-Hash: FTGPNKMYQKQG42LK7HPBUZDX5TVZPSMB X-MailFrom: nathanc@nvidia.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-config-1; header-match-config-2; header-match-config-3; header-match-devel.lists.libvirt.org-0; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; suspicious-header CC: Nicolin Chen , Shameerali Kolothum Thodi , Nathan Chen X-Mailman-Version: 3.2.2 Precedence: list List-Id: Development discussions about the libvirt library & tools Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: From: Nathan Chen via Devel Reply-To: Nathan Chen X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1733963304364116600 Content-Type: text/plain; charset="utf-8" Update the PXB controller busNrs to account for devices we attached in the previous commit, ensuring there are enough VM bus numbers to be assigned for each device attached downstream from each PXB controller. Signed-off-by: Nathan Chen --- src/qemu/qemu_domain_address.c | 52 ++++++++++++++++++++++++++++++++++ 1 file changed, 52 insertions(+) diff --git a/src/qemu/qemu_domain_address.c b/src/qemu/qemu_domain_address.c index dee198a7d2..001d1ec0b9 100644 --- a/src/qemu/qemu_domain_address.c +++ b/src/qemu/qemu_domain_address.c @@ -2678,6 +2678,23 @@ qemuDomainAddressFindNewTargetIndex(virDomainDef *de= f) } =20 =20 +static int +qemuDomainFindAttachedDevicesBusNr(virDomainDef *def, + int lowestBusNr, + unsigned int contIdx) +{ + size_t i; + for (i =3D 0; i < def->ncontrollers; i++) { + virDomainControllerDef *cont =3D def->controllers[i]; + if (cont->info.addr.pci.bus =3D=3D contIdx && cont->idx !=3D 0) + lowestBusNr =3D qemuDomainFindAttachedDevicesBusNr(def, lowest= BusNr - 1, cont->idx); + if (lowestBusNr <=3D 2) + return -1; + } + return lowestBusNr; +} + + static int qemuDomainAddressFindNewBusNr(virDomainDef *def) { @@ -2725,8 +2742,43 @@ qemuDomainAddressFindNewBusNr(virDomainDef *def) */ =20 size_t i; + size_t lowestBusNrContIdx =3D 0; int lowestBusNr =3D 256; =20 + if (def->iommu && def->iommu->model && + def->iommu->model =3D=3D VIR_DOMAIN_IOMMU_MODEL_NESTED_SMMUV3) { + for (i =3D 0; i < def->ncontrollers; i++) { + virDomainControllerDef *cont =3D def->controllers[i]; + if (cont->type =3D=3D VIR_DOMAIN_CONTROLLER_TYPE_PCI) { + int thisBusNr =3D cont->opts.pciopts.busNr; + if (thisBusNr >=3D 0 && thisBusNr < lowestBusNr) { + lowestBusNr =3D thisBusNr; + lowestBusNrContIdx =3D i; + } + } + } + if (lowestBusNr <=3D 2) + return -1; + if (lowestBusNrContIdx =3D=3D 0) { + for (i =3D 0; i < def->ncontrollers; i++) { + if (def->controllers[i]->model =3D=3D VIR_DOMAIN_CONTROLLE= R_MODEL_PCIE_EXPANDER_BUS) { + lowestBusNrContIdx =3D i; + break; + } + } + } else { + for (i =3D lowestBusNrContIdx + 1; i < def->ncontrollers; i++)= { + if (def->controllers[i]->model =3D=3D VIR_DOMAIN_CONTROLLE= R_MODEL_PCIE_EXPANDER_BUS) { + lowestBusNrContIdx =3D i; + break; + } + } + } + lowestBusNr =3D qemuDomainFindAttachedDevicesBusNr(def, lowestBusN= r, + def->controllers[= lowestBusNrContIdx]->idx); + return lowestBusNr - 2; + } + for (i =3D 0; i < def->ncontrollers; i++) { virDomainControllerDef *cont =3D def->controllers[i]; =20 --=20 2.34.1 From nobody Sun Dec 22 03:14:30 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.libvirt.org designates 8.43.85.245 as permitted sender) client-ip=8.43.85.245; envelope-from=devel-bounces@lists.libvirt.org; helo=lists.libvirt.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of lists.libvirt.org designates 8.43.85.245 as permitted sender) smtp.mailfrom=devel-bounces@lists.libvirt.org; arc=fail (BodyHash is different from the expected one); dmarc=pass(p=reject dis=none) header.from=lists.libvirt.org Return-Path: Received: from lists.libvirt.org (lists.libvirt.org [8.43.85.245]) by mx.zohomail.com with SMTPS id 1733963348986917.5782430679129; Wed, 11 Dec 2024 16:29:08 -0800 (PST) Received: by lists.libvirt.org (Postfix, from userid 996) id E94D711A8; Wed, 11 Dec 2024 19:29:07 -0500 (EST) Received: from lists.libvirt.org (localhost [IPv6:::1]) by lists.libvirt.org (Postfix) with ESMTP id 793281392; Wed, 11 Dec 2024 19:25:38 -0500 (EST) Received: by lists.libvirt.org (Postfix, from userid 996) id 24C22E69; Wed, 11 Dec 2024 19:25:34 -0500 (EST) Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2054.outbound.protection.outlook.com [40.107.237.54]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.libvirt.org (Postfix) with ESMTPS id 37BEA12AE for ; Wed, 11 Dec 2024 19:25:10 -0500 (EST) Received: from SN7PR12MB6838.namprd12.prod.outlook.com (2603:10b6:806:266::18) by CY5PR12MB6322.namprd12.prod.outlook.com (2603:10b6:930:21::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8251.14; Thu, 12 Dec 2024 00:25:04 +0000 Received: from SN7PR12MB6838.namprd12.prod.outlook.com ([fe80::529d:478:bc5d:b400]) by SN7PR12MB6838.namprd12.prod.outlook.com ([fe80::529d:478:bc5d:b400%6]) with mapi id 15.20.8251.008; Thu, 12 Dec 2024 00:25:04 +0000 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on lists.libvirt.org X-Spam-Level: X-Spam-Status: No, score=0.2 required=5.0 tests=DKIM_INVALID,DKIM_SIGNED, FORGED_SPF_HELO,MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H2, RCVD_IN_VALIDITY_RPBL_BLOCKED,RCVD_IN_VALIDITY_SAFE_BLOCKED, SPF_HELO_PASS autolearn=no autolearn_force=no version=3.4.4 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=cOkI7gTfxfeLXjL5CB0P4c0mhDlb8jeFyH92w8WMbpv+yJFy0Lhs+m2BWSpHtJTHa5O+TpRk0Shq77KqFUk9ZKrKLcw2O95E8sT6zVoerLB/i7QQuuT2UdHWfRBd21M2pIVgyAcxkXP6yzT5uDfWsVbLR6AB450HRAV09a1YbhqQIzLyCd+82JF385svCDD7ktn/3oFXUdlw0Z/7P4+9woVDvS16V2McPEUyQV2GhyCGAhc2zPuHqNZrll1sua8zg8LrJFwHe64Z5otmxqz3u9dwTp4pWr9FoBeGcb+lRmkGsBszRfl5zp6QbCQZqZxkAofnZZ1/eneSTSS69H6n6g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=JwNegjpvHPDapC53ZQbYNC+oOpok57Dky1wT93CzSpY=; b=Lua5qKBsmSD+gOwfNb/2xP+pXmUPU/wsGE+uS3apiX7JNvcYpDF4AX/jNr0mx+6TP3/p5+s+kH4vZAjq/V1gW2jxvycukfBwTEr9c9I67SHibOrhOAjHwEl8gD9f1WOiZkSTFjyBrKIfM45ZJ3nNrj9jFiPQyiW/JPPcUj/ZLXo9U4/JNTTEwE2Ejy5IncfDZ+U5Ai6IoddTm+sO1OLjHqFKuISi6sHNw5RTzQEGDwQjOtA4FxtPQM83lmTHR/mtzlmBTLTNNQh2MiSAje3Ikzdr41t20LVRZCDbxUZ53lEWHmD9kJ7JMmkNY/Vv6hYp2YruUt+F7s4CipruIN+xww== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=JwNegjpvHPDapC53ZQbYNC+oOpok57Dky1wT93CzSpY=; b=YndvbS8z2rYGsP3VDZqI/F5zDknVBDj72DdSj7wX9WCEq4is8pH6VQNluOtY5PqVS3oyhGDFgddarNVPDMBXRXis1c4gy/kk4as36tiz4X2jGCfXEC6PsoNk31e4GIeL+47YNeRL83zNd6R1YZj8qB1cyTrb60cnWx3B11Qg1l97EgZ06JKbq4f8sO8J04nNeIyCQD4wu1J7+d73LJL7Yt8ARDq7N4L5bbzKSv3talal7lro6aBRDuzXJQ+HlWaetR/9fK3NanP1Va6FNDm0tNHA2Jr+AvgzlNWhl0PIjXv6te0oF6qqYEc7rGX+PxhLVtum2q6GmSDqK0LBT//jQQ== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; To: devel@lists.libvirt.org Subject: [RFC PATCH 5/5] qemu: Add test case for specifying multiple nested SMMUs Date: Wed, 11 Dec 2024 16:24:23 -0800 Message-Id: <20241212002423.1050892-6-nathanc@nvidia.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241212002423.1050892-1-nathanc@nvidia.com> References: <20241212002423.1050892-1-nathanc@nvidia.com> X-ClientProxiedBy: BYAPR05CA0001.namprd05.prod.outlook.com (2603:10b6:a03:c0::14) To SN7PR12MB6838.namprd12.prod.outlook.com (2603:10b6:806:266::18) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SN7PR12MB6838:EE_|CY5PR12MB6322:EE_ X-MS-Office365-Filtering-Correlation-Id: 3cf7a31c-ed95-4274-576c-08dd1a436c84 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|376014|366016; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?YYfAYBYL3DqxhsuNlGcopyr9c0Xebi8kOXSj+Wg25Heo4Os9+pw+vmJJBdVO?= =?us-ascii?Q?1pwU0QVqfhRAifXyhafIvo106PwXQu+6InnvFORRER8HRjbHJlB4gcHa/Mmx?= =?us-ascii?Q?b5ooNwahCS2wH2ecRclaPEhLii2uQ068bk3Au5AS/1r6O8hy101Fo4nK7EHr?= =?us-ascii?Q?EyOdmO3icNGMvaxmjk/EXRcpcbFYhf14clwku4ymUdgwm8hswZEpuka9MFq6?= =?us-ascii?Q?F8AUO+5+Dfn4jj56WEQsqPsq3XzuOQu/nYH+Z2k/xdV5hb3sUZh/c33rdibd?= =?us-ascii?Q?m1jjEOoQW+/0psT7YcwQmPg3CEvponLdc8F0N6mUVwP27mUkDmqB0k6AcsUO?= =?us-ascii?Q?jA0hnN3HIsdxI06s4V4SW1lfrKkRQ+ZMvB+XS6Wdi+DTgBOG/fQKeIWKWDLT?= =?us-ascii?Q?8VN/M0FdGYVfc+M3ABTLZsbjkGJ8Y0u6FWVTKfSqNYFkjSVhIiUo5Zzpl7Xz?= =?us-ascii?Q?soaBR0iuiOlPiDI3o6T69X0bboD6JL+GmAw4MtKLBk7bosxJHS+TrHg2Bnsp?= =?us-ascii?Q?5m58ZP16t31GMVCiTvgC6ZXL+Uf49bvxEU/DKaEB2dU1h08ReOFWVQ6Fq/9i?= =?us-ascii?Q?HjNJ1cGfujLZq0FtcZxwwakl39q7eSlwLIqKMRN7kTyiJcjF6USU8sJPLAxv?= =?us-ascii?Q?4KZhRN5WGNtfLAB9FHbvn5RZZCamEkpm1wnwBnU+gD+KsuppazZjvOxzv3Ye?= =?us-ascii?Q?/lYGvk1u7L6fCEuh/KmGfbVcJ2nGEBpQqJP6CK6kECRrsa5S9KxCVRlO1A9Y?= =?us-ascii?Q?QiWoLspDvYYqR746NsVxkaM5mmuRCOSzdmN21yBswJbkUWWasmEEhm8IU1w+?= =?us-ascii?Q?xeTwDI2D1E9+Caim+TzTWxZ1ss59yREalKmU625fwFcEvHRORmOq2owkg+0P?= =?us-ascii?Q?Au9ivh7i+ahESrE1M+5PMnAULoavv8g2V1e6sAcyum3in3nMFtbr7Gwwtxgj?= =?us-ascii?Q?Cv/8V56t0GNUdOmNXdlhscoWEAbghjPIN8u/8sePKfKEL4ye3pCkRvV3M0gn?= =?us-ascii?Q?KeQJnwyTSzVvEDxWORuHCEuzUV0UwcZAG69k1CX3zT/nvRDxDrQQ0M2VFCwL?= =?us-ascii?Q?3e/zRRGCnIGBur52FhO3uV5LS3YmtvZWgpcEZBKPMhEAbDyJ8RpF1YmPuQm2?= =?us-ascii?Q?o2UG/2Up6fnoy1qbRzfcaF9Hz3Zi/SyejvqpG0Ktu3zQldBrPbJ9o5uyHOIE?= =?us-ascii?Q?TJELWhjGgazw2amwO3KwAYeiZRY5HfKTYezi6ZXS9d1O9CrcccvFOKmWitVa?= =?us-ascii?Q?2zW/v8PhAAKoqhs2Rxd7ss3Ug7bhLspyO0PjRz8COCh6EvmIuC7bYdeAX8jO?= =?us-ascii?Q?ghK8wtOcgO4JOOb3a1sBUzIqxumyxyqOYtg0z9+2VUnnKA4QJhRhAOQ9yYwR?= =?us-ascii?Q?HGgy9lM=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SN7PR12MB6838.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(376014)(366016);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?gcHBhHWZgiyRM8Fhjkhwwbef3x6H8UkZTVxocJAmWtBM2jIpVYqslcR0ZJqX?= =?us-ascii?Q?0U6dlqPqKp8raA7144GKGBr5Ce/aY2JNSbgfb1XjNqQzY3yNZjjDlDXSh9no?= =?us-ascii?Q?SSaT7z/F01npEATMLJqSzxRA/Gx4+LqZa3VFCinbIH9GZEv54Y8uFQ5jzuzy?= =?us-ascii?Q?y+D6ly0pdPD110XcjZXnGEi5+0nm9SAiRCDWKR7jDOOP5ptgdvvNT9zW8KLj?= =?us-ascii?Q?xgyIBzXYWh+G96vPG0dICxGDFc/UAux7a1f3WCr8nS1o+V7v55BfYXzUChCZ?= =?us-ascii?Q?Cvm814KKDuA4SOQ6pMBr4SLc0Xt/HljtbM5zxMyIgtLIYePTXQqyD1fekkgb?= =?us-ascii?Q?AyWwwqkdACwlyJjyrCEoEDDwNJfffGMS74o+xqOwUh1TxBPA0dHm7OLBOUhb?= =?us-ascii?Q?LkNi0Lq6mEX7c3a6+Iy9EI09ADVrsv0WNiKCMy9JwZgyImQ5mfr3/GYjlSYm?= =?us-ascii?Q?XJiaaGMTCaXPeQ/UddwLeopAGboTOsaYcCbxHnTF5ixwdhO9DBv7FKOEEsPW?= =?us-ascii?Q?h1kSXOo9Bq2dUGWfEG/xeXVJKfFjWnpGTmuwiKkxkWKhyu2jBWAv7a25SO5E?= =?us-ascii?Q?Ax79UXW9tNmDcBL0l4R4dHKEwzCH0R5Nk1iIgwC01vB/TaSUQfcKrZYyEpNx?= =?us-ascii?Q?jNb1PCnPLjPz4ICg7QnXnKh3IY7V3QyzmBos6uHWEBcNqqUleDB6X8wUl5SC?= =?us-ascii?Q?q2wgUG7KXOD7rMDODVzjIYLlvcezGPHRIzFMT0BMs6QUrluvpHSQ6sHCDY39?= =?us-ascii?Q?3g23TwOoRH23uRDyvOMxGyL6o2gHQSaLoTgYzBKcec61vNpxDP3WjQ9GomWw?= =?us-ascii?Q?qD6cqRfDvNXX2ftKw1Q851igESF//f0GDyUO6JMR2A5aq+h96f9Mwnsjm+4j?= =?us-ascii?Q?jJXPeswjtxYh2X+PdjqE2x3UQpM1R/ykhy5qb9zepmhx0Bh4LxQEV3l7xDjg?= =?us-ascii?Q?tDpHrG9QPrZFzLGNE3ZrkNUMLbNi7yEyZCFLB4kOlTSMs4VErj1E1mFir11J?= =?us-ascii?Q?qJqvdvpDn64J/XDNbdk+Z5FF4ODALQyRKIvsejLsEa71WQgGQ7iwo+Gow1m3?= =?us-ascii?Q?xwnrZJ05cEIoapsMhpLTgl3dPuJnYo+6G4yU4YpeT1Farp/s/uRAinUsCoEp?= =?us-ascii?Q?Rmrb61fwFHmU7kPom1VqVBd6iW6P18F1Fb1SgFo0tnK2zMb073K3UgbWsbTh?= =?us-ascii?Q?+tGg/nBVmPpdSOXVHuRmyIIeyuxB6HEFQ8lm5vXKqOA23miQWQa9v3SWgxFE?= =?us-ascii?Q?ulkwdeF489qklAqIKpi7vVKbDaWtG22Akg4RiHGc+jXsIsW+/Z1LB1YQI9gm?= =?us-ascii?Q?0cOP7RUizZAHqBEWaktBP+ba0XwP20y6sNtGMDTJtrFfmCLg0cdFKgS8uInV?= =?us-ascii?Q?m8aOIQ2efGjU2X2Qp5iPSzUUPOCDEH3W7yPMqZP7zU8rZcCOn5K/+vpOdRfK?= =?us-ascii?Q?xRRFaziCZI3arK3RUWZ8VhlRjsy4F3N2IGhi9tz0eYLN+wI2KNZkf2wNMQd+?= =?us-ascii?Q?wVArwAc96fbhcZgrcRcTQ8WE6O2yVtHQMVS+0m4qfFUTCTbJJ7mQvNGU9ZJ2?= =?us-ascii?Q?26tev6OVrt46fQZ5E1N860l+4sc3G2+7inZ/mFMG?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 3cf7a31c-ed95-4274-576c-08dd1a436c84 X-MS-Exchange-CrossTenant-AuthSource: SN7PR12MB6838.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Dec 2024 00:25:04.3488 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: dDf2MeYXZ63oEXToQ6XcfWyKfZ0Ntgr5TAmpngNQwMAAcS6RgZ6cB9AHWUr5dYwtFZE6KQJmNmDC9G5+t/Nsmw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY5PR12MB6322 Content-Transfer-Encoding: quoted-printable Message-ID-Hash: DJUJZGCPHIAU4JPFPIF2MNVLMBU7Q6LW X-Message-ID-Hash: DJUJZGCPHIAU4JPFPIF2MNVLMBU7Q6LW X-MailFrom: nathanc@nvidia.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-config-1; header-match-config-2; header-match-config-3; header-match-devel.lists.libvirt.org-0; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; suspicious-header CC: Nicolin Chen , Shameerali Kolothum Thodi , Nathan Chen X-Mailman-Version: 3.2.2 Precedence: list List-Id: Development discussions about the libvirt library & tools Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: From: Nathan Chen via Devel Reply-To: Nathan Chen X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1733963350505116600 Content-Type: text/plain; charset="utf-8" Implement a sub-test in qemuxmlconftest that mocks the scanning of sysfs to determine how many "nestedSmmuv3" devices to assign to the VM when "nestedSmmuv3" IOMMU model is parsed from the VM definition. Add a VM definition example with an associated qemu command line that exercises the detection of "nestedSmmuv3" IOMMU model and auto-creation of PXB controller= s and "nestedSmmuv3" devices. Signed-off-by: Nathan Chen --- src/conf/domain_conf.c | 2 +- src/conf/domain_conf.h | 2 + src/libvirt_private.syms | 2 + tests/meson.build | 1 + .../iommu-nestedsmmuv3.aarch64-latest.args | 38 ++++++++++++ .../iommu-nestedsmmuv3.aarch64-latest.xml | 61 +++++++++++++++++++ tests/qemuxmlconfdata/iommu-nestedsmmuv3.xml | 29 +++++++++ tests/qemuxmlconftest.c | 4 +- tests/virnestedsmmuv3mock.c | 57 +++++++++++++++++ 9 files changed, 194 insertions(+), 2 deletions(-) create mode 100644 tests/qemuxmlconfdata/iommu-nestedsmmuv3.aarch64-latest= .args create mode 100644 tests/qemuxmlconfdata/iommu-nestedsmmuv3.aarch64-latest= .xml create mode 100644 tests/qemuxmlconfdata/iommu-nestedsmmuv3.xml create mode 100644 tests/virnestedsmmuv3mock.c diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index 46f9b9b0cf..d8b473854e 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -22491,7 +22491,7 @@ virDomainDefAddImplicitVideo(virDomainDef *def, vir= DomainXMLOption *xmlopt) return 0; } =20 -static int +int virDomainDefAddImplicitNestedSmmuv3(virDomainDef *def) { // Get the number of host-level SMMUv3 instances diff --git a/src/conf/domain_conf.h b/src/conf/domain_conf.h index edde9b63d6..8e7e8166e3 100644 --- a/src/conf/domain_conf.h +++ b/src/conf/domain_conf.h @@ -3873,6 +3873,8 @@ bool virDomainDefCheckABIStabilityFlags(virDomainDef = *src, int virDomainDefAddImplicitDevices(virDomainDef *def, virDomainXMLOption *xmlopt); =20 +int virDomainDefAddImplicitNestedSmmuv3(virDomainDef *def) G_NO_INLINE; + virDomainIOThreadIDDef *virDomainIOThreadIDFind(const virDomainDef *def, unsigned int iothread_id); virDomainIOThreadIDDef *virDomainIOThreadIDAdd(virDomainDef *def, diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index c931003fad..2527f0844a 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -312,6 +312,7 @@ virDomainCryptoTypeTypeFromString; virDomainCryptoTypeTypeToString; virDomainDefAddController; virDomainDefAddImplicitDevices; +virDomainDefAddImplicitNestedSmmuv3; virDomainDefAddUSBController; virDomainDefCheckABIStability; virDomainDefCheckABIStabilityFlags; @@ -531,6 +532,7 @@ virDomainMemorySourceTypeFromString; virDomainMemorySourceTypeToString; virDomainMouseModeTypeFromString; virDomainMouseModeTypeToString; +virDomainNestedSmmuv3DefNew; virDomainNetAllocateActualDevice; virDomainNetAppendIPAddress; virDomainNetARPInterfaces; diff --git a/tests/meson.build b/tests/meson.build index 0d76d37959..d3b9d4612b 100644 --- a/tests/meson.build +++ b/tests/meson.build @@ -177,6 +177,7 @@ if conf.has('WITH_QEMU') { 'name': 'qemuhotplugmock', 'link_with': [ test_qemu_driver_lib, test= _utils_qemu_lib, test_utils_lib ] }, { 'name': 'qemuxml2argvmock' }, { 'name': 'virhostidmock' }, + { 'name': 'virnestedsmmuv3mock' }, ] else test_qemu_driver_lib =3D [] diff --git a/tests/qemuxmlconfdata/iommu-nestedsmmuv3.aarch64-latest.args b= /tests/qemuxmlconfdata/iommu-nestedsmmuv3.aarch64-latest.args new file mode 100644 index 0000000000..bf094dd20a --- /dev/null +++ b/tests/qemuxmlconfdata/iommu-nestedsmmuv3.aarch64-latest.args @@ -0,0 +1,38 @@ +LC_ALL=3DC \ +PATH=3D/bin \ +HOME=3D/var/lib/libvirt/qemu/domain--1-guest \ +USER=3Dtest \ +LOGNAME=3Dtest \ +XDG_DATA_HOME=3D/var/lib/libvirt/qemu/domain--1-guest/.local/share \ +XDG_CACHE_HOME=3D/var/lib/libvirt/qemu/domain--1-guest/.cache \ +XDG_CONFIG_HOME=3D/var/lib/libvirt/qemu/domain--1-guest/.config \ +/usr/bin/qemu-system-aarch64 \ +-name guest=3Dguest,debug-threads=3Don \ +-S \ +-object '{"qom-type":"secret","id":"masterKey0","format":"raw","file":"/va= r/lib/libvirt/qemu/domain--1-guest/master-key.aes"}' \ +-machine virt,usb=3Doff,gic-version=3D2,dump-guest-core=3Doff,memory-backe= nd=3Dmach-virt.ram,acpi=3Doff \ +-accel tcg \ +-cpu cortex-a15 \ +-m size=3D1048576k \ +-object '{"qom-type":"memory-backend-ram","id":"mach-virt.ram","size":1073= 741824}' \ +-overcommit mem-lock=3Doff \ +-smp 1,sockets=3D1,cores=3D1,threads=3D1 \ +-uuid 1ccfd97d-5eb4-478a-bbe6-88d254c16db7 \ +-display none \ +-no-user-config \ +-nodefaults \ +-chardev socket,id=3Dcharmonitor,fd=3D1729,server=3Don,wait=3Doff \ +-mon chardev=3Dcharmonitor,id=3Dmonitor,mode=3Dcontrol \ +-rtc base=3Dutc \ +-no-shutdown \ +-boot strict=3Don \ +-device '{"driver":"pxb-pcie","bus_nr":254,"id":"pci.1","bus":"pcie.0","ad= dr":"0x1"}' \ +-device '{"driver":"pxb-pcie","bus_nr":252,"id":"pci.2","bus":"pcie.0","ad= dr":"0x2"}' \ +-device '{"driver":"pxb-pcie","bus_nr":250,"id":"pci.3","bus":"pcie.0","ad= dr":"0x3"}' \ +-device '{"driver":"pcie-root-port","port":32,"chassis":4,"id":"pci.4","bu= s":"pcie.0","addr":"0x4"}' \ +-audiodev '{"id":"audio1","driver":"none"}' \ +-device '{"driver":"arm-smmuv3-nested","bus":"pci.1"}' \ +-device '{"driver":"arm-smmuv3-nested","bus":"pci.2"}' \ +-device '{"driver":"arm-smmuv3-nested","bus":"pci.3"}' \ +-sandbox on,obsolete=3Ddeny,elevateprivileges=3Ddeny,spawn=3Ddeny,resource= control=3Ddeny \ +-msg timestamp=3Don diff --git a/tests/qemuxmlconfdata/iommu-nestedsmmuv3.aarch64-latest.xml b/= tests/qemuxmlconfdata/iommu-nestedsmmuv3.aarch64-latest.xml new file mode 100644 index 0000000000..0aa9296eaa --- /dev/null +++ b/tests/qemuxmlconfdata/iommu-nestedsmmuv3.aarch64-latest.xml @@ -0,0 +1,61 @@ + + guest + 1ccfd97d-5eb4-478a-bbe6-88d254c16db7 + 1048576 + 1048576 + 1 + + hvm + + + + + + + cortex-a15 + + + destroy + restart + destroy + + /usr/bin/qemu-system-aarch64 + + + + + +
+ + + + +
+ + + + +
+ + + + +
+ +