From nobody Sun Oct 5 01:50:07 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.libvirt.org designates 8.43.85.245 as permitted sender) client-ip=8.43.85.245; envelope-from=devel-bounces@lists.libvirt.org; helo=lists.libvirt.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of lists.libvirt.org designates 8.43.85.245 as permitted sender) smtp.mailfrom=devel-bounces@lists.libvirt.org; arc=fail (Bad Signature); dmarc=pass(p=reject dis=none) header.from=lists.libvirt.org Return-Path: Received: from lists.libvirt.org (lists.libvirt.org [8.43.85.245]) by mx.zohomail.com with SMTPS id 1759365009344657.4418381625383; Wed, 1 Oct 2025 17:30:09 -0700 (PDT) Received: by lists.libvirt.org (Postfix, from userid 993) id 3F260440BC; Wed, 1 Oct 2025 20:30:08 -0400 (EDT) Received: from [172.19.199.14] (lists.libvirt.org [8.43.85.245]) by lists.libvirt.org (Postfix) with ESMTP id F34A544241; Wed, 1 Oct 2025 20:27:09 -0400 (EDT) Received: by lists.libvirt.org (Postfix, from userid 993) id F071441B06; Wed, 1 Oct 2025 20:24:09 -0400 (EDT) Received: from CY7PR03CU001.outbound.protection.outlook.com (mail-westcentralusazon11010026.outbound.protection.outlook.com [40.93.198.26]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (prime256v1) server-signature RSA-PSS (3072 bits)) (No client certificate requested) by lists.libvirt.org (Postfix) with ESMTPS id 9268341A52 for ; Wed, 1 Oct 2025 20:24:08 -0400 (EDT) Received: from PH7PR12MB6834.namprd12.prod.outlook.com (2603:10b6:510:1b4::18) by DM4PR12MB6232.namprd12.prod.outlook.com (2603:10b6:8:a5::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9160.17; Thu, 2 Oct 2025 00:24:02 +0000 Received: from PH7PR12MB6834.namprd12.prod.outlook.com ([fe80::f432:162b:b94e:d2cb]) by PH7PR12MB6834.namprd12.prod.outlook.com ([fe80::f432:162b:b94e:d2cb%6]) with mapi id 15.20.9160.015; Thu, 2 Oct 2025 00:24:02 +0000 X-Spam-Checker-Version: SpamAssassin 4.0.1 (2024-03-26) on lists.libvirt.org X-Spam-Level: X-Spam-Status: No, score=-3.1 required=5.0 tests=ARC_SIGNED,ARC_VALID, DKIM_INVALID,DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED,RCVD_IN_VALIDITY_RPBL_BLOCKED, RCVD_IN_VALIDITY_SAFE_BLOCKED,SPF_PASS autolearn=unavailable autolearn_force=no version=4.0.1 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=qmp6S1pMfrIdjNH1FMrZoLo96PFsYdCzcwRGp0wNhOKICa+Qdcu+WoQ8413XpCYcTaTCyTyh6cKcS5CH8xIgS/QPgcWVGBdOsjJJQ1I5lrm0/yMUNMxxGFLWklc0zAbnADE/VqOciFBUsKTUX/oGxRMpJyuMhhSumX/kTiv1wIqIDJufUAhlUEfrIDLEKUkhCH6+LUGrZGlIeesc5+nn1LUSbALRQI6FTvkL/o9NkXMlIHHXvP9CnBPDod4Xtc9MN89n7KiKPDFUqG0JyhjPRiO1Ocx/MuEh0mO6x1hqKW5L27z+KwaBHvBwIUbnSrKW5RppJpyUZjXy3G6vSkRUXw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=8ijdlPvuOd3/dcUZd6cHXhVhzqZiIa3wN7nTRwNw+yA=; b=ECsagJ+eFZMySmS0JUARiXF5QR7IHwFUo0Frd26UB1I9+P4a1oZbwAFJTwf5nZSpE2idvXBMuFV8rXQAbh1hHuifKWdpVw/IIgh60m+AggqllOsdqBj5ML4c83BmhCbX/SevjiGlU+vPvhGQT5nivDSWKHs67wuaXkcpCS9acTsXratdEpUoiooEwKaG7M6FGOJkRTKD0Db9MWeF1EeJ3DoW5aklK4RtWvtrw9tNZBSwueoqykQFukhRstFyx/ZyAorMEpUlf0kiHH/twaJJEiCo7PTbJCLvmXCmYNQxwy9W90FLrMSh5bLqeoWMUbmGsKCzdVjdT6XIpA9lHlrEaw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=8ijdlPvuOd3/dcUZd6cHXhVhzqZiIa3wN7nTRwNw+yA=; b=L2FZRCXGFhsIImZzFqqUOicxyNf7NEkvKczc+gjmxLNl+VM1+O8FXpKKVBJVrkN02bh9ciwtGDU/qISl5gtH8m+Gv3qG+pCN9EU2rxO2iJdScl6ubAp1yLQvBU5NRv05Y4zSKBGHtLhD179j8dAL/G8Z1UQnDsvKEz/xiva5NVZDvj1SuYgN3jt8PP5ZpHHW6QFjVQJWnrKmkeyMHQIwhrWk+vKVfxI6wDXGgRitqGagOxRkGMBALw9+XRnzXTw9zxN1VSYoQBLpQGf68YWNFQ6m5oBRaRb+CEeOrSd3uJYwnyGSh1gjNai40ow/v/oEPBMeGnpaRCzRYr8fKyV7cA== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; To: devel@lists.libvirt.org Subject: [PATCH v2 2/3] conf: Support multiple smmuv3Dev IOMMU devices Date: Wed, 1 Oct 2025 17:23:47 -0700 Message-ID: <20251002002348.1536936-3-nathanc@nvidia.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251002002348.1536936-1-nathanc@nvidia.com> References: <20251002002348.1536936-1-nathanc@nvidia.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: SJ0PR05CA0033.namprd05.prod.outlook.com (2603:10b6:a03:33f::8) To PH7PR12MB6834.namprd12.prod.outlook.com (2603:10b6:510:1b4::18) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: PH7PR12MB6834:EE_|DM4PR12MB6232:EE_ X-MS-Office365-Filtering-Correlation-Id: 98d02af0-095c-427c-7520-08de0149fd27 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|366016|376014; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?SGcVpXnz3Vn1h0U0PnZHTNMn+vcB3JtgbISymVB+/vYb+iLurkUwu3a8NN28?= =?us-ascii?Q?+SrbBz8McQRDYXe4vlKLRJf198ZTNzaNV+idZueMJCKDiayd05shU+oxSZbJ?= =?us-ascii?Q?Z7va77jFs6L7o8H/RKSU6FMiLVeI2UNSJDTv7n+eI+LeVSvKIoyA3+/loRuV?= =?us-ascii?Q?Lkp59HzdTZV3NpBMIaKFF40DGjthjoj/bQAE36SeM67OUhH4iz1vb0j00K5V?= =?us-ascii?Q?T6Heypplboyyk74oNeUc5Al2U7AlE8zk9gwrM4RASnInb5SQMxJFzUYM9eq9?= =?us-ascii?Q?MLAClOCXTHvi550k9QmPQwxOe9NJSYtWSBCID5W2s4yXOMi9l+eov4mi58a4?= =?us-ascii?Q?7/Xy3AlUUfQX5kclIbrHW79wmdZdafoZKP6jsg4npWeHaOZv7EPPHWS5OgvB?= =?us-ascii?Q?eN/07AYEVCi5tFExhZEcR9uEyR/vzGWI180zJTnJ2Rn1t0jKmFzFuoWdagDi?= =?us-ascii?Q?JK5Xw1C26hJtiANZGyT4WAZcluCnd94ia4RRVNQs/bYf49/I7KLotMkCd+wA?= =?us-ascii?Q?m6SUYE4GCnZxYDRD79yPedT0/wYMMR+xIceHfD85NdhAtl5EBtYgB8Xa7804?= =?us-ascii?Q?Aa7NY9QExpuvFDJUwuYiUC/sURKyK/1n7TLVpEJUYuFh47jOI/SnlDoUT3VZ?= =?us-ascii?Q?uNE8i2drZO7yg7feCGhWIxuQMcywM3M3ESAJE58EWunqtgUWNBoWjiz3p7PI?= =?us-ascii?Q?SF9IU8UBeA2Mu3JeiWItZjelzUssAhPE3/dp+Sz96xF+BlCjLJ59wIZoj8K4?= =?us-ascii?Q?btiS0j3KKQS9+6klStTmx4W+hlO4j3BK3w2FzOf2eI60k1XNEvTGjbllm9OB?= =?us-ascii?Q?WaGtqjiRWdWvg+KhjlgmKyQNEpfS3WeGwvEgs2A+kDNyAAMiX9fm6b4p7gGB?= =?us-ascii?Q?AHDjgOMwCeM/VE3caQc5zct8CD+NDCcen0DELr4GjtJ5Pqt+zABnBNCuNL0D?= =?us-ascii?Q?IHNMdMuuocu3DOQ98JwFss52UVdDab04ITFpxfJQIKPX83U81DAeLloXMmDs?= =?us-ascii?Q?aUIoMl1/i1H1kItwaYo6deVKPePhMyH+pKzt5eJ39U6JdiTFUsXzeTVwYBa0?= =?us-ascii?Q?U75aoulTteF42S9chdMlIqPMmUM/vrx9rnFZG8KqBOVLnPSg09HwCJ3KleX+?= =?us-ascii?Q?ngRN8wHib5k99QKua4itRwFpUUG6Nz5kgJKwANsNF5JGUTbiJGfJA5ygrFdo?= =?us-ascii?Q?eHH4sQwrIqMyWELD9T3agW2VJweo15SjY3mEloYHn37NFpGDii6HKUQGekWg?= =?us-ascii?Q?Ve0hq03ge5Fnm3LzUBR7rgspTf6mdaEX7k/bwHlqrtN4OXzrUtz4FoEw2w7b?= =?us-ascii?Q?IixJ6fqchjhHC/tW+fC8zrMChervPeOL8aBS8LS7TiMJADg90v2c1vs3lvJY?= =?us-ascii?Q?aRKJ/o+S/b/npvQHuhQdGv4qpuPhxIFXfe2IUGuH9hCNn5o1I4k3qYZDu7iZ?= =?us-ascii?Q?56sqJQv9e8/KWl0gettHL7cl+LttSefn?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PH7PR12MB6834.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(366016)(376014);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?clm9P2rMt9EQgtU0yKjGZYEZ8AyZttYGZb2i3EQekdFyhkfWPic9j4PqK67g?= =?us-ascii?Q?9oJXlGc4luAQGlTeleNwx1djnXvcc6IpX0zVAULCppteiS5Xtfifh7HsydSa?= =?us-ascii?Q?DfuS7y8WnUfhJovlZBFT5MRb01eMiSajZPNtl72sc/+Y2FEfHPYWU6f2pb/d?= =?us-ascii?Q?0KTXrWmlJOsHVrM6lWwgnmhVNmXQkR6BqfbeqJJbr82ofphZzIMiYRjHGe7s?= =?us-ascii?Q?UF2Ky0hd57Bn841zv7kXhmBt3kToNisfkUSWH/OJt/CWce+rG7kPOmwI5tTN?= =?us-ascii?Q?OsR31DlIwtsedxZN4CQL1/GMoXYXomMHsIzV+IUVcUg3habLT6QCTA5ens1r?= =?us-ascii?Q?MJ2gwV0EANt6AyiUEEQzlalt5ByU0/dyGqUaFtgscFaDX8fMrrOIlrvb6EOm?= =?us-ascii?Q?DDvA1K19hS5Xn/OA/Jou2vwtcAq2ITBK64s8PAXOsRL5eBCBII8+Oal3z85T?= =?us-ascii?Q?ZOy+fLbeBDaLPeCCMv+cTAkaknix9shOehd7JELksVzfRx8nkQgeP4PUgC5s?= =?us-ascii?Q?vnNH9Z7nNraf/4eBjQZxOnLsuI8OGd7gmNGMpmsaiNMJH0SWlCv7t4JYbvfm?= =?us-ascii?Q?sgwot01LOfTQ0DE/jE/JsX3sym1p3kr3kGL5NvIDYqIZc1+7OLmC7KAYw5Ir?= =?us-ascii?Q?oQLQETe0lpwzdGuj3wmeFHd3Py3daaJ50k7CjbjGcHE8ZgGmzzirOTX56a6m?= =?us-ascii?Q?oxjcfruPQn62iTpt9t8WvnaoliBl4zmQeOyWFbGdIS4eXXtVYmhH3UGAbNI2?= =?us-ascii?Q?j/GROfpwZRDpHQoEpJVLNyt1bB0FcKodtZygoTfzDnzaFvS5RNUN9kwGk7Qk?= =?us-ascii?Q?93xTzSs2LQvriQlADtpI1m6l4tSPeC5elkKFmTiMG9IdXtQCepKaHPN/Cvs2?= =?us-ascii?Q?cfsMlV2a8wNWDHof/+O9qF75A+bypoQJhH2vwElw7SXZCkt2ZWwYb78YKjar?= =?us-ascii?Q?Dbjesg2/94jQd1QlR2PYWqhEt9PD0NhJjITzkN3J3SkKWSFam+U81nwhzqwE?= =?us-ascii?Q?5EJhLolLifgRNMlrCtCK7+c2ASj+heAMlMi38uJy9SrFUTi6eVg87yNE89Cq?= =?us-ascii?Q?3KFnR77+s1bw0acGMSDMfx2hQ3DM3Ch7iTp+R63G/yVflHMQj3g128Mz+07y?= =?us-ascii?Q?AZ84PAPLrQ0IhlAbgGMPt6cu6ruYwhRBFN8ZDyOisy3H08j2NzdOTJcdulok?= =?us-ascii?Q?TqSzA1/i6NwMCodHe7KNU4YEFf4XcE5A/vRgWBLZZkTIehPYuoKLCQ6/LbMh?= =?us-ascii?Q?QjbXW5+WWeyc7aY7efAuw8WWh1pf8BIAKRkjfPqFhFnnZP1WFVsrk8tcme2P?= =?us-ascii?Q?cs3MlqzoIi0NeMqPqWkj4ZJSvEDVt7YTg9RspOvyi6GTFBwudJlGZk/zb4r3?= =?us-ascii?Q?Ub8RGbNTC756NyArLKEToG2LXUfYmmubHkL/Ynx2xVAVF2JPVNVh7oeGYv4X?= =?us-ascii?Q?9UM1LGUkisZZiPQWvyJduwnjF3K0N69+Bj+zg9htyVl0ZFFWzeDpwmNFUdyQ?= =?us-ascii?Q?xO9Ucy7ioRlqyHgxUGLF3DxYZUJ2fRF9Y7DEk+lr/SJNOd9emyZcwirFvBxR?= =?us-ascii?Q?uLuZavj5iuDcYFcTD4HrtlXvbFvEU3fT9NeafB00?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 98d02af0-095c-427c-7520-08de0149fd27 X-MS-Exchange-CrossTenant-AuthSource: PH7PR12MB6834.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Oct 2025 00:24:02.6220 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: F4/ZMNCWFRvu22BnAVgwILIq5w9VHdyyhkSy48VRW6RvIm/c8I8Z/NRiP4LD5XSkNfGtk/dfFZWEkVSP62nBdA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB6232 Message-ID-Hash: NAMAAPUGEF6ZB2DKVI52C7WGEXL4OWSS X-Message-ID-Hash: NAMAAPUGEF6ZB2DKVI52C7WGEXL4OWSS X-MailFrom: nathanc@nvidia.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; header-match-devel.lists.libvirt.org-0; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: shameerali.kolothum.thodi@huawei.com, nicolinc@nvidia.com, nathanc@nvidia.com, mochs@nvidia.com X-Mailman-Version: 3.3.10 Precedence: list List-Id: Development discussions about the libvirt library & tools Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: From: Nathan Chen via Devel Reply-To: Nathan Chen X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1759365012138116600 Content-Type: text/plain; charset="utf-8" Add support for parsing multiple IOMMU devices from the VM definition when "smmuv3Dev" is the IOMMU model. Signed-off-by: Nathan Chen --- src/conf/domain_conf.c | 84 +++++++++++++---- src/conf/domain_conf.h | 9 +- src/conf/domain_validate.c | 32 ++++--- src/conf/schemas/domaincommon.rng | 4 +- src/libvirt_private.syms | 2 + src/qemu/qemu_alias.c | 15 ++- src/qemu/qemu_command.c | 146 ++++++++++++++++-------------- src/qemu/qemu_domain_address.c | 35 +++---- src/qemu/qemu_driver.c | 8 +- src/qemu/qemu_postparse.c | 11 ++- src/qemu/qemu_validate.c | 2 +- 11 files changed, 215 insertions(+), 133 deletions(-) diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index 6d1adb831d..1c2cf9a2d9 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -4134,7 +4134,8 @@ void virDomainDefFree(virDomainDef *def) virDomainCryptoDefFree(def->cryptos[i]); g_free(def->cryptos); =20 - virDomainIOMMUDefFree(def->iommu); + for (i =3D 0; i < def->niommus; i++) + virDomainIOMMUDefFree(def->iommu[i]); =20 virDomainPstoreDefFree(def->pstore); =20 @@ -5006,9 +5007,9 @@ virDomainDeviceInfoIterateFlags(virDomainDef *def, } =20 device.type =3D VIR_DOMAIN_DEVICE_IOMMU; - if (def->iommu) { - device.data.iommu =3D def->iommu; - if ((rc =3D cb(def, &device, &def->iommu->info, opaque)) !=3D 0) + for (i =3D 0; i < def->niommus; i++) { + device.data.iommu =3D def->iommu[i]; + if ((rc =3D cb(def, &device, &def->iommu[i]->info, opaque)) !=3D 0) return rc; } =20 @@ -16487,6 +16488,43 @@ virDomainInputDefFind(const virDomainDef *def, } =20 =20 +bool +virDomainIOMMUDefEquals(const virDomainIOMMUDef *a, + const virDomainIOMMUDef *b) +{ + if (a->model !=3D b->model || + a->intremap !=3D b->intremap || + a->caching_mode !=3D b->caching_mode || + a->eim !=3D b->eim || + a->iotlb !=3D b->iotlb || + a->aw_bits !=3D b->aw_bits || + a->parent_idx !=3D b->parent_idx || + a->dma_translation !=3D b->dma_translation) + return false; + + if (a->info.type !=3D VIR_DOMAIN_DEVICE_ADDRESS_TYPE_NONE && + !virDomainDeviceInfoAddressIsEqual(&a->info, &b->info)) + return false; + + return true; +} + + +ssize_t +virDomainIOMMUDefFind(const virDomainDef *def, + const virDomainIOMMUDef *iommu) +{ + size_t i; + + for (i =3D 0; i < def->niommus; i++) { + if (virDomainIOMMUDefEquals(iommu, def->iommu[i])) + return i; + } + + return -1; +} + + bool virDomainVsockDefEquals(const virDomainVsockDef *a, const virDomainVsockDef *b) @@ -20154,19 +20192,28 @@ virDomainDefParseXML(xmlXPathContextPtr ctxt, } VIR_FREE(nodes); =20 + /* analysis of iommu devices */ if ((n =3D virXPathNodeSet("./devices/iommu", ctxt, &nodes)) < 0) return NULL; =20 - if (n > 1) { + if (n > 1 && !virXPathBoolean("./devices/iommu/@model =3D 'smmuv3Dev'"= , ctxt)) { virReportError(VIR_ERR_XML_ERROR, "%s", - _("only a single IOMMU device is supported")); + _("multiple IOMMU devices are only supported with m= odel smmuv3Dev")); return NULL; } =20 - if (n > 0) { - if (!(def->iommu =3D virDomainIOMMUDefParseXML(xmlopt, nodes[0], - ctxt, flags))) + if (n > 0) + def->iommu =3D g_new0(virDomainIOMMUDef *, n); + + for (i =3D 0; i < n; i++) { + virDomainIOMMUDef *iommu; + + iommu =3D virDomainIOMMUDefParseXML(xmlopt, nodes[i], ctxt, flags); + + if (!iommu) return NULL; + + def->iommu[def->niommus++] =3D iommu; } VIR_FREE(nodes); =20 @@ -22619,15 +22666,17 @@ virDomainDefCheckABIStabilityFlags(virDomainDef *= src, goto error; } =20 - if (!!src->iommu !=3D !!dst->iommu) { - virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", - _("Target domain IOMMU device count does not match = source")); + if (src->niommus !=3D dst->niommus) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, + _("Target domain IOMMU device count %1$zu does not = match source %2$zu"), + dst->niommus, src->niommus); goto error; } =20 - if (src->iommu && - !virDomainIOMMUDefCheckABIStability(src->iommu, dst->iommu)) - goto error; + for (i =3D 0; i < src->niommus; i++) { + if (!virDomainIOMMUDefCheckABIStability(src->iommu[i], dst->iommu[= i])) + goto error; + } =20 if (!!src->vsock !=3D !!dst->vsock) { virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", @@ -29464,8 +29513,9 @@ virDomainDefFormatInternalSetRootName(virDomainDef = *def, for (n =3D 0; n < def->ncryptos; n++) { virDomainCryptoDefFormat(buf, def->cryptos[n], flags); } - if (def->iommu) - virDomainIOMMUDefFormat(buf, def->iommu); + + for (n =3D 0; n < def->niommus; n++) + virDomainIOMMUDefFormat(buf, def->iommu[n]); =20 if (def->vsock) virDomainVsockDefFormat(buf, def->vsock); diff --git a/src/conf/domain_conf.h b/src/conf/domain_conf.h index 1d0c94a00a..f830fe5226 100644 --- a/src/conf/domain_conf.h +++ b/src/conf/domain_conf.h @@ -3297,6 +3297,9 @@ struct _virDomainDef { size_t nwatchdogs; virDomainWatchdogDef **watchdogs; =20 + size_t niommus; + virDomainIOMMUDef **iommu; + /* At maximum 2 TPMs on the domain if a TPM Proxy is present. */ size_t ntpms; virDomainTPMDef **tpms; @@ -3306,7 +3309,6 @@ struct _virDomainDef { virDomainNVRAMDef *nvram; virCPUDef *cpu; virDomainRedirFilterDef *redirfilter; - virDomainIOMMUDef *iommu; virDomainVsockDef *vsock; virDomainPstoreDef *pstore; =20 @@ -4311,6 +4313,11 @@ virDomainShmemDef *virDomainShmemDefRemove(virDomain= Def *def, size_t idx) ssize_t virDomainInputDefFind(const virDomainDef *def, const virDomainInputDef *input) ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(2) G_GNUC_WARN_UNUSED_RESULT; +bool virDomainIOMMUDefEquals(const virDomainIOMMUDef *a, + const virDomainIOMMUDef *b); +ssize_t virDomainIOMMUDefFind(const virDomainDef *def, + const virDomainIOMMUDef *iommu) + ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(2) G_GNUC_WARN_UNUSED_RESULT; bool virDomainVsockDefEquals(const virDomainVsockDef *a, const virDomainVsockDef *b) ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(2) G_GNUC_WARN_UNUSED_RESULT; diff --git a/src/conf/domain_validate.c b/src/conf/domain_validate.c index 8a599fdbc6..588ecdaa20 100644 --- a/src/conf/domain_validate.c +++ b/src/conf/domain_validate.c @@ -1851,21 +1851,31 @@ virDomainDefCputuneValidate(const virDomainDef *def) static int virDomainDefIOMMUValidate(const virDomainDef *def) { + size_t i; + if (!def->iommu) return 0; =20 - if (def->iommu->intremap =3D=3D VIR_TRISTATE_SWITCH_ON && - def->features[VIR_DOMAIN_FEATURE_IOAPIC] !=3D VIR_DOMAIN_IOAPIC_QE= MU) { - virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", - _("IOMMU interrupt remapping requires split I/O API= C (ioapic driver=3D'qemu')")); - return -1; - } + for (i =3D 0; i < def->niommus; i++) { + virDomainIOMMUDef *iommu =3D def->iommu[i]; + if (def->niommus > 1 && iommu->model !=3D VIR_DOMAIN_IOMMU_MODEL_S= MMUV3_DEV) { + virReportError(VIR_ERR_XML_ERROR, "%s", + _("IOMMU model smmuv3Dev must be specified for = multiple IOMMU definitions")); + } =20 - if (def->iommu->eim =3D=3D VIR_TRISTATE_SWITCH_ON && - def->iommu->intremap !=3D VIR_TRISTATE_SWITCH_ON) { - virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", - _("IOMMU eim requires interrupt remapping to be ena= bled")); - return -1; + if (iommu->intremap =3D=3D VIR_TRISTATE_SWITCH_ON && + def->features[VIR_DOMAIN_FEATURE_IOAPIC] !=3D VIR_DOMAIN_IOAPI= C_QEMU) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", + _("IOMMU interrupt remapping requires split I/O= APIC (ioapic driver=3D'qemu')")); + return -1; + } + + if (iommu->eim =3D=3D VIR_TRISTATE_SWITCH_ON && + iommu->intremap !=3D VIR_TRISTATE_SWITCH_ON) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", + _("IOMMU eim requires interrupt remapping to be= enabled")); + return -1; + } } =20 return 0; diff --git a/src/conf/schemas/domaincommon.rng b/src/conf/schemas/domaincom= mon.rng index 26880b6db2..b9c00e98a5 100644 --- a/src/conf/schemas/domaincommon.rng +++ b/src/conf/schemas/domaincommon.rng @@ -6959,9 +6959,9 @@ - + - + diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index fe72402527..34d9c263d4 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -491,6 +491,8 @@ virDomainInputSourceGrabToggleTypeToString; virDomainInputSourceGrabTypeFromString; virDomainInputSourceGrabTypeToString; virDomainInputTypeToString; +virDomainIOMMUDefEquals; +virDomainIOMMUDefFind; virDomainIOMMUDefFree; virDomainIOMMUDefNew; virDomainIOMMUModelTypeFromString; diff --git a/src/qemu/qemu_alias.c b/src/qemu/qemu_alias.c index a27c688d79..5f2b11b9a6 100644 --- a/src/qemu/qemu_alias.c +++ b/src/qemu/qemu_alias.c @@ -647,10 +647,14 @@ qemuAssignDeviceVsockAlias(virDomainVsockDef *vsock) =20 =20 static void -qemuAssignDeviceIOMMUAlias(virDomainIOMMUDef *iommu) +qemuAssignDeviceIOMMUAlias(virDomainDef *def, + virDomainIOMMUDef **iommu) { - if (!iommu->info.alias) - iommu->info.alias =3D g_strdup("iommu0"); + size_t i; + for (i =3D 0; i < def->niommus; i++) { + if (!iommu[i]->info.alias) + iommu[i]->info.alias =3D g_strdup_printf("iommu%zu", i); + } } =20 =20 @@ -766,8 +770,9 @@ qemuAssignDeviceAliases(virDomainDef *def) if (def->vsock) { qemuAssignDeviceVsockAlias(def->vsock); } - if (def->iommu) - qemuAssignDeviceIOMMUAlias(def->iommu); + if (def->iommu && def->niommus > 0) { + qemuAssignDeviceIOMMUAlias(def, def->iommu); + } for (i =3D 0; i < def->ncryptos; i++) { qemuAssignDeviceCryptoAlias(def, def->cryptos[i]); } diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c index e789e8cf2c..15ae919047 100644 --- a/src/qemu/qemu_command.c +++ b/src/qemu/qemu_command.c @@ -6296,10 +6296,12 @@ qemuBuildBootCommandLine(virCommand *cmd, =20 static virJSONValue * qemuBuildPCISmmuv3DevDevProps(const virDomainDef *def, - const virDomainIOMMUDef *iommu) + const virDomainIOMMUDef *iommu, + size_t id) { g_autoptr(virJSONValue) props =3D NULL; g_autofree char *bus =3D NULL; + g_autofree char *smmuv3_id =3D NULL; size_t i; bool contIsPHB =3D false; =20 @@ -6340,9 +6342,12 @@ qemuBuildPCISmmuv3DevDevProps(const virDomainDef *de= f, bus =3D temp_bus; } =20 + smmuv3_id =3D g_strdup_printf("smmuv3.%zu", id); + if (virJSONValueObjectAdd(&props, "s:driver", "arm-smmuv3", "s:primary-bus", bus, + "s:id", smmuv3_id, NULL) < 0) return NULL; =20 @@ -6355,91 +6360,92 @@ qemuBuildIOMMUCommandLine(virCommand *cmd, const virDomainDef *def, virQEMUCaps *qemuCaps) { + size_t i; g_autoptr(virJSONValue) props =3D NULL; g_autoptr(virJSONValue) wrapperProps =3D NULL; - const virDomainIOMMUDef *iommu =3D def->iommu; - - if (!iommu) + if (!def->iommu || def->niommus <=3D 0) return 0; =20 - switch (iommu->model) { - case VIR_DOMAIN_IOMMU_MODEL_INTEL: - if (virJSONValueObjectAdd(&props, - "s:driver", "intel-iommu", - "s:id", iommu->info.alias, - "S:intremap", qemuOnOffAuto(iommu->intre= map), - "T:caching-mode", iommu->caching_mode, - "S:eim", qemuOnOffAuto(iommu->eim), - "T:device-iotlb", iommu->iotlb, - "z:aw-bits", iommu->aw_bits, - "T:dma-translation", iommu->dma_translat= ion, - NULL) < 0) - return -1; + for (i =3D 0; i < def->niommus; i++) { + virDomainIOMMUDef *iommu =3D def->iommu[i]; + switch (iommu->model) { + case VIR_DOMAIN_IOMMU_MODEL_INTEL: + if (virJSONValueObjectAdd(&props, + "s:driver", "intel-iommu", + "s:id", iommu->info.alias, + "S:intremap", qemuOnOffAuto(iommu->i= ntremap), + "T:caching-mode", iommu->caching_mod= e, + "S:eim", qemuOnOffAuto(iommu->eim), + "T:device-iotlb", iommu->iotlb, + "z:aw-bits", iommu->aw_bits, + "T:dma-translation", iommu->dma_tran= slation, + NULL) < 0) + return -1; =20 - if (qemuBuildDeviceCommandlineFromJSON(cmd, props, def, qemuCaps) = < 0) - return -1; + if (qemuBuildDeviceCommandlineFromJSON(cmd, props, def, qemuCa= ps) < 0) + return -1; =20 - return 0; + return 0; + case VIR_DOMAIN_IOMMU_MODEL_VIRTIO: + if (virJSONValueObjectAdd(&props, + "s:driver", "virtio-iommu", + "s:id", iommu->info.alias, + NULL) < 0) { + return -1; + } =20 - case VIR_DOMAIN_IOMMU_MODEL_VIRTIO: - if (virJSONValueObjectAdd(&props, - "s:driver", "virtio-iommu", - "s:id", iommu->info.alias, - NULL) < 0) { - return -1; - } + if (qemuBuildDeviceAddressProps(props, def, &iommu->info) < 0) + return -1; =20 - if (qemuBuildDeviceAddressProps(props, def, &iommu->info) < 0) - return -1; + if (qemuBuildDeviceCommandlineFromJSON(cmd, props, def, qemuCa= ps) < 0) + return -1; =20 - if (qemuBuildDeviceCommandlineFromJSON(cmd, props, def, qemuCaps) = < 0) - return -1; + return 0; + case VIR_DOMAIN_IOMMU_MODEL_SMMUV3: + /* There is no -device for SMMUv3, so nothing to be done here = */ + return 0; =20 - return 0; + case VIR_DOMAIN_IOMMU_MODEL_AMD: + if (virJSONValueObjectAdd(&wrapperProps, + "s:driver", "AMDVI-PCI", + "s:id", iommu->info.alias, + NULL) < 0) + return -1; =20 - case VIR_DOMAIN_IOMMU_MODEL_SMMUV3: - return 0; + if (qemuBuildDeviceAddressProps(wrapperProps, def, &iommu->inf= o) < 0) + return -1; =20 - case VIR_DOMAIN_IOMMU_MODEL_AMD: - if (virJSONValueObjectAdd(&wrapperProps, - "s:driver", "AMDVI-PCI", - "s:id", iommu->info.alias, - NULL) < 0) - return -1; + if (qemuBuildDeviceCommandlineFromJSON(cmd, wrapperProps, def,= qemuCaps) < 0) + return -1; =20 - if (qemuBuildDeviceAddressProps(wrapperProps, def, &iommu->info) <= 0) - return -1; + if (virJSONValueObjectAdd(&props, + "s:driver", "amd-iommu", + "s:pci-id", iommu->info.alias, + "S:intremap", qemuOnOffAuto(iommu->i= ntremap), + "T:pt", iommu->pt, + "T:xtsup", iommu->xtsup, + "T:device-iotlb", iommu->iotlb, + NULL) < 0) + return -1; =20 - if (qemuBuildDeviceCommandlineFromJSON(cmd, wrapperProps, def, qem= uCaps) < 0) - return -1; + if (qemuBuildDeviceCommandlineFromJSON(cmd, props, def, qemuCa= ps) < 0) + return -1; =20 - if (virJSONValueObjectAdd(&props, - "s:driver", "amd-iommu", - "s:pci-id", iommu->info.alias, - "S:intremap", qemuOnOffAuto(iommu->intre= map), - "T:pt", iommu->pt, - "T:xtsup", iommu->xtsup, - "T:device-iotlb", iommu->iotlb, - NULL) < 0) - return -1; + return 0; =20 - if (qemuBuildDeviceCommandlineFromJSON(cmd, props, def, qemuCaps) = < 0) - return -1; + case VIR_DOMAIN_IOMMU_MODEL_SMMUV3_DEV: + if (!(props =3D qemuBuildPCISmmuv3DevDevProps(def, iommu, i))) + return -1; + if (qemuBuildDeviceCommandlineFromJSON(cmd, props, def, qemuCa= ps) < 0) + return -1; + break; =20 - return 0; =20 - case VIR_DOMAIN_IOMMU_MODEL_SMMUV3_DEV: - if (!(props =3D qemuBuildPCISmmuv3DevDevProps(def, iommu))) - return -1; - if (qemuBuildDeviceCommandlineFromJSON(cmd, props, def, qemuCaps) = < 0) + case VIR_DOMAIN_IOMMU_MODEL_LAST: + default: + virReportEnumRangeError(virDomainIOMMUModel, iommu->model); return -1; - - return 0; - - case VIR_DOMAIN_IOMMU_MODEL_LAST: - default: - virReportEnumRangeError(virDomainIOMMUModel, iommu->model); - return -1; + } } =20 return 0; @@ -7260,8 +7266,8 @@ qemuBuildMachineCommandLine(virCommand *cmd, if (qemuAppendDomainFeaturesMachineParam(&buf, def, qemuCaps) < 0) return -1; =20 - if (def->iommu) { - switch (def->iommu->model) { + if (def->iommu && def->niommus =3D=3D 1) { + switch (def->iommu[0]->model) { case VIR_DOMAIN_IOMMU_MODEL_SMMUV3: virBufferAddLit(&buf, ",iommu=3Dsmmuv3"); break; @@ -7275,7 +7281,7 @@ qemuBuildMachineCommandLine(virCommand *cmd, =20 case VIR_DOMAIN_IOMMU_MODEL_LAST: default: - virReportEnumRangeError(virDomainIOMMUModel, def->iommu->model= ); + virReportEnumRangeError(virDomainIOMMUModel, def->iommu[0]->mo= del); return -1; } } diff --git a/src/qemu/qemu_domain_address.c b/src/qemu/qemu_domain_address.c index 06bf4fab32..2ddc629304 100644 --- a/src/qemu/qemu_domain_address.c +++ b/src/qemu/qemu_domain_address.c @@ -2365,24 +2365,25 @@ qemuDomainAssignDevicePCISlots(virDomainDef *def, /* Nada - none are PCI based (yet) */ } =20 - if (def->iommu) { - virDomainIOMMUDef *iommu =3D def->iommu; - - switch (iommu->model) { - case VIR_DOMAIN_IOMMU_MODEL_VIRTIO: - case VIR_DOMAIN_IOMMU_MODEL_AMD: - if (virDeviceInfoPCIAddressIsWanted(&iommu->info) && - qemuDomainPCIAddressReserveNextAddr(addrs, &iommu->info) <= 0) { - return -1; - } - break; + if (def->iommu && def->niommus > 0) { + for (i =3D 0; i < def->niommus; i++) { + virDomainIOMMUDef *iommu =3D def->iommu[i]; + switch (iommu->model) { + case VIR_DOMAIN_IOMMU_MODEL_VIRTIO: + case VIR_DOMAIN_IOMMU_MODEL_AMD: + if (virDeviceInfoPCIAddressIsWanted(&iommu->info) && + qemuDomainPCIAddressReserveNextAddr(addrs, &iommu->inf= o) < 0) { + return -1; + } + break; =20 - case VIR_DOMAIN_IOMMU_MODEL_INTEL: - case VIR_DOMAIN_IOMMU_MODEL_SMMUV3: - case VIR_DOMAIN_IOMMU_MODEL_SMMUV3_DEV: - case VIR_DOMAIN_IOMMU_MODEL_LAST: - /* These are not PCI devices */ - break; + case VIR_DOMAIN_IOMMU_MODEL_INTEL: + case VIR_DOMAIN_IOMMU_MODEL_SMMUV3: + case VIR_DOMAIN_IOMMU_MODEL_SMMUV3_DEV: + case VIR_DOMAIN_IOMMU_MODEL_LAST: + /* These are not PCI devices */ + break; + } } } =20 diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index ac72ea5cb0..3d65f78c9e 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -6894,12 +6894,12 @@ qemuDomainAttachDeviceConfig(virDomainDef *vmdef, break; =20 case VIR_DOMAIN_DEVICE_IOMMU: - if (vmdef->iommu) { + if (vmdef->iommu && vmdef->niommus > 0) { virReportError(VIR_ERR_OPERATION_INVALID, "%s", _("domain already has an iommu device")); return -1; } - vmdef->iommu =3D g_steal_pointer(&dev->data.iommu); + VIR_APPEND_ELEMENT(vmdef->iommu, vmdef->niommus, dev->data.iommu); break; =20 case VIR_DOMAIN_DEVICE_VIDEO: @@ -7113,12 +7113,12 @@ qemuDomainDetachDeviceConfig(virDomainDef *vmdef, break; =20 case VIR_DOMAIN_DEVICE_IOMMU: - if (!vmdef->iommu) { + if ((idx =3D virDomainIOMMUDefFind(vmdef, dev->data.iommu)) < 0) { virReportError(VIR_ERR_OPERATION_FAILED, "%s", _("matching iommu device not found")); return -1; } - g_clear_pointer(&vmdef->iommu, virDomainIOMMUDefFree); + VIR_DELETE_ELEMENT(vmdef->iommu, idx, vmdef->niommus); break; =20 case VIR_DOMAIN_DEVICE_VIDEO: diff --git a/src/qemu/qemu_postparse.c b/src/qemu/qemu_postparse.c index 9c2427970d..e2744a4a61 100644 --- a/src/qemu/qemu_postparse.c +++ b/src/qemu/qemu_postparse.c @@ -1503,7 +1503,7 @@ qemuDomainDefAddDefaultDevices(virQEMUDriver *driver, } } =20 - if (addIOMMU && !def->iommu && + if (addIOMMU && !def->iommu && def->niommus =3D=3D 0 && virQEMUCapsGet(qemuCaps, QEMU_CAPS_DEVICE_INTEL_IOMMU) && virQEMUCapsGet(qemuCaps, QEMU_CAPS_INTEL_IOMMU_INTREMAP) && virQEMUCapsGet(qemuCaps, QEMU_CAPS_INTEL_IOMMU_EIM)) { @@ -1515,7 +1515,8 @@ qemuDomainDefAddDefaultDevices(virQEMUDriver *driver, iommu->intremap =3D VIR_TRISTATE_SWITCH_ON; iommu->eim =3D VIR_TRISTATE_SWITCH_ON; =20 - def->iommu =3D g_steal_pointer(&iommu); + def->iommu =3D g_new0(virDomainIOMMUDef *, 1); + def->iommu[def->niommus++] =3D g_steal_pointer(&iommu); } =20 if (qemuDomainDefAddDefaultAudioBackend(driver, def) < 0) @@ -1591,9 +1592,9 @@ qemuDomainDefEnableDefaultFeatures(virDomainDef *def, * domain already has IOMMU without inremap. This will be fixed in * qemuDomainIOMMUDefPostParse() but there domain definition can't be * modified so change it now. */ - if (def->iommu && - (def->iommu->intremap =3D=3D VIR_TRISTATE_SWITCH_ON || - qemuDomainNeedsIOMMUWithEIM(def)) && + if (def->iommu && def->niommus =3D=3D 1 && + (def->iommu[0]->intremap =3D=3D VIR_TRISTATE_SWITCH_ON || + qemuDomainNeedsIOMMUWithEIM(def)) && def->features[VIR_DOMAIN_FEATURE_IOAPIC] =3D=3D VIR_DOMAIN_IOAPIC_= NONE) { def->features[VIR_DOMAIN_FEATURE_IOAPIC] =3D VIR_DOMAIN_IOAPIC_QEM= U; } diff --git a/src/qemu/qemu_validate.c b/src/qemu/qemu_validate.c index aac004c544..bdaeefe976 100644 --- a/src/qemu/qemu_validate.c +++ b/src/qemu/qemu_validate.c @@ -851,7 +851,7 @@ qemuValidateDomainVCpuTopology(const virDomainDef *def,= virQEMUCaps *qemuCaps) QEMU_MAX_VCPUS_WITHOUT_EIM); return -1; } - if (!def->iommu || def->iommu->eim !=3D VIR_TRISTATE_SWITCH_ON) { + if (!def->iommu || def->iommu[0]->eim !=3D VIR_TRISTATE_SWITCH_ON)= { virReportError(VIR_ERR_CONFIG_UNSUPPORTED, _("more than %1$d vCPUs require extended interr= upt mode enabled on the iommu device"), QEMU_MAX_VCPUS_WITHOUT_EIM); --=20 2.43.0