From nobody Sun Oct 5 01:49:26 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.libvirt.org designates 8.43.85.245 as permitted sender) client-ip=8.43.85.245; envelope-from=devel-bounces@lists.libvirt.org; helo=lists.libvirt.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of lists.libvirt.org designates 8.43.85.245 as permitted sender) smtp.mailfrom=devel-bounces@lists.libvirt.org; arc=fail (Bad Signature); dmarc=pass(p=reject dis=none) header.from=lists.libvirt.org Return-Path: Received: from lists.libvirt.org (lists.libvirt.org [8.43.85.245]) by mx.zohomail.com with SMTPS id 1759366004233534.9187747494935; Wed, 1 Oct 2025 17:46:44 -0700 (PDT) Received: by lists.libvirt.org (Postfix, from userid 993) id B377D41C5C; Wed, 1 Oct 2025 20:46:42 -0400 (EDT) Received: from [172.19.199.14] (lists.libvirt.org [8.43.85.245]) by lists.libvirt.org (Postfix) with ESMTP id F1899442BF; Wed, 1 Oct 2025 20:39:20 -0400 (EDT) Received: by lists.libvirt.org (Postfix, from userid 993) id DA90943E49; Wed, 1 Oct 2025 20:38:56 -0400 (EDT) Received: from SN4PR0501CU005.outbound.protection.outlook.com (mail-southcentralusazon11011053.outbound.protection.outlook.com [40.93.194.53]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (prime256v1) server-signature RSA-PSS (3072 bits) server-digest SHA256) (No client certificate requested) by lists.libvirt.org (Postfix) with ESMTPS id BC7B343E5D for ; Wed, 1 Oct 2025 20:38:54 -0400 (EDT) Received: from PH7PR12MB6834.namprd12.prod.outlook.com (2603:10b6:510:1b4::18) by PH7PR12MB9152.namprd12.prod.outlook.com (2603:10b6:510:2ec::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9160.18; Thu, 2 Oct 2025 00:38:47 +0000 Received: from PH7PR12MB6834.namprd12.prod.outlook.com ([fe80::f432:162b:b94e:d2cb]) by PH7PR12MB6834.namprd12.prod.outlook.com ([fe80::f432:162b:b94e:d2cb%6]) with mapi id 15.20.9160.015; Thu, 2 Oct 2025 00:38:47 +0000 X-Spam-Checker-Version: SpamAssassin 4.0.1 (2024-03-26) on lists.libvirt.org X-Spam-Level: X-Spam-Status: No, score=-3.1 required=5.0 tests=ARC_SIGNED,ARC_VALID, DKIM_INVALID,DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED,RCVD_IN_VALIDITY_RPBL_BLOCKED, RCVD_IN_VALIDITY_SAFE_BLOCKED,SPF_PASS autolearn=unavailable autolearn_force=no version=4.0.1 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=Cv3Qbk82quyv3Oqf3vBaelCmw4i5nX1tFavzmXkEzHqJYRmUsnzSJKEYbfdDvo7MGFd6pRiGJabjEwpeveEdp8Co1wGozRBSNqZdG1bHcxHyZbdPMYkPx/ygWuOzD0gUV+gos1ygb9Cb/DmE0GYgVe0rEGRIjAvtoiUsDgbi2xmHPyDtbjl1e8a99N0wcO92efXuAhfSmevNgssizOOoWF2+tFZohhhTL8QrMP3Gve4E0IK8yQP3PJauTiiPfcCkBHFSc0w2CDIOgGc+e4sTwyfBbcvf7i4qexrg1BbwDw6z08F1mzwaI284P1IchbrHMNs1BZz/rIG2c4xgFWRM0w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=RZ2pGi7PjSAKnvPoSsjBhCu0xY+MZ07qgkwaNFmCS1M=; b=j/yMGhOAtKVkngMqBKIcwrUTgHHUtCpoxxO++eu5uagvOV98ZRBoIEr6oMHYSb3B2yDMi8MhuHRfROvPlTxC5WbhAaYinxzDOMXveh1c9/kwJcN5SEOWbjMWoBya7Lb31MGkwkhWjGgJfLSwboOEqvvDZ31wvxtF5crL9xzpBmpFLXDn33ZDK8rti1/kPkviEE4q4uMh2yOEoLwFokjXICfdKLAi3U7I4NCnEsxQBJLqI3842dm3cTRwsIEsY5tXkp2VZ8qdlE5M9rp07lvbVJldIOYLwmoLeLTWaBIyU+1BayV4G8z6tAuShTruEc/gOlpQl9VY43JM/CPXkcL7XQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=RZ2pGi7PjSAKnvPoSsjBhCu0xY+MZ07qgkwaNFmCS1M=; b=Q4+8TNaNXNeO9ksNFfdv4Hrc7C+/BtXtaP6szqZlhlFD4Aot95/8M1+fLHYzUyslYV1FCohz6Wwju679Opg4WHsUnJVTA3+OUGcNsTWSFojDcmpZV05exriTsZbXbgcN5x3qQttjRGVbKI00jLD/GN5EpQXtCx2jxf1NwDAXR/B7rhp/lXrsGeArO4JVNnur9nXGdQShfsblP4ayeqjPKbY3LNBgxHNowowTlEXkdiU4Vl6H3MnkcQcw3OpmpvgZf+L57HnGLKmnb0kkilZ+GO4du6OqgFyv6e9LXCmUASOHfC8ahgG2jwNKxRWNP6Gkb9Vq2g1gpzGZcBF3vnWF2w== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; To: devel@lists.libvirt.org Subject: [RFC PATCH v5 2/7] conf: Support multiple smmuv3Dev IOMMU devices Date: Wed, 1 Oct 2025 17:38:32 -0700 Message-ID: <20251002003837.1546646-3-nathanc@nvidia.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251002003837.1546646-1-nathanc@nvidia.com> References: <20251002003837.1546646-1-nathanc@nvidia.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: BYAPR01CA0001.prod.exchangelabs.com (2603:10b6:a02:80::14) To PH7PR12MB6834.namprd12.prod.outlook.com (2603:10b6:510:1b4::18) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: PH7PR12MB6834:EE_|PH7PR12MB9152:EE_ X-MS-Office365-Filtering-Correlation-Id: 0cb6642d-00c4-4324-ebd0-08de014c0ca6 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|376014|366016; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?cJL4+P25D5232WJEHAnphhv22K7ciXQytZfnEdWdXGIE7ce3LzsDYstadQX3?= =?us-ascii?Q?6OhqnyrNGnDw6LreHPSPq1HVnKvK67jWa3QV8Fx+We4bdT8/N/CmweLCX/2I?= =?us-ascii?Q?+v23/k25ddtfXBOW39Wj2CZhPJdXovHfF9+Fdr5uCBky16vYQZMDiuuTDlDY?= =?us-ascii?Q?FOnSxTknXyEM0gTUCD2gIl9j9Em5pWo6m8lwon8StIKDUOfJpifLkBgu1q8f?= =?us-ascii?Q?khg76t22Y6eK9Kg01VnqMs8C+XTKKCA8XwtaHe7rpOosvicFtkElr0b44H+/?= =?us-ascii?Q?0Fx9XpoS+g3KT5LkR5tY/w80JdxeOdo3WKMvwpojXtcRvVvZ/+ZqSmxe1nfF?= =?us-ascii?Q?BYjGQ2CuswKAcg18vJZq0pHOrOLv0vJjTICWgZgrwuD4ByRtezyceigwZdeO?= =?us-ascii?Q?OfKzL/SwCx9INMwptYFgTUohJk5GKKApktLqpdUYfSeM3sXviK9GhQxJT8UX?= =?us-ascii?Q?+sTVEY1HZSy4UfElk83lMxxhpTpn/ECOSjK/9m9gZwaC8UMbN+V4DBDbKeLc?= =?us-ascii?Q?byX6SxjTZFTTn8mREzqEGH7MabqPG+gsRkmD7BIi0fbkEHIuF3TQv/5B95bh?= =?us-ascii?Q?Ad9F4908Vw9IWyDz2GO4OtSvt1U/376qQEhDbE1uzT+mhOHzSgemV9/4RbTy?= =?us-ascii?Q?+Lclf2PO0V6q72VrfDnv82QsCoE8jI3WZgRHo/pB87BUwGvrKKJEm5oIlgL0?= =?us-ascii?Q?SMSXQx/kTq8bq1wSy6FW3fVyNwR1pBNDKsHZ9Vc2YXTltMYncQAfZA5W9O1K?= =?us-ascii?Q?KjtKrXrMOoVn0X2Q4+iQ/wFHXpD81TLYdDaIlyXgsevAT+5ZodWXue5MZz+l?= =?us-ascii?Q?umRPsZwrXFzxe0+hSxF9cO374jIKn+kMBe8A02G1jr8FzRR63JzR5CTwKhBA?= =?us-ascii?Q?opjd0oUve0ioFuXGyI5c4DcmuYFTfwKpcOt02jfxVl9Tb9BCG+90DIzIe6pc?= =?us-ascii?Q?i/bBh+SC8p/fukJjPeoXEpC1Y6RkH3VF7HY/6erglBFYM5l7G21pOKZZ+Cvu?= =?us-ascii?Q?AEp3ZzbKKfvbttS4dLylxn8/CPFFY75YXL4dlE0O9LGMhqGG7k6Cn4Nws8VZ?= =?us-ascii?Q?SS70HC4qgClF/6Jbqo2+cZao2ZDcNBsWFSYMcbz2rfXNVN+a5PFLsUefGrpH?= =?us-ascii?Q?zKu4YiKIadccyqefxSTtPJzij5K/zBQu4JTrjjAtfR7vazII/bxK7vY6zgSb?= =?us-ascii?Q?SjVlKs+G4ZLB1KJfz4w9PT0NELJStfbmdOwNHd2D4m+ZTBYkU5E5EWFEMeSC?= =?us-ascii?Q?HC2qSqMpdcg5mfa2R4kCEbEPRLz0+tnXQSt7/VqHODeS8BC8iQDP5F+nnijZ?= =?us-ascii?Q?sDOTQVDJBguSFoRkWp/00ZhXKuuJPS6E6nu6FkNaTzheubYCaFfhFgruGeiW?= =?us-ascii?Q?liF0azhv2La4B06ex22TfnBvgAxcBYi1ku1876OdmRI0EZUX2aNFYJbEZhbt?= =?us-ascii?Q?x2Cie6AYfAvydlffRn8bKV8/XQ5gYQXa?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PH7PR12MB6834.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(376014)(366016);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?0WtwRGWheZPRvxkeGWk5kVchNvr+ABbleAJuyH9k98NRza1FowlOPF8OlILH?= =?us-ascii?Q?w+x62P6cD/621LrLXyoKFF8MRa0xoRsQCiTsUfBDvmqT9kmTWGhfSpxb7ItC?= =?us-ascii?Q?qwL/tFfVJtyF9WW+t4jBTgdGtDs72iQWBwBvqxUj4iLWYvyMeOu0folGFIkz?= =?us-ascii?Q?I22V8I2F46CbpeyluYimJSf7cFj40rRmuyZt1co2uDxml0T6pkAAkjjwm4kc?= =?us-ascii?Q?1gThueufjLXh035b+WPWevHMa0by3tfZbxltjtVWvWOOWAIAZ5j4mgqsxKYM?= =?us-ascii?Q?bQRycdSSaq7WFuVUVwkq75JtfcM3zkdWYc8w4skrMdH6ytl+fvif/fd1l/HF?= =?us-ascii?Q?bIc1HHylhTDVtNINmKXi6miNxBaNsC5F3iTWInWZ2wArTy6ArcTwn4TTEg0M?= =?us-ascii?Q?O4pBoyTkNFY5S4xErK5WTTL7efKbSy+pAbVEpxKGASV1INLlHU30zl8KERMO?= =?us-ascii?Q?JzqyYDlSUdsS/d3S3FGr/lh0EjR4XkfQyBZ0rZVWzox/5qd4EeIUHfEGwY6z?= =?us-ascii?Q?jpu36w2d8yI4p4bQWmhuIZB84OrT2oxocBcpRk79H8Gom/KR4vNj22InVZDQ?= =?us-ascii?Q?rMDFiGUIQtAw35lw7YhfChR1DL0xPDnBPB935ctDHoSsR88Ml6trO4eTUpTl?= =?us-ascii?Q?ulO24IiwkWSn2dZN+5f+ogydUFY8ktf815wTkp8KhRF2cV+v8ywijb10EjjC?= =?us-ascii?Q?ycWpM8/dfq0Czrw6Uh68ZjLLP7k3eeKOtvTL0u4C5XppMocHb5+0eL0h8v5n?= =?us-ascii?Q?jEldHEjYXepKCPyELitkQisKIh0Xf7pyeNflB6wU615tdL4gpXzqq3X5Xqfc?= =?us-ascii?Q?ebV8BUUUaBt2FnYtZMcCYWtdv1KVU3OILXxc/wDVmVE0vkLj3IPFzAyhyK4L?= =?us-ascii?Q?ghUrjepj4P3BmPgk/Tsw714tpRXpi2FVBU/KNUlCo62VAO2EGMj4L9XR560b?= =?us-ascii?Q?5ymSc2f6f/S7qFxAlqPcH3zSoTa/cQYH97WkKlOzL3RzArAZkoPi0pFkxPTc?= =?us-ascii?Q?QZ3IeZ0Q771NW18LEg8sPLVyIAhYMbcSmC+ndkVG3qUU66DwL3OY3Zur+W5l?= =?us-ascii?Q?0xp2/lYcu4t+XA8GXXRJPMt93e8DRg5QDai/s4A79yhTDFmuMRAZYGiIaNy/?= =?us-ascii?Q?fj67OuOhplDzRamWkStuWMz+U3bipF9K0xnD80U2xfG++ojIBLUzzS/bRzGG?= =?us-ascii?Q?TDesAdze7vn1UzFBUPdN2oB4p5vN/QZd2WNUZ36yb/1eKn6C4CXtZY8zHt3v?= =?us-ascii?Q?AUE9af5Lv5f26zNyfUCn3MVJcptrGKfbtOELF8JJsZw3hwamhXWFArhQYpdM?= =?us-ascii?Q?bXqYjtCKx4s7/W2kLILgf4gjZTjNh/bpI7rJ8KAaT3jFG5AtT6yJW/qBrxdf?= =?us-ascii?Q?VZ+8vHq0m/y66PPj0FAzSREHWc07PmhP7BWjlwVV/jC+zZAzU2DS0TILfeMZ?= =?us-ascii?Q?44yCES+KW20VXc99tyiRuiKOPiJ5Ek9oSC4Vq1ulkOUs8dhUaJyF6dUkG4FG?= =?us-ascii?Q?BEWQvZOV9FgSSUDkgIaaUpZOKgNl66BtQ5MZMSv69NSvUSELhXGQ/DB2xjw3?= =?us-ascii?Q?cO2H7263tT+1GkvEltrB0pteRvlwjMGvm/IGwT1J?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 0cb6642d-00c4-4324-ebd0-08de014c0ca6 X-MS-Exchange-CrossTenant-AuthSource: PH7PR12MB6834.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Oct 2025 00:38:47.6115 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 50m51xsPLpl+ApQE8qZtdl1vuLOE+h35eOd5AwCbDPcMEZTHufuSkxabHb+fyg+oTQECe5n8hSsld1bF4Qf+XA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB9152 Message-ID-Hash: XNVOPWTK7FWDFBRBNUU4N3E7JWDBE44L X-Message-ID-Hash: XNVOPWTK7FWDFBRBNUU4N3E7JWDBE44L X-MailFrom: nathanc@nvidia.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; header-match-devel.lists.libvirt.org-0; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: shameerali.kolothum.thodi@huawei.com, nicolinc@nvidia.com, nathanc@nvidia.com, mochs@nvidia.com X-Mailman-Version: 3.3.10 Precedence: list List-Id: Development discussions about the libvirt library & tools Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: From: Nathan Chen via Devel Reply-To: Nathan Chen X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1759366004982116600 Content-Type: text/plain; charset="utf-8" Add support for parsing multiple IOMMU devices from the VM definition when "smmuv3Dev" is the IOMMU model. Signed-off-by: Nathan Chen --- src/conf/domain_conf.c | 89 ++++++++++++++---- src/conf/domain_conf.h | 9 +- src/conf/domain_validate.c | 32 ++++--- src/conf/schemas/domaincommon.rng | 4 +- src/libvirt_private.syms | 2 + src/qemu/qemu_alias.c | 15 ++- src/qemu/qemu_command.c | 148 ++++++++++++++++-------------- src/qemu/qemu_domain_address.c | 35 +++---- src/qemu/qemu_driver.c | 8 +- src/qemu/qemu_postparse.c | 11 ++- src/qemu/qemu_validate.c | 2 +- 11 files changed, 221 insertions(+), 134 deletions(-) diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index b9a1c29b73..bec4550a67 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -4134,7 +4134,8 @@ void virDomainDefFree(virDomainDef *def) virDomainCryptoDefFree(def->cryptos[i]); g_free(def->cryptos); =20 - virDomainIOMMUDefFree(def->iommu); + for (i =3D 0; i < def->niommus; i++) + virDomainIOMMUDefFree(def->iommu[i]); =20 virDomainPstoreDefFree(def->pstore); =20 @@ -5006,9 +5007,9 @@ virDomainDeviceInfoIterateFlags(virDomainDef *def, } =20 device.type =3D VIR_DOMAIN_DEVICE_IOMMU; - if (def->iommu) { - device.data.iommu =3D def->iommu; - if ((rc =3D cb(def, &device, &def->iommu->info, opaque)) !=3D 0) + for (i =3D 0; i < def->niommus; i++) { + device.data.iommu =3D def->iommu[i]; + if ((rc =3D cb(def, &device, &def->iommu[i]->info, opaque)) !=3D 0) return rc; } =20 @@ -16508,6 +16509,48 @@ virDomainInputDefFind(const virDomainDef *def, } =20 =20 +bool +virDomainIOMMUDefEquals(const virDomainIOMMUDef *a, + const virDomainIOMMUDef *b) +{ + if (a->model !=3D b->model || + a->intremap !=3D b->intremap || + a->caching_mode !=3D b->caching_mode || + a->eim !=3D b->eim || + a->iotlb !=3D b->iotlb || + a->aw_bits !=3D b->aw_bits || + a->parent_idx !=3D b->parent_idx || + a->accel !=3D b->accel || + a->ats !=3D b->ats || + a->ril !=3D b->ril || + a->pasid !=3D b->pasid || + a->oas !=3D b->oas || + a->dma_translation !=3D b->dma_translation) + return false; + + if (a->info.type !=3D VIR_DOMAIN_DEVICE_ADDRESS_TYPE_NONE && + !virDomainDeviceInfoAddressIsEqual(&a->info, &b->info)) + return false; + + return true; +} + + +ssize_t +virDomainIOMMUDefFind(const virDomainDef *def, + const virDomainIOMMUDef *iommu) +{ + size_t i; + + for (i =3D 0; i < def->niommus; i++) { + if (virDomainIOMMUDefEquals(iommu, def->iommu[i])) + return i; + } + + return -1; +} + + bool virDomainVsockDefEquals(const virDomainVsockDef *a, const virDomainVsockDef *b) @@ -20175,19 +20218,28 @@ virDomainDefParseXML(xmlXPathContextPtr ctxt, } VIR_FREE(nodes); =20 + /* analysis of iommu devices */ if ((n =3D virXPathNodeSet("./devices/iommu", ctxt, &nodes)) < 0) return NULL; =20 - if (n > 1) { + if (n > 1 && !virXPathBoolean("./devices/iommu/@model =3D 'smmuv3Dev'"= , ctxt)) { virReportError(VIR_ERR_XML_ERROR, "%s", - _("only a single IOMMU device is supported")); + _("multiple IOMMU devices are only supported with m= odel smmuv3Dev")); return NULL; } =20 - if (n > 0) { - if (!(def->iommu =3D virDomainIOMMUDefParseXML(xmlopt, nodes[0], - ctxt, flags))) + if (n > 0) + def->iommu =3D g_new0(virDomainIOMMUDef *, n); + + for (i =3D 0; i < n; i++) { + virDomainIOMMUDef *iommu; + + iommu =3D virDomainIOMMUDefParseXML(xmlopt, nodes[i], ctxt, flags); + + if (!iommu) return NULL; + + def->iommu[def->niommus++] =3D iommu; } VIR_FREE(nodes); =20 @@ -22670,15 +22722,17 @@ virDomainDefCheckABIStabilityFlags(virDomainDef *= src, goto error; } =20 - if (!!src->iommu !=3D !!dst->iommu) { - virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", - _("Target domain IOMMU device count does not match = source")); + if (src->niommus !=3D dst->niommus) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, + _("Target domain IOMMU device count %1$zu does not = match source %2$zu"), + dst->niommus, src->niommus); goto error; } =20 - if (src->iommu && - !virDomainIOMMUDefCheckABIStability(src->iommu, dst->iommu)) - goto error; + for (i =3D 0; i < src->niommus; i++) { + if (!virDomainIOMMUDefCheckABIStability(src->iommu[i], dst->iommu[= i])) + goto error; + } =20 if (!!src->vsock !=3D !!dst->vsock) { virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", @@ -29535,8 +29589,9 @@ virDomainDefFormatInternalSetRootName(virDomainDef = *def, for (n =3D 0; n < def->ncryptos; n++) { virDomainCryptoDefFormat(buf, def->cryptos[n], flags); } - if (def->iommu) - virDomainIOMMUDefFormat(buf, def->iommu); + + for (n =3D 0; n < def->niommus; n++) + virDomainIOMMUDefFormat(buf, def->iommu[n]); =20 if (def->vsock) virDomainVsockDefFormat(buf, def->vsock); diff --git a/src/conf/domain_conf.h b/src/conf/domain_conf.h index 1bc52e1a63..f3f869a79a 100644 --- a/src/conf/domain_conf.h +++ b/src/conf/domain_conf.h @@ -3302,6 +3302,9 @@ struct _virDomainDef { size_t nwatchdogs; virDomainWatchdogDef **watchdogs; =20 + size_t niommus; + virDomainIOMMUDef **iommu; + /* At maximum 2 TPMs on the domain if a TPM Proxy is present. */ size_t ntpms; virDomainTPMDef **tpms; @@ -3311,7 +3314,6 @@ struct _virDomainDef { virDomainNVRAMDef *nvram; virCPUDef *cpu; virDomainRedirFilterDef *redirfilter; - virDomainIOMMUDef *iommu; virDomainVsockDef *vsock; virDomainPstoreDef *pstore; =20 @@ -4316,6 +4318,11 @@ virDomainShmemDef *virDomainShmemDefRemove(virDomain= Def *def, size_t idx) ssize_t virDomainInputDefFind(const virDomainDef *def, const virDomainInputDef *input) ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(2) G_GNUC_WARN_UNUSED_RESULT; +bool virDomainIOMMUDefEquals(const virDomainIOMMUDef *a, + const virDomainIOMMUDef *b); +ssize_t virDomainIOMMUDefFind(const virDomainDef *def, + const virDomainIOMMUDef *iommu) + ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(2) G_GNUC_WARN_UNUSED_RESULT; bool virDomainVsockDefEquals(const virDomainVsockDef *a, const virDomainVsockDef *b) ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(2) G_GNUC_WARN_UNUSED_RESULT; diff --git a/src/conf/domain_validate.c b/src/conf/domain_validate.c index 117338b4da..c3e3056810 100644 --- a/src/conf/domain_validate.c +++ b/src/conf/domain_validate.c @@ -1851,21 +1851,31 @@ virDomainDefCputuneValidate(const virDomainDef *def) static int virDomainDefIOMMUValidate(const virDomainDef *def) { + size_t i; + if (!def->iommu) return 0; =20 - if (def->iommu->intremap =3D=3D VIR_TRISTATE_SWITCH_ON && - def->features[VIR_DOMAIN_FEATURE_IOAPIC] !=3D VIR_DOMAIN_IOAPIC_QE= MU) { - virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", - _("IOMMU interrupt remapping requires split I/O API= C (ioapic driver=3D'qemu')")); - return -1; - } + for (i =3D 0; i < def->niommus; i++) { + virDomainIOMMUDef *iommu =3D def->iommu[i]; + if (def->niommus > 1 && iommu->model !=3D VIR_DOMAIN_IOMMU_MODEL_S= MMUV3_DEV) { + virReportError(VIR_ERR_XML_ERROR, "%s", + _("IOMMU model smmuv3Dev must be specified for = multiple IOMMU definitions")); + } =20 - if (def->iommu->eim =3D=3D VIR_TRISTATE_SWITCH_ON && - def->iommu->intremap !=3D VIR_TRISTATE_SWITCH_ON) { - virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", - _("IOMMU eim requires interrupt remapping to be ena= bled")); - return -1; + if (iommu->intremap =3D=3D VIR_TRISTATE_SWITCH_ON && + def->features[VIR_DOMAIN_FEATURE_IOAPIC] !=3D VIR_DOMAIN_IOAPI= C_QEMU) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", + _("IOMMU interrupt remapping requires split I/O= APIC (ioapic driver=3D'qemu')")); + return -1; + } + + if (iommu->eim =3D=3D VIR_TRISTATE_SWITCH_ON && + iommu->intremap !=3D VIR_TRISTATE_SWITCH_ON) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", + _("IOMMU eim requires interrupt remapping to be= enabled")); + return -1; + } } =20 return 0; diff --git a/src/conf/schemas/domaincommon.rng b/src/conf/schemas/domaincom= mon.rng index 64fb320ca9..36c1e7c3ef 100644 --- a/src/conf/schemas/domaincommon.rng +++ b/src/conf/schemas/domaincommon.rng @@ -6984,9 +6984,9 @@ - + - + diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index fe72402527..34d9c263d4 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -491,6 +491,8 @@ virDomainInputSourceGrabToggleTypeToString; virDomainInputSourceGrabTypeFromString; virDomainInputSourceGrabTypeToString; virDomainInputTypeToString; +virDomainIOMMUDefEquals; +virDomainIOMMUDefFind; virDomainIOMMUDefFree; virDomainIOMMUDefNew; virDomainIOMMUModelTypeFromString; diff --git a/src/qemu/qemu_alias.c b/src/qemu/qemu_alias.c index a27c688d79..5f2b11b9a6 100644 --- a/src/qemu/qemu_alias.c +++ b/src/qemu/qemu_alias.c @@ -647,10 +647,14 @@ qemuAssignDeviceVsockAlias(virDomainVsockDef *vsock) =20 =20 static void -qemuAssignDeviceIOMMUAlias(virDomainIOMMUDef *iommu) +qemuAssignDeviceIOMMUAlias(virDomainDef *def, + virDomainIOMMUDef **iommu) { - if (!iommu->info.alias) - iommu->info.alias =3D g_strdup("iommu0"); + size_t i; + for (i =3D 0; i < def->niommus; i++) { + if (!iommu[i]->info.alias) + iommu[i]->info.alias =3D g_strdup_printf("iommu%zu", i); + } } =20 =20 @@ -766,8 +770,9 @@ qemuAssignDeviceAliases(virDomainDef *def) if (def->vsock) { qemuAssignDeviceVsockAlias(def->vsock); } - if (def->iommu) - qemuAssignDeviceIOMMUAlias(def->iommu); + if (def->iommu && def->niommus > 0) { + qemuAssignDeviceIOMMUAlias(def, def->iommu); + } for (i =3D 0; i < def->ncryptos; i++) { qemuAssignDeviceCryptoAlias(def, def->cryptos[i]); } diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c index 6ec4801a7e..9f98a18695 100644 --- a/src/qemu/qemu_command.c +++ b/src/qemu/qemu_command.c @@ -6296,10 +6296,12 @@ qemuBuildBootCommandLine(virCommand *cmd, =20 static virJSONValue * qemuBuildPCISmmuv3DevDevProps(const virDomainDef *def, - const virDomainIOMMUDef *iommu) + const virDomainIOMMUDef *iommu, + size_t id) { g_autoptr(virJSONValue) props =3D NULL; g_autofree char *bus =3D NULL; + g_autofree char *smmuv3_id =3D NULL; size_t i; bool contIsPHB =3D false; =20 @@ -6340,9 +6342,12 @@ qemuBuildPCISmmuv3DevDevProps(const virDomainDef *de= f, bus =3D temp_bus; } =20 + smmuv3_id =3D g_strdup_printf("smmuv3.%zu", id); + if (virJSONValueObjectAdd(&props, "s:driver", "arm-smmuv3", - "S:primary-bus", bus, + "s:primary-bus", bus, + "s:id", smmuv3_id, "b:accel", (iommu->accel =3D=3D VIR_TRISTATE= _SWITCH_ON), "b:ats", (iommu->ats =3D=3D VIR_TRISTATE_SWI= TCH_ON), "b:ril", (iommu->ril =3D=3D VIR_TRISTATE_SWI= TCH_ON), @@ -6366,91 +6371,92 @@ qemuBuildIOMMUCommandLine(virCommand *cmd, const virDomainDef *def, virQEMUCaps *qemuCaps) { + size_t i; g_autoptr(virJSONValue) props =3D NULL; g_autoptr(virJSONValue) wrapperProps =3D NULL; - const virDomainIOMMUDef *iommu =3D def->iommu; - - if (!iommu) + if (!def->iommu || def->niommus <=3D 0) return 0; =20 - switch (iommu->model) { - case VIR_DOMAIN_IOMMU_MODEL_INTEL: - if (virJSONValueObjectAdd(&props, - "s:driver", "intel-iommu", - "s:id", iommu->info.alias, - "S:intremap", qemuOnOffAuto(iommu->intre= map), - "T:caching-mode", iommu->caching_mode, - "S:eim", qemuOnOffAuto(iommu->eim), - "T:device-iotlb", iommu->iotlb, - "z:aw-bits", iommu->aw_bits, - "T:dma-translation", iommu->dma_translat= ion, - NULL) < 0) - return -1; + for (i =3D 0; i < def->niommus; i++) { + virDomainIOMMUDef *iommu =3D def->iommu[i]; + switch (iommu->model) { + case VIR_DOMAIN_IOMMU_MODEL_INTEL: + if (virJSONValueObjectAdd(&props, + "s:driver", "intel-iommu", + "s:id", iommu->info.alias, + "S:intremap", qemuOnOffAuto(iommu->i= ntremap), + "T:caching-mode", iommu->caching_mod= e, + "S:eim", qemuOnOffAuto(iommu->eim), + "T:device-iotlb", iommu->iotlb, + "z:aw-bits", iommu->aw_bits, + "T:dma-translation", iommu->dma_tran= slation, + NULL) < 0) + return -1; =20 - if (qemuBuildDeviceCommandlineFromJSON(cmd, props, def, qemuCaps) = < 0) - return -1; + if (qemuBuildDeviceCommandlineFromJSON(cmd, props, def, qemuCa= ps) < 0) + return -1; =20 - return 0; + return 0; + case VIR_DOMAIN_IOMMU_MODEL_VIRTIO: + if (virJSONValueObjectAdd(&props, + "s:driver", "virtio-iommu", + "s:id", iommu->info.alias, + NULL) < 0) { + return -1; + } =20 - case VIR_DOMAIN_IOMMU_MODEL_VIRTIO: - if (virJSONValueObjectAdd(&props, - "s:driver", "virtio-iommu", - "s:id", iommu->info.alias, - NULL) < 0) { - return -1; - } + if (qemuBuildDeviceAddressProps(props, def, &iommu->info) < 0) + return -1; =20 - if (qemuBuildDeviceAddressProps(props, def, &iommu->info) < 0) - return -1; + if (qemuBuildDeviceCommandlineFromJSON(cmd, props, def, qemuCa= ps) < 0) + return -1; =20 - if (qemuBuildDeviceCommandlineFromJSON(cmd, props, def, qemuCaps) = < 0) - return -1; + return 0; + case VIR_DOMAIN_IOMMU_MODEL_SMMUV3: + /* There is no -device for SMMUv3, so nothing to be done here = */ + return 0; =20 - return 0; + case VIR_DOMAIN_IOMMU_MODEL_AMD: + if (virJSONValueObjectAdd(&wrapperProps, + "s:driver", "AMDVI-PCI", + "s:id", iommu->info.alias, + NULL) < 0) + return -1; =20 - case VIR_DOMAIN_IOMMU_MODEL_SMMUV3: - return 0; + if (qemuBuildDeviceAddressProps(wrapperProps, def, &iommu->inf= o) < 0) + return -1; =20 - case VIR_DOMAIN_IOMMU_MODEL_AMD: - if (virJSONValueObjectAdd(&wrapperProps, - "s:driver", "AMDVI-PCI", - "s:id", iommu->info.alias, - NULL) < 0) - return -1; + if (qemuBuildDeviceCommandlineFromJSON(cmd, wrapperProps, def,= qemuCaps) < 0) + return -1; =20 - if (qemuBuildDeviceAddressProps(wrapperProps, def, &iommu->info) <= 0) - return -1; + if (virJSONValueObjectAdd(&props, + "s:driver", "amd-iommu", + "s:pci-id", iommu->info.alias, + "S:intremap", qemuOnOffAuto(iommu->i= ntremap), + "T:pt", iommu->pt, + "T:xtsup", iommu->xtsup, + "T:device-iotlb", iommu->iotlb, + NULL) < 0) + return -1; =20 - if (qemuBuildDeviceCommandlineFromJSON(cmd, wrapperProps, def, qem= uCaps) < 0) - return -1; + if (qemuBuildDeviceCommandlineFromJSON(cmd, props, def, qemuCa= ps) < 0) + return -1; =20 - if (virJSONValueObjectAdd(&props, - "s:driver", "amd-iommu", - "s:pci-id", iommu->info.alias, - "S:intremap", qemuOnOffAuto(iommu->intre= map), - "T:pt", iommu->pt, - "T:xtsup", iommu->xtsup, - "T:device-iotlb", iommu->iotlb, - NULL) < 0) - return -1; + return 0; =20 - if (qemuBuildDeviceCommandlineFromJSON(cmd, props, def, qemuCaps) = < 0) - return -1; + case VIR_DOMAIN_IOMMU_MODEL_SMMUV3_DEV: + if (!(props =3D qemuBuildPCISmmuv3DevDevProps(def, iommu, i))) + return -1; + if (qemuBuildDeviceCommandlineFromJSON(cmd, props, def, qemuCa= ps) < 0) + return -1; + break; =20 - return 0; =20 - case VIR_DOMAIN_IOMMU_MODEL_SMMUV3_DEV: - if (!(props =3D qemuBuildPCISmmuv3DevDevProps(def, iommu))) - return -1; - if (qemuBuildDeviceCommandlineFromJSON(cmd, props, def, qemuCaps) = < 0) + case VIR_DOMAIN_IOMMU_MODEL_LAST: + default: + virReportEnumRangeError(virDomainIOMMUModel, iommu->model); return -1; - - return 0; - - case VIR_DOMAIN_IOMMU_MODEL_LAST: - default: - virReportEnumRangeError(virDomainIOMMUModel, iommu->model); - return -1; + } } =20 return 0; @@ -7271,8 +7277,8 @@ qemuBuildMachineCommandLine(virCommand *cmd, if (qemuAppendDomainFeaturesMachineParam(&buf, def, qemuCaps) < 0) return -1; =20 - if (def->iommu) { - switch (def->iommu->model) { + if (def->iommu && def->niommus =3D=3D 1) { + switch (def->iommu[0]->model) { case VIR_DOMAIN_IOMMU_MODEL_SMMUV3: virBufferAddLit(&buf, ",iommu=3Dsmmuv3"); break; @@ -7286,7 +7292,7 @@ qemuBuildMachineCommandLine(virCommand *cmd, =20 case VIR_DOMAIN_IOMMU_MODEL_LAST: default: - virReportEnumRangeError(virDomainIOMMUModel, def->iommu->model= ); + virReportEnumRangeError(virDomainIOMMUModel, def->iommu[0]->mo= del); return -1; } } diff --git a/src/qemu/qemu_domain_address.c b/src/qemu/qemu_domain_address.c index 06bf4fab32..2ddc629304 100644 --- a/src/qemu/qemu_domain_address.c +++ b/src/qemu/qemu_domain_address.c @@ -2365,24 +2365,25 @@ qemuDomainAssignDevicePCISlots(virDomainDef *def, /* Nada - none are PCI based (yet) */ } =20 - if (def->iommu) { - virDomainIOMMUDef *iommu =3D def->iommu; - - switch (iommu->model) { - case VIR_DOMAIN_IOMMU_MODEL_VIRTIO: - case VIR_DOMAIN_IOMMU_MODEL_AMD: - if (virDeviceInfoPCIAddressIsWanted(&iommu->info) && - qemuDomainPCIAddressReserveNextAddr(addrs, &iommu->info) <= 0) { - return -1; - } - break; + if (def->iommu && def->niommus > 0) { + for (i =3D 0; i < def->niommus; i++) { + virDomainIOMMUDef *iommu =3D def->iommu[i]; + switch (iommu->model) { + case VIR_DOMAIN_IOMMU_MODEL_VIRTIO: + case VIR_DOMAIN_IOMMU_MODEL_AMD: + if (virDeviceInfoPCIAddressIsWanted(&iommu->info) && + qemuDomainPCIAddressReserveNextAddr(addrs, &iommu->inf= o) < 0) { + return -1; + } + break; =20 - case VIR_DOMAIN_IOMMU_MODEL_INTEL: - case VIR_DOMAIN_IOMMU_MODEL_SMMUV3: - case VIR_DOMAIN_IOMMU_MODEL_SMMUV3_DEV: - case VIR_DOMAIN_IOMMU_MODEL_LAST: - /* These are not PCI devices */ - break; + case VIR_DOMAIN_IOMMU_MODEL_INTEL: + case VIR_DOMAIN_IOMMU_MODEL_SMMUV3: + case VIR_DOMAIN_IOMMU_MODEL_SMMUV3_DEV: + case VIR_DOMAIN_IOMMU_MODEL_LAST: + /* These are not PCI devices */ + break; + } } } =20 diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index ac72ea5cb0..3d65f78c9e 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -6894,12 +6894,12 @@ qemuDomainAttachDeviceConfig(virDomainDef *vmdef, break; =20 case VIR_DOMAIN_DEVICE_IOMMU: - if (vmdef->iommu) { + if (vmdef->iommu && vmdef->niommus > 0) { virReportError(VIR_ERR_OPERATION_INVALID, "%s", _("domain already has an iommu device")); return -1; } - vmdef->iommu =3D g_steal_pointer(&dev->data.iommu); + VIR_APPEND_ELEMENT(vmdef->iommu, vmdef->niommus, dev->data.iommu); break; =20 case VIR_DOMAIN_DEVICE_VIDEO: @@ -7113,12 +7113,12 @@ qemuDomainDetachDeviceConfig(virDomainDef *vmdef, break; =20 case VIR_DOMAIN_DEVICE_IOMMU: - if (!vmdef->iommu) { + if ((idx =3D virDomainIOMMUDefFind(vmdef, dev->data.iommu)) < 0) { virReportError(VIR_ERR_OPERATION_FAILED, "%s", _("matching iommu device not found")); return -1; } - g_clear_pointer(&vmdef->iommu, virDomainIOMMUDefFree); + VIR_DELETE_ELEMENT(vmdef->iommu, idx, vmdef->niommus); break; =20 case VIR_DOMAIN_DEVICE_VIDEO: diff --git a/src/qemu/qemu_postparse.c b/src/qemu/qemu_postparse.c index 9c2427970d..e2744a4a61 100644 --- a/src/qemu/qemu_postparse.c +++ b/src/qemu/qemu_postparse.c @@ -1503,7 +1503,7 @@ qemuDomainDefAddDefaultDevices(virQEMUDriver *driver, } } =20 - if (addIOMMU && !def->iommu && + if (addIOMMU && !def->iommu && def->niommus =3D=3D 0 && virQEMUCapsGet(qemuCaps, QEMU_CAPS_DEVICE_INTEL_IOMMU) && virQEMUCapsGet(qemuCaps, QEMU_CAPS_INTEL_IOMMU_INTREMAP) && virQEMUCapsGet(qemuCaps, QEMU_CAPS_INTEL_IOMMU_EIM)) { @@ -1515,7 +1515,8 @@ qemuDomainDefAddDefaultDevices(virQEMUDriver *driver, iommu->intremap =3D VIR_TRISTATE_SWITCH_ON; iommu->eim =3D VIR_TRISTATE_SWITCH_ON; =20 - def->iommu =3D g_steal_pointer(&iommu); + def->iommu =3D g_new0(virDomainIOMMUDef *, 1); + def->iommu[def->niommus++] =3D g_steal_pointer(&iommu); } =20 if (qemuDomainDefAddDefaultAudioBackend(driver, def) < 0) @@ -1591,9 +1592,9 @@ qemuDomainDefEnableDefaultFeatures(virDomainDef *def, * domain already has IOMMU without inremap. This will be fixed in * qemuDomainIOMMUDefPostParse() but there domain definition can't be * modified so change it now. */ - if (def->iommu && - (def->iommu->intremap =3D=3D VIR_TRISTATE_SWITCH_ON || - qemuDomainNeedsIOMMUWithEIM(def)) && + if (def->iommu && def->niommus =3D=3D 1 && + (def->iommu[0]->intremap =3D=3D VIR_TRISTATE_SWITCH_ON || + qemuDomainNeedsIOMMUWithEIM(def)) && def->features[VIR_DOMAIN_FEATURE_IOAPIC] =3D=3D VIR_DOMAIN_IOAPIC_= NONE) { def->features[VIR_DOMAIN_FEATURE_IOAPIC] =3D VIR_DOMAIN_IOAPIC_QEM= U; } diff --git a/src/qemu/qemu_validate.c b/src/qemu/qemu_validate.c index aac004c544..bdaeefe976 100644 --- a/src/qemu/qemu_validate.c +++ b/src/qemu/qemu_validate.c @@ -851,7 +851,7 @@ qemuValidateDomainVCpuTopology(const virDomainDef *def,= virQEMUCaps *qemuCaps) QEMU_MAX_VCPUS_WITHOUT_EIM); return -1; } - if (!def->iommu || def->iommu->eim !=3D VIR_TRISTATE_SWITCH_ON) { + if (!def->iommu || def->iommu[0]->eim !=3D VIR_TRISTATE_SWITCH_ON)= { virReportError(VIR_ERR_CONFIG_UNSUPPORTED, _("more than %1$d vCPUs require extended interr= upt mode enabled on the iommu device"), QEMU_MAX_VCPUS_WITHOUT_EIM); --=20 2.43.0