From nobody Thu May 9 18:49:36 2024 Delivered-To: importer@patchew.org Received-SPF: none (zohomail.com: 8.43.85.245 is neither permitted nor denied by domain of lists.libvirt.org) client-ip=8.43.85.245; envelope-from=devel-bounces@lists.libvirt.org; helo=lists.libvirt.org; Authentication-Results: mx.zohomail.com; spf=none (zohomail.com: 8.43.85.245 is neither permitted nor denied by domain of lists.libvirt.org) smtp.mailfrom=devel-bounces@lists.libvirt.org; dmarc=fail(p=quarantine dis=none) header.from=akamai.com Return-Path: Received: from lists.libvirt.org (lists.libvirt.org [8.43.85.245]) by mx.zohomail.com with SMTPS id 1706566649471437.1945026684508; Mon, 29 Jan 2024 14:17:29 -0800 (PST) Received: by lists.libvirt.org (Postfix, from userid 996) id 44AC11B90; Mon, 29 Jan 2024 17:17:28 -0500 (EST) Received: from lists.libvirt.org.85.43.8.in-addr.arpa (localhost [IPv6:::1]) by lists.libvirt.org (Postfix) with ESMTP id DBD2C1A2A; Mon, 29 Jan 2024 17:15:49 -0500 (EST) Received: by lists.libvirt.org (Postfix, from userid 996) id F032419F6; Mon, 29 Jan 2024 17:15:45 -0500 (EST) Received: from mx0b-00190b01.pphosted.com (mx0b-00190b01.pphosted.com [67.231.157.127]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.libvirt.org (Postfix) with ESMTPS id 547611A06 for ; Mon, 29 Jan 2024 17:15:45 -0500 (EST) Received: from pps.filterd (m0122330.ppops.net [127.0.0.1]) by mx0b-00190b01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 40TI7TCr024131; Mon, 29 Jan 2024 21:44:45 GMT Received: from prod-mail-ppoint2 (prod-mail-ppoint2.akamai.com [184.51.33.19] (may be forged)) by mx0b-00190b01.pphosted.com (PPS) with ESMTPS id 3vvtc10ctt-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 29 Jan 2024 21:44:45 +0000 (GMT) Received: from pps.filterd (prod-mail-ppoint2.akamai.com [127.0.0.1]) by prod-mail-ppoint2.akamai.com (8.17.1.19/8.17.1.19) with ESMTP id 40TL59T0007698; Mon, 29 Jan 2024 16:44:44 -0500 Received: from prod-mail-relay10.akamai.com ([172.27.118.251]) by prod-mail-ppoint2.akamai.com (PPS) with ESMTP id 3vvx22m82e-1; Mon, 29 Jan 2024 16:44:44 -0500 Received: from bos-lhvcg5.bos01.corp.akamai.com (bos-lhvcg5.bos01.corp.akamai.com [172.28.221.19]) by prod-mail-relay10.akamai.com (Postfix) with ESMTP id 21E5F64408; Mon, 29 Jan 2024 21:44:44 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on lists.libvirt.org X-Spam-Level: ** X-Spam-Status: No, score=2.1 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_LOW,RCVD_IN_MSPIKE_H5, RCVD_IN_MSPIKE_WL,RCVD_IN_SBL_CSS,SPF_HELO_NONE,T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.4 From: mgalaxy@akamai.com To: devel@lists.libvirt.org Subject: [PATCH 1/5] qemu.conf changes to support multiple memory backend directories Date: Mon, 29 Jan 2024 16:43:53 -0500 Message-Id: <20240129214357.1281805-2-mgalaxy@akamai.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240129214357.1281805-1-mgalaxy@akamai.com> References: <20240129214357.1281805-1-mgalaxy@akamai.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2024-01-29_14,2024-01-29_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 mlxlogscore=999 malwarescore=0 spamscore=0 adultscore=0 suspectscore=0 phishscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2311290000 definitions=main-2401290159 X-Proofpoint-ORIG-GUID: c29vgePecb_sNvs6DaRWZKqk6_nxn9zQ X-Proofpoint-GUID: c29vgePecb_sNvs6DaRWZKqk6_nxn9zQ X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2024-01-29_14,2024-01-29_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 adultscore=0 mlxlogscore=999 suspectscore=0 clxscore=1015 phishscore=0 priorityscore=1501 spamscore=0 malwarescore=0 mlxscore=0 impostorscore=0 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2401190000 definitions=main-2401290160 Message-ID-Hash: PV4H4XTCY76LUSZT3VFF33CLG6QOL2WV X-Message-ID-Hash: PV4H4XTCY76LUSZT3VFF33CLG6QOL2WV X-MailFrom: mgalaxy@akamai.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-config-1; header-match-config-2; header-match-config-3; header-match-devel.lists.libvirt.org-0; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; suspicious-header CC: steven.sistare@oracle.com, johunt@akamai.com, cweng@akamai.com, bchaney@akamai.com, trakotoz@akamai.com, Michael Galaxy X-Mailman-Version: 3.2.2 Precedence: list List-Id: Development discussions about the libvirt library & tools Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZM-MESSAGEID: 1706566650331100001 From: Michael Galaxy We start by introducing a backwards-compatible, comma-separated specification that will not break existing installations, such as in the following example: $ cat qemu.conf | grep memory_backing_dir memory_backing_dir =3D "/path/to/pmem/0,/path/to/pmem/1" In our case, we almost always have two NUMA nodes, so in that example, we have two PMEM regions which are created on the Linux kernel command line that get mounted into those two locations for libvirt to use. Later patches will recognize this configuration and activate it. Signed-off-by: Michael Galaxy --- src/qemu/qemu_conf.c | 39 +++++++++++++++++++++++++++++++++------ src/qemu/qemu_conf.h | 3 ++- 2 files changed, 35 insertions(+), 7 deletions(-) diff --git a/src/qemu/qemu_conf.c b/src/qemu/qemu_conf.c index 4050a82341..aae9f316d8 100644 --- a/src/qemu/qemu_conf.c +++ b/src/qemu/qemu_conf.c @@ -43,6 +43,7 @@ #include "virfile.h" #include "virstring.h" #include "virutil.h" +#include "virnuma.h" #include "configmake.h" #include "security/security_util.h" =20 @@ -137,6 +138,9 @@ virQEMUDriverConfig *virQEMUDriverConfigNew(bool privil= eged, =20 cfg->cgroupControllers =3D -1; /* -1 =3D=3D auto-detect */ =20 + cfg->memoryBackingDirs =3D g_new0(char *, 1); + cfg->nb_memoryBackingDirs =3D 1; + if (root !=3D NULL) { cfg->logDir =3D g_strdup_printf("%s/log/qemu", root); cfg->swtpmLogDir =3D g_strdup_printf("%s/log/swtpm", root); @@ -153,7 +157,7 @@ virQEMUDriverConfig *virQEMUDriverConfigNew(bool privil= eged, cfg->checkpointDir =3D g_strdup_printf("%s/checkpoint", cfg->libDi= r); cfg->autoDumpPath =3D g_strdup_printf("%s/dump", cfg->libDir); cfg->nvramDir =3D g_strdup_printf("%s/nvram", cfg->libDir); - cfg->memoryBackingDir =3D g_strdup_printf("%s/ram", cfg->libDir); + cfg->memoryBackingDirs[0] =3D g_strdup_printf("%s/ram", cfg->libDi= r); } else if (privileged) { cfg->logDir =3D g_strdup_printf("%s/log/libvirt/qemu", LOCALSTATED= IR); =20 @@ -174,7 +178,8 @@ virQEMUDriverConfig *virQEMUDriverConfigNew(bool privil= eged, cfg->checkpointDir =3D g_strdup_printf("%s/checkpoint", cfg->libDi= r); cfg->autoDumpPath =3D g_strdup_printf("%s/dump", cfg->libDir); cfg->nvramDir =3D g_strdup_printf("%s/nvram", cfg->libDir); - cfg->memoryBackingDir =3D g_strdup_printf("%s/ram", cfg->libDir); + + cfg->memoryBackingDirs[0] =3D g_strdup_printf("%s/ram", cfg->libDi= r); cfg->swtpmStorageDir =3D g_strdup_printf("%s/lib/libvirt/swtpm", LOCALSTATEDIR); } else { @@ -201,7 +206,7 @@ virQEMUDriverConfig *virQEMUDriverConfigNew(bool privil= eged, cfg->configBaseDir); cfg->autoDumpPath =3D g_strdup_printf("%s/qemu/dump", cfg->configB= aseDir); cfg->nvramDir =3D g_strdup_printf("%s/qemu/nvram", cfg->configBase= Dir); - cfg->memoryBackingDir =3D g_strdup_printf("%s/qemu/ram", cfg->conf= igBaseDir); + cfg->memoryBackingDirs[0] =3D g_strdup_printf("%s/qemu/ram", cfg->= configBaseDir); cfg->swtpmStorageDir =3D g_strdup_printf("%s/qemu/swtpm", cfg->configBaseDir); } @@ -369,7 +374,12 @@ static void virQEMUDriverConfigDispose(void *obj) =20 virFirmwareFreeList(cfg->firmwares, cfg->nfirmwares); =20 - g_free(cfg->memoryBackingDir); + for (size_t i =3D 0; i < cfg->nb_memoryBackingDirs; i++) { + g_free(cfg->memoryBackingDirs[i]); + } + + g_free(cfg->memoryBackingDirs); + g_free(cfg->swtpmStorageDir); =20 g_strfreev(cfg->capabilityfilters); @@ -1019,14 +1029,31 @@ virQEMUDriverConfigLoadMemoryEntry(virQEMUDriverCon= fig *cfg, virConf *conf) { g_autofree char *dir =3D NULL; + g_auto(GStrv) params =3D NULL; + GStrv next; int rc; =20 if ((rc =3D virConfGetValueString(conf, "memory_backing_dir", &dir)) <= 0) return -1; =20 if (rc > 0) { - VIR_FREE(cfg->memoryBackingDir); - cfg->memoryBackingDir =3D g_strdup_printf("%s/libvirt/qemu", dir); + size_t i; + for (i =3D 0; i < cfg->nb_memoryBackingDirs; i++) + VIR_FREE(cfg->memoryBackingDirs[i]); + + if (!(params =3D g_strsplit(dir, ",", 0))) { + return -1; + } + + cfg->nb_memoryBackingDirs =3D 0; + for (next =3D params; *next; next++) { + cfg->nb_memoryBackingDirs++; + } + + cfg->memoryBackingDirs =3D g_new0(char *, cfg->nb_memoryBackingDir= s); + for (i =3D 0, next =3D params; *next; next++, i++) { + cfg->memoryBackingDirs[i] =3D g_strdup_printf("%s/libvirt/qemu= ", *next); + } } =20 return 0; diff --git a/src/qemu/qemu_conf.h b/src/qemu/qemu_conf.h index 36049b4bfa..2b8d540df0 100644 --- a/src/qemu/qemu_conf.h +++ b/src/qemu/qemu_conf.h @@ -221,7 +221,8 @@ struct _virQEMUDriverConfig { unsigned int glusterDebugLevel; bool virtiofsdDebug; =20 - char *memoryBackingDir; + char **memoryBackingDirs; + size_t nb_memoryBackingDirs; =20 uid_t swtpm_user; gid_t swtpm_group; --=20 2.25.1 _______________________________________________ Devel mailing list -- devel@lists.libvirt.org To unsubscribe send an email to devel-leave@lists.libvirt.org From nobody Thu May 9 18:49:36 2024 Delivered-To: importer@patchew.org Received-SPF: none (zohomail.com: 8.43.85.245 is neither permitted nor denied by domain of lists.libvirt.org) client-ip=8.43.85.245; envelope-from=devel-bounces@lists.libvirt.org; helo=lists.libvirt.org; Authentication-Results: mx.zohomail.com; spf=none (zohomail.com: 8.43.85.245 is neither permitted nor denied by domain of lists.libvirt.org) smtp.mailfrom=devel-bounces@lists.libvirt.org; dmarc=fail(p=quarantine dis=none) header.from=akamai.com Return-Path: Received: from lists.libvirt.org (lists.libvirt.org [8.43.85.245]) by mx.zohomail.com with SMTPS id 1706567299935435.6632790350383; Mon, 29 Jan 2024 14:28:19 -0800 (PST) Received: by lists.libvirt.org (Postfix, from userid 996) id CB6BB17CA; Mon, 29 Jan 2024 17:28:18 -0500 (EST) Received: from lists.libvirt.org.85.43.8.in-addr.arpa (localhost [IPv6:::1]) by lists.libvirt.org (Postfix) with ESMTP id 39A9B1C8A; Mon, 29 Jan 2024 17:23:58 -0500 (EST) Received: by lists.libvirt.org (Postfix, from userid 996) id 0AE41171E; Mon, 29 Jan 2024 17:23:52 -0500 (EST) Received: from mx0a-00190b01.pphosted.com (mx0a-00190b01.pphosted.com [67.231.149.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.libvirt.org (Postfix) with ESMTPS id A984217CB for ; Mon, 29 Jan 2024 17:23:50 -0500 (EST) Received: from pps.filterd (m0050093.ppops.net [127.0.0.1]) by m0050093.ppops.net-00190b01. (8.17.1.24/8.17.1.24) with ESMTP id 40TI7NsI008789; Mon, 29 Jan 2024 21:44:46 GMT Received: from prod-mail-ppoint5 (prod-mail-ppoint5.akamai.com [184.51.33.60] (may be forged)) by m0050093.ppops.net-00190b01. (PPS) with ESMTPS id 3vvsyudryj-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 29 Jan 2024 21:44:45 +0000 (GMT) Received: from pps.filterd (prod-mail-ppoint5.akamai.com [127.0.0.1]) by prod-mail-ppoint5.akamai.com (8.17.1.19/8.17.1.19) with ESMTP id 40TJp8xk017068; Mon, 29 Jan 2024 13:44:44 -0800 Received: from prod-mail-relay10.akamai.com ([172.27.118.251]) by prod-mail-ppoint5.akamai.com (PPS) with ESMTP id 3vw0pekp4h-1; Mon, 29 Jan 2024 13:44:44 -0800 Received: from bos-lhvcg5.bos01.corp.akamai.com (bos-lhvcg5.bos01.corp.akamai.com [172.28.221.19]) by prod-mail-relay10.akamai.com (Postfix) with ESMTP id 2ECAF6440E; Mon, 29 Jan 2024 21:44:44 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on lists.libvirt.org X-Spam-Level: X-Spam-Status: No, score=-1.5 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_LOW,RCVD_IN_MSPIKE_H4, RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.4 From: mgalaxy@akamai.com To: devel@lists.libvirt.org Subject: [PATCH 2/5] Update cleanup routines to handle multiple memory backing paths instead of just one. Date: Mon, 29 Jan 2024 16:43:54 -0500 Message-Id: <20240129214357.1281805-3-mgalaxy@akamai.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240129214357.1281805-1-mgalaxy@akamai.com> References: <20240129214357.1281805-1-mgalaxy@akamai.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2024-01-29_14,2024-01-29_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 malwarescore=0 phishscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 suspectscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2311290000 definitions=main-2401290159 X-Proofpoint-GUID: Ay_MdEAGBNrBWPdgxVmQCp4eiehFxF3r X-Proofpoint-ORIG-GUID: Ay_MdEAGBNrBWPdgxVmQCp4eiehFxF3r X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2024-01-29_14,2024-01-29_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 priorityscore=1501 bulkscore=0 mlxlogscore=999 suspectscore=0 lowpriorityscore=0 clxscore=1015 phishscore=0 malwarescore=0 mlxscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2401190000 definitions=main-2401290160 Message-ID-Hash: BYZ5CYRQF2MM4DAVWEG5I5SJZYSXKZDC X-Message-ID-Hash: BYZ5CYRQF2MM4DAVWEG5I5SJZYSXKZDC X-MailFrom: mgalaxy@akamai.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-config-1; header-match-config-2; header-match-config-3; header-match-devel.lists.libvirt.org-0; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; suspicious-header CC: steven.sistare@oracle.com, johunt@akamai.com, cweng@akamai.com, bchaney@akamai.com, trakotoz@akamai.com, Michael Galaxy X-Mailman-Version: 3.2.2 Precedence: list List-Id: Development discussions about the libvirt library & tools Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZM-MESSAGEID: 1706567301148100001 From: Michael Galaxy Signed-off-by: Michael Galaxy --- src/qemu/qemu_driver.c | 31 +++++++++++++++++----------- src/qemu/qemu_hotplug.c | 6 +++--- src/qemu/qemu_process.c | 45 ++++++++++++++++++++++++----------------- src/qemu/qemu_process.h | 13 ++++++------ 4 files changed, 54 insertions(+), 41 deletions(-) diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 448e6b1591..98742f83c8 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -653,11 +653,14 @@ qemuStateInitialize(bool privileged, cfg->nvramDir); goto error; } - if (g_mkdir_with_parents(cfg->memoryBackingDir, 0777) < 0) { - virReportSystemError(errno, _("Failed to create memory backing dir= %1$s"), - cfg->memoryBackingDir); - goto error; + for (i =3D 0; i < cfg->nb_memoryBackingDirs; i++) { + if (g_mkdir_with_parents(cfg->memoryBackingDirs[i], 0777) < 0) { + virReportSystemError(errno, _("Failed to create memory backing dir # %1$= lu @ %2$s, total: %3$lu"), + i, cfg->memoryBackingDirs[i], cfg->nb_memoryBackingDirs); + goto error; + } } + if (g_mkdir_with_parents(cfg->slirpStateDir, 0777) < 0) { virReportSystemError(errno, _("Failed to create slirp state dir %1= $s"), cfg->slirpStateDir); @@ -802,13 +805,15 @@ qemuStateInitialize(bool privileged, (int)cfg->group); goto error; } - if (chown(cfg->memoryBackingDir, cfg->user, cfg->group) < 0) { - virReportSystemError(errno, + for (i =3D 0; i < cfg->nb_memoryBackingDirs; i++) { + if (chown(cfg->memoryBackingDirs[i], cfg->user, cfg->group) < = 0) { + virReportSystemError(errno, _("unable to set ownership of '%1$s' to %= 2$d:%3$d"), - cfg->memoryBackingDir, (int)cfg->user, + cfg->memoryBackingDirs[i], (int)cfg->user, (int)cfg->group); - goto error; - } + goto error; + } + } if (chown(cfg->slirpStateDir, cfg->user, cfg->group) < 0) { virReportSystemError(errno, _("unable to set ownership of '%1$s' to %= 2$d:%3$d"), @@ -855,10 +860,12 @@ qemuStateInitialize(bool privileged, goto error; } =20 - if (privileged && - virFileUpdatePerm(cfg->memoryBackingDir, + for (i =3D 0; i < cfg->nb_memoryBackingDirs; i++) { + if (privileged && + virFileUpdatePerm(cfg->memoryBackingDirs[i], 0, S_IXGRP | S_IXOTH) < 0) - goto error; + goto error; + } =20 /* Get all the running persistent or transient configs first */ if (virDomainObjListLoadAllConfigs(qemu_driver->domains, diff --git a/src/qemu/qemu_hotplug.c b/src/qemu/qemu_hotplug.c index 31b00e05ca..c3d636d167 100644 --- a/src/qemu/qemu_hotplug.c +++ b/src/qemu/qemu_hotplug.c @@ -2289,7 +2289,7 @@ qemuDomainAttachMemory(virQEMUDriver *driver, priv, vm->def, mem, true, false, NULL)= < 0) goto cleanup; =20 - if (qemuProcessBuildDestroyMemoryPaths(driver, vm, mem, true) < 0) + if (qemuProcessBuildDestroyMemoryPaths(vm, mem, true) < 0) goto cleanup; =20 if (qemuDomainNamespaceSetupMemory(vm, mem, &teardowndevice) < 0) @@ -2364,7 +2364,7 @@ qemuDomainAttachMemory(virQEMUDriver *driver, qemuDomainObjExitMonitor(vm); =20 if (objAdded && mem) - ignore_value(qemuProcessDestroyMemoryBackingPath(driver, vm, mem)); + ignore_value(qemuProcessDestroyMemoryBackingPath(vm->def, priv, me= m)); =20 virErrorRestore(&orig_err); if (!mem) @@ -4640,7 +4640,7 @@ qemuDomainRemoveMemoryDevice(virQEMUDriver *driver, if (qemuDomainNamespaceTeardownMemory(vm, mem) < 0) VIR_WARN("Unable to remove memory device from /dev"); =20 - if (qemuProcessDestroyMemoryBackingPath(driver, vm, mem) < 0) + if (qemuProcessDestroyMemoryBackingPath(vm->def, priv, mem) < 0) VIR_WARN("Unable to destroy memory backing path"); =20 qemuDomainReleaseMemoryDeviceSlot(vm, mem); diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index c1115b440f..48681ef97f 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -4002,12 +4002,12 @@ qemuProcessBuildDestroyMemoryPathsImpl(virQEMUDrive= r *driver, =20 =20 int -qemuProcessBuildDestroyMemoryPaths(virQEMUDriver *driver, - virDomainObj *vm, +qemuProcessBuildDestroyMemoryPaths(virDomainObj *vm, virDomainMemoryDef *mem, bool build) { - + qemuDomainObjPrivate *priv =3D vm->privateData; + virQEMUDriver *driver =3D priv->driver; g_autoptr(virQEMUDriverConfig) cfg =3D virQEMUDriverGetConfig(driver); size_t i; bool shouldBuildHP =3D false; @@ -4037,13 +4037,15 @@ qemuProcessBuildDestroyMemoryPaths(virQEMUDriver *d= river, } =20 if (!build || shouldBuildMB) { - g_autofree char *path =3D NULL; - if (qemuGetMemoryBackingDomainPath(driver, vm->def, &path) < 0) - return -1; + for (i =3D 0; i < cfg->nb_memoryBackingDirs; i++) { + g_autofree char *path =3D NULL; + if (qemuGetMemoryBackingDomainPath(vm->def, vm->privateData, i, &path) <= 0) + return -1; =20 - if (qemuProcessBuildDestroyMemoryPathsImpl(driver, vm, - path, build) < 0) - return -1; + if (qemuProcessBuildDestroyMemoryPathsImpl(driver, vm, + path, build) < 0) + return -1; + } } =20 return 0; @@ -4051,19 +4053,24 @@ qemuProcessBuildDestroyMemoryPaths(virQEMUDriver *d= river, =20 =20 int -qemuProcessDestroyMemoryBackingPath(virQEMUDriver *driver, - virDomainObj *vm, +qemuProcessDestroyMemoryBackingPath(virDomainDef *def, + qemuDomainObjPrivate *priv, virDomainMemoryDef *mem) { + virQEMUDriver *driver =3D priv->driver; g_autofree char *path =3D NULL; + g_autoptr(virQEMUDriverConfig) cfg =3D virQEMUDriverGetConfig(driver); =20 - if (qemuGetMemoryBackingPath(driver, vm->def, mem->info.alias, &path) = < 0) - return -1; + for (size_t i =3D 0; i < cfg->nb_memoryBackingDirs; i++) { + virDomainXMLPrivateDataCallbacks *privateData =3D (virDomainXMLPrivat= eDataCallbacks *) priv; + if (qemuGetMemoryBackingPath(def, privateData, i, mem->info.alias, &p= ath) < 0) + return -1; =20 - if (unlink(path) < 0 && - errno !=3D ENOENT) { - virReportSystemError(errno, _("Unable to remove %1$s"), path); - return -1; + if (unlink(path) < 0 && + errno !=3D ENOENT) { + virReportSystemError(errno, _("Unable to remove %1$s"), path); + return -1; + } } =20 return 0; @@ -7225,7 +7232,7 @@ qemuProcessPrepareHost(virQEMUDriver *driver, if (qemuProcessPrepareHostBackendChardev(vm) < 0) return -1; =20 - if (qemuProcessBuildDestroyMemoryPaths(driver, vm, NULL, true) < 0) + if (qemuProcessBuildDestroyMemoryPaths(vm, NULL, true) < 0) return -1; =20 /* Ensure no historical cgroup for this VM is lying around bogus @@ -8438,7 +8445,7 @@ void qemuProcessStop(virQEMUDriver *driver, goto endjob; } =20 - qemuProcessBuildDestroyMemoryPaths(driver, vm, NULL, false); + qemuProcessBuildDestroyMemoryPaths(vm, NULL, false); =20 if (!!g_atomic_int_dec_and_test(&driver->nactive) && driver->inhibitCa= llback) driver->inhibitCallback(false, driver->inhibitOpaque); diff --git a/src/qemu/qemu_process.h b/src/qemu/qemu_process.h index c1ea949215..2993ef518a 100644 --- a/src/qemu/qemu_process.h +++ b/src/qemu/qemu_process.h @@ -38,14 +38,13 @@ int qemuProcessStopCPUs(virQEMUDriver *driver, virDomainPausedReason reason, virDomainAsyncJob asyncJob); =20 -int qemuProcessBuildDestroyMemoryPaths(virQEMUDriver *driver, - virDomainObj *vm, - virDomainMemoryDef *mem, - bool build); +int qemuProcessBuildDestroyMemoryPaths(virDomainObj *vm, + virDomainMemoryDef *mem, + bool build); =20 -int qemuProcessDestroyMemoryBackingPath(virQEMUDriver *driver, - virDomainObj *vm, - virDomainMemoryDef *mem); +int qemuProcessDestroyMemoryBackingPath(virDomainDef *def, + qemuDomainObjPrivate *priv, + virDomainMemoryDef *mem); =20 void qemuProcessReconnectAll(virQEMUDriver *driver); =20 --=20 2.25.1 _______________________________________________ Devel mailing list -- devel@lists.libvirt.org To unsubscribe send an email to devel-leave@lists.libvirt.org From nobody Thu May 9 18:49:36 2024 Delivered-To: importer@patchew.org Received-SPF: none (zohomail.com: 8.43.85.245 is neither permitted nor denied by domain of lists.libvirt.org) client-ip=8.43.85.245; envelope-from=devel-bounces@lists.libvirt.org; helo=lists.libvirt.org; Authentication-Results: mx.zohomail.com; spf=none (zohomail.com: 8.43.85.245 is neither permitted nor denied by domain of lists.libvirt.org) smtp.mailfrom=devel-bounces@lists.libvirt.org; dmarc=fail(p=quarantine dis=none) header.from=akamai.com Return-Path: Received: from lists.libvirt.org (lists.libvirt.org [8.43.85.245]) by mx.zohomail.com with SMTPS id 1706567174077210.52791172136415; Mon, 29 Jan 2024 14:26:14 -0800 (PST) Received: by lists.libvirt.org (Postfix, from userid 996) id 9DE3B1C1E; Mon, 29 Jan 2024 17:26:12 -0500 (EST) Received: from lists.libvirt.org.85.43.8.in-addr.arpa (localhost [IPv6:::1]) by lists.libvirt.org (Postfix) with ESMTP id 6D66917ED; Mon, 29 Jan 2024 17:23:55 -0500 (EST) Received: by lists.libvirt.org (Postfix, from userid 996) id C946E17EF; Mon, 29 Jan 2024 17:23:51 -0500 (EST) Received: from mx0a-00190b01.pphosted.com (mx0a-00190b01.pphosted.com [67.231.149.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.libvirt.org (Postfix) with ESMTPS id 6B3B917A6 for ; Mon, 29 Jan 2024 17:23:50 -0500 (EST) Received: from pps.filterd (m0122332.ppops.net [127.0.0.1]) by mx0a-00190b01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 40TI7NvY013422; Mon, 29 Jan 2024 21:44:46 GMT Received: from prod-mail-ppoint6 (prod-mail-ppoint6.akamai.com [184.51.33.61] (may be forged)) by mx0a-00190b01.pphosted.com (PPS) with ESMTPS id 3vvtrn1yp3-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 29 Jan 2024 21:44:45 +0000 (GMT) Received: from pps.filterd (prod-mail-ppoint6.akamai.com [127.0.0.1]) by prod-mail-ppoint6.akamai.com (8.17.1.19/8.17.1.19) with ESMTP id 40TKtlhw002186; Mon, 29 Jan 2024 16:44:44 -0500 Received: from prod-mail-relay10.akamai.com ([172.27.118.251]) by prod-mail-ppoint6.akamai.com (PPS) with ESMTP id 3vvx22v8hj-1; Mon, 29 Jan 2024 16:44:44 -0500 Received: from bos-lhvcg5.bos01.corp.akamai.com (bos-lhvcg5.bos01.corp.akamai.com [172.28.221.19]) by prod-mail-relay10.akamai.com (Postfix) with ESMTP id 3C3E064407; Mon, 29 Jan 2024 21:44:44 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on lists.libvirt.org X-Spam-Level: X-Spam-Status: No, score=-1.5 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_LOW,RCVD_IN_MSPIKE_H4, RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.4 From: mgalaxy@akamai.com To: devel@lists.libvirt.org Subject: [PATCH 3/5] Implement multiple memory backing paths Date: Mon, 29 Jan 2024 16:43:55 -0500 Message-Id: <20240129214357.1281805-4-mgalaxy@akamai.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240129214357.1281805-1-mgalaxy@akamai.com> References: <20240129214357.1281805-1-mgalaxy@akamai.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2024-01-29_14,2024-01-29_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 phishscore=0 bulkscore=0 spamscore=0 adultscore=0 suspectscore=0 mlxlogscore=999 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2311290000 definitions=main-2401290159 X-Proofpoint-GUID: gsrGLJKVJTeMyey7GakJ4apQNcdghkd_ X-Proofpoint-ORIG-GUID: gsrGLJKVJTeMyey7GakJ4apQNcdghkd_ X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2024-01-29_14,2024-01-29_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 clxscore=1015 adultscore=0 mlxscore=0 spamscore=0 malwarescore=0 impostorscore=0 priorityscore=1501 phishscore=0 mlxlogscore=999 lowpriorityscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2401190000 definitions=main-2401290160 Message-ID-Hash: 7MLP6TLJRHEAJZOSAGENYQETSTDMX37M X-Message-ID-Hash: 7MLP6TLJRHEAJZOSAGENYQETSTDMX37M X-MailFrom: mgalaxy@akamai.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-config-1; header-match-config-2; header-match-config-3; header-match-devel.lists.libvirt.org-0; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; suspicious-header CC: steven.sistare@oracle.com, johunt@akamai.com, cweng@akamai.com, bchaney@akamai.com, trakotoz@akamai.com, Michael Galaxy X-Mailman-Version: 3.2.2 Precedence: list List-Id: Development discussions about the libvirt library & tools Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZM-MESSAGEID: 1706567174646100001 From: Michael Galaxy We have different use cases: 1. The domain has multiple NUMA nodes, but they have only specified a single directory pathin qemu.conf (Original default behavior) 2. Domain has multiple NUMA nodes, but we have asked for multiple directory paths as well (new behavior). 3. Domain has single NUMA node, but we have asked for multiple directory paths (new behavior). Each one is elaborated more inline below in the comments. Signed-off-by: Michael Galaxy --- src/qemu/qemu_command.c | 8 +++- src/qemu/qemu_conf.c | 101 ++++++++++++++++++++++++++++++++++++---- src/qemu/qemu_conf.h | 11 +++-- 3 files changed, 106 insertions(+), 14 deletions(-) diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c index b8f071ff2a..818e409d20 100644 --- a/src/qemu/qemu_command.c +++ b/src/qemu/qemu_command.c @@ -3448,7 +3448,9 @@ qemuBuildMemoryBackendProps(virJSONValue **backendPro= ps, } else { /* We can have both pagesize and mem source. If that's the cas= e, * prefer hugepages as those are more specific. */ - if (qemuGetMemoryBackingPath(priv->driver, def, mem->info.alia= s, &memPath) < 0) + + virDomainXMLPrivateDataCallbacks *privateData =3D (virDomainXM= LPrivateDataCallbacks *) priv; + if (qemuGetMemoryBackingPath(def, privateData, mem->targetNode= , mem->info.alias, &memPath) < 0) return -1; } =20 @@ -7291,7 +7293,9 @@ qemuBuildMemPathStr(const virDomainDef *def, return -1; prealloc =3D true; } else if (def->mem.source =3D=3D VIR_DOMAIN_MEMORY_SOURCE_FILE) { - if (qemuGetMemoryBackingPath(priv->driver, def, "ram", &mem_path) = < 0) + // This path should not be reached if NUMA is requested + virDomainXMLPrivateDataCallbacks *privateData =3D (virDomainXMLPri= vateDataCallbacks *) priv; + if (qemuGetMemoryBackingPath(def, privateData, 0, "ram", &mem_path= ) < 0) return -1; } =20 diff --git a/src/qemu/qemu_conf.c b/src/qemu/qemu_conf.c index aae9f316d8..e327a906a3 100644 --- a/src/qemu/qemu_conf.c +++ b/src/qemu/qemu_conf.c @@ -1622,22 +1622,106 @@ qemuGetDomainHupageMemPath(virQEMUDriver *driver, =20 =20 int -qemuGetMemoryBackingDomainPath(virQEMUDriver *driver, - const virDomainDef *def, +qemuGetMemoryBackingDomainPath(const virDomainDef *def, + virDomainXMLPrivateDataCallbacks *priv, + const size_t targetNode, char **path) { + qemuDomainObjPrivate *privateData =3D (qemuDomainObjPrivate *) priv; + virQEMUDriver *driver =3D privateData->driver; g_autoptr(virQEMUDriverConfig) cfg =3D virQEMUDriverGetConfig(driver); const char *root =3D driver->embeddedRoot; g_autofree char *shortName =3D NULL; + size_t path_index =3D 0; // original behavior, described below =20 if (!(shortName =3D virDomainDefGetShortName(def))) return -1; =20 - if (root && !STRPREFIX(cfg->memoryBackingDir, root)) { + /* + * We have three use cases: + * + * 1. Domain has multiple NUMA nodes, but they have only specified + * a single directory path in qemu.conf. (Original default behavior= ). + * + * In this case, we already placed the memory backing path for each= NUMA node + * into the same path location. Preserve the established default be= havior. + * + * 2. Domain has multiple NUMA nodes, but we have asked for multiple d= irectory + * paths as well. + * + * In this case, we will have a one-to-one relationship between the= number + * of NUMA nodes and the order in which the paths are provided. + * If the user does not specify enough paths, then we need to throw= an error. + * NOTE: This is open to comment. The "ordering" of the paths here = is not intially + * configurable to preserve backwards compatibility with the origin= al qemu.conf syntax. + * If controlling the ordering is desired, we would need to revise = the syntax in + * qemu.conf to make that possible. That hasn't been needed so far. + * + * NOTE A): We must check with numatune here, if requested. The num= ber of NUMA nodes + * may be less than or equal to the number of provided paths. If i= t is less, + * we have to respect the choices made by numatune. In this case, = we will map the + * physical NUMA nodes (0, 1, 2...) in the order in which the= y appear in qemu.conf + * + * 3. Domain has a single NUMA node, but we have asked for multiple di= rectory paths. + * + * In this case we also need to check if numatune is requested. If = so, + * we want to pick the path indicated by numatune. + * + * NOTE B): In both cases 2 and 3, if numatune is requested, the path = obviously cannot + * be changed on the fly, like it normally would be in "restri= ctive" mode + * during runtime. So, we will only do this is the mode reques= ted is "strict". + * + * NOTE C): Furthermore, in both cases 2 and 3, if the number of direc= tory paths provided + * is more than one, and one of either: a) no numatune is provided a= t all or + * b) numatune is in fact provided, but the mode is not stric= t, + * then we must thrown error. This is because we cannot know = which backing + * directory path to choose without the user's input. + * + * NOTE D): If one or more directory paths is requested in any of the = cases 1, 2, or 3, + * the numatune cannot specifiy more than one NUMA node, beca= use the only mode + * possible with directory paths is "strict" (e.g. automatic = numa balancing of + * memory will not work). Only one numa node can be requested= by numatune, else + * we must throw an error. + */ + + if (cfg->nb_memoryBackingDirs > 1) { + virDomainNuma *numatune =3D def->numa; + virBitmap *numaBitmap =3D virDomainNumatuneGetNodeset(numatune, priva= teData->autoNodeset, targetNode); + size_t numa_node_count =3D virDomainNumaGetNodeCount(def->numa); + virDomainNumatuneMemMode mode; + + if ((numatune && numaBitmap && virNumaNodesetIsAvailable(numaBitmap))= && + virDomainNumatuneGetMode(def->numa, -1, &mode) =3D=3D 0 && + mode =3D=3D VIR_DOMAIN_NUMATUNE_MEM_STRICT && + virBitmapCountBits(numaBitmap) =3D=3D 1) { + // Is numatune provided? + // Is it strict? + // Does it only specify a single pinning for this target? + // Yes to all 3? then good to go. + + if (cfg->nb_memoryBackingDirs < numa_node_count) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Domain requesting configuration for %lu NUMA nodes, but memor= y backing directory only has (%lu) directory paths available. Either reduce= this to one directory or provide more paths to use."), numa_node_count, cf= g->nb_memoryBackingDirs); + return -1; + } + + path_index =3D virBitmapNextSetBit(numaBitmap, -1); + } else if (numa_node_count > 1 && numa_node_count =3D=3D cfg->nb_memo= ryBackingDirs) { + // Be nice. A valid numatune and pinning has not been specified, but the= number + // of paths matches up exactly, so just assign them one-to-one. + path_index =3D targetNode; + } else { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("There are (%lu) memory directory directories configured. Doma= in must use a 'strict' numatune as well as an associated pinning configurat= ion for each NUMA node before proceeding. An individual NUMA node can only = be pinned to a single backing directory. Please correct the domain configur= ation or remove the memory backing directories and try again."), cfg->nb_me= moryBackingDirs); + return -1; + } + } + + if (root && !STRPREFIX(cfg->memoryBackingDirs[path_index], root)) { g_autofree char * hash =3D virDomainDriverGenerateRootHash("qemu",= root); - *path =3D g_strdup_printf("%s/%s-%s", cfg->memoryBackingDir, hash,= shortName); + *path =3D g_strdup_printf("%s/%s-%s", cfg->memoryBackingDirs[path_= index], hash, shortName); } else { - *path =3D g_strdup_printf("%s/%s", cfg->memoryBackingDir, shortNam= e); + *path =3D g_strdup_printf("%s/%s", cfg->memoryBackingDirs[path_ind= ex], shortName); } =20 return 0; @@ -1657,8 +1741,9 @@ qemuGetMemoryBackingDomainPath(virQEMUDriver *driver, * -1 otherwise (with error reported). */ int -qemuGetMemoryBackingPath(virQEMUDriver *driver, - const virDomainDef *def, +qemuGetMemoryBackingPath(const virDomainDef *def, + virDomainXMLPrivateDataCallbacks *priv, + const size_t targetNode, const char *alias, char **memPath) { @@ -1671,7 +1756,7 @@ qemuGetMemoryBackingPath(virQEMUDriver *driver, return -1; } =20 - if (qemuGetMemoryBackingDomainPath(driver, def, &domainPath) < 0) + if (qemuGetMemoryBackingDomainPath(def, priv, targetNode, &domainPath)= < 0) return -1; =20 *memPath =3D g_strdup_printf("%s/%s", domainPath, alias); diff --git a/src/qemu/qemu_conf.h b/src/qemu/qemu_conf.h index 2b8d540df0..4ae21524f7 100644 --- a/src/qemu/qemu_conf.h +++ b/src/qemu/qemu_conf.h @@ -370,11 +370,14 @@ int qemuGetDomainHupageMemPath(virQEMUDriver *driver, unsigned long long pagesize, char **memPath); =20 -int qemuGetMemoryBackingDomainPath(virQEMUDriver *driver, - const virDomainDef *def, +int qemuGetMemoryBackingDomainPath(const virDomainDef *def, + virDomainXMLPrivateDataCallbacks *priv, + const size_t targetNode, char **path); -int qemuGetMemoryBackingPath(virQEMUDriver *driver, - const virDomainDef *def, + +int qemuGetMemoryBackingPath(const virDomainDef *def, + virDomainXMLPrivateDataCallbacks *priv, + const size_t targetNode, const char *alias, char **memPath); =20 --=20 2.25.1 _______________________________________________ Devel mailing list -- devel@lists.libvirt.org To unsubscribe send an email to devel-leave@lists.libvirt.org From nobody Thu May 9 18:49:36 2024 Delivered-To: importer@patchew.org Received-SPF: none (zohomail.com: 8.43.85.245 is neither permitted nor denied by domain of lists.libvirt.org) client-ip=8.43.85.245; envelope-from=devel-bounces@lists.libvirt.org; helo=lists.libvirt.org; Authentication-Results: mx.zohomail.com; spf=none (zohomail.com: 8.43.85.245 is neither permitted nor denied by domain of lists.libvirt.org) smtp.mailfrom=devel-bounces@lists.libvirt.org; dmarc=fail(p=quarantine dis=none) header.from=akamai.com Return-Path: Received: from lists.libvirt.org (lists.libvirt.org [8.43.85.245]) by mx.zohomail.com with SMTPS id 1706566995604868.7444118463264; Mon, 29 Jan 2024 14:23:15 -0800 (PST) Received: by lists.libvirt.org (Postfix, from userid 996) id 5098419F6; Mon, 29 Jan 2024 17:23:14 -0500 (EST) Received: from lists.libvirt.org.85.43.8.in-addr.arpa (localhost [IPv6:::1]) by lists.libvirt.org (Postfix) with ESMTP id 34272185B; Mon, 29 Jan 2024 17:22:00 -0500 (EST) Received: by lists.libvirt.org (Postfix, from userid 996) id E6CDF19F6; Mon, 29 Jan 2024 17:21:56 -0500 (EST) Received: from mx0a-00190b01.pphosted.com (mx0a-00190b01.pphosted.com [67.231.149.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.libvirt.org (Postfix) with ESMTPS id C89B7185B for ; Mon, 29 Jan 2024 17:21:55 -0500 (EST) Received: from pps.filterd (m0050095.ppops.net [127.0.0.1]) by m0050095.ppops.net-00190b01. (8.17.1.24/8.17.1.24) with ESMTP id 40TI7NLk022513; Mon, 29 Jan 2024 21:44:46 GMT Received: from prod-mail-ppoint1 (prod-mail-ppoint1.akamai.com [184.51.33.18] (may be forged)) by m0050095.ppops.net-00190b01. (PPS) with ESMTPS id 3vvt88bard-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 29 Jan 2024 21:44:45 +0000 (GMT) Received: from pps.filterd (prod-mail-ppoint1.akamai.com [127.0.0.1]) by prod-mail-ppoint1.akamai.com (8.17.1.19/8.17.1.19) with ESMTP id 40TL5gpE013501; Mon, 29 Jan 2024 16:44:44 -0500 Received: from prod-mail-relay10.akamai.com ([172.27.118.251]) by prod-mail-ppoint1.akamai.com (PPS) with ESMTP id 3vvx22v4v8-1; Mon, 29 Jan 2024 16:44:44 -0500 Received: from bos-lhvcg5.bos01.corp.akamai.com (bos-lhvcg5.bos01.corp.akamai.com [172.28.221.19]) by prod-mail-relay10.akamai.com (Postfix) with ESMTP id 40FA364417; Mon, 29 Jan 2024 21:44:44 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on lists.libvirt.org X-Spam-Level: ** X-Spam-Status: No, score=2.1 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_LOW,RCVD_IN_MSPIKE_H4, RCVD_IN_MSPIKE_WL,RCVD_IN_SBL_CSS,SPF_HELO_NONE,T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.4 From: mgalaxy@akamai.com To: devel@lists.libvirt.org Subject: [PATCH 4/5] Support live migration between file-backed memory and anonymous memory. Date: Mon, 29 Jan 2024 16:43:56 -0500 Message-Id: <20240129214357.1281805-5-mgalaxy@akamai.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240129214357.1281805-1-mgalaxy@akamai.com> References: <20240129214357.1281805-1-mgalaxy@akamai.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2024-01-29_14,2024-01-29_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 phishscore=0 malwarescore=0 mlxlogscore=999 bulkscore=0 suspectscore=0 mlxscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2311290000 definitions=main-2401290159 X-Proofpoint-ORIG-GUID: YdoBVtsC5uYLBLXrU3uFhjRy7N7CBaEV X-Proofpoint-GUID: YdoBVtsC5uYLBLXrU3uFhjRy7N7CBaEV X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2024-01-29_14,2024-01-29_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 priorityscore=1501 mlxlogscore=999 clxscore=1015 phishscore=0 spamscore=0 mlxscore=0 suspectscore=0 lowpriorityscore=0 impostorscore=0 malwarescore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2401190000 definitions=main-2401290160 Message-ID-Hash: QCGW6ZPKV7KPCQCT5DPNUJJECZUDMTEZ X-Message-ID-Hash: QCGW6ZPKV7KPCQCT5DPNUJJECZUDMTEZ X-MailFrom: mgalaxy@akamai.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-config-1; header-match-config-2; header-match-config-3; header-match-devel.lists.libvirt.org-0; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; suspicious-header CC: steven.sistare@oracle.com, johunt@akamai.com, cweng@akamai.com, bchaney@akamai.com, trakotoz@akamai.com, Michael Galaxy X-Mailman-Version: 3.2.2 Precedence: list List-Id: Development discussions about the libvirt library & tools Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZM-MESSAGEID: 1706566997276100001 From: Michael Galaxy In our environment, we need to convert VMs into a live-update-comptabile configuration "on-the-fly" (via live migration). So, for this very specific case, this needs to work when PMEM is being enabled. QEMU does not have a problem with this at all, but we need to relax the rules here a bit. Signed-off-by: Michael Galaxy --- src/qemu/qemu_domain.c | 24 +++++++++++++++++++----- 1 file changed, 19 insertions(+), 5 deletions(-) diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c index de36641137..5f2058c58d 100644 --- a/src/qemu/qemu_domain.c +++ b/src/qemu/qemu_domain.c @@ -8520,11 +8520,25 @@ qemuDomainABIStabilityCheck(const virDomainDef *src, size_t i; =20 if (src->mem.source !=3D dst->mem.source) { - virReportError(VIR_ERR_CONFIG_UNSUPPORTED, - _("Target memoryBacking source '%1$s' doesn't match= source memoryBacking source'%2$s'"), - virDomainMemorySourceTypeToString(dst->mem.source), - virDomainMemorySourceTypeToString(src->mem.source)); - return false; + /* + * The current use case for this is the live migration of live-update + * capable CPR guests mounted on PMEM devices at the host + * level (not in-guest PMEM). QEMU has no problem doing these kinds of + * live migrations between these two memory backends, so let them go thro= ugh. + * This allows us to "upgrade" guests from regular memory to file-backed + * memory seemlessly without taking them down. + */ + if (!((src->mem.source =3D=3D VIR_DOMAIN_MEMORY_SOURCE_NONE + && dst->mem.source =3D=3D VIR_DOMAIN_MEMORY_SOURCE_FILE) || + (src->mem.source =3D=3D VIR_DOMAIN_MEMORY_SOURCE_FILE + && dst->mem.source =3D=3D VIR_DOMAIN_MEMORY_SOURCE_NONE))) { + + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, + _("Target memoryBacking source '%1$s' doesn't match source memor= yBacking source'%2$s'"), + virDomainMemorySourceTypeToString(dst->mem.source), + virDomainMemorySourceTypeToString(src->mem.source)); + return false; + } } =20 for (i =3D 0; i < src->nmems; i++) { --=20 2.25.1 _______________________________________________ Devel mailing list -- devel@lists.libvirt.org To unsubscribe send an email to devel-leave@lists.libvirt.org From nobody Thu May 9 18:49:36 2024 Delivered-To: importer@patchew.org Received-SPF: none (zohomail.com: 8.43.85.245 is neither permitted nor denied by domain of lists.libvirt.org) client-ip=8.43.85.245; envelope-from=devel-bounces@lists.libvirt.org; helo=lists.libvirt.org; Authentication-Results: mx.zohomail.com; spf=none (zohomail.com: 8.43.85.245 is neither permitted nor denied by domain of lists.libvirt.org) smtp.mailfrom=devel-bounces@lists.libvirt.org; dmarc=fail(p=quarantine dis=none) header.from=akamai.com Return-Path: Received: from lists.libvirt.org (lists.libvirt.org [8.43.85.245]) by mx.zohomail.com with SMTPS id 170656686301710.451058168252871; Mon, 29 Jan 2024 14:21:03 -0800 (PST) Received: by lists.libvirt.org (Postfix, from userid 996) id D51EF1BB4; Mon, 29 Jan 2024 17:21:01 -0500 (EST) Received: from lists.libvirt.org.85.43.8.in-addr.arpa (localhost [IPv6:::1]) by lists.libvirt.org (Postfix) with ESMTP id EDCFA171E; Mon, 29 Jan 2024 17:19:56 -0500 (EST) Received: by lists.libvirt.org (Postfix, from userid 996) id 3F62119F6; Mon, 29 Jan 2024 17:19:54 -0500 (EST) Received: from mx0a-00190b01.pphosted.com (mx0a-00190b01.pphosted.com [67.231.149.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.libvirt.org (Postfix) with ESMTPS id 81544185B for ; Mon, 29 Jan 2024 17:19:51 -0500 (EST) Received: from pps.filterd (m0122333.ppops.net [127.0.0.1]) by mx0a-00190b01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 40TI7NZT008538; Mon, 29 Jan 2024 21:44:46 GMT Received: from prod-mail-ppoint6 (prod-mail-ppoint6.akamai.com [184.51.33.61] (may be forged)) by mx0a-00190b01.pphosted.com (PPS) with ESMTPS id 3vvsybstmy-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 29 Jan 2024 21:44:46 +0000 (GMT) Received: from pps.filterd (prod-mail-ppoint6.akamai.com [127.0.0.1]) by prod-mail-ppoint6.akamai.com (8.17.1.19/8.17.1.19) with ESMTP id 40TL8DWj002205; Mon, 29 Jan 2024 16:44:45 -0500 Received: from prod-mail-relay10.akamai.com ([172.27.118.251]) by prod-mail-ppoint6.akamai.com (PPS) with ESMTP id 3vvx22v8hm-1; Mon, 29 Jan 2024 16:44:45 -0500 Received: from bos-lhvcg5.bos01.corp.akamai.com (bos-lhvcg5.bos01.corp.akamai.com [172.28.221.19]) by prod-mail-relay10.akamai.com (Postfix) with ESMTP id 579E964409; Mon, 29 Jan 2024 21:44:44 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on lists.libvirt.org X-Spam-Level: X-Spam-Status: No, score=-1.5 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_LOW,RCVD_IN_MSPIKE_H4, RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.4 X-Greylist: delayed 2104 seconds by postgrey-1.37 at lists.libvirt.org; Mon, 29 Jan 2024 17:19:51 EST From: mgalaxy@akamai.com To: devel@lists.libvirt.org Subject: [PATCH 5/5] Update unit test to support multiple memory backends Date: Mon, 29 Jan 2024 16:43:57 -0500 Message-Id: <20240129214357.1281805-6-mgalaxy@akamai.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240129214357.1281805-1-mgalaxy@akamai.com> References: <20240129214357.1281805-1-mgalaxy@akamai.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2024-01-29_14,2024-01-29_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 phishscore=0 bulkscore=0 spamscore=0 adultscore=0 suspectscore=0 mlxlogscore=999 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2311290000 definitions=main-2401290159 X-Proofpoint-ORIG-GUID: bvnRen1QpRTwavr1wGTyRZFjeriDW_3F X-Proofpoint-GUID: bvnRen1QpRTwavr1wGTyRZFjeriDW_3F X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2024-01-29_14,2024-01-29_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 malwarescore=0 clxscore=1015 suspectscore=0 phishscore=0 mlxlogscore=998 impostorscore=0 mlxscore=0 adultscore=0 bulkscore=0 priorityscore=1501 spamscore=0 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2401190000 definitions=main-2401290160 Message-ID-Hash: NAKW3I55MX7P6VFD2DK4UZEKN4T4JBJV X-Message-ID-Hash: NAKW3I55MX7P6VFD2DK4UZEKN4T4JBJV X-MailFrom: mgalaxy@akamai.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-config-1; header-match-config-2; header-match-config-3; header-match-devel.lists.libvirt.org-0; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; suspicious-header CC: steven.sistare@oracle.com, johunt@akamai.com, cweng@akamai.com, bchaney@akamai.com, trakotoz@akamai.com, Michael Galaxy X-Mailman-Version: 3.2.2 Precedence: list List-Id: Development discussions about the libvirt library & tools Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZM-MESSAGEID: 1706566864964100001 From: Michael Galaxy Signed-off-by: Michael Galaxy --- tests/testutilsqemu.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/tests/testutilsqemu.c b/tests/testutilsqemu.c index 9c12a165b1..0da11d83c0 100644 --- a/tests/testutilsqemu.c +++ b/tests/testutilsqemu.c @@ -320,8 +320,9 @@ int qemuTestDriverInit(virQEMUDriver *driver) cfg->libDir =3D g_strdup("/var/lib/libvirt/qemu"); VIR_FREE(cfg->channelTargetDir); cfg->channelTargetDir =3D g_strdup("/var/run/libvirt/qemu/channel"); - VIR_FREE(cfg->memoryBackingDir); - cfg->memoryBackingDir =3D g_strdup("/var/lib/libvirt/qemu/ram"); + VIR_FREE(cfg->memoryBackingDirs); + cfg->memoryBackingDirs =3D g_new0(char *, 1); + cfg->memoryBackingDirs[0] =3D g_strdup("/var/lib/libvirt/qemu/ram"); VIR_FREE(cfg->nvramDir); cfg->nvramDir =3D g_strdup("/var/lib/libvirt/qemu/nvram"); VIR_FREE(cfg->passtStateDir); --=20 2.25.1 _______________________________________________ Devel mailing list -- devel@lists.libvirt.org To unsubscribe send an email to devel-leave@lists.libvirt.org