From nobody Fri Oct 18 05:21:46 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.libvirt.org designates 8.43.85.245 as permitted sender) client-ip=8.43.85.245; envelope-from=devel-bounces@lists.libvirt.org; helo=lists.libvirt.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of lists.libvirt.org designates 8.43.85.245 as permitted sender) smtp.mailfrom=devel-bounces@lists.libvirt.org; dmarc=pass(p=reject dis=none) header.from=lists.libvirt.org ARC-Seal: i=1; a=rsa-sha256; t=1717620059; cv=none; d=zohomail.com; s=zohoarc; b=m2l0XTtCXaGB/+xiLVg9mCMdhUYZ2pTUn4o/6QnQbt9cmE7WCKZNh5MEww+mvfkwqBDiOeJ+E0RQFIivExiij0G/rkoHjl+fqgA4fKGRhOsYU6SP+F0pHKlIkYF244CIJEm5Eq5D4RPNBwviU2YxR9W0Uw4dUxAJLa5lopR5LK8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1717620059; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Reply-To:Reply-To:References:Subject:Subject:To:To:Message-Id; bh=Jq1KDZznSRA/b3za/XGRnD0sFTDKE2tRz3EcJmEKFN4=; b=NAJSfwWqHRJ00Jr8gZxhG6fwpsSPTqW4LFNSVIIzZhDFPKT5owSV381LZiMsjs0JENqtNZbPI5RFLnaTi6lESgWBqvELApFWpD8amw3id0Tt6Sd1i2m/ECI8pMTNXLcgLiAanSZdgzb6PUXfjxSvr/pjkSaIwxdFGd7q4yGpJzs= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of lists.libvirt.org designates 8.43.85.245 as permitted sender) smtp.mailfrom=devel-bounces@lists.libvirt.org; dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.libvirt.org (lists.libvirt.org [8.43.85.245]) by mx.zohomail.com with SMTPS id 1717620059100533.5978522086951; Wed, 5 Jun 2024 13:40:59 -0700 (PDT) Received: by lists.libvirt.org (Postfix, from userid 996) id EC2C32118; Wed, 5 Jun 2024 16:40:57 -0400 (EDT) Received: from lists.libvirt.org (localhost [IPv6:::1]) by lists.libvirt.org (Postfix) with ESMTP id A0F1D21A9; Wed, 5 Jun 2024 16:38:24 -0400 (EDT) Received: by lists.libvirt.org (Postfix, from userid 996) id 5D6952161; Wed, 5 Jun 2024 16:38:15 -0400 (EDT) Received: from mx0b-00190b01.pphosted.com (mx0b-00190b01.pphosted.com [67.231.157.127]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.libvirt.org (Postfix) with ESMTPS id 3D77B2131 for ; Wed, 5 Jun 2024 16:37:58 -0400 (EDT) Received: from pps.filterd (m0050096.ppops.net [127.0.0.1]) by m0050096.ppops.net-00190b01. (8.18.1.2/8.18.1.2) with ESMTP id 455DC7wo017143; Wed, 5 Jun 2024 21:37:57 +0100 Received: from prod-mail-ppoint4 (a72-247-45-32.deploy.static.akamaitechnologies.com [72.247.45.32] (may be forged)) by m0050096.ppops.net-00190b01. (PPS) with ESMTPS id 3yheubpfrr-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 05 Jun 2024 21:37:57 +0100 (BST) Received: from pps.filterd (prod-mail-ppoint4.akamai.com [127.0.0.1]) by prod-mail-ppoint4.akamai.com (8.17.1.19/8.17.1.19) with ESMTP id 455JTJOn002540; Wed, 5 Jun 2024 16:37:56 -0400 Received: from prod-mail-relay10.akamai.com ([172.27.118.251]) by prod-mail-ppoint4.akamai.com (PPS) with ESMTP id 3yjeehna5p-1; Wed, 05 Jun 2024 16:37:56 -0400 Received: from bos-lhvcg5.bos01.corp.akamai.com (bos-lhvcg5.bos01.corp.akamai.com [172.28.222.58]) by prod-mail-relay10.akamai.com (Postfix) with ESMTP id D42B265191; Wed, 5 Jun 2024 20:37:55 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on lists.libvirt.org X-Spam-Level: X-Spam-Status: No, score=-1.5 required=5.0 tests=DKIM_INVALID,DKIM_SIGNED, MAILING_LIST_MULTI,RCVD_IN_DNSWL_LOW,RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.4 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=akamai.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=jan2016.eng; bh=Sb6l2Snv0 +GhLq5GxPYih5lzN9oDXyjbjgQI64wFADI=; b=Wv/6IORuDGZcH3wCFKIAHoorl HhvecrnSd51fG9oXbqPK38FPx0iP8aXb5C3CWpEhLgvPWc3lcGdhn/oEYRMrabVK Imdozq2MfLsCzgERPyc7L4Z8efzkvUsVRRWGhlr2qtEQi9Zv6zhpQvw/pWabSFTH bbexS53ibbZG1dH+eZRk+e//vklwMZb5BuZ8C5lX0u6q83vyRnjghJZ4mzJev7dQ AdyWIKT9lvUCIybSB8R2OASFWY4L3OxQjYkWgT7MGvvGsC+MyULbLRE6WLWWrIDI sMmWg+oFX+uIhY/Pf72223OgLzyjlUUaLDGrg+nJM68Jf+gcXitv8L2NSYsXA== To: devel@lists.libvirt.org Subject: [PATCH v2 1/4] qemu.conf changes to support multiple memory backend Date: Wed, 5 Jun 2024 16:37:35 -0400 Message-Id: <20240605203738.1880333-2-mgalaxy@akamai.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240605203738.1880333-1-mgalaxy@akamai.com> References: <20240605203738.1880333-1-mgalaxy@akamai.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.28.16 definitions=2024-06-05_02,2024-06-05_02,2024-05-17_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 mlxscore=0 adultscore=0 bulkscore=0 mlxlogscore=999 phishscore=0 malwarescore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2405010000 definitions=main-2406050156 X-Proofpoint-GUID: SGZIJCD1c1nHRyfmnKH2m7FEvA5u8Qdq X-Proofpoint-ORIG-GUID: SGZIJCD1c1nHRyfmnKH2m7FEvA5u8Qdq X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.28.16 definitions=2024-06-05_02,2024-06-05_02,2024-05-17_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 lowpriorityscore=0 mlxscore=0 mlxlogscore=999 phishscore=0 bulkscore=0 clxscore=1015 adultscore=0 malwarescore=0 priorityscore=1501 suspectscore=0 impostorscore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2405170001 definitions=main-2406050156 Content-Transfer-Encoding: quoted-printable Message-ID-Hash: UINSYG7VAHFIAUK5UZVSUBDK5I6V54Z7 X-Message-ID-Hash: UINSYG7VAHFIAUK5UZVSUBDK5I6V54Z7 X-MailFrom: mgalaxy@akamai.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-config-1; header-match-config-2; header-match-config-3; header-match-devel.lists.libvirt.org-0; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; suspicious-header CC: steven.sistare@oracle.com, johunt@akamai.com, cweng@akamai.com, bchaney@akamai.com, trakotoz@akamai.com, mkletzan@redhat.com, anthony.yznaga@oracle.com, mprivozn@redhat.com, Michael Galaxy X-Mailman-Version: 3.2.2 Precedence: list List-Id: Development discussions about the libvirt library & tools Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: From: Michael Galaxy via Devel Reply-To: mgalaxy@akamai.com X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1717620059771100001 Content-Type: text/plain; charset="utf-8" From: Michael Galaxy We start by introducing a backwards-compatible, comma-separated specification that will not break existing installations, such as in the following example: $ cat qemu.conf | grep memory_backing_dir memory_backing_dir =3D ["/path/to/pmem/0", "/path/to/pmem/1"] (The old syntax with a single string is also still supported) memory_backing_dir =3D "/path/to/dir" In our case, we almost always have two NUMA nodes, so in that example, we have two PMEM regions which are created on the Linux kernel command line that get mounted into those two locations for libvirt to use. Signed-off-by: Michael Galaxy --- src/qemu/qemu_command.c | 8 ++- src/qemu/qemu_conf.c | 141 +++++++++++++++++++++++++++++++++++----- src/qemu/qemu_conf.h | 14 ++-- src/qemu/qemu_driver.c | 29 +++++---- src/qemu/qemu_hotplug.c | 6 +- src/qemu/qemu_process.c | 44 +++++++------ src/qemu/qemu_process.h | 7 +- 7 files changed, 188 insertions(+), 61 deletions(-) diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c index 2d0eddc79e..9a25fdb0ed 100644 --- a/src/qemu/qemu_command.c +++ b/src/qemu/qemu_command.c @@ -3420,7 +3420,9 @@ qemuBuildMemoryBackendProps(virJSONValue **backendPro= ps, } else { /* We can have both pagesize and mem source. If that's the cas= e, * prefer hugepages as those are more specific. */ - if (qemuGetMemoryBackingPath(priv->driver, def, mem->info.alia= s, &memPath) < 0) + + virDomainXMLPrivateDataCallbacks *privateData =3D (virDomainXM= LPrivateDataCallbacks *) priv; + if (qemuGetMemoryBackingPath(def, privateData, mem->targetNode= , mem->info.alias, &memPath) < 0) return -1; } =20 @@ -7295,7 +7297,9 @@ qemuBuildMemPathStr(const virDomainDef *def, return -1; prealloc =3D true; } else if (def->mem.source =3D=3D VIR_DOMAIN_MEMORY_SOURCE_FILE) { - if (qemuGetMemoryBackingPath(priv->driver, def, "ram", &mem_path) = < 0) + // This path should not be reached if NUMA is requested + virDomainXMLPrivateDataCallbacks *privateData =3D (virDomainXMLPri= vateDataCallbacks *) priv; + if (qemuGetMemoryBackingPath(def, privateData, 0, "ram", &mem_path= ) < 0) return -1; } =20 diff --git a/src/qemu/qemu_conf.c b/src/qemu/qemu_conf.c index 4050a82341..b808e33b77 100644 --- a/src/qemu/qemu_conf.c +++ b/src/qemu/qemu_conf.c @@ -43,6 +43,7 @@ #include "virfile.h" #include "virstring.h" #include "virutil.h" +#include "virnuma.h" #include "configmake.h" #include "security/security_util.h" =20 @@ -137,6 +138,9 @@ virQEMUDriverConfig *virQEMUDriverConfigNew(bool privil= eged, =20 cfg->cgroupControllers =3D -1; /* -1 =3D=3D auto-detect */ =20 + cfg->memoryBackingDirs =3D g_new0(char *, 1); + cfg->nb_memoryBackingDirs =3D 1; + if (root !=3D NULL) { cfg->logDir =3D g_strdup_printf("%s/log/qemu", root); cfg->swtpmLogDir =3D g_strdup_printf("%s/log/swtpm", root); @@ -153,7 +157,7 @@ virQEMUDriverConfig *virQEMUDriverConfigNew(bool privil= eged, cfg->checkpointDir =3D g_strdup_printf("%s/checkpoint", cfg->libDi= r); cfg->autoDumpPath =3D g_strdup_printf("%s/dump", cfg->libDir); cfg->nvramDir =3D g_strdup_printf("%s/nvram", cfg->libDir); - cfg->memoryBackingDir =3D g_strdup_printf("%s/ram", cfg->libDir); + cfg->memoryBackingDirs[0] =3D g_strdup_printf("%s/ram", cfg->libDi= r); } else if (privileged) { cfg->logDir =3D g_strdup_printf("%s/log/libvirt/qemu", LOCALSTATED= IR); =20 @@ -174,7 +178,8 @@ virQEMUDriverConfig *virQEMUDriverConfigNew(bool privil= eged, cfg->checkpointDir =3D g_strdup_printf("%s/checkpoint", cfg->libDi= r); cfg->autoDumpPath =3D g_strdup_printf("%s/dump", cfg->libDir); cfg->nvramDir =3D g_strdup_printf("%s/nvram", cfg->libDir); - cfg->memoryBackingDir =3D g_strdup_printf("%s/ram", cfg->libDir); + + cfg->memoryBackingDirs[0] =3D g_strdup_printf("%s/ram", cfg->libDi= r); cfg->swtpmStorageDir =3D g_strdup_printf("%s/lib/libvirt/swtpm", LOCALSTATEDIR); } else { @@ -201,7 +206,7 @@ virQEMUDriverConfig *virQEMUDriverConfigNew(bool privil= eged, cfg->configBaseDir); cfg->autoDumpPath =3D g_strdup_printf("%s/qemu/dump", cfg->configB= aseDir); cfg->nvramDir =3D g_strdup_printf("%s/qemu/nvram", cfg->configBase= Dir); - cfg->memoryBackingDir =3D g_strdup_printf("%s/qemu/ram", cfg->conf= igBaseDir); + cfg->memoryBackingDirs[0] =3D g_strdup_printf("%s/qemu/ram", cfg->= configBaseDir); cfg->swtpmStorageDir =3D g_strdup_printf("%s/qemu/swtpm", cfg->configBaseDir); } @@ -294,6 +299,7 @@ virQEMUDriverConfig *virQEMUDriverConfigNew(bool privil= eged, static void virQEMUDriverConfigDispose(void *obj) { virQEMUDriverConfig *cfg =3D obj; + size_t i; =20 virBitmapFree(cfg->namespaces); =20 @@ -369,7 +375,12 @@ static void virQEMUDriverConfigDispose(void *obj) =20 virFirmwareFreeList(cfg->firmwares, cfg->nfirmwares); =20 - g_free(cfg->memoryBackingDir); + for (i =3D 0; i < cfg->nb_memoryBackingDirs; i++) { + g_free(cfg->memoryBackingDirs[i]); + } + + g_free(cfg->memoryBackingDirs); + g_free(cfg->swtpmStorageDir); =20 g_strfreev(cfg->capabilityfilters); @@ -1018,15 +1029,22 @@ static int virQEMUDriverConfigLoadMemoryEntry(virQEMUDriverConfig *cfg, virConf *conf) { - g_autofree char *dir =3D NULL; + char **memoryBackingDirs =3D NULL; + g_auto(GStrv) params =3D NULL; int rc; =20 - if ((rc =3D virConfGetValueString(conf, "memory_backing_dir", &dir)) <= 0) + if ((rc =3D virConfGetValueStringList(conf, "memory_backing_dir", true= , &memoryBackingDirs) < 0)) return -1; =20 - if (rc > 0) { - VIR_FREE(cfg->memoryBackingDir); - cfg->memoryBackingDir =3D g_strdup_printf("%s/libvirt/qemu", dir); + if (rc =3D=3D 0) { + size_t i; + for (i =3D 0; i < cfg->nb_memoryBackingDirs; i++) + g_free(cfg->memoryBackingDirs[i]); + + cfg->nb_memoryBackingDirs =3D g_strv_length(memoryBackingDirs); + cfg->memoryBackingDirs =3D g_new0(char *, cfg->nb_memoryBackingDir= s); + for (i =3D 0; i < cfg->nb_memoryBackingDirs; i++) + cfg->memoryBackingDirs[i] =3D g_strdup_printf("%s/libvirt/qemu= ", memoryBackingDirs[i]); } =20 return 0; @@ -1595,22 +1613,110 @@ qemuGetDomainHupageMemPath(virQEMUDriver *driver, =20 =20 int -qemuGetMemoryBackingDomainPath(virQEMUDriver *driver, - const virDomainDef *def, +qemuGetMemoryBackingDomainPath(const virDomainDef *def, + virDomainXMLPrivateDataCallbacks *priv, + const size_t targetNode, char **path) { + qemuDomainObjPrivate *privateData =3D (qemuDomainObjPrivate *) priv; + virQEMUDriver *driver =3D privateData->driver; g_autoptr(virQEMUDriverConfig) cfg =3D virQEMUDriverGetConfig(driver); const char *root =3D driver->embeddedRoot; g_autofree char *shortName =3D NULL; + size_t path_index =3D 0; // original behavior, described below =20 if (!(shortName =3D virDomainDefGetShortName(def))) return -1; =20 - if (root && !STRPREFIX(cfg->memoryBackingDir, root)) { + /* + * We have three use cases: + * + * 1. Domain has multiple NUMA nodes, but they have only specified + * a single directory path in qemu.conf. (Original default behavior= ). + * + * In this case, we already placed the memory backing path for each= NUMA node + * into the same path location. Preserve the established default be= havior. + * + * 2. Domain has multiple NUMA nodes, but we have asked for multiple d= irectory + * paths as well. + * + * In this case, we will have a one-to-one relationship between the= number + * of NUMA nodes and the order in which the paths are provided. + * If the user does not specify enough paths, then we need to throw= an error. + * NOTE: This is open to comment. The "ordering" of the paths here = is not intially + * configurable to preserve backwards compatibility with the origin= al qemu.conf syntax. + * If controlling the ordering is desired, we would need to revise = the syntax in + * qemu.conf to make that possible. That hasn't been needed so far. + * + * NOTE A): We must check with numatune here, if requested. The num= ber of NUMA nodes + * may be less than or equal to the number of provided pat= hs. If it is less, + * we have to respect the choices made by numatune. In thi= s case, we will map the + * physical NUMA nodes (0, 1, 2...) in the order in which = they appear in qemu.conf + * + * 3. Domain has a single NUMA node, but we have asked for multiple di= rectory paths. + * + * In this case we also need to check if numatune is requested. = If so, + * we want to pick the path indicated by numatune. + * + * NOTE B): In both cases 2 and 3, if numatune is requested, the path = obviously cannot + * be changed on the fly, like it normally would be in "restri= ctive" mode + * during runtime. So, we will only do this is the mode reques= ted is "strict". + * + * NOTE C): Furthermore, in both cases 2 and 3, if the number of direc= tory paths provided + * is more than one, and one of either: a) no numatune is pro= vided at all or + * b) numatune is in fact provided, but the mode is not stric= t, + * then we must thrown error. This is because we cannot know = which backing + * directory path to choose without the user's input. + * + * NOTE D): If one or more directory paths is requested in any of the = cases 1, 2, or 3, + * the numatune cannot specifiy more than one NUMA node, beca= use the only mode + * possible with directory paths is "strict" (e.g. automatic = numa balancing of + * memory will not work). Only one numa node can be requested= by numatune, else + * we must throw an error. + */ + + if (cfg->nb_memoryBackingDirs > 1) { + virDomainNuma *numatune =3D def->numa; + virBitmap *numaBitmap =3D virDomainNumatuneGetNodeset(numatune, pr= ivateData->autoNodeset, targetNode); + size_t numa_node_count =3D virDomainNumaGetNodeCount(def->numa); + virDomainNumatuneMemMode mode; + + if ((numatune && numaBitmap && virNumaNodesetIsAvailable(numaBitma= p)) && + virDomainNumatuneGetMode(def->numa, -1, &mode) =3D=3D 0 && + mode =3D=3D VIR_DOMAIN_NUMATUNE_MEM_STRICT && + virBitmapCountBits(numaBitmap) =3D=3D 1) { + + // Is numatune provided? + // Is it strict? + // Does it only specify a single pinning for this target? + // Yes to all 3? then good to go. + + if (cfg->nb_memoryBackingDirs < numa_node_count) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Domain requesting configuration for %1$l= u NUMA nodes, but memory backing directory only has (%2$lu) directory paths= available. Either reduce this to one directory or provide more paths to us= e."), + numa_node_count, + cfg->nb_memoryBackingDirs); + return -1; + } + + path_index =3D virBitmapNextSetBit(numaBitmap, -1); + } else if (numa_node_count > 1 && numa_node_count =3D=3D cfg->nb_m= emoryBackingDirs) { + // Be nice. A valid numatune and pinning has not been specifie= d, but the number + // of paths matches up exactly, so just assign them one-to-one. + path_index =3D targetNode; + } else { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("There are (%1$lu) memory directory directori= es configured. Domain must use a 'strict' numatune as well as an associated= pinning configuration for each NUMA node before proceeding. An individual = NUMA node can only be pinned to a single backing directory. Please correct = the domain configuration or remove the memory backing directories and try a= gain."), + cfg->nb_memoryBackingDirs); + return -1; + } + } + + if (root && !STRPREFIX(cfg->memoryBackingDirs[path_index], root)) { g_autofree char * hash =3D virDomainDriverGenerateRootHash("qemu",= root); - *path =3D g_strdup_printf("%s/%s-%s", cfg->memoryBackingDir, hash,= shortName); + *path =3D g_strdup_printf("%s/%s-%s", cfg->memoryBackingDirs[path_= index], hash, shortName); } else { - *path =3D g_strdup_printf("%s/%s", cfg->memoryBackingDir, shortNam= e); + *path =3D g_strdup_printf("%s/%s", cfg->memoryBackingDirs[path_ind= ex], shortName); } =20 return 0; @@ -1630,8 +1736,9 @@ qemuGetMemoryBackingDomainPath(virQEMUDriver *driver, * -1 otherwise (with error reported). */ int -qemuGetMemoryBackingPath(virQEMUDriver *driver, - const virDomainDef *def, +qemuGetMemoryBackingPath(const virDomainDef *def, + virDomainXMLPrivateDataCallbacks *priv, + const size_t targetNode, const char *alias, char **memPath) { @@ -1644,7 +1751,7 @@ qemuGetMemoryBackingPath(virQEMUDriver *driver, return -1; } =20 - if (qemuGetMemoryBackingDomainPath(driver, def, &domainPath) < 0) + if (qemuGetMemoryBackingDomainPath(def, priv, targetNode, &domainPath)= < 0) return -1; =20 *memPath =3D g_strdup_printf("%s/%s", domainPath, alias); diff --git a/src/qemu/qemu_conf.h b/src/qemu/qemu_conf.h index 36049b4bfa..dabf4d19a4 100644 --- a/src/qemu/qemu_conf.h +++ b/src/qemu/qemu_conf.h @@ -221,7 +221,8 @@ struct _virQEMUDriverConfig { unsigned int glusterDebugLevel; bool virtiofsdDebug; =20 - char *memoryBackingDir; + char **memoryBackingDirs; + size_t nb_memoryBackingDirs; =20 uid_t swtpm_user; gid_t swtpm_group; @@ -369,11 +370,14 @@ int qemuGetDomainHupageMemPath(virQEMUDriver *driver, unsigned long long pagesize, char **memPath); =20 -int qemuGetMemoryBackingDomainPath(virQEMUDriver *driver, - const virDomainDef *def, +int qemuGetMemoryBackingDomainPath(const virDomainDef *def, + virDomainXMLPrivateDataCallbacks *priv, + const size_t targetNode, char **path); -int qemuGetMemoryBackingPath(virQEMUDriver *driver, - const virDomainDef *def, + +int qemuGetMemoryBackingPath(const virDomainDef *def, + virDomainXMLPrivateDataCallbacks *priv, + const size_t targetNode, const char *alias, char **memPath); =20 diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index e2698c7924..51ca7dca78 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -653,11 +653,14 @@ qemuStateInitialize(bool privileged, cfg->nvramDir); goto error; } - if (g_mkdir_with_parents(cfg->memoryBackingDir, 0777) < 0) { - virReportSystemError(errno, _("Failed to create memory backing dir= %1$s"), - cfg->memoryBackingDir); - goto error; + for (i =3D 0; i < cfg->nb_memoryBackingDirs; i++) { + if (g_mkdir_with_parents(cfg->memoryBackingDirs[i], 0777) < 0) { + virReportSystemError(errno, _("Failed to create memory backing= dir # %1$lu @ %2$s, total: %3$lu"), + i, cfg->memoryBackingDirs[i], cfg->nb_memoryBackingDirs); + goto error; + } } + if (g_mkdir_with_parents(cfg->slirpStateDir, 0777) < 0) { virReportSystemError(errno, _("Failed to create slirp state dir %1= $s"), cfg->slirpStateDir); @@ -802,12 +805,14 @@ qemuStateInitialize(bool privileged, (int)cfg->group); goto error; } - if (chown(cfg->memoryBackingDir, cfg->user, cfg->group) < 0) { - virReportSystemError(errno, + for (i =3D 0; i < cfg->nb_memoryBackingDirs; i++) { + if (chown(cfg->memoryBackingDirs[i], cfg->user, cfg->group) < = 0) { + virReportSystemError(errno, _("unable to set ownership of '%1$s' to %= 2$d:%3$d"), - cfg->memoryBackingDir, (int)cfg->user, + cfg->memoryBackingDirs[i], (int)cfg->user, (int)cfg->group); - goto error; + goto error; + } } if (chown(cfg->slirpStateDir, cfg->user, cfg->group) < 0) { virReportSystemError(errno, @@ -855,10 +860,12 @@ qemuStateInitialize(bool privileged, goto error; } =20 - if (privileged && - virFileUpdatePerm(cfg->memoryBackingDir, + for (i =3D 0; i < cfg->nb_memoryBackingDirs; i++) { + if (privileged && + virFileUpdatePerm(cfg->memoryBackingDirs[i], 0, S_IXGRP | S_IXOTH) < 0) - goto error; + goto error; + } =20 /* Get all the running persistent or transient configs first */ if (virDomainObjListLoadAllConfigs(qemu_driver->domains, diff --git a/src/qemu/qemu_hotplug.c b/src/qemu/qemu_hotplug.c index 4a3f4f657e..59db29aada 100644 --- a/src/qemu/qemu_hotplug.c +++ b/src/qemu/qemu_hotplug.c @@ -2276,7 +2276,7 @@ qemuDomainAttachMemory(virQEMUDriver *driver, priv, vm->def, mem, true, false, NULL)= < 0) goto cleanup; =20 - if (qemuProcessBuildDestroyMemoryPaths(driver, vm, mem, true) < 0) + if (qemuProcessBuildDestroyMemoryPaths(vm, mem, true) < 0) goto cleanup; =20 if (qemuDomainNamespaceSetupMemory(vm, mem, &teardowndevice) < 0) @@ -2351,7 +2351,7 @@ qemuDomainAttachMemory(virQEMUDriver *driver, qemuDomainObjExitMonitor(vm); =20 if (objAdded && mem) - ignore_value(qemuProcessDestroyMemoryBackingPath(driver, vm, mem)); + ignore_value(qemuProcessDestroyMemoryBackingPath(vm->def, priv, me= m)); =20 virErrorRestore(&orig_err); if (!mem) @@ -4634,7 +4634,7 @@ qemuDomainRemoveMemoryDevice(virQEMUDriver *driver, if (qemuDomainNamespaceTeardownMemory(vm, mem) < 0) VIR_WARN("Unable to remove memory device from /dev"); =20 - if (qemuProcessDestroyMemoryBackingPath(driver, vm, mem) < 0) + if (qemuProcessDestroyMemoryBackingPath(vm->def, priv, mem) < 0) VIR_WARN("Unable to destroy memory backing path"); =20 qemuDomainReleaseMemoryDeviceSlot(vm, mem); diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index 7ef7040a85..a9af8fe353 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -4046,12 +4046,12 @@ qemuProcessBuildDestroyMemoryPathsImpl(virQEMUDrive= r *driver, =20 =20 int -qemuProcessBuildDestroyMemoryPaths(virQEMUDriver *driver, - virDomainObj *vm, +qemuProcessBuildDestroyMemoryPaths(virDomainObj *vm, virDomainMemoryDef *mem, bool build) { - + qemuDomainObjPrivate *priv =3D vm->privateData; + virQEMUDriver *driver =3D priv->driver; g_autoptr(virQEMUDriverConfig) cfg =3D virQEMUDriverGetConfig(driver); size_t i; bool shouldBuildHP =3D false; @@ -4081,13 +4081,14 @@ qemuProcessBuildDestroyMemoryPaths(virQEMUDriver *d= river, } =20 if (!build || shouldBuildMB) { - g_autofree char *path =3D NULL; - if (qemuGetMemoryBackingDomainPath(driver, vm->def, &path) < 0) - return -1; + for (i =3D 0; i < cfg->nb_memoryBackingDirs; i++) { + g_autofree char *path =3D NULL; + if (qemuGetMemoryBackingDomainPath(vm->def, vm->privateData, i= , &path) < 0) + return -1; =20 - if (qemuProcessBuildDestroyMemoryPathsImpl(driver, vm, - path, build) < 0) - return -1; + if (qemuProcessBuildDestroyMemoryPathsImpl(driver, vm, path, b= uild) < 0) + return -1; + } } =20 return 0; @@ -4095,19 +4096,24 @@ qemuProcessBuildDestroyMemoryPaths(virQEMUDriver *d= river, =20 =20 int -qemuProcessDestroyMemoryBackingPath(virQEMUDriver *driver, - virDomainObj *vm, +qemuProcessDestroyMemoryBackingPath(virDomainDef *def, + qemuDomainObjPrivate *priv, virDomainMemoryDef *mem) { + virQEMUDriver *driver =3D priv->driver; g_autofree char *path =3D NULL; + g_autoptr(virQEMUDriverConfig) cfg =3D virQEMUDriverGetConfig(driver); + size_t i; =20 - if (qemuGetMemoryBackingPath(driver, vm->def, mem->info.alias, &path) = < 0) - return -1; + for (i =3D 0; i < cfg->nb_memoryBackingDirs; i++) { + virDomainXMLPrivateDataCallbacks *privateData =3D (virDomainXMLPri= vateDataCallbacks *) priv; + if (qemuGetMemoryBackingPath(def, privateData, i, mem->info.alias,= &path) < 0) + return -1; =20 - if (unlink(path) < 0 && - errno !=3D ENOENT) { - virReportSystemError(errno, _("Unable to remove %1$s"), path); - return -1; + if (unlink(path) < 0 && errno !=3D ENOENT) { + virReportSystemError(errno, _("Unable to remove %1$s"), path); + return -1; + } } =20 return 0; @@ -7268,7 +7274,7 @@ qemuProcessPrepareHost(virQEMUDriver *driver, if (qemuProcessPrepareHostBackendChardev(vm) < 0) return -1; =20 - if (qemuProcessBuildDestroyMemoryPaths(driver, vm, NULL, true) < 0) + if (qemuProcessBuildDestroyMemoryPaths(vm, NULL, true) < 0) return -1; =20 /* Ensure no historical cgroup for this VM is lying around bogus @@ -8482,7 +8488,7 @@ void qemuProcessStop(virQEMUDriver *driver, goto endjob; } =20 - qemuProcessBuildDestroyMemoryPaths(driver, vm, NULL, false); + qemuProcessBuildDestroyMemoryPaths(vm, NULL, false); =20 if (!!g_atomic_int_dec_and_test(&driver->nactive) && driver->inhibitCa= llback) driver->inhibitCallback(false, driver->inhibitOpaque); diff --git a/src/qemu/qemu_process.h b/src/qemu/qemu_process.h index c1ea949215..eebd0a4d1b 100644 --- a/src/qemu/qemu_process.h +++ b/src/qemu/qemu_process.h @@ -38,13 +38,12 @@ int qemuProcessStopCPUs(virQEMUDriver *driver, virDomainPausedReason reason, virDomainAsyncJob asyncJob); =20 -int qemuProcessBuildDestroyMemoryPaths(virQEMUDriver *driver, - virDomainObj *vm, +int qemuProcessBuildDestroyMemoryPaths(virDomainObj *vm, virDomainMemoryDef *mem, bool build); =20 -int qemuProcessDestroyMemoryBackingPath(virQEMUDriver *driver, - virDomainObj *vm, +int qemuProcessDestroyMemoryBackingPath(virDomainDef *def, + qemuDomainObjPrivate *priv, virDomainMemoryDef *mem); =20 void qemuProcessReconnectAll(virQEMUDriver *driver); --=20 2.34.1 From nobody Fri Oct 18 05:21:46 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.libvirt.org designates 8.43.85.245 as permitted sender) client-ip=8.43.85.245; envelope-from=devel-bounces@lists.libvirt.org; helo=lists.libvirt.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of lists.libvirt.org designates 8.43.85.245 as permitted sender) smtp.mailfrom=devel-bounces@lists.libvirt.org; dmarc=pass(p=reject dis=none) header.from=lists.libvirt.org ARC-Seal: i=1; a=rsa-sha256; t=1717620146; cv=none; d=zohomail.com; s=zohoarc; b=XRxLF9JeEP6t3Q4QqKrKsOogQmazHv7lyHHV50AxDOi+3zak0+o6b8o8ONoh6i9Gt6oolMU3D0OGq0YmsYZr8iJ/Z9f/1svba72ItoDDgkuuUrjo8UaBzIwvCw8lJqVEo4EAuhb1/YO//aplZ3ad2YJeIg+Xu3Vcjp2E9V5wWQc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1717620146; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Reply-To:Reply-To:References:Subject:Subject:To:To:Message-Id; bh=YmcWP1TjjdpPGB7MQ5U/8JqAD65P4mUwSwOii3JeSaI=; b=Sh3rbRK4vSr0AIMLDx0aY1w5r8t9jP85ICttDiGNWSJ+WU8vHfkoCESWueSIFY8dGmQy7VuBmaB1DYj2LDY4p9lSADq/dfQtQyDfqK9izYrCJgnH6Oe5K31xQKgpYV8s3fbr94y1zD540nuLVvIP0Ddjy8qNZZVN30+Lk1SoR9E= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of lists.libvirt.org designates 8.43.85.245 as permitted sender) smtp.mailfrom=devel-bounces@lists.libvirt.org; dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.libvirt.org (lists.libvirt.org [8.43.85.245]) by mx.zohomail.com with SMTPS id 1717620146723170.28897170240828; Wed, 5 Jun 2024 13:42:26 -0700 (PDT) Received: by lists.libvirt.org (Postfix, from userid 996) id 9383E1A46; Wed, 5 Jun 2024 16:42:25 -0400 (EDT) Received: from lists.libvirt.org (localhost [IPv6:::1]) by lists.libvirt.org (Postfix) with ESMTP id 5A6872113; Wed, 5 Jun 2024 16:38:32 -0400 (EDT) Received: by lists.libvirt.org (Postfix, from userid 996) id E07FF1F52; Wed, 5 Jun 2024 16:38:24 -0400 (EDT) Received: from mx0a-00190b01.pphosted.com (mx0a-00190b01.pphosted.com [67.231.149.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.libvirt.org (Postfix) with ESMTPS id F0E17218C for ; Wed, 5 Jun 2024 16:37:59 -0400 (EDT) Received: from pps.filterd (m0122333.ppops.net [127.0.0.1]) by mx0a-00190b01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 455BraCQ011210; Wed, 5 Jun 2024 21:37:58 +0100 Received: from prod-mail-ppoint6 (prod-mail-ppoint6.akamai.com [184.51.33.61] (may be forged)) by mx0a-00190b01.pphosted.com (PPS) with ESMTPS id 3yfw3yb3kv-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 05 Jun 2024 21:37:58 +0100 (BST) Received: from pps.filterd (prod-mail-ppoint6.akamai.com [127.0.0.1]) by prod-mail-ppoint6.akamai.com (8.17.1.19/8.17.1.19) with ESMTP id 455KTrUO004406; Wed, 5 Jun 2024 16:37:57 -0400 Received: from prod-mail-relay10.akamai.com ([172.27.118.251]) by prod-mail-ppoint6.akamai.com (PPS) with ESMTP id 3yjef53c6n-1; Wed, 05 Jun 2024 16:37:57 -0400 Received: from bos-lhvcg5.bos01.corp.akamai.com (bos-lhvcg5.bos01.corp.akamai.com [172.28.222.58]) by prod-mail-relay10.akamai.com (Postfix) with ESMTP id 12EAD65193; Wed, 5 Jun 2024 20:37:57 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on lists.libvirt.org X-Spam-Level: X-Spam-Status: No, score=-1.5 required=5.0 tests=DKIM_INVALID,DKIM_SIGNED, MAILING_LIST_MULTI,RCVD_IN_DNSWL_LOW,RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.4 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=akamai.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=jan2016.eng; bh=UqEPjmc6a Tz396UkIb6TXmD2UHi2/kYOPRK5R+oF9HQ=; b=GIrX7/6UKEvG6iUzNBoPJX0dv 6hYIOuzplsVUkKx2TNTZ5SREeks6ADDubwg2E7GreB+wk+IMcxzVvuamk0YQvIFQ r8zoccLy8r2yAOooWplbeZRGWcHRXpSDwdC8NmYyaKDSCclZF+s84KhU4VqCjP63 tMqfTkocwyh95990S2NZV7d+cw+lwXhCEGSLyrJ+v7Iy6BqttGxd5za1wTcKwh6h QkLC6SWSpRydQ5d0zlMrcBik+smqO/fOrPG81pirBi+Awo8sR7vDB0rvOCZzoqgO JzQEyrgzDWhRbNI75WQv++urSET8moybTNiXV3hL1bG7mCNiHNGsFuTRQi7Ag== To: devel@lists.libvirt.org Subject: [PATCH v2 2/4] Support live migration between file-backed memory and anonymous memory. Date: Wed, 5 Jun 2024 16:37:36 -0400 Message-Id: <20240605203738.1880333-3-mgalaxy@akamai.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240605203738.1880333-1-mgalaxy@akamai.com> References: <20240605203738.1880333-1-mgalaxy@akamai.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.28.16 definitions=2024-06-05_02,2024-06-05_02,2024-05-17_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 bulkscore=0 mlxlogscore=999 adultscore=0 suspectscore=0 malwarescore=0 mlxscore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2405010000 definitions=main-2406050156 X-Proofpoint-ORIG-GUID: NfT_LYWIa0f1IufTzQ6p5W3DX03vrcom X-Proofpoint-GUID: NfT_LYWIa0f1IufTzQ6p5W3DX03vrcom X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.28.16 definitions=2024-06-05_02,2024-06-05_02,2024-05-17_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 clxscore=1015 adultscore=0 mlxlogscore=999 phishscore=0 priorityscore=1501 impostorscore=0 suspectscore=0 malwarescore=0 lowpriorityscore=0 mlxscore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2405170001 definitions=main-2406050156 Content-Transfer-Encoding: quoted-printable Message-ID-Hash: D2KAUOSSHBY7IM4WLFQ4RKAAEW2DIF5N X-Message-ID-Hash: D2KAUOSSHBY7IM4WLFQ4RKAAEW2DIF5N X-MailFrom: mgalaxy@akamai.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-config-1; header-match-config-2; header-match-config-3; header-match-devel.lists.libvirt.org-0; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; suspicious-header CC: steven.sistare@oracle.com, johunt@akamai.com, cweng@akamai.com, bchaney@akamai.com, trakotoz@akamai.com, mkletzan@redhat.com, anthony.yznaga@oracle.com, mprivozn@redhat.com, Michael Galaxy X-Mailman-Version: 3.2.2 Precedence: list List-Id: Development discussions about the libvirt library & tools Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: From: Michael Galaxy via Devel Reply-To: mgalaxy@akamai.com X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1717620148081100001 Content-Type: text/plain; charset="utf-8" From: Michael Galaxy In our environment, we need to convert VMs into a live-update-comptabile configuration "on-the-fly" (via live migration). More specifically: We need to convert between anonymous memory-backed VMs and file-backed memory VMs. So, for this very specific case, this needs to work when host-level PMEM is being enabled. QEMU does not have a problem with this at all, but we need to relax the rules here a bit so that libvirt allows it to go through normally. Signed-off-by: Michael Galaxy --- src/qemu/qemu_domain.c | 24 +++++++++++++++++++----- 1 file changed, 19 insertions(+), 5 deletions(-) diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c index bda62f2e5c..93e98f8dae 100644 --- a/src/qemu/qemu_domain.c +++ b/src/qemu/qemu_domain.c @@ -8641,11 +8641,25 @@ qemuDomainABIStabilityCheck(const virDomainDef *src, size_t i; =20 if (src->mem.source !=3D dst->mem.source) { - virReportError(VIR_ERR_CONFIG_UNSUPPORTED, - _("Target memoryBacking source '%1$s' doesn't match= source memoryBacking source'%2$s'"), - virDomainMemorySourceTypeToString(dst->mem.source), - virDomainMemorySourceTypeToString(src->mem.source)); - return false; + /* + * The current use case for this is the live migration of live-upd= ate + * capable CPR guests mounted on PMEM devices at the host + * level (not in-guest PMEM). QEMU has no problem doing these kind= s of + * live migrations between these two memory backends, so let them = go through. + * This allows us to "upgrade" guests from regular memory to file-= backed + * memory seemlessly without taking them down. + */ + if (!((src->mem.source =3D=3D VIR_DOMAIN_MEMORY_SOURCE_NONE + && dst->mem.source =3D=3D VIR_DOMAIN_MEMORY_SOURCE_FILE) || + (src->mem.source =3D=3D VIR_DOMAIN_MEMORY_SOURCE_FILE + && dst->mem.source =3D=3D VIR_DOMAIN_MEMORY_SOURCE_NONE)))= { + + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, + _("Target memoryBacking source '%1$s' doesn't m= atch source memoryBacking source'%2$s'"), + virDomainMemorySourceTypeToString(dst->mem.sour= ce), + virDomainMemorySourceTypeToString(src->mem.sour= ce)); + return false; + } } =20 for (i =3D 0; i < src->nmems; i++) { --=20 2.34.1 From nobody Fri Oct 18 05:21:46 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.libvirt.org designates 8.43.85.245 as permitted sender) client-ip=8.43.85.245; envelope-from=devel-bounces@lists.libvirt.org; helo=lists.libvirt.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of lists.libvirt.org designates 8.43.85.245 as permitted sender) smtp.mailfrom=devel-bounces@lists.libvirt.org; dmarc=pass(p=reject dis=none) header.from=lists.libvirt.org ARC-Seal: i=1; a=rsa-sha256; t=1717620191; cv=none; d=zohomail.com; s=zohoarc; b=H/VVXNK5TyLVBDKaTx2pWuKmiNw8qNK1t26G4pRmSl/8SRroMLNboDEbZF77dtbd9kSv1JOK2zgq5kS76Ei7RSWWtR7y2V6NUKE31M86ELG9BwyHBOhoVT527cZPd3ikawkuq83bCAdeyLPzBSb3i43EHHwFOGjFO/BvBLIu0HE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1717620191; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Reply-To:Reply-To:References:Subject:Subject:To:To:Message-Id; bh=xvkpA/D0w5TJvoQ7Exxm6xTe6jmhplkvecG8eFx+UJQ=; b=ny7srHif15IePLRSXvNTSMEUA6RjmNHD38va9ZtCr5rCx5BSb9etUdCkrPcr+FriFsb1A0HXX2Kt7CEhzRfZlf6TzMN0Paq6FizHXKWrDUnRsZpNcbjxfg/fkIn14dxSbahX4UkvSOhGFlC+bvjHiEDk/VbwP+Pdj5TII5S/pT4= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of lists.libvirt.org designates 8.43.85.245 as permitted sender) smtp.mailfrom=devel-bounces@lists.libvirt.org; dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.libvirt.org (lists.libvirt.org [8.43.85.245]) by mx.zohomail.com with SMTPS id 1717620191167824.6955485501775; Wed, 5 Jun 2024 13:43:11 -0700 (PDT) Received: by lists.libvirt.org (Postfix, from userid 996) id 2044D21C3; Wed, 5 Jun 2024 16:43:10 -0400 (EDT) Received: from lists.libvirt.org (localhost [IPv6:::1]) by lists.libvirt.org (Postfix) with ESMTP id 689F221CA; Wed, 5 Jun 2024 16:38:37 -0400 (EDT) Received: by lists.libvirt.org (Postfix, from userid 996) id 5B629215B; Wed, 5 Jun 2024 16:38:25 -0400 (EDT) Received: from mx0a-00190b01.pphosted.com (mx0a-00190b01.pphosted.com [67.231.149.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.libvirt.org (Postfix) with ESMTPS id ECC791F56 for ; Wed, 5 Jun 2024 16:38:00 -0400 (EDT) Received: from pps.filterd (m0409409.ppops.net [127.0.0.1]) by m0409409.ppops.net-00190b01. (8.18.1.2/8.18.1.2) with ESMTP id 455BppjT031278; Wed, 5 Jun 2024 21:37:59 +0100 Received: from prod-mail-ppoint1 (prod-mail-ppoint1.akamai.com [184.51.33.18] (may be forged)) by m0409409.ppops.net-00190b01. (PPS) with ESMTPS id 3yj9nqjmjh-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 05 Jun 2024 21:37:59 +0100 (BST) Received: from pps.filterd (prod-mail-ppoint1.akamai.com [127.0.0.1]) by prod-mail-ppoint1.akamai.com (8.17.1.19/8.17.1.19) with ESMTP id 455Hb9wd005253; Wed, 5 Jun 2024 16:37:58 -0400 Received: from prod-mail-relay10.akamai.com ([172.27.118.251]) by prod-mail-ppoint1.akamai.com (PPS) with ESMTP id 3yjeg0bfbv-1; Wed, 05 Jun 2024 16:37:58 -0400 Received: from bos-lhvcg5.bos01.corp.akamai.com (bos-lhvcg5.bos01.corp.akamai.com [172.28.222.58]) by prod-mail-relay10.akamai.com (Postfix) with ESMTP id 2A34D65191; Wed, 5 Jun 2024 20:37:58 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on lists.libvirt.org X-Spam-Level: ** X-Spam-Status: No, score=2.1 required=5.0 tests=DKIM_INVALID,DKIM_SIGNED, MAILING_LIST_MULTI,RCVD_IN_DNSWL_LOW,RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL,RCVD_IN_SBL_CSS,SPF_HELO_NONE,T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.4 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=akamai.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=jan2016.eng; bh=4LfS72COU 8Y6ZGX93YSbxnN5PMjvYS7QcBysrfdf3Vo=; b=GfiQyuwXOjddb7TxgT/NliKUM fXAcwU9wvGP59r/eVN5I5FhKFJZBP1IghNKnKyimrMCwVr4U0UASyd1FspJmhuqw nXvpmf7VZZscSUmj9u/GlNqVDOimMdYPgYcopB9j/zlth2b7VimowyaQLqbGKzWI bEZ6ef6w0m34qUezVqFWdTKrBTj/xr05HfLxG4MxxZwbCb2yVhRJI58slFCOij28 eauwTDdMr0B4ElIgs+AdJOs4dqphxZ/7JwXRq+ExBMu0ld9QLbdGgrTK7x7yvSZ3 iJMSsbasilvjLAx9C9qmQexAtZ+sBPzV8x2b60lYarvzKEHrLvjccKFBEZCdA== To: devel@lists.libvirt.org Subject: [PATCH v2 3/4] Update unit test to support multiple memory backends Date: Wed, 5 Jun 2024 16:37:37 -0400 Message-Id: <20240605203738.1880333-4-mgalaxy@akamai.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240605203738.1880333-1-mgalaxy@akamai.com> References: <20240605203738.1880333-1-mgalaxy@akamai.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.28.16 definitions=2024-06-05_02,2024-06-05_02,2024-05-17_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 malwarescore=0 suspectscore=0 spamscore=0 phishscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2405010000 definitions=main-2406050156 X-Proofpoint-GUID: QT11sV7vDrddaYGqgmOoKjBtuQyljXQt X-Proofpoint-ORIG-GUID: QT11sV7vDrddaYGqgmOoKjBtuQyljXQt X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.28.16 definitions=2024-06-05_02,2024-06-05_02,2024-05-17_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 phishscore=0 bulkscore=0 impostorscore=0 clxscore=1015 priorityscore=1501 malwarescore=0 mlxscore=0 adultscore=0 lowpriorityscore=0 suspectscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2405170001 definitions=main-2406050156 Content-Transfer-Encoding: quoted-printable Message-ID-Hash: 75V4XF6OMTSABRE5EDFM74E7342M4CN4 X-Message-ID-Hash: 75V4XF6OMTSABRE5EDFM74E7342M4CN4 X-MailFrom: mgalaxy@akamai.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-config-1; header-match-config-2; header-match-config-3; header-match-devel.lists.libvirt.org-0; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; suspicious-header CC: steven.sistare@oracle.com, johunt@akamai.com, cweng@akamai.com, bchaney@akamai.com, trakotoz@akamai.com, mkletzan@redhat.com, anthony.yznaga@oracle.com, mprivozn@redhat.com, Michael Galaxy X-Mailman-Version: 3.2.2 Precedence: list List-Id: Development discussions about the libvirt library & tools Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: From: Michael Galaxy via Devel Reply-To: mgalaxy@akamai.com X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1717620192255100001 Content-Type: text/plain; charset="utf-8" From: Michael Galaxy Update the associated test case. Signed-off-by: Michael Galaxy --- tests/testutilsqemu.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/tests/testutilsqemu.c b/tests/testutilsqemu.c index d70850cb5d..dd8db4835e 100644 --- a/tests/testutilsqemu.c +++ b/tests/testutilsqemu.c @@ -320,8 +320,9 @@ int qemuTestDriverInit(virQEMUDriver *driver) cfg->libDir =3D g_strdup("/var/lib/libvirt/qemu"); VIR_FREE(cfg->channelTargetDir); cfg->channelTargetDir =3D g_strdup("/var/run/libvirt/qemu/channel"); - VIR_FREE(cfg->memoryBackingDir); - cfg->memoryBackingDir =3D g_strdup("/var/lib/libvirt/qemu/ram"); + g_free(cfg->memoryBackingDirs); + cfg->memoryBackingDirs =3D g_new0(char *, 1); + cfg->memoryBackingDirs[0] =3D g_strdup("/var/lib/libvirt/qemu/ram"); VIR_FREE(cfg->nvramDir); cfg->nvramDir =3D g_strdup("/var/lib/libvirt/qemu/nvram"); VIR_FREE(cfg->passtStateDir); --=20 2.34.1 From nobody Fri Oct 18 05:21:46 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.libvirt.org designates 8.43.85.245 as permitted sender) client-ip=8.43.85.245; envelope-from=devel-bounces@lists.libvirt.org; helo=lists.libvirt.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of lists.libvirt.org designates 8.43.85.245 as permitted sender) smtp.mailfrom=devel-bounces@lists.libvirt.org; dmarc=pass(p=reject dis=none) header.from=lists.libvirt.org ARC-Seal: i=1; a=rsa-sha256; t=1717620237; cv=none; d=zohomail.com; s=zohoarc; b=k5Z2ON6R6cIMmww9zvFOyGPhKiuQxg+mB5Q1ga0Q/A2mSM4kJAWCnxT1HTuLLOyQs5Srym8DTBErGTDlqWW9A7lEB0IAaRcX2fQ/V/fHB2BaK8BD2IRmC9z9aLMIqgboCW5n7vEiUjX02Nvkwa4LhBR5CI3D674DtF4pSuqOYAo= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1717620237; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Reply-To:Reply-To:References:Subject:Subject:To:To:Message-Id; bh=Zl9q7E/dxuQg+LEqf/eQDNDMsPw6Uo0TuD9MaAHuP9c=; b=SFsMgFWllQUeiqyo5Ua98dhCGkTQ2VScAOXAfxmUZQZW9HIZdGQHVJDouQZE9JlzmxPqmP4D9cDnpky7dsfZp6g/A/rcbOavnSk13ZZRvb/dSpsq11JWTxPGeajOKrCKkFdibdmVXSG1likBOEJfU7ey+IwsVYXraVMxVNN60sw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of lists.libvirt.org designates 8.43.85.245 as permitted sender) smtp.mailfrom=devel-bounces@lists.libvirt.org; dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.libvirt.org (lists.libvirt.org [8.43.85.245]) by mx.zohomail.com with SMTPS id 1717620237981584.8720796023662; Wed, 5 Jun 2024 13:43:57 -0700 (PDT) Received: by lists.libvirt.org (Postfix, from userid 996) id EAB7E216F; Wed, 5 Jun 2024 16:43:56 -0400 (EDT) Received: from lists.libvirt.org (localhost [IPv6:::1]) by lists.libvirt.org (Postfix) with ESMTP id AE1272161; Wed, 5 Jun 2024 16:38:42 -0400 (EDT) Received: by lists.libvirt.org (Postfix, from userid 996) id 690552185; Wed, 5 Jun 2024 16:38:26 -0400 (EDT) Received: from mx0b-00190b01.pphosted.com (mx0b-00190b01.pphosted.com [67.231.157.127]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.libvirt.org (Postfix) with ESMTPS id 20BEB2196 for ; Wed, 5 Jun 2024 16:38:01 -0400 (EDT) Received: from pps.filterd (m0122330.ppops.net [127.0.0.1]) by mx0b-00190b01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 455BqORi015135; Wed, 5 Jun 2024 21:38:00 +0100 Received: from prod-mail-ppoint1 (prod-mail-ppoint1.akamai.com [184.51.33.18] (may be forged)) by mx0b-00190b01.pphosted.com (PPS) with ESMTPS id 3yfw478jc5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 05 Jun 2024 21:38:00 +0100 (BST) Received: from pps.filterd (prod-mail-ppoint1.akamai.com [127.0.0.1]) by prod-mail-ppoint1.akamai.com (8.17.1.19/8.17.1.19) with ESMTP id 455K9kK1005273; Wed, 5 Jun 2024 16:37:59 -0400 Received: from prod-mail-relay10.akamai.com ([172.27.118.251]) by prod-mail-ppoint1.akamai.com (PPS) with ESMTP id 3yjeg0bfbw-1; Wed, 05 Jun 2024 16:37:59 -0400 Received: from bos-lhvcg5.bos01.corp.akamai.com (bos-lhvcg5.bos01.corp.akamai.com [172.28.222.58]) by prod-mail-relay10.akamai.com (Postfix) with ESMTP id 60D2665191; Wed, 5 Jun 2024 20:37:59 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on lists.libvirt.org X-Spam-Level: ** X-Spam-Status: No, score=2.1 required=5.0 tests=DKIM_INVALID,DKIM_SIGNED, MAILING_LIST_MULTI,RCVD_IN_DNSWL_LOW,RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL,RCVD_IN_SBL_CSS,SPF_HELO_NONE,T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.4 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=akamai.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=jan2016.eng; bh=KTYgYE8x5 1+UhoU4CwFLVREC7LIh/pRJ9OPgHQL0bM4=; b=QG9O3LNG5Z9qTYQ0NQQxDL8Li ipc/jze3H1Sk0hcktmaGfvpO5Yl69PAO27fbCu2Ycp5qFCg8W7CNtsJbowSzl35S X75jMmFePYU6Kj84Oxxt5/Hyqos2rhaJt8TP6Z3Kyy7rV5czSo/R6fJWK/mprV4c J3sxrbwqhbCwn+tYxxAXX9MerinTGpvhd3Drx1QxdxCA/X3Jl4If7hPeBHZRYCCk YMNo2Ug28RPnZ6yld7UrktY5d8+pN8dL2KzbUIxpwb5/Dbsjxt5jXKKK9Gdctr0g eVqE0LLefHeSTnBsOoG+85HeOWLpnOPn9mMAQ3f98OBsiucSqSKJXM6HAoENQ== To: devel@lists.libvirt.org Subject: [PATCH v2 4/4] Update documentation to reflect memory_backing_dir change in qemu.conf Date: Wed, 5 Jun 2024 16:37:38 -0400 Message-Id: <20240605203738.1880333-5-mgalaxy@akamai.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240605203738.1880333-1-mgalaxy@akamai.com> References: <20240605203738.1880333-1-mgalaxy@akamai.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.28.16 definitions=2024-06-05_02,2024-06-05_02,2024-05-17_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 malwarescore=0 suspectscore=0 spamscore=0 phishscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2405010000 definitions=main-2406050156 X-Proofpoint-ORIG-GUID: rfeAQsFEloFB4M08f3mlthvcyQR3RhBK X-Proofpoint-GUID: rfeAQsFEloFB4M08f3mlthvcyQR3RhBK X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.28.16 definitions=2024-06-05_02,2024-06-05_02,2024-05-17_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 lowpriorityscore=0 bulkscore=0 mlxscore=0 spamscore=0 impostorscore=0 clxscore=1015 suspectscore=0 phishscore=0 malwarescore=0 priorityscore=1501 mlxlogscore=901 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2405170001 definitions=main-2406050156 Content-Transfer-Encoding: quoted-printable Message-ID-Hash: ZTKHR5SMA6S55MUIPV5KBMJHJEEGIAD4 X-Message-ID-Hash: ZTKHR5SMA6S55MUIPV5KBMJHJEEGIAD4 X-MailFrom: mgalaxy@akamai.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-config-1; header-match-config-2; header-match-config-3; header-match-devel.lists.libvirt.org-0; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; suspicious-header CC: steven.sistare@oracle.com, johunt@akamai.com, cweng@akamai.com, bchaney@akamai.com, trakotoz@akamai.com, mkletzan@redhat.com, anthony.yznaga@oracle.com, mprivozn@redhat.com, Michael Galaxy X-Mailman-Version: 3.2.2 Precedence: list List-Id: Development discussions about the libvirt library & tools Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: From: Michael Galaxy via Devel Reply-To: mgalaxy@akamai.com X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1717620238466100001 Content-Type: text/plain; charset="utf-8" From: Michael Galaxy Update the documentation and changelog accordingly to reflect that the syntax of the memory_backing_dir setting has changed. Signed-off-by: Michael Galaxy --- NEWS.rst | 7 +++++++ docs/kbase/virtiofs.rst | 2 ++ src/qemu/qemu.conf.in | 2 ++ 3 files changed, 11 insertions(+) diff --git a/NEWS.rst b/NEWS.rst index f3ca29443a..64be9a9509 100644 --- a/NEWS.rst +++ b/NEWS.rst @@ -54,6 +54,13 @@ v10.4.0 (2024-06-03) ebtables/iptables). =20 * **Improvements** + * qemu: memory backing directories now support NUMA + + The primary use case of this is when persistent memory is used on + NUMA-aware systems. The setting ``memory_backing_dir`` in qemu.conf + now supports either an array or the original string. This allows + you to provide a array of locations to use that are NUMA-compatible + if needed. =20 * qemu: add zstd to supported compression formats =20 diff --git a/docs/kbase/virtiofs.rst b/docs/kbase/virtiofs.rst index 457c15da7f..931ba96104 100644 --- a/docs/kbase/virtiofs.rst +++ b/docs/kbase/virtiofs.rst @@ -144,6 +144,8 @@ memobject interface). =20 # This directory is used for memoryBacking source if configured as = file. # NOTE: big files will be stored here + # NOTE: In the case of a NUMA-aware system (such has shm/tmpfs or P= MEM), + # one can also do: memory_backing_dir =3D [ "/path/to/node/0"= , "/path/to/node/1" ] memory_backing_dir =3D "/dev/shm/" =20 * Use hugepage-backed memory diff --git a/src/qemu/qemu.conf.in b/src/qemu/qemu.conf.in index 6bc2140dcb..e6d1f8a675 100644 --- a/src/qemu/qemu.conf.in +++ b/src/qemu/qemu.conf.in @@ -897,6 +897,8 @@ =20 # This directory is used for memoryBacking source if configured as file. # NOTE: big files will be stored here +# NOTE: In the case of a NUMA-aware system (such has shm/tmpfs or PMEM), +# one can also do: memory_backing_dir =3D [ "/path/to/node/0", "/pat= h/to/node/1" ] #memory_backing_dir =3D "/var/lib/libvirt/qemu/ram" =20 # Path to the SCSI persistent reservations helper. This helper is --=20 2.34.1