From nobody Thu Sep 19 01:06:07 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.libvirt.org designates 8.43.85.245 as permitted sender) client-ip=8.43.85.245; envelope-from=devel-bounces@lists.libvirt.org; helo=lists.libvirt.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of lists.libvirt.org designates 8.43.85.245 as permitted sender) smtp.mailfrom=devel-bounces@lists.libvirt.org; dmarc=pass(p=reject dis=none) header.from=lists.libvirt.org ARC-Seal: i=1; a=rsa-sha256; t=1723760123; cv=none; d=zohomail.com; s=zohoarc; b=Hsd384eDjyb7lMducVhccYY6GDc750A/NT/0IKsO+n50PPMLHNYMpizvSXk3f3FFXwdm/5Gx0hLgqvUsQdhRgTOoQvPG140fcqeZgGiaGok/4+HDJrSnvOJcrxBmsrnJQBrO5imUKAtuxeDewZtGLcwFT0AomrbTxWQkYcBSCIo= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1723760123; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Reply-To:Reply-To:References:Subject:Subject:To:To:Message-Id; bh=Ki1w01WTY1E3ogFvrkU9mmhwUSf0DyMrXSGHYmNP2mA=; b=MRETJiP4zuElTteMegyPSEUXjxRmn/b6yO2bjehnvF8wFZn+4+x3mR8w8wk6Sb8pd211LMk1EC02Wbq6bvftGEQ/7JgmeEo6bamkO23sQGzuc6UyzE3bGEr5GHIQEBv07GnLENWk9Q4/QzVCuZaoQnX+reMZDThQspPFwp8VEck= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of lists.libvirt.org designates 8.43.85.245 as permitted sender) smtp.mailfrom=devel-bounces@lists.libvirt.org; dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.libvirt.org (lists.libvirt.org [8.43.85.245]) by mx.zohomail.com with SMTPS id 1723760123788457.5758067131176; Thu, 15 Aug 2024 15:15:23 -0700 (PDT) Received: by lists.libvirt.org (Postfix, from userid 996) id AB4B11320; Thu, 15 Aug 2024 18:15:22 -0400 (EDT) Received: from lists.libvirt.org (localhost [IPv6:::1]) by lists.libvirt.org (Postfix) with ESMTP id C04AA14F2; Thu, 15 Aug 2024 18:13:27 -0400 (EDT) Received: by lists.libvirt.org (Postfix, from userid 996) id 4A6411323; Thu, 15 Aug 2024 18:13:21 -0400 (EDT) Received: from mx0a-00190b01.pphosted.com (mx0a-00190b01.pphosted.com [67.231.149.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.libvirt.org (Postfix) with ESMTPS id B0AAD12A3 for ; Thu, 15 Aug 2024 18:13:19 -0400 (EDT) Received: from pps.filterd (m0050095.ppops.net [127.0.0.1]) by m0050095.ppops.net-00190b01. (8.18.1.2/8.18.1.2) with ESMTP id 47FLjdXA012579; Thu, 15 Aug 2024 23:13:18 +0100 Received: from prod-mail-ppoint5 (prod-mail-ppoint5.akamai.com [184.51.33.60] (may be forged)) by m0050095.ppops.net-00190b01. (PPS) with ESMTPS id 40yrkurg34-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 15 Aug 2024 23:13:17 +0100 (BST) Received: from pps.filterd (prod-mail-ppoint5.akamai.com [127.0.0.1]) by prod-mail-ppoint5.akamai.com (8.17.1.19/8.17.1.19) with ESMTP id 47FM7q1F023877; Thu, 15 Aug 2024 15:13:16 -0700 Received: from prod-mail-relay11.akamai.com ([172.27.118.250]) by prod-mail-ppoint5.akamai.com (PPS) with ESMTP id 411rt40921-1; Thu, 15 Aug 2024 15:13:16 -0700 Received: from bos-lhvcg5.bos01.corp.akamai.com (bos-lhvcg5.bos01.corp.akamai.com [172.28.222.58]) by prod-mail-relay11.akamai.com (Postfix) with ESMTP id 80C4D33EE3; Thu, 15 Aug 2024 22:13:15 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on lists.libvirt.org X-Spam-Level: X-Spam-Status: No, score=-1.5 required=5.0 tests=DKIM_INVALID,DKIM_SIGNED, MAILING_LIST_MULTI,RCVD_IN_DNSWL_LOW,RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.4 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=akamai.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=jan2016.eng; bh=apDxJrLbV qjbpQbhFc/Gks+G0ZCkScBlQjSjld6BLgI=; b=Oxt9MIJjcZV/Gz1rh178698+Y TmoRXuxr0QrcEFsyoT3xfzuyRydqVpaeiH2NW6Aiid7iNmoiX4LYrzaYDULo5z/8 637HiUSJCfRhntiNnVznwibTihE2jMffyQejMpwCTonprydv9lsFdg5bO/4l3D+x ofyBqS/Oko1JkLPR9GdgtOEx6d5FKFEds1esAaLnNUqFwcTwjbpx7/6xPQJIwGl5 7hXrmbMNzIcs/SBnmLDEhD94EKtpZ/DgJvpf4G2xnPaRGS3ZuKbL8+6G9ivpFTKc uJPu9+KmhI8IASg6aiHClsV2P8wdDtN3pz0JMrRYRMFeWw1tRs9TGpGSS1nOg== To: devel@lists.libvirt.org Subject: [PATCH v3 1/4] qemu.conf changes to support multiple memory backend Date: Thu, 15 Aug 2024 18:13:05 -0400 Message-Id: <20240815221308.3892813-2-mgalaxy@akamai.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240815221308.3892813-1-mgalaxy@akamai.com> References: <20240815221308.3892813-1-mgalaxy@akamai.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.28.16 definitions=2024-08-15_14,2024-08-15_01,2024-05-17_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 suspectscore=0 malwarescore=0 mlxscore=0 mlxlogscore=999 spamscore=0 phishscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2407110000 definitions=main-2408150161 X-Proofpoint-GUID: d6B5jQFzU3b5GRtg9SGwnay0PEQkdg3p X-Proofpoint-ORIG-GUID: d6B5jQFzU3b5GRtg9SGwnay0PEQkdg3p X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.28.16 definitions=2024-08-15_14,2024-08-15_01,2024-05-17_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 mlxlogscore=999 suspectscore=0 phishscore=0 malwarescore=0 impostorscore=0 lowpriorityscore=0 bulkscore=0 adultscore=0 clxscore=1015 spamscore=0 priorityscore=1501 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2407110000 definitions=main-2408150162 Content-Transfer-Encoding: quoted-printable Message-ID-Hash: 2YWOUUMMMXT4MMZAQAI6PIBQQL3OYTO3 X-Message-ID-Hash: 2YWOUUMMMXT4MMZAQAI6PIBQQL3OYTO3 X-MailFrom: mgalaxy@akamai.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-config-1; header-match-config-2; header-match-config-3; header-match-devel.lists.libvirt.org-0; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; suspicious-header CC: steven.sistare@oracle.com, johunt@akamai.com, cweng@akamai.com, bchaney@akamai.com, trakotoz@akamai.com, mkletzan@redhat.com, anthony.yznaga@oracle.com, mprivozn@redhat.com, Michael Galaxy X-Mailman-Version: 3.2.2 Precedence: list List-Id: Development discussions about the libvirt library & tools Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: From: Michael Galaxy via Devel Reply-To: mgalaxy@akamai.com X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1723760124922116600 Content-Type: text/plain; charset="utf-8" From: Michael Galaxy We start by introducing a backwards-compatible, comma-separated specification that will not break existing installations, such as in the following example: $ cat qemu.conf | grep memory_backing_dir memory_backing_dir =3D ["/path/to/pmem/0", "/path/to/pmem/1"] (The old syntax with a single string is also still supported) memory_backing_dir =3D "/path/to/dir" In our case, we almost always have two NUMA nodes, so in that example, we have two PMEM regions which are created on the Linux kernel command line that get mounted into those two locations for libvirt to use. Signed-off-by: Michael Galaxy --- src/qemu/qemu_command.c | 6 +- src/qemu/qemu_conf.c | 139 +++++++++++++++++++++++++++++++++++----- src/qemu/qemu_conf.h | 16 +++-- src/qemu/qemu_driver.c | 37 +++++++---- src/qemu/qemu_hotplug.c | 6 +- src/qemu/qemu_process.c | 42 ++++++------ src/qemu/qemu_process.h | 6 +- tests/testutilsqemu.c | 5 +- 8 files changed, 191 insertions(+), 66 deletions(-) diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c index 28914c9c34..4227e683c9 100644 --- a/src/qemu/qemu_command.c +++ b/src/qemu/qemu_command.c @@ -3421,7 +3421,8 @@ qemuBuildMemoryBackendProps(virJSONValue **backendPro= ps, } else { /* We can have both pagesize and mem source. If that's the cas= e, * prefer hugepages as those are more specific. */ - if (qemuGetMemoryBackingPath(priv->driver, def, mem->info.alia= s, &memPath) < 0) + + if (qemuGetMemoryBackingPath(def, priv->driver, priv->autoNode= set, mem->targetNode, mem->info.alias, &memPath) < 0) return -1; } =20 @@ -7300,7 +7301,8 @@ qemuBuildMemPathStr(const virDomainDef *def, return -1; prealloc =3D true; } else if (def->mem.source =3D=3D VIR_DOMAIN_MEMORY_SOURCE_FILE) { - if (qemuGetMemoryBackingPath(priv->driver, def, "ram", &mem_path) = < 0) + // This path should not be reached if NUMA is requested + if (qemuGetMemoryBackingPath(def, priv->driver, priv->autoNodeset,= 0, "ram", &mem_path) < 0) return -1; } =20 diff --git a/src/qemu/qemu_conf.c b/src/qemu/qemu_conf.c index b36bede6c3..c2d3e4b400 100644 --- a/src/qemu/qemu_conf.c +++ b/src/qemu/qemu_conf.c @@ -43,6 +43,7 @@ #include "virfile.h" #include "virstring.h" #include "virutil.h" +#include "virnuma.h" #include "configmake.h" #include "security/security_util.h" =20 @@ -137,6 +138,9 @@ virQEMUDriverConfig *virQEMUDriverConfigNew(bool privil= eged, =20 cfg->cgroupControllers =3D -1; /* -1 =3D=3D auto-detect */ =20 + cfg->memoryBackingDirs =3D g_new0(char *, 1); + cfg->nmemoryBackingDirs =3D 1; + if (root !=3D NULL) { cfg->logDir =3D g_strdup_printf("%s/log/qemu", root); cfg->swtpmLogDir =3D g_strdup_printf("%s/log/swtpm", root); @@ -153,7 +157,7 @@ virQEMUDriverConfig *virQEMUDriverConfigNew(bool privil= eged, cfg->checkpointDir =3D g_strdup_printf("%s/checkpoint", cfg->libDi= r); cfg->autoDumpPath =3D g_strdup_printf("%s/dump", cfg->libDir); cfg->nvramDir =3D g_strdup_printf("%s/nvram", cfg->libDir); - cfg->memoryBackingDir =3D g_strdup_printf("%s/ram", cfg->libDir); + cfg->memoryBackingDirs[0] =3D g_strdup_printf("%s/ram", cfg->libDi= r); } else if (privileged) { cfg->logDir =3D g_strdup_printf("%s/log/libvirt/qemu", LOCALSTATED= IR); =20 @@ -174,7 +178,8 @@ virQEMUDriverConfig *virQEMUDriverConfigNew(bool privil= eged, cfg->checkpointDir =3D g_strdup_printf("%s/checkpoint", cfg->libDi= r); cfg->autoDumpPath =3D g_strdup_printf("%s/dump", cfg->libDir); cfg->nvramDir =3D g_strdup_printf("%s/nvram", cfg->libDir); - cfg->memoryBackingDir =3D g_strdup_printf("%s/ram", cfg->libDir); + + cfg->memoryBackingDirs[0] =3D g_strdup_printf("%s/ram", cfg->libDi= r); cfg->swtpmStorageDir =3D g_strdup_printf("%s/lib/libvirt/swtpm", LOCALSTATEDIR); } else { @@ -201,7 +206,7 @@ virQEMUDriverConfig *virQEMUDriverConfigNew(bool privil= eged, cfg->configBaseDir); cfg->autoDumpPath =3D g_strdup_printf("%s/qemu/dump", cfg->configB= aseDir); cfg->nvramDir =3D g_strdup_printf("%s/qemu/nvram", cfg->configBase= Dir); - cfg->memoryBackingDir =3D g_strdup_printf("%s/qemu/ram", cfg->conf= igBaseDir); + cfg->memoryBackingDirs[0] =3D g_strdup_printf("%s/qemu/ram", cfg->= configBaseDir); cfg->swtpmStorageDir =3D g_strdup_printf("%s/qemu/swtpm", cfg->configBaseDir); } @@ -294,6 +299,7 @@ virQEMUDriverConfig *virQEMUDriverConfigNew(bool privil= eged, static void virQEMUDriverConfigDispose(void *obj) { virQEMUDriverConfig *cfg =3D obj; + size_t i; =20 virBitmapFree(cfg->namespaces); =20 @@ -369,7 +375,12 @@ static void virQEMUDriverConfigDispose(void *obj) =20 virFirmwareFreeList(cfg->firmwares, cfg->nfirmwares); =20 - g_free(cfg->memoryBackingDir); + for (i =3D 0; i < cfg->nmemoryBackingDirs; i++) { + g_free(cfg->memoryBackingDirs[i]); + } + + g_free(cfg->memoryBackingDirs); + g_free(cfg->swtpmStorageDir); =20 g_strfreev(cfg->capabilityfilters); @@ -1018,15 +1029,21 @@ static int virQEMUDriverConfigLoadMemoryEntry(virQEMUDriverConfig *cfg, virConf *conf) { - g_autofree char *dir =3D NULL; + char **memoryBackingDirs =3D NULL; int rc; =20 - if ((rc =3D virConfGetValueString(conf, "memory_backing_dir", &dir)) <= 0) + if ((rc =3D virConfGetValueStringList(conf, "memory_backing_dir", true= , &memoryBackingDirs) < 0)) return -1; =20 - if (rc > 0) { - VIR_FREE(cfg->memoryBackingDir); - cfg->memoryBackingDir =3D g_strdup_printf("%s/libvirt/qemu", dir); + if (rc =3D=3D 0) { + size_t i; + for (i =3D 0; i < cfg->nmemoryBackingDirs; i++) + g_free(cfg->memoryBackingDirs[i]); + + cfg->nmemoryBackingDirs =3D g_strv_length(memoryBackingDirs); + cfg->memoryBackingDirs =3D g_new0(char *, cfg->nmemoryBackingDirs); + for (i =3D 0; i < cfg->nmemoryBackingDirs; i++) + cfg->memoryBackingDirs[i] =3D g_strdup_printf("%s/libvirt/qemu= ", memoryBackingDirs[i]); } =20 return 0; @@ -1604,22 +1621,108 @@ qemuGetDomainHupageMemPath(virQEMUDriver *driver, =20 =20 int -qemuGetMemoryBackingDomainPath(virQEMUDriver *driver, - const virDomainDef *def, +qemuGetMemoryBackingDomainPath(const virDomainDef *def, + virQEMUDriver *driver, + virBitmap *autoNodeset, + const size_t targetNode, char **path) { g_autoptr(virQEMUDriverConfig) cfg =3D virQEMUDriverGetConfig(driver); const char *root =3D driver->embeddedRoot; g_autofree char *shortName =3D NULL; + size_t path_index =3D 0; // original behavior, described below =20 if (!(shortName =3D virDomainDefGetShortName(def))) return -1; =20 - if (root && !STRPREFIX(cfg->memoryBackingDir, root)) { + /* + * We have three use cases: + * + * 1. Domain has multiple NUMA nodes, but they have only specified + * a single directory path in qemu.conf. (Original default behavior= ). + * + * In this case, we already placed the memory backing path for each= NUMA node + * into the same path location. Preserve the established default be= havior. + * + * 2. Domain has multiple NUMA nodes, but we have asked for multiple d= irectory + * paths as well. + * + * In this case, we will have a one-to-one relationship between the= number + * of NUMA nodes and the order in which the paths are provided. + * If the user does not specify enough paths, then we need to throw= an error. + * NOTE: This is open to comment. The "ordering" of the paths here = is not intially + * configurable to preserve backwards compatibility with the origin= al qemu.conf syntax. + * If controlling the ordering is desired, we would need to revise = the syntax in + * qemu.conf to make that possible. That hasn't been needed so far. + * + * NOTE A): We must check with numatune here, if requested. The num= ber of NUMA nodes + * may be less than or equal to the number of provided pat= hs. If it is less, + * we have to respect the choices made by numatune. In thi= s case, we will map the + * physical NUMA nodes (0, 1, 2...) in the order in which = they appear in qemu.conf + * + * 3. Domain has a single NUMA node, but we have asked for multiple di= rectory paths. + * + * In this case we also need to check if numatune is requested. = If so, + * we want to pick the path indicated by numatune. + * + * NOTE B): In both cases 2 and 3, if numatune is requested, the path = obviously cannot + * be changed on the fly, like it normally would be in "restri= ctive" mode + * during runtime. So, we will only do this is the mode reques= ted is "strict". + * + * NOTE C): Furthermore, in both cases 2 and 3, if the number of direc= tory paths provided + * is more than one, and one of either: a) no numatune is pro= vided at all or + * b) numatune is in fact provided, but the mode is not stric= t, + * then we must throw an error. This is because we cannot kno= w which backing + * directory path to choose without the user's input. + * + * NOTE D): If one or more directory paths is requested in any of the = cases 1, 2, or 3, + * the numatune cannot specifiy more than one NUMA node, beca= use the only mode + * possible with directory paths is "strict" (e.g. automatic = numa balancing of + * memory will not work). Only one numa node can be requested= by numatune, else + * we must throw an error. + */ + + if (cfg->nmemoryBackingDirs > 1) { + virBitmap *numaBitmap =3D virDomainNumatuneGetNodeset(def->numa, a= utoNodeset, targetNode); + size_t numa_node_count =3D virDomainNumaGetNodeCount(def->numa); + virDomainNumatuneMemMode mode; + + if (def->numa && numaBitmap && virNumaNodesetIsAvailable(numaBitma= p) && + virDomainNumatuneGetMode(def->numa, -1, &mode) =3D=3D 0 && + mode =3D=3D VIR_DOMAIN_NUMATUNE_MEM_STRICT && + virBitmapCountBits(numaBitmap) =3D=3D 1) { + + // Is numatune provided? + // Is it strict? + // Does it only specify a single pinning for this target? + // Yes to all 3? then good to go. + + if (cfg->nmemoryBackingDirs < numa_node_count) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, + _("Domain requesting configuration for %1$l= u NUMA nodes, but memory backing directory only has (%2$lu) directory paths= available. Either reduce this to one directory or provide more paths to us= e."), + numa_node_count, + cfg->nmemoryBackingDirs); + return -1; + } + + path_index =3D virBitmapNextSetBit(numaBitmap, -1); + } else if (numa_node_count > 1 && numa_node_count =3D=3D cfg->nmem= oryBackingDirs) { + // Be nice. A valid numatune and pinning has not been specifie= d, but the number + // of paths matches up exactly, so just assign them one-to-one. + path_index =3D targetNode; + } else { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, + _("There are (%1$lu) memory directory directori= es configured. Domain must use a 'strict' numatune as well as an associated= pinning configuration for each NUMA node before proceeding. An individual = NUMA node can only be pinned to a single backing directory. Please correct = the domain configuration or remove the memory backing directories and try a= gain."), + cfg->nmemoryBackingDirs); + return -1; + } + } + + if (root && !STRPREFIX(cfg->memoryBackingDirs[path_index], root)) { g_autofree char * hash =3D virDomainDriverGenerateRootHash("qemu",= root); - *path =3D g_strdup_printf("%s/%s-%s", cfg->memoryBackingDir, hash,= shortName); + *path =3D g_strdup_printf("%s/%s-%s", cfg->memoryBackingDirs[path_= index], hash, shortName); } else { - *path =3D g_strdup_printf("%s/%s", cfg->memoryBackingDir, shortNam= e); + *path =3D g_strdup_printf("%s/%s", cfg->memoryBackingDirs[path_ind= ex], shortName); } =20 return 0; @@ -1639,8 +1742,10 @@ qemuGetMemoryBackingDomainPath(virQEMUDriver *driver, * -1 otherwise (with error reported). */ int -qemuGetMemoryBackingPath(virQEMUDriver *driver, - const virDomainDef *def, +qemuGetMemoryBackingPath(const virDomainDef *def, + virQEMUDriver *driver, + virBitmap *autoNodeset, + const size_t targetNode, const char *alias, char **memPath) { @@ -1653,7 +1758,7 @@ qemuGetMemoryBackingPath(virQEMUDriver *driver, return -1; } =20 - if (qemuGetMemoryBackingDomainPath(driver, def, &domainPath) < 0) + if (qemuGetMemoryBackingDomainPath(def, driver, autoNodeset, targetNod= e, &domainPath) < 0) return -1; =20 *memPath =3D g_strdup_printf("%s/%s", domainPath, alias); diff --git a/src/qemu/qemu_conf.h b/src/qemu/qemu_conf.h index aa1e1a626c..76f11e681c 100644 --- a/src/qemu/qemu_conf.h +++ b/src/qemu/qemu_conf.h @@ -221,7 +221,8 @@ struct _virQEMUDriverConfig { unsigned int glusterDebugLevel; bool virtiofsdDebug; =20 - char *memoryBackingDir; + char **memoryBackingDirs; + size_t nmemoryBackingDirs; =20 uid_t swtpm_user; gid_t swtpm_group; @@ -368,11 +369,16 @@ int qemuGetDomainHupageMemPath(virQEMUDriver *driver, unsigned long long pagesize, char **memPath); =20 -int qemuGetMemoryBackingDomainPath(virQEMUDriver *driver, - const virDomainDef *def, +int qemuGetMemoryBackingDomainPath(const virDomainDef *def, + virQEMUDriver *driver, + virBitmap *autoNodeset, + const size_t targetNode, char **path); -int qemuGetMemoryBackingPath(virQEMUDriver *driver, - const virDomainDef *def, + +int qemuGetMemoryBackingPath(const virDomainDef *def, + virQEMUDriver *driver, + virBitmap *autoNodeset, + const size_t targetNode, const char *alias, char **memPath); =20 diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 3801ad623a..42caafb692 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -652,11 +652,16 @@ qemuStateInitialize(bool privileged, cfg->nvramDir); goto error; } - if (g_mkdir_with_parents(cfg->memoryBackingDir, 0777) < 0) { - virReportSystemError(errno, _("Failed to create memory backing dir= %1$s"), - cfg->memoryBackingDir); - goto error; + for (i =3D 0; i < cfg->nmemoryBackingDirs; i++) { + if (g_mkdir_with_parents(cfg->memoryBackingDirs[i], 0777) < 0) { + virReportSystemError(errno, + _("Failed to create = memory backing dir # %1$lu @ %2$s, total: %3$lu"), + i, cfg->memoryBackin= gDirs[i], + cfg->nmemoryBackingD= irs); + goto error; + } } + if (g_mkdir_with_parents(cfg->slirpStateDir, 0777) < 0) { virReportSystemError(errno, _("Failed to create slirp state dir %1= $s"), cfg->slirpStateDir); @@ -792,12 +797,14 @@ qemuStateInitialize(bool privileged, (int)cfg->group); goto error; } - if (chown(cfg->memoryBackingDir, cfg->user, cfg->group) < 0) { - virReportSystemError(errno, - _("unable to set ownership of '%1$s' to %= 2$d:%3$d"), - cfg->memoryBackingDir, (int)cfg->user, - (int)cfg->group); - goto error; + for (i =3D 0; i < cfg->nmemoryBackingDirs; i++) { + if (chown(cfg->memoryBackingDirs[i], cfg->user, cfg->group) < = 0) { + virReportSystemError(errno, + _("unable to set ownership of '%1$s' = to %2$d:%3$d"), + cfg->memoryBackingDirs[i], (int)cfg->= user, + (int)cfg->group); + goto error; + } } if (chown(cfg->slirpStateDir, cfg->user, cfg->group) < 0) { virReportSystemError(errno, @@ -842,10 +849,12 @@ qemuStateInitialize(bool privileged, goto error; } =20 - if (privileged && - virFileUpdatePerm(cfg->memoryBackingDir, - 0, S_IXGRP | S_IXOTH) < 0) - goto error; + for (i =3D 0; i < cfg->nmemoryBackingDirs; i++) { + if (privileged && + virFileUpdatePerm(cfg->memoryBackingDirs[i], + 0, S_IXGRP | S_IXOTH) < 0) + goto error; + } =20 /* Get all the running persistent or transient configs first */ if (virDomainObjListLoadAllConfigs(qemu_driver->domains, diff --git a/src/qemu/qemu_hotplug.c b/src/qemu/qemu_hotplug.c index 7cb1800504..a990d1f9f4 100644 --- a/src/qemu/qemu_hotplug.c +++ b/src/qemu/qemu_hotplug.c @@ -2276,7 +2276,7 @@ qemuDomainAttachMemory(virQEMUDriver *driver, priv, vm->def, mem, true, false, NULL)= < 0) goto cleanup; =20 - if (qemuProcessBuildDestroyMemoryPaths(driver, vm, mem, true) < 0) + if (qemuProcessBuildDestroyMemoryPaths(vm, mem, true) < 0) goto cleanup; =20 if (qemuDomainNamespaceSetupMemory(vm, mem, &teardowndevice) < 0) @@ -2351,7 +2351,7 @@ qemuDomainAttachMemory(virQEMUDriver *driver, qemuDomainObjExitMonitor(vm); =20 if (objAdded && mem) - ignore_value(qemuProcessDestroyMemoryBackingPath(driver, vm, mem)); + ignore_value(qemuProcessDestroyMemoryBackingPath(vm, mem)); =20 virErrorRestore(&orig_err); if (!mem) @@ -4646,7 +4646,7 @@ qemuDomainRemoveMemoryDevice(virQEMUDriver *driver, if (qemuDomainNamespaceTeardownMemory(vm, mem) < 0) VIR_WARN("Unable to remove memory device from /dev"); =20 - if (qemuProcessDestroyMemoryBackingPath(driver, vm, mem) < 0) + if (qemuProcessDestroyMemoryBackingPath(vm, mem) < 0) VIR_WARN("Unable to destroy memory backing path"); =20 qemuDomainReleaseMemoryDeviceSlot(vm, mem); diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index a69878e8bb..0b3f19af7f 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -4059,12 +4059,12 @@ qemuProcessBuildDestroyMemoryPathsImpl(virQEMUDrive= r *driver, =20 =20 int -qemuProcessBuildDestroyMemoryPaths(virQEMUDriver *driver, - virDomainObj *vm, +qemuProcessBuildDestroyMemoryPaths(virDomainObj *vm, virDomainMemoryDef *mem, bool build) { - + qemuDomainObjPrivate *priv =3D vm->privateData; + virQEMUDriver *driver =3D priv->driver; g_autoptr(virQEMUDriverConfig) cfg =3D virQEMUDriverGetConfig(driver); size_t i; bool shouldBuildHP =3D false; @@ -4094,13 +4094,14 @@ qemuProcessBuildDestroyMemoryPaths(virQEMUDriver *d= river, } =20 if (!build || shouldBuildMB) { - g_autofree char *path =3D NULL; - if (qemuGetMemoryBackingDomainPath(driver, vm->def, &path) < 0) - return -1; + for (i =3D 0; i < cfg->nmemoryBackingDirs; i++) { + g_autofree char *path =3D NULL; + if (qemuGetMemoryBackingDomainPath(vm->def, driver, priv->auto= Nodeset, i, &path) < 0) + return -1; =20 - if (qemuProcessBuildDestroyMemoryPathsImpl(driver, vm, - path, build) < 0) - return -1; + if (qemuProcessBuildDestroyMemoryPathsImpl(driver, vm, path, b= uild) < 0) + return -1; + } } =20 return 0; @@ -4108,19 +4109,22 @@ qemuProcessBuildDestroyMemoryPaths(virQEMUDriver *d= river, =20 =20 int -qemuProcessDestroyMemoryBackingPath(virQEMUDriver *driver, - virDomainObj *vm, +qemuProcessDestroyMemoryBackingPath(virDomainObj *vm, virDomainMemoryDef *mem) { g_autofree char *path =3D NULL; + qemuDomainObjPrivate *priv =3D vm->privateData; + g_autoptr(virQEMUDriverConfig) cfg =3D virQEMUDriverGetConfig(priv->dr= iver); + size_t i; =20 - if (qemuGetMemoryBackingPath(driver, vm->def, mem->info.alias, &path) = < 0) - return -1; + for (i =3D 0; i < cfg->nmemoryBackingDirs; i++) { + if (qemuGetMemoryBackingPath(vm->def, priv->driver, priv->autoNode= set, i, mem->info.alias, &path) < 0) + return -1; =20 - if (unlink(path) < 0 && - errno !=3D ENOENT) { - virReportSystemError(errno, _("Unable to remove %1$s"), path); - return -1; + if (unlink(path) < 0 && errno !=3D ENOENT) { + virReportSystemError(errno, _("Unable to remove %1$s"), path); + return -1; + } } =20 return 0; @@ -7334,7 +7338,7 @@ qemuProcessPrepareHost(virQEMUDriver *driver, if (qemuProcessPrepareHostBackendChardev(vm) < 0) return -1; =20 - if (qemuProcessBuildDestroyMemoryPaths(driver, vm, NULL, true) < 0) + if (qemuProcessBuildDestroyMemoryPaths(vm, NULL, true) < 0) return -1; =20 /* Ensure no historical cgroup for this VM is lying around bogus @@ -8705,7 +8709,7 @@ void qemuProcessStop(virQEMUDriver *driver, g_clear_pointer(&priv->mon, qemuMonitorClose); } =20 - qemuProcessBuildDestroyMemoryPaths(driver, vm, NULL, false); + qemuProcessBuildDestroyMemoryPaths(vm, NULL, false); =20 /* Do this before we delete the tree and remove pidfile. */ qemuProcessKillManagedPRDaemon(vm); diff --git a/src/qemu/qemu_process.h b/src/qemu/qemu_process.h index 2324aeb7bd..394af9655d 100644 --- a/src/qemu/qemu_process.h +++ b/src/qemu/qemu_process.h @@ -38,13 +38,11 @@ int qemuProcessStopCPUs(virQEMUDriver *driver, virDomainPausedReason reason, virDomainAsyncJob asyncJob); =20 -int qemuProcessBuildDestroyMemoryPaths(virQEMUDriver *driver, - virDomainObj *vm, +int qemuProcessBuildDestroyMemoryPaths(virDomainObj *vm, virDomainMemoryDef *mem, bool build); =20 -int qemuProcessDestroyMemoryBackingPath(virQEMUDriver *driver, - virDomainObj *vm, +int qemuProcessDestroyMemoryBackingPath(virDomainObj *vm, virDomainMemoryDef *mem); =20 void qemuProcessReconnectAll(virQEMUDriver *driver); diff --git a/tests/testutilsqemu.c b/tests/testutilsqemu.c index ee6cae218a..55cf39b4c8 100644 --- a/tests/testutilsqemu.c +++ b/tests/testutilsqemu.c @@ -321,8 +321,9 @@ int qemuTestDriverInit(virQEMUDriver *driver) cfg->libDir =3D g_strdup("/var/lib/libvirt/qemu"); VIR_FREE(cfg->channelTargetDir); cfg->channelTargetDir =3D g_strdup("/var/run/libvirt/qemu/channel"); - VIR_FREE(cfg->memoryBackingDir); - cfg->memoryBackingDir =3D g_strdup("/var/lib/libvirt/qemu/ram"); + g_free(cfg->memoryBackingDirs); + cfg->memoryBackingDirs =3D g_new0(char *, 1); + cfg->memoryBackingDirs[0] =3D g_strdup("/var/lib/libvirt/qemu/ram"); VIR_FREE(cfg->nvramDir); cfg->nvramDir =3D g_strdup("/var/lib/libvirt/qemu/nvram"); VIR_FREE(cfg->passtStateDir); --=20 2.34.1 From nobody Thu Sep 19 01:06:07 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.libvirt.org designates 8.43.85.245 as permitted sender) client-ip=8.43.85.245; envelope-from=devel-bounces@lists.libvirt.org; helo=lists.libvirt.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of lists.libvirt.org designates 8.43.85.245 as permitted sender) smtp.mailfrom=devel-bounces@lists.libvirt.org; dmarc=pass(p=reject dis=none) header.from=lists.libvirt.org ARC-Seal: i=1; a=rsa-sha256; t=1723760069; cv=none; d=zohomail.com; s=zohoarc; b=EFHEKEtTHo5cIIxMDZ0q46v5+H8rMxa0v7zJVFuztY2faChUsoduklubrLcDeYi3YnyOc4nCTMQO9tBLwBLIYdI7yRZAD3cIVCxxV6g2BF2FnyCm1LOk+wLRPtqc5zVRXjncgQqtQNOhfagkVi1os+Lza0yvtYN+u+0EJ5OUnJQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1723760069; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Reply-To:Reply-To:References:Subject:Subject:To:To:Message-Id; bh=03oWxCXvLnc6RVIj3k/kSbYt551BIwqm2eBKdRLDnD4=; b=V5ASg598vKFUu9f3Uah/HkWYvnXJTcEPkiTs0GiQaiYnhdFOcOAq56w1gXyeL/u2MD09YbHLmz6ywhvipM0zfLsTgdnz6ucS4bosIZvBj2XLd4XmLfeszkhBGlhOyDFZjOWwEprG8skd5RIhKFJ7I5aVD5QNFIfAf4NYUSnUt1Q= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of lists.libvirt.org designates 8.43.85.245 as permitted sender) smtp.mailfrom=devel-bounces@lists.libvirt.org; dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.libvirt.org (lists.libvirt.org [8.43.85.245]) by mx.zohomail.com with SMTPS id 1723760069718278.33780465650784; Thu, 15 Aug 2024 15:14:29 -0700 (PDT) Received: by lists.libvirt.org (Postfix, from userid 996) id AB8E313EF; Thu, 15 Aug 2024 18:14:28 -0400 (EDT) Received: from lists.libvirt.org (localhost [IPv6:::1]) by lists.libvirt.org (Postfix) with ESMTP id 33E751525; Thu, 15 Aug 2024 18:13:26 -0400 (EDT) Received: by lists.libvirt.org (Postfix, from userid 996) id 5927312ED; Thu, 15 Aug 2024 18:13:20 -0400 (EDT) Received: from mx0a-00190b01.pphosted.com (mx0a-00190b01.pphosted.com [67.231.149.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.libvirt.org (Postfix) with ESMTPS id B576412B4 for ; Thu, 15 Aug 2024 18:13:19 -0400 (EDT) Received: from pps.filterd (m0409409.ppops.net [127.0.0.1]) by m0409409.ppops.net-00190b01. (8.18.1.2/8.18.1.2) with ESMTP id 47FASpNW029090; Thu, 15 Aug 2024 23:13:18 +0100 Received: from prod-mail-ppoint2 (prod-mail-ppoint2.akamai.com [184.51.33.19] (may be forged)) by m0409409.ppops.net-00190b01. (PPS) with ESMTPS id 41107rpsbd-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 15 Aug 2024 23:13:18 +0100 (BST) Received: from pps.filterd (prod-mail-ppoint2.akamai.com [127.0.0.1]) by prod-mail-ppoint2.akamai.com (8.17.1.19/8.17.1.19) with ESMTP id 47FM7kGF000797; Thu, 15 Aug 2024 18:13:16 -0400 Received: from prod-mail-relay11.akamai.com ([172.27.118.250]) by prod-mail-ppoint2.akamai.com (PPS) with ESMTP id 40x3hyuhhr-1; Thu, 15 Aug 2024 18:13:16 -0400 Received: from bos-lhvcg5.bos01.corp.akamai.com (bos-lhvcg5.bos01.corp.akamai.com [172.28.222.58]) by prod-mail-relay11.akamai.com (Postfix) with ESMTP id 666E133EE3; Thu, 15 Aug 2024 22:13:16 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on lists.libvirt.org X-Spam-Level: ** X-Spam-Status: No, score=2.1 required=5.0 tests=DKIM_INVALID,DKIM_SIGNED, MAILING_LIST_MULTI,RCVD_IN_DNSWL_LOW,RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL,RCVD_IN_SBL_CSS,SPF_HELO_NONE,T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.4 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=akamai.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=jan2016.eng; bh=k0Jyid+tH Fccoug3vsIyUk/akdF88v9Y3tSN8hmP/1Y=; b=hUo0dgqDPl+Pmkq1dWQaL5Fyc Ljcbxolz26d46bPS76uCQDjA/pNvC9vPI6kmjiAa8FQtAC87aDwG0IVp4jZRlWsc ctUxMDplNfaF8//b42xcm2S1XWbU7gIuwEp63vyJX6lUGKeGs9RnF/K+ufwpLxTs jC/2seqR/mwDdKZT0iv08MnF6HhPXoeNqvqjYZ6krDf/8RxZs4COJFdL9IAD/Pdu tCG+3ow/40SSpb/74ApFSRzNUkbFw4z0jO2mK6s2bws32qgrreB9rPD4UN0X11p4 cdA1kPZrewczjbv6BTQJ71fWYEhJoDQdoLA2gKpiTWlswyc1qt4JLKxhfruSA== To: devel@lists.libvirt.org Subject: [PATCH v3 2/4] Support live migration between file-backed memory and anonymous memory. Date: Thu, 15 Aug 2024 18:13:06 -0400 Message-Id: <20240815221308.3892813-3-mgalaxy@akamai.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240815221308.3892813-1-mgalaxy@akamai.com> References: <20240815221308.3892813-1-mgalaxy@akamai.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.28.16 definitions=2024-08-15_14,2024-08-15_01,2024-05-17_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 malwarescore=0 bulkscore=0 mlxscore=0 adultscore=0 spamscore=0 mlxlogscore=999 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2407110000 definitions=main-2408150161 X-Proofpoint-GUID: n2grbnwyykuJ5LxULHoI_HPe2LX9bgC9 X-Proofpoint-ORIG-GUID: n2grbnwyykuJ5LxULHoI_HPe2LX9bgC9 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.28.16 definitions=2024-08-15_14,2024-08-15_01,2024-05-17_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 phishscore=0 mlxscore=0 malwarescore=0 mlxlogscore=999 impostorscore=0 adultscore=0 spamscore=0 lowpriorityscore=0 bulkscore=0 clxscore=1015 priorityscore=1501 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2407110000 definitions=main-2408150162 Content-Transfer-Encoding: quoted-printable Message-ID-Hash: AELTNZR4FMQXIUEUI4DFRA4CKP7CWTHD X-Message-ID-Hash: AELTNZR4FMQXIUEUI4DFRA4CKP7CWTHD X-MailFrom: mgalaxy@akamai.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-config-1; header-match-config-2; header-match-config-3; header-match-devel.lists.libvirt.org-0; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; suspicious-header CC: steven.sistare@oracle.com, johunt@akamai.com, cweng@akamai.com, bchaney@akamai.com, trakotoz@akamai.com, mkletzan@redhat.com, anthony.yznaga@oracle.com, mprivozn@redhat.com, Michael Galaxy X-Mailman-Version: 3.2.2 Precedence: list List-Id: Development discussions about the libvirt library & tools Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: From: Michael Galaxy via Devel Reply-To: mgalaxy@akamai.com X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1723760070708116600 Content-Type: text/plain; charset="utf-8" From: Michael Galaxy In our environment, we need to convert VMs into a live-update-comptabile configuration "on-the-fly" (via live migration). More specifically: We need to convert between anonymous memory-backed VMs and file-backed memory VMs. So, for this very specific case, this needs to work when host-level PMEM is being enabled. QEMU does not have a problem with this at all, but we need to relax the rules here a bit so that libvirt allows it to go through normally. Signed-off-by: Michael Galaxy --- src/qemu/qemu_domain.c | 24 +++++++++++++++++++----- 1 file changed, 19 insertions(+), 5 deletions(-) diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c index f87ba6ba51..d574f6ab98 100644 --- a/src/qemu/qemu_domain.c +++ b/src/qemu/qemu_domain.c @@ -8748,11 +8748,25 @@ qemuDomainABIStabilityCheck(const virDomainDef *src, size_t i; =20 if (src->mem.source !=3D dst->mem.source) { - virReportError(VIR_ERR_CONFIG_UNSUPPORTED, - _("Target memoryBacking source '%1$s' doesn't match= source memoryBacking source'%2$s'"), - virDomainMemorySourceTypeToString(dst->mem.source), - virDomainMemorySourceTypeToString(src->mem.source)); - return false; + /* + * The current use case for this is the live migration of live-upd= ate + * capable CPR guests mounted on PMEM devices at the host + * level (not in-guest PMEM). QEMU has no problem doing these kind= s of + * live migrations between these two memory backends, so let them = go through. + * This allows us to "upgrade" guests from regular memory to file-= backed + * memory seemlessly without taking them down. + */ + if (!((src->mem.source =3D=3D VIR_DOMAIN_MEMORY_SOURCE_NONE + && dst->mem.source =3D=3D VIR_DOMAIN_MEMORY_SOURCE_FILE) || + (src->mem.source =3D=3D VIR_DOMAIN_MEMORY_SOURCE_FILE + && dst->mem.source =3D=3D VIR_DOMAIN_MEMORY_SOURCE_NONE)))= { + + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, + _("Target memoryBacking source '%1$s' doesn't m= atch source memoryBacking source'%2$s'"), + virDomainMemorySourceTypeToString(dst->mem.sour= ce), + virDomainMemorySourceTypeToString(src->mem.sour= ce)); + return false; + } } =20 for (i =3D 0; i < src->nmems; i++) { --=20 2.34.1 From nobody Thu Sep 19 01:06:07 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.libvirt.org designates 8.43.85.245 as permitted sender) client-ip=8.43.85.245; envelope-from=devel-bounces@lists.libvirt.org; helo=lists.libvirt.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of lists.libvirt.org designates 8.43.85.245 as permitted sender) smtp.mailfrom=devel-bounces@lists.libvirt.org; dmarc=pass(p=reject dis=none) header.from=lists.libvirt.org ARC-Seal: i=1; a=rsa-sha256; t=1723760045; cv=none; d=zohomail.com; s=zohoarc; b=j3eFQMYMBMKwLZAEDitEeJvT23wuOFviUj+ImUpntJobEMbzdWxOjtYzeg90Gr5F6YuWpzMyHhhJW/+kcC8xSSxvzhKwW0kCBqFE2q3heD8/sHdKavGUX8ZyZ0i0SWmZYKHCaPygeRRbEMkuRTklnwgPB2eUlfhXwD1V528/l8M= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1723760045; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Reply-To:Reply-To:References:Subject:Subject:To:To:Message-Id; bh=yjaD8n95vbphMSqLk8GIp7dn41eDYgz/3KCy+TYjoCo=; b=jUPJP6zG5rmyUFHGRKEp5+3XHJcv0EIHxqmUwHRgmVCHbKVhTDguB3QRP5Ql+Gn9lAPQYrS/8LT7y9Dmrx4WPlDR8F48Gx/3QMo9hO5uiWbrXyGUw2ZE8yTT79RWC9cSeV4o4mguIKAfQqHQO3MWP1DHn1MLmE0j+shSm01MHOo= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of lists.libvirt.org designates 8.43.85.245 as permitted sender) smtp.mailfrom=devel-bounces@lists.libvirt.org; dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.libvirt.org (lists.libvirt.org [8.43.85.245]) by mx.zohomail.com with SMTPS id 1723760045791533.5789201768855; Thu, 15 Aug 2024 15:14:05 -0700 (PDT) Received: by lists.libvirt.org (Postfix, from userid 996) id A86451381; Thu, 15 Aug 2024 18:14:04 -0400 (EDT) Received: from lists.libvirt.org (localhost [IPv6:::1]) by lists.libvirt.org (Postfix) with ESMTP id 4550714A9; Thu, 15 Aug 2024 18:13:23 -0400 (EDT) Received: by lists.libvirt.org (Postfix, from userid 996) id AF25312B6; Thu, 15 Aug 2024 18:13:19 -0400 (EDT) Received: from mx0b-00190b01.pphosted.com (mx0b-00190b01.pphosted.com [67.231.157.127]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.libvirt.org (Postfix) with ESMTPS id 3083912B4 for ; Thu, 15 Aug 2024 18:13:19 -0400 (EDT) Received: from pps.filterd (m0050096.ppops.net [127.0.0.1]) by m0050096.ppops.net-00190b01. (8.18.1.2/8.18.1.2) with ESMTP id 47FITJIc010077; Thu, 15 Aug 2024 23:13:18 +0100 Received: from prod-mail-ppoint2 (prod-mail-ppoint2.akamai.com [184.51.33.19] (may be forged)) by m0050096.ppops.net-00190b01. (PPS) with ESMTPS id 40x0mepb0d-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 15 Aug 2024 23:13:18 +0100 (BST) Received: from pps.filterd (prod-mail-ppoint2.akamai.com [127.0.0.1]) by prod-mail-ppoint2.akamai.com (8.17.1.19/8.17.1.19) with ESMTP id 47FM7kAT000800; Thu, 15 Aug 2024 18:13:17 -0400 Received: from prod-mail-relay11.akamai.com ([172.27.118.250]) by prod-mail-ppoint2.akamai.com (PPS) with ESMTP id 40x3hyuhhs-1; Thu, 15 Aug 2024 18:13:17 -0400 Received: from bos-lhvcg5.bos01.corp.akamai.com (bos-lhvcg5.bos01.corp.akamai.com [172.28.222.58]) by prod-mail-relay11.akamai.com (Postfix) with ESMTP id 326A033EE3; Thu, 15 Aug 2024 22:13:17 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on lists.libvirt.org X-Spam-Level: ** X-Spam-Status: No, score=2.1 required=5.0 tests=DKIM_INVALID,DKIM_SIGNED, MAILING_LIST_MULTI,RCVD_IN_DNSWL_LOW,RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL,RCVD_IN_SBL_CSS,SPF_HELO_NONE,T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.4 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=akamai.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=jan2016.eng; bh=jDMMzO2S8 y/O7jz9jKaIIecVE1WZ2YKtY5NIls68swI=; b=f3qWc12LlO1iXOReo+u/Lw48R wkLg0F2kT1CwTLli4WPb6+/fYZFlBK8qR7xGENVlZHged5dP6+C08h1bf7nL4xnz LZXmlGVCKIvTzrwsJMK2ouujlJu07JcDuySvg8Ko4yt9MKTG9WzGIkN05zO5D75L XZsUnEqjHoMDfyzV0bPmrwrp0KI9qjpRlczGOvrIdadRXSXE6VsZrczFp0jJIcTj kZW6Rv7gnkkMFBW0u2KWFlerKGsOE/kxh/ZaJbZllMNwihy2Frik1v1+RClkaxUs viElcJBZOcQq29HCLTY3go08jo/K8t3FpcF1y0Q4q1pTlGiwUbmUBqgGx5LRA== To: devel@lists.libvirt.org Subject: [PATCH v3 3/4] Update configuration file documentation to support multiple memory backends Date: Thu, 15 Aug 2024 18:13:07 -0400 Message-Id: <20240815221308.3892813-4-mgalaxy@akamai.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240815221308.3892813-1-mgalaxy@akamai.com> References: <20240815221308.3892813-1-mgalaxy@akamai.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.28.16 definitions=2024-08-15_14,2024-08-15_01,2024-05-17_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 malwarescore=0 bulkscore=0 mlxscore=0 adultscore=0 spamscore=0 mlxlogscore=999 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2407110000 definitions=main-2408150161 X-Proofpoint-GUID: JqGj_-Kle1RafslYhmJSDiKCIp_0eSae X-Proofpoint-ORIG-GUID: JqGj_-Kle1RafslYhmJSDiKCIp_0eSae X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.28.16 definitions=2024-08-15_14,2024-08-15_01,2024-05-17_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 adultscore=0 mlxlogscore=999 lowpriorityscore=0 phishscore=0 clxscore=1015 mlxscore=0 spamscore=0 malwarescore=0 suspectscore=0 impostorscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2407110000 definitions=main-2408150162 Content-Transfer-Encoding: quoted-printable Message-ID-Hash: IMYX6AVUU6CB5ARJJ2ZTMW6AX57PN4JK X-Message-ID-Hash: IMYX6AVUU6CB5ARJJ2ZTMW6AX57PN4JK X-MailFrom: mgalaxy@akamai.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-config-1; header-match-config-2; header-match-config-3; header-match-devel.lists.libvirt.org-0; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; suspicious-header CC: steven.sistare@oracle.com, johunt@akamai.com, cweng@akamai.com, bchaney@akamai.com, trakotoz@akamai.com, mkletzan@redhat.com, anthony.yznaga@oracle.com, mprivozn@redhat.com, Michael Galaxy X-Mailman-Version: 3.2.2 Precedence: list List-Id: Development discussions about the libvirt library & tools Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: From: Michael Galaxy via Devel Reply-To: mgalaxy@akamai.com X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1723760046884116600 Content-Type: text/plain; charset="utf-8" From: Michael Galaxy Signed-off-by: Michael Galaxy --- docs/kbase/virtiofs.rst | 2 ++ src/qemu/qemu.conf.in | 4 ++++ 2 files changed, 6 insertions(+) diff --git a/docs/kbase/virtiofs.rst b/docs/kbase/virtiofs.rst index 457c15da7f..931ba96104 100644 --- a/docs/kbase/virtiofs.rst +++ b/docs/kbase/virtiofs.rst @@ -144,6 +144,8 @@ memobject interface). =20 # This directory is used for memoryBacking source if configured as = file. # NOTE: big files will be stored here + # NOTE: In the case of a NUMA-aware system (such has shm/tmpfs or P= MEM), + # one can also do: memory_backing_dir =3D [ "/path/to/node/0"= , "/path/to/node/1" ] memory_backing_dir =3D "/dev/shm/" =20 * Use hugepage-backed memory diff --git a/src/qemu/qemu.conf.in b/src/qemu/qemu.conf.in index 6bc2140dcb..3c52de0192 100644 --- a/src/qemu/qemu.conf.in +++ b/src/qemu/qemu.conf.in @@ -897,6 +897,10 @@ =20 # This directory is used for memoryBacking source if configured as file. # NOTE: big files will be stored here +# NOTE: In the case of a NUMA-aware system (such has shm/tmpfs or PMEM), +# one can also do: memory_backing_dir =3D [ "/path/to/node/0", "/pat= h/to/node/1" ] +# Be advised that the number of backing directories needs to match +# the number of NUMA nodes on the system. #memory_backing_dir =3D "/var/lib/libvirt/qemu/ram" =20 # Path to the SCSI persistent reservations helper. This helper is --=20 2.34.1 From nobody Thu Sep 19 01:06:07 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.libvirt.org designates 8.43.85.245 as permitted sender) client-ip=8.43.85.245; envelope-from=devel-bounces@lists.libvirt.org; helo=lists.libvirt.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of lists.libvirt.org designates 8.43.85.245 as permitted sender) smtp.mailfrom=devel-bounces@lists.libvirt.org; dmarc=pass(p=reject dis=none) header.from=lists.libvirt.org ARC-Seal: i=1; a=rsa-sha256; t=1723760156; cv=none; d=zohomail.com; s=zohoarc; b=AvEqbgoq/G3EFMbX+qzIsen1uEkqrwjeiH5zwIGsDMiQB1JGXshwdQDvZZMhwaa82Jb34Jp1ZOwl8fwmJu1/ImqVKBNTTWzGs3vKT/Sc6T0JmDCxCzc3F9O+CUK15YE7AQwq0fYXUAykr0mFivetHTw7PnV+z9D9Xm/W3ouq5o4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1723760156; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Reply-To:Reply-To:References:Subject:Subject:To:To:Message-Id; bh=g8gdZ1DOx/hMyFoAEGRjxVmLoGlI07TlGUl5yvGDmv4=; b=Maz4SkKCTPAF/k51/vsqa1twUqLdtl1PCW/vbjBozYN3ugH3AqBRL+ukzrZcOEcdvqLncF6a5Ji/g+uXDh+3ebwlnefmIYRdd/eBx1x+5KJehMm9XnM9JbUfMr2/KbdKYgM2aK9yfjIMS87ZaaIAffaCAJwJypkPw8XEpEBSj1k= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of lists.libvirt.org designates 8.43.85.245 as permitted sender) smtp.mailfrom=devel-bounces@lists.libvirt.org; dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.libvirt.org (lists.libvirt.org [8.43.85.245]) by mx.zohomail.com with SMTPS id 1723760156109791.4182659840959; Thu, 15 Aug 2024 15:15:56 -0700 (PDT) Received: by lists.libvirt.org (Postfix, from userid 996) id 1D8271352; Thu, 15 Aug 2024 18:15:55 -0400 (EDT) Received: from lists.libvirt.org (localhost [IPv6:::1]) by lists.libvirt.org (Postfix) with ESMTP id 43EB41596; Thu, 15 Aug 2024 18:14:04 -0400 (EDT) Received: by lists.libvirt.org (Postfix, from userid 996) id 6E6561406; Thu, 15 Aug 2024 18:14:00 -0400 (EDT) Received: from mx0b-00190b01.pphosted.com (mx0b-00190b01.pphosted.com [67.231.157.127]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.libvirt.org (Postfix) with ESMTPS id BDDCC13CA for ; Thu, 15 Aug 2024 18:13:47 -0400 (EDT) Received: from pps.filterd (m0409411.ppops.net [127.0.0.1]) by m0409411.ppops.net-00190b01. (8.18.1.2/8.18.1.2) with ESMTP id 47FDVg8Z003786; Thu, 15 Aug 2024 23:13:47 +0100 Received: from prod-mail-ppoint1 (prod-mail-ppoint1.akamai.com [184.51.33.18] (may be forged)) by m0409411.ppops.net-00190b01. (PPS) with ESMTPS id 411jms5yqh-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 15 Aug 2024 23:13:46 +0100 (BST) Received: from pps.filterd (prod-mail-ppoint1.akamai.com [127.0.0.1]) by prod-mail-ppoint1.akamai.com (8.17.1.19/8.17.1.19) with ESMTP id 47FM7quS025622; Thu, 15 Aug 2024 18:13:18 -0400 Received: from prod-mail-relay11.akamai.com ([172.27.118.250]) by prod-mail-ppoint1.akamai.com (PPS) with ESMTP id 40x3hykh4h-1; Thu, 15 Aug 2024 18:13:18 -0400 Received: from bos-lhvcg5.bos01.corp.akamai.com (bos-lhvcg5.bos01.corp.akamai.com [172.28.222.58]) by prod-mail-relay11.akamai.com (Postfix) with ESMTP id 032DC33EE3; Thu, 15 Aug 2024 22:13:18 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on lists.libvirt.org X-Spam-Level: X-Spam-Status: No, score=-1.5 required=5.0 tests=DKIM_INVALID,DKIM_SIGNED, MAILING_LIST_MULTI,RCVD_IN_DNSWL_LOW,RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.4 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=akamai.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=jan2016.eng; bh=Ltej/FSAa RPSKZyPjyZ3ByK4gzNPT59SNPYGSJeaTN8=; b=G42OjqhPS+8WsC/8o9yMqxVbh stqbGwPQZCGx6tU3RCHvUE6bJzFHbecQwFi4sEkBbTycNJsOMYVVOM/iUWmysrnj /NF+26c1S+0G0JBPVuZViDp9EGmsgxc97hjagPKcZzR2bg53hb8aA7kfkCZvNJJG 9kCJvHpywZ3AzGCFKkO+aLmMJofkNoPuy9YZAsntrfqS9ALb9W7HTcl4P3nlObM6 DBUntxq8DlIcdP0TKNoDYXfzqm1LnVn+is+Spue56xdgmHwP47BlnaGIIAmhb221 0TSaQJtKezgmRurtmYQc8UeQ0TwgaLVhtP8EP737QNzW68HkTMHLRjdNofiVQ== To: devel@lists.libvirt.org Subject: [PATCH v3 4/4] Add NEWS item for multiple memory backend support Date: Thu, 15 Aug 2024 18:13:08 -0400 Message-Id: <20240815221308.3892813-5-mgalaxy@akamai.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240815221308.3892813-1-mgalaxy@akamai.com> References: <20240815221308.3892813-1-mgalaxy@akamai.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.28.16 definitions=2024-08-15_14,2024-08-15_01,2024-05-17_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 malwarescore=0 adultscore=0 phishscore=0 mlxlogscore=999 bulkscore=0 suspectscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2407110000 definitions=main-2408150161 X-Proofpoint-ORIG-GUID: 2UfJ5AUKhXkNpJWHY5RiX9aVsvGaUxp0 X-Proofpoint-GUID: 2UfJ5AUKhXkNpJWHY5RiX9aVsvGaUxp0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.28.16 definitions=2024-08-15_14,2024-08-15_01,2024-05-17_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 mlxlogscore=920 impostorscore=0 adultscore=0 priorityscore=1501 clxscore=1015 malwarescore=0 lowpriorityscore=0 phishscore=0 bulkscore=0 suspectscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2407110000 definitions=main-2408150162 Content-Transfer-Encoding: quoted-printable Message-ID-Hash: JS7KPBX5LIOTVZJT47MB4EQW2XLAKP7D X-Message-ID-Hash: JS7KPBX5LIOTVZJT47MB4EQW2XLAKP7D X-MailFrom: mgalaxy@akamai.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-config-1; header-match-config-2; header-match-config-3; header-match-devel.lists.libvirt.org-0; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; suspicious-header CC: steven.sistare@oracle.com, johunt@akamai.com, cweng@akamai.com, bchaney@akamai.com, trakotoz@akamai.com, mkletzan@redhat.com, anthony.yznaga@oracle.com, mprivozn@redhat.com, Michael Galaxy X-Mailman-Version: 3.2.2 Precedence: list List-Id: Development discussions about the libvirt library & tools Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: From: Michael Galaxy via Devel Reply-To: mgalaxy@akamai.com X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1723760157194116600 Content-Type: text/plain; charset="utf-8" From: Michael Galaxy Signed-off-by: Michael Galaxy --- NEWS.rst | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/NEWS.rst b/NEWS.rst index bbd30ab3b5..c546f0757a 100644 --- a/NEWS.rst +++ b/NEWS.rst @@ -25,6 +25,14 @@ v10.7.0 (unreleased) backed network devices. So, create new net fds and pass them via SCM_RIGHTS to CH during restore operation. =20 + * qemu: memory backing directories now support NUMA + + The primary use case of this is when persistent memory is used on + NUMA-aware systems. The setting ``memory_backing_dir`` in qemu.conf + now supports either an array or the original string. This allows + you to provide a array of locations to use that are NUMA-compatible + if needed. + * **Bug fixes** =20 =20 --=20 2.34.1