From nobody Wed May 15 01:20:46 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 63.128.21.124 as permitted sender) client-ip=63.128.21.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 63.128.21.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1606897657; cv=none; d=zohomail.com; s=zohoarc; b=JmQ5FA9r/P8lO4vir0Rin8g8sSPNfyBkY0BzaVZHtkWV4VITcs/atqfK5e0r7lopRDe4kWN0XxL8dEkfpthuBoFyoUFaJFkWhDsc1kc3yDUviLKUk+7eNs8iipfJnFCSXQ6xtCiMadS+pdQkyEL/O/Kr/jaZZ7/mlXzF2Np77Jw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1606897657; h=Content-Type:Content-Transfer-Encoding:Date:From:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:To; bh=Oq/65vo8FqkRTigZtvThjUK1o+YDLqOUGe42WLAnKKU=; b=YBxDFimZpG01mtGpZz0quLHie5DdJ67kaFKND7o8+0PhN5MXH6/pSkiRu0J3SknTMWXK+6Ttkk/sLhv8dygWJiAQzOg17onBupPmav0sfUOuj4mG0ctuDpHjyKWrCLerCEZVysJf1EwFvabt+oA6EWpqhxVb6TW8D5TdGMNIfxE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 63.128.21.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [63.128.21.124]) by mx.zohomail.com with SMTPS id 1606897657937873.6683685974238; Wed, 2 Dec 2020 00:27:37 -0800 (PST) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-254-HLzE740EMzu22Hj-cchyiQ-1; Wed, 02 Dec 2020 03:27:34 -0500 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id B086B185E48F; Wed, 2 Dec 2020 08:27:28 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id C6F575D9C6; Wed, 2 Dec 2020 08:27:26 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id C11A64BB7B; Wed, 2 Dec 2020 08:27:22 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 0B28QaSr029998 for ; Wed, 2 Dec 2020 03:26:36 -0500 Received: by smtp.corp.redhat.com (Postfix) id 188CC5C1BD; Wed, 2 Dec 2020 08:26:36 +0000 (UTC) Received: from localhost.localdomain (unknown [10.40.192.81]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8E81A5C1B4 for ; Wed, 2 Dec 2020 08:26:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1606897656; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=Oq/65vo8FqkRTigZtvThjUK1o+YDLqOUGe42WLAnKKU=; b=FMt0jtd2nwirOo/L1auF8ZAlCi3e3Sxwt55vDYGqMEklSl5t1eJ+NcFDmW2PHY5T4jECBN YRMJKLfNbO0bscZbmkOnO7WlDYkR0whemDvF3RNmvL6Ezgm5GHrVgdevf01KyjPUeu0qs2 By3OclC4MayvNO2UjVDTr8UrVbfZqaU= X-MC-Unique: HLzE740EMzu22Hj-cchyiQ-1 From: Michal Privoznik To: libvir-list@redhat.com Subject: [PATCH v2] qemu: Don't cache NUMA caps Date: Wed, 2 Dec 2020 09:26:30 +0100 Message-Id: <21c6dda4cb9ed8dc39e98480e2cc211ccfc1a48f.1606897527.git.mprivozn@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-loop: libvir-list@redhat.com X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" In v6.0.0-rc1~439 (and friends) we tried to cache NUMA capabilities because we assumed they are immutable. And to some extent they are (NUMA hotplug is not a thing, is it). However, our capabilities contain also some runtime info that can change, e.g. hugepages pool allocation sizes or total amount of memory per node (host side memory hotplug might change the value). Because of the caching we might not be reporting the correct runtime info in 'virsh capabilities'. The NUMA caps are used in three places: 1) 'virsh capabilities' 2) domain startup, when parsing numad reply 3) parsing domain private data XML In cases 2) and 3) we need NUMA caps to construct list of physical CPUs that belong to NUMA nodes from numad reply. And while this may seem static, it's not really because of possible CPU hotplug on physical host. There are two possible approaches: 1) build a validation mechanism that would invalidate the cached NUMA caps, or 2) drop the caching and construct NUMA caps from scratch on each use. In this commit, the latter approach is implemented, because it's easier. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=3D1819058 Fixes: 1a1d848694f6c2f1d98a371124928375bc3bb4a3 Signed-off-by: Michal Privoznik Reviewed-by: Daniel Henrique Barboza --- v2 of: https://www.redhat.com/archives/libvir-list/2020-December/msg00023.html diff to v1: - instead of fixing cached capabilities, drop caching completely src/qemu/qemu_conf.c | 23 +---------------------- src/qemu/qemu_conf.h | 6 ------ src/qemu/qemu_domain.c | 7 +++---- src/qemu/qemu_driver.c | 1 - src/qemu/qemu_process.c | 7 +++---- 5 files changed, 7 insertions(+), 37 deletions(-) diff --git a/src/qemu/qemu_conf.c b/src/qemu/qemu_conf.c index cbdde0c0dc..0ae86e5468 100644 --- a/src/qemu/qemu_conf.c +++ b/src/qemu/qemu_conf.c @@ -1289,27 +1289,6 @@ virQEMUDriverCreateXMLConf(virQEMUDriverPtr driver, } =20 =20 -virCapsHostNUMAPtr -virQEMUDriverGetHostNUMACaps(virQEMUDriverPtr driver) -{ - virCapsHostNUMAPtr hostnuma; - - qemuDriverLock(driver); - - if (!driver->hostnuma) - driver->hostnuma =3D virCapabilitiesHostNUMANewHost(); - - hostnuma =3D driver->hostnuma; - - qemuDriverUnlock(driver); - - if (hostnuma) - virCapabilitiesHostNUMARef(hostnuma); - - return hostnuma; -} - - virCPUDefPtr virQEMUDriverGetHostCPU(virQEMUDriverPtr driver) { @@ -1381,7 +1360,7 @@ virCapsPtr virQEMUDriverCreateCapabilities(virQEMUDri= verPtr driver) "DOI \"%s\"", model, doi); } =20 - caps->host.numa =3D virQEMUDriverGetHostNUMACaps(driver); + caps->host.numa =3D virCapabilitiesHostNUMANewHost(); caps->host.cpu =3D virQEMUDriverGetHostCPU(driver); return g_steal_pointer(&caps); } diff --git a/src/qemu/qemu_conf.h b/src/qemu/qemu_conf.h index 411d08db36..7025b5222e 100644 --- a/src/qemu/qemu_conf.h +++ b/src/qemu/qemu_conf.h @@ -270,11 +270,6 @@ struct _virQEMUDriver { */ virCapsPtr caps; =20 - /* Lazy initialized on first use, immutable thereafter. - * Require lock to get the pointer & do optional initialization - */ - virCapsHostNUMAPtr hostnuma; - /* Lazy initialized on first use, immutable thereafter. * Require lock to get the pointer & do optional initialization */ @@ -337,7 +332,6 @@ virQEMUDriverConfigSetDefaults(virQEMUDriverConfigPtr c= fg); =20 virQEMUDriverConfigPtr virQEMUDriverGetConfig(virQEMUDriverPtr driver); =20 -virCapsHostNUMAPtr virQEMUDriverGetHostNUMACaps(virQEMUDriverPtr driver); virCPUDefPtr virQEMUDriverGetHostCPU(virQEMUDriverPtr driver); virCapsPtr virQEMUDriverCreateCapabilities(virQEMUDriverPtr driver); virCapsPtr virQEMUDriverGetCapabilities(virQEMUDriverPtr driver, diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c index 663c0af867..8e9c0224e4 100644 --- a/src/qemu/qemu_domain.c +++ b/src/qemu/qemu_domain.c @@ -2498,8 +2498,7 @@ qemuDomainObjPrivateXMLParseVcpu(xmlNodePtr node, =20 static int qemuDomainObjPrivateXMLParseAutomaticPlacement(xmlXPathContextPtr ctxt, - qemuDomainObjPrivatePtr pri= v, - virQEMUDriverPtr driver) + qemuDomainObjPrivatePtr pri= v) { g_autoptr(virCapsHostNUMA) caps =3D NULL; g_autofree char *nodeset =3D NULL; @@ -2513,7 +2512,7 @@ qemuDomainObjPrivateXMLParseAutomaticPlacement(xmlXPa= thContextPtr ctxt, if (!nodeset && !cpuset) return 0; =20 - if (!(caps =3D virQEMUDriverGetHostNUMACaps(driver))) + if (!(caps =3D virCapabilitiesHostNUMANewHost())) return -1; =20 /* Figure out how big the nodeset bitmap needs to be. @@ -3114,7 +3113,7 @@ qemuDomainObjPrivateXMLParse(xmlXPathContextPtr ctxt, } VIR_FREE(nodes); =20 - if (qemuDomainObjPrivateXMLParseAutomaticPlacement(ctxt, priv, driver)= < 0) + if (qemuDomainObjPrivateXMLParseAutomaticPlacement(ctxt, priv) < 0) goto error; =20 if ((tmp =3D virXPathString("string(./libDir/@path)", ctxt))) diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 8eaa3ce68f..d2077f8f16 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -1136,7 +1136,6 @@ qemuStateCleanup(void) virObjectUnref(qemu_driver->qemuCapsCache); virObjectUnref(qemu_driver->xmlopt); virCPUDefFree(qemu_driver->hostcpu); - virCapabilitiesHostNUMAUnref(qemu_driver->hostnuma); virObjectUnref(qemu_driver->caps); ebtablesContextFree(qemu_driver->ebtables); VIR_FREE(qemu_driver->qemuImgBinary); diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index 579b3c3713..49528f3dab 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -6197,8 +6197,7 @@ qemuProcessUpdateGuestCPU(virDomainDefPtr def, =20 =20 static int -qemuProcessPrepareDomainNUMAPlacement(virQEMUDriverPtr driver, - virDomainObjPtr vm) +qemuProcessPrepareDomainNUMAPlacement(virDomainObjPtr vm) { qemuDomainObjPrivatePtr priv =3D vm->privateData; g_autofree char *nodeset =3D NULL; @@ -6226,7 +6225,7 @@ qemuProcessPrepareDomainNUMAPlacement(virQEMUDriverPt= r driver, if (virBitmapParse(nodeset, &numadNodeset, VIR_DOMAIN_CPUMASK_LEN) < 0) return -1; =20 - if (!(caps =3D virQEMUDriverGetHostNUMACaps(driver))) + if (!(caps =3D virCapabilitiesHostNUMANewHost())) return -1; =20 /* numad may return a nodeset that only contains cpus but cgroups don'= t play @@ -6428,7 +6427,7 @@ qemuProcessPrepareDomain(virQEMUDriverPtr driver, } virDomainAuditSecurityLabel(vm, true); =20 - if (qemuProcessPrepareDomainNUMAPlacement(driver, vm) < 0) + if (qemuProcessPrepareDomainNUMAPlacement(vm) < 0) return -1; } =20 --=20 2.26.2