From nobody Thu May 2 23:24:06 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1492684761773347.74985099316825; Thu, 20 Apr 2017 03:39:21 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 0C1187F6C8; Thu, 20 Apr 2017 10:39:18 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 379B517AFB; Thu, 20 Apr 2017 10:39:17 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 8F35418523CF; Thu, 20 Apr 2017 10:38:55 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id v3KAc4O1006603 for ; Thu, 20 Apr 2017 06:38:04 -0400 Received: by smtp.corp.redhat.com (Postfix) id 7123977E3D; Thu, 20 Apr 2017 10:38:04 +0000 (UTC) Received: from mx1.redhat.com (ext-mx06.extmail.prod.ext.phx2.redhat.com [10.5.110.30]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 9D73977E44; Thu, 20 Apr 2017 10:38:02 +0000 (UTC) Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 268303DBFB; Thu, 20 Apr 2017 10:38:01 +0000 (UTC) Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga105.fm.intel.com with ESMTP; 20 Apr 2017 03:38:00 -0700 Received: from otcloud-gateway.bj.intel.com (HELO z-nuc-2.maas) ([10.238.153.58]) by orsmga005.jf.intel.com with ESMTP; 20 Apr 2017 03:37:58 -0700 DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 0C1187F6C8 Authentication-Results: ext-mx01.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=intel.com Authentication-Results: ext-mx01.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=libvir-list-bounces@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 0C1187F6C8 DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 268303DBFB Authentication-Results: ext-mx06.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=intel.com Authentication-Results: ext-mx06.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=shaohe.feng@intel.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 268303DBFB X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.37,225,1488873600"; d="scan'208";a="90119578" From: Shaohe Feng To: libvir-list@redhat.com Date: Thu, 20 Apr 2017 03:49:29 +0000 Message-Id: <1492660169-6432-1-git-send-email-shaohe.feng@intel.com> X-Greylist: Sender passed SPF test, Sender IP whitelisted by DNSRBL, ACL 203 matched, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Thu, 20 Apr 2017 10:38:01 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Thu, 20 Apr 2017 10:38:01 +0000 (UTC) for IP:'192.55.52.43' DOMAIN:'mga05.intel.com' HELO:'mga05.intel.com' FROM:'shaohe.feng@intel.com' RCPT:'' X-RedHat-Spam-Score: -0.359 (BAYES_50, DATE_IN_PAST_06_12, DCC_REPUT_00_12, RCVD_IN_DNSWL_MED, RP_MATCHES_RCVD, SPF_PASS) 192.55.52.43 mga05.intel.com 192.55.52.43 mga05.intel.com X-Scanned-By: MIMEDefang 2.78 on 10.5.110.30 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-loop: libvir-list@redhat.com Cc: shaohe.feng@intel.com, dolpher.du@intel.com, robert.zyskowski@intel.com, Piotr.Uminski@intel.com, lukasz.daniluk@intel.com, rui.zang@intel.com, he.chen@intel.com, jdenemar@redhat.com Subject: [libvirt] [PATCH] Support CPU less NUMA node. X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.25]); Thu, 20 Apr 2017 10:39:20 +0000 (UTC) X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" Some platform NUMA node does not include cpu, such as Phi. This patch support CPU less NUMA node. But there must be one CPU cell node for a guest. Also must assign the host nodeset for this guest cell node. please enable numactl with "--with-numactl" for libvirt config. Test this patch: 1. define a numa cell without "cpus", such as cell 1. libvirt can edit and start the VM successfully. 2. define a numa cell without "cpus", without nodeset. This case, libvirt will failed to edit the CPUS. --- src/conf/domain_conf.c | 4 +++ src/conf/numa_conf.c | 89 ++++++++++++++++++++++++++++++++++-----------= ---- src/conf/numa_conf.h | 2 ++ src/qemu/qemu_command.c | 12 ++++--- src/util/virbitmap.c | 10 ++++-- 5 files changed, 83 insertions(+), 34 deletions(-) diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index 705deb3..d6e5489 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -17330,6 +17330,10 @@ virDomainDefParseXML(xmlDocPtr xml, ctxt) < 0) goto error; =20 + if (virDomainNumaCPULessCheck(def->numa) < 0){ + goto error; + } + if (virDomainNumatuneHasPlacementAuto(def->numa) && !def->cpumask && !virDomainDefHasVcpuPin(def) && !def->cputune.emulatorpin && diff --git a/src/conf/numa_conf.c b/src/conf/numa_conf.c index bfd3703..1b6f7ff 100644 --- a/src/conf/numa_conf.c +++ b/src/conf/numa_conf.c @@ -742,34 +742,31 @@ virDomainNumaDefCPUParseXML(virDomainNumaPtr def, goto cleanup; } =20 - if (!(tmp =3D virXMLPropString(nodes[i], "cpus"))) { - virReportError(VIR_ERR_XML_ERROR, "%s", - _("Missing 'cpus' attribute in NUMA cell")); - goto cleanup; - } - - if (virBitmapParse(tmp, &def->mem_nodes[cur_cell].cpumask, - VIR_DOMAIN_CPUMASK_LEN) < 0) - goto cleanup; - - if (virBitmapIsAllClear(def->mem_nodes[cur_cell].cpumask)) { - virReportError(VIR_ERR_CONFIG_UNSUPPORTED, - _("NUMA cell %d has no vCPUs assigned"), cur_cel= l); - goto cleanup; - } - VIR_FREE(tmp); - - for (j =3D 0; j < n; j++) { - if (j =3D=3D cur_cell || !def->mem_nodes[j].cpumask) - continue; + tmp =3D virXMLPropString(nodes[i], "cpus"); + if (tmp) { + if (virBitmapParse(tmp, &def->mem_nodes[cur_cell].cpumask, + VIR_DOMAIN_CPUMASK_LEN) < 0) + goto cleanup; =20 - if (virBitmapOverlaps(def->mem_nodes[j].cpumask, - def->mem_nodes[cur_cell].cpumask)) { + if (virBitmapIsAllClear(def->mem_nodes[cur_cell].cpumask)) { virReportError(VIR_ERR_CONFIG_UNSUPPORTED, - _("NUMA cells %u and %zu have overlapping v= CPU ids"), - cur_cell, j); + _("NUMA cell %d has no vCPUs assigned"), cur= _cell); goto cleanup; } + VIR_FREE(tmp); + + for (j =3D 0; j < n; j++) { + if (j =3D=3D cur_cell || !def->mem_nodes[j].cpumask) + continue; + + if (virBitmapOverlaps(def->mem_nodes[j].cpumask, + def->mem_nodes[cur_cell].cpumask)) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, + _("NUMA cells %u and %zu have overlappi= ng vCPU ids"), + cur_cell, j); + goto cleanup; + } + } } =20 ctxt->node =3D nodes[i]; @@ -808,6 +805,7 @@ virDomainNumaDefCPUFormat(virBufferPtr buf, char *cpustr; size_t ncells =3D virDomainNumaGetNodeCount(def); size_t i; + size_t ncpuless =3D 0; =20 if (ncells =3D=3D 0) return 0; @@ -817,12 +815,24 @@ virDomainNumaDefCPUFormat(virBufferPtr buf, for (i =3D 0; i < ncells; i++) { memAccess =3D virDomainNumaGetNodeMemoryAccessMode(def, i); =20 - if (!(cpustr =3D virBitmapFormat(virDomainNumaGetNodeCpumask(def, = i)))) - return -1; + cpustr =3D virBitmapFormat(virDomainNumaGetNodeCpumask(def, i)); =20 virBufferAddLit(buf, "\n"); =20 + if (ncpuless =3D=3D ncells) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, + _("At lease one NUMA node should " + "include CPU element.")); + return -1; + } return 0; } =20 @@ -861,6 +877,7 @@ virDomainNumaGetMaxCPUID(virDomainNumaPtr numa) int bit; =20 bit =3D virBitmapLastSetBit(virDomainNumaGetNodeCpumask(numa, i)); + bit =3D (bit > 0) ? bit : 0; if (bit > ret) ret =3D bit; } @@ -973,3 +990,21 @@ virDomainNumaGetMemorySize(virDomainNumaPtr numa) =20 return ret; } + +// NOTE, For some plateforms, there are some node without cpus, such as Ph= i. +// We just need do some check for these plateforms, we must assigned nodes= et +// for CPU less node. +int +virDomainNumaCPULessCheck(virDomainNumaPtr numa) +{ + int ret =3D 0; + int i =3D 0; + for (i =3D 0; i < numa->nmem_nodes; i++){ + if (!numa->mem_nodes[i].cpumask && !numa->mem_nodes[i].nodeset) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, + _("Cell: %d is a cpuless numa, must specfiy the= nodeset."), i); + return -1; + } + } + return ret; +} diff --git a/src/conf/numa_conf.h b/src/conf/numa_conf.h index b6a5354..1f711e6 100644 --- a/src/conf/numa_conf.h +++ b/src/conf/numa_conf.h @@ -155,5 +155,7 @@ int virDomainNumaDefCPUFormat(virBufferPtr buf, virDoma= inNumaPtr def); =20 unsigned int virDomainNumaGetCPUCountTotal(virDomainNumaPtr numa); =20 +int virDomainNumaCPULessCheck(virDomainNumaPtr numa); + =20 #endif /* __NUMA_CONF_H__ */ diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c index b2e76ca..b07ca66 100644 --- a/src/qemu/qemu_command.c +++ b/src/qemu/qemu_command.c @@ -7665,11 +7665,13 @@ qemuBuildNumaArgStr(virQEMUDriverConfigPtr cfg, virCommandAddArg(cmd, "-numa"); virBufferAsprintf(&buf, "node,nodeid=3D%zu", i); =20 - for (tmpmask =3D cpumask; tmpmask; tmpmask =3D next) { - if ((next =3D strchr(tmpmask, ','))) - *(next++) =3D '\0'; - virBufferAddLit(&buf, ",cpus=3D"); - virBufferAdd(&buf, tmpmask, -1); + if (*cpumask !=3D 0) { + for (tmpmask =3D cpumask; tmpmask; tmpmask =3D next) { + if ((next =3D strchr(tmpmask, ','))) + *(next++) =3D '\0'; + virBufferAddLit(&buf, ",cpus=3D"); + virBufferAdd(&buf, tmpmask, -1); + } } =20 if (needBackend) diff --git a/src/util/virbitmap.c b/src/util/virbitmap.c index eac63d9..b2803ce 100644 --- a/src/util/virbitmap.c +++ b/src/util/virbitmap.c @@ -953,6 +953,9 @@ virBitmapLastSetBit(virBitmapPtr bitmap) unsigned long bits; =20 /* If bitmap is empty then there is no set bit */ + if (!bitmap) + return -1; + if (bitmap->map_len =3D=3D 0) return -1; =20 @@ -1039,9 +1042,12 @@ virBitmapCountBits(virBitmapPtr bitmap) size_t i; size_t ret =3D 0; =20 - for (i =3D 0; i < bitmap->map_len; i++) - ret +=3D count_one_bits_l(bitmap->map[i]); + if (bitmap =3D=3D NULL) + return ret; =20 + for (i =3D 0; i < bitmap->map_len; i++) { + ret +=3D count_one_bits_l(bitmap->map[i]); + } return ret; } =20 --=20 2.7.4 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list