From nobody Fri Apr 26 00:51:59 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1498658311179912.1043277955505; Wed, 28 Jun 2017 06:58:31 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 871AC7C837; Wed, 28 Jun 2017 13:58:26 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 4BFA4955A6; Wed, 28 Jun 2017 13:58:26 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id EC5A71800C89; Wed, 28 Jun 2017 13:58:25 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id v5SDvbW3024490 for ; Wed, 28 Jun 2017 09:57:37 -0400 Received: by smtp.corp.redhat.com (Postfix) id DCA4D5C460; Wed, 28 Jun 2017 13:57:37 +0000 (UTC) Received: from mx1.redhat.com (ext-mx10.extmail.prod.ext.phx2.redhat.com [10.5.110.39]) by smtp.corp.redhat.com (Postfix) with ESMTPS id D602286601 for ; Wed, 28 Jun 2017 13:57:36 +0000 (UTC) Received: from userp1040.oracle.com (userp1040.oracle.com [156.151.31.81]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 0C87F5AFED for ; Wed, 28 Jun 2017 13:57:34 +0000 (UTC) Received: from aserv0021.oracle.com (aserv0021.oracle.com [141.146.126.233]) by userp1040.oracle.com (Sentrion-MTA-4.3.2/Sentrion-MTA-4.3.2) with ESMTP id v5SDvWdE010326 (version=TLSv1 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Wed, 28 Jun 2017 13:57:33 GMT Received: from userv0121.oracle.com (userv0121.oracle.com [156.151.31.72]) by aserv0021.oracle.com (8.13.8/8.14.4) with ESMTP id v5SDvWrd014247 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Wed, 28 Jun 2017 13:57:32 GMT Received: from abhmp0008.oracle.com (abhmp0008.oracle.com [141.146.116.14]) by userv0121.oracle.com (8.14.4/8.13.8) with ESMTP id v5SDvVCn002018; Wed, 28 Jun 2017 13:57:32 GMT Received: from nina.dynamic.ziggo.nl (/10.175.250.211) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 28 Jun 2017 06:57:31 -0700 DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 871AC7C837 Authentication-Results: ext-mx03.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=oracle.com Authentication-Results: ext-mx03.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=libvir-list-bounces@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 871AC7C837 DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 0C87F5AFED Authentication-Results: ext-mx10.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=oracle.com Authentication-Results: ext-mx10.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=wim.ten.have@oracle.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 0C87F5AFED From: Wim Ten Have To: Libvirt Development List Date: Wed, 28 Jun 2017 15:56:36 +0200 Message-Id: <20170628135639.18785-2-wim.ten.have@oracle.com> In-Reply-To: <20170628135639.18785-1-wim.ten.have@oracle.com> References: <20170628135639.18785-1-wim.ten.have@oracle.com> X-Source-IP: aserv0021.oracle.com [141.146.126.233] X-Greylist: Sender passed SPF test, Sender IP whitelisted by DNSRBL, ACL 203 matched, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.39]); Wed, 28 Jun 2017 13:57:34 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.39]); Wed, 28 Jun 2017 13:57:34 +0000 (UTC) for IP:'156.151.31.81' DOMAIN:'userp1040.oracle.com' HELO:'userp1040.oracle.com' FROM:'wim.ten.have@oracle.com' RCPT:'' X-RedHat-Spam-Score: -104.401 (BAYES_50, DCC_REPUT_13_19, RCVD_IN_DNSWL_MED, RCVD_IN_MSPIKE_H2, RP_MATCHES_RCVD, SPF_PASS, UNPARSEABLE_RELAY, USER_IN_WHITELIST) 156.151.31.81 userp1040.oracle.com 156.151.31.81 userp1040.oracle.com X-Scanned-By: MIMEDefang 2.78 on 10.5.110.39 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-loop: libvir-list@redhat.com Cc: Wim ten Have Subject: [libvirt] [RFC PATCH v2 1/4] numa: describe siblings distances within cells X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.27]); Wed, 28 Jun 2017 13:58:27 +0000 (UTC) X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" From: Wim ten Have Add libvirtd NUMA cell domain administration functionality to describe underlying cell id sibling distances in full fashion when configuring HVM guests. [below is an example of a 4 node setup] Changes under this commit concern all those that require adding the valid data structures, virDomainNuma* functional routines and the XML doc schema enhancements to enforce appropriate administration. These changes are accompanied with topic related documentation under docs/formatdomain.html within the "CPU model and topology" extending the "Guest NUMA topology" paragraph. For terminology we refer to sockets as "nodes" where access to each others' distinct resources such as memory make them "siblings" with a designated "distance" between them. A specific design is described under the ACPI (Advanced Configuration and Power Interface Specification) within the chapter explaining the system's SLIT (System Locality Distance Information Table). These patches extend core libvirt's XML description of a virtual machine's hardware to include NUMA distance information for sibling nodes, which is then passed to Xen guests via libxl. Recently qemu landed support for constructing the SLIT since commit 0f203430dd ("numa: Allow setting NUMA distance for different NUMA nodes"), hence these core libvirt extensions can also help other drivers in supporting this feature. These changes alter the docs/schemas/cputypes.rng enforcing domain administration to follow the syntax below per numa cell id. These changes also alter docs/schemas/basictypes.rng to add "numaDistanceValue" which is an "unsignedInt" with a minimum value of 10 as 0-9 are reserved values and can not be used as System Locality Distance Information Table data. Signed-off-by: Wim ten Have --- docs/formatdomain.html.in | 64 ++++++++++- docs/schemas/basictypes.rng | 8 ++ docs/schemas/cputypes.rng | 18 +++ src/conf/cpu_conf.c | 2 +- src/conf/numa_conf.c | 260 ++++++++++++++++++++++++++++++++++++++++= +++- src/conf/numa_conf.h | 25 ++++- src/libvirt_private.syms | 6 + 7 files changed, 376 insertions(+), 7 deletions(-) diff --git a/docs/formatdomain.html.in b/docs/formatdomain.html.in index 36bea67..b9736e3 100644 --- a/docs/formatdomain.html.in +++ b/docs/formatdomain.html.in @@ -1510,7 +1510,69 @@

=20

- This guest NUMA specification is currently available only for QEMU/K= VM. + This guest NUMA specification is currently available only for + QEMU/KVM and Xen. Whereas Xen driver also allows for a distinct + description of NUMA arranged sibling cell + distances Since 3.6.0. +

+ +

+ Under NUMA h/w architecture, distinct resources such as memory + create a designated distance between cell and + siblings that now can be described with the help of + distances. A detailed describtion can be found within + the ACPI (Advanced Configuration and Power Interface Specification) + within the chapter explaining the system's SLIT (System Locality + Distance Information Table). +

+ +
+...
+<cpu>
+  ...
+  <numa>
+    <cell id=3D'0' cpus=3D'0,4-7' memory=3D'512000' unit=3D'KiB'>
+      <distances>
+        <sibling id=3D'0' value=3D'10'/>
+        <sibling id=3D'1' value=3D'21'/>
+        <sibling id=3D'2' value=3D'31'/>
+        <sibling id=3D'3' value=3D'41'/>
+      </distances>
+    </cell>
+    <cell id=3D'1' cpus=3D'1,8-10,12-15' memory=3D'512000' unit=3D'KiB'=
 memAccess=3D'shared'>
+      <distances>
+        <sibling id=3D'0' value=3D'21'/>
+        <sibling id=3D'1' value=3D'10'/>
+        <sibling id=3D'2' value=3D'21'/>
+        <sibling id=3D'3' value=3D'31'/>
+      </distances>
+    </cell>
+    <cell id=3D'2' cpus=3D'2,11' memory=3D'512000' unit=3D'KiB' memAcce=
ss=3D'shared'>
+      <distances>
+        <sibling id=3D'0' value=3D'31'/>
+        <sibling id=3D'1' value=3D'21'/>
+        <sibling id=3D'2' value=3D'10'/>
+        <sibling id=3D'3' value=3D'21'/>
+      </distances>
+    </cell>
+    <cell id=3D'3' cpus=3D'3' memory=3D'512000' unit=3D'KiB'>
+      <distances>
+        <sibling id=3D'0' value=3D'41'/>
+        <sibling id=3D'1' value=3D'31'/>
+        <sibling id=3D'2' value=3D'21'/>
+        <sibling id=3D'3' value=3D'10'/>
+      </distances>
+    </cell>
+  </numa>
+  ...
+</cpu>
+...
+ +

+ Under Xen driver, if no distances are given to describe + the SLIT data between different cells, it will default to a scheme + using 10 for local and 21 for other cells. Which is the assumption + of guest OS when no SLIT is specified.

=20

Events configuration

diff --git a/docs/schemas/basictypes.rng b/docs/schemas/basictypes.rng index 1ea667c..a335b5d 100644 --- a/docs/schemas/basictypes.rng +++ b/docs/schemas/basictypes.rng @@ -77,6 +77,14 @@ =20 + + + + 10 + + + + diff --git a/docs/schemas/cputypes.rng b/docs/schemas/cputypes.rng index 3eef16a..c45b6df 100644 --- a/docs/schemas/cputypes.rng +++ b/docs/schemas/cputypes.rng @@ -129,6 +129,24 @@ + + + + + + + + + + + + + + + + + + =20 diff --git a/src/conf/cpu_conf.c b/src/conf/cpu_conf.c index da40e9b..5d8f7be3 100644 --- a/src/conf/cpu_conf.c +++ b/src/conf/cpu_conf.c @@ -643,7 +643,7 @@ virCPUDefFormatBufFull(virBufferPtr buf, if (virCPUDefFormatBuf(&childrenBuf, def, updateCPU) < 0) goto cleanup; =20 - if (virDomainNumaDefCPUFormat(&childrenBuf, numa) < 0) + if (virDomainNumaDefCPUFormatXML(&childrenBuf, numa) < 0) goto cleanup; =20 /* Put it all together */ diff --git a/src/conf/numa_conf.c b/src/conf/numa_conf.c index bfd3703..1914810 100644 --- a/src/conf/numa_conf.c +++ b/src/conf/numa_conf.c @@ -48,6 +48,8 @@ VIR_ENUM_IMPL(virDomainMemoryAccess, VIR_DOMAIN_MEMORY_AC= CESS_LAST, "shared", "private") =20 +typedef struct _virDomainNumaDistance virDomainNumaDistance; +typedef virDomainNumaDistance *virDomainNumaDistancePtr; =20 typedef struct _virDomainNumaNode virDomainNumaNode; typedef virDomainNumaNode *virDomainNumaNodePtr; @@ -66,6 +68,12 @@ struct _virDomainNuma { virBitmapPtr nodeset; /* host memory nodes where this guest node= resides */ virDomainNumatuneMemMode mode; /* memory mode selection */ virDomainMemoryAccess memAccess; /* shared memory access configura= tion */ + + struct _virDomainNumaDistance { + unsigned int value; /* locality value for node i*j */ + unsigned int cellid; + } *distances; /* remote node distances */ + size_t ndistances; } *mem_nodes; /* guest node configuration */ size_t nmem_nodes; =20 @@ -686,6 +694,95 @@ virDomainNumatuneNodesetIsAvailable(virDomainNumaPtr n= umatune, } =20 =20 +static int +virDomainNumaDefNodeDistanceParseXML(virDomainNumaPtr def, + xmlXPathContextPtr ctxt, + unsigned int cur_cell) +{ + int ret =3D -1; + char *tmp =3D NULL; + size_t i; + xmlNodePtr *nodes =3D NULL; + int ndistances; + virDomainNumaDistancePtr distances =3D NULL; + + + if (!virXPathNode("./distances[1]/sibling", ctxt)) + return 0; + + if ((ndistances =3D virXPathNodeSet("./distances[1]/sibling", ctxt, &n= odes)) <=3D 0) { + virReportError(VIR_ERR_XML_ERROR, "%s", + _("NUMA distances defined without siblings")); + goto cleanup; + } + + if (ndistances < def->nmem_nodes) { + virReportError(VIR_ERR_XML_ERROR, + _("NUMA distances defined with fewer siblings than = nodes for cell id: '%d'"), + cur_cell); + goto cleanup; + } + + if (VIR_ALLOC_N(distances, ndistances) < 0) + goto cleanup; + + for (i =3D 0; i < ndistances; i++) { + unsigned int sibling_id =3D i, sibling_value; + + /* siblings are in order of parsing or explicitly numbered */ + if ((tmp =3D virXMLPropString(nodes[i], "id"))) { + if (virStrToLong_uip(tmp, NULL, 10, &sibling_id) < 0) { + virReportError(VIR_ERR_XML_ERROR, + _("Invalid 'id' attribute in NUMA distances= for sibling: '%s'"), + tmp); + goto cleanup; + } + + if (sibling_id >=3D ndistances) { + virReportError(VIR_ERR_XML_ERROR, "%s", + _("Exactly one 'sibling' element per NUMA d= istance " + "is allowed, non-contiguous ranges or ran= ges not " + "starting from 0 are not allowed")); + goto cleanup; + } + } + VIR_FREE(tmp); + + /* We need a locality value */ + if (!(tmp =3D virXMLPropString(nodes[i], "value"))) { + virReportError(VIR_ERR_XML_ERROR, + _("Missing 'value' attribute in NUMA distances = for sibling id: '%d'"), + sibling_id); + goto cleanup; + } + + /* It needs to be applicable */ + if (virStrToLong_uip(tmp, NULL, 10, &sibling_value) < 0) { + virReportError(VIR_ERR_XML_ERROR, + _("Invalid 'value' attribute in NUMA distances = for sibling id: '%d'"), + sibling_id); + goto cleanup; + } + VIR_FREE(tmp); + + distances[sibling_id].cellid =3D sibling_id; + distances[sibling_id].value =3D sibling_value; + } + + def->mem_nodes[cur_cell].distances =3D distances; + def->mem_nodes[cur_cell].ndistances =3D ndistances; + + ret =3D 0; + + cleanup: + if (ret) + VIR_FREE(distances); + VIR_FREE(nodes); + VIR_FREE(tmp); + + return ret; +} + int virDomainNumaDefCPUParseXML(virDomainNumaPtr def, xmlXPathContextPtr ctxt) @@ -788,6 +885,14 @@ virDomainNumaDefCPUParseXML(virDomainNumaPtr def, def->mem_nodes[cur_cell].memAccess =3D rc; VIR_FREE(tmp); } + + /* Parse NUMA distances info */ + if (virDomainNumaDefNodeDistanceParseXML(def, ctxt, cur_cell) < 0)= { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, + _("NUMA cell %d has incorrect 'distances' config= ured"), + cur_cell); + goto cleanup; + } } =20 ret =3D 0; @@ -801,8 +906,8 @@ virDomainNumaDefCPUParseXML(virDomainNumaPtr def, =20 =20 int -virDomainNumaDefCPUFormat(virBufferPtr buf, - virDomainNumaPtr def) +virDomainNumaDefCPUFormatXML(virBufferPtr buf, + virDomainNumaPtr def) { virDomainMemoryAccess memAccess; char *cpustr; @@ -815,6 +920,8 @@ virDomainNumaDefCPUFormat(virBufferPtr buf, virBufferAddLit(buf, "\n"); virBufferAdjustIndent(buf, 2); for (i =3D 0; i < ncells; i++) { + int ndistances; + memAccess =3D virDomainNumaGetNodeMemoryAccessMode(def, i); =20 if (!(cpustr =3D virBitmapFormat(virDomainNumaGetNodeCpumask(def, = i)))) @@ -829,7 +936,30 @@ virDomainNumaDefCPUFormat(virBufferPtr buf, if (memAccess) virBufferAsprintf(buf, " memAccess=3D'%s'", virDomainMemoryAccessTypeToString(memAccess)= ); - virBufferAddLit(buf, "/>\n"); + + ndistances =3D def->mem_nodes[i].ndistances; + if (!ndistances) { + virBufferAddLit(buf, "/>\n"); + } else { + size_t j; + virDomainNumaDistancePtr distances =3D def->mem_nodes[i].dista= nces; + + virBufferAddLit(buf, ">\n"); + virBufferAdjustIndent(buf, 2); + virBufferAddLit(buf, "\n"); + virBufferAdjustIndent(buf, 2); + for (j =3D 0; j < ndistances; j++) { + virBufferAddLit(buf, "\n"); + } + virBufferAdjustIndent(buf, -2); + virBufferAddLit(buf, "\n"); + virBufferAdjustIndent(buf, -2); + virBufferAddLit(buf, "\n"); + } + VIR_FREE(cpustr); } virBufferAdjustIndent(buf, -2); @@ -922,13 +1052,121 @@ virDomainNumaCheckABIStability(virDomainNumaPtr src, size_t virDomainNumaGetNodeCount(virDomainNumaPtr numa) { - if (!numa) + if (!numa || !numa->mem_nodes) return 0; =20 return numa->nmem_nodes; } =20 =20 +size_t +virDomainNumaSetNodeCount(virDomainNumaPtr numa, size_t nmem_nodes) +{ + if (!numa || !nmem_nodes) + return 0; + + if (numa->mem_nodes) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("Cannot alter an existing nmem_nodes set")); + return 0; + } + + if (VIR_ALLOC_N(numa->mem_nodes, nmem_nodes) < 0) + return 0; + + numa->nmem_nodes =3D nmem_nodes; + + return numa->nmem_nodes; +} + + +size_t +virDomainNumaGetNodeDistance(virDomainNumaPtr numa, + size_t node, + size_t cellid) +{ + virDomainNumaDistancePtr distances; + + if (!numa) + return 0; + + distances =3D numa->mem_nodes[node].distances; + if (!numa->mem_nodes[node].ndistances || !distances) + return 0; + + return distances[cellid].value; +} + + +size_t +virDomainNumaSetNodeDistance(virDomainNumaPtr numa, + size_t node, + size_t cellid, + unsigned int value) +{ + virDomainNumaDistancePtr distances; + + /* + * Advanced Configuration and Power Interface + * Specification version 6.1. Chapter 5.2.17 + * System Locality Distance Information Table + * ... Distance values of 0-9 are reserved. + */ + if (!numa || value < 10) + return 0; + + distances =3D numa->mem_nodes[node].distances; + + if (numa->mem_nodes[node].ndistances > 0 && + distances[cellid].value) + return 0; + + distances[cellid].cellid =3D cellid; + distances[cellid].value =3D value; + + return distances[cellid].value; +} + + +size_t +virDomainNumaGetNodeDistanceCount(virDomainNumaPtr numa, + size_t node) +{ + if (!numa) + return 0; + + return numa->mem_nodes[node].ndistances; +} + + +size_t +virDomainNumaSetNodeDistanceCount(virDomainNumaPtr numa, + size_t node, + size_t ndistances) +{ + virDomainNumaDistancePtr distances; + + if (!numa || !ndistances) + return 0; + + distances =3D numa->mem_nodes[node].distances; + if (distances) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Cannot alter an existing nmem_nodes distances se= t for node: %zu"), + node); + return 0; + } + + if (VIR_ALLOC_N(distances, ndistances) < 0) + return 0; + + numa->mem_nodes[node].distances =3D distances; + numa->mem_nodes[node].ndistances =3D ndistances; + + return numa->mem_nodes[node].ndistances; +} + + virBitmapPtr virDomainNumaGetNodeCpumask(virDomainNumaPtr numa, size_t node) @@ -937,6 +1175,20 @@ virDomainNumaGetNodeCpumask(virDomainNumaPtr numa, } =20 =20 +virBitmapPtr +virDomainNumaSetNodeCpumask(virDomainNumaPtr numa, + size_t node, + virBitmapPtr cpumask) +{ + if (!numa || !cpumask) + return NULL; + + numa->mem_nodes[node].cpumask =3D cpumask; + + return numa->mem_nodes[node].cpumask; +} + + virDomainMemoryAccess virDomainNumaGetNodeMemoryAccessMode(virDomainNumaPtr numa, size_t node) diff --git a/src/conf/numa_conf.h b/src/conf/numa_conf.h index b6a5354..96dda90 100644 --- a/src/conf/numa_conf.h +++ b/src/conf/numa_conf.h @@ -87,12 +87,35 @@ int virDomainNumatuneMaybeGetNodeset(virDomainNumaPtr n= umatune, =20 size_t virDomainNumaGetNodeCount(virDomainNumaPtr numa); =20 +size_t virDomainNumaSetNodeCount(virDomainNumaPtr numa, + size_t nmem_nodes); + +size_t virDomainNumaGetNodeDistance(virDomainNumaPtr numa, + size_t node, + size_t sibling) + ATTRIBUTE_NONNULL(1); +size_t virDomainNumaSetNodeDistance(virDomainNumaPtr numa, + size_t node, + size_t sibling, + unsigned int value) + ATTRIBUTE_NONNULL(1); +size_t virDomainNumaGetNodeDistanceCount(virDomainNumaPtr numa, + size_t node) + ATTRIBUTE_NONNULL(1); +size_t virDomainNumaSetNodeDistanceCount(virDomainNumaPtr numa, + size_t node, + size_t ndistances) + ATTRIBUTE_NONNULL(1); virBitmapPtr virDomainNumaGetNodeCpumask(virDomainNumaPtr numa, size_t node) ATTRIBUTE_NONNULL(1); virDomainMemoryAccess virDomainNumaGetNodeMemoryAccessMode(virDomainNumaPt= r numa, size_t node) ATTRIBUTE_NONNULL(1); +virBitmapPtr virDomainNumaSetNodeCpumask(virDomainNumaPtr numa, + size_t node, + virBitmapPtr cpumask) + ATTRIBUTE_NONNULL(1); unsigned long long virDomainNumaGetNodeMemorySize(virDomainNumaPtr numa, size_t node) ATTRIBUTE_NONNULL(1); @@ -151,7 +174,7 @@ bool virDomainNumatuneNodeSpecified(virDomainNumaPtr nu= matune, int cellid); =20 int virDomainNumaDefCPUParseXML(virDomainNumaPtr def, xmlXPathContextPtr c= txt); -int virDomainNumaDefCPUFormat(virBufferPtr buf, virDomainNumaPtr def); +int virDomainNumaDefCPUFormatXML(virBufferPtr buf, virDomainNumaPtr def); =20 unsigned int virDomainNumaGetCPUCountTotal(virDomainNumaPtr numa); =20 diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index 888412a..b4887c1 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -705,9 +705,15 @@ virDomainNumaGetMaxCPUID; virDomainNumaGetMemorySize; virDomainNumaGetNodeCount; virDomainNumaGetNodeCpumask; +virDomainNumaGetNodeDistance; +virDomainNumaGetNodeDistanceCount; virDomainNumaGetNodeMemoryAccessMode; virDomainNumaGetNodeMemorySize; virDomainNumaNew; +virDomainNumaSetNodeCount; +virDomainNumaSetNodeCpumask; +virDomainNumaSetNodeDistance; +virDomainNumaSetNodeDistanceCount; virDomainNumaSetNodeMemorySize; virDomainNumatuneFormatNodeset; virDomainNumatuneFormatXML; --=20 2.9.4 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Fri Apr 26 00:51:59 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1498658303631463.6124140828608; Wed, 28 Jun 2017 06:58:23 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 673244025E; Wed, 28 Jun 2017 13:58:21 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 3134B87E53; Wed, 28 Jun 2017 13:58:21 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id C137D3FAE4; Wed, 28 Jun 2017 13:58:20 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id v5SDvbWV024483 for ; Wed, 28 Jun 2017 09:57:37 -0400 Received: by smtp.corp.redhat.com (Postfix) id AC9C987E53; Wed, 28 Jun 2017 13:57:37 +0000 (UTC) Received: from mx1.redhat.com (ext-mx07.extmail.prod.ext.phx2.redhat.com [10.5.110.31]) by smtp.corp.redhat.com (Postfix) with ESMTPS id A85C087E59 for ; Wed, 28 Jun 2017 13:57:37 +0000 (UTC) Received: from userp1040.oracle.com (userp1040.oracle.com [156.151.31.81]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 3CCE5C04D2A3 for ; Wed, 28 Jun 2017 13:57:36 +0000 (UTC) Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by userp1040.oracle.com (Sentrion-MTA-4.3.2/Sentrion-MTA-4.3.2) with ESMTP id v5SDvZsq010364 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 28 Jun 2017 13:57:35 GMT Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id v5SDvYHM025514 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Wed, 28 Jun 2017 13:57:35 GMT Received: from abhmp0008.oracle.com (abhmp0008.oracle.com [141.146.116.14]) by aserv0121.oracle.com (8.13.8/8.13.8) with ESMTP id v5SDvWDc027978; Wed, 28 Jun 2017 13:57:33 GMT Received: from nina.dynamic.ziggo.nl (/10.175.250.211) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 28 Jun 2017 06:57:32 -0700 DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 673244025E Authentication-Results: ext-mx06.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=oracle.com Authentication-Results: ext-mx06.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=libvir-list-bounces@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 673244025E DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 3CCE5C04D2A3 Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=oracle.com Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=wim.ten.have@oracle.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 3CCE5C04D2A3 From: Wim Ten Have To: Libvirt Development List Date: Wed, 28 Jun 2017 15:56:37 +0200 Message-Id: <20170628135639.18785-3-wim.ten.have@oracle.com> In-Reply-To: <20170628135639.18785-1-wim.ten.have@oracle.com> References: <20170628135639.18785-1-wim.ten.have@oracle.com> X-Source-IP: aserv0022.oracle.com [141.146.126.234] X-Greylist: Sender passed SPF test, Sender IP whitelisted by DNSRBL, ACL 203 matched, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Wed, 28 Jun 2017 13:57:36 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Wed, 28 Jun 2017 13:57:36 +0000 (UTC) for IP:'156.151.31.81' DOMAIN:'userp1040.oracle.com' HELO:'userp1040.oracle.com' FROM:'wim.ten.have@oracle.com' RCPT:'' X-RedHat-Spam-Score: -104.401 (BAYES_50, DCC_REPUT_13_19, RCVD_IN_DNSWL_MED, RCVD_IN_MSPIKE_H2, RP_MATCHES_RCVD, SPF_PASS, UNPARSEABLE_RELAY, USER_IN_WHITELIST) 156.151.31.81 userp1040.oracle.com 156.151.31.81 userp1040.oracle.com X-Scanned-By: MIMEDefang 2.78 on 10.5.110.31 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-loop: libvir-list@redhat.com Cc: Wim ten Have Subject: [libvirt] [RFC PATCH v2 2/4] libxl: vnuma support X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Wed, 28 Jun 2017 13:58:22 +0000 (UTC) X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" From: Wim ten Have This patch generates a NUMA distance-aware libxl description from the information extracted from a NUMA distance-aware libvirt XML file. By default, if no NUMA node distance information is supplied in the libvirt XML file, this patch uses the distances 10 for local and 21 for remote nodes/sockets." Signed-off-by: Wim ten Have --- src/libxl/libxl_conf.c | 124 +++++++++++++++++++++++++++++++++++++++++++++= ++++ 1 file changed, 124 insertions(+) diff --git a/src/libxl/libxl_conf.c b/src/libxl/libxl_conf.c index 938e09d..7d22229 100644 --- a/src/libxl/libxl_conf.c +++ b/src/libxl/libxl_conf.c @@ -617,6 +617,125 @@ libxlMakeDomBuildInfo(virDomainDefPtr def, return 0; } =20 +#ifdef LIBXL_HAVE_VNUMA +static int +libxlMakeVnumaList(virDomainDefPtr def, + libxl_ctx *ctx, + libxl_domain_config *d_config) +{ + int ret =3D -1; + size_t i, j; + size_t nr_nodes; + size_t num_vnuma; + virBitmapPtr bitmap =3D NULL; + virDomainNumaPtr numa =3D def->numa; + libxl_domain_build_info *b_info =3D &d_config->b_info; + libxl_physinfo physinfo; + libxl_vnode_info *vnuma_nodes =3D NULL; + + if (!numa) + return 0; + + num_vnuma =3D virDomainNumaGetNodeCount(numa); + if (!num_vnuma) + return 0; + + libxl_physinfo_init(&physinfo); + if (libxl_get_physinfo(ctx, &physinfo) < 0) { + libxl_physinfo_dispose(&physinfo); + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("libxl_get_physinfo_info failed")); + return -1; + } + nr_nodes =3D physinfo.nr_nodes; + libxl_physinfo_dispose(&physinfo); + + if (num_vnuma > nr_nodes) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, + _("Number of configured numa cells %zu exceeds the = physical available nodes %zu"), + num_vnuma, nr_nodes); + return -1; + } + + /* + * allocate the vnuma_nodes for assignment under b_info. + */ + if (VIR_ALLOC_N(vnuma_nodes, num_vnuma) < 0) + return -1; + + /* + * parse the vnuma vnodes data. + */ + for (i =3D 0; i < num_vnuma; i++) { + int cpu; + libxl_bitmap vcpu_bitmap; + libxl_vnode_info *p =3D &vnuma_nodes[i]; + + libxl_vnode_info_init(p); + + /* pnode */ + p->pnode =3D i; + + /* memory size */ + p->memkb =3D virDomainNumaGetNodeMemorySize(numa, i); + + /* vcpus */ + bitmap =3D virDomainNumaGetNodeCpumask(numa, i); + if (bitmap =3D=3D NULL) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("vnuma sibling %zu missing vcpus set"), i); + goto cleanup; + } + + if ((cpu =3D virBitmapNextSetBit(bitmap, -1)) < 0) + goto cleanup; + + libxl_bitmap_init(&vcpu_bitmap); + if (libxl_cpu_bitmap_alloc(ctx, &vcpu_bitmap, b_info->max_vcpus)) { + virReportOOMError(); + goto cleanup; + } + + do { + libxl_bitmap_set(&vcpu_bitmap, cpu); + } while ((cpu =3D virBitmapNextSetBit(bitmap, cpu)) >=3D 0); + + libxl_bitmap_copy_alloc(ctx, &p->vcpus, &vcpu_bitmap); + libxl_bitmap_dispose(&vcpu_bitmap); + + /* vdistances */ + if (VIR_ALLOC_N(p->distances, num_vnuma) < 0) + goto cleanup; + p->num_distances =3D num_vnuma; + + /* + * Apply the configured distance value. If not + * available set 10 for local or 21 for remote nodes. + */ + for (j =3D 0; j < num_vnuma; j++) + if (!(p->distances[j] =3D virDomainNumaGetNodeDistance(numa, i= , j))) + p->distances[j] =3D (i =3D=3D j) ? 10 : 21; + } + + b_info->vnuma_nodes =3D vnuma_nodes; + b_info->num_vnuma_nodes =3D num_vnuma; + + ret =3D 0; + + cleanup: + if (ret) { + for (i =3D 0; i < num_vnuma; i++) { + libxl_vnode_info *p =3D &vnuma_nodes[i]; + + VIR_FREE(p->distances); + } + VIR_FREE(vnuma_nodes); + } + + return ret; +} +#endif + static int libxlDiskSetDiscard(libxl_device_disk *x_disk, int discard) { @@ -2207,6 +2326,11 @@ libxlBuildDomainConfig(virPortAllocatorPtr graphicsp= orts, if (libxlMakeDomBuildInfo(def, ctx, caps, d_config) < 0) return -1; =20 +#ifdef LIBXL_HAVE_VNUMA + if (libxlMakeVnumaList(def, ctx, d_config) < 0) + return -1; +#endif + if (libxlMakeDiskList(def, d_config) < 0) return -1; =20 --=20 2.9.4 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Fri Apr 26 00:51:59 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1498658269201798.2000888896179; Wed, 28 Jun 2017 06:57:49 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 2D45A18E687; Wed, 28 Jun 2017 13:57:45 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 6139394C55; Wed, 28 Jun 2017 13:57:44 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id D4E133FADF; Wed, 28 Jun 2017 13:57:42 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id v5SDveib024507 for ; Wed, 28 Jun 2017 09:57:40 -0400 Received: by smtp.corp.redhat.com (Postfix) id 96A808929F; Wed, 28 Jun 2017 13:57:40 +0000 (UTC) Received: from mx1.redhat.com (ext-mx05.extmail.prod.ext.phx2.redhat.com [10.5.110.29]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 8706789281 for ; Wed, 28 Jun 2017 13:57:38 +0000 (UTC) Received: from aserp1040.oracle.com (aserp1040.oracle.com [141.146.126.69]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 1D79B3345B4 for ; Wed, 28 Jun 2017 13:57:36 +0000 (UTC) Received: from userv0021.oracle.com (userv0021.oracle.com [156.151.31.71]) by aserp1040.oracle.com (Sentrion-MTA-4.3.2/Sentrion-MTA-4.3.2) with ESMTP id v5SDvY5W025355 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 28 Jun 2017 13:57:35 GMT Received: from aserv0122.oracle.com (aserv0122.oracle.com [141.146.126.236]) by userv0021.oracle.com (8.14.4/8.14.4) with ESMTP id v5SDvYde019673 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 28 Jun 2017 13:57:34 GMT Received: from abhmp0008.oracle.com (abhmp0008.oracle.com [141.146.116.14]) by aserv0122.oracle.com (8.14.4/8.14.4) with ESMTP id v5SDvXMe005539; Wed, 28 Jun 2017 13:57:33 GMT Received: from nina.dynamic.ziggo.nl (/10.175.250.211) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 28 Jun 2017 06:57:33 -0700 DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 2D45A18E687 Authentication-Results: ext-mx05.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=oracle.com Authentication-Results: ext-mx05.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=libvir-list-bounces@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 2D45A18E687 DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 1D79B3345B4 Authentication-Results: ext-mx05.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=oracle.com Authentication-Results: ext-mx05.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=wim.ten.have@oracle.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 1D79B3345B4 From: Wim Ten Have To: Libvirt Development List Date: Wed, 28 Jun 2017 15:56:38 +0200 Message-Id: <20170628135639.18785-4-wim.ten.have@oracle.com> In-Reply-To: <20170628135639.18785-1-wim.ten.have@oracle.com> References: <20170628135639.18785-1-wim.ten.have@oracle.com> X-Source-IP: userv0021.oracle.com [156.151.31.71] X-Greylist: Sender passed SPF test, Sender IP whitelisted by DNSRBL, ACL 203 matched, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.29]); Wed, 28 Jun 2017 13:57:36 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.29]); Wed, 28 Jun 2017 13:57:36 +0000 (UTC) for IP:'141.146.126.69' DOMAIN:'aserp1040.oracle.com' HELO:'aserp1040.oracle.com' FROM:'wim.ten.have@oracle.com' RCPT:'' X-RedHat-Spam-Score: -101.621 (BAYES_50, DCC_REPUT_13_19, RCVD_IN_DNSWL_MED, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, RP_MATCHES_RCVD, SPF_PASS, UNPARSEABLE_RELAY, USER_IN_WHITELIST) 141.146.126.69 aserp1040.oracle.com 141.146.126.69 aserp1040.oracle.com X-Scanned-By: MIMEDefang 2.78 on 10.5.110.29 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-loop: libvir-list@redhat.com Cc: Wim ten Have Subject: [libvirt] [RFC PATCH v2 3/4] xenconfig: add domxml conversions for xen-xl X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.29]); Wed, 28 Jun 2017 13:57:46 +0000 (UTC) X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" From: Wim ten Have This patch converts NUMA configurations between the Xen libxl configuration file format and libvirt's XML format. XML HVM domain configuration: Xen xl.cfg domain configuration: vnuma =3D [["pnode=3D0","size=3D2048","vcpus=3D0-1","vdistances=3D10,21,3= 1,41"], ["pnode=3D1","size=3D2048","vcpus=3D2-3","vdistances=3D21,10,21,= 31"], ["pnode=3D2","size=3D2048","vcpus=3D4-5","vdistances=3D31,21,10,= 21"], ["pnode=3D3","size=3D2048","vcpus=3D6-7","vdistances=3D41,31,21,= 10"]] If there is no XML description amongst the data the conversion schema from xml to native will generate 10 for local and 21 for all remote instances. Signed-off-by: Wim ten Have --- src/xenconfig/xen_xl.c | 303 +++++++++++++++++++++++++++++++++++++++++++++= ++++ 1 file changed, 303 insertions(+) diff --git a/src/xenconfig/xen_xl.c b/src/xenconfig/xen_xl.c index cac440c..d9f7028 100644 --- a/src/xenconfig/xen_xl.c +++ b/src/xenconfig/xen_xl.c @@ -309,6 +309,168 @@ xenParseXLSpice(virConfPtr conf, virDomainDefPtr def) return -1; } =20 +#ifdef LIBXL_HAVE_VNUMA +static int +xenParseXLVnuma(virConfPtr conf, virDomainDefPtr def) +{ + int ret =3D -1; + char *tmp =3D NULL; + char **token =3D NULL; + size_t vcpus =3D 0; + + virConfValuePtr list; + virDomainNumaPtr numa; + + numa =3D def->numa; + if (numa =3D=3D NULL) + return -1; + + list =3D virConfGetValue(conf, "vnuma"); + if (list && list->type =3D=3D VIR_CONF_LIST) { + size_t nr_nodes =3D 0, vnodeCnt =3D 0; + virConfValuePtr vnode =3D list->list; + virCPUDefPtr cpu; + + vnode =3D list->list; + while (vnode) { + vnode =3D vnode->next; + nr_nodes++; + } + + if (!virDomainNumaSetNodeCount(numa, nr_nodes)) + goto cleanup; + + list =3D list->list; + + if (VIR_ALLOC(cpu) < 0) + goto cleanup; + + while (list) { + + /* Is there a sublist (vnode)? */ + if (list && list->type =3D=3D VIR_CONF_LIST) { + vnode =3D list->list; + + while (vnode && vnode->type =3D=3D VIR_CONF_STRING) { + const char *data; + const char *str =3D vnode->str; + + if (!str || + !(data =3D strrchr(str, '=3D'))) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("vnuma vnode invalid format '%s'"= ), + str); + goto skipvnode; + } + data++; + + if (*data) { + size_t len; + char vtoken[64]; + + if (STRPREFIX(str, "pnode")) { + unsigned int cellid; + + len =3D strlen(data); + if (!virStrncpy(vtoken, data, + len, sizeof(vtoken))) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("vnuma vnode %zu pnode '%= s' too long for destination"), + vnodeCnt, data); + goto cleanup; + } + + if ((virStrToLong_ui(vtoken, NULL, 10, &cellid= ) < 0) || + (cellid >=3D nr_nodes)) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("vnuma vnode %zu invalid = value pnode '%s'"), + vnodeCnt, data); + goto cleanup; + } + } else if (STRPREFIX(str, "size")) { + unsigned long long kbsize; + + len =3D strlen(data); + if (!virStrncpy(vtoken, data, + len, sizeof(vtoken))) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("vnuma vnode %zu size '%s= ' too long for destination"), + vnodeCnt, data); + goto cleanup; + } + + if (virStrToLong_ull(vtoken, NULL, 10, &kbsize= ) < 0) + goto cleanup; + + virDomainNumaSetNodeMemorySize(numa, vnodeCnt,= (kbsize * 1024)); + } else if (STRPREFIX(str, "vcpus")) { + virBitmapPtr cpumask =3D NULL; + + len =3D strlen(data); + if (!virStrncpy(vtoken, data, + len, sizeof(vtoken))) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("vnuma vnode %zu vcpus '%= s' too long for destination"), + vnodeCnt, data); + goto cleanup; + } + + if ((virBitmapParse(vtoken, &cpumask, VIR_DOMA= IN_CPUMASK_LEN) < 0) || + (virDomainNumaSetNodeCpumask(numa, vnodeCn= t, cpumask) =3D=3D NULL)) + goto cleanup; + + vcpus +=3D virBitmapCountBits(cpumask); + } else if (STRPREFIX(str, "vdistances")) { + size_t i, ndistances; + unsigned int value; + + len =3D strlen(data); + if (!virStrncpy(vtoken, data, + len, sizeof(vtoken))) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("vnuma vnode %zu vdistanc= es '%s' too long for destination"), + vnodeCnt, data); + } + + if (VIR_STRDUP(tmp, vtoken) < 0) + goto cleanup; + + if (!(token =3D virStringSplitCount(tmp, ",", = 0, &ndistances)) || + (virDomainNumaSetNodeDistanceCount(numa, v= nodeCnt, ndistances) !=3D ndistances)) + goto cleanup; + + for (i =3D 0; i < ndistances; i++) { + if ((virStrToLong_ui(token[i], NULL, 10, &= value) < 0) || + (virDomainNumaSetNodeDistance(numa, vn= odeCnt, i, value) !=3D value)) + goto cleanup; + } + } else { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("vnuma vnode %zu invalid toke= n '%s'"), + vnodeCnt, str); + goto cleanup; + } + } + skipvnode: + vnode =3D vnode->next; + } + } + list =3D list->next; + vnodeCnt++; + } + cpu->type =3D VIR_CPU_TYPE_GUEST; + def->cpu =3D cpu; + } + + ret =3D 0; + + cleanup: + virStringListFree(token); + VIR_FREE(tmp); + + return ret; +} +#endif =20 static int xenParseXLDiskSrc(virDomainDiskDefPtr disk, char *srcstr) @@ -863,6 +1025,11 @@ xenParseXL(virConfPtr conf, if (xenParseXLOS(conf, def, caps) < 0) goto cleanup; =20 +#ifdef LIBXL_HAVE_VNUMA + if (xenParseXLVnuma(conf, def) < 0) + goto cleanup; +#endif + if (xenParseXLDisk(conf, def) < 0) goto cleanup; =20 @@ -1005,6 +1172,137 @@ xenFormatXLOS(virConfPtr conf, virDomainDefPtr def) return 0; } =20 +#ifdef LIBXL_HAVE_VNUMA +static int +xenFormatXLVnode(virConfValuePtr list, virBufferPtr buf) +{ + int ret =3D -1; + virConfValuePtr numaPnode, tmp; + + if (virBufferCheckError(buf) < 0) + goto cleanup; + + if (VIR_ALLOC(numaPnode) < 0) + goto cleanup; + + /* Place VNODE directive */ + numaPnode->type =3D VIR_CONF_STRING; + numaPnode->str =3D virBufferContentAndReset(buf); + + tmp =3D list->list; + while (tmp && tmp->next) + tmp =3D tmp->next; + if (tmp) + tmp->next =3D numaPnode; + else + list->list =3D numaPnode; + ret =3D 0; + + cleanup: + virBufferFreeAndReset(buf); + return ret; +} + +static int +xenFormatXLVnuma(virConfValuePtr list, + virDomainNumaPtr numa, size_t node, size_t nr_nodes) +{ + int ret =3D -1; + size_t i; + + virBuffer buf =3D VIR_BUFFER_INITIALIZER; + virConfValuePtr numaVnode, tmp; + + size_t nodeSize =3D virDomainNumaGetNodeMemorySize(numa, node) / 1024; + char *nodeVcpus =3D virBitmapFormat(virDomainNumaGetNodeCpumask(numa, = node)); + + if (VIR_ALLOC(numaVnode) < 0) + goto cleanup; + + numaVnode->type =3D VIR_CONF_LIST; + numaVnode->list =3D NULL; + + /* pnode */ + virBufferAsprintf(&buf, "pnode=3D%ld", node); + xenFormatXLVnode(numaVnode, &buf); + + /* size */ + virBufferAsprintf(&buf, "size=3D%ld", nodeSize); + xenFormatXLVnode(numaVnode, &buf); + + /* vcpus */ + virBufferAsprintf(&buf, "vcpus=3D%s", nodeVcpus); + xenFormatXLVnode(numaVnode, &buf); + + /* distances */ + virBufferAddLit(&buf, "vdistances=3D"); + for (i =3D 0; i < nr_nodes; i++) { + size_t distance =3D virDomainNumaGetNodeDistance(numa, node, i); + + /* + * Present the configured distance value. If not + * available set 10 for local or 21 for remote nodes. + */ + if (!distance) + distance =3D (node =3D=3D i) ? 10 : 21; + virBufferAsprintf(&buf, "%zu", distance); + if ((nr_nodes - i) > 1) + virBufferAddLit(&buf, ","); + } + xenFormatXLVnode(numaVnode, &buf); + + tmp =3D list->list; + while (tmp && tmp->next) + tmp =3D tmp->next; + if (tmp) + tmp->next =3D numaVnode; + else + list->list =3D numaVnode; + ret =3D 0; + + cleanup: + VIR_FREE(nodeVcpus); + return ret; +} + +static int +xenFormatXLDomainVnuma(virConfPtr conf, virDomainDefPtr def) +{ + virDomainNumaPtr numa =3D def->numa; + virConfValuePtr vnumaVal; + size_t i; + size_t nr_nodes; + + if (numa =3D=3D NULL) + return -1; + + if (VIR_ALLOC(vnumaVal) < 0) + return -1; + + vnumaVal->type =3D VIR_CONF_LIST; + vnumaVal->list =3D NULL; + + nr_nodes =3D virDomainNumaGetNodeCount(numa); + for (i =3D 0; i < nr_nodes; i++) { + if (xenFormatXLVnuma(vnumaVal, numa, i, nr_nodes) < 0) + goto cleanup; + } + + if (vnumaVal->list !=3D NULL) { + int ret =3D virConfSetValue(conf, "vnuma", vnumaVal); + vnumaVal =3D NULL; + if (ret < 0) + return -1; + } + VIR_FREE(vnumaVal); + + return 0; + + cleanup: + virConfFreeValue(vnumaVal); + return -1; +} +#endif =20 static char * xenFormatXLDiskSrcNet(virStorageSourcePtr src) @@ -1641,6 +1939,11 @@ xenFormatXL(virDomainDefPtr def, virConnectPtr conn) if (xenFormatXLOS(conf, def) < 0) goto cleanup; =20 +#ifdef LIBXL_HAVE_VNUMA + if (xenFormatXLDomainVnuma(conf, def) < 0) + goto cleanup; +#endif + if (xenFormatXLDomainDisks(conf, def) < 0) goto cleanup; =20 --=20 2.9.4 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Fri Apr 26 00:51:59 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1498658286212197.12056083459663; Wed, 28 Jun 2017 06:58:06 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 0BC7E8B134; Wed, 28 Jun 2017 13:58:03 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id BAAF689B7D; Wed, 28 Jun 2017 13:57:56 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 909363FAE0; Wed, 28 Jun 2017 13:57:55 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id v5SDvgDJ024517 for ; Wed, 28 Jun 2017 09:57:42 -0400 Received: by smtp.corp.redhat.com (Postfix) id 81F8C955A8; Wed, 28 Jun 2017 13:57:42 +0000 (UTC) Received: from mx1.redhat.com (ext-mx08.extmail.prod.ext.phx2.redhat.com [10.5.110.32]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 7A893955A4 for ; Wed, 28 Jun 2017 13:57:40 +0000 (UTC) Received: from aserp1040.oracle.com (aserp1040.oracle.com [141.146.126.69]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id A3CDDC05689C for ; Wed, 28 Jun 2017 13:57:38 +0000 (UTC) Received: from userv0021.oracle.com (userv0021.oracle.com [156.151.31.71]) by aserp1040.oracle.com (Sentrion-MTA-4.3.2/Sentrion-MTA-4.3.2) with ESMTP id v5SDvb5Q025386 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 28 Jun 2017 13:57:37 GMT Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by userv0021.oracle.com (8.14.4/8.14.4) with ESMTP id v5SDvaKx019806 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Wed, 28 Jun 2017 13:57:37 GMT Received: from abhmp0008.oracle.com (abhmp0008.oracle.com [141.146.116.14]) by aserv0121.oracle.com (8.13.8/8.13.8) with ESMTP id v5SDvYhD027991; Wed, 28 Jun 2017 13:57:35 GMT Received: from nina.dynamic.ziggo.nl (/10.175.250.211) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 28 Jun 2017 06:57:34 -0700 DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 0BC7E8B134 Authentication-Results: ext-mx04.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=oracle.com Authentication-Results: ext-mx04.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=libvir-list-bounces@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 0BC7E8B134 DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com A3CDDC05689C Authentication-Results: ext-mx08.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=oracle.com Authentication-Results: ext-mx08.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=wim.ten.have@oracle.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com A3CDDC05689C From: Wim Ten Have To: Libvirt Development List Date: Wed, 28 Jun 2017 15:56:39 +0200 Message-Id: <20170628135639.18785-5-wim.ten.have@oracle.com> In-Reply-To: <20170628135639.18785-1-wim.ten.have@oracle.com> References: <20170628135639.18785-1-wim.ten.have@oracle.com> X-Source-IP: userv0021.oracle.com [156.151.31.71] X-Greylist: Sender passed SPF test, Sender IP whitelisted by DNSRBL, ACL 203 matched, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Wed, 28 Jun 2017 13:57:39 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Wed, 28 Jun 2017 13:57:39 +0000 (UTC) for IP:'141.146.126.69' DOMAIN:'aserp1040.oracle.com' HELO:'aserp1040.oracle.com' FROM:'wim.ten.have@oracle.com' RCPT:'' X-RedHat-Spam-Score: -101.621 (BAYES_50, DCC_REPUT_13_19, RCVD_IN_DNSWL_MED, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, RP_MATCHES_RCVD, SPF_PASS, UNPARSEABLE_RELAY, USER_IN_WHITELIST) 141.146.126.69 aserp1040.oracle.com 141.146.126.69 aserp1040.oracle.com X-Scanned-By: MIMEDefang 2.78 on 10.5.110.32 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-loop: libvir-list@redhat.com Cc: Wim ten Have Subject: [libvirt] [RFC PATCH v2 4/4] xlconfigtest: add tests for numa cell sibling distances X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Wed, 28 Jun 2017 13:58:04 +0000 (UTC) X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" From: Wim ten Have Test a bidirectional xen-xl domxml to and from native for numa support administration as brought under this patch series. Signed-off-by: Wim ten Have --- .../test-fullvirt-vnuma-nodistances.cfg | 26 +++++++ .../test-fullvirt-vnuma-nodistances.xml | 53 ++++++++++++++ tests/xlconfigdata/test-fullvirt-vnuma.cfg | 26 +++++++ tests/xlconfigdata/test-fullvirt-vnuma.xml | 81 ++++++++++++++++++= ++++ tests/xlconfigtest.c | 4 ++ 5 files changed, 190 insertions(+) create mode 100644 tests/xlconfigdata/test-fullvirt-vnuma-nodistances.cfg create mode 100644 tests/xlconfigdata/test-fullvirt-vnuma-nodistances.xml create mode 100644 tests/xlconfigdata/test-fullvirt-vnuma.cfg create mode 100644 tests/xlconfigdata/test-fullvirt-vnuma.xml diff --git a/tests/xlconfigdata/test-fullvirt-vnuma-nodistances.cfg b/tests= /xlconfigdata/test-fullvirt-vnuma-nodistances.cfg new file mode 100644 index 0000000..9871f21 --- /dev/null +++ b/tests/xlconfigdata/test-fullvirt-vnuma-nodistances.cfg @@ -0,0 +1,26 @@ +name =3D "XenGuest2" +uuid =3D "c7a5fdb2-cdaf-9455-926a-d65c16db1809" +maxmem =3D 8192 +memory =3D 8192 +vcpus =3D 8 +pae =3D 1 +acpi =3D 1 +apic =3D 1 +viridian =3D 0 +rtc_timeoffset =3D 0 +localtime =3D 0 +on_poweroff =3D "destroy" +on_reboot =3D "restart" +on_crash =3D "restart" +device_model =3D "/usr/lib/xen/bin/qemu-system-i386" +sdl =3D 0 +vnc =3D 1 +vncunused =3D 1 +vnclisten =3D "127.0.0.1" +vif =3D [ "mac=3D00:16:3e:66:92:9c,bridge=3Dxenbr1,script=3Dvif-bridge,mod= el=3De1000" ] +parallel =3D "none" +serial =3D "none" +builder =3D "hvm" +boot =3D "d" +vnuma =3D [ [ "pnode=3D0", "size=3D2048", "vcpus=3D0-1", "vdistances=3D10,= 21,21,21" ], [ "pnode=3D1", "size=3D2048", "vcpus=3D2-3", "vdistances=3D21,= 10,21,21" ], [ "pnode=3D2", "size=3D2048", "vcpus=3D4-5", "vdistances=3D21,= 21,10,21" ], [ "pnode=3D3", "size=3D2048", "vcpus=3D6-7", "vdistances=3D21,= 21,21,10" ] ] +disk =3D [ "format=3Draw,vdev=3Dhda,access=3Drw,backendtype=3Dphy,target= =3D/dev/HostVG/XenGuest2" ] diff --git a/tests/xlconfigdata/test-fullvirt-vnuma-nodistances.xml b/tests= /xlconfigdata/test-fullvirt-vnuma-nodistances.xml new file mode 100644 index 0000000..9cab3ca --- /dev/null +++ b/tests/xlconfigdata/test-fullvirt-vnuma-nodistances.xml @@ -0,0 +1,53 @@ + + XenGuest2 + c7a5fdb2-cdaf-9455-926a-d65c16db1809 + 8388608 + 8388608 + 8 + + hvm + /usr/lib/xen/boot/hvmloader + + + + + + + + + + + + + + + + + destroy + restart + restart + + /usr/lib/xen/bin/qemu-system-i386 + + + + +
+ + + + + +