From nobody Mon Apr 29 06:28:06 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 152414760001464.7735608206425; Thu, 19 Apr 2018 07:20:00 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 590F3C0587E3; Thu, 19 Apr 2018 14:19:58 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 226297E21E; Thu, 19 Apr 2018 14:19:48 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 688394CAA9; Thu, 19 Apr 2018 14:19:47 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id w3JEJjBE007003 for ; Thu, 19 Apr 2018 10:19:45 -0400 Received: by smtp.corp.redhat.com (Postfix) id 16B4B2166BC6; Thu, 19 Apr 2018 14:19:45 +0000 (UTC) Received: from inaba.usersys.redhat.com (unknown [10.43.2.44]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 917682166BAE for ; Thu, 19 Apr 2018 14:19:44 +0000 (UTC) From: Andrea Bolognani To: libvir-list@redhat.com Date: Thu, 19 Apr 2018 16:19:38 +0200 Message-Id: <20180419141939.24253-2-abologna@redhat.com> In-Reply-To: <20180419141939.24253-1-abologna@redhat.com> References: <20180419141939.24253-1-abologna@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH v2 1/2] tests: Create full host NUMA topology in more cases X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Thu, 19 Apr 2018 14:19:59 +0000 (UTC) X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" vircapstest has code to add a full host NUMA topology, that is, one that includes all information about nodes and CPUs including IDs; testQemuCapsInit(), which is used to create a mock virCapsPtr for QEMU tests, however, just fakes it by setting nnumaCell_max to some number. While the latter approach has served us well so far, we're going to need all the information to be filled in soon. In order to do that, we can just move the existing code from vircapstest to testutils and, with some renaming and trivial tweaking, use it as-is. Interestingly, the NUMA topology generated by the function is rigged up so that the NUMA nodes aren't (necessarily) numbered starting from 0, which is a nice way to spot mistaken assumptions in our codebase. Signed-off-by: Andrea Bolognani Reviewed-by: J=EF=BF=BDn Tomko --- tests/testutils.c | 51 +++++++++++++++++++++++++++++++++++++++ tests/testutils.h | 2 ++ tests/testutilsqemu.c | 7 +++++- tests/vircapstest.c | 66 ++++-------------------------------------------= ---- 4 files changed, 64 insertions(+), 62 deletions(-) diff --git a/tests/testutils.c b/tests/testutils.c index 7c095caac9..4b13d11278 100644 --- a/tests/testutils.c +++ b/tests/testutils.c @@ -1228,6 +1228,57 @@ virCapsPtr virTestGenericCapsInit(void) return NULL; } =20 + +#define MAX_CELLS 4 +#define MAX_CPUS_IN_CELL 2 +#define MAX_MEM_IN_CELL 2097152 + +/* + * Build NUMA topology with cell id starting from (0 + seq) + * for testing + */ +int +virTestCapsBuildNUMATopology(virCapsPtr caps, + int seq) +{ + virCapsHostNUMACellCPUPtr cell_cpus =3D NULL; + int core_id, cell_id; + int id; + + id =3D 0; + for (cell_id =3D 0; cell_id < MAX_CELLS; cell_id++) { + if (VIR_ALLOC_N(cell_cpus, MAX_CPUS_IN_CELL) < 0) + goto error; + + for (core_id =3D 0; core_id < MAX_CPUS_IN_CELL; core_id++) { + cell_cpus[core_id].id =3D id + core_id; + cell_cpus[core_id].socket_id =3D cell_id + seq; + cell_cpus[core_id].core_id =3D id + core_id; + if (!(cell_cpus[core_id].siblings =3D + virBitmapNew(MAX_CPUS_IN_CELL))) + goto error; + ignore_value(virBitmapSetBit(cell_cpus[core_id].siblings, id)); + } + id++; + + if (virCapabilitiesAddHostNUMACell(caps, cell_id + seq, + MAX_MEM_IN_CELL, + MAX_CPUS_IN_CELL, cell_cpus, + VIR_ARCH_NONE, NULL, + VIR_ARCH_NONE, NULL) < 0) + goto error; + + cell_cpus =3D NULL; + } + + return 0; + + error: + virCapabilitiesClearHostNUMACellCPUTopology(cell_cpus, MAX_CPUS_IN_CEL= L); + VIR_FREE(cell_cpus); + return -1; +} + static virDomainDefParserConfig virTestGenericDomainDefParserConfig =3D { .features =3D VIR_DOMAIN_DEF_FEATURE_INDIVIDUAL_VCPUS, }; diff --git a/tests/testutils.h b/tests/testutils.h index d840875bc1..3bd7bf1603 100644 --- a/tests/testutils.h +++ b/tests/testutils.h @@ -147,6 +147,8 @@ int virTestMain(int argc, } =20 virCapsPtr virTestGenericCapsInit(void); +int virTestCapsBuildNUMATopology(virCapsPtr caps, + int seq); virDomainXMLOptionPtr virTestGenericDomainXMLConfInit(void); =20 typedef enum { diff --git a/tests/testutilsqemu.c b/tests/testutilsqemu.c index 9671a46f12..01fa1b701f 100644 --- a/tests/testutilsqemu.c +++ b/tests/testutilsqemu.c @@ -400,7 +400,12 @@ virCapsPtr testQemuCapsInit(void) =20 qemuTestSetHostCPU(caps, NULL); =20 - caps->host.nnumaCell_max =3D 4; + /* + * Build a NUMA topology with cell_id (NUMA node id + * being 3(0 + 3),4(1 + 3), 5 and 6 + */ + if (virTestCapsBuildNUMATopology(caps, 3) < 0) + goto cleanup; =20 if (testQemuAddI686Guest(caps) < 0) goto cleanup; diff --git a/tests/vircapstest.c b/tests/vircapstest.c index 664b7da143..1df3fa091f 100644 --- a/tests/vircapstest.c +++ b/tests/vircapstest.c @@ -29,62 +29,6 @@ =20 #define VIR_FROM_THIS VIR_FROM_NONE =20 -#define MAX_CELLS 4 -#define MAX_CPUS_IN_CELL 2 -#define MAX_MEM_IN_CELL 2097152 - - -/* - * Build NUMA Toplogy with cell id starting from (0 + seq) - * for testing - */ -static virCapsPtr -buildNUMATopology(int seq) -{ - virCapsPtr caps; - virCapsHostNUMACellCPUPtr cell_cpus =3D NULL; - int core_id, cell_id; - int id; - - if ((caps =3D virCapabilitiesNew(VIR_ARCH_X86_64, false, false)) =3D= =3D NULL) - goto error; - - id =3D 0; - for (cell_id =3D 0; cell_id < MAX_CELLS; cell_id++) { - if (VIR_ALLOC_N(cell_cpus, MAX_CPUS_IN_CELL) < 0) - goto error; - - for (core_id =3D 0; core_id < MAX_CPUS_IN_CELL; core_id++) { - cell_cpus[core_id].id =3D id + core_id; - cell_cpus[core_id].socket_id =3D cell_id + seq; - cell_cpus[core_id].core_id =3D id + core_id; - if (!(cell_cpus[core_id].siblings =3D - virBitmapNew(MAX_CPUS_IN_CELL))) - goto error; - ignore_value(virBitmapSetBit(cell_cpus[core_id].siblings, id)); - } - id++; - - if (virCapabilitiesAddHostNUMACell(caps, cell_id + seq, - MAX_MEM_IN_CELL, - MAX_CPUS_IN_CELL, cell_cpus, - VIR_ARCH_NONE, NULL, - VIR_ARCH_NONE, NULL) < 0) - goto error; - - cell_cpus =3D NULL; - } - - return caps; - - error: - virCapabilitiesClearHostNUMACellCPUTopology(cell_cpus, MAX_CPUS_IN_CEL= L); - VIR_FREE(cell_cpus); - virObjectUnref(caps); - return NULL; - -} - =20 static int test_virCapabilitiesGetCpusForNodemask(const void *data ATTRIBUTE_UNUSED) @@ -96,11 +40,11 @@ test_virCapabilitiesGetCpusForNodemask(const void *data= ATTRIBUTE_UNUSED) int mask_size =3D 8; int ret =3D -1; =20 - /* - * Build a NUMA topology with cell_id (NUMA node id - * being 3(0 + 3),4(1 + 3), 5 and 6 - */ - if (!(caps =3D buildNUMATopology(3))) + + if (!(caps =3D virCapabilitiesNew(VIR_ARCH_X86_64, false, false))) + goto error; + + if (virTestCapsBuildNUMATopology(caps, 3) < 0) goto error; =20 if (virBitmapParse(nodestr, &nodemask, mask_size) < 0) --=20 2.14.3 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Mon Apr 29 06:28:06 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1524147606882529.1814207777888; Thu, 19 Apr 2018 07:20:06 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 264EE3132597; Thu, 19 Apr 2018 14:20:05 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id AA86E89FC4; Thu, 19 Apr 2018 14:20:04 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 289BA4CAA6; Thu, 19 Apr 2018 14:19:53 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id w3JEJjZQ007010 for ; Thu, 19 Apr 2018 10:19:45 -0400 Received: by smtp.corp.redhat.com (Postfix) id AEA6E2166BC6; Thu, 19 Apr 2018 14:19:45 +0000 (UTC) Received: from inaba.usersys.redhat.com (unknown [10.43.2.44]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 52D7F2166BAE for ; Thu, 19 Apr 2018 14:19:45 +0000 (UTC) From: Andrea Bolognani To: libvir-list@redhat.com Date: Thu, 19 Apr 2018 16:19:39 +0200 Message-Id: <20180419141939.24253-3-abologna@redhat.com> In-Reply-To: <20180419141939.24253-1-abologna@redhat.com> References: <20180419141939.24253-1-abologna@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH v2 2/2] qemu: Figure out nodeset bitmap size correctly X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.48]); Thu, 19 Apr 2018 14:20:05 +0000 (UTC) X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" The current private XML parsing code relies on the assumption that NUMA node IDs start from 0 and are densely allocated, neither of which is necessarily the case. Change it so that the bitmap size is dynamically calculated by looking at NUMA node IDs instead, which ensures all nodes will be able to fit and thus the bitmap will be parsed successfully. Update one of the test cases so that it would fail with the previous approach, but passes with the new one. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=3D1490158 Signed-off-by: Andrea Bolognani Reviewed-by: J=EF=BF=BDn Tomko --- src/qemu/qemu_domain.c | 10 +++++++++- tests/qemustatusxml2xmldata/modern-in.xml | 2 +- 2 files changed, 10 insertions(+), 2 deletions(-) diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c index 21897cb47a..2670a10119 100644 --- a/src/qemu/qemu_domain.c +++ b/src/qemu/qemu_domain.c @@ -2233,6 +2233,8 @@ qemuDomainObjPrivateXMLParseAutomaticPlacement(xmlXPa= thContextPtr ctxt, virCapsPtr caps =3D NULL; char *nodeset; char *cpuset; + int nodesetSize =3D 0; + size_t i; int ret =3D -1; =20 nodeset =3D virXPathString("string(./numad/@nodeset)", ctxt); @@ -2244,8 +2246,14 @@ qemuDomainObjPrivateXMLParseAutomaticPlacement(xmlXP= athContextPtr ctxt, if (!(caps =3D virQEMUDriverGetCapabilities(driver, false))) goto cleanup; =20 + /* Figure out how big the nodeset bitmap needs to be. + * This is necessary because NUMA node IDs are not guaranteed to + * start from 0 or be densely allocated */ + for (i =3D 0; i < caps->host.nnumaCell; i++) + nodesetSize =3D MAX(nodesetSize, caps->host.numaCell[i]->num + 1); + if (nodeset && - virBitmapParse(nodeset, &priv->autoNodeset, caps->host.nnumaCell_m= ax) < 0) + virBitmapParse(nodeset, &priv->autoNodeset, nodesetSize) < 0) goto cleanup; =20 if (cpuset) { diff --git a/tests/qemustatusxml2xmldata/modern-in.xml b/tests/qemustatusxm= l2xmldata/modern-in.xml index 2e166e6e67..c1e57618b6 100644 --- a/tests/qemustatusxml2xmldata/modern-in.xml +++ b/tests/qemustatusxml2xmldata/modern-in.xml @@ -252,7 +252,7 @@ - + --=20 2.14.3 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list