From nobody Thu May 2 03:57:39 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail(p=none dis=none) header.from=intel.com Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1527592972588659.0225331846211; Tue, 29 May 2018 04:22:52 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 93823A7557; Tue, 29 May 2018 11:22:50 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 4285C17A7A; Tue, 29 May 2018 11:22:50 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id D80564BB78; Tue, 29 May 2018 11:22:49 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id w4TBMZvE032638 for ; Tue, 29 May 2018 07:22:35 -0400 Received: by smtp.corp.redhat.com (Postfix) id 50E1D6091E; Tue, 29 May 2018 11:22:35 +0000 (UTC) Received: from mx1.redhat.com (ext-mx07.extmail.prod.ext.phx2.redhat.com [10.5.110.31]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 46B25608F6 for ; Tue, 29 May 2018 11:22:29 +0000 (UTC) Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 616FAC0F8FEF for ; Tue, 29 May 2018 11:22:28 +0000 (UTC) Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga106.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 29 May 2018 04:22:28 -0700 Received: from bing-i9.bj.intel.com ([172.16.182.85]) by orsmga006.jf.intel.com with ESMTP; 29 May 2018 04:22:26 -0700 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.49,456,1520924400"; d="scan'208";a="45609816" From: bing.niu@intel.com To: libvir-list@redhat.com Date: Tue, 29 May 2018 18:58:02 +0800 Message-Id: <1527591483-20151-2-git-send-email-bing.niu@intel.com> In-Reply-To: <1527591483-20151-1-git-send-email-bing.niu@intel.com> References: <1527591483-20151-1-git-send-email-bing.niu@intel.com> X-Greylist: Sender passed SPF test, Sender IP whitelisted by DNSRBL, ACL 207 matched, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Tue, 29 May 2018 11:22:28 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Tue, 29 May 2018 11:22:28 +0000 (UTC) for IP:'192.55.52.136' DOMAIN:'mga12.intel.com' HELO:'mga12.intel.com' FROM:'bing.niu@intel.com' RCPT:'' X-RedHat-Spam-Score: -2.301 (RCVD_IN_DNSWL_MED, SPF_PASS) 192.55.52.136 mga12.intel.com 192.55.52.136 mga12.intel.com X-Scanned-By: MIMEDefang 2.78 on 10.5.110.31 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-loop: libvir-list@redhat.com Cc: shaohe.feng@intel.com, huaqiang.wang@intel.com, Bing Niu , jian-feng.ding@intel.com, rui.zang@yandex.com Subject: [libvirt] [RFC PATCH 1/2] util: Add memory bandwidth support to resctrl X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.26]); Tue, 29 May 2018 11:22:51 +0000 (UTC) X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" From: Bing Niu Add memory bandwidth allocation support basing on existing virresctrl implementation. Two new structures virResctrlInfoMB and virResctrlAllocMB are introduced. virResctrlInfoMB is used to record host system MBA supporting information, e.g., minimum bandwidth allowed, bandwidth granularity, number of bandwidth controller (same as number of last level cache). virResctrlAllocMB is used for allocating memory bandwidth. Following virResctrlAllocPerType, it also employs a nested sparse array to indicate whether allocation is available for particular llc. Overall, the memory bandwidth allocation follows the same sequence as existing virresctrl cache allocation model. It exposes interfaces for probing host's memory bandwidth capability on initialization time and memory bandwidth allocation on runtime. Signed-off-by: Bing Niu --- src/util/virresctrl.c | 415 ++++++++++++++++++++++++++++++++++++++++++++++= +--- src/util/virresctrl.h | 5 + 2 files changed, 401 insertions(+), 19 deletions(-) diff --git a/src/util/virresctrl.c b/src/util/virresctrl.c index fc11635..ad050c7 100644 --- a/src/util/virresctrl.c +++ b/src/util/virresctrl.c @@ -86,6 +86,20 @@ struct _virResctrlInfoPerType { virResctrlInfoPerCache control; }; =20 +/* Information about memory bandwidth allocation */ +typedef struct _virResctrlInfoMB virResctrlInfoMB; +typedef virResctrlInfoMB *virResctrlInfoMBPtr; +struct _virResctrlInfoMB { + /* minimum memory bandwidth allowed*/ + unsigned int min_bandwidth; + /* bandwidth granularity */ + unsigned int bandwidth_gran; + /* level number of llc*/ + unsigned int llc; + /* number of last level cache */ + unsigned int max_id; +}; + typedef struct _virResctrlInfoPerLevel virResctrlInfoPerLevel; typedef virResctrlInfoPerLevel *virResctrlInfoPerLevelPtr; struct _virResctrlInfoPerLevel { @@ -97,6 +111,8 @@ struct _virResctrlInfo { =20 virResctrlInfoPerLevelPtr *levels; size_t nlevels; + + virResctrlInfoMBPtr mb_info; }; =20 static virClassPtr virResctrlInfoClass; @@ -126,6 +142,7 @@ virResctrlInfoDispose(void *obj) VIR_FREE(level); } =20 + VIR_FREE(resctrl->mb_info); VIR_FREE(resctrl->levels); } =20 @@ -201,6 +218,18 @@ struct _virResctrlAllocPerType { size_t nmasks; }; =20 +/* + * virResctrlAllocMB represents one memory bandwidth allocation. Since it = can have + * multiple last level caches in a NUMA system, it is also represented as = a nested + * sparse arrays as virRestrlAllocPerLevel + */ +typedef struct _virResctrlAllocMB virResctrlAllocMB; +typedef virResctrlAllocMB *virResctrlAllocMBPtr; +struct _virResctrlAllocMB { + unsigned int **bandwidth; + size_t nsizes; +}; + typedef struct _virResctrlAllocPerLevel virResctrlAllocPerLevel; typedef virResctrlAllocPerLevel *virResctrlAllocPerLevelPtr; struct _virResctrlAllocPerLevel { @@ -214,7 +243,7 @@ struct _virResctrlAlloc { =20 virResctrlAllocPerLevelPtr *levels; size_t nlevels; - + virResctrlAllocMBPtr mba; /* The identifier (any unique string for now) */ char *id; /* libvirt-generated path in /sys/fs/resctrl for this particular @@ -259,6 +288,13 @@ virResctrlAllocDispose(void *obj) VIR_FREE(level); } =20 + if (resctrl->mba) { + virResctrlAllocMBPtr mba =3D resctrl->mba; + for (i =3D 0; i < mba->nsizes; i++) + VIR_FREE(mba->bandwidth[i]); + } + + VIR_FREE(resctrl->mba); VIR_FREE(resctrl->id); VIR_FREE(resctrl->path); VIR_FREE(resctrl->levels); @@ -364,6 +400,9 @@ virResctrlInfoIsEmpty(virResctrlInfoPtr resctrl) if (!resctrl) return true; =20 + if (resctrl->mb_info) + return false; + for (i =3D 0; i < resctrl->nlevels; i++) { virResctrlInfoPerLevelPtr i_level =3D resctrl->levels[i]; =20 @@ -379,11 +418,62 @@ virResctrlInfoIsEmpty(virResctrlInfoPtr resctrl) return true; } =20 +static int +virResctrlGetMemoryBandwidthInfo(virResctrlInfoPtr resctrl) +{ + DIR *dirp =3D NULL; + int ret =3D -1; + int rv =3D -1; + virResctrlInfoMBPtr i_mb =3D NULL; =20 -#ifdef __linux__ + rv =3D virDirOpenIfExists(&dirp, SYSFS_RESCTRL_PATH "/info"); + if (rv <=3D 0) { + ret =3D rv; + goto cleanup; + } + /* query memory bandwidth allocation info */ + if (VIR_ALLOC(i_mb) < 0) { + ret =3D -1; + goto cleanup; + } + rv =3D virFileReadValueUint(&i_mb->bandwidth_gran, + SYSFS_RESCTRL_PATH "/info/MB/bandwidth_gran"= ); + if (rv =3D=3D -2) { + /* The file doesn't exist, so it's unusable for us, + * properly mba unsupported */ + VIR_WARN("The path '" SYSFS_RESCTRL_PATH "/info/MB/bandwidth_gran'" + "does not exist"); + ret =3D 0; + goto cleanup; + } else if (rv < 0) { + /* File no existing or other failures are fatal, so just quit */ + goto cleanup; + } =20 -int -virResctrlGetInfo(virResctrlInfoPtr resctrl) + rv =3D virFileReadValueUint(&i_mb->min_bandwidth, + SYSFS_RESCTRL_PATH "/info/MB/min_bandwidth"); + if (rv =3D=3D -2) { + /* The file doesn't exist, so it's unusable for us, + * properly mba unsupported */ + VIR_WARN("The path '" SYSFS_RESCTRL_PATH "/info/MB/min_bandwidth'" + "does not exist"); + ret =3D 0; + goto cleanup; + } else if (rv < 0) { + /* File no existing or other failures are fatal, so just quit */ + goto cleanup; + } + + VIR_STEAL_PTR(resctrl->mb_info, i_mb); + ret =3D 0; + cleanup: + VIR_DIR_CLOSE(dirp); + VIR_FREE(i_mb); + return ret; +} + +static int +virResctrlGetCacheInfo(virResctrlInfoPtr resctrl) { DIR *dirp =3D NULL; char *endptr =3D NULL; @@ -512,6 +602,23 @@ virResctrlGetInfo(virResctrlInfoPtr resctrl) return ret; } =20 +#ifdef __linux__ +int +virResctrlGetInfo(virResctrlInfoPtr resctrl) +{ + int ret; + + ret =3D virResctrlGetMemoryBandwidthInfo(resctrl); + if (ret < 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("Get Memory Bandwidth fail")); + goto out; + } + ret =3D virResctrlGetCacheInfo(resctrl); + out: + return ret; +} + #else /* ! __linux__ */ =20 int @@ -534,12 +641,26 @@ virResctrlInfoGetCache(virResctrlInfoPtr resctrl, { virResctrlInfoPerLevelPtr i_level =3D NULL; virResctrlInfoPerTypePtr i_type =3D NULL; + virResctrlInfoMBPtr mb_info =3D NULL; size_t i =3D 0; int ret =3D -1; =20 if (virResctrlInfoIsEmpty(resctrl)) return 0; =20 + /* Let's take the opportunity to update the number of + * last level cache. This number is used to calculate + * free memory bandwidth */ + if (resctrl->mb_info) { + mb_info =3D resctrl->mb_info; + if (level > mb_info->llc) { + mb_info->llc =3D level; + mb_info->max_id =3D 1; + } else if (mb_info->llc =3D=3D level) { + mb_info->max_id++; + } + } + if (level >=3D resctrl->nlevels) return 0; =20 @@ -600,6 +721,9 @@ virResctrlAllocIsEmpty(virResctrlAllocPtr resctrl) if (!resctrl) return true; =20 + if (resctrl->mba) + return false; + for (i =3D 0; i < resctrl->nlevels; i++) { virResctrlAllocPerLevelPtr a_level =3D resctrl->levels[i]; =20 @@ -855,16 +979,218 @@ virResctrlAllocGetID(virResctrlAllocPtr alloc) } =20 =20 -char * -virResctrlAllocFormat(virResctrlAllocPtr resctrl) +int +virResctrlSetMemoryBandwidth(virResctrlAllocPtr resctrl, + unsigned int id, + unsigned int memory_bandwidth) +{ + virResctrlAllocMBPtr mba =3D resctrl->mba; + + if (!mba) { + if (VIR_ALLOC(mba) < 0) + return -1; + resctrl->mba =3D mba; + } + + if (mba->nsizes <=3D id && + VIR_EXPAND_N(mba->bandwidth, mba->nsizes, + id - mba->nsizes + 1) < 0) + return -1; + + if (mba->bandwidth[id]) { + virReportError(VIR_ERR_XML_ERROR, + _("Collision Memory Bandwidth on node %d"), + id); + return -1; + } + + if (VIR_ALLOC(mba->bandwidth[id]) < 0) + return -1; + + *(mba->bandwidth[id]) =3D memory_bandwidth; + return 0; +} + + +static int +virResctrlAllocMemoryBandwidthFormat(virResctrlAllocPtr alloc, virBufferPt= r buf) +{ + int id; + + if (!alloc->mba) + goto out; + + virBufferAddLit(buf, "MB:"); + + for (id =3D 0; id < alloc->mba->nsizes; id++) { + if (alloc->mba->bandwidth[id]) { + virBufferAsprintf(buf, "%u=3D%u;", id, + *(alloc->mba->bandwidth[id])); + } + } + + virBufferTrim(buf, ";", 1); + virBufferAddChar(buf, '\n'); + virBufferCheckError(buf); + out: + return 0; +} + +static int +virResctrlAllocMemoryBandwidth(virResctrlInfoPtr resctrl, + virResctrlAllocPtr alloc, virResctrlAllocPt= r free) +{ + int id; + int ret; + virResctrlAllocMBPtr mb_alloc =3D alloc->mba; + virResctrlAllocMBPtr mb_free =3D free->mba; + virResctrlInfoMBPtr mb_info =3D resctrl->mb_info; + + if (!mb_alloc) + return 0; + + if (mb_alloc && !mb_info) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", + _("RDT Memory Bandwidth allocationi" + " unsupported")); + ret =3D -1; + goto out; + } + + for (id =3D 0; id < mb_alloc->nsizes; id ++) { + if (!mb_alloc->bandwidth[id]) + continue; + + if (*(mb_alloc->bandwidth[id]) % mb_info->bandwidth_gran) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, + _("Memory Bandwidth allocation of size " + "%u is not divisible by granularity %u"), + *(mb_alloc->bandwidth[id]), + mb_info->bandwidth_gran); + ret =3D -1; + goto out; + } + if (*(mb_alloc->bandwidth[id]) < mb_info->min_bandwidth) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, + _("Memory Bandwidth allocation of size " + "%u is smaller than the minimum " + "allowed allocation %u"), + *(mb_alloc->bandwidth[id]), + mb_info->min_bandwidth); + ret =3D -1; + goto out; + } + if (id >=3D mb_info->max_id) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, + _("bandwidth controller %u not exist," + " max controller id %u"), + id, mb_info->max_id - 1); + return -1; + } + if (*(mb_alloc->bandwidth[id]) > *(mb_free->bandwidth[id])) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, + _("Not enough room for allocation of %u " + "bandwidth for node %u%%, freed %u%%"), + id, *(mb_alloc->bandwidth[id]), + *(mb_free->bandwidth[id])); + return -1; + } + } + ret =3D 0; + out: + return ret; +} + + +static int +virResctrlAllocParseMemoryBandwidthLine(virResctrlInfoPtr resctrl, + virResctrlAllocPtr alloc, + char *line) +{ + char **mbs =3D NULL; + char *tmp =3D NULL; + unsigned int bandwidth; + size_t nmb =3D 0; + unsigned int id; + size_t i; + int ret; + + /* For no reason there can be spaces */ + virSkipSpaces((const char **) &line); + + if (STRNEQLEN(line, "MB", 2)) + return 0; + + if (!resctrl || !resctrl->mb_info || + (!resctrl->mb_info->min_bandwidth) + || !(resctrl->mb_info->bandwidth_gran)) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("Missing or inconsistent resctrl info for " + "memory bandwidth allocation")); + } + + if (!alloc->mba) + if (VIR_ALLOC(alloc->mba) < 0) + return -1; + + tmp =3D strchr(line, ':'); + if (!tmp) + return 0; + tmp++; + + mbs =3D virStringSplitCount(tmp, ";", 0, &nmb); + if (!nmb) + return 0; + + for (i =3D 0; i < nmb; i++) { + tmp =3D strchr(mbs[i], '=3D'); + *tmp =3D '\0'; + tmp++; + + if (virStrToLong_uip(mbs[i], NULL, 10, &id) < 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Invalid node id %u "), id); + ret =3D -1; + goto clean; + } + if (virStrToLong_uip(tmp, NULL, 10, &bandwidth) < 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Invalid bandwidth %u"), bandwidth); + ret =3D -1; + goto clean; + } + if (alloc->mba->nsizes <=3D id && + VIR_EXPAND_N(alloc->mba->bandwidth, alloc->mba->nsizes, + id - alloc->mba->nsizes + 1) < 0) { + ret =3D -1; + goto clean; + } + if (!alloc->mba->bandwidth[id]) { + if (VIR_ALLOC(alloc->mba->bandwidth[id]) < 0) { + ret =3D -1; + goto clean; + } + } + + *(alloc->mba->bandwidth[id]) =3D bandwidth; + } + ret =3D 0; + clean: + virStringListFree(mbs); + return ret; +} + + +static int +virResctrlAllocCacheFormat(virResctrlAllocPtr resctrl, virBufferPtr buf) { - virBuffer buf =3D VIR_BUFFER_INITIALIZER; unsigned int level =3D 0; unsigned int type =3D 0; unsigned int cache =3D 0; + int ret =3D 0; =20 if (!resctrl) - return NULL; + return 0; =20 for (level =3D 0; level < resctrl->nlevels; level++) { virResctrlAllocPerLevelPtr a_level =3D resctrl->levels[level]; @@ -878,7 +1204,7 @@ virResctrlAllocFormat(virResctrlAllocPtr resctrl) if (!a_type) continue; =20 - virBufferAsprintf(&buf, "L%u%s:", level, virResctrlTypeToStrin= g(type)); + virBufferAsprintf(buf, "L%u%s:", level, virResctrlTypeToString= (type)); =20 for (cache =3D 0; cache < a_type->nmasks; cache++) { virBitmapPtr mask =3D a_type->masks[cache]; @@ -889,18 +1215,49 @@ virResctrlAllocFormat(virResctrlAllocPtr resctrl) =20 mask_str =3D virBitmapToString(mask, false, true); if (!mask_str) { - virBufferFreeAndReset(&buf); - return NULL; + ret =3D -1; + goto out; } =20 - virBufferAsprintf(&buf, "%u=3D%s;", cache, mask_str); + virBufferAsprintf(buf, "%u=3D%s;", cache, mask_str); VIR_FREE(mask_str); } =20 - virBufferTrim(&buf, ";", 1); - virBufferAddChar(&buf, '\n'); + virBufferTrim(buf, ";", 1); + virBufferAddChar(buf, '\n'); } } + out: + return ret; +} + + +static void +virResctrlMemoryBandwidthSubstract(virResctrlAllocPtr free, + virResctrlAllocPtr used) +{ + size_t i; + + if (used->mba) { + for (i =3D 0; i < used->mba->nsizes; i++) { + if (used->mba->bandwidth[i]) + *(free->mba->bandwidth[i]) -=3D *(used->mba->bandwidth[i]); + } + } +} + + +char * +virResctrlAllocFormat(virResctrlAllocPtr resctrl) +{ + virBuffer buf =3D VIR_BUFFER_INITIALIZER; + + if (!resctrl) + return NULL; + + virResctrlAllocCacheFormat(resctrl, &buf); + + virResctrlAllocMemoryBandwidthFormat(resctrl, &buf); =20 virBufferCheckError(&buf); return virBufferContentAndReset(&buf); @@ -957,9 +1314,8 @@ virResctrlAllocParseProcessCache(virResctrlInfoPtr res= ctrl, return ret; } =20 - static int -virResctrlAllocParseProcessLine(virResctrlInfoPtr resctrl, +virResctrlAllocParseCacheLine(virResctrlInfoPtr resctrl, virResctrlAllocPtr alloc, char *line) { @@ -1029,7 +1385,9 @@ virResctrlAllocParse(virResctrlInfoPtr resctrl, =20 lines =3D virStringSplitCount(schemata, "\n", 0, &nlines); for (i =3D 0; i < nlines; i++) { - if (virResctrlAllocParseProcessLine(resctrl, alloc, lines[i]) < 0) + if (virResctrlAllocParseMemoryBandwidthLine(resctrl, alloc, lines[= i]) < 0) + goto cleanup; + if (virResctrlAllocParseCacheLine(resctrl, alloc, lines[i]) < 0) goto cleanup; } =20 @@ -1165,6 +1523,23 @@ virResctrlAllocNewFromInfo(virResctrlInfoPtr info) } } =20 + if (info->mb_info) { + if (VIR_ALLOC(ret->mba) < 0) + goto error; + + if (info->mb_info->max_id && + VIR_EXPAND_N(ret->mba->bandwidth, ret->mba->nsizes, + info->mb_info->max_id - ret->mba->nsizes) < 0) + goto error; + + for (i =3D 0; i < ret->mba->nsizes; i++) { + if (VIR_ALLOC(ret->mba->bandwidth[i]) < 0) + goto error; + + *(ret->mba->bandwidth[i]) =3D 100; + } + } + cleanup: virBitmapFree(mask); return ret; @@ -1228,6 +1603,7 @@ virResctrlAllocGetUnused(virResctrlInfoPtr resctrl) goto error; } =20 + virResctrlMemoryBandwidthSubstract(ret, alloc); virResctrlAllocSubtract(ret, alloc); virObjectUnref(alloc); alloc =3D NULL; @@ -1389,7 +1765,6 @@ virResctrlAllocFindUnused(virResctrlAllocPtr alloc, return ret; } =20 - static int virResctrlAllocCopyMasks(virResctrlAllocPtr dst, virResctrlAllocPtr src) @@ -1451,6 +1826,9 @@ virResctrlAllocMasksAssign(virResctrlInfoPtr resctrl, if (!alloc_default) goto cleanup; =20 + if (virResctrlAllocMemoryBandwidth(resctrl, alloc, alloc_free) < 0) + goto cleanup; + if (virResctrlAllocCopyMasks(alloc, alloc_default) < 0) goto cleanup; =20 @@ -1524,7 +1902,6 @@ virResctrlAllocDeterminePath(virResctrlAllocPtr alloc, return 0; } =20 - /* This checks if the directory for the alloc exists. If not it tries to = create * it and apply appropriate alloc settings. */ int diff --git a/src/util/virresctrl.h b/src/util/virresctrl.h index 5368ba2..82c7ab1 100644 --- a/src/util/virresctrl.h +++ b/src/util/virresctrl.h @@ -89,6 +89,11 @@ virResctrlAllocSetSize(virResctrlAllocPtr resctrl, unsigned long long size); =20 int +virResctrlSetMemoryBandwidth(virResctrlAllocPtr resctrl, + unsigned int id, + unsigned int mb); + +int virResctrlAllocForeachSize(virResctrlAllocPtr resctrl, virResctrlAllocForeachSizeCallback cb, void *opaque); --=20 2.7.4 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Thu May 2 03:57:39 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail(p=none dis=none) header.from=intel.com Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1527592956955338.92419381263574; Tue, 29 May 2018 04:22:36 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.24]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id F02E330C3E50; Tue, 29 May 2018 11:22:34 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 5F5F3308BDA8; Tue, 29 May 2018 11:22:34 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id D83FD1800C9B; Tue, 29 May 2018 11:22:32 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id w4TBMV0d032618 for ; Tue, 29 May 2018 07:22:31 -0400 Received: by smtp.corp.redhat.com (Postfix) id 13C6A1693B; Tue, 29 May 2018 11:22:31 +0000 (UTC) Received: from mx1.redhat.com (ext-mx07.extmail.prod.ext.phx2.redhat.com [10.5.110.31]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 0B77D65F5A for ; Tue, 29 May 2018 11:22:30 +0000 (UTC) Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id F145BC0F8FC6 for ; Tue, 29 May 2018 11:22:29 +0000 (UTC) Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga106.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 29 May 2018 04:22:29 -0700 Received: from bing-i9.bj.intel.com ([172.16.182.85]) by orsmga006.jf.intel.com with ESMTP; 29 May 2018 04:22:28 -0700 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.49,456,1520924400"; d="scan'208";a="45609826" From: bing.niu@intel.com To: libvir-list@redhat.com Date: Tue, 29 May 2018 18:58:03 +0800 Message-Id: <1527591483-20151-3-git-send-email-bing.niu@intel.com> In-Reply-To: <1527591483-20151-1-git-send-email-bing.niu@intel.com> References: <1527591483-20151-1-git-send-email-bing.niu@intel.com> X-Greylist: Sender passed SPF test, Sender IP whitelisted by DNSRBL, ACL 207 matched, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Tue, 29 May 2018 11:22:30 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Tue, 29 May 2018 11:22:30 +0000 (UTC) for IP:'192.55.52.136' DOMAIN:'mga12.intel.com' HELO:'mga12.intel.com' FROM:'bing.niu@intel.com' RCPT:'' X-RedHat-Spam-Score: -2.301 (RCVD_IN_DNSWL_MED, SPF_PASS) 192.55.52.136 mga12.intel.com 192.55.52.136 mga12.intel.com X-Scanned-By: MIMEDefang 2.78 on 10.5.110.31 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-loop: libvir-list@redhat.com Cc: shaohe.feng@intel.com, huaqiang.wang@intel.com, Bing Niu , jian-feng.ding@intel.com, rui.zang@yandex.com Subject: [libvirt] [RFC PATCH 2/2] conf: Extend cputune/cachetune to support memory bandwidth allocation X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.84 on 10.5.11.24 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.43]); Tue, 29 May 2018 11:22:35 +0000 (UTC) X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" From: Bing Niu Extend current cachetune section to support memory bandwidth allocation. Add a new cachetune element llc for memory allocation. As the example below: id --- on which last level cache memory bandwidth to be set bandwidth --- the memory bandwidth percent to set. Signed-off-by: Bing Niu --- src/conf/domain_conf.c | 86 ++++++++++++++++++++++++++++++++++++++++++++++= ---- src/util/virresctrl.c | 7 ++++ 2 files changed, 86 insertions(+), 7 deletions(-) diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index d6ac47c..aba998d 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -18578,6 +18578,55 @@ virDomainDefParseBootOptions(virDomainDefPtr def, return ret; } =20 +static int +virDomainCachetuneDefParseMemoryBandwidth(xmlXPathContextPtr ctxt, + xmlNodePtr node, + virResctrlAllocPtr alloc) +{ + xmlNodePtr oldnode =3D ctxt->node; + unsigned int id; + unsigned int bandwidth; + char *tmp =3D NULL; + int ret =3D -1; + + ctxt->node =3D node; + + tmp =3D virXMLPropString(node, "id"); + if (!tmp) { + virReportError(VIR_ERR_XML_ERROR, "%s", + _("Missing cachetune attribute 'id'")); + goto cleanup; + } + if (virStrToLong_uip(tmp, NULL, 10, &id) < 0) { + virReportError(VIR_ERR_XML_ERROR, + _("Invalid cachetune attribute 'id' value '%s'"), + tmp); + goto cleanup; + } + VIR_FREE(tmp); + + tmp =3D virXMLPropString(node, "bandwidth"); + if (!tmp) { + virReportError(VIR_ERR_XML_ERROR, "%s", + _("Missing cachetune attribute 'bandwidth'")); + goto cleanup; + } + if (virStrToLong_uip(tmp, NULL, 10, &bandwidth) < 0) { + virReportError(VIR_ERR_XML_ERROR, + _("Invalid cachetune attribute 'bandwidth' value '%= s'"), + tmp); + goto cleanup; + } + VIR_FREE(tmp); + if (virResctrlSetMemoryBandwidth(alloc, id, bandwidth) < 0) + goto cleanup; + + ret =3D 0; + cleanup: + ctxt->node =3D oldnode; + VIR_FREE(tmp); + return ret; +} =20 static int virDomainCachetuneDefParseCache(xmlXPathContextPtr ctxt, @@ -18670,6 +18719,7 @@ virDomainCachetuneDefParse(virDomainDefPtr def, ssize_t i =3D 0; int n; int ret =3D -1; + bool mba_available =3D false; =20 ctxt->node =3D node; =20 @@ -18701,12 +18751,26 @@ virDomainCachetuneDefParse(virDomainDefPtr def, goto cleanup; } =20 + if ((n =3D virXPathNodeSet("./llc", ctxt, &nodes)) < 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("Cannot extract Memory Bandwidth nodes under " + "cachetune. try cache allocation")); + } + + for (i =3D 0; i < n; i++) { + if (virDomainCachetuneDefParseMemoryBandwidth(ctxt, nodes[i], allo= c) < 0) + goto cleanup; + } + + if (n) + mba_available =3D true; + if ((n =3D virXPathNodeSet("./cache", ctxt, &nodes)) < 0) { virReportError(VIR_ERR_INTERNAL_ERROR, "%s", _("Cannot extract cache nodes under cachetune")); - goto cleanup; + if (!mba_available) + goto cleanup; } - for (i =3D 0; i < n; i++) { if (virDomainCachetuneDefParseCache(ctxt, nodes[i], alloc) < 0) goto cleanup; @@ -26394,11 +26458,19 @@ virDomainCachetuneDefFormatHelper(unsigned int le= vel, virBufferPtr buf =3D opaque; unsigned long long short_size =3D virFormatIntPretty(size, &unit); =20 - virBufferAsprintf(buf, - "\n", - cache, level, virCacheTypeToString(type), - short_size, unit); + /* If type is VIR_CACHE_TYPE_LAST, this means it's a memory + * bandwidth allocation formatting request */ + if (type =3D=3D VIR_CACHE_TYPE_LAST) + virBufferAsprintf(buf, + "\n", + cache, size); + + else + virBufferAsprintf(buf, + "\n", + cache, level, virCacheTypeToString(type), + short_size, unit); =20 return 0; } diff --git a/src/util/virresctrl.c b/src/util/virresctrl.c index ad050c7..c720074 100644 --- a/src/util/virresctrl.c +++ b/src/util/virresctrl.c @@ -954,6 +954,13 @@ virResctrlAllocForeachSize(virResctrlAllocPtr resctrl, } } =20 + if (resctrl->mba) { + virResctrlAllocMBPtr mba =3D resctrl->mba; + for (cache =3D 0; cache < mba->nsizes; cache++) + if (mba->bandwidth[cache]) + cb(0, VIR_CACHE_TYPE_LAST, cache, *mba->bandwidth[cache], = opaque); + } + return 0; } =20 --=20 2.7.4 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list