From nobody Sat Feb 7 10:14:47 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.37 as permitted sender) client-ip=209.132.183.37; envelope-from=libvir-list-bounces@redhat.com; helo=mx5-phx2.redhat.com; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.37 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; Return-Path: Received: from mx5-phx2.redhat.com (mx5-phx2.redhat.com [209.132.183.37]) by mx.zohomail.com with SMTPS id 1486972118317761.0883323954517; Sun, 12 Feb 2017 23:48:38 -0800 (PST) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by mx5-phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v1D7jAGI060597; Mon, 13 Feb 2017 02:45:10 -0500 Received: from int-mx13.intmail.prod.int.phx2.redhat.com (int-mx13.intmail.prod.int.phx2.redhat.com [10.5.11.26]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id v1D7j9ni011339 for ; Mon, 13 Feb 2017 02:45:09 -0500 Received: from mx1.redhat.com (ext-mx07.extmail.prod.ext.phx2.redhat.com [10.5.110.31]) by int-mx13.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v1D7j91B020475 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Mon, 13 Feb 2017 02:45:09 -0500 Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 12E82C04B92E; Mon, 13 Feb 2017 07:45:07 +0000 (UTC) Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga103.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 12 Feb 2017 23:45:05 -0800 Received: from s2600wt.sh.intel.com ([10.239.48.158]) by orsmga005.jf.intel.com with ESMTP; 12 Feb 2017 23:45:03 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.35,155,1484035200"; d="scan'208";a="63990353" From: Eli Qiao To: libvir-list@redhat.com Date: Mon, 13 Feb 2017 15:46:39 +0800 Message-Id: <1486972007-15514-2-git-send-email-liyong.qiao@intel.com> In-Reply-To: <1486972007-15514-1-git-send-email-liyong.qiao@intel.com> References: <1486972007-15514-1-git-send-email-liyong.qiao@intel.com> X-Greylist: Sender passed SPF test, Sender IP whitelisted by DNSRBL, ACL 200 matched, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Mon, 13 Feb 2017 07:45:07 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Mon, 13 Feb 2017 07:45:07 +0000 (UTC) for IP:'192.55.52.115' DOMAIN:'mga14.intel.com' HELO:'mga14.intel.com' FROM:'liyong.qiao@intel.com' RCPT:'' X-RedHat-Spam-Score: -4.622 (BAYES_50, DCC_REPUT_00_12, RCVD_IN_DNSWL_HI, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, RP_MATCHES_RCVD, SPF_PASS) 192.55.52.115 mga14.intel.com 192.55.52.115 mga14.intel.com X-Scanned-By: MIMEDefang 2.68 on 10.5.11.26 X-Scanned-By: MIMEDefang 2.78 on 10.5.110.31 X-loop: libvir-list@redhat.com Cc: mkletzan@redhat.com, qiaowei.ren@intel.com, mtosatti@redhat.com, qiaoliyong@gmail.com Subject: [libvirt] [v5 1/9] Resctrl: Add some utils functions X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" This patch adds some utils struct and functions to expose resctrl information. virResCtrlAvailable: If resctrl interface exist on host virResCtrlGet: get specify type resource contral information virResCtrlInit: initialize resctrl struct from the host's sys fs. resctrlall[]: an array to maintain resource control information. Signed-off-by: Eli Qiao --- include/libvirt/virterror.h | 1 + po/POTFILES.in | 1 + src/Makefile.am | 1 + src/libvirt_private.syms | 4 + src/util/virerror.c | 1 + src/util/virresctrl.c | 343 ++++++++++++++++++++++++++++++++++++++++= ++++ src/util/virresctrl.h | 80 +++++++++++ 7 files changed, 431 insertions(+) create mode 100644 src/util/virresctrl.c create mode 100644 src/util/virresctrl.h diff --git a/include/libvirt/virterror.h b/include/libvirt/virterror.h index 2efee8f..3dd2d08 100644 --- a/include/libvirt/virterror.h +++ b/include/libvirt/virterror.h @@ -132,6 +132,7 @@ typedef enum { =20 VIR_FROM_PERF =3D 65, /* Error from perf */ VIR_FROM_LIBSSH =3D 66, /* Error from libssh connection transpor= t */ + VIR_FROM_RESCTRL =3D 67, /* Error from resource control */ =20 # ifdef VIR_ENUM_SENTINELS VIR_ERR_DOMAIN_LAST diff --git a/po/POTFILES.in b/po/POTFILES.in index 365ea66..f7fda98 100644 --- a/po/POTFILES.in +++ b/po/POTFILES.in @@ -240,6 +240,7 @@ src/util/virportallocator.c src/util/virprocess.c src/util/virqemu.c src/util/virrandom.c +src/util/virresctrl.c src/util/virrotatingfile.c src/util/virscsi.c src/util/virscsivhost.c diff --git a/src/Makefile.am b/src/Makefile.am index 2f32d41..b626f29 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -162,6 +162,7 @@ UTIL_SOURCES =3D \ util/virprocess.c util/virprocess.h \ util/virqemu.c util/virqemu.h \ util/virrandom.h util/virrandom.c \ + util/virresctrl.h util/virresctrl.c \ util/virrotatingfile.h util/virrotatingfile.c \ util/virscsi.c util/virscsi.h \ util/virscsivhost.c util/virscsivhost.h \ diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index 8e994c7..743e5ac 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -2313,6 +2313,10 @@ virRandomGenerateWWN; virRandomInt; =20 =20 +# util/virresctrl.h +virResCtrlAvailable; +virResCtrlInit; + # util/virrotatingfile.h virRotatingFileReaderConsume; virRotatingFileReaderFree; diff --git a/src/util/virerror.c b/src/util/virerror.c index ef17fb5..0ba15e6 100644 --- a/src/util/virerror.c +++ b/src/util/virerror.c @@ -139,6 +139,7 @@ VIR_ENUM_IMPL(virErrorDomain, VIR_ERR_DOMAIN_LAST, =20 "Perf", /* 65 */ "Libssh transport layer", + "Resouce Control", ) =20 =20 diff --git a/src/util/virresctrl.c b/src/util/virresctrl.c new file mode 100644 index 0000000..80481bc --- /dev/null +++ b/src/util/virresctrl.c @@ -0,0 +1,343 @@ +/* + * virresctrl.c: methods for managing resource contral + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library. If not, see + * . + * + * Authors: + * Eli Qiao + */ +#include + +#include +#if defined HAVE_SYS_SYSCALL_H +# include +#endif +#include +#include +#include + +#include "virresctrl.h" +#include "viralloc.h" +#include "virerror.h" +#include "virfile.h" +#include "virhostcpu.h" +#include "virlog.h" +#include "virstring.h" +#include "nodeinfo.h" + +VIR_LOG_INIT("util.resctrl"); + +#define VIR_FROM_THIS VIR_FROM_RESCTRL + +#define RESCTRL_DIR "/sys/fs/resctrl" +#define RESCTRL_INFO_DIR "/sys/fs/resctrl/info" +#define SYSFS_SYSTEM_PATH "/sys/devices/system" + +#define MAX_CPU_SOCKET_NUM 8 +#define MAX_CBM_BIT_LEN 32 +#define MAX_SCHEMATA_LEN 1024 +#define MAX_FILE_LEN ( 10 * 1024 * 1024) + + +static unsigned int host_id; + +static virResCtrl resctrlall[] =3D { + { + .name =3D "L3", + .cache_level =3D "l3", + }, + { + .name =3D "L3DATA", + .cache_level =3D "l3", + }, + { + .name =3D "L3CODE", + .cache_level =3D "l3", + }, + { + .name =3D "L2", + .cache_level =3D "l2", + }, +}; + +static int virResCtrlGetInfoStr(const int type, const char *item, char **s= tr) +{ + int ret =3D 0; + char *tmp; + char *path; + + if (virAsprintf(&path, "%s/%s/%s", RESCTRL_INFO_DIR, resctrlall[type].= name, item) < 0) + return -1; + if (virFileReadAll(path, 10, str) < 0) { + ret =3D -1; + goto cleanup; + } + + if ((tmp =3D strchr(*str, '\n'))) *tmp =3D '\0'; + + cleanup: + VIR_FREE(path); + return ret; +} + + +static int virResCtrlGetCPUValue(const char *path, char **value) +{ + int ret =3D -1; + char *tmp; + + if (virFileReadAll(path, 10, value) < 0) goto cleanup; + if ((tmp =3D strchr(*value, '\n'))) *tmp =3D '\0'; + ret =3D 0; + + cleanup: + return ret; +} + +static int virResctrlGetCPUSocketID(const size_t cpu, int *socket_id) +{ + int ret =3D -1; + char *physical_package_path =3D NULL; + char *physical_package =3D NULL; + if (virAsprintf(&physical_package_path, + "%s/cpu/cpu%zu/topology/physical_package_id", + SYSFS_SYSTEM_PATH, cpu) < 0) { + return -1; + } + + if (virResCtrlGetCPUValue(physical_package_path, + &physical_package) < 0) + goto cleanup; + + if (virStrToLong_i(physical_package, NULL, 0, socket_id) < 0) + goto cleanup; + + ret =3D 0; + cleanup: + VIR_FREE(physical_package); + VIR_FREE(physical_package_path); + return ret; +} + +static int virResCtrlGetCPUCache(const size_t cpu, int type, int *cache) +{ + int ret =3D -1; + char *cache_dir =3D NULL; + char *cache_str =3D NULL; + char *tmp; + int carry =3D -1; + + if (virAsprintf(&cache_dir, + "%s/cpu/cpu%zu/cache/index%d/size", + SYSFS_SYSTEM_PATH, cpu, type) < 0) + return -1; + + if (virResCtrlGetCPUValue(cache_dir, &cache_str) < 0) + goto cleanup; + + tmp =3D cache_str; + + while (*tmp !=3D '\0') tmp++; + + if (*(tmp - 1) =3D=3D 'K') { + *(tmp - 1) =3D '\0'; + carry =3D 1; + } else if (*(tmp - 1) =3D=3D 'M') { + *(tmp - 1) =3D '\0'; + carry =3D 1024; + } + + if (virStrToLong_i(cache_str, NULL, 0, cache) < 0) + goto cleanup; + + *cache =3D (*cache) * carry; + + if (*cache < 0) + goto cleanup; + + ret =3D 0; + cleanup: + VIR_FREE(cache_dir); + VIR_FREE(cache_str); + return ret; +} + +/* Fill all cache bank informations */ +static virResCacheBankPtr virResCtrlGetCacheBanks(int type, int *n_sockets) +{ + int npresent_cpus; + int idx =3D -1; + size_t i; + virResCacheBankPtr bank; + + *n_sockets =3D 1; + if ((npresent_cpus =3D virHostCPUGetCount()) < 0) + return NULL; + + if (type =3D=3D VIR_RDT_RESOURCE_L3 + || type =3D=3D VIR_RDT_RESOURCE_L3DATA + || type =3D=3D VIR_RDT_RESOURCE_L3CODE) + idx =3D 3; + else if (type =3D=3D VIR_RDT_RESOURCE_L2) + idx =3D 2; + + if (idx =3D=3D -1) + return NULL; + + if (VIR_ALLOC_N(bank, *n_sockets) < 0) { + *n_sockets =3D 0; + return NULL; + } + + for (i =3D 0; i < npresent_cpus; i ++) { + int s_id; + int cache_size; + + if (virResctrlGetCPUSocketID(i, &s_id) < 0) + goto error; + + if (s_id > (*n_sockets - 1)) { + size_t cur =3D *n_sockets; + size_t exp =3D s_id - (*n_sockets) + 1; + if (VIR_EXPAND_N(bank, cur, exp) < 0) + goto error; + *n_sockets =3D s_id + 1; + } + + if (bank[s_id].cpu_mask =3D=3D NULL) { + if (!(bank[s_id].cpu_mask =3D virBitmapNew(npresent_cpus))) + goto error; + } + + ignore_value(virBitmapSetBit(bank[s_id].cpu_mask, i)); + + if (bank[s_id].cache_size =3D=3D 0) { + if (virResCtrlGetCPUCache(i, idx, &cache_size) < 0) + goto error; + + bank[s_id].cache_size =3D cache_size; + bank[s_id].cache_min =3D cache_size / resctrlall[type].cbm_len; + } + } + return bank; + + error: + *n_sockets =3D 0; + VIR_FREE(bank); + return NULL; +} + +static int virResCtrlGetConfig(int type) +{ + int ret; + size_t i; + char *str; + + /* Read min_cbm_bits from resctrl. + eg: /sys/fs/resctrl/info/L3/num_closids + */ + if ((ret =3D virResCtrlGetInfoStr(type, "num_closids", &str)) < 0) + return ret; + + if (virStrToLong_i(str, NULL, 10, &resctrlall[type].num_closid) < 0) + return -1; + + VIR_FREE(str); + + /* Read min_cbm_bits from resctrl. + eg: /sys/fs/resctrl/info/L3/cbm_mask + */ + if ((ret =3D virResCtrlGetInfoStr(type, "min_cbm_bits", &str)) < 0) + return ret; + + if (virStrToLong_i(str, NULL, 10, &resctrlall[type].min_cbm_bits) < 0) + return -1; + + VIR_FREE(str); + + /* Read cbm_mask string from resctrl. + eg: /sys/fs/resctrl/info/L3/cbm_mask + */ + if ((ret =3D virResCtrlGetInfoStr(type, "cbm_mask", &str)) < 0) + return ret; + + /* Calculate cbm length from the default cbm_mask. */ + resctrlall[type].cbm_len =3D strlen(str) * 4; + VIR_FREE(str); + + /* Get all cache bank informations */ + resctrlall[type].cache_banks =3D virResCtrlGetCacheBanks(type, + &(resctrlall[ty= pe].num_banks)); + + if (resctrlall[type].cache_banks =3D=3D NULL) + return -1; + + for (i =3D 0; i < resctrlall[type].num_banks; i++) { + /*L3CODE and L3DATA shares same L3 resource, so they should + * have same host_id. */ + if (type =3D=3D VIR_RDT_RESOURCE_L3CODE) + resctrlall[type].cache_banks[i].host_id =3D resctrlall[VIR_RDT= _RESOURCE_L3DATA].cache_banks[i].host_id; + else + resctrlall[type].cache_banks[i].host_id =3D host_id++; + } + + resctrlall[type].enabled =3D true; + + return ret; +} + +int +virResCtrlInit(void) +{ + size_t i =3D 0; + char *tmp; + int rc =3D 0; + + for (i =3D 0; i < VIR_RDT_RESOURCE_LAST; i++) { + if ((rc =3D virAsprintf(&tmp, "%s/%s", RESCTRL_INFO_DIR, resctrlal= l[i].name)) < 0) { + VIR_ERROR(_("Failed to initialize resource control config")); + return -1; + } + if (virFileExists(tmp)) { + if ((rc =3D virResCtrlGetConfig(i)) < 0) + VIR_ERROR(_("Failed to get resource control config")); + return -1; + } + + VIR_FREE(tmp); + } + return rc; +} + +/* + * Test whether the host support resource control + */ +bool +virResCtrlAvailable(void) +{ + if (!virFileExists(RESCTRL_INFO_DIR)) + return false; + return true; +} + +/* + * Return an virResCtrlPtr point to virResCtrl object, + * We should not modify it out side of virresctrl.c + */ +virResCtrlPtr +virResCtrlGet(int type) +{ + return &resctrlall[type]; +} diff --git a/src/util/virresctrl.h b/src/util/virresctrl.h new file mode 100644 index 0000000..aa113f4 --- /dev/null +++ b/src/util/virresctrl.h @@ -0,0 +1,80 @@ +/* + * * virresctrl.h: methods for managing rscctrl + * * + * * Copyright (C) 2016 Intel, Inc. + * * + * * This library is free software; you can redistribute it and/or + * * modify it under the terms of the GNU Lesser General Public + * * License as published by the Free Software Foundation; either + * * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library. If not, see + * . + * + * Authors: + * Eli Qiao + */ + +#ifndef __VIR_RESCTRL_H__ +# define __VIR_RESCTRL_H__ + +# include "virutil.h" +# include "virbitmap.h" + +enum { + VIR_RDT_RESOURCE_L3, + VIR_RDT_RESOURCE_L3DATA, + VIR_RDT_RESOURCE_L3CODE, + VIR_RDT_RESOURCE_L2, + /* Must be the last */ + VIR_RDT_RESOURCE_LAST, +}; + + +typedef struct _virResCacheBank virResCacheBank; +typedef virResCacheBank *virResCacheBankPtr; +struct _virResCacheBank { + unsigned int host_id; + unsigned long long cache_size; + unsigned long long cache_left; + unsigned long long cache_min; + virBitmapPtr cpu_mask; +}; + +/** + * struct rdt_resource - attributes of an RDT resource + * @enabled: Is this feature enabled on this machine + * @name: Name to use in "schemata" file + * @num_closid: Number of CLOSIDs available + * @max_cbm: Largest Cache Bit Mask allowed + * @min_cbm_bits: Minimum number of consecutive bits to be s= et + * in a cache bit mask + * @cache_level: Which cache level defines scope of this do= main + * @num_banks: Number of cache bank on this machine. + * @cache_banks: Array of cache bank + */ +typedef struct _virResCtrl virResCtrl; +typedef virResCtrl *virResCtrlPtr; +struct _virResCtrl { + bool enabled; + const char *name; + int num_closid; + int cbm_len; + int min_cbm_bits; + const char* cache_level; + int num_banks; + virResCacheBankPtr cache_banks; +}; + + +bool virResCtrlAvailable(void); +int virResCtrlInit(void); +virResCtrlPtr virResCtrlGet(int); + +#endif --=20 1.9.1 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list