From nobody Fri Apr 26 07:47:41 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.24 as permitted sender) client-ip=209.132.183.24; envelope-from=libvir-list-bounces@redhat.com; helo=mx3-phx2.redhat.com; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.24 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; Return-Path: Received: from mx3-phx2.redhat.com (mx3-phx2.redhat.com [209.132.183.24]) by mx.zohomail.com with SMTPS id 1488793953952965.2381762998209; Mon, 6 Mar 2017 01:52:33 -0800 (PST) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by mx3-phx2.redhat.com (8.13.8/8.13.8) with ESMTP id v269mvQs006895; Mon, 6 Mar 2017 04:48:57 -0500 Received: from int-mx10.intmail.prod.int.phx2.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id v269mjvu029117 for ; Mon, 6 Mar 2017 04:48:45 -0500 Received: from mx1.redhat.com (ext-mx01.extmail.prod.ext.phx2.redhat.com [10.5.110.25]) by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v269miFN006838 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Mon, 6 Mar 2017 04:48:45 -0500 Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id A4A6D81226; Mon, 6 Mar 2017 09:48:42 +0000 (UTC) Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 06 Mar 2017 01:48:39 -0800 Received: from s2600wt.sh.intel.com ([10.239.48.158]) by fmsmga001.fm.intel.com with ESMTP; 06 Mar 2017 01:48:38 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.35,253,1484035200"; d="scan'208";a="1118950626" From: Eli Qiao To: libvir-list@redhat.com Date: Mon, 6 Mar 2017 17:50:32 +0800 Message-Id: <1488793843-50925-2-git-send-email-liyong.qiao@intel.com> In-Reply-To: <1488793843-50925-1-git-send-email-liyong.qiao@intel.com> References: <1488793843-50925-1-git-send-email-liyong.qiao@intel.com> X-Greylist: Sender passed SPF test, Sender IP whitelisted by DNSRBL, ACL 203 matched, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.25]); Mon, 06 Mar 2017 09:48:43 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.25]); Mon, 06 Mar 2017 09:48:43 +0000 (UTC) for IP:'134.134.136.20' DOMAIN:'mga02.intel.com' HELO:'mga02.intel.com' FROM:'liyong.qiao@intel.com' RCPT:'' X-RedHat-Spam-Score: -1.922 (BAYES_50, DCC_REPUT_00_12, RCVD_IN_DNSWL_MED, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, RP_MATCHES_RCVD, SPF_PASS) 134.134.136.20 mga02.intel.com 134.134.136.20 mga02.intel.com X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23 X-Scanned-By: MIMEDefang 2.78 on 10.5.110.25 X-loop: libvir-list@redhat.com Cc: mkletzan@redhat.com, mtosatti@redhat.com, qiaoliyong@gmail.com Subject: [libvirt] [PATH V10 01/12] Resctrl: Add some utils functions X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" This patch adds some utils struct and functions to expose resctrl information. virResCtrlAvailable: if resctrl interface exist on host. virResCtrlGet: get specific type resource control information. virResCtrlInit: initialize resctrl struct from the host's sys fs. resctrlall[]: an array to maintain resource control information. Some of host cpu related information methods was added in virhostcpu.c Signed-off-by: Eli Qiao --- include/libvirt/virterror.h | 1 + po/POTFILES.in | 1 + src/Makefile.am | 1 + src/libvirt_private.syms | 4 + src/util/virerror.c | 1 + src/util/virhostcpu.c | 186 ++++++++++++++++++++++++++++++++++++---- src/util/virhostcpu.h | 6 ++ src/util/virresctrl.c | 201 ++++++++++++++++++++++++++++++++++++++++= ++++ src/util/virresctrl.h | 78 +++++++++++++++++ 9 files changed, 462 insertions(+), 17 deletions(-) create mode 100644 src/util/virresctrl.c create mode 100644 src/util/virresctrl.h diff --git a/include/libvirt/virterror.h b/include/libvirt/virterror.h index 2efee8f..3dd2d08 100644 --- a/include/libvirt/virterror.h +++ b/include/libvirt/virterror.h @@ -132,6 +132,7 @@ typedef enum { =20 VIR_FROM_PERF =3D 65, /* Error from perf */ VIR_FROM_LIBSSH =3D 66, /* Error from libssh connection transpor= t */ + VIR_FROM_RESCTRL =3D 67, /* Error from resource control */ =20 # ifdef VIR_ENUM_SENTINELS VIR_ERR_DOMAIN_LAST diff --git a/po/POTFILES.in b/po/POTFILES.in index 7c7f530..4147bc6 100644 --- a/po/POTFILES.in +++ b/po/POTFILES.in @@ -241,6 +241,7 @@ src/util/virportallocator.c src/util/virprocess.c src/util/virqemu.c src/util/virrandom.c +src/util/virresctrl.c src/util/virrotatingfile.c src/util/virscsi.c src/util/virscsihost.c diff --git a/src/Makefile.am b/src/Makefile.am index 7d42eac..edb946a 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -162,6 +162,7 @@ UTIL_SOURCES =3D \ util/virprocess.c util/virprocess.h \ util/virqemu.c util/virqemu.h \ util/virrandom.h util/virrandom.c \ + util/virresctrl.h util/virresctrl.c \ util/virrotatingfile.h util/virrotatingfile.c \ util/virscsi.c util/virscsi.h \ util/virscsihost.c util/virscsihost.h \ diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index aed1d3d..bb7c3ad 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -2320,6 +2320,10 @@ virRandomGenerateWWN; virRandomInt; =20 =20 +# util/virresctrl.h +virResCtrlAvailable; +virResCtrlInit; + # util/virrotatingfile.h virRotatingFileReaderConsume; virRotatingFileReaderFree; diff --git a/src/util/virerror.c b/src/util/virerror.c index ef17fb5..0ba15e6 100644 --- a/src/util/virerror.c +++ b/src/util/virerror.c @@ -139,6 +139,7 @@ VIR_ENUM_IMPL(virErrorDomain, VIR_ERR_DOMAIN_LAST, =20 "Perf", /* 65 */ "Libssh transport layer", + "Resouce Control", ) =20 =20 diff --git a/src/util/virhostcpu.c b/src/util/virhostcpu.c index f29f312..e6d5102 100644 --- a/src/util/virhostcpu.c +++ b/src/util/virhostcpu.c @@ -206,29 +206,21 @@ void virHostCPUSetSysFSSystemPathLinux(const char *pa= th) sysfs_system_path =3D SYSFS_SYSTEM_PATH; } =20 -/* Return the positive decimal contents of the given - * DIR/cpu%u/FILE, or -1 on error. If DEFAULT_VALUE is non-negative - * and the file could not be found, return that instead of an error; - * this is useful for machines that cannot hot-unplug cpu0, or where - * hot-unplugging is disabled, or where the kernel is too old - * to support NUMA cells, etc. */ +/* Get a String value*/ static int -virHostCPUGetValue(const char *dir, unsigned int cpu, const char *file, - int default_value) +virHostCPUGetStrValue(const char *dir, unsigned int cpu, const char *file,= char *value_str) { char *path; FILE *pathfp; - int value =3D -1; - char value_str[INT_BUFSIZE_BOUND(value)]; - char *tmp; + int ret =3D -1; =20 if (virAsprintf(&path, "%s/cpu%u/%s", dir, cpu, file) < 0) return -1; =20 pathfp =3D fopen(path, "r"); if (pathfp =3D=3D NULL) { - if (default_value >=3D 0 && errno =3D=3D ENOENT) - value =3D default_value; + if (errno =3D=3D ENOENT) + return -2; else virReportSystemError(errno, _("cannot open %s"), path); goto cleanup; @@ -238,17 +230,84 @@ virHostCPUGetValue(const char *dir, unsigned int cpu,= const char *file, virReportSystemError(errno, _("cannot read from %s"), path); goto cleanup; } + + ret =3D 0; + + cleanup: + VIR_FORCE_FCLOSE(pathfp); + VIR_FREE(path); + return ret; +} + + +/* Return the positive decimal contents of the given + * DIR/cpu%u/FILE, or -1 on error. If DEFAULT_VALUE is non-negative + * and the file could not be found, return that instead of an error; + * this is useful for machines that cannot hot-unplug cpu0, or where + * hot-unplugging is disabled, or where the kernel is too old + * to support NUMA cells, etc. */ +static int +virHostCPUGetValue(const char *dir, unsigned int cpu, const char *file, + int default_value) +{ + int value =3D -1; + char value_str[INT_BUFSIZE_BOUND(value)]; + char *tmp; + int ret; + + if ((ret =3D (virHostCPUGetStrValue(dir, cpu, file, value_str))) < 0) { + if (ret =3D=3D -2 && default_value >=3D 0) + return default_value; + else + return -1; + } + if (virStrToLong_i(value_str, &tmp, 10, &value) < 0) { virReportError(VIR_ERR_INTERNAL_ERROR, _("could not convert '%s' to an integer"), value_str); - goto cleanup; + return -1; } + return value; +} =20 - cleanup: - VIR_FORCE_FCLOSE(pathfp); - VIR_FREE(path); +/* Return specific type cache size in KiB of given cpu + -1 on error happened */ +static +int virHostCPUGetCache(unsigned int cpu, unsigned int type) +{ + char *cachedir =3D NULL; + char *cpudir; + char *unit =3D NULL; + char *tmp; + int value =3D -1; + unsigned long long size; + char value_str[INT_BUFSIZE_BOUND(value)]; =20 + if (virAsprintf(&cpudir, "%s/cpu", sysfs_system_path) < 0) + return -1; + + if (virAsprintf(&cachedir, "cache/index%u/size", type) < 0) + goto error; + + if (virHostCPUGetStrValue(cpudir, cpu, cachedir, value_str) < 0) + goto error; + + if ((tmp =3D strchr(value_str, '\n'))) *tmp =3D '\0'; + + if (virStrToLong_i(value_str, &unit, 10, &value) < 0) + goto error; + + size =3D value; + + if (virScaleInteger(&size, unit, 1, ULLONG_MAX) < 0) + goto error; + + return size / 1024; + + error: + VIR_FREE(cpudir); + VIR_FREE(cachedir); return value; } =20 @@ -301,6 +360,23 @@ virHostCPUParseSocket(const char *dir, return ret; } =20 +/* return socket id of a given cpu*/ +static +int virHostCPUGetSocketId(virArch hostarch, unsigned int cpu) +{ + char *cpu_dir; + int ret =3D -1; + + if (virAsprintf(&cpu_dir, "%s/cpu", sysfs_system_path) < 0) + goto cleanup; + + ret =3D virHostCPUParseSocket(cpu_dir, hostarch, cpu); + + cleanup: + VIR_FREE(cpu_dir); + return ret; +} + /* parses a node entry, returning number of processors in the node and * filling arguments */ static int @@ -1346,3 +1422,79 @@ virHostCPUGetKVMMaxVCPUs(void) return -1; } #endif /* HAVE_LINUX_KVM_H */ + +/* Fill all cache bank informations + * Return a list of virResCacheBankPtr, and fill cache bank information + * by loop for all cpus on host, number of cache bank will be set in nbanks + * + * NULL if error happened, and nbanks will be set 0. */ +virResCacheBankPtr virHostCPUGetCacheBanks(virArch arch, int type, size_t = *nbanks, int cbm_len) +{ + int npresent_cpus; + int idx; + size_t i; + virResCacheBankPtr bank; + + *nbanks =3D 0; + if ((npresent_cpus =3D virHostCPUGetCount()) < 0) + return NULL; + + switch (type) { + case VIR_RDT_RESOURCE_L3: + case VIR_RDT_RESOURCE_L3DATA: + case VIR_RDT_RESOURCE_L3CODE: + idx =3D 3; + break; + case VIR_RDT_RESOURCE_L2: + idx =3D 2; + break; + default: + idx =3D -1; + } + + if (idx =3D=3D -1) + return NULL; + + if (VIR_ALLOC_N(bank, 1) < 0) + return NULL; + + *nbanks =3D 1; + + for (i =3D 0; i < npresent_cpus; i ++) { + int s_id; + int cache_size; + + if ((s_id =3D virHostCPUGetSocketId(arch, i)) < 0) + goto error; + + /* Expand cache bank array */ + if (s_id > (*nbanks - 1)) { + size_t cur =3D *nbanks; + size_t exp =3D s_id - (*nbanks) + 1; + if (VIR_EXPAND_N(bank, cur, exp) < 0) + goto error; + *nbanks =3D s_id + 1; + } + + if (bank[s_id].cpu_mask =3D=3D NULL) { + if (!(bank[s_id].cpu_mask =3D virBitmapNew(npresent_cpus))) + goto error; + } + + ignore_value(virBitmapSetBit(bank[s_id].cpu_mask, i)); + + if (bank[s_id].cache_size =3D=3D 0) { + if ((cache_size =3D virHostCPUGetCache(i, idx)) < 0) + goto error; + + bank[s_id].cache_size =3D cache_size; + bank[s_id].cache_min =3D cache_size / cbm_len; + } + } + return bank; + + error: + *nbanks =3D 0; + VIR_FREE(bank); + return NULL; +} diff --git a/src/util/virhostcpu.h b/src/util/virhostcpu.h index 39f7cf8..27f208e 100644 --- a/src/util/virhostcpu.h +++ b/src/util/virhostcpu.h @@ -27,6 +27,7 @@ # include "internal.h" # include "virarch.h" # include "virbitmap.h" +# include "virresctrl.h" =20 # define VIR_HOST_CPU_MASK_LEN 1024 =20 @@ -58,4 +59,9 @@ int virHostCPUStatsAssign(virNodeCPUStatsPtr param, const char *name, unsigned long long value); =20 +virResCacheBankPtr virHostCPUGetCacheBanks(virArch arch, + int type, + size_t *nbanks, + int cbm_len); + #endif /* __VIR_HOSTCPU_H__*/ diff --git a/src/util/virresctrl.c b/src/util/virresctrl.c new file mode 100644 index 0000000..44a47cc --- /dev/null +++ b/src/util/virresctrl.c @@ -0,0 +1,201 @@ +/* + * virresctrl.c: methods for managing resource control + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library. If not, see + * . + * + * Authors: + * Eli Qiao + */ +#include + +#include "virresctrl.h" +#include "viralloc.h" +#include "virerror.h" +#include "virfile.h" +#include "virhostcpu.h" +#include "virlog.h" +#include "virstring.h" +#include "virarch.h" + +VIR_LOG_INIT("util.resctrl"); + +#define VIR_FROM_THIS VIR_FROM_RESCTRL + +#define RESCTRL_DIR "/sys/fs/resctrl" +#define RESCTRL_INFO_DIR "/sys/fs/resctrl/info" +#define SYSFS_SYSTEM_PATH "/sys/devices/system" + +#define MAX_CPU_SOCKET_NUM 8 +#define MAX_CBM_BIT_LEN 32 +#define MAX_SCHEMATA_LEN 1024 +#define MAX_FILE_LEN (10 * 1024 * 1024) + +static unsigned int host_id; + +static virResCtrl resctrlall[] =3D { + { + .name =3D "L3", + .cache_level =3D "l3", + }, + { + .name =3D "L3DATA", + .cache_level =3D "l3", + }, + { + .name =3D "L3CODE", + .cache_level =3D "l3", + }, + { + .name =3D "L2", + .cache_level =3D "l2", + }, +}; + +static int virResCtrlGetInfoStr(const int type, const char *item, char **s= tr) +{ + int ret =3D 0; + char *tmp; + char *path; + + if (virAsprintf(&path, "%s/%s/%s", RESCTRL_INFO_DIR, resctrlall[type].= name, item) < 0) + return -1; + if (virFileReadAll(path, 10, str) < 0) { + ret =3D -1; + goto cleanup; + } + + if ((tmp =3D strchr(*str, '\n'))) *tmp =3D '\0'; + + cleanup: + VIR_FREE(path); + return ret; +} + +static int virResCtrlReadConfig(virArch arch, int type) +{ + int ret; + size_t i, nbanks; + char *str; + + /* Read num_closids from resctrl. + eg: /sys/fs/resctrl/info/L3/num_closids */ + if ((ret =3D virResCtrlGetInfoStr(type, "num_closids", &str)) < 0) + goto error; + + if ((ret =3D virStrToLong_i(str, NULL, 10, &resctrlall[type].num_closi= d)) < 0) + goto error; + + VIR_FREE(str); + + /* Read min_cbm_bits from resctrl. + eg: /sys/fs/resctrl/info/L3/cbm_mask */ + if ((ret =3D virResCtrlGetInfoStr(type, "min_cbm_bits", &str)) < 0) + goto error; + + if ((ret =3D virStrToLong_i(str, NULL, 10, &resctrlall[type].min_cbm_b= its)) < 0) + goto error; + + VIR_FREE(str); + + /* Read cbm_mask string from resctrl. + eg: /sys/fs/resctrl/info/L3/cbm_mask */ + if ((ret =3D virResCtrlGetInfoStr(type, "cbm_mask", &str)) < 0) + goto error; + + /* cbm_mask is in hex, eg: "fffff", calculate cbm length from the defa= ult + cbm_mask. */ + resctrlall[type].cbm_len =3D strlen(str) * 4; + + /* Get all cache bank informations */ + resctrlall[type].cache_banks =3D virHostCPUGetCacheBanks(arch, + type, + &nbanks, resctr= lall[type].cbm_len); + + if (resctrlall[type].cache_banks =3D=3D NULL) + goto error; + + resctrlall[type].num_banks =3D nbanks; + + for (i =3D 0; i < resctrlall[type].num_banks; i++) { + /* L3CODE and L3DATA shares same L3 resource, so they should + * have same host_id. */ + if (type =3D=3D VIR_RDT_RESOURCE_L3CODE) + resctrlall[type].cache_banks[i].host_id =3D resctrlall[VIR_RDT= _RESOURCE_L3DATA].cache_banks[i].host_id; + else + resctrlall[type].cache_banks[i].host_id =3D host_id++; + } + + resctrlall[type].enabled =3D true; + + ret =3D 0; + + error: + VIR_FREE(str); + return ret; +} + +int +virResCtrlInit(void) +{ + size_t i =3D 0; + char *tmp; + int rc =3D 0; + + virArch hostarch; + + hostarch =3D virArchFromHost(); + + for (i =3D 0; i < VIR_RDT_RESOURCE_LAST; i++) { + if ((rc =3D virAsprintf(&tmp, "%s/%s", RESCTRL_INFO_DIR, resctrlal= l[i].name)) < 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("Failed to initialize resource control config= ")); + goto cleanup; + } + + if (virFileExists(tmp)) { + if ((rc =3D virResCtrlReadConfig(hostarch, i)) < 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("Failed to get resource control config")); + goto cleanup; + } + } + VIR_FREE(tmp); + } + + cleanup: + VIR_FREE(tmp); + return rc; +} + +/* + * Test whether the host support resource control + */ +bool +virResCtrlAvailable(void) +{ + if (!virFileExists(RESCTRL_INFO_DIR)) + return false; + return true; +} + +/* + * Return an virResCtrlPtr point to virResCtrl object, + * We should not modify it out side of virresctrl.c + */ +virResCtrlPtr +virResCtrlGet(int type) +{ + return &resctrlall[type]; +} diff --git a/src/util/virresctrl.h b/src/util/virresctrl.h new file mode 100644 index 0000000..5a6a344 --- /dev/null +++ b/src/util/virresctrl.h @@ -0,0 +1,78 @@ +/* + * virresctrl.h: header for managing resctrl control + * + * Copyright (C) 2016 Intel, Inc. + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library. If not, see + * . + * + * Authors: + * Eli Qiao + */ + +#ifndef __VIR_RESCTRL_H__ +# define __VIR_RESCTRL_H__ + +# include "virbitmap.h" + +enum { + VIR_RDT_RESOURCE_L3, + VIR_RDT_RESOURCE_L3DATA, + VIR_RDT_RESOURCE_L3CODE, + VIR_RDT_RESOURCE_L2, + /* Must be the last */ + VIR_RDT_RESOURCE_LAST, +}; + + +typedef struct _virResCacheBank virResCacheBank; +typedef virResCacheBank *virResCacheBankPtr; +struct _virResCacheBank { + unsigned int host_id; + unsigned long long cache_size; + unsigned long long cache_left; + unsigned long long cache_min; + virBitmapPtr cpu_mask; +}; + +/** + * struct rdt_resource - attributes of an RDT resource + * @enabled: Is this feature enabled on this machine + * @name: Name to use in "schemata" file + * @num_closid: Number of CLOSIDs available + * @max_cbm: Largest Cache Bit Mask allowed + * @min_cbm_bits: Minimum number of consecutive bits to be s= et + * in a cache bit mask + * @cache_level: Which cache level defines scope of this do= main + * @num_banks: Number of cache bank on this machine. + * @cache_banks: Array of cache bank + */ +typedef struct _virResCtrl virResCtrl; +typedef virResCtrl *virResCtrlPtr; +struct _virResCtrl { + bool enabled; + const char *name; + int num_closid; + int cbm_len; + int min_cbm_bits; + const char* cache_level; + int num_banks; + virResCacheBankPtr cache_banks; +}; + +bool virResCtrlAvailable(void); +int virResCtrlInit(void); +virResCtrlPtr virResCtrlGet(int); + +#endif --=20 1.9.1 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Fri Apr 26 07:47:41 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.39 as permitted sender) client-ip=209.132.183.39; envelope-from=libvir-list-bounces@redhat.com; helo=mx6-phx2.redhat.com; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.39 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; Return-Path: Received: from mx6-phx2.redhat.com (mx6-phx2.redhat.com [209.132.183.39]) by mx.zohomail.com with SMTPS id 1488793956124775.869856460586; Mon, 6 Mar 2017 01:52:36 -0800 (PST) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by mx6-phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v269n0Ji034109; Mon, 6 Mar 2017 04:49:00 -0500 Received: from int-mx11.intmail.prod.int.phx2.redhat.com (int-mx11.intmail.prod.int.phx2.redhat.com [10.5.11.24]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id v269mlYp029138 for ; Mon, 6 Mar 2017 04:48:47 -0500 Received: from mx1.redhat.com (ext-mx06.extmail.prod.ext.phx2.redhat.com [10.5.110.30]) by int-mx11.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v269mlbJ032370 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Mon, 6 Mar 2017 04:48:47 -0500 Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id A838142BA7; Mon, 6 Mar 2017 09:48:42 +0000 (UTC) Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 06 Mar 2017 01:48:41 -0800 Received: from s2600wt.sh.intel.com ([10.239.48.158]) by fmsmga001.fm.intel.com with ESMTP; 06 Mar 2017 01:48:40 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.35,253,1484035200"; d="scan'208";a="1118950632" From: Eli Qiao To: libvir-list@redhat.com Date: Mon, 6 Mar 2017 17:50:33 +0800 Message-Id: <1488793843-50925-3-git-send-email-liyong.qiao@intel.com> In-Reply-To: <1488793843-50925-1-git-send-email-liyong.qiao@intel.com> References: <1488793843-50925-1-git-send-email-liyong.qiao@intel.com> X-Greylist: Sender passed SPF test, Sender IP whitelisted by DNSRBL, ACL 203 matched, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Mon, 06 Mar 2017 09:48:43 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Mon, 06 Mar 2017 09:48:43 +0000 (UTC) for IP:'134.134.136.20' DOMAIN:'mga02.intel.com' HELO:'mga02.intel.com' FROM:'liyong.qiao@intel.com' RCPT:'' X-RedHat-Spam-Score: -1.922 (BAYES_50, DCC_REPUT_00_12, RCVD_IN_DNSWL_MED, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, RP_MATCHES_RCVD, SPF_PASS) 134.134.136.20 mga02.intel.com 134.134.136.20 mga02.intel.com X-Scanned-By: MIMEDefang 2.68 on 10.5.11.24 X-Scanned-By: MIMEDefang 2.78 on 10.5.110.30 X-loop: libvir-list@redhat.com Cc: mkletzan@redhat.com, mtosatti@redhat.com, qiaoliyong@gmail.com Subject: [libvirt] [PATH V10 02/12] Resctrl: expose cache information to capabilities X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" This patch expose cache information to host's capabilites xml. For l3 cache allocation For l3 cache allocation supported cdp(seperate data/code): RFC on mailing list. https://www.redhat.com/archives/libvir-list/2017-January/msg00644.html Signed-off-by: Eli Qiao --- src/conf/capabilities.c | 56 ++++++++++++++++++++++++++++++++++++++ src/conf/capabilities.h | 23 ++++++++++++++++ src/libvirt_private.syms | 3 +++ src/nodeinfo.c | 64 ++++++++++++++++++++++++++++++++++++++++= ++++ src/nodeinfo.h | 1 + src/qemu/qemu_capabilities.c | 8 ++++++ src/qemu/qemu_driver.c | 4 +++ 7 files changed, 159 insertions(+) diff --git a/src/conf/capabilities.c b/src/conf/capabilities.c index 9ab343b..23e21e4 100644 --- a/src/conf/capabilities.c +++ b/src/conf/capabilities.c @@ -198,6 +198,18 @@ virCapabilitiesClearSecModel(virCapsHostSecModelPtr se= cmodel) } =20 static void +virCapabilitiesClearCacheBank(virCapsHostCacheBankPtr cachebank) +{ + size_t i; + for (i =3D 0; i < cachebank->ncontrol; i++) + VIR_FREE(cachebank->control[i].scope); + + VIR_FREE(cachebank->type); + VIR_FREE(cachebank->cpus); +} + + +static void virCapabilitiesDispose(void *object) { virCapsPtr caps =3D object; @@ -221,6 +233,10 @@ virCapabilitiesDispose(void *object) virCapabilitiesClearSecModel(&caps->host.secModels[i]); VIR_FREE(caps->host.secModels); =20 + for (i =3D 0; i < caps->host.ncachebank; i++) + virCapabilitiesClearCacheBank(caps->host.cachebank[i]); + VIR_FREE(caps->host.cachebank); + VIR_FREE(caps->host.netprefix); VIR_FREE(caps->host.pagesSize); virCPUDefFree(caps->host.cpu); @@ -844,6 +860,41 @@ virCapabilitiesFormatNUMATopology(virBufferPtr buf, return 0; } =20 +static int +virCapabilitiesFormatCache(virBufferPtr buf, + size_t ncachebank, + virCapsHostCacheBankPtr *cachebank) +{ + size_t i; + size_t j; + + virBufferAddLit(buf, "\n"); + virBufferAdjustIndent(buf, 2); + + for (i =3D 0; i < ncachebank; i++) { + virBufferAsprintf(buf, + "\n", + cachebank[i]->id, + cachebank[i]->type, + cachebank[i]->size, + cachebank[i]->cpus); + + virBufferAdjustIndent(buf, 2); + for (j =3D 0; j < cachebank[i]->ncontrol; j++) { + virBufferAsprintf(buf, + "\n", + cachebank[i]->control[j].min, + cachebank[i]->control[j].reserved, + cachebank[i]->control[j].scope); + } + virBufferAdjustIndent(buf, -2); + virBufferAddLit(buf, "\n"); + } + virBufferAdjustIndent(buf, -2); + virBufferAddLit(buf, "\n"); + return 0; +} + /** * virCapabilitiesFormatXML: * @caps: capabilities to format @@ -931,6 +982,11 @@ virCapabilitiesFormatXML(virCapsPtr caps) virBufferAddLit(&buf, "\n"); } =20 + if (caps->host.ncachebank && + virCapabilitiesFormatCache(&buf, caps->host.ncachebank, + caps->host.cachebank) < 0) + return NULL; + if (caps->host.netprefix) virBufferAsprintf(&buf, "%s\n", caps->host.netprefix); diff --git a/src/conf/capabilities.h b/src/conf/capabilities.h index cfdc34a..b446de5 100644 --- a/src/conf/capabilities.h +++ b/src/conf/capabilities.h @@ -138,6 +138,25 @@ struct _virCapsHostSecModel { virCapsHostSecModelLabelPtr labels; }; =20 +typedef struct _virCapsHostCacheControl virCapsHostCacheControl; +typedef virCapsHostCacheControl *virCapsHostCacheControlPtr; +struct _virCapsHostCacheControl { + unsigned long long min; + unsigned long long reserved; + char* scope; +}; + +typedef struct _virCapsHostCacheBank virCapsHostCacheBank; +typedef virCapsHostCacheBank *virCapsHostCacheBankPtr; +struct _virCapsHostCacheBank { + unsigned int id; + char* type; + char* cpus; + unsigned long long size; + size_t ncontrol; + virCapsHostCacheControlPtr control; +}; + typedef struct _virCapsHost virCapsHost; typedef virCapsHost *virCapsHostPtr; struct _virCapsHost { @@ -160,6 +179,10 @@ struct _virCapsHost { size_t nsecModels; virCapsHostSecModelPtr secModels; =20 + size_t ncachebank; + size_t ncachebank_max; + virCapsHostCacheBankPtr *cachebank; + char *netprefix; virCPUDefPtr cpu; int nPagesSize; /* size of pagesSize array */ diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index bb7c3ad..b8445ef 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -1125,6 +1125,7 @@ virLogManagerNew; # nodeinfo.h nodeCapsInitNUMA; nodeGetInfo; +virCapsInitCache; virHostCPUGetCount; virHostCPUGetKVMMaxVCPUs; virHostCPUGetMap; @@ -2322,8 +2323,10 @@ virRandomInt; =20 # util/virresctrl.h virResCtrlAvailable; +virResCtrlGet; virResCtrlInit; =20 + # util/virrotatingfile.h virRotatingFileReaderConsume; virRotatingFileReaderFree; diff --git a/src/nodeinfo.c b/src/nodeinfo.c index f2ded02..a001cd0 100644 --- a/src/nodeinfo.c +++ b/src/nodeinfo.c @@ -48,6 +48,7 @@ #include "virstring.h" #include "virnuma.h" #include "virlog.h" +#include "virresctrl.h" =20 #define VIR_FROM_THIS VIR_FROM_NONE =20 @@ -416,3 +417,66 @@ nodeCapsInitNUMA(virCapsPtr caps) VIR_FREE(pageinfo); return ret; } + +int +virCapsInitCache(virCapsPtr caps) +{ + size_t i, j; + virResCtrlPtr resctrl; + virCapsHostCacheBankPtr bank; + + for (i =3D 0; i < VIR_RDT_RESOURCE_LAST; i++) { + /* L3DATA and L3CODE share L3 resources */ + if (i =3D=3D VIR_RDT_RESOURCE_L3CODE) + continue; + + resctrl =3D virResCtrlGet(i); + + if (resctrl->enabled) { + for (j =3D 0; j < resctrl->num_banks; j++) { + if (VIR_RESIZE_N(caps->host.cachebank, caps->host.ncacheba= nk_max, + caps->host.ncachebank, 1) < 0) + return -1; + + if (VIR_ALLOC(bank) < 0) + return -1; + + bank->id =3D resctrl->cache_banks[j].host_id; + if (VIR_STRDUP(bank->type, resctrl->cache_level) < 0) + goto err; + if (VIR_STRDUP(bank->cpus, virBitmapFormat(resctrl->cache_= banks[j].cpu_mask)) < 0) + goto err; + bank->size =3D resctrl->cache_banks[j].cache_size; + /*L3DATA and L3CODE shares L3 cache resources, so fill the= m to the control element*/ + if (i =3D=3D VIR_RDT_RESOURCE_L3DATA) { + if (VIR_EXPAND_N(bank->control, bank->ncontrol, 2) < 0) + goto err; + + bank->control[0].min =3D virResCtrlGet(VIR_RDT_RESOURC= E_L3DATA)->cache_banks[j].cache_min; + bank->control[0].reserved =3D bank->control[0].min * (= virResCtrlGet(VIR_RDT_RESOURCE_L3DATA)->min_cbm_bits); + if (VIR_STRDUP(bank->control[0].scope, + virResCtrlGet(VIR_RDT_RESOURCE_L3DATA)->= name) < 0) + goto err; + + bank->control[1].min =3D virResCtrlGet(VIR_RDT_RESOURC= E_L3CODE)->cache_banks[j].cache_min; + bank->control[1].reserved =3D bank->control[1].min * (= virResCtrlGet(VIR_RDT_RESOURCE_L3CODE)->min_cbm_bits); + if (VIR_STRDUP(bank->control[1].scope, + virResCtrlGet(VIR_RDT_RESOURCE_L3CODE)-= >name) < 0) + goto err; + } else { + if (VIR_EXPAND_N(bank->control, bank->ncontrol, 1) < 0) + goto err; + bank->control[0].min =3D resctrl->cache_banks[j].cache= _min; + bank->control[0].reserved =3D bank->control[0].min * r= esctrl->min_cbm_bits; + if (VIR_STRDUP(bank->control[0].scope, resctrl->name) = < 0) + goto err; + } + caps->host.cachebank[caps->host.ncachebank++] =3D bank; + } + } + } + return 0; + err: + VIR_FREE(bank); + return -1; +} diff --git a/src/nodeinfo.h b/src/nodeinfo.h index 3c4dc46..5eb0f83 100644 --- a/src/nodeinfo.h +++ b/src/nodeinfo.h @@ -28,5 +28,6 @@ =20 int nodeGetInfo(virNodeInfoPtr nodeinfo); int nodeCapsInitNUMA(virCapsPtr caps); +int virCapsInitCache(virCapsPtr caps); =20 #endif /* __VIR_NODEINFO_H__*/ diff --git a/src/qemu/qemu_capabilities.c b/src/qemu/qemu_capabilities.c index 359a0d8..a0d7254 100644 --- a/src/qemu/qemu_capabilities.c +++ b/src/qemu/qemu_capabilities.c @@ -1100,6 +1100,11 @@ virQEMUCapsInitCPU(virCapsPtr caps, goto cleanup; } =20 +static int +virQEMUCapsInitCache(virCapsPtr caps) +{ + return virCapsInitCache(caps); +} =20 static int virQEMUCapsInitPages(virCapsPtr caps) @@ -1146,6 +1151,9 @@ virCapsPtr virQEMUCapsInit(virQEMUCapsCachePtr cache) if (virQEMUCapsInitCPU(caps, hostarch) < 0) VIR_WARN("Failed to get host CPU"); =20 + if (virQEMUCapsInitCache(caps) < 0) + VIR_WARN("Failed to get host cache"); + /* Add the power management features of the host */ if (virNodeSuspendGetTargetMask(&caps->host.powerMgmt) < 0) VIR_WARN("Failed to get host power management capabilities"); diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 77d8175..3bfb4ec 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -105,6 +105,7 @@ #include "vircgroup.h" #include "virperf.h" #include "virnuma.h" +#include "virresctrl.h" #include "dirname.h" #include "network/bridge_driver.h" =20 @@ -849,6 +850,9 @@ qemuStateInitialize(bool privileged, run_gid =3D cfg->group; } =20 + if (virResCtrlAvailable() && virResCtrlInit() < 0) + VIR_WARN("Faild to initialize resource control."); + qemu_driver->qemuCapsCache =3D virQEMUCapsCacheNew(cfg->libDir, cfg->cacheDir, run_uid, --=20 1.9.1 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Fri Apr 26 07:47:41 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.37 as permitted sender) client-ip=209.132.183.37; envelope-from=libvir-list-bounces@redhat.com; helo=mx5-phx2.redhat.com; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.37 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; Return-Path: Received: from mx5-phx2.redhat.com (mx5-phx2.redhat.com [209.132.183.37]) by mx.zohomail.com with SMTPS id 1488793939349810.0436564161784; Mon, 6 Mar 2017 01:52:19 -0800 (PST) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by mx5-phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v269mxhC026334; Mon, 6 Mar 2017 04:48:59 -0500 Received: from int-mx09.intmail.prod.int.phx2.redhat.com (int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id v269mka9029123 for ; Mon, 6 Mar 2017 04:48:46 -0500 Received: from mx1.redhat.com (ext-mx01.extmail.prod.ext.phx2.redhat.com [10.5.110.25]) by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v269mjxK032221 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Mon, 6 Mar 2017 04:48:45 -0500 Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 096C781F07; Mon, 6 Mar 2017 09:48:44 +0000 (UTC) Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 06 Mar 2017 01:48:42 -0800 Received: from s2600wt.sh.intel.com ([10.239.48.158]) by fmsmga001.fm.intel.com with ESMTP; 06 Mar 2017 01:48:41 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.35,253,1484035200"; d="scan'208";a="1118950634" From: Eli Qiao To: libvir-list@redhat.com Date: Mon, 6 Mar 2017 17:50:34 +0800 Message-Id: <1488793843-50925-4-git-send-email-liyong.qiao@intel.com> In-Reply-To: <1488793843-50925-1-git-send-email-liyong.qiao@intel.com> References: <1488793843-50925-1-git-send-email-liyong.qiao@intel.com> X-Greylist: Sender passed SPF test, Sender IP whitelisted by DNSRBL, ACL 203 matched, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.25]); Mon, 06 Mar 2017 09:48:44 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.25]); Mon, 06 Mar 2017 09:48:44 +0000 (UTC) for IP:'134.134.136.20' DOMAIN:'mga02.intel.com' HELO:'mga02.intel.com' FROM:'liyong.qiao@intel.com' RCPT:'' X-RedHat-Spam-Score: -1.922 (BAYES_50, DCC_REPUT_00_12, RCVD_IN_DNSWL_MED, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, RP_MATCHES_RCVD, SPF_PASS) 134.134.136.20 mga02.intel.com 134.134.136.20 mga02.intel.com X-Scanned-By: MIMEDefang 2.68 on 10.5.11.22 X-Scanned-By: MIMEDefang 2.78 on 10.5.110.25 X-loop: libvir-list@redhat.com Cc: mkletzan@redhat.com, mtosatti@redhat.com, qiaoliyong@gmail.com Subject: [libvirt] [PATH V10 03/12] Resctrl: Add new xml element to support cache tune X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" This patch adds new xml element to support cache tune as: ... ... id: any non-minus number host_id: reference of the host's cache banks id, it's from capabilities type: cache bank type size: should be multiples of the min_size of the bank on host. vcpus: cache allocation on vcpu set, if empty, will apply the allocation on all vcpus --- docs/schemas/domaincommon.rng | 46 +++++++++++++ src/conf/domain_conf.c | 152 ++++++++++++++++++++++++++++++++++++++= ++++ src/conf/domain_conf.h | 19 ++++++ src/util/virresctrl.c | 29 ++++++-- src/util/virresctrl.h | 4 +- 5 files changed, 244 insertions(+), 6 deletions(-) diff --git a/docs/schemas/domaincommon.rng b/docs/schemas/domaincommon.rng index c64544a..efc84c5 100644 --- a/docs/schemas/domaincommon.rng +++ b/docs/schemas/domaincommon.rng @@ -825,6 +825,32 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + @@ -5490,6 +5516,26 @@ -1 + + + [0-9]+ + + + + + [0-9]+ + + + + + (l3) + + + + + KiB + + diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index 97d42fe..652f4ca 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -56,6 +56,7 @@ #include "virstring.h" #include "virnetdev.h" #include "virhostdev.h" +#include "virresctrl.h" =20 #define VIR_FROM_THIS VIR_FROM_DOMAIN =20 @@ -15745,6 +15746,127 @@ virDomainVcpuPinDefParseXML(virDomainDefPtr def, return ret; } =20 +/* Parse the XML definition for cachetune + * and a cachetune has the form + * + */ +static int +virDomainCacheTuneDefParseXML(virDomainDefPtr def, + int n, + xmlNodePtr* nodes) +{ + char* tmp =3D NULL; + size_t i, j; + int type =3D -1; + virDomainCacheBankPtr bank =3D NULL; + virResCtrlPtr resctrl; + + if (VIR_ALLOC_N(bank, n) < 0) + goto cleanup; + + for (i =3D 0; i < n; i++) { + if (!(tmp =3D virXMLPropString(nodes[i], "id"))) { + virReportError(VIR_ERR_XML_ERROR, "%s", _("missing id in cache= tune")); + goto cleanup; + } + if (virStrToLong_uip(tmp, NULL, 10, &(bank[i].id)) < 0) { + virReportError(VIR_ERR_XML_ERROR, + _("invalid setting for cache id '%s'"), tmp); + goto cleanup; + } + + VIR_FREE(tmp); + if (!(tmp =3D virXMLPropString(nodes[i], "host_id"))) { + virReportError(VIR_ERR_XML_ERROR, "%s", _("missing host id in = cache tune")); + goto cleanup; + } + if (virStrToLong_uip(tmp, NULL, 10, &(bank[i].host_id)) < 0) { + virReportError(VIR_ERR_XML_ERROR, + _("invalid setting for cache host id '%s'"), tm= p); + goto cleanup; + } + VIR_FREE(tmp); + + if (!(tmp =3D virXMLPropString(nodes[i], "size"))) { + virReportError(VIR_ERR_XML_ERROR, "%s", _("missing size in cac= he tune")); + goto cleanup; + } + if (virStrToLong_ull(tmp, NULL, 10, &(bank[i].size)) < 0) { + virReportError(VIR_ERR_XML_ERROR, + _("invalid setting for cache size '%s'"), tmp); + goto cleanup; + } + VIR_FREE(tmp); + + if (!(tmp =3D virXMLPropString(nodes[i], "type"))) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("missing cache type")); + goto cleanup; + } + + if ((type =3D virResCtrlTypeFromString(tmp)) < 0) { + virReportError(VIR_ERR_XML_ERROR, + _("'unsupported cache type '%s'"), tmp); + goto cleanup; + } + + resctrl =3D virResCtrlGet(type); + + if (resctrl =3D=3D NULL || !resctrl->enabled) { + virReportError(VIR_ERR_XML_ERROR, + _("'host doesn't enabled cache type '%s'"), tmp); + goto cleanup; + } + + bool found_host_id =3D false; + /* Loop for banks to search host_id */ + for (j =3D 0; j < resctrl->num_banks; j++) { + if (resctrl->cache_banks[j].host_id =3D=3D bank[i].host_id) { + found_host_id =3D true; + break; + } + } + + if (! found_host_id) { + virReportError(VIR_ERR_XML_ERROR, + _("'cache bank's host id %u not found on the host"), + bank[i].host_id); + goto cleanup; + } + + if (bank[i].size =3D=3D 0 || + bank[i].size % resctrl->cache_banks[j].cache_min !=3D 0) { + virReportError(VIR_ERR_XML_ERROR, + _("'the size should be multiples of '%llu'"), + resctrl->cache_banks[j].cache_min); + goto cleanup; + } + + if (VIR_STRDUP(bank[i].type, tmp) < 0) + goto cleanup; + + if ((tmp =3D virXMLPropString(nodes[i], "vcpus"))) { + if (virBitmapParse(tmp, &bank[i].vcpus, + VIR_DOMAIN_CPUMASK_LEN) < 0) + goto cleanup; + + if (virBitmapIsAllClear(bank[i].vcpus)) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, + _("Invalid value of 'vcpus': %s"), tmp); + goto cleanup; + } + } + } + + def->cachetune.cache_banks =3D bank; + def->cachetune.n_banks =3D n; + return 0; + + cleanup: + VIR_FREE(bank); + VIR_FREE(tmp); + return -1; +} =20 /* Parse the XML definition for a iothreadpin * and an iothreadspin has the form @@ -17033,6 +17155,14 @@ virDomainDefParseXML(xmlDocPtr xml, } VIR_FREE(nodes); =20 + if ((n =3D virXPathNodeSet("./cputune/cachetune", ctxt, &nodes)) < 0) + goto error; + + if (virDomainCacheTuneDefParseXML(def, n, nodes) < 0) + goto error; + + VIR_FREE(nodes); + if ((n =3D virXPathNodeSet("./cputune/emulatorpin", ctxt, &nodes)) < 0= ) { virReportError(VIR_ERR_INTERNAL_ERROR, "%s", _("cannot extract emulatorpin nodes")); @@ -23554,6 +23684,26 @@ virDomainSchedulerFormat(virBufferPtr buf, =20 } =20 +static void +virDomainCacheTuneDefFormat(virBufferPtr buf, + virDomainCachetunePtr cache) +{ + size_t i; + for (i =3D 0; i < cache->n_banks; i ++) { + virBufferAsprintf(buf, "cache_banks[i].id, + cache->cache_banks[i].host_id, + cache->cache_banks[i].type, + cache->cache_banks[i].size); + + if (cache->cache_banks[i].vcpus) + virBufferAsprintf(buf, " vcpus=3D'%s'/>\n", + virBitmapFormat(cache->cache_banks[i].vcpus)); + else + virBufferAddLit(buf, "/>\n"); + } +} =20 static int virDomainCputuneDefFormat(virBufferPtr buf, @@ -23617,6 +23767,8 @@ virDomainCputuneDefFormat(virBufferPtr buf, VIR_FREE(cpumask); } =20 + virDomainCacheTuneDefFormat(&childrenBuf, &def->cachetune); + if (def->cputune.emulatorpin) { char *cpumask; virBufferAddLit(&childrenBuf, " Received: from mx4-phx2.redhat.com (mx4-phx2.redhat.com [209.132.183.25]) by mx.zohomail.com with SMTPS id 1488793939223726.8480998355676; Mon, 6 Mar 2017 01:52:19 -0800 (PST) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by mx4-phx2.redhat.com (8.13.8/8.13.8) with ESMTP id v269mm83010808; Mon, 6 Mar 2017 04:48:48 -0500 Received: from smtp.corp.redhat.com (int-mx16.intmail.prod.int.phx2.redhat.com [10.5.11.28]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id v269mkIx029128 for ; Mon, 6 Mar 2017 04:48:46 -0500 Received: by smtp.corp.redhat.com (Postfix) id E5A981DCC0; Mon, 6 Mar 2017 09:48:46 +0000 (UTC) Received: from mx1.redhat.com (ext-mx06.extmail.prod.ext.phx2.redhat.com [10.5.110.30]) by smtp.corp.redhat.com (Postfix) with ESMTPS id D65F62D654; Mon, 6 Mar 2017 09:48:46 +0000 (UTC) Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 0E73942BBA; Mon, 6 Mar 2017 09:48:45 +0000 (UTC) Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 06 Mar 2017 01:48:43 -0800 Received: from s2600wt.sh.intel.com ([10.239.48.158]) by fmsmga001.fm.intel.com with ESMTP; 06 Mar 2017 01:48:42 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.35,253,1484035200"; d="scan'208";a="1118950639" From: Eli Qiao To: libvir-list@redhat.com Date: Mon, 6 Mar 2017 17:50:35 +0800 Message-Id: <1488793843-50925-5-git-send-email-liyong.qiao@intel.com> In-Reply-To: <1488793843-50925-1-git-send-email-liyong.qiao@intel.com> References: <1488793843-50925-1-git-send-email-liyong.qiao@intel.com> X-Greylist: Sender passed SPF test, Sender IP whitelisted by DNSRBL, ACL 203 matched, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Mon, 06 Mar 2017 09:48:45 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Mon, 06 Mar 2017 09:48:45 +0000 (UTC) for IP:'134.134.136.20' DOMAIN:'mga02.intel.com' HELO:'mga02.intel.com' FROM:'liyong.qiao@intel.com' RCPT:'' X-RedHat-Spam-Score: -1.922 (BAYES_50, DCC_REPUT_00_12, RCVD_IN_DNSWL_MED, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, RP_MATCHES_RCVD, SPF_PASS) 134.134.136.20 mga02.intel.com 134.134.136.20 mga02.intel.com X-Scanned-By: MIMEDefang 2.78 on 10.5.110.30 X-Scanned-By: MIMEDefang 2.74 on 10.5.11.28 X-loop: libvir-list@redhat.com Cc: mkletzan@redhat.com, mtosatti@redhat.com, qiaoliyong@gmail.com Subject: [libvirt] [PATH V10 04/12] Resctrl: Add private interfaces to operate cache bank X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" virResCtrlSetCacheBanks: Set cache banks of a libvirt domain. It will create new resource domain under `/sys/fs/resctrl` and fill the schemata according the cache bank configration. virResCtrlUpdate: Destroy resctrl domain directory, and update the schemata after libvirt domain destroy. Signed-off-by: Eli Qiao --- src/libvirt_private.syms | 6 +- src/util/virresctrl.c | 677 +++++++++++++++++++++++++++++++++++++++++++= ++++ src/util/virresctrl.h | 5 +- 3 files changed, 686 insertions(+), 2 deletions(-) diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index b8445ef..9cfffb8 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -2325,7 +2325,11 @@ virRandomInt; virResCtrlAvailable; virResCtrlGet; virResCtrlInit; - +virResCtrlSetCacheBanks; +virResCtrlTypeFromString; +virResCtrlTypeToString; +virResCtrlUpdate; +# =20 # util/virrotatingfile.h virRotatingFileReaderConsume; diff --git a/src/util/virresctrl.c b/src/util/virresctrl.c index eee6675..43af0f5 100644 --- a/src/util/virresctrl.c +++ b/src/util/virresctrl.c @@ -19,6 +19,8 @@ * Eli Qiao */ #include +#include +#include =20 #include "virresctrl.h" #include "viralloc.h" @@ -61,8 +63,58 @@ do { \ =20 #define VIR_RESCTRL_GET_SCHEMATA(count) ((1 << count) - 1) =20 +/** + * a virResSchemata represents a schemata object under a resource control + * domain. + */ +typedef struct _virResSchemataItem virResSchemataItem; +typedef virResSchemataItem *virResSchemataItemPtr; +struct _virResSchemataItem { + unsigned int socket_no; + unsigned schemata; +}; + +typedef struct _virResSchemata virResSchemata; +typedef virResSchemata *virResSchemataPtr; +struct _virResSchemata { + unsigned int n_schemata_items; + virResSchemataItemPtr schemata_items; +}; + +/** + * a virResDomain represents a resource control domain. It's a double link= ed + * list. + */ + +typedef struct _virResDomain virResDomain; +typedef virResDomain *virResDomainPtr; + +struct _virResDomain { + char *name; + virResSchemataPtr schematas[VIR_RDT_RESOURCE_LAST]; + char **tasks; + size_t n_tasks; + size_t n_sockets; + virResDomainPtr pre; + virResDomainPtr next; +}; + +/* All resource control domains on this host*/ +typedef struct _virResCtrlDomain virResCtrlDomain; +typedef virResCtrlDomain *virResCtrlDomainPtr; + +struct _virResCtrlDomain { + unsigned int num_domains; + virResDomainPtr domains; +}; + static unsigned int host_id; =20 +/* Global static struct to be maintained which is a interface */ +static virResCtrlDomain domainall; + +/* Global static struct array to be maintained which indicate + * resource status on a host */ static virResCtrl resctrlall[] =3D { { .name =3D "L3", @@ -82,6 +134,78 @@ static virResCtrl resctrlall[] =3D { }, }; =20 +/* + * How many bits is set in schemata + * eg: + * virResCtrlBitsNum(10110) =3D 2 */ +static int virResCtrlBitsContinuesNum(unsigned schemata) +{ + size_t i; + int ret =3D 0; + for (i =3D 0; i < MAX_CBM_BIT_LEN; i ++) { + if ((schemata & 0x1) =3D=3D 0x1) + ret++; + else + if (ret > 0 || schemata =3D=3D 0) break; + + schemata =3D schemata >> 1; + } + return ret; +} + +/* Position of the highest continue 1 bit of in schemata + * eg: + * virResctrlBitsContinuesPos(10110) =3D 3 */ +static int virResCtrlBitsContinuesPos(unsigned schemata) +{ + size_t i; + int flag =3D 0; + for (i =3D 0; i < MAX_CBM_BIT_LEN; i ++) { + if ((schemata & 0x1) =3D=3D 0x0 && flag =3D=3D 1) + return i; + else if ((schemata & 0x1) =3D=3D 0x1) flag =3D 1; + + schemata =3D schemata >> 1; + } + return 0; +} + +static int virResCtrlGetStr(const char *domain_name, const char *item_name= , char **ret) +{ + char *path; + int rc =3D 0; + + CONSTRUCT_RESCTRL_PATH(domain_name, item_name); + + if (virFileReadAll(path, MAX_FILE_LEN, ret) < 0) { + rc =3D -1; + goto cleanup; + } + + cleanup: + VIR_FREE(path); + return rc; +} + +static int virResCtrlGetSchemata(const int type, const char *name, char **= schemata) +{ + int rc; + char *tmp, *end; + char *buf; + + if ((rc =3D virResCtrlGetStr(name, "schemata", &buf)) < 0) + return rc; + + tmp =3D strstr(buf, resctrlall[type].name); + end =3D strchr(tmp, '\n'); + *end =3D '\0'; + if (VIR_STRDUP(*schemata, tmp) < 0) + rc =3D -1; + + VIR_FREE(buf); + return rc; +} + static int virResCtrlGetInfoStr(const int type, const char *item, char **s= tr) { int ret =3D 0; @@ -102,6 +226,71 @@ static int virResCtrlGetInfoStr(const int type, const = char *item, char **str) return ret; } =20 +/* Return pointer of and ncount of schemata*/ +static virResSchemataPtr virParseSchemata(const char *schemata_str, size_t= *ncount) +{ + const char *p, *q; + int pos; + int ischemata; + virResSchemataPtr schemata; + virResSchemataItemPtr schemataitems, tmpitem; + unsigned int socket_no =3D 0; + char *tmp; + + if (VIR_ALLOC(schemata) < 0) + goto cleanup; + + p =3D q =3D schemata_str; + pos =3D strchr(schemata_str, ':') - p; + + /* calculate cpu socket count */ + *ncount =3D 1; + while ((q =3D strchr(p, ';')) !=3D 0) { + p =3D q + 1; + (*ncount)++; + } + + /* allocat an arrry to store schemata for each socket*/ + if (VIR_ALLOC_N_QUIET(tmpitem, *ncount) < 0) + goto cleanup; + + schemataitems =3D tmpitem; + + p =3D q =3D schemata_str + pos + 1; + + while (*p !=3D '\0') { + if (*p =3D=3D '=3D') { + q =3D p + 1; + + tmpitem->socket_no =3D socket_no++; + + while (*p !=3D ';' && *p !=3D '\0') p++; + + if (VIR_STRNDUP(tmp, q, p-q) < 0) + goto cleanup; + + if (virStrToLong_i(tmp, NULL, 16, &ischemata) < 0) + goto cleanup; + + VIR_FREE(tmp); + tmp =3D NULL; + tmpitem->schemata =3D ischemata; + tmpitem ++; + schemata->n_schemata_items +=3D 1; + } + p++; + } + + schemata->schemata_items =3D schemataitems; + return schemata; + + cleanup: + VIR_FREE(schemata); + VIR_FREE(tmpitem); + return NULL; +} + + static int virResCtrlReadConfig(virArch arch, int type) { int ret; @@ -165,6 +354,488 @@ static int virResCtrlReadConfig(virArch arch, int typ= e) return ret; } =20 +/* Remove the Domain from sysfs, this should only success no pids in tasks + * of a partition. + */ +static +int virResCtrlRemoveDomain(const char *name) +{ + char *path =3D NULL; + int rc =3D 0; + + if ((rc =3D virAsprintf(&path, "%s/%s", RESCTRL_DIR, name)) < 0) + return rc; + rc =3D rmdir(path); + VIR_FREE(path); + return rc; +} + +static +int virResCtrlDestroyDomain(virResDomainPtr p) +{ + size_t i; + int rc; + if ((rc =3D virResCtrlRemoveDomain(p->name)) < 0) + VIR_WARN("Failed to removed partition %s", p->name); + + VIR_FREE(p->name); + p->name =3D NULL; + for (i =3D 0; i < p->n_tasks; i ++) + VIR_FREE(p->tasks[i]); + VIR_FREE(p); + p =3D NULL; + return rc; +} + + +/* assemble schemata string*/ +static +char* virResCtrlAssembleSchemata(virResSchemataPtr schemata, int type) +{ + virBuffer buf =3D VIR_BUFFER_INITIALIZER; + size_t i; + + virBufferAsprintf(&buf, "%s:%u=3D%x", resctrlall[type].name, + schemata->schemata_items[0].socket_no, + schemata->schemata_items[0].schemata); + + for (i =3D 1; i < schemata->n_schemata_items; i++) { + virBufferAsprintf(&buf, ";%u=3D%x", + schemata->schemata_items[i].socket_no, + schemata->schemata_items[i].schemata); + } + + return virBufferContentAndReset(&buf); +} + +/* Refresh default domains' schemata + */ +static +int virResCtrlRefreshSchemata(void) +{ + size_t i, j, k; + unsigned int tmp_schemata; + unsigned int default_schemata; + unsigned int min_schemata; + + virResDomainPtr header, p; + + header =3D domainall.domains; + + if (!header) + return 0; + + for (i =3D 0; i < VIR_RDT_RESOURCE_LAST; i++) { + if (VIR_RESCTRL_ENABLED(i)) { + min_schemata =3D VIR_RESCTRL_GET_SCHEMATA(resctrlall[i].min_cb= m_bits); + + for (j =3D 0; j < header->schematas[i]->n_schemata_items; j ++= ) { + p =3D header->next; + // Reset to default schemata 0xfffff + default_schemata =3D VIR_RESCTRL_GET_SCHEMATA(resctrlall[i= ].cbm_len); + tmp_schemata =3D 0; + /* NOTEs: if only header domain, the schemata will be set = to default one*/ + for (k =3D 1; k < domainall.num_domains; k++) { + if (p->schematas[i]->schemata_items[j].schemata > min_= schemata) + tmp_schemata |=3D p->schematas[i]->schemata_items[= j].schemata; + p =3D p->next; + } + + default_schemata ^=3D tmp_schemata; + + default_schemata &=3D VIR_RESCTRL_GET_SCHEMATA(virResCtrlB= itsContinuesPos(default_schemata)); + header->schematas[i]->schemata_items[j].schemata =3D defau= lt_schemata; + int bitsnum =3D virResCtrlBitsContinuesNum(default_schemat= a); + resctrlall[i].cache_banks[j].cache_left =3D + (bitsnum - resctrlall[i].min_cbm_bits) * resctrlall[i]= .cache_banks[j].cache_min; + } + } + } + + return 0; + +} + +/* Get a domain ptr by domain's name*/ +static +virResDomainPtr virResCtrlGetDomain(const char* name) { + size_t i; + virResDomainPtr p =3D domainall.domains; + for (i =3D 0; i < domainall.num_domains; i++) { + if ((p->name) && STREQ(name, p->name)) + return p; + p =3D p->next; + } + return NULL; +} + +static int +virResCtrlAddTask(virResDomainPtr dom, pid_t pid) +{ + size_t maxtasks; + + if (VIR_RESIZE_N(dom->tasks, maxtasks, dom->n_tasks + 1, 1) < 0) + return -1; + + if (virAsprintf(&(dom->tasks[dom->n_tasks]), "%llu", (long long)pid) <= 0) + return -1; + + dom->n_tasks +=3D 1; + return 0; +} + +static int +virResCtrlWrite(const char *name, const char *item, const char *content) +{ + char *path; + int writefd; + int rc =3D -1; + + CONSTRUCT_RESCTRL_PATH(name, item); + + if (!virFileExists(path)) + goto cleanup; + + if ((writefd =3D open(path, O_WRONLY | O_APPEND, S_IRUSR | S_IWUSR)) <= 0) + goto cleanup; + + if (safewrite(writefd, content, strlen(content)) < 0) + goto cleanup; + + rc =3D 0; + cleanup: + VIR_FREE(path); + VIR_FORCE_CLOSE(writefd); + return rc; +} + +/* if name =3D=3D NULL we load default schemata */ +static +virResDomainPtr virResCtrlLoadDomain(const char *name) +{ + char *schematas; + virResDomainPtr p; + size_t i; + + if (VIR_ALLOC(p) < 0) + goto cleanup; + + for (i =3D 0; i < VIR_RDT_RESOURCE_LAST; i++) { + if (VIR_RESCTRL_ENABLED(i)) { + if (virResCtrlGetSchemata(i, name, &schematas) < 0) + goto cleanup; + p->schematas[i] =3D virParseSchemata(schematas, &(p->n_sockets= )); + VIR_FREE(schematas); + } + } + + p->tasks =3D NULL; + p->n_tasks =3D 0; + + if ((name !=3D NULL) && (VIR_STRDUP(p->name, name)) < 0) + goto cleanup; + + return p; + + cleanup: + VIR_FREE(p); + return NULL; +} + +static +virResDomainPtr virResCtrlCreateDomain(const char *name) +{ + char *path; + mode_t mode =3D 0755; + virResDomainPtr p; + size_t i, j; + + if (virAsprintf(&path, "%s/%s", RESCTRL_DIR, name) < 0) + return NULL; + + if (virDirCreate(path, mode, 0, 0, 0) < 0) + goto cleanup; + + if ((p =3D virResCtrlLoadDomain(name)) =3D=3D NULL) + return p; + + /* sys fs doens't let us use 0. + * reset schemata to min_bits*/ + for (i =3D 0; i < VIR_RDT_RESOURCE_LAST; i++) { + if (VIR_RESCTRL_ENABLED(i)) { + int min_bits =3D VIR_RESCTRL_GET_SCHEMATA(resctrlall[i].min_c= bm_bits); + for (j =3D 0; j < p->n_sockets; j++) + p->schematas[i]->schemata_items[j].schemata =3D min_bits; + } + } + + VIR_FREE(path); + return p; + + cleanup: + VIR_FREE(path); + return NULL; +} + +/* flush domains's information to sysfs*/ +static int +virResCtrlFlushDomainToSysfs(virResDomainPtr dom) +{ + size_t i; + char* schemata; + char* tmp; + int rc =3D -1; + virBuffer buf =3D VIR_BUFFER_INITIALIZER; + + for (i =3D 0; i < VIR_RDT_RESOURCE_LAST; i++) { + if (VIR_RESCTRL_ENABLED(i)) { + tmp =3D virResCtrlAssembleSchemata(dom->schematas[i], i); + virBufferAsprintf(&buf, "%s\n", tmp); + VIR_FREE(tmp); + } + } + + schemata =3D virBufferContentAndReset(&buf); + + if (virResCtrlWrite(dom->name, "schemata", schemata) < 0) + goto cleanup; + + if (dom->n_tasks > 0) { + for (i =3D 0; i < dom->n_tasks; i++) { + if (virResCtrlWrite(dom->name, "tasks", dom->tasks[i]) < 0) + goto cleanup; + } + } + + rc =3D 0; + + cleanup: + VIR_FREE(schemata); + return rc; +} + +static virResDomainPtr virResCtrlGetAllDomains(unsigned int *len) +{ + struct dirent *ent; + DIR *dp =3D NULL; + int direrr; + + *len =3D 0; + virResDomainPtr header, tmp, tmp_pre; + header =3D tmp =3D tmp_pre =3D NULL; + if (virDirOpenQuiet(&dp, RESCTRL_DIR) < 0) { + if (errno =3D=3D ENOENT) + return NULL; + VIR_ERROR(_("Unable to open %s (%d)"), RESCTRL_DIR, errno); + goto cleanup; + } + + header =3D virResCtrlLoadDomain(NULL); + if (header =3D=3D NULL) + goto cleanup; + + header->next =3D NULL; + + *len =3D 1; + + while ((direrr =3D virDirRead(dp, &ent, NULL)) > 0) { + if ((ent->d_type !=3D DT_DIR) || STREQ(ent->d_name, "info")) + continue; + + tmp =3D virResCtrlLoadDomain(ent->d_name); + if (tmp =3D=3D NULL) + goto cleanup; + + tmp->next =3D NULL; + + if (header->next =3D=3D NULL) + header->next =3D tmp; + + if (tmp_pre =3D=3D NULL) { + tmp->pre =3D header; + } else { + tmp->pre =3D tmp_pre; + tmp_pre->next =3D tmp; + } + + tmp_pre =3D tmp; + (*len) ++; + } + return header; + + cleanup: + VIR_DIR_CLOSE(dp); + tmp_pre =3D tmp =3D header; + while (tmp) { + tmp_pre =3D tmp; + tmp =3D tmp->next; + VIR_FREE(tmp_pre); + } + return NULL; +} + +static int +virResCtrlAppendDomain(virResDomainPtr dom) +{ + virResDomainPtr p =3D domainall.domains; + while (p->next !=3D NULL) p =3D p->next; + p->next =3D dom; + dom->pre =3D p; + domainall.num_domains +=3D 1; + return 0; +} + +static int +virResCtrlGetSocketIdByHostID(int type, unsigned int hostid) +{ + size_t i; + for (i =3D 0; i < resctrlall[type].num_banks; i++) { + if (resctrlall[type].cache_banks[i].host_id =3D=3D hostid) + return i; + } + return -1; +} + +static int +virResCtrlCalculateSchemata(int type, + int sid, + unsigned hostid, + unsigned long long size) +{ + size_t i; + int count; + virResDomainPtr p; + unsigned int tmp_schemata; + unsigned int schemata_sum =3D 0; + + if (resctrlall[type].cache_banks[sid].cache_left < size) { + VIR_ERROR(_("Not enough cache left on bank %u"), hostid); + return -1; + } + if ((count =3D size / resctrlall[type].cache_banks[sid].cache_min) <= =3D 0) { + VIR_ERROR(_("Error cache size %llu"), size); + return -1; + } + + tmp_schemata =3D VIR_RESCTRL_GET_SCHEMATA(count); + + p =3D domainall.domains; + p =3D p->next; + for (i =3D 1; i < domainall.num_domains; i ++) { + schemata_sum |=3D p->schematas[type]->schemata_items[sid].schemata; + p =3D p->next; + } + + tmp_schemata =3D tmp_schemata << (resctrlall[type].cbm_len - count); + + while ((tmp_schemata & schemata_sum) !=3D 0) + tmp_schemata =3D tmp_schemata >> 1; + return tmp_schemata; +} + +int virResCtrlSetCacheBanks(virDomainCachetunePtr cachetune, + unsigned char *uuid, pid_t *pids, int npid) +{ + size_t i; + char name[VIR_UUID_STRING_BUFLEN]; + virResDomainPtr p; + int type; + int sid; + int schemata; + + virUUIDFormat(uuid, name); + + for (i =3D 0; i < cachetune->n_banks; i++) { + VIR_DEBUG("cache_banks %u, %u, %llu, %s", + cachetune->cache_banks[i].id, + cachetune->cache_banks[i].host_id, + cachetune->cache_banks[i].size, + cachetune->cache_banks[i].type); + } + + if (cachetune->n_banks < 1) + return 0; + + p =3D virResCtrlGetDomain(name); + if (p =3D=3D NULL) { + VIR_DEBUG("no domain name %s found, create new one!", name); + p =3D virResCtrlCreateDomain(name); + } + + if (p !=3D NULL) { + for (i =3D 0; i < cachetune->n_banks; i++) { + if ((type =3D virResCtrlTypeFromString( + cachetune->cache_banks[i].type)) < 0) { + VIR_WARN("Ignore unknown cache type %s.", + cachetune->cache_banks[i].type); + continue; + } + + if ((sid =3D virResCtrlGetSocketIdByHostID( + type, cachetune->cache_banks[i].host_id)) < 0)= { + VIR_WARN("Can not find cache bank host id %u.", + cachetune->cache_banks[i].host_id); + continue; + } + + if ((schemata =3D virResCtrlCalculateSchemata( + type, sid, cachetune->cache_banks[i].host_id, + cachetune->cache_banks[i].size)) < 0) { + VIR_WARN("Failed to set schemata for cache bank id %u", + cachetune->cache_banks[i].id); + continue; + } + + p->schematas[type]->schemata_items[sid].schemata =3D schemata; + } + + for (i =3D 0; i < npid; i++) + virResCtrlAddTask(p, pids[i]); + + if (virResCtrlFlushDomainToSysfs(p) < 0) { + VIR_ERROR(_("failed to flush domain %s to sysfs"), name); + virResCtrlDestroyDomain(p); + return -1; + } + virResCtrlAppendDomain(p); + } else { + VIR_ERROR(_("Failed to create a domain in sysfs")); + return -1; + } + + virResCtrlRefreshSchemata(); + /* after refresh, flush header's schemata changes to sys fs */ + if (virResCtrlFlushDomainToSysfs(domainall.domains) < 0) + VIR_WARN("failed to flush domain to sysfs"); + + return 0; +} + +/* Should be called after pid disappeared, we recalculate + * schemata of default and flush it to sys fs. + */ +int virResCtrlUpdate(unsigned char *uuid) +{ + char name[VIR_UUID_STRING_BUFLEN]; + virResDomainPtr del; + + virUUIDFormat(uuid, name); + + del =3D virResCtrlGetDomain(name); + + if (del !=3D NULL) { + del->pre->next =3D del->next; + if (del->next !=3D NULL) + del->next->pre =3D del->pre; + virResCtrlDestroyDomain(del); + domainall.num_domains -=3D 1; + virResCtrlRefreshSchemata(); + if (virResCtrlFlushDomainToSysfs(domainall.domains) < 0) + VIR_WARN("failed to flush domain to sysfs"); + } + return 0; +} + int virResCtrlInit(void) { @@ -193,6 +864,12 @@ virResCtrlInit(void) VIR_FREE(tmp); } =20 + domainall.domains =3D virResCtrlGetAllDomains(&(domainall.num_domains)= ); + + if ((rc =3D virResCtrlRefreshSchemata()) < 0) + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("Failed to refresh resource control")); + cleanup: VIR_FREE(tmp); return rc; diff --git a/src/util/virresctrl.h b/src/util/virresctrl.h index 074d307..3a6fb95 100644 --- a/src/util/virresctrl.h +++ b/src/util/virresctrl.h @@ -26,6 +26,7 @@ =20 # include "virbitmap.h" # include "virutil.h" +# include "conf/domain_conf.h" =20 enum { VIR_RDT_RESOURCE_L3, @@ -76,5 +77,7 @@ struct _virResCtrl { bool virResCtrlAvailable(void); int virResCtrlInit(void); virResCtrlPtr virResCtrlGet(int); - +int virResCtrlSetCacheBanks(virDomainCachetunePtr, + unsigned char *, pid_t *, int); +int virResCtrlUpdate(unsigned char *); #endif --=20 1.9.1 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Fri Apr 26 07:47:41 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.39 as permitted sender) client-ip=209.132.183.39; envelope-from=libvir-list-bounces@redhat.com; helo=mx6-phx2.redhat.com; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.39 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; Return-Path: Received: from mx6-phx2.redhat.com (mx6-phx2.redhat.com [209.132.183.39]) by mx.zohomail.com with SMTPS id 1488793950633701.5833535271765; Mon, 6 Mar 2017 01:52:30 -0800 (PST) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by mx6-phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v269n1oC034121; Mon, 6 Mar 2017 04:49:01 -0500 Received: from smtp.corp.redhat.com (int-mx16.intmail.prod.int.phx2.redhat.com [10.5.11.28]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id v269mliE029143 for ; Mon, 6 Mar 2017 04:48:47 -0500 Received: by smtp.corp.redhat.com (Postfix) id E3FE41DCC3; Mon, 6 Mar 2017 09:48:47 +0000 (UTC) Received: from mx1.redhat.com (ext-mx02.extmail.prod.ext.phx2.redhat.com [10.5.110.26]) by smtp.corp.redhat.com (Postfix) with ESMTPS id D86FA2D655; Mon, 6 Mar 2017 09:48:47 +0000 (UTC) Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id A81D2883B1; Mon, 6 Mar 2017 09:48:46 +0000 (UTC) Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 06 Mar 2017 01:48:45 -0800 Received: from s2600wt.sh.intel.com ([10.239.48.158]) by fmsmga001.fm.intel.com with ESMTP; 06 Mar 2017 01:48:43 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.35,253,1484035200"; d="scan'208";a="1118950645" From: Eli Qiao To: libvir-list@redhat.com Date: Mon, 6 Mar 2017 17:50:36 +0800 Message-Id: <1488793843-50925-6-git-send-email-liyong.qiao@intel.com> In-Reply-To: <1488793843-50925-1-git-send-email-liyong.qiao@intel.com> References: <1488793843-50925-1-git-send-email-liyong.qiao@intel.com> X-Greylist: Sender passed SPF test, Sender IP whitelisted by DNSRBL, ACL 203 matched, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.26]); Mon, 06 Mar 2017 09:48:47 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.26]); Mon, 06 Mar 2017 09:48:47 +0000 (UTC) for IP:'134.134.136.20' DOMAIN:'mga02.intel.com' HELO:'mga02.intel.com' FROM:'liyong.qiao@intel.com' RCPT:'' X-RedHat-Spam-Score: -1.922 (BAYES_50, DCC_REPUT_00_12, RCVD_IN_DNSWL_MED, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, RP_MATCHES_RCVD, SPF_PASS) 134.134.136.20 mga02.intel.com 134.134.136.20 mga02.intel.com X-Scanned-By: MIMEDefang 2.78 on 10.5.110.26 X-Scanned-By: MIMEDefang 2.74 on 10.5.11.28 X-loop: libvir-list@redhat.com Cc: mkletzan@redhat.com, mtosatti@redhat.com, qiaoliyong@gmail.com Subject: [libvirt] [PATH V10 05/12] Qemu: Set cache tune while booting a new domain. X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" While user can assign some specific vcpus list in , adds the vcpus' pids to cache bank, else vm->pid will be added to cache bank. Signed-off-by: Eli Qiao --- src/qemu/qemu_driver.c | 6 ++++-- src/qemu/qemu_process.c | 54 +++++++++++++++++++++++++++++++++++++++++++++= ++++ 2 files changed, 58 insertions(+), 2 deletions(-) diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 3bfb4ec..0f11ae2 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -850,8 +850,10 @@ qemuStateInitialize(bool privileged, run_gid =3D cfg->group; } =20 - if (virResCtrlAvailable() && virResCtrlInit() < 0) - VIR_WARN("Faild to initialize resource control."); + if (virResCtrlAvailable() && virResCtrlInit() < 0) { + VIR_ERROR(_("Faild to initialize resource control")); + goto error; + } =20 qemu_driver->qemuCapsCache =3D virQEMUCapsCacheNew(cfg->libDir, cfg->cacheDir, diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index 68378c9..5935502 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -76,6 +76,7 @@ #include "configmake.h" #include "nwfilter_conf.h" #include "netdev_bandwidth_conf.h" +#include "virresctrl.h" =20 #define VIR_FROM_THIS VIR_FROM_QEMU =20 @@ -5055,6 +5056,50 @@ qemuProcessVcpusSortOrder(const void *a, return vcpua->order - vcpub->order; } =20 +static int +qemuProcessSetCacheBanks(virDomainObjPtr vm) +{ + size_t i, j; + virDomainCachetunePtr cachetune; + unsigned int max_vcpus =3D virDomainDefGetVcpusMax(vm->def); + pid_t *pids =3D NULL; + virDomainVcpuDefPtr vcpu; + size_t npid =3D 0; + size_t count =3D 0; + int ret =3D -1; + + cachetune =3D &(vm->def->cachetune); + + for (i =3D 0; i < cachetune->n_banks; i++) { + if (cachetune->cache_banks[i].vcpus) { + for (j =3D 0; j < max_vcpus; j++) { + if (virBitmapIsBitSet(cachetune->cache_banks[i].vcpus, j))= { + + vcpu =3D virDomainDefGetVcpu(vm->def, j); + if (!vcpu->online) + continue; + + if (VIR_RESIZE_N(pids, npid, count, 1) < 0) + goto cleanup; + pids[count ++] =3D qemuDomainGetVcpuPid(vm, j); + } + } + } + } + + /* If not specific vcpus in cachetune, add vm->pid */ + if (pids =3D=3D NULL) { + if (VIR_ALLOC_N(pids, 1) < 0) + goto cleanup; + pids[0] =3D vm->pid; + count =3D 1; + } + ret =3D virResCtrlSetCacheBanks(cachetune, vm->def->uuid, pids, count); + + cleanup: + VIR_FREE(pids); + return ret; +} =20 static int qemuProcessSetupHotpluggableVcpus(virQEMUDriverPtr driver, @@ -5749,6 +5794,11 @@ qemuProcessLaunch(virConnectPtr conn, qemuProcessAutoDestroyAdd(driver, vm, conn) < 0) goto cleanup; =20 + VIR_DEBUG("Cache allocation"); + + if (virResCtrlAvailable() && qemuProcessSetCacheBanks(vm) < 0) + goto cleanup; + ret =3D 0; =20 cleanup: @@ -6251,6 +6301,10 @@ void qemuProcessStop(virQEMUDriverPtr driver, virPerfFree(priv->perf); priv->perf =3D NULL; =20 + if (virResCtrlAvailable() && virResCtrlUpdate(vm->def->uuid) < 0) + VIR_WARN("Failed to update resource control for %s", + vm->def->name); + qemuProcessRemoveDomainStatus(driver, vm); =20 /* Remove VNC and Spice ports from port reservation bitmap, but only if --=20 1.9.1 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Fri Apr 26 07:47:41 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.39 as permitted sender) client-ip=209.132.183.39; envelope-from=libvir-list-bounces@redhat.com; helo=mx6-phx2.redhat.com; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.39 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; Return-Path: Received: from mx6-phx2.redhat.com (mx6-phx2.redhat.com [209.132.183.39]) by mx.zohomail.com with SMTPS id 1488793950913477.49509298313035; Mon, 6 Mar 2017 01:52:30 -0800 (PST) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by mx6-phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v269nRCO034160; Mon, 6 Mar 2017 04:49:27 -0500 Received: from int-mx10.intmail.prod.int.phx2.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id v269mmPQ029156 for ; Mon, 6 Mar 2017 04:48:48 -0500 Received: from mx1.redhat.com (ext-mx06.extmail.prod.ext.phx2.redhat.com [10.5.110.30]) by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v269mmfs006861 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Mon, 6 Mar 2017 04:48:48 -0500 Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 886CB369C3; Mon, 6 Mar 2017 09:48:47 +0000 (UTC) Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 06 Mar 2017 01:48:46 -0800 Received: from s2600wt.sh.intel.com ([10.239.48.158]) by fmsmga001.fm.intel.com with ESMTP; 06 Mar 2017 01:48:45 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.35,253,1484035200"; d="scan'208";a="1118950653" From: Eli Qiao To: libvir-list@redhat.com Date: Mon, 6 Mar 2017 17:50:37 +0800 Message-Id: <1488793843-50925-7-git-send-email-liyong.qiao@intel.com> In-Reply-To: <1488793843-50925-1-git-send-email-liyong.qiao@intel.com> References: <1488793843-50925-1-git-send-email-liyong.qiao@intel.com> X-Greylist: Sender passed SPF test, Sender IP whitelisted by DNSRBL, ACL 203 matched, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Mon, 06 Mar 2017 09:48:48 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Mon, 06 Mar 2017 09:48:48 +0000 (UTC) for IP:'134.134.136.20' DOMAIN:'mga02.intel.com' HELO:'mga02.intel.com' FROM:'liyong.qiao@intel.com' RCPT:'' X-RedHat-Spam-Score: -1.922 (BAYES_50, DCC_REPUT_00_12, RCVD_IN_DNSWL_MED, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, RP_MATCHES_RCVD, SPF_PASS) 134.134.136.20 mga02.intel.com 134.134.136.20 mga02.intel.com X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23 X-Scanned-By: MIMEDefang 2.78 on 10.5.110.30 X-loop: libvir-list@redhat.com Cc: mkletzan@redhat.com, mtosatti@redhat.com, qiaoliyong@gmail.com Subject: [libvirt] [PATH V10 06/12] Resctrl: enable l3code/l3data X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" Enable l3code/l3data while doing cache tune. l3code/l3data should use a continus cbm in their seperated schemata and the cache size are shared between them, so we need to deal them differently with l3 cache. This should enable cdp feature while mounting /sys/fs/resctrl, eg: mount -t resctrl resctrl -o cdp /sys/fs/resctrl Signed-off-by: Eli Qiao --- docs/schemas/domaincommon.rng | 2 +- src/util/virresctrl.c | 27 +++++++++++++++++++++++++-- 2 files changed, 26 insertions(+), 3 deletions(-) diff --git a/docs/schemas/domaincommon.rng b/docs/schemas/domaincommon.rng index efc84c5..ed8bdb9 100644 --- a/docs/schemas/domaincommon.rng +++ b/docs/schemas/domaincommon.rng @@ -5528,7 +5528,7 @@ - (l3) + (l3|l3code|l3data) diff --git a/src/util/virresctrl.c b/src/util/virresctrl.c index 43af0f5..e7376a0 100644 --- a/src/util/virresctrl.c +++ b/src/util/virresctrl.c @@ -417,6 +417,7 @@ int virResCtrlRefreshSchemata(void) unsigned int tmp_schemata; unsigned int default_schemata; unsigned int min_schemata; + int pair_type =3D 0; =20 virResDomainPtr header, p; =20 @@ -429,6 +430,11 @@ int virResCtrlRefreshSchemata(void) if (VIR_RESCTRL_ENABLED(i)) { min_schemata =3D VIR_RESCTRL_GET_SCHEMATA(resctrlall[i].min_cb= m_bits); =20 + if (i =3D=3D VIR_RDT_RESOURCE_L3DATA) + pair_type =3D VIR_RDT_RESOURCE_L3CODE; + if (i =3D=3D VIR_RDT_RESOURCE_L3CODE) + pair_type =3D VIR_RDT_RESOURCE_L3DATA; + for (j =3D 0; j < header->schematas[i]->n_schemata_items; j ++= ) { p =3D header->next; // Reset to default schemata 0xfffff @@ -436,8 +442,11 @@ int virResCtrlRefreshSchemata(void) tmp_schemata =3D 0; /* NOTEs: if only header domain, the schemata will be set = to default one*/ for (k =3D 1; k < domainall.num_domains; k++) { - if (p->schematas[i]->schemata_items[j].schemata > min_= schemata) + if (p->schematas[i]->schemata_items[j].schemata > min_= schemata) { tmp_schemata |=3D p->schematas[i]->schemata_items[= j].schemata; + if (pair_type > 0) + tmp_schemata |=3D p->schematas[pair_type]->sch= emata_items[j].schemata; + } p =3D p->next; } =20 @@ -503,6 +512,7 @@ virResCtrlWrite(const char *name, const char *item, con= st char *content) goto cleanup; =20 rc =3D 0; + cleanup: VIR_FREE(path); VIR_FORCE_CLOSE(writefd); @@ -707,6 +717,7 @@ virResCtrlCalculateSchemata(int type, virResDomainPtr p; unsigned int tmp_schemata; unsigned int schemata_sum =3D 0; + int pair_type =3D 0; =20 if (resctrlall[type].cache_banks[sid].cache_left < size) { VIR_ERROR(_("Not enough cache left on bank %u"), hostid); @@ -721,8 +732,18 @@ virResCtrlCalculateSchemata(int type, =20 p =3D domainall.domains; p =3D p->next; + + /* for type is l3code and l3data, we need to deal them specially*/ + if (type =3D=3D VIR_RDT_RESOURCE_L3DATA) + pair_type =3D VIR_RDT_RESOURCE_L3CODE; + + if (type =3D=3D VIR_RDT_RESOURCE_L3CODE) + pair_type =3D VIR_RDT_RESOURCE_L3DATA; + for (i =3D 1; i < domainall.num_domains; i ++) { schemata_sum |=3D p->schematas[type]->schemata_items[sid].schemata; + if (pair_type > 0) + schemata_sum |=3D p->schematas[pair_type]->schemata_items[sid]= .schemata; p =3D p->next; } =20 @@ -763,6 +784,9 @@ int virResCtrlSetCacheBanks(virDomainCachetunePtr cache= tune, } =20 if (p !=3D NULL) { + + virResCtrlAppendDomain(p); + for (i =3D 0; i < cachetune->n_banks; i++) { if ((type =3D virResCtrlTypeFromString( cachetune->cache_banks[i].type)) < 0) { @@ -797,7 +821,6 @@ int virResCtrlSetCacheBanks(virDomainCachetunePtr cache= tune, virResCtrlDestroyDomain(p); return -1; } - virResCtrlAppendDomain(p); } else { VIR_ERROR(_("Failed to create a domain in sysfs")); return -1; --=20 1.9.1 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Fri Apr 26 07:47:41 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.25 as permitted sender) client-ip=209.132.183.25; envelope-from=libvir-list-bounces@redhat.com; helo=mx4-phx2.redhat.com; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.25 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; Return-Path: Received: from mx4-phx2.redhat.com (mx4-phx2.redhat.com [209.132.183.25]) by mx.zohomail.com with SMTPS id 1488793963177224.19029406089123; Mon, 6 Mar 2017 01:52:43 -0800 (PST) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by mx4-phx2.redhat.com (8.13.8/8.13.8) with ESMTP id v269nUtp010889; Mon, 6 Mar 2017 04:49:30 -0500 Received: from int-mx13.intmail.prod.int.phx2.redhat.com (int-mx13.intmail.prod.int.phx2.redhat.com [10.5.11.26]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id v269mpXp029176 for ; Mon, 6 Mar 2017 04:48:51 -0500 Received: from mx1.redhat.com (ext-mx06.extmail.prod.ext.phx2.redhat.com [10.5.110.30]) by int-mx13.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v269moS9022953 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Mon, 6 Mar 2017 04:48:51 -0500 Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 3C6803B71B; Mon, 6 Mar 2017 09:48:49 +0000 (UTC) Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 06 Mar 2017 01:48:47 -0800 Received: from s2600wt.sh.intel.com ([10.239.48.158]) by fmsmga001.fm.intel.com with ESMTP; 06 Mar 2017 01:48:46 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.35,253,1484035200"; d="scan'208";a="1118950659" From: Eli Qiao To: libvir-list@redhat.com Date: Mon, 6 Mar 2017 17:50:38 +0800 Message-Id: <1488793843-50925-8-git-send-email-liyong.qiao@intel.com> In-Reply-To: <1488793843-50925-1-git-send-email-liyong.qiao@intel.com> References: <1488793843-50925-1-git-send-email-liyong.qiao@intel.com> X-Greylist: Sender passed SPF test, Sender IP whitelisted by DNSRBL, ACL 203 matched, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Mon, 06 Mar 2017 09:48:49 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Mon, 06 Mar 2017 09:48:49 +0000 (UTC) for IP:'134.134.136.20' DOMAIN:'mga02.intel.com' HELO:'mga02.intel.com' FROM:'liyong.qiao@intel.com' RCPT:'' X-RedHat-Spam-Score: -1.922 (BAYES_50, DCC_REPUT_00_12, RCVD_IN_DNSWL_MED, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, RP_MATCHES_RCVD, SPF_PASS) 134.134.136.20 mga02.intel.com 134.134.136.20 mga02.intel.com X-Scanned-By: MIMEDefang 2.68 on 10.5.11.26 X-Scanned-By: MIMEDefang 2.78 on 10.5.110.30 X-loop: libvir-list@redhat.com Cc: mkletzan@redhat.com, mtosatti@redhat.com, qiaoliyong@gmail.com Subject: [libvirt] [PATH V10 07/12] Resctrl: Make sure l3data/l3code are pairs X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" l3data and l3code type of cache banks should be configured pairs. Signed-off-by: Eli Qiao --- src/conf/domain_conf.c | 19 +++++++++++++++++++ src/util/virresctrl.c | 1 - src/util/virresctrl.h | 2 ++ 3 files changed, 21 insertions(+), 1 deletion(-) diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index 652f4ca..86c292d 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -15760,9 +15760,13 @@ virDomainCacheTuneDefParseXML(virDomainDefPtr def, int type =3D -1; virDomainCacheBankPtr bank =3D NULL; virResCtrlPtr resctrl; + /* An array to make sure l3code and l3data are pairs */ + int* sem =3D NULL; =20 if (VIR_ALLOC_N(bank, n) < 0) goto cleanup; + if (VIR_ALLOC_N(sem, MAX_CPU_SOCKET_NUM) < 0) + goto cleanup; =20 for (i =3D 0; i < n; i++) { if (!(tmp =3D virXMLPropString(nodes[i], "id"))) { @@ -15810,6 +15814,12 @@ virDomainCacheTuneDefParseXML(virDomainDefPtr def, goto cleanup; } =20 + /* VIR_RDT_RESOURCE_L3DATA and VIR_RDT_RESOURCE_L3CODE should be p= air */ + if (type =3D=3D VIR_RDT_RESOURCE_L3DATA) + sem[bank[i].host_id] ++; + else if (type =3D=3D VIR_RDT_RESOURCE_L3CODE) + sem[bank[i].host_id] --; + resctrl =3D virResCtrlGet(type); =20 if (resctrl =3D=3D NULL || !resctrl->enabled) { @@ -15858,6 +15868,14 @@ virDomainCacheTuneDefParseXML(virDomainDefPtr def, } } =20 + for (i =3D 0; i < MAX_CPU_SOCKET_NUM; i ++) { + if (sem[i] !=3D 0) { + virReportError(VIR_ERR_XML_ERROR, + _("'l3code and l3data shoud be show up pairs on bank %= zu'"), + i); + goto cleanup; + } + } def->cachetune.cache_banks =3D bank; def->cachetune.n_banks =3D n; return 0; @@ -15865,6 +15883,7 @@ virDomainCacheTuneDefParseXML(virDomainDefPtr def, cleanup: VIR_FREE(bank); VIR_FREE(tmp); + VIR_FREE(sem); return -1; } =20 diff --git a/src/util/virresctrl.c b/src/util/virresctrl.c index e7376a0..ee5f043 100644 --- a/src/util/virresctrl.c +++ b/src/util/virresctrl.c @@ -34,7 +34,6 @@ VIR_LOG_INIT("util.resctrl"); =20 #define VIR_FROM_THIS VIR_FROM_RESCTRL -#define MAX_CPU_SOCKET_NUM 8 #define MAX_CBM_BIT_LEN 32 #define MAX_SCHEMATA_LEN 1024 #define MAX_FILE_LEN (10 * 1024 * 1024) diff --git a/src/util/virresctrl.h b/src/util/virresctrl.h index 3a6fb95..d639de1 100644 --- a/src/util/virresctrl.h +++ b/src/util/virresctrl.h @@ -28,6 +28,8 @@ # include "virutil.h" # include "conf/domain_conf.h" =20 +#define MAX_CPU_SOCKET_NUM 8 + enum { VIR_RDT_RESOURCE_L3, VIR_RDT_RESOURCE_L3DATA, --=20 1.9.1 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Fri Apr 26 07:47:41 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.37 as permitted sender) client-ip=209.132.183.37; envelope-from=libvir-list-bounces@redhat.com; helo=mx5-phx2.redhat.com; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.37 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; Return-Path: Received: from mx5-phx2.redhat.com (mx5-phx2.redhat.com [209.132.183.37]) by mx.zohomail.com with SMTPS id 1488793952722477.67951625801766; Mon, 6 Mar 2017 01:52:32 -0800 (PST) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by mx5-phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v269nTR2026390; Mon, 6 Mar 2017 04:49:29 -0500 Received: from int-mx13.intmail.prod.int.phx2.redhat.com (int-mx13.intmail.prod.int.phx2.redhat.com [10.5.11.26]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id v269mpXw029169 for ; Mon, 6 Mar 2017 04:48:51 -0500 Received: from mx1.redhat.com (ext-mx08.extmail.prod.ext.phx2.redhat.com [10.5.110.32]) by int-mx13.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v269mpQP022978 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Mon, 6 Mar 2017 04:48:51 -0500 Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 186B6C056791; Mon, 6 Mar 2017 09:48:50 +0000 (UTC) Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 06 Mar 2017 01:48:48 -0800 Received: from s2600wt.sh.intel.com ([10.239.48.158]) by fmsmga001.fm.intel.com with ESMTP; 06 Mar 2017 01:48:47 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.35,253,1484035200"; d="scan'208";a="1118950665" From: Eli Qiao To: libvir-list@redhat.com Date: Mon, 6 Mar 2017 17:50:39 +0800 Message-Id: <1488793843-50925-9-git-send-email-liyong.qiao@intel.com> In-Reply-To: <1488793843-50925-1-git-send-email-liyong.qiao@intel.com> References: <1488793843-50925-1-git-send-email-liyong.qiao@intel.com> X-Greylist: Sender passed SPF test, Sender IP whitelisted by DNSRBL, ACL 203 matched, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Mon, 06 Mar 2017 09:48:50 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Mon, 06 Mar 2017 09:48:50 +0000 (UTC) for IP:'134.134.136.20' DOMAIN:'mga02.intel.com' HELO:'mga02.intel.com' FROM:'liyong.qiao@intel.com' RCPT:'' X-RedHat-Spam-Score: -1.922 (BAYES_50, DCC_REPUT_00_12, RCVD_IN_DNSWL_MED, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, RP_MATCHES_RCVD, SPF_PASS) 134.134.136.20 mga02.intel.com 134.134.136.20 mga02.intel.com X-Scanned-By: MIMEDefang 2.68 on 10.5.11.26 X-Scanned-By: MIMEDefang 2.78 on 10.5.110.32 X-loop: libvir-list@redhat.com Cc: mkletzan@redhat.com, mtosatti@redhat.com, qiaoliyong@gmail.com Subject: [libvirt] [PATH V10 08/12] Resctrl: Compatible mode for cdp enabled X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" This patch support l3 cache allocation compatible mode if cdp enabled on host. In this case l3code/l3data has same schemata. Signed-off-by: Eli Qiao --- src/conf/domain_conf.c | 15 +++++++++++++-- src/util/virresctrl.c | 10 ++++++++++ 2 files changed, 23 insertions(+), 2 deletions(-) diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index 86c292d..710c327 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -15823,9 +15823,20 @@ virDomainCacheTuneDefParseXML(virDomainDefPtr def, resctrl =3D virResCtrlGet(type); =20 if (resctrl =3D=3D NULL || !resctrl->enabled) { - virReportError(VIR_ERR_XML_ERROR, + /* support cdp compatible */ + if (type =3D=3D VIR_RDT_RESOURCE_L3) { + resctrl =3D virResCtrlGet(type + 1); + if (resctrl =3D=3D NULL || !resctrl->enabled) { + virReportError(VIR_ERR_XML_ERROR, _("'host doesn't enabled cache type '%s'"), tmp); - goto cleanup; + goto cleanup; + } + VIR_WARN("Use cdp compatible mode."); + } else { + virReportError(VIR_ERR_XML_ERROR, + _("'host doesn't enabled cache type '%s'"), tmp); + goto cleanup; + } } =20 bool found_host_id =3D false; diff --git a/src/util/virresctrl.c b/src/util/virresctrl.c index ee5f043..9f0d05f 100644 --- a/src/util/virresctrl.c +++ b/src/util/virresctrl.c @@ -760,6 +760,7 @@ int virResCtrlSetCacheBanks(virDomainCachetunePtr cache= tune, char name[VIR_UUID_STRING_BUFLEN]; virResDomainPtr p; int type; + int pair_type =3D -1; int sid; int schemata; =20 @@ -793,6 +794,13 @@ int virResCtrlSetCacheBanks(virDomainCachetunePtr cach= etune, cachetune->cache_banks[i].type); continue; } + /* use cdp compatible mode */ + if (!VIR_RESCTRL_ENABLED(type) && + (type =3D=3D VIR_RDT_RESOURCE_L3) && + VIR_RESCTRL_ENABLED(VIR_RDT_RESOURCE_L3DATA)) { + type =3D VIR_RDT_RESOURCE_L3DATA; + pair_type =3D VIR_RDT_RESOURCE_L3CODE; + } =20 if ((sid =3D virResCtrlGetSocketIdByHostID( type, cachetune->cache_banks[i].host_id)) < 0)= { @@ -810,6 +818,8 @@ int virResCtrlSetCacheBanks(virDomainCachetunePtr cache= tune, } =20 p->schematas[type]->schemata_items[sid].schemata =3D schemata; + if (pair_type > 0) + p->schematas[pair_type]->schemata_items[sid].schemata =3D = schemata; } =20 for (i =3D 0; i < npid; i++) --=20 1.9.1 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Fri Apr 26 07:47:41 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.39 as permitted sender) client-ip=209.132.183.39; envelope-from=libvir-list-bounces@redhat.com; helo=mx6-phx2.redhat.com; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.39 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; Return-Path: Received: from mx6-phx2.redhat.com (mx6-phx2.redhat.com [209.132.183.39]) by mx.zohomail.com with SMTPS id 148879397822449.46153970007276; Mon, 6 Mar 2017 01:52:58 -0800 (PST) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by mx6-phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v269nXZ6034173; Mon, 6 Mar 2017 04:49:33 -0500 Received: from int-mx13.intmail.prod.int.phx2.redhat.com (int-mx13.intmail.prod.int.phx2.redhat.com [10.5.11.26]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id v269mqSe029188 for ; Mon, 6 Mar 2017 04:48:52 -0500 Received: from mx1.redhat.com (ext-mx06.extmail.prod.ext.phx2.redhat.com [10.5.110.30]) by int-mx13.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v269mqIQ022985 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Mon, 6 Mar 2017 04:48:52 -0500 Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 4294D3B702; Mon, 6 Mar 2017 09:48:51 +0000 (UTC) Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 06 Mar 2017 01:48:49 -0800 Received: from s2600wt.sh.intel.com ([10.239.48.158]) by fmsmga001.fm.intel.com with ESMTP; 06 Mar 2017 01:48:49 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.35,253,1484035200"; d="scan'208";a="1118950673" From: Eli Qiao To: libvir-list@redhat.com Date: Mon, 6 Mar 2017 17:50:40 +0800 Message-Id: <1488793843-50925-10-git-send-email-liyong.qiao@intel.com> In-Reply-To: <1488793843-50925-1-git-send-email-liyong.qiao@intel.com> References: <1488793843-50925-1-git-send-email-liyong.qiao@intel.com> X-Greylist: Sender passed SPF test, Sender IP whitelisted by DNSRBL, ACL 203 matched, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Mon, 06 Mar 2017 09:48:51 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Mon, 06 Mar 2017 09:48:51 +0000 (UTC) for IP:'134.134.136.20' DOMAIN:'mga02.intel.com' HELO:'mga02.intel.com' FROM:'liyong.qiao@intel.com' RCPT:'' X-RedHat-Spam-Score: -1.922 (BAYES_50, DCC_REPUT_00_12, RCVD_IN_DNSWL_MED, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, RP_MATCHES_RCVD, SPF_PASS) 134.134.136.20 mga02.intel.com 134.134.136.20 mga02.intel.com X-Scanned-By: MIMEDefang 2.68 on 10.5.11.26 X-Scanned-By: MIMEDefang 2.78 on 10.5.110.30 X-loop: libvir-list@redhat.com Cc: mkletzan@redhat.com, mtosatti@redhat.com, qiaoliyong@gmail.com Subject: [libvirt] [PATH V10 09/12] Resctrl: concurrence support X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" The internal struct list domainall is a list which are resctral domain status shared by all VMs, especiall the default domain, each VM should access it concomitantly. Ues a mutex to control it. Each bank's cache_left field is also a global shared resource we need to be care, add a mutex for each bank. We need also to add lock to access /sys/fs/resctrl, use flock. Signed-off-by: Eli Qiao --- src/util/virresctrl.c | 93 ++++++++++++++++++++++++++++++++++++++++++++---= ---- src/util/virresctrl.h | 3 ++ 2 files changed, 83 insertions(+), 13 deletions(-) diff --git a/src/util/virresctrl.c b/src/util/virresctrl.c index 9f0d05f..3c54e44 100644 --- a/src/util/virresctrl.c +++ b/src/util/virresctrl.c @@ -20,7 +20,9 @@ */ #include #include +#include #include +#include =20 #include "virresctrl.h" #include "viralloc.h" @@ -62,6 +64,9 @@ do { \ =20 #define VIR_RESCTRL_GET_SCHEMATA(count) ((1 << count) - 1) =20 +#define VIR_RESCTRL_LOCK(fd, op) flock(fd, op) +#define VIR_RESCTRL_UNLOCK(fd) flock(fd, LOCK_UN) + /** * a virResSchemata represents a schemata object under a resource control * domain. @@ -105,6 +110,8 @@ typedef virResCtrlDomain *virResCtrlDomainPtr; struct _virResCtrlDomain { unsigned int num_domains; virResDomainPtr domains; + + virMutex lock; }; =20 static unsigned int host_id; @@ -176,11 +183,16 @@ static int virResCtrlGetStr(const char *domain_name, = const char *item_name, char =20 CONSTRUCT_RESCTRL_PATH(domain_name, item_name); =20 + if (!virFileExists(path)) + goto cleanup; + if (virFileReadAll(path, MAX_FILE_LEN, ret) < 0) { rc =3D -1; goto cleanup; } =20 + rc =3D 0; + cleanup: VIR_FREE(path); return rc; @@ -687,10 +699,15 @@ static int virResCtrlAppendDomain(virResDomainPtr dom) { virResDomainPtr p =3D domainall.domains; + + virMutexLock(&domainall.lock); + while (p->next !=3D NULL) p =3D p->next; p->next =3D dom; dom->pre =3D p; domainall.num_domains +=3D 1; + + virMutexUnlock(&domainall.lock); return 0; } =20 @@ -713,18 +730,22 @@ virResCtrlCalculateSchemata(int type, { size_t i; int count; + int rc =3D -1; virResDomainPtr p; unsigned int tmp_schemata; unsigned int schemata_sum =3D 0; int pair_type =3D 0; =20 + virMutexLock(&resctrlall[type].cache_banks[sid].lock); + if (resctrlall[type].cache_banks[sid].cache_left < size) { VIR_ERROR(_("Not enough cache left on bank %u"), hostid); - return -1; + goto cleanup; } + if ((count =3D size / resctrlall[type].cache_banks[sid].cache_min) <= =3D 0) { VIR_ERROR(_("Error cache size %llu"), size); - return -1; + goto cleanup; } =20 tmp_schemata =3D VIR_RESCTRL_GET_SCHEMATA(count); @@ -739,7 +760,7 @@ virResCtrlCalculateSchemata(int type, if (type =3D=3D VIR_RDT_RESOURCE_L3CODE) pair_type =3D VIR_RDT_RESOURCE_L3DATA; =20 - for (i =3D 1; i < domainall.num_domains; i ++) { + for (i =3D 1; i < domainall.num_domains; i++) { schemata_sum |=3D p->schematas[type]->schemata_items[sid].schemata; if (pair_type > 0) schemata_sum |=3D p->schematas[pair_type]->schemata_items[sid]= .schemata; @@ -750,7 +771,16 @@ virResCtrlCalculateSchemata(int type, =20 while ((tmp_schemata & schemata_sum) !=3D 0) tmp_schemata =3D tmp_schemata >> 1; - return tmp_schemata; + + resctrlall[type].cache_banks[sid].cache_left -=3D size; + if (pair_type > 0) + resctrlall[pair_type].cache_banks[sid].cache_left =3D resctrlall[t= ype].cache_banks[sid].cache_left; + + rc =3D tmp_schemata; + + cleanup: + virMutexUnlock(&resctrlall[type].cache_banks[sid].lock); + return rc; } =20 int virResCtrlSetCacheBanks(virDomainCachetunePtr cachetune, @@ -763,8 +793,8 @@ int virResCtrlSetCacheBanks(virDomainCachetunePtr cache= tune, int pair_type =3D -1; int sid; int schemata; - - virUUIDFormat(uuid, name); + int lockfd; + int rc =3D -1; =20 for (i =3D 0; i < cachetune->n_banks; i++) { VIR_DEBUG("cache_banks %u, %u, %llu, %s", @@ -777,12 +807,21 @@ int virResCtrlSetCacheBanks(virDomainCachetunePtr cac= hetune, if (cachetune->n_banks < 1) return 0; =20 + virUUIDFormat(uuid, name); + + if ((lockfd =3D open(RESCTRL_DIR, O_RDONLY)) < 0) + goto cleanup; + + if (VIR_RESCTRL_LOCK(lockfd, LOCK_EX) < 0) { + virReportSystemError(errno, _("Unable to lock '%s'"), RESCTRL_DIR); + goto cleanup; + } + p =3D virResCtrlGetDomain(name); if (p =3D=3D NULL) { VIR_DEBUG("no domain name %s found, create new one!", name); p =3D virResCtrlCreateDomain(name); } - if (p !=3D NULL) { =20 virResCtrlAppendDomain(p); @@ -828,19 +867,25 @@ int virResCtrlSetCacheBanks(virDomainCachetunePtr cac= hetune, if (virResCtrlFlushDomainToSysfs(p) < 0) { VIR_ERROR(_("failed to flush domain %s to sysfs"), name); virResCtrlDestroyDomain(p); - return -1; + goto cleanup; } } else { VIR_ERROR(_("Failed to create a domain in sysfs")); - return -1; + goto cleanup; } =20 virResCtrlRefreshSchemata(); /* after refresh, flush header's schemata changes to sys fs */ - if (virResCtrlFlushDomainToSysfs(domainall.domains) < 0) - VIR_WARN("failed to flush domain to sysfs"); + if (virResCtrlFlushDomainToSysfs(domainall.domains) < 0) { + VIR_ERROR(_("failed to flush domain to sysfs")); + goto cleanup; + } =20 - return 0; + rc =3D 0; + + cleanup: + VIR_RESCTRL_UNLOCK(lockfd); + return rc; } =20 /* Should be called after pid disappeared, we recalculate @@ -856,6 +901,9 @@ int virResCtrlUpdate(unsigned char *uuid) del =3D virResCtrlGetDomain(name); =20 if (del !=3D NULL) { + + virMutexLock(&domainall.lock); + del->pre->next =3D del->next; if (del->next !=3D NULL) del->next->pre =3D del->pre; @@ -864,6 +912,7 @@ int virResCtrlUpdate(unsigned char *uuid) virResCtrlRefreshSchemata(); if (virResCtrlFlushDomainToSysfs(domainall.domains) < 0) VIR_WARN("failed to flush domain to sysfs"); + virMutexUnlock(&domainall.lock); } return 0; } @@ -871,7 +920,7 @@ int virResCtrlUpdate(unsigned char *uuid) int virResCtrlInit(void) { - size_t i =3D 0; + size_t i, j; char *tmp; int rc =3D 0; =20 @@ -898,6 +947,24 @@ virResCtrlInit(void) =20 domainall.domains =3D virResCtrlGetAllDomains(&(domainall.num_domains)= ); =20 + for (i =3D 0; i < VIR_RDT_RESOURCE_LAST; i++) { + if (VIR_RESCTRL_ENABLED(i)) { + for (j =3D 0; j < resctrlall[i].num_banks; j++) { + if (virMutexInit(&resctrlall[i].cache_banks[j].lock) < 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("Unable to initialize mutex")); + return -1; + } + } + } + } + + if (virMutexInit(&domainall.lock) < 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("Unable to initialize mutex")); + return -1; + } + if ((rc =3D virResCtrlRefreshSchemata()) < 0) virReportError(VIR_ERR_INTERNAL_ERROR, "%s", _("Failed to refresh resource control")); diff --git a/src/util/virresctrl.h b/src/util/virresctrl.h index d639de1..968e0dc 100644 --- a/src/util/virresctrl.h +++ b/src/util/virresctrl.h @@ -27,6 +27,7 @@ # include "virbitmap.h" # include "virutil.h" # include "conf/domain_conf.h" +# include "virthread.h" =20 #define MAX_CPU_SOCKET_NUM 8 =20 @@ -49,6 +50,8 @@ struct _virResCacheBank { unsigned long long cache_left; unsigned long long cache_min; virBitmapPtr cpu_mask; + + virMutex lock; }; =20 /** --=20 1.9.1 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Fri Apr 26 07:47:41 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.39 as permitted sender) client-ip=209.132.183.39; envelope-from=libvir-list-bounces@redhat.com; helo=mx6-phx2.redhat.com; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.39 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; Return-Path: Received: from mx6-phx2.redhat.com (mx6-phx2.redhat.com [209.132.183.39]) by mx.zohomail.com with SMTPS id 1488793959219281.7999841276687; Mon, 6 Mar 2017 01:52:39 -0800 (PST) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by mx6-phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v269nYmK034185; Mon, 6 Mar 2017 04:49:34 -0500 Received: from int-mx10.intmail.prod.int.phx2.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id v269msqZ029207 for ; Mon, 6 Mar 2017 04:48:54 -0500 Received: from mx1.redhat.com (ext-mx04.extmail.prod.ext.phx2.redhat.com [10.5.110.28]) by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v269msm0006910 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Mon, 6 Mar 2017 04:48:54 -0500 Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 1D7808553C; Mon, 6 Mar 2017 09:48:53 +0000 (UTC) Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 06 Mar 2017 01:48:51 -0800 Received: from s2600wt.sh.intel.com ([10.239.48.158]) by fmsmga001.fm.intel.com with ESMTP; 06 Mar 2017 01:48:50 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.35,253,1484035200"; d="scan'208";a="1118950686" From: Eli Qiao To: libvir-list@redhat.com Date: Mon, 6 Mar 2017 17:50:41 +0800 Message-Id: <1488793843-50925-11-git-send-email-liyong.qiao@intel.com> In-Reply-To: <1488793843-50925-1-git-send-email-liyong.qiao@intel.com> References: <1488793843-50925-1-git-send-email-liyong.qiao@intel.com> X-Greylist: Sender passed SPF test, Sender IP whitelisted by DNSRBL, ACL 203 matched, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Mon, 06 Mar 2017 09:48:53 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Mon, 06 Mar 2017 09:48:53 +0000 (UTC) for IP:'134.134.136.20' DOMAIN:'mga02.intel.com' HELO:'mga02.intel.com' FROM:'liyong.qiao@intel.com' RCPT:'' X-RedHat-Spam-Score: -1.922 (BAYES_50, DCC_REPUT_00_12, RCVD_IN_DNSWL_MED, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, RP_MATCHES_RCVD, SPF_PASS) 134.134.136.20 mga02.intel.com 134.134.136.20 mga02.intel.com X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23 X-Scanned-By: MIMEDefang 2.78 on 10.5.110.28 X-loop: libvir-list@redhat.com Cc: mkletzan@redhat.com, mtosatti@redhat.com, qiaoliyong@gmail.com Subject: [libvirt] [PATH V10 10/12] Resctrl: Scan resctrl before doing cache allocation X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" Other application may touch resctrl while libvirt's running, scan resctrl again before allocating cache information to the newly created VM. Signed-off-by: Eli Qiao --- src/util/virresctrl.c | 40 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+) diff --git a/src/util/virresctrl.c b/src/util/virresctrl.c index 3c54e44..16c01a2 100644 --- a/src/util/virresctrl.c +++ b/src/util/virresctrl.c @@ -711,6 +711,42 @@ virResCtrlAppendDomain(virResDomainPtr dom) return 0; } =20 +/* scan /sys/fs/resctrl again and refresh default schemata */ +static +int virResCtrlScan(void) +{ + struct dirent *ent; + DIR *dp =3D NULL; + int direrr; + virResDomainPtr p; + int rc =3D -1; + + if (virDirOpenQuiet(&dp, RESCTRL_DIR) < 0) { + if (errno =3D=3D ENOENT) + return -1; + VIR_ERROR(_("Unable to open %s (%d)"), RESCTRL_DIR, errno); + goto cleanup; + } + + while ((direrr =3D virDirRead(dp, &ent, NULL)) > 0) { + if ((ent->d_type !=3D DT_DIR) || STREQ(ent->d_name, "info")) + continue; + /* test if we'v tracked all domains */ + p =3D virResCtrlGetDomain(ent->d_name); + if (p =3D=3D NULL) { + p =3D virResCtrlLoadDomain(ent->d_name); + if (p =3D=3D NULL) + continue; + virResCtrlAppendDomain(p); + } + } + rc =3D 0; + + cleanup: + VIR_DIR_CLOSE(dp); + return rc; +} + static int virResCtrlGetSocketIdByHostID(int type, unsigned int hostid) { @@ -817,6 +853,10 @@ int virResCtrlSetCacheBanks(virDomainCachetunePtr cach= etune, goto cleanup; } =20 + if (virResCtrlScan() < 0) { + VIR_ERROR(_("Failed to scan resctrl domain dir")); + goto cleanup; + } p =3D virResCtrlGetDomain(name); if (p =3D=3D NULL) { VIR_DEBUG("no domain name %s found, create new one!", name); --=20 1.9.1 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Fri Apr 26 07:47:41 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.39 as permitted sender) client-ip=209.132.183.39; envelope-from=libvir-list-bounces@redhat.com; helo=mx6-phx2.redhat.com; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.39 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; Return-Path: Received: from mx6-phx2.redhat.com (mx6-phx2.redhat.com [209.132.183.39]) by mx.zohomail.com with SMTPS id 1488793982542471.5444470301492; Mon, 6 Mar 2017 01:53:02 -0800 (PST) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by mx6-phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v269ndwk034196; Mon, 6 Mar 2017 04:49:39 -0500 Received: from int-mx11.intmail.prod.int.phx2.redhat.com (int-mx11.intmail.prod.int.phx2.redhat.com [10.5.11.24]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id v269mugL029221 for ; Mon, 6 Mar 2017 04:48:56 -0500 Received: from mx1.redhat.com (ext-mx05.extmail.prod.ext.phx2.redhat.com [10.5.110.29]) by int-mx11.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v269mua6032457 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Mon, 6 Mar 2017 04:48:56 -0500 Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 3DC6920264; Mon, 6 Mar 2017 09:48:54 +0000 (UTC) Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 06 Mar 2017 01:48:52 -0800 Received: from s2600wt.sh.intel.com ([10.239.48.158]) by fmsmga001.fm.intel.com with ESMTP; 06 Mar 2017 01:48:51 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.35,253,1484035200"; d="scan'208";a="1118950698" From: Eli Qiao To: libvir-list@redhat.com Date: Mon, 6 Mar 2017 17:50:42 +0800 Message-Id: <1488793843-50925-12-git-send-email-liyong.qiao@intel.com> In-Reply-To: <1488793843-50925-1-git-send-email-liyong.qiao@intel.com> References: <1488793843-50925-1-git-send-email-liyong.qiao@intel.com> X-Greylist: Sender passed SPF test, Sender IP whitelisted by DNSRBL, ACL 203 matched, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.29]); Mon, 06 Mar 2017 09:48:54 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.29]); Mon, 06 Mar 2017 09:48:54 +0000 (UTC) for IP:'134.134.136.20' DOMAIN:'mga02.intel.com' HELO:'mga02.intel.com' FROM:'liyong.qiao@intel.com' RCPT:'' X-RedHat-Spam-Score: -1.922 (BAYES_50, DCC_REPUT_00_12, RCVD_IN_DNSWL_MED, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, RP_MATCHES_RCVD, SPF_PASS) 134.134.136.20 mga02.intel.com 134.134.136.20 mga02.intel.com X-Scanned-By: MIMEDefang 2.68 on 10.5.11.24 X-Scanned-By: MIMEDefang 2.78 on 10.5.110.29 X-loop: libvir-list@redhat.com Cc: mkletzan@redhat.com, mtosatti@redhat.com, qiaoliyong@gmail.com Subject: [libvirt] [PATH V10 11/12] Resctrl: Add Public API for nodecachestats X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" This patch expose a public API virNodeCacheStats to query cache stats on a host. Signed-off-by: Eli Qiao --- daemon/remote.c | 67 ++++++++++++++++++++++++++++++++++++++= ++++ include/libvirt/libvirt-host.h | 32 ++++++++++++++++++++ src/driver-hypervisor.h | 7 +++++ src/libvirt-host.c | 41 ++++++++++++++++++++++++++ src/libvirt_public.syms | 1 + src/remote/remote_driver.c | 52 ++++++++++++++++++++++++++++++++ src/remote/remote_protocol.x | 25 +++++++++++++++- src/remote_protocol-structs | 16 ++++++++++ 8 files changed, 240 insertions(+), 1 deletion(-) diff --git a/daemon/remote.c b/daemon/remote.c index f2b9b9a..af291a7 100644 --- a/daemon/remote.c +++ b/daemon/remote.c @@ -7079,3 +7079,70 @@ remoteSerializeDomainDiskErrors(virDomainDiskErrorPt= r errors, } return -1; } + + static int +remoteDispatchNodeGetCacheStats(virNetServerPtr server ATTRIBUTE_UNUSED, + virNetServerClientPtr client ATTRIBUTE_UNU= SED, + virNetMessagePtr msg ATTRIBUTE_UNUSED, + virNetMessageErrorPtr rerr, + remote_node_get_cache_stats_args *args, + remote_node_get_cache_stats_ret *ret) +{ + virNodeCacheStatsPtr params =3D NULL; + size_t i; + int nparams =3D 0; + unsigned int flags; + int rv =3D -1; + struct daemonClientPrivate *priv =3D + virNetServerClientGetPrivateData(client); + + if (!priv->conn) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", _("connection not ope= n")); + goto cleanup; + } + + flags =3D args->flags; + + if (args->nparams && VIR_ALLOC_N(params, args->nparams) < 0) + goto cleanup; + nparams =3D args->nparams; + + if (virNodeGetCacheStats(priv->conn, params, &nparams, flags) < 0) + goto cleanup; + + /* In this case, we need to send back the number of stats + * supported + */ + if (args->nparams =3D=3D 0) { + ret->nparams =3D nparams; + goto success; + } + + /* Serialise the memory parameters. */ + ret->params.params_len =3D nparams; + if (VIR_ALLOC_N(ret->params.params_val, nparams) < 0) + goto cleanup; + + for (i =3D 0; i < nparams; ++i) { + /* remoteDispatchClientRequest will free this: */ + if (VIR_STRDUP(ret->params.params_val[i].field, params[i].field) <= 0) + goto cleanup; + + ret->params.params_val[i].value =3D params[i].value; + } + + success: + rv =3D 0; + + cleanup: + if (rv < 0) { + virNetMessageSaveError(rerr); + if (ret->params.params_val) { + for (i =3D 0; i < nparams; i++) + VIR_FREE(ret->params.params_val[i].field); + VIR_FREE(ret->params.params_val); + } + } + VIR_FREE(params); + return rv; +} diff --git a/include/libvirt/libvirt-host.h b/include/libvirt/libvirt-host.h index 07b5d15..222f361 100644 --- a/include/libvirt/libvirt-host.h +++ b/include/libvirt/libvirt-host.h @@ -140,6 +140,32 @@ struct _virSecurityModel { */ typedef virSecurityModel *virSecurityModelPtr; =20 +/** + * VIR_NODE_CACHE_STATS_FIELD_LENGTH: + * + * Macro providing the field length of virNodeCacheStats + */ + +# define VIR_NODE_CACHE_STATS_FIELD_LENGTH 16 + +/** + * + * virNodeCacheStats is all the cache stats on a host. + */ + +typedef struct _virNodeCacheStats virNodeCacheStats; + +struct _virNodeCacheStats { + char field[VIR_NODE_CACHE_STATS_FIELD_LENGTH]; + unsigned long long value; +}; + +/** + * a virNodeCacheStatsPtr is a pointer to a virNodeCacheStats. + */ + +typedef virNodeCacheStats *virNodeCacheStatsPtr; + =20 /* data types related to virNodePtr */ =20 @@ -603,6 +629,12 @@ int virNodeSuspendForDuration (vir= ConnectPtr conn, unsigned long long dura= tion, unsigned int flags); =20 +int virNodeGetCacheStats (virConnectPtr conn, + virNodeCacheStatsPtr params, + int *nparams, + unsigned int flags); + + /* * NUMA support */ diff --git a/src/driver-hypervisor.h b/src/driver-hypervisor.h index b81420a..a6a1350 100644 --- a/src/driver-hypervisor.h +++ b/src/driver-hypervisor.h @@ -578,6 +578,12 @@ typedef int unsigned int flags); =20 typedef int +(*virDrvNodeGetCacheStats)(virConnectPtr conn, + virNodeCacheStatsPtr params, + int *nparams, + unsigned int flags); + +typedef int (*virDrvNodeGetCellsFreeMemory)(virConnectPtr conn, unsigned long long *freeMems, int startCell, @@ -1458,6 +1464,7 @@ struct _virHypervisorDriver { virDrvConnectSetKeepAlive connectSetKeepAlive; virDrvConnectIsAlive connectIsAlive; virDrvNodeSuspendForDuration nodeSuspendForDuration; + virDrvNodeGetCacheStats nodeGetCacheStats; virDrvDomainGetPerfEvents domainGetPerfEvents; virDrvDomainSetPerfEvents domainSetPerfEvents; virDrvDomainSetBlockIoTune domainSetBlockIoTune; diff --git a/src/libvirt-host.c b/src/libvirt-host.c index 335798a..87c1279 100644 --- a/src/libvirt-host.c +++ b/src/libvirt-host.c @@ -679,6 +679,47 @@ virNodeSuspendForDuration(virConnectPtr conn, return -1; } =20 +/* + * virNodeGetCacheStats: + * @conn: pointer to the hypervisor connection + * @params: pointer to memory parameter object + * (return value, allocated by the caller) + * @nparams: pointer to number of memory parameters; input and output + * @flags: extra flags; not used yet, so callers should always pass 0 + * + * Get all node cache usage stats. + * + * Returns 0 in case of success, and -1 in case of failure. + * +*/ + +int virNodeGetCacheStats(virConnectPtr conn, + virNodeCacheStatsPtr params, + int *nparams, + unsigned int flags) +{ + VIR_DEBUG("conn=3D%p, params=3D%p, nparams=3D%d, flags=3D%x", + conn, params, nparams ? *nparams : -1, flags); + virResetLastError(); + + virCheckConnectReturn(conn, -1); + virCheckNonNullArgGoto(nparams, error); + virCheckNonNegativeArgGoto(*nparams, error); + + if (conn->driver->nodeGetCacheStats) { + int ret; + ret =3D conn->driver->nodeGetCacheStats(conn, params, nparams, fla= gs); + if (ret < 0) + goto error; + return ret; + } + virReportUnsupportedError(); + + error: + virDispatchError(conn); + return -1; + +} =20 /* * virNodeGetMemoryParameters: diff --git a/src/libvirt_public.syms b/src/libvirt_public.syms index 04ef580..a091cab 100644 --- a/src/libvirt_public.syms +++ b/src/libvirt_public.syms @@ -751,6 +751,7 @@ LIBVIRT_3.0.0 { virStorageVolGetInfoFlags; virConnectSecretEventRegisterAny; virConnectSecretEventDeregisterAny; + virNodeGetCacheStats; } LIBVIRT_2.2.0; =20 LIBVIRT_3.1.0 { diff --git a/src/remote/remote_driver.c b/src/remote/remote_driver.c index 0c8bfee..278ffc6 100644 --- a/src/remote/remote_driver.c +++ b/src/remote/remote_driver.c @@ -1708,6 +1708,57 @@ remoteNodeGetCellsFreeMemory(virConnectPtr conn, } =20 static int +remoteNodeGetCacheStats(virConnectPtr conn, + virNodeCacheStatsPtr params, + int *nparams, + unsigned int flags) +{ + int rv =3D -1; + size_t i; + remote_node_get_cache_stats_args args; + remote_node_get_cache_stats_ret ret; + struct private_data *priv =3D conn->privateData; + + remoteDriverLock(priv); + + args.nparams =3D *nparams; + args.flags =3D flags; + + memset(&ret, 0, sizeof(ret)); + if (call(conn, priv, 0, REMOTE_PROC_NODE_GET_CACHE_STATS, + (xdrproc_t) xdr_remote_node_get_cache_stats_args, (char *) &a= rgs, + (xdrproc_t) xdr_remote_node_get_cache_stats_ret, (char *) &re= t) =3D=3D -1) + goto done; + + if (*nparams =3D=3D 0) { + *nparams =3D ret.nparams; + rv =3D 0; + goto cleanup; + } + + *nparams =3D ret.params.params_len; + + /* Deserialise the result. */ + for (i =3D 0; i < *nparams; ++i) { + if (virStrcpyStatic(params[i].field, ret.params.params_val[i].fiel= d) =3D=3D NULL) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Stats %s too big for destination"), + ret.params.params_val[i].field); + goto cleanup; + } + params[i].value =3D ret.params.params_val[i].value; + } + + rv =3D 0; + + cleanup: + xdr_free((xdrproc_t) xdr_remote_node_get_cache_stats_ret, (char *) &re= t); + done: + remoteDriverUnlock(priv); + return rv; +} + +static int remoteConnectListDomains(virConnectPtr conn, int *ids, int maxids) { int rv =3D -1; @@ -8291,6 +8342,7 @@ static virHypervisorDriver hypervisor_driver =3D { .nodeGetMemoryStats =3D remoteNodeGetMemoryStats, /* 0.9.3 */ .nodeGetCellsFreeMemory =3D remoteNodeGetCellsFreeMemory, /* 0.3.3 */ .nodeGetFreeMemory =3D remoteNodeGetFreeMemory, /* 0.3.3 */ + .nodeGetCacheStats =3D remoteNodeGetCacheStats, /* 3.0.0 */ .connectDomainEventRegister =3D remoteConnectDomainEventRegister, /* 0= .5.0 */ .connectDomainEventDeregister =3D remoteConnectDomainEventDeregister, = /* 0.5.0 */ .domainMigratePrepare2 =3D remoteDomainMigratePrepare2, /* 0.5.0 */ diff --git a/src/remote/remote_protocol.x b/src/remote/remote_protocol.x index abe63af..1c349ce 100644 --- a/src/remote/remote_protocol.x +++ b/src/remote/remote_protocol.x @@ -121,6 +121,9 @@ const REMOTE_NODE_CPU_STATS_MAX =3D 16; /* Upper limit on list of node memory stats. */ const REMOTE_NODE_MEMORY_STATS_MAX =3D 16; =20 +/* Upper limit on list of node cache stats */ +const REMOTE_NODE_CACHE_STATS_MAX =3D 16; + /* Upper limit on list of block stats. */ const REMOTE_DOMAIN_BLOCK_STATS_PARAMETERS_MAX =3D 16; =20 @@ -394,6 +397,11 @@ struct remote_node_get_memory_stats { unsigned hyper value; }; =20 +struct remote_node_get_cache_stats { + remote_nonnull_string field; + unsigned hyper value; +}; + struct remote_domain_disk_error { remote_nonnull_string disk; int error; @@ -492,6 +500,16 @@ struct remote_node_get_info_ret { /* insert@1 */ int threads; }; =20 +struct remote_node_get_cache_stats_args { + int nparams; + u_int flags; +}; + +struct remote_node_get_cache_stats_ret { + remote_node_get_cache_stats params; + int nparams; +}; + struct remote_connect_get_capabilities_ret { remote_nonnull_string capabilities; }; @@ -6033,5 +6051,10 @@ enum remote_procedure { * @acl: domain:save:!VIR_DOMAIN_AFFECT_CONFIG|VIR_DOMAIN_AFFECT_LIVE * @acl: domain:save:VIR_DOMAIN_AFFECT_CONFIG */ - REMOTE_PROC_DOMAIN_SET_VCPU =3D 384 + REMOTE_PROC_DOMAIN_SET_VCPU =3D 384, + /** + * @generate: none + * @acl: connect:read + */ + REMOTE_PROC_NODE_GET_CACHE_STATS =3D 385 }; diff --git a/src/remote_protocol-structs b/src/remote_protocol-structs index e1e53d2..c8868d2 100644 --- a/src/remote_protocol-structs +++ b/src/remote_protocol-structs @@ -94,6 +94,10 @@ struct remote_node_get_memory_stats { remote_nonnull_string field; uint64_t value; }; +struct remote_node_get_cache_stats { + remote_nonnull_string field; + uint64_t value; +}; struct remote_domain_disk_error { remote_nonnull_string disk; int error; @@ -2380,6 +2384,17 @@ struct remote_node_get_cpu_map_ret { u_int online; int ret; }; +struct remote_node_get_cache_stats_args { + int nparams; + u_int flags; +}; +struct remote_node_get_cache_stats_ret { + struct { + u_int paramms_len; + remote_node_get_cache_stats * params_val; + } params; + int nparams; +}; struct remote_domain_fstrim_args { remote_nonnull_domain dom; remote_string mountPoint; @@ -3217,4 +3232,5 @@ enum remote_procedure { REMOTE_PROC_SECRET_EVENT_LIFECYCLE =3D 382, REMOTE_PROC_SECRET_EVENT_VALUE_CHANGED =3D 383, REMOTE_PROC_DOMAIN_SET_VCPU =3D 384, + REMOTE_PROC_NODE_GET_CACHE_STATS =3D 385, }; --=20 1.9.1 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Fri Apr 26 07:47:41 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.25 as permitted sender) client-ip=209.132.183.25; envelope-from=libvir-list-bounces@redhat.com; helo=mx4-phx2.redhat.com; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.25 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; Return-Path: Received: from mx4-phx2.redhat.com (mx4-phx2.redhat.com [209.132.183.25]) by mx.zohomail.com with SMTPS id 1488793980359128.76064658265238; Mon, 6 Mar 2017 01:53:00 -0800 (PST) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by mx4-phx2.redhat.com (8.13.8/8.13.8) with ESMTP id v269niLJ010910; Mon, 6 Mar 2017 04:49:44 -0500 Received: from int-mx14.intmail.prod.int.phx2.redhat.com (int-mx14.intmail.prod.int.phx2.redhat.com [10.5.11.27]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id v269mu9l029228 for ; Mon, 6 Mar 2017 04:48:56 -0500 Received: from mx1.redhat.com (ext-mx04.extmail.prod.ext.phx2.redhat.com [10.5.110.28]) by int-mx14.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v269muqY001896 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Mon, 6 Mar 2017 04:48:56 -0500 Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 0322C8553D; Mon, 6 Mar 2017 09:48:55 +0000 (UTC) Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 06 Mar 2017 01:48:53 -0800 Received: from s2600wt.sh.intel.com ([10.239.48.158]) by fmsmga001.fm.intel.com with ESMTP; 06 Mar 2017 01:48:52 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.35,253,1484035200"; d="scan'208";a="1118950706" From: Eli Qiao To: libvir-list@redhat.com Date: Mon, 6 Mar 2017 17:50:43 +0800 Message-Id: <1488793843-50925-13-git-send-email-liyong.qiao@intel.com> In-Reply-To: <1488793843-50925-1-git-send-email-liyong.qiao@intel.com> References: <1488793843-50925-1-git-send-email-liyong.qiao@intel.com> X-Greylist: Sender passed SPF test, Sender IP whitelisted by DNSRBL, ACL 203 matched, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Mon, 06 Mar 2017 09:48:55 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Mon, 06 Mar 2017 09:48:55 +0000 (UTC) for IP:'134.134.136.20' DOMAIN:'mga02.intel.com' HELO:'mga02.intel.com' FROM:'liyong.qiao@intel.com' RCPT:'' X-RedHat-Spam-Score: -1.922 (BAYES_50, DCC_REPUT_00_12, RCVD_IN_DNSWL_MED, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, RP_MATCHES_RCVD, SPF_PASS) 134.134.136.20 mga02.intel.com 134.134.136.20 mga02.intel.com X-Scanned-By: MIMEDefang 2.68 on 10.5.11.27 X-Scanned-By: MIMEDefang 2.78 on 10.5.110.28 X-loop: libvir-list@redhat.com Cc: mkletzan@redhat.com, mtosatti@redhat.com, qiaoliyong@gmail.com Subject: [libvirt] [PATH V10 12/12] Resctrl: Add nodecachestats X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" Add new virsh command line `nodecachestats` to expose the cache usage on a node. Signed-off-by: Eli Qiao --- src/libvirt_private.syms | 3 ++- src/qemu/qemu_driver.c | 12 ++++++++++ src/util/virresctrl.c | 62 ++++++++++++++++++++++++++++++++++++++++++++= ++++ src/util/virresctrl.h | 8 +++++++ tools/virsh-host.c | 49 ++++++++++++++++++++++++++++++++++++++ 5 files changed, 133 insertions(+), 1 deletion(-) diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index 9cfffb8..75a4c98 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -2323,13 +2323,14 @@ virRandomInt; =20 # util/virresctrl.h virResCtrlAvailable; +virResCtrlCacheGetStats; virResCtrlGet; virResCtrlInit; virResCtrlSetCacheBanks; virResCtrlTypeFromString; virResCtrlTypeToString; virResCtrlUpdate; -# + =20 # util/virrotatingfile.h virRotatingFileReaderConsume; diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 0f11ae2..4677406 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -18285,6 +18285,17 @@ qemuNodeGetCPUStats(virConnectPtr conn, return virHostCPUGetStats(cpuNum, params, nparams, flags); } =20 +static int +qemuNodeGetCacheStats(virConnectPtr conn, + virNodeCacheStatsPtr params, + int *nparams, + unsigned int flags) +{ + if (virNodeGetCacheStatsEnsureACL(conn) < 0) + return -1; + + return virResCtrlCacheGetStats(params, nparams, flags); +} =20 static int qemuNodeGetMemoryStats(virConnectPtr conn, @@ -20391,6 +20402,7 @@ static virHypervisorDriver qemuHypervisorDriver =3D= { .domainMemoryPeek =3D qemuDomainMemoryPeek, /* 0.4.4 */ .domainGetBlockInfo =3D qemuDomainGetBlockInfo, /* 0.8.1 */ .nodeGetCPUStats =3D qemuNodeGetCPUStats, /* 0.9.3 */ + .nodeGetCacheStats =3D qemuNodeGetCacheStats, /* 3.1.0 */ .nodeGetMemoryStats =3D qemuNodeGetMemoryStats, /* 0.9.3 */ .nodeGetCellsFreeMemory =3D qemuNodeGetCellsFreeMemory, /* 0.4.4 */ .nodeGetFreeMemory =3D qemuNodeGetFreeMemory, /* 0.4.4 */ diff --git a/src/util/virresctrl.c b/src/util/virresctrl.c index 16c01a2..97f7e84 100644 --- a/src/util/virresctrl.c +++ b/src/util/virresctrl.c @@ -1034,3 +1034,65 @@ virResCtrlGet(int type) { return &resctrlall[type]; } + +int virResCtrlCacheGetStats(virNodeCacheStatsPtr params, + int *nparams, + unsigned int flags) +{ + virCheckFlags(0, -1); + size_t i, j, k; + char *value; + int rc =3D -1; + int lockfd; + + if (*nparams =3D=3D 0) { + for (i =3D 0; i < VIR_RDT_RESOURCE_LAST; i++) { + if (VIR_RESCTRL_ENABLED(i)) + *nparams +=3D resctrlall[i].num_banks; + } + } + if (params =3D=3D NULL) + return 0; + + if ((lockfd =3D open(RESCTRL_DIR, O_RDONLY)) < 0) + goto cleanup; + + if (VIR_RESCTRL_LOCK(lockfd, LOCK_SH) < 0) { + virReportSystemError(errno, _("Unable to lock '%s'"), RESCTRL_DIR); + goto cleanup; + } + if (virResCtrlScan() < 0) { + VIR_ERROR(_("Failed to scan resctrl domain dir")); + goto cleanup; + } + + virResCtrlRefreshSchemata(); + + if ((rc =3D virResCtrlFlushDomainToSysfs(domainall.domains)) < 0) + goto cleanup; + + k =3D 0; + + for (i =3D 0; i < VIR_RDT_RESOURCE_LAST; i++) { + if (VIR_RESCTRL_ENABLED(i)) { + for (j =3D 0; j < resctrlall[i].num_banks; j++) { + + if (virAsprintf(&value, "%s.%zu", + resctrlall[i].name, j) < 0) + goto cleanup; + + if (virStrcpyStatic((¶ms[k])->field, value) =3D=3D NUL= L) + goto cleanup; + + (¶ms[k++])->value =3D resctrlall[i].cache_banks[j].cac= he_left; + } + } + } + + rc =3D 0; + + cleanup: + VIR_FREE(value); + VIR_RESCTRL_UNLOCK(lockfd); + return rc; +} diff --git a/src/util/virresctrl.h b/src/util/virresctrl.h index 968e0dc..eef5370 100644 --- a/src/util/virresctrl.h +++ b/src/util/virresctrl.h @@ -80,9 +80,17 @@ struct _virResCtrl { }; =20 bool virResCtrlAvailable(void); + int virResCtrlInit(void); + virResCtrlPtr virResCtrlGet(int); + int virResCtrlSetCacheBanks(virDomainCachetunePtr, unsigned char *, pid_t *, int); + int virResCtrlUpdate(unsigned char *); + +int virResCtrlCacheGetStats(virNodeCacheStatsPtr params, + int *nparams, + unsigned int flags); #endif diff --git a/tools/virsh-host.c b/tools/virsh-host.c index 24ebde2..c90bd2e 100644 --- a/tools/virsh-host.c +++ b/tools/virsh-host.c @@ -946,6 +946,49 @@ cmdNodeMemStats(vshControl *ctl, const vshCmd *cmd) VIR_FREE(params); return ret; } +/* "nodecachestats" command + */ +static const vshCmdInfo info_nodecachestats[] =3D { + {.name =3D "help", + .data =3D N_("Prints cache stats of the node.") + }, + {.name =3D "desc", + .data =3D N_("Returns cache stats of the node, in kilobytes.") + }, + {.name =3D NULL} +}; + +static bool +cmdNodeCacheStats(vshControl *ctl, const vshCmd *cmd ATTRIBUTE_UNUSED) +{ + virshControlPtr priv =3D ctl->privData; + virNodeCacheStatsPtr params; + int nparams =3D 0; + size_t i; + bool ret =3D false; + + if (virNodeGetCacheStats(priv->conn, NULL, &nparams, 0) !=3D 0) { + vshError(ctl, "%s", + _("Unable to get number of cache stats")); + return false; + } + if (nparams =3D=3D 0) { + /* nothing to output */ + return true; + } + + params =3D vshCalloc(ctl, nparams, sizeof(*params)); + if (virNodeGetCacheStats(priv->conn, params, &nparams, 0) !=3D 0) { + vshError(ctl, "%s", _("Unable to get node cache stats")); + goto cleanup; + } + + for (i =3D 0; i < nparams; i++) + vshPrint(ctl, "%s: %llu KiB\n", params[i].field, params[i].value); + + cleanup: + return ret; +} =20 /* * "nodesuspend" command @@ -1455,6 +1498,12 @@ const vshCmdDef hostAndHypervisorCmds[] =3D { .info =3D info_nodememstats, .flags =3D 0 }, + {.name =3D "nodecachestats", + .handler =3D cmdNodeCacheStats, + .opts =3D NULL, + .info =3D info_nodecachestats, + .flags =3D 0 + }, {.name =3D "nodesuspend", .handler =3D cmdNodeSuspend, .opts =3D opts_node_suspend, --=20 1.9.1 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list