From nobody Sat Feb 7 09:46:56 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail(p=none dis=none) header.from=intel.com Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1535373592403997.0279200458021; Mon, 27 Aug 2018 05:39:52 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 1F9D036A5E6; Mon, 27 Aug 2018 12:39:50 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id DB32A70BA3; Mon, 27 Aug 2018 12:39:49 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 81DDC181A55B; Mon, 27 Aug 2018 12:39:49 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id w7RBO66t024696 for ; Mon, 27 Aug 2018 07:24:06 -0400 Received: by smtp.corp.redhat.com (Postfix) id 6E21A5D9CC; Mon, 27 Aug 2018 11:24:06 +0000 (UTC) Received: from mx1.redhat.com (ext-mx11.extmail.prod.ext.phx2.redhat.com [10.5.110.40]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 6373B5D9CD for ; Mon, 27 Aug 2018 11:24:02 +0000 (UTC) Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 031AE30842AD for ; Mon, 27 Aug 2018 11:24:01 +0000 (UTC) Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 27 Aug 2018 04:23:56 -0700 Received: from david-unc.bj.intel.com ([10.238.145.63]) by fmsmga001.fm.intel.com with ESMTP; 27 Aug 2018 04:23:54 -0700 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.53,295,1531810800"; d="scan'208";a="84804981" From: Wang Huaqiang To: libvir-list@redhat.com Date: Mon, 27 Aug 2018 19:23:13 +0800 Message-Id: <1535368993-24901-11-git-send-email-huaqiang.wang@intel.com> In-Reply-To: <1535368993-24901-1-git-send-email-huaqiang.wang@intel.com> References: <1535368993-24901-1-git-send-email-huaqiang.wang@intel.com> X-Greylist: Sender passed SPF test, Sender IP whitelisted by DNSRBL, ACL 212 matched, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.40]); Mon, 27 Aug 2018 11:24:01 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.40]); Mon, 27 Aug 2018 11:24:01 +0000 (UTC) for IP:'134.134.136.20' DOMAIN:'mga02.intel.com' HELO:'mga02.intel.com' FROM:'huaqiang.wang@intel.com' RCPT:'' X-RedHat-Spam-Score: -2.301 (RCVD_IN_DNSWL_MED, SPF_PASS) 134.134.136.20 mga02.intel.com 134.134.136.20 mga02.intel.com X-Scanned-By: MIMEDefang 2.84 on 10.5.110.40 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-loop: libvir-list@redhat.com X-Mailman-Approved-At: Mon, 27 Aug 2018 08:39:21 -0400 Cc: shaohe.feng@intel.com, Wang Huaqiang , bing.niu@intel.com, jian-feng.ding@intel.com, rui.zang@intel.com Subject: [libvirt] [PATCH 10/10] qemu: Report cache occupancy (CMT) with domstats X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.29]); Mon, 27 Aug 2018 12:39:51 +0000 (UTC) X-ZohoMail: RDMRC_1 RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" Intel x86 RDT CMT is the technology to tell the last level cache occupancy information. Adding the interface in qemu to report this information for resource monitor group through command 'virsh domstats --cpu-total'. Below is a typical output: # virsh domstats 1 --cpu-total Domain: 'ubuntu16.04-base' ... cpu.cache.monitor.count=3D2 cpu.cache.0.name=3Dvcpus_1 cpu.cache.0.vcpus=3D1 cpu.cache.0.bank.count=3D2 cpu.cache.0.bank.0.id=3D0 cpu.cache.0.bank.0.bytes=3D4505600 cpu.cache.0.bank.1.id=3D1 cpu.cache.0.bank.1.bytes=3D5586944 cpu.cache.1.name=3Dvcpus_4-6 cpu.cache.1.vcpus=3D4,5,6 cpu.cache.1.bank.count=3D2 cpu.cache.1.bank.0.id=3D0 cpu.cache.1.bank.0.bytes=3D17571840 cpu.cache.1.bank.1.id=3D1 cpu.cache.1.bank.1.bytes=3D29106176 Signed-off-by: Wang Huaqiang --- src/libvirt-domain.c | 9 ++ src/qemu/qemu_driver.c | 265 +++++++++++++++++++++++++++++++++++++++++++++= ---- 2 files changed, 253 insertions(+), 21 deletions(-) diff --git a/src/libvirt-domain.c b/src/libvirt-domain.c index ef46027..a88e94a 100644 --- a/src/libvirt-domain.c +++ b/src/libvirt-domain.c @@ -11337,6 +11337,15 @@ virConnectGetDomainCapabilities(virConnectPtr conn, * "cpu.user" - user cpu time spent in nanoseconds as unsigned long lo= ng. * "cpu.system" - system cpu time spent in nanoseconds as unsigned long * long. + * "cpu.cache.monitor.count" - tocal cache monitoring groups + * "cpu.cache.M.name" - name for cache monitoring group 'M' + * "cpu.cache.M.vcpus" - vcpus for cache monitoring group 'M' + * "cpu.cache.M.bank.count" - total bank number for cache monitoring + * group 'M' + * "cpu.cache.M.bank.N.id" - OS assigned cache bank id for cache 'N' in + * cache monitoring group 'M' + * "cpu.cache.M.bank.N.bytes" - cache occupancy of cache bank 'N' in + * cache monitoring group 'M' * * VIR_DOMAIN_STATS_BALLOON: * Return memory balloon device information. diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index da8c4e8..193f606 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -107,6 +107,7 @@ #include "virnuma.h" #include "dirname.h" #include "netdev_bandwidth_conf.h" +#include "c-ctype.h" =20 #define VIR_FROM_THIS VIR_FROM_QEMU =20 @@ -19660,6 +19661,225 @@ typedef enum { #define HAVE_JOB(flags) ((flags) & QEMU_DOMAIN_STATS_HAVE_JOB) =20 =20 +/* + * qemuDomainVcpuFormatHelper + * For vcpu string, both '1-3' and '1,3' are valid format and + * representing different vcpu set, but it is not easy to + * differentiate them at first galance, to avoid this case + * substituting all '-' with ',', e.g. substitute string '1-3' + * with '1,2,3'. + */ +static int +qemuDomainVcpuFormatHelper(char **vcpus) +{ + const char *cur =3D NULL; + size_t i =3D 0; + char *tmp =3D NULL; + int start, last; + virBuffer buf =3D VIR_BUFFER_INITIALIZER; + bool firstnum =3D 1; + + if (!*vcpus) + goto error; + + cur =3D *vcpus; + + virSkipSpaces(&cur); + + if (*cur =3D=3D '\0') + goto error; + + while (*cur !=3D 0) { + if (!c_isdigit(*cur)) + goto error; + + if (virStrToLong_i(cur, &tmp, 10, &start) < 0) + goto error; + if (start < 0) + goto error; + + cur =3D tmp; + + virSkipSpaces(&cur); + + if (*cur =3D=3D ',' || *cur =3D=3D 0) { + if (!firstnum) + virBufferAddChar(&buf, ','); + virBufferAsprintf(&buf, "%d", start); + firstnum =3D 0; + } else if (*cur =3D=3D '-') { + cur++; + virSkipSpaces(&cur); + + if (virStrToLong_i(cur, &tmp, 10, &last) < 0) + + goto error; + if (last < start) + goto error; + cur =3D tmp; + + for (i =3D start; i <=3D last; i++) { + if (!firstnum) + + virBufferAddChar(&buf, ','); + virBufferAsprintf(&buf, "%ld", i); + firstnum =3D 0; + } + + virSkipSpaces(&cur); + } + + if (*cur =3D=3D ',') { + cur++; + virSkipSpaces(&cur); + } else if (*cur =3D=3D 0) { + break; + } else { + goto error; + } + } + VIR_FREE(*vcpus); + *vcpus =3D virBufferContentAndReset(&buf); + return 0; + error: + virBufferFreeAndReset(&buf); + return -1; +} + +static int +qemuDomainGetStatsCpuResource(virQEMUDriverPtr driver ATTRIBUTE_UNUSED, + virDomainObjPtr dom, + virDomainStatsRecordPtr record, + int *maxparams, + unsigned int privflags ATTRIBUTE_UNUSED) +{ + size_t i =3D 0; + size_t j =3D 0; + char param_name[VIR_TYPED_PARAM_FIELD_LENGTH]; + virDomainResctrlDefPtr resctrl =3D NULL; + virDomainResctrlMonitorPtr monitor =3D NULL; + unsigned int nvals =3D 0; + unsigned int *ids =3D NULL; + unsigned int *vals =3D NULL; + unsigned int nmonitor =3D NULL; + char *vcpustr =3D NULL; + int ret =3D -1; + + for (i =3D 0; i < dom->def->nresctrls; i++) { + resctrl =3D dom->def->resctrls[i]; + + for (j =3D 0; j < resctrl->nmonitors; j++) { + monitor =3D resctrl->monitors[j]; + if (monitor->vcpus) + nmonitor++; + } + } + + snprintf(param_name, VIR_TYPED_PARAM_FIELD_LENGTH, + "cpu.cache.monitor.count"); + if (virTypedParamsAddUInt(&record->params, + &record->nparams, + maxparams, + param_name, + nmonitor) < 0) + goto cleanup; + + for (i =3D 0; i < dom->def->nresctrls; i++) { + resctrl =3D dom->def->resctrls[i]; + + for (j =3D 0; j < resctrl->nmonitors; j++) { + size_t l =3D 0; + + monitor =3D resctrl->monitors[j]; + + if (!(vcpustr =3D virBitmapFormat(monitor->vcpus))) + goto cleanup; + + if (qemuDomainVcpuFormatHelper(&vcpustr) < 0) + goto cleanup; + + switch ((virDomainResctrlMonType) monitor->type) { + case VIR_DOMAIN_RESCTRL_MONITOR_CACHE: + case VIR_DOMAIN_RESCTRL_MONITOR_CACHE_MEMBW: + if (!monitor->vcpus) + continue; + + if (virResctrlAllocGetCacheOccupancy(resctrl->alloc, + monitor->id, &nvals, + &ids, &vals) < 0) + goto cleanup; + + snprintf(param_name, VIR_TYPED_PARAM_FIELD_LENGTH, + "cpu.cache.%ld.name", i); + if (virTypedParamsAddString(&record->params, + &record->nparams, + maxparams, + param_name, + monitor->id) < 0) + goto cleanup; + + snprintf(param_name, VIR_TYPED_PARAM_FIELD_LENGTH, + "cpu.cache.%ld.vcpus", i); + + if (virTypedParamsAddString(&record->params, + &record->nparams, + maxparams, + param_name, + vcpustr) < 0) + goto cleanup; + + snprintf(param_name, VIR_TYPED_PARAM_FIELD_LENGTH, + "cpu.cache.%ld.bank.count", i); + if (virTypedParamsAddUInt(&record->params, + &record->nparams, + maxparams, + param_name, + nvals) < 0) + goto cleanup; + + for (l =3D 0; l < nvals; l++) { + snprintf(param_name, VIR_TYPED_PARAM_FIELD_LENGTH, + "cpu.cache.%ld.bank.%ld.id", i, l); + if (virTypedParamsAddUInt(&record->params, + &record->nparams, + maxparams, + param_name, + ids[l]) < 0) + goto cleanup; + + + snprintf(param_name, VIR_TYPED_PARAM_FIELD_LENGTH, + "cpu.cache.%ld.bank.%ld.bytes", i, l); + if (virTypedParamsAddUInt(&record->params, + &record->nparams, + maxparams, + param_name, + vals[l]) < 0) + goto cleanup; + } + break; + + case VIR_DOMAIN_RESCTRL_MONITOR_MEMBW: + case VIR_DOMAIN_RESCTRL_MONITOR_LAST: + default: + break; + } + + VIR_FREE(ids); + VIR_FREE(vals); + VIR_FREE(vcpustr); + nvals =3D 0; + } + } + + ret =3D 0; + cleanup: + VIR_FREE(ids); + VIR_FREE(vals); + VIR_FREE(vcpustr); + return ret; +} + static int qemuDomainGetStatsCpu(virQEMUDriverPtr driver ATTRIBUTE_UNUSED, virDomainObjPtr dom, @@ -19673,29 +19893,32 @@ qemuDomainGetStatsCpu(virQEMUDriverPtr driver ATT= RIBUTE_UNUSED, unsigned long long sys_time =3D 0; int err =3D 0; =20 - if (!priv->cgroup) - return 0; + if (priv->cgroup) { + err =3D virCgroupGetCpuacctUsage(priv->cgroup, &cpu_time); + if (!err && virTypedParamsAddULLong(&record->params, + &record->nparams, + maxparams, + "cpu.time", + cpu_time) < 0) + return -1; =20 - err =3D virCgroupGetCpuacctUsage(priv->cgroup, &cpu_time); - if (!err && virTypedParamsAddULLong(&record->params, - &record->nparams, - maxparams, - "cpu.time", - cpu_time) < 0) - return -1; + err =3D virCgroupGetCpuacctStat(priv->cgroup, &user_time, &sys_tim= e); + if (!err && virTypedParamsAddULLong(&record->params, + &record->nparams, + maxparams, + "cpu.user", + user_time) < 0) + return -1; + if (!err && virTypedParamsAddULLong(&record->params, + &record->nparams, + maxparams, + "cpu.system", + sys_time) < 0) + return -1; + } =20 - err =3D virCgroupGetCpuacctStat(priv->cgroup, &user_time, &sys_time); - if (!err && virTypedParamsAddULLong(&record->params, - &record->nparams, - maxparams, - "cpu.user", - user_time) < 0) - return -1; - if (!err && virTypedParamsAddULLong(&record->params, - &record->nparams, - maxparams, - "cpu.system", - sys_time) < 0) + if (qemuDomainGetStatsCpuResource(driver, dom, + record, maxparams, privflags) < 0) return -1; =20 return 0; --=20 2.7.4 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list