From nobody Thu Mar 28 21:46:15 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1499222301506622.224981958032; Tue, 4 Jul 2017 19:38:21 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 1CCF7A145A; Wed, 5 Jul 2017 02:38:19 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 3532B5D72C; Wed, 5 Jul 2017 02:38:18 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 2DB2D4A492; Wed, 5 Jul 2017 02:38:14 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id v652cC2S019431 for ; Tue, 4 Jul 2017 22:38:12 -0400 Received: by smtp.corp.redhat.com (Postfix) id A0A405D744; Wed, 5 Jul 2017 02:38:12 +0000 (UTC) Received: from mx1.redhat.com (ext-mx10.extmail.prod.ext.phx2.redhat.com [10.5.110.39]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 154434F9EE; Wed, 5 Jul 2017 02:38:11 +0000 (UTC) Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id E0180624C6; Wed, 5 Jul 2017 02:38:08 +0000 (UTC) Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga105.jf.intel.com with ESMTP; 04 Jul 2017 19:38:08 -0700 Received: from s2600wt.sh.intel.com ([10.239.48.36]) by orsmga003.jf.intel.com with ESMTP; 04 Jul 2017 19:38:07 -0700 DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 1CCF7A145A Authentication-Results: ext-mx01.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=intel.com Authentication-Results: ext-mx01.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=libvir-list-bounces@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 1CCF7A145A DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com E0180624C6 Authentication-Results: ext-mx10.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=intel.com Authentication-Results: ext-mx10.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=liyong.qiao@intel.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com E0180624C6 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.40,310,1496127600"; d="scan'208";a="988753385" From: Eli Qiao To: libvir-list@redhat.com Date: Wed, 5 Jul 2017 10:39:03 +0800 Message-Id: <1499222344-24961-2-git-send-email-liyong.qiao@intel.com> In-Reply-To: <1499222344-24961-1-git-send-email-liyong.qiao@intel.com> References: <1499222344-24961-1-git-send-email-liyong.qiao@intel.com> X-Greylist: Sender passed SPF test, Sender IP whitelisted by DNSRBL, ACL 203 matched, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.39]); Wed, 05 Jul 2017 02:38:09 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.39]); Wed, 05 Jul 2017 02:38:09 +0000 (UTC) for IP:'134.134.136.100' DOMAIN:'mga07.intel.com' HELO:'mga07.intel.com' FROM:'liyong.qiao@intel.com' RCPT:'' X-RedHat-Spam-Score: -1.902 (BAYES_50, DCC_REPUT_00_12, RCVD_IN_DNSWL_MED, RP_MATCHES_RCVD, SPF_PASS) 134.134.136.100 mga07.intel.com 134.134.136.100 mga07.intel.com X-Scanned-By: MIMEDefang 2.78 on 10.5.110.39 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-loop: libvir-list@redhat.com Cc: mkletzan@redhat.com, liyong.qiao@intel.com Subject: [libvirt] [PATCH RESEND RFC V2 1/2] Resctrl: Add new xml element to support cache tune X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.25]); Wed, 05 Jul 2017 02:38:20 +0000 (UTC) X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" This patch adds new xml element to support cache tune as: ... ... cacheId: reference of the host's cache banks id, it's from capabilities xml. type: cache bank type, it could be both, code, data. sizeKiB: must be multiple of granularity, must be greater than or equal to minimum. vcpus: cache allocation on vcpu set, if empty, will apply the allocation on all vcpus. Signed-off-by: Eli Qiao --- docs/schemas/domaincommon.rng | 29 +++++++++++ src/conf/domain_conf.c | 112 ++++++++++++++++++++++++++++++++++++++= ++++ src/conf/domain_conf.h | 19 +++++++ 3 files changed, 160 insertions(+) diff --git a/docs/schemas/domaincommon.rng b/docs/schemas/domaincommon.rng index e259e3e..6e0262e 100644 --- a/docs/schemas/domaincommon.rng +++ b/docs/schemas/domaincommon.rng @@ -834,6 +834,25 @@ + + + + + + + + + + + + + + + + + + + @@ -5693,6 +5712,16 @@ -1 + + + [0-9]+ + + + + + (both|code|data) + + diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index 0409c62..65a9486 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -2927,6 +2927,11 @@ void virDomainDefFree(virDomainDefPtr def) =20 xmlFreeNode(def->metadata); =20 + for (i =3D 0; i < def->cachetune.n_banks; i++) + virBitmapFree(def->cachetune.cache_banks[i].vcpus); + + VIR_FREE(def->cachetune.cache_banks); + VIR_FREE(def); } =20 @@ -16318,6 +16323,81 @@ virDomainVcpuPinDefParseXML(virDomainDefPtr def, return ret; } =20 +/* Parse the XML definition for cachetune + * and a cachetune has the form + * + */ +static int +virDomainCacheTuneDefParseXML(virDomainDefPtr def, + int n, + xmlNodePtr* nodes) +{ + char* tmp =3D NULL; + size_t i; + int type =3D -1; + virDomainCacheBankPtr bank =3D NULL; + + if (VIR_ALLOC_N(bank, n) < 0) + goto cleanup; + + for (i =3D 0; i < n; i++) { + if (!(tmp =3D virXMLPropString(nodes[i], "cacheId"))) { + virReportError(VIR_ERR_XML_ERROR, "%s", _("missing cacheId in = cache tune")); + goto cleanup; + } + if (virStrToLong_uip(tmp, NULL, 10, &(bank[i].cache_id)) < 0) { + virReportError(VIR_ERR_XML_ERROR, + _("invalid setting for cacheId '%s'"), tmp); + goto cleanup; + } + VIR_FREE(tmp); + + if (!(tmp =3D virXMLPropString(nodes[i], "type"))) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("missing cache type")); + goto cleanup; + } + if ((type =3D virCacheTypeFromString(tmp)) < 0) { + virReportError(VIR_ERR_XML_ERROR, + _("'unsupported cache type '%s'"), tmp); + goto cleanup; + } + VIR_FREE(tmp); + + if (!(tmp =3D virXMLPropString(nodes[i], "sizeKiB"))) { + virReportError(VIR_ERR_XML_ERROR, "%s", _("missing sizeKiB in = cache tune")); + goto cleanup; + } + if (virStrToLong_ull(tmp, NULL, 10, &(bank[i].size)) < 0) { + virReportError(VIR_ERR_XML_ERROR, + _("invalid setting for cache sizeKiB '%s'"), tm= p); + goto cleanup; + } + /* convert to B */ + bank[i].size *=3D 1024; + VIR_FREE(tmp); + + if ((tmp =3D virXMLPropString(nodes[i], "vcpus"))) { + if (virBitmapParse(tmp, &bank[i].vcpus, VIR_DOMAIN_CPUMASK_LEN= ) < 0) + goto cleanup; + + if (virBitmapIsAllClear(bank[i].vcpus)) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, + _("Invalid value of vcpus '%s'"), tmp); + goto cleanup; + } + } + } + + def->cachetune.cache_banks =3D bank; + def->cachetune.n_banks =3D n; + return 0; + + cleanup: + VIR_FREE(bank); + VIR_FREE(tmp); + return -1; +} =20 /* Parse the XML definition for a iothreadpin * and an iothreadspin has the form @@ -17606,6 +17686,14 @@ virDomainDefParseXML(xmlDocPtr xml, } VIR_FREE(nodes); =20 + if ((n =3D virXPathNodeSet("./cputune/cachetune", ctxt, &nodes)) < 0) + goto error; + + if (n > 0 && virDomainCacheTuneDefParseXML(def, n, nodes) < 0) + goto error; + + VIR_FREE(nodes); + if ((n =3D virXPathNodeSet("./cputune/emulatorpin", ctxt, &nodes)) < 0= ) { virReportError(VIR_ERR_INTERNAL_ERROR, "%s", _("cannot extract emulatorpin nodes")); @@ -24390,6 +24478,28 @@ virDomainSchedulerFormat(virBufferPtr buf, } =20 =20 +static void +virDomainCacheTuneDefFormat(virBufferPtr buf, + virDomainCachetunePtr cache) +{ + size_t i; + + for (i =3D 0; i < cache->n_banks; i ++) { + virBufferAsprintf(buf, "cache_banks[i].cache_id, + virCacheTypeToString(cache->cache_banks[i].= type), + cache->cache_banks[i].size / 1024); + + if (cache->cache_banks[i].vcpus) + virBufferAsprintf(buf, " vcpus=3D'%s'/>\n", + virBitmapFormat(cache->cache_banks[i].vcpus)); + else + virBufferAddLit(buf, "/>\n"); + } +} + + static int virDomainCputuneDefFormat(virBufferPtr buf, virDomainDefPtr def) @@ -24436,6 +24546,8 @@ virDomainCputuneDefFormat(virBufferPtr buf, "\n", def->cputune.iothread_quota); =20 + virDomainCacheTuneDefFormat(&childrenBuf, &def->cachetune); + for (i =3D 0; i < def->maxvcpus; i++) { char *cpumask; virDomainVcpuDefPtr vcpu =3D def->vcpus[i]; diff --git a/src/conf/domain_conf.h b/src/conf/domain_conf.h index 6d9ee97..729b676 100644 --- a/src/conf/domain_conf.h +++ b/src/conf/domain_conf.h @@ -2190,6 +2190,23 @@ struct _virDomainMemtune { int allocation; /* enum virDomainMemoryAllocation */ }; =20 +typedef struct _virDomainCacheBank virDomainCacheBank; +typedef virDomainCacheBank *virDomainCacheBankPtr; + +struct _virDomainCacheBank { + unsigned int cache_id; /* cache id to be allocated on */ + int type; /* enum virCacheType */ + unsigned long long size; /* in B */ + virBitmapPtr vcpus; +}; + +typedef struct _virDomainCachetune virDomainCachetune; +typedef virDomainCachetune *virDomainCachetunePtr; +struct _virDomainCachetune { + size_t n_banks; + virDomainCacheBankPtr cache_banks; +}; + typedef struct _virDomainPowerManagement virDomainPowerManagement; typedef virDomainPowerManagement *virDomainPowerManagementPtr; =20 @@ -2263,6 +2280,8 @@ struct _virDomainDef { =20 virDomainCputune cputune; =20 + virDomainCachetune cachetune; + virDomainNumaPtr numa; virDomainResourceDefPtr resource; virDomainIdMapDef idmap; --=20 1.9.1 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Thu Mar 28 21:46:15 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1499222322278472.6849343800053; Tue, 4 Jul 2017 19:38:42 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id A00C8C057EC9; Wed, 5 Jul 2017 02:38:40 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 6FB466A52E; Wed, 5 Jul 2017 02:38:40 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 1F1A61833033; Wed, 5 Jul 2017 02:38:40 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id v652cLpg019452 for ; Tue, 4 Jul 2017 22:38:21 -0400 Received: by smtp.corp.redhat.com (Postfix) id 36BF55C6CA; Wed, 5 Jul 2017 02:38:21 +0000 (UTC) Received: from mx1.redhat.com (ext-mx07.extmail.prod.ext.phx2.redhat.com [10.5.110.31]) by smtp.corp.redhat.com (Postfix) with ESMTPS id B15916F920; Wed, 5 Jul 2017 02:38:17 +0000 (UTC) Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 54EB5C04D2AB; Wed, 5 Jul 2017 02:38:10 +0000 (UTC) Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga105.jf.intel.com with ESMTP; 04 Jul 2017 19:38:09 -0700 Received: from s2600wt.sh.intel.com ([10.239.48.36]) by orsmga003.jf.intel.com with ESMTP; 04 Jul 2017 19:38:08 -0700 DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com A00C8C057EC9 Authentication-Results: ext-mx08.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=intel.com Authentication-Results: ext-mx08.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=libvir-list-bounces@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com A00C8C057EC9 DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 54EB5C04D2AB Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=intel.com Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=liyong.qiao@intel.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 54EB5C04D2AB X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.40,310,1496127600"; d="scan'208";a="988753391" From: Eli Qiao To: libvir-list@redhat.com Date: Wed, 5 Jul 2017 10:39:04 +0800 Message-Id: <1499222344-24961-3-git-send-email-liyong.qiao@intel.com> In-Reply-To: <1499222344-24961-1-git-send-email-liyong.qiao@intel.com> References: <1499222344-24961-1-git-send-email-liyong.qiao@intel.com> X-Greylist: Sender passed SPF test, Sender IP whitelisted by DNSRBL, ACL 203 matched, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Wed, 05 Jul 2017 02:38:11 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Wed, 05 Jul 2017 02:38:11 +0000 (UTC) for IP:'134.134.136.100' DOMAIN:'mga07.intel.com' HELO:'mga07.intel.com' FROM:'liyong.qiao@intel.com' RCPT:'' X-RedHat-Spam-Score: -1.902 (BAYES_50, DCC_REPUT_00_12, RCVD_IN_DNSWL_MED, RP_MATCHES_RCVD, SPF_PASS) 134.134.136.100 mga07.intel.com 134.134.136.100 mga07.intel.com X-Scanned-By: MIMEDefang 2.78 on 10.5.110.31 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-loop: libvir-list@redhat.com Cc: mkletzan@redhat.com, liyong.qiao@intel.com Subject: [libvirt] [PATCH RESEND RFC V2 2/2] Resctrl: Do cache allocation while boot a qemu VM X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Wed, 05 Jul 2017 02:38:41 +0000 (UTC) X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" Set cachetune if user define cachetune in domain xml when using qemu driver. This patch adds 3 major private interface. virResctrlGetFreeCache: return free cache, default cache substract cache allocated. virResctrlSetCachetunes: set cache banks which defined in a domain. virResctrlRemoveCachetunes: remove cache allocation group from the host. Signed-off-by: Eli Qiao --- include/libvirt/virterror.h | 1 + src/Makefile.am | 1 + src/libvirt_private.syms | 9 + src/qemu/qemu_process.c | 81 +++ src/util/virerror.c | 1 + src/util/virresctrl.c | 822 ++++++++++++++++++++++++++= ++++ src/util/virresctrl.h | 88 ++++ tests/Makefile.am | 8 +- tests/virresctrldata/L3-free.schemata | 1 + tests/virresctrldata/L3CODE-free.schemata | 1 + tests/virresctrldata/L3DATA-free.schemata | 1 + tests/virresctrldata/linux-resctrl | 1 + tests/virresctrldata/linux-resctrl-cdp | 1 + tests/virresctrltest.c | 119 +++++ 14 files changed, 1134 insertions(+), 1 deletion(-) create mode 100644 src/util/virresctrl.c create mode 100644 src/util/virresctrl.h create mode 100644 tests/virresctrldata/L3-free.schemata create mode 100644 tests/virresctrldata/L3CODE-free.schemata create mode 100644 tests/virresctrldata/L3DATA-free.schemata create mode 120000 tests/virresctrldata/linux-resctrl create mode 120000 tests/virresctrldata/linux-resctrl-cdp create mode 100644 tests/virresctrltest.c diff --git a/include/libvirt/virterror.h b/include/libvirt/virterror.h index 2efee8f..4bc0c74 100644 --- a/include/libvirt/virterror.h +++ b/include/libvirt/virterror.h @@ -132,6 +132,7 @@ typedef enum { =20 VIR_FROM_PERF =3D 65, /* Error from perf */ VIR_FROM_LIBSSH =3D 66, /* Error from libssh connection transpor= t */ + VIR_FROM_RESCTRL =3D 67, /* Error from resctrl */ =20 # ifdef VIR_ENUM_SENTINELS VIR_ERR_DOMAIN_LAST diff --git a/src/Makefile.am b/src/Makefile.am index eae32dc..8dbb778 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -167,6 +167,7 @@ UTIL_SOURCES =3D \ util/virprocess.c util/virprocess.h \ util/virqemu.c util/virqemu.h \ util/virrandom.h util/virrandom.c \ + util/virresctrl.h util/virresctrl.c \ util/virrotatingfile.h util/virrotatingfile.c \ util/virscsi.c util/virscsi.h \ util/virscsihost.c util/virscsihost.h \ diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index c1e9471..72e5096 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -2439,6 +2439,15 @@ virRandomGenerateWWN; virRandomInt; =20 =20 +# util/virresctrl.h +virResctrlBitmap2String; +virResctrlFreeSchemata; +virResctrlGetFreeCache; +virResctrlRemoveCachetunes; +virResctrlSetCachetunes; +virResctrlTypeToString; + + # util/virrotatingfile.h virRotatingFileReaderConsume; virRotatingFileReaderFree; diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index fa9990e..3b11caa 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -70,6 +70,7 @@ #include "virbitmap.h" #include "viratomic.h" #include "virnuma.h" +#include "virresctrl.h" #include "virstring.h" #include "virhostdev.h" #include "secret_util.h" @@ -5130,6 +5131,77 @@ qemuProcessSetupVcpus(virDomainObjPtr vm) return 0; } =20 +static int +qemuProcessSetCacheBanks(virCapsHostPtr caps, virDomainObjPtr vm) +{ + size_t i, j; + virDomainCachetunePtr cachetune; + virResctrlCachetunePtr resctrl_cachetune; + unsigned int max_vcpus =3D virDomainDefGetVcpusMax(vm->def); + pid_t *pids =3D NULL; + virDomainVcpuDefPtr vcpu; + size_t npids =3D 0; + size_t count =3D 0; + int ret =3D -1; + + cachetune =3D &(vm->def->cachetune); + + if (VIR_ALLOC_N(resctrl_cachetune, cachetune->n_banks) < 0) + goto cleanup; + + /* construct resctrl_cachetune array */ + for (i =3D 0; i < cachetune->n_banks; i++) { + + resctrl_cachetune[i].cache_id =3D cachetune->cache_banks[i].cache_= id; + resctrl_cachetune[i].type =3D cachetune->cache_banks[i].type; + resctrl_cachetune[i].size =3D cachetune->cache_banks[i].size; + + /* get granularity from host's capabilities */ + for (j =3D 0; j < caps->ncaches; j++) { + /* even enable CDP, granularity for code and data are same */ + if (caps->caches[j]->id =3D=3D resctrl_cachetune[i].cache_id && + caps->caches[j]->controls) { + resctrl_cachetune[i].granularity =3D caps->caches[j]->cont= rols[0]->granularity; + break; + } + } + + /* create pids of vcpus array */ + if (cachetune->cache_banks[i].vcpus) { + for (j =3D 0; j < max_vcpus; j++) { + if (virBitmapIsBitSet(cachetune->cache_banks[i].vcpus, j))= { + + vcpu =3D virDomainDefGetVcpu(vm->def, j); + if (!vcpu->online) + continue; + + if (VIR_RESIZE_N(pids, npids, count, 1) < 0) + goto cleanup; + pids[count ++] =3D qemuDomainGetVcpuPid(vm, j); + } + } + } + } + + /* If not specify vcpus in cachetune, add vm->pid */ + if (pids =3D=3D NULL) { + if (VIR_ALLOC_N(pids, 1) < 0) + goto cleanup; + pids[0] =3D vm->pid; + count =3D 1; + } + + ret =3D virResctrlSetCachetunes(vm->def->uuid, + resctrl_cachetune, + cachetune->n_banks, + pids, + count); + + cleanup: + VIR_FREE(resctrl_cachetune); + VIR_FREE(pids); + return ret; +} =20 int qemuProcessSetupIOThread(virDomainObjPtr vm, @@ -5966,6 +6038,12 @@ qemuProcessLaunch(virConnectPtr conn, qemuProcessAutoDestroyAdd(driver, vm, conn) < 0) goto cleanup; =20 + VIR_DEBUG("Cache allocation"); + if (vm->def->cachetune.n_banks > 0 && + qemuProcessSetCacheBanks(&driver->caps->host, + vm) < 0) + goto cleanup; + ret =3D 0; =20 cleanup: @@ -6472,6 +6550,9 @@ void qemuProcessStop(virQEMUDriverPtr driver, virPerfFree(priv->perf); priv->perf =3D NULL; =20 + if (&(vm->def->cachetune) !=3D NULL) + virResctrlRemoveCachetunes(vm->def->uuid); + qemuProcessRemoveDomainStatus(driver, vm); =20 /* Remove VNC and Spice ports from port reservation bitmap, but only if diff --git a/src/util/virerror.c b/src/util/virerror.c index ef17fb5..02fabcc 100644 --- a/src/util/virerror.c +++ b/src/util/virerror.c @@ -139,6 +139,7 @@ VIR_ENUM_IMPL(virErrorDomain, VIR_ERR_DOMAIN_LAST, =20 "Perf", /* 65 */ "Libssh transport layer", + "Resource Control", ) =20 =20 diff --git a/src/util/virresctrl.c b/src/util/virresctrl.c new file mode 100644 index 0000000..541c69c --- /dev/null +++ b/src/util/virresctrl.c @@ -0,0 +1,822 @@ +/* + * virresctrl.c: methods for managing resource control + * + * Copyright (C) 2017 Intel, Inc. + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library. If not, see + * . + * + * Authors: + * Eli Qiao + */ + +#include +#include +#include +#include +#include + +#include "viralloc.h" +#include "virerror.h" +#include "virfile.h" +#include "virlog.h" +#include "virresctrl.h" +#include "virstring.h" +#include "viruuid.h" + +VIR_LOG_INIT("util.resctrl"); + +#define VIR_FROM_THIS VIR_FROM_RESCTRL +#define SYSFS_RESCTRL_PATH "/sys/fs/resctrl" +#define MAX_CBM_LEN 20 +#define VIR_RESCTRL_LOCK(fd, op) flock(fd, op) +#define VIR_RESCTRL_UNLOCK(fd) flock(fd, LOCK_UN) +#define CONSTRUCT_RESCTRL_PATH(domain_name, item_name) \ +do { \ + if (NULL =3D=3D domain_name) { \ + if (virAsprintf(&path, "%s/%s", SYSFS_RESCTRL_PATH, item_name) < 0= ) \ + return -1; \ + } else { \ + if (virAsprintf(&path, "%s/%s/%s", SYSFS_RESCTRL_PATH, domain_name= , \ + item_name) < 0) \ + return -1; \ + } \ +} while (0) + +VIR_ENUM_IMPL(virResctrl, VIR_RESCTRL_TYPE_LAST, + "L3", + "L3CODE", + "L3DATA") + +/** + * a virResctrlGroup represents a resource control group, it's a directory + * under /sys/fs/resctrl. + * e.g. /sys/fs/resctrl/CG1 + * |-- cpus + * |-- schemata + * `-- tasks + * # cat schemata + * L3DATA:0=3Dfffff;1=3Dfffff + * L3CODE:0=3Dfffff;1=3Dfffff + * + * Besides, it can also represent the default resource control group of the + * host. + */ + +typedef struct _virResctrlGroup virResctrlGroup; +typedef virResctrlGroup *virResctrlGroupPtr; +struct _virResctrlGroup { + char *name; /* resource group name, NULL for default host group */ + size_t n_tasks; /* number of tasks assigned to the resource group */ + char **tasks; /* task id list */ + virResctrlSchemataPtr schemata[VIR_RESCTRL_TYPE_LAST]; /* Array for sc= hemata */ +}; + +/* All resource control groups on this host, including default resource gr= oup */ +typedef struct _virResctrlHost virResctrlHost; +typedef virResctrlHost *virResctrlHostPtr; +struct _virResctrlHost { + size_t n_groups; /* number of resource control group */ + virResctrlGroupPtr *groups; /* list of resource control group */ +}; + +void +virResctrlFreeSchemata(virResctrlSchemataPtr ptr) +{ + size_t i; + + if (!ptr) + return; + + for (i =3D 0; i < ptr->n_masks; i++) { + virBitmapFree(ptr->masks[i]->mask); + VIR_FREE(ptr->masks[i]); + } + + VIR_FREE(ptr); + ptr =3D NULL; +} + +static void +virResctrlFreeGroup(virResctrlGroupPtr ptr) +{ + size_t i; + + if (!ptr) + return; + + for (i =3D 0; i < ptr->n_tasks; i++) + VIR_FREE(ptr->tasks[i]); + VIR_FREE(ptr->name); + + for (i =3D 0; i < VIR_RESCTRL_TYPE_LAST; i++) + virResctrlFreeSchemata(ptr->schemata[i]); + + VIR_FREE(ptr); + ptr =3D NULL; +} + +/* Return specify type of schemata string from schematalval. + e.g., 0=3Df;1=3Df */ +static int +virResctrlGetSchemataString(virResctrlType type, + const char *schemataval, + char **schematastr) +{ + int rc =3D -1; + char *prefix =3D NULL; + char **lines =3D NULL; + + if (virAsprintf(&prefix, + "%s:", + virResctrlTypeToString(type)) < 0) + return -1; + + lines =3D virStringSplit(schemataval, "\n", 0); + + if (VIR_STRDUP(*schematastr, + virStringListGetFirstWithPrefix(lines, prefix)) < 0) + goto cleanup; + + if (*schematastr =3D=3D NULL) + rc =3D -1; + else + rc =3D 0; + + cleanup: + VIR_FREE(prefix); + virStringListFree(lines); + return rc; +} + +static int +virResctrlRemoveSysGroup(const char* name) +{ + char *path =3D NULL; + int ret =3D -1; + + if ((ret =3D virAsprintf(&path, "%s/%s", SYSFS_RESCTRL_PATH, name)) < = 0) + return ret; + + ret =3D rmdir(path); + + VIR_FREE(path); + return ret; +} + +static int +virResctrlNewSysGroup(const char *name) +{ + char *path =3D NULL; + int ret =3D -1; + mode_t mode =3D 0755; + + if (virAsprintf(&path, "%s/%s", SYSFS_RESCTRL_PATH, name) < 0) + return -1; + + if (virDirCreate(path, mode, 0, 0, 0) < 0) + goto cleanup; + + ret =3D 0; + + cleanup: + VIR_FREE(path); + return ret; +} + +static int +virResctrlWrite(const char *name, const char *item, const char *content) +{ + char *path; + int writefd; + int rc =3D -1; + + CONSTRUCT_RESCTRL_PATH(name, item); + + if (!virFileExists(path)) + goto cleanup; + + if ((writefd =3D open(path, O_WRONLY | O_APPEND, S_IRUSR | S_IWUSR)) <= 0) + goto cleanup; + + if (safewrite(writefd, content, strlen(content)) < 0) + goto cleanup; + rc =3D 0; + + cleanup: + VIR_FREE(path); + VIR_FORCE_CLOSE(writefd); + return rc; +} + +static +virBitmapPtr virResctrlMask2Bitmap(const char *mask) +{ + virBitmapPtr bitmap; + unsigned int tmp; + size_t i; + + if (virStrToLong_ui(mask, NULL, 16, &tmp) < 0) + return NULL; + + bitmap =3D virBitmapNewEmpty(); + + for (i =3D 0; i < MAX_CBM_LEN; i++) { + if (((tmp & 0x1) =3D=3D 0x1) && + (virBitmapSetBitExpand(bitmap, i) < 0)) + goto error; + tmp =3D tmp >> 1; + } + return bitmap; + + error: + virBitmapFree(bitmap); + return NULL; +} + +char *virResctrlBitmap2String(virBitmapPtr bitmap) +{ + char *tmp; + char *ret =3D NULL; + char *p; + tmp =3D virBitmapString(bitmap); + /* skip "0x" */ + p =3D tmp + 2; + + /* first non-0 position */ + while (*++p =3D=3D '0'); + + if (VIR_STRDUP(ret, p) < 0) + ret =3D NULL; + + VIR_FREE(tmp); + return ret; +} + +static int +virResctrlParseSchemata(const char* schemata_str, + virResctrlSchemataPtr schemata) +{ + VIR_DEBUG("schemata_str=3D%s, schemata=3D%p", schemata_str, schemata); + + int ret =3D -1; + size_t i; + virResctrlMaskPtr mask; + char **schemata_list; + char *mask_str; + + /* parse 0=3Dfffff;1=3Df */ + schemata_list =3D virStringSplit(schemata_str, ";", 0); + + if (!schemata_list) + goto cleanup; + + for (i =3D 0; schemata_list[i] !=3D NULL; i++) { + /* parse 0=3Dfffff */ + mask_str =3D strchr(schemata_list[i], '=3D'); + + if (!mask_str) + goto cleanup; + + if (VIR_ALLOC(mask) < 0) + goto cleanup; + + mask->cache_id =3D i; + mask->mask =3D virResctrlMask2Bitmap(mask_str + 1); + schemata->n_masks +=3D 1; + schemata->masks[i] =3D mask; + + } + ret =3D 0; + + cleanup: + virStringListFree(schemata_list); + return ret; +} + +static int +virResctrlLoadGroup(const char *name, + virResctrlHostPtr host) +{ + VIR_DEBUG("name=3D%s, host=3D%p\n", name, host); + + int ret =3D -1; + char *schemataval =3D NULL; + char *schemata_str =3D NULL; + virResctrlType i; + int rv; + virResctrlGroupPtr grp; + virResctrlSchemataPtr schemata; + + rv =3D virFileReadValueString(&schemataval, + SYSFS_RESCTRL_PATH "/%s/schemata", + name ? name : ""); + + if (rv < 0) + return -1; + + if (VIR_ALLOC(grp) < 0) + goto cleanup; + + if (VIR_STRDUP(grp->name, name) < 0) + goto cleanup; + + for (i =3D 0; i < VIR_RESCTRL_TYPE_LAST; i++) { + rv =3D virResctrlGetSchemataString(i, schemataval, &schemata_str); + + if (rv < 0) + continue; + + if (VIR_ALLOC(schemata) < 0) + goto cleanup; + + schemata->type =3D i; + + if (virResctrlParseSchemata(schemata_str, schemata) < 0) { + VIR_FREE(schemata); + VIR_FREE(schemata_str); + goto cleanup; + } + + grp->schemata[i] =3D schemata; + VIR_FREE(schemata_str); + } + + if (VIR_APPEND_ELEMENT(host->groups, + host->n_groups, + grp) < 0) { + virResctrlFreeGroup(grp); + goto cleanup; + } + + ret =3D 0; + + cleanup: + VIR_FREE(schemataval); + return ret; +} + +static int +virResctrlLoadHost(virResctrlHostPtr host) +{ + int rv =3D -1; + DIR *dirp =3D NULL; + char *path =3D NULL; + struct dirent *ent; + + rv =3D virDirOpenIfExists(&dirp, SYSFS_RESCTRL_PATH); + if (rv < 0) + return -1; + + /* load default group first */ + if (virResctrlLoadGroup(NULL, host) < 0) + return -1; + + while ((rv =3D virDirRead(dirp, &ent, path)) > 0) { + /* resctrl is not hierarchical, only read directory under + /sys/fs/resctrl */ + if ((ent->d_type !=3D DT_DIR) || STREQ(ent->d_name, "info")) + continue; + + if (virResctrlLoadGroup(ent->d_name, host) < 0) + return -1; + } + return 0; +} + +static void +virResctrlRefreshHost(virResctrlHostPtr host) +{ + virResctrlGroupPtr default_grp =3D NULL; + virResctrlSchemataPtr schemata =3D NULL; + size_t i, j; + virResctrlType t; + + default_grp =3D host->groups[0]; + + for (t =3D 0; t < VIR_RESCTRL_TYPE_LAST; t++) { + if (default_grp->schemata[t] !=3D NULL) { + for (i =3D 0; i < default_grp->schemata[t]->n_masks; i++) { + /* Reset default group's mask */ + virBitmapSetAll(default_grp->schemata[t]->masks[i]->mask); + /* Loop each other resource group except default group */ + for (j =3D 1; j < host->n_groups; j++) { + schemata =3D host->groups[j]->schemata[t]; + virBitmapSubtract(default_grp->schemata[t]->masks[i]->= mask, + schemata->masks[i]->mask); + } + } + } + } +} + +static virResctrlGroupPtr +virResctrlGetFreeGroup(void) +{ + size_t i; + virResctrlHostPtr host =3D NULL; + virResctrlGroupPtr grp =3D NULL; + + if (VIR_ALLOC(host) < 0) + return NULL; + + if (virResctrlLoadHost(host) < 0) + goto error; + + virResctrlRefreshHost(host); + + for (i =3D 1; i < host->n_groups; i++) + virResctrlFreeGroup(host->groups[i]); + + grp =3D host->groups[0]; + VIR_FREE(host); + + return grp; + + error: + virResctrlFreeGroup(grp); + return NULL; +} + +virResctrlSchemataPtr +virResctrlGetFreeCache(virResctrlType type) +{ + VIR_DEBUG("type=3D%d", type); + + virResctrlType t; + virResctrlGroupPtr grp =3D NULL; + virResctrlSchemataPtr schemata =3D NULL; + int lockfd =3D -1; + + lockfd =3D open(SYSFS_RESCTRL_PATH, O_DIRECTORY); + if (lockfd < 0) + return NULL; + + VIR_RESCTRL_LOCK(lockfd, LOCK_SH); + + if ((grp =3D virResctrlGetFreeGroup()) =3D=3D NULL) + goto cleanup; + + for (t =3D 0; t < VIR_RESCTRL_TYPE_LAST; t++) { + if (t =3D=3D type) + schemata =3D grp->schemata[t]; + else + virResctrlFreeSchemata(grp->schemata[t]); + } + + cleanup: + VIR_RESCTRL_UNLOCK(lockfd); + VIR_FORCE_CLOSE(lockfd); + return schemata; +} + +static int +virResctrlCalculateCbm(int cbm_len, + virBitmapPtr defaultcbm, + virBitmapPtr newcbm) +{ + VIR_DEBUG("cbm_len=3D%d, defaultcbm=3D%p, newcbm=3D%p", + cbm_len, defaultcbm, newcbm); + + ssize_t pos =3D -1; + size_t i; + + /* not enough cache way to be allocated */ + if (virBitmapCountBits(defaultcbm) < cbm_len + 1) + return -1; + + while ((pos =3D virBitmapNextSetBit(defaultcbm, pos)) >=3D 0) { + for (i =3D 0; i < cbm_len; i++) + ignore_value(virBitmapSetBitExpand(newcbm, i + pos)); + /* Test if newcbm is sub set of defaultcbm */ + if (virBitmapNextClearBit(defaultcbm, pos) > i + pos) { + break; + } else { + pos =3D pos + i - 1; + virBitmapClearAll(newcbm); + } + } + + if (virBitmapCountBits(newcbm) !=3D cbm_len) + return -1; + + /* consume default cbm after allocation */ + virBitmapSubtract(defaultcbm, newcbm); + + return 0; +} + +/* Fill mask value for newly created resource group base on hostcachebank + * and domcachebank */ +static int +virResctrlFillMask(virResctrlGroupPtr grp, + virResctrlGroupPtr free_grp, + virResctrlCachetunePtr cachetune) +{ + VIR_DEBUG("grp=3D%p, free_grp=3D%p, cachetune=3D%p", + grp, free_grp, cachetune); + + unsigned int cache_id; + unsigned int cache_type; + int cbm_candidate_len; + virResctrlSchemataPtr schemata; + virResctrlMaskPtr mask; + + cache_type =3D cachetune->type; + cache_id =3D cachetune->cache_id; + schemata =3D grp->schemata[cache_type]; + + if ((schemata =3D=3D NULL) && (VIR_ALLOC(schemata) < 0)) + return -1; + + if (VIR_ALLOC(mask) < 0) + return -1; + + mask->cache_id =3D cache_id; + mask->mask =3D virBitmapNewEmpty(); + + /* here should be control->granularity and control->min + also domcachebank size should be checked while define domain xml */ + cbm_candidate_len =3D cachetune->size / cachetune->granularity; + if (virResctrlCalculateCbm(cbm_candidate_len, + free_grp->schemata[cache_type]->masks[cache= _id]->mask, + mask->mask) < 0) + goto error; + + schemata->type =3D cache_type; + schemata->n_masks +=3D 1; + schemata->masks[cache_id] =3D mask; + grp->schemata[cache_type] =3D schemata; + + return 0; + + error: + VIR_FREE(schemata); + return -1; +} + +/* only keep the highest consecutive bits */ +static void +virResctrlTrimMask(virResctrlMaskPtr mask) +{ + size_t i; + ssize_t setbit =3D -1; + ssize_t clearbit =3D -1; + + clearbit =3D virBitmapNextClearBit(mask->mask, -1); + setbit =3D virBitmapNextSetBit(mask->mask, -1); + for (i =3D setbit; i < clearbit; i++) + ignore_value(virBitmapClearBit(mask->mask, i)); +} + +static int +virResctrlCompleteMask(virResctrlSchemataPtr schemata, + virResctrlSchemataPtr defaultschemata) +{ + size_t i; + virResctrlMaskPtr mask; + + if (schemata =3D=3D NULL && VIR_ALLOC(schemata) < 0) + return -1; + + if (schemata->n_masks =3D=3D defaultschemata->n_masks) + return 0; + + for (i =3D 0; i < defaultschemata->n_masks; i++) { + if (schemata->masks[i] =3D=3D NULL) { + if (VIR_ALLOC(mask) < 0) + goto error; + + mask->cache_id =3D i; + mask->mask =3D virBitmapNewEmpty(); + schemata->n_masks +=3D 1; + schemata->masks[i] =3D mask; + /* resctrl doesn't allow mask to be zero + use higher bits to fill up the cbm which + domaincache bank doens't provide */ + ignore_value(virBitmapSetBitExpand(mask->mask, + virBitmapLastSetBit(defaultschemata->masks[i]->ma= sk))); + } + /* only keep the highest consecutive bits for default group */ + virResctrlTrimMask(defaultschemata->masks[i]); + } + + return 0; + + error: + VIR_FREE(schemata); + return -1; +} + +/* complete the schemata in the resrouce group before it can be write back + to resctrl */ +static int +virResctrlCompleteGroup(virResctrlGroupPtr grp, + virResctrlGroupPtr default_grp) +{ + virResctrlType t; + virResctrlSchemataPtr schemata; + virResctrlSchemataPtr defaultschemata; + + /* NOTES: resctrl system require we need provide all cache's cbm mask = */ + for (t =3D 0; t < VIR_RESCTRL_TYPE_LAST; t++) { + defaultschemata =3D default_grp->schemata[t]; + if (defaultschemata !=3D NULL) { + schemata =3D grp->schemata[t]; + if (virResctrlCompleteMask(schemata, defaultschemata) < 0) + return -1; + } + } + return 0; +} + +static +char *virResctrlGetSchemataStr(virResctrlSchemataPtr schemata) +{ + virBuffer buf =3D VIR_BUFFER_INITIALIZER; + size_t i; + + virBufferAsprintf(&buf, "%s:%u=3D%s", + virResctrlTypeToString(schemata->type), + schemata->masks[0]->cache_id, + virResctrlBitmap2String(schemata->masks[0]->mask)); + + for (i =3D 1; i < schemata->n_masks; i ++) + virBufferAsprintf(&buf, ";%u=3D%s", + schemata->masks[i]->cache_id, + virResctrlBitmap2String(schemata->masks[i]->mask= )); + + return virBufferContentAndReset(&buf); +} + +static int +virResctrlFlushGroup(virResctrlGroupPtr grp) +{ + int ret =3D -1; + size_t i; + char *schemata_str =3D NULL; + virResctrlType t; + virBuffer buf =3D VIR_BUFFER_INITIALIZER; + + VIR_DEBUG("grp=3D%p", grp); + + if (grp->name !=3D NULL && virResctrlNewSysGroup(grp->name) < 0) + return -1; + + for (t =3D 0; t < VIR_RESCTRL_TYPE_LAST; t++) { + if (grp->schemata[t] !=3D NULL) { + schemata_str =3D virResctrlGetSchemataStr(grp->schemata[t]); + virBufferAsprintf(&buf, "%s\n", schemata_str); + VIR_FREE(schemata_str); + } + } + + schemata_str =3D virBufferContentAndReset(&buf); + + if (virResctrlWrite(grp->name, "schemata", schemata_str) < 0) + goto cleanup; + + for (i =3D 0; i < grp->n_tasks; i++) { + if (virResctrlWrite(grp->name, "tasks", grp->tasks[i]) < 0) + goto cleanup; + } + + ret =3D 0; + + cleanup: + VIR_FREE(schemata_str); + return ret; +} + +int virResctrlSetCachetunes(unsigned char* uuid, + virResctrlCachetunePtr cachetunes, size_t ncac= hetune, + pid_t *pids, size_t npid) +{ + size_t i; + int ret =3D -1; + char name[VIR_UUID_STRING_BUFLEN]; + char *tmp; + int lockfd =3D -1; + virResctrlGroupPtr grp =3D NULL; + virResctrlGroupPtr default_grp =3D NULL; + + virUUIDFormat(uuid, name); + + if (ncachetune < 1) + return 0; + + /* create new resource group */ + if (VIR_ALLOC(grp) < 0) + goto error; + + if (VIR_STRDUP(grp->name, name) < 0) + goto error; + + /* allocate file lock */ + lockfd =3D open(SYSFS_RESCTRL_PATH, O_DIRECTORY); + if (lockfd < 0) + goto error; + + VIR_RESCTRL_LOCK(lockfd, LOCK_EX); + + if ((default_grp =3D virResctrlGetFreeGroup()) =3D=3D NULL) + goto error; + + /* Allocate cache for each cache bank defined in cache tune */ + for (i =3D 0; i < ncachetune; i++) { + /* fill up newly crated grp and consume from default_grp */ + if (virResctrlFillMask(grp, default_grp, &cachetunes[i]) < 0) + goto error; + } + + /* Add tasks to grp */ + for (i =3D 0; i < npid; i++) { + if (virAsprintf(&tmp, "%llu", (long long)pids[i]) < 0) + goto error; + + if (VIR_APPEND_ELEMENT(grp->tasks, + grp->n_tasks, + tmp) < 0) { + VIR_FREE(tmp); + goto error; + } + } + + if (virResctrlCompleteGroup(grp, default_grp) < 0) { + VIR_WARN("Failed to complete group"); + goto error; + } + + if (virResctrlFlushGroup(grp) < 0) + goto error; + + if (virResctrlFlushGroup(default_grp) < 0) { + virResctrlRemoveSysGroup(grp->name); + goto error; + } + + ret =3D 0; + + error: + VIR_RESCTRL_UNLOCK(lockfd); + VIR_FORCE_CLOSE(lockfd); + virResctrlFreeGroup(grp); + virResctrlFreeGroup(default_grp); + return ret; +} + +int virResctrlRemoveCachetunes(unsigned char* uuid) +{ + int ret =3D -1; + int lockfd =3D -1; + size_t i; + virResctrlType t; + virResctrlSchemataPtr schemata; + char name[VIR_UUID_STRING_BUFLEN]; + virResctrlGroupPtr default_grp =3D NULL; + + virUUIDFormat(uuid, name); + + VIR_DEBUG("name=3D%s", name); + + lockfd =3D open(SYSFS_RESCTRL_PATH, O_DIRECTORY); + if (lockfd < 0) + return -1; + + VIR_RESCTRL_LOCK(lockfd, LOCK_SH); + + if (virResctrlRemoveSysGroup(name) < 0) + goto cleanup; + + if ((default_grp =3D virResctrlGetFreeGroup()) =3D=3D NULL) + goto cleanup; + + for (t =3D 0; t < VIR_RESCTRL_TYPE_LAST; t++) { + schemata =3D default_grp->schemata[t]; + if (schemata !=3D NULL) { + for (i =3D 0; i < schemata->n_masks; i++) + virResctrlTrimMask(schemata->masks[i]); + } + } + + if (virResctrlFlushGroup(default_grp) < 0) + goto cleanup; + + ret =3D 0; + + cleanup: + VIR_RESCTRL_UNLOCK(lockfd); + VIR_FORCE_CLOSE(lockfd); + return ret; +} diff --git a/src/util/virresctrl.h b/src/util/virresctrl.h new file mode 100644 index 0000000..49781ad --- /dev/null +++ b/src/util/virresctrl.h @@ -0,0 +1,88 @@ +/* + * virresctrl.h: header for managing resctrl control + * + * Copyright (C) 2017 Intel, Inc. + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library. If not, see + * . + * + * Authors: + * Eli Qiao + */ + +#ifndef __VIR_RESCTRL_H__ +# define __VIR_RESCTRL_H__ + +#include "virutil.h" +#include "virbitmap.h" + +#define MAX_CACHE_ID 16 + +typedef enum { + VIR_RESCTRL_TYPE_L3, + VIR_RESCTRL_TYPE_L3_CODE, + VIR_RESCTRL_TYPE_L3_DATA, + + VIR_RESCTRL_TYPE_LAST +} virResctrlType; + +VIR_ENUM_DECL(virResctrl); + +typedef struct _virResctrlCachetune virResctrlCachetune; +typedef virResctrlCachetune *virResctrlCachetunePtr; +struct _virResctrlCachetune { + unsigned int cache_id; + virResctrlType type; + unsigned long long size; /* in B */ + unsigned long long granularity; /* min size of the cache */ +}; + +/* + * a virResctrlMask represents one of mask object in a + * resource control group. + * e.g., 0=3Df + */ +typedef struct _virResctrlMask virResctrlMask; +typedef virResctrlMask *virResctrlMaskPtr; +struct _virResctrlMask { + unsigned int cache_id; /* cache resource id */ + virBitmapPtr mask; /* the cbm mask */ +}; + +/* + * a virResctrlSchemata represents schemata objects of specific type of + * resource in a resource control group. + * eg: L3:0=3Df,1=3Dff + */ +typedef struct _virResctrlSchemata virResctrlSchemata; +typedef virResctrlSchemata *virResctrlSchemataPtr; +struct _virResctrlSchemata { + virResctrlType type; /* resource control type, e.g., L3 */ + size_t n_masks; /* number of masks */ + virResctrlMaskPtr masks[MAX_CACHE_ID]; /* array of mask, use array for= easy index */ +}; + +/* Get free cache of the host, result saved in schemata */ +virResctrlSchemataPtr virResctrlGetFreeCache(virResctrlType type); + +/* Get mask string from Bitmap */ +char *virResctrlBitmap2String(virBitmapPtr bitmap); + +void virResctrlFreeSchemata(virResctrlSchemataPtr ptr); + +int virResctrlSetCachetunes(unsigned char* uuid, + virResctrlCachetunePtr cachetunes, size_t ncac= hetune, + pid_t *pids, size_t npid); +int virResctrlRemoveCachetunes(unsigned char* uuid); +#endif diff --git a/tests/Makefile.am b/tests/Makefile.am index 19986dc..e0b9923 100644 --- a/tests/Makefile.am +++ b/tests/Makefile.am @@ -229,6 +229,7 @@ if WITH_LINUX test_programs +=3D fchosttest test_programs +=3D scsihosttest test_programs +=3D vircaps2xmltest +test_programs +=3D virresctrltest test_libraries +=3D virusbmock.la \ virnetdevbandwidthmock.la \ virnumamock.la \ @@ -1149,8 +1150,13 @@ virnumamock_la_CFLAGS =3D $(AM_CFLAGS) virnumamock_la_LDFLAGS =3D $(MOCKLIBS_LDFLAGS) virnumamock_la_LIBADD =3D $(MOCKLIBS_LIBS) =20 +virresctrltest_SOURCES =3D \ + virresctrltest.c testutils.h testutils.c virfilewrapper.h virfilewrapper.c +virresctrltest_LDADD =3D $(LDADDS) + else ! WITH_LINUX -EXTRA_DIST +=3D vircaps2xmltest.c virnumamock.c virfilewrapper.c virfilewr= apper.h +EXTRA_DIST +=3D vircaps2xmltest.c virnumamock.c virfilewrapper.c \ + virfilewrapper.h virresctrltest.c endif ! WITH_LINUX =20 if WITH_NSS diff --git a/tests/virresctrldata/L3-free.schemata b/tests/virresctrldata/L= 3-free.schemata new file mode 100644 index 0000000..9b47d25 --- /dev/null +++ b/tests/virresctrldata/L3-free.schemata @@ -0,0 +1 @@ +L3:0=3D1ffff;1=3D1ffff diff --git a/tests/virresctrldata/L3CODE-free.schemata b/tests/virresctrlda= ta/L3CODE-free.schemata new file mode 100644 index 0000000..7039c45 --- /dev/null +++ b/tests/virresctrldata/L3CODE-free.schemata @@ -0,0 +1 @@ +L3CODE:0=3Dcffff;1=3Dcffff diff --git a/tests/virresctrldata/L3DATA-free.schemata b/tests/virresctrlda= ta/L3DATA-free.schemata new file mode 100644 index 0000000..30f1cbd --- /dev/null +++ b/tests/virresctrldata/L3DATA-free.schemata @@ -0,0 +1 @@ +L3DATA:0=3D3ffff;1=3D3ffff diff --git a/tests/virresctrldata/linux-resctrl b/tests/virresctrldata/linu= x-resctrl new file mode 120000 index 0000000..069dfb2 --- /dev/null +++ b/tests/virresctrldata/linux-resctrl @@ -0,0 +1 @@ +../vircaps2xmldata/linux-resctrl \ No newline at end of file diff --git a/tests/virresctrldata/linux-resctrl-cdp b/tests/virresctrldata/= linux-resctrl-cdp new file mode 120000 index 0000000..c5a973f --- /dev/null +++ b/tests/virresctrldata/linux-resctrl-cdp @@ -0,0 +1 @@ +../vircaps2xmldata/linux-resctrl-cdp \ No newline at end of file diff --git a/tests/virresctrltest.c b/tests/virresctrltest.c new file mode 100644 index 0000000..a9d75f1 --- /dev/null +++ b/tests/virresctrltest.c @@ -0,0 +1,119 @@ +/* + * Copyright (C) Intel, Inc. 2017 + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library. If not, see + * . + * + * Authors: + * Eli Qiao + */ + +#include +#include + +#include "testutils.h" +#include "virbitmap.h" +#include "virfilewrapper.h" +#include "virresctrl.h" + + +#define VIR_FROM_THIS VIR_FROM_NONE + +struct virResctrlData { + const char *filename; + virResctrlType type; +}; + +static void +GetSchemataStr(virResctrlSchemataPtr schemata, char **str) +{ + size_t i; + + virBuffer buf =3D VIR_BUFFER_INITIALIZER; + virBufferAsprintf(&buf, "%s:%u=3D%s", + virResctrlTypeToString(schemata->type), + schemata->masks[0]->cache_id, + virResctrlBitmap2String(schemata->masks[0]->ma= sk)); + + for (i =3D 1; i < schemata->n_masks; i ++) + virBufferAsprintf(&buf, ";%u=3D%s", + schemata->masks[i]->cache_id, + virResctrlBitmap2String(schemata->masks[i]= ->mask)); + + *str =3D virBufferContentAndReset(&buf); +} + +static int +test_virResctrl(const void *opaque) +{ + struct virResctrlData *data =3D (struct virResctrlData *) opaque; + char *dir =3D NULL; + char *resctrl =3D NULL; + int ret =3D -1; + virResctrlSchemataPtr schemata =3D NULL; + char *schemata_str =3D NULL; + char *schemata_file; + + if (virAsprintf(&resctrl, "%s/virresctrldata/linux-%s/resctrl", + abs_srcdir, data->filename) < 0) + goto cleanup; + + if (virAsprintf(&schemata_file, "%s/virresctrldata/%s-free.schemata", + abs_srcdir, virResctrlTypeToString(data->type)) < 0) + goto cleanup; + + if (virFileWrapperAddPrefix("/sys/fs/resctrl", resctrl) < 0) + goto cleanup; + + if ((schemata =3D virResctrlGetFreeCache(data->type)) =3D=3D NULL) + goto cleanup; + + virFileWrapperClearPrefixes(); + + GetSchemataStr(schemata, &schemata_str); + + if (virTestCompareToFile(schemata_str, schemata_file) < 0) + goto cleanup; + + ret =3D 0; + + cleanup: + VIR_FREE(dir); + VIR_FREE(resctrl); + VIR_FREE(schemata_str); + virResctrlFreeSchemata(schemata); + return ret; +} + +static int +mymain(void) +{ + int ret =3D 0; + +#define DO_TEST_FULL(filename, type) \ + do { \ + struct virResctrlData data =3D {filename, \ + type}; \ + if (virTestRun(filename, test_virResctrl, &data) < 0) \ + ret =3D -1; \ + } while (0) + + DO_TEST_FULL("resctrl", VIR_RESCTRL_TYPE_L3); + DO_TEST_FULL("resctrl-cdp", VIR_RESCTRL_TYPE_L3_CODE); + DO_TEST_FULL("resctrl-cdp", VIR_RESCTRL_TYPE_L3_DATA); + + return ret; +} + +VIR_TEST_MAIN(mymain) --=20 1.9.1 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list