From nobody Mon Feb 9 02:28:50 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail(p=none dis=none) header.from=intel.com Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 153112153763296.40323472166051; Mon, 9 Jul 2018 00:32:17 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx11.intmail.prod.int.phx2.redhat.com [10.5.11.26]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 47B6530832C2; Mon, 9 Jul 2018 07:32:16 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 177683001A49; Mon, 9 Jul 2018 07:32:16 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id C5CDE1800B6B; Mon, 9 Jul 2018 07:32:15 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.24]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id w697VvDW015250 for ; Mon, 9 Jul 2018 03:31:57 -0400 Received: by smtp.corp.redhat.com (Postfix) id C3974308BDAE; Mon, 9 Jul 2018 07:31:57 +0000 (UTC) Received: from mx1.redhat.com (ext-mx05.extmail.prod.ext.phx2.redhat.com [10.5.110.29]) by smtp.corp.redhat.com (Postfix) with ESMTPS id B5659308BDAD for ; Mon, 9 Jul 2018 07:31:55 +0000 (UTC) Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 0B67F3DE42 for ; Mon, 9 Jul 2018 07:31:54 +0000 (UTC) Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga101.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 09 Jul 2018 00:31:53 -0700 Received: from david-unc.bj.intel.com ([10.238.145.77]) by fmsmga004.fm.intel.com with ESMTP; 09 Jul 2018 00:31:52 -0700 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.51,329,1526367600"; d="scan'208";a="69733974" From: Wang Huaqiang To: libvir-list@redhat.com Date: Mon, 9 Jul 2018 15:00:49 +0800 Message-Id: <1531119658-18549-2-git-send-email-huaqiang.wang@intel.com> In-Reply-To: <1531119658-18549-1-git-send-email-huaqiang.wang@intel.com> References: <1531119658-18549-1-git-send-email-huaqiang.wang@intel.com> X-Greylist: Sender passed SPF test, Sender IP whitelisted by DNSRBL, ACL 207 matched, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.29]); Mon, 09 Jul 2018 07:31:54 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.29]); Mon, 09 Jul 2018 07:31:54 +0000 (UTC) for IP:'192.55.52.88' DOMAIN:'mga01.intel.com' HELO:'mga01.intel.com' FROM:'huaqiang.wang@intel.com' RCPT:'' X-RedHat-Spam-Score: -5.001 (RCVD_IN_DNSWL_HI, SPF_PASS) 192.55.52.88 mga01.intel.com 192.55.52.88 mga01.intel.com X-Scanned-By: MIMEDefang 2.78 on 10.5.110.29 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.24 X-loop: libvir-list@redhat.com Cc: shaohe.feng@intel.com, Wang Huaqiang , bing.niu@intel.com, jian-feng.ding@intel.com, rui.zang@intel.com Subject: [libvirt] [PATCHv2 01/10] util: add Intel x86 RDT/CMT support X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.84 on 10.5.11.26 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.44]); Mon, 09 Jul 2018 07:32:16 +0000 (UTC) X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" Add RDT/CMT feature (Intel x86) by interacting with kernel resctrl file system. Integrate code into util/resctrl. --- src/libvirt_private.syms | 10 ++ src/util/virresctrl.c | 338 +++++++++++++++++++++++++++++++++++++++++++= ++++ src/util/virresctrl.h | 40 ++++++ 3 files changed, 388 insertions(+) diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index 3e30490..b10a3a5 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -2655,6 +2655,16 @@ virResctrlAllocSetID; virResctrlAllocSetSize; virResctrlInfoGetCache; virResctrlInfoNew; +virResctrlMonAddPID; +virResctrlMonCreate; +virResctrlMonDeterminePath; +virResctrlMonGetAlloc; +virResctrlMonGetCacheOccupancy; +virResctrlMonGetID; +virResctrlMonIsRunning; +virResctrlMonNew; +virResctrlMonRemove; +virResctrlMonSetID; =20 =20 # util/virrotatingfile.h diff --git a/src/util/virresctrl.c b/src/util/virresctrl.c index e492a63..3dca937 100644 --- a/src/util/virresctrl.c +++ b/src/util/virresctrl.c @@ -90,6 +90,7 @@ typedef virResctrlAllocPerLevel *virResctrlAllocPerLevelP= tr; /* Class definitions and initializations */ static virClassPtr virResctrlInfoClass; static virClassPtr virResctrlAllocClass; +static virClassPtr virResctrlMonClass; =20 =20 /* virResctrlInfo */ @@ -257,6 +258,33 @@ virResctrlAllocDispose(void *obj) } =20 =20 + +/* virResctrlMon*/ +struct _virResctrlMon { + virObject parent; + + /* alloc: keeps the pointer to a allocation group if sharing same + * resctrlfs subdirectory with a allocation group. + * NULL for a standalone monitoring group.*/ + virResctrlAllocPtr alloc; + /* The identifier (any unique string for now) */ + char *id; + /* libvirt-generated path, be identical to path of 'alloc' if sharing + * same resctrlfs subdirectory with allocation group */ + char *path; +}; + + +static void +virResctrlMonDispose(void *obj) +{ + virResctrlMonPtr resctrlMon =3D obj; + + VIR_FREE(resctrlMon->id); + VIR_FREE(resctrlMon->path); +} + + /* Global initialization for classes */ static int virResctrlOnceInit(void) @@ -267,6 +295,9 @@ virResctrlOnceInit(void) if (!VIR_CLASS_NEW(virResctrlAlloc, virClassForObject())) return -1; =20 + if (!VIR_CLASS_NEW(virResctrlMon, virClassForObject())) + return -1; + return 0; } =20 @@ -1612,3 +1643,310 @@ virResctrlAllocRemove(virResctrlAllocPtr alloc) =20 return ret; } + + +virResctrlMonPtr +virResctrlMonNew(void) +{ + if (virResctrlInitialize() < 0) + return NULL; + + return virObjectNew(virResctrlMonClass); +} + + +int +virResctrlMonSetID(virResctrlMonPtr mon, + const char *id) +{ + if (!id) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("Resctrl mon group 'id' cannot be NULL")); + return -1; + } + + return VIR_STRDUP(mon->id, id); +} + + +const char * +virResctrlMonGetID(virResctrlMonPtr mon) +{ + return mon->id; +} + + +int +virResctrlMonDeterminePath(virResctrlMonPtr mon, + const char *machinename) +{ + char *grouppath =3D NULL; + int ret =3D -1; + + if (!mon->id) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("Resctrl mon group id must be set before creation")); + goto cleanup; + } + + if (!mon->path) { + if (virAsprintf(&grouppath, "%s/mon_groups/%s-%s", + SYSFS_RESCTRL_PATH, machinename, mon->id) < 0) { + + goto cleanup; + } + VIR_STEAL_PTR(mon->path, grouppath); + + } else { + /* if path exists, validate it */ + if (virAsprintf(&grouppath, "%s/%s-%s", + SYSFS_RESCTRL_PATH, machinename, mon->id) < 0) { + goto cleanup; + } + + if (STRNEQ(mon->path, grouppath)) + goto cleanup; + } + + ret =3D 0; + cleanup: + VIR_FREE(grouppath); + return ret; +} + + +int +virResctrlMonAddPID(virResctrlMonPtr mon, + pid_t pid) +{ + char *tasks =3D NULL; + char *pidstr =3D NULL; + int ret =3D 0; + + /* PID only writes to standalone mon group */ + if (mon->alloc) + return 0; + + if (!mon->path) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("Cannot add pid to non-existing resctrl mon group")); + return -1; + } + + VIR_DEBUG("Add PID %d to domain %s\n", + pid, mon->path); + + if (virAsprintf(&tasks, "%s/tasks", mon->path) < 0) + return -1; + + if (virAsprintf(&pidstr, "%lld", (long long int) pid) < 0) + goto cleanup; + + if (virFileWriteStr(tasks, pidstr, 0) < 0) { + virReportSystemError(errno, + _("Cannot write pid in tasks file '%s'"), + tasks); + goto cleanup; + } + + ret =3D 0; + cleanup: + VIR_FREE(tasks); + VIR_FREE(pidstr); + return ret; +} + + +int +virResctrlMonCreate(virResctrlAllocPtr alloc, + virResctrlMonPtr mon, + const char *machinename) +{ + int ret =3D -1; + int lockfd =3D -1; + + if (!mon) + return -1; + + if (alloc) { + if (!virFileExists(alloc->path)) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("resctrl mon group: allocation group exists but not " + "valid")); + return -1; + } + mon->alloc =3D alloc; + + if (virResctrlMonDeterminePath(mon, machinename) < 0) + return -1; + } else { + if (virResctrlMonDeterminePath(mon, machinename) < 0) + return -1; + + lockfd =3D virResctrlLockWrite(); + if (lockfd < 0) + goto cleanup; + + if (virFileExists(mon->path)) { + VIR_DEBUG("Removing resctrl mon group %s", mon->path); + if (rmdir(mon->path) !=3D 0 && errno !=3D ENOENT) { + virReportSystemError(errno, + _("Unable to remove resctrl directory= '%s'"), + mon->path); + goto cleanup; + } + } + + if (virFileMakePath(mon->path) < 0) { + virReportSystemError(errno, + _("Cannot create resctrl directory '%s'"), + mon->path); + goto cleanup; + } + } + + ret =3D 0; + cleanup: + virResctrlUnlock(lockfd); + return ret; +} + + +int +virResctrlMonRemove(virResctrlMonPtr mon) +{ + int ret =3D 0; + + if (!mon->path) + return 0; + + VIR_DEBUG("Removing resctrl mon group %s", mon->path); + if (rmdir(mon->path) !=3D 0 && errno !=3D ENOENT) { + ret =3D -errno; + VIR_ERROR(_("Unable to remove %s (%d)"), mon->path, errno); + } + + return ret; +} + + +bool +virResctrlMonIsRunning(virResctrlMonPtr mon) +{ + bool ret =3D false; + int rv =3D -1; + char *tasks =3D NULL; + + if (mon && virFileExists(mon->path)) { + rv =3D virFileReadValueString(&tasks, "%s/tasks", mon->path); + if (rv < 0) + goto cleanup; + + if (!tasks || !tasks[0]) + goto cleanup; + + ret =3D true; + } + + cleanup: + VIR_FREE(tasks); + return ret; +} + + +static int +virResctrlMonGetStatistic(virResctrlMonPtr mon, + const char *resfile, + unsigned int *value) +{ + DIR *dirp =3D NULL; + int ret =3D -1; + int rv =3D -1; + struct dirent *ent =3D NULL; + unsigned int val =3D 0; + unsigned int valtotal =3D 0; + virBuffer buf =3D VIR_BUFFER_INITIALIZER; + char *mondatapath =3D NULL; + + if (!mon->path) + goto cleanup; + + if (!resfile) + goto cleanup; + + *value =3D 0; + + rv =3D virDirOpenIfExists(&dirp, mon->path); + if (rv <=3D 0) + goto cleanup; + + virBufferAsprintf(&buf, "%s/mon_data", + mon->path); + mondatapath =3D virBufferContentAndReset(&buf); + if (!mondatapath) + goto cleanup; + + VIR_DEBUG("Seek llc_occupancy file from root: %s ", + mondatapath); + + if (virDirOpen(&dirp, mondatapath) < 0) + goto cleanup; + + while ((rv =3D virDirRead(dirp, &ent, mondatapath)) > 0) { + VIR_DEBUG("Parsing file '%s'", ent->d_name); + if (ent->d_type !=3D DT_DIR) + continue; + + if (STRNEQLEN(ent->d_name, "mon_L", 5)) + continue; + + rv =3D virFileReadValueUint(&val, + "%s/%s/%s", + mondatapath, ent->d_name, resfile); + + if (rv =3D=3D -2) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("file %s/%s/%s does not exist"), + mondatapath, ent->d_name, resfile); + goto cleanup; + } else { + if (rv < 0) + goto cleanup; + } + + valtotal +=3D val; + } + + *value =3D valtotal; + ret =3D 0; + cleanup: + VIR_FREE(mondatapath); + VIR_DIR_CLOSE(dirp); + return ret; +} + +int +virResctrlMonGetCacheOccupancy(virResctrlMonPtr mon, + unsigned int * cacheoccu) +{ + const char *cacheoccufile =3D "llc_occupancy"; + unsigned int value =3D 0; + int ret =3D - 1; + + *cacheoccu =3D 0; + + ret =3D virResctrlMonGetStatistic(mon, + cacheoccufile, + &value); + if (ret >=3D 0) + *cacheoccu =3D value; + + return ret; +} + + +virResctrlAllocPtr +virResctrlMonGetAlloc(virResctrlMonPtr mon) +{ + return mon->alloc; +} diff --git a/src/util/virresctrl.h b/src/util/virresctrl.h index 9052a2b..6865ab6 100644 --- a/src/util/virresctrl.h +++ b/src/util/virresctrl.h @@ -116,4 +116,44 @@ virResctrlAllocAddPID(virResctrlAllocPtr alloc, int virResctrlAllocRemove(virResctrlAllocPtr alloc); =20 + +/* Monitoring-related things */ +typedef struct _virResctrlMon virResctrlMon; +typedef virResctrlMon *virResctrlMonPtr; + +virResctrlMonPtr +virResctrlMonNew(void); + +int +virResctrlMonSetID(virResctrlMonPtr mon, + const char *id); + +const char * +virResctrlMonGetID(virResctrlMonPtr mon); + +int +virResctrlMonDeterminePath(virResctrlMonPtr mon, + const char *machinename); + +int +virResctrlMonAddPID(virResctrlMonPtr alloc, + pid_t pid); + +int +virResctrlMonCreate(virResctrlAllocPtr pairedalloc, + virResctrlMonPtr mon, + const char *machinename); + +int +virResctrlMonRemove(virResctrlMonPtr mon); + +bool +virResctrlMonIsRunning(virResctrlMonPtr mon); + +int +virResctrlMonGetCacheOccupancy(virResctrlMonPtr mon, + unsigned int * cacheoccu); + +virResctrlAllocPtr +virResctrlMonGetAlloc(virResctrlMonPtr mon); #endif /* __VIR_RESCTRL_H__ */ --=20 2.7.4 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list