From nobody Fri Apr 26 04:36:46 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.25 as permitted sender) client-ip=209.132.183.25; envelope-from=libvir-list-bounces@redhat.com; helo=mx4-phx2.redhat.com; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.25 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; Return-Path: Received: from mx4-phx2.redhat.com (mx4-phx2.redhat.com [209.132.183.25]) by mx.zohomail.com with SMTPS id 1486709113779516.4943607836263; Thu, 9 Feb 2017 22:45:13 -0800 (PST) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by mx4-phx2.redhat.com (8.13.8/8.13.8) with ESMTP id v1A6fKoP017787; Fri, 10 Feb 2017 01:41:20 -0500 Received: from int-mx10.intmail.prod.int.phx2.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id v1A6enAA009980 for ; Fri, 10 Feb 2017 01:40:49 -0500 Received: from mx1.redhat.com (ext-mx10.extmail.prod.ext.phx2.redhat.com [10.5.110.39]) by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v1A6emar032530 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Fri, 10 Feb 2017 01:40:48 -0500 Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 2ECA361BA5; Fri, 10 Feb 2017 06:40:47 +0000 (UTC) Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga102.jf.intel.com with ESMTP; 09 Feb 2017 22:40:45 -0800 Received: from s2600wt.sh.intel.com ([10.239.48.158]) by fmsmga001.fm.intel.com with ESMTP; 09 Feb 2017 22:40:44 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.35,139,1484035200"; d="scan'208";a="1105518477" From: Eli Qiao To: libvir-list@redhat.com Date: Fri, 10 Feb 2017 14:42:13 +0800 Message-Id: <1486708933-5731-10-git-send-email-liyong.qiao@intel.com> In-Reply-To: <1486708933-5731-1-git-send-email-liyong.qiao@intel.com> References: <1486708933-5731-1-git-send-email-liyong.qiao@intel.com> X-Greylist: Sender passed SPF test, Sender IP whitelisted by DNSRBL, ACL 200 matched, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.39]); Fri, 10 Feb 2017 06:40:47 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.39]); Fri, 10 Feb 2017 06:40:47 +0000 (UTC) for IP:'134.134.136.24' DOMAIN:'mga09.intel.com' HELO:'mga09.intel.com' FROM:'liyong.qiao@intel.com' RCPT:'' X-RedHat-Spam-Score: -4.222 (BAYES_50, RCVD_IN_DNSWL_HI, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, RP_MATCHES_RCVD, SPF_PASS) 134.134.136.24 mga09.intel.com 134.134.136.24 mga09.intel.com X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23 X-Scanned-By: MIMEDefang 2.78 on 10.5.110.39 X-loop: libvir-list@redhat.com Cc: mkletzan@redhat.com, qiaowei.ren@intel.com, mtosatti@redhat.com, qiaoliyong@gmail.com Subject: [libvirt] [v4 9/9] Resctrl: concurrence support X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" The internal struct list domainall is a list which are resctral domain status shared by all VMs, especiall the default domain, each VM should access it concomitantly. Ues a mutex to control it. Each bank's cache_left field is also a global shared resource we need to be care, add a mutex for each bank. We need also to add lock to access /sys/fs/resctrl, use virFileLock. Signed-off-by: Eli Qiao --- src/util/virresctrl.c | 84 +++++++++++++++++++++++++++++++++++++++++++++--= ---- src/util/virresctrl.h | 3 ++ 2 files changed, 78 insertions(+), 9 deletions(-) diff --git a/src/util/virresctrl.c b/src/util/virresctrl.c index 3552761..e949498 100644 --- a/src/util/virresctrl.c +++ b/src/util/virresctrl.c @@ -114,6 +114,8 @@ typedef virResCtrlDomain *virResCtrlDomainPtr; struct _virResCtrlDomain { unsigned int num_domains; virResDomainPtr domains; + + virMutex lock; }; =20 =20 @@ -166,16 +168,34 @@ static int virResCtrlGetStr(const char *domain_name, = const char *item_name, char { char *path; int rc =3D 0; + int readfd; + int len; =20 CONSTRUCT_RESCTRL_PATH(domain_name, item_name); =20 - if (virFileReadAll(path, MAX_FILE_LEN, ret) < 0) { - rc =3D -1; + if (!virFileExists(path)) + goto cleanup; + + if ((readfd =3D open(path, O_RDONLY)) < 0) + goto cleanup; + + /* the lock will be released after readfd closed */ + if (virFileLock(readfd, true, 0, 1, true) < 0) { + virReportSystemError(errno, _("Unable to lock '%s'"), path); + goto cleanup; + } + + len =3D virFileReadLimFD(readfd, MAX_FILE_LEN, ret); + + if (len < 0) { + virReportSystemError(errno, _("Failed to read file '%s'"), path); goto cleanup; } + rc =3D 0; =20 cleanup: VIR_FREE(path); + VIR_FORCE_CLOSE(readfd); return rc; } =20 @@ -616,6 +636,9 @@ virResCtrlRefresh(void) unsigned int origin_count =3D domainall.num_domains; virResDomainPtr p, pre, del; pre =3D domainall.domains; + + virMutexLock(&domainall.lock); + p =3D del =3D NULL; if (pre) p =3D pre->next; @@ -640,10 +663,11 @@ virResCtrlRefresh(void) p =3D p->next; } VIR_FREE(tasks); - } =20 - return virResCtrlRefreshSchemata(); + virResCtrlRefreshSchemata(); + virMutexUnlock(&domainall.lock); + return 0; } =20 /* Get a domain ptr by domain's name*/ @@ -689,6 +713,12 @@ virResCtrlWrite(const char *name, const char *item, co= nst char *content) if ((writefd =3D open(path, O_WRONLY | O_APPEND, S_IRUSR | S_IWUSR)) <= 0) goto cleanup; =20 + /* the lock will be released after writefd closed */ + if (virFileLock(writefd, false, 0, 0, true) < 0) { + virReportSystemError(errno, _("Unable to lock '%s'"), path); + goto cleanup; + } + if (safewrite(writefd, content, strlen(content)) < 0) goto cleanup; =20 @@ -868,10 +898,15 @@ static int virResCtrlAppendDomain(virResDomainPtr dom) { virResDomainPtr p =3D domainall.domains; + + virMutexLock(&domainall.lock); + while (p->next !=3D NULL) p =3D p->next; p->next =3D dom; dom->pre =3D p; domainall.num_domains +=3D 1; + + virMutexUnlock(&domainall.lock); return 0; } =20 @@ -894,18 +929,22 @@ virResCtrlCalculateSchemata(int type, { size_t i; int count; + int rc =3D -1; virResDomainPtr p; unsigned int tmp_schemata; unsigned int schemata_sum =3D 0; int pair_type =3D 0; =20 + virMutexLock(&resctrlall[type].cache_banks[sid].lock); + if (resctrlall[type].cache_banks[sid].cache_left < size) { VIR_ERROR(_("Not enough cache left on bank %u"), hostid); - return -1; + goto cleanup; } + if ((count =3D size / resctrlall[type].cache_banks[sid].cache_min) <= =3D 0) { VIR_ERROR(_("Error cache size %llu"), size); - return -1; + goto cleanup; } =20 tmp_schemata =3D VIR_RESCTRL_GET_SCHEMATA(count); @@ -920,7 +959,7 @@ virResCtrlCalculateSchemata(int type, if (type =3D=3D VIR_RDT_RESOURCE_L3CODE) pair_type =3D VIR_RDT_RESOURCE_L3DATA; =20 - for (i =3D 1; i < domainall.num_domains; i ++) { + for (i =3D 1; i < domainall.num_domains; i++) { schemata_sum |=3D p->schematas[type]->schemata_items[sid].schemata; if (pair_type > 0) schemata_sum |=3D p->schematas[pair_type]->schemata_items[sid]= .schemata; @@ -931,7 +970,16 @@ virResCtrlCalculateSchemata(int type, =20 while ((tmp_schemata & schemata_sum) !=3D 0) tmp_schemata =3D tmp_schemata >> 1; - return tmp_schemata; + + resctrlall[type].cache_banks[sid].cache_left -=3D size; + if (pair_type > 0) + resctrlall[pair_type].cache_banks[sid].cache_left =3D resctrlall[t= ype].cache_banks[sid].cache_left; + + rc =3D tmp_schemata; + + cleanup: + virMutexUnlock(&resctrlall[type].cache_banks[sid].lock); + return rc; } =20 int virResCtrlSetCacheBanks(virDomainCachetunePtr cachetune, @@ -1043,7 +1091,7 @@ int virResCtrlUpdate(void) int virResCtrlInit(void) { - size_t i =3D 0; + size_t i, j; char *tmp; int rc =3D 0; =20 @@ -1063,6 +1111,24 @@ virResCtrlInit(void) =20 domainall.domains =3D virResCtrlGetAllDomains(&(domainall.num_domains)= ); =20 + for (i =3D 0; i < VIR_RDT_RESOURCE_LAST; i++) { + if (VIR_RESCTRL_ENABLED(i)) { + for (j =3D 0; j < resctrlall[i].num_banks; j++) { + if (virMutexInit(&resctrlall[i].cache_banks[j].lock) < 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("Unable to initialize mutex")); + return -1; + } + } + } + } + + if (virMutexInit(&domainall.lock) < 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("Unable to initialize mutex")); + return -1; + } + if ((rc =3D virResCtrlRefresh()) < 0) VIR_ERROR(_("Failed to refresh resource control")); return rc; diff --git a/src/util/virresctrl.h b/src/util/virresctrl.h index ee7e115..07b298d 100644 --- a/src/util/virresctrl.h +++ b/src/util/virresctrl.h @@ -27,6 +27,7 @@ # include "virutil.h" # include "virbitmap.h" # include "domain_conf.h" +# include "virthread.h" =20 #define MAX_CPU_SOCKET_NUM 8 =20 @@ -49,6 +50,8 @@ struct _virResCacheBank { unsigned long long cache_left; unsigned long long cache_min; virBitmapPtr cpu_mask; + + virMutex lock; }; =20 /** --=20 1.9.1 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list