From nobody Sun May 5 13:15:08 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) client-ip=170.10.133.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail(p=none dis=none) header.from=linux.microsoft.com Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mx.zohomail.com with SMTPS id 1639168528974871.1379321216864; Fri, 10 Dec 2021 12:35:28 -0800 (PST) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-158-gonkoO74MH2hlvzNkvXeTw-1; Fri, 10 Dec 2021 15:35:24 -0500 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 1E827192FDA2; Fri, 10 Dec 2021 20:35:20 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id F3AED45D61; Fri, 10 Dec 2021 20:35:19 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id B2A0E180B617; Fri, 10 Dec 2021 20:35:19 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 1BAKYvAh008341 for ; Fri, 10 Dec 2021 15:34:57 -0500 Received: by smtp.corp.redhat.com (Postfix) id 8ECB3400E127; Fri, 10 Dec 2021 20:34:57 +0000 (UTC) Received: from mimecast-mx02.redhat.com (mimecast08.extmail.prod.ext.rdu2.redhat.com [10.11.55.24]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 8A5A0400F3F1 for ; Fri, 10 Dec 2021 20:34:57 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-1.mimecast.com [207.211.31.81]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 6A28D39D30B9 for ; Fri, 10 Dec 2021 20:34:57 +0000 (UTC) Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by relay.mimecast.com with ESMTP id us-mta-143-z_qiVSZjPpinCrrTHEARAQ-1; Fri, 10 Dec 2021 15:34:55 -0500 Received: from prapal-ch2.oiwy50ateaxezb1sqsoezlib2f.xx.internal.cloudapp.net (unknown [20.80.162.67]) by linux.microsoft.com (Postfix) with ESMTPSA id 34A3220B717A; Fri, 10 Dec 2021 12:34:54 -0800 (PST) X-MC-Unique: gonkoO74MH2hlvzNkvXeTw-1 X-MC-Unique: z_qiVSZjPpinCrrTHEARAQ-1 DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 34A3220B717A From: Praveen K Paladugu To: libvir-list@redhat.com Subject: [libvirt PATCH v3 01/13] util: Helper functions to get process info Date: Fri, 10 Dec 2021 20:34:36 +0000 Message-Id: <20211210203448.1579606-2-prapal@linux.microsoft.com> In-Reply-To: <20211210203448.1579606-1-prapal@linux.microsoft.com> References: <20211210203448.1579606-1-prapal@linux.microsoft.com> MIME-Version: 1.0 X-Mimecast-Impersonation-Protect: Policy=CLT - Impersonation Protection Definition; Similar Internal Domain=false; Similar Monitored External Domain=false; Custom External Domain=false; Mimecast External Domain=false; Newly Observed Domain=false; Internal User Name=false; Custom Display Name List=false; Reply-to Address Mismatch=false; Targeted Threat Dictionary=false; Mimecast Threat Dictionary=false; Custom Threat Dictionary=false X-Scanned-By: MIMEDefang 2.84 on 10.11.54.2 X-MIME-Autoconverted: from quoted-printable to 8bit by lists01.pubmisc.prod.ext.phx2.redhat.com id 1BAKYvAh008341 X-loop: libvir-list@redhat.com Cc: william.douglas@intel.com X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZM-MESSAGEID: 1639168529554100001 Content-Type: text/plain; charset="utf-8" Move qemuGetProcessInfo and qemuGetSchedInfo methods to util and share them with ch driver. Signed-off-by: Praveen K Paladugu Reviewed-by: Michal Privoznik --- src/libvirt_private.syms | 2 + src/qemu/qemu_driver.c | 116 ++------------------------------------- src/util/virprocess.c | 108 ++++++++++++++++++++++++++++++++++++ src/util/virprocess.h | 5 ++ 4 files changed, 120 insertions(+), 111 deletions(-) diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index da27ee7b53..56adc192cd 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -3101,8 +3101,10 @@ virProcessGetAffinity; virProcessGetMaxMemLock; virProcessGetNamespaces; virProcessGetPids; +virProcessGetSchedInfo; virProcessGetStartTime; virProcessGetStat; +virProcessGetStatInfo; virProcessGroupGet; virProcessGroupKill; virProcessKill; diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index e444ad2d45..ba3efef42b 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -1317,113 +1317,6 @@ qemuGetSchedstatDelay(unsigned long long *cpudelay, return 0; } =20 - -static int -qemuGetSchedInfo(unsigned long long *cpuWait, - pid_t pid, pid_t tid) -{ - g_autofree char *proc =3D NULL; - g_autofree char *data =3D NULL; - g_auto(GStrv) lines =3D NULL; - size_t i; - double val; - - *cpuWait =3D 0; - - /* In general, we cannot assume pid_t fits in int; but /proc parsing - * is specific to Linux where int works fine. */ - if (tid) - proc =3D g_strdup_printf("/proc/%d/task/%d/sched", (int)pid, (int)= tid); - else - proc =3D g_strdup_printf("/proc/%d/sched", (int)pid); - if (!proc) - return -1; - - /* The file is not guaranteed to exist (needs CONFIG_SCHED_DEBUG) */ - if (access(proc, R_OK) < 0) { - return 0; - } - - if (virFileReadAll(proc, (1<<16), &data) < 0) - return -1; - - lines =3D g_strsplit(data, "\n", 0); - if (!lines) - return -1; - - for (i =3D 0; lines[i] !=3D NULL; i++) { - const char *line =3D lines[i]; - - /* Needs CONFIG_SCHEDSTATS. The second check - * is the old name the kernel used in past */ - if (STRPREFIX(line, "se.statistics.wait_sum") || - STRPREFIX(line, "se.wait_sum")) { - line =3D strchr(line, ':'); - if (!line) { - virReportError(VIR_ERR_INTERNAL_ERROR, - _("Missing separator in sched info '%s'"), - lines[i]); - return -1; - } - line++; - while (*line =3D=3D ' ') - line++; - - if (virStrToDouble(line, NULL, &val) < 0) { - virReportError(VIR_ERR_INTERNAL_ERROR, - _("Unable to parse sched info value '%s'"), - line); - return -1; - } - - *cpuWait =3D (unsigned long long)(val * 1000000); - break; - } - } - - return 0; -} - - -static int -qemuGetProcessInfo(unsigned long long *cpuTime, int *lastCpu, long *vm_rss, - pid_t pid, pid_t tid) -{ - g_auto(GStrv) proc_stat =3D virProcessGetStat(pid, tid); - unsigned long long usertime =3D 0, systime =3D 0; - long rss =3D 0; - int cpu =3D 0; - - if (!proc_stat || - virStrToLong_ullp(proc_stat[VIR_PROCESS_STAT_UTIME], NULL, 10, &us= ertime) < 0 || - virStrToLong_ullp(proc_stat[VIR_PROCESS_STAT_STIME], NULL, 10, &sy= stime) < 0 || - virStrToLong_l(proc_stat[VIR_PROCESS_STAT_RSS], NULL, 10, &rss) < = 0 || - virStrToLong_i(proc_stat[VIR_PROCESS_STAT_PROCESSOR], NULL, 10, &c= pu) < 0) { - VIR_WARN("cannot parse process status data"); - } - - /* We got jiffies - * We want nanoseconds - * _SC_CLK_TCK is jiffies per second - * So calculate thus.... - */ - if (cpuTime) - *cpuTime =3D 1000ull * 1000ull * 1000ull * (usertime + systime) - / (unsigned long long)sysconf(_SC_CLK_TCK); - if (lastCpu) - *lastCpu =3D cpu; - - if (vm_rss) - *vm_rss =3D rss * virGetSystemPageSizeKB(); - - - VIR_DEBUG("Got status for %d/%d user=3D%llu sys=3D%llu cpu=3D%d rss=3D= %ld", - (int)pid, tid, usertime, systime, cpu, rss); - - return 0; -} - - static int qemuDomainHelperGetVcpus(virDomainObj *vm, virVcpuInfoPtr info, @@ -1463,7 +1356,7 @@ qemuDomainHelperGetVcpus(virDomainObj *vm, vcpuinfo->number =3D i; vcpuinfo->state =3D VIR_VCPU_RUNNING; =20 - if (qemuGetProcessInfo(&vcpuinfo->cpuTime, + if (virProcessGetStatInfo(&vcpuinfo->cpuTime, &vcpuinfo->cpu, NULL, vm->pid, vcpupid) < 0) { virReportSystemError(errno, "%s", @@ -1483,7 +1376,7 @@ qemuDomainHelperGetVcpus(virDomainObj *vm, } =20 if (cpuwait) { - if (qemuGetSchedInfo(&(cpuwait[ncpuinfo]), vm->pid, vcpupid) <= 0) + if (virProcessGetSchedInfo(&(cpuwait[ncpuinfo]), vm->pid, vcpu= pid) < 0) return -1; } =20 @@ -2626,7 +2519,8 @@ qemuDomainGetInfo(virDomainPtr dom, } =20 if (virDomainObjIsActive(vm)) { - if (qemuGetProcessInfo(&(info->cpuTime), NULL, NULL, vm->pid, 0) <= 0) { + if (virProcessGetStatInfo(&(info->cpuTime), NULL, NULL, + vm->pid, 0) < 0) { virReportError(VIR_ERR_OPERATION_FAILED, "%s", _("cannot read cputime for domain")); goto cleanup; @@ -10623,7 +10517,7 @@ qemuDomainMemoryStatsInternal(virQEMUDriver *driver, ret =3D 0; } =20 - if (qemuGetProcessInfo(NULL, NULL, &rss, vm->pid, 0) < 0) { + if (virProcessGetStatInfo(NULL, NULL, &rss, vm->pid, 0) < 0) { virReportError(VIR_ERR_OPERATION_FAILED, "%s", _("cannot get RSS for domain")); } else { diff --git a/src/util/virprocess.c b/src/util/virprocess.c index 7b0ad9c97b..3be9080b67 100644 --- a/src/util/virprocess.c +++ b/src/util/virprocess.c @@ -1764,3 +1764,111 @@ virProcessGetStat(pid_t pid, =20 return ret; } + + +int +virProcessGetStatInfo(unsigned long long *cpuTime, int *lastCpu, long *vm_= rss, + pid_t pid, pid_t tid) +{ + g_auto(GStrv) proc_stat =3D virProcessGetStat(pid, tid); + unsigned long long usertime =3D 0, systime =3D 0; + long rss =3D 0; + int cpu =3D 0; + + if (!proc_stat || + virStrToLong_ullp(proc_stat[VIR_PROCESS_STAT_UTIME], NULL, 10, + &usertime) < 0 + || virStrToLong_ullp(proc_stat[VIR_PROCESS_STAT_STIME], NULL, 10, + &systime) < 0 + || virStrToLong_l(proc_stat[VIR_PROCESS_STAT_RSS], NULL, 10, &rss)= < 0 + || virStrToLong_i(proc_stat[VIR_PROCESS_STAT_PROCESSOR], NULL, 10, + &cpu) < 0) { + VIR_WARN("cannot parse process status data"); + } + + /* We got jiffies + * We want nanoseconds + * _SC_CLK_TCK is jiffies per second + * So calculate thus.... + */ + if (cpuTime) + *cpuTime =3D 1000ull * 1000ull * 1000ull * (usertime + systime) + / (unsigned long long) sysconf(_SC_CLK_TCK); + if (lastCpu) + *lastCpu =3D cpu; + + if (vm_rss) + *vm_rss =3D rss * virGetSystemPageSizeKB(); + + + VIR_DEBUG("Got status for %d/%d user=3D%llu sys=3D%llu cpu=3D%d rss=3D= %ld", + (int) pid, tid, usertime, systime, cpu, rss); + + return 0; +} + +int +virProcessGetSchedInfo(unsigned long long *cpuWait, pid_t pid, pid_t tid) +{ + g_autofree char *proc =3D NULL; + g_autofree char *data =3D NULL; + + g_auto(GStrv) lines =3D NULL; + size_t i; + double val; + + *cpuWait =3D 0; + + /* In general, we cannot assume pid_t fits in int; but /proc parsing + * is specific to Linux where int works fine. */ + if (tid) + proc =3D g_strdup_printf("/proc/%d/task/%d/sched", (int) pid, (int= ) tid); + else + proc =3D g_strdup_printf("/proc/%d/sched", (int) pid); + if (!proc) + return -1; + + /* The file is not guaranteed to exist (needs CONFIG_SCHED_DEBUG) */ + if (access(proc, R_OK) < 0) { + return 0; + } + + if (virFileReadAll(proc, (1 << 16), &data) < 0) + return -1; + + lines =3D g_strsplit(data, "\n", 0); + if (!lines) + return -1; + + for (i =3D 0; lines[i] !=3D NULL; i++) { + const char *line =3D lines[i]; + + /* Needs CONFIG_SCHEDSTATS. The second check + * is the old name the kernel used in past */ + if (STRPREFIX(line, "se.statistics.wait_sum") || + STRPREFIX(line, "se.wait_sum")) { + line =3D strchr(line, ':'); + if (!line) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Missing separator in sched info '%s'"), + lines[i]); + return -1; + } + line++; + while (*line =3D=3D ' ') + line++; + + if (virStrToDouble(line, NULL, &val) < 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Unable to parse sched info value '%s'"), + line); + return -1; + } + + *cpuWait =3D (unsigned long long) (val * 1000000); + break; + } + } + + return 0; +} diff --git a/src/util/virprocess.h b/src/util/virprocess.h index 82b7403964..d9d27c29b8 100644 --- a/src/util/virprocess.h +++ b/src/util/virprocess.h @@ -193,3 +193,8 @@ typedef enum { } virProcessNamespaceFlags; =20 int virProcessNamespaceAvailable(unsigned int ns); + +int virProcessGetStatInfo(unsigned long long *cpuTime, int *lastCpu, + long *vm_rss, pid_t pid, pid_t tid); +int virProcessGetSchedInfo(unsigned long long *cpuWait, pid_t pid, + pid_t tid); --=20 2.27.0 From nobody Sun May 5 13:15:08 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) client-ip=170.10.133.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail(p=none dis=none) header.from=linux.microsoft.com Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mx.zohomail.com with SMTPS id 1639168537011941.3081940080373; Fri, 10 Dec 2021 12:35:37 -0800 (PST) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-450-m2Vw4UADNhquJVbpIJKsEA-1; Fri, 10 Dec 2021 15:35:34 -0500 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 7323C192FDA3; Fri, 10 Dec 2021 20:35:28 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 5687360657; Fri, 10 Dec 2021 20:35:28 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 2266D4A700; Fri, 10 Dec 2021 20:35:28 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 1BAKZ0PX008405 for ; Fri, 10 Dec 2021 15:35:00 -0500 Received: by smtp.corp.redhat.com (Postfix) id 54BA92166B2D; Fri, 10 Dec 2021 20:35:00 +0000 (UTC) Received: from mimecast-mx02.redhat.com (mimecast10.extmail.prod.ext.rdu2.redhat.com [10.11.55.26]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 4FF4F2166B26 for ; Fri, 10 Dec 2021 20:34:57 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [207.211.31.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 439B91C8C27A for ; Fri, 10 Dec 2021 20:34:57 +0000 (UTC) Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by relay.mimecast.com with ESMTP id us-mta-211-adwd8SyTOomf7GFgcPUT7A-1; Fri, 10 Dec 2021 15:34:55 -0500 Received: from prapal-ch2.oiwy50ateaxezb1sqsoezlib2f.xx.internal.cloudapp.net (unknown [20.80.162.67]) by linux.microsoft.com (Postfix) with ESMTPSA id 4B34420B717B; Fri, 10 Dec 2021 12:34:54 -0800 (PST) X-MC-Unique: m2Vw4UADNhquJVbpIJKsEA-1 X-MC-Unique: adwd8SyTOomf7GFgcPUT7A-1 DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 4B34420B717B From: Praveen K Paladugu To: libvir-list@redhat.com Subject: [libvirt PATCH v3 02/13] ch_domain: add virCHDomainGetMonitor helper method Date: Fri, 10 Dec 2021 20:34:37 +0000 Message-Id: <20211210203448.1579606-3-prapal@linux.microsoft.com> In-Reply-To: <20211210203448.1579606-1-prapal@linux.microsoft.com> References: <20211210203448.1579606-1-prapal@linux.microsoft.com> MIME-Version: 1.0 X-Mimecast-Impersonation-Protect: Policy=CLT - Impersonation Protection Definition; Similar Internal Domain=false; Similar Monitored External Domain=false; Custom External Domain=false; Mimecast External Domain=false; Newly Observed Domain=false; Internal User Name=false; Custom Display Name List=false; Reply-to Address Mismatch=false; Targeted Threat Dictionary=false; Mimecast Threat Dictionary=false; Custom Threat Dictionary=false X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 X-MIME-Autoconverted: from quoted-printable to 8bit by lists01.pubmisc.prod.ext.phx2.redhat.com id 1BAKZ0PX008405 X-loop: libvir-list@redhat.com Cc: william.douglas@intel.com X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZM-MESSAGEID: 1639168538790100001 Content-Type: text/plain; charset="utf-8" From: Vineeth Pillai Signed-off-by: Vineeth Pillai Signed-off-by: Praveen K Paladugu Reviewed-by: Michal Privoznik --- src/ch/ch_domain.c | 6 ++++++ src/ch/ch_domain.h | 5 +++++ 2 files changed, 11 insertions(+) diff --git a/src/ch/ch_domain.c b/src/ch/ch_domain.c index dd4de9e1ea..bf4ce83595 100644 --- a/src/ch/ch_domain.c +++ b/src/ch/ch_domain.c @@ -290,3 +290,9 @@ virDomainDefParserConfig virCHDriverDomainDefParserConf= ig =3D { .domainPostParseCallback =3D virCHDomainDefPostParse, .deviceValidateCallback =3D chValidateDomainDeviceDef, }; + +virCHMonitor * +virCHDomainGetMonitor(virDomainObj *vm) +{ + return CH_DOMAIN_PRIVATE(vm)->monitor; +} diff --git a/src/ch/ch_domain.h b/src/ch/ch_domain.h index 61b34b0467..c053b25c65 100644 --- a/src/ch/ch_domain.h +++ b/src/ch/ch_domain.h @@ -57,6 +57,11 @@ struct _virCHDomainObjPrivate { virChrdevs *chrdevs; }; =20 +#define CH_DOMAIN_PRIVATE(vm) \ + ((virCHDomainObjPrivate*)(vm)->privateData) + +virCHMonitor *virCHDomainGetMonitor(virDomainObj *vm); + extern virDomainXMLPrivateDataCallbacks virCHDriverPrivateDataCallbacks; extern virDomainDefParserConfig virCHDriverDomainDefParserConfig; =20 --=20 2.27.0 From nobody Sun May 5 13:15:08 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) client-ip=170.10.129.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail(p=none dis=none) header.from=linux.microsoft.com Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mx.zohomail.com with SMTPS id 163916851682011.482999914825655; Fri, 10 Dec 2021 12:35:16 -0800 (PST) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-257--EeOfXP6NtKbzJ2CKE4EVQ-1; Fri, 10 Dec 2021 15:35:12 -0500 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 02938192FDA0; Fri, 10 Dec 2021 20:35:05 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id A738545D71; Fri, 10 Dec 2021 20:35:04 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 6CD764BB7C; Fri, 10 Dec 2021 20:35:00 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 1BAKYvam008340 for ; Fri, 10 Dec 2021 15:34:57 -0500 Received: by smtp.corp.redhat.com (Postfix) id 5A072401DBD; Fri, 10 Dec 2021 20:34:57 +0000 (UTC) Received: from mimecast-mx02.redhat.com (mimecast07.extmail.prod.ext.rdu2.redhat.com [10.11.55.23]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 56755401DB9 for ; Fri, 10 Dec 2021 20:34:57 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [205.139.110.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 3DC8A3C9A02D for ; Fri, 10 Dec 2021 20:34:57 +0000 (UTC) Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by relay.mimecast.com with ESMTP id us-mta-584-8dL7vj_4PWqWs_QXlnsGwA-1; Fri, 10 Dec 2021 15:34:55 -0500 Received: from prapal-ch2.oiwy50ateaxezb1sqsoezlib2f.xx.internal.cloudapp.net (unknown [20.80.162.67]) by linux.microsoft.com (Postfix) with ESMTPSA id 617AC20B7185; Fri, 10 Dec 2021 12:34:54 -0800 (PST) X-MC-Unique: -EeOfXP6NtKbzJ2CKE4EVQ-1 X-MC-Unique: 8dL7vj_4PWqWs_QXlnsGwA-1 DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 617AC20B7185 From: Praveen K Paladugu To: libvir-list@redhat.com Subject: [libvirt PATCH v3 03/13] ch_domain: add methods to manage private vcpu data Date: Fri, 10 Dec 2021 20:34:38 +0000 Message-Id: <20211210203448.1579606-4-prapal@linux.microsoft.com> In-Reply-To: <20211210203448.1579606-1-prapal@linux.microsoft.com> References: <20211210203448.1579606-1-prapal@linux.microsoft.com> MIME-Version: 1.0 X-Mimecast-Impersonation-Protect: Policy=CLT - Impersonation Protection Definition; Similar Internal Domain=false; Similar Monitored External Domain=false; Custom External Domain=false; Mimecast External Domain=false; Newly Observed Domain=false; Internal User Name=false; Custom Display Name List=false; Reply-to Address Mismatch=false; Targeted Threat Dictionary=false; Mimecast Threat Dictionary=false; Custom Threat Dictionary=false X-Scanned-By: MIMEDefang 2.85 on 10.11.54.10 X-MIME-Autoconverted: from quoted-printable to 8bit by lists01.pubmisc.prod.ext.phx2.redhat.com id 1BAKYvam008340 X-loop: libvir-list@redhat.com Cc: william.douglas@intel.com X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZM-MESSAGEID: 1639168517212100005 Content-Type: text/plain; charset="utf-8" From: Vineeth Pillai Signed-off-by: Vineeth Pillai Signed-off-by: Praveen K Paladugu Reviewed-by: Michal Privoznik --- src/ch/ch_domain.c | 50 +++++++++++++++++++++++++++++++++++++++++----- src/ch/ch_domain.h | 11 ++++++++++ 2 files changed, 56 insertions(+), 5 deletions(-) diff --git a/src/ch/ch_domain.c b/src/ch/ch_domain.c index bf4ce83595..36333a8a3f 100644 --- a/src/ch/ch_domain.c +++ b/src/ch/ch_domain.c @@ -164,11 +164,6 @@ virCHDomainObjPrivateFree(void *data) g_free(priv); } =20 -virDomainXMLPrivateDataCallbacks virCHDriverPrivateDataCallbacks =3D { - .alloc =3D virCHDomainObjPrivateAlloc, - .free =3D virCHDomainObjPrivateFree, -}; - static int virCHDomainDefPostParseBasic(virDomainDef *def, void *opaque G_GNUC_UNUSED) @@ -185,6 +180,45 @@ virCHDomainDefPostParseBasic(virDomainDef *def, return 0; } =20 +static virClass *virCHDomainVcpuPrivateClass; +static void virCHDomainVcpuPrivateDispose(void *obj); + +static int +virCHDomainVcpuPrivateOnceInit(void) +{ + if (!VIR_CLASS_NEW(virCHDomainVcpuPrivate, virClassForObject())) + return -1; + + return 0; +} + +VIR_ONCE_GLOBAL_INIT(virCHDomainVcpuPrivate); + +static virObject * +virCHDomainVcpuPrivateNew(void) +{ + virCHDomainVcpuPrivate *priv; + + if (virCHDomainVcpuPrivateInitialize() < 0) + return NULL; + + if (!(priv =3D virObjectNew(virCHDomainVcpuPrivateClass))) + return NULL; + + return (virObject *) priv; +} + + +static void +virCHDomainVcpuPrivateDispose(void *obj) +{ + virCHDomainVcpuPrivate *priv =3D obj; + + priv->tid =3D 0; + + return; +} + static int virCHDomainDefPostParse(virDomainDef *def, unsigned int parseFlags G_GNUC_UNUSED, @@ -203,6 +237,12 @@ virCHDomainDefPostParse(virDomainDef *def, return 0; } =20 +virDomainXMLPrivateDataCallbacks virCHDriverPrivateDataCallbacks =3D { + .alloc =3D virCHDomainObjPrivateAlloc, + .free =3D virCHDomainObjPrivateFree, + .vcpuNew =3D virCHDomainVcpuPrivateNew, +}; + static int chValidateDomainDeviceDef(const virDomainDeviceDef *dev, const virDomainDef *def G_GNUC_UNUSED, diff --git a/src/ch/ch_domain.h b/src/ch/ch_domain.h index c053b25c65..f01c0e5ad0 100644 --- a/src/ch/ch_domain.h +++ b/src/ch/ch_domain.h @@ -62,6 +62,17 @@ struct _virCHDomainObjPrivate { =20 virCHMonitor *virCHDomainGetMonitor(virDomainObj *vm); =20 +typedef struct _virCHDomainVcpuPrivate virCHDomainVcpuPrivate; +struct _virCHDomainVcpuPrivate { + virObject parent; + + pid_t tid; /* vcpu thread id */ + virTristateBool halted; +}; + +#define CH_DOMAIN_VCPU_PRIVATE(vcpu) \ + ((virCHDomainVcpuPrivate *) (vcpu)->privateData) + extern virDomainXMLPrivateDataCallbacks virCHDriverPrivateDataCallbacks; extern virDomainDefParserConfig virCHDriverDomainDefParserConfig; =20 --=20 2.27.0 From nobody Sun May 5 13:15:08 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) client-ip=170.10.129.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail(p=none dis=none) header.from=linux.microsoft.com Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mx.zohomail.com with SMTPS id 1639168545200239.39019952305057; Fri, 10 Dec 2021 12:35:45 -0800 (PST) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-555-vyV5yZJhNRmD82H9W9tCUw-1; Fri, 10 Dec 2021 15:35:39 -0500 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 169ED86A06B; Fri, 10 Dec 2021 20:35:31 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id EFC7F60657; Fri, 10 Dec 2021 20:35:30 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id C28E31802E27; Fri, 10 Dec 2021 20:35:30 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 1BAKZ0Vp008406 for ; Fri, 10 Dec 2021 15:35:00 -0500 Received: by smtp.corp.redhat.com (Postfix) id 54D042166B2F; Fri, 10 Dec 2021 20:35:00 +0000 (UTC) Received: from mimecast-mx02.redhat.com (mimecast04.extmail.prod.ext.rdu2.redhat.com [10.11.55.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 4F3FD2166B25 for ; Fri, 10 Dec 2021 20:34:57 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [205.139.110.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 428C9101A52D for ; Fri, 10 Dec 2021 20:34:57 +0000 (UTC) Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by relay.mimecast.com with ESMTP id us-mta-568-pdBPO7a8OIKT-VU4PyVevw-1; Fri, 10 Dec 2021 15:34:55 -0500 Received: from prapal-ch2.oiwy50ateaxezb1sqsoezlib2f.xx.internal.cloudapp.net (unknown [20.80.162.67]) by linux.microsoft.com (Postfix) with ESMTPSA id 771E520B7186; Fri, 10 Dec 2021 12:34:54 -0800 (PST) X-MC-Unique: vyV5yZJhNRmD82H9W9tCUw-1 X-MC-Unique: pdBPO7a8OIKT-VU4PyVevw-1 DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 771E520B7186 From: Praveen K Paladugu To: libvir-list@redhat.com Subject: [libvirt PATCH v3 04/13] ch_driver, ch_domain: vcpu info getter callbacks Date: Fri, 10 Dec 2021 20:34:39 +0000 Message-Id: <20211210203448.1579606-5-prapal@linux.microsoft.com> In-Reply-To: <20211210203448.1579606-1-prapal@linux.microsoft.com> References: <20211210203448.1579606-1-prapal@linux.microsoft.com> MIME-Version: 1.0 X-Mimecast-Impersonation-Protect: Policy=CLT - Impersonation Protection Definition; Similar Internal Domain=false; Similar Monitored External Domain=false; Custom External Domain=false; Mimecast External Domain=false; Newly Observed Domain=false; Internal User Name=false; Custom Display Name List=false; Reply-to Address Mismatch=false; Targeted Threat Dictionary=false; Mimecast Threat Dictionary=false; Custom Threat Dictionary=false X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 X-MIME-Autoconverted: from quoted-printable to 8bit by lists01.pubmisc.prod.ext.phx2.redhat.com id 1BAKZ0Vp008406 X-loop: libvir-list@redhat.com Cc: william.douglas@intel.com X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZM-MESSAGEID: 1639168546029100001 Content-Type: text/plain; charset="utf-8" Signed-off-by: Vineeth Pillai Signed-off-by: Praveen K Paladugu Reviewed-by: Michal Privoznik --- src/ch/ch_domain.c | 25 +++++++++ src/ch/ch_domain.h | 4 ++ src/ch/ch_driver.c | 134 +++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 163 insertions(+) diff --git a/src/ch/ch_domain.c b/src/ch/ch_domain.c index 36333a8a3f..3f34e87e04 100644 --- a/src/ch/ch_domain.c +++ b/src/ch/ch_domain.c @@ -336,3 +336,28 @@ virCHDomainGetMonitor(virDomainObj *vm) { return CH_DOMAIN_PRIVATE(vm)->monitor; } + +pid_t +virCHDomainGetVcpuPid(virDomainObj *vm, unsigned int vcpuid) +{ + virDomainVcpuDef *vcpu =3D virDomainDefGetVcpu(vm->def, vcpuid); + + return CH_DOMAIN_VCPU_PRIVATE(vcpu)->tid; +} + +bool +virCHDomainHasVcpuPids(virDomainObj *vm) +{ + size_t i; + size_t maxvcpus =3D virDomainDefGetVcpusMax(vm->def); + virDomainVcpuDef *vcpu; + + for (i =3D 0; i < maxvcpus; i++) { + vcpu =3D virDomainDefGetVcpu(vm->def, i); + + if (CH_DOMAIN_VCPU_PRIVATE(vcpu)->tid > 0) + return true; + } + + return false; +} diff --git a/src/ch/ch_domain.h b/src/ch/ch_domain.h index f01c0e5ad0..1ec7643216 100644 --- a/src/ch/ch_domain.h +++ b/src/ch/ch_domain.h @@ -82,3 +82,7 @@ virCHDomainObjBeginJob(virDomainObj *obj, enum virCHDomai= nJob job) =20 void virCHDomainObjEndJob(virDomainObj *obj); + +int virCHDomainRefreshVcpuInfo(virDomainObj *vm); +pid_t virCHDomainGetVcpuPid(virDomainObj *vm, unsigned int vcpuid); +bool virCHDomainHasVcpuPids(virDomainObj *vm); diff --git a/src/ch/ch_driver.c b/src/ch/ch_driver.c index 108644e503..31e9de7f6a 100644 --- a/src/ch/ch_driver.c +++ b/src/ch/ch_driver.c @@ -942,6 +942,137 @@ static int chStateInitialize(bool privileged, return ret; } =20 +static int +chDomainGetVcpusFlags(virDomainPtr dom, unsigned int flags) +{ + virDomainObj *vm; + virDomainDef *def; + int ret =3D -1; + + virCheckFlags(VIR_DOMAIN_AFFECT_LIVE | + VIR_DOMAIN_AFFECT_CONFIG | + VIR_DOMAIN_VCPU_MAXIMUM | VIR_DOMAIN_VCPU_GUEST, -1); + + if (!(vm =3D chDomObjFromDomain(dom))) + return -1; + + if (virDomainGetVcpusFlagsEnsureACL(dom->conn, vm->def, flags) < 0) + goto cleanup; + + if (!(def =3D virDomainObjGetOneDef(vm, flags))) + goto cleanup; + + if (flags & VIR_DOMAIN_VCPU_MAXIMUM) + ret =3D virDomainDefGetVcpusMax(def); + else + ret =3D virDomainDefGetVcpus(def); + + + cleanup: + virDomainObjEndAPI(&vm); + return ret; +} + +static int +chDomainGetMaxVcpus(virDomainPtr dom) +{ + return chDomainGetVcpusFlags(dom, + (VIR_DOMAIN_AFFECT_LIVE | + VIR_DOMAIN_VCPU_MAXIMUM)); +} + +static int +chDomainHelperGetVcpus(virDomainObj *vm, + virVcpuInfoPtr info, + unsigned long long *cpuwait, + int maxinfo, unsigned char *cpumaps, int maplen) +{ + size_t ncpuinfo =3D 0; + size_t i; + + if (maxinfo =3D=3D 0) + return 0; + + if (!virCHDomainHasVcpuPids(vm)) { + virReportError(VIR_ERR_OPERATION_INVALID, + "%s", _("cpu affinity is not supported")); + return -1; + } + + if (info) + memset(info, 0, sizeof(*info) * maxinfo); + + if (cpumaps) + memset(cpumaps, 0, sizeof(*cpumaps) * maxinfo); + + for (i =3D 0; i < virDomainDefGetVcpusMax(vm->def) && ncpuinfo < maxin= fo; i++) { + virDomainVcpuDef *vcpu =3D virDomainDefGetVcpu(vm->def, i); + pid_t vcpupid =3D virCHDomainGetVcpuPid(vm, i); + virVcpuInfoPtr vcpuinfo =3D info + ncpuinfo; + + if (!vcpu->online) + continue; + + if (info) { + vcpuinfo->number =3D i; + vcpuinfo->state =3D VIR_VCPU_RUNNING; + if (virProcessGetStatInfo(&vcpuinfo->cpuTime, + &vcpuinfo->cpu, NULL, + vm->pid, vcpupid) < 0) { + virReportSystemError(errno, "%s", + _("cannot get vCPU placement & pCPU = time")); + return -1; + } + } + + if (cpumaps) { + unsigned char *cpumap =3D VIR_GET_CPUMAP(cpumaps, maplen, ncpu= info); + g_autoptr(virBitmap) map =3D NULL; + + if (!(map =3D virProcessGetAffinity(vcpupid))) + return -1; + + virBitmapToDataBuf(map, cpumap, maplen); + } + + if (cpuwait) { + if (virProcessGetSchedInfo(&(cpuwait[ncpuinfo]), vm->pid, vcpu= pid) < 0) + return -1; + } + + ncpuinfo++; + } + + return ncpuinfo; +} + +static int +chDomainGetVcpus(virDomainPtr dom, + virVcpuInfoPtr info, + int maxinfo, unsigned char *cpumaps, int maplen) +{ + virDomainObj *vm; + int ret =3D -1; + + if (!(vm =3D chDomObjFromDomain(dom))) + goto cleanup; + + if (virDomainGetVcpusEnsureACL(dom->conn, vm->def) < 0) + goto cleanup; + + if (virDomainObjCheckActive(vm) < 0) { + virReportError(VIR_ERR_OPERATION_INVALID, "%s", + _("cannot retrieve vcpu information for inactive do= main")); + goto cleanup; + } + + ret =3D chDomainHelperGetVcpus(vm, info, NULL, maxinfo, cpumaps, maple= n); + + cleanup: + virDomainObjEndAPI(&vm); + return ret; +} + /* Function Tables */ static virHypervisorDriver chHypervisorDriver =3D { .name =3D "CH", @@ -978,6 +1109,9 @@ static virHypervisorDriver chHypervisorDriver =3D { .domainIsActive =3D chDomainIsActive, /* 7.5.0 */ .domainOpenConsole =3D chDomainOpenConsole, /* 7.8.0 */ .nodeGetInfo =3D chNodeGetInfo, /* 7.5.0 */ + .domainGetVcpus =3D chDomainGetVcpus, /* 8.0.0 */ + .domainGetVcpusFlags =3D chDomainGetVcpusFlags, /* 8.0.0 */ + .domainGetMaxVcpus =3D chDomainGetMaxVcpus, /* 8.0.0 */ }; =20 static virConnectDriver chConnectDriver =3D { --=20 2.27.0 From nobody Sun May 5 13:15:08 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) client-ip=170.10.129.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail(p=none dis=none) header.from=linux.microsoft.com Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mx.zohomail.com with SMTPS id 1639168514703841.760449439723; Fri, 10 Dec 2021 12:35:14 -0800 (PST) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-13-2ICQrdzhPL2IsFEf5EqDmQ-1; Fri, 10 Dec 2021 15:35:11 -0500 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id AA06586A061; Fri, 10 Dec 2021 20:35:04 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id DA26B45D72; Fri, 10 Dec 2021 20:35:03 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 920BF1809C87; Fri, 10 Dec 2021 20:35:01 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 1BAKYwEA008364 for ; Fri, 10 Dec 2021 15:34:58 -0500 Received: by smtp.corp.redhat.com (Postfix) id 83C7740C1241; Fri, 10 Dec 2021 20:34:58 +0000 (UTC) Received: from mimecast-mx02.redhat.com (mimecast08.extmail.prod.ext.rdu2.redhat.com [10.11.55.24]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 7FFB4400E127 for ; Fri, 10 Dec 2021 20:34:58 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-2.mimecast.com [207.211.31.81]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 6848239D30B9 for ; Fri, 10 Dec 2021 20:34:58 +0000 (UTC) Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by relay.mimecast.com with ESMTP id us-mta-181-Go723PCbP2e3a8iyNL8nmQ-1; Fri, 10 Dec 2021 15:34:56 -0500 Received: from prapal-ch2.oiwy50ateaxezb1sqsoezlib2f.xx.internal.cloudapp.net (unknown [20.80.162.67]) by linux.microsoft.com (Postfix) with ESMTPSA id 8E06A20B7187; Fri, 10 Dec 2021 12:34:54 -0800 (PST) X-MC-Unique: 2ICQrdzhPL2IsFEf5EqDmQ-1 X-MC-Unique: Go723PCbP2e3a8iyNL8nmQ-1 DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 8E06A20B7187 From: Praveen K Paladugu To: libvir-list@redhat.com Subject: [libvirt PATCH v3 05/13] ch_driver: domainGetVcpuPinInfo and nodeGetCPUMap Date: Fri, 10 Dec 2021 20:34:40 +0000 Message-Id: <20211210203448.1579606-6-prapal@linux.microsoft.com> In-Reply-To: <20211210203448.1579606-1-prapal@linux.microsoft.com> References: <20211210203448.1579606-1-prapal@linux.microsoft.com> MIME-Version: 1.0 X-Mimecast-Impersonation-Protect: Policy=CLT - Impersonation Protection Definition; Similar Internal Domain=false; Similar Monitored External Domain=false; Custom External Domain=false; Mimecast External Domain=false; Newly Observed Domain=false; Internal User Name=false; Custom Display Name List=false; Reply-to Address Mismatch=false; Targeted Threat Dictionary=false; Mimecast Threat Dictionary=false; Custom Threat Dictionary=false X-Scanned-By: MIMEDefang 2.84 on 10.11.54.2 X-MIME-Autoconverted: from quoted-printable to 8bit by lists01.pubmisc.prod.ext.phx2.redhat.com id 1BAKYwEA008364 X-loop: libvir-list@redhat.com Cc: william.douglas@intel.com X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZM-MESSAGEID: 1639168516942100003 Content-Type: text/plain; charset="utf-8" From: Vineeth Pillai Add domainGetVcpuPinInfo and nodeGetCPUMap callbacks to ch driver Signed-off-by: Vineeth Pillai Signed-off-by: Praveen K Paladugu Reviewed-by: Michal Privoznik --- src/ch/ch_domain.h | 10 +++++++-- src/ch/ch_driver.c | 51 ++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 59 insertions(+), 2 deletions(-) diff --git a/src/ch/ch_domain.h b/src/ch/ch_domain.h index 1ec7643216..c36405808b 100644 --- a/src/ch/ch_domain.h +++ b/src/ch/ch_domain.h @@ -23,6 +23,7 @@ #include "ch_conf.h" #include "ch_monitor.h" #include "virchrdev.h" +#include "vircgroup.h" =20 /* Give up waiting for mutex after 30 seconds */ #define CH_JOB_WAIT_TIME (1000ull * 30) @@ -52,9 +53,14 @@ typedef struct _virCHDomainObjPrivate virCHDomainObjPriv= ate; struct _virCHDomainObjPrivate { struct virCHDomainJobObj job; =20 + virChrdevs *chrdevs; + virCgroup *cgroup; + virCHDriver *driver; virCHMonitor *monitor; - - virChrdevs *chrdevs; + char *machineName; + virBitmap *autoNodeset; + virBitmap *autoCpuset; + virChrdevs *devs; }; =20 #define CH_DOMAIN_PRIVATE(vm) \ diff --git a/src/ch/ch_driver.c b/src/ch/ch_driver.c index 31e9de7f6a..ad7383e96e 100644 --- a/src/ch/ch_driver.c +++ b/src/ch/ch_driver.c @@ -981,6 +981,55 @@ chDomainGetMaxVcpus(virDomainPtr dom) VIR_DOMAIN_VCPU_MAXIMUM)); } =20 +static int +chDomainGetVcpuPinInfo(virDomain *dom, + int ncpumaps, + unsigned char *cpumaps, int maplen, unsigned int fl= ags) +{ + virDomainObj *vm =3D NULL; + virDomainDef *def; + bool live; + int ret =3D -1; + + g_autoptr(virBitmap) hostcpus =3D NULL; + virBitmap *autoCpuset =3D NULL; + + virCheckFlags(VIR_DOMAIN_AFFECT_LIVE | VIR_DOMAIN_AFFECT_CONFIG, -1); + + if (!(vm =3D chDomObjFromDomain(dom))) + goto cleanup; + + if (virDomainGetVcpuPinInfoEnsureACL(dom->conn, vm->def) < 0) + goto cleanup; + + if (!(def =3D virDomainObjGetOneDefState(vm, flags, &live))) + goto cleanup; + + if (!(hostcpus =3D virHostCPUGetAvailableCPUsBitmap())) + goto cleanup; + + if (live) + autoCpuset =3D CH_DOMAIN_PRIVATE(vm)->autoCpuset; + + ret =3D virDomainDefGetVcpuPinInfoHelper(def, maplen, ncpumaps, cpumap= s, + hostcpus, autoCpuset); + cleanup: + virDomainObjEndAPI(&vm); + return ret; +} + +static int +chNodeGetCPUMap(virConnectPtr conn, + unsigned char **cpumap, + unsigned int *online, unsigned int flags) +{ + if (virNodeGetCPUMapEnsureACL(conn) < 0) + return -1; + + return virHostCPUGetMap(cpumap, online, flags); +} + + static int chDomainHelperGetVcpus(virDomainObj *vm, virVcpuInfoPtr info, @@ -1112,6 +1161,8 @@ static virHypervisorDriver chHypervisorDriver =3D { .domainGetVcpus =3D chDomainGetVcpus, /* 8.0.0 */ .domainGetVcpusFlags =3D chDomainGetVcpusFlags, /* 8.0.0 */ .domainGetMaxVcpus =3D chDomainGetMaxVcpus, /* 8.0.0 */ + .domainGetVcpuPinInfo =3D chDomainGetVcpuPinInfo, /* 8.0.0 */ + .nodeGetCPUMap =3D chNodeGetCPUMap, /* 8.0.0 */ }; =20 static virConnectDriver chConnectDriver =3D { --=20 2.27.0 From nobody Sun May 5 13:15:08 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) client-ip=170.10.129.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail(p=none dis=none) header.from=linux.microsoft.com Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mx.zohomail.com with SMTPS id 1639168530394812.6954962939765; Fri, 10 Dec 2021 12:35:30 -0800 (PST) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-409-VCMTIBBNOlSaGCupoX3kCQ-1; Fri, 10 Dec 2021 15:35:25 -0500 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 1801763AD2; Fri, 10 Dec 2021 20:35:20 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id F07F660622; Fri, 10 Dec 2021 20:35:19 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id B26884A7C8; Fri, 10 Dec 2021 20:35:19 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 1BAKYwnO008365 for ; Fri, 10 Dec 2021 15:34:58 -0500 Received: by smtp.corp.redhat.com (Postfix) id 8DF46C23DB1; Fri, 10 Dec 2021 20:34:58 +0000 (UTC) Received: from mimecast-mx02.redhat.com (mimecast09.extmail.prod.ext.rdu2.redhat.com [10.11.55.25]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 88CC8C01783 for ; Fri, 10 Dec 2021 20:34:58 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [207.211.31.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 69F172A59562 for ; Fri, 10 Dec 2021 20:34:58 +0000 (UTC) Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by relay.mimecast.com with ESMTP id us-mta-181-INPMcxU7Orq-ujHMTsyDIg-1; Fri, 10 Dec 2021 15:34:56 -0500 Received: from prapal-ch2.oiwy50ateaxezb1sqsoezlib2f.xx.internal.cloudapp.net (unknown [20.80.162.67]) by linux.microsoft.com (Postfix) with ESMTPSA id A49FE20B7189; Fri, 10 Dec 2021 12:34:54 -0800 (PST) X-MC-Unique: VCMTIBBNOlSaGCupoX3kCQ-1 X-MC-Unique: INPMcxU7Orq-ujHMTsyDIg-1 DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com A49FE20B7189 From: Praveen K Paladugu To: libvir-list@redhat.com Subject: [libvirt PATCH v3 06/13] ch_monitor: Get nicindexes in prep for cgroup mgmt Date: Fri, 10 Dec 2021 20:34:41 +0000 Message-Id: <20211210203448.1579606-7-prapal@linux.microsoft.com> In-Reply-To: <20211210203448.1579606-1-prapal@linux.microsoft.com> References: <20211210203448.1579606-1-prapal@linux.microsoft.com> MIME-Version: 1.0 X-Mimecast-Impersonation-Protect: Policy=CLT - Impersonation Protection Definition; Similar Internal Domain=false; Similar Monitored External Domain=false; Custom External Domain=false; Mimecast External Domain=false; Newly Observed Domain=false; Internal User Name=false; Custom Display Name List=false; Reply-to Address Mismatch=false; Targeted Threat Dictionary=false; Mimecast Threat Dictionary=false; Custom Threat Dictionary=false X-Scanned-By: MIMEDefang 2.85 on 10.11.54.8 X-MIME-Autoconverted: from quoted-printable to 8bit by lists01.pubmisc.prod.ext.phx2.redhat.com id 1BAKYwnO008365 X-loop: libvir-list@redhat.com Cc: william.douglas@intel.com X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZM-MESSAGEID: 1639168531741100001 Content-Type: text/plain; charset="utf-8" From: Vineeth Pillai Signed-off-by: Vineeth Pillai Signed-off-by: Praveen K Paladugu Reviewed-by: Michal Privoznik --- src/ch/ch_conf.h | 2 + src/ch/ch_domain.c | 27 ++++++- src/ch/ch_domain.h | 2 + src/ch/ch_driver.c | 1 + src/ch/ch_monitor.c | 185 ++++++++++++++++++++++++++++++-------------- src/ch/ch_monitor.h | 14 +++- src/ch/ch_process.c | 6 +- src/ch/meson.build | 1 + 8 files changed, 179 insertions(+), 59 deletions(-) diff --git a/src/ch/ch_conf.h b/src/ch/ch_conf.h index 37c36d9a09..8fe69c8545 100644 --- a/src/ch/ch_conf.h +++ b/src/ch/ch_conf.h @@ -44,6 +44,8 @@ struct _virCHDriver { virMutex lock; =20 + bool privileged; + /* Require lock to get a reference on the object, * lockless access thereafter */ virCaps *caps; diff --git a/src/ch/ch_domain.c b/src/ch/ch_domain.c index 3f34e87e04..3a32ac63d9 100644 --- a/src/ch/ch_domain.c +++ b/src/ch/ch_domain.c @@ -21,10 +21,12 @@ #include =20 #include "ch_domain.h" +#include "domain_driver.h" #include "viralloc.h" #include "virchrdev.h" #include "virlog.h" #include "virtime.h" +#include "virsystemd.h" =20 #define VIR_FROM_THIS VIR_FROM_CH =20 @@ -134,7 +136,7 @@ virCHDomainObjEndJob(virDomainObj *obj) } =20 static void * -virCHDomainObjPrivateAlloc(void *opaque G_GNUC_UNUSED) +virCHDomainObjPrivateAlloc(void *opaque) { virCHDomainObjPrivate *priv; =20 @@ -150,6 +152,7 @@ virCHDomainObjPrivateAlloc(void *opaque G_GNUC_UNUSED) g_free(priv); return NULL; } + priv->driver =3D opaque; =20 return priv; } @@ -361,3 +364,25 @@ virCHDomainHasVcpuPids(virDomainObj *vm) =20 return false; } + +char * +virCHDomainGetMachineName(virDomainObj *vm) +{ + virCHDomainObjPrivate *priv =3D CH_DOMAIN_PRIVATE(vm); + virCHDriver *driver =3D priv->driver; + char *ret =3D NULL; + + if (vm->pid > 0) { + ret =3D virSystemdGetMachineNameByPID(vm->pid); + if (!ret) + virResetLastError(); + } + + if (!ret) + ret =3D virDomainDriverGenerateMachineName("ch", + NULL, + vm->def->id, vm->def->nam= e, + driver->privileged); + + return ret; +} diff --git a/src/ch/ch_domain.h b/src/ch/ch_domain.h index c36405808b..4fc6251380 100644 --- a/src/ch/ch_domain.h +++ b/src/ch/ch_domain.h @@ -92,3 +92,5 @@ virCHDomainObjEndJob(virDomainObj *obj); int virCHDomainRefreshVcpuInfo(virDomainObj *vm); pid_t virCHDomainGetVcpuPid(virDomainObj *vm, unsigned int vcpuid); bool virCHDomainHasVcpuPids(virDomainObj *vm); + +char *virCHDomainGetMachineName(virDomainObj *vm); diff --git a/src/ch/ch_driver.c b/src/ch/ch_driver.c index ad7383e96e..97fbd76959 100644 --- a/src/ch/ch_driver.c +++ b/src/ch/ch_driver.c @@ -934,6 +934,7 @@ static int chStateInitialize(bool privileged, goto cleanup; } =20 + ch_driver->privileged =3D privileged; ret =3D VIR_DRV_STATE_INIT_COMPLETE; =20 cleanup: diff --git a/src/ch/ch_monitor.c b/src/ch/ch_monitor.c index 12c10da874..d48b057814 100644 --- a/src/ch/ch_monitor.c +++ b/src/ch/ch_monitor.c @@ -226,7 +226,8 @@ virCHMonitorBuildDisksJson(virJSONValue *content, virDo= mainDef *vmdef) } =20 static int -virCHMonitorBuildNetJson(virJSONValue *nets, virDomainNetDef *netdef) +virCHMonitorBuildNetJson(virJSONValue *nets, virDomainNetDef *netdef, + size_t *nnicindexes, int **nicindexes) { virDomainNetType netType =3D virDomainNetGetActualType(netdef); char macaddr[VIR_MAC_STRING_BUFLEN]; @@ -236,59 +237,72 @@ virCHMonitorBuildNetJson(virJSONValue *nets, virDomai= nNetDef *netdef) net =3D virJSONValueNewObject(); =20 switch (netType) { - case VIR_DOMAIN_NET_TYPE_ETHERNET: - if (netdef->guestIP.nips =3D=3D 1) { - const virNetDevIPAddr *ip =3D netdef->guestIP.ips[0]; - g_autofree char *addr =3D NULL; - virSocketAddr netmask; - g_autofree char *netmaskStr =3D NULL; - if (!(addr =3D virSocketAddrFormat(&ip->address))) - return -1; - if (virJSONValueObjectAppendString(net, "ip", addr) < 0) - return -1; + case VIR_DOMAIN_NET_TYPE_ETHERNET: + if (netdef->guestIP.nips =3D=3D 1) { + const virNetDevIPAddr *ip =3D netdef->guestIP.ips[0]; + g_autofree char *addr =3D NULL; + virSocketAddr netmask; + g_autofree char *netmaskStr =3D NULL; + + if (!(addr =3D virSocketAddrFormat(&ip->address))) + return -1; + if (virJSONValueObjectAppendString(net, "ip", addr) < 0) + return -1; + + if (virSocketAddrPrefixToNetmask(ip->prefix, &netmask, AF_= INET) < 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Failed to translate net prefix %d to= netmask"), + ip->prefix); + return -1; + } + if (!(netmaskStr =3D virSocketAddrFormat(&netmask))) + return -1; + if (virJSONValueObjectAppendString(net, "mask", netmaskStr= ) < 0) + return -1; + } else if (netdef->guestIP.nips > 1) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", + _("ethernet type supports a single guest ip= ")); + } =20 - if (virSocketAddrPrefixToNetmask(ip->prefix, &netmask, AF_INET= ) < 0) { - virReportError(VIR_ERR_INTERNAL_ERROR, - _("Failed to translate net prefix %d to net= mask"), - ip->prefix); - return -1; + /* network and bridge use a tap device, and direct uses a + * macvtap device + */ + if (nicindexes && nnicindexes && netdef->ifname) { + int nicindex =3D 0; + + if (virNetDevGetIndex(netdef->ifname, &nicindex) < 0) + return -1; + + VIR_APPEND_ELEMENT(*nicindexes, *nnicindexes, nicindex); } - if (!(netmaskStr =3D virSocketAddrFormat(&netmask))) - return -1; - if (virJSONValueObjectAppendString(net, "mask", netmaskStr) < = 0) - return -1; - } else if (netdef->guestIP.nips > 1) { - virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", - _("ethernet type supports a single guest ip")); - } - break; - case VIR_DOMAIN_NET_TYPE_VHOSTUSER: - if ((virDomainChrType)netdef->data.vhostuser->type !=3D VIR_DOMAIN= _CHR_TYPE_UNIX) { - virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", + break; + case VIR_DOMAIN_NET_TYPE_VHOSTUSER: + if ((virDomainChrType)netdef->data.vhostuser->type !=3D VIR_DO= MAIN_CHR_TYPE_UNIX) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", _("vhost_user type support UNIX socket in this = CH")); - return -1; - } else { - if (virJSONValueObjectAppendString(net, "vhost_socket", netdef= ->data.vhostuser->data.nix.path) < 0) - return -1; - if (virJSONValueObjectAppendBoolean(net, "vhost_user", true) <= 0) return -1; - } - break; - case VIR_DOMAIN_NET_TYPE_BRIDGE: - case VIR_DOMAIN_NET_TYPE_NETWORK: - case VIR_DOMAIN_NET_TYPE_DIRECT: - case VIR_DOMAIN_NET_TYPE_USER: - case VIR_DOMAIN_NET_TYPE_SERVER: - case VIR_DOMAIN_NET_TYPE_CLIENT: - case VIR_DOMAIN_NET_TYPE_MCAST: - case VIR_DOMAIN_NET_TYPE_INTERNAL: - case VIR_DOMAIN_NET_TYPE_HOSTDEV: - case VIR_DOMAIN_NET_TYPE_UDP: - case VIR_DOMAIN_NET_TYPE_VDPA: - case VIR_DOMAIN_NET_TYPE_LAST: - default: - virReportEnumRangeError(virDomainNetType, netType); - return -1; + } else { + if (virJSONValueObjectAppendString(net, "vhost_socket", ne= tdef->data.vhostuser->data.nix.path) < 0) + return -1; + if (virJSONValueObjectAppendBoolean(net, "vhost_user", tru= e) < 0) + return -1; + } + break; + case VIR_DOMAIN_NET_TYPE_BRIDGE: + case VIR_DOMAIN_NET_TYPE_NETWORK: + case VIR_DOMAIN_NET_TYPE_DIRECT: + case VIR_DOMAIN_NET_TYPE_USER: + case VIR_DOMAIN_NET_TYPE_SERVER: + case VIR_DOMAIN_NET_TYPE_CLIENT: + case VIR_DOMAIN_NET_TYPE_MCAST: + case VIR_DOMAIN_NET_TYPE_INTERNAL: + case VIR_DOMAIN_NET_TYPE_HOSTDEV: + case VIR_DOMAIN_NET_TYPE_UDP: + case VIR_DOMAIN_NET_TYPE_VDPA: + case VIR_DOMAIN_NET_TYPE_LAST: + default: + virReportEnumRangeError(virDomainNetType, netType); + return -1; } =20 if (netdef->ifname !=3D NULL) { @@ -329,7 +343,8 @@ virCHMonitorBuildNetJson(virJSONValue *nets, virDomainN= etDef *netdef) } =20 static int -virCHMonitorBuildNetsJson(virJSONValue *content, virDomainDef *vmdef) +virCHMonitorBuildNetsJson(virJSONValue *content, virDomainDef *vmdef, + size_t *nnicindexes, int **nicindexes) { g_autoptr(virJSONValue) nets =3D NULL; size_t i; @@ -338,7 +353,8 @@ virCHMonitorBuildNetsJson(virJSONValue *content, virDom= ainDef *vmdef) nets =3D virJSONValueNewArray(); =20 for (i =3D 0; i < vmdef->nnets; i++) { - if (virCHMonitorBuildNetJson(nets, vmdef->nets[i]) < 0) + if (virCHMonitorBuildNetJson(nets, vmdef->nets[i], + nnicindexes, nicindexes) < 0) return -1; } if (virJSONValueObjectAppend(content, "net", &nets) < 0) @@ -349,7 +365,59 @@ virCHMonitorBuildNetsJson(virJSONValue *content, virDo= mainDef *vmdef) } =20 static int -virCHMonitorBuildVMJson(virDomainDef *vmdef, char **jsonstr) +virCHMonitorBuildDeviceJson(virJSONValue *devices, + virDomainHostdevDef *hostdevdef) +{ + g_autoptr(virJSONValue) device =3D NULL; + + + if (hostdevdef->mode =3D=3D VIR_DOMAIN_HOSTDEV_MODE_SUBSYS && + hostdevdef->source.subsys.type =3D=3D VIR_DOMAIN_HOSTDEV_SUBSYS_TY= PE_PCI) { + g_autofree char *name =3D NULL; + g_autofree char *path =3D NULL; + virDomainHostdevSubsysPCI *pcisrc =3D &hostdevdef->source.subsys.u= .pci; + + device =3D virJSONValueNewObject(); + name =3D virPCIDeviceAddressAsString(&pcisrc->addr); + path =3D g_strdup_printf("/sys/bus/pci/devices/%s/", name); + if (!virFileExists(path)) { + virReportError(VIR_ERR_DEVICE_MISSING, + _("host pci device %s not found"), path); + return -1; + } + if (virJSONValueObjectAppendString(device, "path", path) < 0) + return -1; + if (virJSONValueArrayAppend(devices, &device) < 0) + return -1; + } + + return 0; +} + +static int +virCHMonitorBuildDevicesJson(virJSONValue *content, virDomainDef *vmdef) +{ + size_t i; + + g_autoptr(virJSONValue) devices =3D NULL; + + if (vmdef->nhostdevs =3D=3D 0) + return 0; + + devices =3D virJSONValueNewArray(); + for (i =3D 0; i < vmdef->nhostdevs; i++) { + if (virCHMonitorBuildDeviceJson(devices, vmdef->hostdevs[i]) < 0) + return -1; + } + if (virJSONValueObjectAppend(content, "devices", &devices) < 0) + return -1; + + return 0; +} + +static int +virCHMonitorBuildVMJson(virDomainDef *vmdef, char **jsonstr, + size_t *nnicindexes, int **nicindexes) { g_autoptr(virJSONValue) content =3D virJSONValueNewObject(); =20 @@ -374,7 +442,11 @@ virCHMonitorBuildVMJson(virDomainDef *vmdef, char **js= onstr) if (virCHMonitorBuildDisksJson(content, vmdef) < 0) return -1; =20 - if (virCHMonitorBuildNetsJson(content, vmdef) < 0) + + if (virCHMonitorBuildNetsJson(content, vmdef, nnicindexes, nicindexes)= < 0) + return -1; + + if (virCHMonitorBuildDevicesJson(content, vmdef) < 0) return -1; =20 if (!(*jsonstr =3D virJSONValueToString(content, false))) @@ -671,7 +743,7 @@ virCHMonitorShutdownVMM(virCHMonitor *mon) } =20 int -virCHMonitorCreateVM(virCHMonitor *mon) +virCHMonitorCreateVM(virCHMonitor *mon, size_t *nnicindexes, int **nicinde= xes) { g_autofree char *url =3D NULL; int responseCode =3D 0; @@ -683,7 +755,8 @@ virCHMonitorCreateVM(virCHMonitor *mon) headers =3D curl_slist_append(headers, "Accept: application/json"); headers =3D curl_slist_append(headers, "Content-Type: application/json= "); =20 - if (virCHMonitorBuildVMJson(mon->vm->def, &payload) !=3D 0) + if (virCHMonitorBuildVMJson(mon->vm->def, &payload, + nnicindexes, nicindexes) !=3D 0) return -1; =20 virObjectLock(mon); diff --git a/src/ch/ch_monitor.h b/src/ch/ch_monitor.h index 0f684ca583..8ca9e17a9a 100644 --- a/src/ch/ch_monitor.h +++ b/src/ch/ch_monitor.h @@ -55,10 +55,22 @@ virCHMonitor *virCHMonitorNew(virDomainObj *vm, const c= har *socketdir); void virCHMonitorClose(virCHMonitor *mon); G_DEFINE_AUTOPTR_CLEANUP_FUNC(virCHMonitor, virCHMonitorClose); =20 -int virCHMonitorCreateVM(virCHMonitor *mon); + +int virCHMonitorCreateVM(virCHMonitor *mon, + size_t *nnicindexes, int **nicindexes); int virCHMonitorBootVM(virCHMonitor *mon); int virCHMonitorShutdownVM(virCHMonitor *mon); int virCHMonitorRebootVM(virCHMonitor *mon); int virCHMonitorSuspendVM(virCHMonitor *mon); int virCHMonitorResumeVM(virCHMonitor *mon); int virCHMonitorGetInfo(virCHMonitor *mon, virJSONValue **info); + +typedef struct _virCHMonitorCPUInfo virCHMonitorCPUInfo; +struct _virCHMonitorCPUInfo { + pid_t tid; + bool online; +}; +void virCHMonitorCPUInfoFree(virCHMonitorCPUInfo *cpus); +int virCHMonitorGetCPUInfo(virCHMonitor *mon, + virCHMonitorCPUInfo **vcpus, + size_t maxvcpus); diff --git a/src/ch/ch_process.c b/src/ch/ch_process.c index 9f82e04485..65ca03cfaf 100644 --- a/src/ch/ch_process.c +++ b/src/ch/ch_process.c @@ -147,6 +147,8 @@ int virCHProcessStart(virCHDriver *driver, { int ret =3D -1; virCHDomainObjPrivate *priv =3D vm->privateData; + g_autofree int *nicindexes =3D NULL; + size_t nnicindexes =3D 0; =20 if (!priv->monitor) { /* And we can get the first monitor connection now too */ @@ -156,7 +158,8 @@ int virCHProcessStart(virCHDriver *driver, goto cleanup; } =20 - if (virCHMonitorCreateVM(priv->monitor) < 0) { + if (virCHMonitorCreateVM(priv->monitor, + &nnicindexes, &nicindexes) < 0) { virReportError(VIR_ERR_INTERNAL_ERROR, "%s", _("failed to create guest VM")); goto cleanup; @@ -169,6 +172,7 @@ int virCHProcessStart(virCHDriver *driver, goto cleanup; } =20 + priv->machineName =3D virCHDomainGetMachineName(vm); vm->pid =3D priv->monitor->pid; vm->def->id =3D vm->pid; =20 diff --git a/src/ch/meson.build b/src/ch/meson.build index e34974d56c..e0afdb390a 100644 --- a/src/ch/meson.build +++ b/src/ch/meson.build @@ -29,6 +29,7 @@ if conf.has('WITH_CH') ], include_directories: [ conf_inc_dir, + hypervisor_inc_dir, ], ) =20 --=20 2.27.0 From nobody Sun May 5 13:15:08 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) client-ip=170.10.129.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail(p=none dis=none) header.from=linux.microsoft.com Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mx.zohomail.com with SMTPS id 1639168539974168.88654145680437; Fri, 10 Dec 2021 12:35:39 -0800 (PST) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-535-zF8uOzR0NX25LA-y_X_wEw-1; Fri, 10 Dec 2021 15:35:36 -0500 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id DCB9510144F9; Fri, 10 Dec 2021 20:35:25 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id C017177713; Fri, 10 Dec 2021 20:35:25 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 8EF0A180474A; Fri, 10 Dec 2021 20:35:25 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 1BAKYxJe008377 for ; Fri, 10 Dec 2021 15:34:59 -0500 Received: by smtp.corp.redhat.com (Postfix) id E49E7401DBD; Fri, 10 Dec 2021 20:34:58 +0000 (UTC) Received: from mimecast-mx02.redhat.com (mimecast07.extmail.prod.ext.rdu2.redhat.com [10.11.55.23]) by smtp.corp.redhat.com (Postfix) with ESMTPS id E03DF401DBC for ; Fri, 10 Dec 2021 20:34:58 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-1.mimecast.com [205.139.110.61]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id C01153C9A02E for ; Fri, 10 Dec 2021 20:34:58 +0000 (UTC) Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by relay.mimecast.com with ESMTP id us-mta-40-q1L0XpabNCiASlJcFNJltw-1; Fri, 10 Dec 2021 15:34:56 -0500 Received: from prapal-ch2.oiwy50ateaxezb1sqsoezlib2f.xx.internal.cloudapp.net (unknown [20.80.162.67]) by linux.microsoft.com (Postfix) with ESMTPSA id BE2B120B718A; Fri, 10 Dec 2021 12:34:54 -0800 (PST) X-MC-Unique: zF8uOzR0NX25LA-y_X_wEw-1 X-MC-Unique: q1L0XpabNCiASlJcFNJltw-1 DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com BE2B120B718A From: Praveen K Paladugu To: libvir-list@redhat.com Subject: [libvirt PATCH v3 07/13] qemu, hypervisor: refactor some cgroup mgmt methods Date: Fri, 10 Dec 2021 20:34:42 +0000 Message-Id: <20211210203448.1579606-8-prapal@linux.microsoft.com> In-Reply-To: <20211210203448.1579606-1-prapal@linux.microsoft.com> References: <20211210203448.1579606-1-prapal@linux.microsoft.com> MIME-Version: 1.0 X-Mimecast-Impersonation-Protect: Policy=CLT - Impersonation Protection Definition; Similar Internal Domain=false; Similar Monitored External Domain=false; Custom External Domain=false; Mimecast External Domain=false; Newly Observed Domain=false; Internal User Name=false; Custom Display Name List=false; Reply-to Address Mismatch=false; Targeted Threat Dictionary=false; Mimecast Threat Dictionary=false; Custom Threat Dictionary=false X-Scanned-By: MIMEDefang 2.85 on 10.11.54.10 X-MIME-Autoconverted: from quoted-printable to 8bit by lists01.pubmisc.prod.ext.phx2.redhat.com id 1BAKYxJe008377 X-loop: libvir-list@redhat.com Cc: william.douglas@intel.com X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZM-MESSAGEID: 1639168541295100001 Content-Type: text/plain; charset="utf-8" Refactor some cgroup management methods from qemu into hypervisor. These methods will be shared with ch driver for cgroup management. Signed-off-by: Praveen K Paladugu --- src/hypervisor/domain_cgroup.c | 426 ++++++++++++++++++++++++++++++++- src/hypervisor/domain_cgroup.h | 52 ++++ src/libvirt_private.syms | 13 + src/qemu/qemu_cgroup.c | 410 +------------------------------ src/qemu/qemu_cgroup.h | 11 - src/qemu/qemu_driver.c | 14 +- src/qemu/qemu_hotplug.c | 7 +- src/qemu/qemu_process.c | 20 +- 8 files changed, 522 insertions(+), 431 deletions(-) diff --git a/src/hypervisor/domain_cgroup.c b/src/hypervisor/domain_cgroup.c index 61b54f071c..0a1b3867b1 100644 --- a/src/hypervisor/domain_cgroup.c +++ b/src/hypervisor/domain_cgroup.c @@ -22,11 +22,12 @@ =20 #include "domain_cgroup.h" #include "domain_driver.h" - +#include "util/virnuma.h" +#include "virlog.h" #include "virutil.h" =20 #define VIR_FROM_THIS VIR_FROM_DOMAIN - +VIR_LOG_INIT("domain.cgroup"); =20 int virDomainCgroupSetupBlkio(virCgroup *cgroup, virDomainBlkiotune blkio) @@ -269,3 +270,424 @@ virDomainCgroupSetMemoryLimitParameters(virCgroup *cg= roup, =20 return 0; } + + +int +virSetupBlkioCgroup(virDomainObj *vm, virCgroup *cgroup) +{ + + if (!virCgroupHasController(cgroup, VIR_CGROUP_CONTROLLER_BLKIO)) { + if (vm->def->blkio.weight || vm->def->blkio.ndevices) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", + _("Block I/O tuning is not available on this ho= st")); + return -1; + } else { + return 0; + } + } + + return virDomainCgroupSetupBlkio(cgroup, vm->def->blkio); +} + + +int +virSetupMemoryCgroup(virDomainObj *vm, virCgroup *cgroup) +{ + + if (!virCgroupHasController(cgroup, VIR_CGROUP_CONTROLLER_MEMORY)) { + if (virMemoryLimitIsSet(vm->def->mem.hard_limit) || + virMemoryLimitIsSet(vm->def->mem.soft_limit) || + virMemoryLimitIsSet(vm->def->mem.swap_hard_limit)) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", + _("Memory cgroup is not available on this host"= )); + return -1; + } else { + return 0; + } + } + + return virDomainCgroupSetupMemtune(cgroup, vm->def->mem); +} + + +int +virSetupCpusetCgroup(virCgroup *cgroup) +{ + + if (!virCgroupHasController(cgroup, VIR_CGROUP_CONTROLLER_CPUSET)) + return 0; + + if (virCgroupSetCpusetMemoryMigrate(cgroup, true) < 0) + return -1; + + return 0; +} + + +int +virSetupCpuCgroup(virDomainObj *vm, virCgroup *cgroup) +{ + + if (!virCgroupHasController(cgroup, VIR_CGROUP_CONTROLLER_CPU)) { + if (vm->def->cputune.sharesSpecified) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", + _("CPU tuning is not available on this host")); + return -1; + } else { + return 0; + } + } + + if (vm->def->cputune.sharesSpecified) { + + if (virCgroupSetCpuShares(cgroup, vm->def->cputune.shares) < 0) + return -1; + + } + + return 0; +} + + +int +virInitCgroup(const char *prefix, + virDomainObj * vm, + size_t nnicindexes, int *nicindexes, + virCgroup * cgroup, int cgroupControllers, + unsigned int maxThreadsPerProc, + bool privileged, char *machineName) +{ + if (!privileged) + return 0; + + if (!virCgroupAvailable()) + return 0; + + virCgroupFree(cgroup); + cgroup =3D NULL; + + if (!vm->def->resource) + vm->def->resource =3D g_new0(virDomainResourceDef, 1); + + if (!vm->def->resource->partition) + vm->def->resource->partition =3D g_strdup("/machine"); + + if (!g_path_is_absolute(vm->def->resource->partition)) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, + _("Resource partition '%s' must start with '/'"), + vm->def->resource->partition); + return -1; + } + + if (virCgroupNewMachine(machineName, + prefix, + vm->def->uuid, + NULL, + vm->pid, + false, + nnicindexes, nicindexes, + vm->def->resource->partition, + cgroupControllers, + maxThreadsPerProc, &cgroup) < 0) { + if (virCgroupNewIgnoreError()) + return 0; + + return -1; + } + + return 0; +} + + +void +virRestoreCgroupState(virDomainObj *vm, virCgroup *cgroup) +{ + g_autofree char *mem_mask =3D NULL; + size_t i =3D 0; + + g_autoptr(virBitmap) all_nodes =3D NULL; + + if (!virNumaIsAvailable() || + !virCgroupHasController(cgroup, VIR_CGROUP_CONTROLLER_CPUSET)) + return; + + if (!(all_nodes =3D virNumaGetHostMemoryNodeset())) + goto error; + + if (!(mem_mask =3D virBitmapFormat(all_nodes))) + goto error; + + if (virCgroupHasEmptyTasks(cgroup, VIR_CGROUP_CONTROLLER_CPUSET) <=3D = 0) + goto error; + + if (virCgroupSetCpusetMems(cgroup, mem_mask) < 0) + goto error; + + for (i =3D 0; i < virDomainDefGetVcpusMax(vm->def); i++) { + virDomainVcpuDef *vcpu =3D virDomainDefGetVcpu(vm->def, i); + + if (!vcpu->online) + continue; + + if (virRestoreCgroupThread(cgroup, VIR_CGROUP_THREAD_VCPU, i) < 0) + return; + } + + for (i =3D 0; i < vm->def->niothreadids; i++) { + if (virRestoreCgroupThread(cgroup, VIR_CGROUP_THREAD_IOTHREAD, + vm->def->iothreadids[i]->iothread_id) <= 0) + return; + } + + if (virRestoreCgroupThread(cgroup, VIR_CGROUP_THREAD_EMULATOR, 0) < 0) + return; + + return; + + error: + virResetLastError(); + VIR_DEBUG("Couldn't restore cgroups to meaningful state"); + return; +} + + +int +virRestoreCgroupThread(virCgroup *cgroup, virCgroupThreadName thread, int = id) +{ + g_autoptr(virCgroup) cgroup_temp =3D NULL; + g_autofree char *nodeset =3D NULL; + + if (virCgroupNewThread(cgroup, thread, id, false, &cgroup_temp) < 0) + return -1; + + if (virCgroupSetCpusetMemoryMigrate(cgroup_temp, true) < 0) + return -1; + + if (virCgroupGetCpusetMems(cgroup_temp, &nodeset) < 0) + return -1; + + if (virCgroupSetCpusetMems(cgroup_temp, nodeset) < 0) + return -1; + + return 0; +} + + +int +virConnectCgroup(const char *prefix, virDomainObj *vm, virCgroup *cgroup, + int cgroupControllers, bool privileged, char *machineName) +{ + if (privileged) + return 0; + + if (!virCgroupAvailable()) + return 0; + + virCgroupFree(cgroup); + + if (virCgroupNewDetectMachine(vm->def->name, + prefix, + vm->pid, + cgroupControllers, machineName, &cgroup)= < 0) + return -1; + + virRestoreCgroupState(vm, cgroup); + return 0; +} + + +int +virSetupCgroup(const char *prefix, + virDomainObj * vm, + size_t nnicindexes, int *nicindexes, + virCgroup * cgroup, + int cgroupControllers, + unsigned int maxThreadsPerProc, + bool privileged, char *machineName) +{ + if (!vm->pid) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("Cannot setup cgroups until process is started")); + return -1; + } + + if (virInitCgroup(prefix, vm, nnicindexes, nicindexes, + cgroup, cgroupControllers, + maxThreadsPerProc, privileged, machineName) < 0) + return -1; + + if (!cgroup) + return 0; + + if (virSetupBlkioCgroup(vm, cgroup) < 0) + return -1; + + if (virSetupMemoryCgroup(vm, cgroup) < 0) + return -1; + + if (virSetupCpuCgroup(vm, cgroup) < 0) + return -1; + + if (virSetupCpusetCgroup(cgroup) < 0) + return -1; + + return 0; +} + + +int +virSetupCgroupVcpuBW(virCgroup *cgroup, + unsigned long long period, long long quota) +{ + return virCgroupSetupCpuPeriodQuota(cgroup, period, quota); +} + + +int +virSetupCgroupCpusetCpus(virCgroup *cgroup, virBitmap *cpumask) +{ + return virCgroupSetupCpusetCpus(cgroup, cpumask); +} + + +int +virSetupGlobalCpuCgroup(virDomainObj *vm, virCgroup *cgroup, + virBitmap *autoNodeset) +{ + unsigned long long period =3D vm->def->cputune.global_period; + long long quota =3D vm->def->cputune.global_quota; + g_autofree char *mem_mask =3D NULL; + virDomainNumatuneMemMode mem_mode; + + if ((period || quota) && + !virCgroupHasController(cgroup, VIR_CGROUP_CONTROLLER_CPU)) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", + _("cgroup cpu is required for scheduler tuning")); + return -1; + } + + /* + * If CPU cgroup controller is not initialized here, then we need + * neither period nor quota settings. And if CPUSET controller is + * not initialized either, then there's nothing to do anyway. + */ + if (!virCgroupHasController(cgroup, VIR_CGROUP_CONTROLLER_CPU) && + !virCgroupHasController(cgroup, VIR_CGROUP_CONTROLLER_CPUSET)) + return 0; + + + if (virDomainNumatuneGetMode(vm->def->numa, -1, &mem_mode) =3D=3D 0 && + mem_mode =3D=3D VIR_DOMAIN_NUMATUNE_MEM_STRICT && + virDomainNumatuneMaybeFormatNodeset(vm->def->numa, + autoNodeset, &mem_mask, -1) < = 0) + return -1; + + if (period || quota) { + if (virSetupCgroupVcpuBW(cgroup, period, quota) < 0) + return -1; + } + + return 0; +} + + +int +virRemoveCgroup(virDomainObj *vm, virCgroup *cgroup, char *machineName) +{ + if (cgroup =3D=3D NULL) + return 0; /* Not supported, so claim success */ + + if (virCgroupTerminateMachine(machineName) < 0) { + if (!virCgroupNewIgnoreError()) + VIR_DEBUG("Failed to terminate cgroup for %s", vm->def->name); + } + + return virCgroupRemove(cgroup); +} + + +void +virCgroupEmulatorAllNodesDataFree(virCgroupEmulatorAllNodesData *data) +{ + if (!data) + return; + + virCgroupFree(data->emulatorCgroup); + g_free(data->emulatorMemMask); + g_free(data); +} + + +/** + * virCgroupEmulatorAllNodesAllow: + * @cgroup: domain cgroup pointer + * @retData: filled with structure used to roll back the operation + * + * Allows all NUMA nodes for the cloud hypervisor thread temporarily. This= is + * necessary when hotplugging cpus since it requires memory allocated in t= he + * DMA region. Afterwards the operation can be reverted by + * virCgroupEmulatorAllNodesRestore. + * + * Returns 0 on success -1 on error + */ +int +virCgroupEmulatorAllNodesAllow(virCgroup *cgroup, + virCgroupEmulatorAllNodesData **retData) +{ + virCgroupEmulatorAllNodesData *data =3D NULL; + g_autofree char *all_nodes_str =3D NULL; + + g_autoptr(virBitmap) all_nodes =3D NULL; + int ret =3D -1; + + if (!virNumaIsAvailable() || + !virCgroupHasController(cgroup, VIR_CGROUP_CONTROLLER_CPUSET)) + return 0; + + if (!(all_nodes =3D virNumaGetHostMemoryNodeset())) + goto cleanup; + + if (!(all_nodes_str =3D virBitmapFormat(all_nodes))) + goto cleanup; + + data =3D g_new0(virCgroupEmulatorAllNodesData, 1); + + if (virCgroupNewThread(cgroup, VIR_CGROUP_THREAD_EMULATOR, 0, + false, &data->emulatorCgroup) < 0) + goto cleanup; + + if (virCgroupGetCpusetMems(data->emulatorCgroup, &data->emulatorMemMas= k) < 0 + || virCgroupSetCpusetMems(data->emulatorCgroup, all_nodes_str) < 0) + goto cleanup; + + *retData =3D g_steal_pointer(&data); + ret =3D 0; + + cleanup: + virCgroupEmulatorAllNodesDataFree(data); + + return ret; +} + + +/** + * virCgroupEmulatorAllNodesRestore: + * @data: data structure created by virCgroupEmulatorAllNodesAllow + * + * Rolls back the setting done by virCgroupEmulatorAllNodesAllow and frees= the + * associated data. + */ +void +virCgroupEmulatorAllNodesRestore(virCgroupEmulatorAllNodesData *data) +{ + virError *err; + + if (!data) + return; + + virErrorPreserveLast(&err); + virCgroupSetCpusetMems(data->emulatorCgroup, data->emulatorMemMask); + virErrorRestore(&err); + + virCgroupEmulatorAllNodesDataFree(data); +} diff --git a/src/hypervisor/domain_cgroup.h b/src/hypervisor/domain_cgroup.h index f93e5f74fe..fd8b74a4ee 100644 --- a/src/hypervisor/domain_cgroup.h +++ b/src/hypervisor/domain_cgroup.h @@ -23,6 +23,11 @@ #include "vircgroup.h" #include "domain_conf.h" =20 +typedef struct _virCgroupEmulatorAllNodesData virCgroupEmulatorAllNodesDat= a; +struct _virCgroupEmulatorAllNodesData { + virCgroup *emulatorCgroup; + char *emulatorMemMask; +}; =20 int virDomainCgroupSetupBlkio(virCgroup *cgroup, virDomainBlkiotune blkio); int virDomainCgroupSetupMemtune(virCgroup *cgroup, virDomainMemtune mem); @@ -36,3 +41,50 @@ int virDomainCgroupSetMemoryLimitParameters(virCgroup *c= group, virDomainDef *persistentDef, virTypedParameterPtr params, int nparams); +int +virSetupBlkioCgroup(virDomainObj * vm, virCgroup *cgroup); +int +virSetupMemoryCgroup(virDomainObj * vm, virCgroup *cgroup); +int +virSetupCpusetCgroup(virCgroup *cgroup); +int +virSetupCpuCgroup(virDomainObj * vm, virCgroup *cgroup); +int +virInitCgroup(const char *prefix, virDomainObj * vm, + size_t nnicindexes, int *nicindexes, + virCgroup *cgroup, int cgroupControllers, + unsigned int maxThreadsPerProc, + bool privileged, + char *machineName); +void +virRestoreCgroupState(virDomainObj * vm, virCgroup *cgroup); +int +virConnectCgroup(const char *prefix, virDomainObj * vm, virCgroup *cgroup, + int cgroupControllers, bool privileged, char *machineName= ); +int +virSetupCgroup(const char *prefix, virDomainObj * vm, + size_t nnicindexes, int *nicindexes, + virCgroup *cgroup, int cgroupControllers, + unsigned int maxThreadsPerProc, + bool privileged, + char *machineName); +void +virCgroupEmulatorAllNodesDataFree(virCgroupEmulatorAllNodesData * data); +int +virCgroupEmulatorAllNodesAllow(virCgroup * cgroup, + virCgroupEmulatorAllNodesData ** retData); +void +virCgroupEmulatorAllNodesRestore(virCgroupEmulatorAllNodesData * data); +int +virSetupCgroupVcpuBW(virCgroup * cgroup, + unsigned long long period, long long quota); +int +virSetupCgroupCpusetCpus(virCgroup * cgroup, virBitmap * cpumask); +int +virSetupGlobalCpuCgroup(virDomainObj * vm, virCgroup * cgroup, + virBitmap *autoNodeset); +int +virRemoveCgroup(virDomainObj * vm, virCgroup * cgroup, char *machineName); +int +virRestoreCgroupThread(virCgroup *cgroup, virCgroupThreadName thread, + int id); diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index 56adc192cd..09b1fbb8c4 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -1543,10 +1543,23 @@ virSetConnectStorage; =20 =20 # hypervisor/domain_cgroup.h +virCgroupEmulatorAllNodesAllow; +virCgroupEmulatorAllNodesRestore; +virConnectCgroup; virDomainCgroupSetMemoryLimitParameters; virDomainCgroupSetupBlkio; virDomainCgroupSetupDomainBlkioParameters; virDomainCgroupSetupMemtune; +virInitCgroup; +virRemoveCgroup; +virSetupBlkioCgroup; +virSetupCgroup; +virSetupCgroupCpusetCpus; +virSetupCgroupVcpuBW; +virSetupCpuCgroup; +virSetupCpusetCgroup; +virSetupGlobalCpuCgroup; +virSetupMemoryCgroup; =20 =20 # hypervisor/domain_driver.h diff --git a/src/qemu/qemu_cgroup.c b/src/qemu/qemu_cgroup.c index 308be5b00f..b1d2875959 100644 --- a/src/qemu/qemu_cgroup.c +++ b/src/qemu/qemu_cgroup.c @@ -593,46 +593,6 @@ qemuSetupVideoCgroup(virDomainObj *vm, return ret; } =20 - -static int -qemuSetupBlkioCgroup(virDomainObj *vm) -{ - qemuDomainObjPrivate *priv =3D vm->privateData; - - if (!virCgroupHasController(priv->cgroup, - VIR_CGROUP_CONTROLLER_BLKIO)) { - if (vm->def->blkio.weight || vm->def->blkio.ndevices) { - virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", - _("Block I/O tuning is not available on this ho= st")); - return -1; - } - return 0; - } - - return virDomainCgroupSetupBlkio(priv->cgroup, vm->def->blkio); -} - - -static int -qemuSetupMemoryCgroup(virDomainObj *vm) -{ - qemuDomainObjPrivate *priv =3D vm->privateData; - - if (!virCgroupHasController(priv->cgroup, VIR_CGROUP_CONTROLLER_MEMORY= )) { - if (virMemoryLimitIsSet(vm->def->mem.hard_limit) || - virMemoryLimitIsSet(vm->def->mem.soft_limit) || - virMemoryLimitIsSet(vm->def->mem.swap_hard_limit)) { - virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", - _("Memory cgroup is not available on this host"= )); - return -1; - } - return 0; - } - - return virDomainCgroupSetupMemtune(priv->cgroup, vm->def->mem); -} - - static int qemuSetupFirmwareCgroup(virDomainObj *vm) { @@ -861,44 +821,6 @@ qemuSetupDevicesCgroup(virDomainObj *vm) } =20 =20 -static int -qemuSetupCpusetCgroup(virDomainObj *vm) -{ - qemuDomainObjPrivate *priv =3D vm->privateData; - - if (!virCgroupHasController(priv->cgroup, VIR_CGROUP_CONTROLLER_CPUSET= )) - return 0; - - if (virCgroupSetCpusetMemoryMigrate(priv->cgroup, true) < 0) - return -1; - - return 0; -} - - -static int -qemuSetupCpuCgroup(virDomainObj *vm) -{ - qemuDomainObjPrivate *priv =3D vm->privateData; - - if (!virCgroupHasController(priv->cgroup, VIR_CGROUP_CONTROLLER_CPU)) { - if (vm->def->cputune.sharesSpecified) { - virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", - _("CPU tuning is not available on this host")); - return -1; - } - return 0; - } - - if (vm->def->cputune.sharesSpecified) { - if (virCgroupSetCpuShares(priv->cgroup, vm->def->cputune.shares) <= 0) - return -1; - } - - return 0; -} - - static int qemuSetupCgroupAppid(virDomainObj *vm) { @@ -927,166 +849,13 @@ qemuSetupCgroupAppid(virDomainObj *vm) } =20 =20 -static int -qemuInitCgroup(virDomainObj *vm, - size_t nnicindexes, - int *nicindexes) -{ - qemuDomainObjPrivate *priv =3D vm->privateData; - g_autoptr(virQEMUDriverConfig) cfg =3D virQEMUDriverGetConfig(priv->dr= iver); - - if (!priv->driver->privileged) - return 0; - - if (!virCgroupAvailable()) - return 0; - - virCgroupFree(priv->cgroup); - priv->cgroup =3D NULL; - - if (!vm->def->resource) - vm->def->resource =3D g_new0(virDomainResourceDef, 1); - - if (!vm->def->resource->partition) - vm->def->resource->partition =3D g_strdup("/machine"); - - if (!g_path_is_absolute(vm->def->resource->partition)) { - virReportError(VIR_ERR_CONFIG_UNSUPPORTED, - _("Resource partition '%s' must start with '/'"), - vm->def->resource->partition); - return -1; - } - - if (virCgroupNewMachine(priv->machineName, - "qemu", - vm->def->uuid, - NULL, - vm->pid, - false, - nnicindexes, nicindexes, - vm->def->resource->partition, - cfg->cgroupControllers, - cfg->maxThreadsPerProc, - &priv->cgroup) < 0) { - if (virCgroupNewIgnoreError()) - return 0; - - return -1; - } - - return 0; -} - -static int -qemuRestoreCgroupThread(virCgroup *cgroup, - virCgroupThreadName thread, - int id) -{ - g_autoptr(virCgroup) cgroup_temp =3D NULL; - g_autofree char *nodeset =3D NULL; - - if (virCgroupNewThread(cgroup, thread, id, false, &cgroup_temp) < 0) - return -1; - - if (virCgroupSetCpusetMemoryMigrate(cgroup_temp, true) < 0) - return -1; - - if (virCgroupGetCpusetMems(cgroup_temp, &nodeset) < 0) - return -1; - - if (virCgroupSetCpusetMems(cgroup_temp, nodeset) < 0) - return -1; - - return 0; -} - -static void -qemuRestoreCgroupState(virDomainObj *vm) -{ - g_autofree char *mem_mask =3D NULL; - qemuDomainObjPrivate *priv =3D vm->privateData; - size_t i =3D 0; - g_autoptr(virBitmap) all_nodes =3D NULL; - - if (!virNumaIsAvailable() || - !virCgroupHasController(priv->cgroup, VIR_CGROUP_CONTROLLER_CPUSET= )) - return; - - if (!(all_nodes =3D virNumaGetHostMemoryNodeset())) - goto error; - - if (!(mem_mask =3D virBitmapFormat(all_nodes))) - goto error; - - if (virCgroupHasEmptyTasks(priv->cgroup, - VIR_CGROUP_CONTROLLER_CPUSET) <=3D 0) - goto error; - - if (virCgroupSetCpusetMems(priv->cgroup, mem_mask) < 0) - goto error; - - for (i =3D 0; i < virDomainDefGetVcpusMax(vm->def); i++) { - virDomainVcpuDef *vcpu =3D virDomainDefGetVcpu(vm->def, i); - - if (!vcpu->online) - continue; - - if (qemuRestoreCgroupThread(priv->cgroup, - VIR_CGROUP_THREAD_VCPU, i) < 0) - return; - } - - for (i =3D 0; i < vm->def->niothreadids; i++) { - if (qemuRestoreCgroupThread(priv->cgroup, VIR_CGROUP_THREAD_IOTHRE= AD, - vm->def->iothreadids[i]->iothread_id) = < 0) - return; - } - - if (qemuRestoreCgroupThread(priv->cgroup, - VIR_CGROUP_THREAD_EMULATOR, 0) < 0) - return; - - return; - - error: - virResetLastError(); - VIR_DEBUG("Couldn't restore cgroups to meaningful state"); - return; -} - -int -qemuConnectCgroup(virDomainObj *vm) -{ - qemuDomainObjPrivate *priv =3D vm->privateData; - g_autoptr(virQEMUDriverConfig) cfg =3D virQEMUDriverGetConfig(priv->dr= iver); - - if (!priv->driver->privileged) - return 0; - - if (!virCgroupAvailable()) - return 0; - - virCgroupFree(priv->cgroup); - priv->cgroup =3D NULL; - - if (virCgroupNewDetectMachine(vm->def->name, - "qemu", - vm->pid, - cfg->cgroupControllers, - priv->machineName, - &priv->cgroup) < 0) - return -1; - - qemuRestoreCgroupState(vm); - return 0; -} - int qemuSetupCgroup(virDomainObj *vm, size_t nnicindexes, int *nicindexes) { qemuDomainObjPrivate *priv =3D vm->privateData; + g_autoptr(virQEMUDriverConfig) cfg =3D virQEMUDriverGetConfig(priv->dr= iver); =20 if (!vm->pid) { virReportError(VIR_ERR_INTERNAL_ERROR, "%s", @@ -1094,7 +863,12 @@ qemuSetupCgroup(virDomainObj *vm, return -1; } =20 - if (qemuInitCgroup(vm, nnicindexes, nicindexes) < 0) + if (virInitCgroup("qemu", vm, nnicindexes, nicindexes, + priv->cgroup, + cfg->cgroupControllers, + cfg->maxThreadsPerProc, + priv->driver->privileged, + priv->machineName) < 0) return -1; =20 if (!priv->cgroup) @@ -1103,16 +877,16 @@ qemuSetupCgroup(virDomainObj *vm, if (qemuSetupDevicesCgroup(vm) < 0) return -1; =20 - if (qemuSetupBlkioCgroup(vm) < 0) + if (virSetupBlkioCgroup(vm, priv->cgroup) < 0) return -1; =20 - if (qemuSetupMemoryCgroup(vm) < 0) + if (virSetupMemoryCgroup(vm, priv->cgroup) < 0) return -1; =20 - if (qemuSetupCpuCgroup(vm) < 0) + if (virSetupCpuCgroup(vm, priv->cgroup) < 0) return -1; =20 - if (qemuSetupCpusetCgroup(vm) < 0) + if (virSetupCpusetCgroup(priv->cgroup) < 0) return -1; =20 if (qemuSetupCgroupAppid(vm) < 0) @@ -1121,23 +895,6 @@ qemuSetupCgroup(virDomainObj *vm, return 0; } =20 -int -qemuSetupCgroupVcpuBW(virCgroup *cgroup, - unsigned long long period, - long long quota) -{ - return virCgroupSetupCpuPeriodQuota(cgroup, period, quota); -} - - -int -qemuSetupCgroupCpusetCpus(virCgroup *cgroup, - virBitmap *cpumask) -{ - return virCgroupSetupCpusetCpus(cgroup, cpumask); -} - - int qemuSetupCgroupForExtDevices(virDomainObj *vm, virQEMUDriver *driver) @@ -1164,148 +921,3 @@ qemuSetupCgroupForExtDevices(virDomainObj *vm, =20 return qemuExtDevicesSetupCgroup(driver, vm, cgroup_temp); } - - -int -qemuSetupGlobalCpuCgroup(virDomainObj *vm) -{ - qemuDomainObjPrivate *priv =3D vm->privateData; - unsigned long long period =3D vm->def->cputune.global_period; - long long quota =3D vm->def->cputune.global_quota; - g_autofree char *mem_mask =3D NULL; - virDomainNumatuneMemMode mem_mode; - - if ((period || quota) && - !virCgroupHasController(priv->cgroup, VIR_CGROUP_CONTROLLER_CPU)) { - virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", - _("cgroup cpu is required for scheduler tuning")); - return -1; - } - - /* - * If CPU cgroup controller is not initialized here, then we need - * neither period nor quota settings. And if CPUSET controller is - * not initialized either, then there's nothing to do anyway. - */ - if (!virCgroupHasController(priv->cgroup, VIR_CGROUP_CONTROLLER_CPU) && - !virCgroupHasController(priv->cgroup, VIR_CGROUP_CONTROLLER_CPUSET= )) - return 0; - - - if (virDomainNumatuneGetMode(vm->def->numa, -1, &mem_mode) =3D=3D 0 && - mem_mode =3D=3D VIR_DOMAIN_NUMATUNE_MEM_STRICT && - virDomainNumatuneMaybeFormatNodeset(vm->def->numa, - priv->autoNodeset, - &mem_mask, -1) < 0) - return -1; - - if (period || quota) { - if (qemuSetupCgroupVcpuBW(priv->cgroup, period, quota) < 0) - return -1; - } - - return 0; -} - - -int -qemuRemoveCgroup(virDomainObj *vm) -{ - qemuDomainObjPrivate *priv =3D vm->privateData; - - if (priv->cgroup =3D=3D NULL) - return 0; /* Not supported, so claim success */ - - if (virCgroupTerminateMachine(priv->machineName) < 0) { - if (!virCgroupNewIgnoreError()) - VIR_DEBUG("Failed to terminate cgroup for %s", vm->def->name); - } - - return virCgroupRemove(priv->cgroup); -} - - -static void -qemuCgroupEmulatorAllNodesDataFree(qemuCgroupEmulatorAllNodesData *data) -{ - if (!data) - return; - - virCgroupFree(data->emulatorCgroup); - g_free(data->emulatorMemMask); - g_free(data); -} - - -/** - * qemuCgroupEmulatorAllNodesAllow: - * @cgroup: domain cgroup pointer - * @retData: filled with structure used to roll back the operation - * - * Allows all NUMA nodes for the qemu emulator thread temporarily. This is - * necessary when hotplugging cpus since it requires memory allocated in t= he - * DMA region. Afterwards the operation can be reverted by - * qemuCgroupEmulatorAllNodesRestore. - * - * Returns 0 on success -1 on error - */ -int -qemuCgroupEmulatorAllNodesAllow(virCgroup *cgroup, - qemuCgroupEmulatorAllNodesData **retData) -{ - qemuCgroupEmulatorAllNodesData *data =3D NULL; - g_autofree char *all_nodes_str =3D NULL; - g_autoptr(virBitmap) all_nodes =3D NULL; - int ret =3D -1; - - if (!virNumaIsAvailable() || - !virCgroupHasController(cgroup, VIR_CGROUP_CONTROLLER_CPUSET)) - return 0; - - if (!(all_nodes =3D virNumaGetHostMemoryNodeset())) - goto cleanup; - - if (!(all_nodes_str =3D virBitmapFormat(all_nodes))) - goto cleanup; - - data =3D g_new0(qemuCgroupEmulatorAllNodesData, 1); - - if (virCgroupNewThread(cgroup, VIR_CGROUP_THREAD_EMULATOR, 0, - false, &data->emulatorCgroup) < 0) - goto cleanup; - - if (virCgroupGetCpusetMems(data->emulatorCgroup, &data->emulatorMemMas= k) < 0 || - virCgroupSetCpusetMems(data->emulatorCgroup, all_nodes_str) < 0) - goto cleanup; - - *retData =3D g_steal_pointer(&data); - ret =3D 0; - - cleanup: - qemuCgroupEmulatorAllNodesDataFree(data); - - return ret; -} - - -/** - * qemuCgroupEmulatorAllNodesRestore: - * @data: data structure created by qemuCgroupEmulatorAllNodesAllow - * - * Rolls back the setting done by qemuCgroupEmulatorAllNodesAllow and free= s the - * associated data. - */ -void -qemuCgroupEmulatorAllNodesRestore(qemuCgroupEmulatorAllNodesData *data) -{ - virErrorPtr err; - - if (!data) - return; - - virErrorPreserveLast(&err); - virCgroupSetCpusetMems(data->emulatorCgroup, data->emulatorMemMask); - virErrorRestore(&err); - - qemuCgroupEmulatorAllNodesDataFree(data); -} diff --git a/src/qemu/qemu_cgroup.h b/src/qemu/qemu_cgroup.h index cd537ebd82..f09134947f 100644 --- a/src/qemu/qemu_cgroup.h +++ b/src/qemu/qemu_cgroup.h @@ -56,18 +56,11 @@ int qemuSetupChardevCgroup(virDomainObj *vm, virDomainChrDef *dev); int qemuTeardownChardevCgroup(virDomainObj *vm, virDomainChrDef *dev); -int qemuConnectCgroup(virDomainObj *vm); int qemuSetupCgroup(virDomainObj *vm, size_t nnicindexes, int *nicindexes); -int qemuSetupCgroupVcpuBW(virCgroup *cgroup, - unsigned long long period, - long long quota); -int qemuSetupCgroupCpusetCpus(virCgroup *cgroup, virBitmap *cpumask); -int qemuSetupGlobalCpuCgroup(virDomainObj *vm); int qemuSetupCgroupForExtDevices(virDomainObj *vm, virQEMUDriver *driver); -int qemuRemoveCgroup(virDomainObj *vm); =20 typedef struct _qemuCgroupEmulatorAllNodesData qemuCgroupEmulatorAllNodesD= ata; struct _qemuCgroupEmulatorAllNodesData { @@ -75,8 +68,4 @@ struct _qemuCgroupEmulatorAllNodesData { char *emulatorMemMask; }; =20 -int qemuCgroupEmulatorAllNodesAllow(virCgroup *cgroup, - qemuCgroupEmulatorAllNodesData **data); -void qemuCgroupEmulatorAllNodesRestore(qemuCgroupEmulatorAllNodesData *dat= a); - extern const char *const defaultDeviceACL[]; diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index ba3efef42b..d365036743 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -4424,7 +4424,7 @@ qemuDomainPinVcpuLive(virDomainObj *vm, if (virCgroupNewThread(priv->cgroup, VIR_CGROUP_THREAD_VCPU, v= cpu, false, &cgroup_vcpu) < 0) goto cleanup; - if (qemuSetupCgroupCpusetCpus(cgroup_vcpu, cpumap) < 0) + if (virSetupCgroupCpusetCpus(cgroup_vcpu, cpumap) < 0) goto cleanup; } =20 @@ -4633,7 +4633,7 @@ qemuDomainPinEmulator(virDomainPtr dom, 0, false, &cgroup_emulator) < 0) goto endjob; =20 - if (qemuSetupCgroupCpusetCpus(cgroup_emulator, pcpumap) < 0) { + if (virSetupCgroupCpusetCpus(cgroup_emulator, pcpumap) < 0) { virReportError(VIR_ERR_OPERATION_INVALID, "%s", _("failed to set cpuset.cpus in cgroup" " for emulator threads")); @@ -5038,7 +5038,7 @@ qemuDomainPinIOThread(virDomainPtr dom, if (virCgroupNewThread(priv->cgroup, VIR_CGROUP_THREAD_IOTHREA= D, iothread_id, false, &cgroup_iothread) <= 0) goto endjob; - if (qemuSetupCgroupCpusetCpus(cgroup_iothread, pcpumap) < 0) { + if (virSetupCgroupCpusetCpus(cgroup_iothread, pcpumap) < 0) { virReportError(VIR_ERR_OPERATION_INVALID, _("failed to set cpuset.cpus in cgroup" " for iothread %d"), iothread_id); @@ -8926,7 +8926,7 @@ qemuSetGlobalBWLive(virCgroup *cgroup, unsigned long = long period, if (period =3D=3D 0 && quota =3D=3D 0) return 0; =20 - if (qemuSetupCgroupVcpuBW(cgroup, period, quota) < 0) + if (virSetupCgroupVcpuBW(cgroup, period, quota) < 0) return -1; =20 return 0; @@ -9121,7 +9121,7 @@ qemuSetVcpusBWLive(virDomainObj *vm, virCgroup *cgrou= p, false, &cgroup_vcpu) < 0) return -1; =20 - if (qemuSetupCgroupVcpuBW(cgroup_vcpu, period, quota) < 0) + if (virSetupCgroupVcpuBW(cgroup_vcpu, period, quota) < 0) return -1; } =20 @@ -9142,7 +9142,7 @@ qemuSetEmulatorBandwidthLive(virCgroup *cgroup, false, &cgroup_emulator) < 0) return -1; =20 - if (qemuSetupCgroupVcpuBW(cgroup_emulator, period, quota) < 0) + if (virSetupCgroupVcpuBW(cgroup_emulator, period, quota) < 0) return -1; =20 return 0; @@ -9169,7 +9169,7 @@ qemuSetIOThreadsBWLive(virDomainObj *vm, virCgroup *c= group, false, &cgroup_iothread) < 0) return -1; =20 - if (qemuSetupCgroupVcpuBW(cgroup_iothread, period, quota) < 0) + if (virSetupCgroupVcpuBW(cgroup_iothread, period, quota) < 0) return -1; } =20 diff --git a/src/qemu/qemu_hotplug.c b/src/qemu/qemu_hotplug.c index 04c6600f26..30f04aac57 100644 --- a/src/qemu/qemu_hotplug.c +++ b/src/qemu/qemu_hotplug.c @@ -37,6 +37,7 @@ #include "qemu_snapshot.h" #include "qemu_virtiofs.h" #include "domain_audit.h" +#include "domain_cgroup.h" #include "netdev_bandwidth_conf.h" #include "domain_nwfilter.h" #include "virlog.h" @@ -6551,11 +6552,11 @@ qemuDomainSetVcpusLive(virQEMUDriver *driver, bool enable) { qemuDomainObjPrivate *priv =3D vm->privateData; - qemuCgroupEmulatorAllNodesData *emulatorCgroup =3D NULL; + virCgroupEmulatorAllNodesData *emulatorCgroup =3D NULL; ssize_t nextvcpu =3D -1; int ret =3D -1; =20 - if (qemuCgroupEmulatorAllNodesAllow(priv->cgroup, &emulatorCgroup) < 0) + if (virCgroupEmulatorAllNodesAllow(priv->cgroup, &emulatorCgroup) < 0) goto cleanup; =20 if (enable) { @@ -6576,7 +6577,7 @@ qemuDomainSetVcpusLive(virQEMUDriver *driver, ret =3D 0; =20 cleanup: - qemuCgroupEmulatorAllNodesRestore(emulatorCgroup); + virCgroupEmulatorAllNodesRestore(emulatorCgroup); =20 return ret; } diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index 4963ce383f..21c41e83c9 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -73,6 +73,7 @@ #include "virpidfile.h" #include "virhostcpu.h" #include "domain_audit.h" +#include "domain_cgroup.h" #include "domain_nwfilter.h" #include "domain_validate.h" #include "locking/domain_lock.h" @@ -2744,7 +2745,7 @@ qemuProcessSetupPid(virDomainObj *vm, =20 if (virCgroupHasController(priv->cgroup, VIR_CGROUP_CONTROLLER_CPU= SET)) { if (use_cpumask && - qemuSetupCgroupCpusetCpus(cgroup, use_cpumask) < 0) + virSetupCgroupCpusetCpus(cgroup, use_cpumask) < 0) goto cleanup; =20 if (mem_mask && virCgroupSetCpusetMems(cgroup, mem_mask) < 0) @@ -2753,7 +2754,7 @@ qemuProcessSetupPid(virDomainObj *vm, } =20 if ((period || quota) && - qemuSetupCgroupVcpuBW(cgroup, period, quota) < 0) + virSetupCgroupVcpuBW(cgroup, period, quota) < 0) goto cleanup; =20 /* Move the thread to the sub dir */ @@ -6027,7 +6028,7 @@ qemuProcessSetupHotpluggableVcpus(virQEMUDriver *driv= er, { unsigned int maxvcpus =3D virDomainDefGetVcpusMax(vm->def); qemuDomainObjPrivate *priv =3D vm->privateData; - qemuCgroupEmulatorAllNodesData *emulatorCgroup =3D NULL; + virCgroupEmulatorAllNodesData *emulatorCgroup =3D NULL; virDomainVcpuDef *vcpu; qemuDomainVcpuPrivate *vcpupriv; size_t i; @@ -6055,7 +6056,7 @@ qemuProcessSetupHotpluggableVcpus(virQEMUDriver *driv= er, qsort(bootHotplug, nbootHotplug, sizeof(*bootHotplug), qemuProcessVcpusSortOrder); =20 - if (qemuCgroupEmulatorAllNodesAllow(priv->cgroup, &emulatorCgroup) < 0) + if (virCgroupEmulatorAllNodesAllow(priv->cgroup, &emulatorCgroup) < 0) goto cleanup; =20 for (i =3D 0; i < nbootHotplug; i++) { @@ -6079,7 +6080,7 @@ qemuProcessSetupHotpluggableVcpus(virQEMUDriver *driv= er, ret =3D 0; =20 cleanup: - qemuCgroupEmulatorAllNodesRestore(emulatorCgroup); + virCgroupEmulatorAllNodesRestore(emulatorCgroup); return ret; } =20 @@ -7061,7 +7062,7 @@ qemuProcessPrepareHost(virQEMUDriver *driver, /* Ensure no historical cgroup for this VM is lying around bogus * settings */ VIR_DEBUG("Ensuring no historical cgroup is lying around"); - qemuRemoveCgroup(vm); + virRemoveCgroup(vm, priv->cgroup, priv->machineName); =20 if (g_mkdir_with_parents(cfg->logDir, 0777) < 0) { virReportSystemError(errno, @@ -7678,7 +7679,7 @@ qemuProcessLaunch(virConnectPtr conn, goto cleanup; =20 VIR_DEBUG("Setting global CPU cgroup (if required)"); - if (qemuSetupGlobalCpuCgroup(vm) < 0) + if (virSetupGlobalCpuCgroup(vm, priv->cgroup, priv->autoNodeset) < 0) goto cleanup; =20 VIR_DEBUG("Setting vCPU tuning/settings"); @@ -8290,7 +8291,7 @@ void qemuProcessStop(virQEMUDriver *driver, } =20 retry: - if ((ret =3D qemuRemoveCgroup(vm)) < 0) { + if ((ret =3D virRemoveCgroup(vm, priv->cgroup, priv->machineName)) < 0= ) { if (ret =3D=3D -EBUSY && (retries++ < 5)) { g_usleep(200*1000); goto retry; @@ -8849,7 +8850,8 @@ qemuProcessReconnect(void *opaque) if (!priv->machineName) goto error; =20 - if (qemuConnectCgroup(obj) < 0) + if (virConnectCgroup("qemu", obj, priv->cgroup, cfg->cgroupControllers, + priv->driver->privileged, priv->machineName) < 0) goto error; =20 if (qemuDomainPerfRestart(obj) < 0) --=20 2.27.0 From nobody Sun May 5 13:15:08 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) client-ip=170.10.129.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail(p=none dis=none) header.from=linux.microsoft.com Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mx.zohomail.com with SMTPS id 1639168533481463.8049812885963; Fri, 10 Dec 2021 12:35:33 -0800 (PST) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-361-QiyWjglcMPuOdvInSE3NCQ-1; Fri, 10 Dec 2021 15:35:30 -0500 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 27FE410144EC; Fri, 10 Dec 2021 20:35:23 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 06CF41ABE5; Fri, 10 Dec 2021 20:35:23 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id C54234A7CD; Fri, 10 Dec 2021 20:35:22 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 1BAKYwlK008371 for ; Fri, 10 Dec 2021 15:34:59 -0500 Received: by smtp.corp.redhat.com (Postfix) id D37C7C15E73; Fri, 10 Dec 2021 20:34:58 +0000 (UTC) Received: from mimecast-mx02.redhat.com (mimecast07.extmail.prod.ext.rdu2.redhat.com [10.11.55.23]) by smtp.corp.redhat.com (Postfix) with ESMTPS id CEF09C159F6 for ; Fri, 10 Dec 2021 20:34:58 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [207.211.31.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id B1D453C9A02D for ; Fri, 10 Dec 2021 20:34:58 +0000 (UTC) Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by relay.mimecast.com with ESMTP id us-mta-183-qLgchfqJPXyp8de0_2UutQ-1; Fri, 10 Dec 2021 15:34:56 -0500 Received: from prapal-ch2.oiwy50ateaxezb1sqsoezlib2f.xx.internal.cloudapp.net (unknown [20.80.162.67]) by linux.microsoft.com (Postfix) with ESMTPSA id DB7AA20B718B; Fri, 10 Dec 2021 12:34:54 -0800 (PST) X-MC-Unique: QiyWjglcMPuOdvInSE3NCQ-1 X-MC-Unique: qLgchfqJPXyp8de0_2UutQ-1 DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com DB7AA20B718B From: Praveen K Paladugu To: libvir-list@redhat.com Subject: [libvirt PATCH v3 08/13] ch: methods for cgroup mgmt in ch driver Date: Fri, 10 Dec 2021 20:34:43 +0000 Message-Id: <20211210203448.1579606-9-prapal@linux.microsoft.com> In-Reply-To: <20211210203448.1579606-1-prapal@linux.microsoft.com> References: <20211210203448.1579606-1-prapal@linux.microsoft.com> MIME-Version: 1.0 X-Mimecast-Impersonation-Protect: Policy=CLT - Impersonation Protection Definition; Similar Internal Domain=false; Similar Monitored External Domain=false; Custom External Domain=false; Mimecast External Domain=false; Newly Observed Domain=false; Internal User Name=false; Custom Display Name List=false; Reply-to Address Mismatch=false; Targeted Threat Dictionary=false; Mimecast Threat Dictionary=false; Custom Threat Dictionary=false X-Scanned-By: MIMEDefang 2.85 on 10.11.54.8 X-MIME-Autoconverted: from quoted-printable to 8bit by lists01.pubmisc.prod.ext.phx2.redhat.com id 1BAKYwlK008371 X-loop: libvir-list@redhat.com Cc: william.douglas@intel.com X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZM-MESSAGEID: 1639168534190100003 Content-Type: text/plain; charset="utf-8" From: Vineeth Pillai Signed-off-by: Vineeth Pillai Signed-off-by: Praveen K Paladugu --- src/ch/ch_conf.c | 2 + src/ch/ch_conf.h | 4 +- src/ch/ch_domain.c | 34 +++++ src/ch/ch_domain.h | 3 +- src/ch/ch_monitor.c | 96 ++++++++++++++ src/ch/ch_monitor.h | 54 +++++++- src/ch/ch_process.c | 304 ++++++++++++++++++++++++++++++++++++++++++-- src/ch/ch_process.h | 3 + 8 files changed, 483 insertions(+), 17 deletions(-) diff --git a/src/ch/ch_conf.c b/src/ch/ch_conf.c index ef6f4b5ba8..cfc1174354 100644 --- a/src/ch/ch_conf.c +++ b/src/ch/ch_conf.c @@ -129,6 +129,8 @@ virCHDriverConfigNew(bool privileged) if (!(cfg =3D virObjectNew(virCHDriverConfigClass))) return NULL; =20 + cfg->cgroupControllers =3D -1; /* Auto detect */ + if (privileged) { if (virGetUserID(CH_USER, &cfg->user) < 0) return NULL; diff --git a/src/ch/ch_conf.h b/src/ch/ch_conf.h index 8fe69c8545..1790295ede 100644 --- a/src/ch/ch_conf.h +++ b/src/ch/ch_conf.h @@ -35,11 +35,13 @@ struct _virCHDriverConfig { =20 char *stateDir; char *logDir; - + int cgroupControllers; uid_t user; gid_t group; }; =20 +G_DEFINE_AUTOPTR_CLEANUP_FUNC(virCHDriverConfig, virObjectUnref); + struct _virCHDriver { virMutex lock; diff --git a/src/ch/ch_domain.c b/src/ch/ch_domain.c index 3a32ac63d9..42cb4e7dcf 100644 --- a/src/ch/ch_domain.c +++ b/src/ch/ch_domain.c @@ -324,6 +324,40 @@ chValidateDomainDeviceDef(const virDomainDeviceDef *de= v, _("Serial can only be enabled for a PTY")); return -1; } + return 0; +} + +int +virCHDomainRefreshThreadInfo(virDomainObj *vm) +{ + size_t maxvcpus =3D virDomainDefGetVcpusMax(vm->def); + virCHMonitorThreadInfo *info =3D NULL; + size_t nthreads, ncpus =3D 0; + size_t i; + + nthreads =3D virCHMonitorGetThreadInfo(virCHDomainGetMonitor(vm), + true, &info); + + for (i =3D 0; i < nthreads; i++) { + virCHDomainVcpuPrivate *vcpupriv; + virDomainVcpuDef *vcpu; + virCHMonitorCPUInfo *vcpuInfo; + + if (info[i].type !=3D virCHThreadTypeVcpu) + continue; + + /* TODO: hotplug support */ + vcpuInfo =3D &info[i].vcpuInfo; + vcpu =3D virDomainDefGetVcpu(vm->def, vcpuInfo->cpuid); + vcpupriv =3D CH_DOMAIN_VCPU_PRIVATE(vcpu); + vcpupriv->tid =3D vcpuInfo->tid; + ncpus++; + } + + /* TODO: Remove the warning when hotplug is implemented.*/ + if (ncpus !=3D maxvcpus) + VIR_WARN("Mismatch in the number of cpus, expected: %ld, actual: %= ld", + maxvcpus, ncpus); =20 return 0; } diff --git a/src/ch/ch_domain.h b/src/ch/ch_domain.h index 4fc6251380..7cffe2b4fb 100644 --- a/src/ch/ch_domain.h +++ b/src/ch/ch_domain.h @@ -89,7 +89,8 @@ virCHDomainObjBeginJob(virDomainObj *obj, enum virCHDomai= nJob job) void virCHDomainObjEndJob(virDomainObj *obj); =20 -int virCHDomainRefreshVcpuInfo(virDomainObj *vm); +int virCHDomainRefreshThreadInfo(virDomainObj *vm); + pid_t virCHDomainGetVcpuPid(virDomainObj *vm, unsigned int vcpuid); bool virCHDomainHasVcpuPids(virDomainObj *vm); =20 diff --git a/src/ch/ch_monitor.c b/src/ch/ch_monitor.c index d48b057814..f4e0f88817 100644 --- a/src/ch/ch_monitor.c +++ b/src/ch/ch_monitor.c @@ -41,6 +41,7 @@ VIR_LOG_INIT("ch.ch_monitor"); =20 static virClass *virCHMonitorClass; static void virCHMonitorDispose(void *obj); +static void virCHMonitorThreadInfoFree(virCHMonitor * mon); =20 static int virCHMonitorOnceInit(void) { @@ -571,6 +572,7 @@ static void virCHMonitorDispose(void *opaque) virCHMonitor *mon =3D opaque; =20 VIR_DEBUG("mon=3D%p", mon); + virCHMonitorThreadInfoFree(mon); virObjectUnref(mon->vm); } =20 @@ -736,6 +738,100 @@ virCHMonitorGet(virCHMonitor *mon, const char *endpoi= nt, virJSONValue **response return ret; } =20 +static void +virCHMonitorThreadInfoFree(virCHMonitor *mon) +{ + mon->nthreads =3D 0; + if (mon->threads) + VIR_FREE(mon->threads); +} + +static size_t +virCHMonitorRefreshThreadInfo(virCHMonitor *mon) +{ + virCHMonitorThreadInfo *info =3D NULL; + g_autofree pid_t *tids =3D NULL; + virDomainObj *vm =3D mon->vm; + size_t ntids =3D 0; + size_t i; + + + virCHMonitorThreadInfoFree(mon); + if (virProcessGetPids(vm->pid, &ntids, &tids) < 0) { + mon->threads =3D NULL; + return 0; + } + + info =3D g_new0(virCHMonitorThreadInfo, ntids); + for (i =3D 0; i < ntids; i++) { + g_autofree char *proc =3D NULL; + g_autofree char *data =3D NULL; + + proc =3D g_strdup_printf("/proc/%d/task/%d/comm", + (int)vm->pid, (int)tids[i]); + + if (virFileReadAll(proc, (1 << 16), &data) < 0) { + continue; + } + + VIR_DEBUG("VM PID: %d, TID %d, COMM: %s", + (int)vm->pid, (int)tids[i], data); + if (STRPREFIX(data, "vcpu")) { + int cpuid; + char *tmp; + + if (virStrToLong_i(data + 4, &tmp, 0, &cpuid) < 0) { + VIR_WARN("Index is not specified correctly"); + continue; + } + info[i].type =3D virCHThreadTypeVcpu; + info[i].vcpuInfo.tid =3D tids[i]; + info[i].vcpuInfo.online =3D true; + info[i].vcpuInfo.cpuid =3D cpuid; + VIR_DEBUG("vcpu%d -> tid: %d", cpuid, tids[i]); + } else if (STRPREFIX(data, "_disk") || STRPREFIX(data, "_net") || + STRPREFIX(data, "_rng")) { + /* Prefixes used by cloud-hypervisor for IO Threads are captur= ed at + * https://github.com/cloud-hypervisor/cloud-hypervisor/blob/m= ain/vmm/src/device_manager.rs */ + info[i].type =3D virCHThreadTypeIO; + info[i].ioInfo.tid =3D tids[i]; + virStrcpy(info[i].ioInfo.thrName, data, VIRCH_THREAD_NAME_LEN = - 1); + } else { + info[i].type =3D virCHThreadTypeEmulator; + info[i].emuInfo.tid =3D tids[i]; + virStrcpy(info[i].emuInfo.thrName, data, VIRCH_THREAD_NAME_LEN= - 1); + } + mon->nthreads++; + + } + mon->threads =3D info; + + return mon->nthreads; +} + +/** + * virCHMonitorGetThreadInfo: + * @mon: Pointer to the monitor + * @refresh: Refresh thread info or not + * + * Retrive thread info and store to @threads + * + * Returns count of threads on success. + */ +size_t +virCHMonitorGetThreadInfo(virCHMonitor *mon, bool refresh, + virCHMonitorThreadInfo **threads) +{ + int nthreads =3D 0; + + if (refresh) + nthreads =3D virCHMonitorRefreshThreadInfo(mon); + + *threads =3D mon->threads; + + return nthreads; +} + int virCHMonitorShutdownVMM(virCHMonitor *mon) { diff --git a/src/ch/ch_monitor.h b/src/ch/ch_monitor.h index 8ca9e17a9a..f8c3fa75e8 100644 --- a/src/ch/ch_monitor.h +++ b/src/ch/ch_monitor.h @@ -37,6 +37,50 @@ #define URL_VM_RESUME "vm.resume" #define URL_VM_INFO "vm.info" =20 +#define VIRCH_THREAD_NAME_LEN 16 + +typedef enum { + virCHThreadTypeEmulator, + virCHThreadTypeVcpu, + virCHThreadTypeIO, + virCHThreadTypeMax +} virCHThreadType; + +typedef struct _virCHMonitorCPUInfo virCHMonitorCPUInfo; + +struct _virCHMonitorCPUInfo { + int cpuid; + pid_t tid; + + bool online; +}; + +typedef struct _virCHMonitorEmuThreadInfo virCHMonitorEmuThreadInfo; + +struct _virCHMonitorEmuThreadInfo { + char thrName[VIRCH_THREAD_NAME_LEN]; + pid_t tid; +}; + +typedef struct _virCHMonitorIOThreadInfo virCHMonitorIOThreadInfo; + +struct _virCHMonitorIOThreadInfo { + char thrName[VIRCH_THREAD_NAME_LEN]; + pid_t tid; +}; + +typedef struct _virCHMonitorThreadInfo virCHMonitorThreadInfo; + +struct _virCHMonitorThreadInfo { + virCHThreadType type; + + union { + virCHMonitorCPUInfo vcpuInfo; + virCHMonitorEmuThreadInfo emuInfo; + virCHMonitorIOThreadInfo ioInfo; + }; +}; + typedef struct _virCHMonitor virCHMonitor; =20 struct _virCHMonitor { @@ -49,6 +93,9 @@ struct _virCHMonitor { pid_t pid; =20 virDomainObj *vm; + + size_t nthreads; + virCHMonitorThreadInfo *threads; }; =20 virCHMonitor *virCHMonitorNew(virDomainObj *vm, const char *socketdir); @@ -65,12 +112,9 @@ int virCHMonitorSuspendVM(virCHMonitor *mon); int virCHMonitorResumeVM(virCHMonitor *mon); int virCHMonitorGetInfo(virCHMonitor *mon, virJSONValue **info); =20 -typedef struct _virCHMonitorCPUInfo virCHMonitorCPUInfo; -struct _virCHMonitorCPUInfo { - pid_t tid; - bool online; -}; void virCHMonitorCPUInfoFree(virCHMonitorCPUInfo *cpus); int virCHMonitorGetCPUInfo(virCHMonitor *mon, virCHMonitorCPUInfo **vcpus, size_t maxvcpus); +size_t virCHMonitorGetThreadInfo(virCHMonitor *mon, bool refresh, + virCHMonitorThreadInfo **threads); diff --git a/src/ch/ch_process.c b/src/ch/ch_process.c index 65ca03cfaf..56c8d4216d 100644 --- a/src/ch/ch_process.c +++ b/src/ch/ch_process.c @@ -26,6 +26,8 @@ #include "ch_domain.h" #include "ch_monitor.h" #include "ch_process.h" +#include "domain_cgroup.h" +#include "virnuma.h" #include "viralloc.h" #include "virerror.h" #include "virjson.h" @@ -131,6 +133,254 @@ virCHProcessUpdateInfo(virDomainObj *vm) return 0; } =20 +static int +virCHProcessGetAllCpuAffinity(virBitmap **cpumapRet) +{ + *cpumapRet =3D NULL; + + if (!virHostCPUHasBitmap()) + return 0; + + if (!(*cpumapRet =3D virHostCPUGetOnlineBitmap())) + return -1; + + return 0; +} + +#if defined(WITH_SCHED_GETAFFINITY) || defined(WITH_BSD_CPU_AFFINITY) +static int +virCHProcessInitCpuAffinity(virDomainObj *vm) +{ + g_autoptr(virBitmap) cpumapToSet =3D NULL; + virDomainNumatuneMemMode mem_mode; + virCHDomainObjPrivate *priv =3D vm->privateData; + + if (!vm->pid) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("Cannot setup CPU affinity until process is start= ed")); + return -1; + } + + if (virDomainNumaGetNodeCount(vm->def->numa) <=3D 1 && + virDomainNumatuneGetMode(vm->def->numa, -1, &mem_mode) =3D=3D 0 && + mem_mode =3D=3D VIR_DOMAIN_NUMATUNE_MEM_STRICT) { + virBitmap *nodeset =3D NULL; + + if (virDomainNumatuneMaybeGetNodeset(vm->def->numa, + priv->autoNodeset, + &nodeset, -1) < 0) + return -1; + + if (virNumaNodesetToCPUset(nodeset, &cpumapToSet) < 0) + return -1; + } else if (vm->def->cputune.emulatorpin) { + if (!(cpumapToSet =3D virBitmapNewCopy(vm->def->cputune.emulatorpi= n))) + return -1; + } else { + if (virCHProcessGetAllCpuAffinity(&cpumapToSet) < 0) + return -1; + } + + if (cpumapToSet && virProcessSetAffinity(vm->pid, cpumapToSet, false) = < 0) { + return -1; + } + + return 0; +} +#else /* !defined(WITH_SCHED_GETAFFINITY) && !defined(WITH_BSD_CPU_AFFINIT= Y) */ +static int +virCHProcessInitCpuAffinity(virDomainObj *vm G_GNUC_UNUSED) +{ + return 0; +} +#endif /* !defined(WITH_SCHED_GETAFFINITY) && !defined(WITH_BSD_CPU_AFFINI= TY) */ + +/** + * virCHProcessSetupPid: + * + * This function sets resource properties (affinity, cgroups, + * scheduler) for any PID associated with a domain. It should be used + * to set up emulator PIDs as well as vCPU and I/O thread pids to + * ensure they are all handled the same way. + * + * Returns 0 on success, -1 on error. + */ +static int +virCHProcessSetupPid(virDomainObj *vm, + pid_t pid, + virCgroupThreadName nameval, + int id, + virBitmap *cpumask, + unsigned long long period, + long long quota, + virDomainThreadSchedParam *sched) +{ + virCHDomainObjPrivate *priv =3D vm->privateData; + virDomainNumatuneMemMode mem_mode; + virCgroup *cgroup =3D NULL; + virBitmap *use_cpumask =3D NULL; + virBitmap *affinity_cpumask =3D NULL; + g_autoptr(virBitmap) hostcpumap =3D NULL; + g_autofree char *mem_mask =3D NULL; + int ret =3D -1; + + if ((period || quota) && + !virCgroupHasController(priv->cgroup, VIR_CGROUP_CONTROLLER_CPU)) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", + _("cgroup cpu is required for scheduler tuning")); + goto cleanup; + } + + /* Infer which cpumask shall be used. */ + if (cpumask) { + use_cpumask =3D cpumask; + } else if (vm->def->placement_mode =3D=3D VIR_DOMAIN_CPU_PLACEMENT_MOD= E_AUTO) { + use_cpumask =3D priv->autoCpuset; + } else if (vm->def->cpumask) { + use_cpumask =3D vm->def->cpumask; + } else { + /* we can't assume cloud-hypervisor itself is running on all pCPUs, + * so we need to explicitly set the spawned instance to all pCPUs.= */ + if (virCHProcessGetAllCpuAffinity(&hostcpumap) < 0) + goto cleanup; + affinity_cpumask =3D hostcpumap; + } + + /* + * If CPU cgroup controller is not initialized here, then we need + * neither period nor quota settings. And if CPUSET controller is + * not initialized either, then there's nothing to do anyway. + */ + if (virCgroupHasController(priv->cgroup, VIR_CGROUP_CONTROLLER_CPU) || + virCgroupHasController(priv->cgroup, VIR_CGROUP_CONTROLLER_CPUSET)= ) { + + if (virDomainNumatuneGetMode(vm->def->numa, -1, &mem_mode) =3D=3D = 0 && + mem_mode =3D=3D VIR_DOMAIN_NUMATUNE_MEM_STRICT && + virDomainNumatuneMaybeFormatNodeset(vm->def->numa, + priv->autoNodeset, + &mem_mask, -1) < 0) + goto cleanup; + + if (virCgroupNewThread(priv->cgroup, nameval, id, true, &cgroup) <= 0) + goto cleanup; + + if (virCgroupHasController(priv->cgroup, VIR_CGROUP_CONTROLLER_CPU= SET)) { + if (use_cpumask && + virSetupCgroupCpusetCpus(cgroup, use_cpumask) < 0) + goto cleanup; + + if (mem_mask && virCgroupSetCpusetMems(cgroup, mem_mask) < 0) + goto cleanup; + + } + + if ((period || quota) && + virSetupCgroupVcpuBW(cgroup, period, quota) < 0) + goto cleanup; + + /* Move the thread to the sub dir */ + VIR_INFO("Adding pid %d to cgroup", pid); + if (virCgroupAddThread(cgroup, pid) < 0) + goto cleanup; + + } + + if (!affinity_cpumask) + affinity_cpumask =3D use_cpumask; + + /* Setup legacy affinity. */ + if (affinity_cpumask + && virProcessSetAffinity(pid, affinity_cpumask, false) < 0) + goto cleanup; + + /* Set scheduler type and priority, but not for the main thread. */ + if (sched && + nameval !=3D VIR_CGROUP_THREAD_EMULATOR && + virProcessSetScheduler(pid, sched->policy, sched->priority) < 0) + goto cleanup; + + ret =3D 0; + cleanup: + if (cgroup) { + if (ret < 0) + virCgroupRemove(cgroup); + virCgroupFree(cgroup); + } + + return ret; +} + +/** + * virCHProcessSetupVcpu: + * @vm: domain object + * @vcpuid: id of VCPU to set defaults + * + * This function sets resource properties (cgroups, affinity, scheduler) f= or a + * vCPU. This function expects that the vCPU is online and the vCPU pids w= ere + * correctly detected at the point when it's called. + * + * Returns 0 on success, -1 on error. + */ +int +virCHProcessSetupVcpu(virDomainObj *vm, unsigned int vcpuid) +{ + pid_t vcpupid =3D virCHDomainGetVcpuPid(vm, vcpuid); + virDomainVcpuDef *vcpu =3D virDomainDefGetVcpu(vm->def, vcpuid); + + return virCHProcessSetupPid(vm, vcpupid, VIR_CGROUP_THREAD_VCPU, + vcpuid, vcpu->cpumask, + vm->def->cputune.period, + vm->def->cputune.quota, &vcpu->sched); +} + +static int +virCHProcessSetupVcpus(virDomainObj *vm) +{ + virDomainVcpuDef *vcpu; + unsigned int maxvcpus =3D virDomainDefGetVcpusMax(vm->def); + size_t i; + + if ((vm->def->cputune.period || vm->def->cputune.quota) && + !virCgroupHasController(((virCHDomainObjPrivate *) vm->privateData= )-> + cgroup, VIR_CGROUP_CONTROLLER_CPU)) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", + _("cgroup cpu is required for scheduler tuning")); + return -1; + } + + if (!virCHDomainHasVcpuPids(vm)) { + /* If any CPU has custom affinity that differs from the + * VM default affinity, we must reject it */ + for (i =3D 0; i < maxvcpus; i++) { + vcpu =3D virDomainDefGetVcpu(vm->def, i); + + if (!vcpu->online) + continue; + + if (vcpu->cpumask && + !virBitmapEqual(vm->def->cpumask, vcpu->cpumask)) { + virReportError(VIR_ERR_OPERATION_INVALID, "%s", + _("cpu affinity is not supported")); + return -1; + } + } + + return 0; + } + + for (i =3D 0; i < maxvcpus; i++) { + vcpu =3D virDomainDefGetVcpu(vm->def, i); + + if (!vcpu->online) + continue; + + if (virCHProcessSetupVcpu(vm, i) < 0) + return -1; + } + + return 0; +} + /** * virCHProcessStart: * @driver: pointer to driver structure @@ -141,12 +391,13 @@ virCHProcessUpdateInfo(virDomainObj *vm) * * Returns 0 on success or -1 in case of error */ -int virCHProcessStart(virCHDriver *driver, - virDomainObj *vm, - virDomainRunningReason reason) +int +virCHProcessStart(virCHDriver *driver, + virDomainObj *vm, virDomainRunningReason reason) { int ret =3D -1; virCHDomainObjPrivate *priv =3D vm->privateData; + g_autoptr(virCHDriverConfig) cfg =3D virCHDriverGetConfig(priv->driver= ); g_autofree int *nicindexes =3D NULL; size_t nnicindexes =3D 0; =20 @@ -166,18 +417,39 @@ int virCHProcessStart(virCHDriver *driver, } } =20 + vm->pid =3D priv->monitor->pid; + vm->def->id =3D vm->pid; + priv->machineName =3D virCHDomainGetMachineName(vm); + + if (virSetupCgroup("ch", vm, + nnicindexes, nicindexes, + priv->cgroup, + cfg->cgroupControllers, + 0, /*maxThreadsPerProc*/ + priv->driver->privileged, + priv->machineName) < 0) + goto cleanup; + + if (virCHProcessInitCpuAffinity(vm) < 0) + goto cleanup; + if (virCHMonitorBootVM(priv->monitor) < 0) { virReportError(VIR_ERR_INTERNAL_ERROR, "%s", _("failed to boot guest VM")); goto cleanup; } =20 - priv->machineName =3D virCHDomainGetMachineName(vm); - vm->pid =3D priv->monitor->pid; - vm->def->id =3D vm->pid; + virCHDomainRefreshThreadInfo(vm); =20 - virCHProcessUpdateInfo(vm); + VIR_DEBUG("Setting global CPU cgroup (if required)"); + if (virSetupGlobalCpuCgroup(vm, priv->cgroup, priv->autoNodeset) < 0) + goto cleanup; + + VIR_DEBUG("Setting vCPU tuning/settings"); + if (virCHProcessSetupVcpus(vm) < 0) + goto cleanup; =20 + virCHProcessUpdateInfo(vm); virDomainObjSetState(vm, VIR_DOMAIN_RUNNING, reason); =20 return 0; @@ -189,10 +461,12 @@ int virCHProcessStart(virCHDriver *driver, return ret; } =20 -int virCHProcessStop(virCHDriver *driver G_GNUC_UNUSED, - virDomainObj *vm, - virDomainShutoffReason reason) +int +virCHProcessStop(virCHDriver *driver G_GNUC_UNUSED, + virDomainObj *vm, virDomainShutoffReason reason) { + int ret; + int retries =3D 0; virCHDomainObjPrivate *priv =3D vm->privateData; =20 VIR_DEBUG("Stopping VM name=3D%s pid=3D%d reason=3D%d", @@ -203,6 +477,16 @@ int virCHProcessStop(virCHDriver *driver G_GNUC_UNUSED, priv->monitor =3D NULL; } =20 + retry: + if ((ret =3D virRemoveCgroup(vm, priv->cgroup, priv->machineName))= < 0) { + if (ret =3D=3D -EBUSY && (retries++ < 5)) { + g_usleep(200*1000); + goto retry; + } + VIR_WARN("Failed to remove cgroup for %s", + vm->def->name); + } + vm->pid =3D -1; vm->def->id =3D -1; =20 diff --git a/src/ch/ch_process.h b/src/ch/ch_process.h index abc4915979..800e3f4e23 100644 --- a/src/ch/ch_process.h +++ b/src/ch/ch_process.h @@ -29,3 +29,6 @@ int virCHProcessStart(virCHDriver *driver, int virCHProcessStop(virCHDriver *driver, virDomainObj *vm, virDomainShutoffReason reason); + +int virCHProcessSetupVcpu(virDomainObj *vm, + unsigned int vcpuid); --=20 2.27.0 From nobody Sun May 5 13:15:08 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) client-ip=170.10.129.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail(p=none dis=none) header.from=linux.microsoft.com Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mx.zohomail.com with SMTPS id 1639168534606896.9274984378546; Fri, 10 Dec 2021 12:35:34 -0800 (PST) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-349-DRw4gcZjMoGg6JkhRhDJRQ-1; Fri, 10 Dec 2021 15:35:31 -0500 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id E3CEB192FDB0; Fri, 10 Dec 2021 20:35:22 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id C380E77713; Fri, 10 Dec 2021 20:35:22 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 902171806D2F; Fri, 10 Dec 2021 20:35:22 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 1BAKZ1Ss008439 for ; Fri, 10 Dec 2021 15:35:01 -0500 Received: by smtp.corp.redhat.com (Postfix) id 9093353A9; Fri, 10 Dec 2021 20:35:01 +0000 (UTC) Received: from mimecast-mx02.redhat.com (mimecast10.extmail.prod.ext.rdu2.redhat.com [10.11.55.26]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 8AC0953AC for ; Fri, 10 Dec 2021 20:34:58 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-1.mimecast.com [207.211.31.81]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id C82A21C8C279 for ; Fri, 10 Dec 2021 20:34:58 +0000 (UTC) Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by relay.mimecast.com with ESMTP id us-mta-263-kknJLRVMOIGIkeE_eXkmDA-1; Fri, 10 Dec 2021 15:34:57 -0500 Received: from prapal-ch2.oiwy50ateaxezb1sqsoezlib2f.xx.internal.cloudapp.net (unknown [20.80.162.67]) by linux.microsoft.com (Postfix) with ESMTPSA id 0178C20B718C; Fri, 10 Dec 2021 12:34:55 -0800 (PST) X-MC-Unique: DRw4gcZjMoGg6JkhRhDJRQ-1 X-MC-Unique: kknJLRVMOIGIkeE_eXkmDA-1 DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 0178C20B718C From: Praveen K Paladugu To: libvir-list@redhat.com Subject: [libvirt PATCH v3 09/13] ch_driver, ch_domain: vcpupin callback in ch driver Date: Fri, 10 Dec 2021 20:34:44 +0000 Message-Id: <20211210203448.1579606-10-prapal@linux.microsoft.com> In-Reply-To: <20211210203448.1579606-1-prapal@linux.microsoft.com> References: <20211210203448.1579606-1-prapal@linux.microsoft.com> MIME-Version: 1.0 X-Mimecast-Impersonation-Protect: Policy=CLT - Impersonation Protection Definition; Similar Internal Domain=false; Similar Monitored External Domain=false; Custom External Domain=false; Mimecast External Domain=false; Newly Observed Domain=false; Internal User Name=false; Custom Display Name List=false; Reply-to Address Mismatch=false; Targeted Threat Dictionary=false; Mimecast Threat Dictionary=false; Custom Threat Dictionary=false X-Scanned-By: MIMEDefang 2.79 on 10.11.54.5 X-MIME-Autoconverted: from quoted-printable to 8bit by lists01.pubmisc.prod.ext.phx2.redhat.com id 1BAKZ1Ss008439 X-loop: libvir-list@redhat.com Cc: william.douglas@intel.com X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZM-MESSAGEID: 1639168536477100005 Content-Type: text/plain; charset="utf-8" From: Vineeth Pillai Signed-off-by: Vineeth Pillai Signed-off-by: Praveen K Paladugu --- src/ch/ch_domain.c | 30 +++++++++ src/ch/ch_domain.h | 1 + src/ch/ch_driver.c | 145 ++++++++++++++++++++++++++++++++++++++++++++ src/ch/ch_monitor.c | 2 +- 4 files changed, 177 insertions(+), 1 deletion(-) diff --git a/src/ch/ch_domain.c b/src/ch/ch_domain.c index 42cb4e7dcf..524aa330e1 100644 --- a/src/ch/ch_domain.c +++ b/src/ch/ch_domain.c @@ -20,6 +20,7 @@ =20 #include =20 +#include "datatypes.h" #include "ch_domain.h" #include "domain_driver.h" #include "viralloc.h" @@ -420,3 +421,32 @@ virCHDomainGetMachineName(virDomainObj *vm) =20 return ret; } + +/** + * virCHDomainObjFromDomain: + * @domain: Domain pointer that has to be looked up + * + * This function looks up @domain and returns the appropriate virDomainObj= Ptr + * that has to be released by calling virDomainObjEndAPI(). + * + * Returns the domain object with incremented reference counter which is l= ocked + * on success, NULL otherwise. + */ +virDomainObj * +virCHDomainObjFromDomain(virDomainPtr domain) +{ + virDomainObj *vm; + virCHDriver *driver =3D domain->conn->privateData; + char uuidstr[VIR_UUID_STRING_BUFLEN]; + + vm =3D virDomainObjListFindByUUID(driver->domains, domain->uuid); + if (!vm) { + virUUIDFormat(domain->uuid, uuidstr); + virReportError(VIR_ERR_NO_DOMAIN, + _("no domain with matching uuid '%s' (%s)"), + uuidstr, domain->name); + return NULL; + } + + return vm; +} diff --git a/src/ch/ch_domain.h b/src/ch/ch_domain.h index 7cffe2b4fb..b4a245df54 100644 --- a/src/ch/ch_domain.h +++ b/src/ch/ch_domain.h @@ -95,3 +95,4 @@ pid_t virCHDomainGetVcpuPid(virDomainObj *vm, unsigned in= t vcpuid); bool virCHDomainHasVcpuPids(virDomainObj *vm); =20 char *virCHDomainGetMachineName(virDomainObj *vm); +virDomainObj *virCHDomainObjFromDomain(virDomain *domain); diff --git a/src/ch/ch_driver.c b/src/ch/ch_driver.c index 97fbd76959..0cc08b6c20 100644 --- a/src/ch/ch_driver.c +++ b/src/ch/ch_driver.c @@ -25,6 +25,7 @@ #include "ch_driver.h" #include "ch_monitor.h" #include "ch_process.h" +#include "domain_cgroup.h" #include "datatypes.h" #include "driver.h" #include "viraccessapicheck.h" @@ -1123,6 +1124,148 @@ chDomainGetVcpus(virDomainPtr dom, return ret; } =20 +static int +chDomainPinVcpuLive(virDomainObj *vm, + virDomainDef *def, + int vcpu, + virCHDriver *driver, + virCHDriverConfig *cfg, + virBitmap *cpumap) +{ + g_autoptr(virBitmap) tmpmap =3D NULL; + g_autoptr(virCgroup) cgroup_vcpu =3D NULL; + virDomainVcpuDef *vcpuinfo; + virCHDomainObjPrivate *priv =3D vm->privateData; + + g_autofree char *str =3D NULL; + + if (!virCHDomainHasVcpuPids(vm)) { + virReportError(VIR_ERR_OPERATION_INVALID, + "%s", _("cpu affinity is not supported")); + return -1; + } + + if (!(vcpuinfo =3D virDomainDefGetVcpu(def, vcpu))) { + virReportError(VIR_ERR_INVALID_ARG, + _("vcpu %d is out of range of live cpu count %d"), + vcpu, virDomainDefGetVcpusMax(def)); + return -1; + } + + if (!(tmpmap =3D virBitmapNewCopy(cpumap))) + return -1; + + if (!(str =3D virBitmapFormat(cpumap))) + return -1; + + if (vcpuinfo->online) { + /* Configure the corresponding cpuset cgroup before set affinity. = */ + if (virCgroupHasController(priv->cgroup, VIR_CGROUP_CONTROLLER_CPU= SET)) { + if (virCgroupNewThread(priv->cgroup, VIR_CGROUP_THREAD_VCPU, v= cpu, + false, &cgroup_vcpu) < 0) + return -1; + if (virSetupCgroupCpusetCpus(cgroup_vcpu, cpumap) < 0) + return -1; + } + + if (virProcessSetAffinity(virCHDomainGetVcpuPid(vm, vcpu), cpumap,= false) < 0) + return -1; + } + + virBitmapFree(vcpuinfo->cpumask); + vcpuinfo->cpumask =3D tmpmap; + tmpmap =3D NULL; + + if (virDomainObjSave(vm, driver->xmlopt, cfg->stateDir) < 0) + return -1; + + return 0; +} + + +static int +chDomainPinVcpuFlags(virDomainPtr dom, + unsigned int vcpu, + unsigned char *cpumap, + int maplen, + unsigned int flags) +{ + virCHDriver *driver =3D dom->conn->privateData; + virDomainObj *vm; + virDomainDef *def; + virDomainDef *persistentDef; + int ret =3D -1; + virBitmap *pcpumap =3D NULL; + virDomainVcpuDef *vcpuinfo =3D NULL; + g_autoptr(virCHDriverConfig) cfg =3D NULL; + + virCheckFlags(VIR_DOMAIN_AFFECT_LIVE | + VIR_DOMAIN_AFFECT_CONFIG, -1); + + cfg =3D virCHDriverGetConfig(driver); + + if (!(vm =3D virCHDomainObjFromDomain(dom))) + goto cleanup; + + if (virDomainPinVcpuFlagsEnsureACL(dom->conn, vm->def, flags) < 0) + goto cleanup; + + if (virCHDomainObjBeginJob(vm, CH_JOB_MODIFY) < 0) + goto cleanup; + + if (virDomainObjGetDefs(vm, flags, &def, &persistentDef) < 0) + goto endjob; + + if (persistentDef && + !(vcpuinfo =3D virDomainDefGetVcpu(persistentDef, vcpu))) { + virReportError(VIR_ERR_INVALID_ARG, + _("vcpu %d is out of range of persistent cpu count = %d"), + vcpu, virDomainDefGetVcpus(persistentDef)); + goto endjob; + } + + if (!(pcpumap =3D virBitmapNewData(cpumap, maplen))) + goto endjob; + + if (virBitmapIsAllClear(pcpumap)) { + virReportError(VIR_ERR_INVALID_ARG, "%s", + _("Empty cpu list for pinning")); + goto endjob; + } + + if (def && + chDomainPinVcpuLive(vm, def, vcpu, driver, cfg, pcpumap) < 0) + goto endjob; + + if (persistentDef) { + virBitmapFree(vcpuinfo->cpumask); + vcpuinfo->cpumask =3D pcpumap; + pcpumap =3D NULL; + + goto endjob; + } + + ret =3D 0; + + endjob: + virCHDomainObjEndJob(vm); + + cleanup: + virDomainObjEndAPI(&vm); + virBitmapFree(pcpumap); + return ret; +} + +static int +chDomainPinVcpu(virDomainPtr dom, + unsigned int vcpu, + unsigned char *cpumap, + int maplen) +{ + return chDomainPinVcpuFlags(dom, vcpu, cpumap, maplen, + VIR_DOMAIN_AFFECT_LIVE); +} + /* Function Tables */ static virHypervisorDriver chHypervisorDriver =3D { .name =3D "CH", @@ -1163,6 +1306,8 @@ static virHypervisorDriver chHypervisorDriver =3D { .domainGetVcpusFlags =3D chDomainGetVcpusFlags, /* 8.0.0 */ .domainGetMaxVcpus =3D chDomainGetMaxVcpus, /* 8.0.0 */ .domainGetVcpuPinInfo =3D chDomainGetVcpuPinInfo, /* 8.0.0 */ + .domainPinVcpu =3D chDomainPinVcpu, /* 8.0.0 */ + .domainPinVcpuFlags =3D chDomainPinVcpuFlags, /* 8.0.0 */ .nodeGetCPUMap =3D chNodeGetCPUMap, /* 8.0.0 */ }; =20 diff --git a/src/ch/ch_monitor.c b/src/ch/ch_monitor.c index f4e0f88817..4de86f3c03 100644 --- a/src/ch/ch_monitor.c +++ b/src/ch/ch_monitor.c @@ -41,7 +41,7 @@ VIR_LOG_INIT("ch.ch_monitor"); =20 static virClass *virCHMonitorClass; static void virCHMonitorDispose(void *obj); -static void virCHMonitorThreadInfoFree(virCHMonitor * mon); +static void virCHMonitorThreadInfoFree(virCHMonitor *mon); =20 static int virCHMonitorOnceInit(void) { --=20 2.27.0 From nobody Sun May 5 13:15:08 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) client-ip=170.10.133.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail(p=none dis=none) header.from=linux.microsoft.com Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mx.zohomail.com with SMTPS id 1639168531632722.7339572692887; Fri, 10 Dec 2021 12:35:31 -0800 (PST) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-16-LaR1vX4LNz6Tp1UVbdNXoQ-1; Fri, 10 Dec 2021 15:35:28 -0500 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 1F7EE6409A; Fri, 10 Dec 2021 20:35:23 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 01E8767842; Fri, 10 Dec 2021 20:35:23 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id C175F4A7CB; Fri, 10 Dec 2021 20:35:22 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 1BAKYwOv008366 for ; Fri, 10 Dec 2021 15:34:58 -0500 Received: by smtp.corp.redhat.com (Postfix) id A89021410F39; Fri, 10 Dec 2021 20:34:58 +0000 (UTC) Received: from mimecast-mx02.redhat.com (mimecast10.extmail.prod.ext.rdu2.redhat.com [10.11.55.26]) by smtp.corp.redhat.com (Postfix) with ESMTPS id A3FD21410F35 for ; Fri, 10 Dec 2021 20:34:58 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [207.211.31.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 89E0F1C8C279 for ; Fri, 10 Dec 2021 20:34:58 +0000 (UTC) Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by relay.mimecast.com with ESMTP id us-mta-469-0dg2YSaLPW-X7WXYIdaARg-1; Fri, 10 Dec 2021 15:34:56 -0500 Received: from prapal-ch2.oiwy50ateaxezb1sqsoezlib2f.xx.internal.cloudapp.net (unknown [20.80.162.67]) by linux.microsoft.com (Postfix) with ESMTPSA id 18D8F20B718D; Fri, 10 Dec 2021 12:34:55 -0800 (PST) X-MC-Unique: LaR1vX4LNz6Tp1UVbdNXoQ-1 X-MC-Unique: 0dg2YSaLPW-X7WXYIdaARg-1 DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 18D8F20B718D From: Praveen K Paladugu To: libvir-list@redhat.com Subject: [libvirt PATCH v3 10/13] ch_driver: enable typed param string for numatune Date: Fri, 10 Dec 2021 20:34:45 +0000 Message-Id: <20211210203448.1579606-11-prapal@linux.microsoft.com> In-Reply-To: <20211210203448.1579606-1-prapal@linux.microsoft.com> References: <20211210203448.1579606-1-prapal@linux.microsoft.com> MIME-Version: 1.0 X-Mimecast-Impersonation-Protect: Policy=CLT - Impersonation Protection Definition; Similar Internal Domain=false; Similar Monitored External Domain=false; Custom External Domain=false; Mimecast External Domain=false; Newly Observed Domain=false; Internal User Name=false; Custom Display Name List=false; Reply-to Address Mismatch=false; Targeted Threat Dictionary=false; Mimecast Threat Dictionary=false; Custom Threat Dictionary=false X-Scanned-By: MIMEDefang 2.85 on 10.11.54.7 X-MIME-Autoconverted: from quoted-printable to 8bit by lists01.pubmisc.prod.ext.phx2.redhat.com id 1BAKYwOv008366 X-loop: libvir-list@redhat.com Cc: william.douglas@intel.com X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZM-MESSAGEID: 1639168534028100001 Content-Type: text/plain; charset="utf-8" From: Vineeth Pillai Enable support of VIR_DRV_FEATURE_TYPED_PARAM_STRING to enable numatune Signed-off-by: Vineeth Pillai Signed-off-by: Praveen K Paladugu --- src/ch/ch_driver.c | 31 +++++++++++++++++++++++++++++++ 1 file changed, 31 insertions(+) diff --git a/src/ch/ch_driver.c b/src/ch/ch_driver.c index 0cc08b6c20..03db6b2bc8 100644 --- a/src/ch/ch_driver.c +++ b/src/ch/ch_driver.c @@ -944,6 +944,36 @@ static int chStateInitialize(bool privileged, return ret; } =20 +/* Which features are supported by this driver? */ +static int +chConnectSupportsFeature(virConnectPtr conn, int feature) +{ + if (virConnectSupportsFeatureEnsureACL(conn) < 0) + return -1; + + switch ((virDrvFeature) feature) { + case VIR_DRV_FEATURE_TYPED_PARAM_STRING: + return 1; + case VIR_DRV_FEATURE_MIGRATION_V2: + case VIR_DRV_FEATURE_MIGRATION_V3: + case VIR_DRV_FEATURE_MIGRATION_P2P: + case VIR_DRV_FEATURE_MIGRATE_CHANGE_PROTECTION: + case VIR_DRV_FEATURE_FD_PASSING: + case VIR_DRV_FEATURE_XML_MIGRATABLE: + case VIR_DRV_FEATURE_MIGRATION_OFFLINE: + case VIR_DRV_FEATURE_MIGRATION_PARAMS: + case VIR_DRV_FEATURE_MIGRATION_DIRECT: + case VIR_DRV_FEATURE_MIGRATION_V1: + case VIR_DRV_FEATURE_PROGRAM_KEEPALIVE: + case VIR_DRV_FEATURE_REMOTE: + case VIR_DRV_FEATURE_REMOTE_CLOSE_CALLBACK: + case VIR_DRV_FEATURE_REMOTE_EVENT_CALLBACK: + case VIR_DRV_FEATURE_NETWORK_UPDATE_HAS_CORRECT_ORDER: + default: + return 0; + } +} + static int chDomainGetVcpusFlags(virDomainPtr dom, unsigned int flags) { @@ -1279,6 +1309,7 @@ static virHypervisorDriver chHypervisorDriver =3D { .connectListAllDomains =3D chConnectListAllDomains, /* 7.5.0 */ .connectListDomains =3D chConnectListDomains, /* 7.5.0 */ .connectGetCapabilities =3D chConnectGetCapabilities, /* 7.5.0 */ + .connectSupportsFeature =3D chConnectSupportsFeature, /* 8.0.0 */ .domainCreateXML =3D chDomainCreateXML, /* 7.5.0 */ .domainCreate =3D chDomainCreate, /* 7.5.0 */ .domainCreateWithFlags =3D chDomainCreateWithFlags, /* 7.5.0 */ --=20 2.27.0 From nobody Sun May 5 13:15:08 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) client-ip=170.10.133.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail(p=none dis=none) header.from=linux.microsoft.com Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mx.zohomail.com with SMTPS id 1639168538091824.0451113140828; Fri, 10 Dec 2021 12:35:38 -0800 (PST) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-540-1gMhQG_uPeu8Pl2kn7rQHA-1; Fri, 10 Dec 2021 15:35:33 -0500 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id D39D7192FDA0; Fri, 10 Dec 2021 20:35:28 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id B32AA6107E; Fri, 10 Dec 2021 20:35:28 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 86E731802FE9; Fri, 10 Dec 2021 20:35:28 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 1BAKZ1un008438 for ; Fri, 10 Dec 2021 15:35:01 -0500 Received: by smtp.corp.redhat.com (Postfix) id 8F3A453A8; Fri, 10 Dec 2021 20:35:01 +0000 (UTC) Received: from mimecast-mx02.redhat.com (mimecast01.extmail.prod.ext.rdu2.redhat.com [10.11.55.17]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 8A011518D for ; Fri, 10 Dec 2021 20:34:58 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-1.mimecast.com [205.139.110.61]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 8D34485A5AA for ; Fri, 10 Dec 2021 20:34:58 +0000 (UTC) Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by relay.mimecast.com with ESMTP id us-mta-232-5gwOSsWNPTyrPDh8XZz9qQ-1; Fri, 10 Dec 2021 15:34:56 -0500 Received: from prapal-ch2.oiwy50ateaxezb1sqsoezlib2f.xx.internal.cloudapp.net (unknown [20.80.162.67]) by linux.microsoft.com (Postfix) with ESMTPSA id 2F06420B7193; Fri, 10 Dec 2021 12:34:55 -0800 (PST) X-MC-Unique: 1gMhQG_uPeu8Pl2kn7rQHA-1 X-MC-Unique: 5gwOSsWNPTyrPDh8XZz9qQ-1 DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 2F06420B7193 From: Praveen K Paladugu To: libvir-list@redhat.com Subject: [libvirt PATCH v3 11/13] ch_driver: add numatune callbacks for CH driver Date: Fri, 10 Dec 2021 20:34:46 +0000 Message-Id: <20211210203448.1579606-12-prapal@linux.microsoft.com> In-Reply-To: <20211210203448.1579606-1-prapal@linux.microsoft.com> References: <20211210203448.1579606-1-prapal@linux.microsoft.com> MIME-Version: 1.0 X-Mimecast-Impersonation-Protect: Policy=CLT - Impersonation Protection Definition; Similar Internal Domain=false; Similar Monitored External Domain=false; Custom External Domain=false; Mimecast External Domain=false; Newly Observed Domain=false; Internal User Name=false; Custom Display Name List=false; Reply-to Address Mismatch=false; Targeted Threat Dictionary=false; Mimecast Threat Dictionary=false; Custom Threat Dictionary=false X-Scanned-By: MIMEDefang 2.79 on 10.11.54.5 X-MIME-Autoconverted: from quoted-printable to 8bit by lists01.pubmisc.prod.ext.phx2.redhat.com id 1BAKZ1un008438 X-loop: libvir-list@redhat.com Cc: william.douglas@intel.com X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZM-MESSAGEID: 1639168538943100005 Content-Type: text/plain; charset="utf-8" From: Vineeth Pillai Signed-off-by: Vineeth Pillai Signed-off-by: Praveen K Paladugu --- src/ch/ch_driver.c | 273 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 273 insertions(+) diff --git a/src/ch/ch_driver.c b/src/ch/ch_driver.c index 03db6b2bc8..d7008ef011 100644 --- a/src/ch/ch_driver.c +++ b/src/ch/ch_driver.c @@ -43,6 +43,7 @@ #include "viruri.h" #include "virutil.h" #include "viruuid.h" +#include "virnuma.h" =20 #define VIR_FROM_THIS VIR_FROM_CH =20 @@ -1296,6 +1297,276 @@ chDomainPinVcpu(virDomainPtr dom, VIR_DOMAIN_AFFECT_LIVE); } =20 +#define CH_NB_NUMA_PARAM 2 + +static int +chDomainGetNumaParameters(virDomainPtr dom, + virTypedParameterPtr params, + int *nparams, + unsigned int flags) +{ + size_t i; + virDomainObj *vm =3D NULL; + virDomainNumatuneMemMode tmpmode =3D VIR_DOMAIN_NUMATUNE_MEM_STRICT; + virCHDomainObjPrivate *priv; + g_autofree char *nodeset =3D NULL; + int ret =3D -1; + virDomainDef *def =3D NULL; + bool live =3D false; + virBitmap *autoNodeset =3D NULL; + + virCheckFlags(VIR_DOMAIN_AFFECT_LIVE | + VIR_DOMAIN_AFFECT_CONFIG | + VIR_TYPED_PARAM_STRING_OKAY, -1); + + if (!(vm =3D virCHDomainObjFromDomain(dom))) + return -1; + priv =3D vm->privateData; + + if (virDomainGetNumaParametersEnsureACL(dom->conn, vm->def) < 0) + goto cleanup; + + if (!(def =3D virDomainObjGetOneDefState(vm, flags, &live))) + goto cleanup; + + if (live) + autoNodeset =3D priv->autoNodeset; + + if ((*nparams) =3D=3D 0) { + *nparams =3D CH_NB_NUMA_PARAM; + ret =3D 0; + goto cleanup; + } + + for (i =3D 0; i < CH_NB_NUMA_PARAM && i < *nparams; i++) { + virMemoryParameterPtr param =3D ¶ms[i]; + + switch (i) { + case 0: /* fill numa mode here */ + ignore_value(virDomainNumatuneGetMode(def->numa, -1, &tmpmode)= ); + + if (virTypedParameterAssign(param, VIR_DOMAIN_NUMA_MODE, + VIR_TYPED_PARAM_INT, tmpmode) < 0) + goto cleanup; + + break; + + case 1: /* fill numa nodeset here */ + nodeset =3D virDomainNumatuneFormatNodeset(def->numa, autoNode= set, -1); + + if (!nodeset || + virTypedParameterAssign(param, VIR_DOMAIN_NUMA_NODESET, + VIR_TYPED_PARAM_STRING, nodeset) <= 0) + goto cleanup; + + nodeset =3D NULL; + break; + + /* coverity[dead_error_begin] */ + default: + break; + /* should not hit here */ + } + } + + if (*nparams > CH_NB_NUMA_PARAM) + *nparams =3D CH_NB_NUMA_PARAM; + ret =3D 0; + + cleanup: + virDomainObjEndAPI(&vm); + return ret; +} + +static int +chDomainSetNumaParamsLive(virDomainObj *vm, + virBitmap *nodeset) +{ + g_autoptr(virCgroup) cgroup_temp =3D NULL; + virCHDomainObjPrivate *priv =3D vm->privateData; + g_autofree char *nodeset_str =3D NULL; + virDomainNumatuneMemMode mode; + size_t i =3D 0; + + + if (virDomainNumatuneGetMode(vm->def->numa, -1, &mode) =3D=3D 0 && + mode !=3D VIR_DOMAIN_NUMATUNE_MEM_STRICT) { + virReportError(VIR_ERR_OPERATION_INVALID, "%s", + _("change of nodeset for running domain requires st= rict numa mode")); + return -1; + } + + if (!virNumaNodesetIsAvailable(nodeset)) + return -1; + + /* Ensure the cpuset string is formatted before passing to cgroup */ + if (!(nodeset_str =3D virBitmapFormat(nodeset))) + return -1; + + if (virCgroupNewThread(priv->cgroup, VIR_CGROUP_THREAD_EMULATOR, 0, + false, &cgroup_temp) < 0 || + virCgroupSetCpusetMems(cgroup_temp, nodeset_str) < 0) + return -1; + + + for (i =3D 0; i < virDomainDefGetVcpusMax(vm->def); i++) { + virDomainVcpuDef *vcpu =3D virDomainDefGetVcpu(vm->def, i); + + if (!vcpu->online) + continue; + + if (virCgroupNewThread(priv->cgroup, VIR_CGROUP_THREAD_VCPU, i, + false, &cgroup_temp) < 0 || + virCgroupSetCpusetMems(cgroup_temp, nodeset_str) < 0) + return -1; + + return 0; + } + + for (i =3D 0; i < vm->def->niothreadids; i++) { + if (virCgroupNewThread(priv->cgroup, VIR_CGROUP_THREAD_IOTHREAD, + vm->def->iothreadids[i]->iothread_id, + false, &cgroup_temp) < 0 || + virCgroupSetCpusetMems(cgroup_temp, nodeset_str) < 0) + return -1; + } + + /* set nodeset for root cgroup */ + if (virCgroupSetCpusetMems(priv->cgroup, nodeset_str) < 0) + return -1; + + return 0; +} + +static int +chDomainSetNumaParameters(virDomainPtr dom, + virTypedParameterPtr params, + int nparams, + unsigned int flags) +{ + virCHDriver *driver =3D dom->conn->privateData; + size_t i; + virDomainDef *def; + virDomainDef *persistentDef; + virDomainObj *vm =3D NULL; + int ret =3D -1; + g_autoptr(virCHDriverConfig) cfg =3D NULL; + virCHDomainObjPrivate *priv; + virBitmap *nodeset =3D NULL; + virDomainNumatuneMemMode config_mode; + int mode =3D -1; + + virCheckFlags(VIR_DOMAIN_AFFECT_LIVE | + VIR_DOMAIN_AFFECT_CONFIG, -1); + + if (virTypedParamsValidate(params, nparams, + VIR_DOMAIN_NUMA_MODE, + VIR_TYPED_PARAM_INT, + VIR_DOMAIN_NUMA_NODESET, + VIR_TYPED_PARAM_STRING, + NULL) < 0) + return -1; + + if (!(vm =3D virCHDomainObjFromDomain(dom))) + return -1; + + priv =3D vm->privateData; + cfg =3D virCHDriverGetConfig(driver); + + if (virDomainSetNumaParametersEnsureACL(dom->conn, vm->def, flags) < 0) + goto cleanup; + + for (i =3D 0; i < nparams; i++) { + virTypedParameterPtr param =3D ¶ms[i]; + + if (STREQ(param->field, VIR_DOMAIN_NUMA_MODE)) { + mode =3D param->value.i; + + if (mode < 0 || mode >=3D VIR_DOMAIN_NUMATUNE_MEM_LAST) { + virReportError(VIR_ERR_INVALID_ARG, + _("unsupported numatune mode: '%d'"), mode); + goto cleanup; + } + + } else if (STREQ(param->field, VIR_DOMAIN_NUMA_NODESET)) { + if (virBitmapParse(param->value.s, &nodeset, + VIR_DOMAIN_CPUMASK_LEN) < 0) + goto cleanup; + + if (virBitmapIsAllClear(nodeset)) { + virReportError(VIR_ERR_OPERATION_INVALID, + _("Invalid nodeset of 'numatune': %s"), + param->value.s); + goto cleanup; + } + } + } + + if (virCHDomainObjBeginJob(vm, CH_JOB_MODIFY) < 0) + goto cleanup; + + if (virDomainObjGetDefs(vm, flags, &def, &persistentDef) < 0) + goto endjob; + + if (def) { + if (!driver->privileged) { + virReportError(VIR_ERR_OPERATION_UNSUPPORTED, "%s", + _("NUMA tuning is not available in session mode= ")); + goto endjob; + } + + if (!virCgroupHasController(priv->cgroup, VIR_CGROUP_CONTROLLER_CP= USET)) { + virReportError(VIR_ERR_OPERATION_INVALID, "%s", + _("cgroup cpuset controller is not mounted")); + goto endjob; + } + + if (mode !=3D -1 && + virDomainNumatuneGetMode(def->numa, -1, &config_mode) =3D=3D 0= && + config_mode !=3D mode) { + virReportError(VIR_ERR_OPERATION_INVALID, "%s", + _("can't change numatune mode for running domai= n")); + goto endjob; + } + + if (nodeset && + chDomainSetNumaParamsLive(vm, nodeset) < 0) + goto endjob; + + if (virDomainNumatuneSet(def->numa, + def->placement_mode =3D=3D + VIR_DOMAIN_CPU_PLACEMENT_MODE_STATIC, + -1, mode, nodeset) < 0) + goto endjob; + + if (virDomainObjSave(vm, driver->xmlopt, cfg->stateDir) < 0) + goto endjob; + } + + /* + if (persistentDef) { + if (virDomainNumatuneSet(persistentDef->numa, + persistentDef->placement_mode =3D=3D + VIR_DOMAIN_CPU_PLACEMENT_MODE_STATIC, + -1, mode, nodeset) < 0) + goto endjob; + + if (virDomainDefSave(persistentDef, driver->xmlopt, cfg->configDir= ) < 0) + goto endjob; + } + */ + + ret =3D 0; + + endjob: + virCHDomainObjEndJob(vm); + + cleanup: + virBitmapFree(nodeset); + virDomainObjEndAPI(&vm); + return ret; +} + /* Function Tables */ static virHypervisorDriver chHypervisorDriver =3D { .name =3D "CH", @@ -1340,6 +1611,8 @@ static virHypervisorDriver chHypervisorDriver =3D { .domainPinVcpu =3D chDomainPinVcpu, /* 8.0.0 */ .domainPinVcpuFlags =3D chDomainPinVcpuFlags, /* 8.0.0 */ .nodeGetCPUMap =3D chNodeGetCPUMap, /* 8.0.0 */ + .domainSetNumaParameters =3D chDomainSetNumaParameters, /* 8.0.0 */ + .domainGetNumaParameters =3D chDomainGetNumaParameters, /* 8.0.0 */ }; =20 static virConnectDriver chConnectDriver =3D { --=20 2.27.0 From nobody Sun May 5 13:15:08 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) client-ip=170.10.133.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail(p=none dis=none) header.from=linux.microsoft.com Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mx.zohomail.com with SMTPS id 16391685422411016.2408186300285; Fri, 10 Dec 2021 12:35:42 -0800 (PST) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-559-3eKlxdMsNx2DGEPKzGPJ5Q-1; Fri, 10 Dec 2021 15:35:37 -0500 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 9C00F192FDA9; Fri, 10 Dec 2021 20:35:31 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 817E845D61; Fri, 10 Dec 2021 20:35:31 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 56F771802E2E; Fri, 10 Dec 2021 20:35:31 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 1BAKZ2Gh008457 for ; Fri, 10 Dec 2021 15:35:02 -0500 Received: by smtp.corp.redhat.com (Postfix) id 4CE86202660C; Fri, 10 Dec 2021 20:35:02 +0000 (UTC) Received: from mimecast-mx02.redhat.com (mimecast08.extmail.prod.ext.rdu2.redhat.com [10.11.55.24]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 48B222026D46 for ; Fri, 10 Dec 2021 20:34:58 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-2.mimecast.com [205.139.110.61]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 80DBE39D30BA for ; Fri, 10 Dec 2021 20:34:58 +0000 (UTC) Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by relay.mimecast.com with ESMTP id us-mta-372-g4LKwyhGPiKc5dhY9N7kDQ-1; Fri, 10 Dec 2021 15:34:56 -0500 Received: from prapal-ch2.oiwy50ateaxezb1sqsoezlib2f.xx.internal.cloudapp.net (unknown [20.80.162.67]) by linux.microsoft.com (Postfix) with ESMTPSA id 4473A20B7194; Fri, 10 Dec 2021 12:34:55 -0800 (PST) X-MC-Unique: 3eKlxdMsNx2DGEPKzGPJ5Q-1 X-MC-Unique: g4LKwyhGPiKc5dhY9N7kDQ-1 DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 4473A20B7194 From: Praveen K Paladugu To: libvir-list@redhat.com Subject: [libvirt PATCH v3 12/13] ch_process: Setup emulator and iothread settings Date: Fri, 10 Dec 2021 20:34:47 +0000 Message-Id: <20211210203448.1579606-13-prapal@linux.microsoft.com> In-Reply-To: <20211210203448.1579606-1-prapal@linux.microsoft.com> References: <20211210203448.1579606-1-prapal@linux.microsoft.com> MIME-Version: 1.0 X-Mimecast-Impersonation-Protect: Policy=CLT - Impersonation Protection Definition; Similar Internal Domain=false; Similar Monitored External Domain=false; Custom External Domain=false; Mimecast External Domain=false; Newly Observed Domain=false; Internal User Name=false; Custom Display Name List=false; Reply-to Address Mismatch=false; Targeted Threat Dictionary=false; Mimecast Threat Dictionary=false; Custom Threat Dictionary=false X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-MIME-Autoconverted: from quoted-printable to 8bit by lists01.pubmisc.prod.ext.phx2.redhat.com id 1BAKZ2Gh008457 X-loop: libvir-list@redhat.com Cc: william.douglas@intel.com X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZM-MESSAGEID: 1639168543556100001 Content-Type: text/plain; charset="utf-8" using virCHProcessSetupPid Signed-off-by: Praveen K Paladugu --- src/ch/ch_monitor.c | 60 +++++++++++++++++++++++++++++++++++ src/ch/ch_monitor.h | 2 ++ src/ch/ch_process.c | 77 ++++++++++++++++++++++++++++++++++++++++++++- 3 files changed, 138 insertions(+), 1 deletion(-) diff --git a/src/ch/ch_monitor.c b/src/ch/ch_monitor.c index 4de86f3c03..1709c73181 100644 --- a/src/ch/ch_monitor.c +++ b/src/ch/ch_monitor.c @@ -921,3 +921,63 @@ virCHMonitorGetInfo(virCHMonitor *mon, virJSONValue **= info) { return virCHMonitorGet(mon, URL_VM_INFO, info); } + +/** + * virCHMonitorGetIOThreads: + * @mon: Pointer to the monitor + * @iothreads: Location to return array of IOThreadInfo data + * + * Retrieve the list of iothreads defined/running for the machine + * + * Returns count of IOThreadInfo structures on success + * -1 on error. + */ +int virCHMonitorGetIOThreads(virCHMonitor *mon, + virDomainIOThreadInfo ***iothreads) +{ + size_t nthreads =3D 0, niothreads =3D 0; + int thd_index; + virDomainIOThreadInfo **iothreadinfolist =3D NULL, *iothreadinfo =3D N= ULL; + + *iothreads =3D NULL; + nthreads =3D virCHMonitorRefreshThreadInfo(mon); + + iothreadinfolist =3D g_new0(virDomainIOThreadInfo*, nthreads); + + for (thd_index =3D 0; thd_index < nthreads; thd_index++) { + virBitmap *map =3D NULL; + if (mon->threads[thd_index].type =3D=3D virCHThreadTypeIO) { + iothreadinfo =3D g_new0(virDomainIOThreadInfo, 1); + + iothreadinfo->iothread_id =3D mon->threads[thd_index].ioInfo.t= id; + + if (!(map =3D virProcessGetAffinity(iothreadinfo->iothread_id)= )) + goto cleanup; + + if (virBitmapToData(map, &(iothreadinfo->cpumap), + &(iothreadinfo->cpumaplen)) < 0) { + virBitmapFree(map); + goto cleanup; + } + virBitmapFree(map); + //Append to iothreadinfolist + iothreadinfolist[niothreads] =3D iothreadinfo; + niothreads++; + } + } + VIR_DELETE_ELEMENT_INPLACE(iothreadinfolist, + niothreads, nthreads); + *iothreads =3D iothreadinfolist; + VIR_DEBUG("niothreads =3D %ld", niothreads); + return niothreads; + + cleanup: + if (iothreadinfolist) { + for (thd_index =3D 0; thd_index < niothreads; thd_index++) + VIR_FREE(iothreadinfolist[thd_index]); + VIR_FREE(iothreadinfolist); + } + if (iothreadinfo) + VIR_FREE(iothreadinfo); + return -1; +} diff --git a/src/ch/ch_monitor.h b/src/ch/ch_monitor.h index f8c3fa75e8..98edb0faf9 100644 --- a/src/ch/ch_monitor.h +++ b/src/ch/ch_monitor.h @@ -118,3 +118,5 @@ int virCHMonitorGetCPUInfo(virCHMonitor *mon, size_t maxvcpus); size_t virCHMonitorGetThreadInfo(virCHMonitor *mon, bool refresh, virCHMonitorThreadInfo **threads); +int virCHMonitorGetIOThreads(virCHMonitor *mon, + virDomainIOThreadInfo ***iothreads); diff --git a/src/ch/ch_process.c b/src/ch/ch_process.c index 56c8d4216d..1af89e0d9e 100644 --- a/src/ch/ch_process.c +++ b/src/ch/ch_process.c @@ -41,7 +41,6 @@ VIR_LOG_INIT("ch.ch_process"); #define START_VM_POSTFIX ": starting up vm\n" =20 =20 - static virCHMonitor * virCHProcessConnectMonitor(virCHDriver *driver, virDomainObj *vm) @@ -310,6 +309,74 @@ virCHProcessSetupPid(virDomainObj *vm, return ret; } =20 +static int +virCHProcessSetupIOThread(virDomainObj *vm, + virDomainIOThreadInfo *iothread) +{ + virCHDomainObjPrivate *priv =3D vm->privateData; + return virCHProcessSetupPid(vm, iothread->iothread_id, + VIR_CGROUP_THREAD_IOTHREAD, + iothread->iothread_id, + priv->autoCpuset, // This should be updated= when CLH supports accepting + // iothread settings from input domai= n definition + vm->def->cputune.iothread_period, + vm->def->cputune.iothread_quota, + NULL); // CLH doesn't allow choosing a sche= duler for iothreads. +} + +static int +virCHProcessSetupIOThreads(virDomainObj *vm) +{ + virCHDomainObjPrivate *priv =3D vm->privateData; + virDomainIOThreadInfo **iothreads =3D NULL; + size_t i; + size_t niothreads; + + niothreads =3D virCHMonitorGetIOThreads(priv->monitor, &iothreads); + for (i =3D 0; i < niothreads; i++) { + VIR_DEBUG("IOThread index =3D %ld , tid =3D %d", i, iothreads[i]->= iothread_id); + if (virCHProcessSetupIOThread(vm, iothreads[i]) < 0) + return -1; + } + return 0; +} + + +static int +virCHProcessSetupEmulatorThread(virDomainObj *vm, + virCHMonitorEmuThreadInfo emuthread) +{ + return virCHProcessSetupPid(vm, emuthread.tid, VIR_CGROUP_THREAD_EMULA= TOR, + 0, vm->def->cputune.emulatorpin, + vm->def->cputune.emulator_period, + vm->def->cputune.emulator_quota, + vm->def->cputune.emulatorsched); +} + +static int +virCHProcessSetupEmulatorThreads(virDomainObj *vm) +{ + int thd_index =3D 0; + virCHDomainObjPrivate *priv =3D vm->privateData; + /* + * Cloud-hypervisor start 4 Emulator threads by default: + * vmm + * cloud-hypervisor + * http-server + * signal_handler + */ + for (thd_index =3D 0; thd_index < priv->monitor->nthreads; thd_index++= ) { + if (priv->monitor->threads[thd_index].type =3D=3D virCHThreadTypeE= mulator) { + VIR_DEBUG("Setup tid =3D %d (%s) Emulator thread", priv->monit= or->threads[thd_index].emuInfo.tid, + priv->monitor->threads[thd_index].emuInfo.thrName); + + if (virCHProcessSetupEmulatorThread(vm, priv->monitor->threads= [thd_index].emuInfo) < 0) + return -1; + } + } + return 0; +} + /** * virCHProcessSetupVcpu: * @vm: domain object @@ -441,6 +508,14 @@ virCHProcessStart(virCHDriver *driver, =20 virCHDomainRefreshThreadInfo(vm); =20 + VIR_DEBUG("Setting emulator tuning/settings"); + if (virCHProcessSetupEmulatorThreads(vm) < 0) + goto cleanup; + + VIR_DEBUG("Setting iothread tuning/settings"); + if (virCHProcessSetupIOThreads(vm) < 0) + goto cleanup; + VIR_DEBUG("Setting global CPU cgroup (if required)"); if (virSetupGlobalCpuCgroup(vm, priv->cgroup, priv->autoNodeset) < 0) goto cleanup; --=20 2.27.0 From nobody Sun May 5 13:15:08 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) client-ip=170.10.133.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail(p=none dis=none) header.from=linux.microsoft.com Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mx.zohomail.com with SMTPS id 1639168513623518.7709842203147; Fri, 10 Dec 2021 12:35:13 -0800 (PST) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-460-3fhbURAqNbe53tK2Bp694A-1; Fri, 10 Dec 2021 15:35:09 -0500 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id A9FC6801962; Fri, 10 Dec 2021 20:35:04 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id D419445D61; Fri, 10 Dec 2021 20:35:03 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 5FB2E1809C88; Fri, 10 Dec 2021 20:35:01 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 1BAKYxaA008376 for ; Fri, 10 Dec 2021 15:34:59 -0500 Received: by smtp.corp.redhat.com (Postfix) id DF6CB40CFD12; Fri, 10 Dec 2021 20:34:58 +0000 (UTC) Received: from mimecast-mx02.redhat.com (mimecast05.extmail.prod.ext.rdu2.redhat.com [10.11.55.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id DA89440CFD0A for ; Fri, 10 Dec 2021 20:34:58 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-2.mimecast.com [205.139.110.61]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id C15328001EA for ; Fri, 10 Dec 2021 20:34:58 +0000 (UTC) Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by relay.mimecast.com with ESMTP id us-mta-402-DaT9SjnpNmqhwwS02cRDuw-1; Fri, 10 Dec 2021 15:34:56 -0500 Received: from prapal-ch2.oiwy50ateaxezb1sqsoezlib2f.xx.internal.cloudapp.net (unknown [20.80.162.67]) by linux.microsoft.com (Postfix) with ESMTPSA id 5B6B320B7195; Fri, 10 Dec 2021 12:34:55 -0800 (PST) X-MC-Unique: 3fhbURAqNbe53tK2Bp694A-1 X-MC-Unique: DaT9SjnpNmqhwwS02cRDuw-1 DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 5B6B320B7195 From: Praveen K Paladugu To: libvir-list@redhat.com Subject: [libvirt PATCH v3 13/13] ch_driver: emulator threadinfo & pinning callbacks Date: Fri, 10 Dec 2021 20:34:48 +0000 Message-Id: <20211210203448.1579606-14-prapal@linux.microsoft.com> In-Reply-To: <20211210203448.1579606-1-prapal@linux.microsoft.com> References: <20211210203448.1579606-1-prapal@linux.microsoft.com> MIME-Version: 1.0 X-Mimecast-Impersonation-Protect: Policy=CLT - Impersonation Protection Definition; Similar Internal Domain=false; Similar Monitored External Domain=false; Custom External Domain=false; Mimecast External Domain=false; Newly Observed Domain=false; Internal User Name=false; Custom Display Name List=false; Reply-to Address Mismatch=false; Targeted Threat Dictionary=false; Mimecast Threat Dictionary=false; Custom Threat Dictionary=false X-Scanned-By: MIMEDefang 2.84 on 10.11.54.1 X-MIME-Autoconverted: from quoted-printable to 8bit by lists01.pubmisc.prod.ext.phx2.redhat.com id 1BAKYxaA008376 X-loop: libvir-list@redhat.com Cc: william.douglas@intel.com X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZM-MESSAGEID: 1639168515067100001 Content-Type: text/plain; charset="utf-8" Signed-off-by: Praveen K Paladugu --- src/ch/ch_driver.c | 154 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 154 insertions(+) diff --git a/src/ch/ch_driver.c b/src/ch/ch_driver.c index d7008ef011..1f40f6561a 100644 --- a/src/ch/ch_driver.c +++ b/src/ch/ch_driver.c @@ -1297,6 +1297,158 @@ chDomainPinVcpu(virDomainPtr dom, VIR_DOMAIN_AFFECT_LIVE); } =20 + + +static int +chDomainGetEmulatorPinInfo(virDomainPtr dom, + unsigned char *cpumaps, + int maplen, + unsigned int flags) +{ + virDomainObj *vm =3D NULL; + virDomainDef *def; + virCHDomainObjPrivate *priv; + bool live; + int ret =3D -1; + virBitmap *cpumask =3D NULL; + g_autoptr(virBitmap) bitmap =3D NULL; + virBitmap *autoCpuset =3D NULL; + + virCheckFlags(VIR_DOMAIN_AFFECT_LIVE | + VIR_DOMAIN_AFFECT_CONFIG, -1); + + if (!(vm =3D chDomObjFromDomain(dom))) + goto cleanup; + + if (virDomainGetEmulatorPinInfoEnsureACL(dom->conn, vm->def) < 0) + goto cleanup; + + if (!(def =3D virDomainObjGetOneDefState(vm, flags, &live))) + goto cleanup; + + if (live) { + priv =3D vm->privateData; + autoCpuset =3D priv->autoCpuset; + } + if (def->cputune.emulatorpin) { + cpumask =3D def->cputune.emulatorpin; + } else if (def->cpumask) { + cpumask =3D def->cpumask; + } else if (vm->def->placement_mode =3D=3D VIR_DOMAIN_CPU_PLACEMENT_MOD= E_AUTO && + autoCpuset) { + cpumask =3D autoCpuset; + } else { + if (!(bitmap =3D virHostCPUGetAvailableCPUsBitmap())) + goto cleanup; + cpumask =3D bitmap; + } + + virBitmapToDataBuf(cpumask, cpumaps, maplen); + + ret =3D 1; + + cleanup: + virDomainObjEndAPI(&vm); + return ret; +} + +static int +chDomainPinEmulator(virDomainPtr dom, + unsigned char *cpumap, + int maplen, + unsigned int flags) +{ + virCHDriver *driver =3D dom->conn->privateData; + virDomainObj *vm; + virCgroup *cgroup_emulator =3D NULL; + virDomainDef *def; + virDomainDef *persistentDef; + int ret =3D -1; + virCHDomainObjPrivate *priv; + virBitmap *pcpumap =3D NULL; + g_autoptr(virCHDriverConfig) cfg =3D NULL; + g_autofree char *str =3D NULL; + virTypedParameterPtr eventParams =3D NULL; + int eventNparams =3D 0; + int eventMaxparams =3D 0; + + virCheckFlags(VIR_DOMAIN_AFFECT_LIVE | + VIR_DOMAIN_AFFECT_CONFIG, -1); + + cfg =3D virCHDriverGetConfig(driver); + + if (!(vm =3D chDomObjFromDomain(dom))) + goto cleanup; + + if (virDomainPinEmulatorEnsureACL(dom->conn, vm->def, flags) < 0) + goto cleanup; + + if (virCHDomainObjBeginJob(vm, CH_JOB_MODIFY) < 0) + goto cleanup; + + if (virDomainObjGetDefs(vm, flags, &def, &persistentDef) < 0) + goto endjob; + + priv =3D vm->privateData; + + if (!(pcpumap =3D virBitmapNewData(cpumap, maplen))) + goto endjob; + + if (virBitmapIsAllClear(pcpumap)) { + virReportError(VIR_ERR_INVALID_ARG, "%s", + _("Empty cpu list for pinning")); + goto endjob; + } + + if (def) { + if (virCgroupHasController(priv->cgroup, VIR_CGROUP_CONTROLLER_CPU= SET)) { + if (virCgroupNewThread(priv->cgroup, VIR_CGROUP_THREAD_EMULATO= R, + 0, false, &cgroup_emulator) < 0) + goto endjob; + + if (virSetupCgroupCpusetCpus(cgroup_emulator, pcpumap) < 0) { + virReportError(VIR_ERR_OPERATION_INVALID, "%s", + _("failed to set cpuset.cpus in cgroup" + " for emulator threads")); + goto endjob; + } + } + + if (virProcessSetAffinity(vm->pid, pcpumap, false) < 0) + goto endjob; + + virBitmapFree(def->cputune.emulatorpin); + def->cputune.emulatorpin =3D NULL; + + if (!(def->cputune.emulatorpin =3D virBitmapNewCopy(pcpumap))) + goto endjob; + + if (virDomainObjSave(vm, driver->xmlopt, cfg->stateDir) < 0) + goto endjob; + + str =3D virBitmapFormat(pcpumap); + if (virTypedParamsAddString(&eventParams, &eventNparams, + &eventMaxparams, + VIR_DOMAIN_TUNABLE_CPU_EMULATORPIN, + str) < 0) + goto endjob; + + } + + + ret =3D 0; + + endjob: + virCHDomainObjEndJob(vm); + + cleanup: + if (cgroup_emulator) + virCgroupFree(cgroup_emulator); + virBitmapFree(pcpumap); + virDomainObjEndAPI(&vm); + return ret; +} + #define CH_NB_NUMA_PARAM 2 =20 static int @@ -1610,6 +1762,8 @@ static virHypervisorDriver chHypervisorDriver =3D { .domainGetVcpuPinInfo =3D chDomainGetVcpuPinInfo, /* 8.0.0 */ .domainPinVcpu =3D chDomainPinVcpu, /* 8.0.0 */ .domainPinVcpuFlags =3D chDomainPinVcpuFlags, /* 8.0.0 */ + .domainPinEmulator =3D chDomainPinEmulator, /* 8.0.0 */ + .domainGetEmulatorPinInfo =3D chDomainGetEmulatorPinInfo, /* 8.0.0 */ .nodeGetCPUMap =3D chNodeGetCPUMap, /* 8.0.0 */ .domainSetNumaParameters =3D chDomainSetNumaParameters, /* 8.0.0 */ .domainGetNumaParameters =3D chDomainGetNumaParameters, /* 8.0.0 */ --=20 2.27.0