From nobody Sun Feb 8 06:05:25 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) client-ip=170.10.129.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail(p=none dis=none) header.from=linux.microsoft.com Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mx.zohomail.com with SMTPS id 164312762360452.896884820985406; Tue, 25 Jan 2022 08:20:23 -0800 (PST) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-637-ULgo4B3DPqmel0FcH0tfHA-1; Tue, 25 Jan 2022 11:20:19 -0500 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id BD4B81091DA0; Tue, 25 Jan 2022 16:20:13 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id EFE427DE59; Tue, 25 Jan 2022 16:20:12 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 036581809CB8; Tue, 25 Jan 2022 16:20:12 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 20PGK924002539 for ; Tue, 25 Jan 2022 11:20:10 -0500 Received: by smtp.corp.redhat.com (Postfix) id CCA5A401E77; Tue, 25 Jan 2022 16:20:09 +0000 (UTC) Received: from mimecast-mx02.redhat.com (mimecast01.extmail.prod.ext.rdu2.redhat.com [10.11.55.17]) by smtp.corp.redhat.com (Postfix) with ESMTPS id C8D0A401DAD for ; Tue, 25 Jan 2022 16:20:09 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-2.mimecast.com [207.211.31.81]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id A948585A5A8 for ; Tue, 25 Jan 2022 16:20:09 +0000 (UTC) Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by relay.mimecast.com with ESMTP id us-mta-613-Sn_TiuTUNg-Aro0FjDP7Ew-1; Tue, 25 Jan 2022 11:20:07 -0500 Received: from prapal-ch2.oiwy50ateaxezb1sqsoezlib2f.xx.internal.cloudapp.net (unknown [20.80.162.67]) by linux.microsoft.com (Postfix) with ESMTPSA id 640D020B83C9; Tue, 25 Jan 2022 08:20:05 -0800 (PST) X-MC-Unique: ULgo4B3DPqmel0FcH0tfHA-1 X-MC-Unique: Sn_TiuTUNg-Aro0FjDP7Ew-1 DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 640D020B83C9 From: Praveen K Paladugu To: libvir-list@redhat.com Subject: [libvirt PATCH v5 5/7] ch_driver: add numatune callbacks for CH driver Date: Tue, 25 Jan 2022 16:19:57 +0000 Message-Id: <20220125161959.2805257-6-prapal@linux.microsoft.com> In-Reply-To: <20220125161959.2805257-1-prapal@linux.microsoft.com> References: <20220125161959.2805257-1-prapal@linux.microsoft.com> MIME-Version: 1.0 X-Mimecast-Impersonation-Protect: Policy=CLT - Impersonation Protection Definition; Similar Internal Domain=false; Similar Monitored External Domain=false; Custom External Domain=false; Mimecast External Domain=false; Newly Observed Domain=false; Internal User Name=false; Custom Display Name List=false; Reply-to Address Mismatch=false; Targeted Threat Dictionary=false; Mimecast Threat Dictionary=false; Custom Threat Dictionary=false X-Scanned-By: MIMEDefang 2.85 on 10.11.54.10 X-MIME-Autoconverted: from quoted-printable to 8bit by lists01.pubmisc.prod.ext.phx2.redhat.com id 20PGK924002539 X-loop: libvir-list@redhat.com Cc: william.douglas@intel.com X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZM-MESSAGEID: 1643127625733100001 Content-Type: text/plain; charset="utf-8" From: Vineeth Pillai Signed-off-by: Vineeth Pillai Signed-off-by: Praveen K Paladugu --- src/ch/ch_driver.c | 260 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 260 insertions(+) diff --git a/src/ch/ch_driver.c b/src/ch/ch_driver.c index 32ae15f940..d257c025ef 100644 --- a/src/ch/ch_driver.c +++ b/src/ch/ch_driver.c @@ -43,6 +43,7 @@ #include "viruri.h" #include "virutil.h" #include "viruuid.h" +#include "virnuma.h" =20 #define VIR_FROM_THIS VIR_FROM_CH =20 @@ -1302,6 +1303,263 @@ chDomainPinVcpu(virDomainPtr dom, VIR_DOMAIN_AFFECT_LIVE); } =20 +#define CH_NB_NUMA_PARAM 2 + +static int +chDomainGetNumaParameters(virDomainPtr dom, + virTypedParameterPtr params, + int *nparams, + unsigned int flags) +{ + size_t i; + virDomainObj *vm =3D NULL; + virDomainNumatuneMemMode tmpmode =3D VIR_DOMAIN_NUMATUNE_MEM_STRICT; + virCHDomainObjPrivate *priv; + g_autofree char *nodeset =3D NULL; + int ret =3D -1; + virDomainDef *def =3D NULL; + bool live =3D false; + virBitmap *autoNodeset =3D NULL; + + virCheckFlags(VIR_DOMAIN_AFFECT_LIVE | + VIR_DOMAIN_AFFECT_CONFIG | + VIR_TYPED_PARAM_STRING_OKAY, -1); + + if (!(vm =3D virCHDomainObjFromDomain(dom))) + return -1; + priv =3D vm->privateData; + + if (virDomainGetNumaParametersEnsureACL(dom->conn, vm->def) < 0) + goto cleanup; + + if (!(def =3D virDomainObjGetOneDefState(vm, flags, &live))) + goto cleanup; + + if (live) + autoNodeset =3D priv->autoNodeset; + + if ((*nparams) =3D=3D 0) { + *nparams =3D CH_NB_NUMA_PARAM; + ret =3D 0; + goto cleanup; + } + + for (i =3D 0; i < CH_NB_NUMA_PARAM && i < *nparams; i++) { + virMemoryParameterPtr param =3D ¶ms[i]; + + switch (i) { + case 0: /* fill numa mode here */ + ignore_value(virDomainNumatuneGetMode(def->numa, -1, &tmpmode)= ); + + if (virTypedParameterAssign(param, VIR_DOMAIN_NUMA_MODE, + VIR_TYPED_PARAM_INT, tmpmode) < 0) + goto cleanup; + + break; + + case 1: /* fill numa nodeset here */ + nodeset =3D virDomainNumatuneFormatNodeset(def->numa, autoNode= set, -1); + + if (!nodeset || + virTypedParameterAssign(param, VIR_DOMAIN_NUMA_NODESET, + VIR_TYPED_PARAM_STRING, nodeset) <= 0) + goto cleanup; + + nodeset =3D NULL; + break; + + /* coverity[dead_error_begin] */ + default: + break; + /* should not hit here */ + } + } + + if (*nparams > CH_NB_NUMA_PARAM) + *nparams =3D CH_NB_NUMA_PARAM; + ret =3D 0; + + cleanup: + virDomainObjEndAPI(&vm); + return ret; +} + +static int +chDomainSetNumaParamsLive(virDomainObj *vm, + virBitmap *nodeset) +{ + g_autoptr(virCgroup) cgroup_temp =3D NULL; + virCHDomainObjPrivate *priv =3D vm->privateData; + g_autofree char *nodeset_str =3D NULL; + virDomainNumatuneMemMode mode; + size_t i =3D 0; + + + if (virDomainNumatuneGetMode(vm->def->numa, -1, &mode) =3D=3D 0 && + mode !=3D VIR_DOMAIN_NUMATUNE_MEM_STRICT) { + virReportError(VIR_ERR_OPERATION_INVALID, "%s", + _("change of nodeset for running domain requires st= rict numa mode")); + return -1; + } + + if (!virNumaNodesetIsAvailable(nodeset)) + return -1; + + /* Ensure the cpuset string is formatted before passing to cgroup */ + if (!(nodeset_str =3D virBitmapFormat(nodeset))) + return -1; + + if (virCgroupNewThread(priv->cgroup, VIR_CGROUP_THREAD_EMULATOR, 0, + false, &cgroup_temp) < 0 || + virCgroupSetCpusetMems(cgroup_temp, nodeset_str) < 0) + return -1; + + + for (i =3D 0; i < virDomainDefGetVcpusMax(vm->def); i++) { + virDomainVcpuDef *vcpu =3D virDomainDefGetVcpu(vm->def, i); + + if (!vcpu->online) + continue; + + if (virCgroupNewThread(priv->cgroup, VIR_CGROUP_THREAD_VCPU, i, + false, &cgroup_temp) < 0 || + virCgroupSetCpusetMems(cgroup_temp, nodeset_str) < 0) + return -1; + + return 0; + } + + for (i =3D 0; i < vm->def->niothreadids; i++) { + if (virCgroupNewThread(priv->cgroup, VIR_CGROUP_THREAD_IOTHREAD, + vm->def->iothreadids[i]->iothread_id, + false, &cgroup_temp) < 0 || + virCgroupSetCpusetMems(cgroup_temp, nodeset_str) < 0) + return -1; + } + + /* set nodeset for root cgroup */ + if (virCgroupSetCpusetMems(priv->cgroup, nodeset_str) < 0) + return -1; + + return 0; +} + +static int +chDomainSetNumaParameters(virDomainPtr dom, + virTypedParameterPtr params, + int nparams, + unsigned int flags) +{ + virCHDriver *driver =3D dom->conn->privateData; + size_t i; + virDomainDef *def; + virDomainDef *persistentDef; + virDomainObj *vm =3D NULL; + int ret =3D -1; + g_autoptr(virCHDriverConfig) cfg =3D NULL; + virCHDomainObjPrivate *priv; + virBitmap *nodeset =3D NULL; + virDomainNumatuneMemMode config_mode; + int mode =3D -1; + + virCheckFlags(VIR_DOMAIN_AFFECT_LIVE | + VIR_DOMAIN_AFFECT_CONFIG, -1); + + if (virTypedParamsValidate(params, nparams, + VIR_DOMAIN_NUMA_MODE, + VIR_TYPED_PARAM_INT, + VIR_DOMAIN_NUMA_NODESET, + VIR_TYPED_PARAM_STRING, + NULL) < 0) + return -1; + + if (!(vm =3D virCHDomainObjFromDomain(dom))) + return -1; + + priv =3D vm->privateData; + cfg =3D virCHDriverGetConfig(driver); + + if (virDomainSetNumaParametersEnsureACL(dom->conn, vm->def, flags) < 0) + goto cleanup; + + for (i =3D 0; i < nparams; i++) { + virTypedParameterPtr param =3D ¶ms[i]; + + if (STREQ(param->field, VIR_DOMAIN_NUMA_MODE)) { + mode =3D param->value.i; + + if (mode < 0 || mode >=3D VIR_DOMAIN_NUMATUNE_MEM_LAST) { + virReportError(VIR_ERR_INVALID_ARG, + _("unsupported numatune mode: '%d'"), mode); + goto cleanup; + } + + } else if (STREQ(param->field, VIR_DOMAIN_NUMA_NODESET)) { + if (virBitmapParse(param->value.s, &nodeset, + VIR_DOMAIN_CPUMASK_LEN) < 0) + goto cleanup; + + if (virBitmapIsAllClear(nodeset)) { + virReportError(VIR_ERR_OPERATION_INVALID, + _("Invalid nodeset of 'numatune': %s"), + param->value.s); + goto cleanup; + } + } + } + + if (virCHDomainObjBeginJob(vm, CH_JOB_MODIFY) < 0) + goto cleanup; + + if (virDomainObjGetDefs(vm, flags, &def, &persistentDef) < 0) + goto endjob; + + if (def) { + if (!driver->privileged) { + virReportError(VIR_ERR_OPERATION_UNSUPPORTED, "%s", + _("NUMA tuning is not available in session mode= ")); + goto endjob; + } + + if (!virCgroupHasController(priv->cgroup, VIR_CGROUP_CONTROLLER_CP= USET)) { + virReportError(VIR_ERR_OPERATION_INVALID, "%s", + _("cgroup cpuset controller is not mounted")); + goto endjob; + } + + if (mode !=3D -1 && + virDomainNumatuneGetMode(def->numa, -1, &config_mode) =3D=3D 0= && + config_mode !=3D mode) { + virReportError(VIR_ERR_OPERATION_INVALID, "%s", + _("can't change numatune mode for running domai= n")); + goto endjob; + } + + if (nodeset && + chDomainSetNumaParamsLive(vm, nodeset) < 0) + goto endjob; + + if (virDomainNumatuneSet(def->numa, + def->placement_mode =3D=3D + VIR_DOMAIN_CPU_PLACEMENT_MODE_STATIC, + -1, mode, nodeset) < 0) + goto endjob; + + if (virDomainObjSave(vm, driver->xmlopt, cfg->stateDir) < 0) + goto endjob; + } + + ret =3D 0; + + endjob: + virCHDomainObjEndJob(vm); + + cleanup: + virBitmapFree(nodeset); + virDomainObjEndAPI(&vm); + return ret; +} + /* Function Tables */ static virHypervisorDriver chHypervisorDriver =3D { .name =3D "CH", @@ -1346,6 +1604,8 @@ static virHypervisorDriver chHypervisorDriver =3D { .domainPinVcpu =3D chDomainPinVcpu, /* 8.1.0 */ .domainPinVcpuFlags =3D chDomainPinVcpuFlags, /* 8.1.0 */ .nodeGetCPUMap =3D chNodeGetCPUMap, /* 8.0.0 */ + .domainSetNumaParameters =3D chDomainSetNumaParameters, /* 8.1.0 */ + .domainGetNumaParameters =3D chDomainGetNumaParameters, /* 8.1.0 */ }; =20 static virConnectDriver chConnectDriver =3D { --=20 2.27.0