From nobody Fri May 3 01:22:29 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail(p=none dis=none) header.from=virtuozzo.com Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1535972371228999.0679128492637; Mon, 3 Sep 2018 03:59:31 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx11.intmail.prod.int.phx2.redhat.com [10.5.11.26]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id C43B3308FB9C; Mon, 3 Sep 2018 10:59:28 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id A4D103091373; Mon, 3 Sep 2018 10:59:27 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 0EE1618005DF; Mon, 3 Sep 2018 10:59:26 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id w83AxOrT023336 for ; Mon, 3 Sep 2018 06:59:24 -0400 Received: by smtp.corp.redhat.com (Postfix) id 316A7190D0; Mon, 3 Sep 2018 10:59:24 +0000 (UTC) Received: from mx1.redhat.com (ext-mx13.extmail.prod.ext.phx2.redhat.com [10.5.110.42]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 00A4B6A033 for ; Mon, 3 Sep 2018 10:59:20 +0000 (UTC) Received: from relay.sw.ru (relay.sw.ru [185.231.240.75]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id EBE113082134 for ; Mon, 3 Sep 2018 10:59:15 +0000 (UTC) Received: from [10.94.3.220] (helo=dim-vz7.qa.sw.ru) by relay.sw.ru with esmtp (Exim 4.90_1) (envelope-from ) id 1fwmZZ-0000t8-4h; Mon, 03 Sep 2018 13:59:09 +0300 From: Nikolay Shirokovskiy To: libvir-list@redhat.com Date: Mon, 3 Sep 2018 13:58:31 +0300 Message-Id: <1535972311-452118-1-git-send-email-nshirokovskiy@virtuozzo.com> X-Greylist: Sender passed SPF test, ACL 232 matched, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.42]); Mon, 03 Sep 2018 10:59:17 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.42]); Mon, 03 Sep 2018 10:59:17 +0000 (UTC) for IP:'185.231.240.75' DOMAIN:'relay.sw.ru' HELO:'relay.sw.ru' FROM:'nshirokovskiy@virtuozzo.com' RCPT:'' X-RedHat-Spam-Score: -0.001 (SPF_PASS) 185.231.240.75 relay.sw.ru 185.231.240.75 relay.sw.ru X-Scanned-By: MIMEDefang 2.84 on 10.5.110.42 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-loop: libvir-list@redhat.com Cc: Vladimir Sementsov-Ogievskiy Subject: [libvirt] [PATCH] api,qemu: add block latency histogram X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.84 on 10.5.11.26 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.43]); Mon, 03 Sep 2018 10:59:29 +0000 (UTC) X-ZohoMail: RDMRC_1 RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" This patch adds option to configure/read latency histogram of block device IO operations. The corresponding functionality in qemu still has x- prefix AFAIK. So this patch should help qemu functionality to become non experimental.=20 In short histogram is configured thru new API virDomainSetBlockLatencyHisto= gram and histogram itself is available as new fields of virConnectGetAllDomainStats output. --- include/libvirt/libvirt-domain.h | 27 ++++++ src/access/viraccessperm.c | 3 +- src/access/viraccessperm.h | 6 ++ src/conf/domain_conf.c | 6 ++ src/conf/domain_conf.h | 1 + src/driver-hypervisor.h | 9 ++ src/libvirt-domain.c | 58 +++++++++++++ src/libvirt_private.syms | 2 + src/libvirt_public.syms | 5 ++ src/qemu/qemu_driver.c | 136 ++++++++++++++++++++++++++++++ src/qemu/qemu_monitor.c | 37 +++++++- src/qemu/qemu_monitor.h | 19 +++++ src/qemu/qemu_monitor_json.c | 163 ++++++++++++++++++++++++++++++++= ---- src/qemu/qemu_monitor_json.h | 8 ++ src/remote/remote_daemon_dispatch.c | 39 +++++++++ src/remote/remote_driver.c | 49 ++++++++++- src/remote/remote_protocol.x | 23 ++++- src/remote_protocol-structs | 11 +++ tools/virsh-domain.c | 118 ++++++++++++++++++++++++++ tools/virsh.pod | 18 ++++ 20 files changed, 716 insertions(+), 22 deletions(-) diff --git a/include/libvirt/libvirt-domain.h b/include/libvirt/libvirt-dom= ain.h index fdd2d6b..4fafa3d 100644 --- a/include/libvirt/libvirt-domain.h +++ b/include/libvirt/libvirt-domain.h @@ -4793,4 +4793,31 @@ int virDomainGetLaunchSecurityInfo(virDomainPtr doma= in, int *nparams, unsigned int flags); =20 +/* + * virDomainBlockLatencyHistogram: + * + * Selects a IO operation for which latency histogram is to be configured + */ +typedef enum { + VIR_DOMAIN_BLOCK_LATENCY_ALL =3D 0, + VIR_DOMAIN_BLOCK_LATENCY_READ =3D 1, + VIR_DOMAIN_BLOCK_LATENCY_WRITE =3D 2, + VIR_DOMAIN_BLOCK_LATENCY_FLUSH =3D 3, + +# ifdef VIR_ENUM_SENTINELS + VIR_DOMAIN_BLOCK_LATENCY_LAST + /* + * NB: this enum value will increase over time. It reflects the last s= tate + * supported by this version of the libvirt API. + */ +# endif +} virDomainBlockLatencyHistogram; + +int virDomainSetBlockLatencyHistogram(virDomainPtr dom, + const char *dev, + unsigned int op, + unsigned long long *boundaries, + int nboundaries, + unsigned int flags); + #endif /* __VIR_LIBVIRT_DOMAIN_H__ */ diff --git a/src/access/viraccessperm.c b/src/access/viraccessperm.c index d7cbb70..e08853a 100644 --- a/src/access/viraccessperm.c +++ b/src/access/viraccessperm.c @@ -43,7 +43,8 @@ VIR_ENUM_IMPL(virAccessPermDomain, "fs_trim", "fs_freeze", "block_read", "block_write", "mem_read", "open_graphics", "open_device", "screenshot", - "open_namespace", "set_time", "set_password"); + "open_namespace", "set_time", "set_password", + "block_stats_conf"); =20 VIR_ENUM_IMPL(virAccessPermInterface, VIR_ACCESS_PERM_INTERFACE_LAST, diff --git a/src/access/viraccessperm.h b/src/access/viraccessperm.h index 5ac5ff3..a3097c5 100644 --- a/src/access/viraccessperm.h +++ b/src/access/viraccessperm.h @@ -312,6 +312,12 @@ typedef enum { */ VIR_ACCESS_PERM_DOMAIN_SET_PASSWORD, =20 + /** + * @desc: Configure block stats gathering + * @message: Configuring block stats gathering requires authorization + */ + VIR_ACCESS_PERM_DOMAIN_BLOCK_STATS_CONF, + VIR_ACCESS_PERM_DOMAIN_LAST, } virAccessPermDomain; =20 diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index 38cac07..6095636 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -802,6 +802,12 @@ VIR_ENUM_IMPL(virDomainCrashedReason, VIR_DOMAIN_CRASH= ED_LAST, VIR_ENUM_IMPL(virDomainPMSuspendedReason, VIR_DOMAIN_PMSUSPENDED_LAST, "unknown") =20 +VIR_ENUM_IMPL(virDomainBlockLatencyHistogram, VIR_DOMAIN_BLOCK_LATENCY_LAS= T, + "all", + "read", + "write", + "flush") + VIR_ENUM_IMPL(virDomainSeclabel, VIR_DOMAIN_SECLABEL_LAST, "default", "none", diff --git a/src/conf/domain_conf.h b/src/conf/domain_conf.h index 8a36733..e2674c8 100644 --- a/src/conf/domain_conf.h +++ b/src/conf/domain_conf.h @@ -3466,6 +3466,7 @@ VIR_ENUM_DECL(virDomainShutdownReason) VIR_ENUM_DECL(virDomainShutoffReason) VIR_ENUM_DECL(virDomainCrashedReason) VIR_ENUM_DECL(virDomainPMSuspendedReason) +VIR_ENUM_DECL(virDomainBlockLatencyHistogram) =20 const char *virDomainStateReasonToString(virDomainState state, int reason); int virDomainStateReasonFromString(virDomainState state, const char *reaso= n); diff --git a/src/driver-hypervisor.h b/src/driver-hypervisor.h index eef31eb..968334f 100644 --- a/src/driver-hypervisor.h +++ b/src/driver-hypervisor.h @@ -1322,6 +1322,14 @@ typedef int unsigned int flags); =20 =20 +typedef int +(*virDrvDomainSetBlockLatencyHistogram)(virDomainPtr dom, + const char *dev, + unsigned int op, + unsigned long long *boundaries, + int nboundaries, + unsigned int flags); + typedef struct _virHypervisorDriver virHypervisorDriver; typedef virHypervisorDriver *virHypervisorDriverPtr; =20 @@ -1572,6 +1580,7 @@ struct _virHypervisorDriver { virDrvConnectBaselineHypervisorCPU connectBaselineHypervisorCPU; virDrvNodeGetSEVInfo nodeGetSEVInfo; virDrvDomainGetLaunchSecurityInfo domainGetLaunchSecurityInfo; + virDrvDomainSetBlockLatencyHistogram domainSetBlockLatencyHistogram; }; =20 =20 diff --git a/src/libvirt-domain.c b/src/libvirt-domain.c index ef46027..bfc1947 100644 --- a/src/libvirt-domain.c +++ b/src/libvirt-domain.c @@ -12223,3 +12223,61 @@ int virDomainGetLaunchSecurityInfo(virDomainPtr do= main, virDispatchError(domain->conn); return -1; } + + +/* + * virDomainSetBlockLatencyHistogram: + * @domain: pointer to domain object + * @dev: string specifying the block device + * @op: selects io operation to set histogram for + * @boundaries: borders of histogram bins in nanoseconds, 0 ns and +inf ns + * borders are implicit and should not be specified + * @nboundaries: number of boundaries + * @flags: currently unused, callers should pass 0 + * + * Configures latency histogram parameters for IO operation @op for disk @= dev + * of the @domain. If @op is VIR_DOMAIN_BLOCK_LATENCY_ALL then histograms + * for all IO operations will be configured. + * + * @boundaries is list of histogram iterval boundaries in nanoseconds in a= csending + * order. Boundaries 0 and +inf are implicit and should not be specified. + * For example 10, 50, 100 will configure histogram with intervals + * [0, 10), [10, 50), [50, 100) and [100, +inf). If @boundaries is NULL + * and @op is VIR_DOMAIN_BLOCK_LATENCY_ALL then all the histograms will be= removed + * that is they will not be collected and will not be in the domain stats = output. + * + * Returns 0 on success, -1 on failure. + */ +int virDomainSetBlockLatencyHistogram(virDomainPtr domain, + const char *dev, + unsigned int op, + unsigned long long *boundaries, + int nboundaries, + unsigned int flags) +{ + VIR_DOMAIN_DEBUG(domain, "dev=3D'%s' op=3D%d boundaries=3D%p " + "nboundaries=3D%d flags=3D0x%x", + NULLSTR(dev), op, boundaries, nboundaries, flags); + + virResetLastError(); + + virCheckDomainReturn(domain, -1); + virCheckReadOnlyGoto(domain->conn->flags, error); + + virCheckNonNullArgGoto(dev, error); + + if (domain->conn->driver->domainSetBlockLatencyHistogram) { + int ret; + ret =3D domain->conn->driver->domainSetBlockLatencyHistogram( + domain, dev, op, boundaries, nboundaries, flags); + if (ret < 0) + goto error; + return ret; + } + + virReportUnsupportedError(); + + error: + virDispatchError(domain->conn); + return -1; +} diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index 954ab4b..3eb36f1 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -189,6 +189,8 @@ virDiskNameToIndex; virDomainActualNetDefFree; virDomainBlockedReasonTypeFromString; virDomainBlockedReasonTypeToString; +virDomainBlockLatencyHistogramTypeFromString; +virDomainBlockLatencyHistogramTypeToString; virDomainBootTypeFromString; virDomainBootTypeToString; virDomainCapabilitiesPolicyTypeToString; diff --git a/src/libvirt_public.syms b/src/libvirt_public.syms index d4cdbd8..4f20993 100644 --- a/src/libvirt_public.syms +++ b/src/libvirt_public.syms @@ -809,4 +809,9 @@ LIBVIRT_4.5.0 { virNWFilterBindingGetFilterName; } LIBVIRT_4.4.0; =20 +LIBVIRT_4.8.0 { + global: + virDomainSetBlockLatencyHistogram; +} LIBVIRT_4.5.0; + # .... define new API here using predicted next version number .... diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index a0f7c71..d473ddc 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -20099,6 +20099,48 @@ qemuDomainGetStatsOneBlockRefreshNamed(virStorageS= ourcePtr src, =20 =20 static int +qemuDomainGetBlockLatency(virDomainStatsRecordPtr record, + int *maxparams, + size_t block_idx, + const char *op, + qemuBlockLatencyStatsPtr latency) +{ + char param_name[VIR_TYPED_PARAM_FIELD_LENGTH]; + size_t i; + int ret =3D -1; + + if (!latency->nbins) + return 0; + + snprintf(param_name, VIR_TYPED_PARAM_FIELD_LENGTH, + "block.%zu.latency.%s.bincount", block_idx, op); + if (virTypedParamsAddUInt(&record->params, &record->nparams, maxparams, + param_name, latency->nbins) < 0) + goto cleanup; + + for (i =3D 0; i < latency->nbins; i++) { + snprintf(param_name, VIR_TYPED_PARAM_FIELD_LENGTH, + "block.%zu.latency.%s.bin.%zu", block_idx, op, i); + if (virTypedParamsAddULLong(&record->params, &record->nparams, max= params, + param_name, latency->bins[i]) < 0) + goto cleanup; + } + + for (i =3D 0; i < latency->nbins - 1; i++) { + snprintf(param_name, VIR_TYPED_PARAM_FIELD_LENGTH, + "block.%zu.latency.%s.boundary.%zu", block_idx, op, i); + if (virTypedParamsAddULLong(&record->params, &record->nparams, max= params, + param_name, latency->boundaries[i]) < = 0) + goto cleanup; + } + + ret =3D 0; + cleanup: + return ret; +} + + +static int qemuDomainGetStatsOneBlock(virQEMUDriverPtr driver, virQEMUDriverConfigPtr cfg, virDomainObjPtr dom, @@ -20128,6 +20170,14 @@ qemuDomainGetStatsOneBlock(virQEMUDriverPtr driver, goto cleanup; } =20 + if (qemuDomainGetBlockLatency(record, maxparams, block_idx, "rd", + &entry->rd_latency) < 0 || + qemuDomainGetBlockLatency(record, maxparams, block_idx, "wr", + &entry->wr_latency) < 0 || + qemuDomainGetBlockLatency(record, maxparams, block_idx, "fl", + &entry->flush_latency) < 0) + goto cleanup; + QEMU_ADD_BLOCK_PARAM_ULL(record, maxparams, block_idx, "allocation", entry->wr_highest_offset); =20 @@ -21752,6 +21802,91 @@ qemuDomainGetLaunchSecurityInfo(virDomainPtr domai= n, return ret; } =20 + +static +int qemuDomainSetBlockLatencyHistogram(virDomainPtr dom, + const char *dev, + unsigned int op, + unsigned long long *boundaries, + int nboundaries, + unsigned int flags) +{ + virQEMUDriverPtr driver =3D dom->conn->privateData; + qemuDomainObjPrivatePtr priv; + virDomainObjPtr vm; + virDomainDiskDefPtr disk; + char *device =3D NULL; + bool job =3D false; + int ret =3D -1; + int rc; + + virCheckFlags(0, -1); + + if (op >=3D VIR_DOMAIN_BLOCK_LATENCY_LAST) { + virReportError(VIR_ERR_INVALID_ARG, + _("unknown latency histogram: %d"), op); + return -1; + } + + if (!boundaries && op) { + virReportError(VIR_ERR_OPERATION_UNSUPPORTED, "%s", + _("per operation histogram deletion is not supporte= d")); + return -1; + } + + if (boundaries && nboundaries > 1) { + size_t i; + + for (i =3D 0; i < nboundaries - 1; i++) { + if (boundaries[i] > boundaries[i + 1]) { + virReportError(VIR_ERR_INVALID_ARG, "%s", + _("boundaries should be in ascending order"= )); + return -1; + } + } + } + + if (!(vm =3D qemuDomObjFromDomain(dom))) + return -1; + + if (virDomainSetBlockLatencyHistogramEnsureACL(dom->conn, vm->def) < 0) + goto cleanup; + + if (qemuDomainObjBeginJob(driver, vm, QEMU_JOB_MODIFY) < 0) + goto cleanup; + job =3D true; + + if (!virDomainObjIsActive(vm)) { + virReportError(VIR_ERR_OPERATION_INVALID, + "%s", _("domain is not running")); + goto cleanup; + } + + priv =3D vm->privateData; + + if (!(disk =3D qemuDomainDiskByName(vm->def, dev))) + goto cleanup; + + if (!(device =3D qemuAliasDiskDriveFromDisk(disk))) + goto cleanup; + + qemuDomainObjEnterMonitor(driver, vm); + rc =3D qemuMonitorBlockLatencyHistogramSet(priv->mon, device, op, + boundaries, nboundaries); + if (qemuDomainObjExitMonitor(driver, vm) < 0 || rc < 0) + goto cleanup; + + ret =3D 0; + cleanup: + if (job) + qemuDomainObjEndJob(driver, vm); + virDomainObjEndAPI(&vm); + VIR_FREE(device); + + return ret; +} + + static virHypervisorDriver qemuHypervisorDriver =3D { .name =3D QEMU_DRIVER_NAME, .connectURIProbe =3D qemuConnectURIProbe, @@ -21977,6 +22112,7 @@ static virHypervisorDriver qemuHypervisorDriver =3D= { .connectBaselineHypervisorCPU =3D qemuConnectBaselineHypervisorCPU, /*= 4.4.0 */ .nodeGetSEVInfo =3D qemuNodeGetSEVInfo, /* 4.5.0 */ .domainGetLaunchSecurityInfo =3D qemuDomainGetLaunchSecurityInfo, /* 4= .5.0 */ + .domainSetBlockLatencyHistogram =3D qemuDomainSetBlockLatencyHistogram= , /* 4.8.0 */ }; =20 =20 diff --git a/src/qemu/qemu_monitor.c b/src/qemu/qemu_monitor.c index 4c0002d..064da62 100644 --- a/src/qemu/qemu_monitor.c +++ b/src/qemu/qemu_monitor.c @@ -2257,6 +2257,25 @@ qemuMonitorQueryBlockstats(qemuMonitorPtr mon) } =20 =20 +void +qemuBlockStatsFree(void *value, const void *name ATTRIBUTE_UNUSED) +{ + qemuBlockStatsPtr stats =3D value; + + if (!stats) + return; + + VIR_FREE(stats->rd_latency.bins); + VIR_FREE(stats->rd_latency.boundaries); + VIR_FREE(stats->wr_latency.bins); + VIR_FREE(stats->wr_latency.boundaries); + VIR_FREE(stats->flush_latency.bins); + VIR_FREE(stats->flush_latency.boundaries); + + VIR_FREE(stats); +} + + /** * qemuMonitorGetAllBlockStatsInfo: * @mon: monitor object @@ -2279,7 +2298,7 @@ qemuMonitorGetAllBlockStatsInfo(qemuMonitorPtr mon, =20 QEMU_CHECK_MONITOR(mon); =20 - if (!(*ret_stats =3D virHashCreate(10, virHashValueFree))) + if (!(*ret_stats =3D virHashCreate(10, qemuBlockStatsFree))) goto error; =20 ret =3D qemuMonitorJSONGetAllBlockStatsInfo(mon, *ret_stats, @@ -4429,3 +4448,19 @@ qemuMonitorGetPRManagerInfo(qemuMonitorPtr mon, virHashFree(info); return ret; } + +int +qemuMonitorBlockLatencyHistogramSet(qemuMonitorPtr mon, + const char *device, + unsigned int op, + unsigned long long *boundaries, + int nboundaries) +{ + VIR_DEBUG("mon=3D%p, device=3D%s op=3D%d boundaries=3D%p nboundaries= =3D%d", + mon, device, op, boundaries, nboundaries); + + QEMU_CHECK_MONITOR(mon); + + return qemuMonitorJSONBlockLatencyHistogramSet(mon, device, op, + boundaries, nboundaries= ); +} diff --git a/src/qemu/qemu_monitor.h b/src/qemu/qemu_monitor.h index 48b142a..9295ecb 100644 --- a/src/qemu/qemu_monitor.h +++ b/src/qemu/qemu_monitor.h @@ -569,6 +569,14 @@ virHashTablePtr qemuMonitorGetBlockInfo(qemuMonitorPtr= mon); =20 virJSONValuePtr qemuMonitorQueryBlockstats(qemuMonitorPtr mon); =20 +typedef struct _qemuBlockLatencyStats qemuBlockLatencyStats; +typedef qemuBlockLatencyStats *qemuBlockLatencyStatsPtr; +struct _qemuBlockLatencyStats { + unsigned long long *boundaries; + unsigned long long *bins; + unsigned int nbins; +}; + typedef struct _qemuBlockStats qemuBlockStats; typedef qemuBlockStats *qemuBlockStatsPtr; struct _qemuBlockStats { @@ -580,6 +588,9 @@ struct _qemuBlockStats { long long wr_total_times; long long flush_req; long long flush_total_times; + qemuBlockLatencyStats rd_latency; + qemuBlockLatencyStats wr_latency; + qemuBlockLatencyStats flush_latency; unsigned long long capacity; unsigned long long physical; =20 @@ -592,6 +603,8 @@ struct _qemuBlockStats { unsigned long long write_threshold; }; =20 +void qemuBlockStatsFree(void *value, const void *name); + int qemuMonitorGetAllBlockStatsInfo(qemuMonitorPtr mon, virHashTablePtr *ret_stats, bool backingChain) @@ -1188,4 +1201,10 @@ struct _qemuMonitorPRManagerInfo { int qemuMonitorGetPRManagerInfo(qemuMonitorPtr mon, virHashTablePtr *retinfo); =20 +int qemuMonitorBlockLatencyHistogramSet(qemuMonitorPtr mon, + const char *device, + unsigned int op, + unsigned long long *boundaries, + int nboundaries); + #endif /* QEMU_MONITOR_H */ diff --git a/src/qemu/qemu_monitor_json.c b/src/qemu/qemu_monitor_json.c index 3181805..7622626 100644 --- a/src/qemu/qemu_monitor_json.c +++ b/src/qemu/qemu_monitor_json.c @@ -2319,6 +2319,64 @@ int qemuMonitorJSONGetBlockInfo(qemuMonitorPtr mon, } =20 =20 +static int +qemuMonitorJSONGetBlockLatencyStats(virJSONValuePtr stats, + const char *name, + qemuBlockLatencyStatsPtr latency) +{ + virJSONValuePtr latencyJSON; + virJSONValuePtr bins; + virJSONValuePtr boundaries; + size_t i; + + if (!(latencyJSON =3D virJSONValueObjectGetObject(stats, name))) + return 0; + + if (!(bins =3D virJSONValueObjectGetArray(latencyJSON, "bins"))) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("cannot read bins in latency %s"), name); + return -1; + } + + if (!(boundaries =3D virJSONValueObjectGetArray(latencyJSON, "boundari= es"))) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("cannot read boundaries in latency %s"), name); + return -1; + } + + if (virJSONValueArraySize(bins) !=3D virJSONValueArraySize(boundaries)= + 1) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("bins and boundaries size mismatch in latency %s"= ), name); + return -1; + } + latency->nbins =3D virJSONValueArraySize(bins); + + if (VIR_ALLOC_N(latency->boundaries, latency->nbins - 1) < 0 || + VIR_ALLOC_N(latency->bins, latency->nbins) < 0) + return -1; + + for (i =3D 0; i < latency->nbins; i++) { + if (virJSONValueGetNumberUlong(virJSONValueArrayGet(bins, i), + &latency->bins[i]) < 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("invalid bins in latency %s"), name); + return -1; + } + } + + for (i =3D 0; i < latency->nbins - 1; i++) { + if (virJSONValueGetNumberUlong(virJSONValueArrayGet(boundaries, i), + &latency->boundaries[i]) < 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("invalid boundaries in latency %s"), name); + return -1; + } + } + + return 0; +} + + static qemuBlockStatsPtr qemuMonitorJSONBlockStatsCollectData(virJSONValuePtr dev, int *nstats) @@ -2358,6 +2416,17 @@ qemuMonitorJSONBlockStatsCollectData(virJSONValuePtr= dev, QEMU_MONITOR_BLOCK_STAT_GET("flush_total_time_ns", bstats->flush_total= _times, false); #undef QEMU_MONITOR_BLOCK_STAT_GET =20 +#define QEMU_MONITOR_BLOCK_LATENCY_GET(NAME) \ + if (qemuMonitorJSONGetBlockLatencyStats(stats, \ + #NAME "_latency_histogram", \ + &bstats->NAME ## _latency) < 0= ) \ + goto cleanup; + + QEMU_MONITOR_BLOCK_LATENCY_GET(rd); + QEMU_MONITOR_BLOCK_LATENCY_GET(wr); + QEMU_MONITOR_BLOCK_LATENCY_GET(flush); +#undef QEMU_MONITOR_BLOCK_LATENCY_GET + if ((parent =3D virJSONValueObjectGetObject(dev, "parent")) && (parentstats =3D virJSONValueObjectGetObject(parent, "stats"))) { if (virJSONValueObjectGetNumberUlong(parentstats, "wr_highest_offs= et", @@ -2368,30 +2437,28 @@ qemuMonitorJSONBlockStatsCollectData(virJSONValuePt= r dev, VIR_STEAL_PTR(ret, bstats); =20 cleanup: - VIR_FREE(bstats); + qemuBlockStatsFree(bstats, NULL); return ret; } =20 =20 static int -qemuMonitorJSONAddOneBlockStatsInfo(qemuBlockStatsPtr bstats, +qemuMonitorJSONAddOneBlockStatsInfo(virJSONValuePtr dev, const char *name, virHashTablePtr stats) { - qemuBlockStatsPtr copy =3D NULL; + qemuBlockStatsPtr bstats; + int nstats; =20 - if (VIR_ALLOC(copy) < 0) + if (!(bstats =3D qemuMonitorJSONBlockStatsCollectData(dev, &nstats))) return -1; =20 - if (bstats) - *copy =3D *bstats; - - if (virHashAddEntry(stats, name, copy) < 0) { - VIR_FREE(copy); + if (virHashAddEntry(stats, name, bstats) < 0) { + qemuBlockStatsFree(bstats, NULL); return -1; } =20 - return 0; + return nstats; } =20 =20 @@ -2402,7 +2469,6 @@ qemuMonitorJSONGetOneBlockStatsInfo(virJSONValuePtr d= ev, virHashTablePtr hash, bool backingChain) { - qemuBlockStatsPtr bstats =3D NULL; int ret =3D -1; int nstats =3D 0; const char *qdevname =3D NULL; @@ -2423,19 +2489,16 @@ qemuMonitorJSONGetOneBlockStatsInfo(virJSONValuePtr= dev, goto cleanup; } =20 - if (!(bstats =3D qemuMonitorJSONBlockStatsCollectData(dev, &nstats))) - goto cleanup; - if (devicename && - qemuMonitorJSONAddOneBlockStatsInfo(bstats, devicename, hash) < 0) + (nstats =3D qemuMonitorJSONAddOneBlockStatsInfo(dev, devicename, h= ash)) < 0) goto cleanup; =20 if (qdevname && STRNEQ_NULLABLE(qdevname, devicename) && - qemuMonitorJSONAddOneBlockStatsInfo(bstats, qdevname, hash) < 0) + qemuMonitorJSONAddOneBlockStatsInfo(dev, qdevname, hash) < 0) goto cleanup; =20 if (nodename && - qemuMonitorJSONAddOneBlockStatsInfo(bstats, nodename, hash) < 0) + qemuMonitorJSONAddOneBlockStatsInfo(dev, nodename, hash) < 0) goto cleanup; =20 if (backingChain && @@ -2446,7 +2509,6 @@ qemuMonitorJSONGetOneBlockStatsInfo(virJSONValuePtr d= ev, =20 ret =3D nstats; cleanup: - VIR_FREE(bstats); VIR_FREE(devicename); return ret; } @@ -8396,3 +8458,68 @@ qemuMonitorJSONGetPRManagerInfo(qemuMonitorPtr mon, return ret; =20 } + + +int +qemuMonitorJSONBlockLatencyHistogramSet(qemuMonitorPtr mon, + const char *device, + unsigned int op, + unsigned long long *boundaries, + int nboundaries) +{ + int ret =3D -1; + char *specOp =3D NULL; + virJSONValuePtr cmd =3D NULL; + virJSONValuePtr reply =3D NULL; + const char *specAll =3D "A:boundaries"; + const char *spec; + virJSONValuePtr boundariesJSON =3D NULL; + + if (op !=3D VIR_DOMAIN_BLOCK_LATENCY_ALL) { + if (virAsprintf(&specOp, "A:boundaries-%s", + virDomainBlockLatencyHistogramTypeToString(op)) < = 0) + return -1; + spec =3D specOp; + } else { + spec =3D specAll; + } + + if (boundaries) { + size_t i; + + if (!(boundariesJSON =3D virJSONValueNewArray())) + goto cleanup; + + for (i =3D 0; i < nboundaries; i++) { + virJSONValuePtr val; + + if (!(val =3D virJSONValueNewNumberUlong(boundaries[i])) || + virJSONValueArrayAppend(boundariesJSON, val) < 0) { + virJSONValueFree(val); + goto cleanup; + } + } + } + + if (!(cmd =3D qemuMonitorJSONMakeCommand("block-latency-histogram-set", + "s:device", device, + spec, &boundariesJSON, + NULL))) + return -1; + + if (qemuMonitorJSONCommand(mon, cmd, &reply) < 0) + goto cleanup; + + if (qemuMonitorJSONCheckError(cmd, reply) < 0) + goto cleanup; + + ret =3D 0; + + cleanup: + virJSONValueFree(cmd); + virJSONValueFree(reply); + virJSONValueFree(boundariesJSON); + VIR_FREE(specOp); + + return ret; +} diff --git a/src/qemu/qemu_monitor_json.h b/src/qemu/qemu_monitor_json.h index da267b1..f1636a4 100644 --- a/src/qemu/qemu_monitor_json.h +++ b/src/qemu/qemu_monitor_json.h @@ -577,4 +577,12 @@ int qemuMonitorJSONGetPRManagerInfo(qemuMonitorPtr mon, virHashTablePtr info) ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(2); =20 +int +qemuMonitorJSONBlockLatencyHistogramSet(qemuMonitorPtr mon, + const char *device, + unsigned int op, + unsigned long long *boundaries, + int nboundaries) + ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(2); + #endif /* QEMU_MONITOR_JSON_H */ diff --git a/src/remote/remote_daemon_dispatch.c b/src/remote/remote_daemon= _dispatch.c index e62ebfb..50d2dc4 100644 --- a/src/remote/remote_daemon_dispatch.c +++ b/src/remote/remote_daemon_dispatch.c @@ -7030,6 +7030,45 @@ remoteDispatchStorageVolGetInfoFlags(virNetServerPtr= server ATTRIBUTE_UNUSED, } =20 =20 +static int +remoteDispatchDomainSetBlockLatencyHistogram(virNetServerPtr server ATTRIB= UTE_UNUSED, + virNetServerClientPtr client, + virNetMessagePtr msg ATTRIBUT= E_UNUSED, + virNetMessageErrorPtr rerr, + remote_domain_set_block_laten= cy_histogram_args *args, + remote_domain_set_block_laten= cy_histogram_ret *ret) +{ + int rv =3D -1; + virDomainPtr dom =3D NULL; + int result; + struct daemonClientPrivate *priv =3D virNetServerClientGetPrivateData(= client); + + if (!priv->conn) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", _("connection not ope= n")); + goto cleanup; + } + + if (!(dom =3D get_nonnull_domain(priv->conn, args->dom))) + goto cleanup; + + if ((result =3D virDomainSetBlockLatencyHistogram(dom, args->dev, + args->op, + (unsigned long long *)= args->boundaries.boundaries_val, + args->boundaries.bound= aries_len, + args->flags)) < 0) + goto cleanup; + + ret->result =3D result; + rv =3D 0; + + cleanup: + if (rv < 0) + virNetMessageSaveError(rerr); + virObjectUnref(dom); + return rv; +} + + /*----- Helpers. -----*/ =20 /* get_nonnull_domain and get_nonnull_network turn an on-wire diff --git a/src/remote/remote_driver.c b/src/remote/remote_driver.c index 3b43e21..c5f9295 100644 --- a/src/remote/remote_driver.c +++ b/src/remote/remote_driver.c @@ -8153,6 +8153,52 @@ remoteStorageVolGetInfoFlags(virStorageVolPtr vol, } =20 =20 +static int +remoteDomainSetBlockLatencyHistogram(virDomainPtr dom, + const char *dev, + unsigned int op, + unsigned long long *boundaries, + int nboundaries, + unsigned int flags) +{ + int rv =3D -1; + struct private_data *priv =3D dom->conn->privateData; + remote_domain_set_block_latency_histogram_args args; + remote_domain_set_block_latency_histogram_ret ret; + + remoteDriverLock(priv); + + if (nboundaries > REMOTE_DOMAIN_SET_BLOCK_LATENCY_HISTOGRAM_BOUNDARIES= _MAX) { + virReportError(VIR_ERR_RPC, + _("boundaries length greater than maximum: %d > %d"= ), + nboundaries, + REMOTE_DOMAIN_SET_BLOCK_LATENCY_HISTOGRAM_BOUNDARIE= S_MAX); + goto done; + } + + make_nonnull_domain(&args.dom, dom); + args.dev =3D (char *)dev; + args.op =3D op; + args.boundaries.boundaries_val =3D (uint64_t *) boundaries; + args.boundaries.boundaries_len =3D nboundaries; + args.flags =3D flags; + + memset(&ret, 0, sizeof(ret)); + + if (call(dom->conn, priv, 0, REMOTE_PROC_DOMAIN_SET_BLOCK_LATENCY_HIST= OGRAM, + (xdrproc_t)xdr_remote_domain_set_block_latency_histogram_args= , (char *)&args, + (xdrproc_t)xdr_remote_domain_set_block_latency_histogram_ret,= (char *)&ret) =3D=3D -1) { + goto done; + } + + rv =3D ret.result; + + done: + remoteDriverUnlock(priv); + return rv; +} + + /* get_nonnull_domain and get_nonnull_network turn an on-wire * (name, uuid) pair into virDomainPtr or virNetworkPtr object. * These can return NULL if underlying memory allocations fail, @@ -8536,7 +8582,8 @@ static virHypervisorDriver hypervisor_driver =3D { .connectCompareHypervisorCPU =3D remoteConnectCompareHypervisorCPU, /*= 4.4.0 */ .connectBaselineHypervisorCPU =3D remoteConnectBaselineHypervisorCPU, = /* 4.4.0 */ .nodeGetSEVInfo =3D remoteNodeGetSEVInfo, /* 4.5.0 */ - .domainGetLaunchSecurityInfo =3D remoteDomainGetLaunchSecurityInfo /* = 4.5.0 */ + .domainGetLaunchSecurityInfo =3D remoteDomainGetLaunchSecurityInfo, /*= 4.5.0 */ + .domainSetBlockLatencyHistogram =3D remoteDomainSetBlockLatencyHistogr= am, /* 4.8.0 */ }; =20 static virNetworkDriver network_driver =3D { diff --git a/src/remote/remote_protocol.x b/src/remote/remote_protocol.x index 28c8feb..c4ede63 100644 --- a/src/remote/remote_protocol.x +++ b/src/remote/remote_protocol.x @@ -262,6 +262,9 @@ const REMOTE_NODE_SEV_INFO_MAX =3D 64; /* Upper limit on number of launch security information entries */ const REMOTE_DOMAIN_LAUNCH_SECURITY_INFO_PARAMS_MAX =3D 64; =20 +/* Upper limit on block latency histogram boundaries. */ +const REMOTE_DOMAIN_SET_BLOCK_LATENCY_HISTOGRAM_BOUNDARIES_MAX =3D 1024; + /* UUID. VIR_UUID_BUFLEN definition comes from libvirt.h */ typedef opaque remote_uuid[VIR_UUID_BUFLEN]; =20 @@ -3557,6 +3560,18 @@ struct remote_connect_list_all_nwfilter_bindings_ret= { /* insert@1 */ unsigned int ret; }; =20 +struct remote_domain_set_block_latency_histogram_args { + remote_nonnull_domain dom; + remote_nonnull_string dev; + unsigned int op; + unsigned hyper boundaries; + unsigned int flags; +}; + +struct remote_domain_set_block_latency_histogram_ret { + int result; +}; + /*----- Protocol. -----*/ =20 /* Define the program number, protocol version and procedure numbers here.= */ @@ -6312,5 +6327,11 @@ enum remote_procedure { * @acl: connect:search_nwfilter_bindings * @aclfilter: nwfilter_binding:getattr */ - REMOTE_PROC_CONNECT_LIST_ALL_NWFILTER_BINDINGS =3D 401 + REMOTE_PROC_CONNECT_LIST_ALL_NWFILTER_BINDINGS =3D 401, + + /** + * @generate: none + * @acl: domain:block_stats_conf + */ + REMOTE_PROC_DOMAIN_SET_BLOCK_LATENCY_HISTOGRAM =3D 402 }; diff --git a/src/remote_protocol-structs b/src/remote_protocol-structs index 6343e14..aa85253 100644 --- a/src/remote_protocol-structs +++ b/src/remote_protocol-structs @@ -2966,6 +2966,16 @@ struct remote_connect_list_all_nwfilter_bindings_ret= { } bindings; u_int ret; }; +struct remote_domain_set_block_latency_histogram_args { + remote_nonnull_domain dom; + remote_nonnull_string dev; + u_int op; + struct { + u_int boundaries_len; + uint64_t boundaries_val; + } boundaries; + u_int flags; +}; enum remote_procedure { REMOTE_PROC_CONNECT_OPEN =3D 1, REMOTE_PROC_CONNECT_CLOSE =3D 2, @@ -3368,4 +3378,5 @@ enum remote_procedure { REMOTE_PROC_NWFILTER_BINDING_CREATE_XML =3D 399, REMOTE_PROC_NWFILTER_BINDING_DELETE =3D 400, REMOTE_PROC_CONNECT_LIST_ALL_NWFILTER_BINDINGS =3D 401, + REMOTE_PROC_DOMAIN_SET_BLOCK_LATENCY_HISTOGRAM =3D 402, }; diff --git a/tools/virsh-domain.c b/tools/virsh-domain.c index c1cff9f..e4cf49b 100644 --- a/tools/virsh-domain.c +++ b/tools/virsh-domain.c @@ -13816,6 +13816,118 @@ cmdDomFSInfo(vshControl *ctl, const vshCmd *cmd) return ret >=3D 0; } =20 +/* + * "block-set-latency-histogram" command + */ +static const vshCmdInfo info_block_set_latency_histogram[] =3D { + {.name =3D "help", + .data =3D N_("configures latency histogram for given device for given= operation") + }, + {.name =3D "desc", + .data =3D N_("configures latency histogram for given device for given= operation") + }, + {.name =3D NULL} +}; + +static const vshCmdOptDef opts_block_set_latency_histogram[] =3D { + VIRSH_COMMON_OPT_DOMAIN_FULL(VIR_CONNECT_LIST_DOMAINS_ACTIVE), + {.name =3D "dev", + .type =3D VSH_OT_DATA, + .flags =3D VSH_OFLAG_REQ, + .help =3D N_("device to set latency histogram for") + }, + {.name =3D "boundaries", + .type =3D VSH_OT_STRING, + .help =3D N_("comma separated histogram boundaries in nanoseconds") + }, + {.name =3D "op", + .type =3D VSH_OT_STRING, + .help =3D N_("operation to set latency histogram for, all if omitted") + }, + {.name =3D NULL} +}; + +static bool +cmdBlockSetLatencyHistogram(vshControl *ctl, const vshCmd *cmd) +{ + const char *dev =3D NULL; + const char *opstr =3D NULL; + const char *boundariesstr =3D NULL; + char **boundarieslist =3D NULL; + char **tmp =3D NULL; + unsigned long long *boundaries =3D NULL; + size_t nboundaries =3D 0; + unsigned char op; + virDomainPtr dom; + bool ret =3D false; + + if (!(dom =3D virshCommandOptDomain(ctl, cmd, NULL))) + return false; + + if (vshCommandOptStringReq(ctl, cmd, "dev", &dev) < 0) + goto cleanup; + + if (vshCommandOptStringReq(ctl, cmd, "op", &opstr) < 0) + goto cleanup; + + if (opstr) { + if (STREQ(opstr, "read")) { + op =3D VIR_DOMAIN_BLOCK_LATENCY_READ; + } else if (STREQ(opstr, "write")) { + op =3D VIR_DOMAIN_BLOCK_LATENCY_WRITE; + } else if (STREQ(opstr, "flush")) { + op =3D VIR_DOMAIN_BLOCK_LATENCY_FLUSH; + } else { + vshError(ctl, _("Unknown operation %s value, expecting " + "'read', 'write', 'flush'."), opstr); + goto cleanup; + } + } else { + op =3D VIR_DOMAIN_BLOCK_LATENCY_ALL; + } + + if (vshCommandOptStringReq(ctl, cmd, "boundaries", &boundariesstr) < 0) + goto cleanup; + + if (!boundariesstr && op) { + vshError(ctl, _("per operation histogram deletion is not supported= ")); + goto cleanup; + } + + if (boundariesstr && + !(boundarieslist =3D virStringSplit(boundariesstr, ",", 0))) { + vshError(ctl, "%s", _("Cannot parse boundaries string")); + goto cleanup; + } + + tmp =3D boundarieslist; + while (boundarieslist && *tmp) { + unsigned long long val; + + if (virStrToLong_ull(*tmp, NULL, 10, &val) < 0) { + vshError(ctl, _("Cannot parse boundaries value: %s"), *tmp); + goto cleanup; + } + + if (VIR_APPEND_ELEMENT(boundaries, nboundaries, val) < 0) + goto cleanup; + + tmp++; + } + + if (virDomainSetBlockLatencyHistogram(dom, dev, op, + boundaries, nboundaries, 0) < 0) + goto cleanup; + + ret =3D true; + + cleanup: + virshDomainFree(dom); + virStringListFree(boundarieslist); + VIR_FREE(boundaries); + return ret; +} + const vshCmdDef domManagementCmds[] =3D { {.name =3D "attach-device", .handler =3D cmdAttachDevice, @@ -14425,5 +14537,11 @@ const vshCmdDef domManagementCmds[] =3D { .info =3D info_domblkthreshold, .flags =3D 0 }, + {.name =3D "block-set-latency-histogram", + .handler =3D cmdBlockSetLatencyHistogram, + .opts =3D opts_block_set_latency_histogram, + .info =3D info_block_set_latency_histogram, + .flags =3D 0 + }, {.name =3D NULL} }; diff --git a/tools/virsh.pod b/tools/virsh.pod index 86c041d..faed047 100644 --- a/tools/virsh.pod +++ b/tools/virsh.pod @@ -2988,6 +2988,24 @@ See B for information on I. Output the IP address and port number for the VNC display. If the informat= ion is not available the processes will provide an exit code of 1. =20 +=3Ditem B I I [I] +[I<--op> I] + +Configures latency histogram parameters for IO operation I for = disk +I and resets all histogram's bins to zero. I is one of +C, C or C. If I is omitted then histograms = for +all operations will be configured. Histogram itself is available via +B. + +I is a comma separated list of histogram iterval boundaries in +nanoseconds in acsending order. Boundaries 0 and +inf are implicit and sho= uld +not be specified. For example "10, 50, 100" will configure histogram with +intervals [0, 10), [10, 50), [50, 100) and [100, +inf). + +If both I and I are omitted then the histograms for= all +operations will be removed that is they will not be collected and will not= be +in the B output. + =3Dback =20 =3Dhead1 DEVICE COMMANDS --=20 1.8.3.1 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list