From nobody Wed Nov 5 07:16:03 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 15344837556741012.8068885170322; Thu, 16 Aug 2018 22:29:15 -0700 (PDT) Received: from localhost ([::1]:60032 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fqXJy-0002Wp-CQ for importer@patchew.org; Fri, 17 Aug 2018 01:29:14 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:36277) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fqXA8-0000d7-NM for qemu-devel@nongnu.org; Fri, 17 Aug 2018 01:19:07 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fqXA5-0005Ym-FD for qemu-devel@nongnu.org; Fri, 17 Aug 2018 01:19:04 -0400 Received: from out1-smtp.messagingengine.com ([66.111.4.25]:47645) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fqXA2-0005VT-Iu for qemu-devel@nongnu.org; Fri, 17 Aug 2018 01:18:59 -0400 Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.nyi.internal (Postfix) with ESMTP id 1BE0821E8D; Fri, 17 Aug 2018 01:18:57 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute4.internal (MEProxy); Fri, 17 Aug 2018 01:18:57 -0400 Received: from localhost (flamenco.cs.columbia.edu [128.59.20.216]) by mail.messagingengine.com (Postfix) with ESMTPA id 3F24610291; Fri, 17 Aug 2018 01:18:56 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=braap.org; h=cc :date:from:in-reply-to:message-id:references:subject:to :x-me-sender:x-me-sender:x-sasl-enc; s=mesmtp; bh=qbF8H4qbGfoEMj O33EVYPoFxiO5iK3gXNC0RmbtPy50=; b=zsWaegp6Y5/k4zwXEhIob3txI4Z+ul kRCR+auFeveancredmqOHQP5KGFNo1XTvsWmeyloTyPeLyxLL/9dLeWhQ+eIqEU+ uO8fZwv6XHqC+bGuTDIzccc6XEa6A4eJ+0nlqtu7tHDuuPtz4WDZJippYxLterqt agHjCLz9R/jeQ= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:date:from:in-reply-to:message-id :references:subject:to:x-me-sender:x-me-sender:x-sasl-enc; s= fm3; bh=qbF8H4qbGfoEMjO33EVYPoFxiO5iK3gXNC0RmbtPy50=; b=JuDLi6Tr GCiDlDOpf+0nVH1kDL7ZNR/r4EsuJEE6yyWEZw02O4yAEZVETrZ1hQNXDTzoCESY 34/rfppYo2l9T1odTkTfy2EqFrM4khqOk/fGyxaevjESDAeqyfyOSxlYu3dcyxZE EUS0MOxTmqs9lDBJpgH3s++47YxHk65EgBNz+LJW5aQ6hl8Ac/yyAEhF3RHMDkAs Kvg/PJ07aaq+rbAS+4/KcX25CHU33uR8cF1EhIZx/C7hKCm/nO4DkeEgo04dtD8d 1BagAfdGkC67W3LI03vcNv7nNMMHPc4xgVfWup3wAZEfNNt3Hqk90tw8HWROxIWl YHrzm2dxlK49dQ== X-ME-Proxy: X-ME-Sender: From: "Emilio G. Cota" To: qemu-devel@nongnu.org Date: Fri, 17 Aug 2018 01:18:45 -0400 Message-Id: <20180817051853.23792-2-cota@braap.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180817051853.23792-1-cota@braap.org> References: <20180817051853.23792-1-cota@braap.org> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 66.111.4.25 Subject: [Qemu-devel] [PATCH 1/9] qsp: QEMU's Synchronization Profiler X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , Peter Crosthwaite , Stefan Weil , "Dr. David Alan Gilbert" , Peter Xu , Markus Armbruster , Paolo Bonzini , Richard Henderson Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" The goal of this module is to profile synchronization primitives (i.e. mutexes, recursive mutexes and condition variables) so that scalability issues can be quickly diagnosed. Sync primitives are profiled by QSP based on the vaddr of the object access= ed as well as the call site (file:line_nr). That means the same object called from two different call sites will be tracked in separate entries, which might be reported together or separately (see subsequent commit on call site coalescing). Some perf numbers: Host: Intel(R) Core(TM) i7-6700K CPU @ 4.00GHz Command: taskset -c 0 tests/atomic_add-bench -d 5 -m - Before: 54.80 Mops/s - After: 54.75 Mops/s That is, a negligible slowdown due to the now indirect call to qemu_mutex_lock. Note that using a branch instead of an indirect call introduces a more severe slowdown (53.65 Mops/s, i.e. 2% slowdown). Enabling the profiler (with -p, added in this series) is more interesting: - No profiling: 54.75 Mops/s - W/ profiling: 12.53 Mops/s That is, a 4.36X slowdown. We can break down this slowdown by removing the get_clock calls or the entry lookup: - No profiling: 54.75 Mops/s - W/o get_clock: 25.37 Mops/s - W/o entry lookup: 19.30 Mops/s - W/ profiling: 12.53 Mops/s Signed-off-by: Emilio G. Cota --- include/qemu/qht.h | 1 + include/qemu/qsp.h | 22 ++ include/qemu/thread-posix.h | 4 +- include/qemu/thread-win32.h | 5 +- include/qemu/thread.h | 65 +++- util/qemu-thread-win32.c | 4 +- util/qht.c | 47 ++- util/qsp.c | 584 ++++++++++++++++++++++++++++++++++++ util/Makefile.objs | 1 + 9 files changed, 710 insertions(+), 23 deletions(-) create mode 100644 include/qemu/qsp.h create mode 100644 util/qsp.c diff --git a/include/qemu/qht.h b/include/qemu/qht.h index 1fb9116fa0..c9a11cc29a 100644 --- a/include/qemu/qht.h +++ b/include/qemu/qht.h @@ -46,6 +46,7 @@ typedef bool (*qht_lookup_func_t)(const void *obj, const = void *userp); typedef void (*qht_iter_func_t)(struct qht *ht, void *p, uint32_t h, void = *up); =20 #define QHT_MODE_AUTO_RESIZE 0x1 /* auto-resize when heavily loaded */ +#define QHT_MODE_RAW_MUTEXES 0x2 /* bypass the profiler (QSP) */ =20 /** * qht_init - Initialize a QHT diff --git a/include/qemu/qsp.h b/include/qemu/qsp.h new file mode 100644 index 0000000000..9c2bb60ff0 --- /dev/null +++ b/include/qemu/qsp.h @@ -0,0 +1,22 @@ +/* + * qsp.c - QEMU Synchronization Profiler + * + * Copyright (C) 2018, Emilio G. Cota + * + * License: GNU GPL, version 2 or later. + * See the COPYING file in the top-level directory. + * + * Note: this header file can *only* be included from thread.h. + */ +#ifndef QEMU_QSP_H +#define QEMU_QSP_H + +#include "qemu/fprintf-fn.h" + +void qsp_report(FILE *f, fprintf_function cpu_fprintf, size_t max); + +bool qsp_is_enabled(void); +void qsp_enable(void); +void qsp_disable(void); + +#endif /* QEMU_QSP_H */ diff --git a/include/qemu/thread-posix.h b/include/qemu/thread-posix.h index fd27b34128..c903525062 100644 --- a/include/qemu/thread-posix.h +++ b/include/qemu/thread-posix.h @@ -6,8 +6,8 @@ =20 typedef QemuMutex QemuRecMutex; #define qemu_rec_mutex_destroy qemu_mutex_destroy -#define qemu_rec_mutex_lock qemu_mutex_lock -#define qemu_rec_mutex_trylock qemu_mutex_trylock +#define qemu_rec_mutex_lock_impl qemu_mutex_lock_impl +#define qemu_rec_mutex_trylock_impl qemu_mutex_trylock_impl #define qemu_rec_mutex_unlock qemu_mutex_unlock =20 struct QemuMutex { diff --git a/include/qemu/thread-win32.h b/include/qemu/thread-win32.h index d668d789b4..50af5dd7ab 100644 --- a/include/qemu/thread-win32.h +++ b/include/qemu/thread-win32.h @@ -19,8 +19,9 @@ struct QemuRecMutex { }; =20 void qemu_rec_mutex_destroy(QemuRecMutex *mutex); -void qemu_rec_mutex_lock(QemuRecMutex *mutex); -int qemu_rec_mutex_trylock(QemuRecMutex *mutex); +void qemu_rec_mutex_lock_impl(QemuRecMutex *mutex, const char *file, int l= ine); +int qemu_rec_mutex_trylock_impl(QemuRecMutex *mutex, const char *file, + int line); void qemu_rec_mutex_unlock(QemuRecMutex *mutex); =20 struct QemuCond { diff --git a/include/qemu/thread.h b/include/qemu/thread.h index ef7bd16123..ab508df3cd 100644 --- a/include/qemu/thread.h +++ b/include/qemu/thread.h @@ -16,6 +16,9 @@ typedef struct QemuThread QemuThread; #include "qemu/thread-posix.h" #endif =20 +/* include QSP header once QemuMutex, QemuCond etc. are defined */ +#include "qemu/qsp.h" + #define QEMU_THREAD_JOINABLE 0 #define QEMU_THREAD_DETACHED 1 =20 @@ -25,10 +28,51 @@ int qemu_mutex_trylock_impl(QemuMutex *mutex, const cha= r *file, const int line); void qemu_mutex_lock_impl(QemuMutex *mutex, const char *file, const int li= ne); void qemu_mutex_unlock_impl(QemuMutex *mutex, const char *file, const int = line); =20 -#define qemu_mutex_lock(mutex) \ - qemu_mutex_lock_impl(mutex, __FILE__, __LINE__) -#define qemu_mutex_trylock(mutex) \ - qemu_mutex_trylock_impl(mutex, __FILE__, __LINE__) +typedef void (*QemuMutexLockFunc)(QemuMutex *m, const char *f, int l); +typedef int (*QemuMutexTrylockFunc)(QemuMutex *m, const char *f, int l); +typedef void (*QemuRecMutexLockFunc)(QemuRecMutex *m, const char *f, int l= ); +typedef int (*QemuRecMutexTrylockFunc)(QemuRecMutex *m, const char *f, int= l); +typedef void (*QemuCondWaitFunc)(QemuCond *c, QemuMutex *m, const char *f, + int l); + +extern QemuMutexLockFunc qemu_mutex_lock_func; +extern QemuMutexTrylockFunc qemu_mutex_trylock_func; +extern QemuRecMutexLockFunc qemu_rec_mutex_lock_func; +extern QemuRecMutexTrylockFunc qemu_rec_mutex_trylock_func; +extern QemuCondWaitFunc qemu_cond_wait_func; + +/* convenience macros to bypass the profiler */ +#define qemu_mutex_lock__raw(m) \ + qemu_mutex_lock_impl(m, __FILE__, __LINE__) +#define qemu_mutex_trylock__raw(m) \ + qemu_mutex_trylock_impl(m, __FILE__, __LINE__) + +#define qemu_mutex_lock(m) ({ \ + QemuMutexLockFunc _f =3D atomic_read(&qemu_mutex_lock_func); \ + _f(m, __FILE__, __LINE__); \ + }) + +#define qemu_mutex_trylock(m) ({ \ + QemuMutexTrylockFunc _f =3D atomic_read(&qemu_mutex_trylock_fu= nc); \ + _f(m, __FILE__, __LINE__); \ + }) + +#define qemu_rec_mutex_lock(m) ({ \ + QemuRecMutexLockFunc _f =3D atomic_read(&qemu_rec_mutex_lock_f= unc); \ + _f(m, __FILE__, __LINE__); \ + }) + +#define qemu_rec_trymutex_lock(m) ({ \ + QemuRecMutexTrylockFunc _f; \ + _f =3D atomic_read(&qemu_rec_mutex_trylock_func); \ + _f(m, __FILE__, __LINE__); \ + }) + +#define qemu_cond_wait(c, m) ({ \ + QemuCondWaitFunc _f =3D atomic_read(&qemu_cond_wait_func); \ + _f(c, m, __FILE__, __LINE__); \ + }) + #define qemu_mutex_unlock(mutex) \ qemu_mutex_unlock_impl(mutex, __FILE__, __LINE__) =20 @@ -47,6 +91,16 @@ static inline void (qemu_mutex_unlock)(QemuMutex *mutex) qemu_mutex_unlock(mutex); } =20 +static inline void (qemu_rec_mutex_lock)(QemuRecMutex *mutex) +{ + qemu_rec_mutex_lock(mutex); +} + +static inline int (qemu_rec_mutex_trylock)(QemuRecMutex *mutex) +{ + return qemu_rec_mutex_trylock(mutex); +} + /* Prototypes for other functions are in thread-posix.h/thread-win32.h. */ void qemu_rec_mutex_init(QemuRecMutex *mutex); =20 @@ -63,9 +117,6 @@ void qemu_cond_broadcast(QemuCond *cond); void qemu_cond_wait_impl(QemuCond *cond, QemuMutex *mutex, const char *file, const int line); =20 -#define qemu_cond_wait(cond, mutex) \ - qemu_cond_wait_impl(cond, mutex, __FILE__, __LINE__) - static inline void (qemu_cond_wait)(QemuCond *cond, QemuMutex *mutex) { qemu_cond_wait(cond, mutex); diff --git a/util/qemu-thread-win32.c b/util/qemu-thread-win32.c index b303188a36..4a363ca675 100644 --- a/util/qemu-thread-win32.c +++ b/util/qemu-thread-win32.c @@ -97,13 +97,13 @@ void qemu_rec_mutex_destroy(QemuRecMutex *mutex) DeleteCriticalSection(&mutex->lock); } =20 -void qemu_rec_mutex_lock(QemuRecMutex *mutex) +void qemu_rec_mutex_lock_impl(QemuRecMutex *mutex, const char *file, int l= ine) { assert(mutex->initialized); EnterCriticalSection(&mutex->lock); } =20 -int qemu_rec_mutex_trylock(QemuRecMutex *mutex) +int qemu_rec_mutex_trylock_impl(QemuRecMutex *mutex, const char *file, int= line) { assert(mutex->initialized); return !TryEnterCriticalSection(&mutex->lock); diff --git a/util/qht.c b/util/qht.c index c138777a9c..1e3a072e25 100644 --- a/util/qht.c +++ b/util/qht.c @@ -89,6 +89,33 @@ #define QHT_BUCKET_ENTRIES 4 #endif =20 +/* + * Do _not_ use qemu_mutex_[try]lock directly! Use these macros, otherwise + * the profiler (QSP) will deadlock. + */ +static inline void qht_lock(struct qht *ht) +{ + if (ht->mode & QHT_MODE_RAW_MUTEXES) { + qemu_mutex_lock__raw(&ht->lock); + } else { + qemu_mutex_lock(&ht->lock); + } +} + +static inline int qht_trylock(struct qht *ht) +{ + if (ht->mode & QHT_MODE_RAW_MUTEXES) { + return qemu_mutex_trylock__raw(&(ht)->lock); + } + return qemu_mutex_trylock(&(ht)->lock); +} + +/* this inline is not really necessary, but it helps keep code consistent = */ +static inline void qht_unlock(struct qht *ht) +{ + qemu_mutex_unlock(&ht->lock); +} + /* * Note: reading partially-updated pointers in @pointers could lead to * segfaults. We thus access them with atomic_read/set; this guarantees @@ -254,10 +281,10 @@ void qht_map_lock_buckets__no_stale(struct qht *ht, s= truct qht_map **pmap) qht_map_unlock_buckets(map); =20 /* we raced with a resize; acquire ht->lock to see the updated ht->map= */ - qemu_mutex_lock(&ht->lock); + qht_lock(ht); map =3D ht->map; qht_map_lock_buckets(map); - qemu_mutex_unlock(&ht->lock); + qht_unlock(ht); *pmap =3D map; return; } @@ -288,11 +315,11 @@ struct qht_bucket *qht_bucket_lock__no_stale(struct q= ht *ht, uint32_t hash, qemu_spin_unlock(&b->lock); =20 /* we raced with a resize; acquire ht->lock to see the updated ht->map= */ - qemu_mutex_lock(&ht->lock); + qht_lock(ht); map =3D ht->map; b =3D qht_map_to_bucket(map, hash); qemu_spin_lock(&b->lock); - qemu_mutex_unlock(&ht->lock); + qht_unlock(ht); *pmap =3D map; return b; } @@ -430,13 +457,13 @@ bool qht_reset_size(struct qht *ht, size_t n_elems) =20 n_buckets =3D qht_elems_to_buckets(n_elems); =20 - qemu_mutex_lock(&ht->lock); + qht_lock(ht); map =3D ht->map; if (n_buckets !=3D map->n_buckets) { new =3D qht_map_create(n_buckets); } qht_do_resize_and_reset(ht, new); - qemu_mutex_unlock(&ht->lock); + qht_unlock(ht); =20 return !!new; } @@ -565,7 +592,7 @@ static __attribute__((noinline)) void qht_grow_maybe(st= ruct qht *ht) * If the lock is taken it probably means there's an ongoing resize, * so bail out. */ - if (qemu_mutex_trylock(&ht->lock)) { + if (qht_trylock(ht)) { return; } map =3D ht->map; @@ -575,7 +602,7 @@ static __attribute__((noinline)) void qht_grow_maybe(st= ruct qht *ht) =20 qht_do_resize(ht, new); } - qemu_mutex_unlock(&ht->lock); + qht_unlock(ht); } =20 bool qht_insert(struct qht *ht, void *p, uint32_t hash, void **existing) @@ -788,7 +815,7 @@ bool qht_resize(struct qht *ht, size_t n_elems) size_t n_buckets =3D qht_elems_to_buckets(n_elems); size_t ret =3D false; =20 - qemu_mutex_lock(&ht->lock); + qht_lock(ht); if (n_buckets !=3D ht->map->n_buckets) { struct qht_map *new; =20 @@ -796,7 +823,7 @@ bool qht_resize(struct qht *ht, size_t n_elems) qht_do_resize(ht, new); ret =3D true; } - qemu_mutex_unlock(&ht->lock); + qht_unlock(ht); =20 return ret; } diff --git a/util/qsp.c b/util/qsp.c new file mode 100644 index 0000000000..a02c925fe6 --- /dev/null +++ b/util/qsp.c @@ -0,0 +1,584 @@ +/* + * qsp.c - QEMU Synchronization Profiler + * + * Copyright (C) 2018, Emilio G. Cota + * + * License: GNU GPL, version 2 or later. + * See the COPYING file in the top-level directory. + * + * QSP profiles the time spent in synchronization primitives, which can + * help diagnose performance problems, e.g. scalability issues when + * contention is high. + * + * The primitives currently supported are mutexes, recursive mutexes and + * condition variables. Note that not all related functions are intercepte= d; + * instead we profile only those functions that can have a performance imp= act, + * either due to blocking (e.g. cond_wait, mutex_lock) or cache line + * contention (e.g. mutex_lock, mutex_trylock). + * + * QSP's design focuses on speed and scalability. This is achieved + * by having threads do their profiling entirely on thread-local data. + * The appropriate thread-local data is found via a QHT, i.e. a concurrent= hash + * table. To aggregate data in order to generate a report, we iterate over + * all entries in the hash table. Depending on the number of threads and + * synchronization objects this might be expensive, but note that it is + * very rarely called -- reports are generated only when requested by user= s. + * + * Reports are generated as a table where each row represents a call site.= A + * call site is the triplet formed by the __file__ and __LINE__ of the cal= ler + * as well as the address of the "object" (i.e. mutex, rec. mutex or condv= ar) + * being operated on. Focusing on call sites instead of just on objects mi= ght + * seem puzzling. However, it is a sensible choice since otherwise dealing= with + * dynamically-allocated objects becomes difficult (e.g. what to do when an + * object is destroyed, or reused?). Furthermore, the call site info is of= most + * importance, since it is callers, and not objects, what cause wait time. + * + * Alternative designs considered: + * + * - Use an off-the-shelf profiler such as mutrace. This is not a viable o= ption + * for us because QEMU has __malloc_hook set (by one of the libraries it + * uses); leaving this hook unset is required to avoid deadlock in mutra= ce. + * + * - Use a glib HT for each thread, protecting each HT with its own lock. + * This isn't simpler than the current design, and is 10% slower in the + * atomic_add-bench microbenchmark (-m option). + * + * - For reports, just use a binary tree as we aggregate data, instead of = having + * an intermediate hash table. This would simplify the code only slightl= y, but + * would perform badly if there were many threads and objects to track. + * + * Related Work: + * - Lennart Poettering's mutrace: http://0pointer.de/blog/projects/mutrac= e.html + * - Lozi, David, Thomas, Lawall and Muller. "Remote Core Locking: Migrati= ng + * Critical-Section Execution to Improve the Performance of Multithreaded + * Applications", USENIX ATC'12. + */ +#include "qemu/osdep.h" +#include "qemu/thread.h" +#include "qemu/timer.h" +#include "qemu/qht.h" +#include "exec/tb-hash-xx.h" + +enum QSPType { + QSP_MUTEX, + QSP_REC_MUTEX, + QSP_CONDVAR, +}; + +struct QSPCallSite { + const void *obj; + const char *file; /* i.e. __FILE__; shortened later */ + int line; + enum QSPType type; +}; +typedef struct QSPCallSite QSPCallSite; + +struct QSPEntry { + void *thread_ptr; + const QSPCallSite *callsite; + uint64_t n_acqs; + uint64_t ns; +}; +typedef struct QSPEntry QSPEntry; + +/* initial sizing for hash tables */ +#define QSP_INITIAL_SIZE 64 + +/* If this file is moved, QSP_REL_PATH should be updated accordingly */ +#define QSP_REL_PATH "util/qsp.c" + +/* this file's full path. Used to present all call sites with relative pat= hs */ +static size_t qsp_qemu_path_len; + +/* the address of qsp_thread gives us a unique 'thread ID' */ +static __thread int qsp_thread; + +/* + * Call sites are the same for all threads, so we track them in a separate= hash + * table to save memory. + */ +static struct qht qsp_callsite_ht; + +static struct qht qsp_ht; +static bool qsp_initialized, qsp_initializing; + +static const char * const qsp_typenames[] =3D { + [QSP_MUTEX] =3D "mutex", + [QSP_REC_MUTEX] =3D "rec_mutex", + [QSP_CONDVAR] =3D "condvar", +}; + +QemuMutexLockFunc qemu_mutex_lock_func =3D qemu_mutex_lock_impl; +QemuMutexTrylockFunc qemu_mutex_trylock_func =3D qemu_mutex_trylock_impl; +QemuRecMutexLockFunc qemu_rec_mutex_lock_func =3D qemu_rec_mutex_lock_impl; +QemuRecMutexTrylockFunc qemu_rec_mutex_trylock_func =3D + qemu_rec_mutex_trylock_impl; +QemuCondWaitFunc qemu_cond_wait_func =3D qemu_cond_wait_impl; + +/* + * It pays off to _not_ hash callsite->file; hashing a string is slow, and + * without it we still get a pretty unique hash. + */ +static inline +uint32_t do_qsp_callsite_hash(const QSPCallSite *callsite, uint64_t a) +{ + uint64_t b =3D (uint64_t)callsite->obj; + uint32_t e =3D callsite->line; + uint32_t f =3D callsite->type; + + return tb_hash_func7(a, b, e, f, 0); +} + +static inline +uint32_t qsp_callsite_hash(const QSPCallSite *callsite) +{ + return do_qsp_callsite_hash(callsite, 0); +} + +static inline uint32_t do_qsp_entry_hash(const QSPEntry *entry, uint64_t a) +{ + return do_qsp_callsite_hash(entry->callsite, a); +} + +static uint32_t qsp_entry_hash(const QSPEntry *entry) +{ + return do_qsp_entry_hash(entry, (uint64_t)entry->thread_ptr); +} + +static uint32_t qsp_entry_no_thread_hash(const QSPEntry *entry) +{ + return do_qsp_entry_hash(entry, 0); +} + +static bool qsp_callsite_cmp(const void *ap, const void *bp) +{ + const QSPCallSite *a =3D ap; + const QSPCallSite *b =3D bp; + + return a =3D=3D b || + (a->obj =3D=3D b->obj && + a->line =3D=3D b->line && + a->type =3D=3D b->type && + (a->file =3D=3D b->file || !strcmp(a->file, b->file))); +} + +static bool qsp_entry_no_thread_cmp(const void *ap, const void *bp) +{ + const QSPEntry *a =3D ap; + const QSPEntry *b =3D bp; + + return qsp_callsite_cmp(a->callsite, b->callsite); +} + +static bool qsp_entry_cmp(const void *ap, const void *bp) +{ + const QSPEntry *a =3D ap; + const QSPEntry *b =3D bp; + + return a->thread_ptr =3D=3D b->thread_ptr && + qsp_callsite_cmp(a->callsite, b->callsite); +} + +/* + * Normally we'd call this from a constructor function, but we want it to = work + * via libutil as well. + */ +static void qsp_do_init(void) +{ + /* make sure this file's path in the tree is up to date with QSP_REL_P= ATH */ + g_assert(strstr(__FILE__, QSP_REL_PATH)); + qsp_qemu_path_len =3D strlen(__FILE__) - strlen(QSP_REL_PATH); + + qht_init(&qsp_ht, qsp_entry_cmp, QSP_INITIAL_SIZE, + QHT_MODE_AUTO_RESIZE | QHT_MODE_RAW_MUTEXES); + qht_init(&qsp_callsite_ht, qsp_callsite_cmp, QSP_INITIAL_SIZE, + QHT_MODE_AUTO_RESIZE | QHT_MODE_RAW_MUTEXES); +} + +static __attribute__((noinline)) void qsp_init__slowpath(void) +{ + if (atomic_cmpxchg(&qsp_initializing, false, true) =3D=3D false) { + qsp_do_init(); + atomic_set(&qsp_initialized, true); + } else { + while (!atomic_read(&qsp_initialized)) { + cpu_relax(); + } + } +} + +/* qsp_init() must be called from _all_ exported functions */ +static inline void qsp_init(void) +{ + if (likely(atomic_read(&qsp_initialized))) { + return; + } + qsp_init__slowpath(); +} + +static QSPCallSite *qsp_callsite_find(const QSPCallSite *orig) +{ + QSPCallSite *callsite; + uint32_t hash; + + hash =3D qsp_callsite_hash(orig); + callsite =3D qht_lookup(&qsp_callsite_ht, orig, hash); + if (callsite =3D=3D NULL) { + void *existing =3D NULL; + + callsite =3D g_new(QSPCallSite, 1); + memcpy(callsite, orig, sizeof(*callsite)); + qht_insert(&qsp_callsite_ht, callsite, hash, &existing); + if (unlikely(existing)) { + g_free(callsite); + callsite =3D existing; + } + } + return callsite; +} + +static QSPEntry * +qsp_entry_create(struct qht *ht, const QSPEntry *entry, uint32_t hash) +{ + QSPEntry *e; + void *existing =3D NULL; + + e =3D g_new0(QSPEntry, 1); + e->thread_ptr =3D entry->thread_ptr; + e->callsite =3D qsp_callsite_find(entry->callsite); + + qht_insert(ht, e, hash, &existing); + if (unlikely(existing)) { + g_free(e); + e =3D existing; + } + return e; +} + +static QSPEntry * +qsp_entry_find(struct qht *ht, const QSPEntry *entry, uint32_t hash) +{ + QSPEntry *e; + + e =3D qht_lookup(ht, entry, hash); + if (e =3D=3D NULL) { + e =3D qsp_entry_create(ht, entry, hash); + } + return e; +} + +/* + * Note: Entries are never removed, so callers do not have to be in an RCU + * read-side critical section. + */ +static QSPEntry *qsp_entry_get(const void *obj, const char *file, int line, + enum QSPType type) +{ + QSPCallSite callsite =3D { + .obj =3D obj, + .file =3D file, + .line =3D line, + .type =3D type, + }; + QSPEntry orig; + uint32_t hash; + + qsp_init(); + + orig.thread_ptr =3D &qsp_thread; + orig.callsite =3D &callsite; + + hash =3D qsp_entry_hash(&orig); + return qsp_entry_find(&qsp_ht, &orig, hash); +} + +#define QSP_GEN_VOID(type_, qsp_t_, func_, impl_) \ + static void func_(type_ *obj, const char *file, int line) \ + { \ + QSPEntry *e; \ + int64_t t0, t1; \ + \ + t0 =3D get_clock(); \ + impl_(obj, file, line); \ + t1 =3D get_clock(); \ + \ + e =3D qsp_entry_get(obj, file, line, qsp_t_); \ + atomic_set(&e->ns, e->ns + t1 - t0); \ + atomic_set(&e->n_acqs, e->n_acqs + 1); \ + } + +#define QSP_GEN_RET1(type_, qsp_t_, func_, impl_) \ + static int func_(type_ *obj, const char *file, int line) \ + { \ + QSPEntry *e; \ + int64_t t0, t1; \ + int err; \ + \ + t0 =3D get_clock(); \ + err =3D impl_(obj, file, line); \ + t1 =3D get_clock(); \ + \ + e =3D qsp_entry_get(obj, file, line, qsp_t_); \ + atomic_set(&e->ns, e->ns + t1 - t0); \ + if (!err) { \ + atomic_set(&e->n_acqs, e->n_acqs + 1); \ + } \ + return err; \ + } + +QSP_GEN_VOID(QemuMutex, QSP_MUTEX, qsp_mutex_lock, qemu_mutex_lock_impl) +QSP_GEN_RET1(QemuMutex, QSP_MUTEX, qsp_mutex_trylock, qemu_mutex_trylock_i= mpl) + +QSP_GEN_VOID(QemuRecMutex, QSP_REC_MUTEX, qsp_rec_mutex_lock, + qemu_rec_mutex_lock_impl) +QSP_GEN_RET1(QemuRecMutex, QSP_REC_MUTEX, qsp_rec_mutex_trylock, + qemu_rec_mutex_trylock_impl) + +#undef QSP_GEN_RET1 +#undef QSP_GEN_VOID + +static void +qsp_cond_wait(QemuCond *cond, QemuMutex *mutex, const char *file, int line) +{ + QSPEntry *e; + int64_t t0, t1; + + t0 =3D get_clock(); + qemu_cond_wait_impl(cond, mutex, file, line); + t1 =3D get_clock(); + + e =3D qsp_entry_get(cond, file, line, QSP_CONDVAR); + atomic_set(&e->ns, e->ns + t1 - t0); + atomic_set(&e->n_acqs, e->n_acqs + 1); +} + +bool qsp_is_enabled(void) +{ + return atomic_read(&qemu_mutex_lock_func) =3D=3D qsp_mutex_lock; +} + +void qsp_enable(void) +{ + atomic_set(&qemu_mutex_lock_func, qsp_mutex_lock); + atomic_set(&qemu_mutex_trylock_func, qsp_mutex_trylock); + atomic_set(&qemu_rec_mutex_lock_func, qsp_rec_mutex_lock); + atomic_set(&qemu_rec_mutex_trylock_func, qsp_rec_mutex_trylock); + atomic_set(&qemu_cond_wait_func, qsp_cond_wait); +} + +void qsp_disable(void) +{ + atomic_set(&qemu_mutex_lock_func, qemu_mutex_lock_impl); + atomic_set(&qemu_mutex_trylock_func, qemu_mutex_trylock_impl); + atomic_set(&qemu_rec_mutex_lock_func, qemu_rec_mutex_lock_impl); + atomic_set(&qemu_rec_mutex_trylock_func, qemu_rec_mutex_trylock_impl); + atomic_set(&qemu_cond_wait_func, qemu_cond_wait_impl); +} + +static gint qsp_tree_cmp(gconstpointer ap, gconstpointer bp, gpointer up) +{ + const QSPEntry *a =3D ap; + const QSPEntry *b =3D bp; + const QSPCallSite *ca; + const QSPCallSite *cb; + + if (a->ns > b->ns) { + return -1; + } else if (a->ns < b->ns) { + return 1; + } + ca =3D a->callsite; + cb =3D b->callsite; + /* Break the tie with the object's address */ + if (ca->obj < cb->obj) { + return -1; + } else if (ca->obj > cb->obj) { + return 1; + } else { + int cmp; + + /* same obj. Break the tie with the callsite's file */ + cmp =3D strcmp(ca->file, cb->file); + if (cmp) { + return cmp; + } + /* same callsite file. Break the tie with the callsite's line */ + g_assert(ca->line !=3D cb->line); + if (ca->line < cb->line) { + return -1; + } else if (ca->line > cb->line) { + return 1; + } else { + /* break the tie with the callsite's type */ + return cb->type - ca->type; + } + } +} + +static void qsp_sort(struct qht *ht, void *p, uint32_t h, void *userp) +{ + QSPEntry *e =3D p; + GTree *tree =3D userp; + + g_tree_insert(tree, e, NULL); +} + +static void qsp_aggregate(struct qht *global_ht, void *p, uint32_t h, void= *up) +{ + struct qht *ht =3D up; + const QSPEntry *e =3D p; + QSPEntry *agg; + uint32_t hash; + + hash =3D qsp_entry_no_thread_hash(e); + agg =3D qsp_entry_find(ht, e, hash); + agg->ns +=3D e->ns; + agg->n_acqs +=3D e->n_acqs; +} + +static void qsp_mktree(GTree *tree) +{ + struct qht ht; + + /* Aggregate all results from the global hash table into a local one */ + qht_init(&ht, qsp_entry_no_thread_cmp, QSP_INITIAL_SIZE, + QHT_MODE_AUTO_RESIZE | QHT_MODE_RAW_MUTEXES); + qht_iter(&qsp_ht, qsp_aggregate, &ht); + + /* sort the hash table elements by using a tree */ + qht_iter(&ht, qsp_sort, tree); + + /* free the hash table, but keep the elements (those are in the tree n= ow) */ + qht_destroy(&ht); +} + +/* free string with g_free */ +static char *qsp_at(const QSPCallSite *callsite) +{ + GString *s =3D g_string_new(NULL); + const char *shortened; + + /* remove the absolute path to qemu */ + if (unlikely(strlen(callsite->file) < qsp_qemu_path_len)) { + shortened =3D callsite->file; + } else { + shortened =3D callsite->file + qsp_qemu_path_len; + } + g_string_append_printf(s, "%s:%u", shortened, callsite->line); + return g_string_free(s, FALSE); +} + +struct QSPReportEntry { + const void *obj; + char *callsite_at; + const char *typename; + double time_s; + double ns_avg; + uint64_t n_acqs; +}; +typedef struct QSPReportEntry QSPReportEntry; + +struct QSPReport { + QSPReportEntry *entries; + size_t n_entries; + size_t max_n_entries; +}; +typedef struct QSPReport QSPReport; + +static gboolean qsp_tree_report(gpointer key, gpointer value, gpointer uda= ta) +{ + const QSPEntry *e =3D key; + QSPReport *report =3D udata; + QSPReportEntry *entry; + + if (report->n_entries =3D=3D report->max_n_entries) { + return TRUE; + } + entry =3D &report->entries[report->n_entries]; + report->n_entries++; + + entry->obj =3D e->callsite->obj; + entry->callsite_at =3D qsp_at(e->callsite); + entry->typename =3D qsp_typenames[e->callsite->type]; + entry->time_s =3D e->ns * 1e-9; + entry->n_acqs =3D e->n_acqs; + entry->ns_avg =3D e->n_acqs ? e->ns / e->n_acqs : 0; + return FALSE; +} + +static void +pr_report(const QSPReport *rep, FILE *f, fprintf_function pr) +{ + char *dashes; + size_t max_len =3D 0; + int callsite_len =3D 0; + int callsite_rspace; + int n_dashes; + size_t i; + + /* find out the maximum length of all 'callsite' fields */ + for (i =3D 0; i < rep->n_entries; i++) { + const QSPReportEntry *e =3D &rep->entries[i]; + size_t len =3D strlen(e->callsite_at); + + if (len > max_len) { + max_len =3D len; + } + } + + callsite_len =3D MAX(max_len, strlen("Call site")); + /* white space to leave to the right of "Call site" */ + callsite_rspace =3D callsite_len - strlen("Call site"); + + pr(f, "Type Object Call site%*s Wait Time (s) " + " Count Average (us)\n", callsite_rspace, ""); + + /* build a horizontal rule with dashes */ + n_dashes =3D 79 + callsite_rspace; + dashes =3D g_malloc(n_dashes + 1); + memset(dashes, '-', n_dashes); + dashes[n_dashes] =3D '\0'; + pr(f, "%s\n", dashes); + + for (i =3D 0; i < rep->n_entries; i++) { + const QSPReportEntry *e =3D &rep->entries[i]; + + pr(f, "%-9s %14p %s%*s %13.5f %12" PRIu64 " %12.2f\n", e->typ= ename, + e->obj, e->callsite_at, callsite_len - (int)strlen(e->callsite_= at), + "", e->time_s, e->n_acqs, e->ns_avg * 1e-3); + } + + pr(f, "%s\n", dashes); + g_free(dashes); +} + +static void report_destroy(QSPReport *rep) +{ + size_t i; + + for (i =3D 0; i < rep->n_entries; i++) { + QSPReportEntry *e =3D &rep->entries[i]; + + g_free(e->callsite_at); + } + g_free(rep->entries); +} + +void qsp_report(FILE *f, fprintf_function cpu_fprintf, size_t max) +{ + GTree *tree =3D g_tree_new_full(qsp_tree_cmp, NULL, g_free, NULL); + QSPReport rep; + + qsp_init(); + + rep.entries =3D g_new0(QSPReportEntry, max); + rep.n_entries =3D 0; + rep.max_n_entries =3D max; + + qsp_mktree(tree); + g_tree_foreach(tree, qsp_tree_report, &rep); + g_tree_destroy(tree); + + pr_report(&rep, f, cpu_fprintf); + report_destroy(&rep); +} diff --git a/util/Makefile.objs b/util/Makefile.objs index e1c3fed4dc..e958116c86 100644 --- a/util/Makefile.objs +++ b/util/Makefile.objs @@ -44,6 +44,7 @@ util-obj-y +=3D log.o util-obj-y +=3D pagesize.o util-obj-y +=3D qdist.o util-obj-y +=3D qht.o +util-obj-y +=3D qsp.o util-obj-y +=3D range.o util-obj-y +=3D stats64.o util-obj-y +=3D systemd.o --=20 2.17.1 From nobody Wed Nov 5 07:16:03 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1534483246528755.0977858559467; Thu, 16 Aug 2018 22:20:46 -0700 (PDT) Received: from localhost ([::1]:59990 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fqXBj-0003Mv-FH for importer@patchew.org; Fri, 17 Aug 2018 01:20:43 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:36279) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fqXA8-0000d8-OR for qemu-devel@nongnu.org; Fri, 17 Aug 2018 01:19:05 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fqXA5-0005Yd-F3 for qemu-devel@nongnu.org; Fri, 17 Aug 2018 01:19:04 -0400 Received: from out1-smtp.messagingengine.com ([66.111.4.25]:49223) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fqXA2-0005VS-Is for qemu-devel@nongnu.org; Fri, 17 Aug 2018 01:18:59 -0400 Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.nyi.internal (Postfix) with ESMTP id 11CAC21BE6; Fri, 17 Aug 2018 01:18:57 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute4.internal (MEProxy); Fri, 17 Aug 2018 01:18:57 -0400 Received: from localhost (flamenco.cs.columbia.edu [128.59.20.216]) by mail.messagingengine.com (Postfix) with ESMTPA id 7989310293; Fri, 17 Aug 2018 01:18:56 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=braap.org; h=cc :date:from:in-reply-to:message-id:references:subject:to :x-me-sender:x-me-sender:x-sasl-enc; s=mesmtp; bh=9SBl3tTN/ua/8h KpdVVXJi6PTTvwW0kwifqPGHJwFsU=; b=fLwqSsbysxsNFLLFP4KDTz6QRAMvA7 RqL+e/uvYPTtzk+2vDteRrDOYREkzS6KSl3RIU3C4DGwUP3WFa8kHu9937KnKv2j GiFKtx8QscmxrnQ29Jettgrq/NCxwWu11ekCLg5IYSLj+FlM70DI/G7sv51Cl2aV k/tL41hrthJ+I= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:date:from:in-reply-to:message-id :references:subject:to:x-me-sender:x-me-sender:x-sasl-enc; s= fm3; bh=9SBl3tTN/ua/8hKpdVVXJi6PTTvwW0kwifqPGHJwFsU=; b=DENvjveF bGzqcRFZhRs3HC+PE81XGbFPazwix+Hr/BfG1EBK6JuftZ9y2dlY5lSPKDzL2wBj GjYO4xHMBYa0IYdKcbGkhh1F64cd/b1opgjtvo1KGQKEv6wvo4xEmjkeW11OG6Ji Ja28BdbB8x7vPLUT2U5Y81X4e4d9nYaCnRBtsg2PxfIsvbyLVjKBkgJrlfwKc+dw A359xOQ8SlV5sPMK6M+KOxu0gfxBbp2BLhZpEKDw/kd6OqwjlAgxlj7JVJto+rYe KTxWBIDOD0sHE4BGVIrLZ52+WA/XbXPDZOv49IzMoUA5mEzukKdz+CYdnOxrBKSc mM/tFHjhFCkC2Q== X-ME-Proxy: X-ME-Sender: From: "Emilio G. Cota" To: qemu-devel@nongnu.org Date: Fri, 17 Aug 2018 01:18:46 -0400 Message-Id: <20180817051853.23792-3-cota@braap.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180817051853.23792-1-cota@braap.org> References: <20180817051853.23792-1-cota@braap.org> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 66.111.4.25 Subject: [Qemu-devel] [PATCH 2/9] qsp: add sort_by option to qsp_report X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , Peter Crosthwaite , Stefan Weil , "Dr. David Alan Gilbert" , Peter Xu , Markus Armbruster , Paolo Bonzini , Richard Henderson Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Signed-off-by: Emilio G. Cota --- include/qemu/qsp.h | 8 +++++++- util/qsp.c | 33 +++++++++++++++++++++++++++------ 2 files changed, 34 insertions(+), 7 deletions(-) diff --git a/include/qemu/qsp.h b/include/qemu/qsp.h index 9c2bb60ff0..209480b687 100644 --- a/include/qemu/qsp.h +++ b/include/qemu/qsp.h @@ -13,7 +13,13 @@ =20 #include "qemu/fprintf-fn.h" =20 -void qsp_report(FILE *f, fprintf_function cpu_fprintf, size_t max); +enum QSPSortBy { + QSP_SORT_BY_TOTAL_WAIT_TIME, + QSP_SORT_BY_AVG_WAIT_TIME, +}; + +void qsp_report(FILE *f, fprintf_function cpu_fprintf, size_t max, + enum QSPSortBy sort_by); =20 bool qsp_is_enabled(void); void qsp_enable(void); diff --git a/util/qsp.c b/util/qsp.c index a02c925fe6..b04d4d9986 100644 --- a/util/qsp.c +++ b/util/qsp.c @@ -379,14 +379,34 @@ static gint qsp_tree_cmp(gconstpointer ap, gconstpoin= ter bp, gpointer up) { const QSPEntry *a =3D ap; const QSPEntry *b =3D bp; + enum QSPSortBy sort_by =3D *(enum QSPSortBy *)up; const QSPCallSite *ca; const QSPCallSite *cb; =20 - if (a->ns > b->ns) { - return -1; - } else if (a->ns < b->ns) { - return 1; + switch (sort_by) { + case QSP_SORT_BY_TOTAL_WAIT_TIME: + if (a->ns > b->ns) { + return -1; + } else if (a->ns < b->ns) { + return 1; + } + break; + case QSP_SORT_BY_AVG_WAIT_TIME: + { + double avg_a =3D a->n_acqs ? a->ns / a->n_acqs : 0; + double avg_b =3D b->n_acqs ? b->ns / b->n_acqs : 0; + + if (avg_a > avg_b) { + return -1; + } else if (avg_a < avg_b) { + return 1; + } + break; } + default: + g_assert_not_reached(); + } + ca =3D a->callsite; cb =3D b->callsite; /* Break the tie with the object's address */ @@ -564,9 +584,10 @@ static void report_destroy(QSPReport *rep) g_free(rep->entries); } =20 -void qsp_report(FILE *f, fprintf_function cpu_fprintf, size_t max) +void qsp_report(FILE *f, fprintf_function cpu_fprintf, size_t max, + enum QSPSortBy sort_by) { - GTree *tree =3D g_tree_new_full(qsp_tree_cmp, NULL, g_free, NULL); + GTree *tree =3D g_tree_new_full(qsp_tree_cmp, &sort_by, g_free, NULL); QSPReport rep; =20 qsp_init(); --=20 2.17.1 From nobody Wed Nov 5 07:16:03 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 1534483621143479.1645049916465; Thu, 16 Aug 2018 22:27:01 -0700 (PDT) Received: from localhost ([::1]:60026 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fqXHj-0001ET-2v for importer@patchew.org; Fri, 17 Aug 2018 01:26:55 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:36294) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fqXA9-0000dG-2f for qemu-devel@nongnu.org; Fri, 17 Aug 2018 01:19:06 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fqXA7-0005bR-Ic for qemu-devel@nongnu.org; Fri, 17 Aug 2018 01:19:05 -0400 Received: from out1-smtp.messagingengine.com ([66.111.4.25]:34387) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fqXA6-0005Yn-LV for qemu-devel@nongnu.org; Fri, 17 Aug 2018 01:19:03 -0400 Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.nyi.internal (Postfix) with ESMTP id 895A721EC1; Fri, 17 Aug 2018 01:18:57 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute4.internal (MEProxy); Fri, 17 Aug 2018 01:18:57 -0400 Received: from localhost (flamenco.cs.columbia.edu [128.59.20.216]) by mail.messagingengine.com (Postfix) with ESMTPA id B3F1510297; Fri, 17 Aug 2018 01:18:56 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=braap.org; h=cc :date:from:in-reply-to:message-id:references:subject:to :x-me-sender:x-me-sender:x-sasl-enc; s=mesmtp; bh=byMd+o0Jp/ywwZ LpbPJAPsa5i7q+vRppt9tjeaA9qa0=; b=TNea83UZ0KhqtCdBLX7eGAv0zpTO4w wesSNCeTXe5ZUpS1O/xCGbuo3oz7vq9RP/aGMeRytHy0+NKO9kJNzkqrApkDwUkc r7USK21OYkyh9bCr2S5ydVOEb2rTQHpm7h7t7F8EhiIjQi+zJq019MdpAB1iYgYn r15zB8roVZWkU= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:date:from:in-reply-to:message-id :references:subject:to:x-me-sender:x-me-sender:x-sasl-enc; s= fm3; bh=byMd+o0Jp/ywwZLpbPJAPsa5i7q+vRppt9tjeaA9qa0=; b=hbUOt7T+ 9ZzsDmbIdtz6nLgJHsgJDPKSl9ht4GFUcPRoLEpSpzTB9IYgaBSKU2nK2u4wDb2D 9Md/B14ZXAJJBDPmwAO98vfE0rtmdLM97OTZVLHHaVNrCW2I2YRkJT7WytgA4WR7 IddZPJTvN5pB9M/yFcGbQfOQo5FfjOj/vigzWvhomN/IayBaClzcY++EkRJJ6VaY N3VYtPqtBmBDz2fvqovrQ6sG3UtLkufae/QCwK4Q6PJ5PlqJ+n5nBzEbsnJXWPPu auXgbrvzB+oqIBhCp4XfZfwqUs7OhVptnsip9kxMYLEQWbaZCWDLI0H0vbCexxz2 9BetpKW8bkhvlA== X-ME-Proxy: X-ME-Sender: From: "Emilio G. Cota" To: qemu-devel@nongnu.org Date: Fri, 17 Aug 2018 01:18:47 -0400 Message-Id: <20180817051853.23792-4-cota@braap.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180817051853.23792-1-cota@braap.org> References: <20180817051853.23792-1-cota@braap.org> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 66.111.4.25 Subject: [Qemu-devel] [PATCH 3/9] qsp: add qsp_reset X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , Peter Crosthwaite , Stefan Weil , "Dr. David Alan Gilbert" , Peter Xu , Markus Armbruster , Paolo Bonzini , Richard Henderson Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" I first implemented this by deleting all entries in the global hash table. But doing that safely slows down profiling, since we'd need to introduce rcu_read_lock/unlock in the fast path. What's implemented here avoids messing with the thread-local data in the global hash table. It achieves this by taking a snapshot of the current state, so that subsequent reports present the delta wrt to the snapshot. Signed-off-by: Emilio G. Cota --- include/qemu/qsp.h | 1 + util/qsp.c | 94 ++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 95 insertions(+) diff --git a/include/qemu/qsp.h b/include/qemu/qsp.h index 209480b687..f8c6c9648a 100644 --- a/include/qemu/qsp.h +++ b/include/qemu/qsp.h @@ -24,5 +24,6 @@ void qsp_report(FILE *f, fprintf_function cpu_fprintf, si= ze_t max, bool qsp_is_enabled(void); void qsp_enable(void); void qsp_disable(void); +void qsp_reset(void); =20 #endif /* QEMU_QSP_H */ diff --git a/util/qsp.c b/util/qsp.c index b04d4d9986..ea4bb82a03 100644 --- a/util/qsp.c +++ b/util/qsp.c @@ -47,6 +47,11 @@ * an intermediate hash table. This would simplify the code only slightl= y, but * would perform badly if there were many threads and objects to track. * + * - Wrap operations on qsp entries with RCU read-side critical sections, = so + * that qsp_reset() can delete entries. Unfortunately, the overhead of c= alling + * rcu_read_lock/unlock slows down atomic_add-bench -m by 24%. Having + * a snapshot that is updated on qsp_reset() avoids this overhead. + * * Related Work: * - Lennart Poettering's mutrace: http://0pointer.de/blog/projects/mutrac= e.html * - Lozi, David, Thomas, Lawall and Muller. "Remote Core Locking: Migrati= ng @@ -57,6 +62,7 @@ #include "qemu/thread.h" #include "qemu/timer.h" #include "qemu/qht.h" +#include "qemu/rcu.h" #include "exec/tb-hash-xx.h" =20 enum QSPType { @@ -81,6 +87,12 @@ struct QSPEntry { }; typedef struct QSPEntry QSPEntry; =20 +struct QSPSnapshot { + struct rcu_head rcu; + struct qht ht; +}; +typedef struct QSPSnapshot QSPSnapshot; + /* initial sizing for hash tables */ #define QSP_INITIAL_SIZE 64 =20 @@ -100,6 +112,7 @@ static __thread int qsp_thread; static struct qht qsp_callsite_ht; =20 static struct qht qsp_ht; +static QSPSnapshot *qsp_snapshot; static bool qsp_initialized, qsp_initializing; =20 static const char * const qsp_typenames[] =3D { @@ -456,15 +469,69 @@ static void qsp_aggregate(struct qht *global_ht, void= *p, uint32_t h, void *up) agg->n_acqs +=3D e->n_acqs; } =20 +static void qsp_iter_diff(struct qht *orig, void *p, uint32_t hash, void *= htp) +{ + struct qht *ht =3D htp; + QSPEntry *old =3D p; + QSPEntry *new; + + new =3D qht_lookup(ht, old, hash); + /* entries are never deleted, so we must have this one */ + g_assert(new !=3D NULL); + /* our reading of the stats happened after the snapshot was taken */ + g_assert(new->n_acqs >=3D old->n_acqs); + g_assert(new->ns >=3D old->ns); + + new->n_acqs -=3D old->n_acqs; + new->ns -=3D old->ns; + + /* No point in reporting an empty entry */ + if (new->n_acqs =3D=3D 0 && new->ns =3D=3D 0) { + bool removed =3D qht_remove(ht, new, hash); + + g_assert(removed); + g_free(new); + } +} + +static void qsp_diff(struct qht *orig, struct qht *new) +{ + qht_iter(orig, qsp_iter_diff, new); +} + +static void qsp_ht_delete(struct qht *ht, void *p, uint32_t h, void *htp) +{ + g_free(p); +} + static void qsp_mktree(GTree *tree) { + QSPSnapshot *snap; struct qht ht; =20 + /* + * First, see if there's a prior snapshot, so that we read the global = hash + * table _after_ the snapshot has been created, which guarantees that + * the entries we'll read will be a superset of the snapshot's entries. + * + * We must remain in an RCU read-side critical section until we're done + * with the snapshot. + */ + rcu_read_lock(); + snap =3D atomic_rcu_read(&qsp_snapshot); + /* Aggregate all results from the global hash table into a local one */ qht_init(&ht, qsp_entry_no_thread_cmp, QSP_INITIAL_SIZE, QHT_MODE_AUTO_RESIZE | QHT_MODE_RAW_MUTEXES); qht_iter(&qsp_ht, qsp_aggregate, &ht); =20 + /* compute the difference wrt the snapshot, if any */ + if (snap) { + qsp_diff(&snap->ht, &ht); + } + /* done with the snapshot; RCU can reclaim it */ + rcu_read_unlock(); + /* sort the hash table elements by using a tree */ qht_iter(&ht, qsp_sort, tree); =20 @@ -603,3 +670,30 @@ void qsp_report(FILE *f, fprintf_function cpu_fprintf,= size_t max, pr_report(&rep, f, cpu_fprintf); report_destroy(&rep); } + +static void qsp_snapshot_destroy(QSPSnapshot *snap) +{ + qht_iter(&snap->ht, qsp_ht_delete, NULL); + qht_destroy(&snap->ht); + g_free(snap); +} + +void qsp_reset(void) +{ + QSPSnapshot *new =3D g_new(QSPSnapshot, 1); + QSPSnapshot *old; + + qsp_init(); + + qht_init(&new->ht, qsp_entry_cmp, QSP_INITIAL_SIZE, + QHT_MODE_AUTO_RESIZE | QHT_MODE_RAW_MUTEXES); + + /* take a snapshot of the current state */ + qht_iter(&qsp_ht, qsp_aggregate, &new->ht); + + /* replace the previous snapshot, if any */ + old =3D atomic_xchg(&qsp_snapshot, new); + if (old) { + call_rcu(old, qsp_snapshot_destroy, rcu); + } +} --=20 2.17.1 From nobody Wed Nov 5 07:16:03 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 153448346951694.16797795299897; Thu, 16 Aug 2018 22:24:29 -0700 (PDT) Received: from localhost ([::1]:60011 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fqXFM-00064e-9d for importer@patchew.org; Fri, 17 Aug 2018 01:24:28 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:36285) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fqXA8-0000dD-Oy for qemu-devel@nongnu.org; Fri, 17 Aug 2018 01:19:06 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fqXA5-0005Yo-FV for qemu-devel@nongnu.org; Fri, 17 Aug 2018 01:19:04 -0400 Received: from out1-smtp.messagingengine.com ([66.111.4.25]:48295) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fqXA3-0005Vw-Du for qemu-devel@nongnu.org; Fri, 17 Aug 2018 01:18:59 -0400 Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.nyi.internal (Postfix) with ESMTP id 8955021E56; Fri, 17 Aug 2018 01:18:57 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute4.internal (MEProxy); Fri, 17 Aug 2018 01:18:57 -0400 Received: from localhost (flamenco.cs.columbia.edu [128.59.20.216]) by mail.messagingengine.com (Postfix) with ESMTPA id E87EC10299; Fri, 17 Aug 2018 01:18:56 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=braap.org; h=cc :date:from:in-reply-to:message-id:references:subject:to :x-me-sender:x-me-sender:x-sasl-enc; s=mesmtp; bh=uSJeSYH0b6DA+s Bvn0BXVOE/UALA+7BHr0cJh+lPt9E=; b=FYYL0WO8FOuHY4zL8mhxpKKCerksBB 10Le/1br2LABtOcNaRViNx3pdA0zpxJBZLS27L2RDaasgnHHZ9BbkbB+n7aMaaLx wJ431yy9vIEAuhFWLcKNzw11TV1mGIFk1aQZHjJKotYwC3hesPLgnTogyAXXfb5y TeElF2yzdHfa8= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:date:from:in-reply-to:message-id :references:subject:to:x-me-sender:x-me-sender:x-sasl-enc; s= fm3; bh=uSJeSYH0b6DA+sBvn0BXVOE/UALA+7BHr0cJh+lPt9E=; b=ExXKmbWG QtpPFeUcwqd0EBYvsS3E11Am5zLKKT+9biukaUy7BGtsJA9vssc5Wn43t1KjsaIB lSYEnO/wdAtiVTSYEXmx3Z3Afk8d7+Plo135oZQz0PhzAn/PpXxNETxXF3Bs25ml LR5A0sFQzGP7gt2gEV8gyHIwWFtHJaV2Toi6AMNK7+Lz/c5jLX9hCPOj14/JRyRW YD6jLe4+c7m/l3+ir0CJgB89OJ19PItFG1+H03vDk158SnvAzJBgPCH0RAiX9MBm i2e4yXaEv/HOjSKRYsCryPnYxJVFJttlSg68FgecUacUKRCeGr42BG1h1uMpj+LA ciLY3SoyXzIG8Q== X-ME-Proxy: X-ME-Sender: From: "Emilio G. Cota" To: qemu-devel@nongnu.org Date: Fri, 17 Aug 2018 01:18:48 -0400 Message-Id: <20180817051853.23792-5-cota@braap.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180817051853.23792-1-cota@braap.org> References: <20180817051853.23792-1-cota@braap.org> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 66.111.4.25 Subject: [Qemu-devel] [PATCH 4/9] qsp: support call site coalescing X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , Peter Crosthwaite , Stefan Weil , "Dr. David Alan Gilbert" , Peter Xu , Markus Armbruster , Paolo Bonzini , Richard Henderson Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Signed-off-by: Emilio G. Cota --- include/qemu/qsp.h | 2 +- util/qsp.c | 102 ++++++++++++++++++++++++++++++++++++++------- 2 files changed, 89 insertions(+), 15 deletions(-) diff --git a/include/qemu/qsp.h b/include/qemu/qsp.h index f8c6c9648a..a94c464f90 100644 --- a/include/qemu/qsp.h +++ b/include/qemu/qsp.h @@ -19,7 +19,7 @@ enum QSPSortBy { }; =20 void qsp_report(FILE *f, fprintf_function cpu_fprintf, size_t max, - enum QSPSortBy sort_by); + enum QSPSortBy sort_by, bool callsite_coalesce); =20 bool qsp_is_enabled(void); void qsp_enable(void); diff --git a/util/qsp.c b/util/qsp.c index ea4bb82a03..ea128d3748 100644 --- a/util/qsp.c +++ b/util/qsp.c @@ -27,11 +27,9 @@ * Reports are generated as a table where each row represents a call site.= A * call site is the triplet formed by the __file__ and __LINE__ of the cal= ler * as well as the address of the "object" (i.e. mutex, rec. mutex or condv= ar) - * being operated on. Focusing on call sites instead of just on objects mi= ght - * seem puzzling. However, it is a sensible choice since otherwise dealing= with - * dynamically-allocated objects becomes difficult (e.g. what to do when an - * object is destroyed, or reused?). Furthermore, the call site info is of= most - * importance, since it is callers, and not objects, what cause wait time. + * being operated on. Optionally, call sites that operate on different obj= ects + * of the same type can be coalesced, which can be particularly useful when + * profiling dynamically-allocated objects. * * Alternative designs considered: * @@ -84,6 +82,7 @@ struct QSPEntry { const QSPCallSite *callsite; uint64_t n_acqs; uint64_t ns; + unsigned int n_objs; /* count of coalesced objs; only used for reporti= ng */ }; typedef struct QSPEntry QSPEntry; =20 @@ -163,6 +162,17 @@ static uint32_t qsp_entry_no_thread_hash(const QSPEntr= y *entry) return do_qsp_entry_hash(entry, 0); } =20 +/* without the objects we need to hash the file name to get a decent hash = */ +static uint32_t qsp_entry_no_thread_obj_hash(const QSPEntry *entry) +{ + const QSPCallSite *callsite =3D entry->callsite; + uint64_t a =3D g_str_hash(callsite->file); + uint64_t b =3D callsite->line; + uint32_t e =3D callsite->type; + + return tb_hash_func7(a, b, e, 0, 0); +} + static bool qsp_callsite_cmp(const void *ap, const void *bp) { const QSPCallSite *a =3D ap; @@ -175,6 +185,17 @@ static bool qsp_callsite_cmp(const void *ap, const voi= d *bp) (a->file =3D=3D b->file || !strcmp(a->file, b->file))); } =20 +static bool qsp_callsite_no_obj_cmp(const void *ap, const void *bp) +{ + const QSPCallSite *a =3D ap; + const QSPCallSite *b =3D bp; + + return a =3D=3D b || + (a->line =3D=3D b->line && + a->type =3D=3D b->type && + (a->file =3D=3D b->file || !strcmp(a->file, b->file))); +} + static bool qsp_entry_no_thread_cmp(const void *ap, const void *bp) { const QSPEntry *a =3D ap; @@ -183,6 +204,14 @@ static bool qsp_entry_no_thread_cmp(const void *ap, co= nst void *bp) return qsp_callsite_cmp(a->callsite, b->callsite); } =20 +static bool qsp_entry_no_thread_obj_cmp(const void *ap, const void *bp) +{ + const QSPEntry *a =3D ap; + const QSPEntry *b =3D bp; + + return qsp_callsite_no_obj_cmp(a->callsite, b->callsite); +} + static bool qsp_entry_cmp(const void *ap, const void *bp) { const QSPEntry *a =3D ap; @@ -499,15 +528,36 @@ static void qsp_diff(struct qht *orig, struct qht *ne= w) qht_iter(orig, qsp_iter_diff, new); } =20 +static void +qsp_iter_callsite_coalesce(struct qht *orig, void *p, uint32_t h, void *ht= p) +{ + struct qht *ht =3D htp; + QSPEntry *old =3D p; + QSPEntry *e; + uint32_t hash; + + hash =3D qsp_entry_no_thread_obj_hash(old); + e =3D qht_lookup(ht, old, hash); + if (e =3D=3D NULL) { + e =3D qsp_entry_create(ht, old, hash); + e->n_objs =3D 1; + } else if (e->callsite->obj !=3D old->callsite->obj) { + e->n_objs++; + } + e->ns +=3D old->ns; + e->n_acqs +=3D old->n_acqs; +} + static void qsp_ht_delete(struct qht *ht, void *p, uint32_t h, void *htp) { g_free(p); } =20 -static void qsp_mktree(GTree *tree) +static void qsp_mktree(GTree *tree, bool callsite_coalesce) { QSPSnapshot *snap; - struct qht ht; + struct qht ht, coalesce_ht; + struct qht *htp; =20 /* * First, see if there's a prior snapshot, so that we read the global = hash @@ -532,11 +582,23 @@ static void qsp_mktree(GTree *tree) /* done with the snapshot; RCU can reclaim it */ rcu_read_unlock(); =20 + htp =3D &ht; + if (callsite_coalesce) { + qht_init(&coalesce_ht, qsp_entry_no_thread_obj_cmp, QSP_INITIAL_SI= ZE, + QHT_MODE_AUTO_RESIZE | QHT_MODE_RAW_MUTEXES); + qht_iter(&ht, qsp_iter_callsite_coalesce, &coalesce_ht); + + /* free the previous hash table, and point htp to coalesce_ht */ + qht_iter(&ht, qsp_ht_delete, NULL); + qht_destroy(&ht); + htp =3D &coalesce_ht; + } + /* sort the hash table elements by using a tree */ - qht_iter(&ht, qsp_sort, tree); + qht_iter(htp, qsp_sort, tree); =20 /* free the hash table, but keep the elements (those are in the tree n= ow) */ - qht_destroy(&ht); + qht_destroy(htp); } =20 /* free string with g_free */ @@ -562,6 +624,7 @@ struct QSPReportEntry { double time_s; double ns_avg; uint64_t n_acqs; + unsigned int n_objs; }; typedef struct QSPReportEntry QSPReportEntry; =20 @@ -585,6 +648,7 @@ static gboolean qsp_tree_report(gpointer key, gpointer = value, gpointer udata) report->n_entries++; =20 entry->obj =3D e->callsite->obj; + entry->n_objs =3D e->n_objs; entry->callsite_at =3D qsp_at(e->callsite); entry->typename =3D qsp_typenames[e->callsite->type]; entry->time_s =3D e->ns * 1e-9; @@ -629,10 +693,20 @@ pr_report(const QSPReport *rep, FILE *f, fprintf_func= tion pr) =20 for (i =3D 0; i < rep->n_entries; i++) { const QSPReportEntry *e =3D &rep->entries[i]; + GString *s =3D g_string_new(NULL); =20 - pr(f, "%-9s %14p %s%*s %13.5f %12" PRIu64 " %12.2f\n", e->typ= ename, - e->obj, e->callsite_at, callsite_len - (int)strlen(e->callsite_= at), - "", e->time_s, e->n_acqs, e->ns_avg * 1e-3); + g_string_append_printf(s, "%-9s ", e->typename); + if (e->n_objs > 1) { + g_string_append_printf(s, "[%12u]", e->n_objs); + } else { + g_string_append_printf(s, "%14p", e->obj); + } + g_string_append_printf(s, " %s%*s %13.5f %12" PRIu64 " %12.2f\= n", + e->callsite_at, + callsite_len - (int)strlen(e->callsite_at),= "", + e->time_s, e->n_acqs, e->ns_avg * 1e-3); + pr(f, "%s", s->str); + g_string_free(s, TRUE); } =20 pr(f, "%s\n", dashes); @@ -652,7 +726,7 @@ static void report_destroy(QSPReport *rep) } =20 void qsp_report(FILE *f, fprintf_function cpu_fprintf, size_t max, - enum QSPSortBy sort_by) + enum QSPSortBy sort_by, bool callsite_coalesce) { GTree *tree =3D g_tree_new_full(qsp_tree_cmp, &sort_by, g_free, NULL); QSPReport rep; @@ -663,7 +737,7 @@ void qsp_report(FILE *f, fprintf_function cpu_fprintf, = size_t max, rep.n_entries =3D 0; rep.max_n_entries =3D max; =20 - qsp_mktree(tree); + qsp_mktree(tree, callsite_coalesce); g_tree_foreach(tree, qsp_tree_report, &rep); g_tree_destroy(tree); =20 --=20 2.17.1 From nobody Wed Nov 5 07:16:03 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1534483361923646.9923228514923; Thu, 16 Aug 2018 22:22:41 -0700 (PDT) Received: from localhost ([::1]:60002 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fqXDc-0004pU-OF for importer@patchew.org; Fri, 17 Aug 2018 01:22:40 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:36282) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fqXA8-0000dB-On for qemu-devel@nongnu.org; Fri, 17 Aug 2018 01:19:06 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fqXA5-0005Z1-FM for qemu-devel@nongnu.org; Fri, 17 Aug 2018 01:19:04 -0400 Received: from out1-smtp.messagingengine.com ([66.111.4.25]:59001) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fqXA3-0005Vo-Dt for qemu-devel@nongnu.org; Fri, 17 Aug 2018 01:18:59 -0400 Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.nyi.internal (Postfix) with ESMTP id 72C7021E50; Fri, 17 Aug 2018 01:18:57 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute4.internal (MEProxy); Fri, 17 Aug 2018 01:18:57 -0400 Received: from localhost (flamenco.cs.columbia.edu [128.59.20.216]) by mail.messagingengine.com (Postfix) with ESMTPA id 292811029A; Fri, 17 Aug 2018 01:18:57 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=braap.org; h=cc :date:from:in-reply-to:message-id:references:subject:to :x-me-sender:x-me-sender:x-sasl-enc; s=mesmtp; bh=nubas8S5NqCwjV a0jdcbbyEEBN9414KFvbYG/dDcJgI=; b=o1Q65WqPi61hfdng/uU/fXSeGYyrX/ vrcUaXWKpxSPz9MNXpGtDFtZ5llYrh3j4veb9FIl/R/U3y4jcve5bD1zEJkh7mXP 3w/V3IZkjHh6l75H41o/TGNRbPqNa8z9IhycWVvrZbhU41MkIVNYwmPChzoCONDJ dkU/HhpQ72BMk= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:date:from:in-reply-to:message-id :references:subject:to:x-me-sender:x-me-sender:x-sasl-enc; s= fm3; bh=nubas8S5NqCwjVa0jdcbbyEEBN9414KFvbYG/dDcJgI=; b=mlKvtwhs 6IcaSZmGmdsLyQs6o///Jbgv/UfUy9aUCTCKqzxGtqlmsCS5dHduuvRe7xdvgEZf T1HwwpnS/EVPMGddnVp7dNcqmdK9adadBxJ0RsORPmRx9OfQOzep2VPLnTr2gmhz w+43CN93oZkjiH4Wz932EX+OBXZrnxtrda9Bb+iOhNbQYFOKljNFwyQ9c0mbLpPq KyAyI/x8L9ZXNLg9ecCghUIYgDq/RrkOjtFPL0/iz49hQmyje1TYrH7XwaKAbmGv RgACvt2dvoumrqAyOgxrjiwAq0kBfbcM0hd46FKJshGjWBq7mGlMNCePQsapXf3y OS1irBh3Fv5bNg== X-ME-Proxy: X-ME-Sender: From: "Emilio G. Cota" To: qemu-devel@nongnu.org Date: Fri, 17 Aug 2018 01:18:49 -0400 Message-Id: <20180817051853.23792-6-cota@braap.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180817051853.23792-1-cota@braap.org> References: <20180817051853.23792-1-cota@braap.org> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 66.111.4.25 Subject: [Qemu-devel] [PATCH 5/9] qsp: track BQL callers explicitly X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , Peter Crosthwaite , Stefan Weil , "Dr. David Alan Gilbert" , Peter Xu , Markus Armbruster , Paolo Bonzini , Richard Henderson Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" The BQL is acquired via qemu_mutex_lock_iothread(), which makes the profiler assign the associated wait time (i.e. most of BQL wait time) entirely to that function. This loses the original call site information, which does not help diagnose BQL contention. Fix it by tracking the callers explicitly. Signed-off-by: Emilio G. Cota --- include/qemu/main-loop.h | 4 +++- include/qemu/thread.h | 1 + cpus.c | 10 ++++++++-- stubs/iothread-lock.c | 2 +- util/qsp.c | 6 ++++++ 5 files changed, 19 insertions(+), 4 deletions(-) diff --git a/include/qemu/main-loop.h b/include/qemu/main-loop.h index 721aa2416a..e59f9ae1e9 100644 --- a/include/qemu/main-loop.h +++ b/include/qemu/main-loop.h @@ -276,7 +276,9 @@ bool qemu_mutex_iothread_locked(void); * NOTE: tools currently are single-threaded and qemu_mutex_lock_iothread * is a no-op there. */ -void qemu_mutex_lock_iothread(void); +#define qemu_mutex_lock_iothread() \ + qemu_mutex_lock_iothread_impl(__FILE__, __LINE__) +void qemu_mutex_lock_iothread_impl(const char *file, int line); =20 /** * qemu_mutex_unlock_iothread: Unlock the main loop mutex. diff --git a/include/qemu/thread.h b/include/qemu/thread.h index ab508df3cd..b41c0ae16a 100644 --- a/include/qemu/thread.h +++ b/include/qemu/thread.h @@ -35,6 +35,7 @@ typedef int (*QemuRecMutexTrylockFunc)(QemuRecMutex *m, c= onst char *f, int l); typedef void (*QemuCondWaitFunc)(QemuCond *c, QemuMutex *m, const char *f, int l); =20 +extern QemuMutexLockFunc qemu_bql_mutex_lock_func; extern QemuMutexLockFunc qemu_mutex_lock_func; extern QemuMutexTrylockFunc qemu_mutex_trylock_func; extern QemuRecMutexLockFunc qemu_rec_mutex_lock_func; diff --git a/cpus.c b/cpus.c index b5844b7103..a5ea3eef80 100644 --- a/cpus.c +++ b/cpus.c @@ -1762,10 +1762,16 @@ bool qemu_mutex_iothread_locked(void) return iothread_locked; } =20 -void qemu_mutex_lock_iothread(void) +/* + * The BQL is taken from so many places that it is worth profiling the + * callers directly, instead of funneling them all through a single functi= on. + */ +void qemu_mutex_lock_iothread_impl(const char *file, int line) { + QemuMutexLockFunc bql_lock =3D atomic_read(&qemu_bql_mutex_lock_func); + g_assert(!qemu_mutex_iothread_locked()); - qemu_mutex_lock(&qemu_global_mutex); + bql_lock(&qemu_global_mutex, file, line); iothread_locked =3D true; } =20 diff --git a/stubs/iothread-lock.c b/stubs/iothread-lock.c index 9b6db2e740..eb745d7d6a 100644 --- a/stubs/iothread-lock.c +++ b/stubs/iothread-lock.c @@ -7,7 +7,7 @@ bool qemu_mutex_iothread_locked(void) return true; } =20 -void qemu_mutex_lock_iothread(void) +void qemu_mutex_lock_iothread_impl(const char *file, int line) { } =20 diff --git a/util/qsp.c b/util/qsp.c index ea128d3748..65d9d8f0d6 100644 --- a/util/qsp.c +++ b/util/qsp.c @@ -65,6 +65,7 @@ =20 enum QSPType { QSP_MUTEX, + QSP_BQL_MUTEX, QSP_REC_MUTEX, QSP_CONDVAR, }; @@ -116,10 +117,12 @@ static bool qsp_initialized, qsp_initializing; =20 static const char * const qsp_typenames[] =3D { [QSP_MUTEX] =3D "mutex", + [QSP_BQL_MUTEX] =3D "BQL mutex", [QSP_REC_MUTEX] =3D "rec_mutex", [QSP_CONDVAR] =3D "condvar", }; =20 +QemuMutexLockFunc qemu_bql_mutex_lock_func =3D qemu_mutex_lock_impl; QemuMutexLockFunc qemu_mutex_lock_func =3D qemu_mutex_lock_impl; QemuMutexTrylockFunc qemu_mutex_trylock_func =3D qemu_mutex_trylock_impl; QemuRecMutexLockFunc qemu_rec_mutex_lock_func =3D qemu_rec_mutex_lock_impl; @@ -368,6 +371,7 @@ static QSPEntry *qsp_entry_get(const void *obj, const c= har *file, int line, return err; \ } =20 +QSP_GEN_VOID(QemuMutex, QSP_BQL_MUTEX, qsp_bql_mutex_lock, qemu_mutex_lock= _impl) QSP_GEN_VOID(QemuMutex, QSP_MUTEX, qsp_mutex_lock, qemu_mutex_lock_impl) QSP_GEN_RET1(QemuMutex, QSP_MUTEX, qsp_mutex_trylock, qemu_mutex_trylock_i= mpl) =20 @@ -403,6 +407,7 @@ void qsp_enable(void) { atomic_set(&qemu_mutex_lock_func, qsp_mutex_lock); atomic_set(&qemu_mutex_trylock_func, qsp_mutex_trylock); + atomic_set(&qemu_bql_mutex_lock_func, qsp_bql_mutex_lock); atomic_set(&qemu_rec_mutex_lock_func, qsp_rec_mutex_lock); atomic_set(&qemu_rec_mutex_trylock_func, qsp_rec_mutex_trylock); atomic_set(&qemu_cond_wait_func, qsp_cond_wait); @@ -412,6 +417,7 @@ void qsp_disable(void) { atomic_set(&qemu_mutex_lock_func, qemu_mutex_lock_impl); atomic_set(&qemu_mutex_trylock_func, qemu_mutex_trylock_impl); + atomic_set(&qemu_bql_mutex_lock_func, qemu_mutex_lock_impl); atomic_set(&qemu_rec_mutex_lock_func, qemu_rec_mutex_lock_impl); atomic_set(&qemu_rec_mutex_trylock_func, qemu_rec_mutex_trylock_impl); atomic_set(&qemu_cond_wait_func, qemu_cond_wait_impl); --=20 2.17.1 From nobody Wed Nov 5 07:16:03 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1534483358705593.2758408635152; Thu, 16 Aug 2018 22:22:38 -0700 (PDT) Received: from localhost ([::1]:60001 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fqXDZ-0004nN-JX for importer@patchew.org; Fri, 17 Aug 2018 01:22:37 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:36292) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fqXA8-0000dE-SA for qemu-devel@nongnu.org; Fri, 17 Aug 2018 01:19:05 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fqXA7-0005bZ-IQ for qemu-devel@nongnu.org; Fri, 17 Aug 2018 01:19:04 -0400 Received: from out1-smtp.messagingengine.com ([66.111.4.25]:52175) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fqXA6-0005Yx-KV for qemu-devel@nongnu.org; Fri, 17 Aug 2018 01:19:03 -0400 Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.nyi.internal (Postfix) with ESMTP id BA7B321E6D; Fri, 17 Aug 2018 01:18:57 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute4.internal (MEProxy); Fri, 17 Aug 2018 01:18:57 -0400 Received: from localhost (flamenco.cs.columbia.edu [128.59.20.216]) by mail.messagingengine.com (Postfix) with ESMTPA id 5D06F10269; Fri, 17 Aug 2018 01:18:57 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=braap.org; h=cc :date:from:in-reply-to:message-id:references:subject:to :x-me-sender:x-me-sender:x-sasl-enc; s=mesmtp; bh=93jLgGrA+Mc8l8 BjryG/S4A2BhecIk+smfUBPuecNH0=; b=v7TfBvEo1CofD+RfFoGlo65J7rz9e8 S3S+jS8tFnDu5a0ZNnwUlb2AQ3rNrqQTaH7Zze94q5pVYEt+cYbCxOLh5ODVr9oc Y6V9KBm3OL3CX4ZZDu8IHEv5sJKIZ23XjhdGF3oI1k8y2Hl2dKEnZ1PT6QUVePR4 OEt5nbX1xvG24= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:date:from:in-reply-to:message-id :references:subject:to:x-me-sender:x-me-sender:x-sasl-enc; s= fm3; bh=93jLgGrA+Mc8l8BjryG/S4A2BhecIk+smfUBPuecNH0=; b=XiEER6sA /zYLD9kZa7pt1oec9PJ7LX83kmFytdO0KkHSADq9gxli2gFNSyVkl2R2DcT9A44q rxNNaBZ5QGMuzSUgqmjGoZJ9MJvnzMplgeYiXODPp8W6oknXNdFIPA8BOIk7bTB0 E1bnmASTF/zVQhBbFqkDxTHfLcJc74KbHcjfH+HyoJSXHwboVITqEixNjEfOTefW hsXBYejGSHMovkkJrzaS8UURRoqBC2gO2apRxXq0ck8T26OYzbzi65xR9y+1obs3 Jah22SwV1JfPLw0SEQ4l4w3kZW7xrq+MIG3u+1nBYchKnhWhOPqbOuD3oau+fPGP JXvOuG01kSxG4g== X-ME-Proxy: X-ME-Sender: From: "Emilio G. Cota" To: qemu-devel@nongnu.org Date: Fri, 17 Aug 2018 01:18:50 -0400 Message-Id: <20180817051853.23792-7-cota@braap.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180817051853.23792-1-cota@braap.org> References: <20180817051853.23792-1-cota@braap.org> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 66.111.4.25 Subject: [Qemu-devel] [PATCH 6/9] tests/atomic_add-bench: add -p to enable sync profiler X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , Peter Crosthwaite , Stefan Weil , "Dr. David Alan Gilbert" , Peter Xu , Markus Armbruster , Paolo Bonzini , Richard Henderson Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" When used together with -m, this allows us to benchmark the profiler's performance impact on qemu_mutex_lock. Signed-off-by: Emilio G. Cota --- tests/atomic_add-bench.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/tests/atomic_add-bench.c b/tests/atomic_add-bench.c index f96d448f77..2f6c72f63a 100644 --- a/tests/atomic_add-bench.c +++ b/tests/atomic_add-bench.c @@ -26,6 +26,7 @@ static bool test_stop; static const char commands_string[] =3D " -n =3D number of threads\n" " -m =3D use mutexes instead of atomic increments\n" + " -p =3D enable sync profiler\n" " -d =3D duration in seconds\n" " -r =3D range (will be rounded up to pow2)"; =20 @@ -143,7 +144,7 @@ static void parse_args(int argc, char *argv[]) int c; =20 for (;;) { - c =3D getopt(argc, argv, "hd:n:mr:"); + c =3D getopt(argc, argv, "hd:n:mpr:"); if (c < 0) { break; } @@ -160,6 +161,9 @@ static void parse_args(int argc, char *argv[]) case 'm': use_mutex =3D true; break; + case 'p': + qsp_enable(); + break; case 'r': range =3D pow2ceil(atoi(optarg)); break; --=20 2.17.1 From nobody Wed Nov 5 07:16:03 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1534483249795679.0945762630532; Thu, 16 Aug 2018 22:20:49 -0700 (PDT) Received: from localhost ([::1]:59989 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fqXBh-00035J-4v for importer@patchew.org; Fri, 17 Aug 2018 01:20:41 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:36281) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fqXA8-0000dA-OU for qemu-devel@nongnu.org; Fri, 17 Aug 2018 01:19:05 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fqXA7-0005bT-IN for qemu-devel@nongnu.org; Fri, 17 Aug 2018 01:19:04 -0400 Received: from out1-smtp.messagingengine.com ([66.111.4.25]:39763) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fqXA7-0005Yj-Bn for qemu-devel@nongnu.org; Fri, 17 Aug 2018 01:19:03 -0400 Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.nyi.internal (Postfix) with ESMTP id F196421EDA; Fri, 17 Aug 2018 01:18:57 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute4.internal (MEProxy); Fri, 17 Aug 2018 01:18:57 -0400 Received: from localhost (flamenco.cs.columbia.edu [128.59.20.216]) by mail.messagingengine.com (Postfix) with ESMTPA id 98DC310268; Fri, 17 Aug 2018 01:18:57 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=braap.org; h=cc :date:from:in-reply-to:message-id:references:subject:to :x-me-sender:x-me-sender:x-sasl-enc; s=mesmtp; bh=aQRqgO/wB7hxSX zM7C04XSBCi1sR3cSEr9lN93OtGVk=; b=rxZR47lItBOcC6v0Mz6t28UWC/Zks2 7l1UDeweE0ZG7E7/9hOZBcSDfI4Zja7x5DeLk42MdQW1JSnyE9dSiSv1XVqbZYUu HsABpmgJXUZOjDGgwiUrK/q97eFSEc72bWx8d9dRTKJVoq53B9n7jQxwhs/gKAR4 ykjIuVaFZeZzM= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:date:from:in-reply-to:message-id :references:subject:to:x-me-sender:x-me-sender:x-sasl-enc; s= fm3; bh=aQRqgO/wB7hxSXzM7C04XSBCi1sR3cSEr9lN93OtGVk=; b=q9px7fmL vB7Bi3LThaJSirzgEL58AO58A6EWDOUPICMndGhUUBHsT6DDeQOMFs0XlnIRP3pm kzBZLr9TnJkYAq1d2UBN4bd0uQyWhWVv+pnip+ex+VSCQKOdMnBY9c1NyitgcQ7X LQpBXuoshzRXXiQiOxhPhhe4YS2sV2BopEc5ORu0rT0Rxh32v+82vMX7iIcMNbYa yVyIteC1mspOnOwjlOVYazHTrvl7sEPD5EAlmHhA2ERKm+MHU0AfZuqstfzJwIm8 wI7Jrs98xjllJVBqlS9dPtocSFErEy55To38rxEeyIVMIIVdBGyJ7x02nP1V5zZV UF1Fo14zfFO0uA== X-ME-Proxy: X-ME-Sender: From: "Emilio G. Cota" To: qemu-devel@nongnu.org Date: Fri, 17 Aug 2018 01:18:51 -0400 Message-Id: <20180817051853.23792-8-cota@braap.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180817051853.23792-1-cota@braap.org> References: <20180817051853.23792-1-cota@braap.org> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 66.111.4.25 Subject: [Qemu-devel] [PATCH 7/9] vl: add -enable-sync-profile X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , Peter Crosthwaite , Stefan Weil , "Dr. David Alan Gilbert" , Peter Xu , Markus Armbruster , Paolo Bonzini , Richard Henderson Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Signed-off-by: Emilio G. Cota --- vl.c | 3 +++ qemu-options.hx | 10 ++++++++++ 2 files changed, 13 insertions(+) diff --git a/vl.c b/vl.c index 16b913f9d5..5f4e24f29b 100644 --- a/vl.c +++ b/vl.c @@ -3959,6 +3959,9 @@ int main(int argc, char **argv, char **envp) exit(1); } break; + case QEMU_OPTION_enable_sync_profile: + qsp_enable(); + break; case QEMU_OPTION_nodefconfig: case QEMU_OPTION_nouserconfig: /* Nothing to be parsed here. Especially, do not error out= below. */ diff --git a/qemu-options.hx b/qemu-options.hx index ea4edb4938..466a61d384 100644 --- a/qemu-options.hx +++ b/qemu-options.hx @@ -3963,6 +3963,16 @@ Dump json-encoded vmstate information for current ma= chine type to file in @var{file} ETEXI =20 +DEF("enable-sync-profile", 0, QEMU_OPTION_enable_sync_profile, + "-enable-sync-profile\n" + " enable synchronization profiling\n", + QEMU_ARCH_ALL) +STEXI +@item -enable-sync-profile +@findex -enable-sync-profile +Enable synchronization profiling. +ETEXI + STEXI @end table ETEXI --=20 2.17.1 From nobody Wed Nov 5 07:16:03 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1534483466200321.1530699797879; Thu, 16 Aug 2018 22:24:26 -0700 (PDT) Received: from localhost ([::1]:60010 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fqXFJ-00062U-4M for importer@patchew.org; Fri, 17 Aug 2018 01:24:25 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:36284) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fqXA8-0000dC-Ov for qemu-devel@nongnu.org; Fri, 17 Aug 2018 01:19:05 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fqXA7-0005bg-Ik for qemu-devel@nongnu.org; Fri, 17 Aug 2018 01:19:04 -0400 Received: from out1-smtp.messagingengine.com ([66.111.4.25]:33471) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fqXA6-0005Ys-Ko for qemu-devel@nongnu.org; Fri, 17 Aug 2018 01:19:03 -0400 Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.nyi.internal (Postfix) with ESMTP id 3C4C321EE9; Fri, 17 Aug 2018 01:18:58 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute4.internal (MEProxy); Fri, 17 Aug 2018 01:18:58 -0400 Received: from localhost (flamenco.cs.columbia.edu [128.59.20.216]) by mail.messagingengine.com (Postfix) with ESMTPA id CF85610289; Fri, 17 Aug 2018 01:18:57 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=braap.org; h=cc :date:from:in-reply-to:message-id:references:subject:to :x-me-sender:x-me-sender:x-sasl-enc; s=mesmtp; bh=z6TwBIqv3+8NS0 +Yz1W7WJJExAmF1Wzv8D0whxaKSCY=; b=1y508dTWWBuxUJh6mmvNBG6TOLXfAz zYk543bBiyAEswntMJF789Kb5gWSn233wdd4ZrSP+gJKU4lHGgxE/RpxdSLKu3hB 8nZ2yXtHomSQMwwoprrhg7RDGRGPBirPpIL2q+avpG7F/5X3N2B8TDz6PsKYD5Ji u3oFhvw2RsoEg= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:date:from:in-reply-to:message-id :references:subject:to:x-me-sender:x-me-sender:x-sasl-enc; s= fm3; bh=z6TwBIqv3+8NS0+Yz1W7WJJExAmF1Wzv8D0whxaKSCY=; b=DDdnPIMw GASRw3A2rUqTmR9ry1qWzQyUYpm2sQ2mp7R1Ap/gdrH0aMla6ofmXqfP3wZLiDOQ 5S7mn+4XrRMBNmgALJUqg3YFlPFcdSutaFf1uDwD1CRzPvpJlUTewDlZngPWllRO TtBUB4HHvubercflxoM4JVMqLIKKvegbe6Kre5tXtw3VlKxVMOUnQTjBpt8if+Yy ZeiGoQNxv5aSZdc3gWBg2/cJ21iTKnTh5khHQSmGEbgLYn2gXED7vxDsBugmDkPu ACd4lJWAVhRh0++Yv4/hZA7IHjOvJt7xW1sl5n3aEwimjK1SDPRR57RpgDOP+u3Y 9v9OxsTEiyz+GQ== X-ME-Proxy: X-ME-Sender: From: "Emilio G. Cota" To: qemu-devel@nongnu.org Date: Fri, 17 Aug 2018 01:18:52 -0400 Message-Id: <20180817051853.23792-9-cota@braap.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180817051853.23792-1-cota@braap.org> References: <20180817051853.23792-1-cota@braap.org> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 66.111.4.25 Subject: [Qemu-devel] [PATCH 8/9] hmp-commands: add sync-profile X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , Peter Crosthwaite , Stefan Weil , "Dr. David Alan Gilbert" , Peter Xu , Markus Armbruster , Paolo Bonzini , Richard Henderson Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" The command introduced here is just for developers. This means that: - the interface implemented here could change in the future - the command is only meant to be used from HMP, not from QMP Signed-off-by: Emilio G. Cota Reviewed-by: Dr. David Alan Gilbert --- hmp.h | 1 + hmp.c | 24 ++++++++++++++++++++++++ hmp-commands.hx | 15 +++++++++++++++ 3 files changed, 40 insertions(+) diff --git a/hmp.h b/hmp.h index 33354f1bdd..5f1addcca2 100644 --- a/hmp.h +++ b/hmp.h @@ -42,6 +42,7 @@ void hmp_info_tpm(Monitor *mon, const QDict *qdict); void hmp_info_iothreads(Monitor *mon, const QDict *qdict); void hmp_quit(Monitor *mon, const QDict *qdict); void hmp_stop(Monitor *mon, const QDict *qdict); +void hmp_sync_profile(Monitor *mon, const QDict *qdict); void hmp_system_reset(Monitor *mon, const QDict *qdict); void hmp_system_powerdown(Monitor *mon, const QDict *qdict); void hmp_exit_preconfig(Monitor *mon, const QDict *qdict); diff --git a/hmp.c b/hmp.c index 2aafb50e8e..d94a47f7c7 100644 --- a/hmp.c +++ b/hmp.c @@ -1062,6 +1062,30 @@ void hmp_stop(Monitor *mon, const QDict *qdict) qmp_stop(NULL); } =20 +void hmp_sync_profile(Monitor *mon, const QDict *qdict) +{ + const char *op =3D qdict_get_try_str(qdict, "op"); + + if (op =3D=3D NULL) { + bool on =3D qsp_is_enabled(); + + monitor_printf(mon, "sync-profile is %s\n", on ? "on" : "off"); + return; + } + if (!strcmp(op, "on")) { + qsp_enable(); + } else if (!strcmp(op, "off")) { + qsp_disable(); + } else if (!strcmp(op, "reset")) { + qsp_reset(); + } else { + Error *err =3D NULL; + + error_setg(&err, QERR_INVALID_PARAMETER, op); + hmp_handle_error(mon, &err); + } +} + void hmp_system_reset(Monitor *mon, const QDict *qdict) { qmp_system_reset(NULL); diff --git a/hmp-commands.hx b/hmp-commands.hx index c1fc747403..db0c681f74 100644 --- a/hmp-commands.hx +++ b/hmp-commands.hx @@ -643,6 +643,21 @@ sendkey ctrl-alt-f1 =20 This command is useful to send keys that your graphical user interface intercepts at low level, such as @code{ctrl-alt-f1} in X Window. +ETEXI + { + .name =3D "sync-profile", + .args_type =3D "op:s?", + .params =3D "[on|off|reset]", + .help =3D "enable, disable or reset synchronization profilin= g. " + "With no arguments, prints whether profiling is on o= r off.", + .cmd =3D hmp_sync_profile, + }, + +STEXI +@item sync-profile [on|off|reset] +@findex sync-profile +Enable, disable or reset synchronization profiling. With no arguments, pri= nts +whether profiling is on or off. ETEXI =20 { --=20 2.17.1 From nobody Wed Nov 5 07:16:03 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1534483689123124.42724891498938; Thu, 16 Aug 2018 22:28:09 -0700 (PDT) Received: from localhost ([::1]:60029 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fqXIu-0001sV-0v for importer@patchew.org; Fri, 17 Aug 2018 01:28:08 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:36331) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fqXAA-0000dh-6t for qemu-devel@nongnu.org; Fri, 17 Aug 2018 01:19:07 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fqXA7-0005bm-K5 for qemu-devel@nongnu.org; Fri, 17 Aug 2018 01:19:06 -0400 Received: from out1-smtp.messagingengine.com ([66.111.4.25]:42889) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fqXA7-0005Yz-D2 for qemu-devel@nongnu.org; Fri, 17 Aug 2018 01:19:03 -0400 Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.nyi.internal (Postfix) with ESMTP id 773DA21EED; Fri, 17 Aug 2018 01:18:58 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute4.internal (MEProxy); Fri, 17 Aug 2018 01:18:58 -0400 Received: from localhost (flamenco.cs.columbia.edu [128.59.20.216]) by mail.messagingengine.com (Postfix) with ESMTPA id 1365910269; Fri, 17 Aug 2018 01:18:58 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=braap.org; h=cc :date:from:in-reply-to:message-id:references:subject:to :x-me-sender:x-me-sender:x-sasl-enc; s=mesmtp; bh=RHXtTt+C9jgde1 lML4fhSfKjA4yTmgWHVgxFuMFRmYg=; b=d/1m36zrLUgwZOKVgLs0kfoM9LoRri AK2mSUzoW7o/01XByE+hyvy+jKIdpmxDn6DT9Be99ncP76uxTWEabRhZU8Jpuvtw e7CyFAdq6OSRU0iTpfq9cqTdCjH2HcdmO8VhUnMYLMGMFkEOnq5fLfR9r4o/2NRG 9UwhWo+YkKcbw= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:date:from:in-reply-to:message-id :references:subject:to:x-me-sender:x-me-sender:x-sasl-enc; s= fm3; bh=RHXtTt+C9jgde1lML4fhSfKjA4yTmgWHVgxFuMFRmYg=; b=J9a4OGEc TTiqu7NBFzJmREj7+NXYj7JAZ7pUBRmvb/Q8v7BUZkOpc5FZGhameFaZ+0fa4yN7 LJ3V+dzG1kRmCkK30LwjWry78qa4yJheEmBsK6ccT63L6Mfb0GFSwPnLn3gtyHbW TnnIJ75x5SxCIgkShBFVKtzsQgs5ZR1Cuqy37JCZ/B2ns7JiBcAtflsp2ynA+5Mn EbapXQj/faq2U78bEHEM7XP4KYys5hWs4kL1atsB5WLTk89GG58qWeyz+QAkdZ8p s48HDuoZFd52TGDgeAjjZISrptwP3cmvy4uA/n+gK8pzP+NPSlSKfHavBl8c60Sa 3e8dp6ZAtBe5Aw== X-ME-Proxy: X-ME-Sender: From: "Emilio G. Cota" To: qemu-devel@nongnu.org Date: Fri, 17 Aug 2018 01:18:53 -0400 Message-Id: <20180817051853.23792-10-cota@braap.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180817051853.23792-1-cota@braap.org> References: <20180817051853.23792-1-cota@braap.org> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 66.111.4.25 Subject: [Qemu-devel] [PATCH 9/9] hmp-commands-info: add sync-profile X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , Peter Crosthwaite , Stefan Weil , "Dr. David Alan Gilbert" , Peter Xu , Markus Armbruster , Paolo Bonzini , Richard Henderson Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" The command introduced here is just for developers. This means that: - the info displayed and the output format could change in the future - the command is only meant to be used from HMP, not from QMP Sample output: (qemu) sync-profile sync-profile is off (qemu) info sync-profile Type Object Call site Wait Time (s) Count Average = (us) Reviewed-by: Dr. David Alan Gilbert ---------------------------------------------------------------------------= ---- ---------------------------------------------------------------------------= ---- (qemu) sync-profile on (qemu) sync-profile sync-profile is on (qemu) info sync-profile 15 Type Object Call site Wait Time (s) = Count Average (us) ---------------------------------------------------------------------------= ------------------- condvar 0x55a01813ced0 cpus.c:1165 91.38235 = 2842 32154.24 BQL mutex 0x55a0171b7140 cpus.c:1434 12.56490 = 5787 2171.23 BQL mutex 0x55a0171b7140 accel/tcg/cpu-exec.c:432 7.75846 = 2844 2728.01 BQL mutex 0x55a0171b7140 accel/tcg/cputlb.c:870 5.09889 = 2884 1767.99 BQL mutex 0x55a0171b7140 accel/tcg/cpu-exec.c:529 3.46140 = 3254 1063.74 BQL mutex 0x55a0171b7140 accel/tcg/cputlb.c:804 0.76333 = 8655 88.20 BQL mutex 0x55a0171b7140 cpus.c:1466 0.60893 = 2941 207.05 BQL mutex 0x55a0171b7140 util/main-loop.c:236 0.00894 = 6425 1.39 mutex [ 3] util/qemu-timer.c:520 0.00342 = 50611 0.07 mutex [ 2] util/qemu-timer.c:426 0.00254 = 31336 0.08 mutex [ 3] util/qemu-timer.c:234 0.00107 1= 9275 0.06 mutex 0x55a0171d9960 vl.c:763 0.00043 = 6425 0.07 mutex 0x55a0180d1bb0 monitor.c:458 0.00015 = 1603 0.09 mutex 0x55a0180e4c78 chardev/char.c:109 0.00002 = 217 0.08 mutex 0x55a0180d1bb0 monitor.c:448 0.00001 = 162 0.08 ---------------------------------------------------------------------------= ------------------- (qemu) info sync-profile -m 15 Type Object Call site Wait Time (s) = Count Average (us) ---------------------------------------------------------------------------= ------------------- condvar 0x55a01813ced0 cpus.c:1165 95.11196 = 3051 31174.03 BQL mutex 0x55a0171b7140 accel/tcg/cpu-exec.c:432 7.92108 = 3052 2595.37 BQL mutex 0x55a0171b7140 cpus.c:1434 13.38253 = 6210 2155.00 BQL mutex 0x55a0171b7140 accel/tcg/cputlb.c:870 5.09901 = 3093 1648.57 BQL mutex 0x55a0171b7140 accel/tcg/cpu-exec.c:529 4.21123 = 3468 1214.31 BQL mutex 0x55a0171b7140 cpus.c:1466 0.60895 = 3156 192.95 BQL mutex 0x55a0171b7140 accel/tcg/cputlb.c:804 0.76337 = 9282 82.24 BQL mutex 0x55a0171b7140 util/main-loop.c:236 0.00944 = 6889 1.37 mutex 0x55a01813ce80 tcg/tcg.c:397 0.00000 = 24 0.15 mutex 0x55a0180d1bb0 monitor.c:458 0.00018 = 1922 0.09 mutex [ 2] util/qemu-timer.c:426 0.00266 = 32710 0.08 mutex 0x55a0180e4c78 chardev/char.c:109 0.00002 = 260 0.08 mutex 0x55a0180d1bb0 monitor.c:448 0.00001 = 187 0.08 mutex 0x55a0171d9960 vl.c:763 0.00047 = 6889 0.07 mutex [ 3] util/qemu-timer.c:520 0.00362 = 53377 0.07 ---------------------------------------------------------------------------= ------------------- (qemu) info sync-profile -m -n 15 Type Object Call site Wait Time (s) = Count Average (us) ---------------------------------------------------------------------------= ------------------- condvar 0x55a01813ced0 cpus.c:1165 101.39331 = 3398 29839.12 BQL mutex 0x55a0171b7140 accel/tcg/cpu-exec.c:432 7.92112 = 3399 2330.43 BQL mutex 0x55a0171b7140 cpus.c:1434 14.28280 = 6922 2063.39 BQL mutex 0x55a0171b7140 accel/tcg/cputlb.c:870 5.77505 = 3445 1676.36 BQL mutex 0x55a0171b7140 accel/tcg/cpu-exec.c:529 5.66139 = 3883 1457.99 BQL mutex 0x55a0171b7140 cpus.c:1466 0.60901 = 3519 173.06 BQL mutex 0x55a0171b7140 accel/tcg/cputlb.c:804 0.76351 = 10338 73.85 BQL mutex 0x55a0171b7140 util/main-loop.c:236 0.01032 = 7664 1.35 mutex 0x55a0180e4f08 util/qemu-timer.c:426 0.00041 = 901 0.45 mutex 0x55a01813ce80 tcg/tcg.c:397 0.00000 = 24 0.15 mutex 0x55a0180d1bb0 monitor.c:458 0.00022 = 2319 0.09 mutex 0x55a0180e4c78 chardev/char.c:109 0.00003 = 306 0.08 mutex 0x55a0180e4f08 util/qemu-timer.c:520 0.00068 = 8565 0.08 mutex 0x55a0180d1bb0 monitor.c:448 0.00002 = 215 0.08 mutex 0x55a0180e4f78 util/qemu-timer.c:426 0.00247 = 34224 0.07 ---------------------------------------------------------------------------= ------------------- (qemu) sync-profile reset (qemu) info sync-profile -m 2 Type Object Call site Wait Time (s) Co= unt Average (us) ---------------------------------------------------------------------------= ----------------- condvar 0x55a01813ced0 cpus.c:1165 2.78756 = 99 28157.12 BQL mutex 0x55a0171b7140 accel/tcg/cputlb.c:870 0.33054 = 102 3240.55 ---------------------------------------------------------------------------= ----------------- (qemu) sync-profile off (qemu) sync-profile sync-profile is off (qemu) sync-profile reset (qemu) info sync-profile Type Object Call site Wait Time (s) Count Average = (us) ---------------------------------------------------------------------------= ---- ---------------------------------------------------------------------------= ---- Signed-off-by: Emilio G. Cota --- monitor.c | 11 +++++++++++ hmp-commands-info.hx | 22 ++++++++++++++++++++++ 2 files changed, 33 insertions(+) diff --git a/monitor.c b/monitor.c index a1999e396c..94f673511b 100644 --- a/monitor.c +++ b/monitor.c @@ -1454,6 +1454,17 @@ static void hmp_info_opcount(Monitor *mon, const QDi= ct *qdict) } #endif =20 +static void hmp_info_sync_profile(Monitor *mon, const QDict *qdict) +{ + int64_t max =3D qdict_get_try_int(qdict, "max", 10); + bool mean =3D qdict_get_try_bool(qdict, "mean", false); + bool coalesce =3D !qdict_get_try_bool(qdict, "no_coalesce", false); + enum QSPSortBy sort_by; + + sort_by =3D mean ? QSP_SORT_BY_AVG_WAIT_TIME : QSP_SORT_BY_TOTAL_WAIT_= TIME; + qsp_report((FILE *)mon, monitor_fprintf, max, sort_by, coalesce); +} + static void hmp_info_history(Monitor *mon, const QDict *qdict) { int i; diff --git a/hmp-commands-info.hx b/hmp-commands-info.hx index 70639f656a..cbee8b944d 100644 --- a/hmp-commands-info.hx +++ b/hmp-commands-info.hx @@ -297,6 +297,28 @@ STEXI @item info opcount @findex info opcount Show dynamic compiler opcode counters +ETEXI + + { + .name =3D "sync-profile", + .args_type =3D "mean:-m,no_coalesce:-n,max:i?", + .params =3D "[-m] [-n] [max]", + .help =3D "show synchronization profiling info, up to max en= tries " + "(default: 10), sorted by total wait time. (-m: sort= by " + "mean wait time; -n: do not coalesce objects with th= e " + "same call site)", + .cmd =3D hmp_info_sync_profile, + }, + +STEXI +@item info sync-profile [-m|-n] [@var{max}] +@findex info sync-profile +Show synchronization profiling info, up to @var{max} entries (default: 10), +sorted by total wait time. + -m: sort by mean wait time + -n: do not coalesce objects with the same call site +When different objects that share the same call site are coalesced, the "O= bject" +field shows---enclosed in brackets---the number of objects being coalesced. ETEXI =20 { --=20 2.17.1