From nobody Wed Nov 5 09:26:54 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 153448346951694.16797795299897; Thu, 16 Aug 2018 22:24:29 -0700 (PDT) Received: from localhost ([::1]:60011 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fqXFM-00064e-9d for importer@patchew.org; Fri, 17 Aug 2018 01:24:28 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:36285) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fqXA8-0000dD-Oy for qemu-devel@nongnu.org; Fri, 17 Aug 2018 01:19:06 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fqXA5-0005Yo-FV for qemu-devel@nongnu.org; Fri, 17 Aug 2018 01:19:04 -0400 Received: from out1-smtp.messagingengine.com ([66.111.4.25]:48295) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fqXA3-0005Vw-Du for qemu-devel@nongnu.org; Fri, 17 Aug 2018 01:18:59 -0400 Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.nyi.internal (Postfix) with ESMTP id 8955021E56; Fri, 17 Aug 2018 01:18:57 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute4.internal (MEProxy); Fri, 17 Aug 2018 01:18:57 -0400 Received: from localhost (flamenco.cs.columbia.edu [128.59.20.216]) by mail.messagingengine.com (Postfix) with ESMTPA id E87EC10299; Fri, 17 Aug 2018 01:18:56 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=braap.org; h=cc :date:from:in-reply-to:message-id:references:subject:to :x-me-sender:x-me-sender:x-sasl-enc; s=mesmtp; bh=uSJeSYH0b6DA+s Bvn0BXVOE/UALA+7BHr0cJh+lPt9E=; b=FYYL0WO8FOuHY4zL8mhxpKKCerksBB 10Le/1br2LABtOcNaRViNx3pdA0zpxJBZLS27L2RDaasgnHHZ9BbkbB+n7aMaaLx wJ431yy9vIEAuhFWLcKNzw11TV1mGIFk1aQZHjJKotYwC3hesPLgnTogyAXXfb5y TeElF2yzdHfa8= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:date:from:in-reply-to:message-id :references:subject:to:x-me-sender:x-me-sender:x-sasl-enc; s= fm3; bh=uSJeSYH0b6DA+sBvn0BXVOE/UALA+7BHr0cJh+lPt9E=; b=ExXKmbWG QtpPFeUcwqd0EBYvsS3E11Am5zLKKT+9biukaUy7BGtsJA9vssc5Wn43t1KjsaIB lSYEnO/wdAtiVTSYEXmx3Z3Afk8d7+Plo135oZQz0PhzAn/PpXxNETxXF3Bs25ml LR5A0sFQzGP7gt2gEV8gyHIwWFtHJaV2Toi6AMNK7+Lz/c5jLX9hCPOj14/JRyRW YD6jLe4+c7m/l3+ir0CJgB89OJ19PItFG1+H03vDk158SnvAzJBgPCH0RAiX9MBm i2e4yXaEv/HOjSKRYsCryPnYxJVFJttlSg68FgecUacUKRCeGr42BG1h1uMpj+LA ciLY3SoyXzIG8Q== X-ME-Proxy: X-ME-Sender: From: "Emilio G. Cota" To: qemu-devel@nongnu.org Date: Fri, 17 Aug 2018 01:18:48 -0400 Message-Id: <20180817051853.23792-5-cota@braap.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180817051853.23792-1-cota@braap.org> References: <20180817051853.23792-1-cota@braap.org> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 66.111.4.25 Subject: [Qemu-devel] [PATCH 4/9] qsp: support call site coalescing X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , Peter Crosthwaite , Stefan Weil , "Dr. David Alan Gilbert" , Peter Xu , Markus Armbruster , Paolo Bonzini , Richard Henderson Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Signed-off-by: Emilio G. Cota --- include/qemu/qsp.h | 2 +- util/qsp.c | 102 ++++++++++++++++++++++++++++++++++++++------- 2 files changed, 89 insertions(+), 15 deletions(-) diff --git a/include/qemu/qsp.h b/include/qemu/qsp.h index f8c6c9648a..a94c464f90 100644 --- a/include/qemu/qsp.h +++ b/include/qemu/qsp.h @@ -19,7 +19,7 @@ enum QSPSortBy { }; =20 void qsp_report(FILE *f, fprintf_function cpu_fprintf, size_t max, - enum QSPSortBy sort_by); + enum QSPSortBy sort_by, bool callsite_coalesce); =20 bool qsp_is_enabled(void); void qsp_enable(void); diff --git a/util/qsp.c b/util/qsp.c index ea4bb82a03..ea128d3748 100644 --- a/util/qsp.c +++ b/util/qsp.c @@ -27,11 +27,9 @@ * Reports are generated as a table where each row represents a call site.= A * call site is the triplet formed by the __file__ and __LINE__ of the cal= ler * as well as the address of the "object" (i.e. mutex, rec. mutex or condv= ar) - * being operated on. Focusing on call sites instead of just on objects mi= ght - * seem puzzling. However, it is a sensible choice since otherwise dealing= with - * dynamically-allocated objects becomes difficult (e.g. what to do when an - * object is destroyed, or reused?). Furthermore, the call site info is of= most - * importance, since it is callers, and not objects, what cause wait time. + * being operated on. Optionally, call sites that operate on different obj= ects + * of the same type can be coalesced, which can be particularly useful when + * profiling dynamically-allocated objects. * * Alternative designs considered: * @@ -84,6 +82,7 @@ struct QSPEntry { const QSPCallSite *callsite; uint64_t n_acqs; uint64_t ns; + unsigned int n_objs; /* count of coalesced objs; only used for reporti= ng */ }; typedef struct QSPEntry QSPEntry; =20 @@ -163,6 +162,17 @@ static uint32_t qsp_entry_no_thread_hash(const QSPEntr= y *entry) return do_qsp_entry_hash(entry, 0); } =20 +/* without the objects we need to hash the file name to get a decent hash = */ +static uint32_t qsp_entry_no_thread_obj_hash(const QSPEntry *entry) +{ + const QSPCallSite *callsite =3D entry->callsite; + uint64_t a =3D g_str_hash(callsite->file); + uint64_t b =3D callsite->line; + uint32_t e =3D callsite->type; + + return tb_hash_func7(a, b, e, 0, 0); +} + static bool qsp_callsite_cmp(const void *ap, const void *bp) { const QSPCallSite *a =3D ap; @@ -175,6 +185,17 @@ static bool qsp_callsite_cmp(const void *ap, const voi= d *bp) (a->file =3D=3D b->file || !strcmp(a->file, b->file))); } =20 +static bool qsp_callsite_no_obj_cmp(const void *ap, const void *bp) +{ + const QSPCallSite *a =3D ap; + const QSPCallSite *b =3D bp; + + return a =3D=3D b || + (a->line =3D=3D b->line && + a->type =3D=3D b->type && + (a->file =3D=3D b->file || !strcmp(a->file, b->file))); +} + static bool qsp_entry_no_thread_cmp(const void *ap, const void *bp) { const QSPEntry *a =3D ap; @@ -183,6 +204,14 @@ static bool qsp_entry_no_thread_cmp(const void *ap, co= nst void *bp) return qsp_callsite_cmp(a->callsite, b->callsite); } =20 +static bool qsp_entry_no_thread_obj_cmp(const void *ap, const void *bp) +{ + const QSPEntry *a =3D ap; + const QSPEntry *b =3D bp; + + return qsp_callsite_no_obj_cmp(a->callsite, b->callsite); +} + static bool qsp_entry_cmp(const void *ap, const void *bp) { const QSPEntry *a =3D ap; @@ -499,15 +528,36 @@ static void qsp_diff(struct qht *orig, struct qht *ne= w) qht_iter(orig, qsp_iter_diff, new); } =20 +static void +qsp_iter_callsite_coalesce(struct qht *orig, void *p, uint32_t h, void *ht= p) +{ + struct qht *ht =3D htp; + QSPEntry *old =3D p; + QSPEntry *e; + uint32_t hash; + + hash =3D qsp_entry_no_thread_obj_hash(old); + e =3D qht_lookup(ht, old, hash); + if (e =3D=3D NULL) { + e =3D qsp_entry_create(ht, old, hash); + e->n_objs =3D 1; + } else if (e->callsite->obj !=3D old->callsite->obj) { + e->n_objs++; + } + e->ns +=3D old->ns; + e->n_acqs +=3D old->n_acqs; +} + static void qsp_ht_delete(struct qht *ht, void *p, uint32_t h, void *htp) { g_free(p); } =20 -static void qsp_mktree(GTree *tree) +static void qsp_mktree(GTree *tree, bool callsite_coalesce) { QSPSnapshot *snap; - struct qht ht; + struct qht ht, coalesce_ht; + struct qht *htp; =20 /* * First, see if there's a prior snapshot, so that we read the global = hash @@ -532,11 +582,23 @@ static void qsp_mktree(GTree *tree) /* done with the snapshot; RCU can reclaim it */ rcu_read_unlock(); =20 + htp =3D &ht; + if (callsite_coalesce) { + qht_init(&coalesce_ht, qsp_entry_no_thread_obj_cmp, QSP_INITIAL_SI= ZE, + QHT_MODE_AUTO_RESIZE | QHT_MODE_RAW_MUTEXES); + qht_iter(&ht, qsp_iter_callsite_coalesce, &coalesce_ht); + + /* free the previous hash table, and point htp to coalesce_ht */ + qht_iter(&ht, qsp_ht_delete, NULL); + qht_destroy(&ht); + htp =3D &coalesce_ht; + } + /* sort the hash table elements by using a tree */ - qht_iter(&ht, qsp_sort, tree); + qht_iter(htp, qsp_sort, tree); =20 /* free the hash table, but keep the elements (those are in the tree n= ow) */ - qht_destroy(&ht); + qht_destroy(htp); } =20 /* free string with g_free */ @@ -562,6 +624,7 @@ struct QSPReportEntry { double time_s; double ns_avg; uint64_t n_acqs; + unsigned int n_objs; }; typedef struct QSPReportEntry QSPReportEntry; =20 @@ -585,6 +648,7 @@ static gboolean qsp_tree_report(gpointer key, gpointer = value, gpointer udata) report->n_entries++; =20 entry->obj =3D e->callsite->obj; + entry->n_objs =3D e->n_objs; entry->callsite_at =3D qsp_at(e->callsite); entry->typename =3D qsp_typenames[e->callsite->type]; entry->time_s =3D e->ns * 1e-9; @@ -629,10 +693,20 @@ pr_report(const QSPReport *rep, FILE *f, fprintf_func= tion pr) =20 for (i =3D 0; i < rep->n_entries; i++) { const QSPReportEntry *e =3D &rep->entries[i]; + GString *s =3D g_string_new(NULL); =20 - pr(f, "%-9s %14p %s%*s %13.5f %12" PRIu64 " %12.2f\n", e->typ= ename, - e->obj, e->callsite_at, callsite_len - (int)strlen(e->callsite_= at), - "", e->time_s, e->n_acqs, e->ns_avg * 1e-3); + g_string_append_printf(s, "%-9s ", e->typename); + if (e->n_objs > 1) { + g_string_append_printf(s, "[%12u]", e->n_objs); + } else { + g_string_append_printf(s, "%14p", e->obj); + } + g_string_append_printf(s, " %s%*s %13.5f %12" PRIu64 " %12.2f\= n", + e->callsite_at, + callsite_len - (int)strlen(e->callsite_at),= "", + e->time_s, e->n_acqs, e->ns_avg * 1e-3); + pr(f, "%s", s->str); + g_string_free(s, TRUE); } =20 pr(f, "%s\n", dashes); @@ -652,7 +726,7 @@ static void report_destroy(QSPReport *rep) } =20 void qsp_report(FILE *f, fprintf_function cpu_fprintf, size_t max, - enum QSPSortBy sort_by) + enum QSPSortBy sort_by, bool callsite_coalesce) { GTree *tree =3D g_tree_new_full(qsp_tree_cmp, &sort_by, g_free, NULL); QSPReport rep; @@ -663,7 +737,7 @@ void qsp_report(FILE *f, fprintf_function cpu_fprintf, = size_t max, rep.n_entries =3D 0; rep.max_n_entries =3D max; =20 - qsp_mktree(tree); + qsp_mktree(tree, callsite_coalesce); g_tree_foreach(tree, qsp_tree_report, &rep); g_tree_destroy(tree); =20 --=20 2.17.1