From nobody Wed Nov 5 14:26:05 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1534872416477238.405468238528; Tue, 21 Aug 2018 10:26:56 -0700 (PDT) Received: from localhost ([::1]:55063 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fsAQg-0000GL-S8 for importer@patchew.org; Tue, 21 Aug 2018 13:26:54 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:33415) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fsA4X-0001us-70 for qemu-devel@nongnu.org; Tue, 21 Aug 2018 13:04:07 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fsA4U-0006iI-21 for qemu-devel@nongnu.org; Tue, 21 Aug 2018 13:04:00 -0400 Received: from mail-wm0-x231.google.com ([2a00:1450:400c:c09::231]:50857) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fsA4P-0005y7-1U for qemu-devel@nongnu.org; Tue, 21 Aug 2018 13:03:55 -0400 Received: by mail-wm0-x231.google.com with SMTP id s12-v6so3659467wmc.0 for ; Tue, 21 Aug 2018 10:03:27 -0700 (PDT) Received: from 640k.lan (dynamic-adsl-78-12-184-244.clienti.tiscali.it. [78.12.184.244]) by smtp.gmail.com with ESMTPSA id v6-v6sm2608955wmc.43.2018.08.21.10.03.24 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 21 Aug 2018 10:03:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references; bh=aKdbKm6MOitl3DHt96ctY+9n2ZPEKlbsVln9S2flHIU=; b=E7QBsFzvAtzJiZZJNN7KNmA+WngBme9u0ki6R0BG9spJFpEmssJCR2FSobPLaJTU+z EOL0bgEwIS8WSkgVzXpOGkVYUcrSQUNG/PJLRdt/7BP8nduOlYsPN9qNP4g3P0VtEa+o c+LJ98OS/0BpXLMrLpmm9ZCBVaM/Qu73BZ9RUInZ5qMN7k7GEEpqbDg7Ospim9Al/By9 HqoF68/26D+EHq7QDsTRYP6yGcUMHySOYvyhoj+3hdzQUKxbvl1h0kDXUbq+iK6Nhaui 9nCBaY+Me8N94jmqObgfVvdRoj8BcquqA5dkODWz7pURimTTt2cFdtxOacvv4OR6+0b6 PHfg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references; bh=aKdbKm6MOitl3DHt96ctY+9n2ZPEKlbsVln9S2flHIU=; b=mEkdgHmMEIH9BbU8egT8R6KTR2rDatwrh3B/DbdgrDOobxGmLMkAUr7GEBedJgrg/S dEd46BaUs1KFhsqWNQw/tHVbLf7mFgRQRu0Ncmj8JqI0j7nOpgvm1Zd4Xzp071Ts9mMi 5I6EtfeDWAAEK6R47lLSGcNIcjdmfkrqCO4CzXezXOXdUfdT9Tj3Tjlt4dwnb3XE+fyF gnqyrHa6RlwKpQGu9T9a6mZ+yEl1GfUT3dZ7s1SXApwWDL82n1kaqiBeNraF1vLDjKfS PL6WSHQ/YAylEaOsM7CPExZ6iKyiI+yXx5THMoAErRc5d6n12/8tLu3d/un4NLYmpOWu gCfA== X-Gm-Message-State: APzg51ANB2zc+Jy7DEe0dCTpt98u6aDEGy3Eb/jWZA1zRUrUi8Rk+4IQ 5NRvfsIdhaW1wEHEE7NjxDG/NvpE X-Google-Smtp-Source: ANB0VdY7+9kB/mJOpb9opukfemtdo1uDZ9kRFgN45+F2tK0Ljbhdm8VVHgevlOsyPeM84dd4xosaZQ== X-Received: by 2002:a1c:6d17:: with SMTP id i23-v6mr124513wmc.139.1534871006003; Tue, 21 Aug 2018 10:03:26 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Date: Tue, 21 Aug 2018 19:01:53 +0200 Message-Id: <1534870966-9287-22-git-send-email-pbonzini@redhat.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1534870966-9287-1-git-send-email-pbonzini@redhat.com> References: <1534870966-9287-1-git-send-email-pbonzini@redhat.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:400c:c09::231 Subject: [Qemu-devel] [PULL 21/74] qsp: support call site coalescing X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "Emilio G. Cota" Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZohoMail: RDMRC_1 RDKM_2 RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: "Emilio G. Cota" Signed-off-by: Emilio G. Cota Signed-off-by: Paolo Bonzini --- include/qemu/qsp.h | 2 +- util/qsp.c | 102 +++++++++++++++++++++++++++++++++++++++++++++----= ---- 2 files changed, 89 insertions(+), 15 deletions(-) diff --git a/include/qemu/qsp.h b/include/qemu/qsp.h index f8c6c96..a94c464 100644 --- a/include/qemu/qsp.h +++ b/include/qemu/qsp.h @@ -19,7 +19,7 @@ enum QSPSortBy { }; =20 void qsp_report(FILE *f, fprintf_function cpu_fprintf, size_t max, - enum QSPSortBy sort_by); + enum QSPSortBy sort_by, bool callsite_coalesce); =20 bool qsp_is_enabled(void); void qsp_enable(void); diff --git a/util/qsp.c b/util/qsp.c index 4b1308b..4dc851e 100644 --- a/util/qsp.c +++ b/util/qsp.c @@ -27,11 +27,9 @@ * Reports are generated as a table where each row represents a call site.= A * call site is the triplet formed by the __file__ and __LINE__ of the cal= ler * as well as the address of the "object" (i.e. mutex, rec. mutex or condv= ar) - * being operated on. Focusing on call sites instead of just on objects mi= ght - * seem puzzling. However, it is a sensible choice since otherwise dealing= with - * dynamically-allocated objects becomes difficult (e.g. what to do when an - * object is destroyed, or reused?). Furthermore, the call site info is of= most - * importance, since it is callers, and not objects, what cause wait time. + * being operated on. Optionally, call sites that operate on different obj= ects + * of the same type can be coalesced, which can be particularly useful when + * profiling dynamically-allocated objects. * * Alternative designs considered: * @@ -84,6 +82,7 @@ struct QSPEntry { const QSPCallSite *callsite; uint64_t n_acqs; uint64_t ns; + unsigned int n_objs; /* count of coalesced objs; only used for reporti= ng */ #ifndef CONFIG_ATOMIC64 /* * If we cannot update the counts atomically, then use a seqlock. @@ -170,6 +169,17 @@ static uint32_t qsp_entry_no_thread_hash(const QSPEntr= y *entry) return do_qsp_entry_hash(entry, 0); } =20 +/* without the objects we need to hash the file name to get a decent hash = */ +static uint32_t qsp_entry_no_thread_obj_hash(const QSPEntry *entry) +{ + const QSPCallSite *callsite =3D entry->callsite; + uint64_t a =3D g_str_hash(callsite->file); + uint64_t b =3D callsite->line; + uint32_t e =3D callsite->type; + + return tb_hash_func7(a, b, e, 0, 0); +} + static bool qsp_callsite_cmp(const void *ap, const void *bp) { const QSPCallSite *a =3D ap; @@ -182,6 +192,17 @@ static bool qsp_callsite_cmp(const void *ap, const voi= d *bp) (a->file =3D=3D b->file || !strcmp(a->file, b->file))); } =20 +static bool qsp_callsite_no_obj_cmp(const void *ap, const void *bp) +{ + const QSPCallSite *a =3D ap; + const QSPCallSite *b =3D bp; + + return a =3D=3D b || + (a->line =3D=3D b->line && + a->type =3D=3D b->type && + (a->file =3D=3D b->file || !strcmp(a->file, b->file))); +} + static bool qsp_entry_no_thread_cmp(const void *ap, const void *bp) { const QSPEntry *a =3D ap; @@ -190,6 +211,14 @@ static bool qsp_entry_no_thread_cmp(const void *ap, co= nst void *bp) return qsp_callsite_cmp(a->callsite, b->callsite); } =20 +static bool qsp_entry_no_thread_obj_cmp(const void *ap, const void *bp) +{ + const QSPEntry *a =3D ap; + const QSPEntry *b =3D bp; + + return qsp_callsite_no_obj_cmp(a->callsite, b->callsite); +} + static bool qsp_entry_cmp(const void *ap, const void *bp) { const QSPEntry *a =3D ap; @@ -548,15 +577,36 @@ static void qsp_diff(struct qht *orig, struct qht *ne= w) qht_iter(orig, qsp_iter_diff, new); } =20 +static void +qsp_iter_callsite_coalesce(struct qht *orig, void *p, uint32_t h, void *ht= p) +{ + struct qht *ht =3D htp; + QSPEntry *old =3D p; + QSPEntry *e; + uint32_t hash; + + hash =3D qsp_entry_no_thread_obj_hash(old); + e =3D qht_lookup(ht, old, hash); + if (e =3D=3D NULL) { + e =3D qsp_entry_create(ht, old, hash); + e->n_objs =3D 1; + } else if (e->callsite->obj !=3D old->callsite->obj) { + e->n_objs++; + } + e->ns +=3D old->ns; + e->n_acqs +=3D old->n_acqs; +} + static void qsp_ht_delete(struct qht *ht, void *p, uint32_t h, void *htp) { g_free(p); } =20 -static void qsp_mktree(GTree *tree) +static void qsp_mktree(GTree *tree, bool callsite_coalesce) { QSPSnapshot *snap; - struct qht ht; + struct qht ht, coalesce_ht; + struct qht *htp; =20 /* * First, see if there's a prior snapshot, so that we read the global = hash @@ -581,11 +631,23 @@ static void qsp_mktree(GTree *tree) /* done with the snapshot; RCU can reclaim it */ rcu_read_unlock(); =20 + htp =3D &ht; + if (callsite_coalesce) { + qht_init(&coalesce_ht, qsp_entry_no_thread_obj_cmp, QSP_INITIAL_SI= ZE, + QHT_MODE_AUTO_RESIZE | QHT_MODE_RAW_MUTEXES); + qht_iter(&ht, qsp_iter_callsite_coalesce, &coalesce_ht); + + /* free the previous hash table, and point htp to coalesce_ht */ + qht_iter(&ht, qsp_ht_delete, NULL); + qht_destroy(&ht); + htp =3D &coalesce_ht; + } + /* sort the hash table elements by using a tree */ - qht_iter(&ht, qsp_sort, tree); + qht_iter(htp, qsp_sort, tree); =20 /* free the hash table, but keep the elements (those are in the tree n= ow) */ - qht_destroy(&ht); + qht_destroy(htp); } =20 /* free string with g_free */ @@ -611,6 +673,7 @@ struct QSPReportEntry { double time_s; double ns_avg; uint64_t n_acqs; + unsigned int n_objs; }; typedef struct QSPReportEntry QSPReportEntry; =20 @@ -634,6 +697,7 @@ static gboolean qsp_tree_report(gpointer key, gpointer = value, gpointer udata) report->n_entries++; =20 entry->obj =3D e->callsite->obj; + entry->n_objs =3D e->n_objs; entry->callsite_at =3D qsp_at(e->callsite); entry->typename =3D qsp_typenames[e->callsite->type]; entry->time_s =3D e->ns * 1e-9; @@ -678,10 +742,20 @@ pr_report(const QSPReport *rep, FILE *f, fprintf_func= tion pr) =20 for (i =3D 0; i < rep->n_entries; i++) { const QSPReportEntry *e =3D &rep->entries[i]; + GString *s =3D g_string_new(NULL); =20 - pr(f, "%-9s %14p %s%*s %13.5f %12" PRIu64 " %12.2f\n", e->typ= ename, - e->obj, e->callsite_at, callsite_len - (int)strlen(e->callsite_= at), - "", e->time_s, e->n_acqs, e->ns_avg * 1e-3); + g_string_append_printf(s, "%-9s ", e->typename); + if (e->n_objs > 1) { + g_string_append_printf(s, "[%12u]", e->n_objs); + } else { + g_string_append_printf(s, "%14p", e->obj); + } + g_string_append_printf(s, " %s%*s %13.5f %12" PRIu64 " %12.2f\= n", + e->callsite_at, + callsite_len - (int)strlen(e->callsite_at),= "", + e->time_s, e->n_acqs, e->ns_avg * 1e-3); + pr(f, "%s", s->str); + g_string_free(s, TRUE); } =20 pr(f, "%s\n", dashes); @@ -701,7 +775,7 @@ static void report_destroy(QSPReport *rep) } =20 void qsp_report(FILE *f, fprintf_function cpu_fprintf, size_t max, - enum QSPSortBy sort_by) + enum QSPSortBy sort_by, bool callsite_coalesce) { GTree *tree =3D g_tree_new_full(qsp_tree_cmp, &sort_by, g_free, NULL); QSPReport rep; @@ -712,7 +786,7 @@ void qsp_report(FILE *f, fprintf_function cpu_fprintf, = size_t max, rep.n_entries =3D 0; rep.max_n_entries =3D max; =20 - qsp_mktree(tree); + qsp_mktree(tree, callsite_coalesce); g_tree_foreach(tree, qsp_tree_report, &rep); g_tree_destroy(tree); =20 --=20 1.8.3.1