From nobody Sat Nov 15 11:54:35 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=linaro.org ARC-Seal: i=1; a=rsa-sha256; t=1594670976; cv=none; d=zohomail.com; s=zohoarc; b=BcBtwaUl/xzDYZsWSWELMtRWxVX3LWS+HH3MK8rSk4kCt0CDRV1XYmiMjvlOS5Q2R6nBO5D1n8AjgthTlRzBLvzbbv9Ts83ZN7ELcMbJMi7AaF945cJBSHL2gBQO0Da9Fp6mu8NBUifoj5A52swa1/LWGf97SUyvLWoWRxcDmLs= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1594670976; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=Pdf6aU3FyC9r3Nn2r3fEZmyoTvKCMt4IaqLuhhG9jq4=; b=AobOUedeDLy5xx+/iUG4y2xVaLHUBYfrWcaj/+S6WJnWa8EGH7raIJZhgTSNVbgV7OUJleE78ShBlHHINAjxsYwEmyNFq/6n/SjaGtgqVY7ea1Hc6rKb40A0PC3xKBxo4/pIj2s6hD5N4if4MGXfGwXmp+bwci5oh6rctmgVV2E= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1594670976101936.842819689501; Mon, 13 Jul 2020 13:09:36 -0700 (PDT) Received: from localhost ([::1]:32958 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jv4la-0000ir-RN for importer@patchew.org; Mon, 13 Jul 2020 16:09:34 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:59584) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jv4gl-0008Lf-Gq for qemu-devel@nongnu.org; Mon, 13 Jul 2020 16:04:35 -0400 Received: from mail-wr1-x42e.google.com ([2a00:1450:4864:20::42e]:36918) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jv4gj-0006Ji-3P for qemu-devel@nongnu.org; Mon, 13 Jul 2020 16:04:35 -0400 Received: by mail-wr1-x42e.google.com with SMTP id a6so18135546wrm.4 for ; Mon, 13 Jul 2020 13:04:32 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id d81sm1348870wmc.0.2020.07.13.13.04.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 13 Jul 2020 13:04:23 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id A391A1FF9A; Mon, 13 Jul 2020 21:04:16 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Pdf6aU3FyC9r3Nn2r3fEZmyoTvKCMt4IaqLuhhG9jq4=; b=qMuu8NUF/b5XQzGj6rgCcShJ7YZW8kcH5Cws6JU6MI7d+6hvHv2O8wUnQxD18xN6Ib /GUBwiTEa4mi60XToVO4xC4Vv528IZZWbNckDvFA+EAeTtPf8lNWOU18vdJ9uR2ddDer 9MtAnIARhKWhMJSXUSOlWGNCfqok2+OPyzZ0xwUH0d3l6mjambBEH34WyN/HOBZxjtCS TBY8Vk9tCA3A0DsJ5p3hsMOYAdA3zhejqb+xWlM1Kz5nr/yaFyAJndYCrov4ZNZL84b3 4WVfMvz8Qbw9uBgPv08fMSe2/8/ZjihGHZJaVVw++tRYhUx5PKEzlAeWu4Z1535PuLXO zRrw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Pdf6aU3FyC9r3Nn2r3fEZmyoTvKCMt4IaqLuhhG9jq4=; b=MbVnJqJd6DP0ycZbtdSEiarivYN+Zfu6flCygS76VaNET7bd187quK0+EJpMGOQNW3 hYkOfKlY/XGmFcdV/JMgkxbXSzvhq65n95Ub2NSWphLYRDFLtEnJ23OLBdPu5tw8gTQA OE8ctp0kVxnwHmsKd9eXSKp6GM9wh6X51oU3/NJ8Qb8vYLlfycNVBxXnCN2HAo+YmFP8 jbwN+syBNKdRI76nklr12K+DJLndf0p23L0X0rH/CinmFxRBLi5pDGHpDwFAYJ2fI/9x nivcAp7BGFIM/HkHHtb1ss7JtWrqUz2thSybKrOdOfGvXRVN8coB1PQfOC4Pu+5tTvmp i4gw== X-Gm-Message-State: AOAM533ZZzLU/YVjMqp/ccpe9OxBVCyW5HMjGKTmHFqrK9Fs0CgzT1fZ kdwmpnnjHzE+KLnHtkWpLrYaGA== X-Google-Smtp-Source: ABdhPJwVxlhQ9mVVhwT4VIn7GVK6w/nxlGINZcKsC992JVrFpK6Zxjo0UMSxUODSRdT1QtVrc4vJ5A== X-Received: by 2002:adf:d0d0:: with SMTP id z16mr1230452wrh.95.1594670671470; Mon, 13 Jul 2020 13:04:31 -0700 (PDT) From: =?UTF-8?q?Alex=20Benn=C3=A9e?= To: qemu-devel@nongnu.org Subject: [PATCH v2 11/11] plugins: new hwprofile plugin Date: Mon, 13 Jul 2020 21:04:15 +0100 Message-Id: <20200713200415.26214-12-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200713200415.26214-1-alex.bennee@linaro.org> References: <20200713200415.26214-1-alex.bennee@linaro.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::42e; envelope-from=alex.bennee@linaro.org; helo=mail-wr1-x42e.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, berrange@redhat.com, robert.foley@linaro.org, =?UTF-8?q?Alex=20Benn=C3=A9e?= , richard.henderson@linaro.org, f4bug@amsat.org, robhenry@microsoft.com, =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , aaron@os.amperecomputing.com, cota@braap.org, kuhn.chenqun@huawei.com, peter.puhov@linaro.org, aurelien@aurel32.net Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) This is a plugin intended to help with profiling access to various bits of system hardware. It only really makes sense for system emulation. It takes advantage of the recently exposed helper API that allows us to see the device name (memory region name) associated with a device. You can specify arg=3Dread or arg=3Dwrite to limit the tracking to just reads or writes (by default it does both). The pattern option: -plugin ./tests/plugin/libhwprofile.so,arg=3Dpattern will allow you to see the access pattern to devices, eg: gic_cpu @ 0xffffffc010040000 off:00000000, 8, 1, 8, 1 off:00000000, 4, 1, 4, 1 off:00000000, 2, 1, 2, 1 off:00000000, 1, 1, 1, 1 The source option: -plugin ./tests/plugin/libhwprofile.so,arg=3Dsource will track the virtual source address of the instruction making the access: pl011 @ 0xffffffc010031000 pc:ffffffc0104c785c, 1, 4, 0, 0 pc:ffffffc0104c7898, 1, 4, 0, 0 pc:ffffffc010512bcc, 2, 1867, 0, 0 You cannot mix source and pattern. Finally the match option allow you to limit the tracking to just the devices you care about. Signed-off-by: Alex Benn=C3=A9e Reviewed-by: Robert Foley Tested-by: Robert Foley --- v2 - use 64 bit cpu masks - warn if we will exceed cpu mask capability - don't run in linux-user mode - skip in check-tcg for linux-user test v3 - update device name API - re-factor and clean-up - add source tracking - use direct hash for source/pattern - add match function - expand the commit message - checkpatch cleanups --- tests/plugin/hwprofile.c | 305 ++++++++++++++++++++++++++++++++++++++ tests/plugin/Makefile | 1 + tests/tcg/Makefile.target | 12 +- 3 files changed, 316 insertions(+), 2 deletions(-) create mode 100644 tests/plugin/hwprofile.c diff --git a/tests/plugin/hwprofile.c b/tests/plugin/hwprofile.c new file mode 100644 index 0000000000..6dac1d5f85 --- /dev/null +++ b/tests/plugin/hwprofile.c @@ -0,0 +1,305 @@ +/* + * Copyright (C) 2020, Alex Benn=C3=A9e + * + * HW Profile - breakdown access patterns for IO to devices + * + * License: GNU GPL, version 2 or later. + * See the COPYING file in the top-level directory. + */ + +#include +#include +#include +#include +#include +#include +#include +#include + +#include + +QEMU_PLUGIN_EXPORT int qemu_plugin_version =3D QEMU_PLUGIN_VERSION; + +#define ARRAY_SIZE(x) (sizeof(x) / sizeof((x)[0])) + +typedef struct { + uint64_t cpu_read; + uint64_t cpu_write; + uint64_t reads; + uint64_t writes; +} IOCounts; + +typedef struct { + uint64_t off_or_pc; + IOCounts counts; +} IOLocationCounts; + +typedef struct { + const char *name; + uint64_t base; + IOCounts totals; + GHashTable *detail; +} DeviceCounts; + +static GMutex lock; +static GHashTable *devices; + +/* track the access pattern to a piece of HW */ +static bool pattern; +/* track the source address of access to HW */ +static bool source; +/* track only matched regions of HW */ +static bool check_match; +static gchar **matches; + +static enum qemu_plugin_mem_rw rw =3D QEMU_PLUGIN_MEM_RW; + +static inline bool track_reads(void) +{ + return rw =3D=3D QEMU_PLUGIN_MEM_RW || rw =3D=3D QEMU_PLUGIN_MEM_R; +} + +static inline bool track_writes(void) +{ + return rw =3D=3D QEMU_PLUGIN_MEM_RW || rw =3D=3D QEMU_PLUGIN_MEM_W; +} + +static void plugin_init(void) +{ + devices =3D g_hash_table_new(NULL, NULL); +} + +static gint sort_cmp(gconstpointer a, gconstpointer b) +{ + DeviceCounts *ea =3D (DeviceCounts *) a; + DeviceCounts *eb =3D (DeviceCounts *) b; + return ea->totals.reads + ea->totals.writes > + eb->totals.reads + eb->totals.writes ? -1 : 1; +} + +static gint sort_loc(gconstpointer a, gconstpointer b) +{ + IOLocationCounts *ea =3D (IOLocationCounts *) a; + IOLocationCounts *eb =3D (IOLocationCounts *) b; + return ea->off_or_pc > eb->off_or_pc; +} + +static void fmt_iocount_record(GString *s, IOCounts *rec) +{ + if (track_reads()) { + g_string_append_printf(s, ", %"PRIx64", %"PRId64, + rec->cpu_read, rec->reads); + } + if (track_writes()) { + g_string_append_printf(s, ", %"PRIx64", %"PRId64, + rec->cpu_write, rec->writes); + } +} + +static void fmt_dev_record(GString *s, DeviceCounts *rec) +{ + g_string_append_printf(s, "%s, 0x%"PRIx64, + rec->name, rec->base); + fmt_iocount_record(s, &rec->totals); + g_string_append_c(s, '\n'); +} + +static void plugin_exit(qemu_plugin_id_t id, void *p) +{ + g_autoptr(GString) report =3D g_string_new(""); + GList *counts; + + if (!(pattern || source)) { + g_string_printf(report, "Device, Address"); + if (track_reads()) { + g_string_append_printf(report, ", RCPUs, Reads"); + } + if (track_writes()) { + g_string_append_printf(report, ", WCPUs, Writes"); + } + g_string_append_c(report, '\n'); + } + + counts =3D g_hash_table_get_values(devices); + if (counts && g_list_next(counts)) { + GList *it; + + it =3D g_list_sort(counts, sort_cmp); + + while (it) { + DeviceCounts *rec =3D (DeviceCounts *) it->data; + if (rec->detail) { + GList *accesses =3D g_hash_table_get_values(rec->detail); + GList *io_it =3D g_list_sort(accesses, sort_loc); + const char *prefix =3D pattern ? "off" : "pc"; + g_string_append_printf(report, "%s @ 0x%"PRIx64"\n", + rec->name, rec->base); + while (io_it) { + IOLocationCounts *loc =3D (IOLocationCounts *) io_it->= data; + g_string_append_printf(report, " %s:%08"PRIx64, + prefix, loc->off_or_pc); + fmt_iocount_record(report, &loc->counts); + g_string_append_c(report, '\n'); + io_it =3D io_it->next; + } + } else { + fmt_dev_record(report, rec); + } + it =3D it->next; + }; + g_list_free(it); + } + + qemu_plugin_outs(report->str); +} + +static DeviceCounts *new_count(const char *name, uint64_t base) +{ + DeviceCounts *count =3D g_new0(DeviceCounts, 1); + count->name =3D name; + count->base =3D base; + if (pattern || source) { + count->detail =3D g_hash_table_new(NULL, NULL); + } + g_hash_table_insert(devices, (gpointer) name, count); + return count; +} + +static IOLocationCounts *new_location(GHashTable *table, uint64_t off_or_p= c) +{ + IOLocationCounts *loc =3D g_new0(IOLocationCounts, 1); + loc->off_or_pc =3D off_or_pc; + g_hash_table_insert(table, (gpointer) off_or_pc, loc); + return loc; +} + +static void hwprofile_match_hit(DeviceCounts *rec, uint64_t off) +{ + g_autoptr(GString) report =3D g_string_new("hwprofile: match @ offset"= ); + g_string_append_printf(report, "%"PRIx64", previous hits\n", off); + fmt_dev_record(report, rec); + qemu_plugin_outs(report->str); +} + +static void inc_count(IOCounts *count, bool is_write, unsigned int cpu_ind= ex) +{ + if (is_write) { + count->writes++; + count->cpu_write |=3D (1 << cpu_index); + } else { + count->reads++; + count->cpu_read |=3D (1 << cpu_index); + } +} + +static void vcpu_haddr(unsigned int cpu_index, qemu_plugin_meminfo_t memin= fo, + uint64_t vaddr, void *udata) +{ + struct qemu_plugin_hwaddr *hwaddr =3D qemu_plugin_get_hwaddr(meminfo, = vaddr); + + if (!hwaddr || !qemu_plugin_hwaddr_is_io(hwaddr)) { + return; + } else { + const char *name =3D qemu_plugin_hwaddr_device_name(hwaddr); + uint64_t off =3D qemu_plugin_hwaddr_device_offset(hwaddr); + bool is_write =3D qemu_plugin_mem_is_store(meminfo); + DeviceCounts *counts; + + g_mutex_lock(&lock); + counts =3D (DeviceCounts *) g_hash_table_lookup(devices, name); + + if (!counts) { + uint64_t base =3D vaddr - off; + counts =3D new_count(name, base); + } + + if (check_match) { + if (g_strv_contains((const char * const *)matches, counts->nam= e)) { + hwprofile_match_hit(counts, off); + inc_count(&counts->totals, is_write, cpu_index); + } + } else { + inc_count(&counts->totals, is_write, cpu_index); + } + + /* either track offsets or source of access */ + if (source) { + off =3D (uint64_t) udata; + } + + if (pattern || source) { + IOLocationCounts *io_count =3D g_hash_table_lookup(counts->det= ail, + (gpointer) of= f); + if (!io_count) { + io_count =3D new_location(counts->detail, off); + } + inc_count(&io_count->counts, is_write, cpu_index); + } + + g_mutex_unlock(&lock); + } +} + +static void vcpu_tb_trans(qemu_plugin_id_t id, struct qemu_plugin_tb *tb) +{ + size_t n =3D qemu_plugin_tb_n_insns(tb); + size_t i; + + for (i =3D 0; i < n; i++) { + struct qemu_plugin_insn *insn =3D qemu_plugin_tb_get_insn(tb, i); + gpointer udata =3D (gpointer) (source ? qemu_plugin_insn_vaddr(ins= n) : 0); + qemu_plugin_register_vcpu_mem_cb(insn, vcpu_haddr, + QEMU_PLUGIN_CB_NO_REGS, + rw, udata); + } +} + +QEMU_PLUGIN_EXPORT +int qemu_plugin_install(qemu_plugin_id_t id, const qemu_info_t *info, + int argc, char **argv) +{ + int i; + + for (i =3D 0; i < argc; i++) { + char *opt =3D argv[i]; + if (g_strcmp0(opt, "read") =3D=3D 0) { + rw =3D QEMU_PLUGIN_MEM_R; + } else if (g_strcmp0(opt, "write") =3D=3D 0) { + rw =3D QEMU_PLUGIN_MEM_W; + } else if (g_strcmp0(opt, "pattern") =3D=3D 0) { + pattern =3D true; + } else if (g_strcmp0(opt, "source") =3D=3D 0) { + source =3D true; + } else if (g_str_has_prefix(opt, "match")) { + gchar **parts =3D g_strsplit(opt, "=3D", 2); + check_match =3D true; + matches =3D g_strsplit(parts[1], ",", -1); + g_strfreev(parts); + } else { + fprintf(stderr, "option parsing failed: %s\n", opt); + return -1; + } + } + + if (source && pattern) { + fprintf(stderr, "can only currently track either source or pattern= .\n"); + return -1; + } + + if (!info->system_emulation) { + fprintf(stderr, "hwprofile: plugin only useful for system emulatio= n\n"); + return -1; + } + + /* Just warn about overflow */ + if (info->system.smp_vcpus > 64 || + info->system.max_vcpus > 64) { + fprintf(stderr, "hwprofile: can only track up to 64 CPUs\n"); + } + + plugin_init(); + + qemu_plugin_register_vcpu_tb_trans_cb(id, vcpu_tb_trans); + qemu_plugin_register_atexit_cb(id, plugin_exit, NULL); + return 0; +} diff --git a/tests/plugin/Makefile b/tests/plugin/Makefile index e9348fde4a..e9d1947751 100644 --- a/tests/plugin/Makefile +++ b/tests/plugin/Makefile @@ -21,6 +21,7 @@ NAMES +=3D hotblocks NAMES +=3D howvec NAMES +=3D hotpages NAMES +=3D lockstep +NAMES +=3D hwprofile =20 SONAMES :=3D $(addsuffix .so,$(addprefix lib,$(NAMES))) =20 diff --git a/tests/tcg/Makefile.target b/tests/tcg/Makefile.target index 4b2b696fce..bc595b0588 100644 --- a/tests/tcg/Makefile.target +++ b/tests/tcg/Makefile.target @@ -129,8 +129,16 @@ ifeq ($(CONFIG_PLUGIN),y) PLUGIN_SRC=3D$(SRC_PATH)/tests/plugin PLUGIN_LIB=3D../../plugin VPATH+=3D$(PLUGIN_LIB) -PLUGINS=3D$(filter-out liblockstep.so,\ - $(patsubst %.c, lib%.so, $(notdir $(wildcard $(PLUGIN_SRC)/*.c)))) + +# Some plugins aren't testable automatically +SKIP_PLUGINS=3Dliblockstep.so +ifdef CONFIG_USER_ONLY +SKIP_PLUGINS+=3Dlibhwprofile.so +endif + +PLUGINS=3D$(filter-out $(SKIP_PLUGINS),\ + $(patsubst %.c, lib%.so, \ + $(notdir $(wildcard $(PLUGIN_SRC)/*.c)))) =20 # We need to ensure expand the run-plugin-TEST-with-PLUGIN # pre-requistes manually here as we can't use stems to handle it. We --=20 2.20.1