From nobody Sun Nov 24 06:00:00 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.libvirt.org designates 8.43.85.245 as permitted sender) client-ip=8.43.85.245; envelope-from=devel-bounces@lists.libvirt.org; helo=lists.libvirt.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of lists.libvirt.org designates 8.43.85.245 as permitted sender) smtp.mailfrom=devel-bounces@lists.libvirt.org; dmarc=fail(p=none dis=none) header.from=linaro.org Return-Path: Received: from lists.libvirt.org (lists.libvirt.org [8.43.85.245]) by mx.zohomail.com with SMTPS id 1726476953772161.64545869131734; Mon, 16 Sep 2024 01:55:53 -0700 (PDT) Received: by lists.libvirt.org (Postfix, from userid 996) id A557C1914; Mon, 16 Sep 2024 04:55:52 -0400 (EDT) Received: from lists.libvirt.org (localhost [IPv6:::1]) by lists.libvirt.org (Postfix) with ESMTP id 4F07E1A4C; Mon, 16 Sep 2024 04:54:11 -0400 (EDT) Received: by lists.libvirt.org (Postfix, from userid 996) id 05C1213FE; Mon, 16 Sep 2024 04:54:05 -0400 (EDT) Received: from mail-wr1-f43.google.com (mail-wr1-f43.google.com [209.85.221.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.libvirt.org (Postfix) with ESMTPS id DB3BF13E2 for ; Mon, 16 Sep 2024 04:54:03 -0400 (EDT) Received: by mail-wr1-f43.google.com with SMTP id ffacd0b85a97d-3770320574aso3145937f8f.2 for ; Mon, 16 Sep 2024 01:54:03 -0700 (PDT) Received: from draig.lan ([85.9.250.243]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-378e780e029sm6574810f8f.116.2024.09.16.01.54.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 16 Sep 2024 01:54:01 -0700 (PDT) Received: from draig.lan (localhost [IPv6:::1]) by draig.lan (Postfix) with ESMTP id 8C23C5F8FD; Mon, 16 Sep 2024 09:54:00 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on lists.libvirt.org X-Spam-Level: X-Spam-Status: No, score=-1.6 required=5.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2,SPF_HELO_NONE autolearn=unavailable autolearn_force=no version=3.4.4 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1726476843; x=1727081643; darn=lists.libvirt.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=bsYjToh00NCkqztzoMGpy1iwtNBoYXNP8xjda/hjgcg=; b=epOJQha5fjTen9g0qp10SeCh1WdR6GcqCuSCr43J/NEL4ZcYKgddPv/sZE2jydIJMo K1U+n0B+E8/dL6USss+9EV9pPZsP/WAo21u7ZdPZj+c/j0CwbMt89KzYHS0P5y0mYDw/ 985H4WPQOxDnTOVnQA8XVJgei00F/nG48PaJeHqbAqHLB1pIN5tRuPLG95TFQRt3+6RU Oa0EWA7kN7NEsIUwrzeF1FkyXDrpDQfuEiEPDUFWLyKgCcNIQ2+bDFuyRHe09hflBovR PmTeg5aYuDrAJLf0btFxVUqE2kEAW424sCkUAfqoFXJMafDrKg4JyJMmcDT4jtKNFji+ YCAA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1726476843; x=1727081643; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=bsYjToh00NCkqztzoMGpy1iwtNBoYXNP8xjda/hjgcg=; b=t5wWAaSsN1Fg5plQbSCJnGSQ3Ujdnj0Diiz7JWqa5kqPrNFXbFmFKdMxsqXBqdctrH 3D/72t+kxTXjoLif7+a3HePSX9kNJGXiEzOJI4qah7Uy2ipfL6SzYsg93i69BREhRMbV IiLCnQ2HKiUoIVbl3eEZRiW/Uys2r70rZCnrUZ4bbfQYsHy0H+QFfxHCGjjlPd+9vUi9 BPhIKqT3RDtiRfR5UZTEH1+DBfItJ3mGPhXB0TEk7PPS/nefKFBjbMWfmzWRnWVyBmKl 826qrlzALPED57reJGIo08NI9amcuq0UffQm4x642KM3pjyB6EC21viYu/Il6nloOyBS cdvg== X-Forwarded-Encrypted: i=1; AJvYcCUZgqELJIe+EYxkwL7uYmemslSl3fT1ZPxKx0Wz6KXfJMZNTnIXLqwKFkQimGZgJSMA+6XsQA==@lists.libvirt.org X-Gm-Message-State: AOJu0YwWzB/uvQHW4eP9X6XWigqmZOfivRUJFOZlHsjGDu6matONw6gw EDqVMFSXWiap/hoj3a/jzbjfD32xIIUvDBgdQY3iUmGHMqgFg/DUSHEA1UxiGZE= X-Google-Smtp-Source: AGHT+IE/6gNvAOBAjtlo83VoX375EYWnnZj4iIV3Pym+AC4YXXDlNiWFowhgM6SnhmEWnt2sy2CXiA== X-Received: by 2002:a5d:6183:0:b0:374:c454:dbb3 with SMTP id ffacd0b85a97d-378c2d620femr7509098f8f.55.1726476842277; Mon, 16 Sep 2024 01:54:02 -0700 (PDT) From: =?UTF-8?q?Alex=20Benn=C3=A9e?= To: qemu-devel@nongnu.org Subject: [PATCH v2 03/18] contrib/plugins: control flow plugin Date: Mon, 16 Sep 2024 09:53:45 +0100 Message-Id: <20240916085400.1046925-4-alex.bennee@linaro.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20240916085400.1046925-1-alex.bennee@linaro.org> References: <20240916085400.1046925-1-alex.bennee@linaro.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Message-ID-Hash: K5FWRS6RLPEFWTKLQSD5CZ7N7AJH3QHO X-Message-ID-Hash: K5FWRS6RLPEFWTKLQSD5CZ7N7AJH3QHO X-MailFrom: alex.bennee@linaro.org X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-config-1; header-match-config-2; header-match-config-3; header-match-devel.lists.libvirt.org-0 CC: qemu-s390x@nongnu.org, Thomas Huth , Richard Henderson , qemu-ppc@nongnu.org, Ilya Leoshkevich , Paolo Bonzini , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Marcel Apfelbaum , Alexandre Iooss , Eduardo Habkost , Daniel Henrique Barboza , Yanan Wang , Nicholas Piggin , David Hildenbrand , devel@lists.libvirt.org, Mahmoud Mandour , Zhao Liu , =?UTF-8?q?Alex=20Benn=C3=A9e?= , Pierrick Bouvier , Gustavo Romero X-Mailman-Version: 3.2.2 Precedence: list List-Id: Development discussions about the libvirt library & tools Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1726476954995116600 Content-Type: text/plain; charset="utf-8" This is a simple control flow tracking plugin that uses the latest inline and conditional operations to detect and track control flow changes. It is currently an exercise at seeing how useful the changes are. Reviewed-by: Pierrick Bouvier Message-Id: <20240910140733.4007719-14-alex.bennee@linaro.org> Based-on: <20240312075428.244210-1-pierrick.bouvier@linaro.org> Cc: Gustavo Romero Signed-off-by: Alex Benn=C3=A9e --- v2 - only need a single call back - drop need for INSN_WIDTH - still don't understand the early exits v3 - move initial STORE ops to first instruction to avoid confusion with the conditional callback on the start - filter out non-branches before processing - fix off-by-one with accounting - display "sync fault" or "branch" instead of raw numbers v4 - rename hotdest to hottest (i.e. the hottest branch insn) - rename early to exception - WIP insn structure v5 - dropped #if 0 code and unused vars --- contrib/plugins/cflow.c | 384 +++++++++++++++++++++++++++++++++++++++ contrib/plugins/Makefile | 1 + 2 files changed, 385 insertions(+) create mode 100644 contrib/plugins/cflow.c diff --git a/contrib/plugins/cflow.c b/contrib/plugins/cflow.c new file mode 100644 index 0000000000..c0dc90b0fe --- /dev/null +++ b/contrib/plugins/cflow.c @@ -0,0 +1,384 @@ +/* + * Control Flow plugin + * + * This plugin will track changes to control flow and detect where + * instructions fault. + * + * Copyright (c) 2024 Linaro Ltd + * + * SPDX-License-Identifier: GPL-2.0-or-later + */ +#include +#include +#include +#include +#include +#include + +#include + +QEMU_PLUGIN_EXPORT int qemu_plugin_version =3D QEMU_PLUGIN_VERSION; + +typedef enum { + SORT_HOTTEST, /* hottest branch insn */ + SORT_EXCEPTION, /* most early exits */ + SORT_POPDEST, /* most destinations (usually ret's) */ +} ReportType; + +ReportType report =3D SORT_HOTTEST; +int topn =3D 10; + +typedef struct { + uint64_t daddr; + uint64_t dcount; +} DestData; + +/* A node is an address where we can go to multiple places */ +typedef struct { + GMutex lock; + /* address of the branch point */ + uint64_t addr; + /* array of DestData */ + GArray *dests; + /* early exit/fault count */ + uint64_t early_exit; + /* jump destination count */ + uint64_t dest_count; + /* instruction data */ + char *insn_disas; + /* symbol? */ + const char *symbol; + /* times translated as last in block? */ + int last_count; + /* times translated in the middle of block? */ + int mid_count; +} NodeData; + +typedef enum { + /* last insn in block, expected flow control */ + LAST_INSN =3D (1 << 0), + /* mid-block insn, can only be an exception */ + EXCP_INSN =3D (1 << 1), + /* multiple disassembly, may have changed */ + MULT_INSN =3D (1 << 2), +} InsnTypes; + +typedef struct { + /* address of the branch point */ + uint64_t addr; + /* disassembly */ + char *insn_disas; + /* symbol? */ + const char *symbol; + /* types */ + InsnTypes type_flag; +} InsnData; + +/* We use this to track the current execution state */ +typedef struct { + /* address of end of block */ + uint64_t end_block; + /* next pc after end of block */ + uint64_t pc_after_block; + /* address of last executed PC */ + uint64_t last_pc; +} VCPUScoreBoard; + +/* descriptors for accessing the above scoreboard */ +static qemu_plugin_u64 end_block; +static qemu_plugin_u64 pc_after_block; +static qemu_plugin_u64 last_pc; + + +static GMutex node_lock; +static GHashTable *nodes; +struct qemu_plugin_scoreboard *state; + +/* SORT_HOTTEST */ +static gint hottest(gconstpointer a, gconstpointer b) +{ + NodeData *na =3D (NodeData *) a; + NodeData *nb =3D (NodeData *) b; + + return na->dest_count > nb->dest_count ? -1 : + na->dest_count =3D=3D nb->dest_count ? 0 : 1; +} + +static gint exception(gconstpointer a, gconstpointer b) +{ + NodeData *na =3D (NodeData *) a; + NodeData *nb =3D (NodeData *) b; + + return na->early_exit > nb->early_exit ? -1 : + na->early_exit =3D=3D nb->early_exit ? 0 : 1; +} + +static gint popular(gconstpointer a, gconstpointer b) +{ + NodeData *na =3D (NodeData *) a; + NodeData *nb =3D (NodeData *) b; + + return na->dests->len > nb->dests->len ? -1 : + na->dests->len =3D=3D nb->dests->len ? 0 : 1; +} + +/* Filter out non-branches - returns true to remove entry */ +static gboolean filter_non_branches(gpointer key, gpointer value, gpointer= user_data) +{ + NodeData *node =3D (NodeData *) value; + + return node->dest_count =3D=3D 0; +} + +static void plugin_exit(qemu_plugin_id_t id, void *p) +{ + g_autoptr(GString) result =3D g_string_new("collected "); + GList *data; + GCompareFunc sort =3D &hottest; + int n =3D 0; + + g_mutex_lock(&node_lock); + g_string_append_printf(result, "%d control flow nodes in the hash tabl= e\n", + g_hash_table_size(nodes)); + + /* remove all nodes that didn't branch */ + g_hash_table_foreach_remove(nodes, filter_non_branches, NULL); + + data =3D g_hash_table_get_values(nodes); + + switch (report) { + case SORT_HOTTEST: + sort =3D &hottest; + break; + case SORT_EXCEPTION: + sort =3D &exception; + break; + case SORT_POPDEST: + sort =3D &popular; + break; + } + + data =3D g_list_sort(data, sort); + + for (GList *l =3D data; + l !=3D NULL && n < topn; + l =3D l->next, n++) { + NodeData *n =3D l->data; + const char *type =3D n->mid_count ? "sync fault" : "branch"; + g_string_append_printf(result, " addr: 0x%"PRIx64 " %s: %s (%s)\n= ", + n->addr, n->symbol, n->insn_disas, type); + if (n->early_exit) { + g_string_append_printf(result, " early exits %"PRId64"\n", + n->early_exit); + } + g_string_append_printf(result, " branches %"PRId64"\n", + n->dest_count); + for (int j =3D 0; j < n->dests->len; j++ ) { + DestData *dd =3D &g_array_index(n->dests, DestData, j); + g_string_append_printf(result, " to 0x%"PRIx64" (%"PRId64= ")\n", + dd->daddr, dd->dcount); + } + } + + qemu_plugin_outs(result->str); + + g_mutex_unlock(&node_lock); +} + +static void plugin_init(void) +{ + g_mutex_init(&node_lock); + nodes =3D g_hash_table_new(NULL, g_direct_equal); + state =3D qemu_plugin_scoreboard_new(sizeof(VCPUScoreBoard)); + + /* score board declarations */ + end_block =3D qemu_plugin_scoreboard_u64_in_struct(state, VCPUScoreBoa= rd, end_block); + pc_after_block =3D qemu_plugin_scoreboard_u64_in_struct(state, VCPUSco= reBoard, pc_after_block); + last_pc =3D qemu_plugin_scoreboard_u64_in_struct(state, VCPUScoreBoard= , last_pc); +} + +static NodeData *create_node(uint64_t addr) +{ + NodeData *node =3D g_new0(NodeData, 1); + g_mutex_init(&node->lock); + node->addr =3D addr; + node->dests =3D g_array_new(true, true, sizeof(DestData)); + return node; +} + +static NodeData *fetch_node(uint64_t addr, bool create_if_not_found) +{ + NodeData *node =3D NULL; + + g_mutex_lock(&node_lock); + node =3D (NodeData *) g_hash_table_lookup(nodes, (gconstpointer) addr); + if (!node && create_if_not_found) { + node =3D create_node(addr); + g_hash_table_insert(nodes, (gpointer) addr, (gpointer) node); + } + g_mutex_unlock(&node_lock); + return node; +} + +/* + * Called when we detect a non-linear execution (pc !=3D + * pc_after_block). This could be due to a fault causing some sort of + * exit exception (if last_pc !=3D block_end) or just a taken branch. + */ +static void vcpu_tb_branched_exec(unsigned int cpu_index, void *udata) +{ + uint64_t lpc =3D qemu_plugin_u64_get(last_pc, cpu_index); + uint64_t ebpc =3D qemu_plugin_u64_get(end_block, cpu_index); + uint64_t npc =3D qemu_plugin_u64_get(pc_after_block, cpu_index); + uint64_t pc =3D GPOINTER_TO_UINT(udata); + + /* return early for address 0 */ + if (!lpc) { + return; + } + + NodeData *node =3D fetch_node(lpc, true); + DestData *data =3D NULL; + bool early_exit =3D (lpc !=3D ebpc); + GArray *dests; + + /* the condition should never hit */ + g_assert(pc !=3D npc); + + g_mutex_lock(&node->lock); + + if (early_exit) { + fprintf(stderr, "%s: pc=3D%"PRIx64", epbc=3D%"PRIx64 + " npc=3D%"PRIx64", lpc=3D%"PRIx64", \n", + __func__, pc, ebpc, npc, lpc); + node->early_exit++; + if (!node->mid_count) { + /* count now as we've only just allocated */ + node->mid_count++; + } + } + + dests =3D node->dests; + for (int i =3D 0; i < dests->len; i++) { + if (g_array_index(dests, DestData, i).daddr =3D=3D pc) { + data =3D &g_array_index(dests, DestData, i); + } + } + + /* we've never seen this before, allocate a new entry */ + if (!data) { + DestData new_entry =3D { .daddr =3D pc }; + g_array_append_val(dests, new_entry); + data =3D &g_array_index(dests, DestData, dests->len - 1); + g_assert(data->daddr =3D=3D pc); + } + + data->dcount++; + node->dest_count++; + + g_mutex_unlock(&node->lock); +} + +/* + * At the start of each block we need to resolve two things: + * + * - is last_pc =3D=3D block_end, if not we had an early exit + * - is start of block last_pc + insn width, if not we jumped + * + * Once those are dealt with we can instrument the rest of the + * instructions for their execution. + * + */ +static void vcpu_tb_trans(qemu_plugin_id_t id, struct qemu_plugin_tb *tb) +{ + uint64_t pc =3D qemu_plugin_tb_vaddr(tb); + size_t insns =3D qemu_plugin_tb_n_insns(tb); + struct qemu_plugin_insn *first_insn =3D qemu_plugin_tb_get_insn(tb, 0); + struct qemu_plugin_insn *last_insn =3D qemu_plugin_tb_get_insn(tb, ins= ns - 1); + + /* + * check if we are executing linearly after the last block. We can + * handle both early block exits and normal branches in the + * callback if we hit it. + */ + gpointer udata =3D GUINT_TO_POINTER(pc); + qemu_plugin_register_vcpu_tb_exec_cond_cb( + tb, vcpu_tb_branched_exec, QEMU_PLUGIN_CB_NO_REGS, + QEMU_PLUGIN_COND_NE, pc_after_block, pc, udata); + + /* + * Now we can set start/end for this block so the next block can + * check where we are at. Do this on the first instruction and not + * the TB so we don't get mixed up with above. + */ + qemu_plugin_register_vcpu_insn_exec_inline_per_vcpu(first_insn, + QEMU_PLUGIN_INLINE_S= TORE_U64, + end_block, qemu_plug= in_insn_vaddr(last_insn)); + qemu_plugin_register_vcpu_insn_exec_inline_per_vcpu(first_insn, + QEMU_PLUGIN_INLINE_S= TORE_U64, + pc_after_block, + qemu_plugin_insn_vad= dr(last_insn) + + qemu_plugin_insn_siz= e(last_insn)); + + for (int idx =3D 0; idx < qemu_plugin_tb_n_insns(tb); ++idx) { + struct qemu_plugin_insn *insn =3D qemu_plugin_tb_get_insn(tb, idx); + uint64_t ipc =3D qemu_plugin_insn_vaddr(insn); + /* + * If this is a potential branch point check if we could grab + * the disassembly for it. If it is the last instruction + * always create an entry. + */ + NodeData *node =3D fetch_node(ipc, last_insn); + if (node) { + g_mutex_lock(&node->lock); + if (!node->insn_disas) { + node->insn_disas =3D qemu_plugin_insn_disas(insn); + } + if (!node->symbol) { + node->symbol =3D qemu_plugin_insn_symbol(insn); + } + if (last_insn =3D=3D insn) { + node->last_count++; + } else { + node->mid_count++; + } + g_mutex_unlock(&node->lock); + } + + /* Store the PC of what we are about to execute */ + qemu_plugin_register_vcpu_insn_exec_inline_per_vcpu(insn, + QEMU_PLUGIN_IN= LINE_STORE_U64, + last_pc, ipc); + } +} + +QEMU_PLUGIN_EXPORT +int qemu_plugin_install(qemu_plugin_id_t id, const qemu_info_t *info, + int argc, char **argv) +{ + for (int i =3D 0; i < argc; i++) { + char *opt =3D argv[i]; + g_auto(GStrv) tokens =3D g_strsplit(opt, "=3D", 2); + if (g_strcmp0(tokens[0], "sort") =3D=3D 0) { + if (g_strcmp0(tokens[1], "hottest") =3D=3D 0) { + report =3D SORT_HOTTEST; + } else if (g_strcmp0(tokens[1], "early") =3D=3D 0) { + report =3D SORT_EXCEPTION; + } else if (g_strcmp0(tokens[1], "exceptions") =3D=3D 0) { + report =3D SORT_POPDEST; + } else { + fprintf(stderr, "failed to parse: %s\n", tokens[1]); + return -1; + } + } else { + fprintf(stderr, "option parsing failed: %s\n", opt); + return -1; + } + } + + plugin_init(); + + qemu_plugin_register_vcpu_tb_trans_cb(id, vcpu_tb_trans); + qemu_plugin_register_atexit_cb(id, plugin_exit, NULL); + return 0; +} diff --git a/contrib/plugins/Makefile b/contrib/plugins/Makefile index 05a2a45c5c..d4ac599f93 100644 --- a/contrib/plugins/Makefile +++ b/contrib/plugins/Makefile @@ -29,6 +29,7 @@ NAMES +=3D cache NAMES +=3D drcov NAMES +=3D ips NAMES +=3D stoptrigger +NAMES +=3D cflow =20 ifeq ($(CONFIG_WIN32),y) SO_SUFFIX :=3D .dll --=20 2.39.5