From nobody Sat May 18 13:36:44 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=linaro.org ARC-Seal: i=1; a=rsa-sha256; t=1588094270; cv=none; d=zohomail.com; s=zohoarc; b=eHIP9K4IJKLn+QSusRpgnRQA53UUCSDm/LIWX3lZfl1Bzib6+8qRsF9oWgBIFwBe3aJmy9dduho+9GEYhoSZcIL1oR/tTvB6DelY1aDnsvsJyMCnPQ11AbzRAJ2kSMF/DRKWgKh0EhGCrzP3QXw8oMSAomEqtH+Duwpk5jcm8l0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1588094270; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:To; bh=1pq1lHLSOmmH7N125+R4HUhPyCkUqHE2ejEQyvBBBJ0=; b=lY0v9SLqUJoFd+8xiAaGdZCVoIEYwBwcRz91l8RVD/4KWFGnlS3E7TjG8H8sTK6nZk+NZEKsXVKFjOL7gI8d2aEyQvka8tAJOYXPqiiRhC0btD+1l/BnJU1QZSzn5j3W3Nrznv+WKYPWlz089ivhmSwS1uWejzBqJvI3JZ1EVsY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1588094270465235.32557628551922; Tue, 28 Apr 2020 10:17:50 -0700 (PDT) Received: from localhost ([::1]:40728 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jTTrg-0001wb-VH for importer@patchew.org; Tue, 28 Apr 2020 13:17:49 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:39990) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jTTqp-0001LW-SE for qemu-devel@nongnu.org; Tue, 28 Apr 2020 13:16:58 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.90_1) (envelope-from ) id 1jTTqg-0002tb-JP for qemu-devel@nongnu.org; Tue, 28 Apr 2020 13:16:55 -0400 Received: from mail-wr1-x42a.google.com ([2a00:1450:4864:20::42a]:33896) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jTTqg-0002t5-1c for qemu-devel@nongnu.org; Tue, 28 Apr 2020 13:16:46 -0400 Received: by mail-wr1-x42a.google.com with SMTP id j1so25604261wrt.1 for ; Tue, 28 Apr 2020 10:16:44 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id r2sm4199028wmg.2.2020.04.28.10.16.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 28 Apr 2020 10:16:42 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id 59C091FF7E; Tue, 28 Apr 2020 18:16:41 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=1pq1lHLSOmmH7N125+R4HUhPyCkUqHE2ejEQyvBBBJ0=; b=fRc9Kp4BL4wukY/0nQJpg2aiHYkkTUFAPptu/gIqTYe94FEU1PBi8G7Cax89hS+5cU X0lfZk1Llx9L7tsRFaLQlRhe8AZhR56VQZvnvu+wac1C5HTtIYMbLNr4ip9ljjanEg7c rjpj2AzwCzvRQ7qPbopfxM3T5K1v3q2fFQa44rV8sZCmP8IBt1qa1TkmmcydckbvoJsF NkpvzpWO/0uegOV+IHmVJvtwBs7GVTMSEjtV28DLjFPvy7y5cuDS9wju7ELTHvRyDJ/I 6mCU295luYqZGOoVhugRiDWgkgaZGtlSGCrEd1d9VI1bJBoPYK5p6J/ms/RU+hOKlU6B mqYA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=1pq1lHLSOmmH7N125+R4HUhPyCkUqHE2ejEQyvBBBJ0=; b=MAPd0NlQi7NvpLTZXd2JSYAaM11eGfhS98M2Y1n0vRS34CeBb/VADDwIP1ONODhS2S gYiT2/PEJFuUEdNpM2zrFD24hsiNwNSqjg+yNcD0DtldikhGvk2J0bjAzGeT/Lk4bztn 5WEEJEmnbP/vmLNAaawPG80NxIBaMiKQwAybD1y69RHVIajIWUawUdccCWzTYyl1oXHG lv7463PBmKdycnW0jiQVChZW01cXa8z+W6it1SmnvCmCMnXYoVvJlR7KPI4f6Vmx6N3/ kIlCN4kGZtXyD1Hgkm0anKyqmwEBm7GXJmpM6SZ4VfYhIz28V/WaTEmZyyz26CZDr/3Q Zy3w== X-Gm-Message-State: AGi0Pubkr2ThaL7EtBrh3qz0CgGfAjniw3BpXlwEGFMYMh+Yv2FbsokM w6iyHanan3bgP0riXQiJWYiyBQ== X-Google-Smtp-Source: APiQypLveZJoKkg8Z/+93IyggNTiYpAxrmImkFgS39j9RPIs3/35ZsTrsKYoGYVj3gryAyPQv1cjqg== X-Received: by 2002:adf:e5c8:: with SMTP id a8mr37306195wrn.56.1588094203607; Tue, 28 Apr 2020 10:16:43 -0700 (PDT) From: =?UTF-8?q?Alex=20Benn=C3=A9e?= To: qemu-devel@nongnu.org Subject: [RFC PATCH] plugins: new lockstep plugin for debugging TCG changes Date: Tue, 28 Apr 2020 18:16:33 +0100 Message-Id: <20200428171633.17487-1-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::42a; envelope-from=alex.bennee@linaro.org; helo=mail-wr1-x42a.google.com X-detected-operating-system: by eggs.gnu.org: Error: [-] PROGRAM ABORT : Malformed IPv6 address (bad octet value). Location : parse_addr6(), p0f-client.c:67 X-Received-From: 2a00:1450:4864:20::42a X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?UTF-8?q?Alex=20Benn=C3=A9e?= , Mark Cave-Ayland , richard.henderson@linaro.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) When we make changes to the TCG we sometimes cause regressions that are deep into the execution cycle of the guest. Debugging this often requires comparing large volumes of trace information to figure out where behaviour has diverged. The lockstep plugin utilises a shared socket so two QEMU's running with the plugin will write their current execution position and wait to receive the position of their partner process. When execution diverges the plugins output where they were and the previous few blocks before unloading themselves and letting execution continue. Signed-off-by: Alex Benn=C3=A9e Cc: Richard Henderson Cc: Mark Cave-Ayland --- tests/plugin/lockstep.c | 244 ++++++++++++++++++++++++++++++++++++++++ tests/plugin/Makefile | 1 + 2 files changed, 245 insertions(+) create mode 100644 tests/plugin/lockstep.c diff --git a/tests/plugin/lockstep.c b/tests/plugin/lockstep.c new file mode 100644 index 0000000000..2c386d2b83 --- /dev/null +++ b/tests/plugin/lockstep.c @@ -0,0 +1,244 @@ +/* + * Lockstep Execution Plugin + * + * Allows you to execute two QEMU instances in lockstep and report + * when their execution diverges. This is mainly useful for developers + * who want to see where a change to TCG code generation has + * introduced a subtle and hard to find bug. + * + * Caveats: + * - single-threaded linux-user apps only with non-deterministic syscalls + * - icount based system emulation (no MTTCG) + * + * This code is not thread safe! + * + * Copyright (c) 2020 Linaro Ltd + * + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#include +#include +#include +#include +#include +#include +#include + +#include + +QEMU_PLUGIN_EXPORT int qemu_plugin_version =3D QEMU_PLUGIN_VERSION; + +/* saved so we can uninstall later */ +static qemu_plugin_id_t our_id; + +static unsigned long bb_count; +static unsigned long insn_count; + +typedef struct { + uint64_t pc; + uint64_t insns_in_block; + uint64_t insns_executed; +} BlockInfo; + +static GSList *log; + +static int socket_fd; +static char *path_to_unlink; + + +static void plugin_cleanup(qemu_plugin_id_t id) +{ + + /* Free our block data */ + g_slist_free_full(log, &g_free); + + close(socket_fd); + if (path_to_unlink) { + unlink(path_to_unlink); + } +} + +static void plugin_exit(qemu_plugin_id_t id, void *p) +{ + g_autoptr(GString) out =3D g_string_new("No divergence :-)\n"); + g_string_append_printf(out, "Executed %ld/%d blocks\n", + bb_count, g_slist_length(log)); + g_string_append_printf(out, "Executed ~%ld instructions\n", insn_count= ); + qemu_plugin_outs(out->str); + + plugin_cleanup(id); +} + +static void report_divergance(BlockInfo *us, BlockInfo *them) +{ + int i; + GSList *entry =3D log; + g_autoptr(GString) out =3D g_string_new("I feel a divergence in the fo= rce\n"); + g_string_append_printf(out, "Us @ %#016lx (%ld)\n", + us->pc, us->insns_executed); + g_string_append_printf(out, "Them @ %#016lx (%ld)\n", + them->pc, them->insns_executed); + for (i =3D 0; i < 5; i++) { + BlockInfo *prev =3D (BlockInfo *) entry->data; + g_string_append_printf(out, " previously @ %#016lx\n", prev->pc); + entry =3D g_slist_next(entry); + } + + qemu_plugin_outs(out->str); + + /* we can't do anything else now so uninstall ourselves */ + qemu_plugin_uninstall(our_id, plugin_cleanup); +} + +static void vcpu_tb_exec(unsigned int cpu_index, void *udata) +{ + BlockInfo *bi =3D (BlockInfo *) udata; + BlockInfo remote; + ssize_t bytes; + + bi->insns_executed =3D insn_count; + + /* write our execution state */ + bytes =3D write(socket_fd, bi, sizeof(BlockInfo)); + if (bytes < sizeof(BlockInfo)) { + if (bytes < 0) { + qemu_plugin_outs("problem writing to socket"); + abort(); + } + qemu_plugin_outs("wrote less than expected"); + } + /* read where our peer has reached */ + bytes =3D read(socket_fd, &remote, sizeof(BlockInfo)); + if (bytes < sizeof(BlockInfo)) { + if (bytes < 0) { + qemu_plugin_outs("problem reading from socket"); + abort(); + } + qemu_plugin_outs("read less than expected"); + abort(); + } + + // compare and bail + if ((bi->pc !=3D remote.pc) || + (bi->insns_executed !=3D remote.insns_executed)) { + report_divergance(bi, &remote); + } + + // mark the execution as complete + log =3D g_slist_prepend(log, bi); + insn_count +=3D bi->insns_in_block; + bb_count++; +} + +static void vcpu_tb_trans(qemu_plugin_id_t id, struct qemu_plugin_tb *tb) +{ + BlockInfo *bi =3D g_new0(BlockInfo, 1); + bi->pc =3D qemu_plugin_tb_vaddr(tb); + bi->insns_in_block =3D qemu_plugin_tb_n_insns(tb); + + qemu_plugin_register_vcpu_tb_exec_cb(tb, vcpu_tb_exec, + QEMU_PLUGIN_CB_NO_REGS, (void *)b= i); +} + + +/* + * Instead of encoding master/slave status into what is essentially + * two peers we shall just take the simple approach of checking for + * the existence of the pipe and assuming if it's not there we are the + * first process. + */ +static bool setup_socket(const char *path) +{ + struct sockaddr_un sockaddr; + int fd; + + fd =3D socket(AF_UNIX, SOCK_STREAM, 0); + if (fd < 0) { + perror("create socket"); + return false; + } + + sockaddr.sun_family =3D AF_UNIX; + g_strlcpy(sockaddr.sun_path, path, sizeof(sockaddr.sun_path)-1); + if (bind(fd, (struct sockaddr *)&sockaddr, sizeof(sockaddr)) < 0) { + perror("bind socket"); + close(fd); + return false; + } + + /* remember to clean-up */ + path_to_unlink =3D g_strdup(path); + + if (listen(fd, 1) < 0) { + perror("listen socket"); + close(fd); + return false; + } + + socket_fd =3D accept(fd, NULL, NULL); + if (socket_fd < 0 && errno !=3D EINTR) { + perror("accept socket"); + return false; + } + + qemu_plugin_outs("setup_socket::ready\n"); + + return true; +} + +static bool connect_socket(const char *path) +{ + int fd; + struct sockaddr_un sockaddr; + + fd =3D socket(AF_UNIX, SOCK_STREAM, 0); + if (fd < 0) { + perror("create socket"); + return false; + } + + sockaddr.sun_family =3D AF_UNIX; + g_strlcpy(sockaddr.sun_path, path, sizeof(sockaddr.sun_path)-1); + + if (connect(fd, (struct sockaddr *)&sockaddr, sizeof(sockaddr)) < 0) { + perror("failed to connect"); + return false; + } + + qemu_plugin_outs("connect_socket::ready\n"); + + socket_fd =3D fd; + return true; +} + +static bool setup_unix_socket(const char *path) +{ + if (g_file_test(path, G_FILE_TEST_EXISTS)) { + return connect_socket(path); + } else { + return setup_socket(path); + } +} + + +QEMU_PLUGIN_EXPORT int qemu_plugin_install(qemu_plugin_id_t id, + const qemu_info_t *info, + int argc, char **argv) +{ + if (!argc || !argv[0]) { + qemu_plugin_outs("Need a socket path to talk to other instance."); + return -1; + } + + if (!setup_unix_socket(argv[0])) { + qemu_plugin_outs("Failed to setup socket for communications."); + return -1; + } + + our_id =3D id; + + qemu_plugin_register_vcpu_tb_trans_cb(id, vcpu_tb_trans); + qemu_plugin_register_atexit_cb(id, plugin_exit, NULL); + return 0; +} diff --git a/tests/plugin/Makefile b/tests/plugin/Makefile index 75467b6db8..b3250e2504 100644 --- a/tests/plugin/Makefile +++ b/tests/plugin/Makefile @@ -13,6 +13,7 @@ NAMES +=3D mem NAMES +=3D hotblocks NAMES +=3D howvec NAMES +=3D hotpages +NAMES +=3D lockstep =20 SONAMES :=3D $(addsuffix .so,$(addprefix lib,$(NAMES))) =20 --=20 2.20.1