From nobody Wed May 8 21:40:12 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; dkim=fail spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 149862611146157.13325202098633; Tue, 27 Jun 2017 22:01:51 -0700 (PDT) Received: from localhost ([::1]:59221 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQ56s-0003Ff-2T for importer@patchew.org; Wed, 28 Jun 2017 01:01:50 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:57443) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQ55b-0002bv-Ua for qemu-devel@nongnu.org; Wed, 28 Jun 2017 01:00:33 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dQ55X-0004ip-6G for qemu-devel@nongnu.org; Wed, 28 Jun 2017 01:00:32 -0400 Received: from mail-yw0-x242.google.com ([2607:f8b0:4002:c05::242]:35172) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1dQ55X-0004iG-2G for qemu-devel@nongnu.org; Wed, 28 Jun 2017 01:00:27 -0400 Received: by mail-yw0-x242.google.com with SMTP id z21so2779536ywz.2 for ; Tue, 27 Jun 2017 22:00:25 -0700 (PDT) Received: from localhost.localdomain (c-73-207-178-95.hsd1.ga.comcast.net. [73.207.178.95]) by smtp.gmail.com with ESMTPSA id d2sm521898ywb.53.2017.06.27.22.00.24 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 27 Jun 2017 22:00:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=ujQKbM5GLyl+UJnaL6EjwtfHJdRYGciCHU1Ajpe+13M=; b=Ek8e7mf5GCBIZbNbIQqoWDDA4J0fXSj+KYYWaXfyewkrw3FsZKDAzCpdbHfhsngHF7 BB5hb0Ur7+j854z615VuL4yW8NVfKBQ2Mx5dNg6OoGmUGaaYAgYthsOB1H5MYmPILjpu VPbOLCuQChv8ELT0z410GX+WkuM7Ld6wwTqmjD+4fted/S+6rJkZAqCFRzJ1ESg3/szQ q2A0e4cJFGde4INjjdxylgRzeLKKk2BHMpdl+/ZN81+G1YJ4FboROw+irBYtv13iigpT 8jyKZUMGTCjYtT3O4uZbjTfn+AOJ7yNv3GWQOmmPklglBx4p2vMe5IeGMR5uqtFuy3D7 Bw9w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=ujQKbM5GLyl+UJnaL6EjwtfHJdRYGciCHU1Ajpe+13M=; b=aEqXWj8jJK5l/HPLSDRI7lFmzhBnexR8kEL/QiiK/qvY7w56GYnaglb58QZ7n7PWo7 7AxnByBG7U6BBnNinMkE8Ac1zoUVkPcDER0UxZdiccW4mAWzrBJLbQurhdlVVSKiTd7g njh1jTZ3QjtwfCnsCIURD6ptXs3IuJ/jlJYz1N8HUMw3I1j5o7KgBtk8wr4QELh+g6ep QtSkHnOAQfgHPI2lot/fqKA67ZcLM8scMEwj4kg9ZzCO+DLqgBrvIJ/WSBI9RswybcCn d5BkBc8GhrdfJzp+XNDhIm4cWCL4nnNmpFSxq/jpQC+sTgCCekU6560bFzdHKRCdPi7/ Idkw== X-Gm-Message-State: AKS2vOz4HNy10bKrQiGOYynqtfSQfpa2XSW+Hh2DlrbuAWz7oHEZ5kjk b3BUoPt/jL2pw8KPaIw= X-Received: by 10.129.39.149 with SMTP id n143mr6467192ywn.189.1498626024951; Tue, 27 Jun 2017 22:00:24 -0700 (PDT) From: Pranith Kumar To: alex.bennee@linaro.org Date: Wed, 28 Jun 2017 01:00:02 -0400 Message-Id: <20170628050003.1809-2-bobby.prani@gmail.com> X-Mailer: git-send-email 2.13.0 In-Reply-To: <20170628050003.1809-1-bobby.prani@gmail.com> References: <20170628050003.1809-1-bobby.prani@gmail.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4002:c05::242 Subject: [Qemu-devel] [PATCH 1/2] [TEST] aarch64: Use pmuserenr_el0 register for instrumentation X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-devel@nongnu.org, rth@twiddle.net Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZohoMail: RDKM_2 RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" We need a way for the benchmark running in the guest to indicate us to start/stop our instrumentation. On x86, we could use the 'cpuid' instruction along with an appropriately populated 'eax' register. However, no such dummy instruction exists for aarch64. So we modify the permission bits for 'pmuserenr_el0' register and tap that to instrument the guest code. You can use the following annotations on your region-of-interest to instrument the code. #define magic_enable() \ asm volatile ("msr pmuserenr_el0, %0" :: "r" (0xaaaaaaaa)); #define magic_disable() \ asm volatile ("msr pmuserenr_el0, %0" :: "r" (0xfa11dead)); Signed-off-by: Pranith Kumar --- target/arm/helper.c | 19 +++++++++++++++++-- 1 file changed, 17 insertions(+), 2 deletions(-) diff --git a/target/arm/helper.c b/target/arm/helper.c index 2594faa9b8..dfbf03676c 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -1124,9 +1124,24 @@ static uint64_t pmxevtyper_read(CPUARMState *env, co= nst ARMCPRegInfo *ri) } } =20 +bool enable_instrumentation; + static void pmuserenr_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value) { + ARMCPU *cpu =3D arm_env_get_cpu(env); + CPUState *cs =3D CPU(cpu); + + if (value =3D=3D 0xaaaaaaaa) { + printf("Enabling instrumentation\n"); + enable_instrumentation =3D true; + tb_flush(cs); + } else if (value =3D=3D 0xfa11dead) { + printf("Disabling instrumentation\n"); + enable_instrumentation =3D false; + tb_flush(cs); + } + if (arm_feature(env, ARM_FEATURE_V8)) { env->cp15.c9_pmuserenr =3D value & 0xf; } else { @@ -1316,13 +1331,13 @@ static const ARMCPRegInfo v7_cp_reginfo[] =3D { .access =3D PL0_RW, .type =3D ARM_CP_CONST, .resetvalue =3D 0, .accessfn =3D pmreg_access_xevcntr }, { .name =3D "PMUSERENR", .cp =3D 15, .crn =3D 9, .crm =3D 14, .opc1 = =3D 0, .opc2 =3D 0, - .access =3D PL0_R | PL1_RW, .accessfn =3D access_tpm, + .access =3D PL0_RW | PL1_RW, .accessfn =3D access_tpm, .fieldoffset =3D offsetof(CPUARMState, cp15.c9_pmuserenr), .resetvalue =3D 0, .writefn =3D pmuserenr_write, .raw_writefn =3D raw_write }, { .name =3D "PMUSERENR_EL0", .state =3D ARM_CP_STATE_AA64, .opc0 =3D 3, .opc1 =3D 3, .crn =3D 9, .crm =3D 14, .opc2 =3D 0, - .access =3D PL0_R | PL1_RW, .accessfn =3D access_tpm, .type =3D ARM_= CP_ALIAS, + .access =3D PL0_RW | PL1_RW, .accessfn =3D access_tpm, .type =3D ARM= _CP_ALIAS, .fieldoffset =3D offsetof(CPUARMState, cp15.c9_pmuserenr), .resetvalue =3D 0, .writefn =3D pmuserenr_write, .raw_writefn =3D raw_write }, --=20 2.13.0 From nobody Wed May 8 21:40:12 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; dkim=fail spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1498626192064895.1244106501815; Tue, 27 Jun 2017 22:03:12 -0700 (PDT) Received: from localhost ([::1]:59223 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQ58A-000408-KV for importer@patchew.org; Wed, 28 Jun 2017 01:03:10 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:57444) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQ55b-0002bw-Uk for qemu-devel@nongnu.org; Wed, 28 Jun 2017 01:00:33 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dQ55Y-0004jb-Hv for qemu-devel@nongnu.org; Wed, 28 Jun 2017 01:00:32 -0400 Received: from mail-yw0-x243.google.com ([2607:f8b0:4002:c05::243]:35172) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1dQ55Y-0004iU-CH for qemu-devel@nongnu.org; Wed, 28 Jun 2017 01:00:28 -0400 Received: by mail-yw0-x243.google.com with SMTP id z21so2779545ywz.2 for ; Tue, 27 Jun 2017 22:00:26 -0700 (PDT) Received: from localhost.localdomain (c-73-207-178-95.hsd1.ga.comcast.net. [73.207.178.95]) by smtp.gmail.com with ESMTPSA id d2sm521898ywb.53.2017.06.27.22.00.24 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 27 Jun 2017 22:00:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=TgpzeliW4j6JHxsizAot5GKgWMbwVS126VH6gZBbXk4=; b=bR/8YXK3GfRfD31V2FDq2wmjEtIRBNRo7V/jNMDw8IlGw2HQl6GUiNUVoyFiIMCY8r J/0nnajzPOv4f0FMSah/d4VYMt36PWwwq+jY0LD6wdojcqxQK+LJg2kZJJPEOfoMGoc+ dq5JFP8vrHCL81jlUXW2smL0T1gIHc829GHtBBFpjSe2Ln8YzHXuKDYtyT6xn3ts/kWy Dtt3kaS0eDv/ZOzfAqw6Pxgz+o4U78ORzwnF1YBTGygz2JU8NuQCC+fpp6EX1fyekF55 lGjBfiwe/ZdTZW1QkEkHgu60ZcOYrt06Y4xgxe+HF7BEJYBZjmWDU6Ntum/XDtxHTYQL bu7g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=TgpzeliW4j6JHxsizAot5GKgWMbwVS126VH6gZBbXk4=; b=aHV9eZaHRJBIrEouvX/YQIj6bBjLQNCzOHZFHOPVfV6JjtmtYNFySv+rwVAoSxYIA+ eoNrANq6TdGpKklZFuQECKQwbmEbzT40hXgQxMIqtz5Hm8ziRbwFIwSQGB0CviQfl5HW TOQonyRC4qzKDCMAfviZ+1UU6rvfCcLd6AzvE9497kMr0RfuDV47LtXdMeUO7L33wptP LyfwGhEgelI16owytHnDao+x1ki3SA4kyL/1X8qiMUOYdoD0dNzghzNmCx6vKPKLy31F t/tP61MHIqfyzy1UFNgvSC+iPsDsXaG/9C03zsvyhb+Ji143NA9/xyXznZGPMvv5l2hO qAOQ== X-Gm-Message-State: AKS2vOyJGu2GJmSRNHPeN+egWWsM31zOoaEu8cIDzCv5GqCqoyth2k8C YpeYTuAOAQrxZQ== X-Received: by 10.129.75.196 with SMTP id y187mr6939907ywa.158.1498626026215; Tue, 27 Jun 2017 22:00:26 -0700 (PDT) From: Pranith Kumar To: alex.bennee@linaro.org Date: Wed, 28 Jun 2017 01:00:03 -0400 Message-Id: <20170628050003.1809-3-bobby.prani@gmail.com> X-Mailer: git-send-email 2.13.0 In-Reply-To: <20170628050003.1809-1-bobby.prani@gmail.com> References: <20170628050003.1809-1-bobby.prani@gmail.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4002:c05::243 Subject: [Qemu-devel] [PATCH 2/2] [TEST] Collect TLB and victim TLB hit/miss stats X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-devel@nongnu.org, rth@twiddle.net Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZohoMail: RDKM_2 RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" I used the following patch to collect hit/miss TLB ratios for a few benchmarks. The results can be found here: http://imgur.com/a/gee1o Please note that these results also include boot/shutdown as the per-region instrumentation patch came later. Signed-off-by: Pranith Kumar --- accel/tcg/cputlb.c | 12 ++++++++++++ cpus.c | 26 ++++++++++++++++++++++++++ include/exec/cpu-defs.h | 4 ++++ include/sysemu/cpus.h | 2 ++ target/arm/helper.c | 6 +++++- tcg/i386/tcg-target.inc.c | 16 ++++++++++++++-- vl.c | 3 +++ 7 files changed, 66 insertions(+), 3 deletions(-) diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index ef52a7e5e0..2ac2397431 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -864,12 +864,19 @@ static void io_writex(CPUArchState *env, CPUIOTLBEntr= y *iotlbentry, } } =20 +extern bool enable_instrumentation; + /* Return true if ADDR is present in the victim tlb, and has been copied back to the main tlb. */ static bool victim_tlb_hit(CPUArchState *env, size_t mmu_idx, size_t index, size_t elt_ofs, target_ulong page) { size_t vidx; + + if (enable_instrumentation) { + env->tlb_access_victim++; + } + for (vidx =3D 0; vidx < CPU_VTLB_SIZE; ++vidx) { CPUTLBEntry *vtlb =3D &env->tlb_v_table[mmu_idx][vidx]; target_ulong cmp =3D *(target_ulong *)((uintptr_t)vtlb + elt_ofs); @@ -885,6 +892,11 @@ static bool victim_tlb_hit(CPUArchState *env, size_t m= mu_idx, size_t index, CPUIOTLBEntry tmpio, *io =3D &env->iotlb[mmu_idx][index]; CPUIOTLBEntry *vio =3D &env->iotlb_v[mmu_idx][vidx]; tmpio =3D *io; *io =3D *vio; *vio =3D tmpio; + + if (enable_instrumentation) { + env->tlb_access_victim_hit++; + } + return true; } } diff --git a/cpus.c b/cpus.c index 14bb8d552e..14669b3469 100644 --- a/cpus.c +++ b/cpus.c @@ -1602,6 +1602,32 @@ static bool all_vcpus_paused(void) return true; } =20 +void print_tlb_stats(void) +{ + CPUState *cpu; + CPU_FOREACH(cpu) { + CPUArchState *cs =3D cpu->env_ptr; + + fprintf(stderr, "TLB accesses %lu, hits %lu, victim accesses %lu, = hits %lu\n", + cs->tlb_access_total, cs->tlb_access_hit, cs->tlb_access_v= ictim, + cs->tlb_access_victim_hit); + } +} + +void clear_tlb_stats(void) +{ + CPUState *cpu; + CPU_FOREACH(cpu) { + CPUArchState *cs =3D cpu->env_ptr; + + cs->tlb_access_total =3D 0; + cs->tlb_access_hit =3D 0; + cs->tlb_access_victim =3D 0; + cs->tlb_access_victim =3D 0; + cs->tlb_access_victim_hit =3D 0; + } +} + void pause_all_vcpus(void) { CPUState *cpu; diff --git a/include/exec/cpu-defs.h b/include/exec/cpu-defs.h index 5f4e303635..29b3c2ada8 100644 --- a/include/exec/cpu-defs.h +++ b/include/exec/cpu-defs.h @@ -138,6 +138,10 @@ typedef struct CPUIOTLBEntry { target_ulong tlb_flush_addr; \ target_ulong tlb_flush_mask; \ target_ulong vtlb_index; \ + target_ulong tlb_access_hit; \ + target_ulong tlb_access_total; \ + target_ulong tlb_access_victim; \ + target_ulong tlb_access_victim_hit; \ =20 #else =20 diff --git a/include/sysemu/cpus.h b/include/sysemu/cpus.h index 731756d948..7d8d92646c 100644 --- a/include/sysemu/cpus.h +++ b/include/sysemu/cpus.h @@ -10,6 +10,8 @@ void resume_all_vcpus(void); void pause_all_vcpus(void); void cpu_stop_current(void); void cpu_ticks_init(void); +void print_tlb_stats(void); +void clear_tlb_stats(void); =20 void configure_icount(QemuOpts *opts, Error **errp); extern int use_icount; diff --git a/target/arm/helper.c b/target/arm/helper.c index dfbf03676c..d2e75b0f20 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -1124,7 +1124,9 @@ static uint64_t pmxevtyper_read(CPUARMState *env, con= st ARMCPRegInfo *ri) } } =20 -bool enable_instrumentation; +extern bool enable_instrumentation; +extern void print_tlb_stats(void); +extern void clear_tlb_stats(void); =20 static void pmuserenr_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value) @@ -1139,6 +1141,8 @@ static void pmuserenr_write(CPUARMState *env, const A= RMCPRegInfo *ri, } else if (value =3D=3D 0xfa11dead) { printf("Disabling instrumentation\n"); enable_instrumentation =3D false; + print_tlb_stats(); + clear_tlb_stats(); tb_flush(cs); } =20 diff --git a/tcg/i386/tcg-target.inc.c b/tcg/i386/tcg-target.inc.c index 9d7d25c017..b75bd54c35 100644 --- a/tcg/i386/tcg-target.inc.c +++ b/tcg/i386/tcg-target.inc.c @@ -1250,6 +1250,8 @@ static void * const qemu_st_helpers[16] =3D { [MO_BEQ] =3D helper_be_stq_mmu, }; =20 +extern bool enable_instrumentation; + /* Perform the TLB load and compare. =20 Inputs: @@ -1300,6 +1302,12 @@ static inline void tcg_out_tlb_load(TCGContext *s, T= CGReg addrlo, TCGReg addrhi, } } =20 + if (enable_instrumentation) { + tcg_out_ld(s, TCG_TYPE_I64, r0, TCG_AREG0, offsetof(CPUArchState, = tlb_access_total)); + tcg_out_addi(s, r0, 1); + tcg_out_st(s, TCG_TYPE_I64, r0, TCG_AREG0, offsetof(CPUArchState, = tlb_access_total)); + } + tcg_out_mov(s, tlbtype, r0, addrlo); tlb_mask =3D (target_ulong)TARGET_PAGE_MASK | a_mask; =20 @@ -1348,11 +1356,15 @@ static inline void tcg_out_tlb_load(TCGContext *s, = TCGReg addrlo, TCGReg addrhi, s->code_ptr +=3D 4; } =20 - /* TLB Hit. */ - /* add addend(r0), r1 */ tcg_out_modrm_offset(s, OPC_ADD_GvEv + hrexw, r1, r0, offsetof(CPUTLBEntry, addend) - which); + + if (enable_instrumentation) { + tcg_out_ld(s, TCG_TYPE_I64, r0, TCG_AREG0, offsetof(CPUArchState, = tlb_access_hit)); + tcg_out_addi(s, r0, 1); + tcg_out_st(s, TCG_TYPE_I64, r0, TCG_AREG0, offsetof(CPUArchState, = tlb_access_hit)); + } } =20 /* diff --git a/vl.c b/vl.c index 59fea15488..7fa392c79e 100644 --- a/vl.c +++ b/vl.c @@ -192,6 +192,8 @@ int only_migratable; /* turn it off unless user states = otherwise */ =20 int icount_align_option; =20 +bool enable_instrumentation; + /* The bytes in qemu_uuid are in the order specified by RFC4122, _not_ in = the * little-endian "wire format" described in the SMBIOS 2.6 specification. */ @@ -4761,5 +4763,6 @@ int main(int argc, char **argv, char **envp) qemu_chr_cleanup(); /* TODO: unref root container, check all devices are ok */ =20 + print_tlb_stats(); return 0; } --=20 2.13.0