From nobody Fri Apr 26 18:22:53 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=gmail.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1547207443229736.965868376693; Fri, 11 Jan 2019 03:50:43 -0800 (PST) Received: from localhost ([127.0.0.1]:52360 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ghvKk-0004zu-8D for importer@patchew.org; Fri, 11 Jan 2019 06:50:42 -0500 Received: from eggs.gnu.org ([209.51.188.92]:39637) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ghvJn-0004f0-Sl for qemu-devel@nongnu.org; Fri, 11 Jan 2019 06:49:45 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ghvJl-0006Vj-Nv for qemu-devel@nongnu.org; Fri, 11 Jan 2019 06:49:43 -0500 Received: from mail-lj1-x242.google.com ([2a00:1450:4864:20::242]:35389) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1ghvJl-0006Qy-7z for qemu-devel@nongnu.org; Fri, 11 Jan 2019 06:49:41 -0500 Received: by mail-lj1-x242.google.com with SMTP id x85-v6so12696068ljb.2 for ; Fri, 11 Jan 2019 03:49:37 -0800 (PST) Received: from octofox.hsd1.ca.comcast.net. (jcmvbkbc-1-pt.tunnel.tserv24.sto1.ipv6.he.net. [2001:470:27:1fa::2]) by smtp.gmail.com with ESMTPSA id v17-v6sm15560970ljg.32.2019.01.11.03.49.33 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 11 Jan 2019 03:49:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=Tb1jEbjSkjhdvlKfFHN/e+RlX0syXugb1mMEjdRdNfk=; b=CbeXoB5rKJ8V64dIJXcIeNUtvUIB2WME4s+W5vCJ6/ShZI/y0WaTjrGcMseqbporQ3 kIwZLza946V7VyzffFKgkcHTAzTGF7UDdxD8vJWopEzBJV2kzexvRRqrPRuB9LsMTP0n z9l+VYFCq2G7/QJVClI1jV+oZGqhlGUIQmoDZx19zMHeiYmMKygc2AhLhvi3t8wYDbL1 za/jYNVwuOL0eE/Z/BHnmB2IgWDTe/n9yZoAo+9RDRsoz2jhA+7TnL7Bnp8sunDQS379 fDMzKL6YTd8EfdE/cuSO/udwo1BIerjvMF5kY3pACXAQCVzas5T3wCed2eJQ6zFLUy8M uu3w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=Tb1jEbjSkjhdvlKfFHN/e+RlX0syXugb1mMEjdRdNfk=; b=qHveqkDBC6YWBlqiBpr2um5AUD55gu1E0H5TrgWs9FdrZc1XL91O+9Zby8ZomN/GWe FIJTv4+J8WNcxnrHFt/YYE5zs+hD4b3jj+UyHMD+LtBMvHqfkgMJdmntVGb8zpPHiych fe3fUJ4mMvOnuGCJaFwyDWc34GBINIWc3SFjv40D3reZEQnTvPtptgr13pL7OWBrM4eI kn8jtuC9xBPj1Y27QJABi06mDz81+MjqsMqFG2Ca+hL8+pjj+h5PJXSTnDJyhiZswzq0 BytSxSG91ziKMkYu5ubxP+WrWETMP6XU2Y1zk/YEk1za2g0IGTZ+0McaSAHKNieq4dOZ getw== X-Gm-Message-State: AJcUukfYDh60UEtRnsWQWDEQqAdtLqaKINklavuZM+zCgC6Gat9I/fqr C3opViw+2SrYdxdMeylVyICwcE8m X-Google-Smtp-Source: ALg8bN6+UfO62ne+kHVC9lAH9wXs3e2JFbEU7JrCWib6o3o9sIgAnVKUvSOenWdc8CN3JOIxP0bk7Q== X-Received: by 2002:a2e:8007:: with SMTP id j7-v6mr8379170ljg.50.1547207375610; Fri, 11 Jan 2019 03:49:35 -0800 (PST) From: Max Filippov To: qemu-devel@nongnu.org Date: Fri, 11 Jan 2019 03:49:18 -0800 Message-Id: <20190111114918.27893-1-jcmvbkbc@gmail.com> X-Mailer: git-send-email 2.11.0 X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:4864:20::242 Subject: [Qemu-devel] [PATCH] target/xtensa: rework zero overhead loops implementation X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Max Filippov , Richard Henderson Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Don't invalidate TB with the end of zero overhead loop when LBEG or LEND change. Instead encode the distance from the start of the page where the TB starts to the LEND in the TB cs_base and generate loopback code when the next PC matches encoded LEND. Distance to a destination within the same page and up to a maximum instruction length into the next page is encoded literally, otherwise it's zero. The distance from LEND to LBEG is also encoded in the cs_base: it's encoded literally when less than 256 or as 0 otherwise. This allows for TB chaining for the loopback branch at the end of a loop for the most common loop sizes. With this change the resulting emulation speed is about 10% higher in softmmu mode on uClibc-ng and LTP tests. Emulation speed in linux user mode is a few percent lower because there's no direct TB chaining between different memory pages. Testing with lower limit on direct TB chainig range shows gradual slowdown to ~15% for the block size of 64 bytes and ~50% for the block size of 32 bytes. Signed-off-by: Max Filippov --- target/xtensa/cpu.h | 21 +++++++++++++++ target/xtensa/helper.h | 2 -- target/xtensa/op_helper.c | 24 ----------------- target/xtensa/overlay_tool.h | 1 + target/xtensa/translate.c | 61 +++++++++++++++-------------------------= ---- 5 files changed, 42 insertions(+), 67 deletions(-) diff --git a/target/xtensa/cpu.h b/target/xtensa/cpu.h index 34e5ccd9f1d6..22851d2350a7 100644 --- a/target/xtensa/cpu.h +++ b/target/xtensa/cpu.h @@ -400,6 +400,7 @@ struct XtensaConfig { int excm_level; int ndepc; unsigned inst_fetch_width; + unsigned max_insn_size; uint32_t vecbase; uint32_t exception_vector[EXC_MAX]; unsigned ninterrupt; @@ -695,6 +696,15 @@ static inline int cpu_mmu_index(CPUXtensaState *env, b= ool ifetch) #define XTENSA_TBFLAG_CALLINC_MASK 0x180000 #define XTENSA_TBFLAG_CALLINC_SHIFT 19 =20 +#define XTENSA_CSBASE_LEND_MASK 0x0000ffff +#define XTENSA_CSBASE_LEND_SHIFT 0 +#define XTENSA_CSBASE_LBEG_OFF_MASK 0x00ff0000 +#define XTENSA_CSBASE_LBEG_OFF_SHIFT 16 + +#define LINKABLE_BITS 12 +#define LINKABLE_SIZE (1u << LINKABLE_BITS) +#define LINKABLE_MASK (-LINKABLE_SIZE) + static inline void cpu_get_tb_cpu_state(CPUXtensaState *env, target_ulong = *pc, target_ulong *cs_base, uint32_t *flags) { @@ -706,6 +716,17 @@ static inline void cpu_get_tb_cpu_state(CPUXtensaState= *env, target_ulong *pc, *flags |=3D xtensa_get_ring(env); if (env->sregs[PS] & PS_EXCM) { *flags |=3D XTENSA_TBFLAG_EXCM; + } else if (xtensa_option_enabled(env->config, XTENSA_OPTION_LOOP)) { + target_ulong lend_dist =3D env->sregs[LEND] - (env->pc & LINKABLE_= MASK); + + if (lend_dist < LINKABLE_SIZE + env->config->max_insn_size) { + target_ulong lbeg_off =3D env->sregs[LEND] - env->sregs[LBEG]; + + *cs_base =3D lend_dist; + if (lbeg_off < 256) { + *cs_base |=3D lbeg_off << XTENSA_CSBASE_LBEG_OFF_SHIFT; + } + } } if (xtensa_option_enabled(env->config, XTENSA_OPTION_EXTENDED_L32R) && (env->sregs[LITBASE] & 1)) { diff --git a/target/xtensa/helper.h b/target/xtensa/helper.h index 10153c245360..2ebba0b2c2bf 100644 --- a/target/xtensa/helper.h +++ b/target/xtensa/helper.h @@ -12,8 +12,6 @@ DEF_HELPER_2(rotw, void, env, i32) DEF_HELPER_3(window_check, noreturn, env, i32, i32) DEF_HELPER_1(restore_owb, void, env) DEF_HELPER_2(movsp, void, env, i32) -DEF_HELPER_2(wsr_lbeg, void, env, i32) -DEF_HELPER_2(wsr_lend, void, env, i32) #ifndef CONFIG_USER_ONLY DEF_HELPER_1(simcall, void, env) #endif diff --git a/target/xtensa/op_helper.c b/target/xtensa/op_helper.c index e4b42ab3e56c..078aeb6c2c94 100644 --- a/target/xtensa/op_helper.c +++ b/target/xtensa/op_helper.c @@ -107,13 +107,6 @@ static void tb_invalidate_virtual_addr(CPUXtensaState = *env, uint32_t vaddr) } } =20 -#else - -static void tb_invalidate_virtual_addr(CPUXtensaState *env, uint32_t vaddr) -{ - tb_invalidate_phys_addr(vaddr); -} - #endif =20 void HELPER(exception)(CPUXtensaState *env, uint32_t excp) @@ -370,23 +363,6 @@ void HELPER(movsp)(CPUXtensaState *env, uint32_t pc) } } =20 -void HELPER(wsr_lbeg)(CPUXtensaState *env, uint32_t v) -{ - if (env->sregs[LBEG] !=3D v) { - tb_invalidate_virtual_addr(env, env->sregs[LEND] - 1); - env->sregs[LBEG] =3D v; - } -} - -void HELPER(wsr_lend)(CPUXtensaState *env, uint32_t v) -{ - if (env->sregs[LEND] !=3D v) { - tb_invalidate_virtual_addr(env, env->sregs[LEND] - 1); - env->sregs[LEND] =3D v; - tb_invalidate_virtual_addr(env, env->sregs[LEND] - 1); - } -} - void HELPER(dump_state)(CPUXtensaState *env) { XtensaCPU *cpu =3D xtensa_env_get_cpu(env); diff --git a/target/xtensa/overlay_tool.h b/target/xtensa/overlay_tool.h index ee37a04a176c..12609a0d0c1e 100644 --- a/target/xtensa/overlay_tool.h +++ b/target/xtensa/overlay_tool.h @@ -457,6 +457,7 @@ .nareg =3D XCHAL_NUM_AREGS, \ .ndepc =3D (XCHAL_XEA_VERSION >=3D 2), \ .inst_fetch_width =3D XCHAL_INST_FETCH_WIDTH, \ + .max_insn_size =3D XCHAL_MAX_INSTRUCTION_SIZE, \ EXCEPTIONS_SECTION, \ INTERRUPTS_SECTION, \ TLB_SECTION, \ diff --git a/target/xtensa/translate.c b/target/xtensa/translate.c index 46e13384488e..f200d785550e 100644 --- a/target/xtensa/translate.c +++ b/target/xtensa/translate.c @@ -53,7 +53,7 @@ struct DisasContext { uint32_t pc; int cring; int ring; - uint32_t lbeg; + uint32_t lbeg_off; uint32_t lend; =20 bool sar_5bit; @@ -390,11 +390,9 @@ static void gen_jump(DisasContext *dc, TCGv dest) static void gen_jumpi(DisasContext *dc, uint32_t dest, int slot) { TCGv_i32 tmp =3D tcg_const_i32(dest); -#ifndef CONFIG_USER_ONLY - if (((dc->base.pc_first ^ dest) & TARGET_PAGE_MASK) !=3D 0) { + if (((dc->base.pc_first ^ dest) & LINKABLE_MASK) !=3D 0) { slot =3D -1; } -#endif gen_jump_slot(dc, tmp, slot); tcg_temp_free(tmp); } @@ -420,25 +418,25 @@ static void gen_callw(DisasContext *dc, int callinc, = TCGv_i32 dest) static void gen_callwi(DisasContext *dc, int callinc, uint32_t dest, int s= lot) { TCGv_i32 tmp =3D tcg_const_i32(dest); -#ifndef CONFIG_USER_ONLY - if (((dc->base.pc_first ^ dest) & TARGET_PAGE_MASK) !=3D 0) { + if (((dc->base.pc_first ^ dest) & LINKABLE_MASK) !=3D 0) { slot =3D -1; } -#endif gen_callw_slot(dc, callinc, tmp, slot); tcg_temp_free(tmp); } =20 static bool gen_check_loop_end(DisasContext *dc, int slot) { - if (option_enabled(dc, XTENSA_OPTION_LOOP) && - !(dc->base.tb->flags & XTENSA_TBFLAG_EXCM) && - dc->base.pc_next =3D=3D dc->lend) { + if (dc->base.pc_next =3D=3D dc->lend) { TCGLabel *label =3D gen_new_label(); =20 tcg_gen_brcondi_i32(TCG_COND_EQ, cpu_SR[LCOUNT], 0, label); tcg_gen_subi_i32(cpu_SR[LCOUNT], cpu_SR[LCOUNT], 1); - gen_jumpi(dc, dc->lbeg, slot); + if (dc->lbeg_off) { + gen_jumpi(dc, dc->base.pc_next - dc->lbeg_off, slot); + } else { + gen_jump(dc, cpu_SR[LBEG]); + } gen_set_label(label); gen_jumpi(dc, dc->base.pc_next, -1); return true; @@ -534,16 +532,6 @@ static void gen_rsr(DisasContext *dc, TCGv_i32 d, uint= 32_t sr) } } =20 -static void gen_wsr_lbeg(DisasContext *dc, uint32_t sr, TCGv_i32 s) -{ - gen_helper_wsr_lbeg(cpu_env, s); -} - -static void gen_wsr_lend(DisasContext *dc, uint32_t sr, TCGv_i32 s) -{ - gen_helper_wsr_lend(cpu_env, s); -} - static void gen_wsr_sar(DisasContext *dc, uint32_t sr, TCGv_i32 s) { tcg_gen_andi_i32(cpu_SR[sr], s, 0x3f); @@ -743,8 +731,6 @@ static void gen_wsr(DisasContext *dc, uint32_t sr, TCGv= _i32 s) { static void (* const wsr_handler[256])(DisasContext *dc, uint32_t sr, TCGv_i32 v) =3D { - [LBEG] =3D gen_wsr_lbeg, - [LEND] =3D gen_wsr_lend, [SAR] =3D gen_wsr_sar, [BR] =3D gen_wsr_br, [LITBASE] =3D gen_wsr_litbase, @@ -906,13 +892,6 @@ static void disas_xtensa_insn(CPUXtensaState *env, Dis= asContext *dc) } =20 dc->base.pc_next =3D dc->pc + len; - if (xtensa_option_enabled(dc->config, XTENSA_OPTION_LOOP) && - dc->lbeg =3D=3D dc->pc && - ((dc->pc ^ (dc->base.pc_next - 1)) & -dc->config->inst_fetch_width= )) { - qemu_log_mask(LOG_GUEST_ERROR, - "unaligned first instruction of a loop (pc =3D %08x)= \n", - dc->pc); - } for (i =3D 1; i < len; ++i) { b[i] =3D cpu_ldub_code(env, dc->pc + i); } @@ -1097,8 +1076,10 @@ static void xtensa_tr_init_disas_context(DisasContex= tBase *dcbase, dc->pc =3D dc->base.pc_first; dc->ring =3D tb_flags & XTENSA_TBFLAG_RING_MASK; dc->cring =3D (tb_flags & XTENSA_TBFLAG_EXCM) ? 0 : dc->ring; - dc->lbeg =3D env->sregs[LBEG]; - dc->lend =3D env->sregs[LEND]; + dc->lbeg_off =3D (dc->base.tb->cs_base & XTENSA_CSBASE_LBEG_OFF_MASK) = >> + XTENSA_CSBASE_LBEG_OFF_SHIFT; + dc->lend =3D (dc->base.tb->cs_base & XTENSA_CSBASE_LEND_MASK) + + (dc->base.pc_first & LINKABLE_MASK); dc->debug =3D tb_flags & XTENSA_TBFLAG_DEBUG; dc->icount =3D tb_flags & XTENSA_TBFLAG_ICOUNT; dc->cpenable =3D (tb_flags & XTENSA_TBFLAG_CPENABLE_MASK) >> @@ -1189,10 +1170,10 @@ static void xtensa_tr_translate_insn(DisasContextBa= se *dcbase, CPUState *cpu) } =20 /* End the TB if the next insn will cross into the next page. */ - page_start =3D dc->base.pc_first & TARGET_PAGE_MASK; + page_start =3D dc->base.pc_first & LINKABLE_MASK; if (dc->base.is_jmp =3D=3D DISAS_NEXT && (dc->pc - page_start >=3D TARGET_PAGE_SIZE || - dc->pc - page_start + xtensa_insn_len(env, dc) > TARGET_PAGE_SIZE= )) { + dc->pc - page_start + xtensa_insn_len(env, dc) > LINKABLE_SIZE)) { dc->base.is_jmp =3D DISAS_TOO_MANY; } } @@ -1712,12 +1693,10 @@ static void translate_loop(DisasContext *dc, const = uint32_t arg[], const uint32_t par[]) { uint32_t lend =3D arg[1]; - TCGv_i32 tmp =3D tcg_const_i32(lend); =20 tcg_gen_subi_i32(cpu_SR[LCOUNT], cpu_R[arg[0]], 1); tcg_gen_movi_i32(cpu_SR[LBEG], dc->base.pc_next); - gen_helper_wsr_lend(cpu_env, tmp); - tcg_temp_free(tmp); + tcg_gen_movi_i32(cpu_SR[LEND], lend); =20 if (par[0] !=3D TCG_COND_NEVER) { TCGLabel *label =3D gen_new_label(); @@ -4609,7 +4588,7 @@ static const XtensaOpcodeOps core_ops[] =3D { .translate =3D translate_wsr, .test_ill =3D test_ill_wsr, .par =3D (const uint32_t[]){LBEG}, - .op_flags =3D XTENSA_OP_EXIT_TB_0, + .op_flags =3D XTENSA_OP_EXIT_TB_M1, .windowed_register_op =3D 0x1, }, { .name =3D "wsr.lcount", @@ -4622,7 +4601,7 @@ static const XtensaOpcodeOps core_ops[] =3D { .translate =3D translate_wsr, .test_ill =3D test_ill_wsr, .par =3D (const uint32_t[]){LEND}, - .op_flags =3D XTENSA_OP_EXIT_TB_0, + .op_flags =3D XTENSA_OP_EXIT_TB_M1, .windowed_register_op =3D 0x1, }, { .name =3D "wsr.litbase", @@ -5183,7 +5162,7 @@ static const XtensaOpcodeOps core_ops[] =3D { .translate =3D translate_xsr, .test_ill =3D test_ill_xsr, .par =3D (const uint32_t[]){LBEG}, - .op_flags =3D XTENSA_OP_EXIT_TB_0, + .op_flags =3D XTENSA_OP_EXIT_TB_M1, .windowed_register_op =3D 0x1, }, { .name =3D "xsr.lcount", @@ -5196,7 +5175,7 @@ static const XtensaOpcodeOps core_ops[] =3D { .translate =3D translate_xsr, .test_ill =3D test_ill_xsr, .par =3D (const uint32_t[]){LEND}, - .op_flags =3D XTENSA_OP_EXIT_TB_0, + .op_flags =3D XTENSA_OP_EXIT_TB_M1, .windowed_register_op =3D 0x1, }, { .name =3D "xsr.litbase", --=20 2.11.0