From nobody Thu Nov 6 01:29:20 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1538960921033858.2040947325156; Sun, 7 Oct 2018 18:08:41 -0700 (PDT) Received: from localhost ([::1]:43946 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g9K2D-0000K4-7G for importer@patchew.org; Sun, 07 Oct 2018 21:08:33 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:39913) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g9K0g-0007nu-Gn for qemu-devel@nongnu.org; Sun, 07 Oct 2018 21:07:01 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g9JzG-0001g6-G2 for qemu-devel@nongnu.org; Sun, 07 Oct 2018 21:05:34 -0400 Received: from mail-pf1-x444.google.com ([2607:f8b0:4864:20::444]:38723) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1g9JzE-0001Kr-NT for qemu-devel@nongnu.org; Sun, 07 Oct 2018 21:05:29 -0400 Received: by mail-pf1-x444.google.com with SMTP id f29-v6so4679619pff.5 for ; Sun, 07 Oct 2018 18:05:26 -0700 (PDT) Received: from cloudburst.twiddle.net (97-113-8-179.tukw.qwest.net. [97.113.8.179]) by smtp.gmail.com with ESMTPSA id r65-v6sm21087912pfj.5.2018.10.07.18.05.23 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Sun, 07 Oct 2018 18:05:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id; bh=Fuj3KOB6vftrpPnQlebyRHhBIq64o0dyiN+/bIM1c8k=; b=AiEmQ1aGSsyzOdMg57D3s8Pywy9ohVLO/TMvywL5h6inzJ3/eZ/esidxa7AAb4jYlW ZDUH9HRLDerDDBVMWpt+l6onZRbkpNAYKkgLLfa3XJELJMirEbDgz5jzV9ue6UzP3MZo sPfPV68WGgJCu5ycFVPbW2aDDzVBYFcYn933Y= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=Fuj3KOB6vftrpPnQlebyRHhBIq64o0dyiN+/bIM1c8k=; b=ojWB623L+3HL4aW7Y9zJm0ofVLjnlDc3/ikUR0tR2V4QmZcB1recFSyjgJhloWpnZz c1B8d/x+UowWUh46hfhX6BtdmIZamu4HjglR9s3UneFV7MdJkRSi+0rMIGFC+iTPusDw DXXoU2CIId6g175NRpEsuUu+MihWGjRKvqE5mam3RybkTa79PvapHsh5CrXxDfuZmY0T cPSNXjg+QyhiXd7bAwDuAt9Li2muBJQxgc3Q4qKKlVmDDGm56tmVLfcYeFPECtoioEuA tHHfi/UHV3eQuzAe/xreO1Ta4ShTvrHi3t02SNaymcHsxUz5W9bHibdOObN3UJteACeh ueXw== X-Gm-Message-State: ABuFfoh+O5O1kZzMoEydHGcmzrTAtAng+US6gK1LMiZYgJF2Hl9Inm5V 3ugN+2lQPmCTiqxrO23IROP+VfPTcgY= X-Google-Smtp-Source: ACcGV60AKhK3dXyRQaMbq/fL13f7DXLzQ9qbcvqBYmXEtUkb79wd3aGSDfpdtmC6xrz7Os2KvMJLOA== X-Received: by 2002:a62:564e:: with SMTP id k75-v6mr4102523pfb.33.1538960724767; Sun, 07 Oct 2018 18:05:24 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Date: Sun, 7 Oct 2018 18:05:22 -0700 Message-Id: <20181008010522.7885-1-richard.henderson@linaro.org> X-Mailer: git-send-email 2.17.1 X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::444 Subject: [Qemu-devel] [PATCH] tcg: Add tlb_index and tlb_entry helpers X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: cota@braap.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZohoMail: RDMRC_1 RDKM_2 RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Isolate the computation of an index from an address into a helper before we change that function. Signed-off-by: Richard Henderson --- Emilio, this should make your dynamic tlb sizing patch 1/6 significantly smaller. r~ --- accel/tcg/softmmu_template.h | 68 +++++++++++++++++--------------- include/exec/cpu_ldst.h | 14 +++++++ include/exec/cpu_ldst_template.h | 25 ++++++------ accel/tcg/cputlb.c | 60 +++++++++++++--------------- 4 files changed, 90 insertions(+), 77 deletions(-) diff --git a/accel/tcg/softmmu_template.h b/accel/tcg/softmmu_template.h index f060a693d4..fae901f00d 100644 --- a/accel/tcg/softmmu_template.h +++ b/accel/tcg/softmmu_template.h @@ -111,9 +111,10 @@ static inline DATA_TYPE glue(io_read, SUFFIX)(CPUArchS= tate *env, WORD_TYPE helper_le_ld_name(CPUArchState *env, target_ulong addr, TCGMemOpIdx oi, uintptr_t retaddr) { - unsigned mmu_idx =3D get_mmuidx(oi); - int index =3D (addr >> TARGET_PAGE_BITS) & (CPU_TLB_SIZE - 1); - target_ulong tlb_addr =3D env->tlb_table[mmu_idx][index].ADDR_READ; + uintptr_t mmu_idx =3D get_mmuidx(oi); + uintptr_t index =3D tlb_index(env, mmu_idx, addr); + CPUTLBEntry *entry =3D tlb_entry(env, mmu_idx, index); + target_ulong tlb_addr =3D entry->ADDR_READ; unsigned a_bits =3D get_alignment_bits(get_memop(oi)); uintptr_t haddr; DATA_TYPE res; @@ -129,7 +130,7 @@ WORD_TYPE helper_le_ld_name(CPUArchState *env, target_u= long addr, tlb_fill(ENV_GET_CPU(env), addr, DATA_SIZE, READ_ACCESS_TYPE, mmu_idx, retaddr); } - tlb_addr =3D env->tlb_table[mmu_idx][index].ADDR_READ; + tlb_addr =3D entry->ADDR_READ; } =20 /* Handle an IO access. */ @@ -166,7 +167,7 @@ WORD_TYPE helper_le_ld_name(CPUArchState *env, target_u= long addr, return res; } =20 - haddr =3D addr + env->tlb_table[mmu_idx][index].addend; + haddr =3D addr + entry->addend; #if DATA_SIZE =3D=3D 1 res =3D glue(glue(ld, LSUFFIX), _p)((uint8_t *)haddr); #else @@ -179,9 +180,10 @@ WORD_TYPE helper_le_ld_name(CPUArchState *env, target_= ulong addr, WORD_TYPE helper_be_ld_name(CPUArchState *env, target_ulong addr, TCGMemOpIdx oi, uintptr_t retaddr) { - unsigned mmu_idx =3D get_mmuidx(oi); - int index =3D (addr >> TARGET_PAGE_BITS) & (CPU_TLB_SIZE - 1); - target_ulong tlb_addr =3D env->tlb_table[mmu_idx][index].ADDR_READ; + uintptr_t mmu_idx =3D get_mmuidx(oi); + uintptr_t index =3D tlb_index(env, mmu_idx, addr); + CPUTLBEntry *entry =3D tlb_entry(env, mmu_idx, index); + target_ulong tlb_addr =3D entry->ADDR_READ; unsigned a_bits =3D get_alignment_bits(get_memop(oi)); uintptr_t haddr; DATA_TYPE res; @@ -197,7 +199,7 @@ WORD_TYPE helper_be_ld_name(CPUArchState *env, target_u= long addr, tlb_fill(ENV_GET_CPU(env), addr, DATA_SIZE, READ_ACCESS_TYPE, mmu_idx, retaddr); } - tlb_addr =3D env->tlb_table[mmu_idx][index].ADDR_READ; + tlb_addr =3D entry->ADDR_READ; } =20 /* Handle an IO access. */ @@ -234,7 +236,7 @@ WORD_TYPE helper_be_ld_name(CPUArchState *env, target_u= long addr, return res; } =20 - haddr =3D addr + env->tlb_table[mmu_idx][index].addend; + haddr =3D addr + entry->addend; res =3D glue(glue(ld, LSUFFIX), _be_p)((uint8_t *)haddr); return res; } @@ -275,9 +277,10 @@ static inline void glue(io_write, SUFFIX)(CPUArchState= *env, void helper_le_st_name(CPUArchState *env, target_ulong addr, DATA_TYPE val, TCGMemOpIdx oi, uintptr_t retaddr) { - unsigned mmu_idx =3D get_mmuidx(oi); - int index =3D (addr >> TARGET_PAGE_BITS) & (CPU_TLB_SIZE - 1); - target_ulong tlb_addr =3D env->tlb_table[mmu_idx][index].addr_write; + uintptr_t mmu_idx =3D get_mmuidx(oi); + uintptr_t index =3D tlb_index(env, mmu_idx, addr); + CPUTLBEntry *entry =3D tlb_entry(env, mmu_idx, index); + target_ulong tlb_addr =3D entry->addr_write; unsigned a_bits =3D get_alignment_bits(get_memop(oi)); uintptr_t haddr; =20 @@ -292,7 +295,7 @@ void helper_le_st_name(CPUArchState *env, target_ulong = addr, DATA_TYPE val, tlb_fill(ENV_GET_CPU(env), addr, DATA_SIZE, MMU_DATA_STORE, mmu_idx, retaddr); } - tlb_addr =3D env->tlb_table[mmu_idx][index].addr_write & ~TLB_INVA= LID_MASK; + tlb_addr =3D entry->addr_write & ~TLB_INVALID_MASK; } =20 /* Handle an IO access. */ @@ -313,16 +316,16 @@ void helper_le_st_name(CPUArchState *env, target_ulon= g addr, DATA_TYPE val, if (DATA_SIZE > 1 && unlikely((addr & ~TARGET_PAGE_MASK) + DATA_SIZE - 1 >=3D TARGET_PAGE_SIZE)) { - int i, index2; - target_ulong page2, tlb_addr2; + int i; + target_ulong page2; + CPUTLBEntry *entry2; do_unaligned_access: /* Ensure the second page is in the TLB. Note that the first page is already guaranteed to be filled, and that the second page cannot evict the first. */ page2 =3D (addr + DATA_SIZE) & TARGET_PAGE_MASK; - index2 =3D (page2 >> TARGET_PAGE_BITS) & (CPU_TLB_SIZE - 1); - tlb_addr2 =3D env->tlb_table[mmu_idx][index2].addr_write; - if (!tlb_hit_page(tlb_addr2, page2) + entry2 =3D tlb_entry(env, mmu_idx, page2); + if (!tlb_hit_page(entry2->addr_write, page2) && !VICTIM_TLB_HIT(addr_write, page2)) { tlb_fill(ENV_GET_CPU(env), page2, DATA_SIZE, MMU_DATA_STORE, mmu_idx, retaddr); @@ -340,7 +343,7 @@ void helper_le_st_name(CPUArchState *env, target_ulong = addr, DATA_TYPE val, return; } =20 - haddr =3D addr + env->tlb_table[mmu_idx][index].addend; + haddr =3D addr + entry->addend; #if DATA_SIZE =3D=3D 1 glue(glue(st, SUFFIX), _p)((uint8_t *)haddr, val); #else @@ -352,9 +355,10 @@ void helper_le_st_name(CPUArchState *env, target_ulong= addr, DATA_TYPE val, void helper_be_st_name(CPUArchState *env, target_ulong addr, DATA_TYPE val, TCGMemOpIdx oi, uintptr_t retaddr) { - unsigned mmu_idx =3D get_mmuidx(oi); - int index =3D (addr >> TARGET_PAGE_BITS) & (CPU_TLB_SIZE - 1); - target_ulong tlb_addr =3D env->tlb_table[mmu_idx][index].addr_write; + uintptr_t mmu_idx =3D get_mmuidx(oi); + uintptr_t index =3D tlb_index(env, mmu_idx, addr); + CPUTLBEntry *entry =3D tlb_entry(env, mmu_idx, index); + target_ulong tlb_addr =3D entry->addr_write; unsigned a_bits =3D get_alignment_bits(get_memop(oi)); uintptr_t haddr; =20 @@ -369,7 +373,7 @@ void helper_be_st_name(CPUArchState *env, target_ulong = addr, DATA_TYPE val, tlb_fill(ENV_GET_CPU(env), addr, DATA_SIZE, MMU_DATA_STORE, mmu_idx, retaddr); } - tlb_addr =3D env->tlb_table[mmu_idx][index].addr_write & ~TLB_INVA= LID_MASK; + tlb_addr =3D entry->addr_write & ~TLB_INVALID_MASK; } =20 /* Handle an IO access. */ @@ -381,8 +385,8 @@ void helper_be_st_name(CPUArchState *env, target_ulong = addr, DATA_TYPE val, /* ??? Note that the io helpers always read data in the target byte ordering. We should push the LE/BE request down into io. = */ val =3D TGT_BE(val); - glue(io_write, SUFFIX)(env, mmu_idx, index, val, addr, retaddr, - tlb_addr & TLB_RECHECK); + glue(io_write, SUFFIX)(env, mmu_idx, index, val, addr, + retaddr, tlb_addr & TLB_RECHECK); return; } =20 @@ -390,16 +394,16 @@ void helper_be_st_name(CPUArchState *env, target_ulon= g addr, DATA_TYPE val, if (DATA_SIZE > 1 && unlikely((addr & ~TARGET_PAGE_MASK) + DATA_SIZE - 1 >=3D TARGET_PAGE_SIZE)) { - int i, index2; - target_ulong page2, tlb_addr2; + int i; + target_ulong page2; + CPUTLBEntry *entry2; do_unaligned_access: /* Ensure the second page is in the TLB. Note that the first page is already guaranteed to be filled, and that the second page cannot evict the first. */ page2 =3D (addr + DATA_SIZE) & TARGET_PAGE_MASK; - index2 =3D (page2 >> TARGET_PAGE_BITS) & (CPU_TLB_SIZE - 1); - tlb_addr2 =3D env->tlb_table[mmu_idx][index2].addr_write; - if (!tlb_hit_page(tlb_addr2, page2) + entry2 =3D tlb_entry(env, mmu_idx, page2); + if (!tlb_hit_page(entry2->addr_write, page2) && !VICTIM_TLB_HIT(addr_write, page2)) { tlb_fill(ENV_GET_CPU(env), page2, DATA_SIZE, MMU_DATA_STORE, mmu_idx, retaddr); @@ -417,7 +421,7 @@ void helper_be_st_name(CPUArchState *env, target_ulong = addr, DATA_TYPE val, return; } =20 - haddr =3D addr + env->tlb_table[mmu_idx][index].addend; + haddr =3D addr + entry->addend; glue(glue(st, SUFFIX), _be_p)((uint8_t *)haddr, val); } #endif /* DATA_SIZE > 1 */ diff --git a/include/exec/cpu_ldst.h b/include/exec/cpu_ldst.h index 41ed0526e2..47aaf21d94 100644 --- a/include/exec/cpu_ldst.h +++ b/include/exec/cpu_ldst.h @@ -126,6 +126,20 @@ extern __thread uintptr_t helper_retaddr; /* The memory helpers for tcg-generated code need tcg_target_long etc. */ #include "tcg.h" =20 +/* Find the TLB index corresponding to the mmu_idx + address pair. */ +static inline uintptr_t tlb_index(CPUArchState *env, uintptr_t mmu_idx, + target_ulong addr) +{ + return (addr >> TARGET_PAGE_BITS) & (CPU_TLB_SIZE - 1); +} + +/* Find the TLB entry corresponding to the mmu_idx + address pair. */ +static inline CPUTLBEntry *tlb_entry(CPUArchState *env, uintptr_t mmu_idx, + target_ulong addr) +{ + return &env->tlb_table[mmu_idx][tlb_index(env, mmu_idx, addr)]; +} + #ifdef MMU_MODE0_SUFFIX #define CPU_MMU_INDEX 0 #define MEMSUFFIX MMU_MODE0_SUFFIX diff --git a/include/exec/cpu_ldst_template.h b/include/exec/cpu_ldst_templ= ate.h index 4db2302962..d21a0b59bf 100644 --- a/include/exec/cpu_ldst_template.h +++ b/include/exec/cpu_ldst_template.h @@ -81,7 +81,7 @@ glue(glue(glue(cpu_ld, USUFFIX), MEMSUFFIX), _ra)(CPUArch= State *env, target_ulong ptr, uintptr_t retaddr) { - int page_index; + CPUTLBEntry *entry; RES_TYPE res; target_ulong addr; int mmu_idx; @@ -94,15 +94,15 @@ glue(glue(glue(cpu_ld, USUFFIX), MEMSUFFIX), _ra)(CPUAr= chState *env, #endif =20 addr =3D ptr; - page_index =3D (addr >> TARGET_PAGE_BITS) & (CPU_TLB_SIZE - 1); mmu_idx =3D CPU_MMU_INDEX; - if (unlikely(env->tlb_table[mmu_idx][page_index].ADDR_READ !=3D + entry =3D tlb_entry(env, mmu_idx, addr); + if (unlikely(entry->ADDR_READ !=3D (addr & (TARGET_PAGE_MASK | (DATA_SIZE - 1))))) { oi =3D make_memop_idx(SHIFT, mmu_idx); res =3D glue(glue(helper_ret_ld, URETSUFFIX), MMUSUFFIX)(env, addr, oi, retaddr); } else { - uintptr_t hostaddr =3D addr + env->tlb_table[mmu_idx][page_index].= addend; + uintptr_t hostaddr =3D addr + entry->addend; res =3D glue(glue(ld, USUFFIX), _p)((uint8_t *)hostaddr); } return res; @@ -120,7 +120,8 @@ glue(glue(glue(cpu_lds, SUFFIX), MEMSUFFIX), _ra)(CPUAr= chState *env, target_ulong ptr, uintptr_t retaddr) { - int res, page_index; + CPUTLBEntry *entry; + int res; target_ulong addr; int mmu_idx; TCGMemOpIdx oi; @@ -132,15 +133,15 @@ glue(glue(glue(cpu_lds, SUFFIX), MEMSUFFIX), _ra)(CPU= ArchState *env, #endif =20 addr =3D ptr; - page_index =3D (addr >> TARGET_PAGE_BITS) & (CPU_TLB_SIZE - 1); mmu_idx =3D CPU_MMU_INDEX; - if (unlikely(env->tlb_table[mmu_idx][page_index].ADDR_READ !=3D + entry =3D tlb_entry(env, mmu_idx, addr); + if (unlikely(entry->ADDR_READ !=3D (addr & (TARGET_PAGE_MASK | (DATA_SIZE - 1))))) { oi =3D make_memop_idx(SHIFT, mmu_idx); res =3D (DATA_STYPE)glue(glue(helper_ret_ld, SRETSUFFIX), MMUSUFFIX)(env, addr, oi, retaddr); } else { - uintptr_t hostaddr =3D addr + env->tlb_table[mmu_idx][page_index].= addend; + uintptr_t hostaddr =3D addr + entry->addend; res =3D glue(glue(lds, SUFFIX), _p)((uint8_t *)hostaddr); } return res; @@ -162,7 +163,7 @@ glue(glue(glue(cpu_st, SUFFIX), MEMSUFFIX), _ra)(CPUArc= hState *env, target_ulong ptr, RES_TYPE v, uintptr_t ret= addr) { - int page_index; + CPUTLBEntry *entry; target_ulong addr; int mmu_idx; TCGMemOpIdx oi; @@ -174,15 +175,15 @@ glue(glue(glue(cpu_st, SUFFIX), MEMSUFFIX), _ra)(CPUA= rchState *env, #endif =20 addr =3D ptr; - page_index =3D (addr >> TARGET_PAGE_BITS) & (CPU_TLB_SIZE - 1); mmu_idx =3D CPU_MMU_INDEX; - if (unlikely(env->tlb_table[mmu_idx][page_index].addr_write !=3D + entry =3D tlb_entry(env, mmu_idx, addr); + if (unlikely(entry->addr_write !=3D (addr & (TARGET_PAGE_MASK | (DATA_SIZE - 1))))) { oi =3D make_memop_idx(SHIFT, mmu_idx); glue(glue(helper_ret_st, SUFFIX), MMUSUFFIX)(env, addr, v, oi, retaddr); } else { - uintptr_t hostaddr =3D addr + env->tlb_table[mmu_idx][page_index].= addend; + uintptr_t hostaddr =3D addr + entry->addend; glue(glue(st, SUFFIX), _p)((uint8_t *)hostaddr, v); } } diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index f4702ce91f..015f9d58dd 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -263,7 +263,6 @@ static void tlb_flush_page_async_work(CPUState *cpu, ru= n_on_cpu_data data) { CPUArchState *env =3D cpu->env_ptr; target_ulong addr =3D (target_ulong) data.target_ptr; - int i; int mmu_idx; =20 assert_cpu_is_self(cpu); @@ -281,9 +280,8 @@ static void tlb_flush_page_async_work(CPUState *cpu, ru= n_on_cpu_data data) } =20 addr &=3D TARGET_PAGE_MASK; - i =3D (addr >> TARGET_PAGE_BITS) & (CPU_TLB_SIZE - 1); for (mmu_idx =3D 0; mmu_idx < NB_MMU_MODES; mmu_idx++) { - tlb_flush_entry(&env->tlb_table[mmu_idx][i], addr); + tlb_flush_entry(tlb_entry(env, mmu_idx, addr), addr); tlb_flush_vtlb_page(env, mmu_idx, addr); } =20 @@ -314,17 +312,16 @@ static void tlb_flush_page_by_mmuidx_async_work(CPUSt= ate *cpu, target_ulong addr_and_mmuidx =3D (target_ulong) data.target_ptr; target_ulong addr =3D addr_and_mmuidx & TARGET_PAGE_MASK; unsigned long mmu_idx_bitmap =3D addr_and_mmuidx & ALL_MMUIDX_BITS; - int page =3D (addr >> TARGET_PAGE_BITS) & (CPU_TLB_SIZE - 1); int mmu_idx; =20 assert_cpu_is_self(cpu); =20 - tlb_debug("page:%d addr:"TARGET_FMT_lx" mmu_idx:0x%lx\n", - page, addr, mmu_idx_bitmap); + tlb_debug("flush page addr:"TARGET_FMT_lx" mmu_idx:0x%lx\n", + addr, mmu_idx_bitmap); =20 for (mmu_idx =3D 0; mmu_idx < NB_MMU_MODES; mmu_idx++) { if (test_bit(mmu_idx, &mmu_idx_bitmap)) { - tlb_flush_entry(&env->tlb_table[mmu_idx][page], addr); + tlb_flush_entry(tlb_entry(env, mmu_idx, addr), addr); tlb_flush_vtlb_page(env, mmu_idx, addr); } } @@ -551,15 +548,13 @@ static inline void tlb_set_dirty1(CPUTLBEntry *tlb_en= try, target_ulong vaddr) void tlb_set_dirty(CPUState *cpu, target_ulong vaddr) { CPUArchState *env =3D cpu->env_ptr; - int i; int mmu_idx; =20 assert_cpu_is_self(cpu); =20 vaddr &=3D TARGET_PAGE_MASK; - i =3D (vaddr >> TARGET_PAGE_BITS) & (CPU_TLB_SIZE - 1); for (mmu_idx =3D 0; mmu_idx < NB_MMU_MODES; mmu_idx++) { - tlb_set_dirty1(&env->tlb_table[mmu_idx][i], vaddr); + tlb_set_dirty1(tlb_entry(env, mmu_idx, vaddr), vaddr); } =20 for (mmu_idx =3D 0; mmu_idx < NB_MMU_MODES; mmu_idx++) { @@ -661,8 +656,8 @@ void tlb_set_page_with_attrs(CPUState *cpu, target_ulon= g vaddr, iotlb =3D memory_region_section_get_iotlb(cpu, section, vaddr_page, paddr_page, xlat, prot, &addre= ss); =20 - index =3D (vaddr_page >> TARGET_PAGE_BITS) & (CPU_TLB_SIZE - 1); - te =3D &env->tlb_table[mmu_idx][index]; + index =3D tlb_index(env, mmu_idx, vaddr_page); + te =3D tlb_entry(env, mmu_idx, vaddr_page); =20 /* * Only evict the old entry to the victim tlb if it's for a @@ -773,16 +768,16 @@ static uint64_t io_readx(CPUArchState *env, CPUIOTLBE= ntry *iotlbentry, * repeat the MMU check here. This tlb_fill() call might * longjump out if this access should cause a guest exception. */ - int index; + CPUTLBEntry *entry; target_ulong tlb_addr; =20 tlb_fill(cpu, addr, size, MMU_DATA_LOAD, mmu_idx, retaddr); =20 - index =3D (addr >> TARGET_PAGE_BITS) & (CPU_TLB_SIZE - 1); - tlb_addr =3D env->tlb_table[mmu_idx][index].addr_read; + entry =3D tlb_entry(env, mmu_idx, addr); + tlb_addr =3D entry->addr_read; if (!(tlb_addr & ~(TARGET_PAGE_MASK | TLB_RECHECK))) { /* RAM access */ - uintptr_t haddr =3D addr + env->tlb_table[mmu_idx][index].adde= nd; + uintptr_t haddr =3D addr + entry->addend; =20 return ldn_p((void *)haddr, size); } @@ -840,16 +835,16 @@ static void io_writex(CPUArchState *env, CPUIOTLBEntr= y *iotlbentry, * repeat the MMU check here. This tlb_fill() call might * longjump out if this access should cause a guest exception. */ - int index; + CPUTLBEntry *entry; target_ulong tlb_addr; =20 tlb_fill(cpu, addr, size, MMU_DATA_STORE, mmu_idx, retaddr); =20 - index =3D (addr >> TARGET_PAGE_BITS) & (CPU_TLB_SIZE - 1); - tlb_addr =3D env->tlb_table[mmu_idx][index].addr_write; + entry =3D tlb_entry(env, mmu_idx, addr); + tlb_addr =3D entry->addr_write; if (!(tlb_addr & ~(TARGET_PAGE_MASK | TLB_RECHECK))) { /* RAM access */ - uintptr_t haddr =3D addr + env->tlb_table[mmu_idx][index].adde= nd; + uintptr_t haddr =3D addr + entry->addend; =20 stn_p((void *)haddr, size, val); return; @@ -924,20 +919,19 @@ static bool victim_tlb_hit(CPUArchState *env, size_t = mmu_idx, size_t index, */ tb_page_addr_t get_page_addr_code(CPUArchState *env, target_ulong addr) { - int mmu_idx, index; + uintptr_t mmu_idx =3D cpu_mmu_index(env, true); + uintptr_t index =3D tlb_index(env, mmu_idx, addr); + CPUTLBEntry *entry =3D tlb_entry(env, mmu_idx, addr); void *p; =20 - index =3D (addr >> TARGET_PAGE_BITS) & (CPU_TLB_SIZE - 1); - mmu_idx =3D cpu_mmu_index(env, true); - if (unlikely(!tlb_hit(env->tlb_table[mmu_idx][index].addr_code, addr))= ) { + if (unlikely(!tlb_hit(entry->addr_code, addr))) { if (!VICTIM_TLB_HIT(addr_code, addr)) { tlb_fill(ENV_GET_CPU(env), addr, 0, MMU_INST_FETCH, mmu_idx, 0= ); } - assert(tlb_hit(env->tlb_table[mmu_idx][index].addr_code, addr)); + assert(tlb_hit(entry->addr_code, addr)); } =20 - if (unlikely(env->tlb_table[mmu_idx][index].addr_code & - (TLB_RECHECK | TLB_MMIO))) { + if (unlikely(entry->addr_code & (TLB_RECHECK | TLB_MMIO))) { /* * Return -1 if we can't translate and execute from an entire * page of RAM here, which will cause us to execute by loading @@ -949,7 +943,7 @@ tb_page_addr_t get_page_addr_code(CPUArchState *env, ta= rget_ulong addr) return -1; } =20 - p =3D (void *)((uintptr_t)addr + env->tlb_table[mmu_idx][index].addend= ); + p =3D (void *)((uintptr_t)addr + entry->addend); return qemu_ram_addr_from_host_nofail(p); } =20 @@ -962,10 +956,10 @@ tb_page_addr_t get_page_addr_code(CPUArchState *env, = target_ulong addr) void probe_write(CPUArchState *env, target_ulong addr, int size, int mmu_i= dx, uintptr_t retaddr) { - int index =3D (addr >> TARGET_PAGE_BITS) & (CPU_TLB_SIZE - 1); - target_ulong tlb_addr =3D env->tlb_table[mmu_idx][index].addr_write; + uintptr_t index =3D tlb_index(env, mmu_idx, addr); + CPUTLBEntry *entry =3D tlb_entry(env, mmu_idx, addr); =20 - if (!tlb_hit(tlb_addr, addr)) { + if (!tlb_hit(entry->addr_write, addr)) { /* TLB entry is for a different page */ if (!VICTIM_TLB_HIT(addr_write, addr)) { tlb_fill(ENV_GET_CPU(env), addr, size, MMU_DATA_STORE, @@ -981,8 +975,8 @@ static void *atomic_mmu_lookup(CPUArchState *env, targe= t_ulong addr, NotDirtyInfo *ndi) { size_t mmu_idx =3D get_mmuidx(oi); - size_t index =3D (addr >> TARGET_PAGE_BITS) & (CPU_TLB_SIZE - 1); - CPUTLBEntry *tlbe =3D &env->tlb_table[mmu_idx][index]; + uintptr_t index =3D tlb_index(env, mmu_idx, addr); + CPUTLBEntry *tlbe =3D tlb_entry(env, mmu_idx, addr); target_ulong tlb_addr =3D tlbe->addr_write; TCGMemOp mop =3D get_memop(oi); int a_bits =3D get_alignment_bits(mop); --=20 2.17.1