From nobody Tue Feb 10 07:42:12 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1626817290220274.2295579268987; Tue, 20 Jul 2021 14:41:30 -0700 (PDT) Received: from localhost ([::1]:58216 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1m5xUX-0004OC-4w for importer@patchew.org; Tue, 20 Jul 2021 17:41:29 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50046) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m5xQo-00064Z-Qx for qemu-devel@nongnu.org; Tue, 20 Jul 2021 17:37:39 -0400 Received: from mail-pg1-x531.google.com ([2607:f8b0:4864:20::531]:42595) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1m5xQl-0000DY-DJ for qemu-devel@nongnu.org; Tue, 20 Jul 2021 17:37:38 -0400 Received: by mail-pg1-x531.google.com with SMTP id i16so6208642pgi.9 for ; Tue, 20 Jul 2021 14:37:33 -0700 (PDT) Received: from localhost.localdomain (204-210-126-223.res.spectrum.com. [204.210.126.223]) by smtp.gmail.com with ESMTPSA id r10sm24376813pff.7.2021.07.20.14.37.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Jul 2021 14:37:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=yFBmZYnlqnGmH22vv1l18Boh2+T/swIBlrsgKz43rXg=; b=uKmNJPRq53SWR+Hu6NBupURnnGvlsJUcA+7+JAQpfr9n8a+JaG+dh7PPU4GxV4nFR6 ilCFPPv4puu9P4eCS+RxuWgkGKr19X9qD6hoYOXrzGEcY4iSMUrLGAgCUfh3Rkr2HKFL Y2rqPvmnJwlta0qYyuzAkXqa7W+iJy0+Lvj82yCFMQk+EOGW13W/XprVKnLKGC8L5XOG eAIKhBeInexek6cEeGNxZcrRwUtLqLvsdSEyzaCGRp0mDnTEym5vUbrQfUQD7cvt7AQT vKPTEcID1u/cWdTNkjRaH2FRX31BtEEw5Hnf48QwQupzFh7VnM+08Ac5QhZURIW5DJHy 1TXg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=yFBmZYnlqnGmH22vv1l18Boh2+T/swIBlrsgKz43rXg=; b=Ok49lAHVsRXiGO60INl2eW4CMJbLXnUjyiuor1w9WLS9brmWY3lvQAk9BAmnvitPYL hoZNP1QuujCIlSGow52cchWup2nFq2DSw7OaUN4GGEC0yJpp4m9a2USFqlQ94ho5Oixe IUQdmoBkw9sPV+sRABPbJvduU8OSRGii8mGg0CoIUdglQBEjMP9BKJo+8+7aa1qWQqXG rFmADoDxubuIslpByeX/KigcO+lDqL2Py00uWvDXKUJTbjZFt6w3Q6eeDSRgh25fVp+I V8c8F806YseLIb+BXz3GHoLj5oc6wZxdnL9d+OXAZg0NHP4+DadEYaCOYWU8CCvaVXfm KXIA== X-Gm-Message-State: AOAM532YTTYmIA/AtUdSjZMv5Unp2Cv5jxDwYVMJu2NnF6BPaRX9vOgf SCk8baX8DQMJT9mv3S5Xq8Kacu+TvijY7w== X-Google-Smtp-Source: ABdhPJxZACFdEnolXTKWEZGMajG7gnrDVN2VrfImCxFPA1r7r6LV86BC0PNo1mnjgqtg/spv/6qJDg== X-Received: by 2002:a65:4d4c:: with SMTP id j12mr32379672pgt.311.1626817052125; Tue, 20 Jul 2021 14:37:32 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 04/10] tcg: Rename helper_atomic_*_mmu and provide for user-only Date: Tue, 20 Jul 2021 11:37:17 -1000 Message-Id: <20210720213723.630552-5-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210720213723.630552-1-richard.henderson@linaro.org> References: <20210720213723.630552-1-richard.henderson@linaro.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::531; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x531.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Peter Maydell , Cole Robinson Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1626817291657100001 Content-Type: text/plain; charset="utf-8" Always provide the atomic interface using TCGMemOpIdx oi and uintptr_t retaddr. Rename from helper_* to cpu_* so as to (mostly) match the exec/cpu_ldst.h functions, and to emphasize that they are not callable from TCG directly. Tested-by: Cole Robinson Reviewed-by: Peter Maydell Signed-off-by: Richard Henderson --- include/tcg/tcg.h | 78 ++++++++++++++++------------------- accel/tcg/cputlb.c | 8 ++-- accel/tcg/user-exec.c | 59 ++++++++++++++++---------- target/arm/helper-a64.c | 8 ++-- target/i386/tcg/mem_helper.c | 15 +------ target/m68k/op_helper.c | 19 +++------ target/ppc/mem_helper.c | 16 +++---- target/s390x/tcg/mem_helper.c | 19 ++++----- 8 files changed, 104 insertions(+), 118 deletions(-) diff --git a/include/tcg/tcg.h b/include/tcg/tcg.h index 25dd19d6e1..44ccd86f3e 100644 --- a/include/tcg/tcg.h +++ b/include/tcg/tcg.h @@ -1341,31 +1341,32 @@ void helper_be_stq_mmu(CPUArchState *env, target_ul= ong addr, uint64_t val, # define helper_ret_stl_mmu helper_le_stl_mmu # define helper_ret_stq_mmu helper_le_stq_mmu #endif +#endif /* CONFIG_SOFTMMU */ =20 -uint32_t helper_atomic_cmpxchgb_mmu(CPUArchState *env, target_ulong addr, +uint32_t cpu_atomic_cmpxchgb_mmu(CPUArchState *env, target_ulong addr, + uint32_t cmpv, uint32_t newv, + TCGMemOpIdx oi, uintptr_t retaddr); +uint32_t cpu_atomic_cmpxchgw_le_mmu(CPUArchState *env, target_ulong addr, uint32_t cmpv, uint32_t newv, TCGMemOpIdx oi, uintptr_t retaddr); -uint32_t helper_atomic_cmpxchgw_le_mmu(CPUArchState *env, target_ulong add= r, - uint32_t cmpv, uint32_t newv, - TCGMemOpIdx oi, uintptr_t retaddr); -uint32_t helper_atomic_cmpxchgl_le_mmu(CPUArchState *env, target_ulong add= r, - uint32_t cmpv, uint32_t newv, - TCGMemOpIdx oi, uintptr_t retaddr); -uint64_t helper_atomic_cmpxchgq_le_mmu(CPUArchState *env, target_ulong add= r, - uint64_t cmpv, uint64_t newv, - TCGMemOpIdx oi, uintptr_t retaddr); -uint32_t helper_atomic_cmpxchgw_be_mmu(CPUArchState *env, target_ulong add= r, - uint32_t cmpv, uint32_t newv, - TCGMemOpIdx oi, uintptr_t retaddr); -uint32_t helper_atomic_cmpxchgl_be_mmu(CPUArchState *env, target_ulong add= r, - uint32_t cmpv, uint32_t newv, - TCGMemOpIdx oi, uintptr_t retaddr); -uint64_t helper_atomic_cmpxchgq_be_mmu(CPUArchState *env, target_ulong add= r, - uint64_t cmpv, uint64_t newv, - TCGMemOpIdx oi, uintptr_t retaddr); +uint32_t cpu_atomic_cmpxchgl_le_mmu(CPUArchState *env, target_ulong addr, + uint32_t cmpv, uint32_t newv, + TCGMemOpIdx oi, uintptr_t retaddr); +uint64_t cpu_atomic_cmpxchgq_le_mmu(CPUArchState *env, target_ulong addr, + uint64_t cmpv, uint64_t newv, + TCGMemOpIdx oi, uintptr_t retaddr); +uint32_t cpu_atomic_cmpxchgw_be_mmu(CPUArchState *env, target_ulong addr, + uint32_t cmpv, uint32_t newv, + TCGMemOpIdx oi, uintptr_t retaddr); +uint32_t cpu_atomic_cmpxchgl_be_mmu(CPUArchState *env, target_ulong addr, + uint32_t cmpv, uint32_t newv, + TCGMemOpIdx oi, uintptr_t retaddr); +uint64_t cpu_atomic_cmpxchgq_be_mmu(CPUArchState *env, target_ulong addr, + uint64_t cmpv, uint64_t newv, + TCGMemOpIdx oi, uintptr_t retaddr); =20 #define GEN_ATOMIC_HELPER(NAME, TYPE, SUFFIX) \ -TYPE helper_atomic_ ## NAME ## SUFFIX ## _mmu \ +TYPE cpu_atomic_ ## NAME ## SUFFIX ## _mmu \ (CPUArchState *env, target_ulong addr, TYPE val, \ TCGMemOpIdx oi, uintptr_t retaddr); =20 @@ -1411,31 +1412,22 @@ GEN_ATOMIC_HELPER_ALL(xchg) =20 #undef GEN_ATOMIC_HELPER_ALL #undef GEN_ATOMIC_HELPER -#endif /* CONFIG_SOFTMMU */ =20 -/* - * These aren't really a "proper" helpers because TCG cannot manage Int128. - * However, use the same format as the others, for use by the backends. - * - * The cmpxchg functions are only defined if HAVE_CMPXCHG128; - * the ld/st functions are only defined if HAVE_ATOMIC128, - * as defined by . - */ -Int128 helper_atomic_cmpxchgo_le_mmu(CPUArchState *env, target_ulong addr, - Int128 cmpv, Int128 newv, - TCGMemOpIdx oi, uintptr_t retaddr); -Int128 helper_atomic_cmpxchgo_be_mmu(CPUArchState *env, target_ulong addr, - Int128 cmpv, Int128 newv, - TCGMemOpIdx oi, uintptr_t retaddr); +Int128 cpu_atomic_cmpxchgo_le_mmu(CPUArchState *env, target_ulong addr, + Int128 cmpv, Int128 newv, + TCGMemOpIdx oi, uintptr_t retaddr); +Int128 cpu_atomic_cmpxchgo_be_mmu(CPUArchState *env, target_ulong addr, + Int128 cmpv, Int128 newv, + TCGMemOpIdx oi, uintptr_t retaddr); =20 -Int128 helper_atomic_ldo_le_mmu(CPUArchState *env, target_ulong addr, - TCGMemOpIdx oi, uintptr_t retaddr); -Int128 helper_atomic_ldo_be_mmu(CPUArchState *env, target_ulong addr, - TCGMemOpIdx oi, uintptr_t retaddr); -void helper_atomic_sto_le_mmu(CPUArchState *env, target_ulong addr, Int128= val, - TCGMemOpIdx oi, uintptr_t retaddr); -void helper_atomic_sto_be_mmu(CPUArchState *env, target_ulong addr, Int128= val, - TCGMemOpIdx oi, uintptr_t retaddr); +Int128 cpu_atomic_ldo_le_mmu(CPUArchState *env, target_ulong addr, + TCGMemOpIdx oi, uintptr_t retaddr); +Int128 cpu_atomic_ldo_be_mmu(CPUArchState *env, target_ulong addr, + TCGMemOpIdx oi, uintptr_t retaddr); +void cpu_atomic_sto_le_mmu(CPUArchState *env, target_ulong addr, Int128 va= l, + TCGMemOpIdx oi, uintptr_t retaddr); +void cpu_atomic_sto_be_mmu(CPUArchState *env, target_ulong addr, Int128 va= l, + TCGMemOpIdx oi, uintptr_t retaddr); =20 #ifdef CONFIG_DEBUG_TCG void tcg_assert_listed_vecop(TCGOpcode); diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index b4e15b6aad..63da1cc96f 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -2686,12 +2686,14 @@ void cpu_stq_le_data(CPUArchState *env, target_ulon= g ptr, uint64_t val) cpu_stq_le_data_ra(env, ptr, val, 0); } =20 -/* First set of helpers allows passing in of OI and RETADDR. This makes - them callable from other helpers. */ +/* + * First set of functions passes in OI and RETADDR. + * This makes them callable from other helpers. + */ =20 #define EXTRA_ARGS , TCGMemOpIdx oi, uintptr_t retaddr #define ATOMIC_NAME(X) \ - HELPER(glue(glue(glue(atomic_ ## X, SUFFIX), END), _mmu)) + glue(glue(glue(cpu_atomic_ ## X, SUFFIX), END), _mmu) #define ATOMIC_MMU_DECLS #define ATOMIC_MMU_LOOKUP_RW \ atomic_mmu_lookup(env, addr, oi, DATA_SIZE, PAGE_READ | PAGE_WRITE, re= taddr) diff --git a/accel/tcg/user-exec.c b/accel/tcg/user-exec.c index ba09fd0413..82dbe06f08 100644 --- a/accel/tcg/user-exec.c +++ b/accel/tcg/user-exec.c @@ -1234,19 +1234,23 @@ static void *atomic_mmu_lookup(CPUArchState *env, t= arget_ulong addr, return ret; } =20 -/* Macro to call the above, with local variables from the use context. */ -#define ATOMIC_MMU_DECLS do {} while (0) -#define ATOMIC_MMU_LOOKUP_RW atomic_mmu_lookup(env, addr, DATA_SIZE, GETP= C()) +#include "atomic_common.c.inc" + +/* + * First set of functions passes in OI and RETADDR. + * This makes them callable from other helpers. + */ + +#define EXTRA_ARGS , TCGMemOpIdx oi, uintptr_t retaddr +#define ATOMIC_NAME(X) \ + glue(glue(glue(cpu_atomic_ ## X, SUFFIX), END), _mmu) +#define ATOMIC_MMU_DECLS +#define ATOMIC_MMU_LOOKUP_RW atomic_mmu_lookup(env, addr, DATA_SIZE, reta= ddr) #define ATOMIC_MMU_LOOKUP_R ATOMIC_MMU_LOOKUP_RW #define ATOMIC_MMU_LOOKUP_W ATOMIC_MMU_LOOKUP_RW #define ATOMIC_MMU_CLEANUP do { clear_helper_retaddr(); } while (0) #define ATOMIC_MMU_IDX MMU_USER_IDX =20 -#define ATOMIC_NAME(X) HELPER(glue(glue(atomic_ ## X, SUFFIX), END)) -#define EXTRA_ARGS - -#include "atomic_common.c.inc" - #define DATA_SIZE 1 #include "atomic_template.h" =20 @@ -1261,20 +1265,33 @@ static void *atomic_mmu_lookup(CPUArchState *env, t= arget_ulong addr, #include "atomic_template.h" #endif =20 -/* The following is only callable from other helpers, and matches up - with the softmmu version. */ - #if HAVE_ATOMIC128 || HAVE_CMPXCHG128 - -#undef EXTRA_ARGS -#undef ATOMIC_NAME -#undef ATOMIC_MMU_LOOKUP_RW - -#define EXTRA_ARGS , TCGMemOpIdx oi, uintptr_t retaddr -#define ATOMIC_NAME(X) \ - HELPER(glue(glue(glue(atomic_ ## X, SUFFIX), END), _mmu)) -#define ATOMIC_MMU_LOOKUP_RW atomic_mmu_lookup(env, addr, DATA_SIZE, reta= ddr) - #define DATA_SIZE 16 #include "atomic_template.h" #endif + +/* + * Second set of functions is directly callable from TCG. + */ + +#undef EXTRA_ARGS +#undef ATOMIC_NAME +#undef ATOMIC_MMU_DECLS + +#define ATOMIC_NAME(X) HELPER(glue(glue(atomic_ ## X, SUFFIX), END)) +#define EXTRA_ARGS +#define ATOMIC_MMU_DECLS uintptr_t retaddr =3D GETPC() + +#define DATA_SIZE 1 +#include "atomic_template.h" + +#define DATA_SIZE 2 +#include "atomic_template.h" + +#define DATA_SIZE 4 +#include "atomic_template.h" + +#ifdef CONFIG_ATOMIC64 +#define DATA_SIZE 8 +#include "atomic_template.h" +#endif diff --git a/target/arm/helper-a64.c b/target/arm/helper-a64.c index ac5c4452d5..26f79f9141 100644 --- a/target/arm/helper-a64.c +++ b/target/arm/helper-a64.c @@ -564,7 +564,7 @@ uint64_t HELPER(paired_cmpxchg64_le_parallel)(CPUARMSta= te *env, uint64_t addr, =20 cmpv =3D int128_make128(env->exclusive_val, env->exclusive_high); newv =3D int128_make128(new_lo, new_hi); - oldv =3D helper_atomic_cmpxchgo_le_mmu(env, addr, cmpv, newv, oi, ra); + oldv =3D cpu_atomic_cmpxchgo_le_mmu(env, addr, cmpv, newv, oi, ra); =20 success =3D int128_eq(oldv, cmpv); return !success; @@ -638,7 +638,7 @@ uint64_t HELPER(paired_cmpxchg64_be_parallel)(CPUARMSta= te *env, uint64_t addr, */ cmpv =3D int128_make128(env->exclusive_high, env->exclusive_val); newv =3D int128_make128(new_hi, new_lo); - oldv =3D helper_atomic_cmpxchgo_be_mmu(env, addr, cmpv, newv, oi, ra); + oldv =3D cpu_atomic_cmpxchgo_be_mmu(env, addr, cmpv, newv, oi, ra); =20 success =3D int128_eq(oldv, cmpv); return !success; @@ -660,7 +660,7 @@ void HELPER(casp_le_parallel)(CPUARMState *env, uint32_= t rs, uint64_t addr, =20 cmpv =3D int128_make128(env->xregs[rs], env->xregs[rs + 1]); newv =3D int128_make128(new_lo, new_hi); - oldv =3D helper_atomic_cmpxchgo_le_mmu(env, addr, cmpv, newv, oi, ra); + oldv =3D cpu_atomic_cmpxchgo_le_mmu(env, addr, cmpv, newv, oi, ra); =20 env->xregs[rs] =3D int128_getlo(oldv); env->xregs[rs + 1] =3D int128_gethi(oldv); @@ -681,7 +681,7 @@ void HELPER(casp_be_parallel)(CPUARMState *env, uint32_= t rs, uint64_t addr, =20 cmpv =3D int128_make128(env->xregs[rs + 1], env->xregs[rs]); newv =3D int128_make128(new_lo, new_hi); - oldv =3D helper_atomic_cmpxchgo_be_mmu(env, addr, cmpv, newv, oi, ra); + oldv =3D cpu_atomic_cmpxchgo_be_mmu(env, addr, cmpv, newv, oi, ra); =20 env->xregs[rs + 1] =3D int128_getlo(oldv); env->xregs[rs] =3D int128_gethi(oldv); diff --git a/target/i386/tcg/mem_helper.c b/target/i386/tcg/mem_helper.c index 591f512bff..2da3cd14b6 100644 --- a/target/i386/tcg/mem_helper.c +++ b/target/i386/tcg/mem_helper.c @@ -64,22 +64,12 @@ void helper_cmpxchg8b(CPUX86State *env, target_ulong a0) cmpv =3D deposit64(env->regs[R_EAX], 32, 32, env->regs[R_EDX]); newv =3D deposit64(env->regs[R_EBX], 32, 32, env->regs[R_ECX]); =20 -#ifdef CONFIG_USER_ONLY - { - uint64_t *haddr =3D g2h(env_cpu(env), a0); - cmpv =3D cpu_to_le64(cmpv); - newv =3D cpu_to_le64(newv); - oldv =3D qatomic_cmpxchg__nocheck(haddr, cmpv, newv); - oldv =3D le64_to_cpu(oldv); - } -#else { uintptr_t ra =3D GETPC(); int mem_idx =3D cpu_mmu_index(env, false); TCGMemOpIdx oi =3D make_memop_idx(MO_TEQ, mem_idx); - oldv =3D helper_atomic_cmpxchgq_le_mmu(env, a0, cmpv, newv, oi, ra= ); + oldv =3D cpu_atomic_cmpxchgq_le_mmu(env, a0, cmpv, newv, oi, ra); } -#endif =20 if (oldv =3D=3D cmpv) { eflags |=3D CC_Z; @@ -147,8 +137,7 @@ void helper_cmpxchg16b(CPUX86State *env, target_ulong a= 0) =20 int mem_idx =3D cpu_mmu_index(env, false); TCGMemOpIdx oi =3D make_memop_idx(MO_TEQ | MO_ALIGN_16, mem_idx); - Int128 oldv =3D helper_atomic_cmpxchgo_le_mmu(env, a0, cmpv, - newv, oi, ra); + Int128 oldv =3D cpu_atomic_cmpxchgo_le_mmu(env, a0, cmpv, newv, oi= , ra); =20 if (int128_eq(oldv, cmpv)) { eflags |=3D CC_Z; diff --git a/target/m68k/op_helper.c b/target/m68k/op_helper.c index ae1ba4b437..d006d1cb3e 100644 --- a/target/m68k/op_helper.c +++ b/target/m68k/op_helper.c @@ -22,6 +22,7 @@ #include "exec/exec-all.h" #include "exec/cpu_ldst.h" #include "semihosting/semihost.h" +#include "tcg/tcg.h" =20 #if defined(CONFIG_USER_ONLY) =20 @@ -782,9 +783,9 @@ static void do_cas2l(CPUM68KState *env, uint32_t regs, = uint32_t a1, uint32_t a2, uint32_t u2 =3D env->dregs[Du2]; uint32_t l1, l2; uintptr_t ra =3D GETPC(); -#if defined(CONFIG_ATOMIC64) && !defined(CONFIG_USER_ONLY) +#if defined(CONFIG_ATOMIC64) int mmu_idx =3D cpu_mmu_index(env, 0); - TCGMemOpIdx oi; + TCGMemOpIdx oi =3D make_memop_idx(MO_BEQ, mmu_idx); #endif =20 if (parallel) { @@ -794,23 +795,13 @@ static void do_cas2l(CPUM68KState *env, uint32_t regs= , uint32_t a1, uint32_t a2, if ((a1 & 7) =3D=3D 0 && a2 =3D=3D a1 + 4) { c =3D deposit64(c2, 32, 32, c1); u =3D deposit64(u2, 32, 32, u1); -#ifdef CONFIG_USER_ONLY - l =3D helper_atomic_cmpxchgq_be(env, a1, c, u); -#else - oi =3D make_memop_idx(MO_BEQ, mmu_idx); - l =3D helper_atomic_cmpxchgq_be_mmu(env, a1, c, u, oi, ra); -#endif + l =3D cpu_atomic_cmpxchgq_be_mmu(env, a1, c, u, oi, ra); l1 =3D l >> 32; l2 =3D l; } else if ((a2 & 7) =3D=3D 0 && a1 =3D=3D a2 + 4) { c =3D deposit64(c1, 32, 32, c2); u =3D deposit64(u1, 32, 32, u2); -#ifdef CONFIG_USER_ONLY - l =3D helper_atomic_cmpxchgq_be(env, a2, c, u); -#else - oi =3D make_memop_idx(MO_BEQ, mmu_idx); - l =3D helper_atomic_cmpxchgq_be_mmu(env, a2, c, u, oi, ra); -#endif + l =3D cpu_atomic_cmpxchgq_be_mmu(env, a2, c, u, oi, ra); l2 =3D l >> 32; l1 =3D l; } else diff --git a/target/ppc/mem_helper.c b/target/ppc/mem_helper.c index 444b2a30ef..e2282baa8d 100644 --- a/target/ppc/mem_helper.c +++ b/target/ppc/mem_helper.c @@ -376,7 +376,7 @@ uint64_t helper_lq_le_parallel(CPUPPCState *env, target= _ulong addr, =20 /* We will have raised EXCP_ATOMIC from the translator. */ assert(HAVE_ATOMIC128); - ret =3D helper_atomic_ldo_le_mmu(env, addr, opidx, GETPC()); + ret =3D cpu_atomic_ldo_le_mmu(env, addr, opidx, GETPC()); env->retxh =3D int128_gethi(ret); return int128_getlo(ret); } @@ -388,7 +388,7 @@ uint64_t helper_lq_be_parallel(CPUPPCState *env, target= _ulong addr, =20 /* We will have raised EXCP_ATOMIC from the translator. */ assert(HAVE_ATOMIC128); - ret =3D helper_atomic_ldo_be_mmu(env, addr, opidx, GETPC()); + ret =3D cpu_atomic_ldo_be_mmu(env, addr, opidx, GETPC()); env->retxh =3D int128_gethi(ret); return int128_getlo(ret); } @@ -401,7 +401,7 @@ void helper_stq_le_parallel(CPUPPCState *env, target_ul= ong addr, /* We will have raised EXCP_ATOMIC from the translator. */ assert(HAVE_ATOMIC128); val =3D int128_make128(lo, hi); - helper_atomic_sto_le_mmu(env, addr, val, opidx, GETPC()); + cpu_atomic_sto_le_mmu(env, addr, val, opidx, GETPC()); } =20 void helper_stq_be_parallel(CPUPPCState *env, target_ulong addr, @@ -412,7 +412,7 @@ void helper_stq_be_parallel(CPUPPCState *env, target_ul= ong addr, /* We will have raised EXCP_ATOMIC from the translator. */ assert(HAVE_ATOMIC128); val =3D int128_make128(lo, hi); - helper_atomic_sto_be_mmu(env, addr, val, opidx, GETPC()); + cpu_atomic_sto_be_mmu(env, addr, val, opidx, GETPC()); } =20 uint32_t helper_stqcx_le_parallel(CPUPPCState *env, target_ulong addr, @@ -429,8 +429,8 @@ uint32_t helper_stqcx_le_parallel(CPUPPCState *env, tar= get_ulong addr, =20 cmpv =3D int128_make128(env->reserve_val2, env->reserve_val); newv =3D int128_make128(new_lo, new_hi); - oldv =3D helper_atomic_cmpxchgo_le_mmu(env, addr, cmpv, newv, - opidx, GETPC()); + oldv =3D cpu_atomic_cmpxchgo_le_mmu(env, addr, cmpv, newv, + opidx, GETPC()); success =3D int128_eq(oldv, cmpv); } env->reserve_addr =3D -1; @@ -451,8 +451,8 @@ uint32_t helper_stqcx_be_parallel(CPUPPCState *env, tar= get_ulong addr, =20 cmpv =3D int128_make128(env->reserve_val2, env->reserve_val); newv =3D int128_make128(new_lo, new_hi); - oldv =3D helper_atomic_cmpxchgo_be_mmu(env, addr, cmpv, newv, - opidx, GETPC()); + oldv =3D cpu_atomic_cmpxchgo_be_mmu(env, addr, cmpv, newv, + opidx, GETPC()); success =3D int128_eq(oldv, cmpv); } env->reserve_addr =3D -1; diff --git a/target/s390x/tcg/mem_helper.c b/target/s390x/tcg/mem_helper.c index 9bae13ecf0..21a4de4067 100644 --- a/target/s390x/tcg/mem_helper.c +++ b/target/s390x/tcg/mem_helper.c @@ -1811,7 +1811,7 @@ void HELPER(cdsg_parallel)(CPUS390XState *env, uint64= _t addr, =20 mem_idx =3D cpu_mmu_index(env, false); oi =3D make_memop_idx(MO_TEQ | MO_ALIGN_16, mem_idx); - oldv =3D helper_atomic_cmpxchgo_be_mmu(env, addr, cmpv, newv, oi, ra); + oldv =3D cpu_atomic_cmpxchgo_be_mmu(env, addr, cmpv, newv, oi, ra); fail =3D !int128_eq(oldv, cmpv); =20 env->cc_op =3D fail; @@ -1884,7 +1884,7 @@ static uint32_t do_csst(CPUS390XState *env, uint32_t = r3, uint64_t a1, ov =3D qatomic_cmpxchg__nocheck(haddr, cv, nv); #else TCGMemOpIdx oi =3D make_memop_idx(MO_TEUL | MO_ALIGN, mem_= idx); - ov =3D helper_atomic_cmpxchgl_be_mmu(env, a1, cv, nv, oi, = ra); + ov =3D cpu_atomic_cmpxchgl_be_mmu(env, a1, cv, nv, oi, ra); #endif } else { ov =3D cpu_ldl_data_ra(env, a1, ra); @@ -1903,13 +1903,8 @@ static uint32_t do_csst(CPUS390XState *env, uint32_t= r3, uint64_t a1, =20 if (parallel) { #ifdef CONFIG_ATOMIC64 -# ifdef CONFIG_USER_ONLY - uint64_t *haddr =3D g2h(env_cpu(env), a1); - ov =3D qatomic_cmpxchg__nocheck(haddr, cv, nv); -# else TCGMemOpIdx oi =3D make_memop_idx(MO_TEQ | MO_ALIGN, mem_i= dx); - ov =3D helper_atomic_cmpxchgq_be_mmu(env, a1, cv, nv, oi, = ra); -# endif + ov =3D cpu_atomic_cmpxchgq_be_mmu(env, a1, cv, nv, oi, ra); #else /* Note that we asserted !parallel above. */ g_assert_not_reached(); @@ -1945,7 +1940,7 @@ static uint32_t do_csst(CPUS390XState *env, uint32_t = r3, uint64_t a1, cpu_stq_data_ra(env, a1 + 8, int128_getlo(nv), ra); } else if (HAVE_CMPXCHG128) { TCGMemOpIdx oi =3D make_memop_idx(MO_TEQ | MO_ALIGN_16, me= m_idx); - ov =3D helper_atomic_cmpxchgo_be_mmu(env, a1, cv, nv, oi, = ra); + ov =3D cpu_atomic_cmpxchgo_be_mmu(env, a1, cv, nv, oi, ra); cc =3D !int128_eq(ov, cv); } else { /* Note that we asserted !parallel above. */ @@ -1985,7 +1980,7 @@ static uint32_t do_csst(CPUS390XState *env, uint32_t = r3, uint64_t a1, } else if (HAVE_ATOMIC128) { TCGMemOpIdx oi =3D make_memop_idx(MO_TEQ | MO_ALIGN_16, me= m_idx); Int128 sv =3D int128_make128(svl, svh); - helper_atomic_sto_be_mmu(env, a2, sv, oi, ra); + cpu_atomic_sto_be_mmu(env, a2, sv, oi, ra); } else { /* Note that we asserted !parallel above. */ g_assert_not_reached(); @@ -2486,7 +2481,7 @@ uint64_t HELPER(lpq_parallel)(CPUS390XState *env, uin= t64_t addr) =20 mem_idx =3D cpu_mmu_index(env, false); oi =3D make_memop_idx(MO_TEQ | MO_ALIGN_16, mem_idx); - v =3D helper_atomic_ldo_be_mmu(env, addr, oi, ra); + v =3D cpu_atomic_ldo_be_mmu(env, addr, oi, ra); hi =3D int128_gethi(v); lo =3D int128_getlo(v); =20 @@ -2518,7 +2513,7 @@ void HELPER(stpq_parallel)(CPUS390XState *env, uint64= _t addr, mem_idx =3D cpu_mmu_index(env, false); oi =3D make_memop_idx(MO_TEQ | MO_ALIGN_16, mem_idx); v =3D int128_make128(low, high); - helper_atomic_sto_be_mmu(env, addr, v, oi, ra); + cpu_atomic_sto_be_mmu(env, addr, v, oi, ra); } =20 /* Execute instruction. This instruction executes an insn modified with --=20 2.25.1