Massive pullreq but almost all of that is RTH's SVE refactoring patchset. The other interesting thing here is the fix for compiling on aarch64 macos. thanks -- PMM The following changes since commit f7a1ea403e0282a7f57edd4298c4f65f24165da5: Merge tag 'misc-pull-request' of gitlab.com:marcandre.lureau/qemu into staging (2022-05-29 16:34:56 -0700) are available in the Git repository at: https://git.linaro.org/people/pmaydell/qemu-arm.git tags/pull-target-arm-20220530 for you to fetch changes up to b1071174d2a2ab371082b7d4b5f19e98edc61ac6: target/arm: Remove aa64_sve check from before disas_sve (2022-05-30 17:05:12 +0100) ---------------------------------------------------------------- target-arm queue: * docs/system/arm: Add FEAT_HCX to list of emulated features * target/arm/hvf: Include missing "cpregs.h" * hw/sd/allwinner-sdhost: report FIFO water level as 1 when data ready * SVE: refactor to use TRANS/TRANS_FEAT macros and push SVE feature check down to individual insn level ---------------------------------------------------------------- Icenowy Zheng (1): hw/sd/allwinner-sdhost: report FIFO water level as 1 when data ready Peter Maydell (1): docs/system/arm: Add FEAT_HCX to list of emulated features Philippe Mathieu-Daudé (1): target/arm/hvf: Include missing "cpregs.h" Richard Henderson (114): target/arm: Introduce TRANS, TRANS_FEAT target/arm: Move null function and sve check into gen_gvec_ool_zz target/arm: Use TRANS_FEAT for gen_gvec_ool_zz target/arm: Move null function and sve check into gen_gvec_ool_zzz target/arm: Introduce gen_gvec_ool_arg_zzz target/arm: Use TRANS_FEAT for gen_gvec_ool_arg_zzz target/arm: Use TRANS_FEAT for do_sve2_zzz_ool target/arm: Move null function and sve check into gen_gvec_ool_zzzz target/arm: Use TRANS_FEAT for gen_gvec_ool_zzzz target/arm: Introduce gen_gvec_ool_arg_zzzz target/arm: Use TRANS_FEAT for do_sve2_zzzz_ool target/arm: Use TRANS_FEAT for gen_gvec_ool_arg_zzzz target/arm: Rename do_zzxz_ool to gen_gvec_ool_arg_zzxz target/arm: Use TRANS_FEAT for gen_gvec_ool_arg_zzxz target/arm: Use TRANS_FEAT for do_sve2_zzz_data target/arm: Use TRANS_FEAT for do_sve2_zzzz_data target/arm: Use TRANS_FEAT for do_sve2_zzw_data target/arm: Use TRANS_FEAT for USDOT_zzzz target/arm: Move null function and sve check into gen_gvec_ool_zzp target/arm: Introduce gen_gvec_ool_arg_zpz target/arm: Use TRANS_FEAT for gen_gvec_ool_arg_zpz target/arm: Use TRANS_FEAT for do_sve2_zpz_data target/arm: Rename do_zpzi_ool to gen_gvec_ool_arg_zpzi target/arm: Use TRANS_FEAT for gen_gvec_ool_arg_zpzi target/arm: Move null function and sve check into gen_gvec_ool_zzzp target/arm: Introduce gen_gvec_ool_arg_zpzz target/arm: Use TRANS_FEAT for gen_gvec_ool_arg_zpzz target/arm: Use TRANS_FEAT for do_sve2_zpzz_ool target/arm: Merge gen_gvec_fn_zz into do_mov_z target/arm: Move null function and sve check into gen_gvec_fn_zzz target/arm: Rename do_zzz_fn to gen_gvec_fn_arg_zzz target/arm: More use of gen_gvec_fn_arg_zzz target/arm: Use TRANS_FEAT for gen_gvec_fn_arg_zzz target/arm: Use TRANS_FEAT for do_sve2_fn_zzz target/arm: Use TRANS_FEAT for RAX1 target/arm: Introduce gen_gvec_fn_arg_zzzz target/arm: Use TRANS_FEAT for do_sve2_zzzz_fn target/arm: Introduce gen_gvec_fn_zzi target/arm: Use TRANS_FEAT for do_zz_dbm target/arm: Hoist sve access check through do_sel_z target/arm: Introduce gen_gvec_fn_arg_zzi target/arm: Use TRANS_FEAT for do_sve2_fn2i target/arm: Use TRANS_FEAT for do_vpz_ool target/arm: Use TRANS_FEAT for do_shift_imm target/arm: Introduce do_shift_zpzi target/arm: Use TRANS_FEAT for do_shift_zpzi target/arm: Use TRANS_FEAT for do_zpzzz_ool target/arm: Move sve check into do_index target/arm: Use TRANS_FEAT for do_index target/arm: Use TRANS_FEAT for do_adr target/arm: Use TRANS_FEAT for do_predset target/arm: Use TRANS_FEAT for RDFFR, WRFFR target/arm: Use TRANS_FEAT for do_pfirst_pnext target/arm: Use TRANS_FEAT for do_EXT target/arm: Use TRANS_FEAT for do_perm_pred3 target/arm: Use TRANS_FEAT for do_perm_pred2 target/arm: Move sve zip high_ofs into simd_data target/arm: Use gen_gvec_ool_arg_zzz for do_zip, do_zip_q target/arm: Use TRANS_FEAT for do_zip, do_zip_q target/arm: Use TRANS_FEAT for do_clast_vector target/arm: Use TRANS_FEAT for do_clast_fp target/arm: Use TRANS_FEAT for do_clast_general target/arm: Use TRANS_FEAT for do_last_fp target/arm: Use TRANS_FEAT for do_last_general target/arm: Use TRANS_FEAT for SPLICE target/arm: Use TRANS_FEAT for do_ppzz_flags target/arm: Use TRANS_FEAT for do_sve2_ppzz_flags target/arm: Use TRANS_FEAT for do_ppzi_flags target/arm: Use TRANS_FEAT for do_brk2, do_brk3 target/arm: Use TRANS_FEAT for MUL_zzi target/arm: Reject dup_i w/ shifted byte early target/arm: Reject add/sub w/ shifted byte early target/arm: Reject copy w/ shifted byte early target/arm: Use TRANS_FEAT for ADD_zzi target/arm: Use TRANS_FEAT for do_zzi_sat target/arm: Use TRANS_FEAT for do_zzi_ool target/arm: Introduce gen_gvec_{ptr,fpst}_zzzz target/arm: Use TRANS_FEAT for FMMLA target/arm: Move sve check into gen_gvec_fn_ppp target/arm: Implement NOT (prediates) alias target/arm: Use TRANS_FEAT for SEL_zpzz target/arm: Use TRANS_FEAT for MOVPRFX target/arm: Use TRANS_FEAT for FMLA target/arm: Use TRANS_FEAT for BFMLA target/arm: Rename do_zzz_fp to gen_gvec_ool_fpst_arg_zzz target/arm: Use TRANS_FEAT for DO_FP3 target/arm: Use TRANS_FEAT for FMUL_zzx target/arm: Use TRANS_FEAT for FTMAD target/arm: Move null function and sve check into do_reduce target/arm: Use TRANS_FEAT for do_reduce target/arm: Use TRANS_FEAT for FRECPE, FRSQRTE target/arm: Expand frint_fns for MO_8 target/arm: Rename do_zpz_ptr to gen_gvec_ool_fpst_arg_zpz target/arm: Move null function and sve check into do_frint_mode target/arm: Use TRANS_FEAT for do_frint_mode target/arm: Use TRANS_FEAT for FLOGB target/arm: Use TRANS_FEAT for do_ppz_fp target/arm: Rename do_zpzz_ptr to gen_gvec_fpst_arg_zpzz target/arm: Use TRANS_FEAT for gen_gvec_fpst_arg_zpzz target/arm: Use TRANS_FEAT for FCADD target/arm: Introduce gen_gvec_fpst_zzzzp target/arm: Use TRANS_FEAT for gen_gvec_fpst_zzzzp target/arm: Move null function and sve check into do_fp_imm target/arm: Use TRANS_FEAT for DO_FP_IMM target/arm: Use TRANS_FEAT for DO_FPCMP target/arm: Remove assert in trans_FCMLA_zzxz target/arm: Use TRANS_FEAT for FCMLA_zzxz target/arm: Use TRANS_FEAT for do_narrow_extract target/arm: Use TRANS_FEAT for do_shll_tb target/arm: Use TRANS_FEAT for do_shr_narrow target/arm: Use TRANS_FEAT for do_FMLAL_zzzw target/arm: Use TRANS_FEAT for do_FMLAL_zzxw target/arm: Add sve feature check for remaining trans_* functions target/arm: Remove aa64_sve check from before disas_sve docs/system/arm/emulation.rst | 1 + target/arm/translate.h | 11 + target/arm/sve.decode | 57 +- hw/sd/allwinner-sdhost.c | 7 + target/arm/hvf/hvf.c | 1 + target/arm/sve_helper.c | 6 +- target/arm/translate-a64.c | 2 +- target/arm/translate-sve.c | 5367 +++++++++++++++-------------------------- 8 files changed, 2067 insertions(+), 3385 deletions(-) In commit 5814d587fe861fe9 we added support for emulating FEAT_HCX (Support for the HCRX_EL2 register). However we forgot to add it to the list in emulated.rst. Correct the omission. Fixes: 5814d587fe861fe9 ("target/arm: Enable FEAT_HCX for -cpu max") Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220520084320.424166-1-peter.maydell@linaro.org --- docs/system/arm/emulation.rst | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/system/arm/emulation.rst b/docs/system/arm/emulation.rst index XXXXXXX..XXXXXXX 100644 --- a/docs/system/arm/emulation.rst +++ b/docs/system/arm/emulation.rst @@ -XXX,XX +XXX,XX @@ the following architecture extensions: - FEAT_FRINTTS (Floating-point to integer instructions) - FEAT_FlagM (Flag manipulation instructions v2) - FEAT_FlagM2 (Enhancements to flag manipulation instructions) +- FEAT_HCX (Support for the HCRX_EL2 register) - FEAT_HPDS (Hierarchical permission disables) - FEAT_I8MM (AArch64 Int8 matrix multiplication instructions) - FEAT_IDST (ID space trap handling) -- 2.25.1 From: Philippe Mathieu-Daudé <f4bug@amsat.org> Fix when building HVF on macOS Aarch64: target/arm/hvf/hvf.c:586:15: error: unknown type name 'ARMCPRegInfo'; did you mean 'ARMCPUInfo'? const ARMCPRegInfo *ri; ^~~~~~~~~~~~ ARMCPUInfo target/arm/cpu-qom.h:38:3: note: 'ARMCPUInfo' declared here } ARMCPUInfo; ^ target/arm/hvf/hvf.c:589:14: error: implicit declaration of function 'get_arm_cp_reginfo' is invalid in C99 [-Werror,-Wimplicit-function-declaration] ri = get_arm_cp_reginfo(arm_cpu->cp_regs, key); ^ target/arm/hvf/hvf.c:589:12: warning: incompatible integer to pointer conversion assigning to 'const ARMCPUInfo *' (aka 'const struct ARMCPUInfo *') from 'int' [-Wint-conversion] ri = get_arm_cp_reginfo(arm_cpu->cp_regs, key); ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ target/arm/hvf/hvf.c:591:26: error: no member named 'type' in 'struct ARMCPUInfo' assert(!(ri->type & ARM_CP_NO_RAW)); ~~ ^ /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/assert.h:99:25: note: expanded from macro 'assert' (__builtin_expect(!(e), 0) ? __assert_rtn(__func__, __ASSERT_FILE_NAME, __LINE__, #e) : (void)0) ^ target/arm/hvf/hvf.c:591:33: error: use of undeclared identifier 'ARM_CP_NO_RAW' assert(!(ri->type & ARM_CP_NO_RAW)); ^ 1 warning and 4 errors generated. Fixes: cf7c6d1004 ("target/arm: Split out cpregs.h") Reported-by: Duncan Bayne <duncan@bayne.id.au> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220525161926.34233-1-philmd@fungible.com Resolves: https://gitlab.com/qemu-project/qemu/-/issues/1029 Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/hvf/hvf.c | 1 + 1 file changed, 1 insertion(+) diff --git a/target/arm/hvf/hvf.c b/target/arm/hvf/hvf.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/hvf/hvf.c +++ b/target/arm/hvf/hvf.c @@ -XXX,XX +XXX,XX @@ #include "sysemu/hvf_int.h" #include "sysemu/hw_accel.h" #include "hvf_arm.h" +#include "cpregs.h" #include <mach/mach_time.h> -- 2.25.1 From: Icenowy Zheng <uwu@icenowy.me> U-Boot queries the FIFO water level to reduce checking status register when doing PIO SD card operation. Report a FIFO water level of 1 when data is ready, to prevent the code from trying to read 0 words from the FIFO each time. Signed-off-by: Icenowy Zheng <uwu@icenowy.me> Message-id: 20220520124200.2112699-1-uwu@icenowy.me Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- hw/sd/allwinner-sdhost.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/hw/sd/allwinner-sdhost.c b/hw/sd/allwinner-sdhost.c index XXXXXXX..XXXXXXX 100644 --- a/hw/sd/allwinner-sdhost.c +++ b/hw/sd/allwinner-sdhost.c @@ -XXX,XX +XXX,XX @@ enum { }; enum { + SD_STAR_FIFO_EMPTY = (1 << 2), SD_STAR_CARD_PRESENT = (1 << 8), + SD_STAR_FIFO_LEVEL_1 = (1 << 17), }; enum { @@ -XXX,XX +XXX,XX @@ static uint64_t allwinner_sdhost_read(void *opaque, hwaddr offset, break; case REG_SD_STAR: /* Status */ res = s->status; + if (sdbus_data_ready(&s->sdbus)) { + res |= SD_STAR_FIFO_LEVEL_1; + } else { + res |= SD_STAR_FIFO_EMPTY; + } break; case REG_SD_FWLR: /* FIFO Water Level */ res = s->fifo_wlevel; -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Steal the idea for these leaf function expanders from PowerPC. Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-2-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate.h | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/target/arm/translate.h b/target/arm/translate.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate.h +++ b/target/arm/translate.h @@ -XXX,XX +XXX,XX @@ static inline MemOp finalize_memop(DisasContext *s, MemOp opc) */ uint64_t asimd_imm_const(uint32_t imm, int cmode, int op); +/* + * Helpers for implementing sets of trans_* functions. + * Defer the implementation of NAME to FUNC, with optional extra arguments. + */ +#define TRANS(NAME, FUNC, ...) \ + static bool trans_##NAME(DisasContext *s, arg_##NAME *a) \ + { return FUNC(s, __VA_ARGS__); } +#define TRANS_FEAT(NAME, FEAT, FUNC, ...) \ + static bool trans_##NAME(DisasContext *s, arg_##NAME *a) \ + { return dc_isar_feature(FEAT, s) && FUNC(s, __VA_ARGS__); } + #endif /* TARGET_ARM_TRANSLATE_H */ -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-3-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 36 +++++++++++++++--------------------- 1 file changed, 15 insertions(+), 21 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static int pred_gvec_reg_size(DisasContext *s) } /* Invoke an out-of-line helper on 2 Zregs. */ -static void gen_gvec_ool_zz(DisasContext *s, gen_helper_gvec_2 *fn, +static bool gen_gvec_ool_zz(DisasContext *s, gen_helper_gvec_2 *fn, int rd, int rn, int data) { - unsigned vsz = vec_full_reg_size(s); - tcg_gen_gvec_2_ool(vec_full_reg_offset(s, rd), - vec_full_reg_offset(s, rn), - vsz, vsz, data, fn); + if (fn == NULL) { + return false; + } + if (sve_access_check(s)) { + unsigned vsz = vec_full_reg_size(s); + tcg_gen_gvec_2_ool(vec_full_reg_offset(s, rd), + vec_full_reg_offset(s, rn), + vsz, vsz, data, fn); + } + return true; } /* Invoke an out-of-line helper on 3 Zregs. */ @@ -XXX,XX +XXX,XX @@ static bool trans_FEXPA(DisasContext *s, arg_rr_esz *a) gen_helper_sve_fexpa_s, gen_helper_sve_fexpa_d, }; - if (a->esz == 0) { - return false; - } - if (sve_access_check(s)) { - gen_gvec_ool_zz(s, fns[a->esz], a->rd, a->rn, 0); - } - return true; + return gen_gvec_ool_zz(s, fns[a->esz], a->rd, a->rn, 0); } static bool trans_FTSSEL(DisasContext *s, arg_rrr_esz *a) @@ -XXX,XX +XXX,XX @@ static bool trans_REV_v(DisasContext *s, arg_rr_esz *a) gen_helper_sve_rev_b, gen_helper_sve_rev_h, gen_helper_sve_rev_s, gen_helper_sve_rev_d }; - - if (sve_access_check(s)) { - gen_gvec_ool_zz(s, fns[a->esz], a->rd, a->rn, 0); - } - return true; + return gen_gvec_ool_zz(s, fns[a->esz], a->rd, a->rn, 0); } static bool trans_TBL(DisasContext *s, arg_rrr_esz *a) @@ -XXX,XX +XXX,XX @@ static bool trans_AESMC(DisasContext *s, arg_AESMC *a) if (!dc_isar_feature(aa64_sve2_aes, s)) { return false; } - if (sve_access_check(s)) { - gen_gvec_ool_zz(s, gen_helper_crypto_aesmc, a->rd, a->rd, a->decrypt); - } - return true; + return gen_gvec_ool_zz(s, gen_helper_crypto_aesmc, + a->rd, a->rd, a->decrypt); } static bool do_aese(DisasContext *s, arg_rrr_esz *a, bool decrypt) -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Convert SVE translation functions using gen_gvec_ool_zz to TRANS_FEAT. Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-4-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 39 +++++++++++++------------------------- 1 file changed, 13 insertions(+), 26 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool trans_ADR_u32(DisasContext *s, arg_rrri *a) *** SVE Integer Misc - Unpredicated Group */ -static bool trans_FEXPA(DisasContext *s, arg_rr_esz *a) -{ - static gen_helper_gvec_2 * const fns[4] = { - NULL, - gen_helper_sve_fexpa_h, - gen_helper_sve_fexpa_s, - gen_helper_sve_fexpa_d, - }; - return gen_gvec_ool_zz(s, fns[a->esz], a->rd, a->rn, 0); -} +static gen_helper_gvec_2 * const fexpa_fns[4] = { + NULL, gen_helper_sve_fexpa_h, + gen_helper_sve_fexpa_s, gen_helper_sve_fexpa_d, +}; +TRANS_FEAT(FEXPA, aa64_sve, gen_gvec_ool_zz, + fexpa_fns[a->esz], a->rd, a->rn, 0) static bool trans_FTSSEL(DisasContext *s, arg_rrr_esz *a) { @@ -XXX,XX +XXX,XX @@ static bool trans_INSR_r(DisasContext *s, arg_rrr_esz *a) return true; } -static bool trans_REV_v(DisasContext *s, arg_rr_esz *a) -{ - static gen_helper_gvec_2 * const fns[4] = { - gen_helper_sve_rev_b, gen_helper_sve_rev_h, - gen_helper_sve_rev_s, gen_helper_sve_rev_d - }; - return gen_gvec_ool_zz(s, fns[a->esz], a->rd, a->rn, 0); -} +static gen_helper_gvec_2 * const rev_fns[4] = { + gen_helper_sve_rev_b, gen_helper_sve_rev_h, + gen_helper_sve_rev_s, gen_helper_sve_rev_d +}; +TRANS_FEAT(REV_v, aa64_sve, gen_gvec_ool_zz, rev_fns[a->esz], a->rd, a->rn, 0) static bool trans_TBL(DisasContext *s, arg_rrr_esz *a) { @@ -XXX,XX +XXX,XX @@ static bool trans_USDOT_zzzz(DisasContext *s, arg_USDOT_zzzz *a) return true; } -static bool trans_AESMC(DisasContext *s, arg_AESMC *a) -{ - if (!dc_isar_feature(aa64_sve2_aes, s)) { - return false; - } - return gen_gvec_ool_zz(s, gen_helper_crypto_aesmc, - a->rd, a->rd, a->decrypt); -} +TRANS_FEAT(AESMC, aa64_sve2_aes, gen_gvec_ool_zz, + gen_helper_crypto_aesmc, a->rd, a->rd, a->decrypt) static bool do_aese(DisasContext *s, arg_rrr_esz *a, bool decrypt) { -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-5-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 74 ++++++++++++-------------------------- 1 file changed, 23 insertions(+), 51 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool gen_gvec_ool_zz(DisasContext *s, gen_helper_gvec_2 *fn, } /* Invoke an out-of-line helper on 3 Zregs. */ -static void gen_gvec_ool_zzz(DisasContext *s, gen_helper_gvec_3 *fn, +static bool gen_gvec_ool_zzz(DisasContext *s, gen_helper_gvec_3 *fn, int rd, int rn, int rm, int data) { - unsigned vsz = vec_full_reg_size(s); - tcg_gen_gvec_3_ool(vec_full_reg_offset(s, rd), - vec_full_reg_offset(s, rn), - vec_full_reg_offset(s, rm), - vsz, vsz, data, fn); + if (fn == NULL) { + return false; + } + if (sve_access_check(s)) { + unsigned vsz = vec_full_reg_size(s); + tcg_gen_gvec_3_ool(vec_full_reg_offset(s, rd), + vec_full_reg_offset(s, rn), + vec_full_reg_offset(s, rm), + vsz, vsz, data, fn); + } + return true; } /* Invoke an out-of-line helper on 4 Zregs. */ @@ -XXX,XX +XXX,XX @@ static bool trans_LSL_zzi(DisasContext *s, arg_rri_esz *a) static bool do_zzw_ool(DisasContext *s, arg_rrr_esz *a, gen_helper_gvec_3 *fn) { - if (fn == NULL) { - return false; - } - if (sve_access_check(s)) { - gen_gvec_ool_zzz(s, fn, a->rd, a->rn, a->rm, 0); - } - return true; + return gen_gvec_ool_zzz(s, fn, a->rd, a->rn, a->rm, 0); } #define DO_ZZW(NAME, name) \ @@ -XXX,XX +XXX,XX @@ static bool trans_RDVL(DisasContext *s, arg_RDVL *a) static bool do_adr(DisasContext *s, arg_rrri *a, gen_helper_gvec_3 *fn) { - if (sve_access_check(s)) { - gen_gvec_ool_zzz(s, fn, a->rd, a->rn, a->rm, a->imm); - } - return true; + return gen_gvec_ool_zzz(s, fn, a->rd, a->rn, a->rm, a->imm); } static bool trans_ADR_p32(DisasContext *s, arg_rrri *a) @@ -XXX,XX +XXX,XX @@ static bool trans_FTSSEL(DisasContext *s, arg_rrr_esz *a) gen_helper_sve_ftssel_s, gen_helper_sve_ftssel_d, }; - if (a->esz == 0) { - return false; - } - if (sve_access_check(s)) { - gen_gvec_ool_zzz(s, fns[a->esz], a->rd, a->rn, a->rm, 0); - } - return true; + return gen_gvec_ool_zzz(s, fns[a->esz], a->rd, a->rn, a->rm, 0); } /* @@ -XXX,XX +XXX,XX @@ static bool trans_TBL(DisasContext *s, arg_rrr_esz *a) gen_helper_sve_tbl_b, gen_helper_sve_tbl_h, gen_helper_sve_tbl_s, gen_helper_sve_tbl_d }; - - if (sve_access_check(s)) { - gen_gvec_ool_zzz(s, fns[a->esz], a->rd, a->rn, a->rm, 0); - } - return true; + return gen_gvec_ool_zzz(s, fns[a->esz], a->rd, a->rn, a->rm, 0); } static bool trans_TBL_sve2(DisasContext *s, arg_rrr_esz *a) @@ -XXX,XX +XXX,XX @@ static bool trans_TBX(DisasContext *s, arg_rrr_esz *a) if (!dc_isar_feature(aa64_sve2, s)) { return false; } - if (sve_access_check(s)) { - gen_gvec_ool_zzz(s, fns[a->esz], a->rd, a->rn, a->rm, 0); - } - return true; + return gen_gvec_ool_zzz(s, fns[a->esz], a->rd, a->rn, a->rm, 0); } static bool trans_UNPK(DisasContext *s, arg_UNPK *a) @@ -XXX,XX +XXX,XX @@ static bool do_zip(DisasContext *s, arg_rrr_esz *a, bool high) static bool do_zzz_data_ool(DisasContext *s, arg_rrr_esz *a, int data, gen_helper_gvec_3 *fn) { - if (sve_access_check(s)) { - gen_gvec_ool_zzz(s, fn, a->rd, a->rn, a->rm, data); - } - return true; + return gen_gvec_ool_zzz(s, fn, a->rd, a->rn, a->rm, data); } static bool trans_ZIP1_z(DisasContext *s, arg_rrr_esz *a) @@ -XXX,XX +XXX,XX @@ static bool trans_MUL_zzz(DisasContext *s, arg_rrr_esz *a) static bool do_sve2_zzz_ool(DisasContext *s, arg_rrr_esz *a, gen_helper_gvec_3 *fn) { - if (fn == NULL || !dc_isar_feature(aa64_sve2, s)) { + if (!dc_isar_feature(aa64_sve2, s)) { return false; } - if (sve_access_check(s)) { - gen_gvec_ool_zzz(s, fn, a->rd, a->rn, a->rm, 0); - } - return true; + return gen_gvec_ool_zzz(s, fn, a->rd, a->rn, a->rm, 0); } static bool trans_SMULH_zzz(DisasContext *s, arg_rrr_esz *a) @@ -XXX,XX +XXX,XX @@ static bool do_aese(DisasContext *s, arg_rrr_esz *a, bool decrypt) if (!dc_isar_feature(aa64_sve2_aes, s)) { return false; } - if (sve_access_check(s)) { - gen_gvec_ool_zzz(s, gen_helper_crypto_aese, - a->rd, a->rn, a->rm, decrypt); - } - return true; + return gen_gvec_ool_zzz(s, gen_helper_crypto_aese, + a->rd, a->rn, a->rm, decrypt); } static bool trans_AESE(DisasContext *s, arg_rrr_esz *a) @@ -XXX,XX +XXX,XX @@ static bool do_sm4(DisasContext *s, arg_rrr_esz *a, gen_helper_gvec_3 *fn) if (!dc_isar_feature(aa64_sve2_sm4, s)) { return false; } - if (sve_access_check(s)) { - gen_gvec_ool_zzz(s, fn, a->rd, a->rn, a->rm, 0); - } - return true; + return gen_gvec_ool_zzz(s, fn, a->rd, a->rn, a->rm, 0); } static bool trans_SM4E(DisasContext *s, arg_rrr_esz *a) -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Use gen_gvec_ool_arg_zzz instead of gen_gvec_ool_zzz when the arguments come from arg_rrr_esz. Replaces do_zzw_ool and do_zzz_data_ool. Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-6-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 48 +++++++++++++++++--------------------- 1 file changed, 21 insertions(+), 27 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool gen_gvec_ool_zzz(DisasContext *s, gen_helper_gvec_3 *fn, return true; } +static bool gen_gvec_ool_arg_zzz(DisasContext *s, gen_helper_gvec_3 *fn, + arg_rrr_esz *a, int data) +{ + return gen_gvec_ool_zzz(s, fn, a->rd, a->rn, a->rm, data); +} + /* Invoke an out-of-line helper on 4 Zregs. */ static void gen_gvec_ool_zzzz(DisasContext *s, gen_helper_gvec_4 *fn, int rd, int rn, int rm, int ra, int data) @@ -XXX,XX +XXX,XX @@ static bool trans_LSL_zzi(DisasContext *s, arg_rri_esz *a) return do_shift_imm(s, a, false, tcg_gen_gvec_shli); } -static bool do_zzw_ool(DisasContext *s, arg_rrr_esz *a, gen_helper_gvec_3 *fn) -{ - return gen_gvec_ool_zzz(s, fn, a->rd, a->rn, a->rm, 0); -} - #define DO_ZZW(NAME, name) \ static bool trans_##NAME##_zzw(DisasContext *s, arg_rrr_esz *a) \ { \ @@ -XXX,XX +XXX,XX @@ static bool trans_##NAME##_zzw(DisasContext *s, arg_rrr_esz *a) \ gen_helper_sve_##name##_zzw_b, gen_helper_sve_##name##_zzw_h, \ gen_helper_sve_##name##_zzw_s, NULL \ }; \ - return do_zzw_ool(s, a, fns[a->esz]); \ + return gen_gvec_ool_arg_zzz(s, fns[a->esz], a, 0); \ } DO_ZZW(ASR, asr) @@ -XXX,XX +XXX,XX @@ static bool trans_FTSSEL(DisasContext *s, arg_rrr_esz *a) gen_helper_sve_ftssel_s, gen_helper_sve_ftssel_d, }; - return gen_gvec_ool_zzz(s, fns[a->esz], a->rd, a->rn, a->rm, 0); + return gen_gvec_ool_arg_zzz(s, fns[a->esz], a, 0); } /* @@ -XXX,XX +XXX,XX @@ static bool trans_TBL(DisasContext *s, arg_rrr_esz *a) gen_helper_sve_tbl_b, gen_helper_sve_tbl_h, gen_helper_sve_tbl_s, gen_helper_sve_tbl_d }; - return gen_gvec_ool_zzz(s, fns[a->esz], a->rd, a->rn, a->rm, 0); + return gen_gvec_ool_arg_zzz(s, fns[a->esz], a, 0); } static bool trans_TBL_sve2(DisasContext *s, arg_rrr_esz *a) @@ -XXX,XX +XXX,XX @@ static bool trans_TBX(DisasContext *s, arg_rrr_esz *a) if (!dc_isar_feature(aa64_sve2, s)) { return false; } - return gen_gvec_ool_zzz(s, fns[a->esz], a->rd, a->rn, a->rm, 0); + return gen_gvec_ool_arg_zzz(s, fns[a->esz], a, 0); } static bool trans_UNPK(DisasContext *s, arg_UNPK *a) @@ -XXX,XX +XXX,XX @@ static bool do_zip(DisasContext *s, arg_rrr_esz *a, bool high) return true; } -static bool do_zzz_data_ool(DisasContext *s, arg_rrr_esz *a, int data, - gen_helper_gvec_3 *fn) -{ - return gen_gvec_ool_zzz(s, fn, a->rd, a->rn, a->rm, data); -} - static bool trans_ZIP1_z(DisasContext *s, arg_rrr_esz *a) { return do_zip(s, a, false); @@ -XXX,XX +XXX,XX @@ static gen_helper_gvec_3 * const uzp_fns[4] = { static bool trans_UZP1_z(DisasContext *s, arg_rrr_esz *a) { - return do_zzz_data_ool(s, a, 0, uzp_fns[a->esz]); + return gen_gvec_ool_arg_zzz(s, uzp_fns[a->esz], a, 0); } static bool trans_UZP2_z(DisasContext *s, arg_rrr_esz *a) { - return do_zzz_data_ool(s, a, 1 << a->esz, uzp_fns[a->esz]); + return gen_gvec_ool_arg_zzz(s, uzp_fns[a->esz], a, 1 << a->esz); } static bool trans_UZP1_q(DisasContext *s, arg_rrr_esz *a) @@ -XXX,XX +XXX,XX @@ static bool trans_UZP1_q(DisasContext *s, arg_rrr_esz *a) if (!dc_isar_feature(aa64_sve_f64mm, s)) { return false; } - return do_zzz_data_ool(s, a, 0, gen_helper_sve2_uzp_q); + return gen_gvec_ool_arg_zzz(s, gen_helper_sve2_uzp_q, a, 0); } static bool trans_UZP2_q(DisasContext *s, arg_rrr_esz *a) @@ -XXX,XX +XXX,XX @@ static bool trans_UZP2_q(DisasContext *s, arg_rrr_esz *a) if (!dc_isar_feature(aa64_sve_f64mm, s)) { return false; } - return do_zzz_data_ool(s, a, 16, gen_helper_sve2_uzp_q); + return gen_gvec_ool_arg_zzz(s, gen_helper_sve2_uzp_q, a, 16); } static gen_helper_gvec_3 * const trn_fns[4] = { @@ -XXX,XX +XXX,XX @@ static gen_helper_gvec_3 * const trn_fns[4] = { static bool trans_TRN1_z(DisasContext *s, arg_rrr_esz *a) { - return do_zzz_data_ool(s, a, 0, trn_fns[a->esz]); + return gen_gvec_ool_arg_zzz(s, trn_fns[a->esz], a, 0); } static bool trans_TRN2_z(DisasContext *s, arg_rrr_esz *a) { - return do_zzz_data_ool(s, a, 1 << a->esz, trn_fns[a->esz]); + return gen_gvec_ool_arg_zzz(s, trn_fns[a->esz], a, 1 << a->esz); } static bool trans_TRN1_q(DisasContext *s, arg_rrr_esz *a) @@ -XXX,XX +XXX,XX @@ static bool trans_TRN1_q(DisasContext *s, arg_rrr_esz *a) if (!dc_isar_feature(aa64_sve_f64mm, s)) { return false; } - return do_zzz_data_ool(s, a, 0, gen_helper_sve2_trn_q); + return gen_gvec_ool_arg_zzz(s, gen_helper_sve2_trn_q, a, 0); } static bool trans_TRN2_q(DisasContext *s, arg_rrr_esz *a) @@ -XXX,XX +XXX,XX @@ static bool trans_TRN2_q(DisasContext *s, arg_rrr_esz *a) if (!dc_isar_feature(aa64_sve_f64mm, s)) { return false; } - return do_zzz_data_ool(s, a, 16, gen_helper_sve2_trn_q); + return gen_gvec_ool_arg_zzz(s, gen_helper_sve2_trn_q, a, 16); } /* @@ -XXX,XX +XXX,XX @@ static bool do_sve2_zzz_ool(DisasContext *s, arg_rrr_esz *a, if (!dc_isar_feature(aa64_sve2, s)) { return false; } - return gen_gvec_ool_zzz(s, fn, a->rd, a->rn, a->rm, 0); + return gen_gvec_ool_arg_zzz(s, fn, a, 0); } static bool trans_SMULH_zzz(DisasContext *s, arg_rrr_esz *a) @@ -XXX,XX +XXX,XX @@ static bool do_aese(DisasContext *s, arg_rrr_esz *a, bool decrypt) if (!dc_isar_feature(aa64_sve2_aes, s)) { return false; } - return gen_gvec_ool_zzz(s, gen_helper_crypto_aese, - a->rd, a->rn, a->rm, decrypt); + return gen_gvec_ool_arg_zzz(s, gen_helper_crypto_aese, a, decrypt); } static bool trans_AESE(DisasContext *s, arg_rrr_esz *a) @@ -XXX,XX +XXX,XX @@ static bool do_sm4(DisasContext *s, arg_rrr_esz *a, gen_helper_gvec_3 *fn) if (!dc_isar_feature(aa64_sve2_sm4, s)) { return false; } - return gen_gvec_ool_zzz(s, fn, a->rd, a->rn, a->rm, 0); + return gen_gvec_ool_arg_zzz(s, fn, a, 0); } static bool trans_SM4E(DisasContext *s, arg_rrr_esz *a) -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Convert SVE translation functions using gen_gvec_ool_arg_zzz to TRANS_FEAT. Remove trivial wrappers do_aese, do_sm4. Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-7-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 165 ++++++++++--------------------------- 1 file changed, 45 insertions(+), 120 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool trans_LSL_zzi(DisasContext *s, arg_rri_esz *a) } #define DO_ZZW(NAME, name) \ -static bool trans_##NAME##_zzw(DisasContext *s, arg_rrr_esz *a) \ -{ \ - static gen_helper_gvec_3 * const fns[4] = { \ + static gen_helper_gvec_3 * const name##_zzw_fns[4] = { \ gen_helper_sve_##name##_zzw_b, gen_helper_sve_##name##_zzw_h, \ gen_helper_sve_##name##_zzw_s, NULL \ }; \ - return gen_gvec_ool_arg_zzz(s, fns[a->esz], a, 0); \ -} + TRANS_FEAT(NAME, aa64_sve, gen_gvec_ool_arg_zzz, \ + name##_zzw_fns[a->esz], a, 0) -DO_ZZW(ASR, asr) -DO_ZZW(LSR, lsr) -DO_ZZW(LSL, lsl) +DO_ZZW(ASR_zzw, asr) +DO_ZZW(LSR_zzw, lsr) +DO_ZZW(LSL_zzw, lsl) #undef DO_ZZW @@ -XXX,XX +XXX,XX @@ static gen_helper_gvec_2 * const fexpa_fns[4] = { TRANS_FEAT(FEXPA, aa64_sve, gen_gvec_ool_zz, fexpa_fns[a->esz], a->rd, a->rn, 0) -static bool trans_FTSSEL(DisasContext *s, arg_rrr_esz *a) -{ - static gen_helper_gvec_3 * const fns[4] = { - NULL, - gen_helper_sve_ftssel_h, - gen_helper_sve_ftssel_s, - gen_helper_sve_ftssel_d, - }; - return gen_gvec_ool_arg_zzz(s, fns[a->esz], a, 0); -} +static gen_helper_gvec_3 * const ftssel_fns[4] = { + NULL, gen_helper_sve_ftssel_h, + gen_helper_sve_ftssel_s, gen_helper_sve_ftssel_d, +}; +TRANS_FEAT(FTSSEL, aa64_sve, gen_gvec_ool_arg_zzz, ftssel_fns[a->esz], a, 0) /* *** SVE Predicate Logical Operations Group @@ -XXX,XX +XXX,XX @@ static gen_helper_gvec_2 * const rev_fns[4] = { }; TRANS_FEAT(REV_v, aa64_sve, gen_gvec_ool_zz, rev_fns[a->esz], a->rd, a->rn, 0) -static bool trans_TBL(DisasContext *s, arg_rrr_esz *a) -{ - static gen_helper_gvec_3 * const fns[4] = { - gen_helper_sve_tbl_b, gen_helper_sve_tbl_h, - gen_helper_sve_tbl_s, gen_helper_sve_tbl_d - }; - return gen_gvec_ool_arg_zzz(s, fns[a->esz], a, 0); -} +static gen_helper_gvec_3 * const sve_tbl_fns[4] = { + gen_helper_sve_tbl_b, gen_helper_sve_tbl_h, + gen_helper_sve_tbl_s, gen_helper_sve_tbl_d +}; +TRANS_FEAT(TBL, aa64_sve, gen_gvec_ool_arg_zzz, sve_tbl_fns[a->esz], a, 0) static bool trans_TBL_sve2(DisasContext *s, arg_rrr_esz *a) { @@ -XXX,XX +XXX,XX @@ static bool trans_TBL_sve2(DisasContext *s, arg_rrr_esz *a) return true; } -static bool trans_TBX(DisasContext *s, arg_rrr_esz *a) -{ - static gen_helper_gvec_3 * const fns[4] = { - gen_helper_sve2_tbx_b, gen_helper_sve2_tbx_h, - gen_helper_sve2_tbx_s, gen_helper_sve2_tbx_d - }; - - if (!dc_isar_feature(aa64_sve2, s)) { - return false; - } - return gen_gvec_ool_arg_zzz(s, fns[a->esz], a, 0); -} +static gen_helper_gvec_3 * const tbx_fns[4] = { + gen_helper_sve2_tbx_b, gen_helper_sve2_tbx_h, + gen_helper_sve2_tbx_s, gen_helper_sve2_tbx_d +}; +TRANS_FEAT(TBX, aa64_sve2, gen_gvec_ool_arg_zzz, tbx_fns[a->esz], a, 0) static bool trans_UNPK(DisasContext *s, arg_UNPK *a) { @@ -XXX,XX +XXX,XX @@ static gen_helper_gvec_3 * const uzp_fns[4] = { gen_helper_sve_uzp_s, gen_helper_sve_uzp_d, }; -static bool trans_UZP1_z(DisasContext *s, arg_rrr_esz *a) -{ - return gen_gvec_ool_arg_zzz(s, uzp_fns[a->esz], a, 0); -} +TRANS_FEAT(UZP1_z, aa64_sve, gen_gvec_ool_arg_zzz, + uzp_fns[a->esz], a, 0) +TRANS_FEAT(UZP2_z, aa64_sve, gen_gvec_ool_arg_zzz, + uzp_fns[a->esz], a, 1 << a->esz) -static bool trans_UZP2_z(DisasContext *s, arg_rrr_esz *a) -{ - return gen_gvec_ool_arg_zzz(s, uzp_fns[a->esz], a, 1 << a->esz); -} - -static bool trans_UZP1_q(DisasContext *s, arg_rrr_esz *a) -{ - if (!dc_isar_feature(aa64_sve_f64mm, s)) { - return false; - } - return gen_gvec_ool_arg_zzz(s, gen_helper_sve2_uzp_q, a, 0); -} - -static bool trans_UZP2_q(DisasContext *s, arg_rrr_esz *a) -{ - if (!dc_isar_feature(aa64_sve_f64mm, s)) { - return false; - } - return gen_gvec_ool_arg_zzz(s, gen_helper_sve2_uzp_q, a, 16); -} +TRANS_FEAT(UZP1_q, aa64_sve_f64mm, gen_gvec_ool_arg_zzz, + gen_helper_sve2_uzp_q, a, 0) +TRANS_FEAT(UZP2_q, aa64_sve_f64mm, gen_gvec_ool_arg_zzz, + gen_helper_sve2_uzp_q, a, 16) static gen_helper_gvec_3 * const trn_fns[4] = { gen_helper_sve_trn_b, gen_helper_sve_trn_h, gen_helper_sve_trn_s, gen_helper_sve_trn_d, }; -static bool trans_TRN1_z(DisasContext *s, arg_rrr_esz *a) -{ - return gen_gvec_ool_arg_zzz(s, trn_fns[a->esz], a, 0); -} +TRANS_FEAT(TRN1_z, aa64_sve, gen_gvec_ool_arg_zzz, + trn_fns[a->esz], a, 0) +TRANS_FEAT(TRN2_z, aa64_sve, gen_gvec_ool_arg_zzz, + trn_fns[a->esz], a, 1 << a->esz) -static bool trans_TRN2_z(DisasContext *s, arg_rrr_esz *a) -{ - return gen_gvec_ool_arg_zzz(s, trn_fns[a->esz], a, 1 << a->esz); -} - -static bool trans_TRN1_q(DisasContext *s, arg_rrr_esz *a) -{ - if (!dc_isar_feature(aa64_sve_f64mm, s)) { - return false; - } - return gen_gvec_ool_arg_zzz(s, gen_helper_sve2_trn_q, a, 0); -} - -static bool trans_TRN2_q(DisasContext *s, arg_rrr_esz *a) -{ - if (!dc_isar_feature(aa64_sve_f64mm, s)) { - return false; - } - return gen_gvec_ool_arg_zzz(s, gen_helper_sve2_trn_q, a, 16); -} +TRANS_FEAT(TRN1_q, aa64_sve_f64mm, gen_gvec_ool_arg_zzz, + gen_helper_sve2_trn_q, a, 0) +TRANS_FEAT(TRN2_q, aa64_sve_f64mm, gen_gvec_ool_arg_zzz, + gen_helper_sve2_trn_q, a, 16) /* *** SVE Permute Vector - Predicated Group @@ -XXX,XX +XXX,XX @@ static bool trans_USDOT_zzzz(DisasContext *s, arg_USDOT_zzzz *a) TRANS_FEAT(AESMC, aa64_sve2_aes, gen_gvec_ool_zz, gen_helper_crypto_aesmc, a->rd, a->rd, a->decrypt) -static bool do_aese(DisasContext *s, arg_rrr_esz *a, bool decrypt) -{ - if (!dc_isar_feature(aa64_sve2_aes, s)) { - return false; - } - return gen_gvec_ool_arg_zzz(s, gen_helper_crypto_aese, a, decrypt); -} +TRANS_FEAT(AESE, aa64_sve2_aes, gen_gvec_ool_arg_zzz, + gen_helper_crypto_aese, a, false) +TRANS_FEAT(AESD, aa64_sve2_aes, gen_gvec_ool_arg_zzz, + gen_helper_crypto_aese, a, true) -static bool trans_AESE(DisasContext *s, arg_rrr_esz *a) -{ - return do_aese(s, a, false); -} - -static bool trans_AESD(DisasContext *s, arg_rrr_esz *a) -{ - return do_aese(s, a, true); -} - -static bool do_sm4(DisasContext *s, arg_rrr_esz *a, gen_helper_gvec_3 *fn) -{ - if (!dc_isar_feature(aa64_sve2_sm4, s)) { - return false; - } - return gen_gvec_ool_arg_zzz(s, fn, a, 0); -} - -static bool trans_SM4E(DisasContext *s, arg_rrr_esz *a) -{ - return do_sm4(s, a, gen_helper_crypto_sm4e); -} - -static bool trans_SM4EKEY(DisasContext *s, arg_rrr_esz *a) -{ - return do_sm4(s, a, gen_helper_crypto_sm4ekey); -} +TRANS_FEAT(SM4E, aa64_sve2_sm4, gen_gvec_ool_arg_zzz, + gen_helper_crypto_sm4e, a, 0) +TRANS_FEAT(SM4EKEY, aa64_sve2_sm4, gen_gvec_ool_arg_zzz, + gen_helper_crypto_sm4ekey, a, 0) static bool trans_RAX1(DisasContext *s, arg_rrr_esz *a) { -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Convert SVE translation functions using do_sve2_zzz_ool to use TRANS_FEAT and gen_gvec_ool_arg_zzz. Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-8-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 88 ++++++++++++++------------------------ 1 file changed, 31 insertions(+), 57 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool trans_MUL_zzz(DisasContext *s, arg_rrr_esz *a) return true; } -static bool do_sve2_zzz_ool(DisasContext *s, arg_rrr_esz *a, - gen_helper_gvec_3 *fn) -{ - if (!dc_isar_feature(aa64_sve2, s)) { - return false; - } - return gen_gvec_ool_arg_zzz(s, fn, a, 0); -} +static gen_helper_gvec_3 * const smulh_zzz_fns[4] = { + gen_helper_gvec_smulh_b, gen_helper_gvec_smulh_h, + gen_helper_gvec_smulh_s, gen_helper_gvec_smulh_d, +}; +TRANS_FEAT(SMULH_zzz, aa64_sve2, gen_gvec_ool_arg_zzz, + smulh_zzz_fns[a->esz], a, 0) -static bool trans_SMULH_zzz(DisasContext *s, arg_rrr_esz *a) -{ - static gen_helper_gvec_3 * const fns[4] = { - gen_helper_gvec_smulh_b, gen_helper_gvec_smulh_h, - gen_helper_gvec_smulh_s, gen_helper_gvec_smulh_d, - }; - return do_sve2_zzz_ool(s, a, fns[a->esz]); -} +static gen_helper_gvec_3 * const umulh_zzz_fns[4] = { + gen_helper_gvec_umulh_b, gen_helper_gvec_umulh_h, + gen_helper_gvec_umulh_s, gen_helper_gvec_umulh_d, +}; +TRANS_FEAT(UMULH_zzz, aa64_sve2, gen_gvec_ool_arg_zzz, + umulh_zzz_fns[a->esz], a, 0) -static bool trans_UMULH_zzz(DisasContext *s, arg_rrr_esz *a) -{ - static gen_helper_gvec_3 * const fns[4] = { - gen_helper_gvec_umulh_b, gen_helper_gvec_umulh_h, - gen_helper_gvec_umulh_s, gen_helper_gvec_umulh_d, - }; - return do_sve2_zzz_ool(s, a, fns[a->esz]); -} +TRANS_FEAT(PMUL_zzz, aa64_sve2, gen_gvec_ool_arg_zzz, + gen_helper_gvec_pmul_b, a, 0) -static bool trans_PMUL_zzz(DisasContext *s, arg_rrr_esz *a) -{ - return do_sve2_zzz_ool(s, a, gen_helper_gvec_pmul_b); -} +static gen_helper_gvec_3 * const sqdmulh_zzz_fns[4] = { + gen_helper_sve2_sqdmulh_b, gen_helper_sve2_sqdmulh_h, + gen_helper_sve2_sqdmulh_s, gen_helper_sve2_sqdmulh_d, +}; +TRANS_FEAT(SQDMULH_zzz, aa64_sve2, gen_gvec_ool_arg_zzz, + sqdmulh_zzz_fns[a->esz], a, 0) -static bool trans_SQDMULH_zzz(DisasContext *s, arg_rrr_esz *a) -{ - static gen_helper_gvec_3 * const fns[4] = { - gen_helper_sve2_sqdmulh_b, gen_helper_sve2_sqdmulh_h, - gen_helper_sve2_sqdmulh_s, gen_helper_sve2_sqdmulh_d, - }; - return do_sve2_zzz_ool(s, a, fns[a->esz]); -} - -static bool trans_SQRDMULH_zzz(DisasContext *s, arg_rrr_esz *a) -{ - static gen_helper_gvec_3 * const fns[4] = { - gen_helper_sve2_sqrdmulh_b, gen_helper_sve2_sqrdmulh_h, - gen_helper_sve2_sqrdmulh_s, gen_helper_sve2_sqrdmulh_d, - }; - return do_sve2_zzz_ool(s, a, fns[a->esz]); -} +static gen_helper_gvec_3 * const sqrdmulh_zzz_fns[4] = { + gen_helper_sve2_sqrdmulh_b, gen_helper_sve2_sqrdmulh_h, + gen_helper_sve2_sqrdmulh_s, gen_helper_sve2_sqrdmulh_d, +}; +TRANS_FEAT(SQRDMULH_zzz, aa64_sve2, gen_gvec_ool_arg_zzz, + sqrdmulh_zzz_fns[a->esz], a, 0) /* * SVE2 Integer - Predicated @@ -XXX,XX +XXX,XX @@ static bool trans_UQRSHRNT(DisasContext *s, arg_rri_esz *a) } #define DO_SVE2_ZZZ_NARROW(NAME, name) \ -static bool trans_##NAME(DisasContext *s, arg_rrr_esz *a) \ -{ \ - static gen_helper_gvec_3 * const fns[4] = { \ + static gen_helper_gvec_3 * const name##_fns[4] = { \ NULL, gen_helper_sve2_##name##_h, \ gen_helper_sve2_##name##_s, gen_helper_sve2_##name##_d, \ }; \ - return do_sve2_zzz_ool(s, a, fns[a->esz]); \ -} + TRANS_FEAT(NAME, aa64_sve2, gen_gvec_ool_arg_zzz, \ + name##_fns[a->esz], a, 0) DO_SVE2_ZZZ_NARROW(ADDHNB, addhnb) DO_SVE2_ZZZ_NARROW(ADDHNT, addhnt) @@ -XXX,XX +XXX,XX @@ static bool trans_HISTCNT(DisasContext *s, arg_rprr_esz *a) return do_sve2_zpzz_ool(s, a, fns[a->esz - 2]); } -static bool trans_HISTSEG(DisasContext *s, arg_rrr_esz *a) -{ - if (a->esz != 0) { - return false; - } - return do_sve2_zzz_ool(s, a, gen_helper_sve2_histseg); -} +TRANS_FEAT(HISTSEG, aa64_sve2, gen_gvec_ool_arg_zzz, + a->esz == 0 ? gen_helper_sve2_histseg : NULL, a, 0) static bool do_sve2_zpzz_fp(DisasContext *s, arg_rprr_esz *a, gen_helper_gvec_4_ptr *fn) -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-9-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 102 ++++++++++++++----------------------- 1 file changed, 38 insertions(+), 64 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool gen_gvec_ool_arg_zzz(DisasContext *s, gen_helper_gvec_3 *fn, } /* Invoke an out-of-line helper on 4 Zregs. */ -static void gen_gvec_ool_zzzz(DisasContext *s, gen_helper_gvec_4 *fn, +static bool gen_gvec_ool_zzzz(DisasContext *s, gen_helper_gvec_4 *fn, int rd, int rn, int rm, int ra, int data) { - unsigned vsz = vec_full_reg_size(s); - tcg_gen_gvec_4_ool(vec_full_reg_offset(s, rd), - vec_full_reg_offset(s, rn), - vec_full_reg_offset(s, rm), - vec_full_reg_offset(s, ra), - vsz, vsz, data, fn); + if (fn == NULL) { + return false; + } + if (sve_access_check(s)) { + unsigned vsz = vec_full_reg_size(s); + tcg_gen_gvec_4_ool(vec_full_reg_offset(s, rd), + vec_full_reg_offset(s, rn), + vec_full_reg_offset(s, rm), + vec_full_reg_offset(s, ra), + vsz, vsz, data, fn); + } + return true; } /* Invoke an out-of-line helper on 2 Zregs and a predicate. */ @@ -XXX,XX +XXX,XX @@ static bool trans_TBL_sve2(DisasContext *s, arg_rrr_esz *a) if (!dc_isar_feature(aa64_sve2, s)) { return false; } - if (sve_access_check(s)) { - gen_gvec_ool_zzzz(s, fns[a->esz], a->rd, a->rn, - (a->rn + 1) % 32, a->rm, 0); - } - return true; + return gen_gvec_ool_zzzz(s, fns[a->esz], a->rd, a->rn, + (a->rn + 1) % 32, a->rm, 0); } static gen_helper_gvec_3 * const tbx_fns[4] = { @@ -XXX,XX +XXX,XX @@ static bool trans_DOT_zzzz(DisasContext *s, arg_DOT_zzzz *a) { gen_helper_gvec_sdot_b, gen_helper_gvec_sdot_h }, { gen_helper_gvec_udot_b, gen_helper_gvec_udot_h } }; - - if (sve_access_check(s)) { - gen_gvec_ool_zzzz(s, fns[a->u][a->sz], a->rd, a->rn, a->rm, a->ra, 0); - } - return true; + return gen_gvec_ool_zzzz(s, fns[a->u][a->sz], + a->rd, a->rn, a->rm, a->ra, 0); } /* @@ -XXX,XX +XXX,XX @@ static bool trans_DOT_zzzz(DisasContext *s, arg_DOT_zzzz *a) static bool do_zzxz_ool(DisasContext *s, arg_rrxr_esz *a, gen_helper_gvec_4 *fn) { - if (fn == NULL) { - return false; - } - if (sve_access_check(s)) { - gen_gvec_ool_zzzz(s, fn, a->rd, a->rn, a->rm, a->ra, a->index); - } - return true; + return gen_gvec_ool_zzzz(s, fn, a->rd, a->rn, a->rm, a->ra, a->index); } #define DO_RRXR(NAME, FUNC) \ @@ -XXX,XX +XXX,XX @@ static bool trans_SQCADD_rot270(DisasContext *s, arg_rrr_esz *a) static bool do_sve2_zzzz_ool(DisasContext *s, arg_rrrr_esz *a, gen_helper_gvec_4 *fn, int data) { - if (fn == NULL || !dc_isar_feature(aa64_sve2, s)) { + if (!dc_isar_feature(aa64_sve2, s)) { return false; } - if (sve_access_check(s)) { - gen_gvec_ool_zzzz(s, fn, a->rd, a->rn, a->rm, a->ra, data); - } - return true; + return gen_gvec_ool_zzzz(s, fn, a->rd, a->rn, a->rm, a->ra, data); } static bool do_abal(DisasContext *s, arg_rrrr_esz *a, bool uns, bool sel) @@ -XXX,XX +XXX,XX @@ static bool trans_CMLA_zzzz(DisasContext *s, arg_CMLA_zzzz *a) if (!dc_isar_feature(aa64_sve2, s)) { return false; } - if (sve_access_check(s)) { - gen_gvec_ool_zzzz(s, fns[a->esz], a->rd, a->rn, a->rm, a->ra, a->rot); - } - return true; + return gen_gvec_ool_zzzz(s, fns[a->esz], a->rd, a->rn, + a->rm, a->ra, a->rot); } static bool trans_CDOT_zzzz(DisasContext *s, arg_CMLA_zzzz *a) { - if (!dc_isar_feature(aa64_sve2, s) || a->esz < MO_32) { + static gen_helper_gvec_4 * const fns[] = { + NULL, NULL, gen_helper_sve2_cdot_zzzz_s, gen_helper_sve2_cdot_zzzz_d + }; + + if (!dc_isar_feature(aa64_sve2, s)) { return false; } - if (sve_access_check(s)) { - gen_helper_gvec_4 *fn = (a->esz == MO_32 - ? gen_helper_sve2_cdot_zzzz_s - : gen_helper_sve2_cdot_zzzz_d); - gen_gvec_ool_zzzz(s, fn, a->rd, a->rn, a->rm, a->ra, a->rot); - } - return true; + return gen_gvec_ool_zzzz(s, fns[a->esz], a->rd, a->rn, + a->rm, a->ra, a->rot); } static bool trans_SQRDCMLAH_zzzz(DisasContext *s, arg_SQRDCMLAH_zzzz *a) @@ -XXX,XX +XXX,XX @@ static bool trans_SQRDCMLAH_zzzz(DisasContext *s, arg_SQRDCMLAH_zzzz *a) if (!dc_isar_feature(aa64_sve2, s)) { return false; } - if (sve_access_check(s)) { - gen_gvec_ool_zzzz(s, fns[a->esz], a->rd, a->rn, a->rm, a->ra, a->rot); - } - return true; + return gen_gvec_ool_zzzz(s, fns[a->esz], a->rd, a->rn, + a->rm, a->ra, a->rot); } static bool trans_USDOT_zzzz(DisasContext *s, arg_USDOT_zzzz *a) @@ -XXX,XX +XXX,XX @@ static bool do_i8mm_zzzz_ool(DisasContext *s, arg_rrrr_esz *a, if (!dc_isar_feature(aa64_sve_i8mm, s)) { return false; } - if (sve_access_check(s)) { - gen_gvec_ool_zzzz(s, fn, a->rd, a->rn, a->rm, a->ra, data); - } - return true; + return gen_gvec_ool_zzzz(s, fn, a->rd, a->rn, a->rm, a->ra, data); } static bool trans_SMMLA(DisasContext *s, arg_rrrr_esz *a) @@ -XXX,XX +XXX,XX @@ static bool trans_BFDOT_zzzz(DisasContext *s, arg_rrrr_esz *a) if (!dc_isar_feature(aa64_sve_bf16, s)) { return false; } - if (sve_access_check(s)) { - gen_gvec_ool_zzzz(s, gen_helper_gvec_bfdot, - a->rd, a->rn, a->rm, a->ra, 0); - } - return true; + return gen_gvec_ool_zzzz(s, gen_helper_gvec_bfdot, + a->rd, a->rn, a->rm, a->ra, 0); } static bool trans_BFDOT_zzxz(DisasContext *s, arg_rrxr_esz *a) @@ -XXX,XX +XXX,XX @@ static bool trans_BFDOT_zzxz(DisasContext *s, arg_rrxr_esz *a) if (!dc_isar_feature(aa64_sve_bf16, s)) { return false; } - if (sve_access_check(s)) { - gen_gvec_ool_zzzz(s, gen_helper_gvec_bfdot_idx, - a->rd, a->rn, a->rm, a->ra, a->index); - } - return true; + return gen_gvec_ool_zzzz(s, gen_helper_gvec_bfdot_idx, + a->rd, a->rn, a->rm, a->ra, a->index); } static bool trans_BFMMLA(DisasContext *s, arg_rrrr_esz *a) @@ -XXX,XX +XXX,XX @@ static bool trans_BFMMLA(DisasContext *s, arg_rrrr_esz *a) if (!dc_isar_feature(aa64_sve_bf16, s)) { return false; } - if (sve_access_check(s)) { - gen_gvec_ool_zzzz(s, gen_helper_gvec_bfmmla, - a->rd, a->rn, a->rm, a->ra, 0); - } - return true; + return gen_gvec_ool_zzzz(s, gen_helper_gvec_bfmmla, + a->rd, a->rn, a->rm, a->ra, 0); } static bool do_BFMLAL_zzzw(DisasContext *s, arg_rrrr_esz *a, bool sel) -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Convert SVE translation functions directly using gen_gvec_ool_zzzz to TRANS_FEAT. Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-10-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 89 +++++++++++++------------------------- 1 file changed, 29 insertions(+), 60 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static gen_helper_gvec_3 * const sve_tbl_fns[4] = { }; TRANS_FEAT(TBL, aa64_sve, gen_gvec_ool_arg_zzz, sve_tbl_fns[a->esz], a, 0) -static bool trans_TBL_sve2(DisasContext *s, arg_rrr_esz *a) -{ - static gen_helper_gvec_4 * const fns[4] = { - gen_helper_sve2_tbl_b, gen_helper_sve2_tbl_h, - gen_helper_sve2_tbl_s, gen_helper_sve2_tbl_d - }; - - if (!dc_isar_feature(aa64_sve2, s)) { - return false; - } - return gen_gvec_ool_zzzz(s, fns[a->esz], a->rd, a->rn, - (a->rn + 1) % 32, a->rm, 0); -} +static gen_helper_gvec_4 * const sve2_tbl_fns[4] = { + gen_helper_sve2_tbl_b, gen_helper_sve2_tbl_h, + gen_helper_sve2_tbl_s, gen_helper_sve2_tbl_d +}; +TRANS_FEAT(TBL_sve2, aa64_sve2, gen_gvec_ool_zzzz, sve2_tbl_fns[a->esz], + a->rd, a->rn, (a->rn + 1) % 32, a->rm, 0) static gen_helper_gvec_3 * const tbx_fns[4] = { gen_helper_sve2_tbx_b, gen_helper_sve2_tbx_h, @@ -XXX,XX +XXX,XX @@ DO_ZZI(UMIN, umin) #undef DO_ZZI -static bool trans_DOT_zzzz(DisasContext *s, arg_DOT_zzzz *a) -{ - static gen_helper_gvec_4 * const fns[2][2] = { - { gen_helper_gvec_sdot_b, gen_helper_gvec_sdot_h }, - { gen_helper_gvec_udot_b, gen_helper_gvec_udot_h } - }; - return gen_gvec_ool_zzzz(s, fns[a->u][a->sz], - a->rd, a->rn, a->rm, a->ra, 0); -} +static gen_helper_gvec_4 * const dot_fns[2][2] = { + { gen_helper_gvec_sdot_b, gen_helper_gvec_sdot_h }, + { gen_helper_gvec_udot_b, gen_helper_gvec_udot_h } +}; +TRANS_FEAT(DOT_zzzz, aa64_sve, gen_gvec_ool_zzzz, + dot_fns[a->u][a->sz], a->rd, a->rn, a->rm, a->ra, 0) /* * SVE Multiply - Indexed @@ -XXX,XX +XXX,XX @@ static bool trans_UMLSLT_zzzw(DisasContext *s, arg_rrrr_esz *a) return do_umlsl_zzzw(s, a, true); } -static bool trans_CMLA_zzzz(DisasContext *s, arg_CMLA_zzzz *a) -{ - static gen_helper_gvec_4 * const fns[] = { - gen_helper_sve2_cmla_zzzz_b, gen_helper_sve2_cmla_zzzz_h, - gen_helper_sve2_cmla_zzzz_s, gen_helper_sve2_cmla_zzzz_d, - }; +static gen_helper_gvec_4 * const cmla_fns[] = { + gen_helper_sve2_cmla_zzzz_b, gen_helper_sve2_cmla_zzzz_h, + gen_helper_sve2_cmla_zzzz_s, gen_helper_sve2_cmla_zzzz_d, +}; +TRANS_FEAT(CMLA_zzzz, aa64_sve2, gen_gvec_ool_zzzz, + cmla_fns[a->esz], a->rd, a->rn, a->rm, a->ra, a->rot) - if (!dc_isar_feature(aa64_sve2, s)) { - return false; - } - return gen_gvec_ool_zzzz(s, fns[a->esz], a->rd, a->rn, - a->rm, a->ra, a->rot); -} +static gen_helper_gvec_4 * const cdot_fns[] = { + NULL, NULL, gen_helper_sve2_cdot_zzzz_s, gen_helper_sve2_cdot_zzzz_d +}; +TRANS_FEAT(CDOT_zzzz, aa64_sve2, gen_gvec_ool_zzzz, + cdot_fns[a->esz], a->rd, a->rn, a->rm, a->ra, a->rot) -static bool trans_CDOT_zzzz(DisasContext *s, arg_CMLA_zzzz *a) -{ - static gen_helper_gvec_4 * const fns[] = { - NULL, NULL, gen_helper_sve2_cdot_zzzz_s, gen_helper_sve2_cdot_zzzz_d - }; - - if (!dc_isar_feature(aa64_sve2, s)) { - return false; - } - return gen_gvec_ool_zzzz(s, fns[a->esz], a->rd, a->rn, - a->rm, a->ra, a->rot); -} - -static bool trans_SQRDCMLAH_zzzz(DisasContext *s, arg_SQRDCMLAH_zzzz *a) -{ - static gen_helper_gvec_4 * const fns[] = { - gen_helper_sve2_sqrdcmlah_zzzz_b, gen_helper_sve2_sqrdcmlah_zzzz_h, - gen_helper_sve2_sqrdcmlah_zzzz_s, gen_helper_sve2_sqrdcmlah_zzzz_d, - }; - - if (!dc_isar_feature(aa64_sve2, s)) { - return false; - } - return gen_gvec_ool_zzzz(s, fns[a->esz], a->rd, a->rn, - a->rm, a->ra, a->rot); -} +static gen_helper_gvec_4 * const sqrdcmlah_fns[] = { + gen_helper_sve2_sqrdcmlah_zzzz_b, gen_helper_sve2_sqrdcmlah_zzzz_h, + gen_helper_sve2_sqrdcmlah_zzzz_s, gen_helper_sve2_sqrdcmlah_zzzz_d, +}; +TRANS_FEAT(SQRDCMLAH_zzzz, aa64_sve2, gen_gvec_ool_zzzz, + sqrdcmlah_fns[a->esz], a->rd, a->rn, a->rm, a->ra, a->rot) static bool trans_USDOT_zzzz(DisasContext *s, arg_USDOT_zzzz *a) { -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Use gen_gvec_ool_arg_zzzz instead of gen_gvec_ool_zzzz when the arguments come from arg_rrrr_esz. Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-11-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool gen_gvec_ool_zzzz(DisasContext *s, gen_helper_gvec_4 *fn, return true; } +static bool gen_gvec_ool_arg_zzzz(DisasContext *s, gen_helper_gvec_4 *fn, + arg_rrrr_esz *a, int data) +{ + return gen_gvec_ool_zzzz(s, fn, a->rd, a->rn, a->rm, a->ra, data); +} + /* Invoke an out-of-line helper on 2 Zregs and a predicate. */ static void gen_gvec_ool_zzp(DisasContext *s, gen_helper_gvec_3 *fn, int rd, int rn, int pg, int data) @@ -XXX,XX +XXX,XX @@ static bool do_sve2_zzzz_ool(DisasContext *s, arg_rrrr_esz *a, if (!dc_isar_feature(aa64_sve2, s)) { return false; } - return gen_gvec_ool_zzzz(s, fn, a->rd, a->rn, a->rm, a->ra, data); + return gen_gvec_ool_arg_zzzz(s, fn, a, data); } static bool do_abal(DisasContext *s, arg_rrrr_esz *a, bool uns, bool sel) @@ -XXX,XX +XXX,XX @@ static bool do_i8mm_zzzz_ool(DisasContext *s, arg_rrrr_esz *a, if (!dc_isar_feature(aa64_sve_i8mm, s)) { return false; } - return gen_gvec_ool_zzzz(s, fn, a->rd, a->rn, a->rm, a->ra, data); + return gen_gvec_ool_arg_zzzz(s, fn, a, data); } static bool trans_SMMLA(DisasContext *s, arg_rrrr_esz *a) @@ -XXX,XX +XXX,XX @@ static bool trans_BFDOT_zzzz(DisasContext *s, arg_rrrr_esz *a) if (!dc_isar_feature(aa64_sve_bf16, s)) { return false; } - return gen_gvec_ool_zzzz(s, gen_helper_gvec_bfdot, - a->rd, a->rn, a->rm, a->ra, 0); + return gen_gvec_ool_arg_zzzz(s, gen_helper_gvec_bfdot, a, 0); } static bool trans_BFDOT_zzxz(DisasContext *s, arg_rrxr_esz *a) @@ -XXX,XX +XXX,XX @@ static bool trans_BFMMLA(DisasContext *s, arg_rrrr_esz *a) if (!dc_isar_feature(aa64_sve_bf16, s)) { return false; } - return gen_gvec_ool_zzzz(s, gen_helper_gvec_bfmmla, - a->rd, a->rn, a->rm, a->ra, 0); + return gen_gvec_ool_arg_zzzz(s, gen_helper_gvec_bfmmla, a, 0); } static bool do_BFMLAL_zzzw(DisasContext *s, arg_rrrr_esz *a, bool sel) -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Convert SVE translation functions using do_sve2_zzzz_ool to use TRANS_FEAT and gen_gvec_ool_arg_zzzz. Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-12-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 263 +++++++++++-------------------------- 1 file changed, 79 insertions(+), 184 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool trans_SQCADD_rot270(DisasContext *s, arg_rrr_esz *a) return do_cadd(s, a, true, true); } -static bool do_sve2_zzzz_ool(DisasContext *s, arg_rrrr_esz *a, - gen_helper_gvec_4 *fn, int data) -{ - if (!dc_isar_feature(aa64_sve2, s)) { - return false; - } - return gen_gvec_ool_arg_zzzz(s, fn, a, data); -} +static gen_helper_gvec_4 * const sabal_fns[4] = { + NULL, gen_helper_sve2_sabal_h, + gen_helper_sve2_sabal_s, gen_helper_sve2_sabal_d, +}; +TRANS_FEAT(SABALB, aa64_sve2, gen_gvec_ool_arg_zzzz, sabal_fns[a->esz], a, 0) +TRANS_FEAT(SABALT, aa64_sve2, gen_gvec_ool_arg_zzzz, sabal_fns[a->esz], a, 1) -static bool do_abal(DisasContext *s, arg_rrrr_esz *a, bool uns, bool sel) -{ - static gen_helper_gvec_4 * const fns[2][4] = { - { NULL, gen_helper_sve2_sabal_h, - gen_helper_sve2_sabal_s, gen_helper_sve2_sabal_d }, - { NULL, gen_helper_sve2_uabal_h, - gen_helper_sve2_uabal_s, gen_helper_sve2_uabal_d }, - }; - return do_sve2_zzzz_ool(s, a, fns[uns][a->esz], sel); -} - -static bool trans_SABALB(DisasContext *s, arg_rrrr_esz *a) -{ - return do_abal(s, a, false, false); -} - -static bool trans_SABALT(DisasContext *s, arg_rrrr_esz *a) -{ - return do_abal(s, a, false, true); -} - -static bool trans_UABALB(DisasContext *s, arg_rrrr_esz *a) -{ - return do_abal(s, a, true, false); -} - -static bool trans_UABALT(DisasContext *s, arg_rrrr_esz *a) -{ - return do_abal(s, a, true, true); -} +static gen_helper_gvec_4 * const uabal_fns[4] = { + NULL, gen_helper_sve2_uabal_h, + gen_helper_sve2_uabal_s, gen_helper_sve2_uabal_d, +}; +TRANS_FEAT(UABALB, aa64_sve2, gen_gvec_ool_arg_zzzz, uabal_fns[a->esz], a, 0) +TRANS_FEAT(UABALT, aa64_sve2, gen_gvec_ool_arg_zzzz, uabal_fns[a->esz], a, 1) static bool do_adcl(DisasContext *s, arg_rrrr_esz *a, bool sel) { @@ -XXX,XX +XXX,XX @@ static bool do_adcl(DisasContext *s, arg_rrrr_esz *a, bool sel) * Note that in this case the ESZ field encodes both size and sign. * Split out 'subtract' into bit 1 of the data field for the helper. */ - return do_sve2_zzzz_ool(s, a, fns[a->esz & 1], (a->esz & 2) | sel); + return gen_gvec_ool_arg_zzzz(s, fns[a->esz & 1], a, (a->esz & 2) | sel); } -static bool trans_ADCLB(DisasContext *s, arg_rrrr_esz *a) -{ - return do_adcl(s, a, false); -} - -static bool trans_ADCLT(DisasContext *s, arg_rrrr_esz *a) -{ - return do_adcl(s, a, true); -} +TRANS_FEAT(ADCLB, aa64_sve2, do_adcl, a, false) +TRANS_FEAT(ADCLT, aa64_sve2, do_adcl, a, true) static bool do_sve2_fn2i(DisasContext *s, arg_rri_esz *a, GVecGen2iFn *fn) { @@ -XXX,XX +XXX,XX @@ static bool trans_FMMLA(DisasContext *s, arg_rrrr_esz *a) return true; } -static bool do_sqdmlal_zzzw(DisasContext *s, arg_rrrr_esz *a, - bool sel1, bool sel2) -{ - static gen_helper_gvec_4 * const fns[] = { - NULL, gen_helper_sve2_sqdmlal_zzzw_h, - gen_helper_sve2_sqdmlal_zzzw_s, gen_helper_sve2_sqdmlal_zzzw_d, - }; - return do_sve2_zzzz_ool(s, a, fns[a->esz], (sel2 << 1) | sel1); -} +static gen_helper_gvec_4 * const sqdmlal_zzzw_fns[] = { + NULL, gen_helper_sve2_sqdmlal_zzzw_h, + gen_helper_sve2_sqdmlal_zzzw_s, gen_helper_sve2_sqdmlal_zzzw_d, +}; +TRANS_FEAT(SQDMLALB_zzzw, aa64_sve2, gen_gvec_ool_arg_zzzz, + sqdmlal_zzzw_fns[a->esz], a, 0) +TRANS_FEAT(SQDMLALT_zzzw, aa64_sve2, gen_gvec_ool_arg_zzzz, + sqdmlal_zzzw_fns[a->esz], a, 3) +TRANS_FEAT(SQDMLALBT, aa64_sve2, gen_gvec_ool_arg_zzzz, + sqdmlal_zzzw_fns[a->esz], a, 2) -static bool do_sqdmlsl_zzzw(DisasContext *s, arg_rrrr_esz *a, - bool sel1, bool sel2) -{ - static gen_helper_gvec_4 * const fns[] = { - NULL, gen_helper_sve2_sqdmlsl_zzzw_h, - gen_helper_sve2_sqdmlsl_zzzw_s, gen_helper_sve2_sqdmlsl_zzzw_d, - }; - return do_sve2_zzzz_ool(s, a, fns[a->esz], (sel2 << 1) | sel1); -} +static gen_helper_gvec_4 * const sqdmlsl_zzzw_fns[] = { + NULL, gen_helper_sve2_sqdmlsl_zzzw_h, + gen_helper_sve2_sqdmlsl_zzzw_s, gen_helper_sve2_sqdmlsl_zzzw_d, +}; +TRANS_FEAT(SQDMLSLB_zzzw, aa64_sve2, gen_gvec_ool_arg_zzzz, + sqdmlsl_zzzw_fns[a->esz], a, 0) +TRANS_FEAT(SQDMLSLT_zzzw, aa64_sve2, gen_gvec_ool_arg_zzzz, + sqdmlsl_zzzw_fns[a->esz], a, 3) +TRANS_FEAT(SQDMLSLBT, aa64_sve2, gen_gvec_ool_arg_zzzz, + sqdmlsl_zzzw_fns[a->esz], a, 2) -static bool trans_SQDMLALB_zzzw(DisasContext *s, arg_rrrr_esz *a) -{ - return do_sqdmlal_zzzw(s, a, false, false); -} +static gen_helper_gvec_4 * const sqrdmlah_fns[] = { + gen_helper_sve2_sqrdmlah_b, gen_helper_sve2_sqrdmlah_h, + gen_helper_sve2_sqrdmlah_s, gen_helper_sve2_sqrdmlah_d, +}; +TRANS_FEAT(SQRDMLAH_zzzz, aa64_sve2, gen_gvec_ool_arg_zzzz, + sqrdmlah_fns[a->esz], a, 0) -static bool trans_SQDMLALT_zzzw(DisasContext *s, arg_rrrr_esz *a) -{ - return do_sqdmlal_zzzw(s, a, true, true); -} +static gen_helper_gvec_4 * const sqrdmlsh_fns[] = { + gen_helper_sve2_sqrdmlsh_b, gen_helper_sve2_sqrdmlsh_h, + gen_helper_sve2_sqrdmlsh_s, gen_helper_sve2_sqrdmlsh_d, +}; +TRANS_FEAT(SQRDMLSH_zzzz, aa64_sve2, gen_gvec_ool_arg_zzzz, + sqrdmlsh_fns[a->esz], a, 0) -static bool trans_SQDMLALBT(DisasContext *s, arg_rrrr_esz *a) -{ - return do_sqdmlal_zzzw(s, a, false, true); -} +static gen_helper_gvec_4 * const smlal_zzzw_fns[] = { + NULL, gen_helper_sve2_smlal_zzzw_h, + gen_helper_sve2_smlal_zzzw_s, gen_helper_sve2_smlal_zzzw_d, +}; +TRANS_FEAT(SMLALB_zzzw, aa64_sve2, gen_gvec_ool_arg_zzzz, + smlal_zzzw_fns[a->esz], a, 0) +TRANS_FEAT(SMLALT_zzzw, aa64_sve2, gen_gvec_ool_arg_zzzz, + smlal_zzzw_fns[a->esz], a, 1) -static bool trans_SQDMLSLB_zzzw(DisasContext *s, arg_rrrr_esz *a) -{ - return do_sqdmlsl_zzzw(s, a, false, false); -} +static gen_helper_gvec_4 * const umlal_zzzw_fns[] = { + NULL, gen_helper_sve2_umlal_zzzw_h, + gen_helper_sve2_umlal_zzzw_s, gen_helper_sve2_umlal_zzzw_d, +}; +TRANS_FEAT(UMLALB_zzzw, aa64_sve2, gen_gvec_ool_arg_zzzz, + umlal_zzzw_fns[a->esz], a, 0) +TRANS_FEAT(UMLALT_zzzw, aa64_sve2, gen_gvec_ool_arg_zzzz, + umlal_zzzw_fns[a->esz], a, 1) -static bool trans_SQDMLSLT_zzzw(DisasContext *s, arg_rrrr_esz *a) -{ - return do_sqdmlsl_zzzw(s, a, true, true); -} +static gen_helper_gvec_4 * const smlsl_zzzw_fns[] = { + NULL, gen_helper_sve2_smlsl_zzzw_h, + gen_helper_sve2_smlsl_zzzw_s, gen_helper_sve2_smlsl_zzzw_d, +}; +TRANS_FEAT(SMLSLB_zzzw, aa64_sve2, gen_gvec_ool_arg_zzzz, + smlsl_zzzw_fns[a->esz], a, 0) +TRANS_FEAT(SMLSLT_zzzw, aa64_sve2, gen_gvec_ool_arg_zzzz, + smlsl_zzzw_fns[a->esz], a, 1) -static bool trans_SQDMLSLBT(DisasContext *s, arg_rrrr_esz *a) -{ - return do_sqdmlsl_zzzw(s, a, false, true); -} - -static bool trans_SQRDMLAH_zzzz(DisasContext *s, arg_rrrr_esz *a) -{ - static gen_helper_gvec_4 * const fns[] = { - gen_helper_sve2_sqrdmlah_b, gen_helper_sve2_sqrdmlah_h, - gen_helper_sve2_sqrdmlah_s, gen_helper_sve2_sqrdmlah_d, - }; - return do_sve2_zzzz_ool(s, a, fns[a->esz], 0); -} - -static bool trans_SQRDMLSH_zzzz(DisasContext *s, arg_rrrr_esz *a) -{ - static gen_helper_gvec_4 * const fns[] = { - gen_helper_sve2_sqrdmlsh_b, gen_helper_sve2_sqrdmlsh_h, - gen_helper_sve2_sqrdmlsh_s, gen_helper_sve2_sqrdmlsh_d, - }; - return do_sve2_zzzz_ool(s, a, fns[a->esz], 0); -} - -static bool do_smlal_zzzw(DisasContext *s, arg_rrrr_esz *a, bool sel) -{ - static gen_helper_gvec_4 * const fns[] = { - NULL, gen_helper_sve2_smlal_zzzw_h, - gen_helper_sve2_smlal_zzzw_s, gen_helper_sve2_smlal_zzzw_d, - }; - return do_sve2_zzzz_ool(s, a, fns[a->esz], sel); -} - -static bool trans_SMLALB_zzzw(DisasContext *s, arg_rrrr_esz *a) -{ - return do_smlal_zzzw(s, a, false); -} - -static bool trans_SMLALT_zzzw(DisasContext *s, arg_rrrr_esz *a) -{ - return do_smlal_zzzw(s, a, true); -} - -static bool do_umlal_zzzw(DisasContext *s, arg_rrrr_esz *a, bool sel) -{ - static gen_helper_gvec_4 * const fns[] = { - NULL, gen_helper_sve2_umlal_zzzw_h, - gen_helper_sve2_umlal_zzzw_s, gen_helper_sve2_umlal_zzzw_d, - }; - return do_sve2_zzzz_ool(s, a, fns[a->esz], sel); -} - -static bool trans_UMLALB_zzzw(DisasContext *s, arg_rrrr_esz *a) -{ - return do_umlal_zzzw(s, a, false); -} - -static bool trans_UMLALT_zzzw(DisasContext *s, arg_rrrr_esz *a) -{ - return do_umlal_zzzw(s, a, true); -} - -static bool do_smlsl_zzzw(DisasContext *s, arg_rrrr_esz *a, bool sel) -{ - static gen_helper_gvec_4 * const fns[] = { - NULL, gen_helper_sve2_smlsl_zzzw_h, - gen_helper_sve2_smlsl_zzzw_s, gen_helper_sve2_smlsl_zzzw_d, - }; - return do_sve2_zzzz_ool(s, a, fns[a->esz], sel); -} - -static bool trans_SMLSLB_zzzw(DisasContext *s, arg_rrrr_esz *a) -{ - return do_smlsl_zzzw(s, a, false); -} - -static bool trans_SMLSLT_zzzw(DisasContext *s, arg_rrrr_esz *a) -{ - return do_smlsl_zzzw(s, a, true); -} - -static bool do_umlsl_zzzw(DisasContext *s, arg_rrrr_esz *a, bool sel) -{ - static gen_helper_gvec_4 * const fns[] = { - NULL, gen_helper_sve2_umlsl_zzzw_h, - gen_helper_sve2_umlsl_zzzw_s, gen_helper_sve2_umlsl_zzzw_d, - }; - return do_sve2_zzzz_ool(s, a, fns[a->esz], sel); -} - -static bool trans_UMLSLB_zzzw(DisasContext *s, arg_rrrr_esz *a) -{ - return do_umlsl_zzzw(s, a, false); -} - -static bool trans_UMLSLT_zzzw(DisasContext *s, arg_rrrr_esz *a) -{ - return do_umlsl_zzzw(s, a, true); -} +static gen_helper_gvec_4 * const umlsl_zzzw_fns[] = { + NULL, gen_helper_sve2_umlsl_zzzw_h, + gen_helper_sve2_umlsl_zzzw_s, gen_helper_sve2_umlsl_zzzw_d, +}; +TRANS_FEAT(UMLSLB_zzzw, aa64_sve2, gen_gvec_ool_arg_zzzz, + umlsl_zzzw_fns[a->esz], a, 0) +TRANS_FEAT(UMLSLT_zzzw, aa64_sve2, gen_gvec_ool_arg_zzzz, + umlsl_zzzw_fns[a->esz], a, 1) static gen_helper_gvec_4 * const cmla_fns[] = { gen_helper_sve2_cmla_zzzz_b, gen_helper_sve2_cmla_zzzz_h, -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Convert SVE translation functions directly using gen_gvec_ool_arg_zzzz to TRANS_FEAT. Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-13-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 47 ++++++++------------------------------ 1 file changed, 10 insertions(+), 37 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool trans_FMLSLT_zzxw(DisasContext *s, arg_rrxr_esz *a) return do_FMLAL_zzxw(s, a, true, true); } -static bool do_i8mm_zzzz_ool(DisasContext *s, arg_rrrr_esz *a, - gen_helper_gvec_4 *fn, int data) -{ - if (!dc_isar_feature(aa64_sve_i8mm, s)) { - return false; - } - return gen_gvec_ool_arg_zzzz(s, fn, a, data); -} +TRANS_FEAT(SMMLA, aa64_sve_i8mm, gen_gvec_ool_arg_zzzz, + gen_helper_gvec_smmla_b, a, 0) +TRANS_FEAT(USMMLA, aa64_sve_i8mm, gen_gvec_ool_arg_zzzz, + gen_helper_gvec_usmmla_b, a, 0) +TRANS_FEAT(UMMLA, aa64_sve_i8mm, gen_gvec_ool_arg_zzzz, + gen_helper_gvec_ummla_b, a, 0) -static bool trans_SMMLA(DisasContext *s, arg_rrrr_esz *a) -{ - return do_i8mm_zzzz_ool(s, a, gen_helper_gvec_smmla_b, 0); -} - -static bool trans_USMMLA(DisasContext *s, arg_rrrr_esz *a) -{ - return do_i8mm_zzzz_ool(s, a, gen_helper_gvec_usmmla_b, 0); -} - -static bool trans_UMMLA(DisasContext *s, arg_rrrr_esz *a) -{ - return do_i8mm_zzzz_ool(s, a, gen_helper_gvec_ummla_b, 0); -} - -static bool trans_BFDOT_zzzz(DisasContext *s, arg_rrrr_esz *a) -{ - if (!dc_isar_feature(aa64_sve_bf16, s)) { - return false; - } - return gen_gvec_ool_arg_zzzz(s, gen_helper_gvec_bfdot, a, 0); -} +TRANS_FEAT(BFDOT_zzzz, aa64_sve_bf16, gen_gvec_ool_arg_zzzz, + gen_helper_gvec_bfdot, a, 0) static bool trans_BFDOT_zzxz(DisasContext *s, arg_rrxr_esz *a) { @@ -XXX,XX +XXX,XX @@ static bool trans_BFDOT_zzxz(DisasContext *s, arg_rrxr_esz *a) a->rd, a->rn, a->rm, a->ra, a->index); } -static bool trans_BFMMLA(DisasContext *s, arg_rrrr_esz *a) -{ - if (!dc_isar_feature(aa64_sve_bf16, s)) { - return false; - } - return gen_gvec_ool_arg_zzzz(s, gen_helper_gvec_bfmmla, a, 0); -} +TRANS_FEAT(BFMMLA, aa64_sve_bf16, gen_gvec_ool_arg_zzzz, + gen_helper_gvec_bfmmla, a, 0) static bool do_BFMLAL_zzzw(DisasContext *s, arg_rrrr_esz *a, bool sel) { -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Rename the function to match gen_gvec_ool_arg_zzzz, and move to be adjacent. Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-14-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool gen_gvec_ool_arg_zzzz(DisasContext *s, gen_helper_gvec_4 *fn, return gen_gvec_ool_zzzz(s, fn, a->rd, a->rn, a->rm, a->ra, data); } +static bool gen_gvec_ool_arg_zzxz(DisasContext *s, gen_helper_gvec_4 *fn, + arg_rrxr_esz *a) +{ + return gen_gvec_ool_zzzz(s, fn, a->rd, a->rn, a->rm, a->ra, a->index); +} + /* Invoke an out-of-line helper on 2 Zregs and a predicate. */ static void gen_gvec_ool_zzp(DisasContext *s, gen_helper_gvec_3 *fn, int rd, int rn, int pg, int data) @@ -XXX,XX +XXX,XX @@ TRANS_FEAT(DOT_zzzz, aa64_sve, gen_gvec_ool_zzzz, * SVE Multiply - Indexed */ -static bool do_zzxz_ool(DisasContext *s, arg_rrxr_esz *a, - gen_helper_gvec_4 *fn) -{ - return gen_gvec_ool_zzzz(s, fn, a->rd, a->rn, a->rm, a->ra, a->index); -} - #define DO_RRXR(NAME, FUNC) \ static bool NAME(DisasContext *s, arg_rrxr_esz *a) \ - { return do_zzxz_ool(s, a, FUNC); } + { return gen_gvec_ool_arg_zzxz(s, FUNC, a); } DO_RRXR(trans_SDOT_zzxw_s, gen_helper_gvec_sdot_idx_b) DO_RRXR(trans_SDOT_zzxw_d, gen_helper_gvec_sdot_idx_h) @@ -XXX,XX +XXX,XX @@ static bool trans_SUDOT_zzxw_s(DisasContext *s, arg_rrxr_esz *a) if (!dc_isar_feature(aa64_sve_i8mm, s)) { return false; } - return do_zzxz_ool(s, a, gen_helper_gvec_sudot_idx_b); + return gen_gvec_ool_arg_zzxz(s, gen_helper_gvec_sudot_idx_b, a); } static bool trans_USDOT_zzxw_s(DisasContext *s, arg_rrxr_esz *a) @@ -XXX,XX +XXX,XX @@ static bool trans_USDOT_zzxw_s(DisasContext *s, arg_rrxr_esz *a) if (!dc_isar_feature(aa64_sve_i8mm, s)) { return false; } - return do_zzxz_ool(s, a, gen_helper_gvec_usdot_idx_b); + return gen_gvec_ool_arg_zzxz(s, gen_helper_gvec_usdot_idx_b, a); } #undef DO_RRXR -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Convert SVE translation functions directly using gen_gvec_ool_arg_zzxz to TRANS_FEAT. Also include BFDOT_zzxz, which was using gen_gvec_ool_zzzz. Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-15-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 48 +++++++++++--------------------------- 1 file changed, 14 insertions(+), 34 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ TRANS_FEAT(DOT_zzzz, aa64_sve, gen_gvec_ool_zzzz, * SVE Multiply - Indexed */ -#define DO_RRXR(NAME, FUNC) \ - static bool NAME(DisasContext *s, arg_rrxr_esz *a) \ - { return gen_gvec_ool_arg_zzxz(s, FUNC, a); } +TRANS_FEAT(SDOT_zzxw_s, aa64_sve, gen_gvec_ool_arg_zzxz, + gen_helper_gvec_sdot_idx_b, a) +TRANS_FEAT(SDOT_zzxw_d, aa64_sve, gen_gvec_ool_arg_zzxz, + gen_helper_gvec_sdot_idx_h, a) +TRANS_FEAT(UDOT_zzxw_s, aa64_sve, gen_gvec_ool_arg_zzxz, + gen_helper_gvec_udot_idx_b, a) +TRANS_FEAT(UDOT_zzxw_d, aa64_sve, gen_gvec_ool_arg_zzxz, + gen_helper_gvec_udot_idx_h, a) -DO_RRXR(trans_SDOT_zzxw_s, gen_helper_gvec_sdot_idx_b) -DO_RRXR(trans_SDOT_zzxw_d, gen_helper_gvec_sdot_idx_h) -DO_RRXR(trans_UDOT_zzxw_s, gen_helper_gvec_udot_idx_b) -DO_RRXR(trans_UDOT_zzxw_d, gen_helper_gvec_udot_idx_h) - -static bool trans_SUDOT_zzxw_s(DisasContext *s, arg_rrxr_esz *a) -{ - if (!dc_isar_feature(aa64_sve_i8mm, s)) { - return false; - } - return gen_gvec_ool_arg_zzxz(s, gen_helper_gvec_sudot_idx_b, a); -} - -static bool trans_USDOT_zzxw_s(DisasContext *s, arg_rrxr_esz *a) -{ - if (!dc_isar_feature(aa64_sve_i8mm, s)) { - return false; - } - return gen_gvec_ool_arg_zzxz(s, gen_helper_gvec_usdot_idx_b, a); -} - -#undef DO_RRXR +TRANS_FEAT(SUDOT_zzxw_s, aa64_sve_i8mm, gen_gvec_ool_arg_zzxz, + gen_helper_gvec_sudot_idx_b, a) +TRANS_FEAT(USDOT_zzxw_s, aa64_sve_i8mm, gen_gvec_ool_arg_zzxz, + gen_helper_gvec_usdot_idx_b, a) static bool do_sve2_zzz_data(DisasContext *s, int rd, int rn, int rm, int data, gen_helper_gvec_3 *fn) @@ -XXX,XX +XXX,XX @@ TRANS_FEAT(UMMLA, aa64_sve_i8mm, gen_gvec_ool_arg_zzzz, TRANS_FEAT(BFDOT_zzzz, aa64_sve_bf16, gen_gvec_ool_arg_zzzz, gen_helper_gvec_bfdot, a, 0) - -static bool trans_BFDOT_zzxz(DisasContext *s, arg_rrxr_esz *a) -{ - if (!dc_isar_feature(aa64_sve_bf16, s)) { - return false; - } - return gen_gvec_ool_zzzz(s, gen_helper_gvec_bfdot_idx, - a->rd, a->rn, a->rm, a->ra, a->index); -} +TRANS_FEAT(BFDOT_zzxz, aa64_sve_bf16, gen_gvec_ool_arg_zzxz, + gen_helper_gvec_bfdot_idx, a) TRANS_FEAT(BFMMLA, aa64_sve_bf16, gen_gvec_ool_arg_zzzz, gen_helper_gvec_bfmmla, a, 0) -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Convert SVE translation functions using do_sve2_zzz_data to use TRANS_FEAT and gen_gvec_ool_zzz. Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-16-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 69 ++++++++++++++------------------------ 1 file changed, 25 insertions(+), 44 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ TRANS_FEAT(SUDOT_zzxw_s, aa64_sve_i8mm, gen_gvec_ool_arg_zzxz, TRANS_FEAT(USDOT_zzxw_s, aa64_sve_i8mm, gen_gvec_ool_arg_zzxz, gen_helper_gvec_usdot_idx_b, a) -static bool do_sve2_zzz_data(DisasContext *s, int rd, int rn, int rm, int data, - gen_helper_gvec_3 *fn) -{ - if (fn == NULL || !dc_isar_feature(aa64_sve2, s)) { - return false; - } - if (sve_access_check(s)) { - unsigned vsz = vec_full_reg_size(s); - tcg_gen_gvec_3_ool(vec_full_reg_offset(s, rd), - vec_full_reg_offset(s, rn), - vec_full_reg_offset(s, rm), - vsz, vsz, data, fn); - } - return true; -} - #define DO_SVE2_RRX(NAME, FUNC) \ - static bool NAME(DisasContext *s, arg_rrx_esz *a) \ - { return do_sve2_zzz_data(s, a->rd, a->rn, a->rm, a->index, FUNC); } + TRANS_FEAT(NAME, aa64_sve, gen_gvec_ool_zzz, FUNC, \ + a->rd, a->rn, a->rm, a->index) -DO_SVE2_RRX(trans_MUL_zzx_h, gen_helper_gvec_mul_idx_h) -DO_SVE2_RRX(trans_MUL_zzx_s, gen_helper_gvec_mul_idx_s) -DO_SVE2_RRX(trans_MUL_zzx_d, gen_helper_gvec_mul_idx_d) +DO_SVE2_RRX(MUL_zzx_h, gen_helper_gvec_mul_idx_h) +DO_SVE2_RRX(MUL_zzx_s, gen_helper_gvec_mul_idx_s) +DO_SVE2_RRX(MUL_zzx_d, gen_helper_gvec_mul_idx_d) -DO_SVE2_RRX(trans_SQDMULH_zzx_h, gen_helper_sve2_sqdmulh_idx_h) -DO_SVE2_RRX(trans_SQDMULH_zzx_s, gen_helper_sve2_sqdmulh_idx_s) -DO_SVE2_RRX(trans_SQDMULH_zzx_d, gen_helper_sve2_sqdmulh_idx_d) +DO_SVE2_RRX(SQDMULH_zzx_h, gen_helper_sve2_sqdmulh_idx_h) +DO_SVE2_RRX(SQDMULH_zzx_s, gen_helper_sve2_sqdmulh_idx_s) +DO_SVE2_RRX(SQDMULH_zzx_d, gen_helper_sve2_sqdmulh_idx_d) -DO_SVE2_RRX(trans_SQRDMULH_zzx_h, gen_helper_sve2_sqrdmulh_idx_h) -DO_SVE2_RRX(trans_SQRDMULH_zzx_s, gen_helper_sve2_sqrdmulh_idx_s) -DO_SVE2_RRX(trans_SQRDMULH_zzx_d, gen_helper_sve2_sqrdmulh_idx_d) +DO_SVE2_RRX(SQRDMULH_zzx_h, gen_helper_sve2_sqrdmulh_idx_h) +DO_SVE2_RRX(SQRDMULH_zzx_s, gen_helper_sve2_sqrdmulh_idx_s) +DO_SVE2_RRX(SQRDMULH_zzx_d, gen_helper_sve2_sqrdmulh_idx_d) #undef DO_SVE2_RRX #define DO_SVE2_RRX_TB(NAME, FUNC, TOP) \ - static bool NAME(DisasContext *s, arg_rrx_esz *a) \ - { \ - return do_sve2_zzz_data(s, a->rd, a->rn, a->rm, \ - (a->index << 1) | TOP, FUNC); \ - } + TRANS_FEAT(NAME, aa64_sve, gen_gvec_ool_zzz, FUNC, \ + a->rd, a->rn, a->rm, (a->index << 1) | TOP) -DO_SVE2_RRX_TB(trans_SQDMULLB_zzx_s, gen_helper_sve2_sqdmull_idx_s, false) -DO_SVE2_RRX_TB(trans_SQDMULLB_zzx_d, gen_helper_sve2_sqdmull_idx_d, false) -DO_SVE2_RRX_TB(trans_SQDMULLT_zzx_s, gen_helper_sve2_sqdmull_idx_s, true) -DO_SVE2_RRX_TB(trans_SQDMULLT_zzx_d, gen_helper_sve2_sqdmull_idx_d, true) +DO_SVE2_RRX_TB(SQDMULLB_zzx_s, gen_helper_sve2_sqdmull_idx_s, false) +DO_SVE2_RRX_TB(SQDMULLB_zzx_d, gen_helper_sve2_sqdmull_idx_d, false) +DO_SVE2_RRX_TB(SQDMULLT_zzx_s, gen_helper_sve2_sqdmull_idx_s, true) +DO_SVE2_RRX_TB(SQDMULLT_zzx_d, gen_helper_sve2_sqdmull_idx_d, true) -DO_SVE2_RRX_TB(trans_SMULLB_zzx_s, gen_helper_sve2_smull_idx_s, false) -DO_SVE2_RRX_TB(trans_SMULLB_zzx_d, gen_helper_sve2_smull_idx_d, false) -DO_SVE2_RRX_TB(trans_SMULLT_zzx_s, gen_helper_sve2_smull_idx_s, true) -DO_SVE2_RRX_TB(trans_SMULLT_zzx_d, gen_helper_sve2_smull_idx_d, true) +DO_SVE2_RRX_TB(SMULLB_zzx_s, gen_helper_sve2_smull_idx_s, false) +DO_SVE2_RRX_TB(SMULLB_zzx_d, gen_helper_sve2_smull_idx_d, false) +DO_SVE2_RRX_TB(SMULLT_zzx_s, gen_helper_sve2_smull_idx_s, true) +DO_SVE2_RRX_TB(SMULLT_zzx_d, gen_helper_sve2_smull_idx_d, true) -DO_SVE2_RRX_TB(trans_UMULLB_zzx_s, gen_helper_sve2_umull_idx_s, false) -DO_SVE2_RRX_TB(trans_UMULLB_zzx_d, gen_helper_sve2_umull_idx_d, false) -DO_SVE2_RRX_TB(trans_UMULLT_zzx_s, gen_helper_sve2_umull_idx_s, true) -DO_SVE2_RRX_TB(trans_UMULLT_zzx_d, gen_helper_sve2_umull_idx_d, true) +DO_SVE2_RRX_TB(UMULLB_zzx_s, gen_helper_sve2_umull_idx_s, false) +DO_SVE2_RRX_TB(UMULLB_zzx_d, gen_helper_sve2_umull_idx_d, false) +DO_SVE2_RRX_TB(UMULLT_zzx_s, gen_helper_sve2_umull_idx_s, true) +DO_SVE2_RRX_TB(UMULLT_zzx_d, gen_helper_sve2_umull_idx_d, true) #undef DO_SVE2_RRX_TB -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Convert SVE translation functions using do_sve2_zzzz_data to use TRANS_FEAT and gen_gvec_ool_{zzzz,zzxz}. Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-17-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 106 ++++++++++++++----------------------- 1 file changed, 41 insertions(+), 65 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ DO_SVE2_RRX_TB(UMULLT_zzx_d, gen_helper_sve2_umull_idx_d, true) #undef DO_SVE2_RRX_TB -static bool do_sve2_zzzz_data(DisasContext *s, int rd, int rn, int rm, int ra, - int data, gen_helper_gvec_4 *fn) -{ - if (fn == NULL || !dc_isar_feature(aa64_sve2, s)) { - return false; - } - if (sve_access_check(s)) { - unsigned vsz = vec_full_reg_size(s); - tcg_gen_gvec_4_ool(vec_full_reg_offset(s, rd), - vec_full_reg_offset(s, rn), - vec_full_reg_offset(s, rm), - vec_full_reg_offset(s, ra), - vsz, vsz, data, fn); - } - return true; -} - #define DO_SVE2_RRXR(NAME, FUNC) \ - static bool NAME(DisasContext *s, arg_rrxr_esz *a) \ - { return do_sve2_zzzz_data(s, a->rd, a->rn, a->rm, a->ra, a->index, FUNC); } + TRANS_FEAT(NAME, aa64_sve2, gen_gvec_ool_arg_zzxz, FUNC, a) -DO_SVE2_RRXR(trans_MLA_zzxz_h, gen_helper_gvec_mla_idx_h) -DO_SVE2_RRXR(trans_MLA_zzxz_s, gen_helper_gvec_mla_idx_s) -DO_SVE2_RRXR(trans_MLA_zzxz_d, gen_helper_gvec_mla_idx_d) +DO_SVE2_RRXR(MLA_zzxz_h, gen_helper_gvec_mla_idx_h) +DO_SVE2_RRXR(MLA_zzxz_s, gen_helper_gvec_mla_idx_s) +DO_SVE2_RRXR(MLA_zzxz_d, gen_helper_gvec_mla_idx_d) -DO_SVE2_RRXR(trans_MLS_zzxz_h, gen_helper_gvec_mls_idx_h) -DO_SVE2_RRXR(trans_MLS_zzxz_s, gen_helper_gvec_mls_idx_s) -DO_SVE2_RRXR(trans_MLS_zzxz_d, gen_helper_gvec_mls_idx_d) +DO_SVE2_RRXR(MLS_zzxz_h, gen_helper_gvec_mls_idx_h) +DO_SVE2_RRXR(MLS_zzxz_s, gen_helper_gvec_mls_idx_s) +DO_SVE2_RRXR(MLS_zzxz_d, gen_helper_gvec_mls_idx_d) -DO_SVE2_RRXR(trans_SQRDMLAH_zzxz_h, gen_helper_sve2_sqrdmlah_idx_h) -DO_SVE2_RRXR(trans_SQRDMLAH_zzxz_s, gen_helper_sve2_sqrdmlah_idx_s) -DO_SVE2_RRXR(trans_SQRDMLAH_zzxz_d, gen_helper_sve2_sqrdmlah_idx_d) +DO_SVE2_RRXR(SQRDMLAH_zzxz_h, gen_helper_sve2_sqrdmlah_idx_h) +DO_SVE2_RRXR(SQRDMLAH_zzxz_s, gen_helper_sve2_sqrdmlah_idx_s) +DO_SVE2_RRXR(SQRDMLAH_zzxz_d, gen_helper_sve2_sqrdmlah_idx_d) -DO_SVE2_RRXR(trans_SQRDMLSH_zzxz_h, gen_helper_sve2_sqrdmlsh_idx_h) -DO_SVE2_RRXR(trans_SQRDMLSH_zzxz_s, gen_helper_sve2_sqrdmlsh_idx_s) -DO_SVE2_RRXR(trans_SQRDMLSH_zzxz_d, gen_helper_sve2_sqrdmlsh_idx_d) +DO_SVE2_RRXR(SQRDMLSH_zzxz_h, gen_helper_sve2_sqrdmlsh_idx_h) +DO_SVE2_RRXR(SQRDMLSH_zzxz_s, gen_helper_sve2_sqrdmlsh_idx_s) +DO_SVE2_RRXR(SQRDMLSH_zzxz_d, gen_helper_sve2_sqrdmlsh_idx_d) #undef DO_SVE2_RRXR #define DO_SVE2_RRXR_TB(NAME, FUNC, TOP) \ - static bool NAME(DisasContext *s, arg_rrxr_esz *a) \ - { \ - return do_sve2_zzzz_data(s, a->rd, a->rn, a->rm, a->rd, \ - (a->index << 1) | TOP, FUNC); \ - } + TRANS_FEAT(NAME, aa64_sve2, gen_gvec_ool_zzzz, FUNC, \ + a->rd, a->rn, a->rm, a->ra, (a->index << 1) | TOP) -DO_SVE2_RRXR_TB(trans_SQDMLALB_zzxw_s, gen_helper_sve2_sqdmlal_idx_s, false) -DO_SVE2_RRXR_TB(trans_SQDMLALB_zzxw_d, gen_helper_sve2_sqdmlal_idx_d, false) -DO_SVE2_RRXR_TB(trans_SQDMLALT_zzxw_s, gen_helper_sve2_sqdmlal_idx_s, true) -DO_SVE2_RRXR_TB(trans_SQDMLALT_zzxw_d, gen_helper_sve2_sqdmlal_idx_d, true) +DO_SVE2_RRXR_TB(SQDMLALB_zzxw_s, gen_helper_sve2_sqdmlal_idx_s, false) +DO_SVE2_RRXR_TB(SQDMLALB_zzxw_d, gen_helper_sve2_sqdmlal_idx_d, false) +DO_SVE2_RRXR_TB(SQDMLALT_zzxw_s, gen_helper_sve2_sqdmlal_idx_s, true) +DO_SVE2_RRXR_TB(SQDMLALT_zzxw_d, gen_helper_sve2_sqdmlal_idx_d, true) -DO_SVE2_RRXR_TB(trans_SQDMLSLB_zzxw_s, gen_helper_sve2_sqdmlsl_idx_s, false) -DO_SVE2_RRXR_TB(trans_SQDMLSLB_zzxw_d, gen_helper_sve2_sqdmlsl_idx_d, false) -DO_SVE2_RRXR_TB(trans_SQDMLSLT_zzxw_s, gen_helper_sve2_sqdmlsl_idx_s, true) -DO_SVE2_RRXR_TB(trans_SQDMLSLT_zzxw_d, gen_helper_sve2_sqdmlsl_idx_d, true) +DO_SVE2_RRXR_TB(SQDMLSLB_zzxw_s, gen_helper_sve2_sqdmlsl_idx_s, false) +DO_SVE2_RRXR_TB(SQDMLSLB_zzxw_d, gen_helper_sve2_sqdmlsl_idx_d, false) +DO_SVE2_RRXR_TB(SQDMLSLT_zzxw_s, gen_helper_sve2_sqdmlsl_idx_s, true) +DO_SVE2_RRXR_TB(SQDMLSLT_zzxw_d, gen_helper_sve2_sqdmlsl_idx_d, true) -DO_SVE2_RRXR_TB(trans_SMLALB_zzxw_s, gen_helper_sve2_smlal_idx_s, false) -DO_SVE2_RRXR_TB(trans_SMLALB_zzxw_d, gen_helper_sve2_smlal_idx_d, false) -DO_SVE2_RRXR_TB(trans_SMLALT_zzxw_s, gen_helper_sve2_smlal_idx_s, true) -DO_SVE2_RRXR_TB(trans_SMLALT_zzxw_d, gen_helper_sve2_smlal_idx_d, true) +DO_SVE2_RRXR_TB(SMLALB_zzxw_s, gen_helper_sve2_smlal_idx_s, false) +DO_SVE2_RRXR_TB(SMLALB_zzxw_d, gen_helper_sve2_smlal_idx_d, false) +DO_SVE2_RRXR_TB(SMLALT_zzxw_s, gen_helper_sve2_smlal_idx_s, true) +DO_SVE2_RRXR_TB(SMLALT_zzxw_d, gen_helper_sve2_smlal_idx_d, true) -DO_SVE2_RRXR_TB(trans_UMLALB_zzxw_s, gen_helper_sve2_umlal_idx_s, false) -DO_SVE2_RRXR_TB(trans_UMLALB_zzxw_d, gen_helper_sve2_umlal_idx_d, false) -DO_SVE2_RRXR_TB(trans_UMLALT_zzxw_s, gen_helper_sve2_umlal_idx_s, true) -DO_SVE2_RRXR_TB(trans_UMLALT_zzxw_d, gen_helper_sve2_umlal_idx_d, true) +DO_SVE2_RRXR_TB(UMLALB_zzxw_s, gen_helper_sve2_umlal_idx_s, false) +DO_SVE2_RRXR_TB(UMLALB_zzxw_d, gen_helper_sve2_umlal_idx_d, false) +DO_SVE2_RRXR_TB(UMLALT_zzxw_s, gen_helper_sve2_umlal_idx_s, true) +DO_SVE2_RRXR_TB(UMLALT_zzxw_d, gen_helper_sve2_umlal_idx_d, true) -DO_SVE2_RRXR_TB(trans_SMLSLB_zzxw_s, gen_helper_sve2_smlsl_idx_s, false) -DO_SVE2_RRXR_TB(trans_SMLSLB_zzxw_d, gen_helper_sve2_smlsl_idx_d, false) -DO_SVE2_RRXR_TB(trans_SMLSLT_zzxw_s, gen_helper_sve2_smlsl_idx_s, true) -DO_SVE2_RRXR_TB(trans_SMLSLT_zzxw_d, gen_helper_sve2_smlsl_idx_d, true) +DO_SVE2_RRXR_TB(SMLSLB_zzxw_s, gen_helper_sve2_smlsl_idx_s, false) +DO_SVE2_RRXR_TB(SMLSLB_zzxw_d, gen_helper_sve2_smlsl_idx_d, false) +DO_SVE2_RRXR_TB(SMLSLT_zzxw_s, gen_helper_sve2_smlsl_idx_s, true) +DO_SVE2_RRXR_TB(SMLSLT_zzxw_d, gen_helper_sve2_smlsl_idx_d, true) -DO_SVE2_RRXR_TB(trans_UMLSLB_zzxw_s, gen_helper_sve2_umlsl_idx_s, false) -DO_SVE2_RRXR_TB(trans_UMLSLB_zzxw_d, gen_helper_sve2_umlsl_idx_d, false) -DO_SVE2_RRXR_TB(trans_UMLSLT_zzxw_s, gen_helper_sve2_umlsl_idx_s, true) -DO_SVE2_RRXR_TB(trans_UMLSLT_zzxw_d, gen_helper_sve2_umlsl_idx_d, true) +DO_SVE2_RRXR_TB(UMLSLB_zzxw_s, gen_helper_sve2_umlsl_idx_s, false) +DO_SVE2_RRXR_TB(UMLSLB_zzxw_d, gen_helper_sve2_umlsl_idx_d, false) +DO_SVE2_RRXR_TB(UMLSLT_zzxw_s, gen_helper_sve2_umlsl_idx_s, true) +DO_SVE2_RRXR_TB(UMLSLT_zzxw_d, gen_helper_sve2_umlsl_idx_d, true) #undef DO_SVE2_RRXR_TB #define DO_SVE2_RRXR_ROT(NAME, FUNC) \ - static bool trans_##NAME(DisasContext *s, arg_##NAME *a) \ - { \ - return do_sve2_zzzz_data(s, a->rd, a->rn, a->rm, a->ra, \ - (a->index << 2) | a->rot, FUNC); \ - } + TRANS_FEAT(NAME, aa64_sve2, gen_gvec_ool_zzzz, FUNC, \ + a->rd, a->rn, a->rm, a->ra, (a->index << 2) | a->rot) DO_SVE2_RRXR_ROT(CMLA_zzxz_h, gen_helper_sve2_cmla_idx_h) DO_SVE2_RRXR_ROT(CMLA_zzxz_s, gen_helper_sve2_cmla_idx_s) -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Convert SVE translation functions using do_sve2_zzw_data to use TRANS_FEAT and gen_gvec_ool_arg_zzz. Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-18-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 297 ++++++++++++++++++------------------- 1 file changed, 145 insertions(+), 152 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ DO_SVE2_ZPZZ(USQADD, usqadd) * SVE2 Widening Integer Arithmetic */ -static bool do_sve2_zzw_ool(DisasContext *s, arg_rrr_esz *a, - gen_helper_gvec_3 *fn, int data) -{ - if (fn == NULL || !dc_isar_feature(aa64_sve2, s)) { - return false; - } - if (sve_access_check(s)) { - unsigned vsz = vec_full_reg_size(s); - tcg_gen_gvec_3_ool(vec_full_reg_offset(s, a->rd), - vec_full_reg_offset(s, a->rn), - vec_full_reg_offset(s, a->rm), - vsz, vsz, data, fn); - } - return true; -} +static gen_helper_gvec_3 * const saddl_fns[4] = { + NULL, gen_helper_sve2_saddl_h, + gen_helper_sve2_saddl_s, gen_helper_sve2_saddl_d, +}; +TRANS_FEAT(SADDLB, aa64_sve2, gen_gvec_ool_arg_zzz, + saddl_fns[a->esz], a, 0) +TRANS_FEAT(SADDLT, aa64_sve2, gen_gvec_ool_arg_zzz, + saddl_fns[a->esz], a, 3) +TRANS_FEAT(SADDLBT, aa64_sve2, gen_gvec_ool_arg_zzz, + saddl_fns[a->esz], a, 2) -#define DO_SVE2_ZZZ_TB(NAME, name, SEL1, SEL2) \ -static bool trans_##NAME(DisasContext *s, arg_rrr_esz *a) \ -{ \ - static gen_helper_gvec_3 * const fns[4] = { \ - NULL, gen_helper_sve2_##name##_h, \ - gen_helper_sve2_##name##_s, gen_helper_sve2_##name##_d, \ - }; \ - return do_sve2_zzw_ool(s, a, fns[a->esz], (SEL2 << 1) | SEL1); \ -} +static gen_helper_gvec_3 * const ssubl_fns[4] = { + NULL, gen_helper_sve2_ssubl_h, + gen_helper_sve2_ssubl_s, gen_helper_sve2_ssubl_d, +}; +TRANS_FEAT(SSUBLB, aa64_sve2, gen_gvec_ool_arg_zzz, + ssubl_fns[a->esz], a, 0) +TRANS_FEAT(SSUBLT, aa64_sve2, gen_gvec_ool_arg_zzz, + ssubl_fns[a->esz], a, 3) +TRANS_FEAT(SSUBLBT, aa64_sve2, gen_gvec_ool_arg_zzz, + ssubl_fns[a->esz], a, 2) +TRANS_FEAT(SSUBLTB, aa64_sve2, gen_gvec_ool_arg_zzz, + ssubl_fns[a->esz], a, 1) -DO_SVE2_ZZZ_TB(SADDLB, saddl, false, false) -DO_SVE2_ZZZ_TB(SSUBLB, ssubl, false, false) -DO_SVE2_ZZZ_TB(SABDLB, sabdl, false, false) +static gen_helper_gvec_3 * const sabdl_fns[4] = { + NULL, gen_helper_sve2_sabdl_h, + gen_helper_sve2_sabdl_s, gen_helper_sve2_sabdl_d, +}; +TRANS_FEAT(SABDLB, aa64_sve2, gen_gvec_ool_arg_zzz, + sabdl_fns[a->esz], a, 0) +TRANS_FEAT(SABDLT, aa64_sve2, gen_gvec_ool_arg_zzz, + sabdl_fns[a->esz], a, 3) -DO_SVE2_ZZZ_TB(UADDLB, uaddl, false, false) -DO_SVE2_ZZZ_TB(USUBLB, usubl, false, false) -DO_SVE2_ZZZ_TB(UABDLB, uabdl, false, false) +static gen_helper_gvec_3 * const uaddl_fns[4] = { + NULL, gen_helper_sve2_uaddl_h, + gen_helper_sve2_uaddl_s, gen_helper_sve2_uaddl_d, +}; +TRANS_FEAT(UADDLB, aa64_sve2, gen_gvec_ool_arg_zzz, + uaddl_fns[a->esz], a, 0) +TRANS_FEAT(UADDLT, aa64_sve2, gen_gvec_ool_arg_zzz, + uaddl_fns[a->esz], a, 3) -DO_SVE2_ZZZ_TB(SADDLT, saddl, true, true) -DO_SVE2_ZZZ_TB(SSUBLT, ssubl, true, true) -DO_SVE2_ZZZ_TB(SABDLT, sabdl, true, true) +static gen_helper_gvec_3 * const usubl_fns[4] = { + NULL, gen_helper_sve2_usubl_h, + gen_helper_sve2_usubl_s, gen_helper_sve2_usubl_d, +}; +TRANS_FEAT(USUBLB, aa64_sve2, gen_gvec_ool_arg_zzz, + usubl_fns[a->esz], a, 0) +TRANS_FEAT(USUBLT, aa64_sve2, gen_gvec_ool_arg_zzz, + usubl_fns[a->esz], a, 3) -DO_SVE2_ZZZ_TB(UADDLT, uaddl, true, true) -DO_SVE2_ZZZ_TB(USUBLT, usubl, true, true) -DO_SVE2_ZZZ_TB(UABDLT, uabdl, true, true) +static gen_helper_gvec_3 * const uabdl_fns[4] = { + NULL, gen_helper_sve2_uabdl_h, + gen_helper_sve2_uabdl_s, gen_helper_sve2_uabdl_d, +}; +TRANS_FEAT(UABDLB, aa64_sve2, gen_gvec_ool_arg_zzz, + uabdl_fns[a->esz], a, 0) +TRANS_FEAT(UABDLT, aa64_sve2, gen_gvec_ool_arg_zzz, + uabdl_fns[a->esz], a, 3) -DO_SVE2_ZZZ_TB(SADDLBT, saddl, false, true) -DO_SVE2_ZZZ_TB(SSUBLBT, ssubl, false, true) -DO_SVE2_ZZZ_TB(SSUBLTB, ssubl, true, false) +static gen_helper_gvec_3 * const sqdmull_fns[4] = { + NULL, gen_helper_sve2_sqdmull_zzz_h, + gen_helper_sve2_sqdmull_zzz_s, gen_helper_sve2_sqdmull_zzz_d, +}; +TRANS_FEAT(SQDMULLB_zzz, aa64_sve2, gen_gvec_ool_arg_zzz, + sqdmull_fns[a->esz], a, 0) +TRANS_FEAT(SQDMULLT_zzz, aa64_sve2, gen_gvec_ool_arg_zzz, + sqdmull_fns[a->esz], a, 3) -DO_SVE2_ZZZ_TB(SQDMULLB_zzz, sqdmull_zzz, false, false) -DO_SVE2_ZZZ_TB(SQDMULLT_zzz, sqdmull_zzz, true, true) +static gen_helper_gvec_3 * const smull_fns[4] = { + NULL, gen_helper_sve2_smull_zzz_h, + gen_helper_sve2_smull_zzz_s, gen_helper_sve2_smull_zzz_d, +}; +TRANS_FEAT(SMULLB_zzz, aa64_sve2, gen_gvec_ool_arg_zzz, + smull_fns[a->esz], a, 0) +TRANS_FEAT(SMULLT_zzz, aa64_sve2, gen_gvec_ool_arg_zzz, + smull_fns[a->esz], a, 3) -DO_SVE2_ZZZ_TB(SMULLB_zzz, smull_zzz, false, false) -DO_SVE2_ZZZ_TB(SMULLT_zzz, smull_zzz, true, true) +static gen_helper_gvec_3 * const umull_fns[4] = { + NULL, gen_helper_sve2_umull_zzz_h, + gen_helper_sve2_umull_zzz_s, gen_helper_sve2_umull_zzz_d, +}; +TRANS_FEAT(UMULLB_zzz, aa64_sve2, gen_gvec_ool_arg_zzz, + umull_fns[a->esz], a, 0) +TRANS_FEAT(UMULLT_zzz, aa64_sve2, gen_gvec_ool_arg_zzz, + umull_fns[a->esz], a, 3) -DO_SVE2_ZZZ_TB(UMULLB_zzz, umull_zzz, false, false) -DO_SVE2_ZZZ_TB(UMULLT_zzz, umull_zzz, true, true) - -static bool do_eor_tb(DisasContext *s, arg_rrr_esz *a, bool sel1) -{ - static gen_helper_gvec_3 * const fns[4] = { - gen_helper_sve2_eoril_b, gen_helper_sve2_eoril_h, - gen_helper_sve2_eoril_s, gen_helper_sve2_eoril_d, - }; - return do_sve2_zzw_ool(s, a, fns[a->esz], (!sel1 << 1) | sel1); -} - -static bool trans_EORBT(DisasContext *s, arg_rrr_esz *a) -{ - return do_eor_tb(s, a, false); -} - -static bool trans_EORTB(DisasContext *s, arg_rrr_esz *a) -{ - return do_eor_tb(s, a, true); -} +static gen_helper_gvec_3 * const eoril_fns[4] = { + gen_helper_sve2_eoril_b, gen_helper_sve2_eoril_h, + gen_helper_sve2_eoril_s, gen_helper_sve2_eoril_d, +}; +TRANS_FEAT(EORBT, aa64_sve2, gen_gvec_ool_arg_zzz, eoril_fns[a->esz], a, 2) +TRANS_FEAT(EORTB, aa64_sve2, gen_gvec_ool_arg_zzz, eoril_fns[a->esz], a, 1) static bool do_trans_pmull(DisasContext *s, arg_rrr_esz *a, bool sel) { @@ -XXX,XX +XXX,XX @@ static bool do_trans_pmull(DisasContext *s, arg_rrr_esz *a, bool sel) if (a->esz == 0 && !dc_isar_feature(aa64_sve2_pmull128, s)) { return false; } - return do_sve2_zzw_ool(s, a, fns[a->esz], sel); + return gen_gvec_ool_arg_zzz(s, fns[a->esz], a, sel); } -static bool trans_PMULLB(DisasContext *s, arg_rrr_esz *a) -{ - return do_trans_pmull(s, a, false); -} +TRANS_FEAT(PMULLB, aa64_sve2, do_trans_pmull, a, false) +TRANS_FEAT(PMULLT, aa64_sve2, do_trans_pmull, a, true) -static bool trans_PMULLT(DisasContext *s, arg_rrr_esz *a) -{ - return do_trans_pmull(s, a, true); -} +static gen_helper_gvec_3 * const saddw_fns[4] = { + NULL, gen_helper_sve2_saddw_h, + gen_helper_sve2_saddw_s, gen_helper_sve2_saddw_d, +}; +TRANS_FEAT(SADDWB, aa64_sve2, gen_gvec_ool_arg_zzz, saddw_fns[a->esz], a, 0) +TRANS_FEAT(SADDWT, aa64_sve2, gen_gvec_ool_arg_zzz, saddw_fns[a->esz], a, 1) -#define DO_SVE2_ZZZ_WTB(NAME, name, SEL2) \ -static bool trans_##NAME(DisasContext *s, arg_rrr_esz *a) \ -{ \ - static gen_helper_gvec_3 * const fns[4] = { \ - NULL, gen_helper_sve2_##name##_h, \ - gen_helper_sve2_##name##_s, gen_helper_sve2_##name##_d, \ - }; \ - return do_sve2_zzw_ool(s, a, fns[a->esz], SEL2); \ -} +static gen_helper_gvec_3 * const ssubw_fns[4] = { + NULL, gen_helper_sve2_ssubw_h, + gen_helper_sve2_ssubw_s, gen_helper_sve2_ssubw_d, +}; +TRANS_FEAT(SSUBWB, aa64_sve2, gen_gvec_ool_arg_zzz, ssubw_fns[a->esz], a, 0) +TRANS_FEAT(SSUBWT, aa64_sve2, gen_gvec_ool_arg_zzz, ssubw_fns[a->esz], a, 1) -DO_SVE2_ZZZ_WTB(SADDWB, saddw, false) -DO_SVE2_ZZZ_WTB(SADDWT, saddw, true) -DO_SVE2_ZZZ_WTB(SSUBWB, ssubw, false) -DO_SVE2_ZZZ_WTB(SSUBWT, ssubw, true) +static gen_helper_gvec_3 * const uaddw_fns[4] = { + NULL, gen_helper_sve2_uaddw_h, + gen_helper_sve2_uaddw_s, gen_helper_sve2_uaddw_d, +}; +TRANS_FEAT(UADDWB, aa64_sve2, gen_gvec_ool_arg_zzz, uaddw_fns[a->esz], a, 0) +TRANS_FEAT(UADDWT, aa64_sve2, gen_gvec_ool_arg_zzz, uaddw_fns[a->esz], a, 1) -DO_SVE2_ZZZ_WTB(UADDWB, uaddw, false) -DO_SVE2_ZZZ_WTB(UADDWT, uaddw, true) -DO_SVE2_ZZZ_WTB(USUBWB, usubw, false) -DO_SVE2_ZZZ_WTB(USUBWT, usubw, true) +static gen_helper_gvec_3 * const usubw_fns[4] = { + NULL, gen_helper_sve2_usubw_h, + gen_helper_sve2_usubw_s, gen_helper_sve2_usubw_d, +}; +TRANS_FEAT(USUBWB, aa64_sve2, gen_gvec_ool_arg_zzz, usubw_fns[a->esz], a, 0) +TRANS_FEAT(USUBWT, aa64_sve2, gen_gvec_ool_arg_zzz, usubw_fns[a->esz], a, 1) static void gen_sshll_vec(unsigned vece, TCGv_vec d, TCGv_vec n, int64_t imm) { @@ -XXX,XX +XXX,XX @@ static bool trans_USHLLT(DisasContext *s, arg_rri_esz *a) return do_sve2_shll_tb(s, a, true, true); } -static bool trans_BEXT(DisasContext *s, arg_rrr_esz *a) -{ - static gen_helper_gvec_3 * const fns[4] = { - gen_helper_sve2_bext_b, gen_helper_sve2_bext_h, - gen_helper_sve2_bext_s, gen_helper_sve2_bext_d, - }; - if (!dc_isar_feature(aa64_sve2_bitperm, s)) { - return false; - } - return do_sve2_zzw_ool(s, a, fns[a->esz], 0); -} +static gen_helper_gvec_3 * const bext_fns[4] = { + gen_helper_sve2_bext_b, gen_helper_sve2_bext_h, + gen_helper_sve2_bext_s, gen_helper_sve2_bext_d, +}; +TRANS_FEAT(BEXT, aa64_sve2_bitperm, gen_gvec_ool_arg_zzz, + bext_fns[a->esz], a, 0) -static bool trans_BDEP(DisasContext *s, arg_rrr_esz *a) -{ - static gen_helper_gvec_3 * const fns[4] = { - gen_helper_sve2_bdep_b, gen_helper_sve2_bdep_h, - gen_helper_sve2_bdep_s, gen_helper_sve2_bdep_d, - }; - if (!dc_isar_feature(aa64_sve2_bitperm, s)) { - return false; - } - return do_sve2_zzw_ool(s, a, fns[a->esz], 0); -} +static gen_helper_gvec_3 * const bdep_fns[4] = { + gen_helper_sve2_bdep_b, gen_helper_sve2_bdep_h, + gen_helper_sve2_bdep_s, gen_helper_sve2_bdep_d, +}; +TRANS_FEAT(BDEP, aa64_sve2_bitperm, gen_gvec_ool_arg_zzz, + bdep_fns[a->esz], a, 0) -static bool trans_BGRP(DisasContext *s, arg_rrr_esz *a) -{ - static gen_helper_gvec_3 * const fns[4] = { - gen_helper_sve2_bgrp_b, gen_helper_sve2_bgrp_h, - gen_helper_sve2_bgrp_s, gen_helper_sve2_bgrp_d, - }; - if (!dc_isar_feature(aa64_sve2_bitperm, s)) { - return false; - } - return do_sve2_zzw_ool(s, a, fns[a->esz], 0); -} +static gen_helper_gvec_3 * const bgrp_fns[4] = { + gen_helper_sve2_bgrp_b, gen_helper_sve2_bgrp_h, + gen_helper_sve2_bgrp_s, gen_helper_sve2_bgrp_d, +}; +TRANS_FEAT(BGRP, aa64_sve2_bitperm, gen_gvec_ool_arg_zzz, + bgrp_fns[a->esz], a, 0) -static bool do_cadd(DisasContext *s, arg_rrr_esz *a, bool sq, bool rot) -{ - static gen_helper_gvec_3 * const fns[2][4] = { - { gen_helper_sve2_cadd_b, gen_helper_sve2_cadd_h, - gen_helper_sve2_cadd_s, gen_helper_sve2_cadd_d }, - { gen_helper_sve2_sqcadd_b, gen_helper_sve2_sqcadd_h, - gen_helper_sve2_sqcadd_s, gen_helper_sve2_sqcadd_d }, - }; - return do_sve2_zzw_ool(s, a, fns[sq][a->esz], rot); -} +static gen_helper_gvec_3 * const cadd_fns[4] = { + gen_helper_sve2_cadd_b, gen_helper_sve2_cadd_h, + gen_helper_sve2_cadd_s, gen_helper_sve2_cadd_d, +}; +TRANS_FEAT(CADD_rot90, aa64_sve2, gen_gvec_ool_arg_zzz, + cadd_fns[a->esz], a, 0) +TRANS_FEAT(CADD_rot270, aa64_sve2, gen_gvec_ool_arg_zzz, + cadd_fns[a->esz], a, 1) -static bool trans_CADD_rot90(DisasContext *s, arg_rrr_esz *a) -{ - return do_cadd(s, a, false, false); -} - -static bool trans_CADD_rot270(DisasContext *s, arg_rrr_esz *a) -{ - return do_cadd(s, a, false, true); -} - -static bool trans_SQCADD_rot90(DisasContext *s, arg_rrr_esz *a) -{ - return do_cadd(s, a, true, false); -} - -static bool trans_SQCADD_rot270(DisasContext *s, arg_rrr_esz *a) -{ - return do_cadd(s, a, true, true); -} +static gen_helper_gvec_3 * const sqcadd_fns[4] = { + gen_helper_sve2_sqcadd_b, gen_helper_sve2_sqcadd_h, + gen_helper_sve2_sqcadd_s, gen_helper_sve2_sqcadd_d, +}; +TRANS_FEAT(SQCADD_rot90, aa64_sve2, gen_gvec_ool_arg_zzz, + sqcadd_fns[a->esz], a, 0) +TRANS_FEAT(SQCADD_rot270, aa64_sve2, gen_gvec_ool_arg_zzz, + sqcadd_fns[a->esz], a, 1) static gen_helper_gvec_4 * const sabal_fns[4] = { NULL, gen_helper_sve2_sabal_h, -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> This is the last direct user of tcg_gen_gvec_4_ool. Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-19-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 17 ++--------------- 1 file changed, 2 insertions(+), 15 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static gen_helper_gvec_4 * const sqrdcmlah_fns[] = { TRANS_FEAT(SQRDCMLAH_zzzz, aa64_sve2, gen_gvec_ool_zzzz, sqrdcmlah_fns[a->esz], a->rd, a->rn, a->rm, a->ra, a->rot) -static bool trans_USDOT_zzzz(DisasContext *s, arg_USDOT_zzzz *a) -{ - if (a->esz != 2 || !dc_isar_feature(aa64_sve_i8mm, s)) { - return false; - } - if (sve_access_check(s)) { - unsigned vsz = vec_full_reg_size(s); - tcg_gen_gvec_4_ool(vec_full_reg_offset(s, a->rd), - vec_full_reg_offset(s, a->rn), - vec_full_reg_offset(s, a->rm), - vec_full_reg_offset(s, a->ra), - vsz, vsz, 0, gen_helper_gvec_usdot_b); - } - return true; -} +TRANS_FEAT(USDOT_zzzz, aa64_sve_i8mm, gen_gvec_ool_arg_zzzz, + a->esz == 2 ? gen_helper_gvec_usdot_b : NULL, a, 0) TRANS_FEAT(AESMC, aa64_sve2_aes, gen_gvec_ool_zz, gen_helper_crypto_aesmc, a->rd, a->rd, a->decrypt) -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-20-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 37 +++++++++++++++---------------------- 1 file changed, 15 insertions(+), 22 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool gen_gvec_ool_arg_zzxz(DisasContext *s, gen_helper_gvec_4 *fn, } /* Invoke an out-of-line helper on 2 Zregs and a predicate. */ -static void gen_gvec_ool_zzp(DisasContext *s, gen_helper_gvec_3 *fn, +static bool gen_gvec_ool_zzp(DisasContext *s, gen_helper_gvec_3 *fn, int rd, int rn, int pg, int data) { - unsigned vsz = vec_full_reg_size(s); - tcg_gen_gvec_3_ool(vec_full_reg_offset(s, rd), - vec_full_reg_offset(s, rn), - pred_full_reg_offset(s, pg), - vsz, vsz, data, fn); + if (fn == NULL) { + return false; + } + if (sve_access_check(s)) { + unsigned vsz = vec_full_reg_size(s); + tcg_gen_gvec_3_ool(vec_full_reg_offset(s, rd), + vec_full_reg_offset(s, rn), + pred_full_reg_offset(s, pg), + vsz, vsz, data, fn); + } + return true; } /* Invoke an out-of-line helper on 3 Zregs and a predicate. */ @@ -XXX,XX +XXX,XX @@ static bool trans_SEL_zpzz(DisasContext *s, arg_rprr_esz *a) static bool do_zpz_ool(DisasContext *s, arg_rpr_esz *a, gen_helper_gvec_3 *fn) { - if (fn == NULL) { - return false; - } - if (sve_access_check(s)) { - gen_gvec_ool_zzp(s, fn, a->rd, a->rn, a->pg, 0); - } - return true; + return gen_gvec_ool_zzp(s, fn, a->rd, a->rn, a->pg, 0); } #define DO_ZPZ(NAME, name) \ @@ -XXX,XX +XXX,XX @@ static bool do_movz_zpz(DisasContext *s, int rd, int rn, int pg, gen_helper_sve_movz_b, gen_helper_sve_movz_h, gen_helper_sve_movz_s, gen_helper_sve_movz_d, }; - - if (sve_access_check(s)) { - gen_gvec_ool_zzp(s, fns[esz], rd, rn, pg, invert); - } - return true; + return gen_gvec_ool_zzp(s, fns[esz], rd, rn, pg, invert); } static bool do_zpzi_ool(DisasContext *s, arg_rpri_esz *a, gen_helper_gvec_3 *fn) { - if (sve_access_check(s)) { - gen_gvec_ool_zzp(s, fn, a->rd, a->rn, a->pg, a->imm); - } - return true; + return gen_gvec_ool_zzp(s, fn, a->rd, a->rn, a->pg, a->imm); } static bool trans_ASR_zpzi(DisasContext *s, arg_rpri_esz *a) -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Use gen_gvec_ool_arg_zpz instead of gen_gvec_ool_zzp when the arguments come from arg_rpr_esz. Replaces do_zpz_ool. Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-21-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 45 +++++++++++++++++++++----------------- 1 file changed, 25 insertions(+), 20 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool gen_gvec_ool_zzp(DisasContext *s, gen_helper_gvec_3 *fn, return true; } +static bool gen_gvec_ool_arg_zpz(DisasContext *s, gen_helper_gvec_3 *fn, + arg_rpr_esz *a, int data) +{ + return gen_gvec_ool_zzp(s, fn, a->rd, a->rn, a->pg, data); +} + + /* Invoke an out-of-line helper on 3 Zregs and a predicate. */ static void gen_gvec_ool_zzzp(DisasContext *s, gen_helper_gvec_4 *fn, int rd, int rn, int rm, int pg, int data) @@ -XXX,XX +XXX,XX @@ static bool trans_SEL_zpzz(DisasContext *s, arg_rprr_esz *a) *** SVE Integer Arithmetic - Unary Predicated Group */ -static bool do_zpz_ool(DisasContext *s, arg_rpr_esz *a, gen_helper_gvec_3 *fn) -{ - return gen_gvec_ool_zzp(s, fn, a->rd, a->rn, a->pg, 0); -} - #define DO_ZPZ(NAME, name) \ static bool trans_##NAME(DisasContext *s, arg_rpr_esz *a) \ { \ @@ -XXX,XX +XXX,XX @@ static bool trans_##NAME(DisasContext *s, arg_rpr_esz *a) \ gen_helper_sve_##name##_b, gen_helper_sve_##name##_h, \ gen_helper_sve_##name##_s, gen_helper_sve_##name##_d, \ }; \ - return do_zpz_ool(s, a, fns[a->esz]); \ + return gen_gvec_ool_arg_zpz(s, fns[a->esz], a, 0); \ } DO_ZPZ(CLS, cls) @@ -XXX,XX +XXX,XX @@ static bool trans_FABS(DisasContext *s, arg_rpr_esz *a) gen_helper_sve_fabs_s, gen_helper_sve_fabs_d }; - return do_zpz_ool(s, a, fns[a->esz]); + return gen_gvec_ool_arg_zpz(s, fns[a->esz], a, 0); } static bool trans_FNEG(DisasContext *s, arg_rpr_esz *a) @@ -XXX,XX +XXX,XX @@ static bool trans_FNEG(DisasContext *s, arg_rpr_esz *a) gen_helper_sve_fneg_s, gen_helper_sve_fneg_d }; - return do_zpz_ool(s, a, fns[a->esz]); + return gen_gvec_ool_arg_zpz(s, fns[a->esz], a, 0); } static bool trans_SXTB(DisasContext *s, arg_rpr_esz *a) @@ -XXX,XX +XXX,XX @@ static bool trans_SXTB(DisasContext *s, arg_rpr_esz *a) gen_helper_sve_sxtb_s, gen_helper_sve_sxtb_d }; - return do_zpz_ool(s, a, fns[a->esz]); + return gen_gvec_ool_arg_zpz(s, fns[a->esz], a, 0); } static bool trans_UXTB(DisasContext *s, arg_rpr_esz *a) @@ -XXX,XX +XXX,XX @@ static bool trans_UXTB(DisasContext *s, arg_rpr_esz *a) gen_helper_sve_uxtb_s, gen_helper_sve_uxtb_d }; - return do_zpz_ool(s, a, fns[a->esz]); + return gen_gvec_ool_arg_zpz(s, fns[a->esz], a, 0); } static bool trans_SXTH(DisasContext *s, arg_rpr_esz *a) @@ -XXX,XX +XXX,XX @@ static bool trans_SXTH(DisasContext *s, arg_rpr_esz *a) gen_helper_sve_sxth_s, gen_helper_sve_sxth_d }; - return do_zpz_ool(s, a, fns[a->esz]); + return gen_gvec_ool_arg_zpz(s, fns[a->esz], a, 0); } static bool trans_UXTH(DisasContext *s, arg_rpr_esz *a) @@ -XXX,XX +XXX,XX @@ static bool trans_UXTH(DisasContext *s, arg_rpr_esz *a) gen_helper_sve_uxth_s, gen_helper_sve_uxth_d }; - return do_zpz_ool(s, a, fns[a->esz]); + return gen_gvec_ool_arg_zpz(s, fns[a->esz], a, 0); } static bool trans_SXTW(DisasContext *s, arg_rpr_esz *a) { - return do_zpz_ool(s, a, a->esz == 3 ? gen_helper_sve_sxtw_d : NULL); + return gen_gvec_ool_arg_zpz(s, a->esz == 3 ? gen_helper_sve_sxtw_d + : NULL, a, 0); } static bool trans_UXTW(DisasContext *s, arg_rpr_esz *a) { - return do_zpz_ool(s, a, a->esz == 3 ? gen_helper_sve_uxtw_d : NULL); + return gen_gvec_ool_arg_zpz(s, a->esz == 3 ? gen_helper_sve_uxtw_d + : NULL, a, 0); } #undef DO_ZPZ @@ -XXX,XX +XXX,XX @@ static bool trans_COMPACT(DisasContext *s, arg_rpr_esz *a) static gen_helper_gvec_3 * const fns[4] = { NULL, NULL, gen_helper_sve_compact_s, gen_helper_sve_compact_d }; - return do_zpz_ool(s, a, fns[a->esz]); + return gen_gvec_ool_arg_zpz(s, fns[a->esz], a, 0); } /* Call the helper that computes the ARM LastActiveElement pseudocode @@ -XXX,XX +XXX,XX @@ static bool trans_REVB(DisasContext *s, arg_rpr_esz *a) gen_helper_sve_revb_s, gen_helper_sve_revb_d, }; - return do_zpz_ool(s, a, fns[a->esz]); + return gen_gvec_ool_arg_zpz(s, fns[a->esz], a, 0); } static bool trans_REVH(DisasContext *s, arg_rpr_esz *a) @@ -XXX,XX +XXX,XX @@ static bool trans_REVH(DisasContext *s, arg_rpr_esz *a) gen_helper_sve_revh_s, gen_helper_sve_revh_d, }; - return do_zpz_ool(s, a, fns[a->esz]); + return gen_gvec_ool_arg_zpz(s, fns[a->esz], a, 0); } static bool trans_REVW(DisasContext *s, arg_rpr_esz *a) { - return do_zpz_ool(s, a, a->esz == 3 ? gen_helper_sve_revw_d : NULL); + return gen_gvec_ool_arg_zpz(s, a->esz == 3 ? gen_helper_sve_revw_d + : NULL, a, 0); } static bool trans_RBIT(DisasContext *s, arg_rpr_esz *a) @@ -XXX,XX +XXX,XX @@ static bool trans_RBIT(DisasContext *s, arg_rpr_esz *a) gen_helper_sve_rbit_s, gen_helper_sve_rbit_d, }; - return do_zpz_ool(s, a, fns[a->esz]); + return gen_gvec_ool_arg_zpz(s, fns[a->esz], a, 0); } static bool trans_SPLICE(DisasContext *s, arg_rprr_esz *a) @@ -XXX,XX +XXX,XX @@ static bool do_sve2_zpz_ool(DisasContext *s, arg_rpr_esz *a, if (!dc_isar_feature(aa64_sve2, s)) { return false; } - return do_zpz_ool(s, a, fn); + return gen_gvec_ool_arg_zpz(s, fn, a, 0); } static bool trans_URECPE(DisasContext *s, arg_rpr_esz *a) -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Convert SVE translation functions directly using gen_gvec_ool_arg_zpz to TRANS_FEAT. Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-22-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 189 ++++++++++++------------------------- 1 file changed, 60 insertions(+), 129 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool trans_SEL_zpzz(DisasContext *s, arg_rprr_esz *a) *** SVE Integer Arithmetic - Unary Predicated Group */ -#define DO_ZPZ(NAME, name) \ -static bool trans_##NAME(DisasContext *s, arg_rpr_esz *a) \ -{ \ - static gen_helper_gvec_3 * const fns[4] = { \ - gen_helper_sve_##name##_b, gen_helper_sve_##name##_h, \ - gen_helper_sve_##name##_s, gen_helper_sve_##name##_d, \ +#define DO_ZPZ(NAME, FEAT, name) \ + static gen_helper_gvec_3 * const name##_fns[4] = { \ + gen_helper_##name##_b, gen_helper_##name##_h, \ + gen_helper_##name##_s, gen_helper_##name##_d, \ }; \ - return gen_gvec_ool_arg_zpz(s, fns[a->esz], a, 0); \ -} + TRANS_FEAT(NAME, FEAT, gen_gvec_ool_arg_zpz, name##_fns[a->esz], a, 0) -DO_ZPZ(CLS, cls) -DO_ZPZ(CLZ, clz) -DO_ZPZ(CNT_zpz, cnt_zpz) -DO_ZPZ(CNOT, cnot) -DO_ZPZ(NOT_zpz, not_zpz) -DO_ZPZ(ABS, abs) -DO_ZPZ(NEG, neg) +DO_ZPZ(CLS, aa64_sve, sve_cls) +DO_ZPZ(CLZ, aa64_sve, sve_clz) +DO_ZPZ(CNT_zpz, aa64_sve, sve_cnt_zpz) +DO_ZPZ(CNOT, aa64_sve, sve_cnot) +DO_ZPZ(NOT_zpz, aa64_sve, sve_not_zpz) +DO_ZPZ(ABS, aa64_sve, sve_abs) +DO_ZPZ(NEG, aa64_sve, sve_neg) +DO_ZPZ(RBIT, aa64_sve, sve_rbit) -static bool trans_FABS(DisasContext *s, arg_rpr_esz *a) -{ - static gen_helper_gvec_3 * const fns[4] = { - NULL, - gen_helper_sve_fabs_h, - gen_helper_sve_fabs_s, - gen_helper_sve_fabs_d - }; - return gen_gvec_ool_arg_zpz(s, fns[a->esz], a, 0); -} +static gen_helper_gvec_3 * const fabs_fns[4] = { + NULL, gen_helper_sve_fabs_h, + gen_helper_sve_fabs_s, gen_helper_sve_fabs_d, +}; +TRANS_FEAT(FABS, aa64_sve, gen_gvec_ool_arg_zpz, fabs_fns[a->esz], a, 0) -static bool trans_FNEG(DisasContext *s, arg_rpr_esz *a) -{ - static gen_helper_gvec_3 * const fns[4] = { - NULL, - gen_helper_sve_fneg_h, - gen_helper_sve_fneg_s, - gen_helper_sve_fneg_d - }; - return gen_gvec_ool_arg_zpz(s, fns[a->esz], a, 0); -} +static gen_helper_gvec_3 * const fneg_fns[4] = { + NULL, gen_helper_sve_fneg_h, + gen_helper_sve_fneg_s, gen_helper_sve_fneg_d, +}; +TRANS_FEAT(FNEG, aa64_sve, gen_gvec_ool_arg_zpz, fneg_fns[a->esz], a, 0) -static bool trans_SXTB(DisasContext *s, arg_rpr_esz *a) -{ - static gen_helper_gvec_3 * const fns[4] = { - NULL, - gen_helper_sve_sxtb_h, - gen_helper_sve_sxtb_s, - gen_helper_sve_sxtb_d - }; - return gen_gvec_ool_arg_zpz(s, fns[a->esz], a, 0); -} +static gen_helper_gvec_3 * const sxtb_fns[4] = { + NULL, gen_helper_sve_sxtb_h, + gen_helper_sve_sxtb_s, gen_helper_sve_sxtb_d, +}; +TRANS_FEAT(SXTB, aa64_sve, gen_gvec_ool_arg_zpz, sxtb_fns[a->esz], a, 0) -static bool trans_UXTB(DisasContext *s, arg_rpr_esz *a) -{ - static gen_helper_gvec_3 * const fns[4] = { - NULL, - gen_helper_sve_uxtb_h, - gen_helper_sve_uxtb_s, - gen_helper_sve_uxtb_d - }; - return gen_gvec_ool_arg_zpz(s, fns[a->esz], a, 0); -} +static gen_helper_gvec_3 * const uxtb_fns[4] = { + NULL, gen_helper_sve_uxtb_h, + gen_helper_sve_uxtb_s, gen_helper_sve_uxtb_d, +}; +TRANS_FEAT(UXTB, aa64_sve, gen_gvec_ool_arg_zpz, uxtb_fns[a->esz], a, 0) -static bool trans_SXTH(DisasContext *s, arg_rpr_esz *a) -{ - static gen_helper_gvec_3 * const fns[4] = { - NULL, NULL, - gen_helper_sve_sxth_s, - gen_helper_sve_sxth_d - }; - return gen_gvec_ool_arg_zpz(s, fns[a->esz], a, 0); -} +static gen_helper_gvec_3 * const sxth_fns[4] = { + NULL, NULL, gen_helper_sve_sxth_s, gen_helper_sve_sxth_d +}; +TRANS_FEAT(SXTH, aa64_sve, gen_gvec_ool_arg_zpz, sxth_fns[a->esz], a, 0) -static bool trans_UXTH(DisasContext *s, arg_rpr_esz *a) -{ - static gen_helper_gvec_3 * const fns[4] = { - NULL, NULL, - gen_helper_sve_uxth_s, - gen_helper_sve_uxth_d - }; - return gen_gvec_ool_arg_zpz(s, fns[a->esz], a, 0); -} +static gen_helper_gvec_3 * const uxth_fns[4] = { + NULL, NULL, gen_helper_sve_uxth_s, gen_helper_sve_uxth_d +}; +TRANS_FEAT(UXTH, aa64_sve, gen_gvec_ool_arg_zpz, uxth_fns[a->esz], a, 0) -static bool trans_SXTW(DisasContext *s, arg_rpr_esz *a) -{ - return gen_gvec_ool_arg_zpz(s, a->esz == 3 ? gen_helper_sve_sxtw_d - : NULL, a, 0); -} - -static bool trans_UXTW(DisasContext *s, arg_rpr_esz *a) -{ - return gen_gvec_ool_arg_zpz(s, a->esz == 3 ? gen_helper_sve_uxtw_d - : NULL, a, 0); -} - -#undef DO_ZPZ +TRANS_FEAT(SXTW, aa64_sve, gen_gvec_ool_arg_zpz, + a->esz == 3 ? gen_helper_sve_sxtw_d : NULL, a, 0) +TRANS_FEAT(UXTW, aa64_sve, gen_gvec_ool_arg_zpz, + a->esz == 3 ? gen_helper_sve_uxtw_d : NULL, a, 0) /* *** SVE Integer Reduction Group @@ -XXX,XX +XXX,XX @@ TRANS_FEAT(TRN2_q, aa64_sve_f64mm, gen_gvec_ool_arg_zzz, *** SVE Permute Vector - Predicated Group */ -static bool trans_COMPACT(DisasContext *s, arg_rpr_esz *a) -{ - static gen_helper_gvec_3 * const fns[4] = { - NULL, NULL, gen_helper_sve_compact_s, gen_helper_sve_compact_d - }; - return gen_gvec_ool_arg_zpz(s, fns[a->esz], a, 0); -} +static gen_helper_gvec_3 * const compact_fns[4] = { + NULL, NULL, gen_helper_sve_compact_s, gen_helper_sve_compact_d +}; +TRANS_FEAT(COMPACT, aa64_sve, gen_gvec_ool_arg_zpz, compact_fns[a->esz], a, 0) /* Call the helper that computes the ARM LastActiveElement pseudocode * function, scaled by the element size. This includes the not found @@ -XXX,XX +XXX,XX @@ static bool trans_CPY_m_v(DisasContext *s, arg_rpr_esz *a) return true; } -static bool trans_REVB(DisasContext *s, arg_rpr_esz *a) -{ - static gen_helper_gvec_3 * const fns[4] = { - NULL, - gen_helper_sve_revb_h, - gen_helper_sve_revb_s, - gen_helper_sve_revb_d, - }; - return gen_gvec_ool_arg_zpz(s, fns[a->esz], a, 0); -} +static gen_helper_gvec_3 * const revb_fns[4] = { + NULL, gen_helper_sve_revb_h, + gen_helper_sve_revb_s, gen_helper_sve_revb_d, +}; +TRANS_FEAT(REVB, aa64_sve, gen_gvec_ool_arg_zpz, revb_fns[a->esz], a, 0) -static bool trans_REVH(DisasContext *s, arg_rpr_esz *a) -{ - static gen_helper_gvec_3 * const fns[4] = { - NULL, - NULL, - gen_helper_sve_revh_s, - gen_helper_sve_revh_d, - }; - return gen_gvec_ool_arg_zpz(s, fns[a->esz], a, 0); -} +static gen_helper_gvec_3 * const revh_fns[4] = { + NULL, NULL, gen_helper_sve_revh_s, gen_helper_sve_revh_d, +}; +TRANS_FEAT(REVH, aa64_sve, gen_gvec_ool_arg_zpz, revh_fns[a->esz], a, 0) -static bool trans_REVW(DisasContext *s, arg_rpr_esz *a) -{ - return gen_gvec_ool_arg_zpz(s, a->esz == 3 ? gen_helper_sve_revw_d - : NULL, a, 0); -} - -static bool trans_RBIT(DisasContext *s, arg_rpr_esz *a) -{ - static gen_helper_gvec_3 * const fns[4] = { - gen_helper_sve_rbit_b, - gen_helper_sve_rbit_h, - gen_helper_sve_rbit_s, - gen_helper_sve_rbit_d, - }; - return gen_gvec_ool_arg_zpz(s, fns[a->esz], a, 0); -} +TRANS_FEAT(REVW, aa64_sve, gen_gvec_ool_arg_zpz, + a->esz == 3 ? gen_helper_sve_revw_d : NULL, a, 0) static bool trans_SPLICE(DisasContext *s, arg_rprr_esz *a) { -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Convert SVE translation functions using do_sve2_zpz_data to use TRANS_FEAT and gen_gvec_ool_arg_zpz. Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-23-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 53 ++++++++++---------------------------- 1 file changed, 14 insertions(+), 39 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool trans_UADALP_zpzz(DisasContext *s, arg_rprr_esz *a) * SVE2 integer unary operations (predicated) */ -static bool do_sve2_zpz_ool(DisasContext *s, arg_rpr_esz *a, - gen_helper_gvec_3 *fn) -{ - if (!dc_isar_feature(aa64_sve2, s)) { - return false; - } - return gen_gvec_ool_arg_zpz(s, fn, a, 0); -} +TRANS_FEAT(URECPE, aa64_sve2, gen_gvec_ool_arg_zpz, + a->esz == 2 ? gen_helper_sve2_urecpe_s : NULL, a, 0) -static bool trans_URECPE(DisasContext *s, arg_rpr_esz *a) -{ - if (a->esz != 2) { - return false; - } - return do_sve2_zpz_ool(s, a, gen_helper_sve2_urecpe_s); -} +TRANS_FEAT(URSQRTE, aa64_sve2, gen_gvec_ool_arg_zpz, + a->esz == 2 ? gen_helper_sve2_ursqrte_s : NULL, a, 0) -static bool trans_URSQRTE(DisasContext *s, arg_rpr_esz *a) -{ - if (a->esz != 2) { - return false; - } - return do_sve2_zpz_ool(s, a, gen_helper_sve2_ursqrte_s); -} +static gen_helper_gvec_3 * const sqabs_fns[4] = { + gen_helper_sve2_sqabs_b, gen_helper_sve2_sqabs_h, + gen_helper_sve2_sqabs_s, gen_helper_sve2_sqabs_d, +}; +TRANS_FEAT(SQABS, aa64_sve2, gen_gvec_ool_arg_zpz, sqabs_fns[a->esz], a, 0) -static bool trans_SQABS(DisasContext *s, arg_rpr_esz *a) -{ - static gen_helper_gvec_3 * const fns[4] = { - gen_helper_sve2_sqabs_b, gen_helper_sve2_sqabs_h, - gen_helper_sve2_sqabs_s, gen_helper_sve2_sqabs_d, - }; - return do_sve2_zpz_ool(s, a, fns[a->esz]); -} - -static bool trans_SQNEG(DisasContext *s, arg_rpr_esz *a) -{ - static gen_helper_gvec_3 * const fns[4] = { - gen_helper_sve2_sqneg_b, gen_helper_sve2_sqneg_h, - gen_helper_sve2_sqneg_s, gen_helper_sve2_sqneg_d, - }; - return do_sve2_zpz_ool(s, a, fns[a->esz]); -} +static gen_helper_gvec_3 * const sqneg_fns[4] = { + gen_helper_sve2_sqneg_b, gen_helper_sve2_sqneg_h, + gen_helper_sve2_sqneg_s, gen_helper_sve2_sqneg_d, +}; +TRANS_FEAT(SQNEG, aa64_sve2, gen_gvec_ool_arg_zpz, sqneg_fns[a->esz], a, 0) #define DO_SVE2_ZPZZ(NAME, name) \ static bool trans_##NAME(DisasContext *s, arg_rprr_esz *a) \ -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Rename the function to match gen_gvec_ool_arg_zpz, and move to be adjacent. Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-24-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 29 ++++++++++++++--------------- 1 file changed, 14 insertions(+), 15 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool gen_gvec_ool_arg_zpz(DisasContext *s, gen_helper_gvec_3 *fn, return gen_gvec_ool_zzp(s, fn, a->rd, a->rn, a->pg, data); } +static bool gen_gvec_ool_arg_zpzi(DisasContext *s, gen_helper_gvec_3 *fn, + arg_rpri_esz *a) +{ + return gen_gvec_ool_zzp(s, fn, a->rd, a->rn, a->pg, a->imm); +} /* Invoke an out-of-line helper on 3 Zregs and a predicate. */ static void gen_gvec_ool_zzzp(DisasContext *s, gen_helper_gvec_4 *fn, @@ -XXX,XX +XXX,XX @@ static bool do_movz_zpz(DisasContext *s, int rd, int rn, int pg, return gen_gvec_ool_zzp(s, fns[esz], rd, rn, pg, invert); } -static bool do_zpzi_ool(DisasContext *s, arg_rpri_esz *a, - gen_helper_gvec_3 *fn) -{ - return gen_gvec_ool_zzp(s, fn, a->rd, a->rn, a->pg, a->imm); -} - static bool trans_ASR_zpzi(DisasContext *s, arg_rpri_esz *a) { static gen_helper_gvec_3 * const fns[4] = { @@ -XXX,XX +XXX,XX @@ static bool trans_ASR_zpzi(DisasContext *s, arg_rpri_esz *a) /* Shift by element size is architecturally valid. For arithmetic right-shift, it's the same as by one less. */ a->imm = MIN(a->imm, (8 << a->esz) - 1); - return do_zpzi_ool(s, a, fns[a->esz]); + return gen_gvec_ool_arg_zpzi(s, fns[a->esz], a); } static bool trans_LSR_zpzi(DisasContext *s, arg_rpri_esz *a) @@ -XXX,XX +XXX,XX @@ static bool trans_LSR_zpzi(DisasContext *s, arg_rpri_esz *a) if (a->imm >= (8 << a->esz)) { return do_movz_zpz(s, a->rd, a->rd, a->pg, a->esz, true); } else { - return do_zpzi_ool(s, a, fns[a->esz]); + return gen_gvec_ool_arg_zpzi(s, fns[a->esz], a); } } @@ -XXX,XX +XXX,XX @@ static bool trans_LSL_zpzi(DisasContext *s, arg_rpri_esz *a) if (a->imm >= (8 << a->esz)) { return do_movz_zpz(s, a->rd, a->rd, a->pg, a->esz, true); } else { - return do_zpzi_ool(s, a, fns[a->esz]); + return gen_gvec_ool_arg_zpzi(s, fns[a->esz], a); } } @@ -XXX,XX +XXX,XX @@ static bool trans_ASRD(DisasContext *s, arg_rpri_esz *a) if (a->imm >= (8 << a->esz)) { return do_movz_zpz(s, a->rd, a->rd, a->pg, a->esz, true); } else { - return do_zpzi_ool(s, a, fns[a->esz]); + return gen_gvec_ool_arg_zpzi(s, fns[a->esz], a); } } @@ -XXX,XX +XXX,XX @@ static bool trans_SQSHL_zpzi(DisasContext *s, arg_rpri_esz *a) if (a->esz < 0 || !dc_isar_feature(aa64_sve2, s)) { return false; } - return do_zpzi_ool(s, a, fns[a->esz]); + return gen_gvec_ool_arg_zpzi(s, fns[a->esz], a); } static bool trans_UQSHL_zpzi(DisasContext *s, arg_rpri_esz *a) @@ -XXX,XX +XXX,XX @@ static bool trans_UQSHL_zpzi(DisasContext *s, arg_rpri_esz *a) if (a->esz < 0 || !dc_isar_feature(aa64_sve2, s)) { return false; } - return do_zpzi_ool(s, a, fns[a->esz]); + return gen_gvec_ool_arg_zpzi(s, fns[a->esz], a); } static bool trans_SRSHR(DisasContext *s, arg_rpri_esz *a) @@ -XXX,XX +XXX,XX @@ static bool trans_SRSHR(DisasContext *s, arg_rpri_esz *a) if (a->esz < 0 || !dc_isar_feature(aa64_sve2, s)) { return false; } - return do_zpzi_ool(s, a, fns[a->esz]); + return gen_gvec_ool_arg_zpzi(s, fns[a->esz], a); } static bool trans_URSHR(DisasContext *s, arg_rpri_esz *a) @@ -XXX,XX +XXX,XX @@ static bool trans_URSHR(DisasContext *s, arg_rpri_esz *a) if (a->esz < 0 || !dc_isar_feature(aa64_sve2, s)) { return false; } - return do_zpzi_ool(s, a, fns[a->esz]); + return gen_gvec_ool_arg_zpzi(s, fns[a->esz], a); } static bool trans_SQSHLU(DisasContext *s, arg_rpri_esz *a) @@ -XXX,XX +XXX,XX @@ static bool trans_SQSHLU(DisasContext *s, arg_rpri_esz *a) if (a->esz < 0 || !dc_isar_feature(aa64_sve2, s)) { return false; } - return do_zpzi_ool(s, a, fns[a->esz]); + return gen_gvec_ool_arg_zpzi(s, fns[a->esz], a); } /* -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Convert some SVE translation functions using gen_gvec_ool_arg_zpzi to TRANS_FEAT. Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-25-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 85 ++++++++++++++------------------------ 1 file changed, 30 insertions(+), 55 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool trans_ASRD(DisasContext *s, arg_rpri_esz *a) } } -static bool trans_SQSHL_zpzi(DisasContext *s, arg_rpri_esz *a) -{ - static gen_helper_gvec_3 * const fns[4] = { - gen_helper_sve2_sqshl_zpzi_b, gen_helper_sve2_sqshl_zpzi_h, - gen_helper_sve2_sqshl_zpzi_s, gen_helper_sve2_sqshl_zpzi_d, - }; - if (a->esz < 0 || !dc_isar_feature(aa64_sve2, s)) { - return false; - } - return gen_gvec_ool_arg_zpzi(s, fns[a->esz], a); -} +static gen_helper_gvec_3 * const sqshl_zpzi_fns[4] = { + gen_helper_sve2_sqshl_zpzi_b, gen_helper_sve2_sqshl_zpzi_h, + gen_helper_sve2_sqshl_zpzi_s, gen_helper_sve2_sqshl_zpzi_d, +}; +TRANS_FEAT(SQSHL_zpzi, aa64_sve2, gen_gvec_ool_arg_zpzi, + a->esz < 0 ? NULL : sqshl_zpzi_fns[a->esz], a) -static bool trans_UQSHL_zpzi(DisasContext *s, arg_rpri_esz *a) -{ - static gen_helper_gvec_3 * const fns[4] = { - gen_helper_sve2_uqshl_zpzi_b, gen_helper_sve2_uqshl_zpzi_h, - gen_helper_sve2_uqshl_zpzi_s, gen_helper_sve2_uqshl_zpzi_d, - }; - if (a->esz < 0 || !dc_isar_feature(aa64_sve2, s)) { - return false; - } - return gen_gvec_ool_arg_zpzi(s, fns[a->esz], a); -} +static gen_helper_gvec_3 * const uqshl_zpzi_fns[4] = { + gen_helper_sve2_uqshl_zpzi_b, gen_helper_sve2_uqshl_zpzi_h, + gen_helper_sve2_uqshl_zpzi_s, gen_helper_sve2_uqshl_zpzi_d, +}; +TRANS_FEAT(UQSHL_zpzi, aa64_sve2, gen_gvec_ool_arg_zpzi, + a->esz < 0 ? NULL : uqshl_zpzi_fns[a->esz], a) -static bool trans_SRSHR(DisasContext *s, arg_rpri_esz *a) -{ - static gen_helper_gvec_3 * const fns[4] = { - gen_helper_sve2_srshr_b, gen_helper_sve2_srshr_h, - gen_helper_sve2_srshr_s, gen_helper_sve2_srshr_d, - }; - if (a->esz < 0 || !dc_isar_feature(aa64_sve2, s)) { - return false; - } - return gen_gvec_ool_arg_zpzi(s, fns[a->esz], a); -} +static gen_helper_gvec_3 * const srshr_fns[4] = { + gen_helper_sve2_srshr_b, gen_helper_sve2_srshr_h, + gen_helper_sve2_srshr_s, gen_helper_sve2_srshr_d, +}; +TRANS_FEAT(SRSHR, aa64_sve2, gen_gvec_ool_arg_zpzi, + a->esz < 0 ? NULL : srshr_fns[a->esz], a) -static bool trans_URSHR(DisasContext *s, arg_rpri_esz *a) -{ - static gen_helper_gvec_3 * const fns[4] = { - gen_helper_sve2_urshr_b, gen_helper_sve2_urshr_h, - gen_helper_sve2_urshr_s, gen_helper_sve2_urshr_d, - }; - if (a->esz < 0 || !dc_isar_feature(aa64_sve2, s)) { - return false; - } - return gen_gvec_ool_arg_zpzi(s, fns[a->esz], a); -} +static gen_helper_gvec_3 * const urshr_fns[4] = { + gen_helper_sve2_urshr_b, gen_helper_sve2_urshr_h, + gen_helper_sve2_urshr_s, gen_helper_sve2_urshr_d, +}; +TRANS_FEAT(URSHR, aa64_sve2, gen_gvec_ool_arg_zpzi, + a->esz < 0 ? NULL : urshr_fns[a->esz], a) -static bool trans_SQSHLU(DisasContext *s, arg_rpri_esz *a) -{ - static gen_helper_gvec_3 * const fns[4] = { - gen_helper_sve2_sqshlu_b, gen_helper_sve2_sqshlu_h, - gen_helper_sve2_sqshlu_s, gen_helper_sve2_sqshlu_d, - }; - if (a->esz < 0 || !dc_isar_feature(aa64_sve2, s)) { - return false; - } - return gen_gvec_ool_arg_zpzi(s, fns[a->esz], a); -} +static gen_helper_gvec_3 * const sqshlu_fns[4] = { + gen_helper_sve2_sqshlu_b, gen_helper_sve2_sqshlu_h, + gen_helper_sve2_sqshlu_s, gen_helper_sve2_sqshlu_d, +}; +TRANS_FEAT(SQSHLU, aa64_sve2, gen_gvec_ool_arg_zpzi, + a->esz < 0 ? NULL : sqshlu_fns[a->esz], a) /* *** SVE Bitwise Shift - Predicated Group -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-26-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 42 ++++++++++++++++---------------------- 1 file changed, 18 insertions(+), 24 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool gen_gvec_ool_arg_zpzi(DisasContext *s, gen_helper_gvec_3 *fn, } /* Invoke an out-of-line helper on 3 Zregs and a predicate. */ -static void gen_gvec_ool_zzzp(DisasContext *s, gen_helper_gvec_4 *fn, +static bool gen_gvec_ool_zzzp(DisasContext *s, gen_helper_gvec_4 *fn, int rd, int rn, int rm, int pg, int data) { - unsigned vsz = vec_full_reg_size(s); - tcg_gen_gvec_4_ool(vec_full_reg_offset(s, rd), - vec_full_reg_offset(s, rn), - vec_full_reg_offset(s, rm), - pred_full_reg_offset(s, pg), - vsz, vsz, data, fn); + if (fn == NULL) { + return false; + } + if (sve_access_check(s)) { + unsigned vsz = vec_full_reg_size(s); + tcg_gen_gvec_4_ool(vec_full_reg_offset(s, rd), + vec_full_reg_offset(s, rn), + vec_full_reg_offset(s, rm), + pred_full_reg_offset(s, pg), + vsz, vsz, data, fn); + } + return true; } /* Invoke a vector expander on two Zregs. */ @@ -XXX,XX +XXX,XX @@ static bool trans_UQSUB_zzz(DisasContext *s, arg_rrr_esz *a) static bool do_zpzz_ool(DisasContext *s, arg_rprr_esz *a, gen_helper_gvec_4 *fn) { - if (fn == NULL) { - return false; - } - if (sve_access_check(s)) { - gen_gvec_ool_zzzp(s, fn, a->rd, a->rn, a->rm, a->pg, 0); - } - return true; + return gen_gvec_ool_zzzp(s, fn, a->rd, a->rn, a->rm, a->pg, 0); } /* Select active elememnts from Zn and inactive elements from Zm, @@ -XXX,XX +XXX,XX @@ TRANS_FEAT(REVW, aa64_sve, gen_gvec_ool_arg_zpz, static bool trans_SPLICE(DisasContext *s, arg_rprr_esz *a) { - if (sve_access_check(s)) { - gen_gvec_ool_zzzp(s, gen_helper_sve_splice, - a->rd, a->rn, a->rm, a->pg, a->esz); - } - return true; + return gen_gvec_ool_zzzp(s, gen_helper_sve_splice, + a->rd, a->rn, a->rm, a->pg, a->esz); } static bool trans_SPLICE_sve2(DisasContext *s, arg_rpr_esz *a) @@ -XXX,XX +XXX,XX @@ static bool trans_SPLICE_sve2(DisasContext *s, arg_rpr_esz *a) if (!dc_isar_feature(aa64_sve2, s)) { return false; } - if (sve_access_check(s)) { - gen_gvec_ool_zzzp(s, gen_helper_sve_splice, - a->rd, a->rn, (a->rn + 1) % 32, a->pg, a->esz); - } - return true; + return gen_gvec_ool_zzzp(s, gen_helper_sve_splice, + a->rd, a->rn, (a->rn + 1) % 32, a->pg, a->esz); } /* -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Use gen_gvec_ool_arg_zpzz instead of gen_gvec_ool_zzzp when the arguments come from arg_rprr_esz. Replaces do_zpzz_ool. Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-27-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 21 +++++++++++---------- 1 file changed, 11 insertions(+), 10 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool gen_gvec_ool_zzzp(DisasContext *s, gen_helper_gvec_4 *fn, return true; } +static bool gen_gvec_ool_arg_zpzz(DisasContext *s, gen_helper_gvec_4 *fn, + arg_rprr_esz *a, int data) +{ + return gen_gvec_ool_zzzp(s, fn, a->rd, a->rn, a->rm, a->pg, data); +} + /* Invoke a vector expander on two Zregs. */ static void gen_gvec_fn_zz(DisasContext *s, GVecGen2Fn *gvec_fn, int esz, int rd, int rn) @@ -XXX,XX +XXX,XX @@ static bool trans_UQSUB_zzz(DisasContext *s, arg_rrr_esz *a) *** SVE Integer Arithmetic - Binary Predicated Group */ -static bool do_zpzz_ool(DisasContext *s, arg_rprr_esz *a, gen_helper_gvec_4 *fn) -{ - return gen_gvec_ool_zzzp(s, fn, a->rd, a->rn, a->rm, a->pg, 0); -} - /* Select active elememnts from Zn and inactive elements from Zm, * storing the result in Zd. */ @@ -XXX,XX +XXX,XX @@ static bool trans_##NAME##_zpzz(DisasContext *s, arg_rprr_esz *a) \ gen_helper_sve_##name##_zpzz_b, gen_helper_sve_##name##_zpzz_h, \ gen_helper_sve_##name##_zpzz_s, gen_helper_sve_##name##_zpzz_d, \ }; \ - return do_zpzz_ool(s, a, fns[a->esz]); \ + return gen_gvec_ool_arg_zpzz(s, fns[a->esz], a, 0); \ } DO_ZPZZ(AND, and) @@ -XXX,XX +XXX,XX @@ static bool trans_SDIV_zpzz(DisasContext *s, arg_rprr_esz *a) static gen_helper_gvec_4 * const fns[4] = { NULL, NULL, gen_helper_sve_sdiv_zpzz_s, gen_helper_sve_sdiv_zpzz_d }; - return do_zpzz_ool(s, a, fns[a->esz]); + return gen_gvec_ool_arg_zpzz(s, fns[a->esz], a, 0); } static bool trans_UDIV_zpzz(DisasContext *s, arg_rprr_esz *a) @@ -XXX,XX +XXX,XX @@ static bool trans_UDIV_zpzz(DisasContext *s, arg_rprr_esz *a) static gen_helper_gvec_4 * const fns[4] = { NULL, NULL, gen_helper_sve_udiv_zpzz_s, gen_helper_sve_udiv_zpzz_d }; - return do_zpzz_ool(s, a, fns[a->esz]); + return gen_gvec_ool_arg_zpzz(s, fns[a->esz], a, 0); } static bool trans_SEL_zpzz(DisasContext *s, arg_rprr_esz *a) @@ -XXX,XX +XXX,XX @@ static bool trans_##NAME##_zpzw(DisasContext *s, arg_rprr_esz *a) \ if (a->esz < 0 || a->esz >= 3) { \ return false; \ } \ - return do_zpzz_ool(s, a, fns[a->esz]); \ + return gen_gvec_ool_arg_zpzz(s, fns[a->esz], a, 0); \ } DO_ZPZW(ASR, asr) @@ -XXX,XX +XXX,XX @@ static bool do_sve2_zpzz_ool(DisasContext *s, arg_rprr_esz *a, if (!dc_isar_feature(aa64_sve2, s)) { return false; } - return do_zpzz_ool(s, a, fn); + return gen_gvec_ool_arg_zpzz(s, fn, a, 0); } static bool trans_SADALP_zpzz(DisasContext *s, arg_rprr_esz *a) -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Convert SVE translation functions directly using gen_gvec_ool_arg_zpzz to TRANS_FEAT. Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-28-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 85 ++++++++++++++++---------------------- 1 file changed, 36 insertions(+), 49 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static void do_sel_z(DisasContext *s, int rd, int rn, int rm, int pg, int esz) gen_gvec_ool_zzzp(s, fns[esz], rd, rn, rm, pg, 0); } -#define DO_ZPZZ(NAME, name) \ -static bool trans_##NAME##_zpzz(DisasContext *s, arg_rprr_esz *a) \ -{ \ - static gen_helper_gvec_4 * const fns[4] = { \ - gen_helper_sve_##name##_zpzz_b, gen_helper_sve_##name##_zpzz_h, \ - gen_helper_sve_##name##_zpzz_s, gen_helper_sve_##name##_zpzz_d, \ +#define DO_ZPZZ(NAME, FEAT, name) \ + static gen_helper_gvec_4 * const name##_zpzz_fns[4] = { \ + gen_helper_##name##_zpzz_b, gen_helper_##name##_zpzz_h, \ + gen_helper_##name##_zpzz_s, gen_helper_##name##_zpzz_d, \ }; \ - return gen_gvec_ool_arg_zpzz(s, fns[a->esz], a, 0); \ -} + TRANS_FEAT(NAME, FEAT, gen_gvec_ool_arg_zpzz, \ + name##_zpzz_fns[a->esz], a, 0) -DO_ZPZZ(AND, and) -DO_ZPZZ(EOR, eor) -DO_ZPZZ(ORR, orr) -DO_ZPZZ(BIC, bic) +DO_ZPZZ(AND_zpzz, aa64_sve, sve_and) +DO_ZPZZ(EOR_zpzz, aa64_sve, sve_eor) +DO_ZPZZ(ORR_zpzz, aa64_sve, sve_orr) +DO_ZPZZ(BIC_zpzz, aa64_sve, sve_bic) -DO_ZPZZ(ADD, add) -DO_ZPZZ(SUB, sub) +DO_ZPZZ(ADD_zpzz, aa64_sve, sve_add) +DO_ZPZZ(SUB_zpzz, aa64_sve, sve_sub) -DO_ZPZZ(SMAX, smax) -DO_ZPZZ(UMAX, umax) -DO_ZPZZ(SMIN, smin) -DO_ZPZZ(UMIN, umin) -DO_ZPZZ(SABD, sabd) -DO_ZPZZ(UABD, uabd) +DO_ZPZZ(SMAX_zpzz, aa64_sve, sve_smax) +DO_ZPZZ(UMAX_zpzz, aa64_sve, sve_umax) +DO_ZPZZ(SMIN_zpzz, aa64_sve, sve_smin) +DO_ZPZZ(UMIN_zpzz, aa64_sve, sve_umin) +DO_ZPZZ(SABD_zpzz, aa64_sve, sve_sabd) +DO_ZPZZ(UABD_zpzz, aa64_sve, sve_uabd) -DO_ZPZZ(MUL, mul) -DO_ZPZZ(SMULH, smulh) -DO_ZPZZ(UMULH, umulh) +DO_ZPZZ(MUL_zpzz, aa64_sve, sve_mul) +DO_ZPZZ(SMULH_zpzz, aa64_sve, sve_smulh) +DO_ZPZZ(UMULH_zpzz, aa64_sve, sve_umulh) -DO_ZPZZ(ASR, asr) -DO_ZPZZ(LSR, lsr) -DO_ZPZZ(LSL, lsl) +DO_ZPZZ(ASR_zpzz, aa64_sve, sve_asr) +DO_ZPZZ(LSR_zpzz, aa64_sve, sve_lsr) +DO_ZPZZ(LSL_zpzz, aa64_sve, sve_lsl) -static bool trans_SDIV_zpzz(DisasContext *s, arg_rprr_esz *a) -{ - static gen_helper_gvec_4 * const fns[4] = { - NULL, NULL, gen_helper_sve_sdiv_zpzz_s, gen_helper_sve_sdiv_zpzz_d - }; - return gen_gvec_ool_arg_zpzz(s, fns[a->esz], a, 0); -} +static gen_helper_gvec_4 * const sdiv_fns[4] = { + NULL, NULL, gen_helper_sve_sdiv_zpzz_s, gen_helper_sve_sdiv_zpzz_d +}; +TRANS_FEAT(SDIV_zpzz, aa64_sve, gen_gvec_ool_arg_zpzz, sdiv_fns[a->esz], a, 0) -static bool trans_UDIV_zpzz(DisasContext *s, arg_rprr_esz *a) -{ - static gen_helper_gvec_4 * const fns[4] = { - NULL, NULL, gen_helper_sve_udiv_zpzz_s, gen_helper_sve_udiv_zpzz_d - }; - return gen_gvec_ool_arg_zpzz(s, fns[a->esz], a, 0); -} +static gen_helper_gvec_4 * const udiv_fns[4] = { + NULL, NULL, gen_helper_sve_udiv_zpzz_s, gen_helper_sve_udiv_zpzz_d +}; +TRANS_FEAT(UDIV_zpzz, aa64_sve, gen_gvec_ool_arg_zpzz, udiv_fns[a->esz], a, 0) static bool trans_SEL_zpzz(DisasContext *s, arg_rprr_esz *a) { @@ -XXX,XX +XXX,XX @@ TRANS_FEAT(SQSHLU, aa64_sve2, gen_gvec_ool_arg_zpzi, */ #define DO_ZPZW(NAME, name) \ -static bool trans_##NAME##_zpzw(DisasContext *s, arg_rprr_esz *a) \ -{ \ - static gen_helper_gvec_4 * const fns[3] = { \ + static gen_helper_gvec_4 * const name##_zpzw_fns[4] = { \ gen_helper_sve_##name##_zpzw_b, gen_helper_sve_##name##_zpzw_h, \ - gen_helper_sve_##name##_zpzw_s, \ + gen_helper_sve_##name##_zpzw_s, NULL \ }; \ - if (a->esz < 0 || a->esz >= 3) { \ - return false; \ - } \ - return gen_gvec_ool_arg_zpzz(s, fns[a->esz], a, 0); \ -} + TRANS_FEAT(NAME##_zpzw, aa64_sve, gen_gvec_ool_arg_zpzz, \ + a->esz < 0 ? NULL : name##_zpzw_fns[a->esz], a, 0) DO_ZPZW(ASR, asr) DO_ZPZW(LSR, lsr) -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Convert SVE translation functions using do_sve2_zpzz_ool to use TRANS_FEAT and gen_gvec_ool_arg_zpzz. Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-29-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 118 +++++++++++++------------------------ 1 file changed, 40 insertions(+), 78 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool trans_SEL_zpzz(DisasContext *s, arg_rprr_esz *a) return true; } -#undef DO_ZPZZ - /* *** SVE Integer Arithmetic - Unary Predicated Group */ @@ -XXX,XX +XXX,XX @@ TRANS_FEAT(SQRDMULH_zzz, aa64_sve2, gen_gvec_ool_arg_zzz, * SVE2 Integer - Predicated */ -static bool do_sve2_zpzz_ool(DisasContext *s, arg_rprr_esz *a, - gen_helper_gvec_4 *fn) -{ - if (!dc_isar_feature(aa64_sve2, s)) { - return false; - } - return gen_gvec_ool_arg_zpzz(s, fn, a, 0); -} +static gen_helper_gvec_4 * const sadlp_fns[4] = { + NULL, gen_helper_sve2_sadalp_zpzz_h, + gen_helper_sve2_sadalp_zpzz_s, gen_helper_sve2_sadalp_zpzz_d, +}; +TRANS_FEAT(SADALP_zpzz, aa64_sve2, gen_gvec_ool_arg_zpzz, + sadlp_fns[a->esz], a, 0) -static bool trans_SADALP_zpzz(DisasContext *s, arg_rprr_esz *a) -{ - static gen_helper_gvec_4 * const fns[3] = { - gen_helper_sve2_sadalp_zpzz_h, - gen_helper_sve2_sadalp_zpzz_s, - gen_helper_sve2_sadalp_zpzz_d, - }; - if (a->esz == 0) { - return false; - } - return do_sve2_zpzz_ool(s, a, fns[a->esz - 1]); -} - -static bool trans_UADALP_zpzz(DisasContext *s, arg_rprr_esz *a) -{ - static gen_helper_gvec_4 * const fns[3] = { - gen_helper_sve2_uadalp_zpzz_h, - gen_helper_sve2_uadalp_zpzz_s, - gen_helper_sve2_uadalp_zpzz_d, - }; - if (a->esz == 0) { - return false; - } - return do_sve2_zpzz_ool(s, a, fns[a->esz - 1]); -} +static gen_helper_gvec_4 * const uadlp_fns[4] = { + NULL, gen_helper_sve2_uadalp_zpzz_h, + gen_helper_sve2_uadalp_zpzz_s, gen_helper_sve2_uadalp_zpzz_d, +}; +TRANS_FEAT(UADALP_zpzz, aa64_sve2, gen_gvec_ool_arg_zpzz, + uadlp_fns[a->esz], a, 0) /* * SVE2 integer unary operations (predicated) @@ -XXX,XX +XXX,XX @@ static gen_helper_gvec_3 * const sqneg_fns[4] = { }; TRANS_FEAT(SQNEG, aa64_sve2, gen_gvec_ool_arg_zpz, sqneg_fns[a->esz], a, 0) -#define DO_SVE2_ZPZZ(NAME, name) \ -static bool trans_##NAME(DisasContext *s, arg_rprr_esz *a) \ -{ \ - static gen_helper_gvec_4 * const fns[4] = { \ - gen_helper_sve2_##name##_zpzz_b, gen_helper_sve2_##name##_zpzz_h, \ - gen_helper_sve2_##name##_zpzz_s, gen_helper_sve2_##name##_zpzz_d, \ - }; \ - return do_sve2_zpzz_ool(s, a, fns[a->esz]); \ -} +DO_ZPZZ(SQSHL, aa64_sve2, sve2_sqshl) +DO_ZPZZ(SQRSHL, aa64_sve2, sve2_sqrshl) +DO_ZPZZ(SRSHL, aa64_sve2, sve2_srshl) -DO_SVE2_ZPZZ(SQSHL, sqshl) -DO_SVE2_ZPZZ(SQRSHL, sqrshl) -DO_SVE2_ZPZZ(SRSHL, srshl) +DO_ZPZZ(UQSHL, aa64_sve2, sve2_uqshl) +DO_ZPZZ(UQRSHL, aa64_sve2, sve2_uqrshl) +DO_ZPZZ(URSHL, aa64_sve2, sve2_urshl) -DO_SVE2_ZPZZ(UQSHL, uqshl) -DO_SVE2_ZPZZ(UQRSHL, uqrshl) -DO_SVE2_ZPZZ(URSHL, urshl) +DO_ZPZZ(SHADD, aa64_sve2, sve2_shadd) +DO_ZPZZ(SRHADD, aa64_sve2, sve2_srhadd) +DO_ZPZZ(SHSUB, aa64_sve2, sve2_shsub) -DO_SVE2_ZPZZ(SHADD, shadd) -DO_SVE2_ZPZZ(SRHADD, srhadd) -DO_SVE2_ZPZZ(SHSUB, shsub) +DO_ZPZZ(UHADD, aa64_sve2, sve2_uhadd) +DO_ZPZZ(URHADD, aa64_sve2, sve2_urhadd) +DO_ZPZZ(UHSUB, aa64_sve2, sve2_uhsub) -DO_SVE2_ZPZZ(UHADD, uhadd) -DO_SVE2_ZPZZ(URHADD, urhadd) -DO_SVE2_ZPZZ(UHSUB, uhsub) +DO_ZPZZ(ADDP, aa64_sve2, sve2_addp) +DO_ZPZZ(SMAXP, aa64_sve2, sve2_smaxp) +DO_ZPZZ(UMAXP, aa64_sve2, sve2_umaxp) +DO_ZPZZ(SMINP, aa64_sve2, sve2_sminp) +DO_ZPZZ(UMINP, aa64_sve2, sve2_uminp) -DO_SVE2_ZPZZ(ADDP, addp) -DO_SVE2_ZPZZ(SMAXP, smaxp) -DO_SVE2_ZPZZ(UMAXP, umaxp) -DO_SVE2_ZPZZ(SMINP, sminp) -DO_SVE2_ZPZZ(UMINP, uminp) - -DO_SVE2_ZPZZ(SQADD_zpzz, sqadd) -DO_SVE2_ZPZZ(UQADD_zpzz, uqadd) -DO_SVE2_ZPZZ(SQSUB_zpzz, sqsub) -DO_SVE2_ZPZZ(UQSUB_zpzz, uqsub) -DO_SVE2_ZPZZ(SUQADD, suqadd) -DO_SVE2_ZPZZ(USQADD, usqadd) +DO_ZPZZ(SQADD_zpzz, aa64_sve2, sve2_sqadd) +DO_ZPZZ(UQADD_zpzz, aa64_sve2, sve2_uqadd) +DO_ZPZZ(SQSUB_zpzz, aa64_sve2, sve2_sqsub) +DO_ZPZZ(UQSUB_zpzz, aa64_sve2, sve2_uqsub) +DO_ZPZZ(SUQADD, aa64_sve2, sve2_suqadd) +DO_ZPZZ(USQADD, aa64_sve2, sve2_usqadd) /* * SVE2 Widening Integer Arithmetic @@ -XXX,XX +XXX,XX @@ static bool trans_##NAME(DisasContext *s, arg_rprr_esz *a) \ DO_SVE2_PPZZ_MATCH(MATCH, match) DO_SVE2_PPZZ_MATCH(NMATCH, nmatch) -static bool trans_HISTCNT(DisasContext *s, arg_rprr_esz *a) -{ - static gen_helper_gvec_4 * const fns[2] = { - gen_helper_sve2_histcnt_s, gen_helper_sve2_histcnt_d - }; - if (a->esz < 2) { - return false; - } - return do_sve2_zpzz_ool(s, a, fns[a->esz - 2]); -} +static gen_helper_gvec_4 * const histcnt_fns[4] = { + NULL, NULL, gen_helper_sve2_histcnt_s, gen_helper_sve2_histcnt_d +}; +TRANS_FEAT(HISTCNT, aa64_sve2, gen_gvec_ool_arg_zpzz, + histcnt_fns[a->esz], a, 0) TRANS_FEAT(HISTSEG, aa64_sve2, gen_gvec_ool_arg_zzz, a->esz == 0 ? gen_helper_sve2_histseg : NULL, a, 0) -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> There is only one caller for gen_gvec_fn_zz; inline it. Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-30-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 13 +++---------- 1 file changed, 3 insertions(+), 10 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool gen_gvec_ool_arg_zpzz(DisasContext *s, gen_helper_gvec_4 *fn, return gen_gvec_ool_zzzp(s, fn, a->rd, a->rn, a->rm, a->pg, data); } -/* Invoke a vector expander on two Zregs. */ -static void gen_gvec_fn_zz(DisasContext *s, GVecGen2Fn *gvec_fn, - int esz, int rd, int rn) -{ - unsigned vsz = vec_full_reg_size(s); - gvec_fn(esz, vec_full_reg_offset(s, rd), - vec_full_reg_offset(s, rn), vsz, vsz); -} - /* Invoke a vector expander on three Zregs. */ static void gen_gvec_fn_zzz(DisasContext *s, GVecGen3Fn *gvec_fn, int esz, int rd, int rn, int rm) @@ -XXX,XX +XXX,XX @@ static void gen_gvec_fn_zzzz(DisasContext *s, GVecGen4Fn *gvec_fn, static bool do_mov_z(DisasContext *s, int rd, int rn) { if (sve_access_check(s)) { - gen_gvec_fn_zz(s, tcg_gen_gvec_mov, MO_8, rd, rn); + unsigned vsz = vec_full_reg_size(s); + tcg_gen_gvec_mov(MO_8, vec_full_reg_offset(s, rd), + vec_full_reg_offset(s, rn), vsz, vsz); } return true; } -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-31-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 36 +++++++++++++++--------------------- 1 file changed, 15 insertions(+), 21 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool gen_gvec_ool_arg_zpzz(DisasContext *s, gen_helper_gvec_4 *fn, } /* Invoke a vector expander on three Zregs. */ -static void gen_gvec_fn_zzz(DisasContext *s, GVecGen3Fn *gvec_fn, +static bool gen_gvec_fn_zzz(DisasContext *s, GVecGen3Fn *gvec_fn, int esz, int rd, int rn, int rm) { - unsigned vsz = vec_full_reg_size(s); - gvec_fn(esz, vec_full_reg_offset(s, rd), - vec_full_reg_offset(s, rn), - vec_full_reg_offset(s, rm), vsz, vsz); + if (gvec_fn == NULL) { + return false; + } + if (sve_access_check(s)) { + unsigned vsz = vec_full_reg_size(s); + gvec_fn(esz, vec_full_reg_offset(s, rd), + vec_full_reg_offset(s, rn), + vec_full_reg_offset(s, rm), vsz, vsz); + } + return true; } /* Invoke a vector expander on four Zregs. */ @@ -XXX,XX +XXX,XX @@ const uint64_t pred_esz_masks[4] = { static bool do_zzz_fn(DisasContext *s, arg_rrr_esz *a, GVecGen3Fn *gvec_fn) { - if (sve_access_check(s)) { - gen_gvec_fn_zzz(s, gvec_fn, a->esz, a->rd, a->rn, a->rm); - } - return true; + return gen_gvec_fn_zzz(s, gvec_fn, a->esz, a->rd, a->rn, a->rm); } static bool trans_AND_zzz(DisasContext *s, arg_rrr_esz *a) @@ -XXX,XX +XXX,XX @@ static bool trans_MUL_zzz(DisasContext *s, arg_rrr_esz *a) if (!dc_isar_feature(aa64_sve2, s)) { return false; } - if (sve_access_check(s)) { - gen_gvec_fn_zzz(s, tcg_gen_gvec_mul, a->esz, a->rd, a->rn, a->rm); - } - return true; + return gen_gvec_fn_zzz(s, tcg_gen_gvec_mul, a->esz, a->rd, a->rn, a->rm); } static gen_helper_gvec_3 * const smulh_zzz_fns[4] = { @@ -XXX,XX +XXX,XX @@ static bool do_sve2_fn_zzz(DisasContext *s, arg_rrr_esz *a, GVecGen3Fn *fn) if (!dc_isar_feature(aa64_sve2, s)) { return false; } - if (sve_access_check(s)) { - gen_gvec_fn_zzz(s, fn, a->esz, a->rd, a->rn, a->rm); - } - return true; + return gen_gvec_fn_zzz(s, fn, a->esz, a->rd, a->rn, a->rm); } static bool trans_SABA(DisasContext *s, arg_rrr_esz *a) @@ -XXX,XX +XXX,XX @@ static bool trans_RAX1(DisasContext *s, arg_rrr_esz *a) if (!dc_isar_feature(aa64_sve2_sha3, s)) { return false; } - if (sve_access_check(s)) { - gen_gvec_fn_zzz(s, gen_gvec_rax1, MO_64, a->rd, a->rn, a->rm); - } - return true; + return gen_gvec_fn_zzz(s, gen_gvec_rax1, MO_64, a->rd, a->rn, a->rm); } static bool trans_FCVTNT_sh(DisasContext *s, arg_rpr_esz *a) -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Rename the function to match gen_gvec_fn_zzz, and move to be adjacent. Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-32-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 31 ++++++++++++++++--------------- 1 file changed, 16 insertions(+), 15 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool gen_gvec_fn_zzz(DisasContext *s, GVecGen3Fn *gvec_fn, return true; } +static bool gen_gvec_fn_arg_zzz(DisasContext *s, GVecGen3Fn *fn, + arg_rrr_esz *a) +{ + return gen_gvec_fn_zzz(s, fn, a->esz, a->rd, a->rn, a->rm); +} + /* Invoke a vector expander on four Zregs. */ static void gen_gvec_fn_zzzz(DisasContext *s, GVecGen4Fn *gvec_fn, int esz, int rd, int rn, int rm, int ra) @@ -XXX,XX +XXX,XX @@ const uint64_t pred_esz_masks[4] = { *** SVE Logical - Unpredicated Group */ -static bool do_zzz_fn(DisasContext *s, arg_rrr_esz *a, GVecGen3Fn *gvec_fn) -{ - return gen_gvec_fn_zzz(s, gvec_fn, a->esz, a->rd, a->rn, a->rm); -} - static bool trans_AND_zzz(DisasContext *s, arg_rrr_esz *a) { - return do_zzz_fn(s, a, tcg_gen_gvec_and); + return gen_gvec_fn_arg_zzz(s, tcg_gen_gvec_and, a); } static bool trans_ORR_zzz(DisasContext *s, arg_rrr_esz *a) { - return do_zzz_fn(s, a, tcg_gen_gvec_or); + return gen_gvec_fn_arg_zzz(s, tcg_gen_gvec_or, a); } static bool trans_EOR_zzz(DisasContext *s, arg_rrr_esz *a) { - return do_zzz_fn(s, a, tcg_gen_gvec_xor); + return gen_gvec_fn_arg_zzz(s, tcg_gen_gvec_xor, a); } static bool trans_BIC_zzz(DisasContext *s, arg_rrr_esz *a) { - return do_zzz_fn(s, a, tcg_gen_gvec_andc); + return gen_gvec_fn_arg_zzz(s, tcg_gen_gvec_andc, a); } static void gen_xar8_i64(TCGv_i64 d, TCGv_i64 n, TCGv_i64 m, int64_t sh) @@ -XXX,XX +XXX,XX @@ static bool trans_NBSL(DisasContext *s, arg_rrrr_esz *a) static bool trans_ADD_zzz(DisasContext *s, arg_rrr_esz *a) { - return do_zzz_fn(s, a, tcg_gen_gvec_add); + return gen_gvec_fn_arg_zzz(s, tcg_gen_gvec_add, a); } static bool trans_SUB_zzz(DisasContext *s, arg_rrr_esz *a) { - return do_zzz_fn(s, a, tcg_gen_gvec_sub); + return gen_gvec_fn_arg_zzz(s, tcg_gen_gvec_sub, a); } static bool trans_SQADD_zzz(DisasContext *s, arg_rrr_esz *a) { - return do_zzz_fn(s, a, tcg_gen_gvec_ssadd); + return gen_gvec_fn_arg_zzz(s, tcg_gen_gvec_ssadd, a); } static bool trans_SQSUB_zzz(DisasContext *s, arg_rrr_esz *a) { - return do_zzz_fn(s, a, tcg_gen_gvec_sssub); + return gen_gvec_fn_arg_zzz(s, tcg_gen_gvec_sssub, a); } static bool trans_UQADD_zzz(DisasContext *s, arg_rrr_esz *a) { - return do_zzz_fn(s, a, tcg_gen_gvec_usadd); + return gen_gvec_fn_arg_zzz(s, tcg_gen_gvec_usadd, a); } static bool trans_UQSUB_zzz(DisasContext *s, arg_rrr_esz *a) { - return do_zzz_fn(s, a, tcg_gen_gvec_ussub); + return gen_gvec_fn_arg_zzz(s, tcg_gen_gvec_ussub, a); } /* -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Two uses of gen_gvec_fn_zzz can pass on arg_rrr_esz instead. Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-33-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool trans_MUL_zzz(DisasContext *s, arg_rrr_esz *a) if (!dc_isar_feature(aa64_sve2, s)) { return false; } - return gen_gvec_fn_zzz(s, tcg_gen_gvec_mul, a->esz, a->rd, a->rn, a->rm); + return gen_gvec_fn_arg_zzz(s, tcg_gen_gvec_mul, a); } static gen_helper_gvec_3 * const smulh_zzz_fns[4] = { @@ -XXX,XX +XXX,XX @@ static bool do_sve2_fn_zzz(DisasContext *s, arg_rrr_esz *a, GVecGen3Fn *fn) if (!dc_isar_feature(aa64_sve2, s)) { return false; } - return gen_gvec_fn_zzz(s, fn, a->esz, a->rd, a->rn, a->rm); + return gen_gvec_fn_arg_zzz(s, fn, a); } static bool trans_SABA(DisasContext *s, arg_rrr_esz *a) -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Convert SVE translation functions directly using gen_gvec_fn_arg_zzz to TRANS_FEAT. Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-34-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 66 +++++++------------------------------- 1 file changed, 11 insertions(+), 55 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ const uint64_t pred_esz_masks[4] = { *** SVE Logical - Unpredicated Group */ -static bool trans_AND_zzz(DisasContext *s, arg_rrr_esz *a) -{ - return gen_gvec_fn_arg_zzz(s, tcg_gen_gvec_and, a); -} - -static bool trans_ORR_zzz(DisasContext *s, arg_rrr_esz *a) -{ - return gen_gvec_fn_arg_zzz(s, tcg_gen_gvec_or, a); -} - -static bool trans_EOR_zzz(DisasContext *s, arg_rrr_esz *a) -{ - return gen_gvec_fn_arg_zzz(s, tcg_gen_gvec_xor, a); -} - -static bool trans_BIC_zzz(DisasContext *s, arg_rrr_esz *a) -{ - return gen_gvec_fn_arg_zzz(s, tcg_gen_gvec_andc, a); -} +TRANS_FEAT(AND_zzz, aa64_sve, gen_gvec_fn_arg_zzz, tcg_gen_gvec_and, a) +TRANS_FEAT(ORR_zzz, aa64_sve, gen_gvec_fn_arg_zzz, tcg_gen_gvec_or, a) +TRANS_FEAT(EOR_zzz, aa64_sve, gen_gvec_fn_arg_zzz, tcg_gen_gvec_xor, a) +TRANS_FEAT(BIC_zzz, aa64_sve, gen_gvec_fn_arg_zzz, tcg_gen_gvec_andc, a) static void gen_xar8_i64(TCGv_i64 d, TCGv_i64 n, TCGv_i64 m, int64_t sh) { @@ -XXX,XX +XXX,XX @@ static bool trans_NBSL(DisasContext *s, arg_rrrr_esz *a) *** SVE Integer Arithmetic - Unpredicated Group */ -static bool trans_ADD_zzz(DisasContext *s, arg_rrr_esz *a) -{ - return gen_gvec_fn_arg_zzz(s, tcg_gen_gvec_add, a); -} - -static bool trans_SUB_zzz(DisasContext *s, arg_rrr_esz *a) -{ - return gen_gvec_fn_arg_zzz(s, tcg_gen_gvec_sub, a); -} - -static bool trans_SQADD_zzz(DisasContext *s, arg_rrr_esz *a) -{ - return gen_gvec_fn_arg_zzz(s, tcg_gen_gvec_ssadd, a); -} - -static bool trans_SQSUB_zzz(DisasContext *s, arg_rrr_esz *a) -{ - return gen_gvec_fn_arg_zzz(s, tcg_gen_gvec_sssub, a); -} - -static bool trans_UQADD_zzz(DisasContext *s, arg_rrr_esz *a) -{ - return gen_gvec_fn_arg_zzz(s, tcg_gen_gvec_usadd, a); -} - -static bool trans_UQSUB_zzz(DisasContext *s, arg_rrr_esz *a) -{ - return gen_gvec_fn_arg_zzz(s, tcg_gen_gvec_ussub, a); -} +TRANS_FEAT(ADD_zzz, aa64_sve, gen_gvec_fn_arg_zzz, tcg_gen_gvec_add, a) +TRANS_FEAT(SUB_zzz, aa64_sve, gen_gvec_fn_arg_zzz, tcg_gen_gvec_sub, a) +TRANS_FEAT(SQADD_zzz, aa64_sve, gen_gvec_fn_arg_zzz, tcg_gen_gvec_ssadd, a) +TRANS_FEAT(SQSUB_zzz, aa64_sve, gen_gvec_fn_arg_zzz, tcg_gen_gvec_sssub, a) +TRANS_FEAT(UQADD_zzz, aa64_sve, gen_gvec_fn_arg_zzz, tcg_gen_gvec_usadd, a) +TRANS_FEAT(UQSUB_zzz, aa64_sve, gen_gvec_fn_arg_zzz, tcg_gen_gvec_ussub, a) /* *** SVE Integer Arithmetic - Binary Predicated Group @@ -XXX,XX +XXX,XX @@ static bool trans_MOVPRFX_z(DisasContext *s, arg_rpr_esz *a) * SVE2 Integer Multiply - Unpredicated */ -static bool trans_MUL_zzz(DisasContext *s, arg_rrr_esz *a) -{ - if (!dc_isar_feature(aa64_sve2, s)) { - return false; - } - return gen_gvec_fn_arg_zzz(s, tcg_gen_gvec_mul, a); -} +TRANS_FEAT(MUL_zzz, aa64_sve2, gen_gvec_fn_arg_zzz, tcg_gen_gvec_mul, a) static gen_helper_gvec_3 * const smulh_zzz_fns[4] = { gen_helper_gvec_smulh_b, gen_helper_gvec_smulh_h, -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Convert SVE translation functions using do_sve2_fn_zzz to use TRANS_FEAT and gen_gvec_fn_arg_zzz. Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-35-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 19 ++----------------- 1 file changed, 2 insertions(+), 17 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool trans_SLI(DisasContext *s, arg_rri_esz *a) return do_sve2_fn2i(s, a, gen_gvec_sli); } -static bool do_sve2_fn_zzz(DisasContext *s, arg_rrr_esz *a, GVecGen3Fn *fn) -{ - if (!dc_isar_feature(aa64_sve2, s)) { - return false; - } - return gen_gvec_fn_arg_zzz(s, fn, a); -} - -static bool trans_SABA(DisasContext *s, arg_rrr_esz *a) -{ - return do_sve2_fn_zzz(s, a, gen_gvec_saba); -} - -static bool trans_UABA(DisasContext *s, arg_rrr_esz *a) -{ - return do_sve2_fn_zzz(s, a, gen_gvec_uaba); -} +TRANS_FEAT(SABA, aa64_sve2, gen_gvec_fn_arg_zzz, gen_gvec_saba, a) +TRANS_FEAT(UABA, aa64_sve2, gen_gvec_fn_arg_zzz, gen_gvec_uaba, a) static bool do_sve2_narrow_extract(DisasContext *s, arg_rri_esz *a, const GVecGen2 ops[3]) -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> The decode for RAX1 sets esz to MO_8, because that's what we use by default for "no esz present". We changed that to MO_64 during translation because it is more logical for the operation. However, the esz argument to gen_gvec_rax1 is unused and forces MO_64 within that function, so there is no need to do it here as well. Simplify to use gen_gvec_fn_arg_zzz and TRANS_FEAT. Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-36-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 8 +------- 1 file changed, 1 insertion(+), 7 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ TRANS_FEAT(SM4E, aa64_sve2_sm4, gen_gvec_ool_arg_zzz, TRANS_FEAT(SM4EKEY, aa64_sve2_sm4, gen_gvec_ool_arg_zzz, gen_helper_crypto_sm4ekey, a, 0) -static bool trans_RAX1(DisasContext *s, arg_rrr_esz *a) -{ - if (!dc_isar_feature(aa64_sve2_sha3, s)) { - return false; - } - return gen_gvec_fn_zzz(s, gen_gvec_rax1, MO_64, a->rd, a->rn, a->rm); -} +TRANS_FEAT(RAX1, aa64_sve2_sha3, gen_gvec_fn_arg_zzz, gen_gvec_rax1, a) static bool trans_FCVTNT_sh(DisasContext *s, arg_rpr_esz *a) { -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Merge gen_gvec_fn_zzzz with the sve access check and the dereference of arg_rrrr_esz. Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-37-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 25 ++++++++++++++----------- 1 file changed, 14 insertions(+), 11 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool gen_gvec_fn_arg_zzz(DisasContext *s, GVecGen3Fn *fn, } /* Invoke a vector expander on four Zregs. */ -static void gen_gvec_fn_zzzz(DisasContext *s, GVecGen4Fn *gvec_fn, - int esz, int rd, int rn, int rm, int ra) +static bool gen_gvec_fn_arg_zzzz(DisasContext *s, GVecGen4Fn *gvec_fn, + arg_rrrr_esz *a) { - unsigned vsz = vec_full_reg_size(s); - gvec_fn(esz, vec_full_reg_offset(s, rd), - vec_full_reg_offset(s, rn), - vec_full_reg_offset(s, rm), - vec_full_reg_offset(s, ra), vsz, vsz); + if (gvec_fn == NULL) { + return false; + } + if (sve_access_check(s)) { + unsigned vsz = vec_full_reg_size(s); + gvec_fn(a->esz, vec_full_reg_offset(s, a->rd), + vec_full_reg_offset(s, a->rn), + vec_full_reg_offset(s, a->rm), + vec_full_reg_offset(s, a->ra), vsz, vsz); + } + return true; } /* Invoke a vector move on two Zregs. */ @@ -XXX,XX +XXX,XX @@ static bool do_sve2_zzzz_fn(DisasContext *s, arg_rrrr_esz *a, GVecGen4Fn *fn) if (!dc_isar_feature(aa64_sve2, s)) { return false; } - if (sve_access_check(s)) { - gen_gvec_fn_zzzz(s, fn, a->esz, a->rd, a->rn, a->rm, a->ra); - } - return true; + return gen_gvec_fn_arg_zzzz(s, fn, a); } static void gen_eor3_i64(TCGv_i64 d, TCGv_i64 n, TCGv_i64 m, TCGv_i64 k) -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Convert SVE translation functions using do_sve2_zzzz_fn to use TRANS_FEAT and gen_gvec_fn_arg_zzzz. Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-38-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 38 ++++++-------------------------------- 1 file changed, 6 insertions(+), 32 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool trans_XAR(DisasContext *s, arg_rrri_esz *a) return true; } -static bool do_sve2_zzzz_fn(DisasContext *s, arg_rrrr_esz *a, GVecGen4Fn *fn) -{ - if (!dc_isar_feature(aa64_sve2, s)) { - return false; - } - return gen_gvec_fn_arg_zzzz(s, fn, a); -} - static void gen_eor3_i64(TCGv_i64 d, TCGv_i64 n, TCGv_i64 m, TCGv_i64 k) { tcg_gen_xor_i64(d, n, m); @@ -XXX,XX +XXX,XX @@ static void gen_eor3(unsigned vece, uint32_t d, uint32_t n, uint32_t m, tcg_gen_gvec_4(d, n, m, a, oprsz, maxsz, &op); } -static bool trans_EOR3(DisasContext *s, arg_rrrr_esz *a) -{ - return do_sve2_zzzz_fn(s, a, gen_eor3); -} +TRANS_FEAT(EOR3, aa64_sve2, gen_gvec_fn_arg_zzzz, gen_eor3, a) static void gen_bcax_i64(TCGv_i64 d, TCGv_i64 n, TCGv_i64 m, TCGv_i64 k) { @@ -XXX,XX +XXX,XX @@ static void gen_bcax(unsigned vece, uint32_t d, uint32_t n, uint32_t m, tcg_gen_gvec_4(d, n, m, a, oprsz, maxsz, &op); } -static bool trans_BCAX(DisasContext *s, arg_rrrr_esz *a) -{ - return do_sve2_zzzz_fn(s, a, gen_bcax); -} +TRANS_FEAT(BCAX, aa64_sve2, gen_gvec_fn_arg_zzzz, gen_bcax, a) static void gen_bsl(unsigned vece, uint32_t d, uint32_t n, uint32_t m, uint32_t a, uint32_t oprsz, uint32_t maxsz) @@ -XXX,XX +XXX,XX @@ static void gen_bsl(unsigned vece, uint32_t d, uint32_t n, uint32_t m, tcg_gen_gvec_bitsel(vece, d, a, n, m, oprsz, maxsz); } -static bool trans_BSL(DisasContext *s, arg_rrrr_esz *a) -{ - return do_sve2_zzzz_fn(s, a, gen_bsl); -} +TRANS_FEAT(BSL, aa64_sve2, gen_gvec_fn_arg_zzzz, gen_bsl, a) static void gen_bsl1n_i64(TCGv_i64 d, TCGv_i64 n, TCGv_i64 m, TCGv_i64 k) { @@ -XXX,XX +XXX,XX @@ static void gen_bsl1n(unsigned vece, uint32_t d, uint32_t n, uint32_t m, tcg_gen_gvec_4(d, n, m, a, oprsz, maxsz, &op); } -static bool trans_BSL1N(DisasContext *s, arg_rrrr_esz *a) -{ - return do_sve2_zzzz_fn(s, a, gen_bsl1n); -} +TRANS_FEAT(BSL1N, aa64_sve2, gen_gvec_fn_arg_zzzz, gen_bsl1n, a) static void gen_bsl2n_i64(TCGv_i64 d, TCGv_i64 n, TCGv_i64 m, TCGv_i64 k) { @@ -XXX,XX +XXX,XX @@ static void gen_bsl2n(unsigned vece, uint32_t d, uint32_t n, uint32_t m, tcg_gen_gvec_4(d, n, m, a, oprsz, maxsz, &op); } -static bool trans_BSL2N(DisasContext *s, arg_rrrr_esz *a) -{ - return do_sve2_zzzz_fn(s, a, gen_bsl2n); -} +TRANS_FEAT(BSL2N, aa64_sve2, gen_gvec_fn_arg_zzzz, gen_bsl2n, a) static void gen_nbsl_i64(TCGv_i64 d, TCGv_i64 n, TCGv_i64 m, TCGv_i64 k) { @@ -XXX,XX +XXX,XX @@ static void gen_nbsl(unsigned vece, uint32_t d, uint32_t n, uint32_t m, tcg_gen_gvec_4(d, n, m, a, oprsz, maxsz, &op); } -static bool trans_NBSL(DisasContext *s, arg_rrrr_esz *a) -{ - return do_sve2_zzzz_fn(s, a, gen_nbsl); -} +TRANS_FEAT(NBSL, aa64_sve2, gen_gvec_fn_arg_zzzz, gen_nbsl, a) /* *** SVE Integer Arithmetic - Unpredicated Group -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> We have two places that perform this particular operation. Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-39-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 30 +++++++++++++++++------------- 1 file changed, 17 insertions(+), 13 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool gen_gvec_ool_arg_zpzz(DisasContext *s, gen_helper_gvec_4 *fn, return gen_gvec_ool_zzzp(s, fn, a->rd, a->rn, a->rm, a->pg, data); } +/* Invoke a vector expander on two Zregs and an immediate. */ +static bool gen_gvec_fn_zzi(DisasContext *s, GVecGen2iFn *gvec_fn, + int esz, int rd, int rn, uint64_t imm) +{ + if (gvec_fn == NULL) { + return false; + } + if (sve_access_check(s)) { + unsigned vsz = vec_full_reg_size(s); + gvec_fn(esz, vec_full_reg_offset(s, rd), + vec_full_reg_offset(s, rn), imm, vsz, vsz); + } + return true; +} + /* Invoke a vector expander on three Zregs. */ static bool gen_gvec_fn_zzz(DisasContext *s, GVecGen3Fn *gvec_fn, int esz, int rd, int rn, int rm) @@ -XXX,XX +XXX,XX @@ static bool do_zz_dbm(DisasContext *s, arg_rr_dbm *a, GVecGen2iFn *gvec_fn) extract32(a->dbm, 6, 6))) { return false; } - if (sve_access_check(s)) { - unsigned vsz = vec_full_reg_size(s); - gvec_fn(MO_64, vec_full_reg_offset(s, a->rd), - vec_full_reg_offset(s, a->rn), imm, vsz, vsz); - } - return true; + return gen_gvec_fn_zzi(s, gvec_fn, MO_64, a->rd, a->rn, imm); } static bool trans_AND_zzi(DisasContext *s, arg_rr_dbm *a) @@ -XXX,XX +XXX,XX @@ static bool do_sve2_fn2i(DisasContext *s, arg_rri_esz *a, GVecGen2iFn *fn) if (a->esz < 0 || !dc_isar_feature(aa64_sve2, s)) { return false; } - if (sve_access_check(s)) { - unsigned vsz = vec_full_reg_size(s); - unsigned rd_ofs = vec_full_reg_offset(s, a->rd); - unsigned rn_ofs = vec_full_reg_offset(s, a->rn); - fn(a->esz, rd_ofs, rn_ofs, a->imm, vsz, vsz); - } - return true; + return gen_gvec_fn_zzi(s, fn, a->esz, a->rd, a->rn, a->imm); } static bool trans_SSRA(DisasContext *s, arg_rri_esz *a) -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-40-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 17 +++-------------- 1 file changed, 3 insertions(+), 14 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool do_zz_dbm(DisasContext *s, arg_rr_dbm *a, GVecGen2iFn *gvec_fn) return gen_gvec_fn_zzi(s, gvec_fn, MO_64, a->rd, a->rn, imm); } -static bool trans_AND_zzi(DisasContext *s, arg_rr_dbm *a) -{ - return do_zz_dbm(s, a, tcg_gen_gvec_andi); -} - -static bool trans_ORR_zzi(DisasContext *s, arg_rr_dbm *a) -{ - return do_zz_dbm(s, a, tcg_gen_gvec_ori); -} - -static bool trans_EOR_zzi(DisasContext *s, arg_rr_dbm *a) -{ - return do_zz_dbm(s, a, tcg_gen_gvec_xori); -} +TRANS_FEAT(AND_zzi, aa64_sve, do_zz_dbm, a, tcg_gen_gvec_andi) +TRANS_FEAT(ORR_zzi, aa64_sve, do_zz_dbm, a, tcg_gen_gvec_ori) +TRANS_FEAT(EOR_zzi, aa64_sve, do_zz_dbm, a, tcg_gen_gvec_xori) static bool trans_DUPM(DisasContext *s, arg_DUPM *a) { -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> The check is already done in gen_gvec_ool_zzzp, which is called by do_sel_z; remove from callers. Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-41-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 14 ++++---------- 1 file changed, 4 insertions(+), 10 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ TRANS_FEAT(UQSUB_zzz, aa64_sve, gen_gvec_fn_arg_zzz, tcg_gen_gvec_ussub, a) /* Select active elememnts from Zn and inactive elements from Zm, * storing the result in Zd. */ -static void do_sel_z(DisasContext *s, int rd, int rn, int rm, int pg, int esz) +static bool do_sel_z(DisasContext *s, int rd, int rn, int rm, int pg, int esz) { static gen_helper_gvec_4 * const fns[4] = { gen_helper_sve_sel_zpzz_b, gen_helper_sve_sel_zpzz_h, gen_helper_sve_sel_zpzz_s, gen_helper_sve_sel_zpzz_d }; - gen_gvec_ool_zzzp(s, fns[esz], rd, rn, rm, pg, 0); + return gen_gvec_ool_zzzp(s, fns[esz], rd, rn, rm, pg, 0); } #define DO_ZPZZ(NAME, FEAT, name) \ @@ -XXX,XX +XXX,XX @@ TRANS_FEAT(UDIV_zpzz, aa64_sve, gen_gvec_ool_arg_zpzz, udiv_fns[a->esz], a, 0) static bool trans_SEL_zpzz(DisasContext *s, arg_rprr_esz *a) { - if (sve_access_check(s)) { - do_sel_z(s, a->rd, a->rn, a->rm, a->pg, a->esz); - } - return true; + return do_sel_z(s, a->rd, a->rn, a->rm, a->pg, a->esz); } /* @@ -XXX,XX +XXX,XX @@ static bool trans_MOVPRFX(DisasContext *s, arg_MOVPRFX *a) static bool trans_MOVPRFX_m(DisasContext *s, arg_rpr_esz *a) { - if (sve_access_check(s)) { - do_sel_z(s, a->rd, a->rn, a->rd, a->pg, a->esz); - } - return true; + return do_sel_z(s, a->rd, a->rn, a->rd, a->pg, a->esz); } static bool trans_MOVPRFX_z(DisasContext *s, arg_rpr_esz *a) -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> We have two places that perform this particular operation. Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-42-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 21 +++++++++++++-------- 1 file changed, 13 insertions(+), 8 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool gen_gvec_fn_zzi(DisasContext *s, GVecGen2iFn *gvec_fn, return true; } +static bool gen_gvec_fn_arg_zzi(DisasContext *s, GVecGen2iFn *gvec_fn, + arg_rri_esz *a) +{ + if (a->esz < 0) { + /* Invalid tsz encoding -- see tszimm_esz. */ + return false; + } + return gen_gvec_fn_zzi(s, gvec_fn, a->esz, a->rd, a->rn, a->imm); +} + /* Invoke a vector expander on three Zregs. */ static bool gen_gvec_fn_zzz(DisasContext *s, GVecGen3Fn *gvec_fn, int esz, int rd, int rn, int rm) @@ -XXX,XX +XXX,XX @@ static bool trans_ADD_zzi(DisasContext *s, arg_rri_esz *a) if (a->esz == 0 && extract32(s->insn, 13, 1)) { return false; } - if (sve_access_check(s)) { - unsigned vsz = vec_full_reg_size(s); - tcg_gen_gvec_addi(a->esz, vec_full_reg_offset(s, a->rd), - vec_full_reg_offset(s, a->rn), a->imm, vsz, vsz); - } - return true; + return gen_gvec_fn_arg_zzi(s, tcg_gen_gvec_addi, a); } static bool trans_SUB_zzi(DisasContext *s, arg_rri_esz *a) @@ -XXX,XX +XXX,XX @@ TRANS_FEAT(ADCLT, aa64_sve2, do_adcl, a, true) static bool do_sve2_fn2i(DisasContext *s, arg_rri_esz *a, GVecGen2iFn *fn) { - if (a->esz < 0 || !dc_isar_feature(aa64_sve2, s)) { + if (!dc_isar_feature(aa64_sve2, s)) { return false; } - return gen_gvec_fn_zzi(s, fn, a->esz, a->rd, a->rn, a->imm); + return gen_gvec_fn_arg_zzi(s, fn, a); } static bool trans_SSRA(DisasContext *s, arg_rri_esz *a) -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Convert SVE translation functions using do_sve2_fn2i to use TRANS_FEAT and gen_gvec_fn_arg_zzi. Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-43-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 43 ++++++-------------------------------- 1 file changed, 6 insertions(+), 37 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool do_adcl(DisasContext *s, arg_rrrr_esz *a, bool sel) TRANS_FEAT(ADCLB, aa64_sve2, do_adcl, a, false) TRANS_FEAT(ADCLT, aa64_sve2, do_adcl, a, true) -static bool do_sve2_fn2i(DisasContext *s, arg_rri_esz *a, GVecGen2iFn *fn) -{ - if (!dc_isar_feature(aa64_sve2, s)) { - return false; - } - return gen_gvec_fn_arg_zzi(s, fn, a); -} - -static bool trans_SSRA(DisasContext *s, arg_rri_esz *a) -{ - return do_sve2_fn2i(s, a, gen_gvec_ssra); -} - -static bool trans_USRA(DisasContext *s, arg_rri_esz *a) -{ - return do_sve2_fn2i(s, a, gen_gvec_usra); -} - -static bool trans_SRSRA(DisasContext *s, arg_rri_esz *a) -{ - return do_sve2_fn2i(s, a, gen_gvec_srsra); -} - -static bool trans_URSRA(DisasContext *s, arg_rri_esz *a) -{ - return do_sve2_fn2i(s, a, gen_gvec_ursra); -} - -static bool trans_SRI(DisasContext *s, arg_rri_esz *a) -{ - return do_sve2_fn2i(s, a, gen_gvec_sri); -} - -static bool trans_SLI(DisasContext *s, arg_rri_esz *a) -{ - return do_sve2_fn2i(s, a, gen_gvec_sli); -} +TRANS_FEAT(SSRA, aa64_sve2, gen_gvec_fn_arg_zzi, gen_gvec_ssra, a) +TRANS_FEAT(USRA, aa64_sve2, gen_gvec_fn_arg_zzi, gen_gvec_usra, a) +TRANS_FEAT(SRSRA, aa64_sve2, gen_gvec_fn_arg_zzi, gen_gvec_srsra, a) +TRANS_FEAT(URSRA, aa64_sve2, gen_gvec_fn_arg_zzi, gen_gvec_ursra, a) +TRANS_FEAT(SRI, aa64_sve2, gen_gvec_fn_arg_zzi, gen_gvec_sri, a) +TRANS_FEAT(SLI, aa64_sve2, gen_gvec_fn_arg_zzi, gen_gvec_sli, a) TRANS_FEAT(SABA, aa64_sve2, gen_gvec_fn_arg_zzz, gen_gvec_saba, a) TRANS_FEAT(UABA, aa64_sve2, gen_gvec_fn_arg_zzz, gen_gvec_uaba, a) -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-44-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 20 +++++++------------- 1 file changed, 7 insertions(+), 13 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool do_vpz_ool(DisasContext *s, arg_rpr_esz *a, } #define DO_VPZ(NAME, name) \ -static bool trans_##NAME(DisasContext *s, arg_rpr_esz *a) \ -{ \ - static gen_helper_gvec_reduc * const fns[4] = { \ + static gen_helper_gvec_reduc * const name##_fns[4] = { \ gen_helper_sve_##name##_b, gen_helper_sve_##name##_h, \ gen_helper_sve_##name##_s, gen_helper_sve_##name##_d, \ }; \ - return do_vpz_ool(s, a, fns[a->esz]); \ -} + TRANS_FEAT(NAME, aa64_sve, do_vpz_ool, a, name##_fns[a->esz]) DO_VPZ(ORV, orv) DO_VPZ(ANDV, andv) @@ -XXX,XX +XXX,XX @@ DO_VPZ(UMAXV, umaxv) DO_VPZ(SMINV, sminv) DO_VPZ(UMINV, uminv) -static bool trans_SADDV(DisasContext *s, arg_rpr_esz *a) -{ - static gen_helper_gvec_reduc * const fns[4] = { - gen_helper_sve_saddv_b, gen_helper_sve_saddv_h, - gen_helper_sve_saddv_s, NULL - }; - return do_vpz_ool(s, a, fns[a->esz]); -} +static gen_helper_gvec_reduc * const saddv_fns[4] = { + gen_helper_sve_saddv_b, gen_helper_sve_saddv_h, + gen_helper_sve_saddv_s, NULL +}; +TRANS_FEAT(SADDV, aa64_sve, do_vpz_ool, a, saddv_fns[a->esz]) #undef DO_VPZ -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-45-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 17 +++-------------- 1 file changed, 3 insertions(+), 14 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool do_shift_imm(DisasContext *s, arg_rri_esz *a, bool asr, return true; } -static bool trans_ASR_zzi(DisasContext *s, arg_rri_esz *a) -{ - return do_shift_imm(s, a, true, tcg_gen_gvec_sari); -} - -static bool trans_LSR_zzi(DisasContext *s, arg_rri_esz *a) -{ - return do_shift_imm(s, a, false, tcg_gen_gvec_shri); -} - -static bool trans_LSL_zzi(DisasContext *s, arg_rri_esz *a) -{ - return do_shift_imm(s, a, false, tcg_gen_gvec_shli); -} +TRANS_FEAT(ASR_zzi, aa64_sve, do_shift_imm, a, true, tcg_gen_gvec_sari) +TRANS_FEAT(LSR_zzi, aa64_sve, do_shift_imm, a, false, tcg_gen_gvec_shri) +TRANS_FEAT(LSL_zzi, aa64_sve, do_shift_imm, a, false, tcg_gen_gvec_shli) #define DO_ZZW(NAME, name) \ static gen_helper_gvec_3 * const name##_zzw_fns[4] = { \ -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Share code between the various shifts using arg_rpri_esz. Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-46-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 68 +++++++++++++++++--------------------- 1 file changed, 30 insertions(+), 38 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool do_movz_zpz(DisasContext *s, int rd, int rn, int pg, return gen_gvec_ool_zzp(s, fns[esz], rd, rn, pg, invert); } +static bool do_shift_zpzi(DisasContext *s, arg_rpri_esz *a, bool asr, + gen_helper_gvec_3 * const fns[4]) +{ + int max; + + if (a->esz < 0) { + /* Invalid tsz encoding -- see tszimm_esz. */ + return false; + } + + /* + * Shift by element size is architecturally valid. + * For arithmetic right-shift, it's the same as by one less. + * For logical shifts and ASRD, it is a zeroing operation. + */ + max = 8 << a->esz; + if (a->imm >= max) { + if (asr) { + a->imm = max - 1; + } else { + return do_movz_zpz(s, a->rd, a->rd, a->pg, a->esz, true); + } + } + return gen_gvec_ool_arg_zpzi(s, fns[a->esz], a); +} + static bool trans_ASR_zpzi(DisasContext *s, arg_rpri_esz *a) { static gen_helper_gvec_3 * const fns[4] = { gen_helper_sve_asr_zpzi_b, gen_helper_sve_asr_zpzi_h, gen_helper_sve_asr_zpzi_s, gen_helper_sve_asr_zpzi_d, }; - if (a->esz < 0) { - /* Invalid tsz encoding -- see tszimm_esz. */ - return false; - } - /* Shift by element size is architecturally valid. For - arithmetic right-shift, it's the same as by one less. */ - a->imm = MIN(a->imm, (8 << a->esz) - 1); - return gen_gvec_ool_arg_zpzi(s, fns[a->esz], a); + return do_shift_zpzi(s, a, true, fns); } static bool trans_LSR_zpzi(DisasContext *s, arg_rpri_esz *a) @@ -XXX,XX +XXX,XX @@ static bool trans_LSR_zpzi(DisasContext *s, arg_rpri_esz *a) gen_helper_sve_lsr_zpzi_b, gen_helper_sve_lsr_zpzi_h, gen_helper_sve_lsr_zpzi_s, gen_helper_sve_lsr_zpzi_d, }; - if (a->esz < 0) { - return false; - } - /* Shift by element size is architecturally valid. - For logical shifts, it is a zeroing operation. */ - if (a->imm >= (8 << a->esz)) { - return do_movz_zpz(s, a->rd, a->rd, a->pg, a->esz, true); - } else { - return gen_gvec_ool_arg_zpzi(s, fns[a->esz], a); - } + return do_shift_zpzi(s, a, false, fns); } static bool trans_LSL_zpzi(DisasContext *s, arg_rpri_esz *a) @@ -XXX,XX +XXX,XX @@ static bool trans_LSL_zpzi(DisasContext *s, arg_rpri_esz *a) gen_helper_sve_lsl_zpzi_b, gen_helper_sve_lsl_zpzi_h, gen_helper_sve_lsl_zpzi_s, gen_helper_sve_lsl_zpzi_d, }; - if (a->esz < 0) { - return false; - } - /* Shift by element size is architecturally valid. - For logical shifts, it is a zeroing operation. */ - if (a->imm >= (8 << a->esz)) { - return do_movz_zpz(s, a->rd, a->rd, a->pg, a->esz, true); - } else { - return gen_gvec_ool_arg_zpzi(s, fns[a->esz], a); - } + return do_shift_zpzi(s, a, false, fns); } static bool trans_ASRD(DisasContext *s, arg_rpri_esz *a) @@ -XXX,XX +XXX,XX @@ static bool trans_ASRD(DisasContext *s, arg_rpri_esz *a) gen_helper_sve_asrd_b, gen_helper_sve_asrd_h, gen_helper_sve_asrd_s, gen_helper_sve_asrd_d, }; - if (a->esz < 0) { - return false; - } - /* Shift by element size is architecturally valid. For arithmetic - right shift for division, it is a zeroing operation. */ - if (a->imm >= (8 << a->esz)) { - return do_movz_zpz(s, a->rd, a->rd, a->pg, a->esz, true); - } else { - return gen_gvec_ool_arg_zpzi(s, fns[a->esz], a); - } + return do_shift_zpzi(s, a, false, fns); } static gen_helper_gvec_3 * const sqshl_zpzi_fns[4] = { -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-47-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 52 +++++++++++++++----------------------- 1 file changed, 20 insertions(+), 32 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool do_shift_zpzi(DisasContext *s, arg_rpri_esz *a, bool asr, return gen_gvec_ool_arg_zpzi(s, fns[a->esz], a); } -static bool trans_ASR_zpzi(DisasContext *s, arg_rpri_esz *a) -{ - static gen_helper_gvec_3 * const fns[4] = { - gen_helper_sve_asr_zpzi_b, gen_helper_sve_asr_zpzi_h, - gen_helper_sve_asr_zpzi_s, gen_helper_sve_asr_zpzi_d, - }; - return do_shift_zpzi(s, a, true, fns); -} +static gen_helper_gvec_3 * const asr_zpzi_fns[4] = { + gen_helper_sve_asr_zpzi_b, gen_helper_sve_asr_zpzi_h, + gen_helper_sve_asr_zpzi_s, gen_helper_sve_asr_zpzi_d, +}; +TRANS_FEAT(ASR_zpzi, aa64_sve, do_shift_zpzi, a, true, asr_zpzi_fns) -static bool trans_LSR_zpzi(DisasContext *s, arg_rpri_esz *a) -{ - static gen_helper_gvec_3 * const fns[4] = { - gen_helper_sve_lsr_zpzi_b, gen_helper_sve_lsr_zpzi_h, - gen_helper_sve_lsr_zpzi_s, gen_helper_sve_lsr_zpzi_d, - }; - return do_shift_zpzi(s, a, false, fns); -} +static gen_helper_gvec_3 * const lsr_zpzi_fns[4] = { + gen_helper_sve_lsr_zpzi_b, gen_helper_sve_lsr_zpzi_h, + gen_helper_sve_lsr_zpzi_s, gen_helper_sve_lsr_zpzi_d, +}; +TRANS_FEAT(LSR_zpzi, aa64_sve, do_shift_zpzi, a, false, lsr_zpzi_fns) -static bool trans_LSL_zpzi(DisasContext *s, arg_rpri_esz *a) -{ - static gen_helper_gvec_3 * const fns[4] = { - gen_helper_sve_lsl_zpzi_b, gen_helper_sve_lsl_zpzi_h, - gen_helper_sve_lsl_zpzi_s, gen_helper_sve_lsl_zpzi_d, - }; - return do_shift_zpzi(s, a, false, fns); -} +static gen_helper_gvec_3 * const lsl_zpzi_fns[4] = { + gen_helper_sve_lsl_zpzi_b, gen_helper_sve_lsl_zpzi_h, + gen_helper_sve_lsl_zpzi_s, gen_helper_sve_lsl_zpzi_d, +}; +TRANS_FEAT(LSL_zpzi, aa64_sve, do_shift_zpzi, a, false, lsl_zpzi_fns) -static bool trans_ASRD(DisasContext *s, arg_rpri_esz *a) -{ - static gen_helper_gvec_3 * const fns[4] = { - gen_helper_sve_asrd_b, gen_helper_sve_asrd_h, - gen_helper_sve_asrd_s, gen_helper_sve_asrd_d, - }; - return do_shift_zpzi(s, a, false, fns); -} +static gen_helper_gvec_3 * const asrd_fns[4] = { + gen_helper_sve_asrd_b, gen_helper_sve_asrd_h, + gen_helper_sve_asrd_s, gen_helper_sve_asrd_d, +}; +TRANS_FEAT(ASRD, aa64_sve, do_shift_zpzi, a, false, asrd_fns) static gen_helper_gvec_3 * const sqshl_zpzi_fns[4] = { gen_helper_sve2_sqshl_zpzi_b, gen_helper_sve2_sqshl_zpzi_h, -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Remove the DO_ZPZZZ macro, as it had just the two uses. Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-48-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 23 ++++++++++------------- 1 file changed, 10 insertions(+), 13 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool do_zpzzz_ool(DisasContext *s, arg_rprrr_esz *a, return true; } -#define DO_ZPZZZ(NAME, name) \ -static bool trans_##NAME(DisasContext *s, arg_rprrr_esz *a) \ -{ \ - static gen_helper_gvec_5 * const fns[4] = { \ - gen_helper_sve_##name##_b, gen_helper_sve_##name##_h, \ - gen_helper_sve_##name##_s, gen_helper_sve_##name##_d, \ - }; \ - return do_zpzzz_ool(s, a, fns[a->esz]); \ -} +static gen_helper_gvec_5 * const mla_fns[4] = { + gen_helper_sve_mla_b, gen_helper_sve_mla_h, + gen_helper_sve_mla_s, gen_helper_sve_mla_d, +}; +TRANS_FEAT(MLA, aa64_sve, do_zpzzz_ool, a, mla_fns[a->esz]) -DO_ZPZZZ(MLA, mla) -DO_ZPZZZ(MLS, mls) - -#undef DO_ZPZZZ +static gen_helper_gvec_5 * const mls_fns[4] = { + gen_helper_sve_mls_b, gen_helper_sve_mls_h, + gen_helper_sve_mls_s, gen_helper_sve_mls_d, +}; +TRANS_FEAT(MLS, aa64_sve, do_zpzzz_ool, a, mls_fns[a->esz]) /* *** SVE Index Generation Group -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-49-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 53 ++++++++++++++++++-------------------- 1 file changed, 25 insertions(+), 28 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ TRANS_FEAT(MLS, aa64_sve, do_zpzzz_ool, a, mls_fns[a->esz]) *** SVE Index Generation Group */ -static void do_index(DisasContext *s, int esz, int rd, +static bool do_index(DisasContext *s, int esz, int rd, TCGv_i64 start, TCGv_i64 incr) { - unsigned vsz = vec_full_reg_size(s); - TCGv_i32 desc = tcg_constant_i32(simd_desc(vsz, vsz, 0)); - TCGv_ptr t_zd = tcg_temp_new_ptr(); + unsigned vsz; + TCGv_i32 desc; + TCGv_ptr t_zd; + + if (!sve_access_check(s)) { + return true; + } + + vsz = vec_full_reg_size(s); + desc = tcg_constant_i32(simd_desc(vsz, vsz, 0)); + t_zd = tcg_temp_new_ptr(); tcg_gen_addi_ptr(t_zd, cpu_env, vec_full_reg_offset(s, rd)); if (esz == 3) { @@ -XXX,XX +XXX,XX @@ static void do_index(DisasContext *s, int esz, int rd, tcg_temp_free_i32(i32); } tcg_temp_free_ptr(t_zd); + return true; } static bool trans_INDEX_ii(DisasContext *s, arg_INDEX_ii *a) { - if (sve_access_check(s)) { - TCGv_i64 start = tcg_constant_i64(a->imm1); - TCGv_i64 incr = tcg_constant_i64(a->imm2); - do_index(s, a->esz, a->rd, start, incr); - } - return true; + TCGv_i64 start = tcg_constant_i64(a->imm1); + TCGv_i64 incr = tcg_constant_i64(a->imm2); + return do_index(s, a->esz, a->rd, start, incr); } static bool trans_INDEX_ir(DisasContext *s, arg_INDEX_ir *a) { - if (sve_access_check(s)) { - TCGv_i64 start = tcg_constant_i64(a->imm); - TCGv_i64 incr = cpu_reg(s, a->rm); - do_index(s, a->esz, a->rd, start, incr); - } - return true; + TCGv_i64 start = tcg_constant_i64(a->imm); + TCGv_i64 incr = cpu_reg(s, a->rm); + return do_index(s, a->esz, a->rd, start, incr); } static bool trans_INDEX_ri(DisasContext *s, arg_INDEX_ri *a) { - if (sve_access_check(s)) { - TCGv_i64 start = cpu_reg(s, a->rn); - TCGv_i64 incr = tcg_constant_i64(a->imm); - do_index(s, a->esz, a->rd, start, incr); - } - return true; + TCGv_i64 start = cpu_reg(s, a->rn); + TCGv_i64 incr = tcg_constant_i64(a->imm); + return do_index(s, a->esz, a->rd, start, incr); } static bool trans_INDEX_rr(DisasContext *s, arg_INDEX_rr *a) { - if (sve_access_check(s)) { - TCGv_i64 start = cpu_reg(s, a->rn); - TCGv_i64 incr = cpu_reg(s, a->rm); - do_index(s, a->esz, a->rd, start, incr); - } - return true; + TCGv_i64 start = cpu_reg(s, a->rn); + TCGv_i64 incr = cpu_reg(s, a->rm); + return do_index(s, a->esz, a->rd, start, incr); } /* -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-50-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 35 ++++++++--------------------------- 1 file changed, 8 insertions(+), 27 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool do_index(DisasContext *s, int esz, int rd, return true; } -static bool trans_INDEX_ii(DisasContext *s, arg_INDEX_ii *a) -{ - TCGv_i64 start = tcg_constant_i64(a->imm1); - TCGv_i64 incr = tcg_constant_i64(a->imm2); - return do_index(s, a->esz, a->rd, start, incr); -} - -static bool trans_INDEX_ir(DisasContext *s, arg_INDEX_ir *a) -{ - TCGv_i64 start = tcg_constant_i64(a->imm); - TCGv_i64 incr = cpu_reg(s, a->rm); - return do_index(s, a->esz, a->rd, start, incr); -} - -static bool trans_INDEX_ri(DisasContext *s, arg_INDEX_ri *a) -{ - TCGv_i64 start = cpu_reg(s, a->rn); - TCGv_i64 incr = tcg_constant_i64(a->imm); - return do_index(s, a->esz, a->rd, start, incr); -} - -static bool trans_INDEX_rr(DisasContext *s, arg_INDEX_rr *a) -{ - TCGv_i64 start = cpu_reg(s, a->rn); - TCGv_i64 incr = cpu_reg(s, a->rm); - return do_index(s, a->esz, a->rd, start, incr); -} +TRANS_FEAT(INDEX_ii, aa64_sve, do_index, a->esz, a->rd, + tcg_constant_i64(a->imm1), tcg_constant_i64(a->imm2)) +TRANS_FEAT(INDEX_ir, aa64_sve, do_index, a->esz, a->rd, + tcg_constant_i64(a->imm), cpu_reg(s, a->rm)) +TRANS_FEAT(INDEX_ri, aa64_sve, do_index, a->esz, a->rd, + cpu_reg(s, a->rn), tcg_constant_i64(a->imm)) +TRANS_FEAT(INDEX_rr, aa64_sve, do_index, a->esz, a->rd, + cpu_reg(s, a->rn), cpu_reg(s, a->rm)) /* *** SVE Stack Allocation Group -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-51-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 23 ++++------------------- 1 file changed, 4 insertions(+), 19 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool do_adr(DisasContext *s, arg_rrri *a, gen_helper_gvec_3 *fn) return gen_gvec_ool_zzz(s, fn, a->rd, a->rn, a->rm, a->imm); } -static bool trans_ADR_p32(DisasContext *s, arg_rrri *a) -{ - return do_adr(s, a, gen_helper_sve_adr_p32); -} - -static bool trans_ADR_p64(DisasContext *s, arg_rrri *a) -{ - return do_adr(s, a, gen_helper_sve_adr_p64); -} - -static bool trans_ADR_s32(DisasContext *s, arg_rrri *a) -{ - return do_adr(s, a, gen_helper_sve_adr_s32); -} - -static bool trans_ADR_u32(DisasContext *s, arg_rrri *a) -{ - return do_adr(s, a, gen_helper_sve_adr_u32); -} +TRANS_FEAT(ADR_p32, aa64_sve, do_adr, a, gen_helper_sve_adr_p32) +TRANS_FEAT(ADR_p64, aa64_sve, do_adr, a, gen_helper_sve_adr_p64) +TRANS_FEAT(ADR_s32, aa64_sve, do_adr, a, gen_helper_sve_adr_s32) +TRANS_FEAT(ADR_u32, aa64_sve, do_adr, a, gen_helper_sve_adr_u32) /* *** SVE Integer Misc - Unpredicated Group -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-52-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 19 +++++-------------- 1 file changed, 5 insertions(+), 14 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool do_predset(DisasContext *s, int esz, int rd, int pat, bool setflag) return true; } -static bool trans_PTRUE(DisasContext *s, arg_PTRUE *a) -{ - return do_predset(s, a->esz, a->rd, a->pat, a->s); -} +TRANS_FEAT(PTRUE, aa64_sve, do_predset, a->esz, a->rd, a->pat, a->s) -static bool trans_SETFFR(DisasContext *s, arg_SETFFR *a) -{ - /* Note pat == 31 is #all, to set all elements. */ - return do_predset(s, 0, FFR_PRED_NUM, 31, false); -} +/* Note pat == 31 is #all, to set all elements. */ +TRANS_FEAT(SETFFR, aa64_sve, do_predset, 0, FFR_PRED_NUM, 31, false) -static bool trans_PFALSE(DisasContext *s, arg_PFALSE *a) -{ - /* Note pat == 32 is #unimp, to set no elements. */ - return do_predset(s, 0, a->rd, 32, false); -} +/* Note pat == 32 is #unimp, to set no elements. */ +TRANS_FEAT(PFALSE, aa64_sve, do_predset, 0, a->rd, 32, false) static bool trans_RDFFR_p(DisasContext *s, arg_RDFFR_p *a) { -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-53-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 11 ++--------- 1 file changed, 2 insertions(+), 9 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool trans_RDFFR_p(DisasContext *s, arg_RDFFR_p *a) return trans_AND_pppp(s, &alt_a); } -static bool trans_RDFFR(DisasContext *s, arg_RDFFR *a) -{ - return do_mov_p(s, a->rd, FFR_PRED_NUM); -} - -static bool trans_WRFFR(DisasContext *s, arg_WRFFR *a) -{ - return do_mov_p(s, FFR_PRED_NUM, a->rn); -} +TRANS_FEAT(RDFFR, aa64_sve, do_mov_p, a->rd, FFR_PRED_NUM) +TRANS_FEAT(WRFFR, aa64_sve, do_mov_p, FFR_PRED_NUM, a->rn) static bool do_pfirst_pnext(DisasContext *s, arg_rr_esz *a, void (*gen_fn)(TCGv_i32, TCGv_ptr, -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-54-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 11 ++--------- 1 file changed, 2 insertions(+), 9 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool do_pfirst_pnext(DisasContext *s, arg_rr_esz *a, return true; } -static bool trans_PFIRST(DisasContext *s, arg_rr_esz *a) -{ - return do_pfirst_pnext(s, a, gen_helper_sve_pfirst); -} - -static bool trans_PNEXT(DisasContext *s, arg_rr_esz *a) -{ - return do_pfirst_pnext(s, a, gen_helper_sve_pnext); -} +TRANS_FEAT(PFIRST, aa64_sve, do_pfirst_pnext, a, gen_helper_sve_pfirst) +TRANS_FEAT(PNEXT, aa64_sve, do_pfirst_pnext, a, gen_helper_sve_pnext) /* *** SVE Element Count Group -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-55-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 14 ++------------ 1 file changed, 2 insertions(+), 12 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool do_EXT(DisasContext *s, int rd, int rn, int rm, int imm) return true; } -static bool trans_EXT(DisasContext *s, arg_EXT *a) -{ - return do_EXT(s, a->rd, a->rn, a->rm, a->imm); -} - -static bool trans_EXT_sve2(DisasContext *s, arg_rri *a) -{ - if (!dc_isar_feature(aa64_sve2, s)) { - return false; - } - return do_EXT(s, a->rd, a->rn, (a->rn + 1) % 32, a->imm); -} +TRANS_FEAT(EXT, aa64_sve, do_EXT, a->rd, a->rn, a->rm, a->imm) +TRANS_FEAT(EXT_sve2, aa64_sve2, do_EXT, a->rd, a->rn, (a->rn + 1) % 32, a->imm) /* *** SVE Permute - Unpredicated Group -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-56-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 35 ++++++----------------------------- 1 file changed, 6 insertions(+), 29 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool do_perm_pred2(DisasContext *s, arg_rr_esz *a, bool high_odd, return true; } -static bool trans_ZIP1_p(DisasContext *s, arg_rrr_esz *a) -{ - return do_perm_pred3(s, a, 0, gen_helper_sve_zip_p); -} - -static bool trans_ZIP2_p(DisasContext *s, arg_rrr_esz *a) -{ - return do_perm_pred3(s, a, 1, gen_helper_sve_zip_p); -} - -static bool trans_UZP1_p(DisasContext *s, arg_rrr_esz *a) -{ - return do_perm_pred3(s, a, 0, gen_helper_sve_uzp_p); -} - -static bool trans_UZP2_p(DisasContext *s, arg_rrr_esz *a) -{ - return do_perm_pred3(s, a, 1, gen_helper_sve_uzp_p); -} - -static bool trans_TRN1_p(DisasContext *s, arg_rrr_esz *a) -{ - return do_perm_pred3(s, a, 0, gen_helper_sve_trn_p); -} - -static bool trans_TRN2_p(DisasContext *s, arg_rrr_esz *a) -{ - return do_perm_pred3(s, a, 1, gen_helper_sve_trn_p); -} +TRANS_FEAT(ZIP1_p, aa64_sve, do_perm_pred3, a, 0, gen_helper_sve_zip_p) +TRANS_FEAT(ZIP2_p, aa64_sve, do_perm_pred3, a, 1, gen_helper_sve_zip_p) +TRANS_FEAT(UZP1_p, aa64_sve, do_perm_pred3, a, 0, gen_helper_sve_uzp_p) +TRANS_FEAT(UZP2_p, aa64_sve, do_perm_pred3, a, 1, gen_helper_sve_uzp_p) +TRANS_FEAT(TRN1_p, aa64_sve, do_perm_pred3, a, 0, gen_helper_sve_trn_p) +TRANS_FEAT(TRN2_p, aa64_sve, do_perm_pred3, a, 1, gen_helper_sve_trn_p) static bool trans_REV_p(DisasContext *s, arg_rr_esz *a) { -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-57-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 17 +++-------------- 1 file changed, 3 insertions(+), 14 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ TRANS_FEAT(UZP2_p, aa64_sve, do_perm_pred3, a, 1, gen_helper_sve_uzp_p) TRANS_FEAT(TRN1_p, aa64_sve, do_perm_pred3, a, 0, gen_helper_sve_trn_p) TRANS_FEAT(TRN2_p, aa64_sve, do_perm_pred3, a, 1, gen_helper_sve_trn_p) -static bool trans_REV_p(DisasContext *s, arg_rr_esz *a) -{ - return do_perm_pred2(s, a, 0, gen_helper_sve_rev_p); -} - -static bool trans_PUNPKLO(DisasContext *s, arg_PUNPKLO *a) -{ - return do_perm_pred2(s, a, 0, gen_helper_sve_punpk_p); -} - -static bool trans_PUNPKHI(DisasContext *s, arg_PUNPKHI *a) -{ - return do_perm_pred2(s, a, 1, gen_helper_sve_punpk_p); -} +TRANS_FEAT(REV_p, aa64_sve, do_perm_pred2, a, 0, gen_helper_sve_rev_p) +TRANS_FEAT(PUNPKLO, aa64_sve, do_perm_pred2, a, 0, gen_helper_sve_punpk_p) +TRANS_FEAT(PUNPKHI, aa64_sve, do_perm_pred2, a, 1, gen_helper_sve_punpk_p) /* *** SVE Permute - Interleaving Group -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> This is in line with how we treat uzp, and will eliminate the special case code during translation. Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-58-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/sve_helper.c | 6 ++++-- target/arm/translate-sve.c | 12 ++++++------ 2 files changed, 10 insertions(+), 8 deletions(-) diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -XXX,XX +XXX,XX @@ void HELPER(sve_punpk_p)(void *vd, void *vn, uint32_t pred_desc) void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc) \ { \ intptr_t oprsz = simd_oprsz(desc); \ + intptr_t odd_ofs = simd_data(desc); \ intptr_t i, oprsz_2 = oprsz / 2; \ ARMVectorReg tmp_n, tmp_m; \ /* We produce output faster than we consume input. \ @@ -XXX,XX +XXX,XX @@ void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc) \ vm = memcpy(&tmp_m, vm, oprsz_2); \ } \ for (i = 0; i < oprsz_2; i += sizeof(TYPE)) { \ - *(TYPE *)(vd + H(2 * i + 0)) = *(TYPE *)(vn + H(i)); \ - *(TYPE *)(vd + H(2 * i + sizeof(TYPE))) = *(TYPE *)(vm + H(i)); \ + *(TYPE *)(vd + H(2 * i + 0)) = *(TYPE *)(vn + odd_ofs + H(i)); \ + *(TYPE *)(vd + H(2 * i + sizeof(TYPE))) = \ + *(TYPE *)(vm + odd_ofs + H(i)); \ } \ if (sizeof(TYPE) == 16 && unlikely(oprsz & 16)) { \ memset(vd + oprsz - 16, 0, 16); \ diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool do_zip(DisasContext *s, arg_rrr_esz *a, bool high) unsigned vsz = vec_full_reg_size(s); unsigned high_ofs = high ? vsz / 2 : 0; tcg_gen_gvec_3_ool(vec_full_reg_offset(s, a->rd), - vec_full_reg_offset(s, a->rn) + high_ofs, - vec_full_reg_offset(s, a->rm) + high_ofs, - vsz, vsz, 0, fns[a->esz]); + vec_full_reg_offset(s, a->rn), + vec_full_reg_offset(s, a->rm), + vsz, vsz, high_ofs, fns[a->esz]); } return true; } @@ -XXX,XX +XXX,XX @@ static bool do_zip_q(DisasContext *s, arg_rrr_esz *a, bool high) unsigned vsz = vec_full_reg_size(s); unsigned high_ofs = high ? QEMU_ALIGN_DOWN(vsz, 32) / 2 : 0; tcg_gen_gvec_3_ool(vec_full_reg_offset(s, a->rd), - vec_full_reg_offset(s, a->rn) + high_ofs, - vec_full_reg_offset(s, a->rm) + high_ofs, - vsz, vsz, 0, gen_helper_sve2_zip_q); + vec_full_reg_offset(s, a->rn), + vec_full_reg_offset(s, a->rm), + vsz, vsz, high_ofs, gen_helper_sve2_zip_q); } return true; } -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-59-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 25 +++++++------------------ 1 file changed, 7 insertions(+), 18 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool do_zip(DisasContext *s, arg_rrr_esz *a, bool high) gen_helper_sve_zip_b, gen_helper_sve_zip_h, gen_helper_sve_zip_s, gen_helper_sve_zip_d, }; + unsigned vsz = vec_full_reg_size(s); + unsigned high_ofs = high ? vsz / 2 : 0; - if (sve_access_check(s)) { - unsigned vsz = vec_full_reg_size(s); - unsigned high_ofs = high ? vsz / 2 : 0; - tcg_gen_gvec_3_ool(vec_full_reg_offset(s, a->rd), - vec_full_reg_offset(s, a->rn), - vec_full_reg_offset(s, a->rm), - vsz, vsz, high_ofs, fns[a->esz]); - } - return true; + return gen_gvec_ool_arg_zzz(s, fns[a->esz], a, high_ofs); } static bool trans_ZIP1_z(DisasContext *s, arg_rrr_esz *a) @@ -XXX,XX +XXX,XX @@ static bool trans_ZIP2_z(DisasContext *s, arg_rrr_esz *a) static bool do_zip_q(DisasContext *s, arg_rrr_esz *a, bool high) { + unsigned vsz = vec_full_reg_size(s); + unsigned high_ofs = high ? QEMU_ALIGN_DOWN(vsz, 32) / 2 : 0; + if (!dc_isar_feature(aa64_sve_f64mm, s)) { return false; } - if (sve_access_check(s)) { - unsigned vsz = vec_full_reg_size(s); - unsigned high_ofs = high ? QEMU_ALIGN_DOWN(vsz, 32) / 2 : 0; - tcg_gen_gvec_3_ool(vec_full_reg_offset(s, a->rd), - vec_full_reg_offset(s, a->rn), - vec_full_reg_offset(s, a->rm), - vsz, vsz, high_ofs, gen_helper_sve2_zip_q); - } - return true; + return gen_gvec_ool_arg_zzz(s, gen_helper_sve2_zip_q, a, high_ofs); } static bool trans_ZIP1_q(DisasContext *s, arg_rrr_esz *a) -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Convert SVE translation functions using do_zip* to use TRANS_FEAT and gen_gvec_ool_arg_zzz. Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-60-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 54 +++++++++----------------------------- 1 file changed, 13 insertions(+), 41 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ TRANS_FEAT(PUNPKHI, aa64_sve, do_perm_pred2, a, 1, gen_helper_sve_punpk_p) *** SVE Permute - Interleaving Group */ -static bool do_zip(DisasContext *s, arg_rrr_esz *a, bool high) -{ - static gen_helper_gvec_3 * const fns[4] = { - gen_helper_sve_zip_b, gen_helper_sve_zip_h, - gen_helper_sve_zip_s, gen_helper_sve_zip_d, - }; - unsigned vsz = vec_full_reg_size(s); - unsigned high_ofs = high ? vsz / 2 : 0; +static gen_helper_gvec_3 * const zip_fns[4] = { + gen_helper_sve_zip_b, gen_helper_sve_zip_h, + gen_helper_sve_zip_s, gen_helper_sve_zip_d, +}; +TRANS_FEAT(ZIP1_z, aa64_sve, gen_gvec_ool_arg_zzz, + zip_fns[a->esz], a, 0) +TRANS_FEAT(ZIP2_z, aa64_sve, gen_gvec_ool_arg_zzz, + zip_fns[a->esz], a, vec_full_reg_size(s) / 2) - return gen_gvec_ool_arg_zzz(s, fns[a->esz], a, high_ofs); -} - -static bool trans_ZIP1_z(DisasContext *s, arg_rrr_esz *a) -{ - return do_zip(s, a, false); -} - -static bool trans_ZIP2_z(DisasContext *s, arg_rrr_esz *a) -{ - return do_zip(s, a, true); -} - -static bool do_zip_q(DisasContext *s, arg_rrr_esz *a, bool high) -{ - unsigned vsz = vec_full_reg_size(s); - unsigned high_ofs = high ? QEMU_ALIGN_DOWN(vsz, 32) / 2 : 0; - - if (!dc_isar_feature(aa64_sve_f64mm, s)) { - return false; - } - return gen_gvec_ool_arg_zzz(s, gen_helper_sve2_zip_q, a, high_ofs); -} - -static bool trans_ZIP1_q(DisasContext *s, arg_rrr_esz *a) -{ - return do_zip_q(s, a, false); -} - -static bool trans_ZIP2_q(DisasContext *s, arg_rrr_esz *a) -{ - return do_zip_q(s, a, true); -} +TRANS_FEAT(ZIP1_q, aa64_sve_f64mm, gen_gvec_ool_arg_zzz, + gen_helper_sve2_zip_q, a, 0) +TRANS_FEAT(ZIP2_q, aa64_sve_f64mm, gen_gvec_ool_arg_zzz, + gen_helper_sve2_zip_q, a, + QEMU_ALIGN_DOWN(vec_full_reg_size(s), 32) / 2) static gen_helper_gvec_3 * const uzp_fns[4] = { gen_helper_sve_uzp_b, gen_helper_sve_uzp_h, -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-61-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 11 ++--------- 1 file changed, 2 insertions(+), 9 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool do_clast_vector(DisasContext *s, arg_rprr_esz *a, bool before) return true; } -static bool trans_CLASTA_z(DisasContext *s, arg_rprr_esz *a) -{ - return do_clast_vector(s, a, false); -} - -static bool trans_CLASTB_z(DisasContext *s, arg_rprr_esz *a) -{ - return do_clast_vector(s, a, true); -} +TRANS_FEAT(CLASTA_z, aa64_sve, do_clast_vector, a, false) +TRANS_FEAT(CLASTB_z, aa64_sve, do_clast_vector, a, true) /* Compute CLAST for a scalar. */ static void do_clast_scalar(DisasContext *s, int esz, int pg, int rm, -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-62-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 11 ++--------- 1 file changed, 2 insertions(+), 9 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool do_clast_fp(DisasContext *s, arg_rpr_esz *a, bool before) return true; } -static bool trans_CLASTA_v(DisasContext *s, arg_rpr_esz *a) -{ - return do_clast_fp(s, a, false); -} - -static bool trans_CLASTB_v(DisasContext *s, arg_rpr_esz *a) -{ - return do_clast_fp(s, a, true); -} +TRANS_FEAT(CLASTA_v, aa64_sve, do_clast_fp, a, false) +TRANS_FEAT(CLASTB_v, aa64_sve, do_clast_fp, a, true) /* Compute CLAST for a Xreg. */ static bool do_clast_general(DisasContext *s, arg_rpr_esz *a, bool before) -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-63-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 11 ++--------- 1 file changed, 2 insertions(+), 9 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool do_clast_general(DisasContext *s, arg_rpr_esz *a, bool before) return true; } -static bool trans_CLASTA_r(DisasContext *s, arg_rpr_esz *a) -{ - return do_clast_general(s, a, false); -} - -static bool trans_CLASTB_r(DisasContext *s, arg_rpr_esz *a) -{ - return do_clast_general(s, a, true); -} +TRANS_FEAT(CLASTA_r, aa64_sve, do_clast_general, a, false) +TRANS_FEAT(CLASTB_r, aa64_sve, do_clast_general, a, true) /* Compute LAST for a scalar. */ static TCGv_i64 do_last_scalar(DisasContext *s, int esz, -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-64-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 11 ++--------- 1 file changed, 2 insertions(+), 9 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool do_last_fp(DisasContext *s, arg_rpr_esz *a, bool before) return true; } -static bool trans_LASTA_v(DisasContext *s, arg_rpr_esz *a) -{ - return do_last_fp(s, a, false); -} - -static bool trans_LASTB_v(DisasContext *s, arg_rpr_esz *a) -{ - return do_last_fp(s, a, true); -} +TRANS_FEAT(LASTA_v, aa64_sve, do_last_fp, a, false) +TRANS_FEAT(LASTB_v, aa64_sve, do_last_fp, a, true) /* Compute LAST for a Xreg. */ static bool do_last_general(DisasContext *s, arg_rpr_esz *a, bool before) -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-65-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 11 ++--------- 1 file changed, 2 insertions(+), 9 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool do_last_general(DisasContext *s, arg_rpr_esz *a, bool before) return true; } -static bool trans_LASTA_r(DisasContext *s, arg_rpr_esz *a) -{ - return do_last_general(s, a, false); -} - -static bool trans_LASTB_r(DisasContext *s, arg_rpr_esz *a) -{ - return do_last_general(s, a, true); -} +TRANS_FEAT(LASTA_r, aa64_sve, do_last_general, a, false) +TRANS_FEAT(LASTB_r, aa64_sve, do_last_general, a, true) static bool trans_CPY_m_r(DisasContext *s, arg_rpr_esz *a) { -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-66-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 17 ++++------------- 1 file changed, 4 insertions(+), 13 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ TRANS_FEAT(REVH, aa64_sve, gen_gvec_ool_arg_zpz, revh_fns[a->esz], a, 0) TRANS_FEAT(REVW, aa64_sve, gen_gvec_ool_arg_zpz, a->esz == 3 ? gen_helper_sve_revw_d : NULL, a, 0) -static bool trans_SPLICE(DisasContext *s, arg_rprr_esz *a) -{ - return gen_gvec_ool_zzzp(s, gen_helper_sve_splice, - a->rd, a->rn, a->rm, a->pg, a->esz); -} +TRANS_FEAT(SPLICE, aa64_sve, gen_gvec_ool_arg_zpzz, + gen_helper_sve_splice, a, a->esz) -static bool trans_SPLICE_sve2(DisasContext *s, arg_rpr_esz *a) -{ - if (!dc_isar_feature(aa64_sve2, s)) { - return false; - } - return gen_gvec_ool_zzzp(s, gen_helper_sve_splice, - a->rd, a->rn, (a->rn + 1) % 32, a->pg, a->esz); -} +TRANS_FEAT(SPLICE_sve2, aa64_sve2, gen_gvec_ool_zzzp, gen_helper_sve_splice, + a->rd, a->rn, (a->rn + 1) % 32, a->pg, a->esz) /* *** SVE Integer Compare - Vectors Group -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-67-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 28 ++++++++++++---------------- 1 file changed, 12 insertions(+), 16 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool do_ppzz_flags(DisasContext *s, arg_rprr_esz *a, } #define DO_PPZZ(NAME, name) \ -static bool trans_##NAME##_ppzz(DisasContext *s, arg_rprr_esz *a) \ -{ \ - static gen_helper_gvec_flags_4 * const fns[4] = { \ - gen_helper_sve_##name##_ppzz_b, gen_helper_sve_##name##_ppzz_h, \ - gen_helper_sve_##name##_ppzz_s, gen_helper_sve_##name##_ppzz_d, \ - }; \ - return do_ppzz_flags(s, a, fns[a->esz]); \ -} + static gen_helper_gvec_flags_4 * const name##_ppzz_fns[4] = { \ + gen_helper_sve_##name##_ppzz_b, gen_helper_sve_##name##_ppzz_h, \ + gen_helper_sve_##name##_ppzz_s, gen_helper_sve_##name##_ppzz_d, \ + }; \ + TRANS_FEAT(NAME##_ppzz, aa64_sve, do_ppzz_flags, \ + a, name##_ppzz_fns[a->esz]) DO_PPZZ(CMPEQ, cmpeq) DO_PPZZ(CMPNE, cmpne) @@ -XXX,XX +XXX,XX @@ DO_PPZZ(CMPHS, cmphs) #undef DO_PPZZ #define DO_PPZW(NAME, name) \ -static bool trans_##NAME##_ppzw(DisasContext *s, arg_rprr_esz *a) \ -{ \ - static gen_helper_gvec_flags_4 * const fns[4] = { \ - gen_helper_sve_##name##_ppzw_b, gen_helper_sve_##name##_ppzw_h, \ - gen_helper_sve_##name##_ppzw_s, NULL \ - }; \ - return do_ppzz_flags(s, a, fns[a->esz]); \ -} + static gen_helper_gvec_flags_4 * const name##_ppzw_fns[4] = { \ + gen_helper_sve_##name##_ppzw_b, gen_helper_sve_##name##_ppzw_h, \ + gen_helper_sve_##name##_ppzw_s, NULL \ + }; \ + TRANS_FEAT(NAME##_ppzw, aa64_sve, do_ppzz_flags, \ + a, name##_ppzw_fns[a->esz]) DO_PPZW(CMPEQ, cmpeq) DO_PPZW(CMPNE, cmpne) -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-68-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 28 ++++++++-------------------- 1 file changed, 8 insertions(+), 20 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ DO_SVE2_ZZZ_NARROW(SUBHNT, subhnt) DO_SVE2_ZZZ_NARROW(RSUBHNB, rsubhnb) DO_SVE2_ZZZ_NARROW(RSUBHNT, rsubhnt) -static bool do_sve2_ppzz_flags(DisasContext *s, arg_rprr_esz *a, - gen_helper_gvec_flags_4 *fn) -{ - if (!dc_isar_feature(aa64_sve2, s)) { - return false; - } - return do_ppzz_flags(s, a, fn); -} +static gen_helper_gvec_flags_4 * const match_fns[4] = { + gen_helper_sve2_match_ppzz_b, gen_helper_sve2_match_ppzz_h, NULL, NULL +}; +TRANS_FEAT(MATCH, aa64_sve2, do_ppzz_flags, a, match_fns[a->esz]) -#define DO_SVE2_PPZZ_MATCH(NAME, name) \ -static bool trans_##NAME(DisasContext *s, arg_rprr_esz *a) \ -{ \ - static gen_helper_gvec_flags_4 * const fns[4] = { \ - gen_helper_sve2_##name##_ppzz_b, gen_helper_sve2_##name##_ppzz_h, \ - NULL, NULL \ - }; \ - return do_sve2_ppzz_flags(s, a, fns[a->esz]); \ -} - -DO_SVE2_PPZZ_MATCH(MATCH, match) -DO_SVE2_PPZZ_MATCH(NMATCH, nmatch) +static gen_helper_gvec_flags_4 * const nmatch_fns[4] = { + gen_helper_sve2_nmatch_ppzz_b, gen_helper_sve2_nmatch_ppzz_h, NULL, NULL +}; +TRANS_FEAT(NMATCH, aa64_sve2, do_ppzz_flags, a, nmatch_fns[a->esz]) static gen_helper_gvec_4 * const histcnt_fns[4] = { NULL, NULL, gen_helper_sve2_histcnt_s, gen_helper_sve2_histcnt_d -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-69-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool do_ppzi_flags(DisasContext *s, arg_rpri_esz *a, } #define DO_PPZI(NAME, name) \ -static bool trans_##NAME##_ppzi(DisasContext *s, arg_rpri_esz *a) \ -{ \ - static gen_helper_gvec_flags_3 * const fns[4] = { \ + static gen_helper_gvec_flags_3 * const name##_ppzi_fns[4] = { \ gen_helper_sve_##name##_ppzi_b, gen_helper_sve_##name##_ppzi_h, \ gen_helper_sve_##name##_ppzi_s, gen_helper_sve_##name##_ppzi_d, \ }; \ - return do_ppzi_flags(s, a, fns[a->esz]); \ -} + TRANS_FEAT(NAME##_ppzi, aa64_sve, do_ppzi_flags, a, \ + name##_ppzi_fns[a->esz]) DO_PPZI(CMPEQ, cmpeq) DO_PPZI(CMPNE, cmpne) -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-70-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 45 ++++++++++++-------------------------- 1 file changed, 14 insertions(+), 31 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool do_brk2(DisasContext *s, arg_rpr_s *a, return true; } -static bool trans_BRKPA(DisasContext *s, arg_rprr_s *a) -{ - return do_brk3(s, a, gen_helper_sve_brkpa, gen_helper_sve_brkpas); -} +TRANS_FEAT(BRKPA, aa64_sve, do_brk3, a, + gen_helper_sve_brkpa, gen_helper_sve_brkpas) +TRANS_FEAT(BRKPB, aa64_sve, do_brk3, a, + gen_helper_sve_brkpb, gen_helper_sve_brkpbs) -static bool trans_BRKPB(DisasContext *s, arg_rprr_s *a) -{ - return do_brk3(s, a, gen_helper_sve_brkpb, gen_helper_sve_brkpbs); -} +TRANS_FEAT(BRKA_m, aa64_sve, do_brk2, a, + gen_helper_sve_brka_m, gen_helper_sve_brkas_m) +TRANS_FEAT(BRKB_m, aa64_sve, do_brk2, a, + gen_helper_sve_brkb_m, gen_helper_sve_brkbs_m) -static bool trans_BRKA_m(DisasContext *s, arg_rpr_s *a) -{ - return do_brk2(s, a, gen_helper_sve_brka_m, gen_helper_sve_brkas_m); -} +TRANS_FEAT(BRKA_z, aa64_sve, do_brk2, a, + gen_helper_sve_brka_z, gen_helper_sve_brkas_z) +TRANS_FEAT(BRKB_z, aa64_sve, do_brk2, a, + gen_helper_sve_brkb_z, gen_helper_sve_brkbs_z) -static bool trans_BRKB_m(DisasContext *s, arg_rpr_s *a) -{ - return do_brk2(s, a, gen_helper_sve_brkb_m, gen_helper_sve_brkbs_m); -} - -static bool trans_BRKA_z(DisasContext *s, arg_rpr_s *a) -{ - return do_brk2(s, a, gen_helper_sve_brka_z, gen_helper_sve_brkas_z); -} - -static bool trans_BRKB_z(DisasContext *s, arg_rpr_s *a) -{ - return do_brk2(s, a, gen_helper_sve_brkb_z, gen_helper_sve_brkbs_z); -} - -static bool trans_BRKN(DisasContext *s, arg_rpr_s *a) -{ - return do_brk2(s, a, gen_helper_sve_brkn, gen_helper_sve_brkns); -} +TRANS_FEAT(BRKN, aa64_sve, do_brk2, a, + gen_helper_sve_brkn, gen_helper_sve_brkns) /* *** SVE Predicate Count Group -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-71-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 10 +--------- 1 file changed, 1 insertion(+), 9 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool trans_SUBR_zzi(DisasContext *s, arg_rri_esz *a) return true; } -static bool trans_MUL_zzi(DisasContext *s, arg_rri_esz *a) -{ - if (sve_access_check(s)) { - unsigned vsz = vec_full_reg_size(s); - tcg_gen_gvec_muli(a->esz, vec_full_reg_offset(s, a->rd), - vec_full_reg_offset(s, a->rn), a->imm, vsz, vsz); - } - return true; -} +TRANS_FEAT(MUL_zzi, aa64_sve, gen_gvec_fn_arg_zzi, tcg_gen_gvec_muli, a) static bool do_zzi_sat(DisasContext *s, arg_rri_esz *a, bool u, bool d) { -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Remove the unparsed extraction in trans_DUP_i, which is intended to reject an 8-bit shift of an 8-bit constant for 8-bit element. Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-72-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/sve.decode | 5 ++++- target/arm/translate-sve.c | 10 ++++++---- 2 files changed, 10 insertions(+), 5 deletions(-) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -XXX,XX +XXX,XX @@ WHILE_ptr 00100101 esz:2 1 rm:5 001 100 rn:5 rw:1 rd:4 FDUP 00100101 esz:2 111 00 1110 imm:8 rd:5 # SVE broadcast integer immediate (unpredicated) -DUP_i 00100101 esz:2 111 00 011 . ........ rd:5 imm=%sh8_i8s +{ + INVALID 00100101 00 111 00 011 1 -------- ----- + DUP_i 00100101 esz:2 111 00 011 . ........ rd:5 imm=%sh8_i8s +} # SVE integer add/subtract immediate (unpredicated) ADD_zzi 00100101 .. 100 000 11 . ........ ..... @rdn_sh_i8u diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ const uint64_t pred_esz_masks[4] = { 0x1111111111111111ull, 0x0101010101010101ull }; +static bool trans_INVALID(DisasContext *s, arg_INVALID *a) +{ + unallocated_encoding(s); + return true; +} + /* *** SVE Logical - Unpredicated Group */ @@ -XXX,XX +XXX,XX @@ static bool trans_FDUP(DisasContext *s, arg_FDUP *a) static bool trans_DUP_i(DisasContext *s, arg_DUP_i *a) { - if (a->esz == 0 && extract32(s->insn, 13, 1)) { - return false; - } if (sve_access_check(s)) { unsigned vsz = vec_full_reg_size(s); int dofs = vec_full_reg_offset(s, a->rd); - tcg_gen_gvec_dup_imm(a->esz, dofs, vsz, vsz, a->imm); } return true; -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Remove the unparsed extractions in trans_ADD_zzi, trans_SUBR_zzi, and do_zzi_sat which are intended to reject an 8-bit shift of an 8-bit constant for 8-bit element. Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-73-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/sve.decode | 35 ++++++++++++++++++++++++++++------- target/arm/translate-sve.c | 9 --------- 2 files changed, 28 insertions(+), 16 deletions(-) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -XXX,XX +XXX,XX @@ FDUP 00100101 esz:2 111 00 1110 imm:8 rd:5 } # SVE integer add/subtract immediate (unpredicated) -ADD_zzi 00100101 .. 100 000 11 . ........ ..... @rdn_sh_i8u -SUB_zzi 00100101 .. 100 001 11 . ........ ..... @rdn_sh_i8u -SUBR_zzi 00100101 .. 100 011 11 . ........ ..... @rdn_sh_i8u -SQADD_zzi 00100101 .. 100 100 11 . ........ ..... @rdn_sh_i8u -UQADD_zzi 00100101 .. 100 101 11 . ........ ..... @rdn_sh_i8u -SQSUB_zzi 00100101 .. 100 110 11 . ........ ..... @rdn_sh_i8u -UQSUB_zzi 00100101 .. 100 111 11 . ........ ..... @rdn_sh_i8u +{ + INVALID 00100101 00 100 000 11 1 -------- ----- + ADD_zzi 00100101 .. 100 000 11 . ........ ..... @rdn_sh_i8u +} +{ + INVALID 00100101 00 100 001 11 1 -------- ----- + SUB_zzi 00100101 .. 100 001 11 . ........ ..... @rdn_sh_i8u +} +{ + INVALID 00100101 00 100 011 11 1 -------- ----- + SUBR_zzi 00100101 .. 100 011 11 . ........ ..... @rdn_sh_i8u +} +{ + INVALID 00100101 00 100 100 11 1 -------- ----- + SQADD_zzi 00100101 .. 100 100 11 . ........ ..... @rdn_sh_i8u +} +{ + INVALID 00100101 00 100 101 11 1 -------- ----- + UQADD_zzi 00100101 .. 100 101 11 . ........ ..... @rdn_sh_i8u +} +{ + INVALID 00100101 00 100 110 11 1 -------- ----- + SQSUB_zzi 00100101 .. 100 110 11 . ........ ..... @rdn_sh_i8u +} +{ + INVALID 00100101 00 100 111 11 1 -------- ----- + UQSUB_zzi 00100101 .. 100 111 11 . ........ ..... @rdn_sh_i8u +} # SVE integer min/max immediate (unpredicated) SMAX_zzi 00100101 .. 101 000 110 ........ ..... @rdn_i8s diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool trans_DUP_i(DisasContext *s, arg_DUP_i *a) static bool trans_ADD_zzi(DisasContext *s, arg_rri_esz *a) { - if (a->esz == 0 && extract32(s->insn, 13, 1)) { - return false; - } return gen_gvec_fn_arg_zzi(s, tcg_gen_gvec_addi, a); } @@ -XXX,XX +XXX,XX @@ static bool trans_SUBR_zzi(DisasContext *s, arg_rri_esz *a) .scalar_first = true } }; - if (a->esz == 0 && extract32(s->insn, 13, 1)) { - return false; - } if (sve_access_check(s)) { unsigned vsz = vec_full_reg_size(s); tcg_gen_gvec_2s(vec_full_reg_offset(s, a->rd), @@ -XXX,XX +XXX,XX @@ TRANS_FEAT(MUL_zzi, aa64_sve, gen_gvec_fn_arg_zzi, tcg_gen_gvec_muli, a) static bool do_zzi_sat(DisasContext *s, arg_rri_esz *a, bool u, bool d) { - if (a->esz == 0 && extract32(s->insn, 13, 1)) { - return false; - } if (sve_access_check(s)) { do_sat_addsub_vec(s, a->esz, a->rd, a->rn, tcg_constant_i64(a->imm), u, d); -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Remove the unparsed extractions in trans_CPY_{m,z}_i which are intended to reject an 8-bit shift of an 8-bit constant for 8-bit element. Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-74-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/sve.decode | 10 ++++++++-- target/arm/translate-sve.c | 6 ------ 2 files changed, 8 insertions(+), 8 deletions(-) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -XXX,XX +XXX,XX @@ DUPM 00000101 11 0000 dbm:13 rd:5 FCPY 00000101 .. 01 .... 110 imm:8 ..... @rdn_pg4 # SVE copy integer immediate (predicated) -CPY_m_i 00000101 .. 01 .... 01 . ........ ..... @rdn_pg4 imm=%sh8_i8s -CPY_z_i 00000101 .. 01 .... 00 . ........ ..... @rdn_pg4 imm=%sh8_i8s +{ + INVALID 00000101 00 01 ---- 01 1 -------- ----- + CPY_m_i 00000101 .. 01 .... 01 . ........ ..... @rdn_pg4 imm=%sh8_i8s +} +{ + INVALID 00000101 00 01 ---- 00 1 -------- ----- + CPY_z_i 00000101 .. 01 .... 00 . ........ ..... @rdn_pg4 imm=%sh8_i8s +} ### SVE Permute - Extract Group diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool trans_FCPY(DisasContext *s, arg_FCPY *a) static bool trans_CPY_m_i(DisasContext *s, arg_rpri_esz *a) { - if (a->esz == 0 && extract32(s->insn, 13, 1)) { - return false; - } if (sve_access_check(s)) { do_cpy_m(s, a->esz, a->rd, a->rn, a->pg, tcg_constant_i64(a->imm)); } @@ -XXX,XX +XXX,XX @@ static bool trans_CPY_z_i(DisasContext *s, arg_CPY_z_i *a) gen_helper_sve_cpy_z_s, gen_helper_sve_cpy_z_d, }; - if (a->esz == 0 && extract32(s->insn, 13, 1)) { - return false; - } if (sve_access_check(s)) { unsigned vsz = vec_full_reg_size(s); tcg_gen_gvec_2i_ool(vec_full_reg_offset(s, a->rd), -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-75-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool trans_DUP_i(DisasContext *s, arg_DUP_i *a) return true; } -static bool trans_ADD_zzi(DisasContext *s, arg_rri_esz *a) -{ - return gen_gvec_fn_arg_zzi(s, tcg_gen_gvec_addi, a); -} +TRANS_FEAT(ADD_zzi, aa64_sve, gen_gvec_fn_arg_zzi, tcg_gen_gvec_addi, a) static bool trans_SUB_zzi(DisasContext *s, arg_rri_esz *a) { -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-76-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 23 ++++------------------- 1 file changed, 4 insertions(+), 19 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool do_zzi_sat(DisasContext *s, arg_rri_esz *a, bool u, bool d) return true; } -static bool trans_SQADD_zzi(DisasContext *s, arg_rri_esz *a) -{ - return do_zzi_sat(s, a, false, false); -} - -static bool trans_UQADD_zzi(DisasContext *s, arg_rri_esz *a) -{ - return do_zzi_sat(s, a, true, false); -} - -static bool trans_SQSUB_zzi(DisasContext *s, arg_rri_esz *a) -{ - return do_zzi_sat(s, a, false, true); -} - -static bool trans_UQSUB_zzi(DisasContext *s, arg_rri_esz *a) -{ - return do_zzi_sat(s, a, true, true); -} +TRANS_FEAT(SQADD_zzi, aa64_sve, do_zzi_sat, a, false, false) +TRANS_FEAT(UQADD_zzi, aa64_sve, do_zzi_sat, a, true, false) +TRANS_FEAT(SQSUB_zzi, aa64_sve, do_zzi_sat, a, false, true) +TRANS_FEAT(UQSUB_zzi, aa64_sve, do_zzi_sat, a, true, true) static bool do_zzi_ool(DisasContext *s, arg_rri_esz *a, gen_helper_gvec_2i *fn) { -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-77-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool do_zzi_ool(DisasContext *s, arg_rri_esz *a, gen_helper_gvec_2i *fn) } #define DO_ZZI(NAME, name) \ -static bool trans_##NAME##_zzi(DisasContext *s, arg_rri_esz *a) \ -{ \ - static gen_helper_gvec_2i * const fns[4] = { \ + static gen_helper_gvec_2i * const name##i_fns[4] = { \ gen_helper_sve_##name##i_b, gen_helper_sve_##name##i_h, \ gen_helper_sve_##name##i_s, gen_helper_sve_##name##i_d, \ }; \ - return do_zzi_ool(s, a, fns[a->esz]); \ -} + TRANS_FEAT(NAME##_zzi, aa64_sve, do_zzi_ool, a, name##i_fns[a->esz]) DO_ZZI(SMAX, smax) DO_ZZI(UMAX, umax) -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Use these for the several varieties of floating-point multiply-add instructions. Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-78-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 140 ++++++++++++++----------------------- 1 file changed, 53 insertions(+), 87 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool gen_gvec_ool_arg_zzxz(DisasContext *s, gen_helper_gvec_4 *fn, return gen_gvec_ool_zzzz(s, fn, a->rd, a->rn, a->rm, a->ra, a->index); } +/* Invoke an out-of-line helper on 4 Zregs, plus a pointer. */ +static bool gen_gvec_ptr_zzzz(DisasContext *s, gen_helper_gvec_4_ptr *fn, + int rd, int rn, int rm, int ra, + int data, TCGv_ptr ptr) +{ + if (fn == NULL) { + return false; + } + if (sve_access_check(s)) { + unsigned vsz = vec_full_reg_size(s); + tcg_gen_gvec_4_ptr(vec_full_reg_offset(s, rd), + vec_full_reg_offset(s, rn), + vec_full_reg_offset(s, rm), + vec_full_reg_offset(s, ra), + ptr, vsz, vsz, data, fn); + } + return true; +} + +static bool gen_gvec_fpst_zzzz(DisasContext *s, gen_helper_gvec_4_ptr *fn, + int rd, int rn, int rm, int ra, + int data, ARMFPStatusFlavour flavour) +{ + TCGv_ptr status = fpstatus_ptr(flavour); + bool ret = gen_gvec_ptr_zzzz(s, fn, rd, rn, rm, ra, data, status); + tcg_temp_free_ptr(status); + return ret; +} + /* Invoke an out-of-line helper on 2 Zregs and a predicate. */ static bool gen_gvec_ool_zzp(DisasContext *s, gen_helper_gvec_3 *fn, int rd, int rn, int pg, int data) @@ -XXX,XX +XXX,XX @@ DO_SVE2_RRXR_ROT(CDOT_zzxw_d, gen_helper_sve2_cdot_idx_d) static bool do_FMLA_zzxz(DisasContext *s, arg_rrxr_esz *a, bool sub) { - static gen_helper_gvec_4_ptr * const fns[3] = { + static gen_helper_gvec_4_ptr * const fns[4] = { + NULL, gen_helper_gvec_fmla_idx_h, gen_helper_gvec_fmla_idx_s, gen_helper_gvec_fmla_idx_d, }; - - if (sve_access_check(s)) { - unsigned vsz = vec_full_reg_size(s); - TCGv_ptr status = fpstatus_ptr(a->esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR); - tcg_gen_gvec_4_ptr(vec_full_reg_offset(s, a->rd), - vec_full_reg_offset(s, a->rn), - vec_full_reg_offset(s, a->rm), - vec_full_reg_offset(s, a->ra), - status, vsz, vsz, (a->index << 1) | sub, - fns[a->esz - 1]); - tcg_temp_free_ptr(status); - } - return true; + return gen_gvec_fpst_zzzz(s, fns[a->esz], a->rd, a->rn, a->rm, a->ra, + (a->index << 1) | sub, + a->esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR); } static bool trans_FMLA_zzxz(DisasContext *s, arg_FMLA_zzxz *a) @@ -XXX,XX +XXX,XX @@ static bool trans_FCMLA_zpzzz(DisasContext *s, arg_FCMLA_zpzzz *a) static bool trans_FCMLA_zzxz(DisasContext *s, arg_FCMLA_zzxz *a) { - static gen_helper_gvec_4_ptr * const fns[2] = { + static gen_helper_gvec_4_ptr * const fns[4] = { + NULL, gen_helper_gvec_fcmlah_idx, gen_helper_gvec_fcmlas_idx, + NULL, }; - tcg_debug_assert(a->esz == 1 || a->esz == 2); tcg_debug_assert(a->rd == a->ra); - if (sve_access_check(s)) { - unsigned vsz = vec_full_reg_size(s); - TCGv_ptr status = fpstatus_ptr(a->esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR); - tcg_gen_gvec_4_ptr(vec_full_reg_offset(s, a->rd), - vec_full_reg_offset(s, a->rn), - vec_full_reg_offset(s, a->rm), - vec_full_reg_offset(s, a->ra), - status, vsz, vsz, - a->index * 4 + a->rot, - fns[a->esz - 1]); - tcg_temp_free_ptr(status); - } - return true; + + return gen_gvec_fpst_zzzz(s, fns[a->esz], a->rd, a->rn, a->rm, a->ra, + a->index * 4 + a->rot, + a->esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR); } /* @@ -XXX,XX +XXX,XX @@ static bool trans_FMMLA(DisasContext *s, arg_rrrr_esz *a) return false; } - if (sve_access_check(s)) { - unsigned vsz = vec_full_reg_size(s); - TCGv_ptr status = fpstatus_ptr(FPST_FPCR); - tcg_gen_gvec_4_ptr(vec_full_reg_offset(s, a->rd), - vec_full_reg_offset(s, a->rn), - vec_full_reg_offset(s, a->rm), - vec_full_reg_offset(s, a->ra), - status, vsz, vsz, 0, fn); - tcg_temp_free_ptr(status); - } - return true; + return gen_gvec_fpst_zzzz(s, fn, a->rd, a->rn, a->rm, a->ra, 0, FPST_FPCR); } static gen_helper_gvec_4 * const sqdmlal_zzzw_fns[] = { @@ -XXX,XX +XXX,XX @@ static bool do_FMLAL_zzzw(DisasContext *s, arg_rrrr_esz *a, bool sub, bool sel) if (!dc_isar_feature(aa64_sve2, s)) { return false; } - if (sve_access_check(s)) { - unsigned vsz = vec_full_reg_size(s); - tcg_gen_gvec_4_ptr(vec_full_reg_offset(s, a->rd), - vec_full_reg_offset(s, a->rn), - vec_full_reg_offset(s, a->rm), - vec_full_reg_offset(s, a->ra), - cpu_env, vsz, vsz, (sel << 1) | sub, - gen_helper_sve2_fmlal_zzzw_s); - } - return true; + return gen_gvec_ptr_zzzz(s, gen_helper_sve2_fmlal_zzzw_s, + a->rd, a->rn, a->rm, a->ra, + (sel << 1) | sub, cpu_env); } static bool trans_FMLALB_zzzw(DisasContext *s, arg_rrrr_esz *a) @@ -XXX,XX +XXX,XX @@ static bool do_FMLAL_zzxw(DisasContext *s, arg_rrxr_esz *a, bool sub, bool sel) if (!dc_isar_feature(aa64_sve2, s)) { return false; } - if (sve_access_check(s)) { - unsigned vsz = vec_full_reg_size(s); - tcg_gen_gvec_4_ptr(vec_full_reg_offset(s, a->rd), - vec_full_reg_offset(s, a->rn), - vec_full_reg_offset(s, a->rm), - vec_full_reg_offset(s, a->ra), - cpu_env, vsz, vsz, - (a->index << 2) | (sel << 1) | sub, - gen_helper_sve2_fmlal_zzxw_s); - } - return true; + return gen_gvec_ptr_zzzz(s, gen_helper_sve2_fmlal_zzxw_s, + a->rd, a->rn, a->rm, a->ra, + (a->index << 2) | (sel << 1) | sub, cpu_env); } static bool trans_FMLALB_zzxw(DisasContext *s, arg_rrxr_esz *a) @@ -XXX,XX +XXX,XX @@ static bool do_BFMLAL_zzzw(DisasContext *s, arg_rrrr_esz *a, bool sel) if (!dc_isar_feature(aa64_sve_bf16, s)) { return false; } - if (sve_access_check(s)) { - TCGv_ptr status = fpstatus_ptr(FPST_FPCR); - unsigned vsz = vec_full_reg_size(s); - - tcg_gen_gvec_4_ptr(vec_full_reg_offset(s, a->rd), - vec_full_reg_offset(s, a->rn), - vec_full_reg_offset(s, a->rm), - vec_full_reg_offset(s, a->ra), - status, vsz, vsz, sel, - gen_helper_gvec_bfmlal); - tcg_temp_free_ptr(status); - } - return true; + return gen_gvec_fpst_zzzz(s, gen_helper_gvec_bfmlal, + a->rd, a->rn, a->rm, a->ra, sel, FPST_FPCR); } static bool trans_BFMLALB_zzzw(DisasContext *s, arg_rrrr_esz *a) @@ -XXX,XX +XXX,XX @@ static bool do_BFMLAL_zzxw(DisasContext *s, arg_rrxr_esz *a, bool sel) if (!dc_isar_feature(aa64_sve_bf16, s)) { return false; } - if (sve_access_check(s)) { - TCGv_ptr status = fpstatus_ptr(FPST_FPCR); - unsigned vsz = vec_full_reg_size(s); - - tcg_gen_gvec_4_ptr(vec_full_reg_offset(s, a->rd), - vec_full_reg_offset(s, a->rn), - vec_full_reg_offset(s, a->rm), - vec_full_reg_offset(s, a->ra), - status, vsz, vsz, (a->index << 1) | sel, - gen_helper_gvec_bfmlal_idx); - tcg_temp_free_ptr(status); - } - return true; + return gen_gvec_fpst_zzzz(s, gen_helper_gvec_bfmlal_idx, + a->rd, a->rn, a->rm, a->ra, + (a->index << 1) | sel, FPST_FPCR); } static bool trans_BFMLALB_zzxw(DisasContext *s, arg_rrxr_esz *a) -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Being able to specify the feature predicate in TRANS_FEAT makes it easier to split trans_FMMLA by element size, which also happens to simplify the decode. Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-79-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/sve.decode | 7 +++---- target/arm/translate-sve.c | 27 ++++----------------------- 2 files changed, 7 insertions(+), 27 deletions(-) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -XXX,XX +XXX,XX @@ SQRDCMLAH_zzzz 01000100 esz:2 0 rm:5 0011 rot:2 rn:5 rd:5 ra=%reg_movprfx USDOT_zzzz 01000100 .. 0 ..... 011 110 ..... ..... @rda_rn_rm ### SVE2 floating point matrix multiply accumulate -{ - BFMMLA 01100100 01 1 ..... 111 001 ..... ..... @rda_rn_rm_e0 - FMMLA 01100100 .. 1 ..... 111 001 ..... ..... @rda_rn_rm -} +BFMMLA 01100100 01 1 ..... 111 001 ..... ..... @rda_rn_rm_e0 +FMMLA_s 01100100 10 1 ..... 111 001 ..... ..... @rda_rn_rm_e0 +FMMLA_d 01100100 11 1 ..... 111 001 ..... ..... @rda_rn_rm_e0 ### SVE2 Memory Gather Load Group diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ DO_SVE2_ZPZZ_FP(FMINP, fminp) * SVE Integer Multiply-Add (unpredicated) */ -static bool trans_FMMLA(DisasContext *s, arg_rrrr_esz *a) -{ - gen_helper_gvec_4_ptr *fn; - - switch (a->esz) { - case MO_32: - if (!dc_isar_feature(aa64_sve_f32mm, s)) { - return false; - } - fn = gen_helper_fmmla_s; - break; - case MO_64: - if (!dc_isar_feature(aa64_sve_f64mm, s)) { - return false; - } - fn = gen_helper_fmmla_d; - break; - default: - return false; - } - - return gen_gvec_fpst_zzzz(s, fn, a->rd, a->rn, a->rm, a->ra, 0, FPST_FPCR); -} +TRANS_FEAT(FMMLA_s, aa64_sve_f32mm, gen_gvec_fpst_zzzz, gen_helper_fmmla_s, + a->rd, a->rn, a->rm, a->ra, 0, FPST_FPCR) +TRANS_FEAT(FMMLA_d, aa64_sve_f64mm, gen_gvec_fpst_zzzz, gen_helper_fmmla_d, + a->rd, a->rn, a->rm, a->ra, 0, FPST_FPCR) static gen_helper_gvec_4 * const sqdmlal_zzzw_fns[] = { NULL, gen_helper_sve2_sqdmlal_zzzw_h, -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Combined with the check already present in gen_mov_p, we can simplify some special cases in trans_AND_pppp and trans_BIC_pppp. Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-80-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 30 ++++++++++++------------------ 1 file changed, 12 insertions(+), 18 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static void do_dupi_z(DisasContext *s, int rd, uint64_t word) } /* Invoke a vector expander on three Pregs. */ -static void gen_gvec_fn_ppp(DisasContext *s, GVecGen3Fn *gvec_fn, +static bool gen_gvec_fn_ppp(DisasContext *s, GVecGen3Fn *gvec_fn, int rd, int rn, int rm) { - unsigned psz = pred_gvec_reg_size(s); - gvec_fn(MO_64, pred_full_reg_offset(s, rd), - pred_full_reg_offset(s, rn), - pred_full_reg_offset(s, rm), psz, psz); + if (sve_access_check(s)) { + unsigned psz = pred_gvec_reg_size(s); + gvec_fn(MO_64, pred_full_reg_offset(s, rd), + pred_full_reg_offset(s, rn), + pred_full_reg_offset(s, rm), psz, psz); + } + return true; } /* Invoke a vector move on two Pregs. */ @@ -XXX,XX +XXX,XX @@ static bool trans_AND_pppp(DisasContext *s, arg_rprr_s *a) }; if (!a->s) { - if (!sve_access_check(s)) { - return true; - } if (a->rn == a->rm) { if (a->pg == a->rn) { - do_mov_p(s, a->rd, a->rn); - } else { - gen_gvec_fn_ppp(s, tcg_gen_gvec_and, a->rd, a->rn, a->pg); + return do_mov_p(s, a->rd, a->rn); } - return true; + return gen_gvec_fn_ppp(s, tcg_gen_gvec_and, a->rd, a->rn, a->pg); } else if (a->pg == a->rn || a->pg == a->rm) { - gen_gvec_fn_ppp(s, tcg_gen_gvec_and, a->rd, a->rn, a->rm); - return true; + return gen_gvec_fn_ppp(s, tcg_gen_gvec_and, a->rd, a->rn, a->rm); } } return do_pppp_flags(s, a, &op); @@ -XXX,XX +XXX,XX @@ static bool trans_BIC_pppp(DisasContext *s, arg_rprr_s *a) }; if (!a->s && a->pg == a->rn) { - if (sve_access_check(s)) { - gen_gvec_fn_ppp(s, tcg_gen_gvec_andc, a->rd, a->rn, a->rm); - } - return true; + return gen_gvec_fn_ppp(s, tcg_gen_gvec_andc, a->rd, a->rn, a->rm); } return do_pppp_flags(s, a, &op); } -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> This alias is defined on EOR (prediates). While the same operation could be performed with NAND or NOR, only bother with the official alias. Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-81-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool trans_EOR_pppp(DisasContext *s, arg_rprr_s *a) .fno = gen_helper_sve_eor_pppp, .prefer_i64 = TCG_TARGET_REG_BITS == 64, }; + + /* Alias NOT (predicate) is EOR Pd.B, Pg/Z, Pn.B, Pg.B */ + if (!a->s && a->pg == a->rm) { + return gen_gvec_fn_ppp(s, tcg_gen_gvec_andc, a->rd, a->pg, a->rn); + } return do_pppp_flags(s, a, &op); } -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-82-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static gen_helper_gvec_4 * const udiv_fns[4] = { }; TRANS_FEAT(UDIV_zpzz, aa64_sve, gen_gvec_ool_arg_zpzz, udiv_fns[a->esz], a, 0) -static bool trans_SEL_zpzz(DisasContext *s, arg_rprr_esz *a) -{ - return do_sel_z(s, a->rd, a->rn, a->rm, a->pg, a->esz); -} +TRANS_FEAT(SEL_zpzz, aa64_sve, do_sel_z, a->rd, a->rn, a->rm, a->pg, a->esz) /* *** SVE Integer Arithmetic - Unary Predicated Group -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-83-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 17 +++-------------- 1 file changed, 3 insertions(+), 14 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool trans_PRF_rr(DisasContext *s, arg_PRF_rr *a) * In the meantime, just emit the moves. */ -static bool trans_MOVPRFX(DisasContext *s, arg_MOVPRFX *a) -{ - return do_mov_z(s, a->rd, a->rn); -} - -static bool trans_MOVPRFX_m(DisasContext *s, arg_rpr_esz *a) -{ - return do_sel_z(s, a->rd, a->rn, a->rd, a->pg, a->esz); -} - -static bool trans_MOVPRFX_z(DisasContext *s, arg_rpr_esz *a) -{ - return do_movz_zpz(s, a->rd, a->rn, a->pg, a->esz, false); -} +TRANS_FEAT(MOVPRFX, aa64_sve, do_mov_z, a->rd, a->rn) +TRANS_FEAT(MOVPRFX_m, aa64_sve, do_sel_z, a->rd, a->rn, a->rd, a->pg, a->esz) +TRANS_FEAT(MOVPRFX_z, aa64_sve, do_movz_zpz, a->rd, a->rn, a->pg, a->esz, false) /* * SVE2 Integer Multiply - Unpredicated -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-84-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 11 ++--------- 1 file changed, 2 insertions(+), 9 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool do_FMLA_zzxz(DisasContext *s, arg_rrxr_esz *a, bool sub) a->esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR); } -static bool trans_FMLA_zzxz(DisasContext *s, arg_FMLA_zzxz *a) -{ - return do_FMLA_zzxz(s, a, false); -} - -static bool trans_FMLS_zzxz(DisasContext *s, arg_FMLA_zzxz *a) -{ - return do_FMLA_zzxz(s, a, true); -} +TRANS_FEAT(FMLA_zzxz, aa64_sve, do_FMLA_zzxz, a, false) +TRANS_FEAT(FMLS_zzxz, aa64_sve, do_FMLA_zzxz, a, true) /* *** SVE Floating Point Multiply Indexed Group -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-85-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 28 ++++------------------------ 1 file changed, 4 insertions(+), 24 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ TRANS_FEAT(BFMMLA, aa64_sve_bf16, gen_gvec_ool_arg_zzzz, static bool do_BFMLAL_zzzw(DisasContext *s, arg_rrrr_esz *a, bool sel) { - if (!dc_isar_feature(aa64_sve_bf16, s)) { - return false; - } return gen_gvec_fpst_zzzz(s, gen_helper_gvec_bfmlal, a->rd, a->rn, a->rm, a->ra, sel, FPST_FPCR); } -static bool trans_BFMLALB_zzzw(DisasContext *s, arg_rrrr_esz *a) -{ - return do_BFMLAL_zzzw(s, a, false); -} - -static bool trans_BFMLALT_zzzw(DisasContext *s, arg_rrrr_esz *a) -{ - return do_BFMLAL_zzzw(s, a, true); -} +TRANS_FEAT(BFMLALB_zzzw, aa64_sve_bf16, do_BFMLAL_zzzw, a, false) +TRANS_FEAT(BFMLALT_zzzw, aa64_sve_bf16, do_BFMLAL_zzzw, a, true) static bool do_BFMLAL_zzxw(DisasContext *s, arg_rrxr_esz *a, bool sel) { - if (!dc_isar_feature(aa64_sve_bf16, s)) { - return false; - } return gen_gvec_fpst_zzzz(s, gen_helper_gvec_bfmlal_idx, a->rd, a->rn, a->rm, a->ra, (a->index << 1) | sel, FPST_FPCR); } -static bool trans_BFMLALB_zzxw(DisasContext *s, arg_rrxr_esz *a) -{ - return do_BFMLAL_zzxw(s, a, false); -} - -static bool trans_BFMLALT_zzxw(DisasContext *s, arg_rrxr_esz *a) -{ - return do_BFMLAL_zzxw(s, a, true); -} +TRANS_FEAT(BFMLALB_zzxw, aa64_sve_bf16, do_BFMLAL_zzxw, a, false) +TRANS_FEAT(BFMLALT_zzxw, aa64_sve_bf16, do_BFMLAL_zzxw, a, true) -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Rename the function to match gen_gvec_ool_arg_zzz, and move to be adjacent. Split out gen_gvec_fpst_zzz as a helper while we're at it. Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-86-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 50 +++++++++++++++++++++++--------------- 1 file changed, 30 insertions(+), 20 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool gen_gvec_ool_arg_zzz(DisasContext *s, gen_helper_gvec_3 *fn, return gen_gvec_ool_zzz(s, fn, a->rd, a->rn, a->rm, data); } +/* Invoke an out-of-line helper on 3 Zregs, plus float_status. */ +static bool gen_gvec_fpst_zzz(DisasContext *s, gen_helper_gvec_3_ptr *fn, + int rd, int rn, int rm, + int data, ARMFPStatusFlavour flavour) +{ + if (fn == NULL) { + return false; + } + if (sve_access_check(s)) { + unsigned vsz = vec_full_reg_size(s); + TCGv_ptr status = fpstatus_ptr(flavour); + + tcg_gen_gvec_3_ptr(vec_full_reg_offset(s, rd), + vec_full_reg_offset(s, rn), + vec_full_reg_offset(s, rm), + status, vsz, vsz, data, fn); + + tcg_temp_free_ptr(status); + } + return true; +} + +static bool gen_gvec_fpst_arg_zzz(DisasContext *s, gen_helper_gvec_3_ptr *fn, + arg_rrr_esz *a, int data) +{ + return gen_gvec_fpst_zzz(s, fn, a->rd, a->rn, a->rm, data, + a->esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR); +} + /* Invoke an out-of-line helper on 4 Zregs. */ static bool gen_gvec_ool_zzzz(DisasContext *s, gen_helper_gvec_4 *fn, int rd, int rn, int rm, int ra, int data) @@ -XXX,XX +XXX,XX @@ static bool trans_FADDA(DisasContext *s, arg_rprr_esz *a) *** SVE Floating Point Arithmetic - Unpredicated Group */ -static bool do_zzz_fp(DisasContext *s, arg_rrr_esz *a, - gen_helper_gvec_3_ptr *fn) -{ - if (fn == NULL) { - return false; - } - if (sve_access_check(s)) { - unsigned vsz = vec_full_reg_size(s); - TCGv_ptr status = fpstatus_ptr(a->esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR); - tcg_gen_gvec_3_ptr(vec_full_reg_offset(s, a->rd), - vec_full_reg_offset(s, a->rn), - vec_full_reg_offset(s, a->rm), - status, vsz, vsz, 0, fn); - tcg_temp_free_ptr(status); - } - return true; -} - - #define DO_FP3(NAME, name) \ static bool trans_##NAME(DisasContext *s, arg_rrr_esz *a) \ { \ @@ -XXX,XX +XXX,XX @@ static bool trans_##NAME(DisasContext *s, arg_rrr_esz *a) \ NULL, gen_helper_gvec_##name##_h, \ gen_helper_gvec_##name##_s, gen_helper_gvec_##name##_d \ }; \ - return do_zzz_fp(s, a, fns[a->esz]); \ + return gen_gvec_fpst_arg_zzz(s, fns[a->esz], a, 0); \ } DO_FP3(FADD_zzz, fadd) -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-87-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool trans_FADDA(DisasContext *s, arg_rprr_esz *a) */ #define DO_FP3(NAME, name) \ -static bool trans_##NAME(DisasContext *s, arg_rrr_esz *a) \ -{ \ - static gen_helper_gvec_3_ptr * const fns[4] = { \ + static gen_helper_gvec_3_ptr * const name##_fns[4] = { \ NULL, gen_helper_gvec_##name##_h, \ gen_helper_gvec_##name##_s, gen_helper_gvec_##name##_d \ }; \ - return gen_gvec_fpst_arg_zzz(s, fns[a->esz], a, 0); \ -} + TRANS_FEAT(NAME, aa64_sve, gen_gvec_fpst_arg_zzz, name##_fns[a->esz], a, 0) DO_FP3(FADD_zzz, fadd) DO_FP3(FSUB_zzz, fsub) -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-88-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 26 +++++++------------------- 1 file changed, 7 insertions(+), 19 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ TRANS_FEAT(FMLS_zzxz, aa64_sve, do_FMLA_zzxz, a, true) *** SVE Floating Point Multiply Indexed Group */ -static bool trans_FMUL_zzx(DisasContext *s, arg_FMUL_zzx *a) -{ - static gen_helper_gvec_3_ptr * const fns[3] = { - gen_helper_gvec_fmul_idx_h, - gen_helper_gvec_fmul_idx_s, - gen_helper_gvec_fmul_idx_d, - }; - - if (sve_access_check(s)) { - unsigned vsz = vec_full_reg_size(s); - TCGv_ptr status = fpstatus_ptr(a->esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR); - tcg_gen_gvec_3_ptr(vec_full_reg_offset(s, a->rd), - vec_full_reg_offset(s, a->rn), - vec_full_reg_offset(s, a->rm), - status, vsz, vsz, a->index, fns[a->esz - 1]); - tcg_temp_free_ptr(status); - } - return true; -} +static gen_helper_gvec_3_ptr * const fmul_idx_fns[4] = { + NULL, gen_helper_gvec_fmul_idx_h, + gen_helper_gvec_fmul_idx_s, gen_helper_gvec_fmul_idx_d, +}; +TRANS_FEAT(FMUL_zzx, aa64_sve, gen_gvec_fpst_zzz, + fmul_idx_fns[a->esz], a->rd, a->rn, a->rm, a->index, + a->esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR) /* *** SVE Floating Point Fast Reduction Group -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-89-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 29 +++++++---------------------- 1 file changed, 7 insertions(+), 22 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ DO_PPZ(FCMNE_ppz0, fcmne0) *** SVE floating-point trig multiply-add coefficient */ -static bool trans_FTMAD(DisasContext *s, arg_FTMAD *a) -{ - static gen_helper_gvec_3_ptr * const fns[3] = { - gen_helper_sve_ftmad_h, - gen_helper_sve_ftmad_s, - gen_helper_sve_ftmad_d, - }; - - if (a->esz == 0) { - return false; - } - if (sve_access_check(s)) { - unsigned vsz = vec_full_reg_size(s); - TCGv_ptr status = fpstatus_ptr(a->esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR); - tcg_gen_gvec_3_ptr(vec_full_reg_offset(s, a->rd), - vec_full_reg_offset(s, a->rn), - vec_full_reg_offset(s, a->rm), - status, vsz, vsz, a->imm, fns[a->esz - 1]); - tcg_temp_free_ptr(status); - } - return true; -} +static gen_helper_gvec_3_ptr * const ftmad_fns[4] = { + NULL, gen_helper_sve_ftmad_h, + gen_helper_sve_ftmad_s, gen_helper_sve_ftmad_d, +}; +TRANS_FEAT(FTMAD, aa64_sve, gen_gvec_fpst_zzz, + ftmad_fns[a->esz], a->rd, a->rn, a->rm, a->imm, + a->esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR) /* *** SVE Floating Point Accumulating Reduction Group -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-90-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 30 +++++++++++++++++------------- 1 file changed, 17 insertions(+), 13 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ TRANS_FEAT(FMUL_zzx, aa64_sve, gen_gvec_fpst_zzz, typedef void gen_helper_fp_reduce(TCGv_i64, TCGv_ptr, TCGv_ptr, TCGv_ptr, TCGv_i32); -static void do_reduce(DisasContext *s, arg_rpr_esz *a, +static bool do_reduce(DisasContext *s, arg_rpr_esz *a, gen_helper_fp_reduce *fn) { - unsigned vsz = vec_full_reg_size(s); - unsigned p2vsz = pow2ceil(vsz); - TCGv_i32 t_desc = tcg_constant_i32(simd_desc(vsz, vsz, p2vsz)); + unsigned vsz, p2vsz; + TCGv_i32 t_desc; TCGv_ptr t_zn, t_pg, status; TCGv_i64 temp; + if (fn == NULL) { + return false; + } + if (!sve_access_check(s)) { + return true; + } + + vsz = vec_full_reg_size(s); + p2vsz = pow2ceil(vsz); + t_desc = tcg_constant_i32(simd_desc(vsz, vsz, p2vsz)); temp = tcg_temp_new_i64(); t_zn = tcg_temp_new_ptr(); t_pg = tcg_temp_new_ptr(); @@ -XXX,XX +XXX,XX @@ static void do_reduce(DisasContext *s, arg_rpr_esz *a, write_fp_dreg(s, a->rd, temp); tcg_temp_free_i64(temp); + return true; } #define DO_VPZ(NAME, name) \ static bool trans_##NAME(DisasContext *s, arg_rpr_esz *a) \ { \ - static gen_helper_fp_reduce * const fns[3] = { \ - gen_helper_sve_##name##_h, \ + static gen_helper_fp_reduce * const fns[4] = { \ + NULL, gen_helper_sve_##name##_h, \ gen_helper_sve_##name##_s, \ gen_helper_sve_##name##_d, \ }; \ - if (a->esz == 0) { \ - return false; \ - } \ - if (sve_access_check(s)) { \ - do_reduce(s, a, fns[a->esz - 1]); \ - } \ - return true; \ + return do_reduce(s, a, fns[a->esz]); \ } DO_VPZ(FADDV, faddv) -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-91-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 14 ++++++-------- 1 file changed, 6 insertions(+), 8 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool do_reduce(DisasContext *s, arg_rpr_esz *a, } #define DO_VPZ(NAME, name) \ -static bool trans_##NAME(DisasContext *s, arg_rpr_esz *a) \ -{ \ - static gen_helper_fp_reduce * const fns[4] = { \ - NULL, gen_helper_sve_##name##_h, \ - gen_helper_sve_##name##_s, \ - gen_helper_sve_##name##_d, \ + static gen_helper_fp_reduce * const name##_fns[4] = { \ + NULL, gen_helper_sve_##name##_h, \ + gen_helper_sve_##name##_s, gen_helper_sve_##name##_d, \ }; \ - return do_reduce(s, a, fns[a->esz]); \ -} + TRANS_FEAT(NAME, aa64_sve, do_reduce, a, name##_fns[a->esz]) DO_VPZ(FADDV, faddv) DO_VPZ(FMINNMV, fminnmv) @@ -XXX,XX +XXX,XX @@ DO_VPZ(FMAXNMV, fmaxnmv) DO_VPZ(FMINV, fminv) DO_VPZ(FMAXV, fmaxv) +#undef DO_VPZ + /* *** SVE Floating Point Unary Operations - Unpredicated Group */ -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Rename do_zz_fp to gen_gvec_fpst_arg_zz, and move up. Split out gen_gvec_fpst_zz as a helper while we're at it. Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-92-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 77 ++++++++++++++++++-------------------- 1 file changed, 36 insertions(+), 41 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool gen_gvec_ool_zz(DisasContext *s, gen_helper_gvec_2 *fn, return true; } +static bool gen_gvec_fpst_zz(DisasContext *s, gen_helper_gvec_2_ptr *fn, + int rd, int rn, int data, + ARMFPStatusFlavour flavour) +{ + if (fn == NULL) { + return false; + } + if (sve_access_check(s)) { + unsigned vsz = vec_full_reg_size(s); + TCGv_ptr status = fpstatus_ptr(flavour); + + tcg_gen_gvec_2_ptr(vec_full_reg_offset(s, rd), + vec_full_reg_offset(s, rn), + status, vsz, vsz, data, fn); + tcg_temp_free_ptr(status); + } + return true; +} + +static bool gen_gvec_fpst_arg_zz(DisasContext *s, gen_helper_gvec_2_ptr *fn, + arg_rr_esz *a, int data) +{ + return gen_gvec_fpst_zz(s, fn, a->rd, a->rn, data, + a->esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR); +} + /* Invoke an out-of-line helper on 3 Zregs. */ static bool gen_gvec_ool_zzz(DisasContext *s, gen_helper_gvec_3 *fn, int rd, int rn, int rm, int data) @@ -XXX,XX +XXX,XX @@ DO_VPZ(FMAXV, fmaxv) *** SVE Floating Point Unary Operations - Unpredicated Group */ -static void do_zz_fp(DisasContext *s, arg_rr_esz *a, gen_helper_gvec_2_ptr *fn) -{ - unsigned vsz = vec_full_reg_size(s); - TCGv_ptr status = fpstatus_ptr(a->esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR); +static gen_helper_gvec_2_ptr * const frecpe_fns[] = { + NULL, gen_helper_gvec_frecpe_h, + gen_helper_gvec_frecpe_s, gen_helper_gvec_frecpe_d, +}; +TRANS_FEAT(FRECPE, aa64_sve, gen_gvec_fpst_arg_zz, frecpe_fns[a->esz], a, 0) - tcg_gen_gvec_2_ptr(vec_full_reg_offset(s, a->rd), - vec_full_reg_offset(s, a->rn), - status, vsz, vsz, 0, fn); - tcg_temp_free_ptr(status); -} - -static bool trans_FRECPE(DisasContext *s, arg_rr_esz *a) -{ - static gen_helper_gvec_2_ptr * const fns[3] = { - gen_helper_gvec_frecpe_h, - gen_helper_gvec_frecpe_s, - gen_helper_gvec_frecpe_d, - }; - if (a->esz == 0) { - return false; - } - if (sve_access_check(s)) { - do_zz_fp(s, a, fns[a->esz - 1]); - } - return true; -} - -static bool trans_FRSQRTE(DisasContext *s, arg_rr_esz *a) -{ - static gen_helper_gvec_2_ptr * const fns[3] = { - gen_helper_gvec_frsqrte_h, - gen_helper_gvec_frsqrte_s, - gen_helper_gvec_frsqrte_d, - }; - if (a->esz == 0) { - return false; - } - if (sve_access_check(s)) { - do_zz_fp(s, a, fns[a->esz - 1]); - } - return true; -} +static gen_helper_gvec_2_ptr * const frsqrte_fns[] = { + NULL, gen_helper_gvec_frsqrte_h, + gen_helper_gvec_frsqrte_s, gen_helper_gvec_frsqrte_d, +}; +TRANS_FEAT(FRSQRTE, aa64_sve, gen_gvec_fpst_arg_zz, frsqrte_fns[a->esz], a, 0) /* *** SVE Floating Point Compare with Zero Group -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Simplify indexing of this array. This will allow folding of the illegal esz == 0 into the normal fn == NULL check. Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-93-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool trans_FCVTZU_dd(DisasContext *s, arg_rpr_esz *a) return do_zpz_ptr(s, a->rd, a->rn, a->pg, false, gen_helper_sve_fcvtzu_dd); } -static gen_helper_gvec_3_ptr * const frint_fns[3] = { +static gen_helper_gvec_3_ptr * const frint_fns[] = { + NULL, gen_helper_sve_frint_h, gen_helper_sve_frint_s, gen_helper_sve_frint_d @@ -XXX,XX +XXX,XX @@ static bool trans_FRINTI(DisasContext *s, arg_rpr_esz *a) return false; } return do_zpz_ptr(s, a->rd, a->rn, a->pg, a->esz == MO_16, - frint_fns[a->esz - 1]); + frint_fns[a->esz]); } static bool trans_FRINTX(DisasContext *s, arg_rpr_esz *a) @@ -XXX,XX +XXX,XX @@ static bool trans_FRINTN(DisasContext *s, arg_rpr_esz *a) if (a->esz == 0) { return false; } - return do_frint_mode(s, a, float_round_nearest_even, frint_fns[a->esz - 1]); + return do_frint_mode(s, a, float_round_nearest_even, frint_fns[a->esz]); } static bool trans_FRINTP(DisasContext *s, arg_rpr_esz *a) @@ -XXX,XX +XXX,XX @@ static bool trans_FRINTP(DisasContext *s, arg_rpr_esz *a) if (a->esz == 0) { return false; } - return do_frint_mode(s, a, float_round_up, frint_fns[a->esz - 1]); + return do_frint_mode(s, a, float_round_up, frint_fns[a->esz]); } static bool trans_FRINTM(DisasContext *s, arg_rpr_esz *a) @@ -XXX,XX +XXX,XX @@ static bool trans_FRINTM(DisasContext *s, arg_rpr_esz *a) if (a->esz == 0) { return false; } - return do_frint_mode(s, a, float_round_down, frint_fns[a->esz - 1]); + return do_frint_mode(s, a, float_round_down, frint_fns[a->esz]); } static bool trans_FRINTZ(DisasContext *s, arg_rpr_esz *a) @@ -XXX,XX +XXX,XX @@ static bool trans_FRINTZ(DisasContext *s, arg_rpr_esz *a) if (a->esz == 0) { return false; } - return do_frint_mode(s, a, float_round_to_zero, frint_fns[a->esz - 1]); + return do_frint_mode(s, a, float_round_to_zero, frint_fns[a->esz]); } static bool trans_FRINTA(DisasContext *s, arg_rpr_esz *a) @@ -XXX,XX +XXX,XX @@ static bool trans_FRINTA(DisasContext *s, arg_rpr_esz *a) if (a->esz == 0) { return false; } - return do_frint_mode(s, a, float_round_ties_away, frint_fns[a->esz - 1]); + return do_frint_mode(s, a, float_round_ties_away, frint_fns[a->esz]); } static bool trans_FRECPX(DisasContext *s, arg_rpr_esz *a) -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Rename the function to match other expansion function and move to be adjacent. Split out gen_gvec_fpst_zzp as a helper while we're at it. Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-94-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 392 ++++++++++++------------------------- 1 file changed, 129 insertions(+), 263 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool gen_gvec_ool_arg_zpzi(DisasContext *s, gen_helper_gvec_3 *fn, return gen_gvec_ool_zzp(s, fn, a->rd, a->rn, a->pg, a->imm); } +static bool gen_gvec_fpst_zzp(DisasContext *s, gen_helper_gvec_3_ptr *fn, + int rd, int rn, int pg, int data, + ARMFPStatusFlavour flavour) +{ + if (fn == NULL) { + return false; + } + if (sve_access_check(s)) { + unsigned vsz = vec_full_reg_size(s); + TCGv_ptr status = fpstatus_ptr(flavour); + + tcg_gen_gvec_3_ptr(vec_full_reg_offset(s, rd), + vec_full_reg_offset(s, rn), + pred_full_reg_offset(s, pg), + status, vsz, vsz, data, fn); + tcg_temp_free_ptr(status); + } + return true; +} + +static bool gen_gvec_fpst_arg_zpz(DisasContext *s, gen_helper_gvec_3_ptr *fn, + arg_rpr_esz *a, int data, + ARMFPStatusFlavour flavour) +{ + return gen_gvec_fpst_zzp(s, fn, a->rd, a->rn, a->pg, data, flavour); +} + /* Invoke an out-of-line helper on 3 Zregs and a predicate. */ static bool gen_gvec_ool_zzzp(DisasContext *s, gen_helper_gvec_4 *fn, int rd, int rn, int rm, int pg, int data) @@ -XXX,XX +XXX,XX @@ static bool trans_FCMLA_zzxz(DisasContext *s, arg_FCMLA_zzxz *a) *** SVE Floating Point Unary Operations Predicated Group */ -static bool do_zpz_ptr(DisasContext *s, int rd, int rn, int pg, - bool is_fp16, gen_helper_gvec_3_ptr *fn) -{ - if (sve_access_check(s)) { - unsigned vsz = vec_full_reg_size(s); - TCGv_ptr status = fpstatus_ptr(is_fp16 ? FPST_FPCR_F16 : FPST_FPCR); - tcg_gen_gvec_3_ptr(vec_full_reg_offset(s, rd), - vec_full_reg_offset(s, rn), - pred_full_reg_offset(s, pg), - status, vsz, vsz, 0, fn); - tcg_temp_free_ptr(status); - } - return true; -} +TRANS_FEAT(FCVT_sh, aa64_sve, gen_gvec_fpst_arg_zpz, + gen_helper_sve_fcvt_sh, a, 0, FPST_FPCR) +TRANS_FEAT(FCVT_hs, aa64_sve, gen_gvec_fpst_arg_zpz, + gen_helper_sve_fcvt_hs, a, 0, FPST_FPCR) -static bool trans_FCVT_sh(DisasContext *s, arg_rpr_esz *a) -{ - return do_zpz_ptr(s, a->rd, a->rn, a->pg, false, gen_helper_sve_fcvt_sh); -} +TRANS_FEAT(BFCVT, aa64_sve_bf16, gen_gvec_fpst_arg_zpz, + gen_helper_sve_bfcvt, a, 0, FPST_FPCR) -static bool trans_FCVT_hs(DisasContext *s, arg_rpr_esz *a) -{ - return do_zpz_ptr(s, a->rd, a->rn, a->pg, false, gen_helper_sve_fcvt_hs); -} +TRANS_FEAT(FCVT_dh, aa64_sve, gen_gvec_fpst_arg_zpz, + gen_helper_sve_fcvt_dh, a, 0, FPST_FPCR) +TRANS_FEAT(FCVT_hd, aa64_sve, gen_gvec_fpst_arg_zpz, + gen_helper_sve_fcvt_hd, a, 0, FPST_FPCR) +TRANS_FEAT(FCVT_ds, aa64_sve, gen_gvec_fpst_arg_zpz, + gen_helper_sve_fcvt_ds, a, 0, FPST_FPCR) +TRANS_FEAT(FCVT_sd, aa64_sve, gen_gvec_fpst_arg_zpz, + gen_helper_sve_fcvt_sd, a, 0, FPST_FPCR) -static bool trans_BFCVT(DisasContext *s, arg_rpr_esz *a) -{ - if (!dc_isar_feature(aa64_sve_bf16, s)) { - return false; - } - return do_zpz_ptr(s, a->rd, a->rn, a->pg, false, gen_helper_sve_bfcvt); -} +TRANS_FEAT(FCVTZS_hh, aa64_sve, gen_gvec_fpst_arg_zpz, + gen_helper_sve_fcvtzs_hh, a, 0, FPST_FPCR_F16) +TRANS_FEAT(FCVTZU_hh, aa64_sve, gen_gvec_fpst_arg_zpz, + gen_helper_sve_fcvtzu_hh, a, 0, FPST_FPCR_F16) +TRANS_FEAT(FCVTZS_hs, aa64_sve, gen_gvec_fpst_arg_zpz, + gen_helper_sve_fcvtzs_hs, a, 0, FPST_FPCR_F16) +TRANS_FEAT(FCVTZU_hs, aa64_sve, gen_gvec_fpst_arg_zpz, + gen_helper_sve_fcvtzu_hs, a, 0, FPST_FPCR_F16) +TRANS_FEAT(FCVTZS_hd, aa64_sve, gen_gvec_fpst_arg_zpz, + gen_helper_sve_fcvtzs_hd, a, 0, FPST_FPCR_F16) +TRANS_FEAT(FCVTZU_hd, aa64_sve, gen_gvec_fpst_arg_zpz, + gen_helper_sve_fcvtzu_hd, a, 0, FPST_FPCR_F16) -static bool trans_FCVT_dh(DisasContext *s, arg_rpr_esz *a) -{ - return do_zpz_ptr(s, a->rd, a->rn, a->pg, false, gen_helper_sve_fcvt_dh); -} +TRANS_FEAT(FCVTZS_ss, aa64_sve, gen_gvec_fpst_arg_zpz, + gen_helper_sve_fcvtzs_ss, a, 0, FPST_FPCR) +TRANS_FEAT(FCVTZU_ss, aa64_sve, gen_gvec_fpst_arg_zpz, + gen_helper_sve_fcvtzu_ss, a, 0, FPST_FPCR) +TRANS_FEAT(FCVTZS_sd, aa64_sve, gen_gvec_fpst_arg_zpz, + gen_helper_sve_fcvtzs_sd, a, 0, FPST_FPCR) +TRANS_FEAT(FCVTZU_sd, aa64_sve, gen_gvec_fpst_arg_zpz, + gen_helper_sve_fcvtzu_sd, a, 0, FPST_FPCR) +TRANS_FEAT(FCVTZS_ds, aa64_sve, gen_gvec_fpst_arg_zpz, + gen_helper_sve_fcvtzs_ds, a, 0, FPST_FPCR) +TRANS_FEAT(FCVTZU_ds, aa64_sve, gen_gvec_fpst_arg_zpz, + gen_helper_sve_fcvtzu_ds, a, 0, FPST_FPCR) -static bool trans_FCVT_hd(DisasContext *s, arg_rpr_esz *a) -{ - return do_zpz_ptr(s, a->rd, a->rn, a->pg, false, gen_helper_sve_fcvt_hd); -} - -static bool trans_FCVT_ds(DisasContext *s, arg_rpr_esz *a) -{ - return do_zpz_ptr(s, a->rd, a->rn, a->pg, false, gen_helper_sve_fcvt_ds); -} - -static bool trans_FCVT_sd(DisasContext *s, arg_rpr_esz *a) -{ - return do_zpz_ptr(s, a->rd, a->rn, a->pg, false, gen_helper_sve_fcvt_sd); -} - -static bool trans_FCVTZS_hh(DisasContext *s, arg_rpr_esz *a) -{ - return do_zpz_ptr(s, a->rd, a->rn, a->pg, true, gen_helper_sve_fcvtzs_hh); -} - -static bool trans_FCVTZU_hh(DisasContext *s, arg_rpr_esz *a) -{ - return do_zpz_ptr(s, a->rd, a->rn, a->pg, true, gen_helper_sve_fcvtzu_hh); -} - -static bool trans_FCVTZS_hs(DisasContext *s, arg_rpr_esz *a) -{ - return do_zpz_ptr(s, a->rd, a->rn, a->pg, true, gen_helper_sve_fcvtzs_hs); -} - -static bool trans_FCVTZU_hs(DisasContext *s, arg_rpr_esz *a) -{ - return do_zpz_ptr(s, a->rd, a->rn, a->pg, true, gen_helper_sve_fcvtzu_hs); -} - -static bool trans_FCVTZS_hd(DisasContext *s, arg_rpr_esz *a) -{ - return do_zpz_ptr(s, a->rd, a->rn, a->pg, true, gen_helper_sve_fcvtzs_hd); -} - -static bool trans_FCVTZU_hd(DisasContext *s, arg_rpr_esz *a) -{ - return do_zpz_ptr(s, a->rd, a->rn, a->pg, true, gen_helper_sve_fcvtzu_hd); -} - -static bool trans_FCVTZS_ss(DisasContext *s, arg_rpr_esz *a) -{ - return do_zpz_ptr(s, a->rd, a->rn, a->pg, false, gen_helper_sve_fcvtzs_ss); -} - -static bool trans_FCVTZU_ss(DisasContext *s, arg_rpr_esz *a) -{ - return do_zpz_ptr(s, a->rd, a->rn, a->pg, false, gen_helper_sve_fcvtzu_ss); -} - -static bool trans_FCVTZS_sd(DisasContext *s, arg_rpr_esz *a) -{ - return do_zpz_ptr(s, a->rd, a->rn, a->pg, false, gen_helper_sve_fcvtzs_sd); -} - -static bool trans_FCVTZU_sd(DisasContext *s, arg_rpr_esz *a) -{ - return do_zpz_ptr(s, a->rd, a->rn, a->pg, false, gen_helper_sve_fcvtzu_sd); -} - -static bool trans_FCVTZS_ds(DisasContext *s, arg_rpr_esz *a) -{ - return do_zpz_ptr(s, a->rd, a->rn, a->pg, false, gen_helper_sve_fcvtzs_ds); -} - -static bool trans_FCVTZU_ds(DisasContext *s, arg_rpr_esz *a) -{ - return do_zpz_ptr(s, a->rd, a->rn, a->pg, false, gen_helper_sve_fcvtzu_ds); -} - -static bool trans_FCVTZS_dd(DisasContext *s, arg_rpr_esz *a) -{ - return do_zpz_ptr(s, a->rd, a->rn, a->pg, false, gen_helper_sve_fcvtzs_dd); -} - -static bool trans_FCVTZU_dd(DisasContext *s, arg_rpr_esz *a) -{ - return do_zpz_ptr(s, a->rd, a->rn, a->pg, false, gen_helper_sve_fcvtzu_dd); -} +TRANS_FEAT(FCVTZS_dd, aa64_sve, gen_gvec_fpst_arg_zpz, + gen_helper_sve_fcvtzs_dd, a, 0, FPST_FPCR) +TRANS_FEAT(FCVTZU_dd, aa64_sve, gen_gvec_fpst_arg_zpz, + gen_helper_sve_fcvtzu_dd, a, 0, FPST_FPCR) static gen_helper_gvec_3_ptr * const frint_fns[] = { NULL, @@ -XXX,XX +XXX,XX @@ static gen_helper_gvec_3_ptr * const frint_fns[] = { gen_helper_sve_frint_s, gen_helper_sve_frint_d }; +TRANS_FEAT(FRINTI, aa64_sve, gen_gvec_fpst_arg_zpz, frint_fns[a->esz], + a, 0, a->esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR) -static bool trans_FRINTI(DisasContext *s, arg_rpr_esz *a) -{ - if (a->esz == 0) { - return false; - } - return do_zpz_ptr(s, a->rd, a->rn, a->pg, a->esz == MO_16, - frint_fns[a->esz]); -} - -static bool trans_FRINTX(DisasContext *s, arg_rpr_esz *a) -{ - static gen_helper_gvec_3_ptr * const fns[3] = { - gen_helper_sve_frintx_h, - gen_helper_sve_frintx_s, - gen_helper_sve_frintx_d - }; - if (a->esz == 0) { - return false; - } - return do_zpz_ptr(s, a->rd, a->rn, a->pg, a->esz == MO_16, fns[a->esz - 1]); -} +static gen_helper_gvec_3_ptr * const frintx_fns[] = { + NULL, + gen_helper_sve_frintx_h, + gen_helper_sve_frintx_s, + gen_helper_sve_frintx_d +}; +TRANS_FEAT(FRINTX, aa64_sve, gen_gvec_fpst_arg_zpz, frintx_fns[a->esz], + a, 0, a->esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR); static bool do_frint_mode(DisasContext *s, arg_rpr_esz *a, int mode, gen_helper_gvec_3_ptr *fn) @@ -XXX,XX +XXX,XX @@ static bool trans_FRINTA(DisasContext *s, arg_rpr_esz *a) return do_frint_mode(s, a, float_round_ties_away, frint_fns[a->esz]); } -static bool trans_FRECPX(DisasContext *s, arg_rpr_esz *a) -{ - static gen_helper_gvec_3_ptr * const fns[3] = { - gen_helper_sve_frecpx_h, - gen_helper_sve_frecpx_s, - gen_helper_sve_frecpx_d - }; - if (a->esz == 0) { - return false; - } - return do_zpz_ptr(s, a->rd, a->rn, a->pg, a->esz == MO_16, fns[a->esz - 1]); -} +static gen_helper_gvec_3_ptr * const frecpx_fns[] = { + NULL, gen_helper_sve_frecpx_h, + gen_helper_sve_frecpx_s, gen_helper_sve_frecpx_d, +}; +TRANS_FEAT(FRECPX, aa64_sve, gen_gvec_fpst_arg_zpz, frecpx_fns[a->esz], + a, 0, a->esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR) -static bool trans_FSQRT(DisasContext *s, arg_rpr_esz *a) -{ - static gen_helper_gvec_3_ptr * const fns[3] = { - gen_helper_sve_fsqrt_h, - gen_helper_sve_fsqrt_s, - gen_helper_sve_fsqrt_d - }; - if (a->esz == 0) { - return false; - } - return do_zpz_ptr(s, a->rd, a->rn, a->pg, a->esz == MO_16, fns[a->esz - 1]); -} +static gen_helper_gvec_3_ptr * const fsqrt_fns[] = { + NULL, gen_helper_sve_fsqrt_h, + gen_helper_sve_fsqrt_s, gen_helper_sve_fsqrt_d, +}; +TRANS_FEAT(FSQRT, aa64_sve, gen_gvec_fpst_arg_zpz, fsqrt_fns[a->esz], + a, 0, a->esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR) -static bool trans_SCVTF_hh(DisasContext *s, arg_rpr_esz *a) -{ - return do_zpz_ptr(s, a->rd, a->rn, a->pg, true, gen_helper_sve_scvt_hh); -} +TRANS_FEAT(SCVTF_hh, aa64_sve, gen_gvec_fpst_arg_zpz, + gen_helper_sve_scvt_hh, a, 0, FPST_FPCR_F16) +TRANS_FEAT(SCVTF_sh, aa64_sve, gen_gvec_fpst_arg_zpz, + gen_helper_sve_scvt_sh, a, 0, FPST_FPCR_F16) +TRANS_FEAT(SCVTF_dh, aa64_sve, gen_gvec_fpst_arg_zpz, + gen_helper_sve_scvt_dh, a, 0, FPST_FPCR_F16) -static bool trans_SCVTF_sh(DisasContext *s, arg_rpr_esz *a) -{ - return do_zpz_ptr(s, a->rd, a->rn, a->pg, true, gen_helper_sve_scvt_sh); -} +TRANS_FEAT(SCVTF_ss, aa64_sve, gen_gvec_fpst_arg_zpz, + gen_helper_sve_scvt_ss, a, 0, FPST_FPCR) +TRANS_FEAT(SCVTF_ds, aa64_sve, gen_gvec_fpst_arg_zpz, + gen_helper_sve_scvt_ds, a, 0, FPST_FPCR) -static bool trans_SCVTF_dh(DisasContext *s, arg_rpr_esz *a) -{ - return do_zpz_ptr(s, a->rd, a->rn, a->pg, true, gen_helper_sve_scvt_dh); -} +TRANS_FEAT(SCVTF_sd, aa64_sve, gen_gvec_fpst_arg_zpz, + gen_helper_sve_scvt_sd, a, 0, FPST_FPCR) +TRANS_FEAT(SCVTF_dd, aa64_sve, gen_gvec_fpst_arg_zpz, + gen_helper_sve_scvt_dd, a, 0, FPST_FPCR) -static bool trans_SCVTF_ss(DisasContext *s, arg_rpr_esz *a) -{ - return do_zpz_ptr(s, a->rd, a->rn, a->pg, false, gen_helper_sve_scvt_ss); -} +TRANS_FEAT(UCVTF_hh, aa64_sve, gen_gvec_fpst_arg_zpz, + gen_helper_sve_ucvt_hh, a, 0, FPST_FPCR_F16) +TRANS_FEAT(UCVTF_sh, aa64_sve, gen_gvec_fpst_arg_zpz, + gen_helper_sve_ucvt_sh, a, 0, FPST_FPCR_F16) +TRANS_FEAT(UCVTF_dh, aa64_sve, gen_gvec_fpst_arg_zpz, + gen_helper_sve_ucvt_dh, a, 0, FPST_FPCR_F16) -static bool trans_SCVTF_ds(DisasContext *s, arg_rpr_esz *a) -{ - return do_zpz_ptr(s, a->rd, a->rn, a->pg, false, gen_helper_sve_scvt_ds); -} +TRANS_FEAT(UCVTF_ss, aa64_sve, gen_gvec_fpst_arg_zpz, + gen_helper_sve_ucvt_ss, a, 0, FPST_FPCR) +TRANS_FEAT(UCVTF_ds, aa64_sve, gen_gvec_fpst_arg_zpz, + gen_helper_sve_ucvt_ds, a, 0, FPST_FPCR) +TRANS_FEAT(UCVTF_sd, aa64_sve, gen_gvec_fpst_arg_zpz, + gen_helper_sve_ucvt_sd, a, 0, FPST_FPCR) -static bool trans_SCVTF_sd(DisasContext *s, arg_rpr_esz *a) -{ - return do_zpz_ptr(s, a->rd, a->rn, a->pg, false, gen_helper_sve_scvt_sd); -} - -static bool trans_SCVTF_dd(DisasContext *s, arg_rpr_esz *a) -{ - return do_zpz_ptr(s, a->rd, a->rn, a->pg, false, gen_helper_sve_scvt_dd); -} - -static bool trans_UCVTF_hh(DisasContext *s, arg_rpr_esz *a) -{ - return do_zpz_ptr(s, a->rd, a->rn, a->pg, true, gen_helper_sve_ucvt_hh); -} - -static bool trans_UCVTF_sh(DisasContext *s, arg_rpr_esz *a) -{ - return do_zpz_ptr(s, a->rd, a->rn, a->pg, true, gen_helper_sve_ucvt_sh); -} - -static bool trans_UCVTF_dh(DisasContext *s, arg_rpr_esz *a) -{ - return do_zpz_ptr(s, a->rd, a->rn, a->pg, true, gen_helper_sve_ucvt_dh); -} - -static bool trans_UCVTF_ss(DisasContext *s, arg_rpr_esz *a) -{ - return do_zpz_ptr(s, a->rd, a->rn, a->pg, false, gen_helper_sve_ucvt_ss); -} - -static bool trans_UCVTF_ds(DisasContext *s, arg_rpr_esz *a) -{ - return do_zpz_ptr(s, a->rd, a->rn, a->pg, false, gen_helper_sve_ucvt_ds); -} - -static bool trans_UCVTF_sd(DisasContext *s, arg_rpr_esz *a) -{ - return do_zpz_ptr(s, a->rd, a->rn, a->pg, false, gen_helper_sve_ucvt_sd); -} - -static bool trans_UCVTF_dd(DisasContext *s, arg_rpr_esz *a) -{ - return do_zpz_ptr(s, a->rd, a->rn, a->pg, false, gen_helper_sve_ucvt_dd); -} +TRANS_FEAT(UCVTF_dd, aa64_sve, gen_gvec_fpst_arg_zpz, + gen_helper_sve_ucvt_dd, a, 0, FPST_FPCR) /* *** SVE Memory - 32-bit Gather and Unsized Contiguous Group @@ -XXX,XX +XXX,XX @@ TRANS_FEAT(SM4EKEY, aa64_sve2_sm4, gen_gvec_ool_arg_zzz, TRANS_FEAT(RAX1, aa64_sve2_sha3, gen_gvec_fn_arg_zzz, gen_gvec_rax1, a) -static bool trans_FCVTNT_sh(DisasContext *s, arg_rpr_esz *a) -{ - if (!dc_isar_feature(aa64_sve2, s)) { - return false; - } - return do_zpz_ptr(s, a->rd, a->rn, a->pg, false, gen_helper_sve2_fcvtnt_sh); -} +TRANS_FEAT(FCVTNT_sh, aa64_sve2, gen_gvec_fpst_arg_zpz, + gen_helper_sve2_fcvtnt_sh, a, 0, FPST_FPCR) +TRANS_FEAT(FCVTNT_ds, aa64_sve2, gen_gvec_fpst_arg_zpz, + gen_helper_sve2_fcvtnt_ds, a, 0, FPST_FPCR) -static bool trans_BFCVTNT(DisasContext *s, arg_rpr_esz *a) -{ - if (!dc_isar_feature(aa64_sve_bf16, s)) { - return false; - } - return do_zpz_ptr(s, a->rd, a->rn, a->pg, false, gen_helper_sve_bfcvtnt); -} +TRANS_FEAT(BFCVTNT, aa64_sve_bf16, gen_gvec_fpst_arg_zpz, + gen_helper_sve_bfcvtnt, a, 0, FPST_FPCR) -static bool trans_FCVTNT_ds(DisasContext *s, arg_rpr_esz *a) -{ - if (!dc_isar_feature(aa64_sve2, s)) { - return false; - } - return do_zpz_ptr(s, a->rd, a->rn, a->pg, false, gen_helper_sve2_fcvtnt_ds); -} - -static bool trans_FCVTLT_hs(DisasContext *s, arg_rpr_esz *a) -{ - if (!dc_isar_feature(aa64_sve2, s)) { - return false; - } - return do_zpz_ptr(s, a->rd, a->rn, a->pg, false, gen_helper_sve2_fcvtlt_hs); -} - -static bool trans_FCVTLT_sd(DisasContext *s, arg_rpr_esz *a) -{ - if (!dc_isar_feature(aa64_sve2, s)) { - return false; - } - return do_zpz_ptr(s, a->rd, a->rn, a->pg, false, gen_helper_sve2_fcvtlt_sd); -} +TRANS_FEAT(FCVTLT_hs, aa64_sve2, gen_gvec_fpst_arg_zpz, + gen_helper_sve2_fcvtlt_hs, a, 0, FPST_FPCR) +TRANS_FEAT(FCVTLT_sd, aa64_sve2, gen_gvec_fpst_arg_zpz, + gen_helper_sve2_fcvtlt_sd, a, 0, FPST_FPCR) static bool trans_FCVTX_ds(DisasContext *s, arg_rpr_esz *a) { -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-95-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 52 +++++++++++++++++--------------------- 1 file changed, 23 insertions(+), 29 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ TRANS_FEAT(FRINTX, aa64_sve, gen_gvec_fpst_arg_zpz, frintx_fns[a->esz], static bool do_frint_mode(DisasContext *s, arg_rpr_esz *a, int mode, gen_helper_gvec_3_ptr *fn) { - if (sve_access_check(s)) { - unsigned vsz = vec_full_reg_size(s); - TCGv_i32 tmode = tcg_const_i32(mode); - TCGv_ptr status = fpstatus_ptr(a->esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR); + unsigned vsz; + TCGv_i32 tmode; + TCGv_ptr status; - gen_helper_set_rmode(tmode, tmode, status); - - tcg_gen_gvec_3_ptr(vec_full_reg_offset(s, a->rd), - vec_full_reg_offset(s, a->rn), - pred_full_reg_offset(s, a->pg), - status, vsz, vsz, 0, fn); - - gen_helper_set_rmode(tmode, tmode, status); - tcg_temp_free_i32(tmode); - tcg_temp_free_ptr(status); + if (fn == NULL) { + return false; } + if (!sve_access_check(s)) { + return true; + } + + vsz = vec_full_reg_size(s); + tmode = tcg_const_i32(mode); + status = fpstatus_ptr(a->esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR); + + gen_helper_set_rmode(tmode, tmode, status); + + tcg_gen_gvec_3_ptr(vec_full_reg_offset(s, a->rd), + vec_full_reg_offset(s, a->rn), + pred_full_reg_offset(s, a->pg), + status, vsz, vsz, 0, fn); + + gen_helper_set_rmode(tmode, tmode, status); + tcg_temp_free_i32(tmode); + tcg_temp_free_ptr(status); return true; } static bool trans_FRINTN(DisasContext *s, arg_rpr_esz *a) { - if (a->esz == 0) { - return false; - } return do_frint_mode(s, a, float_round_nearest_even, frint_fns[a->esz]); } static bool trans_FRINTP(DisasContext *s, arg_rpr_esz *a) { - if (a->esz == 0) { - return false; - } return do_frint_mode(s, a, float_round_up, frint_fns[a->esz]); } static bool trans_FRINTM(DisasContext *s, arg_rpr_esz *a) { - if (a->esz == 0) { - return false; - } return do_frint_mode(s, a, float_round_down, frint_fns[a->esz]); } static bool trans_FRINTZ(DisasContext *s, arg_rpr_esz *a) { - if (a->esz == 0) { - return false; - } return do_frint_mode(s, a, float_round_to_zero, frint_fns[a->esz]); } static bool trans_FRINTA(DisasContext *s, arg_rpr_esz *a) { - if (a->esz == 0) { - return false; - } return do_frint_mode(s, a, float_round_ties_away, frint_fns[a->esz]); } -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-96-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 53 ++++++++++---------------------------- 1 file changed, 14 insertions(+), 39 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool do_frint_mode(DisasContext *s, arg_rpr_esz *a, return true; } -static bool trans_FRINTN(DisasContext *s, arg_rpr_esz *a) -{ - return do_frint_mode(s, a, float_round_nearest_even, frint_fns[a->esz]); -} - -static bool trans_FRINTP(DisasContext *s, arg_rpr_esz *a) -{ - return do_frint_mode(s, a, float_round_up, frint_fns[a->esz]); -} - -static bool trans_FRINTM(DisasContext *s, arg_rpr_esz *a) -{ - return do_frint_mode(s, a, float_round_down, frint_fns[a->esz]); -} - -static bool trans_FRINTZ(DisasContext *s, arg_rpr_esz *a) -{ - return do_frint_mode(s, a, float_round_to_zero, frint_fns[a->esz]); -} - -static bool trans_FRINTA(DisasContext *s, arg_rpr_esz *a) -{ - return do_frint_mode(s, a, float_round_ties_away, frint_fns[a->esz]); -} +TRANS_FEAT(FRINTN, aa64_sve, do_frint_mode, a, + float_round_nearest_even, frint_fns[a->esz]) +TRANS_FEAT(FRINTP, aa64_sve, do_frint_mode, a, + float_round_up, frint_fns[a->esz]) +TRANS_FEAT(FRINTM, aa64_sve, do_frint_mode, a, + float_round_down, frint_fns[a->esz]) +TRANS_FEAT(FRINTZ, aa64_sve, do_frint_mode, a, + float_round_to_zero, frint_fns[a->esz]) +TRANS_FEAT(FRINTA, aa64_sve, do_frint_mode, a, + float_round_ties_away, frint_fns[a->esz]) static gen_helper_gvec_3_ptr * const frecpx_fns[] = { NULL, gen_helper_sve_frecpx_h, @@ -XXX,XX +XXX,XX @@ TRANS_FEAT(FCVTLT_hs, aa64_sve2, gen_gvec_fpst_arg_zpz, TRANS_FEAT(FCVTLT_sd, aa64_sve2, gen_gvec_fpst_arg_zpz, gen_helper_sve2_fcvtlt_sd, a, 0, FPST_FPCR) -static bool trans_FCVTX_ds(DisasContext *s, arg_rpr_esz *a) -{ - if (!dc_isar_feature(aa64_sve2, s)) { - return false; - } - return do_frint_mode(s, a, float_round_to_odd, gen_helper_sve_fcvt_ds); -} - -static bool trans_FCVTXNT_ds(DisasContext *s, arg_rpr_esz *a) -{ - if (!dc_isar_feature(aa64_sve2, s)) { - return false; - } - return do_frint_mode(s, a, float_round_to_odd, gen_helper_sve2_fcvtnt_ds); -} +TRANS_FEAT(FCVTX_ds, aa64_sve2, do_frint_mode, a, + float_round_to_odd, gen_helper_sve_fcvt_ds) +TRANS_FEAT(FCVTXNT_ds, aa64_sve2, do_frint_mode, a, + float_round_to_odd, gen_helper_sve2_fcvtnt_ds) static bool trans_FLOGB(DisasContext *s, arg_rpr_esz *a) { -- 2.25.1 From: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20220527181907.189259-97-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 29 ++++++----------------------- 1 file changed, 6 insertions(+), 23 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ TRANS_FEAT(FCVTX_ds, aa64_sve2, do_frint_mode, a, TRANS_FEAT(FCVTXNT_ds, aa64_sve2, do_frint_mode, a, float_round_to_odd, gen_helper_sve2_fcvtnt_ds) -static bool trans_FLOGB(DisasContext *s, arg_rpr_esz *a) -{ - static gen_helper_gvec_3_ptr * const fns[] = { - NULL, gen_helper_flogb_h, - gen_helper_flogb_s, gen_helper_flogb_d - }; - - if (!dc_isar_feature(aa64_sve2, s) || fns[a->esz] == NULL) { - return false; - } - if (sve_access_check(s)) { - TCGv_ptr status = - fpstatus_ptr(a->esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR); - unsigned vsz = vec_full_reg_size(s); - - tcg_gen_gvec_3_ptr(vec_full_reg_offset(s, a->rd), - vec_full_reg_offset(s, a->rn), - pred_full_reg_offset(s, a->pg), - status, vsz, vsz, 0, fns[a->esz]); - tcg_temp_free_ptr(status); - } - return true; -} +static gen_helper_gvec_3_ptr * const flogb_fns[] = { + NULL, gen_helper_flogb_h, + gen_helper_flogb_s, gen_helper_flogb_d +}; +TRANS_FEAT(FLOGB, aa64_sve2, gen_gvec_fpst_arg_zpz, flogb_fns[a->esz], + a, 0, a->esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR) static bool do_FMLAL_zzzw(DisasContext *s, arg_rrrr_esz *a, bool sub, bool sel) { -- 2.25.1 Another very large pullreq (this one mostly because it has RTH's decodetree conversion series in it), but this should be the last of the really large things in my to-review queue... thanks -- PMM The following changes since commit 83aaec1d5a49f158abaa31797a0f976b3c07e5ca: Merge tag 'pull-tcg-20241212' of https://gitlab.com/rth7680/qemu into staging (2024-12-12 18:45:39 -0500) are available in the Git repository at: https://git.linaro.org/people/pmaydell/qemu-arm.git tags/pull-target-arm-20241213 for you to fetch changes up to 48e652c4bd9570f6f24def25355cb3009a7300f8: target/arm: Simplify condition for tlbi_el2_cp_reginfo[] (2024-12-13 15:41:09 +0000) ---------------------------------------------------------------- target-arm queue: * Finish conversion of A64 decoder to decodetree * Use float_round_to_odd in helper_fcvtx_f64_to_f32 * Move TLBI insn emulation code out to its own source file * docs/system/arm: fix broken links, document undocumented properties * MAINTAINERS: correct an email address ---------------------------------------------------------------- Brian Cain (1): MAINTAINERS: correct my email address Peter Maydell (10): target/arm: Move some TLBI insns to their own source file target/arm: Move TLBI insns for AArch32 EL2 to tlbi_insn_helper.c target/arm: Move AArch64 TLBI insns from v8_cp_reginfo[] target/arm: Move the AArch64 EL2 TLBI insns target/arm: Move AArch64 EL3 TLBI insns target/arm: Move TLBI range insns target/arm: Move the TLBI OS insns to tlb-insns.c. target/arm: Move small helper functions to tlb-insns.c target/arm: Move RME TLB insns to tlb-insns.c target/arm: Simplify condition for tlbi_el2_cp_reginfo[] Pierrick Bouvier (4): docs/system/arm/orangepi: update links docs/system/arm/fby35: document execute-in-place property docs/system/arm/xlnx-versal-virt: document ospi-flash property docs/system/arm/virt: document missing properties Richard Henderson (70): target/arm: Add section labels for "Data Processing (register)" target/arm: Convert UDIV, SDIV to decodetree target/arm: Convert LSLV, LSRV, ASRV, RORV to decodetree target/arm: Convert CRC32, CRC32C to decodetree target/arm: Convert SUBP, IRG, GMI to decodetree target/arm: Convert PACGA to decodetree target/arm: Convert RBIT, REV16, REV32, REV64 to decodetree target/arm: Convert CLZ, CLS to decodetree target/arm: Convert PAC[ID]*, AUT[ID]* to decodetree target/arm: Convert XPAC[ID] to decodetree target/arm: Convert disas_logic_reg to decodetree target/arm: Convert disas_add_sub_ext_reg to decodetree target/arm: Convert disas_add_sub_reg to decodetree target/arm: Convert disas_data_proc_3src to decodetree target/arm: Convert disas_adc_sbc to decodetree target/arm: Convert RMIF to decodetree target/arm: Convert SETF8, SETF16 to decodetree target/arm: Convert CCMP, CCMN to decodetree target/arm: Convert disas_cond_select to decodetree target/arm: Introduce fp_access_check_scalar_hsd target/arm: Introduce fp_access_check_vector_hsd target/arm: Convert FCMP, FCMPE, FCCMP, FCCMPE to decodetree target/arm: Fix decode of fp16 vector fabs, fneg, fsqrt target/arm: Convert FMOV, FABS, FNEG (scalar) to decodetree target/arm: Pass fpstatus to vfp_sqrt* target/arm: Remove helper_sqrt_f16 target/arm: Convert FSQRT (scalar) to decodetree target/arm: Convert FRINT[NPMSAXI] (scalar) to decodetree target/arm: Convert BFCVT to decodetree target/arm: Convert FRINT{32, 64}[ZX] (scalar) to decodetree target/arm: Convert FCVT (scalar) to decodetree target/arm: Convert handle_fpfpcvt to decodetree target/arm: Convert FJCVTZS to decodetree target/arm: Convert handle_fmov to decodetree target/arm: Convert SQABS, SQNEG to decodetree target/arm: Convert ABS, NEG to decodetree target/arm: Introduce gen_gvec_cls, gen_gvec_clz target/arm: Convert CLS, CLZ (vector) to decodetree target/arm: Introduce gen_gvec_cnt, gen_gvec_rbit target/arm: Convert CNT, NOT, RBIT (vector) to decodetree target/arm: Convert CMGT, CMGE, GMLT, GMLE, CMEQ (zero) to decodetree target/arm: Introduce gen_gvec_rev{16,32,64} target/arm: Convert handle_rev to decodetree target/arm: Move helper_neon_addlp_{s8, s16} to neon_helper.c target/arm: Introduce gen_gvec_{s,u}{add,ada}lp target/arm: Convert handle_2misc_pairwise to decodetree target/arm: Remove helper_neon_{add,sub}l_u{16,32} target/arm: Introduce clear_vec target/arm: Convert XTN, SQXTUN, SQXTN, UQXTN to decodetree target/arm: Convert FCVTN, BFCVTN to decodetree target/arm: Convert FCVTXN to decodetree target/arm: Convert SHLL to decodetree target/arm: Implement gen_gvec_fabs, gen_gvec_fneg target/arm: Convert FABS, FNEG (vector) to decodetree target/arm: Convert FSQRT (vector) to decodetree target/arm: Convert FRINT* (vector) to decodetree target/arm: Convert FCVT* (vector, integer) scalar to decodetree target/arm: Convert FCVT* (vector, fixed-point) scalar to decodetree target/arm: Convert [US]CVTF (vector, integer) scalar to decodetree target/arm: Convert [US]CVTF (vector, fixed-point) scalar to decodetree target/arm: Rename helper_gvec_vcvt_[hf][su] with _rz target/arm: Convert [US]CVTF (vector) to decodetree target/arm: Convert FCVTZ[SU] (vector, fixed-point) to decodetree target/arm: Convert FCVT* (vector, integer) to decodetree target/arm: Convert handle_2misc_fcmp_zero to decodetree target/arm: Convert FRECPE, FRECPX, FRSQRTE to decodetree target/arm: Introduce gen_gvec_urecpe, gen_gvec_ursqrte target/arm: Convert URECPE and URSQRTE to decodetree target/arm: Convert FCVTL to decodetree target/arm: Use float_round_to_odd in helper_fcvtx_f64_to_f32 MAINTAINERS | 2 +- docs/system/arm/fby35.rst | 5 + docs/system/arm/orangepi.rst | 4 +- docs/system/arm/virt.rst | 16 + docs/system/arm/xlnx-versal-virt.rst | 3 + target/arm/helper.h | 43 +- target/arm/internals.h | 9 + target/arm/tcg/helper-a64.h | 7 - target/arm/tcg/translate.h | 35 + target/arm/tcg/a64.decode | 502 ++- target/arm/helper.c | 1208 +------- target/arm/tcg-stubs.c | 5 + target/arm/tcg/gengvec.c | 369 +++ target/arm/tcg/helper-a64.c | 122 +- target/arm/tcg/neon_helper.c | 106 +- target/arm/tcg/tlb-insns.c | 1266 ++++++++ target/arm/tcg/translate-a64.c | 5670 +++++++++++----------------------- target/arm/tcg/translate-neon.c | 337 +- target/arm/tcg/translate-vfp.c | 6 +- target/arm/tcg/vec_helper.c | 65 +- target/arm/vfp_helper.c | 16 +- target/arm/tcg/meson.build | 1 + 22 files changed, 4203 insertions(+), 5594 deletions(-) create mode 100644 target/arm/tcg/tlb-insns.c From: Richard Henderson <richard.henderson@linaro.org> At the same time, use ### to separate 3rd-level sections. We already use ### for 4.1.92 Data Processing (immediate), but not the two following two third-level sections: 4.1.93 Branches, and 4.1.94 Loads and stores. Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-2-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 19 +++++++++++++++++-- 1 file changed, 17 insertions(+), 2 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ UBFM . 10 100110 . ...... ...... ..... ..... @bitfield_32 EXTR 1 00 100111 1 0 rm:5 imm:6 rn:5 rd:5 &extract sf=1 EXTR 0 00 100111 0 0 rm:5 0 imm:5 rn:5 rd:5 &extract sf=0 -# Branches +### Branches %imm26 0:s26 !function=times_4 @branch . ..... .......................... &i imm=%imm26 @@ -XXX,XX +XXX,XX @@ HLT 1101 0100 010 ................ 000 00 @i16 # DCPS2 1101 0100 101 ................ 000 10 @i16 # DCPS3 1101 0100 101 ................ 000 11 @i16 -# Loads and stores +### Loads and stores &stxr rn rt rt2 rs sz lasr &stlr rn rt sz lasr @@ -XXX,XX +XXX,XX @@ CPYP 00 011 1 01000 ..... .... 01 ..... ..... @cpy CPYM 00 011 1 01010 ..... .... 01 ..... ..... @cpy CPYE 00 011 1 01100 ..... .... 01 ..... ..... @cpy +### Data Processing (register) + +# Data Processing (2-source) +# Data Processing (1-source) +# Logical (shifted reg) +# Add/subtract (shifted reg) +# Add/subtract (extended reg) +# Add/subtract (carry) +# Rotate right into flags +# Evaluate into flags +# Conditional compare (regster) +# Conditional compare (immediate) +# Conditional select +# Data Processing (3-source) + ### Cryptographic AES AESE 01001110 00 10100 00100 10 ..... ..... @r2r_q1e0 -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-3-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 7 ++++ target/arm/tcg/translate-a64.c | 64 +++++++++++++++++----------------- 2 files changed, 39 insertions(+), 32 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ &r rn &ri rd imm &rri_sf rd rn imm sf +&rrr_sf rd rn rm sf &i imm &rr_e rd rn esz &rri_e rd rn imm esz @@ -XXX,XX +XXX,XX @@ CPYE 00 011 1 01100 ..... .... 01 ..... ..... @cpy ### Data Processing (register) # Data Processing (2-source) + +@rrr_sf sf:1 .......... rm:5 ...... rn:5 rd:5 &rrr_sf + +UDIV . 00 11010110 ..... 00001 0 ..... ..... @rrr_sf +SDIV . 00 11010110 ..... 00001 1 ..... ..... @rrr_sf + # Data Processing (1-source) # Logical (shifted reg) # Add/subtract (shifted reg) diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ TRANS(UQRSHRN_si, do_scalar_shift_imm_narrow, a, uqrshrn_fns, 0, false) TRANS(SQSHRUN_si, do_scalar_shift_imm_narrow, a, sqshrun_fns, MO_SIGN, false) TRANS(SQRSHRUN_si, do_scalar_shift_imm_narrow, a, sqrshrun_fns, MO_SIGN, false) +static bool do_div(DisasContext *s, arg_rrr_sf *a, bool is_signed) +{ + TCGv_i64 tcg_n, tcg_m, tcg_rd; + tcg_rd = cpu_reg(s, a->rd); + + if (!a->sf && is_signed) { + tcg_n = tcg_temp_new_i64(); + tcg_m = tcg_temp_new_i64(); + tcg_gen_ext32s_i64(tcg_n, cpu_reg(s, a->rn)); + tcg_gen_ext32s_i64(tcg_m, cpu_reg(s, a->rm)); + } else { + tcg_n = read_cpu_reg(s, a->rn, a->sf); + tcg_m = read_cpu_reg(s, a->rm, a->sf); + } + + if (is_signed) { + gen_helper_sdiv64(tcg_rd, tcg_n, tcg_m); + } else { + gen_helper_udiv64(tcg_rd, tcg_n, tcg_m); + } + + if (!a->sf) { /* zero extend final result */ + tcg_gen_ext32u_i64(tcg_rd, tcg_rd); + } + return true; +} + +TRANS(SDIV, do_div, a, true) +TRANS(UDIV, do_div, a, false) + /* Shift a TCGv src by TCGv shift_amount, put result in dst. * Note that it is the caller's responsibility to ensure that the * shift amount is in range (ie 0..31 or 0..63) and provide the ARM @@ -XXX,XX +XXX,XX @@ static void disas_data_proc_1src(DisasContext *s, uint32_t insn) #undef MAP } -static void handle_div(DisasContext *s, bool is_signed, unsigned int sf, - unsigned int rm, unsigned int rn, unsigned int rd) -{ - TCGv_i64 tcg_n, tcg_m, tcg_rd; - tcg_rd = cpu_reg(s, rd); - - if (!sf && is_signed) { - tcg_n = tcg_temp_new_i64(); - tcg_m = tcg_temp_new_i64(); - tcg_gen_ext32s_i64(tcg_n, cpu_reg(s, rn)); - tcg_gen_ext32s_i64(tcg_m, cpu_reg(s, rm)); - } else { - tcg_n = read_cpu_reg(s, rn, sf); - tcg_m = read_cpu_reg(s, rm, sf); - } - - if (is_signed) { - gen_helper_sdiv64(tcg_rd, tcg_n, tcg_m); - } else { - gen_helper_udiv64(tcg_rd, tcg_n, tcg_m); - } - - if (!sf) { /* zero extend final result */ - tcg_gen_ext32u_i64(tcg_rd, tcg_rd); - } -} /* LSLV, LSRV, ASRV, RORV */ static void handle_shift_reg(DisasContext *s, @@ -XXX,XX +XXX,XX @@ static void disas_data_proc_2src(DisasContext *s, uint32_t insn) } } break; - case 2: /* UDIV */ - handle_div(s, false, sf, rm, rn, rd); - break; - case 3: /* SDIV */ - handle_div(s, true, sf, rm, rn, rd); - break; case 4: /* IRG */ if (sf == 0 || !dc_isar_feature(aa64_mte_insn_reg, s)) { goto do_unallocated; @@ -XXX,XX +XXX,XX @@ static void disas_data_proc_2src(DisasContext *s, uint32_t insn) } default: do_unallocated: + case 2: /* UDIV */ + case 3: /* SDIV */ unallocated_encoding(s); break; } -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-4-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 4 +++ target/arm/tcg/translate-a64.c | 46 ++++++++++++++++------------------ 2 files changed, 25 insertions(+), 25 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ CPYE 00 011 1 01100 ..... .... 01 ..... ..... @cpy UDIV . 00 11010110 ..... 00001 0 ..... ..... @rrr_sf SDIV . 00 11010110 ..... 00001 1 ..... ..... @rrr_sf +LSLV . 00 11010110 ..... 00100 0 ..... ..... @rrr_sf +LSRV . 00 11010110 ..... 00100 1 ..... ..... @rrr_sf +ASRV . 00 11010110 ..... 00101 0 ..... ..... @rrr_sf +RORV . 00 11010110 ..... 00101 1 ..... ..... @rrr_sf # Data Processing (1-source) # Logical (shifted reg) diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static void shift_reg_imm(TCGv_i64 dst, TCGv_i64 src, int sf, } } +static bool do_shift_reg(DisasContext *s, arg_rrr_sf *a, + enum a64_shift_type shift_type) +{ + TCGv_i64 tcg_shift = tcg_temp_new_i64(); + TCGv_i64 tcg_rd = cpu_reg(s, a->rd); + TCGv_i64 tcg_rn = read_cpu_reg(s, a->rn, a->sf); + + tcg_gen_andi_i64(tcg_shift, cpu_reg(s, a->rm), a->sf ? 63 : 31); + shift_reg(tcg_rd, tcg_rn, a->sf, shift_type, tcg_shift); + return true; +} + +TRANS(LSLV, do_shift_reg, a, A64_SHIFT_TYPE_LSL) +TRANS(LSRV, do_shift_reg, a, A64_SHIFT_TYPE_LSR) +TRANS(ASRV, do_shift_reg, a, A64_SHIFT_TYPE_ASR) +TRANS(RORV, do_shift_reg, a, A64_SHIFT_TYPE_ROR) + /* Logical (shifted register) * 31 30 29 28 24 23 22 21 20 16 15 10 9 5 4 0 * +----+-----+-----------+-------+---+------+--------+------+------+ @@ -XXX,XX +XXX,XX @@ static void disas_data_proc_1src(DisasContext *s, uint32_t insn) } -/* LSLV, LSRV, ASRV, RORV */ -static void handle_shift_reg(DisasContext *s, - enum a64_shift_type shift_type, unsigned int sf, - unsigned int rm, unsigned int rn, unsigned int rd) -{ - TCGv_i64 tcg_shift = tcg_temp_new_i64(); - TCGv_i64 tcg_rd = cpu_reg(s, rd); - TCGv_i64 tcg_rn = read_cpu_reg(s, rn, sf); - - tcg_gen_andi_i64(tcg_shift, cpu_reg(s, rm), sf ? 63 : 31); - shift_reg(tcg_rd, tcg_rn, sf, shift_type, tcg_shift); -} - /* CRC32[BHWX], CRC32C[BHWX] */ static void handle_crc32(DisasContext *s, unsigned int sf, unsigned int sz, bool crc32c, @@ -XXX,XX +XXX,XX @@ static void disas_data_proc_2src(DisasContext *s, uint32_t insn) tcg_gen_or_i64(cpu_reg(s, rd), cpu_reg(s, rm), t); } break; - case 8: /* LSLV */ - handle_shift_reg(s, A64_SHIFT_TYPE_LSL, sf, rm, rn, rd); - break; - case 9: /* LSRV */ - handle_shift_reg(s, A64_SHIFT_TYPE_LSR, sf, rm, rn, rd); - break; - case 10: /* ASRV */ - handle_shift_reg(s, A64_SHIFT_TYPE_ASR, sf, rm, rn, rd); - break; - case 11: /* RORV */ - handle_shift_reg(s, A64_SHIFT_TYPE_ROR, sf, rm, rn, rd); - break; case 12: /* PACGA */ if (sf == 0 || !dc_isar_feature(aa64_pauth, s)) { goto do_unallocated; @@ -XXX,XX +XXX,XX @@ static void disas_data_proc_2src(DisasContext *s, uint32_t insn) do_unallocated: case 2: /* UDIV */ case 3: /* SDIV */ + case 8: /* LSLV */ + case 9: /* LSRV */ + case 10: /* ASRV */ + case 11: /* RORV */ unallocated_encoding(s); break; } -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-5-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 12 ++++ target/arm/tcg/translate-a64.c | 101 +++++++++++++-------------------- 2 files changed, 53 insertions(+), 60 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ @rr_d ........ ... ..... ...... rn:5 rd:5 &rr_e esz=3 @rr_sd ........ ... ..... ...... rn:5 rd:5 &rr_e esz=%esz_sd +@rrr_b ........ ... rm:5 ...... rn:5 rd:5 &rrr_e esz=0 @rrr_h ........ ... rm:5 ...... rn:5 rd:5 &rrr_e esz=1 +@rrr_s ........ ... rm:5 ...... rn:5 rd:5 &rrr_e esz=2 @rrr_d ........ ... rm:5 ...... rn:5 rd:5 &rrr_e esz=3 @rrr_sd ........ ... rm:5 ...... rn:5 rd:5 &rrr_e esz=%esz_sd @rrr_hsd ........ ... rm:5 ...... rn:5 rd:5 &rrr_e esz=%esz_hsd @@ -XXX,XX +XXX,XX @@ LSRV . 00 11010110 ..... 00100 1 ..... ..... @rrr_sf ASRV . 00 11010110 ..... 00101 0 ..... ..... @rrr_sf RORV . 00 11010110 ..... 00101 1 ..... ..... @rrr_sf +CRC32 0 00 11010110 ..... 0100 00 ..... ..... @rrr_b +CRC32 0 00 11010110 ..... 0100 01 ..... ..... @rrr_h +CRC32 0 00 11010110 ..... 0100 10 ..... ..... @rrr_s +CRC32 1 00 11010110 ..... 0100 11 ..... ..... @rrr_d + +CRC32C 0 00 11010110 ..... 0101 00 ..... ..... @rrr_b +CRC32C 0 00 11010110 ..... 0101 01 ..... ..... @rrr_h +CRC32C 0 00 11010110 ..... 0101 10 ..... ..... @rrr_s +CRC32C 1 00 11010110 ..... 0101 11 ..... ..... @rrr_d + # Data Processing (1-source) # Logical (shifted reg) # Add/subtract (shifted reg) diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ TRANS(LSRV, do_shift_reg, a, A64_SHIFT_TYPE_LSR) TRANS(ASRV, do_shift_reg, a, A64_SHIFT_TYPE_ASR) TRANS(RORV, do_shift_reg, a, A64_SHIFT_TYPE_ROR) +static bool do_crc32(DisasContext *s, arg_rrr_e *a, bool crc32c) +{ + TCGv_i64 tcg_acc, tcg_val, tcg_rd; + TCGv_i32 tcg_bytes; + + switch (a->esz) { + case MO_8: + case MO_16: + case MO_32: + tcg_val = tcg_temp_new_i64(); + tcg_gen_extract_i64(tcg_val, cpu_reg(s, a->rm), 0, 8 << a->esz); + break; + case MO_64: + tcg_val = cpu_reg(s, a->rm); + break; + default: + g_assert_not_reached(); + } + tcg_acc = cpu_reg(s, a->rn); + tcg_bytes = tcg_constant_i32(1 << a->esz); + tcg_rd = cpu_reg(s, a->rd); + + if (crc32c) { + gen_helper_crc32c_64(tcg_rd, tcg_acc, tcg_val, tcg_bytes); + } else { + gen_helper_crc32_64(tcg_rd, tcg_acc, tcg_val, tcg_bytes); + } + return true; +} + +TRANS_FEAT(CRC32, aa64_crc32, do_crc32, a, false) +TRANS_FEAT(CRC32C, aa64_crc32, do_crc32, a, true) + /* Logical (shifted register) * 31 30 29 28 24 23 22 21 20 16 15 10 9 5 4 0 * +----+-----+-----------+-------+---+------+--------+------+------+ @@ -XXX,XX +XXX,XX @@ static void disas_data_proc_1src(DisasContext *s, uint32_t insn) } -/* CRC32[BHWX], CRC32C[BHWX] */ -static void handle_crc32(DisasContext *s, - unsigned int sf, unsigned int sz, bool crc32c, - unsigned int rm, unsigned int rn, unsigned int rd) -{ - TCGv_i64 tcg_acc, tcg_val; - TCGv_i32 tcg_bytes; - - if (!dc_isar_feature(aa64_crc32, s) - || (sf == 1 && sz != 3) - || (sf == 0 && sz == 3)) { - unallocated_encoding(s); - return; - } - - if (sz == 3) { - tcg_val = cpu_reg(s, rm); - } else { - uint64_t mask; - switch (sz) { - case 0: - mask = 0xFF; - break; - case 1: - mask = 0xFFFF; - break; - case 2: - mask = 0xFFFFFFFF; - break; - default: - g_assert_not_reached(); - } - tcg_val = tcg_temp_new_i64(); - tcg_gen_andi_i64(tcg_val, cpu_reg(s, rm), mask); - } - - tcg_acc = cpu_reg(s, rn); - tcg_bytes = tcg_constant_i32(1 << sz); - - if (crc32c) { - gen_helper_crc32c_64(cpu_reg(s, rd), tcg_acc, tcg_val, tcg_bytes); - } else { - gen_helper_crc32_64(cpu_reg(s, rd), tcg_acc, tcg_val, tcg_bytes); - } -} - /* Data-processing (2 source) * 31 30 29 28 21 20 16 15 10 9 5 4 0 * +----+---+---+-----------------+------+--------+------+------+ @@ -XXX,XX +XXX,XX @@ static void disas_data_proc_2src(DisasContext *s, uint32_t insn) gen_helper_pacga(cpu_reg(s, rd), tcg_env, cpu_reg(s, rn), cpu_reg_sp(s, rm)); break; - case 16: - case 17: - case 18: - case 19: - case 20: - case 21: - case 22: - case 23: /* CRC32 */ - { - int sz = extract32(opcode, 0, 2); - bool crc32c = extract32(opcode, 2, 1); - handle_crc32(s, sf, sz, crc32c, rm, rn, rd); - break; - } default: do_unallocated: case 2: /* UDIV */ @@ -XXX,XX +XXX,XX @@ static void disas_data_proc_2src(DisasContext *s, uint32_t insn) case 9: /* LSRV */ case 10: /* ASRV */ case 11: /* RORV */ + case 16: + case 17: + case 18: + case 19: + case 20: + case 21: + case 22: + case 23: /* CRC32 */ unallocated_encoding(s); break; } -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-6-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 7 +++ target/arm/tcg/translate-a64.c | 94 +++++++++++++++++++--------------- 2 files changed, 59 insertions(+), 42 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ %hlm 11:1 20:2 &r rn +&rrr rd rn rm &ri rd imm &rri_sf rd rn imm sf &rrr_sf rd rn rm sf @@ -XXX,XX +XXX,XX @@ CPYE 00 011 1 01100 ..... .... 01 ..... ..... @cpy # Data Processing (2-source) +@rrr . .......... rm:5 ...... rn:5 rd:5 &rrr @rrr_sf sf:1 .......... rm:5 ...... rn:5 rd:5 &rrr_sf UDIV . 00 11010110 ..... 00001 0 ..... ..... @rrr_sf @@ -XXX,XX +XXX,XX @@ CRC32C 0 00 11010110 ..... 0101 01 ..... ..... @rrr_h CRC32C 0 00 11010110 ..... 0101 10 ..... ..... @rrr_s CRC32C 1 00 11010110 ..... 0101 11 ..... ..... @rrr_d +SUBP 1 00 11010110 ..... 000000 ..... ..... @rrr +SUBPS 1 01 11010110 ..... 000000 ..... ..... @rrr +IRG 1 00 11010110 ..... 000100 ..... ..... @rrr +GMI 1 00 11010110 ..... 000101 ..... ..... @rrr + # Data Processing (1-source) # Logical (shifted reg) # Add/subtract (shifted reg) diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static bool do_crc32(DisasContext *s, arg_rrr_e *a, bool crc32c) TRANS_FEAT(CRC32, aa64_crc32, do_crc32, a, false) TRANS_FEAT(CRC32C, aa64_crc32, do_crc32, a, true) +static bool do_subp(DisasContext *s, arg_rrr *a, bool setflag) +{ + TCGv_i64 tcg_n = read_cpu_reg_sp(s, a->rn, true); + TCGv_i64 tcg_m = read_cpu_reg_sp(s, a->rm, true); + TCGv_i64 tcg_d = cpu_reg(s, a->rd); + + tcg_gen_sextract_i64(tcg_n, tcg_n, 0, 56); + tcg_gen_sextract_i64(tcg_m, tcg_m, 0, 56); + + if (setflag) { + gen_sub_CC(true, tcg_d, tcg_n, tcg_m); + } else { + tcg_gen_sub_i64(tcg_d, tcg_n, tcg_m); + } + return true; +} + +TRANS_FEAT(SUBP, aa64_mte_insn_reg, do_subp, a, false) +TRANS_FEAT(SUBPS, aa64_mte_insn_reg, do_subp, a, true) + +static bool trans_IRG(DisasContext *s, arg_rrr *a) +{ + if (dc_isar_feature(aa64_mte_insn_reg, s)) { + TCGv_i64 tcg_rd = cpu_reg_sp(s, a->rd); + TCGv_i64 tcg_rn = cpu_reg_sp(s, a->rn); + + if (s->ata[0]) { + gen_helper_irg(tcg_rd, tcg_env, tcg_rn, cpu_reg(s, a->rm)); + } else { + gen_address_with_allocation_tag0(tcg_rd, tcg_rn); + } + return true; + } + return false; +} + +static bool trans_GMI(DisasContext *s, arg_rrr *a) +{ + if (dc_isar_feature(aa64_mte_insn_reg, s)) { + TCGv_i64 t = tcg_temp_new_i64(); + + tcg_gen_extract_i64(t, cpu_reg_sp(s, a->rn), 56, 4); + tcg_gen_shl_i64(t, tcg_constant_i64(1), t); + tcg_gen_or_i64(cpu_reg(s, a->rd), cpu_reg(s, a->rm), t); + return true; + } + return false; +} + /* Logical (shifted register) * 31 30 29 28 24 23 22 21 20 16 15 10 9 5 4 0 * +----+-----+-----------+-------+---+------+--------+------+------+ @@ -XXX,XX +XXX,XX @@ static void disas_data_proc_2src(DisasContext *s, uint32_t insn) } switch (opcode) { - case 0: /* SUBP(S) */ - if (sf == 0 || !dc_isar_feature(aa64_mte_insn_reg, s)) { - goto do_unallocated; - } else { - TCGv_i64 tcg_n, tcg_m, tcg_d; - - tcg_n = read_cpu_reg_sp(s, rn, true); - tcg_m = read_cpu_reg_sp(s, rm, true); - tcg_gen_sextract_i64(tcg_n, tcg_n, 0, 56); - tcg_gen_sextract_i64(tcg_m, tcg_m, 0, 56); - tcg_d = cpu_reg(s, rd); - - if (setflag) { - gen_sub_CC(true, tcg_d, tcg_n, tcg_m); - } else { - tcg_gen_sub_i64(tcg_d, tcg_n, tcg_m); - } - } - break; - case 4: /* IRG */ - if (sf == 0 || !dc_isar_feature(aa64_mte_insn_reg, s)) { - goto do_unallocated; - } - if (s->ata[0]) { - gen_helper_irg(cpu_reg_sp(s, rd), tcg_env, - cpu_reg_sp(s, rn), cpu_reg(s, rm)); - } else { - gen_address_with_allocation_tag0(cpu_reg_sp(s, rd), - cpu_reg_sp(s, rn)); - } - break; - case 5: /* GMI */ - if (sf == 0 || !dc_isar_feature(aa64_mte_insn_reg, s)) { - goto do_unallocated; - } else { - TCGv_i64 t = tcg_temp_new_i64(); - - tcg_gen_extract_i64(t, cpu_reg_sp(s, rn), 56, 4); - tcg_gen_shl_i64(t, tcg_constant_i64(1), t); - tcg_gen_or_i64(cpu_reg(s, rd), cpu_reg(s, rm), t); - } - break; case 12: /* PACGA */ if (sf == 0 || !dc_isar_feature(aa64_pauth, s)) { goto do_unallocated; @@ -XXX,XX +XXX,XX @@ static void disas_data_proc_2src(DisasContext *s, uint32_t insn) break; default: do_unallocated: + case 0: /* SUBP(S) */ case 2: /* UDIV */ case 3: /* SDIV */ + case 4: /* IRG */ + case 5: /* GMI */ case 8: /* LSLV */ case 9: /* LSRV */ case 10: /* ASRV */ -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Remove disas_data_proc_2src, as this was the last insn decoded by that function. Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-7-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 2 ++ target/arm/tcg/translate-a64.c | 65 ++++++---------------------------- 2 files changed, 13 insertions(+), 54 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ SUBPS 1 01 11010110 ..... 000000 ..... ..... @rrr IRG 1 00 11010110 ..... 000100 ..... ..... @rrr GMI 1 00 11010110 ..... 000101 ..... ..... @rrr +PACGA 1 00 11010110 ..... 001100 ..... ..... @rrr + # Data Processing (1-source) # Logical (shifted reg) # Add/subtract (shifted reg) diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static bool trans_GMI(DisasContext *s, arg_rrr *a) return false; } +static bool trans_PACGA(DisasContext *s, arg_rrr *a) +{ + if (dc_isar_feature(aa64_pauth, s)) { + gen_helper_pacga(cpu_reg(s, a->rd), tcg_env, + cpu_reg(s, a->rn), cpu_reg_sp(s, a->rm)); + return true; + } + return false; +} + /* Logical (shifted register) * 31 30 29 28 24 23 22 21 20 16 15 10 9 5 4 0 * +----+-----+-----------+-------+---+------+--------+------+------+ @@ -XXX,XX +XXX,XX @@ static void disas_data_proc_1src(DisasContext *s, uint32_t insn) } -/* Data-processing (2 source) - * 31 30 29 28 21 20 16 15 10 9 5 4 0 - * +----+---+---+-----------------+------+--------+------+------+ - * | sf | 0 | S | 1 1 0 1 0 1 1 0 | Rm | opcode | Rn | Rd | - * +----+---+---+-----------------+------+--------+------+------+ - */ -static void disas_data_proc_2src(DisasContext *s, uint32_t insn) -{ - unsigned int sf, rm, opcode, rn, rd, setflag; - sf = extract32(insn, 31, 1); - setflag = extract32(insn, 29, 1); - rm = extract32(insn, 16, 5); - opcode = extract32(insn, 10, 6); - rn = extract32(insn, 5, 5); - rd = extract32(insn, 0, 5); - - if (setflag && opcode != 0) { - unallocated_encoding(s); - return; - } - - switch (opcode) { - case 12: /* PACGA */ - if (sf == 0 || !dc_isar_feature(aa64_pauth, s)) { - goto do_unallocated; - } - gen_helper_pacga(cpu_reg(s, rd), tcg_env, - cpu_reg(s, rn), cpu_reg_sp(s, rm)); - break; - default: - do_unallocated: - case 0: /* SUBP(S) */ - case 2: /* UDIV */ - case 3: /* SDIV */ - case 4: /* IRG */ - case 5: /* GMI */ - case 8: /* LSLV */ - case 9: /* LSRV */ - case 10: /* ASRV */ - case 11: /* RORV */ - case 16: - case 17: - case 18: - case 19: - case 20: - case 21: - case 22: - case 23: /* CRC32 */ - unallocated_encoding(s); - break; - } -} - /* * Data processing - register * 31 30 29 28 25 21 20 16 10 0 @@ -XXX,XX +XXX,XX @@ static void disas_data_proc_reg(DisasContext *s, uint32_t insn) if (op0) { /* (1 source) */ disas_data_proc_1src(s, insn); } else { /* (2 source) */ - disas_data_proc_2src(s, insn); + goto do_unallocated; } break; case 0x8 ... 0xf: /* (3 source) */ -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-8-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 11 +++ target/arm/tcg/translate-a64.c | 137 +++++++++++++++------------------ 2 files changed, 72 insertions(+), 76 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ &r rn &rrr rd rn rm &ri rd imm +&rr rd rn +&rr_sf rd rn sf &rri_sf rd rn imm sf &rrr_sf rd rn rm sf &i imm @@ -XXX,XX +XXX,XX @@ GMI 1 00 11010110 ..... 000101 ..... ..... @rrr PACGA 1 00 11010110 ..... 001100 ..... ..... @rrr # Data Processing (1-source) + +@rr . .......... ..... ...... rn:5 rd:5 &rr +@rr_sf sf:1 .......... ..... ...... rn:5 rd:5 &rr_sf + +RBIT . 10 11010110 00000 000000 ..... ..... @rr_sf +REV16 . 10 11010110 00000 000001 ..... ..... @rr_sf +REV32 . 10 11010110 00000 000010 ..... ..... @rr_sf +REV64 1 10 11010110 00000 000011 ..... ..... @rr + # Logical (shifted reg) # Add/subtract (shifted reg) # Add/subtract (extended reg) diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static bool trans_PACGA(DisasContext *s, arg_rrr *a) return false; } +typedef void ArithOneOp(TCGv_i64, TCGv_i64); + +static bool gen_rr(DisasContext *s, int rd, int rn, ArithOneOp fn) +{ + fn(cpu_reg(s, rd), cpu_reg(s, rn)); + return true; +} + +static void gen_rbit32(TCGv_i64 tcg_rd, TCGv_i64 tcg_rn) +{ + TCGv_i32 t32 = tcg_temp_new_i32(); + + tcg_gen_extrl_i64_i32(t32, tcg_rn); + gen_helper_rbit(t32, t32); + tcg_gen_extu_i32_i64(tcg_rd, t32); +} + +static void gen_rev16_xx(TCGv_i64 tcg_rd, TCGv_i64 tcg_rn, TCGv_i64 mask) +{ + TCGv_i64 tcg_tmp = tcg_temp_new_i64(); + + tcg_gen_shri_i64(tcg_tmp, tcg_rn, 8); + tcg_gen_and_i64(tcg_rd, tcg_rn, mask); + tcg_gen_and_i64(tcg_tmp, tcg_tmp, mask); + tcg_gen_shli_i64(tcg_rd, tcg_rd, 8); + tcg_gen_or_i64(tcg_rd, tcg_rd, tcg_tmp); +} + +static void gen_rev16_32(TCGv_i64 tcg_rd, TCGv_i64 tcg_rn) +{ + gen_rev16_xx(tcg_rd, tcg_rn, tcg_constant_i64(0x00ff00ff)); +} + +static void gen_rev16_64(TCGv_i64 tcg_rd, TCGv_i64 tcg_rn) +{ + gen_rev16_xx(tcg_rd, tcg_rn, tcg_constant_i64(0x00ff00ff00ff00ffull)); +} + +static void gen_rev_32(TCGv_i64 tcg_rd, TCGv_i64 tcg_rn) +{ + tcg_gen_bswap32_i64(tcg_rd, tcg_rn, TCG_BSWAP_OZ); +} + +static void gen_rev32(TCGv_i64 tcg_rd, TCGv_i64 tcg_rn) +{ + tcg_gen_bswap64_i64(tcg_rd, tcg_rn); + tcg_gen_rotri_i64(tcg_rd, tcg_rd, 32); +} + +TRANS(RBIT, gen_rr, a->rd, a->rn, a->sf ? gen_helper_rbit64 : gen_rbit32) +TRANS(REV16, gen_rr, a->rd, a->rn, a->sf ? gen_rev16_64 : gen_rev16_32) +TRANS(REV32, gen_rr, a->rd, a->rn, a->sf ? gen_rev32 : gen_rev_32) +TRANS(REV64, gen_rr, a->rd, a->rn, tcg_gen_bswap64_i64) + /* Logical (shifted register) * 31 30 29 28 24 23 22 21 20 16 15 10 9 5 4 0 * +----+-----+-----------+-------+---+------+--------+------+------+ @@ -XXX,XX +XXX,XX @@ static void handle_cls(DisasContext *s, unsigned int sf, } } -static void handle_rbit(DisasContext *s, unsigned int sf, - unsigned int rn, unsigned int rd) -{ - TCGv_i64 tcg_rd, tcg_rn; - tcg_rd = cpu_reg(s, rd); - tcg_rn = cpu_reg(s, rn); - - if (sf) { - gen_helper_rbit64(tcg_rd, tcg_rn); - } else { - TCGv_i32 tcg_tmp32 = tcg_temp_new_i32(); - tcg_gen_extrl_i64_i32(tcg_tmp32, tcg_rn); - gen_helper_rbit(tcg_tmp32, tcg_tmp32); - tcg_gen_extu_i32_i64(tcg_rd, tcg_tmp32); - } -} - -/* REV with sf==1, opcode==3 ("REV64") */ -static void handle_rev64(DisasContext *s, unsigned int sf, - unsigned int rn, unsigned int rd) -{ - if (!sf) { - unallocated_encoding(s); - return; - } - tcg_gen_bswap64_i64(cpu_reg(s, rd), cpu_reg(s, rn)); -} - -/* REV with sf==0, opcode==2 - * REV32 (sf==1, opcode==2) - */ -static void handle_rev32(DisasContext *s, unsigned int sf, - unsigned int rn, unsigned int rd) -{ - TCGv_i64 tcg_rd = cpu_reg(s, rd); - TCGv_i64 tcg_rn = cpu_reg(s, rn); - - if (sf) { - tcg_gen_bswap64_i64(tcg_rd, tcg_rn); - tcg_gen_rotri_i64(tcg_rd, tcg_rd, 32); - } else { - tcg_gen_bswap32_i64(tcg_rd, tcg_rn, TCG_BSWAP_OZ); - } -} - -/* REV16 (opcode==1) */ -static void handle_rev16(DisasContext *s, unsigned int sf, - unsigned int rn, unsigned int rd) -{ - TCGv_i64 tcg_rd = cpu_reg(s, rd); - TCGv_i64 tcg_tmp = tcg_temp_new_i64(); - TCGv_i64 tcg_rn = read_cpu_reg(s, rn, sf); - TCGv_i64 mask = tcg_constant_i64(sf ? 0x00ff00ff00ff00ffull : 0x00ff00ff); - - tcg_gen_shri_i64(tcg_tmp, tcg_rn, 8); - tcg_gen_and_i64(tcg_rd, tcg_rn, mask); - tcg_gen_and_i64(tcg_tmp, tcg_tmp, mask); - tcg_gen_shli_i64(tcg_rd, tcg_rd, 8); - tcg_gen_or_i64(tcg_rd, tcg_rd, tcg_tmp); -} - /* Data-processing (1 source) * 31 30 29 28 21 20 16 15 10 9 5 4 0 * +----+---+---+-----------------+---------+--------+------+------+ @@ -XXX,XX +XXX,XX @@ static void disas_data_proc_1src(DisasContext *s, uint32_t insn) #define MAP(SF, O2, O1) ((SF) | (O1 << 1) | (O2 << 7)) switch (MAP(sf, opcode2, opcode)) { - case MAP(0, 0x00, 0x00): /* RBIT */ - case MAP(1, 0x00, 0x00): - handle_rbit(s, sf, rn, rd); - break; - case MAP(0, 0x00, 0x01): /* REV16 */ - case MAP(1, 0x00, 0x01): - handle_rev16(s, sf, rn, rd); - break; - case MAP(0, 0x00, 0x02): /* REV/REV32 */ - case MAP(1, 0x00, 0x02): - handle_rev32(s, sf, rn, rd); - break; - case MAP(1, 0x00, 0x03): /* REV64 */ - handle_rev64(s, sf, rn, rd); - break; case MAP(0, 0x00, 0x04): /* CLZ */ case MAP(1, 0x00, 0x04): handle_clz(s, sf, rn, rd); @@ -XXX,XX +XXX,XX @@ static void disas_data_proc_1src(DisasContext *s, uint32_t insn) break; default: do_unallocated: + case MAP(0, 0x00, 0x00): /* RBIT */ + case MAP(1, 0x00, 0x00): + case MAP(0, 0x00, 0x01): /* REV16 */ + case MAP(1, 0x00, 0x01): + case MAP(0, 0x00, 0x02): /* REV/REV32 */ + case MAP(1, 0x00, 0x02): + case MAP(1, 0x00, 0x03): /* REV64 */ unallocated_encoding(s); break; } -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-9-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 3 ++ target/arm/tcg/translate-a64.c | 72 ++++++++++++++-------------------- 2 files changed, 33 insertions(+), 42 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ REV16 . 10 11010110 00000 000001 ..... ..... @rr_sf REV32 . 10 11010110 00000 000010 ..... ..... @rr_sf REV64 1 10 11010110 00000 000011 ..... ..... @rr +CLZ . 10 11010110 00000 000100 ..... ..... @rr_sf +CLS . 10 11010110 00000 000101 ..... ..... @rr_sf + # Logical (shifted reg) # Add/subtract (shifted reg) # Add/subtract (extended reg) diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ TRANS(REV16, gen_rr, a->rd, a->rn, a->sf ? gen_rev16_64 : gen_rev16_32) TRANS(REV32, gen_rr, a->rd, a->rn, a->sf ? gen_rev32 : gen_rev_32) TRANS(REV64, gen_rr, a->rd, a->rn, tcg_gen_bswap64_i64) +static void gen_clz32(TCGv_i64 tcg_rd, TCGv_i64 tcg_rn) +{ + TCGv_i32 t32 = tcg_temp_new_i32(); + + tcg_gen_extrl_i64_i32(t32, tcg_rn); + tcg_gen_clzi_i32(t32, t32, 32); + tcg_gen_extu_i32_i64(tcg_rd, t32); +} + +static void gen_clz64(TCGv_i64 tcg_rd, TCGv_i64 tcg_rn) +{ + tcg_gen_clzi_i64(tcg_rd, tcg_rn, 64); +} + +static void gen_cls32(TCGv_i64 tcg_rd, TCGv_i64 tcg_rn) +{ + TCGv_i32 t32 = tcg_temp_new_i32(); + + tcg_gen_extrl_i64_i32(t32, tcg_rn); + tcg_gen_clrsb_i32(t32, t32); + tcg_gen_extu_i32_i64(tcg_rd, t32); +} + +TRANS(CLZ, gen_rr, a->rd, a->rn, a->sf ? gen_clz64 : gen_clz32) +TRANS(CLS, gen_rr, a->rd, a->rn, a->sf ? tcg_gen_clrsb_i64 : gen_cls32) + /* Logical (shifted register) * 31 30 29 28 24 23 22 21 20 16 15 10 9 5 4 0 * +----+-----+-----------+-------+---+------+--------+------+------+ @@ -XXX,XX +XXX,XX @@ static void disas_cond_select(DisasContext *s, uint32_t insn) } } -static void handle_clz(DisasContext *s, unsigned int sf, - unsigned int rn, unsigned int rd) -{ - TCGv_i64 tcg_rd, tcg_rn; - tcg_rd = cpu_reg(s, rd); - tcg_rn = cpu_reg(s, rn); - - if (sf) { - tcg_gen_clzi_i64(tcg_rd, tcg_rn, 64); - } else { - TCGv_i32 tcg_tmp32 = tcg_temp_new_i32(); - tcg_gen_extrl_i64_i32(tcg_tmp32, tcg_rn); - tcg_gen_clzi_i32(tcg_tmp32, tcg_tmp32, 32); - tcg_gen_extu_i32_i64(tcg_rd, tcg_tmp32); - } -} - -static void handle_cls(DisasContext *s, unsigned int sf, - unsigned int rn, unsigned int rd) -{ - TCGv_i64 tcg_rd, tcg_rn; - tcg_rd = cpu_reg(s, rd); - tcg_rn = cpu_reg(s, rn); - - if (sf) { - tcg_gen_clrsb_i64(tcg_rd, tcg_rn); - } else { - TCGv_i32 tcg_tmp32 = tcg_temp_new_i32(); - tcg_gen_extrl_i64_i32(tcg_tmp32, tcg_rn); - tcg_gen_clrsb_i32(tcg_tmp32, tcg_tmp32); - tcg_gen_extu_i32_i64(tcg_rd, tcg_tmp32); - } -} - /* Data-processing (1 source) * 31 30 29 28 21 20 16 15 10 9 5 4 0 * +----+---+---+-----------------+---------+--------+------+------+ @@ -XXX,XX +XXX,XX @@ static void disas_data_proc_1src(DisasContext *s, uint32_t insn) #define MAP(SF, O2, O1) ((SF) | (O1 << 1) | (O2 << 7)) switch (MAP(sf, opcode2, opcode)) { - case MAP(0, 0x00, 0x04): /* CLZ */ - case MAP(1, 0x00, 0x04): - handle_clz(s, sf, rn, rd); - break; - case MAP(0, 0x00, 0x05): /* CLS */ - case MAP(1, 0x00, 0x05): - handle_cls(s, sf, rn, rd); - break; case MAP(1, 0x01, 0x00): /* PACIA */ if (s->pauth_active) { tcg_rd = cpu_reg(s, rd); @@ -XXX,XX +XXX,XX @@ static void disas_data_proc_1src(DisasContext *s, uint32_t insn) case MAP(0, 0x00, 0x02): /* REV/REV32 */ case MAP(1, 0x00, 0x02): case MAP(1, 0x00, 0x03): /* REV64 */ + case MAP(0, 0x00, 0x04): /* CLZ */ + case MAP(1, 0x00, 0x04): + case MAP(0, 0x00, 0x05): /* CLS */ + case MAP(1, 0x00, 0x05): unallocated_encoding(s); break; } -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> This includes PACIA, PACIZA, PACIB, PACIZB, PACDA, PACDZA, PACDB, PACDZB, AUTIA, AUTIZA, AUTIB, AUTIZB, AUTDA, AUTDZA, AUTDB, AUTDZB. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-10-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 13 +++ target/arm/tcg/translate-a64.c | 173 +++++++++------------------------ 2 files changed, 58 insertions(+), 128 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ REV64 1 10 11010110 00000 000011 ..... ..... @rr CLZ . 10 11010110 00000 000100 ..... ..... @rr_sf CLS . 10 11010110 00000 000101 ..... ..... @rr_sf +&pacaut rd rn z +@pacaut . .. ........ ..... .. z:1 ... rn:5 rd:5 &pacaut + +PACIA 1 10 11010110 00001 00.000 ..... ..... @pacaut +PACIB 1 10 11010110 00001 00.001 ..... ..... @pacaut +PACDA 1 10 11010110 00001 00.010 ..... ..... @pacaut +PACDB 1 10 11010110 00001 00.011 ..... ..... @pacaut + +AUTIA 1 10 11010110 00001 00.100 ..... ..... @pacaut +AUTIB 1 10 11010110 00001 00.101 ..... ..... @pacaut +AUTDA 1 10 11010110 00001 00.110 ..... ..... @pacaut +AUTDB 1 10 11010110 00001 00.111 ..... ..... @pacaut + # Logical (shifted reg) # Add/subtract (shifted reg) # Add/subtract (extended reg) diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static void gen_cls32(TCGv_i64 tcg_rd, TCGv_i64 tcg_rn) TRANS(CLZ, gen_rr, a->rd, a->rn, a->sf ? gen_clz64 : gen_clz32) TRANS(CLS, gen_rr, a->rd, a->rn, a->sf ? tcg_gen_clrsb_i64 : gen_cls32) +static bool gen_pacaut(DisasContext *s, arg_pacaut *a, NeonGenTwo64OpEnvFn fn) +{ + TCGv_i64 tcg_rd, tcg_rn; + + if (a->z) { + if (a->rn != 31) { + return false; + } + tcg_rn = tcg_constant_i64(0); + } else { + tcg_rn = cpu_reg_sp(s, a->rn); + } + if (s->pauth_active) { + tcg_rd = cpu_reg(s, a->rd); + fn(tcg_rd, tcg_env, tcg_rd, tcg_rn); + } + return true; +} + +TRANS_FEAT(PACIA, aa64_pauth, gen_pacaut, a, gen_helper_pacia) +TRANS_FEAT(PACIB, aa64_pauth, gen_pacaut, a, gen_helper_pacib) +TRANS_FEAT(PACDA, aa64_pauth, gen_pacaut, a, gen_helper_pacda) +TRANS_FEAT(PACDB, aa64_pauth, gen_pacaut, a, gen_helper_pacdb) + +TRANS_FEAT(AUTIA, aa64_pauth, gen_pacaut, a, gen_helper_autia) +TRANS_FEAT(AUTIB, aa64_pauth, gen_pacaut, a, gen_helper_autib) +TRANS_FEAT(AUTDA, aa64_pauth, gen_pacaut, a, gen_helper_autda) +TRANS_FEAT(AUTDB, aa64_pauth, gen_pacaut, a, gen_helper_autdb) + /* Logical (shifted register) * 31 30 29 28 24 23 22 21 20 16 15 10 9 5 4 0 * +----+-----+-----------+-------+---+------+--------+------+------+ @@ -XXX,XX +XXX,XX @@ static void disas_data_proc_1src(DisasContext *s, uint32_t insn) #define MAP(SF, O2, O1) ((SF) | (O1 << 1) | (O2 << 7)) switch (MAP(sf, opcode2, opcode)) { - case MAP(1, 0x01, 0x00): /* PACIA */ - if (s->pauth_active) { - tcg_rd = cpu_reg(s, rd); - gen_helper_pacia(tcg_rd, tcg_env, tcg_rd, cpu_reg_sp(s, rn)); - } else if (!dc_isar_feature(aa64_pauth, s)) { - goto do_unallocated; - } - break; - case MAP(1, 0x01, 0x01): /* PACIB */ - if (s->pauth_active) { - tcg_rd = cpu_reg(s, rd); - gen_helper_pacib(tcg_rd, tcg_env, tcg_rd, cpu_reg_sp(s, rn)); - } else if (!dc_isar_feature(aa64_pauth, s)) { - goto do_unallocated; - } - break; - case MAP(1, 0x01, 0x02): /* PACDA */ - if (s->pauth_active) { - tcg_rd = cpu_reg(s, rd); - gen_helper_pacda(tcg_rd, tcg_env, tcg_rd, cpu_reg_sp(s, rn)); - } else if (!dc_isar_feature(aa64_pauth, s)) { - goto do_unallocated; - } - break; - case MAP(1, 0x01, 0x03): /* PACDB */ - if (s->pauth_active) { - tcg_rd = cpu_reg(s, rd); - gen_helper_pacdb(tcg_rd, tcg_env, tcg_rd, cpu_reg_sp(s, rn)); - } else if (!dc_isar_feature(aa64_pauth, s)) { - goto do_unallocated; - } - break; - case MAP(1, 0x01, 0x04): /* AUTIA */ - if (s->pauth_active) { - tcg_rd = cpu_reg(s, rd); - gen_helper_autia(tcg_rd, tcg_env, tcg_rd, cpu_reg_sp(s, rn)); - } else if (!dc_isar_feature(aa64_pauth, s)) { - goto do_unallocated; - } - break; - case MAP(1, 0x01, 0x05): /* AUTIB */ - if (s->pauth_active) { - tcg_rd = cpu_reg(s, rd); - gen_helper_autib(tcg_rd, tcg_env, tcg_rd, cpu_reg_sp(s, rn)); - } else if (!dc_isar_feature(aa64_pauth, s)) { - goto do_unallocated; - } - break; - case MAP(1, 0x01, 0x06): /* AUTDA */ - if (s->pauth_active) { - tcg_rd = cpu_reg(s, rd); - gen_helper_autda(tcg_rd, tcg_env, tcg_rd, cpu_reg_sp(s, rn)); - } else if (!dc_isar_feature(aa64_pauth, s)) { - goto do_unallocated; - } - break; - case MAP(1, 0x01, 0x07): /* AUTDB */ - if (s->pauth_active) { - tcg_rd = cpu_reg(s, rd); - gen_helper_autdb(tcg_rd, tcg_env, tcg_rd, cpu_reg_sp(s, rn)); - } else if (!dc_isar_feature(aa64_pauth, s)) { - goto do_unallocated; - } - break; - case MAP(1, 0x01, 0x08): /* PACIZA */ - if (!dc_isar_feature(aa64_pauth, s) || rn != 31) { - goto do_unallocated; - } else if (s->pauth_active) { - tcg_rd = cpu_reg(s, rd); - gen_helper_pacia(tcg_rd, tcg_env, tcg_rd, tcg_constant_i64(0)); - } - break; - case MAP(1, 0x01, 0x09): /* PACIZB */ - if (!dc_isar_feature(aa64_pauth, s) || rn != 31) { - goto do_unallocated; - } else if (s->pauth_active) { - tcg_rd = cpu_reg(s, rd); - gen_helper_pacib(tcg_rd, tcg_env, tcg_rd, tcg_constant_i64(0)); - } - break; - case MAP(1, 0x01, 0x0a): /* PACDZA */ - if (!dc_isar_feature(aa64_pauth, s) || rn != 31) { - goto do_unallocated; - } else if (s->pauth_active) { - tcg_rd = cpu_reg(s, rd); - gen_helper_pacda(tcg_rd, tcg_env, tcg_rd, tcg_constant_i64(0)); - } - break; - case MAP(1, 0x01, 0x0b): /* PACDZB */ - if (!dc_isar_feature(aa64_pauth, s) || rn != 31) { - goto do_unallocated; - } else if (s->pauth_active) { - tcg_rd = cpu_reg(s, rd); - gen_helper_pacdb(tcg_rd, tcg_env, tcg_rd, tcg_constant_i64(0)); - } - break; - case MAP(1, 0x01, 0x0c): /* AUTIZA */ - if (!dc_isar_feature(aa64_pauth, s) || rn != 31) { - goto do_unallocated; - } else if (s->pauth_active) { - tcg_rd = cpu_reg(s, rd); - gen_helper_autia(tcg_rd, tcg_env, tcg_rd, tcg_constant_i64(0)); - } - break; - case MAP(1, 0x01, 0x0d): /* AUTIZB */ - if (!dc_isar_feature(aa64_pauth, s) || rn != 31) { - goto do_unallocated; - } else if (s->pauth_active) { - tcg_rd = cpu_reg(s, rd); - gen_helper_autib(tcg_rd, tcg_env, tcg_rd, tcg_constant_i64(0)); - } - break; - case MAP(1, 0x01, 0x0e): /* AUTDZA */ - if (!dc_isar_feature(aa64_pauth, s) || rn != 31) { - goto do_unallocated; - } else if (s->pauth_active) { - tcg_rd = cpu_reg(s, rd); - gen_helper_autda(tcg_rd, tcg_env, tcg_rd, tcg_constant_i64(0)); - } - break; - case MAP(1, 0x01, 0x0f): /* AUTDZB */ - if (!dc_isar_feature(aa64_pauth, s) || rn != 31) { - goto do_unallocated; - } else if (s->pauth_active) { - tcg_rd = cpu_reg(s, rd); - gen_helper_autdb(tcg_rd, tcg_env, tcg_rd, tcg_constant_i64(0)); - } - break; case MAP(1, 0x01, 0x10): /* XPACI */ if (!dc_isar_feature(aa64_pauth, s) || rn != 31) { goto do_unallocated; @@ -XXX,XX +XXX,XX @@ static void disas_data_proc_1src(DisasContext *s, uint32_t insn) case MAP(1, 0x00, 0x04): case MAP(0, 0x00, 0x05): /* CLS */ case MAP(1, 0x00, 0x05): + case MAP(1, 0x01, 0x00): /* PACIA */ + case MAP(1, 0x01, 0x01): /* PACIB */ + case MAP(1, 0x01, 0x02): /* PACDA */ + case MAP(1, 0x01, 0x03): /* PACDB */ + case MAP(1, 0x01, 0x04): /* AUTIA */ + case MAP(1, 0x01, 0x05): /* AUTIB */ + case MAP(1, 0x01, 0x06): /* AUTDA */ + case MAP(1, 0x01, 0x07): /* AUTDB */ + case MAP(1, 0x01, 0x08): /* PACIZA */ + case MAP(1, 0x01, 0x09): /* PACIZB */ + case MAP(1, 0x01, 0x0a): /* PACDZA */ + case MAP(1, 0x01, 0x0b): /* PACDZB */ + case MAP(1, 0x01, 0x0c): /* AUTIZA */ + case MAP(1, 0x01, 0x0d): /* AUTIZB */ + case MAP(1, 0x01, 0x0e): /* AUTDZA */ + case MAP(1, 0x01, 0x0f): /* AUTDZB */ unallocated_encoding(s); break; } -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Remove disas_data_proc_1src, as these were the last insns decoded by that function. Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-11-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 3 ++ target/arm/tcg/translate-a64.c | 99 +++++----------------------------- 2 files changed, 16 insertions(+), 86 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ AUTIB 1 10 11010110 00001 00.101 ..... ..... @pacaut AUTDA 1 10 11010110 00001 00.110 ..... ..... @pacaut AUTDB 1 10 11010110 00001 00.111 ..... ..... @pacaut +XPACI 1 10 11010110 00001 010000 11111 rd:5 +XPACD 1 10 11010110 00001 010001 11111 rd:5 + # Logical (shifted reg) # Add/subtract (shifted reg) # Add/subtract (extended reg) diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ TRANS_FEAT(AUTIB, aa64_pauth, gen_pacaut, a, gen_helper_autib) TRANS_FEAT(AUTDA, aa64_pauth, gen_pacaut, a, gen_helper_autda) TRANS_FEAT(AUTDB, aa64_pauth, gen_pacaut, a, gen_helper_autdb) +static bool do_xpac(DisasContext *s, int rd, NeonGenOne64OpEnvFn *fn) +{ + if (s->pauth_active) { + TCGv_i64 tcg_rd = cpu_reg(s, rd); + fn(tcg_rd, tcg_env, tcg_rd); + } + return true; +} + +TRANS_FEAT(XPACI, aa64_pauth, do_xpac, a->rd, gen_helper_xpaci) +TRANS_FEAT(XPACD, aa64_pauth, do_xpac, a->rd, gen_helper_xpacd) + /* Logical (shifted register) * 31 30 29 28 24 23 22 21 20 16 15 10 9 5 4 0 * +----+-----+-----------+-------+---+------+--------+------+------+ @@ -XXX,XX +XXX,XX @@ static void disas_cond_select(DisasContext *s, uint32_t insn) } } -/* Data-processing (1 source) - * 31 30 29 28 21 20 16 15 10 9 5 4 0 - * +----+---+---+-----------------+---------+--------+------+------+ - * | sf | 1 | S | 1 1 0 1 0 1 1 0 | opcode2 | opcode | Rn | Rd | - * +----+---+---+-----------------+---------+--------+------+------+ - */ -static void disas_data_proc_1src(DisasContext *s, uint32_t insn) -{ - unsigned int sf, opcode, opcode2, rn, rd; - TCGv_i64 tcg_rd; - - if (extract32(insn, 29, 1)) { - unallocated_encoding(s); - return; - } - - sf = extract32(insn, 31, 1); - opcode = extract32(insn, 10, 6); - opcode2 = extract32(insn, 16, 5); - rn = extract32(insn, 5, 5); - rd = extract32(insn, 0, 5); - -#define MAP(SF, O2, O1) ((SF) | (O1 << 1) | (O2 << 7)) - - switch (MAP(sf, opcode2, opcode)) { - case MAP(1, 0x01, 0x10): /* XPACI */ - if (!dc_isar_feature(aa64_pauth, s) || rn != 31) { - goto do_unallocated; - } else if (s->pauth_active) { - tcg_rd = cpu_reg(s, rd); - gen_helper_xpaci(tcg_rd, tcg_env, tcg_rd); - } - break; - case MAP(1, 0x01, 0x11): /* XPACD */ - if (!dc_isar_feature(aa64_pauth, s) || rn != 31) { - goto do_unallocated; - } else if (s->pauth_active) { - tcg_rd = cpu_reg(s, rd); - gen_helper_xpacd(tcg_rd, tcg_env, tcg_rd); - } - break; - default: - do_unallocated: - case MAP(0, 0x00, 0x00): /* RBIT */ - case MAP(1, 0x00, 0x00): - case MAP(0, 0x00, 0x01): /* REV16 */ - case MAP(1, 0x00, 0x01): - case MAP(0, 0x00, 0x02): /* REV/REV32 */ - case MAP(1, 0x00, 0x02): - case MAP(1, 0x00, 0x03): /* REV64 */ - case MAP(0, 0x00, 0x04): /* CLZ */ - case MAP(1, 0x00, 0x04): - case MAP(0, 0x00, 0x05): /* CLS */ - case MAP(1, 0x00, 0x05): - case MAP(1, 0x01, 0x00): /* PACIA */ - case MAP(1, 0x01, 0x01): /* PACIB */ - case MAP(1, 0x01, 0x02): /* PACDA */ - case MAP(1, 0x01, 0x03): /* PACDB */ - case MAP(1, 0x01, 0x04): /* AUTIA */ - case MAP(1, 0x01, 0x05): /* AUTIB */ - case MAP(1, 0x01, 0x06): /* AUTDA */ - case MAP(1, 0x01, 0x07): /* AUTDB */ - case MAP(1, 0x01, 0x08): /* PACIZA */ - case MAP(1, 0x01, 0x09): /* PACIZB */ - case MAP(1, 0x01, 0x0a): /* PACDZA */ - case MAP(1, 0x01, 0x0b): /* PACDZB */ - case MAP(1, 0x01, 0x0c): /* AUTIZA */ - case MAP(1, 0x01, 0x0d): /* AUTIZB */ - case MAP(1, 0x01, 0x0e): /* AUTDZA */ - case MAP(1, 0x01, 0x0f): /* AUTDZB */ - unallocated_encoding(s); - break; - } - -#undef MAP -} - - /* * Data processing - register * 31 30 29 28 25 21 20 16 10 0 @@ -XXX,XX +XXX,XX @@ static void disas_data_proc_1src(DisasContext *s, uint32_t insn) */ static void disas_data_proc_reg(DisasContext *s, uint32_t insn) { - int op0 = extract32(insn, 30, 1); int op1 = extract32(insn, 28, 1); int op2 = extract32(insn, 21, 4); int op3 = extract32(insn, 10, 6); @@ -XXX,XX +XXX,XX @@ static void disas_data_proc_reg(DisasContext *s, uint32_t insn) disas_cond_select(s, insn); break; - case 0x6: /* Data-processing */ - if (op0) { /* (1 source) */ - disas_data_proc_1src(s, insn); - } else { /* (2 source) */ - goto do_unallocated; - } - break; case 0x8 ... 0xf: /* (3 source) */ disas_data_proc_3src(s, insn); break; default: do_unallocated: + case 0x6: /* Data-processing */ unallocated_encoding(s); break; } -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> This includes AND, BIC, ORR, ORN, EOR, EON, ANDS, BICS (shifted reg). Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-12-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 9 +++ target/arm/tcg/translate-a64.c | 117 ++++++++++++--------------------- 2 files changed, 51 insertions(+), 75 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ XPACI 1 10 11010110 00001 010000 11111 rd:5 XPACD 1 10 11010110 00001 010001 11111 rd:5 # Logical (shifted reg) + +&logic_shift rd rn rm sf sa st n +@logic_shift sf:1 .. ..... st:2 n:1 rm:5 sa:6 rn:5 rd:5 &logic_shift + +AND_r . 00 01010 .. . ..... ...... ..... ..... @logic_shift +ORR_r . 01 01010 .. . ..... ...... ..... ..... @logic_shift +EOR_r . 10 01010 .. . ..... ...... ..... ..... @logic_shift +ANDS_r . 11 01010 .. . ..... ...... ..... ..... @logic_shift + # Add/subtract (shifted reg) # Add/subtract (extended reg) # Add/subtract (carry) diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static bool do_xpac(DisasContext *s, int rd, NeonGenOne64OpEnvFn *fn) TRANS_FEAT(XPACI, aa64_pauth, do_xpac, a->rd, gen_helper_xpaci) TRANS_FEAT(XPACD, aa64_pauth, do_xpac, a->rd, gen_helper_xpacd) -/* Logical (shifted register) - * 31 30 29 28 24 23 22 21 20 16 15 10 9 5 4 0 - * +----+-----+-----------+-------+---+------+--------+------+------+ - * | sf | opc | 0 1 0 1 0 | shift | N | Rm | imm6 | Rn | Rd | - * +----+-----+-----------+-------+---+------+--------+------+------+ - */ -static void disas_logic_reg(DisasContext *s, uint32_t insn) +static bool do_logic_reg(DisasContext *s, arg_logic_shift *a, + ArithTwoOp *fn, ArithTwoOp *inv_fn, bool setflags) { TCGv_i64 tcg_rd, tcg_rn, tcg_rm; - unsigned int sf, opc, shift_type, invert, rm, shift_amount, rn, rd; - sf = extract32(insn, 31, 1); - opc = extract32(insn, 29, 2); - shift_type = extract32(insn, 22, 2); - invert = extract32(insn, 21, 1); - rm = extract32(insn, 16, 5); - shift_amount = extract32(insn, 10, 6); - rn = extract32(insn, 5, 5); - rd = extract32(insn, 0, 5); - - if (!sf && (shift_amount & (1 << 5))) { - unallocated_encoding(s); - return; + if (!a->sf && (a->sa & (1 << 5))) { + return false; } - tcg_rd = cpu_reg(s, rd); + tcg_rd = cpu_reg(s, a->rd); + tcg_rn = cpu_reg(s, a->rn); - if (opc == 1 && shift_amount == 0 && shift_type == 0 && rn == 31) { - /* Unshifted ORR and ORN with WZR/XZR is the standard encoding for - * register-register MOV and MVN, so it is worth special casing. - */ - tcg_rm = cpu_reg(s, rm); - if (invert) { + tcg_rm = read_cpu_reg(s, a->rm, a->sf); + if (a->sa) { + shift_reg_imm(tcg_rm, tcg_rm, a->sf, a->st, a->sa); + } + + (a->n ? inv_fn : fn)(tcg_rd, tcg_rn, tcg_rm); + if (!a->sf) { + tcg_gen_ext32u_i64(tcg_rd, tcg_rd); + } + if (setflags) { + gen_logic_CC(a->sf, tcg_rd); + } + return true; +} + +static bool trans_ORR_r(DisasContext *s, arg_logic_shift *a) +{ + /* + * Unshifted ORR and ORN with WZR/XZR is the standard encoding for + * register-register MOV and MVN, so it is worth special casing. + */ + if (a->sa == 0 && a->st == 0 && a->rn == 31) { + TCGv_i64 tcg_rd = cpu_reg(s, a->rd); + TCGv_i64 tcg_rm = cpu_reg(s, a->rm); + + if (a->n) { tcg_gen_not_i64(tcg_rd, tcg_rm); - if (!sf) { + if (!a->sf) { tcg_gen_ext32u_i64(tcg_rd, tcg_rd); } } else { - if (sf) { + if (a->sf) { tcg_gen_mov_i64(tcg_rd, tcg_rm); } else { tcg_gen_ext32u_i64(tcg_rd, tcg_rm); } } - return; + return true; } - tcg_rm = read_cpu_reg(s, rm, sf); - - if (shift_amount) { - shift_reg_imm(tcg_rm, tcg_rm, sf, shift_type, shift_amount); - } - - tcg_rn = cpu_reg(s, rn); - - switch (opc | (invert << 2)) { - case 0: /* AND */ - case 3: /* ANDS */ - tcg_gen_and_i64(tcg_rd, tcg_rn, tcg_rm); - break; - case 1: /* ORR */ - tcg_gen_or_i64(tcg_rd, tcg_rn, tcg_rm); - break; - case 2: /* EOR */ - tcg_gen_xor_i64(tcg_rd, tcg_rn, tcg_rm); - break; - case 4: /* BIC */ - case 7: /* BICS */ - tcg_gen_andc_i64(tcg_rd, tcg_rn, tcg_rm); - break; - case 5: /* ORN */ - tcg_gen_orc_i64(tcg_rd, tcg_rn, tcg_rm); - break; - case 6: /* EON */ - tcg_gen_eqv_i64(tcg_rd, tcg_rn, tcg_rm); - break; - default: - assert(FALSE); - break; - } - - if (!sf) { - tcg_gen_ext32u_i64(tcg_rd, tcg_rd); - } - - if (opc == 3) { - gen_logic_CC(sf, tcg_rd); - } + return do_logic_reg(s, a, tcg_gen_or_i64, tcg_gen_orc_i64, false); } +TRANS(AND_r, do_logic_reg, a, tcg_gen_and_i64, tcg_gen_andc_i64, false) +TRANS(ANDS_r, do_logic_reg, a, tcg_gen_and_i64, tcg_gen_andc_i64, true) +TRANS(EOR_r, do_logic_reg, a, tcg_gen_xor_i64, tcg_gen_eqv_i64, false) + /* * Add/subtract (extended register) * @@ -XXX,XX +XXX,XX @@ static void disas_data_proc_reg(DisasContext *s, uint32_t insn) /* Add/sub (shifted register) */ disas_add_sub_reg(s, insn); } - } else { - /* Logical (shifted register) */ - disas_logic_reg(s, insn); + return; } - return; + goto do_unallocated; } switch (op2) { -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> This includes ADD, SUB, ADDS, SUBS (extended register). Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-13-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 9 +++++ target/arm/tcg/translate-a64.c | 65 +++++++++++----------------------- 2 files changed, 29 insertions(+), 45 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ ANDS_r . 11 01010 .. . ..... ...... ..... ..... @logic_shift # Add/subtract (shifted reg) # Add/subtract (extended reg) + +&addsub_ext rd rn rm sf sa st +@addsub_ext sf:1 .. ........ rm:5 st:3 sa:3 rn:5 rd:5 &addsub_ext + +ADD_ext . 00 01011001 ..... ... ... ..... ..... @addsub_ext +SUB_ext . 10 01011001 ..... ... ... ..... ..... @addsub_ext +ADDS_ext . 01 01011001 ..... ... ... ..... ..... @addsub_ext +SUBS_ext . 11 01011001 ..... ... ... ..... ..... @addsub_ext + # Add/subtract (carry) # Rotate right into flags # Evaluate into flags diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ TRANS(AND_r, do_logic_reg, a, tcg_gen_and_i64, tcg_gen_andc_i64, false) TRANS(ANDS_r, do_logic_reg, a, tcg_gen_and_i64, tcg_gen_andc_i64, true) TRANS(EOR_r, do_logic_reg, a, tcg_gen_xor_i64, tcg_gen_eqv_i64, false) -/* - * Add/subtract (extended register) - * - * 31|30|29|28 24|23 22|21|20 16|15 13|12 10|9 5|4 0| - * +--+--+--+-----------+-----+--+-------+------+------+----+----+ - * |sf|op| S| 0 1 0 1 1 | opt | 1| Rm |option| imm3 | Rn | Rd | - * +--+--+--+-----------+-----+--+-------+------+------+----+----+ - * - * sf: 0 -> 32bit, 1 -> 64bit - * op: 0 -> add , 1 -> sub - * S: 1 -> set flags - * opt: 00 - * option: extension type (see DecodeRegExtend) - * imm3: optional shift to Rm - * - * Rd = Rn + LSL(extend(Rm), amount) - */ -static void disas_add_sub_ext_reg(DisasContext *s, uint32_t insn) +static bool do_addsub_ext(DisasContext *s, arg_addsub_ext *a, + bool sub_op, bool setflags) { - int rd = extract32(insn, 0, 5); - int rn = extract32(insn, 5, 5); - int imm3 = extract32(insn, 10, 3); - int option = extract32(insn, 13, 3); - int rm = extract32(insn, 16, 5); - int opt = extract32(insn, 22, 2); - bool setflags = extract32(insn, 29, 1); - bool sub_op = extract32(insn, 30, 1); - bool sf = extract32(insn, 31, 1); + TCGv_i64 tcg_rm, tcg_rn, tcg_rd, tcg_result; - TCGv_i64 tcg_rm, tcg_rn; /* temps */ - TCGv_i64 tcg_rd; - TCGv_i64 tcg_result; - - if (imm3 > 4 || opt != 0) { - unallocated_encoding(s); - return; + if (a->sa > 4) { + return false; } /* non-flag setting ops may use SP */ if (!setflags) { - tcg_rd = cpu_reg_sp(s, rd); + tcg_rd = cpu_reg_sp(s, a->rd); } else { - tcg_rd = cpu_reg(s, rd); + tcg_rd = cpu_reg(s, a->rd); } - tcg_rn = read_cpu_reg_sp(s, rn, sf); + tcg_rn = read_cpu_reg_sp(s, a->rn, a->sf); - tcg_rm = read_cpu_reg(s, rm, sf); - ext_and_shift_reg(tcg_rm, tcg_rm, option, imm3); + tcg_rm = read_cpu_reg(s, a->rm, a->sf); + ext_and_shift_reg(tcg_rm, tcg_rm, a->st, a->sa); tcg_result = tcg_temp_new_i64(); - if (!setflags) { if (sub_op) { tcg_gen_sub_i64(tcg_result, tcg_rn, tcg_rm); @@ -XXX,XX +XXX,XX @@ static void disas_add_sub_ext_reg(DisasContext *s, uint32_t insn) } } else { if (sub_op) { - gen_sub_CC(sf, tcg_result, tcg_rn, tcg_rm); + gen_sub_CC(a->sf, tcg_result, tcg_rn, tcg_rm); } else { - gen_add_CC(sf, tcg_result, tcg_rn, tcg_rm); + gen_add_CC(a->sf, tcg_result, tcg_rn, tcg_rm); } } - if (sf) { + if (a->sf) { tcg_gen_mov_i64(tcg_rd, tcg_result); } else { tcg_gen_ext32u_i64(tcg_rd, tcg_result); } + return true; } +TRANS(ADD_ext, do_addsub_ext, a, false, false) +TRANS(SUB_ext, do_addsub_ext, a, true, false) +TRANS(ADDS_ext, do_addsub_ext, a, false, true) +TRANS(SUBS_ext, do_addsub_ext, a, true, true) + /* * Add/subtract (shifted register) * @@ -XXX,XX +XXX,XX @@ static void disas_data_proc_reg(DisasContext *s, uint32_t insn) if (!op1) { if (op2 & 8) { if (op2 & 1) { - /* Add/sub (extended register) */ - disas_add_sub_ext_reg(s, insn); + goto do_unallocated; } else { /* Add/sub (shifted register) */ disas_add_sub_reg(s, insn); -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> This includes ADD, SUB, ADDS, SUBS (shifted register). Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-14-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 9 +++++ target/arm/tcg/translate-a64.c | 64 ++++++++++------------------------ 2 files changed, 27 insertions(+), 46 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ EOR_r . 10 01010 .. . ..... ...... ..... ..... @logic_shift ANDS_r . 11 01010 .. . ..... ...... ..... ..... @logic_shift # Add/subtract (shifted reg) + +&addsub_shift rd rn rm sf sa st +@addsub_shift sf:1 .. ..... st:2 . rm:5 sa:6 rn:5 rd:5 &addsub_shift + +ADD_r . 00 01011 .. 0 ..... ...... ..... ..... @addsub_shift +SUB_r . 10 01011 .. 0 ..... ...... ..... ..... @addsub_shift +ADDS_r . 01 01011 .. 0 ..... ...... ..... ..... @addsub_shift +SUBS_r . 11 01011 .. 0 ..... ...... ..... ..... @addsub_shift + # Add/subtract (extended reg) &addsub_ext rd rn rm sf sa st diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ TRANS(SUB_ext, do_addsub_ext, a, true, false) TRANS(ADDS_ext, do_addsub_ext, a, false, true) TRANS(SUBS_ext, do_addsub_ext, a, true, true) -/* - * Add/subtract (shifted register) - * - * 31 30 29 28 24 23 22 21 20 16 15 10 9 5 4 0 - * +--+--+--+-----------+-----+--+-------+---------+------+------+ - * |sf|op| S| 0 1 0 1 1 |shift| 0| Rm | imm6 | Rn | Rd | - * +--+--+--+-----------+-----+--+-------+---------+------+------+ - * - * sf: 0 -> 32bit, 1 -> 64bit - * op: 0 -> add , 1 -> sub - * S: 1 -> set flags - * shift: 00 -> LSL, 01 -> LSR, 10 -> ASR, 11 -> RESERVED - * imm6: Shift amount to apply to Rm before the add/sub - */ -static void disas_add_sub_reg(DisasContext *s, uint32_t insn) +static bool do_addsub_reg(DisasContext *s, arg_addsub_shift *a, + bool sub_op, bool setflags) { - int rd = extract32(insn, 0, 5); - int rn = extract32(insn, 5, 5); - int imm6 = extract32(insn, 10, 6); - int rm = extract32(insn, 16, 5); - int shift_type = extract32(insn, 22, 2); - bool setflags = extract32(insn, 29, 1); - bool sub_op = extract32(insn, 30, 1); - bool sf = extract32(insn, 31, 1); + TCGv_i64 tcg_rd, tcg_rn, tcg_rm, tcg_result; - TCGv_i64 tcg_rd = cpu_reg(s, rd); - TCGv_i64 tcg_rn, tcg_rm; - TCGv_i64 tcg_result; - - if ((shift_type == 3) || (!sf && (imm6 > 31))) { - unallocated_encoding(s); - return; + if (a->st == 3 || (!a->sf && (a->sa & 32))) { + return false; } - tcg_rn = read_cpu_reg(s, rn, sf); - tcg_rm = read_cpu_reg(s, rm, sf); + tcg_rd = cpu_reg(s, a->rd); + tcg_rn = read_cpu_reg(s, a->rn, a->sf); + tcg_rm = read_cpu_reg(s, a->rm, a->sf); - shift_reg_imm(tcg_rm, tcg_rm, sf, shift_type, imm6); + shift_reg_imm(tcg_rm, tcg_rm, a->sf, a->st, a->sa); tcg_result = tcg_temp_new_i64(); - if (!setflags) { if (sub_op) { tcg_gen_sub_i64(tcg_result, tcg_rn, tcg_rm); @@ -XXX,XX +XXX,XX @@ static void disas_add_sub_reg(DisasContext *s, uint32_t insn) } } else { if (sub_op) { - gen_sub_CC(sf, tcg_result, tcg_rn, tcg_rm); + gen_sub_CC(a->sf, tcg_result, tcg_rn, tcg_rm); } else { - gen_add_CC(sf, tcg_result, tcg_rn, tcg_rm); + gen_add_CC(a->sf, tcg_result, tcg_rn, tcg_rm); } } - if (sf) { + if (a->sf) { tcg_gen_mov_i64(tcg_rd, tcg_result); } else { tcg_gen_ext32u_i64(tcg_rd, tcg_result); } + return true; } +TRANS(ADD_r, do_addsub_reg, a, false, false) +TRANS(SUB_r, do_addsub_reg, a, true, false) +TRANS(ADDS_r, do_addsub_reg, a, false, true) +TRANS(SUBS_r, do_addsub_reg, a, true, true) + /* Data-processing (3 source) * * 31 30 29 28 24 23 21 20 16 15 14 10 9 5 4 0 @@ -XXX,XX +XXX,XX @@ static void disas_data_proc_reg(DisasContext *s, uint32_t insn) int op3 = extract32(insn, 10, 6); if (!op1) { - if (op2 & 8) { - if (op2 & 1) { - goto do_unallocated; - } else { - /* Add/sub (shifted register) */ - disas_add_sub_reg(s, insn); - } - return; - } goto do_unallocated; } -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> This includes MADD, MSUB, SMADDL, SMSUBL, UMADDL, UMSUBL, SMULH, UMULH. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-15-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 16 +++++ target/arm/tcg/translate-a64.c | 119 ++++++++++++--------------------- 2 files changed, 59 insertions(+), 76 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ SUBS_ext . 11 01011001 ..... ... ... ..... ..... @addsub_ext # Conditional select # Data Processing (3-source) +&rrrr rd rn rm ra +@rrrr . .. ........ rm:5 . ra:5 rn:5 rd:5 &rrrr + +MADD_w 0 00 11011000 ..... 0 ..... ..... ..... @rrrr +MSUB_w 0 00 11011000 ..... 1 ..... ..... ..... @rrrr +MADD_x 1 00 11011000 ..... 0 ..... ..... ..... @rrrr +MSUB_x 1 00 11011000 ..... 1 ..... ..... ..... @rrrr + +SMADDL 1 00 11011001 ..... 0 ..... ..... ..... @rrrr +SMSUBL 1 00 11011001 ..... 1 ..... ..... ..... @rrrr +UMADDL 1 00 11011101 ..... 0 ..... ..... ..... @rrrr +UMSUBL 1 00 11011101 ..... 1 ..... ..... ..... @rrrr + +SMULH 1 00 11011010 ..... 0 11111 ..... ..... @rrr +UMULH 1 00 11011110 ..... 0 11111 ..... ..... @rrr + ### Cryptographic AES AESE 01001110 00 10100 00100 10 ..... ..... @r2r_q1e0 diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ TRANS(SUB_r, do_addsub_reg, a, true, false) TRANS(ADDS_r, do_addsub_reg, a, false, true) TRANS(SUBS_r, do_addsub_reg, a, true, true) -/* Data-processing (3 source) - * - * 31 30 29 28 24 23 21 20 16 15 14 10 9 5 4 0 - * +--+------+-----------+------+------+----+------+------+------+ - * |sf| op54 | 1 1 0 1 1 | op31 | Rm | o0 | Ra | Rn | Rd | - * +--+------+-----------+------+------+----+------+------+------+ - */ -static void disas_data_proc_3src(DisasContext *s, uint32_t insn) +static bool do_mulh(DisasContext *s, arg_rrr *a, + void (*fn)(TCGv_i64, TCGv_i64, TCGv_i64, TCGv_i64)) { - int rd = extract32(insn, 0, 5); - int rn = extract32(insn, 5, 5); - int ra = extract32(insn, 10, 5); - int rm = extract32(insn, 16, 5); - int op_id = (extract32(insn, 29, 3) << 4) | - (extract32(insn, 21, 3) << 1) | - extract32(insn, 15, 1); - bool sf = extract32(insn, 31, 1); - bool is_sub = extract32(op_id, 0, 1); - bool is_high = extract32(op_id, 2, 1); - bool is_signed = false; - TCGv_i64 tcg_op1; - TCGv_i64 tcg_op2; - TCGv_i64 tcg_tmp; + TCGv_i64 discard = tcg_temp_new_i64(); + TCGv_i64 tcg_rd = cpu_reg(s, a->rd); + TCGv_i64 tcg_rn = cpu_reg(s, a->rn); + TCGv_i64 tcg_rm = cpu_reg(s, a->rm); - /* Note that op_id is sf:op54:op31:o0 so it includes the 32/64 size flag */ - switch (op_id) { - case 0x42: /* SMADDL */ - case 0x43: /* SMSUBL */ - case 0x44: /* SMULH */ - is_signed = true; - break; - case 0x0: /* MADD (32bit) */ - case 0x1: /* MSUB (32bit) */ - case 0x40: /* MADD (64bit) */ - case 0x41: /* MSUB (64bit) */ - case 0x4a: /* UMADDL */ - case 0x4b: /* UMSUBL */ - case 0x4c: /* UMULH */ - break; - default: - unallocated_encoding(s); - return; - } + fn(discard, tcg_rd, tcg_rn, tcg_rm); + return true; +} - if (is_high) { - TCGv_i64 low_bits = tcg_temp_new_i64(); /* low bits discarded */ - TCGv_i64 tcg_rd = cpu_reg(s, rd); - TCGv_i64 tcg_rn = cpu_reg(s, rn); - TCGv_i64 tcg_rm = cpu_reg(s, rm); +TRANS(SMULH, do_mulh, a, tcg_gen_muls2_i64) +TRANS(UMULH, do_mulh, a, tcg_gen_mulu2_i64) - if (is_signed) { - tcg_gen_muls2_i64(low_bits, tcg_rd, tcg_rn, tcg_rm); - } else { - tcg_gen_mulu2_i64(low_bits, tcg_rd, tcg_rn, tcg_rm); - } - return; - } +static bool do_muladd(DisasContext *s, arg_rrrr *a, + bool sf, bool is_sub, MemOp mop) +{ + TCGv_i64 tcg_rd = cpu_reg(s, a->rd); + TCGv_i64 tcg_op1, tcg_op2; - tcg_op1 = tcg_temp_new_i64(); - tcg_op2 = tcg_temp_new_i64(); - tcg_tmp = tcg_temp_new_i64(); - - if (op_id < 0x42) { - tcg_gen_mov_i64(tcg_op1, cpu_reg(s, rn)); - tcg_gen_mov_i64(tcg_op2, cpu_reg(s, rm)); + if (mop == MO_64) { + tcg_op1 = cpu_reg(s, a->rn); + tcg_op2 = cpu_reg(s, a->rm); } else { - if (is_signed) { - tcg_gen_ext32s_i64(tcg_op1, cpu_reg(s, rn)); - tcg_gen_ext32s_i64(tcg_op2, cpu_reg(s, rm)); - } else { - tcg_gen_ext32u_i64(tcg_op1, cpu_reg(s, rn)); - tcg_gen_ext32u_i64(tcg_op2, cpu_reg(s, rm)); - } + tcg_op1 = tcg_temp_new_i64(); + tcg_op2 = tcg_temp_new_i64(); + tcg_gen_ext_i64(tcg_op1, cpu_reg(s, a->rn), mop); + tcg_gen_ext_i64(tcg_op2, cpu_reg(s, a->rm), mop); } - if (ra == 31 && !is_sub) { + if (a->ra == 31 && !is_sub) { /* Special-case MADD with rA == XZR; it is the standard MUL alias */ - tcg_gen_mul_i64(cpu_reg(s, rd), tcg_op1, tcg_op2); + tcg_gen_mul_i64(tcg_rd, tcg_op1, tcg_op2); } else { + TCGv_i64 tcg_tmp = tcg_temp_new_i64(); + TCGv_i64 tcg_ra = cpu_reg(s, a->ra); + tcg_gen_mul_i64(tcg_tmp, tcg_op1, tcg_op2); if (is_sub) { - tcg_gen_sub_i64(cpu_reg(s, rd), cpu_reg(s, ra), tcg_tmp); + tcg_gen_sub_i64(tcg_rd, tcg_ra, tcg_tmp); } else { - tcg_gen_add_i64(cpu_reg(s, rd), cpu_reg(s, ra), tcg_tmp); + tcg_gen_add_i64(tcg_rd, tcg_ra, tcg_tmp); } } if (!sf) { - tcg_gen_ext32u_i64(cpu_reg(s, rd), cpu_reg(s, rd)); + tcg_gen_ext32u_i64(tcg_rd, tcg_rd); } + return true; } +TRANS(MADD_w, do_muladd, a, false, false, MO_64) +TRANS(MSUB_w, do_muladd, a, false, true, MO_64) +TRANS(MADD_x, do_muladd, a, true, false, MO_64) +TRANS(MSUB_x, do_muladd, a, true, true, MO_64) + +TRANS(SMADDL, do_muladd, a, true, false, MO_SL) +TRANS(SMSUBL, do_muladd, a, true, true, MO_SL) +TRANS(UMADDL, do_muladd, a, true, false, MO_UL) +TRANS(UMSUBL, do_muladd, a, true, true, MO_UL) + /* Add/subtract (with carry) * 31 30 29 28 27 26 25 24 23 22 21 20 16 15 10 9 5 4 0 * +--+--+--+------------------------+------+-------------+------+-----+ @@ -XXX,XX +XXX,XX @@ static void disas_data_proc_reg(DisasContext *s, uint32_t insn) disas_cond_select(s, insn); break; - case 0x8 ... 0xf: /* (3 source) */ - disas_data_proc_3src(s, insn); - break; - default: do_unallocated: case 0x6: /* Data-processing */ + case 0x8 ... 0xf: /* (3 source) */ unallocated_encoding(s); break; } -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> This includes ADC, SBC, ADCS, SBCS. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-16-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 6 +++++ target/arm/tcg/translate-a64.c | 43 +++++++++++++--------------------- 2 files changed, 22 insertions(+), 27 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ ADDS_ext . 01 01011001 ..... ... ... ..... ..... @addsub_ext SUBS_ext . 11 01011001 ..... ... ... ..... ..... @addsub_ext # Add/subtract (carry) + +ADC . 00 11010000 ..... 000000 ..... ..... @rrr_sf +ADCS . 01 11010000 ..... 000000 ..... ..... @rrr_sf +SBC . 10 11010000 ..... 000000 ..... ..... @rrr_sf +SBCS . 11 11010000 ..... 000000 ..... ..... @rrr_sf + # Rotate right into flags # Evaluate into flags # Conditional compare (regster) diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ TRANS(SMSUBL, do_muladd, a, true, true, MO_SL) TRANS(UMADDL, do_muladd, a, true, false, MO_UL) TRANS(UMSUBL, do_muladd, a, true, true, MO_UL) -/* Add/subtract (with carry) - * 31 30 29 28 27 26 25 24 23 22 21 20 16 15 10 9 5 4 0 - * +--+--+--+------------------------+------+-------------+------+-----+ - * |sf|op| S| 1 1 0 1 0 0 0 0 | rm | 0 0 0 0 0 0 | Rn | Rd | - * +--+--+--+------------------------+------+-------------+------+-----+ - */ - -static void disas_adc_sbc(DisasContext *s, uint32_t insn) +static bool do_adc_sbc(DisasContext *s, arg_rrr_sf *a, + bool is_sub, bool setflags) { - unsigned int sf, op, setflags, rm, rn, rd; TCGv_i64 tcg_y, tcg_rn, tcg_rd; - sf = extract32(insn, 31, 1); - op = extract32(insn, 30, 1); - setflags = extract32(insn, 29, 1); - rm = extract32(insn, 16, 5); - rn = extract32(insn, 5, 5); - rd = extract32(insn, 0, 5); + tcg_rd = cpu_reg(s, a->rd); + tcg_rn = cpu_reg(s, a->rn); - tcg_rd = cpu_reg(s, rd); - tcg_rn = cpu_reg(s, rn); - - if (op) { + if (is_sub) { tcg_y = tcg_temp_new_i64(); - tcg_gen_not_i64(tcg_y, cpu_reg(s, rm)); + tcg_gen_not_i64(tcg_y, cpu_reg(s, a->rm)); } else { - tcg_y = cpu_reg(s, rm); + tcg_y = cpu_reg(s, a->rm); } if (setflags) { - gen_adc_CC(sf, tcg_rd, tcg_rn, tcg_y); + gen_adc_CC(a->sf, tcg_rd, tcg_rn, tcg_y); } else { - gen_adc(sf, tcg_rd, tcg_rn, tcg_y); + gen_adc(a->sf, tcg_rd, tcg_rn, tcg_y); } + return true; } +TRANS(ADC, do_adc_sbc, a, false, false) +TRANS(SBC, do_adc_sbc, a, true, false) +TRANS(ADCS, do_adc_sbc, a, false, true) +TRANS(SBCS, do_adc_sbc, a, true, true) + /* * Rotate right into flags * 31 30 29 21 15 10 5 4 0 @@ -XXX,XX +XXX,XX @@ static void disas_data_proc_reg(DisasContext *s, uint32_t insn) switch (op2) { case 0x0: switch (op3) { - case 0x00: /* Add/subtract (with carry) */ - disas_adc_sbc(s, insn); - break; - case 0x01: /* Rotate right into flags */ case 0x21: disas_rotate_right_into_flags(s, insn); @@ -XXX,XX +XXX,XX @@ static void disas_data_proc_reg(DisasContext *s, uint32_t insn) break; default: + case 0x00: /* Add/subtract (with carry) */ goto do_unallocated; } break; -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-17-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 3 +++ target/arm/tcg/translate-a64.c | 32 +++++++++----------------------- 2 files changed, 12 insertions(+), 23 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ SBC . 10 11010000 ..... 000000 ..... ..... @rrr_sf SBCS . 11 11010000 ..... 000000 ..... ..... @rrr_sf # Rotate right into flags + +RMIF 1 01 11010000 imm:6 00001 rn:5 0 mask:4 + # Evaluate into flags # Conditional compare (regster) # Conditional compare (immediate) diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ TRANS(SBC, do_adc_sbc, a, true, false) TRANS(ADCS, do_adc_sbc, a, false, true) TRANS(SBCS, do_adc_sbc, a, true, true) -/* - * Rotate right into flags - * 31 30 29 21 15 10 5 4 0 - * +--+--+--+-----------------+--------+-----------+------+--+------+ - * |sf|op| S| 1 1 0 1 0 0 0 0 | imm6 | 0 0 0 0 1 | Rn |o2| mask | - * +--+--+--+-----------------+--------+-----------+------+--+------+ - */ -static void disas_rotate_right_into_flags(DisasContext *s, uint32_t insn) +static bool trans_RMIF(DisasContext *s, arg_RMIF *a) { - int mask = extract32(insn, 0, 4); - int o2 = extract32(insn, 4, 1); - int rn = extract32(insn, 5, 5); - int imm6 = extract32(insn, 15, 6); - int sf_op_s = extract32(insn, 29, 3); + int mask = a->mask; TCGv_i64 tcg_rn; TCGv_i32 nzcv; - if (sf_op_s != 5 || o2 != 0 || !dc_isar_feature(aa64_condm_4, s)) { - unallocated_encoding(s); - return; + if (!dc_isar_feature(aa64_condm_4, s)) { + return false; } - tcg_rn = read_cpu_reg(s, rn, 1); - tcg_gen_rotri_i64(tcg_rn, tcg_rn, imm6); + tcg_rn = read_cpu_reg(s, a->rn, 1); + tcg_gen_rotri_i64(tcg_rn, tcg_rn, a->imm); nzcv = tcg_temp_new_i32(); tcg_gen_extrl_i64_i32(nzcv, tcg_rn); @@ -XXX,XX +XXX,XX @@ static void disas_rotate_right_into_flags(DisasContext *s, uint32_t insn) if (mask & 1) { /* V */ tcg_gen_shli_i32(cpu_VF, nzcv, 31 - 0); } + return true; } /* @@ -XXX,XX +XXX,XX @@ static void disas_data_proc_reg(DisasContext *s, uint32_t insn) switch (op2) { case 0x0: switch (op3) { - case 0x01: /* Rotate right into flags */ - case 0x21: - disas_rotate_right_into_flags(s, insn); - break; - case 0x02: /* Evaluate into flags */ case 0x12: case 0x22: @@ -XXX,XX +XXX,XX @@ static void disas_data_proc_reg(DisasContext *s, uint32_t insn) default: case 0x00: /* Add/subtract (with carry) */ + case 0x01: /* Rotate right into flags */ + case 0x21: goto do_unallocated; } break; -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-18-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 4 +++ target/arm/tcg/translate-a64.c | 48 +++++----------------------------- 2 files changed, 11 insertions(+), 41 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ SBCS . 11 11010000 ..... 000000 ..... ..... @rrr_sf RMIF 1 01 11010000 imm:6 00001 rn:5 0 mask:4 # Evaluate into flags + +SETF8 0 01 11010000 00000 000010 rn:5 01101 +SETF16 0 01 11010000 00000 010010 rn:5 01101 + # Conditional compare (regster) # Conditional compare (immediate) # Conditional select diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static bool trans_RMIF(DisasContext *s, arg_RMIF *a) return true; } -/* - * Evaluate into flags - * 31 30 29 21 15 14 10 5 4 0 - * +--+--+--+-----------------+---------+----+---------+------+--+------+ - * |sf|op| S| 1 1 0 1 0 0 0 0 | opcode2 | sz | 0 0 1 0 | Rn |o3| mask | - * +--+--+--+-----------------+---------+----+---------+------+--+------+ - */ -static void disas_evaluate_into_flags(DisasContext *s, uint32_t insn) +static bool do_setf(DisasContext *s, int rn, int shift) { - int o3_mask = extract32(insn, 0, 5); - int rn = extract32(insn, 5, 5); - int o2 = extract32(insn, 15, 6); - int sz = extract32(insn, 14, 1); - int sf_op_s = extract32(insn, 29, 3); - TCGv_i32 tmp; - int shift; + TCGv_i32 tmp = tcg_temp_new_i32(); - if (sf_op_s != 1 || o2 != 0 || o3_mask != 0xd || - !dc_isar_feature(aa64_condm_4, s)) { - unallocated_encoding(s); - return; - } - shift = sz ? 16 : 24; /* SETF16 or SETF8 */ - - tmp = tcg_temp_new_i32(); tcg_gen_extrl_i64_i32(tmp, cpu_reg(s, rn)); tcg_gen_shli_i32(cpu_NF, tmp, shift); tcg_gen_shli_i32(cpu_VF, tmp, shift - 1); tcg_gen_mov_i32(cpu_ZF, cpu_NF); tcg_gen_xor_i32(cpu_VF, cpu_VF, cpu_NF); + return true; } +TRANS_FEAT(SETF8, aa64_condm_4, do_setf, a->rn, 24) +TRANS_FEAT(SETF16, aa64_condm_4, do_setf, a->rn, 16) + /* Conditional compare (immediate / register) * 31 30 29 28 27 26 25 24 23 22 21 20 16 15 12 11 10 9 5 4 3 0 * +--+--+--+------------------------+--------+------+----+--+------+--+-----+ @@ -XXX,XX +XXX,XX @@ static void disas_data_proc_reg(DisasContext *s, uint32_t insn) { int op1 = extract32(insn, 28, 1); int op2 = extract32(insn, 21, 4); - int op3 = extract32(insn, 10, 6); if (!op1) { goto do_unallocated; } switch (op2) { - case 0x0: - switch (op3) { - case 0x02: /* Evaluate into flags */ - case 0x12: - case 0x22: - case 0x32: - disas_evaluate_into_flags(s, insn); - break; - - default: - case 0x00: /* Add/subtract (with carry) */ - case 0x01: /* Rotate right into flags */ - case 0x21: - goto do_unallocated; - } - break; - case 0x2: /* Conditional compare */ disas_cc(s, insn); /* both imm and reg forms */ break; @@ -XXX,XX +XXX,XX @@ static void disas_data_proc_reg(DisasContext *s, uint32_t insn) default: do_unallocated: + case 0x0: case 0x6: /* Data-processing */ case 0x8 ... 0xf: /* (3 source) */ unallocated_encoding(s); -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-19-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 6 ++-- target/arm/tcg/translate-a64.c | 66 +++++++++++----------------------- 2 files changed, 25 insertions(+), 47 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ RMIF 1 01 11010000 imm:6 00001 rn:5 0 mask:4 SETF8 0 01 11010000 00000 000010 rn:5 01101 SETF16 0 01 11010000 00000 010010 rn:5 01101 -# Conditional compare (regster) -# Conditional compare (immediate) +# Conditional compare + +CCMP sf:1 op:1 1 11010010 y:5 cond:4 imm:1 0 rn:5 0 nzcv:4 + # Conditional select # Data Processing (3-source) diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static bool do_setf(DisasContext *s, int rn, int shift) TRANS_FEAT(SETF8, aa64_condm_4, do_setf, a->rn, 24) TRANS_FEAT(SETF16, aa64_condm_4, do_setf, a->rn, 16) -/* Conditional compare (immediate / register) - * 31 30 29 28 27 26 25 24 23 22 21 20 16 15 12 11 10 9 5 4 3 0 - * +--+--+--+------------------------+--------+------+----+--+------+--+-----+ - * |sf|op| S| 1 1 0 1 0 0 1 0 |imm5/rm | cond |i/r |o2| Rn |o3|nzcv | - * +--+--+--+------------------------+--------+------+----+--+------+--+-----+ - * [1] y [0] [0] - */ -static void disas_cc(DisasContext *s, uint32_t insn) +/* CCMP, CCMN */ +static bool trans_CCMP(DisasContext *s, arg_CCMP *a) { - unsigned int sf, op, y, cond, rn, nzcv, is_imm; - TCGv_i32 tcg_t0, tcg_t1, tcg_t2; - TCGv_i64 tcg_tmp, tcg_y, tcg_rn; + TCGv_i32 tcg_t0 = tcg_temp_new_i32(); + TCGv_i32 tcg_t1 = tcg_temp_new_i32(); + TCGv_i32 tcg_t2 = tcg_temp_new_i32(); + TCGv_i64 tcg_tmp = tcg_temp_new_i64(); + TCGv_i64 tcg_rn, tcg_y; DisasCompare c; - - if (!extract32(insn, 29, 1)) { - unallocated_encoding(s); - return; - } - if (insn & (1 << 10 | 1 << 4)) { - unallocated_encoding(s); - return; - } - sf = extract32(insn, 31, 1); - op = extract32(insn, 30, 1); - is_imm = extract32(insn, 11, 1); - y = extract32(insn, 16, 5); /* y = rm (reg) or imm5 (imm) */ - cond = extract32(insn, 12, 4); - rn = extract32(insn, 5, 5); - nzcv = extract32(insn, 0, 4); + unsigned nzcv; /* Set T0 = !COND. */ - tcg_t0 = tcg_temp_new_i32(); - arm_test_cc(&c, cond); + arm_test_cc(&c, a->cond); tcg_gen_setcondi_i32(tcg_invert_cond(c.cond), tcg_t0, c.value, 0); /* Load the arguments for the new comparison. */ - if (is_imm) { - tcg_y = tcg_temp_new_i64(); - tcg_gen_movi_i64(tcg_y, y); + if (a->imm) { + tcg_y = tcg_constant_i64(a->y); } else { - tcg_y = cpu_reg(s, y); + tcg_y = cpu_reg(s, a->y); } - tcg_rn = cpu_reg(s, rn); + tcg_rn = cpu_reg(s, a->rn); /* Set the flags for the new comparison. */ - tcg_tmp = tcg_temp_new_i64(); - if (op) { - gen_sub_CC(sf, tcg_tmp, tcg_rn, tcg_y); + if (a->op) { + gen_sub_CC(a->sf, tcg_tmp, tcg_rn, tcg_y); } else { - gen_add_CC(sf, tcg_tmp, tcg_rn, tcg_y); + gen_add_CC(a->sf, tcg_tmp, tcg_rn, tcg_y); } - /* If COND was false, force the flags to #nzcv. Compute two masks + /* + * If COND was false, force the flags to #nzcv. Compute two masks * to help with this: T1 = (COND ? 0 : -1), T2 = (COND ? -1 : 0). * For tcg hosts that support ANDC, we can make do with just T1. * In either case, allow the tcg optimizer to delete any unused mask. */ - tcg_t1 = tcg_temp_new_i32(); - tcg_t2 = tcg_temp_new_i32(); tcg_gen_neg_i32(tcg_t1, tcg_t0); tcg_gen_subi_i32(tcg_t2, tcg_t0, 1); + nzcv = a->nzcv; if (nzcv & 8) { /* N */ tcg_gen_or_i32(cpu_NF, cpu_NF, tcg_t1); } else { @@ -XXX,XX +XXX,XX @@ static void disas_cc(DisasContext *s, uint32_t insn) tcg_gen_and_i32(cpu_VF, cpu_VF, tcg_t2); } } + return true; } /* Conditional select @@ -XXX,XX +XXX,XX @@ static void disas_data_proc_reg(DisasContext *s, uint32_t insn) } switch (op2) { - case 0x2: /* Conditional compare */ - disas_cc(s, insn); /* both imm and reg forms */ - break; - case 0x4: /* Conditional select */ disas_cond_select(s, insn); break; @@ -XXX,XX +XXX,XX @@ static void disas_data_proc_reg(DisasContext *s, uint32_t insn) default: do_unallocated: case 0x0: + case 0x2: /* Conditional compare */ case 0x6: /* Data-processing */ case 0x8 ... 0xf: /* (3 source) */ unallocated_encoding(s); -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> This includes CSEL, CSINC, CSINV, CSNEG. Remove disas_data_proc_reg, as these were the last insns decoded by that function. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-20-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 3 ++ target/arm/tcg/translate-a64.c | 84 ++++++---------------------------- 2 files changed, 17 insertions(+), 70 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ SETF16 0 01 11010000 00000 010010 rn:5 01101 CCMP sf:1 op:1 1 11010010 y:5 cond:4 imm:1 0 rn:5 0 nzcv:4 # Conditional select + +CSEL sf:1 else_inv:1 011010100 rm:5 cond:4 0 else_inc:1 rn:5 rd:5 + # Data Processing (3-source) &rrrr rd rn rm ra diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static bool trans_CCMP(DisasContext *s, arg_CCMP *a) return true; } -/* Conditional select - * 31 30 29 28 21 20 16 15 12 11 10 9 5 4 0 - * +----+----+---+-----------------+------+------+-----+------+------+ - * | sf | op | S | 1 1 0 1 0 1 0 0 | Rm | cond | op2 | Rn | Rd | - * +----+----+---+-----------------+------+------+-----+------+------+ - */ -static void disas_cond_select(DisasContext *s, uint32_t insn) +static bool trans_CSEL(DisasContext *s, arg_CSEL *a) { - unsigned int sf, else_inv, rm, cond, else_inc, rn, rd; - TCGv_i64 tcg_rd, zero; + TCGv_i64 tcg_rd = cpu_reg(s, a->rd); + TCGv_i64 zero = tcg_constant_i64(0); DisasCompare64 c; - if (extract32(insn, 29, 1) || extract32(insn, 11, 1)) { - /* S == 1 or op2<1> == 1 */ - unallocated_encoding(s); - return; - } - sf = extract32(insn, 31, 1); - else_inv = extract32(insn, 30, 1); - rm = extract32(insn, 16, 5); - cond = extract32(insn, 12, 4); - else_inc = extract32(insn, 10, 1); - rn = extract32(insn, 5, 5); - rd = extract32(insn, 0, 5); + a64_test_cc(&c, a->cond); - tcg_rd = cpu_reg(s, rd); - - a64_test_cc(&c, cond); - zero = tcg_constant_i64(0); - - if (rn == 31 && rm == 31 && (else_inc ^ else_inv)) { + if (a->rn == 31 && a->rm == 31 && (a->else_inc ^ a->else_inv)) { /* CSET & CSETM. */ - if (else_inv) { + if (a->else_inv) { tcg_gen_negsetcond_i64(tcg_invert_cond(c.cond), tcg_rd, c.value, zero); } else { @@ -XXX,XX +XXX,XX @@ static void disas_cond_select(DisasContext *s, uint32_t insn) tcg_rd, c.value, zero); } } else { - TCGv_i64 t_true = cpu_reg(s, rn); - TCGv_i64 t_false = read_cpu_reg(s, rm, 1); - if (else_inv && else_inc) { + TCGv_i64 t_true = cpu_reg(s, a->rn); + TCGv_i64 t_false = read_cpu_reg(s, a->rm, 1); + + if (a->else_inv && a->else_inc) { tcg_gen_neg_i64(t_false, t_false); - } else if (else_inv) { + } else if (a->else_inv) { tcg_gen_not_i64(t_false, t_false); - } else if (else_inc) { + } else if (a->else_inc) { tcg_gen_addi_i64(t_false, t_false, 1); } tcg_gen_movcond_i64(c.cond, tcg_rd, c.value, zero, t_true, t_false); } - if (!sf) { + if (!a->sf) { tcg_gen_ext32u_i64(tcg_rd, tcg_rd); } -} - -/* - * Data processing - register - * 31 30 29 28 25 21 20 16 10 0 - * +--+---+--+---+-------+-----+-------+-------+---------+ - * | |op0| |op1| 1 0 1 | op2 | | op3 | | - * +--+---+--+---+-------+-----+-------+-------+---------+ - */ -static void disas_data_proc_reg(DisasContext *s, uint32_t insn) -{ - int op1 = extract32(insn, 28, 1); - int op2 = extract32(insn, 21, 4); - - if (!op1) { - goto do_unallocated; - } - - switch (op2) { - case 0x4: /* Conditional select */ - disas_cond_select(s, insn); - break; - - default: - do_unallocated: - case 0x0: - case 0x2: /* Conditional compare */ - case 0x6: /* Data-processing */ - case 0x8 ... 0xf: /* (3 source) */ - unallocated_encoding(s); - break; - } + return true; } static void handle_fp_compare(DisasContext *s, int size, @@ -XXX,XX +XXX,XX @@ static bool btype_destination_ok(uint32_t insn, bool bt, int btype) static void disas_a64_legacy(DisasContext *s, uint32_t insn) { switch (extract32(insn, 25, 4)) { - case 0x5: - case 0xd: /* Data processing - register */ - disas_data_proc_reg(s, insn); - break; case 0x7: case 0xf: /* Data processing - SIMD and floating point */ disas_data_proc_simd_fp(s, insn); -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Provide a simple way to check for float64, float32, and float16 support, as well as the fpu enabled. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-21-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/translate-a64.c | 62 ++++++++++++++++++---------------- 1 file changed, 32 insertions(+), 30 deletions(-) diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static bool fp_access_check(DisasContext *s) return true; } +/* + * Return <0 for non-supported element sizes, with MO_16 controlled by + * FEAT_FP16; return 0 for fp disabled; otherwise return >0 for success. + */ +static int fp_access_check_scalar_hsd(DisasContext *s, MemOp esz) +{ + switch (esz) { + case MO_64: + case MO_32: + break; + case MO_16: + if (!dc_isar_feature(aa64_fp16, s)) { + return -1; + } + break; + default: + return -1; + } + return fp_access_check(s); +} + /* * Check that SVE access is enabled. If it is, return true. * If not, emit code to generate an appropriate exception and return false. @@ -XXX,XX +XXX,XX @@ static bool trans_FCSEL(DisasContext *s, arg_FCSEL *a) { TCGv_i64 t_true, t_false; DisasCompare64 c; + int check = fp_access_check_scalar_hsd(s, a->esz); - switch (a->esz) { - case MO_32: - case MO_64: - break; - case MO_16: - if (!dc_isar_feature(aa64_fp16, s)) { - return false; - } - break; - default: - return false; - } - - if (!fp_access_check(s)) { - return true; + if (check <= 0) { + return check == 0; } /* Zero extend sreg & hreg inputs to 64 bits now. */ @@ -XXX,XX +XXX,XX @@ TRANS(FMINV_s, do_fp_reduction, a, gen_helper_vfp_mins) static bool trans_FMOVI_s(DisasContext *s, arg_FMOVI_s *a) { - switch (a->esz) { - case MO_32: - case MO_64: - break; - case MO_16: - if (!dc_isar_feature(aa64_fp16, s)) { - return false; - } - break; - default: - return false; - } - if (fp_access_check(s)) { - uint64_t imm = vfp_expand_imm(a->esz, a->imm); - write_fp_dreg(s, a->rd, tcg_constant_i64(imm)); + int check = fp_access_check_scalar_hsd(s, a->esz); + uint64_t imm; + + if (check <= 0) { + return check == 0; } + + imm = vfp_expand_imm(a->esz, a->imm); + write_fp_dreg(s, a->rd, tcg_constant_i64(imm)); return true; } -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Provide a simple way to check for float64, float32, and float16 support vs vector width, as well as the fpu enabled. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-22-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/translate-a64.c | 135 +++++++++++++-------------------- 1 file changed, 54 insertions(+), 81 deletions(-) diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static int fp_access_check_scalar_hsd(DisasContext *s, MemOp esz) return fp_access_check(s); } +/* Likewise, but vector MO_64 must have two elements. */ +static int fp_access_check_vector_hsd(DisasContext *s, bool is_q, MemOp esz) +{ + switch (esz) { + case MO_64: + if (!is_q) { + return -1; + } + break; + case MO_32: + break; + case MO_16: + if (!dc_isar_feature(aa64_fp16, s)) { + return -1; + } + break; + default: + return -1; + } + return fp_access_check(s); +} + /* * Check that SVE access is enabled. If it is, return true. * If not, emit code to generate an appropriate exception and return false. @@ -XXX,XX +XXX,XX @@ static bool do_fp3_vector(DisasContext *s, arg_qrrr_e *a, int data, gen_helper_gvec_3_ptr * const fns[3]) { MemOp esz = a->esz; + int check = fp_access_check_vector_hsd(s, a->q, esz); - switch (esz) { - case MO_64: - if (!a->q) { - return false; - } - break; - case MO_32: - break; - case MO_16: - if (!dc_isar_feature(aa64_fp16, s)) { - return false; - } - break; - default: - return false; - } - if (fp_access_check(s)) { - gen_gvec_op3_fpst(s, a->q, a->rd, a->rn, a->rm, - esz == MO_16, data, fns[esz - 1]); + if (check <= 0) { + return check == 0; } + + gen_gvec_op3_fpst(s, a->q, a->rd, a->rn, a->rm, + esz == MO_16, data, fns[esz - 1]); return true; } @@ -XXX,XX +XXX,XX @@ TRANS_FEAT(FCADD_270, aa64_fcma, do_fp3_vector, a, 1, f_vector_fcadd) static bool trans_FCMLA_v(DisasContext *s, arg_FCMLA_v *a) { - gen_helper_gvec_4_ptr *fn; + static gen_helper_gvec_4_ptr * const fn[] = { + [MO_16] = gen_helper_gvec_fcmlah, + [MO_32] = gen_helper_gvec_fcmlas, + [MO_64] = gen_helper_gvec_fcmlad, + }; + int check; if (!dc_isar_feature(aa64_fcma, s)) { return false; } - switch (a->esz) { - case MO_64: - if (!a->q) { - return false; - } - fn = gen_helper_gvec_fcmlad; - break; - case MO_32: - fn = gen_helper_gvec_fcmlas; - break; - case MO_16: - if (!dc_isar_feature(aa64_fp16, s)) { - return false; - } - fn = gen_helper_gvec_fcmlah; - break; - default: - return false; - } - if (fp_access_check(s)) { - gen_gvec_op4_fpst(s, a->q, a->rd, a->rn, a->rm, a->rd, - a->esz == MO_16, a->rot, fn); + + check = fp_access_check_vector_hsd(s, a->q, a->esz); + if (check <= 0) { + return check == 0; } + + gen_gvec_op4_fpst(s, a->q, a->rd, a->rn, a->rm, a->rd, + a->esz == MO_16, a->rot, fn[a->esz]); return true; } @@ -XXX,XX +XXX,XX @@ static bool do_fp3_vector_idx(DisasContext *s, arg_qrrx_e *a, gen_helper_gvec_3_ptr * const fns[3]) { MemOp esz = a->esz; + int check = fp_access_check_vector_hsd(s, a->q, esz); - switch (esz) { - case MO_64: - if (!a->q) { - return false; - } - break; - case MO_32: - break; - case MO_16: - if (!dc_isar_feature(aa64_fp16, s)) { - return false; - } - break; - default: - g_assert_not_reached(); - } - if (fp_access_check(s)) { - gen_gvec_op3_fpst(s, a->q, a->rd, a->rn, a->rm, - esz == MO_16, a->idx, fns[esz - 1]); + if (check <= 0) { + return check == 0; } + + gen_gvec_op3_fpst(s, a->q, a->rd, a->rn, a->rm, + esz == MO_16, a->idx, fns[esz - 1]); return true; } @@ -XXX,XX +XXX,XX @@ static bool do_fmla_vector_idx(DisasContext *s, arg_qrrx_e *a, bool neg) gen_helper_gvec_fmla_idx_d, }; MemOp esz = a->esz; + int check = fp_access_check_vector_hsd(s, a->q, esz); - switch (esz) { - case MO_64: - if (!a->q) { - return false; - } - break; - case MO_32: - break; - case MO_16: - if (!dc_isar_feature(aa64_fp16, s)) { - return false; - } - break; - default: - g_assert_not_reached(); - } - if (fp_access_check(s)) { - gen_gvec_op4_fpst(s, a->q, a->rd, a->rn, a->rm, a->rd, - esz == MO_16, (a->idx << 1) | neg, - fns[esz - 1]); + if (check <= 0) { + return check == 0; } + + gen_gvec_op4_fpst(s, a->q, a->rd, a->rn, a->rm, a->rd, + esz == MO_16, (a->idx << 1) | neg, + fns[esz - 1]); return true; } -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-23-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 8 + target/arm/tcg/translate-a64.c | 283 ++++++++++++--------------------- 2 files changed, 112 insertions(+), 179 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ FMINV_s 0110 1110 10 11000 01111 10 ..... ..... @rr_q1e2 FMOVI_s 0001 1110 .. 1 imm:8 100 00000 rd:5 esz=%esz_hsd +# Floating-point Compare + +FCMP 00011110 .. 1 rm:5 001000 rn:5 e:1 z:1 000 esz=%esz_hsd + +# Floating-point Conditional Compare + +FCCMP 00011110 .. 1 rm:5 cond:4 01 rn:5 e:1 nzcv:4 esz=%esz_hsd + # Advanced SIMD Modified Immediate / Shift by Immediate %abcdefgh 16:3 5:5 diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static bool trans_FMOVI_s(DisasContext *s, arg_FMOVI_s *a) return true; } +/* + * Floating point compare, conditional compare + */ + +static void handle_fp_compare(DisasContext *s, int size, + unsigned int rn, unsigned int rm, + bool cmp_with_zero, bool signal_all_nans) +{ + TCGv_i64 tcg_flags = tcg_temp_new_i64(); + TCGv_ptr fpst = fpstatus_ptr(size == MO_16 ? FPST_FPCR_F16 : FPST_FPCR); + + if (size == MO_64) { + TCGv_i64 tcg_vn, tcg_vm; + + tcg_vn = read_fp_dreg(s, rn); + if (cmp_with_zero) { + tcg_vm = tcg_constant_i64(0); + } else { + tcg_vm = read_fp_dreg(s, rm); + } + if (signal_all_nans) { + gen_helper_vfp_cmped_a64(tcg_flags, tcg_vn, tcg_vm, fpst); + } else { + gen_helper_vfp_cmpd_a64(tcg_flags, tcg_vn, tcg_vm, fpst); + } + } else { + TCGv_i32 tcg_vn = tcg_temp_new_i32(); + TCGv_i32 tcg_vm = tcg_temp_new_i32(); + + read_vec_element_i32(s, tcg_vn, rn, 0, size); + if (cmp_with_zero) { + tcg_gen_movi_i32(tcg_vm, 0); + } else { + read_vec_element_i32(s, tcg_vm, rm, 0, size); + } + + switch (size) { + case MO_32: + if (signal_all_nans) { + gen_helper_vfp_cmpes_a64(tcg_flags, tcg_vn, tcg_vm, fpst); + } else { + gen_helper_vfp_cmps_a64(tcg_flags, tcg_vn, tcg_vm, fpst); + } + break; + case MO_16: + if (signal_all_nans) { + gen_helper_vfp_cmpeh_a64(tcg_flags, tcg_vn, tcg_vm, fpst); + } else { + gen_helper_vfp_cmph_a64(tcg_flags, tcg_vn, tcg_vm, fpst); + } + break; + default: + g_assert_not_reached(); + } + } + + gen_set_nzcv(tcg_flags); +} + +/* FCMP, FCMPE */ +static bool trans_FCMP(DisasContext *s, arg_FCMP *a) +{ + int check = fp_access_check_scalar_hsd(s, a->esz); + + if (check <= 0) { + return check == 0; + } + + handle_fp_compare(s, a->esz, a->rn, a->rm, a->z, a->e); + return true; +} + +/* FCCMP, FCCMPE */ +static bool trans_FCCMP(DisasContext *s, arg_FCCMP *a) +{ + TCGLabel *label_continue = NULL; + int check = fp_access_check_scalar_hsd(s, a->esz); + + if (check <= 0) { + return check == 0; + } + + if (a->cond < 0x0e) { /* not always */ + TCGLabel *label_match = gen_new_label(); + label_continue = gen_new_label(); + arm_gen_test_cc(a->cond, label_match); + /* nomatch: */ + gen_set_nzcv(tcg_constant_i64(a->nzcv << 28)); + tcg_gen_br(label_continue); + gen_set_label(label_match); + } + + handle_fp_compare(s, a->esz, a->rn, a->rm, false, a->e); + + if (label_continue) { + gen_set_label(label_continue); + } + return true; +} + /* * Advanced SIMD Modified Immediate */ @@ -XXX,XX +XXX,XX @@ static bool trans_CSEL(DisasContext *s, arg_CSEL *a) return true; } -static void handle_fp_compare(DisasContext *s, int size, - unsigned int rn, unsigned int rm, - bool cmp_with_zero, bool signal_all_nans) -{ - TCGv_i64 tcg_flags = tcg_temp_new_i64(); - TCGv_ptr fpst = fpstatus_ptr(size == MO_16 ? FPST_FPCR_F16 : FPST_FPCR); - - if (size == MO_64) { - TCGv_i64 tcg_vn, tcg_vm; - - tcg_vn = read_fp_dreg(s, rn); - if (cmp_with_zero) { - tcg_vm = tcg_constant_i64(0); - } else { - tcg_vm = read_fp_dreg(s, rm); - } - if (signal_all_nans) { - gen_helper_vfp_cmped_a64(tcg_flags, tcg_vn, tcg_vm, fpst); - } else { - gen_helper_vfp_cmpd_a64(tcg_flags, tcg_vn, tcg_vm, fpst); - } - } else { - TCGv_i32 tcg_vn = tcg_temp_new_i32(); - TCGv_i32 tcg_vm = tcg_temp_new_i32(); - - read_vec_element_i32(s, tcg_vn, rn, 0, size); - if (cmp_with_zero) { - tcg_gen_movi_i32(tcg_vm, 0); - } else { - read_vec_element_i32(s, tcg_vm, rm, 0, size); - } - - switch (size) { - case MO_32: - if (signal_all_nans) { - gen_helper_vfp_cmpes_a64(tcg_flags, tcg_vn, tcg_vm, fpst); - } else { - gen_helper_vfp_cmps_a64(tcg_flags, tcg_vn, tcg_vm, fpst); - } - break; - case MO_16: - if (signal_all_nans) { - gen_helper_vfp_cmpeh_a64(tcg_flags, tcg_vn, tcg_vm, fpst); - } else { - gen_helper_vfp_cmph_a64(tcg_flags, tcg_vn, tcg_vm, fpst); - } - break; - default: - g_assert_not_reached(); - } - } - - gen_set_nzcv(tcg_flags); -} - -/* Floating point compare - * 31 30 29 28 24 23 22 21 20 16 15 14 13 10 9 5 4 0 - * +---+---+---+-----------+------+---+------+-----+---------+------+-------+ - * | M | 0 | S | 1 1 1 1 0 | type | 1 | Rm | op | 1 0 0 0 | Rn | op2 | - * +---+---+---+-----------+------+---+------+-----+---------+------+-------+ - */ -static void disas_fp_compare(DisasContext *s, uint32_t insn) -{ - unsigned int mos, type, rm, op, rn, opc, op2r; - int size; - - mos = extract32(insn, 29, 3); - type = extract32(insn, 22, 2); - rm = extract32(insn, 16, 5); - op = extract32(insn, 14, 2); - rn = extract32(insn, 5, 5); - opc = extract32(insn, 3, 2); - op2r = extract32(insn, 0, 3); - - if (mos || op || op2r) { - unallocated_encoding(s); - return; - } - - switch (type) { - case 0: - size = MO_32; - break; - case 1: - size = MO_64; - break; - case 3: - size = MO_16; - if (dc_isar_feature(aa64_fp16, s)) { - break; - } - /* fallthru */ - default: - unallocated_encoding(s); - return; - } - - if (!fp_access_check(s)) { - return; - } - - handle_fp_compare(s, size, rn, rm, opc & 1, opc & 2); -} - -/* Floating point conditional compare - * 31 30 29 28 24 23 22 21 20 16 15 12 11 10 9 5 4 3 0 - * +---+---+---+-----------+------+---+------+------+-----+------+----+------+ - * | M | 0 | S | 1 1 1 1 0 | type | 1 | Rm | cond | 0 1 | Rn | op | nzcv | - * +---+---+---+-----------+------+---+------+------+-----+------+----+------+ - */ -static void disas_fp_ccomp(DisasContext *s, uint32_t insn) -{ - unsigned int mos, type, rm, cond, rn, op, nzcv; - TCGLabel *label_continue = NULL; - int size; - - mos = extract32(insn, 29, 3); - type = extract32(insn, 22, 2); - rm = extract32(insn, 16, 5); - cond = extract32(insn, 12, 4); - rn = extract32(insn, 5, 5); - op = extract32(insn, 4, 1); - nzcv = extract32(insn, 0, 4); - - if (mos) { - unallocated_encoding(s); - return; - } - - switch (type) { - case 0: - size = MO_32; - break; - case 1: - size = MO_64; - break; - case 3: - size = MO_16; - if (dc_isar_feature(aa64_fp16, s)) { - break; - } - /* fallthru */ - default: - unallocated_encoding(s); - return; - } - - if (!fp_access_check(s)) { - return; - } - - if (cond < 0x0e) { /* not always */ - TCGLabel *label_match = gen_new_label(); - label_continue = gen_new_label(); - arm_gen_test_cc(cond, label_match); - /* nomatch: */ - gen_set_nzcv(tcg_constant_i64(nzcv << 28)); - tcg_gen_br(label_continue); - gen_set_label(label_match); - } - - handle_fp_compare(s, size, rn, rm, false, op); - - if (cond < 0x0e) { - gen_set_label(label_continue); - } -} - /* Floating-point data-processing (1 source) - half precision */ static void handle_fp_1src_half(DisasContext *s, int opcode, int rd, int rn) { @@ -XXX,XX +XXX,XX @@ static void disas_data_proc_fp(DisasContext *s, uint32_t insn) disas_fp_fixed_conv(s, insn); } else { switch (extract32(insn, 10, 2)) { - case 1: - /* Floating point conditional compare */ - disas_fp_ccomp(s, insn); - break; - case 2: - /* Floating point data-processing (2 source) */ - unallocated_encoding(s); /* in decodetree */ - break; - case 3: - /* Floating point conditional select */ + case 1: /* Floating point conditional compare */ + case 2: /* Floating point data-processing (2 source) */ + case 3: /* Floating point conditional select */ unallocated_encoding(s); /* in decodetree */ break; case 0: @@ -XXX,XX +XXX,XX @@ static void disas_data_proc_fp(DisasContext *s, uint32_t insn) break; case 1: /* [15:12] == xx10 */ /* Floating point compare */ - disas_fp_compare(s, insn); + unallocated_encoding(s); /* in decodetree */ break; case 2: /* [15:12] == x100 */ /* Floating point data-processing (1 source) */ -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> These opcodes are only supported as vector operations, not as advsimd scalar. Set only_in_vector, and remove the unreachable implementation of scalar fneg. Reported-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Message-id: 20241211163036.2297116-24-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/translate-a64.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc_fp16(DisasContext *s, uint32_t insn) break; case 0x2f: /* FABS */ case 0x6f: /* FNEG */ + only_in_vector = true; need_fpst = false; break; case 0x7d: /* FRSQRTE */ + break; case 0x7f: /* FSQRT (vector) */ + only_in_vector = true; break; default: unallocated_encoding(s); @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc_fp16(DisasContext *s, uint32_t insn) case 0x7b: /* FCVTZU */ gen_helper_advsimd_f16touinth(tcg_res, tcg_op, tcg_fpstatus); break; - case 0x6f: /* FNEG */ - tcg_gen_xori_i32(tcg_res, tcg_op, 0x8000); - break; case 0x7d: /* FRSQRTE */ gen_helper_rsqrte_f16(tcg_res, tcg_op, tcg_fpstatus); break; -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-25-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 7 +++ target/arm/tcg/translate-a64.c | 105 +++++++++++++++++++++++---------- 2 files changed, 81 insertions(+), 31 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ @rr_h ........ ... ..... ...... rn:5 rd:5 &rr_e esz=1 @rr_d ........ ... ..... ...... rn:5 rd:5 &rr_e esz=3 @rr_sd ........ ... ..... ...... rn:5 rd:5 &rr_e esz=%esz_sd +@rr_hsd ........ ... ..... ...... rn:5 rd:5 &rr_e esz=%esz_hsd @rrr_b ........ ... rm:5 ...... rn:5 rd:5 &rrr_e esz=0 @rrr_h ........ ... rm:5 ...... rn:5 rd:5 &rrr_e esz=1 @@ -XXX,XX +XXX,XX @@ FMAXV_s 0110 1110 00 11000 01111 10 ..... ..... @rr_q1e2 FMINV_h 0.00 1110 10 11000 01111 10 ..... ..... @qrr_h FMINV_s 0110 1110 10 11000 01111 10 ..... ..... @rr_q1e2 +# Floating-point data processing (1 source) + +FMOV_s 00011110 .. 1 000000 10000 ..... ..... @rr_hsd +FABS_s 00011110 .. 1 000001 10000 ..... ..... @rr_hsd +FNEG_s 00011110 .. 1 000010 10000 ..... ..... @rr_hsd + # Floating-point Immediate FMOVI_s 0001 1110 .. 1 imm:8 100 00000 rd:5 esz=%esz_hsd diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static bool trans_CSEL(DisasContext *s, arg_CSEL *a) return true; } +typedef struct FPScalar1Int { + void (*gen_h)(TCGv_i32, TCGv_i32); + void (*gen_s)(TCGv_i32, TCGv_i32); + void (*gen_d)(TCGv_i64, TCGv_i64); +} FPScalar1Int; + +static bool do_fp1_scalar_int(DisasContext *s, arg_rr_e *a, + const FPScalar1Int *f) +{ + switch (a->esz) { + case MO_64: + if (fp_access_check(s)) { + TCGv_i64 t = read_fp_dreg(s, a->rn); + f->gen_d(t, t); + write_fp_dreg(s, a->rd, t); + } + break; + case MO_32: + if (fp_access_check(s)) { + TCGv_i32 t = read_fp_sreg(s, a->rn); + f->gen_s(t, t); + write_fp_sreg(s, a->rd, t); + } + break; + case MO_16: + if (!dc_isar_feature(aa64_fp16, s)) { + return false; + } + if (fp_access_check(s)) { + TCGv_i32 t = read_fp_hreg(s, a->rn); + f->gen_h(t, t); + write_fp_sreg(s, a->rd, t); + } + break; + default: + return false; + } + return true; +} + +static const FPScalar1Int f_scalar_fmov = { + tcg_gen_mov_i32, + tcg_gen_mov_i32, + tcg_gen_mov_i64, +}; +TRANS(FMOV_s, do_fp1_scalar_int, a, &f_scalar_fmov) + +static const FPScalar1Int f_scalar_fabs = { + gen_vfp_absh, + gen_vfp_abss, + gen_vfp_absd, +}; +TRANS(FABS_s, do_fp1_scalar_int, a, &f_scalar_fabs) + +static const FPScalar1Int f_scalar_fneg = { + gen_vfp_negh, + gen_vfp_negs, + gen_vfp_negd, +}; +TRANS(FNEG_s, do_fp1_scalar_int, a, &f_scalar_fneg) + /* Floating-point data-processing (1 source) - half precision */ static void handle_fp_1src_half(DisasContext *s, int opcode, int rd, int rn) { @@ -XXX,XX +XXX,XX @@ static void handle_fp_1src_half(DisasContext *s, int opcode, int rd, int rn) TCGv_i32 tcg_res = tcg_temp_new_i32(); switch (opcode) { - case 0x0: /* FMOV */ - tcg_gen_mov_i32(tcg_res, tcg_op); - break; - case 0x1: /* FABS */ - gen_vfp_absh(tcg_res, tcg_op); - break; - case 0x2: /* FNEG */ - gen_vfp_negh(tcg_res, tcg_op); - break; case 0x3: /* FSQRT */ fpst = fpstatus_ptr(FPST_FPCR_F16); gen_helper_sqrt_f16(tcg_res, tcg_op, fpst); @@ -XXX,XX +XXX,XX @@ static void handle_fp_1src_half(DisasContext *s, int opcode, int rd, int rn) gen_helper_advsimd_rinth(tcg_res, tcg_op, fpst); break; default: + case 0x0: /* FMOV */ + case 0x1: /* FABS */ + case 0x2: /* FNEG */ g_assert_not_reached(); } @@ -XXX,XX +XXX,XX @@ static void handle_fp_1src_single(DisasContext *s, int opcode, int rd, int rn) tcg_res = tcg_temp_new_i32(); switch (opcode) { - case 0x0: /* FMOV */ - tcg_gen_mov_i32(tcg_res, tcg_op); - goto done; - case 0x1: /* FABS */ - gen_vfp_abss(tcg_res, tcg_op); - goto done; - case 0x2: /* FNEG */ - gen_vfp_negs(tcg_res, tcg_op); - goto done; case 0x3: /* FSQRT */ gen_helper_vfp_sqrts(tcg_res, tcg_op, tcg_env); goto done; @@ -XXX,XX +XXX,XX @@ static void handle_fp_1src_single(DisasContext *s, int opcode, int rd, int rn) gen_fpst = gen_helper_frint64_s; break; default: + case 0x0: /* FMOV */ + case 0x1: /* FABS */ + case 0x2: /* FNEG */ g_assert_not_reached(); } @@ -XXX,XX +XXX,XX @@ static void handle_fp_1src_double(DisasContext *s, int opcode, int rd, int rn) TCGv_ptr fpst; int rmode = -1; - switch (opcode) { - case 0x0: /* FMOV */ - gen_gvec_fn2(s, false, rd, rn, tcg_gen_gvec_mov, 0); - return; - } - tcg_op = read_fp_dreg(s, rn); tcg_res = tcg_temp_new_i64(); switch (opcode) { - case 0x1: /* FABS */ - gen_vfp_absd(tcg_res, tcg_op); - goto done; - case 0x2: /* FNEG */ - gen_vfp_negd(tcg_res, tcg_op); - goto done; case 0x3: /* FSQRT */ gen_helper_vfp_sqrtd(tcg_res, tcg_op, tcg_env); goto done; @@ -XXX,XX +XXX,XX @@ static void handle_fp_1src_double(DisasContext *s, int opcode, int rd, int rn) gen_fpst = gen_helper_frint64_d; break; default: + case 0x0: /* FMOV */ + case 0x1: /* FABS */ + case 0x2: /* FNEG */ g_assert_not_reached(); } @@ -XXX,XX +XXX,XX @@ static void disas_fp_1src(DisasContext *s, uint32_t insn) goto do_unallocated; } /* fall through */ - case 0x0 ... 0x3: + case 0x3: case 0x8 ... 0xc: case 0xe ... 0xf: /* 32-to-32 and 64-to-64 ops */ @@ -XXX,XX +XXX,XX @@ static void disas_fp_1src(DisasContext *s, uint32_t insn) default: do_unallocated: + case 0x0: /* FMOV */ + case 0x1: /* FABS */ + case 0x2: /* FNEG */ unallocated_encoding(s); break; } -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Pass fpstatus not env, like most other fp helpers. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-26-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/helper.h | 6 +++--- target/arm/tcg/translate-a64.c | 15 +++++++-------- target/arm/tcg/translate-vfp.c | 6 +++--- target/arm/vfp_helper.c | 12 ++++++------ 4 files changed, 19 insertions(+), 20 deletions(-) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_3(vfp_maxnumd, f64, f64, f64, ptr) DEF_HELPER_3(vfp_minnumh, f16, f16, f16, ptr) DEF_HELPER_3(vfp_minnums, f32, f32, f32, ptr) DEF_HELPER_3(vfp_minnumd, f64, f64, f64, ptr) -DEF_HELPER_2(vfp_sqrth, f16, f16, env) -DEF_HELPER_2(vfp_sqrts, f32, f32, env) -DEF_HELPER_2(vfp_sqrtd, f64, f64, env) +DEF_HELPER_2(vfp_sqrth, f16, f16, ptr) +DEF_HELPER_2(vfp_sqrts, f32, f32, ptr) +DEF_HELPER_2(vfp_sqrtd, f64, f64, ptr) DEF_HELPER_3(vfp_cmph, void, f16, f16, env) DEF_HELPER_3(vfp_cmps, void, f32, f32, env) DEF_HELPER_3(vfp_cmpd, void, f64, f64, env) diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static void handle_fp_1src_single(DisasContext *s, int opcode, int rd, int rn) switch (opcode) { case 0x3: /* FSQRT */ - gen_helper_vfp_sqrts(tcg_res, tcg_op, tcg_env); - goto done; + gen_fpst = gen_helper_vfp_sqrts; + break; case 0x6: /* BFCVT */ gen_fpst = gen_helper_bfcvt; break; @@ -XXX,XX +XXX,XX @@ static void handle_fp_1src_single(DisasContext *s, int opcode, int rd, int rn) gen_fpst(tcg_res, tcg_op, fpst); } - done: write_fp_sreg(s, rd, tcg_res); } @@ -XXX,XX +XXX,XX @@ static void handle_fp_1src_double(DisasContext *s, int opcode, int rd, int rn) switch (opcode) { case 0x3: /* FSQRT */ - gen_helper_vfp_sqrtd(tcg_res, tcg_op, tcg_env); - goto done; + gen_fpst = gen_helper_vfp_sqrtd; + break; case 0x8: /* FRINTN */ case 0x9: /* FRINTP */ case 0xa: /* FRINTM */ @@ -XXX,XX +XXX,XX @@ static void handle_fp_1src_double(DisasContext *s, int opcode, int rd, int rn) gen_fpst(tcg_res, tcg_op, fpst); } - done: write_fp_dreg(s, rd, tcg_res); } @@ -XXX,XX +XXX,XX @@ static void handle_2misc_64(DisasContext *s, int opcode, bool u, gen_vfp_negd(tcg_rd, tcg_rn); break; case 0x7f: /* FSQRT */ - gen_helper_vfp_sqrtd(tcg_rd, tcg_rn, tcg_env); + gen_helper_vfp_sqrtd(tcg_rd, tcg_rn, tcg_fpstatus); break; case 0x1a: /* FCVTNS */ case 0x1b: /* FCVTMS */ @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) handle_2misc_fcmp_zero(s, opcode, false, u, is_q, size, rn, rd); return; case 0x7f: /* FSQRT */ + need_fpstatus = true; if (size == 3 && !is_q) { unallocated_encoding(s); return; @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) gen_vfp_negs(tcg_res, tcg_op); break; case 0x7f: /* FSQRT */ - gen_helper_vfp_sqrts(tcg_res, tcg_op, tcg_env); + gen_helper_vfp_sqrts(tcg_res, tcg_op, tcg_fpstatus); break; case 0x1a: /* FCVTNS */ case 0x1b: /* FCVTMS */ diff --git a/target/arm/tcg/translate-vfp.c b/target/arm/tcg/translate-vfp.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-vfp.c +++ b/target/arm/tcg/translate-vfp.c @@ -XXX,XX +XXX,XX @@ DO_VFP_2OP(VNEG, dp, gen_vfp_negd, aa32_fpdp_v2) static void gen_VSQRT_hp(TCGv_i32 vd, TCGv_i32 vm) { - gen_helper_vfp_sqrth(vd, vm, tcg_env); + gen_helper_vfp_sqrth(vd, vm, fpstatus_ptr(FPST_FPCR_F16)); } static void gen_VSQRT_sp(TCGv_i32 vd, TCGv_i32 vm) { - gen_helper_vfp_sqrts(vd, vm, tcg_env); + gen_helper_vfp_sqrts(vd, vm, fpstatus_ptr(FPST_FPCR)); } static void gen_VSQRT_dp(TCGv_i64 vd, TCGv_i64 vm) { - gen_helper_vfp_sqrtd(vd, vm, tcg_env); + gen_helper_vfp_sqrtd(vd, vm, fpstatus_ptr(FPST_FPCR)); } DO_VFP_2OP(VSQRT, hp, gen_VSQRT_hp, aa32_fp16_arith) diff --git a/target/arm/vfp_helper.c b/target/arm/vfp_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vfp_helper.c +++ b/target/arm/vfp_helper.c @@ -XXX,XX +XXX,XX @@ VFP_BINOP(minnum) VFP_BINOP(maxnum) #undef VFP_BINOP -dh_ctype_f16 VFP_HELPER(sqrt, h)(dh_ctype_f16 a, CPUARMState *env) +dh_ctype_f16 VFP_HELPER(sqrt, h)(dh_ctype_f16 a, void *fpstp) { - return float16_sqrt(a, &env->vfp.fp_status_f16); + return float16_sqrt(a, fpstp); } -float32 VFP_HELPER(sqrt, s)(float32 a, CPUARMState *env) +float32 VFP_HELPER(sqrt, s)(float32 a, void *fpstp) { - return float32_sqrt(a, &env->vfp.fp_status); + return float32_sqrt(a, fpstp); } -float64 VFP_HELPER(sqrt, d)(float64 a, CPUARMState *env) +float64 VFP_HELPER(sqrt, d)(float64 a, void *fpstp) { - return float64_sqrt(a, &env->vfp.fp_status); + return float64_sqrt(a, fpstp); } static void softfloat_to_vfp_compare(CPUARMState *env, FloatRelation cmp) -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> This function is identical with helper_vfp_sqrth. Replace all uses. Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-27-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/helper-a64.h | 1 - target/arm/tcg/helper-a64.c | 11 ----------- target/arm/tcg/translate-a64.c | 4 ++-- 3 files changed, 2 insertions(+), 14 deletions(-) diff --git a/target/arm/tcg/helper-a64.h b/target/arm/tcg/helper-a64.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/helper-a64.h +++ b/target/arm/tcg/helper-a64.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_2(advsimd_rinth_exact, f16, f16, ptr) DEF_HELPER_2(advsimd_rinth, f16, f16, ptr) DEF_HELPER_2(advsimd_f16tosinth, i32, f16, ptr) DEF_HELPER_2(advsimd_f16touinth, i32, f16, ptr) -DEF_HELPER_2(sqrt_f16, f16, f16, ptr) DEF_HELPER_2(exception_return, void, env, i64) DEF_HELPER_FLAGS_2(dc_zva, TCG_CALL_NO_WG, void, env, i64) diff --git a/target/arm/tcg/helper-a64.c b/target/arm/tcg/helper-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/helper-a64.c +++ b/target/arm/tcg/helper-a64.c @@ -XXX,XX +XXX,XX @@ illegal_return: "resuming execution at 0x%" PRIx64 "\n", cur_el, env->pc); } -/* - * Square Root and Reciprocal square root - */ - -uint32_t HELPER(sqrt_f16)(uint32_t a, void *fpstp) -{ - float_status *s = fpstp; - - return float16_sqrt(a, s); -} - void HELPER(dc_zva)(CPUARMState *env, uint64_t vaddr_in) { uintptr_t ra = GETPC(); diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static void handle_fp_1src_half(DisasContext *s, int opcode, int rd, int rn) switch (opcode) { case 0x3: /* FSQRT */ fpst = fpstatus_ptr(FPST_FPCR_F16); - gen_helper_sqrt_f16(tcg_res, tcg_op, fpst); + gen_helper_vfp_sqrth(tcg_res, tcg_op, fpst); break; case 0x8: /* FRINTN */ case 0x9: /* FRINTP */ @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc_fp16(DisasContext *s, uint32_t insn) gen_helper_rsqrte_f16(tcg_res, tcg_op, tcg_fpstatus); break; case 0x7f: /* FSQRT */ - gen_helper_sqrt_f16(tcg_res, tcg_op, tcg_fpstatus); + gen_helper_vfp_sqrth(tcg_res, tcg_op, tcg_fpstatus); break; default: g_assert_not_reached(); -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-28-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 1 + target/arm/tcg/translate-a64.c | 72 ++++++++++++++++++++++++++++------ 2 files changed, 62 insertions(+), 11 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ FMINV_s 0110 1110 10 11000 01111 10 ..... ..... @rr_q1e2 FMOV_s 00011110 .. 1 000000 10000 ..... ..... @rr_hsd FABS_s 00011110 .. 1 000001 10000 ..... ..... @rr_hsd FNEG_s 00011110 .. 1 000010 10000 ..... ..... @rr_hsd +FSQRT_s 00011110 .. 1 000011 10000 ..... ..... @rr_hsd # Floating-point Immediate diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static const FPScalar1Int f_scalar_fneg = { }; TRANS(FNEG_s, do_fp1_scalar_int, a, &f_scalar_fneg) +typedef struct FPScalar1 { + void (*gen_h)(TCGv_i32, TCGv_i32, TCGv_ptr); + void (*gen_s)(TCGv_i32, TCGv_i32, TCGv_ptr); + void (*gen_d)(TCGv_i64, TCGv_i64, TCGv_ptr); +} FPScalar1; + +static bool do_fp1_scalar(DisasContext *s, arg_rr_e *a, + const FPScalar1 *f, int rmode) +{ + TCGv_i32 tcg_rmode = NULL; + TCGv_ptr fpst; + TCGv_i64 t64; + TCGv_i32 t32; + int check = fp_access_check_scalar_hsd(s, a->esz); + + if (check <= 0) { + return check == 0; + } + + fpst = fpstatus_ptr(a->esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR); + if (rmode >= 0) { + tcg_rmode = gen_set_rmode(rmode, fpst); + } + + switch (a->esz) { + case MO_64: + t64 = read_fp_dreg(s, a->rn); + f->gen_d(t64, t64, fpst); + write_fp_dreg(s, a->rd, t64); + break; + case MO_32: + t32 = read_fp_sreg(s, a->rn); + f->gen_s(t32, t32, fpst); + write_fp_sreg(s, a->rd, t32); + break; + case MO_16: + t32 = read_fp_hreg(s, a->rn); + f->gen_h(t32, t32, fpst); + write_fp_sreg(s, a->rd, t32); + break; + default: + g_assert_not_reached(); + } + + if (rmode >= 0) { + gen_restore_rmode(tcg_rmode, fpst); + } + return true; +} + +static const FPScalar1 f_scalar_fsqrt = { + gen_helper_vfp_sqrth, + gen_helper_vfp_sqrts, + gen_helper_vfp_sqrtd, +}; +TRANS(FSQRT_s, do_fp1_scalar, a, &f_scalar_fsqrt, -1) + /* Floating-point data-processing (1 source) - half precision */ static void handle_fp_1src_half(DisasContext *s, int opcode, int rd, int rn) { @@ -XXX,XX +XXX,XX @@ static void handle_fp_1src_half(DisasContext *s, int opcode, int rd, int rn) TCGv_i32 tcg_res = tcg_temp_new_i32(); switch (opcode) { - case 0x3: /* FSQRT */ - fpst = fpstatus_ptr(FPST_FPCR_F16); - gen_helper_vfp_sqrth(tcg_res, tcg_op, fpst); - break; case 0x8: /* FRINTN */ case 0x9: /* FRINTP */ case 0xa: /* FRINTM */ @@ -XXX,XX +XXX,XX @@ static void handle_fp_1src_half(DisasContext *s, int opcode, int rd, int rn) case 0x0: /* FMOV */ case 0x1: /* FABS */ case 0x2: /* FNEG */ + case 0x3: /* FSQRT */ g_assert_not_reached(); } @@ -XXX,XX +XXX,XX @@ static void handle_fp_1src_single(DisasContext *s, int opcode, int rd, int rn) tcg_res = tcg_temp_new_i32(); switch (opcode) { - case 0x3: /* FSQRT */ - gen_fpst = gen_helper_vfp_sqrts; - break; case 0x6: /* BFCVT */ gen_fpst = gen_helper_bfcvt; break; @@ -XXX,XX +XXX,XX @@ static void handle_fp_1src_single(DisasContext *s, int opcode, int rd, int rn) case 0x0: /* FMOV */ case 0x1: /* FABS */ case 0x2: /* FNEG */ + case 0x3: /* FSQRT */ g_assert_not_reached(); } @@ -XXX,XX +XXX,XX @@ static void handle_fp_1src_double(DisasContext *s, int opcode, int rd, int rn) tcg_res = tcg_temp_new_i64(); switch (opcode) { - case 0x3: /* FSQRT */ - gen_fpst = gen_helper_vfp_sqrtd; - break; case 0x8: /* FRINTN */ case 0x9: /* FRINTP */ case 0xa: /* FRINTM */ @@ -XXX,XX +XXX,XX @@ static void handle_fp_1src_double(DisasContext *s, int opcode, int rd, int rn) case 0x0: /* FMOV */ case 0x1: /* FABS */ case 0x2: /* FNEG */ + case 0x3: /* FSQRT */ g_assert_not_reached(); } @@ -XXX,XX +XXX,XX @@ static void disas_fp_1src(DisasContext *s, uint32_t insn) goto do_unallocated; } /* fall through */ - case 0x3: case 0x8 ... 0xc: case 0xe ... 0xf: /* 32-to-32 and 64-to-64 ops */ @@ -XXX,XX +XXX,XX @@ static void disas_fp_1src(DisasContext *s, uint32_t insn) case 0x0: /* FMOV */ case 0x1: /* FABS */ case 0x2: /* FNEG */ + case 0x3: /* FSQRT */ unallocated_encoding(s); break; } -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Remove handle_fp_1src_half as these were the last insns decoded by that function. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-29-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 8 +++ target/arm/tcg/translate-a64.c | 117 +++++++++++---------------------- 2 files changed, 46 insertions(+), 79 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ FABS_s 00011110 .. 1 000001 10000 ..... ..... @rr_hsd FNEG_s 00011110 .. 1 000010 10000 ..... ..... @rr_hsd FSQRT_s 00011110 .. 1 000011 10000 ..... ..... @rr_hsd +FRINTN_s 00011110 .. 1 001000 10000 ..... ..... @rr_hsd +FRINTP_s 00011110 .. 1 001001 10000 ..... ..... @rr_hsd +FRINTM_s 00011110 .. 1 001010 10000 ..... ..... @rr_hsd +FRINTZ_s 00011110 .. 1 001011 10000 ..... ..... @rr_hsd +FRINTA_s 00011110 .. 1 001100 10000 ..... ..... @rr_hsd +FRINTX_s 00011110 .. 1 001110 10000 ..... ..... @rr_hsd +FRINTI_s 00011110 .. 1 001111 10000 ..... ..... @rr_hsd + # Floating-point Immediate FMOVI_s 0001 1110 .. 1 imm:8 100 00000 rd:5 esz=%esz_hsd diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static const FPScalar1 f_scalar_fsqrt = { }; TRANS(FSQRT_s, do_fp1_scalar, a, &f_scalar_fsqrt, -1) -/* Floating-point data-processing (1 source) - half precision */ -static void handle_fp_1src_half(DisasContext *s, int opcode, int rd, int rn) -{ - TCGv_ptr fpst = NULL; - TCGv_i32 tcg_op = read_fp_hreg(s, rn); - TCGv_i32 tcg_res = tcg_temp_new_i32(); +static const FPScalar1 f_scalar_frint = { + gen_helper_advsimd_rinth, + gen_helper_rints, + gen_helper_rintd, +}; +TRANS(FRINTN_s, do_fp1_scalar, a, &f_scalar_frint, FPROUNDING_TIEEVEN) +TRANS(FRINTP_s, do_fp1_scalar, a, &f_scalar_frint, FPROUNDING_POSINF) +TRANS(FRINTM_s, do_fp1_scalar, a, &f_scalar_frint, FPROUNDING_NEGINF) +TRANS(FRINTZ_s, do_fp1_scalar, a, &f_scalar_frint, FPROUNDING_ZERO) +TRANS(FRINTA_s, do_fp1_scalar, a, &f_scalar_frint, FPROUNDING_TIEAWAY) +TRANS(FRINTI_s, do_fp1_scalar, a, &f_scalar_frint, -1) - switch (opcode) { - case 0x8: /* FRINTN */ - case 0x9: /* FRINTP */ - case 0xa: /* FRINTM */ - case 0xb: /* FRINTZ */ - case 0xc: /* FRINTA */ - { - TCGv_i32 tcg_rmode; - - fpst = fpstatus_ptr(FPST_FPCR_F16); - tcg_rmode = gen_set_rmode(opcode & 7, fpst); - gen_helper_advsimd_rinth(tcg_res, tcg_op, fpst); - gen_restore_rmode(tcg_rmode, fpst); - break; - } - case 0xe: /* FRINTX */ - fpst = fpstatus_ptr(FPST_FPCR_F16); - gen_helper_advsimd_rinth_exact(tcg_res, tcg_op, fpst); - break; - case 0xf: /* FRINTI */ - fpst = fpstatus_ptr(FPST_FPCR_F16); - gen_helper_advsimd_rinth(tcg_res, tcg_op, fpst); - break; - default: - case 0x0: /* FMOV */ - case 0x1: /* FABS */ - case 0x2: /* FNEG */ - case 0x3: /* FSQRT */ - g_assert_not_reached(); - } - - write_fp_sreg(s, rd, tcg_res); -} +static const FPScalar1 f_scalar_frintx = { + gen_helper_advsimd_rinth_exact, + gen_helper_rints_exact, + gen_helper_rintd_exact, +}; +TRANS(FRINTX_s, do_fp1_scalar, a, &f_scalar_frintx, -1) /* Floating-point data-processing (1 source) - single precision */ static void handle_fp_1src_single(DisasContext *s, int opcode, int rd, int rn) @@ -XXX,XX +XXX,XX @@ static void handle_fp_1src_single(DisasContext *s, int opcode, int rd, int rn) case 0x6: /* BFCVT */ gen_fpst = gen_helper_bfcvt; break; - case 0x8: /* FRINTN */ - case 0x9: /* FRINTP */ - case 0xa: /* FRINTM */ - case 0xb: /* FRINTZ */ - case 0xc: /* FRINTA */ - rmode = opcode & 7; - gen_fpst = gen_helper_rints; - break; - case 0xe: /* FRINTX */ - gen_fpst = gen_helper_rints_exact; - break; - case 0xf: /* FRINTI */ - gen_fpst = gen_helper_rints; - break; case 0x10: /* FRINT32Z */ rmode = FPROUNDING_ZERO; gen_fpst = gen_helper_frint32_s; @@ -XXX,XX +XXX,XX @@ static void handle_fp_1src_single(DisasContext *s, int opcode, int rd, int rn) case 0x1: /* FABS */ case 0x2: /* FNEG */ case 0x3: /* FSQRT */ + case 0x8: /* FRINTN */ + case 0x9: /* FRINTP */ + case 0xa: /* FRINTM */ + case 0xb: /* FRINTZ */ + case 0xc: /* FRINTA */ + case 0xe: /* FRINTX */ + case 0xf: /* FRINTI */ g_assert_not_reached(); } @@ -XXX,XX +XXX,XX @@ static void handle_fp_1src_double(DisasContext *s, int opcode, int rd, int rn) tcg_res = tcg_temp_new_i64(); switch (opcode) { - case 0x8: /* FRINTN */ - case 0x9: /* FRINTP */ - case 0xa: /* FRINTM */ - case 0xb: /* FRINTZ */ - case 0xc: /* FRINTA */ - rmode = opcode & 7; - gen_fpst = gen_helper_rintd; - break; - case 0xe: /* FRINTX */ - gen_fpst = gen_helper_rintd_exact; - break; - case 0xf: /* FRINTI */ - gen_fpst = gen_helper_rintd; - break; case 0x10: /* FRINT32Z */ rmode = FPROUNDING_ZERO; gen_fpst = gen_helper_frint32_d; @@ -XXX,XX +XXX,XX @@ static void handle_fp_1src_double(DisasContext *s, int opcode, int rd, int rn) case 0x1: /* FABS */ case 0x2: /* FNEG */ case 0x3: /* FSQRT */ + case 0x8: /* FRINTN */ + case 0x9: /* FRINTP */ + case 0xa: /* FRINTM */ + case 0xb: /* FRINTZ */ + case 0xc: /* FRINTA */ + case 0xe: /* FRINTX */ + case 0xf: /* FRINTI */ g_assert_not_reached(); } @@ -XXX,XX +XXX,XX @@ static void disas_fp_1src(DisasContext *s, uint32_t insn) if (type > 1 || !dc_isar_feature(aa64_frint, s)) { goto do_unallocated; } - /* fall through */ - case 0x8 ... 0xc: - case 0xe ... 0xf: /* 32-to-32 and 64-to-64 ops */ switch (type) { case 0: @@ -XXX,XX +XXX,XX @@ static void disas_fp_1src(DisasContext *s, uint32_t insn) handle_fp_1src_double(s, opcode, rd, rn); break; case 3: - if (!dc_isar_feature(aa64_fp16, s)) { - goto do_unallocated; - } - - if (!fp_access_check(s)) { - return; - } - handle_fp_1src_half(s, opcode, rd, rn); - break; default: goto do_unallocated; } @@ -XXX,XX +XXX,XX @@ static void disas_fp_1src(DisasContext *s, uint32_t insn) case 0x1: /* FABS */ case 0x2: /* FNEG */ case 0x3: /* FSQRT */ + case 0x8: /* FRINTN */ + case 0x9: /* FRINTP */ + case 0xa: /* FRINTM */ + case 0xb: /* FRINTZ */ + case 0xc: /* FRINTA */ + case 0xe: /* FRINTX */ + case 0xf: /* FRINTI */ unallocated_encoding(s); break; } -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-30-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 3 +++ target/arm/tcg/translate-a64.c | 26 +++++++------------------- 2 files changed, 10 insertions(+), 19 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ &qrrrr_e q rd rn rm ra esz @rr_h ........ ... ..... ...... rn:5 rd:5 &rr_e esz=1 +@rr_s ........ ... ..... ...... rn:5 rd:5 &rr_e esz=2 @rr_d ........ ... ..... ...... rn:5 rd:5 &rr_e esz=3 @rr_sd ........ ... ..... ...... rn:5 rd:5 &rr_e esz=%esz_sd @rr_hsd ........ ... ..... ...... rn:5 rd:5 &rr_e esz=%esz_hsd @@ -XXX,XX +XXX,XX @@ FRINTA_s 00011110 .. 1 001100 10000 ..... ..... @rr_hsd FRINTX_s 00011110 .. 1 001110 10000 ..... ..... @rr_hsd FRINTI_s 00011110 .. 1 001111 10000 ..... ..... @rr_hsd +BFCVT_s 00011110 01 1 000110 10000 ..... ..... @rr_s + # Floating-point Immediate FMOVI_s 0001 1110 .. 1 imm:8 100 00000 rd:5 esz=%esz_hsd diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static const FPScalar1 f_scalar_frintx = { }; TRANS(FRINTX_s, do_fp1_scalar, a, &f_scalar_frintx, -1) +static const FPScalar1 f_scalar_bfcvt = { + .gen_s = gen_helper_bfcvt, +}; +TRANS_FEAT(BFCVT_s, aa64_bf16, do_fp1_scalar, a, &f_scalar_bfcvt, -1) + /* Floating-point data-processing (1 source) - single precision */ static void handle_fp_1src_single(DisasContext *s, int opcode, int rd, int rn) { @@ -XXX,XX +XXX,XX @@ static void handle_fp_1src_single(DisasContext *s, int opcode, int rd, int rn) tcg_res = tcg_temp_new_i32(); switch (opcode) { - case 0x6: /* BFCVT */ - gen_fpst = gen_helper_bfcvt; - break; case 0x10: /* FRINT32Z */ rmode = FPROUNDING_ZERO; gen_fpst = gen_helper_frint32_s; @@ -XXX,XX +XXX,XX @@ static void handle_fp_1src_single(DisasContext *s, int opcode, int rd, int rn) case 0x1: /* FABS */ case 0x2: /* FNEG */ case 0x3: /* FSQRT */ + case 0x6: /* BFCVT */ case 0x8: /* FRINTN */ case 0x9: /* FRINTP */ case 0xa: /* FRINTM */ @@ -XXX,XX +XXX,XX @@ static void disas_fp_1src(DisasContext *s, uint32_t insn) } break; - case 0x6: - switch (type) { - case 1: /* BFCVT */ - if (!dc_isar_feature(aa64_bf16, s)) { - goto do_unallocated; - } - if (!fp_access_check(s)) { - return; - } - handle_fp_1src_single(s, opcode, rd, rn); - break; - default: - goto do_unallocated; - } - break; - default: do_unallocated: case 0x0: /* FMOV */ case 0x1: /* FABS */ case 0x2: /* FNEG */ case 0x3: /* FSQRT */ + case 0x6: /* BFCVT */ case 0x8: /* FRINTN */ case 0x9: /* FRINTP */ case 0xa: /* FRINTM */ -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Remove handle_fp_1src_single and handle_fp_1src_double as these were the last insns decoded by those functions. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-31-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 5 ++ target/arm/tcg/translate-a64.c | 146 ++++----------------------------- 2 files changed, 22 insertions(+), 129 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ FRINTI_s 00011110 .. 1 001111 10000 ..... ..... @rr_hsd BFCVT_s 00011110 01 1 000110 10000 ..... ..... @rr_s +FRINT32Z_s 00011110 0. 1 010000 10000 ..... ..... @rr_sd +FRINT32X_s 00011110 0. 1 010001 10000 ..... ..... @rr_sd +FRINT64Z_s 00011110 0. 1 010010 10000 ..... ..... @rr_sd +FRINT64X_s 00011110 0. 1 010011 10000 ..... ..... @rr_sd + # Floating-point Immediate FMOVI_s 0001 1110 .. 1 imm:8 100 00000 rd:5 esz=%esz_hsd diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static const FPScalar1 f_scalar_bfcvt = { }; TRANS_FEAT(BFCVT_s, aa64_bf16, do_fp1_scalar, a, &f_scalar_bfcvt, -1) -/* Floating-point data-processing (1 source) - single precision */ -static void handle_fp_1src_single(DisasContext *s, int opcode, int rd, int rn) -{ - void (*gen_fpst)(TCGv_i32, TCGv_i32, TCGv_ptr); - TCGv_i32 tcg_op, tcg_res; - TCGv_ptr fpst; - int rmode = -1; +static const FPScalar1 f_scalar_frint32 = { + NULL, + gen_helper_frint32_s, + gen_helper_frint32_d, +}; +TRANS_FEAT(FRINT32Z_s, aa64_frint, do_fp1_scalar, a, + &f_scalar_frint32, FPROUNDING_ZERO) +TRANS_FEAT(FRINT32X_s, aa64_frint, do_fp1_scalar, a, &f_scalar_frint32, -1) - tcg_op = read_fp_sreg(s, rn); - tcg_res = tcg_temp_new_i32(); - - switch (opcode) { - case 0x10: /* FRINT32Z */ - rmode = FPROUNDING_ZERO; - gen_fpst = gen_helper_frint32_s; - break; - case 0x11: /* FRINT32X */ - gen_fpst = gen_helper_frint32_s; - break; - case 0x12: /* FRINT64Z */ - rmode = FPROUNDING_ZERO; - gen_fpst = gen_helper_frint64_s; - break; - case 0x13: /* FRINT64X */ - gen_fpst = gen_helper_frint64_s; - break; - default: - case 0x0: /* FMOV */ - case 0x1: /* FABS */ - case 0x2: /* FNEG */ - case 0x3: /* FSQRT */ - case 0x6: /* BFCVT */ - case 0x8: /* FRINTN */ - case 0x9: /* FRINTP */ - case 0xa: /* FRINTM */ - case 0xb: /* FRINTZ */ - case 0xc: /* FRINTA */ - case 0xe: /* FRINTX */ - case 0xf: /* FRINTI */ - g_assert_not_reached(); - } - - fpst = fpstatus_ptr(FPST_FPCR); - if (rmode >= 0) { - TCGv_i32 tcg_rmode = gen_set_rmode(rmode, fpst); - gen_fpst(tcg_res, tcg_op, fpst); - gen_restore_rmode(tcg_rmode, fpst); - } else { - gen_fpst(tcg_res, tcg_op, fpst); - } - - write_fp_sreg(s, rd, tcg_res); -} - -/* Floating-point data-processing (1 source) - double precision */ -static void handle_fp_1src_double(DisasContext *s, int opcode, int rd, int rn) -{ - void (*gen_fpst)(TCGv_i64, TCGv_i64, TCGv_ptr); - TCGv_i64 tcg_op, tcg_res; - TCGv_ptr fpst; - int rmode = -1; - - tcg_op = read_fp_dreg(s, rn); - tcg_res = tcg_temp_new_i64(); - - switch (opcode) { - case 0x10: /* FRINT32Z */ - rmode = FPROUNDING_ZERO; - gen_fpst = gen_helper_frint32_d; - break; - case 0x11: /* FRINT32X */ - gen_fpst = gen_helper_frint32_d; - break; - case 0x12: /* FRINT64Z */ - rmode = FPROUNDING_ZERO; - gen_fpst = gen_helper_frint64_d; - break; - case 0x13: /* FRINT64X */ - gen_fpst = gen_helper_frint64_d; - break; - default: - case 0x0: /* FMOV */ - case 0x1: /* FABS */ - case 0x2: /* FNEG */ - case 0x3: /* FSQRT */ - case 0x8: /* FRINTN */ - case 0x9: /* FRINTP */ - case 0xa: /* FRINTM */ - case 0xb: /* FRINTZ */ - case 0xc: /* FRINTA */ - case 0xe: /* FRINTX */ - case 0xf: /* FRINTI */ - g_assert_not_reached(); - } - - fpst = fpstatus_ptr(FPST_FPCR); - if (rmode >= 0) { - TCGv_i32 tcg_rmode = gen_set_rmode(rmode, fpst); - gen_fpst(tcg_res, tcg_op, fpst); - gen_restore_rmode(tcg_rmode, fpst); - } else { - gen_fpst(tcg_res, tcg_op, fpst); - } - - write_fp_dreg(s, rd, tcg_res); -} +static const FPScalar1 f_scalar_frint64 = { + NULL, + gen_helper_frint64_s, + gen_helper_frint64_d, +}; +TRANS_FEAT(FRINT64Z_s, aa64_frint, do_fp1_scalar, a, + &f_scalar_frint64, FPROUNDING_ZERO) +TRANS_FEAT(FRINT64X_s, aa64_frint, do_fp1_scalar, a, &f_scalar_frint64, -1) static void handle_fp_fcvt(DisasContext *s, int opcode, int rd, int rn, int dtype, int ntype) @@ -XXX,XX +XXX,XX @@ static void disas_fp_1src(DisasContext *s, uint32_t insn) break; } - case 0x10 ... 0x13: /* FRINT{32,64}{X,Z} */ - if (type > 1 || !dc_isar_feature(aa64_frint, s)) { - goto do_unallocated; - } - /* 32-to-32 and 64-to-64 ops */ - switch (type) { - case 0: - if (!fp_access_check(s)) { - return; - } - handle_fp_1src_single(s, opcode, rd, rn); - break; - case 1: - if (!fp_access_check(s)) { - return; - } - handle_fp_1src_double(s, opcode, rd, rn); - break; - case 3: - default: - goto do_unallocated; - } - break; - default: do_unallocated: case 0x0: /* FMOV */ @@ -XXX,XX +XXX,XX @@ static void disas_fp_1src(DisasContext *s, uint32_t insn) case 0xc: /* FRINTA */ case 0xe: /* FRINTX */ case 0xf: /* FRINTI */ + case 0x10 ... 0x13: /* FRINT{32,64}{X,Z} */ unallocated_encoding(s); break; } -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Remove handle_fp_fcvt and disas_fp_1src as these were the last insns decoded by those functions. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-32-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 7 ++ target/arm/tcg/translate-a64.c | 172 +++++++++++++-------------------- 2 files changed, 74 insertions(+), 105 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ FRINT32X_s 00011110 0. 1 010001 10000 ..... ..... @rr_sd FRINT64Z_s 00011110 0. 1 010010 10000 ..... ..... @rr_sd FRINT64X_s 00011110 0. 1 010011 10000 ..... ..... @rr_sd +FCVT_s_ds 00011110 00 1 000101 10000 ..... ..... @rr +FCVT_s_hs 00011110 00 1 000111 10000 ..... ..... @rr +FCVT_s_sd 00011110 01 1 000100 10000 ..... ..... @rr +FCVT_s_hd 00011110 01 1 000111 10000 ..... ..... @rr +FCVT_s_sh 00011110 11 1 000100 10000 ..... ..... @rr +FCVT_s_dh 00011110 11 1 000101 10000 ..... ..... @rr + # Floating-point Immediate FMOVI_s 0001 1110 .. 1 imm:8 100 00000 rd:5 esz=%esz_hsd diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ TRANS_FEAT(FRINT64Z_s, aa64_frint, do_fp1_scalar, a, &f_scalar_frint64, FPROUNDING_ZERO) TRANS_FEAT(FRINT64X_s, aa64_frint, do_fp1_scalar, a, &f_scalar_frint64, -1) -static void handle_fp_fcvt(DisasContext *s, int opcode, - int rd, int rn, int dtype, int ntype) +static bool trans_FCVT_s_ds(DisasContext *s, arg_rr *a) { - switch (ntype) { - case 0x0: - { - TCGv_i32 tcg_rn = read_fp_sreg(s, rn); - if (dtype == 1) { - /* Single to double */ - TCGv_i64 tcg_rd = tcg_temp_new_i64(); - gen_helper_vfp_fcvtds(tcg_rd, tcg_rn, tcg_env); - write_fp_dreg(s, rd, tcg_rd); - } else { - /* Single to half */ - TCGv_i32 tcg_rd = tcg_temp_new_i32(); - TCGv_i32 ahp = get_ahp_flag(); - TCGv_ptr fpst = fpstatus_ptr(FPST_FPCR); + if (fp_access_check(s)) { + TCGv_i32 tcg_rn = read_fp_sreg(s, a->rn); + TCGv_i64 tcg_rd = tcg_temp_new_i64(); - gen_helper_vfp_fcvt_f32_to_f16(tcg_rd, tcg_rn, fpst, ahp); - /* write_fp_sreg is OK here because top half of tcg_rd is zero */ - write_fp_sreg(s, rd, tcg_rd); - } - break; - } - case 0x1: - { - TCGv_i64 tcg_rn = read_fp_dreg(s, rn); - TCGv_i32 tcg_rd = tcg_temp_new_i32(); - if (dtype == 0) { - /* Double to single */ - gen_helper_vfp_fcvtsd(tcg_rd, tcg_rn, tcg_env); - } else { - TCGv_ptr fpst = fpstatus_ptr(FPST_FPCR); - TCGv_i32 ahp = get_ahp_flag(); - /* Double to half */ - gen_helper_vfp_fcvt_f64_to_f16(tcg_rd, tcg_rn, fpst, ahp); - /* write_fp_sreg is OK here because top half of tcg_rd is zero */ - } - write_fp_sreg(s, rd, tcg_rd); - break; - } - case 0x3: - { - TCGv_i32 tcg_rn = read_fp_sreg(s, rn); - TCGv_ptr tcg_fpst = fpstatus_ptr(FPST_FPCR); - TCGv_i32 tcg_ahp = get_ahp_flag(); - tcg_gen_ext16u_i32(tcg_rn, tcg_rn); - if (dtype == 0) { - /* Half to single */ - TCGv_i32 tcg_rd = tcg_temp_new_i32(); - gen_helper_vfp_fcvt_f16_to_f32(tcg_rd, tcg_rn, tcg_fpst, tcg_ahp); - write_fp_sreg(s, rd, tcg_rd); - } else { - /* Half to double */ - TCGv_i64 tcg_rd = tcg_temp_new_i64(); - gen_helper_vfp_fcvt_f16_to_f64(tcg_rd, tcg_rn, tcg_fpst, tcg_ahp); - write_fp_dreg(s, rd, tcg_rd); - } - break; - } - default: - g_assert_not_reached(); + gen_helper_vfp_fcvtds(tcg_rd, tcg_rn, tcg_env); + write_fp_dreg(s, a->rd, tcg_rd); } + return true; } -/* Floating point data-processing (1 source) - * 31 30 29 28 24 23 22 21 20 15 14 10 9 5 4 0 - * +---+---+---+-----------+------+---+--------+-----------+------+------+ - * | M | 0 | S | 1 1 1 1 0 | type | 1 | opcode | 1 0 0 0 0 | Rn | Rd | - * +---+---+---+-----------+------+---+--------+-----------+------+------+ - */ -static void disas_fp_1src(DisasContext *s, uint32_t insn) +static bool trans_FCVT_s_hs(DisasContext *s, arg_rr *a) { - int mos = extract32(insn, 29, 3); - int type = extract32(insn, 22, 2); - int opcode = extract32(insn, 15, 6); - int rn = extract32(insn, 5, 5); - int rd = extract32(insn, 0, 5); + if (fp_access_check(s)) { + TCGv_i32 tmp = read_fp_sreg(s, a->rn); + TCGv_i32 ahp = get_ahp_flag(); + TCGv_ptr fpst = fpstatus_ptr(FPST_FPCR); - if (mos) { - goto do_unallocated; + gen_helper_vfp_fcvt_f32_to_f16(tmp, tmp, fpst, ahp); + /* write_fp_sreg is OK here because top half of result is zero */ + write_fp_sreg(s, a->rd, tmp); } + return true; +} - switch (opcode) { - case 0x4: case 0x5: case 0x7: - { - /* FCVT between half, single and double precision */ - int dtype = extract32(opcode, 0, 2); - if (type == 2 || dtype == type) { - goto do_unallocated; - } - if (!fp_access_check(s)) { - return; - } +static bool trans_FCVT_s_sd(DisasContext *s, arg_rr *a) +{ + if (fp_access_check(s)) { + TCGv_i64 tcg_rn = read_fp_dreg(s, a->rn); + TCGv_i32 tcg_rd = tcg_temp_new_i32(); - handle_fp_fcvt(s, opcode, rd, rn, dtype, type); - break; + gen_helper_vfp_fcvtsd(tcg_rd, tcg_rn, tcg_env); + write_fp_sreg(s, a->rd, tcg_rd); } + return true; +} - default: - do_unallocated: - case 0x0: /* FMOV */ - case 0x1: /* FABS */ - case 0x2: /* FNEG */ - case 0x3: /* FSQRT */ - case 0x6: /* BFCVT */ - case 0x8: /* FRINTN */ - case 0x9: /* FRINTP */ - case 0xa: /* FRINTM */ - case 0xb: /* FRINTZ */ - case 0xc: /* FRINTA */ - case 0xe: /* FRINTX */ - case 0xf: /* FRINTI */ - case 0x10 ... 0x13: /* FRINT{32,64}{X,Z} */ - unallocated_encoding(s); - break; +static bool trans_FCVT_s_hd(DisasContext *s, arg_rr *a) +{ + if (fp_access_check(s)) { + TCGv_i64 tcg_rn = read_fp_dreg(s, a->rn); + TCGv_i32 tcg_rd = tcg_temp_new_i32(); + TCGv_i32 ahp = get_ahp_flag(); + TCGv_ptr fpst = fpstatus_ptr(FPST_FPCR); + + gen_helper_vfp_fcvt_f64_to_f16(tcg_rd, tcg_rn, fpst, ahp); + /* write_fp_sreg is OK here because top half of tcg_rd is zero */ + write_fp_sreg(s, a->rd, tcg_rd); } + return true; +} + +static bool trans_FCVT_s_sh(DisasContext *s, arg_rr *a) +{ + if (fp_access_check(s)) { + TCGv_i32 tcg_rn = read_fp_hreg(s, a->rn); + TCGv_i32 tcg_rd = tcg_temp_new_i32(); + TCGv_ptr tcg_fpst = fpstatus_ptr(FPST_FPCR); + TCGv_i32 tcg_ahp = get_ahp_flag(); + + gen_helper_vfp_fcvt_f16_to_f32(tcg_rd, tcg_rn, tcg_fpst, tcg_ahp); + write_fp_sreg(s, a->rd, tcg_rd); + } + return true; +} + +static bool trans_FCVT_s_dh(DisasContext *s, arg_rr *a) +{ + if (fp_access_check(s)) { + TCGv_i32 tcg_rn = read_fp_hreg(s, a->rn); + TCGv_i64 tcg_rd = tcg_temp_new_i64(); + TCGv_ptr tcg_fpst = fpstatus_ptr(FPST_FPCR); + TCGv_i32 tcg_ahp = get_ahp_flag(); + + gen_helper_vfp_fcvt_f16_to_f64(tcg_rd, tcg_rn, tcg_fpst, tcg_ahp); + write_fp_dreg(s, a->rd, tcg_rd); + } + return true; } /* Handle floating point <=> fixed point conversions. Note that we can @@ -XXX,XX +XXX,XX @@ static void disas_data_proc_fp(DisasContext *s, uint32_t insn) break; case 2: /* [15:12] == x100 */ /* Floating point data-processing (1 source) */ - disas_fp_1src(s, insn); + unallocated_encoding(s); /* in decodetree */ break; case 3: /* [15:12] == 1000 */ unallocated_encoding(s); -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> This includes SCVTF, UCVTF, FCVT{N,P,M,Z,A}{S,U}. Remove disas_fp_fixed_conv as those were the last insns decoded by that function. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-33-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 40 ++++ target/arm/tcg/translate-a64.c | 391 ++++++++++++++------------------- 2 files changed, 209 insertions(+), 222 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ FMAXV_s 0110 1110 00 11000 01111 10 ..... ..... @rr_q1e2 FMINV_h 0.00 1110 10 11000 01111 10 ..... ..... @qrr_h FMINV_s 0110 1110 10 11000 01111 10 ..... ..... @rr_q1e2 +# Conversion between floating-point and fixed-point (general register) + +&fcvt rd rn esz sf shift +%fcvt_shift32 10:5 !function=rsub_32 +%fcvt_shift64 10:6 !function=rsub_64 + +@fcvt32 0 ....... .. ...... 1..... rn:5 rd:5 \ + &fcvt sf=0 esz=%esz_hsd shift=%fcvt_shift32 +@fcvt64 1 ....... .. ...... ...... rn:5 rd:5 \ + &fcvt sf=1 esz=%esz_hsd shift=%fcvt_shift64 + +SCVTF_g . 0011110 .. 000010 ...... ..... ..... @fcvt32 +SCVTF_g . 0011110 .. 000010 ...... ..... ..... @fcvt64 +UCVTF_g . 0011110 .. 000011 ...... ..... ..... @fcvt32 +UCVTF_g . 0011110 .. 000011 ...... ..... ..... @fcvt64 + +FCVTZS_g . 0011110 .. 011000 ...... ..... ..... @fcvt32 +FCVTZS_g . 0011110 .. 011000 ...... ..... ..... @fcvt64 +FCVTZU_g . 0011110 .. 011001 ...... ..... ..... @fcvt32 +FCVTZU_g . 0011110 .. 011001 ...... ..... ..... @fcvt64 + +# Conversion between floating-point and integer (general register) + +@icvt sf:1 ....... .. ...... ...... rn:5 rd:5 \ + &fcvt esz=%esz_hsd shift=0 + +SCVTF_g . 0011110 .. 100010 000000 ..... ..... @icvt +UCVTF_g . 0011110 .. 100011 000000 ..... ..... @icvt + +FCVTNS_g . 0011110 .. 100000 000000 ..... ..... @icvt +FCVTNU_g . 0011110 .. 100001 000000 ..... ..... @icvt +FCVTPS_g . 0011110 .. 101000 000000 ..... ..... @icvt +FCVTPU_g . 0011110 .. 101001 000000 ..... ..... @icvt +FCVTMS_g . 0011110 .. 110000 000000 ..... ..... @icvt +FCVTMU_g . 0011110 .. 110001 000000 ..... ..... @icvt +FCVTZS_g . 0011110 .. 111000 000000 ..... ..... @icvt +FCVTZU_g . 0011110 .. 111001 000000 ..... ..... @icvt +FCVTAS_g . 0011110 .. 100100 000000 ..... ..... @icvt +FCVTAU_g . 0011110 .. 100101 000000 ..... ..... @icvt + # Floating-point data processing (1 source) FMOV_s 00011110 .. 1 000000 10000 ..... ..... @rr_hsd diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static bool trans_FCVT_s_dh(DisasContext *s, arg_rr *a) return true; } -/* Handle floating point <=> fixed point conversions. Note that we can - * also deal with fp <=> integer conversions as a special case (scale == 64) - * OPTME: consider handling that special case specially or at least skipping - * the call to scalbn in the helpers for zero shifts. - */ -static void handle_fpfpcvt(DisasContext *s, int rd, int rn, int opcode, - bool itof, int rmode, int scale, int sf, int type) +static bool do_cvtf_scalar(DisasContext *s, MemOp esz, int rd, int shift, + TCGv_i64 tcg_int, bool is_signed) { - bool is_signed = !(opcode & 1); TCGv_ptr tcg_fpstatus; TCGv_i32 tcg_shift, tcg_single; TCGv_i64 tcg_double; - tcg_fpstatus = fpstatus_ptr(type == 3 ? FPST_FPCR_F16 : FPST_FPCR); + tcg_fpstatus = fpstatus_ptr(esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR); + tcg_shift = tcg_constant_i32(shift); - tcg_shift = tcg_constant_i32(64 - scale); - - if (itof) { - TCGv_i64 tcg_int = cpu_reg(s, rn); - if (!sf) { - TCGv_i64 tcg_extend = tcg_temp_new_i64(); - - if (is_signed) { - tcg_gen_ext32s_i64(tcg_extend, tcg_int); - } else { - tcg_gen_ext32u_i64(tcg_extend, tcg_int); - } - - tcg_int = tcg_extend; + switch (esz) { + case MO_64: + tcg_double = tcg_temp_new_i64(); + if (is_signed) { + gen_helper_vfp_sqtod(tcg_double, tcg_int, tcg_shift, tcg_fpstatus); + } else { + gen_helper_vfp_uqtod(tcg_double, tcg_int, tcg_shift, tcg_fpstatus); } + write_fp_dreg(s, rd, tcg_double); + break; - switch (type) { - case 1: /* float64 */ - tcg_double = tcg_temp_new_i64(); - if (is_signed) { - gen_helper_vfp_sqtod(tcg_double, tcg_int, - tcg_shift, tcg_fpstatus); - } else { - gen_helper_vfp_uqtod(tcg_double, tcg_int, - tcg_shift, tcg_fpstatus); - } - write_fp_dreg(s, rd, tcg_double); - break; - - case 0: /* float32 */ - tcg_single = tcg_temp_new_i32(); - if (is_signed) { - gen_helper_vfp_sqtos(tcg_single, tcg_int, - tcg_shift, tcg_fpstatus); - } else { - gen_helper_vfp_uqtos(tcg_single, tcg_int, - tcg_shift, tcg_fpstatus); - } - write_fp_sreg(s, rd, tcg_single); - break; - - case 3: /* float16 */ - tcg_single = tcg_temp_new_i32(); - if (is_signed) { - gen_helper_vfp_sqtoh(tcg_single, tcg_int, - tcg_shift, tcg_fpstatus); - } else { - gen_helper_vfp_uqtoh(tcg_single, tcg_int, - tcg_shift, tcg_fpstatus); - } - write_fp_sreg(s, rd, tcg_single); - break; - - default: - g_assert_not_reached(); + case MO_32: + tcg_single = tcg_temp_new_i32(); + if (is_signed) { + gen_helper_vfp_sqtos(tcg_single, tcg_int, tcg_shift, tcg_fpstatus); + } else { + gen_helper_vfp_uqtos(tcg_single, tcg_int, tcg_shift, tcg_fpstatus); } - } else { - TCGv_i64 tcg_int = cpu_reg(s, rd); - TCGv_i32 tcg_rmode; + write_fp_sreg(s, rd, tcg_single); + break; - if (extract32(opcode, 2, 1)) { - /* There are too many rounding modes to all fit into rmode, - * so FCVTA[US] is a special case. - */ - rmode = FPROUNDING_TIEAWAY; + case MO_16: + tcg_single = tcg_temp_new_i32(); + if (is_signed) { + gen_helper_vfp_sqtoh(tcg_single, tcg_int, tcg_shift, tcg_fpstatus); + } else { + gen_helper_vfp_uqtoh(tcg_single, tcg_int, tcg_shift, tcg_fpstatus); } + write_fp_sreg(s, rd, tcg_single); + break; - tcg_rmode = gen_set_rmode(rmode, tcg_fpstatus); - - switch (type) { - case 1: /* float64 */ - tcg_double = read_fp_dreg(s, rn); - if (is_signed) { - if (!sf) { - gen_helper_vfp_tosld(tcg_int, tcg_double, - tcg_shift, tcg_fpstatus); - } else { - gen_helper_vfp_tosqd(tcg_int, tcg_double, - tcg_shift, tcg_fpstatus); - } - } else { - if (!sf) { - gen_helper_vfp_tould(tcg_int, tcg_double, - tcg_shift, tcg_fpstatus); - } else { - gen_helper_vfp_touqd(tcg_int, tcg_double, - tcg_shift, tcg_fpstatus); - } - } - if (!sf) { - tcg_gen_ext32u_i64(tcg_int, tcg_int); - } - break; - - case 0: /* float32 */ - tcg_single = read_fp_sreg(s, rn); - if (sf) { - if (is_signed) { - gen_helper_vfp_tosqs(tcg_int, tcg_single, - tcg_shift, tcg_fpstatus); - } else { - gen_helper_vfp_touqs(tcg_int, tcg_single, - tcg_shift, tcg_fpstatus); - } - } else { - TCGv_i32 tcg_dest = tcg_temp_new_i32(); - if (is_signed) { - gen_helper_vfp_tosls(tcg_dest, tcg_single, - tcg_shift, tcg_fpstatus); - } else { - gen_helper_vfp_touls(tcg_dest, tcg_single, - tcg_shift, tcg_fpstatus); - } - tcg_gen_extu_i32_i64(tcg_int, tcg_dest); - } - break; - - case 3: /* float16 */ - tcg_single = read_fp_sreg(s, rn); - if (sf) { - if (is_signed) { - gen_helper_vfp_tosqh(tcg_int, tcg_single, - tcg_shift, tcg_fpstatus); - } else { - gen_helper_vfp_touqh(tcg_int, tcg_single, - tcg_shift, tcg_fpstatus); - } - } else { - TCGv_i32 tcg_dest = tcg_temp_new_i32(); - if (is_signed) { - gen_helper_vfp_toslh(tcg_dest, tcg_single, - tcg_shift, tcg_fpstatus); - } else { - gen_helper_vfp_toulh(tcg_dest, tcg_single, - tcg_shift, tcg_fpstatus); - } - tcg_gen_extu_i32_i64(tcg_int, tcg_dest); - } - break; - - default: - g_assert_not_reached(); - } - - gen_restore_rmode(tcg_rmode, tcg_fpstatus); + default: + g_assert_not_reached(); } + return true; } -/* Floating point <-> fixed point conversions - * 31 30 29 28 24 23 22 21 20 19 18 16 15 10 9 5 4 0 - * +----+---+---+-----------+------+---+-------+--------+-------+------+------+ - * | sf | 0 | S | 1 1 1 1 0 | type | 0 | rmode | opcode | scale | Rn | Rd | - * +----+---+---+-----------+------+---+-------+--------+-------+------+------+ - */ -static void disas_fp_fixed_conv(DisasContext *s, uint32_t insn) +static bool do_cvtf_g(DisasContext *s, arg_fcvt *a, bool is_signed) { - int rd = extract32(insn, 0, 5); - int rn = extract32(insn, 5, 5); - int scale = extract32(insn, 10, 6); - int opcode = extract32(insn, 16, 3); - int rmode = extract32(insn, 19, 2); - int type = extract32(insn, 22, 2); - bool sbit = extract32(insn, 29, 1); - bool sf = extract32(insn, 31, 1); - bool itof; + TCGv_i64 tcg_int; + int check = fp_access_check_scalar_hsd(s, a->esz); - if (sbit || (!sf && scale < 32)) { - unallocated_encoding(s); - return; + if (check <= 0) { + return check == 0; } - switch (type) { - case 0: /* float32 */ - case 1: /* float64 */ - break; - case 3: /* float16 */ - if (dc_isar_feature(aa64_fp16, s)) { - break; + if (a->sf) { + tcg_int = cpu_reg(s, a->rn); + } else { + tcg_int = read_cpu_reg(s, a->rn, true); + if (is_signed) { + tcg_gen_ext32s_i64(tcg_int, tcg_int); + } else { + tcg_gen_ext32u_i64(tcg_int, tcg_int); } - /* fallthru */ - default: - unallocated_encoding(s); - return; } - - switch ((rmode << 3) | opcode) { - case 0x2: /* SCVTF */ - case 0x3: /* UCVTF */ - itof = true; - break; - case 0x18: /* FCVTZS */ - case 0x19: /* FCVTZU */ - itof = false; - break; - default: - unallocated_encoding(s); - return; - } - - if (!fp_access_check(s)) { - return; - } - - handle_fpfpcvt(s, rd, rn, opcode, itof, FPROUNDING_ZERO, scale, sf, type); + return do_cvtf_scalar(s, a->esz, a->rd, a->shift, tcg_int, is_signed); } +TRANS(SCVTF_g, do_cvtf_g, a, true) +TRANS(UCVTF_g, do_cvtf_g, a, false) + +static void do_fcvt_scalar(DisasContext *s, MemOp out, MemOp esz, + TCGv_i64 tcg_out, int shift, int rn, + ARMFPRounding rmode) +{ + TCGv_ptr tcg_fpstatus; + TCGv_i32 tcg_shift, tcg_rmode, tcg_single; + + tcg_fpstatus = fpstatus_ptr(esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR); + tcg_shift = tcg_constant_i32(shift); + tcg_rmode = gen_set_rmode(rmode, tcg_fpstatus); + + switch (esz) { + case MO_64: + read_vec_element(s, tcg_out, rn, 0, MO_64); + switch (out) { + case MO_64 | MO_SIGN: + gen_helper_vfp_tosqd(tcg_out, tcg_out, tcg_shift, tcg_fpstatus); + break; + case MO_64: + gen_helper_vfp_touqd(tcg_out, tcg_out, tcg_shift, tcg_fpstatus); + break; + case MO_32 | MO_SIGN: + gen_helper_vfp_tosld(tcg_out, tcg_out, tcg_shift, tcg_fpstatus); + break; + case MO_32: + gen_helper_vfp_tould(tcg_out, tcg_out, tcg_shift, tcg_fpstatus); + break; + default: + g_assert_not_reached(); + } + break; + + case MO_32: + tcg_single = read_fp_sreg(s, rn); + switch (out) { + case MO_64 | MO_SIGN: + gen_helper_vfp_tosqs(tcg_out, tcg_single, tcg_shift, tcg_fpstatus); + break; + case MO_64: + gen_helper_vfp_touqs(tcg_out, tcg_single, tcg_shift, tcg_fpstatus); + break; + case MO_32 | MO_SIGN: + gen_helper_vfp_tosls(tcg_single, tcg_single, + tcg_shift, tcg_fpstatus); + tcg_gen_extu_i32_i64(tcg_out, tcg_single); + break; + case MO_32: + gen_helper_vfp_touls(tcg_single, tcg_single, + tcg_shift, tcg_fpstatus); + tcg_gen_extu_i32_i64(tcg_out, tcg_single); + break; + default: + g_assert_not_reached(); + } + break; + + case MO_16: + tcg_single = read_fp_hreg(s, rn); + switch (out) { + case MO_64 | MO_SIGN: + gen_helper_vfp_tosqh(tcg_out, tcg_single, tcg_shift, tcg_fpstatus); + break; + case MO_64: + gen_helper_vfp_touqh(tcg_out, tcg_single, tcg_shift, tcg_fpstatus); + break; + case MO_32 | MO_SIGN: + gen_helper_vfp_toslh(tcg_single, tcg_single, + tcg_shift, tcg_fpstatus); + tcg_gen_extu_i32_i64(tcg_out, tcg_single); + break; + case MO_32: + gen_helper_vfp_toulh(tcg_single, tcg_single, + tcg_shift, tcg_fpstatus); + tcg_gen_extu_i32_i64(tcg_out, tcg_single); + break; + default: + g_assert_not_reached(); + } + break; + + default: + g_assert_not_reached(); + } + + gen_restore_rmode(tcg_rmode, tcg_fpstatus); +} + +static bool do_fcvt_g(DisasContext *s, arg_fcvt *a, + ARMFPRounding rmode, bool is_signed) +{ + TCGv_i64 tcg_int; + int check = fp_access_check_scalar_hsd(s, a->esz); + + if (check <= 0) { + return check == 0; + } + + tcg_int = cpu_reg(s, a->rd); + do_fcvt_scalar(s, (a->sf ? MO_64 : MO_32) | (is_signed ? MO_SIGN : 0), + a->esz, tcg_int, a->shift, a->rn, rmode); + + if (!a->sf) { + tcg_gen_ext32u_i64(tcg_int, tcg_int); + } + return true; +} + +TRANS(FCVTNS_g, do_fcvt_g, a, FPROUNDING_TIEEVEN, true) +TRANS(FCVTNU_g, do_fcvt_g, a, FPROUNDING_TIEEVEN, false) +TRANS(FCVTPS_g, do_fcvt_g, a, FPROUNDING_POSINF, true) +TRANS(FCVTPU_g, do_fcvt_g, a, FPROUNDING_POSINF, false) +TRANS(FCVTMS_g, do_fcvt_g, a, FPROUNDING_NEGINF, true) +TRANS(FCVTMU_g, do_fcvt_g, a, FPROUNDING_NEGINF, false) +TRANS(FCVTZS_g, do_fcvt_g, a, FPROUNDING_ZERO, true) +TRANS(FCVTZU_g, do_fcvt_g, a, FPROUNDING_ZERO, false) +TRANS(FCVTAS_g, do_fcvt_g, a, FPROUNDING_TIEAWAY, true) +TRANS(FCVTAU_g, do_fcvt_g, a, FPROUNDING_TIEAWAY, false) + static void handle_fmov(DisasContext *s, int rd, int rn, int type, bool itof) { /* FMOV: gpr to or from float, double, or top half of quad fp reg, @@ -XXX,XX +XXX,XX @@ static void disas_fp_int_conv(DisasContext *s, uint32_t insn) switch (opcode) { case 2: /* SCVTF */ case 3: /* UCVTF */ - itof = true; - /* fallthru */ case 4: /* FCVTAS */ case 5: /* FCVTAU */ - if (rmode != 0) { - goto do_unallocated; - } - /* fallthru */ case 0: /* FCVT[NPMZ]S */ case 1: /* FCVT[NPMZ]U */ - switch (type) { - case 0: /* float32 */ - case 1: /* float64 */ - break; - case 3: /* float16 */ - if (!dc_isar_feature(aa64_fp16, s)) { - goto do_unallocated; - } - break; - default: - goto do_unallocated; - } - if (!fp_access_check(s)) { - return; - } - handle_fpfpcvt(s, rd, rn, opcode, itof, rmode, 64, sf, type); - break; + goto do_unallocated; default: switch (sf << 7 | type << 5 | rmode << 3 | opcode) { @@ -XXX,XX +XXX,XX @@ static void disas_data_proc_fp(DisasContext *s, uint32_t insn) unallocated_encoding(s); /* in decodetree */ } else if (extract32(insn, 21, 1) == 0) { /* Floating point to fixed point conversions */ - disas_fp_fixed_conv(s, insn); + unallocated_encoding(s); /* in decodetree */ } else { switch (extract32(insn, 10, 2)) { case 1: /* Floating point conditional compare */ -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-34-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 2 ++ target/arm/tcg/translate-a64.c | 41 +++++++++++++++++----------------- 2 files changed, 22 insertions(+), 21 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ FCVTZU_g . 0011110 .. 111001 000000 ..... ..... @icvt FCVTAS_g . 0011110 .. 100100 000000 ..... ..... @icvt FCVTAU_g . 0011110 .. 100101 000000 ..... ..... @icvt +FJCVTZS 0 0011110 01 111110 000000 ..... ..... @rr + # Floating-point data processing (1 source) FMOV_s 00011110 .. 1 000000 10000 ..... ..... @rr_hsd diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ TRANS(FCVTZU_g, do_fcvt_g, a, FPROUNDING_ZERO, false) TRANS(FCVTAS_g, do_fcvt_g, a, FPROUNDING_TIEAWAY, true) TRANS(FCVTAU_g, do_fcvt_g, a, FPROUNDING_TIEAWAY, false) +static bool trans_FJCVTZS(DisasContext *s, arg_FJCVTZS *a) +{ + if (!dc_isar_feature(aa64_jscvt, s)) { + return false; + } + if (fp_access_check(s)) { + TCGv_i64 t = read_fp_dreg(s, a->rn); + TCGv_ptr fpstatus = fpstatus_ptr(FPST_FPCR); + + gen_helper_fjcvtzs(t, t, fpstatus); + + tcg_gen_ext32u_i64(cpu_reg(s, a->rd), t); + tcg_gen_extrh_i64_i32(cpu_ZF, t); + tcg_gen_movi_i32(cpu_CF, 0); + tcg_gen_movi_i32(cpu_NF, 0); + tcg_gen_movi_i32(cpu_VF, 0); + } + return true; +} + static void handle_fmov(DisasContext *s, int rd, int rn, int type, bool itof) { /* FMOV: gpr to or from float, double, or top half of quad fp reg, @@ -XXX,XX +XXX,XX @@ static void handle_fmov(DisasContext *s, int rd, int rn, int type, bool itof) } } -static void handle_fjcvtzs(DisasContext *s, int rd, int rn) -{ - TCGv_i64 t = read_fp_dreg(s, rn); - TCGv_ptr fpstatus = fpstatus_ptr(FPST_FPCR); - - gen_helper_fjcvtzs(t, t, fpstatus); - - tcg_gen_ext32u_i64(cpu_reg(s, rd), t); - tcg_gen_extrh_i64_i32(cpu_ZF, t); - tcg_gen_movi_i32(cpu_CF, 0); - tcg_gen_movi_i32(cpu_NF, 0); - tcg_gen_movi_i32(cpu_VF, 0); -} - /* Floating point <-> integer conversions * 31 30 29 28 24 23 22 21 20 19 18 16 15 10 9 5 4 0 * +----+---+---+-----------+------+---+-------+-----+-------------+----+----+ @@ -XXX,XX +XXX,XX @@ static void disas_fp_int_conv(DisasContext *s, uint32_t insn) break; case 0b00111110: /* FJCVTZS */ - if (!dc_isar_feature(aa64_jscvt, s)) { - goto do_unallocated; - } else if (fp_access_check(s)) { - handle_fjcvtzs(s, rd, rn); - } - break; - default: do_unallocated: unallocated_encoding(s); -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Remove disas_fp_int_conv and disas_data_proc_fp as these were the last insns decoded by those functions. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-35-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 14 ++ target/arm/tcg/translate-a64.c | 232 ++++++++++----------------------- 2 files changed, 86 insertions(+), 160 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ FCVTAU_g . 0011110 .. 100101 000000 ..... ..... @icvt FJCVTZS 0 0011110 01 111110 000000 ..... ..... @rr +FMOV_ws 0 0011110 00 100110 000000 ..... ..... @rr +FMOV_sw 0 0011110 00 100111 000000 ..... ..... @rr + +FMOV_xd 1 0011110 01 100110 000000 ..... ..... @rr +FMOV_dx 1 0011110 01 100111 000000 ..... ..... @rr + +# Move to/from upper half of 128-bit +FMOV_xu 1 0011110 10 101110 000000 ..... ..... @rr +FMOV_ux 1 0011110 10 101111 000000 ..... ..... @rr + +# Half-precision allows both sf=0 and sf=1 with identical results +FMOV_xh - 0011110 11 100110 000000 ..... ..... @rr +FMOV_hx - 0011110 11 100111 000000 ..... ..... @rr + # Floating-point data processing (1 source) FMOV_s 00011110 .. 1 000000 10000 ..... ..... @rr_hsd diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static bool trans_FJCVTZS(DisasContext *s, arg_FJCVTZS *a) return true; } -static void handle_fmov(DisasContext *s, int rd, int rn, int type, bool itof) +static bool trans_FMOV_hx(DisasContext *s, arg_rr *a) { - /* FMOV: gpr to or from float, double, or top half of quad fp reg, - * without conversion. - */ - - if (itof) { - TCGv_i64 tcg_rn = cpu_reg(s, rn); - TCGv_i64 tmp; - - switch (type) { - case 0: - /* 32 bit */ - tmp = tcg_temp_new_i64(); - tcg_gen_ext32u_i64(tmp, tcg_rn); - write_fp_dreg(s, rd, tmp); - break; - case 1: - /* 64 bit */ - write_fp_dreg(s, rd, tcg_rn); - break; - case 2: - /* 64 bit to top half. */ - tcg_gen_st_i64(tcg_rn, tcg_env, fp_reg_hi_offset(s, rd)); - clear_vec_high(s, true, rd); - break; - case 3: - /* 16 bit */ - tmp = tcg_temp_new_i64(); - tcg_gen_ext16u_i64(tmp, tcg_rn); - write_fp_dreg(s, rd, tmp); - break; - default: - g_assert_not_reached(); - } - } else { - TCGv_i64 tcg_rd = cpu_reg(s, rd); - - switch (type) { - case 0: - /* 32 bit */ - tcg_gen_ld32u_i64(tcg_rd, tcg_env, fp_reg_offset(s, rn, MO_32)); - break; - case 1: - /* 64 bit */ - tcg_gen_ld_i64(tcg_rd, tcg_env, fp_reg_offset(s, rn, MO_64)); - break; - case 2: - /* 64 bits from top half */ - tcg_gen_ld_i64(tcg_rd, tcg_env, fp_reg_hi_offset(s, rn)); - break; - case 3: - /* 16 bit */ - tcg_gen_ld16u_i64(tcg_rd, tcg_env, fp_reg_offset(s, rn, MO_16)); - break; - default: - g_assert_not_reached(); - } + if (!dc_isar_feature(aa64_fp16, s)) { + return false; } + if (fp_access_check(s)) { + TCGv_i64 tcg_rn = cpu_reg(s, a->rn); + TCGv_i64 tmp = tcg_temp_new_i64(); + tcg_gen_ext16u_i64(tmp, tcg_rn); + write_fp_dreg(s, a->rd, tmp); + } + return true; } -/* Floating point <-> integer conversions - * 31 30 29 28 24 23 22 21 20 19 18 16 15 10 9 5 4 0 - * +----+---+---+-----------+------+---+-------+-----+-------------+----+----+ - * | sf | 0 | S | 1 1 1 1 0 | type | 1 | rmode | opc | 0 0 0 0 0 0 | Rn | Rd | - * +----+---+---+-----------+------+---+-------+-----+-------------+----+----+ - */ -static void disas_fp_int_conv(DisasContext *s, uint32_t insn) +static bool trans_FMOV_sw(DisasContext *s, arg_rr *a) { - int rd = extract32(insn, 0, 5); - int rn = extract32(insn, 5, 5); - int opcode = extract32(insn, 16, 3); - int rmode = extract32(insn, 19, 2); - int type = extract32(insn, 22, 2); - bool sbit = extract32(insn, 29, 1); - bool sf = extract32(insn, 31, 1); - bool itof = false; - - if (sbit) { - goto do_unallocated; - } - - switch (opcode) { - case 2: /* SCVTF */ - case 3: /* UCVTF */ - case 4: /* FCVTAS */ - case 5: /* FCVTAU */ - case 0: /* FCVT[NPMZ]S */ - case 1: /* FCVT[NPMZ]U */ - goto do_unallocated; - - default: - switch (sf << 7 | type << 5 | rmode << 3 | opcode) { - case 0b01100110: /* FMOV half <-> 32-bit int */ - case 0b01100111: - case 0b11100110: /* FMOV half <-> 64-bit int */ - case 0b11100111: - if (!dc_isar_feature(aa64_fp16, s)) { - goto do_unallocated; - } - /* fallthru */ - case 0b00000110: /* FMOV 32-bit */ - case 0b00000111: - case 0b10100110: /* FMOV 64-bit */ - case 0b10100111: - case 0b11001110: /* FMOV top half of 128-bit */ - case 0b11001111: - if (!fp_access_check(s)) { - return; - } - itof = opcode & 1; - handle_fmov(s, rd, rn, type, itof); - break; - - case 0b00111110: /* FJCVTZS */ - default: - do_unallocated: - unallocated_encoding(s); - return; - } - break; + if (fp_access_check(s)) { + TCGv_i64 tcg_rn = cpu_reg(s, a->rn); + TCGv_i64 tmp = tcg_temp_new_i64(); + tcg_gen_ext32u_i64(tmp, tcg_rn); + write_fp_dreg(s, a->rd, tmp); } + return true; } -/* FP-specific subcases of table C3-6 (SIMD and FP data processing) - * 31 30 29 28 25 24 0 - * +---+---+---+---------+-----------------------------+ - * | | 0 | | 1 1 1 1 | | - * +---+---+---+---------+-----------------------------+ - */ -static void disas_data_proc_fp(DisasContext *s, uint32_t insn) +static bool trans_FMOV_dx(DisasContext *s, arg_rr *a) { - if (extract32(insn, 24, 1)) { - unallocated_encoding(s); /* in decodetree */ - } else if (extract32(insn, 21, 1) == 0) { - /* Floating point to fixed point conversions */ - unallocated_encoding(s); /* in decodetree */ - } else { - switch (extract32(insn, 10, 2)) { - case 1: /* Floating point conditional compare */ - case 2: /* Floating point data-processing (2 source) */ - case 3: /* Floating point conditional select */ - unallocated_encoding(s); /* in decodetree */ - break; - case 0: - switch (ctz32(extract32(insn, 12, 4))) { - case 0: /* [15:12] == xxx1 */ - /* Floating point immediate */ - unallocated_encoding(s); /* in decodetree */ - break; - case 1: /* [15:12] == xx10 */ - /* Floating point compare */ - unallocated_encoding(s); /* in decodetree */ - break; - case 2: /* [15:12] == x100 */ - /* Floating point data-processing (1 source) */ - unallocated_encoding(s); /* in decodetree */ - break; - case 3: /* [15:12] == 1000 */ - unallocated_encoding(s); - break; - default: /* [15:12] == 0000 */ - /* Floating point <-> integer conversions */ - disas_fp_int_conv(s, insn); - break; - } - break; - } + if (fp_access_check(s)) { + TCGv_i64 tcg_rn = cpu_reg(s, a->rn); + write_fp_dreg(s, a->rd, tcg_rn); } + return true; +} + +static bool trans_FMOV_ux(DisasContext *s, arg_rr *a) +{ + if (fp_access_check(s)) { + TCGv_i64 tcg_rn = cpu_reg(s, a->rn); + tcg_gen_st_i64(tcg_rn, tcg_env, fp_reg_hi_offset(s, a->rd)); + clear_vec_high(s, true, a->rd); + } + return true; +} + +static bool trans_FMOV_xh(DisasContext *s, arg_rr *a) +{ + if (!dc_isar_feature(aa64_fp16, s)) { + return false; + } + if (fp_access_check(s)) { + TCGv_i64 tcg_rd = cpu_reg(s, a->rd); + tcg_gen_ld16u_i64(tcg_rd, tcg_env, fp_reg_offset(s, a->rn, MO_16)); + } + return true; +} + +static bool trans_FMOV_ws(DisasContext *s, arg_rr *a) +{ + if (fp_access_check(s)) { + TCGv_i64 tcg_rd = cpu_reg(s, a->rd); + tcg_gen_ld32u_i64(tcg_rd, tcg_env, fp_reg_offset(s, a->rn, MO_32)); + } + return true; +} + +static bool trans_FMOV_xd(DisasContext *s, arg_rr *a) +{ + if (fp_access_check(s)) { + TCGv_i64 tcg_rd = cpu_reg(s, a->rd); + tcg_gen_ld_i64(tcg_rd, tcg_env, fp_reg_offset(s, a->rn, MO_64)); + } + return true; +} + +static bool trans_FMOV_xu(DisasContext *s, arg_rr *a) +{ + if (fp_access_check(s)) { + TCGv_i64 tcg_rd = cpu_reg(s, a->rd); + tcg_gen_ld_i64(tcg_rd, tcg_env, fp_reg_hi_offset(s, a->rn)); + } + return true; } /* Common vector code for handling integer to FP conversion */ @@ -XXX,XX +XXX,XX @@ static void disas_data_proc_simd(DisasContext *s, uint32_t insn) static void disas_data_proc_simd_fp(DisasContext *s, uint32_t insn) { if (extract32(insn, 28, 1) == 1 && extract32(insn, 30, 1) == 0) { - disas_data_proc_fp(s, insn); + unallocated_encoding(s); /* in decodetree */ } else { /* SIMD, including crypto */ disas_data_proc_simd(s, insn); -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-36-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 11 +++ target/arm/tcg/translate-a64.c | 123 +++++++++++++++++++++------------ 2 files changed, 89 insertions(+), 45 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ @rr_h ........ ... ..... ...... rn:5 rd:5 &rr_e esz=1 @rr_s ........ ... ..... ...... rn:5 rd:5 &rr_e esz=2 @rr_d ........ ... ..... ...... rn:5 rd:5 &rr_e esz=3 +@rr_e ........ esz:2 . ..... ...... rn:5 rd:5 &rr_e @rr_sd ........ ... ..... ...... rn:5 rd:5 &rr_e esz=%esz_sd @rr_hsd ........ ... ..... ...... rn:5 rd:5 &rr_e esz=%esz_hsd @@ -XXX,XX +XXX,XX @@ UQRSHRN_si 0111 11110 .... ... 10011 1 ..... ..... @shri_s SQRSHRUN_si 0111 11110 .... ... 10001 1 ..... ..... @shri_b SQRSHRUN_si 0111 11110 .... ... 10001 1 ..... ..... @shri_h SQRSHRUN_si 0111 11110 .... ... 10001 1 ..... ..... @shri_s + +# Advanced SIMD scalar two-register miscellaneous + +SQABS_s 0101 1110 ..1 00000 01111 0 ..... ..... @rr_e +SQNEG_s 0111 1110 ..1 00000 01111 0 ..... ..... @rr_e + +# Advanced SIMD two-register miscellaneous + +SQABS_v 0.00 1110 ..1 00000 01111 0 ..... ..... @qrr_e +SQNEG_v 0.10 1110 ..1 00000 01111 0 ..... ..... @qrr_e diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static bool trans_FMOV_xu(DisasContext *s, arg_rr *a) return true; } +typedef struct ENVScalar1 { + NeonGenOneOpEnvFn *gen_bhs[3]; + NeonGenOne64OpEnvFn *gen_d; +} ENVScalar1; + +static bool do_env_scalar1(DisasContext *s, arg_rr_e *a, const ENVScalar1 *f) +{ + if (!fp_access_check(s)) { + return true; + } + if (a->esz == MO_64) { + TCGv_i64 t = read_fp_dreg(s, a->rn); + f->gen_d(t, tcg_env, t); + write_fp_dreg(s, a->rd, t); + } else { + TCGv_i32 t = tcg_temp_new_i32(); + + read_vec_element_i32(s, t, a->rn, 0, a->esz); + f->gen_bhs[a->esz](t, tcg_env, t); + write_fp_sreg(s, a->rd, t); + } + return true; +} + +static bool do_env_vector1(DisasContext *s, arg_qrr_e *a, const ENVScalar1 *f) +{ + if (a->esz == MO_64 && !a->q) { + return false; + } + if (!fp_access_check(s)) { + return true; + } + if (a->esz == MO_64) { + TCGv_i64 t = tcg_temp_new_i64(); + + for (int i = 0; i < 2; ++i) { + read_vec_element(s, t, a->rn, i, MO_64); + f->gen_d(t, tcg_env, t); + write_vec_element(s, t, a->rd, i, MO_64); + } + } else { + TCGv_i32 t = tcg_temp_new_i32(); + int n = (a->q ? 16 : 8) >> a->esz; + + for (int i = 0; i < n; ++i) { + read_vec_element_i32(s, t, a->rn, i, a->esz); + f->gen_bhs[a->esz](t, tcg_env, t); + write_vec_element_i32(s, t, a->rd, i, a->esz); + } + } + clear_vec_high(s, a->q, a->rd); + return true; +} + +static const ENVScalar1 f_scalar_sqabs = { + { gen_helper_neon_qabs_s8, + gen_helper_neon_qabs_s16, + gen_helper_neon_qabs_s32 }, + gen_helper_neon_qabs_s64, +}; +TRANS(SQABS_s, do_env_scalar1, a, &f_scalar_sqabs) +TRANS(SQABS_v, do_env_vector1, a, &f_scalar_sqabs) + +static const ENVScalar1 f_scalar_sqneg = { + { gen_helper_neon_qneg_s8, + gen_helper_neon_qneg_s16, + gen_helper_neon_qneg_s32 }, + gen_helper_neon_qneg_s64, +}; +TRANS(SQNEG_s, do_env_scalar1, a, &f_scalar_sqneg) +TRANS(SQNEG_v, do_env_vector1, a, &f_scalar_sqneg) + /* Common vector code for handling integer to FP conversion */ static void handle_simd_intfp_conv(DisasContext *s, int rd, int rn, int elements, int is_signed, @@ -XXX,XX +XXX,XX @@ static void handle_2misc_64(DisasContext *s, int opcode, bool u, */ tcg_gen_not_i64(tcg_rd, tcg_rn); break; - case 0x7: /* SQABS, SQNEG */ - if (u) { - gen_helper_neon_qneg_s64(tcg_rd, tcg_env, tcg_rn); - } else { - gen_helper_neon_qabs_s64(tcg_rd, tcg_env, tcg_rn); - } - break; case 0xa: /* CMLT */ cond = TCG_COND_LT; do_cmop: @@ -XXX,XX +XXX,XX @@ static void handle_2misc_64(DisasContext *s, int opcode, bool u, gen_helper_frint64_d(tcg_rd, tcg_rn, tcg_fpstatus); break; default: + case 0x7: /* SQABS, SQNEG */ g_assert_not_reached(); } } @@ -XXX,XX +XXX,XX @@ static void disas_simd_scalar_two_reg_misc(DisasContext *s, uint32_t insn) TCGv_ptr tcg_fpstatus; switch (opcode) { - case 0x7: /* SQABS / SQNEG */ - break; case 0xa: /* CMLT */ if (u) { unallocated_encoding(s); @@ -XXX,XX +XXX,XX @@ static void disas_simd_scalar_two_reg_misc(DisasContext *s, uint32_t insn) break; default: case 0x3: /* USQADD / SUQADD */ + case 0x7: /* SQABS / SQNEG */ unallocated_encoding(s); return; } @@ -XXX,XX +XXX,XX @@ static void disas_simd_scalar_two_reg_misc(DisasContext *s, uint32_t insn) read_vec_element_i32(s, tcg_rn, rn, 0, size); switch (opcode) { - case 0x7: /* SQABS, SQNEG */ - { - NeonGenOneOpEnvFn *genfn; - static NeonGenOneOpEnvFn * const fns[3][2] = { - { gen_helper_neon_qabs_s8, gen_helper_neon_qneg_s8 }, - { gen_helper_neon_qabs_s16, gen_helper_neon_qneg_s16 }, - { gen_helper_neon_qabs_s32, gen_helper_neon_qneg_s32 }, - }; - genfn = fns[size][u]; - genfn(tcg_rd, tcg_env, tcg_rn); - break; - } case 0x1a: /* FCVTNS */ case 0x1b: /* FCVTMS */ case 0x1c: /* FCVTAS */ @@ -XXX,XX +XXX,XX @@ static void disas_simd_scalar_two_reg_misc(DisasContext *s, uint32_t insn) tcg_fpstatus); break; default: + case 0x7: /* SQABS, SQNEG */ g_assert_not_reached(); } @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) return; } break; - case 0x7: /* SQABS, SQNEG */ - if (size == 3 && !is_q) { - unallocated_encoding(s); - return; - } - break; case 0xc ... 0xf: case 0x16 ... 0x1f: { @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) } default: case 0x3: /* SUQADD, USQADD */ + case 0x7: /* SQABS, SQNEG */ unallocated_encoding(s); return; } @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) tcg_gen_clrsb_i32(tcg_res, tcg_op); } break; - case 0x7: /* SQABS, SQNEG */ - if (u) { - gen_helper_neon_qneg_s32(tcg_res, tcg_env, tcg_op); - } else { - gen_helper_neon_qabs_s32(tcg_res, tcg_env, tcg_op); - } - break; case 0x2f: /* FABS */ gen_vfp_abss(tcg_res, tcg_op); break; @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) gen_helper_frint64_s(tcg_res, tcg_op, tcg_fpstatus); break; default: + case 0x7: /* SQABS, SQNEG */ g_assert_not_reached(); } } else { @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) gen_helper_neon_cnt_u8(tcg_res, tcg_op); } break; - case 0x7: /* SQABS, SQNEG */ - { - NeonGenOneOpEnvFn *genfn; - static NeonGenOneOpEnvFn * const fns[2][2] = { - { gen_helper_neon_qabs_s8, gen_helper_neon_qneg_s8 }, - { gen_helper_neon_qabs_s16, gen_helper_neon_qneg_s16 }, - }; - genfn = fns[size][u]; - genfn(tcg_res, tcg_env, tcg_op); - break; - } case 0x4: /* CLS, CLZ */ if (u) { if (size == 0) { @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) } break; default: + case 0x7: /* SQABS, SQNEG */ g_assert_not_reached(); } } -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-37-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 4 +++ target/arm/tcg/translate-a64.c | 46 +++++++++++++++++++++++----------- 2 files changed, 35 insertions(+), 15 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ SQRSHRUN_si 0111 11110 .... ... 10001 1 ..... ..... @shri_s SQABS_s 0101 1110 ..1 00000 01111 0 ..... ..... @rr_e SQNEG_s 0111 1110 ..1 00000 01111 0 ..... ..... @rr_e +ABS_s 0101 1110 111 00000 10111 0 ..... ..... @rr +NEG_s 0111 1110 111 00000 10111 0 ..... ..... @rr # Advanced SIMD two-register miscellaneous SQABS_v 0.00 1110 ..1 00000 01111 0 ..... ..... @qrr_e SQNEG_v 0.10 1110 ..1 00000 01111 0 ..... ..... @qrr_e +ABS_v 0.00 1110 ..1 00000 10111 0 ..... ..... @qrr_e +NEG_v 0.10 1110 ..1 00000 10111 0 ..... ..... @qrr_e diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static const ENVScalar1 f_scalar_sqneg = { TRANS(SQNEG_s, do_env_scalar1, a, &f_scalar_sqneg) TRANS(SQNEG_v, do_env_vector1, a, &f_scalar_sqneg) +static bool do_scalar1_d(DisasContext *s, arg_rr *a, ArithOneOp *f) +{ + if (fp_access_check(s)) { + TCGv_i64 t = read_fp_dreg(s, a->rn); + f(t, t); + write_fp_dreg(s, a->rd, t); + } + return true; +} + +TRANS(ABS_s, do_scalar1_d, a, tcg_gen_abs_i64) +TRANS(NEG_s, do_scalar1_d, a, tcg_gen_neg_i64) + +static bool do_gvec_fn2(DisasContext *s, arg_qrr_e *a, GVecGen2Fn *fn) +{ + if (!a->q && a->esz == MO_64) { + return false; + } + if (fp_access_check(s)) { + gen_gvec_fn2(s, a->q, a->rd, a->rn, fn, a->esz); + } + return true; +} + +TRANS(ABS_v, do_gvec_fn2, a, tcg_gen_gvec_abs) +TRANS(NEG_v, do_gvec_fn2, a, tcg_gen_gvec_neg) + /* Common vector code for handling integer to FP conversion */ static void handle_simd_intfp_conv(DisasContext *s, int rd, int rn, int elements, int is_signed, @@ -XXX,XX +XXX,XX @@ static void handle_2misc_64(DisasContext *s, int opcode, bool u, case 0x9: /* CMEQ, CMLE */ cond = u ? TCG_COND_LE : TCG_COND_EQ; goto do_cmop; - case 0xb: /* ABS, NEG */ - if (u) { - tcg_gen_neg_i64(tcg_rd, tcg_rn); - } else { - tcg_gen_abs_i64(tcg_rd, tcg_rn); - } - break; case 0x2f: /* FABS */ gen_vfp_absd(tcg_rd, tcg_rn); break; @@ -XXX,XX +XXX,XX @@ static void handle_2misc_64(DisasContext *s, int opcode, bool u, break; default: case 0x7: /* SQABS, SQNEG */ + case 0xb: /* ABS, NEG */ g_assert_not_reached(); } } @@ -XXX,XX +XXX,XX @@ static void disas_simd_scalar_two_reg_misc(DisasContext *s, uint32_t insn) /* fall through */ case 0x8: /* CMGT, CMGE */ case 0x9: /* CMEQ, CMLE */ - case 0xb: /* ABS, NEG */ if (size != 3) { unallocated_encoding(s); return; @@ -XXX,XX +XXX,XX @@ static void disas_simd_scalar_two_reg_misc(DisasContext *s, uint32_t insn) default: case 0x3: /* USQADD / SUQADD */ case 0x7: /* SQABS / SQNEG */ + case 0xb: /* ABS, NEG */ unallocated_encoding(s); return; } @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) /* fall through */ case 0x8: /* CMGT, CMGE */ case 0x9: /* CMEQ, CMLE */ - case 0xb: /* ABS, NEG */ if (size == 3 && !is_q) { unallocated_encoding(s); return; @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) default: case 0x3: /* SUQADD, USQADD */ case 0x7: /* SQABS, SQNEG */ + case 0xb: /* ABS, NEG */ unallocated_encoding(s); return; } @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) gen_gvec_fn2(s, is_q, rd, rn, gen_gvec_clt0, size); return; case 0xb: - if (u) { /* ABS, NEG */ - gen_gvec_fn2(s, is_q, rd, rn, tcg_gen_gvec_neg, size); - } else { - gen_gvec_fn2(s, is_q, rd, rn, tcg_gen_gvec_abs, size); - } - return; + g_assert_not_reached(); } if (size == 3) { -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Add gvec interfaces for CLS and CLZ operations. Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-38-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/translate.h | 5 +++++ target/arm/tcg/gengvec.c | 35 +++++++++++++++++++++++++++++++++ target/arm/tcg/translate-a64.c | 29 +++++++-------------------- target/arm/tcg/translate-neon.c | 29 ++------------------------- 4 files changed, 49 insertions(+), 49 deletions(-) diff --git a/target/arm/tcg/translate.h b/target/arm/tcg/translate.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate.h +++ b/target/arm/tcg/translate.h @@ -XXX,XX +XXX,XX @@ void gen_gvec_umaxp(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, void gen_gvec_uminp(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz); +void gen_gvec_cls(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, + uint32_t opr_sz, uint32_t max_sz); +void gen_gvec_clz(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, + uint32_t opr_sz, uint32_t max_sz); + /* * Forward to the isar_feature_* tests given a DisasContext pointer. */ diff --git a/target/arm/tcg/gengvec.c b/target/arm/tcg/gengvec.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/gengvec.c +++ b/target/arm/tcg/gengvec.c @@ -XXX,XX +XXX,XX @@ void gen_gvec_urhadd(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, assert(vece <= MO_32); tcg_gen_gvec_3(rd_ofs, rn_ofs, rm_ofs, opr_sz, max_sz, &g[vece]); } + +void gen_gvec_cls(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, + uint32_t opr_sz, uint32_t max_sz) +{ + static const GVecGen2 g[] = { + { .fni4 = gen_helper_neon_cls_s8, + .vece = MO_8 }, + { .fni4 = gen_helper_neon_cls_s16, + .vece = MO_16 }, + { .fni4 = tcg_gen_clrsb_i32, + .vece = MO_32 }, + }; + assert(vece <= MO_32); + tcg_gen_gvec_2(rd_ofs, rn_ofs, opr_sz, max_sz, &g[vece]); +} + +static void gen_clz32_i32(TCGv_i32 d, TCGv_i32 n) +{ + tcg_gen_clzi_i32(d, n, 32); +} + +void gen_gvec_clz(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, + uint32_t opr_sz, uint32_t max_sz) +{ + static const GVecGen2 g[] = { + { .fni4 = gen_helper_neon_clz_u8, + .vece = MO_8 }, + { .fni4 = gen_helper_neon_clz_u16, + .vece = MO_16 }, + { .fni4 = gen_clz32_i32, + .vece = MO_32 }, + }; + assert(vece <= MO_32); + tcg_gen_gvec_2(rd_ofs, rn_ofs, opr_sz, max_sz, &g[vece]); +} diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) } switch (opcode) { + case 0x4: /* CLZ, CLS */ + if (u) { + gen_gvec_fn2(s, is_q, rd, rn, gen_gvec_clz, size); + } else { + gen_gvec_fn2(s, is_q, rd, rn, gen_gvec_cls, size); + } + return; case 0x5: if (u && size == 0) { /* NOT */ gen_gvec_fn2(s, is_q, rd, rn, tcg_gen_gvec_not, 0); @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) if (size == 2) { /* Special cases for 32 bit elements */ switch (opcode) { - case 0x4: /* CLS */ - if (u) { - tcg_gen_clzi_i32(tcg_res, tcg_op, 32); - } else { - tcg_gen_clrsb_i32(tcg_res, tcg_op); - } - break; case 0x2f: /* FABS */ gen_vfp_abss(tcg_res, tcg_op); break; @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) gen_helper_neon_cnt_u8(tcg_res, tcg_op); } break; - case 0x4: /* CLS, CLZ */ - if (u) { - if (size == 0) { - gen_helper_neon_clz_u8(tcg_res, tcg_op); - } else { - gen_helper_neon_clz_u16(tcg_res, tcg_op); - } - } else { - if (size == 0) { - gen_helper_neon_cls_s8(tcg_res, tcg_op); - } else { - gen_helper_neon_cls_s16(tcg_res, tcg_op); - } - } - break; default: case 0x7: /* SQABS, SQNEG */ g_assert_not_reached(); diff --git a/target/arm/tcg/translate-neon.c b/target/arm/tcg/translate-neon.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-neon.c +++ b/target/arm/tcg/translate-neon.c @@ -XXX,XX +XXX,XX @@ DO_2MISC_VEC(VCGT0, gen_gvec_cgt0) DO_2MISC_VEC(VCLE0, gen_gvec_cle0) DO_2MISC_VEC(VCGE0, gen_gvec_cge0) DO_2MISC_VEC(VCLT0, gen_gvec_clt0) +DO_2MISC_VEC(VCLS, gen_gvec_cls) +DO_2MISC_VEC(VCLZ, gen_gvec_clz) static bool trans_VMVN(DisasContext *s, arg_2misc *a) { @@ -XXX,XX +XXX,XX @@ static bool trans_VREV16(DisasContext *s, arg_2misc *a) return do_2misc(s, a, gen_rev16); } -static bool trans_VCLS(DisasContext *s, arg_2misc *a) -{ - static NeonGenOneOpFn * const fn[] = { - gen_helper_neon_cls_s8, - gen_helper_neon_cls_s16, - gen_helper_neon_cls_s32, - NULL, - }; - return do_2misc(s, a, fn[a->size]); -} - -static void do_VCLZ_32(TCGv_i32 rd, TCGv_i32 rm) -{ - tcg_gen_clzi_i32(rd, rm, 32); -} - -static bool trans_VCLZ(DisasContext *s, arg_2misc *a) -{ - static NeonGenOneOpFn * const fn[] = { - gen_helper_neon_clz_u8, - gen_helper_neon_clz_u16, - do_VCLZ_32, - NULL, - }; - return do_2misc(s, a, fn[a->size]); -} - static bool trans_VCNT(DisasContext *s, arg_2misc *a) { if (a->size != 0) { -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-39-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 2 ++ target/arm/tcg/translate-a64.c | 37 ++++++++++++++++------------------ 2 files changed, 19 insertions(+), 20 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ SQABS_v 0.00 1110 ..1 00000 01111 0 ..... ..... @qrr_e SQNEG_v 0.10 1110 ..1 00000 01111 0 ..... ..... @qrr_e ABS_v 0.00 1110 ..1 00000 10111 0 ..... ..... @qrr_e NEG_v 0.10 1110 ..1 00000 10111 0 ..... ..... @qrr_e +CLS_v 0.00 1110 ..1 00000 01001 0 ..... ..... @qrr_e +CLZ_v 0.10 1110 ..1 00000 01001 0 ..... ..... @qrr_e diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static bool do_gvec_fn2(DisasContext *s, arg_qrr_e *a, GVecGen2Fn *fn) TRANS(ABS_v, do_gvec_fn2, a, tcg_gen_gvec_abs) TRANS(NEG_v, do_gvec_fn2, a, tcg_gen_gvec_neg) +static bool do_gvec_fn2_bhs(DisasContext *s, arg_qrr_e *a, GVecGen2Fn *fn) +{ + if (a->esz == MO_64) { + return false; + } + if (fp_access_check(s)) { + gen_gvec_fn2(s, a->q, a->rd, a->rn, fn, a->esz); + } + return true; +} + +TRANS(CLS_v, do_gvec_fn2_bhs, a, gen_gvec_cls) +TRANS(CLZ_v, do_gvec_fn2_bhs, a, gen_gvec_clz) + /* Common vector code for handling integer to FP conversion */ static void handle_simd_intfp_conv(DisasContext *s, int rd, int rn, int elements, int is_signed, @@ -XXX,XX +XXX,XX @@ static void handle_2misc_64(DisasContext *s, int opcode, bool u, TCGCond cond; switch (opcode) { - case 0x4: /* CLS, CLZ */ - if (u) { - tcg_gen_clzi_i64(tcg_rd, tcg_rn, 64); - } else { - tcg_gen_clrsb_i64(tcg_rd, tcg_rn); - } - break; case 0x5: /* NOT */ /* This opcode is shared with CNT and RBIT but we have earlier * enforced that size == 3 if and only if this is the NOT insn. @@ -XXX,XX +XXX,XX @@ static void handle_2misc_64(DisasContext *s, int opcode, bool u, gen_helper_frint64_d(tcg_rd, tcg_rn, tcg_fpstatus); break; default: + case 0x4: /* CLS, CLZ */ case 0x7: /* SQABS, SQNEG */ case 0xb: /* ABS, NEG */ g_assert_not_reached(); @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) handle_2misc_narrow(s, false, opcode, u, is_q, size, rn, rd); return; - case 0x4: /* CLS, CLZ */ - if (size == 3) { - unallocated_encoding(s); - return; - } - break; case 0x2: /* SADDLP, UADDLP */ case 0x6: /* SADALP, UADALP */ if (size == 3) { @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) } default: case 0x3: /* SUQADD, USQADD */ + case 0x4: /* CLS, CLZ */ case 0x7: /* SQABS, SQNEG */ case 0xb: /* ABS, NEG */ unallocated_encoding(s); @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) } switch (opcode) { - case 0x4: /* CLZ, CLS */ - if (u) { - gen_gvec_fn2(s, is_q, rd, rn, gen_gvec_clz, size); - } else { - gen_gvec_fn2(s, is_q, rd, rn, gen_gvec_cls, size); - } - return; case 0x5: if (u && size == 0) { /* NOT */ gen_gvec_fn2(s, is_q, rd, rn, tcg_gen_gvec_not, 0); @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) case 0xa: /* CMLT */ gen_gvec_fn2(s, is_q, rd, rn, gen_gvec_clt0, size); return; + case 0x4: /* CLZ, CLS */ case 0xb: g_assert_not_reached(); } -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Add gvec interfaces for CNT and RBIT operations. Use ctpop8 for CNT and revbit+bswap for RBIT. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-40-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/helper.h | 4 ++-- target/arm/tcg/translate.h | 4 ++++ target/arm/tcg/gengvec.c | 16 ++++++++++++++++ target/arm/tcg/neon_helper.c | 21 --------------------- target/arm/tcg/translate-a64.c | 32 +++++++++----------------------- target/arm/tcg/translate-neon.c | 16 ++++++++-------- target/arm/tcg/vec_helper.c | 24 ++++++++++++++++++++++++ 7 files changed, 63 insertions(+), 54 deletions(-) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_1(neon_clz_u16, i32, i32) DEF_HELPER_1(neon_cls_s8, i32, i32) DEF_HELPER_1(neon_cls_s16, i32, i32) DEF_HELPER_1(neon_cls_s32, i32, i32) -DEF_HELPER_1(neon_cnt_u8, i32, i32) -DEF_HELPER_FLAGS_1(neon_rbit_u8, TCG_CALL_NO_RWG_SE, i32, i32) +DEF_HELPER_FLAGS_3(gvec_cnt_b, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(gvec_rbit_b, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_3(neon_qdmulh_s16, i32, env, i32, i32) DEF_HELPER_3(neon_qrdmulh_s16, i32, env, i32, i32) diff --git a/target/arm/tcg/translate.h b/target/arm/tcg/translate.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate.h +++ b/target/arm/tcg/translate.h @@ -XXX,XX +XXX,XX @@ void gen_gvec_cls(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, uint32_t opr_sz, uint32_t max_sz); void gen_gvec_clz(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, uint32_t opr_sz, uint32_t max_sz); +void gen_gvec_cnt(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, + uint32_t opr_sz, uint32_t max_sz); +void gen_gvec_rbit(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, + uint32_t opr_sz, uint32_t max_sz); /* * Forward to the isar_feature_* tests given a DisasContext pointer. diff --git a/target/arm/tcg/gengvec.c b/target/arm/tcg/gengvec.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/gengvec.c +++ b/target/arm/tcg/gengvec.c @@ -XXX,XX +XXX,XX @@ void gen_gvec_clz(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, assert(vece <= MO_32); tcg_gen_gvec_2(rd_ofs, rn_ofs, opr_sz, max_sz, &g[vece]); } + +void gen_gvec_cnt(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, + uint32_t opr_sz, uint32_t max_sz) +{ + assert(vece == MO_8); + tcg_gen_gvec_2_ool(rd_ofs, rn_ofs, opr_sz, max_sz, 0, + gen_helper_gvec_cnt_b); +} + +void gen_gvec_rbit(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, + uint32_t opr_sz, uint32_t max_sz) +{ + assert(vece == MO_8); + tcg_gen_gvec_2_ool(rd_ofs, rn_ofs, opr_sz, max_sz, 0, + gen_helper_gvec_rbit_b); +} diff --git a/target/arm/tcg/neon_helper.c b/target/arm/tcg/neon_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/neon_helper.c +++ b/target/arm/tcg/neon_helper.c @@ -XXX,XX +XXX,XX @@ uint32_t HELPER(neon_cls_s32)(uint32_t x) return count - 1; } -/* Bit count. */ -uint32_t HELPER(neon_cnt_u8)(uint32_t x) -{ - x = (x & 0x55555555) + ((x >> 1) & 0x55555555); - x = (x & 0x33333333) + ((x >> 2) & 0x33333333); - x = (x & 0x0f0f0f0f) + ((x >> 4) & 0x0f0f0f0f); - return x; -} - -/* Reverse bits in each 8 bit word */ -uint32_t HELPER(neon_rbit_u8)(uint32_t x) -{ - x = ((x & 0xf0f0f0f0) >> 4) - | ((x & 0x0f0f0f0f) << 4); - x = ((x & 0x88888888) >> 3) - | ((x & 0x44444444) >> 1) - | ((x & 0x22222222) << 1) - | ((x & 0x11111111) << 3); - return x; -} - #define NEON_QDMULH16(dest, src1, src2, round) do { \ uint32_t tmp = (int32_t)(int16_t) src1 * (int16_t) src2; \ if ((tmp ^ (tmp << 1)) & SIGNBIT) { \ diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) } switch (opcode) { - case 0x5: - if (u && size == 0) { /* NOT */ + case 0x5: /* CNT, NOT, RBIT */ + if (!u) { + gen_gvec_fn2(s, is_q, rd, rn, gen_gvec_cnt, 0); + } else if (size) { + gen_gvec_fn2(s, is_q, rd, rn, gen_gvec_rbit, 0); + } else { gen_gvec_fn2(s, is_q, rd, rn, tcg_gen_gvec_not, 0); - return; } - break; + return; case 0x8: /* CMGT, CMGE */ if (u) { gen_gvec_fn2(s, is_q, rd, rn, gen_gvec_cge0, size); @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) } else { int pass; + assert(size == 2); for (pass = 0; pass < (is_q ? 4 : 2); pass++) { TCGv_i32 tcg_op = tcg_temp_new_i32(); TCGv_i32 tcg_res = tcg_temp_new_i32(); read_vec_element_i32(s, tcg_op, rn, pass, MO_32); - if (size == 2) { + { /* Special cases for 32 bit elements */ switch (opcode) { case 0x2f: /* FABS */ @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) case 0x7: /* SQABS, SQNEG */ g_assert_not_reached(); } - } else { - /* Use helpers for 8 and 16 bit elements */ - switch (opcode) { - case 0x5: /* CNT, RBIT */ - /* For these two insns size is part of the opcode specifier - * (handled earlier); they always operate on byte elements. - */ - if (u) { - gen_helper_neon_rbit_u8(tcg_res, tcg_op); - } else { - gen_helper_neon_cnt_u8(tcg_res, tcg_op); - } - break; - default: - case 0x7: /* SQABS, SQNEG */ - g_assert_not_reached(); - } } - write_vec_element_i32(s, tcg_res, rd, pass, MO_32); } } diff --git a/target/arm/tcg/translate-neon.c b/target/arm/tcg/translate-neon.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-neon.c +++ b/target/arm/tcg/translate-neon.c @@ -XXX,XX +XXX,XX @@ static bool trans_VMVN(DisasContext *s, arg_2misc *a) return do_2misc_vec(s, a, tcg_gen_gvec_not); } +static bool trans_VCNT(DisasContext *s, arg_2misc *a) +{ + if (a->size != 0) { + return false; + } + return do_2misc_vec(s, a, gen_gvec_cnt); +} + #define WRAP_2M_3_OOL_FN(WRAPNAME, FUNC, DATA) \ static void WRAPNAME(unsigned vece, uint32_t rd_ofs, \ uint32_t rm_ofs, uint32_t oprsz, \ @@ -XXX,XX +XXX,XX @@ static bool trans_VREV16(DisasContext *s, arg_2misc *a) return do_2misc(s, a, gen_rev16); } -static bool trans_VCNT(DisasContext *s, arg_2misc *a) -{ - if (a->size != 0) { - return false; - } - return do_2misc(s, a, gen_helper_neon_cnt_u8); -} - static void gen_VABS_F(unsigned vece, uint32_t rd_ofs, uint32_t rm_ofs, uint32_t oprsz, uint32_t maxsz) { diff --git a/target/arm/tcg/vec_helper.c b/target/arm/tcg/vec_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/vec_helper.c +++ b/target/arm/tcg/vec_helper.c @@ -XXX,XX +XXX,XX @@ DO_CLAMP(gvec_uclamp_b, uint8_t) DO_CLAMP(gvec_uclamp_h, uint16_t) DO_CLAMP(gvec_uclamp_s, uint32_t) DO_CLAMP(gvec_uclamp_d, uint64_t) + +/* Bit count in each 8-bit word. */ +void HELPER(gvec_cnt_b)(void *vd, void *vn, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + uint8_t *d = vd, *n = vn; + + for (i = 0; i < opr_sz; ++i) { + d[i] = ctpop8(n[i]); + } + clear_tail(d, opr_sz, simd_maxsz(desc)); +} + +/* Reverse bits in each 8 bit word */ +void HELPER(gvec_rbit_b)(void *vd, void *vn, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + uint64_t *d = vd, *n = vn; + + for (i = 0; i < opr_sz / 8; ++i) { + d[i] = revbit64(bswap64(n[i])); + } + clear_tail(d, opr_sz, simd_maxsz(desc)); +} -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-41-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 4 ++++ target/arm/tcg/translate-a64.c | 34 ++++++---------------------------- 2 files changed, 10 insertions(+), 28 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ @rrr_q1e3 ........ ... rm:5 ...... rn:5 rd:5 &qrrr_e q=1 esz=3 @rrrr_q1e3 ........ ... rm:5 . ra:5 rn:5 rd:5 &qrrrr_e q=1 esz=3 +@qrr_b . q:1 ...... .. ...... ...... rn:5 rd:5 &qrr_e esz=0 @qrr_h . q:1 ...... .. ...... ...... rn:5 rd:5 &qrr_e esz=1 @qrr_e . q:1 ...... esz:2 ...... ...... rn:5 rd:5 &qrr_e @@ -XXX,XX +XXX,XX @@ ABS_v 0.00 1110 ..1 00000 10111 0 ..... ..... @qrr_e NEG_v 0.10 1110 ..1 00000 10111 0 ..... ..... @qrr_e CLS_v 0.00 1110 ..1 00000 01001 0 ..... ..... @qrr_e CLZ_v 0.10 1110 ..1 00000 01001 0 ..... ..... @qrr_e +CNT_v 0.00 1110 001 00000 01011 0 ..... ..... @qrr_b +NOT_v 0.10 1110 001 00000 01011 0 ..... ..... @qrr_b +RBIT_v 0.10 1110 011 00000 01011 0 ..... ..... @qrr_b diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static bool do_gvec_fn2(DisasContext *s, arg_qrr_e *a, GVecGen2Fn *fn) TRANS(ABS_v, do_gvec_fn2, a, tcg_gen_gvec_abs) TRANS(NEG_v, do_gvec_fn2, a, tcg_gen_gvec_neg) +TRANS(NOT_v, do_gvec_fn2, a, tcg_gen_gvec_not) +TRANS(CNT_v, do_gvec_fn2, a, gen_gvec_cnt) +TRANS(RBIT_v, do_gvec_fn2, a, gen_gvec_rbit) static bool do_gvec_fn2_bhs(DisasContext *s, arg_qrr_e *a, GVecGen2Fn *fn) { @@ -XXX,XX +XXX,XX @@ static void handle_2misc_64(DisasContext *s, int opcode, bool u, TCGCond cond; switch (opcode) { - case 0x5: /* NOT */ - /* This opcode is shared with CNT and RBIT but we have earlier - * enforced that size == 3 if and only if this is the NOT insn. - */ - tcg_gen_not_i64(tcg_rd, tcg_rn); - break; case 0xa: /* CMLT */ cond = TCG_COND_LT; do_cmop: @@ -XXX,XX +XXX,XX @@ static void handle_2misc_64(DisasContext *s, int opcode, bool u, break; default: case 0x4: /* CLS, CLZ */ + case 0x5: /* NOT */ case 0x7: /* SQABS, SQNEG */ case 0xb: /* ABS, NEG */ g_assert_not_reached(); @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) case 0x1: /* REV16 */ handle_rev(s, opcode, u, is_q, size, rn, rd); return; - case 0x5: /* CNT, NOT, RBIT */ - if (u && size == 0) { - /* NOT */ - break; - } else if (u && size == 1) { - /* RBIT */ - break; - } else if (!u && size == 0) { - /* CNT */ - break; - } - unallocated_encoding(s); - return; case 0x12: /* XTN, XTN2, SQXTUN, SQXTUN2 */ case 0x14: /* SQXTN, SQXTN2, UQXTN, UQXTN2 */ if (size == 3) { @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) default: case 0x3: /* SUQADD, USQADD */ case 0x4: /* CLS, CLZ */ + case 0x5: /* CNT, NOT, RBIT */ case 0x7: /* SQABS, SQNEG */ case 0xb: /* ABS, NEG */ unallocated_encoding(s); @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) } switch (opcode) { - case 0x5: /* CNT, NOT, RBIT */ - if (!u) { - gen_gvec_fn2(s, is_q, rd, rn, gen_gvec_cnt, 0); - } else if (size) { - gen_gvec_fn2(s, is_q, rd, rn, gen_gvec_rbit, 0); - } else { - gen_gvec_fn2(s, is_q, rd, rn, tcg_gen_gvec_not, 0); - } - return; case 0x8: /* CMGT, CMGE */ if (u) { gen_gvec_fn2(s, is_q, rd, rn, gen_gvec_cge0, size); @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) gen_gvec_fn2(s, is_q, rd, rn, gen_gvec_clt0, size); return; case 0x4: /* CLZ, CLS */ + case 0x5: /* CNT, NOT, RBIT */ case 0xb: g_assert_not_reached(); } -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-42-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 10 ++++ target/arm/tcg/translate-a64.c | 94 +++++++++++----------------------- 2 files changed, 40 insertions(+), 64 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ SQABS_s 0101 1110 ..1 00000 01111 0 ..... ..... @rr_e SQNEG_s 0111 1110 ..1 00000 01111 0 ..... ..... @rr_e ABS_s 0101 1110 111 00000 10111 0 ..... ..... @rr NEG_s 0111 1110 111 00000 10111 0 ..... ..... @rr +CMGT0_s 0101 1110 111 00000 10001 0 ..... ..... @rr +CMGE0_s 0111 1110 111 00000 10001 0 ..... ..... @rr +CMEQ0_s 0101 1110 111 00000 10011 0 ..... ..... @rr +CMLE0_s 0111 1110 111 00000 10011 0 ..... ..... @rr +CMLT0_s 0101 1110 111 00000 10101 0 ..... ..... @rr # Advanced SIMD two-register miscellaneous @@ -XXX,XX +XXX,XX @@ CLZ_v 0.10 1110 ..1 00000 01001 0 ..... ..... @qrr_e CNT_v 0.00 1110 001 00000 01011 0 ..... ..... @qrr_b NOT_v 0.10 1110 001 00000 01011 0 ..... ..... @qrr_b RBIT_v 0.10 1110 011 00000 01011 0 ..... ..... @qrr_b +CMGT0_v 0.00 1110 ..1 00000 10001 0 ..... ..... @qrr_e +CMGE0_v 0.10 1110 ..1 00000 10001 0 ..... ..... @qrr_e +CMEQ0_v 0.00 1110 ..1 00000 10011 0 ..... ..... @qrr_e +CMLE0_v 0.10 1110 ..1 00000 10011 0 ..... ..... @qrr_e +CMLT0_v 0.00 1110 ..1 00000 10101 0 ..... ..... @qrr_e diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static bool do_scalar1_d(DisasContext *s, arg_rr *a, ArithOneOp *f) TRANS(ABS_s, do_scalar1_d, a, tcg_gen_abs_i64) TRANS(NEG_s, do_scalar1_d, a, tcg_gen_neg_i64) +static bool do_cmop0_d(DisasContext *s, arg_rr *a, TCGCond cond) +{ + if (fp_access_check(s)) { + TCGv_i64 t = read_fp_dreg(s, a->rn); + tcg_gen_negsetcond_i64(cond, t, t, tcg_constant_i64(0)); + write_fp_dreg(s, a->rd, t); + } + return true; +} + +TRANS(CMGT0_s, do_cmop0_d, a, TCG_COND_GT) +TRANS(CMGE0_s, do_cmop0_d, a, TCG_COND_GE) +TRANS(CMLE0_s, do_cmop0_d, a, TCG_COND_LE) +TRANS(CMLT0_s, do_cmop0_d, a, TCG_COND_LT) +TRANS(CMEQ0_s, do_cmop0_d, a, TCG_COND_EQ) + static bool do_gvec_fn2(DisasContext *s, arg_qrr_e *a, GVecGen2Fn *fn) { if (!a->q && a->esz == MO_64) { @@ -XXX,XX +XXX,XX @@ TRANS(NEG_v, do_gvec_fn2, a, tcg_gen_gvec_neg) TRANS(NOT_v, do_gvec_fn2, a, tcg_gen_gvec_not) TRANS(CNT_v, do_gvec_fn2, a, gen_gvec_cnt) TRANS(RBIT_v, do_gvec_fn2, a, gen_gvec_rbit) +TRANS(CMGT0_v, do_gvec_fn2, a, gen_gvec_cgt0) +TRANS(CMGE0_v, do_gvec_fn2, a, gen_gvec_cge0) +TRANS(CMLT0_v, do_gvec_fn2, a, gen_gvec_clt0) +TRANS(CMLE0_v, do_gvec_fn2, a, gen_gvec_cle0) +TRANS(CMEQ0_v, do_gvec_fn2, a, gen_gvec_ceq0) static bool do_gvec_fn2_bhs(DisasContext *s, arg_qrr_e *a, GVecGen2Fn *fn) { @@ -XXX,XX +XXX,XX @@ static void handle_2misc_64(DisasContext *s, int opcode, bool u, * The caller only need provide tcg_rmode and tcg_fpstatus if the op * requires them. */ - TCGCond cond; - switch (opcode) { - case 0xa: /* CMLT */ - cond = TCG_COND_LT; - do_cmop: - /* 64 bit integer comparison against zero, result is test ? -1 : 0. */ - tcg_gen_negsetcond_i64(cond, tcg_rd, tcg_rn, tcg_constant_i64(0)); - break; - case 0x8: /* CMGT, CMGE */ - cond = u ? TCG_COND_GE : TCG_COND_GT; - goto do_cmop; - case 0x9: /* CMEQ, CMLE */ - cond = u ? TCG_COND_LE : TCG_COND_EQ; - goto do_cmop; case 0x2f: /* FABS */ gen_vfp_absd(tcg_rd, tcg_rn); break; @@ -XXX,XX +XXX,XX @@ static void handle_2misc_64(DisasContext *s, int opcode, bool u, case 0x4: /* CLS, CLZ */ case 0x5: /* NOT */ case 0x7: /* SQABS, SQNEG */ + case 0x8: /* CMGT, CMGE */ + case 0x9: /* CMEQ, CMLE */ + case 0xa: /* CMLT */ case 0xb: /* ABS, NEG */ g_assert_not_reached(); } @@ -XXX,XX +XXX,XX @@ static void disas_simd_scalar_two_reg_misc(DisasContext *s, uint32_t insn) TCGv_ptr tcg_fpstatus; switch (opcode) { - case 0xa: /* CMLT */ - if (u) { - unallocated_encoding(s); - return; - } - /* fall through */ - case 0x8: /* CMGT, CMGE */ - case 0x9: /* CMEQ, CMLE */ - if (size != 3) { - unallocated_encoding(s); - return; - } - break; case 0x12: /* SQXTUN */ if (!u) { unallocated_encoding(s); @@ -XXX,XX +XXX,XX @@ static void disas_simd_scalar_two_reg_misc(DisasContext *s, uint32_t insn) default: case 0x3: /* USQADD / SUQADD */ case 0x7: /* SQABS / SQNEG */ + case 0x8: /* CMGT, CMGE */ + case 0x9: /* CMEQ, CMLE */ + case 0xa: /* CMLT */ case 0xb: /* ABS, NEG */ unallocated_encoding(s); return; @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) } handle_shll(s, is_q, size, rn, rd); return; - case 0xa: /* CMLT */ - if (u == 1) { - unallocated_encoding(s); - return; - } - /* fall through */ - case 0x8: /* CMGT, CMGE */ - case 0x9: /* CMEQ, CMLE */ - if (size == 3 && !is_q) { - unallocated_encoding(s); - return; - } - break; case 0xc ... 0xf: case 0x16 ... 0x1f: { @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) case 0x4: /* CLS, CLZ */ case 0x5: /* CNT, NOT, RBIT */ case 0x7: /* SQABS, SQNEG */ + case 0x8: /* CMGT, CMGE */ + case 0x9: /* CMEQ, CMLE */ + case 0xa: /* CMLT */ case 0xb: /* ABS, NEG */ unallocated_encoding(s); return; @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) tcg_rmode = NULL; } - switch (opcode) { - case 0x8: /* CMGT, CMGE */ - if (u) { - gen_gvec_fn2(s, is_q, rd, rn, gen_gvec_cge0, size); - } else { - gen_gvec_fn2(s, is_q, rd, rn, gen_gvec_cgt0, size); - } - return; - case 0x9: /* CMEQ, CMLE */ - if (u) { - gen_gvec_fn2(s, is_q, rd, rn, gen_gvec_cle0, size); - } else { - gen_gvec_fn2(s, is_q, rd, rn, gen_gvec_ceq0, size); - } - return; - case 0xa: /* CMLT */ - gen_gvec_fn2(s, is_q, rd, rn, gen_gvec_clt0, size); - return; - case 0x4: /* CLZ, CLS */ - case 0x5: /* CNT, NOT, RBIT */ - case 0xb: - g_assert_not_reached(); - } - if (size == 3) { /* All 64-bit element operations can be shared with scalar 2misc */ int pass; -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-43-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/translate.h | 6 +++ target/arm/tcg/gengvec.c | 58 ++++++++++++++++++++++ target/arm/tcg/translate-neon.c | 88 +++++++-------------------------- 3 files changed, 81 insertions(+), 71 deletions(-) diff --git a/target/arm/tcg/translate.h b/target/arm/tcg/translate.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate.h +++ b/target/arm/tcg/translate.h @@ -XXX,XX +XXX,XX @@ void gen_gvec_cnt(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, uint32_t opr_sz, uint32_t max_sz); void gen_gvec_rbit(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, uint32_t opr_sz, uint32_t max_sz); +void gen_gvec_rev16(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, + uint32_t opr_sz, uint32_t max_sz); +void gen_gvec_rev32(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, + uint32_t opr_sz, uint32_t max_sz); +void gen_gvec_rev64(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, + uint32_t opr_sz, uint32_t max_sz); /* * Forward to the isar_feature_* tests given a DisasContext pointer. diff --git a/target/arm/tcg/gengvec.c b/target/arm/tcg/gengvec.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/gengvec.c +++ b/target/arm/tcg/gengvec.c @@ -XXX,XX +XXX,XX @@ void gen_gvec_rbit(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, tcg_gen_gvec_2_ool(rd_ofs, rn_ofs, opr_sz, max_sz, 0, gen_helper_gvec_rbit_b); } + +void gen_gvec_rev16(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, + uint32_t opr_sz, uint32_t max_sz) +{ + assert(vece == MO_8); + tcg_gen_gvec_rotli(MO_16, rd_ofs, rn_ofs, 8, opr_sz, max_sz); +} + +static void gen_bswap32_i64(TCGv_i64 d, TCGv_i64 n) +{ + tcg_gen_bswap64_i64(d, n); + tcg_gen_rotli_i64(d, d, 32); +} + +void gen_gvec_rev32(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, + uint32_t opr_sz, uint32_t max_sz) +{ + static const GVecGen2 g = { + .fni8 = gen_bswap32_i64, + .fni4 = tcg_gen_bswap32_i32, + .prefer_i64 = TCG_TARGET_REG_BITS == 64, + .vece = MO_32 + }; + + switch (vece) { + case MO_16: + tcg_gen_gvec_rotli(MO_32, rd_ofs, rn_ofs, 16, opr_sz, max_sz); + break; + case MO_8: + tcg_gen_gvec_2(rd_ofs, rn_ofs, opr_sz, max_sz, &g); + break; + default: + g_assert_not_reached(); + } +} + +void gen_gvec_rev64(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, + uint32_t opr_sz, uint32_t max_sz) +{ + static const GVecGen2 g[] = { + { .fni8 = tcg_gen_bswap64_i64, + .vece = MO_64 }, + { .fni8 = tcg_gen_hswap_i64, + .vece = MO_64 }, + }; + + switch (vece) { + case MO_32: + tcg_gen_gvec_rotli(MO_64, rd_ofs, rn_ofs, 32, opr_sz, max_sz); + break; + case MO_8: + case MO_16: + tcg_gen_gvec_2(rd_ofs, rn_ofs, opr_sz, max_sz, &g[vece]); + break; + default: + g_assert_not_reached(); + } +} diff --git a/target/arm/tcg/translate-neon.c b/target/arm/tcg/translate-neon.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-neon.c +++ b/target/arm/tcg/translate-neon.c @@ -XXX,XX +XXX,XX @@ static bool trans_VDUP_scalar(DisasContext *s, arg_VDUP_scalar *a) return true; } -static bool trans_VREV64(DisasContext *s, arg_VREV64 *a) -{ - int pass, half; - TCGv_i32 tmp[2]; - - if (!arm_dc_feature(s, ARM_FEATURE_NEON)) { - return false; - } - - /* UNDEF accesses to D16-D31 if they don't exist. */ - if (!dc_isar_feature(aa32_simd_r32, s) && - ((a->vd | a->vm) & 0x10)) { - return false; - } - - if ((a->vd | a->vm) & a->q) { - return false; - } - - if (a->size == 3) { - return false; - } - - if (!vfp_access_check(s)) { - return true; - } - - tmp[0] = tcg_temp_new_i32(); - tmp[1] = tcg_temp_new_i32(); - - for (pass = 0; pass < (a->q ? 2 : 1); pass++) { - for (half = 0; half < 2; half++) { - read_neon_element32(tmp[half], a->vm, pass * 2 + half, MO_32); - switch (a->size) { - case 0: - tcg_gen_bswap32_i32(tmp[half], tmp[half]); - break; - case 1: - gen_swap_half(tmp[half], tmp[half]); - break; - case 2: - break; - default: - g_assert_not_reached(); - } - } - write_neon_element32(tmp[1], a->vd, pass * 2, MO_32); - write_neon_element32(tmp[0], a->vd, pass * 2 + 1, MO_32); - } - return true; -} - static bool do_2misc_pairwise(DisasContext *s, arg_2misc *a, NeonGenWidenFn *widenfn, NeonGenTwo64OpFn *opfn, @@ -XXX,XX +XXX,XX @@ DO_2MISC_VEC(VCGE0, gen_gvec_cge0) DO_2MISC_VEC(VCLT0, gen_gvec_clt0) DO_2MISC_VEC(VCLS, gen_gvec_cls) DO_2MISC_VEC(VCLZ, gen_gvec_clz) +DO_2MISC_VEC(VREV64, gen_gvec_rev64) static bool trans_VMVN(DisasContext *s, arg_2misc *a) { @@ -XXX,XX +XXX,XX @@ static bool trans_VCNT(DisasContext *s, arg_2misc *a) return do_2misc_vec(s, a, gen_gvec_cnt); } +static bool trans_VREV16(DisasContext *s, arg_2misc *a) +{ + if (a->size != 0) { + return false; + } + return do_2misc_vec(s, a, gen_gvec_rev16); +} + +static bool trans_VREV32(DisasContext *s, arg_2misc *a) +{ + if (a->size != 0 && a->size != 1) { + return false; + } + return do_2misc_vec(s, a, gen_gvec_rev32); +} + #define WRAP_2M_3_OOL_FN(WRAPNAME, FUNC, DATA) \ static void WRAPNAME(unsigned vece, uint32_t rd_ofs, \ uint32_t rm_ofs, uint32_t oprsz, \ @@ -XXX,XX +XXX,XX @@ static bool do_2misc(DisasContext *s, arg_2misc *a, NeonGenOneOpFn *fn) return true; } -static bool trans_VREV32(DisasContext *s, arg_2misc *a) -{ - static NeonGenOneOpFn * const fn[] = { - tcg_gen_bswap32_i32, - gen_swap_half, - NULL, - NULL, - }; - return do_2misc(s, a, fn[a->size]); -} - -static bool trans_VREV16(DisasContext *s, arg_2misc *a) -{ - if (a->size != 0) { - return false; - } - return do_2misc(s, a, gen_rev16); -} - static void gen_VABS_F(unsigned vece, uint32_t rd_ofs, uint32_t rm_ofs, uint32_t oprsz, uint32_t maxsz) { -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> This includes REV16, REV32, REV64. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-44-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 5 +++ target/arm/tcg/translate-a64.c | 79 +++------------------------------- 2 files changed, 10 insertions(+), 74 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ @qrr_b . q:1 ...... .. ...... ...... rn:5 rd:5 &qrr_e esz=0 @qrr_h . q:1 ...... .. ...... ...... rn:5 rd:5 &qrr_e esz=1 +@qrr_bh . q:1 ...... . esz:1 ...... ...... rn:5 rd:5 &qrr_e @qrr_e . q:1 ...... esz:2 ...... ...... rn:5 rd:5 &qrr_e @qrrr_b . q:1 ...... ... rm:5 ...... rn:5 rd:5 &qrrr_e esz=0 @@ -XXX,XX +XXX,XX @@ CMGE0_v 0.10 1110 ..1 00000 10001 0 ..... ..... @qrr_e CMEQ0_v 0.00 1110 ..1 00000 10011 0 ..... ..... @qrr_e CMLE0_v 0.10 1110 ..1 00000 10011 0 ..... ..... @qrr_e CMLT0_v 0.00 1110 ..1 00000 10101 0 ..... ..... @qrr_e + +REV16_v 0.00 1110 001 00000 00011 0 ..... ..... @qrr_b +REV32_v 0.10 1110 0.1 00000 00001 0 ..... ..... @qrr_bh +REV64_v 0.00 1110 ..1 00000 00001 0 ..... ..... @qrr_e diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ TRANS(CMGE0_v, do_gvec_fn2, a, gen_gvec_cge0) TRANS(CMLT0_v, do_gvec_fn2, a, gen_gvec_clt0) TRANS(CMLE0_v, do_gvec_fn2, a, gen_gvec_cle0) TRANS(CMEQ0_v, do_gvec_fn2, a, gen_gvec_ceq0) +TRANS(REV16_v, do_gvec_fn2, a, gen_gvec_rev16) +TRANS(REV32_v, do_gvec_fn2, a, gen_gvec_rev32) static bool do_gvec_fn2_bhs(DisasContext *s, arg_qrr_e *a, GVecGen2Fn *fn) { @@ -XXX,XX +XXX,XX @@ static bool do_gvec_fn2_bhs(DisasContext *s, arg_qrr_e *a, GVecGen2Fn *fn) TRANS(CLS_v, do_gvec_fn2_bhs, a, gen_gvec_cls) TRANS(CLZ_v, do_gvec_fn2_bhs, a, gen_gvec_clz) +TRANS(REV64_v, do_gvec_fn2_bhs, a, gen_gvec_rev64) /* Common vector code for handling integer to FP conversion */ static void handle_simd_intfp_conv(DisasContext *s, int rd, int rn, @@ -XXX,XX +XXX,XX @@ static void handle_2misc_widening(DisasContext *s, int opcode, bool is_q, } } -static void handle_rev(DisasContext *s, int opcode, bool u, - bool is_q, int size, int rn, int rd) -{ - int op = (opcode << 1) | u; - int opsz = op + size; - int grp_size = 3 - opsz; - int dsize = is_q ? 128 : 64; - int i; - - if (opsz >= 3) { - unallocated_encoding(s); - return; - } - - if (!fp_access_check(s)) { - return; - } - - if (size == 0) { - /* Special case bytes, use bswap op on each group of elements */ - int groups = dsize / (8 << grp_size); - - for (i = 0; i < groups; i++) { - TCGv_i64 tcg_tmp = tcg_temp_new_i64(); - - read_vec_element(s, tcg_tmp, rn, i, grp_size); - switch (grp_size) { - case MO_16: - tcg_gen_bswap16_i64(tcg_tmp, tcg_tmp, TCG_BSWAP_IZ); - break; - case MO_32: - tcg_gen_bswap32_i64(tcg_tmp, tcg_tmp, TCG_BSWAP_IZ); - break; - case MO_64: - tcg_gen_bswap64_i64(tcg_tmp, tcg_tmp); - break; - default: - g_assert_not_reached(); - } - write_vec_element(s, tcg_tmp, rd, i, grp_size); - } - clear_vec_high(s, is_q, rd); - } else { - int revmask = (1 << grp_size) - 1; - int esize = 8 << size; - int elements = dsize / esize; - TCGv_i64 tcg_rn = tcg_temp_new_i64(); - TCGv_i64 tcg_rd[2]; - - for (i = 0; i < 2; i++) { - tcg_rd[i] = tcg_temp_new_i64(); - tcg_gen_movi_i64(tcg_rd[i], 0); - } - - for (i = 0; i < elements; i++) { - int e_rev = (i & 0xf) ^ revmask; - int w = (e_rev * esize) / 64; - int o = (e_rev * esize) % 64; - - read_vec_element(s, tcg_rn, rn, i, size); - tcg_gen_deposit_i64(tcg_rd[w], tcg_rd[w], tcg_rn, o, esize); - } - - for (i = 0; i < 2; i++) { - write_vec_element(s, tcg_rd[i], rd, i, MO_64); - } - clear_vec_high(s, true, rd); - } -} - static void handle_2misc_pairwise(DisasContext *s, int opcode, bool u, bool is_q, int size, int rn, int rd) { @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) TCGv_ptr tcg_fpstatus; switch (opcode) { - case 0x0: /* REV64, REV32 */ - case 0x1: /* REV16 */ - handle_rev(s, opcode, u, is_q, size, rn, rd); - return; case 0x12: /* XTN, XTN2, SQXTUN, SQXTUN2 */ case 0x14: /* SQXTN, SQXTN2, UQXTN, UQXTN2 */ if (size == 3) { @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) break; } default: + case 0x0: /* REV64, REV32 */ + case 0x1: /* REV16 */ case 0x3: /* SUQADD, USQADD */ case 0x4: /* CLS, CLZ */ case 0x5: /* CNT, NOT, RBIT */ -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Move from helper-a64.c to neon_helper.c so that these functions are available for arm32 code as well. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-45-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/helper.h | 2 ++ target/arm/tcg/helper-a64.h | 2 -- target/arm/tcg/helper-a64.c | 43 ------------------------------------ target/arm/tcg/neon_helper.c | 43 ++++++++++++++++++++++++++++++++++++ 4 files changed, 45 insertions(+), 45 deletions(-) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_2(neon_addl_u16, i64, i64, i64) DEF_HELPER_2(neon_addl_u32, i64, i64, i64) DEF_HELPER_2(neon_paddl_u16, i64, i64, i64) DEF_HELPER_2(neon_paddl_u32, i64, i64, i64) +DEF_HELPER_FLAGS_1(neon_addlp_s8, TCG_CALL_NO_RWG_SE, i64, i64) +DEF_HELPER_FLAGS_1(neon_addlp_s16, TCG_CALL_NO_RWG_SE, i64, i64) DEF_HELPER_2(neon_subl_u16, i64, i64, i64) DEF_HELPER_2(neon_subl_u32, i64, i64, i64) DEF_HELPER_3(neon_addl_saturate_s32, i64, env, i64, i64) diff --git a/target/arm/tcg/helper-a64.h b/target/arm/tcg/helper-a64.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/helper-a64.h +++ b/target/arm/tcg/helper-a64.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_3(recpsf_f64, TCG_CALL_NO_RWG, f64, f64, f64, ptr) DEF_HELPER_FLAGS_3(rsqrtsf_f16, TCG_CALL_NO_RWG, f16, f16, f16, ptr) DEF_HELPER_FLAGS_3(rsqrtsf_f32, TCG_CALL_NO_RWG, f32, f32, f32, ptr) DEF_HELPER_FLAGS_3(rsqrtsf_f64, TCG_CALL_NO_RWG, f64, f64, f64, ptr) -DEF_HELPER_FLAGS_1(neon_addlp_s8, TCG_CALL_NO_RWG_SE, i64, i64) DEF_HELPER_FLAGS_1(neon_addlp_u8, TCG_CALL_NO_RWG_SE, i64, i64) -DEF_HELPER_FLAGS_1(neon_addlp_s16, TCG_CALL_NO_RWG_SE, i64, i64) DEF_HELPER_FLAGS_1(neon_addlp_u16, TCG_CALL_NO_RWG_SE, i64, i64) DEF_HELPER_FLAGS_2(frecpx_f64, TCG_CALL_NO_RWG, f64, f64, ptr) DEF_HELPER_FLAGS_2(frecpx_f32, TCG_CALL_NO_RWG, f32, f32, ptr) diff --git a/target/arm/tcg/helper-a64.c b/target/arm/tcg/helper-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/helper-a64.c +++ b/target/arm/tcg/helper-a64.c @@ -XXX,XX +XXX,XX @@ float64 HELPER(rsqrtsf_f64)(float64 a, float64 b, void *fpstp) return float64_muladd(a, b, float64_three, float_muladd_halve_result, fpst); } -/* Pairwise long add: add pairs of adjacent elements into - * double-width elements in the result (eg _s8 is an 8x8->16 op) - */ -uint64_t HELPER(neon_addlp_s8)(uint64_t a) -{ - uint64_t nsignmask = 0x0080008000800080ULL; - uint64_t wsignmask = 0x8000800080008000ULL; - uint64_t elementmask = 0x00ff00ff00ff00ffULL; - uint64_t tmp1, tmp2; - uint64_t res, signres; - - /* Extract odd elements, sign extend each to a 16 bit field */ - tmp1 = a & elementmask; - tmp1 ^= nsignmask; - tmp1 |= wsignmask; - tmp1 = (tmp1 - nsignmask) ^ wsignmask; - /* Ditto for the even elements */ - tmp2 = (a >> 8) & elementmask; - tmp2 ^= nsignmask; - tmp2 |= wsignmask; - tmp2 = (tmp2 - nsignmask) ^ wsignmask; - - /* calculate the result by summing bits 0..14, 16..22, etc, - * and then adjusting the sign bits 15, 23, etc manually. - * This ensures the addition can't overflow the 16 bit field. - */ - signres = (tmp1 ^ tmp2) & wsignmask; - res = (tmp1 & ~wsignmask) + (tmp2 & ~wsignmask); - res ^= signres; - - return res; -} - uint64_t HELPER(neon_addlp_u8)(uint64_t a) { uint64_t tmp; @@ -XXX,XX +XXX,XX @@ uint64_t HELPER(neon_addlp_u8)(uint64_t a) return tmp; } -uint64_t HELPER(neon_addlp_s16)(uint64_t a) -{ - int32_t reslo, reshi; - - reslo = (int32_t)(int16_t)a + (int32_t)(int16_t)(a >> 16); - reshi = (int32_t)(int16_t)(a >> 32) + (int32_t)(int16_t)(a >> 48); - - return (uint32_t)reslo | (((uint64_t)reshi) << 32); -} - uint64_t HELPER(neon_addlp_u16)(uint64_t a) { uint64_t tmp; diff --git a/target/arm/tcg/neon_helper.c b/target/arm/tcg/neon_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/neon_helper.c +++ b/target/arm/tcg/neon_helper.c @@ -XXX,XX +XXX,XX @@ uint64_t HELPER(neon_paddl_u32)(uint64_t a, uint64_t b) return low + ((uint64_t)high << 32); } +/* Pairwise long add: add pairs of adjacent elements into + * double-width elements in the result (eg _s8 is an 8x8->16 op) + */ +uint64_t HELPER(neon_addlp_s8)(uint64_t a) +{ + uint64_t nsignmask = 0x0080008000800080ULL; + uint64_t wsignmask = 0x8000800080008000ULL; + uint64_t elementmask = 0x00ff00ff00ff00ffULL; + uint64_t tmp1, tmp2; + uint64_t res, signres; + + /* Extract odd elements, sign extend each to a 16 bit field */ + tmp1 = a & elementmask; + tmp1 ^= nsignmask; + tmp1 |= wsignmask; + tmp1 = (tmp1 - nsignmask) ^ wsignmask; + /* Ditto for the even elements */ + tmp2 = (a >> 8) & elementmask; + tmp2 ^= nsignmask; + tmp2 |= wsignmask; + tmp2 = (tmp2 - nsignmask) ^ wsignmask; + + /* calculate the result by summing bits 0..14, 16..22, etc, + * and then adjusting the sign bits 15, 23, etc manually. + * This ensures the addition can't overflow the 16 bit field. + */ + signres = (tmp1 ^ tmp2) & wsignmask; + res = (tmp1 & ~wsignmask) + (tmp2 & ~wsignmask); + res ^= signres; + + return res; +} + +uint64_t HELPER(neon_addlp_s16)(uint64_t a) +{ + int32_t reslo, reshi; + + reslo = (int32_t)(int16_t)a + (int32_t)(int16_t)(a >> 16); + reshi = (int32_t)(int16_t)(a >> 32) + (int32_t)(int16_t)(a >> 48); + + return (uint32_t)reslo | (((uint64_t)reshi) << 32); +} + uint64_t HELPER(neon_subl_u16)(uint64_t a, uint64_t b) { uint64_t mask; -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Pairwise addition with and without accumulation. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-46-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/helper.h | 2 - target/arm/tcg/translate.h | 9 ++ target/arm/tcg/gengvec.c | 230 ++++++++++++++++++++++++++++++++ target/arm/tcg/neon_helper.c | 22 --- target/arm/tcg/translate-neon.c | 150 +-------------------- 5 files changed, 243 insertions(+), 170 deletions(-) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_1(neon_widen_s16, i64, i32) DEF_HELPER_2(neon_addl_u16, i64, i64, i64) DEF_HELPER_2(neon_addl_u32, i64, i64, i64) -DEF_HELPER_2(neon_paddl_u16, i64, i64, i64) -DEF_HELPER_2(neon_paddl_u32, i64, i64, i64) DEF_HELPER_FLAGS_1(neon_addlp_s8, TCG_CALL_NO_RWG_SE, i64, i64) DEF_HELPER_FLAGS_1(neon_addlp_s16, TCG_CALL_NO_RWG_SE, i64, i64) DEF_HELPER_2(neon_subl_u16, i64, i64, i64) diff --git a/target/arm/tcg/translate.h b/target/arm/tcg/translate.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate.h +++ b/target/arm/tcg/translate.h @@ -XXX,XX +XXX,XX @@ void gen_gvec_rev32(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, void gen_gvec_rev64(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, uint32_t opr_sz, uint32_t max_sz); +void gen_gvec_saddlp(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, + uint32_t opr_sz, uint32_t max_sz); +void gen_gvec_sadalp(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, + uint32_t opr_sz, uint32_t max_sz); +void gen_gvec_uaddlp(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, + uint32_t opr_sz, uint32_t max_sz); +void gen_gvec_uadalp(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, + uint32_t opr_sz, uint32_t max_sz); + /* * Forward to the isar_feature_* tests given a DisasContext pointer. */ diff --git a/target/arm/tcg/gengvec.c b/target/arm/tcg/gengvec.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/gengvec.c +++ b/target/arm/tcg/gengvec.c @@ -XXX,XX +XXX,XX @@ void gen_gvec_rev64(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, g_assert_not_reached(); } } + +static void gen_saddlp_vec(unsigned vece, TCGv_vec d, TCGv_vec n) +{ + int half = 4 << vece; + TCGv_vec t = tcg_temp_new_vec_matching(d); + + tcg_gen_shli_vec(vece, t, n, half); + tcg_gen_sari_vec(vece, d, n, half); + tcg_gen_sari_vec(vece, t, t, half); + tcg_gen_add_vec(vece, d, d, t); +} + +static void gen_saddlp_s_i64(TCGv_i64 d, TCGv_i64 n) +{ + TCGv_i64 t = tcg_temp_new_i64(); + + tcg_gen_ext32s_i64(t, n); + tcg_gen_sari_i64(d, n, 32); + tcg_gen_add_i64(d, d, t); +} + +void gen_gvec_saddlp(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, + uint32_t opr_sz, uint32_t max_sz) +{ + static const TCGOpcode vecop_list[] = { + INDEX_op_sari_vec, INDEX_op_shli_vec, INDEX_op_add_vec, 0 + }; + static const GVecGen2 g[] = { + { .fniv = gen_saddlp_vec, + .fni8 = gen_helper_neon_addlp_s8, + .opt_opc = vecop_list, + .vece = MO_16 }, + { .fniv = gen_saddlp_vec, + .fni8 = gen_helper_neon_addlp_s16, + .opt_opc = vecop_list, + .vece = MO_32 }, + { .fniv = gen_saddlp_vec, + .fni8 = gen_saddlp_s_i64, + .opt_opc = vecop_list, + .vece = MO_64 }, + }; + assert(vece <= MO_32); + tcg_gen_gvec_2(rd_ofs, rn_ofs, opr_sz, max_sz, &g[vece]); +} + +static void gen_sadalp_vec(unsigned vece, TCGv_vec d, TCGv_vec n) +{ + TCGv_vec t = tcg_temp_new_vec_matching(d); + + gen_saddlp_vec(vece, t, n); + tcg_gen_add_vec(vece, d, d, t); +} + +static void gen_sadalp_b_i64(TCGv_i64 d, TCGv_i64 n) +{ + TCGv_i64 t = tcg_temp_new_i64(); + + gen_helper_neon_addlp_s8(t, n); + tcg_gen_vec_add16_i64(d, d, t); +} + +static void gen_sadalp_h_i64(TCGv_i64 d, TCGv_i64 n) +{ + TCGv_i64 t = tcg_temp_new_i64(); + + gen_helper_neon_addlp_s16(t, n); + tcg_gen_vec_add32_i64(d, d, t); +} + +static void gen_sadalp_s_i64(TCGv_i64 d, TCGv_i64 n) +{ + TCGv_i64 t = tcg_temp_new_i64(); + + gen_saddlp_s_i64(t, n); + tcg_gen_add_i64(d, d, t); +} + +void gen_gvec_sadalp(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, + uint32_t opr_sz, uint32_t max_sz) +{ + static const TCGOpcode vecop_list[] = { + INDEX_op_sari_vec, INDEX_op_shli_vec, INDEX_op_add_vec, 0 + }; + static const GVecGen2 g[] = { + { .fniv = gen_sadalp_vec, + .fni8 = gen_sadalp_b_i64, + .opt_opc = vecop_list, + .load_dest = true, + .vece = MO_16 }, + { .fniv = gen_sadalp_vec, + .fni8 = gen_sadalp_h_i64, + .opt_opc = vecop_list, + .load_dest = true, + .vece = MO_32 }, + { .fniv = gen_sadalp_vec, + .fni8 = gen_sadalp_s_i64, + .opt_opc = vecop_list, + .load_dest = true, + .vece = MO_64 }, + }; + assert(vece <= MO_32); + tcg_gen_gvec_2(rd_ofs, rn_ofs, opr_sz, max_sz, &g[vece]); +} + +static void gen_uaddlp_vec(unsigned vece, TCGv_vec d, TCGv_vec n) +{ + int half = 4 << vece; + TCGv_vec t = tcg_temp_new_vec_matching(d); + TCGv_vec m = tcg_constant_vec_matching(d, vece, MAKE_64BIT_MASK(0, half)); + + tcg_gen_shri_vec(vece, t, n, half); + tcg_gen_and_vec(vece, d, n, m); + tcg_gen_add_vec(vece, d, d, t); +} + +static void gen_uaddlp_b_i64(TCGv_i64 d, TCGv_i64 n) +{ + TCGv_i64 t = tcg_temp_new_i64(); + TCGv_i64 m = tcg_constant_i64(dup_const(MO_16, 0xff)); + + tcg_gen_shri_i64(t, n, 8); + tcg_gen_and_i64(d, n, m); + tcg_gen_and_i64(t, t, m); + /* No carry between widened unsigned elements. */ + tcg_gen_add_i64(d, d, t); +} + +static void gen_uaddlp_h_i64(TCGv_i64 d, TCGv_i64 n) +{ + TCGv_i64 t = tcg_temp_new_i64(); + TCGv_i64 m = tcg_constant_i64(dup_const(MO_32, 0xffff)); + + tcg_gen_shri_i64(t, n, 16); + tcg_gen_and_i64(d, n, m); + tcg_gen_and_i64(t, t, m); + /* No carry between widened unsigned elements. */ + tcg_gen_add_i64(d, d, t); +} + +static void gen_uaddlp_s_i64(TCGv_i64 d, TCGv_i64 n) +{ + TCGv_i64 t = tcg_temp_new_i64(); + + tcg_gen_ext32u_i64(t, n); + tcg_gen_shri_i64(d, n, 32); + tcg_gen_add_i64(d, d, t); +} + +void gen_gvec_uaddlp(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, + uint32_t opr_sz, uint32_t max_sz) +{ + static const TCGOpcode vecop_list[] = { + INDEX_op_shri_vec, INDEX_op_add_vec, 0 + }; + static const GVecGen2 g[] = { + { .fniv = gen_uaddlp_vec, + .fni8 = gen_uaddlp_b_i64, + .opt_opc = vecop_list, + .vece = MO_16 }, + { .fniv = gen_uaddlp_vec, + .fni8 = gen_uaddlp_h_i64, + .opt_opc = vecop_list, + .vece = MO_32 }, + { .fniv = gen_uaddlp_vec, + .fni8 = gen_uaddlp_s_i64, + .opt_opc = vecop_list, + .vece = MO_64 }, + }; + assert(vece <= MO_32); + tcg_gen_gvec_2(rd_ofs, rn_ofs, opr_sz, max_sz, &g[vece]); +} + +static void gen_uadalp_vec(unsigned vece, TCGv_vec d, TCGv_vec n) +{ + TCGv_vec t = tcg_temp_new_vec_matching(d); + + gen_uaddlp_vec(vece, t, n); + tcg_gen_add_vec(vece, d, d, t); +} + +static void gen_uadalp_b_i64(TCGv_i64 d, TCGv_i64 n) +{ + TCGv_i64 t = tcg_temp_new_i64(); + + gen_uaddlp_b_i64(t, n); + tcg_gen_vec_add16_i64(d, d, t); +} + +static void gen_uadalp_h_i64(TCGv_i64 d, TCGv_i64 n) +{ + TCGv_i64 t = tcg_temp_new_i64(); + + gen_uaddlp_h_i64(t, n); + tcg_gen_vec_add32_i64(d, d, t); +} + +static void gen_uadalp_s_i64(TCGv_i64 d, TCGv_i64 n) +{ + TCGv_i64 t = tcg_temp_new_i64(); + + gen_uaddlp_s_i64(t, n); + tcg_gen_add_i64(d, d, t); +} + +void gen_gvec_uadalp(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, + uint32_t opr_sz, uint32_t max_sz) +{ + static const TCGOpcode vecop_list[] = { + INDEX_op_shri_vec, INDEX_op_add_vec, 0 + }; + static const GVecGen2 g[] = { + { .fniv = gen_uadalp_vec, + .fni8 = gen_uadalp_b_i64, + .load_dest = true, + .opt_opc = vecop_list, + .vece = MO_16 }, + { .fniv = gen_uadalp_vec, + .fni8 = gen_uadalp_h_i64, + .load_dest = true, + .opt_opc = vecop_list, + .vece = MO_32 }, + { .fniv = gen_uadalp_vec, + .fni8 = gen_uadalp_s_i64, + .load_dest = true, + .opt_opc = vecop_list, + .vece = MO_64 }, + }; + assert(vece <= MO_32); + tcg_gen_gvec_2(rd_ofs, rn_ofs, opr_sz, max_sz, &g[vece]); +} diff --git a/target/arm/tcg/neon_helper.c b/target/arm/tcg/neon_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/neon_helper.c +++ b/target/arm/tcg/neon_helper.c @@ -XXX,XX +XXX,XX @@ uint64_t HELPER(neon_addl_u32)(uint64_t a, uint64_t b) return (a + b) ^ mask; } -uint64_t HELPER(neon_paddl_u16)(uint64_t a, uint64_t b) -{ - uint64_t tmp; - uint64_t tmp2; - - tmp = a & 0x0000ffff0000ffffull; - tmp += (a >> 16) & 0x0000ffff0000ffffull; - tmp2 = b & 0xffff0000ffff0000ull; - tmp2 += (b << 16) & 0xffff0000ffff0000ull; - return ( tmp & 0xffff) - | ((tmp >> 16) & 0xffff0000ull) - | ((tmp2 << 16) & 0xffff00000000ull) - | ( tmp2 & 0xffff000000000000ull); -} - -uint64_t HELPER(neon_paddl_u32)(uint64_t a, uint64_t b) -{ - uint32_t low = a + (a >> 32); - uint32_t high = b + (b >> 32); - return low + ((uint64_t)high << 32); -} - /* Pairwise long add: add pairs of adjacent elements into * double-width elements in the result (eg _s8 is an 8x8->16 op) */ diff --git a/target/arm/tcg/translate-neon.c b/target/arm/tcg/translate-neon.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-neon.c +++ b/target/arm/tcg/translate-neon.c @@ -XXX,XX +XXX,XX @@ static bool trans_VDUP_scalar(DisasContext *s, arg_VDUP_scalar *a) return true; } -static bool do_2misc_pairwise(DisasContext *s, arg_2misc *a, - NeonGenWidenFn *widenfn, - NeonGenTwo64OpFn *opfn, - NeonGenTwo64OpFn *accfn) -{ - /* - * Pairwise long operations: widen both halves of the pair, - * combine the pairs with the opfn, and then possibly accumulate - * into the destination with the accfn. - */ - int pass; - - if (!arm_dc_feature(s, ARM_FEATURE_NEON)) { - return false; - } - - /* UNDEF accesses to D16-D31 if they don't exist. */ - if (!dc_isar_feature(aa32_simd_r32, s) && - ((a->vd | a->vm) & 0x10)) { - return false; - } - - if ((a->vd | a->vm) & a->q) { - return false; - } - - if (!widenfn) { - return false; - } - - if (!vfp_access_check(s)) { - return true; - } - - for (pass = 0; pass < a->q + 1; pass++) { - TCGv_i32 tmp; - TCGv_i64 rm0_64, rm1_64, rd_64; - - rm0_64 = tcg_temp_new_i64(); - rm1_64 = tcg_temp_new_i64(); - rd_64 = tcg_temp_new_i64(); - - tmp = tcg_temp_new_i32(); - read_neon_element32(tmp, a->vm, pass * 2, MO_32); - widenfn(rm0_64, tmp); - read_neon_element32(tmp, a->vm, pass * 2 + 1, MO_32); - widenfn(rm1_64, tmp); - - opfn(rd_64, rm0_64, rm1_64); - - if (accfn) { - TCGv_i64 tmp64 = tcg_temp_new_i64(); - read_neon_element64(tmp64, a->vd, pass, MO_64); - accfn(rd_64, tmp64, rd_64); - } - write_neon_element64(rd_64, a->vd, pass, MO_64); - } - return true; -} - -static bool trans_VPADDL_S(DisasContext *s, arg_2misc *a) -{ - static NeonGenWidenFn * const widenfn[] = { - gen_helper_neon_widen_s8, - gen_helper_neon_widen_s16, - tcg_gen_ext_i32_i64, - NULL, - }; - static NeonGenTwo64OpFn * const opfn[] = { - gen_helper_neon_paddl_u16, - gen_helper_neon_paddl_u32, - tcg_gen_add_i64, - NULL, - }; - - return do_2misc_pairwise(s, a, widenfn[a->size], opfn[a->size], NULL); -} - -static bool trans_VPADDL_U(DisasContext *s, arg_2misc *a) -{ - static NeonGenWidenFn * const widenfn[] = { - gen_helper_neon_widen_u8, - gen_helper_neon_widen_u16, - tcg_gen_extu_i32_i64, - NULL, - }; - static NeonGenTwo64OpFn * const opfn[] = { - gen_helper_neon_paddl_u16, - gen_helper_neon_paddl_u32, - tcg_gen_add_i64, - NULL, - }; - - return do_2misc_pairwise(s, a, widenfn[a->size], opfn[a->size], NULL); -} - -static bool trans_VPADAL_S(DisasContext *s, arg_2misc *a) -{ - static NeonGenWidenFn * const widenfn[] = { - gen_helper_neon_widen_s8, - gen_helper_neon_widen_s16, - tcg_gen_ext_i32_i64, - NULL, - }; - static NeonGenTwo64OpFn * const opfn[] = { - gen_helper_neon_paddl_u16, - gen_helper_neon_paddl_u32, - tcg_gen_add_i64, - NULL, - }; - static NeonGenTwo64OpFn * const accfn[] = { - gen_helper_neon_addl_u16, - gen_helper_neon_addl_u32, - tcg_gen_add_i64, - NULL, - }; - - return do_2misc_pairwise(s, a, widenfn[a->size], opfn[a->size], - accfn[a->size]); -} - -static bool trans_VPADAL_U(DisasContext *s, arg_2misc *a) -{ - static NeonGenWidenFn * const widenfn[] = { - gen_helper_neon_widen_u8, - gen_helper_neon_widen_u16, - tcg_gen_extu_i32_i64, - NULL, - }; - static NeonGenTwo64OpFn * const opfn[] = { - gen_helper_neon_paddl_u16, - gen_helper_neon_paddl_u32, - tcg_gen_add_i64, - NULL, - }; - static NeonGenTwo64OpFn * const accfn[] = { - gen_helper_neon_addl_u16, - gen_helper_neon_addl_u32, - tcg_gen_add_i64, - NULL, - }; - - return do_2misc_pairwise(s, a, widenfn[a->size], opfn[a->size], - accfn[a->size]); -} - typedef void ZipFn(TCGv_ptr, TCGv_ptr); static bool do_zip_uzp(DisasContext *s, arg_2misc *a, @@ -XXX,XX +XXX,XX @@ DO_2MISC_VEC(VCLT0, gen_gvec_clt0) DO_2MISC_VEC(VCLS, gen_gvec_cls) DO_2MISC_VEC(VCLZ, gen_gvec_clz) DO_2MISC_VEC(VREV64, gen_gvec_rev64) +DO_2MISC_VEC(VPADDL_S, gen_gvec_saddlp) +DO_2MISC_VEC(VPADDL_U, gen_gvec_uaddlp) +DO_2MISC_VEC(VPADAL_S, gen_gvec_sadalp) +DO_2MISC_VEC(VPADAL_U, gen_gvec_uadalp) static bool trans_VMVN(DisasContext *s, arg_2misc *a) { -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> This includes SADDLP, UADDLP, SADALP, UADALP. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-47-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/helper-a64.h | 2 - target/arm/tcg/a64.decode | 5 ++ target/arm/tcg/helper-a64.c | 18 -------- target/arm/tcg/translate-a64.c | 84 +++------------------------------- 4 files changed, 11 insertions(+), 98 deletions(-) diff --git a/target/arm/tcg/helper-a64.h b/target/arm/tcg/helper-a64.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/helper-a64.h +++ b/target/arm/tcg/helper-a64.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_3(recpsf_f64, TCG_CALL_NO_RWG, f64, f64, f64, ptr) DEF_HELPER_FLAGS_3(rsqrtsf_f16, TCG_CALL_NO_RWG, f16, f16, f16, ptr) DEF_HELPER_FLAGS_3(rsqrtsf_f32, TCG_CALL_NO_RWG, f32, f32, f32, ptr) DEF_HELPER_FLAGS_3(rsqrtsf_f64, TCG_CALL_NO_RWG, f64, f64, f64, ptr) -DEF_HELPER_FLAGS_1(neon_addlp_u8, TCG_CALL_NO_RWG_SE, i64, i64) -DEF_HELPER_FLAGS_1(neon_addlp_u16, TCG_CALL_NO_RWG_SE, i64, i64) DEF_HELPER_FLAGS_2(frecpx_f64, TCG_CALL_NO_RWG, f64, f64, ptr) DEF_HELPER_FLAGS_2(frecpx_f32, TCG_CALL_NO_RWG, f32, f32, ptr) DEF_HELPER_FLAGS_2(frecpx_f16, TCG_CALL_NO_RWG, f16, f16, ptr) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ CMLT0_v 0.00 1110 ..1 00000 10101 0 ..... ..... @qrr_e REV16_v 0.00 1110 001 00000 00011 0 ..... ..... @qrr_b REV32_v 0.10 1110 0.1 00000 00001 0 ..... ..... @qrr_bh REV64_v 0.00 1110 ..1 00000 00001 0 ..... ..... @qrr_e + +SADDLP_v 0.00 1110 ..1 00000 00101 0 ..... ..... @qrr_e +UADDLP_v 0.10 1110 ..1 00000 00101 0 ..... ..... @qrr_e +SADALP_v 0.00 1110 ..1 00000 01101 0 ..... ..... @qrr_e +UADALP_v 0.10 1110 ..1 00000 01101 0 ..... ..... @qrr_e diff --git a/target/arm/tcg/helper-a64.c b/target/arm/tcg/helper-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/helper-a64.c +++ b/target/arm/tcg/helper-a64.c @@ -XXX,XX +XXX,XX @@ float64 HELPER(rsqrtsf_f64)(float64 a, float64 b, void *fpstp) return float64_muladd(a, b, float64_three, float_muladd_halve_result, fpst); } -uint64_t HELPER(neon_addlp_u8)(uint64_t a) -{ - uint64_t tmp; - - tmp = a & 0x00ff00ff00ff00ffULL; - tmp += (a >> 8) & 0x00ff00ff00ff00ffULL; - return tmp; -} - -uint64_t HELPER(neon_addlp_u16)(uint64_t a) -{ - uint64_t tmp; - - tmp = a & 0x0000ffff0000ffffULL; - tmp += (a >> 16) & 0x0000ffff0000ffffULL; - return tmp; -} - /* Floating-point reciprocal exponent - see FPRecpX in ARM ARM */ uint32_t HELPER(frecpx_f16)(uint32_t a, void *fpstp) { diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static bool do_gvec_fn2_bhs(DisasContext *s, arg_qrr_e *a, GVecGen2Fn *fn) TRANS(CLS_v, do_gvec_fn2_bhs, a, gen_gvec_cls) TRANS(CLZ_v, do_gvec_fn2_bhs, a, gen_gvec_clz) TRANS(REV64_v, do_gvec_fn2_bhs, a, gen_gvec_rev64) +TRANS(SADDLP_v, do_gvec_fn2_bhs, a, gen_gvec_saddlp) +TRANS(UADDLP_v, do_gvec_fn2_bhs, a, gen_gvec_uaddlp) +TRANS(SADALP_v, do_gvec_fn2_bhs, a, gen_gvec_sadalp) +TRANS(UADALP_v, do_gvec_fn2_bhs, a, gen_gvec_uadalp) /* Common vector code for handling integer to FP conversion */ static void handle_simd_intfp_conv(DisasContext *s, int rd, int rn, @@ -XXX,XX +XXX,XX @@ static void handle_2misc_widening(DisasContext *s, int opcode, bool is_q, } } -static void handle_2misc_pairwise(DisasContext *s, int opcode, bool u, - bool is_q, int size, int rn, int rd) -{ - /* Implement the pairwise operations from 2-misc: - * SADDLP, UADDLP, SADALP, UADALP. - * These all add pairs of elements in the input to produce a - * double-width result element in the output (possibly accumulating). - */ - bool accum = (opcode == 0x6); - int maxpass = is_q ? 2 : 1; - int pass; - TCGv_i64 tcg_res[2]; - - if (size == 2) { - /* 32 + 32 -> 64 op */ - MemOp memop = size + (u ? 0 : MO_SIGN); - - for (pass = 0; pass < maxpass; pass++) { - TCGv_i64 tcg_op1 = tcg_temp_new_i64(); - TCGv_i64 tcg_op2 = tcg_temp_new_i64(); - - tcg_res[pass] = tcg_temp_new_i64(); - - read_vec_element(s, tcg_op1, rn, pass * 2, memop); - read_vec_element(s, tcg_op2, rn, pass * 2 + 1, memop); - tcg_gen_add_i64(tcg_res[pass], tcg_op1, tcg_op2); - if (accum) { - read_vec_element(s, tcg_op1, rd, pass, MO_64); - tcg_gen_add_i64(tcg_res[pass], tcg_res[pass], tcg_op1); - } - } - } else { - for (pass = 0; pass < maxpass; pass++) { - TCGv_i64 tcg_op = tcg_temp_new_i64(); - NeonGenOne64OpFn *genfn; - static NeonGenOne64OpFn * const fns[2][2] = { - { gen_helper_neon_addlp_s8, gen_helper_neon_addlp_u8 }, - { gen_helper_neon_addlp_s16, gen_helper_neon_addlp_u16 }, - }; - - genfn = fns[size][u]; - - tcg_res[pass] = tcg_temp_new_i64(); - - read_vec_element(s, tcg_op, rn, pass, MO_64); - genfn(tcg_res[pass], tcg_op); - - if (accum) { - read_vec_element(s, tcg_op, rd, pass, MO_64); - if (size == 0) { - gen_helper_neon_addl_u16(tcg_res[pass], - tcg_res[pass], tcg_op); - } else { - gen_helper_neon_addl_u32(tcg_res[pass], - tcg_res[pass], tcg_op); - } - } - } - } - if (!is_q) { - tcg_res[1] = tcg_constant_i64(0); - } - for (pass = 0; pass < 2; pass++) { - write_vec_element(s, tcg_res[pass], rd, pass, MO_64); - } -} - static void handle_shll(DisasContext *s, bool is_q, int size, int rn, int rd) { /* Implement SHLL and SHLL2 */ @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) handle_2misc_narrow(s, false, opcode, u, is_q, size, rn, rd); return; - case 0x2: /* SADDLP, UADDLP */ - case 0x6: /* SADALP, UADALP */ - if (size == 3) { - unallocated_encoding(s); - return; - } - if (!fp_access_check(s)) { - return; - } - handle_2misc_pairwise(s, opcode, u, is_q, size, rn, rd); - return; case 0x13: /* SHLL, SHLL2 */ if (u == 0 || size == 3) { unallocated_encoding(s); @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) default: case 0x0: /* REV64, REV32 */ case 0x1: /* REV16 */ + case 0x2: /* SADDLP, UADDLP */ case 0x3: /* SUQADD, USQADD */ case 0x4: /* CLS, CLZ */ case 0x5: /* CNT, NOT, RBIT */ + case 0x6: /* SADALP, UADALP */ case 0x7: /* SQABS, SQNEG */ case 0x8: /* CMGT, CMGE */ case 0x9: /* CMEQ, CMLE */ -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> These have generic equivalents: tcg_gen_vec_{add,sub}{16,32}_i64. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-48-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/helper.h | 4 ---- target/arm/tcg/neon_helper.c | 36 --------------------------------- target/arm/tcg/translate-neon.c | 22 ++++++++++---------- 3 files changed, 11 insertions(+), 51 deletions(-) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_1(neon_widen_s8, i64, i32) DEF_HELPER_1(neon_widen_u16, i64, i32) DEF_HELPER_1(neon_widen_s16, i64, i32) -DEF_HELPER_2(neon_addl_u16, i64, i64, i64) -DEF_HELPER_2(neon_addl_u32, i64, i64, i64) DEF_HELPER_FLAGS_1(neon_addlp_s8, TCG_CALL_NO_RWG_SE, i64, i64) DEF_HELPER_FLAGS_1(neon_addlp_s16, TCG_CALL_NO_RWG_SE, i64, i64) -DEF_HELPER_2(neon_subl_u16, i64, i64, i64) -DEF_HELPER_2(neon_subl_u32, i64, i64, i64) DEF_HELPER_3(neon_addl_saturate_s32, i64, env, i64, i64) DEF_HELPER_3(neon_addl_saturate_s64, i64, env, i64, i64) DEF_HELPER_2(neon_abdl_u16, i64, i32, i32) diff --git a/target/arm/tcg/neon_helper.c b/target/arm/tcg/neon_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/neon_helper.c +++ b/target/arm/tcg/neon_helper.c @@ -XXX,XX +XXX,XX @@ uint64_t HELPER(neon_widen_s16)(uint32_t x) return ((uint32_t)(int16_t)x) | (high << 32); } -uint64_t HELPER(neon_addl_u16)(uint64_t a, uint64_t b) -{ - uint64_t mask; - mask = (a ^ b) & 0x8000800080008000ull; - a &= ~0x8000800080008000ull; - b &= ~0x8000800080008000ull; - return (a + b) ^ mask; -} - -uint64_t HELPER(neon_addl_u32)(uint64_t a, uint64_t b) -{ - uint64_t mask; - mask = (a ^ b) & 0x8000000080000000ull; - a &= ~0x8000000080000000ull; - b &= ~0x8000000080000000ull; - return (a + b) ^ mask; -} - /* Pairwise long add: add pairs of adjacent elements into * double-width elements in the result (eg _s8 is an 8x8->16 op) */ @@ -XXX,XX +XXX,XX @@ uint64_t HELPER(neon_addlp_s16)(uint64_t a) return (uint32_t)reslo | (((uint64_t)reshi) << 32); } -uint64_t HELPER(neon_subl_u16)(uint64_t a, uint64_t b) -{ - uint64_t mask; - mask = (a ^ ~b) & 0x8000800080008000ull; - a |= 0x8000800080008000ull; - b &= ~0x8000800080008000ull; - return (a - b) ^ mask; -} - -uint64_t HELPER(neon_subl_u32)(uint64_t a, uint64_t b) -{ - uint64_t mask; - mask = (a ^ ~b) & 0x8000000080000000ull; - a |= 0x8000000080000000ull; - b &= ~0x8000000080000000ull; - return (a - b) ^ mask; -} - uint64_t HELPER(neon_addl_saturate_s32)(CPUARMState *env, uint64_t a, uint64_t b) { uint32_t x, y; diff --git a/target/arm/tcg/translate-neon.c b/target/arm/tcg/translate-neon.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-neon.c +++ b/target/arm/tcg/translate-neon.c @@ -XXX,XX +XXX,XX @@ static bool do_prewiden_3d(DisasContext *s, arg_3diff *a, NULL, NULL, \ }; \ static NeonGenTwo64OpFn * const addfn[] = { \ - gen_helper_neon_##OP##l_u16, \ - gen_helper_neon_##OP##l_u32, \ + tcg_gen_vec_##OP##16_i64, \ + tcg_gen_vec_##OP##32_i64, \ tcg_gen_##OP##_i64, \ NULL, \ }; \ @@ -XXX,XX +XXX,XX @@ static bool do_narrow_3d(DisasContext *s, arg_3diff *a, static bool trans_##INSN##_3d(DisasContext *s, arg_3diff *a) \ { \ static NeonGenTwo64OpFn * const addfn[] = { \ - gen_helper_neon_##OP##l_u16, \ - gen_helper_neon_##OP##l_u32, \ + tcg_gen_vec_##OP##16_i64, \ + tcg_gen_vec_##OP##32_i64, \ tcg_gen_##OP##_i64, \ NULL, \ }; \ @@ -XXX,XX +XXX,XX @@ static bool trans_VABAL_S_3d(DisasContext *s, arg_3diff *a) NULL, }; static NeonGenTwo64OpFn * const addfn[] = { - gen_helper_neon_addl_u16, - gen_helper_neon_addl_u32, + tcg_gen_vec_add16_i64, + tcg_gen_vec_add32_i64, tcg_gen_add_i64, NULL, }; @@ -XXX,XX +XXX,XX @@ static bool trans_VABAL_U_3d(DisasContext *s, arg_3diff *a) NULL, }; static NeonGenTwo64OpFn * const addfn[] = { - gen_helper_neon_addl_u16, - gen_helper_neon_addl_u32, + tcg_gen_vec_add16_i64, + tcg_gen_vec_add32_i64, tcg_gen_add_i64, NULL, }; @@ -XXX,XX +XXX,XX @@ static bool trans_VMULL_U_3d(DisasContext *s, arg_3diff *a) NULL, \ }; \ static NeonGenTwo64OpFn * const accfn[] = { \ - gen_helper_neon_##ACC##l_u16, \ - gen_helper_neon_##ACC##l_u32, \ + tcg_gen_vec_##ACC##16_i64, \ + tcg_gen_vec_##ACC##32_i64, \ tcg_gen_##ACC##_i64, \ NULL, \ }; \ @@ -XXX,XX +XXX,XX @@ static bool trans_VMULL_U_2sc(DisasContext *s, arg_2scalar *a) }; \ static NeonGenTwo64OpFn * const accfn[] = { \ NULL, \ - gen_helper_neon_##ACC##l_u32, \ + tcg_gen_vec_##ACC##32_i64, \ tcg_gen_##ACC##_i64, \ NULL, \ }; \ -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> In a couple of places, clearing the entire vector before storing one element is the easiest solution. Wrap that into a helper function. Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-49-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/translate-a64.c | 21 ++++++++++++--------- 1 file changed, 12 insertions(+), 9 deletions(-) diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static TCGv_i32 read_fp_hreg(DisasContext *s, int reg) return v; } -/* Clear the bits above an N-bit vector, for N = (is_q ? 128 : 64). +static void clear_vec(DisasContext *s, int rd) +{ + unsigned ofs = fp_reg_offset(s, rd, MO_64); + unsigned vsz = vec_full_reg_size(s); + + tcg_gen_gvec_dup_imm(MO_64, ofs, vsz, vsz, 0); +} + +/* + * Clear the bits above an N-bit vector, for N = (is_q ? 128 : 64). * If SVE is not enabled, then there are only 128 bits in the vector. */ static void clear_vec_high(DisasContext *s, bool is_q, int rd) @@ -XXX,XX +XXX,XX @@ static bool trans_SM3SS1(DisasContext *s, arg_SM3SS1 *a) TCGv_i32 tcg_op2 = tcg_temp_new_i32(); TCGv_i32 tcg_op3 = tcg_temp_new_i32(); TCGv_i32 tcg_res = tcg_temp_new_i32(); - unsigned vsz, dofs; read_vec_element_i32(s, tcg_op1, a->rn, 3, MO_32); read_vec_element_i32(s, tcg_op2, a->rm, 3, MO_32); @@ -XXX,XX +XXX,XX @@ static bool trans_SM3SS1(DisasContext *s, arg_SM3SS1 *a) tcg_gen_rotri_i32(tcg_res, tcg_res, 25); /* Clear the whole register first, then store bits [127:96]. */ - vsz = vec_full_reg_size(s); - dofs = vec_full_reg_offset(s, a->rd); - tcg_gen_gvec_dup_imm(MO_64, dofs, vsz, vsz, 0); + clear_vec(s, a->rd); write_vec_element_i32(s, tcg_res, a->rd, 3, MO_32); } return true; @@ -XXX,XX +XXX,XX @@ static bool do_scalar_muladd_widening_idx(DisasContext *s, arg_rrx_e *a, TCGv_i64 t0 = tcg_temp_new_i64(); TCGv_i64 t1 = tcg_temp_new_i64(); TCGv_i64 t2 = tcg_temp_new_i64(); - unsigned vsz, dofs; if (acc) { read_vec_element(s, t0, a->rd, 0, a->esz + 1); @@ -XXX,XX +XXX,XX @@ static bool do_scalar_muladd_widening_idx(DisasContext *s, arg_rrx_e *a, fn(t0, t1, t2); /* Clear the whole register first, then store scalar. */ - vsz = vec_full_reg_size(s); - dofs = vec_full_reg_offset(s, a->rd); - tcg_gen_gvec_dup_imm(MO_64, dofs, vsz, vsz, 0); + clear_vec(s, a->rd); write_vec_element(s, t0, a->rd, 0, a->esz + 1); } return true; -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-50-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 9 ++ target/arm/tcg/translate-a64.c | 153 ++++++++++++++++++++------------- 2 files changed, 102 insertions(+), 60 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ CMEQ0_s 0101 1110 111 00000 10011 0 ..... ..... @rr CMLE0_s 0111 1110 111 00000 10011 0 ..... ..... @rr CMLT0_s 0101 1110 111 00000 10101 0 ..... ..... @rr +SQXTUN_s 0111 1110 ..1 00001 00101 0 ..... ..... @rr_e +SQXTN_s 0101 1110 ..1 00001 01001 0 ..... ..... @rr_e +UQXTN_s 0111 1110 ..1 00001 01001 0 ..... ..... @rr_e + # Advanced SIMD two-register miscellaneous SQABS_v 0.00 1110 ..1 00000 01111 0 ..... ..... @qrr_e @@ -XXX,XX +XXX,XX @@ SADDLP_v 0.00 1110 ..1 00000 00101 0 ..... ..... @qrr_e UADDLP_v 0.10 1110 ..1 00000 00101 0 ..... ..... @qrr_e SADALP_v 0.00 1110 ..1 00000 01101 0 ..... ..... @qrr_e UADALP_v 0.10 1110 ..1 00000 01101 0 ..... ..... @qrr_e + +XTN 0.00 1110 ..1 00001 00101 0 ..... ..... @qrr_e +SQXTUN_v 0.10 1110 ..1 00001 00101 0 ..... ..... @qrr_e +SQXTN_v 0.00 1110 ..1 00001 01001 0 ..... ..... @qrr_e +UQXTN_v 0.10 1110 ..1 00001 01001 0 ..... ..... @qrr_e diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ TRANS(CMLE0_s, do_cmop0_d, a, TCG_COND_LE) TRANS(CMLT0_s, do_cmop0_d, a, TCG_COND_LT) TRANS(CMEQ0_s, do_cmop0_d, a, TCG_COND_EQ) +static bool do_2misc_narrow_scalar(DisasContext *s, arg_rr_e *a, + ArithOneOp * const fn[3]) +{ + if (a->esz == MO_64) { + return false; + } + if (fp_access_check(s)) { + TCGv_i64 t = tcg_temp_new_i64(); + + read_vec_element(s, t, a->rn, 0, a->esz + 1); + fn[a->esz](t, t); + clear_vec(s, a->rd); + write_vec_element(s, t, a->rd, 0, a->esz); + } + return true; +} + +#define WRAP_ENV(NAME) \ + static void gen_##NAME(TCGv_i64 d, TCGv_i64 n) \ + { gen_helper_##NAME(d, tcg_env, n); } + +WRAP_ENV(neon_unarrow_sat8) +WRAP_ENV(neon_unarrow_sat16) +WRAP_ENV(neon_unarrow_sat32) + +static ArithOneOp * const f_scalar_sqxtun[] = { + gen_neon_unarrow_sat8, + gen_neon_unarrow_sat16, + gen_neon_unarrow_sat32, +}; +TRANS(SQXTUN_s, do_2misc_narrow_scalar, a, f_scalar_sqxtun) + +WRAP_ENV(neon_narrow_sat_s8) +WRAP_ENV(neon_narrow_sat_s16) +WRAP_ENV(neon_narrow_sat_s32) + +static ArithOneOp * const f_scalar_sqxtn[] = { + gen_neon_narrow_sat_s8, + gen_neon_narrow_sat_s16, + gen_neon_narrow_sat_s32, +}; +TRANS(SQXTN_s, do_2misc_narrow_scalar, a, f_scalar_sqxtn) + +WRAP_ENV(neon_narrow_sat_u8) +WRAP_ENV(neon_narrow_sat_u16) +WRAP_ENV(neon_narrow_sat_u32) + +static ArithOneOp * const f_scalar_uqxtn[] = { + gen_neon_narrow_sat_u8, + gen_neon_narrow_sat_u16, + gen_neon_narrow_sat_u32, +}; +TRANS(UQXTN_s, do_2misc_narrow_scalar, a, f_scalar_uqxtn) + +#undef WRAP_ENV + static bool do_gvec_fn2(DisasContext *s, arg_qrr_e *a, GVecGen2Fn *fn) { if (!a->q && a->esz == MO_64) { @@ -XXX,XX +XXX,XX @@ TRANS(UADDLP_v, do_gvec_fn2_bhs, a, gen_gvec_uaddlp) TRANS(SADALP_v, do_gvec_fn2_bhs, a, gen_gvec_sadalp) TRANS(UADALP_v, do_gvec_fn2_bhs, a, gen_gvec_uadalp) +static bool do_2misc_narrow_vector(DisasContext *s, arg_qrr_e *a, + ArithOneOp * const fn[3]) +{ + if (a->esz == MO_64) { + return false; + } + if (fp_access_check(s)) { + TCGv_i64 t0 = tcg_temp_new_i64(); + TCGv_i64 t1 = tcg_temp_new_i64(); + + read_vec_element(s, t0, a->rn, 0, MO_64); + read_vec_element(s, t1, a->rn, 1, MO_64); + fn[a->esz](t0, t0); + fn[a->esz](t1, t1); + write_vec_element(s, t0, a->rd, a->q ? 2 : 0, MO_32); + write_vec_element(s, t1, a->rd, a->q ? 3 : 1, MO_32); + clear_vec_high(s, a->q, a->rd); + } + return true; +} + +static ArithOneOp * const f_scalar_xtn[] = { + gen_helper_neon_narrow_u8, + gen_helper_neon_narrow_u16, + tcg_gen_ext32u_i64, +}; +TRANS(XTN, do_2misc_narrow_vector, a, f_scalar_xtn) +TRANS(SQXTUN_v, do_2misc_narrow_vector, a, f_scalar_sqxtun) +TRANS(SQXTN_v, do_2misc_narrow_vector, a, f_scalar_sqxtn) +TRANS(UQXTN_v, do_2misc_narrow_vector, a, f_scalar_uqxtn) + /* Common vector code for handling integer to FP conversion */ static void handle_simd_intfp_conv(DisasContext *s, int rd, int rn, int elements, int is_signed, @@ -XXX,XX +XXX,XX @@ static void handle_2misc_narrow(DisasContext *s, bool scalar, tcg_res[pass] = tcg_temp_new_i64(); switch (opcode) { - case 0x12: /* XTN, SQXTUN */ - { - static NeonGenOne64OpFn * const xtnfns[3] = { - gen_helper_neon_narrow_u8, - gen_helper_neon_narrow_u16, - tcg_gen_ext32u_i64, - }; - static NeonGenOne64OpEnvFn * const sqxtunfns[3] = { - gen_helper_neon_unarrow_sat8, - gen_helper_neon_unarrow_sat16, - gen_helper_neon_unarrow_sat32, - }; - if (u) { - genenvfn = sqxtunfns[size]; - } else { - genfn = xtnfns[size]; - } - break; - } - case 0x14: /* SQXTN, UQXTN */ - { - static NeonGenOne64OpEnvFn * const fns[3][2] = { - { gen_helper_neon_narrow_sat_s8, - gen_helper_neon_narrow_sat_u8 }, - { gen_helper_neon_narrow_sat_s16, - gen_helper_neon_narrow_sat_u16 }, - { gen_helper_neon_narrow_sat_s32, - gen_helper_neon_narrow_sat_u32 }, - }; - genenvfn = fns[size][u]; - break; - } case 0x16: /* FCVTN, FCVTN2 */ /* 32 bit to 16 bit or 64 bit to 32 bit float conversion */ if (size == 2) { @@ -XXX,XX +XXX,XX @@ static void handle_2misc_narrow(DisasContext *s, bool scalar, } break; default: + case 0x12: /* XTN, SQXTUN */ + case 0x14: /* SQXTN, UQXTN */ g_assert_not_reached(); } @@ -XXX,XX +XXX,XX @@ static void disas_simd_scalar_two_reg_misc(DisasContext *s, uint32_t insn) TCGv_ptr tcg_fpstatus; switch (opcode) { - case 0x12: /* SQXTUN */ - if (!u) { - unallocated_encoding(s); - return; - } - /* fall through */ - case 0x14: /* SQXTN, UQXTN */ - if (size == 3) { - unallocated_encoding(s); - return; - } - if (!fp_access_check(s)) { - return; - } - handle_2misc_narrow(s, true, opcode, u, false, size, rn, rd); - return; case 0xc ... 0xf: case 0x16 ... 0x1d: case 0x1f: @@ -XXX,XX +XXX,XX @@ static void disas_simd_scalar_two_reg_misc(DisasContext *s, uint32_t insn) case 0x9: /* CMEQ, CMLE */ case 0xa: /* CMLT */ case 0xb: /* ABS, NEG */ + case 0x12: /* SQXTUN */ + case 0x14: /* SQXTN, UQXTN */ unallocated_encoding(s); return; } @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) TCGv_ptr tcg_fpstatus; switch (opcode) { - case 0x12: /* XTN, XTN2, SQXTUN, SQXTUN2 */ - case 0x14: /* SQXTN, SQXTN2, UQXTN, UQXTN2 */ - if (size == 3) { - unallocated_encoding(s); - return; - } - if (!fp_access_check(s)) { - return; - } - - handle_2misc_narrow(s, false, opcode, u, is_q, size, rn, rd); - return; case 0x13: /* SHLL, SHLL2 */ if (u == 0 || size == 3) { unallocated_encoding(s); @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) case 0x9: /* CMEQ, CMLE */ case 0xa: /* CMLT */ case 0xb: /* ABS, NEG */ + case 0x12: /* XTN, XTN2, SQXTUN, SQXTUN2 */ + case 0x14: /* SQXTN, SQXTN2, UQXTN, UQXTN2 */ unallocated_encoding(s); return; } -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-51-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 5 ++ target/arm/tcg/translate-a64.c | 89 ++++++++++++++++++---------------- 2 files changed, 52 insertions(+), 42 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ %rd 0:5 %esz_sd 22:1 !function=plus_2 +%esz_hs 22:1 !function=plus_1 %esz_hsd 22:2 !function=xor_2 %hl 11:1 21:1 %hlm 11:1 20:2 @@ -XXX,XX +XXX,XX @@ @qrr_b . q:1 ...... .. ...... ...... rn:5 rd:5 &qrr_e esz=0 @qrr_h . q:1 ...... .. ...... ...... rn:5 rd:5 &qrr_e esz=1 @qrr_bh . q:1 ...... . esz:1 ...... ...... rn:5 rd:5 &qrr_e +@qrr_hs . q:1 ...... .. ...... ...... rn:5 rd:5 &qrr_e esz=%esz_hs @qrr_e . q:1 ...... esz:2 ...... ...... rn:5 rd:5 &qrr_e @qrrr_b . q:1 ...... ... rm:5 ...... rn:5 rd:5 &qrrr_e esz=0 @@ -XXX,XX +XXX,XX @@ XTN 0.00 1110 ..1 00001 00101 0 ..... ..... @qrr_e SQXTUN_v 0.10 1110 ..1 00001 00101 0 ..... ..... @qrr_e SQXTN_v 0.00 1110 ..1 00001 01001 0 ..... ..... @qrr_e UQXTN_v 0.10 1110 ..1 00001 01001 0 ..... ..... @qrr_e + +FCVTN_v 0.00 1110 0.1 00001 01101 0 ..... ..... @qrr_hs +BFCVTN_v 0.00 1110 101 00001 01101 0 ..... ..... @qrr_h diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ TRANS(SQXTUN_v, do_2misc_narrow_vector, a, f_scalar_sqxtun) TRANS(SQXTN_v, do_2misc_narrow_vector, a, f_scalar_sqxtn) TRANS(UQXTN_v, do_2misc_narrow_vector, a, f_scalar_uqxtn) +static void gen_fcvtn_hs(TCGv_i64 d, TCGv_i64 n) +{ + TCGv_i32 tcg_lo = tcg_temp_new_i32(); + TCGv_i32 tcg_hi = tcg_temp_new_i32(); + TCGv_ptr fpst = fpstatus_ptr(FPST_FPCR); + TCGv_i32 ahp = get_ahp_flag(); + + tcg_gen_extr_i64_i32(tcg_lo, tcg_hi, n); + gen_helper_vfp_fcvt_f32_to_f16(tcg_lo, tcg_lo, fpst, ahp); + gen_helper_vfp_fcvt_f32_to_f16(tcg_hi, tcg_hi, fpst, ahp); + tcg_gen_deposit_i32(tcg_lo, tcg_lo, tcg_hi, 16, 16); + tcg_gen_extu_i32_i64(d, tcg_lo); +} + +static void gen_fcvtn_sd(TCGv_i64 d, TCGv_i64 n) +{ + TCGv_i32 tmp = tcg_temp_new_i32(); + gen_helper_vfp_fcvtsd(tmp, n, tcg_env); + tcg_gen_extu_i32_i64(d, tmp); +} + +static ArithOneOp * const f_vector_fcvtn[] = { + NULL, + gen_fcvtn_hs, + gen_fcvtn_sd, +}; +TRANS(FCVTN_v, do_2misc_narrow_vector, a, f_vector_fcvtn) + +static void gen_bfcvtn_hs(TCGv_i64 d, TCGv_i64 n) +{ + TCGv_ptr fpst = fpstatus_ptr(FPST_FPCR); + TCGv_i32 tmp = tcg_temp_new_i32(); + gen_helper_bfcvt_pair(tmp, n, fpst); + tcg_gen_extu_i32_i64(d, tmp); +} + +static ArithOneOp * const f_vector_bfcvtn[] = { + NULL, + gen_bfcvtn_hs, + NULL, +}; +TRANS_FEAT(BFCVTN_v, aa64_bf16, do_2misc_narrow_vector, a, f_vector_bfcvtn) + /* Common vector code for handling integer to FP conversion */ static void handle_simd_intfp_conv(DisasContext *s, int rd, int rn, int elements, int is_signed, @@ -XXX,XX +XXX,XX @@ static void handle_2misc_narrow(DisasContext *s, bool scalar, tcg_res[pass] = tcg_temp_new_i64(); switch (opcode) { - case 0x16: /* FCVTN, FCVTN2 */ - /* 32 bit to 16 bit or 64 bit to 32 bit float conversion */ - if (size == 2) { - TCGv_i32 tmp = tcg_temp_new_i32(); - gen_helper_vfp_fcvtsd(tmp, tcg_op, tcg_env); - tcg_gen_extu_i32_i64(tcg_res[pass], tmp); - } else { - TCGv_i32 tcg_lo = tcg_temp_new_i32(); - TCGv_i32 tcg_hi = tcg_temp_new_i32(); - TCGv_ptr fpst = fpstatus_ptr(FPST_FPCR); - TCGv_i32 ahp = get_ahp_flag(); - - tcg_gen_extr_i64_i32(tcg_lo, tcg_hi, tcg_op); - gen_helper_vfp_fcvt_f32_to_f16(tcg_lo, tcg_lo, fpst, ahp); - gen_helper_vfp_fcvt_f32_to_f16(tcg_hi, tcg_hi, fpst, ahp); - tcg_gen_deposit_i32(tcg_lo, tcg_lo, tcg_hi, 16, 16); - tcg_gen_extu_i32_i64(tcg_res[pass], tcg_lo); - } - break; - case 0x36: /* BFCVTN, BFCVTN2 */ - { - TCGv_ptr fpst = fpstatus_ptr(FPST_FPCR); - TCGv_i32 tmp = tcg_temp_new_i32(); - gen_helper_bfcvt_pair(tmp, tcg_op, fpst); - tcg_gen_extu_i32_i64(tcg_res[pass], tmp); - } - break; case 0x56: /* FCVTXN, FCVTXN2 */ { /* @@ -XXX,XX +XXX,XX @@ static void handle_2misc_narrow(DisasContext *s, bool scalar, default: case 0x12: /* XTN, SQXTUN */ case 0x14: /* SQXTN, UQXTN */ + case 0x16: /* FCVTN, FCVTN2 */ + case 0x36: /* BFCVTN, BFCVTN2 */ g_assert_not_reached(); } @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) unallocated_encoding(s); return; } - /* fall through */ - case 0x16: /* FCVTN, FCVTN2 */ - /* handle_2misc_narrow does a 2*size -> size operation, but these - * instructions encode the source size rather than dest size. - */ - if (!fp_access_check(s)) { - return; - } - handle_2misc_narrow(s, false, opcode, 0, is_q, size - 1, rn, rd); - return; - case 0x36: /* BFCVTN, BFCVTN2 */ - if (!dc_isar_feature(aa64_bf16, s) || size != 2) { - unallocated_encoding(s); - return; - } if (!fp_access_check(s)) { return; } @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) } break; default: + case 0x16: /* FCVTN, FCVTN2 */ + case 0x36: /* BFCVTN, BFCVTN2 */ unallocated_encoding(s); return; } -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Remove handle_2misc_narrow as this was the last insn decoded by that function. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-52-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 4 ++ target/arm/tcg/translate-a64.c | 101 +++++++-------------------------- 2 files changed, 24 insertions(+), 81 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ @qrr_b . q:1 ...... .. ...... ...... rn:5 rd:5 &qrr_e esz=0 @qrr_h . q:1 ...... .. ...... ...... rn:5 rd:5 &qrr_e esz=1 +@qrr_s . q:1 ...... .. ...... ...... rn:5 rd:5 &qrr_e esz=2 @qrr_bh . q:1 ...... . esz:1 ...... ...... rn:5 rd:5 &qrr_e @qrr_hs . q:1 ...... .. ...... ...... rn:5 rd:5 &qrr_e esz=%esz_hs @qrr_e . q:1 ...... esz:2 ...... ...... rn:5 rd:5 &qrr_e @@ -XXX,XX +XXX,XX @@ SQXTUN_s 0111 1110 ..1 00001 00101 0 ..... ..... @rr_e SQXTN_s 0101 1110 ..1 00001 01001 0 ..... ..... @rr_e UQXTN_s 0111 1110 ..1 00001 01001 0 ..... ..... @rr_e +FCVTXN_s 0111 1110 011 00001 01101 0 ..... ..... @rr_s + # Advanced SIMD two-register miscellaneous SQABS_v 0.00 1110 ..1 00000 01111 0 ..... ..... @qrr_e @@ -XXX,XX +XXX,XX @@ SQXTN_v 0.00 1110 ..1 00001 01001 0 ..... ..... @qrr_e UQXTN_v 0.10 1110 ..1 00001 01001 0 ..... ..... @qrr_e FCVTN_v 0.00 1110 0.1 00001 01101 0 ..... ..... @qrr_hs +FCVTXN_v 0.10 1110 011 00001 01101 0 ..... ..... @qrr_s BFCVTN_v 0.00 1110 101 00001 01101 0 ..... ..... @qrr_h diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static ArithOneOp * const f_scalar_uqxtn[] = { }; TRANS(UQXTN_s, do_2misc_narrow_scalar, a, f_scalar_uqxtn) +static void gen_fcvtxn_sd(TCGv_i64 d, TCGv_i64 n) +{ + /* + * 64 bit to 32 bit float conversion + * with von Neumann rounding (round to odd) + */ + TCGv_i32 tmp = tcg_temp_new_i32(); + gen_helper_fcvtx_f64_to_f32(tmp, n, tcg_env); + tcg_gen_extu_i32_i64(d, tmp); +} + +static ArithOneOp * const f_scalar_fcvtxn[] = { + NULL, + NULL, + gen_fcvtxn_sd, +}; +TRANS(FCVTXN_s, do_2misc_narrow_scalar, a, f_scalar_fcvtxn) + #undef WRAP_ENV static bool do_gvec_fn2(DisasContext *s, arg_qrr_e *a, GVecGen2Fn *fn) @@ -XXX,XX +XXX,XX @@ static ArithOneOp * const f_vector_fcvtn[] = { gen_fcvtn_sd, }; TRANS(FCVTN_v, do_2misc_narrow_vector, a, f_vector_fcvtn) +TRANS(FCVTXN_v, do_2misc_narrow_vector, a, f_scalar_fcvtxn) static void gen_bfcvtn_hs(TCGv_i64 d, TCGv_i64 n) { @@ -XXX,XX +XXX,XX @@ static void handle_2misc_reciprocal(DisasContext *s, int opcode, } } -static void handle_2misc_narrow(DisasContext *s, bool scalar, - int opcode, bool u, bool is_q, - int size, int rn, int rd) -{ - /* Handle 2-reg-misc ops which are narrowing (so each 2*size element - * in the source becomes a size element in the destination). - */ - int pass; - TCGv_i64 tcg_res[2]; - int destelt = is_q ? 2 : 0; - int passes = scalar ? 1 : 2; - - if (scalar) { - tcg_res[1] = tcg_constant_i64(0); - } - - for (pass = 0; pass < passes; pass++) { - TCGv_i64 tcg_op = tcg_temp_new_i64(); - NeonGenOne64OpFn *genfn = NULL; - NeonGenOne64OpEnvFn *genenvfn = NULL; - - if (scalar) { - read_vec_element(s, tcg_op, rn, pass, size + 1); - } else { - read_vec_element(s, tcg_op, rn, pass, MO_64); - } - tcg_res[pass] = tcg_temp_new_i64(); - - switch (opcode) { - case 0x56: /* FCVTXN, FCVTXN2 */ - { - /* - * 64 bit to 32 bit float conversion - * with von Neumann rounding (round to odd) - */ - TCGv_i32 tmp = tcg_temp_new_i32(); - assert(size == 2); - gen_helper_fcvtx_f64_to_f32(tmp, tcg_op, tcg_env); - tcg_gen_extu_i32_i64(tcg_res[pass], tmp); - } - break; - default: - case 0x12: /* XTN, SQXTUN */ - case 0x14: /* SQXTN, UQXTN */ - case 0x16: /* FCVTN, FCVTN2 */ - case 0x36: /* BFCVTN, BFCVTN2 */ - g_assert_not_reached(); - } - - if (genfn) { - genfn(tcg_res[pass], tcg_op); - } else if (genenvfn) { - genenvfn(tcg_res[pass], tcg_env, tcg_op); - } - } - - for (pass = 0; pass < 2; pass++) { - write_vec_element(s, tcg_res[pass], rd, destelt + pass, MO_32); - } - clear_vec_high(s, is_q, rd); -} - /* AdvSIMD scalar two reg misc * 31 30 29 28 24 23 22 21 17 16 12 11 10 9 5 4 0 * +-----+---+-----------+------+-----------+--------+-----+------+------+ @@ -XXX,XX +XXX,XX @@ static void disas_simd_scalar_two_reg_misc(DisasContext *s, uint32_t insn) rmode = FPROUNDING_TIEAWAY; break; case 0x56: /* FCVTXN, FCVTXN2 */ - if (size == 2) { - unallocated_encoding(s); - return; - } - if (!fp_access_check(s)) { - return; - } - handle_2misc_narrow(s, true, opcode, u, false, size - 1, rn, rd); - return; default: unallocated_encoding(s); return; @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) } handle_2misc_reciprocal(s, opcode, false, u, is_q, size, rn, rd); return; - case 0x56: /* FCVTXN, FCVTXN2 */ - if (size == 2) { - unallocated_encoding(s); - return; - } - if (!fp_access_check(s)) { - return; - } - handle_2misc_narrow(s, false, opcode, 0, is_q, size - 1, rn, rd); - return; case 0x17: /* FCVTL, FCVTL2 */ if (!fp_access_check(s)) { return; @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) default: case 0x16: /* FCVTN, FCVTN2 */ case 0x36: /* BFCVTN, BFCVTN2 */ + case 0x56: /* FCVTXN, FCVTXN2 */ unallocated_encoding(s); return; } -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-53-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 2 + target/arm/tcg/translate-a64.c | 75 +++++++++++++++++----------------- 2 files changed, 40 insertions(+), 37 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ UQXTN_v 0.10 1110 ..1 00001 01001 0 ..... ..... @qrr_e FCVTN_v 0.00 1110 0.1 00001 01101 0 ..... ..... @qrr_hs FCVTXN_v 0.10 1110 011 00001 01101 0 ..... ..... @qrr_s BFCVTN_v 0.00 1110 101 00001 01101 0 ..... ..... @qrr_h + +SHLL_v 0.10 1110 ..1 00001 00111 0 ..... ..... @qrr_e diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static ArithOneOp * const f_vector_bfcvtn[] = { }; TRANS_FEAT(BFCVTN_v, aa64_bf16, do_2misc_narrow_vector, a, f_vector_bfcvtn) +static bool trans_SHLL_v(DisasContext *s, arg_qrr_e *a) +{ + static NeonGenWidenFn * const widenfns[3] = { + gen_helper_neon_widen_u8, + gen_helper_neon_widen_u16, + tcg_gen_extu_i32_i64, + }; + NeonGenWidenFn *widenfn; + TCGv_i64 tcg_res[2]; + TCGv_i32 tcg_op; + int part, pass; + + if (a->esz == MO_64) { + return false; + } + if (!fp_access_check(s)) { + return true; + } + + tcg_op = tcg_temp_new_i32(); + widenfn = widenfns[a->esz]; + part = a->q ? 2 : 0; + + for (pass = 0; pass < 2; pass++) { + read_vec_element_i32(s, tcg_op, a->rn, part + pass, MO_32); + tcg_res[pass] = tcg_temp_new_i64(); + widenfn(tcg_res[pass], tcg_op); + tcg_gen_shli_i64(tcg_res[pass], tcg_res[pass], 8 << a->esz); + } + + for (pass = 0; pass < 2; pass++) { + write_vec_element(s, tcg_res[pass], a->rd, pass, MO_64); + } + return true; +} + + /* Common vector code for handling integer to FP conversion */ static void handle_simd_intfp_conv(DisasContext *s, int rd, int rn, int elements, int is_signed, @@ -XXX,XX +XXX,XX @@ static void handle_2misc_widening(DisasContext *s, int opcode, bool is_q, } } -static void handle_shll(DisasContext *s, bool is_q, int size, int rn, int rd) -{ - /* Implement SHLL and SHLL2 */ - int pass; - int part = is_q ? 2 : 0; - TCGv_i64 tcg_res[2]; - - for (pass = 0; pass < 2; pass++) { - static NeonGenWidenFn * const widenfns[3] = { - gen_helper_neon_widen_u8, - gen_helper_neon_widen_u16, - tcg_gen_extu_i32_i64, - }; - NeonGenWidenFn *widenfn = widenfns[size]; - TCGv_i32 tcg_op = tcg_temp_new_i32(); - - read_vec_element_i32(s, tcg_op, rn, part + pass, MO_32); - tcg_res[pass] = tcg_temp_new_i64(); - widenfn(tcg_res[pass], tcg_op); - tcg_gen_shli_i64(tcg_res[pass], tcg_res[pass], 8 << size); - } - - for (pass = 0; pass < 2; pass++) { - write_vec_element(s, tcg_res[pass], rd, pass, MO_64); - } -} - /* AdvSIMD two reg misc * 31 30 29 28 24 23 22 21 17 16 12 11 10 9 5 4 0 * +---+---+---+-----------+------+-----------+--------+-----+------+------+ @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) TCGv_ptr tcg_fpstatus; switch (opcode) { - case 0x13: /* SHLL, SHLL2 */ - if (u == 0 || size == 3) { - unallocated_encoding(s); - return; - } - if (!fp_access_check(s)) { - return; - } - handle_shll(s, is_q, size, rn, rd); - return; case 0xc ... 0xf: case 0x16 ... 0x1f: { @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) case 0xa: /* CMLT */ case 0xb: /* ABS, NEG */ case 0x12: /* XTN, XTN2, SQXTUN, SQXTUN2 */ + case 0x13: /* SHLL, SHLL2 */ case 0x14: /* SQXTN, SQXTN2, UQXTN, UQXTN2 */ unallocated_encoding(s); return; -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Move the current implementation out of translate-neon.c, and extend to handle all element sizes. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-54-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/translate.h | 6 ++++++ target/arm/tcg/gengvec.c | 14 ++++++++++++++ target/arm/tcg/translate-neon.c | 20 ++------------------ 3 files changed, 22 insertions(+), 18 deletions(-) diff --git a/target/arm/tcg/translate.h b/target/arm/tcg/translate.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate.h +++ b/target/arm/tcg/translate.h @@ -XXX,XX +XXX,XX @@ void gen_gvec_uaddlp(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, void gen_gvec_uadalp(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, uint32_t opr_sz, uint32_t max_sz); +/* These exclusively manipulate the sign bit. */ +void gen_gvec_fabs(unsigned vece, uint32_t dofs, uint32_t aofs, + uint32_t oprsz, uint32_t maxsz); +void gen_gvec_fneg(unsigned vece, uint32_t dofs, uint32_t aofs, + uint32_t oprsz, uint32_t maxsz); + /* * Forward to the isar_feature_* tests given a DisasContext pointer. */ diff --git a/target/arm/tcg/gengvec.c b/target/arm/tcg/gengvec.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/gengvec.c +++ b/target/arm/tcg/gengvec.c @@ -XXX,XX +XXX,XX @@ void gen_gvec_uadalp(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, assert(vece <= MO_32); tcg_gen_gvec_2(rd_ofs, rn_ofs, opr_sz, max_sz, &g[vece]); } + +void gen_gvec_fabs(unsigned vece, uint32_t dofs, uint32_t aofs, + uint32_t oprsz, uint32_t maxsz) +{ + uint64_t s_bit = 1ull << ((8 << vece) - 1); + tcg_gen_gvec_andi(vece, dofs, aofs, s_bit - 1, oprsz, maxsz); +} + +void gen_gvec_fneg(unsigned vece, uint32_t dofs, uint32_t aofs, + uint32_t oprsz, uint32_t maxsz) +{ + uint64_t s_bit = 1ull << ((8 << vece) - 1); + tcg_gen_gvec_xori(vece, dofs, aofs, s_bit, oprsz, maxsz); +} diff --git a/target/arm/tcg/translate-neon.c b/target/arm/tcg/translate-neon.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-neon.c +++ b/target/arm/tcg/translate-neon.c @@ -XXX,XX +XXX,XX @@ static bool do_2misc(DisasContext *s, arg_2misc *a, NeonGenOneOpFn *fn) return true; } -static void gen_VABS_F(unsigned vece, uint32_t rd_ofs, uint32_t rm_ofs, - uint32_t oprsz, uint32_t maxsz) -{ - tcg_gen_gvec_andi(vece, rd_ofs, rm_ofs, - vece == MO_16 ? 0x7fff : 0x7fffffff, - oprsz, maxsz); -} - static bool trans_VABS_F(DisasContext *s, arg_2misc *a) { if (a->size == MO_16) { @@ -XXX,XX +XXX,XX @@ static bool trans_VABS_F(DisasContext *s, arg_2misc *a) } else if (a->size != MO_32) { return false; } - return do_2misc_vec(s, a, gen_VABS_F); -} - -static void gen_VNEG_F(unsigned vece, uint32_t rd_ofs, uint32_t rm_ofs, - uint32_t oprsz, uint32_t maxsz) -{ - tcg_gen_gvec_xori(vece, rd_ofs, rm_ofs, - vece == MO_16 ? 0x8000 : 0x80000000, - oprsz, maxsz); + return do_2misc_vec(s, a, gen_gvec_fabs); } static bool trans_VNEG_F(DisasContext *s, arg_2misc *a) @@ -XXX,XX +XXX,XX @@ static bool trans_VNEG_F(DisasContext *s, arg_2misc *a) } else if (a->size != MO_32) { return false; } - return do_2misc_vec(s, a, gen_VNEG_F); + return do_2misc_vec(s, a, gen_gvec_fneg); } static bool trans_VRECPE(DisasContext *s, arg_2misc *a) -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-55-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 7 +++++ target/arm/tcg/translate-a64.c | 54 +++++++++++++++------------------- 2 files changed, 31 insertions(+), 30 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ @qrr_s . q:1 ...... .. ...... ...... rn:5 rd:5 &qrr_e esz=2 @qrr_bh . q:1 ...... . esz:1 ...... ...... rn:5 rd:5 &qrr_e @qrr_hs . q:1 ...... .. ...... ...... rn:5 rd:5 &qrr_e esz=%esz_hs +@qrr_sd . q:1 ...... .. ...... ...... rn:5 rd:5 &qrr_e esz=%esz_sd @qrr_e . q:1 ...... esz:2 ...... ...... rn:5 rd:5 &qrr_e @qrrr_b . q:1 ...... ... rm:5 ...... rn:5 rd:5 &qrrr_e esz=0 @@ -XXX,XX +XXX,XX @@ FCVTXN_v 0.10 1110 011 00001 01101 0 ..... ..... @qrr_s BFCVTN_v 0.00 1110 101 00001 01101 0 ..... ..... @qrr_h SHLL_v 0.10 1110 ..1 00001 00111 0 ..... ..... @qrr_e + +FABS_v 0.00 1110 111 11000 11111 0 ..... ..... @qrr_h +FABS_v 0.00 1110 1.1 00000 11111 0 ..... ..... @qrr_sd + +FNEG_v 0.10 1110 111 11000 11111 0 ..... ..... @qrr_h +FNEG_v 0.10 1110 1.1 00000 11111 0 ..... ..... @qrr_sd diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static bool trans_SHLL_v(DisasContext *s, arg_qrr_e *a) return true; } +static bool do_fabs_fneg_v(DisasContext *s, arg_qrr_e *a, GVecGen2Fn *fn) +{ + int check = fp_access_check_vector_hsd(s, a->q, a->esz); + + if (check <= 0) { + return check == 0; + } + + gen_gvec_fn2(s, a->q, a->rd, a->rn, fn, a->esz); + return true; +} + +TRANS(FABS_v, do_fabs_fneg_v, a, gen_gvec_fabs) +TRANS(FNEG_v, do_fabs_fneg_v, a, gen_gvec_fneg) /* Common vector code for handling integer to FP conversion */ static void handle_simd_intfp_conv(DisasContext *s, int rd, int rn, @@ -XXX,XX +XXX,XX @@ static void handle_2misc_64(DisasContext *s, int opcode, bool u, * requires them. */ switch (opcode) { - case 0x2f: /* FABS */ - gen_vfp_absd(tcg_rd, tcg_rn); - break; - case 0x6f: /* FNEG */ - gen_vfp_negd(tcg_rd, tcg_rn); - break; case 0x7f: /* FSQRT */ gen_helper_vfp_sqrtd(tcg_rd, tcg_rn, tcg_fpstatus); break; @@ -XXX,XX +XXX,XX @@ static void handle_2misc_64(DisasContext *s, int opcode, bool u, case 0x9: /* CMEQ, CMLE */ case 0xa: /* CMLT */ case 0xb: /* ABS, NEG */ + case 0x2f: /* FABS */ + case 0x6f: /* FNEG */ g_assert_not_reached(); } } @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) opcode |= (extract32(size, 1, 1) << 5) | (u << 6); size = is_double ? 3 : 2; switch (opcode) { - case 0x2f: /* FABS */ - case 0x6f: /* FNEG */ - if (size == 3 && !is_q) { - unallocated_encoding(s); - return; - } - break; case 0x1d: /* SCVTF */ case 0x5d: /* UCVTF */ { @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) case 0x16: /* FCVTN, FCVTN2 */ case 0x36: /* BFCVTN, BFCVTN2 */ case 0x56: /* FCVTXN, FCVTXN2 */ + case 0x2f: /* FABS */ + case 0x6f: /* FNEG */ unallocated_encoding(s); return; } @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) { /* Special cases for 32 bit elements */ switch (opcode) { - case 0x2f: /* FABS */ - gen_vfp_abss(tcg_res, tcg_op); - break; - case 0x6f: /* FNEG */ - gen_vfp_negs(tcg_res, tcg_op); - break; case 0x7f: /* FSQRT */ gen_helper_vfp_sqrts(tcg_res, tcg_op, tcg_fpstatus); break; @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) break; default: case 0x7: /* SQABS, SQNEG */ + case 0x2f: /* FABS */ + case 0x6f: /* FNEG */ g_assert_not_reached(); } } @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc_fp16(DisasContext *s, uint32_t insn) case 0x7b: /* FCVTZU */ rmode = FPROUNDING_ZERO; break; - case 0x2f: /* FABS */ - case 0x6f: /* FNEG */ - only_in_vector = true; - need_fpst = false; - break; case 0x7d: /* FRSQRTE */ break; case 0x7f: /* FSQRT (vector) */ only_in_vector = true; break; default: + case 0x2f: /* FABS */ + case 0x6f: /* FNEG */ unallocated_encoding(s); return; } @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc_fp16(DisasContext *s, uint32_t insn) case 0x59: /* FRINTX */ gen_helper_advsimd_rinth_exact(tcg_res, tcg_op, tcg_fpstatus); break; - case 0x2f: /* FABS */ - tcg_gen_andi_i32(tcg_res, tcg_op, 0x7fff); - break; - case 0x6f: /* FNEG */ - tcg_gen_xori_i32(tcg_res, tcg_op, 0x8000); - break; case 0x7d: /* FRSQRTE */ gen_helper_rsqrte_f16(tcg_res, tcg_op, tcg_fpstatus); break; @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc_fp16(DisasContext *s, uint32_t insn) gen_helper_vfp_sqrth(tcg_res, tcg_op, tcg_fpstatus); break; default: + case 0x2f: /* FABS */ + case 0x6f: /* FNEG */ g_assert_not_reached(); } -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-56-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 3 ++ target/arm/tcg/translate-a64.c | 69 ++++++++++++++++++++++++---------- 2 files changed, 53 insertions(+), 19 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ FABS_v 0.00 1110 1.1 00000 11111 0 ..... ..... @qrr_sd FNEG_v 0.10 1110 111 11000 11111 0 ..... ..... @qrr_h FNEG_v 0.10 1110 1.1 00000 11111 0 ..... ..... @qrr_sd + +FSQRT_v 0.10 1110 111 11001 11111 0 ..... ..... @qrr_h +FSQRT_v 0.10 1110 1.1 00001 11111 0 ..... ..... @qrr_sd diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static bool do_fabs_fneg_v(DisasContext *s, arg_qrr_e *a, GVecGen2Fn *fn) TRANS(FABS_v, do_fabs_fneg_v, a, gen_gvec_fabs) TRANS(FNEG_v, do_fabs_fneg_v, a, gen_gvec_fneg) +static bool do_fp1_vector(DisasContext *s, arg_qrr_e *a, + const FPScalar1 *f, int rmode) +{ + TCGv_i32 tcg_rmode = NULL; + TCGv_ptr fpst; + int check = fp_access_check_vector_hsd(s, a->q, a->esz); + + if (check <= 0) { + return check == 0; + } + + fpst = fpstatus_ptr(a->esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR); + if (rmode >= 0) { + tcg_rmode = gen_set_rmode(rmode, fpst); + } + + if (a->esz == MO_64) { + TCGv_i64 t64 = tcg_temp_new_i64(); + + for (int pass = 0; pass < 2; ++pass) { + read_vec_element(s, t64, a->rn, pass, MO_64); + f->gen_d(t64, t64, fpst); + write_vec_element(s, t64, a->rd, pass, MO_64); + } + } else { + TCGv_i32 t32 = tcg_temp_new_i32(); + void (*gen)(TCGv_i32, TCGv_i32, TCGv_ptr) + = (a->esz == MO_16 ? f->gen_h : f->gen_s); + + for (int pass = 0, n = (a->q ? 16 : 8) >> a->esz; pass < n; ++pass) { + read_vec_element_i32(s, t32, a->rn, pass, a->esz); + gen(t32, t32, fpst); + write_vec_element_i32(s, t32, a->rd, pass, a->esz); + } + } + clear_vec_high(s, a->q, a->rd); + + if (rmode >= 0) { + gen_restore_rmode(tcg_rmode, fpst); + } + return true; +} + +TRANS(FSQRT_v, do_fp1_vector, a, &f_scalar_fsqrt, -1) + /* Common vector code for handling integer to FP conversion */ static void handle_simd_intfp_conv(DisasContext *s, int rd, int rn, int elements, int is_signed, @@ -XXX,XX +XXX,XX @@ static void handle_2misc_64(DisasContext *s, int opcode, bool u, * requires them. */ switch (opcode) { - case 0x7f: /* FSQRT */ - gen_helper_vfp_sqrtd(tcg_rd, tcg_rn, tcg_fpstatus); - break; case 0x1a: /* FCVTNS */ case 0x1b: /* FCVTMS */ case 0x1c: /* FCVTAS */ @@ -XXX,XX +XXX,XX @@ static void handle_2misc_64(DisasContext *s, int opcode, bool u, case 0xb: /* ABS, NEG */ case 0x2f: /* FABS */ case 0x6f: /* FNEG */ + case 0x7f: /* FSQRT */ g_assert_not_reached(); } } @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) } handle_2misc_fcmp_zero(s, opcode, false, u, is_q, size, rn, rd); return; - case 0x7f: /* FSQRT */ - need_fpstatus = true; - if (size == 3 && !is_q) { - unallocated_encoding(s); - return; - } - break; case 0x1a: /* FCVTNS */ case 0x1b: /* FCVTMS */ case 0x3a: /* FCVTPS */ @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) case 0x56: /* FCVTXN, FCVTXN2 */ case 0x2f: /* FABS */ case 0x6f: /* FNEG */ + case 0x7f: /* FSQRT */ unallocated_encoding(s); return; } @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) { /* Special cases for 32 bit elements */ switch (opcode) { - case 0x7f: /* FSQRT */ - gen_helper_vfp_sqrts(tcg_res, tcg_op, tcg_fpstatus); - break; case 0x1a: /* FCVTNS */ case 0x1b: /* FCVTMS */ case 0x1c: /* FCVTAS */ @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) case 0x7: /* SQABS, SQNEG */ case 0x2f: /* FABS */ case 0x6f: /* FNEG */ + case 0x7f: /* FSQRT */ g_assert_not_reached(); } } @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc_fp16(DisasContext *s, uint32_t insn) break; case 0x7d: /* FRSQRTE */ break; - case 0x7f: /* FSQRT (vector) */ - only_in_vector = true; - break; default: case 0x2f: /* FABS */ case 0x6f: /* FNEG */ + case 0x7f: /* FSQRT (vector) */ unallocated_encoding(s); return; } @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc_fp16(DisasContext *s, uint32_t insn) case 0x7d: /* FRSQRTE */ gen_helper_rsqrte_f16(tcg_res, tcg_op, tcg_fpstatus); break; - case 0x7f: /* FSQRT */ - gen_helper_vfp_sqrth(tcg_res, tcg_op, tcg_fpstatus); - break; default: case 0x2f: /* FABS */ case 0x6f: /* FNEG */ + case 0x7f: /* FSQRT */ g_assert_not_reached(); } -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-57-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 26 +++++ target/arm/tcg/translate-a64.c | 176 ++++++++++++--------------------- 2 files changed, 88 insertions(+), 114 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ FNEG_v 0.10 1110 1.1 00000 11111 0 ..... ..... @qrr_sd FSQRT_v 0.10 1110 111 11001 11111 0 ..... ..... @qrr_h FSQRT_v 0.10 1110 1.1 00001 11111 0 ..... ..... @qrr_sd + +FRINTN_v 0.00 1110 011 11001 10001 0 ..... ..... @qrr_h +FRINTN_v 0.00 1110 0.1 00001 10001 0 ..... ..... @qrr_sd + +FRINTM_v 0.00 1110 011 11001 10011 0 ..... ..... @qrr_h +FRINTM_v 0.00 1110 0.1 00001 10011 0 ..... ..... @qrr_sd + +FRINTP_v 0.00 1110 111 11001 10001 0 ..... ..... @qrr_h +FRINTP_v 0.00 1110 1.1 00001 10001 0 ..... ..... @qrr_sd + +FRINTZ_v 0.00 1110 111 11001 10011 0 ..... ..... @qrr_h +FRINTZ_v 0.00 1110 1.1 00001 10011 0 ..... ..... @qrr_sd + +FRINTA_v 0.10 1110 011 11001 10001 0 ..... ..... @qrr_h +FRINTA_v 0.10 1110 0.1 00001 10001 0 ..... ..... @qrr_sd + +FRINTX_v 0.10 1110 011 11001 10011 0 ..... ..... @qrr_h +FRINTX_v 0.10 1110 0.1 00001 10011 0 ..... ..... @qrr_sd + +FRINTI_v 0.10 1110 111 11001 10011 0 ..... ..... @qrr_h +FRINTI_v 0.10 1110 1.1 00001 10011 0 ..... ..... @qrr_sd + +FRINT32Z_v 0.00 1110 0.1 00001 11101 0 ..... ..... @qrr_sd +FRINT32X_v 0.10 1110 0.1 00001 11101 0 ..... ..... @qrr_sd +FRINT64Z_v 0.00 1110 0.1 00001 11111 0 ..... ..... @qrr_sd +FRINT64X_v 0.10 1110 0.1 00001 11111 0 ..... ..... @qrr_sd diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static bool do_fp1_vector(DisasContext *s, arg_qrr_e *a, TRANS(FSQRT_v, do_fp1_vector, a, &f_scalar_fsqrt, -1) +TRANS(FRINTN_v, do_fp1_vector, a, &f_scalar_frint, FPROUNDING_TIEEVEN) +TRANS(FRINTP_v, do_fp1_vector, a, &f_scalar_frint, FPROUNDING_POSINF) +TRANS(FRINTM_v, do_fp1_vector, a, &f_scalar_frint, FPROUNDING_NEGINF) +TRANS(FRINTZ_v, do_fp1_vector, a, &f_scalar_frint, FPROUNDING_ZERO) +TRANS(FRINTA_v, do_fp1_vector, a, &f_scalar_frint, FPROUNDING_TIEAWAY) +TRANS(FRINTI_v, do_fp1_vector, a, &f_scalar_frint, -1) +TRANS(FRINTX_v, do_fp1_vector, a, &f_scalar_frintx, -1) + +TRANS_FEAT(FRINT32Z_v, aa64_frint, do_fp1_vector, a, + &f_scalar_frint32, FPROUNDING_ZERO) +TRANS_FEAT(FRINT32X_v, aa64_frint, do_fp1_vector, a, &f_scalar_frint32, -1) +TRANS_FEAT(FRINT64Z_v, aa64_frint, do_fp1_vector, a, + &f_scalar_frint64, FPROUNDING_ZERO) +TRANS_FEAT(FRINT64X_v, aa64_frint, do_fp1_vector, a, &f_scalar_frint64, -1) + /* Common vector code for handling integer to FP conversion */ static void handle_simd_intfp_conv(DisasContext *s, int rd, int rn, int elements, int is_signed, @@ -XXX,XX +XXX,XX @@ static void handle_2misc_64(DisasContext *s, int opcode, bool u, case 0x7b: /* FCVTZU */ gen_helper_vfp_touqd(tcg_rd, tcg_rn, tcg_constant_i32(0), tcg_fpstatus); break; - case 0x18: /* FRINTN */ - case 0x19: /* FRINTM */ - case 0x38: /* FRINTP */ - case 0x39: /* FRINTZ */ - case 0x58: /* FRINTA */ - case 0x79: /* FRINTI */ - gen_helper_rintd(tcg_rd, tcg_rn, tcg_fpstatus); - break; - case 0x59: /* FRINTX */ - gen_helper_rintd_exact(tcg_rd, tcg_rn, tcg_fpstatus); - break; - case 0x1e: /* FRINT32Z */ - case 0x5e: /* FRINT32X */ - gen_helper_frint32_d(tcg_rd, tcg_rn, tcg_fpstatus); - break; - case 0x1f: /* FRINT64Z */ - case 0x5f: /* FRINT64X */ - gen_helper_frint64_d(tcg_rd, tcg_rn, tcg_fpstatus); - break; default: case 0x4: /* CLS, CLZ */ case 0x5: /* NOT */ @@ -XXX,XX +XXX,XX @@ static void handle_2misc_64(DisasContext *s, int opcode, bool u, case 0x2f: /* FABS */ case 0x6f: /* FNEG */ case 0x7f: /* FSQRT */ + case 0x18: /* FRINTN */ + case 0x19: /* FRINTM */ + case 0x38: /* FRINTP */ + case 0x39: /* FRINTZ */ + case 0x58: /* FRINTA */ + case 0x79: /* FRINTI */ + case 0x59: /* FRINTX */ + case 0x1e: /* FRINT32Z */ + case 0x5e: /* FRINT32X */ + case 0x1f: /* FRINT64Z */ + case 0x5f: /* FRINT64X */ g_assert_not_reached(); } } @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) } handle_2misc_widening(s, opcode, is_q, size, rn, rd); return; - case 0x18: /* FRINTN */ - case 0x19: /* FRINTM */ - case 0x38: /* FRINTP */ - case 0x39: /* FRINTZ */ - rmode = extract32(opcode, 5, 1) | (extract32(opcode, 0, 1) << 1); - /* fall through */ - case 0x59: /* FRINTX */ - case 0x79: /* FRINTI */ - need_fpstatus = true; - if (size == 3 && !is_q) { - unallocated_encoding(s); - return; - } - break; - case 0x58: /* FRINTA */ - rmode = FPROUNDING_TIEAWAY; - need_fpstatus = true; - if (size == 3 && !is_q) { - unallocated_encoding(s); - return; - } - break; case 0x7c: /* URSQRTE */ if (size == 3) { unallocated_encoding(s); return; } break; - case 0x1e: /* FRINT32Z */ - case 0x1f: /* FRINT64Z */ - rmode = FPROUNDING_ZERO; - /* fall through */ - case 0x5e: /* FRINT32X */ - case 0x5f: /* FRINT64X */ - need_fpstatus = true; - if ((size == 3 && !is_q) || !dc_isar_feature(aa64_frint, s)) { - unallocated_encoding(s); - return; - } - break; default: case 0x16: /* FCVTN, FCVTN2 */ case 0x36: /* BFCVTN, BFCVTN2 */ @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) case 0x2f: /* FABS */ case 0x6f: /* FNEG */ case 0x7f: /* FSQRT */ + case 0x18: /* FRINTN */ + case 0x19: /* FRINTM */ + case 0x38: /* FRINTP */ + case 0x39: /* FRINTZ */ + case 0x59: /* FRINTX */ + case 0x79: /* FRINTI */ + case 0x58: /* FRINTA */ + case 0x1e: /* FRINT32Z */ + case 0x1f: /* FRINT64Z */ + case 0x5e: /* FRINT32X */ + case 0x5f: /* FRINT64X */ unallocated_encoding(s); return; } @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) gen_helper_vfp_touls(tcg_res, tcg_op, tcg_constant_i32(0), tcg_fpstatus); break; - case 0x18: /* FRINTN */ - case 0x19: /* FRINTM */ - case 0x38: /* FRINTP */ - case 0x39: /* FRINTZ */ - case 0x58: /* FRINTA */ - case 0x79: /* FRINTI */ - gen_helper_rints(tcg_res, tcg_op, tcg_fpstatus); - break; - case 0x59: /* FRINTX */ - gen_helper_rints_exact(tcg_res, tcg_op, tcg_fpstatus); - break; case 0x7c: /* URSQRTE */ gen_helper_rsqrte_u32(tcg_res, tcg_op); break; - case 0x1e: /* FRINT32Z */ - case 0x5e: /* FRINT32X */ - gen_helper_frint32_s(tcg_res, tcg_op, tcg_fpstatus); - break; - case 0x1f: /* FRINT64Z */ - case 0x5f: /* FRINT64X */ - gen_helper_frint64_s(tcg_res, tcg_op, tcg_fpstatus); - break; default: case 0x7: /* SQABS, SQNEG */ case 0x2f: /* FABS */ case 0x6f: /* FNEG */ case 0x7f: /* FSQRT */ + case 0x18: /* FRINTN */ + case 0x19: /* FRINTM */ + case 0x38: /* FRINTP */ + case 0x39: /* FRINTZ */ + case 0x58: /* FRINTA */ + case 0x79: /* FRINTI */ + case 0x59: /* FRINTX */ + case 0x1e: /* FRINT32Z */ + case 0x5e: /* FRINT32X */ + case 0x1f: /* FRINT64Z */ + case 0x5f: /* FRINT64X */ g_assert_not_reached(); } } @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc_fp16(DisasContext *s, uint32_t insn) int rn, rd; bool is_q; bool is_scalar; - bool only_in_vector = false; int pass; TCGv_i32 tcg_rmode = NULL; @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc_fp16(DisasContext *s, uint32_t insn) case 0x3d: /* FRECPE */ case 0x3f: /* FRECPX */ break; - case 0x18: /* FRINTN */ - only_in_vector = true; - rmode = FPROUNDING_TIEEVEN; - break; - case 0x19: /* FRINTM */ - only_in_vector = true; - rmode = FPROUNDING_NEGINF; - break; - case 0x38: /* FRINTP */ - only_in_vector = true; - rmode = FPROUNDING_POSINF; - break; - case 0x39: /* FRINTZ */ - only_in_vector = true; - rmode = FPROUNDING_ZERO; - break; - case 0x58: /* FRINTA */ - only_in_vector = true; - rmode = FPROUNDING_TIEAWAY; - break; - case 0x59: /* FRINTX */ - case 0x79: /* FRINTI */ - only_in_vector = true; - /* current rounding mode */ - break; case 0x1a: /* FCVTNS */ rmode = FPROUNDING_TIEEVEN; break; @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc_fp16(DisasContext *s, uint32_t insn) case 0x2f: /* FABS */ case 0x6f: /* FNEG */ case 0x7f: /* FSQRT (vector) */ + case 0x18: /* FRINTN */ + case 0x19: /* FRINTM */ + case 0x38: /* FRINTP */ + case 0x39: /* FRINTZ */ + case 0x58: /* FRINTA */ + case 0x59: /* FRINTX */ + case 0x79: /* FRINTI */ unallocated_encoding(s); return; } @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc_fp16(DisasContext *s, uint32_t insn) unallocated_encoding(s); return; } - /* FRINTxx is only in the vector form */ - if (only_in_vector) { - unallocated_encoding(s); - return; - } } if (!fp_access_check(s)) { @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc_fp16(DisasContext *s, uint32_t insn) case 0x7b: /* FCVTZU */ gen_helper_advsimd_f16touinth(tcg_res, tcg_op, tcg_fpstatus); break; - case 0x18: /* FRINTN */ - case 0x19: /* FRINTM */ - case 0x38: /* FRINTP */ - case 0x39: /* FRINTZ */ - case 0x58: /* FRINTA */ - case 0x79: /* FRINTI */ - gen_helper_advsimd_rinth(tcg_res, tcg_op, tcg_fpstatus); - break; - case 0x59: /* FRINTX */ - gen_helper_advsimd_rinth_exact(tcg_res, tcg_op, tcg_fpstatus); - break; case 0x7d: /* FRSQRTE */ gen_helper_rsqrte_f16(tcg_res, tcg_op, tcg_fpstatus); break; @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc_fp16(DisasContext *s, uint32_t insn) case 0x2f: /* FABS */ case 0x6f: /* FNEG */ case 0x7f: /* FSQRT */ + case 0x18: /* FRINTN */ + case 0x19: /* FRINTM */ + case 0x38: /* FRINTP */ + case 0x39: /* FRINTZ */ + case 0x58: /* FRINTA */ + case 0x79: /* FRINTI */ + case 0x59: /* FRINTX */ g_assert_not_reached(); } -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Arm silliness with naming, the scalar insns described as part of the vector instructions, as separate from the "regular" scalar insns which output to general registers. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-58-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 30 ++++++++ target/arm/tcg/translate-a64.c | 133 ++++++++++++++------------------- 2 files changed, 86 insertions(+), 77 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ UQXTN_s 0111 1110 ..1 00001 01001 0 ..... ..... @rr_e FCVTXN_s 0111 1110 011 00001 01101 0 ..... ..... @rr_s +@icvt_h . ....... .. ...... ...... rn:5 rd:5 \ + &fcvt sf=0 esz=1 shift=0 +@icvt_sd . ....... .. ...... ...... rn:5 rd:5 \ + &fcvt sf=0 esz=%esz_sd shift=0 + +FCVTNS_f 0101 1110 011 11001 10101 0 ..... ..... @icvt_h +FCVTNS_f 0101 1110 0.1 00001 10101 0 ..... ..... @icvt_sd +FCVTNU_f 0111 1110 011 11001 10101 0 ..... ..... @icvt_h +FCVTNU_f 0111 1110 0.1 00001 10101 0 ..... ..... @icvt_sd + +FCVTPS_f 0101 1110 111 11001 10101 0 ..... ..... @icvt_h +FCVTPS_f 0101 1110 1.1 00001 10101 0 ..... ..... @icvt_sd +FCVTPU_f 0111 1110 111 11001 10101 0 ..... ..... @icvt_h +FCVTPU_f 0111 1110 1.1 00001 10101 0 ..... ..... @icvt_sd + +FCVTMS_f 0101 1110 011 11001 10111 0 ..... ..... @icvt_h +FCVTMS_f 0101 1110 0.1 00001 10111 0 ..... ..... @icvt_sd +FCVTMU_f 0111 1110 011 11001 10111 0 ..... ..... @icvt_h +FCVTMU_f 0111 1110 0.1 00001 10111 0 ..... ..... @icvt_sd + +FCVTZS_f 0101 1110 111 11001 10111 0 ..... ..... @icvt_h +FCVTZS_f 0101 1110 1.1 00001 10111 0 ..... ..... @icvt_sd +FCVTZU_f 0111 1110 111 11001 10111 0 ..... ..... @icvt_h +FCVTZU_f 0111 1110 1.1 00001 10111 0 ..... ..... @icvt_sd + +FCVTAS_f 0101 1110 011 11001 11001 0 ..... ..... @icvt_h +FCVTAS_f 0101 1110 0.1 00001 11001 0 ..... ..... @icvt_sd +FCVTAU_f 0111 1110 011 11001 11001 0 ..... ..... @icvt_h +FCVTAU_f 0111 1110 0.1 00001 11001 0 ..... ..... @icvt_sd + # Advanced SIMD two-register miscellaneous SQABS_v 0.00 1110 ..1 00000 01111 0 ..... ..... @qrr_e diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static void do_fcvt_scalar(DisasContext *s, MemOp out, MemOp esz, tcg_shift, tcg_fpstatus); tcg_gen_extu_i32_i64(tcg_out, tcg_single); break; + case MO_16 | MO_SIGN: + gen_helper_vfp_toshh(tcg_single, tcg_single, + tcg_shift, tcg_fpstatus); + tcg_gen_extu_i32_i64(tcg_out, tcg_single); + break; + case MO_16: + gen_helper_vfp_touhh(tcg_single, tcg_single, + tcg_shift, tcg_fpstatus); + tcg_gen_extu_i32_i64(tcg_out, tcg_single); + break; default: g_assert_not_reached(); } @@ -XXX,XX +XXX,XX @@ TRANS(FCVTZU_g, do_fcvt_g, a, FPROUNDING_ZERO, false) TRANS(FCVTAS_g, do_fcvt_g, a, FPROUNDING_TIEAWAY, true) TRANS(FCVTAU_g, do_fcvt_g, a, FPROUNDING_TIEAWAY, false) +/* + * FCVT* (vector), scalar version. + * Which sounds weird, but really just means output to fp register + * instead of output to general register. Input and output element + * size are always equal. + */ +static bool do_fcvt_f(DisasContext *s, arg_fcvt *a, + ARMFPRounding rmode, bool is_signed) +{ + TCGv_i64 tcg_int; + int check = fp_access_check_scalar_hsd(s, a->esz); + + if (check <= 0) { + return check == 0; + } + + tcg_int = tcg_temp_new_i64(); + do_fcvt_scalar(s, a->esz | (is_signed ? MO_SIGN : 0), + a->esz, tcg_int, a->shift, a->rn, rmode); + + clear_vec(s, a->rd); + write_vec_element(s, tcg_int, a->rd, 0, a->esz); + return true; +} + +TRANS(FCVTNS_f, do_fcvt_f, a, FPROUNDING_TIEEVEN, true) +TRANS(FCVTNU_f, do_fcvt_f, a, FPROUNDING_TIEEVEN, false) +TRANS(FCVTPS_f, do_fcvt_f, a, FPROUNDING_POSINF, true) +TRANS(FCVTPU_f, do_fcvt_f, a, FPROUNDING_POSINF, false) +TRANS(FCVTMS_f, do_fcvt_f, a, FPROUNDING_NEGINF, true) +TRANS(FCVTMU_f, do_fcvt_f, a, FPROUNDING_NEGINF, false) +TRANS(FCVTZS_f, do_fcvt_f, a, FPROUNDING_ZERO, true) +TRANS(FCVTZU_f, do_fcvt_f, a, FPROUNDING_ZERO, false) +TRANS(FCVTAS_f, do_fcvt_f, a, FPROUNDING_TIEAWAY, true) +TRANS(FCVTAU_f, do_fcvt_f, a, FPROUNDING_TIEAWAY, false) + static bool trans_FJCVTZS(DisasContext *s, arg_FJCVTZS *a) { if (!dc_isar_feature(aa64_jscvt, s)) { @@ -XXX,XX +XXX,XX @@ static void disas_simd_scalar_two_reg_misc(DisasContext *s, uint32_t insn) int opcode = extract32(insn, 12, 5); int size = extract32(insn, 22, 2); bool u = extract32(insn, 29, 1); - bool is_fcvt = false; - int rmode; - TCGv_i32 tcg_rmode; - TCGv_ptr tcg_fpstatus; switch (opcode) { case 0xc ... 0xf: @@ -XXX,XX +XXX,XX @@ static void disas_simd_scalar_two_reg_misc(DisasContext *s, uint32_t insn) case 0x5b: /* FCVTMU */ case 0x7a: /* FCVTPU */ case 0x7b: /* FCVTZU */ - is_fcvt = true; - rmode = extract32(opcode, 5, 1) | (extract32(opcode, 0, 1) << 1); - break; case 0x1c: /* FCVTAS */ case 0x5c: /* FCVTAU */ - /* TIEAWAY doesn't fit in the usual rounding mode encoding */ - is_fcvt = true; - rmode = FPROUNDING_TIEAWAY; - break; case 0x56: /* FCVTXN, FCVTXN2 */ default: unallocated_encoding(s); @@ -XXX,XX +XXX,XX @@ static void disas_simd_scalar_two_reg_misc(DisasContext *s, uint32_t insn) unallocated_encoding(s); return; } - - if (!fp_access_check(s)) { - return; - } - - if (is_fcvt) { - tcg_fpstatus = fpstatus_ptr(FPST_FPCR); - tcg_rmode = gen_set_rmode(rmode, tcg_fpstatus); - } else { - tcg_fpstatus = NULL; - tcg_rmode = NULL; - } - - if (size == 3) { - TCGv_i64 tcg_rn = read_fp_dreg(s, rn); - TCGv_i64 tcg_rd = tcg_temp_new_i64(); - - handle_2misc_64(s, opcode, u, tcg_rd, tcg_rn, tcg_rmode, tcg_fpstatus); - write_fp_dreg(s, rd, tcg_rd); - } else { - TCGv_i32 tcg_rn = tcg_temp_new_i32(); - TCGv_i32 tcg_rd = tcg_temp_new_i32(); - - read_vec_element_i32(s, tcg_rn, rn, 0, size); - - switch (opcode) { - case 0x1a: /* FCVTNS */ - case 0x1b: /* FCVTMS */ - case 0x1c: /* FCVTAS */ - case 0x3a: /* FCVTPS */ - case 0x3b: /* FCVTZS */ - gen_helper_vfp_tosls(tcg_rd, tcg_rn, tcg_constant_i32(0), - tcg_fpstatus); - break; - case 0x5a: /* FCVTNU */ - case 0x5b: /* FCVTMU */ - case 0x5c: /* FCVTAU */ - case 0x7a: /* FCVTPU */ - case 0x7b: /* FCVTZU */ - gen_helper_vfp_touls(tcg_rd, tcg_rn, tcg_constant_i32(0), - tcg_fpstatus); - break; - default: - case 0x7: /* SQABS, SQNEG */ - g_assert_not_reached(); - } - - write_fp_sreg(s, rd, tcg_rd); - } - - if (is_fcvt) { - gen_restore_rmode(tcg_rmode, tcg_fpstatus); - } + g_assert_not_reached(); } /* AdvSIMD shift by immediate @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc_fp16(DisasContext *s, uint32_t insn) TCGv_i32 tcg_res = tcg_temp_new_i32(); switch (fpop) { - case 0x1a: /* FCVTNS */ - case 0x1b: /* FCVTMS */ - case 0x1c: /* FCVTAS */ - case 0x3a: /* FCVTPS */ - case 0x3b: /* FCVTZS */ - gen_helper_advsimd_f16tosinth(tcg_res, tcg_op, tcg_fpstatus); - break; case 0x3d: /* FRECPE */ gen_helper_recpe_f16(tcg_res, tcg_op, tcg_fpstatus); break; case 0x3f: /* FRECPX */ gen_helper_frecpx_f16(tcg_res, tcg_op, tcg_fpstatus); break; + case 0x7d: /* FRSQRTE */ + gen_helper_rsqrte_f16(tcg_res, tcg_op, tcg_fpstatus); + break; + default: + case 0x1a: /* FCVTNS */ + case 0x1b: /* FCVTMS */ + case 0x1c: /* FCVTAS */ + case 0x3a: /* FCVTPS */ + case 0x3b: /* FCVTZS */ case 0x5a: /* FCVTNU */ case 0x5b: /* FCVTMU */ case 0x5c: /* FCVTAU */ case 0x7a: /* FCVTPU */ case 0x7b: /* FCVTZU */ - gen_helper_advsimd_f16touinth(tcg_res, tcg_op, tcg_fpstatus); - break; - case 0x7d: /* FRSQRTE */ - gen_helper_rsqrte_f16(tcg_res, tcg_op, tcg_fpstatus); - break; - default: g_assert_not_reached(); } -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-59-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 19 +++++++++++++++++++ target/arm/tcg/translate-a64.c | 4 +--- 2 files changed, 20 insertions(+), 3 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ FCVTAS_f 0101 1110 0.1 00001 11001 0 ..... ..... @icvt_sd FCVTAU_f 0111 1110 011 11001 11001 0 ..... ..... @icvt_h FCVTAU_f 0111 1110 0.1 00001 11001 0 ..... ..... @icvt_sd +%fcvt_f_sh_h 16:4 !function=rsub_16 +%fcvt_f_sh_s 16:5 !function=rsub_32 +%fcvt_f_sh_d 16:6 !function=rsub_64 + +@fcvt_fixed_h .... .... . 001 .... ...... rn:5 rd:5 \ + &fcvt sf=0 esz=1 shift=%fcvt_f_sh_h +@fcvt_fixed_s .... .... . 01 ..... ...... rn:5 rd:5 \ + &fcvt sf=0 esz=2 shift=%fcvt_f_sh_s +@fcvt_fixed_d .... .... . 1 ...... ...... rn:5 rd:5 \ + &fcvt sf=0 esz=3 shift=%fcvt_f_sh_d + +FCVTZS_f 0101 1111 0 ....... 111111 ..... ..... @fcvt_fixed_h +FCVTZS_f 0101 1111 0 ....... 111111 ..... ..... @fcvt_fixed_s +FCVTZS_f 0101 1111 0 ....... 111111 ..... ..... @fcvt_fixed_d + +FCVTZU_f 0111 1111 0 ....... 111111 ..... ..... @fcvt_fixed_h +FCVTZU_f 0111 1111 0 ....... 111111 ..... ..... @fcvt_fixed_s +FCVTZU_f 0111 1111 0 ....... 111111 ..... ..... @fcvt_fixed_d + # Advanced SIMD two-register miscellaneous SQABS_v 0.00 1110 ..1 00000 01111 0 ..... ..... @qrr_e diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static void disas_simd_scalar_shift_imm(DisasContext *s, uint32_t insn) handle_simd_shift_intfp_conv(s, true, false, is_u, immh, immb, opcode, rn, rd); break; - case 0x1f: /* FCVTZS, FCVTZU */ - handle_simd_shift_fpint_conv(s, true, false, is_u, immh, immb, rn, rd); - break; default: case 0x00: /* SSHR / USHR */ case 0x02: /* SSRA / USRA */ @@ -XXX,XX +XXX,XX @@ static void disas_simd_scalar_shift_imm(DisasContext *s, uint32_t insn) case 0x11: /* SQRSHRUN */ case 0x12: /* SQSHRN, UQSHRN */ case 0x13: /* SQRSHRN, UQRSHRN */ + case 0x1f: /* FCVTZS, FCVTZU */ unallocated_encoding(s); break; } -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-60-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 6 ++++++ target/arm/tcg/translate-a64.c | 35 ++++++++++++++++++++++++---------- 2 files changed, 31 insertions(+), 10 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ FCVTXN_s 0111 1110 011 00001 01101 0 ..... ..... @rr_s @icvt_sd . ....... .. ...... ...... rn:5 rd:5 \ &fcvt sf=0 esz=%esz_sd shift=0 +SCVTF_f 0101 1110 011 11001 11011 0 ..... ..... @icvt_h +SCVTF_f 0101 1110 0.1 00001 11011 0 ..... ..... @icvt_sd + +UCVTF_f 0111 1110 011 11001 11011 0 ..... ..... @icvt_h +UCVTF_f 0111 1110 0.1 00001 11011 0 ..... ..... @icvt_sd + FCVTNS_f 0101 1110 011 11001 10101 0 ..... ..... @icvt_h FCVTNS_f 0101 1110 0.1 00001 10101 0 ..... ..... @icvt_sd FCVTNU_f 0111 1110 011 11001 10101 0 ..... ..... @icvt_h diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static bool do_cvtf_g(DisasContext *s, arg_fcvt *a, bool is_signed) TRANS(SCVTF_g, do_cvtf_g, a, true) TRANS(UCVTF_g, do_cvtf_g, a, false) +/* + * [US]CVTF (vector), scalar version. + * Which sounds weird, but really just means input from fp register + * instead of input from general register. Input and output element + * size are always equal. + */ +static bool do_cvtf_f(DisasContext *s, arg_fcvt *a, bool is_signed) +{ + TCGv_i64 tcg_int; + int check = fp_access_check_scalar_hsd(s, a->esz); + + if (check <= 0) { + return check == 0; + } + + tcg_int = tcg_temp_new_i64(); + read_vec_element(s, tcg_int, a->rn, 0, a->esz | (is_signed ? MO_SIGN : 0)); + return do_cvtf_scalar(s, a->esz, a->rd, a->shift, tcg_int, is_signed); +} + +TRANS(SCVTF_f, do_cvtf_f, a, true) +TRANS(UCVTF_f, do_cvtf_f, a, false) + static void do_fcvt_scalar(DisasContext *s, MemOp out, MemOp esz, TCGv_i64 tcg_out, int shift, int rn, ARMFPRounding rmode) @@ -XXX,XX +XXX,XX @@ static void disas_simd_scalar_two_reg_misc(DisasContext *s, uint32_t insn) case 0x6d: /* FCMLE (zero) */ handle_2misc_fcmp_zero(s, opcode, true, u, true, size, rn, rd); return; - case 0x1d: /* SCVTF */ - case 0x5d: /* UCVTF */ - { - bool is_signed = (opcode == 0x1d); - if (!fp_access_check(s)) { - return; - } - handle_simd_intfp_conv(s, rd, rn, 1, is_signed, 0, size); - return; - } case 0x3d: /* FRECPE */ case 0x3f: /* FRECPX */ case 0x7d: /* FRSQRTE */ @@ -XXX,XX +XXX,XX @@ static void disas_simd_scalar_two_reg_misc(DisasContext *s, uint32_t insn) case 0x1c: /* FCVTAS */ case 0x5c: /* FCVTAU */ case 0x56: /* FCVTXN, FCVTXN2 */ + case 0x1d: /* SCVTF */ + case 0x5d: /* UCVTF */ default: unallocated_encoding(s); return; -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Remove disas_simd_scalar_shift_imm as these were the last insns decoded by that function. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-61-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 8 ++++++ target/arm/tcg/translate-a64.c | 47 ---------------------------------- 2 files changed, 8 insertions(+), 47 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ FCVTAU_f 0111 1110 0.1 00001 11001 0 ..... ..... @icvt_sd @fcvt_fixed_d .... .... . 1 ...... ...... rn:5 rd:5 \ &fcvt sf=0 esz=3 shift=%fcvt_f_sh_d +SCVTF_f 0101 1111 0 ....... 111001 ..... ..... @fcvt_fixed_h +SCVTF_f 0101 1111 0 ....... 111001 ..... ..... @fcvt_fixed_s +SCVTF_f 0101 1111 0 ....... 111001 ..... ..... @fcvt_fixed_d + +UCVTF_f 0111 1111 0 ....... 111001 ..... ..... @fcvt_fixed_h +UCVTF_f 0111 1111 0 ....... 111001 ..... ..... @fcvt_fixed_s +UCVTF_f 0111 1111 0 ....... 111001 ..... ..... @fcvt_fixed_d + FCVTZS_f 0101 1111 0 ....... 111111 ..... ..... @fcvt_fixed_h FCVTZS_f 0101 1111 0 ....... 111111 ..... ..... @fcvt_fixed_s FCVTZS_f 0101 1111 0 ....... 111111 ..... ..... @fcvt_fixed_d diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static void handle_simd_shift_fpint_conv(DisasContext *s, bool is_scalar, gen_restore_rmode(tcg_rmode, tcg_fpstatus); } -/* AdvSIMD scalar shift by immediate - * 31 30 29 28 23 22 19 18 16 15 11 10 9 5 4 0 - * +-----+---+-------------+------+------+--------+---+------+------+ - * | 0 1 | U | 1 1 1 1 1 0 | immh | immb | opcode | 1 | Rn | Rd | - * +-----+---+-------------+------+------+--------+---+------+------+ - * - * This is the scalar version so it works on a fixed sized registers - */ -static void disas_simd_scalar_shift_imm(DisasContext *s, uint32_t insn) -{ - int rd = extract32(insn, 0, 5); - int rn = extract32(insn, 5, 5); - int opcode = extract32(insn, 11, 5); - int immb = extract32(insn, 16, 3); - int immh = extract32(insn, 19, 4); - bool is_u = extract32(insn, 29, 1); - - if (immh == 0) { - unallocated_encoding(s); - return; - } - - switch (opcode) { - case 0x1c: /* SCVTF, UCVTF */ - handle_simd_shift_intfp_conv(s, true, false, is_u, immh, immb, - opcode, rn, rd); - break; - default: - case 0x00: /* SSHR / USHR */ - case 0x02: /* SSRA / USRA */ - case 0x04: /* SRSHR / URSHR */ - case 0x06: /* SRSRA / URSRA */ - case 0x08: /* SRI */ - case 0x0a: /* SHL / SLI */ - case 0x0c: /* SQSHLU */ - case 0x0e: /* SQSHL, UQSHL */ - case 0x10: /* SQSHRUN */ - case 0x11: /* SQRSHRUN */ - case 0x12: /* SQSHRN, UQSHRN */ - case 0x13: /* SQRSHRN, UQRSHRN */ - case 0x1f: /* FCVTZS, FCVTZU */ - unallocated_encoding(s); - break; - } -} - static void handle_2misc_64(DisasContext *s, int opcode, bool u, TCGv_i64 tcg_rd, TCGv_i64 tcg_rn, TCGv_i32 tcg_rmode, TCGv_ptr tcg_fpstatus) @@ -XXX,XX +XXX,XX @@ static const AArch64DecodeTable data_proc_simd[] = { { 0x0e200800, 0x9f3e0c00, disas_simd_two_reg_misc }, { 0x0f000400, 0x9f800400, disas_simd_shift_imm }, { 0x5e200800, 0xdf3e0c00, disas_simd_scalar_two_reg_misc }, - { 0x5f000400, 0xdf800400, disas_simd_scalar_shift_imm }, { 0x0e780800, 0x8f7e0c00, disas_simd_two_reg_misc_fp16 }, { 0x00000000, 0x00000000, NULL } }; -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Emphasize that these functions use round-to-zero mode. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-62-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/helper.h | 8 ++++---- target/arm/tcg/translate-neon.c | 8 ++++---- target/arm/tcg/vec_helper.c | 8 ++++---- 3 files changed, 12 insertions(+), 12 deletions(-) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_4(gvec_touizs, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_vcvt_sf, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_vcvt_uf, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(gvec_vcvt_fs, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(gvec_vcvt_fu, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_vcvt_rz_fs, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_vcvt_rz_fu, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_vcvt_sh, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_vcvt_uh, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(gvec_vcvt_hs, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(gvec_vcvt_hu, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_vcvt_rz_hs, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_vcvt_rz_hu, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_vcvt_rm_ss, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_vcvt_rm_us, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) diff --git a/target/arm/tcg/translate-neon.c b/target/arm/tcg/translate-neon.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-neon.c +++ b/target/arm/tcg/translate-neon.c @@ -XXX,XX +XXX,XX @@ static bool do_fp_2sh(DisasContext *s, arg_2reg_shift *a, DO_FP_2SH(VCVT_SF, gen_helper_gvec_vcvt_sf) DO_FP_2SH(VCVT_UF, gen_helper_gvec_vcvt_uf) -DO_FP_2SH(VCVT_FS, gen_helper_gvec_vcvt_fs) -DO_FP_2SH(VCVT_FU, gen_helper_gvec_vcvt_fu) +DO_FP_2SH(VCVT_FS, gen_helper_gvec_vcvt_rz_fs) +DO_FP_2SH(VCVT_FU, gen_helper_gvec_vcvt_rz_fu) DO_FP_2SH(VCVT_SH, gen_helper_gvec_vcvt_sh) DO_FP_2SH(VCVT_UH, gen_helper_gvec_vcvt_uh) -DO_FP_2SH(VCVT_HS, gen_helper_gvec_vcvt_hs) -DO_FP_2SH(VCVT_HU, gen_helper_gvec_vcvt_hu) +DO_FP_2SH(VCVT_HS, gen_helper_gvec_vcvt_rz_hs) +DO_FP_2SH(VCVT_HU, gen_helper_gvec_vcvt_rz_hu) static bool do_1reg_imm(DisasContext *s, arg_1reg_imm *a, GVecGen2iFn *fn) diff --git a/target/arm/tcg/vec_helper.c b/target/arm/tcg/vec_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/vec_helper.c +++ b/target/arm/tcg/vec_helper.c @@ -XXX,XX +XXX,XX @@ DO_3OP_PAIR(gvec_uminp_s, MIN, uint32_t, H4) DO_VCVT_FIXED(gvec_vcvt_sf, helper_vfp_sltos, uint32_t) DO_VCVT_FIXED(gvec_vcvt_uf, helper_vfp_ultos, uint32_t) -DO_VCVT_FIXED(gvec_vcvt_fs, helper_vfp_tosls_round_to_zero, uint32_t) -DO_VCVT_FIXED(gvec_vcvt_fu, helper_vfp_touls_round_to_zero, uint32_t) +DO_VCVT_FIXED(gvec_vcvt_rz_fs, helper_vfp_tosls_round_to_zero, uint32_t) +DO_VCVT_FIXED(gvec_vcvt_rz_fu, helper_vfp_touls_round_to_zero, uint32_t) DO_VCVT_FIXED(gvec_vcvt_sh, helper_vfp_shtoh, uint16_t) DO_VCVT_FIXED(gvec_vcvt_uh, helper_vfp_uhtoh, uint16_t) -DO_VCVT_FIXED(gvec_vcvt_hs, helper_vfp_toshh_round_to_zero, uint16_t) -DO_VCVT_FIXED(gvec_vcvt_hu, helper_vfp_touhh_round_to_zero, uint16_t) +DO_VCVT_FIXED(gvec_vcvt_rz_hs, helper_vfp_toshh_round_to_zero, uint16_t) +DO_VCVT_FIXED(gvec_vcvt_rz_hu, helper_vfp_touhh_round_to_zero, uint16_t) #undef DO_VCVT_FIXED -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Remove handle_simd_intfp_conv and handle_simd_shift_intfp_conv as these were the last insns decoded by those functions. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-63-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/helper.h | 3 + target/arm/tcg/a64.decode | 22 ++++ target/arm/tcg/translate-a64.c | 201 ++++++--------------------------- target/arm/tcg/vec_helper.c | 7 +- 4 files changed, 66 insertions(+), 167 deletions(-) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_4(gvec_vcvt_uh, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_vcvt_rz_hs, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_vcvt_rz_hu, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_vcvt_sd, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_vcvt_ud, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_4(gvec_vcvt_rm_ss, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_vcvt_rm_us, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_vcvt_rm_sh, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ FRINT32Z_v 0.00 1110 0.1 00001 11101 0 ..... ..... @qrr_sd FRINT32X_v 0.10 1110 0.1 00001 11101 0 ..... ..... @qrr_sd FRINT64Z_v 0.00 1110 0.1 00001 11111 0 ..... ..... @qrr_sd FRINT64X_v 0.10 1110 0.1 00001 11111 0 ..... ..... @qrr_sd + +SCVTF_vi 0.00 1110 011 11001 11011 0 ..... ..... @qrr_h +SCVTF_vi 0.00 1110 0.1 00001 11011 0 ..... ..... @qrr_sd + +UCVTF_vi 0.10 1110 011 11001 11011 0 ..... ..... @qrr_h +UCVTF_vi 0.10 1110 0.1 00001 11011 0 ..... ..... @qrr_sd + +&fcvt_q rd rn esz q shift +@fcvtq_h . q:1 . ...... 001 .... ...... rn:5 rd:5 \ + &fcvt_q esz=1 shift=%fcvt_f_sh_h +@fcvtq_s . q:1 . ...... 01 ..... ...... rn:5 rd:5 \ + &fcvt_q esz=2 shift=%fcvt_f_sh_s +@fcvtq_d . q:1 . ...... 1 ...... ...... rn:5 rd:5 \ + &fcvt_q esz=3 shift=%fcvt_f_sh_d + +SCVTF_vf 0.00 11110 ....... 111001 ..... ..... @fcvtq_h +SCVTF_vf 0.00 11110 ....... 111001 ..... ..... @fcvtq_s +SCVTF_vf 0.00 11110 ....... 111001 ..... ..... @fcvtq_d + +UCVTF_vf 0.10 11110 ....... 111001 ..... ..... @fcvtq_h +UCVTF_vf 0.10 11110 ....... 111001 ..... ..... @fcvtq_s +UCVTF_vf 0.10 11110 ....... 111001 ..... ..... @fcvtq_d diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ TRANS_FEAT(FRINT64Z_v, aa64_frint, do_fp1_vector, a, &f_scalar_frint64, FPROUNDING_ZERO) TRANS_FEAT(FRINT64X_v, aa64_frint, do_fp1_vector, a, &f_scalar_frint64, -1) -/* Common vector code for handling integer to FP conversion */ -static void handle_simd_intfp_conv(DisasContext *s, int rd, int rn, - int elements, int is_signed, - int fracbits, int size) +static bool do_gvec_op2_fpst(DisasContext *s, MemOp esz, bool is_q, + int rd, int rn, int data, + gen_helper_gvec_2_ptr * const fns[3]) { - TCGv_ptr tcg_fpst = fpstatus_ptr(size == MO_16 ? FPST_FPCR_F16 : FPST_FPCR); - TCGv_i32 tcg_shift = NULL; + int check = fp_access_check_vector_hsd(s, is_q, esz); + TCGv_ptr fpst; - MemOp mop = size | (is_signed ? MO_SIGN : 0); - int pass; - - if (fracbits || size == MO_64) { - tcg_shift = tcg_constant_i32(fracbits); + if (check <= 0) { + return check == 0; } - if (size == MO_64) { - TCGv_i64 tcg_int64 = tcg_temp_new_i64(); - TCGv_i64 tcg_double = tcg_temp_new_i64(); - - for (pass = 0; pass < elements; pass++) { - read_vec_element(s, tcg_int64, rn, pass, mop); - - if (is_signed) { - gen_helper_vfp_sqtod(tcg_double, tcg_int64, - tcg_shift, tcg_fpst); - } else { - gen_helper_vfp_uqtod(tcg_double, tcg_int64, - tcg_shift, tcg_fpst); - } - if (elements == 1) { - write_fp_dreg(s, rd, tcg_double); - } else { - write_vec_element(s, tcg_double, rd, pass, MO_64); - } - } - } else { - TCGv_i32 tcg_int32 = tcg_temp_new_i32(); - TCGv_i32 tcg_float = tcg_temp_new_i32(); - - for (pass = 0; pass < elements; pass++) { - read_vec_element_i32(s, tcg_int32, rn, pass, mop); - - switch (size) { - case MO_32: - if (fracbits) { - if (is_signed) { - gen_helper_vfp_sltos(tcg_float, tcg_int32, - tcg_shift, tcg_fpst); - } else { - gen_helper_vfp_ultos(tcg_float, tcg_int32, - tcg_shift, tcg_fpst); - } - } else { - if (is_signed) { - gen_helper_vfp_sitos(tcg_float, tcg_int32, tcg_fpst); - } else { - gen_helper_vfp_uitos(tcg_float, tcg_int32, tcg_fpst); - } - } - break; - case MO_16: - if (fracbits) { - if (is_signed) { - gen_helper_vfp_sltoh(tcg_float, tcg_int32, - tcg_shift, tcg_fpst); - } else { - gen_helper_vfp_ultoh(tcg_float, tcg_int32, - tcg_shift, tcg_fpst); - } - } else { - if (is_signed) { - gen_helper_vfp_sitoh(tcg_float, tcg_int32, tcg_fpst); - } else { - gen_helper_vfp_uitoh(tcg_float, tcg_int32, tcg_fpst); - } - } - break; - default: - g_assert_not_reached(); - } - - if (elements == 1) { - write_fp_sreg(s, rd, tcg_float); - } else { - write_vec_element_i32(s, tcg_float, rd, pass, size); - } - } - } - - clear_vec_high(s, elements << size == 16, rd); + fpst = fpstatus_ptr(esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR); + tcg_gen_gvec_2_ptr(vec_full_reg_offset(s, rd), + vec_full_reg_offset(s, rn), fpst, + is_q ? 16 : 8, vec_full_reg_size(s), + data, fns[esz - 1]); + return true; } -/* UCVTF/SCVTF - Integer to FP conversion */ -static void handle_simd_shift_intfp_conv(DisasContext *s, bool is_scalar, - bool is_q, bool is_u, - int immh, int immb, int opcode, - int rn, int rd) -{ - int size, elements, fracbits; - int immhb = immh << 3 | immb; +static gen_helper_gvec_2_ptr * const f_scvtf_v[] = { + gen_helper_gvec_vcvt_sh, + gen_helper_gvec_vcvt_sf, + gen_helper_gvec_vcvt_sd, +}; +TRANS(SCVTF_vi, do_gvec_op2_fpst, + a->esz, a->q, a->rd, a->rn, 0, f_scvtf_v) +TRANS(SCVTF_vf, do_gvec_op2_fpst, + a->esz, a->q, a->rd, a->rn, a->shift, f_scvtf_v) - if (immh & 8) { - size = MO_64; - if (!is_scalar && !is_q) { - unallocated_encoding(s); - return; - } - } else if (immh & 4) { - size = MO_32; - } else if (immh & 2) { - size = MO_16; - if (!dc_isar_feature(aa64_fp16, s)) { - unallocated_encoding(s); - return; - } - } else { - /* immh == 0 would be a failure of the decode logic */ - g_assert(immh == 1); - unallocated_encoding(s); - return; - } - - if (is_scalar) { - elements = 1; - } else { - elements = (8 << is_q) >> size; - } - fracbits = (16 << size) - immhb; - - if (!fp_access_check(s)) { - return; - } - - handle_simd_intfp_conv(s, rd, rn, elements, !is_u, fracbits, size); -} +static gen_helper_gvec_2_ptr * const f_ucvtf_v[] = { + gen_helper_gvec_vcvt_uh, + gen_helper_gvec_vcvt_uf, + gen_helper_gvec_vcvt_ud, +}; +TRANS(UCVTF_vi, do_gvec_op2_fpst, + a->esz, a->q, a->rd, a->rn, 0, f_ucvtf_v) +TRANS(UCVTF_vf, do_gvec_op2_fpst, + a->esz, a->q, a->rd, a->rn, a->shift, f_ucvtf_v) /* FCVTZS, FVCVTZU - FP to fixedpoint conversion */ static void handle_simd_shift_fpint_conv(DisasContext *s, bool is_scalar, @@ -XXX,XX +XXX,XX @@ static void disas_simd_shift_imm(DisasContext *s, uint32_t insn) } switch (opcode) { - case 0x1c: /* SCVTF / UCVTF */ - handle_simd_shift_intfp_conv(s, false, is_q, is_u, immh, immb, - opcode, rn, rd); - break; case 0x1f: /* FCVTZS/ FCVTZU */ handle_simd_shift_fpint_conv(s, false, is_q, is_u, immh, immb, rn, rd); return; @@ -XXX,XX +XXX,XX @@ static void disas_simd_shift_imm(DisasContext *s, uint32_t insn) case 0x12: /* SQSHRN / UQSHRN */ case 0x13: /* SQRSHRN / UQRSHRN */ case 0x14: /* SSHLL / USHLL */ + case 0x1c: /* SCVTF / UCVTF */ unallocated_encoding(s); return; } @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) opcode |= (extract32(size, 1, 1) << 5) | (u << 6); size = is_double ? 3 : 2; switch (opcode) { - case 0x1d: /* SCVTF */ - case 0x5d: /* UCVTF */ - { - bool is_signed = (opcode == 0x1d) ? true : false; - int elements = is_double ? 2 : is_q ? 4 : 2; - if (is_double && !is_q) { - unallocated_encoding(s); - return; - } - if (!fp_access_check(s)) { - return; - } - handle_simd_intfp_conv(s, rd, rn, elements, is_signed, 0, size); - return; - } case 0x2c: /* FCMGT (zero) */ case 0x2d: /* FCMEQ (zero) */ case 0x2e: /* FCMLT (zero) */ @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) case 0x1f: /* FRINT64Z */ case 0x5e: /* FRINT32X */ case 0x5f: /* FRINT64X */ + case 0x1d: /* SCVTF */ + case 0x5d: /* UCVTF */ unallocated_encoding(s); return; } @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc_fp16(DisasContext *s, uint32_t insn) fpop = deposit32(fpop, 6, 1, u); switch (fpop) { - case 0x1d: /* SCVTF */ - case 0x5d: /* UCVTF */ - { - int elements; - - if (is_scalar) { - elements = 1; - } else { - elements = (is_q ? 8 : 4); - } - - if (!fp_access_check(s)) { - return; - } - handle_simd_intfp_conv(s, rd, rn, elements, !u, 0, MO_16); - return; - } - break; case 0x2c: /* FCMGT (zero) */ case 0x2d: /* FCMEQ (zero) */ case 0x2e: /* FCMLT (zero) */ @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc_fp16(DisasContext *s, uint32_t insn) case 0x58: /* FRINTA */ case 0x59: /* FRINTX */ case 0x79: /* FRINTI */ + case 0x1d: /* SCVTF */ + case 0x5d: /* UCVTF */ unallocated_encoding(s); return; } diff --git a/target/arm/tcg/vec_helper.c b/target/arm/tcg/vec_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/vec_helper.c +++ b/target/arm/tcg/vec_helper.c @@ -XXX,XX +XXX,XX @@ DO_3OP_PAIR(gvec_uminp_s, MIN, uint32_t, H4) clear_tail(d, oprsz, simd_maxsz(desc)); \ } +DO_VCVT_FIXED(gvec_vcvt_sd, helper_vfp_sqtod, uint64_t) +DO_VCVT_FIXED(gvec_vcvt_ud, helper_vfp_uqtod, uint64_t) DO_VCVT_FIXED(gvec_vcvt_sf, helper_vfp_sltos, uint32_t) DO_VCVT_FIXED(gvec_vcvt_uf, helper_vfp_ultos, uint32_t) -DO_VCVT_FIXED(gvec_vcvt_rz_fs, helper_vfp_tosls_round_to_zero, uint32_t) -DO_VCVT_FIXED(gvec_vcvt_rz_fu, helper_vfp_touls_round_to_zero, uint32_t) DO_VCVT_FIXED(gvec_vcvt_sh, helper_vfp_shtoh, uint16_t) DO_VCVT_FIXED(gvec_vcvt_uh, helper_vfp_uhtoh, uint16_t) + +DO_VCVT_FIXED(gvec_vcvt_rz_fs, helper_vfp_tosls_round_to_zero, uint32_t) +DO_VCVT_FIXED(gvec_vcvt_rz_fu, helper_vfp_touls_round_to_zero, uint32_t) DO_VCVT_FIXED(gvec_vcvt_rz_hs, helper_vfp_toshh_round_to_zero, uint16_t) DO_VCVT_FIXED(gvec_vcvt_rz_hu, helper_vfp_touhh_round_to_zero, uint16_t) -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Remove handle_simd_shift_fpint_conv and disas_simd_shift_imm as these were the last insns decoded by those functions. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-64-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/helper.h | 4 + target/arm/tcg/a64.decode | 8 ++ target/arm/tcg/translate-a64.c | 160 +++------------------------------ target/arm/tcg/vec_helper.c | 2 + target/arm/vfp_helper.c | 4 + 5 files changed, 32 insertions(+), 146 deletions(-) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_3(vfp_touhs_round_to_zero, i32, f32, i32, ptr) DEF_HELPER_3(vfp_touls_round_to_zero, i32, f32, i32, ptr) DEF_HELPER_3(vfp_toshd_round_to_zero, i64, f64, i32, ptr) DEF_HELPER_3(vfp_tosld_round_to_zero, i64, f64, i32, ptr) +DEF_HELPER_3(vfp_tosqd_round_to_zero, i64, f64, i32, ptr) DEF_HELPER_3(vfp_touhd_round_to_zero, i64, f64, i32, ptr) DEF_HELPER_3(vfp_tould_round_to_zero, i64, f64, i32, ptr) +DEF_HELPER_3(vfp_touqd_round_to_zero, i64, f64, i32, ptr) DEF_HELPER_3(vfp_touhh, i32, f16, i32, ptr) DEF_HELPER_3(vfp_toshh, i32, f16, i32, ptr) DEF_HELPER_3(vfp_toulh, i32, f16, i32, ptr) @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_4(gvec_vcvt_rz_hu, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_vcvt_sd, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_vcvt_ud, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_vcvt_rz_ds, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_vcvt_rz_du, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_vcvt_rm_ss, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_vcvt_rm_us, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ SCVTF_vf 0.00 11110 ....... 111001 ..... ..... @fcvtq_d UCVTF_vf 0.10 11110 ....... 111001 ..... ..... @fcvtq_h UCVTF_vf 0.10 11110 ....... 111001 ..... ..... @fcvtq_s UCVTF_vf 0.10 11110 ....... 111001 ..... ..... @fcvtq_d + +FCVTZS_vf 0.00 11110 ....... 111111 ..... ..... @fcvtq_h +FCVTZS_vf 0.00 11110 ....... 111111 ..... ..... @fcvtq_s +FCVTZS_vf 0.00 11110 ....... 111111 ..... ..... @fcvtq_d + +FCVTZU_vf 0.10 11110 ....... 111111 ..... ..... @fcvtq_h +FCVTZU_vf 0.10 11110 ....... 111111 ..... ..... @fcvtq_s +FCVTZU_vf 0.10 11110 ....... 111111 ..... ..... @fcvtq_d diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ TRANS(UCVTF_vi, do_gvec_op2_fpst, TRANS(UCVTF_vf, do_gvec_op2_fpst, a->esz, a->q, a->rd, a->rn, a->shift, f_ucvtf_v) -/* FCVTZS, FVCVTZU - FP to fixedpoint conversion */ -static void handle_simd_shift_fpint_conv(DisasContext *s, bool is_scalar, - bool is_q, bool is_u, - int immh, int immb, int rn, int rd) -{ - int immhb = immh << 3 | immb; - int pass, size, fracbits; - TCGv_ptr tcg_fpstatus; - TCGv_i32 tcg_rmode, tcg_shift; +static gen_helper_gvec_2_ptr * const f_fcvtzs_vf[] = { + gen_helper_gvec_vcvt_rz_hs, + gen_helper_gvec_vcvt_rz_fs, + gen_helper_gvec_vcvt_rz_ds, +}; +TRANS(FCVTZS_vf, do_gvec_op2_fpst, + a->esz, a->q, a->rd, a->rn, a->shift, f_fcvtzs_vf) - if (immh & 0x8) { - size = MO_64; - if (!is_scalar && !is_q) { - unallocated_encoding(s); - return; - } - } else if (immh & 0x4) { - size = MO_32; - } else if (immh & 0x2) { - size = MO_16; - if (!dc_isar_feature(aa64_fp16, s)) { - unallocated_encoding(s); - return; - } - } else { - /* Should have split out AdvSIMD modified immediate earlier. */ - assert(immh == 1); - unallocated_encoding(s); - return; - } - - if (!fp_access_check(s)) { - return; - } - - assert(!(is_scalar && is_q)); - - tcg_fpstatus = fpstatus_ptr(size == MO_16 ? FPST_FPCR_F16 : FPST_FPCR); - tcg_rmode = gen_set_rmode(FPROUNDING_ZERO, tcg_fpstatus); - fracbits = (16 << size) - immhb; - tcg_shift = tcg_constant_i32(fracbits); - - if (size == MO_64) { - int maxpass = is_scalar ? 1 : 2; - - for (pass = 0; pass < maxpass; pass++) { - TCGv_i64 tcg_op = tcg_temp_new_i64(); - - read_vec_element(s, tcg_op, rn, pass, MO_64); - if (is_u) { - gen_helper_vfp_touqd(tcg_op, tcg_op, tcg_shift, tcg_fpstatus); - } else { - gen_helper_vfp_tosqd(tcg_op, tcg_op, tcg_shift, tcg_fpstatus); - } - write_vec_element(s, tcg_op, rd, pass, MO_64); - } - clear_vec_high(s, is_q, rd); - } else { - void (*fn)(TCGv_i32, TCGv_i32, TCGv_i32, TCGv_ptr); - int maxpass = is_scalar ? 1 : ((8 << is_q) >> size); - - switch (size) { - case MO_16: - if (is_u) { - fn = gen_helper_vfp_touhh; - } else { - fn = gen_helper_vfp_toshh; - } - break; - case MO_32: - if (is_u) { - fn = gen_helper_vfp_touls; - } else { - fn = gen_helper_vfp_tosls; - } - break; - default: - g_assert_not_reached(); - } - - for (pass = 0; pass < maxpass; pass++) { - TCGv_i32 tcg_op = tcg_temp_new_i32(); - - read_vec_element_i32(s, tcg_op, rn, pass, size); - fn(tcg_op, tcg_op, tcg_shift, tcg_fpstatus); - if (is_scalar) { - if (size == MO_16 && !is_u) { - tcg_gen_ext16u_i32(tcg_op, tcg_op); - } - write_fp_sreg(s, rd, tcg_op); - } else { - write_vec_element_i32(s, tcg_op, rd, pass, size); - } - } - if (!is_scalar) { - clear_vec_high(s, is_q, rd); - } - } - - gen_restore_rmode(tcg_rmode, tcg_fpstatus); -} +static gen_helper_gvec_2_ptr * const f_fcvtzu_vf[] = { + gen_helper_gvec_vcvt_rz_hu, + gen_helper_gvec_vcvt_rz_fu, + gen_helper_gvec_vcvt_rz_du, +}; +TRANS(FCVTZU_vf, do_gvec_op2_fpst, + a->esz, a->q, a->rd, a->rn, a->shift, f_fcvtzu_vf) static void handle_2misc_64(DisasContext *s, int opcode, bool u, TCGv_i64 tcg_rd, TCGv_i64 tcg_rn, @@ -XXX,XX +XXX,XX @@ static void disas_simd_scalar_two_reg_misc(DisasContext *s, uint32_t insn) g_assert_not_reached(); } -/* AdvSIMD shift by immediate - * 31 30 29 28 23 22 19 18 16 15 11 10 9 5 4 0 - * +---+---+---+-------------+------+------+--------+---+------+------+ - * | 0 | Q | U | 0 1 1 1 1 0 | immh | immb | opcode | 1 | Rn | Rd | - * +---+---+---+-------------+------+------+--------+---+------+------+ - */ -static void disas_simd_shift_imm(DisasContext *s, uint32_t insn) -{ - int rd = extract32(insn, 0, 5); - int rn = extract32(insn, 5, 5); - int opcode = extract32(insn, 11, 5); - int immb = extract32(insn, 16, 3); - int immh = extract32(insn, 19, 4); - bool is_u = extract32(insn, 29, 1); - bool is_q = extract32(insn, 30, 1); - - if (immh == 0) { - unallocated_encoding(s); - return; - } - - switch (opcode) { - case 0x1f: /* FCVTZS/ FCVTZU */ - handle_simd_shift_fpint_conv(s, false, is_q, is_u, immh, immb, rn, rd); - return; - default: - case 0x00: /* SSHR / USHR */ - case 0x02: /* SSRA / USRA (accumulate) */ - case 0x04: /* SRSHR / URSHR (rounding) */ - case 0x06: /* SRSRA / URSRA (accum + rounding) */ - case 0x08: /* SRI */ - case 0x0a: /* SHL / SLI */ - case 0x0c: /* SQSHLU */ - case 0x0e: /* SQSHL, UQSHL */ - case 0x10: /* SHRN / SQSHRUN */ - case 0x11: /* RSHRN / SQRSHRUN */ - case 0x12: /* SQSHRN / UQSHRN */ - case 0x13: /* SQRSHRN / UQRSHRN */ - case 0x14: /* SSHLL / USHLL */ - case 0x1c: /* SCVTF / UCVTF */ - unallocated_encoding(s); - return; - } -} - static void handle_2misc_widening(DisasContext *s, int opcode, bool is_q, int size, int rn, int rd) { @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc_fp16(DisasContext *s, uint32_t insn) static const AArch64DecodeTable data_proc_simd[] = { /* pattern , mask , fn */ { 0x0e200800, 0x9f3e0c00, disas_simd_two_reg_misc }, - { 0x0f000400, 0x9f800400, disas_simd_shift_imm }, { 0x5e200800, 0xdf3e0c00, disas_simd_scalar_two_reg_misc }, { 0x0e780800, 0x8f7e0c00, disas_simd_two_reg_misc_fp16 }, { 0x00000000, 0x00000000, NULL } diff --git a/target/arm/tcg/vec_helper.c b/target/arm/tcg/vec_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/vec_helper.c +++ b/target/arm/tcg/vec_helper.c @@ -XXX,XX +XXX,XX @@ DO_VCVT_FIXED(gvec_vcvt_uf, helper_vfp_ultos, uint32_t) DO_VCVT_FIXED(gvec_vcvt_sh, helper_vfp_shtoh, uint16_t) DO_VCVT_FIXED(gvec_vcvt_uh, helper_vfp_uhtoh, uint16_t) +DO_VCVT_FIXED(gvec_vcvt_rz_ds, helper_vfp_tosqd_round_to_zero, uint64_t) +DO_VCVT_FIXED(gvec_vcvt_rz_du, helper_vfp_touqd_round_to_zero, uint64_t) DO_VCVT_FIXED(gvec_vcvt_rz_fs, helper_vfp_tosls_round_to_zero, uint32_t) DO_VCVT_FIXED(gvec_vcvt_rz_fu, helper_vfp_touls_round_to_zero, uint32_t) DO_VCVT_FIXED(gvec_vcvt_rz_hs, helper_vfp_toshh_round_to_zero, uint16_t) diff --git a/target/arm/vfp_helper.c b/target/arm/vfp_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vfp_helper.c +++ b/target/arm/vfp_helper.c @@ -XXX,XX +XXX,XX @@ VFP_CONV_FIX_A64(sq, h, 16, dh_ctype_f16, 64, int64) VFP_CONV_FIX(uh, h, 16, dh_ctype_f16, 32, uint16) VFP_CONV_FIX(ul, h, 16, dh_ctype_f16, 32, uint32) VFP_CONV_FIX_A64(uq, h, 16, dh_ctype_f16, 64, uint64) +VFP_CONV_FLOAT_FIX_ROUND(sq, d, 64, float64, 64, int64, + float_round_to_zero, _round_to_zero) +VFP_CONV_FLOAT_FIX_ROUND(uq, d, 64, float64, 64, uint64, + float_round_to_zero, _round_to_zero) #undef VFP_CONV_FIX #undef VFP_CONV_FIX_FLOAT -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Remove handle_2misc_64 as these were the last insns decoded by that function. Remove helper_advsimd_f16to[su]inth as unused; we now always go through helper_vfp_to[su]hh or a specialized vector function instead. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-65-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/helper.h | 2 + target/arm/tcg/helper-a64.h | 2 - target/arm/tcg/a64.decode | 25 ++++ target/arm/tcg/helper-a64.c | 32 ----- target/arm/tcg/translate-a64.c | 227 +++++++++++---------------------- target/arm/tcg/vec_helper.c | 2 + 6 files changed, 102 insertions(+), 188 deletions(-) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_4(gvec_vcvt_ud, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_vcvt_rz_ds, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_vcvt_rz_du, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_vcvt_rm_sd, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_vcvt_rm_ud, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_vcvt_rm_ss, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_vcvt_rm_us, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_vcvt_rm_sh, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) diff --git a/target/arm/tcg/helper-a64.h b/target/arm/tcg/helper-a64.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/helper-a64.h +++ b/target/arm/tcg/helper-a64.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_3(advsimd_mulx2h, i32, i32, i32, ptr) DEF_HELPER_4(advsimd_muladd2h, i32, i32, i32, i32, ptr) DEF_HELPER_2(advsimd_rinth_exact, f16, f16, ptr) DEF_HELPER_2(advsimd_rinth, f16, f16, ptr) -DEF_HELPER_2(advsimd_f16tosinth, i32, f16, ptr) -DEF_HELPER_2(advsimd_f16touinth, i32, f16, ptr) DEF_HELPER_2(exception_return, void, env, i64) DEF_HELPER_FLAGS_2(dc_zva, TCG_CALL_NO_WG, void, env, i64) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ SCVTF_vi 0.00 1110 0.1 00001 11011 0 ..... ..... @qrr_sd UCVTF_vi 0.10 1110 011 11001 11011 0 ..... ..... @qrr_h UCVTF_vi 0.10 1110 0.1 00001 11011 0 ..... ..... @qrr_sd +FCVTNS_vi 0.00 1110 011 11001 10101 0 ..... ..... @qrr_h +FCVTNS_vi 0.00 1110 0.1 00001 10101 0 ..... ..... @qrr_sd +FCVTNU_vi 0.10 1110 011 11001 10101 0 ..... ..... @qrr_h +FCVTNU_vi 0.10 1110 0.1 00001 10101 0 ..... ..... @qrr_sd + +FCVTPS_vi 0.00 1110 111 11001 10101 0 ..... ..... @qrr_h +FCVTPS_vi 0.00 1110 1.1 00001 10101 0 ..... ..... @qrr_sd +FCVTPU_vi 0.10 1110 111 11001 10101 0 ..... ..... @qrr_h +FCVTPU_vi 0.10 1110 1.1 00001 10101 0 ..... ..... @qrr_sd + +FCVTMS_vi 0.00 1110 011 11001 10111 0 ..... ..... @qrr_h +FCVTMS_vi 0.00 1110 0.1 00001 10111 0 ..... ..... @qrr_sd +FCVTMU_vi 0.10 1110 011 11001 10111 0 ..... ..... @qrr_h +FCVTMU_vi 0.10 1110 0.1 00001 10111 0 ..... ..... @qrr_sd + +FCVTZS_vi 0.00 1110 111 11001 10111 0 ..... ..... @qrr_h +FCVTZS_vi 0.00 1110 1.1 00001 10111 0 ..... ..... @qrr_sd +FCVTZU_vi 0.10 1110 111 11001 10111 0 ..... ..... @qrr_h +FCVTZU_vi 0.10 1110 1.1 00001 10111 0 ..... ..... @qrr_sd + +FCVTAS_vi 0.00 1110 011 11001 11001 0 ..... ..... @qrr_h +FCVTAS_vi 0.00 1110 0.1 00001 11001 0 ..... ..... @qrr_sd +FCVTAU_vi 0.10 1110 011 11001 11001 0 ..... ..... @qrr_h +FCVTAU_vi 0.10 1110 0.1 00001 11001 0 ..... ..... @qrr_sd + &fcvt_q rd rn esz q shift @fcvtq_h . q:1 . ...... 001 .... ...... rn:5 rd:5 \ &fcvt_q esz=1 shift=%fcvt_f_sh_h diff --git a/target/arm/tcg/helper-a64.c b/target/arm/tcg/helper-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/helper-a64.c +++ b/target/arm/tcg/helper-a64.c @@ -XXX,XX +XXX,XX @@ uint32_t HELPER(advsimd_rinth)(uint32_t x, void *fp_status) return ret; } -/* - * Half-precision floating point conversion functions - * - * There are a multitude of conversion functions with various - * different rounding modes. This is dealt with by the calling code - * setting the mode appropriately before calling the helper. - */ - -uint32_t HELPER(advsimd_f16tosinth)(uint32_t a, void *fpstp) -{ - float_status *fpst = fpstp; - - /* Invalid if we are passed a NaN */ - if (float16_is_any_nan(a)) { - float_raise(float_flag_invalid, fpst); - return 0; - } - return float16_to_int16(a, fpst); -} - -uint32_t HELPER(advsimd_f16touinth)(uint32_t a, void *fpstp) -{ - float_status *fpst = fpstp; - - /* Invalid if we are passed a NaN */ - if (float16_is_any_nan(a)) { - float_raise(float_flag_invalid, fpst); - return 0; - } - return float16_to_uint16(a, fpst); -} - static int el_from_spsr(uint32_t spsr) { /* Return the exception level that this SPSR is requesting a return to, diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static gen_helper_gvec_2_ptr * const f_fcvtzu_vf[] = { TRANS(FCVTZU_vf, do_gvec_op2_fpst, a->esz, a->q, a->rd, a->rn, a->shift, f_fcvtzu_vf) -static void handle_2misc_64(DisasContext *s, int opcode, bool u, - TCGv_i64 tcg_rd, TCGv_i64 tcg_rn, - TCGv_i32 tcg_rmode, TCGv_ptr tcg_fpstatus) -{ - /* Handle 64->64 opcodes which are shared between the scalar and - * vector 2-reg-misc groups. We cover every integer opcode where size == 3 - * is valid in either group and also the double-precision fp ops. - * The caller only need provide tcg_rmode and tcg_fpstatus if the op - * requires them. - */ - switch (opcode) { - case 0x1a: /* FCVTNS */ - case 0x1b: /* FCVTMS */ - case 0x1c: /* FCVTAS */ - case 0x3a: /* FCVTPS */ - case 0x3b: /* FCVTZS */ - gen_helper_vfp_tosqd(tcg_rd, tcg_rn, tcg_constant_i32(0), tcg_fpstatus); - break; - case 0x5a: /* FCVTNU */ - case 0x5b: /* FCVTMU */ - case 0x5c: /* FCVTAU */ - case 0x7a: /* FCVTPU */ - case 0x7b: /* FCVTZU */ - gen_helper_vfp_touqd(tcg_rd, tcg_rn, tcg_constant_i32(0), tcg_fpstatus); - break; - default: - case 0x4: /* CLS, CLZ */ - case 0x5: /* NOT */ - case 0x7: /* SQABS, SQNEG */ - case 0x8: /* CMGT, CMGE */ - case 0x9: /* CMEQ, CMLE */ - case 0xa: /* CMLT */ - case 0xb: /* ABS, NEG */ - case 0x2f: /* FABS */ - case 0x6f: /* FNEG */ - case 0x7f: /* FSQRT */ - case 0x18: /* FRINTN */ - case 0x19: /* FRINTM */ - case 0x38: /* FRINTP */ - case 0x39: /* FRINTZ */ - case 0x58: /* FRINTA */ - case 0x79: /* FRINTI */ - case 0x59: /* FRINTX */ - case 0x1e: /* FRINT32Z */ - case 0x5e: /* FRINT32X */ - case 0x1f: /* FRINT64Z */ - case 0x5f: /* FRINT64X */ - g_assert_not_reached(); - } -} +static gen_helper_gvec_2_ptr * const f_fcvt_s_vi[] = { + gen_helper_gvec_vcvt_rm_sh, + gen_helper_gvec_vcvt_rm_ss, + gen_helper_gvec_vcvt_rm_sd, +}; + +static gen_helper_gvec_2_ptr * const f_fcvt_u_vi[] = { + gen_helper_gvec_vcvt_rm_uh, + gen_helper_gvec_vcvt_rm_us, + gen_helper_gvec_vcvt_rm_ud, +}; + +TRANS(FCVTNS_vi, do_gvec_op2_fpst, + a->esz, a->q, a->rd, a->rn, float_round_nearest_even, f_fcvt_s_vi) +TRANS(FCVTNU_vi, do_gvec_op2_fpst, + a->esz, a->q, a->rd, a->rn, float_round_nearest_even, f_fcvt_u_vi) +TRANS(FCVTPS_vi, do_gvec_op2_fpst, + a->esz, a->q, a->rd, a->rn, float_round_up, f_fcvt_s_vi) +TRANS(FCVTPU_vi, do_gvec_op2_fpst, + a->esz, a->q, a->rd, a->rn, float_round_up, f_fcvt_u_vi) +TRANS(FCVTMS_vi, do_gvec_op2_fpst, + a->esz, a->q, a->rd, a->rn, float_round_down, f_fcvt_s_vi) +TRANS(FCVTMU_vi, do_gvec_op2_fpst, + a->esz, a->q, a->rd, a->rn, float_round_down, f_fcvt_u_vi) +TRANS(FCVTZS_vi, do_gvec_op2_fpst, + a->esz, a->q, a->rd, a->rn, float_round_to_zero, f_fcvt_s_vi) +TRANS(FCVTZU_vi, do_gvec_op2_fpst, + a->esz, a->q, a->rd, a->rn, float_round_to_zero, f_fcvt_u_vi) +TRANS(FCVTAS_vi, do_gvec_op2_fpst, + a->esz, a->q, a->rd, a->rn, float_round_ties_away, f_fcvt_s_vi) +TRANS(FCVTAU_vi, do_gvec_op2_fpst, + a->esz, a->q, a->rd, a->rn, float_round_ties_away, f_fcvt_u_vi) static void handle_2misc_fcmp_zero(DisasContext *s, int opcode, bool is_scalar, bool is_u, bool is_q, @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) } handle_2misc_fcmp_zero(s, opcode, false, u, is_q, size, rn, rd); return; - case 0x1a: /* FCVTNS */ - case 0x1b: /* FCVTMS */ - case 0x3a: /* FCVTPS */ - case 0x3b: /* FCVTZS */ - case 0x5a: /* FCVTNU */ - case 0x5b: /* FCVTMU */ - case 0x7a: /* FCVTPU */ - case 0x7b: /* FCVTZU */ - need_fpstatus = true; - rmode = extract32(opcode, 5, 1) | (extract32(opcode, 0, 1) << 1); - if (size == 3 && !is_q) { - unallocated_encoding(s); - return; - } - break; - case 0x5c: /* FCVTAU */ - case 0x1c: /* FCVTAS */ - need_fpstatus = true; - rmode = FPROUNDING_TIEAWAY; - if (size == 3 && !is_q) { - unallocated_encoding(s); - return; - } - break; case 0x3c: /* URECPE */ if (size == 3) { unallocated_encoding(s); @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) case 0x5f: /* FRINT64X */ case 0x1d: /* SCVTF */ case 0x5d: /* UCVTF */ + case 0x1a: /* FCVTNS */ + case 0x1b: /* FCVTMS */ + case 0x3a: /* FCVTPS */ + case 0x3b: /* FCVTZS */ + case 0x5a: /* FCVTNU */ + case 0x5b: /* FCVTMU */ + case 0x7a: /* FCVTPU */ + case 0x7b: /* FCVTZU */ + case 0x5c: /* FCVTAU */ + case 0x1c: /* FCVTAS */ unallocated_encoding(s); return; } @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) tcg_rmode = NULL; } - if (size == 3) { - /* All 64-bit element operations can be shared with scalar 2misc */ - int pass; - - /* Coverity claims (size == 3 && !is_q) has been eliminated - * from all paths leading to here. - */ - tcg_debug_assert(is_q); - for (pass = 0; pass < 2; pass++) { - TCGv_i64 tcg_op = tcg_temp_new_i64(); - TCGv_i64 tcg_res = tcg_temp_new_i64(); - - read_vec_element(s, tcg_op, rn, pass, MO_64); - - handle_2misc_64(s, opcode, u, tcg_res, tcg_op, - tcg_rmode, tcg_fpstatus); - - write_vec_element(s, tcg_res, rd, pass, MO_64); - } - } else { + { int pass; assert(size == 2); @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) { /* Special cases for 32 bit elements */ switch (opcode) { - case 0x1a: /* FCVTNS */ - case 0x1b: /* FCVTMS */ - case 0x1c: /* FCVTAS */ - case 0x3a: /* FCVTPS */ - case 0x3b: /* FCVTZS */ - gen_helper_vfp_tosls(tcg_res, tcg_op, - tcg_constant_i32(0), tcg_fpstatus); - break; - case 0x5a: /* FCVTNU */ - case 0x5b: /* FCVTMU */ - case 0x5c: /* FCVTAU */ - case 0x7a: /* FCVTPU */ - case 0x7b: /* FCVTZU */ - gen_helper_vfp_touls(tcg_res, tcg_op, - tcg_constant_i32(0), tcg_fpstatus); - break; case 0x7c: /* URSQRTE */ gen_helper_rsqrte_u32(tcg_res, tcg_op); break; @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) case 0x5e: /* FRINT32X */ case 0x1f: /* FRINT64Z */ case 0x5f: /* FRINT64X */ + case 0x1a: /* FCVTNS */ + case 0x1b: /* FCVTMS */ + case 0x1c: /* FCVTAS */ + case 0x3a: /* FCVTPS */ + case 0x3b: /* FCVTZS */ + case 0x5a: /* FCVTNU */ + case 0x5b: /* FCVTMU */ + case 0x5c: /* FCVTAU */ + case 0x7a: /* FCVTPU */ + case 0x7b: /* FCVTZU */ g_assert_not_reached(); } } @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc_fp16(DisasContext *s, uint32_t insn) case 0x3d: /* FRECPE */ case 0x3f: /* FRECPX */ break; - case 0x1a: /* FCVTNS */ - rmode = FPROUNDING_TIEEVEN; - break; - case 0x1b: /* FCVTMS */ - rmode = FPROUNDING_NEGINF; - break; - case 0x1c: /* FCVTAS */ - rmode = FPROUNDING_TIEAWAY; - break; - case 0x3a: /* FCVTPS */ - rmode = FPROUNDING_POSINF; - break; - case 0x3b: /* FCVTZS */ - rmode = FPROUNDING_ZERO; - break; - case 0x5a: /* FCVTNU */ - rmode = FPROUNDING_TIEEVEN; - break; - case 0x5b: /* FCVTMU */ - rmode = FPROUNDING_NEGINF; - break; - case 0x5c: /* FCVTAU */ - rmode = FPROUNDING_TIEAWAY; - break; - case 0x7a: /* FCVTPU */ - rmode = FPROUNDING_POSINF; - break; - case 0x7b: /* FCVTZU */ - rmode = FPROUNDING_ZERO; - break; case 0x7d: /* FRSQRTE */ break; default: @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc_fp16(DisasContext *s, uint32_t insn) case 0x79: /* FRINTI */ case 0x1d: /* SCVTF */ case 0x5d: /* UCVTF */ + case 0x1a: /* FCVTNS */ + case 0x1b: /* FCVTMS */ + case 0x1c: /* FCVTAS */ + case 0x3a: /* FCVTPS */ + case 0x3b: /* FCVTZS */ + case 0x5a: /* FCVTNU */ + case 0x5b: /* FCVTMU */ + case 0x5c: /* FCVTAU */ + case 0x7a: /* FCVTPU */ + case 0x7b: /* FCVTZU */ unallocated_encoding(s); return; } @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc_fp16(DisasContext *s, uint32_t insn) read_vec_element_i32(s, tcg_op, rn, pass, MO_16); switch (fpop) { - case 0x1a: /* FCVTNS */ - case 0x1b: /* FCVTMS */ - case 0x1c: /* FCVTAS */ - case 0x3a: /* FCVTPS */ - case 0x3b: /* FCVTZS */ - gen_helper_advsimd_f16tosinth(tcg_res, tcg_op, tcg_fpstatus); - break; case 0x3d: /* FRECPE */ gen_helper_recpe_f16(tcg_res, tcg_op, tcg_fpstatus); break; - case 0x5a: /* FCVTNU */ - case 0x5b: /* FCVTMU */ - case 0x5c: /* FCVTAU */ - case 0x7a: /* FCVTPU */ - case 0x7b: /* FCVTZU */ - gen_helper_advsimd_f16touinth(tcg_res, tcg_op, tcg_fpstatus); - break; case 0x7d: /* FRSQRTE */ gen_helper_rsqrte_f16(tcg_res, tcg_op, tcg_fpstatus); break; @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc_fp16(DisasContext *s, uint32_t insn) case 0x58: /* FRINTA */ case 0x79: /* FRINTI */ case 0x59: /* FRINTX */ + case 0x1a: /* FCVTNS */ + case 0x1b: /* FCVTMS */ + case 0x1c: /* FCVTAS */ + case 0x3a: /* FCVTPS */ + case 0x3b: /* FCVTZS */ + case 0x5a: /* FCVTNU */ + case 0x5b: /* FCVTMU */ + case 0x5c: /* FCVTAU */ + case 0x7a: /* FCVTPU */ + case 0x7b: /* FCVTZU */ g_assert_not_reached(); } diff --git a/target/arm/tcg/vec_helper.c b/target/arm/tcg/vec_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/vec_helper.c +++ b/target/arm/tcg/vec_helper.c @@ -XXX,XX +XXX,XX @@ DO_VCVT_FIXED(gvec_vcvt_rz_hu, helper_vfp_touhh_round_to_zero, uint16_t) clear_tail(d, oprsz, simd_maxsz(desc)); \ } +DO_VCVT_RMODE(gvec_vcvt_rm_sd, helper_vfp_tosqd, uint64_t) +DO_VCVT_RMODE(gvec_vcvt_rm_ud, helper_vfp_touqd, uint64_t) DO_VCVT_RMODE(gvec_vcvt_rm_ss, helper_vfp_tosls, uint32_t) DO_VCVT_RMODE(gvec_vcvt_rm_us, helper_vfp_touls, uint32_t) DO_VCVT_RMODE(gvec_vcvt_rm_sh, helper_vfp_toshh, uint16_t) -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> This includes FCMEQ, FCMGT, FCMGE, FCMLT, FCMLE. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-66-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/helper.h | 5 + target/arm/tcg/a64.decode | 30 ++++ target/arm/tcg/translate-a64.c | 249 +++++++++++++-------------------- target/arm/tcg/vec_helper.c | 4 +- 4 files changed, 138 insertions(+), 150 deletions(-) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_4(gvec_frsqrte_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_fcgt0_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_fcgt0_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_fcgt0_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_fcge0_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_fcge0_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_fcge0_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_fceq0_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_fceq0_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_fceq0_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_fcle0_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_fcle0_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_fcle0_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_fclt0_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_fclt0_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_fclt0_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_fadd_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_fadd_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ UQXTN_s 0111 1110 ..1 00001 01001 0 ..... ..... @rr_e FCVTXN_s 0111 1110 011 00001 01101 0 ..... ..... @rr_s +FCMGT0_s 0101 1110 111 11000 11001 0 ..... ..... @rr_h +FCMGT0_s 0101 1110 1.1 00000 11001 0 ..... ..... @rr_sd + +FCMGE0_s 0111 1110 111 11000 11001 0 ..... ..... @rr_h +FCMGE0_s 0111 1110 1.1 00000 11001 0 ..... ..... @rr_sd + +FCMEQ0_s 0101 1110 111 11000 11011 0 ..... ..... @rr_h +FCMEQ0_s 0101 1110 1.1 00000 11011 0 ..... ..... @rr_sd + +FCMLE0_s 0111 1110 111 11000 11011 0 ..... ..... @rr_h +FCMLE0_s 0111 1110 1.1 00000 11011 0 ..... ..... @rr_sd + +FCMLT0_s 0101 1110 111 11000 11101 0 ..... ..... @rr_h +FCMLT0_s 0101 1110 1.1 00000 11101 0 ..... ..... @rr_sd + @icvt_h . ....... .. ...... ...... rn:5 rd:5 \ &fcvt sf=0 esz=1 shift=0 @icvt_sd . ....... .. ...... ...... rn:5 rd:5 \ @@ -XXX,XX +XXX,XX @@ FCVTAS_vi 0.00 1110 0.1 00001 11001 0 ..... ..... @qrr_sd FCVTAU_vi 0.10 1110 011 11001 11001 0 ..... ..... @qrr_h FCVTAU_vi 0.10 1110 0.1 00001 11001 0 ..... ..... @qrr_sd +FCMGT0_v 0.00 1110 111 11000 11001 0 ..... ..... @qrr_h +FCMGT0_v 0.00 1110 1.1 00000 11001 0 ..... ..... @qrr_sd + +FCMGE0_v 0.10 1110 111 11000 11001 0 ..... ..... @qrr_h +FCMGE0_v 0.10 1110 1.1 00000 11001 0 ..... ..... @qrr_sd + +FCMEQ0_v 0.00 1110 111 11000 11011 0 ..... ..... @qrr_h +FCMEQ0_v 0.00 1110 1.1 00000 11011 0 ..... ..... @qrr_sd + +FCMLE0_v 0.10 1110 111 11000 11011 0 ..... ..... @qrr_h +FCMLE0_v 0.10 1110 1.1 00000 11011 0 ..... ..... @qrr_sd + +FCMLT0_v 0.00 1110 111 11000 11101 0 ..... ..... @qrr_h +FCMLT0_v 0.00 1110 1.1 00000 11101 0 ..... ..... @qrr_sd + &fcvt_q rd rn esz q shift @fcvtq_h . q:1 . ...... 001 .... ...... rn:5 rd:5 \ &fcvt_q esz=1 shift=%fcvt_f_sh_h diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static const FPScalar f_scalar_frsqrts = { }; TRANS(FRSQRTS_s, do_fp3_scalar, a, &f_scalar_frsqrts) +static bool do_fcmp0_s(DisasContext *s, arg_rr_e *a, + const FPScalar *f, bool swap) +{ + switch (a->esz) { + case MO_64: + if (fp_access_check(s)) { + TCGv_i64 t0 = read_fp_dreg(s, a->rn); + TCGv_i64 t1 = tcg_constant_i64(0); + if (swap) { + f->gen_d(t0, t1, t0, fpstatus_ptr(FPST_FPCR)); + } else { + f->gen_d(t0, t0, t1, fpstatus_ptr(FPST_FPCR)); + } + write_fp_dreg(s, a->rd, t0); + } + break; + case MO_32: + if (fp_access_check(s)) { + TCGv_i32 t0 = read_fp_sreg(s, a->rn); + TCGv_i32 t1 = tcg_constant_i32(0); + if (swap) { + f->gen_s(t0, t1, t0, fpstatus_ptr(FPST_FPCR)); + } else { + f->gen_s(t0, t0, t1, fpstatus_ptr(FPST_FPCR)); + } + write_fp_sreg(s, a->rd, t0); + } + break; + case MO_16: + if (!dc_isar_feature(aa64_fp16, s)) { + return false; + } + if (fp_access_check(s)) { + TCGv_i32 t0 = read_fp_hreg(s, a->rn); + TCGv_i32 t1 = tcg_constant_i32(0); + if (swap) { + f->gen_h(t0, t1, t0, fpstatus_ptr(FPST_FPCR_F16)); + } else { + f->gen_h(t0, t0, t1, fpstatus_ptr(FPST_FPCR_F16)); + } + write_fp_sreg(s, a->rd, t0); + } + break; + default: + return false; + } + return true; +} + +TRANS(FCMEQ0_s, do_fcmp0_s, a, &f_scalar_fcmeq, false) +TRANS(FCMGT0_s, do_fcmp0_s, a, &f_scalar_fcmgt, false) +TRANS(FCMGE0_s, do_fcmp0_s, a, &f_scalar_fcmge, false) +TRANS(FCMLT0_s, do_fcmp0_s, a, &f_scalar_fcmgt, true) +TRANS(FCMLE0_s, do_fcmp0_s, a, &f_scalar_fcmge, true) + static bool do_satacc_s(DisasContext *s, arg_rrr_e *a, MemOp sgn_n, MemOp sgn_m, void (*gen_bhs)(TCGv_i64, TCGv_i64, TCGv_i64, TCGv_i64, MemOp), @@ -XXX,XX +XXX,XX @@ TRANS(FCVTAS_vi, do_gvec_op2_fpst, TRANS(FCVTAU_vi, do_gvec_op2_fpst, a->esz, a->q, a->rd, a->rn, float_round_ties_away, f_fcvt_u_vi) -static void handle_2misc_fcmp_zero(DisasContext *s, int opcode, - bool is_scalar, bool is_u, bool is_q, - int size, int rn, int rd) -{ - bool is_double = (size == MO_64); - TCGv_ptr fpst; +static gen_helper_gvec_2_ptr * const f_fceq0[] = { + gen_helper_gvec_fceq0_h, + gen_helper_gvec_fceq0_s, + gen_helper_gvec_fceq0_d, +}; +TRANS(FCMEQ0_v, do_gvec_op2_fpst, a->esz, a->q, a->rd, a->rn, 0, f_fceq0) - if (!fp_access_check(s)) { - return; - } +static gen_helper_gvec_2_ptr * const f_fcgt0[] = { + gen_helper_gvec_fcgt0_h, + gen_helper_gvec_fcgt0_s, + gen_helper_gvec_fcgt0_d, +}; +TRANS(FCMGT0_v, do_gvec_op2_fpst, a->esz, a->q, a->rd, a->rn, 0, f_fcgt0) - fpst = fpstatus_ptr(size == MO_16 ? FPST_FPCR_F16 : FPST_FPCR); +static gen_helper_gvec_2_ptr * const f_fcge0[] = { + gen_helper_gvec_fcge0_h, + gen_helper_gvec_fcge0_s, + gen_helper_gvec_fcge0_d, +}; +TRANS(FCMGE0_v, do_gvec_op2_fpst, a->esz, a->q, a->rd, a->rn, 0, f_fcge0) - if (is_double) { - TCGv_i64 tcg_op = tcg_temp_new_i64(); - TCGv_i64 tcg_zero = tcg_constant_i64(0); - TCGv_i64 tcg_res = tcg_temp_new_i64(); - NeonGenTwoDoubleOpFn *genfn; - bool swap = false; - int pass; +static gen_helper_gvec_2_ptr * const f_fclt0[] = { + gen_helper_gvec_fclt0_h, + gen_helper_gvec_fclt0_s, + gen_helper_gvec_fclt0_d, +}; +TRANS(FCMLT0_v, do_gvec_op2_fpst, a->esz, a->q, a->rd, a->rn, 0, f_fclt0) - switch (opcode) { - case 0x2e: /* FCMLT (zero) */ - swap = true; - /* fallthrough */ - case 0x2c: /* FCMGT (zero) */ - genfn = gen_helper_neon_cgt_f64; - break; - case 0x2d: /* FCMEQ (zero) */ - genfn = gen_helper_neon_ceq_f64; - break; - case 0x6d: /* FCMLE (zero) */ - swap = true; - /* fall through */ - case 0x6c: /* FCMGE (zero) */ - genfn = gen_helper_neon_cge_f64; - break; - default: - g_assert_not_reached(); - } - - for (pass = 0; pass < (is_scalar ? 1 : 2); pass++) { - read_vec_element(s, tcg_op, rn, pass, MO_64); - if (swap) { - genfn(tcg_res, tcg_zero, tcg_op, fpst); - } else { - genfn(tcg_res, tcg_op, tcg_zero, fpst); - } - write_vec_element(s, tcg_res, rd, pass, MO_64); - } - - clear_vec_high(s, !is_scalar, rd); - } else { - TCGv_i32 tcg_op = tcg_temp_new_i32(); - TCGv_i32 tcg_zero = tcg_constant_i32(0); - TCGv_i32 tcg_res = tcg_temp_new_i32(); - NeonGenTwoSingleOpFn *genfn; - bool swap = false; - int pass, maxpasses; - - if (size == MO_16) { - switch (opcode) { - case 0x2e: /* FCMLT (zero) */ - swap = true; - /* fall through */ - case 0x2c: /* FCMGT (zero) */ - genfn = gen_helper_advsimd_cgt_f16; - break; - case 0x2d: /* FCMEQ (zero) */ - genfn = gen_helper_advsimd_ceq_f16; - break; - case 0x6d: /* FCMLE (zero) */ - swap = true; - /* fall through */ - case 0x6c: /* FCMGE (zero) */ - genfn = gen_helper_advsimd_cge_f16; - break; - default: - g_assert_not_reached(); - } - } else { - switch (opcode) { - case 0x2e: /* FCMLT (zero) */ - swap = true; - /* fall through */ - case 0x2c: /* FCMGT (zero) */ - genfn = gen_helper_neon_cgt_f32; - break; - case 0x2d: /* FCMEQ (zero) */ - genfn = gen_helper_neon_ceq_f32; - break; - case 0x6d: /* FCMLE (zero) */ - swap = true; - /* fall through */ - case 0x6c: /* FCMGE (zero) */ - genfn = gen_helper_neon_cge_f32; - break; - default: - g_assert_not_reached(); - } - } - - if (is_scalar) { - maxpasses = 1; - } else { - int vector_size = 8 << is_q; - maxpasses = vector_size >> size; - } - - for (pass = 0; pass < maxpasses; pass++) { - read_vec_element_i32(s, tcg_op, rn, pass, size); - if (swap) { - genfn(tcg_res, tcg_zero, tcg_op, fpst); - } else { - genfn(tcg_res, tcg_op, tcg_zero, fpst); - } - if (is_scalar) { - write_fp_sreg(s, rd, tcg_res); - } else { - write_vec_element_i32(s, tcg_res, rd, pass, size); - } - } - - if (!is_scalar) { - clear_vec_high(s, is_q, rd); - } - } -} +static gen_helper_gvec_2_ptr * const f_fcle0[] = { + gen_helper_gvec_fcle0_h, + gen_helper_gvec_fcle0_s, + gen_helper_gvec_fcle0_d, +}; +TRANS(FCMLE0_v, do_gvec_op2_fpst, a->esz, a->q, a->rd, a->rn, 0, f_fcle0) static void handle_2misc_reciprocal(DisasContext *s, int opcode, bool is_scalar, bool is_u, bool is_q, @@ -XXX,XX +XXX,XX @@ static void disas_simd_scalar_two_reg_misc(DisasContext *s, uint32_t insn) opcode |= (extract32(size, 1, 1) << 5) | (u << 6); size = extract32(size, 0, 1) ? 3 : 2; switch (opcode) { - case 0x2c: /* FCMGT (zero) */ - case 0x2d: /* FCMEQ (zero) */ - case 0x2e: /* FCMLT (zero) */ - case 0x6c: /* FCMGE (zero) */ - case 0x6d: /* FCMLE (zero) */ - handle_2misc_fcmp_zero(s, opcode, true, u, true, size, rn, rd); - return; case 0x3d: /* FRECPE */ case 0x3f: /* FRECPX */ case 0x7d: /* FRSQRTE */ @@ -XXX,XX +XXX,XX @@ static void disas_simd_scalar_two_reg_misc(DisasContext *s, uint32_t insn) case 0x56: /* FCVTXN, FCVTXN2 */ case 0x1d: /* SCVTF */ case 0x5d: /* UCVTF */ + case 0x2c: /* FCMGT (zero) */ + case 0x2d: /* FCMEQ (zero) */ + case 0x2e: /* FCMLT (zero) */ + case 0x6c: /* FCMGE (zero) */ + case 0x6d: /* FCMLE (zero) */ default: unallocated_encoding(s); return; @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) opcode |= (extract32(size, 1, 1) << 5) | (u << 6); size = is_double ? 3 : 2; switch (opcode) { - case 0x2c: /* FCMGT (zero) */ - case 0x2d: /* FCMEQ (zero) */ - case 0x2e: /* FCMLT (zero) */ - case 0x6c: /* FCMGE (zero) */ - case 0x6d: /* FCMLE (zero) */ - if (size == 3 && !is_q) { - unallocated_encoding(s); - return; - } - handle_2misc_fcmp_zero(s, opcode, false, u, is_q, size, rn, rd); - return; case 0x3c: /* URECPE */ if (size == 3) { unallocated_encoding(s); @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) case 0x7b: /* FCVTZU */ case 0x5c: /* FCVTAU */ case 0x1c: /* FCVTAS */ + case 0x2c: /* FCMGT (zero) */ + case 0x2d: /* FCMEQ (zero) */ + case 0x2e: /* FCMLT (zero) */ + case 0x6c: /* FCMGE (zero) */ + case 0x6d: /* FCMLE (zero) */ unallocated_encoding(s); return; } @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc_fp16(DisasContext *s, uint32_t insn) fpop = deposit32(fpop, 6, 1, u); switch (fpop) { - case 0x2c: /* FCMGT (zero) */ - case 0x2d: /* FCMEQ (zero) */ - case 0x2e: /* FCMLT (zero) */ - case 0x6c: /* FCMGE (zero) */ - case 0x6d: /* FCMLE (zero) */ - handle_2misc_fcmp_zero(s, fpop, is_scalar, 0, is_q, MO_16, rn, rd); - return; case 0x3d: /* FRECPE */ case 0x3f: /* FRECPX */ break; @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc_fp16(DisasContext *s, uint32_t insn) case 0x5c: /* FCVTAU */ case 0x7a: /* FCVTPU */ case 0x7b: /* FCVTZU */ + case 0x2c: /* FCMGT (zero) */ + case 0x2d: /* FCMEQ (zero) */ + case 0x2e: /* FCMLT (zero) */ + case 0x6c: /* FCMGE (zero) */ + case 0x6d: /* FCMLE (zero) */ unallocated_encoding(s); return; } diff --git a/target/arm/tcg/vec_helper.c b/target/arm/tcg/vec_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/vec_helper.c +++ b/target/arm/tcg/vec_helper.c @@ -XXX,XX +XXX,XX @@ DO_2OP(gvec_touszh, vfp_touszh, float16) #define DO_2OP_CMP0(FN, CMPOP, DIRN) \ WRAP_CMP0_##DIRN(FN, CMPOP, float16) \ WRAP_CMP0_##DIRN(FN, CMPOP, float32) \ + WRAP_CMP0_##DIRN(FN, CMPOP, float64) \ DO_2OP(gvec_f##FN##0_h, float16_##FN##0, float16) \ - DO_2OP(gvec_f##FN##0_s, float32_##FN##0, float32) + DO_2OP(gvec_f##FN##0_s, float32_##FN##0, float32) \ + DO_2OP(gvec_f##FN##0_d, float64_##FN##0, float64) DO_2OP_CMP0(cgt, cgt, FWD) DO_2OP_CMP0(cge, cge, FWD) -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Remove disas_simd_scalar_two_reg_misc and disas_simd_two_reg_misc_fp16 as these were the last insns decoded by those functions. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-67-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 15 ++ target/arm/tcg/translate-a64.c | 329 ++++----------------------------- 2 files changed, 53 insertions(+), 291 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ FCMLE0_s 0111 1110 1.1 00000 11011 0 ..... ..... @rr_sd FCMLT0_s 0101 1110 111 11000 11101 0 ..... ..... @rr_h FCMLT0_s 0101 1110 1.1 00000 11101 0 ..... ..... @rr_sd +FRECPE_s 0101 1110 111 11001 11011 0 ..... ..... @rr_h +FRECPE_s 0101 1110 1.1 00001 11011 0 ..... ..... @rr_sd + +FRECPX_s 0101 1110 111 11001 11111 0 ..... ..... @rr_h +FRECPX_s 0101 1110 1.1 00001 11111 0 ..... ..... @rr_sd + +FRSQRTE_s 0111 1110 111 11001 11011 0 ..... ..... @rr_h +FRSQRTE_s 0111 1110 1.1 00001 11011 0 ..... ..... @rr_sd + @icvt_h . ....... .. ...... ...... rn:5 rd:5 \ &fcvt sf=0 esz=1 shift=0 @icvt_sd . ....... .. ...... ...... rn:5 rd:5 \ @@ -XXX,XX +XXX,XX @@ FCMLE0_v 0.10 1110 1.1 00000 11011 0 ..... ..... @qrr_sd FCMLT0_v 0.00 1110 111 11000 11101 0 ..... ..... @qrr_h FCMLT0_v 0.00 1110 1.1 00000 11101 0 ..... ..... @qrr_sd +FRECPE_v 0.00 1110 111 11001 11011 0 ..... ..... @qrr_h +FRECPE_v 0.00 1110 1.1 00001 11011 0 ..... ..... @qrr_sd + +FRSQRTE_v 0.10 1110 111 11001 11011 0 ..... ..... @qrr_h +FRSQRTE_v 0.10 1110 1.1 00001 11011 0 ..... ..... @qrr_sd + &fcvt_q rd rn esz q shift @fcvtq_h . q:1 . ...... 001 .... ...... rn:5 rd:5 \ &fcvt_q esz=1 shift=%fcvt_f_sh_h diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ TRANS_FEAT(FRINT64Z_s, aa64_frint, do_fp1_scalar, a, &f_scalar_frint64, FPROUNDING_ZERO) TRANS_FEAT(FRINT64X_s, aa64_frint, do_fp1_scalar, a, &f_scalar_frint64, -1) +static const FPScalar1 f_scalar_frecpe = { + gen_helper_recpe_f16, + gen_helper_recpe_f32, + gen_helper_recpe_f64, +}; +TRANS(FRECPE_s, do_fp1_scalar, a, &f_scalar_frecpe, -1) + +static const FPScalar1 f_scalar_frecpx = { + gen_helper_frecpx_f16, + gen_helper_frecpx_f32, + gen_helper_frecpx_f64, +}; +TRANS(FRECPX_s, do_fp1_scalar, a, &f_scalar_frecpx, -1) + +static const FPScalar1 f_scalar_frsqrte = { + gen_helper_rsqrte_f16, + gen_helper_rsqrte_f32, + gen_helper_rsqrte_f64, +}; +TRANS(FRSQRTE_s, do_fp1_scalar, a, &f_scalar_frsqrte, -1) + static bool trans_FCVT_s_ds(DisasContext *s, arg_rr *a) { if (fp_access_check(s)) { @@ -XXX,XX +XXX,XX @@ static gen_helper_gvec_2_ptr * const f_fcle0[] = { }; TRANS(FCMLE0_v, do_gvec_op2_fpst, a->esz, a->q, a->rd, a->rn, 0, f_fcle0) +static gen_helper_gvec_2_ptr * const f_frecpe[] = { + gen_helper_gvec_frecpe_h, + gen_helper_gvec_frecpe_s, + gen_helper_gvec_frecpe_d, +}; +TRANS(FRECPE_v, do_gvec_op2_fpst, a->esz, a->q, a->rd, a->rn, 0, f_frecpe) + +static gen_helper_gvec_2_ptr * const f_frsqrte[] = { + gen_helper_gvec_frsqrte_h, + gen_helper_gvec_frsqrte_s, + gen_helper_gvec_frsqrte_d, +}; +TRANS(FRSQRTE_v, do_gvec_op2_fpst, a->esz, a->q, a->rd, a->rn, 0, f_frsqrte) + static void handle_2misc_reciprocal(DisasContext *s, int opcode, bool is_scalar, bool is_u, bool is_q, int size, int rn, int rd) { bool is_double = (size == 3); - TCGv_ptr fpst = fpstatus_ptr(FPST_FPCR); if (is_double) { - TCGv_i64 tcg_op = tcg_temp_new_i64(); - TCGv_i64 tcg_res = tcg_temp_new_i64(); - int pass; - - for (pass = 0; pass < (is_scalar ? 1 : 2); pass++) { - read_vec_element(s, tcg_op, rn, pass, MO_64); - switch (opcode) { - case 0x3d: /* FRECPE */ - gen_helper_recpe_f64(tcg_res, tcg_op, fpst); - break; - case 0x3f: /* FRECPX */ - gen_helper_frecpx_f64(tcg_res, tcg_op, fpst); - break; - case 0x7d: /* FRSQRTE */ - gen_helper_rsqrte_f64(tcg_res, tcg_op, fpst); - break; - default: - g_assert_not_reached(); - } - write_vec_element(s, tcg_res, rd, pass, MO_64); - } - clear_vec_high(s, !is_scalar, rd); + g_assert_not_reached(); } else { TCGv_i32 tcg_op = tcg_temp_new_i32(); TCGv_i32 tcg_res = tcg_temp_new_i32(); @@ -XXX,XX +XXX,XX @@ static void handle_2misc_reciprocal(DisasContext *s, int opcode, gen_helper_recpe_u32(tcg_res, tcg_op); break; case 0x3d: /* FRECPE */ - gen_helper_recpe_f32(tcg_res, tcg_op, fpst); - break; case 0x3f: /* FRECPX */ - gen_helper_frecpx_f32(tcg_res, tcg_op, fpst); - break; case 0x7d: /* FRSQRTE */ - gen_helper_rsqrte_f32(tcg_res, tcg_op, fpst); - break; default: g_assert_not_reached(); } @@ -XXX,XX +XXX,XX @@ static void handle_2misc_reciprocal(DisasContext *s, int opcode, } } -/* AdvSIMD scalar two reg misc - * 31 30 29 28 24 23 22 21 17 16 12 11 10 9 5 4 0 - * +-----+---+-----------+------+-----------+--------+-----+------+------+ - * | 0 1 | U | 1 1 1 1 0 | size | 1 0 0 0 0 | opcode | 1 0 | Rn | Rd | - * +-----+---+-----------+------+-----------+--------+-----+------+------+ - */ -static void disas_simd_scalar_two_reg_misc(DisasContext *s, uint32_t insn) -{ - int rd = extract32(insn, 0, 5); - int rn = extract32(insn, 5, 5); - int opcode = extract32(insn, 12, 5); - int size = extract32(insn, 22, 2); - bool u = extract32(insn, 29, 1); - - switch (opcode) { - case 0xc ... 0xf: - case 0x16 ... 0x1d: - case 0x1f: - /* Floating point: U, size[1] and opcode indicate operation; - * size[0] indicates single or double precision. - */ - opcode |= (extract32(size, 1, 1) << 5) | (u << 6); - size = extract32(size, 0, 1) ? 3 : 2; - switch (opcode) { - case 0x3d: /* FRECPE */ - case 0x3f: /* FRECPX */ - case 0x7d: /* FRSQRTE */ - if (!fp_access_check(s)) { - return; - } - handle_2misc_reciprocal(s, opcode, true, u, true, size, rn, rd); - return; - case 0x1a: /* FCVTNS */ - case 0x1b: /* FCVTMS */ - case 0x3a: /* FCVTPS */ - case 0x3b: /* FCVTZS */ - case 0x5a: /* FCVTNU */ - case 0x5b: /* FCVTMU */ - case 0x7a: /* FCVTPU */ - case 0x7b: /* FCVTZU */ - case 0x1c: /* FCVTAS */ - case 0x5c: /* FCVTAU */ - case 0x56: /* FCVTXN, FCVTXN2 */ - case 0x1d: /* SCVTF */ - case 0x5d: /* UCVTF */ - case 0x2c: /* FCMGT (zero) */ - case 0x2d: /* FCMEQ (zero) */ - case 0x2e: /* FCMLT (zero) */ - case 0x6c: /* FCMGE (zero) */ - case 0x6d: /* FCMLE (zero) */ - default: - unallocated_encoding(s); - return; - } - break; - default: - case 0x3: /* USQADD / SUQADD */ - case 0x7: /* SQABS / SQNEG */ - case 0x8: /* CMGT, CMGE */ - case 0x9: /* CMEQ, CMLE */ - case 0xa: /* CMLT */ - case 0xb: /* ABS, NEG */ - case 0x12: /* SQXTUN */ - case 0x14: /* SQXTN, UQXTN */ - unallocated_encoding(s); - return; - } - g_assert_not_reached(); -} - static void handle_2misc_widening(DisasContext *s, int opcode, bool is_q, int size, int rn, int rd) { @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) unallocated_encoding(s); return; } - /* fall through */ - case 0x3d: /* FRECPE */ - case 0x7d: /* FRSQRTE */ - if (size == 3 && !is_q) { - unallocated_encoding(s); - return; - } if (!fp_access_check(s)) { return; } @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) case 0x2e: /* FCMLT (zero) */ case 0x6c: /* FCMGE (zero) */ case 0x6d: /* FCMLE (zero) */ + case 0x3d: /* FRECPE */ + case 0x7d: /* FRSQRTE */ unallocated_encoding(s); return; } @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) } } -/* AdvSIMD [scalar] two register miscellaneous (FP16) - * - * 31 30 29 28 27 24 23 22 21 17 16 12 11 10 9 5 4 0 - * +---+---+---+---+---------+---+-------------+--------+-----+------+------+ - * | 0 | Q | U | S | 1 1 1 0 | a | 1 1 1 1 0 0 | opcode | 1 0 | Rn | Rd | - * +---+---+---+---+---------+---+-------------+--------+-----+------+------+ - * mask: 1000 1111 0111 1110 0000 1100 0000 0000 0x8f7e 0c00 - * val: 0000 1110 0111 1000 0000 1000 0000 0000 0x0e78 0800 - * - * This actually covers two groups where scalar access is governed by - * bit 28. A bunch of the instructions (float to integral) only exist - * in the vector form and are un-allocated for the scalar decode. Also - * in the scalar decode Q is always 1. - */ -static void disas_simd_two_reg_misc_fp16(DisasContext *s, uint32_t insn) -{ - int fpop, opcode, a, u; - int rn, rd; - bool is_q; - bool is_scalar; - - int pass; - TCGv_i32 tcg_rmode = NULL; - TCGv_ptr tcg_fpstatus = NULL; - bool need_fpst = true; - int rmode = -1; - - if (!dc_isar_feature(aa64_fp16, s)) { - unallocated_encoding(s); - return; - } - - rd = extract32(insn, 0, 5); - rn = extract32(insn, 5, 5); - - a = extract32(insn, 23, 1); - u = extract32(insn, 29, 1); - is_scalar = extract32(insn, 28, 1); - is_q = extract32(insn, 30, 1); - - opcode = extract32(insn, 12, 5); - fpop = deposit32(opcode, 5, 1, a); - fpop = deposit32(fpop, 6, 1, u); - - switch (fpop) { - case 0x3d: /* FRECPE */ - case 0x3f: /* FRECPX */ - break; - case 0x7d: /* FRSQRTE */ - break; - default: - case 0x2f: /* FABS */ - case 0x6f: /* FNEG */ - case 0x7f: /* FSQRT (vector) */ - case 0x18: /* FRINTN */ - case 0x19: /* FRINTM */ - case 0x38: /* FRINTP */ - case 0x39: /* FRINTZ */ - case 0x58: /* FRINTA */ - case 0x59: /* FRINTX */ - case 0x79: /* FRINTI */ - case 0x1d: /* SCVTF */ - case 0x5d: /* UCVTF */ - case 0x1a: /* FCVTNS */ - case 0x1b: /* FCVTMS */ - case 0x1c: /* FCVTAS */ - case 0x3a: /* FCVTPS */ - case 0x3b: /* FCVTZS */ - case 0x5a: /* FCVTNU */ - case 0x5b: /* FCVTMU */ - case 0x5c: /* FCVTAU */ - case 0x7a: /* FCVTPU */ - case 0x7b: /* FCVTZU */ - case 0x2c: /* FCMGT (zero) */ - case 0x2d: /* FCMEQ (zero) */ - case 0x2e: /* FCMLT (zero) */ - case 0x6c: /* FCMGE (zero) */ - case 0x6d: /* FCMLE (zero) */ - unallocated_encoding(s); - return; - } - - - /* Check additional constraints for the scalar encoding */ - if (is_scalar) { - if (!is_q) { - unallocated_encoding(s); - return; - } - } - - if (!fp_access_check(s)) { - return; - } - - if (rmode >= 0 || need_fpst) { - tcg_fpstatus = fpstatus_ptr(FPST_FPCR_F16); - } - - if (rmode >= 0) { - tcg_rmode = gen_set_rmode(rmode, tcg_fpstatus); - } - - if (is_scalar) { - TCGv_i32 tcg_op = read_fp_hreg(s, rn); - TCGv_i32 tcg_res = tcg_temp_new_i32(); - - switch (fpop) { - case 0x3d: /* FRECPE */ - gen_helper_recpe_f16(tcg_res, tcg_op, tcg_fpstatus); - break; - case 0x3f: /* FRECPX */ - gen_helper_frecpx_f16(tcg_res, tcg_op, tcg_fpstatus); - break; - case 0x7d: /* FRSQRTE */ - gen_helper_rsqrte_f16(tcg_res, tcg_op, tcg_fpstatus); - break; - default: - case 0x1a: /* FCVTNS */ - case 0x1b: /* FCVTMS */ - case 0x1c: /* FCVTAS */ - case 0x3a: /* FCVTPS */ - case 0x3b: /* FCVTZS */ - case 0x5a: /* FCVTNU */ - case 0x5b: /* FCVTMU */ - case 0x5c: /* FCVTAU */ - case 0x7a: /* FCVTPU */ - case 0x7b: /* FCVTZU */ - g_assert_not_reached(); - } - - /* limit any sign extension going on */ - tcg_gen_andi_i32(tcg_res, tcg_res, 0xffff); - write_fp_sreg(s, rd, tcg_res); - } else { - for (pass = 0; pass < (is_q ? 8 : 4); pass++) { - TCGv_i32 tcg_op = tcg_temp_new_i32(); - TCGv_i32 tcg_res = tcg_temp_new_i32(); - - read_vec_element_i32(s, tcg_op, rn, pass, MO_16); - - switch (fpop) { - case 0x3d: /* FRECPE */ - gen_helper_recpe_f16(tcg_res, tcg_op, tcg_fpstatus); - break; - case 0x7d: /* FRSQRTE */ - gen_helper_rsqrte_f16(tcg_res, tcg_op, tcg_fpstatus); - break; - default: - case 0x2f: /* FABS */ - case 0x6f: /* FNEG */ - case 0x7f: /* FSQRT */ - case 0x18: /* FRINTN */ - case 0x19: /* FRINTM */ - case 0x38: /* FRINTP */ - case 0x39: /* FRINTZ */ - case 0x58: /* FRINTA */ - case 0x79: /* FRINTI */ - case 0x59: /* FRINTX */ - case 0x1a: /* FCVTNS */ - case 0x1b: /* FCVTMS */ - case 0x1c: /* FCVTAS */ - case 0x3a: /* FCVTPS */ - case 0x3b: /* FCVTZS */ - case 0x5a: /* FCVTNU */ - case 0x5b: /* FCVTMU */ - case 0x5c: /* FCVTAU */ - case 0x7a: /* FCVTPU */ - case 0x7b: /* FCVTZU */ - g_assert_not_reached(); - } - - write_vec_element_i32(s, tcg_res, rd, pass, MO_16); - } - - clear_vec_high(s, is_q, rd); - } - - if (tcg_rmode) { - gen_restore_rmode(tcg_rmode, tcg_fpstatus); - } -} - /* C3.6 Data processing - SIMD, inc Crypto * * As the decode gets a little complex we are using a table based @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc_fp16(DisasContext *s, uint32_t insn) static const AArch64DecodeTable data_proc_simd[] = { /* pattern , mask , fn */ { 0x0e200800, 0x9f3e0c00, disas_simd_two_reg_misc }, - { 0x5e200800, 0xdf3e0c00, disas_simd_scalar_two_reg_misc }, - { 0x0e780800, 0x8f7e0c00, disas_simd_two_reg_misc_fp16 }, { 0x00000000, 0x00000000, NULL } }; -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-68-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/helper.h | 3 +++ target/arm/tcg/translate.h | 5 +++++ target/arm/tcg/gengvec.c | 16 ++++++++++++++++ target/arm/tcg/translate-neon.c | 4 ++-- target/arm/tcg/vec_helper.c | 22 ++++++++++++++++++++++ 5 files changed, 48 insertions(+), 2 deletions(-) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_4(gvec_uminp_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_uminp_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_uminp_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(gvec_urecpe_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(gvec_ursqrte_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) + #ifdef TARGET_AARCH64 #include "tcg/helper-a64.h" #include "tcg/helper-sve.h" diff --git a/target/arm/tcg/translate.h b/target/arm/tcg/translate.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate.h +++ b/target/arm/tcg/translate.h @@ -XXX,XX +XXX,XX @@ void gen_gvec_fabs(unsigned vece, uint32_t dofs, uint32_t aofs, void gen_gvec_fneg(unsigned vece, uint32_t dofs, uint32_t aofs, uint32_t oprsz, uint32_t maxsz); +void gen_gvec_urecpe(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, + uint32_t opr_sz, uint32_t max_sz); +void gen_gvec_ursqrte(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, + uint32_t opr_sz, uint32_t max_sz); + /* * Forward to the isar_feature_* tests given a DisasContext pointer. */ diff --git a/target/arm/tcg/gengvec.c b/target/arm/tcg/gengvec.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/gengvec.c +++ b/target/arm/tcg/gengvec.c @@ -XXX,XX +XXX,XX @@ void gen_gvec_fneg(unsigned vece, uint32_t dofs, uint32_t aofs, uint64_t s_bit = 1ull << ((8 << vece) - 1); tcg_gen_gvec_xori(vece, dofs, aofs, s_bit, oprsz, maxsz); } + +void gen_gvec_urecpe(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, + uint32_t opr_sz, uint32_t max_sz) +{ + assert(vece == MO_32); + tcg_gen_gvec_2_ool(rd_ofs, rn_ofs, opr_sz, max_sz, 0, + gen_helper_gvec_urecpe_s); +} + +void gen_gvec_ursqrte(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, + uint32_t opr_sz, uint32_t max_sz) +{ + assert(vece == MO_32); + tcg_gen_gvec_2_ool(rd_ofs, rn_ofs, opr_sz, max_sz, 0, + gen_helper_gvec_ursqrte_s); +} diff --git a/target/arm/tcg/translate-neon.c b/target/arm/tcg/translate-neon.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-neon.c +++ b/target/arm/tcg/translate-neon.c @@ -XXX,XX +XXX,XX @@ static bool trans_VRECPE(DisasContext *s, arg_2misc *a) if (a->size != 2) { return false; } - return do_2misc(s, a, gen_helper_recpe_u32); + return do_2misc_vec(s, a, gen_gvec_urecpe); } static bool trans_VRSQRTE(DisasContext *s, arg_2misc *a) @@ -XXX,XX +XXX,XX @@ static bool trans_VRSQRTE(DisasContext *s, arg_2misc *a) if (a->size != 2) { return false; } - return do_2misc(s, a, gen_helper_rsqrte_u32); + return do_2misc_vec(s, a, gen_gvec_ursqrte); } #define WRAP_1OP_ENV_FN(WRAPNAME, FUNC) \ diff --git a/target/arm/tcg/vec_helper.c b/target/arm/tcg/vec_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/vec_helper.c +++ b/target/arm/tcg/vec_helper.c @@ -XXX,XX +XXX,XX @@ void HELPER(gvec_rbit_b)(void *vd, void *vn, uint32_t desc) } clear_tail(d, opr_sz, simd_maxsz(desc)); } + +void HELPER(gvec_urecpe_s)(void *vd, void *vn, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + uint32_t *d = vd, *n = vn; + + for (i = 0; i < opr_sz / 4; ++i) { + d[i] = helper_recpe_u32(n[i]); + } + clear_tail(d, opr_sz, simd_maxsz(desc)); +} + +void HELPER(gvec_ursqrte_s)(void *vd, void *vn, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + uint32_t *d = vd, *n = vn; + + for (i = 0; i < opr_sz / 4; ++i) { + d[i] = helper_rsqrte_u32(n[i]); + } + clear_tail(d, opr_sz, simd_maxsz(desc)); +} -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Remove handle_2misc_reciprocal as these were the last insns decoded by that function. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-69-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 3 + target/arm/tcg/translate-a64.c | 139 ++------------------------------- 2 files changed, 8 insertions(+), 134 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ FRECPE_v 0.00 1110 1.1 00001 11011 0 ..... ..... @qrr_sd FRSQRTE_v 0.10 1110 111 11001 11011 0 ..... ..... @qrr_h FRSQRTE_v 0.10 1110 1.1 00001 11011 0 ..... ..... @qrr_sd +URECPE_v 0.00 1110 101 00001 11001 0 ..... ..... @qrr_s +URSQRTE_v 0.10 1110 101 00001 11001 0 ..... ..... @qrr_s + &fcvt_q rd rn esz q shift @fcvtq_h . q:1 . ...... 001 .... ...... rn:5 rd:5 \ &fcvt_q esz=1 shift=%fcvt_f_sh_h diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ TRANS(CMLE0_v, do_gvec_fn2, a, gen_gvec_cle0) TRANS(CMEQ0_v, do_gvec_fn2, a, gen_gvec_ceq0) TRANS(REV16_v, do_gvec_fn2, a, gen_gvec_rev16) TRANS(REV32_v, do_gvec_fn2, a, gen_gvec_rev32) +TRANS(URECPE_v, do_gvec_fn2, a, gen_gvec_urecpe) +TRANS(URSQRTE_v, do_gvec_fn2, a, gen_gvec_ursqrte) static bool do_gvec_fn2_bhs(DisasContext *s, arg_qrr_e *a, GVecGen2Fn *fn) { @@ -XXX,XX +XXX,XX @@ static gen_helper_gvec_2_ptr * const f_frsqrte[] = { }; TRANS(FRSQRTE_v, do_gvec_op2_fpst, a->esz, a->q, a->rd, a->rn, 0, f_frsqrte) -static void handle_2misc_reciprocal(DisasContext *s, int opcode, - bool is_scalar, bool is_u, bool is_q, - int size, int rn, int rd) -{ - bool is_double = (size == 3); - - if (is_double) { - g_assert_not_reached(); - } else { - TCGv_i32 tcg_op = tcg_temp_new_i32(); - TCGv_i32 tcg_res = tcg_temp_new_i32(); - int pass, maxpasses; - - if (is_scalar) { - maxpasses = 1; - } else { - maxpasses = is_q ? 4 : 2; - } - - for (pass = 0; pass < maxpasses; pass++) { - read_vec_element_i32(s, tcg_op, rn, pass, MO_32); - - switch (opcode) { - case 0x3c: /* URECPE */ - gen_helper_recpe_u32(tcg_res, tcg_op); - break; - case 0x3d: /* FRECPE */ - case 0x3f: /* FRECPX */ - case 0x7d: /* FRSQRTE */ - default: - g_assert_not_reached(); - } - - if (is_scalar) { - write_fp_sreg(s, rd, tcg_res); - } else { - write_vec_element_i32(s, tcg_res, rd, pass, MO_32); - } - } - if (!is_scalar) { - clear_vec_high(s, is_q, rd); - } - } -} - static void handle_2misc_widening(DisasContext *s, int opcode, bool is_q, int size, int rn, int rd) { @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) bool is_q = extract32(insn, 30, 1); int rn = extract32(insn, 5, 5); int rd = extract32(insn, 0, 5); - bool need_fpstatus = false; - int rmode = -1; - TCGv_i32 tcg_rmode; - TCGv_ptr tcg_fpstatus; switch (opcode) { case 0xc ... 0xf: @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) opcode |= (extract32(size, 1, 1) << 5) | (u << 6); size = is_double ? 3 : 2; switch (opcode) { - case 0x3c: /* URECPE */ - if (size == 3) { - unallocated_encoding(s); - return; - } - if (!fp_access_check(s)) { - return; - } - handle_2misc_reciprocal(s, opcode, false, u, is_q, size, rn, rd); - return; case 0x17: /* FCVTL, FCVTL2 */ if (!fp_access_check(s)) { return; } handle_2misc_widening(s, opcode, is_q, size, rn, rd); return; - case 0x7c: /* URSQRTE */ - if (size == 3) { - unallocated_encoding(s); - return; - } - break; default: case 0x16: /* FCVTN, FCVTN2 */ case 0x36: /* BFCVTN, BFCVTN2 */ @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) case 0x6d: /* FCMLE (zero) */ case 0x3d: /* FRECPE */ case 0x7d: /* FRSQRTE */ + case 0x3c: /* URECPE */ + case 0x7c: /* URSQRTE */ unallocated_encoding(s); return; } @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) unallocated_encoding(s); return; } - - if (!fp_access_check(s)) { - return; - } - - if (need_fpstatus || rmode >= 0) { - tcg_fpstatus = fpstatus_ptr(FPST_FPCR); - } else { - tcg_fpstatus = NULL; - } - if (rmode >= 0) { - tcg_rmode = gen_set_rmode(rmode, tcg_fpstatus); - } else { - tcg_rmode = NULL; - } - - { - int pass; - - assert(size == 2); - for (pass = 0; pass < (is_q ? 4 : 2); pass++) { - TCGv_i32 tcg_op = tcg_temp_new_i32(); - TCGv_i32 tcg_res = tcg_temp_new_i32(); - - read_vec_element_i32(s, tcg_op, rn, pass, MO_32); - - { - /* Special cases for 32 bit elements */ - switch (opcode) { - case 0x7c: /* URSQRTE */ - gen_helper_rsqrte_u32(tcg_res, tcg_op); - break; - default: - case 0x7: /* SQABS, SQNEG */ - case 0x2f: /* FABS */ - case 0x6f: /* FNEG */ - case 0x7f: /* FSQRT */ - case 0x18: /* FRINTN */ - case 0x19: /* FRINTM */ - case 0x38: /* FRINTP */ - case 0x39: /* FRINTZ */ - case 0x58: /* FRINTA */ - case 0x79: /* FRINTI */ - case 0x59: /* FRINTX */ - case 0x1e: /* FRINT32Z */ - case 0x5e: /* FRINT32X */ - case 0x1f: /* FRINT64Z */ - case 0x5f: /* FRINT64X */ - case 0x1a: /* FCVTNS */ - case 0x1b: /* FCVTMS */ - case 0x1c: /* FCVTAS */ - case 0x3a: /* FCVTPS */ - case 0x3b: /* FCVTZS */ - case 0x5a: /* FCVTNU */ - case 0x5b: /* FCVTMU */ - case 0x5c: /* FCVTAU */ - case 0x7a: /* FCVTPU */ - case 0x7b: /* FCVTZU */ - g_assert_not_reached(); - } - } - write_vec_element_i32(s, tcg_res, rd, pass, MO_32); - } - } - clear_vec_high(s, is_q, rd); - - if (tcg_rmode) { - gen_restore_rmode(tcg_rmode, tcg_fpstatus); - } + g_assert_not_reached(); } /* C3.6 Data processing - SIMD, inc Crypto -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Remove lookup_disas_fn, handle_2misc_widening, disas_simd_two_reg_misc, disas_data_proc_simd, disas_data_proc_simd_fp, disas_a64_legacy, as this is the final insn to be converted. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-70-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 2 + target/arm/tcg/translate-a64.c | 202 +++------------------------------ 2 files changed, 18 insertions(+), 186 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ FRSQRTE_v 0.10 1110 1.1 00001 11011 0 ..... ..... @qrr_sd URECPE_v 0.00 1110 101 00001 11001 0 ..... ..... @qrr_s URSQRTE_v 0.10 1110 101 00001 11001 0 ..... ..... @qrr_s +FCVTL_v 0.00 1110 0.1 00001 01111 0 ..... ..... @qrr_sd + &fcvt_q rd rn esz q shift @fcvtq_h . q:1 . ...... 001 .... ...... rn:5 rd:5 \ &fcvt_q esz=1 shift=%fcvt_f_sh_h diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static inline void gen_check_sp_alignment(DisasContext *s) */ } -/* - * This provides a simple table based table lookup decoder. It is - * intended to be used when the relevant bits for decode are too - * awkwardly placed and switch/if based logic would be confusing and - * deeply nested. Since it's a linear search through the table, tables - * should be kept small. - * - * It returns the first handler where insn & mask == pattern, or - * NULL if there is no match. - * The table is terminated by an empty mask (i.e. 0) - */ -static inline AArch64DecodeFn *lookup_disas_fn(const AArch64DecodeTable *table, - uint32_t insn) -{ - const AArch64DecodeTable *tptr = table; - - while (tptr->mask) { - if ((insn & tptr->mask) == tptr->pattern) { - return tptr->disas_fn; - } - tptr++; - } - return NULL; -} - /* * The instruction disassembly implemented here matches * the instruction encoding classifications in chapter C4 @@ -XXX,XX +XXX,XX @@ static gen_helper_gvec_2_ptr * const f_frsqrte[] = { }; TRANS(FRSQRTE_v, do_gvec_op2_fpst, a->esz, a->q, a->rd, a->rn, 0, f_frsqrte) -static void handle_2misc_widening(DisasContext *s, int opcode, bool is_q, - int size, int rn, int rd) +static bool trans_FCVTL_v(DisasContext *s, arg_qrr_e *a) { /* Handle 2-reg-misc ops which are widening (so each size element * in the source becomes a 2*size element in the destination. @@ -XXX,XX +XXX,XX @@ static void handle_2misc_widening(DisasContext *s, int opcode, bool is_q, */ int pass; - if (size == 3) { + if (!fp_access_check(s)) { + return true; + } + + if (a->esz == MO_64) { /* 32 -> 64 bit fp conversion */ TCGv_i64 tcg_res[2]; - int srcelt = is_q ? 2 : 0; + TCGv_i32 tcg_op = tcg_temp_new_i32(); + int srcelt = a->q ? 2 : 0; for (pass = 0; pass < 2; pass++) { - TCGv_i32 tcg_op = tcg_temp_new_i32(); tcg_res[pass] = tcg_temp_new_i64(); - - read_vec_element_i32(s, tcg_op, rn, srcelt + pass, MO_32); + read_vec_element_i32(s, tcg_op, a->rn, srcelt + pass, MO_32); gen_helper_vfp_fcvtds(tcg_res[pass], tcg_op, tcg_env); } for (pass = 0; pass < 2; pass++) { - write_vec_element(s, tcg_res[pass], rd, pass, MO_64); + write_vec_element(s, tcg_res[pass], a->rd, pass, MO_64); } } else { /* 16 -> 32 bit fp conversion */ - int srcelt = is_q ? 4 : 0; + int srcelt = a->q ? 4 : 0; TCGv_i32 tcg_res[4]; TCGv_ptr fpst = fpstatus_ptr(FPST_FPCR); TCGv_i32 ahp = get_ahp_flag(); for (pass = 0; pass < 4; pass++) { tcg_res[pass] = tcg_temp_new_i32(); - - read_vec_element_i32(s, tcg_res[pass], rn, srcelt + pass, MO_16); + read_vec_element_i32(s, tcg_res[pass], a->rn, srcelt + pass, MO_16); gen_helper_vfp_fcvt_f16_to_f32(tcg_res[pass], tcg_res[pass], fpst, ahp); } for (pass = 0; pass < 4; pass++) { - write_vec_element_i32(s, tcg_res[pass], rd, pass, MO_32); + write_vec_element_i32(s, tcg_res[pass], a->rd, pass, MO_32); } } -} - -/* AdvSIMD two reg misc - * 31 30 29 28 24 23 22 21 17 16 12 11 10 9 5 4 0 - * +---+---+---+-----------+------+-----------+--------+-----+------+------+ - * | 0 | Q | U | 0 1 1 1 0 | size | 1 0 0 0 0 | opcode | 1 0 | Rn | Rd | - * +---+---+---+-----------+------+-----------+--------+-----+------+------+ - */ -static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) -{ - int size = extract32(insn, 22, 2); - int opcode = extract32(insn, 12, 5); - bool u = extract32(insn, 29, 1); - bool is_q = extract32(insn, 30, 1); - int rn = extract32(insn, 5, 5); - int rd = extract32(insn, 0, 5); - - switch (opcode) { - case 0xc ... 0xf: - case 0x16 ... 0x1f: - { - /* Floating point: U, size[1] and opcode indicate operation; - * size[0] indicates single or double precision. - */ - int is_double = extract32(size, 0, 1); - opcode |= (extract32(size, 1, 1) << 5) | (u << 6); - size = is_double ? 3 : 2; - switch (opcode) { - case 0x17: /* FCVTL, FCVTL2 */ - if (!fp_access_check(s)) { - return; - } - handle_2misc_widening(s, opcode, is_q, size, rn, rd); - return; - default: - case 0x16: /* FCVTN, FCVTN2 */ - case 0x36: /* BFCVTN, BFCVTN2 */ - case 0x56: /* FCVTXN, FCVTXN2 */ - case 0x2f: /* FABS */ - case 0x6f: /* FNEG */ - case 0x7f: /* FSQRT */ - case 0x18: /* FRINTN */ - case 0x19: /* FRINTM */ - case 0x38: /* FRINTP */ - case 0x39: /* FRINTZ */ - case 0x59: /* FRINTX */ - case 0x79: /* FRINTI */ - case 0x58: /* FRINTA */ - case 0x1e: /* FRINT32Z */ - case 0x1f: /* FRINT64Z */ - case 0x5e: /* FRINT32X */ - case 0x5f: /* FRINT64X */ - case 0x1d: /* SCVTF */ - case 0x5d: /* UCVTF */ - case 0x1a: /* FCVTNS */ - case 0x1b: /* FCVTMS */ - case 0x3a: /* FCVTPS */ - case 0x3b: /* FCVTZS */ - case 0x5a: /* FCVTNU */ - case 0x5b: /* FCVTMU */ - case 0x7a: /* FCVTPU */ - case 0x7b: /* FCVTZU */ - case 0x5c: /* FCVTAU */ - case 0x1c: /* FCVTAS */ - case 0x2c: /* FCMGT (zero) */ - case 0x2d: /* FCMEQ (zero) */ - case 0x2e: /* FCMLT (zero) */ - case 0x6c: /* FCMGE (zero) */ - case 0x6d: /* FCMLE (zero) */ - case 0x3d: /* FRECPE */ - case 0x7d: /* FRSQRTE */ - case 0x3c: /* URECPE */ - case 0x7c: /* URSQRTE */ - unallocated_encoding(s); - return; - } - break; - } - default: - case 0x0: /* REV64, REV32 */ - case 0x1: /* REV16 */ - case 0x2: /* SADDLP, UADDLP */ - case 0x3: /* SUQADD, USQADD */ - case 0x4: /* CLS, CLZ */ - case 0x5: /* CNT, NOT, RBIT */ - case 0x6: /* SADALP, UADALP */ - case 0x7: /* SQABS, SQNEG */ - case 0x8: /* CMGT, CMGE */ - case 0x9: /* CMEQ, CMLE */ - case 0xa: /* CMLT */ - case 0xb: /* ABS, NEG */ - case 0x12: /* XTN, XTN2, SQXTUN, SQXTUN2 */ - case 0x13: /* SHLL, SHLL2 */ - case 0x14: /* SQXTN, SQXTN2, UQXTN, UQXTN2 */ - unallocated_encoding(s); - return; - } - g_assert_not_reached(); -} - -/* C3.6 Data processing - SIMD, inc Crypto - * - * As the decode gets a little complex we are using a table based - * approach for this part of the decode. - */ -static const AArch64DecodeTable data_proc_simd[] = { - /* pattern , mask , fn */ - { 0x0e200800, 0x9f3e0c00, disas_simd_two_reg_misc }, - { 0x00000000, 0x00000000, NULL } -}; - -static void disas_data_proc_simd(DisasContext *s, uint32_t insn) -{ - /* Note that this is called with all non-FP cases from - * table C3-6 so it must UNDEF for entries not specifically - * allocated to instructions in that table. - */ - AArch64DecodeFn *fn = lookup_disas_fn(&data_proc_simd[0], insn); - if (fn) { - fn(s, insn); - } else { - unallocated_encoding(s); - } -} - -/* C3.6 Data processing - SIMD and floating point */ -static void disas_data_proc_simd_fp(DisasContext *s, uint32_t insn) -{ - if (extract32(insn, 28, 1) == 1 && extract32(insn, 30, 1) == 0) { - unallocated_encoding(s); /* in decodetree */ - } else { - /* SIMD, including crypto */ - disas_data_proc_simd(s, insn); - } + clear_vec_high(s, true, a->rd); + return true; } static bool trans_OK(DisasContext *s, arg_OK *a) @@ -XXX,XX +XXX,XX @@ static bool btype_destination_ok(uint32_t insn, bool bt, int btype) return false; } -/* C3.1 A64 instruction index by encoding */ -static void disas_a64_legacy(DisasContext *s, uint32_t insn) -{ - switch (extract32(insn, 25, 4)) { - case 0x7: - case 0xf: /* Data processing - SIMD and floating point */ - disas_data_proc_simd_fp(s, insn); - break; - default: - unallocated_encoding(s); - break; - } -} - static void aarch64_tr_init_disas_context(DisasContextBase *dcbase, CPUState *cpu) { @@ -XXX,XX +XXX,XX @@ static void aarch64_tr_translate_insn(DisasContextBase *dcbase, CPUState *cpu) if (!disas_a64(s, insn) && !disas_sme(s, insn) && !disas_sve(s, insn)) { - disas_a64_legacy(s, insn); + unallocated_encoding(s); } /* -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Softfloat has native support for round-to-odd. Use it. Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241206031428.78634-1-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/helper-a64.c | 18 ++++-------------- 1 file changed, 4 insertions(+), 14 deletions(-) diff --git a/target/arm/tcg/helper-a64.c b/target/arm/tcg/helper-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/helper-a64.c +++ b/target/arm/tcg/helper-a64.c @@ -XXX,XX +XXX,XX @@ float64 HELPER(frecpx_f64)(float64 a, void *fpstp) float32 HELPER(fcvtx_f64_to_f32)(float64 a, CPUARMState *env) { - /* Von Neumann rounding is implemented by using round-to-zero - * and then setting the LSB of the result if Inexact was raised. - */ float32 r; float_status *fpst = &env->vfp.fp_status; - float_status tstat = *fpst; - int exflags; + int old = get_float_rounding_mode(fpst); - set_float_rounding_mode(float_round_to_zero, &tstat); - set_float_exception_flags(0, &tstat); - r = float64_to_float32(a, &tstat); - exflags = get_float_exception_flags(&tstat); - if (exflags & float_flag_inexact) { - r = make_float32(float32_val(r) | 1); - } - exflags |= get_float_exception_flags(fpst); - set_float_exception_flags(exflags, fpst); + set_float_rounding_mode(float_round_to_odd, fpst); + r = float64_to_float32(a, fpst); + set_float_rounding_mode(old, fpst); return r; } -- 2.34.1 From: Pierrick Bouvier <pierrick.bouvier@linaro.org> www.orangepi.org does not support https, it's expected to stick to http. Reviewed-by: Niek Linnenbank <nieklinnenbank@gmail.com> Signed-off-by: Pierrick Bouvier <pierrick.bouvier@linaro.org> Message-id: 20241206192254.3889131-2-pierrick.bouvier@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- docs/system/arm/orangepi.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/system/arm/orangepi.rst b/docs/system/arm/orangepi.rst index XXXXXXX..XXXXXXX 100644 --- a/docs/system/arm/orangepi.rst +++ b/docs/system/arm/orangepi.rst @@ -XXX,XX +XXX,XX @@ Orange Pi PC images Note that the mainline kernel does not have a root filesystem. You may provide it with an official Orange Pi PC image from the official website: - http://www.orangepi.org/downloadresources/ + http://www.orangepi.org/html/serviceAndSupport/index.html Another possibility is to run an Armbian image for Orange Pi PC which can be downloaded from: @@ -XXX,XX +XXX,XX @@ including the Orange Pi PC. NetBSD 9.0 is known to work best for the Orange Pi P board and provides a fully working system with serial console, networking and storage. For the Orange Pi PC machine, get the 'evbarm-earmv7hf' based image from: - https://cdn.netbsd.org/pub/NetBSD/NetBSD-9.0/evbarm-earmv7hf/binary/gzimg/armv7.img.gz + https://archive.netbsd.org/pub/NetBSD-archive/NetBSD-9.0/evbarm-earmv7hf/binary/gzimg/armv7.img.gz The image requires manually installing U-Boot in the image. Build U-Boot with the orangepi_pc_defconfig configuration as described in the previous section. -- 2.34.1 From: Pierrick Bouvier <pierrick.bouvier@linaro.org> Reviewed-by: Cédric Le Goater <clg@redhat.com> Signed-off-by: Pierrick Bouvier <pierrick.bouvier@linaro.org> Message-id: 20241206192254.3889131-3-pierrick.bouvier@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- docs/system/arm/fby35.rst | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/docs/system/arm/fby35.rst b/docs/system/arm/fby35.rst index XXXXXXX..XXXXXXX 100644 --- a/docs/system/arm/fby35.rst +++ b/docs/system/arm/fby35.rst @@ -XXX,XX +XXX,XX @@ process starts. $ screen /dev/tty0 # In a separate TMUX pane, terminal window, etc. $ screen /dev/tty1 $ (qemu) c # Start the boot process once screen is setup. + +This machine model supports emulation of the boot from the CE0 flash device by +setting option ``execute-in-place``. When using this option, the CPU fetches +instructions to execute by reading CE0 and not from a preloaded ROM +initialized at machine init time. As a result, execution will be slower. -- 2.34.1 From: Pierrick Bouvier <pierrick.bouvier@linaro.org> Signed-off-by: Pierrick Bouvier <pierrick.bouvier@linaro.org> Message-id: 20241206192254.3889131-4-pierrick.bouvier@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> --- docs/system/arm/xlnx-versal-virt.rst | 3 +++ 1 file changed, 3 insertions(+) diff --git a/docs/system/arm/xlnx-versal-virt.rst b/docs/system/arm/xlnx-versal-virt.rst index XXXXXXX..XXXXXXX 100644 --- a/docs/system/arm/xlnx-versal-virt.rst +++ b/docs/system/arm/xlnx-versal-virt.rst @@ -XXX,XX +XXX,XX @@ Run the following at the U-Boot prompt: fdt set /chosen/dom0 reg <0x00000000 0x40000000 0x0 0x03100000> booti 30000000 - 20000000 +It's possible to change the OSPI flash model emulated by using the machine model +option ``ospi-flash``. + BBRAM File Backend """""""""""""""""" BBRAM can have an optional file backend, which must be a seekable -- 2.34.1 From: Pierrick Bouvier <pierrick.bouvier@linaro.org> Signed-off-by: Pierrick Bouvier <pierrick.bouvier@linaro.org> Message-id: 20241206192254.3889131-5-pierrick.bouvier@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> --- docs/system/arm/virt.rst | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/docs/system/arm/virt.rst b/docs/system/arm/virt.rst index XXXXXXX..XXXXXXX 100644 --- a/docs/system/arm/virt.rst +++ b/docs/system/arm/virt.rst @@ -XXX,XX +XXX,XX @@ iommu ``smmuv3`` Create an SMMUv3 +default-bus-bypass-iommu + Set ``on``/``off`` to enable/disable `bypass_iommu + <https://gitlab.com/qemu-project/qemu/-/blob/master/docs/bypass-iommu.txt>`_ + for default root bus. + ras Set ``on``/``off`` to enable/disable reporting host memory errors to a guest using ACPI and guest external abort exceptions. The default is off. +acpi + Set ``on``/``off``/``auto`` to enable/disable ACPI. + dtb-randomness Set ``on``/``off`` to pass random seeds via the guest DTB rng-seed and kaslr-seed nodes (in both "/chosen" and @@ -XXX,XX +XXX,XX @@ dtb-randomness dtb-kaslr-seed A deprecated synonym for dtb-randomness. +x-oem-id + Set string (up to 6 bytes) to override the default value of field OEMID in ACPI + table header. + +x-oem-table-id + Set string (up to 8 bytes) to override the default value of field OEM Table ID + in ACPI table header. + Linux guest kernel configuration """""""""""""""""""""""""""""""" -- 2.34.1 From: Brian Cain <brian.cain@oss.qualcomm.com> Mea culpa, I don't know how I got this wrong in 2dfe93699c. Still getting used to the new address, I suppose. Somehow I got it right in the mailmap, though. Signed-off-by: Brian Cain <brian.cain@oss.qualcomm.com> Message-id: 20241209181242.1434231-1-brian.cain@oss.qualcomm.com Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- MAINTAINERS | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/MAINTAINERS b/MAINTAINERS index XXXXXXX..XXXXXXX 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -XXX,XX +XXX,XX @@ F: target/avr/ F: tests/functional/test_avr_mega2560.py Hexagon TCG CPUs -M: Brian Cain <bcain@oss.qualcomm.com> +M: Brian Cain <brian.cain@oss.qualcomm.com> S: Supported F: target/hexagon/ X: target/hexagon/idef-parser/ -- 2.34.1 target/arm/helper.c is very large and unwieldy. One subset of code that we can pull out into its own file is the cpreg arrays and corresponding functions for the TLBI instructions. Because these are instructions they are only relevant for TCG and we can make the new file only be built for CONFIG_TCG. In this commit we move the AArch32 instructions from: not_v7_cp_reginfo[] v7_cp_reginfo[] v7mp_cp_reginfo[] v8_cp_reginfo[] into a new file target/arm/tcg/tlb-insns.c. A few small functions are used both by functions we haven't yet moved across and by functions we have already moved. We temporarily make these global with a prototype in cpregs.h; when the move of all TLBI insns is complete these will return to being file-local. For CONFIG_TCG, this is just moving code around. For a KVM only build, these cpregs will no longer be added to the cpregs hashtable for the CPU. However this should not be a behaviour change, because: * we never try to migration sync or otherwise include ARM_CP_NO_RAW cpregs * for migration we treat the kernel's list of system registers as the authoritative one, so these TLBI insns were never in it anyway The no-tcg stub of define_tlb_insn_regs() therefore does nothing. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241210160452.2427965-2-peter.maydell@linaro.org --- target/arm/cpregs.h | 14 +++ target/arm/internals.h | 3 + target/arm/helper.c | 231 ++-------------------------------- target/arm/tcg-stubs.c | 5 + target/arm/tcg/tlb-insns.c | 246 +++++++++++++++++++++++++++++++++++++ target/arm/tcg/meson.build | 1 + 6 files changed, 280 insertions(+), 220 deletions(-) create mode 100644 target/arm/tcg/tlb-insns.c diff --git a/target/arm/cpregs.h b/target/arm/cpregs.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpregs.h +++ b/target/arm/cpregs.h @@ -XXX,XX +XXX,XX @@ static inline bool arm_cpreg_traps_in_nv(const ARMCPRegInfo *ri) return ri->opc1 == 4 || ri->opc1 == 5; } +/* + * Temporary declarations of functions until the move to tlb_insn_helper.c + * is complete and we can make the functions static again + */ +CPAccessResult access_ttlb(CPUARMState *env, const ARMCPRegInfo *ri, + bool isread); +CPAccessResult access_ttlbis(CPUARMState *env, const ARMCPRegInfo *ri, + bool isread); +bool tlb_force_broadcast(CPUARMState *env); +void tlbimva_hyp_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value); +void tlbimva_hyp_is_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value); + #endif /* TARGET_ARM_CPREGS_H */ diff --git a/target/arm/internals.h b/target/arm/internals.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/internals.h +++ b/target/arm/internals.h @@ -XXX,XX +XXX,XX @@ static inline uint64_t pauth_ptr_mask(ARMVAParameters param) /* Add the cpreg definitions for debug related system registers */ void define_debug_regs(ARMCPU *cpu); +/* Add the cpreg definitions for TLBI instructions */ +void define_tlb_insn_regs(ARMCPU *cpu); + /* Effective value of MDCR_EL2 */ static inline uint64_t arm_mdcr_el2_eff(CPUARMState *env) { diff --git a/target/arm/helper.c b/target/arm/helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -XXX,XX +XXX,XX @@ static CPAccessResult access_tacr(CPUARMState *env, const ARMCPRegInfo *ri, } /* Check for traps from EL1 due to HCR_EL2.TTLB. */ -static CPAccessResult access_ttlb(CPUARMState *env, const ARMCPRegInfo *ri, - bool isread) +CPAccessResult access_ttlb(CPUARMState *env, const ARMCPRegInfo *ri, + bool isread) { if (arm_current_el(env) == 1 && (arm_hcr_el2_eff(env) & HCR_TTLB)) { return CP_ACCESS_TRAP_EL2; @@ -XXX,XX +XXX,XX @@ static CPAccessResult access_ttlb(CPUARMState *env, const ARMCPRegInfo *ri, } /* Check for traps from EL1 due to HCR_EL2.TTLB or TTLBIS. */ -static CPAccessResult access_ttlbis(CPUARMState *env, const ARMCPRegInfo *ri, - bool isread) +CPAccessResult access_ttlbis(CPUARMState *env, const ARMCPRegInfo *ri, + bool isread) { if (arm_current_el(env) == 1 && (arm_hcr_el2_eff(env) & (HCR_TTLB | HCR_TTLBIS))) { @@ -XXX,XX +XXX,XX @@ static int alle1_tlbmask(CPUARMState *env) ARMMMUIdxBit_Stage2_S); } - -/* IS variants of TLB operations must affect all cores */ -static void tlbiall_is_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) -{ - CPUState *cs = env_cpu(env); - - tlb_flush_all_cpus_synced(cs); -} - -static void tlbiasid_is_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) -{ - CPUState *cs = env_cpu(env); - - tlb_flush_all_cpus_synced(cs); -} - -static void tlbimva_is_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) -{ - CPUState *cs = env_cpu(env); - - tlb_flush_page_all_cpus_synced(cs, value & TARGET_PAGE_MASK); -} - -static void tlbimvaa_is_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) -{ - CPUState *cs = env_cpu(env); - - tlb_flush_page_all_cpus_synced(cs, value & TARGET_PAGE_MASK); -} - /* * Non-IS variants of TLB operations are upgraded to * IS versions if we are at EL1 and HCR_EL2.FB is effectively set to * force broadcast of these operations. */ -static bool tlb_force_broadcast(CPUARMState *env) +bool tlb_force_broadcast(CPUARMState *env) { return arm_current_el(env) == 1 && (arm_hcr_el2_eff(env) & HCR_FB); } -static void tlbiall_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) -{ - /* Invalidate all (TLBIALL) */ - CPUState *cs = env_cpu(env); - - if (tlb_force_broadcast(env)) { - tlb_flush_all_cpus_synced(cs); - } else { - tlb_flush(cs); - } -} - -static void tlbimva_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) -{ - /* Invalidate single TLB entry by MVA and ASID (TLBIMVA) */ - CPUState *cs = env_cpu(env); - - value &= TARGET_PAGE_MASK; - if (tlb_force_broadcast(env)) { - tlb_flush_page_all_cpus_synced(cs, value); - } else { - tlb_flush_page(cs, value); - } -} - -static void tlbiasid_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) -{ - /* Invalidate by ASID (TLBIASID) */ - CPUState *cs = env_cpu(env); - - if (tlb_force_broadcast(env)) { - tlb_flush_all_cpus_synced(cs); - } else { - tlb_flush(cs); - } -} - -static void tlbimvaa_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) -{ - /* Invalidate single entry by MVA, all ASIDs (TLBIMVAA) */ - CPUState *cs = env_cpu(env); - - value &= TARGET_PAGE_MASK; - if (tlb_force_broadcast(env)) { - tlb_flush_page_all_cpus_synced(cs, value); - } else { - tlb_flush_page(cs, value); - } -} - static void tlbiall_nsnh_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value) { @@ -XXX,XX +XXX,XX @@ static void tlbiall_hyp_is_write(CPUARMState *env, const ARMCPRegInfo *ri, tlb_flush_by_mmuidx_all_cpus_synced(cs, ARMMMUIdxBit_E2); } -static void tlbimva_hyp_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) +void tlbimva_hyp_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) { CPUState *cs = env_cpu(env); uint64_t pageaddr = value & ~MAKE_64BIT_MASK(0, 12); @@ -XXX,XX +XXX,XX @@ static void tlbimva_hyp_write(CPUARMState *env, const ARMCPRegInfo *ri, tlb_flush_page_by_mmuidx(cs, pageaddr, ARMMMUIdxBit_E2); } -static void tlbimva_hyp_is_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) +void tlbimva_hyp_is_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) { CPUState *cs = env_cpu(env); uint64_t pageaddr = value & ~MAKE_64BIT_MASK(0, 12); @@ -XXX,XX +XXX,XX @@ static void tlbimva_hyp_is_write(CPUARMState *env, const ARMCPRegInfo *ri, ARMMMUIdxBit_E2); } -static void tlbiipas2_hyp_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) -{ - CPUState *cs = env_cpu(env); - uint64_t pageaddr = (value & MAKE_64BIT_MASK(0, 28)) << 12; - - tlb_flush_page_by_mmuidx(cs, pageaddr, ARMMMUIdxBit_Stage2); -} - -static void tlbiipas2is_hyp_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) -{ - CPUState *cs = env_cpu(env); - uint64_t pageaddr = (value & MAKE_64BIT_MASK(0, 28)) << 12; - - tlb_flush_page_by_mmuidx_all_cpus_synced(cs, pageaddr, ARMMMUIdxBit_Stage2); -} - static const ARMCPRegInfo cp_reginfo[] = { /* * Define the secure and non-secure FCSE identifier CP registers @@ -XXX,XX +XXX,XX @@ static const ARMCPRegInfo not_v7_cp_reginfo[] = { */ { .name = "DBGDIDR", .cp = 14, .crn = 0, .crm = 0, .opc1 = 0, .opc2 = 0, .access = PL0_R, .type = ARM_CP_CONST, .resetvalue = 0 }, - /* - * MMU TLB control. Note that the wildcarding means we cover not just - * the unified TLB ops but also the dside/iside/inner-shareable variants. - */ - { .name = "TLBIALL", .cp = 15, .crn = 8, .crm = CP_ANY, - .opc1 = CP_ANY, .opc2 = 0, .access = PL1_W, .writefn = tlbiall_write, - .type = ARM_CP_NO_RAW }, - { .name = "TLBIMVA", .cp = 15, .crn = 8, .crm = CP_ANY, - .opc1 = CP_ANY, .opc2 = 1, .access = PL1_W, .writefn = tlbimva_write, - .type = ARM_CP_NO_RAW }, - { .name = "TLBIASID", .cp = 15, .crn = 8, .crm = CP_ANY, - .opc1 = CP_ANY, .opc2 = 2, .access = PL1_W, .writefn = tlbiasid_write, - .type = ARM_CP_NO_RAW }, - { .name = "TLBIMVAA", .cp = 15, .crn = 8, .crm = CP_ANY, - .opc1 = CP_ANY, .opc2 = 3, .access = PL1_W, .writefn = tlbimvaa_write, - .type = ARM_CP_NO_RAW }, { .name = "PRRR", .cp = 15, .crn = 10, .crm = 2, .opc1 = 0, .opc2 = 0, .access = PL1_RW, .type = ARM_CP_NOP }, { .name = "NMRR", .cp = 15, .crn = 10, .crm = 2, @@ -XXX,XX +XXX,XX @@ static const ARMCPRegInfo v7_cp_reginfo[] = { .opc0 = 3, .opc1 = 0, .crn = 12, .crm = 1, .opc2 = 0, .fgt = FGT_ISR_EL1, .type = ARM_CP_NO_RAW, .access = PL1_R, .readfn = isr_read }, - /* 32 bit ITLB invalidates */ - { .name = "ITLBIALL", .cp = 15, .opc1 = 0, .crn = 8, .crm = 5, .opc2 = 0, - .type = ARM_CP_NO_RAW, .access = PL1_W, .accessfn = access_ttlb, - .writefn = tlbiall_write }, - { .name = "ITLBIMVA", .cp = 15, .opc1 = 0, .crn = 8, .crm = 5, .opc2 = 1, - .type = ARM_CP_NO_RAW, .access = PL1_W, .accessfn = access_ttlb, - .writefn = tlbimva_write }, - { .name = "ITLBIASID", .cp = 15, .opc1 = 0, .crn = 8, .crm = 5, .opc2 = 2, - .type = ARM_CP_NO_RAW, .access = PL1_W, .accessfn = access_ttlb, - .writefn = tlbiasid_write }, - /* 32 bit DTLB invalidates */ - { .name = "DTLBIALL", .cp = 15, .opc1 = 0, .crn = 8, .crm = 6, .opc2 = 0, - .type = ARM_CP_NO_RAW, .access = PL1_W, .accessfn = access_ttlb, - .writefn = tlbiall_write }, - { .name = "DTLBIMVA", .cp = 15, .opc1 = 0, .crn = 8, .crm = 6, .opc2 = 1, - .type = ARM_CP_NO_RAW, .access = PL1_W, .accessfn = access_ttlb, - .writefn = tlbimva_write }, - { .name = "DTLBIASID", .cp = 15, .opc1 = 0, .crn = 8, .crm = 6, .opc2 = 2, - .type = ARM_CP_NO_RAW, .access = PL1_W, .accessfn = access_ttlb, - .writefn = tlbiasid_write }, - /* 32 bit TLB invalidates */ - { .name = "TLBIALL", .cp = 15, .opc1 = 0, .crn = 8, .crm = 7, .opc2 = 0, - .type = ARM_CP_NO_RAW, .access = PL1_W, .accessfn = access_ttlb, - .writefn = tlbiall_write }, - { .name = "TLBIMVA", .cp = 15, .opc1 = 0, .crn = 8, .crm = 7, .opc2 = 1, - .type = ARM_CP_NO_RAW, .access = PL1_W, .accessfn = access_ttlb, - .writefn = tlbimva_write }, - { .name = "TLBIASID", .cp = 15, .opc1 = 0, .crn = 8, .crm = 7, .opc2 = 2, - .type = ARM_CP_NO_RAW, .access = PL1_W, .accessfn = access_ttlb, - .writefn = tlbiasid_write }, - { .name = "TLBIMVAA", .cp = 15, .opc1 = 0, .crn = 8, .crm = 7, .opc2 = 3, - .type = ARM_CP_NO_RAW, .access = PL1_W, .accessfn = access_ttlb, - .writefn = tlbimvaa_write }, -}; - -static const ARMCPRegInfo v7mp_cp_reginfo[] = { - /* 32 bit TLB invalidates, Inner Shareable */ - { .name = "TLBIALLIS", .cp = 15, .opc1 = 0, .crn = 8, .crm = 3, .opc2 = 0, - .type = ARM_CP_NO_RAW, .access = PL1_W, .accessfn = access_ttlbis, - .writefn = tlbiall_is_write }, - { .name = "TLBIMVAIS", .cp = 15, .opc1 = 0, .crn = 8, .crm = 3, .opc2 = 1, - .type = ARM_CP_NO_RAW, .access = PL1_W, .accessfn = access_ttlbis, - .writefn = tlbimva_is_write }, - { .name = "TLBIASIDIS", .cp = 15, .opc1 = 0, .crn = 8, .crm = 3, .opc2 = 2, - .type = ARM_CP_NO_RAW, .access = PL1_W, .accessfn = access_ttlbis, - .writefn = tlbiasid_is_write }, - { .name = "TLBIMVAAIS", .cp = 15, .opc1 = 0, .crn = 8, .crm = 3, .opc2 = 3, - .type = ARM_CP_NO_RAW, .access = PL1_W, .accessfn = access_ttlbis, - .writefn = tlbimvaa_is_write }, }; static const ARMCPRegInfo pmovsset_cp_reginfo[] = { @@ -XXX,XX +XXX,XX @@ static const ARMCPRegInfo v8_cp_reginfo[] = { .fieldoffset = offsetof(CPUARMState, cp15.par_el[1]), .writefn = par_write }, #endif - /* TLB invalidate last level of translation table walk */ - { .name = "TLBIMVALIS", .cp = 15, .opc1 = 0, .crn = 8, .crm = 3, .opc2 = 5, - .type = ARM_CP_NO_RAW, .access = PL1_W, .accessfn = access_ttlbis, - .writefn = tlbimva_is_write }, - { .name = "TLBIMVAALIS", .cp = 15, .opc1 = 0, .crn = 8, .crm = 3, .opc2 = 7, - .type = ARM_CP_NO_RAW, .access = PL1_W, .accessfn = access_ttlbis, - .writefn = tlbimvaa_is_write }, - { .name = "TLBIMVAL", .cp = 15, .opc1 = 0, .crn = 8, .crm = 7, .opc2 = 5, - .type = ARM_CP_NO_RAW, .access = PL1_W, .accessfn = access_ttlb, - .writefn = tlbimva_write }, - { .name = "TLBIMVAAL", .cp = 15, .opc1 = 0, .crn = 8, .crm = 7, .opc2 = 7, - .type = ARM_CP_NO_RAW, .access = PL1_W, .accessfn = access_ttlb, - .writefn = tlbimvaa_write }, - { .name = "TLBIMVALH", .cp = 15, .opc1 = 4, .crn = 8, .crm = 7, .opc2 = 5, - .type = ARM_CP_NO_RAW, .access = PL2_W, - .writefn = tlbimva_hyp_write }, - { .name = "TLBIMVALHIS", - .cp = 15, .opc1 = 4, .crn = 8, .crm = 3, .opc2 = 5, - .type = ARM_CP_NO_RAW, .access = PL2_W, - .writefn = tlbimva_hyp_is_write }, - { .name = "TLBIIPAS2", - .cp = 15, .opc1 = 4, .crn = 8, .crm = 4, .opc2 = 1, - .type = ARM_CP_NO_RAW, .access = PL2_W, - .writefn = tlbiipas2_hyp_write }, - { .name = "TLBIIPAS2IS", - .cp = 15, .opc1 = 4, .crn = 8, .crm = 0, .opc2 = 1, - .type = ARM_CP_NO_RAW, .access = PL2_W, - .writefn = tlbiipas2is_hyp_write }, - { .name = "TLBIIPAS2L", - .cp = 15, .opc1 = 4, .crn = 8, .crm = 4, .opc2 = 5, - .type = ARM_CP_NO_RAW, .access = PL2_W, - .writefn = tlbiipas2_hyp_write }, - { .name = "TLBIIPAS2LIS", - .cp = 15, .opc1 = 4, .crn = 8, .crm = 0, .opc2 = 5, - .type = ARM_CP_NO_RAW, .access = PL2_W, - .writefn = tlbiipas2is_hyp_write }, /* 32 bit cache operations */ { .name = "ICIALLUIS", .cp = 15, .opc1 = 0, .crn = 7, .crm = 1, .opc2 = 0, .type = ARM_CP_NOP, .access = PL1_W, .accessfn = access_ticab }, @@ -XXX,XX +XXX,XX @@ void register_cp_regs_for_features(ARMCPU *cpu) define_arm_cp_regs(cpu, not_v8_cp_reginfo); } + define_tlb_insn_regs(cpu); + if (arm_feature(env, ARM_FEATURE_V6)) { /* The ID registers all have impdef reset values */ ARMCPRegInfo v6_idregs[] = { @@ -XXX,XX +XXX,XX @@ void register_cp_regs_for_features(ARMCPU *cpu) if (arm_feature(env, ARM_FEATURE_V6K)) { define_arm_cp_regs(cpu, v6k_cp_reginfo); } - if (arm_feature(env, ARM_FEATURE_V7MP) && - !arm_feature(env, ARM_FEATURE_PMSA)) { - define_arm_cp_regs(cpu, v7mp_cp_reginfo); - } if (arm_feature(env, ARM_FEATURE_V7VE)) { define_arm_cp_regs(cpu, pmovsset_cp_reginfo); } diff --git a/target/arm/tcg-stubs.c b/target/arm/tcg-stubs.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg-stubs.c +++ b/target/arm/tcg-stubs.c @@ -XXX,XX +XXX,XX @@ void raise_exception_ra(CPUARMState *env, uint32_t excp, uint32_t syndrome, void assert_hflags_rebuild_correctly(CPUARMState *env) { } + +/* TLBI insns are only used by TCG, so we don't need to do anything for KVM */ +void define_tlb_insn_regs(ARMCPU *cpu) +{ +} diff --git a/target/arm/tcg/tlb-insns.c b/target/arm/tcg/tlb-insns.c new file mode 100644 index XXXXXXX..XXXXXXX --- /dev/null +++ b/target/arm/tcg/tlb-insns.c @@ -XXX,XX +XXX,XX @@ +/* + * Helpers for TLBI insns + * + * This code is licensed under the GNU GPL v2 or later. + * + * SPDX-License-Identifier: GPL-2.0-or-later + */ +#include "qemu/osdep.h" +#include "exec/exec-all.h" +#include "cpu.h" +#include "internals.h" +#include "cpu-features.h" +#include "cpregs.h" + +/* IS variants of TLB operations must affect all cores */ +static void tlbiall_is_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) +{ + CPUState *cs = env_cpu(env); + + tlb_flush_all_cpus_synced(cs); +} + +static void tlbiasid_is_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) +{ + CPUState *cs = env_cpu(env); + + tlb_flush_all_cpus_synced(cs); +} + +static void tlbimva_is_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) +{ + CPUState *cs = env_cpu(env); + + tlb_flush_page_all_cpus_synced(cs, value & TARGET_PAGE_MASK); +} + +static void tlbimvaa_is_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) +{ + CPUState *cs = env_cpu(env); + + tlb_flush_page_all_cpus_synced(cs, value & TARGET_PAGE_MASK); +} + +static void tlbiall_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) +{ + /* Invalidate all (TLBIALL) */ + CPUState *cs = env_cpu(env); + + if (tlb_force_broadcast(env)) { + tlb_flush_all_cpus_synced(cs); + } else { + tlb_flush(cs); + } +} + +static void tlbimva_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) +{ + /* Invalidate single TLB entry by MVA and ASID (TLBIMVA) */ + CPUState *cs = env_cpu(env); + + value &= TARGET_PAGE_MASK; + if (tlb_force_broadcast(env)) { + tlb_flush_page_all_cpus_synced(cs, value); + } else { + tlb_flush_page(cs, value); + } +} + +static void tlbiasid_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) +{ + /* Invalidate by ASID (TLBIASID) */ + CPUState *cs = env_cpu(env); + + if (tlb_force_broadcast(env)) { + tlb_flush_all_cpus_synced(cs); + } else { + tlb_flush(cs); + } +} + +static void tlbimvaa_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) +{ + /* Invalidate single entry by MVA, all ASIDs (TLBIMVAA) */ + CPUState *cs = env_cpu(env); + + value &= TARGET_PAGE_MASK; + if (tlb_force_broadcast(env)) { + tlb_flush_page_all_cpus_synced(cs, value); + } else { + tlb_flush_page(cs, value); + } +} + +static void tlbiipas2_hyp_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) +{ + CPUState *cs = env_cpu(env); + uint64_t pageaddr = (value & MAKE_64BIT_MASK(0, 28)) << 12; + + tlb_flush_page_by_mmuidx(cs, pageaddr, ARMMMUIdxBit_Stage2); +} + +static void tlbiipas2is_hyp_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) +{ + CPUState *cs = env_cpu(env); + uint64_t pageaddr = (value & MAKE_64BIT_MASK(0, 28)) << 12; + + tlb_flush_page_by_mmuidx_all_cpus_synced(cs, pageaddr, ARMMMUIdxBit_Stage2); +} + +static const ARMCPRegInfo tlbi_not_v7_cp_reginfo[] = { + /* + * MMU TLB control. Note that the wildcarding means we cover not just + * the unified TLB ops but also the dside/iside/inner-shareable variants. + */ + { .name = "TLBIALL", .cp = 15, .crn = 8, .crm = CP_ANY, + .opc1 = CP_ANY, .opc2 = 0, .access = PL1_W, .writefn = tlbiall_write, + .type = ARM_CP_NO_RAW }, + { .name = "TLBIMVA", .cp = 15, .crn = 8, .crm = CP_ANY, + .opc1 = CP_ANY, .opc2 = 1, .access = PL1_W, .writefn = tlbimva_write, + .type = ARM_CP_NO_RAW }, + { .name = "TLBIASID", .cp = 15, .crn = 8, .crm = CP_ANY, + .opc1 = CP_ANY, .opc2 = 2, .access = PL1_W, .writefn = tlbiasid_write, + .type = ARM_CP_NO_RAW }, + { .name = "TLBIMVAA", .cp = 15, .crn = 8, .crm = CP_ANY, + .opc1 = CP_ANY, .opc2 = 3, .access = PL1_W, .writefn = tlbimvaa_write, + .type = ARM_CP_NO_RAW }, +}; + +static const ARMCPRegInfo tlbi_v7_cp_reginfo[] = { + /* 32 bit ITLB invalidates */ + { .name = "ITLBIALL", .cp = 15, .opc1 = 0, .crn = 8, .crm = 5, .opc2 = 0, + .type = ARM_CP_NO_RAW, .access = PL1_W, .accessfn = access_ttlb, + .writefn = tlbiall_write }, + { .name = "ITLBIMVA", .cp = 15, .opc1 = 0, .crn = 8, .crm = 5, .opc2 = 1, + .type = ARM_CP_NO_RAW, .access = PL1_W, .accessfn = access_ttlb, + .writefn = tlbimva_write }, + { .name = "ITLBIASID", .cp = 15, .opc1 = 0, .crn = 8, .crm = 5, .opc2 = 2, + .type = ARM_CP_NO_RAW, .access = PL1_W, .accessfn = access_ttlb, + .writefn = tlbiasid_write }, + /* 32 bit DTLB invalidates */ + { .name = "DTLBIALL", .cp = 15, .opc1 = 0, .crn = 8, .crm = 6, .opc2 = 0, + .type = ARM_CP_NO_RAW, .access = PL1_W, .accessfn = access_ttlb, + .writefn = tlbiall_write }, + { .name = "DTLBIMVA", .cp = 15, .opc1 = 0, .crn = 8, .crm = 6, .opc2 = 1, + .type = ARM_CP_NO_RAW, .access = PL1_W, .accessfn = access_ttlb, + .writefn = tlbimva_write }, + { .name = "DTLBIASID", .cp = 15, .opc1 = 0, .crn = 8, .crm = 6, .opc2 = 2, + .type = ARM_CP_NO_RAW, .access = PL1_W, .accessfn = access_ttlb, + .writefn = tlbiasid_write }, + /* 32 bit TLB invalidates */ + { .name = "TLBIALL", .cp = 15, .opc1 = 0, .crn = 8, .crm = 7, .opc2 = 0, + .type = ARM_CP_NO_RAW, .access = PL1_W, .accessfn = access_ttlb, + .writefn = tlbiall_write }, + { .name = "TLBIMVA", .cp = 15, .opc1 = 0, .crn = 8, .crm = 7, .opc2 = 1, + .type = ARM_CP_NO_RAW, .access = PL1_W, .accessfn = access_ttlb, + .writefn = tlbimva_write }, + { .name = "TLBIASID", .cp = 15, .opc1 = 0, .crn = 8, .crm = 7, .opc2 = 2, + .type = ARM_CP_NO_RAW, .access = PL1_W, .accessfn = access_ttlb, + .writefn = tlbiasid_write }, + { .name = "TLBIMVAA", .cp = 15, .opc1 = 0, .crn = 8, .crm = 7, .opc2 = 3, + .type = ARM_CP_NO_RAW, .access = PL1_W, .accessfn = access_ttlb, + .writefn = tlbimvaa_write }, +}; + +static const ARMCPRegInfo tlbi_v7mp_cp_reginfo[] = { + /* 32 bit TLB invalidates, Inner Shareable */ + { .name = "TLBIALLIS", .cp = 15, .opc1 = 0, .crn = 8, .crm = 3, .opc2 = 0, + .type = ARM_CP_NO_RAW, .access = PL1_W, .accessfn = access_ttlbis, + .writefn = tlbiall_is_write }, + { .name = "TLBIMVAIS", .cp = 15, .opc1 = 0, .crn = 8, .crm = 3, .opc2 = 1, + .type = ARM_CP_NO_RAW, .access = PL1_W, .accessfn = access_ttlbis, + .writefn = tlbimva_is_write }, + { .name = "TLBIASIDIS", .cp = 15, .opc1 = 0, .crn = 8, .crm = 3, .opc2 = 2, + .type = ARM_CP_NO_RAW, .access = PL1_W, .accessfn = access_ttlbis, + .writefn = tlbiasid_is_write }, + { .name = "TLBIMVAAIS", .cp = 15, .opc1 = 0, .crn = 8, .crm = 3, .opc2 = 3, + .type = ARM_CP_NO_RAW, .access = PL1_W, .accessfn = access_ttlbis, + .writefn = tlbimvaa_is_write }, +}; + +static const ARMCPRegInfo tlbi_v8_cp_reginfo[] = { + /* AArch32 TLB invalidate last level of translation table walk */ + { .name = "TLBIMVALIS", .cp = 15, .opc1 = 0, .crn = 8, .crm = 3, .opc2 = 5, + .type = ARM_CP_NO_RAW, .access = PL1_W, .accessfn = access_ttlbis, + .writefn = tlbimva_is_write }, + { .name = "TLBIMVAALIS", .cp = 15, .opc1 = 0, .crn = 8, .crm = 3, .opc2 = 7, + .type = ARM_CP_NO_RAW, .access = PL1_W, .accessfn = access_ttlbis, + .writefn = tlbimvaa_is_write }, + { .name = "TLBIMVAL", .cp = 15, .opc1 = 0, .crn = 8, .crm = 7, .opc2 = 5, + .type = ARM_CP_NO_RAW, .access = PL1_W, .accessfn = access_ttlb, + .writefn = tlbimva_write }, + { .name = "TLBIMVAAL", .cp = 15, .opc1 = 0, .crn = 8, .crm = 7, .opc2 = 7, + .type = ARM_CP_NO_RAW, .access = PL1_W, .accessfn = access_ttlb, + .writefn = tlbimvaa_write }, + { .name = "TLBIMVALH", .cp = 15, .opc1 = 4, .crn = 8, .crm = 7, .opc2 = 5, + .type = ARM_CP_NO_RAW, .access = PL2_W, + .writefn = tlbimva_hyp_write }, + { .name = "TLBIMVALHIS", + .cp = 15, .opc1 = 4, .crn = 8, .crm = 3, .opc2 = 5, + .type = ARM_CP_NO_RAW, .access = PL2_W, + .writefn = tlbimva_hyp_is_write }, + { .name = "TLBIIPAS2", + .cp = 15, .opc1 = 4, .crn = 8, .crm = 4, .opc2 = 1, + .type = ARM_CP_NO_RAW, .access = PL2_W, + .writefn = tlbiipas2_hyp_write }, + { .name = "TLBIIPAS2IS", + .cp = 15, .opc1 = 4, .crn = 8, .crm = 0, .opc2 = 1, + .type = ARM_CP_NO_RAW, .access = PL2_W, + .writefn = tlbiipas2is_hyp_write }, + { .name = "TLBIIPAS2L", + .cp = 15, .opc1 = 4, .crn = 8, .crm = 4, .opc2 = 5, + .type = ARM_CP_NO_RAW, .access = PL2_W, + .writefn = tlbiipas2_hyp_write }, + { .name = "TLBIIPAS2LIS", + .cp = 15, .opc1 = 4, .crn = 8, .crm = 0, .opc2 = 5, + .type = ARM_CP_NO_RAW, .access = PL2_W, + .writefn = tlbiipas2is_hyp_write }, +}; + +void define_tlb_insn_regs(ARMCPU *cpu) +{ + CPUARMState *env = &cpu->env; + + if (!arm_feature(env, ARM_FEATURE_V7)) { + define_arm_cp_regs(cpu, tlbi_not_v7_cp_reginfo); + } else { + define_arm_cp_regs(cpu, tlbi_v7_cp_reginfo); + } + if (arm_feature(env, ARM_FEATURE_V7MP) && + !arm_feature(env, ARM_FEATURE_PMSA)) { + define_arm_cp_regs(cpu, tlbi_v7mp_cp_reginfo); + } + if (arm_feature(env, ARM_FEATURE_V8)) { + define_arm_cp_regs(cpu, tlbi_v8_cp_reginfo); + } +} diff --git a/target/arm/tcg/meson.build b/target/arm/tcg/meson.build index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/meson.build +++ b/target/arm/tcg/meson.build @@ -XXX,XX +XXX,XX @@ arm_ss.add(files( 'op_helper.c', 'tlb_helper.c', 'vec_helper.c', + 'tlb-insns.c', )) arm_ss.add(when: 'TARGET_AARCH64', if_true: files( -- 2.34.1 Move the AArch32 TLBI insns for AArch32 EL2 to tlbi_insn_helper.c. To keep this as an obviously pure code-movement, we retain the same condition for registering tlbi_el2_cp_reginfo that we use for el2_cp_reginfo. We'll be able to simplify this condition later, since the need to define the reginfo for EL3-without-EL2 doesn't apply for the TLBI ops specifically. This move brings all the uses of tlbimva_hyp_write() and tlbimva_hyp_is_write() back into a single file, so we can move those also, and make them file-local again. The helper alle1_tlbmask() is an exception to the pattern that we only need to make these functions global temporarily, because once this refactoring is complete it will be called by both code in helper.c (vttbr_write()) and by code in tlb-insns.c. We therefore put its prototype in a permanent home in internals.h. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241210160452.2427965-3-peter.maydell@linaro.org --- target/arm/cpregs.h | 4 -- target/arm/internals.h | 6 +++ target/arm/helper.c | 74 +-------------------------------- target/arm/tcg/tlb-insns.c | 85 ++++++++++++++++++++++++++++++++++++++ 4 files changed, 92 insertions(+), 77 deletions(-) diff --git a/target/arm/cpregs.h b/target/arm/cpregs.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpregs.h +++ b/target/arm/cpregs.h @@ -XXX,XX +XXX,XX @@ CPAccessResult access_ttlb(CPUARMState *env, const ARMCPRegInfo *ri, CPAccessResult access_ttlbis(CPUARMState *env, const ARMCPRegInfo *ri, bool isread); bool tlb_force_broadcast(CPUARMState *env); -void tlbimva_hyp_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value); -void tlbimva_hyp_is_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value); #endif /* TARGET_ARM_CPREGS_H */ diff --git a/target/arm/internals.h b/target/arm/internals.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/internals.h +++ b/target/arm/internals.h @@ -XXX,XX +XXX,XX @@ uint64_t gt_get_countervalue(CPUARMState *env); * and CNTVCT_EL0 (this will be either 0 or the value of CNTVOFF_EL2). */ uint64_t gt_virt_cnt_offset(CPUARMState *env); + +/* + * Return mask of ARMMMUIdxBit values corresponding to an "invalidate + * all EL1" scope; this covers stage 1 and stage 2. + */ +int alle1_tlbmask(CPUARMState *env); #endif diff --git a/target/arm/helper.c b/target/arm/helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -XXX,XX +XXX,XX @@ static void contextidr_write(CPUARMState *env, const ARMCPRegInfo *ri, raw_write(env, ri, value); } -static int alle1_tlbmask(CPUARMState *env) +int alle1_tlbmask(CPUARMState *env) { /* * Note that the 'ALL' scope must invalidate both stage 1 and @@ -XXX,XX +XXX,XX @@ bool tlb_force_broadcast(CPUARMState *env) return arm_current_el(env) == 1 && (arm_hcr_el2_eff(env) & HCR_FB); } -static void tlbiall_nsnh_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) -{ - CPUState *cs = env_cpu(env); - - tlb_flush_by_mmuidx(cs, alle1_tlbmask(env)); -} - -static void tlbiall_nsnh_is_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) -{ - CPUState *cs = env_cpu(env); - - tlb_flush_by_mmuidx_all_cpus_synced(cs, alle1_tlbmask(env)); -} - - -static void tlbiall_hyp_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) -{ - CPUState *cs = env_cpu(env); - - tlb_flush_by_mmuidx(cs, ARMMMUIdxBit_E2); -} - -static void tlbiall_hyp_is_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) -{ - CPUState *cs = env_cpu(env); - - tlb_flush_by_mmuidx_all_cpus_synced(cs, ARMMMUIdxBit_E2); -} - -void tlbimva_hyp_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) -{ - CPUState *cs = env_cpu(env); - uint64_t pageaddr = value & ~MAKE_64BIT_MASK(0, 12); - - tlb_flush_page_by_mmuidx(cs, pageaddr, ARMMMUIdxBit_E2); -} - -void tlbimva_hyp_is_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) -{ - CPUState *cs = env_cpu(env); - uint64_t pageaddr = value & ~MAKE_64BIT_MASK(0, 12); - - tlb_flush_page_by_mmuidx_all_cpus_synced(cs, pageaddr, - ARMMMUIdxBit_E2); -} - static const ARMCPRegInfo cp_reginfo[] = { /* * Define the secure and non-secure FCSE identifier CP registers @@ -XXX,XX +XXX,XX @@ static const ARMCPRegInfo el2_cp_reginfo[] = { { .name = "HTTBR", .cp = 15, .opc1 = 4, .crm = 2, .access = PL2_RW, .type = ARM_CP_64BIT | ARM_CP_ALIAS, .fieldoffset = offsetof(CPUARMState, cp15.ttbr0_el[2]) }, - { .name = "TLBIALLNSNH", - .cp = 15, .opc1 = 4, .crn = 8, .crm = 7, .opc2 = 4, - .type = ARM_CP_NO_RAW, .access = PL2_W, - .writefn = tlbiall_nsnh_write }, - { .name = "TLBIALLNSNHIS", - .cp = 15, .opc1 = 4, .crn = 8, .crm = 3, .opc2 = 4, - .type = ARM_CP_NO_RAW, .access = PL2_W, - .writefn = tlbiall_nsnh_is_write }, - { .name = "TLBIALLH", .cp = 15, .opc1 = 4, .crn = 8, .crm = 7, .opc2 = 0, - .type = ARM_CP_NO_RAW, .access = PL2_W, - .writefn = tlbiall_hyp_write }, - { .name = "TLBIALLHIS", .cp = 15, .opc1 = 4, .crn = 8, .crm = 3, .opc2 = 0, - .type = ARM_CP_NO_RAW, .access = PL2_W, - .writefn = tlbiall_hyp_is_write }, - { .name = "TLBIMVAH", .cp = 15, .opc1 = 4, .crn = 8, .crm = 7, .opc2 = 1, - .type = ARM_CP_NO_RAW, .access = PL2_W, - .writefn = tlbimva_hyp_write }, - { .name = "TLBIMVAHIS", .cp = 15, .opc1 = 4, .crn = 8, .crm = 3, .opc2 = 1, - .type = ARM_CP_NO_RAW, .access = PL2_W, - .writefn = tlbimva_hyp_is_write }, { .name = "TLBI_ALLE2", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 7, .opc2 = 0, .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF, diff --git a/target/arm/tcg/tlb-insns.c b/target/arm/tcg/tlb-insns.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/tlb-insns.c +++ b/target/arm/tcg/tlb-insns.c @@ -XXX,XX +XXX,XX @@ static void tlbimvaa_write(CPUARMState *env, const ARMCPRegInfo *ri, } } +static void tlbimva_hyp_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) +{ + CPUState *cs = env_cpu(env); + uint64_t pageaddr = value & ~MAKE_64BIT_MASK(0, 12); + + tlb_flush_page_by_mmuidx(cs, pageaddr, ARMMMUIdxBit_E2); +} + +static void tlbimva_hyp_is_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) +{ + CPUState *cs = env_cpu(env); + uint64_t pageaddr = value & ~MAKE_64BIT_MASK(0, 12); + + tlb_flush_page_by_mmuidx_all_cpus_synced(cs, pageaddr, + ARMMMUIdxBit_E2); +} + static void tlbiipas2_hyp_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value) { @@ -XXX,XX +XXX,XX @@ static void tlbiipas2is_hyp_write(CPUARMState *env, const ARMCPRegInfo *ri, tlb_flush_page_by_mmuidx_all_cpus_synced(cs, pageaddr, ARMMMUIdxBit_Stage2); } +static void tlbiall_nsnh_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) +{ + CPUState *cs = env_cpu(env); + + tlb_flush_by_mmuidx(cs, alle1_tlbmask(env)); +} + +static void tlbiall_nsnh_is_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) +{ + CPUState *cs = env_cpu(env); + + tlb_flush_by_mmuidx_all_cpus_synced(cs, alle1_tlbmask(env)); +} + + +static void tlbiall_hyp_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) +{ + CPUState *cs = env_cpu(env); + + tlb_flush_by_mmuidx(cs, ARMMMUIdxBit_E2); +} + +static void tlbiall_hyp_is_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) +{ + CPUState *cs = env_cpu(env); + + tlb_flush_by_mmuidx_all_cpus_synced(cs, ARMMMUIdxBit_E2); +} + static const ARMCPRegInfo tlbi_not_v7_cp_reginfo[] = { /* * MMU TLB control. Note that the wildcarding means we cover not just @@ -XXX,XX +XXX,XX @@ static const ARMCPRegInfo tlbi_v8_cp_reginfo[] = { .writefn = tlbiipas2is_hyp_write }, }; +static const ARMCPRegInfo tlbi_el2_cp_reginfo[] = { + { .name = "TLBIALLNSNH", + .cp = 15, .opc1 = 4, .crn = 8, .crm = 7, .opc2 = 4, + .type = ARM_CP_NO_RAW, .access = PL2_W, + .writefn = tlbiall_nsnh_write }, + { .name = "TLBIALLNSNHIS", + .cp = 15, .opc1 = 4, .crn = 8, .crm = 3, .opc2 = 4, + .type = ARM_CP_NO_RAW, .access = PL2_W, + .writefn = tlbiall_nsnh_is_write }, + { .name = "TLBIALLH", .cp = 15, .opc1 = 4, .crn = 8, .crm = 7, .opc2 = 0, + .type = ARM_CP_NO_RAW, .access = PL2_W, + .writefn = tlbiall_hyp_write }, + { .name = "TLBIALLHIS", .cp = 15, .opc1 = 4, .crn = 8, .crm = 3, .opc2 = 0, + .type = ARM_CP_NO_RAW, .access = PL2_W, + .writefn = tlbiall_hyp_is_write }, + { .name = "TLBIMVAH", .cp = 15, .opc1 = 4, .crn = 8, .crm = 7, .opc2 = 1, + .type = ARM_CP_NO_RAW, .access = PL2_W, + .writefn = tlbimva_hyp_write }, + { .name = "TLBIMVAHIS", .cp = 15, .opc1 = 4, .crn = 8, .crm = 3, .opc2 = 1, + .type = ARM_CP_NO_RAW, .access = PL2_W, + .writefn = tlbimva_hyp_is_write }, +}; + void define_tlb_insn_regs(ARMCPU *cpu) { CPUARMState *env = &cpu->env; @@ -XXX,XX +XXX,XX @@ void define_tlb_insn_regs(ARMCPU *cpu) if (arm_feature(env, ARM_FEATURE_V8)) { define_arm_cp_regs(cpu, tlbi_v8_cp_reginfo); } + /* + * We retain the existing logic for when to register these TLBI + * ops (i.e. matching the condition for el2_cp_reginfo[] in + * helper.c), but we will be able to simplify this later. + */ + if (arm_feature(env, ARM_FEATURE_EL2) + || (arm_feature(env, ARM_FEATURE_EL3) + && arm_feature(env, ARM_FEATURE_V8))) { + define_arm_cp_regs(cpu, tlbi_el2_cp_reginfo); + } } -- 2.34.1 Move the AArch64 TLBI insns that are declared in v8_cp_reginfo[] into tlb-insns.c. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241210160452.2427965-4-peter.maydell@linaro.org --- target/arm/cpregs.h | 11 +++ target/arm/helper.c | 182 +++---------------------------------- target/arm/tcg/tlb-insns.c | 160 ++++++++++++++++++++++++++++++++ 3 files changed, 182 insertions(+), 171 deletions(-) diff --git a/target/arm/cpregs.h b/target/arm/cpregs.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpregs.h +++ b/target/arm/cpregs.h @@ -XXX,XX +XXX,XX @@ CPAccessResult access_ttlb(CPUARMState *env, const ARMCPRegInfo *ri, CPAccessResult access_ttlbis(CPUARMState *env, const ARMCPRegInfo *ri, bool isread); bool tlb_force_broadcast(CPUARMState *env); +int tlbbits_for_regime(CPUARMState *env, ARMMMUIdx mmu_idx, + uint64_t addr); +int vae1_tlbbits(CPUARMState *env, uint64_t addr); +int vae1_tlbmask(CPUARMState *env); +int ipas2e1_tlbmask(CPUARMState *env, int64_t value); +void tlbi_aa64_vmalle1is_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value); +void tlbi_aa64_alle1is_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value); +void tlbi_aa64_vae1is_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value); #endif /* TARGET_ARM_CPREGS_H */ diff --git a/target/arm/helper.c b/target/arm/helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -XXX,XX +XXX,XX @@ static CPAccessResult access_tocu(CPUARMState *env, const ARMCPRegInfo *ri, * Page D4-1736 (DDI0487A.b) */ -static int vae1_tlbmask(CPUARMState *env) +int vae1_tlbmask(CPUARMState *env) { uint64_t hcr = arm_hcr_el2_eff(env); uint16_t mask; @@ -XXX,XX +XXX,XX @@ static int vae2_tlbmask(CPUARMState *env) } /* Return 56 if TBI is enabled, 64 otherwise. */ -static int tlbbits_for_regime(CPUARMState *env, ARMMMUIdx mmu_idx, - uint64_t addr) +int tlbbits_for_regime(CPUARMState *env, ARMMMUIdx mmu_idx, + uint64_t addr) { uint64_t tcr = regime_tcr(env, mmu_idx); int tbi = aa64_va_parameter_tbi(tcr, mmu_idx); @@ -XXX,XX +XXX,XX @@ static int tlbbits_for_regime(CPUARMState *env, ARMMMUIdx mmu_idx, return (tbi >> select) & 1 ? 56 : 64; } -static int vae1_tlbbits(CPUARMState *env, uint64_t addr) +int vae1_tlbbits(CPUARMState *env, uint64_t addr) { uint64_t hcr = arm_hcr_el2_eff(env); ARMMMUIdx mmu_idx; @@ -XXX,XX +XXX,XX @@ static int vae2_tlbbits(CPUARMState *env, uint64_t addr) return tlbbits_for_regime(env, mmu_idx, addr); } -static void tlbi_aa64_vmalle1is_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) +void tlbi_aa64_vmalle1is_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) { CPUState *cs = env_cpu(env); int mask = vae1_tlbmask(env); @@ -XXX,XX +XXX,XX @@ static void tlbi_aa64_vmalle1is_write(CPUARMState *env, const ARMCPRegInfo *ri, tlb_flush_by_mmuidx_all_cpus_synced(cs, mask); } -static void tlbi_aa64_vmalle1_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) -{ - CPUState *cs = env_cpu(env); - int mask = vae1_tlbmask(env); - - if (tlb_force_broadcast(env)) { - tlb_flush_by_mmuidx_all_cpus_synced(cs, mask); - } else { - tlb_flush_by_mmuidx(cs, mask); - } -} - static int e2_tlbmask(CPUARMState *env) { return (ARMMMUIdxBit_E20_0 | @@ -XXX,XX +XXX,XX @@ static int e2_tlbmask(CPUARMState *env) ARMMMUIdxBit_E2); } -static void tlbi_aa64_alle1_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) -{ - CPUState *cs = env_cpu(env); - int mask = alle1_tlbmask(env); - - tlb_flush_by_mmuidx(cs, mask); -} - static void tlbi_aa64_alle2_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value) { @@ -XXX,XX +XXX,XX @@ static void tlbi_aa64_alle3_write(CPUARMState *env, const ARMCPRegInfo *ri, tlb_flush_by_mmuidx(cs, ARMMMUIdxBit_E3); } -static void tlbi_aa64_alle1is_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) +void tlbi_aa64_alle1is_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) { CPUState *cs = env_cpu(env); int mask = alle1_tlbmask(env); @@ -XXX,XX +XXX,XX @@ static void tlbi_aa64_vae3_write(CPUARMState *env, const ARMCPRegInfo *ri, tlb_flush_page_by_mmuidx(cs, pageaddr, ARMMMUIdxBit_E3); } -static void tlbi_aa64_vae1is_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) +void tlbi_aa64_vae1is_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) { CPUState *cs = env_cpu(env); int mask = vae1_tlbmask(env); @@ -XXX,XX +XXX,XX @@ static void tlbi_aa64_vae1is_write(CPUARMState *env, const ARMCPRegInfo *ri, tlb_flush_page_bits_by_mmuidx_all_cpus_synced(cs, pageaddr, mask, bits); } -static void tlbi_aa64_vae1_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) -{ - /* - * Invalidate by VA, EL1&0 (AArch64 version). - * Currently handles all of VAE1, VAAE1, VAALE1 and VALE1, - * since we don't support flush-for-specific-ASID-only or - * flush-last-level-only. - */ - CPUState *cs = env_cpu(env); - int mask = vae1_tlbmask(env); - uint64_t pageaddr = sextract64(value << 12, 0, 56); - int bits = vae1_tlbbits(env, pageaddr); - - if (tlb_force_broadcast(env)) { - tlb_flush_page_bits_by_mmuidx_all_cpus_synced(cs, pageaddr, mask, bits); - } else { - tlb_flush_page_bits_by_mmuidx(cs, pageaddr, mask, bits); - } -} - static void tlbi_aa64_vae2is_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value) { @@ -XXX,XX +XXX,XX @@ static void tlbi_aa64_vae3is_write(CPUARMState *env, const ARMCPRegInfo *ri, ARMMMUIdxBit_E3, bits); } -static int ipas2e1_tlbmask(CPUARMState *env, int64_t value) +int ipas2e1_tlbmask(CPUARMState *env, int64_t value) { /* * The MSB of value is the NS field, which only applies if SEL2 @@ -XXX,XX +XXX,XX @@ static int ipas2e1_tlbmask(CPUARMState *env, int64_t value) : ARMMMUIdxBit_Stage2); } -static void tlbi_aa64_ipas2e1_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) -{ - CPUState *cs = env_cpu(env); - int mask = ipas2e1_tlbmask(env, value); - uint64_t pageaddr = sextract64(value << 12, 0, 56); - - if (tlb_force_broadcast(env)) { - tlb_flush_page_by_mmuidx_all_cpus_synced(cs, pageaddr, mask); - } else { - tlb_flush_page_by_mmuidx(cs, pageaddr, mask); - } -} - -static void tlbi_aa64_ipas2e1is_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) -{ - CPUState *cs = env_cpu(env); - int mask = ipas2e1_tlbmask(env, value); - uint64_t pageaddr = sextract64(value << 12, 0, 56); - - tlb_flush_page_by_mmuidx_all_cpus_synced(cs, pageaddr, mask); -} - #ifdef TARGET_AARCH64 typedef struct { uint64_t base; @@ -XXX,XX +XXX,XX @@ static const ARMCPRegInfo v8_cp_reginfo[] = { .opc0 = 1, .opc1 = 0, .crn = 7, .crm = 14, .opc2 = 2, .fgt = FGT_DCCISW, .access = PL1_W, .accessfn = access_tsw, .type = ARM_CP_NOP }, - /* TLBI operations */ - { .name = "TLBI_VMALLE1IS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 3, .opc2 = 0, - .access = PL1_W, .accessfn = access_ttlbis, .type = ARM_CP_NO_RAW, - .fgt = FGT_TLBIVMALLE1IS, - .writefn = tlbi_aa64_vmalle1is_write }, - { .name = "TLBI_VAE1IS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 3, .opc2 = 1, - .access = PL1_W, .accessfn = access_ttlbis, .type = ARM_CP_NO_RAW, - .fgt = FGT_TLBIVAE1IS, - .writefn = tlbi_aa64_vae1is_write }, - { .name = "TLBI_ASIDE1IS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 3, .opc2 = 2, - .access = PL1_W, .accessfn = access_ttlbis, .type = ARM_CP_NO_RAW, - .fgt = FGT_TLBIASIDE1IS, - .writefn = tlbi_aa64_vmalle1is_write }, - { .name = "TLBI_VAAE1IS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 3, .opc2 = 3, - .access = PL1_W, .accessfn = access_ttlbis, .type = ARM_CP_NO_RAW, - .fgt = FGT_TLBIVAAE1IS, - .writefn = tlbi_aa64_vae1is_write }, - { .name = "TLBI_VALE1IS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 3, .opc2 = 5, - .access = PL1_W, .accessfn = access_ttlbis, .type = ARM_CP_NO_RAW, - .fgt = FGT_TLBIVALE1IS, - .writefn = tlbi_aa64_vae1is_write }, - { .name = "TLBI_VAALE1IS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 3, .opc2 = 7, - .access = PL1_W, .accessfn = access_ttlbis, .type = ARM_CP_NO_RAW, - .fgt = FGT_TLBIVAALE1IS, - .writefn = tlbi_aa64_vae1is_write }, - { .name = "TLBI_VMALLE1", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 7, .opc2 = 0, - .access = PL1_W, .accessfn = access_ttlb, .type = ARM_CP_NO_RAW, - .fgt = FGT_TLBIVMALLE1, - .writefn = tlbi_aa64_vmalle1_write }, - { .name = "TLBI_VAE1", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 7, .opc2 = 1, - .access = PL1_W, .accessfn = access_ttlb, .type = ARM_CP_NO_RAW, - .fgt = FGT_TLBIVAE1, - .writefn = tlbi_aa64_vae1_write }, - { .name = "TLBI_ASIDE1", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 7, .opc2 = 2, - .access = PL1_W, .accessfn = access_ttlb, .type = ARM_CP_NO_RAW, - .fgt = FGT_TLBIASIDE1, - .writefn = tlbi_aa64_vmalle1_write }, - { .name = "TLBI_VAAE1", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 7, .opc2 = 3, - .access = PL1_W, .accessfn = access_ttlb, .type = ARM_CP_NO_RAW, - .fgt = FGT_TLBIVAAE1, - .writefn = tlbi_aa64_vae1_write }, - { .name = "TLBI_VALE1", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 7, .opc2 = 5, - .access = PL1_W, .accessfn = access_ttlb, .type = ARM_CP_NO_RAW, - .fgt = FGT_TLBIVALE1, - .writefn = tlbi_aa64_vae1_write }, - { .name = "TLBI_VAALE1", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 7, .opc2 = 7, - .access = PL1_W, .accessfn = access_ttlb, .type = ARM_CP_NO_RAW, - .fgt = FGT_TLBIVAALE1, - .writefn = tlbi_aa64_vae1_write }, - { .name = "TLBI_IPAS2E1IS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 0, .opc2 = 1, - .access = PL2_W, .type = ARM_CP_NO_RAW, - .writefn = tlbi_aa64_ipas2e1is_write }, - { .name = "TLBI_IPAS2LE1IS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 0, .opc2 = 5, - .access = PL2_W, .type = ARM_CP_NO_RAW, - .writefn = tlbi_aa64_ipas2e1is_write }, - { .name = "TLBI_ALLE1IS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 3, .opc2 = 4, - .access = PL2_W, .type = ARM_CP_NO_RAW, - .writefn = tlbi_aa64_alle1is_write }, - { .name = "TLBI_VMALLS12E1IS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 3, .opc2 = 6, - .access = PL2_W, .type = ARM_CP_NO_RAW, - .writefn = tlbi_aa64_alle1is_write }, - { .name = "TLBI_IPAS2E1", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 4, .opc2 = 1, - .access = PL2_W, .type = ARM_CP_NO_RAW, - .writefn = tlbi_aa64_ipas2e1_write }, - { .name = "TLBI_IPAS2LE1", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 4, .opc2 = 5, - .access = PL2_W, .type = ARM_CP_NO_RAW, - .writefn = tlbi_aa64_ipas2e1_write }, - { .name = "TLBI_ALLE1", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 7, .opc2 = 4, - .access = PL2_W, .type = ARM_CP_NO_RAW, - .writefn = tlbi_aa64_alle1_write }, - { .name = "TLBI_VMALLS12E1", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 7, .opc2 = 6, - .access = PL2_W, .type = ARM_CP_NO_RAW, - .writefn = tlbi_aa64_alle1is_write }, #ifndef CONFIG_USER_ONLY /* 64 bit address translation operations */ { .name = "AT_S1E1R", .state = ARM_CP_STATE_AA64, diff --git a/target/arm/tcg/tlb-insns.c b/target/arm/tcg/tlb-insns.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/tlb-insns.c +++ b/target/arm/tcg/tlb-insns.c @@ -XXX,XX +XXX,XX @@ static void tlbiall_hyp_is_write(CPUARMState *env, const ARMCPRegInfo *ri, tlb_flush_by_mmuidx_all_cpus_synced(cs, ARMMMUIdxBit_E2); } +static void tlbi_aa64_vmalle1_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) +{ + CPUState *cs = env_cpu(env); + int mask = vae1_tlbmask(env); + + if (tlb_force_broadcast(env)) { + tlb_flush_by_mmuidx_all_cpus_synced(cs, mask); + } else { + tlb_flush_by_mmuidx(cs, mask); + } +} + +static void tlbi_aa64_alle1_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) +{ + CPUState *cs = env_cpu(env); + int mask = alle1_tlbmask(env); + + tlb_flush_by_mmuidx(cs, mask); +} + +static void tlbi_aa64_vae1_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) +{ + /* + * Invalidate by VA, EL1&0 (AArch64 version). + * Currently handles all of VAE1, VAAE1, VAALE1 and VALE1, + * since we don't support flush-for-specific-ASID-only or + * flush-last-level-only. + */ + CPUState *cs = env_cpu(env); + int mask = vae1_tlbmask(env); + uint64_t pageaddr = sextract64(value << 12, 0, 56); + int bits = vae1_tlbbits(env, pageaddr); + + if (tlb_force_broadcast(env)) { + tlb_flush_page_bits_by_mmuidx_all_cpus_synced(cs, pageaddr, mask, bits); + } else { + tlb_flush_page_bits_by_mmuidx(cs, pageaddr, mask, bits); + } +} + +static void tlbi_aa64_ipas2e1_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) +{ + CPUState *cs = env_cpu(env); + int mask = ipas2e1_tlbmask(env, value); + uint64_t pageaddr = sextract64(value << 12, 0, 56); + + if (tlb_force_broadcast(env)) { + tlb_flush_page_by_mmuidx_all_cpus_synced(cs, pageaddr, mask); + } else { + tlb_flush_page_by_mmuidx(cs, pageaddr, mask); + } +} + +static void tlbi_aa64_ipas2e1is_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) +{ + CPUState *cs = env_cpu(env); + int mask = ipas2e1_tlbmask(env, value); + uint64_t pageaddr = sextract64(value << 12, 0, 56); + + tlb_flush_page_by_mmuidx_all_cpus_synced(cs, pageaddr, mask); +} + static const ARMCPRegInfo tlbi_not_v7_cp_reginfo[] = { /* * MMU TLB control. Note that the wildcarding means we cover not just @@ -XXX,XX +XXX,XX @@ static const ARMCPRegInfo tlbi_v8_cp_reginfo[] = { .cp = 15, .opc1 = 4, .crn = 8, .crm = 0, .opc2 = 5, .type = ARM_CP_NO_RAW, .access = PL2_W, .writefn = tlbiipas2is_hyp_write }, + /* AArch64 TLBI operations */ + { .name = "TLBI_VMALLE1IS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 3, .opc2 = 0, + .access = PL1_W, .accessfn = access_ttlbis, .type = ARM_CP_NO_RAW, + .fgt = FGT_TLBIVMALLE1IS, + .writefn = tlbi_aa64_vmalle1is_write }, + { .name = "TLBI_VAE1IS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 3, .opc2 = 1, + .access = PL1_W, .accessfn = access_ttlbis, .type = ARM_CP_NO_RAW, + .fgt = FGT_TLBIVAE1IS, + .writefn = tlbi_aa64_vae1is_write }, + { .name = "TLBI_ASIDE1IS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 3, .opc2 = 2, + .access = PL1_W, .accessfn = access_ttlbis, .type = ARM_CP_NO_RAW, + .fgt = FGT_TLBIASIDE1IS, + .writefn = tlbi_aa64_vmalle1is_write }, + { .name = "TLBI_VAAE1IS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 3, .opc2 = 3, + .access = PL1_W, .accessfn = access_ttlbis, .type = ARM_CP_NO_RAW, + .fgt = FGT_TLBIVAAE1IS, + .writefn = tlbi_aa64_vae1is_write }, + { .name = "TLBI_VALE1IS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 3, .opc2 = 5, + .access = PL1_W, .accessfn = access_ttlbis, .type = ARM_CP_NO_RAW, + .fgt = FGT_TLBIVALE1IS, + .writefn = tlbi_aa64_vae1is_write }, + { .name = "TLBI_VAALE1IS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 3, .opc2 = 7, + .access = PL1_W, .accessfn = access_ttlbis, .type = ARM_CP_NO_RAW, + .fgt = FGT_TLBIVAALE1IS, + .writefn = tlbi_aa64_vae1is_write }, + { .name = "TLBI_VMALLE1", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 7, .opc2 = 0, + .access = PL1_W, .accessfn = access_ttlb, .type = ARM_CP_NO_RAW, + .fgt = FGT_TLBIVMALLE1, + .writefn = tlbi_aa64_vmalle1_write }, + { .name = "TLBI_VAE1", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 7, .opc2 = 1, + .access = PL1_W, .accessfn = access_ttlb, .type = ARM_CP_NO_RAW, + .fgt = FGT_TLBIVAE1, + .writefn = tlbi_aa64_vae1_write }, + { .name = "TLBI_ASIDE1", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 7, .opc2 = 2, + .access = PL1_W, .accessfn = access_ttlb, .type = ARM_CP_NO_RAW, + .fgt = FGT_TLBIASIDE1, + .writefn = tlbi_aa64_vmalle1_write }, + { .name = "TLBI_VAAE1", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 7, .opc2 = 3, + .access = PL1_W, .accessfn = access_ttlb, .type = ARM_CP_NO_RAW, + .fgt = FGT_TLBIVAAE1, + .writefn = tlbi_aa64_vae1_write }, + { .name = "TLBI_VALE1", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 7, .opc2 = 5, + .access = PL1_W, .accessfn = access_ttlb, .type = ARM_CP_NO_RAW, + .fgt = FGT_TLBIVALE1, + .writefn = tlbi_aa64_vae1_write }, + { .name = "TLBI_VAALE1", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 7, .opc2 = 7, + .access = PL1_W, .accessfn = access_ttlb, .type = ARM_CP_NO_RAW, + .fgt = FGT_TLBIVAALE1, + .writefn = tlbi_aa64_vae1_write }, + { .name = "TLBI_IPAS2E1IS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 0, .opc2 = 1, + .access = PL2_W, .type = ARM_CP_NO_RAW, + .writefn = tlbi_aa64_ipas2e1is_write }, + { .name = "TLBI_IPAS2LE1IS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 0, .opc2 = 5, + .access = PL2_W, .type = ARM_CP_NO_RAW, + .writefn = tlbi_aa64_ipas2e1is_write }, + { .name = "TLBI_ALLE1IS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 3, .opc2 = 4, + .access = PL2_W, .type = ARM_CP_NO_RAW, + .writefn = tlbi_aa64_alle1is_write }, + { .name = "TLBI_VMALLS12E1IS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 3, .opc2 = 6, + .access = PL2_W, .type = ARM_CP_NO_RAW, + .writefn = tlbi_aa64_alle1is_write }, + { .name = "TLBI_IPAS2E1", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 4, .opc2 = 1, + .access = PL2_W, .type = ARM_CP_NO_RAW, + .writefn = tlbi_aa64_ipas2e1_write }, + { .name = "TLBI_IPAS2LE1", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 4, .opc2 = 5, + .access = PL2_W, .type = ARM_CP_NO_RAW, + .writefn = tlbi_aa64_ipas2e1_write }, + { .name = "TLBI_ALLE1", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 7, .opc2 = 4, + .access = PL2_W, .type = ARM_CP_NO_RAW, + .writefn = tlbi_aa64_alle1_write }, + { .name = "TLBI_VMALLS12E1", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 7, .opc2 = 6, + .access = PL2_W, .type = ARM_CP_NO_RAW, + .writefn = tlbi_aa64_alle1is_write }, }; static const ARMCPRegInfo tlbi_el2_cp_reginfo[] = { -- 2.34.1 Move the AArch64 EL2 TLBI insn definitions that were in el2_cp_reginfo[] across to tlb-insns.c. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241210160452.2427965-5-peter.maydell@linaro.org --- target/arm/cpregs.h | 7 +++++ target/arm/helper.c | 61 ++++---------------------------------- target/arm/tcg/tlb-insns.c | 49 ++++++++++++++++++++++++++++++ 3 files changed, 62 insertions(+), 55 deletions(-) diff --git a/target/arm/cpregs.h b/target/arm/cpregs.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpregs.h +++ b/target/arm/cpregs.h @@ -XXX,XX +XXX,XX @@ bool tlb_force_broadcast(CPUARMState *env); int tlbbits_for_regime(CPUARMState *env, ARMMMUIdx mmu_idx, uint64_t addr); int vae1_tlbbits(CPUARMState *env, uint64_t addr); +int vae2_tlbbits(CPUARMState *env, uint64_t addr); int vae1_tlbmask(CPUARMState *env); +int vae2_tlbmask(CPUARMState *env); int ipas2e1_tlbmask(CPUARMState *env, int64_t value); +int e2_tlbmask(CPUARMState *env); void tlbi_aa64_vmalle1is_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value); void tlbi_aa64_alle1is_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value); void tlbi_aa64_vae1is_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value); +void tlbi_aa64_alle2is_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value); +void tlbi_aa64_vae2is_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value); #endif /* TARGET_ARM_CPREGS_H */ diff --git a/target/arm/helper.c b/target/arm/helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -XXX,XX +XXX,XX @@ int vae1_tlbmask(CPUARMState *env) return mask; } -static int vae2_tlbmask(CPUARMState *env) +int vae2_tlbmask(CPUARMState *env) { uint64_t hcr = arm_hcr_el2_eff(env); uint16_t mask; @@ -XXX,XX +XXX,XX @@ int vae1_tlbbits(CPUARMState *env, uint64_t addr) return tlbbits_for_regime(env, mmu_idx, addr); } -static int vae2_tlbbits(CPUARMState *env, uint64_t addr) +int vae2_tlbbits(CPUARMState *env, uint64_t addr) { uint64_t hcr = arm_hcr_el2_eff(env); ARMMMUIdx mmu_idx; @@ -XXX,XX +XXX,XX @@ void tlbi_aa64_vmalle1is_write(CPUARMState *env, const ARMCPRegInfo *ri, tlb_flush_by_mmuidx_all_cpus_synced(cs, mask); } -static int e2_tlbmask(CPUARMState *env) +int e2_tlbmask(CPUARMState *env) { return (ARMMMUIdxBit_E20_0 | ARMMMUIdxBit_E20_2 | @@ -XXX,XX +XXX,XX @@ static int e2_tlbmask(CPUARMState *env) ARMMMUIdxBit_E2); } -static void tlbi_aa64_alle2_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) -{ - CPUState *cs = env_cpu(env); - int mask = e2_tlbmask(env); - - tlb_flush_by_mmuidx(cs, mask); -} - static void tlbi_aa64_alle3_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value) { @@ -XXX,XX +XXX,XX @@ void tlbi_aa64_alle1is_write(CPUARMState *env, const ARMCPRegInfo *ri, tlb_flush_by_mmuidx_all_cpus_synced(cs, mask); } -static void tlbi_aa64_alle2is_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) +void tlbi_aa64_alle2is_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) { CPUState *cs = env_cpu(env); int mask = e2_tlbmask(env); @@ -XXX,XX +XXX,XX @@ static void tlbi_aa64_alle3is_write(CPUARMState *env, const ARMCPRegInfo *ri, tlb_flush_by_mmuidx_all_cpus_synced(cs, ARMMMUIdxBit_E3); } -static void tlbi_aa64_vae2_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) -{ - /* - * Invalidate by VA, EL2 - * Currently handles both VAE2 and VALE2, since we don't support - * flush-last-level-only. - */ - CPUState *cs = env_cpu(env); - int mask = vae2_tlbmask(env); - uint64_t pageaddr = sextract64(value << 12, 0, 56); - int bits = vae2_tlbbits(env, pageaddr); - - tlb_flush_page_bits_by_mmuidx(cs, pageaddr, mask, bits); -} - static void tlbi_aa64_vae3_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value) { @@ -XXX,XX +XXX,XX @@ void tlbi_aa64_vae1is_write(CPUARMState *env, const ARMCPRegInfo *ri, tlb_flush_page_bits_by_mmuidx_all_cpus_synced(cs, pageaddr, mask, bits); } -static void tlbi_aa64_vae2is_write(CPUARMState *env, const ARMCPRegInfo *ri, +void tlbi_aa64_vae2is_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value) { CPUState *cs = env_cpu(env); @@ -XXX,XX +XXX,XX @@ static const ARMCPRegInfo el2_cp_reginfo[] = { { .name = "HTTBR", .cp = 15, .opc1 = 4, .crm = 2, .access = PL2_RW, .type = ARM_CP_64BIT | ARM_CP_ALIAS, .fieldoffset = offsetof(CPUARMState, cp15.ttbr0_el[2]) }, - { .name = "TLBI_ALLE2", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 7, .opc2 = 0, - .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF, - .writefn = tlbi_aa64_alle2_write }, - { .name = "TLBI_VAE2", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 7, .opc2 = 1, - .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF, - .writefn = tlbi_aa64_vae2_write }, - { .name = "TLBI_VALE2", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 7, .opc2 = 5, - .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF, - .writefn = tlbi_aa64_vae2_write }, - { .name = "TLBI_ALLE2IS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 3, .opc2 = 0, - .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF, - .writefn = tlbi_aa64_alle2is_write }, - { .name = "TLBI_VAE2IS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 3, .opc2 = 1, - .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF, - .writefn = tlbi_aa64_vae2is_write }, - { .name = "TLBI_VALE2IS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 3, .opc2 = 5, - .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF, - .writefn = tlbi_aa64_vae2is_write }, #ifndef CONFIG_USER_ONLY /* * Unlike the other EL2-related AT operations, these must diff --git a/target/arm/tcg/tlb-insns.c b/target/arm/tcg/tlb-insns.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/tlb-insns.c +++ b/target/arm/tcg/tlb-insns.c @@ -XXX,XX +XXX,XX @@ static void tlbi_aa64_alle1_write(CPUARMState *env, const ARMCPRegInfo *ri, tlb_flush_by_mmuidx(cs, mask); } +static void tlbi_aa64_alle2_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) +{ + CPUState *cs = env_cpu(env); + int mask = e2_tlbmask(env); + + tlb_flush_by_mmuidx(cs, mask); +} + +static void tlbi_aa64_vae2_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) +{ + /* + * Invalidate by VA, EL2 + * Currently handles both VAE2 and VALE2, since we don't support + * flush-last-level-only. + */ + CPUState *cs = env_cpu(env); + int mask = vae2_tlbmask(env); + uint64_t pageaddr = sextract64(value << 12, 0, 56); + int bits = vae2_tlbbits(env, pageaddr); + + tlb_flush_page_bits_by_mmuidx(cs, pageaddr, mask, bits); +} + static void tlbi_aa64_vae1_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value) { @@ -XXX,XX +XXX,XX @@ static const ARMCPRegInfo tlbi_el2_cp_reginfo[] = { { .name = "TLBIMVAHIS", .cp = 15, .opc1 = 4, .crn = 8, .crm = 3, .opc2 = 1, .type = ARM_CP_NO_RAW, .access = PL2_W, .writefn = tlbimva_hyp_is_write }, + { .name = "TLBI_ALLE2", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 7, .opc2 = 0, + .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF, + .writefn = tlbi_aa64_alle2_write }, + { .name = "TLBI_VAE2", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 7, .opc2 = 1, + .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF, + .writefn = tlbi_aa64_vae2_write }, + { .name = "TLBI_VALE2", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 7, .opc2 = 5, + .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF, + .writefn = tlbi_aa64_vae2_write }, + { .name = "TLBI_ALLE2IS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 3, .opc2 = 0, + .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF, + .writefn = tlbi_aa64_alle2is_write }, + { .name = "TLBI_VAE2IS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 3, .opc2 = 1, + .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF, + .writefn = tlbi_aa64_vae2is_write }, + { .name = "TLBI_VALE2IS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 3, .opc2 = 5, + .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF, + .writefn = tlbi_aa64_vae2is_write }, }; void define_tlb_insn_regs(ARMCPU *cpu) -- 2.34.1 Move the AArch64 EL3 TLBI insns from el3_cp_reginfo[] across to tlb-insns.c. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241210160452.2427965-6-peter.maydell@linaro.org --- target/arm/cpregs.h | 4 +++ target/arm/helper.c | 56 +++----------------------------------- target/arm/tcg/tlb-insns.c | 54 ++++++++++++++++++++++++++++++++++++ 3 files changed, 62 insertions(+), 52 deletions(-) diff --git a/target/arm/cpregs.h b/target/arm/cpregs.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpregs.h +++ b/target/arm/cpregs.h @@ -XXX,XX +XXX,XX @@ void tlbi_aa64_alle2is_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value); void tlbi_aa64_vae2is_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value); +void tlbi_aa64_vae3is_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value); +void tlbi_aa64_alle3is_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value); #endif /* TARGET_ARM_CPREGS_H */ diff --git a/target/arm/helper.c b/target/arm/helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -XXX,XX +XXX,XX @@ int e2_tlbmask(CPUARMState *env) ARMMMUIdxBit_E2); } -static void tlbi_aa64_alle3_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) -{ - ARMCPU *cpu = env_archcpu(env); - CPUState *cs = CPU(cpu); - - tlb_flush_by_mmuidx(cs, ARMMMUIdxBit_E3); -} - void tlbi_aa64_alle1is_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value) { @@ -XXX,XX +XXX,XX @@ void tlbi_aa64_alle2is_write(CPUARMState *env, const ARMCPRegInfo *ri, tlb_flush_by_mmuidx_all_cpus_synced(cs, mask); } -static void tlbi_aa64_alle3is_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) +void tlbi_aa64_alle3is_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) { CPUState *cs = env_cpu(env); tlb_flush_by_mmuidx_all_cpus_synced(cs, ARMMMUIdxBit_E3); } -static void tlbi_aa64_vae3_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) -{ - /* - * Invalidate by VA, EL3 - * Currently handles both VAE3 and VALE3, since we don't support - * flush-last-level-only. - */ - ARMCPU *cpu = env_archcpu(env); - CPUState *cs = CPU(cpu); - uint64_t pageaddr = sextract64(value << 12, 0, 56); - - tlb_flush_page_by_mmuidx(cs, pageaddr, ARMMMUIdxBit_E3); -} - void tlbi_aa64_vae1is_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value) { @@ -XXX,XX +XXX,XX @@ void tlbi_aa64_vae2is_write(CPUARMState *env, const ARMCPRegInfo *ri, tlb_flush_page_bits_by_mmuidx_all_cpus_synced(cs, pageaddr, mask, bits); } -static void tlbi_aa64_vae3is_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) +void tlbi_aa64_vae3is_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) { CPUState *cs = env_cpu(env); uint64_t pageaddr = sextract64(value << 12, 0, 56); @@ -XXX,XX +XXX,XX @@ static const ARMCPRegInfo el3_cp_reginfo[] = { .opc0 = 3, .opc1 = 6, .crn = 5, .crm = 1, .opc2 = 1, .access = PL3_RW, .type = ARM_CP_CONST, .resetvalue = 0 }, - { .name = "TLBI_ALLE3IS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 3, .opc2 = 0, - .access = PL3_W, .type = ARM_CP_NO_RAW, - .writefn = tlbi_aa64_alle3is_write }, - { .name = "TLBI_VAE3IS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 3, .opc2 = 1, - .access = PL3_W, .type = ARM_CP_NO_RAW, - .writefn = tlbi_aa64_vae3is_write }, - { .name = "TLBI_VALE3IS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 3, .opc2 = 5, - .access = PL3_W, .type = ARM_CP_NO_RAW, - .writefn = tlbi_aa64_vae3is_write }, - { .name = "TLBI_ALLE3", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 7, .opc2 = 0, - .access = PL3_W, .type = ARM_CP_NO_RAW, - .writefn = tlbi_aa64_alle3_write }, - { .name = "TLBI_VAE3", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 7, .opc2 = 1, - .access = PL3_W, .type = ARM_CP_NO_RAW, - .writefn = tlbi_aa64_vae3_write }, - { .name = "TLBI_VALE3", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 7, .opc2 = 5, - .access = PL3_W, .type = ARM_CP_NO_RAW, - .writefn = tlbi_aa64_vae3_write }, }; #ifndef CONFIG_USER_ONLY diff --git a/target/arm/tcg/tlb-insns.c b/target/arm/tcg/tlb-insns.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/tlb-insns.c +++ b/target/arm/tcg/tlb-insns.c @@ -XXX,XX +XXX,XX @@ static void tlbi_aa64_alle2_write(CPUARMState *env, const ARMCPRegInfo *ri, tlb_flush_by_mmuidx(cs, mask); } +static void tlbi_aa64_alle3_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) +{ + ARMCPU *cpu = env_archcpu(env); + CPUState *cs = CPU(cpu); + + tlb_flush_by_mmuidx(cs, ARMMMUIdxBit_E3); +} + static void tlbi_aa64_vae2_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value) { @@ -XXX,XX +XXX,XX @@ static void tlbi_aa64_vae2_write(CPUARMState *env, const ARMCPRegInfo *ri, tlb_flush_page_bits_by_mmuidx(cs, pageaddr, mask, bits); } +static void tlbi_aa64_vae3_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) +{ + /* + * Invalidate by VA, EL3 + * Currently handles both VAE3 and VALE3, since we don't support + * flush-last-level-only. + */ + ARMCPU *cpu = env_archcpu(env); + CPUState *cs = CPU(cpu); + uint64_t pageaddr = sextract64(value << 12, 0, 56); + + tlb_flush_page_by_mmuidx(cs, pageaddr, ARMMMUIdxBit_E3); +} + static void tlbi_aa64_vae1_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value) { @@ -XXX,XX +XXX,XX @@ static const ARMCPRegInfo tlbi_el2_cp_reginfo[] = { .writefn = tlbi_aa64_vae2is_write }, }; +static const ARMCPRegInfo tlbi_el3_cp_reginfo[] = { + { .name = "TLBI_ALLE3IS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 3, .opc2 = 0, + .access = PL3_W, .type = ARM_CP_NO_RAW, + .writefn = tlbi_aa64_alle3is_write }, + { .name = "TLBI_VAE3IS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 3, .opc2 = 1, + .access = PL3_W, .type = ARM_CP_NO_RAW, + .writefn = tlbi_aa64_vae3is_write }, + { .name = "TLBI_VALE3IS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 3, .opc2 = 5, + .access = PL3_W, .type = ARM_CP_NO_RAW, + .writefn = tlbi_aa64_vae3is_write }, + { .name = "TLBI_ALLE3", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 7, .opc2 = 0, + .access = PL3_W, .type = ARM_CP_NO_RAW, + .writefn = tlbi_aa64_alle3_write }, + { .name = "TLBI_VAE3", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 7, .opc2 = 1, + .access = PL3_W, .type = ARM_CP_NO_RAW, + .writefn = tlbi_aa64_vae3_write }, + { .name = "TLBI_VALE3", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 7, .opc2 = 5, + .access = PL3_W, .type = ARM_CP_NO_RAW, + .writefn = tlbi_aa64_vae3_write }, +}; + void define_tlb_insn_regs(ARMCPU *cpu) { CPUARMState *env = &cpu->env; @@ -XXX,XX +XXX,XX @@ void define_tlb_insn_regs(ARMCPU *cpu) && arm_feature(env, ARM_FEATURE_V8))) { define_arm_cp_regs(cpu, tlbi_el2_cp_reginfo); } + if (arm_feature(env, ARM_FEATURE_EL3)) { + define_arm_cp_regs(cpu, tlbi_el3_cp_reginfo); + } } -- 2.34.1 Move the TLBI invalidate-range insns across to tlb-insns.c. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241210160452.2427965-7-peter.maydell@linaro.org --- target/arm/cpregs.h | 2 + target/arm/helper.c | 330 +------------------------------------ target/arm/tcg/tlb-insns.c | 329 ++++++++++++++++++++++++++++++++++++ 3 files changed, 333 insertions(+), 328 deletions(-) diff --git a/target/arm/cpregs.h b/target/arm/cpregs.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpregs.h +++ b/target/arm/cpregs.h @@ -XXX,XX +XXX,XX @@ CPAccessResult access_ttlb(CPUARMState *env, const ARMCPRegInfo *ri, bool isread); CPAccessResult access_ttlbis(CPUARMState *env, const ARMCPRegInfo *ri, bool isread); +CPAccessResult access_ttlbos(CPUARMState *env, const ARMCPRegInfo *ri, + bool isread); bool tlb_force_broadcast(CPUARMState *env); int tlbbits_for_regime(CPUARMState *env, ARMMMUIdx mmu_idx, uint64_t addr); diff --git a/target/arm/helper.c b/target/arm/helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -XXX,XX +XXX,XX @@ CPAccessResult access_ttlbis(CPUARMState *env, const ARMCPRegInfo *ri, #ifdef TARGET_AARCH64 /* Check for traps from EL1 due to HCR_EL2.TTLB or TTLBOS. */ -static CPAccessResult access_ttlbos(CPUARMState *env, const ARMCPRegInfo *ri, - bool isread) +CPAccessResult access_ttlbos(CPUARMState *env, const ARMCPRegInfo *ri, + bool isread) { if (arm_current_el(env) == 1 && (arm_hcr_el2_eff(env) & (HCR_TTLB | HCR_TTLBOS))) { @@ -XXX,XX +XXX,XX @@ int ipas2e1_tlbmask(CPUARMState *env, int64_t value) : ARMMMUIdxBit_Stage2); } -#ifdef TARGET_AARCH64 -typedef struct { - uint64_t base; - uint64_t length; -} TLBIRange; - -static ARMGranuleSize tlbi_range_tg_to_gran_size(int tg) -{ - /* - * Note that the TLBI range TG field encoding differs from both - * TG0 and TG1 encodings. - */ - switch (tg) { - case 1: - return Gran4K; - case 2: - return Gran16K; - case 3: - return Gran64K; - default: - return GranInvalid; - } -} - -static TLBIRange tlbi_aa64_get_range(CPUARMState *env, ARMMMUIdx mmuidx, - uint64_t value) -{ - unsigned int page_size_granule, page_shift, num, scale, exponent; - /* Extract one bit to represent the va selector in use. */ - uint64_t select = sextract64(value, 36, 1); - ARMVAParameters param = aa64_va_parameters(env, select, mmuidx, true, false); - TLBIRange ret = { }; - ARMGranuleSize gran; - - page_size_granule = extract64(value, 46, 2); - gran = tlbi_range_tg_to_gran_size(page_size_granule); - - /* The granule encoded in value must match the granule in use. */ - if (gran != param.gran) { - qemu_log_mask(LOG_GUEST_ERROR, "Invalid tlbi page size granule %d\n", - page_size_granule); - return ret; - } - - page_shift = arm_granule_bits(gran); - num = extract64(value, 39, 5); - scale = extract64(value, 44, 2); - exponent = (5 * scale) + 1; - - ret.length = (num + 1) << (exponent + page_shift); - - if (param.select) { - ret.base = sextract64(value, 0, 37); - } else { - ret.base = extract64(value, 0, 37); - } - if (param.ds) { - /* - * With DS=1, BaseADDR is always shifted 16 so that it is able - * to address all 52 va bits. The input address is perforce - * aligned on a 64k boundary regardless of translation granule. - */ - page_shift = 16; - } - ret.base <<= page_shift; - - return ret; -} - -static void do_rvae_write(CPUARMState *env, uint64_t value, - int idxmap, bool synced) -{ - ARMMMUIdx one_idx = ARM_MMU_IDX_A | ctz32(idxmap); - TLBIRange range; - int bits; - - range = tlbi_aa64_get_range(env, one_idx, value); - bits = tlbbits_for_regime(env, one_idx, range.base); - - if (synced) { - tlb_flush_range_by_mmuidx_all_cpus_synced(env_cpu(env), - range.base, - range.length, - idxmap, - bits); - } else { - tlb_flush_range_by_mmuidx(env_cpu(env), range.base, - range.length, idxmap, bits); - } -} - -static void tlbi_aa64_rvae1_write(CPUARMState *env, - const ARMCPRegInfo *ri, - uint64_t value) -{ - /* - * Invalidate by VA range, EL1&0. - * Currently handles all of RVAE1, RVAAE1, RVAALE1 and RVALE1, - * since we don't support flush-for-specific-ASID-only or - * flush-last-level-only. - */ - - do_rvae_write(env, value, vae1_tlbmask(env), - tlb_force_broadcast(env)); -} - -static void tlbi_aa64_rvae1is_write(CPUARMState *env, - const ARMCPRegInfo *ri, - uint64_t value) -{ - /* - * Invalidate by VA range, Inner/Outer Shareable EL1&0. - * Currently handles all of RVAE1IS, RVAE1OS, RVAAE1IS, RVAAE1OS, - * RVAALE1IS, RVAALE1OS, RVALE1IS and RVALE1OS, since we don't support - * flush-for-specific-ASID-only, flush-last-level-only or inner/outer - * shareable specific flushes. - */ - - do_rvae_write(env, value, vae1_tlbmask(env), true); -} - -static void tlbi_aa64_rvae2_write(CPUARMState *env, - const ARMCPRegInfo *ri, - uint64_t value) -{ - /* - * Invalidate by VA range, EL2. - * Currently handles all of RVAE2 and RVALE2, - * since we don't support flush-for-specific-ASID-only or - * flush-last-level-only. - */ - - do_rvae_write(env, value, vae2_tlbmask(env), - tlb_force_broadcast(env)); - - -} - -static void tlbi_aa64_rvae2is_write(CPUARMState *env, - const ARMCPRegInfo *ri, - uint64_t value) -{ - /* - * Invalidate by VA range, Inner/Outer Shareable, EL2. - * Currently handles all of RVAE2IS, RVAE2OS, RVALE2IS and RVALE2OS, - * since we don't support flush-for-specific-ASID-only, - * flush-last-level-only or inner/outer shareable specific flushes. - */ - - do_rvae_write(env, value, vae2_tlbmask(env), true); - -} - -static void tlbi_aa64_rvae3_write(CPUARMState *env, - const ARMCPRegInfo *ri, - uint64_t value) -{ - /* - * Invalidate by VA range, EL3. - * Currently handles all of RVAE3 and RVALE3, - * since we don't support flush-for-specific-ASID-only or - * flush-last-level-only. - */ - - do_rvae_write(env, value, ARMMMUIdxBit_E3, tlb_force_broadcast(env)); -} - -static void tlbi_aa64_rvae3is_write(CPUARMState *env, - const ARMCPRegInfo *ri, - uint64_t value) -{ - /* - * Invalidate by VA range, EL3, Inner/Outer Shareable. - * Currently handles all of RVAE3IS, RVAE3OS, RVALE3IS and RVALE3OS, - * since we don't support flush-for-specific-ASID-only, - * flush-last-level-only or inner/outer specific flushes. - */ - - do_rvae_write(env, value, ARMMMUIdxBit_E3, true); -} - -static void tlbi_aa64_ripas2e1_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) -{ - do_rvae_write(env, value, ipas2e1_tlbmask(env, value), - tlb_force_broadcast(env)); -} - -static void tlbi_aa64_ripas2e1is_write(CPUARMState *env, - const ARMCPRegInfo *ri, - uint64_t value) -{ - do_rvae_write(env, value, ipas2e1_tlbmask(env, value), true); -} -#endif - static CPAccessResult aa64_zva_access(CPUARMState *env, const ARMCPRegInfo *ri, bool isread) { @@ -XXX,XX +XXX,XX @@ static const ARMCPRegInfo pauth_reginfo[] = { .fieldoffset = offsetof(CPUARMState, keys.apib.hi) }, }; -static const ARMCPRegInfo tlbirange_reginfo[] = { - { .name = "TLBI_RVAE1IS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 2, .opc2 = 1, - .access = PL1_W, .accessfn = access_ttlbis, .type = ARM_CP_NO_RAW, - .fgt = FGT_TLBIRVAE1IS, - .writefn = tlbi_aa64_rvae1is_write }, - { .name = "TLBI_RVAAE1IS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 2, .opc2 = 3, - .access = PL1_W, .accessfn = access_ttlbis, .type = ARM_CP_NO_RAW, - .fgt = FGT_TLBIRVAAE1IS, - .writefn = tlbi_aa64_rvae1is_write }, - { .name = "TLBI_RVALE1IS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 2, .opc2 = 5, - .access = PL1_W, .accessfn = access_ttlbis, .type = ARM_CP_NO_RAW, - .fgt = FGT_TLBIRVALE1IS, - .writefn = tlbi_aa64_rvae1is_write }, - { .name = "TLBI_RVAALE1IS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 2, .opc2 = 7, - .access = PL1_W, .accessfn = access_ttlbis, .type = ARM_CP_NO_RAW, - .fgt = FGT_TLBIRVAALE1IS, - .writefn = tlbi_aa64_rvae1is_write }, - { .name = "TLBI_RVAE1OS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 5, .opc2 = 1, - .access = PL1_W, .accessfn = access_ttlbos, .type = ARM_CP_NO_RAW, - .fgt = FGT_TLBIRVAE1OS, - .writefn = tlbi_aa64_rvae1is_write }, - { .name = "TLBI_RVAAE1OS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 5, .opc2 = 3, - .access = PL1_W, .accessfn = access_ttlbos, .type = ARM_CP_NO_RAW, - .fgt = FGT_TLBIRVAAE1OS, - .writefn = tlbi_aa64_rvae1is_write }, - { .name = "TLBI_RVALE1OS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 5, .opc2 = 5, - .access = PL1_W, .accessfn = access_ttlbos, .type = ARM_CP_NO_RAW, - .fgt = FGT_TLBIRVALE1OS, - .writefn = tlbi_aa64_rvae1is_write }, - { .name = "TLBI_RVAALE1OS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 5, .opc2 = 7, - .access = PL1_W, .accessfn = access_ttlbos, .type = ARM_CP_NO_RAW, - .fgt = FGT_TLBIRVAALE1OS, - .writefn = tlbi_aa64_rvae1is_write }, - { .name = "TLBI_RVAE1", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 6, .opc2 = 1, - .access = PL1_W, .accessfn = access_ttlb, .type = ARM_CP_NO_RAW, - .fgt = FGT_TLBIRVAE1, - .writefn = tlbi_aa64_rvae1_write }, - { .name = "TLBI_RVAAE1", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 6, .opc2 = 3, - .access = PL1_W, .accessfn = access_ttlb, .type = ARM_CP_NO_RAW, - .fgt = FGT_TLBIRVAAE1, - .writefn = tlbi_aa64_rvae1_write }, - { .name = "TLBI_RVALE1", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 6, .opc2 = 5, - .access = PL1_W, .accessfn = access_ttlb, .type = ARM_CP_NO_RAW, - .fgt = FGT_TLBIRVALE1, - .writefn = tlbi_aa64_rvae1_write }, - { .name = "TLBI_RVAALE1", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 6, .opc2 = 7, - .access = PL1_W, .accessfn = access_ttlb, .type = ARM_CP_NO_RAW, - .fgt = FGT_TLBIRVAALE1, - .writefn = tlbi_aa64_rvae1_write }, - { .name = "TLBI_RIPAS2E1IS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 0, .opc2 = 2, - .access = PL2_W, .type = ARM_CP_NO_RAW, - .writefn = tlbi_aa64_ripas2e1is_write }, - { .name = "TLBI_RIPAS2LE1IS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 0, .opc2 = 6, - .access = PL2_W, .type = ARM_CP_NO_RAW, - .writefn = tlbi_aa64_ripas2e1is_write }, - { .name = "TLBI_RVAE2IS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 2, .opc2 = 1, - .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF, - .writefn = tlbi_aa64_rvae2is_write }, - { .name = "TLBI_RVALE2IS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 2, .opc2 = 5, - .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF, - .writefn = tlbi_aa64_rvae2is_write }, - { .name = "TLBI_RIPAS2E1", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 4, .opc2 = 2, - .access = PL2_W, .type = ARM_CP_NO_RAW, - .writefn = tlbi_aa64_ripas2e1_write }, - { .name = "TLBI_RIPAS2LE1", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 4, .opc2 = 6, - .access = PL2_W, .type = ARM_CP_NO_RAW, - .writefn = tlbi_aa64_ripas2e1_write }, - { .name = "TLBI_RVAE2OS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 5, .opc2 = 1, - .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF, - .writefn = tlbi_aa64_rvae2is_write }, - { .name = "TLBI_RVALE2OS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 5, .opc2 = 5, - .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF, - .writefn = tlbi_aa64_rvae2is_write }, - { .name = "TLBI_RVAE2", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 6, .opc2 = 1, - .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF, - .writefn = tlbi_aa64_rvae2_write }, - { .name = "TLBI_RVALE2", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 6, .opc2 = 5, - .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF, - .writefn = tlbi_aa64_rvae2_write }, - { .name = "TLBI_RVAE3IS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 2, .opc2 = 1, - .access = PL3_W, .type = ARM_CP_NO_RAW, - .writefn = tlbi_aa64_rvae3is_write }, - { .name = "TLBI_RVALE3IS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 2, .opc2 = 5, - .access = PL3_W, .type = ARM_CP_NO_RAW, - .writefn = tlbi_aa64_rvae3is_write }, - { .name = "TLBI_RVAE3OS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 5, .opc2 = 1, - .access = PL3_W, .type = ARM_CP_NO_RAW, - .writefn = tlbi_aa64_rvae3is_write }, - { .name = "TLBI_RVALE3OS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 5, .opc2 = 5, - .access = PL3_W, .type = ARM_CP_NO_RAW, - .writefn = tlbi_aa64_rvae3is_write }, - { .name = "TLBI_RVAE3", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 6, .opc2 = 1, - .access = PL3_W, .type = ARM_CP_NO_RAW, - .writefn = tlbi_aa64_rvae3_write }, - { .name = "TLBI_RVALE3", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 6, .opc2 = 5, - .access = PL3_W, .type = ARM_CP_NO_RAW, - .writefn = tlbi_aa64_rvae3_write }, -}; - static const ARMCPRegInfo tlbios_reginfo[] = { { .name = "TLBI_VMALLE1OS", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 1, .opc2 = 0, @@ -XXX,XX +XXX,XX @@ void register_cp_regs_for_features(ARMCPU *cpu) if (cpu_isar_feature(aa64_rndr, cpu)) { define_arm_cp_regs(cpu, rndr_reginfo); } - if (cpu_isar_feature(aa64_tlbirange, cpu)) { - define_arm_cp_regs(cpu, tlbirange_reginfo); - } if (cpu_isar_feature(aa64_tlbios, cpu)) { define_arm_cp_regs(cpu, tlbios_reginfo); } diff --git a/target/arm/tcg/tlb-insns.c b/target/arm/tcg/tlb-insns.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/tlb-insns.c +++ b/target/arm/tcg/tlb-insns.c @@ -XXX,XX +XXX,XX @@ * SPDX-License-Identifier: GPL-2.0-or-later */ #include "qemu/osdep.h" +#include "qemu/log.h" #include "exec/exec-all.h" #include "cpu.h" #include "internals.h" @@ -XXX,XX +XXX,XX @@ static const ARMCPRegInfo tlbi_el3_cp_reginfo[] = { .writefn = tlbi_aa64_vae3_write }, }; +#ifdef TARGET_AARCH64 +typedef struct { + uint64_t base; + uint64_t length; +} TLBIRange; + +static ARMGranuleSize tlbi_range_tg_to_gran_size(int tg) +{ + /* + * Note that the TLBI range TG field encoding differs from both + * TG0 and TG1 encodings. + */ + switch (tg) { + case 1: + return Gran4K; + case 2: + return Gran16K; + case 3: + return Gran64K; + default: + return GranInvalid; + } +} + +static TLBIRange tlbi_aa64_get_range(CPUARMState *env, ARMMMUIdx mmuidx, + uint64_t value) +{ + unsigned int page_size_granule, page_shift, num, scale, exponent; + /* Extract one bit to represent the va selector in use. */ + uint64_t select = sextract64(value, 36, 1); + ARMVAParameters param = aa64_va_parameters(env, select, mmuidx, true, false); + TLBIRange ret = { }; + ARMGranuleSize gran; + + page_size_granule = extract64(value, 46, 2); + gran = tlbi_range_tg_to_gran_size(page_size_granule); + + /* The granule encoded in value must match the granule in use. */ + if (gran != param.gran) { + qemu_log_mask(LOG_GUEST_ERROR, "Invalid tlbi page size granule %d\n", + page_size_granule); + return ret; + } + + page_shift = arm_granule_bits(gran); + num = extract64(value, 39, 5); + scale = extract64(value, 44, 2); + exponent = (5 * scale) + 1; + + ret.length = (num + 1) << (exponent + page_shift); + + if (param.select) { + ret.base = sextract64(value, 0, 37); + } else { + ret.base = extract64(value, 0, 37); + } + if (param.ds) { + /* + * With DS=1, BaseADDR is always shifted 16 so that it is able + * to address all 52 va bits. The input address is perforce + * aligned on a 64k boundary regardless of translation granule. + */ + page_shift = 16; + } + ret.base <<= page_shift; + + return ret; +} + +static void do_rvae_write(CPUARMState *env, uint64_t value, + int idxmap, bool synced) +{ + ARMMMUIdx one_idx = ARM_MMU_IDX_A | ctz32(idxmap); + TLBIRange range; + int bits; + + range = tlbi_aa64_get_range(env, one_idx, value); + bits = tlbbits_for_regime(env, one_idx, range.base); + + if (synced) { + tlb_flush_range_by_mmuidx_all_cpus_synced(env_cpu(env), + range.base, + range.length, + idxmap, + bits); + } else { + tlb_flush_range_by_mmuidx(env_cpu(env), range.base, + range.length, idxmap, bits); + } +} + +static void tlbi_aa64_rvae1_write(CPUARMState *env, + const ARMCPRegInfo *ri, + uint64_t value) +{ + /* + * Invalidate by VA range, EL1&0. + * Currently handles all of RVAE1, RVAAE1, RVAALE1 and RVALE1, + * since we don't support flush-for-specific-ASID-only or + * flush-last-level-only. + */ + + do_rvae_write(env, value, vae1_tlbmask(env), + tlb_force_broadcast(env)); +} + +static void tlbi_aa64_rvae1is_write(CPUARMState *env, + const ARMCPRegInfo *ri, + uint64_t value) +{ + /* + * Invalidate by VA range, Inner/Outer Shareable EL1&0. + * Currently handles all of RVAE1IS, RVAE1OS, RVAAE1IS, RVAAE1OS, + * RVAALE1IS, RVAALE1OS, RVALE1IS and RVALE1OS, since we don't support + * flush-for-specific-ASID-only, flush-last-level-only or inner/outer + * shareable specific flushes. + */ + + do_rvae_write(env, value, vae1_tlbmask(env), true); +} + +static void tlbi_aa64_rvae2_write(CPUARMState *env, + const ARMCPRegInfo *ri, + uint64_t value) +{ + /* + * Invalidate by VA range, EL2. + * Currently handles all of RVAE2 and RVALE2, + * since we don't support flush-for-specific-ASID-only or + * flush-last-level-only. + */ + + do_rvae_write(env, value, vae2_tlbmask(env), + tlb_force_broadcast(env)); + + +} + +static void tlbi_aa64_rvae2is_write(CPUARMState *env, + const ARMCPRegInfo *ri, + uint64_t value) +{ + /* + * Invalidate by VA range, Inner/Outer Shareable, EL2. + * Currently handles all of RVAE2IS, RVAE2OS, RVALE2IS and RVALE2OS, + * since we don't support flush-for-specific-ASID-only, + * flush-last-level-only or inner/outer shareable specific flushes. + */ + + do_rvae_write(env, value, vae2_tlbmask(env), true); + +} + +static void tlbi_aa64_rvae3_write(CPUARMState *env, + const ARMCPRegInfo *ri, + uint64_t value) +{ + /* + * Invalidate by VA range, EL3. + * Currently handles all of RVAE3 and RVALE3, + * since we don't support flush-for-specific-ASID-only or + * flush-last-level-only. + */ + + do_rvae_write(env, value, ARMMMUIdxBit_E3, tlb_force_broadcast(env)); +} + +static void tlbi_aa64_rvae3is_write(CPUARMState *env, + const ARMCPRegInfo *ri, + uint64_t value) +{ + /* + * Invalidate by VA range, EL3, Inner/Outer Shareable. + * Currently handles all of RVAE3IS, RVAE3OS, RVALE3IS and RVALE3OS, + * since we don't support flush-for-specific-ASID-only, + * flush-last-level-only or inner/outer specific flushes. + */ + + do_rvae_write(env, value, ARMMMUIdxBit_E3, true); +} + +static void tlbi_aa64_ripas2e1_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) +{ + do_rvae_write(env, value, ipas2e1_tlbmask(env, value), + tlb_force_broadcast(env)); +} + +static void tlbi_aa64_ripas2e1is_write(CPUARMState *env, + const ARMCPRegInfo *ri, + uint64_t value) +{ + do_rvae_write(env, value, ipas2e1_tlbmask(env, value), true); +} + +static const ARMCPRegInfo tlbirange_reginfo[] = { + { .name = "TLBI_RVAE1IS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 2, .opc2 = 1, + .access = PL1_W, .accessfn = access_ttlbis, .type = ARM_CP_NO_RAW, + .fgt = FGT_TLBIRVAE1IS, + .writefn = tlbi_aa64_rvae1is_write }, + { .name = "TLBI_RVAAE1IS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 2, .opc2 = 3, + .access = PL1_W, .accessfn = access_ttlbis, .type = ARM_CP_NO_RAW, + .fgt = FGT_TLBIRVAAE1IS, + .writefn = tlbi_aa64_rvae1is_write }, + { .name = "TLBI_RVALE1IS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 2, .opc2 = 5, + .access = PL1_W, .accessfn = access_ttlbis, .type = ARM_CP_NO_RAW, + .fgt = FGT_TLBIRVALE1IS, + .writefn = tlbi_aa64_rvae1is_write }, + { .name = "TLBI_RVAALE1IS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 2, .opc2 = 7, + .access = PL1_W, .accessfn = access_ttlbis, .type = ARM_CP_NO_RAW, + .fgt = FGT_TLBIRVAALE1IS, + .writefn = tlbi_aa64_rvae1is_write }, + { .name = "TLBI_RVAE1OS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 5, .opc2 = 1, + .access = PL1_W, .accessfn = access_ttlbos, .type = ARM_CP_NO_RAW, + .fgt = FGT_TLBIRVAE1OS, + .writefn = tlbi_aa64_rvae1is_write }, + { .name = "TLBI_RVAAE1OS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 5, .opc2 = 3, + .access = PL1_W, .accessfn = access_ttlbos, .type = ARM_CP_NO_RAW, + .fgt = FGT_TLBIRVAAE1OS, + .writefn = tlbi_aa64_rvae1is_write }, + { .name = "TLBI_RVALE1OS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 5, .opc2 = 5, + .access = PL1_W, .accessfn = access_ttlbos, .type = ARM_CP_NO_RAW, + .fgt = FGT_TLBIRVALE1OS, + .writefn = tlbi_aa64_rvae1is_write }, + { .name = "TLBI_RVAALE1OS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 5, .opc2 = 7, + .access = PL1_W, .accessfn = access_ttlbos, .type = ARM_CP_NO_RAW, + .fgt = FGT_TLBIRVAALE1OS, + .writefn = tlbi_aa64_rvae1is_write }, + { .name = "TLBI_RVAE1", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 6, .opc2 = 1, + .access = PL1_W, .accessfn = access_ttlb, .type = ARM_CP_NO_RAW, + .fgt = FGT_TLBIRVAE1, + .writefn = tlbi_aa64_rvae1_write }, + { .name = "TLBI_RVAAE1", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 6, .opc2 = 3, + .access = PL1_W, .accessfn = access_ttlb, .type = ARM_CP_NO_RAW, + .fgt = FGT_TLBIRVAAE1, + .writefn = tlbi_aa64_rvae1_write }, + { .name = "TLBI_RVALE1", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 6, .opc2 = 5, + .access = PL1_W, .accessfn = access_ttlb, .type = ARM_CP_NO_RAW, + .fgt = FGT_TLBIRVALE1, + .writefn = tlbi_aa64_rvae1_write }, + { .name = "TLBI_RVAALE1", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 6, .opc2 = 7, + .access = PL1_W, .accessfn = access_ttlb, .type = ARM_CP_NO_RAW, + .fgt = FGT_TLBIRVAALE1, + .writefn = tlbi_aa64_rvae1_write }, + { .name = "TLBI_RIPAS2E1IS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 0, .opc2 = 2, + .access = PL2_W, .type = ARM_CP_NO_RAW, + .writefn = tlbi_aa64_ripas2e1is_write }, + { .name = "TLBI_RIPAS2LE1IS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 0, .opc2 = 6, + .access = PL2_W, .type = ARM_CP_NO_RAW, + .writefn = tlbi_aa64_ripas2e1is_write }, + { .name = "TLBI_RVAE2IS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 2, .opc2 = 1, + .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF, + .writefn = tlbi_aa64_rvae2is_write }, + { .name = "TLBI_RVALE2IS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 2, .opc2 = 5, + .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF, + .writefn = tlbi_aa64_rvae2is_write }, + { .name = "TLBI_RIPAS2E1", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 4, .opc2 = 2, + .access = PL2_W, .type = ARM_CP_NO_RAW, + .writefn = tlbi_aa64_ripas2e1_write }, + { .name = "TLBI_RIPAS2LE1", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 4, .opc2 = 6, + .access = PL2_W, .type = ARM_CP_NO_RAW, + .writefn = tlbi_aa64_ripas2e1_write }, + { .name = "TLBI_RVAE2OS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 5, .opc2 = 1, + .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF, + .writefn = tlbi_aa64_rvae2is_write }, + { .name = "TLBI_RVALE2OS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 5, .opc2 = 5, + .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF, + .writefn = tlbi_aa64_rvae2is_write }, + { .name = "TLBI_RVAE2", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 6, .opc2 = 1, + .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF, + .writefn = tlbi_aa64_rvae2_write }, + { .name = "TLBI_RVALE2", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 6, .opc2 = 5, + .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF, + .writefn = tlbi_aa64_rvae2_write }, + { .name = "TLBI_RVAE3IS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 2, .opc2 = 1, + .access = PL3_W, .type = ARM_CP_NO_RAW, + .writefn = tlbi_aa64_rvae3is_write }, + { .name = "TLBI_RVALE3IS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 2, .opc2 = 5, + .access = PL3_W, .type = ARM_CP_NO_RAW, + .writefn = tlbi_aa64_rvae3is_write }, + { .name = "TLBI_RVAE3OS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 5, .opc2 = 1, + .access = PL3_W, .type = ARM_CP_NO_RAW, + .writefn = tlbi_aa64_rvae3is_write }, + { .name = "TLBI_RVALE3OS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 5, .opc2 = 5, + .access = PL3_W, .type = ARM_CP_NO_RAW, + .writefn = tlbi_aa64_rvae3is_write }, + { .name = "TLBI_RVAE3", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 6, .opc2 = 1, + .access = PL3_W, .type = ARM_CP_NO_RAW, + .writefn = tlbi_aa64_rvae3_write }, + { .name = "TLBI_RVALE3", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 6, .opc2 = 5, + .access = PL3_W, .type = ARM_CP_NO_RAW, + .writefn = tlbi_aa64_rvae3_write }, +}; +#endif + void define_tlb_insn_regs(ARMCPU *cpu) { CPUARMState *env = &cpu->env; @@ -XXX,XX +XXX,XX @@ void define_tlb_insn_regs(ARMCPU *cpu) if (arm_feature(env, ARM_FEATURE_EL3)) { define_arm_cp_regs(cpu, tlbi_el3_cp_reginfo); } +#ifdef TARGET_AARCH64 + if (cpu_isar_feature(aa64_tlbirange, cpu)) { + define_arm_cp_regs(cpu, tlbirange_reginfo); + } +#endif } -- 2.34.1 Move the TLBI OS insns across to tlb-insns.c. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241210160452.2427965-8-peter.maydell@linaro.org --- target/arm/helper.c | 80 -------------------------------------- target/arm/tcg/tlb-insns.c | 80 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 80 insertions(+), 80 deletions(-) diff --git a/target/arm/helper.c b/target/arm/helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -XXX,XX +XXX,XX @@ static const ARMCPRegInfo pauth_reginfo[] = { .fieldoffset = offsetof(CPUARMState, keys.apib.hi) }, }; -static const ARMCPRegInfo tlbios_reginfo[] = { - { .name = "TLBI_VMALLE1OS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 1, .opc2 = 0, - .access = PL1_W, .accessfn = access_ttlbos, .type = ARM_CP_NO_RAW, - .fgt = FGT_TLBIVMALLE1OS, - .writefn = tlbi_aa64_vmalle1is_write }, - { .name = "TLBI_VAE1OS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 1, .opc2 = 1, - .fgt = FGT_TLBIVAE1OS, - .access = PL1_W, .accessfn = access_ttlbos, .type = ARM_CP_NO_RAW, - .writefn = tlbi_aa64_vae1is_write }, - { .name = "TLBI_ASIDE1OS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 1, .opc2 = 2, - .access = PL1_W, .accessfn = access_ttlbos, .type = ARM_CP_NO_RAW, - .fgt = FGT_TLBIASIDE1OS, - .writefn = tlbi_aa64_vmalle1is_write }, - { .name = "TLBI_VAAE1OS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 1, .opc2 = 3, - .access = PL1_W, .accessfn = access_ttlbos, .type = ARM_CP_NO_RAW, - .fgt = FGT_TLBIVAAE1OS, - .writefn = tlbi_aa64_vae1is_write }, - { .name = "TLBI_VALE1OS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 1, .opc2 = 5, - .access = PL1_W, .accessfn = access_ttlbos, .type = ARM_CP_NO_RAW, - .fgt = FGT_TLBIVALE1OS, - .writefn = tlbi_aa64_vae1is_write }, - { .name = "TLBI_VAALE1OS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 1, .opc2 = 7, - .access = PL1_W, .accessfn = access_ttlbos, .type = ARM_CP_NO_RAW, - .fgt = FGT_TLBIVAALE1OS, - .writefn = tlbi_aa64_vae1is_write }, - { .name = "TLBI_ALLE2OS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 1, .opc2 = 0, - .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF, - .writefn = tlbi_aa64_alle2is_write }, - { .name = "TLBI_VAE2OS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 1, .opc2 = 1, - .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF, - .writefn = tlbi_aa64_vae2is_write }, - { .name = "TLBI_ALLE1OS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 1, .opc2 = 4, - .access = PL2_W, .type = ARM_CP_NO_RAW, - .writefn = tlbi_aa64_alle1is_write }, - { .name = "TLBI_VALE2OS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 1, .opc2 = 5, - .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF, - .writefn = tlbi_aa64_vae2is_write }, - { .name = "TLBI_VMALLS12E1OS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 1, .opc2 = 6, - .access = PL2_W, .type = ARM_CP_NO_RAW, - .writefn = tlbi_aa64_alle1is_write }, - { .name = "TLBI_IPAS2E1OS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 4, .opc2 = 0, - .access = PL2_W, .type = ARM_CP_NOP }, - { .name = "TLBI_RIPAS2E1OS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 4, .opc2 = 3, - .access = PL2_W, .type = ARM_CP_NOP }, - { .name = "TLBI_IPAS2LE1OS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 4, .opc2 = 4, - .access = PL2_W, .type = ARM_CP_NOP }, - { .name = "TLBI_RIPAS2LE1OS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 4, .opc2 = 7, - .access = PL2_W, .type = ARM_CP_NOP }, - { .name = "TLBI_ALLE3OS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 1, .opc2 = 0, - .access = PL3_W, .type = ARM_CP_NO_RAW, - .writefn = tlbi_aa64_alle3is_write }, - { .name = "TLBI_VAE3OS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 1, .opc2 = 1, - .access = PL3_W, .type = ARM_CP_NO_RAW, - .writefn = tlbi_aa64_vae3is_write }, - { .name = "TLBI_VALE3OS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 1, .opc2 = 5, - .access = PL3_W, .type = ARM_CP_NO_RAW, - .writefn = tlbi_aa64_vae3is_write }, -}; - static uint64_t rndr_readfn(CPUARMState *env, const ARMCPRegInfo *ri) { Error *err = NULL; @@ -XXX,XX +XXX,XX @@ void register_cp_regs_for_features(ARMCPU *cpu) if (cpu_isar_feature(aa64_rndr, cpu)) { define_arm_cp_regs(cpu, rndr_reginfo); } - if (cpu_isar_feature(aa64_tlbios, cpu)) { - define_arm_cp_regs(cpu, tlbios_reginfo); - } /* Data Cache clean instructions up to PoP */ if (cpu_isar_feature(aa64_dcpop, cpu)) { define_one_arm_cp_reg(cpu, dcpop_reg); diff --git a/target/arm/tcg/tlb-insns.c b/target/arm/tcg/tlb-insns.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/tlb-insns.c +++ b/target/arm/tcg/tlb-insns.c @@ -XXX,XX +XXX,XX @@ static const ARMCPRegInfo tlbirange_reginfo[] = { .access = PL3_W, .type = ARM_CP_NO_RAW, .writefn = tlbi_aa64_rvae3_write }, }; + +static const ARMCPRegInfo tlbios_reginfo[] = { + { .name = "TLBI_VMALLE1OS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 1, .opc2 = 0, + .access = PL1_W, .accessfn = access_ttlbos, .type = ARM_CP_NO_RAW, + .fgt = FGT_TLBIVMALLE1OS, + .writefn = tlbi_aa64_vmalle1is_write }, + { .name = "TLBI_VAE1OS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 1, .opc2 = 1, + .fgt = FGT_TLBIVAE1OS, + .access = PL1_W, .accessfn = access_ttlbos, .type = ARM_CP_NO_RAW, + .writefn = tlbi_aa64_vae1is_write }, + { .name = "TLBI_ASIDE1OS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 1, .opc2 = 2, + .access = PL1_W, .accessfn = access_ttlbos, .type = ARM_CP_NO_RAW, + .fgt = FGT_TLBIASIDE1OS, + .writefn = tlbi_aa64_vmalle1is_write }, + { .name = "TLBI_VAAE1OS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 1, .opc2 = 3, + .access = PL1_W, .accessfn = access_ttlbos, .type = ARM_CP_NO_RAW, + .fgt = FGT_TLBIVAAE1OS, + .writefn = tlbi_aa64_vae1is_write }, + { .name = "TLBI_VALE1OS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 1, .opc2 = 5, + .access = PL1_W, .accessfn = access_ttlbos, .type = ARM_CP_NO_RAW, + .fgt = FGT_TLBIVALE1OS, + .writefn = tlbi_aa64_vae1is_write }, + { .name = "TLBI_VAALE1OS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 1, .opc2 = 7, + .access = PL1_W, .accessfn = access_ttlbos, .type = ARM_CP_NO_RAW, + .fgt = FGT_TLBIVAALE1OS, + .writefn = tlbi_aa64_vae1is_write }, + { .name = "TLBI_ALLE2OS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 1, .opc2 = 0, + .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF, + .writefn = tlbi_aa64_alle2is_write }, + { .name = "TLBI_VAE2OS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 1, .opc2 = 1, + .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF, + .writefn = tlbi_aa64_vae2is_write }, + { .name = "TLBI_ALLE1OS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 1, .opc2 = 4, + .access = PL2_W, .type = ARM_CP_NO_RAW, + .writefn = tlbi_aa64_alle1is_write }, + { .name = "TLBI_VALE2OS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 1, .opc2 = 5, + .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF, + .writefn = tlbi_aa64_vae2is_write }, + { .name = "TLBI_VMALLS12E1OS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 1, .opc2 = 6, + .access = PL2_W, .type = ARM_CP_NO_RAW, + .writefn = tlbi_aa64_alle1is_write }, + { .name = "TLBI_IPAS2E1OS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 4, .opc2 = 0, + .access = PL2_W, .type = ARM_CP_NOP }, + { .name = "TLBI_RIPAS2E1OS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 4, .opc2 = 3, + .access = PL2_W, .type = ARM_CP_NOP }, + { .name = "TLBI_IPAS2LE1OS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 4, .opc2 = 4, + .access = PL2_W, .type = ARM_CP_NOP }, + { .name = "TLBI_RIPAS2LE1OS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 4, .opc2 = 7, + .access = PL2_W, .type = ARM_CP_NOP }, + { .name = "TLBI_ALLE3OS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 1, .opc2 = 0, + .access = PL3_W, .type = ARM_CP_NO_RAW, + .writefn = tlbi_aa64_alle3is_write }, + { .name = "TLBI_VAE3OS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 1, .opc2 = 1, + .access = PL3_W, .type = ARM_CP_NO_RAW, + .writefn = tlbi_aa64_vae3is_write }, + { .name = "TLBI_VALE3OS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 1, .opc2 = 5, + .access = PL3_W, .type = ARM_CP_NO_RAW, + .writefn = tlbi_aa64_vae3is_write }, +}; #endif void define_tlb_insn_regs(ARMCPU *cpu) @@ -XXX,XX +XXX,XX @@ void define_tlb_insn_regs(ARMCPU *cpu) if (cpu_isar_feature(aa64_tlbirange, cpu)) { define_arm_cp_regs(cpu, tlbirange_reginfo); } + if (cpu_isar_feature(aa64_tlbios, cpu)) { + define_arm_cp_regs(cpu, tlbios_reginfo); + } #endif } -- 2.34.1 The remaining functions that we temporarily made global are now used only from callsits in tlb-insns.c; move them across and make them file-local again. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241210160452.2427965-9-peter.maydell@linaro.org --- target/arm/cpregs.h | 34 ------ target/arm/helper.c | 220 ------------------------------------- target/arm/tcg/tlb-insns.c | 220 +++++++++++++++++++++++++++++++++++++ 3 files changed, 220 insertions(+), 254 deletions(-) diff --git a/target/arm/cpregs.h b/target/arm/cpregs.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpregs.h +++ b/target/arm/cpregs.h @@ -XXX,XX +XXX,XX @@ static inline bool arm_cpreg_traps_in_nv(const ARMCPRegInfo *ri) return ri->opc1 == 4 || ri->opc1 == 5; } -/* - * Temporary declarations of functions until the move to tlb_insn_helper.c - * is complete and we can make the functions static again - */ -CPAccessResult access_ttlb(CPUARMState *env, const ARMCPRegInfo *ri, - bool isread); -CPAccessResult access_ttlbis(CPUARMState *env, const ARMCPRegInfo *ri, - bool isread); -CPAccessResult access_ttlbos(CPUARMState *env, const ARMCPRegInfo *ri, - bool isread); -bool tlb_force_broadcast(CPUARMState *env); -int tlbbits_for_regime(CPUARMState *env, ARMMMUIdx mmu_idx, - uint64_t addr); -int vae1_tlbbits(CPUARMState *env, uint64_t addr); -int vae2_tlbbits(CPUARMState *env, uint64_t addr); -int vae1_tlbmask(CPUARMState *env); -int vae2_tlbmask(CPUARMState *env); -int ipas2e1_tlbmask(CPUARMState *env, int64_t value); -int e2_tlbmask(CPUARMState *env); -void tlbi_aa64_vmalle1is_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value); -void tlbi_aa64_alle1is_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value); -void tlbi_aa64_vae1is_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value); -void tlbi_aa64_alle2is_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value); -void tlbi_aa64_vae2is_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value); -void tlbi_aa64_vae3is_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value); -void tlbi_aa64_alle3is_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value); - #endif /* TARGET_ARM_CPREGS_H */ diff --git a/target/arm/helper.c b/target/arm/helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -XXX,XX +XXX,XX @@ static CPAccessResult access_tacr(CPUARMState *env, const ARMCPRegInfo *ri, return CP_ACCESS_OK; } -/* Check for traps from EL1 due to HCR_EL2.TTLB. */ -CPAccessResult access_ttlb(CPUARMState *env, const ARMCPRegInfo *ri, - bool isread) -{ - if (arm_current_el(env) == 1 && (arm_hcr_el2_eff(env) & HCR_TTLB)) { - return CP_ACCESS_TRAP_EL2; - } - return CP_ACCESS_OK; -} - -/* Check for traps from EL1 due to HCR_EL2.TTLB or TTLBIS. */ -CPAccessResult access_ttlbis(CPUARMState *env, const ARMCPRegInfo *ri, - bool isread) -{ - if (arm_current_el(env) == 1 && - (arm_hcr_el2_eff(env) & (HCR_TTLB | HCR_TTLBIS))) { - return CP_ACCESS_TRAP_EL2; - } - return CP_ACCESS_OK; -} - -#ifdef TARGET_AARCH64 -/* Check for traps from EL1 due to HCR_EL2.TTLB or TTLBOS. */ -CPAccessResult access_ttlbos(CPUARMState *env, const ARMCPRegInfo *ri, - bool isread) -{ - if (arm_current_el(env) == 1 && - (arm_hcr_el2_eff(env) & (HCR_TTLB | HCR_TTLBOS))) { - return CP_ACCESS_TRAP_EL2; - } - return CP_ACCESS_OK; -} -#endif - static void dacr_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value) { ARMCPU *cpu = env_archcpu(env); @@ -XXX,XX +XXX,XX @@ int alle1_tlbmask(CPUARMState *env) ARMMMUIdxBit_Stage2_S); } -/* - * Non-IS variants of TLB operations are upgraded to - * IS versions if we are at EL1 and HCR_EL2.FB is effectively set to - * force broadcast of these operations. - */ -bool tlb_force_broadcast(CPUARMState *env) -{ - return arm_current_el(env) == 1 && (arm_hcr_el2_eff(env) & HCR_FB); -} - static const ARMCPRegInfo cp_reginfo[] = { /* * Define the secure and non-secure FCSE identifier CP registers @@ -XXX,XX +XXX,XX @@ static CPAccessResult access_tocu(CPUARMState *env, const ARMCPRegInfo *ri, return do_cacheop_pou_access(env, HCR_TOCU | HCR_TPU); } -/* - * See: D4.7.2 TLB maintenance requirements and the TLB maintenance instructions - * Page D4-1736 (DDI0487A.b) - */ - -int vae1_tlbmask(CPUARMState *env) -{ - uint64_t hcr = arm_hcr_el2_eff(env); - uint16_t mask; - - assert(arm_feature(env, ARM_FEATURE_AARCH64)); - - if ((hcr & (HCR_E2H | HCR_TGE)) == (HCR_E2H | HCR_TGE)) { - mask = ARMMMUIdxBit_E20_2 | - ARMMMUIdxBit_E20_2_PAN | - ARMMMUIdxBit_E20_0; - } else { - /* This is AArch64 only, so we don't need to touch the EL30_x TLBs */ - mask = ARMMMUIdxBit_E10_1 | - ARMMMUIdxBit_E10_1_PAN | - ARMMMUIdxBit_E10_0; - } - return mask; -} - -int vae2_tlbmask(CPUARMState *env) -{ - uint64_t hcr = arm_hcr_el2_eff(env); - uint16_t mask; - - if (hcr & HCR_E2H) { - mask = ARMMMUIdxBit_E20_2 | - ARMMMUIdxBit_E20_2_PAN | - ARMMMUIdxBit_E20_0; - } else { - mask = ARMMMUIdxBit_E2; - } - return mask; -} - -/* Return 56 if TBI is enabled, 64 otherwise. */ -int tlbbits_for_regime(CPUARMState *env, ARMMMUIdx mmu_idx, - uint64_t addr) -{ - uint64_t tcr = regime_tcr(env, mmu_idx); - int tbi = aa64_va_parameter_tbi(tcr, mmu_idx); - int select = extract64(addr, 55, 1); - - return (tbi >> select) & 1 ? 56 : 64; -} - -int vae1_tlbbits(CPUARMState *env, uint64_t addr) -{ - uint64_t hcr = arm_hcr_el2_eff(env); - ARMMMUIdx mmu_idx; - - assert(arm_feature(env, ARM_FEATURE_AARCH64)); - - /* Only the regime of the mmu_idx below is significant. */ - if ((hcr & (HCR_E2H | HCR_TGE)) == (HCR_E2H | HCR_TGE)) { - mmu_idx = ARMMMUIdx_E20_0; - } else { - mmu_idx = ARMMMUIdx_E10_0; - } - - return tlbbits_for_regime(env, mmu_idx, addr); -} - -int vae2_tlbbits(CPUARMState *env, uint64_t addr) -{ - uint64_t hcr = arm_hcr_el2_eff(env); - ARMMMUIdx mmu_idx; - - /* - * Only the regime of the mmu_idx below is significant. - * Regime EL2&0 has two ranges with separate TBI configuration, while EL2 - * only has one. - */ - if (hcr & HCR_E2H) { - mmu_idx = ARMMMUIdx_E20_2; - } else { - mmu_idx = ARMMMUIdx_E2; - } - - return tlbbits_for_regime(env, mmu_idx, addr); -} - -void tlbi_aa64_vmalle1is_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) -{ - CPUState *cs = env_cpu(env); - int mask = vae1_tlbmask(env); - - tlb_flush_by_mmuidx_all_cpus_synced(cs, mask); -} - -int e2_tlbmask(CPUARMState *env) -{ - return (ARMMMUIdxBit_E20_0 | - ARMMMUIdxBit_E20_2 | - ARMMMUIdxBit_E20_2_PAN | - ARMMMUIdxBit_E2); -} - -void tlbi_aa64_alle1is_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) -{ - CPUState *cs = env_cpu(env); - int mask = alle1_tlbmask(env); - - tlb_flush_by_mmuidx_all_cpus_synced(cs, mask); -} - -void tlbi_aa64_alle2is_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) -{ - CPUState *cs = env_cpu(env); - int mask = e2_tlbmask(env); - - tlb_flush_by_mmuidx_all_cpus_synced(cs, mask); -} - -void tlbi_aa64_alle3is_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) -{ - CPUState *cs = env_cpu(env); - - tlb_flush_by_mmuidx_all_cpus_synced(cs, ARMMMUIdxBit_E3); -} - -void tlbi_aa64_vae1is_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) -{ - CPUState *cs = env_cpu(env); - int mask = vae1_tlbmask(env); - uint64_t pageaddr = sextract64(value << 12, 0, 56); - int bits = vae1_tlbbits(env, pageaddr); - - tlb_flush_page_bits_by_mmuidx_all_cpus_synced(cs, pageaddr, mask, bits); -} - -void tlbi_aa64_vae2is_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) -{ - CPUState *cs = env_cpu(env); - int mask = vae2_tlbmask(env); - uint64_t pageaddr = sextract64(value << 12, 0, 56); - int bits = vae2_tlbbits(env, pageaddr); - - tlb_flush_page_bits_by_mmuidx_all_cpus_synced(cs, pageaddr, mask, bits); -} - -void tlbi_aa64_vae3is_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) -{ - CPUState *cs = env_cpu(env); - uint64_t pageaddr = sextract64(value << 12, 0, 56); - int bits = tlbbits_for_regime(env, ARMMMUIdx_E3, pageaddr); - - tlb_flush_page_bits_by_mmuidx_all_cpus_synced(cs, pageaddr, - ARMMMUIdxBit_E3, bits); -} - -int ipas2e1_tlbmask(CPUARMState *env, int64_t value) -{ - /* - * The MSB of value is the NS field, which only applies if SEL2 - * is implemented and SCR_EL3.NS is not set (i.e. in secure mode). - */ - return (value >= 0 - && cpu_isar_feature(aa64_sel2, env_archcpu(env)) - && arm_is_secure_below_el3(env) - ? ARMMMUIdxBit_Stage2_S - : ARMMMUIdxBit_Stage2); -} - static CPAccessResult aa64_zva_access(CPUARMState *env, const ARMCPRegInfo *ri, bool isread) { diff --git a/target/arm/tcg/tlb-insns.c b/target/arm/tcg/tlb-insns.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/tlb-insns.c +++ b/target/arm/tcg/tlb-insns.c @@ -XXX,XX +XXX,XX @@ #include "cpu-features.h" #include "cpregs.h" +/* Check for traps from EL1 due to HCR_EL2.TTLB. */ +static CPAccessResult access_ttlb(CPUARMState *env, const ARMCPRegInfo *ri, + bool isread) +{ + if (arm_current_el(env) == 1 && (arm_hcr_el2_eff(env) & HCR_TTLB)) { + return CP_ACCESS_TRAP_EL2; + } + return CP_ACCESS_OK; +} + +/* Check for traps from EL1 due to HCR_EL2.TTLB or TTLBIS. */ +static CPAccessResult access_ttlbis(CPUARMState *env, const ARMCPRegInfo *ri, + bool isread) +{ + if (arm_current_el(env) == 1 && + (arm_hcr_el2_eff(env) & (HCR_TTLB | HCR_TTLBIS))) { + return CP_ACCESS_TRAP_EL2; + } + return CP_ACCESS_OK; +} + +#ifdef TARGET_AARCH64 +/* Check for traps from EL1 due to HCR_EL2.TTLB or TTLBOS. */ +static CPAccessResult access_ttlbos(CPUARMState *env, const ARMCPRegInfo *ri, + bool isread) +{ + if (arm_current_el(env) == 1 && + (arm_hcr_el2_eff(env) & (HCR_TTLB | HCR_TTLBOS))) { + return CP_ACCESS_TRAP_EL2; + } + return CP_ACCESS_OK; +} +#endif + /* IS variants of TLB operations must affect all cores */ static void tlbiall_is_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value) @@ -XXX,XX +XXX,XX @@ static void tlbimvaa_is_write(CPUARMState *env, const ARMCPRegInfo *ri, tlb_flush_page_all_cpus_synced(cs, value & TARGET_PAGE_MASK); } +/* + * Non-IS variants of TLB operations are upgraded to + * IS versions if we are at EL1 and HCR_EL2.FB is effectively set to + * force broadcast of these operations. + */ +static bool tlb_force_broadcast(CPUARMState *env) +{ + return arm_current_el(env) == 1 && (arm_hcr_el2_eff(env) & HCR_FB); +} + static void tlbiall_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value) { @@ -XXX,XX +XXX,XX @@ static void tlbiall_hyp_is_write(CPUARMState *env, const ARMCPRegInfo *ri, tlb_flush_by_mmuidx_all_cpus_synced(cs, ARMMMUIdxBit_E2); } +/* + * See: D4.7.2 TLB maintenance requirements and the TLB maintenance instructions + * Page D4-1736 (DDI0487A.b) + */ + +static int vae1_tlbmask(CPUARMState *env) +{ + uint64_t hcr = arm_hcr_el2_eff(env); + uint16_t mask; + + assert(arm_feature(env, ARM_FEATURE_AARCH64)); + + if ((hcr & (HCR_E2H | HCR_TGE)) == (HCR_E2H | HCR_TGE)) { + mask = ARMMMUIdxBit_E20_2 | + ARMMMUIdxBit_E20_2_PAN | + ARMMMUIdxBit_E20_0; + } else { + /* This is AArch64 only, so we don't need to touch the EL30_x TLBs */ + mask = ARMMMUIdxBit_E10_1 | + ARMMMUIdxBit_E10_1_PAN | + ARMMMUIdxBit_E10_0; + } + return mask; +} + +static int vae2_tlbmask(CPUARMState *env) +{ + uint64_t hcr = arm_hcr_el2_eff(env); + uint16_t mask; + + if (hcr & HCR_E2H) { + mask = ARMMMUIdxBit_E20_2 | + ARMMMUIdxBit_E20_2_PAN | + ARMMMUIdxBit_E20_0; + } else { + mask = ARMMMUIdxBit_E2; + } + return mask; +} + +/* Return 56 if TBI is enabled, 64 otherwise. */ +static int tlbbits_for_regime(CPUARMState *env, ARMMMUIdx mmu_idx, + uint64_t addr) +{ + uint64_t tcr = regime_tcr(env, mmu_idx); + int tbi = aa64_va_parameter_tbi(tcr, mmu_idx); + int select = extract64(addr, 55, 1); + + return (tbi >> select) & 1 ? 56 : 64; +} + +static int vae1_tlbbits(CPUARMState *env, uint64_t addr) +{ + uint64_t hcr = arm_hcr_el2_eff(env); + ARMMMUIdx mmu_idx; + + assert(arm_feature(env, ARM_FEATURE_AARCH64)); + + /* Only the regime of the mmu_idx below is significant. */ + if ((hcr & (HCR_E2H | HCR_TGE)) == (HCR_E2H | HCR_TGE)) { + mmu_idx = ARMMMUIdx_E20_0; + } else { + mmu_idx = ARMMMUIdx_E10_0; + } + + return tlbbits_for_regime(env, mmu_idx, addr); +} + +static int vae2_tlbbits(CPUARMState *env, uint64_t addr) +{ + uint64_t hcr = arm_hcr_el2_eff(env); + ARMMMUIdx mmu_idx; + + /* + * Only the regime of the mmu_idx below is significant. + * Regime EL2&0 has two ranges with separate TBI configuration, while EL2 + * only has one. + */ + if (hcr & HCR_E2H) { + mmu_idx = ARMMMUIdx_E20_2; + } else { + mmu_idx = ARMMMUIdx_E2; + } + + return tlbbits_for_regime(env, mmu_idx, addr); +} + +static void tlbi_aa64_vmalle1is_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) +{ + CPUState *cs = env_cpu(env); + int mask = vae1_tlbmask(env); + + tlb_flush_by_mmuidx_all_cpus_synced(cs, mask); +} + static void tlbi_aa64_vmalle1_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value) { @@ -XXX,XX +XXX,XX @@ static void tlbi_aa64_vmalle1_write(CPUARMState *env, const ARMCPRegInfo *ri, } } +static int e2_tlbmask(CPUARMState *env) +{ + return (ARMMMUIdxBit_E20_0 | + ARMMMUIdxBit_E20_2 | + ARMMMUIdxBit_E20_2_PAN | + ARMMMUIdxBit_E2); +} + static void tlbi_aa64_alle1_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value) { @@ -XXX,XX +XXX,XX @@ static void tlbi_aa64_alle3_write(CPUARMState *env, const ARMCPRegInfo *ri, tlb_flush_by_mmuidx(cs, ARMMMUIdxBit_E3); } +static void tlbi_aa64_alle1is_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) +{ + CPUState *cs = env_cpu(env); + int mask = alle1_tlbmask(env); + + tlb_flush_by_mmuidx_all_cpus_synced(cs, mask); +} + +static void tlbi_aa64_alle2is_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) +{ + CPUState *cs = env_cpu(env); + int mask = e2_tlbmask(env); + + tlb_flush_by_mmuidx_all_cpus_synced(cs, mask); +} + +static void tlbi_aa64_alle3is_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) +{ + CPUState *cs = env_cpu(env); + + tlb_flush_by_mmuidx_all_cpus_synced(cs, ARMMMUIdxBit_E3); +} + static void tlbi_aa64_vae2_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value) { @@ -XXX,XX +XXX,XX @@ static void tlbi_aa64_vae3_write(CPUARMState *env, const ARMCPRegInfo *ri, tlb_flush_page_by_mmuidx(cs, pageaddr, ARMMMUIdxBit_E3); } +static void tlbi_aa64_vae1is_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) +{ + CPUState *cs = env_cpu(env); + int mask = vae1_tlbmask(env); + uint64_t pageaddr = sextract64(value << 12, 0, 56); + int bits = vae1_tlbbits(env, pageaddr); + + tlb_flush_page_bits_by_mmuidx_all_cpus_synced(cs, pageaddr, mask, bits); +} + static void tlbi_aa64_vae1_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value) { @@ -XXX,XX +XXX,XX @@ static void tlbi_aa64_vae1_write(CPUARMState *env, const ARMCPRegInfo *ri, } } +static void tlbi_aa64_vae2is_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) +{ + CPUState *cs = env_cpu(env); + int mask = vae2_tlbmask(env); + uint64_t pageaddr = sextract64(value << 12, 0, 56); + int bits = vae2_tlbbits(env, pageaddr); + + tlb_flush_page_bits_by_mmuidx_all_cpus_synced(cs, pageaddr, mask, bits); +} + +static void tlbi_aa64_vae3is_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) +{ + CPUState *cs = env_cpu(env); + uint64_t pageaddr = sextract64(value << 12, 0, 56); + int bits = tlbbits_for_regime(env, ARMMMUIdx_E3, pageaddr); + + tlb_flush_page_bits_by_mmuidx_all_cpus_synced(cs, pageaddr, + ARMMMUIdxBit_E3, bits); +} + +static int ipas2e1_tlbmask(CPUARMState *env, int64_t value) +{ + /* + * The MSB of value is the NS field, which only applies if SEL2 + * is implemented and SCR_EL3.NS is not set (i.e. in secure mode). + */ + return (value >= 0 + && cpu_isar_feature(aa64_sel2, env_archcpu(env)) + && arm_is_secure_below_el3(env) + ? ARMMMUIdxBit_Stage2_S + : ARMMMUIdxBit_Stage2); +} + static void tlbi_aa64_ipas2e1_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value) { -- 2.34.1 Move the FEAT_RME specific TLB insns across to tlb-insns.c. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241210160452.2427965-10-peter.maydell@linaro.org --- target/arm/helper.c | 38 -------------------------------- target/arm/tcg/tlb-insns.c | 45 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 45 insertions(+), 38 deletions(-) diff --git a/target/arm/helper.c b/target/arm/helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -XXX,XX +XXX,XX @@ static const ARMCPRegInfo sme_reginfo[] = { .type = ARM_CP_CONST, .resetvalue = 0 }, }; -static void tlbi_aa64_paall_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) -{ - CPUState *cs = env_cpu(env); - - tlb_flush(cs); -} - static void gpccr_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value) { @@ -XXX,XX +XXX,XX @@ static void gpccr_reset(CPUARMState *env, const ARMCPRegInfo *ri) env_archcpu(env)->reset_l0gptsz); } -static void tlbi_aa64_paallos_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) -{ - CPUState *cs = env_cpu(env); - - tlb_flush_all_cpus_synced(cs); -} - static const ARMCPRegInfo rme_reginfo[] = { { .name = "GPCCR_EL3", .state = ARM_CP_STATE_AA64, .opc0 = 3, .opc1 = 6, .crn = 2, .crm = 1, .opc2 = 6, @@ -XXX,XX +XXX,XX @@ static const ARMCPRegInfo rme_reginfo[] = { { .name = "MFAR_EL3", .state = ARM_CP_STATE_AA64, .opc0 = 3, .opc1 = 6, .crn = 6, .crm = 0, .opc2 = 5, .access = PL3_RW, .fieldoffset = offsetof(CPUARMState, cp15.mfar_el3) }, - { .name = "TLBI_PAALL", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 7, .opc2 = 4, - .access = PL3_W, .type = ARM_CP_NO_RAW, - .writefn = tlbi_aa64_paall_write }, - { .name = "TLBI_PAALLOS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 1, .opc2 = 4, - .access = PL3_W, .type = ARM_CP_NO_RAW, - .writefn = tlbi_aa64_paallos_write }, - /* - * QEMU does not have a way to invalidate by physical address, thus - * invalidating a range of physical addresses is accomplished by - * flushing all tlb entries in the outer shareable domain, - * just like PAALLOS. - */ - { .name = "TLBI_RPALOS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 4, .opc2 = 7, - .access = PL3_W, .type = ARM_CP_NO_RAW, - .writefn = tlbi_aa64_paallos_write }, - { .name = "TLBI_RPAOS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 4, .opc2 = 3, - .access = PL3_W, .type = ARM_CP_NO_RAW, - .writefn = tlbi_aa64_paallos_write }, { .name = "DC_CIPAPA", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 6, .crn = 7, .crm = 14, .opc2 = 1, .access = PL3_W, .type = ARM_CP_NOP }, diff --git a/target/arm/tcg/tlb-insns.c b/target/arm/tcg/tlb-insns.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/tlb-insns.c +++ b/target/arm/tcg/tlb-insns.c @@ -XXX,XX +XXX,XX @@ static const ARMCPRegInfo tlbios_reginfo[] = { .access = PL3_W, .type = ARM_CP_NO_RAW, .writefn = tlbi_aa64_vae3is_write }, }; + +static void tlbi_aa64_paall_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) +{ + CPUState *cs = env_cpu(env); + + tlb_flush(cs); +} + +static void tlbi_aa64_paallos_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) +{ + CPUState *cs = env_cpu(env); + + tlb_flush_all_cpus_synced(cs); +} + +static const ARMCPRegInfo tlbi_rme_reginfo[] = { + { .name = "TLBI_PAALL", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 7, .opc2 = 4, + .access = PL3_W, .type = ARM_CP_NO_RAW, + .writefn = tlbi_aa64_paall_write }, + { .name = "TLBI_PAALLOS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 1, .opc2 = 4, + .access = PL3_W, .type = ARM_CP_NO_RAW, + .writefn = tlbi_aa64_paallos_write }, + /* + * QEMU does not have a way to invalidate by physical address, thus + * invalidating a range of physical addresses is accomplished by + * flushing all tlb entries in the outer shareable domain, + * just like PAALLOS. + */ + { .name = "TLBI_RPALOS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 4, .opc2 = 7, + .access = PL3_W, .type = ARM_CP_NO_RAW, + .writefn = tlbi_aa64_paallos_write }, + { .name = "TLBI_RPAOS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 4, .opc2 = 3, + .access = PL3_W, .type = ARM_CP_NO_RAW, + .writefn = tlbi_aa64_paallos_write }, +}; + #endif void define_tlb_insn_regs(ARMCPU *cpu) @@ -XXX,XX +XXX,XX @@ void define_tlb_insn_regs(ARMCPU *cpu) if (cpu_isar_feature(aa64_tlbios, cpu)) { define_arm_cp_regs(cpu, tlbios_reginfo); } + if (cpu_isar_feature(aa64_rme, cpu)) { + define_arm_cp_regs(cpu, tlbi_rme_reginfo); + } #endif } -- 2.34.1 We currently register the tlbi_el2_cp_reginfo[] TLBI insns if EL2 is implemented, or if EL3 and v8 is implemented. This is a copy of the logic used for el2_cp_reginfo[], but for the specific case of the TLBI insns we can simplify it. This is because we do not need the "if EL2 does not exist but EL3 does then EL2 registers should exist and be RAZ/WI" handling here: all our cpregs are for instructions, which UNDEF when EL3 exists and EL2 does not. Simplify the condition down to just "if EL2 exists". This is not a behaviour change because: * for AArch64 insns we marked them with ARM_CP_EL3_NO_EL2_UNDEF, which meant that define_arm_cp_regs() would ignore them if EL2 wasn't present * for AArch32 insns, the .access = PL2_W meant that if EL2 was not present the only way to get at them was from AArch32 EL3; but we have no CPUs which have ARM_FEATURE_V8 but start in AArch32 Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241210160452.2427965-11-peter.maydell@linaro.org --- target/arm/tcg/tlb-insns.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/target/arm/tcg/tlb-insns.c b/target/arm/tcg/tlb-insns.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/tlb-insns.c +++ b/target/arm/tcg/tlb-insns.c @@ -XXX,XX +XXX,XX @@ void define_tlb_insn_regs(ARMCPU *cpu) * ops (i.e. matching the condition for el2_cp_reginfo[] in * helper.c), but we will be able to simplify this later. */ - if (arm_feature(env, ARM_FEATURE_EL2) - || (arm_feature(env, ARM_FEATURE_EL3) - && arm_feature(env, ARM_FEATURE_V8))) { + if (arm_feature(env, ARM_FEATURE_EL2)) { define_arm_cp_regs(cpu, tlbi_el2_cp_reginfo); } if (arm_feature(env, ARM_FEATURE_EL3)) { -- 2.34.1