Just my fp16 work, plus some small stuff for the sbsa-ref board; but my rule of thumb is to send a pullreq once I get over about 30 patches... -- PMM The following changes since commit 2f4c51c0f384d7888a04b4815861e6d5fd244d75: Merge remote-tracking branch 'remotes/kraxel/tags/usb-20200831-pull-request' into staging (2020-08-31 19:39:13 +0100) are available in the Git repository at: https://git.linaro.org/people/pmaydell/qemu-arm.git tags/pull-target-arm-20200901 for you to fetch changes up to 3f462bf0f6ea6382dd1502d4eb1fcd33c8e774f5: hw/arm/sbsa-ref : Add embedded controller in secure memory (2020-09-01 14:01:34 +0100) ---------------------------------------------------------------- target-arm queue: * Implement fp16 support for AArch32 VFP and Neon * hw/arm/sbsa-ref: add "reg" property to DT cpu nodes * hw/arm/sbsa-ref : Add embedded controller in secure memory ---------------------------------------------------------------- Graeme Gregory (2): hw/misc/sbsa_ec : Add an embedded controller for sbsa-ref hw/arm/sbsa-ref : Add embedded controller in secure memory Leif Lindholm (1): hw/arm/sbsa-ref: add "reg" property to DT cpu nodes Peter Maydell (44): target/arm: Remove local definitions of float constants target/arm: Use correct ID register check for aa32_fp16_arith target/arm: Implement VFP fp16 for VFP_BINOP operations target/arm: Implement VFP fp16 VMLA, VMLS, VNMLS, VNMLA, VNMUL target/arm: Macroify trans functions for VFMA, VFMS, VFNMA, VFNMS target/arm: Implement VFP fp16 for fused-multiply-add target/arm: Macroify uses of do_vfp_2op_sp() and do_vfp_2op_dp() target/arm: Implement VFP fp16 for VABS, VNEG, VSQRT target/arm: Implement VFP fp16 for VMOV immediate target/arm: Implement VFP fp16 VCMP target/arm: Implement VFP fp16 VLDR and VSTR target/arm: Implement VFP fp16 VCVT between float and integer target/arm: Make VFP_CONV_FIX macros take separate float type and float size target/arm: Use macros instead of open-coding fp16 conversion helpers target/arm: Implement VFP fp16 VCVT between float and fixed-point target/arm: Implement VFP vp16 VCVT-with-specified-rounding-mode target/arm: Implement VFP fp16 VSEL target/arm: Implement VFP fp16 VRINT* target/arm: Implement new VFP fp16 insn VINS target/arm: Implement new VFP fp16 insn VMOVX target/arm: Implement VFP fp16 VMOV between gp and halfprec registers target/arm: Implement FP16 for Neon VADD, VSUB, VABD, VMUL target/arm: Implement fp16 for Neon VRECPE, VRSQRTE using gvec target/arm: Implement fp16 for Neon VABS, VNEG of floats target/arm: Implement fp16 for VCEQ, VCGE, VCGT comparisons target/arm: Implement fp16 for VACGE, VACGT target/arm: Implement fp16 for Neon VMAX, VMIN target/arm: Implement fp16 for Neon VMAXNM, VMINNM target/arm: Implement fp16 for Neon VMLA, VMLS operations target/arm: Implement fp16 for Neon VFMA, VMFS target/arm: Implement fp16 for Neon fp compare-vs-0 target/arm: Implement fp16 for Neon VRECPS target/arm: Implement fp16 for Neon VRSQRTS target/arm: Implement fp16 for Neon pairwise fp ops target/arm: Implement fp16 for Neon float-integer VCVT target/arm: Convert Neon VCVT fixed-point to gvec target/arm: Implement fp16 for Neon VCVT fixed-point target/arm: Implement fp16 for Neon VCVT with rounding modes target/arm: Implement fp16 for Neon VRINT-with-specified-rounding-mode target/arm: Implement fp16 for Neon VRINTX target/arm/vec_helper: Handle oprsz less than 16 bytes in indexed operations target/arm/vec_helper: Add gvec fp indexed multiply-and-add operations target/arm: Implement fp16 for Neon VMUL, VMLA, VMLS target/arm: Enable FP16 in '-cpu max' target/arm/cpu.h | 7 +- target/arm/helper.h | 133 ++++++- target/arm/neon-dp.decode | 8 +- target/arm/vfp-uncond.decode | 27 +- target/arm/vfp.decode | 34 +- hw/arm/sbsa-ref.c | 43 ++- hw/misc/sbsa_ec.c | 98 +++++ target/arm/cpu.c | 3 +- target/arm/cpu64.c | 10 +- target/arm/helper-a64.c | 11 - target/arm/translate-sve.c | 4 - target/arm/vec_helper.c | 431 ++++++++++++++++++++- target/arm/vfp_helper.c | 244 +++++------- hw/misc/meson.build | 2 + target/arm/translate-neon.c.inc | 755 +++++++++++++------------------------ target/arm/translate-vfp.c.inc | 810 ++++++++++++++++++++++++++++++++++++---- 16 files changed, 1819 insertions(+), 801 deletions(-) create mode 100644 hw/misc/sbsa_ec.c In several places the target/arm code defines local float constants for 2, 3 and 1.5, which are also provided by include/fpu/softfloat.h. Remove the unnecessary local duplicate versions. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-2-peter.maydell@linaro.org --- target/arm/helper-a64.c | 11 ----------- target/arm/translate-sve.c | 4 ---- target/arm/vfp_helper.c | 4 ---- 3 files changed, 19 deletions(-) diff --git a/target/arm/helper-a64.c b/target/arm/helper-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper-a64.c +++ b/target/arm/helper-a64.c @@ -XXX,XX +XXX,XX @@ uint64_t HELPER(neon_cgt_f64)(float64 a, float64 b, void *fpstp) * versions, these do a fully fused multiply-add or * multiply-add-and-halve. */ -#define float16_two make_float16(0x4000) -#define float16_three make_float16(0x4200) -#define float16_one_point_five make_float16(0x3e00) - -#define float32_two make_float32(0x40000000) -#define float32_three make_float32(0x40400000) -#define float32_one_point_five make_float32(0x3fc00000) - -#define float64_two make_float64(0x4000000000000000ULL) -#define float64_three make_float64(0x4008000000000000ULL) -#define float64_one_point_five make_float64(0x3FF8000000000000ULL) uint32_t HELPER(recpsf_f16)(uint32_t a, uint32_t b, void *fpstp) { diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool trans_##NAME##_zpzi(DisasContext *s, arg_rpri_esz *a) \ return true; \ } -#define float16_two make_float16(0x4000) -#define float32_two make_float32(0x40000000) -#define float64_two make_float64(0x4000000000000000ULL) - DO_FP_IMM(FADD, fadds, half, one) DO_FP_IMM(FSUB, fsubs, half, one) DO_FP_IMM(FMUL, fmuls, half, two) diff --git a/target/arm/vfp_helper.c b/target/arm/vfp_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vfp_helper.c +++ b/target/arm/vfp_helper.c @@ -XXX,XX +XXX,XX @@ uint32_t HELPER(vfp_fcvt_f64_to_f16)(float64 a, void *fpstp, uint32_t ahp_mode) return r; } -#define float32_two make_float32(0x40000000) -#define float32_three make_float32(0x40400000) -#define float32_one_point_five make_float32(0x3fc00000) - float32 HELPER(recps_f32)(CPUARMState *env, float32 a, float32 b) { float_status *s = &env->vfp.standard_fp_status; -- 2.20.1 The aa32_fp16_arith feature check function currently looks at the AArch64 ID_AA64PFR0 register. This is (as the comment notes) not correct. The bogus check was put in mostly to allow testing of the fp16 variants of the VCMLA instructions and it was something of a mistake that we allowed them to exist in master. Switch the feature check function to testing VMFR1.FPHP, which is what it ought to be. This will remove emulation of the VCMLA and VCADD insns from AArch32 code running on an AArch64 '-cpu max' using system emulation. (They were never enabled for aarch32 linux-user and system-emulation.) Since we weren't advertising their existence via the AArch32 ID register, well-behaved guests wouldn't have been using them anyway. Once we have implemented all the AArch32 support for the FP16 extension we will advertise it in the MVFR1 ID register field, which will reenable these insns along with all the others. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-3-peter.maydell@linaro.org --- target/arm/cpu.h | 7 +------ 1 file changed, 1 insertion(+), 6 deletions(-) diff --git a/target/arm/cpu.h b/target/arm/cpu.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -XXX,XX +XXX,XX @@ static inline bool isar_feature_aa32_predinv(const ARMISARegisters *id) static inline bool isar_feature_aa32_fp16_arith(const ARMISARegisters *id) { - /* - * This is a placeholder for use by VCMA until the rest of - * the ARMv8.2-FP16 extension is implemented for aa32 mode. - * At which point we can properly set and check MVFR1.FPHP. - */ - return FIELD_EX64(id->id_aa64pfr0, ID_AA64PFR0, FP) == 1; + return FIELD_EX32(id->mvfr1, MVFR1, FPHP) >= 3; } static inline bool isar_feature_aa32_vfp_simd(const ARMISARegisters *id) -- 2.20.1 Implmeent VFP fp16 support for simple binary-operator VFP insns VADD, VSUB, VMUL, VDIV, VMINNM and VMAXNM: * make the VFP_BINOP() macro generate float16 helpers as well as float32 and float64 * implement a do_vfp_3op_hp() function similar to the existing do_vfp_3op_sp() * add decode for the half-precision insn patterns Note that the VFP_BINOP macro use creates a couple of unused helper functions vfp_maxh and vfp_minh, but they're small so it's not worth splitting the BINOP operations into "needs halfprec" and "no halfprec" groups. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-4-peter.maydell@linaro.org --- target/arm/helper.h | 8 ++++ target/arm/vfp-uncond.decode | 3 ++ target/arm/vfp.decode | 4 ++ target/arm/vfp_helper.c | 5 ++ target/arm/translate-vfp.c.inc | 86 ++++++++++++++++++++++++++++++++++ 5 files changed, 106 insertions(+) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_5(probe_access, TCG_CALL_NO_WG, void, env, tl, i32, i32, i32) DEF_HELPER_1(vfp_get_fpscr, i32, env) DEF_HELPER_2(vfp_set_fpscr, void, env, i32) +DEF_HELPER_3(vfp_addh, f16, f16, f16, ptr) DEF_HELPER_3(vfp_adds, f32, f32, f32, ptr) DEF_HELPER_3(vfp_addd, f64, f64, f64, ptr) +DEF_HELPER_3(vfp_subh, f16, f16, f16, ptr) DEF_HELPER_3(vfp_subs, f32, f32, f32, ptr) DEF_HELPER_3(vfp_subd, f64, f64, f64, ptr) +DEF_HELPER_3(vfp_mulh, f16, f16, f16, ptr) DEF_HELPER_3(vfp_muls, f32, f32, f32, ptr) DEF_HELPER_3(vfp_muld, f64, f64, f64, ptr) +DEF_HELPER_3(vfp_divh, f16, f16, f16, ptr) DEF_HELPER_3(vfp_divs, f32, f32, f32, ptr) DEF_HELPER_3(vfp_divd, f64, f64, f64, ptr) +DEF_HELPER_3(vfp_maxh, f16, f16, f16, ptr) DEF_HELPER_3(vfp_maxs, f32, f32, f32, ptr) DEF_HELPER_3(vfp_maxd, f64, f64, f64, ptr) +DEF_HELPER_3(vfp_minh, f16, f16, f16, ptr) DEF_HELPER_3(vfp_mins, f32, f32, f32, ptr) DEF_HELPER_3(vfp_mind, f64, f64, f64, ptr) +DEF_HELPER_3(vfp_maxnumh, f16, f16, f16, ptr) DEF_HELPER_3(vfp_maxnums, f32, f32, f32, ptr) DEF_HELPER_3(vfp_maxnumd, f64, f64, f64, ptr) +DEF_HELPER_3(vfp_minnumh, f16, f16, f16, ptr) DEF_HELPER_3(vfp_minnums, f32, f32, f32, ptr) DEF_HELPER_3(vfp_minnumd, f64, f64, f64, ptr) DEF_HELPER_1(vfp_negs, f32, f32) diff --git a/target/arm/vfp-uncond.decode b/target/arm/vfp-uncond.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vfp-uncond.decode +++ b/target/arm/vfp-uncond.decode @@ -XXX,XX +XXX,XX @@ VSEL 1111 1110 0. cc:2 .... .... 1010 .0.0 .... \ VSEL 1111 1110 0. cc:2 .... .... 1011 .0.0 .... \ vm=%vm_dp vn=%vn_dp vd=%vd_dp dp=1 +VMAXNM_hp 1111 1110 1.00 .... .... 1001 .0.0 .... @vfp_dnm_s +VMINNM_hp 1111 1110 1.00 .... .... 1001 .1.0 .... @vfp_dnm_s + VMAXNM_sp 1111 1110 1.00 .... .... 1010 .0.0 .... @vfp_dnm_s VMINNM_sp 1111 1110 1.00 .... .... 1010 .1.0 .... @vfp_dnm_s diff --git a/target/arm/vfp.decode b/target/arm/vfp.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vfp.decode +++ b/target/arm/vfp.decode @@ -XXX,XX +XXX,XX @@ VNMLS_dp ---- 1110 0.01 .... .... 1011 .0.0 .... @vfp_dnm_d VNMLA_sp ---- 1110 0.01 .... .... 1010 .1.0 .... @vfp_dnm_s VNMLA_dp ---- 1110 0.01 .... .... 1011 .1.0 .... @vfp_dnm_d +VMUL_hp ---- 1110 0.10 .... .... 1001 .0.0 .... @vfp_dnm_s VMUL_sp ---- 1110 0.10 .... .... 1010 .0.0 .... @vfp_dnm_s VMUL_dp ---- 1110 0.10 .... .... 1011 .0.0 .... @vfp_dnm_d VNMUL_sp ---- 1110 0.10 .... .... 1010 .1.0 .... @vfp_dnm_s VNMUL_dp ---- 1110 0.10 .... .... 1011 .1.0 .... @vfp_dnm_d +VADD_hp ---- 1110 0.11 .... .... 1001 .0.0 .... @vfp_dnm_s VADD_sp ---- 1110 0.11 .... .... 1010 .0.0 .... @vfp_dnm_s VADD_dp ---- 1110 0.11 .... .... 1011 .0.0 .... @vfp_dnm_d +VSUB_hp ---- 1110 0.11 .... .... 1001 .1.0 .... @vfp_dnm_s VSUB_sp ---- 1110 0.11 .... .... 1010 .1.0 .... @vfp_dnm_s VSUB_dp ---- 1110 0.11 .... .... 1011 .1.0 .... @vfp_dnm_d +VDIV_hp ---- 1110 1.00 .... .... 1001 .0.0 .... @vfp_dnm_s VDIV_sp ---- 1110 1.00 .... .... 1010 .0.0 .... @vfp_dnm_s VDIV_dp ---- 1110 1.00 .... .... 1011 .0.0 .... @vfp_dnm_d diff --git a/target/arm/vfp_helper.c b/target/arm/vfp_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vfp_helper.c +++ b/target/arm/vfp_helper.c @@ -XXX,XX +XXX,XX @@ void vfp_set_fpscr(CPUARMState *env, uint32_t val) #define VFP_HELPER(name, p) HELPER(glue(glue(vfp_,name),p)) #define VFP_BINOP(name) \ +dh_ctype_f16 VFP_HELPER(name, h)(dh_ctype_f16 a, dh_ctype_f16 b, void *fpstp) \ +{ \ + float_status *fpst = fpstp; \ + return float16_ ## name(a, b, fpst); \ +} \ float32 VFP_HELPER(name, s)(float32 a, float32 b, void *fpstp) \ { \ float_status *fpst = fpstp; \ diff --git a/target/arm/translate-vfp.c.inc b/target/arm/translate-vfp.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-vfp.c.inc +++ b/target/arm/translate-vfp.c.inc @@ -XXX,XX +XXX,XX @@ static bool do_vfp_3op_sp(DisasContext *s, VFPGen3OpSPFn *fn, return true; } +static bool do_vfp_3op_hp(DisasContext *s, VFPGen3OpSPFn *fn, + int vd, int vn, int vm, bool reads_vd) +{ + /* + * Do a half-precision operation. Functionally this is + * the same as do_vfp_3op_sp(), except: + * - it uses the FPST_FPCR_F16 + * - it doesn't need the VFP vector handling (fp16 is a + * v8 feature, and in v8 VFP vectors don't exist) + * - it does the aa32_fp16_arith feature test + */ + TCGv_i32 f0, f1, fd; + TCGv_ptr fpst; + + if (!dc_isar_feature(aa32_fp16_arith, s)) { + return false; + } + + if (s->vec_len != 0 || s->vec_stride != 0) { + return false; + } + + if (!vfp_access_check(s)) { + return true; + } + + f0 = tcg_temp_new_i32(); + f1 = tcg_temp_new_i32(); + fd = tcg_temp_new_i32(); + fpst = fpstatus_ptr(FPST_FPCR_F16); + + neon_load_reg32(f0, vn); + neon_load_reg32(f1, vm); + + if (reads_vd) { + neon_load_reg32(fd, vd); + } + fn(fd, f0, f1, fpst); + neon_store_reg32(fd, vd); + + tcg_temp_free_i32(f0); + tcg_temp_free_i32(f1); + tcg_temp_free_i32(fd); + tcg_temp_free_ptr(fpst); + + return true; +} + static bool do_vfp_3op_dp(DisasContext *s, VFPGen3OpDPFn *fn, int vd, int vn, int vm, bool reads_vd) { @@ -XXX,XX +XXX,XX @@ static bool trans_VNMLA_dp(DisasContext *s, arg_VNMLA_dp *a) return do_vfp_3op_dp(s, gen_VNMLA_dp, a->vd, a->vn, a->vm, true); } +static bool trans_VMUL_hp(DisasContext *s, arg_VMUL_sp *a) +{ + return do_vfp_3op_hp(s, gen_helper_vfp_mulh, a->vd, a->vn, a->vm, false); +} + static bool trans_VMUL_sp(DisasContext *s, arg_VMUL_sp *a) { return do_vfp_3op_sp(s, gen_helper_vfp_muls, a->vd, a->vn, a->vm, false); @@ -XXX,XX +XXX,XX @@ static bool trans_VNMUL_dp(DisasContext *s, arg_VNMUL_dp *a) return do_vfp_3op_dp(s, gen_VNMUL_dp, a->vd, a->vn, a->vm, false); } +static bool trans_VADD_hp(DisasContext *s, arg_VADD_sp *a) +{ + return do_vfp_3op_hp(s, gen_helper_vfp_addh, a->vd, a->vn, a->vm, false); +} + static bool trans_VADD_sp(DisasContext *s, arg_VADD_sp *a) { return do_vfp_3op_sp(s, gen_helper_vfp_adds, a->vd, a->vn, a->vm, false); @@ -XXX,XX +XXX,XX @@ static bool trans_VADD_dp(DisasContext *s, arg_VADD_dp *a) return do_vfp_3op_dp(s, gen_helper_vfp_addd, a->vd, a->vn, a->vm, false); } +static bool trans_VSUB_hp(DisasContext *s, arg_VSUB_sp *a) +{ + return do_vfp_3op_hp(s, gen_helper_vfp_subh, a->vd, a->vn, a->vm, false); +} + static bool trans_VSUB_sp(DisasContext *s, arg_VSUB_sp *a) { return do_vfp_3op_sp(s, gen_helper_vfp_subs, a->vd, a->vn, a->vm, false); @@ -XXX,XX +XXX,XX @@ static bool trans_VSUB_dp(DisasContext *s, arg_VSUB_dp *a) return do_vfp_3op_dp(s, gen_helper_vfp_subd, a->vd, a->vn, a->vm, false); } +static bool trans_VDIV_hp(DisasContext *s, arg_VDIV_sp *a) +{ + return do_vfp_3op_hp(s, gen_helper_vfp_divh, a->vd, a->vn, a->vm, false); +} + static bool trans_VDIV_sp(DisasContext *s, arg_VDIV_sp *a) { return do_vfp_3op_sp(s, gen_helper_vfp_divs, a->vd, a->vn, a->vm, false); @@ -XXX,XX +XXX,XX @@ static bool trans_VDIV_dp(DisasContext *s, arg_VDIV_dp *a) return do_vfp_3op_dp(s, gen_helper_vfp_divd, a->vd, a->vn, a->vm, false); } +static bool trans_VMINNM_hp(DisasContext *s, arg_VMINNM_sp *a) +{ + if (!dc_isar_feature(aa32_vminmaxnm, s)) { + return false; + } + return do_vfp_3op_hp(s, gen_helper_vfp_minnumh, + a->vd, a->vn, a->vm, false); +} + +static bool trans_VMAXNM_hp(DisasContext *s, arg_VMAXNM_sp *a) +{ + if (!dc_isar_feature(aa32_vminmaxnm, s)) { + return false; + } + return do_vfp_3op_hp(s, gen_helper_vfp_maxnumh, + a->vd, a->vn, a->vm, false); +} + static bool trans_VMINNM_sp(DisasContext *s, arg_VMINNM_sp *a) { if (!dc_isar_feature(aa32_vminmaxnm, s)) { -- 2.20.1 Implement fp16 versions of the VFP VMLA, VMLS, VNMLS, VNMLA, VNMUL instructions. (These are all the remaining ones which we implement via do_vfp_3op_[hsd]p().) Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-5-peter.maydell@linaro.org --- target/arm/helper.h | 1 + target/arm/vfp.decode | 5 ++ target/arm/vfp_helper.c | 5 ++ target/arm/translate-vfp.c.inc | 84 ++++++++++++++++++++++++++++++++++ 4 files changed, 95 insertions(+) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_3(vfp_maxnumd, f64, f64, f64, ptr) DEF_HELPER_3(vfp_minnumh, f16, f16, f16, ptr) DEF_HELPER_3(vfp_minnums, f32, f32, f32, ptr) DEF_HELPER_3(vfp_minnumd, f64, f64, f64, ptr) +DEF_HELPER_1(vfp_negh, f16, f16) DEF_HELPER_1(vfp_negs, f32, f32) DEF_HELPER_1(vfp_negd, f64, f64) DEF_HELPER_1(vfp_abss, f32, f32) diff --git a/target/arm/vfp.decode b/target/arm/vfp.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vfp.decode +++ b/target/arm/vfp.decode @@ -XXX,XX +XXX,XX @@ VLDM_VSTM_dp ---- 1101 0.1 l:1 rn:4 .... 1011 imm:8 \ vd=%vd_dp p=1 u=0 w=1 # 3-register VFP data-processing; bits [23,21:20,6] identify the operation. +VMLA_hp ---- 1110 0.00 .... .... 1001 .0.0 .... @vfp_dnm_s VMLA_sp ---- 1110 0.00 .... .... 1010 .0.0 .... @vfp_dnm_s VMLA_dp ---- 1110 0.00 .... .... 1011 .0.0 .... @vfp_dnm_d +VMLS_hp ---- 1110 0.00 .... .... 1001 .1.0 .... @vfp_dnm_s VMLS_sp ---- 1110 0.00 .... .... 1010 .1.0 .... @vfp_dnm_s VMLS_dp ---- 1110 0.00 .... .... 1011 .1.0 .... @vfp_dnm_d +VNMLS_hp ---- 1110 0.01 .... .... 1001 .0.0 .... @vfp_dnm_s VNMLS_sp ---- 1110 0.01 .... .... 1010 .0.0 .... @vfp_dnm_s VNMLS_dp ---- 1110 0.01 .... .... 1011 .0.0 .... @vfp_dnm_d +VNMLA_hp ---- 1110 0.01 .... .... 1001 .1.0 .... @vfp_dnm_s VNMLA_sp ---- 1110 0.01 .... .... 1010 .1.0 .... @vfp_dnm_s VNMLA_dp ---- 1110 0.01 .... .... 1011 .1.0 .... @vfp_dnm_d @@ -XXX,XX +XXX,XX @@ VMUL_hp ---- 1110 0.10 .... .... 1001 .0.0 .... @vfp_dnm_s VMUL_sp ---- 1110 0.10 .... .... 1010 .0.0 .... @vfp_dnm_s VMUL_dp ---- 1110 0.10 .... .... 1011 .0.0 .... @vfp_dnm_d +VNMUL_hp ---- 1110 0.10 .... .... 1001 .1.0 .... @vfp_dnm_s VNMUL_sp ---- 1110 0.10 .... .... 1010 .1.0 .... @vfp_dnm_s VNMUL_dp ---- 1110 0.10 .... .... 1011 .1.0 .... @vfp_dnm_d diff --git a/target/arm/vfp_helper.c b/target/arm/vfp_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vfp_helper.c +++ b/target/arm/vfp_helper.c @@ -XXX,XX +XXX,XX @@ VFP_BINOP(minnum) VFP_BINOP(maxnum) #undef VFP_BINOP +dh_ctype_f16 VFP_HELPER(neg, h)(dh_ctype_f16 a) +{ + return float16_chs(a); +} + float32 VFP_HELPER(neg, s)(float32 a) { return float32_chs(a); diff --git a/target/arm/translate-vfp.c.inc b/target/arm/translate-vfp.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-vfp.c.inc +++ b/target/arm/translate-vfp.c.inc @@ -XXX,XX +XXX,XX @@ static bool do_vfp_2op_dp(DisasContext *s, VFPGen2OpDPFn *fn, int vd, int vm) return true; } +static void gen_VMLA_hp(TCGv_i32 vd, TCGv_i32 vn, TCGv_i32 vm, TCGv_ptr fpst) +{ + /* Note that order of inputs to the add matters for NaNs */ + TCGv_i32 tmp = tcg_temp_new_i32(); + + gen_helper_vfp_mulh(tmp, vn, vm, fpst); + gen_helper_vfp_addh(vd, vd, tmp, fpst); + tcg_temp_free_i32(tmp); +} + +static bool trans_VMLA_hp(DisasContext *s, arg_VMLA_sp *a) +{ + return do_vfp_3op_hp(s, gen_VMLA_hp, a->vd, a->vn, a->vm, true); +} + static void gen_VMLA_sp(TCGv_i32 vd, TCGv_i32 vn, TCGv_i32 vm, TCGv_ptr fpst) { /* Note that order of inputs to the add matters for NaNs */ @@ -XXX,XX +XXX,XX @@ static bool trans_VMLA_dp(DisasContext *s, arg_VMLA_dp *a) return do_vfp_3op_dp(s, gen_VMLA_dp, a->vd, a->vn, a->vm, true); } +static void gen_VMLS_hp(TCGv_i32 vd, TCGv_i32 vn, TCGv_i32 vm, TCGv_ptr fpst) +{ + /* + * VMLS: vd = vd + -(vn * vm) + * Note that order of inputs to the add matters for NaNs. + */ + TCGv_i32 tmp = tcg_temp_new_i32(); + + gen_helper_vfp_mulh(tmp, vn, vm, fpst); + gen_helper_vfp_negh(tmp, tmp); + gen_helper_vfp_addh(vd, vd, tmp, fpst); + tcg_temp_free_i32(tmp); +} + +static bool trans_VMLS_hp(DisasContext *s, arg_VMLS_sp *a) +{ + return do_vfp_3op_hp(s, gen_VMLS_hp, a->vd, a->vn, a->vm, true); +} + static void gen_VMLS_sp(TCGv_i32 vd, TCGv_i32 vn, TCGv_i32 vm, TCGv_ptr fpst) { /* @@ -XXX,XX +XXX,XX @@ static bool trans_VMLS_dp(DisasContext *s, arg_VMLS_dp *a) return do_vfp_3op_dp(s, gen_VMLS_dp, a->vd, a->vn, a->vm, true); } +static void gen_VNMLS_hp(TCGv_i32 vd, TCGv_i32 vn, TCGv_i32 vm, TCGv_ptr fpst) +{ + /* + * VNMLS: -fd + (fn * fm) + * Note that it isn't valid to replace (-A + B) with (B - A) or similar + * plausible looking simplifications because this will give wrong results + * for NaNs. + */ + TCGv_i32 tmp = tcg_temp_new_i32(); + + gen_helper_vfp_mulh(tmp, vn, vm, fpst); + gen_helper_vfp_negh(vd, vd); + gen_helper_vfp_addh(vd, vd, tmp, fpst); + tcg_temp_free_i32(tmp); +} + +static bool trans_VNMLS_hp(DisasContext *s, arg_VNMLS_sp *a) +{ + return do_vfp_3op_hp(s, gen_VNMLS_hp, a->vd, a->vn, a->vm, true); +} + static void gen_VNMLS_sp(TCGv_i32 vd, TCGv_i32 vn, TCGv_i32 vm, TCGv_ptr fpst) { /* @@ -XXX,XX +XXX,XX @@ static bool trans_VNMLS_dp(DisasContext *s, arg_VNMLS_dp *a) return do_vfp_3op_dp(s, gen_VNMLS_dp, a->vd, a->vn, a->vm, true); } +static void gen_VNMLA_hp(TCGv_i32 vd, TCGv_i32 vn, TCGv_i32 vm, TCGv_ptr fpst) +{ + /* VNMLA: -fd + -(fn * fm) */ + TCGv_i32 tmp = tcg_temp_new_i32(); + + gen_helper_vfp_mulh(tmp, vn, vm, fpst); + gen_helper_vfp_negh(tmp, tmp); + gen_helper_vfp_negh(vd, vd); + gen_helper_vfp_addh(vd, vd, tmp, fpst); + tcg_temp_free_i32(tmp); +} + +static bool trans_VNMLA_hp(DisasContext *s, arg_VNMLA_sp *a) +{ + return do_vfp_3op_hp(s, gen_VNMLA_hp, a->vd, a->vn, a->vm, true); +} + static void gen_VNMLA_sp(TCGv_i32 vd, TCGv_i32 vn, TCGv_i32 vm, TCGv_ptr fpst) { /* VNMLA: -fd + -(fn * fm) */ @@ -XXX,XX +XXX,XX @@ static bool trans_VMUL_dp(DisasContext *s, arg_VMUL_dp *a) return do_vfp_3op_dp(s, gen_helper_vfp_muld, a->vd, a->vn, a->vm, false); } +static void gen_VNMUL_hp(TCGv_i32 vd, TCGv_i32 vn, TCGv_i32 vm, TCGv_ptr fpst) +{ + /* VNMUL: -(fn * fm) */ + gen_helper_vfp_mulh(vd, vn, vm, fpst); + gen_helper_vfp_negh(vd, vd); +} + +static bool trans_VNMUL_hp(DisasContext *s, arg_VNMUL_sp *a) +{ + return do_vfp_3op_hp(s, gen_VNMUL_hp, a->vd, a->vn, a->vm, false); +} + static void gen_VNMUL_sp(TCGv_i32 vd, TCGv_i32 vn, TCGv_i32 vm, TCGv_ptr fpst) { /* VNMUL: -(fn * fm) */ -- 2.20.1 Macroify creation of the trans functions for single and double precision VFMA, VFMS, VFNMA, VFNMS. The repetition was OK for two sizes, but we're about to add halfprec and it will get a bit more than seems reasonable. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-6-peter.maydell@linaro.org --- target/arm/translate-vfp.c.inc | 50 +++++++++------------------------- 1 file changed, 13 insertions(+), 37 deletions(-) diff --git a/target/arm/translate-vfp.c.inc b/target/arm/translate-vfp.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-vfp.c.inc +++ b/target/arm/translate-vfp.c.inc @@ -XXX,XX +XXX,XX @@ static bool do_vfm_sp(DisasContext *s, arg_VFMA_sp *a, bool neg_n, bool neg_d) return true; } -static bool trans_VFMA_sp(DisasContext *s, arg_VFMA_sp *a) -{ - return do_vfm_sp(s, a, false, false); -} - -static bool trans_VFMS_sp(DisasContext *s, arg_VFMS_sp *a) -{ - return do_vfm_sp(s, a, true, false); -} - -static bool trans_VFNMA_sp(DisasContext *s, arg_VFNMA_sp *a) -{ - return do_vfm_sp(s, a, false, true); -} - -static bool trans_VFNMS_sp(DisasContext *s, arg_VFNMS_sp *a) -{ - return do_vfm_sp(s, a, true, true); -} - static bool do_vfm_dp(DisasContext *s, arg_VFMA_dp *a, bool neg_n, bool neg_d) { /* @@ -XXX,XX +XXX,XX @@ static bool do_vfm_dp(DisasContext *s, arg_VFMA_dp *a, bool neg_n, bool neg_d) return true; } -static bool trans_VFMA_dp(DisasContext *s, arg_VFMA_dp *a) -{ - return do_vfm_dp(s, a, false, false); -} +#define MAKE_ONE_VFM_TRANS_FN(INSN, PREC, NEGN, NEGD) \ + static bool trans_##INSN##_##PREC(DisasContext *s, \ + arg_##INSN##_##PREC *a) \ + { \ + return do_vfm_##PREC(s, a, NEGN, NEGD); \ + } -static bool trans_VFMS_dp(DisasContext *s, arg_VFMS_dp *a) -{ - return do_vfm_dp(s, a, true, false); -} +#define MAKE_VFM_TRANS_FNS(PREC) \ + MAKE_ONE_VFM_TRANS_FN(VFMA, PREC, false, false) \ + MAKE_ONE_VFM_TRANS_FN(VFMS, PREC, true, false) \ + MAKE_ONE_VFM_TRANS_FN(VFNMA, PREC, false, true) \ + MAKE_ONE_VFM_TRANS_FN(VFNMS, PREC, true, true) -static bool trans_VFNMA_dp(DisasContext *s, arg_VFNMA_dp *a) -{ - return do_vfm_dp(s, a, false, true); -} - -static bool trans_VFNMS_dp(DisasContext *s, arg_VFNMS_dp *a) -{ - return do_vfm_dp(s, a, true, true); -} +MAKE_VFM_TRANS_FNS(sp) +MAKE_VFM_TRANS_FNS(dp) static bool trans_VMOV_imm_sp(DisasContext *s, arg_VMOV_imm_sp *a) { -- 2.20.1 Implement VFP fp16 support for fused multiply-add insns VFNMA, VFNMS, VFMA, VFMS. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-7-peter.maydell@linaro.org --- target/arm/helper.h | 1 + target/arm/vfp.decode | 5 +++ target/arm/vfp_helper.c | 7 ++++ target/arm/translate-vfp.c.inc | 64 ++++++++++++++++++++++++++++++++++ 4 files changed, 77 insertions(+) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_3(vfp_fcvt_f64_to_f16, TCG_CALL_NO_RWG, f16, f64, ptr, i32) DEF_HELPER_4(vfp_muladdd, f64, f64, f64, f64, ptr) DEF_HELPER_4(vfp_muladds, f32, f32, f32, f32, ptr) +DEF_HELPER_4(vfp_muladdh, f16, f16, f16, f16, ptr) DEF_HELPER_3(recps_f32, f32, env, f32, f32) DEF_HELPER_3(rsqrts_f32, f32, env, f32, f32) diff --git a/target/arm/vfp.decode b/target/arm/vfp.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vfp.decode +++ b/target/arm/vfp.decode @@ -XXX,XX +XXX,XX @@ VDIV_hp ---- 1110 1.00 .... .... 1001 .0.0 .... @vfp_dnm_s VDIV_sp ---- 1110 1.00 .... .... 1010 .0.0 .... @vfp_dnm_s VDIV_dp ---- 1110 1.00 .... .... 1011 .0.0 .... @vfp_dnm_d +VFMA_hp ---- 1110 1.10 .... .... 1001 .0. 0 .... @vfp_dnm_s +VFMS_hp ---- 1110 1.10 .... .... 1001 .1. 0 .... @vfp_dnm_s +VFNMA_hp ---- 1110 1.01 .... .... 1001 .0. 0 .... @vfp_dnm_s +VFNMS_hp ---- 1110 1.01 .... .... 1001 .1. 0 .... @vfp_dnm_s + VFMA_sp ---- 1110 1.10 .... .... 1010 .0. 0 .... @vfp_dnm_s VFMS_sp ---- 1110 1.10 .... .... 1010 .1. 0 .... @vfp_dnm_s VFNMA_sp ---- 1110 1.01 .... .... 1010 .0. 0 .... @vfp_dnm_s diff --git a/target/arm/vfp_helper.c b/target/arm/vfp_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vfp_helper.c +++ b/target/arm/vfp_helper.c @@ -XXX,XX +XXX,XX @@ uint32_t HELPER(rsqrte_u32)(uint32_t a) } /* VFPv4 fused multiply-accumulate */ +dh_ctype_f16 VFP_HELPER(muladd, h)(dh_ctype_f16 a, dh_ctype_f16 b, + dh_ctype_f16 c, void *fpstp) +{ + float_status *fpst = fpstp; + return float16_muladd(a, b, c, 0, fpst); +} + float32 VFP_HELPER(muladd, s)(float32 a, float32 b, float32 c, void *fpstp) { float_status *fpst = fpstp; diff --git a/target/arm/translate-vfp.c.inc b/target/arm/translate-vfp.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-vfp.c.inc +++ b/target/arm/translate-vfp.c.inc @@ -XXX,XX +XXX,XX @@ static bool trans_VMAXNM_dp(DisasContext *s, arg_VMAXNM_dp *a) a->vd, a->vn, a->vm, false); } +static bool do_vfm_hp(DisasContext *s, arg_VFMA_sp *a, bool neg_n, bool neg_d) +{ + /* + * VFNMA : fd = muladd(-fd, fn, fm) + * VFNMS : fd = muladd(-fd, -fn, fm) + * VFMA : fd = muladd( fd, fn, fm) + * VFMS : fd = muladd( fd, -fn, fm) + * + * These are fused multiply-add, and must be done as one floating + * point operation with no rounding between the multiplication and + * addition steps. NB that doing the negations here as separate + * steps is correct : an input NaN should come out with its sign + * bit flipped if it is a negated-input. + */ + TCGv_ptr fpst; + TCGv_i32 vn, vm, vd; + + /* + * Present in VFPv4 only, and only with the FP16 extension. + * Note that we can't rely on the SIMDFMAC check alone, because + * in a Neon-no-VFP core that ID register field will be non-zero. + */ + if (!dc_isar_feature(aa32_fp16_arith, s) || + !dc_isar_feature(aa32_simdfmac, s) || + !dc_isar_feature(aa32_fpsp_v2, s)) { + return false; + } + + if (s->vec_len != 0 || s->vec_stride != 0) { + return false; + } + + if (!vfp_access_check(s)) { + return true; + } + + vn = tcg_temp_new_i32(); + vm = tcg_temp_new_i32(); + vd = tcg_temp_new_i32(); + + neon_load_reg32(vn, a->vn); + neon_load_reg32(vm, a->vm); + if (neg_n) { + /* VFNMS, VFMS */ + gen_helper_vfp_negh(vn, vn); + } + neon_load_reg32(vd, a->vd); + if (neg_d) { + /* VFNMA, VFNMS */ + gen_helper_vfp_negh(vd, vd); + } + fpst = fpstatus_ptr(FPST_FPCR_F16); + gen_helper_vfp_muladdh(vd, vn, vm, vd, fpst); + neon_store_reg32(vd, a->vd); + + tcg_temp_free_ptr(fpst); + tcg_temp_free_i32(vn); + tcg_temp_free_i32(vm); + tcg_temp_free_i32(vd); + + return true; +} + static bool do_vfm_sp(DisasContext *s, arg_VFMA_sp *a, bool neg_n, bool neg_d) { /* @@ -XXX,XX +XXX,XX @@ static bool do_vfm_dp(DisasContext *s, arg_VFMA_dp *a, bool neg_n, bool neg_d) MAKE_ONE_VFM_TRANS_FN(VFNMA, PREC, false, true) \ MAKE_ONE_VFM_TRANS_FN(VFNMS, PREC, true, true) +MAKE_VFM_TRANS_FNS(hp) MAKE_VFM_TRANS_FNS(sp) MAKE_VFM_TRANS_FNS(dp) -- 2.20.1 Macroify the uses of do_vfp_2op_sp() and do_vfp_2op_dp(); this will make it easier to add the halfprec support. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-8-peter.maydell@linaro.org --- target/arm/translate-vfp.c.inc | 49 ++++++++++------------------------ 1 file changed, 14 insertions(+), 35 deletions(-) diff --git a/target/arm/translate-vfp.c.inc b/target/arm/translate-vfp.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-vfp.c.inc +++ b/target/arm/translate-vfp.c.inc @@ -XXX,XX +XXX,XX @@ static bool trans_VMOV_imm_dp(DisasContext *s, arg_VMOV_imm_dp *a) return true; } -static bool trans_VMOV_reg_sp(DisasContext *s, arg_VMOV_reg_sp *a) -{ - return do_vfp_2op_sp(s, tcg_gen_mov_i32, a->vd, a->vm); -} +#define DO_VFP_2OP(INSN, PREC, FN) \ + static bool trans_##INSN##_##PREC(DisasContext *s, \ + arg_##INSN##_##PREC *a) \ + { \ + return do_vfp_2op_##PREC(s, FN, a->vd, a->vm); \ + } -static bool trans_VMOV_reg_dp(DisasContext *s, arg_VMOV_reg_dp *a) -{ - return do_vfp_2op_dp(s, tcg_gen_mov_i64, a->vd, a->vm); -} +DO_VFP_2OP(VMOV_reg, sp, tcg_gen_mov_i32) +DO_VFP_2OP(VMOV_reg, dp, tcg_gen_mov_i64) -static bool trans_VABS_sp(DisasContext *s, arg_VABS_sp *a) -{ - return do_vfp_2op_sp(s, gen_helper_vfp_abss, a->vd, a->vm); -} +DO_VFP_2OP(VABS, sp, gen_helper_vfp_abss) +DO_VFP_2OP(VABS, dp, gen_helper_vfp_absd) -static bool trans_VABS_dp(DisasContext *s, arg_VABS_dp *a) -{ - return do_vfp_2op_dp(s, gen_helper_vfp_absd, a->vd, a->vm); -} - -static bool trans_VNEG_sp(DisasContext *s, arg_VNEG_sp *a) -{ - return do_vfp_2op_sp(s, gen_helper_vfp_negs, a->vd, a->vm); -} - -static bool trans_VNEG_dp(DisasContext *s, arg_VNEG_dp *a) -{ - return do_vfp_2op_dp(s, gen_helper_vfp_negd, a->vd, a->vm); -} +DO_VFP_2OP(VNEG, sp, gen_helper_vfp_negs) +DO_VFP_2OP(VNEG, dp, gen_helper_vfp_negd) static void gen_VSQRT_sp(TCGv_i32 vd, TCGv_i32 vm) { gen_helper_vfp_sqrts(vd, vm, cpu_env); } -static bool trans_VSQRT_sp(DisasContext *s, arg_VSQRT_sp *a) -{ - return do_vfp_2op_sp(s, gen_VSQRT_sp, a->vd, a->vm); -} - static void gen_VSQRT_dp(TCGv_i64 vd, TCGv_i64 vm) { gen_helper_vfp_sqrtd(vd, vm, cpu_env); } -static bool trans_VSQRT_dp(DisasContext *s, arg_VSQRT_dp *a) -{ - return do_vfp_2op_dp(s, gen_VSQRT_dp, a->vd, a->vm); -} +DO_VFP_2OP(VSQRT, sp, gen_VSQRT_sp) +DO_VFP_2OP(VSQRT, dp, gen_VSQRT_dp) static bool trans_VCMP_sp(DisasContext *s, arg_VCMP_sp *a) { -- 2.20.1 Implement VFP fp16 for VABS, VNEG and VSQRT. This is all the fp16 insns that use the DO_VFP_2OP macro, because there is no fp16 version of VMOV_reg. Notes: * the gen_helper_vfp_negh already exists as we needed to create it for the fp16 multiply-add insns * as usual we need to use the f16 version of the fp_status; this is only relevant for VSQRT Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-9-peter.maydell@linaro.org --- target/arm/helper.h | 2 ++ target/arm/vfp.decode | 3 +++ target/arm/vfp_helper.c | 10 +++++++++ target/arm/translate-vfp.c.inc | 40 ++++++++++++++++++++++++++++++++++ 4 files changed, 55 insertions(+) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_3(vfp_minnumd, f64, f64, f64, ptr) DEF_HELPER_1(vfp_negh, f16, f16) DEF_HELPER_1(vfp_negs, f32, f32) DEF_HELPER_1(vfp_negd, f64, f64) +DEF_HELPER_1(vfp_absh, f16, f16) DEF_HELPER_1(vfp_abss, f32, f32) DEF_HELPER_1(vfp_absd, f64, f64) +DEF_HELPER_2(vfp_sqrth, f16, f16, env) DEF_HELPER_2(vfp_sqrts, f32, f32, env) DEF_HELPER_2(vfp_sqrtd, f64, f64, env) DEF_HELPER_3(vfp_cmps, void, f32, f32, env) diff --git a/target/arm/vfp.decode b/target/arm/vfp.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vfp.decode +++ b/target/arm/vfp.decode @@ -XXX,XX +XXX,XX @@ VMOV_imm_dp ---- 1110 1.11 .... .... 1011 0000 .... \ VMOV_reg_sp ---- 1110 1.11 0000 .... 1010 01.0 .... @vfp_dm_ss VMOV_reg_dp ---- 1110 1.11 0000 .... 1011 01.0 .... @vfp_dm_dd +VABS_hp ---- 1110 1.11 0000 .... 1001 11.0 .... @vfp_dm_ss VABS_sp ---- 1110 1.11 0000 .... 1010 11.0 .... @vfp_dm_ss VABS_dp ---- 1110 1.11 0000 .... 1011 11.0 .... @vfp_dm_dd +VNEG_hp ---- 1110 1.11 0001 .... 1001 01.0 .... @vfp_dm_ss VNEG_sp ---- 1110 1.11 0001 .... 1010 01.0 .... @vfp_dm_ss VNEG_dp ---- 1110 1.11 0001 .... 1011 01.0 .... @vfp_dm_dd +VSQRT_hp ---- 1110 1.11 0001 .... 1001 11.0 .... @vfp_dm_ss VSQRT_sp ---- 1110 1.11 0001 .... 1010 11.0 .... @vfp_dm_ss VSQRT_dp ---- 1110 1.11 0001 .... 1011 11.0 .... @vfp_dm_dd diff --git a/target/arm/vfp_helper.c b/target/arm/vfp_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vfp_helper.c +++ b/target/arm/vfp_helper.c @@ -XXX,XX +XXX,XX @@ float64 VFP_HELPER(neg, d)(float64 a) return float64_chs(a); } +dh_ctype_f16 VFP_HELPER(abs, h)(dh_ctype_f16 a) +{ + return float16_abs(a); +} + float32 VFP_HELPER(abs, s)(float32 a) { return float32_abs(a); @@ -XXX,XX +XXX,XX @@ float64 VFP_HELPER(abs, d)(float64 a) return float64_abs(a); } +dh_ctype_f16 VFP_HELPER(sqrt, h)(dh_ctype_f16 a, CPUARMState *env) +{ + return float16_sqrt(a, &env->vfp.fp_status_f16); +} + float32 VFP_HELPER(sqrt, s)(float32 a, CPUARMState *env) { return float32_sqrt(a, &env->vfp.fp_status); diff --git a/target/arm/translate-vfp.c.inc b/target/arm/translate-vfp.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-vfp.c.inc +++ b/target/arm/translate-vfp.c.inc @@ -XXX,XX +XXX,XX @@ static bool do_vfp_2op_sp(DisasContext *s, VFPGen2OpSPFn *fn, int vd, int vm) return true; } +static bool do_vfp_2op_hp(DisasContext *s, VFPGen2OpSPFn *fn, int vd, int vm) +{ + /* + * Do a half-precision operation. Functionally this is + * the same as do_vfp_2op_sp(), except: + * - it doesn't need the VFP vector handling (fp16 is a + * v8 feature, and in v8 VFP vectors don't exist) + * - it does the aa32_fp16_arith feature test + */ + TCGv_i32 f0; + + if (!dc_isar_feature(aa32_fp16_arith, s)) { + return false; + } + + if (s->vec_len != 0 || s->vec_stride != 0) { + return false; + } + + if (!vfp_access_check(s)) { + return true; + } + + f0 = tcg_temp_new_i32(); + neon_load_reg32(f0, vm); + fn(f0, f0); + neon_store_reg32(f0, vd); + tcg_temp_free_i32(f0); + + return true; +} + static bool do_vfp_2op_dp(DisasContext *s, VFPGen2OpDPFn *fn, int vd, int vm) { uint32_t delta_m = 0; @@ -XXX,XX +XXX,XX @@ static bool trans_VMOV_imm_dp(DisasContext *s, arg_VMOV_imm_dp *a) DO_VFP_2OP(VMOV_reg, sp, tcg_gen_mov_i32) DO_VFP_2OP(VMOV_reg, dp, tcg_gen_mov_i64) +DO_VFP_2OP(VABS, hp, gen_helper_vfp_absh) DO_VFP_2OP(VABS, sp, gen_helper_vfp_abss) DO_VFP_2OP(VABS, dp, gen_helper_vfp_absd) +DO_VFP_2OP(VNEG, hp, gen_helper_vfp_negh) DO_VFP_2OP(VNEG, sp, gen_helper_vfp_negs) DO_VFP_2OP(VNEG, dp, gen_helper_vfp_negd) +static void gen_VSQRT_hp(TCGv_i32 vd, TCGv_i32 vm) +{ + gen_helper_vfp_sqrth(vd, vm, cpu_env); +} + static void gen_VSQRT_sp(TCGv_i32 vd, TCGv_i32 vm) { gen_helper_vfp_sqrts(vd, vm, cpu_env); @@ -XXX,XX +XXX,XX @@ static void gen_VSQRT_dp(TCGv_i64 vd, TCGv_i64 vm) gen_helper_vfp_sqrtd(vd, vm, cpu_env); } +DO_VFP_2OP(VSQRT, hp, gen_VSQRT_hp) DO_VFP_2OP(VSQRT, sp, gen_VSQRT_sp) DO_VFP_2OP(VSQRT, dp, gen_VSQRT_dp) -- 2.20.1 Implement VFP fp16 support for the VMOV immediate insn. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-10-peter.maydell@linaro.org --- target/arm/vfp.decode | 2 ++ target/arm/translate-vfp.c.inc | 22 ++++++++++++++++++++++ 2 files changed, 24 insertions(+) diff --git a/target/arm/vfp.decode b/target/arm/vfp.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vfp.decode +++ b/target/arm/vfp.decode @@ -XXX,XX +XXX,XX @@ VFMS_dp ---- 1110 1.10 .... .... 1011 .1.0 .... @vfp_dnm_d VFNMA_dp ---- 1110 1.01 .... .... 1011 .0.0 .... @vfp_dnm_d VFNMS_dp ---- 1110 1.01 .... .... 1011 .1.0 .... @vfp_dnm_d +VMOV_imm_hp ---- 1110 1.11 .... .... 1001 0000 .... \ + vd=%vd_sp imm=%vmov_imm VMOV_imm_sp ---- 1110 1.11 .... .... 1010 0000 .... \ vd=%vd_sp imm=%vmov_imm VMOV_imm_dp ---- 1110 1.11 .... .... 1011 0000 .... \ diff --git a/target/arm/translate-vfp.c.inc b/target/arm/translate-vfp.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-vfp.c.inc +++ b/target/arm/translate-vfp.c.inc @@ -XXX,XX +XXX,XX @@ MAKE_VFM_TRANS_FNS(hp) MAKE_VFM_TRANS_FNS(sp) MAKE_VFM_TRANS_FNS(dp) +static bool trans_VMOV_imm_hp(DisasContext *s, arg_VMOV_imm_sp *a) +{ + TCGv_i32 fd; + + if (!dc_isar_feature(aa32_fp16_arith, s)) { + return false; + } + + if (s->vec_len != 0 || s->vec_stride != 0) { + return false; + } + + if (!vfp_access_check(s)) { + return true; + } + + fd = tcg_const_i32(vfp_expand_imm(MO_16, a->imm)); + neon_store_reg32(fd, a->vd); + tcg_temp_free_i32(fd); + return true; +} + static bool trans_VMOV_imm_sp(DisasContext *s, arg_VMOV_imm_sp *a) { uint32_t delta_d = 0; -- 2.20.1 Implement fp16 version of VCMP. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-11-peter.maydell@linaro.org --- target/arm/helper.h | 2 ++ target/arm/vfp.decode | 2 ++ target/arm/vfp_helper.c | 15 +++++++------ target/arm/translate-vfp.c.inc | 39 ++++++++++++++++++++++++++++++++++ 4 files changed, 51 insertions(+), 7 deletions(-) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_1(vfp_absd, f64, f64) DEF_HELPER_2(vfp_sqrth, f16, f16, env) DEF_HELPER_2(vfp_sqrts, f32, f32, env) DEF_HELPER_2(vfp_sqrtd, f64, f64, env) +DEF_HELPER_3(vfp_cmph, void, f16, f16, env) DEF_HELPER_3(vfp_cmps, void, f32, f32, env) DEF_HELPER_3(vfp_cmpd, void, f64, f64, env) +DEF_HELPER_3(vfp_cmpeh, void, f16, f16, env) DEF_HELPER_3(vfp_cmpes, void, f32, f32, env) DEF_HELPER_3(vfp_cmped, void, f64, f64, env) diff --git a/target/arm/vfp.decode b/target/arm/vfp.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vfp.decode +++ b/target/arm/vfp.decode @@ -XXX,XX +XXX,XX @@ VSQRT_hp ---- 1110 1.11 0001 .... 1001 11.0 .... @vfp_dm_ss VSQRT_sp ---- 1110 1.11 0001 .... 1010 11.0 .... @vfp_dm_ss VSQRT_dp ---- 1110 1.11 0001 .... 1011 11.0 .... @vfp_dm_dd +VCMP_hp ---- 1110 1.11 010 z:1 .... 1001 e:1 1.0 .... \ + vd=%vd_sp vm=%vm_sp VCMP_sp ---- 1110 1.11 010 z:1 .... 1010 e:1 1.0 .... \ vd=%vd_sp vm=%vm_sp VCMP_dp ---- 1110 1.11 010 z:1 .... 1011 e:1 1.0 .... \ diff --git a/target/arm/vfp_helper.c b/target/arm/vfp_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vfp_helper.c +++ b/target/arm/vfp_helper.c @@ -XXX,XX +XXX,XX @@ static void softfloat_to_vfp_compare(CPUARMState *env, FloatRelation cmp) } /* XXX: check quiet/signaling case */ -#define DO_VFP_cmp(p, type) \ -void VFP_HELPER(cmp, p)(type a, type b, CPUARMState *env) \ +#define DO_VFP_cmp(P, FLOATTYPE, ARGTYPE, FPST) \ +void VFP_HELPER(cmp, P)(ARGTYPE a, ARGTYPE b, CPUARMState *env) \ { \ softfloat_to_vfp_compare(env, \ - type ## _compare_quiet(a, b, &env->vfp.fp_status)); \ + FLOATTYPE ## _compare_quiet(a, b, &env->vfp.FPST)); \ } \ -void VFP_HELPER(cmpe, p)(type a, type b, CPUARMState *env) \ +void VFP_HELPER(cmpe, P)(ARGTYPE a, ARGTYPE b, CPUARMState *env) \ { \ softfloat_to_vfp_compare(env, \ - type ## _compare(a, b, &env->vfp.fp_status)); \ + FLOATTYPE ## _compare(a, b, &env->vfp.FPST)); \ } -DO_VFP_cmp(s, float32) -DO_VFP_cmp(d, float64) +DO_VFP_cmp(h, float16, dh_ctype_f16, fp_status_f16) +DO_VFP_cmp(s, float32, float32, fp_status) +DO_VFP_cmp(d, float64, float64, fp_status) #undef DO_VFP_cmp /* Integer to float and float to integer conversions */ diff --git a/target/arm/translate-vfp.c.inc b/target/arm/translate-vfp.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-vfp.c.inc +++ b/target/arm/translate-vfp.c.inc @@ -XXX,XX +XXX,XX @@ DO_VFP_2OP(VSQRT, hp, gen_VSQRT_hp) DO_VFP_2OP(VSQRT, sp, gen_VSQRT_sp) DO_VFP_2OP(VSQRT, dp, gen_VSQRT_dp) +static bool trans_VCMP_hp(DisasContext *s, arg_VCMP_sp *a) +{ + TCGv_i32 vd, vm; + + if (!dc_isar_feature(aa32_fp16_arith, s)) { + return false; + } + + /* Vm/M bits must be zero for the Z variant */ + if (a->z && a->vm != 0) { + return false; + } + + if (!vfp_access_check(s)) { + return true; + } + + vd = tcg_temp_new_i32(); + vm = tcg_temp_new_i32(); + + neon_load_reg32(vd, a->vd); + if (a->z) { + tcg_gen_movi_i32(vm, 0); + } else { + neon_load_reg32(vm, a->vm); + } + + if (a->e) { + gen_helper_vfp_cmpeh(vd, vm, cpu_env); + } else { + gen_helper_vfp_cmph(vd, vm, cpu_env); + } + + tcg_temp_free_i32(vd); + tcg_temp_free_i32(vm); + + return true; +} + static bool trans_VCMP_sp(DisasContext *s, arg_VCMP_sp *a) { TCGv_i32 vd, vm; -- 2.20.1 Implement the fp16 versions of the VFP VLDR/VSTR (immediate). Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-12-peter.maydell@linaro.org --- target/arm/vfp.decode | 3 +-- target/arm/translate-vfp.c.inc | 35 ++++++++++++++++++++++++++++++++++ 2 files changed, 36 insertions(+), 2 deletions(-) diff --git a/target/arm/vfp.decode b/target/arm/vfp.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vfp.decode +++ b/target/arm/vfp.decode @@ -XXX,XX +XXX,XX @@ VMOV_single ---- 1110 000 l:1 .... rt:4 1010 . 001 0000 vn=%vn_sp VMOV_64_sp ---- 1100 010 op:1 rt2:4 rt:4 1010 00.1 .... vm=%vm_sp VMOV_64_dp ---- 1100 010 op:1 rt2:4 rt:4 1011 00.1 .... vm=%vm_dp -# Note that the half-precision variants of VLDR and VSTR are -# not part of this decodetree at all because they have bits [9:8] == 0b01 +VLDR_VSTR_hp ---- 1101 u:1 .0 l:1 rn:4 .... 1001 imm:8 vd=%vd_sp VLDR_VSTR_sp ---- 1101 u:1 .0 l:1 rn:4 .... 1010 imm:8 vd=%vd_sp VLDR_VSTR_dp ---- 1101 u:1 .0 l:1 rn:4 .... 1011 imm:8 vd=%vd_dp diff --git a/target/arm/translate-vfp.c.inc b/target/arm/translate-vfp.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-vfp.c.inc +++ b/target/arm/translate-vfp.c.inc @@ -XXX,XX +XXX,XX @@ static bool trans_VMOV_64_dp(DisasContext *s, arg_VMOV_64_dp *a) return true; } +static bool trans_VLDR_VSTR_hp(DisasContext *s, arg_VLDR_VSTR_sp *a) +{ + uint32_t offset; + TCGv_i32 addr, tmp; + + if (!dc_isar_feature(aa32_fp16_arith, s)) { + return false; + } + + if (!vfp_access_check(s)) { + return true; + } + + /* imm8 field is offset/2 for fp16, unlike fp32 and fp64 */ + offset = a->imm << 1; + if (!a->u) { + offset = -offset; + } + + /* For thumb, use of PC is UNPREDICTABLE. */ + addr = add_reg_for_lit(s, a->rn, offset); + tmp = tcg_temp_new_i32(); + if (a->l) { + gen_aa32_ld16u(s, tmp, addr, get_mem_index(s)); + neon_store_reg32(tmp, a->vd); + } else { + neon_load_reg32(tmp, a->vd); + gen_aa32_st16(s, tmp, addr, get_mem_index(s)); + } + tcg_temp_free_i32(tmp); + tcg_temp_free_i32(addr); + + return true; +} + static bool trans_VLDR_VSTR_sp(DisasContext *s, arg_VLDR_VSTR_sp *a) { uint32_t offset; -- 2.20.1 Implement the fp16 versions of the VFP VCVT instruction forms which convert between floating point and integer. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-13-peter.maydell@linaro.org --- target/arm/vfp.decode | 4 +++ target/arm/translate-vfp.c.inc | 65 ++++++++++++++++++++++++++++++++++ 2 files changed, 69 insertions(+) diff --git a/target/arm/vfp.decode b/target/arm/vfp.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vfp.decode +++ b/target/arm/vfp.decode @@ -XXX,XX +XXX,XX @@ VCVT_sp ---- 1110 1.11 0111 .... 1010 11.0 .... @vfp_dm_ds VCVT_dp ---- 1110 1.11 0111 .... 1011 11.0 .... @vfp_dm_sd # VCVT from integer to floating point: Vm always single; Vd depends on size +VCVT_int_hp ---- 1110 1.11 1000 .... 1001 s:1 1.0 .... \ + vd=%vd_sp vm=%vm_sp VCVT_int_sp ---- 1110 1.11 1000 .... 1010 s:1 1.0 .... \ vd=%vd_sp vm=%vm_sp VCVT_int_dp ---- 1110 1.11 1000 .... 1011 s:1 1.0 .... \ @@ -XXX,XX +XXX,XX @@ VCVT_fix_dp ---- 1110 1.11 1.1. .... 1011 .1.0 .... \ vd=%vd_dp imm=%vm_sp opc=%vcvt_fix_op # VCVT float to integer (VCVT and VCVTR): Vd always single; Vd depends on size +VCVT_hp_int ---- 1110 1.11 110 s:1 .... 1001 rz:1 1.0 .... \ + vd=%vd_sp vm=%vm_sp VCVT_sp_int ---- 1110 1.11 110 s:1 .... 1010 rz:1 1.0 .... \ vd=%vd_sp vm=%vm_sp VCVT_dp_int ---- 1110 1.11 110 s:1 .... 1011 rz:1 1.0 .... \ diff --git a/target/arm/translate-vfp.c.inc b/target/arm/translate-vfp.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-vfp.c.inc +++ b/target/arm/translate-vfp.c.inc @@ -XXX,XX +XXX,XX @@ static bool trans_VCVT_dp(DisasContext *s, arg_VCVT_dp *a) return true; } +static bool trans_VCVT_int_hp(DisasContext *s, arg_VCVT_int_sp *a) +{ + TCGv_i32 vm; + TCGv_ptr fpst; + + if (!dc_isar_feature(aa32_fp16_arith, s)) { + return false; + } + + if (!vfp_access_check(s)) { + return true; + } + + vm = tcg_temp_new_i32(); + neon_load_reg32(vm, a->vm); + fpst = fpstatus_ptr(FPST_FPCR_F16); + if (a->s) { + /* i32 -> f16 */ + gen_helper_vfp_sitoh(vm, vm, fpst); + } else { + /* u32 -> f16 */ + gen_helper_vfp_uitoh(vm, vm, fpst); + } + neon_store_reg32(vm, a->vd); + tcg_temp_free_i32(vm); + tcg_temp_free_ptr(fpst); + return true; +} + static bool trans_VCVT_int_sp(DisasContext *s, arg_VCVT_int_sp *a) { TCGv_i32 vm; @@ -XXX,XX +XXX,XX @@ static bool trans_VCVT_fix_dp(DisasContext *s, arg_VCVT_fix_dp *a) return true; } +static bool trans_VCVT_hp_int(DisasContext *s, arg_VCVT_sp_int *a) +{ + TCGv_i32 vm; + TCGv_ptr fpst; + + if (!dc_isar_feature(aa32_fp16_arith, s)) { + return false; + } + + if (!vfp_access_check(s)) { + return true; + } + + fpst = fpstatus_ptr(FPST_FPCR_F16); + vm = tcg_temp_new_i32(); + neon_load_reg32(vm, a->vm); + + if (a->s) { + if (a->rz) { + gen_helper_vfp_tosizh(vm, vm, fpst); + } else { + gen_helper_vfp_tosih(vm, vm, fpst); + } + } else { + if (a->rz) { + gen_helper_vfp_touizh(vm, vm, fpst); + } else { + gen_helper_vfp_touih(vm, vm, fpst); + } + } + neon_store_reg32(vm, a->vd); + tcg_temp_free_i32(vm); + tcg_temp_free_ptr(fpst); + return true; +} + static bool trans_VCVT_sp_int(DisasContext *s, arg_VCVT_sp_int *a) { TCGv_i32 vm; -- 2.20.1 Currently the VFP_CONV_FIX macros take a single fsz argument for the size of the float type, which is used both to select the name of the functions to call (eg float32_is_any_nan()) and also for the type to use for the float inputs and outputs (eg float32). Separate these into fsz and ftype arguments, so that we can use them for fp16, which uses 'float16' in the function names but is still passing inputs and outputs in a 32-bit sized type. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-14-peter.maydell@linaro.org --- target/arm/vfp_helper.c | 46 ++++++++++++++++++++--------------------- 1 file changed, 23 insertions(+), 23 deletions(-) diff --git a/target/arm/vfp_helper.c b/target/arm/vfp_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vfp_helper.c +++ b/target/arm/vfp_helper.c @@ -XXX,XX +XXX,XX @@ float32 VFP_HELPER(fcvts, d)(float64 x, CPUARMState *env) } /* VFP3 fixed point conversion. */ -#define VFP_CONV_FIX_FLOAT(name, p, fsz, isz, itype) \ -float##fsz HELPER(vfp_##name##to##p)(uint##isz##_t x, uint32_t shift, \ +#define VFP_CONV_FIX_FLOAT(name, p, fsz, ftype, isz, itype) \ +ftype HELPER(vfp_##name##to##p)(uint##isz##_t x, uint32_t shift, \ void *fpstp) \ { return itype##_to_##float##fsz##_scalbn(x, -shift, fpstp); } -#define VFP_CONV_FLOAT_FIX_ROUND(name, p, fsz, isz, itype, ROUND, suff) \ -uint##isz##_t HELPER(vfp_to##name##p##suff)(float##fsz x, uint32_t shift, \ +#define VFP_CONV_FLOAT_FIX_ROUND(name, p, fsz, ftype, isz, itype, ROUND, suff) \ +uint##isz##_t HELPER(vfp_to##name##p##suff)(ftype x, uint32_t shift, \ void *fpst) \ { \ if (unlikely(float##fsz##_is_any_nan(x))) { \ @@ -XXX,XX +XXX,XX @@ uint##isz##_t HELPER(vfp_to##name##p##suff)(float##fsz x, uint32_t shift, \ return float##fsz##_to_##itype##_scalbn(x, ROUND, shift, fpst); \ } -#define VFP_CONV_FIX(name, p, fsz, isz, itype) \ -VFP_CONV_FIX_FLOAT(name, p, fsz, isz, itype) \ -VFP_CONV_FLOAT_FIX_ROUND(name, p, fsz, isz, itype, \ +#define VFP_CONV_FIX(name, p, fsz, ftype, isz, itype) \ +VFP_CONV_FIX_FLOAT(name, p, fsz, ftype, isz, itype) \ +VFP_CONV_FLOAT_FIX_ROUND(name, p, fsz, ftype, isz, itype, \ float_round_to_zero, _round_to_zero) \ -VFP_CONV_FLOAT_FIX_ROUND(name, p, fsz, isz, itype, \ +VFP_CONV_FLOAT_FIX_ROUND(name, p, fsz, ftype, isz, itype, \ get_float_rounding_mode(fpst), ) -#define VFP_CONV_FIX_A64(name, p, fsz, isz, itype) \ -VFP_CONV_FIX_FLOAT(name, p, fsz, isz, itype) \ -VFP_CONV_FLOAT_FIX_ROUND(name, p, fsz, isz, itype, \ +#define VFP_CONV_FIX_A64(name, p, fsz, ftype, isz, itype) \ +VFP_CONV_FIX_FLOAT(name, p, fsz, ftype, isz, itype) \ +VFP_CONV_FLOAT_FIX_ROUND(name, p, fsz, ftype, isz, itype, \ get_float_rounding_mode(fpst), ) -VFP_CONV_FIX(sh, d, 64, 64, int16) -VFP_CONV_FIX(sl, d, 64, 64, int32) -VFP_CONV_FIX_A64(sq, d, 64, 64, int64) -VFP_CONV_FIX(uh, d, 64, 64, uint16) -VFP_CONV_FIX(ul, d, 64, 64, uint32) -VFP_CONV_FIX_A64(uq, d, 64, 64, uint64) -VFP_CONV_FIX(sh, s, 32, 32, int16) -VFP_CONV_FIX(sl, s, 32, 32, int32) -VFP_CONV_FIX_A64(sq, s, 32, 64, int64) -VFP_CONV_FIX(uh, s, 32, 32, uint16) -VFP_CONV_FIX(ul, s, 32, 32, uint32) -VFP_CONV_FIX_A64(uq, s, 32, 64, uint64) +VFP_CONV_FIX(sh, d, 64, float64, 64, int16) +VFP_CONV_FIX(sl, d, 64, float64, 64, int32) +VFP_CONV_FIX_A64(sq, d, 64, float64, 64, int64) +VFP_CONV_FIX(uh, d, 64, float64, 64, uint16) +VFP_CONV_FIX(ul, d, 64, float64, 64, uint32) +VFP_CONV_FIX_A64(uq, d, 64, float64, 64, uint64) +VFP_CONV_FIX(sh, s, 32, float32, 32, int16) +VFP_CONV_FIX(sl, s, 32, float32, 32, int32) +VFP_CONV_FIX_A64(sq, s, 32, float32, 64, int64) +VFP_CONV_FIX(uh, s, 32, float32, 32, uint16) +VFP_CONV_FIX(ul, s, 32, float32, 32, uint32) +VFP_CONV_FIX_A64(uq, s, 32, float32, 64, uint64) #undef VFP_CONV_FIX #undef VFP_CONV_FIX_FLOAT -- 2.20.1 Now the VFP_CONV_FIX macros can handle fp16's distinction between the width of the operation and the width of the type used to pass operands, use the macros rather than the open-coded functions. This creates an extra six helper functions, all of which we are going to need for the AArch32 VFP fp16 instructions. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-15-peter.maydell@linaro.org --- target/arm/helper.h | 6 +++ target/arm/vfp_helper.c | 86 +++-------------------------------------- 2 files changed, 12 insertions(+), 80 deletions(-) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_2(vfp_tosizh, s32, f16, ptr) DEF_HELPER_2(vfp_tosizs, s32, f32, ptr) DEF_HELPER_2(vfp_tosizd, s32, f64, ptr) +DEF_HELPER_3(vfp_toshh_round_to_zero, i32, f16, i32, ptr) +DEF_HELPER_3(vfp_toslh_round_to_zero, i32, f16, i32, ptr) +DEF_HELPER_3(vfp_touhh_round_to_zero, i32, f16, i32, ptr) +DEF_HELPER_3(vfp_toulh_round_to_zero, i32, f16, i32, ptr) DEF_HELPER_3(vfp_toshs_round_to_zero, i32, f32, i32, ptr) DEF_HELPER_3(vfp_tosls_round_to_zero, i32, f32, i32, ptr) DEF_HELPER_3(vfp_touhs_round_to_zero, i32, f32, i32, ptr) @@ -XXX,XX +XXX,XX @@ DEF_HELPER_3(vfp_sqtod, f64, i64, i32, ptr) DEF_HELPER_3(vfp_uhtod, f64, i64, i32, ptr) DEF_HELPER_3(vfp_ultod, f64, i64, i32, ptr) DEF_HELPER_3(vfp_uqtod, f64, i64, i32, ptr) +DEF_HELPER_3(vfp_shtoh, f16, i32, i32, ptr) +DEF_HELPER_3(vfp_uhtoh, f16, i32, i32, ptr) DEF_HELPER_3(vfp_sltoh, f16, i32, i32, ptr) DEF_HELPER_3(vfp_ultoh, f16, i32, i32, ptr) DEF_HELPER_3(vfp_sqtoh, f16, i64, i32, ptr) diff --git a/target/arm/vfp_helper.c b/target/arm/vfp_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vfp_helper.c +++ b/target/arm/vfp_helper.c @@ -XXX,XX +XXX,XX @@ VFP_CONV_FIX_A64(sq, s, 32, float32, 64, int64) VFP_CONV_FIX(uh, s, 32, float32, 32, uint16) VFP_CONV_FIX(ul, s, 32, float32, 32, uint32) VFP_CONV_FIX_A64(uq, s, 32, float32, 64, uint64) +VFP_CONV_FIX(sh, h, 16, dh_ctype_f16, 32, int16) +VFP_CONV_FIX(sl, h, 16, dh_ctype_f16, 32, int32) +VFP_CONV_FIX_A64(sq, h, 16, dh_ctype_f16, 64, int64) +VFP_CONV_FIX(uh, h, 16, dh_ctype_f16, 32, uint16) +VFP_CONV_FIX(ul, h, 16, dh_ctype_f16, 32, uint32) +VFP_CONV_FIX_A64(uq, h, 16, dh_ctype_f16, 64, uint64) #undef VFP_CONV_FIX #undef VFP_CONV_FIX_FLOAT #undef VFP_CONV_FLOAT_FIX_ROUND #undef VFP_CONV_FIX_A64 -uint32_t HELPER(vfp_sltoh)(uint32_t x, uint32_t shift, void *fpst) -{ - return int32_to_float16_scalbn(x, -shift, fpst); -} - -uint32_t HELPER(vfp_ultoh)(uint32_t x, uint32_t shift, void *fpst) -{ - return uint32_to_float16_scalbn(x, -shift, fpst); -} - -uint32_t HELPER(vfp_sqtoh)(uint64_t x, uint32_t shift, void *fpst) -{ - return int64_to_float16_scalbn(x, -shift, fpst); -} - -uint32_t HELPER(vfp_uqtoh)(uint64_t x, uint32_t shift, void *fpst) -{ - return uint64_to_float16_scalbn(x, -shift, fpst); -} - -uint32_t HELPER(vfp_toshh)(uint32_t x, uint32_t shift, void *fpst) -{ - if (unlikely(float16_is_any_nan(x))) { - float_raise(float_flag_invalid, fpst); - return 0; - } - return float16_to_int16_scalbn(x, get_float_rounding_mode(fpst), - shift, fpst); -} - -uint32_t HELPER(vfp_touhh)(uint32_t x, uint32_t shift, void *fpst) -{ - if (unlikely(float16_is_any_nan(x))) { - float_raise(float_flag_invalid, fpst); - return 0; - } - return float16_to_uint16_scalbn(x, get_float_rounding_mode(fpst), - shift, fpst); -} - -uint32_t HELPER(vfp_toslh)(uint32_t x, uint32_t shift, void *fpst) -{ - if (unlikely(float16_is_any_nan(x))) { - float_raise(float_flag_invalid, fpst); - return 0; - } - return float16_to_int32_scalbn(x, get_float_rounding_mode(fpst), - shift, fpst); -} - -uint32_t HELPER(vfp_toulh)(uint32_t x, uint32_t shift, void *fpst) -{ - if (unlikely(float16_is_any_nan(x))) { - float_raise(float_flag_invalid, fpst); - return 0; - } - return float16_to_uint32_scalbn(x, get_float_rounding_mode(fpst), - shift, fpst); -} - -uint64_t HELPER(vfp_tosqh)(uint32_t x, uint32_t shift, void *fpst) -{ - if (unlikely(float16_is_any_nan(x))) { - float_raise(float_flag_invalid, fpst); - return 0; - } - return float16_to_int64_scalbn(x, get_float_rounding_mode(fpst), - shift, fpst); -} - -uint64_t HELPER(vfp_touqh)(uint32_t x, uint32_t shift, void *fpst) -{ - if (unlikely(float16_is_any_nan(x))) { - float_raise(float_flag_invalid, fpst); - return 0; - } - return float16_to_uint64_scalbn(x, get_float_rounding_mode(fpst), - shift, fpst); -} - /* Set the current fp rounding mode and return the old one. * The argument is a softfloat float_round_ value. */ -- 2.20.1 Implement the fp16 versions of the VFP VCVT instruction forms which convert between floating point and fixed-point. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-16-peter.maydell@linaro.org --- target/arm/vfp.decode | 2 ++ target/arm/translate-vfp.c.inc | 59 ++++++++++++++++++++++++++++++++++ 2 files changed, 61 insertions(+) diff --git a/target/arm/vfp.decode b/target/arm/vfp.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vfp.decode +++ b/target/arm/vfp.decode @@ -XXX,XX +XXX,XX @@ VJCVT ---- 1110 1.11 1001 .... 1011 11.0 .... @vfp_dm_sd # We assemble bits 18 (op), 16 (u) and 7 (sx) into a single opc field # for the convenience of the trans_VCVT_fix functions. %vcvt_fix_op 18:1 16:1 7:1 +VCVT_fix_hp ---- 1110 1.11 1.1. .... 1001 .1.0 .... \ + vd=%vd_sp imm=%vm_sp opc=%vcvt_fix_op VCVT_fix_sp ---- 1110 1.11 1.1. .... 1010 .1.0 .... \ vd=%vd_sp imm=%vm_sp opc=%vcvt_fix_op VCVT_fix_dp ---- 1110 1.11 1.1. .... 1011 .1.0 .... \ diff --git a/target/arm/translate-vfp.c.inc b/target/arm/translate-vfp.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-vfp.c.inc +++ b/target/arm/translate-vfp.c.inc @@ -XXX,XX +XXX,XX @@ static bool trans_VJCVT(DisasContext *s, arg_VJCVT *a) return true; } +static bool trans_VCVT_fix_hp(DisasContext *s, arg_VCVT_fix_sp *a) +{ + TCGv_i32 vd, shift; + TCGv_ptr fpst; + int frac_bits; + + if (!dc_isar_feature(aa32_fp16_arith, s)) { + return false; + } + + if (!vfp_access_check(s)) { + return true; + } + + frac_bits = (a->opc & 1) ? (32 - a->imm) : (16 - a->imm); + + vd = tcg_temp_new_i32(); + neon_load_reg32(vd, a->vd); + + fpst = fpstatus_ptr(FPST_FPCR_F16); + shift = tcg_const_i32(frac_bits); + + /* Switch on op:U:sx bits */ + switch (a->opc) { + case 0: + gen_helper_vfp_shtoh(vd, vd, shift, fpst); + break; + case 1: + gen_helper_vfp_sltoh(vd, vd, shift, fpst); + break; + case 2: + gen_helper_vfp_uhtoh(vd, vd, shift, fpst); + break; + case 3: + gen_helper_vfp_ultoh(vd, vd, shift, fpst); + break; + case 4: + gen_helper_vfp_toshh_round_to_zero(vd, vd, shift, fpst); + break; + case 5: + gen_helper_vfp_toslh_round_to_zero(vd, vd, shift, fpst); + break; + case 6: + gen_helper_vfp_touhh_round_to_zero(vd, vd, shift, fpst); + break; + case 7: + gen_helper_vfp_toulh_round_to_zero(vd, vd, shift, fpst); + break; + default: + g_assert_not_reached(); + } + + neon_store_reg32(vd, a->vd); + tcg_temp_free_i32(vd); + tcg_temp_free_i32(shift); + tcg_temp_free_ptr(fpst); + return true; +} + static bool trans_VCVT_fix_sp(DisasContext *s, arg_VCVT_fix_sp *a) { TCGv_i32 vd, shift; -- 2.20.1 Implement the fp16 versions of the VFP VCVT instruction forms which convert between floating point and integer with a specified rounding mode. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-17-peter.maydell@linaro.org --- target/arm/vfp-uncond.decode | 6 ++++-- target/arm/translate-vfp.c.inc | 32 ++++++++++++++++++++++++-------- 2 files changed, 28 insertions(+), 10 deletions(-) diff --git a/target/arm/vfp-uncond.decode b/target/arm/vfp-uncond.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vfp-uncond.decode +++ b/target/arm/vfp-uncond.decode @@ -XXX,XX +XXX,XX @@ VRINT 1111 1110 1.11 10 rm:2 .... 1011 01.0 .... \ vm=%vm_dp vd=%vd_dp dp=1 # VCVT float to int with specified rounding mode; Vd is always single-precision +VCVT 1111 1110 1.11 11 rm:2 .... 1001 op:1 1.0 .... \ + vm=%vm_sp vd=%vd_sp sz=1 VCVT 1111 1110 1.11 11 rm:2 .... 1010 op:1 1.0 .... \ - vm=%vm_sp vd=%vd_sp dp=0 + vm=%vm_sp vd=%vd_sp sz=2 VCVT 1111 1110 1.11 11 rm:2 .... 1011 op:1 1.0 .... \ - vm=%vm_dp vd=%vd_sp dp=1 + vm=%vm_dp vd=%vd_sp sz=3 diff --git a/target/arm/translate-vfp.c.inc b/target/arm/translate-vfp.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-vfp.c.inc +++ b/target/arm/translate-vfp.c.inc @@ -XXX,XX +XXX,XX @@ static bool trans_VRINT(DisasContext *s, arg_VRINT *a) static bool trans_VCVT(DisasContext *s, arg_VCVT *a) { uint32_t rd, rm; - bool dp = a->dp; + int sz = a->sz; TCGv_ptr fpst; TCGv_i32 tcg_rmode, tcg_shift; int rounding = fp_decode_rm[a->rm]; @@ -XXX,XX +XXX,XX @@ static bool trans_VCVT(DisasContext *s, arg_VCVT *a) return false; } - if (dp && !dc_isar_feature(aa32_fpdp_v2, s)) { + if (sz == 3 && !dc_isar_feature(aa32_fpdp_v2, s)) { + return false; + } + + if (sz == 1 && !dc_isar_feature(aa32_fp16_arith, s)) { return false; } /* UNDEF accesses to D16-D31 if they don't exist */ - if (dp && !dc_isar_feature(aa32_simd_r32, s) && (a->vm & 0x10)) { + if (sz == 3 && !dc_isar_feature(aa32_simd_r32, s) && (a->vm & 0x10)) { return false; } @@ -XXX,XX +XXX,XX @@ static bool trans_VCVT(DisasContext *s, arg_VCVT *a) return true; } - fpst = fpstatus_ptr(FPST_FPCR); + if (sz == 1) { + fpst = fpstatus_ptr(FPST_FPCR_F16); + } else { + fpst = fpstatus_ptr(FPST_FPCR); + } tcg_shift = tcg_const_i32(0); tcg_rmode = tcg_const_i32(arm_rmode_to_sf(rounding)); gen_helper_set_rmode(tcg_rmode, tcg_rmode, fpst); - if (dp) { + if (sz == 3) { TCGv_i64 tcg_double, tcg_res; TCGv_i32 tcg_tmp; tcg_double = tcg_temp_new_i64(); @@ -XXX,XX +XXX,XX @@ static bool trans_VCVT(DisasContext *s, arg_VCVT *a) tcg_single = tcg_temp_new_i32(); tcg_res = tcg_temp_new_i32(); neon_load_reg32(tcg_single, rm); - if (is_signed) { - gen_helper_vfp_tosls(tcg_res, tcg_single, tcg_shift, fpst); + if (sz == 1) { + if (is_signed) { + gen_helper_vfp_toslh(tcg_res, tcg_single, tcg_shift, fpst); + } else { + gen_helper_vfp_toulh(tcg_res, tcg_single, tcg_shift, fpst); + } } else { - gen_helper_vfp_touls(tcg_res, tcg_single, tcg_shift, fpst); + if (is_signed) { + gen_helper_vfp_tosls(tcg_res, tcg_single, tcg_shift, fpst); + } else { + gen_helper_vfp_touls(tcg_res, tcg_single, tcg_shift, fpst); + } } neon_store_reg32(tcg_res, rd); tcg_temp_free_i32(tcg_res); -- 2.20.1 Implement the fp16 versions of the VFP VSEL instruction. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-18-peter.maydell@linaro.org --- target/arm/vfp-uncond.decode | 6 ++++-- target/arm/translate-vfp.c.inc | 16 ++++++++++++---- 2 files changed, 16 insertions(+), 6 deletions(-) diff --git a/target/arm/vfp-uncond.decode b/target/arm/vfp-uncond.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vfp-uncond.decode +++ b/target/arm/vfp-uncond.decode @@ -XXX,XX +XXX,XX @@ @vfp_dnm_s ................................ vm=%vm_sp vn=%vn_sp vd=%vd_sp @vfp_dnm_d ................................ vm=%vm_dp vn=%vn_dp vd=%vd_dp +VSEL 1111 1110 0. cc:2 .... .... 1001 .0.0 .... \ + vm=%vm_sp vn=%vn_sp vd=%vd_sp sz=1 VSEL 1111 1110 0. cc:2 .... .... 1010 .0.0 .... \ - vm=%vm_sp vn=%vn_sp vd=%vd_sp dp=0 + vm=%vm_sp vn=%vn_sp vd=%vd_sp sz=2 VSEL 1111 1110 0. cc:2 .... .... 1011 .0.0 .... \ - vm=%vm_dp vn=%vn_dp vd=%vd_dp dp=1 + vm=%vm_dp vn=%vn_dp vd=%vd_dp sz=3 VMAXNM_hp 1111 1110 1.00 .... .... 1001 .0.0 .... @vfp_dnm_s VMINNM_hp 1111 1110 1.00 .... .... 1001 .1.0 .... @vfp_dnm_s diff --git a/target/arm/translate-vfp.c.inc b/target/arm/translate-vfp.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-vfp.c.inc +++ b/target/arm/translate-vfp.c.inc @@ -XXX,XX +XXX,XX @@ static bool vfp_access_check(DisasContext *s) static bool trans_VSEL(DisasContext *s, arg_VSEL *a) { uint32_t rd, rn, rm; - bool dp = a->dp; + int sz = a->sz; if (!dc_isar_feature(aa32_vsel, s)) { return false; } - if (dp && !dc_isar_feature(aa32_fpdp_v2, s)) { + if (sz == 3 && !dc_isar_feature(aa32_fpdp_v2, s)) { + return false; + } + + if (sz == 1 && !dc_isar_feature(aa32_fp16_arith, s)) { return false; } /* UNDEF accesses to D16-D31 if they don't exist */ - if (dp && !dc_isar_feature(aa32_simd_r32, s) && + if (sz == 3 && !dc_isar_feature(aa32_simd_r32, s) && ((a->vm | a->vn | a->vd) & 0x10)) { return false; } @@ -XXX,XX +XXX,XX @@ static bool trans_VSEL(DisasContext *s, arg_VSEL *a) return true; } - if (dp) { + if (sz == 3) { TCGv_i64 frn, frm, dest; TCGv_i64 tmp, zero, zf, nf, vf; @@ -XXX,XX +XXX,XX @@ static bool trans_VSEL(DisasContext *s, arg_VSEL *a) tcg_temp_free_i32(tmp); break; } + /* For fp16 the top half is always zeroes */ + if (sz == 1) { + tcg_gen_andi_i32(dest, dest, 0xffff); + } neon_store_reg32(dest, rd); tcg_temp_free_i32(frn); tcg_temp_free_i32(frm); -- 2.20.1 Implement the fp16 version of the VFP VRINT* insns. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-19-peter.maydell@linaro.org --- target/arm/helper.h | 2 + target/arm/vfp-uncond.decode | 6 ++- target/arm/vfp.decode | 3 ++ target/arm/vfp_helper.c | 21 ++++++++ target/arm/translate-vfp.c.inc | 98 +++++++++++++++++++++++++++++++--- 5 files changed, 122 insertions(+), 8 deletions(-) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_3(shr_cc, i32, env, i32, i32) DEF_HELPER_3(sar_cc, i32, env, i32, i32) DEF_HELPER_3(ror_cc, i32, env, i32, i32) +DEF_HELPER_FLAGS_2(rinth_exact, TCG_CALL_NO_RWG, f16, f16, ptr) DEF_HELPER_FLAGS_2(rints_exact, TCG_CALL_NO_RWG, f32, f32, ptr) DEF_HELPER_FLAGS_2(rintd_exact, TCG_CALL_NO_RWG, f64, f64, ptr) +DEF_HELPER_FLAGS_2(rinth, TCG_CALL_NO_RWG, f16, f16, ptr) DEF_HELPER_FLAGS_2(rints, TCG_CALL_NO_RWG, f32, f32, ptr) DEF_HELPER_FLAGS_2(rintd, TCG_CALL_NO_RWG, f64, f64, ptr) diff --git a/target/arm/vfp-uncond.decode b/target/arm/vfp-uncond.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vfp-uncond.decode +++ b/target/arm/vfp-uncond.decode @@ -XXX,XX +XXX,XX @@ VMINNM_sp 1111 1110 1.00 .... .... 1010 .1.0 .... @vfp_dnm_s VMAXNM_dp 1111 1110 1.00 .... .... 1011 .0.0 .... @vfp_dnm_d VMINNM_dp 1111 1110 1.00 .... .... 1011 .1.0 .... @vfp_dnm_d +VRINT 1111 1110 1.11 10 rm:2 .... 1001 01.0 .... \ + vm=%vm_sp vd=%vd_sp sz=1 VRINT 1111 1110 1.11 10 rm:2 .... 1010 01.0 .... \ - vm=%vm_sp vd=%vd_sp dp=0 + vm=%vm_sp vd=%vd_sp sz=2 VRINT 1111 1110 1.11 10 rm:2 .... 1011 01.0 .... \ - vm=%vm_dp vd=%vd_dp dp=1 + vm=%vm_dp vd=%vd_dp sz=3 # VCVT float to int with specified rounding mode; Vd is always single-precision VCVT 1111 1110 1.11 11 rm:2 .... 1001 op:1 1.0 .... \ diff --git a/target/arm/vfp.decode b/target/arm/vfp.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vfp.decode +++ b/target/arm/vfp.decode @@ -XXX,XX +XXX,XX @@ VCVT_f16_f32 ---- 1110 1.11 0011 .... 1010 t:1 1.0 .... \ VCVT_f16_f64 ---- 1110 1.11 0011 .... 1011 t:1 1.0 .... \ vd=%vd_sp vm=%vm_dp +VRINTR_hp ---- 1110 1.11 0110 .... 1001 01.0 .... @vfp_dm_ss VRINTR_sp ---- 1110 1.11 0110 .... 1010 01.0 .... @vfp_dm_ss VRINTR_dp ---- 1110 1.11 0110 .... 1011 01.0 .... @vfp_dm_dd +VRINTZ_hp ---- 1110 1.11 0110 .... 1001 11.0 .... @vfp_dm_ss VRINTZ_sp ---- 1110 1.11 0110 .... 1010 11.0 .... @vfp_dm_ss VRINTZ_dp ---- 1110 1.11 0110 .... 1011 11.0 .... @vfp_dm_dd +VRINTX_hp ---- 1110 1.11 0111 .... 1001 01.0 .... @vfp_dm_ss VRINTX_sp ---- 1110 1.11 0111 .... 1010 01.0 .... @vfp_dm_ss VRINTX_dp ---- 1110 1.11 0111 .... 1011 01.0 .... @vfp_dm_dd diff --git a/target/arm/vfp_helper.c b/target/arm/vfp_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vfp_helper.c +++ b/target/arm/vfp_helper.c @@ -XXX,XX +XXX,XX @@ float64 VFP_HELPER(muladd, d)(float64 a, float64 b, float64 c, void *fpstp) } /* ARMv8 round to integral */ +dh_ctype_f16 HELPER(rinth_exact)(dh_ctype_f16 x, void *fp_status) +{ + return float16_round_to_int(x, fp_status); +} + float32 HELPER(rints_exact)(float32 x, void *fp_status) { return float32_round_to_int(x, fp_status); @@ -XXX,XX +XXX,XX @@ float64 HELPER(rintd_exact)(float64 x, void *fp_status) return float64_round_to_int(x, fp_status); } +dh_ctype_f16 HELPER(rinth)(dh_ctype_f16 x, void *fp_status) +{ + int old_flags = get_float_exception_flags(fp_status), new_flags; + float16 ret; + + ret = float16_round_to_int(x, fp_status); + + /* Suppress any inexact exceptions the conversion produced */ + if (!(old_flags & float_flag_inexact)) { + new_flags = get_float_exception_flags(fp_status); + set_float_exception_flags(new_flags & ~float_flag_inexact, fp_status); + } + + return ret; +} + float32 HELPER(rints)(float32 x, void *fp_status) { int old_flags = get_float_exception_flags(fp_status), new_flags; diff --git a/target/arm/translate-vfp.c.inc b/target/arm/translate-vfp.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-vfp.c.inc +++ b/target/arm/translate-vfp.c.inc @@ -XXX,XX +XXX,XX @@ static const uint8_t fp_decode_rm[] = { static bool trans_VRINT(DisasContext *s, arg_VRINT *a) { uint32_t rd, rm; - bool dp = a->dp; + int sz = a->sz; TCGv_ptr fpst; TCGv_i32 tcg_rmode; int rounding = fp_decode_rm[a->rm]; @@ -XXX,XX +XXX,XX @@ static bool trans_VRINT(DisasContext *s, arg_VRINT *a) return false; } - if (dp && !dc_isar_feature(aa32_fpdp_v2, s)) { + if (sz == 3 && !dc_isar_feature(aa32_fpdp_v2, s)) { + return false; + } + + if (sz == 1 && !dc_isar_feature(aa32_fp16_arith, s)) { return false; } /* UNDEF accesses to D16-D31 if they don't exist */ - if (dp && !dc_isar_feature(aa32_simd_r32, s) && + if (sz == 3 && !dc_isar_feature(aa32_simd_r32, s) && ((a->vm | a->vd) & 0x10)) { return false; } @@ -XXX,XX +XXX,XX @@ static bool trans_VRINT(DisasContext *s, arg_VRINT *a) return true; } - fpst = fpstatus_ptr(FPST_FPCR); + if (sz == 1) { + fpst = fpstatus_ptr(FPST_FPCR_F16); + } else { + fpst = fpstatus_ptr(FPST_FPCR); + } tcg_rmode = tcg_const_i32(arm_rmode_to_sf(rounding)); gen_helper_set_rmode(tcg_rmode, tcg_rmode, fpst); - if (dp) { + if (sz == 3) { TCGv_i64 tcg_op; TCGv_i64 tcg_res; tcg_op = tcg_temp_new_i64(); @@ -XXX,XX +XXX,XX @@ static bool trans_VRINT(DisasContext *s, arg_VRINT *a) tcg_op = tcg_temp_new_i32(); tcg_res = tcg_temp_new_i32(); neon_load_reg32(tcg_op, rm); - gen_helper_rints(tcg_res, tcg_op, fpst); + if (sz == 1) { + gen_helper_rinth(tcg_res, tcg_op, fpst); + } else { + gen_helper_rints(tcg_res, tcg_op, fpst); + } neon_store_reg32(tcg_res, rd); tcg_temp_free_i32(tcg_op); tcg_temp_free_i32(tcg_res); @@ -XXX,XX +XXX,XX @@ static bool trans_VCVT_f16_f64(DisasContext *s, arg_VCVT_f16_f64 *a) return true; } +static bool trans_VRINTR_hp(DisasContext *s, arg_VRINTR_sp *a) +{ + TCGv_ptr fpst; + TCGv_i32 tmp; + + if (!dc_isar_feature(aa32_fp16_arith, s)) { + return false; + } + + if (!vfp_access_check(s)) { + return true; + } + + tmp = tcg_temp_new_i32(); + neon_load_reg32(tmp, a->vm); + fpst = fpstatus_ptr(FPST_FPCR_F16); + gen_helper_rinth(tmp, tmp, fpst); + neon_store_reg32(tmp, a->vd); + tcg_temp_free_ptr(fpst); + tcg_temp_free_i32(tmp); + return true; +} + static bool trans_VRINTR_sp(DisasContext *s, arg_VRINTR_sp *a) { TCGv_ptr fpst; @@ -XXX,XX +XXX,XX @@ static bool trans_VRINTR_dp(DisasContext *s, arg_VRINTR_dp *a) return true; } +static bool trans_VRINTZ_hp(DisasContext *s, arg_VRINTZ_sp *a) +{ + TCGv_ptr fpst; + TCGv_i32 tmp; + TCGv_i32 tcg_rmode; + + if (!dc_isar_feature(aa32_fp16_arith, s)) { + return false; + } + + if (!vfp_access_check(s)) { + return true; + } + + tmp = tcg_temp_new_i32(); + neon_load_reg32(tmp, a->vm); + fpst = fpstatus_ptr(FPST_FPCR_F16); + tcg_rmode = tcg_const_i32(float_round_to_zero); + gen_helper_set_rmode(tcg_rmode, tcg_rmode, fpst); + gen_helper_rinth(tmp, tmp, fpst); + gen_helper_set_rmode(tcg_rmode, tcg_rmode, fpst); + neon_store_reg32(tmp, a->vd); + tcg_temp_free_ptr(fpst); + tcg_temp_free_i32(tcg_rmode); + tcg_temp_free_i32(tmp); + return true; +} + static bool trans_VRINTZ_sp(DisasContext *s, arg_VRINTZ_sp *a) { TCGv_ptr fpst; @@ -XXX,XX +XXX,XX @@ static bool trans_VRINTZ_dp(DisasContext *s, arg_VRINTZ_dp *a) return true; } +static bool trans_VRINTX_hp(DisasContext *s, arg_VRINTX_sp *a) +{ + TCGv_ptr fpst; + TCGv_i32 tmp; + + if (!dc_isar_feature(aa32_fp16_arith, s)) { + return false; + } + + if (!vfp_access_check(s)) { + return true; + } + + tmp = tcg_temp_new_i32(); + neon_load_reg32(tmp, a->vm); + fpst = fpstatus_ptr(FPST_FPCR_F16); + gen_helper_rinth_exact(tmp, tmp, fpst); + neon_store_reg32(tmp, a->vd); + tcg_temp_free_ptr(fpst); + tcg_temp_free_i32(tmp); + return true; +} + static bool trans_VRINTX_sp(DisasContext *s, arg_VRINTX_sp *a) { TCGv_ptr fpst; -- 2.20.1 The fp16 extension includes a new instruction VINS, which copies the lower 16 bits of a 32-bit source VFP register into the upper 16 bits of the destination. Implement it. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-20-peter.maydell@linaro.org --- target/arm/vfp-uncond.decode | 3 +++ target/arm/translate-vfp.c.inc | 28 ++++++++++++++++++++++++++++ 2 files changed, 31 insertions(+) diff --git a/target/arm/vfp-uncond.decode b/target/arm/vfp-uncond.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vfp-uncond.decode +++ b/target/arm/vfp-uncond.decode @@ -XXX,XX +XXX,XX @@ VCVT 1111 1110 1.11 11 rm:2 .... 1010 op:1 1.0 .... \ vm=%vm_sp vd=%vd_sp sz=2 VCVT 1111 1110 1.11 11 rm:2 .... 1011 op:1 1.0 .... \ vm=%vm_dp vd=%vd_sp sz=3 + +VINS 1111 1110 1.11 0000 .... 1010 11 . 0 .... \ + vd=%vd_sp vm=%vm_sp diff --git a/target/arm/translate-vfp.c.inc b/target/arm/translate-vfp.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-vfp.c.inc +++ b/target/arm/translate-vfp.c.inc @@ -XXX,XX +XXX,XX @@ static bool trans_NOCP(DisasContext *s, arg_NOCP *a) return false; } + +static bool trans_VINS(DisasContext *s, arg_VINS *a) +{ + TCGv_i32 rd, rm; + + if (!dc_isar_feature(aa32_fp16_arith, s)) { + return false; + } + + if (s->vec_len != 0 || s->vec_stride != 0) { + return false; + } + + if (!vfp_access_check(s)) { + return true; + } + + /* Insert low half of Vm into high half of Vd */ + rm = tcg_temp_new_i32(); + rd = tcg_temp_new_i32(); + neon_load_reg32(rm, a->vm); + neon_load_reg32(rd, a->vd); + tcg_gen_deposit_i32(rd, rd, rm, 16, 16); + neon_store_reg32(rd, a->vd); + tcg_temp_free_i32(rm); + tcg_temp_free_i32(rd); + return true; +} -- 2.20.1 The fp16 extension includes a new instruction VMOVX, which copies the upper 16 bits of a 32-bit source VFP register into the lower 16 bits of the destination and zeroes the high half of the destination. Implement it. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-21-peter.maydell@linaro.org --- target/arm/vfp-uncond.decode | 3 +++ target/arm/translate-vfp.c.inc | 25 +++++++++++++++++++++++++ 2 files changed, 28 insertions(+) diff --git a/target/arm/vfp-uncond.decode b/target/arm/vfp-uncond.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vfp-uncond.decode +++ b/target/arm/vfp-uncond.decode @@ -XXX,XX +XXX,XX @@ VCVT 1111 1110 1.11 11 rm:2 .... 1010 op:1 1.0 .... \ VCVT 1111 1110 1.11 11 rm:2 .... 1011 op:1 1.0 .... \ vm=%vm_dp vd=%vd_sp sz=3 +VMOVX 1111 1110 1.11 0000 .... 1010 01 . 0 .... \ + vd=%vd_sp vm=%vm_sp + VINS 1111 1110 1.11 0000 .... 1010 11 . 0 .... \ vd=%vd_sp vm=%vm_sp diff --git a/target/arm/translate-vfp.c.inc b/target/arm/translate-vfp.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-vfp.c.inc +++ b/target/arm/translate-vfp.c.inc @@ -XXX,XX +XXX,XX @@ static bool trans_VINS(DisasContext *s, arg_VINS *a) tcg_temp_free_i32(rd); return true; } + +static bool trans_VMOVX(DisasContext *s, arg_VINS *a) +{ + TCGv_i32 rm; + + if (!dc_isar_feature(aa32_fp16_arith, s)) { + return false; + } + + if (s->vec_len != 0 || s->vec_stride != 0) { + return false; + } + + if (!vfp_access_check(s)) { + return true; + } + + /* Set Vd to high half of Vm */ + rm = tcg_temp_new_i32(); + neon_load_reg32(rm, a->vm); + tcg_gen_shri_i32(rm, rm, 16); + neon_store_reg32(rm, a->vd); + tcg_temp_free_i32(rm); + return true; +} -- 2.20.1 Implement the VFP fp16 variant of VMOV that transfers a 16-bit value between a general purpose register and a VFP register. Note that Rt == 15 is UNPREDICTABLE; since this insn is v8 and later only we have no need to replicate the old "updates CPSR.NZCV" behaviour that the singleprec version of this insn does. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-22-peter.maydell@linaro.org --- target/arm/vfp.decode | 1 + target/arm/translate-vfp.c.inc | 34 ++++++++++++++++++++++++++++++++++ 2 files changed, 35 insertions(+) diff --git a/target/arm/vfp.decode b/target/arm/vfp.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vfp.decode +++ b/target/arm/vfp.decode @@ -XXX,XX +XXX,XX @@ VDUP ---- 1110 1 b:1 q:1 0 .... rt:4 1011 . 0 e:1 1 0000 \ vn=%vn_dp VMSR_VMRS ---- 1110 111 l:1 reg:4 rt:4 1010 0001 0000 +VMOV_half ---- 1110 000 l:1 .... rt:4 1001 . 001 0000 vn=%vn_sp VMOV_single ---- 1110 000 l:1 .... rt:4 1010 . 001 0000 vn=%vn_sp VMOV_64_sp ---- 1100 010 op:1 rt2:4 rt:4 1010 00.1 .... vm=%vm_sp diff --git a/target/arm/translate-vfp.c.inc b/target/arm/translate-vfp.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-vfp.c.inc +++ b/target/arm/translate-vfp.c.inc @@ -XXX,XX +XXX,XX @@ static bool trans_VMSR_VMRS(DisasContext *s, arg_VMSR_VMRS *a) return true; } +static bool trans_VMOV_half(DisasContext *s, arg_VMOV_single *a) +{ + TCGv_i32 tmp; + + if (!dc_isar_feature(aa32_fp16_arith, s)) { + return false; + } + + if (a->rt == 15) { + /* UNPREDICTABLE; we choose to UNDEF */ + return false; + } + + if (!vfp_access_check(s)) { + return true; + } + + if (a->l) { + /* VFP to general purpose register */ + tmp = tcg_temp_new_i32(); + neon_load_reg32(tmp, a->vn); + tcg_gen_andi_i32(tmp, tmp, 0xffff); + store_reg(s, a->rt, tmp); + } else { + /* general purpose register to VFP */ + tmp = load_reg(s, a->rt); + tcg_gen_andi_i32(tmp, tmp, 0xffff); + neon_store_reg32(tmp, a->vn); + tcg_temp_free_i32(tmp); + } + + return true; +} + static bool trans_VMOV_single(DisasContext *s, arg_VMOV_single *a) { TCGv_i32 tmp; -- 2.20.1 Implement FP16 support for the Neon insns which use the DO_3S_FP_GVEC macro: VADD, VSUB, VABD, VMUL. For VABD this requires us to implement a new gvec_fabd_h helper using the machinery we have already for the other helpers. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-24-peter.maydell@linaro.org --- target/arm/helper.h | 1 + target/arm/vec_helper.c | 6 ++++++ target/arm/translate-neon.c.inc | 36 +++++++++++++++++---------------- 3 files changed, 26 insertions(+), 17 deletions(-) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_5(gvec_fmul_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_fmul_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_fmul_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_fabd_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_fabd_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_ftsmul_h, TCG_CALL_NO_RWG, diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -XXX,XX +XXX,XX @@ static float64 float64_ftsmul(float64 op1, uint64_t op2, float_status *stat) return result; } +static float16 float16_abd(float16 op1, float16 op2, float_status *stat) +{ + return float16_abs(float16_sub(op1, op2, stat)); +} + static float32 float32_abd(float32 op1, float32 op2, float_status *stat) { return float32_abs(float32_sub(op1, op2, stat)); @@ -XXX,XX +XXX,XX @@ DO_3OP(gvec_ftsmul_h, float16_ftsmul, float16) DO_3OP(gvec_ftsmul_s, float32_ftsmul, float32) DO_3OP(gvec_ftsmul_d, float64_ftsmul, float64) +DO_3OP(gvec_fabd_h, float16_abd, float16) DO_3OP(gvec_fabd_s, float32_abd, float32) #ifdef TARGET_AARCH64 diff --git a/target/arm/translate-neon.c.inc b/target/arm/translate-neon.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-neon.c.inc +++ b/target/arm/translate-neon.c.inc @@ -XXX,XX +XXX,XX @@ static bool do_3same_fp(DisasContext *s, arg_3same *a, VFPGen3OpSPFn *fn, return true; } -/* - * For all the functions using this macro, size == 1 means fp16, - * which is an architecture extension we don't implement yet. - */ -#define DO_3S_FP_GVEC(INSN,FUNC) \ - static void gen_##INSN##_3s(unsigned vece, uint32_t rd_ofs, \ - uint32_t rn_ofs, uint32_t rm_ofs, \ - uint32_t oprsz, uint32_t maxsz) \ +#define WRAP_FP_GVEC(WRAPNAME, FPST, FUNC) \ + static void WRAPNAME(unsigned vece, uint32_t rd_ofs, \ + uint32_t rn_ofs, uint32_t rm_ofs, \ + uint32_t oprsz, uint32_t maxsz) \ { \ - TCGv_ptr fpst = fpstatus_ptr(FPST_STD); \ + TCGv_ptr fpst = fpstatus_ptr(FPST); \ tcg_gen_gvec_3_ptr(rd_ofs, rn_ofs, rm_ofs, fpst, \ oprsz, maxsz, 0, FUNC); \ tcg_temp_free_ptr(fpst); \ - } \ + } + +#define DO_3S_FP_GVEC(INSN,SFUNC,HFUNC) \ + WRAP_FP_GVEC(gen_##INSN##_fp32_3s, FPST_STD, SFUNC) \ + WRAP_FP_GVEC(gen_##INSN##_fp16_3s, FPST_STD_F16, HFUNC) \ static bool trans_##INSN##_fp_3s(DisasContext *s, arg_3same *a) \ { \ if (a->size != 0) { \ - /* TODO fp16 support */ \ - return false; \ + if (!dc_isar_feature(aa32_fp16_arith, s)) { \ + return false; \ + } \ + return do_3same(s, a, gen_##INSN##_fp16_3s); \ } \ - return do_3same(s, a, gen_##INSN##_3s); \ + return do_3same(s, a, gen_##INSN##_fp32_3s); \ } -DO_3S_FP_GVEC(VADD, gen_helper_gvec_fadd_s) -DO_3S_FP_GVEC(VSUB, gen_helper_gvec_fsub_s) -DO_3S_FP_GVEC(VABD, gen_helper_gvec_fabd_s) -DO_3S_FP_GVEC(VMUL, gen_helper_gvec_fmul_s) +DO_3S_FP_GVEC(VADD, gen_helper_gvec_fadd_s, gen_helper_gvec_fadd_h) +DO_3S_FP_GVEC(VSUB, gen_helper_gvec_fsub_s, gen_helper_gvec_fsub_h) +DO_3S_FP_GVEC(VABD, gen_helper_gvec_fabd_s, gen_helper_gvec_fabd_h) +DO_3S_FP_GVEC(VMUL, gen_helper_gvec_fmul_s, gen_helper_gvec_fmul_h) /* * For all the functions using this macro, size == 1 means fp16, -- 2.20.1 We already have gvec helpers for floating point VRECPE and VRQSRTE, so convert the Neon decoder to use them and add the fp16 support. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-25-peter.maydell@linaro.org --- target/arm/translate-neon.c.inc | 31 +++++++++++++++++++++++++++++-- 1 file changed, 29 insertions(+), 2 deletions(-) diff --git a/target/arm/translate-neon.c.inc b/target/arm/translate-neon.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-neon.c.inc +++ b/target/arm/translate-neon.c.inc @@ -XXX,XX +XXX,XX @@ static bool do_2misc_fp(DisasContext *s, arg_2misc *a, return do_2misc_fp(s, a, FUNC); \ } -DO_2MISC_FP(VRECPE_F, gen_helper_recpe_f32) -DO_2MISC_FP(VRSQRTE_F, gen_helper_rsqrte_f32) DO_2MISC_FP(VCVT_FS, gen_helper_vfp_sitos) DO_2MISC_FP(VCVT_FU, gen_helper_vfp_uitos) DO_2MISC_FP(VCVT_SF, gen_helper_vfp_tosizs) DO_2MISC_FP(VCVT_UF, gen_helper_vfp_touizs) +#define DO_2MISC_FP_VEC(INSN, HFUNC, SFUNC) \ + static void gen_##INSN(unsigned vece, uint32_t rd_ofs, \ + uint32_t rm_ofs, \ + uint32_t oprsz, uint32_t maxsz) \ + { \ + static gen_helper_gvec_2_ptr * const fns[4] = { \ + NULL, HFUNC, SFUNC, NULL, \ + }; \ + TCGv_ptr fpst; \ + fpst = fpstatus_ptr(vece == MO_16 ? FPST_STD_F16 : FPST_STD); \ + tcg_gen_gvec_2_ptr(rd_ofs, rm_ofs, fpst, oprsz, maxsz, 0, \ + fns[vece]); \ + tcg_temp_free_ptr(fpst); \ + } \ + static bool trans_##INSN(DisasContext *s, arg_2misc *a) \ + { \ + if (a->size == MO_16) { \ + if (!dc_isar_feature(aa32_fp16_arith, s)) { \ + return false; \ + } \ + } else if (a->size != MO_32) { \ + return false; \ + } \ + return do_2misc_vec(s, a, gen_##INSN); \ + } + +DO_2MISC_FP_VEC(VRECPE_F, gen_helper_gvec_frecpe_h, gen_helper_gvec_frecpe_s) +DO_2MISC_FP_VEC(VRSQRTE_F, gen_helper_gvec_frsqrte_h, gen_helper_gvec_frsqrte_s) + static bool trans_VRINTX(DisasContext *s, arg_2misc *a) { if (!arm_dc_feature(s, ARM_FEATURE_V8)) { -- 2.20.1 Rewrite Neon VABS/VNEG of floats to use gvec logical AND and XOR, so that we can implement the fp16 version of the insns. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-26-peter.maydell@linaro.org --- target/arm/translate-neon.c.inc | 34 +++++++++++++++++++++++++++------ 1 file changed, 28 insertions(+), 6 deletions(-) diff --git a/target/arm/translate-neon.c.inc b/target/arm/translate-neon.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-neon.c.inc +++ b/target/arm/translate-neon.c.inc @@ -XXX,XX +XXX,XX @@ static bool trans_VCNT(DisasContext *s, arg_2misc *a) return do_2misc(s, a, gen_helper_neon_cnt_u8); } +static void gen_VABS_F(unsigned vece, uint32_t rd_ofs, uint32_t rm_ofs, + uint32_t oprsz, uint32_t maxsz) +{ + tcg_gen_gvec_andi(vece, rd_ofs, rm_ofs, + vece == MO_16 ? 0x7fff : 0x7fffffff, + oprsz, maxsz); +} + static bool trans_VABS_F(DisasContext *s, arg_2misc *a) { - if (a->size != 2) { + if (a->size == MO_16) { + if (!dc_isar_feature(aa32_fp16_arith, s)) { + return false; + } + } else if (a->size != MO_32) { return false; } - /* TODO: FP16 : size == 1 */ - return do_2misc(s, a, gen_helper_vfp_abss); + return do_2misc_vec(s, a, gen_VABS_F); +} + +static void gen_VNEG_F(unsigned vece, uint32_t rd_ofs, uint32_t rm_ofs, + uint32_t oprsz, uint32_t maxsz) +{ + tcg_gen_gvec_xori(vece, rd_ofs, rm_ofs, + vece == MO_16 ? 0x8000 : 0x80000000, + oprsz, maxsz); } static bool trans_VNEG_F(DisasContext *s, arg_2misc *a) { - if (a->size != 2) { + if (a->size == MO_16) { + if (!dc_isar_feature(aa32_fp16_arith, s)) { + return false; + } + } else if (a->size != MO_32) { return false; } - /* TODO: FP16 : size == 1 */ - return do_2misc(s, a, gen_helper_vfp_negs); + return do_2misc_vec(s, a, gen_VNEG_F); } static bool trans_VRECPE(DisasContext *s, arg_2misc *a) -- 2.20.1 Convert the Neon floating-point vector comparison ops VCEQ, VCGE and VCGT over to using a gvec helper and use this to implement the fp16 case. (We put the float16_ceq() etc functions above the DO_2OP() macro definition because later when we convert the compare-against-zero instructions we'll want their definitions to be visible at that point in the source file.) Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-27-peter.maydell@linaro.org --- target/arm/helper.h | 9 +++++++ target/arm/vec_helper.c | 44 +++++++++++++++++++++++++++++++++ target/arm/translate-neon.c.inc | 6 ++--- 3 files changed, 56 insertions(+), 3 deletions(-) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_5(gvec_fmul_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_fabd_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_fabd_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_fceq_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_fceq_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(gvec_fcge_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_fcge_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(gvec_fcgt_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_fcgt_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_5(gvec_ftsmul_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_ftsmul_s, TCG_CALL_NO_RWG, diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -XXX,XX +XXX,XX @@ void HELPER(gvec_fcmlad)(void *vd, void *vn, void *vm, clear_tail(d, opr_sz, simd_maxsz(desc)); } +/* + * Floating point comparisons producing an integer result (all 1s or all 0s). + * Note that EQ doesn't signal InvalidOp for QNaNs but GE and GT do. + * Softfloat routines return 0/1, which we convert to the 0/-1 Neon requires. + */ +static uint16_t float16_ceq(float16 op1, float16 op2, float_status *stat) +{ + return -float16_eq_quiet(op1, op2, stat); +} + +static uint32_t float32_ceq(float32 op1, float32 op2, float_status *stat) +{ + return -float32_eq_quiet(op1, op2, stat); +} + +static uint16_t float16_cge(float16 op1, float16 op2, float_status *stat) +{ + return -float16_le(op2, op1, stat); +} + +static uint32_t float32_cge(float32 op1, float32 op2, float_status *stat) +{ + return -float32_le(op2, op1, stat); +} + +static uint16_t float16_cgt(float16 op1, float16 op2, float_status *stat) +{ + return -float16_lt(op2, op1, stat); +} + +static uint32_t float32_cgt(float32 op1, float32 op2, float_status *stat) +{ + return -float32_lt(op2, op1, stat); +} + #define DO_2OP(NAME, FUNC, TYPE) \ void HELPER(NAME)(void *vd, void *vn, void *stat, uint32_t desc) \ { \ @@ -XXX,XX +XXX,XX @@ DO_3OP(gvec_ftsmul_d, float64_ftsmul, float64) DO_3OP(gvec_fabd_h, float16_abd, float16) DO_3OP(gvec_fabd_s, float32_abd, float32) +DO_3OP(gvec_fceq_h, float16_ceq, float16) +DO_3OP(gvec_fceq_s, float32_ceq, float32) + +DO_3OP(gvec_fcge_h, float16_cge, float16) +DO_3OP(gvec_fcge_s, float32_cge, float32) + +DO_3OP(gvec_fcgt_h, float16_cgt, float16) +DO_3OP(gvec_fcgt_s, float32_cgt, float32) + #ifdef TARGET_AARCH64 DO_3OP(gvec_recps_h, helper_recpsf_f16, float16) diff --git a/target/arm/translate-neon.c.inc b/target/arm/translate-neon.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-neon.c.inc +++ b/target/arm/translate-neon.c.inc @@ -XXX,XX +XXX,XX @@ DO_3S_FP_GVEC(VADD, gen_helper_gvec_fadd_s, gen_helper_gvec_fadd_h) DO_3S_FP_GVEC(VSUB, gen_helper_gvec_fsub_s, gen_helper_gvec_fsub_h) DO_3S_FP_GVEC(VABD, gen_helper_gvec_fabd_s, gen_helper_gvec_fabd_h) DO_3S_FP_GVEC(VMUL, gen_helper_gvec_fmul_s, gen_helper_gvec_fmul_h) +DO_3S_FP_GVEC(VCEQ, gen_helper_gvec_fceq_s, gen_helper_gvec_fceq_h) +DO_3S_FP_GVEC(VCGE, gen_helper_gvec_fcge_s, gen_helper_gvec_fcge_h) +DO_3S_FP_GVEC(VCGT, gen_helper_gvec_fcgt_s, gen_helper_gvec_fcgt_h) /* * For all the functions using this macro, size == 1 means fp16, @@ -XXX,XX +XXX,XX @@ DO_3S_FP_GVEC(VMUL, gen_helper_gvec_fmul_s, gen_helper_gvec_fmul_h) return do_3same_fp(s, a, FUNC, READS_VD); \ } -DO_3S_FP(VCEQ, gen_helper_neon_ceq_f32, false) -DO_3S_FP(VCGE, gen_helper_neon_cge_f32, false) -DO_3S_FP(VCGT, gen_helper_neon_cgt_f32, false) DO_3S_FP(VACGE, gen_helper_neon_acge_f32, false) DO_3S_FP(VACGT, gen_helper_neon_acgt_f32, false) DO_3S_FP(VMAX, gen_helper_vfp_maxs, false) -- 2.20.1 Convert the neon floating-point vector absolute comparison ops VACGE and VACGT over to using a gvec hepler and use this to implement the fp16 case. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-28-peter.maydell@linaro.org --- target/arm/helper.h | 6 ++++++ target/arm/vec_helper.c | 26 ++++++++++++++++++++++++++ target/arm/translate-neon.c.inc | 4 ++-- 3 files changed, 34 insertions(+), 2 deletions(-) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_5(gvec_fcge_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_fcgt_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_fcgt_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_facge_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_facge_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(gvec_facgt_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_facgt_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_5(gvec_ftsmul_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_ftsmul_s, TCG_CALL_NO_RWG, diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -XXX,XX +XXX,XX @@ static uint32_t float32_cgt(float32 op1, float32 op2, float_status *stat) return -float32_lt(op2, op1, stat); } +static uint16_t float16_acge(float16 op1, float16 op2, float_status *stat) +{ + return -float16_le(float16_abs(op2), float16_abs(op1), stat); +} + +static uint32_t float32_acge(float32 op1, float32 op2, float_status *stat) +{ + return -float32_le(float32_abs(op2), float32_abs(op1), stat); +} + +static uint16_t float16_acgt(float16 op1, float16 op2, float_status *stat) +{ + return -float16_lt(float16_abs(op2), float16_abs(op1), stat); +} + +static uint32_t float32_acgt(float32 op1, float32 op2, float_status *stat) +{ + return -float32_lt(float32_abs(op2), float32_abs(op1), stat); +} + #define DO_2OP(NAME, FUNC, TYPE) \ void HELPER(NAME)(void *vd, void *vn, void *stat, uint32_t desc) \ { \ @@ -XXX,XX +XXX,XX @@ DO_3OP(gvec_fcge_s, float32_cge, float32) DO_3OP(gvec_fcgt_h, float16_cgt, float16) DO_3OP(gvec_fcgt_s, float32_cgt, float32) +DO_3OP(gvec_facge_h, float16_acge, float16) +DO_3OP(gvec_facge_s, float32_acge, float32) + +DO_3OP(gvec_facgt_h, float16_acgt, float16) +DO_3OP(gvec_facgt_s, float32_acgt, float32) + #ifdef TARGET_AARCH64 DO_3OP(gvec_recps_h, helper_recpsf_f16, float16) diff --git a/target/arm/translate-neon.c.inc b/target/arm/translate-neon.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-neon.c.inc +++ b/target/arm/translate-neon.c.inc @@ -XXX,XX +XXX,XX @@ DO_3S_FP_GVEC(VMUL, gen_helper_gvec_fmul_s, gen_helper_gvec_fmul_h) DO_3S_FP_GVEC(VCEQ, gen_helper_gvec_fceq_s, gen_helper_gvec_fceq_h) DO_3S_FP_GVEC(VCGE, gen_helper_gvec_fcge_s, gen_helper_gvec_fcge_h) DO_3S_FP_GVEC(VCGT, gen_helper_gvec_fcgt_s, gen_helper_gvec_fcgt_h) +DO_3S_FP_GVEC(VACGE, gen_helper_gvec_facge_s, gen_helper_gvec_facge_h) +DO_3S_FP_GVEC(VACGT, gen_helper_gvec_facgt_s, gen_helper_gvec_facgt_h) /* * For all the functions using this macro, size == 1 means fp16, @@ -XXX,XX +XXX,XX @@ DO_3S_FP_GVEC(VCGT, gen_helper_gvec_fcgt_s, gen_helper_gvec_fcgt_h) return do_3same_fp(s, a, FUNC, READS_VD); \ } -DO_3S_FP(VACGE, gen_helper_neon_acge_f32, false) -DO_3S_FP(VACGT, gen_helper_neon_acgt_f32, false) DO_3S_FP(VMAX, gen_helper_vfp_maxs, false) DO_3S_FP(VMIN, gen_helper_vfp_mins, false) -- 2.20.1 Convert the Neon float-point VMAX and VMIN insns over to using a gvec helper, and use this to implement the fp16 case. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-29-peter.maydell@linaro.org --- target/arm/helper.h | 6 ++++++ target/arm/vec_helper.c | 6 ++++++ target/arm/translate-neon.c.inc | 5 ++--- 3 files changed, 14 insertions(+), 3 deletions(-) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_5(gvec_facge_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_facgt_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_facgt_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_fmax_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_fmax_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(gvec_fmin_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_fmin_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_5(gvec_ftsmul_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_ftsmul_s, TCG_CALL_NO_RWG, diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -XXX,XX +XXX,XX @@ DO_3OP(gvec_facge_s, float32_acge, float32) DO_3OP(gvec_facgt_h, float16_acgt, float16) DO_3OP(gvec_facgt_s, float32_acgt, float32) +DO_3OP(gvec_fmax_h, float16_max, float16) +DO_3OP(gvec_fmax_s, float32_max, float32) + +DO_3OP(gvec_fmin_h, float16_min, float16) +DO_3OP(gvec_fmin_s, float32_min, float32) + #ifdef TARGET_AARCH64 DO_3OP(gvec_recps_h, helper_recpsf_f16, float16) diff --git a/target/arm/translate-neon.c.inc b/target/arm/translate-neon.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-neon.c.inc +++ b/target/arm/translate-neon.c.inc @@ -XXX,XX +XXX,XX @@ DO_3S_FP_GVEC(VCGE, gen_helper_gvec_fcge_s, gen_helper_gvec_fcge_h) DO_3S_FP_GVEC(VCGT, gen_helper_gvec_fcgt_s, gen_helper_gvec_fcgt_h) DO_3S_FP_GVEC(VACGE, gen_helper_gvec_facge_s, gen_helper_gvec_facge_h) DO_3S_FP_GVEC(VACGT, gen_helper_gvec_facgt_s, gen_helper_gvec_facgt_h) +DO_3S_FP_GVEC(VMAX, gen_helper_gvec_fmax_s, gen_helper_gvec_fmax_h) +DO_3S_FP_GVEC(VMIN, gen_helper_gvec_fmin_s, gen_helper_gvec_fmin_h) /* * For all the functions using this macro, size == 1 means fp16, @@ -XXX,XX +XXX,XX @@ DO_3S_FP_GVEC(VACGT, gen_helper_gvec_facgt_s, gen_helper_gvec_facgt_h) return do_3same_fp(s, a, FUNC, READS_VD); \ } -DO_3S_FP(VMAX, gen_helper_vfp_maxs, false) -DO_3S_FP(VMIN, gen_helper_vfp_mins, false) - static void gen_VMLA_fp_3s(TCGv_i32 vd, TCGv_i32 vn, TCGv_i32 vm, TCGv_ptr fpstatus) { -- 2.20.1 Convert the Neon floating point VMAXNM and VMINNM insns to using a gvec helper and use this to implement the fp16 case. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-30-peter.maydell@linaro.org --- target/arm/helper.h | 6 ++++++ target/arm/vec_helper.c | 6 ++++++ target/arm/translate-neon.c.inc | 23 +++++++++++++++-------- 3 files changed, 27 insertions(+), 8 deletions(-) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_5(gvec_fmax_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_fmin_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_fmin_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_fmaxnum_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_fmaxnum_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(gvec_fminnum_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_fminnum_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_5(gvec_ftsmul_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_ftsmul_s, TCG_CALL_NO_RWG, diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -XXX,XX +XXX,XX @@ DO_3OP(gvec_fmax_s, float32_max, float32) DO_3OP(gvec_fmin_h, float16_min, float16) DO_3OP(gvec_fmin_s, float32_min, float32) +DO_3OP(gvec_fmaxnum_h, float16_maxnum, float16) +DO_3OP(gvec_fmaxnum_s, float32_maxnum, float32) + +DO_3OP(gvec_fminnum_h, float16_minnum, float16) +DO_3OP(gvec_fminnum_s, float32_minnum, float32) + #ifdef TARGET_AARCH64 DO_3OP(gvec_recps_h, helper_recpsf_f16, float16) diff --git a/target/arm/translate-neon.c.inc b/target/arm/translate-neon.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-neon.c.inc +++ b/target/arm/translate-neon.c.inc @@ -XXX,XX +XXX,XX @@ static void gen_VMLS_fp_3s(TCGv_i32 vd, TCGv_i32 vn, TCGv_i32 vm, DO_3S_FP(VMLA, gen_VMLA_fp_3s, true) DO_3S_FP(VMLS, gen_VMLS_fp_3s, true) +WRAP_FP_GVEC(gen_VMAXNM_fp32_3s, FPST_STD, gen_helper_gvec_fmaxnum_s) +WRAP_FP_GVEC(gen_VMAXNM_fp16_3s, FPST_STD_F16, gen_helper_gvec_fmaxnum_h) +WRAP_FP_GVEC(gen_VMINNM_fp32_3s, FPST_STD, gen_helper_gvec_fminnum_s) +WRAP_FP_GVEC(gen_VMINNM_fp16_3s, FPST_STD_F16, gen_helper_gvec_fminnum_h) + static bool trans_VMAXNM_fp_3s(DisasContext *s, arg_3same *a) { if (!arm_dc_feature(s, ARM_FEATURE_V8)) { @@ -XXX,XX +XXX,XX @@ static bool trans_VMAXNM_fp_3s(DisasContext *s, arg_3same *a) } if (a->size != 0) { - /* TODO fp16 support */ - return false; + if (!dc_isar_feature(aa32_fp16_arith, s)) { + return false; + } + return do_3same(s, a, gen_VMAXNM_fp16_3s); } - - return do_3same_fp(s, a, gen_helper_vfp_maxnums, false); + return do_3same(s, a, gen_VMAXNM_fp32_3s); } static bool trans_VMINNM_fp_3s(DisasContext *s, arg_3same *a) @@ -XXX,XX +XXX,XX @@ static bool trans_VMINNM_fp_3s(DisasContext *s, arg_3same *a) } if (a->size != 0) { - /* TODO fp16 support */ - return false; + if (!dc_isar_feature(aa32_fp16_arith, s)) { + return false; + } + return do_3same(s, a, gen_VMINNM_fp16_3s); } - - return do_3same_fp(s, a, gen_helper_vfp_minnums, false); + return do_3same(s, a, gen_VMINNM_fp32_3s); } WRAP_ENV_FN(gen_VRECPS_tramp, gen_helper_recps_f32) -- 2.20.1 Convert the Neon floating-point VMLA and VMLS insns over to using a gvec helper, and use this to implement the fp16 case. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-31-peter.maydell@linaro.org --- target/arm/helper.h | 6 +++++ target/arm/vec_helper.c | 42 +++++++++++++++++++++++++++++++++ target/arm/translate-neon.c.inc | 33 ++------------------------ 3 files changed, 50 insertions(+), 31 deletions(-) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_5(gvec_fmaxnum_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i3 DEF_HELPER_FLAGS_5(gvec_fminnum_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_fminnum_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_fmla_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_fmla_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(gvec_fmls_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_fmls_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_5(gvec_ftsmul_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_ftsmul_s, TCG_CALL_NO_RWG, diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -XXX,XX +XXX,XX @@ DO_3OP(gvec_rsqrts_d, helper_rsqrtsf_f64, float64) #endif #undef DO_3OP +/* Non-fused multiply-add (unlike float16_muladd etc, which are fused) */ +static float16 float16_muladd_nf(float16 dest, float16 op1, float16 op2, + float_status *stat) +{ + return float16_add(dest, float16_mul(op1, op2, stat), stat); +} + +static float32 float32_muladd_nf(float32 dest, float32 op1, float32 op2, + float_status *stat) +{ + return float32_add(dest, float32_mul(op1, op2, stat), stat); +} + +static float16 float16_mulsub_nf(float16 dest, float16 op1, float16 op2, + float_status *stat) +{ + return float16_sub(dest, float16_mul(op1, op2, stat), stat); +} + +static float32 float32_mulsub_nf(float32 dest, float32 op1, float32 op2, + float_status *stat) +{ + return float32_sub(dest, float32_mul(op1, op2, stat), stat); +} + +#define DO_MULADD(NAME, FUNC, TYPE) \ +void HELPER(NAME)(void *vd, void *vn, void *vm, void *stat, uint32_t desc) \ +{ \ + intptr_t i, oprsz = simd_oprsz(desc); \ + TYPE *d = vd, *n = vn, *m = vm; \ + for (i = 0; i < oprsz / sizeof(TYPE); i++) { \ + d[i] = FUNC(d[i], n[i], m[i], stat); \ + } \ + clear_tail(d, oprsz, simd_maxsz(desc)); \ +} + +DO_MULADD(gvec_fmla_h, float16_muladd_nf, float16) +DO_MULADD(gvec_fmla_s, float32_muladd_nf, float32) + +DO_MULADD(gvec_fmls_h, float16_mulsub_nf, float16) +DO_MULADD(gvec_fmls_s, float32_mulsub_nf, float32) + /* For the indexed ops, SVE applies the index per 128-bit vector segment. * For AdvSIMD, there is of course only one such vector segment. */ diff --git a/target/arm/translate-neon.c.inc b/target/arm/translate-neon.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-neon.c.inc +++ b/target/arm/translate-neon.c.inc @@ -XXX,XX +XXX,XX @@ DO_3S_FP_GVEC(VACGE, gen_helper_gvec_facge_s, gen_helper_gvec_facge_h) DO_3S_FP_GVEC(VACGT, gen_helper_gvec_facgt_s, gen_helper_gvec_facgt_h) DO_3S_FP_GVEC(VMAX, gen_helper_gvec_fmax_s, gen_helper_gvec_fmax_h) DO_3S_FP_GVEC(VMIN, gen_helper_gvec_fmin_s, gen_helper_gvec_fmin_h) - -/* - * For all the functions using this macro, size == 1 means fp16, - * which is an architecture extension we don't implement yet. - */ -#define DO_3S_FP(INSN,FUNC,READS_VD) \ - static bool trans_##INSN##_fp_3s(DisasContext *s, arg_3same *a) \ - { \ - if (a->size != 0) { \ - /* TODO fp16 support */ \ - return false; \ - } \ - return do_3same_fp(s, a, FUNC, READS_VD); \ - } - -static void gen_VMLA_fp_3s(TCGv_i32 vd, TCGv_i32 vn, TCGv_i32 vm, - TCGv_ptr fpstatus) -{ - gen_helper_vfp_muls(vn, vn, vm, fpstatus); - gen_helper_vfp_adds(vd, vd, vn, fpstatus); -} - -static void gen_VMLS_fp_3s(TCGv_i32 vd, TCGv_i32 vn, TCGv_i32 vm, - TCGv_ptr fpstatus) -{ - gen_helper_vfp_muls(vn, vn, vm, fpstatus); - gen_helper_vfp_subs(vd, vd, vn, fpstatus); -} - -DO_3S_FP(VMLA, gen_VMLA_fp_3s, true) -DO_3S_FP(VMLS, gen_VMLS_fp_3s, true) +DO_3S_FP_GVEC(VMLA, gen_helper_gvec_fmla_s, gen_helper_gvec_fmla_h) +DO_3S_FP_GVEC(VMLS, gen_helper_gvec_fmls_s, gen_helper_gvec_fmls_h) WRAP_FP_GVEC(gen_VMAXNM_fp32_3s, FPST_STD, gen_helper_gvec_fmaxnum_s) WRAP_FP_GVEC(gen_VMAXNM_fp16_3s, FPST_STD_F16, gen_helper_gvec_fmaxnum_h) -- 2.20.1 Convert the neon floating-point vector operations VFMA and VFMS to use a gvec helper, and use this to implement the fp16 case. This is the last use of do_3same_fp() so we can now delete that function. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-32-peter.maydell@linaro.org --- target/arm/helper.h | 6 +++ target/arm/vec_helper.c | 33 +++++++++++- target/arm/translate-neon.c.inc | 92 +-------------------------------- 3 files changed, 40 insertions(+), 91 deletions(-) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_5(gvec_fmla_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_fmls_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_fmls_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_vfma_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_vfma_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(gvec_vfms_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_vfms_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_5(gvec_ftsmul_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_ftsmul_s, TCG_CALL_NO_RWG, diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -XXX,XX +XXX,XX @@ static float32 float32_mulsub_nf(float32 dest, float32 op1, float32 op2, return float32_sub(dest, float32_mul(op1, op2, stat), stat); } -#define DO_MULADD(NAME, FUNC, TYPE) \ +/* Fused versions; these have the semantics Neon VFMA/VFMS want */ +static float16 float16_muladd_f(float16 dest, float16 op1, float16 op2, + float_status *stat) +{ + return float16_muladd(op1, op2, dest, 0, stat); +} + +static float32 float32_muladd_f(float32 dest, float32 op1, float32 op2, + float_status *stat) +{ + return float32_muladd(op1, op2, dest, 0, stat); +} + +static float16 float16_mulsub_f(float16 dest, float16 op1, float16 op2, + float_status *stat) +{ + return float16_muladd(float16_chs(op1), op2, dest, 0, stat); +} + +static float32 float32_mulsub_f(float32 dest, float32 op1, float32 op2, + float_status *stat) +{ + return float32_muladd(float32_chs(op1), op2, dest, 0, stat); +} + +#define DO_MULADD(NAME, FUNC, TYPE) \ void HELPER(NAME)(void *vd, void *vn, void *vm, void *stat, uint32_t desc) \ { \ intptr_t i, oprsz = simd_oprsz(desc); \ @@ -XXX,XX +XXX,XX @@ DO_MULADD(gvec_fmla_s, float32_muladd_nf, float32) DO_MULADD(gvec_fmls_h, float16_mulsub_nf, float16) DO_MULADD(gvec_fmls_s, float32_mulsub_nf, float32) +DO_MULADD(gvec_vfma_h, float16_muladd_f, float16) +DO_MULADD(gvec_vfma_s, float32_muladd_f, float32) + +DO_MULADD(gvec_vfms_h, float16_mulsub_f, float16) +DO_MULADD(gvec_vfms_s, float32_mulsub_f, float32) + /* For the indexed ops, SVE applies the index per 128-bit vector segment. * For AdvSIMD, there is of course only one such vector segment. */ diff --git a/target/arm/translate-neon.c.inc b/target/arm/translate-neon.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-neon.c.inc +++ b/target/arm/translate-neon.c.inc @@ -XXX,XX +XXX,XX @@ DO_3SAME_PAIR(VPADD, padd_u) DO_3SAME_VQDMULH(VQDMULH, qdmulh) DO_3SAME_VQDMULH(VQRDMULH, qrdmulh) -static bool do_3same_fp(DisasContext *s, arg_3same *a, VFPGen3OpSPFn *fn, - bool reads_vd) -{ - /* - * FP operations handled elementwise 32 bits at a time. - * If reads_vd is true then the old value of Vd will be - * loaded before calling the callback function. This is - * used for multiply-accumulate type operations. - */ - TCGv_i32 tmp, tmp2; - int pass; - - if (!arm_dc_feature(s, ARM_FEATURE_NEON)) { - return false; - } - - /* UNDEF accesses to D16-D31 if they don't exist. */ - if (!dc_isar_feature(aa32_simd_r32, s) && - ((a->vd | a->vn | a->vm) & 0x10)) { - return false; - } - - if ((a->vn | a->vm | a->vd) & a->q) { - return false; - } - - if (!vfp_access_check(s)) { - return true; - } - - TCGv_ptr fpstatus = fpstatus_ptr(FPST_STD); - for (pass = 0; pass < (a->q ? 4 : 2); pass++) { - tmp = neon_load_reg(a->vn, pass); - tmp2 = neon_load_reg(a->vm, pass); - if (reads_vd) { - TCGv_i32 tmp_rd = neon_load_reg(a->vd, pass); - fn(tmp_rd, tmp, tmp2, fpstatus); - neon_store_reg(a->vd, pass, tmp_rd); - tcg_temp_free_i32(tmp); - } else { - fn(tmp, tmp, tmp2, fpstatus); - neon_store_reg(a->vd, pass, tmp); - } - tcg_temp_free_i32(tmp2); - } - tcg_temp_free_ptr(fpstatus); - return true; -} - #define WRAP_FP_GVEC(WRAPNAME, FPST, FUNC) \ static void WRAPNAME(unsigned vece, uint32_t rd_ofs, \ uint32_t rn_ofs, uint32_t rm_ofs, \ @@ -XXX,XX +XXX,XX @@ DO_3S_FP_GVEC(VMAX, gen_helper_gvec_fmax_s, gen_helper_gvec_fmax_h) DO_3S_FP_GVEC(VMIN, gen_helper_gvec_fmin_s, gen_helper_gvec_fmin_h) DO_3S_FP_GVEC(VMLA, gen_helper_gvec_fmla_s, gen_helper_gvec_fmla_h) DO_3S_FP_GVEC(VMLS, gen_helper_gvec_fmls_s, gen_helper_gvec_fmls_h) +DO_3S_FP_GVEC(VFMA, gen_helper_gvec_vfma_s, gen_helper_gvec_vfma_h) +DO_3S_FP_GVEC(VFMS, gen_helper_gvec_vfms_s, gen_helper_gvec_vfms_h) WRAP_FP_GVEC(gen_VMAXNM_fp32_3s, FPST_STD, gen_helper_gvec_fmaxnum_s) WRAP_FP_GVEC(gen_VMAXNM_fp16_3s, FPST_STD_F16, gen_helper_gvec_fmaxnum_h) @@ -XXX,XX +XXX,XX @@ static bool trans_VRSQRTS_fp_3s(DisasContext *s, arg_3same *a) return do_3same(s, a, gen_VRSQRTS_fp_3s); } -static void gen_VFMA_fp_3s(TCGv_i32 vd, TCGv_i32 vn, TCGv_i32 vm, - TCGv_ptr fpstatus) -{ - gen_helper_vfp_muladds(vd, vn, vm, vd, fpstatus); -} - -static bool trans_VFMA_fp_3s(DisasContext *s, arg_3same *a) -{ - if (!dc_isar_feature(aa32_simdfmac, s)) { - return false; - } - - if (a->size != 0) { - /* TODO fp16 support */ - return false; - } - - return do_3same_fp(s, a, gen_VFMA_fp_3s, true); -} - -static void gen_VFMS_fp_3s(TCGv_i32 vd, TCGv_i32 vn, TCGv_i32 vm, - TCGv_ptr fpstatus) -{ - gen_helper_vfp_negs(vn, vn); - gen_helper_vfp_muladds(vd, vn, vm, vd, fpstatus); -} - -static bool trans_VFMS_fp_3s(DisasContext *s, arg_3same *a) -{ - if (!dc_isar_feature(aa32_simdfmac, s)) { - return false; - } - - if (a->size != 0) { - /* TODO fp16 support */ - return false; - } - - return do_3same_fp(s, a, gen_VFMS_fp_3s, true); -} - static bool do_3same_fp_pair(DisasContext *s, arg_3same *a, VFPGen3OpSPFn *fn) { /* FP operations handled pairwise 32 bits at a time */ -- 2.20.1 Convert the neon floating-point vector compare-vs-0 insns VCEQ0, VCGT0, VCLE0, VCGE0 and VCLT0 to use a gvec helper, and use this to implement the fp16 case. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-33-peter.maydell@linaro.org --- target/arm/helper.h | 15 +++++++++++++++ target/arm/vec_helper.c | 25 +++++++++++++++++++++++++ target/arm/translate-neon.c.inc | 33 +++++---------------------------- 3 files changed, 45 insertions(+), 28 deletions(-) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_4(gvec_frsqrte_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_frsqrte_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_frsqrte_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_fcgt0_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_fcgt0_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(gvec_fcge0_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_fcge0_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(gvec_fceq0_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_fceq0_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(gvec_fcle0_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_fcle0_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(gvec_fclt0_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_fclt0_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_5(gvec_fadd_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_fadd_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_fadd_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -XXX,XX +XXX,XX @@ DO_2OP(gvec_frsqrte_h, helper_rsqrte_f16, float16) DO_2OP(gvec_frsqrte_s, helper_rsqrte_f32, float32) DO_2OP(gvec_frsqrte_d, helper_rsqrte_f64, float64) +#define WRAP_CMP0_FWD(FN, CMPOP, TYPE) \ + static TYPE TYPE##_##FN##0(TYPE op, float_status *stat) \ + { \ + return TYPE##_##CMPOP(op, TYPE##_zero, stat); \ + } + +#define WRAP_CMP0_REV(FN, CMPOP, TYPE) \ + static TYPE TYPE##_##FN##0(TYPE op, float_status *stat) \ + { \ + return TYPE##_##CMPOP(TYPE##_zero, op, stat); \ + } + +#define DO_2OP_CMP0(FN, CMPOP, DIRN) \ + WRAP_CMP0_##DIRN(FN, CMPOP, float16) \ + WRAP_CMP0_##DIRN(FN, CMPOP, float32) \ + DO_2OP(gvec_f##FN##0_h, float16_##FN##0, float16) \ + DO_2OP(gvec_f##FN##0_s, float32_##FN##0, float32) + +DO_2OP_CMP0(cgt, cgt, FWD) +DO_2OP_CMP0(cge, cge, FWD) +DO_2OP_CMP0(ceq, ceq, FWD) +DO_2OP_CMP0(clt, cgt, REV) +DO_2OP_CMP0(cle, cge, REV) + #undef DO_2OP +#undef DO_2OP_CMP0 /* Floating-point trigonometric starting value. * See the ARM ARM pseudocode function FPTrigSMul. diff --git a/target/arm/translate-neon.c.inc b/target/arm/translate-neon.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-neon.c.inc +++ b/target/arm/translate-neon.c.inc @@ -XXX,XX +XXX,XX @@ DO_2MISC_FP(VCVT_UF, gen_helper_vfp_touizs) DO_2MISC_FP_VEC(VRECPE_F, gen_helper_gvec_frecpe_h, gen_helper_gvec_frecpe_s) DO_2MISC_FP_VEC(VRSQRTE_F, gen_helper_gvec_frsqrte_h, gen_helper_gvec_frsqrte_s) +DO_2MISC_FP_VEC(VCGT0_F, gen_helper_gvec_fcgt0_h, gen_helper_gvec_fcgt0_s) +DO_2MISC_FP_VEC(VCGE0_F, gen_helper_gvec_fcge0_h, gen_helper_gvec_fcge0_s) +DO_2MISC_FP_VEC(VCEQ0_F, gen_helper_gvec_fceq0_h, gen_helper_gvec_fceq0_s) +DO_2MISC_FP_VEC(VCLT0_F, gen_helper_gvec_fclt0_h, gen_helper_gvec_fclt0_s) +DO_2MISC_FP_VEC(VCLE0_F, gen_helper_gvec_fcle0_h, gen_helper_gvec_fcle0_s) static bool trans_VRINTX(DisasContext *s, arg_2misc *a) { @@ -XXX,XX +XXX,XX @@ static bool trans_VRINTX(DisasContext *s, arg_2misc *a) return do_2misc_fp(s, a, gen_helper_rints_exact); } -#define WRAP_FP_CMP0_FWD(WRAPNAME, FUNC) \ - static void WRAPNAME(TCGv_i32 d, TCGv_i32 m, TCGv_ptr fpst) \ - { \ - TCGv_i32 zero = tcg_const_i32(0); \ - FUNC(d, m, zero, fpst); \ - tcg_temp_free_i32(zero); \ - } -#define WRAP_FP_CMP0_REV(WRAPNAME, FUNC) \ - static void WRAPNAME(TCGv_i32 d, TCGv_i32 m, TCGv_ptr fpst) \ - { \ - TCGv_i32 zero = tcg_const_i32(0); \ - FUNC(d, zero, m, fpst); \ - tcg_temp_free_i32(zero); \ - } - -#define DO_FP_CMP0(INSN, FUNC, REV) \ - WRAP_FP_CMP0_##REV(gen_##INSN, FUNC) \ - static bool trans_##INSN(DisasContext *s, arg_2misc *a) \ - { \ - return do_2misc_fp(s, a, gen_##INSN); \ - } - -DO_FP_CMP0(VCGT0_F, gen_helper_neon_cgt_f32, FWD) -DO_FP_CMP0(VCGE0_F, gen_helper_neon_cge_f32, FWD) -DO_FP_CMP0(VCEQ0_F, gen_helper_neon_ceq_f32, FWD) -DO_FP_CMP0(VCLE0_F, gen_helper_neon_cge_f32, REV) -DO_FP_CMP0(VCLT0_F, gen_helper_neon_cgt_f32, REV) - static bool do_vrint(DisasContext *s, arg_2misc *a, int rmode) { /* -- 2.20.1 Convert the Neon VRECPS insn to using a gvec helper, and use this to implement the fp16 case. The phrasing of the new float32_recps_nf() is slightly different from the old recps_f32() so that it parallels the f16 version; for f16 we can't assume that flush-to-zero is always enabled. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-34-peter.maydell@linaro.org --- target/arm/helper.h | 4 +++- target/arm/vec_helper.c | 31 +++++++++++++++++++++++++++++++ target/arm/vfp_helper.c | 13 ------------- target/arm/translate-neon.c.inc | 21 +-------------------- 4 files changed, 35 insertions(+), 34 deletions(-) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_4(vfp_muladdd, f64, f64, f64, f64, ptr) DEF_HELPER_4(vfp_muladds, f32, f32, f32, f32, ptr) DEF_HELPER_4(vfp_muladdh, f16, f16, f16, f16, ptr) -DEF_HELPER_3(recps_f32, f32, env, f32, f32) DEF_HELPER_3(rsqrts_f32, f32, env, f32, f32) DEF_HELPER_FLAGS_2(recpe_f16, TCG_CALL_NO_RWG, f16, f16, ptr) DEF_HELPER_FLAGS_2(recpe_f32, TCG_CALL_NO_RWG, f32, f32, ptr) @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_5(gvec_fmaxnum_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i3 DEF_HELPER_FLAGS_5(gvec_fminnum_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_fminnum_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_recps_nf_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_recps_nf_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_5(gvec_fmla_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_fmla_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -XXX,XX +XXX,XX @@ static float32 float32_abd(float32 op1, float32 op2, float_status *stat) return float32_abs(float32_sub(op1, op2, stat)); } +/* + * Reciprocal step. These are the AArch32 version which uses a + * non-fused multiply-and-subtract. + */ +static float16 float16_recps_nf(float16 op1, float16 op2, float_status *stat) +{ + op1 = float16_squash_input_denormal(op1, stat); + op2 = float16_squash_input_denormal(op2, stat); + + if ((float16_is_infinity(op1) && float16_is_zero(op2)) || + (float16_is_infinity(op2) && float16_is_zero(op1))) { + return float16_two; + } + return float16_sub(float16_two, float16_mul(op1, op2, stat), stat); +} + +static float32 float32_recps_nf(float32 op1, float32 op2, float_status *stat) +{ + op1 = float32_squash_input_denormal(op1, stat); + op2 = float32_squash_input_denormal(op2, stat); + + if ((float32_is_infinity(op1) && float32_is_zero(op2)) || + (float32_is_infinity(op2) && float32_is_zero(op1))) { + return float32_two; + } + return float32_sub(float32_two, float32_mul(op1, op2, stat), stat); +} + #define DO_3OP(NAME, FUNC, TYPE) \ void HELPER(NAME)(void *vd, void *vn, void *vm, void *stat, uint32_t desc) \ { \ @@ -XXX,XX +XXX,XX @@ DO_3OP(gvec_fmaxnum_s, float32_maxnum, float32) DO_3OP(gvec_fminnum_h, float16_minnum, float16) DO_3OP(gvec_fminnum_s, float32_minnum, float32) +DO_3OP(gvec_recps_nf_h, float16_recps_nf, float16) +DO_3OP(gvec_recps_nf_s, float32_recps_nf, float32) + #ifdef TARGET_AARCH64 DO_3OP(gvec_recps_h, helper_recpsf_f16, float16) diff --git a/target/arm/vfp_helper.c b/target/arm/vfp_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vfp_helper.c +++ b/target/arm/vfp_helper.c @@ -XXX,XX +XXX,XX @@ uint32_t HELPER(vfp_fcvt_f64_to_f16)(float64 a, void *fpstp, uint32_t ahp_mode) return r; } -float32 HELPER(recps_f32)(CPUARMState *env, float32 a, float32 b) -{ - float_status *s = &env->vfp.standard_fp_status; - if ((float32_is_infinity(a) && float32_is_zero_or_denormal(b)) || - (float32_is_infinity(b) && float32_is_zero_or_denormal(a))) { - if (!(float32_is_zero(a) || float32_is_zero(b))) { - float_raise(float_flag_input_denormal, s); - } - return float32_two; - } - return float32_sub(float32_two, float32_mul(a, b, s), s); -} - float32 HELPER(rsqrts_f32)(CPUARMState *env, float32 a, float32 b) { float_status *s = &env->vfp.standard_fp_status; diff --git a/target/arm/translate-neon.c.inc b/target/arm/translate-neon.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-neon.c.inc +++ b/target/arm/translate-neon.c.inc @@ -XXX,XX +XXX,XX @@ DO_3S_FP_GVEC(VMLA, gen_helper_gvec_fmla_s, gen_helper_gvec_fmla_h) DO_3S_FP_GVEC(VMLS, gen_helper_gvec_fmls_s, gen_helper_gvec_fmls_h) DO_3S_FP_GVEC(VFMA, gen_helper_gvec_vfma_s, gen_helper_gvec_vfma_h) DO_3S_FP_GVEC(VFMS, gen_helper_gvec_vfms_s, gen_helper_gvec_vfms_h) +DO_3S_FP_GVEC(VRECPS, gen_helper_gvec_recps_nf_s, gen_helper_gvec_recps_nf_h) WRAP_FP_GVEC(gen_VMAXNM_fp32_3s, FPST_STD, gen_helper_gvec_fmaxnum_s) WRAP_FP_GVEC(gen_VMAXNM_fp16_3s, FPST_STD_F16, gen_helper_gvec_fmaxnum_h) @@ -XXX,XX +XXX,XX @@ static bool trans_VMINNM_fp_3s(DisasContext *s, arg_3same *a) return do_3same(s, a, gen_VMINNM_fp32_3s); } -WRAP_ENV_FN(gen_VRECPS_tramp, gen_helper_recps_f32) - -static void gen_VRECPS_fp_3s(unsigned vece, uint32_t rd_ofs, - uint32_t rn_ofs, uint32_t rm_ofs, - uint32_t oprsz, uint32_t maxsz) -{ - static const GVecGen3 ops = { .fni4 = gen_VRECPS_tramp }; - tcg_gen_gvec_3(rd_ofs, rn_ofs, rm_ofs, oprsz, maxsz, &ops); -} - -static bool trans_VRECPS_fp_3s(DisasContext *s, arg_3same *a) -{ - if (a->size != 0) { - /* TODO fp16 support */ - return false; - } - - return do_3same(s, a, gen_VRECPS_fp_3s); -} - WRAP_ENV_FN(gen_VRSQRTS_tramp, gen_helper_rsqrts_f32) static void gen_VRSQRTS_fp_3s(unsigned vece, uint32_t rd_ofs, -- 2.20.1 Convert the Neon VRSQRTS insn to using a gvec helper, and use this to implement the fp16 case. As with VRECPS, we adjust the phrasing of the new implementation slightly so that the fp32 version parallels the fp16 one. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-35-peter.maydell@linaro.org --- target/arm/helper.h | 4 +++- target/arm/vec_helper.c | 30 ++++++++++++++++++++++++++++++ target/arm/vfp_helper.c | 15 --------------- target/arm/translate-neon.c.inc | 21 +-------------------- 4 files changed, 34 insertions(+), 36 deletions(-) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_4(vfp_muladdd, f64, f64, f64, f64, ptr) DEF_HELPER_4(vfp_muladds, f32, f32, f32, f32, ptr) DEF_HELPER_4(vfp_muladdh, f16, f16, f16, f16, ptr) -DEF_HELPER_3(rsqrts_f32, f32, env, f32, f32) DEF_HELPER_FLAGS_2(recpe_f16, TCG_CALL_NO_RWG, f16, f16, ptr) DEF_HELPER_FLAGS_2(recpe_f32, TCG_CALL_NO_RWG, f32, f32, ptr) DEF_HELPER_FLAGS_2(recpe_f64, TCG_CALL_NO_RWG, f64, f64, ptr) @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_5(gvec_fminnum_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i3 DEF_HELPER_FLAGS_5(gvec_recps_nf_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_recps_nf_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_rsqrts_nf_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_rsqrts_nf_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_5(gvec_fmla_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_fmla_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -XXX,XX +XXX,XX @@ static float32 float32_recps_nf(float32 op1, float32 op2, float_status *stat) return float32_sub(float32_two, float32_mul(op1, op2, stat), stat); } +/* Reciprocal square-root step. AArch32 non-fused semantics. */ +static float16 float16_rsqrts_nf(float16 op1, float16 op2, float_status *stat) +{ + op1 = float16_squash_input_denormal(op1, stat); + op2 = float16_squash_input_denormal(op2, stat); + + if ((float16_is_infinity(op1) && float16_is_zero(op2)) || + (float16_is_infinity(op2) && float16_is_zero(op1))) { + return float16_one_point_five; + } + op1 = float16_sub(float16_three, float16_mul(op1, op2, stat), stat); + return float16_div(op1, float16_two, stat); +} + +static float32 float32_rsqrts_nf(float32 op1, float32 op2, float_status *stat) +{ + op1 = float32_squash_input_denormal(op1, stat); + op2 = float32_squash_input_denormal(op2, stat); + + if ((float32_is_infinity(op1) && float32_is_zero(op2)) || + (float32_is_infinity(op2) && float32_is_zero(op1))) { + return float32_one_point_five; + } + op1 = float32_sub(float32_three, float32_mul(op1, op2, stat), stat); + return float32_div(op1, float32_two, stat); +} + #define DO_3OP(NAME, FUNC, TYPE) \ void HELPER(NAME)(void *vd, void *vn, void *vm, void *stat, uint32_t desc) \ { \ @@ -XXX,XX +XXX,XX @@ DO_3OP(gvec_fminnum_s, float32_minnum, float32) DO_3OP(gvec_recps_nf_h, float16_recps_nf, float16) DO_3OP(gvec_recps_nf_s, float32_recps_nf, float32) +DO_3OP(gvec_rsqrts_nf_h, float16_rsqrts_nf, float16) +DO_3OP(gvec_rsqrts_nf_s, float32_rsqrts_nf, float32) + #ifdef TARGET_AARCH64 DO_3OP(gvec_recps_h, helper_recpsf_f16, float16) diff --git a/target/arm/vfp_helper.c b/target/arm/vfp_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vfp_helper.c +++ b/target/arm/vfp_helper.c @@ -XXX,XX +XXX,XX @@ uint32_t HELPER(vfp_fcvt_f64_to_f16)(float64 a, void *fpstp, uint32_t ahp_mode) return r; } -float32 HELPER(rsqrts_f32)(CPUARMState *env, float32 a, float32 b) -{ - float_status *s = &env->vfp.standard_fp_status; - float32 product; - if ((float32_is_infinity(a) && float32_is_zero_or_denormal(b)) || - (float32_is_infinity(b) && float32_is_zero_or_denormal(a))) { - if (!(float32_is_zero(a) || float32_is_zero(b))) { - float_raise(float_flag_input_denormal, s); - } - return float32_one_point_five; - } - product = float32_mul(a, b, s); - return float32_div(float32_sub(float32_three, product, s), float32_two, s); -} - /* NEON helpers. */ /* Constants 256 and 512 are used in some helpers; we avoid relying on diff --git a/target/arm/translate-neon.c.inc b/target/arm/translate-neon.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-neon.c.inc +++ b/target/arm/translate-neon.c.inc @@ -XXX,XX +XXX,XX @@ DO_3S_FP_GVEC(VMLS, gen_helper_gvec_fmls_s, gen_helper_gvec_fmls_h) DO_3S_FP_GVEC(VFMA, gen_helper_gvec_vfma_s, gen_helper_gvec_vfma_h) DO_3S_FP_GVEC(VFMS, gen_helper_gvec_vfms_s, gen_helper_gvec_vfms_h) DO_3S_FP_GVEC(VRECPS, gen_helper_gvec_recps_nf_s, gen_helper_gvec_recps_nf_h) +DO_3S_FP_GVEC(VRSQRTS, gen_helper_gvec_rsqrts_nf_s, gen_helper_gvec_rsqrts_nf_h) WRAP_FP_GVEC(gen_VMAXNM_fp32_3s, FPST_STD, gen_helper_gvec_fmaxnum_s) WRAP_FP_GVEC(gen_VMAXNM_fp16_3s, FPST_STD_F16, gen_helper_gvec_fmaxnum_h) @@ -XXX,XX +XXX,XX @@ static bool trans_VMINNM_fp_3s(DisasContext *s, arg_3same *a) return do_3same(s, a, gen_VMINNM_fp32_3s); } -WRAP_ENV_FN(gen_VRSQRTS_tramp, gen_helper_rsqrts_f32) - -static void gen_VRSQRTS_fp_3s(unsigned vece, uint32_t rd_ofs, - uint32_t rn_ofs, uint32_t rm_ofs, - uint32_t oprsz, uint32_t maxsz) -{ - static const GVecGen3 ops = { .fni4 = gen_VRSQRTS_tramp }; - tcg_gen_gvec_3(rd_ofs, rn_ofs, rm_ofs, oprsz, maxsz, &ops); -} - -static bool trans_VRSQRTS_fp_3s(DisasContext *s, arg_3same *a) -{ - if (a->size != 0) { - /* TODO fp16 support */ - return false; - } - - return do_3same(s, a, gen_VRSQRTS_fp_3s); -} - static bool do_3same_fp_pair(DisasContext *s, arg_3same *a, VFPGen3OpSPFn *fn) { /* FP operations handled pairwise 32 bits at a time */ -- 2.20.1 Convert the Neon pairwise fp ops to use a single gvic-style helper to do the full operation instead of one helper call for each 32-bit part. This allows us to use the same framework to implement the fp16. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-36-peter.maydell@linaro.org --- target/arm/helper.h | 7 +++++ target/arm/vec_helper.c | 45 +++++++++++++++++++++++++++++++++ target/arm/translate-neon.c.inc | 42 ++++++++++++------------------ 3 files changed, 68 insertions(+), 26 deletions(-) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_5(gvec_fcmlas_idx, TCG_CALL_NO_RWG, DEF_HELPER_FLAGS_5(gvec_fcmlad, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(neon_paddh, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(neon_pmaxh, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(neon_pminh, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(neon_padds, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(neon_pmaxs, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(neon_pmins, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_4(gvec_frecpe_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_frecpe_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_frecpe_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -XXX,XX +XXX,XX @@ DO_ABA(gvec_uaba_s, uint32_t) DO_ABA(gvec_uaba_d, uint64_t) #undef DO_ABA + +#define DO_NEON_PAIRWISE(NAME, OP) \ + void HELPER(NAME##s)(void *vd, void *vn, void *vm, \ + void *stat, uint32_t oprsz) \ + { \ + float_status *fpst = stat; \ + float32 *d = vd; \ + float32 *n = vn; \ + float32 *m = vm; \ + float32 r0, r1; \ + \ + /* Read all inputs before writing outputs in case vm == vd */ \ + r0 = float32_##OP(n[H4(0)], n[H4(1)], fpst); \ + r1 = float32_##OP(m[H4(0)], m[H4(1)], fpst); \ + \ + d[H4(0)] = r0; \ + d[H4(1)] = r1; \ + } \ + \ + void HELPER(NAME##h)(void *vd, void *vn, void *vm, \ + void *stat, uint32_t oprsz) \ + { \ + float_status *fpst = stat; \ + float16 *d = vd; \ + float16 *n = vn; \ + float16 *m = vm; \ + float16 r0, r1, r2, r3; \ + \ + /* Read all inputs before writing outputs in case vm == vd */ \ + r0 = float16_##OP(n[H2(0)], n[H2(1)], fpst); \ + r1 = float16_##OP(n[H2(2)], n[H2(3)], fpst); \ + r2 = float16_##OP(m[H2(0)], m[H2(1)], fpst); \ + r3 = float16_##OP(m[H2(2)], m[H2(3)], fpst); \ + \ + d[H4(0)] = r0; \ + d[H4(1)] = r1; \ + d[H4(2)] = r2; \ + d[H4(3)] = r3; \ + } + +DO_NEON_PAIRWISE(neon_padd, add) +DO_NEON_PAIRWISE(neon_pmax, max) +DO_NEON_PAIRWISE(neon_pmin, min) + +#undef DO_NEON_PAIRWISE diff --git a/target/arm/translate-neon.c.inc b/target/arm/translate-neon.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-neon.c.inc +++ b/target/arm/translate-neon.c.inc @@ -XXX,XX +XXX,XX @@ static bool trans_VMINNM_fp_3s(DisasContext *s, arg_3same *a) return do_3same(s, a, gen_VMINNM_fp32_3s); } -static bool do_3same_fp_pair(DisasContext *s, arg_3same *a, VFPGen3OpSPFn *fn) +static bool do_3same_fp_pair(DisasContext *s, arg_3same *a, + gen_helper_gvec_3_ptr *fn) { - /* FP operations handled pairwise 32 bits at a time */ - TCGv_i32 tmp, tmp2, tmp3; + /* FP pairwise operations */ TCGv_ptr fpstatus; if (!arm_dc_feature(s, ARM_FEATURE_NEON)) { @@ -XXX,XX +XXX,XX @@ static bool do_3same_fp_pair(DisasContext *s, arg_3same *a, VFPGen3OpSPFn *fn) assert(a->q == 0); /* enforced by decode patterns */ - /* - * Note that we have to be careful not to clobber the source operands - * in the "vm == vd" case by storing the result of the first pass too - * early. Since Q is 0 there are always just two passes, so instead - * of a complicated loop over each pass we just unroll. - */ - fpstatus = fpstatus_ptr(FPST_STD); - tmp = neon_load_reg(a->vn, 0); - tmp2 = neon_load_reg(a->vn, 1); - fn(tmp, tmp, tmp2, fpstatus); - tcg_temp_free_i32(tmp2); - tmp3 = neon_load_reg(a->vm, 0); - tmp2 = neon_load_reg(a->vm, 1); - fn(tmp3, tmp3, tmp2, fpstatus); - tcg_temp_free_i32(tmp2); + fpstatus = fpstatus_ptr(a->size != 0 ? FPST_STD_F16 : FPST_STD); + tcg_gen_gvec_3_ptr(vfp_reg_offset(1, a->vd), + vfp_reg_offset(1, a->vn), + vfp_reg_offset(1, a->vm), + fpstatus, 8, 8, 0, fn); tcg_temp_free_ptr(fpstatus); - neon_store_reg(a->vd, 0, tmp); - neon_store_reg(a->vd, 1, tmp3); return true; } @@ -XXX,XX +XXX,XX @@ static bool do_3same_fp_pair(DisasContext *s, arg_3same *a, VFPGen3OpSPFn *fn) static bool trans_##INSN##_fp_3s(DisasContext *s, arg_3same *a) \ { \ if (a->size != 0) { \ - /* TODO fp16 support */ \ - return false; \ + if (!dc_isar_feature(aa32_fp16_arith, s)) { \ + return false; \ + } \ + return do_3same_fp_pair(s, a, FUNC##h); \ } \ - return do_3same_fp_pair(s, a, FUNC); \ + return do_3same_fp_pair(s, a, FUNC##s); \ } -DO_3S_FP_PAIR(VPADD, gen_helper_vfp_adds) -DO_3S_FP_PAIR(VPMAX, gen_helper_vfp_maxs) -DO_3S_FP_PAIR(VPMIN, gen_helper_vfp_mins) +DO_3S_FP_PAIR(VPADD, gen_helper_neon_padd) +DO_3S_FP_PAIR(VPMAX, gen_helper_neon_pmax) +DO_3S_FP_PAIR(VPMIN, gen_helper_neon_pmin) static bool do_vector_2sh(DisasContext *s, arg_2reg_shift *a, GVecGen2iFn *fn) { -- 2.20.1 Convert the Neon float-integer VCVT insns to gvec, and use this to implement fp16 support for them. Note that unlike the VFP int<->fp16 VCVT insns we converted earlier and which convert to/from a 32-bit integer, these Neon insns convert to/from 16-bit integers. So we can use the existing vfp conversion helpers for the f32<->u32/i32 case but need to provide our own for f16<->u16/i16. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-37-peter.maydell@linaro.org --- target/arm/helper.h | 9 +++++++++ target/arm/vec_helper.c | 29 +++++++++++++++++++++++++++++ target/arm/translate-neon.c.inc | 15 ++++----------- 3 files changed, 42 insertions(+), 11 deletions(-) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_5(neon_padds, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(neon_pmaxs, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(neon_pmins, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_sstoh, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_sitos, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_ustoh, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_uitos, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_tosszh, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_tosizs, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_touszh, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_touizs, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_4(gvec_frecpe_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_frecpe_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_frecpe_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -XXX,XX +XXX,XX @@ static uint32_t float32_acgt(float32 op1, float32 op2, float_status *stat) return -float32_lt(float32_abs(op2), float32_abs(op1), stat); } +static int16_t vfp_tosszh(float16 x, void *fpstp) +{ + float_status *fpst = fpstp; + if (float16_is_any_nan(x)) { + float_raise(float_flag_invalid, fpst); + return 0; + } + return float16_to_int16_round_to_zero(x, fpst); +} + +static uint16_t vfp_touszh(float16 x, void *fpstp) +{ + float_status *fpst = fpstp; + if (float16_is_any_nan(x)) { + float_raise(float_flag_invalid, fpst); + return 0; + } + return float16_to_uint16_round_to_zero(x, fpst); +} + #define DO_2OP(NAME, FUNC, TYPE) \ void HELPER(NAME)(void *vd, void *vn, void *stat, uint32_t desc) \ { \ @@ -XXX,XX +XXX,XX @@ DO_2OP(gvec_frsqrte_h, helper_rsqrte_f16, float16) DO_2OP(gvec_frsqrte_s, helper_rsqrte_f32, float32) DO_2OP(gvec_frsqrte_d, helper_rsqrte_f64, float64) +DO_2OP(gvec_sitos, helper_vfp_sitos, int32_t) +DO_2OP(gvec_uitos, helper_vfp_uitos, uint32_t) +DO_2OP(gvec_tosizs, helper_vfp_tosizs, float32) +DO_2OP(gvec_touizs, helper_vfp_touizs, float32) +DO_2OP(gvec_sstoh, int16_to_float16, int16_t) +DO_2OP(gvec_ustoh, uint16_to_float16, uint16_t) +DO_2OP(gvec_tosszh, vfp_tosszh, float16) +DO_2OP(gvec_touszh, vfp_touszh, float16) + #define WRAP_CMP0_FWD(FN, CMPOP, TYPE) \ static TYPE TYPE##_##FN##0(TYPE op, float_status *stat) \ { \ diff --git a/target/arm/translate-neon.c.inc b/target/arm/translate-neon.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-neon.c.inc +++ b/target/arm/translate-neon.c.inc @@ -XXX,XX +XXX,XX @@ static bool do_2misc_fp(DisasContext *s, arg_2misc *a, return true; } -#define DO_2MISC_FP(INSN, FUNC) \ - static bool trans_##INSN(DisasContext *s, arg_2misc *a) \ - { \ - return do_2misc_fp(s, a, FUNC); \ - } - -DO_2MISC_FP(VCVT_FS, gen_helper_vfp_sitos) -DO_2MISC_FP(VCVT_FU, gen_helper_vfp_uitos) -DO_2MISC_FP(VCVT_SF, gen_helper_vfp_tosizs) -DO_2MISC_FP(VCVT_UF, gen_helper_vfp_touizs) - #define DO_2MISC_FP_VEC(INSN, HFUNC, SFUNC) \ static void gen_##INSN(unsigned vece, uint32_t rd_ofs, \ uint32_t rm_ofs, \ @@ -XXX,XX +XXX,XX @@ DO_2MISC_FP_VEC(VCGE0_F, gen_helper_gvec_fcge0_h, gen_helper_gvec_fcge0_s) DO_2MISC_FP_VEC(VCEQ0_F, gen_helper_gvec_fceq0_h, gen_helper_gvec_fceq0_s) DO_2MISC_FP_VEC(VCLT0_F, gen_helper_gvec_fclt0_h, gen_helper_gvec_fclt0_s) DO_2MISC_FP_VEC(VCLE0_F, gen_helper_gvec_fcle0_h, gen_helper_gvec_fcle0_s) +DO_2MISC_FP_VEC(VCVT_FS, gen_helper_gvec_sstoh, gen_helper_gvec_sitos) +DO_2MISC_FP_VEC(VCVT_FU, gen_helper_gvec_ustoh, gen_helper_gvec_uitos) +DO_2MISC_FP_VEC(VCVT_SF, gen_helper_gvec_tosszh, gen_helper_gvec_tosizs) +DO_2MISC_FP_VEC(VCVT_UF, gen_helper_gvec_touszh, gen_helper_gvec_touizs) static bool trans_VRINTX(DisasContext *s, arg_2misc *a) { -- 2.20.1 Convert the Neon VCVT float<->fixed-point insns to a gvec style, in preparation for adding fp16 support. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-38-peter.maydell@linaro.org --- target/arm/helper.h | 5 +++++ target/arm/vec_helper.c | 20 +++++++++++++++++++ target/arm/translate-neon.c.inc | 35 +++++++++++++++++---------------- 3 files changed, 43 insertions(+), 17 deletions(-) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_4(gvec_tosizs, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_touszh, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_touizs, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_vcvt_sf, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_vcvt_uf, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_vcvt_fs, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_vcvt_fu, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_4(gvec_frecpe_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_frecpe_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_frecpe_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -XXX,XX +XXX,XX @@ DO_NEON_PAIRWISE(neon_pmax, max) DO_NEON_PAIRWISE(neon_pmin, min) #undef DO_NEON_PAIRWISE + +#define DO_VCVT_FIXED(NAME, FUNC, TYPE) \ + void HELPER(NAME)(void *vd, void *vn, void *stat, uint32_t desc) \ + { \ + intptr_t i, oprsz = simd_oprsz(desc); \ + int shift = simd_data(desc); \ + TYPE *d = vd, *n = vn; \ + float_status *fpst = stat; \ + for (i = 0; i < oprsz / sizeof(TYPE); i++) { \ + d[i] = FUNC(n[i], shift, fpst); \ + } \ + clear_tail(d, oprsz, simd_maxsz(desc)); \ + } + +DO_VCVT_FIXED(gvec_vcvt_sf, helper_vfp_sltos, uint32_t) +DO_VCVT_FIXED(gvec_vcvt_uf, helper_vfp_ultos, uint32_t) +DO_VCVT_FIXED(gvec_vcvt_fs, helper_vfp_tosls_round_to_zero, uint32_t) +DO_VCVT_FIXED(gvec_vcvt_fu, helper_vfp_touls_round_to_zero, uint32_t) + +#undef DO_VCVT_FIXED diff --git a/target/arm/translate-neon.c.inc b/target/arm/translate-neon.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-neon.c.inc +++ b/target/arm/translate-neon.c.inc @@ -XXX,XX +XXX,XX @@ static bool trans_VSHLL_U_2sh(DisasContext *s, arg_2reg_shift *a) } static bool do_fp_2sh(DisasContext *s, arg_2reg_shift *a, - NeonGenTwoSingleOpFn *fn) + gen_helper_gvec_2_ptr *fn) { /* FP operations in 2-reg-and-shift group */ - TCGv_i32 tmp, shiftv; - TCGv_ptr fpstatus; - int pass; + int vec_size = a->q ? 16 : 8; + int rd_ofs = neon_reg_offset(a->vd, 0); + int rm_ofs = neon_reg_offset(a->vm, 0); + TCGv_ptr fpst; if (!arm_dc_feature(s, ARM_FEATURE_NEON)) { return false; } + if (a->size != 0) { + if (!dc_isar_feature(aa32_fp16_arith, s)) { + return false; + } + } + /* UNDEF accesses to D16-D31 if they don't exist. */ if (!dc_isar_feature(aa32_simd_r32, s) && ((a->vd | a->vm) & 0x10)) { @@ -XXX,XX +XXX,XX @@ static bool do_fp_2sh(DisasContext *s, arg_2reg_shift *a, return true; } - fpstatus = fpstatus_ptr(FPST_STD); - shiftv = tcg_const_i32(a->shift); - for (pass = 0; pass < (a->q ? 4 : 2); pass++) { - tmp = neon_load_reg(a->vm, pass); - fn(tmp, tmp, shiftv, fpstatus); - neon_store_reg(a->vd, pass, tmp); - } - tcg_temp_free_ptr(fpstatus); - tcg_temp_free_i32(shiftv); + fpst = fpstatus_ptr(a->size ? FPST_STD_F16 : FPST_STD); + tcg_gen_gvec_2_ptr(rd_ofs, rm_ofs, fpst, vec_size, vec_size, a->shift, fn); + tcg_temp_free_ptr(fpst); return true; } @@ -XXX,XX +XXX,XX @@ static bool do_fp_2sh(DisasContext *s, arg_2reg_shift *a, return do_fp_2sh(s, a, FUNC); \ } -DO_FP_2SH(VCVT_SF, gen_helper_vfp_sltos) -DO_FP_2SH(VCVT_UF, gen_helper_vfp_ultos) -DO_FP_2SH(VCVT_FS, gen_helper_vfp_tosls_round_to_zero) -DO_FP_2SH(VCVT_FU, gen_helper_vfp_touls_round_to_zero) +DO_FP_2SH(VCVT_SF, gen_helper_gvec_vcvt_sf) +DO_FP_2SH(VCVT_UF, gen_helper_gvec_vcvt_uf) +DO_FP_2SH(VCVT_FS, gen_helper_gvec_vcvt_fs) +DO_FP_2SH(VCVT_FU, gen_helper_gvec_vcvt_fu) static uint64_t asimd_imm_const(uint32_t imm, int cmode, int op) { -- 2.20.1 Implement fp16 for the Neon VCVT insns which convert between float and fixed-point. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-39-peter.maydell@linaro.org --- target/arm/helper.h | 5 +++++ target/arm/neon-dp.decode | 8 +++++++- target/arm/vec_helper.c | 4 ++++ target/arm/translate-neon.c.inc | 5 +++++ 4 files changed, 21 insertions(+), 1 deletion(-) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_4(gvec_vcvt_uf, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_vcvt_fs, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_vcvt_fu, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_vcvt_sh, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_vcvt_uh, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_vcvt_hs, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_vcvt_hu, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_4(gvec_frecpe_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_frecpe_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_frecpe_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) diff --git a/target/arm/neon-dp.decode b/target/arm/neon-dp.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/neon-dp.decode +++ b/target/arm/neon-dp.decode @@ -XXX,XX +XXX,XX @@ VMINNM_fp_3s 1111 001 1 0 . 1 . .... .... 1111 ... 1 .... @3same_fp # We use size=0 for fp32 and size=1 for fp16 to match the 3-same encodings. @2reg_vcvt .... ... . . . 1 ..... .... .... . q:1 . . .... \ &2reg_shift vm=%vm_dp vd=%vd_dp size=0 shift=%neon_rshift_i5 +@2reg_vcvt_f16 .... ... . . . 11 .... .... .... . q:1 . . .... \ + &2reg_shift vm=%vm_dp vd=%vd_dp size=1 shift=%neon_rshift_i4 VSHR_S_2sh 1111 001 0 1 . ...... .... 0000 . . . 1 .... @2reg_shr_d VSHR_S_2sh 1111 001 0 1 . ...... .... 0000 . . . 1 .... @2reg_shr_s @@ -XXX,XX +XXX,XX @@ VSHLL_U_2sh 1111 001 1 1 . ...... .... 1010 . 0 . 1 .... @2reg_shll_h VSHLL_U_2sh 1111 001 1 1 . ...... .... 1010 . 0 . 1 .... @2reg_shll_b # VCVT fixed<->float conversions -# TODO: FP16 fixed<->float conversions are opc==0b1100 and 0b1101 +VCVT_SH_2sh 1111 001 0 1 . ...... .... 1100 0 . . 1 .... @2reg_vcvt_f16 +VCVT_UH_2sh 1111 001 1 1 . ...... .... 1100 0 . . 1 .... @2reg_vcvt_f16 +VCVT_HS_2sh 1111 001 0 1 . ...... .... 1101 0 . . 1 .... @2reg_vcvt_f16 +VCVT_HU_2sh 1111 001 1 1 . ...... .... 1101 0 . . 1 .... @2reg_vcvt_f16 + VCVT_SF_2sh 1111 001 0 1 . ...... .... 1110 0 . . 1 .... @2reg_vcvt VCVT_UF_2sh 1111 001 1 1 . ...... .... 1110 0 . . 1 .... @2reg_vcvt VCVT_FS_2sh 1111 001 0 1 . ...... .... 1111 0 . . 1 .... @2reg_vcvt diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -XXX,XX +XXX,XX @@ DO_VCVT_FIXED(gvec_vcvt_sf, helper_vfp_sltos, uint32_t) DO_VCVT_FIXED(gvec_vcvt_uf, helper_vfp_ultos, uint32_t) DO_VCVT_FIXED(gvec_vcvt_fs, helper_vfp_tosls_round_to_zero, uint32_t) DO_VCVT_FIXED(gvec_vcvt_fu, helper_vfp_touls_round_to_zero, uint32_t) +DO_VCVT_FIXED(gvec_vcvt_sh, helper_vfp_shtoh, uint16_t) +DO_VCVT_FIXED(gvec_vcvt_uh, helper_vfp_uhtoh, uint16_t) +DO_VCVT_FIXED(gvec_vcvt_hs, helper_vfp_toshh_round_to_zero, uint16_t) +DO_VCVT_FIXED(gvec_vcvt_hu, helper_vfp_touhh_round_to_zero, uint16_t) #undef DO_VCVT_FIXED diff --git a/target/arm/translate-neon.c.inc b/target/arm/translate-neon.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-neon.c.inc +++ b/target/arm/translate-neon.c.inc @@ -XXX,XX +XXX,XX @@ DO_FP_2SH(VCVT_UF, gen_helper_gvec_vcvt_uf) DO_FP_2SH(VCVT_FS, gen_helper_gvec_vcvt_fs) DO_FP_2SH(VCVT_FU, gen_helper_gvec_vcvt_fu) +DO_FP_2SH(VCVT_SH, gen_helper_gvec_vcvt_sh) +DO_FP_2SH(VCVT_UH, gen_helper_gvec_vcvt_uh) +DO_FP_2SH(VCVT_HS, gen_helper_gvec_vcvt_hs) +DO_FP_2SH(VCVT_HU, gen_helper_gvec_vcvt_hu) + static uint64_t asimd_imm_const(uint32_t imm, int cmode, int op) { /* -- 2.20.1 Convert the Neon VCVT with-specified-rounding-mode instructions to gvec, and use this to implement fp16 support for them. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-40-peter.maydell@linaro.org --- target/arm/helper.h | 5 ++ target/arm/vec_helper.c | 23 +++++++ target/arm/translate-neon.c.inc | 105 ++++++++++++-------------------- 3 files changed, 66 insertions(+), 67 deletions(-) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_4(gvec_vcvt_uh, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_vcvt_hs, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_vcvt_hu, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_vcvt_rm_ss, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_vcvt_rm_us, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_vcvt_rm_sh, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_vcvt_rm_uh, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_4(gvec_frecpe_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_frecpe_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_frecpe_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -XXX,XX +XXX,XX @@ DO_VCVT_FIXED(gvec_vcvt_hs, helper_vfp_toshh_round_to_zero, uint16_t) DO_VCVT_FIXED(gvec_vcvt_hu, helper_vfp_touhh_round_to_zero, uint16_t) #undef DO_VCVT_FIXED + +#define DO_VCVT_RMODE(NAME, FUNC, TYPE) \ + void HELPER(NAME)(void *vd, void *vn, void *stat, uint32_t desc) \ + { \ + float_status *fpst = stat; \ + intptr_t i, oprsz = simd_oprsz(desc); \ + uint32_t rmode = simd_data(desc); \ + uint32_t prev_rmode = get_float_rounding_mode(fpst); \ + TYPE *d = vd, *n = vn; \ + set_float_rounding_mode(rmode, fpst); \ + for (i = 0; i < oprsz / sizeof(TYPE); i++) { \ + d[i] = FUNC(n[i], 0, fpst); \ + } \ + set_float_rounding_mode(prev_rmode, fpst); \ + clear_tail(d, oprsz, simd_maxsz(desc)); \ + } + +DO_VCVT_RMODE(gvec_vcvt_rm_ss, helper_vfp_tosls, uint32_t) +DO_VCVT_RMODE(gvec_vcvt_rm_us, helper_vfp_touls, uint32_t) +DO_VCVT_RMODE(gvec_vcvt_rm_sh, helper_vfp_toshh, uint16_t) +DO_VCVT_RMODE(gvec_vcvt_rm_uh, helper_vfp_touhh, uint16_t) + +#undef DO_VCVT_RMODE diff --git a/target/arm/translate-neon.c.inc b/target/arm/translate-neon.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-neon.c.inc +++ b/target/arm/translate-neon.c.inc @@ -XXX,XX +XXX,XX @@ DO_VRINT(VRINTZ, FPROUNDING_ZERO) DO_VRINT(VRINTM, FPROUNDING_NEGINF) DO_VRINT(VRINTP, FPROUNDING_POSINF) -static bool do_vcvt(DisasContext *s, arg_2misc *a, int rmode, bool is_signed) -{ - /* - * Handle a VCVT* operation by iterating 32 bits at a time, - * with a specified rounding mode in operation. - */ - int pass; - TCGv_ptr fpst; - TCGv_i32 tcg_rmode, tcg_shift; - - if (!arm_dc_feature(s, ARM_FEATURE_NEON) || - !arm_dc_feature(s, ARM_FEATURE_V8)) { - return false; +#define DO_VEC_RMODE(INSN, RMODE, OP) \ + static void gen_##INSN(unsigned vece, uint32_t rd_ofs, \ + uint32_t rm_ofs, \ + uint32_t oprsz, uint32_t maxsz) \ + { \ + static gen_helper_gvec_2_ptr * const fns[4] = { \ + NULL, \ + gen_helper_gvec_##OP##h, \ + gen_helper_gvec_##OP##s, \ + NULL, \ + }; \ + TCGv_ptr fpst; \ + fpst = fpstatus_ptr(vece == 1 ? FPST_STD_F16 : FPST_STD); \ + tcg_gen_gvec_2_ptr(rd_ofs, rm_ofs, fpst, oprsz, maxsz, \ + arm_rmode_to_sf(RMODE), fns[vece]); \ + tcg_temp_free_ptr(fpst); \ + } \ + static bool trans_##INSN(DisasContext *s, arg_2misc *a) \ + { \ + if (!arm_dc_feature(s, ARM_FEATURE_V8)) { \ + return false; \ + } \ + if (a->size == MO_16) { \ + if (!dc_isar_feature(aa32_fp16_arith, s)) { \ + return false; \ + } \ + } else if (a->size != MO_32) { \ + return false; \ + } \ + return do_2misc_vec(s, a, gen_##INSN); \ } - /* UNDEF accesses to D16-D31 if they don't exist. */ - if (!dc_isar_feature(aa32_simd_r32, s) && - ((a->vd | a->vm) & 0x10)) { - return false; - } - - if (a->size != 2) { - /* TODO: FP16 will be the size == 1 case */ - return false; - } - - if ((a->vd | a->vm) & a->q) { - return false; - } - - if (!vfp_access_check(s)) { - return true; - } - - fpst = fpstatus_ptr(FPST_STD); - tcg_shift = tcg_const_i32(0); - tcg_rmode = tcg_const_i32(arm_rmode_to_sf(rmode)); - gen_helper_set_neon_rmode(tcg_rmode, tcg_rmode, cpu_env); - for (pass = 0; pass < (a->q ? 4 : 2); pass++) { - TCGv_i32 tmp = neon_load_reg(a->vm, pass); - if (is_signed) { - gen_helper_vfp_tosls(tmp, tmp, tcg_shift, fpst); - } else { - gen_helper_vfp_touls(tmp, tmp, tcg_shift, fpst); - } - neon_store_reg(a->vd, pass, tmp); - } - gen_helper_set_neon_rmode(tcg_rmode, tcg_rmode, cpu_env); - tcg_temp_free_i32(tcg_rmode); - tcg_temp_free_i32(tcg_shift); - tcg_temp_free_ptr(fpst); - - return true; -} - -#define DO_VCVT(INSN, RMODE, SIGNED) \ - static bool trans_##INSN(DisasContext *s, arg_2misc *a) \ - { \ - return do_vcvt(s, a, RMODE, SIGNED); \ - } - -DO_VCVT(VCVTAU, FPROUNDING_TIEAWAY, false) -DO_VCVT(VCVTAS, FPROUNDING_TIEAWAY, true) -DO_VCVT(VCVTNU, FPROUNDING_TIEEVEN, false) -DO_VCVT(VCVTNS, FPROUNDING_TIEEVEN, true) -DO_VCVT(VCVTPU, FPROUNDING_POSINF, false) -DO_VCVT(VCVTPS, FPROUNDING_POSINF, true) -DO_VCVT(VCVTMU, FPROUNDING_NEGINF, false) -DO_VCVT(VCVTMS, FPROUNDING_NEGINF, true) +DO_VEC_RMODE(VCVTAU, FPROUNDING_TIEAWAY, vcvt_rm_u) +DO_VEC_RMODE(VCVTAS, FPROUNDING_TIEAWAY, vcvt_rm_s) +DO_VEC_RMODE(VCVTNU, FPROUNDING_TIEEVEN, vcvt_rm_u) +DO_VEC_RMODE(VCVTNS, FPROUNDING_TIEEVEN, vcvt_rm_s) +DO_VEC_RMODE(VCVTPU, FPROUNDING_POSINF, vcvt_rm_u) +DO_VEC_RMODE(VCVTPS, FPROUNDING_POSINF, vcvt_rm_s) +DO_VEC_RMODE(VCVTMU, FPROUNDING_NEGINF, vcvt_rm_u) +DO_VEC_RMODE(VCVTMS, FPROUNDING_NEGINF, vcvt_rm_s) static bool trans_VSWP(DisasContext *s, arg_2misc *a) { -- 2.20.1 Convert the Neon VRINT-with-specified-rounding-mode insns to gvec, and use this to implement the fp16 versions. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-41-peter.maydell@linaro.org --- target/arm/helper.h | 4 +- target/arm/vec_helper.c | 21 +++++++++++ target/arm/vfp_helper.c | 17 --------- target/arm/translate-neon.c.inc | 67 +++------------------------------ 4 files changed, 30 insertions(+), 79 deletions(-) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_3(vfp_sqtoh, f16, i64, i32, ptr) DEF_HELPER_3(vfp_uqtoh, f16, i64, i32, ptr) DEF_HELPER_FLAGS_2(set_rmode, TCG_CALL_NO_RWG, i32, i32, ptr) -DEF_HELPER_FLAGS_2(set_neon_rmode, TCG_CALL_NO_RWG, i32, i32, env) DEF_HELPER_FLAGS_3(vfp_fcvt_f16_to_f32, TCG_CALL_NO_RWG, f32, f16, ptr, i32) DEF_HELPER_FLAGS_3(vfp_fcvt_f32_to_f16, TCG_CALL_NO_RWG, f16, f32, ptr, i32) @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_4(gvec_vcvt_rm_us, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_vcvt_rm_sh, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_vcvt_rm_uh, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_vrint_rm_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_vrint_rm_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_4(gvec_frecpe_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_frecpe_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_frecpe_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -XXX,XX +XXX,XX @@ DO_VCVT_RMODE(gvec_vcvt_rm_sh, helper_vfp_toshh, uint16_t) DO_VCVT_RMODE(gvec_vcvt_rm_uh, helper_vfp_touhh, uint16_t) #undef DO_VCVT_RMODE + +#define DO_VRINT_RMODE(NAME, FUNC, TYPE) \ + void HELPER(NAME)(void *vd, void *vn, void *stat, uint32_t desc) \ + { \ + float_status *fpst = stat; \ + intptr_t i, oprsz = simd_oprsz(desc); \ + uint32_t rmode = simd_data(desc); \ + uint32_t prev_rmode = get_float_rounding_mode(fpst); \ + TYPE *d = vd, *n = vn; \ + set_float_rounding_mode(rmode, fpst); \ + for (i = 0; i < oprsz / sizeof(TYPE); i++) { \ + d[i] = FUNC(n[i], fpst); \ + } \ + set_float_rounding_mode(prev_rmode, fpst); \ + clear_tail(d, oprsz, simd_maxsz(desc)); \ + } + +DO_VRINT_RMODE(gvec_vrint_rm_h, helper_rinth, uint16_t) +DO_VRINT_RMODE(gvec_vrint_rm_s, helper_rints, uint32_t) + +#undef DO_VRINT_RMODE diff --git a/target/arm/vfp_helper.c b/target/arm/vfp_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vfp_helper.c +++ b/target/arm/vfp_helper.c @@ -XXX,XX +XXX,XX @@ uint32_t HELPER(set_rmode)(uint32_t rmode, void *fpstp) return prev_rmode; } -/* Set the current fp rounding mode in the standard fp status and return - * the old one. This is for NEON instructions that need to change the - * rounding mode but wish to use the standard FPSCR values for everything - * else. Always set the rounding mode back to the correct value after - * modifying it. - * The argument is a softfloat float_round_ value. - */ -uint32_t HELPER(set_neon_rmode)(uint32_t rmode, CPUARMState *env) -{ - float_status *fp_status = &env->vfp.standard_fp_status; - - uint32_t prev_rmode = get_float_rounding_mode(fp_status); - set_float_rounding_mode(rmode, fp_status); - - return prev_rmode; -} - /* Half precision conversions. */ float32 HELPER(vfp_fcvt_f16_to_f32)(uint32_t a, void *fpstp, uint32_t ahp_mode) { diff --git a/target/arm/translate-neon.c.inc b/target/arm/translate-neon.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-neon.c.inc +++ b/target/arm/translate-neon.c.inc @@ -XXX,XX +XXX,XX @@ static bool trans_VRINTX(DisasContext *s, arg_2misc *a) return do_2misc_fp(s, a, gen_helper_rints_exact); } -static bool do_vrint(DisasContext *s, arg_2misc *a, int rmode) -{ - /* - * Handle a VRINT* operation by iterating 32 bits at a time, - * with a specified rounding mode in operation. - */ - int pass; - TCGv_ptr fpst; - TCGv_i32 tcg_rmode; - - if (!arm_dc_feature(s, ARM_FEATURE_NEON) || - !arm_dc_feature(s, ARM_FEATURE_V8)) { - return false; - } - - /* UNDEF accesses to D16-D31 if they don't exist. */ - if (!dc_isar_feature(aa32_simd_r32, s) && - ((a->vd | a->vm) & 0x10)) { - return false; - } - - if (a->size != 2) { - /* TODO: FP16 will be the size == 1 case */ - return false; - } - - if ((a->vd | a->vm) & a->q) { - return false; - } - - if (!vfp_access_check(s)) { - return true; - } - - fpst = fpstatus_ptr(FPST_STD); - tcg_rmode = tcg_const_i32(arm_rmode_to_sf(rmode)); - gen_helper_set_neon_rmode(tcg_rmode, tcg_rmode, cpu_env); - for (pass = 0; pass < (a->q ? 4 : 2); pass++) { - TCGv_i32 tmp = neon_load_reg(a->vm, pass); - gen_helper_rints(tmp, tmp, fpst); - neon_store_reg(a->vd, pass, tmp); - } - gen_helper_set_neon_rmode(tcg_rmode, tcg_rmode, cpu_env); - tcg_temp_free_i32(tcg_rmode); - tcg_temp_free_ptr(fpst); - - return true; -} - -#define DO_VRINT(INSN, RMODE) \ - static bool trans_##INSN(DisasContext *s, arg_2misc *a) \ - { \ - return do_vrint(s, a, RMODE); \ - } - -DO_VRINT(VRINTN, FPROUNDING_TIEEVEN) -DO_VRINT(VRINTA, FPROUNDING_TIEAWAY) -DO_VRINT(VRINTZ, FPROUNDING_ZERO) -DO_VRINT(VRINTM, FPROUNDING_NEGINF) -DO_VRINT(VRINTP, FPROUNDING_POSINF) - #define DO_VEC_RMODE(INSN, RMODE, OP) \ static void gen_##INSN(unsigned vece, uint32_t rd_ofs, \ uint32_t rm_ofs, \ @@ -XXX,XX +XXX,XX @@ DO_VEC_RMODE(VCVTPS, FPROUNDING_POSINF, vcvt_rm_s) DO_VEC_RMODE(VCVTMU, FPROUNDING_NEGINF, vcvt_rm_u) DO_VEC_RMODE(VCVTMS, FPROUNDING_NEGINF, vcvt_rm_s) +DO_VEC_RMODE(VRINTN, FPROUNDING_TIEEVEN, vrint_rm_) +DO_VEC_RMODE(VRINTA, FPROUNDING_TIEAWAY, vrint_rm_) +DO_VEC_RMODE(VRINTZ, FPROUNDING_ZERO, vrint_rm_) +DO_VEC_RMODE(VRINTM, FPROUNDING_NEGINF, vrint_rm_) +DO_VEC_RMODE(VRINTP, FPROUNDING_POSINF, vrint_rm_) + static bool trans_VSWP(DisasContext *s, arg_2misc *a) { TCGv_i64 rm, rd; -- 2.20.1 Convert the Neon VRINTX insn to use gvec, and use this to implement fp16 support for it. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-42-peter.maydell@linaro.org --- target/arm/helper.h | 3 +++ target/arm/vec_helper.c | 3 +++ target/arm/translate-neon.c.inc | 45 +++------------------------------ 3 files changed, 9 insertions(+), 42 deletions(-) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_4(gvec_vcvt_rm_uh, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_vrint_rm_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_vrint_rm_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_vrintx_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_vrintx_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_4(gvec_frecpe_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_frecpe_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_frecpe_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -XXX,XX +XXX,XX @@ DO_2OP(gvec_frsqrte_h, helper_rsqrte_f16, float16) DO_2OP(gvec_frsqrte_s, helper_rsqrte_f32, float32) DO_2OP(gvec_frsqrte_d, helper_rsqrte_f64, float64) +DO_2OP(gvec_vrintx_h, float16_round_to_int, float16) +DO_2OP(gvec_vrintx_s, float32_round_to_int, float32) + DO_2OP(gvec_sitos, helper_vfp_sitos, int32_t) DO_2OP(gvec_uitos, helper_vfp_uitos, uint32_t) DO_2OP(gvec_tosizs, helper_vfp_tosizs, float32) diff --git a/target/arm/translate-neon.c.inc b/target/arm/translate-neon.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-neon.c.inc +++ b/target/arm/translate-neon.c.inc @@ -XXX,XX +XXX,XX @@ static bool trans_VQNEG(DisasContext *s, arg_2misc *a) return do_2misc(s, a, fn[a->size]); } -static bool do_2misc_fp(DisasContext *s, arg_2misc *a, - NeonGenOneSingleOpFn *fn) -{ - int pass; - TCGv_ptr fpst; - - /* Handle a 2-reg-misc operation by iterating 32 bits at a time */ - if (!arm_dc_feature(s, ARM_FEATURE_NEON)) { - return false; - } - - /* UNDEF accesses to D16-D31 if they don't exist. */ - if (!dc_isar_feature(aa32_simd_r32, s) && - ((a->vd | a->vm) & 0x10)) { - return false; - } - - if (a->size != 2) { - /* TODO: FP16 will be the size == 1 case */ - return false; - } - - if ((a->vd | a->vm) & a->q) { - return false; - } - - if (!vfp_access_check(s)) { - return true; - } - - fpst = fpstatus_ptr(FPST_STD); - for (pass = 0; pass < (a->q ? 4 : 2); pass++) { - TCGv_i32 tmp = neon_load_reg(a->vm, pass); - fn(tmp, tmp, fpst); - neon_store_reg(a->vd, pass, tmp); - } - tcg_temp_free_ptr(fpst); - - return true; -} - #define DO_2MISC_FP_VEC(INSN, HFUNC, SFUNC) \ static void gen_##INSN(unsigned vece, uint32_t rd_ofs, \ uint32_t rm_ofs, \ @@ -XXX,XX +XXX,XX @@ DO_2MISC_FP_VEC(VCVT_FU, gen_helper_gvec_ustoh, gen_helper_gvec_uitos) DO_2MISC_FP_VEC(VCVT_SF, gen_helper_gvec_tosszh, gen_helper_gvec_tosizs) DO_2MISC_FP_VEC(VCVT_UF, gen_helper_gvec_touszh, gen_helper_gvec_touizs) +DO_2MISC_FP_VEC(VRINTX_impl, gen_helper_gvec_vrintx_h, gen_helper_gvec_vrintx_s) + static bool trans_VRINTX(DisasContext *s, arg_2misc *a) { if (!arm_dc_feature(s, ARM_FEATURE_V8)) { return false; } - return do_2misc_fp(s, a, gen_helper_rints_exact); + return trans_VRINTX_impl(s, a); } #define DO_VEC_RMODE(INSN, RMODE, OP) \ -- 2.20.1 In the gvec helper functions for indexed operations, for AArch32 Neon the oprsz (total size of the vector) can be less than 16 bytes if the operation is on a D reg. Since the inner loop in these helpers always goes from 0 to segment, we must clamp it based on oprsz to avoid processing a full 16 byte segment when asked to handle an 8 byte wide vector. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-43-peter.maydell@linaro.org --- target/arm/vec_helper.c | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -XXX,XX +XXX,XX @@ DO_MULADD(gvec_vfms_s, float32_mulsub_f, float32) #define DO_MUL_IDX(NAME, TYPE, H) \ void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc) \ { \ - intptr_t i, j, oprsz = simd_oprsz(desc), segment = 16 / sizeof(TYPE); \ + intptr_t i, j, oprsz = simd_oprsz(desc); \ + intptr_t segment = MIN(16, oprsz) / sizeof(TYPE); \ intptr_t idx = simd_data(desc); \ TYPE *d = vd, *n = vn, *m = vm; \ for (i = 0; i < oprsz / sizeof(TYPE); i += segment) { \ @@ -XXX,XX +XXX,XX @@ DO_MUL_IDX(gvec_mul_idx_d, uint64_t, ) #define DO_MLA_IDX(NAME, TYPE, OP, H) \ void HELPER(NAME)(void *vd, void *vn, void *vm, void *va, uint32_t desc) \ { \ - intptr_t i, j, oprsz = simd_oprsz(desc), segment = 16 / sizeof(TYPE); \ + intptr_t i, j, oprsz = simd_oprsz(desc); \ + intptr_t segment = MIN(16, oprsz) / sizeof(TYPE); \ intptr_t idx = simd_data(desc); \ TYPE *d = vd, *n = vn, *m = vm, *a = va; \ for (i = 0; i < oprsz / sizeof(TYPE); i += segment) { \ @@ -XXX,XX +XXX,XX @@ DO_MLA_IDX(gvec_mls_idx_d, uint64_t, -, ) #define DO_FMUL_IDX(NAME, TYPE, H) \ void HELPER(NAME)(void *vd, void *vn, void *vm, void *stat, uint32_t desc) \ { \ - intptr_t i, j, oprsz = simd_oprsz(desc), segment = 16 / sizeof(TYPE); \ + intptr_t i, j, oprsz = simd_oprsz(desc); \ + intptr_t segment = MIN(16, oprsz) / sizeof(TYPE); \ intptr_t idx = simd_data(desc); \ TYPE *d = vd, *n = vn, *m = vm; \ for (i = 0; i < oprsz / sizeof(TYPE); i += segment) { \ @@ -XXX,XX +XXX,XX @@ DO_FMUL_IDX(gvec_fmul_idx_d, float64, ) void HELPER(NAME)(void *vd, void *vn, void *vm, void *va, \ void *stat, uint32_t desc) \ { \ - intptr_t i, j, oprsz = simd_oprsz(desc), segment = 16 / sizeof(TYPE); \ + intptr_t i, j, oprsz = simd_oprsz(desc); \ + intptr_t segment = MIN(16, oprsz) / sizeof(TYPE); \ TYPE op1_neg = extract32(desc, SIMD_DATA_SHIFT, 1); \ intptr_t idx = desc >> (SIMD_DATA_SHIFT + 1); \ TYPE *d = vd, *n = vn, *m = vm, *a = va; \ -- 2.20.1 Add gvec helpers for doing Neon-style indexed non-fused fp multiply-and-accumulate operations. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Message-id: 20200828183354.27913-44-peter.maydell@linaro.org --- target/arm/helper.h | 10 ++++++++++ target/arm/vec_helper.c | 27 ++++++++++++++++++++++----- 2 files changed, 32 insertions(+), 5 deletions(-) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_5(gvec_fmul_idx_s, TCG_CALL_NO_RWG, DEF_HELPER_FLAGS_5(gvec_fmul_idx_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_fmla_nf_idx_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_fmla_nf_idx_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(gvec_fmls_nf_idx_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_fmls_nf_idx_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_6(gvec_fmla_idx_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_6(gvec_fmla_idx_s, TCG_CALL_NO_RWG, diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -XXX,XX +XXX,XX @@ DO_MLA_IDX(gvec_mls_idx_d, uint64_t, -, ) #undef DO_MLA_IDX -#define DO_FMUL_IDX(NAME, TYPE, H) \ +#define DO_FMUL_IDX(NAME, ADD, TYPE, H) \ void HELPER(NAME)(void *vd, void *vn, void *vm, void *stat, uint32_t desc) \ { \ intptr_t i, j, oprsz = simd_oprsz(desc); \ @@ -XXX,XX +XXX,XX @@ void HELPER(NAME)(void *vd, void *vn, void *vm, void *stat, uint32_t desc) \ for (i = 0; i < oprsz / sizeof(TYPE); i += segment) { \ TYPE mm = m[H(i + idx)]; \ for (j = 0; j < segment; j++) { \ - d[i + j] = TYPE##_mul(n[i + j], mm, stat); \ + d[i + j] = TYPE##_##ADD(d[i + j], \ + TYPE##_mul(n[i + j], mm, stat), stat); \ } \ } \ clear_tail(d, oprsz, simd_maxsz(desc)); \ } -DO_FMUL_IDX(gvec_fmul_idx_h, float16, H2) -DO_FMUL_IDX(gvec_fmul_idx_s, float32, H4) -DO_FMUL_IDX(gvec_fmul_idx_d, float64, ) +#define float16_nop(N, M, S) (M) +#define float32_nop(N, M, S) (M) +#define float64_nop(N, M, S) (M) +DO_FMUL_IDX(gvec_fmul_idx_h, nop, float16, H2) +DO_FMUL_IDX(gvec_fmul_idx_s, nop, float32, H4) +DO_FMUL_IDX(gvec_fmul_idx_d, nop, float64, ) + +/* + * Non-fused multiply-accumulate operations, for Neon. NB that unlike + * the fused ops below they assume accumulate both from and into Vd. + */ +DO_FMUL_IDX(gvec_fmla_nf_idx_h, add, float16, H2) +DO_FMUL_IDX(gvec_fmla_nf_idx_s, add, float32, H4) +DO_FMUL_IDX(gvec_fmls_nf_idx_h, sub, float16, H2) +DO_FMUL_IDX(gvec_fmls_nf_idx_s, sub, float32, H4) + +#undef float16_nop +#undef float32_nop +#undef float64_nop #undef DO_FMUL_IDX #define DO_FMLA_IDX(NAME, TYPE, H) \ -- 2.20.1 Convert the Neon floating-point VMUL, VMLA and VMLS to use gvec, and use this to implement fp16 support. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-45-peter.maydell@linaro.org --- target/arm/translate-neon.c.inc | 114 ++++++++++++++++---------------- 1 file changed, 57 insertions(+), 57 deletions(-) diff --git a/target/arm/translate-neon.c.inc b/target/arm/translate-neon.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-neon.c.inc +++ b/target/arm/translate-neon.c.inc @@ -XXX,XX +XXX,XX @@ static bool trans_VMLS_2sc(DisasContext *s, arg_2scalar *a) return do_2scalar(s, a, opfn[a->size], accfn[a->size]); } -/* - * Rather than have a float-specific version of do_2scalar just for - * three insns, we wrap a NeonGenTwoSingleOpFn to turn it into - * a NeonGenTwoOpFn. - */ -#define WRAP_FP_FN(WRAPNAME, FUNC) \ - static void WRAPNAME(TCGv_i32 rd, TCGv_i32 rn, TCGv_i32 rm) \ - { \ - TCGv_ptr fpstatus = fpstatus_ptr(FPST_STD); \ - FUNC(rd, rn, rm, fpstatus); \ - tcg_temp_free_ptr(fpstatus); \ +static bool do_2scalar_fp_vec(DisasContext *s, arg_2scalar *a, + gen_helper_gvec_3_ptr *fn) +{ + /* Two registers and a scalar, using gvec */ + int vec_size = a->q ? 16 : 8; + int rd_ofs = neon_reg_offset(a->vd, 0); + int rn_ofs = neon_reg_offset(a->vn, 0); + int rm_ofs; + int idx; + TCGv_ptr fpstatus; + + if (!arm_dc_feature(s, ARM_FEATURE_NEON)) { + return false; } -WRAP_FP_FN(gen_VMUL_F_mul, gen_helper_vfp_muls) -WRAP_FP_FN(gen_VMUL_F_add, gen_helper_vfp_adds) -WRAP_FP_FN(gen_VMUL_F_sub, gen_helper_vfp_subs) + /* UNDEF accesses to D16-D31 if they don't exist. */ + if (!dc_isar_feature(aa32_simd_r32, s) && + ((a->vd | a->vn | a->vm) & 0x10)) { + return false; + } -static bool trans_VMUL_F_2sc(DisasContext *s, arg_2scalar *a) -{ - static NeonGenTwoOpFn * const opfn[] = { - NULL, - NULL, /* TODO: fp16 support */ - gen_VMUL_F_mul, - NULL, - }; + if (!fn) { + /* Bad size (including size == 3, which is a different insn group) */ + return false; + } - return do_2scalar(s, a, opfn[a->size], NULL); + if (a->q && ((a->vd | a->vn) & 1)) { + return false; + } + + if (!vfp_access_check(s)) { + return true; + } + + /* a->vm is M:Vm, which encodes both register and index */ + idx = extract32(a->vm, a->size + 2, 2); + a->vm = extract32(a->vm, 0, a->size + 2); + rm_ofs = neon_reg_offset(a->vm, 0); + + fpstatus = fpstatus_ptr(a->size == 1 ? FPST_STD_F16 : FPST_STD); + tcg_gen_gvec_3_ptr(rd_ofs, rn_ofs, rm_ofs, fpstatus, + vec_size, vec_size, idx, fn); + tcg_temp_free_ptr(fpstatus); + return true; } -static bool trans_VMLA_F_2sc(DisasContext *s, arg_2scalar *a) -{ - static NeonGenTwoOpFn * const opfn[] = { - NULL, - NULL, /* TODO: fp16 support */ - gen_VMUL_F_mul, - NULL, - }; - static NeonGenTwoOpFn * const accfn[] = { - NULL, - NULL, /* TODO: fp16 support */ - gen_VMUL_F_add, - NULL, - }; +#define DO_VMUL_F_2sc(NAME, FUNC) \ + static bool trans_##NAME##_F_2sc(DisasContext *s, arg_2scalar *a) \ + { \ + static gen_helper_gvec_3_ptr * const opfn[] = { \ + NULL, \ + gen_helper_##FUNC##_h, \ + gen_helper_##FUNC##_s, \ + NULL, \ + }; \ + if (a->size == MO_16 && !dc_isar_feature(aa32_fp16_arith, s)) { \ + return false; \ + } \ + return do_2scalar_fp_vec(s, a, opfn[a->size]); \ + } - return do_2scalar(s, a, opfn[a->size], accfn[a->size]); -} - -static bool trans_VMLS_F_2sc(DisasContext *s, arg_2scalar *a) -{ - static NeonGenTwoOpFn * const opfn[] = { - NULL, - NULL, /* TODO: fp16 support */ - gen_VMUL_F_mul, - NULL, - }; - static NeonGenTwoOpFn * const accfn[] = { - NULL, - NULL, /* TODO: fp16 support */ - gen_VMUL_F_sub, - NULL, - }; - - return do_2scalar(s, a, opfn[a->size], accfn[a->size]); -} +DO_VMUL_F_2sc(VMUL, gvec_fmul_idx) +DO_VMUL_F_2sc(VMLA, gvec_fmla_nf_idx) +DO_VMUL_F_2sc(VMLS, gvec_fmls_nf_idx) WRAP_ENV_FN(gen_VQDMULH_16, gen_helper_neon_qdmulh_s16) WRAP_ENV_FN(gen_VQDMULH_32, gen_helper_neon_qdmulh_s32) -- 2.20.1 Set the MVFR1 ID register FPHP and SIMDHP fields to indicate that our "-cpu max" has v8.2-FP16. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-46-peter.maydell@linaro.org --- target/arm/cpu.c | 3 ++- target/arm/cpu64.c | 10 ++++------ 2 files changed, 6 insertions(+), 7 deletions(-) diff --git a/target/arm/cpu.c b/target/arm/cpu.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpu.c +++ b/target/arm/cpu.c @@ -XXX,XX +XXX,XX @@ static void arm_max_initfn(Object *obj) cpu->isar.id_isar6 = t; t = cpu->isar.mvfr1; - t = FIELD_DP32(t, MVFR1, FPHP, 2); /* v8.0 FP support */ + t = FIELD_DP32(t, MVFR1, FPHP, 3); /* v8.2-FP16 */ + t = FIELD_DP32(t, MVFR1, SIMDHP, 2); /* v8.2-FP16 */ cpu->isar.mvfr1 = t; t = cpu->isar.mvfr2; diff --git a/target/arm/cpu64.c b/target/arm/cpu64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpu64.c +++ b/target/arm/cpu64.c @@ -XXX,XX +XXX,XX @@ static void aarch64_max_initfn(Object *obj) u = FIELD_DP32(u, ID_DFR0, PERFMON, 5); /* v8.4-PMU */ cpu->isar.id_dfr0 = u; - /* - * FIXME: We do not yet support ARMv8.2-fp16 for AArch32 yet, - * so do not set MVFR1.FPHP. Strictly speaking this is not legal, - * but it is also not legal to enable SVE without support for FP16, - * and enabling SVE in system mode is more useful in the short term. - */ + u = cpu->isar.mvfr1; + u = FIELD_DP32(u, MVFR1, FPHP, 3); /* v8.2-FP16 */ + u = FIELD_DP32(u, MVFR1, SIMDHP, 2); /* v8.2-FP16 */ + cpu->isar.mvfr1 = u; #ifdef CONFIG_USER_ONLY /* For usermode -cpu max we can use a larger and more efficient DCZ -- 2.20.1 From: Leif Lindholm <leif@nuviainc.com> The sbsa-ref platform uses a minimal device tree to pass amount of memory as well as number of cpus to the firmware. However, when dumping that minimal dtb (with -M sbsa-virt,dumpdtb=<file>), the resulting blob generates a warning when decompiled by dtc due to lack of reg property. Add a simple reg property per cpu, representing a 64-bit MPIDR_EL1. This also ends up being cleaner than having the firmware calculating its own IDs for generating APCI. Signed-off-by: Leif Lindholm <leif@nuviainc.com> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Message-id: 20200827124335.30586-1-leif@nuviainc.com Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- hw/arm/sbsa-ref.c | 29 +++++++++++++++++++++++------ 1 file changed, 23 insertions(+), 6 deletions(-) diff --git a/hw/arm/sbsa-ref.c b/hw/arm/sbsa-ref.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/sbsa-ref.c +++ b/hw/arm/sbsa-ref.c @@ -XXX,XX +XXX,XX @@ static const int sbsa_ref_irqmap[] = { [SBSA_EHCI] = 11, }; +static uint64_t sbsa_ref_cpu_mp_affinity(SBSAMachineState *sms, int idx) +{ + uint8_t clustersz = ARM_DEFAULT_CPUS_PER_CLUSTER; + return arm_cpu_mp_affinity(idx, clustersz); +} + /* * Firmware on this machine only uses ACPI table to load OS, these limited * device tree nodes are just to let firmware know the info which varies from @@ -XXX,XX +XXX,XX @@ static void create_fdt(SBSAMachineState *sms) g_free(matrix); } + /* + * From Documentation/devicetree/bindings/arm/cpus.yaml + * On ARM v8 64-bit systems this property is required + * and matches the MPIDR_EL1 register affinity bits. + * + * * If cpus node's #address-cells property is set to 2 + * + * The first reg cell bits [7:0] must be set to + * bits [39:32] of MPIDR_EL1. + * + * The second reg cell bits [23:0] must be set to + * bits [23:0] of MPIDR_EL1. + */ qemu_fdt_add_subnode(sms->fdt, "/cpus"); + qemu_fdt_setprop_cell(sms->fdt, "/cpus", "#address-cells", 2); + qemu_fdt_setprop_cell(sms->fdt, "/cpus", "#size-cells", 0x0); for (cpu = sms->smp_cpus - 1; cpu >= 0; cpu--) { char *nodename = g_strdup_printf("/cpus/cpu@%d", cpu); ARMCPU *armcpu = ARM_CPU(qemu_get_cpu(cpu)); CPUState *cs = CPU(armcpu); + uint64_t mpidr = sbsa_ref_cpu_mp_affinity(sms, cpu); qemu_fdt_add_subnode(sms->fdt, nodename); + qemu_fdt_setprop_u64(sms->fdt, nodename, "reg", mpidr); if (ms->possible_cpus->cpus[cs->cpu_index].props.has_node_id) { qemu_fdt_setprop_cell(sms->fdt, nodename, "numa-node-id", @@ -XXX,XX +XXX,XX @@ static void sbsa_ref_init(MachineState *machine) arm_load_kernel(ARM_CPU(first_cpu), machine, &sms->bootinfo); } -static uint64_t sbsa_ref_cpu_mp_affinity(SBSAMachineState *sms, int idx) -{ - uint8_t clustersz = ARM_DEFAULT_CPUS_PER_CLUSTER; - return arm_cpu_mp_affinity(idx, clustersz); -} - static const CPUArchIdList *sbsa_ref_possible_cpu_arch_ids(MachineState *ms) { unsigned int max_cpus = ms->smp.max_cpus; -- 2.20.1 From: Graeme Gregory <graeme@nuviainc.com> A difference between sbsa platform and the virt platform is PSCI is handled by ARM-TF in the sbsa platform. This means that the PSCI code there needs to communicate some of the platform power changes down to the qemu code for things like shutdown/reset control. Space has been left to extend the EC if we find other use cases in future where ARM-TF and qemu need to communicate. Signed-off-by: Graeme Gregory <graeme@nuviainc.com> Reviewed-by: Leif Lindholm <leif@nuviainc.com> Tested-by: Leif Lindholm <leif@nuviainc.com> Message-id: 20200826141952.136164-2-graeme@nuviainc.com Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- hw/misc/sbsa_ec.c | 98 +++++++++++++++++++++++++++++++++++++++++++++ hw/misc/meson.build | 2 + 2 files changed, 100 insertions(+) create mode 100644 hw/misc/sbsa_ec.c diff --git a/hw/misc/sbsa_ec.c b/hw/misc/sbsa_ec.c new file mode 100644 index XXXXXXX..XXXXXXX --- /dev/null +++ b/hw/misc/sbsa_ec.c @@ -XXX,XX +XXX,XX @@ +/* + * ARM SBSA Reference Platform Embedded Controller + * + * A device to allow PSCI running in the secure side of sbsa-ref machine + * to communicate platform power states to qemu. + * + * Copyright (c) 2020 Nuvia Inc + * Written by Graeme Gregory <graeme@nuviainc.com> + * + * SPDX-License-Identifer: GPL-2.0-or-later + */ + +#include "qemu/osdep.h" +#include "qemu-common.h" +#include "qemu/log.h" +#include "hw/sysbus.h" +#include "sysemu/runstate.h" + +typedef struct { + SysBusDevice parent_obj; + MemoryRegion iomem; +} SECUREECState; + +#define TYPE_SBSA_EC "sbsa-ec" +#define SECURE_EC(obj) OBJECT_CHECK(SECUREECState, (obj), TYPE_SBSA_EC) + +enum sbsa_ec_powerstates { + SBSA_EC_CMD_POWEROFF = 0x01, + SBSA_EC_CMD_REBOOT = 0x02, +}; + +static uint64_t sbsa_ec_read(void *opaque, hwaddr offset, unsigned size) +{ + /* No use for this currently */ + qemu_log_mask(LOG_GUEST_ERROR, "sbsa-ec: no readable registers"); + return 0; +} + +static void sbsa_ec_write(void *opaque, hwaddr offset, + uint64_t value, unsigned size) +{ + if (offset == 0) { /* PSCI machine power command register */ + switch (value) { + case SBSA_EC_CMD_POWEROFF: + qemu_system_shutdown_request(SHUTDOWN_CAUSE_GUEST_SHUTDOWN); + break; + case SBSA_EC_CMD_REBOOT: + qemu_system_reset_request(SHUTDOWN_CAUSE_GUEST_RESET); + break; + default: + qemu_log_mask(LOG_GUEST_ERROR, + "sbsa-ec: unknown power command"); + } + } else { + qemu_log_mask(LOG_GUEST_ERROR, "sbsa-ec: unknown EC register"); + } +} + +static const MemoryRegionOps sbsa_ec_ops = { + .read = sbsa_ec_read, + .write = sbsa_ec_write, + .endianness = DEVICE_NATIVE_ENDIAN, + .valid.min_access_size = 4, + .valid.max_access_size = 4, +}; + +static void sbsa_ec_init(Object *obj) +{ + SECUREECState *s = SECURE_EC(obj); + SysBusDevice *dev = SYS_BUS_DEVICE(obj); + + memory_region_init_io(&s->iomem, obj, &sbsa_ec_ops, s, "sbsa-ec", + 0x1000); + sysbus_init_mmio(dev, &s->iomem); +} + +static void sbsa_ec_class_init(ObjectClass *klass, void *data) +{ + DeviceClass *dc = DEVICE_CLASS(klass); + + /* No vmstate or reset required: device has no internal state */ + dc->user_creatable = false; +} + +static const TypeInfo sbsa_ec_info = { + .name = TYPE_SBSA_EC, + .parent = TYPE_SYS_BUS_DEVICE, + .instance_size = sizeof(SECUREECState), + .instance_init = sbsa_ec_init, + .class_init = sbsa_ec_class_init, +}; + +static void sbsa_ec_register_type(void) +{ + type_register_static(&sbsa_ec_info); +} + +type_init(sbsa_ec_register_type); diff --git a/hw/misc/meson.build b/hw/misc/meson.build index XXXXXXX..XXXXXXX 100644 --- a/hw/misc/meson.build +++ b/hw/misc/meson.build @@ -XXX,XX +XXX,XX @@ specific_ss.add(when: 'CONFIG_MAC_VIA', if_true: files('mac_via.c')) specific_ss.add(when: 'CONFIG_MIPS_CPS', if_true: files('mips_cmgcr.c', 'mips_cpc.c')) specific_ss.add(when: 'CONFIG_MIPS_ITU', if_true: files('mips_itu.c')) + +specific_ss.add(when: 'CONFIG_SBSA_REF', if_true: files('sbsa_ec.c')) -- 2.20.1 From: Graeme Gregory <graeme@nuviainc.com> Add the previously created sbsa-ec device to the sbsa-ref machine in secure memory so the PSCI implementation in ARM-TF can access it, but not expose it to non secure firmware or OS except by via ARM-TF. Signed-off-by: Graeme Gregory <graeme@nuviainc.com> Reviewed-by: Leif Lindholm <leif@nuviainc.com> Tested-by: Leif Lindholm <leif@nuviainc.com> Message-id: 20200826141952.136164-3-graeme@nuviainc.com Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- hw/arm/sbsa-ref.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/hw/arm/sbsa-ref.c b/hw/arm/sbsa-ref.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/sbsa-ref.c +++ b/hw/arm/sbsa-ref.c @@ -XXX,XX +XXX,XX @@ enum { SBSA_CPUPERIPHS, SBSA_GIC_DIST, SBSA_GIC_REDIST, + SBSA_SECURE_EC, SBSA_SMMU, SBSA_UART, SBSA_RTC, @@ -XXX,XX +XXX,XX @@ static const MemMapEntry sbsa_ref_memmap[] = { [SBSA_CPUPERIPHS] = { 0x40000000, 0x00040000 }, [SBSA_GIC_DIST] = { 0x40060000, 0x00010000 }, [SBSA_GIC_REDIST] = { 0x40080000, 0x04000000 }, + [SBSA_SECURE_EC] = { 0x50000000, 0x00001000 }, [SBSA_UART] = { 0x60000000, 0x00001000 }, [SBSA_RTC] = { 0x60010000, 0x00001000 }, [SBSA_GPIO] = { 0x60020000, 0x00001000 }, @@ -XXX,XX +XXX,XX @@ static void *sbsa_ref_dtb(const struct arm_boot_info *binfo, int *fdt_size) return board->fdt; } +static void create_secure_ec(MemoryRegion *mem) +{ + hwaddr base = sbsa_ref_memmap[SBSA_SECURE_EC].base; + DeviceState *dev = qdev_new("sbsa-ec"); + SysBusDevice *s = SYS_BUS_DEVICE(dev); + + memory_region_add_subregion(mem, base, + sysbus_mmio_get_region(s, 0)); +} + static void sbsa_ref_init(MachineState *machine) { unsigned int smp_cpus = machine->smp.cpus; @@ -XXX,XX +XXX,XX @@ static void sbsa_ref_init(MachineState *machine) create_pcie(sms); + create_secure_ec(secure_sysmem); + sms->bootinfo.ram_size = machine->ram_size; sms->bootinfo.nb_cpus = smp_cpus; sms->bootinfo.board_id = -1; -- 2.20.1 The following changes since commit 8f6330a807f2642dc2a3cdf33347aa28a4c00a87: Merge tag 'pull-maintainer-updates-060324-1' of https://gitlab.com/stsquad/qemu into staging (2024-03-06 16:56:20 +0000) are available in the Git repository at: https://git.linaro.org/people/pmaydell/qemu-arm.git tags/pull-target-arm-20240308 for you to fetch changes up to bbf6c6dbead82292a20951eb1204442a6b838de9: target/arm: Move v7m-related code from cpu32.c into a separate file (2024-03-08 14:45:03 +0000) ---------------------------------------------------------------- target-arm queue: * Implement FEAT_ECV * STM32L4x5: Implement GPIO device * Fix 32-bit SMOPA * Refactor v7m related code from cpu32.c into its own file * hw/rtc/sun4v-rtc: Relicense to GPLv2-or-later ---------------------------------------------------------------- Inès Varhol (3): hw/gpio: Implement STM32L4x5 GPIO hw/arm: Connect STM32L4x5 GPIO to STM32L4x5 SoC tests/qtest: Add STM32L4x5 GPIO QTest testcase Peter Maydell (9): target/arm: Move some register related defines to internals.h target/arm: Timer _EL02 registers UNDEF for E2H == 0 target/arm: use FIELD macro for CNTHCTL bit definitions target/arm: Don't allow RES0 CNTHCTL_EL2 bits to be written target/arm: Implement new FEAT_ECV trap bits target/arm: Define CNTPCTSS_EL0 and CNTVCTSS_EL0 target/arm: Implement FEAT_ECV CNTPOFF_EL2 handling target/arm: Enable FEAT_ECV for 'max' CPU hw/rtc/sun4v-rtc: Relicense to GPLv2-or-later Richard Henderson (1): target/arm: Fix 32-bit SMOPA Thomas Huth (1): target/arm: Move v7m-related code from cpu32.c into a separate file MAINTAINERS | 1 + docs/system/arm/b-l475e-iot01a.rst | 2 +- docs/system/arm/emulation.rst | 1 + include/hw/arm/stm32l4x5_soc.h | 2 + include/hw/gpio/stm32l4x5_gpio.h | 71 +++++ include/hw/misc/stm32l4x5_syscfg.h | 3 +- include/hw/rtc/sun4v-rtc.h | 2 +- target/arm/cpu-features.h | 10 + target/arm/cpu.h | 129 +-------- target/arm/internals.h | 151 ++++++++++ hw/arm/stm32l4x5_soc.c | 71 ++++- hw/gpio/stm32l4x5_gpio.c | 477 ++++++++++++++++++++++++++++++++ hw/misc/stm32l4x5_syscfg.c | 1 + hw/rtc/sun4v-rtc.c | 2 +- target/arm/helper.c | 189 ++++++++++++- target/arm/tcg/cpu-v7m.c | 290 +++++++++++++++++++ target/arm/tcg/cpu32.c | 261 ------------------ target/arm/tcg/cpu64.c | 1 + target/arm/tcg/sme_helper.c | 77 +++--- tests/qtest/stm32l4x5_gpio-test.c | 551 +++++++++++++++++++++++++++++++++++++ tests/tcg/aarch64/sme-smopa-1.c | 47 ++++ tests/tcg/aarch64/sme-smopa-2.c | 54 ++++ hw/arm/Kconfig | 3 +- hw/gpio/Kconfig | 3 + hw/gpio/meson.build | 1 + hw/gpio/trace-events | 6 + target/arm/meson.build | 3 + target/arm/tcg/meson.build | 3 + target/arm/trace-events | 1 + tests/qtest/meson.build | 3 +- tests/tcg/aarch64/Makefile.target | 2 +- 31 files changed, 1962 insertions(+), 456 deletions(-) create mode 100644 include/hw/gpio/stm32l4x5_gpio.h create mode 100644 hw/gpio/stm32l4x5_gpio.c create mode 100644 target/arm/tcg/cpu-v7m.c create mode 100644 tests/qtest/stm32l4x5_gpio-test.c create mode 100644 tests/tcg/aarch64/sme-smopa-1.c create mode 100644 tests/tcg/aarch64/sme-smopa-2.c cpu.h has a lot of #defines relating to CPU register fields. Most of these aren't actually used outside target/arm code, so there's no point in cluttering up the cpu.h file with them. Move some easy ones to internals.h. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20240301183219.2424889-2-peter.maydell@linaro.org --- target/arm/cpu.h | 128 ----------------------------------------- target/arm/internals.h | 128 +++++++++++++++++++++++++++++++++++++++++ 2 files changed, 128 insertions(+), 128 deletions(-) diff --git a/target/arm/cpu.h b/target/arm/cpu.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -XXX,XX +XXX,XX @@ typedef struct ARMGenericTimer { uint64_t ctl; /* Timer Control register */ } ARMGenericTimer; -#define VTCR_NSW (1u << 29) -#define VTCR_NSA (1u << 30) -#define VSTCR_SW VTCR_NSW -#define VSTCR_SA VTCR_NSA - /* Define a maximum sized vector register. * For 32-bit, this is a 128-bit NEON/AdvSIMD register. * For 64-bit, this is a 2048-bit SVE register. @@ -XXX,XX +XXX,XX @@ void pmu_init(ARMCPU *cpu); #define SCTLR_SPINTMASK (1ULL << 62) /* FEAT_NMI */ #define SCTLR_TIDCP (1ULL << 63) /* FEAT_TIDCP1 */ -/* Bit definitions for CPACR (AArch32 only) */ -FIELD(CPACR, CP10, 20, 2) -FIELD(CPACR, CP11, 22, 2) -FIELD(CPACR, TRCDIS, 28, 1) /* matches CPACR_EL1.TTA */ -FIELD(CPACR, D32DIS, 30, 1) /* up to v7; RAZ in v8 */ -FIELD(CPACR, ASEDIS, 31, 1) - -/* Bit definitions for CPACR_EL1 (AArch64 only) */ -FIELD(CPACR_EL1, ZEN, 16, 2) -FIELD(CPACR_EL1, FPEN, 20, 2) -FIELD(CPACR_EL1, SMEN, 24, 2) -FIELD(CPACR_EL1, TTA, 28, 1) /* matches CPACR.TRCDIS */ - -/* Bit definitions for HCPTR (AArch32 only) */ -FIELD(HCPTR, TCP10, 10, 1) -FIELD(HCPTR, TCP11, 11, 1) -FIELD(HCPTR, TASE, 15, 1) -FIELD(HCPTR, TTA, 20, 1) -FIELD(HCPTR, TAM, 30, 1) /* matches CPTR_EL2.TAM */ -FIELD(HCPTR, TCPAC, 31, 1) /* matches CPTR_EL2.TCPAC */ - -/* Bit definitions for CPTR_EL2 (AArch64 only) */ -FIELD(CPTR_EL2, TZ, 8, 1) /* !E2H */ -FIELD(CPTR_EL2, TFP, 10, 1) /* !E2H, matches HCPTR.TCP10 */ -FIELD(CPTR_EL2, TSM, 12, 1) /* !E2H */ -FIELD(CPTR_EL2, ZEN, 16, 2) /* E2H */ -FIELD(CPTR_EL2, FPEN, 20, 2) /* E2H */ -FIELD(CPTR_EL2, SMEN, 24, 2) /* E2H */ -FIELD(CPTR_EL2, TTA, 28, 1) -FIELD(CPTR_EL2, TAM, 30, 1) /* matches HCPTR.TAM */ -FIELD(CPTR_EL2, TCPAC, 31, 1) /* matches HCPTR.TCPAC */ - -/* Bit definitions for CPTR_EL3 (AArch64 only) */ -FIELD(CPTR_EL3, EZ, 8, 1) -FIELD(CPTR_EL3, TFP, 10, 1) -FIELD(CPTR_EL3, ESM, 12, 1) -FIELD(CPTR_EL3, TTA, 20, 1) -FIELD(CPTR_EL3, TAM, 30, 1) -FIELD(CPTR_EL3, TCPAC, 31, 1) - -#define MDCR_MTPME (1U << 28) -#define MDCR_TDCC (1U << 27) -#define MDCR_HLP (1U << 26) /* MDCR_EL2 */ -#define MDCR_SCCD (1U << 23) /* MDCR_EL3 */ -#define MDCR_HCCD (1U << 23) /* MDCR_EL2 */ -#define MDCR_EPMAD (1U << 21) -#define MDCR_EDAD (1U << 20) -#define MDCR_TTRF (1U << 19) -#define MDCR_STE (1U << 18) /* MDCR_EL3 */ -#define MDCR_SPME (1U << 17) /* MDCR_EL3 */ -#define MDCR_HPMD (1U << 17) /* MDCR_EL2 */ -#define MDCR_SDD (1U << 16) -#define MDCR_SPD (3U << 14) -#define MDCR_TDRA (1U << 11) -#define MDCR_TDOSA (1U << 10) -#define MDCR_TDA (1U << 9) -#define MDCR_TDE (1U << 8) -#define MDCR_HPME (1U << 7) -#define MDCR_TPM (1U << 6) -#define MDCR_TPMCR (1U << 5) -#define MDCR_HPMN (0x1fU) - -/* Not all of the MDCR_EL3 bits are present in the 32-bit SDCR */ -#define SDCR_VALID_MASK (MDCR_MTPME | MDCR_TDCC | MDCR_SCCD | \ - MDCR_EPMAD | MDCR_EDAD | MDCR_TTRF | \ - MDCR_STE | MDCR_SPME | MDCR_SPD) - #define CPSR_M (0x1fU) #define CPSR_T (1U << 5) #define CPSR_F (1U << 6) @@ -XXX,XX +XXX,XX @@ FIELD(CPTR_EL3, TCPAC, 31, 1) #define XPSR_NZCV CPSR_NZCV #define XPSR_IT CPSR_IT -#define TTBCR_N (7U << 0) /* TTBCR.EAE==0 */ -#define TTBCR_T0SZ (7U << 0) /* TTBCR.EAE==1 */ -#define TTBCR_PD0 (1U << 4) -#define TTBCR_PD1 (1U << 5) -#define TTBCR_EPD0 (1U << 7) -#define TTBCR_IRGN0 (3U << 8) -#define TTBCR_ORGN0 (3U << 10) -#define TTBCR_SH0 (3U << 12) -#define TTBCR_T1SZ (3U << 16) -#define TTBCR_A1 (1U << 22) -#define TTBCR_EPD1 (1U << 23) -#define TTBCR_IRGN1 (3U << 24) -#define TTBCR_ORGN1 (3U << 26) -#define TTBCR_SH1 (1U << 28) -#define TTBCR_EAE (1U << 31) - -FIELD(VTCR, T0SZ, 0, 6) -FIELD(VTCR, SL0, 6, 2) -FIELD(VTCR, IRGN0, 8, 2) -FIELD(VTCR, ORGN0, 10, 2) -FIELD(VTCR, SH0, 12, 2) -FIELD(VTCR, TG0, 14, 2) -FIELD(VTCR, PS, 16, 3) -FIELD(VTCR, VS, 19, 1) -FIELD(VTCR, HA, 21, 1) -FIELD(VTCR, HD, 22, 1) -FIELD(VTCR, HWU59, 25, 1) -FIELD(VTCR, HWU60, 26, 1) -FIELD(VTCR, HWU61, 27, 1) -FIELD(VTCR, HWU62, 28, 1) -FIELD(VTCR, NSW, 29, 1) -FIELD(VTCR, NSA, 30, 1) -FIELD(VTCR, DS, 32, 1) -FIELD(VTCR, SL2, 33, 1) - /* Bit definitions for ARMv8 SPSR (PSTATE) format. * Only these are valid when in AArch64 mode; in * AArch32 mode SPSRs are basically CPSR-format. @@ -XXX,XX +XXX,XX @@ static inline void xpsr_write(CPUARMState *env, uint32_t val, uint32_t mask) #define HCR_TWEDEN (1ULL << 59) #define HCR_TWEDEL MAKE_64BIT_MASK(60, 4) -#define HCRX_ENAS0 (1ULL << 0) -#define HCRX_ENALS (1ULL << 1) -#define HCRX_ENASR (1ULL << 2) -#define HCRX_FNXS (1ULL << 3) -#define HCRX_FGTNXS (1ULL << 4) -#define HCRX_SMPME (1ULL << 5) -#define HCRX_TALLINT (1ULL << 6) -#define HCRX_VINMI (1ULL << 7) -#define HCRX_VFNMI (1ULL << 8) -#define HCRX_CMOW (1ULL << 9) -#define HCRX_MCE2 (1ULL << 10) -#define HCRX_MSCEN (1ULL << 11) - -#define HPFAR_NS (1ULL << 63) - #define SCR_NS (1ULL << 0) #define SCR_IRQ (1ULL << 1) #define SCR_FIQ (1ULL << 2) @@ -XXX,XX +XXX,XX @@ static inline void xpsr_write(CPUARMState *env, uint32_t val, uint32_t mask) #define SCR_GPF (1ULL << 48) #define SCR_NSE (1ULL << 62) -#define HSTR_TTEE (1 << 16) -#define HSTR_TJDBX (1 << 17) - -#define CNTHCTL_CNTVMASK (1 << 18) -#define CNTHCTL_CNTPMASK (1 << 19) - /* Return the current FPSCR value. */ uint32_t vfp_get_fpscr(CPUARMState *env); void vfp_set_fpscr(CPUARMState *env, uint32_t val); diff --git a/target/arm/internals.h b/target/arm/internals.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/internals.h +++ b/target/arm/internals.h @@ -XXX,XX +XXX,XX @@ FIELD(DBGWCR, WT, 20, 1) FIELD(DBGWCR, MASK, 24, 5) FIELD(DBGWCR, SSCE, 29, 1) +#define VTCR_NSW (1u << 29) +#define VTCR_NSA (1u << 30) +#define VSTCR_SW VTCR_NSW +#define VSTCR_SA VTCR_NSA + +/* Bit definitions for CPACR (AArch32 only) */ +FIELD(CPACR, CP10, 20, 2) +FIELD(CPACR, CP11, 22, 2) +FIELD(CPACR, TRCDIS, 28, 1) /* matches CPACR_EL1.TTA */ +FIELD(CPACR, D32DIS, 30, 1) /* up to v7; RAZ in v8 */ +FIELD(CPACR, ASEDIS, 31, 1) + +/* Bit definitions for CPACR_EL1 (AArch64 only) */ +FIELD(CPACR_EL1, ZEN, 16, 2) +FIELD(CPACR_EL1, FPEN, 20, 2) +FIELD(CPACR_EL1, SMEN, 24, 2) +FIELD(CPACR_EL1, TTA, 28, 1) /* matches CPACR.TRCDIS */ + +/* Bit definitions for HCPTR (AArch32 only) */ +FIELD(HCPTR, TCP10, 10, 1) +FIELD(HCPTR, TCP11, 11, 1) +FIELD(HCPTR, TASE, 15, 1) +FIELD(HCPTR, TTA, 20, 1) +FIELD(HCPTR, TAM, 30, 1) /* matches CPTR_EL2.TAM */ +FIELD(HCPTR, TCPAC, 31, 1) /* matches CPTR_EL2.TCPAC */ + +/* Bit definitions for CPTR_EL2 (AArch64 only) */ +FIELD(CPTR_EL2, TZ, 8, 1) /* !E2H */ +FIELD(CPTR_EL2, TFP, 10, 1) /* !E2H, matches HCPTR.TCP10 */ +FIELD(CPTR_EL2, TSM, 12, 1) /* !E2H */ +FIELD(CPTR_EL2, ZEN, 16, 2) /* E2H */ +FIELD(CPTR_EL2, FPEN, 20, 2) /* E2H */ +FIELD(CPTR_EL2, SMEN, 24, 2) /* E2H */ +FIELD(CPTR_EL2, TTA, 28, 1) +FIELD(CPTR_EL2, TAM, 30, 1) /* matches HCPTR.TAM */ +FIELD(CPTR_EL2, TCPAC, 31, 1) /* matches HCPTR.TCPAC */ + +/* Bit definitions for CPTR_EL3 (AArch64 only) */ +FIELD(CPTR_EL3, EZ, 8, 1) +FIELD(CPTR_EL3, TFP, 10, 1) +FIELD(CPTR_EL3, ESM, 12, 1) +FIELD(CPTR_EL3, TTA, 20, 1) +FIELD(CPTR_EL3, TAM, 30, 1) +FIELD(CPTR_EL3, TCPAC, 31, 1) + +#define MDCR_MTPME (1U << 28) +#define MDCR_TDCC (1U << 27) +#define MDCR_HLP (1U << 26) /* MDCR_EL2 */ +#define MDCR_SCCD (1U << 23) /* MDCR_EL3 */ +#define MDCR_HCCD (1U << 23) /* MDCR_EL2 */ +#define MDCR_EPMAD (1U << 21) +#define MDCR_EDAD (1U << 20) +#define MDCR_TTRF (1U << 19) +#define MDCR_STE (1U << 18) /* MDCR_EL3 */ +#define MDCR_SPME (1U << 17) /* MDCR_EL3 */ +#define MDCR_HPMD (1U << 17) /* MDCR_EL2 */ +#define MDCR_SDD (1U << 16) +#define MDCR_SPD (3U << 14) +#define MDCR_TDRA (1U << 11) +#define MDCR_TDOSA (1U << 10) +#define MDCR_TDA (1U << 9) +#define MDCR_TDE (1U << 8) +#define MDCR_HPME (1U << 7) +#define MDCR_TPM (1U << 6) +#define MDCR_TPMCR (1U << 5) +#define MDCR_HPMN (0x1fU) + +/* Not all of the MDCR_EL3 bits are present in the 32-bit SDCR */ +#define SDCR_VALID_MASK (MDCR_MTPME | MDCR_TDCC | MDCR_SCCD | \ + MDCR_EPMAD | MDCR_EDAD | MDCR_TTRF | \ + MDCR_STE | MDCR_SPME | MDCR_SPD) + +#define TTBCR_N (7U << 0) /* TTBCR.EAE==0 */ +#define TTBCR_T0SZ (7U << 0) /* TTBCR.EAE==1 */ +#define TTBCR_PD0 (1U << 4) +#define TTBCR_PD1 (1U << 5) +#define TTBCR_EPD0 (1U << 7) +#define TTBCR_IRGN0 (3U << 8) +#define TTBCR_ORGN0 (3U << 10) +#define TTBCR_SH0 (3U << 12) +#define TTBCR_T1SZ (3U << 16) +#define TTBCR_A1 (1U << 22) +#define TTBCR_EPD1 (1U << 23) +#define TTBCR_IRGN1 (3U << 24) +#define TTBCR_ORGN1 (3U << 26) +#define TTBCR_SH1 (1U << 28) +#define TTBCR_EAE (1U << 31) + +FIELD(VTCR, T0SZ, 0, 6) +FIELD(VTCR, SL0, 6, 2) +FIELD(VTCR, IRGN0, 8, 2) +FIELD(VTCR, ORGN0, 10, 2) +FIELD(VTCR, SH0, 12, 2) +FIELD(VTCR, TG0, 14, 2) +FIELD(VTCR, PS, 16, 3) +FIELD(VTCR, VS, 19, 1) +FIELD(VTCR, HA, 21, 1) +FIELD(VTCR, HD, 22, 1) +FIELD(VTCR, HWU59, 25, 1) +FIELD(VTCR, HWU60, 26, 1) +FIELD(VTCR, HWU61, 27, 1) +FIELD(VTCR, HWU62, 28, 1) +FIELD(VTCR, NSW, 29, 1) +FIELD(VTCR, NSA, 30, 1) +FIELD(VTCR, DS, 32, 1) +FIELD(VTCR, SL2, 33, 1) + +#define HCRX_ENAS0 (1ULL << 0) +#define HCRX_ENALS (1ULL << 1) +#define HCRX_ENASR (1ULL << 2) +#define HCRX_FNXS (1ULL << 3) +#define HCRX_FGTNXS (1ULL << 4) +#define HCRX_SMPME (1ULL << 5) +#define HCRX_TALLINT (1ULL << 6) +#define HCRX_VINMI (1ULL << 7) +#define HCRX_VFNMI (1ULL << 8) +#define HCRX_CMOW (1ULL << 9) +#define HCRX_MCE2 (1ULL << 10) +#define HCRX_MSCEN (1ULL << 11) + +#define HPFAR_NS (1ULL << 63) + +#define HSTR_TTEE (1 << 16) +#define HSTR_TJDBX (1 << 17) + +#define CNTHCTL_CNTVMASK (1 << 18) +#define CNTHCTL_CNTPMASK (1 << 19) + /* We use a few fake FSR values for internal purposes in M profile. * M profile cores don't have A/R format FSRs, but currently our * get_phys_addr() code assumes A/R profile and reports failures via -- 2.34.1 The timer _EL02 registers should UNDEF for invalid accesses from EL2 or EL3 when HCR_EL2.E2H == 0, not take a cp access trap. We were delivering the exception to EL2 with the wrong syndrome. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20240301183219.2424889-3-peter.maydell@linaro.org --- target/arm/helper.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/target/arm/helper.c b/target/arm/helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -XXX,XX +XXX,XX @@ static CPAccessResult e2h_access(CPUARMState *env, const ARMCPRegInfo *ri, return CP_ACCESS_OK; } if (!(arm_hcr_el2_eff(env) & HCR_E2H)) { - return CP_ACCESS_TRAP; + return CP_ACCESS_TRAP_UNCATEGORIZED; } return CP_ACCESS_OK; } -- 2.34.1 We prefer the FIELD macro over ad-hoc #defines for register bits; switch CNTHCTL to that style before we add any more bits. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20240301183219.2424889-4-peter.maydell@linaro.org --- target/arm/internals.h | 27 +++++++++++++++++++++++++-- target/arm/helper.c | 9 ++++----- 2 files changed, 29 insertions(+), 7 deletions(-) diff --git a/target/arm/internals.h b/target/arm/internals.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/internals.h +++ b/target/arm/internals.h @@ -XXX,XX +XXX,XX @@ FIELD(VTCR, SL2, 33, 1) #define HSTR_TTEE (1 << 16) #define HSTR_TJDBX (1 << 17) -#define CNTHCTL_CNTVMASK (1 << 18) -#define CNTHCTL_CNTPMASK (1 << 19) +/* + * Depending on the value of HCR_EL2.E2H, bits 0 and 1 + * have different bit definitions, and EL1PCTEN might be + * bit 0 or bit 10. We use _E2H1 and _E2H0 suffixes to + * disambiguate if necessary. + */ +FIELD(CNTHCTL, EL0PCTEN_E2H1, 0, 1) +FIELD(CNTHCTL, EL0VCTEN_E2H1, 1, 1) +FIELD(CNTHCTL, EL1PCTEN_E2H0, 0, 1) +FIELD(CNTHCTL, EL1PCEN_E2H0, 1, 1) +FIELD(CNTHCTL, EVNTEN, 2, 1) +FIELD(CNTHCTL, EVNTDIR, 3, 1) +FIELD(CNTHCTL, EVNTI, 4, 4) +FIELD(CNTHCTL, EL0VTEN, 8, 1) +FIELD(CNTHCTL, EL0PTEN, 9, 1) +FIELD(CNTHCTL, EL1PCTEN_E2H1, 10, 1) +FIELD(CNTHCTL, EL1PTEN, 11, 1) +FIELD(CNTHCTL, ECV, 12, 1) +FIELD(CNTHCTL, EL1TVT, 13, 1) +FIELD(CNTHCTL, EL1TVCT, 14, 1) +FIELD(CNTHCTL, EL1NVPCT, 15, 1) +FIELD(CNTHCTL, EL1NVVCT, 16, 1) +FIELD(CNTHCTL, EVNTIS, 17, 1) +FIELD(CNTHCTL, CNTVMASK, 18, 1) +FIELD(CNTHCTL, CNTPMASK, 19, 1) /* We use a few fake FSR values for internal purposes in M profile. * M profile cores don't have A/R format FSRs, but currently our diff --git a/target/arm/helper.c b/target/arm/helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -XXX,XX +XXX,XX @@ static void gt_update_irq(ARMCPU *cpu, int timeridx) * It is RES0 in Secure and NonSecure state. */ if ((ss == ARMSS_Root || ss == ARMSS_Realm) && - ((timeridx == GTIMER_VIRT && (cnthctl & CNTHCTL_CNTVMASK)) || - (timeridx == GTIMER_PHYS && (cnthctl & CNTHCTL_CNTPMASK)))) { + ((timeridx == GTIMER_VIRT && (cnthctl & R_CNTHCTL_CNTVMASK_MASK)) || + (timeridx == GTIMER_PHYS && (cnthctl & R_CNTHCTL_CNTPMASK_MASK)))) { irqstate = 0; } @@ -XXX,XX +XXX,XX @@ static void gt_cnthctl_write(CPUARMState *env, const ARMCPRegInfo *ri, { ARMCPU *cpu = env_archcpu(env); uint32_t oldval = env->cp15.cnthctl_el2; - raw_write(env, ri, value); - if ((oldval ^ value) & CNTHCTL_CNTVMASK) { + if ((oldval ^ value) & R_CNTHCTL_CNTVMASK_MASK) { gt_update_irq(cpu, GTIMER_VIRT); - } else if ((oldval ^ value) & CNTHCTL_CNTPMASK) { + } else if ((oldval ^ value) & R_CNTHCTL_CNTPMASK_MASK) { gt_update_irq(cpu, GTIMER_PHYS); } } -- 2.34.1 Don't allow the guest to write CNTHCTL_EL2 bits which don't exist. This is not strictly architecturally required, but it is how we've tended to implement registers more recently. In particular, bits [19:18] are only present with FEAT_RME, and bits [17:12] will only be present with FEAT_ECV. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20240301183219.2424889-5-peter.maydell@linaro.org --- target/arm/helper.c | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/target/arm/helper.c b/target/arm/helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -XXX,XX +XXX,XX @@ static void gt_cnthctl_write(CPUARMState *env, const ARMCPRegInfo *ri, { ARMCPU *cpu = env_archcpu(env); uint32_t oldval = env->cp15.cnthctl_el2; + uint32_t valid_mask = + R_CNTHCTL_EL0PCTEN_E2H1_MASK | + R_CNTHCTL_EL0VCTEN_E2H1_MASK | + R_CNTHCTL_EVNTEN_MASK | + R_CNTHCTL_EVNTDIR_MASK | + R_CNTHCTL_EVNTI_MASK | + R_CNTHCTL_EL0VTEN_MASK | + R_CNTHCTL_EL0PTEN_MASK | + R_CNTHCTL_EL1PCTEN_E2H1_MASK | + R_CNTHCTL_EL1PTEN_MASK; + + if (cpu_isar_feature(aa64_rme, cpu)) { + valid_mask |= R_CNTHCTL_CNTVMASK_MASK | R_CNTHCTL_CNTPMASK_MASK; + } + + /* Clear RES0 bits */ + value &= valid_mask; + raw_write(env, ri, value); if ((oldval ^ value) & R_CNTHCTL_CNTVMASK_MASK) { -- 2.34.1 The functionality defined by ID_AA64MMFR0_EL1.ECV == 1 is: * four new trap bits for various counter and timer registers * the CNTHCTL_EL2.EVNTIS and CNTKCTL_EL1.EVNTIS bits which control scaling of the event stream. This is a no-op for us, because we don't implement the event stream (our WFE is a NOP): all we need to do is allow CNTHCTL_EL2.ENVTIS to be read and written. * extensions to PMSCR_EL1.PCT, PMSCR_EL2.PCT, TRFCR_EL1.TS and TRFCR_EL2.TS: these are all no-ops for us, because we don't implement FEAT_SPE or FEAT_TRF. * new registers CNTPCTSS_EL0 and NCTVCTSS_EL0 which are "self-sychronizing" views of the CNTPCT_EL0 and CNTVCT_EL0, meaning that no barriers are needed around their accesses. For us these are just the same as the normal views, because all our sysregs are inherently self-sychronizing. In this commit we implement the trap handling and permit the new CNTHCTL_EL2 bits to be written. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20240301183219.2424889-6-peter.maydell@linaro.org --- target/arm/cpu-features.h | 5 ++++ target/arm/helper.c | 51 +++++++++++++++++++++++++++++++++++---- 2 files changed, 51 insertions(+), 5 deletions(-) diff --git a/target/arm/cpu-features.h b/target/arm/cpu-features.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpu-features.h +++ b/target/arm/cpu-features.h @@ -XXX,XX +XXX,XX @@ static inline bool isar_feature_aa64_fgt(const ARMISARegisters *id) return FIELD_EX64(id->id_aa64mmfr0, ID_AA64MMFR0, FGT) != 0; } +static inline bool isar_feature_aa64_ecv_traps(const ARMISARegisters *id) +{ + return FIELD_EX64(id->id_aa64mmfr0, ID_AA64MMFR0, ECV) > 0; +} + static inline bool isar_feature_aa64_vh(const ARMISARegisters *id) { return FIELD_EX64(id->id_aa64mmfr1, ID_AA64MMFR1, VH) != 0; diff --git a/target/arm/helper.c b/target/arm/helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -XXX,XX +XXX,XX @@ static CPAccessResult gt_counter_access(CPUARMState *env, int timeridx, : !extract32(env->cp15.cnthctl_el2, 0, 1))) { return CP_ACCESS_TRAP_EL2; } + if (has_el2 && timeridx == GTIMER_VIRT) { + if (FIELD_EX64(env->cp15.cnthctl_el2, CNTHCTL, EL1TVCT)) { + return CP_ACCESS_TRAP_EL2; + } + } break; } return CP_ACCESS_OK; @@ -XXX,XX +XXX,XX @@ static CPAccessResult gt_timer_access(CPUARMState *env, int timeridx, } } } + if (has_el2 && timeridx == GTIMER_VIRT) { + if (FIELD_EX64(env->cp15.cnthctl_el2, CNTHCTL, EL1TVT)) { + return CP_ACCESS_TRAP_EL2; + } + } break; } return CP_ACCESS_OK; @@ -XXX,XX +XXX,XX @@ static void gt_cnthctl_write(CPUARMState *env, const ARMCPRegInfo *ri, if (cpu_isar_feature(aa64_rme, cpu)) { valid_mask |= R_CNTHCTL_CNTVMASK_MASK | R_CNTHCTL_CNTPMASK_MASK; } + if (cpu_isar_feature(aa64_ecv_traps, cpu)) { + valid_mask |= + R_CNTHCTL_EL1TVT_MASK | + R_CNTHCTL_EL1TVCT_MASK | + R_CNTHCTL_EL1NVPCT_MASK | + R_CNTHCTL_EL1NVVCT_MASK | + R_CNTHCTL_EVNTIS_MASK; + } /* Clear RES0 bits */ value &= valid_mask; @@ -XXX,XX +XXX,XX @@ static CPAccessResult e2h_access(CPUARMState *env, const ARMCPRegInfo *ri, { if (arm_current_el(env) == 1) { /* This must be a FEAT_NV access */ - /* TODO: FEAT_ECV will need to check CNTHCTL_EL2 here */ return CP_ACCESS_OK; } if (!(arm_hcr_el2_eff(env) & HCR_E2H)) { @@ -XXX,XX +XXX,XX @@ static CPAccessResult e2h_access(CPUARMState *env, const ARMCPRegInfo *ri, return CP_ACCESS_OK; } +static CPAccessResult access_el1nvpct(CPUARMState *env, const ARMCPRegInfo *ri, + bool isread) +{ + if (arm_current_el(env) == 1) { + /* This must be a FEAT_NV access with NVx == 101 */ + if (FIELD_EX64(env->cp15.cnthctl_el2, CNTHCTL, EL1NVPCT)) { + return CP_ACCESS_TRAP_EL2; + } + } + return e2h_access(env, ri, isread); +} + +static CPAccessResult access_el1nvvct(CPUARMState *env, const ARMCPRegInfo *ri, + bool isread) +{ + if (arm_current_el(env) == 1) { + /* This must be a FEAT_NV access with NVx == 101 */ + if (FIELD_EX64(env->cp15.cnthctl_el2, CNTHCTL, EL1NVVCT)) { + return CP_ACCESS_TRAP_EL2; + } + } + return e2h_access(env, ri, isread); +} + /* Test if system register redirection is to occur in the current state. */ static bool redirect_for_e2h(CPUARMState *env) { @@ -XXX,XX +XXX,XX @@ static const ARMCPRegInfo vhe_reginfo[] = { { .name = "CNTP_CTL_EL02", .state = ARM_CP_STATE_AA64, .opc0 = 3, .opc1 = 5, .crn = 14, .crm = 2, .opc2 = 1, .type = ARM_CP_IO | ARM_CP_ALIAS, - .access = PL2_RW, .accessfn = e2h_access, + .access = PL2_RW, .accessfn = access_el1nvpct, .nv2_redirect_offset = 0x180 | NV2_REDIR_NO_NV1, .fieldoffset = offsetof(CPUARMState, cp15.c14_timer[GTIMER_PHYS].ctl), .writefn = gt_phys_ctl_write, .raw_writefn = raw_write }, { .name = "CNTV_CTL_EL02", .state = ARM_CP_STATE_AA64, .opc0 = 3, .opc1 = 5, .crn = 14, .crm = 3, .opc2 = 1, .type = ARM_CP_IO | ARM_CP_ALIAS, - .access = PL2_RW, .accessfn = e2h_access, + .access = PL2_RW, .accessfn = access_el1nvvct, .nv2_redirect_offset = 0x170 | NV2_REDIR_NO_NV1, .fieldoffset = offsetof(CPUARMState, cp15.c14_timer[GTIMER_VIRT].ctl), .writefn = gt_virt_ctl_write, .raw_writefn = raw_write }, @@ -XXX,XX +XXX,XX @@ static const ARMCPRegInfo vhe_reginfo[] = { .type = ARM_CP_IO | ARM_CP_ALIAS, .fieldoffset = offsetof(CPUARMState, cp15.c14_timer[GTIMER_PHYS].cval), .nv2_redirect_offset = 0x178 | NV2_REDIR_NO_NV1, - .access = PL2_RW, .accessfn = e2h_access, + .access = PL2_RW, .accessfn = access_el1nvpct, .writefn = gt_phys_cval_write, .raw_writefn = raw_write }, { .name = "CNTV_CVAL_EL02", .state = ARM_CP_STATE_AA64, .opc0 = 3, .opc1 = 5, .crn = 14, .crm = 3, .opc2 = 2, .type = ARM_CP_IO | ARM_CP_ALIAS, .nv2_redirect_offset = 0x168 | NV2_REDIR_NO_NV1, .fieldoffset = offsetof(CPUARMState, cp15.c14_timer[GTIMER_VIRT].cval), - .access = PL2_RW, .accessfn = e2h_access, + .access = PL2_RW, .accessfn = access_el1nvvct, .writefn = gt_virt_cval_write, .raw_writefn = raw_write }, #endif }; -- 2.34.1 For FEAT_ECV, new registers CNTPCTSS_EL0 and CNTVCTSS_EL0 are defined, which are "self-synchronized" views of the physical and virtual counts as seen in the CNTPCT_EL0 and CNTVCT_EL0 registers (meaning that no barriers are needed around accesses to them to ensure that reads of them do not occur speculatively and out-of-order with other instructions). For QEMU, all our system registers are self-synchronized, so we can simply copy the existing implementation of CNTPCT_EL0 and CNTVCT_EL0 to the new register encodings. This means we now implement all the functionality required for ID_AA64MMFR0_EL1.ECV == 0b0001. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20240301183219.2424889-7-peter.maydell@linaro.org --- target/arm/helper.c | 43 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 43 insertions(+) diff --git a/target/arm/helper.c b/target/arm/helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -XXX,XX +XXX,XX @@ static const ARMCPRegInfo generic_timer_cp_reginfo[] = { }, }; +/* + * FEAT_ECV adds extra views of CNTVCT_EL0 and CNTPCT_EL0 which + * are "self-synchronizing". For QEMU all sysregs are self-synchronizing, + * so our implementations here are identical to the normal registers. + */ +static const ARMCPRegInfo gen_timer_ecv_cp_reginfo[] = { + { .name = "CNTVCTSS", .cp = 15, .crm = 14, .opc1 = 9, + .access = PL0_R, .type = ARM_CP_64BIT | ARM_CP_NO_RAW | ARM_CP_IO, + .accessfn = gt_vct_access, + .readfn = gt_virt_cnt_read, .resetfn = arm_cp_reset_ignore, + }, + { .name = "CNTVCTSS_EL0", .state = ARM_CP_STATE_AA64, + .opc0 = 3, .opc1 = 3, .crn = 14, .crm = 0, .opc2 = 6, + .access = PL0_R, .type = ARM_CP_NO_RAW | ARM_CP_IO, + .accessfn = gt_vct_access, .readfn = gt_virt_cnt_read, + }, + { .name = "CNTPCTSS", .cp = 15, .crm = 14, .opc1 = 8, + .access = PL0_R, .type = ARM_CP_64BIT | ARM_CP_NO_RAW | ARM_CP_IO, + .accessfn = gt_pct_access, + .readfn = gt_cnt_read, .resetfn = arm_cp_reset_ignore, + }, + { .name = "CNTPCTSS_EL0", .state = ARM_CP_STATE_AA64, + .opc0 = 3, .opc1 = 3, .crn = 14, .crm = 0, .opc2 = 5, + .access = PL0_R, .type = ARM_CP_NO_RAW | ARM_CP_IO, + .accessfn = gt_pct_access, .readfn = gt_cnt_read, + }, +}; + #else /* @@ -XXX,XX +XXX,XX @@ static const ARMCPRegInfo generic_timer_cp_reginfo[] = { }, }; +/* + * CNTVCTSS_EL0 has the same trap conditions as CNTVCT_EL0, so it also + * is exposed to userspace by Linux. + */ +static const ARMCPRegInfo gen_timer_ecv_cp_reginfo[] = { + { .name = "CNTVCTSS_EL0", .state = ARM_CP_STATE_AA64, + .opc0 = 3, .opc1 = 3, .crn = 14, .crm = 0, .opc2 = 6, + .access = PL0_R, .type = ARM_CP_NO_RAW | ARM_CP_IO, + .readfn = gt_virt_cnt_read, + }, +}; + #endif static void par_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value) @@ -XXX,XX +XXX,XX @@ void register_cp_regs_for_features(ARMCPU *cpu) if (arm_feature(env, ARM_FEATURE_GENERIC_TIMER)) { define_arm_cp_regs(cpu, generic_timer_cp_reginfo); } + if (cpu_isar_feature(aa64_ecv_traps, cpu)) { + define_arm_cp_regs(cpu, gen_timer_ecv_cp_reginfo); + } if (arm_feature(env, ARM_FEATURE_VAPA)) { ARMCPRegInfo vapa_cp_reginfo[] = { { .name = "PAR", .cp = 15, .crn = 7, .crm = 4, .opc1 = 0, .opc2 = 0, -- 2.34.1 When ID_AA64MMFR0_EL1.ECV is 0b0010, a new register CNTPOFF_EL2 is implemented. This is similar to the existing CNTVOFF_EL2, except that it controls a hypervisor-adjustable offset made to the physical counter and timer. Implement the handling for this register, which includes control/trap bits in SCR_EL3 and CNTHCTL_EL2. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20240301183219.2424889-8-peter.maydell@linaro.org --- target/arm/cpu-features.h | 5 +++ target/arm/cpu.h | 1 + target/arm/helper.c | 68 +++++++++++++++++++++++++++++++++++++-- target/arm/trace-events | 1 + 4 files changed, 73 insertions(+), 2 deletions(-) diff --git a/target/arm/cpu-features.h b/target/arm/cpu-features.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpu-features.h +++ b/target/arm/cpu-features.h @@ -XXX,XX +XXX,XX @@ static inline bool isar_feature_aa64_ecv_traps(const ARMISARegisters *id) return FIELD_EX64(id->id_aa64mmfr0, ID_AA64MMFR0, ECV) > 0; } +static inline bool isar_feature_aa64_ecv(const ARMISARegisters *id) +{ + return FIELD_EX64(id->id_aa64mmfr0, ID_AA64MMFR0, ECV) > 1; +} + static inline bool isar_feature_aa64_vh(const ARMISARegisters *id) { return FIELD_EX64(id->id_aa64mmfr1, ID_AA64MMFR1, VH) != 0; diff --git a/target/arm/cpu.h b/target/arm/cpu.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -XXX,XX +XXX,XX @@ typedef struct CPUArchState { uint64_t c14_cntkctl; /* Timer Control register */ uint64_t cnthctl_el2; /* Counter/Timer Hyp Control register */ uint64_t cntvoff_el2; /* Counter Virtual Offset register */ + uint64_t cntpoff_el2; /* Counter Physical Offset register */ ARMGenericTimer c14_timer[NUM_GTIMERS]; uint32_t c15_cpar; /* XScale Coprocessor Access Register */ uint32_t c15_ticonfig; /* TI925T configuration byte. */ diff --git a/target/arm/helper.c b/target/arm/helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -XXX,XX +XXX,XX @@ static void scr_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value) if (cpu_isar_feature(aa64_rme, cpu)) { valid_mask |= SCR_NSE | SCR_GPF; } + if (cpu_isar_feature(aa64_ecv, cpu)) { + valid_mask |= SCR_ECVEN; + } } else { valid_mask &= ~(SCR_RW | SCR_ST); if (cpu_isar_feature(aa32_ras, cpu)) { @@ -XXX,XX +XXX,XX @@ void gt_rme_post_el_change(ARMCPU *cpu, void *ignored) gt_update_irq(cpu, GTIMER_PHYS); } +static uint64_t gt_phys_raw_cnt_offset(CPUARMState *env) +{ + if ((env->cp15.scr_el3 & SCR_ECVEN) && + FIELD_EX64(env->cp15.cnthctl_el2, CNTHCTL, ECV) && + arm_is_el2_enabled(env) && + (arm_hcr_el2_eff(env) & (HCR_E2H | HCR_TGE)) != (HCR_E2H | HCR_TGE)) { + return env->cp15.cntpoff_el2; + } + return 0; +} + +static uint64_t gt_phys_cnt_offset(CPUARMState *env) +{ + if (arm_current_el(env) >= 2) { + return 0; + } + return gt_phys_raw_cnt_offset(env); +} + static void gt_recalc_timer(ARMCPU *cpu, int timeridx) { ARMGenericTimer *gt = &cpu->env.cp15.c14_timer[timeridx]; @@ -XXX,XX +XXX,XX @@ static void gt_recalc_timer(ARMCPU *cpu, int timeridx) * reset timer to when ISTATUS next has to change */ uint64_t offset = timeridx == GTIMER_VIRT ? - cpu->env.cp15.cntvoff_el2 : 0; + cpu->env.cp15.cntvoff_el2 : gt_phys_raw_cnt_offset(&cpu->env); uint64_t count = gt_get_countervalue(&cpu->env); /* Note that this must be unsigned 64 bit arithmetic: */ int istatus = count - offset >= gt->cval; @@ -XXX,XX +XXX,XX @@ static void gt_timer_reset(CPUARMState *env, const ARMCPRegInfo *ri, static uint64_t gt_cnt_read(CPUARMState *env, const ARMCPRegInfo *ri) { - return gt_get_countervalue(env); + return gt_get_countervalue(env) - gt_phys_cnt_offset(env); } static uint64_t gt_virt_cnt_offset(CPUARMState *env) @@ -XXX,XX +XXX,XX @@ static uint64_t gt_tval_read(CPUARMState *env, const ARMCPRegInfo *ri, case GTIMER_HYPVIRT: offset = gt_virt_cnt_offset(env); break; + case GTIMER_PHYS: + offset = gt_phys_cnt_offset(env); + break; } return (uint32_t)(env->cp15.c14_timer[timeridx].cval - @@ -XXX,XX +XXX,XX @@ static void gt_tval_write(CPUARMState *env, const ARMCPRegInfo *ri, case GTIMER_HYPVIRT: offset = gt_virt_cnt_offset(env); break; + case GTIMER_PHYS: + offset = gt_phys_cnt_offset(env); + break; } trace_arm_gt_tval_write(timeridx, value); @@ -XXX,XX +XXX,XX @@ static void gt_cnthctl_write(CPUARMState *env, const ARMCPRegInfo *ri, R_CNTHCTL_EL1NVVCT_MASK | R_CNTHCTL_EVNTIS_MASK; } + if (cpu_isar_feature(aa64_ecv, cpu)) { + valid_mask |= R_CNTHCTL_ECV_MASK; + } /* Clear RES0 bits */ value &= valid_mask; @@ -XXX,XX +XXX,XX @@ static const ARMCPRegInfo gen_timer_ecv_cp_reginfo[] = { }, }; +static CPAccessResult gt_cntpoff_access(CPUARMState *env, + const ARMCPRegInfo *ri, + bool isread) +{ + if (arm_current_el(env) == 2 && !(env->cp15.scr_el3 & SCR_ECVEN)) { + return CP_ACCESS_TRAP_EL3; + } + return CP_ACCESS_OK; +} + +static void gt_cntpoff_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) +{ + ARMCPU *cpu = env_archcpu(env); + + trace_arm_gt_cntpoff_write(value); + raw_write(env, ri, value); + gt_recalc_timer(cpu, GTIMER_PHYS); +} + +static const ARMCPRegInfo gen_timer_cntpoff_reginfo = { + .name = "CNTPOFF_EL2", .state = ARM_CP_STATE_AA64, + .opc0 = 3, .opc1 = 4, .crn = 14, .crm = 0, .opc2 = 6, + .access = PL2_RW, .type = ARM_CP_IO, .resetvalue = 0, + .accessfn = gt_cntpoff_access, .writefn = gt_cntpoff_write, + .nv2_redirect_offset = 0x1a8, + .fieldoffset = offsetof(CPUARMState, cp15.cntpoff_el2), +}; #else /* @@ -XXX,XX +XXX,XX @@ void register_cp_regs_for_features(ARMCPU *cpu) if (cpu_isar_feature(aa64_ecv_traps, cpu)) { define_arm_cp_regs(cpu, gen_timer_ecv_cp_reginfo); } +#ifndef CONFIG_USER_ONLY + if (cpu_isar_feature(aa64_ecv, cpu)) { + define_one_arm_cp_reg(cpu, &gen_timer_cntpoff_reginfo); + } +#endif if (arm_feature(env, ARM_FEATURE_VAPA)) { ARMCPRegInfo vapa_cp_reginfo[] = { { .name = "PAR", .cp = 15, .crn = 7, .crm = 4, .opc1 = 0, .opc2 = 0, diff --git a/target/arm/trace-events b/target/arm/trace-events index XXXXXXX..XXXXXXX 100644 --- a/target/arm/trace-events +++ b/target/arm/trace-events @@ -XXX,XX +XXX,XX @@ arm_gt_tval_write(int timer, uint64_t value) "gt_tval_write: timer %d value 0x%" arm_gt_ctl_write(int timer, uint64_t value) "gt_ctl_write: timer %d value 0x%" PRIx64 arm_gt_imask_toggle(int timer) "gt_ctl_write: timer %d IMASK toggle" arm_gt_cntvoff_write(uint64_t value) "gt_cntvoff_write: value 0x%" PRIx64 +arm_gt_cntpoff_write(uint64_t value) "gt_cntpoff_write: value 0x%" PRIx64 arm_gt_update_irq(int timer, int irqstate) "gt_update_irq: timer %d irqstate %d" # kvm.c -- 2.34.1 Enable all FEAT_ECV features on the 'max' CPU. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20240301183219.2424889-9-peter.maydell@linaro.org --- docs/system/arm/emulation.rst | 1 + target/arm/tcg/cpu64.c | 1 + 2 files changed, 2 insertions(+) diff --git a/docs/system/arm/emulation.rst b/docs/system/arm/emulation.rst index XXXXXXX..XXXXXXX 100644 --- a/docs/system/arm/emulation.rst +++ b/docs/system/arm/emulation.rst @@ -XXX,XX +XXX,XX @@ the following architecture extensions: - FEAT_DotProd (Advanced SIMD dot product instructions) - FEAT_DoubleFault (Double Fault Extension) - FEAT_E0PD (Preventing EL0 access to halves of address maps) +- FEAT_ECV (Enhanced Counter Virtualization) - FEAT_EPAC (Enhanced pointer authentication) - FEAT_ETS (Enhanced Translation Synchronization) - FEAT_EVT (Enhanced Virtualization Traps) diff --git a/target/arm/tcg/cpu64.c b/target/arm/tcg/cpu64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/cpu64.c +++ b/target/arm/tcg/cpu64.c @@ -XXX,XX +XXX,XX @@ void aarch64_max_tcg_initfn(Object *obj) t = FIELD_DP64(t, ID_AA64MMFR0, TGRAN64_2, 2); /* 64k stage2 supported */ t = FIELD_DP64(t, ID_AA64MMFR0, TGRAN4_2, 2); /* 4k stage2 supported */ t = FIELD_DP64(t, ID_AA64MMFR0, FGT, 1); /* FEAT_FGT */ + t = FIELD_DP64(t, ID_AA64MMFR0, ECV, 2); /* FEAT_ECV */ cpu->isar.id_aa64mmfr0 = t; t = cpu->isar.id_aa64mmfr1; -- 2.34.1 From: Inès Varhol <ines.varhol@telecom-paris.fr> Features supported : - the 8 STM32L4x5 GPIOs are initialized with their reset values (except IDR, see below) - input mode : setting a pin in input mode "externally" (using input irqs) results in an out irq (transmitted to SYSCFG) - output mode : setting a bit in ODR sets the corresponding out irq (if this line is configured in output mode) - pull-up, pull-down - push-pull, open-drain Difference with the real GPIOs : - Alternate Function and Analog mode aren't implemented : pins in AF/Analog behave like pins in input mode - floating pins stay at their last value - register IDR reset values differ from the real one : values are coherent with the other registers reset values and the fact that AF/Analog modes aren't implemented - setting I/O output speed isn't supported - locking port bits isn't supported - ADC function isn't supported - GPIOH has 16 pins instead of 2 pins - writing to registers LCKR, AFRL, AFRH and ASCR is ineffective Signed-off-by: Arnaud Minier <arnaud.minier@telecom-paris.fr> Signed-off-by: Inès Varhol <ines.varhol@telecom-paris.fr> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Acked-by: Alistair Francis <alistair.francis@wdc.com> Message-id: 20240305210444.310665-2-ines.varhol@telecom-paris.fr Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- MAINTAINERS | 1 + docs/system/arm/b-l475e-iot01a.rst | 2 +- include/hw/gpio/stm32l4x5_gpio.h | 70 +++++ hw/gpio/stm32l4x5_gpio.c | 477 +++++++++++++++++++++++++++++ hw/gpio/Kconfig | 3 + hw/gpio/meson.build | 1 + hw/gpio/trace-events | 6 + 7 files changed, 559 insertions(+), 1 deletion(-) create mode 100644 include/hw/gpio/stm32l4x5_gpio.h create mode 100644 hw/gpio/stm32l4x5_gpio.c diff --git a/MAINTAINERS b/MAINTAINERS index XXXXXXX..XXXXXXX 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -XXX,XX +XXX,XX @@ F: hw/arm/stm32l4x5_soc.c F: hw/misc/stm32l4x5_exti.c F: hw/misc/stm32l4x5_syscfg.c F: hw/misc/stm32l4x5_rcc.c +F: hw/gpio/stm32l4x5_gpio.c F: include/hw/*/stm32l4x5_*.h B-L475E-IOT01A IoT Node diff --git a/docs/system/arm/b-l475e-iot01a.rst b/docs/system/arm/b-l475e-iot01a.rst index XXXXXXX..XXXXXXX 100644 --- a/docs/system/arm/b-l475e-iot01a.rst +++ b/docs/system/arm/b-l475e-iot01a.rst @@ -XXX,XX +XXX,XX @@ Currently B-L475E-IOT01A machine's only supports the following devices: - STM32L4x5 EXTI (Extended interrupts and events controller) - STM32L4x5 SYSCFG (System configuration controller) - STM32L4x5 RCC (Reset and clock control) +- STM32L4x5 GPIOs (General-purpose I/Os) Missing devices """"""""""""""" @@ -XXX,XX +XXX,XX @@ Missing devices The B-L475E-IOT01A does *not* support the following devices: - Serial ports (UART) -- General-purpose I/Os (GPIO) - Analog to Digital Converter (ADC) - SPI controller - Timer controller (TIMER) diff --git a/include/hw/gpio/stm32l4x5_gpio.h b/include/hw/gpio/stm32l4x5_gpio.h new file mode 100644 index XXXXXXX..XXXXXXX --- /dev/null +++ b/include/hw/gpio/stm32l4x5_gpio.h @@ -XXX,XX +XXX,XX @@ +/* + * STM32L4x5 GPIO (General Purpose Input/Ouput) + * + * Copyright (c) 2024 Arnaud Minier <arnaud.minier@telecom-paris.fr> + * Copyright (c) 2024 Inès Varhol <ines.varhol@telecom-paris.fr> + * + * SPDX-License-Identifier: GPL-2.0-or-later + * + * This work is licensed under the terms of the GNU GPL, version 2 or later. + * See the COPYING file in the top-level directory. + */ + +/* + * The reference used is the STMicroElectronics RM0351 Reference manual + * for STM32L4x5 and STM32L4x6 advanced Arm ® -based 32-bit MCUs. + * https://www.st.com/en/microcontrollers-microprocessors/stm32l4x5/documentation.html + */ + +#ifndef HW_STM32L4X5_GPIO_H +#define HW_STM32L4X5_GPIO_H + +#include "hw/sysbus.h" +#include "qom/object.h" + +#define TYPE_STM32L4X5_GPIO "stm32l4x5-gpio" +OBJECT_DECLARE_SIMPLE_TYPE(Stm32l4x5GpioState, STM32L4X5_GPIO) + +#define GPIO_NUM_PINS 16 + +struct Stm32l4x5GpioState { + SysBusDevice parent_obj; + + MemoryRegion mmio; + + /* GPIO registers */ + uint32_t moder; + uint32_t otyper; + uint32_t ospeedr; + uint32_t pupdr; + uint32_t idr; + uint32_t odr; + uint32_t lckr; + uint32_t afrl; + uint32_t afrh; + uint32_t ascr; + + /* GPIO registers reset values */ + uint32_t moder_reset; + uint32_t ospeedr_reset; + uint32_t pupdr_reset; + + /* + * External driving of pins. + * The pins can be set externally through the device + * anonymous input GPIOs lines under certain conditions. + * The pin must not be in push-pull output mode, + * and can't be set high in open-drain mode. + * Pins driven externally and configured to + * output mode will in general be "disconnected" + * (see `get_gpio_pinmask_to_disconnect()`) + */ + uint16_t disconnected_pins; + uint16_t pins_connected_high; + + char *name; + Clock *clk; + qemu_irq pin[GPIO_NUM_PINS]; +}; + +#endif diff --git a/hw/gpio/stm32l4x5_gpio.c b/hw/gpio/stm32l4x5_gpio.c new file mode 100644 index XXXXXXX..XXXXXXX --- /dev/null +++ b/hw/gpio/stm32l4x5_gpio.c @@ -XXX,XX +XXX,XX @@ +/* + * STM32L4x5 GPIO (General Purpose Input/Ouput) + * + * Copyright (c) 2024 Arnaud Minier <arnaud.minier@telecom-paris.fr> + * Copyright (c) 2024 Inès Varhol <ines.varhol@telecom-paris.fr> + * + * SPDX-License-Identifier: GPL-2.0-or-later + * + * This work is licensed under the terms of the GNU GPL, version 2 or later. + * See the COPYING file in the top-level directory. + */ + +/* + * The reference used is the STMicroElectronics RM0351 Reference manual + * for STM32L4x5 and STM32L4x6 advanced Arm ® -based 32-bit MCUs. + * https://www.st.com/en/microcontrollers-microprocessors/stm32l4x5/documentation.html + */ + +#include "qemu/osdep.h" +#include "qemu/log.h" +#include "hw/gpio/stm32l4x5_gpio.h" +#include "hw/irq.h" +#include "hw/qdev-clock.h" +#include "hw/qdev-properties.h" +#include "qapi/visitor.h" +#include "qapi/error.h" +#include "migration/vmstate.h" +#include "trace.h" + +#define GPIO_MODER 0x00 +#define GPIO_OTYPER 0x04 +#define GPIO_OSPEEDR 0x08 +#define GPIO_PUPDR 0x0C +#define GPIO_IDR 0x10 +#define GPIO_ODR 0x14 +#define GPIO_BSRR 0x18 +#define GPIO_LCKR 0x1C +#define GPIO_AFRL 0x20 +#define GPIO_AFRH 0x24 +#define GPIO_BRR 0x28 +#define GPIO_ASCR 0x2C + +/* 0b11111111_11111111_00000000_00000000 */ +#define RESERVED_BITS_MASK 0xFFFF0000 + +static void update_gpio_idr(Stm32l4x5GpioState *s); + +static bool is_pull_up(Stm32l4x5GpioState *s, unsigned pin) +{ + return extract32(s->pupdr, 2 * pin, 2) == 1; +} + +static bool is_pull_down(Stm32l4x5GpioState *s, unsigned pin) +{ + return extract32(s->pupdr, 2 * pin, 2) == 2; +} + +static bool is_output(Stm32l4x5GpioState *s, unsigned pin) +{ + return extract32(s->moder, 2 * pin, 2) == 1; +} + +static bool is_open_drain(Stm32l4x5GpioState *s, unsigned pin) +{ + return extract32(s->otyper, pin, 1) == 1; +} + +static bool is_push_pull(Stm32l4x5GpioState *s, unsigned pin) +{ + return extract32(s->otyper, pin, 1) == 0; +} + +static void stm32l4x5_gpio_reset_hold(Object *obj) +{ + Stm32l4x5GpioState *s = STM32L4X5_GPIO(obj); + + s->moder = s->moder_reset; + s->otyper = 0x00000000; + s->ospeedr = s->ospeedr_reset; + s->pupdr = s->pupdr_reset; + s->idr = 0x00000000; + s->odr = 0x00000000; + s->lckr = 0x00000000; + s->afrl = 0x00000000; + s->afrh = 0x00000000; + s->ascr = 0x00000000; + + s->disconnected_pins = 0xFFFF; + s->pins_connected_high = 0x0000; + update_gpio_idr(s); +} + +static void stm32l4x5_gpio_set(void *opaque, int line, int level) +{ + Stm32l4x5GpioState *s = opaque; + /* + * The pin isn't set if line is configured in output mode + * except if level is 0 and the output is open-drain. + * This way there will be no short-circuit prone situations. + */ + if (is_output(s, line) && !(is_open_drain(s, line) && (level == 0))) { + qemu_log_mask(LOG_GUEST_ERROR, "Line %d can't be driven externally\n", + line); + return; + } + + s->disconnected_pins &= ~(1 << line); + if (level) { + s->pins_connected_high |= (1 << line); + } else { + s->pins_connected_high &= ~(1 << line); + } + trace_stm32l4x5_gpio_pins(s->name, s->disconnected_pins, + s->pins_connected_high); + update_gpio_idr(s); +} + + +static void update_gpio_idr(Stm32l4x5GpioState *s) +{ + uint32_t new_idr_mask = 0; + uint32_t new_idr = s->odr; + uint32_t old_idr = s->idr; + int new_pin_state, old_pin_state; + + for (int i = 0; i < GPIO_NUM_PINS; i++) { + if (is_output(s, i)) { + if (is_push_pull(s, i)) { + new_idr_mask |= (1 << i); + } else if (!(s->odr & (1 << i))) { + /* open-drain ODR 0 */ + new_idr_mask |= (1 << i); + /* open-drain ODR 1 */ + } else if (!(s->disconnected_pins & (1 << i)) && + !(s->pins_connected_high & (1 << i))) { + /* open-drain ODR 1 with pin connected low */ + new_idr_mask |= (1 << i); + new_idr &= ~(1 << i); + /* open-drain ODR 1 with unactive pin */ + } else if (is_pull_up(s, i)) { + new_idr_mask |= (1 << i); + } else if (is_pull_down(s, i)) { + new_idr_mask |= (1 << i); + new_idr &= ~(1 << i); + } + /* + * The only case left is for open-drain ODR 1 + * with unactive pin without pull-up or pull-down : + * the value is floating. + */ + /* input or analog mode with connected pin */ + } else if (!(s->disconnected_pins & (1 << i))) { + if (s->pins_connected_high & (1 << i)) { + /* pin high */ + new_idr_mask |= (1 << i); + new_idr |= (1 << i); + } else { + /* pin low */ + new_idr_mask |= (1 << i); + new_idr &= ~(1 << i); + } + /* input or analog mode with disconnected pin */ + } else { + if (is_pull_up(s, i)) { + /* pull-up */ + new_idr_mask |= (1 << i); + new_idr |= (1 << i); + } else if (is_pull_down(s, i)) { + /* pull-down */ + new_idr_mask |= (1 << i); + new_idr &= ~(1 << i); + } + /* + * The only case left is for a disconnected pin + * without pull-up or pull-down : + * the value is floating. + */ + } + } + + s->idr = (old_idr & ~new_idr_mask) | (new_idr & new_idr_mask); + trace_stm32l4x5_gpio_update_idr(s->name, old_idr, s->idr); + + for (int i = 0; i < GPIO_NUM_PINS; i++) { + if (new_idr_mask & (1 << i)) { + new_pin_state = (new_idr & (1 << i)) > 0; + old_pin_state = (old_idr & (1 << i)) > 0; + if (new_pin_state > old_pin_state) { + qemu_irq_raise(s->pin[i]); + } else if (new_pin_state < old_pin_state) { + qemu_irq_lower(s->pin[i]); + } + } + } +} + +/* + * Return mask of pins that are both configured in output + * mode and externally driven (except pins in open-drain + * mode externally set to 0). + */ +static uint32_t get_gpio_pinmask_to_disconnect(Stm32l4x5GpioState *s) +{ + uint32_t pins_to_disconnect = 0; + for (int i = 0; i < GPIO_NUM_PINS; i++) { + /* for each connected pin in output mode */ + if (!(s->disconnected_pins & (1 << i)) && is_output(s, i)) { + /* if either push-pull or high level */ + if (is_push_pull(s, i) || s->pins_connected_high & (1 << i)) { + pins_to_disconnect |= (1 << i); + qemu_log_mask(LOG_GUEST_ERROR, + "Line %d can't be driven externally\n", + i); + } + } + } + return pins_to_disconnect; +} + +/* + * Set field `disconnected_pins` and call `update_gpio_idr()` + */ +static void disconnect_gpio_pins(Stm32l4x5GpioState *s, uint16_t lines) +{ + s->disconnected_pins |= lines; + trace_stm32l4x5_gpio_pins(s->name, s->disconnected_pins, + s->pins_connected_high); + update_gpio_idr(s); +} + +static void disconnected_pins_set(Object *obj, Visitor *v, + const char *name, void *opaque, Error **errp) +{ + Stm32l4x5GpioState *s = STM32L4X5_GPIO(obj); + uint16_t value; + if (!visit_type_uint16(v, name, &value, errp)) { + return; + } + disconnect_gpio_pins(s, value); +} + +static void disconnected_pins_get(Object *obj, Visitor *v, + const char *name, void *opaque, Error **errp) +{ + visit_type_uint16(v, name, (uint16_t *)opaque, errp); +} + +static void clock_freq_get(Object *obj, Visitor *v, + const char *name, void *opaque, Error **errp) +{ + Stm32l4x5GpioState *s = STM32L4X5_GPIO(obj); + uint32_t clock_freq_hz = clock_get_hz(s->clk); + visit_type_uint32(v, name, &clock_freq_hz, errp); +} + +static void stm32l4x5_gpio_write(void *opaque, hwaddr addr, + uint64_t val64, unsigned int size) +{ + Stm32l4x5GpioState *s = opaque; + + uint32_t value = val64; + trace_stm32l4x5_gpio_write(s->name, addr, val64); + + switch (addr) { + case GPIO_MODER: + s->moder = value; + disconnect_gpio_pins(s, get_gpio_pinmask_to_disconnect(s)); + qemu_log_mask(LOG_UNIMP, + "%s: Analog and AF modes aren't supported\n\ + Analog and AF mode behave like input mode\n", + __func__); + return; + case GPIO_OTYPER: + s->otyper = value & ~RESERVED_BITS_MASK; + disconnect_gpio_pins(s, get_gpio_pinmask_to_disconnect(s)); + return; + case GPIO_OSPEEDR: + qemu_log_mask(LOG_UNIMP, + "%s: Changing I/O output speed isn't supported\n\ + I/O speed is already maximal\n", + __func__); + s->ospeedr = value; + return; + case GPIO_PUPDR: + s->pupdr = value; + update_gpio_idr(s); + return; + case GPIO_IDR: + qemu_log_mask(LOG_UNIMP, + "%s: GPIO->IDR is read-only\n", + __func__); + return; + case GPIO_ODR: + s->odr = value & ~RESERVED_BITS_MASK; + update_gpio_idr(s); + return; + case GPIO_BSRR: { + uint32_t bits_to_reset = (value & RESERVED_BITS_MASK) >> GPIO_NUM_PINS; + uint32_t bits_to_set = value & ~RESERVED_BITS_MASK; + /* If both BSx and BRx are set, BSx has priority.*/ + s->odr &= ~bits_to_reset; + s->odr |= bits_to_set; + update_gpio_idr(s); + return; + } + case GPIO_LCKR: + qemu_log_mask(LOG_UNIMP, + "%s: Locking port bits configuration isn't supported\n", + __func__); + s->lckr = value & ~RESERVED_BITS_MASK; + return; + case GPIO_AFRL: + qemu_log_mask(LOG_UNIMP, + "%s: Alternate functions aren't supported\n", + __func__); + s->afrl = value; + return; + case GPIO_AFRH: + qemu_log_mask(LOG_UNIMP, + "%s: Alternate functions aren't supported\n", + __func__); + s->afrh = value; + return; + case GPIO_BRR: { + uint32_t bits_to_reset = value & ~RESERVED_BITS_MASK; + s->odr &= ~bits_to_reset; + update_gpio_idr(s); + return; + } + case GPIO_ASCR: + qemu_log_mask(LOG_UNIMP, + "%s: ADC function isn't supported\n", + __func__); + s->ascr = value & ~RESERVED_BITS_MASK; + return; + default: + qemu_log_mask(LOG_GUEST_ERROR, + "%s: Bad offset 0x%" HWADDR_PRIx "\n", __func__, addr); + } +} + +static uint64_t stm32l4x5_gpio_read(void *opaque, hwaddr addr, + unsigned int size) +{ + Stm32l4x5GpioState *s = opaque; + + trace_stm32l4x5_gpio_read(s->name, addr); + + switch (addr) { + case GPIO_MODER: + return s->moder; + case GPIO_OTYPER: + return s->otyper; + case GPIO_OSPEEDR: + return s->ospeedr; + case GPIO_PUPDR: + return s->pupdr; + case GPIO_IDR: + return s->idr; + case GPIO_ODR: + return s->odr; + case GPIO_BSRR: + return 0; + case GPIO_LCKR: + return s->lckr; + case GPIO_AFRL: + return s->afrl; + case GPIO_AFRH: + return s->afrh; + case GPIO_BRR: + return 0; + case GPIO_ASCR: + return s->ascr; + default: + qemu_log_mask(LOG_GUEST_ERROR, + "%s: Bad offset 0x%" HWADDR_PRIx "\n", __func__, addr); + return 0; + } +} + +static const MemoryRegionOps stm32l4x5_gpio_ops = { + .read = stm32l4x5_gpio_read, + .write = stm32l4x5_gpio_write, + .endianness = DEVICE_NATIVE_ENDIAN, + .impl = { + .min_access_size = 4, + .max_access_size = 4, + .unaligned = false, + }, + .valid = { + .min_access_size = 4, + .max_access_size = 4, + .unaligned = false, + }, +}; + +static void stm32l4x5_gpio_init(Object *obj) +{ + Stm32l4x5GpioState *s = STM32L4X5_GPIO(obj); + + memory_region_init_io(&s->mmio, obj, &stm32l4x5_gpio_ops, s, + TYPE_STM32L4X5_GPIO, 0x400); + + sysbus_init_mmio(SYS_BUS_DEVICE(obj), &s->mmio); + + qdev_init_gpio_out(DEVICE(obj), s->pin, GPIO_NUM_PINS); + qdev_init_gpio_in(DEVICE(obj), stm32l4x5_gpio_set, GPIO_NUM_PINS); + + s->clk = qdev_init_clock_in(DEVICE(s), "clk", NULL, s, 0); + + object_property_add(obj, "disconnected-pins", "uint16", + disconnected_pins_get, disconnected_pins_set, + NULL, &s->disconnected_pins); + object_property_add(obj, "clock-freq-hz", "uint32", + clock_freq_get, NULL, NULL, NULL); +} + +static void stm32l4x5_gpio_realize(DeviceState *dev, Error **errp) +{ + Stm32l4x5GpioState *s = STM32L4X5_GPIO(dev); + if (!clock_has_source(s->clk)) { + error_setg(errp, "GPIO: clk input must be connected"); + return; + } +} + +static const VMStateDescription vmstate_stm32l4x5_gpio = { + .name = TYPE_STM32L4X5_GPIO, + .version_id = 1, + .minimum_version_id = 1, + .fields = (VMStateField[]){ + VMSTATE_UINT32(moder, Stm32l4x5GpioState), + VMSTATE_UINT32(otyper, Stm32l4x5GpioState), + VMSTATE_UINT32(ospeedr, Stm32l4x5GpioState), + VMSTATE_UINT32(pupdr, Stm32l4x5GpioState), + VMSTATE_UINT32(idr, Stm32l4x5GpioState), + VMSTATE_UINT32(odr, Stm32l4x5GpioState), + VMSTATE_UINT32(lckr, Stm32l4x5GpioState), + VMSTATE_UINT32(afrl, Stm32l4x5GpioState), + VMSTATE_UINT32(afrh, Stm32l4x5GpioState), + VMSTATE_UINT32(ascr, Stm32l4x5GpioState), + VMSTATE_UINT16(disconnected_pins, Stm32l4x5GpioState), + VMSTATE_UINT16(pins_connected_high, Stm32l4x5GpioState), + VMSTATE_END_OF_LIST() + } +}; + +static Property stm32l4x5_gpio_properties[] = { + DEFINE_PROP_STRING("name", Stm32l4x5GpioState, name), + DEFINE_PROP_UINT32("mode-reset", Stm32l4x5GpioState, moder_reset, 0), + DEFINE_PROP_UINT32("ospeed-reset", Stm32l4x5GpioState, ospeedr_reset, 0), + DEFINE_PROP_UINT32("pupd-reset", Stm32l4x5GpioState, pupdr_reset, 0), + DEFINE_PROP_END_OF_LIST(), +}; + +static void stm32l4x5_gpio_class_init(ObjectClass *klass, void *data) +{ + DeviceClass *dc = DEVICE_CLASS(klass); + ResettableClass *rc = RESETTABLE_CLASS(klass); + + device_class_set_props(dc, stm32l4x5_gpio_properties); + dc->vmsd = &vmstate_stm32l4x5_gpio; + dc->realize = stm32l4x5_gpio_realize; + rc->phases.hold = stm32l4x5_gpio_reset_hold; +} + +static const TypeInfo stm32l4x5_gpio_types[] = { + { + .name = TYPE_STM32L4X5_GPIO, + .parent = TYPE_SYS_BUS_DEVICE, + .instance_size = sizeof(Stm32l4x5GpioState), + .instance_init = stm32l4x5_gpio_init, + .class_init = stm32l4x5_gpio_class_init, + }, +}; + +DEFINE_TYPES(stm32l4x5_gpio_types) diff --git a/hw/gpio/Kconfig b/hw/gpio/Kconfig index XXXXXXX..XXXXXXX 100644 --- a/hw/gpio/Kconfig +++ b/hw/gpio/Kconfig @@ -XXX,XX +XXX,XX @@ config GPIO_PWR config SIFIVE_GPIO bool + +config STM32L4X5_GPIO + bool diff --git a/hw/gpio/meson.build b/hw/gpio/meson.build index XXXXXXX..XXXXXXX 100644 --- a/hw/gpio/meson.build +++ b/hw/gpio/meson.build @@ -XXX,XX +XXX,XX @@ system_ss.add(when: 'CONFIG_RASPI', if_true: files( 'bcm2835_gpio.c', 'bcm2838_gpio.c' )) +system_ss.add(when: 'CONFIG_STM32L4X5_SOC', if_true: files('stm32l4x5_gpio.c')) system_ss.add(when: 'CONFIG_ASPEED_SOC', if_true: files('aspeed_gpio.c')) system_ss.add(when: 'CONFIG_SIFIVE_GPIO', if_true: files('sifive_gpio.c')) diff --git a/hw/gpio/trace-events b/hw/gpio/trace-events index XXXXXXX..XXXXXXX 100644 --- a/hw/gpio/trace-events +++ b/hw/gpio/trace-events @@ -XXX,XX +XXX,XX @@ sifive_gpio_update_output_irq(int64_t line, int64_t value) "line %" PRIi64 " val # aspeed_gpio.c aspeed_gpio_read(uint64_t offset, uint64_t value) "offset: 0x%" PRIx64 " value 0x%" PRIx64 aspeed_gpio_write(uint64_t offset, uint64_t value) "offset: 0x%" PRIx64 " value 0x%" PRIx64 + +# stm32l4x5_gpio.c +stm32l4x5_gpio_read(char *gpio, uint64_t addr) "GPIO%s addr: 0x%" PRIx64 " " +stm32l4x5_gpio_write(char *gpio, uint64_t addr, uint64_t data) "GPIO%s addr: 0x%" PRIx64 " val: 0x%" PRIx64 "" +stm32l4x5_gpio_update_idr(char *gpio, uint32_t old_idr, uint32_t new_idr) "GPIO%s from: 0x%x to: 0x%x" +stm32l4x5_gpio_pins(char *gpio, uint16_t disconnected, uint16_t high) "GPIO%s disconnected pins: 0x%x levels: 0x%x" -- 2.34.1 From: Inès Varhol <ines.varhol@telecom-paris.fr> Signed-off-by: Arnaud Minier <arnaud.minier@telecom-paris.fr> Signed-off-by: Inès Varhol <ines.varhol@telecom-paris.fr> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Acked-by: Alistair Francis <alistair.francis@wdc.com> Message-id: 20240305210444.310665-3-ines.varhol@telecom-paris.fr Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- include/hw/arm/stm32l4x5_soc.h | 2 + include/hw/gpio/stm32l4x5_gpio.h | 1 + include/hw/misc/stm32l4x5_syscfg.h | 3 +- hw/arm/stm32l4x5_soc.c | 71 +++++++++++++++++++++++------- hw/misc/stm32l4x5_syscfg.c | 1 + hw/arm/Kconfig | 3 +- 6 files changed, 63 insertions(+), 18 deletions(-) diff --git a/include/hw/arm/stm32l4x5_soc.h b/include/hw/arm/stm32l4x5_soc.h index XXXXXXX..XXXXXXX 100644 --- a/include/hw/arm/stm32l4x5_soc.h +++ b/include/hw/arm/stm32l4x5_soc.h @@ -XXX,XX +XXX,XX @@ #include "hw/misc/stm32l4x5_syscfg.h" #include "hw/misc/stm32l4x5_exti.h" #include "hw/misc/stm32l4x5_rcc.h" +#include "hw/gpio/stm32l4x5_gpio.h" #include "qom/object.h" #define TYPE_STM32L4X5_SOC "stm32l4x5-soc" @@ -XXX,XX +XXX,XX @@ struct Stm32l4x5SocState { OrIRQState exti_or_gates[NUM_EXTI_OR_GATES]; Stm32l4x5SyscfgState syscfg; Stm32l4x5RccState rcc; + Stm32l4x5GpioState gpio[NUM_GPIOS]; MemoryRegion sram1; MemoryRegion sram2; diff --git a/include/hw/gpio/stm32l4x5_gpio.h b/include/hw/gpio/stm32l4x5_gpio.h index XXXXXXX..XXXXXXX 100644 --- a/include/hw/gpio/stm32l4x5_gpio.h +++ b/include/hw/gpio/stm32l4x5_gpio.h @@ -XXX,XX +XXX,XX @@ #define TYPE_STM32L4X5_GPIO "stm32l4x5-gpio" OBJECT_DECLARE_SIMPLE_TYPE(Stm32l4x5GpioState, STM32L4X5_GPIO) +#define NUM_GPIOS 8 #define GPIO_NUM_PINS 16 struct Stm32l4x5GpioState { diff --git a/include/hw/misc/stm32l4x5_syscfg.h b/include/hw/misc/stm32l4x5_syscfg.h index XXXXXXX..XXXXXXX 100644 --- a/include/hw/misc/stm32l4x5_syscfg.h +++ b/include/hw/misc/stm32l4x5_syscfg.h @@ -XXX,XX +XXX,XX @@ #include "hw/sysbus.h" #include "qom/object.h" +#include "hw/gpio/stm32l4x5_gpio.h" #define TYPE_STM32L4X5_SYSCFG "stm32l4x5-syscfg" OBJECT_DECLARE_SIMPLE_TYPE(Stm32l4x5SyscfgState, STM32L4X5_SYSCFG) -#define NUM_GPIOS 8 -#define GPIO_NUM_PINS 16 #define SYSCFG_NUM_EXTICR 4 struct Stm32l4x5SyscfgState { diff --git a/hw/arm/stm32l4x5_soc.c b/hw/arm/stm32l4x5_soc.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/stm32l4x5_soc.c +++ b/hw/arm/stm32l4x5_soc.c @@ -XXX,XX +XXX,XX @@ #include "sysemu/sysemu.h" #include "hw/or-irq.h" #include "hw/arm/stm32l4x5_soc.h" +#include "hw/gpio/stm32l4x5_gpio.h" #include "hw/qdev-clock.h" #include "hw/misc/unimp.h" @@ -XXX,XX +XXX,XX @@ static const int exti_or_gate1_lines_in[EXTI_OR_GATE1_NUM_LINES_IN] = { 16, 35, 36, 37, 38, }; +static const struct { + uint32_t addr; + uint32_t moder_reset; + uint32_t ospeedr_reset; + uint32_t pupdr_reset; +} stm32l4x5_gpio_cfg[NUM_GPIOS] = { + { 0x48000000, 0xABFFFFFF, 0x0C000000, 0x64000000 }, + { 0x48000400, 0xFFFFFEBF, 0x00000000, 0x00000100 }, + { 0x48000800, 0xFFFFFFFF, 0x00000000, 0x00000000 }, + { 0x48000C00, 0xFFFFFFFF, 0x00000000, 0x00000000 }, + { 0x48001000, 0xFFFFFFFF, 0x00000000, 0x00000000 }, + { 0x48001400, 0xFFFFFFFF, 0x00000000, 0x00000000 }, + { 0x48001800, 0xFFFFFFFF, 0x00000000, 0x00000000 }, + { 0x48001C00, 0x0000000F, 0x00000000, 0x00000000 }, +}; + static void stm32l4x5_soc_initfn(Object *obj) { Stm32l4x5SocState *s = STM32L4X5_SOC(obj); @@ -XXX,XX +XXX,XX @@ static void stm32l4x5_soc_initfn(Object *obj) } object_initialize_child(obj, "syscfg", &s->syscfg, TYPE_STM32L4X5_SYSCFG); object_initialize_child(obj, "rcc", &s->rcc, TYPE_STM32L4X5_RCC); + + for (unsigned i = 0; i < NUM_GPIOS; i++) { + g_autofree char *name = g_strdup_printf("gpio%c", 'a' + i); + object_initialize_child(obj, name, &s->gpio[i], TYPE_STM32L4X5_GPIO); + } } static void stm32l4x5_soc_realize(DeviceState *dev_soc, Error **errp) @@ -XXX,XX +XXX,XX @@ static void stm32l4x5_soc_realize(DeviceState *dev_soc, Error **errp) Stm32l4x5SocState *s = STM32L4X5_SOC(dev_soc); const Stm32l4x5SocClass *sc = STM32L4X5_SOC_GET_CLASS(dev_soc); MemoryRegion *system_memory = get_system_memory(); - DeviceState *armv7m; + DeviceState *armv7m, *dev; SysBusDevice *busdev; + uint32_t pin_index; if (!memory_region_init_rom(&s->flash, OBJECT(dev_soc), "flash", sc->flash_size, errp)) { @@ -XXX,XX +XXX,XX @@ static void stm32l4x5_soc_realize(DeviceState *dev_soc, Error **errp) return; } + /* GPIOs */ + for (unsigned i = 0; i < NUM_GPIOS; i++) { + g_autofree char *name = g_strdup_printf("%c", 'A' + i); + dev = DEVICE(&s->gpio[i]); + qdev_prop_set_string(dev, "name", name); + qdev_prop_set_uint32(dev, "mode-reset", + stm32l4x5_gpio_cfg[i].moder_reset); + qdev_prop_set_uint32(dev, "ospeed-reset", + stm32l4x5_gpio_cfg[i].ospeedr_reset); + qdev_prop_set_uint32(dev, "pupd-reset", + stm32l4x5_gpio_cfg[i].pupdr_reset); + busdev = SYS_BUS_DEVICE(&s->gpio[i]); + g_free(name); + name = g_strdup_printf("gpio%c-out", 'a' + i); + qdev_connect_clock_in(DEVICE(&s->gpio[i]), "clk", + qdev_get_clock_out(DEVICE(&(s->rcc)), name)); + if (!sysbus_realize(busdev, errp)) { + return; + } + sysbus_mmio_map(busdev, 0, stm32l4x5_gpio_cfg[i].addr); + } + /* System configuration controller */ busdev = SYS_BUS_DEVICE(&s->syscfg); if (!sysbus_realize(busdev, errp)) { return; } sysbus_mmio_map(busdev, 0, SYSCFG_ADDR); - /* - * TODO: when the GPIO device is implemented, connect it - * to SYCFG using `qdev_connect_gpio_out`, NUM_GPIOS and - * GPIO_NUM_PINS. - */ + + for (unsigned i = 0; i < NUM_GPIOS; i++) { + for (unsigned j = 0; j < GPIO_NUM_PINS; j++) { + pin_index = GPIO_NUM_PINS * i + j; + qdev_connect_gpio_out(DEVICE(&s->gpio[i]), j, + qdev_get_gpio_in(DEVICE(&s->syscfg), + pin_index)); + } + } /* EXTI device */ busdev = SYS_BUS_DEVICE(&s->exti); @@ -XXX,XX +XXX,XX @@ static void stm32l4x5_soc_realize(DeviceState *dev_soc, Error **errp) } } - for (unsigned i = 0; i < 16; i++) { + for (unsigned i = 0; i < GPIO_NUM_PINS; i++) { qdev_connect_gpio_out(DEVICE(&s->syscfg), i, qdev_get_gpio_in(DEVICE(&s->exti), i)); } @@ -XXX,XX +XXX,XX @@ static void stm32l4x5_soc_realize(DeviceState *dev_soc, Error **errp) /* RESERVED: 0x40024400, 0x7FDBC00 */ /* AHB2 BUS */ - create_unimplemented_device("GPIOA", 0x48000000, 0x400); - create_unimplemented_device("GPIOB", 0x48000400, 0x400); - create_unimplemented_device("GPIOC", 0x48000800, 0x400); - create_unimplemented_device("GPIOD", 0x48000C00, 0x400); - create_unimplemented_device("GPIOE", 0x48001000, 0x400); - create_unimplemented_device("GPIOF", 0x48001400, 0x400); - create_unimplemented_device("GPIOG", 0x48001800, 0x400); - create_unimplemented_device("GPIOH", 0x48001C00, 0x400); /* RESERVED: 0x48002000, 0x7FDBC00 */ create_unimplemented_device("OTG_FS", 0x50000000, 0x40000); create_unimplemented_device("ADC", 0x50040000, 0x400); diff --git a/hw/misc/stm32l4x5_syscfg.c b/hw/misc/stm32l4x5_syscfg.c index XXXXXXX..XXXXXXX 100644 --- a/hw/misc/stm32l4x5_syscfg.c +++ b/hw/misc/stm32l4x5_syscfg.c @@ -XXX,XX +XXX,XX @@ #include "hw/irq.h" #include "migration/vmstate.h" #include "hw/misc/stm32l4x5_syscfg.h" +#include "hw/gpio/stm32l4x5_gpio.h" #define SYSCFG_MEMRMP 0x00 #define SYSCFG_CFGR1 0x04 diff --git a/hw/arm/Kconfig b/hw/arm/Kconfig index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/Kconfig +++ b/hw/arm/Kconfig @@ -XXX,XX +XXX,XX @@ config STM32L4X5_SOC bool select ARM_V7M select OR_IRQ - select STM32L4X5_SYSCFG select STM32L4X5_EXTI + select STM32L4X5_SYSCFG select STM32L4X5_RCC + select STM32L4X5_GPIO config XLNX_ZYNQMP_ARM bool -- 2.34.1 From: Inès Varhol <ines.varhol@telecom-paris.fr> The testcase contains : - `test_idr_reset_value()` : Checks the reset values of MODER, OTYPER, PUPDR, ODR and IDR. - `test_gpio_output_mode()` : Checks that writing a bit in register ODR results in the corresponding pin rising or lowering, if this pin is configured in output mode. - `test_gpio_input_mode()` : Checks that a input pin set high or low externally results in the pin rising and lowering. - `test_pull_up_pull_down()` : Checks that a floating pin in pull-up/down mode is actually high/down. - `test_push_pull()` : Checks that a pin set externally is disconnected when configured in push-pull output mode, and can't be set externally while in this mode. - `test_open_drain()` : Checks that a pin set externally high is disconnected when configured in open-drain output mode, and can't be set high while in this mode. - `test_bsrr_brr()` : Checks that writing to BSRR and BRR has the desired result in ODR. - `test_clock_enable()` : Checks that GPIO clock is at the right frequency after enabling it. Acked-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Arnaud Minier <arnaud.minier@telecom-paris.fr> Signed-off-by: Inès Varhol <ines.varhol@telecom-paris.fr> Message-id: 20240305210444.310665-4-ines.varhol@telecom-paris.fr Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- tests/qtest/stm32l4x5_gpio-test.c | 551 ++++++++++++++++++++++++++++++ tests/qtest/meson.build | 3 +- 2 files changed, 553 insertions(+), 1 deletion(-) create mode 100644 tests/qtest/stm32l4x5_gpio-test.c diff --git a/tests/qtest/stm32l4x5_gpio-test.c b/tests/qtest/stm32l4x5_gpio-test.c new file mode 100644 index XXXXXXX..XXXXXXX --- /dev/null +++ b/tests/qtest/stm32l4x5_gpio-test.c @@ -XXX,XX +XXX,XX @@ +/* + * QTest testcase for STM32L4x5_GPIO + * + * Copyright (c) 2024 Arnaud Minier <arnaud.minier@telecom-paris.fr> + * Copyright (c) 2024 Inès Varhol <ines.varhol@telecom-paris.fr> + * + * This work is licensed under the terms of the GNU GPL, version 2 or later. + * See the COPYING file in the top-level directory. + */ + +#include "qemu/osdep.h" +#include "libqtest-single.h" + +#define GPIO_BASE_ADDR 0x48000000 +#define GPIO_SIZE 0x400 +#define NUM_GPIOS 8 +#define NUM_GPIO_PINS 16 + +#define GPIO_A 0x48000000 +#define GPIO_B 0x48000400 +#define GPIO_C 0x48000800 +#define GPIO_D 0x48000C00 +#define GPIO_E 0x48001000 +#define GPIO_F 0x48001400 +#define GPIO_G 0x48001800 +#define GPIO_H 0x48001C00 + +#define MODER 0x00 +#define OTYPER 0x04 +#define PUPDR 0x0C +#define IDR 0x10 +#define ODR 0x14 +#define BSRR 0x18 +#define BRR 0x28 + +#define MODER_INPUT 0 +#define MODER_OUTPUT 1 + +#define PUPDR_NONE 0 +#define PUPDR_PULLUP 1 +#define PUPDR_PULLDOWN 2 + +#define OTYPER_PUSH_PULL 0 +#define OTYPER_OPEN_DRAIN 1 + +const uint32_t moder_reset[NUM_GPIOS] = { + 0xABFFFFFF, + 0xFFFFFEBF, + 0xFFFFFFFF, + 0xFFFFFFFF, + 0xFFFFFFFF, + 0xFFFFFFFF, + 0xFFFFFFFF, + 0x0000000F +}; + +const uint32_t pupdr_reset[NUM_GPIOS] = { + 0x64000000, + 0x00000100, + 0x00000000, + 0x00000000, + 0x00000000, + 0x00000000, + 0x00000000, + 0x00000000 +}; + +const uint32_t idr_reset[NUM_GPIOS] = { + 0x0000A000, + 0x00000010, + 0x00000000, + 0x00000000, + 0x00000000, + 0x00000000, + 0x00000000, + 0x00000000 +}; + +static uint32_t gpio_readl(unsigned int gpio, unsigned int offset) +{ + return readl(gpio + offset); +} + +static void gpio_writel(unsigned int gpio, unsigned int offset, uint32_t value) +{ + writel(gpio + offset, value); +} + +static void gpio_set_bit(unsigned int gpio, unsigned int reg, + unsigned int pin, uint32_t value) +{ + uint32_t mask = 0xFFFFFFFF & ~(0x1 << pin); + gpio_writel(gpio, reg, (gpio_readl(gpio, reg) & mask) | value << pin); +} + +static void gpio_set_2bits(unsigned int gpio, unsigned int reg, + unsigned int pin, uint32_t value) +{ + uint32_t offset = 2 * pin; + uint32_t mask = 0xFFFFFFFF & ~(0x3 << offset); + gpio_writel(gpio, reg, (gpio_readl(gpio, reg) & mask) | value << offset); +} + +static unsigned int get_gpio_id(uint32_t gpio_addr) +{ + return (gpio_addr - GPIO_BASE_ADDR) / GPIO_SIZE; +} + +static void gpio_set_irq(unsigned int gpio, int num, int level) +{ + g_autofree char *name = g_strdup_printf("/machine/soc/gpio%c", + get_gpio_id(gpio) + 'a'); + qtest_set_irq_in(global_qtest, name, NULL, num, level); +} + +static void disconnect_all_pins(unsigned int gpio) +{ + g_autofree char *path = g_strdup_printf("/machine/soc/gpio%c", + get_gpio_id(gpio) + 'a'); + QDict *r; + + r = qtest_qmp(global_qtest, "{ 'execute': 'qom-set', 'arguments': " + "{ 'path': %s, 'property': 'disconnected-pins', 'value': %d } }", + path, 0xFFFF); + g_assert_false(qdict_haskey(r, "error")); + qobject_unref(r); +} + +static uint32_t get_disconnected_pins(unsigned int gpio) +{ + g_autofree char *path = g_strdup_printf("/machine/soc/gpio%c", + get_gpio_id(gpio) + 'a'); + uint32_t disconnected_pins = 0; + QDict *r; + + r = qtest_qmp(global_qtest, "{ 'execute': 'qom-get', 'arguments':" + " { 'path': %s, 'property': 'disconnected-pins'} }", path); + g_assert_false(qdict_haskey(r, "error")); + disconnected_pins = qdict_get_int(r, "return"); + qobject_unref(r); + return disconnected_pins; +} + +static uint32_t reset(uint32_t gpio, unsigned int offset) +{ + switch (offset) { + case MODER: + return moder_reset[get_gpio_id(gpio)]; + case PUPDR: + return pupdr_reset[get_gpio_id(gpio)]; + case IDR: + return idr_reset[get_gpio_id(gpio)]; + } + return 0x0; +} + +static void system_reset(void) +{ + QDict *r; + r = qtest_qmp(global_qtest, "{'execute': 'system_reset'}"); + g_assert_false(qdict_haskey(r, "error")); + qobject_unref(r); +} + +static void test_idr_reset_value(void) +{ + /* + * Checks that the values in MODER, OTYPER, PUPDR and ODR + * after reset are correct, and that the value in IDR is + * coherent. + * Since AF and analog modes aren't implemented, IDR reset + * values aren't the same as with a real board. + * + * Register IDR contains the actual values of all GPIO pins. + * Its value depends on the pins' configuration + * (intput/output/analog : register MODER, push-pull/open-drain : + * register OTYPER, pull-up/pull-down/none : register PUPDR) + * and on the values stored in register ODR + * (in case the pin is in output mode). + */ + + gpio_writel(GPIO_A, MODER, 0xDEADBEEF); + gpio_writel(GPIO_A, ODR, 0xDEADBEEF); + gpio_writel(GPIO_A, OTYPER, 0xDEADBEEF); + gpio_writel(GPIO_A, PUPDR, 0xDEADBEEF); + + gpio_writel(GPIO_B, MODER, 0xDEADBEEF); + gpio_writel(GPIO_B, ODR, 0xDEADBEEF); + gpio_writel(GPIO_B, OTYPER, 0xDEADBEEF); + gpio_writel(GPIO_B, PUPDR, 0xDEADBEEF); + + gpio_writel(GPIO_C, MODER, 0xDEADBEEF); + gpio_writel(GPIO_C, ODR, 0xDEADBEEF); + gpio_writel(GPIO_C, OTYPER, 0xDEADBEEF); + gpio_writel(GPIO_C, PUPDR, 0xDEADBEEF); + + gpio_writel(GPIO_H, MODER, 0xDEADBEEF); + gpio_writel(GPIO_H, ODR, 0xDEADBEEF); + gpio_writel(GPIO_H, OTYPER, 0xDEADBEEF); + gpio_writel(GPIO_H, PUPDR, 0xDEADBEEF); + + system_reset(); + + uint32_t moder = gpio_readl(GPIO_A, MODER); + uint32_t odr = gpio_readl(GPIO_A, ODR); + uint32_t otyper = gpio_readl(GPIO_A, OTYPER); + uint32_t pupdr = gpio_readl(GPIO_A, PUPDR); + uint32_t idr = gpio_readl(GPIO_A, IDR); + /* 15: AF, 14: AF, 13: AF, 12: Analog ... */ + /* here AF is the same as Analog and Input mode */ + g_assert_cmphex(moder, ==, reset(GPIO_A, MODER)); + g_assert_cmphex(odr, ==, reset(GPIO_A, ODR)); + g_assert_cmphex(otyper, ==, reset(GPIO_A, OTYPER)); + /* 15: pull-up, 14: pull-down, 13: pull-up, 12: neither ... */ + g_assert_cmphex(pupdr, ==, reset(GPIO_A, PUPDR)); + /* 15 : 1, 14: 0, 13: 1, 12 : reset value ... */ + g_assert_cmphex(idr, ==, reset(GPIO_A, IDR)); + + moder = gpio_readl(GPIO_B, MODER); + odr = gpio_readl(GPIO_B, ODR); + otyper = gpio_readl(GPIO_B, OTYPER); + pupdr = gpio_readl(GPIO_B, PUPDR); + idr = gpio_readl(GPIO_B, IDR); + /* ... 5: Analog, 4: AF, 3: AF, 2: Analog ... */ + /* here AF is the same as Analog and Input mode */ + g_assert_cmphex(moder, ==, reset(GPIO_B, MODER)); + g_assert_cmphex(odr, ==, reset(GPIO_B, ODR)); + g_assert_cmphex(otyper, ==, reset(GPIO_B, OTYPER)); + /* ... 5: neither, 4: pull-up, 3: neither ... */ + g_assert_cmphex(pupdr, ==, reset(GPIO_B, PUPDR)); + /* ... 5 : reset value, 4 : 1, 3 : reset value ... */ + g_assert_cmphex(idr, ==, reset(GPIO_B, IDR)); + + moder = gpio_readl(GPIO_C, MODER); + odr = gpio_readl(GPIO_C, ODR); + otyper = gpio_readl(GPIO_C, OTYPER); + pupdr = gpio_readl(GPIO_C, PUPDR); + idr = gpio_readl(GPIO_C, IDR); + /* Analog, same as Input mode*/ + g_assert_cmphex(moder, ==, reset(GPIO_C, MODER)); + g_assert_cmphex(odr, ==, reset(GPIO_C, ODR)); + g_assert_cmphex(otyper, ==, reset(GPIO_C, OTYPER)); + /* no pull-up or pull-down */ + g_assert_cmphex(pupdr, ==, reset(GPIO_C, PUPDR)); + /* reset value */ + g_assert_cmphex(idr, ==, reset(GPIO_C, IDR)); + + moder = gpio_readl(GPIO_H, MODER); + odr = gpio_readl(GPIO_H, ODR); + otyper = gpio_readl(GPIO_H, OTYPER); + pupdr = gpio_readl(GPIO_H, PUPDR); + idr = gpio_readl(GPIO_H, IDR); + /* Analog, same as Input mode */ + g_assert_cmphex(moder, ==, reset(GPIO_H, MODER)); + g_assert_cmphex(odr, ==, reset(GPIO_H, ODR)); + g_assert_cmphex(otyper, ==, reset(GPIO_H, OTYPER)); + /* no pull-up or pull-down */ + g_assert_cmphex(pupdr, ==, reset(GPIO_H, PUPDR)); + /* reset value */ + g_assert_cmphex(idr, ==, reset(GPIO_H, IDR)); +} + +static void test_gpio_output_mode(const void *data) +{ + /* + * Checks that setting a bit in ODR sets the corresponding + * GPIO line high : it should set the right bit in IDR + * and send an irq to syscfg. + * Additionally, it checks that values written to ODR + * when not in output mode are stored and not discarded. + */ + unsigned int pin = ((uint64_t)data) & 0xF; + uint32_t gpio = ((uint64_t)data) >> 32; + unsigned int gpio_id = get_gpio_id(gpio); + + qtest_irq_intercept_in(global_qtest, "/machine/soc/syscfg"); + + /* Set a bit in ODR and check nothing happens */ + gpio_set_bit(gpio, ODR, pin, 1); + g_assert_cmphex(gpio_readl(gpio, IDR), ==, reset(gpio, IDR)); + g_assert_false(get_irq(gpio_id * NUM_GPIO_PINS + pin)); + + /* Configure the relevant line as output and check the pin is high */ + gpio_set_2bits(gpio, MODER, pin, MODER_OUTPUT); + g_assert_cmphex(gpio_readl(gpio, IDR), ==, reset(gpio, IDR) | (1 << pin)); + g_assert_true(get_irq(gpio_id * NUM_GPIO_PINS + pin)); + + /* Reset the bit in ODR and check the pin is low */ + gpio_set_bit(gpio, ODR, pin, 0); + g_assert_cmphex(gpio_readl(gpio, IDR), ==, reset(gpio, IDR) & ~(1 << pin)); + g_assert_false(get_irq(gpio_id * NUM_GPIO_PINS + pin)); + + /* Clean the test */ + gpio_writel(gpio, ODR, reset(gpio, ODR)); + gpio_writel(gpio, MODER, reset(gpio, MODER)); + g_assert_cmphex(gpio_readl(gpio, IDR), ==, reset(gpio, IDR)); + g_assert_false(get_irq(gpio_id * NUM_GPIO_PINS + pin)); +} + +static void test_gpio_input_mode(const void *data) +{ + /* + * Test that setting a line high/low externally sets the + * corresponding GPIO line high/low : it should set the + * right bit in IDR and send an irq to syscfg. + */ + unsigned int pin = ((uint64_t)data) & 0xF; + uint32_t gpio = ((uint64_t)data) >> 32; + unsigned int gpio_id = get_gpio_id(gpio); + + qtest_irq_intercept_in(global_qtest, "/machine/soc/syscfg"); + + /* Configure a line as input, raise it, and check that the pin is high */ + gpio_set_2bits(gpio, MODER, pin, MODER_INPUT); + gpio_set_irq(gpio, pin, 1); + g_assert_cmphex(gpio_readl(gpio, IDR), ==, reset(gpio, IDR) | (1 << pin)); + g_assert_true(get_irq(gpio_id * NUM_GPIO_PINS + pin)); + + /* Lower the line and check that the pin is low */ + gpio_set_irq(gpio, pin, 0); + g_assert_cmphex(gpio_readl(gpio, IDR), ==, reset(gpio, IDR) & ~(1 << pin)); + g_assert_false(get_irq(gpio_id * NUM_GPIO_PINS + pin)); + + /* Clean the test */ + gpio_writel(gpio, MODER, reset(gpio, MODER)); + disconnect_all_pins(gpio); + g_assert_cmphex(gpio_readl(gpio, IDR), ==, reset(gpio, IDR)); +} + +static void test_pull_up_pull_down(const void *data) +{ + /* + * Test that a floating pin with pull-up sets the pin + * high and vice-versa. + */ + unsigned int pin = ((uint64_t)data) & 0xF; + uint32_t gpio = ((uint64_t)data) >> 32; + unsigned int gpio_id = get_gpio_id(gpio); + + qtest_irq_intercept_in(global_qtest, "/machine/soc/syscfg"); + + /* Configure a line as input with pull-up, check the line is set high */ + gpio_set_2bits(gpio, MODER, pin, MODER_INPUT); + gpio_set_2bits(gpio, PUPDR, pin, PUPDR_PULLUP); + g_assert_cmphex(gpio_readl(gpio, IDR), ==, reset(gpio, IDR) | (1 << pin)); + g_assert_true(get_irq(gpio_id * NUM_GPIO_PINS + pin)); + + /* Configure the line with pull-down, check the line is low */ + gpio_set_2bits(gpio, PUPDR, pin, PUPDR_PULLDOWN); + g_assert_cmphex(gpio_readl(gpio, IDR), ==, reset(gpio, IDR) & ~(1 << pin)); + g_assert_false(get_irq(gpio_id * NUM_GPIO_PINS + pin)); + + /* Clean the test */ + gpio_writel(gpio, MODER, reset(gpio, MODER)); + gpio_writel(gpio, PUPDR, reset(gpio, PUPDR)); + g_assert_cmphex(gpio_readl(gpio, IDR), ==, reset(gpio, IDR)); +} + +static void test_push_pull(const void *data) +{ + /* + * Test that configuring a line in push-pull output mode + * disconnects the pin, that the pin can't be set or reset + * externally afterwards. + */ + unsigned int pin = ((uint64_t)data) & 0xF; + uint32_t gpio = ((uint64_t)data) >> 32; + uint32_t gpio2 = GPIO_BASE_ADDR + (GPIO_H - gpio); + + qtest_irq_intercept_in(global_qtest, "/machine/soc/syscfg"); + + /* Setting a line high externally, configuring it in push-pull output */ + /* And checking the pin was disconnected */ + gpio_set_irq(gpio, pin, 1); + gpio_set_2bits(gpio, MODER, pin, MODER_OUTPUT); + g_assert_cmphex(get_disconnected_pins(gpio), ==, 0xFFFF); + g_assert_cmphex(gpio_readl(gpio, IDR), ==, reset(gpio, IDR) & ~(1 << pin)); + + /* Setting a line low externally, configuring it in push-pull output */ + /* And checking the pin was disconnected */ + gpio_set_irq(gpio2, pin, 0); + gpio_set_bit(gpio2, ODR, pin, 1); + gpio_set_2bits(gpio2, MODER, pin, MODER_OUTPUT); + g_assert_cmphex(get_disconnected_pins(gpio2), ==, 0xFFFF); + g_assert_cmphex(gpio_readl(gpio2, IDR), ==, reset(gpio2, IDR) | (1 << pin)); + + /* Trying to set a push-pull output pin, checking it doesn't work */ + gpio_set_irq(gpio, pin, 1); + g_assert_cmphex(get_disconnected_pins(gpio), ==, 0xFFFF); + g_assert_cmphex(gpio_readl(gpio, IDR), ==, reset(gpio, IDR) & ~(1 << pin)); + + /* Trying to reset a push-pull output pin, checking it doesn't work */ + gpio_set_irq(gpio2, pin, 0); + g_assert_cmphex(get_disconnected_pins(gpio2), ==, 0xFFFF); + g_assert_cmphex(gpio_readl(gpio2, IDR), ==, reset(gpio2, IDR) | (1 << pin)); + + /* Clean the test */ + gpio_writel(gpio, MODER, reset(gpio, MODER)); + gpio_writel(gpio2, ODR, reset(gpio2, ODR)); + gpio_writel(gpio2, MODER, reset(gpio2, MODER)); +} + +static void test_open_drain(const void *data) +{ + /* + * Test that configuring a line in open-drain output mode + * disconnects a pin set high externally and that the pin + * can't be set high externally while configured in open-drain. + * + * However a pin set low externally shouldn't be disconnected, + * and it can be set low externally when in open-drain mode. + */ + unsigned int pin = ((uint64_t)data) & 0xF; + uint32_t gpio = ((uint64_t)data) >> 32; + uint32_t gpio2 = GPIO_BASE_ADDR + (GPIO_H - gpio); + + qtest_irq_intercept_in(global_qtest, "/machine/soc/syscfg"); + + /* Setting a line high externally, configuring it in open-drain output */ + /* And checking the pin was disconnected */ + gpio_set_irq(gpio, pin, 1); + gpio_set_bit(gpio, OTYPER, pin, OTYPER_OPEN_DRAIN); + gpio_set_2bits(gpio, MODER, pin, MODER_OUTPUT); + g_assert_cmphex(get_disconnected_pins(gpio), ==, 0xFFFF); + g_assert_cmphex(gpio_readl(gpio, IDR), ==, reset(gpio, IDR) & ~(1 << pin)); + + /* Setting a line low externally, configuring it in open-drain output */ + /* And checking the pin wasn't disconnected */ + gpio_set_irq(gpio2, pin, 0); + gpio_set_bit(gpio2, ODR, pin, 1); + gpio_set_bit(gpio2, OTYPER, pin, OTYPER_OPEN_DRAIN); + gpio_set_2bits(gpio2, MODER, pin, MODER_OUTPUT); + g_assert_cmphex(get_disconnected_pins(gpio2), ==, 0xFFFF & ~(1 << pin)); + g_assert_cmphex(gpio_readl(gpio2, IDR), ==, + reset(gpio2, IDR) & ~(1 << pin)); + + /* Trying to set a open-drain output pin, checking it doesn't work */ + gpio_set_irq(gpio, pin, 1); + g_assert_cmphex(get_disconnected_pins(gpio), ==, 0xFFFF); + g_assert_cmphex(gpio_readl(gpio, IDR), ==, reset(gpio, IDR) & ~(1 << pin)); + + /* Trying to reset a open-drain output pin, checking it works */ + gpio_set_bit(gpio, ODR, pin, 1); + gpio_set_irq(gpio, pin, 0); + g_assert_cmphex(get_disconnected_pins(gpio2), ==, 0xFFFF & ~(1 << pin)); + g_assert_cmphex(gpio_readl(gpio2, IDR), ==, + reset(gpio2, IDR) & ~(1 << pin)); + + /* Clean the test */ + disconnect_all_pins(gpio2); + gpio_writel(gpio2, OTYPER, reset(gpio2, OTYPER)); + gpio_writel(gpio2, ODR, reset(gpio2, ODR)); + gpio_writel(gpio2, MODER, reset(gpio2, MODER)); + g_assert_cmphex(gpio_readl(gpio2, IDR), ==, reset(gpio2, IDR)); + disconnect_all_pins(gpio); + gpio_writel(gpio, OTYPER, reset(gpio, OTYPER)); + gpio_writel(gpio, ODR, reset(gpio, ODR)); + gpio_writel(gpio, MODER, reset(gpio, MODER)); + g_assert_cmphex(gpio_readl(gpio, IDR), ==, reset(gpio, IDR)); +} + +static void test_bsrr_brr(const void *data) +{ + /* + * Test that writing a '1' in BSS and BSRR + * has the desired effect on ODR. + * In BSRR, BSx has priority over BRx. + */ + unsigned int pin = ((uint64_t)data) & 0xF; + uint32_t gpio = ((uint64_t)data) >> 32; + + gpio_writel(gpio, BSRR, (1 << pin)); + g_assert_cmphex(gpio_readl(gpio, ODR), ==, reset(gpio, ODR) | (1 << pin)); + + gpio_writel(gpio, BSRR, (1 << (pin + NUM_GPIO_PINS))); + g_assert_cmphex(gpio_readl(gpio, ODR), ==, reset(gpio, ODR)); + + gpio_writel(gpio, BSRR, (1 << pin)); + g_assert_cmphex(gpio_readl(gpio, ODR), ==, reset(gpio, ODR) | (1 << pin)); + + gpio_writel(gpio, BRR, (1 << pin)); + g_assert_cmphex(gpio_readl(gpio, ODR), ==, reset(gpio, ODR)); + + /* BSx should have priority over BRx */ + gpio_writel(gpio, BSRR, (1 << pin) | (1 << (pin + NUM_GPIO_PINS))); + g_assert_cmphex(gpio_readl(gpio, ODR), ==, reset(gpio, ODR) | (1 << pin)); + + gpio_writel(gpio, BRR, (1 << pin)); + g_assert_cmphex(gpio_readl(gpio, ODR), ==, reset(gpio, ODR)); + + gpio_writel(gpio, ODR, reset(gpio, ODR)); +} + +int main(int argc, char **argv) +{ + int ret; + + g_test_init(&argc, &argv, NULL); + g_test_set_nonfatal_assertions(); + qtest_add_func("stm32l4x5/gpio/test_idr_reset_value", + test_idr_reset_value); + /* + * The inputs for the tests (gpio and pin) can be changed, + * but the tests don't work for pins that are high at reset + * (GPIOA15, GPIO13 and GPIOB5). + * Specifically, rising the pin then checking `get_irq()` + * is problematic since the pin was already high. + */ + qtest_add_data_func("stm32l4x5/gpio/test_gpioc5_output_mode", + (void *)((uint64_t)GPIO_C << 32 | 5), + test_gpio_output_mode); + qtest_add_data_func("stm32l4x5/gpio/test_gpioh3_output_mode", + (void *)((uint64_t)GPIO_H << 32 | 3), + test_gpio_output_mode); + qtest_add_data_func("stm32l4x5/gpio/test_gpio_input_mode1", + (void *)((uint64_t)GPIO_D << 32 | 6), + test_gpio_input_mode); + qtest_add_data_func("stm32l4x5/gpio/test_gpio_input_mode2", + (void *)((uint64_t)GPIO_C << 32 | 10), + test_gpio_input_mode); + qtest_add_data_func("stm32l4x5/gpio/test_gpio_pull_up_pull_down1", + (void *)((uint64_t)GPIO_B << 32 | 5), + test_pull_up_pull_down); + qtest_add_data_func("stm32l4x5/gpio/test_gpio_pull_up_pull_down2", + (void *)((uint64_t)GPIO_F << 32 | 1), + test_pull_up_pull_down); + qtest_add_data_func("stm32l4x5/gpio/test_gpio_push_pull1", + (void *)((uint64_t)GPIO_G << 32 | 6), + test_push_pull); + qtest_add_data_func("stm32l4x5/gpio/test_gpio_push_pull2", + (void *)((uint64_t)GPIO_H << 32 | 3), + test_push_pull); + qtest_add_data_func("stm32l4x5/gpio/test_gpio_open_drain1", + (void *)((uint64_t)GPIO_C << 32 | 4), + test_open_drain); + qtest_add_data_func("stm32l4x5/gpio/test_gpio_open_drain2", + (void *)((uint64_t)GPIO_E << 32 | 11), + test_open_drain); + qtest_add_data_func("stm32l4x5/gpio/test_bsrr_brr1", + (void *)((uint64_t)GPIO_A << 32 | 12), + test_bsrr_brr); + qtest_add_data_func("stm32l4x5/gpio/test_bsrr_brr2", + (void *)((uint64_t)GPIO_D << 32 | 0), + test_bsrr_brr); + + qtest_start("-machine b-l475e-iot01a"); + ret = g_test_run(); + qtest_end(); + + return ret; +} diff --git a/tests/qtest/meson.build b/tests/qtest/meson.build index XXXXXXX..XXXXXXX 100644 --- a/tests/qtest/meson.build +++ b/tests/qtest/meson.build @@ -XXX,XX +XXX,XX @@ qtests_aspeed = \ qtests_stm32l4x5 = \ ['stm32l4x5_exti-test', 'stm32l4x5_syscfg-test', - 'stm32l4x5_rcc-test'] + 'stm32l4x5_rcc-test', + 'stm32l4x5_gpio-test'] qtests_arm = \ (config_all_devices.has_key('CONFIG_MPS2') ? ['sse-timer-test'] : []) + \ -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> While the 8-bit input elements are sequential in the input vector, the 32-bit output elements are not sequential in the output matrix. Do not attempt to compute 2 32-bit outputs at the same time. Cc: qemu-stable@nongnu.org Fixes: 23a5e3859f5 ("target/arm: Implement SME integer outer product") Resolves: https://gitlab.com/qemu-project/qemu/-/issues/2083 Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Message-id: 20240305163931.242795-1-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/sme_helper.c | 77 ++++++++++++++++++------------- tests/tcg/aarch64/sme-smopa-1.c | 47 +++++++++++++++++++ tests/tcg/aarch64/sme-smopa-2.c | 54 ++++++++++++++++++++++ tests/tcg/aarch64/Makefile.target | 2 +- 4 files changed, 147 insertions(+), 33 deletions(-) create mode 100644 tests/tcg/aarch64/sme-smopa-1.c create mode 100644 tests/tcg/aarch64/sme-smopa-2.c diff --git a/target/arm/tcg/sme_helper.c b/target/arm/tcg/sme_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/sme_helper.c +++ b/target/arm/tcg/sme_helper.c @@ -XXX,XX +XXX,XX @@ void HELPER(sme_bfmopa)(void *vza, void *vzn, void *vzm, void *vpn, } } -typedef uint64_t IMOPFn(uint64_t, uint64_t, uint64_t, uint8_t, bool); +typedef uint32_t IMOPFn32(uint32_t, uint32_t, uint32_t, uint8_t, bool); +static inline void do_imopa_s(uint32_t *za, uint32_t *zn, uint32_t *zm, + uint8_t *pn, uint8_t *pm, + uint32_t desc, IMOPFn32 *fn) +{ + intptr_t row, col, oprsz = simd_oprsz(desc) / 4; + bool neg = simd_data(desc); -static inline void do_imopa(uint64_t *za, uint64_t *zn, uint64_t *zm, - uint8_t *pn, uint8_t *pm, - uint32_t desc, IMOPFn *fn) + for (row = 0; row < oprsz; ++row) { + uint8_t pa = (pn[H1(row >> 1)] >> ((row & 1) * 4)) & 0xf; + uint32_t *za_row = &za[tile_vslice_index(row)]; + uint32_t n = zn[H4(row)]; + + for (col = 0; col < oprsz; ++col) { + uint8_t pb = pm[H1(col >> 1)] >> ((col & 1) * 4); + uint32_t *a = &za_row[H4(col)]; + + *a = fn(n, zm[H4(col)], *a, pa & pb, neg); + } + } +} + +typedef uint64_t IMOPFn64(uint64_t, uint64_t, uint64_t, uint8_t, bool); +static inline void do_imopa_d(uint64_t *za, uint64_t *zn, uint64_t *zm, + uint8_t *pn, uint8_t *pm, + uint32_t desc, IMOPFn64 *fn) { intptr_t row, col, oprsz = simd_oprsz(desc) / 8; bool neg = simd_data(desc); @@ -XXX,XX +XXX,XX @@ static inline void do_imopa(uint64_t *za, uint64_t *zn, uint64_t *zm, } #define DEF_IMOP_32(NAME, NTYPE, MTYPE) \ -static uint64_t NAME(uint64_t n, uint64_t m, uint64_t a, uint8_t p, bool neg) \ +static uint32_t NAME(uint32_t n, uint32_t m, uint32_t a, uint8_t p, bool neg) \ { \ - uint32_t sum0 = 0, sum1 = 0; \ + uint32_t sum = 0; \ /* Apply P to N as a mask, making the inactive elements 0. */ \ n &= expand_pred_b(p); \ - sum0 += (NTYPE)(n >> 0) * (MTYPE)(m >> 0); \ - sum0 += (NTYPE)(n >> 8) * (MTYPE)(m >> 8); \ - sum0 += (NTYPE)(n >> 16) * (MTYPE)(m >> 16); \ - sum0 += (NTYPE)(n >> 24) * (MTYPE)(m >> 24); \ - sum1 += (NTYPE)(n >> 32) * (MTYPE)(m >> 32); \ - sum1 += (NTYPE)(n >> 40) * (MTYPE)(m >> 40); \ - sum1 += (NTYPE)(n >> 48) * (MTYPE)(m >> 48); \ - sum1 += (NTYPE)(n >> 56) * (MTYPE)(m >> 56); \ - if (neg) { \ - sum0 = (uint32_t)a - sum0, sum1 = (uint32_t)(a >> 32) - sum1; \ - } else { \ - sum0 = (uint32_t)a + sum0, sum1 = (uint32_t)(a >> 32) + sum1; \ - } \ - return ((uint64_t)sum1 << 32) | sum0; \ + sum += (NTYPE)(n >> 0) * (MTYPE)(m >> 0); \ + sum += (NTYPE)(n >> 8) * (MTYPE)(m >> 8); \ + sum += (NTYPE)(n >> 16) * (MTYPE)(m >> 16); \ + sum += (NTYPE)(n >> 24) * (MTYPE)(m >> 24); \ + return neg ? a - sum : a + sum; \ } #define DEF_IMOP_64(NAME, NTYPE, MTYPE) \ @@ -XXX,XX +XXX,XX @@ DEF_IMOP_64(umopa_d, uint16_t, uint16_t) DEF_IMOP_64(sumopa_d, int16_t, uint16_t) DEF_IMOP_64(usmopa_d, uint16_t, int16_t) -#define DEF_IMOPH(NAME) \ - void HELPER(sme_##NAME)(void *vza, void *vzn, void *vzm, void *vpn, \ - void *vpm, uint32_t desc) \ - { do_imopa(vza, vzn, vzm, vpn, vpm, desc, NAME); } +#define DEF_IMOPH(NAME, S) \ + void HELPER(sme_##NAME##_##S)(void *vza, void *vzn, void *vzm, \ + void *vpn, void *vpm, uint32_t desc) \ + { do_imopa_##S(vza, vzn, vzm, vpn, vpm, desc, NAME##_##S); } -DEF_IMOPH(smopa_s) -DEF_IMOPH(umopa_s) -DEF_IMOPH(sumopa_s) -DEF_IMOPH(usmopa_s) -DEF_IMOPH(smopa_d) -DEF_IMOPH(umopa_d) -DEF_IMOPH(sumopa_d) -DEF_IMOPH(usmopa_d) +DEF_IMOPH(smopa, s) +DEF_IMOPH(umopa, s) +DEF_IMOPH(sumopa, s) +DEF_IMOPH(usmopa, s) + +DEF_IMOPH(smopa, d) +DEF_IMOPH(umopa, d) +DEF_IMOPH(sumopa, d) +DEF_IMOPH(usmopa, d) diff --git a/tests/tcg/aarch64/sme-smopa-1.c b/tests/tcg/aarch64/sme-smopa-1.c new file mode 100644 index XXXXXXX..XXXXXXX --- /dev/null +++ b/tests/tcg/aarch64/sme-smopa-1.c @@ -XXX,XX +XXX,XX @@ +#include <stdio.h> +#include <string.h> + +int main() +{ + static const int cmp[4][4] = { + { 110, 134, 158, 182 }, + { 390, 478, 566, 654 }, + { 670, 822, 974, 1126 }, + { 950, 1166, 1382, 1598 } + }; + int dst[4][4]; + int *tmp = &dst[0][0]; + + asm volatile( + ".arch armv8-r+sme\n\t" + "smstart\n\t" + "index z0.b, #0, #1\n\t" + "movprfx z1, z0\n\t" + "add z1.b, z1.b, #16\n\t" + "ptrue p0.b\n\t" + "smopa za0.s, p0/m, p0/m, z0.b, z1.b\n\t" + "ptrue p0.s, vl4\n\t" + "mov w12, #0\n\t" + "st1w { za0h.s[w12, #0] }, p0, [%0]\n\t" + "add %0, %0, #16\n\t" + "st1w { za0h.s[w12, #1] }, p0, [%0]\n\t" + "add %0, %0, #16\n\t" + "st1w { za0h.s[w12, #2] }, p0, [%0]\n\t" + "add %0, %0, #16\n\t" + "st1w { za0h.s[w12, #3] }, p0, [%0]\n\t" + "smstop" + : "+r"(tmp) : : "memory"); + + if (memcmp(cmp, dst, sizeof(dst)) == 0) { + return 0; + } + + /* See above for correct results. */ + for (int i = 0; i < 4; ++i) { + for (int j = 0; j < 4; ++j) { + printf("%6d", dst[i][j]); + } + printf("\n"); + } + return 1; +} diff --git a/tests/tcg/aarch64/sme-smopa-2.c b/tests/tcg/aarch64/sme-smopa-2.c new file mode 100644 index XXXXXXX..XXXXXXX --- /dev/null +++ b/tests/tcg/aarch64/sme-smopa-2.c @@ -XXX,XX +XXX,XX @@ +#include <stdio.h> +#include <string.h> + +int main() +{ + static const long cmp[4][4] = { + { 110, 134, 158, 182 }, + { 390, 478, 566, 654 }, + { 670, 822, 974, 1126 }, + { 950, 1166, 1382, 1598 } + }; + long dst[4][4]; + long *tmp = &dst[0][0]; + long svl; + + /* Validate that we have a wide enough vector for 4 elements. */ + asm(".arch armv8-r+sme-i64\n\trdsvl %0, #1" : "=r"(svl)); + if (svl < 32) { + return 0; + } + + asm volatile( + "smstart\n\t" + "index z0.h, #0, #1\n\t" + "movprfx z1, z0\n\t" + "add z1.h, z1.h, #16\n\t" + "ptrue p0.b\n\t" + "smopa za0.d, p0/m, p0/m, z0.h, z1.h\n\t" + "ptrue p0.d, vl4\n\t" + "mov w12, #0\n\t" + "st1d { za0h.d[w12, #0] }, p0, [%0]\n\t" + "add %0, %0, #32\n\t" + "st1d { za0h.d[w12, #1] }, p0, [%0]\n\t" + "mov w12, #2\n\t" + "add %0, %0, #32\n\t" + "st1d { za0h.d[w12, #0] }, p0, [%0]\n\t" + "add %0, %0, #32\n\t" + "st1d { za0h.d[w12, #1] }, p0, [%0]\n\t" + "smstop" + : "+r"(tmp) : : "memory"); + + if (memcmp(cmp, dst, sizeof(dst)) == 0) { + return 0; + } + + /* See above for correct results. */ + for (int i = 0; i < 4; ++i) { + for (int j = 0; j < 4; ++j) { + printf("%6ld", dst[i][j]); + } + printf("\n"); + } + return 1; +} diff --git a/tests/tcg/aarch64/Makefile.target b/tests/tcg/aarch64/Makefile.target index XXXXXXX..XXXXXXX 100644 --- a/tests/tcg/aarch64/Makefile.target +++ b/tests/tcg/aarch64/Makefile.target @@ -XXX,XX +XXX,XX @@ endif # SME Tests ifneq ($(CROSS_AS_HAS_ARMV9_SME),) -AARCH64_TESTS += sme-outprod1 +AARCH64_TESTS += sme-outprod1 sme-smopa-1 sme-smopa-2 endif # System Registers Tests -- 2.34.1 The sun4v RTC device model added under commit a0e893039cf2ce0 in 2016 was unfortunately added with a license of GPL-v3-or-later, which is not compatible with other QEMU code which has a GPL-v2-only license. Relicense the code in the .c and the .h file to GPL-v2-or-later, to make it compatible with the rest of QEMU. Cc: qemu-stable@nongnu.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Paolo Bonzini (for Red Hat) <pbonzini@redhat.com> Signed-off-by: Artyom Tarasenko <atar4qemu@gmail.com> Signed-off-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Acked-by: Alex Bennée <alex.bennee@linaro.org> Message-id: 20240223161300.938542-1-peter.maydell@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- include/hw/rtc/sun4v-rtc.h | 2 +- hw/rtc/sun4v-rtc.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/include/hw/rtc/sun4v-rtc.h b/include/hw/rtc/sun4v-rtc.h index XXXXXXX..XXXXXXX 100644 --- a/include/hw/rtc/sun4v-rtc.h +++ b/include/hw/rtc/sun4v-rtc.h @@ -XXX,XX +XXX,XX @@ * * Copyright (c) 2016 Artyom Tarasenko * - * This code is licensed under the GNU GPL v3 or (at your option) any later + * This code is licensed under the GNU GPL v2 or (at your option) any later * version. */ diff --git a/hw/rtc/sun4v-rtc.c b/hw/rtc/sun4v-rtc.c index XXXXXXX..XXXXXXX 100644 --- a/hw/rtc/sun4v-rtc.c +++ b/hw/rtc/sun4v-rtc.c @@ -XXX,XX +XXX,XX @@ * * Copyright (c) 2016 Artyom Tarasenko * - * This code is licensed under the GNU GPL v3 or (at your option) any later + * This code is licensed under the GNU GPL v2 or (at your option) any later * version. */ -- 2.34.1 From: Thomas Huth <thuth@redhat.com> Move the code to a separate file so that we do not have to compile it anymore if CONFIG_ARM_V7M is not set. Signed-off-by: Thomas Huth <thuth@redhat.com> Message-id: 20240308141051.536599-2-thuth@redhat.com Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/cpu-v7m.c | 290 +++++++++++++++++++++++++++++++++++++ target/arm/tcg/cpu32.c | 261 --------------------------------- target/arm/meson.build | 3 + target/arm/tcg/meson.build | 3 + 4 files changed, 296 insertions(+), 261 deletions(-) create mode 100644 target/arm/tcg/cpu-v7m.c diff --git a/target/arm/tcg/cpu-v7m.c b/target/arm/tcg/cpu-v7m.c new file mode 100644 index XXXXXXX..XXXXXXX --- /dev/null +++ b/target/arm/tcg/cpu-v7m.c @@ -XXX,XX +XXX,XX @@ +/* + * QEMU ARMv7-M TCG-only CPUs. + * + * Copyright (c) 2012 SUSE LINUX Products GmbH + * + * This code is licensed under the GNU GPL v2 or later. + * + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#include "qemu/osdep.h" +#include "cpu.h" +#include "hw/core/tcg-cpu-ops.h" +#include "internals.h" + +#if !defined(CONFIG_USER_ONLY) + +#include "hw/intc/armv7m_nvic.h" + +static bool arm_v7m_cpu_exec_interrupt(CPUState *cs, int interrupt_request) +{ + CPUClass *cc = CPU_GET_CLASS(cs); + ARMCPU *cpu = ARM_CPU(cs); + CPUARMState *env = &cpu->env; + bool ret = false; + + /* + * ARMv7-M interrupt masking works differently than -A or -R. + * There is no FIQ/IRQ distinction. Instead of I and F bits + * masking FIQ and IRQ interrupts, an exception is taken only + * if it is higher priority than the current execution priority + * (which depends on state like BASEPRI, FAULTMASK and the + * currently active exception). + */ + if (interrupt_request & CPU_INTERRUPT_HARD + && (armv7m_nvic_can_take_pending_exception(env->nvic))) { + cs->exception_index = EXCP_IRQ; + cc->tcg_ops->do_interrupt(cs); + ret = true; + } + return ret; +} + +#endif /* !CONFIG_USER_ONLY */ + +static void cortex_m0_initfn(Object *obj) +{ + ARMCPU *cpu = ARM_CPU(obj); + set_feature(&cpu->env, ARM_FEATURE_V6); + set_feature(&cpu->env, ARM_FEATURE_M); + + cpu->midr = 0x410cc200; + + /* + * These ID register values are not guest visible, because + * we do not implement the Main Extension. They must be set + * to values corresponding to the Cortex-M0's implemented + * features, because QEMU generally controls its emulation + * by looking at ID register fields. We use the same values as + * for the M3. + */ + cpu->isar.id_pfr0 = 0x00000030; + cpu->isar.id_pfr1 = 0x00000200; + cpu->isar.id_dfr0 = 0x00100000; + cpu->id_afr0 = 0x00000000; + cpu->isar.id_mmfr0 = 0x00000030; + cpu->isar.id_mmfr1 = 0x00000000; + cpu->isar.id_mmfr2 = 0x00000000; + cpu->isar.id_mmfr3 = 0x00000000; + cpu->isar.id_isar0 = 0x01141110; + cpu->isar.id_isar1 = 0x02111000; + cpu->isar.id_isar2 = 0x21112231; + cpu->isar.id_isar3 = 0x01111110; + cpu->isar.id_isar4 = 0x01310102; + cpu->isar.id_isar5 = 0x00000000; + cpu->isar.id_isar6 = 0x00000000; +} + +static void cortex_m3_initfn(Object *obj) +{ + ARMCPU *cpu = ARM_CPU(obj); + set_feature(&cpu->env, ARM_FEATURE_V7); + set_feature(&cpu->env, ARM_FEATURE_M); + set_feature(&cpu->env, ARM_FEATURE_M_MAIN); + cpu->midr = 0x410fc231; + cpu->pmsav7_dregion = 8; + cpu->isar.id_pfr0 = 0x00000030; + cpu->isar.id_pfr1 = 0x00000200; + cpu->isar.id_dfr0 = 0x00100000; + cpu->id_afr0 = 0x00000000; + cpu->isar.id_mmfr0 = 0x00000030; + cpu->isar.id_mmfr1 = 0x00000000; + cpu->isar.id_mmfr2 = 0x00000000; + cpu->isar.id_mmfr3 = 0x00000000; + cpu->isar.id_isar0 = 0x01141110; + cpu->isar.id_isar1 = 0x02111000; + cpu->isar.id_isar2 = 0x21112231; + cpu->isar.id_isar3 = 0x01111110; + cpu->isar.id_isar4 = 0x01310102; + cpu->isar.id_isar5 = 0x00000000; + cpu->isar.id_isar6 = 0x00000000; +} + +static void cortex_m4_initfn(Object *obj) +{ + ARMCPU *cpu = ARM_CPU(obj); + + set_feature(&cpu->env, ARM_FEATURE_V7); + set_feature(&cpu->env, ARM_FEATURE_M); + set_feature(&cpu->env, ARM_FEATURE_M_MAIN); + set_feature(&cpu->env, ARM_FEATURE_THUMB_DSP); + cpu->midr = 0x410fc240; /* r0p0 */ + cpu->pmsav7_dregion = 8; + cpu->isar.mvfr0 = 0x10110021; + cpu->isar.mvfr1 = 0x11000011; + cpu->isar.mvfr2 = 0x00000000; + cpu->isar.id_pfr0 = 0x00000030; + cpu->isar.id_pfr1 = 0x00000200; + cpu->isar.id_dfr0 = 0x00100000; + cpu->id_afr0 = 0x00000000; + cpu->isar.id_mmfr0 = 0x00000030; + cpu->isar.id_mmfr1 = 0x00000000; + cpu->isar.id_mmfr2 = 0x00000000; + cpu->isar.id_mmfr3 = 0x00000000; + cpu->isar.id_isar0 = 0x01141110; + cpu->isar.id_isar1 = 0x02111000; + cpu->isar.id_isar2 = 0x21112231; + cpu->isar.id_isar3 = 0x01111110; + cpu->isar.id_isar4 = 0x01310102; + cpu->isar.id_isar5 = 0x00000000; + cpu->isar.id_isar6 = 0x00000000; +} + +static void cortex_m7_initfn(Object *obj) +{ + ARMCPU *cpu = ARM_CPU(obj); + + set_feature(&cpu->env, ARM_FEATURE_V7); + set_feature(&cpu->env, ARM_FEATURE_M); + set_feature(&cpu->env, ARM_FEATURE_M_MAIN); + set_feature(&cpu->env, ARM_FEATURE_THUMB_DSP); + cpu->midr = 0x411fc272; /* r1p2 */ + cpu->pmsav7_dregion = 8; + cpu->isar.mvfr0 = 0x10110221; + cpu->isar.mvfr1 = 0x12000011; + cpu->isar.mvfr2 = 0x00000040; + cpu->isar.id_pfr0 = 0x00000030; + cpu->isar.id_pfr1 = 0x00000200; + cpu->isar.id_dfr0 = 0x00100000; + cpu->id_afr0 = 0x00000000; + cpu->isar.id_mmfr0 = 0x00100030; + cpu->isar.id_mmfr1 = 0x00000000; + cpu->isar.id_mmfr2 = 0x01000000; + cpu->isar.id_mmfr3 = 0x00000000; + cpu->isar.id_isar0 = 0x01101110; + cpu->isar.id_isar1 = 0x02112000; + cpu->isar.id_isar2 = 0x20232231; + cpu->isar.id_isar3 = 0x01111131; + cpu->isar.id_isar4 = 0x01310132; + cpu->isar.id_isar5 = 0x00000000; + cpu->isar.id_isar6 = 0x00000000; +} + +static void cortex_m33_initfn(Object *obj) +{ + ARMCPU *cpu = ARM_CPU(obj); + + set_feature(&cpu->env, ARM_FEATURE_V8); + set_feature(&cpu->env, ARM_FEATURE_M); + set_feature(&cpu->env, ARM_FEATURE_M_MAIN); + set_feature(&cpu->env, ARM_FEATURE_M_SECURITY); + set_feature(&cpu->env, ARM_FEATURE_THUMB_DSP); + cpu->midr = 0x410fd213; /* r0p3 */ + cpu->pmsav7_dregion = 16; + cpu->sau_sregion = 8; + cpu->isar.mvfr0 = 0x10110021; + cpu->isar.mvfr1 = 0x11000011; + cpu->isar.mvfr2 = 0x00000040; + cpu->isar.id_pfr0 = 0x00000030; + cpu->isar.id_pfr1 = 0x00000210; + cpu->isar.id_dfr0 = 0x00200000; + cpu->id_afr0 = 0x00000000; + cpu->isar.id_mmfr0 = 0x00101F40; + cpu->isar.id_mmfr1 = 0x00000000; + cpu->isar.id_mmfr2 = 0x01000000; + cpu->isar.id_mmfr3 = 0x00000000; + cpu->isar.id_isar0 = 0x01101110; + cpu->isar.id_isar1 = 0x02212000; + cpu->isar.id_isar2 = 0x20232232; + cpu->isar.id_isar3 = 0x01111131; + cpu->isar.id_isar4 = 0x01310132; + cpu->isar.id_isar5 = 0x00000000; + cpu->isar.id_isar6 = 0x00000000; + cpu->clidr = 0x00000000; + cpu->ctr = 0x8000c000; +} + +static void cortex_m55_initfn(Object *obj) +{ + ARMCPU *cpu = ARM_CPU(obj); + + set_feature(&cpu->env, ARM_FEATURE_V8); + set_feature(&cpu->env, ARM_FEATURE_V8_1M); + set_feature(&cpu->env, ARM_FEATURE_M); + set_feature(&cpu->env, ARM_FEATURE_M_MAIN); + set_feature(&cpu->env, ARM_FEATURE_M_SECURITY); + set_feature(&cpu->env, ARM_FEATURE_THUMB_DSP); + cpu->midr = 0x410fd221; /* r0p1 */ + cpu->revidr = 0; + cpu->pmsav7_dregion = 16; + cpu->sau_sregion = 8; + /* These are the MVFR* values for the FPU + full MVE configuration */ + cpu->isar.mvfr0 = 0x10110221; + cpu->isar.mvfr1 = 0x12100211; + cpu->isar.mvfr2 = 0x00000040; + cpu->isar.id_pfr0 = 0x20000030; + cpu->isar.id_pfr1 = 0x00000230; + cpu->isar.id_dfr0 = 0x10200000; + cpu->id_afr0 = 0x00000000; + cpu->isar.id_mmfr0 = 0x00111040; + cpu->isar.id_mmfr1 = 0x00000000; + cpu->isar.id_mmfr2 = 0x01000000; + cpu->isar.id_mmfr3 = 0x00000011; + cpu->isar.id_isar0 = 0x01103110; + cpu->isar.id_isar1 = 0x02212000; + cpu->isar.id_isar2 = 0x20232232; + cpu->isar.id_isar3 = 0x01111131; + cpu->isar.id_isar4 = 0x01310132; + cpu->isar.id_isar5 = 0x00000000; + cpu->isar.id_isar6 = 0x00000000; + cpu->clidr = 0x00000000; /* caches not implemented */ + cpu->ctr = 0x8303c003; +} + +static const TCGCPUOps arm_v7m_tcg_ops = { + .initialize = arm_translate_init, + .synchronize_from_tb = arm_cpu_synchronize_from_tb, + .debug_excp_handler = arm_debug_excp_handler, + .restore_state_to_opc = arm_restore_state_to_opc, + +#ifdef CONFIG_USER_ONLY + .record_sigsegv = arm_cpu_record_sigsegv, + .record_sigbus = arm_cpu_record_sigbus, +#else + .tlb_fill = arm_cpu_tlb_fill, + .cpu_exec_interrupt = arm_v7m_cpu_exec_interrupt, + .do_interrupt = arm_v7m_cpu_do_interrupt, + .do_transaction_failed = arm_cpu_do_transaction_failed, + .do_unaligned_access = arm_cpu_do_unaligned_access, + .adjust_watchpoint_address = arm_adjust_watchpoint_address, + .debug_check_watchpoint = arm_debug_check_watchpoint, + .debug_check_breakpoint = arm_debug_check_breakpoint, +#endif /* !CONFIG_USER_ONLY */ +}; + +static void arm_v7m_class_init(ObjectClass *oc, void *data) +{ + ARMCPUClass *acc = ARM_CPU_CLASS(oc); + CPUClass *cc = CPU_CLASS(oc); + + acc->info = data; + cc->tcg_ops = &arm_v7m_tcg_ops; + cc->gdb_core_xml_file = "arm-m-profile.xml"; +} + +static const ARMCPUInfo arm_v7m_cpus[] = { + { .name = "cortex-m0", .initfn = cortex_m0_initfn, + .class_init = arm_v7m_class_init }, + { .name = "cortex-m3", .initfn = cortex_m3_initfn, + .class_init = arm_v7m_class_init }, + { .name = "cortex-m4", .initfn = cortex_m4_initfn, + .class_init = arm_v7m_class_init }, + { .name = "cortex-m7", .initfn = cortex_m7_initfn, + .class_init = arm_v7m_class_init }, + { .name = "cortex-m33", .initfn = cortex_m33_initfn, + .class_init = arm_v7m_class_init }, + { .name = "cortex-m55", .initfn = cortex_m55_initfn, + .class_init = arm_v7m_class_init }, +}; + +static void arm_v7m_cpu_register_types(void) +{ + size_t i; + + for (i = 0; i < ARRAY_SIZE(arm_v7m_cpus); ++i) { + arm_cpu_register(&arm_v7m_cpus[i]); + } +} + +type_init(arm_v7m_cpu_register_types) diff --git a/target/arm/tcg/cpu32.c b/target/arm/tcg/cpu32.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/cpu32.c +++ b/target/arm/tcg/cpu32.c @@ -XXX,XX +XXX,XX @@ #include "hw/boards.h" #endif #include "cpregs.h" -#if !defined(CONFIG_USER_ONLY) && defined(CONFIG_TCG) -#include "hw/intc/armv7m_nvic.h" -#endif /* Share AArch32 -cpu max features with AArch64. */ @@ -XXX,XX +XXX,XX @@ void aa32_max_features(ARMCPU *cpu) /* CPU models. These are not needed for the AArch64 linux-user build. */ #if !defined(CONFIG_USER_ONLY) || !defined(TARGET_AARCH64) -#if !defined(CONFIG_USER_ONLY) -static bool arm_v7m_cpu_exec_interrupt(CPUState *cs, int interrupt_request) -{ - CPUClass *cc = CPU_GET_CLASS(cs); - ARMCPU *cpu = ARM_CPU(cs); - CPUARMState *env = &cpu->env; - bool ret = false; - - /* - * ARMv7-M interrupt masking works differently than -A or -R. - * There is no FIQ/IRQ distinction. Instead of I and F bits - * masking FIQ and IRQ interrupts, an exception is taken only - * if it is higher priority than the current execution priority - * (which depends on state like BASEPRI, FAULTMASK and the - * currently active exception). - */ - if (interrupt_request & CPU_INTERRUPT_HARD - && (armv7m_nvic_can_take_pending_exception(env->nvic))) { - cs->exception_index = EXCP_IRQ; - cc->tcg_ops->do_interrupt(cs); - ret = true; - } - return ret; -} -#endif /* !CONFIG_USER_ONLY */ - static void arm926_initfn(Object *obj) { ARMCPU *cpu = ARM_CPU(obj); @@ -XXX,XX +XXX,XX @@ static void cortex_a15_initfn(Object *obj) define_arm_cp_regs(cpu, cortexa15_cp_reginfo); } -static void cortex_m0_initfn(Object *obj) -{ - ARMCPU *cpu = ARM_CPU(obj); - set_feature(&cpu->env, ARM_FEATURE_V6); - set_feature(&cpu->env, ARM_FEATURE_M); - - cpu->midr = 0x410cc200; - - /* - * These ID register values are not guest visible, because - * we do not implement the Main Extension. They must be set - * to values corresponding to the Cortex-M0's implemented - * features, because QEMU generally controls its emulation - * by looking at ID register fields. We use the same values as - * for the M3. - */ - cpu->isar.id_pfr0 = 0x00000030; - cpu->isar.id_pfr1 = 0x00000200; - cpu->isar.id_dfr0 = 0x00100000; - cpu->id_afr0 = 0x00000000; - cpu->isar.id_mmfr0 = 0x00000030; - cpu->isar.id_mmfr1 = 0x00000000; - cpu->isar.id_mmfr2 = 0x00000000; - cpu->isar.id_mmfr3 = 0x00000000; - cpu->isar.id_isar0 = 0x01141110; - cpu->isar.id_isar1 = 0x02111000; - cpu->isar.id_isar2 = 0x21112231; - cpu->isar.id_isar3 = 0x01111110; - cpu->isar.id_isar4 = 0x01310102; - cpu->isar.id_isar5 = 0x00000000; - cpu->isar.id_isar6 = 0x00000000; -} - -static void cortex_m3_initfn(Object *obj) -{ - ARMCPU *cpu = ARM_CPU(obj); - set_feature(&cpu->env, ARM_FEATURE_V7); - set_feature(&cpu->env, ARM_FEATURE_M); - set_feature(&cpu->env, ARM_FEATURE_M_MAIN); - cpu->midr = 0x410fc231; - cpu->pmsav7_dregion = 8; - cpu->isar.id_pfr0 = 0x00000030; - cpu->isar.id_pfr1 = 0x00000200; - cpu->isar.id_dfr0 = 0x00100000; - cpu->id_afr0 = 0x00000000; - cpu->isar.id_mmfr0 = 0x00000030; - cpu->isar.id_mmfr1 = 0x00000000; - cpu->isar.id_mmfr2 = 0x00000000; - cpu->isar.id_mmfr3 = 0x00000000; - cpu->isar.id_isar0 = 0x01141110; - cpu->isar.id_isar1 = 0x02111000; - cpu->isar.id_isar2 = 0x21112231; - cpu->isar.id_isar3 = 0x01111110; - cpu->isar.id_isar4 = 0x01310102; - cpu->isar.id_isar5 = 0x00000000; - cpu->isar.id_isar6 = 0x00000000; -} - -static void cortex_m4_initfn(Object *obj) -{ - ARMCPU *cpu = ARM_CPU(obj); - - set_feature(&cpu->env, ARM_FEATURE_V7); - set_feature(&cpu->env, ARM_FEATURE_M); - set_feature(&cpu->env, ARM_FEATURE_M_MAIN); - set_feature(&cpu->env, ARM_FEATURE_THUMB_DSP); - cpu->midr = 0x410fc240; /* r0p0 */ - cpu->pmsav7_dregion = 8; - cpu->isar.mvfr0 = 0x10110021; - cpu->isar.mvfr1 = 0x11000011; - cpu->isar.mvfr2 = 0x00000000; - cpu->isar.id_pfr0 = 0x00000030; - cpu->isar.id_pfr1 = 0x00000200; - cpu->isar.id_dfr0 = 0x00100000; - cpu->id_afr0 = 0x00000000; - cpu->isar.id_mmfr0 = 0x00000030; - cpu->isar.id_mmfr1 = 0x00000000; - cpu->isar.id_mmfr2 = 0x00000000; - cpu->isar.id_mmfr3 = 0x00000000; - cpu->isar.id_isar0 = 0x01141110; - cpu->isar.id_isar1 = 0x02111000; - cpu->isar.id_isar2 = 0x21112231; - cpu->isar.id_isar3 = 0x01111110; - cpu->isar.id_isar4 = 0x01310102; - cpu->isar.id_isar5 = 0x00000000; - cpu->isar.id_isar6 = 0x00000000; -} - -static void cortex_m7_initfn(Object *obj) -{ - ARMCPU *cpu = ARM_CPU(obj); - - set_feature(&cpu->env, ARM_FEATURE_V7); - set_feature(&cpu->env, ARM_FEATURE_M); - set_feature(&cpu->env, ARM_FEATURE_M_MAIN); - set_feature(&cpu->env, ARM_FEATURE_THUMB_DSP); - cpu->midr = 0x411fc272; /* r1p2 */ - cpu->pmsav7_dregion = 8; - cpu->isar.mvfr0 = 0x10110221; - cpu->isar.mvfr1 = 0x12000011; - cpu->isar.mvfr2 = 0x00000040; - cpu->isar.id_pfr0 = 0x00000030; - cpu->isar.id_pfr1 = 0x00000200; - cpu->isar.id_dfr0 = 0x00100000; - cpu->id_afr0 = 0x00000000; - cpu->isar.id_mmfr0 = 0x00100030; - cpu->isar.id_mmfr1 = 0x00000000; - cpu->isar.id_mmfr2 = 0x01000000; - cpu->isar.id_mmfr3 = 0x00000000; - cpu->isar.id_isar0 = 0x01101110; - cpu->isar.id_isar1 = 0x02112000; - cpu->isar.id_isar2 = 0x20232231; - cpu->isar.id_isar3 = 0x01111131; - cpu->isar.id_isar4 = 0x01310132; - cpu->isar.id_isar5 = 0x00000000; - cpu->isar.id_isar6 = 0x00000000; -} - -static void cortex_m33_initfn(Object *obj) -{ - ARMCPU *cpu = ARM_CPU(obj); - - set_feature(&cpu->env, ARM_FEATURE_V8); - set_feature(&cpu->env, ARM_FEATURE_M); - set_feature(&cpu->env, ARM_FEATURE_M_MAIN); - set_feature(&cpu->env, ARM_FEATURE_M_SECURITY); - set_feature(&cpu->env, ARM_FEATURE_THUMB_DSP); - cpu->midr = 0x410fd213; /* r0p3 */ - cpu->pmsav7_dregion = 16; - cpu->sau_sregion = 8; - cpu->isar.mvfr0 = 0x10110021; - cpu->isar.mvfr1 = 0x11000011; - cpu->isar.mvfr2 = 0x00000040; - cpu->isar.id_pfr0 = 0x00000030; - cpu->isar.id_pfr1 = 0x00000210; - cpu->isar.id_dfr0 = 0x00200000; - cpu->id_afr0 = 0x00000000; - cpu->isar.id_mmfr0 = 0x00101F40; - cpu->isar.id_mmfr1 = 0x00000000; - cpu->isar.id_mmfr2 = 0x01000000; - cpu->isar.id_mmfr3 = 0x00000000; - cpu->isar.id_isar0 = 0x01101110; - cpu->isar.id_isar1 = 0x02212000; - cpu->isar.id_isar2 = 0x20232232; - cpu->isar.id_isar3 = 0x01111131; - cpu->isar.id_isar4 = 0x01310132; - cpu->isar.id_isar5 = 0x00000000; - cpu->isar.id_isar6 = 0x00000000; - cpu->clidr = 0x00000000; - cpu->ctr = 0x8000c000; -} - -static void cortex_m55_initfn(Object *obj) -{ - ARMCPU *cpu = ARM_CPU(obj); - - set_feature(&cpu->env, ARM_FEATURE_V8); - set_feature(&cpu->env, ARM_FEATURE_V8_1M); - set_feature(&cpu->env, ARM_FEATURE_M); - set_feature(&cpu->env, ARM_FEATURE_M_MAIN); - set_feature(&cpu->env, ARM_FEATURE_M_SECURITY); - set_feature(&cpu->env, ARM_FEATURE_THUMB_DSP); - cpu->midr = 0x410fd221; /* r0p1 */ - cpu->revidr = 0; - cpu->pmsav7_dregion = 16; - cpu->sau_sregion = 8; - /* These are the MVFR* values for the FPU + full MVE configuration */ - cpu->isar.mvfr0 = 0x10110221; - cpu->isar.mvfr1 = 0x12100211; - cpu->isar.mvfr2 = 0x00000040; - cpu->isar.id_pfr0 = 0x20000030; - cpu->isar.id_pfr1 = 0x00000230; - cpu->isar.id_dfr0 = 0x10200000; - cpu->id_afr0 = 0x00000000; - cpu->isar.id_mmfr0 = 0x00111040; - cpu->isar.id_mmfr1 = 0x00000000; - cpu->isar.id_mmfr2 = 0x01000000; - cpu->isar.id_mmfr3 = 0x00000011; - cpu->isar.id_isar0 = 0x01103110; - cpu->isar.id_isar1 = 0x02212000; - cpu->isar.id_isar2 = 0x20232232; - cpu->isar.id_isar3 = 0x01111131; - cpu->isar.id_isar4 = 0x01310132; - cpu->isar.id_isar5 = 0x00000000; - cpu->isar.id_isar6 = 0x00000000; - cpu->clidr = 0x00000000; /* caches not implemented */ - cpu->ctr = 0x8303c003; -} - static const ARMCPRegInfo cortexr5_cp_reginfo[] = { /* Dummy the TCM region regs for the moment */ { .name = "ATCM", .cp = 15, .opc1 = 0, .crn = 9, .crm = 1, .opc2 = 0, @@ -XXX,XX +XXX,XX @@ static void pxa270c5_initfn(Object *obj) cpu->reset_sctlr = 0x00000078; } -static const TCGCPUOps arm_v7m_tcg_ops = { - .initialize = arm_translate_init, - .synchronize_from_tb = arm_cpu_synchronize_from_tb, - .debug_excp_handler = arm_debug_excp_handler, - .restore_state_to_opc = arm_restore_state_to_opc, - -#ifdef CONFIG_USER_ONLY - .record_sigsegv = arm_cpu_record_sigsegv, - .record_sigbus = arm_cpu_record_sigbus, -#else - .tlb_fill = arm_cpu_tlb_fill, - .cpu_exec_interrupt = arm_v7m_cpu_exec_interrupt, - .do_interrupt = arm_v7m_cpu_do_interrupt, - .do_transaction_failed = arm_cpu_do_transaction_failed, - .do_unaligned_access = arm_cpu_do_unaligned_access, - .adjust_watchpoint_address = arm_adjust_watchpoint_address, - .debug_check_watchpoint = arm_debug_check_watchpoint, - .debug_check_breakpoint = arm_debug_check_breakpoint, -#endif /* !CONFIG_USER_ONLY */ -}; - -static void arm_v7m_class_init(ObjectClass *oc, void *data) -{ - ARMCPUClass *acc = ARM_CPU_CLASS(oc); - CPUClass *cc = CPU_CLASS(oc); - - acc->info = data; - cc->tcg_ops = &arm_v7m_tcg_ops; - cc->gdb_core_xml_file = "arm-m-profile.xml"; -} - #ifndef TARGET_AARCH64 /* * -cpu max: a CPU with as many features enabled as our emulation supports. @@ -XXX,XX +XXX,XX @@ static const ARMCPUInfo arm_tcg_cpus[] = { { .name = "cortex-a8", .initfn = cortex_a8_initfn }, { .name = "cortex-a9", .initfn = cortex_a9_initfn }, { .name = "cortex-a15", .initfn = cortex_a15_initfn }, - { .name = "cortex-m0", .initfn = cortex_m0_initfn, - .class_init = arm_v7m_class_init }, - { .name = "cortex-m3", .initfn = cortex_m3_initfn, - .class_init = arm_v7m_class_init }, - { .name = "cortex-m4", .initfn = cortex_m4_initfn, - .class_init = arm_v7m_class_init }, - { .name = "cortex-m7", .initfn = cortex_m7_initfn, - .class_init = arm_v7m_class_init }, - { .name = "cortex-m33", .initfn = cortex_m33_initfn, - .class_init = arm_v7m_class_init }, - { .name = "cortex-m55", .initfn = cortex_m55_initfn, - .class_init = arm_v7m_class_init }, { .name = "cortex-r5", .initfn = cortex_r5_initfn }, { .name = "cortex-r5f", .initfn = cortex_r5f_initfn }, { .name = "cortex-r52", .initfn = cortex_r52_initfn }, diff --git a/target/arm/meson.build b/target/arm/meson.build index XXXXXXX..XXXXXXX 100644 --- a/target/arm/meson.build +++ b/target/arm/meson.build @@ -XXX,XX +XXX,XX @@ arm_system_ss.add(files( 'ptw.c', )) +arm_user_ss = ss.source_set() + subdir('hvf') if 'CONFIG_TCG' in config_all_accel @@ -XXX,XX +XXX,XX @@ endif target_arch += {'arm': arm_ss} target_system_arch += {'arm': arm_system_ss} +target_user_arch += {'arm': arm_user_ss} diff --git a/target/arm/tcg/meson.build b/target/arm/tcg/meson.build index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/meson.build +++ b/target/arm/tcg/meson.build @@ -XXX,XX +XXX,XX @@ arm_ss.add(when: 'TARGET_AARCH64', if_true: files( arm_system_ss.add(files( 'psci.c', )) + +arm_system_ss.add(when: 'CONFIG_ARM_V7M', if_true: files('cpu-v7m.c')) +arm_user_ss.add(when: 'TARGET_AARCH64', if_false: files('cpu-v7m.c')) -- 2.34.1