First target-arm queue for 9.2. I know I have more stuff in my to-review queue after this... -- PMM The following changes since commit cab1afb393ea0943b3086188e91d71d594ede6bf: Merge tag 'hppa-v9.1-fixes-pull-request' of https://github.com/hdeller/qemu-hppa into staging (2024-09-04 13:20:17 +0100) are available in the Git repository at: https://git.linaro.org/people/pmaydell/qemu-arm.git tags/pull-target-arm-20240905 for you to fetch changes up to 99ec7b440a1d6a6ef07450b68687d24d13a25fb5: platform-bus: fix refcount leak (2024-09-05 13:12:37 +0100) ---------------------------------------------------------------- target-arm queue: * Implement FEAT_EBF16 emulation * accel/tcg: Remove dead code from rr_cpu_thread_fn() * hw: add compat machines for 9.2 * virt: default to two-stage SMMU from virt-9.2 * sbsa-ref: use two-stage SMMU * hw: Various minor memory leak fixes * target/arm: Correct names of VFP VFNMA and VFNMS insns * hw/arm/xilinx_zynq: Enable Security Extensions * hw/arm/boot: Report error msg if loading elf/dtb failed ---------------------------------------------------------------- Changbin Du (1): hw/arm/boot: Report error msg if loading elf/dtb failed Cornelia Huck (1): hw: add compat machines for 9.2 Gao Shiyuan (1): platform-bus: fix refcount leak Peter Maydell (20): target/arm: Allow setting the FPCR.EBF bit for FEAT_EBF16 target/arm: Pass env pointer through to sme_bfmopa helper target/arm: Pass env pointer through to gvec_bfdot helper target/arm: Pass env pointer through to gvec_bfdot_idx helper target/arm: Pass env pointer through to gvec_bfmmla helper target/arm: Prepare bfdotadd() callers for FEAT_EBF support target/arm: Implement FPCR.EBF=1 semantics for bfdotadd() target/arm: Enable FEAT_EBF16 in the "max" CPU accel/tcg: Remove dead code from rr_cpu_thread_fn() hw/arm/smmuv3: Update comment documenting "stage" property hw/arm/virt: Default to two-stage SMMU from virt-9.2 hw/arm/sbsa-ref: Use two-stage SMMU hw/misc/xlnx-versal-cfu: destroy fifo in finalize hw/misc/xlnx-versal-trng: Free s->prng in finalize, not unrealize hw/nvram/xlnx-bbram: Call register_finalize_block hw/nvram/xlnx-zynqmp-efuse: Call register_finalize_block hw/misc/xlnx-versal-trng: Call register_finalize_block hm/nvram/xlnx-versal-efuse-ctrl: Call register_finalize_block hw/arm/sbsa-ref: Don't leak string in sbsa_fdt_add_gic_node() target/arm: Correct names of VFP VFNMA and VFNMS insns Philippe Mathieu-Daudé (1): hw/arm/boot: Explain why load_elf_hdr() error is ignored Sebastian Huber (1): hw/arm/xilinx_zynq: Enable Security Extensions docs/system/arm/emulation.rst | 1 + include/hw/arm/virt.h | 1 + include/hw/boards.h | 3 + include/hw/i386/pc.h | 3 + include/hw/misc/xlnx-versal-trng.h | 1 + include/hw/nvram/xlnx-bbram.h | 1 + include/hw/nvram/xlnx-versal-efuse.h | 1 + include/hw/nvram/xlnx-zynqmp-efuse.h | 1 + target/arm/cpu-features.h | 5 + target/arm/cpu.h | 1 + target/arm/helper.h | 12 +-- target/arm/tcg/helper-sme.h | 4 +- target/arm/tcg/vec_internal.h | 37 ++++++- target/arm/tcg/vfp.decode | 12 +-- accel/tcg/tcg-accel-ops-rr.c | 4 +- hw/arm/boot.c | 10 +- hw/arm/sbsa-ref.c | 16 ++- hw/arm/smmuv3.c | 1 + hw/arm/virt.c | 19 +++- hw/arm/xilinx_zynq.c | 8 -- hw/core/machine.c | 3 + hw/core/platform-bus.c | 5 +- hw/i386/pc.c | 3 + hw/i386/pc_piix.c | 15 ++- hw/i386/pc_q35.c | 13 ++- hw/m68k/virt.c | 11 +- hw/misc/xlnx-versal-cfu.c | 8 ++ hw/misc/xlnx-versal-trng.c | 12 +-- hw/nvram/xlnx-bbram.c | 13 ++- hw/nvram/xlnx-versal-efuse-ctrl.c | 6 +- hw/nvram/xlnx-zynqmp-efuse.c | 13 ++- hw/ppc/spapr.c | 17 ++- hw/s390x/s390-virtio-ccw.c | 14 ++- target/arm/tcg/cpu64.c | 4 +- target/arm/tcg/sme_helper.c | 78 +++++++++----- target/arm/tcg/translate-a64.c | 40 ++++++- target/arm/tcg/translate-neon.c | 43 ++++++-- target/arm/tcg/translate-sme.c | 3 +- target/arm/tcg/translate-sve.c | 25 ++++- target/arm/tcg/translate-vfp.c | 8 +- target/arm/tcg/vec_helper.c | 200 ++++++++++++++++++++++++++++------- target/arm/vfp_helper.c | 8 +- 42 files changed, 529 insertions(+), 154 deletions(-) FEAT_EBF16 adds one new bit to the FPCR floating point control register. Allow this bit to be read and written when the ID registers indicate the presence of the feature. Note that because this new bit is not in FPSCR_FPCR_MASK the bit is not visible in the AArch32 FPSCR, and FPSCR writes do not affect it. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> --- target/arm/cpu-features.h | 5 +++++ target/arm/cpu.h | 1 + target/arm/vfp_helper.c | 8 ++++++-- 3 files changed, 12 insertions(+), 2 deletions(-) diff --git a/target/arm/cpu-features.h b/target/arm/cpu-features.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpu-features.h +++ b/target/arm/cpu-features.h @@ -XXX,XX +XXX,XX @@ static inline bool isar_feature_aa64_bf16(const ARMISARegisters *id) return FIELD_EX64(id->id_aa64isar1, ID_AA64ISAR1, BF16) != 0; } +static inline bool isar_feature_aa64_ebf16(const ARMISARegisters *id) +{ + return FIELD_EX64(id->id_aa64isar1, ID_AA64ISAR1, BF16) > 1; +} + static inline bool isar_feature_aa64_rcpc_8_3(const ARMISARegisters *id) { return FIELD_EX64(id->id_aa64isar1, ID_AA64ISAR1, LRCPC) != 0; diff --git a/target/arm/cpu.h b/target/arm/cpu.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -XXX,XX +XXX,XX @@ void vfp_set_fpscr(CPUARMState *env, uint32_t val); #define FPCR_OFE (1 << 10) /* Overflow exception trap enable */ #define FPCR_UFE (1 << 11) /* Underflow exception trap enable */ #define FPCR_IXE (1 << 12) /* Inexact exception trap enable */ +#define FPCR_EBF (1 << 13) /* Extended BFloat16 behaviors */ #define FPCR_IDE (1 << 15) /* Input Denormal exception trap enable */ #define FPCR_LEN_MASK (7 << 16) /* LEN, A-profile only */ #define FPCR_FZ16 (1 << 19) /* ARMv8.2+, FP16 flush-to-zero */ diff --git a/target/arm/vfp_helper.c b/target/arm/vfp_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vfp_helper.c +++ b/target/arm/vfp_helper.c @@ -XXX,XX +XXX,XX @@ static void vfp_set_fpcr_masked(CPUARMState *env, uint32_t val, uint32_t mask) val &= ~FPCR_FZ16; } + if (!cpu_isar_feature(aa64_ebf16, cpu)) { + val &= ~FPCR_EBF; + } + vfp_set_fpcr_to_host(env, val, mask); if (mask & (FPCR_LEN_MASK | FPCR_STRIDE_MASK)) { @@ -XXX,XX +XXX,XX @@ static void vfp_set_fpcr_masked(CPUARMState *env, uint32_t val, uint32_t mask) * We don't implement trapped exception handling, so the * trap enable bits, IDE|IXE|UFE|OFE|DZE|IOE are all RAZ/WI (not RES0!) * - * The FPCR bits we keep in vfp.fpcr are AHP, DN, FZ, RMode + * The FPCR bits we keep in vfp.fpcr are AHP, DN, FZ, RMode, EBF * and FZ16. Len, Stride and LTPSIZE we just handled. Store those bits * there, and zero any of the other FPCR bits and the RES0 and RAZ/WI * bits. */ - val &= FPCR_AHP | FPCR_DN | FPCR_FZ | FPCR_RMODE_MASK | FPCR_FZ16; + val &= FPCR_AHP | FPCR_DN | FPCR_FZ | FPCR_RMODE_MASK | FPCR_FZ16 | FPCR_EBF; env->vfp.fpcr &= ~mask; env->vfp.fpcr |= val; } -- 2.34.1 To implement the FEAT_EBF16 semantics, we are going to need the CPUARMState env pointer in every helper function which calls bfdotadd(). Pass the env pointer through from generated code to the sme_bfmopa helper. (We'll add the code that uses it when we've adjusted all the helpers to have access to the env pointer.) Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> --- target/arm/tcg/helper-sme.h | 4 ++-- target/arm/tcg/sme_helper.c | 4 ++-- target/arm/tcg/translate-sme.c | 2 +- 3 files changed, 5 insertions(+), 5 deletions(-) diff --git a/target/arm/tcg/helper-sme.h b/target/arm/tcg/helper-sme.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/helper-sme.h +++ b/target/arm/tcg/helper-sme.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_7(sme_fmopa_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_7(sme_fmopa_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_6(sme_bfmopa, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_7(sme_bfmopa, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, env, i32) DEF_HELPER_FLAGS_6(sme_smopa_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_6(sme_umopa_s, TCG_CALL_NO_RWG, diff --git a/target/arm/tcg/sme_helper.c b/target/arm/tcg/sme_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/sme_helper.c +++ b/target/arm/tcg/sme_helper.c @@ -XXX,XX +XXX,XX @@ void HELPER(sme_fmopa_h)(void *vza, void *vzn, void *vzm, void *vpn, } } -void HELPER(sme_bfmopa)(void *vza, void *vzn, void *vzm, void *vpn, - void *vpm, uint32_t desc) +void HELPER(sme_bfmopa)(void *vza, void *vzn, void *vzm, + void *vpn, void *vpm, CPUARMState *env, uint32_t desc) { intptr_t row, col, oprsz = simd_maxsz(desc); uint32_t neg = simd_data(desc) * 0x80008000u; diff --git a/target/arm/tcg/translate-sme.c b/target/arm/tcg/translate-sme.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-sme.c +++ b/target/arm/tcg/translate-sme.c @@ -XXX,XX +XXX,XX @@ TRANS_FEAT(FMOPA_d, aa64_sme_f64f64, do_outprod_fpst, a, MO_64, FPST_FPCR, gen_helper_sme_fmopa_d) /* TODO: FEAT_EBF16 */ -TRANS_FEAT(BFMOPA, aa64_sme, do_outprod, a, MO_32, gen_helper_sme_bfmopa) +TRANS_FEAT(BFMOPA, aa64_sme, do_outprod_env, a, MO_32, gen_helper_sme_bfmopa) TRANS_FEAT(SMOPA_s, aa64_sme, do_outprod, a, MO_32, gen_helper_sme_smopa_s) TRANS_FEAT(UMOPA_s, aa64_sme, do_outprod, a, MO_32, gen_helper_sme_umopa_s) -- 2.34.1 Pass the env pointer through to the gvec_bfdot helper, so we can use it to add support for FEAT_EBF16. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> --- target/arm/helper.h | 4 ++-- target/arm/tcg/translate-a64.c | 27 ++++++++++++++++++++++++- target/arm/tcg/translate-neon.c | 35 +++++++++++++++++++++++++++++++-- target/arm/tcg/translate-sve.c | 15 +++++++++++++- target/arm/tcg/vec_helper.c | 3 ++- 5 files changed, 77 insertions(+), 7 deletions(-) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_5(gvec_ummla_b, TCG_CALL_NO_RWG, DEF_HELPER_FLAGS_5(gvec_usmmla_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(gvec_bfdot, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_6(gvec_bfdot, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, env, i32) DEF_HELPER_FLAGS_5(gvec_bfdot_idx, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static void gen_gvec_op4_ool(DisasContext *s, bool is_q, int rd, int rn, is_q ? 16 : 8, vec_full_reg_size(s), data, fn); } +/* + * Expand a 4-operand operation using an out-of-line helper that takes + * a pointer to the CPU env. + */ +static void gen_gvec_op4_env(DisasContext *s, bool is_q, int rd, int rn, + int rm, int ra, int data, + gen_helper_gvec_4_ptr *fn) +{ + tcg_gen_gvec_4_ptr(vec_full_reg_offset(s, rd), + vec_full_reg_offset(s, rn), + vec_full_reg_offset(s, rm), + vec_full_reg_offset(s, ra), + tcg_env, + is_q ? 16 : 8, vec_full_reg_size(s), data, fn); +} + /* * Expand a 4-operand + fpstatus pointer + simd data value operation using * an out-of-line helper. @@ -XXX,XX +XXX,XX @@ static bool do_dot_vector(DisasContext *s, arg_qrrr_e *a, return true; } +static bool do_dot_vector_env(DisasContext *s, arg_qrrr_e *a, + gen_helper_gvec_4_ptr *fn) +{ + if (fp_access_check(s)) { + gen_gvec_op4_env(s, a->q, a->rd, a->rn, a->rm, a->rd, 0, fn); + } + return true; +} + TRANS_FEAT(SDOT_v, aa64_dp, do_dot_vector, a, gen_helper_gvec_sdot_b) TRANS_FEAT(UDOT_v, aa64_dp, do_dot_vector, a, gen_helper_gvec_udot_b) TRANS_FEAT(USDOT_v, aa64_i8mm, do_dot_vector, a, gen_helper_gvec_usdot_b) -TRANS_FEAT(BFDOT_v, aa64_bf16, do_dot_vector, a, gen_helper_gvec_bfdot) +TRANS_FEAT(BFDOT_v, aa64_bf16, do_dot_vector_env, a, gen_helper_gvec_bfdot) TRANS_FEAT(BFMMLA, aa64_bf16, do_dot_vector, a, gen_helper_gvec_bfmmla) TRANS_FEAT(SMMLA, aa64_i8mm, do_dot_vector, a, gen_helper_gvec_smmla_b) TRANS_FEAT(UMMLA, aa64_i8mm, do_dot_vector, a, gen_helper_gvec_ummla_b) diff --git a/target/arm/tcg/translate-neon.c b/target/arm/tcg/translate-neon.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-neon.c +++ b/target/arm/tcg/translate-neon.c @@ -XXX,XX +XXX,XX @@ static bool do_neon_ddda(DisasContext *s, int q, int vd, int vn, int vm, return true; } +static bool do_neon_ddda_env(DisasContext *s, int q, int vd, int vn, int vm, + int data, gen_helper_gvec_4_ptr *fn_gvec) +{ + /* UNDEF accesses to D16-D31 if they don't exist. */ + if (((vd | vn | vm) & 0x10) && !dc_isar_feature(aa32_simd_r32, s)) { + return false; + } + + /* + * UNDEF accesses to odd registers for each bit of Q. + * Q will be 0b111 for all Q-reg instructions, otherwise + * when we have mixed Q- and D-reg inputs. + */ + if (((vd & 1) * 4 | (vn & 1) * 2 | (vm & 1)) & q) { + return false; + } + + if (!vfp_access_check(s)) { + return true; + } + + int opr_sz = q ? 16 : 8; + tcg_gen_gvec_4_ptr(vfp_reg_offset(1, vd), + vfp_reg_offset(1, vn), + vfp_reg_offset(1, vm), + vfp_reg_offset(1, vd), + tcg_env, + opr_sz, opr_sz, data, fn_gvec); + return true; +} + static bool do_neon_ddda_fpst(DisasContext *s, int q, int vd, int vn, int vm, int data, ARMFPStatusFlavour fp_flavour, gen_helper_gvec_4_ptr *fn_gvec_ptr) @@ -XXX,XX +XXX,XX @@ static bool trans_VDOT_b16(DisasContext *s, arg_VDOT_b16 *a) if (!dc_isar_feature(aa32_bf16, s)) { return false; } - return do_neon_ddda(s, a->q * 7, a->vd, a->vn, a->vm, 0, - gen_helper_gvec_bfdot); + return do_neon_ddda_env(s, a->q * 7, a->vd, a->vn, a->vm, 0, + gen_helper_gvec_bfdot); } static bool trans_VFML(DisasContext *s, arg_VFML *a) diff --git a/target/arm/tcg/translate-sve.c b/target/arm/tcg/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-sve.c +++ b/target/arm/tcg/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool gen_gvec_fpst_zzzz(DisasContext *s, gen_helper_gvec_4_ptr *fn, return ret; } +static bool gen_gvec_env_zzzz(DisasContext *s, gen_helper_gvec_4_ptr *fn, + int rd, int rn, int rm, int ra, + int data) +{ + return gen_gvec_ptr_zzzz(s, fn, rd, rn, rm, ra, data, tcg_env); +} + +static bool gen_gvec_env_arg_zzzz(DisasContext *s, gen_helper_gvec_4_ptr *fn, + arg_rrrr_esz *a, int data) +{ + return gen_gvec_env_zzzz(s, fn, a->rd, a->rn, a->rm, a->ra, data); +} + /* Invoke an out-of-line helper on 4 Zregs, 1 Preg, plus fpst. */ static bool gen_gvec_fpst_zzzzp(DisasContext *s, gen_helper_gvec_5_ptr *fn, int rd, int rn, int rm, int ra, int pg, @@ -XXX,XX +XXX,XX @@ TRANS_FEAT_NONSTREAMING(USMMLA, aa64_sve_i8mm, gen_gvec_ool_arg_zzzz, TRANS_FEAT_NONSTREAMING(UMMLA, aa64_sve_i8mm, gen_gvec_ool_arg_zzzz, gen_helper_gvec_ummla_b, a, 0) -TRANS_FEAT(BFDOT_zzzz, aa64_sve_bf16, gen_gvec_ool_arg_zzzz, +TRANS_FEAT(BFDOT_zzzz, aa64_sve_bf16, gen_gvec_env_arg_zzzz, gen_helper_gvec_bfdot, a, 0) TRANS_FEAT(BFDOT_zzxz, aa64_sve_bf16, gen_gvec_ool_arg_zzxz, gen_helper_gvec_bfdot_idx, a) diff --git a/target/arm/tcg/vec_helper.c b/target/arm/tcg/vec_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/vec_helper.c +++ b/target/arm/tcg/vec_helper.c @@ -XXX,XX +XXX,XX @@ float32 bfdotadd(float32 sum, uint32_t e1, uint32_t e2) return t1; } -void HELPER(gvec_bfdot)(void *vd, void *vn, void *vm, void *va, uint32_t desc) +void HELPER(gvec_bfdot)(void *vd, void *vn, void *vm, void *va, + CPUARMState *env, uint32_t desc) { intptr_t i, opr_sz = simd_oprsz(desc); float32 *d = vd, *a = va; -- 2.34.1 Pass the env pointer through to the gvec_bfdot_idx helper, so we can use it to add support for FEAT_EBF16. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> --- target/arm/helper.h | 4 ++-- target/arm/tcg/translate-a64.c | 11 ++++++++++- target/arm/tcg/translate-neon.c | 4 ++-- target/arm/tcg/translate-sve.c | 8 +++++++- target/arm/tcg/vec_helper.c | 2 +- 5 files changed, 22 insertions(+), 7 deletions(-) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_5(gvec_usmmla_b, TCG_CALL_NO_RWG, DEF_HELPER_FLAGS_6(gvec_bfdot, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, env, i32) -DEF_HELPER_FLAGS_5(gvec_bfdot_idx, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_6(gvec_bfdot_idx, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, env, i32) DEF_HELPER_FLAGS_5(gvec_bfmmla, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static bool do_dot_vector_idx(DisasContext *s, arg_qrrx_e *a, return true; } +static bool do_dot_vector_idx_env(DisasContext *s, arg_qrrx_e *a, + gen_helper_gvec_4_ptr *fn) +{ + if (fp_access_check(s)) { + gen_gvec_op4_env(s, a->q, a->rd, a->rn, a->rm, a->rd, a->idx, fn); + } + return true; +} + TRANS_FEAT(SDOT_vi, aa64_dp, do_dot_vector_idx, a, gen_helper_gvec_sdot_idx_b) TRANS_FEAT(UDOT_vi, aa64_dp, do_dot_vector_idx, a, gen_helper_gvec_udot_idx_b) TRANS_FEAT(SUDOT_vi, aa64_i8mm, do_dot_vector_idx, a, gen_helper_gvec_sudot_idx_b) TRANS_FEAT(USDOT_vi, aa64_i8mm, do_dot_vector_idx, a, gen_helper_gvec_usdot_idx_b) -TRANS_FEAT(BFDOT_vi, aa64_bf16, do_dot_vector_idx, a, +TRANS_FEAT(BFDOT_vi, aa64_bf16, do_dot_vector_idx_env, a, gen_helper_gvec_bfdot_idx) static bool trans_BFMLAL_vi(DisasContext *s, arg_qrrx_e *a) diff --git a/target/arm/tcg/translate-neon.c b/target/arm/tcg/translate-neon.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-neon.c +++ b/target/arm/tcg/translate-neon.c @@ -XXX,XX +XXX,XX @@ static bool trans_VDOT_b16_scal(DisasContext *s, arg_VDOT_b16_scal *a) if (!dc_isar_feature(aa32_bf16, s)) { return false; } - return do_neon_ddda(s, a->q * 6, a->vd, a->vn, a->vm, a->index, - gen_helper_gvec_bfdot_idx); + return do_neon_ddda_env(s, a->q * 6, a->vd, a->vn, a->vm, a->index, + gen_helper_gvec_bfdot_idx); } static bool trans_VFML_scalar(DisasContext *s, arg_VFML_scalar *a) diff --git a/target/arm/tcg/translate-sve.c b/target/arm/tcg/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-sve.c +++ b/target/arm/tcg/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool gen_gvec_env_arg_zzzz(DisasContext *s, gen_helper_gvec_4_ptr *fn, return gen_gvec_env_zzzz(s, fn, a->rd, a->rn, a->rm, a->ra, data); } +static bool gen_gvec_env_arg_zzxz(DisasContext *s, gen_helper_gvec_4_ptr *fn, + arg_rrxr_esz *a) +{ + return gen_gvec_env_zzzz(s, fn, a->rd, a->rn, a->rm, a->ra, a->index); +} + /* Invoke an out-of-line helper on 4 Zregs, 1 Preg, plus fpst. */ static bool gen_gvec_fpst_zzzzp(DisasContext *s, gen_helper_gvec_5_ptr *fn, int rd, int rn, int rm, int ra, int pg, @@ -XXX,XX +XXX,XX @@ TRANS_FEAT_NONSTREAMING(UMMLA, aa64_sve_i8mm, gen_gvec_ool_arg_zzzz, TRANS_FEAT(BFDOT_zzzz, aa64_sve_bf16, gen_gvec_env_arg_zzzz, gen_helper_gvec_bfdot, a, 0) -TRANS_FEAT(BFDOT_zzxz, aa64_sve_bf16, gen_gvec_ool_arg_zzxz, +TRANS_FEAT(BFDOT_zzxz, aa64_sve_bf16, gen_gvec_env_arg_zzxz, gen_helper_gvec_bfdot_idx, a) TRANS_FEAT_NONSTREAMING(BFMMLA, aa64_sve_bf16, gen_gvec_ool_arg_zzzz, diff --git a/target/arm/tcg/vec_helper.c b/target/arm/tcg/vec_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/vec_helper.c +++ b/target/arm/tcg/vec_helper.c @@ -XXX,XX +XXX,XX @@ void HELPER(gvec_bfdot)(void *vd, void *vn, void *vm, void *va, } void HELPER(gvec_bfdot_idx)(void *vd, void *vn, void *vm, - void *va, uint32_t desc) + void *va, CPUARMState *env, uint32_t desc) { intptr_t i, j, opr_sz = simd_oprsz(desc); intptr_t index = simd_data(desc); -- 2.34.1 Pass the env pointer through to the gvec_bfmmla helper, so we can use it to add support for FEAT_EBF16. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> --- target/arm/helper.h | 4 ++-- target/arm/tcg/translate-a64.c | 2 +- target/arm/tcg/translate-neon.c | 4 ++-- target/arm/tcg/translate-sve.c | 2 +- target/arm/tcg/vec_helper.c | 3 ++- 5 files changed, 8 insertions(+), 7 deletions(-) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_6(gvec_bfdot, TCG_CALL_NO_RWG, DEF_HELPER_FLAGS_6(gvec_bfdot_idx, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, env, i32) -DEF_HELPER_FLAGS_5(gvec_bfmmla, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_6(gvec_bfmmla, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, env, i32) DEF_HELPER_FLAGS_6(gvec_bfmlal, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ TRANS_FEAT(SDOT_v, aa64_dp, do_dot_vector, a, gen_helper_gvec_sdot_b) TRANS_FEAT(UDOT_v, aa64_dp, do_dot_vector, a, gen_helper_gvec_udot_b) TRANS_FEAT(USDOT_v, aa64_i8mm, do_dot_vector, a, gen_helper_gvec_usdot_b) TRANS_FEAT(BFDOT_v, aa64_bf16, do_dot_vector_env, a, gen_helper_gvec_bfdot) -TRANS_FEAT(BFMMLA, aa64_bf16, do_dot_vector, a, gen_helper_gvec_bfmmla) +TRANS_FEAT(BFMMLA, aa64_bf16, do_dot_vector_env, a, gen_helper_gvec_bfmmla) TRANS_FEAT(SMMLA, aa64_i8mm, do_dot_vector, a, gen_helper_gvec_smmla_b) TRANS_FEAT(UMMLA, aa64_i8mm, do_dot_vector, a, gen_helper_gvec_ummla_b) TRANS_FEAT(USMMLA, aa64_i8mm, do_dot_vector, a, gen_helper_gvec_usmmla_b) diff --git a/target/arm/tcg/translate-neon.c b/target/arm/tcg/translate-neon.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-neon.c +++ b/target/arm/tcg/translate-neon.c @@ -XXX,XX +XXX,XX @@ static bool trans_VMMLA_b16(DisasContext *s, arg_VMMLA_b16 *a) if (!dc_isar_feature(aa32_bf16, s)) { return false; } - return do_neon_ddda(s, 7, a->vd, a->vn, a->vm, 0, - gen_helper_gvec_bfmmla); + return do_neon_ddda_env(s, 7, a->vd, a->vn, a->vm, 0, + gen_helper_gvec_bfmmla); } static bool trans_VFMA_b16(DisasContext *s, arg_VFMA_b16 *a) diff --git a/target/arm/tcg/translate-sve.c b/target/arm/tcg/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-sve.c +++ b/target/arm/tcg/translate-sve.c @@ -XXX,XX +XXX,XX @@ TRANS_FEAT(BFDOT_zzzz, aa64_sve_bf16, gen_gvec_env_arg_zzzz, TRANS_FEAT(BFDOT_zzxz, aa64_sve_bf16, gen_gvec_env_arg_zzxz, gen_helper_gvec_bfdot_idx, a) -TRANS_FEAT_NONSTREAMING(BFMMLA, aa64_sve_bf16, gen_gvec_ool_arg_zzzz, +TRANS_FEAT_NONSTREAMING(BFMMLA, aa64_sve_bf16, gen_gvec_env_arg_zzzz, gen_helper_gvec_bfmmla, a, 0) static bool do_BFMLAL_zzzw(DisasContext *s, arg_rrrr_esz *a, bool sel) diff --git a/target/arm/tcg/vec_helper.c b/target/arm/tcg/vec_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/vec_helper.c +++ b/target/arm/tcg/vec_helper.c @@ -XXX,XX +XXX,XX @@ void HELPER(gvec_bfdot_idx)(void *vd, void *vn, void *vm, clear_tail(d, opr_sz, simd_maxsz(desc)); } -void HELPER(gvec_bfmmla)(void *vd, void *vn, void *vm, void *va, uint32_t desc) +void HELPER(gvec_bfmmla)(void *vd, void *vn, void *vm, void *va, + CPUARMState *env, uint32_t desc) { intptr_t s, opr_sz = simd_oprsz(desc); float32 *d = vd, *a = va; -- 2.34.1 We use bfdotadd() in four callsites for various helper functions. Currently this all assumes that we have the FPCR.EBF=0 semantics. For FPCR.EBF=1 we will need to: * call a different routine to bfdotadd() because we need to do a fused multiply-add rather than separate multiply and add steps * use a different float_status that honours the FPCR rounding mode and denormal-flushing fields * pass in an extra float_status that has been set up to perform round-to-odd rounding To prepare for this, refactor all the callsites so that instead of for (...) { x = bfdotadd(...); } they are: float_status fpst, fpst_odd; if (is_ebf(env, &fpst, &fpst_odd)) { for (...) { x = bfdotadd_ebf(..., &fpst, &fpst_odd); } } else { for (...) { x = bfdotadd(..., &fpst); } } For the moment the is_ebf() function always returns false, sets up fpst for EBF=0 semantics and never sets up fpst_odd; bfdotadd_ebf() will assert if called. We'll fill in the handling for EBF=1 in the next commit. This change should be a zero-behaviour-change refactor. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> --- target/arm/tcg/vec_internal.h | 37 ++++++++- target/arm/tcg/sme_helper.c | 74 ++++++++++++------ target/arm/tcg/vec_helper.c | 139 +++++++++++++++++++++++++--------- 3 files changed, 189 insertions(+), 61 deletions(-) diff --git a/target/arm/tcg/vec_internal.h b/target/arm/tcg/vec_internal.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/vec_internal.h +++ b/target/arm/tcg/vec_internal.h @@ -XXX,XX +XXX,XX @@ int64_t do_sqrdmlah_d(int64_t, int64_t, int64_t, bool, bool); * bfdotadd: * @sum: addend * @e1, @e2: multiplicand vectors + * @fpst: floating-point status to use * * BFloat16 2-way dot product of @e1 & @e2, accumulating with @sum. * The @e1 and @e2 operands correspond to the 32-bit source vector * slots and contain two Bfloat16 values each. * - * Corresponds to the ARM pseudocode function BFDotAdd. + * Corresponds to the ARM pseudocode function BFDotAdd, specialized + * for the FPCR.EBF == 0 case. */ -float32 bfdotadd(float32 sum, uint32_t e1, uint32_t e2); +float32 bfdotadd(float32 sum, uint32_t e1, uint32_t e2, float_status *fpst); +/** + * bfdotadd_ebf: + * @sum: addend + * @e1, @e2: multiplicand vectors + * @fpst: floating-point status to use + * @fpst_odd: floating-point status to use for round-to-odd operations + * + * BFloat16 2-way dot product of @e1 & @e2, accumulating with @sum. + * The @e1 and @e2 operands correspond to the 32-bit source vector + * slots and contain two Bfloat16 values each. + * + * Corresponds to the ARM pseudocode function BFDotAdd, specialized + * for the FPCR.EBF == 1 case. + */ +float32 bfdotadd_ebf(float32 sum, uint32_t e1, uint32_t e2, + float_status *fpst, float_status *fpst_odd); + +/** + * is_ebf: + * @env: CPU state + * @statusp: pointer to floating point status to fill in + * @oddstatusp: pointer to floating point status to fill in for round-to-odd + * + * Determine whether a BFDotAdd operation should use FPCR.EBF = 0 + * or FPCR.EBF = 1 semantics. On return, has initialized *statusp + * and *oddstatusp to suitable float_status arguments to use with either + * bfdotadd() or bfdotadd_ebf(). + * Returns true for EBF = 1, false for EBF = 0. (The caller should use this + * to decide whether to call bfdotadd() or bfdotadd_ebf().) + */ +bool is_ebf(CPUARMState *env, float_status *statusp, float_status *oddstatusp); #endif /* TARGET_ARM_VEC_INTERNAL_H */ diff --git a/target/arm/tcg/sme_helper.c b/target/arm/tcg/sme_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/sme_helper.c +++ b/target/arm/tcg/sme_helper.c @@ -XXX,XX +XXX,XX @@ void HELPER(sme_bfmopa)(void *vza, void *vzn, void *vzm, intptr_t row, col, oprsz = simd_maxsz(desc); uint32_t neg = simd_data(desc) * 0x80008000u; uint16_t *pn = vpn, *pm = vpm; + float_status fpst, fpst_odd; - for (row = 0; row < oprsz; ) { - uint16_t prow = pn[H2(row >> 4)]; - do { - void *vza_row = vza + tile_vslice_offset(row); - uint32_t n = *(uint32_t *)(vzn + H1_4(row)); + if (is_ebf(env, &fpst, &fpst_odd)) { + for (row = 0; row < oprsz; ) { + uint16_t prow = pn[H2(row >> 4)]; + do { + void *vza_row = vza + tile_vslice_offset(row); + uint32_t n = *(uint32_t *)(vzn + H1_4(row)); - n = f16mop_adj_pair(n, prow, neg); + n = f16mop_adj_pair(n, prow, neg); - for (col = 0; col < oprsz; ) { - uint16_t pcol = pm[H2(col >> 4)]; - do { - if (prow & pcol & 0b0101) { - uint32_t *a = vza_row + H1_4(col); - uint32_t m = *(uint32_t *)(vzm + H1_4(col)); + for (col = 0; col < oprsz; ) { + uint16_t pcol = pm[H2(col >> 4)]; + do { + if (prow & pcol & 0b0101) { + uint32_t *a = vza_row + H1_4(col); + uint32_t m = *(uint32_t *)(vzm + H1_4(col)); - m = f16mop_adj_pair(m, pcol, 0); - *a = bfdotadd(*a, n, m); - } - col += 4; - pcol >>= 4; - } while (col & 15); - } - row += 4; - prow >>= 4; - } while (row & 15); + m = f16mop_adj_pair(m, pcol, 0); + *a = bfdotadd_ebf(*a, n, m, &fpst, &fpst_odd); + } + col += 4; + pcol >>= 4; + } while (col & 15); + } + row += 4; + prow >>= 4; + } while (row & 15); + } + } else { + for (row = 0; row < oprsz; ) { + uint16_t prow = pn[H2(row >> 4)]; + do { + void *vza_row = vza + tile_vslice_offset(row); + uint32_t n = *(uint32_t *)(vzn + H1_4(row)); + + n = f16mop_adj_pair(n, prow, neg); + + for (col = 0; col < oprsz; ) { + uint16_t pcol = pm[H2(col >> 4)]; + do { + if (prow & pcol & 0b0101) { + uint32_t *a = vza_row + H1_4(col); + uint32_t m = *(uint32_t *)(vzm + H1_4(col)); + + m = f16mop_adj_pair(m, pcol, 0); + *a = bfdotadd(*a, n, m, &fpst); + } + col += 4; + pcol >>= 4; + } while (col & 15); + } + row += 4; + prow >>= 4; + } while (row & 15); + } } } diff --git a/target/arm/tcg/vec_helper.c b/target/arm/tcg/vec_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/vec_helper.c +++ b/target/arm/tcg/vec_helper.c @@ -XXX,XX +XXX,XX @@ DO_MMLA_B(gvec_usmmla_b, do_usmmla_b) * BFloat16 Dot Product */ -float32 bfdotadd(float32 sum, uint32_t e1, uint32_t e2) +bool is_ebf(CPUARMState *env, float_status *statusp, float_status *oddstatusp) { /* FPCR is ignored for BFDOT and BFMMLA. */ - float_status bf_status = { + *statusp = (float_status){ .tininess_before_rounding = float_tininess_before_rounding, .float_rounding_mode = float_round_to_odd_inf, .flush_to_zero = true, .flush_inputs_to_zero = true, .default_nan_mode = true, }; + + return false; +} + +float32 bfdotadd(float32 sum, uint32_t e1, uint32_t e2, float_status *fpst) +{ float32 t1, t2; /* * Extract each BFloat16 from the element pair, and shift * them such that they become float32. */ - t1 = float32_mul(e1 << 16, e2 << 16, &bf_status); - t2 = float32_mul(e1 & 0xffff0000u, e2 & 0xffff0000u, &bf_status); - t1 = float32_add(t1, t2, &bf_status); - t1 = float32_add(sum, t1, &bf_status); + t1 = float32_mul(e1 << 16, e2 << 16, fpst); + t2 = float32_mul(e1 & 0xffff0000u, e2 & 0xffff0000u, fpst); + t1 = float32_add(t1, t2, fpst); + t1 = float32_add(sum, t1, fpst); return t1; } +float32 bfdotadd_ebf(float32 sum, uint32_t e1, uint32_t e2, + float_status *fpst, float_status *fpst_odd) +{ + g_assert_not_reached(); +} + void HELPER(gvec_bfdot)(void *vd, void *vn, void *vm, void *va, CPUARMState *env, uint32_t desc) { intptr_t i, opr_sz = simd_oprsz(desc); float32 *d = vd, *a = va; uint32_t *n = vn, *m = vm; + float_status fpst, fpst_odd; - for (i = 0; i < opr_sz / 4; ++i) { - d[i] = bfdotadd(a[i], n[i], m[i]); + if (is_ebf(env, &fpst, &fpst_odd)) { + for (i = 0; i < opr_sz / 4; ++i) { + d[i] = bfdotadd_ebf(a[i], n[i], m[i], &fpst, &fpst_odd); + } + } else { + for (i = 0; i < opr_sz / 4; ++i) { + d[i] = bfdotadd(a[i], n[i], m[i], &fpst); + } } clear_tail(d, opr_sz, simd_maxsz(desc)); } @@ -XXX,XX +XXX,XX @@ void HELPER(gvec_bfdot_idx)(void *vd, void *vn, void *vm, intptr_t eltspersegment = MIN(16 / 4, elements); float32 *d = vd, *a = va; uint32_t *n = vn, *m = vm; + float_status fpst, fpst_odd; - for (i = 0; i < elements; i += eltspersegment) { - uint32_t m_idx = m[i + H4(index)]; + if (is_ebf(env, &fpst, &fpst_odd)) { + for (i = 0; i < elements; i += eltspersegment) { + uint32_t m_idx = m[i + H4(index)]; - for (j = i; j < i + eltspersegment; j++) { - d[j] = bfdotadd(a[j], n[j], m_idx); + for (j = i; j < i + eltspersegment; j++) { + d[j] = bfdotadd_ebf(a[j], n[j], m_idx, &fpst, &fpst_odd); + } + } + } else { + for (i = 0; i < elements; i += eltspersegment) { + uint32_t m_idx = m[i + H4(index)]; + + for (j = i; j < i + eltspersegment; j++) { + d[j] = bfdotadd(a[j], n[j], m_idx, &fpst); + } } } clear_tail(d, opr_sz, simd_maxsz(desc)); @@ -XXX,XX +XXX,XX @@ void HELPER(gvec_bfmmla)(void *vd, void *vn, void *vm, void *va, intptr_t s, opr_sz = simd_oprsz(desc); float32 *d = vd, *a = va; uint32_t *n = vn, *m = vm; + float_status fpst, fpst_odd; - for (s = 0; s < opr_sz / 4; s += 4) { - float32 sum00, sum01, sum10, sum11; + if (is_ebf(env, &fpst, &fpst_odd)) { + for (s = 0; s < opr_sz / 4; s += 4) { + float32 sum00, sum01, sum10, sum11; - /* - * Process the entire segment at once, writing back the - * results only after we've consumed all of the inputs. - * - * Key to indices by column: - * i j i k j k - */ - sum00 = a[s + H4(0 + 0)]; - sum00 = bfdotadd(sum00, n[s + H4(0 + 0)], m[s + H4(0 + 0)]); - sum00 = bfdotadd(sum00, n[s + H4(0 + 1)], m[s + H4(0 + 1)]); + /* + * Process the entire segment at once, writing back the + * results only after we've consumed all of the inputs. + * + * Key to indices by column: + * i j i k j k + */ + sum00 = a[s + H4(0 + 0)]; + sum00 = bfdotadd_ebf(sum00, n[s + H4(0 + 0)], m[s + H4(0 + 0)], &fpst, &fpst_odd); + sum00 = bfdotadd_ebf(sum00, n[s + H4(0 + 1)], m[s + H4(0 + 1)], &fpst, &fpst_odd); - sum01 = a[s + H4(0 + 1)]; - sum01 = bfdotadd(sum01, n[s + H4(0 + 0)], m[s + H4(2 + 0)]); - sum01 = bfdotadd(sum01, n[s + H4(0 + 1)], m[s + H4(2 + 1)]); + sum01 = a[s + H4(0 + 1)]; + sum01 = bfdotadd_ebf(sum01, n[s + H4(0 + 0)], m[s + H4(2 + 0)], &fpst, &fpst_odd); + sum01 = bfdotadd_ebf(sum01, n[s + H4(0 + 1)], m[s + H4(2 + 1)], &fpst, &fpst_odd); - sum10 = a[s + H4(2 + 0)]; - sum10 = bfdotadd(sum10, n[s + H4(2 + 0)], m[s + H4(0 + 0)]); - sum10 = bfdotadd(sum10, n[s + H4(2 + 1)], m[s + H4(0 + 1)]); + sum10 = a[s + H4(2 + 0)]; + sum10 = bfdotadd_ebf(sum10, n[s + H4(2 + 0)], m[s + H4(0 + 0)], &fpst, &fpst_odd); + sum10 = bfdotadd_ebf(sum10, n[s + H4(2 + 1)], m[s + H4(0 + 1)], &fpst, &fpst_odd); - sum11 = a[s + H4(2 + 1)]; - sum11 = bfdotadd(sum11, n[s + H4(2 + 0)], m[s + H4(2 + 0)]); - sum11 = bfdotadd(sum11, n[s + H4(2 + 1)], m[s + H4(2 + 1)]); + sum11 = a[s + H4(2 + 1)]; + sum11 = bfdotadd_ebf(sum11, n[s + H4(2 + 0)], m[s + H4(2 + 0)], &fpst, &fpst_odd); + sum11 = bfdotadd_ebf(sum11, n[s + H4(2 + 1)], m[s + H4(2 + 1)], &fpst, &fpst_odd); - d[s + H4(0 + 0)] = sum00; - d[s + H4(0 + 1)] = sum01; - d[s + H4(2 + 0)] = sum10; - d[s + H4(2 + 1)] = sum11; + d[s + H4(0 + 0)] = sum00; + d[s + H4(0 + 1)] = sum01; + d[s + H4(2 + 0)] = sum10; + d[s + H4(2 + 1)] = sum11; + } + } else { + for (s = 0; s < opr_sz / 4; s += 4) { + float32 sum00, sum01, sum10, sum11; + + /* + * Process the entire segment at once, writing back the + * results only after we've consumed all of the inputs. + * + * Key to indices by column: + * i j i k j k + */ + sum00 = a[s + H4(0 + 0)]; + sum00 = bfdotadd(sum00, n[s + H4(0 + 0)], m[s + H4(0 + 0)], &fpst); + sum00 = bfdotadd(sum00, n[s + H4(0 + 1)], m[s + H4(0 + 1)], &fpst); + + sum01 = a[s + H4(0 + 1)]; + sum01 = bfdotadd(sum01, n[s + H4(0 + 0)], m[s + H4(2 + 0)], &fpst); + sum01 = bfdotadd(sum01, n[s + H4(0 + 1)], m[s + H4(2 + 1)], &fpst); + + sum10 = a[s + H4(2 + 0)]; + sum10 = bfdotadd(sum10, n[s + H4(2 + 0)], m[s + H4(0 + 0)], &fpst); + sum10 = bfdotadd(sum10, n[s + H4(2 + 1)], m[s + H4(0 + 1)], &fpst); + + sum11 = a[s + H4(2 + 1)]; + sum11 = bfdotadd(sum11, n[s + H4(2 + 0)], m[s + H4(2 + 0)], &fpst); + sum11 = bfdotadd(sum11, n[s + H4(2 + 1)], m[s + H4(2 + 1)], &fpst); + + d[s + H4(0 + 0)] = sum00; + d[s + H4(0 + 1)] = sum01; + d[s + H4(2 + 0)] = sum10; + d[s + H4(2 + 1)] = sum11; + } } clear_tail(d, opr_sz, simd_maxsz(desc)); } -- 2.34.1 Implement the FPCR.EBF=1 semantics for bfdotadd() operations: * is_ebf() sets up fpst and fpst_odd * bfdotadd_ebf() implements the fused paired-multiply-and-add operation that we need The paired-multiply-and-add is similar to f16_dotadd() and we use the same trick here as in that function, but the inputs here are bfloat16 rather than float16. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> --- target/arm/tcg/vec_helper.c | 57 +++++++++++++++++++++++++++++++++++-- 1 file changed, 54 insertions(+), 3 deletions(-) diff --git a/target/arm/tcg/vec_helper.c b/target/arm/tcg/vec_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/vec_helper.c +++ b/target/arm/tcg/vec_helper.c @@ -XXX,XX +XXX,XX @@ DO_MMLA_B(gvec_usmmla_b, do_usmmla_b) bool is_ebf(CPUARMState *env, float_status *statusp, float_status *oddstatusp) { - /* FPCR is ignored for BFDOT and BFMMLA. */ + /* + * For BFDOT, BFMMLA, etc, the behaviour depends on FPCR.EBF. + * For EBF = 0, we ignore the FPCR bits which determine rounding + * mode and denormal-flushing, and we do unfused multiplies and + * additions with intermediate rounding of all products and sums. + * For EBF = 1, we honour FPCR rounding mode and denormal-flushing bits, + * and we perform a fused two-way sum-of-products without intermediate + * rounding of the products. + * In either case, we don't set fp exception flags. + * + * EBF is AArch64 only, so even if it's set in the FPCR it has + * no effect on AArch32 instructions. + */ + bool ebf = is_a64(env) && env->vfp.fpcr & FPCR_EBF; *statusp = (float_status){ .tininess_before_rounding = float_tininess_before_rounding, .float_rounding_mode = float_round_to_odd_inf, @@ -XXX,XX +XXX,XX @@ bool is_ebf(CPUARMState *env, float_status *statusp, float_status *oddstatusp) .default_nan_mode = true, }; - return false; + if (ebf) { + float_status *fpst = &env->vfp.fp_status; + set_flush_to_zero(get_flush_to_zero(fpst), statusp); + set_flush_inputs_to_zero(get_flush_inputs_to_zero(fpst), statusp); + set_float_rounding_mode(get_float_rounding_mode(fpst), statusp); + + /* EBF=1 needs to do a step with round-to-odd semantics */ + *oddstatusp = *statusp; + set_float_rounding_mode(float_round_to_odd, oddstatusp); + } + + return ebf; } float32 bfdotadd(float32 sum, uint32_t e1, uint32_t e2, float_status *fpst) @@ -XXX,XX +XXX,XX @@ float32 bfdotadd(float32 sum, uint32_t e1, uint32_t e2, float_status *fpst) float32 bfdotadd_ebf(float32 sum, uint32_t e1, uint32_t e2, float_status *fpst, float_status *fpst_odd) { - g_assert_not_reached(); + /* + * Compare f16_dotadd() in sme_helper.c, but here we have + * bfloat16 inputs. In particular that means that we do not + * want the FPCR.FZ16 flush semantics, so we use the normal + * float_status for the input handling here. + */ + float64 e1r = float32_to_float64(e1 << 16, fpst); + float64 e1c = float32_to_float64(e1 & 0xffff0000u, fpst); + float64 e2r = float32_to_float64(e2 << 16, fpst); + float64 e2c = float32_to_float64(e2 & 0xffff0000u, fpst); + float64 t64; + float32 t32; + + /* + * The ARM pseudocode function FPDot performs both multiplies + * and the add with a single rounding operation. Emulate this + * by performing the first multiply in round-to-odd, then doing + * the second multiply as fused multiply-add, and rounding to + * float32 all in one step. + */ + t64 = float64_mul(e1r, e2r, fpst_odd); + t64 = float64r32_muladd(e1c, e2c, t64, 0, fpst); + + /* This conversion is exact, because we've already rounded. */ + t32 = float64_to_float32(t64, fpst); + + /* The final accumulation step is not fused. */ + return float32_add(sum, t32, fpst); } void HELPER(gvec_bfdot)(void *vd, void *vn, void *vm, void *va, -- 2.34.1 Now that we've implemented the required behaviour for FEAT_EBF16, we can enable it for the "max" CPU type, list it in our documentation, and delete a TODO comment about it being missing. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> --- docs/system/arm/emulation.rst | 1 + target/arm/tcg/cpu64.c | 4 ++-- target/arm/tcg/translate-sme.c | 1 - 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/system/arm/emulation.rst b/docs/system/arm/emulation.rst index XXXXXXX..XXXXXXX 100644 --- a/docs/system/arm/emulation.rst +++ b/docs/system/arm/emulation.rst @@ -XXX,XX +XXX,XX @@ the following architecture extensions: - FEAT_DotProd (Advanced SIMD dot product instructions) - FEAT_DoubleFault (Double Fault Extension) - FEAT_E0PD (Preventing EL0 access to halves of address maps) +- FEAT_EBF16 (AArch64 Extended BFloat16 instructions) - FEAT_ECV (Enhanced Counter Virtualization) - FEAT_EL0 (Support for execution at EL0) - FEAT_EL1 (Support for execution at EL1) diff --git a/target/arm/tcg/cpu64.c b/target/arm/tcg/cpu64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/cpu64.c +++ b/target/arm/tcg/cpu64.c @@ -XXX,XX +XXX,XX @@ void aarch64_max_tcg_initfn(Object *obj) t = FIELD_DP64(t, ID_AA64ISAR1, FRINTTS, 1); /* FEAT_FRINTTS */ t = FIELD_DP64(t, ID_AA64ISAR1, SB, 1); /* FEAT_SB */ t = FIELD_DP64(t, ID_AA64ISAR1, SPECRES, 1); /* FEAT_SPECRES */ - t = FIELD_DP64(t, ID_AA64ISAR1, BF16, 1); /* FEAT_BF16 */ + t = FIELD_DP64(t, ID_AA64ISAR1, BF16, 2); /* FEAT_BF16, FEAT_EBF16 */ t = FIELD_DP64(t, ID_AA64ISAR1, DGH, 1); /* FEAT_DGH */ t = FIELD_DP64(t, ID_AA64ISAR1, I8MM, 1); /* FEAT_I8MM */ cpu->isar.id_aa64isar1 = t; @@ -XXX,XX +XXX,XX @@ void aarch64_max_tcg_initfn(Object *obj) t = FIELD_DP64(t, ID_AA64ZFR0, SVEVER, 1); t = FIELD_DP64(t, ID_AA64ZFR0, AES, 2); /* FEAT_SVE_PMULL128 */ t = FIELD_DP64(t, ID_AA64ZFR0, BITPERM, 1); /* FEAT_SVE_BitPerm */ - t = FIELD_DP64(t, ID_AA64ZFR0, BFLOAT16, 1); /* FEAT_BF16 */ + t = FIELD_DP64(t, ID_AA64ZFR0, BFLOAT16, 2); /* FEAT_BF16, FEAT_EBF16 */ t = FIELD_DP64(t, ID_AA64ZFR0, SHA3, 1); /* FEAT_SVE_SHA3 */ t = FIELD_DP64(t, ID_AA64ZFR0, SM4, 1); /* FEAT_SVE_SM4 */ t = FIELD_DP64(t, ID_AA64ZFR0, I8MM, 1); /* FEAT_I8MM */ diff --git a/target/arm/tcg/translate-sme.c b/target/arm/tcg/translate-sme.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-sme.c +++ b/target/arm/tcg/translate-sme.c @@ -XXX,XX +XXX,XX @@ TRANS_FEAT(FMOPA_s, aa64_sme, do_outprod_fpst, a, TRANS_FEAT(FMOPA_d, aa64_sme_f64f64, do_outprod_fpst, a, MO_64, FPST_FPCR, gen_helper_sme_fmopa_d) -/* TODO: FEAT_EBF16 */ TRANS_FEAT(BFMOPA, aa64_sme, do_outprod_env, a, MO_32, gen_helper_sme_bfmopa) TRANS_FEAT(SMOPA_s, aa64_sme, do_outprod, a, MO_32, gen_helper_sme_smopa_s) -- 2.34.1 The main loop in rr_cpu_thread_fn() can never terminate, so the code at the end of the function to clean up the RCU subsystem is dead code. Replace it with g_assert_not_reached(). (This is different from the other cpu_thread_fn for e.g. MTTCG or for the KVM accelerator -- those can exit, if the vCPU they are responsible for is unplugged. But the RR cpu thread fn handles all CPUs in the system in a round-robin way, so even if one is unplugged it keeps looping.) Resolves: Coverity CID 1547782 Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20240815143634.3413679-1-peter.maydell@linaro.org --- accel/tcg/tcg-accel-ops-rr.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/accel/tcg/tcg-accel-ops-rr.c b/accel/tcg/tcg-accel-ops-rr.c index XXXXXXX..XXXXXXX 100644 --- a/accel/tcg/tcg-accel-ops-rr.c +++ b/accel/tcg/tcg-accel-ops-rr.c @@ -XXX,XX +XXX,XX @@ static void *rr_cpu_thread_fn(void *arg) rr_deal_with_unplugged_cpus(); } - rcu_remove_force_rcu_notifier(&force_rcu); - rcu_unregister_thread(); - return NULL; + g_assert_not_reached(); } void rr_start_vcpu_thread(CPUState *cpu) -- 2.34.1 From: Cornelia Huck <cohuck@redhat.com> Add 9.2 machine types for arm/i440fx/m68k/q35/s390x/spapr. Signed-off-by: Cornelia Huck <cohuck@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Eric Auger <eric.auger@redhat.com> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Message-id: 20240816161350.3706332-2-peter.maydell@linaro.org Message-id: 20240816103723.2325982-1-cohuck@redhat.com Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- include/hw/boards.h | 3 +++ include/hw/i386/pc.h | 3 +++ hw/arm/virt.c | 11 +++++++++-- hw/core/machine.c | 3 +++ hw/i386/pc.c | 3 +++ hw/i386/pc_piix.c | 15 ++++++++++++--- hw/i386/pc_q35.c | 13 +++++++++++-- hw/m68k/virt.c | 11 +++++++++-- hw/ppc/spapr.c | 17 ++++++++++++++--- hw/s390x/s390-virtio-ccw.c | 14 +++++++++++++- 10 files changed, 80 insertions(+), 13 deletions(-) diff --git a/include/hw/boards.h b/include/hw/boards.h index XXXXXXX..XXXXXXX 100644 --- a/include/hw/boards.h +++ b/include/hw/boards.h @@ -XXX,XX +XXX,XX @@ struct MachineState { } \ type_init(machine_initfn##_register_types) +extern GlobalProperty hw_compat_9_1[]; +extern const size_t hw_compat_9_1_len; + extern GlobalProperty hw_compat_9_0[]; extern const size_t hw_compat_9_0_len; diff --git a/include/hw/i386/pc.h b/include/hw/i386/pc.h index XXXXXXX..XXXXXXX 100644 --- a/include/hw/i386/pc.h +++ b/include/hw/i386/pc.h @@ -XXX,XX +XXX,XX @@ void pc_system_parse_ovmf_flash(uint8_t *flash_ptr, size_t flash_size); /* sgx.c */ void pc_machine_init_sgx_epc(PCMachineState *pcms); +extern GlobalProperty pc_compat_9_1[]; +extern const size_t pc_compat_9_1_len; + extern GlobalProperty pc_compat_9_0[]; extern const size_t pc_compat_9_0_len; diff --git a/hw/arm/virt.c b/hw/arm/virt.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/virt.c +++ b/hw/arm/virt.c @@ -XXX,XX +XXX,XX @@ static void machvirt_machine_init(void) } type_init(machvirt_machine_init); -static void virt_machine_9_1_options(MachineClass *mc) +static void virt_machine_9_2_options(MachineClass *mc) { } -DEFINE_VIRT_MACHINE_AS_LATEST(9, 1) +DEFINE_VIRT_MACHINE_AS_LATEST(9, 2) + +static void virt_machine_9_1_options(MachineClass *mc) +{ + virt_machine_9_2_options(mc); + compat_props_add(mc->compat_props, hw_compat_9_1, hw_compat_9_1_len); +} +DEFINE_VIRT_MACHINE(9, 1) static void virt_machine_9_0_options(MachineClass *mc) { diff --git a/hw/core/machine.c b/hw/core/machine.c index XXXXXXX..XXXXXXX 100644 --- a/hw/core/machine.c +++ b/hw/core/machine.c @@ -XXX,XX +XXX,XX @@ #include "hw/virtio/virtio-iommu.h" #include "audio/audio.h" +GlobalProperty hw_compat_9_1[] = {}; +const size_t hw_compat_9_1_len = G_N_ELEMENTS(hw_compat_9_1); + GlobalProperty hw_compat_9_0[] = { {"arm-cpu", "backcompat-cntfrq", "true" }, { "scsi-hd", "migrate-emulated-scsi-request", "false" }, diff --git a/hw/i386/pc.c b/hw/i386/pc.c index XXXXXXX..XXXXXXX 100644 --- a/hw/i386/pc.c +++ b/hw/i386/pc.c @@ -XXX,XX +XXX,XX @@ { "qemu64-" TYPE_X86_CPU, "model-id", "QEMU Virtual CPU version " v, },\ { "athlon-" TYPE_X86_CPU, "model-id", "QEMU Virtual CPU version " v, }, +GlobalProperty pc_compat_9_1[] = {}; +const size_t pc_compat_9_1_len = G_N_ELEMENTS(pc_compat_9_1); + GlobalProperty pc_compat_9_0[] = { { TYPE_X86_CPU, "x-amd-topoext-features-only", "false" }, { TYPE_X86_CPU, "x-l1-cache-per-thread", "false" }, diff --git a/hw/i386/pc_piix.c b/hw/i386/pc_piix.c index XXXXXXX..XXXXXXX 100644 --- a/hw/i386/pc_piix.c +++ b/hw/i386/pc_piix.c @@ -XXX,XX +XXX,XX @@ static void pc_i440fx_machine_options(MachineClass *m) "Use a different south bridge than PIIX3"); } -static void pc_i440fx_machine_9_1_options(MachineClass *m) +static void pc_i440fx_machine_9_2_options(MachineClass *m) { pc_i440fx_machine_options(m); m->alias = "pc"; m->is_default = true; } +DEFINE_I440FX_MACHINE(9, 2); + +static void pc_i440fx_machine_9_1_options(MachineClass *m) +{ + pc_i440fx_machine_9_2_options(m); + m->alias = NULL; + m->is_default = false; + compat_props_add(m->compat_props, hw_compat_9_1, hw_compat_9_1_len); + compat_props_add(m->compat_props, pc_compat_9_1, pc_compat_9_1_len); +} + DEFINE_I440FX_MACHINE(9, 1); static void pc_i440fx_machine_9_0_options(MachineClass *m) @@ -XXX,XX +XXX,XX @@ static void pc_i440fx_machine_9_0_options(MachineClass *m) PCMachineClass *pcmc = PC_MACHINE_CLASS(m); pc_i440fx_machine_9_1_options(m); - m->alias = NULL; - m->is_default = false; m->smbios_memory_device_size = 16 * GiB; compat_props_add(m->compat_props, hw_compat_9_0, hw_compat_9_0_len); diff --git a/hw/i386/pc_q35.c b/hw/i386/pc_q35.c index XXXXXXX..XXXXXXX 100644 --- a/hw/i386/pc_q35.c +++ b/hw/i386/pc_q35.c @@ -XXX,XX +XXX,XX @@ static void pc_q35_machine_options(MachineClass *m) pc_q35_compat_defaults, pc_q35_compat_defaults_len); } -static void pc_q35_machine_9_1_options(MachineClass *m) +static void pc_q35_machine_9_2_options(MachineClass *m) { pc_q35_machine_options(m); m->alias = "q35"; } +DEFINE_Q35_MACHINE(9, 2); + +static void pc_q35_machine_9_1_options(MachineClass *m) +{ + pc_q35_machine_9_2_options(m); + m->alias = NULL; + compat_props_add(m->compat_props, hw_compat_9_1, hw_compat_9_1_len); + compat_props_add(m->compat_props, pc_compat_9_1, pc_compat_9_1_len); +} + DEFINE_Q35_MACHINE(9, 1); static void pc_q35_machine_9_0_options(MachineClass *m) { PCMachineClass *pcmc = PC_MACHINE_CLASS(m); pc_q35_machine_9_1_options(m); - m->alias = NULL; m->smbios_memory_device_size = 16 * GiB; compat_props_add(m->compat_props, hw_compat_9_0, hw_compat_9_0_len); compat_props_add(m->compat_props, pc_compat_9_0, pc_compat_9_0_len); diff --git a/hw/m68k/virt.c b/hw/m68k/virt.c index XXXXXXX..XXXXXXX 100644 --- a/hw/m68k/virt.c +++ b/hw/m68k/virt.c @@ -XXX,XX +XXX,XX @@ type_init(virt_machine_register_types) #define DEFINE_VIRT_MACHINE(major, minor) \ DEFINE_VIRT_MACHINE_IMPL(false, major, minor) -static void virt_machine_9_1_options(MachineClass *mc) +static void virt_machine_9_2_options(MachineClass *mc) { } -DEFINE_VIRT_MACHINE_AS_LATEST(9, 1) +DEFINE_VIRT_MACHINE_AS_LATEST(9, 2) + +static void virt_machine_9_1_options(MachineClass *mc) +{ + virt_machine_9_2_options(mc); + compat_props_add(mc->compat_props, hw_compat_9_1, hw_compat_9_1_len); +} +DEFINE_VIRT_MACHINE(9, 1) static void virt_machine_9_0_options(MachineClass *mc) { diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c index XXXXXXX..XXXXXXX 100644 --- a/hw/ppc/spapr.c +++ b/hw/ppc/spapr.c @@ -XXX,XX +XXX,XX @@ static void spapr_machine_latest_class_options(MachineClass *mc) DEFINE_SPAPR_MACHINE_IMPL(false, major, minor, _, tag) /* - * pseries-9.1 + * pseries-9.2 */ -static void spapr_machine_9_1_class_options(MachineClass *mc) +static void spapr_machine_9_2_class_options(MachineClass *mc) { /* Defaults for the latest behaviour inherited from the base class */ } -DEFINE_SPAPR_MACHINE_AS_LATEST(9, 1); +DEFINE_SPAPR_MACHINE_AS_LATEST(9, 2); + +/* + * pseries-9.1 + */ +static void spapr_machine_9_1_class_options(MachineClass *mc) +{ + spapr_machine_9_2_class_options(mc); + compat_props_add(mc->compat_props, hw_compat_9_1, hw_compat_9_1_len); +} + +DEFINE_SPAPR_MACHINE(9, 1); /* * pseries-9.0 diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c index XXXXXXX..XXXXXXX 100644 --- a/hw/s390x/s390-virtio-ccw.c +++ b/hw/s390x/s390-virtio-ccw.c @@ -XXX,XX +XXX,XX @@ static const TypeInfo ccw_machine_info = { DEFINE_CCW_MACHINE_IMPL(false, major, minor) +static void ccw_machine_9_2_instance_options(MachineState *machine) +{ +} + +static void ccw_machine_9_2_class_options(MachineClass *mc) +{ +} +DEFINE_CCW_MACHINE_AS_LATEST(9, 2); + static void ccw_machine_9_1_instance_options(MachineState *machine) { + ccw_machine_9_2_instance_options(machine); } static void ccw_machine_9_1_class_options(MachineClass *mc) { + ccw_machine_9_2_class_options(mc); + compat_props_add(mc->compat_props, hw_compat_9_1, hw_compat_9_1_len); } -DEFINE_CCW_MACHINE_AS_LATEST(9, 1); +DEFINE_CCW_MACHINE(9, 1); static void ccw_machine_9_0_instance_options(MachineState *machine) { -- 2.34.1 When we added support for nested (stage 1 + stage 2) translation to the SMMU in commit 58377c363291d we forgot to update the comment that documents the valid values of the "stage" property. Add the new "nested" value to it. Fixes: 58377c363291d ("hw/arm/smmuv3: Support and advertise nesting") Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Eric Auger <eric.auger@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Message-id: 20240816161350.3706332-3-peter.maydell@linaro.org --- hw/arm/smmuv3.c | 1 + 1 file changed, 1 insertion(+) diff --git a/hw/arm/smmuv3.c b/hw/arm/smmuv3.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/smmuv3.c +++ b/hw/arm/smmuv3.c @@ -XXX,XX +XXX,XX @@ static Property smmuv3_properties[] = { * Stages of translation advertised. * "1": Stage 1 * "2": Stage 2 + * "nested": Both stage 1 and stage 2 * Defaults to stage 1 */ DEFINE_PROP_STRING("stage", SMMUv3State, stage), -- 2.34.1 Now that our SMMU model supports enabling both stages of translation at once, we can enable this in the virt board. This is no change in behaviour for guests, because if they simply ignore stage 2 and never configure it then it has no effect. For the usual backwards compatibility reasons we enable this only for machine types starting with 9.2. (Note that the SMMU is disabled by default on the virt board and is only created if the user passes the 'iommu=smmuv3' machine option.) Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Eric Auger <eric.auger@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Message-id: 20240816161350.3706332-4-peter.maydell@linaro.org --- include/hw/arm/virt.h | 1 + hw/arm/virt.c | 8 ++++++++ 2 files changed, 9 insertions(+) diff --git a/include/hw/arm/virt.h b/include/hw/arm/virt.h index XXXXXXX..XXXXXXX 100644 --- a/include/hw/arm/virt.h +++ b/include/hw/arm/virt.h @@ -XXX,XX +XXX,XX @@ struct VirtMachineClass { bool no_cpu_topology; bool no_tcg_lpa2; bool no_ns_el2_virt_timer_irq; + bool no_nested_smmu; }; struct VirtMachineState { diff --git a/hw/arm/virt.c b/hw/arm/virt.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/virt.c +++ b/hw/arm/virt.c @@ -XXX,XX +XXX,XX @@ static void create_pcie_irq_map(const MachineState *ms, static void create_smmu(const VirtMachineState *vms, PCIBus *bus) { + VirtMachineClass *vmc = VIRT_MACHINE_GET_CLASS(vms); char *node; const char compat[] = "arm,smmu-v3"; int irq = vms->irqmap[VIRT_SMMU]; @@ -XXX,XX +XXX,XX @@ static void create_smmu(const VirtMachineState *vms, dev = qdev_new(TYPE_ARM_SMMUV3); + if (!vmc->no_nested_smmu) { + object_property_set_str(OBJECT(dev), "stage", "nested", &error_fatal); + } object_property_set_link(OBJECT(dev), "primary-bus", OBJECT(bus), &error_abort); sysbus_realize_and_unref(SYS_BUS_DEVICE(dev), &error_fatal); @@ -XXX,XX +XXX,XX @@ DEFINE_VIRT_MACHINE_AS_LATEST(9, 2) static void virt_machine_9_1_options(MachineClass *mc) { + VirtMachineClass *vmc = VIRT_MACHINE_CLASS(OBJECT_CLASS(mc)); + virt_machine_9_2_options(mc); compat_props_add(mc->compat_props, hw_compat_9_1, hw_compat_9_1_len); + /* 9.1 and earlier have only a stage-1 SMMU, not a nested s1+2 one */ + vmc->no_nested_smmu = true; } DEFINE_VIRT_MACHINE(9, 1) -- 2.34.1 Now that our SMMU model supports enabling both stages of translation at once, we can enable this in the sbsa-ref board. Existing guest code that only programs stage 1 and doesn't care about stage 2 should continue to run with the same behaviour, but guests that do want to do nested SMMU configurations can now do so. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Eric Auger <eric.auger@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org> Message-id: 20240816161350.3706332-5-peter.maydell@linaro.org --- hw/arm/sbsa-ref.c | 1 + 1 file changed, 1 insertion(+) diff --git a/hw/arm/sbsa-ref.c b/hw/arm/sbsa-ref.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/sbsa-ref.c +++ b/hw/arm/sbsa-ref.c @@ -XXX,XX +XXX,XX @@ static void create_smmu(const SBSAMachineState *sms, PCIBus *bus) dev = qdev_new(TYPE_ARM_SMMUV3); + object_property_set_str(OBJECT(dev), "stage", "nested", &error_abort); object_property_set_link(OBJECT(dev), "primary-bus", OBJECT(bus), &error_abort); sysbus_realize_and_unref(SYS_BUS_DEVICE(dev), &error_fatal); -- 2.34.1 Since the TYPE_XNLX_VERSAL_CFU_FDRO device creates a FIFO in its instance_init method, we must destroy the FIFO in instance_finalize to avoid a memory leak for the QOM introspection "instantiate-examine-finalize" cycle: Direct leak of 8192 byte(s) in 1 object(s) allocated from: #0 0x55ec89eae7ee in malloc (/mnt/nvmedisk/linaro/qemu-from-laptop/qemu/build/asan/qemu-system-aarch64+0x294d7ee) (BuildId: 6d508874816cc47d17c8dd775e8f809ae520e8cb) #1 0x7f697018f738 in g_malloc debian/build/deb/../../../glib/gmem.c:128:13 #2 0x55ec8d98d98d in fifo8_create util/fifo8.c:27:18 #3 0x55ec8aa2a624 in fifo32_create /mnt/nvmedisk/linaro/qemu-from-laptop/qemu/include/qemu/fifo32.h:35:5 #4 0x55ec8aa2a33c in cfu_fdro_init hw/misc/xlnx-versal-cfu.c:397:5 #5 0x55ec8ce75da1 in object_init_with_type qom/object.c:420:9 #6 0x55ec8ce5d07b in object_initialize_with_type qom/object.c:562:5 #7 0x55ec8ce5e91d in object_new_with_type qom/object.c:782:5 #8 0x55ec8ce5e9f1 in object_new qom/object.c:797:12 #9 0x55ec8d65c81d in qmp_device_list_properties qom/qom-qmp-cmds.c:144:11 Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Edgar E. Iglesias <edgar.iglesias@amd.com> Reviewed-by: Francisco Iglesias <francisco.iglesias@amd.com> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> Message-id: 20240822162127.705879-2-peter.maydell@linaro.org --- hw/misc/xlnx-versal-cfu.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/hw/misc/xlnx-versal-cfu.c b/hw/misc/xlnx-versal-cfu.c index XXXXXXX..XXXXXXX 100644 --- a/hw/misc/xlnx-versal-cfu.c +++ b/hw/misc/xlnx-versal-cfu.c @@ -XXX,XX +XXX,XX @@ static void cfu_fdro_init(Object *obj) fifo32_create(&s->fdro_data, 8 * KiB / sizeof(uint32_t)); } +static void cfu_fdro_finalize(Object *obj) +{ + XlnxVersalCFUFDRO *s = XLNX_VERSAL_CFU_FDRO(obj); + + fifo32_destroy(&s->fdro_data); +} + static void cfu_fdro_reset_enter(Object *obj, ResetType type) { XlnxVersalCFUFDRO *s = XLNX_VERSAL_CFU_FDRO(obj); @@ -XXX,XX +XXX,XX @@ static const TypeInfo cfu_fdro_info = { .instance_size = sizeof(XlnxVersalCFUFDRO), .class_init = cfu_fdro_class_init, .instance_init = cfu_fdro_init, + .instance_finalize = cfu_fdro_finalize, .interfaces = (InterfaceInfo[]) { { TYPE_XLNX_CFI_IF }, { } -- 2.34.1 The TYPE_XLNX_VERSAL_TRNG device creates s->prng with g_rand_new() in its init method, but it frees it in its unrealize method. This results in a leak in the QOM introspection "initialize-inspect-finalize" lifecycle: Direct leak of 2500 byte(s) in 1 object(s) allocated from: #0 0x55ec89eae9d8 in __interceptor_calloc (/mnt/nvmedisk/linaro/qemu-from-laptop/qemu/build/asan/qemu-system-aarch64+0x294d9d8) (BuildId: 6d5 08874816cc47d17c8dd775e8f809ae520e8cb) #1 0x7f697018fc50 in g_malloc0 debian/build/deb/../../../glib/gmem.c:161:13 #2 0x7f6970197738 in g_rand_new_with_seed_array debian/build/deb/../../../glib/grand.c:202:17 #3 0x7f6970197816 in g_rand_new debian/build/deb/../../../glib/grand.c:286:10 #4 0x55ec8aa3656a in trng_init hw/misc/xlnx-versal-trng.c:624:15 #5 0x55ec8ce75da1 in object_init_with_type qom/object.c:420:9 #6 0x55ec8ce5d07b in object_initialize_with_type qom/object.c:562:5 #7 0x55ec8ce5e91d in object_new_with_type qom/object.c:782:5 #8 0x55ec8ce5e9f1 in object_new qom/object.c:797:12 #9 0x55ec8d65c81d in qmp_device_list_properties qom/qom-qmp-cmds.c:144:11 Move the free to finalize so it matches where we are initing s->prng. Since that's the only thing our unrealize method was doing, this essentially switches the whole function to be a finalize implementation. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Edgar E. Iglesias <edgar.iglesias@amd.com> Reviewed-by: Francisco Iglesias <francisco.iglesias@amd.com> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> Message-id: 20240822162127.705879-3-peter.maydell@linaro.org --- hw/misc/xlnx-versal-trng.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/hw/misc/xlnx-versal-trng.c b/hw/misc/xlnx-versal-trng.c index XXXXXXX..XXXXXXX 100644 --- a/hw/misc/xlnx-versal-trng.c +++ b/hw/misc/xlnx-versal-trng.c @@ -XXX,XX +XXX,XX @@ static void trng_init(Object *obj) s->prng = g_rand_new(); } -static void trng_unrealize(DeviceState *dev) +static void trng_finalize(Object *obj) { - XlnxVersalTRng *s = XLNX_VERSAL_TRNG(dev); + XlnxVersalTRng *s = XLNX_VERSAL_TRNG(obj); g_rand_free(s->prng); s->prng = NULL; @@ -XXX,XX +XXX,XX @@ static void trng_class_init(ObjectClass *klass, void *data) ResettableClass *rc = RESETTABLE_CLASS(klass); dc->vmsd = &vmstate_trng; - dc->unrealize = trng_unrealize; rc->phases.hold = trng_reset_hold; /* Clone uint64 property with set allowed after realized */ @@ -XXX,XX +XXX,XX @@ static const TypeInfo trng_info = { .instance_size = sizeof(XlnxVersalTRng), .class_init = trng_class_init, .instance_init = trng_init, + .instance_finalize = trng_finalize, }; static void trng_register_types(void) -- 2.34.1 The TYPE_XLNX_BBRAM device creates a register block with register_init_block32() in its instance_init method; we must therefore destroy it in our instance_finalize method to avoid a leak in the QOM introspection "init-inspect-finalize" lifecycle: Direct leak of 304 byte(s) in 1 object(s) allocated from: #0 0x5641518ca9d8 in __interceptor_calloc (/mnt/nvmedisk/linaro/qemu-from-laptop/qemu/build/asan/qemu-system-aarch64+0x294d9d8) (BuildId: 4a6 18cb63d57d5a19ed45cfc262b08da47eaafe5) #1 0x7ff1aab31c50 in g_malloc0 debian/build/deb/../../../glib/gmem.c:161:13 #2 0x564151cffc5d in register_init_block hw/core/register.c:248:34 #3 0x564151d006be in register_init_block32 hw/core/register.c:299:12 #4 0x56415293df75 in bbram_ctrl_init hw/nvram/xlnx-bbram.c:462:9 #5 0x564154891dc1 in object_init_with_type qom/object.c:420:9 #6 0x56415487909b in object_initialize_with_type qom/object.c:562:5 #7 0x56415487a93d in object_new_with_type qom/object.c:782:5 #8 0x56415487aa11 in object_new qom/object.c:797:12 #9 0x56415507883d in qmp_device_list_properties qom/qom-qmp-cmds.c:144:11 Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Edgar E. Iglesias <edgar.iglesias@amd.com> Reviewed-by: Francisco Iglesias <francisco.iglesias@amd.com> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> Message-id: 20240822162127.705879-4-peter.maydell@linaro.org --- include/hw/nvram/xlnx-bbram.h | 1 + hw/nvram/xlnx-bbram.c | 13 ++++++++++--- 2 files changed, 11 insertions(+), 3 deletions(-) diff --git a/include/hw/nvram/xlnx-bbram.h b/include/hw/nvram/xlnx-bbram.h index XXXXXXX..XXXXXXX 100644 --- a/include/hw/nvram/xlnx-bbram.h +++ b/include/hw/nvram/xlnx-bbram.h @@ -XXX,XX +XXX,XX @@ struct XlnxBBRam { bool bbram8_wo; bool blk_ro; + RegisterInfoArray *reg_array; uint32_t regs[RMAX_XLNX_BBRAM]; RegisterInfo regs_info[RMAX_XLNX_BBRAM]; }; diff --git a/hw/nvram/xlnx-bbram.c b/hw/nvram/xlnx-bbram.c index XXXXXXX..XXXXXXX 100644 --- a/hw/nvram/xlnx-bbram.c +++ b/hw/nvram/xlnx-bbram.c @@ -XXX,XX +XXX,XX @@ static void bbram_ctrl_init(Object *obj) { XlnxBBRam *s = XLNX_BBRAM(obj); SysBusDevice *sbd = SYS_BUS_DEVICE(obj); - RegisterInfoArray *reg_array; - reg_array = + s->reg_array = register_init_block32(DEVICE(obj), bbram_ctrl_regs_info, ARRAY_SIZE(bbram_ctrl_regs_info), s->regs_info, s->regs, @@ -XXX,XX +XXX,XX @@ static void bbram_ctrl_init(Object *obj) XLNX_BBRAM_ERR_DEBUG, R_MAX * 4); - sysbus_init_mmio(sbd, &reg_array->mem); + sysbus_init_mmio(sbd, &s->reg_array->mem); sysbus_init_irq(sbd, &s->irq_bbram); } +static void bbram_ctrl_finalize(Object *obj) +{ + XlnxBBRam *s = XLNX_BBRAM(obj); + + register_finalize_block(s->reg_array); +} + static void bbram_prop_set_drive(Object *obj, Visitor *v, const char *name, void *opaque, Error **errp) { @@ -XXX,XX +XXX,XX @@ static const TypeInfo bbram_ctrl_info = { .instance_size = sizeof(XlnxBBRam), .class_init = bbram_ctrl_class_init, .instance_init = bbram_ctrl_init, + .instance_finalize = bbram_ctrl_finalize, }; static void bbram_ctrl_register_types(void) -- 2.34.1 The TYPE_XLNX_ZYNQMP_EFUSE device creates a register block with register_init_block32() in its instance_init method; we must therefore destroy it in our instance_finalize method to avoid a leak in the QOM introspection "init-inspect-finalize" lifecycle: Direct leak of 304 byte(s) in 1 object(s) allocated from: #0 0x55f3ff5839d8 in __interceptor_calloc (/mnt/nvmedisk/linaro/qemu-from-laptop/qemu/build/asan/qemu-system-aarch64+0x294d9d8) (BuildId: 23cf931c66865a71b6cc4da95156d03bc106fa72) #1 0x7f3f31c6bc50 in g_malloc0 debian/build/deb/../../../glib/gmem.c:161:13 #2 0x55f3ff9b8c5d in register_init_block hw/core/register.c:248:34 #3 0x55f3ff9b96be in register_init_block32 hw/core/register.c:299:12 #4 0x55f4005e5b25 in efuse_ctrl_init hw/nvram/xlnx-versal-efuse-ctrl.c:718:9 #5 0x55f40254afb1 in object_init_with_type qom/object.c:420:9 #6 0x55f40253228b in object_initialize_with_type qom/object.c:562:5 #7 0x55f402533b2d in object_new_with_type qom/object.c:782:5 #8 0x55f402533c01 in object_new qom/object.c:797:12 #9 0x55f402d31a2d in qmp_device_list_properties qom/qom-qmp-cmds.c:144:11 Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Edgar E. Iglesias <edgar.iglesias@amd.com> Reviewed-by: Francisco Iglesias <francisco.iglesias@amd.com> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> Message-id: 20240822162127.705879-5-peter.maydell@linaro.org --- include/hw/nvram/xlnx-zynqmp-efuse.h | 1 + hw/nvram/xlnx-zynqmp-efuse.c | 13 ++++++++++--- 2 files changed, 11 insertions(+), 3 deletions(-) diff --git a/include/hw/nvram/xlnx-zynqmp-efuse.h b/include/hw/nvram/xlnx-zynqmp-efuse.h index XXXXXXX..XXXXXXX 100644 --- a/include/hw/nvram/xlnx-zynqmp-efuse.h +++ b/include/hw/nvram/xlnx-zynqmp-efuse.h @@ -XXX,XX +XXX,XX @@ struct XlnxZynqMPEFuse { qemu_irq irq; XlnxEFuse *efuse; + RegisterInfoArray *reg_array; uint32_t regs[XLNX_ZYNQMP_EFUSE_R_MAX]; RegisterInfo regs_info[XLNX_ZYNQMP_EFUSE_R_MAX]; }; diff --git a/hw/nvram/xlnx-zynqmp-efuse.c b/hw/nvram/xlnx-zynqmp-efuse.c index XXXXXXX..XXXXXXX 100644 --- a/hw/nvram/xlnx-zynqmp-efuse.c +++ b/hw/nvram/xlnx-zynqmp-efuse.c @@ -XXX,XX +XXX,XX @@ static void zynqmp_efuse_init(Object *obj) { XlnxZynqMPEFuse *s = XLNX_ZYNQMP_EFUSE(obj); SysBusDevice *sbd = SYS_BUS_DEVICE(obj); - RegisterInfoArray *reg_array; - reg_array = + s->reg_array = register_init_block32(DEVICE(obj), zynqmp_efuse_regs_info, ARRAY_SIZE(zynqmp_efuse_regs_info), s->regs_info, s->regs, @@ -XXX,XX +XXX,XX @@ static void zynqmp_efuse_init(Object *obj) ZYNQMP_EFUSE_ERR_DEBUG, R_MAX * 4); - sysbus_init_mmio(sbd, &reg_array->mem); + sysbus_init_mmio(sbd, &s->reg_array->mem); sysbus_init_irq(sbd, &s->irq); } +static void zynqmp_efuse_finalize(Object *obj) +{ + XlnxZynqMPEFuse *s = XLNX_ZYNQMP_EFUSE(obj); + + register_finalize_block(s->reg_array); +} + static const VMStateDescription vmstate_efuse = { .name = TYPE_XLNX_ZYNQMP_EFUSE, .version_id = 1, @@ -XXX,XX +XXX,XX @@ static const TypeInfo efuse_info = { .instance_size = sizeof(XlnxZynqMPEFuse), .class_init = zynqmp_efuse_class_init, .instance_init = zynqmp_efuse_init, + .instance_finalize = zynqmp_efuse_finalize, }; static void efuse_register_types(void) -- 2.34.1 The TYPE_XLNX_VERSAL_TRNG device creates a register block with register_init_block32() in its instance_init method; we must therefore destroy it in our instance_finalize method to avoid a leak in the QOM introspection "init-inspect-finalize" lifecycle: Direct leak of 304 byte(s) in 1 object(s) allocated from: #0 0x55842ec799d8 in __interceptor_calloc (/mnt/nvmedisk/linaro/qemu-from-laptop/qemu/build/asan/qemu-system-aarch64+0x294e9d8) (BuildId: 47496e53f3e779f1c7e9b82cbea07407152b498b) #1 0x7fe793c75c50 in g_malloc0 debian/build/deb/../../../glib/gmem.c:161:13 #2 0x55842f0aec5d in register_init_block hw/core/register.c:248:34 #3 0x55842f0af6be in register_init_block32 hw/core/register.c:299:12 #4 0x55842f801588 in trng_init hw/misc/xlnx-versal-trng.c:614:9 #5 0x558431c411a1 in object_init_with_type qom/object.c:420:9 #6 0x558431c2847b in object_initialize_with_type qom/object.c:562:5 #7 0x558431c29d1d in object_new_with_type qom/object.c:782:5 #8 0x558431c29df1 in object_new qom/object.c:797:12 #9 0x558432427c1d in qmp_device_list_properties qom/qom-qmp-cmds.c:144:11 Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Edgar E. Iglesias <edgar.iglesias@amd.com> Reviewed-by: Francisco Iglesias <francisco.iglesias@amd.com> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> Message-id: 20240822162127.705879-6-peter.maydell@linaro.org --- include/hw/misc/xlnx-versal-trng.h | 1 + hw/misc/xlnx-versal-trng.c | 6 +++--- 2 files changed, 4 insertions(+), 3 deletions(-) diff --git a/include/hw/misc/xlnx-versal-trng.h b/include/hw/misc/xlnx-versal-trng.h index XXXXXXX..XXXXXXX 100644 --- a/include/hw/misc/xlnx-versal-trng.h +++ b/include/hw/misc/xlnx-versal-trng.h @@ -XXX,XX +XXX,XX @@ typedef struct XlnxVersalTRng { uint64_t forced_prng_count; uint64_t tst_seed[2]; + RegisterInfoArray *reg_array; uint32_t regs[RMAX_XLNX_VERSAL_TRNG]; RegisterInfo regs_info[RMAX_XLNX_VERSAL_TRNG]; } XlnxVersalTRng; diff --git a/hw/misc/xlnx-versal-trng.c b/hw/misc/xlnx-versal-trng.c index XXXXXXX..XXXXXXX 100644 --- a/hw/misc/xlnx-versal-trng.c +++ b/hw/misc/xlnx-versal-trng.c @@ -XXX,XX +XXX,XX @@ static void trng_init(Object *obj) { XlnxVersalTRng *s = XLNX_VERSAL_TRNG(obj); SysBusDevice *sbd = SYS_BUS_DEVICE(obj); - RegisterInfoArray *reg_array; - reg_array = + s->reg_array = register_init_block32(DEVICE(obj), trng_regs_info, ARRAY_SIZE(trng_regs_info), s->regs_info, s->regs, @@ -XXX,XX +XXX,XX @@ static void trng_init(Object *obj) XLNX_VERSAL_TRNG_ERR_DEBUG, R_MAX * 4); - sysbus_init_mmio(sbd, &reg_array->mem); + sysbus_init_mmio(sbd, &s->reg_array->mem); sysbus_init_irq(sbd, &s->irq); s->prng = g_rand_new(); @@ -XXX,XX +XXX,XX @@ static void trng_finalize(Object *obj) { XlnxVersalTRng *s = XLNX_VERSAL_TRNG(obj); + register_finalize_block(s->reg_array); g_rand_free(s->prng); s->prng = NULL; } -- 2.34.1 The TYPE_XLNX_VERSAL_EFUSE_CTRL device creates a register block with register_init_block32() in its instance_init method; we must therefore destroy it in our instance_finalize method to avoid a leak in the QOM introspection "init-inspect-finalize" lifecycle: Direct leak of 304 byte(s) in 1 object(s) allocated from: #0 0x55f222b5b9d8 in __interceptor_calloc (/mnt/nvmedisk/linaro/qemu-from-laptop/qemu/build/asan/qemu-system-aarch64+0x294e9d8) (BuildId: 420 43d49e1139e3f3071b1f22fac1e3e7249c9a6) #1 0x7fbb10669c50 in g_malloc0 debian/build/deb/../../../glib/gmem.c:161:13 #2 0x55f222f90c5d in register_init_block hw/core/register.c:248:34 #3 0x55f222f916be in register_init_block32 hw/core/register.c:299:12 #4 0x55f223bbdd15 in efuse_ctrl_init hw/nvram/xlnx-versal-efuse-ctrl.c:718:9 #5 0x55f225b23391 in object_init_with_type qom/object.c:420:9 #6 0x55f225b0a66b in object_initialize_with_type qom/object.c:562:5 #7 0x55f225b0bf0d in object_new_with_type qom/object.c:782:5 #8 0x55f225b0bfe1 in object_new qom/object.c:797:12 #9 0x55f226309e0d in qmp_device_list_properties qom/qom-qmp-cmds.c:144:11 Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Edgar E. Iglesias <edgar.iglesias@amd.com> Reviewed-by: Francisco Iglesias <francisco.iglesias@amd.com> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> Message-id: 20240822162127.705879-7-peter.maydell@linaro.org --- include/hw/nvram/xlnx-versal-efuse.h | 1 + hw/nvram/xlnx-versal-efuse-ctrl.c | 6 +++--- 2 files changed, 4 insertions(+), 3 deletions(-) diff --git a/include/hw/nvram/xlnx-versal-efuse.h b/include/hw/nvram/xlnx-versal-efuse.h index XXXXXXX..XXXXXXX 100644 --- a/include/hw/nvram/xlnx-versal-efuse.h +++ b/include/hw/nvram/xlnx-versal-efuse.h @@ -XXX,XX +XXX,XX @@ struct XlnxVersalEFuseCtrl { void *extra_pg0_lock_spec; /* Opaque property */ uint32_t extra_pg0_lock_n16; + RegisterInfoArray *reg_array; uint32_t regs[XLNX_VERSAL_EFUSE_CTRL_R_MAX]; RegisterInfo regs_info[XLNX_VERSAL_EFUSE_CTRL_R_MAX]; }; diff --git a/hw/nvram/xlnx-versal-efuse-ctrl.c b/hw/nvram/xlnx-versal-efuse-ctrl.c index XXXXXXX..XXXXXXX 100644 --- a/hw/nvram/xlnx-versal-efuse-ctrl.c +++ b/hw/nvram/xlnx-versal-efuse-ctrl.c @@ -XXX,XX +XXX,XX @@ static void efuse_ctrl_init(Object *obj) { XlnxVersalEFuseCtrl *s = XLNX_VERSAL_EFUSE_CTRL(obj); SysBusDevice *sbd = SYS_BUS_DEVICE(obj); - RegisterInfoArray *reg_array; - reg_array = + s->reg_array = register_init_block32(DEVICE(obj), efuse_ctrl_regs_info, ARRAY_SIZE(efuse_ctrl_regs_info), s->regs_info, s->regs, @@ -XXX,XX +XXX,XX @@ static void efuse_ctrl_init(Object *obj) XLNX_VERSAL_EFUSE_CTRL_ERR_DEBUG, R_MAX * 4); - sysbus_init_mmio(sbd, &reg_array->mem); + sysbus_init_mmio(sbd, &s->reg_array->mem); sysbus_init_irq(sbd, &s->irq_efuse_imr); } @@ -XXX,XX +XXX,XX @@ static void efuse_ctrl_finalize(Object *obj) { XlnxVersalEFuseCtrl *s = XLNX_VERSAL_EFUSE_CTRL(obj); + register_finalize_block(s->reg_array); g_free(s->extra_pg0_lock_spec); } -- 2.34.1 In sbsa_fdt_add_gic_node() we g_strdup_printf() two nodename strings, but only free one. Since the string is actually entirely constant and we don't make any use of printf's format-string operations, we can drop the g_strdup_printf() use entirely. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Reviewed-by: Gavin Shan <gshan@redhat.com> Message-id: 20240822162323.706382-1-peter.maydell@linaro.org --- hw/arm/sbsa-ref.c | 15 ++++++--------- 1 file changed, 6 insertions(+), 9 deletions(-) diff --git a/hw/arm/sbsa-ref.c b/hw/arm/sbsa-ref.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/sbsa-ref.c +++ b/hw/arm/sbsa-ref.c @@ -XXX,XX +XXX,XX @@ static uint64_t sbsa_ref_cpu_mp_affinity(SBSAMachineState *sms, int idx) static void sbsa_fdt_add_gic_node(SBSAMachineState *sms) { - char *nodename; + const char *intc_nodename = "/intc"; + const char *its_nodename = "/intc/its"; - nodename = g_strdup_printf("/intc"); - qemu_fdt_add_subnode(sms->fdt, nodename); - qemu_fdt_setprop_sized_cells(sms->fdt, nodename, "reg", + qemu_fdt_add_subnode(sms->fdt, intc_nodename); + qemu_fdt_setprop_sized_cells(sms->fdt, intc_nodename, "reg", 2, sbsa_ref_memmap[SBSA_GIC_DIST].base, 2, sbsa_ref_memmap[SBSA_GIC_DIST].size, 2, sbsa_ref_memmap[SBSA_GIC_REDIST].base, 2, sbsa_ref_memmap[SBSA_GIC_REDIST].size); - nodename = g_strdup_printf("/intc/its"); - qemu_fdt_add_subnode(sms->fdt, nodename); - qemu_fdt_setprop_sized_cells(sms->fdt, nodename, "reg", + qemu_fdt_add_subnode(sms->fdt, its_nodename); + qemu_fdt_setprop_sized_cells(sms->fdt, its_nodename, "reg", 2, sbsa_ref_memmap[SBSA_GIC_ITS].base, 2, sbsa_ref_memmap[SBSA_GIC_ITS].size); - - g_free(nodename); } /* -- 2.34.1 In vfp.decode we have the names of the VFNMA and VFNMS instructions the wrong way around. The architecture says that bit 6 is the 'op' bit, which is 1 for VFNMA and 0 for VFNMS, but we label these two lines of decode the other way around. This doesn't cause any user-visible problem because in the handling of these functions in translate-vfp.c we give VFNMA the behaviour specified for VFNMS and vice-versa, but it's confusing when reading the code. Switch the names of the VFP VFNMA and VFNMS instructions in the decode file and flip the behaviour also. NB: the instructions VFMA and VFMS *are* decoded with op=0 for VFMA and op=1 for VFMS; the confusion probably arose because we assumed VFNMA and VFNMS to be the same way around. Resolves: https://gitlab.com/qemu-project/qemu/-/issues/2536 Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Message-id: 20240830152156.2046590-1-peter.maydell@linaro.org Reviewed-by: Richard Henderson <richard.henderson@linaro.org> --- target/arm/tcg/vfp.decode | 12 ++++++------ target/arm/tcg/translate-vfp.c | 8 ++++---- 2 files changed, 10 insertions(+), 10 deletions(-) diff --git a/target/arm/tcg/vfp.decode b/target/arm/tcg/vfp.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/vfp.decode +++ b/target/arm/tcg/vfp.decode @@ -XXX,XX +XXX,XX @@ VDIV_dp ---- 1110 1.00 .... .... 1011 .0.0 .... @vfp_dnm_d VFMA_hp ---- 1110 1.10 .... .... 1001 .0. 0 .... @vfp_dnm_s VFMS_hp ---- 1110 1.10 .... .... 1001 .1. 0 .... @vfp_dnm_s -VFNMA_hp ---- 1110 1.01 .... .... 1001 .0. 0 .... @vfp_dnm_s -VFNMS_hp ---- 1110 1.01 .... .... 1001 .1. 0 .... @vfp_dnm_s +VFNMS_hp ---- 1110 1.01 .... .... 1001 .0. 0 .... @vfp_dnm_s +VFNMA_hp ---- 1110 1.01 .... .... 1001 .1. 0 .... @vfp_dnm_s VFMA_sp ---- 1110 1.10 .... .... 1010 .0. 0 .... @vfp_dnm_s VFMS_sp ---- 1110 1.10 .... .... 1010 .1. 0 .... @vfp_dnm_s -VFNMA_sp ---- 1110 1.01 .... .... 1010 .0. 0 .... @vfp_dnm_s -VFNMS_sp ---- 1110 1.01 .... .... 1010 .1. 0 .... @vfp_dnm_s +VFNMS_sp ---- 1110 1.01 .... .... 1010 .0. 0 .... @vfp_dnm_s +VFNMA_sp ---- 1110 1.01 .... .... 1010 .1. 0 .... @vfp_dnm_s VFMA_dp ---- 1110 1.10 .... .... 1011 .0.0 .... @vfp_dnm_d VFMS_dp ---- 1110 1.10 .... .... 1011 .1.0 .... @vfp_dnm_d -VFNMA_dp ---- 1110 1.01 .... .... 1011 .0.0 .... @vfp_dnm_d -VFNMS_dp ---- 1110 1.01 .... .... 1011 .1.0 .... @vfp_dnm_d +VFNMS_dp ---- 1110 1.01 .... .... 1011 .0.0 .... @vfp_dnm_d +VFNMA_dp ---- 1110 1.01 .... .... 1011 .1.0 .... @vfp_dnm_d VMOV_imm_hp ---- 1110 1.11 .... .... 1001 0000 .... \ vd=%vd_sp imm=%vmov_imm diff --git a/target/arm/tcg/translate-vfp.c b/target/arm/tcg/translate-vfp.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-vfp.c +++ b/target/arm/tcg/translate-vfp.c @@ -XXX,XX +XXX,XX @@ static bool do_vfm_sp(DisasContext *s, arg_VFMA_sp *a, bool neg_n, bool neg_d) static bool do_vfm_dp(DisasContext *s, arg_VFMA_dp *a, bool neg_n, bool neg_d) { /* - * VFNMA : fd = muladd(-fd, fn, fm) - * VFNMS : fd = muladd(-fd, -fn, fm) + * VFNMA : fd = muladd(-fd, -fn, fm) + * VFNMS : fd = muladd(-fd, fn, fm) * VFMA : fd = muladd( fd, fn, fm) * VFMS : fd = muladd( fd, -fn, fm) * @@ -XXX,XX +XXX,XX @@ static bool do_vfm_dp(DisasContext *s, arg_VFMA_dp *a, bool neg_n, bool neg_d) #define MAKE_VFM_TRANS_FNS(PREC) \ MAKE_ONE_VFM_TRANS_FN(VFMA, PREC, false, false) \ MAKE_ONE_VFM_TRANS_FN(VFMS, PREC, true, false) \ - MAKE_ONE_VFM_TRANS_FN(VFNMA, PREC, false, true) \ - MAKE_ONE_VFM_TRANS_FN(VFNMS, PREC, true, true) + MAKE_ONE_VFM_TRANS_FN(VFNMS, PREC, false, true) \ + MAKE_ONE_VFM_TRANS_FN(VFNMA, PREC, true, true) MAKE_VFM_TRANS_FNS(hp) MAKE_VFM_TRANS_FNS(sp) -- 2.34.1 From: Sebastian Huber <sebastian.huber@embedded-brains.de> The system supports the Security Extensions (core and GIC). This change is necessary to run tests which pass on the real hardware. Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de> Reviewed-by: Edgar E. Iglesias <edgar.iglesias@amd.com> Tested-by: Edgar E. Iglesias <edgar.iglesias@amd.com> Message-id: 20240828005019.57705-1-sebastian.huber@embedded-brains.de Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- hw/arm/xilinx_zynq.c | 8 -------- 1 file changed, 8 deletions(-) diff --git a/hw/arm/xilinx_zynq.c b/hw/arm/xilinx_zynq.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/xilinx_zynq.c +++ b/hw/arm/xilinx_zynq.c @@ -XXX,XX +XXX,XX @@ static void zynq_init(MachineState *machine) for (n = 0; n < smp_cpus; n++) { Object *cpuobj = object_new(machine->cpu_type); - /* - * By default A9 CPUs have EL3 enabled. This board does not currently - * support EL3 so the CPU EL3 property is disabled before realization. - */ - if (object_property_find(cpuobj, "has_el3")) { - object_property_set_bool(cpuobj, "has_el3", false, &error_fatal); - } - object_property_set_int(cpuobj, "midr", ZYNQ_BOARD_MIDR, &error_fatal); object_property_set_int(cpuobj, "reset-cbar", MPCORE_PERIPHBASE, -- 2.34.1 From: Changbin Du <changbin.du@huawei.com> Print errors before exit. Do not exit silently. Cc: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Changbin Du <changbin.du@huawei.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Message-id: 20240903133940.3447430-1-changbin.du@huawei.com Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- hw/arm/boot.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/hw/arm/boot.c b/hw/arm/boot.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/boot.c +++ b/hw/arm/boot.c @@ -XXX,XX +XXX,XX @@ static ssize_t arm_load_elf(struct arm_boot_info *info, uint64_t *pentry, 1, data_swab, as); if (ret <= 0) { /* The header loaded but the image didn't */ + error_report("Couldn't load elf '%s': %s", + info->kernel_filename, load_elf_strerror(ret)); exit(1); } -- 2.34.1 From: Philippe Mathieu-Daudé <philmd@linaro.org> If the file is not an ELF file, arm_setup_direct_kernel_boot() falls back to try it as a uimage or an AArch64 Image file or as last resort a bare raw binary. We can discard load_elf_hdr() error and silently return. Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org> Message-id: 20240903144154.17135-1-philmd@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> --- hw/arm/boot.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/hw/arm/boot.c b/hw/arm/boot.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/boot.c +++ b/hw/arm/boot.c @@ -XXX,XX +XXX,XX @@ static ssize_t arm_load_elf(struct arm_boot_info *info, uint64_t *pentry, } elf_header; int data_swab = 0; bool big_endian; - ssize_t ret = -1; + ssize_t ret; Error *err = NULL; load_elf_hdr(info->kernel_filename, &elf_header, &elf_is64, &err); if (err) { + /* + * If the file is not an ELF file we silently return. + * The caller will fall back to try other formats. + */ error_free(err); - return ret; + return -1; } if (elf_is64) { -- 2.34.1 From: Gao Shiyuan <gaoshiyuan@baidu.com> memory_region_find() returns an MR which it is the caller's responsibility to unref, but platform_bus_map_mmio() was forgetting to do so, thus leaking the MR. Signed-off-by: Gao Shiyuan <gaoshiyuan@baidu.com> Message-id: 20240829131005.9196-1-gaoshiyuan@baidu.com Reviewed-by: Peter Maydell <peter.maydell@linaro.org> [PMM: tweaked commit message] Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- hw/core/platform-bus.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/hw/core/platform-bus.c b/hw/core/platform-bus.c index XXXXXXX..XXXXXXX 100644 --- a/hw/core/platform-bus.c +++ b/hw/core/platform-bus.c @@ -XXX,XX +XXX,XX @@ static void platform_bus_map_mmio(PlatformBusDevice *pbus, SysBusDevice *sbdev, * the target device's memory region */ for (off = 0; off < pbus->mmio_size; off += alignment) { - if (!memory_region_find(&pbus->mmio, off, size).mr) { + MemoryRegion *mr = memory_region_find(&pbus->mmio, off, size).mr; + if (!mr) { found_region = true; break; + } else { + memory_region_unref(mr); } } -- 2.34.1 v3->v4: Windows headers define an INT type which clashed with an enum value name in arm_gicv3_its.c... The following changes since commit eae587e8e3694b1aceab23239493fb4c7e1a80f5: Merge remote-tracking branch 'remotes/armbru/tags/pull-qapi-2021-09-13' into staging (2021-09-13 11:00:30 +0100) are available in the Git repository at: https://git.linaro.org/people/pmaydell/qemu-arm.git tags/pull-target-arm-20210913-3 for you to fetch changes up to 28e987a7e7edaa3ca7feeac65edca26145df8814: hw/arm/mps2.c: Mark internal-only I2C buses as 'full' (2021-09-13 21:01:08 +0100) ---------------------------------------------------------------- target-arm queue: * mark MPS2/MPS3 board-internal i2c buses as 'full' so that command line user-created devices are not plugged into them * Take an exception if PSTATE.IL is set * Support an emulated ITS in the virt board * Add support for kudo-bmc board * Probe for KVM_CAP_ARM_VM_IPA_SIZE when creating scratch VM * cadence_uart: Fix clock handling issues that prevented u-boot from running ---------------------------------------------------------------- Bin Meng (6): hw/misc: zynq_slcr: Correctly compute output clocks in the reset exit phase hw/char: cadence_uart: Disable transmit when input clock is disabled hw/char: cadence_uart: Move clock/reset check to uart_can_receive() hw/char: cadence_uart: Convert to memop_with_attrs() ops hw/char: cadence_uart: Ignore access when unclocked or in reset for uart_{read, write}() hw/char: cadence_uart: Log a guest error when device is unclocked or in reset Chris Rauer (1): hw/arm: Add support for kudo-bmc board. Marc Zyngier (1): hw/arm/virt: KVM: Probe for KVM_CAP_ARM_VM_IPA_SIZE when creating scratch VM Peter Maydell (5): target/arm: Take an exception if PSTATE.IL is set qdev: Support marking individual buses as 'full' hw/arm/mps2-tz.c: Add extra data parameter to MakeDevFn hw/arm/mps2-tz.c: Mark internal-only I2C buses as 'full' hw/arm/mps2.c: Mark internal-only I2C buses as 'full' Richard Henderson (1): target/arm: Merge disas_a64_insn into aarch64_tr_translate_insn Shashi Mallela (9): hw/intc: GICv3 ITS initial framework hw/intc: GICv3 ITS register definitions added hw/intc: GICv3 ITS command queue framework hw/intc: GICv3 ITS Command processing hw/intc: GICv3 ITS Feature enablement hw/intc: GICv3 redistributor ITS processing tests/data/acpi/virt: Add IORT files for ITS hw/arm/virt: add ITS support in virt GIC tests/data/acpi/virt: Update IORT files for ITS docs/system/arm/nuvoton.rst | 1 + hw/intc/gicv3_internal.h | 188 ++++- include/hw/arm/virt.h | 2 + include/hw/intc/arm_gicv3_common.h | 13 + include/hw/intc/arm_gicv3_its_common.h | 32 +- include/hw/qdev-core.h | 24 + target/arm/cpu.h | 1 + target/arm/kvm_arm.h | 4 +- target/arm/syndrome.h | 5 + target/arm/translate.h | 2 + hw/arm/mps2-tz.c | 92 ++- hw/arm/mps2.c | 12 +- hw/arm/npcm7xx_boards.c | 34 + hw/arm/virt.c | 29 +- hw/char/cadence_uart.c | 61 +- hw/intc/arm_gicv3.c | 14 + hw/intc/arm_gicv3_common.c | 13 + hw/intc/arm_gicv3_cpuif.c | 7 +- hw/intc/arm_gicv3_dist.c | 5 +- hw/intc/arm_gicv3_its.c | 1322 ++++++++++++++++++++++++++++++++ hw/intc/arm_gicv3_its_common.c | 7 +- hw/intc/arm_gicv3_its_kvm.c | 2 +- hw/intc/arm_gicv3_redist.c | 153 +++- hw/misc/zynq_slcr.c | 31 +- softmmu/qdev-monitor.c | 7 +- target/arm/helper-a64.c | 1 + target/arm/helper.c | 8 + target/arm/kvm.c | 7 +- target/arm/translate-a64.c | 255 +++--- target/arm/translate.c | 21 + hw/intc/meson.build | 1 + tests/data/acpi/virt/IORT | Bin 0 -> 124 bytes tests/data/acpi/virt/IORT.memhp | Bin 0 -> 124 bytes tests/data/acpi/virt/IORT.numamem | Bin 0 -> 124 bytes tests/data/acpi/virt/IORT.pxb | Bin 0 -> 124 bytes 35 files changed, 2144 insertions(+), 210 deletions(-) create mode 100644 hw/intc/arm_gicv3_its.c create mode 100644 tests/data/acpi/virt/IORT create mode 100644 tests/data/acpi/virt/IORT.memhp create mode 100644 tests/data/acpi/virt/IORT.numamem create mode 100644 tests/data/acpi/virt/IORT.pxb