:p
atchew
Login
From: Frank Chang <frank.chang@sifive.com> Support the true Zicclsm extension so that we can trap misaligned accesses when Zicclsm is disabled. To enable/disable Zicclsm, simply set zicclsm=[true|false], e.g.: -cpu rv64,zicclsm=[true|false] QEMU will raise a misaligned load/store exception when executing misaligned load/store instructions if Zicclsm is disabled. Changelog: v7: * Raise the misaligned exceptions for floating-point instructions (RVF, RVD, Zfh) when Zicclsm is disabled. * Add Zicclsm unit tests. v6: * Reorder and squash Zicclsm CPU property and RVA22U64 profile patches to make every single commit to pass `make check`. v5: * Add Zicclsm to RVA22U64 profile explicitly. * Rebase to the latest riscv-to-apply.next. v4: * Align ROM reset vector data at 8-byte aligned offsets. v3: * Enable Zicclsm for the compatible CPUs. * Rebase to the latest riscv-to-apply.next. v2: * Use (size_memop(size) | mo_endian_env(env)) to calculate MemOp. * Use (log2_esz << MO_ASHIFT) to calculate aligment MemOp for vector load/store whole register instructions. Frank Chang (8): target/riscv: Add Zicclsm CPU option target/riscv: Enable Zicclsm for the compatible CPUs hw/riscv: sifive_u: Align ROM reset vector data target/riscv: Support raising misaligned exceptions for scalar loads/stores target/riscv: Support raising misaligned exceptions for floating-point loads/stores target/riscv: Support raising misaligned exceptions for vector loads/stores target/riscv: Expose Zicclsm as a CPU property and update RVA22U64 profile tests/tcg/riscv64: Add test for Zicclsm hw/riscv/sifive_u.c | 11 +- target/riscv/cpu.c | 10 +- target/riscv/cpu_cfg_fields.h.inc | 2 + target/riscv/tcg/insn_trans/trans_rvd.c.inc | 6 + target/riscv/tcg/insn_trans/trans_rvf.c.inc | 6 + target/riscv/tcg/insn_trans/trans_rvi.c.inc | 6 + target/riscv/tcg/insn_trans/trans_rvv.c.inc | 18 +- target/riscv/tcg/insn_trans/trans_rvzfh.c.inc | 6 + target/riscv/tcg/vector_helper.c | 65 +++- tests/tcg/riscv64/Makefile.softmmu-target | 22 ++ tests/tcg/riscv64/test-zicclsm.S | 368 ++++++++++++++++++ 11 files changed, 496 insertions(+), 24 deletions(-) create mode 100644 tests/tcg/riscv64/test-zicclsm.S -- 2.43.0
From: Frank Chang <frank.chang@sifive.com> Add Zicclsm CPU option so that user can turn on/off misaligned loads and stores support. Signed-off-by: Frank Chang <frank.chang@sifive.com> Reviewed-by: Max Chou <max.chou@sifive.com> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> --- target/riscv/cpu_cfg_fields.h.inc | 2 ++ 1 file changed, 2 insertions(+) diff --git a/target/riscv/cpu_cfg_fields.h.inc b/target/riscv/cpu_cfg_fields.h.inc index XXXXXXX..XXXXXXX 100644 --- a/target/riscv/cpu_cfg_fields.h.inc +++ b/target/riscv/cpu_cfg_fields.h.inc @@ -XXX,XX +XXX,XX @@ BOOL_FIELD(has_priv_1_11) /* Always enabled for TCG if has_priv_1_11 */ BOOL_FIELD(ext_ziccrse) +BOOL_FIELD(ext_zicclsm) + /* Vendor-specific custom extensions */ BOOL_FIELD(ext_xtheadba) BOOL_FIELD(ext_xtheadbb) -- 2.43.0
From: Frank Chang <frank.chang@sifive.com> To support Zicclsm, let's enable Zicclsm for the following compatible CPUs: - Base 32 CPU (to be backward compatible) - Base 64 CPU (to be backward compatible) - XuanTie (T-Head) C908 - Tenstorrent Ascalon - Ventana Veyron V1 - XiangShan Kunminghu - MIPS P8700 (ISA doesn't include Zicclsm, but their datasheet claims that it has unaligned load/store support in hardware) Signed-off-by: Frank Chang <frank.chang@sifive.com> Acked-by: Alistair Francis <alistair.francis@wdc.com> --- target/riscv/cpu.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/target/riscv/cpu.c b/target/riscv/cpu.c index XXXXXXX..XXXXXXX 100644 --- a/target/riscv/cpu.c +++ b/target/riscv/cpu.c @@ -XXX,XX +XXX,XX @@ static const TypeInfo riscv_cpu_type_infos[] = { .cfg.ext_zicbom = true, .cfg.ext_zicbop = true, .cfg.ext_zicboz = true, + .cfg.ext_zicclsm = true, .cfg.ext_zicntr = true, .cfg.ext_zicsr = true, .cfg.ext_zifencei = true, @@ -XXX,XX +XXX,XX @@ static const TypeInfo riscv_cpu_type_infos[] = { .cfg.ext_zicbom = true, .cfg.ext_zicbop = true, .cfg.ext_zicboz = true, + .cfg.ext_zicclsm = true, .cfg.ext_zicntr = true, .cfg.ext_zicsr = true, .cfg.ext_zifencei = true, @@ -XXX,XX +XXX,XX @@ static const TypeInfo riscv_cpu_type_infos[] = { .cfg.ext_zbs = true, .cfg.ext_zkt = true, .cfg.ext_zbkc = true, + .cfg.ext_zicclsm = true, .cfg.ext_zicsr = true, .cfg.ext_zifencei = true, .cfg.ext_zihintpause = true, @@ -XXX,XX +XXX,XX @@ static const TypeInfo riscv_cpu_type_infos[] = { .cfg.ext_zicbom = true, .cfg.ext_zicbop = true, .cfg.ext_zicboz = true, + .cfg.ext_zicclsm = true, .cfg.ext_zicntr = true, .cfg.ext_zicond = true, .cfg.ext_zicsr = true, @@ -XXX,XX +XXX,XX @@ static const TypeInfo riscv_cpu_type_infos[] = { /* ISA extensions */ .cfg.mmu = true, + .cfg.ext_zicclsm = true, .cfg.ext_zifencei = true, .cfg.ext_zicsr = true, .cfg.pmp = true, @@ -XXX,XX +XXX,XX @@ static const TypeInfo riscv_cpu_type_infos[] = { * The RISC-V Instruction Set Manual: Volume I * Unprivileged Architecture */ + .cfg.ext_zicclsm = true, .cfg.ext_zicntr = true, .cfg.ext_zihpm = true, .cfg.ext_zihintntl = true, @@ -XXX,XX +XXX,XX @@ static const TypeInfo riscv_cpu_type_infos[] = { .misa_ext = RVI | RVM | RVA | RVF | RVD | RVC | RVS | RVU, .priv_spec = PRIV_VERSION_1_12_0, .cfg.max_satp_mode = VM_1_10_SV48, + .cfg.ext_zicclsm = true, .cfg.ext_zifencei = true, .cfg.ext_zicsr = true, .cfg.mmu = true, -- 2.43.0
From: Frank Chang <frank.chang@sifive.com> The SiFive U ROM reset vector data needs proper 8-byte alignment for RV64 ld instructions. The misaligned load will cause exception when Zicclsm is supported as SiFive U CPU doesn't support hardware misaligned loads and stores. Add padding to ensure start_addr and fdt_load_addr are placed at 8-byte aligned offsets and adjust the load instruction offsets to match the new data layout. Signed-off-by: Frank Chang <frank.chang@sifive.com> --- hw/riscv/sifive_u.c | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/hw/riscv/sifive_u.c b/hw/riscv/sifive_u.c index XXXXXXX..XXXXXXX 100644 --- a/hw/riscv/sifive_u.c +++ b/hw/riscv/sifive_u.c @@ -XXX,XX +XXX,XX @@ static void sifive_u_machine_init(MachineState *machine) 0, 0, 0x00028067, /* jr t0 */ + 0x00000000, /* padding for alignment */ start_addr, /* start: .dword */ start_addr_hi32, fdt_load_addr, /* fdt_laddr: .dword */ fdt_load_addr_hi32, - 0x00000000, /* fw_dyn: */ }; + if (riscv_is_32bit(&s->soc.u_cpus)) { - reset_vec[4] = 0x0202a583; /* lw a1, 32(t0) */ - reset_vec[5] = 0x0182a283; /* lw t0, 24(t0) */ + reset_vec[4] = 0x0242a583; /* lw a1, 36(t0) */ + reset_vec[5] = 0x01c2a283; /* lw t0, 28(t0) */ } else { - reset_vec[4] = 0x0202b583; /* ld a1, 32(t0) */ - reset_vec[5] = 0x0182b283; /* ld t0, 24(t0) */ + reset_vec[4] = 0x0242b583; /* ld a1, 36(t0) */ + reset_vec[5] = 0x01c2b283; /* ld t0, 28(t0) */ } -- 2.43.0
From: Frank Chang <frank.chang@sifive.com> When the Zicclsm extension is not enabled, raise misaligned load/store exceptions for misaligned accesses from scalar load/store instructions. Signed-off-by: Frank Chang <frank.chang@sifive.com> Reviewed-by: Max Chou <max.chou@sifive.com> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> --- target/riscv/tcg/insn_trans/trans_rvi.c.inc | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/target/riscv/tcg/insn_trans/trans_rvi.c.inc b/target/riscv/tcg/insn_trans/trans_rvi.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/riscv/tcg/insn_trans/trans_rvi.c.inc +++ b/target/riscv/tcg/insn_trans/trans_rvi.c.inc @@ -XXX,XX +XXX,XX @@ static bool gen_load(DisasContext *ctx, arg_lb *a, MemOp memop) if (ctx->cfg_ptr->ext_zama16b) { memop |= MO_ATOM_WITHIN16; } + if (!ctx->cfg_ptr->ext_zicclsm) { + memop |= MO_ALIGN; + } decode_save_opc(ctx, 0); if (get_xl(ctx) == MXL_RV128) { out = gen_load_i128(ctx, a, memop); @@ -XXX,XX +XXX,XX @@ static bool gen_store(DisasContext *ctx, arg_sb *a, MemOp memop) if (ctx->cfg_ptr->ext_zama16b) { memop |= MO_ATOM_WITHIN16; } + if (!ctx->cfg_ptr->ext_zicclsm) { + memop |= MO_ALIGN; + } decode_save_opc(ctx, 0); if (get_xl(ctx) == MXL_RV128) { return gen_store_i128(ctx, a, memop); -- 2.43.0
From: Frank Chang <frank.chang@sifive.com> When the Zicclsm extension is not enabled, raise misaligned load/store exceptions for misaligned accesses from floating-point load/store instructions (RVF, RVD, Zfh). Signed-off-by: Frank Chang <frank.chang@sifive.com> --- target/riscv/tcg/insn_trans/trans_rvd.c.inc | 6 ++++++ target/riscv/tcg/insn_trans/trans_rvf.c.inc | 6 ++++++ target/riscv/tcg/insn_trans/trans_rvzfh.c.inc | 6 ++++++ 3 files changed, 18 insertions(+) diff --git a/target/riscv/tcg/insn_trans/trans_rvd.c.inc b/target/riscv/tcg/insn_trans/trans_rvd.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/riscv/tcg/insn_trans/trans_rvd.c.inc +++ b/target/riscv/tcg/insn_trans/trans_rvd.c.inc @@ -XXX,XX +XXX,XX @@ static bool trans_fld(DisasContext *ctx, arg_fld *a) memop |= MO_ATOM_IFALIGN; } memop |= ctx->mo_endianness; + if (!ctx->cfg_ptr->ext_zicclsm) { + memop |= MO_ALIGN; + } decode_save_opc(ctx, 0); addr = get_address(ctx, a->rs1, a->imm); @@ -XXX,XX +XXX,XX @@ static bool trans_fsd(DisasContext *ctx, arg_fsd *a) memop |= MO_ATOM_IFALIGN; } memop |= ctx->mo_endianness; + if (!ctx->cfg_ptr->ext_zicclsm) { + memop |= MO_ALIGN; + } decode_save_opc(ctx, 0); addr = get_address(ctx, a->rs1, a->imm); diff --git a/target/riscv/tcg/insn_trans/trans_rvf.c.inc b/target/riscv/tcg/insn_trans/trans_rvf.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/riscv/tcg/insn_trans/trans_rvf.c.inc +++ b/target/riscv/tcg/insn_trans/trans_rvf.c.inc @@ -XXX,XX +XXX,XX @@ static bool trans_flw(DisasContext *ctx, arg_flw *a) if (ctx->cfg_ptr->ext_zama16b) { memop |= MO_ATOM_WITHIN16; } + if (!ctx->cfg_ptr->ext_zicclsm) { + memop |= MO_ALIGN; + } decode_save_opc(ctx, 0); addr = get_address(ctx, a->rs1, a->imm); @@ -XXX,XX +XXX,XX @@ static bool trans_fsw(DisasContext *ctx, arg_fsw *a) if (ctx->cfg_ptr->ext_zama16b) { memop |= MO_ATOM_WITHIN16; } + if (!ctx->cfg_ptr->ext_zicclsm) { + memop |= MO_ALIGN; + } decode_save_opc(ctx, 0); addr = get_address(ctx, a->rs1, a->imm); diff --git a/target/riscv/tcg/insn_trans/trans_rvzfh.c.inc b/target/riscv/tcg/insn_trans/trans_rvzfh.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/riscv/tcg/insn_trans/trans_rvzfh.c.inc +++ b/target/riscv/tcg/insn_trans/trans_rvzfh.c.inc @@ -XXX,XX +XXX,XX @@ static bool trans_flh(DisasContext *ctx, arg_flh *a) REQUIRE_ZFHMIN_OR_ZFBFMIN(ctx); memop |= ctx->mo_endianness; + if (!ctx->cfg_ptr->ext_zicclsm) { + memop |= MO_ALIGN; + } decode_save_opc(ctx, 0); t0 = get_gpr(ctx, a->rs1, EXT_NONE); if (a->imm) { @@ -XXX,XX +XXX,XX @@ static bool trans_fsh(DisasContext *ctx, arg_fsh *a) REQUIRE_ZFHMIN_OR_ZFBFMIN(ctx); memop |= ctx->mo_endianness; + if (!ctx->cfg_ptr->ext_zicclsm) { + memop |= MO_ALIGN; + } decode_save_opc(ctx, 0); t0 = get_gpr(ctx, a->rs1, EXT_NONE); if (a->imm) { -- 2.43.0
From: Frank Chang <frank.chang@sifive.com> When the Zicclsm extension is not enabled, raise misaligned load/store exceptions for misaligned accesses from vector load/store instructions. We will skip the host fast-path and fall back to the slow TLB-path to raise misaligned load/store exceptions for the misaligned accesses when Zicclsm extension is disabled. Signed-off-by: Frank Chang <frank.chang@sifive.com> Reviewed-by: Max Chou <max.chou@sifive.com> Acked-by: Alistair Francis <alistair.francis@wdc.com> --- target/riscv/tcg/insn_trans/trans_rvv.c.inc | 18 +++++- target/riscv/tcg/vector_helper.c | 65 ++++++++++++++++----- 2 files changed, 65 insertions(+), 18 deletions(-) diff --git a/target/riscv/tcg/insn_trans/trans_rvv.c.inc b/target/riscv/tcg/insn_trans/trans_rvv.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/riscv/tcg/insn_trans/trans_rvv.c.inc +++ b/target/riscv/tcg/insn_trans/trans_rvv.c.inc @@ -XXX,XX +XXX,XX @@ static bool ldst_whole_trans(uint32_t vd, uint32_t rs1, uint32_t nf, * Use the helper function if either: * - vstart is not 0. */ - bool use_helper_fn = !s->vstart_eq_zero; if (!use_helper_fn) { uint32_t size = s->cfg_ptr->vlenb * nf; TCGv_i64 t8 = tcg_temp_new_i64(); MemOp atomicity = MO_ATOM_NONE; + MemOp alignment = MO_UNALN; + + /* + * If Zicclsm is disabled, require alignment based on element size. + * Use MO_ALIGN_* based on log2_esz (0 = MO_UNALN, 1 = MO_ALIGN_2, etc). + */ + if (!s->cfg_ptr->ext_zicclsm) { + alignment = log2_esz << MO_ASHIFT; + } + if (log2_esz == 0) { atomicity = MO_ATOM_NONE; } else { atomicity = MO_ATOM_IFALIGN_PAIR; } + for (int i = 0; i < size; i += 8) { TCGv addr = get_address(s, rs1, i); if (is_load) { - tcg_gen_qemu_ld_i64(t8, addr, s->mem_idx, MO_LEUQ | atomicity); + tcg_gen_qemu_ld_i64(t8, addr, s->mem_idx, + MO_LEUQ | atomicity | alignment); tcg_gen_st_i64(t8, tcg_env, vreg_ofs(s, vd) + i); } else { tcg_gen_ld_i64(t8, tcg_env, vreg_ofs(s, vd) + i); - tcg_gen_qemu_st_i64(t8, addr, s->mem_idx, MO_LEUQ | atomicity); + tcg_gen_qemu_st_i64(t8, addr, s->mem_idx, + MO_LEUQ | atomicity | alignment); } if (i == size - 8) { tcg_gen_movi_i32(cpu_vstart, 0); diff --git a/target/riscv/tcg/vector_helper.c b/target/riscv/tcg/vector_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/riscv/tcg/vector_helper.c +++ b/target/riscv/tcg/vector_helper.c @@ -XXX,XX +XXX,XX @@ static inline void vext_set_elem_mask(void *v0, int index, ((uint64_t *)v0)[idx] = deposit64(old, pos, 1, value); } +static inline MemOpIdx vext_make_memop_idx(CPURISCVState *env, size_t size) +{ + int mmu_idx = riscv_env_mmu_index(env, false); + MemOp memop = size_memop(size) | mo_endian_env(env); + + if (!riscv_cpu_cfg(env)->ext_zicclsm) { + memop |= MO_ALIGN; + } + + return make_memop_idx(memop, mmu_idx); +} + /* elements operations for load and store */ typedef void vext_ldst_elem_fn_tlb(CPURISCVState *env, abi_ptr addr, uint32_t idx, void *vd, uintptr_t retaddr); typedef void vext_ldst_elem_fn_host(void *vd, uint32_t idx, void *host); -#define GEN_VEXT_LD_ELEM(NAME, ETYPE, H, LDSUF) \ +#define GEN_VEXT_TLB_LD_ELEM(NAME, ETYPE, H, LDSUF) \ static inline QEMU_ALWAYS_INLINE \ void NAME##_tlb(CPURISCVState *env, abi_ptr addr, \ uint32_t idx, void *vd, uintptr_t retaddr) \ { \ ETYPE *cur = ((ETYPE *)vd + H(idx)); \ - *cur = cpu_##LDSUF##_data_ra(env, addr, retaddr); \ + MemOpIdx oi = vext_make_memop_idx(env, sizeof(ETYPE)); \ + *cur = cpu_##LDSUF##_mmu(env, addr, oi, retaddr); \ } \ - \ + +#define GEN_VEXT_HOST_LD_ELEM(NAME, ETYPE, H, LDSUF) \ static inline QEMU_ALWAYS_INLINE \ void NAME##_host(void *vd, uint32_t idx, void *host) \ { \ @@ -XXX,XX +XXX,XX @@ void NAME##_host(void *vd, uint32_t idx, void *host) \ *cur = (ETYPE)LDSUF##_p(host); \ } -GEN_VEXT_LD_ELEM(lde_b, uint8_t, H1, ldub) -GEN_VEXT_LD_ELEM(lde_h, uint16_t, H2, lduw_le) -GEN_VEXT_LD_ELEM(lde_w, uint32_t, H4, ldl_le) -GEN_VEXT_LD_ELEM(lde_d, uint64_t, H8, ldq_le) +GEN_VEXT_TLB_LD_ELEM(lde_b, uint8_t, H1, ldb) +GEN_VEXT_TLB_LD_ELEM(lde_h, uint16_t, H2, ldw) +GEN_VEXT_TLB_LD_ELEM(lde_w, uint32_t, H4, ldl) +GEN_VEXT_TLB_LD_ELEM(lde_d, uint64_t, H8, ldq) -#define GEN_VEXT_ST_ELEM(NAME, ETYPE, H, STSUF) \ +GEN_VEXT_HOST_LD_ELEM(lde_b, uint8_t, H1, ldub) +GEN_VEXT_HOST_LD_ELEM(lde_h, uint16_t, H2, lduw_le) +GEN_VEXT_HOST_LD_ELEM(lde_w, uint32_t, H4, ldl_le) +GEN_VEXT_HOST_LD_ELEM(lde_d, uint64_t, H8, ldq_le) + +#define GEN_VEXT_TLB_ST_ELEM(NAME, ETYPE, H, STSUF) \ static inline QEMU_ALWAYS_INLINE \ void NAME##_tlb(CPURISCVState *env, abi_ptr addr, \ uint32_t idx, void *vd, uintptr_t retaddr) \ { \ ETYPE data = *((ETYPE *)vd + H(idx)); \ - cpu_##STSUF##_data_ra(env, addr, data, retaddr); \ + MemOpIdx oi = vext_make_memop_idx(env, sizeof(ETYPE)); \ + cpu_##STSUF##_mmu(env, addr, data, oi, retaddr); \ } \ - \ + +#define GEN_VEXT_HOST_ST_ELEM(NAME, ETYPE, H, STSUF) \ static inline QEMU_ALWAYS_INLINE \ void NAME##_host(void *vd, uint32_t idx, void *host) \ { \ @@ -XXX,XX +XXX,XX @@ void NAME##_host(void *vd, uint32_t idx, void *host) \ STSUF##_p(host, data); \ } -GEN_VEXT_ST_ELEM(ste_b, uint8_t, H1, stb) -GEN_VEXT_ST_ELEM(ste_h, uint16_t, H2, stw_le) -GEN_VEXT_ST_ELEM(ste_w, uint32_t, H4, stl_le) -GEN_VEXT_ST_ELEM(ste_d, uint64_t, H8, stq_le) +GEN_VEXT_TLB_ST_ELEM(ste_b, uint8_t, H1, stb) +GEN_VEXT_TLB_ST_ELEM(ste_h, uint16_t, H2, stw) +GEN_VEXT_TLB_ST_ELEM(ste_w, uint32_t, H4, stl) +GEN_VEXT_TLB_ST_ELEM(ste_d, uint64_t, H8, stq) + +GEN_VEXT_HOST_ST_ELEM(ste_b, uint8_t, H1, stb) +GEN_VEXT_HOST_ST_ELEM(ste_h, uint16_t, H2, stw_le) +GEN_VEXT_HOST_ST_ELEM(ste_w, uint32_t, H4, stl_le) +GEN_VEXT_HOST_ST_ELEM(ste_d, uint64_t, H8, stq_le) static inline QEMU_ALWAYS_INLINE void vext_continuous_ldst_tlb(CPURISCVState *env, vext_ldst_elem_fn_tlb *ldst_tlb, @@ -XXX,XX +XXX,XX @@ vext_page_ldst_us(CPURISCVState *env, void *vd, target_ulong addr, probe_pages(env, addr, size, ra, access_type, mmu_index, &host, &flags, true); - if (flags == 0) { + bool misaligned = addr & (esz - 1); + + /* + * Allow the host fast-pash when: + * 1. Page permission/pmp/watchpoint are checked and we have a contigous + * host mapping. + * 2. Zicclsm is enabled or load/store is not a misaligned access. + * Otherwise, we will fall back to the slow TLB-path. + */ + if (flags == 0 && (riscv_cpu_cfg(env)->ext_zicclsm || !misaligned)) { if (nf == 1) { vext_continuous_ldst_host(env, ldst_host, vd, evl, env->vstart, host, esz, is_load); -- 2.43.0
From: Frank Chang <frank.chang@sifive.com> Update Zicclsm ISA string and expose it as a CPU property to allow user to turn on/off Zicclsm extension. In addition, Zicclsm extension is mandatory for the RVA22U64 profile. Previously, Zicclsm was enabled automatically when has_priv_1_11 was true. Now that Zicclsm has been converted to an explicit CPU option, it must be explicitly added to the RVA22U64 profile's extension list to ensure the profile remains compliant with the specification. Signed-off-by: Frank Chang <frank.chang@sifive.com> --- target/riscv/cpu.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/target/riscv/cpu.c b/target/riscv/cpu.c index XXXXXXX..XXXXXXX 100644 --- a/target/riscv/cpu.c +++ b/target/riscv/cpu.c @@ -XXX,XX +XXX,XX @@ const RISCVIsaExtData isa_edata_arr[] = { ISA_EXT_DATA_ENTRY(zicboz, PRIV_VERSION_1_12_0, ext_zicboz), ISA_INTERNAL_EXT_DATA_ENTRY(ziccamoa, PRIV_VERSION_1_11_0, has_priv_1_11), ISA_INTERNAL_EXT_DATA_ENTRY(ziccif, PRIV_VERSION_1_11_0, has_priv_1_11), - ISA_INTERNAL_EXT_DATA_ENTRY(zicclsm, PRIV_VERSION_1_11_0, has_priv_1_11), + ISA_EXT_DATA_ENTRY(zicclsm, PRIV_VERSION_1_11_0, ext_zicclsm), ISA_EXT_DATA_ENTRY(ziccrse, PRIV_VERSION_1_11_0, ext_ziccrse), ISA_EXT_DATA_ENTRY(zicfilp, PRIV_VERSION_1_12_0, ext_zicfilp), ISA_EXT_DATA_ENTRY(zicfiss, PRIV_VERSION_1_13_0, ext_zicfiss), @@ -XXX,XX +XXX,XX @@ static RISCVCPUProfile RVA22U64 = { CPU_CFG_OFFSET(ext_zkt), CPU_CFG_OFFSET(ext_zicntr), CPU_CFG_OFFSET(ext_zihpm), CPU_CFG_OFFSET(ext_zicbom), CPU_CFG_OFFSET(ext_zicbop), CPU_CFG_OFFSET(ext_zicboz), + CPU_CFG_OFFSET(ext_zicclsm), /* mandatory named features for this profile */ CPU_CFG_OFFSET(ext_zic64b), -- 2.43.0
From: Frank Chang <frank.chang@sifive.com> Cover scalar, floating-point, vector, and segmented misaligned accesses with Zicclsm enabled and disabled. Clean up both generated test binaries. To build and run the tests: make -C build/tests/tcg/riscv64-softmmu \ CC=riscv64-unknown-elf-gcc LD=riscv64-unknown-elf-ld \ test-zicclsm test-zicclsm-off make -C build/tests/tcg/riscv64-softmmu \ run-test-zicclsm run-test-zicclsm-off To clean the generated binaries and objects: make -C build/tests/tcg/riscv64-softmmu clean Signed-off-by: Frank Chang <frank.chang@sifive.com> --- tests/tcg/riscv64/Makefile.softmmu-target | 22 ++ tests/tcg/riscv64/test-zicclsm.S | 368 ++++++++++++++++++++++ 2 files changed, 390 insertions(+) create mode 100644 tests/tcg/riscv64/test-zicclsm.S diff --git a/tests/tcg/riscv64/Makefile.softmmu-target b/tests/tcg/riscv64/Makefile.softmmu-target index XXXXXXX..XXXXXXX 100644 --- a/tests/tcg/riscv64/Makefile.softmmu-target +++ b/tests/tcg/riscv64/Makefile.softmmu-target @@ -XXX,XX +XXX,XX @@ comma:= , run-test-crc32: test-crc32 $(call run-test, $<, $(QEMU) -cpu rv64$(comma)xlrbr=true $(QEMU_OPTS)$<) +# Zicclsm: misaligned load/store support. Assemble one source twice: the +# default build expects every misaligned access to succeed (zicclsm=true), +# the -DZICCLSM_DISABLED build expects every one to trap (zicclsm=false). +ZICCLSM_MARCH = -march=rv64gcv_zfh +CLEANFILES += test-zicclsm test-zicclsm-off + +test-zicclsm: test-zicclsm.S $(LINK_SCRIPT) + $(CC) $(CFLAGS) $(ZICCLSM_MARCH) $< -Wa,--noexecstack -c -o test-zicclsm.o + $(LD) $(LDFLAGS) test-zicclsm.o -o $@ + +test-zicclsm-off: test-zicclsm.S $(LINK_SCRIPT) + $(CC) $(CFLAGS) $(ZICCLSM_MARCH) -DZICCLSM_DISABLED $< -Wa,--noexecstack -c -o test-zicclsm-off.o + $(LD) $(LDFLAGS) test-zicclsm-off.o -o $@ + +EXTRA_RUNS += run-test-zicclsm +run-test-zicclsm: test-zicclsm + $(call run-test, $<, $(QEMU) -cpu rv64$(comma)v=true$(comma)zfh=true$(comma)zicclsm=true $(QEMU_OPTS)$<) + +EXTRA_RUNS += run-test-zicclsm-off +run-test-zicclsm-off: test-zicclsm-off + $(call run-test, $<, $(QEMU) -cpu rv64$(comma)v=true$(comma)zfh=true$(comma)zicclsm=false $(QEMU_OPTS)$<) + # We don't currently support the multiarch system tests undefine MULTIARCH_TESTS diff --git a/tests/tcg/riscv64/test-zicclsm.S b/tests/tcg/riscv64/test-zicclsm.S new file mode 100644 index XXXXXXX..XXXXXXX --- /dev/null +++ b/tests/tcg/riscv64/test-zicclsm.S @@ -XXX,XX +XXX,XX @@ +/* + * Test the Zicclsm extension (misaligned load/store support). + * + * This single source is assembled twice: + * - test-zicclsm : run on a CPU with zicclsm=true. Every misaligned + * scalar integer, floating-point and vector + * load/store must complete WITHOUT raising a trap. + * - test-zicclsm-off : built with -DZICCLSM_DISABLED and run on a CPU with + * zicclsm=false. Every misaligned access must raise a + * misaligned load/store exception, with the correct + * mcause and mtval. + * + * Zicclsm governs all regular scalar loads/stores (integer and F/D/Zfh + * floating-point) as well as vector element loads/stores. Floating-point + * loads/stores (flh/flw/fld, fsh/fsw/fsd) are therefore exercised here. + * + * Atomic (A/Zacas/...) accesses are intentionally excluded: they always + * require natural alignment regardless of Zicclsm. Likewise cm.push/cm.pop + * (Zcmp) are excluded, as they are not regular loads/stores. + * + * Register conventions (persist across the whole test; the trap handler only + * clobbers t0-t4): + * s1 = expected mcause for the pending misaligned access + * s2 = expected mtval (the misaligned address) + * s3 = trap counter (incremented by the handler) + * s4 = base address of the aligned data buffer + * s5 = snapshot of s3 taken before an access, used to check the delta + * + * SPDX-License-Identifier: GPL-2.0-or-later + */ + + .option norelax + .option norvc + +/* RISC-V exception causes (see target/riscv/cpu_bits.h). */ +#define CAUSE_LOAD_MISALIGNED 0x4 +#define CAUSE_STORE_MISALIGNED 0x6 + +/* + * EXPECT sets up the expectation for the access that immediately follows and + * snapshots the trap counter. + * \cause = expected mcause if the access traps + * \off = byte offset from the buffer base; also the expected mtval + */ + .macro EXPECT cause, off + li s1, \cause + addi s2, s4, \off + mv s5, s3 + .endm + +/* + * CHECK validates the outcome of the preceding access. + * - When Zicclsm is disabled, exactly one trap must have fired. + * - When Zicclsm is enabled, no trap must have fired. + */ + .macro CHECK +#ifdef ZICCLSM_DISABLED + addi s5, s5, 1 + bne s3, s5, fail +#else + bne s3, s5, fail +#endif + .endm + +/* In the enabled case, also verify the value returned by scalar loads. */ + .macro CHECK_VALUE value +#ifndef ZICCLSM_DISABLED + li t0, \value + bne a2, t0, fail +#endif + .endm + +/* Verify bytes written by an enabled scalar store. */ + .macro CHECK_BYTE off, value +#ifndef ZICCLSM_DISABLED + lbu t0, \off(s4) + li t1, \value + bne t0, t1, fail +#endif + .endm + + .text + .global _start +_start: + /* Install the trap handler. */ + lla t0, trap + csrw mtvec, t0 + + /* Enable the FP (FS) and Vector (VS) unit state so F/D/V instructions + * do not trap as illegal. 0x6600 = FS[14:13]=11 | VS[10:9]=11. */ + li t0, 0x6600 + csrs mstatus, t0 + + /* Initialise persistent state. */ + li s3, 0 /* trap counter */ + lla s4, buf /* aligned buffer base */ + + /* + * ---- Scalar integer loads ---- + * lh/lhu need 2-byte alignment; lw/lwu 4-byte; ld 8-byte. + */ + EXPECT CAUSE_LOAD_MISALIGNED, 1 + lh a2, 1(s4) + CHECK + CHECK_VALUE 0x2211 + EXPECT CAUSE_LOAD_MISALIGNED, 1 + lhu a2, 1(s4) + CHECK + CHECK_VALUE 0x2211 + + EXPECT CAUSE_LOAD_MISALIGNED, 1 + lw a2, 1(s4) + CHECK + CHECK_VALUE 0x44332211 + EXPECT CAUSE_LOAD_MISALIGNED, 3 + lw a2, 3(s4) + CHECK + CHECK_VALUE 0x66554433 + EXPECT CAUSE_LOAD_MISALIGNED, 1 + lwu a2, 1(s4) + CHECK + CHECK_VALUE 0x44332211 + + EXPECT CAUSE_LOAD_MISALIGNED, 1 + ld a2, 1(s4) + CHECK + CHECK_VALUE 0x8877665544332211 + EXPECT CAUSE_LOAD_MISALIGNED, 3 + ld a2, 3(s4) + CHECK + CHECK_VALUE 0xaa99887766554433 + EXPECT CAUSE_LOAD_MISALIGNED, 7 + ld a2, 7(s4) + CHECK + CHECK_VALUE 0xeeddccbbaa998877 + + /* + * ---- Scalar integer stores ---- + */ + li t6, 0x1122334455667788 + EXPECT CAUSE_STORE_MISALIGNED, 1 + sh t6, 1(s4) + CHECK + CHECK_BYTE 1, 0x88 + CHECK_BYTE 2, 0x77 + + EXPECT CAUSE_STORE_MISALIGNED, 1 + sw t6, 1(s4) + CHECK + CHECK_BYTE 1, 0x88 + CHECK_BYTE 2, 0x77 + CHECK_BYTE 3, 0x66 + CHECK_BYTE 4, 0x55 + EXPECT CAUSE_STORE_MISALIGNED, 3 + sw t6, 3(s4) + CHECK + CHECK_BYTE 3, 0x88 + CHECK_BYTE 4, 0x77 + CHECK_BYTE 5, 0x66 + CHECK_BYTE 6, 0x55 + + EXPECT CAUSE_STORE_MISALIGNED, 1 + sd t6, 1(s4) + CHECK + CHECK_BYTE 1, 0x88 + CHECK_BYTE 2, 0x77 + CHECK_BYTE 3, 0x66 + CHECK_BYTE 4, 0x55 + CHECK_BYTE 5, 0x44 + CHECK_BYTE 6, 0x33 + CHECK_BYTE 7, 0x22 + CHECK_BYTE 8, 0x11 + EXPECT CAUSE_STORE_MISALIGNED, 7 + sd t6, 7(s4) + CHECK + CHECK_BYTE 7, 0x88 + CHECK_BYTE 8, 0x77 + CHECK_BYTE 9, 0x66 + CHECK_BYTE 10, 0x55 + CHECK_BYTE 11, 0x44 + CHECK_BYTE 12, 0x33 + CHECK_BYTE 13, 0x22 + CHECK_BYTE 14, 0x11 + + /* + * ---- Floating-point loads ---- + * flh needs 2-byte alignment; flw 4-byte; fld 8-byte. Their alignment + * is governed by Zicclsm just like the scalar integer forms. + */ + EXPECT CAUSE_LOAD_MISALIGNED, 1 + flh fa0, 1(s4) + CHECK + + EXPECT CAUSE_LOAD_MISALIGNED, 1 + flw fa0, 1(s4) + CHECK + EXPECT CAUSE_LOAD_MISALIGNED, 3 + flw fa0, 3(s4) + CHECK + + EXPECT CAUSE_LOAD_MISALIGNED, 1 + fld fa0, 1(s4) + CHECK + EXPECT CAUSE_LOAD_MISALIGNED, 7 + fld fa0, 7(s4) + CHECK + + /* + * ---- Floating-point stores ---- + */ + EXPECT CAUSE_STORE_MISALIGNED, 1 + fsh fa0, 1(s4) + CHECK + + EXPECT CAUSE_STORE_MISALIGNED, 1 + fsw fa0, 1(s4) + CHECK + EXPECT CAUSE_STORE_MISALIGNED, 3 + fsw fa0, 3(s4) + CHECK + + EXPECT CAUSE_STORE_MISALIGNED, 1 + fsd fa0, 1(s4) + CHECK + EXPECT CAUSE_STORE_MISALIGNED, 7 + fsd fa0, 7(s4) + CHECK + + /* + * ---- Vector unit-stride loads / stores ---- + * A base address that is not aligned to the element size (SEW) is + * misaligned for the first element access. + */ + vsetvli t1, x0, e16, m1, ta, ma + EXPECT CAUSE_LOAD_MISALIGNED, 1 + addi a0, s4, 1 + vle16.v v0, (a0) + CHECK + EXPECT CAUSE_STORE_MISALIGNED, 1 + addi a0, s4, 1 + vse16.v v0, (a0) + CHECK + + vsetvli t1, x0, e32, m1, ta, ma + EXPECT CAUSE_LOAD_MISALIGNED, 1 + addi a0, s4, 1 + vle32.v v0, (a0) + CHECK + EXPECT CAUSE_STORE_MISALIGNED, 3 + addi a0, s4, 3 + vse32.v v0, (a0) + CHECK + + vsetvli t1, x0, e64, m1, ta, ma + EXPECT CAUSE_LOAD_MISALIGNED, 1 + addi a0, s4, 1 + vle64.v v0, (a0) + CHECK + EXPECT CAUSE_STORE_MISALIGNED, 7 + addi a0, s4, 7 + vse64.v v0, (a0) + CHECK + + /* + * ---- Vector strided loads / stores ---- + */ + vsetvli t1, x0, e32, m1, ta, ma + li a1, 8 /* stride in bytes */ + EXPECT CAUSE_LOAD_MISALIGNED, 1 + addi a0, s4, 1 + vlse32.v v0, (a0), a1 + CHECK + EXPECT CAUSE_STORE_MISALIGNED, 1 + addi a0, s4, 1 + vsse32.v v0, (a0), a1 + CHECK + + /* ---- Vector indexed loads / stores ---- */ + /* + * Zero indices keep the first element at the deliberately misaligned base. + */ + vmv.v.i v1, 0 + EXPECT CAUSE_LOAD_MISALIGNED, 1 + addi a0, s4, 1 + vluxei32.v v0, (a0), v1 + CHECK + EXPECT CAUSE_STORE_MISALIGNED, 1 + addi a0, s4, 1 + vsuxei32.v v0, (a0), v1 + CHECK + + /* ---- Vector segmented loads / stores ---- */ + EXPECT CAUSE_LOAD_MISALIGNED, 1 + addi a0, s4, 1 + vlseg2e32.v v0, (a0) + CHECK + EXPECT CAUSE_STORE_MISALIGNED, 1 + addi a0, s4, 1 + vsseg2e32.v v0, (a0) + CHECK + + /* + * ---- Vector whole-register load ---- + * Only the whole-register *load* forms carry an element width + * (vl1re32.v => EEW=32), so only they enforce alignment when Zicclsm is + * off. The whole-register store form (vs1r.v) is defined as EEW=8 + * (byte granular) and therefore never faults on misalignment, so it is + * not exercised here. + */ + EXPECT CAUSE_LOAD_MISALIGNED, 1 + addi a0, s4, 1 + vl1re32.v v1, (a0) + CHECK + + /* Success. */ + li a0, 0 + j _exit + + /* + * Trap handler: validate mcause and mtval against the expectation, bump + * the trap counter, then skip past the faulting instruction. The + * instruction length is decoded from its low two bits (0b11 => 4 bytes, + * otherwise a 2-byte compressed instruction). + */ + .balign 4 +trap: + csrr t0, mcause + bne t0, s1, fail + csrr t1, mtval + bne t1, s2, fail + addi s3, s3, 1 + + csrr t0, mepc + lhu t2, 0(t0) + andi t3, t2, 3 + li t4, 3 + bne t3, t4, 1f + addi t0, t0, 4 /* 32-bit instruction */ + j 2f +1: + addi t0, t0, 2 /* 16-bit compressed instruction */ +2: + csrw mepc, t0 + mret + +fail: + li a0, 1 +_exit: + lla a1, semiargs + li t0, 0x20026 /* ADP_Stopped_ApplicationExit */ + sd t0, 0(a1) + sd a0, 8(a1) + li a0, 0x20 /* TARGET_SYS_EXIT_EXTENDED */ + .balign 16 + slli zero, zero, 0x1f + ebreak + srai zero, zero, 0x7 + j . + + .data + .balign 16 +semiargs: + .space 16 + .balign 64 +buf: + .byte 0x00, 0x11, 0x22, 0x33, 0x44, 0x55, 0x66, 0x77 + .byte 0x88, 0x99, 0xaa, 0xbb, 0xcc, 0xdd, 0xee, 0xff + .space 240 -- 2.43.0
From: Frank Chang <frank.chang@sifive.com> Support the true Zicclsm extension so that we can trap misaligned accesses when Zicclsm is disabled. To enable/disable Zicclsm, simply set zicclsm=[true|false], e.g.: -cpu rv64,zicclsm=[true|false] QEMU will raise a misaligned load/store exception when executing misaligned load/store instructions if Zicclsm is disabled. Changelog: v8: * Squash patch#1 into patch#2. v7: * Raise the misaligned exceptions for floating-point instructions (RVF, RVD, Zfh) when Zicclsm is disabled. * Add Zicclsm unit tests. v6: * Reorder and squash Zicclsm CPU property and RVA22U64 profile patches to make every single commit to pass `make check`. v5: * Add Zicclsm to RVA22U64 profile explicitly. * Rebase to the latest riscv-to-apply.next. v4: * Align ROM reset vector data at 8-byte aligned offsets. v3: * Enable Zicclsm for the compatible CPUs. * Rebase to the latest riscv-to-apply.next. v2: * Use (size_memop(size) | mo_endian_env(env)) to calculate MemOp. * Use (log2_esz << MO_ASHIFT) to calculate aligment MemOp for vector load/store whole register instructions. Frank Chang (7): target/riscv: Add Zicclsm CPU option and enable it for the eligible CPUs hw/riscv: sifive_u: Align ROM reset vector data target/riscv: Support raising misaligned exceptions for scalar loads/stores target/riscv: Support raising misaligned exceptions for floating-point loads/stores target/riscv: Support raising misaligned exceptions for vector loads/stores target/riscv: Expose Zicclsm as a CPU property and update RVA22U64 profile tests/tcg/riscv64: Add test for Zicclsm hw/riscv/sifive_u.c | 11 +- target/riscv/cpu.c | 10 +- target/riscv/cpu_cfg_fields.h.inc | 2 + target/riscv/tcg/insn_trans/trans_rvd.c.inc | 6 + target/riscv/tcg/insn_trans/trans_rvf.c.inc | 6 + target/riscv/tcg/insn_trans/trans_rvi.c.inc | 6 + target/riscv/tcg/insn_trans/trans_rvv.c.inc | 18 +- target/riscv/tcg/insn_trans/trans_rvzfh.c.inc | 6 + target/riscv/tcg/vector_helper.c | 65 +++- tests/tcg/riscv64/Makefile.softmmu-target | 22 ++ tests/tcg/riscv64/test-zicclsm.S | 368 ++++++++++++++++++ 11 files changed, 496 insertions(+), 24 deletions(-) create mode 100644 tests/tcg/riscv64/test-zicclsm.S -- 2.43.0
From: Frank Chang <frank.chang@sifive.com> Add Zicclsm CPU option and enable it for the following eligible CPUs: - Base 32 CPU (to be backward compatible) - Base 64 CPU (to be backward compatible) - XuanTie (T-Head) C908 - Tenstorrent Ascalon - Ventana Veyron V1 - XiangShan Kunminghu - MIPS P8700 (ISA doesn't include Zicclsm, but their datasheet claims that it has unaligned load/store support in hardware) Signed-off-by: Frank Chang <frank.chang@sifive.com> Reviewed-by: Max Chou <max.chou@sifive.com> Acked-by: Alistair Francis <alistair.francis@wdc.com> Reviewed-by: Daniel Henrique Barboza <daniel.barboza@oss.qualcomm.com> --- target/riscv/cpu.c | 7 +++++++ target/riscv/cpu_cfg_fields.h.inc | 2 ++ 2 files changed, 9 insertions(+) diff --git a/target/riscv/cpu.c b/target/riscv/cpu.c index XXXXXXX..XXXXXXX 100644 --- a/target/riscv/cpu.c +++ b/target/riscv/cpu.c @@ -XXX,XX +XXX,XX @@ static const TypeInfo riscv_cpu_type_infos[] = { .cfg.ext_zicbom = true, .cfg.ext_zicbop = true, .cfg.ext_zicboz = true, + .cfg.ext_zicclsm = true, .cfg.ext_zicntr = true, .cfg.ext_zicsr = true, .cfg.ext_zifencei = true, @@ -XXX,XX +XXX,XX @@ static const TypeInfo riscv_cpu_type_infos[] = { .cfg.ext_zicbom = true, .cfg.ext_zicbop = true, .cfg.ext_zicboz = true, + .cfg.ext_zicclsm = true, .cfg.ext_zicntr = true, .cfg.ext_zicsr = true, .cfg.ext_zifencei = true, @@ -XXX,XX +XXX,XX @@ static const TypeInfo riscv_cpu_type_infos[] = { .cfg.ext_zbs = true, .cfg.ext_zkt = true, .cfg.ext_zbkc = true, + .cfg.ext_zicclsm = true, .cfg.ext_zicsr = true, .cfg.ext_zifencei = true, .cfg.ext_zihintpause = true, @@ -XXX,XX +XXX,XX @@ static const TypeInfo riscv_cpu_type_infos[] = { .cfg.ext_zicbom = true, .cfg.ext_zicbop = true, .cfg.ext_zicboz = true, + .cfg.ext_zicclsm = true, .cfg.ext_zicntr = true, .cfg.ext_zicond = true, .cfg.ext_zicsr = true, @@ -XXX,XX +XXX,XX @@ static const TypeInfo riscv_cpu_type_infos[] = { /* ISA extensions */ .cfg.mmu = true, + .cfg.ext_zicclsm = true, .cfg.ext_zifencei = true, .cfg.ext_zicsr = true, .cfg.pmp = true, @@ -XXX,XX +XXX,XX @@ static const TypeInfo riscv_cpu_type_infos[] = { * The RISC-V Instruction Set Manual: Volume I * Unprivileged Architecture */ + .cfg.ext_zicclsm = true, .cfg.ext_zicntr = true, .cfg.ext_zihpm = true, .cfg.ext_zihintntl = true, @@ -XXX,XX +XXX,XX @@ static const TypeInfo riscv_cpu_type_infos[] = { .misa_ext = RVI | RVM | RVA | RVF | RVD | RVC | RVS | RVU, .priv_spec = PRIV_VERSION_1_12_0, .cfg.max_satp_mode = VM_1_10_SV48, + .cfg.ext_zicclsm = true, .cfg.ext_zifencei = true, .cfg.ext_zicsr = true, .cfg.mmu = true, diff --git a/target/riscv/cpu_cfg_fields.h.inc b/target/riscv/cpu_cfg_fields.h.inc index XXXXXXX..XXXXXXX 100644 --- a/target/riscv/cpu_cfg_fields.h.inc +++ b/target/riscv/cpu_cfg_fields.h.inc @@ -XXX,XX +XXX,XX @@ BOOL_FIELD(has_priv_1_11) /* Always enabled for TCG if has_priv_1_11 */ BOOL_FIELD(ext_ziccrse) +BOOL_FIELD(ext_zicclsm) + /* Vendor-specific custom extensions */ BOOL_FIELD(ext_xtheadba) BOOL_FIELD(ext_xtheadbb) -- 2.43.0
From: Frank Chang <frank.chang@sifive.com> The SiFive U ROM reset vector data needs proper 8-byte alignment for RV64 ld instructions. The misaligned load will cause exception when Zicclsm is supported as SiFive U CPU doesn't support hardware misaligned loads and stores. Add padding to ensure start_addr and fdt_load_addr are placed at 8-byte aligned offsets and adjust the load instruction offsets to match the new data layout. Signed-off-by: Frank Chang <frank.chang@sifive.com> Reviewed-by: Daniel Henrique Barboza <daniel.barboza@oss.qualcomm.com> --- hw/riscv/sifive_u.c | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/hw/riscv/sifive_u.c b/hw/riscv/sifive_u.c index XXXXXXX..XXXXXXX 100644 --- a/hw/riscv/sifive_u.c +++ b/hw/riscv/sifive_u.c @@ -XXX,XX +XXX,XX @@ static void sifive_u_machine_init(MachineState *machine) 0, 0, 0x00028067, /* jr t0 */ + 0x00000000, /* padding for alignment */ start_addr, /* start: .dword */ start_addr_hi32, fdt_load_addr, /* fdt_laddr: .dword */ fdt_load_addr_hi32, - 0x00000000, /* fw_dyn: */ }; + if (riscv_is_32bit(&s->soc.u_cpus)) { - reset_vec[4] = 0x0202a583; /* lw a1, 32(t0) */ - reset_vec[5] = 0x0182a283; /* lw t0, 24(t0) */ + reset_vec[4] = 0x0242a583; /* lw a1, 36(t0) */ + reset_vec[5] = 0x01c2a283; /* lw t0, 28(t0) */ } else { - reset_vec[4] = 0x0202b583; /* ld a1, 32(t0) */ - reset_vec[5] = 0x0182b283; /* ld t0, 24(t0) */ + reset_vec[4] = 0x0242b583; /* ld a1, 36(t0) */ + reset_vec[5] = 0x01c2b283; /* ld t0, 28(t0) */ } -- 2.43.0
From: Frank Chang <frank.chang@sifive.com> When the Zicclsm extension is not enabled, raise misaligned load/store exceptions for misaligned accesses from scalar load/store instructions. Signed-off-by: Frank Chang <frank.chang@sifive.com> Reviewed-by: Max Chou <max.chou@sifive.com> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> --- target/riscv/tcg/insn_trans/trans_rvi.c.inc | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/target/riscv/tcg/insn_trans/trans_rvi.c.inc b/target/riscv/tcg/insn_trans/trans_rvi.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/riscv/tcg/insn_trans/trans_rvi.c.inc +++ b/target/riscv/tcg/insn_trans/trans_rvi.c.inc @@ -XXX,XX +XXX,XX @@ static bool gen_load(DisasContext *ctx, arg_lb *a, MemOp memop) if (ctx->cfg_ptr->ext_zama16b) { memop |= MO_ATOM_WITHIN16; } + if (!ctx->cfg_ptr->ext_zicclsm) { + memop |= MO_ALIGN; + } decode_save_opc(ctx, 0); if (get_xl(ctx) == MXL_RV128) { out = gen_load_i128(ctx, a, memop); @@ -XXX,XX +XXX,XX @@ static bool gen_store(DisasContext *ctx, arg_sb *a, MemOp memop) if (ctx->cfg_ptr->ext_zama16b) { memop |= MO_ATOM_WITHIN16; } + if (!ctx->cfg_ptr->ext_zicclsm) { + memop |= MO_ALIGN; + } decode_save_opc(ctx, 0); if (get_xl(ctx) == MXL_RV128) { return gen_store_i128(ctx, a, memop); -- 2.43.0
From: Frank Chang <frank.chang@sifive.com> When the Zicclsm extension is not enabled, raise misaligned load/store exceptions for misaligned accesses from floating-point load/store instructions (RVF, RVD, Zfh). Signed-off-by: Frank Chang <frank.chang@sifive.com> Reviewed-by: Daniel Henrique Barboza <daniel.barboza@oss.qualcomm.com> --- target/riscv/tcg/insn_trans/trans_rvd.c.inc | 6 ++++++ target/riscv/tcg/insn_trans/trans_rvf.c.inc | 6 ++++++ target/riscv/tcg/insn_trans/trans_rvzfh.c.inc | 6 ++++++ 3 files changed, 18 insertions(+) diff --git a/target/riscv/tcg/insn_trans/trans_rvd.c.inc b/target/riscv/tcg/insn_trans/trans_rvd.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/riscv/tcg/insn_trans/trans_rvd.c.inc +++ b/target/riscv/tcg/insn_trans/trans_rvd.c.inc @@ -XXX,XX +XXX,XX @@ static bool trans_fld(DisasContext *ctx, arg_fld *a) memop |= MO_ATOM_IFALIGN; } memop |= ctx->mo_endianness; + if (!ctx->cfg_ptr->ext_zicclsm) { + memop |= MO_ALIGN; + } decode_save_opc(ctx, 0); addr = get_address(ctx, a->rs1, a->imm); @@ -XXX,XX +XXX,XX @@ static bool trans_fsd(DisasContext *ctx, arg_fsd *a) memop |= MO_ATOM_IFALIGN; } memop |= ctx->mo_endianness; + if (!ctx->cfg_ptr->ext_zicclsm) { + memop |= MO_ALIGN; + } decode_save_opc(ctx, 0); addr = get_address(ctx, a->rs1, a->imm); diff --git a/target/riscv/tcg/insn_trans/trans_rvf.c.inc b/target/riscv/tcg/insn_trans/trans_rvf.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/riscv/tcg/insn_trans/trans_rvf.c.inc +++ b/target/riscv/tcg/insn_trans/trans_rvf.c.inc @@ -XXX,XX +XXX,XX @@ static bool trans_flw(DisasContext *ctx, arg_flw *a) if (ctx->cfg_ptr->ext_zama16b) { memop |= MO_ATOM_WITHIN16; } + if (!ctx->cfg_ptr->ext_zicclsm) { + memop |= MO_ALIGN; + } decode_save_opc(ctx, 0); addr = get_address(ctx, a->rs1, a->imm); @@ -XXX,XX +XXX,XX @@ static bool trans_fsw(DisasContext *ctx, arg_fsw *a) if (ctx->cfg_ptr->ext_zama16b) { memop |= MO_ATOM_WITHIN16; } + if (!ctx->cfg_ptr->ext_zicclsm) { + memop |= MO_ALIGN; + } decode_save_opc(ctx, 0); addr = get_address(ctx, a->rs1, a->imm); diff --git a/target/riscv/tcg/insn_trans/trans_rvzfh.c.inc b/target/riscv/tcg/insn_trans/trans_rvzfh.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/riscv/tcg/insn_trans/trans_rvzfh.c.inc +++ b/target/riscv/tcg/insn_trans/trans_rvzfh.c.inc @@ -XXX,XX +XXX,XX @@ static bool trans_flh(DisasContext *ctx, arg_flh *a) REQUIRE_ZFHMIN_OR_ZFBFMIN(ctx); memop |= ctx->mo_endianness; + if (!ctx->cfg_ptr->ext_zicclsm) { + memop |= MO_ALIGN; + } decode_save_opc(ctx, 0); t0 = get_gpr(ctx, a->rs1, EXT_NONE); if (a->imm) { @@ -XXX,XX +XXX,XX @@ static bool trans_fsh(DisasContext *ctx, arg_fsh *a) REQUIRE_ZFHMIN_OR_ZFBFMIN(ctx); memop |= ctx->mo_endianness; + if (!ctx->cfg_ptr->ext_zicclsm) { + memop |= MO_ALIGN; + } decode_save_opc(ctx, 0); t0 = get_gpr(ctx, a->rs1, EXT_NONE); if (a->imm) { -- 2.43.0
From: Frank Chang <frank.chang@sifive.com> When the Zicclsm extension is not enabled, raise misaligned load/store exceptions for misaligned accesses from vector load/store instructions. We will skip the host fast-path and fall back to the slow TLB-path to raise misaligned load/store exceptions for the misaligned accesses when Zicclsm extension is disabled. Signed-off-by: Frank Chang <frank.chang@sifive.com> Reviewed-by: Max Chou <max.chou@sifive.com> Acked-by: Alistair Francis <alistair.francis@wdc.com> --- target/riscv/tcg/insn_trans/trans_rvv.c.inc | 18 +++++- target/riscv/tcg/vector_helper.c | 65 ++++++++++++++++----- 2 files changed, 65 insertions(+), 18 deletions(-) diff --git a/target/riscv/tcg/insn_trans/trans_rvv.c.inc b/target/riscv/tcg/insn_trans/trans_rvv.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/riscv/tcg/insn_trans/trans_rvv.c.inc +++ b/target/riscv/tcg/insn_trans/trans_rvv.c.inc @@ -XXX,XX +XXX,XX @@ static bool ldst_whole_trans(uint32_t vd, uint32_t rs1, uint32_t nf, * Use the helper function if either: * - vstart is not 0. */ - bool use_helper_fn = !s->vstart_eq_zero; if (!use_helper_fn) { uint32_t size = s->cfg_ptr->vlenb * nf; TCGv_i64 t8 = tcg_temp_new_i64(); MemOp atomicity = MO_ATOM_NONE; + MemOp alignment = MO_UNALN; + + /* + * If Zicclsm is disabled, require alignment based on element size. + * Use MO_ALIGN_* based on log2_esz (0 = MO_UNALN, 1 = MO_ALIGN_2, etc). + */ + if (!s->cfg_ptr->ext_zicclsm) { + alignment = log2_esz << MO_ASHIFT; + } + if (log2_esz == 0) { atomicity = MO_ATOM_NONE; } else { atomicity = MO_ATOM_IFALIGN_PAIR; } + for (int i = 0; i < size; i += 8) { TCGv addr = get_address(s, rs1, i); if (is_load) { - tcg_gen_qemu_ld_i64(t8, addr, s->mem_idx, MO_LEUQ | atomicity); + tcg_gen_qemu_ld_i64(t8, addr, s->mem_idx, + MO_LEUQ | atomicity | alignment); tcg_gen_st_i64(t8, tcg_env, vreg_ofs(s, vd) + i); } else { tcg_gen_ld_i64(t8, tcg_env, vreg_ofs(s, vd) + i); - tcg_gen_qemu_st_i64(t8, addr, s->mem_idx, MO_LEUQ | atomicity); + tcg_gen_qemu_st_i64(t8, addr, s->mem_idx, + MO_LEUQ | atomicity | alignment); } if (i == size - 8) { tcg_gen_movi_i32(cpu_vstart, 0); diff --git a/target/riscv/tcg/vector_helper.c b/target/riscv/tcg/vector_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/riscv/tcg/vector_helper.c +++ b/target/riscv/tcg/vector_helper.c @@ -XXX,XX +XXX,XX @@ static inline void vext_set_elem_mask(void *v0, int index, ((uint64_t *)v0)[idx] = deposit64(old, pos, 1, value); } +static inline MemOpIdx vext_make_memop_idx(CPURISCVState *env, size_t size) +{ + int mmu_idx = riscv_env_mmu_index(env, false); + MemOp memop = size_memop(size) | mo_endian_env(env); + + if (!riscv_cpu_cfg(env)->ext_zicclsm) { + memop |= MO_ALIGN; + } + + return make_memop_idx(memop, mmu_idx); +} + /* elements operations for load and store */ typedef void vext_ldst_elem_fn_tlb(CPURISCVState *env, abi_ptr addr, uint32_t idx, void *vd, uintptr_t retaddr); typedef void vext_ldst_elem_fn_host(void *vd, uint32_t idx, void *host); -#define GEN_VEXT_LD_ELEM(NAME, ETYPE, H, LDSUF) \ +#define GEN_VEXT_TLB_LD_ELEM(NAME, ETYPE, H, LDSUF) \ static inline QEMU_ALWAYS_INLINE \ void NAME##_tlb(CPURISCVState *env, abi_ptr addr, \ uint32_t idx, void *vd, uintptr_t retaddr) \ { \ ETYPE *cur = ((ETYPE *)vd + H(idx)); \ - *cur = cpu_##LDSUF##_data_ra(env, addr, retaddr); \ + MemOpIdx oi = vext_make_memop_idx(env, sizeof(ETYPE)); \ + *cur = cpu_##LDSUF##_mmu(env, addr, oi, retaddr); \ } \ - \ + +#define GEN_VEXT_HOST_LD_ELEM(NAME, ETYPE, H, LDSUF) \ static inline QEMU_ALWAYS_INLINE \ void NAME##_host(void *vd, uint32_t idx, void *host) \ { \ @@ -XXX,XX +XXX,XX @@ void NAME##_host(void *vd, uint32_t idx, void *host) \ *cur = (ETYPE)LDSUF##_p(host); \ } -GEN_VEXT_LD_ELEM(lde_b, uint8_t, H1, ldub) -GEN_VEXT_LD_ELEM(lde_h, uint16_t, H2, lduw_le) -GEN_VEXT_LD_ELEM(lde_w, uint32_t, H4, ldl_le) -GEN_VEXT_LD_ELEM(lde_d, uint64_t, H8, ldq_le) +GEN_VEXT_TLB_LD_ELEM(lde_b, uint8_t, H1, ldb) +GEN_VEXT_TLB_LD_ELEM(lde_h, uint16_t, H2, ldw) +GEN_VEXT_TLB_LD_ELEM(lde_w, uint32_t, H4, ldl) +GEN_VEXT_TLB_LD_ELEM(lde_d, uint64_t, H8, ldq) -#define GEN_VEXT_ST_ELEM(NAME, ETYPE, H, STSUF) \ +GEN_VEXT_HOST_LD_ELEM(lde_b, uint8_t, H1, ldub) +GEN_VEXT_HOST_LD_ELEM(lde_h, uint16_t, H2, lduw_le) +GEN_VEXT_HOST_LD_ELEM(lde_w, uint32_t, H4, ldl_le) +GEN_VEXT_HOST_LD_ELEM(lde_d, uint64_t, H8, ldq_le) + +#define GEN_VEXT_TLB_ST_ELEM(NAME, ETYPE, H, STSUF) \ static inline QEMU_ALWAYS_INLINE \ void NAME##_tlb(CPURISCVState *env, abi_ptr addr, \ uint32_t idx, void *vd, uintptr_t retaddr) \ { \ ETYPE data = *((ETYPE *)vd + H(idx)); \ - cpu_##STSUF##_data_ra(env, addr, data, retaddr); \ + MemOpIdx oi = vext_make_memop_idx(env, sizeof(ETYPE)); \ + cpu_##STSUF##_mmu(env, addr, data, oi, retaddr); \ } \ - \ + +#define GEN_VEXT_HOST_ST_ELEM(NAME, ETYPE, H, STSUF) \ static inline QEMU_ALWAYS_INLINE \ void NAME##_host(void *vd, uint32_t idx, void *host) \ { \ @@ -XXX,XX +XXX,XX @@ void NAME##_host(void *vd, uint32_t idx, void *host) \ STSUF##_p(host, data); \ } -GEN_VEXT_ST_ELEM(ste_b, uint8_t, H1, stb) -GEN_VEXT_ST_ELEM(ste_h, uint16_t, H2, stw_le) -GEN_VEXT_ST_ELEM(ste_w, uint32_t, H4, stl_le) -GEN_VEXT_ST_ELEM(ste_d, uint64_t, H8, stq_le) +GEN_VEXT_TLB_ST_ELEM(ste_b, uint8_t, H1, stb) +GEN_VEXT_TLB_ST_ELEM(ste_h, uint16_t, H2, stw) +GEN_VEXT_TLB_ST_ELEM(ste_w, uint32_t, H4, stl) +GEN_VEXT_TLB_ST_ELEM(ste_d, uint64_t, H8, stq) + +GEN_VEXT_HOST_ST_ELEM(ste_b, uint8_t, H1, stb) +GEN_VEXT_HOST_ST_ELEM(ste_h, uint16_t, H2, stw_le) +GEN_VEXT_HOST_ST_ELEM(ste_w, uint32_t, H4, stl_le) +GEN_VEXT_HOST_ST_ELEM(ste_d, uint64_t, H8, stq_le) static inline QEMU_ALWAYS_INLINE void vext_continuous_ldst_tlb(CPURISCVState *env, vext_ldst_elem_fn_tlb *ldst_tlb, @@ -XXX,XX +XXX,XX @@ vext_page_ldst_us(CPURISCVState *env, void *vd, target_ulong addr, probe_pages(env, addr, size, ra, access_type, mmu_index, &host, &flags, true); - if (flags == 0) { + bool misaligned = addr & (esz - 1); + + /* + * Allow the host fast-pash when: + * 1. Page permission/pmp/watchpoint are checked and we have a contigous + * host mapping. + * 2. Zicclsm is enabled or load/store is not a misaligned access. + * Otherwise, we will fall back to the slow TLB-path. + */ + if (flags == 0 && (riscv_cpu_cfg(env)->ext_zicclsm || !misaligned)) { if (nf == 1) { vext_continuous_ldst_host(env, ldst_host, vd, evl, env->vstart, host, esz, is_load); -- 2.43.0
From: Frank Chang <frank.chang@sifive.com> Update Zicclsm ISA string and expose it as a CPU property to allow user to turn on/off Zicclsm extension. In addition, Zicclsm extension is mandatory for the RVA22U64 profile. Previously, Zicclsm was enabled automatically when has_priv_1_11 was true. Now that Zicclsm has been converted to an explicit CPU option, it must be explicitly added to the RVA22U64 profile's extension list to ensure the profile remains compliant with the specification. Signed-off-by: Frank Chang <frank.chang@sifive.com> Reviewed-by: Daniel Henrique Barboza <daniel.barboza@oss.qualcomm.com> --- target/riscv/cpu.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/target/riscv/cpu.c b/target/riscv/cpu.c index XXXXXXX..XXXXXXX 100644 --- a/target/riscv/cpu.c +++ b/target/riscv/cpu.c @@ -XXX,XX +XXX,XX @@ const RISCVIsaExtData isa_edata_arr[] = { ISA_EXT_DATA_ENTRY(zicboz, PRIV_VERSION_1_12_0, ext_zicboz), ISA_INTERNAL_EXT_DATA_ENTRY(ziccamoa, PRIV_VERSION_1_11_0, has_priv_1_11), ISA_INTERNAL_EXT_DATA_ENTRY(ziccif, PRIV_VERSION_1_11_0, has_priv_1_11), - ISA_INTERNAL_EXT_DATA_ENTRY(zicclsm, PRIV_VERSION_1_11_0, has_priv_1_11), + ISA_EXT_DATA_ENTRY(zicclsm, PRIV_VERSION_1_11_0, ext_zicclsm), ISA_EXT_DATA_ENTRY(ziccrse, PRIV_VERSION_1_11_0, ext_ziccrse), ISA_EXT_DATA_ENTRY(zicfilp, PRIV_VERSION_1_12_0, ext_zicfilp), ISA_EXT_DATA_ENTRY(zicfiss, PRIV_VERSION_1_13_0, ext_zicfiss), @@ -XXX,XX +XXX,XX @@ static RISCVCPUProfile RVA22U64 = { CPU_CFG_OFFSET(ext_zkt), CPU_CFG_OFFSET(ext_zicntr), CPU_CFG_OFFSET(ext_zihpm), CPU_CFG_OFFSET(ext_zicbom), CPU_CFG_OFFSET(ext_zicbop), CPU_CFG_OFFSET(ext_zicboz), + CPU_CFG_OFFSET(ext_zicclsm), /* mandatory named features for this profile */ CPU_CFG_OFFSET(ext_zic64b), -- 2.43.0
From: Frank Chang <frank.chang@sifive.com> Cover scalar, floating-point, vector, and segmented misaligned accesses with Zicclsm enabled and disabled. Clean up both generated test binaries. To build and run the tests: make -C build/tests/tcg/riscv64-softmmu \ CC=riscv64-unknown-elf-gcc LD=riscv64-unknown-elf-ld \ test-zicclsm test-zicclsm-off make -C build/tests/tcg/riscv64-softmmu \ run-test-zicclsm run-test-zicclsm-off To clean the generated binaries and objects: make -C build/tests/tcg/riscv64-softmmu clean Signed-off-by: Frank Chang <frank.chang@sifive.com> Reviewed-by: Daniel Henrique Barboza <daniel.barboza@oss.qualcomm.com> --- tests/tcg/riscv64/Makefile.softmmu-target | 22 ++ tests/tcg/riscv64/test-zicclsm.S | 368 ++++++++++++++++++++++ 2 files changed, 390 insertions(+) create mode 100644 tests/tcg/riscv64/test-zicclsm.S diff --git a/tests/tcg/riscv64/Makefile.softmmu-target b/tests/tcg/riscv64/Makefile.softmmu-target index XXXXXXX..XXXXXXX 100644 --- a/tests/tcg/riscv64/Makefile.softmmu-target +++ b/tests/tcg/riscv64/Makefile.softmmu-target @@ -XXX,XX +XXX,XX @@ comma:= , run-test-crc32: test-crc32 $(call run-test, $<, $(QEMU) -cpu rv64$(comma)xlrbr=true $(QEMU_OPTS)$<) +# Zicclsm: misaligned load/store support. Assemble one source twice: the +# default build expects every misaligned access to succeed (zicclsm=true), +# the -DZICCLSM_DISABLED build expects every one to trap (zicclsm=false). +ZICCLSM_MARCH = -march=rv64gcv_zfh +CLEANFILES += test-zicclsm test-zicclsm-off + +test-zicclsm: test-zicclsm.S $(LINK_SCRIPT) + $(CC) $(CFLAGS) $(ZICCLSM_MARCH) $< -Wa,--noexecstack -c -o test-zicclsm.o + $(LD) $(LDFLAGS) test-zicclsm.o -o $@ + +test-zicclsm-off: test-zicclsm.S $(LINK_SCRIPT) + $(CC) $(CFLAGS) $(ZICCLSM_MARCH) -DZICCLSM_DISABLED $< -Wa,--noexecstack -c -o test-zicclsm-off.o + $(LD) $(LDFLAGS) test-zicclsm-off.o -o $@ + +EXTRA_RUNS += run-test-zicclsm +run-test-zicclsm: test-zicclsm + $(call run-test, $<, $(QEMU) -cpu rv64$(comma)v=true$(comma)zfh=true$(comma)zicclsm=true $(QEMU_OPTS)$<) + +EXTRA_RUNS += run-test-zicclsm-off +run-test-zicclsm-off: test-zicclsm-off + $(call run-test, $<, $(QEMU) -cpu rv64$(comma)v=true$(comma)zfh=true$(comma)zicclsm=false $(QEMU_OPTS)$<) + # We don't currently support the multiarch system tests undefine MULTIARCH_TESTS diff --git a/tests/tcg/riscv64/test-zicclsm.S b/tests/tcg/riscv64/test-zicclsm.S new file mode 100644 index XXXXXXX..XXXXXXX --- /dev/null +++ b/tests/tcg/riscv64/test-zicclsm.S @@ -XXX,XX +XXX,XX @@ +/* + * Test the Zicclsm extension (misaligned load/store support). + * + * This single source is assembled twice: + * - test-zicclsm : run on a CPU with zicclsm=true. Every misaligned + * scalar integer, floating-point and vector + * load/store must complete WITHOUT raising a trap. + * - test-zicclsm-off : built with -DZICCLSM_DISABLED and run on a CPU with + * zicclsm=false. Every misaligned access must raise a + * misaligned load/store exception, with the correct + * mcause and mtval. + * + * Zicclsm governs all regular scalar loads/stores (integer and F/D/Zfh + * floating-point) as well as vector element loads/stores. Floating-point + * loads/stores (flh/flw/fld, fsh/fsw/fsd) are therefore exercised here. + * + * Atomic (A/Zacas/...) accesses are intentionally excluded: they always + * require natural alignment regardless of Zicclsm. Likewise cm.push/cm.pop + * (Zcmp) are excluded, as they are not regular loads/stores. + * + * Register conventions (persist across the whole test; the trap handler only + * clobbers t0-t4): + * s1 = expected mcause for the pending misaligned access + * s2 = expected mtval (the misaligned address) + * s3 = trap counter (incremented by the handler) + * s4 = base address of the aligned data buffer + * s5 = snapshot of s3 taken before an access, used to check the delta + * + * SPDX-License-Identifier: GPL-2.0-or-later + */ + + .option norelax + .option norvc + +/* RISC-V exception causes (see target/riscv/cpu_bits.h). */ +#define CAUSE_LOAD_MISALIGNED 0x4 +#define CAUSE_STORE_MISALIGNED 0x6 + +/* + * EXPECT sets up the expectation for the access that immediately follows and + * snapshots the trap counter. + * \cause = expected mcause if the access traps + * \off = byte offset from the buffer base; also the expected mtval + */ + .macro EXPECT cause, off + li s1, \cause + addi s2, s4, \off + mv s5, s3 + .endm + +/* + * CHECK validates the outcome of the preceding access. + * - When Zicclsm is disabled, exactly one trap must have fired. + * - When Zicclsm is enabled, no trap must have fired. + */ + .macro CHECK +#ifdef ZICCLSM_DISABLED + addi s5, s5, 1 + bne s3, s5, fail +#else + bne s3, s5, fail +#endif + .endm + +/* In the enabled case, also verify the value returned by scalar loads. */ + .macro CHECK_VALUE value +#ifndef ZICCLSM_DISABLED + li t0, \value + bne a2, t0, fail +#endif + .endm + +/* Verify bytes written by an enabled scalar store. */ + .macro CHECK_BYTE off, value +#ifndef ZICCLSM_DISABLED + lbu t0, \off(s4) + li t1, \value + bne t0, t1, fail +#endif + .endm + + .text + .global _start +_start: + /* Install the trap handler. */ + lla t0, trap + csrw mtvec, t0 + + /* Enable the FP (FS) and Vector (VS) unit state so F/D/V instructions + * do not trap as illegal. 0x6600 = FS[14:13]=11 | VS[10:9]=11. */ + li t0, 0x6600 + csrs mstatus, t0 + + /* Initialise persistent state. */ + li s3, 0 /* trap counter */ + lla s4, buf /* aligned buffer base */ + + /* + * ---- Scalar integer loads ---- + * lh/lhu need 2-byte alignment; lw/lwu 4-byte; ld 8-byte. + */ + EXPECT CAUSE_LOAD_MISALIGNED, 1 + lh a2, 1(s4) + CHECK + CHECK_VALUE 0x2211 + EXPECT CAUSE_LOAD_MISALIGNED, 1 + lhu a2, 1(s4) + CHECK + CHECK_VALUE 0x2211 + + EXPECT CAUSE_LOAD_MISALIGNED, 1 + lw a2, 1(s4) + CHECK + CHECK_VALUE 0x44332211 + EXPECT CAUSE_LOAD_MISALIGNED, 3 + lw a2, 3(s4) + CHECK + CHECK_VALUE 0x66554433 + EXPECT CAUSE_LOAD_MISALIGNED, 1 + lwu a2, 1(s4) + CHECK + CHECK_VALUE 0x44332211 + + EXPECT CAUSE_LOAD_MISALIGNED, 1 + ld a2, 1(s4) + CHECK + CHECK_VALUE 0x8877665544332211 + EXPECT CAUSE_LOAD_MISALIGNED, 3 + ld a2, 3(s4) + CHECK + CHECK_VALUE 0xaa99887766554433 + EXPECT CAUSE_LOAD_MISALIGNED, 7 + ld a2, 7(s4) + CHECK + CHECK_VALUE 0xeeddccbbaa998877 + + /* + * ---- Scalar integer stores ---- + */ + li t6, 0x1122334455667788 + EXPECT CAUSE_STORE_MISALIGNED, 1 + sh t6, 1(s4) + CHECK + CHECK_BYTE 1, 0x88 + CHECK_BYTE 2, 0x77 + + EXPECT CAUSE_STORE_MISALIGNED, 1 + sw t6, 1(s4) + CHECK + CHECK_BYTE 1, 0x88 + CHECK_BYTE 2, 0x77 + CHECK_BYTE 3, 0x66 + CHECK_BYTE 4, 0x55 + EXPECT CAUSE_STORE_MISALIGNED, 3 + sw t6, 3(s4) + CHECK + CHECK_BYTE 3, 0x88 + CHECK_BYTE 4, 0x77 + CHECK_BYTE 5, 0x66 + CHECK_BYTE 6, 0x55 + + EXPECT CAUSE_STORE_MISALIGNED, 1 + sd t6, 1(s4) + CHECK + CHECK_BYTE 1, 0x88 + CHECK_BYTE 2, 0x77 + CHECK_BYTE 3, 0x66 + CHECK_BYTE 4, 0x55 + CHECK_BYTE 5, 0x44 + CHECK_BYTE 6, 0x33 + CHECK_BYTE 7, 0x22 + CHECK_BYTE 8, 0x11 + EXPECT CAUSE_STORE_MISALIGNED, 7 + sd t6, 7(s4) + CHECK + CHECK_BYTE 7, 0x88 + CHECK_BYTE 8, 0x77 + CHECK_BYTE 9, 0x66 + CHECK_BYTE 10, 0x55 + CHECK_BYTE 11, 0x44 + CHECK_BYTE 12, 0x33 + CHECK_BYTE 13, 0x22 + CHECK_BYTE 14, 0x11 + + /* + * ---- Floating-point loads ---- + * flh needs 2-byte alignment; flw 4-byte; fld 8-byte. Their alignment + * is governed by Zicclsm just like the scalar integer forms. + */ + EXPECT CAUSE_LOAD_MISALIGNED, 1 + flh fa0, 1(s4) + CHECK + + EXPECT CAUSE_LOAD_MISALIGNED, 1 + flw fa0, 1(s4) + CHECK + EXPECT CAUSE_LOAD_MISALIGNED, 3 + flw fa0, 3(s4) + CHECK + + EXPECT CAUSE_LOAD_MISALIGNED, 1 + fld fa0, 1(s4) + CHECK + EXPECT CAUSE_LOAD_MISALIGNED, 7 + fld fa0, 7(s4) + CHECK + + /* + * ---- Floating-point stores ---- + */ + EXPECT CAUSE_STORE_MISALIGNED, 1 + fsh fa0, 1(s4) + CHECK + + EXPECT CAUSE_STORE_MISALIGNED, 1 + fsw fa0, 1(s4) + CHECK + EXPECT CAUSE_STORE_MISALIGNED, 3 + fsw fa0, 3(s4) + CHECK + + EXPECT CAUSE_STORE_MISALIGNED, 1 + fsd fa0, 1(s4) + CHECK + EXPECT CAUSE_STORE_MISALIGNED, 7 + fsd fa0, 7(s4) + CHECK + + /* + * ---- Vector unit-stride loads / stores ---- + * A base address that is not aligned to the element size (SEW) is + * misaligned for the first element access. + */ + vsetvli t1, x0, e16, m1, ta, ma + EXPECT CAUSE_LOAD_MISALIGNED, 1 + addi a0, s4, 1 + vle16.v v0, (a0) + CHECK + EXPECT CAUSE_STORE_MISALIGNED, 1 + addi a0, s4, 1 + vse16.v v0, (a0) + CHECK + + vsetvli t1, x0, e32, m1, ta, ma + EXPECT CAUSE_LOAD_MISALIGNED, 1 + addi a0, s4, 1 + vle32.v v0, (a0) + CHECK + EXPECT CAUSE_STORE_MISALIGNED, 3 + addi a0, s4, 3 + vse32.v v0, (a0) + CHECK + + vsetvli t1, x0, e64, m1, ta, ma + EXPECT CAUSE_LOAD_MISALIGNED, 1 + addi a0, s4, 1 + vle64.v v0, (a0) + CHECK + EXPECT CAUSE_STORE_MISALIGNED, 7 + addi a0, s4, 7 + vse64.v v0, (a0) + CHECK + + /* + * ---- Vector strided loads / stores ---- + */ + vsetvli t1, x0, e32, m1, ta, ma + li a1, 8 /* stride in bytes */ + EXPECT CAUSE_LOAD_MISALIGNED, 1 + addi a0, s4, 1 + vlse32.v v0, (a0), a1 + CHECK + EXPECT CAUSE_STORE_MISALIGNED, 1 + addi a0, s4, 1 + vsse32.v v0, (a0), a1 + CHECK + + /* ---- Vector indexed loads / stores ---- */ + /* + * Zero indices keep the first element at the deliberately misaligned base. + */ + vmv.v.i v1, 0 + EXPECT CAUSE_LOAD_MISALIGNED, 1 + addi a0, s4, 1 + vluxei32.v v0, (a0), v1 + CHECK + EXPECT CAUSE_STORE_MISALIGNED, 1 + addi a0, s4, 1 + vsuxei32.v v0, (a0), v1 + CHECK + + /* ---- Vector segmented loads / stores ---- */ + EXPECT CAUSE_LOAD_MISALIGNED, 1 + addi a0, s4, 1 + vlseg2e32.v v0, (a0) + CHECK + EXPECT CAUSE_STORE_MISALIGNED, 1 + addi a0, s4, 1 + vsseg2e32.v v0, (a0) + CHECK + + /* + * ---- Vector whole-register load ---- + * Only the whole-register *load* forms carry an element width + * (vl1re32.v => EEW=32), so only they enforce alignment when Zicclsm is + * off. The whole-register store form (vs1r.v) is defined as EEW=8 + * (byte granular) and therefore never faults on misalignment, so it is + * not exercised here. + */ + EXPECT CAUSE_LOAD_MISALIGNED, 1 + addi a0, s4, 1 + vl1re32.v v1, (a0) + CHECK + + /* Success. */ + li a0, 0 + j _exit + + /* + * Trap handler: validate mcause and mtval against the expectation, bump + * the trap counter, then skip past the faulting instruction. The + * instruction length is decoded from its low two bits (0b11 => 4 bytes, + * otherwise a 2-byte compressed instruction). + */ + .balign 4 +trap: + csrr t0, mcause + bne t0, s1, fail + csrr t1, mtval + bne t1, s2, fail + addi s3, s3, 1 + + csrr t0, mepc + lhu t2, 0(t0) + andi t3, t2, 3 + li t4, 3 + bne t3, t4, 1f + addi t0, t0, 4 /* 32-bit instruction */ + j 2f +1: + addi t0, t0, 2 /* 16-bit compressed instruction */ +2: + csrw mepc, t0 + mret + +fail: + li a0, 1 +_exit: + lla a1, semiargs + li t0, 0x20026 /* ADP_Stopped_ApplicationExit */ + sd t0, 0(a1) + sd a0, 8(a1) + li a0, 0x20 /* TARGET_SYS_EXIT_EXTENDED */ + .balign 16 + slli zero, zero, 0x1f + ebreak + srai zero, zero, 0x7 + j . + + .data + .balign 16 +semiargs: + .space 16 + .balign 64 +buf: + .byte 0x00, 0x11, 0x22, 0x33, 0x44, 0x55, 0x66, 0x77 + .byte 0x88, 0x99, 0xaa, 0xbb, 0xcc, 0xdd, 0xee, 0xff + .space 240 -- 2.43.0