Arm queue; the big bit here is RTH's MTE for user-mode series. -- PMM The following changes since commit 83339e21d05c824ebc9131d644f25c23d0e41ecf: Merge remote-tracking branch 'remotes/stefanha-gitlab/tags/block-pull-request' into staging (2021-02-10 15:42:20 +0000) are available in the Git repository at: https://git.linaro.org/people/pmaydell/qemu-arm.git tags/pull-target-arm-20210211 for you to fetch changes up to 5213c78932ecf4bae18d62baf8735724e25fb478: target/arm: Correctly initialize MDCR_EL2.HPMN (2021-02-11 11:50:16 +0000) ---------------------------------------------------------------- target-arm queue: * Correctly initialize MDCR_EL2.HPMN * versal: Use nr_apu_cpus in favor of hard coding 2 * npcm7xx: Add ethernet device * Enable ARMv8.4-MemTag for user-mode emulation * accel/tcg: Add URL of clang bug to comment about our workaround * Add support for FEAT_DIT, Data Independent Timing * Remove GPIO from unimplemented NPCM7XX * Fix SCR RES1 handling * Don't migrate CPUARMState.features ---------------------------------------------------------------- Aaron Lindsay (1): target/arm: Don't migrate CPUARMState.features Daniel Müller (1): target/arm: Correctly initialize MDCR_EL2.HPMN Doug Evans (3): hw/net: Add npcm7xx emc model hw/arm: Add npcm7xx emc model tests/qtests: Add npcm7xx emc model test Edgar E. Iglesias (1): hw/arm: versal: Use nr_apu_cpus in favor of hard coding 2 Hao Wu (1): hw/arm: Remove GPIO from unimplemented NPCM7XX Mike Nawrocki (1): target/arm: Fix SCR RES1 handling Peter Maydell (2): arm: Update infocenter.arm.com URLs accel/tcg: Add URL of clang bug to comment about our workaround Rebecca Cran (4): target/arm: Add support for FEAT_DIT, Data Independent Timing target/arm: Support AA32 DIT by moving PSTATE_SS from cpsr into env->pstate target/arm: Set ID_AA64PFR0.DIT and ID_PFR0.DIT to 1 for "max" AA64 CPU target/arm: Set ID_PFR0.DIT to 1 for "max" 32-bit CPU Richard Henderson (31): tcg: Introduce target-specific page data for user-only linux-user: Introduce PAGE_ANON exec: Use uintptr_t for guest_base exec: Use uintptr_t in cpu_ldst.h exec: Improve types for guest_addr_valid linux-user: Check for overflow in access_ok linux-user: Tidy VERIFY_READ/VERIFY_WRITE bsd-user: Tidy VERIFY_READ/VERIFY_WRITE linux-user: Do not use guest_addr_valid for h2g_valid linux-user: Fix guest_addr_valid vs reserved_va exec: Introduce cpu_untagged_addr exec: Use cpu_untagged_addr in g2h; split out g2h_untagged linux-user: Explicitly untag memory management syscalls linux-user: Use guest_range_valid in access_ok exec: Rename guest_{addr,range}_valid to *_untagged linux-user: Use cpu_untagged_addr in access_ok; split out *_untagged linux-user: Move lock_user et al out of line linux-user: Fix types in uaccess.c linux-user: Handle tags in lock_user/unlock_user linux-user/aarch64: Implement PR_TAGGED_ADDR_ENABLE target/arm: Improve gen_top_byte_ignore target/arm: Use the proper TBI settings for linux-user linux-user/aarch64: Implement PR_MTE_TCF and PR_MTE_TAG linux-user/aarch64: Implement PROT_MTE target/arm: Split out syndrome.h from internals.h linux-user/aarch64: Pass syndrome to EXC_*_ABORT linux-user/aarch64: Signal SEGV_MTESERR for sync tag check fault linux-user/aarch64: Signal SEGV_MTEAERR for async tag check error target/arm: Add allocation tag storage for user mode target/arm: Enable MTE for user-only tests/tcg/aarch64: Add mte smoke tests docs/system/arm/nuvoton.rst | 3 +- bsd-user/qemu.h | 9 +- include/exec/cpu-all.h | 47 +- include/exec/cpu_ldst.h | 39 +- include/exec/exec-all.h | 2 +- include/hw/arm/npcm7xx.h | 2 + include/hw/dma/pl080.h | 7 +- include/hw/misc/arm_integrator_debug.h | 2 +- include/hw/net/npcm7xx_emc.h | 286 +++++++++++ include/hw/ssi/pl022.h | 5 +- linux-user/aarch64/target_signal.h | 3 + linux-user/aarch64/target_syscall.h | 13 + linux-user/qemu.h | 76 +-- linux-user/syscall_defs.h | 1 + target/arm/cpu-param.h | 3 + target/arm/cpu.h | 49 ++ target/arm/internals.h | 255 +--------- target/arm/syndrome.h | 273 +++++++++++ tests/tcg/aarch64/mte.h | 60 +++ accel/tcg/cpu-exec.c | 25 +- accel/tcg/translate-all.c | 32 +- accel/tcg/user-exec.c | 51 +- bsd-user/main.c | 4 +- hw/arm/aspeed_ast2600.c | 2 +- hw/arm/musca.c | 4 +- hw/arm/npcm7xx.c | 58 ++- hw/arm/xlnx-versal.c | 4 +- hw/misc/arm_integrator_debug.c | 2 +- hw/net/npcm7xx_emc.c | 857 +++++++++++++++++++++++++++++++++ hw/timer/arm_timer.c | 7 +- linux-user/aarch64/cpu_loop.c | 38 +- linux-user/elfload.c | 18 +- linux-user/flatload.c | 2 +- linux-user/hppa/cpu_loop.c | 39 +- linux-user/i386/cpu_loop.c | 6 +- linux-user/i386/signal.c | 5 +- linux-user/main.c | 4 +- linux-user/mmap.c | 86 ++-- linux-user/ppc/signal.c | 4 +- linux-user/syscall.c | 165 +++++-- linux-user/uaccess.c | 82 +++- target/arm/cpu.c | 29 +- target/arm/cpu64.c | 5 + target/arm/helper-a64.c | 31 +- target/arm/helper.c | 71 ++- target/arm/machine.c | 2 +- target/arm/mte_helper.c | 39 +- target/arm/op_helper.c | 9 +- target/arm/tlb_helper.c | 15 +- target/arm/translate-a64.c | 37 +- target/hppa/op_helper.c | 2 +- target/i386/tcg/mem_helper.c | 2 +- target/s390x/mem_helper.c | 4 +- tests/qtest/npcm7xx_emc-test.c | 812 +++++++++++++++++++++++++++++++ tests/tcg/aarch64/mte-1.c | 28 ++ tests/tcg/aarch64/mte-2.c | 45 ++ tests/tcg/aarch64/mte-3.c | 51 ++ tests/tcg/aarch64/mte-4.c | 45 ++ tests/tcg/aarch64/pauth-2.c | 1 - hw/net/meson.build | 1 + hw/net/trace-events | 17 + tests/qtest/meson.build | 1 + tests/tcg/aarch64/Makefile.target | 6 + tests/tcg/configure.sh | 4 + 64 files changed, 3312 insertions(+), 575 deletions(-) create mode 100644 include/hw/net/npcm7xx_emc.h create mode 100644 target/arm/syndrome.h create mode 100644 tests/tcg/aarch64/mte.h create mode 100644 hw/net/npcm7xx_emc.c create mode 100644 tests/qtest/npcm7xx_emc-test.c create mode 100644 tests/tcg/aarch64/mte-1.c create mode 100644 tests/tcg/aarch64/mte-2.c create mode 100644 tests/tcg/aarch64/mte-3.c create mode 100644 tests/tcg/aarch64/mte-4.c From: Aaron Lindsay <aaron@os.amperecomputing.com> As feature flags are added or removed, the meanings of bits in the `features` field can change between QEMU versions, causing migration failures. Additionally, migrating the field is not useful because it is a constant function of the CPU being used. Fixes: LP:1914696 Signed-off-by: Aaron Lindsay <aaron@os.amperecomputing.com> Suggested-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Andrew Jones <drjones@redhat.com> Tested-by: Andrew Jones <drjones@redhat.com> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/machine.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/target/arm/machine.c b/target/arm/machine.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/machine.c +++ b/target/arm/machine.c @@ -XXX,XX +XXX,XX @@ const VMStateDescription vmstate_arm_cpu = { VMSTATE_UINT64(env.exclusive_addr, ARMCPU), VMSTATE_UINT64(env.exclusive_val, ARMCPU), VMSTATE_UINT64(env.exclusive_high, ARMCPU), - VMSTATE_UINT64(env.features, ARMCPU), + VMSTATE_UNUSED(sizeof(uint64_t)), VMSTATE_UINT32(env.exception.syndrome, ARMCPU), VMSTATE_UINT32(env.exception.fsr, ARMCPU), VMSTATE_UINT64(env.exception.vaddress, ARMCPU), -- 2.20.1 From: Mike Nawrocki <michael.nawrocki@gtri.gatech.edu> The FW and AW bits of SCR_EL3 are RES1 only in some contexts. Force them to 1 only when there is no support for AArch32 at EL1 or above. The reset value will be 0x30 only if the CPU is AArch64-only; if there is support for AArch32 at EL1 or above, it will be reset to 0. Also adds helper function isar_feature_aa64_aa32_el1 to check if AArch32 is supported at EL1 or above. Signed-off-by: Mike Nawrocki <michael.nawrocki@gtri.gatech.edu> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20210203165552.16306-2-michael.nawrocki@gtri.gatech.edu Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/cpu.h | 5 +++++ target/arm/helper.c | 16 ++++++++++++++-- 2 files changed, 19 insertions(+), 2 deletions(-) diff --git a/target/arm/cpu.h b/target/arm/cpu.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -XXX,XX +XXX,XX @@ static inline bool isar_feature_aa64_aa32(const ARMISARegisters *id) return FIELD_EX64(id->id_aa64pfr0, ID_AA64PFR0, EL0) >= 2; } +static inline bool isar_feature_aa64_aa32_el1(const ARMISARegisters *id) +{ + return FIELD_EX64(id->id_aa64pfr0, ID_AA64PFR0, EL1) >= 2; +} + static inline bool isar_feature_aa64_sve(const ARMISARegisters *id) { return FIELD_EX64(id->id_aa64pfr0, ID_AA64PFR0, SVE) != 0; diff --git a/target/arm/helper.c b/target/arm/helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -XXX,XX +XXX,XX @@ static void scr_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value) ARMCPU *cpu = env_archcpu(env); if (ri->state == ARM_CP_STATE_AA64) { - value |= SCR_FW | SCR_AW; /* these two bits are RES1. */ + if (arm_feature(env, ARM_FEATURE_AARCH64) && + !cpu_isar_feature(aa64_aa32_el1, cpu)) { + value |= SCR_FW | SCR_AW; /* these two bits are RES1. */ + } valid_mask &= ~SCR_NET; if (cpu_isar_feature(aa64_lor, cpu)) { @@ -XXX,XX +XXX,XX @@ static void scr_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value) raw_write(env, ri, value); } +static void scr_reset(CPUARMState *env, const ARMCPRegInfo *ri) +{ + /* + * scr_write will set the RES1 bits on an AArch64-only CPU. + * The reset value will be 0x30 on an AArch64-only CPU and 0 otherwise. + */ + scr_write(env, ri, 0); +} + static CPAccessResult access_aa64_tid2(CPUARMState *env, const ARMCPRegInfo *ri, bool isread) @@ -XXX,XX +XXX,XX @@ static const ARMCPRegInfo el3_cp_reginfo[] = { { .name = "SCR_EL3", .state = ARM_CP_STATE_AA64, .opc0 = 3, .opc1 = 6, .crn = 1, .crm = 1, .opc2 = 0, .access = PL3_RW, .fieldoffset = offsetof(CPUARMState, cp15.scr_el3), - .resetvalue = 0, .writefn = scr_write }, + .resetfn = scr_reset, .writefn = scr_write }, { .name = "SCR", .type = ARM_CP_ALIAS | ARM_CP_NEWEL, .cp = 15, .opc1 = 0, .crn = 1, .crm = 1, .opc2 = 0, .access = PL1_RW, .accessfn = access_trap_aa32s_el1, -- 2.20.1 From: Hao Wu <wuhaotsh@google.com> NPCM7XX GPIO devices have been implemented in hw/gpio/npcm7xx-gpio.c. So we removed them from the unimplemented devices list. Reviewed-by: Doug Evans<dje@google.com> Reviewed-by: Tyrong Ting<kfting@nuvoton.com> Signed-off-by: Hao Wu<wuhaotsh@google.com> Message-id: 20210129005845.416272-2-wuhaotsh@google.com Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- hw/arm/npcm7xx.c | 8 -------- 1 file changed, 8 deletions(-) diff --git a/hw/arm/npcm7xx.c b/hw/arm/npcm7xx.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/npcm7xx.c +++ b/hw/arm/npcm7xx.c @@ -XXX,XX +XXX,XX @@ static void npcm7xx_realize(DeviceState *dev, Error **errp) create_unimplemented_device("npcm7xx.pcierc", 0xe1000000, 64 * KiB); create_unimplemented_device("npcm7xx.kcs", 0xf0007000, 4 * KiB); create_unimplemented_device("npcm7xx.gfxi", 0xf000e000, 4 * KiB); - create_unimplemented_device("npcm7xx.gpio[0]", 0xf0010000, 4 * KiB); - create_unimplemented_device("npcm7xx.gpio[1]", 0xf0011000, 4 * KiB); - create_unimplemented_device("npcm7xx.gpio[2]", 0xf0012000, 4 * KiB); - create_unimplemented_device("npcm7xx.gpio[3]", 0xf0013000, 4 * KiB); - create_unimplemented_device("npcm7xx.gpio[4]", 0xf0014000, 4 * KiB); - create_unimplemented_device("npcm7xx.gpio[5]", 0xf0015000, 4 * KiB); - create_unimplemented_device("npcm7xx.gpio[6]", 0xf0016000, 4 * KiB); - create_unimplemented_device("npcm7xx.gpio[7]", 0xf0017000, 4 * KiB); create_unimplemented_device("npcm7xx.smbus[0]", 0xf0080000, 4 * KiB); create_unimplemented_device("npcm7xx.smbus[1]", 0xf0081000, 4 * KiB); create_unimplemented_device("npcm7xx.smbus[2]", 0xf0082000, 4 * KiB); -- 2.20.1 From: Rebecca Cran <rebecca@nuviainc.com> Add support for FEAT_DIT. DIT (Data Independent Timing) is a required feature for ARMv8.4. Since virtual machine execution is largely nondeterministic and TCG is outside of the security domain, it's implemented as a NOP. Signed-off-by: Rebecca Cran <rebecca@nuviainc.com> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20210208065700.19454-2-rebecca@nuviainc.com Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/cpu.h | 12 ++++++++++++ target/arm/internals.h | 6 ++++++ target/arm/helper.c | 22 ++++++++++++++++++++++ target/arm/translate-a64.c | 12 ++++++++++++ 4 files changed, 52 insertions(+) diff --git a/target/arm/cpu.h b/target/arm/cpu.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -XXX,XX +XXX,XX @@ void pmu_init(ARMCPU *cpu); #define CPSR_IT_2_7 (0xfc00U) #define CPSR_GE (0xfU << 16) #define CPSR_IL (1U << 20) +#define CPSR_DIT (1U << 21) #define CPSR_PAN (1U << 22) #define CPSR_J (1U << 24) #define CPSR_IT_0_1 (3U << 25) @@ -XXX,XX +XXX,XX @@ void pmu_init(ARMCPU *cpu); #define PSTATE_SS (1U << 21) #define PSTATE_PAN (1U << 22) #define PSTATE_UAO (1U << 23) +#define PSTATE_DIT (1U << 24) #define PSTATE_TCO (1U << 25) #define PSTATE_V (1U << 28) #define PSTATE_C (1U << 29) @@ -XXX,XX +XXX,XX @@ static inline bool isar_feature_aa32_tts2uxn(const ARMISARegisters *id) return FIELD_EX32(id->id_mmfr4, ID_MMFR4, XNX) != 0; } +static inline bool isar_feature_aa32_dit(const ARMISARegisters *id) +{ + return FIELD_EX32(id->id_pfr0, ID_PFR0, DIT) != 0; +} + /* * 64-bit feature tests via id registers. */ @@ -XXX,XX +XXX,XX @@ static inline bool isar_feature_aa64_tts2uxn(const ARMISARegisters *id) return FIELD_EX64(id->id_aa64mmfr1, ID_AA64MMFR1, XNX) != 0; } +static inline bool isar_feature_aa64_dit(const ARMISARegisters *id) +{ + return FIELD_EX64(id->id_aa64pfr0, ID_AA64PFR0, DIT) != 0; +} + /* * Feature tests for "does this exist in either 32-bit or 64-bit?" */ diff --git a/target/arm/internals.h b/target/arm/internals.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/internals.h +++ b/target/arm/internals.h @@ -XXX,XX +XXX,XX @@ static inline uint32_t aarch32_cpsr_valid_mask(uint64_t features, if (isar_feature_aa32_pan(id)) { valid |= CPSR_PAN; } + if (isar_feature_aa32_dit(id)) { + valid |= CPSR_DIT; + } return valid; } @@ -XXX,XX +XXX,XX @@ static inline uint32_t aarch64_pstate_valid_mask(const ARMISARegisters *id) if (isar_feature_aa64_uao(id)) { valid |= PSTATE_UAO; } + if (isar_feature_aa64_dit(id)) { + valid |= PSTATE_DIT; + } if (isar_feature_aa64_mte(id)) { valid |= PSTATE_TCO; } diff --git a/target/arm/helper.c b/target/arm/helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -XXX,XX +XXX,XX @@ static const ARMCPRegInfo uao_reginfo = { .readfn = aa64_uao_read, .writefn = aa64_uao_write }; +static uint64_t aa64_dit_read(CPUARMState *env, const ARMCPRegInfo *ri) +{ + return env->pstate & PSTATE_DIT; +} + +static void aa64_dit_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) +{ + env->pstate = (env->pstate & ~PSTATE_DIT) | (value & PSTATE_DIT); +} + +static const ARMCPRegInfo dit_reginfo = { + .name = "DIT", .state = ARM_CP_STATE_AA64, + .opc0 = 3, .opc1 = 3, .crn = 4, .crm = 2, .opc2 = 5, + .type = ARM_CP_NO_RAW, .access = PL0_RW, + .readfn = aa64_dit_read, .writefn = aa64_dit_write +}; + static CPAccessResult aa64_cacheop_poc_access(CPUARMState *env, const ARMCPRegInfo *ri, bool isread) @@ -XXX,XX +XXX,XX @@ void register_cp_regs_for_features(ARMCPU *cpu) define_one_arm_cp_reg(cpu, &uao_reginfo); } + if (cpu_isar_feature(aa64_dit, cpu)) { + define_one_arm_cp_reg(cpu, &dit_reginfo); + } + if (arm_feature(env, ARM_FEATURE_EL2) && cpu_isar_feature(aa64_vh, cpu)) { define_arm_cp_regs(cpu, vhe_reginfo); } diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-a64.c +++ b/target/arm/translate-a64.c @@ -XXX,XX +XXX,XX @@ static void handle_msr_i(DisasContext *s, uint32_t insn, tcg_temp_free_i32(t1); break; + case 0x1a: /* DIT */ + if (!dc_isar_feature(aa64_dit, s)) { + goto do_unallocated; + } + if (crm & 1) { + set_pstate_bits(PSTATE_DIT); + } else { + clear_pstate_bits(PSTATE_DIT); + } + /* There's no need to rebuild hflags because DIT is a nop */ + break; + case 0x1e: /* DAIFSet */ t1 = tcg_const_i32(crm); gen_helper_msr_i_daifset(cpu_env, t1); -- 2.20.1 From: Rebecca Cran <rebecca@nuviainc.com> cpsr has been treated as being the same as spsr, but it isn't. Since PSTATE_SS isn't in cpsr, remove it and move it into env->pstate. This allows us to add support for CPSR_DIT, adding helper functions to merge SPSR_ELx to and from CPSR. Signed-off-by: Rebecca Cran <rebecca@nuviainc.com> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20210208065700.19454-3-rebecca@nuviainc.com Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/helper-a64.c | 27 +++++++++++++++++++++++---- target/arm/helper.c | 24 ++++++++++++++++++------ target/arm/op_helper.c | 9 +-------- 3 files changed, 42 insertions(+), 18 deletions(-) diff --git a/target/arm/helper-a64.c b/target/arm/helper-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper-a64.c +++ b/target/arm/helper-a64.c @@ -XXX,XX +XXX,XX @@ static int el_from_spsr(uint32_t spsr) } } +static void cpsr_write_from_spsr_elx(CPUARMState *env, + uint32_t val) +{ + uint32_t mask; + + /* Save SPSR_ELx.SS into PSTATE. */ + env->pstate = (env->pstate & ~PSTATE_SS) | (val & PSTATE_SS); + val &= ~PSTATE_SS; + + /* Move DIT to the correct location for CPSR */ + if (val & PSTATE_DIT) { + val &= ~PSTATE_DIT; + val |= CPSR_DIT; + } + + mask = aarch32_cpsr_valid_mask(env->features, \ + &env_archcpu(env)->isar); + cpsr_write(env, val, mask, CPSRWriteRaw); +} + void HELPER(exception_return)(CPUARMState *env, uint64_t new_pc) { int cur_el = arm_current_el(env); unsigned int spsr_idx = aarch64_banked_spsr_index(cur_el); - uint32_t mask, spsr = env->banked_spsr[spsr_idx]; + uint32_t spsr = env->banked_spsr[spsr_idx]; int new_el; bool return_to_aa64 = (spsr & PSTATE_nRW) == 0; @@ -XXX,XX +XXX,XX @@ void HELPER(exception_return)(CPUARMState *env, uint64_t new_pc) * will sort the register banks out for us, and we've already * caught all the bad-mode cases in el_from_spsr(). */ - mask = aarch32_cpsr_valid_mask(env->features, &env_archcpu(env)->isar); - cpsr_write(env, spsr, mask, CPSRWriteRaw); + cpsr_write_from_spsr_elx(env, spsr); if (!arm_singlestep_active(env)) { - env->uncached_cpsr &= ~PSTATE_SS; + env->pstate &= ~PSTATE_SS; } aarch64_sync_64_to_32(env); diff --git a/target/arm/helper.c b/target/arm/helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -XXX,XX +XXX,XX @@ static void take_aarch32_exception(CPUARMState *env, int new_mode, * For exceptions taken to AArch32 we must clear the SS bit in both * PSTATE and in the old-state value we save to SPSR_<mode>, so zero it now. */ - env->uncached_cpsr &= ~PSTATE_SS; + env->pstate &= ~PSTATE_SS; env->spsr = cpsr_read(env); /* Clear IT bits. */ env->condexec_bits = 0; @@ -XXX,XX +XXX,XX @@ static int aarch64_regnum(CPUARMState *env, int aarch32_reg) } } +static uint32_t cpsr_read_for_spsr_elx(CPUARMState *env) +{ + uint32_t ret = cpsr_read(env); + + /* Move DIT to the correct location for SPSR_ELx */ + if (ret & CPSR_DIT) { + ret &= ~CPSR_DIT; + ret |= PSTATE_DIT; + } + /* Merge PSTATE.SS into SPSR_ELx */ + ret |= env->pstate & PSTATE_SS; + + return ret; +} + /* Handle exception entry to a target EL which is using AArch64 */ static void arm_cpu_do_interrupt_aarch64(CPUState *cs) { @@ -XXX,XX +XXX,XX @@ static void arm_cpu_do_interrupt_aarch64(CPUState *cs) aarch64_save_sp(env, arm_current_el(env)); env->elr_el[new_el] = env->pc; } else { - old_mode = cpsr_read(env); + old_mode = cpsr_read_for_spsr_elx(env); env->elr_el[new_el] = env->regs[15]; aarch64_sync_32_to_64(env); @@ -XXX,XX +XXX,XX @@ void cpu_get_tb_cpu_state(CPUARMState *env, target_ulong *pc, target_ulong *cs_base, uint32_t *pflags) { uint32_t flags = env->hflags; - uint32_t pstate_for_ss; *cs_base = 0; assert_hflags_rebuild_correctly(env); @@ -XXX,XX +XXX,XX @@ void cpu_get_tb_cpu_state(CPUARMState *env, target_ulong *pc, if (cpu_isar_feature(aa64_bti, env_archcpu(env))) { flags = FIELD_DP32(flags, TBFLAG_A64, BTYPE, env->btype); } - pstate_for_ss = env->pstate; } else { *pc = env->regs[15]; @@ -XXX,XX +XXX,XX @@ void cpu_get_tb_cpu_state(CPUARMState *env, target_ulong *pc, flags = FIELD_DP32(flags, TBFLAG_AM32, THUMB, env->thumb); flags = FIELD_DP32(flags, TBFLAG_AM32, CONDEXEC, env->condexec_bits); - pstate_for_ss = env->uncached_cpsr; } /* @@ -XXX,XX +XXX,XX @@ void cpu_get_tb_cpu_state(CPUARMState *env, target_ulong *pc, * SS_ACTIVE is set in hflags; PSTATE_SS is computed every TB. */ if (FIELD_EX32(flags, TBFLAG_ANY, SS_ACTIVE) && - (pstate_for_ss & PSTATE_SS)) { + (env->pstate & PSTATE_SS)) { flags = FIELD_DP32(flags, TBFLAG_ANY, PSTATE_SS, 1); } diff --git a/target/arm/op_helper.c b/target/arm/op_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/op_helper.c +++ b/target/arm/op_helper.c @@ -XXX,XX +XXX,XX @@ void HELPER(exception_bkpt_insn)(CPUARMState *env, uint32_t syndrome) uint32_t HELPER(cpsr_read)(CPUARMState *env) { - /* - * We store the ARMv8 PSTATE.SS bit in env->uncached_cpsr. - * This is convenient for populating SPSR_ELx, but must be - * hidden from aarch32 mode, where it is not visible. - * - * TODO: ARMv8.4-DIT -- need to move SS somewhere else. - */ - return cpsr_read(env) & ~(CPSR_EXEC | PSTATE_SS); + return cpsr_read(env) & ~CPSR_EXEC; } void HELPER(cpsr_write)(CPUARMState *env, uint32_t val, uint32_t mask) -- 2.20.1 From: Rebecca Cran <rebecca@nuviainc.com> Enable FEAT_DIT for the "max" AARCH64 CPU. Signed-off-by: Rebecca Cran <rebecca@nuviainc.com> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20210208065700.19454-4-rebecca@nuviainc.com Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/cpu64.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/target/arm/cpu64.c b/target/arm/cpu64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpu64.c +++ b/target/arm/cpu64.c @@ -XXX,XX +XXX,XX @@ static void aarch64_max_initfn(Object *obj) t = FIELD_DP64(t, ID_AA64PFR0, FP, 1); t = FIELD_DP64(t, ID_AA64PFR0, ADVSIMD, 1); t = FIELD_DP64(t, ID_AA64PFR0, SEL2, 1); + t = FIELD_DP64(t, ID_AA64PFR0, DIT, 1); cpu->isar.id_aa64pfr0 = t; t = cpu->isar.id_aa64pfr1; @@ -XXX,XX +XXX,XX @@ static void aarch64_max_initfn(Object *obj) u = FIELD_DP32(u, ID_ISAR6, SPECRES, 1); cpu->isar.id_isar6 = u; + u = cpu->isar.id_pfr0; + u = FIELD_DP32(u, ID_PFR0, DIT, 1); + cpu->isar.id_pfr0 = u; + u = cpu->isar.id_mmfr3; u = FIELD_DP32(u, ID_MMFR3, PAN, 2); /* ATS1E1 */ cpu->isar.id_mmfr3 = u; -- 2.20.1 From: Rebecca Cran <rebecca@nuviainc.com> Enable FEAT_DIT for the "max" 32-bit CPU. Signed-off-by: Rebecca Cran <rebecca@nuviainc.com> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20210208065700.19454-5-rebecca@nuviainc.com Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/cpu.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/target/arm/cpu.c b/target/arm/cpu.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpu.c +++ b/target/arm/cpu.c @@ -XXX,XX +XXX,XX @@ static void arm_max_initfn(Object *obj) t = FIELD_DP32(t, ID_MMFR4, CNP, 1); /* TTCNP */ t = FIELD_DP32(t, ID_MMFR4, XNX, 1); /* TTS2UXN */ cpu->isar.id_mmfr4 = t; + + t = cpu->isar.id_pfr0; + t = FIELD_DP32(t, ID_PFR0, DIT, 1); + cpu->isar.id_pfr0 = t; } #endif } -- 2.20.1 Update infocenter.arm.com URLs for various pieces of Arm documentation to the new developer.arm.com equivalents. (There is a redirection in place from the old URLs, but we might as well update our comments in case the redirect ever disappears in future.) This patch covers all the URLs which are not MPS2/SSE-200/IoTKit related (those are dealt with in a different patch). Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Message-id: 20210205171456.19939-1-peter.maydell@linaro.org --- include/hw/dma/pl080.h | 7 ++++--- include/hw/misc/arm_integrator_debug.h | 2 +- include/hw/ssi/pl022.h | 5 +++-- hw/arm/aspeed_ast2600.c | 2 +- hw/arm/musca.c | 4 ++-- hw/misc/arm_integrator_debug.c | 2 +- hw/timer/arm_timer.c | 7 ++++--- 7 files changed, 16 insertions(+), 13 deletions(-) diff --git a/include/hw/dma/pl080.h b/include/hw/dma/pl080.h index XXXXXXX..XXXXXXX 100644 --- a/include/hw/dma/pl080.h +++ b/include/hw/dma/pl080.h @@ -XXX,XX +XXX,XX @@ * (at your option) any later version. */ -/* This is a model of the Arm PrimeCell PL080/PL081 DMA controller: +/* + * This is a model of the Arm PrimeCell PL080/PL081 DMA controller: * The PL080 TRM is: - * http://infocenter.arm.com/help/topic/com.arm.doc.ddi0196g/DDI0196.pdf + * https://developer.arm.com/documentation/ddi0196/latest * and the PL081 TRM is: - * http://infocenter.arm.com/help/topic/com.arm.doc.ddi0218e/DDI0218.pdf + * https://developer.arm.com/documentation/ddi0218/latest * * QEMU interface: * + sysbus IRQ 0: DMACINTR combined interrupt line diff --git a/include/hw/misc/arm_integrator_debug.h b/include/hw/misc/arm_integrator_debug.h index XXXXXXX..XXXXXXX 100644 --- a/include/hw/misc/arm_integrator_debug.h +++ b/include/hw/misc/arm_integrator_debug.h @@ -XXX,XX +XXX,XX @@ * * Browse the data sheet: * - * http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.dui0159b/Babbfijf.html + * https://developer.arm.com/documentation/dui0159/b/peripherals-and-interfaces/debug-leds-and-dip-switch-interface * * Copyright (c) 2013 Alex Bennée <alex@bennee.com> * diff --git a/include/hw/ssi/pl022.h b/include/hw/ssi/pl022.h index XXXXXXX..XXXXXXX 100644 --- a/include/hw/ssi/pl022.h +++ b/include/hw/ssi/pl022.h @@ -XXX,XX +XXX,XX @@ * (at your option) any later version. */ -/* This is a model of the Arm PrimeCell PL022 synchronous serial port. +/* + * This is a model of the Arm PrimeCell PL022 synchronous serial port. * The PL022 TRM is: - * http://infocenter.arm.com/help/topic/com.arm.doc.ddi0194h/DDI0194H_ssp_pl022_trm.pdf + * https://developer.arm.com/documentation/ddi0194/latest * * QEMU interface: * + sysbus IRQ: SSPINTR combined interrupt line diff --git a/hw/arm/aspeed_ast2600.c b/hw/arm/aspeed_ast2600.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/aspeed_ast2600.c +++ b/hw/arm/aspeed_ast2600.c @@ -XXX,XX +XXX,XX @@ static void aspeed_soc_ast2600_init(Object *obj) /* * ASPEED ast2600 has 0xf as cluster ID * - * http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.ddi0388e/CIHEBGFG.html + * https://developer.arm.com/documentation/ddi0388/e/the-system-control-coprocessors/summary-of-system-control-coprocessor-registers/multiprocessor-affinity-register */ static uint64_t aspeed_calc_affinity(int cpu) { diff --git a/hw/arm/musca.c b/hw/arm/musca.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/musca.c +++ b/hw/arm/musca.c @@ -XXX,XX +XXX,XX @@ * https://developer.arm.com/products/system-design/development-boards/iot-test-chips-and-boards/musca-a-test-chip-board * https://developer.arm.com/products/system-design/development-boards/iot-test-chips-and-boards/musca-b-test-chip-board * We model the A and B1 variants of this board, as described in the TRMs: - * http://infocenter.arm.com/help/topic/com.arm.doc.101107_0000_00_en/index.html - * http://infocenter.arm.com/help/topic/com.arm.doc.101312_0000_00_en/index.html + * https://developer.arm.com/documentation/101107/latest/ + * https://developer.arm.com/documentation/101312/latest/ */ #include "qemu/osdep.h" diff --git a/hw/misc/arm_integrator_debug.c b/hw/misc/arm_integrator_debug.c index XXXXXXX..XXXXXXX 100644 --- a/hw/misc/arm_integrator_debug.c +++ b/hw/misc/arm_integrator_debug.c @@ -XXX,XX +XXX,XX @@ * to this area. * * The real h/w is described at: - * http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.dui0159b/Babbfijf.html + * https://developer.arm.com/documentation/dui0159/b/peripherals-and-interfaces/debug-leds-and-dip-switch-interface * * Copyright (c) 2013 Alex Bennée <alex@bennee.com> * diff --git a/hw/timer/arm_timer.c b/hw/timer/arm_timer.c index XXXXXXX..XXXXXXX 100644 --- a/hw/timer/arm_timer.c +++ b/hw/timer/arm_timer.c @@ -XXX,XX +XXX,XX @@ static arm_timer_state *arm_timer_init(uint32_t freq) return s; } -/* ARM PrimeCell SP804 dual timer module. +/* + * ARM PrimeCell SP804 dual timer module. * Docs at - * http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.ddi0271d/index.html -*/ + * https://developer.arm.com/documentation/ddi0271/latest/ + */ #define TYPE_SP804 "sp804" OBJECT_DECLARE_SIMPLE_TYPE(SP804State, SP804) -- 2.20.1 In cpu_exec() we have a longstanding workaround for compilers which do not correctly implement the part of the sigsetjmp()/siglongjmp() spec which requires that local variables which are not changed between the setjmp and the longjmp retain their value. I recently ran across the upstream clang bug report for this; add a link to it to the comment describing the workaround, and generally expand the comment, so that we have a reasonable chance in future of understanding why it's there and determining when we can remove it, assuming clang eventually fixes the bug. Remove the /* buggy compiler */ comments on the #else and #endif: they don't add anything to understanding and are somewhat misleading since they're sandwiching the code path for *non*-buggy compilers. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Message-id: 20210129130330.30820-1-peter.maydell@linaro.org --- accel/tcg/cpu-exec.c | 25 +++++++++++++++++++------ 1 file changed, 19 insertions(+), 6 deletions(-) diff --git a/accel/tcg/cpu-exec.c b/accel/tcg/cpu-exec.c index XXXXXXX..XXXXXXX 100644 --- a/accel/tcg/cpu-exec.c +++ b/accel/tcg/cpu-exec.c @@ -XXX,XX +XXX,XX @@ int cpu_exec(CPUState *cpu) /* prepare setjmp context for exception handling */ if (sigsetjmp(cpu->jmp_env, 0) != 0) { #if defined(__clang__) - /* Some compilers wrongly smash all local variables after - * siglongjmp. There were bug reports for gcc 4.5.0 and clang. + /* + * Some compilers wrongly smash all local variables after + * siglongjmp (the spec requires that only non-volatile locals + * which are changed between the sigsetjmp and siglongjmp are + * permitted to be trashed). There were bug reports for gcc + * 4.5.0 and clang. The bug is fixed in all versions of gcc + * that we support, but is still unfixed in clang: + * https://bugs.llvm.org/show_bug.cgi?id=21183 + * * Reload essential local variables here for those compilers. - * Newer versions of gcc would complain about this code (-Wclobbered). */ + * Newer versions of gcc would complain about this code (-Wclobbered), + * so we only perform the workaround for clang. + */ cpu = current_cpu; cc = CPU_GET_CLASS(cpu); -#else /* buggy compiler */ - /* Assert that the compiler does not smash local variables. */ +#else + /* + * Non-buggy compilers preserve these locals; assert that + * they have the correct value. + */ g_assert(cpu == current_cpu); g_assert(cc == CPU_GET_CLASS(cpu)); -#endif /* buggy compiler */ +#endif + #ifndef CONFIG_SOFTMMU tcg_debug_assert(!have_mmap_lock()); #endif -- 2.20.1 From: Richard Henderson <richard.henderson@linaro.org> This data can be allocated by page_alloc_target_data() and released by page_set_flags(start, end, prot | PAGE_RESET). This data will be used to hold tag memory for AArch64 MTE. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20210210000223.884088-2-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- include/exec/cpu-all.h | 42 +++++++++++++++++++++++++++++++++------ accel/tcg/translate-all.c | 28 ++++++++++++++++++++++++++ linux-user/mmap.c | 4 +++- linux-user/syscall.c | 4 ++-- 4 files changed, 69 insertions(+), 9 deletions(-) diff --git a/include/exec/cpu-all.h b/include/exec/cpu-all.h index XXXXXXX..XXXXXXX 100644 --- a/include/exec/cpu-all.h +++ b/include/exec/cpu-all.h @@ -XXX,XX +XXX,XX @@ extern intptr_t qemu_host_page_mask; #define PAGE_EXEC 0x0004 #define PAGE_BITS (PAGE_READ | PAGE_WRITE | PAGE_EXEC) #define PAGE_VALID 0x0008 -/* original state of the write flag (used when tracking self-modifying - code */ +/* + * Original state of the write flag (used when tracking self-modifying code) + */ #define PAGE_WRITE_ORG 0x0010 -/* Invalidate the TLB entry immediately, helpful for s390x - * Low-Address-Protection. Used with PAGE_WRITE in tlb_set_page_with_attrs() */ -#define PAGE_WRITE_INV 0x0040 +/* + * Invalidate the TLB entry immediately, helpful for s390x + * Low-Address-Protection. Used with PAGE_WRITE in tlb_set_page_with_attrs() + */ +#define PAGE_WRITE_INV 0x0020 +/* For use with page_set_flags: page is being replaced; target_data cleared. */ +#define PAGE_RESET 0x0040 + #if defined(CONFIG_BSD) && defined(CONFIG_USER_ONLY) /* FIXME: Code that sets/uses this is broken and needs to go away. */ -#define PAGE_RESERVED 0x0020 +#define PAGE_RESERVED 0x0100 #endif /* Target-specific bits that will be used via page_get_flags(). */ #define PAGE_TARGET_1 0x0080 @@ -XXX,XX +XXX,XX @@ int walk_memory_regions(void *, walk_memory_regions_fn); int page_get_flags(target_ulong address); void page_set_flags(target_ulong start, target_ulong end, int flags); int page_check_range(target_ulong start, target_ulong len, int flags); + +/** + * page_alloc_target_data(address, size) + * @address: guest virtual address + * @size: size of data to allocate + * + * Allocate @size bytes of out-of-band data to associate with the + * guest page at @address. If the page is not mapped, NULL will + * be returned. If there is existing data associated with @address, + * no new memory will be allocated. + * + * The memory will be freed when the guest page is deallocated, + * e.g. with the munmap system call. + */ +void *page_alloc_target_data(target_ulong address, size_t size); + +/** + * page_get_target_data(address) + * @address: guest virtual address + * + * Return any out-of-bound memory assocated with the guest page + * at @address, as per page_alloc_target_data. + */ +void *page_get_target_data(target_ulong address); #endif CPUArchState *cpu_copy(CPUArchState *env); diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c index XXXXXXX..XXXXXXX 100644 --- a/accel/tcg/translate-all.c +++ b/accel/tcg/translate-all.c @@ -XXX,XX +XXX,XX @@ typedef struct PageDesc { unsigned int code_write_count; #else unsigned long flags; + void *target_data; #endif #ifndef CONFIG_USER_ONLY QemuSpin lock; @@ -XXX,XX +XXX,XX @@ int page_get_flags(target_ulong address) void page_set_flags(target_ulong start, target_ulong end, int flags) { target_ulong addr, len; + bool reset_target_data; /* This function should never be called with addresses outside the guest address space. If this assert fires, it probably indicates @@ -XXX,XX +XXX,XX @@ void page_set_flags(target_ulong start, target_ulong end, int flags) if (flags & PAGE_WRITE) { flags |= PAGE_WRITE_ORG; } + reset_target_data = !(flags & PAGE_VALID) || (flags & PAGE_RESET); + flags &= ~PAGE_RESET; for (addr = start, len = end - start; len != 0; @@ -XXX,XX +XXX,XX @@ void page_set_flags(target_ulong start, target_ulong end, int flags) p->first_tb) { tb_invalidate_phys_page(addr, 0); } + if (reset_target_data && p->target_data) { + g_free(p->target_data); + p->target_data = NULL; + } p->flags = flags; } } +void *page_get_target_data(target_ulong address) +{ + PageDesc *p = page_find(address >> TARGET_PAGE_BITS); + return p ? p->target_data : NULL; +} + +void *page_alloc_target_data(target_ulong address, size_t size) +{ + PageDesc *p = page_find(address >> TARGET_PAGE_BITS); + void *ret = NULL; + + if (p->flags & PAGE_VALID) { + ret = p->target_data; + if (!ret) { + p->target_data = ret = g_malloc0(size); + } + } + return ret; +} + int page_check_range(target_ulong start, target_ulong len, int flags) { PageDesc *p; diff --git a/linux-user/mmap.c b/linux-user/mmap.c index XXXXXXX..XXXXXXX 100644 --- a/linux-user/mmap.c +++ b/linux-user/mmap.c @@ -XXX,XX +XXX,XX @@ abi_long target_mmap(abi_ulong start, abi_ulong len, int target_prot, } } the_end1: + page_flags |= PAGE_RESET; page_set_flags(start, start + len, page_flags); the_end: trace_target_mmap_complete(start); @@ -XXX,XX +XXX,XX @@ abi_long target_mremap(abi_ulong old_addr, abi_ulong old_size, new_addr = h2g(host_addr); prot = page_get_flags(old_addr); page_set_flags(old_addr, old_addr + old_size, 0); - page_set_flags(new_addr, new_addr + new_size, prot | PAGE_VALID); + page_set_flags(new_addr, new_addr + new_size, + prot | PAGE_VALID | PAGE_RESET); } tb_invalidate_phys_range(new_addr, new_addr + new_size); mmap_unlock(); diff --git a/linux-user/syscall.c b/linux-user/syscall.c index XXXXXXX..XXXXXXX 100644 --- a/linux-user/syscall.c +++ b/linux-user/syscall.c @@ -XXX,XX +XXX,XX @@ static inline abi_ulong do_shmat(CPUArchState *cpu_env, raddr=h2g((unsigned long)host_raddr); page_set_flags(raddr, raddr + shm_info.shm_segsz, - PAGE_VALID | PAGE_READ | - ((shmflg & SHM_RDONLY)? 0 : PAGE_WRITE)); + PAGE_VALID | PAGE_RESET | PAGE_READ | + (shmflg & SHM_RDONLY ? 0 : PAGE_WRITE)); for (i = 0; i < N_SHM_REGIONS; i++) { if (!shm_regions[i].in_use) { -- 2.20.1 From: Richard Henderson <richard.henderson@linaro.org> Record whether the backing page is anonymous, or if it has file backing. This will allow us to get close to the Linux AArch64 ABI for MTE, which allows tag memory only on ram-backed VMAs. The real ABI allows tag memory on files, when those files are on ram-backed filesystems, such as tmpfs. We will not be able to implement that in QEMU linux-user. Thankfully, anonymous memory for malloc arenas is the primary consumer of this feature, so this restricted version should still be of use. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20210210000223.884088-3-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- include/exec/cpu-all.h | 2 ++ linux-user/mmap.c | 3 +++ 2 files changed, 5 insertions(+) diff --git a/include/exec/cpu-all.h b/include/exec/cpu-all.h index XXXXXXX..XXXXXXX 100644 --- a/include/exec/cpu-all.h +++ b/include/exec/cpu-all.h @@ -XXX,XX +XXX,XX @@ extern intptr_t qemu_host_page_mask; #define PAGE_WRITE_INV 0x0020 /* For use with page_set_flags: page is being replaced; target_data cleared. */ #define PAGE_RESET 0x0040 +/* For linux-user, indicates that the page is MAP_ANON. */ +#define PAGE_ANON 0x0080 #if defined(CONFIG_BSD) && defined(CONFIG_USER_ONLY) /* FIXME: Code that sets/uses this is broken and needs to go away. */ diff --git a/linux-user/mmap.c b/linux-user/mmap.c index XXXXXXX..XXXXXXX 100644 --- a/linux-user/mmap.c +++ b/linux-user/mmap.c @@ -XXX,XX +XXX,XX @@ abi_long target_mmap(abi_ulong start, abi_ulong len, int target_prot, } } the_end1: + if (flags & MAP_ANONYMOUS) { + page_flags |= PAGE_ANON; + } page_flags |= PAGE_RESET; page_set_flags(start, start + len, page_flags); the_end: -- 2.20.1 From: Richard Henderson <richard.henderson@linaro.org> This is more descriptive than 'unsigned long'. No functional change, since these match on all linux+bsd hosts. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20210210000223.884088-4-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- include/exec/cpu-all.h | 2 +- bsd-user/main.c | 4 ++-- linux-user/elfload.c | 4 ++-- linux-user/main.c | 4 ++-- 4 files changed, 7 insertions(+), 7 deletions(-) diff --git a/include/exec/cpu-all.h b/include/exec/cpu-all.h index XXXXXXX..XXXXXXX 100644 --- a/include/exec/cpu-all.h +++ b/include/exec/cpu-all.h @@ -XXX,XX +XXX,XX @@ static inline void tswap64s(uint64_t *s) /* On some host systems the guest address space is reserved on the host. * This allows the guest address space to be offset to a convenient location. */ -extern unsigned long guest_base; +extern uintptr_t guest_base; extern bool have_guest_base; extern unsigned long reserved_va; diff --git a/bsd-user/main.c b/bsd-user/main.c index XXXXXXX..XXXXXXX 100644 --- a/bsd-user/main.c +++ b/bsd-user/main.c @@ -XXX,XX +XXX,XX @@ int singlestep; unsigned long mmap_min_addr; -unsigned long guest_base; +uintptr_t guest_base; bool have_guest_base; unsigned long reserved_va; @@ -XXX,XX +XXX,XX @@ int main(int argc, char **argv) g_free(target_environ); if (qemu_loglevel_mask(CPU_LOG_PAGE)) { - qemu_log("guest_base 0x%lx\n", guest_base); + qemu_log("guest_base %p\n", (void *)guest_base); log_page_dump("binary load"); qemu_log("start_brk 0x" TARGET_ABI_FMT_lx "\n", info->start_brk); diff --git a/linux-user/elfload.c b/linux-user/elfload.c index XXXXXXX..XXXXXXX 100644 --- a/linux-user/elfload.c +++ b/linux-user/elfload.c @@ -XXX,XX +XXX,XX @@ static void pgb_have_guest_base(const char *image_name, abi_ulong guest_loaddr, void *addr, *test; if (!QEMU_IS_ALIGNED(guest_base, align)) { - fprintf(stderr, "Requested guest base 0x%lx does not satisfy " + fprintf(stderr, "Requested guest base %p does not satisfy " "host minimum alignment (0x%lx)\n", - guest_base, align); + (void *)guest_base, align); exit(EXIT_FAILURE); } diff --git a/linux-user/main.c b/linux-user/main.c index XXXXXXX..XXXXXXX 100644 --- a/linux-user/main.c +++ b/linux-user/main.c @@ -XXX,XX +XXX,XX @@ static const char *cpu_model; static const char *cpu_type; static const char *seed_optarg; unsigned long mmap_min_addr; -unsigned long guest_base; +uintptr_t guest_base; bool have_guest_base; /* @@ -XXX,XX +XXX,XX @@ int main(int argc, char **argv, char **envp) g_free(target_environ); if (qemu_loglevel_mask(CPU_LOG_PAGE)) { - qemu_log("guest_base 0x%lx\n", guest_base); + qemu_log("guest_base %p\n", (void *)guest_base); log_page_dump("binary load"); qemu_log("start_brk 0x" TARGET_ABI_FMT_lx "\n", info->start_brk); -- 2.20.1 From: Richard Henderson <richard.henderson@linaro.org> This is more descriptive than 'unsigned long'. No functional change, since these match on all linux+bsd hosts. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20210210000223.884088-5-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- include/exec/cpu_ldst.h | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/include/exec/cpu_ldst.h b/include/exec/cpu_ldst.h index XXXXXXX..XXXXXXX 100644 --- a/include/exec/cpu_ldst.h +++ b/include/exec/cpu_ldst.h @@ -XXX,XX +XXX,XX @@ typedef uint64_t abi_ptr; #endif /* All direct uses of g2h and h2g need to go away for usermode softmmu. */ -#define g2h(x) ((void *)((unsigned long)(abi_ptr)(x) + guest_base)) +#define g2h(x) ((void *)((uintptr_t)(abi_ptr)(x) + guest_base)) #if HOST_LONG_BITS <= TARGET_VIRT_ADDR_SPACE_BITS #define guest_addr_valid(x) (1) #else #define guest_addr_valid(x) ((x) <= GUEST_ADDR_MAX) #endif -#define h2g_valid(x) guest_addr_valid((unsigned long)(x) - guest_base) +#define h2g_valid(x) guest_addr_valid((uintptr_t)(x) - guest_base) static inline int guest_range_valid(unsigned long start, unsigned long len) { @@ -XXX,XX +XXX,XX @@ static inline int guest_range_valid(unsigned long start, unsigned long len) } #define h2g_nocheck(x) ({ \ - unsigned long __ret = (unsigned long)(x) - guest_base; \ + uintptr_t __ret = (uintptr_t)(x) - guest_base; \ (abi_ptr)__ret; \ }) -- 2.20.1 From: Richard Henderson <richard.henderson@linaro.org> Return bool not int; pass abi_ulong not 'unsigned long'. All callers use abi_ulong already, so the change in type has no effect. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20210210000223.884088-6-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- include/exec/cpu_ldst.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/include/exec/cpu_ldst.h b/include/exec/cpu_ldst.h index XXXXXXX..XXXXXXX 100644 --- a/include/exec/cpu_ldst.h +++ b/include/exec/cpu_ldst.h @@ -XXX,XX +XXX,XX @@ typedef uint64_t abi_ptr; #endif #define h2g_valid(x) guest_addr_valid((uintptr_t)(x) - guest_base) -static inline int guest_range_valid(unsigned long start, unsigned long len) +static inline bool guest_range_valid(abi_ulong start, abi_ulong len) { return len - 1 <= GUEST_ADDR_MAX && start <= GUEST_ADDR_MAX - len + 1; } -- 2.20.1 From: Richard Henderson <richard.henderson@linaro.org> Verify that addr + size - 1 does not wrap around. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20210210000223.884088-7-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- linux-user/qemu.h | 17 ++++++++++++----- 1 file changed, 12 insertions(+), 5 deletions(-) diff --git a/linux-user/qemu.h b/linux-user/qemu.h index XXXXXXX..XXXXXXX 100644 --- a/linux-user/qemu.h +++ b/linux-user/qemu.h @@ -XXX,XX +XXX,XX @@ extern unsigned long guest_stack_size; #define VERIFY_READ 0 #define VERIFY_WRITE 1 /* implies read access */ -static inline int access_ok(int type, abi_ulong addr, abi_ulong size) +static inline bool access_ok(int type, abi_ulong addr, abi_ulong size) { - return guest_addr_valid(addr) && - (size == 0 || guest_addr_valid(addr + size - 1)) && - page_check_range((target_ulong)addr, size, - (type == VERIFY_READ) ? PAGE_READ : (PAGE_READ | PAGE_WRITE)) == 0; + if (!guest_addr_valid(addr)) { + return false; + } + if (size != 0 && + (addr + size - 1 < addr || + !guest_addr_valid(addr + size - 1))) { + return false; + } + return page_check_range((target_ulong)addr, size, + (type == VERIFY_READ) ? PAGE_READ : + (PAGE_READ | PAGE_WRITE)) == 0; } /* NOTE __get_user and __put_user use host pointers and don't check access. -- 2.20.1 From: Richard Henderson <richard.henderson@linaro.org> These constants are only ever used with access_ok, and friends. Rather than translating them to PAGE_* bits, let them equal the PAGE_* bits to begin. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20210210000223.884088-8-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- linux-user/qemu.h | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/linux-user/qemu.h b/linux-user/qemu.h index XXXXXXX..XXXXXXX 100644 --- a/linux-user/qemu.h +++ b/linux-user/qemu.h @@ -XXX,XX +XXX,XX @@ extern unsigned long guest_stack_size; /* user access */ -#define VERIFY_READ 0 -#define VERIFY_WRITE 1 /* implies read access */ +#define VERIFY_READ PAGE_READ +#define VERIFY_WRITE (PAGE_READ | PAGE_WRITE) static inline bool access_ok(int type, abi_ulong addr, abi_ulong size) { @@ -XXX,XX +XXX,XX @@ static inline bool access_ok(int type, abi_ulong addr, abi_ulong size) !guest_addr_valid(addr + size - 1))) { return false; } - return page_check_range((target_ulong)addr, size, - (type == VERIFY_READ) ? PAGE_READ : - (PAGE_READ | PAGE_WRITE)) == 0; + return page_check_range((target_ulong)addr, size, type) == 0; } /* NOTE __get_user and __put_user use host pointers and don't check access. -- 2.20.1 From: Richard Henderson <richard.henderson@linaro.org> These constants are only ever used with access_ok, and friends. Rather than translating them to PAGE_* bits, let them equal the PAGE_* bits to begin. Reviewed-by: Warner Losh <imp@bsdimp.com> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20210210000223.884088-9-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- bsd-user/qemu.h | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/bsd-user/qemu.h b/bsd-user/qemu.h index XXXXXXX..XXXXXXX 100644 --- a/bsd-user/qemu.h +++ b/bsd-user/qemu.h @@ -XXX,XX +XXX,XX @@ extern unsigned long x86_stack_size; /* user access */ -#define VERIFY_READ 0 -#define VERIFY_WRITE 1 /* implies read access */ +#define VERIFY_READ PAGE_READ +#define VERIFY_WRITE (PAGE_READ | PAGE_WRITE) -static inline int access_ok(int type, abi_ulong addr, abi_ulong size) +static inline bool access_ok(int type, abi_ulong addr, abi_ulong size) { - return page_check_range((target_ulong)addr, size, - (type == VERIFY_READ) ? PAGE_READ : (PAGE_READ | PAGE_WRITE)) == 0; + return page_check_range((target_ulong)addr, size, type) == 0; } /* NOTE __get_user and __put_user use host pointers and don't check access. */ -- 2.20.1 From: Richard Henderson <richard.henderson@linaro.org> This is the only use of guest_addr_valid that does not begin with a guest address, but a host address being transformed to a guest address. We will shortly adjust guest_addr_valid to handle guest memory tags, and the host address should not be subjected to that. Move h2g_valid adjacent to the other h2g macros. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20210210000223.884088-10-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- include/exec/cpu_ldst.h | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/include/exec/cpu_ldst.h b/include/exec/cpu_ldst.h index XXXXXXX..XXXXXXX 100644 --- a/include/exec/cpu_ldst.h +++ b/include/exec/cpu_ldst.h @@ -XXX,XX +XXX,XX @@ typedef uint64_t abi_ptr; #else #define guest_addr_valid(x) ((x) <= GUEST_ADDR_MAX) #endif -#define h2g_valid(x) guest_addr_valid((uintptr_t)(x) - guest_base) static inline bool guest_range_valid(abi_ulong start, abi_ulong len) { return len - 1 <= GUEST_ADDR_MAX && start <= GUEST_ADDR_MAX - len + 1; } +#define h2g_valid(x) \ + (HOST_LONG_BITS <= TARGET_VIRT_ADDR_SPACE_BITS || \ + (uintptr_t)(x) - guest_base <= GUEST_ADDR_MAX) + #define h2g_nocheck(x) ({ \ uintptr_t __ret = (uintptr_t)(x) - guest_base; \ (abi_ptr)__ret; \ -- 2.20.1 From: Richard Henderson <richard.henderson@linaro.org> We must always use GUEST_ADDR_MAX, because even 32-bit hosts can use -R <reserved_va> to restrict the memory address of the guest. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20210210000223.884088-11-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- include/exec/cpu_ldst.h | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/include/exec/cpu_ldst.h b/include/exec/cpu_ldst.h index XXXXXXX..XXXXXXX 100644 --- a/include/exec/cpu_ldst.h +++ b/include/exec/cpu_ldst.h @@ -XXX,XX +XXX,XX @@ typedef uint64_t abi_ptr; /* All direct uses of g2h and h2g need to go away for usermode softmmu. */ #define g2h(x) ((void *)((uintptr_t)(abi_ptr)(x) + guest_base)) -#if HOST_LONG_BITS <= TARGET_VIRT_ADDR_SPACE_BITS -#define guest_addr_valid(x) (1) -#else -#define guest_addr_valid(x) ((x) <= GUEST_ADDR_MAX) -#endif +static inline bool guest_addr_valid(abi_ulong x) +{ + return x <= GUEST_ADDR_MAX; +} static inline bool guest_range_valid(abi_ulong start, abi_ulong len) { -- 2.20.1 From: Richard Henderson <richard.henderson@linaro.org> Provide an identity fallback for target that do not use tagged addresses. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20210210000223.884088-12-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- include/exec/cpu_ldst.h | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/include/exec/cpu_ldst.h b/include/exec/cpu_ldst.h index XXXXXXX..XXXXXXX 100644 --- a/include/exec/cpu_ldst.h +++ b/include/exec/cpu_ldst.h @@ -XXX,XX +XXX,XX @@ typedef uint64_t abi_ptr; #define TARGET_ABI_FMT_ptr "%"PRIx64 #endif +#ifndef TARGET_TAGGED_ADDRESSES +static inline abi_ptr cpu_untagged_addr(CPUState *cs, abi_ptr x) +{ + return x; +} +#endif + /* All direct uses of g2h and h2g need to go away for usermode softmmu. */ #define g2h(x) ((void *)((uintptr_t)(abi_ptr)(x) + guest_base)) -- 2.20.1 From: Richard Henderson <richard.henderson@linaro.org> Use g2h_untagged in contexts that have no cpu, e.g. the binary loaders that operate before the primary cpu is created. As a colollary, target_mmap and friends must use untagged addresses, since they are used by the loaders. Use g2h_untagged on values returned from target_mmap, as the kernel never applies a tag itself. Use g2h_untagged on all pc values. The only current user of tags, aarch64, removes tags from code addresses upon branch, so "pc" is always untagged. Use g2h with the cpu context on hand wherever possible. Use g2h_untagged in lock_user, which will be updated soon. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20210210000223.884088-13-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- include/exec/cpu_ldst.h | 12 +++++- include/exec/exec-all.h | 2 +- linux-user/qemu.h | 6 +-- accel/tcg/translate-all.c | 4 +- accel/tcg/user-exec.c | 48 ++++++++++++------------ linux-user/elfload.c | 12 +++--- linux-user/flatload.c | 2 +- linux-user/hppa/cpu_loop.c | 31 ++++++++-------- linux-user/i386/cpu_loop.c | 4 +- linux-user/mmap.c | 45 +++++++++++----------- linux-user/ppc/signal.c | 4 +- linux-user/syscall.c | 72 +++++++++++++++++++----------------- target/arm/helper-a64.c | 4 +- target/hppa/op_helper.c | 2 +- target/i386/tcg/mem_helper.c | 2 +- target/s390x/mem_helper.c | 4 +- 16 files changed, 135 insertions(+), 119 deletions(-) diff --git a/include/exec/cpu_ldst.h b/include/exec/cpu_ldst.h index XXXXXXX..XXXXXXX 100644 --- a/include/exec/cpu_ldst.h +++ b/include/exec/cpu_ldst.h @@ -XXX,XX +XXX,XX @@ static inline abi_ptr cpu_untagged_addr(CPUState *cs, abi_ptr x) #endif /* All direct uses of g2h and h2g need to go away for usermode softmmu. */ -#define g2h(x) ((void *)((uintptr_t)(abi_ptr)(x) + guest_base)) +static inline void *g2h_untagged(abi_ptr x) +{ + return (void *)((uintptr_t)(x) + guest_base); +} + +static inline void *g2h(CPUState *cs, abi_ptr x) +{ + return g2h_untagged(cpu_untagged_addr(cs, x)); +} static inline bool guest_addr_valid(abi_ulong x) { @@ -XXX,XX +XXX,XX @@ static inline int cpu_ldsw_code(CPUArchState *env, abi_ptr addr) static inline void *tlb_vaddr_to_host(CPUArchState *env, abi_ptr addr, MMUAccessType access_type, int mmu_idx) { - return g2h(addr); + return g2h(env_cpu(env), addr); } #else void *tlb_vaddr_to_host(CPUArchState *env, abi_ptr addr, diff --git a/include/exec/exec-all.h b/include/exec/exec-all.h index XXXXXXX..XXXXXXX 100644 --- a/include/exec/exec-all.h +++ b/include/exec/exec-all.h @@ -XXX,XX +XXX,XX @@ static inline tb_page_addr_t get_page_addr_code_hostp(CPUArchState *env, void **hostp) { if (hostp) { - *hostp = g2h(addr); + *hostp = g2h_untagged(addr); } return addr; } diff --git a/linux-user/qemu.h b/linux-user/qemu.h index XXXXXXX..XXXXXXX 100644 --- a/linux-user/qemu.h +++ b/linux-user/qemu.h @@ -XXX,XX +XXX,XX @@ static inline void *lock_user(int type, abi_ulong guest_addr, long len, int copy return addr; } #else - return g2h(guest_addr); + return g2h_untagged(guest_addr); #endif } @@ -XXX,XX +XXX,XX @@ static inline void unlock_user(void *host_ptr, abi_ulong guest_addr, #ifdef DEBUG_REMAP if (!host_ptr) return; - if (host_ptr == g2h(guest_addr)) + if (host_ptr == g2h_untagged(guest_addr)) return; if (len > 0) - memcpy(g2h(guest_addr), host_ptr, len); + memcpy(g2h_untagged(guest_addr), host_ptr, len); g_free(host_ptr); #endif } diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c index XXXXXXX..XXXXXXX 100644 --- a/accel/tcg/translate-all.c +++ b/accel/tcg/translate-all.c @@ -XXX,XX +XXX,XX @@ static inline void tb_page_add(PageDesc *p, TranslationBlock *tb, prot |= p2->flags; p2->flags &= ~PAGE_WRITE; } - mprotect(g2h(page_addr), qemu_host_page_size, + mprotect(g2h_untagged(page_addr), qemu_host_page_size, (prot & PAGE_BITS) & ~PAGE_WRITE); if (DEBUG_TB_INVALIDATE_GATE) { printf("protecting code page: 0x" TB_PAGE_ADDR_FMT "\n", page_addr); @@ -XXX,XX +XXX,XX @@ int page_unprotect(target_ulong address, uintptr_t pc) } #endif } - mprotect((void *)g2h(host_start), qemu_host_page_size, + mprotect((void *)g2h_untagged(host_start), qemu_host_page_size, prot & PAGE_BITS); } mmap_unlock(); diff --git a/accel/tcg/user-exec.c b/accel/tcg/user-exec.c index XXXXXXX..XXXXXXX 100644 --- a/accel/tcg/user-exec.c +++ b/accel/tcg/user-exec.c @@ -XXX,XX +XXX,XX @@ int probe_access_flags(CPUArchState *env, target_ulong addr, int flags; flags = probe_access_internal(env, addr, 0, access_type, nonfault, ra); - *phost = flags ? NULL : g2h(addr); + *phost = flags ? NULL : g2h(env_cpu(env), addr); return flags; } @@ -XXX,XX +XXX,XX @@ void *probe_access(CPUArchState *env, target_ulong addr, int size, flags = probe_access_internal(env, addr, size, access_type, false, ra); g_assert(flags == 0); - return size ? g2h(addr) : NULL; + return size ? g2h(env_cpu(env), addr) : NULL; } #if defined(__i386__) @@ -XXX,XX +XXX,XX @@ uint32_t cpu_ldub_data(CPUArchState *env, abi_ptr ptr) uint16_t meminfo = trace_mem_get_info(MO_UB, MMU_USER_IDX, false); trace_guest_mem_before_exec(env_cpu(env), ptr, meminfo); - ret = ldub_p(g2h(ptr)); + ret = ldub_p(g2h(env_cpu(env), ptr)); qemu_plugin_vcpu_mem_cb(env_cpu(env), ptr, meminfo); return ret; } @@ -XXX,XX +XXX,XX @@ int cpu_ldsb_data(CPUArchState *env, abi_ptr ptr) uint16_t meminfo = trace_mem_get_info(MO_SB, MMU_USER_IDX, false); trace_guest_mem_before_exec(env_cpu(env), ptr, meminfo); - ret = ldsb_p(g2h(ptr)); + ret = ldsb_p(g2h(env_cpu(env), ptr)); qemu_plugin_vcpu_mem_cb(env_cpu(env), ptr, meminfo); return ret; } @@ -XXX,XX +XXX,XX @@ uint32_t cpu_lduw_be_data(CPUArchState *env, abi_ptr ptr) uint16_t meminfo = trace_mem_get_info(MO_BEUW, MMU_USER_IDX, false); trace_guest_mem_before_exec(env_cpu(env), ptr, meminfo); - ret = lduw_be_p(g2h(ptr)); + ret = lduw_be_p(g2h(env_cpu(env), ptr)); qemu_plugin_vcpu_mem_cb(env_cpu(env), ptr, meminfo); return ret; } @@ -XXX,XX +XXX,XX @@ int cpu_ldsw_be_data(CPUArchState *env, abi_ptr ptr) uint16_t meminfo = trace_mem_get_info(MO_BESW, MMU_USER_IDX, false); trace_guest_mem_before_exec(env_cpu(env), ptr, meminfo); - ret = ldsw_be_p(g2h(ptr)); + ret = ldsw_be_p(g2h(env_cpu(env), ptr)); qemu_plugin_vcpu_mem_cb(env_cpu(env), ptr, meminfo); return ret; } @@ -XXX,XX +XXX,XX @@ uint32_t cpu_ldl_be_data(CPUArchState *env, abi_ptr ptr) uint16_t meminfo = trace_mem_get_info(MO_BEUL, MMU_USER_IDX, false); trace_guest_mem_before_exec(env_cpu(env), ptr, meminfo); - ret = ldl_be_p(g2h(ptr)); + ret = ldl_be_p(g2h(env_cpu(env), ptr)); qemu_plugin_vcpu_mem_cb(env_cpu(env), ptr, meminfo); return ret; } @@ -XXX,XX +XXX,XX @@ uint64_t cpu_ldq_be_data(CPUArchState *env, abi_ptr ptr) uint16_t meminfo = trace_mem_get_info(MO_BEQ, MMU_USER_IDX, false); trace_guest_mem_before_exec(env_cpu(env), ptr, meminfo); - ret = ldq_be_p(g2h(ptr)); + ret = ldq_be_p(g2h(env_cpu(env), ptr)); qemu_plugin_vcpu_mem_cb(env_cpu(env), ptr, meminfo); return ret; } @@ -XXX,XX +XXX,XX @@ uint32_t cpu_lduw_le_data(CPUArchState *env, abi_ptr ptr) uint16_t meminfo = trace_mem_get_info(MO_LEUW, MMU_USER_IDX, false); trace_guest_mem_before_exec(env_cpu(env), ptr, meminfo); - ret = lduw_le_p(g2h(ptr)); + ret = lduw_le_p(g2h(env_cpu(env), ptr)); qemu_plugin_vcpu_mem_cb(env_cpu(env), ptr, meminfo); return ret; } @@ -XXX,XX +XXX,XX @@ int cpu_ldsw_le_data(CPUArchState *env, abi_ptr ptr) uint16_t meminfo = trace_mem_get_info(MO_LESW, MMU_USER_IDX, false); trace_guest_mem_before_exec(env_cpu(env), ptr, meminfo); - ret = ldsw_le_p(g2h(ptr)); + ret = ldsw_le_p(g2h(env_cpu(env), ptr)); qemu_plugin_vcpu_mem_cb(env_cpu(env), ptr, meminfo); return ret; } @@ -XXX,XX +XXX,XX @@ uint32_t cpu_ldl_le_data(CPUArchState *env, abi_ptr ptr) uint16_t meminfo = trace_mem_get_info(MO_LEUL, MMU_USER_IDX, false); trace_guest_mem_before_exec(env_cpu(env), ptr, meminfo); - ret = ldl_le_p(g2h(ptr)); + ret = ldl_le_p(g2h(env_cpu(env), ptr)); qemu_plugin_vcpu_mem_cb(env_cpu(env), ptr, meminfo); return ret; } @@ -XXX,XX +XXX,XX @@ uint64_t cpu_ldq_le_data(CPUArchState *env, abi_ptr ptr) uint16_t meminfo = trace_mem_get_info(MO_LEQ, MMU_USER_IDX, false); trace_guest_mem_before_exec(env_cpu(env), ptr, meminfo); - ret = ldq_le_p(g2h(ptr)); + ret = ldq_le_p(g2h(env_cpu(env), ptr)); qemu_plugin_vcpu_mem_cb(env_cpu(env), ptr, meminfo); return ret; } @@ -XXX,XX +XXX,XX @@ void cpu_stb_data(CPUArchState *env, abi_ptr ptr, uint32_t val) uint16_t meminfo = trace_mem_get_info(MO_UB, MMU_USER_IDX, true); trace_guest_mem_before_exec(env_cpu(env), ptr, meminfo); - stb_p(g2h(ptr), val); + stb_p(g2h(env_cpu(env), ptr), val); qemu_plugin_vcpu_mem_cb(env_cpu(env), ptr, meminfo); } @@ -XXX,XX +XXX,XX @@ void cpu_stw_be_data(CPUArchState *env, abi_ptr ptr, uint32_t val) uint16_t meminfo = trace_mem_get_info(MO_BEUW, MMU_USER_IDX, true); trace_guest_mem_before_exec(env_cpu(env), ptr, meminfo); - stw_be_p(g2h(ptr), val); + stw_be_p(g2h(env_cpu(env), ptr), val); qemu_plugin_vcpu_mem_cb(env_cpu(env), ptr, meminfo); } @@ -XXX,XX +XXX,XX @@ void cpu_stl_be_data(CPUArchState *env, abi_ptr ptr, uint32_t val) uint16_t meminfo = trace_mem_get_info(MO_BEUL, MMU_USER_IDX, true); trace_guest_mem_before_exec(env_cpu(env), ptr, meminfo); - stl_be_p(g2h(ptr), val); + stl_be_p(g2h(env_cpu(env), ptr), val); qemu_plugin_vcpu_mem_cb(env_cpu(env), ptr, meminfo); } @@ -XXX,XX +XXX,XX @@ void cpu_stq_be_data(CPUArchState *env, abi_ptr ptr, uint64_t val) uint16_t meminfo = trace_mem_get_info(MO_BEQ, MMU_USER_IDX, true); trace_guest_mem_before_exec(env_cpu(env), ptr, meminfo); - stq_be_p(g2h(ptr), val); + stq_be_p(g2h(env_cpu(env), ptr), val); qemu_plugin_vcpu_mem_cb(env_cpu(env), ptr, meminfo); } @@ -XXX,XX +XXX,XX @@ void cpu_stw_le_data(CPUArchState *env, abi_ptr ptr, uint32_t val) uint16_t meminfo = trace_mem_get_info(MO_LEUW, MMU_USER_IDX, true); trace_guest_mem_before_exec(env_cpu(env), ptr, meminfo); - stw_le_p(g2h(ptr), val); + stw_le_p(g2h(env_cpu(env), ptr), val); qemu_plugin_vcpu_mem_cb(env_cpu(env), ptr, meminfo); } @@ -XXX,XX +XXX,XX @@ void cpu_stl_le_data(CPUArchState *env, abi_ptr ptr, uint32_t val) uint16_t meminfo = trace_mem_get_info(MO_LEUL, MMU_USER_IDX, true); trace_guest_mem_before_exec(env_cpu(env), ptr, meminfo); - stl_le_p(g2h(ptr), val); + stl_le_p(g2h(env_cpu(env), ptr), val); qemu_plugin_vcpu_mem_cb(env_cpu(env), ptr, meminfo); } @@ -XXX,XX +XXX,XX @@ void cpu_stq_le_data(CPUArchState *env, abi_ptr ptr, uint64_t val) uint16_t meminfo = trace_mem_get_info(MO_LEQ, MMU_USER_IDX, true); trace_guest_mem_before_exec(env_cpu(env), ptr, meminfo); - stq_le_p(g2h(ptr), val); + stq_le_p(g2h(env_cpu(env), ptr), val); qemu_plugin_vcpu_mem_cb(env_cpu(env), ptr, meminfo); } @@ -XXX,XX +XXX,XX @@ uint32_t cpu_ldub_code(CPUArchState *env, abi_ptr ptr) uint32_t ret; set_helper_retaddr(1); - ret = ldub_p(g2h(ptr)); + ret = ldub_p(g2h_untagged(ptr)); clear_helper_retaddr(); return ret; } @@ -XXX,XX +XXX,XX @@ uint32_t cpu_lduw_code(CPUArchState *env, abi_ptr ptr) uint32_t ret; set_helper_retaddr(1); - ret = lduw_p(g2h(ptr)); + ret = lduw_p(g2h_untagged(ptr)); clear_helper_retaddr(); return ret; } @@ -XXX,XX +XXX,XX @@ uint32_t cpu_ldl_code(CPUArchState *env, abi_ptr ptr) uint32_t ret; set_helper_retaddr(1); - ret = ldl_p(g2h(ptr)); + ret = ldl_p(g2h_untagged(ptr)); clear_helper_retaddr(); return ret; } @@ -XXX,XX +XXX,XX @@ uint64_t cpu_ldq_code(CPUArchState *env, abi_ptr ptr) uint64_t ret; set_helper_retaddr(1); - ret = ldq_p(g2h(ptr)); + ret = ldq_p(g2h_untagged(ptr)); clear_helper_retaddr(); return ret; } @@ -XXX,XX +XXX,XX @@ static void *atomic_mmu_lookup(CPUArchState *env, target_ulong addr, if (unlikely(addr & (size - 1))) { cpu_loop_exit_atomic(env_cpu(env), retaddr); } - void *ret = g2h(addr); + void *ret = g2h(env_cpu(env), addr); set_helper_retaddr(retaddr); return ret; } diff --git a/linux-user/elfload.c b/linux-user/elfload.c index XXXXXXX..XXXXXXX 100644 --- a/linux-user/elfload.c +++ b/linux-user/elfload.c @@ -XXX,XX +XXX,XX @@ enum { static bool init_guest_commpage(void) { - void *want = g2h(ARM_COMMPAGE & -qemu_host_page_size); + void *want = g2h_untagged(ARM_COMMPAGE & -qemu_host_page_size); void *addr = mmap(want, qemu_host_page_size, PROT_READ | PROT_WRITE, MAP_ANONYMOUS | MAP_PRIVATE | MAP_FIXED, -1, 0); @@ -XXX,XX +XXX,XX @@ static bool init_guest_commpage(void) } /* Set kernel helper versions; rest of page is 0. */ - __put_user(5, (uint32_t *)g2h(0xffff0ffcu)); + __put_user(5, (uint32_t *)g2h_untagged(0xffff0ffcu)); if (mprotect(addr, qemu_host_page_size, PROT_READ)) { perror("Protecting guest commpage"); @@ -XXX,XX +XXX,XX @@ static void zero_bss(abi_ulong elf_bss, abi_ulong last_bss, int prot) here is still actually needed. For now, continue with it, but merge it with the "normal" mmap that would allocate the bss. */ - host_start = (uintptr_t) g2h(elf_bss); - host_end = (uintptr_t) g2h(last_bss); + host_start = (uintptr_t) g2h_untagged(elf_bss); + host_end = (uintptr_t) g2h_untagged(last_bss); host_map_start = REAL_HOST_PAGE_ALIGN(host_start); if (host_map_start < host_end) { @@ -XXX,XX +XXX,XX @@ static void pgb_have_guest_base(const char *image_name, abi_ulong guest_loaddr, } /* Reserve the address space for the binary, or reserved_va. */ - test = g2h(guest_loaddr); + test = g2h_untagged(guest_loaddr); addr = mmap(test, guest_hiaddr - guest_loaddr, PROT_NONE, flags, -1, 0); if (test != addr) { pgb_fail_in_use(image_name); @@ -XXX,XX +XXX,XX @@ static void pgb_reserved_va(const char *image_name, abi_ulong guest_loaddr, /* Reserve the memory on the host. */ assert(guest_base != 0); - test = g2h(0); + test = g2h_untagged(0); addr = mmap(test, reserved_va, PROT_NONE, flags, -1, 0); if (addr == MAP_FAILED || addr != test) { error_report("Unable to reserve 0x%lx bytes of virtual address " diff --git a/linux-user/flatload.c b/linux-user/flatload.c index XXXXXXX..XXXXXXX 100644 --- a/linux-user/flatload.c +++ b/linux-user/flatload.c @@ -XXX,XX +XXX,XX @@ static int load_flat_file(struct linux_binprm * bprm, } /* zero the BSS. */ - memset(g2h(datapos + data_len), 0, bss_len); + memset(g2h_untagged(datapos + data_len), 0, bss_len); return 0; } diff --git a/linux-user/hppa/cpu_loop.c b/linux-user/hppa/cpu_loop.c index XXXXXXX..XXXXXXX 100644 --- a/linux-user/hppa/cpu_loop.c +++ b/linux-user/hppa/cpu_loop.c @@ -XXX,XX +XXX,XX @@ static abi_ulong hppa_lws(CPUHPPAState *env) { + CPUState *cs = env_cpu(env); uint32_t which = env->gr[20]; abi_ulong addr = env->gr[26]; abi_ulong old = env->gr[25]; @@ -XXX,XX +XXX,XX @@ static abi_ulong hppa_lws(CPUHPPAState *env) } old = tswap32(old); new = tswap32(new); - ret = qatomic_cmpxchg((uint32_t *)g2h(addr), old, new); + ret = qatomic_cmpxchg((uint32_t *)g2h(cs, addr), old, new); ret = tswap32(ret); break; @@ -XXX,XX +XXX,XX @@ static abi_ulong hppa_lws(CPUHPPAState *env) can be host-endian as well. */ switch (size) { case 0: - old = *(uint8_t *)g2h(old); - new = *(uint8_t *)g2h(new); - ret = qatomic_cmpxchg((uint8_t *)g2h(addr), old, new); + old = *(uint8_t *)g2h(cs, old); + new = *(uint8_t *)g2h(cs, new); + ret = qatomic_cmpxchg((uint8_t *)g2h(cs, addr), old, new); ret = ret != old; break; case 1: - old = *(uint16_t *)g2h(old); - new = *(uint16_t *)g2h(new); - ret = qatomic_cmpxchg((uint16_t *)g2h(addr), old, new); + old = *(uint16_t *)g2h(cs, old); + new = *(uint16_t *)g2h(cs, new); + ret = qatomic_cmpxchg((uint16_t *)g2h(cs, addr), old, new); ret = ret != old; break; case 2: - old = *(uint32_t *)g2h(old); - new = *(uint32_t *)g2h(new); - ret = qatomic_cmpxchg((uint32_t *)g2h(addr), old, new); + old = *(uint32_t *)g2h(cs, old); + new = *(uint32_t *)g2h(cs, new); + ret = qatomic_cmpxchg((uint32_t *)g2h(cs, addr), old, new); ret = ret != old; break; case 3: { uint64_t o64, n64, r64; - o64 = *(uint64_t *)g2h(old); - n64 = *(uint64_t *)g2h(new); + o64 = *(uint64_t *)g2h(cs, old); + n64 = *(uint64_t *)g2h(cs, new); #ifdef CONFIG_ATOMIC64 - r64 = qatomic_cmpxchg__nocheck((uint64_t *)g2h(addr), + r64 = qatomic_cmpxchg__nocheck((uint64_t *)g2h(cs, addr), o64, n64); ret = r64 != o64; #else start_exclusive(); - r64 = *(uint64_t *)g2h(addr); + r64 = *(uint64_t *)g2h(cs, addr); ret = 1; if (r64 == o64) { - *(uint64_t *)g2h(addr) = n64; + *(uint64_t *)g2h(cs, addr) = n64; ret = 0; } end_exclusive(); diff --git a/linux-user/i386/cpu_loop.c b/linux-user/i386/cpu_loop.c index XXXXXXX..XXXXXXX 100644 --- a/linux-user/i386/cpu_loop.c +++ b/linux-user/i386/cpu_loop.c @@ -XXX,XX +XXX,XX @@ void target_cpu_copy_regs(CPUArchState *env, struct target_pt_regs *regs) env->idt.base = target_mmap(0, sizeof(uint64_t) * (env->idt.limit + 1), PROT_READ|PROT_WRITE, MAP_ANONYMOUS|MAP_PRIVATE, -1, 0); - idt_table = g2h(env->idt.base); + idt_table = g2h_untagged(env->idt.base); set_idt(0, 0); set_idt(1, 0); set_idt(2, 0); @@ -XXX,XX +XXX,XX @@ void target_cpu_copy_regs(CPUArchState *env, struct target_pt_regs *regs) PROT_READ|PROT_WRITE, MAP_ANONYMOUS|MAP_PRIVATE, -1, 0); env->gdt.limit = sizeof(uint64_t) * TARGET_GDT_ENTRIES - 1; - gdt_table = g2h(env->gdt.base); + gdt_table = g2h_untagged(env->gdt.base); #ifdef TARGET_ABI32 write_dt(&gdt_table[__USER_CS >> 3], 0, 0xfffff, DESC_G_MASK | DESC_B_MASK | DESC_P_MASK | DESC_S_MASK | diff --git a/linux-user/mmap.c b/linux-user/mmap.c index XXXXXXX..XXXXXXX 100644 --- a/linux-user/mmap.c +++ b/linux-user/mmap.c @@ -XXX,XX +XXX,XX @@ int target_mprotect(abi_ulong start, abi_ulong len, int target_prot) } end = host_end; } - ret = mprotect(g2h(host_start), qemu_host_page_size, + ret = mprotect(g2h_untagged(host_start), qemu_host_page_size, prot1 & PAGE_BITS); if (ret != 0) { goto error; @@ -XXX,XX +XXX,XX @@ int target_mprotect(abi_ulong start, abi_ulong len, int target_prot) for (addr = end; addr < host_end; addr += TARGET_PAGE_SIZE) { prot1 |= page_get_flags(addr); } - ret = mprotect(g2h(host_end - qemu_host_page_size), + ret = mprotect(g2h_untagged(host_end - qemu_host_page_size), qemu_host_page_size, prot1 & PAGE_BITS); if (ret != 0) { goto error; @@ -XXX,XX +XXX,XX @@ int target_mprotect(abi_ulong start, abi_ulong len, int target_prot) /* handle the pages in the middle */ if (host_start < host_end) { - ret = mprotect(g2h(host_start), host_end - host_start, host_prot); + ret = mprotect(g2h_untagged(host_start), + host_end - host_start, host_prot); if (ret != 0) { goto error; } @@ -XXX,XX +XXX,XX @@ static int mmap_frag(abi_ulong real_start, int prot1, prot_new; real_end = real_start + qemu_host_page_size; - host_start = g2h(real_start); + host_start = g2h_untagged(real_start); /* get the protection of the target pages outside the mapping */ prot1 = 0; @@ -XXX,XX +XXX,XX @@ static int mmap_frag(abi_ulong real_start, mprotect(host_start, qemu_host_page_size, prot1 | PROT_WRITE); /* read the corresponding file data */ - if (pread(fd, g2h(start), end - start, offset) == -1) + if (pread(fd, g2h_untagged(start), end - start, offset) == -1) return -1; /* put final protection */ @@ -XXX,XX +XXX,XX @@ static int mmap_frag(abi_ulong real_start, mprotect(host_start, qemu_host_page_size, prot_new); } if (prot_new & PROT_WRITE) { - memset(g2h(start), 0, end - start); + memset(g2h_untagged(start), 0, end - start); } } return 0; @@ -XXX,XX +XXX,XX @@ abi_ulong mmap_find_vma(abi_ulong start, abi_ulong size, abi_ulong align) * - mremap() with MREMAP_FIXED flag * - shmat() with SHM_REMAP flag */ - ptr = mmap(g2h(addr), size, PROT_NONE, + ptr = mmap(g2h_untagged(addr), size, PROT_NONE, MAP_ANONYMOUS|MAP_PRIVATE|MAP_NORESERVE, -1, 0); /* ENOMEM, if host address space has no memory */ @@ -XXX,XX +XXX,XX @@ abi_long target_mmap(abi_ulong start, abi_ulong len, int target_prot, /* Note: we prefer to control the mapping address. It is especially important if qemu_host_page_size > qemu_real_host_page_size */ - p = mmap(g2h(start), host_len, host_prot, + p = mmap(g2h_untagged(start), host_len, host_prot, flags | MAP_FIXED | MAP_ANONYMOUS, -1, 0); if (p == MAP_FAILED) { goto fail; @@ -XXX,XX +XXX,XX @@ abi_long target_mmap(abi_ulong start, abi_ulong len, int target_prot, /* update start so that it points to the file position at 'offset' */ host_start = (unsigned long)p; if (!(flags & MAP_ANONYMOUS)) { - p = mmap(g2h(start), len, host_prot, + p = mmap(g2h_untagged(start), len, host_prot, flags | MAP_FIXED, fd, host_offset); if (p == MAP_FAILED) { - munmap(g2h(start), host_len); + munmap(g2h_untagged(start), host_len); goto fail; } host_start += offset - host_offset; @@ -XXX,XX +XXX,XX @@ abi_long target_mmap(abi_ulong start, abi_ulong len, int target_prot, -1, 0); if (retaddr == -1) goto fail; - if (pread(fd, g2h(start), len, offset) == -1) + if (pread(fd, g2h_untagged(start), len, offset) == -1) goto fail; if (!(host_prot & PROT_WRITE)) { ret = target_mprotect(start, len, target_prot); @@ -XXX,XX +XXX,XX @@ abi_long target_mmap(abi_ulong start, abi_ulong len, int target_prot, offset1 = 0; else offset1 = offset + real_start - start; - p = mmap(g2h(real_start), real_end - real_start, + p = mmap(g2h_untagged(real_start), real_end - real_start, host_prot, flags, fd, offset1); if (p == MAP_FAILED) goto fail; @@ -XXX,XX +XXX,XX @@ static void mmap_reserve(abi_ulong start, abi_ulong size) real_end -= qemu_host_page_size; } if (real_start != real_end) { - mmap(g2h(real_start), real_end - real_start, PROT_NONE, + mmap(g2h_untagged(real_start), real_end - real_start, PROT_NONE, MAP_FIXED | MAP_ANONYMOUS | MAP_PRIVATE | MAP_NORESERVE, -1, 0); } @@ -XXX,XX +XXX,XX @@ int target_munmap(abi_ulong start, abi_ulong len) if (reserved_va) { mmap_reserve(real_start, real_end - real_start); } else { - ret = munmap(g2h(real_start), real_end - real_start); + ret = munmap(g2h_untagged(real_start), real_end - real_start); } } @@ -XXX,XX +XXX,XX @@ abi_long target_mremap(abi_ulong old_addr, abi_ulong old_size, mmap_lock(); if (flags & MREMAP_FIXED) { - host_addr = mremap(g2h(old_addr), old_size, new_size, - flags, g2h(new_addr)); + host_addr = mremap(g2h_untagged(old_addr), old_size, new_size, + flags, g2h_untagged(new_addr)); if (reserved_va && host_addr != MAP_FAILED) { /* If new and old addresses overlap then the above mremap will @@ -XXX,XX +XXX,XX @@ abi_long target_mremap(abi_ulong old_addr, abi_ulong old_size, errno = ENOMEM; host_addr = MAP_FAILED; } else { - host_addr = mremap(g2h(old_addr), old_size, new_size, - flags | MREMAP_FIXED, g2h(mmap_start)); + host_addr = mremap(g2h_untagged(old_addr), old_size, new_size, + flags | MREMAP_FIXED, + g2h_untagged(mmap_start)); if (reserved_va) { mmap_reserve(old_addr, old_size); } @@ -XXX,XX +XXX,XX @@ abi_long target_mremap(abi_ulong old_addr, abi_ulong old_size, } } if (prot == 0) { - host_addr = mremap(g2h(old_addr), old_size, new_size, flags); + host_addr = mremap(g2h_untagged(old_addr), + old_size, new_size, flags); if (host_addr != MAP_FAILED) { /* Check if address fits target address space */ if (!guest_range_valid(h2g(host_addr), new_size)) { /* Revert mremap() changes */ - host_addr = mremap(g2h(old_addr), new_size, old_size, - flags); + host_addr = mremap(g2h_untagged(old_addr), + new_size, old_size, flags); errno = ENOMEM; host_addr = MAP_FAILED; } else if (reserved_va && old_size > new_size) { diff --git a/linux-user/ppc/signal.c b/linux-user/ppc/signal.c index XXXXXXX..XXXXXXX 100644 --- a/linux-user/ppc/signal.c +++ b/linux-user/ppc/signal.c @@ -XXX,XX +XXX,XX @@ static void restore_user_regs(CPUPPCState *env, uint64_t v_addr; /* 64-bit needs to recover the pointer to the vectors from the frame */ __get_user(v_addr, &frame->v_regs); - v_regs = g2h(v_addr); + v_regs = g2h(env_cpu(env), v_addr); #else v_regs = (ppc_avr_t *)frame->mc_vregs.altivec; #endif @@ -XXX,XX +XXX,XX @@ void setup_rt_frame(int sig, struct target_sigaction *ka, if (get_ppc64_abi(image) < 2) { /* ELFv1 PPC64 function pointers are pointers to OPD entries. */ struct target_func_ptr *handler = - (struct target_func_ptr *)g2h(ka->_sa_handler); + (struct target_func_ptr *)g2h(env_cpu(env), ka->_sa_handler); env->nip = tswapl(handler->entry); env->gpr[2] = tswapl(handler->toc); } else { diff --git a/linux-user/syscall.c b/linux-user/syscall.c index XXXXXXX..XXXXXXX 100644 --- a/linux-user/syscall.c +++ b/linux-user/syscall.c @@ -XXX,XX +XXX,XX @@ abi_long do_brk(abi_ulong new_brk) /* Heap contents are initialized to zero, as for anonymous * mapped pages. */ if (new_brk > target_brk) { - memset(g2h(target_brk), 0, new_brk - target_brk); + memset(g2h_untagged(target_brk), 0, new_brk - target_brk); } target_brk = new_brk; DEBUGF_BRK(TARGET_ABI_FMT_lx " (new_brk <= brk_page)\n", target_brk); @@ -XXX,XX +XXX,XX @@ abi_long do_brk(abi_ulong new_brk) * come from the remaining part of the previous page: it may * contains garbage data due to a previous heap usage (grown * then shrunken). */ - memset(g2h(target_brk), 0, brk_page - target_brk); + memset(g2h_untagged(target_brk), 0, brk_page - target_brk); target_brk = new_brk; brk_page = HOST_PAGE_ALIGN(target_brk); @@ -XXX,XX +XXX,XX @@ static inline abi_ulong do_shmat(CPUArchState *cpu_env, mmap_lock(); if (shmaddr) - host_raddr = shmat(shmid, (void *)g2h(shmaddr), shmflg); + host_raddr = shmat(shmid, (void *)g2h_untagged(shmaddr), shmflg); else { abi_ulong mmap_start; @@ -XXX,XX +XXX,XX @@ static inline abi_ulong do_shmat(CPUArchState *cpu_env, errno = ENOMEM; host_raddr = (void *)-1; } else - host_raddr = shmat(shmid, g2h(mmap_start), shmflg | SHM_REMAP); + host_raddr = shmat(shmid, g2h_untagged(mmap_start), + shmflg | SHM_REMAP); } if (host_raddr == (void *)-1) { @@ -XXX,XX +XXX,XX @@ static inline abi_long do_shmdt(abi_ulong shmaddr) break; } } - rv = get_errno(shmdt(g2h(shmaddr))); + rv = get_errno(shmdt(g2h_untagged(shmaddr))); mmap_unlock(); @@ -XXX,XX +XXX,XX @@ static abi_long write_ldt(CPUX86State *env, MAP_ANONYMOUS|MAP_PRIVATE, -1, 0); if (env->ldt.base == -1) return -TARGET_ENOMEM; - memset(g2h(env->ldt.base), 0, + memset(g2h_untagged(env->ldt.base), 0, TARGET_LDT_ENTRIES * TARGET_LDT_ENTRY_SIZE); env->ldt.limit = 0xffff; - ldt_table = g2h(env->ldt.base); + ldt_table = g2h_untagged(env->ldt.base); } /* NOTE: same code as Linux kernel */ @@ -XXX,XX +XXX,XX @@ static abi_long do_modify_ldt(CPUX86State *env, int func, abi_ulong ptr, #if defined(TARGET_ABI32) abi_long do_set_thread_area(CPUX86State *env, abi_ulong ptr) { - uint64_t *gdt_table = g2h(env->gdt.base); + uint64_t *gdt_table = g2h_untagged(env->gdt.base); struct target_modify_ldt_ldt_s ldt_info; struct target_modify_ldt_ldt_s *target_ldt_info; int seg_32bit, contents, read_exec_only, limit_in_pages; @@ -XXX,XX +XXX,XX @@ install: static abi_long do_get_thread_area(CPUX86State *env, abi_ulong ptr) { struct target_modify_ldt_ldt_s *target_ldt_info; - uint64_t *gdt_table = g2h(env->gdt.base); + uint64_t *gdt_table = g2h_untagged(env->gdt.base); uint32_t base_addr, limit, flags; int seg_32bit, contents, read_exec_only, limit_in_pages, idx; int seg_not_present, useable, lm; @@ -XXX,XX +XXX,XX @@ static int do_safe_futex(int *uaddr, int op, int val, tricky. However they're probably useless because guest atomic operations won't work either. */ #if defined(TARGET_NR_futex) -static int do_futex(target_ulong uaddr, int op, int val, target_ulong timeout, - target_ulong uaddr2, int val3) +static int do_futex(CPUState *cpu, target_ulong uaddr, int op, int val, + target_ulong timeout, target_ulong uaddr2, int val3) { struct timespec ts, *pts; int base_op; @@ -XXX,XX +XXX,XX @@ static int do_futex(target_ulong uaddr, int op, int val, target_ulong timeout, } else { pts = NULL; } - return do_safe_futex(g2h(uaddr), op, tswap32(val), pts, NULL, val3); + return do_safe_futex(g2h(cpu, uaddr), + op, tswap32(val), pts, NULL, val3); case FUTEX_WAKE: - return do_safe_futex(g2h(uaddr), op, val, NULL, NULL, 0); + return do_safe_futex(g2h(cpu, uaddr), + op, val, NULL, NULL, 0); case FUTEX_FD: - return do_safe_futex(g2h(uaddr), op, val, NULL, NULL, 0); + return do_safe_futex(g2h(cpu, uaddr), + op, val, NULL, NULL, 0); case FUTEX_REQUEUE: case FUTEX_CMP_REQUEUE: case FUTEX_WAKE_OP: @@ -XXX,XX +XXX,XX @@ static int do_futex(target_ulong uaddr, int op, int val, target_ulong timeout, to satisfy the compiler. We do not need to tswap TIMEOUT since it's not compared to guest memory. */ pts = (struct timespec *)(uintptr_t) timeout; - return do_safe_futex(g2h(uaddr), op, val, pts, g2h(uaddr2), + return do_safe_futex(g2h(cpu, uaddr), op, val, pts, g2h(cpu, uaddr2), (base_op == FUTEX_CMP_REQUEUE - ? tswap32(val3) - : val3)); + ? tswap32(val3) : val3)); default: return -TARGET_ENOSYS; } @@ -XXX,XX +XXX,XX @@ static int do_futex(target_ulong uaddr, int op, int val, target_ulong timeout, #endif #if defined(TARGET_NR_futex_time64) -static int do_futex_time64(target_ulong uaddr, int op, int val, target_ulong timeout, +static int do_futex_time64(CPUState *cpu, target_ulong uaddr, int op, + int val, target_ulong timeout, target_ulong uaddr2, int val3) { struct timespec ts, *pts; @@ -XXX,XX +XXX,XX @@ static int do_futex_time64(target_ulong uaddr, int op, int val, target_ulong tim } else { pts = NULL; } - return do_safe_futex(g2h(uaddr), op, tswap32(val), pts, NULL, val3); + return do_safe_futex(g2h(cpu, uaddr), op, + tswap32(val), pts, NULL, val3); case FUTEX_WAKE: - return do_safe_futex(g2h(uaddr), op, val, NULL, NULL, 0); + return do_safe_futex(g2h(cpu, uaddr), op, val, NULL, NULL, 0); case FUTEX_FD: - return do_safe_futex(g2h(uaddr), op, val, NULL, NULL, 0); + return do_safe_futex(g2h(cpu, uaddr), op, val, NULL, NULL, 0); case FUTEX_REQUEUE: case FUTEX_CMP_REQUEUE: case FUTEX_WAKE_OP: @@ -XXX,XX +XXX,XX @@ static int do_futex_time64(target_ulong uaddr, int op, int val, target_ulong tim to satisfy the compiler. We do not need to tswap TIMEOUT since it's not compared to guest memory. */ pts = (struct timespec *)(uintptr_t) timeout; - return do_safe_futex(g2h(uaddr), op, val, pts, g2h(uaddr2), + return do_safe_futex(g2h(cpu, uaddr), op, val, pts, g2h(cpu, uaddr2), (base_op == FUTEX_CMP_REQUEUE - ? tswap32(val3) - : val3)); + ? tswap32(val3) : val3)); default: return -TARGET_ENOSYS; } @@ -XXX,XX +XXX,XX @@ static int open_self_maps(void *cpu_env, int fd) const char *path; max = h2g_valid(max - 1) ? - max : (uintptr_t) g2h(GUEST_ADDR_MAX) + 1; + max : (uintptr_t) g2h_untagged(GUEST_ADDR_MAX) + 1; if (page_check_range(h2g(min), max - min, flags) == -1) { continue; @@ -XXX,XX +XXX,XX @@ static abi_long do_syscall1(void *cpu_env, int num, abi_long arg1, if (ts->child_tidptr) { put_user_u32(0, ts->child_tidptr); - do_sys_futex(g2h(ts->child_tidptr), FUTEX_WAKE, INT_MAX, - NULL, NULL, 0); + do_sys_futex(g2h(cpu, ts->child_tidptr), + FUTEX_WAKE, INT_MAX, NULL, NULL, 0); } thread_cpu = NULL; g_free(ts); @@ -XXX,XX +XXX,XX @@ static abi_long do_syscall1(void *cpu_env, int num, abi_long arg1, if (!arg5) { ret = mount(p, p2, p3, (unsigned long)arg4, NULL); } else { - ret = mount(p, p2, p3, (unsigned long)arg4, g2h(arg5)); + ret = mount(p, p2, p3, (unsigned long)arg4, g2h(cpu, arg5)); } ret = get_errno(ret); @@ -XXX,XX +XXX,XX @@ static abi_long do_syscall1(void *cpu_env, int num, abi_long arg1, /* ??? msync/mlock/munlock are broken for softmmu. */ #ifdef TARGET_NR_msync case TARGET_NR_msync: - return get_errno(msync(g2h(arg1), arg2, arg3)); + return get_errno(msync(g2h(cpu, arg1), arg2, arg3)); #endif #ifdef TARGET_NR_mlock case TARGET_NR_mlock: - return get_errno(mlock(g2h(arg1), arg2)); + return get_errno(mlock(g2h(cpu, arg1), arg2)); #endif #ifdef TARGET_NR_munlock case TARGET_NR_munlock: - return get_errno(munlock(g2h(arg1), arg2)); + return get_errno(munlock(g2h(cpu, arg1), arg2)); #endif #ifdef TARGET_NR_mlockall case TARGET_NR_mlockall: @@ -XXX,XX +XXX,XX @@ static abi_long do_syscall1(void *cpu_env, int num, abi_long arg1, #if defined(TARGET_NR_set_tid_address) && defined(__NR_set_tid_address) case TARGET_NR_set_tid_address: - return get_errno(set_tid_address((int *)g2h(arg1))); + return get_errno(set_tid_address((int *)g2h(cpu, arg1))); #endif case TARGET_NR_tkill: @@ -XXX,XX +XXX,XX @@ static abi_long do_syscall1(void *cpu_env, int num, abi_long arg1, #endif #ifdef TARGET_NR_futex case TARGET_NR_futex: - return do_futex(arg1, arg2, arg3, arg4, arg5, arg6); + return do_futex(cpu, arg1, arg2, arg3, arg4, arg5, arg6); #endif #ifdef TARGET_NR_futex_time64 case TARGET_NR_futex_time64: - return do_futex_time64(arg1, arg2, arg3, arg4, arg5, arg6); + return do_futex_time64(cpu, arg1, arg2, arg3, arg4, arg5, arg6); #endif #if defined(TARGET_NR_inotify_init) && defined(__NR_inotify_init) case TARGET_NR_inotify_init: diff --git a/target/arm/helper-a64.c b/target/arm/helper-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper-a64.c +++ b/target/arm/helper-a64.c @@ -XXX,XX +XXX,XX @@ uint64_t HELPER(paired_cmpxchg64_le)(CPUARMState *env, uint64_t addr, #ifdef CONFIG_USER_ONLY /* ??? Enforce alignment. */ - uint64_t *haddr = g2h(addr); + uint64_t *haddr = g2h(env_cpu(env), addr); set_helper_retaddr(ra); o0 = ldq_le_p(haddr + 0); @@ -XXX,XX +XXX,XX @@ uint64_t HELPER(paired_cmpxchg64_be)(CPUARMState *env, uint64_t addr, #ifdef CONFIG_USER_ONLY /* ??? Enforce alignment. */ - uint64_t *haddr = g2h(addr); + uint64_t *haddr = g2h(env_cpu(env), addr); set_helper_retaddr(ra); o1 = ldq_be_p(haddr + 0); diff --git a/target/hppa/op_helper.c b/target/hppa/op_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/hppa/op_helper.c +++ b/target/hppa/op_helper.c @@ -XXX,XX +XXX,XX @@ static void atomic_store_3(CPUHPPAState *env, target_ulong addr, uint32_t val, #ifdef CONFIG_USER_ONLY uint32_t old, new, cmp; - uint32_t *haddr = g2h(addr - 1); + uint32_t *haddr = g2h(env_cpu(env), addr - 1); old = *haddr; while (1) { new = (old & ~mask) | (val & mask); diff --git a/target/i386/tcg/mem_helper.c b/target/i386/tcg/mem_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/i386/tcg/mem_helper.c +++ b/target/i386/tcg/mem_helper.c @@ -XXX,XX +XXX,XX @@ void helper_cmpxchg8b(CPUX86State *env, target_ulong a0) #ifdef CONFIG_USER_ONLY { - uint64_t *haddr = g2h(a0); + uint64_t *haddr = g2h(env_cpu(env), a0); cmpv = cpu_to_le64(cmpv); newv = cpu_to_le64(newv); oldv = qatomic_cmpxchg__nocheck(haddr, cmpv, newv); diff --git a/target/s390x/mem_helper.c b/target/s390x/mem_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/s390x/mem_helper.c +++ b/target/s390x/mem_helper.c @@ -XXX,XX +XXX,XX @@ static uint32_t do_csst(CPUS390XState *env, uint32_t r3, uint64_t a1, if (parallel) { #ifdef CONFIG_USER_ONLY - uint32_t *haddr = g2h(a1); + uint32_t *haddr = g2h(env_cpu(env), a1); ov = qatomic_cmpxchg__nocheck(haddr, cv, nv); #else TCGMemOpIdx oi = make_memop_idx(MO_TEUL | MO_ALIGN, mem_idx); @@ -XXX,XX +XXX,XX @@ static uint32_t do_csst(CPUS390XState *env, uint32_t r3, uint64_t a1, if (parallel) { #ifdef CONFIG_ATOMIC64 # ifdef CONFIG_USER_ONLY - uint64_t *haddr = g2h(a1); + uint64_t *haddr = g2h(env_cpu(env), a1); ov = qatomic_cmpxchg__nocheck(haddr, cv, nv); # else TCGMemOpIdx oi = make_memop_idx(MO_TEQ | MO_ALIGN, mem_idx); -- 2.20.1 From: Richard Henderson <richard.henderson@linaro.org> We define target_mmap et al as untagged, so that they can be used from the binary loaders. Explicitly call cpu_untagged_addr for munmap, mprotect, mremap syscall entry points. Add a few comments for the syscalls that are exempted by the kernel's tagged-address-abi.rst. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20210210000223.884088-14-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- linux-user/syscall.c | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/linux-user/syscall.c b/linux-user/syscall.c index XXXXXXX..XXXXXXX 100644 --- a/linux-user/syscall.c +++ b/linux-user/syscall.c @@ -XXX,XX +XXX,XX @@ abi_long do_brk(abi_ulong new_brk) abi_long mapped_addr; abi_ulong new_alloc_size; + /* brk pointers are always untagged */ + DEBUGF_BRK("do_brk(" TARGET_ABI_FMT_lx ") -> ", new_brk); if (!new_brk) { @@ -XXX,XX +XXX,XX @@ static inline abi_ulong do_shmat(CPUArchState *cpu_env, int i,ret; abi_ulong shmlba; + /* shmat pointers are always untagged */ + /* find out the length of the shared memory segment */ ret = get_errno(shmctl(shmid, IPC_STAT, &shm_info)); if (is_error(ret)) { @@ -XXX,XX +XXX,XX @@ static inline abi_long do_shmdt(abi_ulong shmaddr) int i; abi_long rv; + /* shmdt pointers are always untagged */ + mmap_lock(); for (i = 0; i < N_SHM_REGIONS; ++i) { @@ -XXX,XX +XXX,XX @@ static abi_long do_syscall1(void *cpu_env, int num, abi_long arg1, v5, v6)); } #else + /* mmap pointers are always untagged */ ret = get_errno(target_mmap(arg1, arg2, arg3, target_to_host_bitmask(arg4, mmap_flags_tbl), arg5, @@ -XXX,XX +XXX,XX @@ static abi_long do_syscall1(void *cpu_env, int num, abi_long arg1, return get_errno(ret); #endif case TARGET_NR_munmap: + arg1 = cpu_untagged_addr(cpu, arg1); return get_errno(target_munmap(arg1, arg2)); case TARGET_NR_mprotect: + arg1 = cpu_untagged_addr(cpu, arg1); { TaskState *ts = cpu->opaque; /* Special hack to detect libc making the stack executable. */ @@ -XXX,XX +XXX,XX @@ static abi_long do_syscall1(void *cpu_env, int num, abi_long arg1, return get_errno(target_mprotect(arg1, arg2, arg3)); #ifdef TARGET_NR_mremap case TARGET_NR_mremap: + arg1 = cpu_untagged_addr(cpu, arg1); + /* mremap new_addr (arg5) is always untagged */ return get_errno(target_mremap(arg1, arg2, arg3, arg4, arg5)); #endif /* ??? msync/mlock/munlock are broken for softmmu. */ -- 2.20.1 From: Richard Henderson <richard.henderson@linaro.org> We're currently open-coding the range check in access_ok; use guest_range_valid when size != 0. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20210210000223.884088-15-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- linux-user/qemu.h | 9 +++------ 1 file changed, 3 insertions(+), 6 deletions(-) diff --git a/linux-user/qemu.h b/linux-user/qemu.h index XXXXXXX..XXXXXXX 100644 --- a/linux-user/qemu.h +++ b/linux-user/qemu.h @@ -XXX,XX +XXX,XX @@ extern unsigned long guest_stack_size; static inline bool access_ok(int type, abi_ulong addr, abi_ulong size) { - if (!guest_addr_valid(addr)) { - return false; - } - if (size != 0 && - (addr + size - 1 < addr || - !guest_addr_valid(addr + size - 1))) { + if (size == 0 + ? !guest_addr_valid(addr) + : !guest_range_valid(addr, size)) { return false; } return page_check_range((target_ulong)addr, size, type) == 0; -- 2.20.1 From: Richard Henderson <richard.henderson@linaro.org> The places that use these are better off using untagged addresses, so do not provide a tagged versions. Rename to make it clear about the address type. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20210210000223.884088-16-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- include/exec/cpu_ldst.h | 4 ++-- linux-user/qemu.h | 4 ++-- accel/tcg/user-exec.c | 3 ++- linux-user/mmap.c | 12 ++++++------ linux-user/syscall.c | 2 +- 5 files changed, 13 insertions(+), 12 deletions(-) diff --git a/include/exec/cpu_ldst.h b/include/exec/cpu_ldst.h index XXXXXXX..XXXXXXX 100644 --- a/include/exec/cpu_ldst.h +++ b/include/exec/cpu_ldst.h @@ -XXX,XX +XXX,XX @@ static inline void *g2h(CPUState *cs, abi_ptr x) return g2h_untagged(cpu_untagged_addr(cs, x)); } -static inline bool guest_addr_valid(abi_ulong x) +static inline bool guest_addr_valid_untagged(abi_ulong x) { return x <= GUEST_ADDR_MAX; } -static inline bool guest_range_valid(abi_ulong start, abi_ulong len) +static inline bool guest_range_valid_untagged(abi_ulong start, abi_ulong len) { return len - 1 <= GUEST_ADDR_MAX && start <= GUEST_ADDR_MAX - len + 1; } diff --git a/linux-user/qemu.h b/linux-user/qemu.h index XXXXXXX..XXXXXXX 100644 --- a/linux-user/qemu.h +++ b/linux-user/qemu.h @@ -XXX,XX +XXX,XX @@ extern unsigned long guest_stack_size; static inline bool access_ok(int type, abi_ulong addr, abi_ulong size) { if (size == 0 - ? !guest_addr_valid(addr) - : !guest_range_valid(addr, size)) { + ? !guest_addr_valid_untagged(addr) + : !guest_range_valid_untagged(addr, size)) { return false; } return page_check_range((target_ulong)addr, size, type) == 0; diff --git a/accel/tcg/user-exec.c b/accel/tcg/user-exec.c index XXXXXXX..XXXXXXX 100644 --- a/accel/tcg/user-exec.c +++ b/accel/tcg/user-exec.c @@ -XXX,XX +XXX,XX @@ static int probe_access_internal(CPUArchState *env, target_ulong addr, g_assert_not_reached(); } - if (!guest_addr_valid(addr) || page_check_range(addr, 1, flags) < 0) { + if (!guest_addr_valid_untagged(addr) || + page_check_range(addr, 1, flags) < 0) { if (nonfault) { return TLB_INVALID_MASK; } else { diff --git a/linux-user/mmap.c b/linux-user/mmap.c index XXXXXXX..XXXXXXX 100644 --- a/linux-user/mmap.c +++ b/linux-user/mmap.c @@ -XXX,XX +XXX,XX @@ int target_mprotect(abi_ulong start, abi_ulong len, int target_prot) } len = TARGET_PAGE_ALIGN(len); end = start + len; - if (!guest_range_valid(start, len)) { + if (!guest_range_valid_untagged(start, len)) { return -TARGET_ENOMEM; } if (len == 0) { @@ -XXX,XX +XXX,XX @@ abi_long target_mmap(abi_ulong start, abi_ulong len, int target_prot, * It can fail only on 64-bit host with 32-bit target. * On any other target/host host mmap() handles this error correctly. */ - if (end < start || !guest_range_valid(start, len)) { + if (end < start || !guest_range_valid_untagged(start, len)) { errno = ENOMEM; goto fail; } @@ -XXX,XX +XXX,XX @@ int target_munmap(abi_ulong start, abi_ulong len) if (start & ~TARGET_PAGE_MASK) return -TARGET_EINVAL; len = TARGET_PAGE_ALIGN(len); - if (len == 0 || !guest_range_valid(start, len)) { + if (len == 0 || !guest_range_valid_untagged(start, len)) { return -TARGET_EINVAL; } @@ -XXX,XX +XXX,XX @@ abi_long target_mremap(abi_ulong old_addr, abi_ulong old_size, int prot; void *host_addr; - if (!guest_range_valid(old_addr, old_size) || + if (!guest_range_valid_untagged(old_addr, old_size) || ((flags & MREMAP_FIXED) && - !guest_range_valid(new_addr, new_size))) { + !guest_range_valid_untagged(new_addr, new_size))) { errno = ENOMEM; return -1; } @@ -XXX,XX +XXX,XX @@ abi_long target_mremap(abi_ulong old_addr, abi_ulong old_size, if (host_addr != MAP_FAILED) { /* Check if address fits target address space */ - if (!guest_range_valid(h2g(host_addr), new_size)) { + if (!guest_range_valid_untagged(h2g(host_addr), new_size)) { /* Revert mremap() changes */ host_addr = mremap(g2h_untagged(old_addr), new_size, old_size, flags); diff --git a/linux-user/syscall.c b/linux-user/syscall.c index XXXXXXX..XXXXXXX 100644 --- a/linux-user/syscall.c +++ b/linux-user/syscall.c @@ -XXX,XX +XXX,XX @@ static inline abi_ulong do_shmat(CPUArchState *cpu_env, return -TARGET_EINVAL; } } - if (!guest_range_valid(shmaddr, shm_info.shm_segsz)) { + if (!guest_range_valid_untagged(shmaddr, shm_info.shm_segsz)) { return -TARGET_EINVAL; } -- 2.20.1 From: Richard Henderson <richard.henderson@linaro.org> Provide both tagged and untagged versions of access_ok. In a few places use thread_cpu, as the user is several callees removed from do_syscall1. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20210210000223.884088-17-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- linux-user/qemu.h | 11 +++++++++-- linux-user/elfload.c | 2 +- linux-user/hppa/cpu_loop.c | 8 ++++---- linux-user/i386/cpu_loop.c | 2 +- linux-user/i386/signal.c | 5 +++-- linux-user/syscall.c | 9 ++++++--- 6 files changed, 24 insertions(+), 13 deletions(-) diff --git a/linux-user/qemu.h b/linux-user/qemu.h index XXXXXXX..XXXXXXX 100644 --- a/linux-user/qemu.h +++ b/linux-user/qemu.h @@ -XXX,XX +XXX,XX @@ extern unsigned long guest_stack_size; #define VERIFY_READ PAGE_READ #define VERIFY_WRITE (PAGE_READ | PAGE_WRITE) -static inline bool access_ok(int type, abi_ulong addr, abi_ulong size) +static inline bool access_ok_untagged(int type, abi_ulong addr, abi_ulong size) { if (size == 0 ? !guest_addr_valid_untagged(addr) @@ -XXX,XX +XXX,XX @@ static inline bool access_ok(int type, abi_ulong addr, abi_ulong size) return page_check_range((target_ulong)addr, size, type) == 0; } +static inline bool access_ok(CPUState *cpu, int type, + abi_ulong addr, abi_ulong size) +{ + return access_ok_untagged(type, cpu_untagged_addr(cpu, addr), size); +} + /* NOTE __get_user and __put_user use host pointers and don't check access. These are usually used to access struct data members once the struct has been locked - usually with lock_user_struct. */ @@ -XXX,XX +XXX,XX @@ abi_long copy_to_user(abi_ulong gaddr, void *hptr, size_t len); host area will have the same contents as the guest. */ static inline void *lock_user(int type, abi_ulong guest_addr, long len, int copy) { - if (!access_ok(type, guest_addr, len)) + if (!access_ok_untagged(type, guest_addr, len)) { return NULL; + } #ifdef DEBUG_REMAP { void *addr; diff --git a/linux-user/elfload.c b/linux-user/elfload.c index XXXXXXX..XXXXXXX 100644 --- a/linux-user/elfload.c +++ b/linux-user/elfload.c @@ -XXX,XX +XXX,XX @@ static int vma_get_mapping_count(const struct mm_struct *mm) static abi_ulong vma_dump_size(const struct vm_area_struct *vma) { /* if we cannot even read the first page, skip it */ - if (!access_ok(VERIFY_READ, vma->vma_start, TARGET_PAGE_SIZE)) + if (!access_ok_untagged(VERIFY_READ, vma->vma_start, TARGET_PAGE_SIZE)) return (0); /* diff --git a/linux-user/hppa/cpu_loop.c b/linux-user/hppa/cpu_loop.c index XXXXXXX..XXXXXXX 100644 --- a/linux-user/hppa/cpu_loop.c +++ b/linux-user/hppa/cpu_loop.c @@ -XXX,XX +XXX,XX @@ static abi_ulong hppa_lws(CPUHPPAState *env) return -TARGET_ENOSYS; case 0: /* elf32 atomic 32bit cmpxchg */ - if ((addr & 3) || !access_ok(VERIFY_WRITE, addr, 4)) { + if ((addr & 3) || !access_ok(cs, VERIFY_WRITE, addr, 4)) { return -TARGET_EFAULT; } old = tswap32(old); @@ -XXX,XX +XXX,XX @@ static abi_ulong hppa_lws(CPUHPPAState *env) return -TARGET_ENOSYS; } if (((addr | old | new) & ((1 << size) - 1)) - || !access_ok(VERIFY_WRITE, addr, 1 << size) - || !access_ok(VERIFY_READ, old, 1 << size) - || !access_ok(VERIFY_READ, new, 1 << size)) { + || !access_ok(cs, VERIFY_WRITE, addr, 1 << size) + || !access_ok(cs, VERIFY_READ, old, 1 << size) + || !access_ok(cs, VERIFY_READ, new, 1 << size)) { return -TARGET_EFAULT; } /* Note that below we use host-endian loads so that the cmpxchg diff --git a/linux-user/i386/cpu_loop.c b/linux-user/i386/cpu_loop.c index XXXXXXX..XXXXXXX 100644 --- a/linux-user/i386/cpu_loop.c +++ b/linux-user/i386/cpu_loop.c @@ -XXX,XX +XXX,XX @@ static bool write_ok_or_segv(CPUX86State *env, abi_ptr addr, size_t len) * For all the vsyscalls, NULL means "don't write anything" not * "write it at address 0". */ - if (addr == 0 || access_ok(VERIFY_WRITE, addr, len)) { + if (addr == 0 || access_ok(env_cpu(env), VERIFY_WRITE, addr, len)) { return true; } diff --git a/linux-user/i386/signal.c b/linux-user/i386/signal.c index XXXXXXX..XXXXXXX 100644 --- a/linux-user/i386/signal.c +++ b/linux-user/i386/signal.c @@ -XXX,XX +XXX,XX @@ restore_sigcontext(CPUX86State *env, struct target_sigcontext *sc) fpstate_addr = tswapl(sc->fpstate); if (fpstate_addr != 0) { - if (!access_ok(VERIFY_READ, fpstate_addr, - sizeof(struct target_fpstate))) + if (!access_ok(env_cpu(env), VERIFY_READ, fpstate_addr, + sizeof(struct target_fpstate))) { goto badframe; + } #ifndef TARGET_X86_64 cpu_x86_frstor(env, fpstate_addr, 1); #else diff --git a/linux-user/syscall.c b/linux-user/syscall.c index XXXXXXX..XXXXXXX 100644 --- a/linux-user/syscall.c +++ b/linux-user/syscall.c @@ -XXX,XX +XXX,XX @@ static abi_long do_accept4(int fd, abi_ulong target_addr, return -TARGET_EINVAL; } - if (!access_ok(VERIFY_WRITE, target_addr, addrlen)) + if (!access_ok(thread_cpu, VERIFY_WRITE, target_addr, addrlen)) { return -TARGET_EFAULT; + } addr = alloca(addrlen); @@ -XXX,XX +XXX,XX @@ static abi_long do_getpeername(int fd, abi_ulong target_addr, return -TARGET_EINVAL; } - if (!access_ok(VERIFY_WRITE, target_addr, addrlen)) + if (!access_ok(thread_cpu, VERIFY_WRITE, target_addr, addrlen)) { return -TARGET_EFAULT; + } addr = alloca(addrlen); @@ -XXX,XX +XXX,XX @@ static abi_long do_getsockname(int fd, abi_ulong target_addr, return -TARGET_EINVAL; } - if (!access_ok(VERIFY_WRITE, target_addr, addrlen)) + if (!access_ok(thread_cpu, VERIFY_WRITE, target_addr, addrlen)) { return -TARGET_EFAULT; + } addr = alloca(addrlen); -- 2.20.1 From: Richard Henderson <richard.henderson@linaro.org> These functions are not small, except for unlock_user without debugging enabled. Move them out of line, and add missing braces on the way. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20210210000223.884088-18-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- linux-user/qemu.h | 45 ++++++------------------------------------- linux-user/uaccess.c | 46 ++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 52 insertions(+), 39 deletions(-) diff --git a/linux-user/qemu.h b/linux-user/qemu.h index XXXXXXX..XXXXXXX 100644 --- a/linux-user/qemu.h +++ b/linux-user/qemu.h @@ -XXX,XX +XXX,XX @@ abi_long copy_to_user(abi_ulong gaddr, void *hptr, size_t len); /* Lock an area of guest memory into the host. If copy is true then the host area will have the same contents as the guest. */ -static inline void *lock_user(int type, abi_ulong guest_addr, long len, int copy) -{ - if (!access_ok_untagged(type, guest_addr, len)) { - return NULL; - } -#ifdef DEBUG_REMAP - { - void *addr; - addr = g_malloc(len); - if (copy) - memcpy(addr, g2h(guest_addr), len); - else - memset(addr, 0, len); - return addr; - } -#else - return g2h_untagged(guest_addr); -#endif -} +void *lock_user(int type, abi_ulong guest_addr, long len, int copy); /* Unlock an area of guest memory. The first LEN bytes must be flushed back to guest memory. host_ptr = NULL is explicitly allowed and does nothing. */ -static inline void unlock_user(void *host_ptr, abi_ulong guest_addr, - long len) -{ - #ifdef DEBUG_REMAP - if (!host_ptr) - return; - if (host_ptr == g2h_untagged(guest_addr)) - return; - if (len > 0) - memcpy(g2h_untagged(guest_addr), host_ptr, len); - g_free(host_ptr); +static inline void unlock_user(void *host_ptr, abi_ulong guest_addr, long len) +{ } +#else +void unlock_user(void *host_ptr, abi_ulong guest_addr, long len); #endif -} /* Return the length of a string in target memory or -TARGET_EFAULT if access error. */ abi_long target_strlen(abi_ulong gaddr); /* Like lock_user but for null terminated strings. */ -static inline void *lock_user_string(abi_ulong guest_addr) -{ - abi_long len; - len = target_strlen(guest_addr); - if (len < 0) - return NULL; - return lock_user(VERIFY_READ, guest_addr, (long)(len + 1), 1); -} +void *lock_user_string(abi_ulong guest_addr); /* Helper macros for locking/unlocking a target struct. */ #define lock_user_struct(type, host_ptr, guest_addr, copy) \ diff --git a/linux-user/uaccess.c b/linux-user/uaccess.c index XXXXXXX..XXXXXXX 100644 --- a/linux-user/uaccess.c +++ b/linux-user/uaccess.c @@ -XXX,XX +XXX,XX @@ #include "qemu.h" +void *lock_user(int type, abi_ulong guest_addr, long len, int copy) +{ + if (!access_ok_untagged(type, guest_addr, len)) { + return NULL; + } +#ifdef DEBUG_REMAP + { + void *addr; + addr = g_malloc(len); + if (copy) { + memcpy(addr, g2h(guest_addr), len); + } else { + memset(addr, 0, len); + } + return addr; + } +#else + return g2h_untagged(guest_addr); +#endif +} + +#ifdef DEBUG_REMAP +void unlock_user(void *host_ptr, abi_ulong guest_addr, long len); +{ + if (!host_ptr) { + return; + } + if (host_ptr == g2h_untagged(guest_addr)) { + return; + } + if (len > 0) { + memcpy(g2h_untagged(guest_addr), host_ptr, len); + } + g_free(host_ptr); +} +#endif + +void *lock_user_string(abi_ulong guest_addr) +{ + abi_long len = target_strlen(guest_addr); + if (len < 0) { + return NULL; + } + return lock_user(VERIFY_READ, guest_addr, (long)(len + 1), 1); +} + /* copy_from_user() and copy_to_user() are usually used to copy data * buffers between the target and host. These internally perform * locking/unlocking of the memory. -- 2.20.1 From: Richard Henderson <richard.henderson@linaro.org> For copy_*_user, only 0 and -TARGET_EFAULT are returned; no need to involve abi_long. Use size_t for lengths. Use bool for the lock_user copy argument. Use ssize_t for target_strlen, because we can't overflow the host memory space. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20210210000223.884088-19-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- linux-user/qemu.h | 14 ++++++-------- linux-user/uaccess.c | 45 ++++++++++++++++++++++---------------------- 2 files changed, 29 insertions(+), 30 deletions(-) diff --git a/linux-user/qemu.h b/linux-user/qemu.h index XXXXXXX..XXXXXXX 100644 --- a/linux-user/qemu.h +++ b/linux-user/qemu.h @@ -XXX,XX +XXX,XX @@ #include "exec/cpu_ldst.h" #undef DEBUG_REMAP -#ifdef DEBUG_REMAP -#endif /* DEBUG_REMAP */ #include "exec/user/abitypes.h" @@ -XXX,XX +XXX,XX @@ static inline bool access_ok(CPUState *cpu, int type, * buffers between the target and host. These internally perform * locking/unlocking of the memory. */ -abi_long copy_from_user(void *hptr, abi_ulong gaddr, size_t len); -abi_long copy_to_user(abi_ulong gaddr, void *hptr, size_t len); +int copy_from_user(void *hptr, abi_ulong gaddr, size_t len); +int copy_to_user(abi_ulong gaddr, void *hptr, size_t len); /* Functions for accessing guest memory. The tget and tput functions read/write single values, byteswapping as necessary. The lock_user function @@ -XXX,XX +XXX,XX @@ abi_long copy_to_user(abi_ulong gaddr, void *hptr, size_t len); /* Lock an area of guest memory into the host. If copy is true then the host area will have the same contents as the guest. */ -void *lock_user(int type, abi_ulong guest_addr, long len, int copy); +void *lock_user(int type, abi_ulong guest_addr, size_t len, bool copy); /* Unlock an area of guest memory. The first LEN bytes must be flushed back to guest memory. host_ptr = NULL is explicitly allowed and does nothing. */ -#ifdef DEBUG_REMAP -static inline void unlock_user(void *host_ptr, abi_ulong guest_addr, long len) +#ifndef DEBUG_REMAP +static inline void unlock_user(void *host_ptr, abi_ulong guest_addr, size_t len) { } #else void unlock_user(void *host_ptr, abi_ulong guest_addr, long len); @@ -XXX,XX +XXX,XX @@ void unlock_user(void *host_ptr, abi_ulong guest_addr, long len); /* Return the length of a string in target memory or -TARGET_EFAULT if access error. */ -abi_long target_strlen(abi_ulong gaddr); +ssize_t target_strlen(abi_ulong gaddr); /* Like lock_user but for null terminated strings. */ void *lock_user_string(abi_ulong guest_addr); diff --git a/linux-user/uaccess.c b/linux-user/uaccess.c index XXXXXXX..XXXXXXX 100644 --- a/linux-user/uaccess.c +++ b/linux-user/uaccess.c @@ -XXX,XX +XXX,XX @@ #include "qemu.h" -void *lock_user(int type, abi_ulong guest_addr, long len, int copy) +void *lock_user(int type, abi_ulong guest_addr, size_t len, bool copy) { if (!access_ok_untagged(type, guest_addr, len)) { return NULL; @@ -XXX,XX +XXX,XX @@ void *lock_user(int type, abi_ulong guest_addr, long len, int copy) } #ifdef DEBUG_REMAP -void unlock_user(void *host_ptr, abi_ulong guest_addr, long len); +void unlock_user(void *host_ptr, abi_ulong guest_addr, size_t len); { if (!host_ptr) { return; @@ -XXX,XX +XXX,XX @@ void unlock_user(void *host_ptr, abi_ulong guest_addr, long len); if (host_ptr == g2h_untagged(guest_addr)) { return; } - if (len > 0) { + if (len != 0) { memcpy(g2h_untagged(guest_addr), host_ptr, len); } g_free(host_ptr); @@ -XXX,XX +XXX,XX @@ void unlock_user(void *host_ptr, abi_ulong guest_addr, long len); void *lock_user_string(abi_ulong guest_addr) { - abi_long len = target_strlen(guest_addr); + ssize_t len = target_strlen(guest_addr); if (len < 0) { return NULL; } - return lock_user(VERIFY_READ, guest_addr, (long)(len + 1), 1); + return lock_user(VERIFY_READ, guest_addr, (size_t)len + 1, 1); } /* copy_from_user() and copy_to_user() are usually used to copy data * buffers between the target and host. These internally perform * locking/unlocking of the memory. */ -abi_long copy_from_user(void *hptr, abi_ulong gaddr, size_t len) +int copy_from_user(void *hptr, abi_ulong gaddr, size_t len) { - abi_long ret = 0; - void *ghptr; + int ret = 0; + void *ghptr = lock_user(VERIFY_READ, gaddr, len, 1); - if ((ghptr = lock_user(VERIFY_READ, gaddr, len, 1))) { + if (ghptr) { memcpy(hptr, ghptr, len); unlock_user(ghptr, gaddr, 0); - } else + } else { ret = -TARGET_EFAULT; - + } return ret; } - -abi_long copy_to_user(abi_ulong gaddr, void *hptr, size_t len) +int copy_to_user(abi_ulong gaddr, void *hptr, size_t len) { - abi_long ret = 0; - void *ghptr; + int ret = 0; + void *ghptr = lock_user(VERIFY_WRITE, gaddr, len, 0); - if ((ghptr = lock_user(VERIFY_WRITE, gaddr, len, 0))) { + if (ghptr) { memcpy(ghptr, hptr, len); unlock_user(ghptr, gaddr, len); - } else + } else { ret = -TARGET_EFAULT; + } return ret; } /* Return the length of a string in target memory or -TARGET_EFAULT if access error */ -abi_long target_strlen(abi_ulong guest_addr1) +ssize_t target_strlen(abi_ulong guest_addr1) { uint8_t *ptr; abi_ulong guest_addr; - int max_len, len; + size_t max_len, len; guest_addr = guest_addr1; for(;;) { @@ -XXX,XX +XXX,XX @@ abi_long target_strlen(abi_ulong guest_addr1) unlock_user(ptr, guest_addr, 0); guest_addr += len; /* we don't allow wrapping or integer overflow */ - if (guest_addr == 0 || - (guest_addr - guest_addr1) > 0x7fffffff) + if (guest_addr == 0 || (guest_addr - guest_addr1) > 0x7fffffff) { return -TARGET_EFAULT; - if (len != max_len) + } + if (len != max_len) { break; + } } return guest_addr - guest_addr1; } -- 2.20.1 From: Richard Henderson <richard.henderson@linaro.org> Resolve the untagged address once, using thread_cpu. Tidy the DEBUG_REMAP code using glib routines. Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Message-id: 20210210000223.884088-20-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- linux-user/uaccess.c | 27 ++++++++++++++------------- 1 file changed, 14 insertions(+), 13 deletions(-) diff --git a/linux-user/uaccess.c b/linux-user/uaccess.c index XXXXXXX..XXXXXXX 100644 --- a/linux-user/uaccess.c +++ b/linux-user/uaccess.c @@ -XXX,XX +XXX,XX @@ void *lock_user(int type, abi_ulong guest_addr, size_t len, bool copy) { + void *host_addr; + + guest_addr = cpu_untagged_addr(thread_cpu, guest_addr); if (!access_ok_untagged(type, guest_addr, len)) { return NULL; } + host_addr = g2h_untagged(guest_addr); #ifdef DEBUG_REMAP - { - void *addr; - addr = g_malloc(len); - if (copy) { - memcpy(addr, g2h(guest_addr), len); - } else { - memset(addr, 0, len); - } - return addr; + if (copy) { + host_addr = g_memdup(host_addr, len); + } else { + host_addr = g_malloc0(len); } -#else - return g2h_untagged(guest_addr); #endif + return host_addr; } #ifdef DEBUG_REMAP void unlock_user(void *host_ptr, abi_ulong guest_addr, size_t len); { + void *host_ptr_conv; + if (!host_ptr) { return; } - if (host_ptr == g2h_untagged(guest_addr)) { + host_ptr_conv = g2h(thread_cpu, guest_addr); + if (host_ptr == host_ptr_conv) { return; } if (len != 0) { - memcpy(g2h_untagged(guest_addr), host_ptr, len); + memcpy(host_ptr_conv, host_ptr, len); } g_free(host_ptr); } -- 2.20.1 From: Richard Henderson <richard.henderson@linaro.org> This is the prctl bit that controls whether syscalls accept tagged addresses. See Documentation/arm64/tagged-address-abi.rst in the linux kernel. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20210210000223.884088-21-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- linux-user/aarch64/target_syscall.h | 4 ++++ target/arm/cpu-param.h | 3 +++ target/arm/cpu.h | 31 +++++++++++++++++++++++++++++ linux-user/syscall.c | 24 ++++++++++++++++++++++ 4 files changed, 62 insertions(+) diff --git a/linux-user/aarch64/target_syscall.h b/linux-user/aarch64/target_syscall.h index XXXXXXX..XXXXXXX 100644 --- a/linux-user/aarch64/target_syscall.h +++ b/linux-user/aarch64/target_syscall.h @@ -XXX,XX +XXX,XX @@ struct target_pt_regs { # define TARGET_PR_PAC_APDBKEY (1 << 3) # define TARGET_PR_PAC_APGAKEY (1 << 4) +#define TARGET_PR_SET_TAGGED_ADDR_CTRL 55 +#define TARGET_PR_GET_TAGGED_ADDR_CTRL 56 +# define TARGET_PR_TAGGED_ADDR_ENABLE (1UL << 0) + #endif /* AARCH64_TARGET_SYSCALL_H */ diff --git a/target/arm/cpu-param.h b/target/arm/cpu-param.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpu-param.h +++ b/target/arm/cpu-param.h @@ -XXX,XX +XXX,XX @@ #ifdef CONFIG_USER_ONLY #define TARGET_PAGE_BITS 12 +# ifdef TARGET_AARCH64 +# define TARGET_TAGGED_ADDRESSES +# endif #else /* * ARMv7 and later CPUs have 4K pages minimum, but ARMv5 and v6 diff --git a/target/arm/cpu.h b/target/arm/cpu.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -XXX,XX +XXX,XX @@ typedef struct CPUARMState { const struct arm_boot_info *boot_info; /* Store GICv3CPUState to access from this struct */ void *gicv3state; + +#ifdef TARGET_TAGGED_ADDRESSES + /* Linux syscall tagged address support */ + bool tagged_addr_enable; +#endif } CPUARMState; static inline void set_feature(CPUARMState *env, int feature) @@ -XXX,XX +XXX,XX @@ static inline MemTxAttrs *typecheck_memtxattrs(MemTxAttrs *x) */ #define PAGE_BTI PAGE_TARGET_1 +#ifdef TARGET_TAGGED_ADDRESSES +/** + * cpu_untagged_addr: + * @cs: CPU context + * @x: tagged address + * + * Remove any address tag from @x. This is explicitly related to the + * linux syscall TIF_TAGGED_ADDR setting, not TBI in general. + * + * There should be a better place to put this, but we need this in + * include/exec/cpu_ldst.h, and not some place linux-user specific. + */ +static inline target_ulong cpu_untagged_addr(CPUState *cs, target_ulong x) +{ + ARMCPU *cpu = ARM_CPU(cs); + if (cpu->env.tagged_addr_enable) { + /* + * TBI is enabled for userspace but not kernelspace addresses. + * Only clear the tag if bit 55 is clear. + */ + x &= sextract64(x, 0, 56); + } + return x; +} +#endif + /* * Naming convention for isar_feature functions: * Functions which test 32-bit ID registers should have _aa32_ in diff --git a/linux-user/syscall.c b/linux-user/syscall.c index XXXXXXX..XXXXXXX 100644 --- a/linux-user/syscall.c +++ b/linux-user/syscall.c @@ -XXX,XX +XXX,XX @@ static abi_long do_syscall1(void *cpu_env, int num, abi_long arg1, } } return -TARGET_EINVAL; + case TARGET_PR_SET_TAGGED_ADDR_CTRL: + { + abi_ulong valid_mask = TARGET_PR_TAGGED_ADDR_ENABLE; + CPUARMState *env = cpu_env; + + if ((arg2 & ~valid_mask) || arg3 || arg4 || arg5) { + return -TARGET_EINVAL; + } + env->tagged_addr_enable = arg2 & TARGET_PR_TAGGED_ADDR_ENABLE; + return 0; + } + case TARGET_PR_GET_TAGGED_ADDR_CTRL: + { + abi_long ret = 0; + CPUARMState *env = cpu_env; + + if (arg2 || arg3 || arg4 || arg5) { + return -TARGET_EINVAL; + } + if (env->tagged_addr_enable) { + ret |= TARGET_PR_TAGGED_ADDR_ENABLE; + } + return ret; + } #endif /* AARCH64 */ case PR_GET_SECCOMP: case PR_SET_SECCOMP: -- 2.20.1 From: Richard Henderson <richard.henderson@linaro.org> Use simple arithmetic instead of a conditional move when tbi0 != tbi1. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20210210000223.884088-22-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-a64.c | 25 ++++++++++++++----------- 1 file changed, 14 insertions(+), 11 deletions(-) diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-a64.c +++ b/target/arm/translate-a64.c @@ -XXX,XX +XXX,XX @@ static void gen_top_byte_ignore(DisasContext *s, TCGv_i64 dst, /* Sign-extend from bit 55. */ tcg_gen_sextract_i64(dst, src, 0, 56); - if (tbi != 3) { - TCGv_i64 tcg_zero = tcg_const_i64(0); - - /* - * The two TBI bits differ. - * If tbi0, then !tbi1: only use the extension if positive. - * if !tbi0, then tbi1: only use the extension if negative. - */ - tcg_gen_movcond_i64(tbi == 1 ? TCG_COND_GE : TCG_COND_LT, - dst, dst, tcg_zero, dst, src); - tcg_temp_free_i64(tcg_zero); + switch (tbi) { + case 1: + /* tbi0 but !tbi1: only use the extension if positive */ + tcg_gen_and_i64(dst, dst, src); + break; + case 2: + /* !tbi0 but tbi1: only use the extension if negative */ + tcg_gen_or_i64(dst, dst, src); + break; + case 3: + /* tbi0 and tbi1: always use the extension */ + break; + default: + g_assert_not_reached(); } } } -- 2.20.1 From: Richard Henderson <richard.henderson@linaro.org> We were fudging TBI1 enabled to speed up the generated code. Now that we've improved the code generation, remove this. Also, tidy the comment to reflect the current code. The pauth test was testing a kernel address (-1) and making incorrect assumptions about TBI1; stick to userland addresses. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20210210000223.884088-23-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/internals.h | 4 ++-- target/arm/cpu.c | 10 +++------- tests/tcg/aarch64/pauth-2.c | 1 - 3 files changed, 5 insertions(+), 10 deletions(-) diff --git a/target/arm/internals.h b/target/arm/internals.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/internals.h +++ b/target/arm/internals.h @@ -XXX,XX +XXX,XX @@ static inline bool tcma_check(uint32_t desc, int bit55, int ptr_tag) */ static inline uint64_t useronly_clean_ptr(uint64_t ptr) { - /* TBI is known to be enabled. */ #ifdef CONFIG_USER_ONLY - ptr = sextract64(ptr, 0, 56); + /* TBI0 is known to be enabled, while TBI1 is disabled. */ + ptr &= sextract64(ptr, 0, 56); #endif return ptr; } diff --git a/target/arm/cpu.c b/target/arm/cpu.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpu.c +++ b/target/arm/cpu.c @@ -XXX,XX +XXX,XX @@ static void arm_cpu_reset(DeviceState *dev) env->vfp.zcr_el[1] = MIN(cpu->sve_max_vq - 1, 3); } /* - * Enable TBI0 and TBI1. While the real kernel only enables TBI0, - * turning on both here will produce smaller code and otherwise - * make no difference to the user-level emulation. - * - * In sve_probe_page, we assume that this is set. - * Do not modify this without other changes. + * Enable TBI0 but not TBI1. + * Note that this must match useronly_clean_ptr. */ - env->cp15.tcr_el[1].raw_tcr = (3ULL << 37); + env->cp15.tcr_el[1].raw_tcr = (1ULL << 37); #else /* Reset into the highest available EL */ if (arm_feature(env, ARM_FEATURE_EL3)) { diff --git a/tests/tcg/aarch64/pauth-2.c b/tests/tcg/aarch64/pauth-2.c index XXXXXXX..XXXXXXX 100644 --- a/tests/tcg/aarch64/pauth-2.c +++ b/tests/tcg/aarch64/pauth-2.c @@ -XXX,XX +XXX,XX @@ void do_test(uint64_t value) int main() { do_test(0); - do_test(-1); do_test(0xda004acedeadbeefull); return 0; } -- 2.20.1 From: Richard Henderson <richard.henderson@linaro.org> These prctl fields are required for the function of MTE. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20210210000223.884088-24-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- linux-user/aarch64/target_syscall.h | 9 ++++++ linux-user/syscall.c | 43 +++++++++++++++++++++++++++++ 2 files changed, 52 insertions(+) diff --git a/linux-user/aarch64/target_syscall.h b/linux-user/aarch64/target_syscall.h index XXXXXXX..XXXXXXX 100644 --- a/linux-user/aarch64/target_syscall.h +++ b/linux-user/aarch64/target_syscall.h @@ -XXX,XX +XXX,XX @@ struct target_pt_regs { #define TARGET_PR_SET_TAGGED_ADDR_CTRL 55 #define TARGET_PR_GET_TAGGED_ADDR_CTRL 56 # define TARGET_PR_TAGGED_ADDR_ENABLE (1UL << 0) +/* MTE tag check fault modes */ +# define TARGET_PR_MTE_TCF_SHIFT 1 +# define TARGET_PR_MTE_TCF_NONE (0UL << TARGET_PR_MTE_TCF_SHIFT) +# define TARGET_PR_MTE_TCF_SYNC (1UL << TARGET_PR_MTE_TCF_SHIFT) +# define TARGET_PR_MTE_TCF_ASYNC (2UL << TARGET_PR_MTE_TCF_SHIFT) +# define TARGET_PR_MTE_TCF_MASK (3UL << TARGET_PR_MTE_TCF_SHIFT) +/* MTE tag inclusion mask */ +# define TARGET_PR_MTE_TAG_SHIFT 3 +# define TARGET_PR_MTE_TAG_MASK (0xffffUL << TARGET_PR_MTE_TAG_SHIFT) #endif /* AARCH64_TARGET_SYSCALL_H */ diff --git a/linux-user/syscall.c b/linux-user/syscall.c index XXXXXXX..XXXXXXX 100644 --- a/linux-user/syscall.c +++ b/linux-user/syscall.c @@ -XXX,XX +XXX,XX @@ static abi_long do_syscall1(void *cpu_env, int num, abi_long arg1, { abi_ulong valid_mask = TARGET_PR_TAGGED_ADDR_ENABLE; CPUARMState *env = cpu_env; + ARMCPU *cpu = env_archcpu(env); + + if (cpu_isar_feature(aa64_mte, cpu)) { + valid_mask |= TARGET_PR_MTE_TCF_MASK; + valid_mask |= TARGET_PR_MTE_TAG_MASK; + } if ((arg2 & ~valid_mask) || arg3 || arg4 || arg5) { return -TARGET_EINVAL; } env->tagged_addr_enable = arg2 & TARGET_PR_TAGGED_ADDR_ENABLE; + + if (cpu_isar_feature(aa64_mte, cpu)) { + switch (arg2 & TARGET_PR_MTE_TCF_MASK) { + case TARGET_PR_MTE_TCF_NONE: + case TARGET_PR_MTE_TCF_SYNC: + case TARGET_PR_MTE_TCF_ASYNC: + break; + default: + return -EINVAL; + } + + /* + * Write PR_MTE_TCF to SCTLR_EL1[TCF0]. + * Note that the syscall values are consistent with hw. + */ + env->cp15.sctlr_el[1] = + deposit64(env->cp15.sctlr_el[1], 38, 2, + arg2 >> TARGET_PR_MTE_TCF_SHIFT); + + /* + * Write PR_MTE_TAG to GCR_EL1[Exclude]. + * Note that the syscall uses an include mask, + * and hardware uses an exclude mask -- invert. + */ + env->cp15.gcr_el1 = + deposit64(env->cp15.gcr_el1, 0, 16, + ~arg2 >> TARGET_PR_MTE_TAG_SHIFT); + arm_rebuild_hflags(env); + } return 0; } case TARGET_PR_GET_TAGGED_ADDR_CTRL: { abi_long ret = 0; CPUARMState *env = cpu_env; + ARMCPU *cpu = env_archcpu(env); if (arg2 || arg3 || arg4 || arg5) { return -TARGET_EINVAL; @@ -XXX,XX +XXX,XX @@ static abi_long do_syscall1(void *cpu_env, int num, abi_long arg1, if (env->tagged_addr_enable) { ret |= TARGET_PR_TAGGED_ADDR_ENABLE; } + if (cpu_isar_feature(aa64_mte, cpu)) { + /* See above. */ + ret |= (extract64(env->cp15.sctlr_el[1], 38, 2) + << TARGET_PR_MTE_TCF_SHIFT); + ret = deposit64(ret, TARGET_PR_MTE_TAG_SHIFT, 16, + ~env->cp15.gcr_el1); + } return ret; } #endif /* AARCH64 */ -- 2.20.1 From: Richard Henderson <richard.henderson@linaro.org> Remember the PROT_MTE bit as PAGE_MTE/PAGE_TARGET_2. Otherwise this does not yet have effect. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20210210000223.884088-25-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- include/exec/cpu-all.h | 1 + linux-user/syscall_defs.h | 1 + target/arm/cpu.h | 1 + linux-user/mmap.c | 22 ++++++++++++++-------- 4 files changed, 17 insertions(+), 8 deletions(-) diff --git a/include/exec/cpu-all.h b/include/exec/cpu-all.h index XXXXXXX..XXXXXXX 100644 --- a/include/exec/cpu-all.h +++ b/include/exec/cpu-all.h @@ -XXX,XX +XXX,XX @@ extern intptr_t qemu_host_page_mask; #endif /* Target-specific bits that will be used via page_get_flags(). */ #define PAGE_TARGET_1 0x0080 +#define PAGE_TARGET_2 0x0200 #if defined(CONFIG_USER_ONLY) void page_dump(FILE *f); diff --git a/linux-user/syscall_defs.h b/linux-user/syscall_defs.h index XXXXXXX..XXXXXXX 100644 --- a/linux-user/syscall_defs.h +++ b/linux-user/syscall_defs.h @@ -XXX,XX +XXX,XX @@ struct target_winsize { #ifdef TARGET_AARCH64 #define TARGET_PROT_BTI 0x10 +#define TARGET_PROT_MTE 0x20 #endif /* Common */ diff --git a/target/arm/cpu.h b/target/arm/cpu.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -XXX,XX +XXX,XX @@ static inline MemTxAttrs *typecheck_memtxattrs(MemTxAttrs *x) * AArch64 usage of the PAGE_TARGET_* bits for linux-user. */ #define PAGE_BTI PAGE_TARGET_1 +#define PAGE_MTE PAGE_TARGET_2 #ifdef TARGET_TAGGED_ADDRESSES /** diff --git a/linux-user/mmap.c b/linux-user/mmap.c index XXXXXXX..XXXXXXX 100644 --- a/linux-user/mmap.c +++ b/linux-user/mmap.c @@ -XXX,XX +XXX,XX @@ static int validate_prot_to_pageflags(int *host_prot, int prot) | (prot & PROT_EXEC ? PROT_READ : 0); #ifdef TARGET_AARCH64 - /* - * The PROT_BTI bit is only accepted if the cpu supports the feature. - * Since this is the unusual case, don't bother checking unless - * the bit has been requested. If set and valid, record the bit - * within QEMU's page_flags. - */ - if (prot & TARGET_PROT_BTI) { + { ARMCPU *cpu = ARM_CPU(thread_cpu); - if (cpu_isar_feature(aa64_bti, cpu)) { + + /* + * The PROT_BTI bit is only accepted if the cpu supports the feature. + * Since this is the unusual case, don't bother checking unless + * the bit has been requested. If set and valid, record the bit + * within QEMU's page_flags. + */ + if ((prot & TARGET_PROT_BTI) && cpu_isar_feature(aa64_bti, cpu)) { valid |= TARGET_PROT_BTI; page_flags |= PAGE_BTI; } + /* Similarly for the PROT_MTE bit. */ + if ((prot & TARGET_PROT_MTE) && cpu_isar_feature(aa64_mte, cpu)) { + valid |= TARGET_PROT_MTE; + page_flags |= PAGE_MTE; + } } #endif -- 2.20.1 From: Richard Henderson <richard.henderson@linaro.org> Move everything related to syndromes to a new file, which can be shared with linux-user. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20210210000223.884088-26-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/internals.h | 245 +----------------------------------- target/arm/syndrome.h | 273 +++++++++++++++++++++++++++++++++++++++++ 2 files changed, 274 insertions(+), 244 deletions(-) create mode 100644 target/arm/syndrome.h diff --git a/target/arm/internals.h b/target/arm/internals.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/internals.h +++ b/target/arm/internals.h @@ -XXX,XX +XXX,XX @@ #define TARGET_ARM_INTERNALS_H #include "hw/registerfields.h" +#include "syndrome.h" /* register banks for CPU modes */ #define BANK_USRSYS 0 @@ -XXX,XX +XXX,XX @@ static inline bool extended_addresses_enabled(CPUARMState *env) (arm_feature(env, ARM_FEATURE_LPAE) && (tcr->raw_tcr & TTBCR_EAE)); } -/* Valid Syndrome Register EC field values */ -enum arm_exception_class { - EC_UNCATEGORIZED = 0x00, - EC_WFX_TRAP = 0x01, - EC_CP15RTTRAP = 0x03, - EC_CP15RRTTRAP = 0x04, - EC_CP14RTTRAP = 0x05, - EC_CP14DTTRAP = 0x06, - EC_ADVSIMDFPACCESSTRAP = 0x07, - EC_FPIDTRAP = 0x08, - EC_PACTRAP = 0x09, - EC_CP14RRTTRAP = 0x0c, - EC_BTITRAP = 0x0d, - EC_ILLEGALSTATE = 0x0e, - EC_AA32_SVC = 0x11, - EC_AA32_HVC = 0x12, - EC_AA32_SMC = 0x13, - EC_AA64_SVC = 0x15, - EC_AA64_HVC = 0x16, - EC_AA64_SMC = 0x17, - EC_SYSTEMREGISTERTRAP = 0x18, - EC_SVEACCESSTRAP = 0x19, - EC_INSNABORT = 0x20, - EC_INSNABORT_SAME_EL = 0x21, - EC_PCALIGNMENT = 0x22, - EC_DATAABORT = 0x24, - EC_DATAABORT_SAME_EL = 0x25, - EC_SPALIGNMENT = 0x26, - EC_AA32_FPTRAP = 0x28, - EC_AA64_FPTRAP = 0x2c, - EC_SERROR = 0x2f, - EC_BREAKPOINT = 0x30, - EC_BREAKPOINT_SAME_EL = 0x31, - EC_SOFTWARESTEP = 0x32, - EC_SOFTWARESTEP_SAME_EL = 0x33, - EC_WATCHPOINT = 0x34, - EC_WATCHPOINT_SAME_EL = 0x35, - EC_AA32_BKPT = 0x38, - EC_VECTORCATCH = 0x3a, - EC_AA64_BKPT = 0x3c, -}; - -#define ARM_EL_EC_SHIFT 26 -#define ARM_EL_IL_SHIFT 25 -#define ARM_EL_ISV_SHIFT 24 -#define ARM_EL_IL (1 << ARM_EL_IL_SHIFT) -#define ARM_EL_ISV (1 << ARM_EL_ISV_SHIFT) - -static inline uint32_t syn_get_ec(uint32_t syn) -{ - return syn >> ARM_EL_EC_SHIFT; -} - -/* Utility functions for constructing various kinds of syndrome value. - * Note that in general we follow the AArch64 syndrome values; in a - * few cases the value in HSR for exceptions taken to AArch32 Hyp - * mode differs slightly, and we fix this up when populating HSR in - * arm_cpu_do_interrupt_aarch32_hyp(). - * The exception is FP/SIMD access traps -- these report extra information - * when taking an exception to AArch32. For those we include the extra coproc - * and TA fields, and mask them out when taking the exception to AArch64. - */ -static inline uint32_t syn_uncategorized(void) -{ - return (EC_UNCATEGORIZED << ARM_EL_EC_SHIFT) | ARM_EL_IL; -} - -static inline uint32_t syn_aa64_svc(uint32_t imm16) -{ - return (EC_AA64_SVC << ARM_EL_EC_SHIFT) | ARM_EL_IL | (imm16 & 0xffff); -} - -static inline uint32_t syn_aa64_hvc(uint32_t imm16) -{ - return (EC_AA64_HVC << ARM_EL_EC_SHIFT) | ARM_EL_IL | (imm16 & 0xffff); -} - -static inline uint32_t syn_aa64_smc(uint32_t imm16) -{ - return (EC_AA64_SMC << ARM_EL_EC_SHIFT) | ARM_EL_IL | (imm16 & 0xffff); -} - -static inline uint32_t syn_aa32_svc(uint32_t imm16, bool is_16bit) -{ - return (EC_AA32_SVC << ARM_EL_EC_SHIFT) | (imm16 & 0xffff) - | (is_16bit ? 0 : ARM_EL_IL); -} - -static inline uint32_t syn_aa32_hvc(uint32_t imm16) -{ - return (EC_AA32_HVC << ARM_EL_EC_SHIFT) | ARM_EL_IL | (imm16 & 0xffff); -} - -static inline uint32_t syn_aa32_smc(void) -{ - return (EC_AA32_SMC << ARM_EL_EC_SHIFT) | ARM_EL_IL; -} - -static inline uint32_t syn_aa64_bkpt(uint32_t imm16) -{ - return (EC_AA64_BKPT << ARM_EL_EC_SHIFT) | ARM_EL_IL | (imm16 & 0xffff); -} - -static inline uint32_t syn_aa32_bkpt(uint32_t imm16, bool is_16bit) -{ - return (EC_AA32_BKPT << ARM_EL_EC_SHIFT) | (imm16 & 0xffff) - | (is_16bit ? 0 : ARM_EL_IL); -} - -static inline uint32_t syn_aa64_sysregtrap(int op0, int op1, int op2, - int crn, int crm, int rt, - int isread) -{ - return (EC_SYSTEMREGISTERTRAP << ARM_EL_EC_SHIFT) | ARM_EL_IL - | (op0 << 20) | (op2 << 17) | (op1 << 14) | (crn << 10) | (rt << 5) - | (crm << 1) | isread; -} - -static inline uint32_t syn_cp14_rt_trap(int cv, int cond, int opc1, int opc2, - int crn, int crm, int rt, int isread, - bool is_16bit) -{ - return (EC_CP14RTTRAP << ARM_EL_EC_SHIFT) - | (is_16bit ? 0 : ARM_EL_IL) - | (cv << 24) | (cond << 20) | (opc2 << 17) | (opc1 << 14) - | (crn << 10) | (rt << 5) | (crm << 1) | isread; -} - -static inline uint32_t syn_cp15_rt_trap(int cv, int cond, int opc1, int opc2, - int crn, int crm, int rt, int isread, - bool is_16bit) -{ - return (EC_CP15RTTRAP << ARM_EL_EC_SHIFT) - | (is_16bit ? 0 : ARM_EL_IL) - | (cv << 24) | (cond << 20) | (opc2 << 17) | (opc1 << 14) - | (crn << 10) | (rt << 5) | (crm << 1) | isread; -} - -static inline uint32_t syn_cp14_rrt_trap(int cv, int cond, int opc1, int crm, - int rt, int rt2, int isread, - bool is_16bit) -{ - return (EC_CP14RRTTRAP << ARM_EL_EC_SHIFT) - | (is_16bit ? 0 : ARM_EL_IL) - | (cv << 24) | (cond << 20) | (opc1 << 16) - | (rt2 << 10) | (rt << 5) | (crm << 1) | isread; -} - -static inline uint32_t syn_cp15_rrt_trap(int cv, int cond, int opc1, int crm, - int rt, int rt2, int isread, - bool is_16bit) -{ - return (EC_CP15RRTTRAP << ARM_EL_EC_SHIFT) - | (is_16bit ? 0 : ARM_EL_IL) - | (cv << 24) | (cond << 20) | (opc1 << 16) - | (rt2 << 10) | (rt << 5) | (crm << 1) | isread; -} - -static inline uint32_t syn_fp_access_trap(int cv, int cond, bool is_16bit) -{ - /* AArch32 FP trap or any AArch64 FP/SIMD trap: TA == 0 coproc == 0xa */ - return (EC_ADVSIMDFPACCESSTRAP << ARM_EL_EC_SHIFT) - | (is_16bit ? 0 : ARM_EL_IL) - | (cv << 24) | (cond << 20) | 0xa; -} - -static inline uint32_t syn_simd_access_trap(int cv, int cond, bool is_16bit) -{ - /* AArch32 SIMD trap: TA == 1 coproc == 0 */ - return (EC_ADVSIMDFPACCESSTRAP << ARM_EL_EC_SHIFT) - | (is_16bit ? 0 : ARM_EL_IL) - | (cv << 24) | (cond << 20) | (1 << 5); -} - -static inline uint32_t syn_sve_access_trap(void) -{ - return EC_SVEACCESSTRAP << ARM_EL_EC_SHIFT; -} - -static inline uint32_t syn_pactrap(void) -{ - return EC_PACTRAP << ARM_EL_EC_SHIFT; -} - -static inline uint32_t syn_btitrap(int btype) -{ - return (EC_BTITRAP << ARM_EL_EC_SHIFT) | btype; -} - -static inline uint32_t syn_insn_abort(int same_el, int ea, int s1ptw, int fsc) -{ - return (EC_INSNABORT << ARM_EL_EC_SHIFT) | (same_el << ARM_EL_EC_SHIFT) - | ARM_EL_IL | (ea << 9) | (s1ptw << 7) | fsc; -} - -static inline uint32_t syn_data_abort_no_iss(int same_el, int fnv, - int ea, int cm, int s1ptw, - int wnr, int fsc) -{ - return (EC_DATAABORT << ARM_EL_EC_SHIFT) | (same_el << ARM_EL_EC_SHIFT) - | ARM_EL_IL - | (fnv << 10) | (ea << 9) | (cm << 8) | (s1ptw << 7) - | (wnr << 6) | fsc; -} - -static inline uint32_t syn_data_abort_with_iss(int same_el, - int sas, int sse, int srt, - int sf, int ar, - int ea, int cm, int s1ptw, - int wnr, int fsc, - bool is_16bit) -{ - return (EC_DATAABORT << ARM_EL_EC_SHIFT) | (same_el << ARM_EL_EC_SHIFT) - | (is_16bit ? 0 : ARM_EL_IL) - | ARM_EL_ISV | (sas << 22) | (sse << 21) | (srt << 16) - | (sf << 15) | (ar << 14) - | (ea << 9) | (cm << 8) | (s1ptw << 7) | (wnr << 6) | fsc; -} - -static inline uint32_t syn_swstep(int same_el, int isv, int ex) -{ - return (EC_SOFTWARESTEP << ARM_EL_EC_SHIFT) | (same_el << ARM_EL_EC_SHIFT) - | ARM_EL_IL | (isv << 24) | (ex << 6) | 0x22; -} - -static inline uint32_t syn_watchpoint(int same_el, int cm, int wnr) -{ - return (EC_WATCHPOINT << ARM_EL_EC_SHIFT) | (same_el << ARM_EL_EC_SHIFT) - | ARM_EL_IL | (cm << 8) | (wnr << 6) | 0x22; -} - -static inline uint32_t syn_breakpoint(int same_el) -{ - return (EC_BREAKPOINT << ARM_EL_EC_SHIFT) | (same_el << ARM_EL_EC_SHIFT) - | ARM_EL_IL | 0x22; -} - -static inline uint32_t syn_wfx(int cv, int cond, int ti, bool is_16bit) -{ - return (EC_WFX_TRAP << ARM_EL_EC_SHIFT) | - (is_16bit ? 0 : (1 << ARM_EL_IL_SHIFT)) | - (cv << 24) | (cond << 20) | ti; -} - /* Update a QEMU watchpoint based on the information the guest has set in the * DBGWCR<n>_EL1 and DBGWVR<n>_EL1 registers. */ diff --git a/target/arm/syndrome.h b/target/arm/syndrome.h new file mode 100644 index XXXXXXX..XXXXXXX --- /dev/null +++ b/target/arm/syndrome.h @@ -XXX,XX +XXX,XX @@ +/* + * QEMU ARM CPU -- syndrome functions and types + * + * Copyright (c) 2014 Linaro Ltd + * + * This program is free software; you can redistribute it and/or + * modify it under the terms of the GNU General Public License + * as published by the Free Software Foundation; either version 2 + * of the License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, see + * <http://www.gnu.org/licenses/gpl-2.0.html> + * + * This header defines functions, types, etc which need to be shared + * between different source files within target/arm/ but which are + * private to it and not required by the rest of QEMU. + */ + +#ifndef TARGET_ARM_SYNDROME_H +#define TARGET_ARM_SYNDROME_H + +/* Valid Syndrome Register EC field values */ +enum arm_exception_class { + EC_UNCATEGORIZED = 0x00, + EC_WFX_TRAP = 0x01, + EC_CP15RTTRAP = 0x03, + EC_CP15RRTTRAP = 0x04, + EC_CP14RTTRAP = 0x05, + EC_CP14DTTRAP = 0x06, + EC_ADVSIMDFPACCESSTRAP = 0x07, + EC_FPIDTRAP = 0x08, + EC_PACTRAP = 0x09, + EC_CP14RRTTRAP = 0x0c, + EC_BTITRAP = 0x0d, + EC_ILLEGALSTATE = 0x0e, + EC_AA32_SVC = 0x11, + EC_AA32_HVC = 0x12, + EC_AA32_SMC = 0x13, + EC_AA64_SVC = 0x15, + EC_AA64_HVC = 0x16, + EC_AA64_SMC = 0x17, + EC_SYSTEMREGISTERTRAP = 0x18, + EC_SVEACCESSTRAP = 0x19, + EC_INSNABORT = 0x20, + EC_INSNABORT_SAME_EL = 0x21, + EC_PCALIGNMENT = 0x22, + EC_DATAABORT = 0x24, + EC_DATAABORT_SAME_EL = 0x25, + EC_SPALIGNMENT = 0x26, + EC_AA32_FPTRAP = 0x28, + EC_AA64_FPTRAP = 0x2c, + EC_SERROR = 0x2f, + EC_BREAKPOINT = 0x30, + EC_BREAKPOINT_SAME_EL = 0x31, + EC_SOFTWARESTEP = 0x32, + EC_SOFTWARESTEP_SAME_EL = 0x33, + EC_WATCHPOINT = 0x34, + EC_WATCHPOINT_SAME_EL = 0x35, + EC_AA32_BKPT = 0x38, + EC_VECTORCATCH = 0x3a, + EC_AA64_BKPT = 0x3c, +}; + +#define ARM_EL_EC_SHIFT 26 +#define ARM_EL_IL_SHIFT 25 +#define ARM_EL_ISV_SHIFT 24 +#define ARM_EL_IL (1 << ARM_EL_IL_SHIFT) +#define ARM_EL_ISV (1 << ARM_EL_ISV_SHIFT) + +static inline uint32_t syn_get_ec(uint32_t syn) +{ + return syn >> ARM_EL_EC_SHIFT; +} + +/* + * Utility functions for constructing various kinds of syndrome value. + * Note that in general we follow the AArch64 syndrome values; in a + * few cases the value in HSR for exceptions taken to AArch32 Hyp + * mode differs slightly, and we fix this up when populating HSR in + * arm_cpu_do_interrupt_aarch32_hyp(). + * The exception is FP/SIMD access traps -- these report extra information + * when taking an exception to AArch32. For those we include the extra coproc + * and TA fields, and mask them out when taking the exception to AArch64. + */ +static inline uint32_t syn_uncategorized(void) +{ + return (EC_UNCATEGORIZED << ARM_EL_EC_SHIFT) | ARM_EL_IL; +} + +static inline uint32_t syn_aa64_svc(uint32_t imm16) +{ + return (EC_AA64_SVC << ARM_EL_EC_SHIFT) | ARM_EL_IL | (imm16 & 0xffff); +} + +static inline uint32_t syn_aa64_hvc(uint32_t imm16) +{ + return (EC_AA64_HVC << ARM_EL_EC_SHIFT) | ARM_EL_IL | (imm16 & 0xffff); +} + +static inline uint32_t syn_aa64_smc(uint32_t imm16) +{ + return (EC_AA64_SMC << ARM_EL_EC_SHIFT) | ARM_EL_IL | (imm16 & 0xffff); +} + +static inline uint32_t syn_aa32_svc(uint32_t imm16, bool is_16bit) +{ + return (EC_AA32_SVC << ARM_EL_EC_SHIFT) | (imm16 & 0xffff) + | (is_16bit ? 0 : ARM_EL_IL); +} + +static inline uint32_t syn_aa32_hvc(uint32_t imm16) +{ + return (EC_AA32_HVC << ARM_EL_EC_SHIFT) | ARM_EL_IL | (imm16 & 0xffff); +} + +static inline uint32_t syn_aa32_smc(void) +{ + return (EC_AA32_SMC << ARM_EL_EC_SHIFT) | ARM_EL_IL; +} + +static inline uint32_t syn_aa64_bkpt(uint32_t imm16) +{ + return (EC_AA64_BKPT << ARM_EL_EC_SHIFT) | ARM_EL_IL | (imm16 & 0xffff); +} + +static inline uint32_t syn_aa32_bkpt(uint32_t imm16, bool is_16bit) +{ + return (EC_AA32_BKPT << ARM_EL_EC_SHIFT) | (imm16 & 0xffff) + | (is_16bit ? 0 : ARM_EL_IL); +} + +static inline uint32_t syn_aa64_sysregtrap(int op0, int op1, int op2, + int crn, int crm, int rt, + int isread) +{ + return (EC_SYSTEMREGISTERTRAP << ARM_EL_EC_SHIFT) | ARM_EL_IL + | (op0 << 20) | (op2 << 17) | (op1 << 14) | (crn << 10) | (rt << 5) + | (crm << 1) | isread; +} + +static inline uint32_t syn_cp14_rt_trap(int cv, int cond, int opc1, int opc2, + int crn, int crm, int rt, int isread, + bool is_16bit) +{ + return (EC_CP14RTTRAP << ARM_EL_EC_SHIFT) + | (is_16bit ? 0 : ARM_EL_IL) + | (cv << 24) | (cond << 20) | (opc2 << 17) | (opc1 << 14) + | (crn << 10) | (rt << 5) | (crm << 1) | isread; +} + +static inline uint32_t syn_cp15_rt_trap(int cv, int cond, int opc1, int opc2, + int crn, int crm, int rt, int isread, + bool is_16bit) +{ + return (EC_CP15RTTRAP << ARM_EL_EC_SHIFT) + | (is_16bit ? 0 : ARM_EL_IL) + | (cv << 24) | (cond << 20) | (opc2 << 17) | (opc1 << 14) + | (crn << 10) | (rt << 5) | (crm << 1) | isread; +} + +static inline uint32_t syn_cp14_rrt_trap(int cv, int cond, int opc1, int crm, + int rt, int rt2, int isread, + bool is_16bit) +{ + return (EC_CP14RRTTRAP << ARM_EL_EC_SHIFT) + | (is_16bit ? 0 : ARM_EL_IL) + | (cv << 24) | (cond << 20) | (opc1 << 16) + | (rt2 << 10) | (rt << 5) | (crm << 1) | isread; +} + +static inline uint32_t syn_cp15_rrt_trap(int cv, int cond, int opc1, int crm, + int rt, int rt2, int isread, + bool is_16bit) +{ + return (EC_CP15RRTTRAP << ARM_EL_EC_SHIFT) + | (is_16bit ? 0 : ARM_EL_IL) + | (cv << 24) | (cond << 20) | (opc1 << 16) + | (rt2 << 10) | (rt << 5) | (crm << 1) | isread; +} + +static inline uint32_t syn_fp_access_trap(int cv, int cond, bool is_16bit) +{ + /* AArch32 FP trap or any AArch64 FP/SIMD trap: TA == 0 coproc == 0xa */ + return (EC_ADVSIMDFPACCESSTRAP << ARM_EL_EC_SHIFT) + | (is_16bit ? 0 : ARM_EL_IL) + | (cv << 24) | (cond << 20) | 0xa; +} + +static inline uint32_t syn_simd_access_trap(int cv, int cond, bool is_16bit) +{ + /* AArch32 SIMD trap: TA == 1 coproc == 0 */ + return (EC_ADVSIMDFPACCESSTRAP << ARM_EL_EC_SHIFT) + | (is_16bit ? 0 : ARM_EL_IL) + | (cv << 24) | (cond << 20) | (1 << 5); +} + +static inline uint32_t syn_sve_access_trap(void) +{ + return EC_SVEACCESSTRAP << ARM_EL_EC_SHIFT; +} + +static inline uint32_t syn_pactrap(void) +{ + return EC_PACTRAP << ARM_EL_EC_SHIFT; +} + +static inline uint32_t syn_btitrap(int btype) +{ + return (EC_BTITRAP << ARM_EL_EC_SHIFT) | btype; +} + +static inline uint32_t syn_insn_abort(int same_el, int ea, int s1ptw, int fsc) +{ + return (EC_INSNABORT << ARM_EL_EC_SHIFT) | (same_el << ARM_EL_EC_SHIFT) + | ARM_EL_IL | (ea << 9) | (s1ptw << 7) | fsc; +} + +static inline uint32_t syn_data_abort_no_iss(int same_el, int fnv, + int ea, int cm, int s1ptw, + int wnr, int fsc) +{ + return (EC_DATAABORT << ARM_EL_EC_SHIFT) | (same_el << ARM_EL_EC_SHIFT) + | ARM_EL_IL + | (fnv << 10) | (ea << 9) | (cm << 8) | (s1ptw << 7) + | (wnr << 6) | fsc; +} + +static inline uint32_t syn_data_abort_with_iss(int same_el, + int sas, int sse, int srt, + int sf, int ar, + int ea, int cm, int s1ptw, + int wnr, int fsc, + bool is_16bit) +{ + return (EC_DATAABORT << ARM_EL_EC_SHIFT) | (same_el << ARM_EL_EC_SHIFT) + | (is_16bit ? 0 : ARM_EL_IL) + | ARM_EL_ISV | (sas << 22) | (sse << 21) | (srt << 16) + | (sf << 15) | (ar << 14) + | (ea << 9) | (cm << 8) | (s1ptw << 7) | (wnr << 6) | fsc; +} + +static inline uint32_t syn_swstep(int same_el, int isv, int ex) +{ + return (EC_SOFTWARESTEP << ARM_EL_EC_SHIFT) | (same_el << ARM_EL_EC_SHIFT) + | ARM_EL_IL | (isv << 24) | (ex << 6) | 0x22; +} + +static inline uint32_t syn_watchpoint(int same_el, int cm, int wnr) +{ + return (EC_WATCHPOINT << ARM_EL_EC_SHIFT) | (same_el << ARM_EL_EC_SHIFT) + | ARM_EL_IL | (cm << 8) | (wnr << 6) | 0x22; +} + +static inline uint32_t syn_breakpoint(int same_el) +{ + return (EC_BREAKPOINT << ARM_EL_EC_SHIFT) | (same_el << ARM_EL_EC_SHIFT) + | ARM_EL_IL | 0x22; +} + +static inline uint32_t syn_wfx(int cv, int cond, int ti, bool is_16bit) +{ + return (EC_WFX_TRAP << ARM_EL_EC_SHIFT) | + (is_16bit ? 0 : (1 << ARM_EL_IL_SHIFT)) | + (cv << 24) | (cond << 20) | ti; +} + +#endif /* TARGET_ARM_SYNDROME_H */ -- 2.20.1 From: Richard Henderson <richard.henderson@linaro.org> A proper syndrome is required to fill in the proper si_code. Use page_get_flags to determine permission vs translation for user-only. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20210210000223.884088-27-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- linux-user/aarch64/cpu_loop.c | 24 +++++++++++++++++++++--- target/arm/tlb_helper.c | 15 +++++++++------ 2 files changed, 30 insertions(+), 9 deletions(-) diff --git a/linux-user/aarch64/cpu_loop.c b/linux-user/aarch64/cpu_loop.c index XXXXXXX..XXXXXXX 100644 --- a/linux-user/aarch64/cpu_loop.c +++ b/linux-user/aarch64/cpu_loop.c @@ -XXX,XX +XXX,XX @@ #include "cpu_loop-common.h" #include "qemu/guest-random.h" #include "hw/semihosting/common-semi.h" +#include "target/arm/syndrome.h" #define get_user_code_u32(x, gaddr, env) \ ({ abi_long __r = get_user_u32((x), (gaddr)); \ @@ -XXX,XX +XXX,XX @@ void cpu_loop(CPUARMState *env) { CPUState *cs = env_cpu(env); - int trapnr; + int trapnr, ec, fsc; abi_long ret; target_siginfo_t info; @@ -XXX,XX +XXX,XX @@ void cpu_loop(CPUARMState *env) case EXCP_DATA_ABORT: info.si_signo = TARGET_SIGSEGV; info.si_errno = 0; - /* XXX: check env->error_code */ - info.si_code = TARGET_SEGV_MAPERR; info._sifields._sigfault._addr = env->exception.vaddress; + + /* We should only arrive here with EC in {DATAABORT, INSNABORT}. */ + ec = syn_get_ec(env->exception.syndrome); + assert(ec == EC_DATAABORT || ec == EC_INSNABORT); + + /* Both EC have the same format for FSC, or close enough. */ + fsc = extract32(env->exception.syndrome, 0, 6); + switch (fsc) { + case 0x04 ... 0x07: /* Translation fault, level {0-3} */ + info.si_code = TARGET_SEGV_MAPERR; + break; + case 0x09 ... 0x0b: /* Access flag fault, level {1-3} */ + case 0x0d ... 0x0f: /* Permission fault, level {1-3} */ + info.si_code = TARGET_SEGV_ACCERR; + break; + default: + g_assert_not_reached(); + } + queue_signal(env, info.si_signo, QEMU_SI_FAULT, &info); break; case EXCP_DEBUG: diff --git a/target/arm/tlb_helper.c b/target/arm/tlb_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tlb_helper.c +++ b/target/arm/tlb_helper.c @@ -XXX,XX +XXX,XX @@ bool arm_cpu_tlb_fill(CPUState *cs, vaddr address, int size, bool probe, uintptr_t retaddr) { ARMCPU *cpu = ARM_CPU(cs); + ARMMMUFaultInfo fi = {}; #ifdef CONFIG_USER_ONLY - cpu->env.exception.vaddress = address; - if (access_type == MMU_INST_FETCH) { - cs->exception_index = EXCP_PREFETCH_ABORT; + int flags = page_get_flags(useronly_clean_ptr(address)); + if (flags & PAGE_VALID) { + fi.type = ARMFault_Permission; } else { - cs->exception_index = EXCP_DATA_ABORT; + fi.type = ARMFault_Translation; } - cpu_loop_exit_restore(cs, retaddr); + + /* now we have a real cpu fault */ + cpu_restore_state(cs, retaddr, true); + arm_deliver_fault(cpu, address, access_type, mmu_idx, &fi); #else hwaddr phys_addr; target_ulong page_size; int prot, ret; MemTxAttrs attrs = {}; - ARMMMUFaultInfo fi = {}; ARMCacheAttrs cacheattrs = {}; /* -- 2.20.1 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20210210000223.884088-28-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- linux-user/aarch64/target_signal.h | 2 ++ linux-user/aarch64/cpu_loop.c | 3 +++ 2 files changed, 5 insertions(+) diff --git a/linux-user/aarch64/target_signal.h b/linux-user/aarch64/target_signal.h index XXXXXXX..XXXXXXX 100644 --- a/linux-user/aarch64/target_signal.h +++ b/linux-user/aarch64/target_signal.h @@ -XXX,XX +XXX,XX @@ typedef struct target_sigaltstack { #include "../generic/signal.h" +#define TARGET_SEGV_MTESERR 9 /* Synchronous ARM MTE exception */ + #define TARGET_ARCH_HAS_SETUP_FRAME #endif /* AARCH64_TARGET_SIGNAL_H */ diff --git a/linux-user/aarch64/cpu_loop.c b/linux-user/aarch64/cpu_loop.c index XXXXXXX..XXXXXXX 100644 --- a/linux-user/aarch64/cpu_loop.c +++ b/linux-user/aarch64/cpu_loop.c @@ -XXX,XX +XXX,XX @@ void cpu_loop(CPUARMState *env) case 0x0d ... 0x0f: /* Permission fault, level {1-3} */ info.si_code = TARGET_SEGV_ACCERR; break; + case 0x11: /* Synchronous Tag Check Fault */ + info.si_code = TARGET_SEGV_MTESERR; + break; default: g_assert_not_reached(); } -- 2.20.1 From: Richard Henderson <richard.henderson@linaro.org> The real kernel collects _TIF_MTE_ASYNC_FAULT into the current thread's state on any kernel entry (interrupt, exception etc), and then delivers the signal in advance of resuming the thread. This means that while the signal won't be delivered immediately, it will not be delayed forever -- at minimum it will be delivered after the next clock interrupt. We don't have a clock interrupt in linux-user, so we issue a cpu_kick to signal a return to the main loop at the end of the current TB. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20210210000223.884088-29-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- linux-user/aarch64/target_signal.h | 1 + linux-user/aarch64/cpu_loop.c | 11 +++++++++++ target/arm/mte_helper.c | 10 ++++++++++ 3 files changed, 22 insertions(+) diff --git a/linux-user/aarch64/target_signal.h b/linux-user/aarch64/target_signal.h index XXXXXXX..XXXXXXX 100644 --- a/linux-user/aarch64/target_signal.h +++ b/linux-user/aarch64/target_signal.h @@ -XXX,XX +XXX,XX @@ typedef struct target_sigaltstack { #include "../generic/signal.h" +#define TARGET_SEGV_MTEAERR 8 /* Asynchronous ARM MTE error */ #define TARGET_SEGV_MTESERR 9 /* Synchronous ARM MTE exception */ #define TARGET_ARCH_HAS_SETUP_FRAME diff --git a/linux-user/aarch64/cpu_loop.c b/linux-user/aarch64/cpu_loop.c index XXXXXXX..XXXXXXX 100644 --- a/linux-user/aarch64/cpu_loop.c +++ b/linux-user/aarch64/cpu_loop.c @@ -XXX,XX +XXX,XX @@ void cpu_loop(CPUARMState *env) EXCP_DUMP(env, "qemu: unhandled CPU exception 0x%x - aborting\n", trapnr); abort(); } + + /* Check for MTE asynchronous faults */ + if (unlikely(env->cp15.tfsr_el[0])) { + env->cp15.tfsr_el[0] = 0; + info.si_signo = TARGET_SIGSEGV; + info.si_errno = 0; + info._sifields._sigfault._addr = 0; + info.si_code = TARGET_SEGV_MTEAERR; + queue_signal(env, info.si_signo, QEMU_SI_FAULT, &info); + } + process_pending_signals(env); /* Exception return on AArch64 always clears the exclusive monitor, * so any return to running guest code implies this. diff --git a/target/arm/mte_helper.c b/target/arm/mte_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/mte_helper.c +++ b/target/arm/mte_helper.c @@ -XXX,XX +XXX,XX @@ static void mte_check_fail(CPUARMState *env, uint32_t desc, select = 0; } env->cp15.tfsr_el[el] |= 1 << select; +#ifdef CONFIG_USER_ONLY + /* + * Stand in for a timer irq, setting _TIF_MTE_ASYNC_FAULT, + * which then sends a SIGSEGV when the thread is next scheduled. + * This cpu will return to the main loop at the end of the TB, + * which is rather sooner than "normal". But the alternative + * is waiting until the next syscall. + */ + qemu_cpu_kick(env_cpu(env)); +#endif break; default: -- 2.20.1 From: Richard Henderson <richard.henderson@linaro.org> Use the now-saved PAGE_ANON and PAGE_MTE bits, and the per-page saved data. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20210210000223.884088-30-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/mte_helper.c | 29 +++++++++++++++++++++++++++-- 1 file changed, 27 insertions(+), 2 deletions(-) diff --git a/target/arm/mte_helper.c b/target/arm/mte_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/mte_helper.c +++ b/target/arm/mte_helper.c @@ -XXX,XX +XXX,XX @@ static uint8_t *allocation_tag_mem(CPUARMState *env, int ptr_mmu_idx, int tag_size, uintptr_t ra) { #ifdef CONFIG_USER_ONLY - /* Tag storage not implemented. */ - return NULL; + uint64_t clean_ptr = useronly_clean_ptr(ptr); + int flags = page_get_flags(clean_ptr); + uint8_t *tags; + uintptr_t index; + + if (!(flags & (ptr_access == MMU_DATA_STORE ? PAGE_WRITE : PAGE_READ))) { + /* SIGSEGV */ + arm_cpu_tlb_fill(env_cpu(env), ptr, ptr_size, ptr_access, + ptr_mmu_idx, false, ra); + g_assert_not_reached(); + } + + /* Require both MAP_ANON and PROT_MTE for the page. */ + if (!(flags & PAGE_ANON) || !(flags & PAGE_MTE)) { + return NULL; + } + + tags = page_get_target_data(clean_ptr); + if (tags == NULL) { + size_t alloc_size = TARGET_PAGE_SIZE >> (LOG2_TAG_GRANULE + 1); + tags = page_alloc_target_data(clean_ptr, alloc_size); + assert(tags != NULL); + } + + index = extract32(ptr, LOG2_TAG_GRANULE + 1, + TARGET_PAGE_BITS - LOG2_TAG_GRANULE - 1); + return tags + index; #else uintptr_t index; CPUIOTLBEntry *iotlbentry; -- 2.20.1 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20210210000223.884088-31-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/cpu.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/target/arm/cpu.c b/target/arm/cpu.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpu.c +++ b/target/arm/cpu.c @@ -XXX,XX +XXX,XX @@ static void arm_cpu_reset(DeviceState *dev) * Note that this must match useronly_clean_ptr. */ env->cp15.tcr_el[1].raw_tcr = (1ULL << 37); + + /* Enable MTE */ + if (cpu_isar_feature(aa64_mte, cpu)) { + /* Enable tag access, but leave TCF0 as No Effect (0). */ + env->cp15.sctlr_el[1] |= SCTLR_ATA0; + /* + * Exclude all tags, so that tag 0 is always used. + * This corresponds to Linux current->thread.gcr_incl = 0. + * + * Set RRND, so that helper_irg() will generate a seed later. + * Here in cpu_reset(), the crypto subsystem has not yet been + * initialized. + */ + env->cp15.gcr_el1 = 0x1ffff; + } #else /* Reset into the highest available EL */ if (arm_feature(env, ARM_FEATURE_EL3)) { -- 2.20.1 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20210210000223.884088-32-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- tests/tcg/aarch64/mte.h | 60 +++++++++++++++++++++++++++++++ tests/tcg/aarch64/mte-1.c | 28 +++++++++++++++ tests/tcg/aarch64/mte-2.c | 45 +++++++++++++++++++++++ tests/tcg/aarch64/mte-3.c | 51 ++++++++++++++++++++++++++ tests/tcg/aarch64/mte-4.c | 45 +++++++++++++++++++++++ tests/tcg/aarch64/Makefile.target | 6 ++++ tests/tcg/configure.sh | 4 +++ 7 files changed, 239 insertions(+) create mode 100644 tests/tcg/aarch64/mte.h create mode 100644 tests/tcg/aarch64/mte-1.c create mode 100644 tests/tcg/aarch64/mte-2.c create mode 100644 tests/tcg/aarch64/mte-3.c create mode 100644 tests/tcg/aarch64/mte-4.c diff --git a/tests/tcg/aarch64/mte.h b/tests/tcg/aarch64/mte.h new file mode 100644 index XXXXXXX..XXXXXXX --- /dev/null +++ b/tests/tcg/aarch64/mte.h @@ -XXX,XX +XXX,XX @@ +/* + * Linux kernel fallback API definitions for MTE and test helpers. + * + * Copyright (c) 2021 Linaro Ltd + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#include <assert.h> +#include <string.h> +#include <stdlib.h> +#include <stdio.h> +#include <unistd.h> +#include <signal.h> +#include <sys/mman.h> +#include <sys/prctl.h> + +#ifndef PR_SET_TAGGED_ADDR_CTRL +# define PR_SET_TAGGED_ADDR_CTRL 55 +#endif +#ifndef PR_TAGGED_ADDR_ENABLE +# define PR_TAGGED_ADDR_ENABLE (1UL << 0) +#endif +#ifndef PR_MTE_TCF_SHIFT +# define PR_MTE_TCF_SHIFT 1 +# define PR_MTE_TCF_NONE (0UL << PR_MTE_TCF_SHIFT) +# define PR_MTE_TCF_SYNC (1UL << PR_MTE_TCF_SHIFT) +# define PR_MTE_TCF_ASYNC (2UL << PR_MTE_TCF_SHIFT) +# define PR_MTE_TAG_SHIFT 3 +#endif + +#ifndef PROT_MTE +# define PROT_MTE 0x20 +#endif + +#ifndef SEGV_MTEAERR +# define SEGV_MTEAERR 8 +# define SEGV_MTESERR 9 +#endif + +static void enable_mte(int tcf) +{ + int r = prctl(PR_SET_TAGGED_ADDR_CTRL, + PR_TAGGED_ADDR_ENABLE | tcf | (0xfffe << PR_MTE_TAG_SHIFT), + 0, 0, 0); + if (r < 0) { + perror("PR_SET_TAGGED_ADDR_CTRL"); + exit(2); + } +} + +static void *alloc_mte_mem(size_t size) +{ + void *p = mmap(NULL, size, PROT_READ | PROT_WRITE | PROT_MTE, + MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); + if (p == MAP_FAILED) { + perror("mmap PROT_MTE"); + exit(2); + } + return p; +} diff --git a/tests/tcg/aarch64/mte-1.c b/tests/tcg/aarch64/mte-1.c new file mode 100644 index XXXXXXX..XXXXXXX --- /dev/null +++ b/tests/tcg/aarch64/mte-1.c @@ -XXX,XX +XXX,XX @@ +/* + * Memory tagging, basic pass cases. + * + * Copyright (c) 2021 Linaro Ltd + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#include "mte.h" + +int main(int ac, char **av) +{ + int *p0, *p1, *p2; + long c; + + enable_mte(PR_MTE_TCF_NONE); + p0 = alloc_mte_mem(sizeof(*p0)); + + asm("irg %0,%1,%2" : "=r"(p1) : "r"(p0), "r"(1)); + assert(p1 != p0); + asm("subp %0,%1,%2" : "=r"(c) : "r"(p0), "r"(p1)); + assert(c == 0); + + asm("stg %0, [%0]" : : "r"(p1)); + asm("ldg %0, [%1]" : "=r"(p2) : "r"(p0), "0"(p0)); + assert(p1 == p2); + + return 0; +} diff --git a/tests/tcg/aarch64/mte-2.c b/tests/tcg/aarch64/mte-2.c new file mode 100644 index XXXXXXX..XXXXXXX --- /dev/null +++ b/tests/tcg/aarch64/mte-2.c @@ -XXX,XX +XXX,XX @@ +/* + * Memory tagging, basic fail cases, synchronous signals. + * + * Copyright (c) 2021 Linaro Ltd + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#include "mte.h" + +void pass(int sig, siginfo_t *info, void *uc) +{ + assert(info->si_code == SEGV_MTESERR); + exit(0); +} + +int main(int ac, char **av) +{ + struct sigaction sa; + int *p0, *p1, *p2; + long excl = 1; + + enable_mte(PR_MTE_TCF_SYNC); + p0 = alloc_mte_mem(sizeof(*p0)); + + /* Create two differently tagged pointers. */ + asm("irg %0,%1,%2" : "=r"(p1) : "r"(p0), "r"(excl)); + asm("gmi %0,%1,%0" : "+r"(excl) : "r" (p1)); + assert(excl != 1); + asm("irg %0,%1,%2" : "=r"(p2) : "r"(p0), "r"(excl)); + assert(p1 != p2); + + /* Store the tag from the first pointer. */ + asm("stg %0, [%0]" : : "r"(p1)); + + *p1 = 0; + + memset(&sa, 0, sizeof(sa)); + sa.sa_sigaction = pass; + sa.sa_flags = SA_SIGINFO; + sigaction(SIGSEGV, &sa, NULL); + + *p2 = 0; + + abort(); +} diff --git a/tests/tcg/aarch64/mte-3.c b/tests/tcg/aarch64/mte-3.c new file mode 100644 index XXXXXXX..XXXXXXX --- /dev/null +++ b/tests/tcg/aarch64/mte-3.c @@ -XXX,XX +XXX,XX @@ +/* + * Memory tagging, basic fail cases, asynchronous signals. + * + * Copyright (c) 2021 Linaro Ltd + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#include "mte.h" + +void pass(int sig, siginfo_t *info, void *uc) +{ + assert(info->si_code == SEGV_MTEAERR); + exit(0); +} + +int main(int ac, char **av) +{ + struct sigaction sa; + long *p0, *p1, *p2; + long excl = 1; + + enable_mte(PR_MTE_TCF_ASYNC); + p0 = alloc_mte_mem(sizeof(*p0)); + + /* Create two differently tagged pointers. */ + asm("irg %0,%1,%2" : "=r"(p1) : "r"(p0), "r"(excl)); + asm("gmi %0,%1,%0" : "+r"(excl) : "r" (p1)); + assert(excl != 1); + asm("irg %0,%1,%2" : "=r"(p2) : "r"(p0), "r"(excl)); + assert(p1 != p2); + + /* Store the tag from the first pointer. */ + asm("stg %0, [%0]" : : "r"(p1)); + + *p1 = 0; + + memset(&sa, 0, sizeof(sa)); + sa.sa_sigaction = pass; + sa.sa_flags = SA_SIGINFO; + sigaction(SIGSEGV, &sa, NULL); + + /* + * Signal for async error will happen eventually. + * For a real kernel this should be after the next IRQ (e.g. timer). + * For qemu linux-user, we kick the cpu and exit at the next TB. + * In either case, loop until this happens (or killed by timeout). + * For extra sauce, yield, producing EXCP_YIELD to cpu_loop(). + */ + asm("str %0, [%0]; yield" : : "r"(p2)); + while (1); +} diff --git a/tests/tcg/aarch64/mte-4.c b/tests/tcg/aarch64/mte-4.c new file mode 100644 index XXXXXXX..XXXXXXX --- /dev/null +++ b/tests/tcg/aarch64/mte-4.c @@ -XXX,XX +XXX,XX @@ +/* + * Memory tagging, re-reading tag checks. + * + * Copyright (c) 2021 Linaro Ltd + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#include "mte.h" + +void __attribute__((noinline)) tagset(void *p, size_t size) +{ + size_t i; + for (i = 0; i < size; i += 16) { + asm("stg %0, [%0]" : : "r"(p + i)); + } +} + +void __attribute__((noinline)) tagcheck(void *p, size_t size) +{ + size_t i; + void *c; + + for (i = 0; i < size; i += 16) { + asm("ldg %0, [%1]" : "=r"(c) : "r"(p + i), "0"(p)); + assert(c == p); + } +} + +int main(int ac, char **av) +{ + size_t size = getpagesize() * 4; + long excl = 1; + int *p0, *p1; + + enable_mte(PR_MTE_TCF_ASYNC); + p0 = alloc_mte_mem(size); + + /* Tag the pointer. */ + asm("irg %0,%1,%2" : "=r"(p1) : "r"(p0), "r"(excl)); + + tagset(p1, size); + tagcheck(p1, size); + + return 0; +} diff --git a/tests/tcg/aarch64/Makefile.target b/tests/tcg/aarch64/Makefile.target index XXXXXXX..XXXXXXX 100644 --- a/tests/tcg/aarch64/Makefile.target +++ b/tests/tcg/aarch64/Makefile.target @@ -XXX,XX +XXX,XX @@ endif # bti-2 tests PROT_BTI, so no special compiler support required. AARCH64_TESTS += bti-2 +# MTE Tests +ifneq ($(DOCKER_IMAGE)$(CROSS_CC_HAS_ARMV8_MTE),) +AARCH64_TESTS += mte-1 mte-2 mte-3 mte-4 +mte-%: CFLAGS += -march=armv8.5-a+memtag +endif + # Semihosting smoke test for linux-user AARCH64_TESTS += semihosting run-semihosting: semihosting diff --git a/tests/tcg/configure.sh b/tests/tcg/configure.sh index XXXXXXX..XXXXXXX 100755 --- a/tests/tcg/configure.sh +++ b/tests/tcg/configure.sh @@ -XXX,XX +XXX,XX @@ for target in $target_list; do -mbranch-protection=standard -o $TMPE $TMPC; then echo "CROSS_CC_HAS_ARMV8_BTI=y" >> $config_target_mak fi + if do_compiler "$target_compiler" $target_compiler_cflags \ + -march=armv8.5-a+memtag -o $TMPE $TMPC; then + echo "CROSS_CC_HAS_ARMV8_MTE=y" >> $config_target_mak + fi ;; esac -- 2.20.1 From: Doug Evans <dje@google.com> This is a 10/100 ethernet device that has several features. Only the ones needed by the Linux driver have been implemented. See npcm7xx_emc.c for a list of unimplemented features. Reviewed-by: Hao Wu <wuhaotsh@google.com> Reviewed-by: Avi Fishman <avi.fishman@nuvoton.com> Signed-off-by: Doug Evans <dje@google.com> Message-id: 20210209015541.778833-2-dje@google.com Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- include/hw/net/npcm7xx_emc.h | 286 ++++++++++++ hw/net/npcm7xx_emc.c | 857 +++++++++++++++++++++++++++++++++++ hw/net/meson.build | 1 + hw/net/trace-events | 17 + 4 files changed, 1161 insertions(+) create mode 100644 include/hw/net/npcm7xx_emc.h create mode 100644 hw/net/npcm7xx_emc.c diff --git a/include/hw/net/npcm7xx_emc.h b/include/hw/net/npcm7xx_emc.h new file mode 100644 index XXXXXXX..XXXXXXX --- /dev/null +++ b/include/hw/net/npcm7xx_emc.h @@ -XXX,XX +XXX,XX @@ +/* + * Nuvoton NPCM7xx EMC Module + * + * Copyright 2020 Google LLC + * + * This program is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License as published by the + * Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License + * for more details. + */ + +#ifndef NPCM7XX_EMC_H +#define NPCM7XX_EMC_H + +#include "hw/irq.h" +#include "hw/sysbus.h" +#include "net/net.h" + +/* 32-bit register indices. */ +enum NPCM7xxPWMRegister { + /* Control registers. */ + REG_CAMCMR, + REG_CAMEN, + + /* There are 16 CAMn[ML] registers. */ + REG_CAMM_BASE, + REG_CAML_BASE, + REG_CAMML_LAST = 0x21, + + REG_TXDLSA = 0x22, + REG_RXDLSA, + REG_MCMDR, + REG_MIID, + REG_MIIDA, + REG_FFTCR, + REG_TSDR, + REG_RSDR, + REG_DMARFC, + REG_MIEN, + + /* Status registers. */ + REG_MISTA, + REG_MGSTA, + REG_MPCNT, + REG_MRPC, + REG_MRPCC, + REG_MREPC, + REG_DMARFS, + REG_CTXDSA, + REG_CTXBSA, + REG_CRXDSA, + REG_CRXBSA, + + NPCM7XX_NUM_EMC_REGS, +}; + +/* REG_CAMCMR fields */ +/* Enable CAM Compare */ +#define REG_CAMCMR_ECMP (1 << 4) +/* Complement CAM Compare */ +#define REG_CAMCMR_CCAM (1 << 3) +/* Accept Broadcast Packet */ +#define REG_CAMCMR_ABP (1 << 2) +/* Accept Multicast Packet */ +#define REG_CAMCMR_AMP (1 << 1) +/* Accept Unicast Packet */ +#define REG_CAMCMR_AUP (1 << 0) + +/* REG_MCMDR fields */ +/* Software Reset */ +#define REG_MCMDR_SWR (1 << 24) +/* Internal Loopback Select */ +#define REG_MCMDR_LBK (1 << 21) +/* Operation Mode Select */ +#define REG_MCMDR_OPMOD (1 << 20) +/* Enable MDC Clock Generation */ +#define REG_MCMDR_ENMDC (1 << 19) +/* Full-Duplex Mode Select */ +#define REG_MCMDR_FDUP (1 << 18) +/* Enable SQE Checking */ +#define REG_MCMDR_ENSEQ (1 << 17) +/* Send PAUSE Frame */ +#define REG_MCMDR_SDPZ (1 << 16) +/* No Defer */ +#define REG_MCMDR_NDEF (1 << 9) +/* Frame Transmission On */ +#define REG_MCMDR_TXON (1 << 8) +/* Strip CRC Checksum */ +#define REG_MCMDR_SPCRC (1 << 5) +/* Accept CRC Error Packet */ +#define REG_MCMDR_AEP (1 << 4) +/* Accept Control Packet */ +#define REG_MCMDR_ACP (1 << 3) +/* Accept Runt Packet */ +#define REG_MCMDR_ARP (1 << 2) +/* Accept Long Packet */ +#define REG_MCMDR_ALP (1 << 1) +/* Frame Reception On */ +#define REG_MCMDR_RXON (1 << 0) + +/* REG_MIEN fields */ +/* Enable Transmit Descriptor Unavailable Interrupt */ +#define REG_MIEN_ENTDU (1 << 23) +/* Enable Transmit Completion Interrupt */ +#define REG_MIEN_ENTXCP (1 << 18) +/* Enable Transmit Interrupt */ +#define REG_MIEN_ENTXINTR (1 << 16) +/* Enable Receive Descriptor Unavailable Interrupt */ +#define REG_MIEN_ENRDU (1 << 10) +/* Enable Receive Good Interrupt */ +#define REG_MIEN_ENRXGD (1 << 4) +/* Enable Receive Interrupt */ +#define REG_MIEN_ENRXINTR (1 << 0) + +/* REG_MISTA fields */ +/* TODO: Add error fields and support simulated errors? */ +/* Transmit Bus Error Interrupt */ +#define REG_MISTA_TXBERR (1 << 24) +/* Transmit Descriptor Unavailable Interrupt */ +#define REG_MISTA_TDU (1 << 23) +/* Transmit Completion Interrupt */ +#define REG_MISTA_TXCP (1 << 18) +/* Transmit Interrupt */ +#define REG_MISTA_TXINTR (1 << 16) +/* Receive Bus Error Interrupt */ +#define REG_MISTA_RXBERR (1 << 11) +/* Receive Descriptor Unavailable Interrupt */ +#define REG_MISTA_RDU (1 << 10) +/* DMA Early Notification Interrupt */ +#define REG_MISTA_DENI (1 << 9) +/* Maximum Frame Length Interrupt */ +#define REG_MISTA_DFOI (1 << 8) +/* Receive Good Interrupt */ +#define REG_MISTA_RXGD (1 << 4) +/* Packet Too Long Interrupt */ +#define REG_MISTA_PTLE (1 << 3) +/* Receive Interrupt */ +#define REG_MISTA_RXINTR (1 << 0) + +/* REG_MGSTA fields */ +/* Transmission Halted */ +#define REG_MGSTA_TXHA (1 << 11) +/* Receive Halted */ +#define REG_MGSTA_RXHA (1 << 11) + +/* REG_DMARFC fields */ +/* Maximum Receive Frame Length */ +#define REG_DMARFC_RXMS(word) extract32((word), 0, 16) + +/* REG MIIDA fields */ +/* Busy Bit */ +#define REG_MIIDA_BUSY (1 << 17) + +/* Transmit and receive descriptors */ +typedef struct NPCM7xxEMCTxDesc NPCM7xxEMCTxDesc; +typedef struct NPCM7xxEMCRxDesc NPCM7xxEMCRxDesc; + +struct NPCM7xxEMCTxDesc { + uint32_t flags; + uint32_t txbsa; + uint32_t status_and_length; + uint32_t ntxdsa; +}; + +struct NPCM7xxEMCRxDesc { + uint32_t status_and_length; + uint32_t rxbsa; + uint32_t reserved; + uint32_t nrxdsa; +}; + +/* NPCM7xxEMCTxDesc.flags values */ +/* Owner: 0 = cpu, 1 = emc */ +#define TX_DESC_FLAG_OWNER_MASK (1 << 31) +/* Transmit interrupt enable */ +#define TX_DESC_FLAG_INTEN (1 << 2) +/* CRC append */ +#define TX_DESC_FLAG_CRCAPP (1 << 1) +/* Padding enable */ +#define TX_DESC_FLAG_PADEN (1 << 0) + +/* NPCM7xxEMCTxDesc.status_and_length values */ +/* Collision count */ +#define TX_DESC_STATUS_CCNT_SHIFT 28 +#define TX_DESC_STATUS_CCNT_BITSIZE 4 +/* SQE error */ +#define TX_DESC_STATUS_SQE (1 << 26) +/* Transmission paused */ +#define TX_DESC_STATUS_PAU (1 << 25) +/* P transmission halted */ +#define TX_DESC_STATUS_TXHA (1 << 24) +/* Late collision */ +#define TX_DESC_STATUS_LC (1 << 23) +/* Transmission abort */ +#define TX_DESC_STATUS_TXABT (1 << 22) +/* No carrier sense */ +#define TX_DESC_STATUS_NCS (1 << 21) +/* Defer exceed */ +#define TX_DESC_STATUS_EXDEF (1 << 20) +/* Transmission complete */ +#define TX_DESC_STATUS_TXCP (1 << 19) +/* Transmission deferred */ +#define TX_DESC_STATUS_DEF (1 << 17) +/* Transmit interrupt */ +#define TX_DESC_STATUS_TXINTR (1 << 16) + +#define TX_DESC_PKT_LEN(word) extract32((word), 0, 16) + +/* Transmit buffer start address */ +#define TX_DESC_TXBSA(word) ((uint32_t) (word) & ~3u) + +/* Next transmit descriptor start address */ +#define TX_DESC_NTXDSA(word) ((uint32_t) (word) & ~3u) + +/* NPCM7xxEMCRxDesc.status_and_length values */ +/* Owner: 0b00 = cpu, 0b01 = undefined, 0b10 = emc, 0b11 = undefined */ +#define RX_DESC_STATUS_OWNER_SHIFT 30 +#define RX_DESC_STATUS_OWNER_BITSIZE 2 +#define RX_DESC_STATUS_OWNER_MASK (3 << RX_DESC_STATUS_OWNER_SHIFT) +/* Runt packet */ +#define RX_DESC_STATUS_RP (1 << 22) +/* Alignment error */ +#define RX_DESC_STATUS_ALIE (1 << 21) +/* Frame reception complete */ +#define RX_DESC_STATUS_RXGD (1 << 20) +/* Packet too long */ +#define RX_DESC_STATUS_PTLE (1 << 19) +/* CRC error */ +#define RX_DESC_STATUS_CRCE (1 << 17) +/* Receive interrupt */ +#define RX_DESC_STATUS_RXINTR (1 << 16) + +#define RX_DESC_PKT_LEN(word) extract32((word), 0, 16) + +/* Receive buffer start address */ +#define RX_DESC_RXBSA(word) ((uint32_t) (word) & ~3u) + +/* Next receive descriptor start address */ +#define RX_DESC_NRXDSA(word) ((uint32_t) (word) & ~3u) + +/* Minimum packet length, when TX_DESC_FLAG_PADEN is set. */ +#define MIN_PACKET_LENGTH 64 + +struct NPCM7xxEMCState { + /*< private >*/ + SysBusDevice parent; + /*< public >*/ + + MemoryRegion iomem; + + qemu_irq tx_irq; + qemu_irq rx_irq; + + NICState *nic; + NICConf conf; + + /* 0 or 1, for log messages */ + uint8_t emc_num; + + uint32_t regs[NPCM7XX_NUM_EMC_REGS]; + + /* + * tx is active. Set to true by TSDR and then switches off when out of + * descriptors. If the TXON bit in REG_MCMDR is off then this is off. + */ + bool tx_active; + + /* + * rx is active. Set to true by RSDR and then switches off when out of + * descriptors. If the RXON bit in REG_MCMDR is off then this is off. + */ + bool rx_active; +}; + +typedef struct NPCM7xxEMCState NPCM7xxEMCState; + +#define TYPE_NPCM7XX_EMC "npcm7xx-emc" +#define NPCM7XX_EMC(obj) \ + OBJECT_CHECK(NPCM7xxEMCState, (obj), TYPE_NPCM7XX_EMC) + +#endif /* NPCM7XX_EMC_H */ diff --git a/hw/net/npcm7xx_emc.c b/hw/net/npcm7xx_emc.c new file mode 100644 index XXXXXXX..XXXXXXX --- /dev/null +++ b/hw/net/npcm7xx_emc.c @@ -XXX,XX +XXX,XX @@ +/* + * Nuvoton NPCM7xx EMC Module + * + * Copyright 2020 Google LLC + * + * This program is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License as published by the + * Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License + * for more details. + * + * Unsupported/unimplemented features: + * - MCMDR.FDUP (full duplex) is ignored, half duplex is not supported + * - Only CAM0 is supported, CAM[1-15] are not + * - writes to CAMEN.[1-15] are ignored, these bits always read as zeroes + * - MII is not implemented, MIIDA.BUSY and MIID always return zero + * - MCMDR.LBK is not implemented + * - MCMDR.{OPMOD,ENSQE,AEP,ARP} are not supported + * - H/W FIFOs are not supported, MCMDR.FFTCR is ignored + * - MGSTA.SQE is not supported + * - pause and control frames are not implemented + * - MGSTA.CCNT is not supported + * - MPCNT, DMARFS are not implemented + */ + +#include "qemu/osdep.h" + +/* For crc32 */ +#include <zlib.h> + +#include "qemu-common.h" +#include "hw/irq.h" +#include "hw/qdev-clock.h" +#include "hw/qdev-properties.h" +#include "hw/net/npcm7xx_emc.h" +#include "net/eth.h" +#include "migration/vmstate.h" +#include "qemu/bitops.h" +#include "qemu/error-report.h" +#include "qemu/log.h" +#include "qemu/module.h" +#include "qemu/units.h" +#include "sysemu/dma.h" +#include "trace.h" + +#define CRC_LENGTH 4 + +/* + * The maximum size of a (layer 2) ethernet frame as defined by 802.3. + * 1518 = 6(dest macaddr) + 6(src macaddr) + 2(proto) + 4(crc) + 1500(payload) + * This does not include an additional 4 for the vlan field (802.1q). + */ +#define MAX_ETH_FRAME_SIZE 1518 + +static const char *emc_reg_name(int regno) +{ +#define REG(name) case REG_ ## name: return #name; + switch (regno) { + REG(CAMCMR) + REG(CAMEN) + REG(TXDLSA) + REG(RXDLSA) + REG(MCMDR) + REG(MIID) + REG(MIIDA) + REG(FFTCR) + REG(TSDR) + REG(RSDR) + REG(DMARFC) + REG(MIEN) + REG(MISTA) + REG(MGSTA) + REG(MPCNT) + REG(MRPC) + REG(MRPCC) + REG(MREPC) + REG(DMARFS) + REG(CTXDSA) + REG(CTXBSA) + REG(CRXDSA) + REG(CRXBSA) + case REG_CAMM_BASE + 0: return "CAM0M"; + case REG_CAML_BASE + 0: return "CAM0L"; + case REG_CAMM_BASE + 2 ... REG_CAMML_LAST: + /* Only CAM0 is supported, fold the others into something simple. */ + if (regno & 1) { + return "CAM<n>L"; + } else { + return "CAM<n>M"; + } + default: return "UNKNOWN"; + } +#undef REG +} + +static void emc_reset(NPCM7xxEMCState *emc) +{ + trace_npcm7xx_emc_reset(emc->emc_num); + + memset(&emc->regs[0], 0, sizeof(emc->regs)); + + /* These regs have non-zero reset values. */ + emc->regs[REG_TXDLSA] = 0xfffffffc; + emc->regs[REG_RXDLSA] = 0xfffffffc; + emc->regs[REG_MIIDA] = 0x00900000; + emc->regs[REG_FFTCR] = 0x0101; + emc->regs[REG_DMARFC] = 0x0800; + emc->regs[REG_MPCNT] = 0x7fff; + + emc->tx_active = false; + emc->rx_active = false; +} + +static void npcm7xx_emc_reset(DeviceState *dev) +{ + NPCM7xxEMCState *emc = NPCM7XX_EMC(dev); + emc_reset(emc); +} + +static void emc_soft_reset(NPCM7xxEMCState *emc) +{ + /* + * The docs say at least MCMDR.{LBK,OPMOD} bits are not changed during a + * soft reset, but does not go into further detail. For now, KISS. + */ + uint32_t mcmdr = emc->regs[REG_MCMDR]; + emc_reset(emc); + emc->regs[REG_MCMDR] = mcmdr & (REG_MCMDR_LBK | REG_MCMDR_OPMOD); + + qemu_set_irq(emc->tx_irq, 0); + qemu_set_irq(emc->rx_irq, 0); +} + +static void emc_set_link(NetClientState *nc) +{ + /* Nothing to do yet. */ +} + +/* MISTA.TXINTR is the union of the individual bits with their enables. */ +static void emc_update_mista_txintr(NPCM7xxEMCState *emc) +{ + /* Only look at the bits we support. */ + uint32_t mask = (REG_MISTA_TXBERR | + REG_MISTA_TDU | + REG_MISTA_TXCP); + if (emc->regs[REG_MISTA] & emc->regs[REG_MIEN] & mask) { + emc->regs[REG_MISTA] |= REG_MISTA_TXINTR; + } else { + emc->regs[REG_MISTA] &= ~REG_MISTA_TXINTR; + } +} + +/* MISTA.RXINTR is the union of the individual bits with their enables. */ +static void emc_update_mista_rxintr(NPCM7xxEMCState *emc) +{ + /* Only look at the bits we support. */ + uint32_t mask = (REG_MISTA_RXBERR | + REG_MISTA_RDU | + REG_MISTA_RXGD); + if (emc->regs[REG_MISTA] & emc->regs[REG_MIEN] & mask) { + emc->regs[REG_MISTA] |= REG_MISTA_RXINTR; + } else { + emc->regs[REG_MISTA] &= ~REG_MISTA_RXINTR; + } +} + +/* N.B. emc_update_mista_txintr must have already been called. */ +static void emc_update_tx_irq(NPCM7xxEMCState *emc) +{ + int level = !!(emc->regs[REG_MISTA] & + emc->regs[REG_MIEN] & + REG_MISTA_TXINTR); + trace_npcm7xx_emc_update_tx_irq(level); + qemu_set_irq(emc->tx_irq, level); +} + +/* N.B. emc_update_mista_rxintr must have already been called. */ +static void emc_update_rx_irq(NPCM7xxEMCState *emc) +{ + int level = !!(emc->regs[REG_MISTA] & + emc->regs[REG_MIEN] & + REG_MISTA_RXINTR); + trace_npcm7xx_emc_update_rx_irq(level); + qemu_set_irq(emc->rx_irq, level); +} + +/* Update IRQ states due to changes in MIEN,MISTA. */ +static void emc_update_irq_from_reg_change(NPCM7xxEMCState *emc) +{ + emc_update_mista_txintr(emc); + emc_update_tx_irq(emc); + + emc_update_mista_rxintr(emc); + emc_update_rx_irq(emc); +} + +static int emc_read_tx_desc(dma_addr_t addr, NPCM7xxEMCTxDesc *desc) +{ + if (dma_memory_read(&address_space_memory, addr, desc, sizeof(*desc))) { + qemu_log_mask(LOG_GUEST_ERROR, "%s: Failed to read descriptor @ 0x%" + HWADDR_PRIx "\n", __func__, addr); + return -1; + } + desc->flags = le32_to_cpu(desc->flags); + desc->txbsa = le32_to_cpu(desc->txbsa); + desc->status_and_length = le32_to_cpu(desc->status_and_length); + desc->ntxdsa = le32_to_cpu(desc->ntxdsa); + return 0; +} + +static int emc_write_tx_desc(const NPCM7xxEMCTxDesc *desc, dma_addr_t addr) +{ + NPCM7xxEMCTxDesc le_desc; + + le_desc.flags = cpu_to_le32(desc->flags); + le_desc.txbsa = cpu_to_le32(desc->txbsa); + le_desc.status_and_length = cpu_to_le32(desc->status_and_length); + le_desc.ntxdsa = cpu_to_le32(desc->ntxdsa); + if (dma_memory_write(&address_space_memory, addr, &le_desc, + sizeof(le_desc))) { + qemu_log_mask(LOG_GUEST_ERROR, "%s: Failed to write descriptor @ 0x%" + HWADDR_PRIx "\n", __func__, addr); + return -1; + } + return 0; +} + +static int emc_read_rx_desc(dma_addr_t addr, NPCM7xxEMCRxDesc *desc) +{ + if (dma_memory_read(&address_space_memory, addr, desc, sizeof(*desc))) { + qemu_log_mask(LOG_GUEST_ERROR, "%s: Failed to read descriptor @ 0x%" + HWADDR_PRIx "\n", __func__, addr); + return -1; + } + desc->status_and_length = le32_to_cpu(desc->status_and_length); + desc->rxbsa = le32_to_cpu(desc->rxbsa); + desc->reserved = le32_to_cpu(desc->reserved); + desc->nrxdsa = le32_to_cpu(desc->nrxdsa); + return 0; +} + +static int emc_write_rx_desc(const NPCM7xxEMCRxDesc *desc, dma_addr_t addr) +{ + NPCM7xxEMCRxDesc le_desc; + + le_desc.status_and_length = cpu_to_le32(desc->status_and_length); + le_desc.rxbsa = cpu_to_le32(desc->rxbsa); + le_desc.reserved = cpu_to_le32(desc->reserved); + le_desc.nrxdsa = cpu_to_le32(desc->nrxdsa); + if (dma_memory_write(&address_space_memory, addr, &le_desc, + sizeof(le_desc))) { + qemu_log_mask(LOG_GUEST_ERROR, "%s: Failed to write descriptor @ 0x%" + HWADDR_PRIx "\n", __func__, addr); + return -1; + } + return 0; +} + +static void emc_set_mista(NPCM7xxEMCState *emc, uint32_t flags) +{ + trace_npcm7xx_emc_set_mista(flags); + emc->regs[REG_MISTA] |= flags; + if (extract32(flags, 16, 16)) { + emc_update_mista_txintr(emc); + } + if (extract32(flags, 0, 16)) { + emc_update_mista_rxintr(emc); + } +} + +static void emc_halt_tx(NPCM7xxEMCState *emc, uint32_t mista_flag) +{ + emc->tx_active = false; + emc_set_mista(emc, mista_flag); +} + +static void emc_halt_rx(NPCM7xxEMCState *emc, uint32_t mista_flag) +{ + emc->rx_active = false; + emc_set_mista(emc, mista_flag); +} + +static void emc_set_next_tx_descriptor(NPCM7xxEMCState *emc, + const NPCM7xxEMCTxDesc *tx_desc, + uint32_t desc_addr) +{ + /* Update the current descriptor, if only to reset the owner flag. */ + if (emc_write_tx_desc(tx_desc, desc_addr)) { + /* + * We just read it so this shouldn't generally happen. + * Error already reported. + */ + emc_set_mista(emc, REG_MISTA_TXBERR); + } + emc->regs[REG_CTXDSA] = TX_DESC_NTXDSA(tx_desc->ntxdsa); +} + +static void emc_set_next_rx_descriptor(NPCM7xxEMCState *emc, + const NPCM7xxEMCRxDesc *rx_desc, + uint32_t desc_addr) +{ + /* Update the current descriptor, if only to reset the owner flag. */ + if (emc_write_rx_desc(rx_desc, desc_addr)) { + /* + * We just read it so this shouldn't generally happen. + * Error already reported. + */ + emc_set_mista(emc, REG_MISTA_RXBERR); + } + emc->regs[REG_CRXDSA] = RX_DESC_NRXDSA(rx_desc->nrxdsa); +} + +static void emc_try_send_next_packet(NPCM7xxEMCState *emc) +{ + /* Working buffer for sending out packets. Most packets fit in this. */ +#define TX_BUFFER_SIZE 2048 + uint8_t tx_send_buffer[TX_BUFFER_SIZE]; + uint32_t desc_addr = TX_DESC_NTXDSA(emc->regs[REG_CTXDSA]); + NPCM7xxEMCTxDesc tx_desc; + uint32_t next_buf_addr, length; + uint8_t *buf; + g_autofree uint8_t *malloced_buf = NULL; + + if (emc_read_tx_desc(desc_addr, &tx_desc)) { + /* Error reading descriptor, already reported. */ + emc_halt_tx(emc, REG_MISTA_TXBERR); + emc_update_tx_irq(emc); + return; + } + + /* Nothing we can do if we don't own the descriptor. */ + if (!(tx_desc.flags & TX_DESC_FLAG_OWNER_MASK)) { + trace_npcm7xx_emc_cpu_owned_desc(desc_addr); + emc_halt_tx(emc, REG_MISTA_TDU); + emc_update_tx_irq(emc); + return; + } + + /* Give the descriptor back regardless of what happens. */ + tx_desc.flags &= ~TX_DESC_FLAG_OWNER_MASK; + tx_desc.status_and_length &= 0xffff; + + /* + * Despite the h/w documentation saying the tx buffer is word aligned, + * the linux driver does not word align the buffer. There is value in not + * aligning the buffer: See the description of NET_IP_ALIGN in linux + * kernel sources. + */ + next_buf_addr = tx_desc.txbsa; + emc->regs[REG_CTXBSA] = next_buf_addr; + length = TX_DESC_PKT_LEN(tx_desc.status_and_length); + buf = &tx_send_buffer[0]; + + if (length > sizeof(tx_send_buffer)) { + malloced_buf = g_malloc(length); + buf = malloced_buf; + } + + if (dma_memory_read(&address_space_memory, next_buf_addr, buf, length)) { + qemu_log_mask(LOG_GUEST_ERROR, "%s: Failed to read packet @ 0x%x\n", + __func__, next_buf_addr); + emc_set_mista(emc, REG_MISTA_TXBERR); + emc_set_next_tx_descriptor(emc, &tx_desc, desc_addr); + emc_update_tx_irq(emc); + trace_npcm7xx_emc_tx_done(emc->regs[REG_CTXDSA]); + return; + } + + if ((tx_desc.flags & TX_DESC_FLAG_PADEN) && (length < MIN_PACKET_LENGTH)) { + memset(buf + length, 0, MIN_PACKET_LENGTH - length); + length = MIN_PACKET_LENGTH; + } + + /* N.B. emc_receive can get called here. */ + qemu_send_packet(qemu_get_queue(emc->nic), buf, length); + trace_npcm7xx_emc_sent_packet(length); + + tx_desc.status_and_length |= TX_DESC_STATUS_TXCP; + if (tx_desc.flags & TX_DESC_FLAG_INTEN) { + emc_set_mista(emc, REG_MISTA_TXCP); + } + if (emc->regs[REG_MISTA] & emc->regs[REG_MIEN] & REG_MISTA_TXINTR) { + tx_desc.status_and_length |= TX_DESC_STATUS_TXINTR; + } + + emc_set_next_tx_descriptor(emc, &tx_desc, desc_addr); + emc_update_tx_irq(emc); + trace_npcm7xx_emc_tx_done(emc->regs[REG_CTXDSA]); +} + +static bool emc_can_receive(NetClientState *nc) +{ + NPCM7xxEMCState *emc = NPCM7XX_EMC(qemu_get_nic_opaque(nc)); + + bool can_receive = emc->rx_active; + trace_npcm7xx_emc_can_receive(can_receive); + return can_receive; +} + +/* If result is false then *fail_reason contains the reason. */ +static bool emc_receive_filter1(NPCM7xxEMCState *emc, const uint8_t *buf, + size_t len, const char **fail_reason) +{ + eth_pkt_types_e pkt_type = get_eth_packet_type(PKT_GET_ETH_HDR(buf)); + + switch (pkt_type) { + case ETH_PKT_BCAST: + if (emc->regs[REG_CAMCMR] & REG_CAMCMR_CCAM) { + return true; + } else { + *fail_reason = "Broadcast packet disabled"; + return !!(emc->regs[REG_CAMCMR] & REG_CAMCMR_ABP); + } + case ETH_PKT_MCAST: + if (emc->regs[REG_CAMCMR] & REG_CAMCMR_CCAM) { + return true; + } else { + *fail_reason = "Multicast packet disabled"; + return !!(emc->regs[REG_CAMCMR] & REG_CAMCMR_AMP); + } + case ETH_PKT_UCAST: { + bool matches; + if (emc->regs[REG_CAMCMR] & REG_CAMCMR_AUP) { + return true; + } + matches = ((emc->regs[REG_CAMCMR] & REG_CAMCMR_ECMP) && + /* We only support one CAM register, CAM0. */ + (emc->regs[REG_CAMEN] & (1 << 0)) && + memcmp(buf, emc->conf.macaddr.a, ETH_ALEN) == 0); + if (emc->regs[REG_CAMCMR] & REG_CAMCMR_CCAM) { + *fail_reason = "MACADDR matched, comparison complemented"; + return !matches; + } else { + *fail_reason = "MACADDR didn't match"; + return matches; + } + } + default: + g_assert_not_reached(); + } +} + +static bool emc_receive_filter(NPCM7xxEMCState *emc, const uint8_t *buf, + size_t len) +{ + const char *fail_reason = NULL; + bool ok = emc_receive_filter1(emc, buf, len, &fail_reason); + if (!ok) { + trace_npcm7xx_emc_packet_filtered_out(fail_reason); + } + return ok; +} + +static ssize_t emc_receive(NetClientState *nc, const uint8_t *buf, size_t len1) +{ + NPCM7xxEMCState *emc = NPCM7XX_EMC(qemu_get_nic_opaque(nc)); + const uint32_t len = len1; + size_t max_frame_len; + bool long_frame; + uint32_t desc_addr; + NPCM7xxEMCRxDesc rx_desc; + uint32_t crc; + uint8_t *crc_ptr; + uint32_t buf_addr; + + trace_npcm7xx_emc_receiving_packet(len); + + if (!emc_can_receive(nc)) { + qemu_log_mask(LOG_GUEST_ERROR, "%s: Unexpected packet\n", __func__); + return -1; + } + + if (len < ETH_HLEN || + /* Defensive programming: drop unsupportable large packets. */ + len > 0xffff - CRC_LENGTH) { + qemu_log_mask(LOG_GUEST_ERROR, "%s: Dropped frame of %u bytes\n", + __func__, len); + return len; + } + + /* + * DENI is set if EMC received the Length/Type field of the incoming + * packet, so it will be set regardless of what happens next. + */ + emc_set_mista(emc, REG_MISTA_DENI); + + if (!emc_receive_filter(emc, buf, len)) { + emc_update_rx_irq(emc); + return len; + } + + /* Huge frames (> DMARFC) are dropped. */ + max_frame_len = REG_DMARFC_RXMS(emc->regs[REG_DMARFC]); + if (len + CRC_LENGTH > max_frame_len) { + trace_npcm7xx_emc_packet_dropped(len); + emc_set_mista(emc, REG_MISTA_DFOI); + emc_update_rx_irq(emc); + return len; + } + + /* + * Long Frames (> MAX_ETH_FRAME_SIZE) are also dropped, unless MCMDR.ALP + * is set. + */ + long_frame = false; + if (len + CRC_LENGTH > MAX_ETH_FRAME_SIZE) { + if (emc->regs[REG_MCMDR] & REG_MCMDR_ALP) { + long_frame = true; + } else { + trace_npcm7xx_emc_packet_dropped(len); + emc_set_mista(emc, REG_MISTA_PTLE); + emc_update_rx_irq(emc); + return len; + } + } + + desc_addr = RX_DESC_NRXDSA(emc->regs[REG_CRXDSA]); + if (emc_read_rx_desc(desc_addr, &rx_desc)) { + /* Error reading descriptor, already reported. */ + emc_halt_rx(emc, REG_MISTA_RXBERR); + emc_update_rx_irq(emc); + return len; + } + + /* Nothing we can do if we don't own the descriptor. */ + if (!(rx_desc.status_and_length & RX_DESC_STATUS_OWNER_MASK)) { + trace_npcm7xx_emc_cpu_owned_desc(desc_addr); + emc_halt_rx(emc, REG_MISTA_RDU); + emc_update_rx_irq(emc); + return len; + } + + crc = 0; + crc_ptr = (uint8_t *) &crc; + if (!(emc->regs[REG_MCMDR] & REG_MCMDR_SPCRC)) { + crc = cpu_to_be32(crc32(~0, buf, len)); + } + + /* Give the descriptor back regardless of what happens. */ + rx_desc.status_and_length &= ~RX_DESC_STATUS_OWNER_MASK; + + buf_addr = rx_desc.rxbsa; + emc->regs[REG_CRXBSA] = buf_addr; + if (dma_memory_write(&address_space_memory, buf_addr, buf, len) || + (!(emc->regs[REG_MCMDR] & REG_MCMDR_SPCRC) && + dma_memory_write(&address_space_memory, buf_addr + len, crc_ptr, + 4))) { + qemu_log_mask(LOG_GUEST_ERROR, "%s: Bus error writing packet\n", + __func__); + emc_set_mista(emc, REG_MISTA_RXBERR); + emc_set_next_rx_descriptor(emc, &rx_desc, desc_addr); + emc_update_rx_irq(emc); + trace_npcm7xx_emc_rx_done(emc->regs[REG_CRXDSA]); + return len; + } + + trace_npcm7xx_emc_received_packet(len); + + /* Note: We've already verified len+4 <= 0xffff. */ + rx_desc.status_and_length = len; + if (!(emc->regs[REG_MCMDR] & REG_MCMDR_SPCRC)) { + rx_desc.status_and_length += 4; + } + rx_desc.status_and_length |= RX_DESC_STATUS_RXGD; + emc_set_mista(emc, REG_MISTA_RXGD); + + if (emc->regs[REG_MISTA] & emc->regs[REG_MIEN] & REG_MISTA_RXINTR) { + rx_desc.status_and_length |= RX_DESC_STATUS_RXINTR; + } + if (long_frame) { + rx_desc.status_and_length |= RX_DESC_STATUS_PTLE; + } + + emc_set_next_rx_descriptor(emc, &rx_desc, desc_addr); + emc_update_rx_irq(emc); + trace_npcm7xx_emc_rx_done(emc->regs[REG_CRXDSA]); + return len; +} + +static void emc_try_receive_next_packet(NPCM7xxEMCState *emc) +{ + if (emc_can_receive(qemu_get_queue(emc->nic))) { + qemu_flush_queued_packets(qemu_get_queue(emc->nic)); + } +} + +static uint64_t npcm7xx_emc_read(void *opaque, hwaddr offset, unsigned size) +{ + NPCM7xxEMCState *emc = opaque; + uint32_t reg = offset / sizeof(uint32_t); + uint32_t result; + + if (reg >= NPCM7XX_NUM_EMC_REGS) { + qemu_log_mask(LOG_GUEST_ERROR, + "%s: Invalid offset 0x%04" HWADDR_PRIx "\n", + __func__, offset); + return 0; + } + + switch (reg) { + case REG_MIID: + /* + * We don't implement MII. For determinism, always return zero as + * writes record the last value written for debugging purposes. + */ + qemu_log_mask(LOG_UNIMP, "%s: Read of MIID, returning 0\n", __func__); + result = 0; + break; + case REG_TSDR: + case REG_RSDR: + qemu_log_mask(LOG_GUEST_ERROR, + "%s: Read of write-only reg, %s/%d\n", + __func__, emc_reg_name(reg), reg); + return 0; + default: + result = emc->regs[reg]; + break; + } + + trace_npcm7xx_emc_reg_read(emc->emc_num, result, emc_reg_name(reg), reg); + return result; +} + +static void npcm7xx_emc_write(void *opaque, hwaddr offset, + uint64_t v, unsigned size) +{ + NPCM7xxEMCState *emc = opaque; + uint32_t reg = offset / sizeof(uint32_t); + uint32_t value = v; + + g_assert(size == sizeof(uint32_t)); + + if (reg >= NPCM7XX_NUM_EMC_REGS) { + qemu_log_mask(LOG_GUEST_ERROR, + "%s: Invalid offset 0x%04" HWADDR_PRIx "\n", + __func__, offset); + return; + } + + trace_npcm7xx_emc_reg_write(emc->emc_num, emc_reg_name(reg), reg, value); + + switch (reg) { + case REG_CAMCMR: + emc->regs[reg] = value; + break; + case REG_CAMEN: + /* Only CAM0 is supported, don't pretend otherwise. */ + if (value & ~1) { + qemu_log_mask(LOG_GUEST_ERROR, + "%s: Only CAM0 is supported, cannot enable others" + ": 0x%x\n", + __func__, value); + } + emc->regs[reg] = value & 1; + break; + case REG_CAMM_BASE + 0: + emc->regs[reg] = value; + emc->conf.macaddr.a[0] = value >> 24; + emc->conf.macaddr.a[1] = value >> 16; + emc->conf.macaddr.a[2] = value >> 8; + emc->conf.macaddr.a[3] = value >> 0; + break; + case REG_CAML_BASE + 0: + emc->regs[reg] = value; + emc->conf.macaddr.a[4] = value >> 24; + emc->conf.macaddr.a[5] = value >> 16; + break; + case REG_MCMDR: { + uint32_t prev; + if (value & REG_MCMDR_SWR) { + emc_soft_reset(emc); + /* On h/w the reset happens over multiple cycles. For now KISS. */ + break; + } + prev = emc->regs[reg]; + emc->regs[reg] = value; + /* Update tx state. */ + if (!(prev & REG_MCMDR_TXON) && + (value & REG_MCMDR_TXON)) { + emc->regs[REG_CTXDSA] = emc->regs[REG_TXDLSA]; + /* + * Linux kernel turns TX on with CPU still holding descriptor, + * which suggests we should wait for a write to TSDR before trying + * to send a packet: so we don't send one here. + */ + } else if ((prev & REG_MCMDR_TXON) && + !(value & REG_MCMDR_TXON)) { + emc->regs[REG_MGSTA] |= REG_MGSTA_TXHA; + } + if (!(value & REG_MCMDR_TXON)) { + emc_halt_tx(emc, 0); + } + /* Update rx state. */ + if (!(prev & REG_MCMDR_RXON) && + (value & REG_MCMDR_RXON)) { + emc->regs[REG_CRXDSA] = emc->regs[REG_RXDLSA]; + } else if ((prev & REG_MCMDR_RXON) && + !(value & REG_MCMDR_RXON)) { + emc->regs[REG_MGSTA] |= REG_MGSTA_RXHA; + } + if (!(value & REG_MCMDR_RXON)) { + emc_halt_rx(emc, 0); + } + break; + } + case REG_TXDLSA: + case REG_RXDLSA: + case REG_DMARFC: + case REG_MIID: + emc->regs[reg] = value; + break; + case REG_MIEN: + emc->regs[reg] = value; + emc_update_irq_from_reg_change(emc); + break; + case REG_MISTA: + /* Clear the bits that have 1 in "value". */ + emc->regs[reg] &= ~value; + emc_update_irq_from_reg_change(emc); + break; + case REG_MGSTA: + /* Clear the bits that have 1 in "value". */ + emc->regs[reg] &= ~value; + break; + case REG_TSDR: + if (emc->regs[REG_MCMDR] & REG_MCMDR_TXON) { + emc->tx_active = true; + /* Keep trying to send packets until we run out. */ + while (emc->tx_active) { + emc_try_send_next_packet(emc); + } + } + break; + case REG_RSDR: + if (emc->regs[REG_MCMDR] & REG_MCMDR_RXON) { + emc->rx_active = true; + emc_try_receive_next_packet(emc); + } + break; + case REG_MIIDA: + emc->regs[reg] = value & ~REG_MIIDA_BUSY; + break; + case REG_MRPC: + case REG_MRPCC: + case REG_MREPC: + case REG_CTXDSA: + case REG_CTXBSA: + case REG_CRXDSA: + case REG_CRXBSA: + qemu_log_mask(LOG_GUEST_ERROR, + "%s: Write to read-only reg %s/%d\n", + __func__, emc_reg_name(reg), reg); + break; + default: + qemu_log_mask(LOG_UNIMP, "%s: Write to unimplemented reg %s/%d\n", + __func__, emc_reg_name(reg), reg); + break; + } +} + +static const struct MemoryRegionOps npcm7xx_emc_ops = { + .read = npcm7xx_emc_read, + .write = npcm7xx_emc_write, + .endianness = DEVICE_LITTLE_ENDIAN, + .valid = { + .min_access_size = 4, + .max_access_size = 4, + .unaligned = false, + }, +}; + +static void emc_cleanup(NetClientState *nc) +{ + /* Nothing to do yet. */ +} + +static NetClientInfo net_npcm7xx_emc_info = { + .type = NET_CLIENT_DRIVER_NIC, + .size = sizeof(NICState), + .can_receive = emc_can_receive, + .receive = emc_receive, + .cleanup = emc_cleanup, + .link_status_changed = emc_set_link, +}; + +static void npcm7xx_emc_realize(DeviceState *dev, Error **errp) +{ + NPCM7xxEMCState *emc = NPCM7XX_EMC(dev); + SysBusDevice *sbd = SYS_BUS_DEVICE(emc); + + memory_region_init_io(&emc->iomem, OBJECT(emc), &npcm7xx_emc_ops, emc, + TYPE_NPCM7XX_EMC, 4 * KiB); + sysbus_init_mmio(sbd, &emc->iomem); + sysbus_init_irq(sbd, &emc->tx_irq); + sysbus_init_irq(sbd, &emc->rx_irq); + + qemu_macaddr_default_if_unset(&emc->conf.macaddr); + emc->nic = qemu_new_nic(&net_npcm7xx_emc_info, &emc->conf, + object_get_typename(OBJECT(dev)), dev->id, emc); + qemu_format_nic_info_str(qemu_get_queue(emc->nic), emc->conf.macaddr.a); +} + +static void npcm7xx_emc_unrealize(DeviceState *dev) +{ + NPCM7xxEMCState *emc = NPCM7XX_EMC(dev); + + qemu_del_nic(emc->nic); +} + +static const VMStateDescription vmstate_npcm7xx_emc = { + .name = TYPE_NPCM7XX_EMC, + .version_id = 0, + .minimum_version_id = 0, + .fields = (VMStateField[]) { + VMSTATE_UINT8(emc_num, NPCM7xxEMCState), + VMSTATE_UINT32_ARRAY(regs, NPCM7xxEMCState, NPCM7XX_NUM_EMC_REGS), + VMSTATE_BOOL(tx_active, NPCM7xxEMCState), + VMSTATE_BOOL(rx_active, NPCM7xxEMCState), + VMSTATE_END_OF_LIST(), + }, +}; + +static Property npcm7xx_emc_properties[] = { + DEFINE_NIC_PROPERTIES(NPCM7xxEMCState, conf), + DEFINE_PROP_END_OF_LIST(), +}; + +static void npcm7xx_emc_class_init(ObjectClass *klass, void *data) +{ + DeviceClass *dc = DEVICE_CLASS(klass); + + set_bit(DEVICE_CATEGORY_NETWORK, dc->categories); + dc->desc = "NPCM7xx EMC Controller"; + dc->realize = npcm7xx_emc_realize; + dc->unrealize = npcm7xx_emc_unrealize; + dc->reset = npcm7xx_emc_reset; + dc->vmsd = &vmstate_npcm7xx_emc; + device_class_set_props(dc, npcm7xx_emc_properties); +} + +static const TypeInfo npcm7xx_emc_info = { + .name = TYPE_NPCM7XX_EMC, + .parent = TYPE_SYS_BUS_DEVICE, + .instance_size = sizeof(NPCM7xxEMCState), + .class_init = npcm7xx_emc_class_init, +}; + +static void npcm7xx_emc_register_type(void) +{ + type_register_static(&npcm7xx_emc_info); +} + +type_init(npcm7xx_emc_register_type) diff --git a/hw/net/meson.build b/hw/net/meson.build index XXXXXXX..XXXXXXX 100644 --- a/hw/net/meson.build +++ b/hw/net/meson.build @@ -XXX,XX +XXX,XX @@ softmmu_ss.add(when: 'CONFIG_I82596_COMMON', if_true: files('i82596.c')) softmmu_ss.add(when: 'CONFIG_SUNHME', if_true: files('sunhme.c')) softmmu_ss.add(when: 'CONFIG_FTGMAC100', if_true: files('ftgmac100.c')) softmmu_ss.add(when: 'CONFIG_SUNGEM', if_true: files('sungem.c')) +softmmu_ss.add(when: 'CONFIG_NPCM7XX', if_true: files('npcm7xx_emc.c')) softmmu_ss.add(when: 'CONFIG_ETRAXFS', if_true: files('etraxfs_eth.c')) softmmu_ss.add(when: 'CONFIG_COLDFIRE', if_true: files('mcf_fec.c')) diff --git a/hw/net/trace-events b/hw/net/trace-events index XXXXXXX..XXXXXXX 100644 --- a/hw/net/trace-events +++ b/hw/net/trace-events @@ -XXX,XX +XXX,XX @@ imx_fec_receive_last(int last) "rx frame flags 0x%04x" imx_enet_receive(size_t size) "len %zu" imx_enet_receive_len(uint64_t addr, int len) "rx_bd 0x%"PRIx64" length %d" imx_enet_receive_last(int last) "rx frame flags 0x%04x" + +# npcm7xx_emc.c +npcm7xx_emc_reset(int emc_num) "Resetting emc%d" +npcm7xx_emc_update_tx_irq(int level) "Setting tx irq to %d" +npcm7xx_emc_update_rx_irq(int level) "Setting rx irq to %d" +npcm7xx_emc_set_mista(uint32_t flags) "ORing 0x%x into MISTA" +npcm7xx_emc_cpu_owned_desc(uint32_t addr) "Can't process cpu-owned descriptor @0x%x" +npcm7xx_emc_sent_packet(uint32_t len) "Sent %u byte packet" +npcm7xx_emc_tx_done(uint32_t ctxdsa) "TX done, CTXDSA=0x%x" +npcm7xx_emc_can_receive(int can_receive) "Can receive: %d" +npcm7xx_emc_packet_filtered_out(const char* fail_reason) "Packet filtered out: %s" +npcm7xx_emc_packet_dropped(uint32_t len) "%u byte packet dropped" +npcm7xx_emc_receiving_packet(uint32_t len) "Receiving %u byte packet" +npcm7xx_emc_received_packet(uint32_t len) "Received %u byte packet" +npcm7xx_emc_rx_done(uint32_t crxdsa) "RX done, CRXDSA=0x%x" +npcm7xx_emc_reg_read(int emc_num, uint32_t result, const char *name, int regno) "emc%d: 0x%x = reg[%s/%d]" +npcm7xx_emc_reg_write(int emc_num, const char *name, int regno, uint32_t value) "emc%d: reg[%s/%d] = 0x%x" -- 2.20.1 From: Doug Evans <dje@google.com> This is a 10/100 ethernet device that has several features. Only the ones needed by the Linux driver have been implemented. See npcm7xx_emc.c for a list of unimplemented features. Reviewed-by: Hao Wu <wuhaotsh@google.com> Reviewed-by: Avi Fishman <avi.fishman@nuvoton.com> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Doug Evans <dje@google.com> Message-id: 20210209015541.778833-3-dje@google.com Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- docs/system/arm/nuvoton.rst | 3 ++- include/hw/arm/npcm7xx.h | 2 ++ hw/arm/npcm7xx.c | 50 +++++++++++++++++++++++++++++++++++-- 3 files changed, 52 insertions(+), 3 deletions(-) diff --git a/docs/system/arm/nuvoton.rst b/docs/system/arm/nuvoton.rst index XXXXXXX..XXXXXXX 100644 --- a/docs/system/arm/nuvoton.rst +++ b/docs/system/arm/nuvoton.rst @@ -XXX,XX +XXX,XX @@ Supported devices * GPIO controller * Analog to Digital Converter (ADC) * Pulse Width Modulation (PWM) + * Ethernet controller (EMC) Missing devices --------------- @@ -XXX,XX +XXX,XX @@ Missing devices * Shared memory (SHM) * eSPI slave interface - * Ethernet controllers (GMAC and EMC) + * Ethernet controller (GMAC) * USB device (USBD) * SMBus controller (SMBF) * Peripheral SPI controller (PSPI) diff --git a/include/hw/arm/npcm7xx.h b/include/hw/arm/npcm7xx.h index XXXXXXX..XXXXXXX 100644 --- a/include/hw/arm/npcm7xx.h +++ b/include/hw/arm/npcm7xx.h @@ -XXX,XX +XXX,XX @@ #include "hw/misc/npcm7xx_gcr.h" #include "hw/misc/npcm7xx_pwm.h" #include "hw/misc/npcm7xx_rng.h" +#include "hw/net/npcm7xx_emc.h" #include "hw/nvram/npcm7xx_otp.h" #include "hw/timer/npcm7xx_timer.h" #include "hw/ssi/npcm7xx_fiu.h" @@ -XXX,XX +XXX,XX @@ typedef struct NPCM7xxState { EHCISysBusState ehci; OHCISysBusState ohci; NPCM7xxFIUState fiu[2]; + NPCM7xxEMCState emc[2]; } NPCM7xxState; #define TYPE_NPCM7XX "npcm7xx" diff --git a/hw/arm/npcm7xx.c b/hw/arm/npcm7xx.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/npcm7xx.c +++ b/hw/arm/npcm7xx.c @@ -XXX,XX +XXX,XX @@ enum NPCM7xxInterrupt { NPCM7XX_UART1_IRQ, NPCM7XX_UART2_IRQ, NPCM7XX_UART3_IRQ, + NPCM7XX_EMC1RX_IRQ = 15, + NPCM7XX_EMC1TX_IRQ, NPCM7XX_TIMER0_IRQ = 32, /* Timer Module 0 */ NPCM7XX_TIMER1_IRQ, NPCM7XX_TIMER2_IRQ, @@ -XXX,XX +XXX,XX @@ enum NPCM7xxInterrupt { NPCM7XX_OHCI_IRQ = 62, NPCM7XX_PWM0_IRQ = 93, /* PWM module 0 */ NPCM7XX_PWM1_IRQ, /* PWM module 1 */ + NPCM7XX_EMC2RX_IRQ = 114, + NPCM7XX_EMC2TX_IRQ, NPCM7XX_GPIO0_IRQ = 116, NPCM7XX_GPIO1_IRQ, NPCM7XX_GPIO2_IRQ, @@ -XXX,XX +XXX,XX @@ static const hwaddr npcm7xx_pwm_addr[] = { 0xf0104000, }; +/* Register base address for each EMC Module */ +static const hwaddr npcm7xx_emc_addr[] = { + 0xf0825000, + 0xf0826000, +}; + static const struct { hwaddr regs_addr; uint32_t unconnected_pins; @@ -XXX,XX +XXX,XX @@ static void npcm7xx_init(Object *obj) for (i = 0; i < ARRAY_SIZE(s->pwm); i++) { object_initialize_child(obj, "pwm[*]", &s->pwm[i], TYPE_NPCM7XX_PWM); } + + for (i = 0; i < ARRAY_SIZE(s->emc); i++) { + object_initialize_child(obj, "emc[*]", &s->emc[i], TYPE_NPCM7XX_EMC); + } } static void npcm7xx_realize(DeviceState *dev, Error **errp) @@ -XXX,XX +XXX,XX @@ static void npcm7xx_realize(DeviceState *dev, Error **errp) sysbus_connect_irq(sbd, i, npcm7xx_irq(s, NPCM7XX_PWM0_IRQ + i)); } + /* + * EMC Modules. Cannot fail. + * The mapping of the device to its netdev backend works as follows: + * emc[i] = nd_table[i] + * This works around the inability to specify the netdev property for the + * emc device: it's not pluggable and thus the -device option can't be + * used. + */ + QEMU_BUILD_BUG_ON(ARRAY_SIZE(npcm7xx_emc_addr) != ARRAY_SIZE(s->emc)); + QEMU_BUILD_BUG_ON(ARRAY_SIZE(s->emc) != 2); + for (i = 0; i < ARRAY_SIZE(s->emc); i++) { + s->emc[i].emc_num = i; + SysBusDevice *sbd = SYS_BUS_DEVICE(&s->emc[i]); + if (nd_table[i].used) { + qemu_check_nic_model(&nd_table[i], TYPE_NPCM7XX_EMC); + qdev_set_nic_properties(DEVICE(sbd), &nd_table[i]); + } + /* + * The device exists regardless of whether it's connected to a QEMU + * netdev backend. So always instantiate it even if there is no + * backend. + */ + sysbus_realize(sbd, &error_abort); + sysbus_mmio_map(sbd, 0, npcm7xx_emc_addr[i]); + int tx_irq = i == 0 ? NPCM7XX_EMC1TX_IRQ : NPCM7XX_EMC2TX_IRQ; + int rx_irq = i == 0 ? NPCM7XX_EMC1RX_IRQ : NPCM7XX_EMC2RX_IRQ; + /* + * N.B. The values for the second argument sysbus_connect_irq are + * chosen to match the registration order in npcm7xx_emc_realize. + */ + sysbus_connect_irq(sbd, 0, npcm7xx_irq(s, tx_irq)); + sysbus_connect_irq(sbd, 1, npcm7xx_irq(s, rx_irq)); + } + /* * Flash Interface Unit (FIU). Can fail if incorrect number of chip selects * specified, but this is a programming error. @@ -XXX,XX +XXX,XX @@ static void npcm7xx_realize(DeviceState *dev, Error **errp) create_unimplemented_device("npcm7xx.vcd", 0xf0810000, 64 * KiB); create_unimplemented_device("npcm7xx.ece", 0xf0820000, 8 * KiB); create_unimplemented_device("npcm7xx.vdma", 0xf0822000, 8 * KiB); - create_unimplemented_device("npcm7xx.emc1", 0xf0825000, 4 * KiB); - create_unimplemented_device("npcm7xx.emc2", 0xf0826000, 4 * KiB); create_unimplemented_device("npcm7xx.usbd[0]", 0xf0830000, 4 * KiB); create_unimplemented_device("npcm7xx.usbd[1]", 0xf0831000, 4 * KiB); create_unimplemented_device("npcm7xx.usbd[2]", 0xf0832000, 4 * KiB); -- 2.20.1 From: Doug Evans <dje@google.com> Reviewed-by: Hao Wu <wuhaotsh@google.com> Reviewed-by: Avi Fishman <avi.fishman@nuvoton.com> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Doug Evans <dje@google.com> Message-id: 20210209015541.778833-4-dje@google.com Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- tests/qtest/npcm7xx_emc-test.c | 812 +++++++++++++++++++++++++++++++++ tests/qtest/meson.build | 1 + 2 files changed, 813 insertions(+) create mode 100644 tests/qtest/npcm7xx_emc-test.c diff --git a/tests/qtest/npcm7xx_emc-test.c b/tests/qtest/npcm7xx_emc-test.c new file mode 100644 index XXXXXXX..XXXXXXX --- /dev/null +++ b/tests/qtest/npcm7xx_emc-test.c @@ -XXX,XX +XXX,XX @@ +/* + * QTests for Nuvoton NPCM7xx EMC Modules. + * + * Copyright 2020 Google LLC + * + * This program is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License as published by the + * Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License + * for more details. + */ + +#include "qemu/osdep.h" +#include "qemu-common.h" +#include "libqos/libqos.h" +#include "qapi/qmp/qdict.h" +#include "qapi/qmp/qnum.h" +#include "qemu/bitops.h" +#include "qemu/iov.h" + +/* Name of the emc device. */ +#define TYPE_NPCM7XX_EMC "npcm7xx-emc" + +/* Timeout for various operations, in seconds. */ +#define TIMEOUT_SECONDS 10 + +/* Address in memory of the descriptor. */ +#define DESC_ADDR (1 << 20) /* 1 MiB */ + +/* Address in memory of the data packet. */ +#define DATA_ADDR (DESC_ADDR + 4096) + +#define CRC_LENGTH 4 + +#define NUM_TX_DESCRIPTORS 3 +#define NUM_RX_DESCRIPTORS 2 + +/* Size of tx,rx test buffers. */ +#define TX_DATA_LEN 64 +#define RX_DATA_LEN 64 + +#define TX_STEP_COUNT 10000 +#define RX_STEP_COUNT 10000 + +/* 32-bit register indices. */ +typedef enum NPCM7xxPWMRegister { + /* Control registers. */ + REG_CAMCMR, + REG_CAMEN, + + /* There are 16 CAMn[ML] registers. */ + REG_CAMM_BASE, + REG_CAML_BASE, + + REG_TXDLSA = 0x22, + REG_RXDLSA, + REG_MCMDR, + REG_MIID, + REG_MIIDA, + REG_FFTCR, + REG_TSDR, + REG_RSDR, + REG_DMARFC, + REG_MIEN, + + /* Status registers. */ + REG_MISTA, + REG_MGSTA, + REG_MPCNT, + REG_MRPC, + REG_MRPCC, + REG_MREPC, + REG_DMARFS, + REG_CTXDSA, + REG_CTXBSA, + REG_CRXDSA, + REG_CRXBSA, + + NPCM7XX_NUM_EMC_REGS, +} NPCM7xxPWMRegister; + +enum { NUM_CAMML_REGS = 16 }; + +/* REG_CAMCMR fields */ +/* Enable CAM Compare */ +#define REG_CAMCMR_ECMP (1 << 4) +/* Accept Unicast Packet */ +#define REG_CAMCMR_AUP (1 << 0) + +/* REG_MCMDR fields */ +/* Software Reset */ +#define REG_MCMDR_SWR (1 << 24) +/* Frame Transmission On */ +#define REG_MCMDR_TXON (1 << 8) +/* Accept Long Packet */ +#define REG_MCMDR_ALP (1 << 1) +/* Frame Reception On */ +#define REG_MCMDR_RXON (1 << 0) + +/* REG_MIEN fields */ +/* Enable Transmit Completion Interrupt */ +#define REG_MIEN_ENTXCP (1 << 18) +/* Enable Transmit Interrupt */ +#define REG_MIEN_ENTXINTR (1 << 16) +/* Enable Receive Good Interrupt */ +#define REG_MIEN_ENRXGD (1 << 4) +/* ENable Receive Interrupt */ +#define REG_MIEN_ENRXINTR (1 << 0) + +/* REG_MISTA fields */ +/* Transmit Bus Error Interrupt */ +#define REG_MISTA_TXBERR (1 << 24) +/* Transmit Descriptor Unavailable Interrupt */ +#define REG_MISTA_TDU (1 << 23) +/* Transmit Completion Interrupt */ +#define REG_MISTA_TXCP (1 << 18) +/* Transmit Interrupt */ +#define REG_MISTA_TXINTR (1 << 16) +/* Receive Bus Error Interrupt */ +#define REG_MISTA_RXBERR (1 << 11) +/* Receive Descriptor Unavailable Interrupt */ +#define REG_MISTA_RDU (1 << 10) +/* DMA Early Notification Interrupt */ +#define REG_MISTA_DENI (1 << 9) +/* Maximum Frame Length Interrupt */ +#define REG_MISTA_DFOI (1 << 8) +/* Receive Good Interrupt */ +#define REG_MISTA_RXGD (1 << 4) +/* Packet Too Long Interrupt */ +#define REG_MISTA_PTLE (1 << 3) +/* Receive Interrupt */ +#define REG_MISTA_RXINTR (1 << 0) + +typedef struct NPCM7xxEMCTxDesc NPCM7xxEMCTxDesc; +typedef struct NPCM7xxEMCRxDesc NPCM7xxEMCRxDesc; + +struct NPCM7xxEMCTxDesc { + uint32_t flags; + uint32_t txbsa; + uint32_t status_and_length; + uint32_t ntxdsa; +}; + +struct NPCM7xxEMCRxDesc { + uint32_t status_and_length; + uint32_t rxbsa; + uint32_t reserved; + uint32_t nrxdsa; +}; + +/* NPCM7xxEMCTxDesc.flags values */ +/* Owner: 0 = cpu, 1 = emc */ +#define TX_DESC_FLAG_OWNER_MASK (1 << 31) +/* Transmit interrupt enable */ +#define TX_DESC_FLAG_INTEN (1 << 2) + +/* NPCM7xxEMCTxDesc.status_and_length values */ +/* Transmission complete */ +#define TX_DESC_STATUS_TXCP (1 << 19) +/* Transmit interrupt */ +#define TX_DESC_STATUS_TXINTR (1 << 16) + +/* NPCM7xxEMCRxDesc.status_and_length values */ +/* Owner: 0b00 = cpu, 0b10 = emc */ +#define RX_DESC_STATUS_OWNER_SHIFT 30 +#define RX_DESC_STATUS_OWNER_MASK 0xc0000000 +/* Frame Reception Complete */ +#define RX_DESC_STATUS_RXGD (1 << 20) +/* Packet too long */ +#define RX_DESC_STATUS_PTLE (1 << 19) +/* Receive Interrupt */ +#define RX_DESC_STATUS_RXINTR (1 << 16) + +#define RX_DESC_PKT_LEN(word) ((uint32_t) (word) & 0xffff) + +typedef struct EMCModule { + int rx_irq; + int tx_irq; + uint64_t base_addr; +} EMCModule; + +typedef struct TestData { + const EMCModule *module; +} TestData; + +static const EMCModule emc_module_list[] = { + { + .rx_irq = 15, + .tx_irq = 16, + .base_addr = 0xf0825000 + }, + { + .rx_irq = 114, + .tx_irq = 115, + .base_addr = 0xf0826000 + } +}; + +/* Returns the index of the EMC module. */ +static int emc_module_index(const EMCModule *mod) +{ + ptrdiff_t diff = mod - emc_module_list; + + g_assert_true(diff >= 0 && diff < ARRAY_SIZE(emc_module_list)); + + return diff; +} + +static void packet_test_clear(void *sockets) +{ + int *test_sockets = sockets; + + close(test_sockets[0]); + g_free(test_sockets); +} + +static int *packet_test_init(int module_num, GString *cmd_line) +{ + int *test_sockets = g_new(int, 2); + int ret = socketpair(PF_UNIX, SOCK_STREAM, 0, test_sockets); + g_assert_cmpint(ret, != , -1); + + /* + * KISS and use -nic. We specify two nics (both emc{0,1}) because there's + * currently no way to specify only emc1: The driver implicitly relies on + * emc[i] == nd_table[i]. + */ + if (module_num == 0) { + g_string_append_printf(cmd_line, + " -nic socket,fd=%d,model=" TYPE_NPCM7XX_EMC " " + " -nic user,model=" TYPE_NPCM7XX_EMC " ", + test_sockets[1]); + } else { + g_string_append_printf(cmd_line, + " -nic user,model=" TYPE_NPCM7XX_EMC " " + " -nic socket,fd=%d,model=" TYPE_NPCM7XX_EMC " ", + test_sockets[1]); + } + + g_test_queue_destroy(packet_test_clear, test_sockets); + return test_sockets; +} + +static uint32_t emc_read(QTestState *qts, const EMCModule *mod, + NPCM7xxPWMRegister regno) +{ + return qtest_readl(qts, mod->base_addr + regno * sizeof(uint32_t)); +} + +static void emc_write(QTestState *qts, const EMCModule *mod, + NPCM7xxPWMRegister regno, uint32_t value) +{ + qtest_writel(qts, mod->base_addr + regno * sizeof(uint32_t), value); +} + +/* + * Reset the EMC module. + * The module must be reset before, e.g., TXDLSA,RXDLSA are changed. + */ +static bool emc_soft_reset(QTestState *qts, const EMCModule *mod) +{ + uint32_t val; + uint64_t end_time; + + emc_write(qts, mod, REG_MCMDR, REG_MCMDR_SWR); + + /* + * Wait for device to reset as the linux driver does. + * During reset the AHB reads 0 for all registers. So first wait for + * something that resets to non-zero, and then wait for SWR becoming 0. + */ + end_time = g_get_monotonic_time() + TIMEOUT_SECONDS * G_TIME_SPAN_SECOND; + + do { + qtest_clock_step(qts, 100); + val = emc_read(qts, mod, REG_FFTCR); + } while (val == 0 && g_get_monotonic_time() < end_time); + if (val != 0) { + do { + qtest_clock_step(qts, 100); + val = emc_read(qts, mod, REG_MCMDR); + if ((val & REG_MCMDR_SWR) == 0) { + /* + * N.B. The CAMs have been reset here, so macaddr matching of + * incoming packets will not work. + */ + return true; + } + } while (g_get_monotonic_time() < end_time); + } + + g_message("%s: Timeout expired", __func__); + return false; +} + +/* Check emc registers are reset to default value. */ +static void test_init(gconstpointer test_data) +{ + const TestData *td = test_data; + const EMCModule *mod = td->module; + QTestState *qts = qtest_init("-machine quanta-gsj"); + int i; + +#define CHECK_REG(regno, value) \ + do { \ + g_assert_cmphex(emc_read(qts, mod, (regno)), ==, (value)); \ + } while (0) + + CHECK_REG(REG_CAMCMR, 0); + CHECK_REG(REG_CAMEN, 0); + CHECK_REG(REG_TXDLSA, 0xfffffffc); + CHECK_REG(REG_RXDLSA, 0xfffffffc); + CHECK_REG(REG_MCMDR, 0); + CHECK_REG(REG_MIID, 0); + CHECK_REG(REG_MIIDA, 0x00900000); + CHECK_REG(REG_FFTCR, 0x0101); + CHECK_REG(REG_DMARFC, 0x0800); + CHECK_REG(REG_MIEN, 0); + CHECK_REG(REG_MISTA, 0); + CHECK_REG(REG_MGSTA, 0); + CHECK_REG(REG_MPCNT, 0x7fff); + CHECK_REG(REG_MRPC, 0); + CHECK_REG(REG_MRPCC, 0); + CHECK_REG(REG_MREPC, 0); + CHECK_REG(REG_DMARFS, 0); + CHECK_REG(REG_CTXDSA, 0); + CHECK_REG(REG_CTXBSA, 0); + CHECK_REG(REG_CRXDSA, 0); + CHECK_REG(REG_CRXBSA, 0); + +#undef CHECK_REG + + for (i = 0; i < NUM_CAMML_REGS; ++i) { + g_assert_cmpuint(emc_read(qts, mod, REG_CAMM_BASE + i * 2), ==, + 0); + g_assert_cmpuint(emc_read(qts, mod, REG_CAML_BASE + i * 2), ==, + 0); + } + + qtest_quit(qts); +} + +static bool emc_wait_irq(QTestState *qts, const EMCModule *mod, int step, + bool is_tx) +{ + uint64_t end_time = + g_get_monotonic_time() + TIMEOUT_SECONDS * G_TIME_SPAN_SECOND; + + do { + if (qtest_get_irq(qts, is_tx ? mod->tx_irq : mod->rx_irq)) { + return true; + } + qtest_clock_step(qts, step); + } while (g_get_monotonic_time() < end_time); + + g_message("%s: Timeout expired", __func__); + return false; +} + +static bool emc_wait_mista(QTestState *qts, const EMCModule *mod, int step, + uint32_t flag) +{ + uint64_t end_time = + g_get_monotonic_time() + TIMEOUT_SECONDS * G_TIME_SPAN_SECOND; + + do { + uint32_t mista = emc_read(qts, mod, REG_MISTA); + if (mista & flag) { + return true; + } + qtest_clock_step(qts, step); + } while (g_get_monotonic_time() < end_time); + + g_message("%s: Timeout expired", __func__); + return false; +} + +static bool wait_socket_readable(int fd) +{ + fd_set read_fds; + struct timeval tv; + int rv; + + FD_ZERO(&read_fds); + FD_SET(fd, &read_fds); + tv.tv_sec = TIMEOUT_SECONDS; + tv.tv_usec = 0; + rv = select(fd + 1, &read_fds, NULL, NULL, &tv); + if (rv == -1) { + perror("select"); + } else if (rv == 0) { + g_message("%s: Timeout expired", __func__); + } + return rv == 1; +} + +static void init_tx_desc(NPCM7xxEMCTxDesc *desc, size_t count, + uint32_t desc_addr) +{ + g_assert(count >= 2); + memset(&desc[0], 0, sizeof(*desc) * count); + /* Leave the last one alone, owned by the cpu -> stops transmission. */ + for (size_t i = 0; i < count - 1; ++i) { + desc[i].flags = + cpu_to_le32(TX_DESC_FLAG_OWNER_MASK | /* owner = 1: emc */ + TX_DESC_FLAG_INTEN | + 0 | /* crc append = 0 */ + 0 /* padding enable = 0 */); + desc[i].status_and_length = + cpu_to_le32(0 | /* collision count = 0 */ + 0 | /* SQE = 0 */ + 0 | /* PAU = 0 */ + 0 | /* TXHA = 0 */ + 0 | /* LC = 0 */ + 0 | /* TXABT = 0 */ + 0 | /* NCS = 0 */ + 0 | /* EXDEF = 0 */ + 0 | /* TXCP = 0 */ + 0 | /* DEF = 0 */ + 0 | /* TXINTR = 0 */ + 0 /* length filled in later */); + desc[i].ntxdsa = cpu_to_le32(desc_addr + (i + 1) * sizeof(*desc)); + } +} + +static void enable_tx(QTestState *qts, const EMCModule *mod, + const NPCM7xxEMCTxDesc *desc, size_t count, + uint32_t desc_addr, uint32_t mien_flags) +{ + /* Write the descriptors to guest memory. */ + qtest_memwrite(qts, desc_addr, desc, sizeof(*desc) * count); + + /* Trigger sending the packet. */ + /* The module must be reset before changing TXDLSA. */ + g_assert(emc_soft_reset(qts, mod)); + emc_write(qts, mod, REG_TXDLSA, desc_addr); + emc_write(qts, mod, REG_CTXDSA, ~0); + emc_write(qts, mod, REG_MIEN, REG_MIEN_ENTXCP | mien_flags); + { + uint32_t mcmdr = emc_read(qts, mod, REG_MCMDR); + mcmdr |= REG_MCMDR_TXON; + emc_write(qts, mod, REG_MCMDR, mcmdr); + } + + /* Prod the device to send the packet. */ + emc_write(qts, mod, REG_TSDR, 1); +} + +static void emc_send_verify1(QTestState *qts, const EMCModule *mod, int fd, + bool with_irq, uint32_t desc_addr, + uint32_t next_desc_addr, + const char *test_data, int test_size) +{ + NPCM7xxEMCTxDesc result_desc; + uint32_t expected_mask, expected_value, recv_len; + int ret; + char buffer[TX_DATA_LEN]; + + g_assert(wait_socket_readable(fd)); + + /* Read the descriptor back. */ + qtest_memread(qts, desc_addr, &result_desc, sizeof(result_desc)); + /* Descriptor should be owned by cpu now. */ + g_assert((result_desc.flags & TX_DESC_FLAG_OWNER_MASK) == 0); + /* Test the status bits, ignoring the length field. */ + expected_mask = 0xffff << 16; + expected_value = TX_DESC_STATUS_TXCP; + if (with_irq) { + expected_value |= TX_DESC_STATUS_TXINTR; + } + g_assert_cmphex((result_desc.status_and_length & expected_mask), ==, + expected_value); + + /* Check data sent to the backend. */ + recv_len = ~0; + ret = qemu_recv(fd, &recv_len, sizeof(recv_len), MSG_DONTWAIT); + g_assert_cmpint(ret, == , sizeof(recv_len)); + + g_assert(wait_socket_readable(fd)); + memset(buffer, 0xff, sizeof(buffer)); + ret = qemu_recv(fd, buffer, test_size, MSG_DONTWAIT); + g_assert_cmpmem(buffer, ret, test_data, test_size); +} + +static void emc_send_verify(QTestState *qts, const EMCModule *mod, int fd, + bool with_irq) +{ + NPCM7xxEMCTxDesc desc[NUM_TX_DESCRIPTORS]; + uint32_t desc_addr = DESC_ADDR; + static const char test1_data[] = "TEST1"; + static const char test2_data[] = "Testing 1 2 3 ..."; + uint32_t data1_addr = DATA_ADDR; + uint32_t data2_addr = data1_addr + sizeof(test1_data); + bool got_tdu; + uint32_t end_desc_addr; + + /* Prepare test data buffer. */ + qtest_memwrite(qts, data1_addr, test1_data, sizeof(test1_data)); + qtest_memwrite(qts, data2_addr, test2_data, sizeof(test2_data)); + + init_tx_desc(&desc[0], NUM_TX_DESCRIPTORS, desc_addr); + desc[0].txbsa = cpu_to_le32(data1_addr); + desc[0].status_and_length |= sizeof(test1_data); + desc[1].txbsa = cpu_to_le32(data2_addr); + desc[1].status_and_length |= sizeof(test2_data); + + enable_tx(qts, mod, &desc[0], NUM_TX_DESCRIPTORS, desc_addr, + with_irq ? REG_MIEN_ENTXINTR : 0); + + /* + * It's problematic to observe the interrupt for each packet. + * Instead just wait until all the packets go out. + */ + got_tdu = false; + while (!got_tdu) { + if (with_irq) { + g_assert_true(emc_wait_irq(qts, mod, TX_STEP_COUNT, + /*is_tx=*/true)); + } else { + g_assert_true(emc_wait_mista(qts, mod, TX_STEP_COUNT, + REG_MISTA_TXINTR)); + } + got_tdu = !!(emc_read(qts, mod, REG_MISTA) & REG_MISTA_TDU); + /* If we don't have TDU yet, reset the interrupt. */ + if (!got_tdu) { + emc_write(qts, mod, REG_MISTA, + emc_read(qts, mod, REG_MISTA) & 0xffff0000); + } + } + + end_desc_addr = desc_addr + 2 * sizeof(desc[0]); + g_assert_cmphex(emc_read(qts, mod, REG_CTXDSA), ==, end_desc_addr); + g_assert_cmphex(emc_read(qts, mod, REG_MISTA), ==, + REG_MISTA_TXCP | REG_MISTA_TXINTR | REG_MISTA_TDU); + + emc_send_verify1(qts, mod, fd, with_irq, + desc_addr, end_desc_addr, + test1_data, sizeof(test1_data)); + emc_send_verify1(qts, mod, fd, with_irq, + desc_addr + sizeof(desc[0]), end_desc_addr, + test2_data, sizeof(test2_data)); +} + +static void init_rx_desc(NPCM7xxEMCRxDesc *desc, size_t count, + uint32_t desc_addr, uint32_t data_addr) +{ + g_assert_true(count >= 2); + memset(desc, 0, sizeof(*desc) * count); + desc[0].rxbsa = cpu_to_le32(data_addr); + desc[0].status_and_length = + cpu_to_le32(0b10 << RX_DESC_STATUS_OWNER_SHIFT | /* owner = 10: emc */ + 0 | /* RP = 0 */ + 0 | /* ALIE = 0 */ + 0 | /* RXGD = 0 */ + 0 | /* PTLE = 0 */ + 0 | /* CRCE = 0 */ + 0 | /* RXINTR = 0 */ + 0 /* length (filled in later) */); + /* Leave the last one alone, owned by the cpu -> stops transmission. */ + desc[0].nrxdsa = cpu_to_le32(desc_addr + sizeof(*desc)); +} + +static void enable_rx(QTestState *qts, const EMCModule *mod, + const NPCM7xxEMCRxDesc *desc, size_t count, + uint32_t desc_addr, uint32_t mien_flags, + uint32_t mcmdr_flags) +{ + /* + * Write the descriptor to guest memory. + * FWIW, IWBN if the docs said the buffer needs to be at least DMARFC + * bytes. + */ + qtest_memwrite(qts, desc_addr, desc, sizeof(*desc) * count); + + /* Trigger receiving the packet. */ + /* The module must be reset before changing RXDLSA. */ + g_assert(emc_soft_reset(qts, mod)); + emc_write(qts, mod, REG_RXDLSA, desc_addr); + emc_write(qts, mod, REG_MIEN, REG_MIEN_ENRXGD | mien_flags); + + /* + * We don't know what the device's macaddr is, so just accept all + * unicast packets (AUP). + */ + emc_write(qts, mod, REG_CAMCMR, REG_CAMCMR_AUP); + emc_write(qts, mod, REG_CAMEN, 1 << 0); + { + uint32_t mcmdr = emc_read(qts, mod, REG_MCMDR); + mcmdr |= REG_MCMDR_RXON | mcmdr_flags; + emc_write(qts, mod, REG_MCMDR, mcmdr); + } + + /* Prod the device to accept a packet. */ + emc_write(qts, mod, REG_RSDR, 1); +} + +static void emc_recv_verify(QTestState *qts, const EMCModule *mod, int fd, + bool with_irq) +{ + NPCM7xxEMCRxDesc desc[NUM_RX_DESCRIPTORS]; + uint32_t desc_addr = DESC_ADDR; + uint32_t data_addr = DATA_ADDR; + int ret; + uint32_t expected_mask, expected_value; + NPCM7xxEMCRxDesc result_desc; + + /* Prepare test data buffer. */ + const char test[RX_DATA_LEN] = "TEST"; + int len = htonl(sizeof(test)); + const struct iovec iov[] = { + { + .iov_base = &len, + .iov_len = sizeof(len), + },{ + .iov_base = (char *) test, + .iov_len = sizeof(test), + }, + }; + + /* + * Reset the device BEFORE sending a test packet, otherwise the packet + * may get swallowed by an active device of an earlier test. + */ + init_rx_desc(&desc[0], NUM_RX_DESCRIPTORS, desc_addr, data_addr); + enable_rx(qts, mod, &desc[0], NUM_RX_DESCRIPTORS, desc_addr, + with_irq ? REG_MIEN_ENRXINTR : 0, 0); + + /* Send test packet to device's socket. */ + ret = iov_send(fd, iov, 2, 0, sizeof(len) + sizeof(test)); + g_assert_cmpint(ret, == , sizeof(test) + sizeof(len)); + + /* Wait for RX interrupt. */ + if (with_irq) { + g_assert_true(emc_wait_irq(qts, mod, RX_STEP_COUNT, /*is_tx=*/false)); + } else { + g_assert_true(emc_wait_mista(qts, mod, RX_STEP_COUNT, REG_MISTA_RXGD)); + } + + g_assert_cmphex(emc_read(qts, mod, REG_CRXDSA), ==, + desc_addr + sizeof(desc[0])); + + expected_mask = 0xffff; + expected_value = (REG_MISTA_DENI | + REG_MISTA_RXGD | + REG_MISTA_RXINTR); + g_assert_cmphex((emc_read(qts, mod, REG_MISTA) & expected_mask), + ==, expected_value); + + /* Read the descriptor back. */ + qtest_memread(qts, desc_addr, &result_desc, sizeof(result_desc)); + /* Descriptor should be owned by cpu now. */ + g_assert((result_desc.status_and_length & RX_DESC_STATUS_OWNER_MASK) == 0); + /* Test the status bits, ignoring the length field. */ + expected_mask = 0xffff << 16; + expected_value = RX_DESC_STATUS_RXGD; + if (with_irq) { + expected_value |= RX_DESC_STATUS_RXINTR; + } + g_assert_cmphex((result_desc.status_and_length & expected_mask), ==, + expected_value); + g_assert_cmpint(RX_DESC_PKT_LEN(result_desc.status_and_length), ==, + RX_DATA_LEN + CRC_LENGTH); + + { + char buffer[RX_DATA_LEN]; + qtest_memread(qts, data_addr, buffer, sizeof(buffer)); + g_assert_cmpstr(buffer, == , "TEST"); + } +} + +static void emc_test_ptle(QTestState *qts, const EMCModule *mod, int fd) +{ + NPCM7xxEMCRxDesc desc[NUM_RX_DESCRIPTORS]; + uint32_t desc_addr = DESC_ADDR; + uint32_t data_addr = DATA_ADDR; + int ret; + NPCM7xxEMCRxDesc result_desc; + uint32_t expected_mask, expected_value; + + /* Prepare test data buffer. */ +#define PTLE_DATA_LEN 1600 + char test_data[PTLE_DATA_LEN]; + int len = htonl(sizeof(test_data)); + const struct iovec iov[] = { + { + .iov_base = &len, + .iov_len = sizeof(len), + },{ + .iov_base = (char *) test_data, + .iov_len = sizeof(test_data), + }, + }; + memset(test_data, 42, sizeof(test_data)); + + /* + * Reset the device BEFORE sending a test packet, otherwise the packet + * may get swallowed by an active device of an earlier test. + */ + init_rx_desc(&desc[0], NUM_RX_DESCRIPTORS, desc_addr, data_addr); + enable_rx(qts, mod, &desc[0], NUM_RX_DESCRIPTORS, desc_addr, + REG_MIEN_ENRXINTR, REG_MCMDR_ALP); + + /* Send test packet to device's socket. */ + ret = iov_send(fd, iov, 2, 0, sizeof(len) + sizeof(test_data)); + g_assert_cmpint(ret, == , sizeof(test_data) + sizeof(len)); + + /* Wait for RX interrupt. */ + g_assert_true(emc_wait_irq(qts, mod, RX_STEP_COUNT, /*is_tx=*/false)); + + /* Read the descriptor back. */ + qtest_memread(qts, desc_addr, &result_desc, sizeof(result_desc)); + /* Descriptor should be owned by cpu now. */ + g_assert((result_desc.status_and_length & RX_DESC_STATUS_OWNER_MASK) == 0); + /* Test the status bits, ignoring the length field. */ + expected_mask = 0xffff << 16; + expected_value = (RX_DESC_STATUS_RXGD | + RX_DESC_STATUS_PTLE | + RX_DESC_STATUS_RXINTR); + g_assert_cmphex((result_desc.status_and_length & expected_mask), ==, + expected_value); + g_assert_cmpint(RX_DESC_PKT_LEN(result_desc.status_and_length), ==, + PTLE_DATA_LEN + CRC_LENGTH); + + { + char buffer[PTLE_DATA_LEN]; + qtest_memread(qts, data_addr, buffer, sizeof(buffer)); + g_assert(memcmp(buffer, test_data, PTLE_DATA_LEN) == 0); + } +} + +static void test_tx(gconstpointer test_data) +{ + const TestData *td = test_data; + GString *cmd_line = g_string_new("-machine quanta-gsj"); + int *test_sockets = packet_test_init(emc_module_index(td->module), + cmd_line); + QTestState *qts = qtest_init(cmd_line->str); + + /* + * TODO: For pedantic correctness test_sockets[0] should be closed after + * the fork and before the exec, but that will require some harness + * improvements. + */ + close(test_sockets[1]); + /* Defensive programming */ + test_sockets[1] = -1; + + qtest_irq_intercept_in(qts, "/machine/soc/a9mpcore/gic"); + + emc_send_verify(qts, td->module, test_sockets[0], /*with_irq=*/false); + emc_send_verify(qts, td->module, test_sockets[0], /*with_irq=*/true); + + qtest_quit(qts); +} + +static void test_rx(gconstpointer test_data) +{ + const TestData *td = test_data; + GString *cmd_line = g_string_new("-machine quanta-gsj"); + int *test_sockets = packet_test_init(emc_module_index(td->module), + cmd_line); + QTestState *qts = qtest_init(cmd_line->str); + + /* + * TODO: For pedantic correctness test_sockets[0] should be closed after + * the fork and before the exec, but that will require some harness + * improvements. + */ + close(test_sockets[1]); + /* Defensive programming */ + test_sockets[1] = -1; + + qtest_irq_intercept_in(qts, "/machine/soc/a9mpcore/gic"); + + emc_recv_verify(qts, td->module, test_sockets[0], /*with_irq=*/false); + emc_recv_verify(qts, td->module, test_sockets[0], /*with_irq=*/true); + emc_test_ptle(qts, td->module, test_sockets[0]); + + qtest_quit(qts); +} + +static void emc_add_test(const char *name, const TestData* td, + GTestDataFunc fn) +{ + g_autofree char *full_name = g_strdup_printf( + "npcm7xx_emc/emc[%d]/%s", emc_module_index(td->module), name); + qtest_add_data_func(full_name, td, fn); +} +#define add_test(name, td) emc_add_test(#name, td, test_##name) + +int main(int argc, char **argv) +{ + TestData test_data_list[ARRAY_SIZE(emc_module_list)]; + + g_test_init(&argc, &argv, NULL); + + for (int i = 0; i < ARRAY_SIZE(emc_module_list); ++i) { + TestData *td = &test_data_list[i]; + + td->module = &emc_module_list[i]; + + add_test(init, td); + add_test(tx, td); + add_test(rx, td); + } + + return g_test_run(); +} diff --git a/tests/qtest/meson.build b/tests/qtest/meson.build index XXXXXXX..XXXXXXX 100644 --- a/tests/qtest/meson.build +++ b/tests/qtest/meson.build @@ -XXX,XX +XXX,XX @@ qtests_sparc64 = \ qtests_npcm7xx = \ ['npcm7xx_adc-test', + 'npcm7xx_emc-test', 'npcm7xx_gpio-test', 'npcm7xx_pwm-test', 'npcm7xx_rng-test', -- 2.20.1 From: "Edgar E. Iglesias" <edgar.iglesias@xilinx.com> Use nr_apu_cpus in favor of hard coding 2. Signed-off-by: Edgar E. Iglesias <edgar.iglesias@xilinx.com> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Reviewed-by: Luc Michel <luc@lmichel.fr> Message-id: 20210210142048.3125878-2-edgar.iglesias@gmail.com Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- hw/arm/xlnx-versal.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/hw/arm/xlnx-versal.c b/hw/arm/xlnx-versal.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/xlnx-versal.c +++ b/hw/arm/xlnx-versal.c @@ -XXX,XX +XXX,XX @@ static void versal_create_apu_gic(Versal *s, qemu_irq *pic) gicbusdev = SYS_BUS_DEVICE(&s->fpd.apu.gic); gicdev = DEVICE(&s->fpd.apu.gic); qdev_prop_set_uint32(gicdev, "revision", 3); - qdev_prop_set_uint32(gicdev, "num-cpu", 2); + qdev_prop_set_uint32(gicdev, "num-cpu", nr_apu_cpus); qdev_prop_set_uint32(gicdev, "num-irq", XLNX_VERSAL_NR_IRQS + 32); qdev_prop_set_uint32(gicdev, "len-redist-region-count", 1); - qdev_prop_set_uint32(gicdev, "redist-region-count[0]", 2); + qdev_prop_set_uint32(gicdev, "redist-region-count[0]", nr_apu_cpus); qdev_prop_set_bit(gicdev, "has-security-extensions", true); sysbus_realize(SYS_BUS_DEVICE(&s->fpd.apu.gic), &error_fatal); -- 2.20.1 From: Daniel Müller <muellerd@fb.com> When working with performance monitoring counters, we look at MDCR_EL2.HPMN as part of the check whether a counter is enabled. This check fails, because MDCR_EL2.HPMN is reset to 0, meaning that no counters are "enabled" for < EL2. That's in violation of the Arm specification, which states that > On a Warm reset, this field [MDCR_EL2.HPMN] resets to the value in > PMCR_EL0.N That's also what a comment in the code acknowledges, but the necessary adjustment seems to have been forgotten when support for more counters was added. This change fixes the issue by setting the reset value to PMCR.N, which is four. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/helper.c | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/target/arm/helper.c b/target/arm/helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -XXX,XX +XXX,XX @@ #endif #define ARM_CPU_FREQ 1000000000 /* FIXME: 1 GHz, should be configurable */ +#define PMCR_NUM_COUNTERS 4 /* QEMU IMPDEF choice */ #ifndef CONFIG_USER_ONLY @@ -XXX,XX +XXX,XX @@ static const ARMCPRegInfo el2_cp_reginfo[] = { .writefn = gt_hyp_ctl_write, .raw_writefn = raw_write }, #endif /* The only field of MDCR_EL2 that has a defined architectural reset value - * is MDCR_EL2.HPMN which should reset to the value of PMCR_EL0.N; but we - * don't implement any PMU event counters, so using zero as a reset - * value for MDCR_EL2 is okay + * is MDCR_EL2.HPMN which should reset to the value of PMCR_EL0.N. */ { .name = "MDCR_EL2", .state = ARM_CP_STATE_BOTH, .opc0 = 3, .opc1 = 4, .crn = 1, .crm = 1, .opc2 = 1, - .access = PL2_RW, .resetvalue = 0, + .access = PL2_RW, .resetvalue = PMCR_NUM_COUNTERS, .fieldoffset = offsetof(CPUARMState, cp15.mdcr_el2), }, { .name = "HPFAR", .state = ARM_CP_STATE_AA32, .cp = 15, .opc1 = 4, .crn = 6, .crm = 0, .opc2 = 4, @@ -XXX,XX +XXX,XX @@ static void define_pmu_regs(ARMCPU *cpu) * field as main ID register, and we implement four counters in * addition to the cycle count register. */ - unsigned int i, pmcrn = 4; + unsigned int i, pmcrn = PMCR_NUM_COUNTERS; ARMCPRegInfo pmcr = { .name = "PMCR", .cp = 15, .crn = 9, .crm = 12, .opc1 = 0, .opc2 = 0, .access = PL0_RW, -- 2.20.1 Hi; this is one last arm pullreq before the end of the year. Mostly minor cleanups, and also implementation of the FEAT_XS architectural feature. thanks -- PMM The following changes since commit 8032c78e556cd0baec111740a6c636863f9bd7c8: Merge tag 'firmware-20241216-pull-request' of https://gitlab.com/kraxel/qemu into staging (2024-12-16 14:20:33 -0500) are available in the Git repository at: https://git.linaro.org/people/pmaydell/qemu-arm.git tags/pull-target-arm-20241217 for you to fetch changes up to e91254250acb8570bd7b8a8f89d30e6d18291d02: tests/functional: update sbsa-ref firmware used in test (2024-12-17 15:21:06 +0000) ---------------------------------------------------------------- target-arm queue: * remove a line of redundant code * convert various TCG helper fns to use 'fpst' alias * Use float_status in helper_fcvtx_f64_to_f32 * Use float_status in helper_vfp_fcvt{ds,sd} * Implement FEAT_XS * hw/intc/arm_gicv3_its: Zero initialize local DTEntry etc structs * tests/functional: update sbsa-ref firmware used in test ---------------------------------------------------------------- Denis Rastyogin (1): target/arm: remove redundant code Manos Pitsidianakis (3): target/arm: Add decodetree entry for DSB nXS variant target/arm: Enable FEAT_XS for the max cpu tests/tcg/aarch64: add system test for FEAT_XS Marcin Juszkiewicz (1): tests/functional: update sbsa-ref firmware used in test Peter Maydell (4): target/arm: Implement fine-grained-trap handling for FEAT_XS target/arm: Add ARM_CP_ADD_TLBI_NXS type flag for NXS insns target/arm: Add ARM_CP_ADD_TLBI_NXS type flag to TLBI insns hw/intc/arm_gicv3_its: Zero initialize local DTEntry etc structs Richard Henderson (10): target/arm: Convert vfp_helper.c to fpst alias target/arm: Convert helper-a64.c to fpst alias target/arm: Convert vec_helper.c to fpst alias target/arm: Convert neon_helper.c to fpst alias target/arm: Convert sve_helper.c to fpst alias target/arm: Convert sme_helper.c to fpst alias target/arm: Convert vec_helper.c to use env alias target/arm: Convert neon_helper.c to use env alias target/arm: Use float_status in helper_fcvtx_f64_to_f32 target/arm: Use float_status in helper_vfp_fcvt{ds,sd} docs/system/arm/emulation.rst | 1 + target/arm/cpregs.h | 80 ++-- target/arm/cpu-features.h | 5 + target/arm/helper.h | 638 +++++++++++++++---------------- target/arm/tcg/helper-a64.h | 116 +++--- target/arm/tcg/helper-sme.h | 4 +- target/arm/tcg/helper-sve.h | 426 ++++++++++----------- target/arm/tcg/a64.decode | 3 + hw/intc/arm_gicv3_its.c | 44 +-- target/arm/helper.c | 30 +- target/arm/tcg/cpu64.c | 1 + target/arm/tcg/helper-a64.c | 101 ++--- target/arm/tcg/neon_helper.c | 27 +- target/arm/tcg/op_helper.c | 11 +- target/arm/tcg/sme_helper.c | 8 +- target/arm/tcg/sve_helper.c | 96 ++--- target/arm/tcg/tlb-insns.c | 202 ++++++---- target/arm/tcg/translate-a64.c | 26 +- target/arm/tcg/translate-vfp.c | 4 +- target/arm/tcg/vec_helper.c | 81 ++-- target/arm/vfp_helper.c | 130 +++---- tests/tcg/aarch64/system/feat-xs.c | 27 ++ tests/functional/test_aarch64_sbsaref.py | 20 +- 23 files changed, 1083 insertions(+), 998 deletions(-) create mode 100644 tests/tcg/aarch64/system/feat-xs.c From: Denis Rastyogin <gerben@altlinux.org> This call is redundant as it only retrieves a value that is not used further. Found by Linux Verification Center (linuxtesting.org) with SVACE. Signed-off-by: Denis Rastyogin <gerben@altlinux.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241212120618.518369-1-gerben@altlinux.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/vfp_helper.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/target/arm/vfp_helper.c b/target/arm/vfp_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vfp_helper.c +++ b/target/arm/vfp_helper.c @@ -XXX,XX +XXX,XX @@ float64 HELPER(rintd)(float64 x, void *fp_status) ret = float64_round_to_int(x, fp_status); - new_flags = get_float_exception_flags(fp_status); - /* Suppress any inexact exceptions the conversion produced */ if (!(old_flags & float_flag_inexact)) { new_flags = get_float_exception_flags(fp_status); -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Message-id: 20241206031224.78525-3-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/helper.h | 268 ++++++++++++++++++++-------------------- target/arm/vfp_helper.c | 120 ++++++++---------- 2 files changed, 186 insertions(+), 202 deletions(-) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_5(probe_access, TCG_CALL_NO_WG, void, env, tl, i32, i32, i32) DEF_HELPER_1(vfp_get_fpscr, i32, env) DEF_HELPER_2(vfp_set_fpscr, void, env, i32) -DEF_HELPER_3(vfp_addh, f16, f16, f16, ptr) -DEF_HELPER_3(vfp_adds, f32, f32, f32, ptr) -DEF_HELPER_3(vfp_addd, f64, f64, f64, ptr) -DEF_HELPER_3(vfp_subh, f16, f16, f16, ptr) -DEF_HELPER_3(vfp_subs, f32, f32, f32, ptr) -DEF_HELPER_3(vfp_subd, f64, f64, f64, ptr) -DEF_HELPER_3(vfp_mulh, f16, f16, f16, ptr) -DEF_HELPER_3(vfp_muls, f32, f32, f32, ptr) -DEF_HELPER_3(vfp_muld, f64, f64, f64, ptr) -DEF_HELPER_3(vfp_divh, f16, f16, f16, ptr) -DEF_HELPER_3(vfp_divs, f32, f32, f32, ptr) -DEF_HELPER_3(vfp_divd, f64, f64, f64, ptr) -DEF_HELPER_3(vfp_maxh, f16, f16, f16, ptr) -DEF_HELPER_3(vfp_maxs, f32, f32, f32, ptr) -DEF_HELPER_3(vfp_maxd, f64, f64, f64, ptr) -DEF_HELPER_3(vfp_minh, f16, f16, f16, ptr) -DEF_HELPER_3(vfp_mins, f32, f32, f32, ptr) -DEF_HELPER_3(vfp_mind, f64, f64, f64, ptr) -DEF_HELPER_3(vfp_maxnumh, f16, f16, f16, ptr) -DEF_HELPER_3(vfp_maxnums, f32, f32, f32, ptr) -DEF_HELPER_3(vfp_maxnumd, f64, f64, f64, ptr) -DEF_HELPER_3(vfp_minnumh, f16, f16, f16, ptr) -DEF_HELPER_3(vfp_minnums, f32, f32, f32, ptr) -DEF_HELPER_3(vfp_minnumd, f64, f64, f64, ptr) -DEF_HELPER_2(vfp_sqrth, f16, f16, ptr) -DEF_HELPER_2(vfp_sqrts, f32, f32, ptr) -DEF_HELPER_2(vfp_sqrtd, f64, f64, ptr) +DEF_HELPER_3(vfp_addh, f16, f16, f16, fpst) +DEF_HELPER_3(vfp_adds, f32, f32, f32, fpst) +DEF_HELPER_3(vfp_addd, f64, f64, f64, fpst) +DEF_HELPER_3(vfp_subh, f16, f16, f16, fpst) +DEF_HELPER_3(vfp_subs, f32, f32, f32, fpst) +DEF_HELPER_3(vfp_subd, f64, f64, f64, fpst) +DEF_HELPER_3(vfp_mulh, f16, f16, f16, fpst) +DEF_HELPER_3(vfp_muls, f32, f32, f32, fpst) +DEF_HELPER_3(vfp_muld, f64, f64, f64, fpst) +DEF_HELPER_3(vfp_divh, f16, f16, f16, fpst) +DEF_HELPER_3(vfp_divs, f32, f32, f32, fpst) +DEF_HELPER_3(vfp_divd, f64, f64, f64, fpst) +DEF_HELPER_3(vfp_maxh, f16, f16, f16, fpst) +DEF_HELPER_3(vfp_maxs, f32, f32, f32, fpst) +DEF_HELPER_3(vfp_maxd, f64, f64, f64, fpst) +DEF_HELPER_3(vfp_minh, f16, f16, f16, fpst) +DEF_HELPER_3(vfp_mins, f32, f32, f32, fpst) +DEF_HELPER_3(vfp_mind, f64, f64, f64, fpst) +DEF_HELPER_3(vfp_maxnumh, f16, f16, f16, fpst) +DEF_HELPER_3(vfp_maxnums, f32, f32, f32, fpst) +DEF_HELPER_3(vfp_maxnumd, f64, f64, f64, fpst) +DEF_HELPER_3(vfp_minnumh, f16, f16, f16, fpst) +DEF_HELPER_3(vfp_minnums, f32, f32, f32, fpst) +DEF_HELPER_3(vfp_minnumd, f64, f64, f64, fpst) +DEF_HELPER_2(vfp_sqrth, f16, f16, fpst) +DEF_HELPER_2(vfp_sqrts, f32, f32, fpst) +DEF_HELPER_2(vfp_sqrtd, f64, f64, fpst) DEF_HELPER_3(vfp_cmph, void, f16, f16, env) DEF_HELPER_3(vfp_cmps, void, f32, f32, env) DEF_HELPER_3(vfp_cmpd, void, f64, f64, env) @@ -XXX,XX +XXX,XX @@ DEF_HELPER_3(vfp_cmped, void, f64, f64, env) DEF_HELPER_2(vfp_fcvtds, f64, f32, env) DEF_HELPER_2(vfp_fcvtsd, f32, f64, env) -DEF_HELPER_FLAGS_2(bfcvt, TCG_CALL_NO_RWG, i32, f32, ptr) -DEF_HELPER_FLAGS_2(bfcvt_pair, TCG_CALL_NO_RWG, i32, i64, ptr) +DEF_HELPER_FLAGS_2(bfcvt, TCG_CALL_NO_RWG, i32, f32, fpst) +DEF_HELPER_FLAGS_2(bfcvt_pair, TCG_CALL_NO_RWG, i32, i64, fpst) -DEF_HELPER_2(vfp_uitoh, f16, i32, ptr) -DEF_HELPER_2(vfp_uitos, f32, i32, ptr) -DEF_HELPER_2(vfp_uitod, f64, i32, ptr) -DEF_HELPER_2(vfp_sitoh, f16, i32, ptr) -DEF_HELPER_2(vfp_sitos, f32, i32, ptr) -DEF_HELPER_2(vfp_sitod, f64, i32, ptr) +DEF_HELPER_2(vfp_uitoh, f16, i32, fpst) +DEF_HELPER_2(vfp_uitos, f32, i32, fpst) +DEF_HELPER_2(vfp_uitod, f64, i32, fpst) +DEF_HELPER_2(vfp_sitoh, f16, i32, fpst) +DEF_HELPER_2(vfp_sitos, f32, i32, fpst) +DEF_HELPER_2(vfp_sitod, f64, i32, fpst) -DEF_HELPER_2(vfp_touih, i32, f16, ptr) -DEF_HELPER_2(vfp_touis, i32, f32, ptr) -DEF_HELPER_2(vfp_touid, i32, f64, ptr) -DEF_HELPER_2(vfp_touizh, i32, f16, ptr) -DEF_HELPER_2(vfp_touizs, i32, f32, ptr) -DEF_HELPER_2(vfp_touizd, i32, f64, ptr) -DEF_HELPER_2(vfp_tosih, s32, f16, ptr) -DEF_HELPER_2(vfp_tosis, s32, f32, ptr) -DEF_HELPER_2(vfp_tosid, s32, f64, ptr) -DEF_HELPER_2(vfp_tosizh, s32, f16, ptr) -DEF_HELPER_2(vfp_tosizs, s32, f32, ptr) -DEF_HELPER_2(vfp_tosizd, s32, f64, ptr) +DEF_HELPER_2(vfp_touih, i32, f16, fpst) +DEF_HELPER_2(vfp_touis, i32, f32, fpst) +DEF_HELPER_2(vfp_touid, i32, f64, fpst) +DEF_HELPER_2(vfp_touizh, i32, f16, fpst) +DEF_HELPER_2(vfp_touizs, i32, f32, fpst) +DEF_HELPER_2(vfp_touizd, i32, f64, fpst) +DEF_HELPER_2(vfp_tosih, s32, f16, fpst) +DEF_HELPER_2(vfp_tosis, s32, f32, fpst) +DEF_HELPER_2(vfp_tosid, s32, f64, fpst) +DEF_HELPER_2(vfp_tosizh, s32, f16, fpst) +DEF_HELPER_2(vfp_tosizs, s32, f32, fpst) +DEF_HELPER_2(vfp_tosizd, s32, f64, fpst) -DEF_HELPER_3(vfp_toshh_round_to_zero, i32, f16, i32, ptr) -DEF_HELPER_3(vfp_toslh_round_to_zero, i32, f16, i32, ptr) -DEF_HELPER_3(vfp_touhh_round_to_zero, i32, f16, i32, ptr) -DEF_HELPER_3(vfp_toulh_round_to_zero, i32, f16, i32, ptr) -DEF_HELPER_3(vfp_toshs_round_to_zero, i32, f32, i32, ptr) -DEF_HELPER_3(vfp_tosls_round_to_zero, i32, f32, i32, ptr) -DEF_HELPER_3(vfp_touhs_round_to_zero, i32, f32, i32, ptr) -DEF_HELPER_3(vfp_touls_round_to_zero, i32, f32, i32, ptr) -DEF_HELPER_3(vfp_toshd_round_to_zero, i64, f64, i32, ptr) -DEF_HELPER_3(vfp_tosld_round_to_zero, i64, f64, i32, ptr) -DEF_HELPER_3(vfp_tosqd_round_to_zero, i64, f64, i32, ptr) -DEF_HELPER_3(vfp_touhd_round_to_zero, i64, f64, i32, ptr) -DEF_HELPER_3(vfp_tould_round_to_zero, i64, f64, i32, ptr) -DEF_HELPER_3(vfp_touqd_round_to_zero, i64, f64, i32, ptr) -DEF_HELPER_3(vfp_touhh, i32, f16, i32, ptr) -DEF_HELPER_3(vfp_toshh, i32, f16, i32, ptr) -DEF_HELPER_3(vfp_toulh, i32, f16, i32, ptr) -DEF_HELPER_3(vfp_toslh, i32, f16, i32, ptr) -DEF_HELPER_3(vfp_touqh, i64, f16, i32, ptr) -DEF_HELPER_3(vfp_tosqh, i64, f16, i32, ptr) -DEF_HELPER_3(vfp_toshs, i32, f32, i32, ptr) -DEF_HELPER_3(vfp_tosls, i32, f32, i32, ptr) -DEF_HELPER_3(vfp_tosqs, i64, f32, i32, ptr) -DEF_HELPER_3(vfp_touhs, i32, f32, i32, ptr) -DEF_HELPER_3(vfp_touls, i32, f32, i32, ptr) -DEF_HELPER_3(vfp_touqs, i64, f32, i32, ptr) -DEF_HELPER_3(vfp_toshd, i64, f64, i32, ptr) -DEF_HELPER_3(vfp_tosld, i64, f64, i32, ptr) -DEF_HELPER_3(vfp_tosqd, i64, f64, i32, ptr) -DEF_HELPER_3(vfp_touhd, i64, f64, i32, ptr) -DEF_HELPER_3(vfp_tould, i64, f64, i32, ptr) -DEF_HELPER_3(vfp_touqd, i64, f64, i32, ptr) -DEF_HELPER_3(vfp_shtos, f32, i32, i32, ptr) -DEF_HELPER_3(vfp_sltos, f32, i32, i32, ptr) -DEF_HELPER_3(vfp_sqtos, f32, i64, i32, ptr) -DEF_HELPER_3(vfp_uhtos, f32, i32, i32, ptr) -DEF_HELPER_3(vfp_ultos, f32, i32, i32, ptr) -DEF_HELPER_3(vfp_uqtos, f32, i64, i32, ptr) -DEF_HELPER_3(vfp_shtod, f64, i64, i32, ptr) -DEF_HELPER_3(vfp_sltod, f64, i64, i32, ptr) -DEF_HELPER_3(vfp_sqtod, f64, i64, i32, ptr) -DEF_HELPER_3(vfp_uhtod, f64, i64, i32, ptr) -DEF_HELPER_3(vfp_ultod, f64, i64, i32, ptr) -DEF_HELPER_3(vfp_uqtod, f64, i64, i32, ptr) -DEF_HELPER_3(vfp_shtoh, f16, i32, i32, ptr) -DEF_HELPER_3(vfp_uhtoh, f16, i32, i32, ptr) -DEF_HELPER_3(vfp_sltoh, f16, i32, i32, ptr) -DEF_HELPER_3(vfp_ultoh, f16, i32, i32, ptr) -DEF_HELPER_3(vfp_sqtoh, f16, i64, i32, ptr) -DEF_HELPER_3(vfp_uqtoh, f16, i64, i32, ptr) +DEF_HELPER_3(vfp_toshh_round_to_zero, i32, f16, i32, fpst) +DEF_HELPER_3(vfp_toslh_round_to_zero, i32, f16, i32, fpst) +DEF_HELPER_3(vfp_touhh_round_to_zero, i32, f16, i32, fpst) +DEF_HELPER_3(vfp_toulh_round_to_zero, i32, f16, i32, fpst) +DEF_HELPER_3(vfp_toshs_round_to_zero, i32, f32, i32, fpst) +DEF_HELPER_3(vfp_tosls_round_to_zero, i32, f32, i32, fpst) +DEF_HELPER_3(vfp_touhs_round_to_zero, i32, f32, i32, fpst) +DEF_HELPER_3(vfp_touls_round_to_zero, i32, f32, i32, fpst) +DEF_HELPER_3(vfp_toshd_round_to_zero, i64, f64, i32, fpst) +DEF_HELPER_3(vfp_tosld_round_to_zero, i64, f64, i32, fpst) +DEF_HELPER_3(vfp_tosqd_round_to_zero, i64, f64, i32, fpst) +DEF_HELPER_3(vfp_touhd_round_to_zero, i64, f64, i32, fpst) +DEF_HELPER_3(vfp_tould_round_to_zero, i64, f64, i32, fpst) +DEF_HELPER_3(vfp_touqd_round_to_zero, i64, f64, i32, fpst) +DEF_HELPER_3(vfp_touhh, i32, f16, i32, fpst) +DEF_HELPER_3(vfp_toshh, i32, f16, i32, fpst) +DEF_HELPER_3(vfp_toulh, i32, f16, i32, fpst) +DEF_HELPER_3(vfp_toslh, i32, f16, i32, fpst) +DEF_HELPER_3(vfp_touqh, i64, f16, i32, fpst) +DEF_HELPER_3(vfp_tosqh, i64, f16, i32, fpst) +DEF_HELPER_3(vfp_toshs, i32, f32, i32, fpst) +DEF_HELPER_3(vfp_tosls, i32, f32, i32, fpst) +DEF_HELPER_3(vfp_tosqs, i64, f32, i32, fpst) +DEF_HELPER_3(vfp_touhs, i32, f32, i32, fpst) +DEF_HELPER_3(vfp_touls, i32, f32, i32, fpst) +DEF_HELPER_3(vfp_touqs, i64, f32, i32, fpst) +DEF_HELPER_3(vfp_toshd, i64, f64, i32, fpst) +DEF_HELPER_3(vfp_tosld, i64, f64, i32, fpst) +DEF_HELPER_3(vfp_tosqd, i64, f64, i32, fpst) +DEF_HELPER_3(vfp_touhd, i64, f64, i32, fpst) +DEF_HELPER_3(vfp_tould, i64, f64, i32, fpst) +DEF_HELPER_3(vfp_touqd, i64, f64, i32, fpst) +DEF_HELPER_3(vfp_shtos, f32, i32, i32, fpst) +DEF_HELPER_3(vfp_sltos, f32, i32, i32, fpst) +DEF_HELPER_3(vfp_sqtos, f32, i64, i32, fpst) +DEF_HELPER_3(vfp_uhtos, f32, i32, i32, fpst) +DEF_HELPER_3(vfp_ultos, f32, i32, i32, fpst) +DEF_HELPER_3(vfp_uqtos, f32, i64, i32, fpst) +DEF_HELPER_3(vfp_shtod, f64, i64, i32, fpst) +DEF_HELPER_3(vfp_sltod, f64, i64, i32, fpst) +DEF_HELPER_3(vfp_sqtod, f64, i64, i32, fpst) +DEF_HELPER_3(vfp_uhtod, f64, i64, i32, fpst) +DEF_HELPER_3(vfp_ultod, f64, i64, i32, fpst) +DEF_HELPER_3(vfp_uqtod, f64, i64, i32, fpst) +DEF_HELPER_3(vfp_shtoh, f16, i32, i32, fpst) +DEF_HELPER_3(vfp_uhtoh, f16, i32, i32, fpst) +DEF_HELPER_3(vfp_sltoh, f16, i32, i32, fpst) +DEF_HELPER_3(vfp_ultoh, f16, i32, i32, fpst) +DEF_HELPER_3(vfp_sqtoh, f16, i64, i32, fpst) +DEF_HELPER_3(vfp_uqtoh, f16, i64, i32, fpst) -DEF_HELPER_3(vfp_shtos_round_to_nearest, f32, i32, i32, ptr) -DEF_HELPER_3(vfp_sltos_round_to_nearest, f32, i32, i32, ptr) -DEF_HELPER_3(vfp_uhtos_round_to_nearest, f32, i32, i32, ptr) -DEF_HELPER_3(vfp_ultos_round_to_nearest, f32, i32, i32, ptr) -DEF_HELPER_3(vfp_shtod_round_to_nearest, f64, i64, i32, ptr) -DEF_HELPER_3(vfp_sltod_round_to_nearest, f64, i64, i32, ptr) -DEF_HELPER_3(vfp_uhtod_round_to_nearest, f64, i64, i32, ptr) -DEF_HELPER_3(vfp_ultod_round_to_nearest, f64, i64, i32, ptr) -DEF_HELPER_3(vfp_shtoh_round_to_nearest, f16, i32, i32, ptr) -DEF_HELPER_3(vfp_uhtoh_round_to_nearest, f16, i32, i32, ptr) -DEF_HELPER_3(vfp_sltoh_round_to_nearest, f16, i32, i32, ptr) -DEF_HELPER_3(vfp_ultoh_round_to_nearest, f16, i32, i32, ptr) +DEF_HELPER_3(vfp_shtos_round_to_nearest, f32, i32, i32, fpst) +DEF_HELPER_3(vfp_sltos_round_to_nearest, f32, i32, i32, fpst) +DEF_HELPER_3(vfp_uhtos_round_to_nearest, f32, i32, i32, fpst) +DEF_HELPER_3(vfp_ultos_round_to_nearest, f32, i32, i32, fpst) +DEF_HELPER_3(vfp_shtod_round_to_nearest, f64, i64, i32, fpst) +DEF_HELPER_3(vfp_sltod_round_to_nearest, f64, i64, i32, fpst) +DEF_HELPER_3(vfp_uhtod_round_to_nearest, f64, i64, i32, fpst) +DEF_HELPER_3(vfp_ultod_round_to_nearest, f64, i64, i32, fpst) +DEF_HELPER_3(vfp_shtoh_round_to_nearest, f16, i32, i32, fpst) +DEF_HELPER_3(vfp_uhtoh_round_to_nearest, f16, i32, i32, fpst) +DEF_HELPER_3(vfp_sltoh_round_to_nearest, f16, i32, i32, fpst) +DEF_HELPER_3(vfp_ultoh_round_to_nearest, f16, i32, i32, fpst) -DEF_HELPER_FLAGS_2(set_rmode, TCG_CALL_NO_RWG, i32, i32, ptr) +DEF_HELPER_FLAGS_2(set_rmode, TCG_CALL_NO_RWG, i32, i32, fpst) -DEF_HELPER_FLAGS_3(vfp_fcvt_f16_to_f32, TCG_CALL_NO_RWG, f32, f16, ptr, i32) -DEF_HELPER_FLAGS_3(vfp_fcvt_f32_to_f16, TCG_CALL_NO_RWG, f16, f32, ptr, i32) -DEF_HELPER_FLAGS_3(vfp_fcvt_f16_to_f64, TCG_CALL_NO_RWG, f64, f16, ptr, i32) -DEF_HELPER_FLAGS_3(vfp_fcvt_f64_to_f16, TCG_CALL_NO_RWG, f16, f64, ptr, i32) +DEF_HELPER_FLAGS_3(vfp_fcvt_f16_to_f32, TCG_CALL_NO_RWG, f32, f16, fpst, i32) +DEF_HELPER_FLAGS_3(vfp_fcvt_f32_to_f16, TCG_CALL_NO_RWG, f16, f32, fpst, i32) +DEF_HELPER_FLAGS_3(vfp_fcvt_f16_to_f64, TCG_CALL_NO_RWG, f64, f16, fpst, i32) +DEF_HELPER_FLAGS_3(vfp_fcvt_f64_to_f16, TCG_CALL_NO_RWG, f16, f64, fpst, i32) -DEF_HELPER_4(vfp_muladdd, f64, f64, f64, f64, ptr) -DEF_HELPER_4(vfp_muladds, f32, f32, f32, f32, ptr) -DEF_HELPER_4(vfp_muladdh, f16, f16, f16, f16, ptr) +DEF_HELPER_4(vfp_muladdd, f64, f64, f64, f64, fpst) +DEF_HELPER_4(vfp_muladds, f32, f32, f32, f32, fpst) +DEF_HELPER_4(vfp_muladdh, f16, f16, f16, f16, fpst) -DEF_HELPER_FLAGS_2(recpe_f16, TCG_CALL_NO_RWG, f16, f16, ptr) -DEF_HELPER_FLAGS_2(recpe_f32, TCG_CALL_NO_RWG, f32, f32, ptr) -DEF_HELPER_FLAGS_2(recpe_f64, TCG_CALL_NO_RWG, f64, f64, ptr) -DEF_HELPER_FLAGS_2(rsqrte_f16, TCG_CALL_NO_RWG, f16, f16, ptr) -DEF_HELPER_FLAGS_2(rsqrte_f32, TCG_CALL_NO_RWG, f32, f32, ptr) -DEF_HELPER_FLAGS_2(rsqrte_f64, TCG_CALL_NO_RWG, f64, f64, ptr) +DEF_HELPER_FLAGS_2(recpe_f16, TCG_CALL_NO_RWG, f16, f16, fpst) +DEF_HELPER_FLAGS_2(recpe_f32, TCG_CALL_NO_RWG, f32, f32, fpst) +DEF_HELPER_FLAGS_2(recpe_f64, TCG_CALL_NO_RWG, f64, f64, fpst) +DEF_HELPER_FLAGS_2(rsqrte_f16, TCG_CALL_NO_RWG, f16, f16, fpst) +DEF_HELPER_FLAGS_2(rsqrte_f32, TCG_CALL_NO_RWG, f32, f32, fpst) +DEF_HELPER_FLAGS_2(rsqrte_f64, TCG_CALL_NO_RWG, f64, f64, fpst) DEF_HELPER_FLAGS_1(recpe_u32, TCG_CALL_NO_RWG, i32, i32) DEF_HELPER_FLAGS_1(rsqrte_u32, TCG_CALL_NO_RWG, i32, i32) DEF_HELPER_FLAGS_4(neon_tbl, TCG_CALL_NO_RWG, i64, env, i32, i64, i64) @@ -XXX,XX +XXX,XX @@ DEF_HELPER_3(shr_cc, i32, env, i32, i32) DEF_HELPER_3(sar_cc, i32, env, i32, i32) DEF_HELPER_3(ror_cc, i32, env, i32, i32) -DEF_HELPER_FLAGS_2(rinth_exact, TCG_CALL_NO_RWG, f16, f16, ptr) -DEF_HELPER_FLAGS_2(rints_exact, TCG_CALL_NO_RWG, f32, f32, ptr) -DEF_HELPER_FLAGS_2(rintd_exact, TCG_CALL_NO_RWG, f64, f64, ptr) -DEF_HELPER_FLAGS_2(rinth, TCG_CALL_NO_RWG, f16, f16, ptr) -DEF_HELPER_FLAGS_2(rints, TCG_CALL_NO_RWG, f32, f32, ptr) -DEF_HELPER_FLAGS_2(rintd, TCG_CALL_NO_RWG, f64, f64, ptr) +DEF_HELPER_FLAGS_2(rinth_exact, TCG_CALL_NO_RWG, f16, f16, fpst) +DEF_HELPER_FLAGS_2(rints_exact, TCG_CALL_NO_RWG, f32, f32, fpst) +DEF_HELPER_FLAGS_2(rintd_exact, TCG_CALL_NO_RWG, f64, f64, fpst) +DEF_HELPER_FLAGS_2(rinth, TCG_CALL_NO_RWG, f16, f16, fpst) +DEF_HELPER_FLAGS_2(rints, TCG_CALL_NO_RWG, f32, f32, fpst) +DEF_HELPER_FLAGS_2(rintd, TCG_CALL_NO_RWG, f64, f64, fpst) DEF_HELPER_FLAGS_2(vjcvt, TCG_CALL_NO_RWG, i32, f64, env) -DEF_HELPER_FLAGS_2(fjcvtzs, TCG_CALL_NO_RWG, i64, f64, ptr) +DEF_HELPER_FLAGS_2(fjcvtzs, TCG_CALL_NO_RWG, i64, f64, fpst) DEF_HELPER_FLAGS_3(check_hcr_el2_trap, TCG_CALL_NO_WG, void, env, i32, i32) @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_5(gvec_fmlal_idx_a32, TCG_CALL_NO_RWG, DEF_HELPER_FLAGS_5(gvec_fmlal_idx_a64, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_2(frint32_s, TCG_CALL_NO_RWG, f32, f32, ptr) -DEF_HELPER_FLAGS_2(frint64_s, TCG_CALL_NO_RWG, f32, f32, ptr) -DEF_HELPER_FLAGS_2(frint32_d, TCG_CALL_NO_RWG, f64, f64, ptr) -DEF_HELPER_FLAGS_2(frint64_d, TCG_CALL_NO_RWG, f64, f64, ptr) +DEF_HELPER_FLAGS_2(frint32_s, TCG_CALL_NO_RWG, f32, f32, fpst) +DEF_HELPER_FLAGS_2(frint64_s, TCG_CALL_NO_RWG, f32, f32, fpst) +DEF_HELPER_FLAGS_2(frint32_d, TCG_CALL_NO_RWG, f64, f64, fpst) +DEF_HELPER_FLAGS_2(frint64_d, TCG_CALL_NO_RWG, f64, f64, fpst) DEF_HELPER_FLAGS_3(gvec_ceq0_b, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_3(gvec_ceq0_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) diff --git a/target/arm/vfp_helper.c b/target/arm/vfp_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vfp_helper.c +++ b/target/arm/vfp_helper.c @@ -XXX,XX +XXX,XX @@ void vfp_set_fpscr(CPUARMState *env, uint32_t val) #define VFP_HELPER(name, p) HELPER(glue(glue(vfp_,name),p)) #define VFP_BINOP(name) \ -dh_ctype_f16 VFP_HELPER(name, h)(dh_ctype_f16 a, dh_ctype_f16 b, void *fpstp) \ +dh_ctype_f16 VFP_HELPER(name, h)(dh_ctype_f16 a, dh_ctype_f16 b, float_status *fpst) \ { \ - float_status *fpst = fpstp; \ return float16_ ## name(a, b, fpst); \ } \ -float32 VFP_HELPER(name, s)(float32 a, float32 b, void *fpstp) \ +float32 VFP_HELPER(name, s)(float32 a, float32 b, float_status *fpst) \ { \ - float_status *fpst = fpstp; \ return float32_ ## name(a, b, fpst); \ } \ -float64 VFP_HELPER(name, d)(float64 a, float64 b, void *fpstp) \ +float64 VFP_HELPER(name, d)(float64 a, float64 b, float_status *fpst) \ { \ - float_status *fpst = fpstp; \ return float64_ ## name(a, b, fpst); \ } VFP_BINOP(add) @@ -XXX,XX +XXX,XX @@ VFP_BINOP(minnum) VFP_BINOP(maxnum) #undef VFP_BINOP -dh_ctype_f16 VFP_HELPER(sqrt, h)(dh_ctype_f16 a, void *fpstp) +dh_ctype_f16 VFP_HELPER(sqrt, h)(dh_ctype_f16 a, float_status *fpst) { - return float16_sqrt(a, fpstp); + return float16_sqrt(a, fpst); } -float32 VFP_HELPER(sqrt, s)(float32 a, void *fpstp) +float32 VFP_HELPER(sqrt, s)(float32 a, float_status *fpst) { - return float32_sqrt(a, fpstp); + return float32_sqrt(a, fpst); } -float64 VFP_HELPER(sqrt, d)(float64 a, void *fpstp) +float64 VFP_HELPER(sqrt, d)(float64 a, float_status *fpst) { - return float64_sqrt(a, fpstp); + return float64_sqrt(a, fpst); } static void softfloat_to_vfp_compare(CPUARMState *env, FloatRelation cmp) @@ -XXX,XX +XXX,XX @@ DO_VFP_cmp(d, float64, float64, fp_status) /* Integer to float and float to integer conversions */ #define CONV_ITOF(name, ftype, fsz, sign) \ -ftype HELPER(name)(uint32_t x, void *fpstp) \ +ftype HELPER(name)(uint32_t x, float_status *fpst) \ { \ - float_status *fpst = fpstp; \ return sign##int32_to_##float##fsz((sign##int32_t)x, fpst); \ } #define CONV_FTOI(name, ftype, fsz, sign, round) \ -sign##int32_t HELPER(name)(ftype x, void *fpstp) \ +sign##int32_t HELPER(name)(ftype x, float_status *fpst) \ { \ - float_status *fpst = fpstp; \ if (float##fsz##_is_any_nan(x)) { \ float_raise(float_flag_invalid, fpst); \ return 0; \ @@ -XXX,XX +XXX,XX @@ float32 VFP_HELPER(fcvts, d)(float64 x, CPUARMState *env) return float64_to_float32(x, &env->vfp.fp_status); } -uint32_t HELPER(bfcvt)(float32 x, void *status) +uint32_t HELPER(bfcvt)(float32 x, float_status *status) { return float32_to_bfloat16(x, status); } -uint32_t HELPER(bfcvt_pair)(uint64_t pair, void *status) +uint32_t HELPER(bfcvt_pair)(uint64_t pair, float_status *status) { bfloat16 lo = float32_to_bfloat16(extract64(pair, 0, 32), status); bfloat16 hi = float32_to_bfloat16(extract64(pair, 32, 32), status); @@ -XXX,XX +XXX,XX @@ uint32_t HELPER(bfcvt_pair)(uint64_t pair, void *status) */ #define VFP_CONV_FIX_FLOAT(name, p, fsz, ftype, isz, itype) \ ftype HELPER(vfp_##name##to##p)(uint##isz##_t x, uint32_t shift, \ - void *fpstp) \ -{ return itype##_to_##float##fsz##_scalbn(x, -shift, fpstp); } + float_status *fpst) \ +{ return itype##_to_##float##fsz##_scalbn(x, -shift, fpst); } #define VFP_CONV_FIX_FLOAT_ROUND(name, p, fsz, ftype, isz, itype) \ ftype HELPER(vfp_##name##to##p##_round_to_nearest)(uint##isz##_t x, \ uint32_t shift, \ - void *fpstp) \ + float_status *fpst) \ { \ ftype ret; \ - float_status *fpst = fpstp; \ FloatRoundMode oldmode = fpst->float_rounding_mode; \ fpst->float_rounding_mode = float_round_nearest_even; \ - ret = itype##_to_##float##fsz##_scalbn(x, -shift, fpstp); \ + ret = itype##_to_##float##fsz##_scalbn(x, -shift, fpst); \ fpst->float_rounding_mode = oldmode; \ return ret; \ } #define VFP_CONV_FLOAT_FIX_ROUND(name, p, fsz, ftype, isz, itype, ROUND, suff) \ uint##isz##_t HELPER(vfp_to##name##p##suff)(ftype x, uint32_t shift, \ - void *fpst) \ + float_status *fpst) \ { \ if (unlikely(float##fsz##_is_any_nan(x))) { \ float_raise(float_flag_invalid, fpst); \ @@ -XXX,XX +XXX,XX @@ VFP_CONV_FLOAT_FIX_ROUND(uq, d, 64, float64, 64, uint64, /* Set the current fp rounding mode and return the old one. * The argument is a softfloat float_round_ value. */ -uint32_t HELPER(set_rmode)(uint32_t rmode, void *fpstp) +uint32_t HELPER(set_rmode)(uint32_t rmode, float_status *fp_status) { - float_status *fp_status = fpstp; - uint32_t prev_rmode = get_float_rounding_mode(fp_status); set_float_rounding_mode(rmode, fp_status); @@ -XXX,XX +XXX,XX @@ uint32_t HELPER(set_rmode)(uint32_t rmode, void *fpstp) } /* Half precision conversions. */ -float32 HELPER(vfp_fcvt_f16_to_f32)(uint32_t a, void *fpstp, uint32_t ahp_mode) +float32 HELPER(vfp_fcvt_f16_to_f32)(uint32_t a, float_status *fpst, + uint32_t ahp_mode) { /* Squash FZ16 to 0 for the duration of conversion. In this case, * it would affect flushing input denormals. */ - float_status *fpst = fpstp; bool save = get_flush_inputs_to_zero(fpst); set_flush_inputs_to_zero(false, fpst); float32 r = float16_to_float32(a, !ahp_mode, fpst); @@ -XXX,XX +XXX,XX @@ float32 HELPER(vfp_fcvt_f16_to_f32)(uint32_t a, void *fpstp, uint32_t ahp_mode) return r; } -uint32_t HELPER(vfp_fcvt_f32_to_f16)(float32 a, void *fpstp, uint32_t ahp_mode) +uint32_t HELPER(vfp_fcvt_f32_to_f16)(float32 a, float_status *fpst, + uint32_t ahp_mode) { /* Squash FZ16 to 0 for the duration of conversion. In this case, * it would affect flushing output denormals. */ - float_status *fpst = fpstp; bool save = get_flush_to_zero(fpst); set_flush_to_zero(false, fpst); float16 r = float32_to_float16(a, !ahp_mode, fpst); @@ -XXX,XX +XXX,XX @@ uint32_t HELPER(vfp_fcvt_f32_to_f16)(float32 a, void *fpstp, uint32_t ahp_mode) return r; } -float64 HELPER(vfp_fcvt_f16_to_f64)(uint32_t a, void *fpstp, uint32_t ahp_mode) +float64 HELPER(vfp_fcvt_f16_to_f64)(uint32_t a, float_status *fpst, + uint32_t ahp_mode) { /* Squash FZ16 to 0 for the duration of conversion. In this case, * it would affect flushing input denormals. */ - float_status *fpst = fpstp; bool save = get_flush_inputs_to_zero(fpst); set_flush_inputs_to_zero(false, fpst); float64 r = float16_to_float64(a, !ahp_mode, fpst); @@ -XXX,XX +XXX,XX @@ float64 HELPER(vfp_fcvt_f16_to_f64)(uint32_t a, void *fpstp, uint32_t ahp_mode) return r; } -uint32_t HELPER(vfp_fcvt_f64_to_f16)(float64 a, void *fpstp, uint32_t ahp_mode) +uint32_t HELPER(vfp_fcvt_f64_to_f16)(float64 a, float_status *fpst, + uint32_t ahp_mode) { /* Squash FZ16 to 0 for the duration of conversion. In this case, * it would affect flushing output denormals. */ - float_status *fpst = fpstp; bool save = get_flush_to_zero(fpst); set_flush_to_zero(false, fpst); float16 r = float64_to_float16(a, !ahp_mode, fpst); @@ -XXX,XX +XXX,XX @@ static bool round_to_inf(float_status *fpst, bool sign_bit) } } -uint32_t HELPER(recpe_f16)(uint32_t input, void *fpstp) +uint32_t HELPER(recpe_f16)(uint32_t input, float_status *fpst) { - float_status *fpst = fpstp; float16 f16 = float16_squash_input_denormal(input, fpst); uint32_t f16_val = float16_val(f16); uint32_t f16_sign = float16_is_neg(f16); @@ -XXX,XX +XXX,XX @@ uint32_t HELPER(recpe_f16)(uint32_t input, void *fpstp) return make_float16(f16_val); } -float32 HELPER(recpe_f32)(float32 input, void *fpstp) +float32 HELPER(recpe_f32)(float32 input, float_status *fpst) { - float_status *fpst = fpstp; float32 f32 = float32_squash_input_denormal(input, fpst); uint32_t f32_val = float32_val(f32); bool f32_sign = float32_is_neg(f32); @@ -XXX,XX +XXX,XX @@ float32 HELPER(recpe_f32)(float32 input, void *fpstp) return make_float32(f32_val); } -float64 HELPER(recpe_f64)(float64 input, void *fpstp) +float64 HELPER(recpe_f64)(float64 input, float_status *fpst) { - float_status *fpst = fpstp; float64 f64 = float64_squash_input_denormal(input, fpst); uint64_t f64_val = float64_val(f64); bool f64_sign = float64_is_neg(f64); @@ -XXX,XX +XXX,XX @@ static uint64_t recip_sqrt_estimate(int *exp , int exp_off, uint64_t frac) return extract64(estimate, 0, 8) << 44; } -uint32_t HELPER(rsqrte_f16)(uint32_t input, void *fpstp) +uint32_t HELPER(rsqrte_f16)(uint32_t input, float_status *s) { - float_status *s = fpstp; float16 f16 = float16_squash_input_denormal(input, s); uint16_t val = float16_val(f16); bool f16_sign = float16_is_neg(f16); @@ -XXX,XX +XXX,XX @@ uint32_t HELPER(rsqrte_f16)(uint32_t input, void *fpstp) if (float16_is_signaling_nan(f16, s)) { float_raise(float_flag_invalid, s); if (!s->default_nan_mode) { - nan = float16_silence_nan(f16, fpstp); + nan = float16_silence_nan(f16, s); } } if (s->default_nan_mode) { @@ -XXX,XX +XXX,XX @@ uint32_t HELPER(rsqrte_f16)(uint32_t input, void *fpstp) return make_float16(val); } -float32 HELPER(rsqrte_f32)(float32 input, void *fpstp) +float32 HELPER(rsqrte_f32)(float32 input, float_status *s) { - float_status *s = fpstp; float32 f32 = float32_squash_input_denormal(input, s); uint32_t val = float32_val(f32); uint32_t f32_sign = float32_is_neg(f32); @@ -XXX,XX +XXX,XX @@ float32 HELPER(rsqrte_f32)(float32 input, void *fpstp) if (float32_is_signaling_nan(f32, s)) { float_raise(float_flag_invalid, s); if (!s->default_nan_mode) { - nan = float32_silence_nan(f32, fpstp); + nan = float32_silence_nan(f32, s); } } if (s->default_nan_mode) { @@ -XXX,XX +XXX,XX @@ float32 HELPER(rsqrte_f32)(float32 input, void *fpstp) return make_float32(val); } -float64 HELPER(rsqrte_f64)(float64 input, void *fpstp) +float64 HELPER(rsqrte_f64)(float64 input, float_status *s) { - float_status *s = fpstp; float64 f64 = float64_squash_input_denormal(input, s); uint64_t val = float64_val(f64); bool f64_sign = float64_is_neg(f64); @@ -XXX,XX +XXX,XX @@ float64 HELPER(rsqrte_f64)(float64 input, void *fpstp) if (float64_is_signaling_nan(f64, s)) { float_raise(float_flag_invalid, s); if (!s->default_nan_mode) { - nan = float64_silence_nan(f64, fpstp); + nan = float64_silence_nan(f64, s); } } if (s->default_nan_mode) { @@ -XXX,XX +XXX,XX @@ uint32_t HELPER(rsqrte_u32)(uint32_t a) /* VFPv4 fused multiply-accumulate */ dh_ctype_f16 VFP_HELPER(muladd, h)(dh_ctype_f16 a, dh_ctype_f16 b, - dh_ctype_f16 c, void *fpstp) + dh_ctype_f16 c, float_status *fpst) { - float_status *fpst = fpstp; return float16_muladd(a, b, c, 0, fpst); } -float32 VFP_HELPER(muladd, s)(float32 a, float32 b, float32 c, void *fpstp) +float32 VFP_HELPER(muladd, s)(float32 a, float32 b, float32 c, + float_status *fpst) { - float_status *fpst = fpstp; return float32_muladd(a, b, c, 0, fpst); } -float64 VFP_HELPER(muladd, d)(float64 a, float64 b, float64 c, void *fpstp) +float64 VFP_HELPER(muladd, d)(float64 a, float64 b, float64 c, + float_status *fpst) { - float_status *fpst = fpstp; return float64_muladd(a, b, c, 0, fpst); } /* ARMv8 round to integral */ -dh_ctype_f16 HELPER(rinth_exact)(dh_ctype_f16 x, void *fp_status) +dh_ctype_f16 HELPER(rinth_exact)(dh_ctype_f16 x, float_status *fp_status) { return float16_round_to_int(x, fp_status); } -float32 HELPER(rints_exact)(float32 x, void *fp_status) +float32 HELPER(rints_exact)(float32 x, float_status *fp_status) { return float32_round_to_int(x, fp_status); } -float64 HELPER(rintd_exact)(float64 x, void *fp_status) +float64 HELPER(rintd_exact)(float64 x, float_status *fp_status) { return float64_round_to_int(x, fp_status); } -dh_ctype_f16 HELPER(rinth)(dh_ctype_f16 x, void *fp_status) +dh_ctype_f16 HELPER(rinth)(dh_ctype_f16 x, float_status *fp_status) { int old_flags = get_float_exception_flags(fp_status), new_flags; float16 ret; @@ -XXX,XX +XXX,XX @@ dh_ctype_f16 HELPER(rinth)(dh_ctype_f16 x, void *fp_status) return ret; } -float32 HELPER(rints)(float32 x, void *fp_status) +float32 HELPER(rints)(float32 x, float_status *fp_status) { int old_flags = get_float_exception_flags(fp_status), new_flags; float32 ret; @@ -XXX,XX +XXX,XX @@ float32 HELPER(rints)(float32 x, void *fp_status) return ret; } -float64 HELPER(rintd)(float64 x, void *fp_status) +float64 HELPER(rintd)(float64 x, float_status *fp_status) { int old_flags = get_float_exception_flags(fp_status), new_flags; float64 ret; @@ -XXX,XX +XXX,XX @@ const FloatRoundMode arm_rmode_to_sf_map[] = { * Implement float64 to int32_t conversion without saturation; * the result is supplied modulo 2^32. */ -uint64_t HELPER(fjcvtzs)(float64 value, void *vstatus) +uint64_t HELPER(fjcvtzs)(float64 value, float_status *status) { - float_status *status = vstatus; uint32_t frac, e_old, e_new; bool inexact; @@ -XXX,XX +XXX,XX @@ static float32 frint_s(float32 f, float_status *fpst, int intsize) return (0x100u + 126u + intsize) << 23; } -float32 HELPER(frint32_s)(float32 f, void *fpst) +float32 HELPER(frint32_s)(float32 f, float_status *fpst) { return frint_s(f, fpst, 32); } -float32 HELPER(frint64_s)(float32 f, void *fpst) +float32 HELPER(frint64_s)(float32 f, float_status *fpst) { return frint_s(f, fpst, 64); } @@ -XXX,XX +XXX,XX @@ static float64 frint_d(float64 f, float_status *fpst, int intsize) return (uint64_t)(0x800 + 1022 + intsize) << 52; } -float64 HELPER(frint32_d)(float64 f, void *fpst) +float64 HELPER(frint32_d)(float64 f, float_status *fpst) { return frint_d(f, fpst, 32); } -float64 HELPER(frint64_d)(float64 f, void *fpst) +float64 HELPER(frint64_d)(float64 f, float_status *fpst) { return frint_d(f, fpst, 64); } -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Message-id: 20241206031224.78525-4-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/helper-a64.h | 94 +++++++++++++++++------------------ target/arm/tcg/helper-a64.c | 98 +++++++++++++------------------------ 2 files changed, 80 insertions(+), 112 deletions(-) diff --git a/target/arm/tcg/helper-a64.h b/target/arm/tcg/helper-a64.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/helper-a64.h +++ b/target/arm/tcg/helper-a64.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_2(msr_i_spsel, void, env, i32) DEF_HELPER_2(msr_i_daifset, void, env, i32) DEF_HELPER_2(msr_i_daifclear, void, env, i32) DEF_HELPER_1(msr_set_allint_el1, void, env) -DEF_HELPER_3(vfp_cmph_a64, i64, f16, f16, ptr) -DEF_HELPER_3(vfp_cmpeh_a64, i64, f16, f16, ptr) -DEF_HELPER_3(vfp_cmps_a64, i64, f32, f32, ptr) -DEF_HELPER_3(vfp_cmpes_a64, i64, f32, f32, ptr) -DEF_HELPER_3(vfp_cmpd_a64, i64, f64, f64, ptr) -DEF_HELPER_3(vfp_cmped_a64, i64, f64, f64, ptr) +DEF_HELPER_3(vfp_cmph_a64, i64, f16, f16, fpst) +DEF_HELPER_3(vfp_cmpeh_a64, i64, f16, f16, fpst) +DEF_HELPER_3(vfp_cmps_a64, i64, f32, f32, fpst) +DEF_HELPER_3(vfp_cmpes_a64, i64, f32, f32, fpst) +DEF_HELPER_3(vfp_cmpd_a64, i64, f64, f64, fpst) +DEF_HELPER_3(vfp_cmped_a64, i64, f64, f64, fpst) DEF_HELPER_FLAGS_4(simd_tblx, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_3(vfp_mulxs, TCG_CALL_NO_RWG, f32, f32, f32, ptr) -DEF_HELPER_FLAGS_3(vfp_mulxd, TCG_CALL_NO_RWG, f64, f64, f64, ptr) -DEF_HELPER_FLAGS_3(neon_ceq_f64, TCG_CALL_NO_RWG, i64, i64, i64, ptr) -DEF_HELPER_FLAGS_3(neon_cge_f64, TCG_CALL_NO_RWG, i64, i64, i64, ptr) -DEF_HELPER_FLAGS_3(neon_cgt_f64, TCG_CALL_NO_RWG, i64, i64, i64, ptr) -DEF_HELPER_FLAGS_3(recpsf_f16, TCG_CALL_NO_RWG, f16, f16, f16, ptr) -DEF_HELPER_FLAGS_3(recpsf_f32, TCG_CALL_NO_RWG, f32, f32, f32, ptr) -DEF_HELPER_FLAGS_3(recpsf_f64, TCG_CALL_NO_RWG, f64, f64, f64, ptr) -DEF_HELPER_FLAGS_3(rsqrtsf_f16, TCG_CALL_NO_RWG, f16, f16, f16, ptr) -DEF_HELPER_FLAGS_3(rsqrtsf_f32, TCG_CALL_NO_RWG, f32, f32, f32, ptr) -DEF_HELPER_FLAGS_3(rsqrtsf_f64, TCG_CALL_NO_RWG, f64, f64, f64, ptr) -DEF_HELPER_FLAGS_2(frecpx_f64, TCG_CALL_NO_RWG, f64, f64, ptr) -DEF_HELPER_FLAGS_2(frecpx_f32, TCG_CALL_NO_RWG, f32, f32, ptr) -DEF_HELPER_FLAGS_2(frecpx_f16, TCG_CALL_NO_RWG, f16, f16, ptr) +DEF_HELPER_FLAGS_3(vfp_mulxs, TCG_CALL_NO_RWG, f32, f32, f32, fpst) +DEF_HELPER_FLAGS_3(vfp_mulxd, TCG_CALL_NO_RWG, f64, f64, f64, fpst) +DEF_HELPER_FLAGS_3(neon_ceq_f64, TCG_CALL_NO_RWG, i64, i64, i64, fpst) +DEF_HELPER_FLAGS_3(neon_cge_f64, TCG_CALL_NO_RWG, i64, i64, i64, fpst) +DEF_HELPER_FLAGS_3(neon_cgt_f64, TCG_CALL_NO_RWG, i64, i64, i64, fpst) +DEF_HELPER_FLAGS_3(recpsf_f16, TCG_CALL_NO_RWG, f16, f16, f16, fpst) +DEF_HELPER_FLAGS_3(recpsf_f32, TCG_CALL_NO_RWG, f32, f32, f32, fpst) +DEF_HELPER_FLAGS_3(recpsf_f64, TCG_CALL_NO_RWG, f64, f64, f64, fpst) +DEF_HELPER_FLAGS_3(rsqrtsf_f16, TCG_CALL_NO_RWG, f16, f16, f16, fpst) +DEF_HELPER_FLAGS_3(rsqrtsf_f32, TCG_CALL_NO_RWG, f32, f32, f32, fpst) +DEF_HELPER_FLAGS_3(rsqrtsf_f64, TCG_CALL_NO_RWG, f64, f64, f64, fpst) +DEF_HELPER_FLAGS_2(frecpx_f64, TCG_CALL_NO_RWG, f64, f64, fpst) +DEF_HELPER_FLAGS_2(frecpx_f32, TCG_CALL_NO_RWG, f32, f32, fpst) +DEF_HELPER_FLAGS_2(frecpx_f16, TCG_CALL_NO_RWG, f16, f16, fpst) DEF_HELPER_FLAGS_2(fcvtx_f64_to_f32, TCG_CALL_NO_RWG, f32, f64, env) DEF_HELPER_FLAGS_3(crc32_64, TCG_CALL_NO_RWG_SE, i64, i64, i64, i32) DEF_HELPER_FLAGS_3(crc32c_64, TCG_CALL_NO_RWG_SE, i64, i64, i64, i32) -DEF_HELPER_FLAGS_3(advsimd_maxh, TCG_CALL_NO_RWG, f16, f16, f16, ptr) -DEF_HELPER_FLAGS_3(advsimd_minh, TCG_CALL_NO_RWG, f16, f16, f16, ptr) -DEF_HELPER_FLAGS_3(advsimd_maxnumh, TCG_CALL_NO_RWG, f16, f16, f16, ptr) -DEF_HELPER_FLAGS_3(advsimd_minnumh, TCG_CALL_NO_RWG, f16, f16, f16, ptr) -DEF_HELPER_3(advsimd_addh, f16, f16, f16, ptr) -DEF_HELPER_3(advsimd_subh, f16, f16, f16, ptr) -DEF_HELPER_3(advsimd_mulh, f16, f16, f16, ptr) -DEF_HELPER_3(advsimd_divh, f16, f16, f16, ptr) -DEF_HELPER_3(advsimd_ceq_f16, i32, f16, f16, ptr) -DEF_HELPER_3(advsimd_cge_f16, i32, f16, f16, ptr) -DEF_HELPER_3(advsimd_cgt_f16, i32, f16, f16, ptr) -DEF_HELPER_3(advsimd_acge_f16, i32, f16, f16, ptr) -DEF_HELPER_3(advsimd_acgt_f16, i32, f16, f16, ptr) -DEF_HELPER_3(advsimd_mulxh, f16, f16, f16, ptr) -DEF_HELPER_4(advsimd_muladdh, f16, f16, f16, f16, ptr) -DEF_HELPER_3(advsimd_add2h, i32, i32, i32, ptr) -DEF_HELPER_3(advsimd_sub2h, i32, i32, i32, ptr) -DEF_HELPER_3(advsimd_mul2h, i32, i32, i32, ptr) -DEF_HELPER_3(advsimd_div2h, i32, i32, i32, ptr) -DEF_HELPER_3(advsimd_max2h, i32, i32, i32, ptr) -DEF_HELPER_3(advsimd_min2h, i32, i32, i32, ptr) -DEF_HELPER_3(advsimd_maxnum2h, i32, i32, i32, ptr) -DEF_HELPER_3(advsimd_minnum2h, i32, i32, i32, ptr) -DEF_HELPER_3(advsimd_mulx2h, i32, i32, i32, ptr) -DEF_HELPER_4(advsimd_muladd2h, i32, i32, i32, i32, ptr) -DEF_HELPER_2(advsimd_rinth_exact, f16, f16, ptr) -DEF_HELPER_2(advsimd_rinth, f16, f16, ptr) +DEF_HELPER_FLAGS_3(advsimd_maxh, TCG_CALL_NO_RWG, f16, f16, f16, fpst) +DEF_HELPER_FLAGS_3(advsimd_minh, TCG_CALL_NO_RWG, f16, f16, f16, fpst) +DEF_HELPER_FLAGS_3(advsimd_maxnumh, TCG_CALL_NO_RWG, f16, f16, f16, fpst) +DEF_HELPER_FLAGS_3(advsimd_minnumh, TCG_CALL_NO_RWG, f16, f16, f16, fpst) +DEF_HELPER_3(advsimd_addh, f16, f16, f16, fpst) +DEF_HELPER_3(advsimd_subh, f16, f16, f16, fpst) +DEF_HELPER_3(advsimd_mulh, f16, f16, f16, fpst) +DEF_HELPER_3(advsimd_divh, f16, f16, f16, fpst) +DEF_HELPER_3(advsimd_ceq_f16, i32, f16, f16, fpst) +DEF_HELPER_3(advsimd_cge_f16, i32, f16, f16, fpst) +DEF_HELPER_3(advsimd_cgt_f16, i32, f16, f16, fpst) +DEF_HELPER_3(advsimd_acge_f16, i32, f16, f16, fpst) +DEF_HELPER_3(advsimd_acgt_f16, i32, f16, f16, fpst) +DEF_HELPER_3(advsimd_mulxh, f16, f16, f16, fpst) +DEF_HELPER_4(advsimd_muladdh, f16, f16, f16, f16, fpst) +DEF_HELPER_3(advsimd_add2h, i32, i32, i32, fpst) +DEF_HELPER_3(advsimd_sub2h, i32, i32, i32, fpst) +DEF_HELPER_3(advsimd_mul2h, i32, i32, i32, fpst) +DEF_HELPER_3(advsimd_div2h, i32, i32, i32, fpst) +DEF_HELPER_3(advsimd_max2h, i32, i32, i32, fpst) +DEF_HELPER_3(advsimd_min2h, i32, i32, i32, fpst) +DEF_HELPER_3(advsimd_maxnum2h, i32, i32, i32, fpst) +DEF_HELPER_3(advsimd_minnum2h, i32, i32, i32, fpst) +DEF_HELPER_3(advsimd_mulx2h, i32, i32, i32, fpst) +DEF_HELPER_4(advsimd_muladd2h, i32, i32, i32, i32, fpst) +DEF_HELPER_2(advsimd_rinth_exact, f16, f16, fpst) +DEF_HELPER_2(advsimd_rinth, f16, f16, fpst) DEF_HELPER_2(exception_return, void, env, i64) DEF_HELPER_FLAGS_2(dc_zva, TCG_CALL_NO_WG, void, env, i64) diff --git a/target/arm/tcg/helper-a64.c b/target/arm/tcg/helper-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/helper-a64.c +++ b/target/arm/tcg/helper-a64.c @@ -XXX,XX +XXX,XX @@ static inline uint32_t float_rel_to_flags(int res) return flags; } -uint64_t HELPER(vfp_cmph_a64)(uint32_t x, uint32_t y, void *fp_status) +uint64_t HELPER(vfp_cmph_a64)(uint32_t x, uint32_t y, float_status *fp_status) { return float_rel_to_flags(float16_compare_quiet(x, y, fp_status)); } -uint64_t HELPER(vfp_cmpeh_a64)(uint32_t x, uint32_t y, void *fp_status) +uint64_t HELPER(vfp_cmpeh_a64)(uint32_t x, uint32_t y, float_status *fp_status) { return float_rel_to_flags(float16_compare(x, y, fp_status)); } -uint64_t HELPER(vfp_cmps_a64)(float32 x, float32 y, void *fp_status) +uint64_t HELPER(vfp_cmps_a64)(float32 x, float32 y, float_status *fp_status) { return float_rel_to_flags(float32_compare_quiet(x, y, fp_status)); } -uint64_t HELPER(vfp_cmpes_a64)(float32 x, float32 y, void *fp_status) +uint64_t HELPER(vfp_cmpes_a64)(float32 x, float32 y, float_status *fp_status) { return float_rel_to_flags(float32_compare(x, y, fp_status)); } -uint64_t HELPER(vfp_cmpd_a64)(float64 x, float64 y, void *fp_status) +uint64_t HELPER(vfp_cmpd_a64)(float64 x, float64 y, float_status *fp_status) { return float_rel_to_flags(float64_compare_quiet(x, y, fp_status)); } -uint64_t HELPER(vfp_cmped_a64)(float64 x, float64 y, void *fp_status) +uint64_t HELPER(vfp_cmped_a64)(float64 x, float64 y, float_status *fp_status) { return float_rel_to_flags(float64_compare(x, y, fp_status)); } -float32 HELPER(vfp_mulxs)(float32 a, float32 b, void *fpstp) +float32 HELPER(vfp_mulxs)(float32 a, float32 b, float_status *fpst) { - float_status *fpst = fpstp; - a = float32_squash_input_denormal(a, fpst); b = float32_squash_input_denormal(b, fpst); @@ -XXX,XX +XXX,XX @@ float32 HELPER(vfp_mulxs)(float32 a, float32 b, void *fpstp) return float32_mul(a, b, fpst); } -float64 HELPER(vfp_mulxd)(float64 a, float64 b, void *fpstp) +float64 HELPER(vfp_mulxd)(float64 a, float64 b, float_status *fpst) { - float_status *fpst = fpstp; - a = float64_squash_input_denormal(a, fpst); b = float64_squash_input_denormal(b, fpst); @@ -XXX,XX +XXX,XX @@ float64 HELPER(vfp_mulxd)(float64 a, float64 b, void *fpstp) } /* 64bit/double versions of the neon float compare functions */ -uint64_t HELPER(neon_ceq_f64)(float64 a, float64 b, void *fpstp) +uint64_t HELPER(neon_ceq_f64)(float64 a, float64 b, float_status *fpst) { - float_status *fpst = fpstp; return -float64_eq_quiet(a, b, fpst); } -uint64_t HELPER(neon_cge_f64)(float64 a, float64 b, void *fpstp) +uint64_t HELPER(neon_cge_f64)(float64 a, float64 b, float_status *fpst) { - float_status *fpst = fpstp; return -float64_le(b, a, fpst); } -uint64_t HELPER(neon_cgt_f64)(float64 a, float64 b, void *fpstp) +uint64_t HELPER(neon_cgt_f64)(float64 a, float64 b, float_status *fpst) { - float_status *fpst = fpstp; return -float64_lt(b, a, fpst); } @@ -XXX,XX +XXX,XX @@ uint64_t HELPER(neon_cgt_f64)(float64 a, float64 b, void *fpstp) * multiply-add-and-halve. */ -uint32_t HELPER(recpsf_f16)(uint32_t a, uint32_t b, void *fpstp) +uint32_t HELPER(recpsf_f16)(uint32_t a, uint32_t b, float_status *fpst) { - float_status *fpst = fpstp; - a = float16_squash_input_denormal(a, fpst); b = float16_squash_input_denormal(b, fpst); @@ -XXX,XX +XXX,XX @@ uint32_t HELPER(recpsf_f16)(uint32_t a, uint32_t b, void *fpstp) return float16_muladd(a, b, float16_two, 0, fpst); } -float32 HELPER(recpsf_f32)(float32 a, float32 b, void *fpstp) +float32 HELPER(recpsf_f32)(float32 a, float32 b, float_status *fpst) { - float_status *fpst = fpstp; - a = float32_squash_input_denormal(a, fpst); b = float32_squash_input_denormal(b, fpst); @@ -XXX,XX +XXX,XX @@ float32 HELPER(recpsf_f32)(float32 a, float32 b, void *fpstp) return float32_muladd(a, b, float32_two, 0, fpst); } -float64 HELPER(recpsf_f64)(float64 a, float64 b, void *fpstp) +float64 HELPER(recpsf_f64)(float64 a, float64 b, float_status *fpst) { - float_status *fpst = fpstp; - a = float64_squash_input_denormal(a, fpst); b = float64_squash_input_denormal(b, fpst); @@ -XXX,XX +XXX,XX @@ float64 HELPER(recpsf_f64)(float64 a, float64 b, void *fpstp) return float64_muladd(a, b, float64_two, 0, fpst); } -uint32_t HELPER(rsqrtsf_f16)(uint32_t a, uint32_t b, void *fpstp) +uint32_t HELPER(rsqrtsf_f16)(uint32_t a, uint32_t b, float_status *fpst) { - float_status *fpst = fpstp; - a = float16_squash_input_denormal(a, fpst); b = float16_squash_input_denormal(b, fpst); @@ -XXX,XX +XXX,XX @@ uint32_t HELPER(rsqrtsf_f16)(uint32_t a, uint32_t b, void *fpstp) return float16_muladd(a, b, float16_three, float_muladd_halve_result, fpst); } -float32 HELPER(rsqrtsf_f32)(float32 a, float32 b, void *fpstp) +float32 HELPER(rsqrtsf_f32)(float32 a, float32 b, float_status *fpst) { - float_status *fpst = fpstp; - a = float32_squash_input_denormal(a, fpst); b = float32_squash_input_denormal(b, fpst); @@ -XXX,XX +XXX,XX @@ float32 HELPER(rsqrtsf_f32)(float32 a, float32 b, void *fpstp) return float32_muladd(a, b, float32_three, float_muladd_halve_result, fpst); } -float64 HELPER(rsqrtsf_f64)(float64 a, float64 b, void *fpstp) +float64 HELPER(rsqrtsf_f64)(float64 a, float64 b, float_status *fpst) { - float_status *fpst = fpstp; - a = float64_squash_input_denormal(a, fpst); b = float64_squash_input_denormal(b, fpst); @@ -XXX,XX +XXX,XX @@ float64 HELPER(rsqrtsf_f64)(float64 a, float64 b, void *fpstp) } /* Floating-point reciprocal exponent - see FPRecpX in ARM ARM */ -uint32_t HELPER(frecpx_f16)(uint32_t a, void *fpstp) +uint32_t HELPER(frecpx_f16)(uint32_t a, float_status *fpst) { - float_status *fpst = fpstp; uint16_t val16, sbit; int16_t exp; @@ -XXX,XX +XXX,XX @@ uint32_t HELPER(frecpx_f16)(uint32_t a, void *fpstp) } } -float32 HELPER(frecpx_f32)(float32 a, void *fpstp) +float32 HELPER(frecpx_f32)(float32 a, float_status *fpst) { - float_status *fpst = fpstp; uint32_t val32, sbit; int32_t exp; @@ -XXX,XX +XXX,XX @@ float32 HELPER(frecpx_f32)(float32 a, void *fpstp) } } -float64 HELPER(frecpx_f64)(float64 a, void *fpstp) +float64 HELPER(frecpx_f64)(float64 a, float_status *fpst) { - float_status *fpst = fpstp; uint64_t val64, sbit; int64_t exp; @@ -XXX,XX +XXX,XX @@ uint64_t HELPER(crc32c_64)(uint64_t acc, uint64_t val, uint32_t bytes) #define ADVSIMD_HELPER(name, suffix) HELPER(glue(glue(advsimd_, name), suffix)) #define ADVSIMD_HALFOP(name) \ -uint32_t ADVSIMD_HELPER(name, h)(uint32_t a, uint32_t b, void *fpstp) \ +uint32_t ADVSIMD_HELPER(name, h)(uint32_t a, uint32_t b, float_status *fpst) \ { \ - float_status *fpst = fpstp; \ return float16_ ## name(a, b, fpst); \ } @@ -XXX,XX +XXX,XX @@ ADVSIMD_HALFOP(minnum) ADVSIMD_HALFOP(maxnum) #define ADVSIMD_TWOHALFOP(name) \ -uint32_t ADVSIMD_HELPER(name, 2h)(uint32_t two_a, uint32_t two_b, void *fpstp) \ +uint32_t ADVSIMD_HELPER(name, 2h)(uint32_t two_a, uint32_t two_b, \ + float_status *fpst) \ { \ float16 a1, a2, b1, b2; \ uint32_t r1, r2; \ - float_status *fpst = fpstp; \ a1 = extract32(two_a, 0, 16); \ a2 = extract32(two_a, 16, 16); \ b1 = extract32(two_b, 0, 16); \ @@ -XXX,XX +XXX,XX @@ ADVSIMD_TWOHALFOP(minnum) ADVSIMD_TWOHALFOP(maxnum) /* Data processing - scalar floating-point and advanced SIMD */ -static float16 float16_mulx(float16 a, float16 b, void *fpstp) +static float16 float16_mulx(float16 a, float16 b, float_status *fpst) { - float_status *fpst = fpstp; - a = float16_squash_input_denormal(a, fpst); b = float16_squash_input_denormal(b, fpst); @@ -XXX,XX +XXX,XX @@ ADVSIMD_TWOHALFOP(mulx) /* fused multiply-accumulate */ uint32_t HELPER(advsimd_muladdh)(uint32_t a, uint32_t b, uint32_t c, - void *fpstp) + float_status *fpst) { - float_status *fpst = fpstp; return float16_muladd(a, b, c, 0, fpst); } uint32_t HELPER(advsimd_muladd2h)(uint32_t two_a, uint32_t two_b, - uint32_t two_c, void *fpstp) + uint32_t two_c, float_status *fpst) { - float_status *fpst = fpstp; float16 a1, a2, b1, b2, c1, c2; uint32_t r1, r2; a1 = extract32(two_a, 0, 16); @@ -XXX,XX +XXX,XX @@ uint32_t HELPER(advsimd_muladd2h)(uint32_t two_a, uint32_t two_b, #define ADVSIMD_CMPRES(test) (test) ? 0xffff : 0 -uint32_t HELPER(advsimd_ceq_f16)(uint32_t a, uint32_t b, void *fpstp) +uint32_t HELPER(advsimd_ceq_f16)(uint32_t a, uint32_t b, float_status *fpst) { - float_status *fpst = fpstp; int compare = float16_compare_quiet(a, b, fpst); return ADVSIMD_CMPRES(compare == float_relation_equal); } -uint32_t HELPER(advsimd_cge_f16)(uint32_t a, uint32_t b, void *fpstp) +uint32_t HELPER(advsimd_cge_f16)(uint32_t a, uint32_t b, float_status *fpst) { - float_status *fpst = fpstp; int compare = float16_compare(a, b, fpst); return ADVSIMD_CMPRES(compare == float_relation_greater || compare == float_relation_equal); } -uint32_t HELPER(advsimd_cgt_f16)(uint32_t a, uint32_t b, void *fpstp) +uint32_t HELPER(advsimd_cgt_f16)(uint32_t a, uint32_t b, float_status *fpst) { - float_status *fpst = fpstp; int compare = float16_compare(a, b, fpst); return ADVSIMD_CMPRES(compare == float_relation_greater); } -uint32_t HELPER(advsimd_acge_f16)(uint32_t a, uint32_t b, void *fpstp) +uint32_t HELPER(advsimd_acge_f16)(uint32_t a, uint32_t b, float_status *fpst) { - float_status *fpst = fpstp; float16 f0 = float16_abs(a); float16 f1 = float16_abs(b); int compare = float16_compare(f0, f1, fpst); @@ -XXX,XX +XXX,XX @@ uint32_t HELPER(advsimd_acge_f16)(uint32_t a, uint32_t b, void *fpstp) compare == float_relation_equal); } -uint32_t HELPER(advsimd_acgt_f16)(uint32_t a, uint32_t b, void *fpstp) +uint32_t HELPER(advsimd_acgt_f16)(uint32_t a, uint32_t b, float_status *fpst) { - float_status *fpst = fpstp; float16 f0 = float16_abs(a); float16 f1 = float16_abs(b); int compare = float16_compare(f0, f1, fpst); @@ -XXX,XX +XXX,XX @@ uint32_t HELPER(advsimd_acgt_f16)(uint32_t a, uint32_t b, void *fpstp) } /* round to integral */ -uint32_t HELPER(advsimd_rinth_exact)(uint32_t x, void *fp_status) +uint32_t HELPER(advsimd_rinth_exact)(uint32_t x, float_status *fp_status) { return float16_round_to_int(x, fp_status); } -uint32_t HELPER(advsimd_rinth)(uint32_t x, void *fp_status) +uint32_t HELPER(advsimd_rinth)(uint32_t x, float_status *fp_status) { int old_flags = get_float_exception_flags(fp_status), new_flags; float16 ret; -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Message-id: 20241206031224.78525-5-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/helper.h | 284 ++++++++++++++++++------------------ target/arm/tcg/helper-a64.h | 18 +-- target/arm/tcg/helper-sve.h | 12 +- target/arm/tcg/vec_helper.c | 60 ++++---- 4 files changed, 183 insertions(+), 191 deletions(-) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_5(gvec_usdot_idx_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_fcaddh, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(gvec_fcadds, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(gvec_fcaddd, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(gvec_fcmlah, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(gvec_fcmlah_idx, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(gvec_fcmlas, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(gvec_fcmlas_idx, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(gvec_fcmlad, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) -DEF_HELPER_FLAGS_4(gvec_sstoh, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(gvec_sitos, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(gvec_ustoh, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(gvec_uitos, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(gvec_tosszh, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(gvec_tosizs, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(gvec_touszh, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(gvec_touizs, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_sstoh, TCG_CALL_NO_RWG, void, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_4(gvec_sitos, TCG_CALL_NO_RWG, void, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_4(gvec_ustoh, TCG_CALL_NO_RWG, void, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_4(gvec_uitos, TCG_CALL_NO_RWG, void, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_4(gvec_tosszh, TCG_CALL_NO_RWG, void, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_4(gvec_tosizs, TCG_CALL_NO_RWG, void, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_4(gvec_touszh, TCG_CALL_NO_RWG, void, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_4(gvec_touizs, TCG_CALL_NO_RWG, void, ptr, ptr, fpst, i32) -DEF_HELPER_FLAGS_4(gvec_vcvt_sf, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(gvec_vcvt_uf, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(gvec_vcvt_rz_fs, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(gvec_vcvt_rz_fu, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_vcvt_sf, TCG_CALL_NO_RWG, void, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_4(gvec_vcvt_uf, TCG_CALL_NO_RWG, void, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_4(gvec_vcvt_rz_fs, TCG_CALL_NO_RWG, void, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_4(gvec_vcvt_rz_fu, TCG_CALL_NO_RWG, void, ptr, ptr, fpst, i32) -DEF_HELPER_FLAGS_4(gvec_vcvt_sh, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(gvec_vcvt_uh, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(gvec_vcvt_rz_hs, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(gvec_vcvt_rz_hu, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_vcvt_sh, TCG_CALL_NO_RWG, void, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_4(gvec_vcvt_uh, TCG_CALL_NO_RWG, void, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_4(gvec_vcvt_rz_hs, TCG_CALL_NO_RWG, void, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_4(gvec_vcvt_rz_hu, TCG_CALL_NO_RWG, void, ptr, ptr, fpst, i32) -DEF_HELPER_FLAGS_4(gvec_vcvt_sd, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(gvec_vcvt_ud, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(gvec_vcvt_rz_ds, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(gvec_vcvt_rz_du, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_vcvt_sd, TCG_CALL_NO_RWG, void, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_4(gvec_vcvt_ud, TCG_CALL_NO_RWG, void, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_4(gvec_vcvt_rz_ds, TCG_CALL_NO_RWG, void, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_4(gvec_vcvt_rz_du, TCG_CALL_NO_RWG, void, ptr, ptr, fpst, i32) -DEF_HELPER_FLAGS_4(gvec_vcvt_rm_sd, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(gvec_vcvt_rm_ud, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(gvec_vcvt_rm_ss, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(gvec_vcvt_rm_us, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(gvec_vcvt_rm_sh, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(gvec_vcvt_rm_uh, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_vcvt_rm_sd, TCG_CALL_NO_RWG, void, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_4(gvec_vcvt_rm_ud, TCG_CALL_NO_RWG, void, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_4(gvec_vcvt_rm_ss, TCG_CALL_NO_RWG, void, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_4(gvec_vcvt_rm_us, TCG_CALL_NO_RWG, void, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_4(gvec_vcvt_rm_sh, TCG_CALL_NO_RWG, void, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_4(gvec_vcvt_rm_uh, TCG_CALL_NO_RWG, void, ptr, ptr, fpst, i32) -DEF_HELPER_FLAGS_4(gvec_vrint_rm_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(gvec_vrint_rm_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_vrint_rm_h, TCG_CALL_NO_RWG, void, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_4(gvec_vrint_rm_s, TCG_CALL_NO_RWG, void, ptr, ptr, fpst, i32) -DEF_HELPER_FLAGS_4(gvec_vrintx_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(gvec_vrintx_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_vrintx_h, TCG_CALL_NO_RWG, void, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_4(gvec_vrintx_s, TCG_CALL_NO_RWG, void, ptr, ptr, fpst, i32) -DEF_HELPER_FLAGS_4(gvec_frecpe_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(gvec_frecpe_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(gvec_frecpe_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_frecpe_h, TCG_CALL_NO_RWG, void, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_4(gvec_frecpe_s, TCG_CALL_NO_RWG, void, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_4(gvec_frecpe_d, TCG_CALL_NO_RWG, void, ptr, ptr, fpst, i32) -DEF_HELPER_FLAGS_4(gvec_frsqrte_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(gvec_frsqrte_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(gvec_frsqrte_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_frsqrte_h, TCG_CALL_NO_RWG, void, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_4(gvec_frsqrte_s, TCG_CALL_NO_RWG, void, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_4(gvec_frsqrte_d, TCG_CALL_NO_RWG, void, ptr, ptr, fpst, i32) -DEF_HELPER_FLAGS_4(gvec_fcgt0_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(gvec_fcgt0_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(gvec_fcgt0_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_fcgt0_h, TCG_CALL_NO_RWG, void, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_4(gvec_fcgt0_s, TCG_CALL_NO_RWG, void, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_4(gvec_fcgt0_d, TCG_CALL_NO_RWG, void, ptr, ptr, fpst, i32) -DEF_HELPER_FLAGS_4(gvec_fcge0_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(gvec_fcge0_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(gvec_fcge0_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_fcge0_h, TCG_CALL_NO_RWG, void, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_4(gvec_fcge0_s, TCG_CALL_NO_RWG, void, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_4(gvec_fcge0_d, TCG_CALL_NO_RWG, void, ptr, ptr, fpst, i32) -DEF_HELPER_FLAGS_4(gvec_fceq0_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(gvec_fceq0_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(gvec_fceq0_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_fceq0_h, TCG_CALL_NO_RWG, void, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_4(gvec_fceq0_s, TCG_CALL_NO_RWG, void, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_4(gvec_fceq0_d, TCG_CALL_NO_RWG, void, ptr, ptr, fpst, i32) -DEF_HELPER_FLAGS_4(gvec_fcle0_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(gvec_fcle0_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(gvec_fcle0_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_fcle0_h, TCG_CALL_NO_RWG, void, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_4(gvec_fcle0_s, TCG_CALL_NO_RWG, void, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_4(gvec_fcle0_d, TCG_CALL_NO_RWG, void, ptr, ptr, fpst, i32) -DEF_HELPER_FLAGS_4(gvec_fclt0_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(gvec_fclt0_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(gvec_fclt0_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_fclt0_h, TCG_CALL_NO_RWG, void, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_4(gvec_fclt0_s, TCG_CALL_NO_RWG, void, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_4(gvec_fclt0_d, TCG_CALL_NO_RWG, void, ptr, ptr, fpst, i32) -DEF_HELPER_FLAGS_5(gvec_fadd_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(gvec_fadd_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(gvec_fadd_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_fadd_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_5(gvec_fadd_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_5(gvec_fadd_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) -DEF_HELPER_FLAGS_5(gvec_fsub_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(gvec_fsub_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(gvec_fsub_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_fsub_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_5(gvec_fsub_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_5(gvec_fsub_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) -DEF_HELPER_FLAGS_5(gvec_fmul_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(gvec_fmul_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(gvec_fmul_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_fmul_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_5(gvec_fmul_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_5(gvec_fmul_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) -DEF_HELPER_FLAGS_5(gvec_fabd_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(gvec_fabd_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(gvec_fabd_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_fabd_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_5(gvec_fabd_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_5(gvec_fabd_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) -DEF_HELPER_FLAGS_5(gvec_fceq_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(gvec_fceq_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(gvec_fceq_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_fceq_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_5(gvec_fceq_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_5(gvec_fceq_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) -DEF_HELPER_FLAGS_5(gvec_fcge_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(gvec_fcge_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(gvec_fcge_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_fcge_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_5(gvec_fcge_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_5(gvec_fcge_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) -DEF_HELPER_FLAGS_5(gvec_fcgt_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(gvec_fcgt_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(gvec_fcgt_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_fcgt_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_5(gvec_fcgt_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_5(gvec_fcgt_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) -DEF_HELPER_FLAGS_5(gvec_facge_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(gvec_facge_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(gvec_facge_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_facge_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_5(gvec_facge_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_5(gvec_facge_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) -DEF_HELPER_FLAGS_5(gvec_facgt_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(gvec_facgt_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(gvec_facgt_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_facgt_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_5(gvec_facgt_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_5(gvec_facgt_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) -DEF_HELPER_FLAGS_5(gvec_fmax_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(gvec_fmax_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(gvec_fmax_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_fmax_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_5(gvec_fmax_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_5(gvec_fmax_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) -DEF_HELPER_FLAGS_5(gvec_fmin_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(gvec_fmin_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(gvec_fmin_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_fmin_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_5(gvec_fmin_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_5(gvec_fmin_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) -DEF_HELPER_FLAGS_5(gvec_fmaxnum_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(gvec_fmaxnum_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(gvec_fmaxnum_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_fmaxnum_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_5(gvec_fmaxnum_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_5(gvec_fmaxnum_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) -DEF_HELPER_FLAGS_5(gvec_fminnum_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(gvec_fminnum_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(gvec_fminnum_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_fminnum_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_5(gvec_fminnum_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_5(gvec_fminnum_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) -DEF_HELPER_FLAGS_5(gvec_recps_nf_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(gvec_recps_nf_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_recps_nf_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_5(gvec_recps_nf_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) -DEF_HELPER_FLAGS_5(gvec_rsqrts_nf_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(gvec_rsqrts_nf_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_rsqrts_nf_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_5(gvec_rsqrts_nf_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) -DEF_HELPER_FLAGS_5(gvec_fmla_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(gvec_fmla_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_fmla_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_5(gvec_fmla_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) -DEF_HELPER_FLAGS_5(gvec_fmls_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(gvec_fmls_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_fmls_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_5(gvec_fmls_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) -DEF_HELPER_FLAGS_5(gvec_vfma_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(gvec_vfma_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(gvec_vfma_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_vfma_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_5(gvec_vfma_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_5(gvec_vfma_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) -DEF_HELPER_FLAGS_5(gvec_vfms_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(gvec_vfms_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(gvec_vfms_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_vfms_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_5(gvec_vfms_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_5(gvec_vfms_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(gvec_ftsmul_h, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(gvec_ftsmul_s, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(gvec_ftsmul_d, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(gvec_fmul_idx_h, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(gvec_fmul_idx_s, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(gvec_fmul_idx_d, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(gvec_fmla_nf_idx_h, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(gvec_fmla_nf_idx_s, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(gvec_fmls_nf_idx_h, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(gvec_fmls_nf_idx_s, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(gvec_fmla_idx_h, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(gvec_fmla_idx_s, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(gvec_fmla_idx_d, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(gvec_uqadd_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_6(gvec_bfmmla, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, env, i32) DEF_HELPER_FLAGS_6(gvec_bfmlal, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(gvec_bfmlal_idx, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(gvec_sclamp_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_5(gvec_uclamp_s, TCG_CALL_NO_RWG, DEF_HELPER_FLAGS_5(gvec_uclamp_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(gvec_faddp_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(gvec_faddp_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(gvec_faddp_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_faddp_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_5(gvec_faddp_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_5(gvec_faddp_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) -DEF_HELPER_FLAGS_5(gvec_fmaxp_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(gvec_fmaxp_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(gvec_fmaxp_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_fmaxp_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_5(gvec_fmaxp_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_5(gvec_fmaxp_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) -DEF_HELPER_FLAGS_5(gvec_fminp_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(gvec_fminp_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(gvec_fminp_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_fminp_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_5(gvec_fminp_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_5(gvec_fminp_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) -DEF_HELPER_FLAGS_5(gvec_fmaxnump_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(gvec_fmaxnump_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(gvec_fmaxnump_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_fmaxnump_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_5(gvec_fmaxnump_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_5(gvec_fmaxnump_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) -DEF_HELPER_FLAGS_5(gvec_fminnump_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(gvec_fminnump_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(gvec_fminnump_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_fminnump_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_5(gvec_fminnump_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_5(gvec_fminnump_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_4(gvec_addp_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_addp_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) diff --git a/target/arm/tcg/helper-a64.h b/target/arm/tcg/helper-a64.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/helper-a64.h +++ b/target/arm/tcg/helper-a64.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_4(cpyfe, void, env, i32, i32, i32) DEF_HELPER_FLAGS_1(guarded_page_check, TCG_CALL_NO_WG, void, env) DEF_HELPER_FLAGS_2(guarded_page_br, TCG_CALL_NO_RWG, void, env, tl) -DEF_HELPER_FLAGS_5(gvec_fdiv_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(gvec_fdiv_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(gvec_fdiv_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_fdiv_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_5(gvec_fdiv_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_5(gvec_fdiv_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) -DEF_HELPER_FLAGS_5(gvec_fmulx_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(gvec_fmulx_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(gvec_fmulx_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_fmulx_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_5(gvec_fmulx_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_5(gvec_fmulx_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) -DEF_HELPER_FLAGS_5(gvec_fmulx_idx_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(gvec_fmulx_idx_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(gvec_fmulx_idx_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_fmulx_idx_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_5(gvec_fmulx_idx_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_5(gvec_fmulx_idx_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) diff --git a/target/arm/tcg/helper-sve.h b/target/arm/tcg/helper-sve.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/helper-sve.h +++ b/target/arm/tcg/helper-sve.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_4(sve_umini_s, TCG_CALL_NO_RWG, void, ptr, ptr, i64, i32) DEF_HELPER_FLAGS_4(sve_umini_d, TCG_CALL_NO_RWG, void, ptr, ptr, i64, i32) DEF_HELPER_FLAGS_5(gvec_recps_h, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(gvec_recps_s, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(gvec_recps_d, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(gvec_rsqrts_h, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(gvec_rsqrts_s, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(gvec_rsqrts_d, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_4(sve_faddv_h, TCG_CALL_NO_RWG, i64, ptr, ptr, ptr, i32) diff --git a/target/arm/tcg/vec_helper.c b/target/arm/tcg/vec_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/vec_helper.c +++ b/target/arm/tcg/vec_helper.c @@ -XXX,XX +XXX,XX @@ DO_DOT_IDX(gvec_sdot_idx_h, int64_t, int16_t, int16_t, H8) DO_DOT_IDX(gvec_udot_idx_h, uint64_t, uint16_t, uint16_t, H8) void HELPER(gvec_fcaddh)(void *vd, void *vn, void *vm, - void *vfpst, uint32_t desc) + float_status *fpst, uint32_t desc) { uintptr_t opr_sz = simd_oprsz(desc); float16 *d = vd; float16 *n = vn; float16 *m = vm; - float_status *fpst = vfpst; uint32_t neg_real = extract32(desc, SIMD_DATA_SHIFT, 1); uint32_t neg_imag = neg_real ^ 1; uintptr_t i; @@ -XXX,XX +XXX,XX @@ void HELPER(gvec_fcaddh)(void *vd, void *vn, void *vm, } void HELPER(gvec_fcadds)(void *vd, void *vn, void *vm, - void *vfpst, uint32_t desc) + float_status *fpst, uint32_t desc) { uintptr_t opr_sz = simd_oprsz(desc); float32 *d = vd; float32 *n = vn; float32 *m = vm; - float_status *fpst = vfpst; uint32_t neg_real = extract32(desc, SIMD_DATA_SHIFT, 1); uint32_t neg_imag = neg_real ^ 1; uintptr_t i; @@ -XXX,XX +XXX,XX @@ void HELPER(gvec_fcadds)(void *vd, void *vn, void *vm, } void HELPER(gvec_fcaddd)(void *vd, void *vn, void *vm, - void *vfpst, uint32_t desc) + float_status *fpst, uint32_t desc) { uintptr_t opr_sz = simd_oprsz(desc); float64 *d = vd; float64 *n = vn; float64 *m = vm; - float_status *fpst = vfpst; uint64_t neg_real = extract64(desc, SIMD_DATA_SHIFT, 1); uint64_t neg_imag = neg_real ^ 1; uintptr_t i; @@ -XXX,XX +XXX,XX @@ void HELPER(gvec_fcaddd)(void *vd, void *vn, void *vm, } void HELPER(gvec_fcmlah)(void *vd, void *vn, void *vm, void *va, - void *vfpst, uint32_t desc) + float_status *fpst, uint32_t desc) { uintptr_t opr_sz = simd_oprsz(desc); float16 *d = vd, *n = vn, *m = vm, *a = va; - float_status *fpst = vfpst; intptr_t flip = extract32(desc, SIMD_DATA_SHIFT, 1); uint32_t neg_imag = extract32(desc, SIMD_DATA_SHIFT + 1, 1); uint32_t neg_real = flip ^ neg_imag; @@ -XXX,XX +XXX,XX @@ void HELPER(gvec_fcmlah)(void *vd, void *vn, void *vm, void *va, } void HELPER(gvec_fcmlah_idx)(void *vd, void *vn, void *vm, void *va, - void *vfpst, uint32_t desc) + float_status *fpst, uint32_t desc) { uintptr_t opr_sz = simd_oprsz(desc); float16 *d = vd, *n = vn, *m = vm, *a = va; - float_status *fpst = vfpst; intptr_t flip = extract32(desc, SIMD_DATA_SHIFT, 1); uint32_t neg_imag = extract32(desc, SIMD_DATA_SHIFT + 1, 1); intptr_t index = extract32(desc, SIMD_DATA_SHIFT + 2, 2); @@ -XXX,XX +XXX,XX @@ void HELPER(gvec_fcmlah_idx)(void *vd, void *vn, void *vm, void *va, } void HELPER(gvec_fcmlas)(void *vd, void *vn, void *vm, void *va, - void *vfpst, uint32_t desc) + float_status *fpst, uint32_t desc) { uintptr_t opr_sz = simd_oprsz(desc); float32 *d = vd, *n = vn, *m = vm, *a = va; - float_status *fpst = vfpst; intptr_t flip = extract32(desc, SIMD_DATA_SHIFT, 1); uint32_t neg_imag = extract32(desc, SIMD_DATA_SHIFT + 1, 1); uint32_t neg_real = flip ^ neg_imag; @@ -XXX,XX +XXX,XX @@ void HELPER(gvec_fcmlas)(void *vd, void *vn, void *vm, void *va, } void HELPER(gvec_fcmlas_idx)(void *vd, void *vn, void *vm, void *va, - void *vfpst, uint32_t desc) + float_status *fpst, uint32_t desc) { uintptr_t opr_sz = simd_oprsz(desc); float32 *d = vd, *n = vn, *m = vm, *a = va; - float_status *fpst = vfpst; intptr_t flip = extract32(desc, SIMD_DATA_SHIFT, 1); uint32_t neg_imag = extract32(desc, SIMD_DATA_SHIFT + 1, 1); intptr_t index = extract32(desc, SIMD_DATA_SHIFT + 2, 2); @@ -XXX,XX +XXX,XX @@ void HELPER(gvec_fcmlas_idx)(void *vd, void *vn, void *vm, void *va, } void HELPER(gvec_fcmlad)(void *vd, void *vn, void *vm, void *va, - void *vfpst, uint32_t desc) + float_status *fpst, uint32_t desc) { uintptr_t opr_sz = simd_oprsz(desc); float64 *d = vd, *n = vn, *m = vm, *a = va; - float_status *fpst = vfpst; intptr_t flip = extract32(desc, SIMD_DATA_SHIFT, 1); uint64_t neg_imag = extract32(desc, SIMD_DATA_SHIFT + 1, 1); uint64_t neg_real = flip ^ neg_imag; @@ -XXX,XX +XXX,XX @@ static uint64_t float64_acgt(float64 op1, float64 op2, float_status *stat) return -float64_lt(float64_abs(op2), float64_abs(op1), stat); } -static int16_t vfp_tosszh(float16 x, void *fpstp) +static int16_t vfp_tosszh(float16 x, float_status *fpst) { - float_status *fpst = fpstp; if (float16_is_any_nan(x)) { float_raise(float_flag_invalid, fpst); return 0; @@ -XXX,XX +XXX,XX @@ static int16_t vfp_tosszh(float16 x, void *fpstp) return float16_to_int16_round_to_zero(x, fpst); } -static uint16_t vfp_touszh(float16 x, void *fpstp) +static uint16_t vfp_touszh(float16 x, float_status *fpst) { - float_status *fpst = fpstp; if (float16_is_any_nan(x)) { float_raise(float_flag_invalid, fpst); return 0; @@ -XXX,XX +XXX,XX @@ static uint16_t vfp_touszh(float16 x, void *fpstp) } #define DO_2OP(NAME, FUNC, TYPE) \ -void HELPER(NAME)(void *vd, void *vn, void *stat, uint32_t desc) \ +void HELPER(NAME)(void *vd, void *vn, float_status *stat, uint32_t desc) \ { \ intptr_t i, oprsz = simd_oprsz(desc); \ TYPE *d = vd, *n = vn; \ @@ -XXX,XX +XXX,XX @@ static float32 float32_rsqrts_nf(float32 op1, float32 op2, float_status *stat) } #define DO_3OP(NAME, FUNC, TYPE) \ -void HELPER(NAME)(void *vd, void *vn, void *vm, void *stat, uint32_t desc) \ +void HELPER(NAME)(void *vd, void *vn, void *vm, \ + float_status *stat, uint32_t desc) \ { \ intptr_t i, oprsz = simd_oprsz(desc); \ TYPE *d = vd, *n = vn, *m = vm; \ @@ -XXX,XX +XXX,XX @@ static float64 float64_mulsub_f(float64 dest, float64 op1, float64 op2, return float64_muladd(float64_chs(op1), op2, dest, 0, stat); } -#define DO_MULADD(NAME, FUNC, TYPE) \ -void HELPER(NAME)(void *vd, void *vn, void *vm, void *stat, uint32_t desc) \ +#define DO_MULADD(NAME, FUNC, TYPE) \ +void HELPER(NAME)(void *vd, void *vn, void *vm, \ + float_status *stat, uint32_t desc) \ { \ intptr_t i, oprsz = simd_oprsz(desc); \ TYPE *d = vd, *n = vn, *m = vm; \ @@ -XXX,XX +XXX,XX @@ DO_MLA_IDX(gvec_mls_idx_d, uint64_t, -, H8) #undef DO_MLA_IDX #define DO_FMUL_IDX(NAME, ADD, MUL, TYPE, H) \ -void HELPER(NAME)(void *vd, void *vn, void *vm, void *stat, uint32_t desc) \ +void HELPER(NAME)(void *vd, void *vn, void *vm, \ + float_status *stat, uint32_t desc) \ { \ intptr_t i, j, oprsz = simd_oprsz(desc); \ intptr_t segment = MIN(16, oprsz) / sizeof(TYPE); \ @@ -XXX,XX +XXX,XX @@ DO_FMUL_IDX(gvec_fmls_nf_idx_s, float32_sub, float32_mul, float32, H4) #define DO_FMLA_IDX(NAME, TYPE, H) \ void HELPER(NAME)(void *vd, void *vn, void *vm, void *va, \ - void *stat, uint32_t desc) \ + float_status *stat, uint32_t desc) \ { \ intptr_t i, j, oprsz = simd_oprsz(desc); \ intptr_t segment = MIN(16, oprsz) / sizeof(TYPE); \ @@ -XXX,XX +XXX,XX @@ DO_ABA(gvec_uaba_d, uint64_t) #undef DO_ABA #define DO_3OP_PAIR(NAME, FUNC, TYPE, H) \ -void HELPER(NAME)(void *vd, void *vn, void *vm, void *stat, uint32_t desc) \ +void HELPER(NAME)(void *vd, void *vn, void *vm, \ + float_status *stat, uint32_t desc) \ { \ ARMVectorReg scratch; \ intptr_t oprsz = simd_oprsz(desc); \ @@ -XXX,XX +XXX,XX @@ DO_3OP_PAIR(gvec_uminp_s, MIN, uint32_t, H4) #undef DO_3OP_PAIR #define DO_VCVT_FIXED(NAME, FUNC, TYPE) \ - void HELPER(NAME)(void *vd, void *vn, void *stat, uint32_t desc) \ + void HELPER(NAME)(void *vd, void *vn, float_status *stat, uint32_t desc) \ { \ intptr_t i, oprsz = simd_oprsz(desc); \ int shift = simd_data(desc); \ @@ -XXX,XX +XXX,XX @@ DO_VCVT_FIXED(gvec_vcvt_rz_hu, helper_vfp_touhh_round_to_zero, uint16_t) #undef DO_VCVT_FIXED #define DO_VCVT_RMODE(NAME, FUNC, TYPE) \ - void HELPER(NAME)(void *vd, void *vn, void *stat, uint32_t desc) \ + void HELPER(NAME)(void *vd, void *vn, float_status *fpst, uint32_t desc) \ { \ - float_status *fpst = stat; \ intptr_t i, oprsz = simd_oprsz(desc); \ uint32_t rmode = simd_data(desc); \ uint32_t prev_rmode = get_float_rounding_mode(fpst); \ @@ -XXX,XX +XXX,XX @@ DO_VCVT_RMODE(gvec_vcvt_rm_uh, helper_vfp_touhh, uint16_t) #undef DO_VCVT_RMODE #define DO_VRINT_RMODE(NAME, FUNC, TYPE) \ - void HELPER(NAME)(void *vd, void *vn, void *stat, uint32_t desc) \ + void HELPER(NAME)(void *vd, void *vn, float_status *fpst, uint32_t desc) \ { \ - float_status *fpst = stat; \ intptr_t i, oprsz = simd_oprsz(desc); \ uint32_t rmode = simd_data(desc); \ uint32_t prev_rmode = get_float_rounding_mode(fpst); \ @@ -XXX,XX +XXX,XX @@ void HELPER(gvec_bfmmla)(void *vd, void *vn, void *vm, void *va, } void HELPER(gvec_bfmlal)(void *vd, void *vn, void *vm, void *va, - void *stat, uint32_t desc) + float_status *stat, uint32_t desc) { intptr_t i, opr_sz = simd_oprsz(desc); intptr_t sel = simd_data(desc); @@ -XXX,XX +XXX,XX @@ void HELPER(gvec_bfmlal)(void *vd, void *vn, void *vm, void *va, } void HELPER(gvec_bfmlal_idx)(void *vd, void *vn, void *vm, - void *va, void *stat, uint32_t desc) + void *va, float_status *stat, uint32_t desc) { intptr_t i, j, opr_sz = simd_oprsz(desc); intptr_t sel = extract32(desc, SIMD_DATA_SHIFT, 1); -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Message-id: 20241206031224.78525-6-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/helper.h | 14 +++++++------- target/arm/tcg/neon_helper.c | 21 +++++++-------------- 2 files changed, 14 insertions(+), 21 deletions(-) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_2(neon_qneg_s16, TCG_CALL_NO_RWG, i32, env, i32) DEF_HELPER_FLAGS_2(neon_qneg_s32, TCG_CALL_NO_RWG, i32, env, i32) DEF_HELPER_FLAGS_2(neon_qneg_s64, TCG_CALL_NO_RWG, i64, env, i64) -DEF_HELPER_3(neon_ceq_f32, i32, i32, i32, ptr) -DEF_HELPER_3(neon_cge_f32, i32, i32, i32, ptr) -DEF_HELPER_3(neon_cgt_f32, i32, i32, i32, ptr) -DEF_HELPER_3(neon_acge_f32, i32, i32, i32, ptr) -DEF_HELPER_3(neon_acgt_f32, i32, i32, i32, ptr) -DEF_HELPER_3(neon_acge_f64, i64, i64, i64, ptr) -DEF_HELPER_3(neon_acgt_f64, i64, i64, i64, ptr) +DEF_HELPER_3(neon_ceq_f32, i32, i32, i32, fpst) +DEF_HELPER_3(neon_cge_f32, i32, i32, i32, fpst) +DEF_HELPER_3(neon_cgt_f32, i32, i32, i32, fpst) +DEF_HELPER_3(neon_acge_f32, i32, i32, i32, fpst) +DEF_HELPER_3(neon_acgt_f32, i32, i32, i32, fpst) +DEF_HELPER_3(neon_acge_f64, i64, i64, i64, fpst) +DEF_HELPER_3(neon_acgt_f64, i64, i64, i64, fpst) /* iwmmxt_helper.c */ DEF_HELPER_2(iwmmxt_maddsq, i64, i64, i64) diff --git a/target/arm/tcg/neon_helper.c b/target/arm/tcg/neon_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/neon_helper.c +++ b/target/arm/tcg/neon_helper.c @@ -XXX,XX +XXX,XX @@ uint64_t HELPER(neon_qneg_s64)(CPUARMState *env, uint64_t x) * Note that EQ doesn't signal InvalidOp for QNaNs but GE and GT do. * Softfloat routines return 0/1, which we convert to the 0/-1 Neon requires. */ -uint32_t HELPER(neon_ceq_f32)(uint32_t a, uint32_t b, void *fpstp) +uint32_t HELPER(neon_ceq_f32)(uint32_t a, uint32_t b, float_status *fpst) { - float_status *fpst = fpstp; return -float32_eq_quiet(make_float32(a), make_float32(b), fpst); } -uint32_t HELPER(neon_cge_f32)(uint32_t a, uint32_t b, void *fpstp) +uint32_t HELPER(neon_cge_f32)(uint32_t a, uint32_t b, float_status *fpst) { - float_status *fpst = fpstp; return -float32_le(make_float32(b), make_float32(a), fpst); } -uint32_t HELPER(neon_cgt_f32)(uint32_t a, uint32_t b, void *fpstp) +uint32_t HELPER(neon_cgt_f32)(uint32_t a, uint32_t b, float_status *fpst) { - float_status *fpst = fpstp; return -float32_lt(make_float32(b), make_float32(a), fpst); } -uint32_t HELPER(neon_acge_f32)(uint32_t a, uint32_t b, void *fpstp) +uint32_t HELPER(neon_acge_f32)(uint32_t a, uint32_t b, float_status *fpst) { - float_status *fpst = fpstp; float32 f0 = float32_abs(make_float32(a)); float32 f1 = float32_abs(make_float32(b)); return -float32_le(f1, f0, fpst); } -uint32_t HELPER(neon_acgt_f32)(uint32_t a, uint32_t b, void *fpstp) +uint32_t HELPER(neon_acgt_f32)(uint32_t a, uint32_t b, float_status *fpst) { - float_status *fpst = fpstp; float32 f0 = float32_abs(make_float32(a)); float32 f1 = float32_abs(make_float32(b)); return -float32_lt(f1, f0, fpst); } -uint64_t HELPER(neon_acge_f64)(uint64_t a, uint64_t b, void *fpstp) +uint64_t HELPER(neon_acge_f64)(uint64_t a, uint64_t b, float_status *fpst) { - float_status *fpst = fpstp; float64 f0 = float64_abs(make_float64(a)); float64 f1 = float64_abs(make_float64(b)); return -float64_le(f1, f0, fpst); } -uint64_t HELPER(neon_acgt_f64)(uint64_t a, uint64_t b, void *fpstp) +uint64_t HELPER(neon_acgt_f64)(uint64_t a, uint64_t b, float_status *fpst) { - float_status *fpst = fpstp; float64 f0 = float64_abs(make_float64(a)); float64 f1 = float64_abs(make_float64(b)); return -float64_lt(f1, f0, fpst); -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Message-id: 20241206031224.78525-7-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/helper-sve.h | 414 ++++++++++++++++++------------------ target/arm/tcg/sve_helper.c | 96 +++++---- 2 files changed, 258 insertions(+), 252 deletions(-) diff --git a/target/arm/tcg/helper-sve.h b/target/arm/tcg/helper-sve.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/helper-sve.h +++ b/target/arm/tcg/helper-sve.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_5(gvec_rsqrts_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_4(sve_faddv_h, TCG_CALL_NO_RWG, - i64, ptr, ptr, ptr, i32) + i64, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_4(sve_faddv_s, TCG_CALL_NO_RWG, - i64, ptr, ptr, ptr, i32) + i64, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_4(sve_faddv_d, TCG_CALL_NO_RWG, - i64, ptr, ptr, ptr, i32) + i64, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_4(sve_fmaxnmv_h, TCG_CALL_NO_RWG, - i64, ptr, ptr, ptr, i32) + i64, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_4(sve_fmaxnmv_s, TCG_CALL_NO_RWG, - i64, ptr, ptr, ptr, i32) + i64, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_4(sve_fmaxnmv_d, TCG_CALL_NO_RWG, - i64, ptr, ptr, ptr, i32) + i64, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_4(sve_fminnmv_h, TCG_CALL_NO_RWG, - i64, ptr, ptr, ptr, i32) + i64, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_4(sve_fminnmv_s, TCG_CALL_NO_RWG, - i64, ptr, ptr, ptr, i32) + i64, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_4(sve_fminnmv_d, TCG_CALL_NO_RWG, - i64, ptr, ptr, ptr, i32) + i64, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_4(sve_fmaxv_h, TCG_CALL_NO_RWG, - i64, ptr, ptr, ptr, i32) + i64, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_4(sve_fmaxv_s, TCG_CALL_NO_RWG, - i64, ptr, ptr, ptr, i32) + i64, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_4(sve_fmaxv_d, TCG_CALL_NO_RWG, - i64, ptr, ptr, ptr, i32) + i64, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_4(sve_fminv_h, TCG_CALL_NO_RWG, - i64, ptr, ptr, ptr, i32) + i64, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_4(sve_fminv_s, TCG_CALL_NO_RWG, - i64, ptr, ptr, ptr, i32) + i64, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_4(sve_fminv_d, TCG_CALL_NO_RWG, - i64, ptr, ptr, ptr, i32) + i64, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(sve_fadda_h, TCG_CALL_NO_RWG, - i64, i64, ptr, ptr, ptr, i32) + i64, i64, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(sve_fadda_s, TCG_CALL_NO_RWG, - i64, i64, ptr, ptr, ptr, i32) + i64, i64, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(sve_fadda_d, TCG_CALL_NO_RWG, - i64, i64, ptr, ptr, ptr, i32) + i64, i64, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(sve_fcmge0_h, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(sve_fcmge0_s, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(sve_fcmge0_d, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(sve_fcmgt0_h, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(sve_fcmgt0_s, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(sve_fcmgt0_d, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(sve_fcmlt0_h, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(sve_fcmlt0_s, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(sve_fcmlt0_d, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(sve_fcmle0_h, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(sve_fcmle0_s, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(sve_fcmle0_d, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(sve_fcmeq0_h, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(sve_fcmeq0_s, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(sve_fcmeq0_d, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(sve_fcmne0_h, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(sve_fcmne0_s, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(sve_fcmne0_d, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(sve_fadd_h, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(sve_fadd_s, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(sve_fadd_d, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(sve_fsub_h, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(sve_fsub_s, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(sve_fsub_d, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(sve_fmul_h, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(sve_fmul_s, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(sve_fmul_d, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(sve_fdiv_h, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(sve_fdiv_s, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(sve_fdiv_d, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(sve_fmin_h, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(sve_fmin_s, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(sve_fmin_d, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(sve_fmax_h, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(sve_fmax_s, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(sve_fmax_d, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(sve_fminnum_h, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(sve_fminnum_s, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(sve_fminnum_d, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(sve_fmaxnum_h, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(sve_fmaxnum_s, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(sve_fmaxnum_d, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(sve_fabd_h, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(sve_fabd_s, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(sve_fabd_d, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(sve_fscalbn_h, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(sve_fscalbn_s, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(sve_fscalbn_d, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(sve_fmulx_h, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(sve_fmulx_s, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(sve_fmulx_d, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(sve_fadds_h, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, i64, ptr, i32) + void, ptr, ptr, ptr, i64, fpst, i32) DEF_HELPER_FLAGS_6(sve_fadds_s, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, i64, ptr, i32) + void, ptr, ptr, ptr, i64, fpst, i32) DEF_HELPER_FLAGS_6(sve_fadds_d, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, i64, ptr, i32) + void, ptr, ptr, ptr, i64, fpst, i32) DEF_HELPER_FLAGS_6(sve_fsubs_h, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, i64, ptr, i32) + void, ptr, ptr, ptr, i64, fpst, i32) DEF_HELPER_FLAGS_6(sve_fsubs_s, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, i64, ptr, i32) + void, ptr, ptr, ptr, i64, fpst, i32) DEF_HELPER_FLAGS_6(sve_fsubs_d, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, i64, ptr, i32) + void, ptr, ptr, ptr, i64, fpst, i32) DEF_HELPER_FLAGS_6(sve_fmuls_h, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, i64, ptr, i32) + void, ptr, ptr, ptr, i64, fpst, i32) DEF_HELPER_FLAGS_6(sve_fmuls_s, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, i64, ptr, i32) + void, ptr, ptr, ptr, i64, fpst, i32) DEF_HELPER_FLAGS_6(sve_fmuls_d, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, i64, ptr, i32) + void, ptr, ptr, ptr, i64, fpst, i32) DEF_HELPER_FLAGS_6(sve_fsubrs_h, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, i64, ptr, i32) + void, ptr, ptr, ptr, i64, fpst, i32) DEF_HELPER_FLAGS_6(sve_fsubrs_s, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, i64, ptr, i32) + void, ptr, ptr, ptr, i64, fpst, i32) DEF_HELPER_FLAGS_6(sve_fsubrs_d, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, i64, ptr, i32) + void, ptr, ptr, ptr, i64, fpst, i32) DEF_HELPER_FLAGS_6(sve_fmaxnms_h, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, i64, ptr, i32) + void, ptr, ptr, ptr, i64, fpst, i32) DEF_HELPER_FLAGS_6(sve_fmaxnms_s, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, i64, ptr, i32) + void, ptr, ptr, ptr, i64, fpst, i32) DEF_HELPER_FLAGS_6(sve_fmaxnms_d, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, i64, ptr, i32) + void, ptr, ptr, ptr, i64, fpst, i32) DEF_HELPER_FLAGS_6(sve_fminnms_h, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, i64, ptr, i32) + void, ptr, ptr, ptr, i64, fpst, i32) DEF_HELPER_FLAGS_6(sve_fminnms_s, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, i64, ptr, i32) + void, ptr, ptr, ptr, i64, fpst, i32) DEF_HELPER_FLAGS_6(sve_fminnms_d, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, i64, ptr, i32) + void, ptr, ptr, ptr, i64, fpst, i32) DEF_HELPER_FLAGS_6(sve_fmaxs_h, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, i64, ptr, i32) + void, ptr, ptr, ptr, i64, fpst, i32) DEF_HELPER_FLAGS_6(sve_fmaxs_s, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, i64, ptr, i32) + void, ptr, ptr, ptr, i64, fpst, i32) DEF_HELPER_FLAGS_6(sve_fmaxs_d, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, i64, ptr, i32) + void, ptr, ptr, ptr, i64, fpst, i32) DEF_HELPER_FLAGS_6(sve_fmins_h, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, i64, ptr, i32) + void, ptr, ptr, ptr, i64, fpst, i32) DEF_HELPER_FLAGS_6(sve_fmins_s, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, i64, ptr, i32) + void, ptr, ptr, ptr, i64, fpst, i32) DEF_HELPER_FLAGS_6(sve_fmins_d, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, i64, ptr, i32) + void, ptr, ptr, ptr, i64, fpst, i32) DEF_HELPER_FLAGS_5(sve_fcvt_sh, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(sve_fcvt_dh, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(sve_fcvt_hs, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(sve_fcvt_ds, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(sve_fcvt_hd, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(sve_fcvt_sd, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(sve_bfcvt, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(sve_fcvtzs_hh, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(sve_fcvtzs_hs, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(sve_fcvtzs_ss, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(sve_fcvtzs_ds, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(sve_fcvtzs_hd, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(sve_fcvtzs_sd, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(sve_fcvtzs_dd, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(sve_fcvtzu_hh, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(sve_fcvtzu_hs, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(sve_fcvtzu_ss, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(sve_fcvtzu_ds, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(sve_fcvtzu_hd, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(sve_fcvtzu_sd, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(sve_fcvtzu_dd, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(sve_frint_h, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(sve_frint_s, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(sve_frint_d, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(sve_frintx_h, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(sve_frintx_s, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(sve_frintx_d, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(sve_frecpx_h, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(sve_frecpx_s, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(sve_frecpx_d, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(sve_fsqrt_h, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(sve_fsqrt_s, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(sve_fsqrt_d, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(sve_scvt_hh, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(sve_scvt_sh, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(sve_scvt_dh, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(sve_scvt_ss, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(sve_scvt_sd, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(sve_scvt_ds, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(sve_scvt_dd, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(sve_ucvt_hh, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(sve_ucvt_sh, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(sve_ucvt_dh, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(sve_ucvt_ss, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(sve_ucvt_sd, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(sve_ucvt_ds, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(sve_ucvt_dd, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(sve_fcmge_h, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(sve_fcmge_s, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(sve_fcmge_d, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(sve_fcmgt_h, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(sve_fcmgt_s, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(sve_fcmgt_d, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(sve_fcmeq_h, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(sve_fcmeq_s, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(sve_fcmeq_d, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(sve_fcmne_h, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(sve_fcmne_s, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(sve_fcmne_d, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(sve_fcmuo_h, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(sve_fcmuo_s, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(sve_fcmuo_d, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(sve_facge_h, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(sve_facge_s, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(sve_facge_d, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(sve_facgt_h, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(sve_facgt_s, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(sve_facgt_d, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(sve_fcadd_h, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(sve_fcadd_s, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(sve_fcadd_d, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_7(sve_fmla_zpzzz_h, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_7(sve_fmla_zpzzz_s, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_7(sve_fmla_zpzzz_d, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_7(sve_fmls_zpzzz_h, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_7(sve_fmls_zpzzz_s, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_7(sve_fmls_zpzzz_d, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_7(sve_fnmla_zpzzz_h, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_7(sve_fnmla_zpzzz_s, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_7(sve_fnmla_zpzzz_d, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_7(sve_fnmls_zpzzz_h, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_7(sve_fnmls_zpzzz_s, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_7(sve_fnmls_zpzzz_d, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_7(sve_fcmla_zpzzz_h, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_7(sve_fcmla_zpzzz_s, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_7(sve_fcmla_zpzzz_d, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, ptr, fpst, i32) -DEF_HELPER_FLAGS_5(sve_ftmad_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(sve_ftmad_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(sve_ftmad_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve_ftmad_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_5(sve_ftmad_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_5(sve_ftmad_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_4(sve2_saddl_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve2_saddl_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_4(sve2_xar_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve2_xar_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_6(sve2_faddp_zpzz_h, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(sve2_faddp_zpzz_s, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(sve2_faddp_zpzz_d, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(sve2_fmaxnmp_zpzz_h, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(sve2_fmaxnmp_zpzz_s, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(sve2_fmaxnmp_zpzz_d, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(sve2_fminnmp_zpzz_h, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(sve2_fminnmp_zpzz_s, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(sve2_fminnmp_zpzz_d, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(sve2_fmaxp_zpzz_h, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(sve2_fmaxp_zpzz_s, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(sve2_fmaxp_zpzz_d, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(sve2_fminp_zpzz_h, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(sve2_fminp_zpzz_s, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(sve2_fminp_zpzz_d, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(sve2_eor3, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve2_bcax, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_5(sve2_sqrdcmlah_zzzz_s, TCG_CALL_NO_RWG, DEF_HELPER_FLAGS_5(sve2_sqrdcmlah_zzzz_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_6(fmmla_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_6(fmmla_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_6(fmmla_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_6(fmmla_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(sve2_sqrdmlah_idx_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_5(sve2_cdot_idx_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve2_fcvtnt_sh, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(sve2_fcvtnt_ds, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(sve_bfcvtnt, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(sve2_fcvtlt_hs, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_5(sve2_fcvtlt_sd, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, fpst, i32) -DEF_HELPER_FLAGS_5(flogb_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(flogb_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(flogb_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(flogb_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_5(flogb_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) +DEF_HELPER_FLAGS_5(flogb_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_4(sve2_sqshl_zpzi_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) diff --git a/target/arm/tcg/sve_helper.c b/target/arm/tcg/sve_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/sve_helper.c +++ b/target/arm/tcg/sve_helper.c @@ -XXX,XX +XXX,XX @@ DO_ZPZZ_PAIR_D(sve2_sminp_zpzz_d, int64_t, DO_MIN) #define DO_ZPZZ_PAIR_FP(NAME, TYPE, H, OP) \ void HELPER(NAME)(void *vd, void *vn, void *vm, void *vg, \ - void *status, uint32_t desc) \ + float_status *status, uint32_t desc) \ { \ intptr_t i, opr_sz = simd_oprsz(desc); \ for (i = 0; i < opr_sz; ) { \ @@ -XXX,XX +XXX,XX @@ static TYPE NAME##_reduce(TYPE *data, float_status *status, uintptr_t n) \ return TYPE##_##FUNC(lo, hi, status); \ } \ } \ -uint64_t HELPER(NAME)(void *vn, void *vg, void *vs, uint32_t desc) \ +uint64_t HELPER(NAME)(void *vn, void *vg, float_status *s, uint32_t desc) \ { \ uintptr_t i, oprsz = simd_oprsz(desc), maxsz = simd_data(desc); \ TYPE data[sizeof(ARMVectorReg) / sizeof(TYPE)]; \ @@ -XXX,XX +XXX,XX @@ uint64_t HELPER(NAME)(void *vn, void *vg, void *vs, uint32_t desc) \ for (; i < maxsz; i += sizeof(TYPE)) { \ *(TYPE *)((void *)data + i) = IDENT; \ } \ - return NAME##_reduce(data, vs, maxsz / sizeof(TYPE)); \ + return NAME##_reduce(data, s, maxsz / sizeof(TYPE)); \ } DO_REDUCE(sve_faddv_h, float16, H1_2, add, float16_zero) @@ -XXX,XX +XXX,XX @@ DO_REDUCE(sve_fmaxv_d, float64, H1_8, max, float64_chs(float64_infinity)) #undef DO_REDUCE uint64_t HELPER(sve_fadda_h)(uint64_t nn, void *vm, void *vg, - void *status, uint32_t desc) + float_status *status, uint32_t desc) { intptr_t i = 0, opr_sz = simd_oprsz(desc); float16 result = nn; @@ -XXX,XX +XXX,XX @@ uint64_t HELPER(sve_fadda_h)(uint64_t nn, void *vm, void *vg, } uint64_t HELPER(sve_fadda_s)(uint64_t nn, void *vm, void *vg, - void *status, uint32_t desc) + float_status *status, uint32_t desc) { intptr_t i = 0, opr_sz = simd_oprsz(desc); float32 result = nn; @@ -XXX,XX +XXX,XX @@ uint64_t HELPER(sve_fadda_s)(uint64_t nn, void *vm, void *vg, } uint64_t HELPER(sve_fadda_d)(uint64_t nn, void *vm, void *vg, - void *status, uint32_t desc) + float_status *status, uint32_t desc) { intptr_t i = 0, opr_sz = simd_oprsz(desc) / 8; uint64_t *m = vm; @@ -XXX,XX +XXX,XX @@ uint64_t HELPER(sve_fadda_d)(uint64_t nn, void *vm, void *vg, */ #define DO_ZPZZ_FP(NAME, TYPE, H, OP) \ void HELPER(NAME)(void *vd, void *vn, void *vm, void *vg, \ - void *status, uint32_t desc) \ + float_status *status, uint32_t desc) \ { \ intptr_t i = simd_oprsz(desc); \ uint64_t *g = vg; \ @@ -XXX,XX +XXX,XX @@ DO_ZPZZ_FP(sve_fmulx_d, uint64_t, H1_8, helper_vfp_mulxd) */ #define DO_ZPZS_FP(NAME, TYPE, H, OP) \ void HELPER(NAME)(void *vd, void *vn, void *vg, uint64_t scalar, \ - void *status, uint32_t desc) \ + float_status *status, uint32_t desc) \ { \ intptr_t i = simd_oprsz(desc); \ uint64_t *g = vg; \ @@ -XXX,XX +XXX,XX @@ DO_ZPZS_FP(sve_fmins_d, float64, H1_8, float64_min) * With the extra float_status parameter. */ #define DO_ZPZ_FP(NAME, TYPE, H, OP) \ -void HELPER(NAME)(void *vd, void *vn, void *vg, void *status, uint32_t desc) \ +void HELPER(NAME)(void *vd, void *vn, void *vg, \ + float_status *status, uint32_t desc) \ { \ intptr_t i = simd_oprsz(desc); \ uint64_t *g = vg; \ @@ -XXX,XX +XXX,XX @@ static void do_fmla_zpzzz_h(void *vd, void *vn, void *vm, void *va, void *vg, } void HELPER(sve_fmla_zpzzz_h)(void *vd, void *vn, void *vm, void *va, - void *vg, void *status, uint32_t desc) + void *vg, float_status *status, uint32_t desc) { do_fmla_zpzzz_h(vd, vn, vm, va, vg, status, desc, 0, 0); } void HELPER(sve_fmls_zpzzz_h)(void *vd, void *vn, void *vm, void *va, - void *vg, void *status, uint32_t desc) + void *vg, float_status *status, uint32_t desc) { do_fmla_zpzzz_h(vd, vn, vm, va, vg, status, desc, 0x8000, 0); } void HELPER(sve_fnmla_zpzzz_h)(void *vd, void *vn, void *vm, void *va, - void *vg, void *status, uint32_t desc) + void *vg, float_status *status, uint32_t desc) { do_fmla_zpzzz_h(vd, vn, vm, va, vg, status, desc, 0x8000, 0x8000); } void HELPER(sve_fnmls_zpzzz_h)(void *vd, void *vn, void *vm, void *va, - void *vg, void *status, uint32_t desc) + void *vg, float_status *status, uint32_t desc) { do_fmla_zpzzz_h(vd, vn, vm, va, vg, status, desc, 0, 0x8000); } @@ -XXX,XX +XXX,XX @@ static void do_fmla_zpzzz_s(void *vd, void *vn, void *vm, void *va, void *vg, } void HELPER(sve_fmla_zpzzz_s)(void *vd, void *vn, void *vm, void *va, - void *vg, void *status, uint32_t desc) + void *vg, float_status *status, uint32_t desc) { do_fmla_zpzzz_s(vd, vn, vm, va, vg, status, desc, 0, 0); } void HELPER(sve_fmls_zpzzz_s)(void *vd, void *vn, void *vm, void *va, - void *vg, void *status, uint32_t desc) + void *vg, float_status *status, uint32_t desc) { do_fmla_zpzzz_s(vd, vn, vm, va, vg, status, desc, 0x80000000, 0); } void HELPER(sve_fnmla_zpzzz_s)(void *vd, void *vn, void *vm, void *va, - void *vg, void *status, uint32_t desc) + void *vg, float_status *status, uint32_t desc) { do_fmla_zpzzz_s(vd, vn, vm, va, vg, status, desc, 0x80000000, 0x80000000); } void HELPER(sve_fnmls_zpzzz_s)(void *vd, void *vn, void *vm, void *va, - void *vg, void *status, uint32_t desc) + void *vg, float_status *status, uint32_t desc) { do_fmla_zpzzz_s(vd, vn, vm, va, vg, status, desc, 0, 0x80000000); } @@ -XXX,XX +XXX,XX @@ static void do_fmla_zpzzz_d(void *vd, void *vn, void *vm, void *va, void *vg, } void HELPER(sve_fmla_zpzzz_d)(void *vd, void *vn, void *vm, void *va, - void *vg, void *status, uint32_t desc) + void *vg, float_status *status, uint32_t desc) { do_fmla_zpzzz_d(vd, vn, vm, va, vg, status, desc, 0, 0); } void HELPER(sve_fmls_zpzzz_d)(void *vd, void *vn, void *vm, void *va, - void *vg, void *status, uint32_t desc) + void *vg, float_status *status, uint32_t desc) { do_fmla_zpzzz_d(vd, vn, vm, va, vg, status, desc, INT64_MIN, 0); } void HELPER(sve_fnmla_zpzzz_d)(void *vd, void *vn, void *vm, void *va, - void *vg, void *status, uint32_t desc) + void *vg, float_status *status, uint32_t desc) { do_fmla_zpzzz_d(vd, vn, vm, va, vg, status, desc, INT64_MIN, INT64_MIN); } void HELPER(sve_fnmls_zpzzz_d)(void *vd, void *vn, void *vm, void *va, - void *vg, void *status, uint32_t desc) + void *vg, float_status *status, uint32_t desc) { do_fmla_zpzzz_d(vd, vn, vm, va, vg, status, desc, 0, INT64_MIN); } @@ -XXX,XX +XXX,XX @@ void HELPER(sve_fnmls_zpzzz_d)(void *vd, void *vn, void *vm, void *va, */ #define DO_FPCMP_PPZZ(NAME, TYPE, H, OP) \ void HELPER(NAME)(void *vd, void *vn, void *vm, void *vg, \ - void *status, uint32_t desc) \ + float_status *status, uint32_t desc) \ { \ intptr_t i = simd_oprsz(desc), j = (i - 1) >> 6; \ uint64_t *d = vd, *g = vg; \ @@ -XXX,XX +XXX,XX @@ DO_FPCMP_PPZZ_ALL(sve_facgt, DO_FACGT) */ #define DO_FPCMP_PPZ0(NAME, TYPE, H, OP) \ void HELPER(NAME)(void *vd, void *vn, void *vg, \ - void *status, uint32_t desc) \ + float_status *status, uint32_t desc) \ { \ intptr_t i = simd_oprsz(desc), j = (i - 1) >> 6; \ uint64_t *d = vd, *g = vg; \ @@ -XXX,XX +XXX,XX @@ DO_FPCMP_PPZ0_ALL(sve_fcmne0, DO_FCMNE) /* FP Trig Multiply-Add. */ -void HELPER(sve_ftmad_h)(void *vd, void *vn, void *vm, void *vs, uint32_t desc) +void HELPER(sve_ftmad_h)(void *vd, void *vn, void *vm, + float_status *s, uint32_t desc) { static const float16 coeff[16] = { 0x3c00, 0xb155, 0x2030, 0x0000, 0x0000, 0x0000, 0x0000, 0x0000, @@ -XXX,XX +XXX,XX @@ void HELPER(sve_ftmad_h)(void *vd, void *vn, void *vm, void *vs, uint32_t desc) mm = float16_abs(mm); xx += 8; } - d[i] = float16_muladd(n[i], mm, coeff[xx], 0, vs); + d[i] = float16_muladd(n[i], mm, coeff[xx], 0, s); } } -void HELPER(sve_ftmad_s)(void *vd, void *vn, void *vm, void *vs, uint32_t desc) +void HELPER(sve_ftmad_s)(void *vd, void *vn, void *vm, + float_status *s, uint32_t desc) { static const float32 coeff[16] = { 0x3f800000, 0xbe2aaaab, 0x3c088886, 0xb95008b9, @@ -XXX,XX +XXX,XX @@ void HELPER(sve_ftmad_s)(void *vd, void *vn, void *vm, void *vs, uint32_t desc) mm = float32_abs(mm); xx += 8; } - d[i] = float32_muladd(n[i], mm, coeff[xx], 0, vs); + d[i] = float32_muladd(n[i], mm, coeff[xx], 0, s); } } -void HELPER(sve_ftmad_d)(void *vd, void *vn, void *vm, void *vs, uint32_t desc) +void HELPER(sve_ftmad_d)(void *vd, void *vn, void *vm, + float_status *s, uint32_t desc) { static const float64 coeff[16] = { 0x3ff0000000000000ull, 0xbfc5555555555543ull, @@ -XXX,XX +XXX,XX @@ void HELPER(sve_ftmad_d)(void *vd, void *vn, void *vm, void *vs, uint32_t desc) mm = float64_abs(mm); xx += 8; } - d[i] = float64_muladd(n[i], mm, coeff[xx], 0, vs); + d[i] = float64_muladd(n[i], mm, coeff[xx], 0, s); } } @@ -XXX,XX +XXX,XX @@ void HELPER(sve_ftmad_d)(void *vd, void *vn, void *vm, void *vs, uint32_t desc) */ void HELPER(sve_fcadd_h)(void *vd, void *vn, void *vm, void *vg, - void *vs, uint32_t desc) + float_status *s, uint32_t desc) { intptr_t j, i = simd_oprsz(desc); uint64_t *g = vg; @@ -XXX,XX +XXX,XX @@ void HELPER(sve_fcadd_h)(void *vd, void *vn, void *vm, void *vg, e3 = *(float16 *)(vm + H1_2(i)) ^ neg_imag; if (likely((pg >> (i & 63)) & 1)) { - *(float16 *)(vd + H1_2(i)) = float16_add(e0, e1, vs); + *(float16 *)(vd + H1_2(i)) = float16_add(e0, e1, s); } if (likely((pg >> (j & 63)) & 1)) { - *(float16 *)(vd + H1_2(j)) = float16_add(e2, e3, vs); + *(float16 *)(vd + H1_2(j)) = float16_add(e2, e3, s); } } while (i & 63); } while (i != 0); } void HELPER(sve_fcadd_s)(void *vd, void *vn, void *vm, void *vg, - void *vs, uint32_t desc) + float_status *s, uint32_t desc) { intptr_t j, i = simd_oprsz(desc); uint64_t *g = vg; @@ -XXX,XX +XXX,XX @@ void HELPER(sve_fcadd_s)(void *vd, void *vn, void *vm, void *vg, e3 = *(float32 *)(vm + H1_2(i)) ^ neg_imag; if (likely((pg >> (i & 63)) & 1)) { - *(float32 *)(vd + H1_2(i)) = float32_add(e0, e1, vs); + *(float32 *)(vd + H1_2(i)) = float32_add(e0, e1, s); } if (likely((pg >> (j & 63)) & 1)) { - *(float32 *)(vd + H1_2(j)) = float32_add(e2, e3, vs); + *(float32 *)(vd + H1_2(j)) = float32_add(e2, e3, s); } } while (i & 63); } while (i != 0); } void HELPER(sve_fcadd_d)(void *vd, void *vn, void *vm, void *vg, - void *vs, uint32_t desc) + float_status *s, uint32_t desc) { intptr_t j, i = simd_oprsz(desc); uint64_t *g = vg; @@ -XXX,XX +XXX,XX @@ void HELPER(sve_fcadd_d)(void *vd, void *vn, void *vm, void *vg, e3 = *(float64 *)(vm + H1_2(i)) ^ neg_imag; if (likely((pg >> (i & 63)) & 1)) { - *(float64 *)(vd + H1_2(i)) = float64_add(e0, e1, vs); + *(float64 *)(vd + H1_2(i)) = float64_add(e0, e1, s); } if (likely((pg >> (j & 63)) & 1)) { - *(float64 *)(vd + H1_2(j)) = float64_add(e2, e3, vs); + *(float64 *)(vd + H1_2(j)) = float64_add(e2, e3, s); } } while (i & 63); } while (i != 0); @@ -XXX,XX +XXX,XX @@ void HELPER(sve_fcadd_d)(void *vd, void *vn, void *vm, void *vg, */ void HELPER(sve_fcmla_zpzzz_h)(void *vd, void *vn, void *vm, void *va, - void *vg, void *status, uint32_t desc) + void *vg, float_status *status, uint32_t desc) { intptr_t j, i = simd_oprsz(desc); unsigned rot = simd_data(desc); @@ -XXX,XX +XXX,XX @@ void HELPER(sve_fcmla_zpzzz_h)(void *vd, void *vn, void *vm, void *va, } void HELPER(sve_fcmla_zpzzz_s)(void *vd, void *vn, void *vm, void *va, - void *vg, void *status, uint32_t desc) + void *vg, float_status *status, uint32_t desc) { intptr_t j, i = simd_oprsz(desc); unsigned rot = simd_data(desc); @@ -XXX,XX +XXX,XX @@ void HELPER(sve_fcmla_zpzzz_s)(void *vd, void *vn, void *vm, void *va, } void HELPER(sve_fcmla_zpzzz_d)(void *vd, void *vn, void *vm, void *va, - void *vg, void *status, uint32_t desc) + void *vg, float_status *status, uint32_t desc) { intptr_t j, i = simd_oprsz(desc); unsigned rot = simd_data(desc); @@ -XXX,XX +XXX,XX @@ void HELPER(sve2_xar_s)(void *vd, void *vn, void *vm, uint32_t desc) } void HELPER(fmmla_s)(void *vd, void *vn, void *vm, void *va, - void *status, uint32_t desc) + float_status *status, uint32_t desc) { intptr_t s, opr_sz = simd_oprsz(desc) / (sizeof(float32) * 4); @@ -XXX,XX +XXX,XX @@ void HELPER(fmmla_s)(void *vd, void *vn, void *vm, void *va, } void HELPER(fmmla_d)(void *vd, void *vn, void *vm, void *va, - void *status, uint32_t desc) + float_status *status, uint32_t desc) { intptr_t s, opr_sz = simd_oprsz(desc) / (sizeof(float64) * 4); @@ -XXX,XX +XXX,XX @@ void HELPER(fmmla_d)(void *vd, void *vn, void *vm, void *va, } #define DO_FCVTNT(NAME, TYPEW, TYPEN, HW, HN, OP) \ -void HELPER(NAME)(void *vd, void *vn, void *vg, void *status, uint32_t desc) \ +void HELPER(NAME)(void *vd, void *vn, void *vg, \ + float_status *status, uint32_t desc) \ { \ intptr_t i = simd_oprsz(desc); \ uint64_t *g = vg; \ @@ -XXX,XX +XXX,XX @@ DO_FCVTNT(sve2_fcvtnt_sh, uint32_t, uint16_t, H1_4, H1_2, sve_f32_to_f16) DO_FCVTNT(sve2_fcvtnt_ds, uint64_t, uint32_t, H1_8, H1_4, float64_to_float32) #define DO_FCVTLT(NAME, TYPEW, TYPEN, HW, HN, OP) \ -void HELPER(NAME)(void *vd, void *vn, void *vg, void *status, uint32_t desc) \ +void HELPER(NAME)(void *vd, void *vn, void *vg, \ + float_status *status, uint32_t desc) \ { \ intptr_t i = simd_oprsz(desc); \ uint64_t *g = vg; \ -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Message-id: 20241206031224.78525-8-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/helper-sme.h | 4 ++-- target/arm/tcg/sme_helper.c | 8 ++++---- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/target/arm/tcg/helper-sme.h b/target/arm/tcg/helper-sme.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/helper-sme.h +++ b/target/arm/tcg/helper-sme.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_5(sme_addva_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_7(sme_fmopa_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, ptr, env, i32) DEF_HELPER_FLAGS_7(sme_fmopa_s, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_7(sme_fmopa_d, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_7(sme_bfmopa, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, ptr, env, i32) DEF_HELPER_FLAGS_6(sme_smopa_s, TCG_CALL_NO_RWG, diff --git a/target/arm/tcg/sme_helper.c b/target/arm/tcg/sme_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/sme_helper.c +++ b/target/arm/tcg/sme_helper.c @@ -XXX,XX +XXX,XX @@ void HELPER(sme_addva_d)(void *vzda, void *vzn, void *vpn, } void HELPER(sme_fmopa_s)(void *vza, void *vzn, void *vzm, void *vpn, - void *vpm, void *vst, uint32_t desc) + void *vpm, float_status *fpst_in, uint32_t desc) { intptr_t row, col, oprsz = simd_maxsz(desc); uint32_t neg = simd_data(desc) << 31; @@ -XXX,XX +XXX,XX @@ void HELPER(sme_fmopa_s)(void *vza, void *vzn, void *vzm, void *vpn, * update the cumulative fp exception status. It also produces * default nans. */ - fpst = *(float_status *)vst; + fpst = *fpst_in; set_default_nan_mode(true, &fpst); for (row = 0; row < oprsz; ) { @@ -XXX,XX +XXX,XX @@ void HELPER(sme_fmopa_s)(void *vza, void *vzn, void *vzm, void *vpn, } void HELPER(sme_fmopa_d)(void *vza, void *vzn, void *vzm, void *vpn, - void *vpm, void *vst, uint32_t desc) + void *vpm, float_status *fpst_in, uint32_t desc) { intptr_t row, col, oprsz = simd_oprsz(desc) / 8; uint64_t neg = (uint64_t)simd_data(desc) << 63; uint64_t *za = vza, *zn = vzn, *zm = vzm; uint8_t *pn = vpn, *pm = vpm; - float_status fpst = *(float_status *)vst; + float_status fpst = *fpst_in; set_default_nan_mode(true, &fpst); -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Allow the helpers to receive CPUARMState* directly instead of via void*. Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Message-id: 20241206031224.78525-9-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/helper.h | 12 ++++++------ target/arm/tcg/helper-a64.h | 2 +- target/arm/tcg/vec_helper.c | 21 +++++++-------------- 3 files changed, 14 insertions(+), 21 deletions(-) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_5(gvec_suqadd_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_fmlal_a32, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, env, i32) DEF_HELPER_FLAGS_5(gvec_fmlal_a64, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, env, i32) DEF_HELPER_FLAGS_5(gvec_fmlal_idx_a32, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, env, i32) DEF_HELPER_FLAGS_5(gvec_fmlal_idx_a64, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, env, i32) DEF_HELPER_FLAGS_2(frint32_s, TCG_CALL_NO_RWG, f32, f32, fpst) DEF_HELPER_FLAGS_2(frint64_s, TCG_CALL_NO_RWG, f32, f32, fpst) @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_4(sve2_sqrdmulh_idx_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_6(sve2_fmlal_zzzw_s, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, env, i32) DEF_HELPER_FLAGS_6(sve2_fmlal_zzxw_s, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, ptr, i32) + void, ptr, ptr, ptr, ptr, env, i32) DEF_HELPER_FLAGS_4(gvec_xar_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) diff --git a/target/arm/tcg/helper-a64.h b/target/arm/tcg/helper-a64.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/helper-a64.h +++ b/target/arm/tcg/helper-a64.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_3(vfp_cmps_a64, i64, f32, f32, fpst) DEF_HELPER_3(vfp_cmpes_a64, i64, f32, f32, fpst) DEF_HELPER_3(vfp_cmpd_a64, i64, f64, f64, fpst) DEF_HELPER_3(vfp_cmped_a64, i64, f64, f64, fpst) -DEF_HELPER_FLAGS_4(simd_tblx, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(simd_tblx, TCG_CALL_NO_RWG, void, ptr, ptr, env, i32) DEF_HELPER_FLAGS_3(vfp_mulxs, TCG_CALL_NO_RWG, f32, f32, f32, fpst) DEF_HELPER_FLAGS_3(vfp_mulxd, TCG_CALL_NO_RWG, f64, f64, f64, fpst) DEF_HELPER_FLAGS_3(neon_ceq_f64, TCG_CALL_NO_RWG, i64, i64, i64, fpst) diff --git a/target/arm/tcg/vec_helper.c b/target/arm/tcg/vec_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/vec_helper.c +++ b/target/arm/tcg/vec_helper.c @@ -XXX,XX +XXX,XX @@ static void do_fmlal(float32 *d, void *vn, void *vm, float_status *fpst, } void HELPER(gvec_fmlal_a32)(void *vd, void *vn, void *vm, - void *venv, uint32_t desc) + CPUARMState *env, uint32_t desc) { - CPUARMState *env = venv; do_fmlal(vd, vn, vm, &env->vfp.standard_fp_status, desc, get_flush_inputs_to_zero(&env->vfp.fp_status_f16)); } void HELPER(gvec_fmlal_a64)(void *vd, void *vn, void *vm, - void *venv, uint32_t desc) + CPUARMState *env, uint32_t desc) { - CPUARMState *env = venv; do_fmlal(vd, vn, vm, &env->vfp.fp_status, desc, get_flush_inputs_to_zero(&env->vfp.fp_status_f16)); } void HELPER(sve2_fmlal_zzzw_s)(void *vd, void *vn, void *vm, void *va, - void *venv, uint32_t desc) + CPUARMState *env, uint32_t desc) { intptr_t i, oprsz = simd_oprsz(desc); uint16_t negn = extract32(desc, SIMD_DATA_SHIFT, 1) << 15; intptr_t sel = extract32(desc, SIMD_DATA_SHIFT + 1, 1) * sizeof(float16); - CPUARMState *env = venv; float_status *status = &env->vfp.fp_status; bool fz16 = get_flush_inputs_to_zero(&env->vfp.fp_status_f16); @@ -XXX,XX +XXX,XX @@ static void do_fmlal_idx(float32 *d, void *vn, void *vm, float_status *fpst, } void HELPER(gvec_fmlal_idx_a32)(void *vd, void *vn, void *vm, - void *venv, uint32_t desc) + CPUARMState *env, uint32_t desc) { - CPUARMState *env = venv; do_fmlal_idx(vd, vn, vm, &env->vfp.standard_fp_status, desc, get_flush_inputs_to_zero(&env->vfp.fp_status_f16)); } void HELPER(gvec_fmlal_idx_a64)(void *vd, void *vn, void *vm, - void *venv, uint32_t desc) + CPUARMState *env, uint32_t desc) { - CPUARMState *env = venv; do_fmlal_idx(vd, vn, vm, &env->vfp.fp_status, desc, get_flush_inputs_to_zero(&env->vfp.fp_status_f16)); } void HELPER(sve2_fmlal_zzxw_s)(void *vd, void *vn, void *vm, void *va, - void *venv, uint32_t desc) + CPUARMState *env, uint32_t desc) { intptr_t i, j, oprsz = simd_oprsz(desc); uint16_t negn = extract32(desc, SIMD_DATA_SHIFT, 1) << 15; intptr_t sel = extract32(desc, SIMD_DATA_SHIFT + 1, 1) * sizeof(float16); intptr_t idx = extract32(desc, SIMD_DATA_SHIFT + 2, 3) * sizeof(float16); - CPUARMState *env = venv; float_status *status = &env->vfp.fp_status; bool fz16 = get_flush_inputs_to_zero(&env->vfp.fp_status_f16); @@ -XXX,XX +XXX,XX @@ DO_VRINT_RMODE(gvec_vrint_rm_s, helper_rints, uint32_t) #undef DO_VRINT_RMODE #ifdef TARGET_AARCH64 -void HELPER(simd_tblx)(void *vd, void *vm, void *venv, uint32_t desc) +void HELPER(simd_tblx)(void *vd, void *vm, CPUARMState *env, uint32_t desc) { const uint8_t *indices = vm; - CPUARMState *env = venv; size_t oprsz = simd_oprsz(desc); uint32_t rn = extract32(desc, SIMD_DATA_SHIFT, 5); bool is_tbx = extract32(desc, SIMD_DATA_SHIFT + 5, 1); -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Message-id: 20241206031224.78525-10-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/helper.h | 56 ++++++++++++++++++------------------ target/arm/tcg/neon_helper.c | 6 ++-- 2 files changed, 30 insertions(+), 32 deletions(-) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_3(neon_qrshl_u32, i32, env, i32, i32) DEF_HELPER_3(neon_qrshl_s32, i32, env, i32, i32) DEF_HELPER_3(neon_qrshl_u64, i64, env, i64, i64) DEF_HELPER_3(neon_qrshl_s64, i64, env, i64, i64) -DEF_HELPER_FLAGS_5(neon_sqshl_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(neon_sqshl_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(neon_sqshl_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(neon_sqshl_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(neon_uqshl_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(neon_uqshl_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(neon_uqshl_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(neon_uqshl_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(neon_sqrshl_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(neon_sqrshl_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(neon_sqrshl_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(neon_sqrshl_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(neon_uqrshl_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(neon_uqrshl_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(neon_uqrshl_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(neon_uqrshl_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(neon_sqshli_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(neon_sqshli_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(neon_sqshli_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(neon_sqshli_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(neon_uqshli_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(neon_uqshli_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(neon_uqshli_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(neon_uqshli_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(neon_sqshlui_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(neon_sqshlui_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(neon_sqshlui_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(neon_sqshlui_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(neon_sqshl_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, env, i32) +DEF_HELPER_FLAGS_5(neon_sqshl_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, env, i32) +DEF_HELPER_FLAGS_5(neon_sqshl_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, env, i32) +DEF_HELPER_FLAGS_5(neon_sqshl_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, env, i32) +DEF_HELPER_FLAGS_5(neon_uqshl_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, env, i32) +DEF_HELPER_FLAGS_5(neon_uqshl_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, env, i32) +DEF_HELPER_FLAGS_5(neon_uqshl_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, env, i32) +DEF_HELPER_FLAGS_5(neon_uqshl_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, env, i32) +DEF_HELPER_FLAGS_5(neon_sqrshl_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, env, i32) +DEF_HELPER_FLAGS_5(neon_sqrshl_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, env, i32) +DEF_HELPER_FLAGS_5(neon_sqrshl_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, env, i32) +DEF_HELPER_FLAGS_5(neon_sqrshl_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, env, i32) +DEF_HELPER_FLAGS_5(neon_uqrshl_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, env, i32) +DEF_HELPER_FLAGS_5(neon_uqrshl_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, env, i32) +DEF_HELPER_FLAGS_5(neon_uqrshl_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, env, i32) +DEF_HELPER_FLAGS_5(neon_uqrshl_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, env, i32) +DEF_HELPER_FLAGS_4(neon_sqshli_b, TCG_CALL_NO_RWG, void, ptr, ptr, env, i32) +DEF_HELPER_FLAGS_4(neon_sqshli_h, TCG_CALL_NO_RWG, void, ptr, ptr, env, i32) +DEF_HELPER_FLAGS_4(neon_sqshli_s, TCG_CALL_NO_RWG, void, ptr, ptr, env, i32) +DEF_HELPER_FLAGS_4(neon_sqshli_d, TCG_CALL_NO_RWG, void, ptr, ptr, env, i32) +DEF_HELPER_FLAGS_4(neon_uqshli_b, TCG_CALL_NO_RWG, void, ptr, ptr, env, i32) +DEF_HELPER_FLAGS_4(neon_uqshli_h, TCG_CALL_NO_RWG, void, ptr, ptr, env, i32) +DEF_HELPER_FLAGS_4(neon_uqshli_s, TCG_CALL_NO_RWG, void, ptr, ptr, env, i32) +DEF_HELPER_FLAGS_4(neon_uqshli_d, TCG_CALL_NO_RWG, void, ptr, ptr, env, i32) +DEF_HELPER_FLAGS_4(neon_sqshlui_b, TCG_CALL_NO_RWG, void, ptr, ptr, env, i32) +DEF_HELPER_FLAGS_4(neon_sqshlui_h, TCG_CALL_NO_RWG, void, ptr, ptr, env, i32) +DEF_HELPER_FLAGS_4(neon_sqshlui_s, TCG_CALL_NO_RWG, void, ptr, ptr, env, i32) +DEF_HELPER_FLAGS_4(neon_sqshlui_d, TCG_CALL_NO_RWG, void, ptr, ptr, env, i32) DEF_HELPER_FLAGS_4(gvec_srshl_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_srshl_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) diff --git a/target/arm/tcg/neon_helper.c b/target/arm/tcg/neon_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/neon_helper.c +++ b/target/arm/tcg/neon_helper.c @@ -XXX,XX +XXX,XX @@ void HELPER(name)(void *vd, void *vn, void *vm, uint32_t desc) \ } #define NEON_GVEC_VOP2_ENV(name, vtype) \ -void HELPER(name)(void *vd, void *vn, void *vm, void *venv, uint32_t desc) \ +void HELPER(name)(void *vd, void *vn, void *vm, CPUARMState *env, uint32_t desc) \ { \ intptr_t i, opr_sz = simd_oprsz(desc); \ vtype *d = vd, *n = vn, *m = vm; \ - CPUARMState *env = venv; \ for (i = 0; i < opr_sz / sizeof(vtype); i++) { \ NEON_FN(d[i], n[i], m[i]); \ } \ @@ -XXX,XX +XXX,XX @@ void HELPER(name)(void *vd, void *vn, void *vm, void *venv, uint32_t desc) \ } #define NEON_GVEC_VOP2i_ENV(name, vtype) \ -void HELPER(name)(void *vd, void *vn, void *venv, uint32_t desc) \ +void HELPER(name)(void *vd, void *vn, CPUARMState *env, uint32_t desc) \ { \ intptr_t i, opr_sz = simd_oprsz(desc); \ int imm = simd_data(desc); \ vtype *d = vd, *n = vn; \ - CPUARMState *env = venv; \ for (i = 0; i < opr_sz / sizeof(vtype); i++) { \ NEON_FN(d[i], n[i], imm); \ } \ -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Pass float_status not env to match other functions. Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Message-id: 20241206031952.78776-2-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/helper-a64.h | 2 +- target/arm/tcg/helper-a64.c | 3 +-- target/arm/tcg/translate-a64.c | 2 +- 3 files changed, 3 insertions(+), 4 deletions(-) diff --git a/target/arm/tcg/helper-a64.h b/target/arm/tcg/helper-a64.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/helper-a64.h +++ b/target/arm/tcg/helper-a64.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_3(rsqrtsf_f64, TCG_CALL_NO_RWG, f64, f64, f64, fpst) DEF_HELPER_FLAGS_2(frecpx_f64, TCG_CALL_NO_RWG, f64, f64, fpst) DEF_HELPER_FLAGS_2(frecpx_f32, TCG_CALL_NO_RWG, f32, f32, fpst) DEF_HELPER_FLAGS_2(frecpx_f16, TCG_CALL_NO_RWG, f16, f16, fpst) -DEF_HELPER_FLAGS_2(fcvtx_f64_to_f32, TCG_CALL_NO_RWG, f32, f64, env) +DEF_HELPER_FLAGS_2(fcvtx_f64_to_f32, TCG_CALL_NO_RWG, f32, f64, fpst) DEF_HELPER_FLAGS_3(crc32_64, TCG_CALL_NO_RWG_SE, i64, i64, i64, i32) DEF_HELPER_FLAGS_3(crc32c_64, TCG_CALL_NO_RWG_SE, i64, i64, i64, i32) DEF_HELPER_FLAGS_3(advsimd_maxh, TCG_CALL_NO_RWG, f16, f16, f16, fpst) diff --git a/target/arm/tcg/helper-a64.c b/target/arm/tcg/helper-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/helper-a64.c +++ b/target/arm/tcg/helper-a64.c @@ -XXX,XX +XXX,XX @@ float64 HELPER(frecpx_f64)(float64 a, float_status *fpst) } } -float32 HELPER(fcvtx_f64_to_f32)(float64 a, CPUARMState *env) +float32 HELPER(fcvtx_f64_to_f32)(float64 a, float_status *fpst) { float32 r; - float_status *fpst = &env->vfp.fp_status; int old = get_float_rounding_mode(fpst); set_float_rounding_mode(float_round_to_odd, fpst); diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static void gen_fcvtxn_sd(TCGv_i64 d, TCGv_i64 n) * with von Neumann rounding (round to odd) */ TCGv_i32 tmp = tcg_temp_new_i32(); - gen_helper_fcvtx_f64_to_f32(tmp, n, tcg_env); + gen_helper_fcvtx_f64_to_f32(tmp, n, fpstatus_ptr(FPST_FPCR)); tcg_gen_extu_i32_i64(d, tmp); } -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Pass float_status not env to match other functions. Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Message-id: 20241206031952.78776-3-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/helper.h | 4 ++-- target/arm/tcg/translate-a64.c | 15 ++++++++++----- target/arm/tcg/translate-vfp.c | 4 ++-- target/arm/vfp_helper.c | 8 ++++---- 4 files changed, 18 insertions(+), 13 deletions(-) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_3(vfp_cmpeh, void, f16, f16, env) DEF_HELPER_3(vfp_cmpes, void, f32, f32, env) DEF_HELPER_3(vfp_cmped, void, f64, f64, env) -DEF_HELPER_2(vfp_fcvtds, f64, f32, env) -DEF_HELPER_2(vfp_fcvtsd, f32, f64, env) +DEF_HELPER_2(vfp_fcvtds, f64, f32, fpst) +DEF_HELPER_2(vfp_fcvtsd, f32, f64, fpst) DEF_HELPER_FLAGS_2(bfcvt, TCG_CALL_NO_RWG, i32, f32, fpst) DEF_HELPER_FLAGS_2(bfcvt_pair, TCG_CALL_NO_RWG, i32, i64, fpst) diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static bool trans_FCVT_s_ds(DisasContext *s, arg_rr *a) if (fp_access_check(s)) { TCGv_i32 tcg_rn = read_fp_sreg(s, a->rn); TCGv_i64 tcg_rd = tcg_temp_new_i64(); + TCGv_ptr fpst = fpstatus_ptr(FPST_FPCR); - gen_helper_vfp_fcvtds(tcg_rd, tcg_rn, tcg_env); + gen_helper_vfp_fcvtds(tcg_rd, tcg_rn, fpst); write_fp_dreg(s, a->rd, tcg_rd); } return true; @@ -XXX,XX +XXX,XX @@ static bool trans_FCVT_s_sd(DisasContext *s, arg_rr *a) if (fp_access_check(s)) { TCGv_i64 tcg_rn = read_fp_dreg(s, a->rn); TCGv_i32 tcg_rd = tcg_temp_new_i32(); + TCGv_ptr fpst = fpstatus_ptr(FPST_FPCR); - gen_helper_vfp_fcvtsd(tcg_rd, tcg_rn, tcg_env); + gen_helper_vfp_fcvtsd(tcg_rd, tcg_rn, fpst); write_fp_sreg(s, a->rd, tcg_rd); } return true; @@ -XXX,XX +XXX,XX @@ static void gen_fcvtn_hs(TCGv_i64 d, TCGv_i64 n) static void gen_fcvtn_sd(TCGv_i64 d, TCGv_i64 n) { TCGv_i32 tmp = tcg_temp_new_i32(); - gen_helper_vfp_fcvtsd(tmp, n, tcg_env); + TCGv_ptr fpst = fpstatus_ptr(FPST_FPCR); + + gen_helper_vfp_fcvtsd(tmp, n, fpst); tcg_gen_extu_i32_i64(d, tmp); } @@ -XXX,XX +XXX,XX @@ static bool trans_FCVTL_v(DisasContext *s, arg_qrr_e *a) * The only instruction like this is FCVTL. */ int pass; + TCGv_ptr fpst; if (!fp_access_check(s)) { return true; } + fpst = fpstatus_ptr(FPST_FPCR); if (a->esz == MO_64) { /* 32 -> 64 bit fp conversion */ TCGv_i64 tcg_res[2]; @@ -XXX,XX +XXX,XX @@ static bool trans_FCVTL_v(DisasContext *s, arg_qrr_e *a) for (pass = 0; pass < 2; pass++) { tcg_res[pass] = tcg_temp_new_i64(); read_vec_element_i32(s, tcg_op, a->rn, srcelt + pass, MO_32); - gen_helper_vfp_fcvtds(tcg_res[pass], tcg_op, tcg_env); + gen_helper_vfp_fcvtds(tcg_res[pass], tcg_op, fpst); } for (pass = 0; pass < 2; pass++) { write_vec_element(s, tcg_res[pass], a->rd, pass, MO_64); @@ -XXX,XX +XXX,XX @@ static bool trans_FCVTL_v(DisasContext *s, arg_qrr_e *a) /* 16 -> 32 bit fp conversion */ int srcelt = a->q ? 4 : 0; TCGv_i32 tcg_res[4]; - TCGv_ptr fpst = fpstatus_ptr(FPST_FPCR); TCGv_i32 ahp = get_ahp_flag(); for (pass = 0; pass < 4; pass++) { diff --git a/target/arm/tcg/translate-vfp.c b/target/arm/tcg/translate-vfp.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-vfp.c +++ b/target/arm/tcg/translate-vfp.c @@ -XXX,XX +XXX,XX @@ static bool trans_VCVT_sp(DisasContext *s, arg_VCVT_sp *a) vm = tcg_temp_new_i32(); vd = tcg_temp_new_i64(); vfp_load_reg32(vm, a->vm); - gen_helper_vfp_fcvtds(vd, vm, tcg_env); + gen_helper_vfp_fcvtds(vd, vm, fpstatus_ptr(FPST_FPCR)); vfp_store_reg64(vd, a->vd); return true; } @@ -XXX,XX +XXX,XX @@ static bool trans_VCVT_dp(DisasContext *s, arg_VCVT_dp *a) vd = tcg_temp_new_i32(); vm = tcg_temp_new_i64(); vfp_load_reg64(vm, a->vm); - gen_helper_vfp_fcvtsd(vd, vm, tcg_env); + gen_helper_vfp_fcvtsd(vd, vm, fpstatus_ptr(FPST_FPCR)); vfp_store_reg32(vd, a->vd); return true; } diff --git a/target/arm/vfp_helper.c b/target/arm/vfp_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vfp_helper.c +++ b/target/arm/vfp_helper.c @@ -XXX,XX +XXX,XX @@ FLOAT_CONVS(ui, d, float64, 64, u) #undef FLOAT_CONVS /* floating point conversion */ -float64 VFP_HELPER(fcvtd, s)(float32 x, CPUARMState *env) +float64 VFP_HELPER(fcvtd, s)(float32 x, float_status *status) { - return float32_to_float64(x, &env->vfp.fp_status); + return float32_to_float64(x, status); } -float32 VFP_HELPER(fcvts, d)(float64 x, CPUARMState *env) +float32 VFP_HELPER(fcvts, d)(float64 x, float_status *status) { - return float64_to_float32(x, &env->vfp.fp_status); + return float64_to_float32(x, status); } uint32_t HELPER(bfcvt)(float32 x, float_status *status) -- 2.34.1 FEAT_XS introduces a set of new TLBI maintenance instructions with an "nXS" qualifier. These behave like the stardard ones except that they do not wait for memory accesses with the XS attribute to complete. They have an interaction with the fine-grained-trap handling: the FGT bits that a hypervisor can use to trap TLBI maintenance instructions normally trap also the nXS variants, but the hypervisor can elect to not trap the nXS variants by setting HCRX_EL2.FGTnXS to 1. Add support to our FGT mechanism for these TLBI bits. For each TLBI-trapping FGT bit we define, for example: * FGT_TLBIVAE1 -- the same value we do at present for the normal variant of the insn * FGT_TLBIVAE1NXS -- for the nXS qualified insn; the value of this enum has an NXS bit ORed into it In access_check_cp_reg() we can then ignore the trap bit for an access where ri->fgt has the NXS bit set and HCRX_EL2.FGTnXS is 1. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211144440.2700268-2-peter.maydell@linaro.org --- target/arm/cpregs.h | 72 ++++++++++++++++++++++---------------- target/arm/cpu-features.h | 5 +++ target/arm/helper.c | 5 ++- target/arm/tcg/op_helper.c | 11 +++++- 4 files changed, 61 insertions(+), 32 deletions(-) diff --git a/target/arm/cpregs.h b/target/arm/cpregs.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpregs.h +++ b/target/arm/cpregs.h @@ -XXX,XX +XXX,XX @@ FIELD(HDFGWTR_EL2, NBRBCTL, 60, 1) FIELD(HDFGWTR_EL2, NBRBDATA, 61, 1) FIELD(HDFGWTR_EL2, NPMSNEVFR_EL1, 62, 1) +FIELD(FGT, NXS, 13, 1) /* Honour HCR_EL2.FGTnXS to suppress FGT */ /* Which fine-grained trap bit register to check, if any */ FIELD(FGT, TYPE, 10, 3) FIELD(FGT, REV, 9, 1) /* Is bit sense reversed? */ @@ -XXX,XX +XXX,XX @@ FIELD(FGT, BITPOS, 0, 6) /* Bit position within the uint64_t */ #define DO_REV_BIT(REG, BITNAME) \ FGT_##BITNAME = FGT_##REG | FGT_REV | R_##REG##_EL2_##BITNAME##_SHIFT +/* + * The FGT bits for TLBI maintenance instructions accessible at EL1 always + * affect the "normal" TLBI insns; they affect the corresponding TLBI insns + * with the nXS qualifier only if HCRX_EL2.FGTnXS is 0. We define e.g. + * FGT_TLBIVAE1 to use for the normal insn, and FGT_TLBIVAE1NXS to use + * for the nXS qualified insn. + */ +#define DO_TLBINXS_BIT(REG, BITNAME) \ + FGT_##BITNAME = FGT_##REG | R_##REG##_EL2_##BITNAME##_SHIFT, \ + FGT_##BITNAME##NXS = FGT_##BITNAME | R_FGT_NXS_MASK + typedef enum FGTBit { /* * These bits tell us which register arrays to use: @@ -XXX,XX +XXX,XX @@ typedef enum FGTBit { DO_BIT(HFGITR, ATS1E0W), DO_BIT(HFGITR, ATS1E1RP), DO_BIT(HFGITR, ATS1E1WP), - DO_BIT(HFGITR, TLBIVMALLE1OS), - DO_BIT(HFGITR, TLBIVAE1OS), - DO_BIT(HFGITR, TLBIASIDE1OS), - DO_BIT(HFGITR, TLBIVAAE1OS), - DO_BIT(HFGITR, TLBIVALE1OS), - DO_BIT(HFGITR, TLBIVAALE1OS), - DO_BIT(HFGITR, TLBIRVAE1OS), - DO_BIT(HFGITR, TLBIRVAAE1OS), - DO_BIT(HFGITR, TLBIRVALE1OS), - DO_BIT(HFGITR, TLBIRVAALE1OS), - DO_BIT(HFGITR, TLBIVMALLE1IS), - DO_BIT(HFGITR, TLBIVAE1IS), - DO_BIT(HFGITR, TLBIASIDE1IS), - DO_BIT(HFGITR, TLBIVAAE1IS), - DO_BIT(HFGITR, TLBIVALE1IS), - DO_BIT(HFGITR, TLBIVAALE1IS), - DO_BIT(HFGITR, TLBIRVAE1IS), - DO_BIT(HFGITR, TLBIRVAAE1IS), - DO_BIT(HFGITR, TLBIRVALE1IS), - DO_BIT(HFGITR, TLBIRVAALE1IS), - DO_BIT(HFGITR, TLBIRVAE1), - DO_BIT(HFGITR, TLBIRVAAE1), - DO_BIT(HFGITR, TLBIRVALE1), - DO_BIT(HFGITR, TLBIRVAALE1), - DO_BIT(HFGITR, TLBIVMALLE1), - DO_BIT(HFGITR, TLBIVAE1), - DO_BIT(HFGITR, TLBIASIDE1), - DO_BIT(HFGITR, TLBIVAAE1), - DO_BIT(HFGITR, TLBIVALE1), - DO_BIT(HFGITR, TLBIVAALE1), + DO_TLBINXS_BIT(HFGITR, TLBIVMALLE1OS), + DO_TLBINXS_BIT(HFGITR, TLBIVAE1OS), + DO_TLBINXS_BIT(HFGITR, TLBIASIDE1OS), + DO_TLBINXS_BIT(HFGITR, TLBIVAAE1OS), + DO_TLBINXS_BIT(HFGITR, TLBIVALE1OS), + DO_TLBINXS_BIT(HFGITR, TLBIVAALE1OS), + DO_TLBINXS_BIT(HFGITR, TLBIRVAE1OS), + DO_TLBINXS_BIT(HFGITR, TLBIRVAAE1OS), + DO_TLBINXS_BIT(HFGITR, TLBIRVALE1OS), + DO_TLBINXS_BIT(HFGITR, TLBIRVAALE1OS), + DO_TLBINXS_BIT(HFGITR, TLBIVMALLE1IS), + DO_TLBINXS_BIT(HFGITR, TLBIVAE1IS), + DO_TLBINXS_BIT(HFGITR, TLBIASIDE1IS), + DO_TLBINXS_BIT(HFGITR, TLBIVAAE1IS), + DO_TLBINXS_BIT(HFGITR, TLBIVALE1IS), + DO_TLBINXS_BIT(HFGITR, TLBIVAALE1IS), + DO_TLBINXS_BIT(HFGITR, TLBIRVAE1IS), + DO_TLBINXS_BIT(HFGITR, TLBIRVAAE1IS), + DO_TLBINXS_BIT(HFGITR, TLBIRVALE1IS), + DO_TLBINXS_BIT(HFGITR, TLBIRVAALE1IS), + DO_TLBINXS_BIT(HFGITR, TLBIRVAE1), + DO_TLBINXS_BIT(HFGITR, TLBIRVAAE1), + DO_TLBINXS_BIT(HFGITR, TLBIRVALE1), + DO_TLBINXS_BIT(HFGITR, TLBIRVAALE1), + DO_TLBINXS_BIT(HFGITR, TLBIVMALLE1), + DO_TLBINXS_BIT(HFGITR, TLBIVAE1), + DO_TLBINXS_BIT(HFGITR, TLBIASIDE1), + DO_TLBINXS_BIT(HFGITR, TLBIVAAE1), + DO_TLBINXS_BIT(HFGITR, TLBIVALE1), + DO_TLBINXS_BIT(HFGITR, TLBIVAALE1), DO_BIT(HFGITR, CFPRCTX), DO_BIT(HFGITR, DVPRCTX), DO_BIT(HFGITR, CPPRCTX), diff --git a/target/arm/cpu-features.h b/target/arm/cpu-features.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpu-features.h +++ b/target/arm/cpu-features.h @@ -XXX,XX +XXX,XX @@ static inline bool isar_feature_aa64_fcma(const ARMISARegisters *id) return FIELD_EX64(id->id_aa64isar1, ID_AA64ISAR1, FCMA) != 0; } +static inline bool isar_feature_aa64_xs(const ARMISARegisters *id) +{ + return FIELD_EX64(id->id_aa64isar1, ID_AA64ISAR1, XS) != 0; +} + /* * These are the values from APA/API/APA3. * In general these must be compared '>=', per the normal Arm ARM diff --git a/target/arm/helper.c b/target/arm/helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -XXX,XX +XXX,XX @@ static void hcrx_write(CPUARMState *env, const ARMCPRegInfo *ri, valid_mask |= HCRX_TALLINT | HCRX_VINMI | HCRX_VFNMI; } /* FEAT_CMOW adds CMOW */ - if (cpu_isar_feature(aa64_cmow, cpu)) { valid_mask |= HCRX_CMOW; } + /* FEAT_XS adds FGTnXS, FnXS */ + if (cpu_isar_feature(aa64_xs, cpu)) { + valid_mask |= HCRX_FGTNXS | HCRX_FNXS; + } /* Clear RES0 bits. */ env->cp15.hcrx_el2 = value & valid_mask; diff --git a/target/arm/tcg/op_helper.c b/target/arm/tcg/op_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/op_helper.c +++ b/target/arm/tcg/op_helper.c @@ -XXX,XX +XXX,XX @@ const void *HELPER(access_check_cp_reg)(CPUARMState *env, uint32_t key, unsigned int idx = FIELD_EX32(ri->fgt, FGT, IDX); unsigned int bitpos = FIELD_EX32(ri->fgt, FGT, BITPOS); bool rev = FIELD_EX32(ri->fgt, FGT, REV); + bool nxs = FIELD_EX32(ri->fgt, FGT, NXS); bool trapbit; if (ri->fgt & FGT_EXEC) { @@ -XXX,XX +XXX,XX @@ const void *HELPER(access_check_cp_reg)(CPUARMState *env, uint32_t key, trapword = env->cp15.fgt_write[idx]; } - trapbit = extract64(trapword, bitpos, 1); + if (nxs && (arm_hcrx_el2_eff(env) & HCRX_FGTNXS)) { + /* + * If HCRX_EL2.FGTnXS is 1 then the fine-grained trap for + * TLBI maintenance insns does *not* apply to the nXS variant. + */ + trapbit = 0; + } else { + trapbit = extract64(trapword, bitpos, 1); + } if (trapbit != rev) { res = CP_ACCESS_TRAP_EL2; goto fail; -- 2.34.1 All of the TLBI insns with an NXS variant put that variant at the same encoding but with a CRn field that is one greater than for the original TLBI insn. To avoid having to define every TLBI insn effectively twice, once in the normal way and once in a set of cpreg arrays that are only registered when FEAT_XS is present, we define a new ARM_CP_ADD_TLB_NXS type flag for cpregs. When this flag is set in a cpreg struct and FEAT_XS is present, define_one_arm_cp_reg_with_opaque() will automatically add a second cpreg to the hash table for the TLBI NXS insn with: * the crn+1 encoding * an FGT field that indicates that it should honour HCR_EL2.FGTnXS * a name with the "NXS" suffix (If there are future TLBI NXS insns that don't use this same encoding convention, it is also possible to define them manually.) Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211144440.2700268-3-peter.maydell@linaro.org --- target/arm/cpregs.h | 8 ++++++++ target/arm/helper.c | 25 +++++++++++++++++++++++++ 2 files changed, 33 insertions(+) diff --git a/target/arm/cpregs.h b/target/arm/cpregs.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpregs.h +++ b/target/arm/cpregs.h @@ -XXX,XX +XXX,XX @@ enum { * equivalent EL1 register when FEAT_NV2 is enabled. */ ARM_CP_NV2_REDIRECT = 1 << 20, + /* + * Flag: this is a TLBI insn which (when FEAT_XS is present) also has + * an NXS variant at the same encoding except that crn is 1 greater, + * so when registering this cpreg automatically also register one + * for the TLBI NXS variant. (For QEMU the NXS variant behaves + * identically to the normal one, other than FGT trapping handling.) + */ + ARM_CP_ADD_TLBI_NXS = 1 << 21, }; /* diff --git a/target/arm/helper.c b/target/arm/helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -XXX,XX +XXX,XX @@ void define_one_arm_cp_reg_with_opaque(ARMCPU *cpu, if (r->state != state && r->state != ARM_CP_STATE_BOTH) { continue; } + if ((r->type & ARM_CP_ADD_TLBI_NXS) && + cpu_isar_feature(aa64_xs, cpu)) { + /* + * This is a TLBI insn which has an NXS variant. The + * NXS variant is at the same encoding except that + * crn is +1, and has the same behaviour except for + * fine-grained trapping. Add the NXS insn here and + * then fall through to add the normal register. + * add_cpreg_to_hashtable() copies the cpreg struct + * and name that it is passed, so it's OK to use + * a local struct here. + */ + ARMCPRegInfo nxs_ri = *r; + g_autofree char *name = g_strdup_printf("%sNXS", r->name); + + assert(state == ARM_CP_STATE_AA64); + assert(nxs_ri.crn < 0xf); + nxs_ri.crn++; + if (nxs_ri.fgt) { + nxs_ri.fgt |= R_FGT_NXS_MASK; + } + add_cpreg_to_hashtable(cpu, &nxs_ri, opaque, state, + ARM_CP_SECSTATE_NS, + crm, opc1, opc2, name); + } if (state == ARM_CP_STATE_AA32) { /* * Under AArch32 CP registers can be common -- 2.34.1 Add the ARM_CP_ADD_TLBI_NXS to the TLBI insns with an NXS variant. This is every AArch64 TLBI encoding except for the four FEAT_RME TLBI insns. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211144440.2700268-4-peter.maydell@linaro.org --- target/arm/tcg/tlb-insns.c | 202 +++++++++++++++++++++++-------------- 1 file changed, 124 insertions(+), 78 deletions(-) diff --git a/target/arm/tcg/tlb-insns.c b/target/arm/tcg/tlb-insns.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/tlb-insns.c +++ b/target/arm/tcg/tlb-insns.c @@ -XXX,XX +XXX,XX @@ static const ARMCPRegInfo tlbi_v8_cp_reginfo[] = { /* AArch64 TLBI operations */ { .name = "TLBI_VMALLE1IS", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 3, .opc2 = 0, - .access = PL1_W, .accessfn = access_ttlbis, .type = ARM_CP_NO_RAW, + .access = PL1_W, .accessfn = access_ttlbis, + .type = ARM_CP_NO_RAW | ARM_CP_ADD_TLBI_NXS, .fgt = FGT_TLBIVMALLE1IS, .writefn = tlbi_aa64_vmalle1is_write }, { .name = "TLBI_VAE1IS", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 3, .opc2 = 1, - .access = PL1_W, .accessfn = access_ttlbis, .type = ARM_CP_NO_RAW, + .access = PL1_W, .accessfn = access_ttlbis, + .type = ARM_CP_NO_RAW | ARM_CP_ADD_TLBI_NXS, .fgt = FGT_TLBIVAE1IS, .writefn = tlbi_aa64_vae1is_write }, { .name = "TLBI_ASIDE1IS", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 3, .opc2 = 2, - .access = PL1_W, .accessfn = access_ttlbis, .type = ARM_CP_NO_RAW, + .access = PL1_W, .accessfn = access_ttlbis, + .type = ARM_CP_NO_RAW | ARM_CP_ADD_TLBI_NXS, .fgt = FGT_TLBIASIDE1IS, .writefn = tlbi_aa64_vmalle1is_write }, { .name = "TLBI_VAAE1IS", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 3, .opc2 = 3, - .access = PL1_W, .accessfn = access_ttlbis, .type = ARM_CP_NO_RAW, + .access = PL1_W, .accessfn = access_ttlbis, + .type = ARM_CP_NO_RAW | ARM_CP_ADD_TLBI_NXS, .fgt = FGT_TLBIVAAE1IS, .writefn = tlbi_aa64_vae1is_write }, { .name = "TLBI_VALE1IS", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 3, .opc2 = 5, - .access = PL1_W, .accessfn = access_ttlbis, .type = ARM_CP_NO_RAW, + .access = PL1_W, .accessfn = access_ttlbis, + .type = ARM_CP_NO_RAW | ARM_CP_ADD_TLBI_NXS, .fgt = FGT_TLBIVALE1IS, .writefn = tlbi_aa64_vae1is_write }, { .name = "TLBI_VAALE1IS", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 3, .opc2 = 7, - .access = PL1_W, .accessfn = access_ttlbis, .type = ARM_CP_NO_RAW, + .access = PL1_W, .accessfn = access_ttlbis, + .type = ARM_CP_NO_RAW | ARM_CP_ADD_TLBI_NXS, .fgt = FGT_TLBIVAALE1IS, .writefn = tlbi_aa64_vae1is_write }, { .name = "TLBI_VMALLE1", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 7, .opc2 = 0, - .access = PL1_W, .accessfn = access_ttlb, .type = ARM_CP_NO_RAW, + .access = PL1_W, .accessfn = access_ttlb, + .type = ARM_CP_NO_RAW | ARM_CP_ADD_TLBI_NXS, .fgt = FGT_TLBIVMALLE1, .writefn = tlbi_aa64_vmalle1_write }, { .name = "TLBI_VAE1", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 7, .opc2 = 1, - .access = PL1_W, .accessfn = access_ttlb, .type = ARM_CP_NO_RAW, + .access = PL1_W, .accessfn = access_ttlb, + .type = ARM_CP_NO_RAW | ARM_CP_ADD_TLBI_NXS, .fgt = FGT_TLBIVAE1, .writefn = tlbi_aa64_vae1_write }, { .name = "TLBI_ASIDE1", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 7, .opc2 = 2, - .access = PL1_W, .accessfn = access_ttlb, .type = ARM_CP_NO_RAW, + .access = PL1_W, .accessfn = access_ttlb, + .type = ARM_CP_NO_RAW | ARM_CP_ADD_TLBI_NXS, .fgt = FGT_TLBIASIDE1, .writefn = tlbi_aa64_vmalle1_write }, { .name = "TLBI_VAAE1", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 7, .opc2 = 3, - .access = PL1_W, .accessfn = access_ttlb, .type = ARM_CP_NO_RAW, + .access = PL1_W, .accessfn = access_ttlb, + .type = ARM_CP_NO_RAW | ARM_CP_ADD_TLBI_NXS, .fgt = FGT_TLBIVAAE1, .writefn = tlbi_aa64_vae1_write }, { .name = "TLBI_VALE1", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 7, .opc2 = 5, - .access = PL1_W, .accessfn = access_ttlb, .type = ARM_CP_NO_RAW, + .access = PL1_W, .accessfn = access_ttlb, + .type = ARM_CP_NO_RAW | ARM_CP_ADD_TLBI_NXS, .fgt = FGT_TLBIVALE1, .writefn = tlbi_aa64_vae1_write }, { .name = "TLBI_VAALE1", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 7, .opc2 = 7, - .access = PL1_W, .accessfn = access_ttlb, .type = ARM_CP_NO_RAW, + .access = PL1_W, .accessfn = access_ttlb, + .type = ARM_CP_NO_RAW | ARM_CP_ADD_TLBI_NXS, .fgt = FGT_TLBIVAALE1, .writefn = tlbi_aa64_vae1_write }, { .name = "TLBI_IPAS2E1IS", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 0, .opc2 = 1, - .access = PL2_W, .type = ARM_CP_NO_RAW, + .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_ADD_TLBI_NXS, .writefn = tlbi_aa64_ipas2e1is_write }, { .name = "TLBI_IPAS2LE1IS", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 0, .opc2 = 5, - .access = PL2_W, .type = ARM_CP_NO_RAW, + .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_ADD_TLBI_NXS, .writefn = tlbi_aa64_ipas2e1is_write }, { .name = "TLBI_ALLE1IS", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 3, .opc2 = 4, - .access = PL2_W, .type = ARM_CP_NO_RAW, + .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_ADD_TLBI_NXS, .writefn = tlbi_aa64_alle1is_write }, { .name = "TLBI_VMALLS12E1IS", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 3, .opc2 = 6, - .access = PL2_W, .type = ARM_CP_NO_RAW, + .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_ADD_TLBI_NXS, .writefn = tlbi_aa64_alle1is_write }, { .name = "TLBI_IPAS2E1", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 4, .opc2 = 1, - .access = PL2_W, .type = ARM_CP_NO_RAW, + .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_ADD_TLBI_NXS, .writefn = tlbi_aa64_ipas2e1_write }, { .name = "TLBI_IPAS2LE1", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 4, .opc2 = 5, - .access = PL2_W, .type = ARM_CP_NO_RAW, + .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_ADD_TLBI_NXS, .writefn = tlbi_aa64_ipas2e1_write }, { .name = "TLBI_ALLE1", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 7, .opc2 = 4, - .access = PL2_W, .type = ARM_CP_NO_RAW, + .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_ADD_TLBI_NXS, .writefn = tlbi_aa64_alle1_write }, { .name = "TLBI_VMALLS12E1", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 7, .opc2 = 6, - .access = PL2_W, .type = ARM_CP_NO_RAW, + .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_ADD_TLBI_NXS, .writefn = tlbi_aa64_alle1is_write }, }; @@ -XXX,XX +XXX,XX @@ static const ARMCPRegInfo tlbi_el2_cp_reginfo[] = { .writefn = tlbimva_hyp_is_write }, { .name = "TLBI_ALLE2", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 7, .opc2 = 0, - .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF, + .access = PL2_W, + .type = ARM_CP_NO_RAW | ARM_CP_ADD_TLBI_NXS | ARM_CP_EL3_NO_EL2_UNDEF, .writefn = tlbi_aa64_alle2_write }, { .name = "TLBI_VAE2", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 7, .opc2 = 1, - .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF, + .access = PL2_W, + .type = ARM_CP_NO_RAW | ARM_CP_ADD_TLBI_NXS | ARM_CP_EL3_NO_EL2_UNDEF, .writefn = tlbi_aa64_vae2_write }, { .name = "TLBI_VALE2", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 7, .opc2 = 5, - .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF, + .access = PL2_W, + .type = ARM_CP_NO_RAW | ARM_CP_ADD_TLBI_NXS | ARM_CP_EL3_NO_EL2_UNDEF, .writefn = tlbi_aa64_vae2_write }, { .name = "TLBI_ALLE2IS", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 3, .opc2 = 0, - .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF, + .access = PL2_W, + .type = ARM_CP_NO_RAW | ARM_CP_ADD_TLBI_NXS | ARM_CP_EL3_NO_EL2_UNDEF, .writefn = tlbi_aa64_alle2is_write }, { .name = "TLBI_VAE2IS", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 3, .opc2 = 1, - .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF, + .access = PL2_W, + .type = ARM_CP_NO_RAW | ARM_CP_ADD_TLBI_NXS | ARM_CP_EL3_NO_EL2_UNDEF, .writefn = tlbi_aa64_vae2is_write }, { .name = "TLBI_VALE2IS", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 3, .opc2 = 5, - .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF, + .access = PL2_W, + .type = ARM_CP_NO_RAW | ARM_CP_ADD_TLBI_NXS | ARM_CP_EL3_NO_EL2_UNDEF, .writefn = tlbi_aa64_vae2is_write }, }; static const ARMCPRegInfo tlbi_el3_cp_reginfo[] = { { .name = "TLBI_ALLE3IS", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 3, .opc2 = 0, - .access = PL3_W, .type = ARM_CP_NO_RAW, + .access = PL3_W, .type = ARM_CP_NO_RAW | ARM_CP_ADD_TLBI_NXS, .writefn = tlbi_aa64_alle3is_write }, { .name = "TLBI_VAE3IS", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 3, .opc2 = 1, - .access = PL3_W, .type = ARM_CP_NO_RAW, + .access = PL3_W, .type = ARM_CP_NO_RAW | ARM_CP_ADD_TLBI_NXS, .writefn = tlbi_aa64_vae3is_write }, { .name = "TLBI_VALE3IS", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 3, .opc2 = 5, - .access = PL3_W, .type = ARM_CP_NO_RAW, + .access = PL3_W, .type = ARM_CP_NO_RAW | ARM_CP_ADD_TLBI_NXS, .writefn = tlbi_aa64_vae3is_write }, { .name = "TLBI_ALLE3", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 7, .opc2 = 0, - .access = PL3_W, .type = ARM_CP_NO_RAW, + .access = PL3_W, .type = ARM_CP_NO_RAW | ARM_CP_ADD_TLBI_NXS, .writefn = tlbi_aa64_alle3_write }, { .name = "TLBI_VAE3", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 7, .opc2 = 1, - .access = PL3_W, .type = ARM_CP_NO_RAW, + .access = PL3_W, .type = ARM_CP_NO_RAW | ARM_CP_ADD_TLBI_NXS, .writefn = tlbi_aa64_vae3_write }, { .name = "TLBI_VALE3", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 7, .opc2 = 5, - .access = PL3_W, .type = ARM_CP_NO_RAW, + .access = PL3_W, .type = ARM_CP_NO_RAW | ARM_CP_ADD_TLBI_NXS, .writefn = tlbi_aa64_vae3_write }, }; @@ -XXX,XX +XXX,XX @@ static void tlbi_aa64_ripas2e1is_write(CPUARMState *env, static const ARMCPRegInfo tlbirange_reginfo[] = { { .name = "TLBI_RVAE1IS", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 2, .opc2 = 1, - .access = PL1_W, .accessfn = access_ttlbis, .type = ARM_CP_NO_RAW, + .access = PL1_W, .accessfn = access_ttlbis, + .type = ARM_CP_NO_RAW | ARM_CP_ADD_TLBI_NXS, .fgt = FGT_TLBIRVAE1IS, .writefn = tlbi_aa64_rvae1is_write }, { .name = "TLBI_RVAAE1IS", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 2, .opc2 = 3, - .access = PL1_W, .accessfn = access_ttlbis, .type = ARM_CP_NO_RAW, + .access = PL1_W, .accessfn = access_ttlbis, + .type = ARM_CP_NO_RAW | ARM_CP_ADD_TLBI_NXS, .fgt = FGT_TLBIRVAAE1IS, .writefn = tlbi_aa64_rvae1is_write }, { .name = "TLBI_RVALE1IS", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 2, .opc2 = 5, - .access = PL1_W, .accessfn = access_ttlbis, .type = ARM_CP_NO_RAW, + .access = PL1_W, .accessfn = access_ttlbis, + .type = ARM_CP_NO_RAW | ARM_CP_ADD_TLBI_NXS, .fgt = FGT_TLBIRVALE1IS, .writefn = tlbi_aa64_rvae1is_write }, { .name = "TLBI_RVAALE1IS", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 2, .opc2 = 7, - .access = PL1_W, .accessfn = access_ttlbis, .type = ARM_CP_NO_RAW, + .access = PL1_W, .accessfn = access_ttlbis, + .type = ARM_CP_NO_RAW | ARM_CP_ADD_TLBI_NXS, .fgt = FGT_TLBIRVAALE1IS, .writefn = tlbi_aa64_rvae1is_write }, { .name = "TLBI_RVAE1OS", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 5, .opc2 = 1, - .access = PL1_W, .accessfn = access_ttlbos, .type = ARM_CP_NO_RAW, + .access = PL1_W, .accessfn = access_ttlbos, + .type = ARM_CP_NO_RAW | ARM_CP_ADD_TLBI_NXS, .fgt = FGT_TLBIRVAE1OS, .writefn = tlbi_aa64_rvae1is_write }, { .name = "TLBI_RVAAE1OS", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 5, .opc2 = 3, - .access = PL1_W, .accessfn = access_ttlbos, .type = ARM_CP_NO_RAW, + .access = PL1_W, .accessfn = access_ttlbos, + .type = ARM_CP_NO_RAW | ARM_CP_ADD_TLBI_NXS, .fgt = FGT_TLBIRVAAE1OS, .writefn = tlbi_aa64_rvae1is_write }, { .name = "TLBI_RVALE1OS", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 5, .opc2 = 5, - .access = PL1_W, .accessfn = access_ttlbos, .type = ARM_CP_NO_RAW, + .access = PL1_W, .accessfn = access_ttlbos, + .type = ARM_CP_NO_RAW | ARM_CP_ADD_TLBI_NXS, .fgt = FGT_TLBIRVALE1OS, .writefn = tlbi_aa64_rvae1is_write }, { .name = "TLBI_RVAALE1OS", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 5, .opc2 = 7, - .access = PL1_W, .accessfn = access_ttlbos, .type = ARM_CP_NO_RAW, + .access = PL1_W, .accessfn = access_ttlbos, + .type = ARM_CP_NO_RAW | ARM_CP_ADD_TLBI_NXS, .fgt = FGT_TLBIRVAALE1OS, .writefn = tlbi_aa64_rvae1is_write }, { .name = "TLBI_RVAE1", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 6, .opc2 = 1, - .access = PL1_W, .accessfn = access_ttlb, .type = ARM_CP_NO_RAW, + .access = PL1_W, .accessfn = access_ttlb, + .type = ARM_CP_NO_RAW | ARM_CP_ADD_TLBI_NXS, .fgt = FGT_TLBIRVAE1, .writefn = tlbi_aa64_rvae1_write }, { .name = "TLBI_RVAAE1", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 6, .opc2 = 3, - .access = PL1_W, .accessfn = access_ttlb, .type = ARM_CP_NO_RAW, + .access = PL1_W, .accessfn = access_ttlb, + .type = ARM_CP_NO_RAW | ARM_CP_ADD_TLBI_NXS, .fgt = FGT_TLBIRVAAE1, .writefn = tlbi_aa64_rvae1_write }, { .name = "TLBI_RVALE1", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 6, .opc2 = 5, - .access = PL1_W, .accessfn = access_ttlb, .type = ARM_CP_NO_RAW, + .access = PL1_W, .accessfn = access_ttlb, + .type = ARM_CP_NO_RAW | ARM_CP_ADD_TLBI_NXS, .fgt = FGT_TLBIRVALE1, .writefn = tlbi_aa64_rvae1_write }, { .name = "TLBI_RVAALE1", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 6, .opc2 = 7, - .access = PL1_W, .accessfn = access_ttlb, .type = ARM_CP_NO_RAW, + .access = PL1_W, .accessfn = access_ttlb, + .type = ARM_CP_NO_RAW | ARM_CP_ADD_TLBI_NXS, .fgt = FGT_TLBIRVAALE1, .writefn = tlbi_aa64_rvae1_write }, { .name = "TLBI_RIPAS2E1IS", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 0, .opc2 = 2, - .access = PL2_W, .type = ARM_CP_NO_RAW, + .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_ADD_TLBI_NXS, .writefn = tlbi_aa64_ripas2e1is_write }, { .name = "TLBI_RIPAS2LE1IS", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 0, .opc2 = 6, - .access = PL2_W, .type = ARM_CP_NO_RAW, + .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_ADD_TLBI_NXS, .writefn = tlbi_aa64_ripas2e1is_write }, { .name = "TLBI_RVAE2IS", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 2, .opc2 = 1, - .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF, + .access = PL2_W, + .type = ARM_CP_NO_RAW | ARM_CP_ADD_TLBI_NXS | ARM_CP_EL3_NO_EL2_UNDEF, .writefn = tlbi_aa64_rvae2is_write }, { .name = "TLBI_RVALE2IS", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 2, .opc2 = 5, - .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF, + .access = PL2_W, + .type = ARM_CP_NO_RAW | ARM_CP_ADD_TLBI_NXS | ARM_CP_EL3_NO_EL2_UNDEF, .writefn = tlbi_aa64_rvae2is_write }, { .name = "TLBI_RIPAS2E1", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 4, .opc2 = 2, - .access = PL2_W, .type = ARM_CP_NO_RAW, + .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_ADD_TLBI_NXS, .writefn = tlbi_aa64_ripas2e1_write }, { .name = "TLBI_RIPAS2LE1", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 4, .opc2 = 6, - .access = PL2_W, .type = ARM_CP_NO_RAW, + .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_ADD_TLBI_NXS, .writefn = tlbi_aa64_ripas2e1_write }, { .name = "TLBI_RVAE2OS", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 5, .opc2 = 1, - .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF, + .access = PL2_W, + .type = ARM_CP_NO_RAW | ARM_CP_ADD_TLBI_NXS | ARM_CP_EL3_NO_EL2_UNDEF, .writefn = tlbi_aa64_rvae2is_write }, { .name = "TLBI_RVALE2OS", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 5, .opc2 = 5, - .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF, + .access = PL2_W, + .type = ARM_CP_NO_RAW | ARM_CP_ADD_TLBI_NXS | ARM_CP_EL3_NO_EL2_UNDEF, .writefn = tlbi_aa64_rvae2is_write }, { .name = "TLBI_RVAE2", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 6, .opc2 = 1, - .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF, + .access = PL2_W, + .type = ARM_CP_NO_RAW | ARM_CP_ADD_TLBI_NXS | ARM_CP_EL3_NO_EL2_UNDEF, .writefn = tlbi_aa64_rvae2_write }, { .name = "TLBI_RVALE2", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 6, .opc2 = 5, - .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF, + .access = PL2_W, + .type = ARM_CP_NO_RAW | ARM_CP_ADD_TLBI_NXS | ARM_CP_EL3_NO_EL2_UNDEF, .writefn = tlbi_aa64_rvae2_write }, { .name = "TLBI_RVAE3IS", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 2, .opc2 = 1, - .access = PL3_W, .type = ARM_CP_NO_RAW, + .access = PL3_W, .type = ARM_CP_NO_RAW | ARM_CP_ADD_TLBI_NXS, .writefn = tlbi_aa64_rvae3is_write }, { .name = "TLBI_RVALE3IS", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 2, .opc2 = 5, - .access = PL3_W, .type = ARM_CP_NO_RAW, + .access = PL3_W, .type = ARM_CP_NO_RAW | ARM_CP_ADD_TLBI_NXS, .writefn = tlbi_aa64_rvae3is_write }, { .name = "TLBI_RVAE3OS", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 5, .opc2 = 1, - .access = PL3_W, .type = ARM_CP_NO_RAW, + .access = PL3_W, .type = ARM_CP_NO_RAW | ARM_CP_ADD_TLBI_NXS, .writefn = tlbi_aa64_rvae3is_write }, { .name = "TLBI_RVALE3OS", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 5, .opc2 = 5, - .access = PL3_W, .type = ARM_CP_NO_RAW, + .access = PL3_W, .type = ARM_CP_NO_RAW | ARM_CP_ADD_TLBI_NXS, .writefn = tlbi_aa64_rvae3is_write }, { .name = "TLBI_RVAE3", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 6, .opc2 = 1, - .access = PL3_W, .type = ARM_CP_NO_RAW, + .access = PL3_W, .type = ARM_CP_NO_RAW | ARM_CP_ADD_TLBI_NXS, .writefn = tlbi_aa64_rvae3_write }, { .name = "TLBI_RVALE3", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 6, .opc2 = 5, - .access = PL3_W, .type = ARM_CP_NO_RAW, + .access = PL3_W, .type = ARM_CP_NO_RAW | ARM_CP_ADD_TLBI_NXS, .writefn = tlbi_aa64_rvae3_write }, }; static const ARMCPRegInfo tlbios_reginfo[] = { { .name = "TLBI_VMALLE1OS", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 1, .opc2 = 0, - .access = PL1_W, .accessfn = access_ttlbos, .type = ARM_CP_NO_RAW, + .access = PL1_W, .accessfn = access_ttlbos, + .type = ARM_CP_NO_RAW | ARM_CP_ADD_TLBI_NXS, .fgt = FGT_TLBIVMALLE1OS, .writefn = tlbi_aa64_vmalle1is_write }, { .name = "TLBI_VAE1OS", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 1, .opc2 = 1, .fgt = FGT_TLBIVAE1OS, - .access = PL1_W, .accessfn = access_ttlbos, .type = ARM_CP_NO_RAW, + .access = PL1_W, .accessfn = access_ttlbos, + .type = ARM_CP_NO_RAW | ARM_CP_ADD_TLBI_NXS, .writefn = tlbi_aa64_vae1is_write }, { .name = "TLBI_ASIDE1OS", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 1, .opc2 = 2, - .access = PL1_W, .accessfn = access_ttlbos, .type = ARM_CP_NO_RAW, + .access = PL1_W, .accessfn = access_ttlbos, + .type = ARM_CP_NO_RAW | ARM_CP_ADD_TLBI_NXS, .fgt = FGT_TLBIASIDE1OS, .writefn = tlbi_aa64_vmalle1is_write }, { .name = "TLBI_VAAE1OS", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 1, .opc2 = 3, - .access = PL1_W, .accessfn = access_ttlbos, .type = ARM_CP_NO_RAW, + .access = PL1_W, .accessfn = access_ttlbos, + .type = ARM_CP_NO_RAW | ARM_CP_ADD_TLBI_NXS, .fgt = FGT_TLBIVAAE1OS, .writefn = tlbi_aa64_vae1is_write }, { .name = "TLBI_VALE1OS", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 1, .opc2 = 5, - .access = PL1_W, .accessfn = access_ttlbos, .type = ARM_CP_NO_RAW, + .access = PL1_W, .accessfn = access_ttlbos, + .type = ARM_CP_NO_RAW | ARM_CP_ADD_TLBI_NXS, .fgt = FGT_TLBIVALE1OS, .writefn = tlbi_aa64_vae1is_write }, { .name = "TLBI_VAALE1OS", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 1, .opc2 = 7, - .access = PL1_W, .accessfn = access_ttlbos, .type = ARM_CP_NO_RAW, + .access = PL1_W, .accessfn = access_ttlbos, + .type = ARM_CP_NO_RAW | ARM_CP_ADD_TLBI_NXS, .fgt = FGT_TLBIVAALE1OS, .writefn = tlbi_aa64_vae1is_write }, { .name = "TLBI_ALLE2OS", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 1, .opc2 = 0, - .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF, + .access = PL2_W, + .type = ARM_CP_NO_RAW | ARM_CP_ADD_TLBI_NXS | ARM_CP_EL3_NO_EL2_UNDEF, .writefn = tlbi_aa64_alle2is_write }, { .name = "TLBI_VAE2OS", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 1, .opc2 = 1, - .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF, + .access = PL2_W, + .type = ARM_CP_NO_RAW | ARM_CP_ADD_TLBI_NXS | ARM_CP_EL3_NO_EL2_UNDEF, .writefn = tlbi_aa64_vae2is_write }, { .name = "TLBI_ALLE1OS", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 1, .opc2 = 4, - .access = PL2_W, .type = ARM_CP_NO_RAW, + .access = PL2_W, + .type = ARM_CP_NO_RAW | ARM_CP_ADD_TLBI_NXS, .writefn = tlbi_aa64_alle1is_write }, { .name = "TLBI_VALE2OS", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 1, .opc2 = 5, - .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF, + .access = PL2_W, + .type = ARM_CP_NO_RAW | ARM_CP_ADD_TLBI_NXS | ARM_CP_EL3_NO_EL2_UNDEF, .writefn = tlbi_aa64_vae2is_write }, { .name = "TLBI_VMALLS12E1OS", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 1, .opc2 = 6, - .access = PL2_W, .type = ARM_CP_NO_RAW, + .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_ADD_TLBI_NXS, .writefn = tlbi_aa64_alle1is_write }, { .name = "TLBI_IPAS2E1OS", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 4, .opc2 = 0, - .access = PL2_W, .type = ARM_CP_NOP }, + .access = PL2_W, .type = ARM_CP_NOP | ARM_CP_ADD_TLBI_NXS }, { .name = "TLBI_RIPAS2E1OS", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 4, .opc2 = 3, - .access = PL2_W, .type = ARM_CP_NOP }, + .access = PL2_W, .type = ARM_CP_NOP | ARM_CP_ADD_TLBI_NXS }, { .name = "TLBI_IPAS2LE1OS", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 4, .opc2 = 4, - .access = PL2_W, .type = ARM_CP_NOP }, + .access = PL2_W, .type = ARM_CP_NOP | ARM_CP_ADD_TLBI_NXS }, { .name = "TLBI_RIPAS2LE1OS", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 4, .opc2 = 7, - .access = PL2_W, .type = ARM_CP_NOP }, + .access = PL2_W, .type = ARM_CP_NOP | ARM_CP_ADD_TLBI_NXS }, { .name = "TLBI_ALLE3OS", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 1, .opc2 = 0, - .access = PL3_W, .type = ARM_CP_NO_RAW, + .access = PL3_W, .type = ARM_CP_NO_RAW | ARM_CP_ADD_TLBI_NXS, .writefn = tlbi_aa64_alle3is_write }, { .name = "TLBI_VAE3OS", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 1, .opc2 = 1, - .access = PL3_W, .type = ARM_CP_NO_RAW, + .access = PL3_W, .type = ARM_CP_NO_RAW | ARM_CP_ADD_TLBI_NXS, .writefn = tlbi_aa64_vae3is_write }, { .name = "TLBI_VALE3OS", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 1, .opc2 = 5, - .access = PL3_W, .type = ARM_CP_NO_RAW, + .access = PL3_W, .type = ARM_CP_NO_RAW | ARM_CP_ADD_TLBI_NXS, .writefn = tlbi_aa64_vae3is_write }, }; -- 2.34.1 From: Manos Pitsidianakis <manos.pitsidianakis@linaro.org> The DSB nXS variant is always both a reads and writes request type. Ignore the domain field like we do in plain DSB and perform a full system barrier operation. The DSB nXS variant is part of FEAT_XS made mandatory from Armv8.7. Signed-off-by: Manos Pitsidianakis <manos.pitsidianakis@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211144440.2700268-5-peter.maydell@linaro.org [PMM: added missing "UNDEF unless feature present" check] Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 3 +++ target/arm/tcg/translate-a64.c | 9 +++++++++ 2 files changed, 12 insertions(+) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ WFIT 1101 0101 0000 0011 0001 0000 001 rd:5 CLREX 1101 0101 0000 0011 0011 ---- 010 11111 DSB_DMB 1101 0101 0000 0011 0011 domain:2 types:2 10- 11111 +# For the DSB nXS variant, types always equals MBReqTypes_All and we ignore the +# domain bits. +DSB_nXS 1101 0101 0000 0011 0011 -- 10 001 11111 ISB 1101 0101 0000 0011 0011 ---- 110 11111 SB 1101 0101 0000 0011 0011 0000 111 11111 diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static bool trans_DSB_DMB(DisasContext *s, arg_DSB_DMB *a) return true; } +static bool trans_DSB_nXS(DisasContext *s, arg_DSB_nXS *a) +{ + if (!dc_isar_feature(aa64_xs, s)) { + return false; + } + tcg_gen_mb(TCG_BAR_SC | TCG_MO_ALL); + return true; +} + static bool trans_ISB(DisasContext *s, arg_ISB *a) { /* -- 2.34.1 From: Manos Pitsidianakis <manos.pitsidianakis@linaro.org> Add FEAT_XS feature report value in max cpu's ID_AA64ISAR1 sys register. Signed-off-by: Manos Pitsidianakis <manos.pitsidianakis@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211144440.2700268-6-peter.maydell@linaro.org [PMM: Add entry for FEAT_XS to documentation] Reviewed-by: Richard Henderson <richard.henderson@linaro.org> --- docs/system/arm/emulation.rst | 1 + target/arm/tcg/cpu64.c | 1 + 2 files changed, 2 insertions(+) diff --git a/docs/system/arm/emulation.rst b/docs/system/arm/emulation.rst index XXXXXXX..XXXXXXX 100644 --- a/docs/system/arm/emulation.rst +++ b/docs/system/arm/emulation.rst @@ -XXX,XX +XXX,XX @@ the following architecture extensions: - FEAT_VMID16 (16-bit VMID) - FEAT_WFxT (WFE and WFI instructions with timeout) - FEAT_XNX (Translation table stage 2 Unprivileged Execute-never) +- FEAT_XS (XS attribute) For information on the specifics of these extensions, please refer to the `Arm Architecture Reference Manual for A-profile architecture diff --git a/target/arm/tcg/cpu64.c b/target/arm/tcg/cpu64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/cpu64.c +++ b/target/arm/tcg/cpu64.c @@ -XXX,XX +XXX,XX @@ void aarch64_max_tcg_initfn(Object *obj) t = FIELD_DP64(t, ID_AA64ISAR1, BF16, 2); /* FEAT_BF16, FEAT_EBF16 */ t = FIELD_DP64(t, ID_AA64ISAR1, DGH, 1); /* FEAT_DGH */ t = FIELD_DP64(t, ID_AA64ISAR1, I8MM, 1); /* FEAT_I8MM */ + t = FIELD_DP64(t, ID_AA64ISAR1, XS, 1); /* FEAT_XS */ cpu->isar.id_aa64isar1 = t; t = cpu->isar.id_aa64isar2; -- 2.34.1 From: Manos Pitsidianakis <manos.pitsidianakis@linaro.org> Add system test to make sure FEAT_XS is enabled for max cpu emulation and that QEMU doesn't crash when encountering an NXS instruction variant. Signed-off-by: Manos Pitsidianakis <manos.pitsidianakis@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Message-id: 20241211144440.2700268-7-peter.maydell@linaro.org [PMM: In ISAR field test, mask with 0xf, not 0xff; use < rather than an equality test to follow the standard ID register field check guidelines] Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- tests/tcg/aarch64/system/feat-xs.c | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+) create mode 100644 tests/tcg/aarch64/system/feat-xs.c diff --git a/tests/tcg/aarch64/system/feat-xs.c b/tests/tcg/aarch64/system/feat-xs.c new file mode 100644 index XXXXXXX..XXXXXXX --- /dev/null +++ b/tests/tcg/aarch64/system/feat-xs.c @@ -XXX,XX +XXX,XX @@ +/* + * FEAT_XS Test + * + * Copyright (c) 2024 Linaro Ltd + * + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#include <minilib.h> +#include <stdint.h> + +int main(void) +{ + uint64_t isar1; + + asm volatile ("mrs %0, id_aa64isar1_el1" : "=r"(isar1)); + if (((isar1 >> 56) & 0xf) < 1) { + ml_printf("FEAT_XS not supported by CPU"); + return 1; + } + /* VMALLE1NXS */ + asm volatile (".inst 0xd508971f"); + /* VMALLE1OSNXS */ + asm volatile (".inst 0xd508911f"); + + return 0; +} -- 2.34.1 In the GICv3 ITS model, we have a common coding pattern which has a local C struct like "DTEntry dte", which is a C representation of an in-guest-memory data structure, and we call a function such as get_dte() to read guest memory and fill in the C struct. These functions to read in the struct sometimes have cases where they will leave early and not fill in the whole struct (for instance get_dte() will set "dte->valid = false" and nothing else for the case where it is passed an entry_addr implying that there is no L2 table entry for the DTE). This then causes potential use of uninitialized memory later, for instance when we call a trace event which prints all the fields of the struct. Sufficiently advanced compilers may produce -Wmaybe-uninitialized warnings about this, especially if LTO is enabled. Rather than trying to carefully separate out these trace events into "only the 'valid' field is initialized" and "all fields can be printed", zero-init all the structs when we define them. None of these structs are large (the biggest is 24 bytes) and having consistent behaviour is less likely to be buggy. Cc: qemu-stable@nongnu.org Resolves: https://gitlab.com/qemu-project/qemu/-/issues/2718 Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Message-id: 20241213182337.3343068-1-peter.maydell@linaro.org --- hw/intc/arm_gicv3_its.c | 44 ++++++++++++++++++++--------------------- 1 file changed, 22 insertions(+), 22 deletions(-) diff --git a/hw/intc/arm_gicv3_its.c b/hw/intc/arm_gicv3_its.c index XXXXXXX..XXXXXXX 100644 --- a/hw/intc/arm_gicv3_its.c +++ b/hw/intc/arm_gicv3_its.c @@ -XXX,XX +XXX,XX @@ static ItsCmdResult lookup_vte(GICv3ITSState *s, const char *who, static ItsCmdResult process_its_cmd_phys(GICv3ITSState *s, const ITEntry *ite, int irqlevel) { - CTEntry cte; + CTEntry cte = {}; ItsCmdResult cmdres; cmdres = lookup_cte(s, __func__, ite->icid, &cte); @@ -XXX,XX +XXX,XX @@ static ItsCmdResult process_its_cmd_phys(GICv3ITSState *s, const ITEntry *ite, static ItsCmdResult process_its_cmd_virt(GICv3ITSState *s, const ITEntry *ite, int irqlevel) { - VTEntry vte; + VTEntry vte = {}; ItsCmdResult cmdres; cmdres = lookup_vte(s, __func__, ite->vpeid, &vte); @@ -XXX,XX +XXX,XX @@ static ItsCmdResult process_its_cmd_virt(GICv3ITSState *s, const ITEntry *ite, static ItsCmdResult do_process_its_cmd(GICv3ITSState *s, uint32_t devid, uint32_t eventid, ItsCmdType cmd) { - DTEntry dte; - ITEntry ite; + DTEntry dte = {}; + ITEntry ite = {}; ItsCmdResult cmdres; int irqlevel; @@ -XXX,XX +XXX,XX @@ static ItsCmdResult process_mapti(GICv3ITSState *s, const uint64_t *cmdpkt, uint32_t pIntid = 0; uint64_t num_eventids; uint16_t icid = 0; - DTEntry dte; - ITEntry ite; + DTEntry dte = {}; + ITEntry ite = {}; devid = (cmdpkt[0] & DEVID_MASK) >> DEVID_SHIFT; eventid = cmdpkt[1] & EVENTID_MASK; @@ -XXX,XX +XXX,XX @@ static ItsCmdResult process_vmapti(GICv3ITSState *s, const uint64_t *cmdpkt, { uint32_t devid, eventid, vintid, doorbell, vpeid; uint32_t num_eventids; - DTEntry dte; - ITEntry ite; + DTEntry dte = {}; + ITEntry ite = {}; if (!its_feature_virtual(s)) { return CMD_CONTINUE; @@ -XXX,XX +XXX,XX @@ static bool update_cte(GICv3ITSState *s, uint16_t icid, const CTEntry *cte) static ItsCmdResult process_mapc(GICv3ITSState *s, const uint64_t *cmdpkt) { uint16_t icid; - CTEntry cte; + CTEntry cte = {}; icid = cmdpkt[2] & ICID_MASK; cte.valid = cmdpkt[2] & CMD_FIELD_VALID_MASK; @@ -XXX,XX +XXX,XX @@ static bool update_dte(GICv3ITSState *s, uint32_t devid, const DTEntry *dte) static ItsCmdResult process_mapd(GICv3ITSState *s, const uint64_t *cmdpkt) { uint32_t devid; - DTEntry dte; + DTEntry dte = {}; devid = (cmdpkt[0] & DEVID_MASK) >> DEVID_SHIFT; dte.size = cmdpkt[1] & SIZE_MASK; @@ -XXX,XX +XXX,XX @@ static ItsCmdResult process_movi(GICv3ITSState *s, const uint64_t *cmdpkt) { uint32_t devid, eventid; uint16_t new_icid; - DTEntry dte; - CTEntry old_cte, new_cte; - ITEntry old_ite; + DTEntry dte = {}; + CTEntry old_cte = {}, new_cte = {}; + ITEntry old_ite = {}; ItsCmdResult cmdres; devid = FIELD_EX64(cmdpkt[0], MOVI_0, DEVICEID); @@ -XXX,XX +XXX,XX @@ static bool update_vte(GICv3ITSState *s, uint32_t vpeid, const VTEntry *vte) static ItsCmdResult process_vmapp(GICv3ITSState *s, const uint64_t *cmdpkt) { - VTEntry vte; + VTEntry vte = {}; uint32_t vpeid; if (!its_feature_virtual(s)) { @@ -XXX,XX +XXX,XX @@ static void vmovp_callback(gpointer data, gpointer opaque) */ GICv3ITSState *s = data; VmovpCallbackData *cbdata = opaque; - VTEntry vte; + VTEntry vte = {}; ItsCmdResult cmdres; cmdres = lookup_vte(s, __func__, cbdata->vpeid, &vte); @@ -XXX,XX +XXX,XX @@ static ItsCmdResult process_vmovi(GICv3ITSState *s, const uint64_t *cmdpkt) { uint32_t devid, eventid, vpeid, doorbell; bool doorbell_valid; - DTEntry dte; - ITEntry ite; - VTEntry old_vte, new_vte; + DTEntry dte = {}; + ITEntry ite = {}; + VTEntry old_vte = {}, new_vte = {}; ItsCmdResult cmdres; if (!its_feature_virtual(s)) { @@ -XXX,XX +XXX,XX @@ static ItsCmdResult process_vinvall(GICv3ITSState *s, const uint64_t *cmdpkt) static ItsCmdResult process_inv(GICv3ITSState *s, const uint64_t *cmdpkt) { uint32_t devid, eventid; - ITEntry ite; - DTEntry dte; - CTEntry cte; - VTEntry vte; + ITEntry ite = {}; + DTEntry dte = {}; + CTEntry cte = {}; + VTEntry vte = {}; ItsCmdResult cmdres; devid = FIELD_EX64(cmdpkt[0], INV_0, DEVICEID); -- 2.34.1 From: Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org> Update the URLs for the binaries we use for the firmware in the sbsa-ref functional tests. The firmware is built using Debian 'bookworm' cross toolchain (gcc 12.2.0). Used versions: - Trusted Firmware v2.12.0 - Tianocore EDK2 stable202411 - Tianocore EDK2 Platforms code commit 4b3530d This allows us to move away from "some git commit on trunk" to a stable release for both TF-A and EDK2. Signed-off-by: Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org> Message-id: 20241125125448.185504-1-marcin.juszkiewicz@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- tests/functional/test_aarch64_sbsaref.py | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/tests/functional/test_aarch64_sbsaref.py b/tests/functional/test_aarch64_sbsaref.py index XXXXXXX..XXXXXXX 100755 --- a/tests/functional/test_aarch64_sbsaref.py +++ b/tests/functional/test_aarch64_sbsaref.py @@ -XXX,XX +XXX,XX @@ def fetch_firmware(test): Used components: - - Trusted Firmware v2.11.0 - - Tianocore EDK2 4d4f569924 - - Tianocore EDK2-platforms 3f08401 + - Trusted Firmware v2.12.0 + - Tianocore EDK2 edk2-stable202411 + - Tianocore EDK2-platforms 4b3530d """ @@ -XXX,XX +XXX,XX @@ class Aarch64SbsarefMachine(QemuSystemTest): ASSET_FLASH0 = Asset( ('https://artifacts.codelinaro.org/artifactory/linaro-419-sbsa-ref/' - '20240619-148232/edk2/SBSA_FLASH0.fd.xz'), - '0c954842a590988f526984de22e21ae0ab9cb351a0c99a8a58e928f0c7359cf7') + '20241122-189881/edk2/SBSA_FLASH0.fd.xz'), + '76eb89d42eebe324e4395329f47447cda9ac920aabcf99aca85424609c3384a5') ASSET_FLASH1 = Asset( ('https://artifacts.codelinaro.org/artifactory/linaro-419-sbsa-ref/' - '20240619-148232/edk2/SBSA_FLASH1.fd.xz'), - 'c6ec39374c4d79bb9e9cdeeb6db44732d90bb4a334cec92002b3f4b9cac4b5ee') + '20241122-189881/edk2/SBSA_FLASH1.fd.xz'), + 'f850f243bd8dbd49c51e061e0f79f1697546938f454aeb59ab7d93e5f0d412fc') def test_sbsaref_edk2_firmware(self): @@ -XXX,XX +XXX,XX @@ def test_sbsaref_edk2_firmware(self): # AP Trusted ROM wait_for_console_pattern(self, "Booting Trusted Firmware") - wait_for_console_pattern(self, "BL1: v2.11.0(release):") + wait_for_console_pattern(self, "BL1: v2.12.0(release):") wait_for_console_pattern(self, "BL1: Booting BL2") # Trusted Boot Firmware - wait_for_console_pattern(self, "BL2: v2.11.0(release)") + wait_for_console_pattern(self, "BL2: v2.12.0(release)") wait_for_console_pattern(self, "Booting BL31") # EL3 Runtime Software - wait_for_console_pattern(self, "BL31: v2.11.0(release)") + wait_for_console_pattern(self, "BL31: v2.12.0(release)") # Non-trusted Firmware wait_for_console_pattern(self, "UEFI firmware (version 1.0") -- 2.34.1