Most of this is the Neon decodetree patches, followed by Edgar's versal cleanups. thanks -- PMM The following changes since commit 2ef486e76d64436be90f7359a3071fb2a56ce835: Merge remote-tracking branch 'remotes/marcel/tags/rdma-pull-request' into staging (2020-05-03 14:12:56 +0100) are available in the Git repository at: https://git.linaro.org/people/pmaydell/qemu-arm.git tags/pull-target-arm-20200504 for you to fetch changes up to 9aefc6cf9b73f66062d2f914a0136756e7a28211: target/arm: Move gen_ function typedefs to translate.h (2020-05-04 12:59:26 +0100) ---------------------------------------------------------------- target-arm queue: * Start of conversion of Neon insns to decodetree * versal board: support SD and RTC * Implement ARMv8.2-TTS2UXN * Make VQDMULL undefined when U=1 * Some minor code cleanups ---------------------------------------------------------------- Edgar E. Iglesias (11): hw/arm: versal: Remove inclusion of arm_gicv3_common.h hw/arm: versal: Move misplaced comment hw/arm: versal-virt: Fix typo xlnx-ve -> xlnx-versal hw/arm: versal: Embed the UARTs into the SoC type hw/arm: versal: Embed the GEMs into the SoC type hw/arm: versal: Embed the ADMAs into the SoC type hw/arm: versal: Embed the APUs into the SoC type hw/arm: versal: Add support for SD hw/arm: versal: Add support for the RTC hw/arm: versal-virt: Add support for SD hw/arm: versal-virt: Add support for the RTC Fredrik Strupe (1): target/arm: Make VQDMULL undefined when U=1 Peter Maydell (25): target/arm: Don't use a TLB for ARMMMUIdx_Stage2 target/arm: Use enum constant in get_phys_addr_lpae() call target/arm: Add new 's1_is_el0' argument to get_phys_addr_lpae() target/arm: Implement ARMv8.2-TTS2UXN target/arm: Use correct variable for setting 'max' cpu's ID_AA64DFR0 target/arm/translate-vfp.inc.c: Remove duplicate simd_r32 check target/arm: Don't allow Thumb Neon insns without FEATURE_NEON target/arm: Add stubs for AArch32 Neon decodetree target/arm: Convert VCMLA (vector) to decodetree target/arm: Convert VCADD (vector) to decodetree target/arm: Convert V[US]DOT (vector) to decodetree target/arm: Convert VFM[AS]L (vector) to decodetree target/arm: Convert VCMLA (scalar) to decodetree target/arm: Convert V[US]DOT (scalar) to decodetree target/arm: Convert VFM[AS]L (scalar) to decodetree target/arm: Convert Neon load/store multiple structures to decodetree target/arm: Convert Neon 'load single structure to all lanes' to decodetree target/arm: Convert Neon 'load/store single structure' to decodetree target/arm: Convert Neon 3-reg-same VADD/VSUB to decodetree target/arm: Convert Neon 3-reg-same logic ops to decodetree target/arm: Convert Neon 3-reg-same VMAX/VMIN to decodetree target/arm: Convert Neon 3-reg-same comparisons to decodetree target/arm: Convert Neon 3-reg-same VQADD/VQSUB to decodetree target/arm: Convert Neon 3-reg-same VMUL, VMLA, VMLS, VSHL to decodetree target/arm: Move gen_ function typedefs to translate.h Philippe Mathieu-Daudé (2): hw/arm/mps2-tz: Use TYPE_IOTKIT instead of hardcoded string target/arm: Use uint64_t for midr field in CPU state struct include/hw/arm/xlnx-versal.h | 31 +- target/arm/cpu-param.h | 2 +- target/arm/cpu.h | 38 ++- target/arm/translate-a64.h | 9 - target/arm/translate.h | 26 ++ target/arm/neon-dp.decode | 86 +++++ target/arm/neon-ls.decode | 52 +++ target/arm/neon-shared.decode | 66 ++++ hw/arm/mps2-tz.c | 2 +- hw/arm/xlnx-versal-virt.c | 74 ++++- hw/arm/xlnx-versal.c | 115 +++++-- target/arm/cpu.c | 3 +- target/arm/cpu64.c | 8 +- target/arm/helper.c | 183 ++++------ target/arm/translate-a64.c | 17 - target/arm/translate-neon.inc.c | 714 +++++++++++++++++++++++++++++++++++++++ target/arm/translate-vfp.inc.c | 6 - target/arm/translate.c | 716 +++------------------------------------- target/arm/Makefile.objs | 18 + 19 files changed, 1302 insertions(+), 864 deletions(-) create mode 100644 target/arm/neon-dp.decode create mode 100644 target/arm/neon-ls.decode create mode 100644 target/arm/neon-shared.decode create mode 100644 target/arm/translate-neon.inc.c From: Fredrik Strupe <fredrik@strupe.net> According to Arm ARM, VQDMULL is only valid when U=0, while having U=1 is unallocated. Signed-off-by: Fredrik Strupe <fredrik@strupe.net> Fixes: 695272dcb976 ("target-arm: Handle UNDEF cases for Neon 3-regs-different-widths") Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/target/arm/translate.c b/target/arm/translate.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate.c +++ b/target/arm/translate.c @@ -XXX,XX +XXX,XX @@ static int disas_neon_data_insn(DisasContext *s, uint32_t insn) {0, 0, 0, 0}, /* VMLSL */ {0, 0, 0, 9}, /* VQDMLSL */ {0, 0, 0, 0}, /* Integer VMULL */ - {0, 0, 0, 1}, /* VQDMULL */ + {0, 0, 0, 9}, /* VQDMULL */ {0, 0, 0, 0xa}, /* Polynomial VMULL */ {0, 0, 0, 7}, /* Reserved: always UNDEF */ }; -- 2.20.1 From: Philippe Mathieu-Daudé <f4bug@amsat.org> By using the TYPE_* definitions for devices, we can: - quickly find where devices are used with 'git-grep' - easily rename a device (one-line change). Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Message-id: 20200428154650.21991-1-f4bug@amsat.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- hw/arm/mps2-tz.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/hw/arm/mps2-tz.c b/hw/arm/mps2-tz.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/mps2-tz.c +++ b/hw/arm/mps2-tz.c @@ -XXX,XX +XXX,XX @@ static void mps2tz_common_init(MachineState *machine) exit(EXIT_FAILURE); } - sysbus_init_child_obj(OBJECT(machine), "iotkit", &mms->iotkit, + sysbus_init_child_obj(OBJECT(machine), TYPE_IOTKIT, &mms->iotkit, sizeof(mms->iotkit), mmc->armsse_type); iotkitdev = DEVICE(&mms->iotkit); object_property_set_link(OBJECT(&mms->iotkit), OBJECT(system_memory), -- 2.20.1 We define ARMMMUIdx_Stage2 as being an MMU index which uses a QEMU TLB. However we never actually use the TLB -- all stage 2 lookups are done by direct calls to get_phys_addr_lpae() followed by a physical address load via address_space_ld*(). Remove Stage2 from the list of ARM MMU indexes which correspond to real core MMU indexes, and instead put it in the set of "NOTLB" ARM MMU indexes. This allows us to drop NB_MMU_MODES to 11. It also means we can safely add support for the ARMv8.3-TTS2UXN extension, which adds permission bits to the stage 2 descriptors which define execute permission separatel for EL0 and EL1; supporting that while keeping Stage2 in a QEMU TLB would require us to use separate TLBs for "Stage2 for an EL0 access" and "Stage2 for an EL1 access", which is a lot of extra complication given we aren't even using the QEMU TLB. In the process of updating the comment on our MMU index use, fix a couple of other minor errors: * NS EL2 EL2&0 was missing from the list in the comment * some text hadn't been updated from when we bumped NB_MMU_MODES above 8 Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Edgar E. Iglesias <edgar.iglesias@xilinx.com> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200330210400.11724-2-peter.maydell@linaro.org --- target/arm/cpu-param.h | 2 +- target/arm/cpu.h | 21 +++++--- target/arm/helper.c | 112 ++++------------------------------------- 3 files changed, 27 insertions(+), 108 deletions(-) diff --git a/target/arm/cpu-param.h b/target/arm/cpu-param.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpu-param.h +++ b/target/arm/cpu-param.h @@ -XXX,XX +XXX,XX @@ # define TARGET_PAGE_BITS_MIN 10 #endif -#define NB_MMU_MODES 12 +#define NB_MMU_MODES 11 #endif diff --git a/target/arm/cpu.h b/target/arm/cpu.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -XXX,XX +XXX,XX @@ bool write_cpustate_to_list(ARMCPU *cpu, bool kvm_sync); * handling via the TLB. The only way to do a stage 1 translation without * the immediate stage 2 translation is via the ATS or AT system insns, * which can be slow-pathed and always do a page table walk. + * The only use of stage 2 translations is either as part of an s1+2 + * lookup or when loading the descriptors during a stage 1 page table walk, + * and in both those cases we don't use the TLB. * 4. we can also safely fold together the "32 bit EL3" and "64 bit EL3" * translation regimes, because they map reasonably well to each other * and they can't both be active at the same time. @@ -XXX,XX +XXX,XX @@ bool write_cpustate_to_list(ARMCPU *cpu, bool kvm_sync); * NS EL1 EL1&0 stage 1+2 (aka NS PL1) * NS EL1 EL1&0 stage 1+2 +PAN * NS EL0 EL2&0 + * NS EL2 EL2&0 * NS EL2 EL2&0 +PAN * NS EL2 (aka NS PL2) * S EL0 EL1&0 (aka S PL0) * S EL1 EL1&0 (not used if EL3 is 32 bit) * S EL1 EL1&0 +PAN * S EL3 (aka S PL1) - * NS EL1&0 stage 2 * - * for a total of 12 different mmu_idx. + * for a total of 11 different mmu_idx. * * R profile CPUs have an MPU, but can use the same set of MMU indexes * as A profile. They only need to distinguish NS EL0 and NS EL1 (and @@ -XXX,XX +XXX,XX @@ bool write_cpustate_to_list(ARMCPU *cpu, bool kvm_sync); * are not quite the same -- different CPU types (most notably M profile * vs A/R profile) would like to use MMU indexes with different semantics, * but since we don't ever need to use all of those in a single CPU we - * can avoid setting NB_MMU_MODES to more than 8. The lower bits of + * can avoid having to set NB_MMU_MODES to "total number of A profile MMU + * modes + total number of M profile MMU modes". The lower bits of * ARMMMUIdx are the core TLB mmu index, and the higher bits are always * the same for any particular CPU. * Variables of type ARMMUIdx are always full values, and the core @@ -XXX,XX +XXX,XX @@ typedef enum ARMMMUIdx { ARMMMUIdx_SE10_1_PAN = 9 | ARM_MMU_IDX_A, ARMMMUIdx_SE3 = 10 | ARM_MMU_IDX_A, - ARMMMUIdx_Stage2 = 11 | ARM_MMU_IDX_A, - /* * These are not allocated TLBs and are used only for AT system * instructions or for the first stage of an S12 page table walk. @@ -XXX,XX +XXX,XX @@ typedef enum ARMMMUIdx { ARMMMUIdx_Stage1_E0 = 0 | ARM_MMU_IDX_NOTLB, ARMMMUIdx_Stage1_E1 = 1 | ARM_MMU_IDX_NOTLB, ARMMMUIdx_Stage1_E1_PAN = 2 | ARM_MMU_IDX_NOTLB, + /* + * Not allocated a TLB: used only for second stage of an S12 page + * table walk, or for descriptor loads during first stage of an S1 + * page table walk. Note that if we ever want to have a TLB for this + * then various TLB flush insns which currently are no-ops or flush + * only stage 1 MMU indexes will need to change to flush stage 2. + */ + ARMMMUIdx_Stage2 = 3 | ARM_MMU_IDX_NOTLB, /* * M-profile. @@ -XXX,XX +XXX,XX @@ typedef enum ARMMMUIdxBit { TO_CORE_BIT(SE10_1), TO_CORE_BIT(SE10_1_PAN), TO_CORE_BIT(SE3), - TO_CORE_BIT(Stage2), TO_CORE_BIT(MUser), TO_CORE_BIT(MPriv), diff --git a/target/arm/helper.c b/target/arm/helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -XXX,XX +XXX,XX @@ static void tlbiall_nsnh_write(CPUARMState *env, const ARMCPRegInfo *ri, tlb_flush_by_mmuidx(cs, ARMMMUIdxBit_E10_1 | ARMMMUIdxBit_E10_1_PAN | - ARMMMUIdxBit_E10_0 | - ARMMMUIdxBit_Stage2); + ARMMMUIdxBit_E10_0); } static void tlbiall_nsnh_is_write(CPUARMState *env, const ARMCPRegInfo *ri, @@ -XXX,XX +XXX,XX @@ static void tlbiall_nsnh_is_write(CPUARMState *env, const ARMCPRegInfo *ri, tlb_flush_by_mmuidx_all_cpus_synced(cs, ARMMMUIdxBit_E10_1 | ARMMMUIdxBit_E10_1_PAN | - ARMMMUIdxBit_E10_0 | - ARMMMUIdxBit_Stage2); + ARMMMUIdxBit_E10_0); } -static void tlbiipas2_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) -{ - /* Invalidate by IPA. This has to invalidate any structures that - * contain only stage 2 translation information, but does not need - * to apply to structures that contain combined stage 1 and stage 2 - * translation information. - * This must NOP if EL2 isn't implemented or SCR_EL3.NS is zero. - */ - CPUState *cs = env_cpu(env); - uint64_t pageaddr; - - if (!arm_feature(env, ARM_FEATURE_EL2) || !(env->cp15.scr_el3 & SCR_NS)) { - return; - } - - pageaddr = sextract64(value << 12, 0, 40); - - tlb_flush_page_by_mmuidx(cs, pageaddr, ARMMMUIdxBit_Stage2); -} - -static void tlbiipas2_is_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) -{ - CPUState *cs = env_cpu(env); - uint64_t pageaddr; - - if (!arm_feature(env, ARM_FEATURE_EL2) || !(env->cp15.scr_el3 & SCR_NS)) { - return; - } - - pageaddr = sextract64(value << 12, 0, 40); - - tlb_flush_page_by_mmuidx_all_cpus_synced(cs, pageaddr, - ARMMMUIdxBit_Stage2); -} static void tlbiall_hyp_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value) @@ -XXX,XX +XXX,XX @@ static void vttbr_write(CPUARMState *env, const ARMCPRegInfo *ri, tlb_flush_by_mmuidx(cs, ARMMMUIdxBit_E10_1 | ARMMMUIdxBit_E10_1_PAN | - ARMMMUIdxBit_E10_0 | - ARMMMUIdxBit_Stage2); + ARMMMUIdxBit_E10_0); raw_write(env, ri, value); } } @@ -XXX,XX +XXX,XX @@ static int alle1_tlbmask(CPUARMState *env) return ARMMMUIdxBit_SE10_1 | ARMMMUIdxBit_SE10_1_PAN | ARMMMUIdxBit_SE10_0; - } else if (arm_feature(env, ARM_FEATURE_EL2)) { - return ARMMMUIdxBit_E10_1 | - ARMMMUIdxBit_E10_1_PAN | - ARMMMUIdxBit_E10_0 | - ARMMMUIdxBit_Stage2; } else { return ARMMMUIdxBit_E10_1 | ARMMMUIdxBit_E10_1_PAN | @@ -XXX,XX +XXX,XX @@ static void tlbi_aa64_vae3is_write(CPUARMState *env, const ARMCPRegInfo *ri, ARMMMUIdxBit_SE3); } -static void tlbi_aa64_ipas2e1_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) -{ - /* Invalidate by IPA. This has to invalidate any structures that - * contain only stage 2 translation information, but does not need - * to apply to structures that contain combined stage 1 and stage 2 - * translation information. - * This must NOP if EL2 isn't implemented or SCR_EL3.NS is zero. - */ - ARMCPU *cpu = env_archcpu(env); - CPUState *cs = CPU(cpu); - uint64_t pageaddr; - - if (!arm_feature(env, ARM_FEATURE_EL2) || !(env->cp15.scr_el3 & SCR_NS)) { - return; - } - - pageaddr = sextract64(value << 12, 0, 48); - - tlb_flush_page_by_mmuidx(cs, pageaddr, ARMMMUIdxBit_Stage2); -} - -static void tlbi_aa64_ipas2e1is_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) -{ - CPUState *cs = env_cpu(env); - uint64_t pageaddr; - - if (!arm_feature(env, ARM_FEATURE_EL2) || !(env->cp15.scr_el3 & SCR_NS)) { - return; - } - - pageaddr = sextract64(value << 12, 0, 48); - - tlb_flush_page_by_mmuidx_all_cpus_synced(cs, pageaddr, - ARMMMUIdxBit_Stage2); -} - static CPAccessResult aa64_zva_access(CPUARMState *env, const ARMCPRegInfo *ri, bool isread) { @@ -XXX,XX +XXX,XX @@ static const ARMCPRegInfo v8_cp_reginfo[] = { .writefn = tlbi_aa64_vae1_write }, { .name = "TLBI_IPAS2E1IS", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 0, .opc2 = 1, - .access = PL2_W, .type = ARM_CP_NO_RAW, - .writefn = tlbi_aa64_ipas2e1is_write }, + .access = PL2_W, .type = ARM_CP_NOP }, { .name = "TLBI_IPAS2LE1IS", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 0, .opc2 = 5, - .access = PL2_W, .type = ARM_CP_NO_RAW, - .writefn = tlbi_aa64_ipas2e1is_write }, + .access = PL2_W, .type = ARM_CP_NOP }, { .name = "TLBI_ALLE1IS", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 3, .opc2 = 4, .access = PL2_W, .type = ARM_CP_NO_RAW, @@ -XXX,XX +XXX,XX @@ static const ARMCPRegInfo v8_cp_reginfo[] = { .writefn = tlbi_aa64_alle1is_write }, { .name = "TLBI_IPAS2E1", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 4, .opc2 = 1, - .access = PL2_W, .type = ARM_CP_NO_RAW, - .writefn = tlbi_aa64_ipas2e1_write }, + .access = PL2_W, .type = ARM_CP_NOP }, { .name = "TLBI_IPAS2LE1", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 4, .opc2 = 5, - .access = PL2_W, .type = ARM_CP_NO_RAW, - .writefn = tlbi_aa64_ipas2e1_write }, + .access = PL2_W, .type = ARM_CP_NOP }, { .name = "TLBI_ALLE1", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 7, .opc2 = 4, .access = PL2_W, .type = ARM_CP_NO_RAW, @@ -XXX,XX +XXX,XX @@ static const ARMCPRegInfo v8_cp_reginfo[] = { .writefn = tlbimva_hyp_is_write }, { .name = "TLBIIPAS2", .cp = 15, .opc1 = 4, .crn = 8, .crm = 4, .opc2 = 1, - .type = ARM_CP_NO_RAW, .access = PL2_W, - .writefn = tlbiipas2_write }, + .type = ARM_CP_NOP, .access = PL2_W }, { .name = "TLBIIPAS2IS", .cp = 15, .opc1 = 4, .crn = 8, .crm = 0, .opc2 = 1, - .type = ARM_CP_NO_RAW, .access = PL2_W, - .writefn = tlbiipas2_is_write }, + .type = ARM_CP_NOP, .access = PL2_W }, { .name = "TLBIIPAS2L", .cp = 15, .opc1 = 4, .crn = 8, .crm = 4, .opc2 = 5, - .type = ARM_CP_NO_RAW, .access = PL2_W, - .writefn = tlbiipas2_write }, + .type = ARM_CP_NOP, .access = PL2_W }, { .name = "TLBIIPAS2LIS", .cp = 15, .opc1 = 4, .crn = 8, .crm = 0, .opc2 = 5, - .type = ARM_CP_NO_RAW, .access = PL2_W, - .writefn = tlbiipas2_is_write }, + .type = ARM_CP_NOP, .access = PL2_W }, /* 32 bit cache operations */ { .name = "ICIALLUIS", .cp = 15, .opc1 = 0, .crn = 7, .crm = 1, .opc2 = 0, .type = ARM_CP_NOP, .access = PL1_W, .accessfn = aa64_cacheop_pou_access }, -- 2.20.1 The access_type argument to get_phys_addr_lpae() is an MMUAccessType; use the enum constant MMU_DATA_LOAD rather than a literal 0 when we call it in S1_ptw_translate(). Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Edgar E. Iglesias <edgar.iglesias@xilinx.com> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200330210400.11724-3-peter.maydell@linaro.org --- target/arm/helper.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/target/arm/helper.c b/target/arm/helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -XXX,XX +XXX,XX @@ static hwaddr S1_ptw_translate(CPUARMState *env, ARMMMUIdx mmu_idx, pcacheattrs = &cacheattrs; } - ret = get_phys_addr_lpae(env, addr, 0, ARMMMUIdx_Stage2, &s2pa, - &txattrs, &s2prot, &s2size, fi, pcacheattrs); + ret = get_phys_addr_lpae(env, addr, MMU_DATA_LOAD, ARMMMUIdx_Stage2, + &s2pa, &txattrs, &s2prot, &s2size, fi, + pcacheattrs); if (ret) { assert(fi->type != ARMFault_None); fi->s2addr = addr; -- 2.20.1 For ARMv8.2-TTS2UXN, the stage 2 page table walk wants to know whether the stage 1 access is for EL0 or not, because whether exec permission is given can depend on whether this is an EL0 or EL1 access. Add a new argument to get_phys_addr_lpae() so the call sites can pass this information in. Since get_phys_addr_lpae() doesn't already have a doc comment, add one so we have a place to put the documentation of the semantics of the new s1_is_el0 argument. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Edgar E. Iglesias <edgar.iglesias@xilinx.com> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200330210400.11724-4-peter.maydell@linaro.org --- target/arm/helper.c | 29 ++++++++++++++++++++++++++++- 1 file changed, 28 insertions(+), 1 deletion(-) diff --git a/target/arm/helper.c b/target/arm/helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -XXX,XX +XXX,XX @@ static bool get_phys_addr_lpae(CPUARMState *env, target_ulong address, MMUAccessType access_type, ARMMMUIdx mmu_idx, + bool s1_is_el0, hwaddr *phys_ptr, MemTxAttrs *txattrs, int *prot, target_ulong *page_size_ptr, ARMMMUFaultInfo *fi, ARMCacheAttrs *cacheattrs); @@ -XXX,XX +XXX,XX @@ static hwaddr S1_ptw_translate(CPUARMState *env, ARMMMUIdx mmu_idx, } ret = get_phys_addr_lpae(env, addr, MMU_DATA_LOAD, ARMMMUIdx_Stage2, + false, &s2pa, &txattrs, &s2prot, &s2size, fi, pcacheattrs); if (ret) { @@ -XXX,XX +XXX,XX @@ static ARMVAParameters aa32_va_parameters(CPUARMState *env, uint32_t va, }; } +/** + * get_phys_addr_lpae: perform one stage of page table walk, LPAE format + * + * Returns false if the translation was successful. Otherwise, phys_ptr, attrs, + * prot and page_size may not be filled in, and the populated fsr value provides + * information on why the translation aborted, in the format of a long-format + * DFSR/IFSR fault register, with the following caveats: + * * the WnR bit is never set (the caller must do this). + * + * @env: CPUARMState + * @address: virtual address to get physical address for + * @access_type: MMU_DATA_LOAD, MMU_DATA_STORE or MMU_INST_FETCH + * @mmu_idx: MMU index indicating required translation regime + * @s1_is_el0: if @mmu_idx is ARMMMUIdx_Stage2 (so this is a stage 2 page table + * walk), must be true if this is stage 2 of a stage 1+2 walk for an + * EL0 access). If @mmu_idx is anything else, @s1_is_el0 is ignored. + * @phys_ptr: set to the physical address corresponding to the virtual address + * @attrs: set to the memory transaction attributes to use + * @prot: set to the permissions for the page containing phys_ptr + * @page_size_ptr: set to the size of the page containing phys_ptr + * @fi: set to fault info if the translation fails + * @cacheattrs: (if non-NULL) set to the cacheability/shareability attributes + */ static bool get_phys_addr_lpae(CPUARMState *env, target_ulong address, MMUAccessType access_type, ARMMMUIdx mmu_idx, + bool s1_is_el0, hwaddr *phys_ptr, MemTxAttrs *txattrs, int *prot, target_ulong *page_size_ptr, ARMMMUFaultInfo *fi, ARMCacheAttrs *cacheattrs) @@ -XXX,XX +XXX,XX @@ bool get_phys_addr(CPUARMState *env, target_ulong address, /* S1 is done. Now do S2 translation. */ ret = get_phys_addr_lpae(env, ipa, access_type, ARMMMUIdx_Stage2, + mmu_idx == ARMMMUIdx_E10_0, phys_ptr, attrs, &s2_prot, page_size, fi, cacheattrs != NULL ? &cacheattrs2 : NULL); @@ -XXX,XX +XXX,XX @@ bool get_phys_addr(CPUARMState *env, target_ulong address, } if (regime_using_lpae_format(env, mmu_idx)) { - return get_phys_addr_lpae(env, address, access_type, mmu_idx, + return get_phys_addr_lpae(env, address, access_type, mmu_idx, false, phys_ptr, attrs, prot, page_size, fi, cacheattrs); } else if (regime_sctlr(env, mmu_idx) & SCTLR_XP) { -- 2.20.1 The ARMv8.2-TTS2UXN feature extends the XN field in stage 2 translation table descriptors from just bit [54] to bits [54:53], allowing stage 2 to control execution permissions separately for EL0 and EL1. Implement the new semantics of the XN field and enable the feature for our 'max' CPU. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Edgar E. Iglesias <edgar.iglesias@xilinx.com> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200330210400.11724-5-peter.maydell@linaro.org --- target/arm/cpu.h | 15 +++++++++++++++ target/arm/cpu.c | 1 + target/arm/cpu64.c | 2 ++ target/arm/helper.c | 37 +++++++++++++++++++++++++++++++------ 4 files changed, 49 insertions(+), 6 deletions(-) diff --git a/target/arm/cpu.h b/target/arm/cpu.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -XXX,XX +XXX,XX @@ static inline bool isar_feature_aa32_ccidx(const ARMISARegisters *id) return FIELD_EX32(id->id_mmfr4, ID_MMFR4, CCIDX) != 0; } +static inline bool isar_feature_aa32_tts2uxn(const ARMISARegisters *id) +{ + return FIELD_EX32(id->id_mmfr4, ID_MMFR4, XNX) != 0; +} + /* * 64-bit feature tests via id registers. */ @@ -XXX,XX +XXX,XX @@ static inline bool isar_feature_aa64_ccidx(const ARMISARegisters *id) return FIELD_EX64(id->id_aa64mmfr2, ID_AA64MMFR2, CCIDX) != 0; } +static inline bool isar_feature_aa64_tts2uxn(const ARMISARegisters *id) +{ + return FIELD_EX64(id->id_aa64mmfr1, ID_AA64MMFR1, XNX) != 0; +} + /* * Feature tests for "does this exist in either 32-bit or 64-bit?" */ @@ -XXX,XX +XXX,XX @@ static inline bool isar_feature_any_ccidx(const ARMISARegisters *id) return isar_feature_aa64_ccidx(id) || isar_feature_aa32_ccidx(id); } +static inline bool isar_feature_any_tts2uxn(const ARMISARegisters *id) +{ + return isar_feature_aa64_tts2uxn(id) || isar_feature_aa32_tts2uxn(id); +} + /* * Forward to the above feature tests given an ARMCPU pointer. */ diff --git a/target/arm/cpu.c b/target/arm/cpu.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpu.c +++ b/target/arm/cpu.c @@ -XXX,XX +XXX,XX @@ static void arm_max_initfn(Object *obj) t = FIELD_DP32(t, ID_MMFR4, HPDS, 1); /* AA32HPD */ t = FIELD_DP32(t, ID_MMFR4, AC2, 1); /* ACTLR2, HACTLR2 */ t = FIELD_DP32(t, ID_MMFR4, CNP, 1); /* TTCNP */ + t = FIELD_DP32(t, ID_MMFR4, XNX, 1); /* TTS2UXN */ cpu->isar.id_mmfr4 = t; } #endif diff --git a/target/arm/cpu64.c b/target/arm/cpu64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpu64.c +++ b/target/arm/cpu64.c @@ -XXX,XX +XXX,XX @@ static void aarch64_max_initfn(Object *obj) t = FIELD_DP64(t, ID_AA64MMFR1, VH, 1); t = FIELD_DP64(t, ID_AA64MMFR1, PAN, 2); /* ATS1E1 */ t = FIELD_DP64(t, ID_AA64MMFR1, VMIDBITS, 2); /* VMID16 */ + t = FIELD_DP64(t, ID_AA64MMFR1, XNX, 1); /* TTS2UXN */ cpu->isar.id_aa64mmfr1 = t; t = cpu->isar.id_aa64mmfr2; @@ -XXX,XX +XXX,XX @@ static void aarch64_max_initfn(Object *obj) u = FIELD_DP32(u, ID_MMFR4, HPDS, 1); /* AA32HPD */ u = FIELD_DP32(u, ID_MMFR4, AC2, 1); /* ACTLR2, HACTLR2 */ u = FIELD_DP32(u, ID_MMFR4, CNP, 1); /* TTCNP */ + u = FIELD_DP32(u, ID_MMFR4, XNX, 1); /* TTS2UXN */ cpu->isar.id_mmfr4 = u; u = cpu->isar.id_aa64dfr0; diff --git a/target/arm/helper.c b/target/arm/helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -XXX,XX +XXX,XX @@ simple_ap_to_rw_prot(CPUARMState *env, ARMMMUIdx mmu_idx, int ap) * * @env: CPUARMState * @s2ap: The 2-bit stage2 access permissions (S2AP) - * @xn: XN (execute-never) bit + * @xn: XN (execute-never) bits + * @s1_is_el0: true if this is S2 of an S1+2 walk for EL0 */ -static int get_S2prot(CPUARMState *env, int s2ap, int xn) +static int get_S2prot(CPUARMState *env, int s2ap, int xn, bool s1_is_el0) { int prot = 0; @@ -XXX,XX +XXX,XX @@ static int get_S2prot(CPUARMState *env, int s2ap, int xn) if (s2ap & 2) { prot |= PAGE_WRITE; } - if (!xn) { - if (arm_el_is_aa64(env, 2) || prot & PAGE_READ) { + + if (cpu_isar_feature(any_tts2uxn, env_archcpu(env))) { + switch (xn) { + case 0: prot |= PAGE_EXEC; + break; + case 1: + if (s1_is_el0) { + prot |= PAGE_EXEC; + } + break; + case 2: + break; + case 3: + if (!s1_is_el0) { + prot |= PAGE_EXEC; + } + break; + default: + g_assert_not_reached(); + } + } else { + if (!extract32(xn, 1, 1)) { + if (arm_el_is_aa64(env, 2) || prot & PAGE_READ) { + prot |= PAGE_EXEC; + } } } return prot; @@ -XXX,XX +XXX,XX @@ static bool get_phys_addr_lpae(CPUARMState *env, target_ulong address, } ap = extract32(attrs, 4, 2); - xn = extract32(attrs, 12, 1); if (mmu_idx == ARMMMUIdx_Stage2) { ns = true; - *prot = get_S2prot(env, ap, xn); + xn = extract32(attrs, 11, 2); + *prot = get_S2prot(env, ap, xn, s1_is_el0); } else { ns = extract32(attrs, 3, 1); + xn = extract32(attrs, 12, 1); pxn = extract32(attrs, 11, 1); *prot = get_S1prot(env, mmu_idx, aarch64, ap, ns, xn, pxn); } -- 2.20.1 In aarch64_max_initfn() we update both 32-bit and 64-bit ID registers. The intended pattern is that for 64-bit ID registers we use FIELD_DP64 and the uint64_t 't' register, while 32-bit ID registers use FIELD_DP32 and the uint32_t 'u' register. For ID_AA64DFR0 we accidentally used 'u', meaning that the top 32 bits of this 64-bit ID register would end up always zero. Luckily at the moment that's what they should be anyway, so this bug has no visible effects. Use the right-sized variable. Fixes: 3bec78447a958d481991 Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Laurent Desnogues <laurent.desnogues@gmail.com> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Message-id: 20200423110915.10527-1-peter.maydell@linaro.org --- target/arm/cpu64.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/target/arm/cpu64.c b/target/arm/cpu64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpu64.c +++ b/target/arm/cpu64.c @@ -XXX,XX +XXX,XX @@ static void aarch64_max_initfn(Object *obj) u = FIELD_DP32(u, ID_MMFR4, XNX, 1); /* TTS2UXN */ cpu->isar.id_mmfr4 = u; - u = cpu->isar.id_aa64dfr0; - u = FIELD_DP64(u, ID_AA64DFR0, PMUVER, 5); /* v8.4-PMU */ - cpu->isar.id_aa64dfr0 = u; + t = cpu->isar.id_aa64dfr0; + t = FIELD_DP64(t, ID_AA64DFR0, PMUVER, 5); /* v8.4-PMU */ + cpu->isar.id_aa64dfr0 = t; u = cpu->isar.id_dfr0; u = FIELD_DP32(u, ID_DFR0, PERFMON, 5); /* v8.4-PMU */ -- 2.20.1 From: Philippe Mathieu-Daudé <f4bug@amsat.org> MIDR_EL1 is a 64-bit system register with the top 32-bit being RES0. Represent it in QEMU's ARMCPU struct with a uint64_t, not a uint32_t. This fixes an error when compiling with -Werror=conversion because we were manipulating the register value using a local uint64_t variable: target/arm/cpu64.c: In function ‘aarch64_max_initfn’: target/arm/cpu64.c:628:21: error: conversion from ‘uint64_t’ {aka ‘long unsigned int’} to ‘uint32_t’ {aka ‘unsigned int’} may change value [-Werror=conversion] 628 | cpu->midr = t; | ^ and future-proofs us against a possible future architecture change using some of the top 32 bits. Suggested-by: Laurent Desnogues <laurent.desnogues@gmail.com> Suggested-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Reviewed-by: Laurent Desnogues <laurent.desnogues@gmail.com> Message-id: 20200428172634.29707-1-f4bug@amsat.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/cpu.h | 2 +- target/arm/cpu.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/target/arm/cpu.h b/target/arm/cpu.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -XXX,XX +XXX,XX @@ struct ARMCPU { uint64_t id_aa64dfr0; uint64_t id_aa64dfr1; } isar; - uint32_t midr; + uint64_t midr; uint32_t revidr; uint32_t reset_fpsid; uint32_t ctr; diff --git a/target/arm/cpu.c b/target/arm/cpu.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpu.c +++ b/target/arm/cpu.c @@ -XXX,XX +XXX,XX @@ static const ARMCPUInfo arm_cpus[] = { static Property arm_cpu_properties[] = { DEFINE_PROP_BOOL("start-powered-off", ARMCPU, start_powered_off, false), DEFINE_PROP_UINT32("psci-conduit", ARMCPU, psci_conduit, 0), - DEFINE_PROP_UINT32("midr", ARMCPU, midr, 0), + DEFINE_PROP_UINT64("midr", ARMCPU, midr, 0), DEFINE_PROP_UINT64("mp-affinity", ARMCPU, mp_affinity, ARM64_AFFINITY_INVALID), DEFINE_PROP_INT32("node-id", ARMCPU, node_id, CPU_UNSET_NUMA_NODE_ID), -- 2.20.1 From: "Edgar E. Iglesias" <edgar.iglesias@xilinx.com> Remove inclusion of arm_gicv3_common.h, this already gets included via xlnx-versal.h. Signed-off-by: Edgar E. Iglesias <edgar.iglesias@xilinx.com> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> Reviewed-by: Luc Michel <luc.michel@greensocs.com> Message-id: 20200427181649.26851-2-edgar.iglesias@gmail.com Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- hw/arm/xlnx-versal.c | 1 - 1 file changed, 1 deletion(-) diff --git a/hw/arm/xlnx-versal.c b/hw/arm/xlnx-versal.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/xlnx-versal.c +++ b/hw/arm/xlnx-versal.c @@ -XXX,XX +XXX,XX @@ #include "hw/arm/boot.h" #include "kvm_arm.h" #include "hw/misc/unimp.h" -#include "hw/intc/arm_gicv3_common.h" #include "hw/arm/xlnx-versal.h" #include "hw/char/pl011.h" -- 2.20.1 From: "Edgar E. Iglesias" <edgar.iglesias@xilinx.com> Move misplaced comment. Signed-off-by: Edgar E. Iglesias <edgar.iglesias@xilinx.com> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Reviewed-by: Luc Michel <luc.michel@greensocs.com> Message-id: 20200427181649.26851-3-edgar.iglesias@gmail.com Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- hw/arm/xlnx-versal.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/hw/arm/xlnx-versal.c b/hw/arm/xlnx-versal.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/xlnx-versal.c +++ b/hw/arm/xlnx-versal.c @@ -XXX,XX +XXX,XX @@ static void versal_create_apu_cpus(Versal *s) obj = object_new(XLNX_VERSAL_ACPU_TYPE); if (!obj) { - /* Secondary CPUs start in PSCI powered-down state */ error_report("Unable to create apu.cpu[%d] of type %s", i, XLNX_VERSAL_ACPU_TYPE); exit(EXIT_FAILURE); @@ -XXX,XX +XXX,XX @@ static void versal_create_apu_cpus(Versal *s) object_property_set_int(obj, s->cfg.psci_conduit, "psci-conduit", &error_abort); if (i) { + /* Secondary CPUs start in PSCI powered-down state */ object_property_set_bool(obj, true, "start-powered-off", &error_abort); } -- 2.20.1 From: "Edgar E. Iglesias" <edgar.iglesias@xilinx.com> Fix typo xlnx-ve -> xlnx-versal. Signed-off-by: Edgar E. Iglesias <edgar.iglesias@xilinx.com> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Reviewed-by: Luc Michel <luc.michel@greensocs.com> Message-id: 20200427181649.26851-4-edgar.iglesias@gmail.com Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- hw/arm/xlnx-versal-virt.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/hw/arm/xlnx-versal-virt.c b/hw/arm/xlnx-versal-virt.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/xlnx-versal-virt.c +++ b/hw/arm/xlnx-versal-virt.c @@ -XXX,XX +XXX,XX @@ static void versal_virt_init(MachineState *machine) psci_conduit = QEMU_PSCI_CONDUIT_SMC; } - sysbus_init_child_obj(OBJECT(machine), "xlnx-ve", &s->soc, + sysbus_init_child_obj(OBJECT(machine), "xlnx-versal", &s->soc, sizeof(s->soc), TYPE_XLNX_VERSAL); object_property_set_link(OBJECT(&s->soc), OBJECT(machine->ram), "ddr", &error_abort); -- 2.20.1 From: "Edgar E. Iglesias" <edgar.iglesias@xilinx.com> Embed the UARTs into the SoC type. Suggested-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Edgar E. Iglesias <edgar.iglesias@xilinx.com> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Reviewed-by: Luc Michel <luc.michel@greensocs.com> Message-id: 20200427181649.26851-5-edgar.iglesias@gmail.com Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- include/hw/arm/xlnx-versal.h | 3 ++- hw/arm/xlnx-versal.c | 12 ++++++------ 2 files changed, 8 insertions(+), 7 deletions(-) diff --git a/include/hw/arm/xlnx-versal.h b/include/hw/arm/xlnx-versal.h index XXXXXXX..XXXXXXX 100644 --- a/include/hw/arm/xlnx-versal.h +++ b/include/hw/arm/xlnx-versal.h @@ -XXX,XX +XXX,XX @@ #include "hw/sysbus.h" #include "hw/arm/boot.h" #include "hw/intc/arm_gicv3.h" +#include "hw/char/pl011.h" #define TYPE_XLNX_VERSAL "xlnx-versal" #define XLNX_VERSAL(obj) OBJECT_CHECK(Versal, (obj), TYPE_XLNX_VERSAL) @@ -XXX,XX +XXX,XX @@ typedef struct Versal { MemoryRegion mr_ocm; struct { - SysBusDevice *uart[XLNX_VERSAL_NR_UARTS]; + PL011State uart[XLNX_VERSAL_NR_UARTS]; SysBusDevice *gem[XLNX_VERSAL_NR_GEMS]; SysBusDevice *adma[XLNX_VERSAL_NR_ADMAS]; } iou; diff --git a/hw/arm/xlnx-versal.c b/hw/arm/xlnx-versal.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/xlnx-versal.c +++ b/hw/arm/xlnx-versal.c @@ -XXX,XX +XXX,XX @@ #include "kvm_arm.h" #include "hw/misc/unimp.h" #include "hw/arm/xlnx-versal.h" -#include "hw/char/pl011.h" #define XLNX_VERSAL_ACPU_TYPE ARM_CPU_TYPE_NAME("cortex-a72") #define GEM_REVISION 0x40070106 @@ -XXX,XX +XXX,XX @@ static void versal_create_uarts(Versal *s, qemu_irq *pic) DeviceState *dev; MemoryRegion *mr; - dev = qdev_create(NULL, TYPE_PL011); - s->lpd.iou.uart[i] = SYS_BUS_DEVICE(dev); + sysbus_init_child_obj(OBJECT(s), name, + &s->lpd.iou.uart[i], sizeof(s->lpd.iou.uart[i]), + TYPE_PL011); + dev = DEVICE(&s->lpd.iou.uart[i]); qdev_prop_set_chr(dev, "chardev", serial_hd(i)); - object_property_add_child(OBJECT(s), name, OBJECT(dev), &error_fatal); qdev_init_nofail(dev); - mr = sysbus_mmio_get_region(s->lpd.iou.uart[i], 0); + mr = sysbus_mmio_get_region(SYS_BUS_DEVICE(dev), 0); memory_region_add_subregion(&s->mr_ps, addrs[i], mr); - sysbus_connect_irq(s->lpd.iou.uart[i], 0, pic[irqs[i]]); + sysbus_connect_irq(SYS_BUS_DEVICE(dev), 0, pic[irqs[i]]); g_free(name); } } -- 2.20.1 From: "Edgar E. Iglesias" <edgar.iglesias@xilinx.com> Embed the GEMs into the SoC type. Suggested-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Edgar E. Iglesias <edgar.iglesias@xilinx.com> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Reviewed-by: Luc Michel <luc.michel@greensocs.com> Message-id: 20200427181649.26851-6-edgar.iglesias@gmail.com Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- include/hw/arm/xlnx-versal.h | 3 ++- hw/arm/xlnx-versal.c | 15 ++++++++------- 2 files changed, 10 insertions(+), 8 deletions(-) diff --git a/include/hw/arm/xlnx-versal.h b/include/hw/arm/xlnx-versal.h index XXXXXXX..XXXXXXX 100644 --- a/include/hw/arm/xlnx-versal.h +++ b/include/hw/arm/xlnx-versal.h @@ -XXX,XX +XXX,XX @@ #include "hw/arm/boot.h" #include "hw/intc/arm_gicv3.h" #include "hw/char/pl011.h" +#include "hw/net/cadence_gem.h" #define TYPE_XLNX_VERSAL "xlnx-versal" #define XLNX_VERSAL(obj) OBJECT_CHECK(Versal, (obj), TYPE_XLNX_VERSAL) @@ -XXX,XX +XXX,XX @@ typedef struct Versal { struct { PL011State uart[XLNX_VERSAL_NR_UARTS]; - SysBusDevice *gem[XLNX_VERSAL_NR_GEMS]; + CadenceGEMState gem[XLNX_VERSAL_NR_GEMS]; SysBusDevice *adma[XLNX_VERSAL_NR_ADMAS]; } iou; } lpd; diff --git a/hw/arm/xlnx-versal.c b/hw/arm/xlnx-versal.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/xlnx-versal.c +++ b/hw/arm/xlnx-versal.c @@ -XXX,XX +XXX,XX @@ static void versal_create_gems(Versal *s, qemu_irq *pic) DeviceState *dev; MemoryRegion *mr; - dev = qdev_create(NULL, "cadence_gem"); - s->lpd.iou.gem[i] = SYS_BUS_DEVICE(dev); - object_property_add_child(OBJECT(s), name, OBJECT(dev), &error_fatal); + sysbus_init_child_obj(OBJECT(s), name, + &s->lpd.iou.gem[i], sizeof(s->lpd.iou.gem[i]), + TYPE_CADENCE_GEM); + dev = DEVICE(&s->lpd.iou.gem[i]); if (nd->used) { qemu_check_nic_model(nd, "cadence_gem"); qdev_set_nic_properties(dev, nd); } - object_property_set_int(OBJECT(s->lpd.iou.gem[i]), + object_property_set_int(OBJECT(dev), 2, "num-priority-queues", &error_abort); - object_property_set_link(OBJECT(s->lpd.iou.gem[i]), + object_property_set_link(OBJECT(dev), OBJECT(&s->mr_ps), "dma", &error_abort); qdev_init_nofail(dev); - mr = sysbus_mmio_get_region(s->lpd.iou.gem[i], 0); + mr = sysbus_mmio_get_region(SYS_BUS_DEVICE(dev), 0); memory_region_add_subregion(&s->mr_ps, addrs[i], mr); - sysbus_connect_irq(s->lpd.iou.gem[i], 0, pic[irqs[i]]); + sysbus_connect_irq(SYS_BUS_DEVICE(dev), 0, pic[irqs[i]]); g_free(name); } } -- 2.20.1 From: "Edgar E. Iglesias" <edgar.iglesias@xilinx.com> Embed the ADMAs into the SoC type. Suggested-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Edgar E. Iglesias <edgar.iglesias@xilinx.com> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Reviewed-by: Luc Michel <luc.michel@greensocs.com> Message-id: 20200427181649.26851-7-edgar.iglesias@gmail.com Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- include/hw/arm/xlnx-versal.h | 3 ++- hw/arm/xlnx-versal.c | 14 +++++++------- 2 files changed, 9 insertions(+), 8 deletions(-) diff --git a/include/hw/arm/xlnx-versal.h b/include/hw/arm/xlnx-versal.h index XXXXXXX..XXXXXXX 100644 --- a/include/hw/arm/xlnx-versal.h +++ b/include/hw/arm/xlnx-versal.h @@ -XXX,XX +XXX,XX @@ #include "hw/arm/boot.h" #include "hw/intc/arm_gicv3.h" #include "hw/char/pl011.h" +#include "hw/dma/xlnx-zdma.h" #include "hw/net/cadence_gem.h" #define TYPE_XLNX_VERSAL "xlnx-versal" @@ -XXX,XX +XXX,XX @@ typedef struct Versal { struct { PL011State uart[XLNX_VERSAL_NR_UARTS]; CadenceGEMState gem[XLNX_VERSAL_NR_GEMS]; - SysBusDevice *adma[XLNX_VERSAL_NR_ADMAS]; + XlnxZDMA adma[XLNX_VERSAL_NR_ADMAS]; } iou; } lpd; diff --git a/hw/arm/xlnx-versal.c b/hw/arm/xlnx-versal.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/xlnx-versal.c +++ b/hw/arm/xlnx-versal.c @@ -XXX,XX +XXX,XX @@ static void versal_create_admas(Versal *s, qemu_irq *pic) DeviceState *dev; MemoryRegion *mr; - dev = qdev_create(NULL, "xlnx.zdma"); - s->lpd.iou.adma[i] = SYS_BUS_DEVICE(dev); - object_property_set_int(OBJECT(s->lpd.iou.adma[i]), 128, "bus-width", - &error_abort); - object_property_add_child(OBJECT(s), name, OBJECT(dev), &error_fatal); + sysbus_init_child_obj(OBJECT(s), name, + &s->lpd.iou.adma[i], sizeof(s->lpd.iou.adma[i]), + TYPE_XLNX_ZDMA); + dev = DEVICE(&s->lpd.iou.adma[i]); + object_property_set_int(OBJECT(dev), 128, "bus-width", &error_abort); qdev_init_nofail(dev); - mr = sysbus_mmio_get_region(s->lpd.iou.adma[i], 0); + mr = sysbus_mmio_get_region(SYS_BUS_DEVICE(dev), 0); memory_region_add_subregion(&s->mr_ps, MM_ADMA_CH0 + i * MM_ADMA_CH0_SIZE, mr); - sysbus_connect_irq(s->lpd.iou.adma[i], 0, pic[VERSAL_ADMA_IRQ_0 + i]); + sysbus_connect_irq(SYS_BUS_DEVICE(dev), 0, pic[VERSAL_ADMA_IRQ_0 + i]); g_free(name); } } -- 2.20.1 From: "Edgar E. Iglesias" <edgar.iglesias@xilinx.com> Embed the APUs into the SoC type. Suggested-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Edgar E. Iglesias <edgar.iglesias@xilinx.com> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Reviewed-by: Luc Michel <luc.michel@greensocs.com> Message-id: 20200427181649.26851-8-edgar.iglesias@gmail.com Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- include/hw/arm/xlnx-versal.h | 2 +- hw/arm/xlnx-versal-virt.c | 4 ++-- hw/arm/xlnx-versal.c | 19 +++++-------------- 3 files changed, 8 insertions(+), 17 deletions(-) diff --git a/include/hw/arm/xlnx-versal.h b/include/hw/arm/xlnx-versal.h index XXXXXXX..XXXXXXX 100644 --- a/include/hw/arm/xlnx-versal.h +++ b/include/hw/arm/xlnx-versal.h @@ -XXX,XX +XXX,XX @@ typedef struct Versal { struct { struct { MemoryRegion mr; - ARMCPU *cpu[XLNX_VERSAL_NR_ACPUS]; + ARMCPU cpu[XLNX_VERSAL_NR_ACPUS]; GICv3State gic; } apu; } fpd; diff --git a/hw/arm/xlnx-versal-virt.c b/hw/arm/xlnx-versal-virt.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/xlnx-versal-virt.c +++ b/hw/arm/xlnx-versal-virt.c @@ -XXX,XX +XXX,XX @@ static void versal_virt_init(MachineState *machine) s->binfo.get_dtb = versal_virt_get_dtb; s->binfo.modify_dtb = versal_virt_modify_dtb; if (machine->kernel_filename) { - arm_load_kernel(s->soc.fpd.apu.cpu[0], machine, &s->binfo); + arm_load_kernel(&s->soc.fpd.apu.cpu[0], machine, &s->binfo); } else { - AddressSpace *as = arm_boot_address_space(s->soc.fpd.apu.cpu[0], + AddressSpace *as = arm_boot_address_space(&s->soc.fpd.apu.cpu[0], &s->binfo); /* Some boot-loaders (e.g u-boot) don't like blobs at address 0 (NULL). * Offset things by 4K. */ diff --git a/hw/arm/xlnx-versal.c b/hw/arm/xlnx-versal.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/xlnx-versal.c +++ b/hw/arm/xlnx-versal.c @@ -XXX,XX +XXX,XX @@ static void versal_create_apu_cpus(Versal *s) for (i = 0; i < ARRAY_SIZE(s->fpd.apu.cpu); i++) { Object *obj; - char *name; - - obj = object_new(XLNX_VERSAL_ACPU_TYPE); - if (!obj) { - error_report("Unable to create apu.cpu[%d] of type %s", - i, XLNX_VERSAL_ACPU_TYPE); - exit(EXIT_FAILURE); - } - - name = g_strdup_printf("apu-cpu[%d]", i); - object_property_add_child(OBJECT(s), name, obj, &error_fatal); - g_free(name); + object_initialize_child(OBJECT(s), "apu-cpu[*]", + &s->fpd.apu.cpu[i], sizeof(s->fpd.apu.cpu[i]), + XLNX_VERSAL_ACPU_TYPE, &error_abort, NULL); + obj = OBJECT(&s->fpd.apu.cpu[i]); object_property_set_int(obj, s->cfg.psci_conduit, "psci-conduit", &error_abort); if (i) { @@ -XXX,XX +XXX,XX @@ static void versal_create_apu_cpus(Versal *s) object_property_set_link(obj, OBJECT(&s->fpd.apu.mr), "memory", &error_abort); object_property_set_bool(obj, true, "realized", &error_fatal); - s->fpd.apu.cpu[i] = ARM_CPU(obj); } } @@ -XXX,XX +XXX,XX @@ static void versal_create_apu_gic(Versal *s, qemu_irq *pic) } for (i = 0; i < nr_apu_cpus; i++) { - DeviceState *cpudev = DEVICE(s->fpd.apu.cpu[i]); + DeviceState *cpudev = DEVICE(&s->fpd.apu.cpu[i]); int ppibase = XLNX_VERSAL_NR_IRQS + i * GIC_INTERNAL + GIC_NR_SGIS; qemu_irq maint_irq; int ti; -- 2.20.1 From: "Edgar E. Iglesias" <edgar.iglesias@xilinx.com> Add support for SD. Signed-off-by: Edgar E. Iglesias <edgar.iglesias@xilinx.com> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Reviewed-by: Luc Michel <luc.michel@greensocs.com> Message-id: 20200427181649.26851-9-edgar.iglesias@gmail.com Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- include/hw/arm/xlnx-versal.h | 12 ++++++++++++ hw/arm/xlnx-versal.c | 31 +++++++++++++++++++++++++++++++ 2 files changed, 43 insertions(+) diff --git a/include/hw/arm/xlnx-versal.h b/include/hw/arm/xlnx-versal.h index XXXXXXX..XXXXXXX 100644 --- a/include/hw/arm/xlnx-versal.h +++ b/include/hw/arm/xlnx-versal.h @@ -XXX,XX +XXX,XX @@ #include "hw/sysbus.h" #include "hw/arm/boot.h" +#include "hw/sd/sdhci.h" #include "hw/intc/arm_gicv3.h" #include "hw/char/pl011.h" #include "hw/dma/xlnx-zdma.h" @@ -XXX,XX +XXX,XX @@ #define XLNX_VERSAL_NR_UARTS 2 #define XLNX_VERSAL_NR_GEMS 2 #define XLNX_VERSAL_NR_ADMAS 8 +#define XLNX_VERSAL_NR_SDS 2 #define XLNX_VERSAL_NR_IRQS 192 typedef struct Versal { @@ -XXX,XX +XXX,XX @@ typedef struct Versal { } iou; } lpd; + /* The Platform Management Controller subsystem. */ + struct { + struct { + SDHCIState sd[XLNX_VERSAL_NR_SDS]; + } iou; + } pmc; + struct { MemoryRegion *mr_ddr; uint32_t psci_conduit; @@ -XXX,XX +XXX,XX @@ typedef struct Versal { #define VERSAL_GEM1_IRQ_0 58 #define VERSAL_GEM1_WAKE_IRQ_0 59 #define VERSAL_ADMA_IRQ_0 60 +#define VERSAL_SD0_IRQ_0 126 /* Architecturally reserved IRQs suitable for virtualization. */ #define VERSAL_RSVD_IRQ_FIRST 111 @@ -XXX,XX +XXX,XX @@ typedef struct Versal { #define MM_FPD_CRF 0xfd1a0000U #define MM_FPD_CRF_SIZE 0x140000 +#define MM_PMC_SD0 0xf1040000U +#define MM_PMC_SD0_SIZE 0x10000 #define MM_PMC_CRP 0xf1260000U #define MM_PMC_CRP_SIZE 0x10000 #endif diff --git a/hw/arm/xlnx-versal.c b/hw/arm/xlnx-versal.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/xlnx-versal.c +++ b/hw/arm/xlnx-versal.c @@ -XXX,XX +XXX,XX @@ static void versal_create_admas(Versal *s, qemu_irq *pic) } } +#define SDHCI_CAPABILITIES 0x280737ec6481 /* Same as on ZynqMP. */ +static void versal_create_sds(Versal *s, qemu_irq *pic) +{ + int i; + + for (i = 0; i < ARRAY_SIZE(s->pmc.iou.sd); i++) { + DeviceState *dev; + MemoryRegion *mr; + + sysbus_init_child_obj(OBJECT(s), "sd[*]", + &s->pmc.iou.sd[i], sizeof(s->pmc.iou.sd[i]), + TYPE_SYSBUS_SDHCI); + dev = DEVICE(&s->pmc.iou.sd[i]); + + object_property_set_uint(OBJECT(dev), + 3, "sd-spec-version", &error_fatal); + object_property_set_uint(OBJECT(dev), SDHCI_CAPABILITIES, "capareg", + &error_fatal); + object_property_set_uint(OBJECT(dev), UHS_I, "uhs", &error_fatal); + qdev_init_nofail(dev); + + mr = sysbus_mmio_get_region(SYS_BUS_DEVICE(dev), 0); + memory_region_add_subregion(&s->mr_ps, + MM_PMC_SD0 + i * MM_PMC_SD0_SIZE, mr); + + sysbus_connect_irq(SYS_BUS_DEVICE(dev), 0, + pic[VERSAL_SD0_IRQ_0 + i * 2]); + } +} + /* This takes the board allocated linear DDR memory and creates aliases * for each split DDR range/aperture on the Versal address map. */ @@ -XXX,XX +XXX,XX @@ static void versal_realize(DeviceState *dev, Error **errp) versal_create_uarts(s, pic); versal_create_gems(s, pic); versal_create_admas(s, pic); + versal_create_sds(s, pic); versal_map_ddr(s); versal_unimp(s); -- 2.20.1 From: "Edgar E. Iglesias" <edgar.iglesias@xilinx.com> hw/arm: versal: Add support for the RTC. Signed-off-by: Edgar E. Iglesias <edgar.iglesias@xilinx.com> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> Reviewed-by: Luc Michel <luc.michel@greensocs.com> Message-id: 20200427181649.26851-10-edgar.iglesias@gmail.com Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- include/hw/arm/xlnx-versal.h | 8 ++++++++ hw/arm/xlnx-versal.c | 21 +++++++++++++++++++++ 2 files changed, 29 insertions(+) diff --git a/include/hw/arm/xlnx-versal.h b/include/hw/arm/xlnx-versal.h index XXXXXXX..XXXXXXX 100644 --- a/include/hw/arm/xlnx-versal.h +++ b/include/hw/arm/xlnx-versal.h @@ -XXX,XX +XXX,XX @@ #include "hw/char/pl011.h" #include "hw/dma/xlnx-zdma.h" #include "hw/net/cadence_gem.h" +#include "hw/rtc/xlnx-zynqmp-rtc.h" #define TYPE_XLNX_VERSAL "xlnx-versal" #define XLNX_VERSAL(obj) OBJECT_CHECK(Versal, (obj), TYPE_XLNX_VERSAL) @@ -XXX,XX +XXX,XX @@ typedef struct Versal { struct { SDHCIState sd[XLNX_VERSAL_NR_SDS]; } iou; + + XlnxZynqMPRTC rtc; } pmc; struct { @@ -XXX,XX +XXX,XX @@ typedef struct Versal { #define VERSAL_GEM1_IRQ_0 58 #define VERSAL_GEM1_WAKE_IRQ_0 59 #define VERSAL_ADMA_IRQ_0 60 +#define VERSAL_RTC_APB_ERR_IRQ 121 #define VERSAL_SD0_IRQ_0 126 +#define VERSAL_RTC_ALARM_IRQ 142 +#define VERSAL_RTC_SECONDS_IRQ 143 /* Architecturally reserved IRQs suitable for virtualization. */ #define VERSAL_RSVD_IRQ_FIRST 111 @@ -XXX,XX +XXX,XX @@ typedef struct Versal { #define MM_PMC_SD0_SIZE 0x10000 #define MM_PMC_CRP 0xf1260000U #define MM_PMC_CRP_SIZE 0x10000 +#define MM_PMC_RTC 0xf12a0000 +#define MM_PMC_RTC_SIZE 0x10000 #endif diff --git a/hw/arm/xlnx-versal.c b/hw/arm/xlnx-versal.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/xlnx-versal.c +++ b/hw/arm/xlnx-versal.c @@ -XXX,XX +XXX,XX @@ static void versal_create_sds(Versal *s, qemu_irq *pic) } } +static void versal_create_rtc(Versal *s, qemu_irq *pic) +{ + SysBusDevice *sbd; + MemoryRegion *mr; + + sysbus_init_child_obj(OBJECT(s), "rtc", &s->pmc.rtc, sizeof(s->pmc.rtc), + TYPE_XLNX_ZYNQMP_RTC); + sbd = SYS_BUS_DEVICE(&s->pmc.rtc); + qdev_init_nofail(DEVICE(sbd)); + + mr = sysbus_mmio_get_region(sbd, 0); + memory_region_add_subregion(&s->mr_ps, MM_PMC_RTC, mr); + + /* + * TODO: Connect the ALARM and SECONDS interrupts once our RTC model + * supports them. + */ + sysbus_connect_irq(sbd, 1, pic[VERSAL_RTC_APB_ERR_IRQ]); +} + /* This takes the board allocated linear DDR memory and creates aliases * for each split DDR range/aperture on the Versal address map. */ @@ -XXX,XX +XXX,XX @@ static void versal_realize(DeviceState *dev, Error **errp) versal_create_gems(s, pic); versal_create_admas(s, pic); versal_create_sds(s, pic); + versal_create_rtc(s, pic); versal_map_ddr(s); versal_unimp(s); -- 2.20.1 From: "Edgar E. Iglesias" <edgar.iglesias@xilinx.com> Add support for SD. Signed-off-by: Edgar E. Iglesias <edgar.iglesias@xilinx.com> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> Reviewed-by: Luc Michel <luc.michel@greensocs.com> Message-id: 20200427181649.26851-11-edgar.iglesias@gmail.com Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- hw/arm/xlnx-versal-virt.c | 46 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 46 insertions(+) diff --git a/hw/arm/xlnx-versal-virt.c b/hw/arm/xlnx-versal-virt.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/xlnx-versal-virt.c +++ b/hw/arm/xlnx-versal-virt.c @@ -XXX,XX +XXX,XX @@ #include "hw/arm/sysbus-fdt.h" #include "hw/arm/fdt.h" #include "cpu.h" +#include "hw/qdev-properties.h" #include "hw/arm/xlnx-versal.h" #define TYPE_XLNX_VERSAL_VIRT_MACHINE MACHINE_TYPE_NAME("xlnx-versal-virt") @@ -XXX,XX +XXX,XX @@ static void fdt_add_zdma_nodes(VersalVirt *s) } } +static void fdt_add_sd_nodes(VersalVirt *s) +{ + const char clocknames[] = "clk_xin\0clk_ahb"; + const char compat[] = "arasan,sdhci-8.9a"; + int i; + + for (i = ARRAY_SIZE(s->soc.pmc.iou.sd) - 1; i >= 0; i--) { + uint64_t addr = MM_PMC_SD0 + MM_PMC_SD0_SIZE * i; + char *name = g_strdup_printf("/sdhci@%" PRIx64, addr); + + qemu_fdt_add_subnode(s->fdt, name); + + qemu_fdt_setprop_cells(s->fdt, name, "clocks", + s->phandle.clk_25Mhz, s->phandle.clk_25Mhz); + qemu_fdt_setprop(s->fdt, name, "clock-names", + clocknames, sizeof(clocknames)); + qemu_fdt_setprop_cells(s->fdt, name, "interrupts", + GIC_FDT_IRQ_TYPE_SPI, VERSAL_SD0_IRQ_0 + i * 2, + GIC_FDT_IRQ_FLAGS_LEVEL_HI); + qemu_fdt_setprop_sized_cells(s->fdt, name, "reg", + 2, addr, 2, MM_PMC_SD0_SIZE); + qemu_fdt_setprop(s->fdt, name, "compatible", compat, sizeof(compat)); + g_free(name); + } +} + static void fdt_nop_memory_nodes(void *fdt, Error **errp) { Error *err = NULL; @@ -XXX,XX +XXX,XX @@ static void create_virtio_regions(VersalVirt *s) } } +static void sd_plugin_card(SDHCIState *sd, DriveInfo *di) +{ + BlockBackend *blk = di ? blk_by_legacy_dinfo(di) : NULL; + DeviceState *card; + + card = qdev_create(qdev_get_child_bus(DEVICE(sd), "sd-bus"), TYPE_SD_CARD); + object_property_add_child(OBJECT(sd), "card[*]", OBJECT(card), + &error_fatal); + qdev_prop_set_drive(card, "drive", blk, &error_fatal); + object_property_set_bool(OBJECT(card), true, "realized", &error_fatal); +} + static void versal_virt_init(MachineState *machine) { VersalVirt *s = XLNX_VERSAL_VIRT_MACHINE(machine); int psci_conduit = QEMU_PSCI_CONDUIT_DISABLED; + int i; /* * If the user provides an Operating System to be loaded, we expect them @@ -XXX,XX +XXX,XX @@ static void versal_virt_init(MachineState *machine) fdt_add_gic_nodes(s); fdt_add_timer_nodes(s); fdt_add_zdma_nodes(s); + fdt_add_sd_nodes(s); fdt_add_cpu_nodes(s, psci_conduit); fdt_add_clk_node(s, "/clk125", 125000000, s->phandle.clk_125Mhz); fdt_add_clk_node(s, "/clk25", 25000000, s->phandle.clk_25Mhz); @@ -XXX,XX +XXX,XX @@ static void versal_virt_init(MachineState *machine) memory_region_add_subregion_overlap(get_system_memory(), 0, &s->soc.fpd.apu.mr, 0); + /* Plugin SD cards. */ + for (i = 0; i < ARRAY_SIZE(s->soc.pmc.iou.sd); i++) { + sd_plugin_card(&s->soc.pmc.iou.sd[i], drive_get_next(IF_SD)); + } + s->binfo.ram_size = machine->ram_size; s->binfo.loader_start = 0x0; s->binfo.get_dtb = versal_virt_get_dtb; -- 2.20.1 From: "Edgar E. Iglesias" <edgar.iglesias@xilinx.com> Add support for the RTC. Signed-off-by: Edgar E. Iglesias <edgar.iglesias@xilinx.com> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> Reviewed-by: Luc Michel <luc.michel@greensocs.com> Message-id: 20200427181649.26851-12-edgar.iglesias@gmail.com Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- hw/arm/xlnx-versal-virt.c | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/hw/arm/xlnx-versal-virt.c b/hw/arm/xlnx-versal-virt.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/xlnx-versal-virt.c +++ b/hw/arm/xlnx-versal-virt.c @@ -XXX,XX +XXX,XX @@ static void fdt_add_sd_nodes(VersalVirt *s) } } +static void fdt_add_rtc_node(VersalVirt *s) +{ + const char compat[] = "xlnx,zynqmp-rtc"; + const char interrupt_names[] = "alarm\0sec"; + char *name = g_strdup_printf("/rtc@%x", MM_PMC_RTC); + + qemu_fdt_add_subnode(s->fdt, name); + + qemu_fdt_setprop_cells(s->fdt, name, "interrupts", + GIC_FDT_IRQ_TYPE_SPI, VERSAL_RTC_ALARM_IRQ, + GIC_FDT_IRQ_FLAGS_LEVEL_HI, + GIC_FDT_IRQ_TYPE_SPI, VERSAL_RTC_SECONDS_IRQ, + GIC_FDT_IRQ_FLAGS_LEVEL_HI); + qemu_fdt_setprop(s->fdt, name, "interrupt-names", + interrupt_names, sizeof(interrupt_names)); + qemu_fdt_setprop_sized_cells(s->fdt, name, "reg", + 2, MM_PMC_RTC, 2, MM_PMC_RTC_SIZE); + qemu_fdt_setprop(s->fdt, name, "compatible", compat, sizeof(compat)); + g_free(name); +} + static void fdt_nop_memory_nodes(void *fdt, Error **errp) { Error *err = NULL; @@ -XXX,XX +XXX,XX @@ static void versal_virt_init(MachineState *machine) fdt_add_timer_nodes(s); fdt_add_zdma_nodes(s); fdt_add_sd_nodes(s); + fdt_add_rtc_node(s); fdt_add_cpu_nodes(s, psci_conduit); fdt_add_clk_node(s, "/clk125", 125000000, s->phandle.clk_125Mhz); fdt_add_clk_node(s, "/clk25", 25000000, s->phandle.clk_25Mhz); -- 2.20.1 Somewhere along theline we accidentally added a duplicate "using D16-D31 when they don't exist" check to do_vfm_dp() (probably an artifact of a patchseries rebase). Remove it. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Message-id: 20200430181003.21682-2-peter.maydell@linaro.org --- target/arm/translate-vfp.inc.c | 6 ------ 1 file changed, 6 deletions(-) diff --git a/target/arm/translate-vfp.inc.c b/target/arm/translate-vfp.inc.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-vfp.inc.c +++ b/target/arm/translate-vfp.inc.c @@ -XXX,XX +XXX,XX @@ static bool do_vfm_dp(DisasContext *s, arg_VFMA_dp *a, bool neg_n, bool neg_d) return false; } - /* UNDEF accesses to D16-D31 if they don't exist. */ - if (!dc_isar_feature(aa32_simd_r32, s) && - ((a->vd | a->vn | a->vm) & 0x10)) { - return false; - } - if (!vfp_access_check(s)) { return true; } -- 2.20.1 We were accidentally permitting decode of Thumb Neon insns even if the CPU didn't have the FEATURE_NEON bit set, because the feature check was being done before the call to disas_neon_data_insn() and disas_neon_ls_insn() in the Arm decoder but was omitted from the Thumb decoder. Push the feature bit check down into the called functions so it is done for both Arm and Thumb encodings. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Message-id: 20200430181003.21682-3-peter.maydell@linaro.org --- target/arm/translate.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/target/arm/translate.c b/target/arm/translate.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate.c +++ b/target/arm/translate.c @@ -XXX,XX +XXX,XX @@ static int disas_neon_ls_insn(DisasContext *s, uint32_t insn) TCGv_i32 tmp2; TCGv_i64 tmp64; + if (!arm_dc_feature(s, ARM_FEATURE_NEON)) { + return 1; + } + /* FIXME: this access check should not take precedence over UNDEF * for invalid encodings; we will generate incorrect syndrome information * for attempts to execute invalid vfp/neon encodings with FP disabled. @@ -XXX,XX +XXX,XX @@ static int disas_neon_data_insn(DisasContext *s, uint32_t insn) TCGv_ptr ptr1, ptr2, ptr3; TCGv_i64 tmp64; + if (!arm_dc_feature(s, ARM_FEATURE_NEON)) { + return 1; + } + /* FIXME: this access check should not take precedence over UNDEF * for invalid encodings; we will generate incorrect syndrome information * for attempts to execute invalid vfp/neon encodings with FP disabled. @@ -XXX,XX +XXX,XX @@ static void disas_arm_insn(DisasContext *s, unsigned int insn) if (((insn >> 25) & 7) == 1) { /* NEON Data processing. */ - if (!arm_dc_feature(s, ARM_FEATURE_NEON)) { - goto illegal_op; - } - if (disas_neon_data_insn(s, insn)) { goto illegal_op; } @@ -XXX,XX +XXX,XX @@ static void disas_arm_insn(DisasContext *s, unsigned int insn) } if ((insn & 0x0f100000) == 0x04000000) { /* NEON load/store. */ - if (!arm_dc_feature(s, ARM_FEATURE_NEON)) { - goto illegal_op; - } - if (disas_neon_ls_insn(s, insn)) { goto illegal_op; } -- 2.20.1 Add the infrastructure for building and invoking a decodetree decoder for the AArch32 Neon encodings. At the moment the new decoder covers nothing, so we always fall back to the existing hand-written decode. We follow the same pattern we did for the VFP decodetree conversion (commit 78e138bc1f672c145ef6ace74617d and following): code that deals with Neon will be moving gradually out to translate-neon.vfp.inc, which we #include into translate.c. In order to share the decode files between A32 and T32, we split Neon into 3 parts: * data-processing * load-store * 'shared' encodings The first two groups of instructions have similar but not identical A32 and T32 encodings, so we need to manually transform the T32 encoding into the A32 one before calling the decoder; the third group covers the Neon instructions which are identical in A32 and T32. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200430181003.21682-4-peter.maydell@linaro.org --- target/arm/neon-dp.decode | 29 ++++++++++++++++++++++++++ target/arm/neon-ls.decode | 29 ++++++++++++++++++++++++++ target/arm/neon-shared.decode | 27 +++++++++++++++++++++++++ target/arm/translate-neon.inc.c | 32 +++++++++++++++++++++++++++++ target/arm/translate.c | 36 +++++++++++++++++++++++++++++++-- target/arm/Makefile.objs | 18 +++++++++++++++++ 6 files changed, 169 insertions(+), 2 deletions(-) create mode 100644 target/arm/neon-dp.decode create mode 100644 target/arm/neon-ls.decode create mode 100644 target/arm/neon-shared.decode create mode 100644 target/arm/translate-neon.inc.c diff --git a/target/arm/neon-dp.decode b/target/arm/neon-dp.decode new file mode 100644 index XXXXXXX..XXXXXXX --- /dev/null +++ b/target/arm/neon-dp.decode @@ -XXX,XX +XXX,XX @@ +# AArch32 Neon data-processing instruction descriptions +# +# Copyright (c) 2020 Linaro, Ltd +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library; if not, see <http://www.gnu.org/licenses/>. + +# +# This file is processed by scripts/decodetree.py +# + +# Encodings for Neon data processing instructions where the T32 encoding +# is a simple transformation of the A32 encoding. +# More specifically, this file covers instructions where the A32 encoding is +# 0b1111_001p_qqqq_qqqq_qqqq_qqqq_qqqq_qqqq +# and the T32 encoding is +# 0b111p_1111_qqqq_qqqq_qqqq_qqqq_qqqq_qqqq +# This file works on the A32 encoding only; calling code for T32 has to +# transform the insn into the A32 version first. diff --git a/target/arm/neon-ls.decode b/target/arm/neon-ls.decode new file mode 100644 index XXXXXXX..XXXXXXX --- /dev/null +++ b/target/arm/neon-ls.decode @@ -XXX,XX +XXX,XX @@ +# AArch32 Neon load/store instruction descriptions +# +# Copyright (c) 2020 Linaro, Ltd +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library; if not, see <http://www.gnu.org/licenses/>. + +# +# This file is processed by scripts/decodetree.py +# + +# Encodings for Neon load/store instructions where the T32 encoding +# is a simple transformation of the A32 encoding. +# More specifically, this file covers instructions where the A32 encoding is +# 0b1111_0100_xxx0_xxxx_xxxx_xxxx_xxxx_xxxx +# and the T32 encoding is +# 0b1111_1001_xxx0_xxxx_xxxx_xxxx_xxxx_xxxx +# This file works on the A32 encoding only; calling code for T32 has to +# transform the insn into the A32 version first. diff --git a/target/arm/neon-shared.decode b/target/arm/neon-shared.decode new file mode 100644 index XXXXXXX..XXXXXXX --- /dev/null +++ b/target/arm/neon-shared.decode @@ -XXX,XX +XXX,XX @@ +# AArch32 Neon instruction descriptions +# +# Copyright (c) 2020 Linaro, Ltd +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library; if not, see <http://www.gnu.org/licenses/>. + +# +# This file is processed by scripts/decodetree.py +# + +# Encodings for Neon instructions whose encoding is the same for +# both A32 and T32. + +# More specifically, this covers: +# 2reg scalar ext: 0b1111_1110_xxxx_xxxx_xxxx_1x0x_xxxx_xxxx +# 3same ext: 0b1111_110x_xxxx_xxxx_xxxx_1x0x_xxxx_xxxx diff --git a/target/arm/translate-neon.inc.c b/target/arm/translate-neon.inc.c new file mode 100644 index XXXXXXX..XXXXXXX --- /dev/null +++ b/target/arm/translate-neon.inc.c @@ -XXX,XX +XXX,XX @@ +/* + * ARM translation: AArch32 Neon instructions + * + * Copyright (c) 2003 Fabrice Bellard + * Copyright (c) 2005-2007 CodeSourcery + * Copyright (c) 2007 OpenedHand, Ltd. + * Copyright (c) 2020 Linaro, Ltd. + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library; if not, see <http://www.gnu.org/licenses/>. + */ + +/* + * This file is intended to be included from translate.c; it uses + * some macros and definitions provided by that file. + * It might be possible to convert it to a standalone .c file eventually. + */ + +/* Include the generated Neon decoder */ +#include "decode-neon-dp.inc.c" +#include "decode-neon-ls.inc.c" +#include "decode-neon-shared.inc.c" diff --git a/target/arm/translate.c b/target/arm/translate.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate.c +++ b/target/arm/translate.c @@ -XXX,XX +XXX,XX @@ static TCGv_ptr vfp_reg_ptr(bool dp, int reg) #define ARM_CP_RW_BIT (1 << 20) -/* Include the VFP decoder */ +/* Include the VFP and Neon decoders */ #include "translate-vfp.inc.c" +#include "translate-neon.inc.c" static inline void iwmmxt_load_reg(TCGv_i64 var, int reg) { @@ -XXX,XX +XXX,XX @@ static void disas_arm_insn(DisasContext *s, unsigned int insn) /* Unconditional instructions. */ /* TODO: Perhaps merge these into one decodetree output file. */ if (disas_a32_uncond(s, insn) || - disas_vfp_uncond(s, insn)) { + disas_vfp_uncond(s, insn) || + disas_neon_dp(s, insn) || + disas_neon_ls(s, insn) || + disas_neon_shared(s, insn)) { return; } /* fall back to legacy decoder */ @@ -XXX,XX +XXX,XX @@ static void disas_thumb2_insn(DisasContext *s, uint32_t insn) ARCH(6T2); } + if ((insn & 0xef000000) == 0xef000000) { + /* + * T32 encodings 0b111p_1111_qqqq_qqqq_qqqq_qqqq_qqqq_qqqq + * transform into + * A32 encodings 0b1111_001p_qqqq_qqqq_qqqq_qqqq_qqqq_qqqq + */ + uint32_t a32_insn = (insn & 0xe2ffffff) | + ((insn & (1 << 28)) >> 4) | (1 << 28); + + if (disas_neon_dp(s, a32_insn)) { + return; + } + } + + if ((insn & 0xff100000) == 0xf9000000) { + /* + * T32 encodings 0b1111_1001_ppp0_qqqq_qqqq_qqqq_qqqq_qqqq + * transform into + * A32 encodings 0b1111_0100_ppp0_qqqq_qqqq_qqqq_qqqq_qqqq + */ + uint32_t a32_insn = (insn & 0x00ffffff) | 0xf4000000; + + if (disas_neon_ls(s, a32_insn)) { + return; + } + } + /* * TODO: Perhaps merge these into one decodetree output file. * Note disas_vfp is written for a32 with cond field in the @@ -XXX,XX +XXX,XX @@ static void disas_thumb2_insn(DisasContext *s, uint32_t insn) */ if (disas_t32(s, insn) || disas_vfp_uncond(s, insn) || + disas_neon_shared(s, insn) || ((insn >> 28) == 0xe && disas_vfp(s, insn))) { return; } diff --git a/target/arm/Makefile.objs b/target/arm/Makefile.objs index XXXXXXX..XXXXXXX 100644 --- a/target/arm/Makefile.objs +++ b/target/arm/Makefile.objs @@ -XXX,XX +XXX,XX @@ target/arm/decode-sve.inc.c: $(SRC_PATH)/target/arm/sve.decode $(DECODETREE) $(PYTHON) $(DECODETREE) --decode disas_sve -o $@ $<,\ "GEN", $(TARGET_DIR)$@) +target/arm/decode-neon-shared.inc.c: $(SRC_PATH)/target/arm/neon-shared.decode $(DECODETREE) + $(call quiet-command,\ + $(PYTHON) $(DECODETREE) --static-decode disas_neon_shared -o $@ $<,\ + "GEN", $(TARGET_DIR)$@) + +target/arm/decode-neon-dp.inc.c: $(SRC_PATH)/target/arm/neon-dp.decode $(DECODETREE) + $(call quiet-command,\ + $(PYTHON) $(DECODETREE) --static-decode disas_neon_dp -o $@ $<,\ + "GEN", $(TARGET_DIR)$@) + +target/arm/decode-neon-ls.inc.c: $(SRC_PATH)/target/arm/neon-ls.decode $(DECODETREE) + $(call quiet-command,\ + $(PYTHON) $(DECODETREE) --static-decode disas_neon_ls -o $@ $<,\ + "GEN", $(TARGET_DIR)$@) + target/arm/decode-vfp.inc.c: $(SRC_PATH)/target/arm/vfp.decode $(DECODETREE) $(call quiet-command,\ $(PYTHON) $(DECODETREE) --static-decode disas_vfp -o $@ $<,\ @@ -XXX,XX +XXX,XX @@ target/arm/decode-t16.inc.c: $(SRC_PATH)/target/arm/t16.decode $(DECODETREE) "GEN", $(TARGET_DIR)$@) target/arm/translate-sve.o: target/arm/decode-sve.inc.c +target/arm/translate.o: target/arm/decode-neon-shared.inc.c +target/arm/translate.o: target/arm/decode-neon-dp.inc.c +target/arm/translate.o: target/arm/decode-neon-ls.inc.c target/arm/translate.o: target/arm/decode-vfp.inc.c target/arm/translate.o: target/arm/decode-vfp-uncond.inc.c target/arm/translate.o: target/arm/decode-a32.inc.c -- 2.20.1 Convert the VCMLA (vector) insns in the 3same extension group to decodetree. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200430181003.21682-5-peter.maydell@linaro.org --- target/arm/neon-shared.decode | 11 ++++++++++ target/arm/translate-neon.inc.c | 37 +++++++++++++++++++++++++++++++++ target/arm/translate.c | 11 +--------- 3 files changed, 49 insertions(+), 10 deletions(-) diff --git a/target/arm/neon-shared.decode b/target/arm/neon-shared.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/neon-shared.decode +++ b/target/arm/neon-shared.decode @@ -XXX,XX +XXX,XX @@ # More specifically, this covers: # 2reg scalar ext: 0b1111_1110_xxxx_xxxx_xxxx_1x0x_xxxx_xxxx # 3same ext: 0b1111_110x_xxxx_xxxx_xxxx_1x0x_xxxx_xxxx + +# VFP/Neon register fields; same as vfp.decode +%vm_dp 5:1 0:4 +%vm_sp 0:4 5:1 +%vn_dp 7:1 16:4 +%vn_sp 16:4 7:1 +%vd_dp 22:1 12:4 +%vd_sp 12:4 22:1 + +VCMLA 1111 110 rot:2 . 1 size:1 .... .... 1000 . q:1 . 0 .... \ + vm=%vm_dp vn=%vn_dp vd=%vd_dp diff --git a/target/arm/translate-neon.inc.c b/target/arm/translate-neon.inc.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-neon.inc.c +++ b/target/arm/translate-neon.inc.c @@ -XXX,XX +XXX,XX @@ #include "decode-neon-dp.inc.c" #include "decode-neon-ls.inc.c" #include "decode-neon-shared.inc.c" + +static bool trans_VCMLA(DisasContext *s, arg_VCMLA *a) +{ + int opr_sz; + TCGv_ptr fpst; + gen_helper_gvec_3_ptr *fn_gvec_ptr; + + if (!dc_isar_feature(aa32_vcma, s) + || (!a->size && !dc_isar_feature(aa32_fp16_arith, s))) { + return false; + } + + /* UNDEF accesses to D16-D31 if they don't exist. */ + if (!dc_isar_feature(aa32_simd_r32, s) && + ((a->vd | a->vn | a->vm) & 0x10)) { + return false; + } + + if ((a->vn | a->vm | a->vd) & a->q) { + return false; + } + + if (!vfp_access_check(s)) { + return true; + } + + opr_sz = (1 + a->q) * 8; + fpst = get_fpstatus_ptr(1); + fn_gvec_ptr = a->size ? gen_helper_gvec_fcmlas : gen_helper_gvec_fcmlah; + tcg_gen_gvec_3_ptr(vfp_reg_offset(1, a->vd), + vfp_reg_offset(1, a->vn), + vfp_reg_offset(1, a->vm), + fpst, opr_sz, opr_sz, a->rot, + fn_gvec_ptr); + tcg_temp_free_ptr(fpst); + return true; +} diff --git a/target/arm/translate.c b/target/arm/translate.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate.c +++ b/target/arm/translate.c @@ -XXX,XX +XXX,XX @@ static int disas_neon_insn_3same_ext(DisasContext *s, uint32_t insn) bool is_long = false, q = extract32(insn, 6, 1); bool ptr_is_env = false; - if ((insn & 0xfe200f10) == 0xfc200800) { - /* VCMLA -- 1111 110R R.1S .... .... 1000 ...0 .... */ - int size = extract32(insn, 20, 1); - data = extract32(insn, 23, 2); /* rot */ - if (!dc_isar_feature(aa32_vcma, s) - || (!size && !dc_isar_feature(aa32_fp16_arith, s))) { - return 1; - } - fn_gvec_ptr = size ? gen_helper_gvec_fcmlas : gen_helper_gvec_fcmlah; - } else if ((insn & 0xfea00f10) == 0xfc800800) { + if ((insn & 0xfea00f10) == 0xfc800800) { /* VCADD -- 1111 110R 1.0S .... .... 1000 ...0 .... */ int size = extract32(insn, 20, 1); data = extract32(insn, 24, 1); /* rot */ -- 2.20.1 Convert the VCADD (vector) insns to decodetree. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200430181003.21682-6-peter.maydell@linaro.org --- target/arm/neon-shared.decode | 3 +++ target/arm/translate-neon.inc.c | 37 +++++++++++++++++++++++++++++++++ target/arm/translate.c | 11 +--------- 3 files changed, 41 insertions(+), 10 deletions(-) diff --git a/target/arm/neon-shared.decode b/target/arm/neon-shared.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/neon-shared.decode +++ b/target/arm/neon-shared.decode @@ -XXX,XX +XXX,XX @@ VCMLA 1111 110 rot:2 . 1 size:1 .... .... 1000 . q:1 . 0 .... \ vm=%vm_dp vn=%vn_dp vd=%vd_dp + +VCADD 1111 110 rot:1 1 . 0 size:1 .... .... 1000 . q:1 . 0 .... \ + vm=%vm_dp vn=%vn_dp vd=%vd_dp diff --git a/target/arm/translate-neon.inc.c b/target/arm/translate-neon.inc.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-neon.inc.c +++ b/target/arm/translate-neon.inc.c @@ -XXX,XX +XXX,XX @@ static bool trans_VCMLA(DisasContext *s, arg_VCMLA *a) tcg_temp_free_ptr(fpst); return true; } + +static bool trans_VCADD(DisasContext *s, arg_VCADD *a) +{ + int opr_sz; + TCGv_ptr fpst; + gen_helper_gvec_3_ptr *fn_gvec_ptr; + + if (!dc_isar_feature(aa32_vcma, s) + || (!a->size && !dc_isar_feature(aa32_fp16_arith, s))) { + return false; + } + + /* UNDEF accesses to D16-D31 if they don't exist. */ + if (!dc_isar_feature(aa32_simd_r32, s) && + ((a->vd | a->vn | a->vm) & 0x10)) { + return false; + } + + if ((a->vn | a->vm | a->vd) & a->q) { + return false; + } + + if (!vfp_access_check(s)) { + return true; + } + + opr_sz = (1 + a->q) * 8; + fpst = get_fpstatus_ptr(1); + fn_gvec_ptr = a->size ? gen_helper_gvec_fcadds : gen_helper_gvec_fcaddh; + tcg_gen_gvec_3_ptr(vfp_reg_offset(1, a->vd), + vfp_reg_offset(1, a->vn), + vfp_reg_offset(1, a->vm), + fpst, opr_sz, opr_sz, a->rot, + fn_gvec_ptr); + tcg_temp_free_ptr(fpst); + return true; +} diff --git a/target/arm/translate.c b/target/arm/translate.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate.c +++ b/target/arm/translate.c @@ -XXX,XX +XXX,XX @@ static int disas_neon_insn_3same_ext(DisasContext *s, uint32_t insn) bool is_long = false, q = extract32(insn, 6, 1); bool ptr_is_env = false; - if ((insn & 0xfea00f10) == 0xfc800800) { - /* VCADD -- 1111 110R 1.0S .... .... 1000 ...0 .... */ - int size = extract32(insn, 20, 1); - data = extract32(insn, 24, 1); /* rot */ - if (!dc_isar_feature(aa32_vcma, s) - || (!size && !dc_isar_feature(aa32_fp16_arith, s))) { - return 1; - } - fn_gvec_ptr = size ? gen_helper_gvec_fcadds : gen_helper_gvec_fcaddh; - } else if ((insn & 0xfeb00f00) == 0xfc200d00) { + if ((insn & 0xfeb00f00) == 0xfc200d00) { /* V[US]DOT -- 1111 1100 0.10 .... .... 1101 .Q.U .... */ bool u = extract32(insn, 4, 1); if (!dc_isar_feature(aa32_dp, s)) { -- 2.20.1 Convert the V[US]DOT (vector) insns to decodetree. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200430181003.21682-7-peter.maydell@linaro.org --- target/arm/neon-shared.decode | 4 ++++ target/arm/translate-neon.inc.c | 32 ++++++++++++++++++++++++++++++++ target/arm/translate.c | 9 +-------- 3 files changed, 37 insertions(+), 8 deletions(-) diff --git a/target/arm/neon-shared.decode b/target/arm/neon-shared.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/neon-shared.decode +++ b/target/arm/neon-shared.decode @@ -XXX,XX +XXX,XX @@ VCMLA 1111 110 rot:2 . 1 size:1 .... .... 1000 . q:1 . 0 .... \ VCADD 1111 110 rot:1 1 . 0 size:1 .... .... 1000 . q:1 . 0 .... \ vm=%vm_dp vn=%vn_dp vd=%vd_dp + +# VUDOT and VSDOT +VDOT 1111 110 00 . 10 .... .... 1101 . q:1 . u:1 .... \ + vm=%vm_dp vn=%vn_dp vd=%vd_dp diff --git a/target/arm/translate-neon.inc.c b/target/arm/translate-neon.inc.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-neon.inc.c +++ b/target/arm/translate-neon.inc.c @@ -XXX,XX +XXX,XX @@ static bool trans_VCADD(DisasContext *s, arg_VCADD *a) tcg_temp_free_ptr(fpst); return true; } + +static bool trans_VDOT(DisasContext *s, arg_VDOT *a) +{ + int opr_sz; + gen_helper_gvec_3 *fn_gvec; + + if (!dc_isar_feature(aa32_dp, s)) { + return false; + } + + /* UNDEF accesses to D16-D31 if they don't exist. */ + if (!dc_isar_feature(aa32_simd_r32, s) && + ((a->vd | a->vn | a->vm) & 0x10)) { + return false; + } + + if ((a->vn | a->vm | a->vd) & a->q) { + return false; + } + + if (!vfp_access_check(s)) { + return true; + } + + opr_sz = (1 + a->q) * 8; + fn_gvec = a->u ? gen_helper_gvec_udot_b : gen_helper_gvec_sdot_b; + tcg_gen_gvec_3_ool(vfp_reg_offset(1, a->vd), + vfp_reg_offset(1, a->vn), + vfp_reg_offset(1, a->vm), + opr_sz, opr_sz, 0, fn_gvec); + return true; +} diff --git a/target/arm/translate.c b/target/arm/translate.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate.c +++ b/target/arm/translate.c @@ -XXX,XX +XXX,XX @@ static int disas_neon_insn_3same_ext(DisasContext *s, uint32_t insn) bool is_long = false, q = extract32(insn, 6, 1); bool ptr_is_env = false; - if ((insn & 0xfeb00f00) == 0xfc200d00) { - /* V[US]DOT -- 1111 1100 0.10 .... .... 1101 .Q.U .... */ - bool u = extract32(insn, 4, 1); - if (!dc_isar_feature(aa32_dp, s)) { - return 1; - } - fn_gvec = u ? gen_helper_gvec_udot_b : gen_helper_gvec_sdot_b; - } else if ((insn & 0xff300f10) == 0xfc200810) { + if ((insn & 0xff300f10) == 0xfc200810) { /* VFM[AS]L -- 1111 1100 S.10 .... .... 1000 .Q.1 .... */ int is_s = extract32(insn, 23, 1); if (!dc_isar_feature(aa32_fhm, s)) { -- 2.20.1 Convert the VFM[AS]L (vector) insns to decodetree. This is the last insn in the legacy decoder for the 3same_ext group, so we can delete the legacy decoder function for the group entirely. Note that in disas_thumb2_insn() the parts of this encoding space where the decodetree decoder returns false will correctly be directed to illegal_op by the "(insn & (1 << 28))" check so they won't fall into disas_coproc_insn() by mistake. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200430181003.21682-8-peter.maydell@linaro.org --- target/arm/neon-shared.decode | 6 +++ target/arm/translate-neon.inc.c | 31 +++++++++++ target/arm/translate.c | 92 +-------------------------------- 3 files changed, 38 insertions(+), 91 deletions(-) diff --git a/target/arm/neon-shared.decode b/target/arm/neon-shared.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/neon-shared.decode +++ b/target/arm/neon-shared.decode @@ -XXX,XX +XXX,XX @@ VCADD 1111 110 rot:1 1 . 0 size:1 .... .... 1000 . q:1 . 0 .... \ # VUDOT and VSDOT VDOT 1111 110 00 . 10 .... .... 1101 . q:1 . u:1 .... \ vm=%vm_dp vn=%vn_dp vd=%vd_dp + +# VFM[AS]L +VFML 1111 110 0 s:1 . 10 .... .... 1000 . 0 . 1 .... \ + vm=%vm_sp vn=%vn_sp vd=%vd_dp q=0 +VFML 1111 110 0 s:1 . 10 .... .... 1000 . 1 . 1 .... \ + vm=%vm_dp vn=%vn_dp vd=%vd_dp q=1 diff --git a/target/arm/translate-neon.inc.c b/target/arm/translate-neon.inc.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-neon.inc.c +++ b/target/arm/translate-neon.inc.c @@ -XXX,XX +XXX,XX @@ static bool trans_VDOT(DisasContext *s, arg_VDOT *a) opr_sz, opr_sz, 0, fn_gvec); return true; } + +static bool trans_VFML(DisasContext *s, arg_VFML *a) +{ + int opr_sz; + + if (!dc_isar_feature(aa32_fhm, s)) { + return false; + } + + /* UNDEF accesses to D16-D31 if they don't exist. */ + if (!dc_isar_feature(aa32_simd_r32, s) && + (a->vd & 0x10)) { + return false; + } + + if (a->vd & a->q) { + return false; + } + + if (!vfp_access_check(s)) { + return true; + } + + opr_sz = (1 + a->q) * 8; + tcg_gen_gvec_3_ptr(vfp_reg_offset(1, a->vd), + vfp_reg_offset(a->q, a->vn), + vfp_reg_offset(a->q, a->vm), + cpu_env, opr_sz, opr_sz, a->s, /* is_2 == 0 */ + gen_helper_gvec_fmlal_a32); + return true; +} diff --git a/target/arm/translate.c b/target/arm/translate.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate.c +++ b/target/arm/translate.c @@ -XXX,XX +XXX,XX @@ static int disas_neon_data_insn(DisasContext *s, uint32_t insn) return 0; } -/* Advanced SIMD three registers of the same length extension. - * 31 25 23 22 20 16 12 11 10 9 8 3 0 - * +---------------+-----+---+-----+----+----+---+----+---+----+---------+----+ - * | 1 1 1 1 1 1 0 | op1 | D | op2 | Vn | Vd | 1 | o3 | 0 | o4 | N Q M U | Vm | - * +---------------+-----+---+-----+----+----+---+----+---+----+---------+----+ - */ -static int disas_neon_insn_3same_ext(DisasContext *s, uint32_t insn) -{ - gen_helper_gvec_3 *fn_gvec = NULL; - gen_helper_gvec_3_ptr *fn_gvec_ptr = NULL; - int rd, rn, rm, opr_sz; - int data = 0; - int off_rn, off_rm; - bool is_long = false, q = extract32(insn, 6, 1); - bool ptr_is_env = false; - - if ((insn & 0xff300f10) == 0xfc200810) { - /* VFM[AS]L -- 1111 1100 S.10 .... .... 1000 .Q.1 .... */ - int is_s = extract32(insn, 23, 1); - if (!dc_isar_feature(aa32_fhm, s)) { - return 1; - } - is_long = true; - data = is_s; /* is_2 == 0 */ - fn_gvec_ptr = gen_helper_gvec_fmlal_a32; - ptr_is_env = true; - } else { - return 1; - } - - VFP_DREG_D(rd, insn); - if (rd & q) { - return 1; - } - if (q || !is_long) { - VFP_DREG_N(rn, insn); - VFP_DREG_M(rm, insn); - if ((rn | rm) & q & !is_long) { - return 1; - } - off_rn = vfp_reg_offset(1, rn); - off_rm = vfp_reg_offset(1, rm); - } else { - rn = VFP_SREG_N(insn); - rm = VFP_SREG_M(insn); - off_rn = vfp_reg_offset(0, rn); - off_rm = vfp_reg_offset(0, rm); - } - - if (s->fp_excp_el) { - gen_exception_insn(s, s->pc_curr, EXCP_UDEF, - syn_simd_access_trap(1, 0xe, false), s->fp_excp_el); - return 0; - } - if (!s->vfp_enabled) { - return 1; - } - - opr_sz = (1 + q) * 8; - if (fn_gvec_ptr) { - TCGv_ptr ptr; - if (ptr_is_env) { - ptr = cpu_env; - } else { - ptr = get_fpstatus_ptr(1); - } - tcg_gen_gvec_3_ptr(vfp_reg_offset(1, rd), off_rn, off_rm, ptr, - opr_sz, opr_sz, data, fn_gvec_ptr); - if (!ptr_is_env) { - tcg_temp_free_ptr(ptr); - } - } else { - tcg_gen_gvec_3_ool(vfp_reg_offset(1, rd), off_rn, off_rm, - opr_sz, opr_sz, data, fn_gvec); - } - return 0; -} - /* Advanced SIMD two registers and a scalar extension. * 31 24 23 22 20 16 12 11 10 9 8 3 0 * +-----------------+----+---+----+----+----+---+----+---+----+---------+----+ @@ -XXX,XX +XXX,XX @@ static void disas_arm_insn(DisasContext *s, unsigned int insn) } } } - } else if ((insn & 0x0e000a00) == 0x0c000800 - && arm_dc_feature(s, ARM_FEATURE_V8)) { - if (disas_neon_insn_3same_ext(s, insn)) { - goto illegal_op; - } - return; } else if ((insn & 0x0f000a00) == 0x0e000800 && arm_dc_feature(s, ARM_FEATURE_V8)) { if (disas_neon_insn_2reg_scalar_ext(s, insn)) { @@ -XXX,XX +XXX,XX @@ static void disas_thumb2_insn(DisasContext *s, uint32_t insn) } break; } - if ((insn & 0xfe000a00) == 0xfc000800 + if ((insn & 0xff000a00) == 0xfe000800 && arm_dc_feature(s, ARM_FEATURE_V8)) { /* The Thumb2 and ARM encodings are identical. */ - if (disas_neon_insn_3same_ext(s, insn)) { - goto illegal_op; - } - } else if ((insn & 0xff000a00) == 0xfe000800 - && arm_dc_feature(s, ARM_FEATURE_V8)) { - /* The Thumb2 and ARM encodings are identical. */ if (disas_neon_insn_2reg_scalar_ext(s, insn)) { goto illegal_op; } -- 2.20.1 Convert VCMLA (scalar) in the 2reg-scalar-ext group to decodetree. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200430181003.21682-9-peter.maydell@linaro.org --- target/arm/neon-shared.decode | 5 +++++ target/arm/translate-neon.inc.c | 40 +++++++++++++++++++++++++++++++++ target/arm/translate.c | 26 +-------------------- 3 files changed, 46 insertions(+), 25 deletions(-) diff --git a/target/arm/neon-shared.decode b/target/arm/neon-shared.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/neon-shared.decode +++ b/target/arm/neon-shared.decode @@ -XXX,XX +XXX,XX @@ VFML 1111 110 0 s:1 . 10 .... .... 1000 . 0 . 1 .... \ vm=%vm_sp vn=%vn_sp vd=%vd_dp q=0 VFML 1111 110 0 s:1 . 10 .... .... 1000 . 1 . 1 .... \ vm=%vm_dp vn=%vn_dp vd=%vd_dp q=1 + +VCMLA_scalar 1111 1110 0 . rot:2 .... .... 1000 . q:1 index:1 0 vm:4 \ + vn=%vn_dp vd=%vd_dp size=0 +VCMLA_scalar 1111 1110 1 . rot:2 .... .... 1000 . q:1 . 0 .... \ + vm=%vm_dp vn=%vn_dp vd=%vd_dp size=1 index=0 diff --git a/target/arm/translate-neon.inc.c b/target/arm/translate-neon.inc.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-neon.inc.c +++ b/target/arm/translate-neon.inc.c @@ -XXX,XX +XXX,XX @@ static bool trans_VFML(DisasContext *s, arg_VFML *a) gen_helper_gvec_fmlal_a32); return true; } + +static bool trans_VCMLA_scalar(DisasContext *s, arg_VCMLA_scalar *a) +{ + gen_helper_gvec_3_ptr *fn_gvec_ptr; + int opr_sz; + TCGv_ptr fpst; + + if (!dc_isar_feature(aa32_vcma, s)) { + return false; + } + if (a->size == 0 && !dc_isar_feature(aa32_fp16_arith, s)) { + return false; + } + + /* UNDEF accesses to D16-D31 if they don't exist. */ + if (!dc_isar_feature(aa32_simd_r32, s) && + ((a->vd | a->vn | a->vm) & 0x10)) { + return false; + } + + if ((a->vd | a->vn) & a->q) { + return false; + } + + if (!vfp_access_check(s)) { + return true; + } + + fn_gvec_ptr = (a->size ? gen_helper_gvec_fcmlas_idx + : gen_helper_gvec_fcmlah_idx); + opr_sz = (1 + a->q) * 8; + fpst = get_fpstatus_ptr(1); + tcg_gen_gvec_3_ptr(vfp_reg_offset(1, a->vd), + vfp_reg_offset(1, a->vn), + vfp_reg_offset(1, a->vm), + fpst, opr_sz, opr_sz, + (a->index << 2) | a->rot, fn_gvec_ptr); + tcg_temp_free_ptr(fpst); + return true; +} diff --git a/target/arm/translate.c b/target/arm/translate.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate.c +++ b/target/arm/translate.c @@ -XXX,XX +XXX,XX @@ static int disas_neon_insn_2reg_scalar_ext(DisasContext *s, uint32_t insn) bool is_long = false, q = extract32(insn, 6, 1); bool ptr_is_env = false; - if ((insn & 0xff000f10) == 0xfe000800) { - /* VCMLA (indexed) -- 1111 1110 S.RR .... .... 1000 ...0 .... */ - int rot = extract32(insn, 20, 2); - int size = extract32(insn, 23, 1); - int index; - - if (!dc_isar_feature(aa32_vcma, s)) { - return 1; - } - if (size == 0) { - if (!dc_isar_feature(aa32_fp16_arith, s)) { - return 1; - } - /* For fp16, rm is just Vm, and index is M. */ - rm = extract32(insn, 0, 4); - index = extract32(insn, 5, 1); - } else { - /* For fp32, rm is the usual M:Vm, and index is 0. */ - VFP_DREG_M(rm, insn); - index = 0; - } - data = (index << 2) | rot; - fn_gvec_ptr = (size ? gen_helper_gvec_fcmlas_idx - : gen_helper_gvec_fcmlah_idx); - } else if ((insn & 0xffb00f00) == 0xfe200d00) { + if ((insn & 0xffb00f00) == 0xfe200d00) { /* V[US]DOT -- 1111 1110 0.10 .... .... 1101 .Q.U .... */ int u = extract32(insn, 4, 1); -- 2.20.1 Convert the V[US]DOT (scalar) insns in the 2reg-scalar-ext group to decodetree. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200430181003.21682-10-peter.maydell@linaro.org --- target/arm/neon-shared.decode | 3 +++ target/arm/translate-neon.inc.c | 35 +++++++++++++++++++++++++++++++++ target/arm/translate.c | 13 +----------- 3 files changed, 39 insertions(+), 12 deletions(-) diff --git a/target/arm/neon-shared.decode b/target/arm/neon-shared.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/neon-shared.decode +++ b/target/arm/neon-shared.decode @@ -XXX,XX +XXX,XX @@ VCMLA_scalar 1111 1110 0 . rot:2 .... .... 1000 . q:1 index:1 0 vm:4 \ vn=%vn_dp vd=%vd_dp size=0 VCMLA_scalar 1111 1110 1 . rot:2 .... .... 1000 . q:1 . 0 .... \ vm=%vm_dp vn=%vn_dp vd=%vd_dp size=1 index=0 + +VDOT_scalar 1111 1110 0 . 10 .... .... 1101 . q:1 index:1 u:1 rm:4 \ + vm=%vm_dp vn=%vn_dp vd=%vd_dp diff --git a/target/arm/translate-neon.inc.c b/target/arm/translate-neon.inc.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-neon.inc.c +++ b/target/arm/translate-neon.inc.c @@ -XXX,XX +XXX,XX @@ static bool trans_VCMLA_scalar(DisasContext *s, arg_VCMLA_scalar *a) tcg_temp_free_ptr(fpst); return true; } + +static bool trans_VDOT_scalar(DisasContext *s, arg_VDOT_scalar *a) +{ + gen_helper_gvec_3 *fn_gvec; + int opr_sz; + TCGv_ptr fpst; + + if (!dc_isar_feature(aa32_dp, s)) { + return false; + } + + /* UNDEF accesses to D16-D31 if they don't exist. */ + if (!dc_isar_feature(aa32_simd_r32, s) && + ((a->vd | a->vn) & 0x10)) { + return false; + } + + if ((a->vd | a->vn) & a->q) { + return false; + } + + if (!vfp_access_check(s)) { + return true; + } + + fn_gvec = a->u ? gen_helper_gvec_udot_idx_b : gen_helper_gvec_sdot_idx_b; + opr_sz = (1 + a->q) * 8; + fpst = get_fpstatus_ptr(1); + tcg_gen_gvec_3_ool(vfp_reg_offset(1, a->vd), + vfp_reg_offset(1, a->vn), + vfp_reg_offset(1, a->rm), + opr_sz, opr_sz, a->index, fn_gvec); + tcg_temp_free_ptr(fpst); + return true; +} diff --git a/target/arm/translate.c b/target/arm/translate.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate.c +++ b/target/arm/translate.c @@ -XXX,XX +XXX,XX @@ static int disas_neon_insn_2reg_scalar_ext(DisasContext *s, uint32_t insn) bool is_long = false, q = extract32(insn, 6, 1); bool ptr_is_env = false; - if ((insn & 0xffb00f00) == 0xfe200d00) { - /* V[US]DOT -- 1111 1110 0.10 .... .... 1101 .Q.U .... */ - int u = extract32(insn, 4, 1); - - if (!dc_isar_feature(aa32_dp, s)) { - return 1; - } - fn_gvec = u ? gen_helper_gvec_udot_idx_b : gen_helper_gvec_sdot_idx_b; - /* rm is just Vm, and index is M. */ - data = extract32(insn, 5, 1); /* index */ - rm = extract32(insn, 0, 4); - } else if ((insn & 0xffa00f10) == 0xfe000810) { + if ((insn & 0xffa00f10) == 0xfe000810) { /* VFM[AS]L -- 1111 1110 0.0S .... .... 1000 .Q.1 .... */ int is_s = extract32(insn, 20, 1); int vm20 = extract32(insn, 0, 3); -- 2.20.1 Convert the VFM[AS]L (scalar) insns in the 2reg-scalar-ext group to decodetree. These are the last ones in the group so we can remove all the legacy decode for the group. Note that in disas_thumb2_insn() the parts of this encoding space where the decodetree decoder returns false will correctly be directed to illegal_op by the "(insn & (1 << 28))" check so they won't fall into disas_coproc_insn() by mistake. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200430181003.21682-11-peter.maydell@linaro.org --- target/arm/neon-shared.decode | 7 +++ target/arm/translate-neon.inc.c | 32 ++++++++++ target/arm/translate.c | 107 +------------------------------- 3 files changed, 40 insertions(+), 106 deletions(-) diff --git a/target/arm/neon-shared.decode b/target/arm/neon-shared.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/neon-shared.decode +++ b/target/arm/neon-shared.decode @@ -XXX,XX +XXX,XX @@ VCMLA_scalar 1111 1110 1 . rot:2 .... .... 1000 . q:1 . 0 .... \ VDOT_scalar 1111 1110 0 . 10 .... .... 1101 . q:1 index:1 u:1 rm:4 \ vm=%vm_dp vn=%vn_dp vd=%vd_dp + +%vfml_scalar_q0_rm 0:3 5:1 +%vfml_scalar_q1_index 5:1 3:1 +VFML_scalar 1111 1110 0 . 0 s:1 .... .... 1000 . 0 . 1 index:1 ... \ + rm=%vfml_scalar_q0_rm vn=%vn_sp vd=%vd_dp q=0 +VFML_scalar 1111 1110 0 . 0 s:1 .... .... 1000 . 1 . 1 . rm:3 \ + index=%vfml_scalar_q1_index vn=%vn_dp vd=%vd_dp q=1 diff --git a/target/arm/translate-neon.inc.c b/target/arm/translate-neon.inc.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-neon.inc.c +++ b/target/arm/translate-neon.inc.c @@ -XXX,XX +XXX,XX @@ static bool trans_VDOT_scalar(DisasContext *s, arg_VDOT_scalar *a) tcg_temp_free_ptr(fpst); return true; } + +static bool trans_VFML_scalar(DisasContext *s, arg_VFML_scalar *a) +{ + int opr_sz; + + if (!dc_isar_feature(aa32_fhm, s)) { + return false; + } + + /* UNDEF accesses to D16-D31 if they don't exist. */ + if (!dc_isar_feature(aa32_simd_r32, s) && + ((a->vd & 0x10) || (a->q && (a->vn & 0x10)))) { + return false; + } + + if (a->vd & a->q) { + return false; + } + + if (!vfp_access_check(s)) { + return true; + } + + opr_sz = (1 + a->q) * 8; + tcg_gen_gvec_3_ptr(vfp_reg_offset(1, a->vd), + vfp_reg_offset(a->q, a->vn), + vfp_reg_offset(a->q, a->rm), + cpu_env, opr_sz, opr_sz, + (a->index << 2) | a->s, /* is_2 == 0 */ + gen_helper_gvec_fmlal_idx_a32); + return true; +} diff --git a/target/arm/translate.c b/target/arm/translate.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate.c +++ b/target/arm/translate.c @@ -XXX,XX +XXX,XX @@ static int disas_dsp_insn(DisasContext *s, uint32_t insn) } #define VFP_REG_SHR(x, n) (((n) > 0) ? (x) >> (n) : (x) << -(n)) -#define VFP_SREG(insn, bigbit, smallbit) \ - ((VFP_REG_SHR(insn, bigbit - 1) & 0x1e) | (((insn) >> (smallbit)) & 1)) #define VFP_DREG(reg, insn, bigbit, smallbit) do { \ if (dc_isar_feature(aa32_simd_r32, s)) { \ reg = (((insn) >> (bigbit)) & 0x0f) \ @@ -XXX,XX +XXX,XX @@ static int disas_dsp_insn(DisasContext *s, uint32_t insn) reg = ((insn) >> (bigbit)) & 0x0f; \ }} while (0) -#define VFP_SREG_D(insn) VFP_SREG(insn, 12, 22) #define VFP_DREG_D(reg, insn) VFP_DREG(reg, insn, 12, 22) -#define VFP_SREG_N(insn) VFP_SREG(insn, 16, 7) #define VFP_DREG_N(reg, insn) VFP_DREG(reg, insn, 16, 7) -#define VFP_SREG_M(insn) VFP_SREG(insn, 0, 5) #define VFP_DREG_M(reg, insn) VFP_DREG(reg, insn, 0, 5) static void gen_neon_dup_low16(TCGv_i32 var) @@ -XXX,XX +XXX,XX @@ static int disas_neon_data_insn(DisasContext *s, uint32_t insn) return 0; } -/* Advanced SIMD two registers and a scalar extension. - * 31 24 23 22 20 16 12 11 10 9 8 3 0 - * +-----------------+----+---+----+----+----+---+----+---+----+---------+----+ - * | 1 1 1 1 1 1 1 0 | o1 | D | o2 | Vn | Vd | 1 | o3 | 0 | o4 | N Q M U | Vm | - * +-----------------+----+---+----+----+----+---+----+---+----+---------+----+ - * - */ - -static int disas_neon_insn_2reg_scalar_ext(DisasContext *s, uint32_t insn) -{ - gen_helper_gvec_3 *fn_gvec = NULL; - gen_helper_gvec_3_ptr *fn_gvec_ptr = NULL; - int rd, rn, rm, opr_sz, data; - int off_rn, off_rm; - bool is_long = false, q = extract32(insn, 6, 1); - bool ptr_is_env = false; - - if ((insn & 0xffa00f10) == 0xfe000810) { - /* VFM[AS]L -- 1111 1110 0.0S .... .... 1000 .Q.1 .... */ - int is_s = extract32(insn, 20, 1); - int vm20 = extract32(insn, 0, 3); - int vm3 = extract32(insn, 3, 1); - int m = extract32(insn, 5, 1); - int index; - - if (!dc_isar_feature(aa32_fhm, s)) { - return 1; - } - if (q) { - rm = vm20; - index = m * 2 + vm3; - } else { - rm = vm20 * 2 + m; - index = vm3; - } - is_long = true; - data = (index << 2) | is_s; /* is_2 == 0 */ - fn_gvec_ptr = gen_helper_gvec_fmlal_idx_a32; - ptr_is_env = true; - } else { - return 1; - } - - VFP_DREG_D(rd, insn); - if (rd & q) { - return 1; - } - if (q || !is_long) { - VFP_DREG_N(rn, insn); - if (rn & q & !is_long) { - return 1; - } - off_rn = vfp_reg_offset(1, rn); - off_rm = vfp_reg_offset(1, rm); - } else { - rn = VFP_SREG_N(insn); - off_rn = vfp_reg_offset(0, rn); - off_rm = vfp_reg_offset(0, rm); - } - if (s->fp_excp_el) { - gen_exception_insn(s, s->pc_curr, EXCP_UDEF, - syn_simd_access_trap(1, 0xe, false), s->fp_excp_el); - return 0; - } - if (!s->vfp_enabled) { - return 1; - } - - opr_sz = (1 + q) * 8; - if (fn_gvec_ptr) { - TCGv_ptr ptr; - if (ptr_is_env) { - ptr = cpu_env; - } else { - ptr = get_fpstatus_ptr(1); - } - tcg_gen_gvec_3_ptr(vfp_reg_offset(1, rd), off_rn, off_rm, ptr, - opr_sz, opr_sz, data, fn_gvec_ptr); - if (!ptr_is_env) { - tcg_temp_free_ptr(ptr); - } - } else { - tcg_gen_gvec_3_ool(vfp_reg_offset(1, rd), off_rn, off_rm, - opr_sz, opr_sz, data, fn_gvec); - } - return 0; -} - static int disas_coproc_insn(DisasContext *s, uint32_t insn) { int cpnum, is64, crn, crm, opc1, opc2, isread, rt, rt2; @@ -XXX,XX +XXX,XX @@ static void disas_arm_insn(DisasContext *s, unsigned int insn) } } } - } else if ((insn & 0x0f000a00) == 0x0e000800 - && arm_dc_feature(s, ARM_FEATURE_V8)) { - if (disas_neon_insn_2reg_scalar_ext(s, insn)) { - goto illegal_op; - } - return; } goto illegal_op; } @@ -XXX,XX +XXX,XX @@ static void disas_thumb2_insn(DisasContext *s, uint32_t insn) } break; } - if ((insn & 0xff000a00) == 0xfe000800 - && arm_dc_feature(s, ARM_FEATURE_V8)) { - /* The Thumb2 and ARM encodings are identical. */ - if (disas_neon_insn_2reg_scalar_ext(s, insn)) { - goto illegal_op; - } - } else if (((insn >> 24) & 3) == 3) { + if (((insn >> 24) & 3) == 3) { /* Translate into the equivalent ARM encoding. */ insn = (insn & 0xe2ffffff) | ((insn & (1 << 28)) >> 4) | (1 << 28); if (disas_neon_data_insn(s, insn)) { -- 2.20.1 Convert the Neon "load/store multiple structures" insns to decodetree. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200430181003.21682-12-peter.maydell@linaro.org --- target/arm/neon-ls.decode | 7 ++ target/arm/translate-neon.inc.c | 124 ++++++++++++++++++++++++++++++++ target/arm/translate.c | 91 +---------------------- 3 files changed, 133 insertions(+), 89 deletions(-) diff --git a/target/arm/neon-ls.decode b/target/arm/neon-ls.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/neon-ls.decode +++ b/target/arm/neon-ls.decode @@ -XXX,XX +XXX,XX @@ # 0b1111_1001_xxx0_xxxx_xxxx_xxxx_xxxx_xxxx # This file works on the A32 encoding only; calling code for T32 has to # transform the insn into the A32 version first. + +%vd_dp 22:1 12:4 + +# Neon load/store multiple structures + +VLDST_multiple 1111 0100 0 . l:1 0 rn:4 .... itype:4 size:2 align:2 rm:4 \ + vd=%vd_dp diff --git a/target/arm/translate-neon.inc.c b/target/arm/translate-neon.inc.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-neon.inc.c +++ b/target/arm/translate-neon.inc.c @@ -XXX,XX +XXX,XX @@ static bool trans_VFML_scalar(DisasContext *s, arg_VFML_scalar *a) gen_helper_gvec_fmlal_idx_a32); return true; } + +static struct { + int nregs; + int interleave; + int spacing; +} const neon_ls_element_type[11] = { + {1, 4, 1}, + {1, 4, 2}, + {4, 1, 1}, + {2, 2, 2}, + {1, 3, 1}, + {1, 3, 2}, + {3, 1, 1}, + {1, 1, 1}, + {1, 2, 1}, + {1, 2, 2}, + {2, 1, 1} +}; + +static void gen_neon_ldst_base_update(DisasContext *s, int rm, int rn, + int stride) +{ + if (rm != 15) { + TCGv_i32 base; + + base = load_reg(s, rn); + if (rm == 13) { + tcg_gen_addi_i32(base, base, stride); + } else { + TCGv_i32 index; + index = load_reg(s, rm); + tcg_gen_add_i32(base, base, index); + tcg_temp_free_i32(index); + } + store_reg(s, rn, base); + } +} + +static bool trans_VLDST_multiple(DisasContext *s, arg_VLDST_multiple *a) +{ + /* Neon load/store multiple structures */ + int nregs, interleave, spacing, reg, n; + MemOp endian = s->be_data; + int mmu_idx = get_mem_index(s); + int size = a->size; + TCGv_i64 tmp64; + TCGv_i32 addr, tmp; + + if (!arm_dc_feature(s, ARM_FEATURE_NEON)) { + return false; + } + + /* UNDEF accesses to D16-D31 if they don't exist */ + if (!dc_isar_feature(aa32_simd_r32, s) && (a->vd & 0x10)) { + return false; + } + if (a->itype > 10) { + return false; + } + /* Catch UNDEF cases for bad values of align field */ + switch (a->itype & 0xc) { + case 4: + if (a->align >= 2) { + return false; + } + break; + case 8: + if (a->align == 3) { + return false; + } + break; + default: + break; + } + nregs = neon_ls_element_type[a->itype].nregs; + interleave = neon_ls_element_type[a->itype].interleave; + spacing = neon_ls_element_type[a->itype].spacing; + if (size == 3 && (interleave | spacing) != 1) { + return false; + } + + if (!vfp_access_check(s)) { + return true; + } + + /* For our purposes, bytes are always little-endian. */ + if (size == 0) { + endian = MO_LE; + } + /* + * Consecutive little-endian elements from a single register + * can be promoted to a larger little-endian operation. + */ + if (interleave == 1 && endian == MO_LE) { + size = 3; + } + tmp64 = tcg_temp_new_i64(); + addr = tcg_temp_new_i32(); + tmp = tcg_const_i32(1 << size); + load_reg_var(s, addr, a->rn); + for (reg = 0; reg < nregs; reg++) { + for (n = 0; n < 8 >> size; n++) { + int xs; + for (xs = 0; xs < interleave; xs++) { + int tt = a->vd + reg + spacing * xs; + + if (a->l) { + gen_aa32_ld_i64(s, tmp64, addr, mmu_idx, endian | size); + neon_store_element64(tt, n, size, tmp64); + } else { + neon_load_element64(tmp64, tt, n, size); + gen_aa32_st_i64(s, tmp64, addr, mmu_idx, endian | size); + } + tcg_gen_add_i32(addr, addr, tmp); + } + } + } + tcg_temp_free_i32(addr); + tcg_temp_free_i32(tmp); + tcg_temp_free_i64(tmp64); + + gen_neon_ldst_base_update(s, a->rm, a->rn, nregs * interleave * 8); + return true; +} diff --git a/target/arm/translate.c b/target/arm/translate.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate.c +++ b/target/arm/translate.c @@ -XXX,XX +XXX,XX @@ static void gen_neon_trn_u16(TCGv_i32 t0, TCGv_i32 t1) } -static struct { - int nregs; - int interleave; - int spacing; -} const neon_ls_element_type[11] = { - {1, 4, 1}, - {1, 4, 2}, - {4, 1, 1}, - {2, 2, 2}, - {1, 3, 1}, - {1, 3, 2}, - {3, 1, 1}, - {1, 1, 1}, - {1, 2, 1}, - {1, 2, 2}, - {2, 1, 1} -}; - /* Translate a NEON load/store element instruction. Return nonzero if the instruction is invalid. */ static int disas_neon_ls_insn(DisasContext *s, uint32_t insn) { int rd, rn, rm; - int op; int nregs; - int interleave; - int spacing; int stride; int size; int reg; int load; - int n; int vec_size; - int mmu_idx; - MemOp endian; TCGv_i32 addr; TCGv_i32 tmp; - TCGv_i32 tmp2; - TCGv_i64 tmp64; if (!arm_dc_feature(s, ARM_FEATURE_NEON)) { return 1; @@ -XXX,XX +XXX,XX @@ static int disas_neon_ls_insn(DisasContext *s, uint32_t insn) rn = (insn >> 16) & 0xf; rm = insn & 0xf; load = (insn & (1 << 21)) != 0; - endian = s->be_data; - mmu_idx = get_mem_index(s); if ((insn & (1 << 23)) == 0) { - /* Load store all elements. */ - op = (insn >> 8) & 0xf; - size = (insn >> 6) & 3; - if (op > 10) - return 1; - /* Catch UNDEF cases for bad values of align field */ - switch (op & 0xc) { - case 4: - if (((insn >> 5) & 1) == 1) { - return 1; - } - break; - case 8: - if (((insn >> 4) & 3) == 3) { - return 1; - } - break; - default: - break; - } - nregs = neon_ls_element_type[op].nregs; - interleave = neon_ls_element_type[op].interleave; - spacing = neon_ls_element_type[op].spacing; - if (size == 3 && (interleave | spacing) != 1) { - return 1; - } - /* For our purposes, bytes are always little-endian. */ - if (size == 0) { - endian = MO_LE; - } - /* Consecutive little-endian elements from a single register - * can be promoted to a larger little-endian operation. - */ - if (interleave == 1 && endian == MO_LE) { - size = 3; - } - tmp64 = tcg_temp_new_i64(); - addr = tcg_temp_new_i32(); - tmp2 = tcg_const_i32(1 << size); - load_reg_var(s, addr, rn); - for (reg = 0; reg < nregs; reg++) { - for (n = 0; n < 8 >> size; n++) { - int xs; - for (xs = 0; xs < interleave; xs++) { - int tt = rd + reg + spacing * xs; - - if (load) { - gen_aa32_ld_i64(s, tmp64, addr, mmu_idx, endian | size); - neon_store_element64(tt, n, size, tmp64); - } else { - neon_load_element64(tmp64, tt, n, size); - gen_aa32_st_i64(s, tmp64, addr, mmu_idx, endian | size); - } - tcg_gen_add_i32(addr, addr, tmp2); - } - } - } - tcg_temp_free_i32(addr); - tcg_temp_free_i32(tmp2); - tcg_temp_free_i64(tmp64); - stride = nregs * interleave * 8; + /* Load store all elements -- handled already by decodetree */ + return 1; } else { size = (insn >> 10) & 3; if (size == 3) { -- 2.20.1 Convert the Neon "load single structure to all lanes" insns to decodetree. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200430181003.21682-13-peter.maydell@linaro.org --- target/arm/neon-ls.decode | 5 +++ target/arm/translate-neon.inc.c | 73 +++++++++++++++++++++++++++++++++ target/arm/translate.c | 55 +------------------------ 3 files changed, 80 insertions(+), 53 deletions(-) diff --git a/target/arm/neon-ls.decode b/target/arm/neon-ls.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/neon-ls.decode +++ b/target/arm/neon-ls.decode @@ -XXX,XX +XXX,XX @@ VLDST_multiple 1111 0100 0 . l:1 0 rn:4 .... itype:4 size:2 align:2 rm:4 \ vd=%vd_dp + +# Neon load single element to all lanes + +VLD_all_lanes 1111 0100 1 . 1 0 rn:4 .... 11 n:2 size:2 t:1 a:1 rm:4 \ + vd=%vd_dp diff --git a/target/arm/translate-neon.inc.c b/target/arm/translate-neon.inc.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-neon.inc.c +++ b/target/arm/translate-neon.inc.c @@ -XXX,XX +XXX,XX @@ static bool trans_VLDST_multiple(DisasContext *s, arg_VLDST_multiple *a) gen_neon_ldst_base_update(s, a->rm, a->rn, nregs * interleave * 8); return true; } + +static bool trans_VLD_all_lanes(DisasContext *s, arg_VLD_all_lanes *a) +{ + /* Neon load single structure to all lanes */ + int reg, stride, vec_size; + int vd = a->vd; + int size = a->size; + int nregs = a->n + 1; + TCGv_i32 addr, tmp; + + if (!arm_dc_feature(s, ARM_FEATURE_NEON)) { + return false; + } + + /* UNDEF accesses to D16-D31 if they don't exist */ + if (!dc_isar_feature(aa32_simd_r32, s) && (a->vd & 0x10)) { + return false; + } + + if (size == 3) { + if (nregs != 4 || a->a == 0) { + return false; + } + /* For VLD4 size == 3 a == 1 means 32 bits at 16 byte alignment */ + size = 2; + } + if (nregs == 1 && a->a == 1 && size == 0) { + return false; + } + if (nregs == 3 && a->a == 1) { + return false; + } + + if (!vfp_access_check(s)) { + return true; + } + + /* + * VLD1 to all lanes: T bit indicates how many Dregs to write. + * VLD2/3/4 to all lanes: T bit indicates register stride. + */ + stride = a->t ? 2 : 1; + vec_size = nregs == 1 ? stride * 8 : 8; + + tmp = tcg_temp_new_i32(); + addr = tcg_temp_new_i32(); + load_reg_var(s, addr, a->rn); + for (reg = 0; reg < nregs; reg++) { + gen_aa32_ld_i32(s, tmp, addr, get_mem_index(s), + s->be_data | size); + if ((vd & 1) && vec_size == 16) { + /* + * We cannot write 16 bytes at once because the + * destination is unaligned. + */ + tcg_gen_gvec_dup_i32(size, neon_reg_offset(vd, 0), + 8, 8, tmp); + tcg_gen_gvec_mov(0, neon_reg_offset(vd + 1, 0), + neon_reg_offset(vd, 0), 8, 8); + } else { + tcg_gen_gvec_dup_i32(size, neon_reg_offset(vd, 0), + vec_size, vec_size, tmp); + } + tcg_gen_addi_i32(addr, addr, 1 << size); + vd += stride; + } + tcg_temp_free_i32(tmp); + tcg_temp_free_i32(addr); + + gen_neon_ldst_base_update(s, a->rm, a->rn, (1 << size) * nregs); + + return true; +} diff --git a/target/arm/translate.c b/target/arm/translate.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate.c +++ b/target/arm/translate.c @@ -XXX,XX +XXX,XX @@ static int disas_neon_ls_insn(DisasContext *s, uint32_t insn) int size; int reg; int load; - int vec_size; TCGv_i32 addr; TCGv_i32 tmp; @@ -XXX,XX +XXX,XX @@ static int disas_neon_ls_insn(DisasContext *s, uint32_t insn) } else { size = (insn >> 10) & 3; if (size == 3) { - /* Load single element to all lanes. */ - int a = (insn >> 4) & 1; - if (!load) { - return 1; - } - size = (insn >> 6) & 3; - nregs = ((insn >> 8) & 3) + 1; - - if (size == 3) { - if (nregs != 4 || a == 0) { - return 1; - } - /* For VLD4 size==3 a == 1 means 32 bits at 16 byte alignment */ - size = 2; - } - if (nregs == 1 && a == 1 && size == 0) { - return 1; - } - if (nregs == 3 && a == 1) { - return 1; - } - addr = tcg_temp_new_i32(); - load_reg_var(s, addr, rn); - - /* VLD1 to all lanes: bit 5 indicates how many Dregs to write. - * VLD2/3/4 to all lanes: bit 5 indicates register stride. - */ - stride = (insn & (1 << 5)) ? 2 : 1; - vec_size = nregs == 1 ? stride * 8 : 8; - - tmp = tcg_temp_new_i32(); - for (reg = 0; reg < nregs; reg++) { - gen_aa32_ld_i32(s, tmp, addr, get_mem_index(s), - s->be_data | size); - if ((rd & 1) && vec_size == 16) { - /* We cannot write 16 bytes at once because the - * destination is unaligned. - */ - tcg_gen_gvec_dup_i32(size, neon_reg_offset(rd, 0), - 8, 8, tmp); - tcg_gen_gvec_mov(0, neon_reg_offset(rd + 1, 0), - neon_reg_offset(rd, 0), 8, 8); - } else { - tcg_gen_gvec_dup_i32(size, neon_reg_offset(rd, 0), - vec_size, vec_size, tmp); - } - tcg_gen_addi_i32(addr, addr, 1 << size); - rd += stride; - } - tcg_temp_free_i32(tmp); - tcg_temp_free_i32(addr); - stride = (1 << size) * nregs; + /* Load single element to all lanes -- handled by decodetree */ + return 1; } else { /* Single element. */ int idx = (insn >> 4) & 0xf; -- 2.20.1 Convert the Neon "load/store single structure to one lane" insns to decodetree. As this is the last set of insns in the neon load/store group, we can remove the whole disas_neon_ls_insn() function. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200430181003.21682-14-peter.maydell@linaro.org --- target/arm/neon-ls.decode | 11 +++ target/arm/translate-neon.inc.c | 89 +++++++++++++++++++ target/arm/translate.c | 147 -------------------------------- 3 files changed, 100 insertions(+), 147 deletions(-) diff --git a/target/arm/neon-ls.decode b/target/arm/neon-ls.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/neon-ls.decode +++ b/target/arm/neon-ls.decode @@ -XXX,XX +XXX,XX @@ VLDST_multiple 1111 0100 0 . l:1 0 rn:4 .... itype:4 size:2 align:2 rm:4 \ VLD_all_lanes 1111 0100 1 . 1 0 rn:4 .... 11 n:2 size:2 t:1 a:1 rm:4 \ vd=%vd_dp + +# Neon load/store single structure to one lane +%imm1_5_p1 5:1 !function=plus1 +%imm1_6_p1 6:1 !function=plus1 + +VLDST_single 1111 0100 1 . l:1 0 rn:4 .... 00 n:2 reg_idx:3 align:1 rm:4 \ + vd=%vd_dp size=0 stride=1 +VLDST_single 1111 0100 1 . l:1 0 rn:4 .... 01 n:2 reg_idx:2 align:2 rm:4 \ + vd=%vd_dp size=1 stride=%imm1_5_p1 +VLDST_single 1111 0100 1 . l:1 0 rn:4 .... 10 n:2 reg_idx:1 align:3 rm:4 \ + vd=%vd_dp size=2 stride=%imm1_6_p1 diff --git a/target/arm/translate-neon.inc.c b/target/arm/translate-neon.inc.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-neon.inc.c +++ b/target/arm/translate-neon.inc.c @@ -XXX,XX +XXX,XX @@ * It might be possible to convert it to a standalone .c file eventually. */ +static inline int plus1(DisasContext *s, int x) +{ + return x + 1; +} + /* Include the generated Neon decoder */ #include "decode-neon-dp.inc.c" #include "decode-neon-ls.inc.c" @@ -XXX,XX +XXX,XX @@ static bool trans_VLD_all_lanes(DisasContext *s, arg_VLD_all_lanes *a) return true; } + +static bool trans_VLDST_single(DisasContext *s, arg_VLDST_single *a) +{ + /* Neon load/store single structure to one lane */ + int reg; + int nregs = a->n + 1; + int vd = a->vd; + TCGv_i32 addr, tmp; + + if (!arm_dc_feature(s, ARM_FEATURE_NEON)) { + return false; + } + + /* UNDEF accesses to D16-D31 if they don't exist */ + if (!dc_isar_feature(aa32_simd_r32, s) && (a->vd & 0x10)) { + return false; + } + + /* Catch the UNDEF cases. This is unavoidably a bit messy. */ + switch (nregs) { + case 1: + if (((a->align & (1 << a->size)) != 0) || + (a->size == 2 && ((a->align & 3) == 1 || (a->align & 3) == 2))) { + return false; + } + break; + case 3: + if ((a->align & 1) != 0) { + return false; + } + /* fall through */ + case 2: + if (a->size == 2 && (a->align & 2) != 0) { + return false; + } + break; + case 4: + if ((a->size == 2) && ((a->align & 3) == 3)) { + return false; + } + break; + default: + abort(); + } + if ((vd + a->stride * (nregs - 1)) > 31) { + /* + * Attempts to write off the end of the register file are + * UNPREDICTABLE; we choose to UNDEF because otherwise we would + * access off the end of the array that holds the register data. + */ + return false; + } + + if (!vfp_access_check(s)) { + return true; + } + + tmp = tcg_temp_new_i32(); + addr = tcg_temp_new_i32(); + load_reg_var(s, addr, a->rn); + /* + * TODO: if we implemented alignment exceptions, we should check + * addr against the alignment encoded in a->align here. + */ + for (reg = 0; reg < nregs; reg++) { + if (a->l) { + gen_aa32_ld_i32(s, tmp, addr, get_mem_index(s), + s->be_data | a->size); + neon_store_element(vd, a->reg_idx, a->size, tmp); + } else { /* Store */ + neon_load_element(tmp, vd, a->reg_idx, a->size); + gen_aa32_st_i32(s, tmp, addr, get_mem_index(s), + s->be_data | a->size); + } + vd += a->stride; + tcg_gen_addi_i32(addr, addr, 1 << a->size); + } + tcg_temp_free_i32(addr); + tcg_temp_free_i32(tmp); + + gen_neon_ldst_base_update(s, a->rm, a->rn, (1 << a->size) * nregs); + + return true; +} diff --git a/target/arm/translate.c b/target/arm/translate.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate.c +++ b/target/arm/translate.c @@ -XXX,XX +XXX,XX @@ static void gen_neon_trn_u16(TCGv_i32 t0, TCGv_i32 t1) tcg_temp_free_i32(rd); } - -/* Translate a NEON load/store element instruction. Return nonzero if the - instruction is invalid. */ -static int disas_neon_ls_insn(DisasContext *s, uint32_t insn) -{ - int rd, rn, rm; - int nregs; - int stride; - int size; - int reg; - int load; - TCGv_i32 addr; - TCGv_i32 tmp; - - if (!arm_dc_feature(s, ARM_FEATURE_NEON)) { - return 1; - } - - /* FIXME: this access check should not take precedence over UNDEF - * for invalid encodings; we will generate incorrect syndrome information - * for attempts to execute invalid vfp/neon encodings with FP disabled. - */ - if (s->fp_excp_el) { - gen_exception_insn(s, s->pc_curr, EXCP_UDEF, - syn_simd_access_trap(1, 0xe, false), s->fp_excp_el); - return 0; - } - - if (!s->vfp_enabled) - return 1; - VFP_DREG_D(rd, insn); - rn = (insn >> 16) & 0xf; - rm = insn & 0xf; - load = (insn & (1 << 21)) != 0; - if ((insn & (1 << 23)) == 0) { - /* Load store all elements -- handled already by decodetree */ - return 1; - } else { - size = (insn >> 10) & 3; - if (size == 3) { - /* Load single element to all lanes -- handled by decodetree */ - return 1; - } else { - /* Single element. */ - int idx = (insn >> 4) & 0xf; - int reg_idx; - switch (size) { - case 0: - reg_idx = (insn >> 5) & 7; - stride = 1; - break; - case 1: - reg_idx = (insn >> 6) & 3; - stride = (insn & (1 << 5)) ? 2 : 1; - break; - case 2: - reg_idx = (insn >> 7) & 1; - stride = (insn & (1 << 6)) ? 2 : 1; - break; - default: - abort(); - } - nregs = ((insn >> 8) & 3) + 1; - /* Catch the UNDEF cases. This is unavoidably a bit messy. */ - switch (nregs) { - case 1: - if (((idx & (1 << size)) != 0) || - (size == 2 && ((idx & 3) == 1 || (idx & 3) == 2))) { - return 1; - } - break; - case 3: - if ((idx & 1) != 0) { - return 1; - } - /* fall through */ - case 2: - if (size == 2 && (idx & 2) != 0) { - return 1; - } - break; - case 4: - if ((size == 2) && ((idx & 3) == 3)) { - return 1; - } - break; - default: - abort(); - } - if ((rd + stride * (nregs - 1)) > 31) { - /* Attempts to write off the end of the register file - * are UNPREDICTABLE; we choose to UNDEF because otherwise - * the neon_load_reg() would write off the end of the array. - */ - return 1; - } - tmp = tcg_temp_new_i32(); - addr = tcg_temp_new_i32(); - load_reg_var(s, addr, rn); - for (reg = 0; reg < nregs; reg++) { - if (load) { - gen_aa32_ld_i32(s, tmp, addr, get_mem_index(s), - s->be_data | size); - neon_store_element(rd, reg_idx, size, tmp); - } else { /* Store */ - neon_load_element(tmp, rd, reg_idx, size); - gen_aa32_st_i32(s, tmp, addr, get_mem_index(s), - s->be_data | size); - } - rd += stride; - tcg_gen_addi_i32(addr, addr, 1 << size); - } - tcg_temp_free_i32(addr); - tcg_temp_free_i32(tmp); - stride = nregs * (1 << size); - } - } - if (rm != 15) { - TCGv_i32 base; - - base = load_reg(s, rn); - if (rm == 13) { - tcg_gen_addi_i32(base, base, stride); - } else { - TCGv_i32 index; - index = load_reg(s, rm); - tcg_gen_add_i32(base, base, index); - tcg_temp_free_i32(index); - } - store_reg(s, rn, base); - } - return 0; -} - static inline void gen_neon_narrow(int size, TCGv_i32 dest, TCGv_i64 src) { switch (size) { @@ -XXX,XX +XXX,XX @@ static void disas_arm_insn(DisasContext *s, unsigned int insn) } return; } - if ((insn & 0x0f100000) == 0x04000000) { - /* NEON load/store. */ - if (disas_neon_ls_insn(s, insn)) { - goto illegal_op; - } - return; - } if ((insn & 0x0e000f00) == 0x0c000100) { if (arm_dc_feature(s, ARM_FEATURE_IWMMXT)) { /* iWMMXt register transfer. */ @@ -XXX,XX +XXX,XX @@ static void disas_thumb2_insn(DisasContext *s, uint32_t insn) } break; case 12: - if ((insn & 0x01100000) == 0x01000000) { - if (disas_neon_ls_insn(s, insn)) { - goto illegal_op; - } - break; - } goto illegal_op; default: illegal_op: -- 2.20.1 Convert the Neon 3-reg-same VADD and VSUB insns to decodetree. Note that we don't need the neon_3r_sizes[op] check here because all size values are OK for VADD and VSUB; we'll add this when we convert the first insn that has size restrictions. For this we need one of the GVecGen*Fn typedefs currently in translate-a64.h; move them all to translate.h as a block so they are visible to the 32-bit decoder. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200430181003.21682-15-peter.maydell@linaro.org --- target/arm/translate-a64.h | 9 -------- target/arm/translate.h | 9 ++++++++ target/arm/neon-dp.decode | 17 +++++++++++++++ target/arm/translate-neon.inc.c | 38 +++++++++++++++++++++++++++++++++ target/arm/translate.c | 14 ++++-------- 5 files changed, 68 insertions(+), 19 deletions(-) diff --git a/target/arm/translate-a64.h b/target/arm/translate-a64.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-a64.h +++ b/target/arm/translate-a64.h @@ -XXX,XX +XXX,XX @@ static inline int vec_full_reg_size(DisasContext *s) bool disas_sve(DisasContext *, uint32_t); -/* Note that the gvec expanders operate on offsets + sizes. */ -typedef void GVecGen2Fn(unsigned, uint32_t, uint32_t, uint32_t, uint32_t); -typedef void GVecGen2iFn(unsigned, uint32_t, uint32_t, int64_t, - uint32_t, uint32_t); -typedef void GVecGen3Fn(unsigned, uint32_t, uint32_t, - uint32_t, uint32_t, uint32_t); -typedef void GVecGen4Fn(unsigned, uint32_t, uint32_t, uint32_t, - uint32_t, uint32_t, uint32_t); - #endif /* TARGET_ARM_TRANSLATE_A64_H */ diff --git a/target/arm/translate.h b/target/arm/translate.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate.h +++ b/target/arm/translate.h @@ -XXX,XX +XXX,XX @@ void gen_sshl_i64(TCGv_i64 d, TCGv_i64 a, TCGv_i64 b); #define dc_isar_feature(name, ctx) \ ({ DisasContext *ctx_ = (ctx); isar_feature_##name(ctx_->isar); }) +/* Note that the gvec expanders operate on offsets + sizes. */ +typedef void GVecGen2Fn(unsigned, uint32_t, uint32_t, uint32_t, uint32_t); +typedef void GVecGen2iFn(unsigned, uint32_t, uint32_t, int64_t, + uint32_t, uint32_t); +typedef void GVecGen3Fn(unsigned, uint32_t, uint32_t, + uint32_t, uint32_t, uint32_t); +typedef void GVecGen4Fn(unsigned, uint32_t, uint32_t, uint32_t, + uint32_t, uint32_t, uint32_t); + #endif /* TARGET_ARM_TRANSLATE_H */ diff --git a/target/arm/neon-dp.decode b/target/arm/neon-dp.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/neon-dp.decode +++ b/target/arm/neon-dp.decode @@ -XXX,XX +XXX,XX @@ # # This file is processed by scripts/decodetree.py # +# VFP/Neon register fields; same as vfp.decode +%vm_dp 5:1 0:4 +%vn_dp 7:1 16:4 +%vd_dp 22:1 12:4 # Encodings for Neon data processing instructions where the T32 encoding # is a simple transformation of the A32 encoding. @@ -XXX,XX +XXX,XX @@ # 0b111p_1111_qqqq_qqqq_qqqq_qqqq_qqqq_qqqq # This file works on the A32 encoding only; calling code for T32 has to # transform the insn into the A32 version first. + +###################################################################### +# 3-reg-same grouping: +# 1111 001 U 0 D sz:2 Vn:4 Vd:4 opc:4 N Q M op Vm:4 +###################################################################### + +&3same vm vn vd q size + +@3same .... ... . . . size:2 .... .... .... . q:1 . . .... \ + &3same vm=%vm_dp vn=%vn_dp vd=%vd_dp + +VADD_3s 1111 001 0 0 . .. .... .... 1000 . . . 0 .... @3same +VSUB_3s 1111 001 1 0 . .. .... .... 1000 . . . 0 .... @3same diff --git a/target/arm/translate-neon.inc.c b/target/arm/translate-neon.inc.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-neon.inc.c +++ b/target/arm/translate-neon.inc.c @@ -XXX,XX +XXX,XX @@ static bool trans_VLDST_single(DisasContext *s, arg_VLDST_single *a) return true; } + +static bool do_3same(DisasContext *s, arg_3same *a, GVecGen3Fn fn) +{ + int vec_size = a->q ? 16 : 8; + int rd_ofs = neon_reg_offset(a->vd, 0); + int rn_ofs = neon_reg_offset(a->vn, 0); + int rm_ofs = neon_reg_offset(a->vm, 0); + + if (!arm_dc_feature(s, ARM_FEATURE_NEON)) { + return false; + } + + /* UNDEF accesses to D16-D31 if they don't exist. */ + if (!dc_isar_feature(aa32_simd_r32, s) && + ((a->vd | a->vn | a->vm) & 0x10)) { + return false; + } + + if ((a->vn | a->vm | a->vd) & a->q) { + return false; + } + + if (!vfp_access_check(s)) { + return true; + } + + fn(a->size, rd_ofs, rn_ofs, rm_ofs, vec_size, vec_size); + return true; +} + +#define DO_3SAME(INSN, FUNC) \ + static bool trans_##INSN##_3s(DisasContext *s, arg_3same *a) \ + { \ + return do_3same(s, a, FUNC); \ + } + +DO_3SAME(VADD, tcg_gen_gvec_add) +DO_3SAME(VSUB, tcg_gen_gvec_sub) diff --git a/target/arm/translate.c b/target/arm/translate.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate.c +++ b/target/arm/translate.c @@ -XXX,XX +XXX,XX @@ static int disas_neon_data_insn(DisasContext *s, uint32_t insn) } return 0; - case NEON_3R_VADD_VSUB: - if (u) { - tcg_gen_gvec_sub(size, rd_ofs, rn_ofs, rm_ofs, - vec_size, vec_size); - } else { - tcg_gen_gvec_add(size, rd_ofs, rn_ofs, rm_ofs, - vec_size, vec_size); - } - return 0; - case NEON_3R_VQADD: tcg_gen_gvec_4(rd_ofs, offsetof(CPUARMState, vfp.qc), rn_ofs, rm_ofs, vec_size, vec_size, @@ -XXX,XX +XXX,XX @@ static int disas_neon_data_insn(DisasContext *s, uint32_t insn) tcg_gen_gvec_3(rd_ofs, rm_ofs, rn_ofs, vec_size, vec_size, u ? &ushl_op[size] : &sshl_op[size]); return 0; + + case NEON_3R_VADD_VSUB: + /* Already handled by decodetree */ + return 1; } if (size == 3) { -- 2.20.1 Convert the Neon logic ops in the 3-reg-same grouping to decodetree. Note that for the logic ops the 'size' field forms part of their decode and the actual operations are always bitwise. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200430181003.21682-16-peter.maydell@linaro.org --- target/arm/neon-dp.decode | 12 +++++++++++ target/arm/translate-neon.inc.c | 19 +++++++++++++++++ target/arm/translate.c | 38 +-------------------------------- 3 files changed, 32 insertions(+), 37 deletions(-) diff --git a/target/arm/neon-dp.decode b/target/arm/neon-dp.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/neon-dp.decode +++ b/target/arm/neon-dp.decode @@ -XXX,XX +XXX,XX @@ @3same .... ... . . . size:2 .... .... .... . q:1 . . .... \ &3same vm=%vm_dp vn=%vn_dp vd=%vd_dp +@3same_logic .... ... . . . .. .... .... .... . q:1 .. .... \ + &3same vm=%vm_dp vn=%vn_dp vd=%vd_dp size=0 + +VAND_3s 1111 001 0 0 . 00 .... .... 0001 ... 1 .... @3same_logic +VBIC_3s 1111 001 0 0 . 01 .... .... 0001 ... 1 .... @3same_logic +VORR_3s 1111 001 0 0 . 10 .... .... 0001 ... 1 .... @3same_logic +VORN_3s 1111 001 0 0 . 11 .... .... 0001 ... 1 .... @3same_logic +VEOR_3s 1111 001 1 0 . 00 .... .... 0001 ... 1 .... @3same_logic +VBSL_3s 1111 001 1 0 . 01 .... .... 0001 ... 1 .... @3same_logic +VBIT_3s 1111 001 1 0 . 10 .... .... 0001 ... 1 .... @3same_logic +VBIF_3s 1111 001 1 0 . 11 .... .... 0001 ... 1 .... @3same_logic + VADD_3s 1111 001 0 0 . .. .... .... 1000 . . . 0 .... @3same VSUB_3s 1111 001 1 0 . .. .... .... 1000 . . . 0 .... @3same diff --git a/target/arm/translate-neon.inc.c b/target/arm/translate-neon.inc.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-neon.inc.c +++ b/target/arm/translate-neon.inc.c @@ -XXX,XX +XXX,XX @@ static bool do_3same(DisasContext *s, arg_3same *a, GVecGen3Fn fn) DO_3SAME(VADD, tcg_gen_gvec_add) DO_3SAME(VSUB, tcg_gen_gvec_sub) +DO_3SAME(VAND, tcg_gen_gvec_and) +DO_3SAME(VBIC, tcg_gen_gvec_andc) +DO_3SAME(VORR, tcg_gen_gvec_or) +DO_3SAME(VORN, tcg_gen_gvec_orc) +DO_3SAME(VEOR, tcg_gen_gvec_xor) + +/* These insns are all gvec_bitsel but with the inputs in various orders. */ +#define DO_3SAME_BITSEL(INSN, O1, O2, O3) \ + static void gen_##INSN##_3s(unsigned vece, uint32_t rd_ofs, \ + uint32_t rn_ofs, uint32_t rm_ofs, \ + uint32_t oprsz, uint32_t maxsz) \ + { \ + tcg_gen_gvec_bitsel(vece, rd_ofs, O1, O2, O3, oprsz, maxsz); \ + } \ + DO_3SAME(INSN, gen_##INSN##_3s) + +DO_3SAME_BITSEL(VBSL, rd_ofs, rn_ofs, rm_ofs) +DO_3SAME_BITSEL(VBIT, rm_ofs, rn_ofs, rd_ofs) +DO_3SAME_BITSEL(VBIF, rm_ofs, rd_ofs, rn_ofs) diff --git a/target/arm/translate.c b/target/arm/translate.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate.c +++ b/target/arm/translate.c @@ -XXX,XX +XXX,XX @@ static int disas_neon_data_insn(DisasContext *s, uint32_t insn) } return 1; - case NEON_3R_LOGIC: /* Logic ops. */ - switch ((u << 2) | size) { - case 0: /* VAND */ - tcg_gen_gvec_and(0, rd_ofs, rn_ofs, rm_ofs, - vec_size, vec_size); - break; - case 1: /* VBIC */ - tcg_gen_gvec_andc(0, rd_ofs, rn_ofs, rm_ofs, - vec_size, vec_size); - break; - case 2: /* VORR */ - tcg_gen_gvec_or(0, rd_ofs, rn_ofs, rm_ofs, - vec_size, vec_size); - break; - case 3: /* VORN */ - tcg_gen_gvec_orc(0, rd_ofs, rn_ofs, rm_ofs, - vec_size, vec_size); - break; - case 4: /* VEOR */ - tcg_gen_gvec_xor(0, rd_ofs, rn_ofs, rm_ofs, - vec_size, vec_size); - break; - case 5: /* VBSL */ - tcg_gen_gvec_bitsel(MO_8, rd_ofs, rd_ofs, rn_ofs, rm_ofs, - vec_size, vec_size); - break; - case 6: /* VBIT */ - tcg_gen_gvec_bitsel(MO_8, rd_ofs, rm_ofs, rn_ofs, rd_ofs, - vec_size, vec_size); - break; - case 7: /* VBIF */ - tcg_gen_gvec_bitsel(MO_8, rd_ofs, rm_ofs, rd_ofs, rn_ofs, - vec_size, vec_size); - break; - } - return 0; - case NEON_3R_VQADD: tcg_gen_gvec_4(rd_ofs, offsetof(CPUARMState, vfp.qc), rn_ofs, rm_ofs, vec_size, vec_size, @@ -XXX,XX +XXX,XX @@ static int disas_neon_data_insn(DisasContext *s, uint32_t insn) return 0; case NEON_3R_VADD_VSUB: + case NEON_3R_LOGIC: /* Already handled by decodetree */ return 1; } -- 2.20.1 Convert the Neon 3-reg-same VMAX and VMIN insns to decodetree. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200430181003.21682-17-peter.maydell@linaro.org --- target/arm/neon-dp.decode | 5 +++++ target/arm/translate-neon.inc.c | 14 ++++++++++++++ target/arm/translate.c | 21 ++------------------- 3 files changed, 21 insertions(+), 19 deletions(-) diff --git a/target/arm/neon-dp.decode b/target/arm/neon-dp.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/neon-dp.decode +++ b/target/arm/neon-dp.decode @@ -XXX,XX +XXX,XX @@ VBSL_3s 1111 001 1 0 . 01 .... .... 0001 ... 1 .... @3same_logic VBIT_3s 1111 001 1 0 . 10 .... .... 0001 ... 1 .... @3same_logic VBIF_3s 1111 001 1 0 . 11 .... .... 0001 ... 1 .... @3same_logic +VMAX_S_3s 1111 001 0 0 . .. .... .... 0110 . . . 0 .... @3same +VMAX_U_3s 1111 001 1 0 . .. .... .... 0110 . . . 0 .... @3same +VMIN_S_3s 1111 001 0 0 . .. .... .... 0110 . . . 1 .... @3same +VMIN_U_3s 1111 001 1 0 . .. .... .... 0110 . . . 1 .... @3same + VADD_3s 1111 001 0 0 . .. .... .... 1000 . . . 0 .... @3same VSUB_3s 1111 001 1 0 . .. .... .... 1000 . . . 0 .... @3same diff --git a/target/arm/translate-neon.inc.c b/target/arm/translate-neon.inc.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-neon.inc.c +++ b/target/arm/translate-neon.inc.c @@ -XXX,XX +XXX,XX @@ DO_3SAME(VEOR, tcg_gen_gvec_xor) DO_3SAME_BITSEL(VBSL, rd_ofs, rn_ofs, rm_ofs) DO_3SAME_BITSEL(VBIT, rm_ofs, rn_ofs, rd_ofs) DO_3SAME_BITSEL(VBIF, rm_ofs, rd_ofs, rn_ofs) + +#define DO_3SAME_NO_SZ_3(INSN, FUNC) \ + static bool trans_##INSN##_3s(DisasContext *s, arg_3same *a) \ + { \ + if (a->size == 3) { \ + return false; \ + } \ + return do_3same(s, a, FUNC); \ + } + +DO_3SAME_NO_SZ_3(VMAX_S, tcg_gen_gvec_smax) +DO_3SAME_NO_SZ_3(VMAX_U, tcg_gen_gvec_umax) +DO_3SAME_NO_SZ_3(VMIN_S, tcg_gen_gvec_smin) +DO_3SAME_NO_SZ_3(VMIN_U, tcg_gen_gvec_umin) diff --git a/target/arm/translate.c b/target/arm/translate.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate.c +++ b/target/arm/translate.c @@ -XXX,XX +XXX,XX @@ static int disas_neon_data_insn(DisasContext *s, uint32_t insn) rd_ofs, rn_ofs, rm_ofs, vec_size, vec_size); return 0; - case NEON_3R_VMAX: - if (u) { - tcg_gen_gvec_umax(size, rd_ofs, rn_ofs, rm_ofs, - vec_size, vec_size); - } else { - tcg_gen_gvec_smax(size, rd_ofs, rn_ofs, rm_ofs, - vec_size, vec_size); - } - return 0; - case NEON_3R_VMIN: - if (u) { - tcg_gen_gvec_umin(size, rd_ofs, rn_ofs, rm_ofs, - vec_size, vec_size); - } else { - tcg_gen_gvec_smin(size, rd_ofs, rn_ofs, rm_ofs, - vec_size, vec_size); - } - return 0; - case NEON_3R_VSHL: /* Note the operation is vshl vd,vm,vn */ tcg_gen_gvec_3(rd_ofs, rm_ofs, rn_ofs, vec_size, vec_size, @@ -XXX,XX +XXX,XX @@ static int disas_neon_data_insn(DisasContext *s, uint32_t insn) case NEON_3R_VADD_VSUB: case NEON_3R_LOGIC: + case NEON_3R_VMAX: + case NEON_3R_VMIN: /* Already handled by decodetree */ return 1; } -- 2.20.1 Convert the Neon comparison ops in the 3-reg-same grouping to decodetree. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200430181003.21682-18-peter.maydell@linaro.org --- target/arm/neon-dp.decode | 8 ++++++++ target/arm/translate-neon.inc.c | 22 ++++++++++++++++++++++ target/arm/translate.c | 23 +++-------------------- 3 files changed, 33 insertions(+), 20 deletions(-) diff --git a/target/arm/neon-dp.decode b/target/arm/neon-dp.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/neon-dp.decode +++ b/target/arm/neon-dp.decode @@ -XXX,XX +XXX,XX @@ VBSL_3s 1111 001 1 0 . 01 .... .... 0001 ... 1 .... @3same_logic VBIT_3s 1111 001 1 0 . 10 .... .... 0001 ... 1 .... @3same_logic VBIF_3s 1111 001 1 0 . 11 .... .... 0001 ... 1 .... @3same_logic +VCGT_S_3s 1111 001 0 0 . .. .... .... 0011 . . . 0 .... @3same +VCGT_U_3s 1111 001 1 0 . .. .... .... 0011 . . . 0 .... @3same +VCGE_S_3s 1111 001 0 0 . .. .... .... 0011 . . . 1 .... @3same +VCGE_U_3s 1111 001 1 0 . .. .... .... 0011 . . . 1 .... @3same + VMAX_S_3s 1111 001 0 0 . .. .... .... 0110 . . . 0 .... @3same VMAX_U_3s 1111 001 1 0 . .. .... .... 0110 . . . 0 .... @3same VMIN_S_3s 1111 001 0 0 . .. .... .... 0110 . . . 1 .... @3same @@ -XXX,XX +XXX,XX @@ VMIN_U_3s 1111 001 1 0 . .. .... .... 0110 . . . 1 .... @3same VADD_3s 1111 001 0 0 . .. .... .... 1000 . . . 0 .... @3same VSUB_3s 1111 001 1 0 . .. .... .... 1000 . . . 0 .... @3same + +VTST_3s 1111 001 0 0 . .. .... .... 1000 . . . 1 .... @3same +VCEQ_3s 1111 001 1 0 . .. .... .... 1000 . . . 1 .... @3same diff --git a/target/arm/translate-neon.inc.c b/target/arm/translate-neon.inc.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-neon.inc.c +++ b/target/arm/translate-neon.inc.c @@ -XXX,XX +XXX,XX @@ DO_3SAME_NO_SZ_3(VMAX_S, tcg_gen_gvec_smax) DO_3SAME_NO_SZ_3(VMAX_U, tcg_gen_gvec_umax) DO_3SAME_NO_SZ_3(VMIN_S, tcg_gen_gvec_smin) DO_3SAME_NO_SZ_3(VMIN_U, tcg_gen_gvec_umin) + +#define DO_3SAME_CMP(INSN, COND) \ + static void gen_##INSN##_3s(unsigned vece, uint32_t rd_ofs, \ + uint32_t rn_ofs, uint32_t rm_ofs, \ + uint32_t oprsz, uint32_t maxsz) \ + { \ + tcg_gen_gvec_cmp(COND, vece, rd_ofs, rn_ofs, rm_ofs, oprsz, maxsz); \ + } \ + DO_3SAME_NO_SZ_3(INSN, gen_##INSN##_3s) + +DO_3SAME_CMP(VCGT_S, TCG_COND_GT) +DO_3SAME_CMP(VCGT_U, TCG_COND_GTU) +DO_3SAME_CMP(VCGE_S, TCG_COND_GE) +DO_3SAME_CMP(VCGE_U, TCG_COND_GEU) +DO_3SAME_CMP(VCEQ, TCG_COND_EQ) + +static void gen_VTST_3s(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, + uint32_t rm_ofs, uint32_t oprsz, uint32_t maxsz) +{ + tcg_gen_gvec_3(rd_ofs, rn_ofs, rm_ofs, oprsz, maxsz, &cmtst_op[vece]); +} +DO_3SAME_NO_SZ_3(VTST, gen_VTST_3s) diff --git a/target/arm/translate.c b/target/arm/translate.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate.c +++ b/target/arm/translate.c @@ -XXX,XX +XXX,XX @@ static int disas_neon_data_insn(DisasContext *s, uint32_t insn) u ? &mls_op[size] : &mla_op[size]); return 0; - case NEON_3R_VTST_VCEQ: - if (u) { /* VCEQ */ - tcg_gen_gvec_cmp(TCG_COND_EQ, size, rd_ofs, rn_ofs, rm_ofs, - vec_size, vec_size); - } else { /* VTST */ - tcg_gen_gvec_3(rd_ofs, rn_ofs, rm_ofs, - vec_size, vec_size, &cmtst_op[size]); - } - return 0; - - case NEON_3R_VCGT: - tcg_gen_gvec_cmp(u ? TCG_COND_GTU : TCG_COND_GT, size, - rd_ofs, rn_ofs, rm_ofs, vec_size, vec_size); - return 0; - - case NEON_3R_VCGE: - tcg_gen_gvec_cmp(u ? TCG_COND_GEU : TCG_COND_GE, size, - rd_ofs, rn_ofs, rm_ofs, vec_size, vec_size); - return 0; - case NEON_3R_VSHL: /* Note the operation is vshl vd,vm,vn */ tcg_gen_gvec_3(rd_ofs, rm_ofs, rn_ofs, vec_size, vec_size, @@ -XXX,XX +XXX,XX @@ static int disas_neon_data_insn(DisasContext *s, uint32_t insn) case NEON_3R_LOGIC: case NEON_3R_VMAX: case NEON_3R_VMIN: + case NEON_3R_VTST_VCEQ: + case NEON_3R_VCGT: + case NEON_3R_VCGE: /* Already handled by decodetree */ return 1; } -- 2.20.1 Convert the Neon VQADD/VQSUB insns in the 3-reg-same grouping to decodetree. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200430181003.21682-19-peter.maydell@linaro.org --- target/arm/neon-dp.decode | 6 ++++++ target/arm/translate-neon.inc.c | 15 +++++++++++++++ target/arm/translate.c | 14 ++------------ 3 files changed, 23 insertions(+), 12 deletions(-) diff --git a/target/arm/neon-dp.decode b/target/arm/neon-dp.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/neon-dp.decode +++ b/target/arm/neon-dp.decode @@ -XXX,XX +XXX,XX @@ @3same .... ... . . . size:2 .... .... .... . q:1 . . .... \ &3same vm=%vm_dp vn=%vn_dp vd=%vd_dp +VQADD_S_3s 1111 001 0 0 . .. .... .... 0000 . . . 1 .... @3same +VQADD_U_3s 1111 001 1 0 . .. .... .... 0000 . . . 1 .... @3same + @3same_logic .... ... . . . .. .... .... .... . q:1 .. .... \ &3same vm=%vm_dp vn=%vn_dp vd=%vd_dp size=0 @@ -XXX,XX +XXX,XX @@ VBSL_3s 1111 001 1 0 . 01 .... .... 0001 ... 1 .... @3same_logic VBIT_3s 1111 001 1 0 . 10 .... .... 0001 ... 1 .... @3same_logic VBIF_3s 1111 001 1 0 . 11 .... .... 0001 ... 1 .... @3same_logic +VQSUB_S_3s 1111 001 0 0 . .. .... .... 0010 . . . 1 .... @3same +VQSUB_U_3s 1111 001 1 0 . .. .... .... 0010 . . . 1 .... @3same + VCGT_S_3s 1111 001 0 0 . .. .... .... 0011 . . . 0 .... @3same VCGT_U_3s 1111 001 1 0 . .. .... .... 0011 . . . 0 .... @3same VCGE_S_3s 1111 001 0 0 . .. .... .... 0011 . . . 1 .... @3same diff --git a/target/arm/translate-neon.inc.c b/target/arm/translate-neon.inc.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-neon.inc.c +++ b/target/arm/translate-neon.inc.c @@ -XXX,XX +XXX,XX @@ static void gen_VTST_3s(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, tcg_gen_gvec_3(rd_ofs, rn_ofs, rm_ofs, oprsz, maxsz, &cmtst_op[vece]); } DO_3SAME_NO_SZ_3(VTST, gen_VTST_3s) + +#define DO_3SAME_GVEC4(INSN, OPARRAY) \ + static void gen_##INSN##_3s(unsigned vece, uint32_t rd_ofs, \ + uint32_t rn_ofs, uint32_t rm_ofs, \ + uint32_t oprsz, uint32_t maxsz) \ + { \ + tcg_gen_gvec_4(rd_ofs, offsetof(CPUARMState, vfp.qc), \ + rn_ofs, rm_ofs, oprsz, maxsz, &OPARRAY[vece]); \ + } \ + DO_3SAME(INSN, gen_##INSN##_3s) + +DO_3SAME_GVEC4(VQADD_S, sqadd_op) +DO_3SAME_GVEC4(VQADD_U, uqadd_op) +DO_3SAME_GVEC4(VQSUB_S, sqsub_op) +DO_3SAME_GVEC4(VQSUB_U, uqsub_op) diff --git a/target/arm/translate.c b/target/arm/translate.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate.c +++ b/target/arm/translate.c @@ -XXX,XX +XXX,XX @@ static int disas_neon_data_insn(DisasContext *s, uint32_t insn) } return 1; - case NEON_3R_VQADD: - tcg_gen_gvec_4(rd_ofs, offsetof(CPUARMState, vfp.qc), - rn_ofs, rm_ofs, vec_size, vec_size, - (u ? uqadd_op : sqadd_op) + size); - return 0; - - case NEON_3R_VQSUB: - tcg_gen_gvec_4(rd_ofs, offsetof(CPUARMState, vfp.qc), - rn_ofs, rm_ofs, vec_size, vec_size, - (u ? uqsub_op : sqsub_op) + size); - return 0; - case NEON_3R_VMUL: /* VMUL */ if (u) { /* Polynomial case allows only P8. */ @@ -XXX,XX +XXX,XX @@ static int disas_neon_data_insn(DisasContext *s, uint32_t insn) case NEON_3R_VTST_VCEQ: case NEON_3R_VCGT: case NEON_3R_VCGE: + case NEON_3R_VQADD: + case NEON_3R_VQSUB: /* Already handled by decodetree */ return 1; } -- 2.20.1 Convert the Neon VMUL, VMLA, VMLS and VSHL insns in the 3-reg-same grouping to decodetree. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200430181003.21682-20-peter.maydell@linaro.org --- target/arm/neon-dp.decode | 9 +++++++ target/arm/translate-neon.inc.c | 44 +++++++++++++++++++++++++++++++++ target/arm/translate.c | 28 +++------------------ 3 files changed, 56 insertions(+), 25 deletions(-) diff --git a/target/arm/neon-dp.decode b/target/arm/neon-dp.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/neon-dp.decode +++ b/target/arm/neon-dp.decode @@ -XXX,XX +XXX,XX @@ VCGT_U_3s 1111 001 1 0 . .. .... .... 0011 . . . 0 .... @3same VCGE_S_3s 1111 001 0 0 . .. .... .... 0011 . . . 1 .... @3same VCGE_U_3s 1111 001 1 0 . .. .... .... 0011 . . . 1 .... @3same +VSHL_S_3s 1111 001 0 0 . .. .... .... 0100 . . . 0 .... @3same +VSHL_U_3s 1111 001 1 0 . .. .... .... 0100 . . . 0 .... @3same + VMAX_S_3s 1111 001 0 0 . .. .... .... 0110 . . . 0 .... @3same VMAX_U_3s 1111 001 1 0 . .. .... .... 0110 . . . 0 .... @3same VMIN_S_3s 1111 001 0 0 . .. .... .... 0110 . . . 1 .... @3same @@ -XXX,XX +XXX,XX @@ VSUB_3s 1111 001 1 0 . .. .... .... 1000 . . . 0 .... @3same VTST_3s 1111 001 0 0 . .. .... .... 1000 . . . 1 .... @3same VCEQ_3s 1111 001 1 0 . .. .... .... 1000 . . . 1 .... @3same + +VMLA_3s 1111 001 0 0 . .. .... .... 1001 . . . 0 .... @3same +VMLS_3s 1111 001 1 0 . .. .... .... 1001 . . . 0 .... @3same + +VMUL_3s 1111 001 0 0 . .. .... .... 1001 . . . 1 .... @3same +VMUL_p_3s 1111 001 1 0 . .. .... .... 1001 . . . 1 .... @3same diff --git a/target/arm/translate-neon.inc.c b/target/arm/translate-neon.inc.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-neon.inc.c +++ b/target/arm/translate-neon.inc.c @@ -XXX,XX +XXX,XX @@ DO_3SAME_NO_SZ_3(VMAX_S, tcg_gen_gvec_smax) DO_3SAME_NO_SZ_3(VMAX_U, tcg_gen_gvec_umax) DO_3SAME_NO_SZ_3(VMIN_S, tcg_gen_gvec_smin) DO_3SAME_NO_SZ_3(VMIN_U, tcg_gen_gvec_umin) +DO_3SAME_NO_SZ_3(VMUL, tcg_gen_gvec_mul) #define DO_3SAME_CMP(INSN, COND) \ static void gen_##INSN##_3s(unsigned vece, uint32_t rd_ofs, \ @@ -XXX,XX +XXX,XX @@ DO_3SAME_GVEC4(VQADD_S, sqadd_op) DO_3SAME_GVEC4(VQADD_U, uqadd_op) DO_3SAME_GVEC4(VQSUB_S, sqsub_op) DO_3SAME_GVEC4(VQSUB_U, uqsub_op) + +static void gen_VMUL_p_3s(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, + uint32_t rm_ofs, uint32_t oprsz, uint32_t maxsz) +{ + tcg_gen_gvec_3_ool(rd_ofs, rn_ofs, rm_ofs, oprsz, maxsz, + 0, gen_helper_gvec_pmul_b); +} + +static bool trans_VMUL_p_3s(DisasContext *s, arg_3same *a) +{ + if (a->size != 0) { + return false; + } + return do_3same(s, a, gen_VMUL_p_3s); +} + +#define DO_3SAME_GVEC3_NO_SZ_3(INSN, OPARRAY) \ + static void gen_##INSN##_3s(unsigned vece, uint32_t rd_ofs, \ + uint32_t rn_ofs, uint32_t rm_ofs, \ + uint32_t oprsz, uint32_t maxsz) \ + { \ + tcg_gen_gvec_3(rd_ofs, rn_ofs, rm_ofs, \ + oprsz, maxsz, &OPARRAY[vece]); \ + } \ + DO_3SAME_NO_SZ_3(INSN, gen_##INSN##_3s) + + +DO_3SAME_GVEC3_NO_SZ_3(VMLA, mla_op) +DO_3SAME_GVEC3_NO_SZ_3(VMLS, mls_op) + +#define DO_3SAME_GVEC3_SHIFT(INSN, OPARRAY) \ + static void gen_##INSN##_3s(unsigned vece, uint32_t rd_ofs, \ + uint32_t rn_ofs, uint32_t rm_ofs, \ + uint32_t oprsz, uint32_t maxsz) \ + { \ + /* Note the operation is vshl vd,vm,vn */ \ + tcg_gen_gvec_3(rd_ofs, rm_ofs, rn_ofs, \ + oprsz, maxsz, &OPARRAY[vece]); \ + } \ + DO_3SAME(INSN, gen_##INSN##_3s) + +DO_3SAME_GVEC3_SHIFT(VSHL_S, sshl_op) +DO_3SAME_GVEC3_SHIFT(VSHL_U, ushl_op) diff --git a/target/arm/translate.c b/target/arm/translate.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate.c +++ b/target/arm/translate.c @@ -XXX,XX +XXX,XX @@ static int disas_neon_data_insn(DisasContext *s, uint32_t insn) } return 1; - case NEON_3R_VMUL: /* VMUL */ - if (u) { - /* Polynomial case allows only P8. */ - if (size != 0) { - return 1; - } - tcg_gen_gvec_3_ool(rd_ofs, rn_ofs, rm_ofs, vec_size, vec_size, - 0, gen_helper_gvec_pmul_b); - } else { - tcg_gen_gvec_mul(size, rd_ofs, rn_ofs, rm_ofs, - vec_size, vec_size); - } - return 0; - - case NEON_3R_VML: /* VMLA, VMLS */ - tcg_gen_gvec_3(rd_ofs, rn_ofs, rm_ofs, vec_size, vec_size, - u ? &mls_op[size] : &mla_op[size]); - return 0; - - case NEON_3R_VSHL: - /* Note the operation is vshl vd,vm,vn */ - tcg_gen_gvec_3(rd_ofs, rm_ofs, rn_ofs, vec_size, vec_size, - u ? &ushl_op[size] : &sshl_op[size]); - return 0; - case NEON_3R_VADD_VSUB: case NEON_3R_LOGIC: case NEON_3R_VMAX: @@ -XXX,XX +XXX,XX @@ static int disas_neon_data_insn(DisasContext *s, uint32_t insn) case NEON_3R_VCGE: case NEON_3R_VQADD: case NEON_3R_VQSUB: + case NEON_3R_VMUL: + case NEON_3R_VML: + case NEON_3R_VSHL: /* Already handled by decodetree */ return 1; } -- 2.20.1 We're going to want at least some of the NeonGen* typedefs for the refactored 32-bit Neon decoder, so move them all to translate.h since it makes more sense to keep them in one group. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200430181003.21682-23-peter.maydell@linaro.org --- target/arm/translate.h | 17 +++++++++++++++++ target/arm/translate-a64.c | 17 ----------------- 2 files changed, 17 insertions(+), 17 deletions(-) diff --git a/target/arm/translate.h b/target/arm/translate.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate.h +++ b/target/arm/translate.h @@ -XXX,XX +XXX,XX @@ typedef void GVecGen3Fn(unsigned, uint32_t, uint32_t, typedef void GVecGen4Fn(unsigned, uint32_t, uint32_t, uint32_t, uint32_t, uint32_t, uint32_t); +/* Function prototype for gen_ functions for calling Neon helpers */ +typedef void NeonGenOneOpEnvFn(TCGv_i32, TCGv_ptr, TCGv_i32); +typedef void NeonGenTwoOpFn(TCGv_i32, TCGv_i32, TCGv_i32); +typedef void NeonGenTwoOpEnvFn(TCGv_i32, TCGv_ptr, TCGv_i32, TCGv_i32); +typedef void NeonGenTwo64OpFn(TCGv_i64, TCGv_i64, TCGv_i64); +typedef void NeonGenTwo64OpEnvFn(TCGv_i64, TCGv_ptr, TCGv_i64, TCGv_i64); +typedef void NeonGenNarrowFn(TCGv_i32, TCGv_i64); +typedef void NeonGenNarrowEnvFn(TCGv_i32, TCGv_ptr, TCGv_i64); +typedef void NeonGenWidenFn(TCGv_i64, TCGv_i32); +typedef void NeonGenTwoSingleOPFn(TCGv_i32, TCGv_i32, TCGv_i32, TCGv_ptr); +typedef void NeonGenTwoDoubleOPFn(TCGv_i64, TCGv_i64, TCGv_i64, TCGv_ptr); +typedef void NeonGenOneOpFn(TCGv_i64, TCGv_i64); +typedef void CryptoTwoOpFn(TCGv_ptr, TCGv_ptr); +typedef void CryptoThreeOpIntFn(TCGv_ptr, TCGv_ptr, TCGv_i32); +typedef void CryptoThreeOpFn(TCGv_ptr, TCGv_ptr, TCGv_ptr); +typedef void AtomicThreeOpFn(TCGv_i64, TCGv_i64, TCGv_i64, TCGArg, MemOp); + #endif /* TARGET_ARM_TRANSLATE_H */ diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-a64.c +++ b/target/arm/translate-a64.c @@ -XXX,XX +XXX,XX @@ typedef struct AArch64DecodeTable { AArch64DecodeFn *disas_fn; } AArch64DecodeTable; -/* Function prototype for gen_ functions for calling Neon helpers */ -typedef void NeonGenOneOpEnvFn(TCGv_i32, TCGv_ptr, TCGv_i32); -typedef void NeonGenTwoOpFn(TCGv_i32, TCGv_i32, TCGv_i32); -typedef void NeonGenTwoOpEnvFn(TCGv_i32, TCGv_ptr, TCGv_i32, TCGv_i32); -typedef void NeonGenTwo64OpFn(TCGv_i64, TCGv_i64, TCGv_i64); -typedef void NeonGenTwo64OpEnvFn(TCGv_i64, TCGv_ptr, TCGv_i64, TCGv_i64); -typedef void NeonGenNarrowFn(TCGv_i32, TCGv_i64); -typedef void NeonGenNarrowEnvFn(TCGv_i32, TCGv_ptr, TCGv_i64); -typedef void NeonGenWidenFn(TCGv_i64, TCGv_i32); -typedef void NeonGenTwoSingleOPFn(TCGv_i32, TCGv_i32, TCGv_i32, TCGv_ptr); -typedef void NeonGenTwoDoubleOPFn(TCGv_i64, TCGv_i64, TCGv_i64, TCGv_ptr); -typedef void NeonGenOneOpFn(TCGv_i64, TCGv_i64); -typedef void CryptoTwoOpFn(TCGv_ptr, TCGv_ptr); -typedef void CryptoThreeOpIntFn(TCGv_ptr, TCGv_ptr, TCGv_i32); -typedef void CryptoThreeOpFn(TCGv_ptr, TCGv_ptr, TCGv_ptr); -typedef void AtomicThreeOpFn(TCGv_i64, TCGv_i64, TCGv_i64, TCGArg, MemOp); - /* initialize TCG globals. */ void a64_translate_init(void) { -- 2.20.1 arm queue: big stuff here is my MVE codegen optimisation, and Alex's Apple Silicon hvf support. -- PMM The following changes since commit 7adb961995a3744f51396502b33ad04a56a317c3: Merge remote-tracking branch 'remotes/dgilbert-gitlab/tags/pull-virtiofs-20210916' into staging (2021-09-19 18:53:29 +0100) are available in the Git repository at: https://git.linaro.org/people/pmaydell/qemu-arm.git tags/pull-target-arm-20210920 for you to fetch changes up to 1dc5a60bfe406bc1122d68cbdefda38d23134b27: target/arm: Optimize MVE 1op-immediate insns (2021-09-20 14:18:01 +0100) ---------------------------------------------------------------- target-arm queue: * Optimize codegen for MVE when predication not active * hvf: Add Apple Silicon support * hw/intc: Set GIC maintenance interrupt level to only 0 or 1 * Fix mishandling of MVE FPSCR.LTPSIZE reset for usermode emulator * elf2dmp: Fix coverity nits ---------------------------------------------------------------- Alexander Graf (7): arm: Move PMC register definitions to internals.h hvf: Add execute to dirty log permission bitmap hvf: Introduce hvf_arch_init() callback hvf: Add Apple Silicon support hvf: arm: Implement PSCI handling arm: Add Hypervisor.framework build target hvf: arm: Add rudimentary PMC support Peter Collingbourne (1): arm/hvf: Add a WFI handler Peter Maydell (18): elf2dmp: Check curl_easy_setopt() return value elf2dmp: Fail cleanly if PDB file specifies zero block_size target/arm: Don't skip M-profile reset entirely in user mode target/arm: Always clear exclusive monitor on reset target/arm: Consolidate ifdef blocks in reset hvf: arm: Implement -cpu host target/arm: Avoid goto_tb if we're trying to exit to the main loop target/arm: Enforce that FPDSCR.LTPSIZE is 4 on inbound migration target/arm: Add TB flag for "MVE insns not predicated" target/arm: Optimize MVE logic ops target/arm: Optimize MVE arithmetic ops target/arm: Optimize MVE VNEG, VABS target/arm: Optimize MVE VDUP target/arm: Optimize MVE VMVN target/arm: Optimize MVE VSHL, VSHR immediate forms target/arm: Optimize MVE VSHLL and VMOVL target/arm: Optimize MVE VSLI and VSRI target/arm: Optimize MVE 1op-immediate insns Shashi Mallela (1): hw/intc: Set GIC maintenance interrupt level to only 0 or 1 meson.build | 8 + include/sysemu/hvf_int.h | 12 +- target/arm/cpu.h | 6 +- target/arm/hvf_arm.h | 18 + target/arm/internals.h | 44 ++ target/arm/kvm_arm.h | 2 - target/arm/translate.h | 2 + accel/hvf/hvf-accel-ops.c | 21 +- contrib/elf2dmp/download.c | 22 +- contrib/elf2dmp/pdb.c | 4 + hw/intc/arm_gicv3_cpuif.c | 5 +- target/arm/cpu.c | 56 +- target/arm/helper.c | 77 ++- target/arm/hvf/hvf.c | 1278 +++++++++++++++++++++++++++++++++++++++++ target/arm/machine.c | 13 + target/arm/translate-m-nocp.c | 8 +- target/arm/translate-mve.c | 310 +++++++--- target/arm/translate-vfp.c | 33 +- target/arm/translate.c | 42 +- target/i386/hvf/hvf.c | 10 + MAINTAINERS | 5 + target/arm/hvf/meson.build | 3 + target/arm/hvf/trace-events | 11 + target/arm/meson.build | 2 + 24 files changed, 1824 insertions(+), 168 deletions(-) create mode 100644 target/arm/hvf_arm.h create mode 100644 target/arm/hvf/hvf.c create mode 100644 target/arm/hvf/meson.build create mode 100644 target/arm/hvf/trace-events Coverity points out that we aren't checking the return value from curl_easy_setopt(). Fixes: Coverity CID 1458895 Inspired-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com> Reviewed-by: Viktor Prutyanov <viktor.prutyanov@phystech.edu> Tested-by: Viktor Prutyanov <viktor.prutyanov@phystech.edu> Message-id: 20210910170656.366592-2-philmd@redhat.com Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- contrib/elf2dmp/download.c | 22 ++++++++++------------ 1 file changed, 10 insertions(+), 12 deletions(-) diff --git a/contrib/elf2dmp/download.c b/contrib/elf2dmp/download.c index XXXXXXX..XXXXXXX 100644 --- a/contrib/elf2dmp/download.c +++ b/contrib/elf2dmp/download.c @@ -XXX,XX +XXX,XX @@ int download_url(const char *name, const char *url) goto out_curl; } - curl_easy_setopt(curl, CURLOPT_URL, url); - curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, NULL); - curl_easy_setopt(curl, CURLOPT_WRITEDATA, file); - curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1); - curl_easy_setopt(curl, CURLOPT_NOPROGRESS, 0); - - if (curl_easy_perform(curl) != CURLE_OK) { - err = 1; - fclose(file); + if (curl_easy_setopt(curl, CURLOPT_URL, url) != CURLE_OK + || curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, NULL) != CURLE_OK + || curl_easy_setopt(curl, CURLOPT_WRITEDATA, file) != CURLE_OK + || curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1) != CURLE_OK + || curl_easy_setopt(curl, CURLOPT_NOPROGRESS, 0) != CURLE_OK + || curl_easy_perform(curl) != CURLE_OK) { unlink(name); - goto out_curl; + fclose(file); + err = 1; + } else { + err = fclose(file); } - err = fclose(file); - out_curl: curl_easy_cleanup(curl); -- 2.20.1 Coverity points out that if the PDB file we're trying to read has a header specifying a block_size of zero then we will end up trying to divide by zero in pdb_ds_read_file(). Check for this and fail cleanly instead. Fixes: Coverity CID 1458869 Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Viktor Prutyanov <viktor.prutyanov@phystech.edu> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com> Tested-by: Viktor Prutyanov <viktor.prutyanov@phystech.edu> Message-id: 20210910170656.366592-3-philmd@redhat.com Message-Id: <20210901143910.17112-3-peter.maydell@linaro.org> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com> --- contrib/elf2dmp/pdb.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/contrib/elf2dmp/pdb.c b/contrib/elf2dmp/pdb.c index XXXXXXX..XXXXXXX 100644 --- a/contrib/elf2dmp/pdb.c +++ b/contrib/elf2dmp/pdb.c @@ -XXX,XX +XXX,XX @@ out_symbols: static int pdb_reader_ds_init(struct pdb_reader *r, PDB_DS_HEADER *hdr) { + if (hdr->block_size == 0) { + return 1; + } + memset(r->file_used, 0, sizeof(r->file_used)); r->ds.header = hdr; r->ds.toc = pdb_ds_read(hdr, (uint32_t *)((uint8_t *)hdr + -- 2.20.1 Currently all of the M-profile specific code in arm_cpu_reset() is inside a !defined(CONFIG_USER_ONLY) ifdef block. This is unintentional: it happened because originally the only M-profile-specific handling was the setup of the initial SP and PC from the vector table, which is system-emulation only. But then we added a lot of other M-profile setup to the same "if (ARM_FEATURE_M)" code block without noticing that it was all inside a not-user-mode ifdef. This has generally been harmless, but with the addition of v8.1M low-overhead-loop support we ran into a problem: the reset of FPSCR.LTPSIZE to 4 was only being done for system emulation mode, so if a user-mode guest tried to execute the LE instruction it would incorrectly take a UsageFault. Adjust the ifdefs so only the really system-emulation specific parts are covered. Because this means we now run some reset code that sets up initial values in the FPCCR and similar FPU related registers, explicitly set up the registers controlling FPU context handling in user-emulation mode so that the FPU works by design and not by chance. Resolves: https://gitlab.com/qemu-project/qemu/-/issues/613 Cc: qemu-stable@nongnu.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20210914120725.24992-2-peter.maydell@linaro.org --- target/arm/cpu.c | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/target/arm/cpu.c b/target/arm/cpu.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpu.c +++ b/target/arm/cpu.c @@ -XXX,XX +XXX,XX @@ static void arm_cpu_reset(DeviceState *dev) env->uncached_cpsr = ARM_CPU_MODE_SVC; } env->daif = PSTATE_D | PSTATE_A | PSTATE_I | PSTATE_F; +#endif if (arm_feature(env, ARM_FEATURE_M)) { +#ifndef CONFIG_USER_ONLY uint32_t initial_msp; /* Loaded from 0x0 */ uint32_t initial_pc; /* Loaded from 0x4 */ uint8_t *rom; uint32_t vecbase; +#endif if (cpu_isar_feature(aa32_lob, cpu)) { /* @@ -XXX,XX +XXX,XX @@ static void arm_cpu_reset(DeviceState *dev) env->v7m.fpccr[M_REG_S] = R_V7M_FPCCR_ASPEN_MASK | R_V7M_FPCCR_LSPEN_MASK | R_V7M_FPCCR_S_MASK; } + +#ifndef CONFIG_USER_ONLY /* Unlike A/R profile, M profile defines the reset LR value */ env->regs[14] = 0xffffffff; @@ -XXX,XX +XXX,XX @@ static void arm_cpu_reset(DeviceState *dev) env->regs[13] = initial_msp & 0xFFFFFFFC; env->regs[15] = initial_pc & ~1; env->thumb = initial_pc & 1; +#else + /* + * For user mode we run non-secure and with access to the FPU. + * The FPU context is active (ie does not need further setup) + * and is owned by non-secure. + */ + env->v7m.secure = false; + env->v7m.nsacr = 0xcff; + env->v7m.cpacr[M_REG_NS] = 0xf0ffff; + env->v7m.fpccr[M_REG_S] &= + ~(R_V7M_FPCCR_LSPEN_MASK | R_V7M_FPCCR_S_MASK); + env->v7m.control[M_REG_S] |= R_V7M_CONTROL_FPCA_MASK; +#endif } +#ifndef CONFIG_USER_ONLY /* AArch32 has a hard highvec setting of 0xFFFF0000. If we are currently * executing as AArch32 then check if highvecs are enabled and * adjust the PC accordingly. -- 2.20.1 There's no particular reason why the exclusive monitor should be only cleared on reset in system emulation mode. It doesn't hurt if it isn't cleared in user mode, but we might as well reduce the amount of code we have that's inside an ifdef. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20210914120725.24992-3-peter.maydell@linaro.org --- target/arm/cpu.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/target/arm/cpu.c b/target/arm/cpu.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpu.c +++ b/target/arm/cpu.c @@ -XXX,XX +XXX,XX @@ static void arm_cpu_reset(DeviceState *dev) env->regs[15] = 0xFFFF0000; } + env->vfp.xregs[ARM_VFP_FPEXC] = 0; +#endif + /* M profile requires that reset clears the exclusive monitor; * A profile does not, but clearing it makes more sense than having it * set with an exclusive access on address zero. */ arm_clear_exclusive(env); - env->vfp.xregs[ARM_VFP_FPEXC] = 0; -#endif - if (arm_feature(env, ARM_FEATURE_PMSA)) { if (cpu->pmsav7_dregion > 0) { if (arm_feature(env, ARM_FEATURE_V8)) { -- 2.20.1 Move an ifndef CONFIG_USER_ONLY code block up in arm_cpu_reset() so it can be merged with another earlier one. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20210914120725.24992-4-peter.maydell@linaro.org --- target/arm/cpu.c | 22 ++++++++++------------ 1 file changed, 10 insertions(+), 12 deletions(-) diff --git a/target/arm/cpu.c b/target/arm/cpu.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpu.c +++ b/target/arm/cpu.c @@ -XXX,XX +XXX,XX @@ static void arm_cpu_reset(DeviceState *dev) env->uncached_cpsr = ARM_CPU_MODE_SVC; } env->daif = PSTATE_D | PSTATE_A | PSTATE_I | PSTATE_F; + + /* AArch32 has a hard highvec setting of 0xFFFF0000. If we are currently + * executing as AArch32 then check if highvecs are enabled and + * adjust the PC accordingly. + */ + if (A32_BANKED_CURRENT_REG_GET(env, sctlr) & SCTLR_V) { + env->regs[15] = 0xFFFF0000; + } + + env->vfp.xregs[ARM_VFP_FPEXC] = 0; #endif if (arm_feature(env, ARM_FEATURE_M)) { @@ -XXX,XX +XXX,XX @@ static void arm_cpu_reset(DeviceState *dev) #endif } -#ifndef CONFIG_USER_ONLY - /* AArch32 has a hard highvec setting of 0xFFFF0000. If we are currently - * executing as AArch32 then check if highvecs are enabled and - * adjust the PC accordingly. - */ - if (A32_BANKED_CURRENT_REG_GET(env, sctlr) & SCTLR_V) { - env->regs[15] = 0xFFFF0000; - } - - env->vfp.xregs[ARM_VFP_FPEXC] = 0; -#endif - /* M profile requires that reset clears the exclusive monitor; * A profile does not, but clearing it makes more sense than having it * set with an exclusive access on address zero. -- 2.20.1 From: Shashi Mallela <shashi.mallela@linaro.org> During sbsa acs level 3 testing, it is seen that the GIC maintenance interrupts are not triggered and the related test cases fail. This is because we were incorrectly passing the value of the MISR register (from maintenance_interrupt_state()) to qemu_set_irq() as the level argument, whereas the device on the other end of this irq line expects a 0/1 value. Fix the logic to pass a 0/1 level indication, rather than a 0/not-0 value. Fixes: c5fc89b36c0 ("hw/intc/arm_gicv3: Implement gicv3_cpuif_virt_update()") Signed-off-by: Shashi Mallela <shashi.mallela@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Message-id: 20210915205809.59068-1-shashi.mallela@linaro.org [PMM: tweaked commit message; collapsed nested if()s into one] Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- hw/intc/arm_gicv3_cpuif.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/hw/intc/arm_gicv3_cpuif.c b/hw/intc/arm_gicv3_cpuif.c index XXXXXXX..XXXXXXX 100644 --- a/hw/intc/arm_gicv3_cpuif.c +++ b/hw/intc/arm_gicv3_cpuif.c @@ -XXX,XX +XXX,XX @@ static void gicv3_cpuif_virt_update(GICv3CPUState *cs) } } - if (cs->ich_hcr_el2 & ICH_HCR_EL2_EN) { - maintlevel = maintenance_interrupt_state(cs); + if ((cs->ich_hcr_el2 & ICH_HCR_EL2_EN) && + maintenance_interrupt_state(cs) != 0) { + maintlevel = 1; } trace_gicv3_cpuif_virt_set_irqs(gicv3_redist_affid(cs), fiqlevel, -- 2.20.1 From: Alexander Graf <agraf@csgraf.de> We will need PMC register definitions in accel specific code later. Move all constant definitions to common arm headers so we can reuse them. Signed-off-by: Alexander Graf <agraf@csgraf.de> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Message-id: 20210916155404.86958-2-agraf@csgraf.de Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/internals.h | 44 ++++++++++++++++++++++++++++++++++++++++++ target/arm/helper.c | 44 ------------------------------------------ 2 files changed, 44 insertions(+), 44 deletions(-) diff --git a/target/arm/internals.h b/target/arm/internals.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/internals.h +++ b/target/arm/internals.h @@ -XXX,XX +XXX,XX @@ enum MVEECIState { /* All other values reserved */ }; +/* Definitions for the PMU registers */ +#define PMCRN_MASK 0xf800 +#define PMCRN_SHIFT 11 +#define PMCRLC 0x40 +#define PMCRDP 0x20 +#define PMCRX 0x10 +#define PMCRD 0x8 +#define PMCRC 0x4 +#define PMCRP 0x2 +#define PMCRE 0x1 +/* + * Mask of PMCR bits writeable by guest (not including WO bits like C, P, + * which can be written as 1 to trigger behaviour but which stay RAZ). + */ +#define PMCR_WRITEABLE_MASK (PMCRLC | PMCRDP | PMCRX | PMCRD | PMCRE) + +#define PMXEVTYPER_P 0x80000000 +#define PMXEVTYPER_U 0x40000000 +#define PMXEVTYPER_NSK 0x20000000 +#define PMXEVTYPER_NSU 0x10000000 +#define PMXEVTYPER_NSH 0x08000000 +#define PMXEVTYPER_M 0x04000000 +#define PMXEVTYPER_MT 0x02000000 +#define PMXEVTYPER_EVTCOUNT 0x0000ffff +#define PMXEVTYPER_MASK (PMXEVTYPER_P | PMXEVTYPER_U | PMXEVTYPER_NSK | \ + PMXEVTYPER_NSU | PMXEVTYPER_NSH | \ + PMXEVTYPER_M | PMXEVTYPER_MT | \ + PMXEVTYPER_EVTCOUNT) + +#define PMCCFILTR 0xf8000000 +#define PMCCFILTR_M PMXEVTYPER_M +#define PMCCFILTR_EL0 (PMCCFILTR | PMCCFILTR_M) + +static inline uint32_t pmu_num_counters(CPUARMState *env) +{ + return (env->cp15.c9_pmcr & PMCRN_MASK) >> PMCRN_SHIFT; +} + +/* Bits allowed to be set/cleared for PMCNTEN* and PMINTEN* */ +static inline uint64_t pmu_counter_mask(CPUARMState *env) +{ + return (1 << 31) | ((1 << pmu_num_counters(env)) - 1); +} + #endif diff --git a/target/arm/helper.c b/target/arm/helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -XXX,XX +XXX,XX @@ static const ARMCPRegInfo v6_cp_reginfo[] = { REGINFO_SENTINEL }; -/* Definitions for the PMU registers */ -#define PMCRN_MASK 0xf800 -#define PMCRN_SHIFT 11 -#define PMCRLC 0x40 -#define PMCRDP 0x20 -#define PMCRX 0x10 -#define PMCRD 0x8 -#define PMCRC 0x4 -#define PMCRP 0x2 -#define PMCRE 0x1 -/* - * Mask of PMCR bits writeable by guest (not including WO bits like C, P, - * which can be written as 1 to trigger behaviour but which stay RAZ). - */ -#define PMCR_WRITEABLE_MASK (PMCRLC | PMCRDP | PMCRX | PMCRD | PMCRE) - -#define PMXEVTYPER_P 0x80000000 -#define PMXEVTYPER_U 0x40000000 -#define PMXEVTYPER_NSK 0x20000000 -#define PMXEVTYPER_NSU 0x10000000 -#define PMXEVTYPER_NSH 0x08000000 -#define PMXEVTYPER_M 0x04000000 -#define PMXEVTYPER_MT 0x02000000 -#define PMXEVTYPER_EVTCOUNT 0x0000ffff -#define PMXEVTYPER_MASK (PMXEVTYPER_P | PMXEVTYPER_U | PMXEVTYPER_NSK | \ - PMXEVTYPER_NSU | PMXEVTYPER_NSH | \ - PMXEVTYPER_M | PMXEVTYPER_MT | \ - PMXEVTYPER_EVTCOUNT) - -#define PMCCFILTR 0xf8000000 -#define PMCCFILTR_M PMXEVTYPER_M -#define PMCCFILTR_EL0 (PMCCFILTR | PMCCFILTR_M) - -static inline uint32_t pmu_num_counters(CPUARMState *env) -{ - return (env->cp15.c9_pmcr & PMCRN_MASK) >> PMCRN_SHIFT; -} - -/* Bits allowed to be set/cleared for PMCNTEN* and PMINTEN* */ -static inline uint64_t pmu_counter_mask(CPUARMState *env) -{ - return (1 << 31) | ((1 << pmu_num_counters(env)) - 1); -} - typedef struct pm_event { uint16_t number; /* PMEVTYPER.evtCount is 16 bits wide */ /* If the event is supported on this CPU (used to generate PMCEID[01]) */ -- 2.20.1 From: Alexander Graf <agraf@csgraf.de> Hvf's permission bitmap during and after dirty logging does not include the HV_MEMORY_EXEC permission. At least on Apple Silicon, this leads to instruction faults once dirty logging was enabled. Add the bit to make it work properly. Signed-off-by: Alexander Graf <agraf@csgraf.de> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Message-id: 20210916155404.86958-3-agraf@csgraf.de Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- accel/hvf/hvf-accel-ops.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/accel/hvf/hvf-accel-ops.c b/accel/hvf/hvf-accel-ops.c index XXXXXXX..XXXXXXX 100644 --- a/accel/hvf/hvf-accel-ops.c +++ b/accel/hvf/hvf-accel-ops.c @@ -XXX,XX +XXX,XX @@ static void hvf_set_dirty_tracking(MemoryRegionSection *section, bool on) if (on) { slot->flags |= HVF_SLOT_LOG; hv_vm_protect((uintptr_t)slot->start, (size_t)slot->size, - HV_MEMORY_READ); + HV_MEMORY_READ | HV_MEMORY_EXEC); /* stop tracking region*/ } else { slot->flags &= ~HVF_SLOT_LOG; hv_vm_protect((uintptr_t)slot->start, (size_t)slot->size, - HV_MEMORY_READ | HV_MEMORY_WRITE); + HV_MEMORY_READ | HV_MEMORY_WRITE | HV_MEMORY_EXEC); } } -- 2.20.1 From: Alexander Graf <agraf@csgraf.de> We will need to install a migration helper for the ARM hvf backend. Let's introduce an arch callback for the overall hvf init chain to do so. Signed-off-by: Alexander Graf <agraf@csgraf.de> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Message-id: 20210916155404.86958-4-agraf@csgraf.de Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- include/sysemu/hvf_int.h | 1 + accel/hvf/hvf-accel-ops.c | 3 ++- target/i386/hvf/hvf.c | 5 +++++ 3 files changed, 8 insertions(+), 1 deletion(-) diff --git a/include/sysemu/hvf_int.h b/include/sysemu/hvf_int.h index XXXXXXX..XXXXXXX 100644 --- a/include/sysemu/hvf_int.h +++ b/include/sysemu/hvf_int.h @@ -XXX,XX +XXX,XX @@ struct hvf_vcpu_state { }; void assert_hvf_ok(hv_return_t ret); +int hvf_arch_init(void); int hvf_arch_init_vcpu(CPUState *cpu); void hvf_arch_vcpu_destroy(CPUState *cpu); int hvf_vcpu_exec(CPUState *); diff --git a/accel/hvf/hvf-accel-ops.c b/accel/hvf/hvf-accel-ops.c index XXXXXXX..XXXXXXX 100644 --- a/accel/hvf/hvf-accel-ops.c +++ b/accel/hvf/hvf-accel-ops.c @@ -XXX,XX +XXX,XX @@ static int hvf_accel_init(MachineState *ms) hvf_state = s; memory_listener_register(&hvf_memory_listener, &address_space_memory); - return 0; + + return hvf_arch_init(); } static void hvf_accel_class_init(ObjectClass *oc, void *data) diff --git a/target/i386/hvf/hvf.c b/target/i386/hvf/hvf.c index XXXXXXX..XXXXXXX 100644 --- a/target/i386/hvf/hvf.c +++ b/target/i386/hvf/hvf.c @@ -XXX,XX +XXX,XX @@ static inline bool apic_bus_freq_is_known(CPUX86State *env) return env->apic_bus_freq != 0; } +int hvf_arch_init(void) +{ + return 0; +} + int hvf_arch_init_vcpu(CPUState *cpu) { X86CPU *x86cpu = X86_CPU(cpu); -- 2.20.1 From: Alexander Graf <agraf@csgraf.de> With Apple Silicon available to the masses, it's a good time to add support for driving its virtualization extensions from QEMU. This patch adds all necessary architecture specific code to get basic VMs working, including save/restore. Known limitations: - WFI handling is missing (follows in later patch) - No watchpoint/breakpoint support Signed-off-by: Alexander Graf <agraf@csgraf.de> Reviewed-by: Roman Bolshakov <r.bolshakov@yadro.com> Reviewed-by: Sergio Lopez <slp@redhat.com> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Message-id: 20210916155404.86958-5-agraf@csgraf.de Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- meson.build | 1 + include/sysemu/hvf_int.h | 10 +- accel/hvf/hvf-accel-ops.c | 9 + target/arm/hvf/hvf.c | 794 ++++++++++++++++++++++++++++++++++++ target/i386/hvf/hvf.c | 5 + MAINTAINERS | 5 + target/arm/hvf/trace-events | 10 + 7 files changed, 833 insertions(+), 1 deletion(-) create mode 100644 target/arm/hvf/hvf.c create mode 100644 target/arm/hvf/trace-events diff --git a/meson.build b/meson.build index XXXXXXX..XXXXXXX 100644 --- a/meson.build +++ b/meson.build @@ -XXX,XX +XXX,XX @@ if have_system or have_user 'accel/tcg', 'hw/core', 'target/arm', + 'target/arm/hvf', 'target/hppa', 'target/i386', 'target/i386/kvm', diff --git a/include/sysemu/hvf_int.h b/include/sysemu/hvf_int.h index XXXXXXX..XXXXXXX 100644 --- a/include/sysemu/hvf_int.h +++ b/include/sysemu/hvf_int.h @@ -XXX,XX +XXX,XX @@ #ifndef HVF_INT_H #define HVF_INT_H +#ifdef __aarch64__ +#include <Hypervisor/Hypervisor.h> +#else #include <Hypervisor/hv.h> +#endif /* hvf_slot flags */ #define HVF_SLOT_LOG (1 << 0) @@ -XXX,XX +XXX,XX @@ struct HVFState { int num_slots; hvf_vcpu_caps *hvf_caps; + uint64_t vtimer_offset; }; extern HVFState *hvf_state; struct hvf_vcpu_state { - int fd; + uint64_t fd; + void *exit; + bool vtimer_masked; }; void assert_hvf_ok(hv_return_t ret); @@ -XXX,XX +XXX,XX @@ int hvf_vcpu_exec(CPUState *); hvf_slot *hvf_find_overlap_slot(uint64_t, uint64_t); int hvf_put_registers(CPUState *); int hvf_get_registers(CPUState *); +void hvf_kick_vcpu_thread(CPUState *cpu); #endif diff --git a/accel/hvf/hvf-accel-ops.c b/accel/hvf/hvf-accel-ops.c index XXXXXXX..XXXXXXX 100644 --- a/accel/hvf/hvf-accel-ops.c +++ b/accel/hvf/hvf-accel-ops.c @@ -XXX,XX +XXX,XX @@ HVFState *hvf_state; +#ifdef __aarch64__ +#define HV_VM_DEFAULT NULL +#endif + /* Memory slots */ hvf_slot *hvf_find_overlap_slot(uint64_t start, uint64_t size) @@ -XXX,XX +XXX,XX @@ static int hvf_init_vcpu(CPUState *cpu) pthread_sigmask(SIG_BLOCK, NULL, &set); sigdelset(&set, SIG_IPI); +#ifdef __aarch64__ + r = hv_vcpu_create(&cpu->hvf->fd, (hv_vcpu_exit_t **)&cpu->hvf->exit, NULL); +#else r = hv_vcpu_create((hv_vcpuid_t *)&cpu->hvf->fd, HV_VCPU_DEFAULT); +#endif cpu->vcpu_dirty = 1; assert_hvf_ok(r); @@ -XXX,XX +XXX,XX @@ static void hvf_accel_ops_class_init(ObjectClass *oc, void *data) AccelOpsClass *ops = ACCEL_OPS_CLASS(oc); ops->create_vcpu_thread = hvf_start_vcpu_thread; + ops->kick_vcpu_thread = hvf_kick_vcpu_thread; ops->synchronize_post_reset = hvf_cpu_synchronize_post_reset; ops->synchronize_post_init = hvf_cpu_synchronize_post_init; diff --git a/target/arm/hvf/hvf.c b/target/arm/hvf/hvf.c new file mode 100644 index XXXXXXX..XXXXXXX --- /dev/null +++ b/target/arm/hvf/hvf.c @@ -XXX,XX +XXX,XX @@ +/* + * QEMU Hypervisor.framework support for Apple Silicon + + * Copyright 2020 Alexander Graf <agraf@csgraf.de> + * + * This work is licensed under the terms of the GNU GPL, version 2 or later. + * See the COPYING file in the top-level directory. + * + */ + +#include "qemu/osdep.h" +#include "qemu-common.h" +#include "qemu/error-report.h" + +#include "sysemu/runstate.h" +#include "sysemu/hvf.h" +#include "sysemu/hvf_int.h" +#include "sysemu/hw_accel.h" + +#include <mach/mach_time.h> + +#include "exec/address-spaces.h" +#include "hw/irq.h" +#include "qemu/main-loop.h" +#include "sysemu/cpus.h" +#include "target/arm/cpu.h" +#include "target/arm/internals.h" +#include "trace/trace-target_arm_hvf.h" +#include "migration/vmstate.h" + +#define HVF_SYSREG(crn, crm, op0, op1, op2) \ + ENCODE_AA64_CP_REG(CP_REG_ARM64_SYSREG_CP, crn, crm, op0, op1, op2) +#define PL1_WRITE_MASK 0x4 + +#define SYSREG(op0, op1, crn, crm, op2) \ + ((op0 << 20) | (op2 << 17) | (op1 << 14) | (crn << 10) | (crm << 1)) +#define SYSREG_MASK SYSREG(0x3, 0x7, 0xf, 0xf, 0x7) +#define SYSREG_OSLAR_EL1 SYSREG(2, 0, 1, 0, 4) +#define SYSREG_OSLSR_EL1 SYSREG(2, 0, 1, 1, 4) +#define SYSREG_OSDLR_EL1 SYSREG(2, 0, 1, 3, 4) +#define SYSREG_CNTPCT_EL0 SYSREG(3, 3, 14, 0, 1) + +#define WFX_IS_WFE (1 << 0) + +#define TMR_CTL_ENABLE (1 << 0) +#define TMR_CTL_IMASK (1 << 1) +#define TMR_CTL_ISTATUS (1 << 2) + +typedef struct HVFVTimer { + /* Vtimer value during migration and paused state */ + uint64_t vtimer_val; +} HVFVTimer; + +static HVFVTimer vtimer; + +struct hvf_reg_match { + int reg; + uint64_t offset; +}; + +static const struct hvf_reg_match hvf_reg_match[] = { + { HV_REG_X0, offsetof(CPUARMState, xregs[0]) }, + { HV_REG_X1, offsetof(CPUARMState, xregs[1]) }, + { HV_REG_X2, offsetof(CPUARMState, xregs[2]) }, + { HV_REG_X3, offsetof(CPUARMState, xregs[3]) }, + { HV_REG_X4, offsetof(CPUARMState, xregs[4]) }, + { HV_REG_X5, offsetof(CPUARMState, xregs[5]) }, + { HV_REG_X6, offsetof(CPUARMState, xregs[6]) }, + { HV_REG_X7, offsetof(CPUARMState, xregs[7]) }, + { HV_REG_X8, offsetof(CPUARMState, xregs[8]) }, + { HV_REG_X9, offsetof(CPUARMState, xregs[9]) }, + { HV_REG_X10, offsetof(CPUARMState, xregs[10]) }, + { HV_REG_X11, offsetof(CPUARMState, xregs[11]) }, + { HV_REG_X12, offsetof(CPUARMState, xregs[12]) }, + { HV_REG_X13, offsetof(CPUARMState, xregs[13]) }, + { HV_REG_X14, offsetof(CPUARMState, xregs[14]) }, + { HV_REG_X15, offsetof(CPUARMState, xregs[15]) }, + { HV_REG_X16, offsetof(CPUARMState, xregs[16]) }, + { HV_REG_X17, offsetof(CPUARMState, xregs[17]) }, + { HV_REG_X18, offsetof(CPUARMState, xregs[18]) }, + { HV_REG_X19, offsetof(CPUARMState, xregs[19]) }, + { HV_REG_X20, offsetof(CPUARMState, xregs[20]) }, + { HV_REG_X21, offsetof(CPUARMState, xregs[21]) }, + { HV_REG_X22, offsetof(CPUARMState, xregs[22]) }, + { HV_REG_X23, offsetof(CPUARMState, xregs[23]) }, + { HV_REG_X24, offsetof(CPUARMState, xregs[24]) }, + { HV_REG_X25, offsetof(CPUARMState, xregs[25]) }, + { HV_REG_X26, offsetof(CPUARMState, xregs[26]) }, + { HV_REG_X27, offsetof(CPUARMState, xregs[27]) }, + { HV_REG_X28, offsetof(CPUARMState, xregs[28]) }, + { HV_REG_X29, offsetof(CPUARMState, xregs[29]) }, + { HV_REG_X30, offsetof(CPUARMState, xregs[30]) }, + { HV_REG_PC, offsetof(CPUARMState, pc) }, +}; + +static const struct hvf_reg_match hvf_fpreg_match[] = { + { HV_SIMD_FP_REG_Q0, offsetof(CPUARMState, vfp.zregs[0]) }, + { HV_SIMD_FP_REG_Q1, offsetof(CPUARMState, vfp.zregs[1]) }, + { HV_SIMD_FP_REG_Q2, offsetof(CPUARMState, vfp.zregs[2]) }, + { HV_SIMD_FP_REG_Q3, offsetof(CPUARMState, vfp.zregs[3]) }, + { HV_SIMD_FP_REG_Q4, offsetof(CPUARMState, vfp.zregs[4]) }, + { HV_SIMD_FP_REG_Q5, offsetof(CPUARMState, vfp.zregs[5]) }, + { HV_SIMD_FP_REG_Q6, offsetof(CPUARMState, vfp.zregs[6]) }, + { HV_SIMD_FP_REG_Q7, offsetof(CPUARMState, vfp.zregs[7]) }, + { HV_SIMD_FP_REG_Q8, offsetof(CPUARMState, vfp.zregs[8]) }, + { HV_SIMD_FP_REG_Q9, offsetof(CPUARMState, vfp.zregs[9]) }, + { HV_SIMD_FP_REG_Q10, offsetof(CPUARMState, vfp.zregs[10]) }, + { HV_SIMD_FP_REG_Q11, offsetof(CPUARMState, vfp.zregs[11]) }, + { HV_SIMD_FP_REG_Q12, offsetof(CPUARMState, vfp.zregs[12]) }, + { HV_SIMD_FP_REG_Q13, offsetof(CPUARMState, vfp.zregs[13]) }, + { HV_SIMD_FP_REG_Q14, offsetof(CPUARMState, vfp.zregs[14]) }, + { HV_SIMD_FP_REG_Q15, offsetof(CPUARMState, vfp.zregs[15]) }, + { HV_SIMD_FP_REG_Q16, offsetof(CPUARMState, vfp.zregs[16]) }, + { HV_SIMD_FP_REG_Q17, offsetof(CPUARMState, vfp.zregs[17]) }, + { HV_SIMD_FP_REG_Q18, offsetof(CPUARMState, vfp.zregs[18]) }, + { HV_SIMD_FP_REG_Q19, offsetof(CPUARMState, vfp.zregs[19]) }, + { HV_SIMD_FP_REG_Q20, offsetof(CPUARMState, vfp.zregs[20]) }, + { HV_SIMD_FP_REG_Q21, offsetof(CPUARMState, vfp.zregs[21]) }, + { HV_SIMD_FP_REG_Q22, offsetof(CPUARMState, vfp.zregs[22]) }, + { HV_SIMD_FP_REG_Q23, offsetof(CPUARMState, vfp.zregs[23]) }, + { HV_SIMD_FP_REG_Q24, offsetof(CPUARMState, vfp.zregs[24]) }, + { HV_SIMD_FP_REG_Q25, offsetof(CPUARMState, vfp.zregs[25]) }, + { HV_SIMD_FP_REG_Q26, offsetof(CPUARMState, vfp.zregs[26]) }, + { HV_SIMD_FP_REG_Q27, offsetof(CPUARMState, vfp.zregs[27]) }, + { HV_SIMD_FP_REG_Q28, offsetof(CPUARMState, vfp.zregs[28]) }, + { HV_SIMD_FP_REG_Q29, offsetof(CPUARMState, vfp.zregs[29]) }, + { HV_SIMD_FP_REG_Q30, offsetof(CPUARMState, vfp.zregs[30]) }, + { HV_SIMD_FP_REG_Q31, offsetof(CPUARMState, vfp.zregs[31]) }, +}; + +struct hvf_sreg_match { + int reg; + uint32_t key; + uint32_t cp_idx; +}; + +static struct hvf_sreg_match hvf_sreg_match[] = { + { HV_SYS_REG_DBGBVR0_EL1, HVF_SYSREG(0, 0, 14, 0, 4) }, + { HV_SYS_REG_DBGBCR0_EL1, HVF_SYSREG(0, 0, 14, 0, 5) }, + { HV_SYS_REG_DBGWVR0_EL1, HVF_SYSREG(0, 0, 14, 0, 6) }, + { HV_SYS_REG_DBGWCR0_EL1, HVF_SYSREG(0, 0, 14, 0, 7) }, + + { HV_SYS_REG_DBGBVR1_EL1, HVF_SYSREG(0, 1, 14, 0, 4) }, + { HV_SYS_REG_DBGBCR1_EL1, HVF_SYSREG(0, 1, 14, 0, 5) }, + { HV_SYS_REG_DBGWVR1_EL1, HVF_SYSREG(0, 1, 14, 0, 6) }, + { HV_SYS_REG_DBGWCR1_EL1, HVF_SYSREG(0, 1, 14, 0, 7) }, + + { HV_SYS_REG_DBGBVR2_EL1, HVF_SYSREG(0, 2, 14, 0, 4) }, + { HV_SYS_REG_DBGBCR2_EL1, HVF_SYSREG(0, 2, 14, 0, 5) }, + { HV_SYS_REG_DBGWVR2_EL1, HVF_SYSREG(0, 2, 14, 0, 6) }, + { HV_SYS_REG_DBGWCR2_EL1, HVF_SYSREG(0, 2, 14, 0, 7) }, + + { HV_SYS_REG_DBGBVR3_EL1, HVF_SYSREG(0, 3, 14, 0, 4) }, + { HV_SYS_REG_DBGBCR3_EL1, HVF_SYSREG(0, 3, 14, 0, 5) }, + { HV_SYS_REG_DBGWVR3_EL1, HVF_SYSREG(0, 3, 14, 0, 6) }, + { HV_SYS_REG_DBGWCR3_EL1, HVF_SYSREG(0, 3, 14, 0, 7) }, + + { HV_SYS_REG_DBGBVR4_EL1, HVF_SYSREG(0, 4, 14, 0, 4) }, + { HV_SYS_REG_DBGBCR4_EL1, HVF_SYSREG(0, 4, 14, 0, 5) }, + { HV_SYS_REG_DBGWVR4_EL1, HVF_SYSREG(0, 4, 14, 0, 6) }, + { HV_SYS_REG_DBGWCR4_EL1, HVF_SYSREG(0, 4, 14, 0, 7) }, + + { HV_SYS_REG_DBGBVR5_EL1, HVF_SYSREG(0, 5, 14, 0, 4) }, + { HV_SYS_REG_DBGBCR5_EL1, HVF_SYSREG(0, 5, 14, 0, 5) }, + { HV_SYS_REG_DBGWVR5_EL1, HVF_SYSREG(0, 5, 14, 0, 6) }, + { HV_SYS_REG_DBGWCR5_EL1, HVF_SYSREG(0, 5, 14, 0, 7) }, + + { HV_SYS_REG_DBGBVR6_EL1, HVF_SYSREG(0, 6, 14, 0, 4) }, + { HV_SYS_REG_DBGBCR6_EL1, HVF_SYSREG(0, 6, 14, 0, 5) }, + { HV_SYS_REG_DBGWVR6_EL1, HVF_SYSREG(0, 6, 14, 0, 6) }, + { HV_SYS_REG_DBGWCR6_EL1, HVF_SYSREG(0, 6, 14, 0, 7) }, + + { HV_SYS_REG_DBGBVR7_EL1, HVF_SYSREG(0, 7, 14, 0, 4) }, + { HV_SYS_REG_DBGBCR7_EL1, HVF_SYSREG(0, 7, 14, 0, 5) }, + { HV_SYS_REG_DBGWVR7_EL1, HVF_SYSREG(0, 7, 14, 0, 6) }, + { HV_SYS_REG_DBGWCR7_EL1, HVF_SYSREG(0, 7, 14, 0, 7) }, + + { HV_SYS_REG_DBGBVR8_EL1, HVF_SYSREG(0, 8, 14, 0, 4) }, + { HV_SYS_REG_DBGBCR8_EL1, HVF_SYSREG(0, 8, 14, 0, 5) }, + { HV_SYS_REG_DBGWVR8_EL1, HVF_SYSREG(0, 8, 14, 0, 6) }, + { HV_SYS_REG_DBGWCR8_EL1, HVF_SYSREG(0, 8, 14, 0, 7) }, + + { HV_SYS_REG_DBGBVR9_EL1, HVF_SYSREG(0, 9, 14, 0, 4) }, + { HV_SYS_REG_DBGBCR9_EL1, HVF_SYSREG(0, 9, 14, 0, 5) }, + { HV_SYS_REG_DBGWVR9_EL1, HVF_SYSREG(0, 9, 14, 0, 6) }, + { HV_SYS_REG_DBGWCR9_EL1, HVF_SYSREG(0, 9, 14, 0, 7) }, + + { HV_SYS_REG_DBGBVR10_EL1, HVF_SYSREG(0, 10, 14, 0, 4) }, + { HV_SYS_REG_DBGBCR10_EL1, HVF_SYSREG(0, 10, 14, 0, 5) }, + { HV_SYS_REG_DBGWVR10_EL1, HVF_SYSREG(0, 10, 14, 0, 6) }, + { HV_SYS_REG_DBGWCR10_EL1, HVF_SYSREG(0, 10, 14, 0, 7) }, + + { HV_SYS_REG_DBGBVR11_EL1, HVF_SYSREG(0, 11, 14, 0, 4) }, + { HV_SYS_REG_DBGBCR11_EL1, HVF_SYSREG(0, 11, 14, 0, 5) }, + { HV_SYS_REG_DBGWVR11_EL1, HVF_SYSREG(0, 11, 14, 0, 6) }, + { HV_SYS_REG_DBGWCR11_EL1, HVF_SYSREG(0, 11, 14, 0, 7) }, + + { HV_SYS_REG_DBGBVR12_EL1, HVF_SYSREG(0, 12, 14, 0, 4) }, + { HV_SYS_REG_DBGBCR12_EL1, HVF_SYSREG(0, 12, 14, 0, 5) }, + { HV_SYS_REG_DBGWVR12_EL1, HVF_SYSREG(0, 12, 14, 0, 6) }, + { HV_SYS_REG_DBGWCR12_EL1, HVF_SYSREG(0, 12, 14, 0, 7) }, + + { HV_SYS_REG_DBGBVR13_EL1, HVF_SYSREG(0, 13, 14, 0, 4) }, + { HV_SYS_REG_DBGBCR13_EL1, HVF_SYSREG(0, 13, 14, 0, 5) }, + { HV_SYS_REG_DBGWVR13_EL1, HVF_SYSREG(0, 13, 14, 0, 6) }, + { HV_SYS_REG_DBGWCR13_EL1, HVF_SYSREG(0, 13, 14, 0, 7) }, + + { HV_SYS_REG_DBGBVR14_EL1, HVF_SYSREG(0, 14, 14, 0, 4) }, + { HV_SYS_REG_DBGBCR14_EL1, HVF_SYSREG(0, 14, 14, 0, 5) }, + { HV_SYS_REG_DBGWVR14_EL1, HVF_SYSREG(0, 14, 14, 0, 6) }, + { HV_SYS_REG_DBGWCR14_EL1, HVF_SYSREG(0, 14, 14, 0, 7) }, + + { HV_SYS_REG_DBGBVR15_EL1, HVF_SYSREG(0, 15, 14, 0, 4) }, + { HV_SYS_REG_DBGBCR15_EL1, HVF_SYSREG(0, 15, 14, 0, 5) }, + { HV_SYS_REG_DBGWVR15_EL1, HVF_SYSREG(0, 15, 14, 0, 6) }, + { HV_SYS_REG_DBGWCR15_EL1, HVF_SYSREG(0, 15, 14, 0, 7) }, + +#ifdef SYNC_NO_RAW_REGS + /* + * The registers below are manually synced on init because they are + * marked as NO_RAW. We still list them to make number space sync easier. + */ + { HV_SYS_REG_MDCCINT_EL1, HVF_SYSREG(0, 2, 2, 0, 0) }, + { HV_SYS_REG_MIDR_EL1, HVF_SYSREG(0, 0, 3, 0, 0) }, + { HV_SYS_REG_MPIDR_EL1, HVF_SYSREG(0, 0, 3, 0, 5) }, + { HV_SYS_REG_ID_AA64PFR0_EL1, HVF_SYSREG(0, 4, 3, 0, 0) }, +#endif + { HV_SYS_REG_ID_AA64PFR1_EL1, HVF_SYSREG(0, 4, 3, 0, 2) }, + { HV_SYS_REG_ID_AA64DFR0_EL1, HVF_SYSREG(0, 5, 3, 0, 0) }, + { HV_SYS_REG_ID_AA64DFR1_EL1, HVF_SYSREG(0, 5, 3, 0, 1) }, + { HV_SYS_REG_ID_AA64ISAR0_EL1, HVF_SYSREG(0, 6, 3, 0, 0) }, + { HV_SYS_REG_ID_AA64ISAR1_EL1, HVF_SYSREG(0, 6, 3, 0, 1) }, +#ifdef SYNC_NO_MMFR0 + /* We keep the hardware MMFR0 around. HW limits are there anyway */ + { HV_SYS_REG_ID_AA64MMFR0_EL1, HVF_SYSREG(0, 7, 3, 0, 0) }, +#endif + { HV_SYS_REG_ID_AA64MMFR1_EL1, HVF_SYSREG(0, 7, 3, 0, 1) }, + { HV_SYS_REG_ID_AA64MMFR2_EL1, HVF_SYSREG(0, 7, 3, 0, 2) }, + + { HV_SYS_REG_MDSCR_EL1, HVF_SYSREG(0, 2, 2, 0, 2) }, + { HV_SYS_REG_SCTLR_EL1, HVF_SYSREG(1, 0, 3, 0, 0) }, + { HV_SYS_REG_CPACR_EL1, HVF_SYSREG(1, 0, 3, 0, 2) }, + { HV_SYS_REG_TTBR0_EL1, HVF_SYSREG(2, 0, 3, 0, 0) }, + { HV_SYS_REG_TTBR1_EL1, HVF_SYSREG(2, 0, 3, 0, 1) }, + { HV_SYS_REG_TCR_EL1, HVF_SYSREG(2, 0, 3, 0, 2) }, + + { HV_SYS_REG_APIAKEYLO_EL1, HVF_SYSREG(2, 1, 3, 0, 0) }, + { HV_SYS_REG_APIAKEYHI_EL1, HVF_SYSREG(2, 1, 3, 0, 1) }, + { HV_SYS_REG_APIBKEYLO_EL1, HVF_SYSREG(2, 1, 3, 0, 2) }, + { HV_SYS_REG_APIBKEYHI_EL1, HVF_SYSREG(2, 1, 3, 0, 3) }, + { HV_SYS_REG_APDAKEYLO_EL1, HVF_SYSREG(2, 2, 3, 0, 0) }, + { HV_SYS_REG_APDAKEYHI_EL1, HVF_SYSREG(2, 2, 3, 0, 1) }, + { HV_SYS_REG_APDBKEYLO_EL1, HVF_SYSREG(2, 2, 3, 0, 2) }, + { HV_SYS_REG_APDBKEYHI_EL1, HVF_SYSREG(2, 2, 3, 0, 3) }, + { HV_SYS_REG_APGAKEYLO_EL1, HVF_SYSREG(2, 3, 3, 0, 0) }, + { HV_SYS_REG_APGAKEYHI_EL1, HVF_SYSREG(2, 3, 3, 0, 1) }, + + { HV_SYS_REG_SPSR_EL1, HVF_SYSREG(4, 0, 3, 0, 0) }, + { HV_SYS_REG_ELR_EL1, HVF_SYSREG(4, 0, 3, 0, 1) }, + { HV_SYS_REG_SP_EL0, HVF_SYSREG(4, 1, 3, 0, 0) }, + { HV_SYS_REG_AFSR0_EL1, HVF_SYSREG(5, 1, 3, 0, 0) }, + { HV_SYS_REG_AFSR1_EL1, HVF_SYSREG(5, 1, 3, 0, 1) }, + { HV_SYS_REG_ESR_EL1, HVF_SYSREG(5, 2, 3, 0, 0) }, + { HV_SYS_REG_FAR_EL1, HVF_SYSREG(6, 0, 3, 0, 0) }, + { HV_SYS_REG_PAR_EL1, HVF_SYSREG(7, 4, 3, 0, 0) }, + { HV_SYS_REG_MAIR_EL1, HVF_SYSREG(10, 2, 3, 0, 0) }, + { HV_SYS_REG_AMAIR_EL1, HVF_SYSREG(10, 3, 3, 0, 0) }, + { HV_SYS_REG_VBAR_EL1, HVF_SYSREG(12, 0, 3, 0, 0) }, + { HV_SYS_REG_CONTEXTIDR_EL1, HVF_SYSREG(13, 0, 3, 0, 1) }, + { HV_SYS_REG_TPIDR_EL1, HVF_SYSREG(13, 0, 3, 0, 4) }, + { HV_SYS_REG_CNTKCTL_EL1, HVF_SYSREG(14, 1, 3, 0, 0) }, + { HV_SYS_REG_CSSELR_EL1, HVF_SYSREG(0, 0, 3, 2, 0) }, + { HV_SYS_REG_TPIDR_EL0, HVF_SYSREG(13, 0, 3, 3, 2) }, + { HV_SYS_REG_TPIDRRO_EL0, HVF_SYSREG(13, 0, 3, 3, 3) }, + { HV_SYS_REG_CNTV_CTL_EL0, HVF_SYSREG(14, 3, 3, 3, 1) }, + { HV_SYS_REG_CNTV_CVAL_EL0, HVF_SYSREG(14, 3, 3, 3, 2) }, + { HV_SYS_REG_SP_EL1, HVF_SYSREG(4, 1, 3, 4, 0) }, +}; + +int hvf_get_registers(CPUState *cpu) +{ + ARMCPU *arm_cpu = ARM_CPU(cpu); + CPUARMState *env = &arm_cpu->env; + hv_return_t ret; + uint64_t val; + hv_simd_fp_uchar16_t fpval; + int i; + + for (i = 0; i < ARRAY_SIZE(hvf_reg_match); i++) { + ret = hv_vcpu_get_reg(cpu->hvf->fd, hvf_reg_match[i].reg, &val); + *(uint64_t *)((void *)env + hvf_reg_match[i].offset) = val; + assert_hvf_ok(ret); + } + + for (i = 0; i < ARRAY_SIZE(hvf_fpreg_match); i++) { + ret = hv_vcpu_get_simd_fp_reg(cpu->hvf->fd, hvf_fpreg_match[i].reg, + &fpval); + memcpy((void *)env + hvf_fpreg_match[i].offset, &fpval, sizeof(fpval)); + assert_hvf_ok(ret); + } + + val = 0; + ret = hv_vcpu_get_reg(cpu->hvf->fd, HV_REG_FPCR, &val); + assert_hvf_ok(ret); + vfp_set_fpcr(env, val); + + val = 0; + ret = hv_vcpu_get_reg(cpu->hvf->fd, HV_REG_FPSR, &val); + assert_hvf_ok(ret); + vfp_set_fpsr(env, val); + + ret = hv_vcpu_get_reg(cpu->hvf->fd, HV_REG_CPSR, &val); + assert_hvf_ok(ret); + pstate_write(env, val); + + for (i = 0; i < ARRAY_SIZE(hvf_sreg_match); i++) { + if (hvf_sreg_match[i].cp_idx == -1) { + continue; + } + + ret = hv_vcpu_get_sys_reg(cpu->hvf->fd, hvf_sreg_match[i].reg, &val); + assert_hvf_ok(ret); + + arm_cpu->cpreg_values[hvf_sreg_match[i].cp_idx] = val; + } + assert(write_list_to_cpustate(arm_cpu)); + + aarch64_restore_sp(env, arm_current_el(env)); + + return 0; +} + +int hvf_put_registers(CPUState *cpu) +{ + ARMCPU *arm_cpu = ARM_CPU(cpu); + CPUARMState *env = &arm_cpu->env; + hv_return_t ret; + uint64_t val; + hv_simd_fp_uchar16_t fpval; + int i; + + for (i = 0; i < ARRAY_SIZE(hvf_reg_match); i++) { + val = *(uint64_t *)((void *)env + hvf_reg_match[i].offset); + ret = hv_vcpu_set_reg(cpu->hvf->fd, hvf_reg_match[i].reg, val); + assert_hvf_ok(ret); + } + + for (i = 0; i < ARRAY_SIZE(hvf_fpreg_match); i++) { + memcpy(&fpval, (void *)env + hvf_fpreg_match[i].offset, sizeof(fpval)); + ret = hv_vcpu_set_simd_fp_reg(cpu->hvf->fd, hvf_fpreg_match[i].reg, + fpval); + assert_hvf_ok(ret); + } + + ret = hv_vcpu_set_reg(cpu->hvf->fd, HV_REG_FPCR, vfp_get_fpcr(env)); + assert_hvf_ok(ret); + + ret = hv_vcpu_set_reg(cpu->hvf->fd, HV_REG_FPSR, vfp_get_fpsr(env)); + assert_hvf_ok(ret); + + ret = hv_vcpu_set_reg(cpu->hvf->fd, HV_REG_CPSR, pstate_read(env)); + assert_hvf_ok(ret); + + aarch64_save_sp(env, arm_current_el(env)); + + assert(write_cpustate_to_list(arm_cpu, false)); + for (i = 0; i < ARRAY_SIZE(hvf_sreg_match); i++) { + if (hvf_sreg_match[i].cp_idx == -1) { + continue; + } + + val = arm_cpu->cpreg_values[hvf_sreg_match[i].cp_idx]; + ret = hv_vcpu_set_sys_reg(cpu->hvf->fd, hvf_sreg_match[i].reg, val); + assert_hvf_ok(ret); + } + + ret = hv_vcpu_set_vtimer_offset(cpu->hvf->fd, hvf_state->vtimer_offset); + assert_hvf_ok(ret); + + return 0; +} + +static void flush_cpu_state(CPUState *cpu) +{ + if (cpu->vcpu_dirty) { + hvf_put_registers(cpu); + cpu->vcpu_dirty = false; + } +} + +static void hvf_set_reg(CPUState *cpu, int rt, uint64_t val) +{ + hv_return_t r; + + flush_cpu_state(cpu); + + if (rt < 31) { + r = hv_vcpu_set_reg(cpu->hvf->fd, HV_REG_X0 + rt, val); + assert_hvf_ok(r); + } +} + +static uint64_t hvf_get_reg(CPUState *cpu, int rt) +{ + uint64_t val = 0; + hv_return_t r; + + flush_cpu_state(cpu); + + if (rt < 31) { + r = hv_vcpu_get_reg(cpu->hvf->fd, HV_REG_X0 + rt, &val); + assert_hvf_ok(r); + } + + return val; +} + +void hvf_arch_vcpu_destroy(CPUState *cpu) +{ +} + +int hvf_arch_init_vcpu(CPUState *cpu) +{ + ARMCPU *arm_cpu = ARM_CPU(cpu); + CPUARMState *env = &arm_cpu->env; + uint32_t sregs_match_len = ARRAY_SIZE(hvf_sreg_match); + uint32_t sregs_cnt = 0; + uint64_t pfr; + hv_return_t ret; + int i; + + env->aarch64 = 1; + asm volatile("mrs %0, cntfrq_el0" : "=r"(arm_cpu->gt_cntfrq_hz)); + + /* Allocate enough space for our sysreg sync */ + arm_cpu->cpreg_indexes = g_renew(uint64_t, arm_cpu->cpreg_indexes, + sregs_match_len); + arm_cpu->cpreg_values = g_renew(uint64_t, arm_cpu->cpreg_values, + sregs_match_len); + arm_cpu->cpreg_vmstate_indexes = g_renew(uint64_t, + arm_cpu->cpreg_vmstate_indexes, + sregs_match_len); + arm_cpu->cpreg_vmstate_values = g_renew(uint64_t, + arm_cpu->cpreg_vmstate_values, + sregs_match_len); + + memset(arm_cpu->cpreg_values, 0, sregs_match_len * sizeof(uint64_t)); + + /* Populate cp list for all known sysregs */ + for (i = 0; i < sregs_match_len; i++) { + const ARMCPRegInfo *ri; + uint32_t key = hvf_sreg_match[i].key; + + ri = get_arm_cp_reginfo(arm_cpu->cp_regs, key); + if (ri) { + assert(!(ri->type & ARM_CP_NO_RAW)); + hvf_sreg_match[i].cp_idx = sregs_cnt; + arm_cpu->cpreg_indexes[sregs_cnt++] = cpreg_to_kvm_id(key); + } else { + hvf_sreg_match[i].cp_idx = -1; + } + } + arm_cpu->cpreg_array_len = sregs_cnt; + arm_cpu->cpreg_vmstate_array_len = sregs_cnt; + + assert(write_cpustate_to_list(arm_cpu, false)); + + /* Set CP_NO_RAW system registers on init */ + ret = hv_vcpu_set_sys_reg(cpu->hvf->fd, HV_SYS_REG_MIDR_EL1, + arm_cpu->midr); + assert_hvf_ok(ret); + + ret = hv_vcpu_set_sys_reg(cpu->hvf->fd, HV_SYS_REG_MPIDR_EL1, + arm_cpu->mp_affinity); + assert_hvf_ok(ret); + + ret = hv_vcpu_get_sys_reg(cpu->hvf->fd, HV_SYS_REG_ID_AA64PFR0_EL1, &pfr); + assert_hvf_ok(ret); + pfr |= env->gicv3state ? (1 << 24) : 0; + ret = hv_vcpu_set_sys_reg(cpu->hvf->fd, HV_SYS_REG_ID_AA64PFR0_EL1, pfr); + assert_hvf_ok(ret); + + /* We're limited to underlying hardware caps, override internal versions */ + ret = hv_vcpu_get_sys_reg(cpu->hvf->fd, HV_SYS_REG_ID_AA64MMFR0_EL1, + &arm_cpu->isar.id_aa64mmfr0); + assert_hvf_ok(ret); + + return 0; +} + +void hvf_kick_vcpu_thread(CPUState *cpu) +{ + hv_vcpus_exit(&cpu->hvf->fd, 1); +} + +static void hvf_raise_exception(CPUState *cpu, uint32_t excp, + uint32_t syndrome) +{ + ARMCPU *arm_cpu = ARM_CPU(cpu); + CPUARMState *env = &arm_cpu->env; + + cpu->exception_index = excp; + env->exception.target_el = 1; + env->exception.syndrome = syndrome; + + arm_cpu_do_interrupt(cpu); +} + +static int hvf_sysreg_read(CPUState *cpu, uint32_t reg, uint32_t rt) +{ + ARMCPU *arm_cpu = ARM_CPU(cpu); + CPUARMState *env = &arm_cpu->env; + uint64_t val = 0; + + switch (reg) { + case SYSREG_CNTPCT_EL0: + val = qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL) / + gt_cntfrq_period_ns(arm_cpu); + break; + case SYSREG_OSLSR_EL1: + val = env->cp15.oslsr_el1; + break; + case SYSREG_OSDLR_EL1: + /* Dummy register */ + break; + default: + cpu_synchronize_state(cpu); + trace_hvf_unhandled_sysreg_read(env->pc, reg, + (reg >> 20) & 0x3, + (reg >> 14) & 0x7, + (reg >> 10) & 0xf, + (reg >> 1) & 0xf, + (reg >> 17) & 0x7); + hvf_raise_exception(cpu, EXCP_UDEF, syn_uncategorized()); + return 1; + } + + trace_hvf_sysreg_read(reg, + (reg >> 20) & 0x3, + (reg >> 14) & 0x7, + (reg >> 10) & 0xf, + (reg >> 1) & 0xf, + (reg >> 17) & 0x7, + val); + hvf_set_reg(cpu, rt, val); + + return 0; +} + +static int hvf_sysreg_write(CPUState *cpu, uint32_t reg, uint64_t val) +{ + ARMCPU *arm_cpu = ARM_CPU(cpu); + CPUARMState *env = &arm_cpu->env; + + trace_hvf_sysreg_write(reg, + (reg >> 20) & 0x3, + (reg >> 14) & 0x7, + (reg >> 10) & 0xf, + (reg >> 1) & 0xf, + (reg >> 17) & 0x7, + val); + + switch (reg) { + case SYSREG_OSLAR_EL1: + env->cp15.oslsr_el1 = val & 1; + break; + case SYSREG_OSDLR_EL1: + /* Dummy register */ + break; + default: + cpu_synchronize_state(cpu); + trace_hvf_unhandled_sysreg_write(env->pc, reg, + (reg >> 20) & 0x3, + (reg >> 14) & 0x7, + (reg >> 10) & 0xf, + (reg >> 1) & 0xf, + (reg >> 17) & 0x7); + hvf_raise_exception(cpu, EXCP_UDEF, syn_uncategorized()); + return 1; + } + + return 0; +} + +static int hvf_inject_interrupts(CPUState *cpu) +{ + if (cpu->interrupt_request & CPU_INTERRUPT_FIQ) { + trace_hvf_inject_fiq(); + hv_vcpu_set_pending_interrupt(cpu->hvf->fd, HV_INTERRUPT_TYPE_FIQ, + true); + } + + if (cpu->interrupt_request & CPU_INTERRUPT_HARD) { + trace_hvf_inject_irq(); + hv_vcpu_set_pending_interrupt(cpu->hvf->fd, HV_INTERRUPT_TYPE_IRQ, + true); + } + + return 0; +} + +static uint64_t hvf_vtimer_val_raw(void) +{ + /* + * mach_absolute_time() returns the vtimer value without the VM + * offset that we define. Add our own offset on top. + */ + return mach_absolute_time() - hvf_state->vtimer_offset; +} + +static void hvf_sync_vtimer(CPUState *cpu) +{ + ARMCPU *arm_cpu = ARM_CPU(cpu); + hv_return_t r; + uint64_t ctl; + bool irq_state; + + if (!cpu->hvf->vtimer_masked) { + /* We will get notified on vtimer changes by hvf, nothing to do */ + return; + } + + r = hv_vcpu_get_sys_reg(cpu->hvf->fd, HV_SYS_REG_CNTV_CTL_EL0, &ctl); + assert_hvf_ok(r); + + irq_state = (ctl & (TMR_CTL_ENABLE | TMR_CTL_IMASK | TMR_CTL_ISTATUS)) == + (TMR_CTL_ENABLE | TMR_CTL_ISTATUS); + qemu_set_irq(arm_cpu->gt_timer_outputs[GTIMER_VIRT], irq_state); + + if (!irq_state) { + /* Timer no longer asserting, we can unmask it */ + hv_vcpu_set_vtimer_mask(cpu->hvf->fd, false); + cpu->hvf->vtimer_masked = false; + } +} + +int hvf_vcpu_exec(CPUState *cpu) +{ + ARMCPU *arm_cpu = ARM_CPU(cpu); + CPUARMState *env = &arm_cpu->env; + hv_vcpu_exit_t *hvf_exit = cpu->hvf->exit; + hv_return_t r; + bool advance_pc = false; + + if (hvf_inject_interrupts(cpu)) { + return EXCP_INTERRUPT; + } + + if (cpu->halted) { + return EXCP_HLT; + } + + flush_cpu_state(cpu); + + qemu_mutex_unlock_iothread(); + assert_hvf_ok(hv_vcpu_run(cpu->hvf->fd)); + + /* handle VMEXIT */ + uint64_t exit_reason = hvf_exit->reason; + uint64_t syndrome = hvf_exit->exception.syndrome; + uint32_t ec = syn_get_ec(syndrome); + + qemu_mutex_lock_iothread(); + switch (exit_reason) { + case HV_EXIT_REASON_EXCEPTION: + /* This is the main one, handle below. */ + break; + case HV_EXIT_REASON_VTIMER_ACTIVATED: + qemu_set_irq(arm_cpu->gt_timer_outputs[GTIMER_VIRT], 1); + cpu->hvf->vtimer_masked = true; + return 0; + case HV_EXIT_REASON_CANCELED: + /* we got kicked, no exit to process */ + return 0; + default: + assert(0); + } + + hvf_sync_vtimer(cpu); + + switch (ec) { + case EC_DATAABORT: { + bool isv = syndrome & ARM_EL_ISV; + bool iswrite = (syndrome >> 6) & 1; + bool s1ptw = (syndrome >> 7) & 1; + uint32_t sas = (syndrome >> 22) & 3; + uint32_t len = 1 << sas; + uint32_t srt = (syndrome >> 16) & 0x1f; + uint64_t val = 0; + + trace_hvf_data_abort(env->pc, hvf_exit->exception.virtual_address, + hvf_exit->exception.physical_address, isv, + iswrite, s1ptw, len, srt); + + assert(isv); + + if (iswrite) { + val = hvf_get_reg(cpu, srt); + address_space_write(&address_space_memory, + hvf_exit->exception.physical_address, + MEMTXATTRS_UNSPECIFIED, &val, len); + } else { + address_space_read(&address_space_memory, + hvf_exit->exception.physical_address, + MEMTXATTRS_UNSPECIFIED, &val, len); + hvf_set_reg(cpu, srt, val); + } + + advance_pc = true; + break; + } + case EC_SYSTEMREGISTERTRAP: { + bool isread = (syndrome >> 0) & 1; + uint32_t rt = (syndrome >> 5) & 0x1f; + uint32_t reg = syndrome & SYSREG_MASK; + uint64_t val; + int ret = 0; + + if (isread) { + ret = hvf_sysreg_read(cpu, reg, rt); + } else { + val = hvf_get_reg(cpu, rt); + ret = hvf_sysreg_write(cpu, reg, val); + } + + advance_pc = !ret; + break; + } + case EC_WFX_TRAP: + advance_pc = true; + break; + case EC_AA64_HVC: + cpu_synchronize_state(cpu); + trace_hvf_unknown_hvc(env->xregs[0]); + /* SMCCC 1.3 section 5.2 says every unknown SMCCC call returns -1 */ + env->xregs[0] = -1; + break; + case EC_AA64_SMC: + cpu_synchronize_state(cpu); + trace_hvf_unknown_smc(env->xregs[0]); + hvf_raise_exception(cpu, EXCP_UDEF, syn_uncategorized()); + break; + default: + cpu_synchronize_state(cpu); + trace_hvf_exit(syndrome, ec, env->pc); + error_report("0x%llx: unhandled exception ec=0x%x", env->pc, ec); + } + + if (advance_pc) { + uint64_t pc; + + flush_cpu_state(cpu); + + r = hv_vcpu_get_reg(cpu->hvf->fd, HV_REG_PC, &pc); + assert_hvf_ok(r); + pc += 4; + r = hv_vcpu_set_reg(cpu->hvf->fd, HV_REG_PC, pc); + assert_hvf_ok(r); + } + + return 0; +} + +static const VMStateDescription vmstate_hvf_vtimer = { + .name = "hvf-vtimer", + .version_id = 1, + .minimum_version_id = 1, + .fields = (VMStateField[]) { + VMSTATE_UINT64(vtimer_val, HVFVTimer), + VMSTATE_END_OF_LIST() + }, +}; + +static void hvf_vm_state_change(void *opaque, bool running, RunState state) +{ + HVFVTimer *s = opaque; + + if (running) { + /* Update vtimer offset on all CPUs */ + hvf_state->vtimer_offset = mach_absolute_time() - s->vtimer_val; + cpu_synchronize_all_states(); + } else { + /* Remember vtimer value on every pause */ + s->vtimer_val = hvf_vtimer_val_raw(); + } +} + +int hvf_arch_init(void) +{ + hvf_state->vtimer_offset = mach_absolute_time(); + vmstate_register(NULL, 0, &vmstate_hvf_vtimer, &vtimer); + qemu_add_vm_change_state_handler(hvf_vm_state_change, &vtimer); + return 0; +} diff --git a/target/i386/hvf/hvf.c b/target/i386/hvf/hvf.c index XXXXXXX..XXXXXXX 100644 --- a/target/i386/hvf/hvf.c +++ b/target/i386/hvf/hvf.c @@ -XXX,XX +XXX,XX @@ static inline bool apic_bus_freq_is_known(CPUX86State *env) return env->apic_bus_freq != 0; } +void hvf_kick_vcpu_thread(CPUState *cpu) +{ + cpus_kick_thread(cpu); +} + int hvf_arch_init(void) { return 0; diff --git a/MAINTAINERS b/MAINTAINERS index XXXXXXX..XXXXXXX 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -XXX,XX +XXX,XX @@ F: accel/accel-*.c F: accel/Makefile.objs F: accel/stubs/Makefile.objs +Apple Silicon HVF CPUs +M: Alexander Graf <agraf@csgraf.de> +S: Maintained +F: target/arm/hvf/ + X86 HVF CPUs M: Cameron Esfahani <dirty@apple.com> M: Roman Bolshakov <r.bolshakov@yadro.com> diff --git a/target/arm/hvf/trace-events b/target/arm/hvf/trace-events new file mode 100644 index XXXXXXX..XXXXXXX --- /dev/null +++ b/target/arm/hvf/trace-events @@ -XXX,XX +XXX,XX @@ +hvf_unhandled_sysreg_read(uint64_t pc, uint32_t reg, uint32_t op0, uint32_t op1, uint32_t crn, uint32_t crm, uint32_t op2) "unhandled sysreg read at pc=0x%"PRIx64": 0x%08x (op0=%d op1=%d crn=%d crm=%d op2=%d)" +hvf_unhandled_sysreg_write(uint64_t pc, uint32_t reg, uint32_t op0, uint32_t op1, uint32_t crn, uint32_t crm, uint32_t op2) "unhandled sysreg write at pc=0x%"PRIx64": 0x%08x (op0=%d op1=%d crn=%d crm=%d op2=%d)" +hvf_inject_fiq(void) "injecting FIQ" +hvf_inject_irq(void) "injecting IRQ" +hvf_data_abort(uint64_t pc, uint64_t va, uint64_t pa, bool isv, bool iswrite, bool s1ptw, uint32_t len, uint32_t srt) "data abort: [pc=0x%"PRIx64" va=0x%016"PRIx64" pa=0x%016"PRIx64" isv=%d iswrite=%d s1ptw=%d len=%d srt=%d]" +hvf_sysreg_read(uint32_t reg, uint32_t op0, uint32_t op1, uint32_t crn, uint32_t crm, uint32_t op2, uint64_t val) "sysreg read 0x%08x (op0=%d op1=%d crn=%d crm=%d op2=%d) = 0x%016"PRIx64 +hvf_sysreg_write(uint32_t reg, uint32_t op0, uint32_t op1, uint32_t crn, uint32_t crm, uint32_t op2, uint64_t val) "sysreg write 0x%08x (op0=%d op1=%d crn=%d crm=%d op2=%d, val=0x%016"PRIx64")" +hvf_unknown_hvc(uint64_t x0) "unknown HVC! 0x%016"PRIx64 +hvf_unknown_smc(uint64_t x0) "unknown SMC! 0x%016"PRIx64 +hvf_exit(uint64_t syndrome, uint32_t ec, uint64_t pc) "exit: 0x%"PRIx64" [ec=0x%x pc=0x%"PRIx64"]" -- 2.20.1 From: Peter Collingbourne <pcc@google.com> Sleep on WFI until the VTIMER is due but allow ourselves to be woken up on IPI. In this implementation IPI is blocked on the CPU thread at startup and pselect() is used to atomically unblock the signal and begin sleeping. The signal is sent unconditionally so there's no need to worry about races between actually sleeping and the "we think we're sleeping" state. It may lead to an extra wakeup but that's better than missing it entirely. Signed-off-by: Peter Collingbourne <pcc@google.com> Signed-off-by: Alexander Graf <agraf@csgraf.de> Acked-by: Roman Bolshakov <r.bolshakov@yadro.com> Reviewed-by: Sergio Lopez <slp@redhat.com> Message-id: 20210916155404.86958-6-agraf@csgraf.de [agraf: Remove unused 'set' variable, always advance PC on WFX trap, support vm stop / continue operations and cntv offsets] Signed-off-by: Alexander Graf <agraf@csgraf.de> Acked-by: Roman Bolshakov <r.bolshakov@yadro.com> Reviewed-by: Sergio Lopez <slp@redhat.com> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- include/sysemu/hvf_int.h | 1 + accel/hvf/hvf-accel-ops.c | 5 +-- target/arm/hvf/hvf.c | 79 +++++++++++++++++++++++++++++++++++++++ 3 files changed, 82 insertions(+), 3 deletions(-) diff --git a/include/sysemu/hvf_int.h b/include/sysemu/hvf_int.h index XXXXXXX..XXXXXXX 100644 --- a/include/sysemu/hvf_int.h +++ b/include/sysemu/hvf_int.h @@ -XXX,XX +XXX,XX @@ struct hvf_vcpu_state { uint64_t fd; void *exit; bool vtimer_masked; + sigset_t unblock_ipi_mask; }; void assert_hvf_ok(hv_return_t ret); diff --git a/accel/hvf/hvf-accel-ops.c b/accel/hvf/hvf-accel-ops.c index XXXXXXX..XXXXXXX 100644 --- a/accel/hvf/hvf-accel-ops.c +++ b/accel/hvf/hvf-accel-ops.c @@ -XXX,XX +XXX,XX @@ static int hvf_init_vcpu(CPUState *cpu) cpu->hvf = g_malloc0(sizeof(*cpu->hvf)); /* init cpu signals */ - sigset_t set; struct sigaction sigact; memset(&sigact, 0, sizeof(sigact)); sigact.sa_handler = dummy_signal; sigaction(SIG_IPI, &sigact, NULL); - pthread_sigmask(SIG_BLOCK, NULL, &set); - sigdelset(&set, SIG_IPI); + pthread_sigmask(SIG_BLOCK, NULL, &cpu->hvf->unblock_ipi_mask); + sigdelset(&cpu->hvf->unblock_ipi_mask, SIG_IPI); #ifdef __aarch64__ r = hv_vcpu_create(&cpu->hvf->fd, (hv_vcpu_exit_t **)&cpu->hvf->exit, NULL); diff --git a/target/arm/hvf/hvf.c b/target/arm/hvf/hvf.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/hvf/hvf.c +++ b/target/arm/hvf/hvf.c @@ -XXX,XX +XXX,XX @@ * QEMU Hypervisor.framework support for Apple Silicon * Copyright 2020 Alexander Graf <agraf@csgraf.de> + * Copyright 2020 Google LLC * * This work is licensed under the terms of the GNU GPL, version 2 or later. * See the COPYING file in the top-level directory. @@ -XXX,XX +XXX,XX @@ int hvf_arch_init_vcpu(CPUState *cpu) void hvf_kick_vcpu_thread(CPUState *cpu) { + cpus_kick_thread(cpu); hv_vcpus_exit(&cpu->hvf->fd, 1); } @@ -XXX,XX +XXX,XX @@ static uint64_t hvf_vtimer_val_raw(void) return mach_absolute_time() - hvf_state->vtimer_offset; } +static uint64_t hvf_vtimer_val(void) +{ + if (!runstate_is_running()) { + /* VM is paused, the vtimer value is in vtimer.vtimer_val */ + return vtimer.vtimer_val; + } + + return hvf_vtimer_val_raw(); +} + +static void hvf_wait_for_ipi(CPUState *cpu, struct timespec *ts) +{ + /* + * Use pselect to sleep so that other threads can IPI us while we're + * sleeping. + */ + qatomic_mb_set(&cpu->thread_kicked, false); + qemu_mutex_unlock_iothread(); + pselect(0, 0, 0, 0, ts, &cpu->hvf->unblock_ipi_mask); + qemu_mutex_lock_iothread(); +} + +static void hvf_wfi(CPUState *cpu) +{ + ARMCPU *arm_cpu = ARM_CPU(cpu); + struct timespec ts; + hv_return_t r; + uint64_t ctl; + uint64_t cval; + int64_t ticks_to_sleep; + uint64_t seconds; + uint64_t nanos; + uint32_t cntfrq; + + if (cpu->interrupt_request & (CPU_INTERRUPT_HARD | CPU_INTERRUPT_FIQ)) { + /* Interrupt pending, no need to wait */ + return; + } + + r = hv_vcpu_get_sys_reg(cpu->hvf->fd, HV_SYS_REG_CNTV_CTL_EL0, &ctl); + assert_hvf_ok(r); + + if (!(ctl & 1) || (ctl & 2)) { + /* Timer disabled or masked, just wait for an IPI. */ + hvf_wait_for_ipi(cpu, NULL); + return; + } + + r = hv_vcpu_get_sys_reg(cpu->hvf->fd, HV_SYS_REG_CNTV_CVAL_EL0, &cval); + assert_hvf_ok(r); + + ticks_to_sleep = cval - hvf_vtimer_val(); + if (ticks_to_sleep < 0) { + return; + } + + cntfrq = gt_cntfrq_period_ns(arm_cpu); + seconds = muldiv64(ticks_to_sleep, cntfrq, NANOSECONDS_PER_SECOND); + ticks_to_sleep -= muldiv64(seconds, NANOSECONDS_PER_SECOND, cntfrq); + nanos = ticks_to_sleep * cntfrq; + + /* + * Don't sleep for less than the time a context switch would take, + * so that we can satisfy fast timer requests on the same CPU. + * Measurements on M1 show the sweet spot to be ~2ms. + */ + if (!seconds && nanos < (2 * SCALE_MS)) { + return; + } + + ts = (struct timespec) { seconds, nanos }; + hvf_wait_for_ipi(cpu, &ts); +} + static void hvf_sync_vtimer(CPUState *cpu) { ARMCPU *arm_cpu = ARM_CPU(cpu); @@ -XXX,XX +XXX,XX @@ int hvf_vcpu_exec(CPUState *cpu) } case EC_WFX_TRAP: advance_pc = true; + if (!(syndrome & WFX_IS_WFE)) { + hvf_wfi(cpu); + } break; case EC_AA64_HVC: cpu_synchronize_state(cpu); -- 2.20.1 Now that we have working system register sync, we push more target CPU properties into the virtual machine. That might be useful in some situations, but is not the typical case that users want. So let's add a -cpu host option that allows them to explicitly pass all CPU capabilities of their host CPU into the guest. Signed-off-by: Alexander Graf <agraf@csgraf.de> Acked-by: Roman Bolshakov <r.bolshakov@yadro.com> Reviewed-by: Sergio Lopez <slp@redhat.com> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Message-id: 20210916155404.86958-7-agraf@csgraf.de [PMM: drop unnecessary #include line from .h file] Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/cpu.h | 2 + target/arm/hvf_arm.h | 18 +++++++++ target/arm/kvm_arm.h | 2 - target/arm/cpu.c | 13 ++++-- target/arm/hvf/hvf.c | 95 ++++++++++++++++++++++++++++++++++++++++++++ 5 files changed, 124 insertions(+), 6 deletions(-) create mode 100644 target/arm/hvf_arm.h diff --git a/target/arm/cpu.h b/target/arm/cpu.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -XXX,XX +XXX,XX @@ bool write_cpustate_to_list(ARMCPU *cpu, bool kvm_sync); #define ARM_CPU_TYPE_NAME(name) (name ARM_CPU_TYPE_SUFFIX) #define CPU_RESOLVING_TYPE TYPE_ARM_CPU +#define TYPE_ARM_HOST_CPU "host-" TYPE_ARM_CPU + #define cpu_signal_handler cpu_arm_signal_handler #define cpu_list arm_cpu_list diff --git a/target/arm/hvf_arm.h b/target/arm/hvf_arm.h new file mode 100644 index XXXXXXX..XXXXXXX --- /dev/null +++ b/target/arm/hvf_arm.h @@ -XXX,XX +XXX,XX @@ +/* + * QEMU Hypervisor.framework (HVF) support -- ARM specifics + * + * Copyright (c) 2021 Alexander Graf + * + * This work is licensed under the terms of the GNU GPL, version 2 or later. + * See the COPYING file in the top-level directory. + * + */ + +#ifndef QEMU_HVF_ARM_H +#define QEMU_HVF_ARM_H + +#include "cpu.h" + +void hvf_arm_set_cpu_features_from_host(struct ARMCPU *cpu); + +#endif diff --git a/target/arm/kvm_arm.h b/target/arm/kvm_arm.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/kvm_arm.h +++ b/target/arm/kvm_arm.h @@ -XXX,XX +XXX,XX @@ bool kvm_arm_create_scratch_host_vcpu(const uint32_t *cpus_to_try, */ void kvm_arm_destroy_scratch_host_vcpu(int *fdarray); -#define TYPE_ARM_HOST_CPU "host-" TYPE_ARM_CPU - /** * ARMHostCPUFeatures: information about the host CPU (identified * by asking the host kernel) diff --git a/target/arm/cpu.c b/target/arm/cpu.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpu.c +++ b/target/arm/cpu.c @@ -XXX,XX +XXX,XX @@ #include "sysemu/tcg.h" #include "sysemu/hw_accel.h" #include "kvm_arm.h" +#include "hvf_arm.h" #include "disas/capstone.h" #include "fpu/softfloat.h" @@ -XXX,XX +XXX,XX @@ static void arm_cpu_realizefn(DeviceState *dev, Error **errp) * this is the first point where we can report it. */ if (cpu->host_cpu_probe_failed) { - if (!kvm_enabled()) { - error_setg(errp, "The 'host' CPU type can only be used with KVM"); + if (!kvm_enabled() && !hvf_enabled()) { + error_setg(errp, "The 'host' CPU type can only be used with KVM or HVF"); } else { error_setg(errp, "Failed to retrieve host CPU features"); } @@ -XXX,XX +XXX,XX @@ static void arm_cpu_class_init(ObjectClass *oc, void *data) #endif /* CONFIG_TCG */ } -#ifdef CONFIG_KVM +#if defined(CONFIG_KVM) || defined(CONFIG_HVF) static void arm_host_initfn(Object *obj) { ARMCPU *cpu = ARM_CPU(obj); +#ifdef CONFIG_KVM kvm_arm_set_cpu_features_from_host(cpu); if (arm_feature(&cpu->env, ARM_FEATURE_AARCH64)) { aarch64_add_sve_properties(obj); } +#else + hvf_arm_set_cpu_features_from_host(cpu); +#endif arm_cpu_post_init(obj); } @@ -XXX,XX +XXX,XX @@ static void arm_cpu_register_types(void) { type_register_static(&arm_cpu_type_info); -#ifdef CONFIG_KVM +#if defined(CONFIG_KVM) || defined(CONFIG_HVF) type_register_static(&host_arm_cpu_type_info); #endif } diff --git a/target/arm/hvf/hvf.c b/target/arm/hvf/hvf.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/hvf/hvf.c +++ b/target/arm/hvf/hvf.c @@ -XXX,XX +XXX,XX @@ #include "sysemu/hvf.h" #include "sysemu/hvf_int.h" #include "sysemu/hw_accel.h" +#include "hvf_arm.h" #include <mach/mach_time.h> @@ -XXX,XX +XXX,XX @@ typedef struct HVFVTimer { static HVFVTimer vtimer; +typedef struct ARMHostCPUFeatures { + ARMISARegisters isar; + uint64_t features; + uint64_t midr; + uint32_t reset_sctlr; + const char *dtb_compatible; +} ARMHostCPUFeatures; + +static ARMHostCPUFeatures arm_host_cpu_features; + struct hvf_reg_match { int reg; uint64_t offset; @@ -XXX,XX +XXX,XX @@ static uint64_t hvf_get_reg(CPUState *cpu, int rt) return val; } +static bool hvf_arm_get_host_cpu_features(ARMHostCPUFeatures *ahcf) +{ + ARMISARegisters host_isar = {}; + const struct isar_regs { + int reg; + uint64_t *val; + } regs[] = { + { HV_SYS_REG_ID_AA64PFR0_EL1, &host_isar.id_aa64pfr0 }, + { HV_SYS_REG_ID_AA64PFR1_EL1, &host_isar.id_aa64pfr1 }, + { HV_SYS_REG_ID_AA64DFR0_EL1, &host_isar.id_aa64dfr0 }, + { HV_SYS_REG_ID_AA64DFR1_EL1, &host_isar.id_aa64dfr1 }, + { HV_SYS_REG_ID_AA64ISAR0_EL1, &host_isar.id_aa64isar0 }, + { HV_SYS_REG_ID_AA64ISAR1_EL1, &host_isar.id_aa64isar1 }, + { HV_SYS_REG_ID_AA64MMFR0_EL1, &host_isar.id_aa64mmfr0 }, + { HV_SYS_REG_ID_AA64MMFR1_EL1, &host_isar.id_aa64mmfr1 }, + { HV_SYS_REG_ID_AA64MMFR2_EL1, &host_isar.id_aa64mmfr2 }, + }; + hv_vcpu_t fd; + hv_return_t r = HV_SUCCESS; + hv_vcpu_exit_t *exit; + int i; + + ahcf->dtb_compatible = "arm,arm-v8"; + ahcf->features = (1ULL << ARM_FEATURE_V8) | + (1ULL << ARM_FEATURE_NEON) | + (1ULL << ARM_FEATURE_AARCH64) | + (1ULL << ARM_FEATURE_PMU) | + (1ULL << ARM_FEATURE_GENERIC_TIMER); + + /* We set up a small vcpu to extract host registers */ + + if (hv_vcpu_create(&fd, &exit, NULL) != HV_SUCCESS) { + return false; + } + + for (i = 0; i < ARRAY_SIZE(regs); i++) { + r |= hv_vcpu_get_sys_reg(fd, regs[i].reg, regs[i].val); + } + r |= hv_vcpu_get_sys_reg(fd, HV_SYS_REG_MIDR_EL1, &ahcf->midr); + r |= hv_vcpu_destroy(fd); + + ahcf->isar = host_isar; + + /* + * A scratch vCPU returns SCTLR 0, so let's fill our default with the M1 + * boot SCTLR from https://github.com/AsahiLinux/m1n1/issues/97 + */ + ahcf->reset_sctlr = 0x30100180; + /* + * SPAN is disabled by default when SCTLR.SPAN=1. To improve compatibility, + * let's disable it on boot and then allow guest software to turn it on by + * setting it to 0. + */ + ahcf->reset_sctlr |= 0x00800000; + + /* Make sure we don't advertise AArch32 support for EL0/EL1 */ + if ((host_isar.id_aa64pfr0 & 0xff) != 0x11) { + return false; + } + + return r == HV_SUCCESS; +} + +void hvf_arm_set_cpu_features_from_host(ARMCPU *cpu) +{ + if (!arm_host_cpu_features.dtb_compatible) { + if (!hvf_enabled() || + !hvf_arm_get_host_cpu_features(&arm_host_cpu_features)) { + /* + * We can't report this error yet, so flag that we need to + * in arm_cpu_realizefn(). + */ + cpu->host_cpu_probe_failed = true; + return; + } + } + + cpu->dtb_compatible = arm_host_cpu_features.dtb_compatible; + cpu->isar = arm_host_cpu_features.isar; + cpu->env.features = arm_host_cpu_features.features; + cpu->midr = arm_host_cpu_features.midr; + cpu->reset_sctlr = arm_host_cpu_features.reset_sctlr; +} + void hvf_arch_vcpu_destroy(CPUState *cpu) { } -- 2.20.1 From: Alexander Graf <agraf@csgraf.de> We need to handle PSCI calls. Most of the TCG code works for us, but we can simplify it to only handle aa64 mode and we need to handle SUSPEND differently. This patch takes the TCG code as template and duplicates it in HVF. To tell the guest that we support PSCI 0.2 now, update the check in arm_cpu_initfn() as well. Signed-off-by: Alexander Graf <agraf@csgraf.de> Reviewed-by: Sergio Lopez <slp@redhat.com> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Message-id: 20210916155404.86958-8-agraf@csgraf.de Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/cpu.c | 4 +- target/arm/hvf/hvf.c | 141 ++++++++++++++++++++++++++++++++++-- target/arm/hvf/trace-events | 1 + 3 files changed, 139 insertions(+), 7 deletions(-) diff --git a/target/arm/cpu.c b/target/arm/cpu.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpu.c +++ b/target/arm/cpu.c @@ -XXX,XX +XXX,XX @@ static void arm_cpu_initfn(Object *obj) cpu->psci_version = 1; /* By default assume PSCI v0.1 */ cpu->kvm_target = QEMU_KVM_ARM_TARGET_NONE; - if (tcg_enabled()) { - cpu->psci_version = 2; /* TCG implements PSCI 0.2 */ + if (tcg_enabled() || hvf_enabled()) { + cpu->psci_version = 2; /* TCG and HVF implement PSCI 0.2 */ } } diff --git a/target/arm/hvf/hvf.c b/target/arm/hvf/hvf.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/hvf/hvf.c +++ b/target/arm/hvf/hvf.c @@ -XXX,XX +XXX,XX @@ #include "hw/irq.h" #include "qemu/main-loop.h" #include "sysemu/cpus.h" +#include "arm-powerctl.h" #include "target/arm/cpu.h" #include "target/arm/internals.h" #include "trace/trace-target_arm_hvf.h" @@ -XXX,XX +XXX,XX @@ #define TMR_CTL_IMASK (1 << 1) #define TMR_CTL_ISTATUS (1 << 2) +static void hvf_wfi(CPUState *cpu); + typedef struct HVFVTimer { /* Vtimer value during migration and paused state */ uint64_t vtimer_val; @@ -XXX,XX +XXX,XX @@ static void hvf_raise_exception(CPUState *cpu, uint32_t excp, arm_cpu_do_interrupt(cpu); } +static void hvf_psci_cpu_off(ARMCPU *arm_cpu) +{ + int32_t ret = arm_set_cpu_off(arm_cpu->mp_affinity); + assert(ret == QEMU_ARM_POWERCTL_RET_SUCCESS); +} + +/* + * Handle a PSCI call. + * + * Returns 0 on success + * -1 when the PSCI call is unknown, + */ +static bool hvf_handle_psci_call(CPUState *cpu) +{ + ARMCPU *arm_cpu = ARM_CPU(cpu); + CPUARMState *env = &arm_cpu->env; + uint64_t param[4] = { + env->xregs[0], + env->xregs[1], + env->xregs[2], + env->xregs[3] + }; + uint64_t context_id, mpidr; + bool target_aarch64 = true; + CPUState *target_cpu_state; + ARMCPU *target_cpu; + target_ulong entry; + int target_el = 1; + int32_t ret = 0; + + trace_hvf_psci_call(param[0], param[1], param[2], param[3], + arm_cpu->mp_affinity); + + switch (param[0]) { + case QEMU_PSCI_0_2_FN_PSCI_VERSION: + ret = QEMU_PSCI_0_2_RET_VERSION_0_2; + break; + case QEMU_PSCI_0_2_FN_MIGRATE_INFO_TYPE: + ret = QEMU_PSCI_0_2_RET_TOS_MIGRATION_NOT_REQUIRED; /* No trusted OS */ + break; + case QEMU_PSCI_0_2_FN_AFFINITY_INFO: + case QEMU_PSCI_0_2_FN64_AFFINITY_INFO: + mpidr = param[1]; + + switch (param[2]) { + case 0: + target_cpu_state = arm_get_cpu_by_id(mpidr); + if (!target_cpu_state) { + ret = QEMU_PSCI_RET_INVALID_PARAMS; + break; + } + target_cpu = ARM_CPU(target_cpu_state); + + ret = target_cpu->power_state; + break; + default: + /* Everything above affinity level 0 is always on. */ + ret = 0; + } + break; + case QEMU_PSCI_0_2_FN_SYSTEM_RESET: + qemu_system_reset_request(SHUTDOWN_CAUSE_GUEST_RESET); + /* + * QEMU reset and shutdown are async requests, but PSCI + * mandates that we never return from the reset/shutdown + * call, so power the CPU off now so it doesn't execute + * anything further. + */ + hvf_psci_cpu_off(arm_cpu); + break; + case QEMU_PSCI_0_2_FN_SYSTEM_OFF: + qemu_system_shutdown_request(SHUTDOWN_CAUSE_GUEST_SHUTDOWN); + hvf_psci_cpu_off(arm_cpu); + break; + case QEMU_PSCI_0_1_FN_CPU_ON: + case QEMU_PSCI_0_2_FN_CPU_ON: + case QEMU_PSCI_0_2_FN64_CPU_ON: + mpidr = param[1]; + entry = param[2]; + context_id = param[3]; + ret = arm_set_cpu_on(mpidr, entry, context_id, + target_el, target_aarch64); + break; + case QEMU_PSCI_0_1_FN_CPU_OFF: + case QEMU_PSCI_0_2_FN_CPU_OFF: + hvf_psci_cpu_off(arm_cpu); + break; + case QEMU_PSCI_0_1_FN_CPU_SUSPEND: + case QEMU_PSCI_0_2_FN_CPU_SUSPEND: + case QEMU_PSCI_0_2_FN64_CPU_SUSPEND: + /* Affinity levels are not supported in QEMU */ + if (param[1] & 0xfffe0000) { + ret = QEMU_PSCI_RET_INVALID_PARAMS; + break; + } + /* Powerdown is not supported, we always go into WFI */ + env->xregs[0] = 0; + hvf_wfi(cpu); + break; + case QEMU_PSCI_0_1_FN_MIGRATE: + case QEMU_PSCI_0_2_FN_MIGRATE: + ret = QEMU_PSCI_RET_NOT_SUPPORTED; + break; + default: + return false; + } + + env->xregs[0] = ret; + return true; +} + static int hvf_sysreg_read(CPUState *cpu, uint32_t reg, uint32_t rt) { ARMCPU *arm_cpu = ARM_CPU(cpu); @@ -XXX,XX +XXX,XX @@ int hvf_vcpu_exec(CPUState *cpu) break; case EC_AA64_HVC: cpu_synchronize_state(cpu); - trace_hvf_unknown_hvc(env->xregs[0]); - /* SMCCC 1.3 section 5.2 says every unknown SMCCC call returns -1 */ - env->xregs[0] = -1; + if (arm_cpu->psci_conduit == QEMU_PSCI_CONDUIT_HVC) { + if (!hvf_handle_psci_call(cpu)) { + trace_hvf_unknown_hvc(env->xregs[0]); + /* SMCCC 1.3 section 5.2 says every unknown SMCCC call returns -1 */ + env->xregs[0] = -1; + } + } else { + trace_hvf_unknown_hvc(env->xregs[0]); + hvf_raise_exception(cpu, EXCP_UDEF, syn_uncategorized()); + } break; case EC_AA64_SMC: cpu_synchronize_state(cpu); - trace_hvf_unknown_smc(env->xregs[0]); - hvf_raise_exception(cpu, EXCP_UDEF, syn_uncategorized()); + if (arm_cpu->psci_conduit == QEMU_PSCI_CONDUIT_SMC) { + advance_pc = true; + + if (!hvf_handle_psci_call(cpu)) { + trace_hvf_unknown_smc(env->xregs[0]); + /* SMCCC 1.3 section 5.2 says every unknown SMCCC call returns -1 */ + env->xregs[0] = -1; + } + } else { + trace_hvf_unknown_smc(env->xregs[0]); + hvf_raise_exception(cpu, EXCP_UDEF, syn_uncategorized()); + } break; default: cpu_synchronize_state(cpu); diff --git a/target/arm/hvf/trace-events b/target/arm/hvf/trace-events index XXXXXXX..XXXXXXX 100644 --- a/target/arm/hvf/trace-events +++ b/target/arm/hvf/trace-events @@ -XXX,XX +XXX,XX @@ hvf_sysreg_write(uint32_t reg, uint32_t op0, uint32_t op1, uint32_t crn, uint32_ hvf_unknown_hvc(uint64_t x0) "unknown HVC! 0x%016"PRIx64 hvf_unknown_smc(uint64_t x0) "unknown SMC! 0x%016"PRIx64 hvf_exit(uint64_t syndrome, uint32_t ec, uint64_t pc) "exit: 0x%"PRIx64" [ec=0x%x pc=0x%"PRIx64"]" +hvf_psci_call(uint64_t x0, uint64_t x1, uint64_t x2, uint64_t x3, uint32_t cpuid) "PSCI Call x0=0x%016"PRIx64" x1=0x%016"PRIx64" x2=0x%016"PRIx64" x3=0x%016"PRIx64" cpu=0x%x" -- 2.20.1 From: Alexander Graf <agraf@csgraf.de> Now that we have all logic in place that we need to handle Hypervisor.framework on Apple Silicon systems, let's add CONFIG_HVF for aarch64 as well so that we can build it. Signed-off-by: Alexander Graf <agraf@csgraf.de> Reviewed-by: Roman Bolshakov <r.bolshakov@yadro.com> Tested-by: Roman Bolshakov <r.bolshakov@yadro.com> (x86 only) Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Sergio Lopez <slp@redhat.com> Message-id: 20210916155404.86958-9-agraf@csgraf.de Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- meson.build | 7 +++++++ target/arm/hvf/meson.build | 3 +++ target/arm/meson.build | 2 ++ 3 files changed, 12 insertions(+) create mode 100644 target/arm/hvf/meson.build diff --git a/meson.build b/meson.build index XXXXXXX..XXXXXXX 100644 --- a/meson.build +++ b/meson.build @@ -XXX,XX +XXX,XX @@ else endif accelerator_targets = { 'CONFIG_KVM': kvm_targets } + +if cpu in ['aarch64'] + accelerator_targets += { + 'CONFIG_HVF': ['aarch64-softmmu'] + } +endif + if cpu in ['x86', 'x86_64', 'arm', 'aarch64'] # i386 emulator provides xenpv machine type for multiple architectures accelerator_targets += { diff --git a/target/arm/hvf/meson.build b/target/arm/hvf/meson.build new file mode 100644 index XXXXXXX..XXXXXXX --- /dev/null +++ b/target/arm/hvf/meson.build @@ -XXX,XX +XXX,XX @@ +arm_softmmu_ss.add(when: [hvf, 'CONFIG_HVF'], if_true: files( + 'hvf.c', +)) diff --git a/target/arm/meson.build b/target/arm/meson.build index XXXXXXX..XXXXXXX 100644 --- a/target/arm/meson.build +++ b/target/arm/meson.build @@ -XXX,XX +XXX,XX @@ arm_softmmu_ss.add(files( 'psci.c', )) +subdir('hvf') + target_arch += {'arm': arm_ss} target_softmmu_arch += {'arm': arm_softmmu_ss} -- 2.20.1 From: Alexander Graf <agraf@csgraf.de> We can expose cycle counters on the PMU easily. To be as compatible as possible, let's do so, but make sure we don't expose any other architectural counters that we can not model yet. This allows OSs to work that require PMU support. Signed-off-by: Alexander Graf <agraf@csgraf.de> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Message-id: 20210916155404.86958-10-agraf@csgraf.de Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/hvf/hvf.c | 179 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 179 insertions(+) diff --git a/target/arm/hvf/hvf.c b/target/arm/hvf/hvf.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/hvf/hvf.c +++ b/target/arm/hvf/hvf.c @@ -XXX,XX +XXX,XX @@ #define SYSREG_OSLSR_EL1 SYSREG(2, 0, 1, 1, 4) #define SYSREG_OSDLR_EL1 SYSREG(2, 0, 1, 3, 4) #define SYSREG_CNTPCT_EL0 SYSREG(3, 3, 14, 0, 1) +#define SYSREG_PMCR_EL0 SYSREG(3, 3, 9, 12, 0) +#define SYSREG_PMUSERENR_EL0 SYSREG(3, 3, 9, 14, 0) +#define SYSREG_PMCNTENSET_EL0 SYSREG(3, 3, 9, 12, 1) +#define SYSREG_PMCNTENCLR_EL0 SYSREG(3, 3, 9, 12, 2) +#define SYSREG_PMINTENCLR_EL1 SYSREG(3, 0, 9, 14, 2) +#define SYSREG_PMOVSCLR_EL0 SYSREG(3, 3, 9, 12, 3) +#define SYSREG_PMSWINC_EL0 SYSREG(3, 3, 9, 12, 4) +#define SYSREG_PMSELR_EL0 SYSREG(3, 3, 9, 12, 5) +#define SYSREG_PMCEID0_EL0 SYSREG(3, 3, 9, 12, 6) +#define SYSREG_PMCEID1_EL0 SYSREG(3, 3, 9, 12, 7) +#define SYSREG_PMCCNTR_EL0 SYSREG(3, 3, 9, 13, 0) +#define SYSREG_PMCCFILTR_EL0 SYSREG(3, 3, 14, 15, 7) #define WFX_IS_WFE (1 << 0) @@ -XXX,XX +XXX,XX @@ static int hvf_sysreg_read(CPUState *cpu, uint32_t reg, uint32_t rt) val = qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL) / gt_cntfrq_period_ns(arm_cpu); break; + case SYSREG_PMCR_EL0: + val = env->cp15.c9_pmcr; + break; + case SYSREG_PMCCNTR_EL0: + pmu_op_start(env); + val = env->cp15.c15_ccnt; + pmu_op_finish(env); + break; + case SYSREG_PMCNTENCLR_EL0: + val = env->cp15.c9_pmcnten; + break; + case SYSREG_PMOVSCLR_EL0: + val = env->cp15.c9_pmovsr; + break; + case SYSREG_PMSELR_EL0: + val = env->cp15.c9_pmselr; + break; + case SYSREG_PMINTENCLR_EL1: + val = env->cp15.c9_pminten; + break; + case SYSREG_PMCCFILTR_EL0: + val = env->cp15.pmccfiltr_el0; + break; + case SYSREG_PMCNTENSET_EL0: + val = env->cp15.c9_pmcnten; + break; + case SYSREG_PMUSERENR_EL0: + val = env->cp15.c9_pmuserenr; + break; + case SYSREG_PMCEID0_EL0: + case SYSREG_PMCEID1_EL0: + /* We can't really count anything yet, declare all events invalid */ + val = 0; + break; case SYSREG_OSLSR_EL1: val = env->cp15.oslsr_el1; break; @@ -XXX,XX +XXX,XX @@ static int hvf_sysreg_read(CPUState *cpu, uint32_t reg, uint32_t rt) return 0; } +static void pmu_update_irq(CPUARMState *env) +{ + ARMCPU *cpu = env_archcpu(env); + qemu_set_irq(cpu->pmu_interrupt, (env->cp15.c9_pmcr & PMCRE) && + (env->cp15.c9_pminten & env->cp15.c9_pmovsr)); +} + +static bool pmu_event_supported(uint16_t number) +{ + return false; +} + +/* Returns true if the counter (pass 31 for PMCCNTR) should count events using + * the current EL, security state, and register configuration. + */ +static bool pmu_counter_enabled(CPUARMState *env, uint8_t counter) +{ + uint64_t filter; + bool enabled, filtered = true; + int el = arm_current_el(env); + + enabled = (env->cp15.c9_pmcr & PMCRE) && + (env->cp15.c9_pmcnten & (1 << counter)); + + if (counter == 31) { + filter = env->cp15.pmccfiltr_el0; + } else { + filter = env->cp15.c14_pmevtyper[counter]; + } + + if (el == 0) { + filtered = filter & PMXEVTYPER_U; + } else if (el == 1) { + filtered = filter & PMXEVTYPER_P; + } + + if (counter != 31) { + /* + * If not checking PMCCNTR, ensure the counter is setup to an event we + * support + */ + uint16_t event = filter & PMXEVTYPER_EVTCOUNT; + if (!pmu_event_supported(event)) { + return false; + } + } + + return enabled && !filtered; +} + +static void pmswinc_write(CPUARMState *env, uint64_t value) +{ + unsigned int i; + for (i = 0; i < pmu_num_counters(env); i++) { + /* Increment a counter's count iff: */ + if ((value & (1 << i)) && /* counter's bit is set */ + /* counter is enabled and not filtered */ + pmu_counter_enabled(env, i) && + /* counter is SW_INCR */ + (env->cp15.c14_pmevtyper[i] & PMXEVTYPER_EVTCOUNT) == 0x0) { + /* + * Detect if this write causes an overflow since we can't predict + * PMSWINC overflows like we can for other events + */ + uint32_t new_pmswinc = env->cp15.c14_pmevcntr[i] + 1; + + if (env->cp15.c14_pmevcntr[i] & ~new_pmswinc & INT32_MIN) { + env->cp15.c9_pmovsr |= (1 << i); + pmu_update_irq(env); + } + + env->cp15.c14_pmevcntr[i] = new_pmswinc; + } + } +} + static int hvf_sysreg_write(CPUState *cpu, uint32_t reg, uint64_t val) { ARMCPU *arm_cpu = ARM_CPU(cpu); @@ -XXX,XX +XXX,XX @@ static int hvf_sysreg_write(CPUState *cpu, uint32_t reg, uint64_t val) val); switch (reg) { + case SYSREG_PMCCNTR_EL0: + pmu_op_start(env); + env->cp15.c15_ccnt = val; + pmu_op_finish(env); + break; + case SYSREG_PMCR_EL0: + pmu_op_start(env); + + if (val & PMCRC) { + /* The counter has been reset */ + env->cp15.c15_ccnt = 0; + } + + if (val & PMCRP) { + unsigned int i; + for (i = 0; i < pmu_num_counters(env); i++) { + env->cp15.c14_pmevcntr[i] = 0; + } + } + + env->cp15.c9_pmcr &= ~PMCR_WRITEABLE_MASK; + env->cp15.c9_pmcr |= (val & PMCR_WRITEABLE_MASK); + + pmu_op_finish(env); + break; + case SYSREG_PMUSERENR_EL0: + env->cp15.c9_pmuserenr = val & 0xf; + break; + case SYSREG_PMCNTENSET_EL0: + env->cp15.c9_pmcnten |= (val & pmu_counter_mask(env)); + break; + case SYSREG_PMCNTENCLR_EL0: + env->cp15.c9_pmcnten &= ~(val & pmu_counter_mask(env)); + break; + case SYSREG_PMINTENCLR_EL1: + pmu_op_start(env); + env->cp15.c9_pminten |= val; + pmu_op_finish(env); + break; + case SYSREG_PMOVSCLR_EL0: + pmu_op_start(env); + env->cp15.c9_pmovsr &= ~val; + pmu_op_finish(env); + break; + case SYSREG_PMSWINC_EL0: + pmu_op_start(env); + pmswinc_write(env, val); + pmu_op_finish(env); + break; + case SYSREG_PMSELR_EL0: + env->cp15.c9_pmselr = val & 0x1f; + break; + case SYSREG_PMCCFILTR_EL0: + pmu_op_start(env); + env->cp15.pmccfiltr_el0 = val & PMCCFILTR_EL0; + pmu_op_finish(env); + break; case SYSREG_OSLAR_EL1: env->cp15.oslsr_el1 = val & 1; break; -- 2.20.1 Currently gen_jmp_tb() assumes that if it is called then the jump it is handling is the only reason that we might be trying to end the TB, so it will use goto_tb if it can. This is usually the case: mostly "we did something that means we must end the TB" happens on a non-branch instruction. However, there are cases where we decide early in handling an instruction that we need to end the TB and return to the main loop, and then the insn is a complex one that involves gen_jmp_tb(). For instance, for M-profile FP instructions, in gen_preserve_fp_state() which is called from vfp_access_check() we want to force an exit to the main loop if lazy state preservation is active and we are in icount mode. Make gen_jmp_tb() look at the current value of is_jmp, and only use goto_tb if the previous is_jmp was DISAS_NEXT or DISAS_TOO_MANY. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20210913095440.13462-2-peter.maydell@linaro.org --- target/arm/translate.c | 34 +++++++++++++++++++++++++++++++++- 1 file changed, 33 insertions(+), 1 deletion(-) diff --git a/target/arm/translate.c b/target/arm/translate.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate.c +++ b/target/arm/translate.c @@ -XXX,XX +XXX,XX @@ static inline void gen_jmp_tb(DisasContext *s, uint32_t dest, int tbno) /* An indirect jump so that we still trigger the debug exception. */ gen_set_pc_im(s, dest); s->base.is_jmp = DISAS_JUMP; - } else { + return; + } + switch (s->base.is_jmp) { + case DISAS_NEXT: + case DISAS_TOO_MANY: + case DISAS_NORETURN: + /* + * The normal case: just go to the destination TB. + * NB: NORETURN happens if we generate code like + * gen_brcondi(l); + * gen_jmp(); + * gen_set_label(l); + * gen_jmp(); + * on the second call to gen_jmp(). + */ gen_goto_tb(s, tbno, dest); + break; + case DISAS_UPDATE_NOCHAIN: + case DISAS_UPDATE_EXIT: + /* + * We already decided we're leaving the TB for some other reason. + * Avoid using goto_tb so we really do exit back to the main loop + * and don't chain to another TB. + */ + gen_set_pc_im(s, dest); + gen_goto_ptr(); + s->base.is_jmp = DISAS_NORETURN; + break; + default: + /* + * We shouldn't be emitting code for a jump and also have + * is_jmp set to one of the special cases like DISAS_SWI. + */ + g_assert_not_reached(); } } -- 2.20.1 Architecturally, for an M-profile CPU with the LOB feature the LTPSIZE field in FPDSCR is always constant 4. QEMU's implementation enforces this everywhere, except that we don't check that it is true in incoming migration data. We're going to add come in gen_update_fp_context() which relies on the "always 4" property. Since this is TCG-only, we don't actually need to be robust to bogus incoming migration data, and the effect of it being wrong would be wrong code generation rather than a QEMU crash; but if it did ever happen somehow it would be very difficult to track down the cause. Add a check so that we fail the inbound migration if the FPDSCR.LTPSIZE value is incorrect. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20210913095440.13462-3-peter.maydell@linaro.org --- target/arm/machine.c | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/target/arm/machine.c b/target/arm/machine.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/machine.c +++ b/target/arm/machine.c @@ -XXX,XX +XXX,XX @@ static int cpu_post_load(void *opaque, int version_id) hw_breakpoint_update_all(cpu); hw_watchpoint_update_all(cpu); + /* + * TCG gen_update_fp_context() relies on the invariant that + * FPDSCR.LTPSIZE is constant 4 for M-profile with the LOB extension; + * forbid bogus incoming data with some other value. + */ + if (arm_feature(env, ARM_FEATURE_M) && cpu_isar_feature(aa32_lob, cpu)) { + if (extract32(env->v7m.fpdscr[M_REG_NS], + FPCR_LTPSIZE_SHIFT, FPCR_LTPSIZE_LENGTH) != 4 || + extract32(env->v7m.fpdscr[M_REG_S], + FPCR_LTPSIZE_SHIFT, FPCR_LTPSIZE_LENGTH) != 4) { + return -1; + } + } if (!kvm_enabled()) { pmu_op_finish(&cpu->env); } -- 2.20.1 Our current codegen for MVE always calls out to helper functions, because some byte lanes might be predicated. The common case is that in fact there is no predication active and all lanes should be updated together, so we can produce better code by detecting that and using the TCG generic vector infrastructure. Add a TB flag that is set when we can guarantee that there is no active MVE predication, and a bool in the DisasContext. Subsequent patches will use this flag to generate improved code for some instructions. In most cases when the predication state changes we simply end the TB after that instruction. For the code called from vfp_access_check() that handles lazy state preservation and creating a new FP context, we can usually avoid having to try to end the TB because luckily the new value of the flag following the register changes in those sequences doesn't depend on any runtime decisions. We do have to end the TB if the guest has enabled lazy FP state preservation but not automatic state preservation, but this is an odd corner case that is not going to be common in real-world code. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20210913095440.13462-4-peter.maydell@linaro.org --- target/arm/cpu.h | 4 +++- target/arm/translate.h | 2 ++ target/arm/helper.c | 33 +++++++++++++++++++++++++++++++++ target/arm/translate-m-nocp.c | 8 +++++++- target/arm/translate-mve.c | 13 ++++++++++++- target/arm/translate-vfp.c | 33 +++++++++++++++++++++++++++------ target/arm/translate.c | 8 ++++++++ 7 files changed, 92 insertions(+), 9 deletions(-) diff --git a/target/arm/cpu.h b/target/arm/cpu.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -XXX,XX +XXX,XX @@ typedef ARMCPU ArchCPU; * | TBFLAG_AM32 | +-----+----------+ * | | |TBFLAG_M32| * +-------------+----------------+----------+ - * 31 23 5 4 0 + * 31 23 6 5 0 * * Unless otherwise noted, these bits are cached in env->hflags. */ @@ -XXX,XX +XXX,XX @@ FIELD(TBFLAG_M32, LSPACT, 2, 1) /* Not cached. */ FIELD(TBFLAG_M32, NEW_FP_CTXT_NEEDED, 3, 1) /* Not cached. */ /* Set if FPCCR.S does not match current security state */ FIELD(TBFLAG_M32, FPCCR_S_WRONG, 4, 1) /* Not cached. */ +/* Set if MVE insns are definitely not predicated by VPR or LTPSIZE */ +FIELD(TBFLAG_M32, MVE_NO_PRED, 5, 1) /* Not cached. */ /* * Bit usage when in AArch64 state diff --git a/target/arm/translate.h b/target/arm/translate.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate.h +++ b/target/arm/translate.h @@ -XXX,XX +XXX,XX @@ typedef struct DisasContext { bool align_mem; /* True if PSTATE.IL is set */ bool pstate_il; + /* True if MVE insns are definitely not predicated by VPR or LTPSIZE */ + bool mve_no_pred; /* * >= 0, a copy of PSTATE.BTYPE, which will be 0 without v8.5-BTI. * < 0, set by the current instruction. diff --git a/target/arm/helper.c b/target/arm/helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -XXX,XX +XXX,XX @@ static inline void assert_hflags_rebuild_correctly(CPUARMState *env) #endif } +static bool mve_no_pred(CPUARMState *env) +{ + /* + * Return true if there is definitely no predication of MVE + * instructions by VPR or LTPSIZE. (Returning false even if there + * isn't any predication is OK; generated code will just be + * a little worse.) + * If the CPU does not implement MVE then this TB flag is always 0. + * + * NOTE: if you change this logic, the "recalculate s->mve_no_pred" + * logic in gen_update_fp_context() needs to be updated to match. + * + * We do not include the effect of the ECI bits here -- they are + * tracked in other TB flags. This simplifies the logic for + * "when did we emit code that changes the MVE_NO_PRED TB flag + * and thus need to end the TB?". + */ + if (cpu_isar_feature(aa32_mve, env_archcpu(env))) { + return false; + } + if (env->v7m.vpr) { + return false; + } + if (env->v7m.ltpsize < 4) { + return false; + } + return true; +} + void cpu_get_tb_cpu_state(CPUARMState *env, target_ulong *pc, target_ulong *cs_base, uint32_t *pflags) { @@ -XXX,XX +XXX,XX @@ void cpu_get_tb_cpu_state(CPUARMState *env, target_ulong *pc, if (env->v7m.fpccr[is_secure] & R_V7M_FPCCR_LSPACT_MASK) { DP_TBFLAG_M32(flags, LSPACT, 1); } + + if (mve_no_pred(env)) { + DP_TBFLAG_M32(flags, MVE_NO_PRED, 1); + } } else { /* * Note that XSCALE_CPAR shares bits with VECSTRIDE. diff --git a/target/arm/translate-m-nocp.c b/target/arm/translate-m-nocp.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-m-nocp.c +++ b/target/arm/translate-m-nocp.c @@ -XXX,XX +XXX,XX @@ static bool trans_VLLDM_VLSTM(DisasContext *s, arg_VLLDM_VLSTM *a) clear_eci_state(s); - /* End the TB, because we have updated FP control bits */ + /* + * End the TB, because we have updated FP control bits, + * and possibly VPR or LTPSIZE. + */ s->base.is_jmp = DISAS_UPDATE_EXIT; return true; } @@ -XXX,XX +XXX,XX @@ static bool gen_M_fp_sysreg_write(DisasContext *s, int regno, store_cpu_field(control, v7m.control[M_REG_S]); tcg_gen_andi_i32(tmp, tmp, ~FPCR_NZCV_MASK); gen_helper_vfp_set_fpscr(cpu_env, tmp); + s->base.is_jmp = DISAS_UPDATE_NOCHAIN; tcg_temp_free_i32(tmp); tcg_temp_free_i32(sfpa); break; @@ -XXX,XX +XXX,XX @@ static bool gen_M_fp_sysreg_write(DisasContext *s, int regno, } tmp = loadfn(s, opaque, true); store_cpu_field(tmp, v7m.vpr); + s->base.is_jmp = DISAS_UPDATE_NOCHAIN; break; case ARM_VFP_P0: { @@ -XXX,XX +XXX,XX @@ static bool gen_M_fp_sysreg_write(DisasContext *s, int regno, tcg_gen_deposit_i32(vpr, vpr, tmp, R_V7M_VPR_P0_SHIFT, R_V7M_VPR_P0_LENGTH); store_cpu_field(vpr, v7m.vpr); + s->base.is_jmp = DISAS_UPDATE_NOCHAIN; tcg_temp_free_i32(tmp); break; } diff --git a/target/arm/translate-mve.c b/target/arm/translate-mve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-mve.c +++ b/target/arm/translate-mve.c @@ -XXX,XX +XXX,XX @@ DO_LOGIC(VORR, gen_helper_mve_vorr) DO_LOGIC(VORN, gen_helper_mve_vorn) DO_LOGIC(VEOR, gen_helper_mve_veor) -DO_LOGIC(VPSEL, gen_helper_mve_vpsel) +static bool trans_VPSEL(DisasContext *s, arg_2op *a) +{ + /* This insn updates predication bits */ + s->base.is_jmp = DISAS_UPDATE_NOCHAIN; + return do_2op(s, a, gen_helper_mve_vpsel); +} #define DO_2OP(INSN, FN) \ static bool trans_##INSN(DisasContext *s, arg_2op *a) \ @@ -XXX,XX +XXX,XX @@ static bool trans_VPNOT(DisasContext *s, arg_VPNOT *a) } gen_helper_mve_vpnot(cpu_env); + /* This insn updates predication bits */ + s->base.is_jmp = DISAS_UPDATE_NOCHAIN; mve_update_eci(s); return true; } @@ -XXX,XX +XXX,XX @@ static bool do_vcmp(DisasContext *s, arg_vcmp *a, MVEGenCmpFn *fn) /* VPT */ gen_vpst(s, a->mask); } + /* This insn updates predication bits */ + s->base.is_jmp = DISAS_UPDATE_NOCHAIN; mve_update_eci(s); return true; } @@ -XXX,XX +XXX,XX @@ static bool do_vcmp_scalar(DisasContext *s, arg_vcmp_scalar *a, /* VPT */ gen_vpst(s, a->mask); } + /* This insn updates predication bits */ + s->base.is_jmp = DISAS_UPDATE_NOCHAIN; mve_update_eci(s); return true; } diff --git a/target/arm/translate-vfp.c b/target/arm/translate-vfp.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-vfp.c +++ b/target/arm/translate-vfp.c @@ -XXX,XX +XXX,XX @@ static inline long vfp_f16_offset(unsigned reg, bool top) * Generate code for M-profile lazy FP state preservation if needed; * this corresponds to the pseudocode PreserveFPState() function. */ -static void gen_preserve_fp_state(DisasContext *s) +static void gen_preserve_fp_state(DisasContext *s, bool skip_context_update) { if (s->v7m_lspact) { /* @@ -XXX,XX +XXX,XX @@ static void gen_preserve_fp_state(DisasContext *s) * any further FP insns in this TB. */ s->v7m_lspact = false; + /* + * The helper might have zeroed VPR, so we do not know the + * correct value for the MVE_NO_PRED TB flag any more. + * If we're about to create a new fp context then that + * will precisely determine the MVE_NO_PRED value (see + * gen_update_fp_context()). Otherwise, we must: + * - set s->mve_no_pred to false, so this instruction + * is generated to use helper functions + * - end the TB now, without chaining to the next TB + */ + if (skip_context_update || !s->v7m_new_fp_ctxt_needed) { + s->mve_no_pred = false; + s->base.is_jmp = DISAS_UPDATE_NOCHAIN; + } } } @@ -XXX,XX +XXX,XX @@ static void gen_update_fp_context(DisasContext *s) TCGv_i32 z32 = tcg_const_i32(0); store_cpu_field(z32, v7m.vpr); } - /* - * We don't need to arrange to end the TB, because the only - * parts of FPSCR which we cache in the TB flags are the VECLEN - * and VECSTRIDE, and those don't exist for M-profile. + * We just updated the FPSCR and VPR. Some of this state is cached + * in the MVE_NO_PRED TB flag. We want to avoid having to end the + * TB here, which means we need the new value of the MVE_NO_PRED + * flag to be exactly known here and the same for all executions. + * Luckily FPDSCR.LTPSIZE is always constant 4 and the VPR is + * always set to 0, so the new MVE_NO_PRED flag is always 1 + * if and only if we have MVE. + * + * (The other FPSCR state cached in TB flags is VECLEN and VECSTRIDE, + * but those do not exist for M-profile, so are not relevant here.) */ + s->mve_no_pred = dc_isar_feature(aa32_mve, s); if (s->v8m_secure) { bits |= R_V7M_CONTROL_SFPA_MASK; @@ -XXX,XX +XXX,XX @@ bool vfp_access_check_m(DisasContext *s, bool skip_context_update) /* Handle M-profile lazy FP state mechanics */ /* Trigger lazy-state preservation if necessary */ - gen_preserve_fp_state(s); + gen_preserve_fp_state(s, skip_context_update); if (!skip_context_update) { /* Update ownership of FP context and create new FP context if needed */ diff --git a/target/arm/translate.c b/target/arm/translate.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate.c +++ b/target/arm/translate.c @@ -XXX,XX +XXX,XX @@ static bool trans_DLS(DisasContext *s, arg_DLS *a) /* DLSTP: set FPSCR.LTPSIZE */ tmp = tcg_const_i32(a->size); store_cpu_field(tmp, v7m.ltpsize); + s->base.is_jmp = DISAS_UPDATE_NOCHAIN; } return true; } @@ -XXX,XX +XXX,XX @@ static bool trans_WLS(DisasContext *s, arg_WLS *a) assert(ok); tmp = tcg_const_i32(a->size); store_cpu_field(tmp, v7m.ltpsize); + /* + * LTPSIZE updated, but MVE_NO_PRED will always be the same thing (0) + * when we take this upcoming exit from this TB, so gen_jmp_tb() is OK. + */ } gen_jmp_tb(s, s->base.pc_next, 1); @@ -XXX,XX +XXX,XX @@ static bool trans_VCTP(DisasContext *s, arg_VCTP *a) gen_helper_mve_vctp(cpu_env, masklen); tcg_temp_free_i32(masklen); tcg_temp_free_i32(rn_shifted); + /* This insn updates predication bits */ + s->base.is_jmp = DISAS_UPDATE_NOCHAIN; mve_update_eci(s); return true; } @@ -XXX,XX +XXX,XX @@ static void arm_tr_init_disas_context(DisasContextBase *dcbase, CPUState *cs) dc->v7m_new_fp_ctxt_needed = EX_TBFLAG_M32(tb_flags, NEW_FP_CTXT_NEEDED); dc->v7m_lspact = EX_TBFLAG_M32(tb_flags, LSPACT); + dc->mve_no_pred = EX_TBFLAG_M32(tb_flags, MVE_NO_PRED); } else { dc->debug_target_el = EX_TBFLAG_ANY(tb_flags, DEBUG_TARGET_EL); dc->sctlr_b = EX_TBFLAG_A32(tb_flags, SCTLR__B); -- 2.20.1 When not predicating, implement the MVE bitwise logical insns directly using TCG vector operations. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20210913095440.13462-5-peter.maydell@linaro.org --- target/arm/translate-mve.c | 51 +++++++++++++++++++++++++++----------- 1 file changed, 36 insertions(+), 15 deletions(-) diff --git a/target/arm/translate-mve.c b/target/arm/translate-mve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-mve.c +++ b/target/arm/translate-mve.c @@ -XXX,XX +XXX,XX @@ static TCGv_ptr mve_qreg_ptr(unsigned reg) return ret; } +static bool mve_no_predication(DisasContext *s) +{ + /* + * Return true if we are executing the entire MVE instruction + * with no predication or partial-execution, and so we can safely + * use an inline TCG vector implementation. + */ + return s->eci == 0 && s->mve_no_pred; +} + static bool mve_check_qreg_bank(DisasContext *s, int qmask) { /* @@ -XXX,XX +XXX,XX @@ static bool trans_VNEG_fp(DisasContext *s, arg_1op *a) return do_1op(s, a, fns[a->size]); } -static bool do_2op(DisasContext *s, arg_2op *a, MVEGenTwoOpFn fn) +static bool do_2op_vec(DisasContext *s, arg_2op *a, MVEGenTwoOpFn fn, + GVecGen3Fn *vecfn) { TCGv_ptr qd, qn, qm; @@ -XXX,XX +XXX,XX @@ static bool do_2op(DisasContext *s, arg_2op *a, MVEGenTwoOpFn fn) return true; } - qd = mve_qreg_ptr(a->qd); - qn = mve_qreg_ptr(a->qn); - qm = mve_qreg_ptr(a->qm); - fn(cpu_env, qd, qn, qm); - tcg_temp_free_ptr(qd); - tcg_temp_free_ptr(qn); - tcg_temp_free_ptr(qm); + if (vecfn && mve_no_predication(s)) { + vecfn(a->size, mve_qreg_offset(a->qd), mve_qreg_offset(a->qn), + mve_qreg_offset(a->qm), 16, 16); + } else { + qd = mve_qreg_ptr(a->qd); + qn = mve_qreg_ptr(a->qn); + qm = mve_qreg_ptr(a->qm); + fn(cpu_env, qd, qn, qm); + tcg_temp_free_ptr(qd); + tcg_temp_free_ptr(qn); + tcg_temp_free_ptr(qm); + } mve_update_eci(s); return true; } -#define DO_LOGIC(INSN, HELPER) \ +static bool do_2op(DisasContext *s, arg_2op *a, MVEGenTwoOpFn *fn) +{ + return do_2op_vec(s, a, fn, NULL); +} + +#define DO_LOGIC(INSN, HELPER, VECFN) \ static bool trans_##INSN(DisasContext *s, arg_2op *a) \ { \ - return do_2op(s, a, HELPER); \ + return do_2op_vec(s, a, HELPER, VECFN); \ } -DO_LOGIC(VAND, gen_helper_mve_vand) -DO_LOGIC(VBIC, gen_helper_mve_vbic) -DO_LOGIC(VORR, gen_helper_mve_vorr) -DO_LOGIC(VORN, gen_helper_mve_vorn) -DO_LOGIC(VEOR, gen_helper_mve_veor) +DO_LOGIC(VAND, gen_helper_mve_vand, tcg_gen_gvec_and) +DO_LOGIC(VBIC, gen_helper_mve_vbic, tcg_gen_gvec_andc) +DO_LOGIC(VORR, gen_helper_mve_vorr, tcg_gen_gvec_or) +DO_LOGIC(VORN, gen_helper_mve_vorn, tcg_gen_gvec_orc) +DO_LOGIC(VEOR, gen_helper_mve_veor, tcg_gen_gvec_xor) static bool trans_VPSEL(DisasContext *s, arg_2op *a) { -- 2.20.1 Optimize MVE arithmetic ops when we have a TCG vector operation we can use. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20210913095440.13462-6-peter.maydell@linaro.org --- target/arm/translate-mve.c | 20 +++++++++++--------- 1 file changed, 11 insertions(+), 9 deletions(-) diff --git a/target/arm/translate-mve.c b/target/arm/translate-mve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-mve.c +++ b/target/arm/translate-mve.c @@ -XXX,XX +XXX,XX @@ static bool trans_VPSEL(DisasContext *s, arg_2op *a) return do_2op(s, a, gen_helper_mve_vpsel); } -#define DO_2OP(INSN, FN) \ +#define DO_2OP_VEC(INSN, FN, VECFN) \ static bool trans_##INSN(DisasContext *s, arg_2op *a) \ { \ static MVEGenTwoOpFn * const fns[] = { \ @@ -XXX,XX +XXX,XX @@ static bool trans_VPSEL(DisasContext *s, arg_2op *a) gen_helper_mve_##FN##w, \ NULL, \ }; \ - return do_2op(s, a, fns[a->size]); \ + return do_2op_vec(s, a, fns[a->size], VECFN); \ } -DO_2OP(VADD, vadd) -DO_2OP(VSUB, vsub) -DO_2OP(VMUL, vmul) +#define DO_2OP(INSN, FN) DO_2OP_VEC(INSN, FN, NULL) + +DO_2OP_VEC(VADD, vadd, tcg_gen_gvec_add) +DO_2OP_VEC(VSUB, vsub, tcg_gen_gvec_sub) +DO_2OP_VEC(VMUL, vmul, tcg_gen_gvec_mul) DO_2OP(VMULH_S, vmulhs) DO_2OP(VMULH_U, vmulhu) DO_2OP(VRMULH_S, vrmulhs) DO_2OP(VRMULH_U, vrmulhu) -DO_2OP(VMAX_S, vmaxs) -DO_2OP(VMAX_U, vmaxu) -DO_2OP(VMIN_S, vmins) -DO_2OP(VMIN_U, vminu) +DO_2OP_VEC(VMAX_S, vmaxs, tcg_gen_gvec_smax) +DO_2OP_VEC(VMAX_U, vmaxu, tcg_gen_gvec_umax) +DO_2OP_VEC(VMIN_S, vmins, tcg_gen_gvec_smin) +DO_2OP_VEC(VMIN_U, vminu, tcg_gen_gvec_umin) DO_2OP(VABD_S, vabds) DO_2OP(VABD_U, vabdu) DO_2OP(VHADD_S, vhadds) -- 2.20.1 Optimize the MVE VNEG and VABS insns by using TCG vector ops when possible. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20210913095440.13462-7-peter.maydell@linaro.org --- target/arm/translate-mve.c | 32 ++++++++++++++++++++++---------- 1 file changed, 22 insertions(+), 10 deletions(-) diff --git a/target/arm/translate-mve.c b/target/arm/translate-mve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-mve.c +++ b/target/arm/translate-mve.c @@ -XXX,XX +XXX,XX @@ static bool trans_VDUP(DisasContext *s, arg_VDUP *a) return true; } -static bool do_1op(DisasContext *s, arg_1op *a, MVEGenOneOpFn fn) +static bool do_1op_vec(DisasContext *s, arg_1op *a, MVEGenOneOpFn fn, + GVecGen2Fn vecfn) { TCGv_ptr qd, qm; @@ -XXX,XX +XXX,XX @@ static bool do_1op(DisasContext *s, arg_1op *a, MVEGenOneOpFn fn) return true; } - qd = mve_qreg_ptr(a->qd); - qm = mve_qreg_ptr(a->qm); - fn(cpu_env, qd, qm); - tcg_temp_free_ptr(qd); - tcg_temp_free_ptr(qm); + if (vecfn && mve_no_predication(s)) { + vecfn(a->size, mve_qreg_offset(a->qd), mve_qreg_offset(a->qm), 16, 16); + } else { + qd = mve_qreg_ptr(a->qd); + qm = mve_qreg_ptr(a->qm); + fn(cpu_env, qd, qm); + tcg_temp_free_ptr(qd); + tcg_temp_free_ptr(qm); + } mve_update_eci(s); return true; } -#define DO_1OP(INSN, FN) \ +static bool do_1op(DisasContext *s, arg_1op *a, MVEGenOneOpFn fn) +{ + return do_1op_vec(s, a, fn, NULL); +} + +#define DO_1OP_VEC(INSN, FN, VECFN) \ static bool trans_##INSN(DisasContext *s, arg_1op *a) \ { \ static MVEGenOneOpFn * const fns[] = { \ @@ -XXX,XX +XXX,XX @@ static bool do_1op(DisasContext *s, arg_1op *a, MVEGenOneOpFn fn) gen_helper_mve_##FN##w, \ NULL, \ }; \ - return do_1op(s, a, fns[a->size]); \ + return do_1op_vec(s, a, fns[a->size], VECFN); \ } +#define DO_1OP(INSN, FN) DO_1OP_VEC(INSN, FN, NULL) + DO_1OP(VCLZ, vclz) DO_1OP(VCLS, vcls) -DO_1OP(VABS, vabs) -DO_1OP(VNEG, vneg) +DO_1OP_VEC(VABS, vabs, tcg_gen_gvec_abs) +DO_1OP_VEC(VNEG, vneg, tcg_gen_gvec_neg) DO_1OP(VQABS, vqabs) DO_1OP(VQNEG, vqneg) DO_1OP(VMAXA, vmaxa) -- 2.20.1 Optimize the MVE VDUP insns by using TCG vector ops when possible. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20210913095440.13462-8-peter.maydell@linaro.org --- target/arm/translate-mve.c | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/target/arm/translate-mve.c b/target/arm/translate-mve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-mve.c +++ b/target/arm/translate-mve.c @@ -XXX,XX +XXX,XX @@ static bool trans_VDUP(DisasContext *s, arg_VDUP *a) return true; } - qd = mve_qreg_ptr(a->qd); rt = load_reg(s, a->rt); - tcg_gen_dup_i32(a->size, rt, rt); - gen_helper_mve_vdup(cpu_env, qd, rt); - tcg_temp_free_ptr(qd); + if (mve_no_predication(s)) { + tcg_gen_gvec_dup_i32(a->size, mve_qreg_offset(a->qd), 16, 16, rt); + } else { + qd = mve_qreg_ptr(a->qd); + tcg_gen_dup_i32(a->size, rt, rt); + gen_helper_mve_vdup(cpu_env, qd, rt); + tcg_temp_free_ptr(qd); + } tcg_temp_free_i32(rt); mve_update_eci(s); return true; -- 2.20.1 Optimize the MVE VMVN insn by using TCG vector ops when possible. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20210913095440.13462-9-peter.maydell@linaro.org --- target/arm/translate-mve.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/target/arm/translate-mve.c b/target/arm/translate-mve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-mve.c +++ b/target/arm/translate-mve.c @@ -XXX,XX +XXX,XX @@ static bool trans_VREV64(DisasContext *s, arg_1op *a) static bool trans_VMVN(DisasContext *s, arg_1op *a) { - return do_1op(s, a, gen_helper_mve_vmvn); + return do_1op_vec(s, a, gen_helper_mve_vmvn, tcg_gen_gvec_not); } static bool trans_VABS_fp(DisasContext *s, arg_1op *a) -- 2.20.1 Optimize the MVE VSHL and VSHR immediate forms by using TCG vector ops when possible. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20210913095440.13462-10-peter.maydell@linaro.org --- target/arm/translate-mve.c | 83 +++++++++++++++++++++++++++++--------- 1 file changed, 63 insertions(+), 20 deletions(-) diff --git a/target/arm/translate-mve.c b/target/arm/translate-mve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-mve.c +++ b/target/arm/translate-mve.c @@ -XXX,XX +XXX,XX @@ static bool trans_Vimm_1r(DisasContext *s, arg_1imm *a) return do_1imm(s, a, fn); } -static bool do_2shift(DisasContext *s, arg_2shift *a, MVEGenTwoOpShiftFn fn, - bool negateshift) +static bool do_2shift_vec(DisasContext *s, arg_2shift *a, MVEGenTwoOpShiftFn fn, + bool negateshift, GVecGen2iFn vecfn) { TCGv_ptr qd, qm; int shift = a->shift; @@ -XXX,XX +XXX,XX @@ static bool do_2shift(DisasContext *s, arg_2shift *a, MVEGenTwoOpShiftFn fn, shift = -shift; } - qd = mve_qreg_ptr(a->qd); - qm = mve_qreg_ptr(a->qm); - fn(cpu_env, qd, qm, tcg_constant_i32(shift)); - tcg_temp_free_ptr(qd); - tcg_temp_free_ptr(qm); + if (vecfn && mve_no_predication(s)) { + vecfn(a->size, mve_qreg_offset(a->qd), mve_qreg_offset(a->qm), + shift, 16, 16); + } else { + qd = mve_qreg_ptr(a->qd); + qm = mve_qreg_ptr(a->qm); + fn(cpu_env, qd, qm, tcg_constant_i32(shift)); + tcg_temp_free_ptr(qd); + tcg_temp_free_ptr(qm); + } mve_update_eci(s); return true; } -#define DO_2SHIFT(INSN, FN, NEGATESHIFT) \ - static bool trans_##INSN(DisasContext *s, arg_2shift *a) \ - { \ - static MVEGenTwoOpShiftFn * const fns[] = { \ - gen_helper_mve_##FN##b, \ - gen_helper_mve_##FN##h, \ - gen_helper_mve_##FN##w, \ - NULL, \ - }; \ - return do_2shift(s, a, fns[a->size], NEGATESHIFT); \ +static bool do_2shift(DisasContext *s, arg_2shift *a, MVEGenTwoOpShiftFn fn, + bool negateshift) +{ + return do_2shift_vec(s, a, fn, negateshift, NULL); +} + +#define DO_2SHIFT_VEC(INSN, FN, NEGATESHIFT, VECFN) \ + static bool trans_##INSN(DisasContext *s, arg_2shift *a) \ + { \ + static MVEGenTwoOpShiftFn * const fns[] = { \ + gen_helper_mve_##FN##b, \ + gen_helper_mve_##FN##h, \ + gen_helper_mve_##FN##w, \ + NULL, \ + }; \ + return do_2shift_vec(s, a, fns[a->size], NEGATESHIFT, VECFN); \ } -DO_2SHIFT(VSHLI, vshli_u, false) +#define DO_2SHIFT(INSN, FN, NEGATESHIFT) \ + DO_2SHIFT_VEC(INSN, FN, NEGATESHIFT, NULL) + +static void do_gvec_shri_s(unsigned vece, uint32_t dofs, uint32_t aofs, + int64_t shift, uint32_t oprsz, uint32_t maxsz) +{ + /* + * We get here with a negated shift count, and we must handle + * shifts by the element size, which tcg_gen_gvec_sari() does not do. + */ + shift = -shift; + if (shift == (8 << vece)) { + shift--; + } + tcg_gen_gvec_sari(vece, dofs, aofs, shift, oprsz, maxsz); +} + +static void do_gvec_shri_u(unsigned vece, uint32_t dofs, uint32_t aofs, + int64_t shift, uint32_t oprsz, uint32_t maxsz) +{ + /* + * We get here with a negated shift count, and we must handle + * shifts by the element size, which tcg_gen_gvec_shri() does not do. + */ + shift = -shift; + if (shift == (8 << vece)) { + tcg_gen_gvec_dup_imm(vece, dofs, oprsz, maxsz, 0); + } else { + tcg_gen_gvec_shri(vece, dofs, aofs, shift, oprsz, maxsz); + } +} + +DO_2SHIFT_VEC(VSHLI, vshli_u, false, tcg_gen_gvec_shli) DO_2SHIFT(VQSHLI_S, vqshli_s, false) DO_2SHIFT(VQSHLI_U, vqshli_u, false) DO_2SHIFT(VQSHLUI, vqshlui_s, false) /* These right shifts use a left-shift helper with negated shift count */ -DO_2SHIFT(VSHRI_S, vshli_s, true) -DO_2SHIFT(VSHRI_U, vshli_u, true) +DO_2SHIFT_VEC(VSHRI_S, vshli_s, true, do_gvec_shri_s) +DO_2SHIFT_VEC(VSHRI_U, vshli_u, true, do_gvec_shri_u) DO_2SHIFT(VRSHRI_S, vrshli_s, true) DO_2SHIFT(VRSHRI_U, vrshli_u, true) -- 2.20.1 Optimize the MVE VSHLL insns by using TCG vector ops when possible. This includes the VMOVL insn, which we handle in mve.decode as "VSHLL with zero shift count". Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20210913095440.13462-11-peter.maydell@linaro.org --- target/arm/translate-mve.c | 67 +++++++++++++++++++++++++++++++++----- 1 file changed, 59 insertions(+), 8 deletions(-) diff --git a/target/arm/translate-mve.c b/target/arm/translate-mve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-mve.c +++ b/target/arm/translate-mve.c @@ -XXX,XX +XXX,XX @@ DO_2SHIFT_SCALAR(VQSHL_U_scalar, vqshli_u) DO_2SHIFT_SCALAR(VQRSHL_S_scalar, vqrshli_s) DO_2SHIFT_SCALAR(VQRSHL_U_scalar, vqrshli_u) -#define DO_VSHLL(INSN, FN) \ - static bool trans_##INSN(DisasContext *s, arg_2shift *a) \ - { \ - static MVEGenTwoOpShiftFn * const fns[] = { \ - gen_helper_mve_##FN##b, \ - gen_helper_mve_##FN##h, \ - }; \ - return do_2shift(s, a, fns[a->size], false); \ +#define DO_VSHLL(INSN, FN) \ + static bool trans_##INSN(DisasContext *s, arg_2shift *a) \ + { \ + static MVEGenTwoOpShiftFn * const fns[] = { \ + gen_helper_mve_##FN##b, \ + gen_helper_mve_##FN##h, \ + }; \ + return do_2shift_vec(s, a, fns[a->size], false, do_gvec_##FN); \ } +/* + * For the VSHLL vector helpers, the vece is the size of the input + * (ie MO_8 or MO_16); the helpers want to work in the output size. + * The shift count can be 0..<input size>, inclusive. (0 is VMOVL.) + */ +static void do_gvec_vshllbs(unsigned vece, uint32_t dofs, uint32_t aofs, + int64_t shift, uint32_t oprsz, uint32_t maxsz) +{ + unsigned ovece = vece + 1; + unsigned ibits = vece == MO_8 ? 8 : 16; + tcg_gen_gvec_shli(ovece, dofs, aofs, ibits, oprsz, maxsz); + tcg_gen_gvec_sari(ovece, dofs, dofs, ibits - shift, oprsz, maxsz); +} + +static void do_gvec_vshllbu(unsigned vece, uint32_t dofs, uint32_t aofs, + int64_t shift, uint32_t oprsz, uint32_t maxsz) +{ + unsigned ovece = vece + 1; + tcg_gen_gvec_andi(ovece, dofs, aofs, + ovece == MO_16 ? 0xff : 0xffff, oprsz, maxsz); + tcg_gen_gvec_shli(ovece, dofs, dofs, shift, oprsz, maxsz); +} + +static void do_gvec_vshllts(unsigned vece, uint32_t dofs, uint32_t aofs, + int64_t shift, uint32_t oprsz, uint32_t maxsz) +{ + unsigned ovece = vece + 1; + unsigned ibits = vece == MO_8 ? 8 : 16; + if (shift == 0) { + tcg_gen_gvec_sari(ovece, dofs, aofs, ibits, oprsz, maxsz); + } else { + tcg_gen_gvec_andi(ovece, dofs, aofs, + ovece == MO_16 ? 0xff00 : 0xffff0000, oprsz, maxsz); + tcg_gen_gvec_sari(ovece, dofs, dofs, ibits - shift, oprsz, maxsz); + } +} + +static void do_gvec_vshlltu(unsigned vece, uint32_t dofs, uint32_t aofs, + int64_t shift, uint32_t oprsz, uint32_t maxsz) +{ + unsigned ovece = vece + 1; + unsigned ibits = vece == MO_8 ? 8 : 16; + if (shift == 0) { + tcg_gen_gvec_shri(ovece, dofs, aofs, ibits, oprsz, maxsz); + } else { + tcg_gen_gvec_andi(ovece, dofs, aofs, + ovece == MO_16 ? 0xff00 : 0xffff0000, oprsz, maxsz); + tcg_gen_gvec_shri(ovece, dofs, dofs, ibits - shift, oprsz, maxsz); + } +} + DO_VSHLL(VSHLL_BS, vshllbs) DO_VSHLL(VSHLL_BU, vshllbu) DO_VSHLL(VSHLL_TS, vshllts) -- 2.20.1 Optimize the MVE shift-and-insert insns by using TCG vector ops when possible. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20210913095440.13462-12-peter.maydell@linaro.org --- target/arm/translate-mve.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/target/arm/translate-mve.c b/target/arm/translate-mve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-mve.c +++ b/target/arm/translate-mve.c @@ -XXX,XX +XXX,XX @@ DO_2SHIFT_VEC(VSHRI_U, vshli_u, true, do_gvec_shri_u) DO_2SHIFT(VRSHRI_S, vrshli_s, true) DO_2SHIFT(VRSHRI_U, vrshli_u, true) -DO_2SHIFT(VSRI, vsri, false) -DO_2SHIFT(VSLI, vsli, false) +DO_2SHIFT_VEC(VSRI, vsri, false, gen_gvec_sri) +DO_2SHIFT_VEC(VSLI, vsli, false, gen_gvec_sli) #define DO_2SHIFT_FP(INSN, FN) \ static bool trans_##INSN(DisasContext *s, arg_2shift *a) \ -- 2.20.1 Optimize the MVE 1op-immediate insns (VORR, VBIC, VMOV) to use TCG vector ops when possible. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20210913095440.13462-13-peter.maydell@linaro.org --- target/arm/translate-mve.c | 26 +++++++++++++++++++++----- 1 file changed, 21 insertions(+), 5 deletions(-) diff --git a/target/arm/translate-mve.c b/target/arm/translate-mve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-mve.c +++ b/target/arm/translate-mve.c @@ -XXX,XX +XXX,XX @@ static bool trans_VADDLV(DisasContext *s, arg_VADDLV *a) return true; } -static bool do_1imm(DisasContext *s, arg_1imm *a, MVEGenOneOpImmFn *fn) +static bool do_1imm(DisasContext *s, arg_1imm *a, MVEGenOneOpImmFn *fn, + GVecGen2iFn *vecfn) { TCGv_ptr qd; uint64_t imm; @@ -XXX,XX +XXX,XX @@ static bool do_1imm(DisasContext *s, arg_1imm *a, MVEGenOneOpImmFn *fn) imm = asimd_imm_const(a->imm, a->cmode, a->op); - qd = mve_qreg_ptr(a->qd); - fn(cpu_env, qd, tcg_constant_i64(imm)); - tcg_temp_free_ptr(qd); + if (vecfn && mve_no_predication(s)) { + vecfn(MO_64, mve_qreg_offset(a->qd), mve_qreg_offset(a->qd), + imm, 16, 16); + } else { + qd = mve_qreg_ptr(a->qd); + fn(cpu_env, qd, tcg_constant_i64(imm)); + tcg_temp_free_ptr(qd); + } mve_update_eci(s); return true; } +static void gen_gvec_vmovi(unsigned vece, uint32_t dofs, uint32_t aofs, + int64_t c, uint32_t oprsz, uint32_t maxsz) +{ + tcg_gen_gvec_dup_imm(vece, dofs, oprsz, maxsz, c); +} + static bool trans_Vimm_1r(DisasContext *s, arg_1imm *a) { /* Handle decode of cmode/op here between VORR/VBIC/VMOV */ MVEGenOneOpImmFn *fn; + GVecGen2iFn *vecfn; if ((a->cmode & 1) && a->cmode < 12) { if (a->op) { @@ -XXX,XX +XXX,XX @@ static bool trans_Vimm_1r(DisasContext *s, arg_1imm *a) * so the VBIC becomes a logical AND operation. */ fn = gen_helper_mve_vandi; + vecfn = tcg_gen_gvec_andi; } else { fn = gen_helper_mve_vorri; + vecfn = tcg_gen_gvec_ori; } } else { /* There is one unallocated cmode/op combination in this space */ @@ -XXX,XX +XXX,XX @@ static bool trans_Vimm_1r(DisasContext *s, arg_1imm *a) } /* asimd_imm_const() sorts out VMVNI vs VMOVI for us */ fn = gen_helper_mve_vmovi; + vecfn = gen_gvec_vmovi; } - return do_1imm(s, a, fn); + return do_1imm(s, a, fn, vecfn); } static bool do_2shift_vec(DisasContext *s, arg_2shift *a, MVEGenTwoOpShiftFn fn, -- 2.20.1