Another target-arm queue, since we're over 30 patches already. Most of this is RTH's SVE-patches-part-1. thanks -- PMM The following changes since commit d32e41a1188e929cc0fb16829ce3736046951e39: Merge remote-tracking branch 'remotes/famz/tags/docker-and-block-pull-request' into staging (2018-05-18 14:11:52 +0100) are available in the Git repository at: git://git.linaro.org/people/pmaydell/qemu-arm.git tags/pull-target-arm-20180518 for you to fetch changes up to b94f8f60bd841c5b737185cd38263e26822f77ab: target/arm: Implement SVE Permute - Extract Group (2018-05-18 17:48:09 +0100) ---------------------------------------------------------------- target-arm queue: * Initial part of SVE implementation (currently disabled) * smmuv3: fix some minor Coverity issues * add model of Xilinx ZynqMP generic DMA controller * expose (most) Arm coprocessor/system registers to gdb via QEMU's gdbstub, for reads only ---------------------------------------------------------------- Abdallah Bouassida (3): target/arm: Add "ARM_CP_NO_GDB" as a new bit field for ARMCPRegInfo type target/arm: Add "_S" suffix to the secure version of a sysreg target/arm: Add the XML dynamic generation Eric Auger (2): hw/arm/smmuv3: Fix Coverity issue in smmuv3_record_event hw/arm/smmu-common: Fix coverity issue in get_block_pte_address Francisco Iglesias (2): xlnx-zdma: Add a model of the Xilinx ZynqMP generic DMA xlnx-zynqmp: Connect the ZynqMP GDMA and ADMA Richard Henderson (25): target/arm: Introduce translate-a64.h target/arm: Add SVE decode skeleton target/arm: Implement SVE Bitwise Logical - Unpredicated Group target/arm: Implement SVE load vector/predicate target/arm: Implement SVE predicate test target/arm: Implement SVE Predicate Logical Operations Group target/arm: Implement SVE Predicate Misc Group target/arm: Implement SVE Integer Binary Arithmetic - Predicated Group target/arm: Implement SVE Integer Reduction Group target/arm: Implement SVE bitwise shift by immediate (predicated) target/arm: Implement SVE bitwise shift by vector (predicated) target/arm: Implement SVE bitwise shift by wide elements (predicated) target/arm: Implement SVE Integer Arithmetic - Unary Predicated Group target/arm: Implement SVE Integer Multiply-Add Group target/arm: Implement SVE Integer Arithmetic - Unpredicated Group target/arm: Implement SVE Index Generation Group target/arm: Implement SVE Stack Allocation Group target/arm: Implement SVE Bitwise Shift - Unpredicated Group target/arm: Implement SVE Compute Vector Address Group target/arm: Implement SVE floating-point exponential accelerator target/arm: Implement SVE floating-point trig select coefficient target/arm: Implement SVE Element Count Group target/arm: Implement SVE Bitwise Immediate Group target/arm: Implement SVE Integer Wide Immediate - Predicated Group target/arm: Implement SVE Permute - Extract Group hw/dma/Makefile.objs | 1 + target/arm/Makefile.objs | 10 + include/hw/arm/xlnx-zynqmp.h | 5 + include/hw/dma/xlnx-zdma.h | 84 ++ include/qom/cpu.h | 5 +- target/arm/cpu.h | 37 +- target/arm/helper-sve.h | 427 +++++++++ target/arm/helper.h | 1 + target/arm/translate-a64.h | 118 +++ gdbstub.c | 10 + hw/arm/smmu-common.c | 4 +- hw/arm/smmuv3.c | 2 +- hw/arm/xlnx-zynqmp.c | 53 ++ hw/dma/xlnx-zdma.c | 832 +++++++++++++++++ target/arm/cpu.c | 1 + target/arm/gdbstub.c | 76 ++ target/arm/helper.c | 57 +- target/arm/sve_helper.c | 1562 +++++++++++++++++++++++++++++++ target/arm/translate-a64.c | 119 +-- target/arm/translate-sve.c | 2070 ++++++++++++++++++++++++++++++++++++++++++ .gitignore | 1 + target/arm/sve.decode | 419 +++++++++ 22 files changed, 5778 insertions(+), 116 deletions(-) create mode 100644 include/hw/dma/xlnx-zdma.h create mode 100644 target/arm/helper-sve.h create mode 100644 target/arm/translate-a64.h create mode 100644 hw/dma/xlnx-zdma.c create mode 100644 target/arm/sve_helper.c create mode 100644 target/arm/translate-sve.c create mode 100644 target/arm/sve.decode From: Abdallah Bouassida <abdallah.bouassida@lauterbach.com> This is a preparation for the coming feature of creating dynamically an XML description for the ARM sysregs. A register has ARM_CP_NO_GDB enabled will not be shown in the dynamic XML. This bit is enabled automatically when creating CP_ANY wildcard aliases. This bit could be enabled manually for any register we want to remove from the dynamic XML description. Signed-off-by: Abdallah Bouassida <abdallah.bouassida@lauterbach.com> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Tested-by: Alex Bennée <alex.bennee@linaro.org> Message-id: 1524153386-3550-2-git-send-email-abdallah.bouassida@lauterbach.com Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/cpu.h | 3 ++- target/arm/helper.c | 2 +- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/target/arm/cpu.h b/target/arm/cpu.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -XXX,XX +XXX,XX @@ static inline uint64_t cpreg_to_kvm_id(uint32_t cpregid) #define ARM_LAST_SPECIAL ARM_CP_DC_ZVA #define ARM_CP_FPU 0x1000 #define ARM_CP_SVE 0x2000 +#define ARM_CP_NO_GDB 0x4000 /* Used only as a terminator for ARMCPRegInfo lists */ #define ARM_CP_SENTINEL 0xffff /* Mask of only the flag bits in a type field */ -#define ARM_CP_FLAG_MASK 0x30ff +#define ARM_CP_FLAG_MASK 0x70ff /* Valid values for ARMCPRegInfo state field, indicating which of * the AArch32 and AArch64 execution states this register is visible in. diff --git a/target/arm/helper.c b/target/arm/helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -XXX,XX +XXX,XX @@ static void add_cpreg_to_hashtable(ARMCPU *cpu, const ARMCPRegInfo *r, if (((r->crm == CP_ANY) && crm != 0) || ((r->opc1 == CP_ANY) && opc1 != 0) || ((r->opc2 == CP_ANY) && opc2 != 0)) { - r2->type |= ARM_CP_ALIAS; + r2->type |= ARM_CP_ALIAS | ARM_CP_NO_GDB; } /* Check that raw accesses are either forbidden or handled. Note that -- 2.17.0 From: Abdallah Bouassida <abdallah.bouassida@lauterbach.com> This is a preparation for the coming feature of creating dynamically an XML description for the ARM sysregs. Add "_S" suffix to the secure version of sysregs that have both S and NS views Replace (S) and (NS) by _S and _NS for the register that are manually defined, so all the registers follow the same convention. Signed-off-by: Abdallah Bouassida <abdallah.bouassida@lauterbach.com> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Tested-by: Alex Bennée <alex.bennee@linaro.org> Message-id: 1524153386-3550-3-git-send-email-abdallah.bouassida@lauterbach.com Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/helper.c | 29 ++++++++++++++++++----------- 1 file changed, 18 insertions(+), 11 deletions(-) diff --git a/target/arm/helper.c b/target/arm/helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -XXX,XX +XXX,XX @@ static const ARMCPRegInfo cp_reginfo[] = { * the secure register to be properly reset and migrated. There is also no * v8 EL1 version of the register so the non-secure instance stands alone. */ - { .name = "FCSEIDR(NS)", + { .name = "FCSEIDR", .cp = 15, .opc1 = 0, .crn = 13, .crm = 0, .opc2 = 0, .access = PL1_RW, .secure = ARM_CP_SECSTATE_NS, .fieldoffset = offsetof(CPUARMState, cp15.fcseidr_ns), .resetvalue = 0, .writefn = fcse_write, .raw_writefn = raw_write, }, - { .name = "FCSEIDR(S)", + { .name = "FCSEIDR_S", .cp = 15, .opc1 = 0, .crn = 13, .crm = 0, .opc2 = 0, .access = PL1_RW, .secure = ARM_CP_SECSTATE_S, .fieldoffset = offsetof(CPUARMState, cp15.fcseidr_s), @@ -XXX,XX +XXX,XX @@ static const ARMCPRegInfo cp_reginfo[] = { .access = PL1_RW, .secure = ARM_CP_SECSTATE_NS, .fieldoffset = offsetof(CPUARMState, cp15.contextidr_el[1]), .resetvalue = 0, .writefn = contextidr_write, .raw_writefn = raw_write, }, - { .name = "CONTEXTIDR(S)", .state = ARM_CP_STATE_AA32, + { .name = "CONTEXTIDR_S", .state = ARM_CP_STATE_AA32, .cp = 15, .opc1 = 0, .crn = 13, .crm = 0, .opc2 = 1, .access = PL1_RW, .secure = ARM_CP_SECSTATE_S, .fieldoffset = offsetof(CPUARMState, cp15.contextidr_s), @@ -XXX,XX +XXX,XX @@ static const ARMCPRegInfo generic_timer_cp_reginfo[] = { cp15.c14_timer[GTIMER_PHYS].ctl), .writefn = gt_phys_ctl_write, .raw_writefn = raw_write, }, - { .name = "CNTP_CTL(S)", + { .name = "CNTP_CTL_S", .cp = 15, .crn = 14, .crm = 2, .opc1 = 0, .opc2 = 1, .secure = ARM_CP_SECSTATE_S, .type = ARM_CP_IO | ARM_CP_ALIAS, .access = PL1_RW | PL0_R, @@ -XXX,XX +XXX,XX @@ static const ARMCPRegInfo generic_timer_cp_reginfo[] = { .accessfn = gt_ptimer_access, .readfn = gt_phys_tval_read, .writefn = gt_phys_tval_write, }, - { .name = "CNTP_TVAL(S)", + { .name = "CNTP_TVAL_S", .cp = 15, .crn = 14, .crm = 2, .opc1 = 0, .opc2 = 0, .secure = ARM_CP_SECSTATE_S, .type = ARM_CP_NO_RAW | ARM_CP_IO, .access = PL1_RW | PL0_R, @@ -XXX,XX +XXX,XX @@ static const ARMCPRegInfo generic_timer_cp_reginfo[] = { .accessfn = gt_ptimer_access, .writefn = gt_phys_cval_write, .raw_writefn = raw_write, }, - { .name = "CNTP_CVAL(S)", .cp = 15, .crm = 14, .opc1 = 2, + { .name = "CNTP_CVAL_S", .cp = 15, .crm = 14, .opc1 = 2, .secure = ARM_CP_SECSTATE_S, .access = PL1_RW | PL0_R, .type = ARM_CP_64BIT | ARM_CP_IO | ARM_CP_ALIAS, @@ -XXX,XX +XXX,XX @@ CpuDefinitionInfoList *arch_query_cpu_definitions(Error **errp) static void add_cpreg_to_hashtable(ARMCPU *cpu, const ARMCPRegInfo *r, void *opaque, int state, int secstate, - int crm, int opc1, int opc2) + int crm, int opc1, int opc2, + const char *name) { /* Private utility function for define_one_arm_cp_reg_with_opaque(): * add a single reginfo struct to the hash table. @@ -XXX,XX +XXX,XX @@ static void add_cpreg_to_hashtable(ARMCPU *cpu, const ARMCPRegInfo *r, int is64 = (r->type & ARM_CP_64BIT) ? 1 : 0; int ns = (secstate & ARM_CP_SECSTATE_NS) ? 1 : 0; + r2->name = g_strdup(name); /* Reset the secure state to the specific incoming state. This is * necessary as the register may have been defined with both states. */ @@ -XXX,XX +XXX,XX @@ void define_one_arm_cp_reg_with_opaque(ARMCPU *cpu, /* Under AArch32 CP registers can be common * (same for secure and non-secure world) or banked. */ + char *name; + switch (r->secure) { case ARM_CP_SECSTATE_S: case ARM_CP_SECSTATE_NS: add_cpreg_to_hashtable(cpu, r, opaque, state, - r->secure, crm, opc1, opc2); + r->secure, crm, opc1, opc2, + r->name); break; default: + name = g_strdup_printf("%s_S", r->name); add_cpreg_to_hashtable(cpu, r, opaque, state, ARM_CP_SECSTATE_S, - crm, opc1, opc2); + crm, opc1, opc2, name); + g_free(name); add_cpreg_to_hashtable(cpu, r, opaque, state, ARM_CP_SECSTATE_NS, - crm, opc1, opc2); + crm, opc1, opc2, r->name); break; } } else { @@ -XXX,XX +XXX,XX @@ void define_one_arm_cp_reg_with_opaque(ARMCPU *cpu, * of AArch32 */ add_cpreg_to_hashtable(cpu, r, opaque, state, ARM_CP_SECSTATE_NS, - crm, opc1, opc2); + crm, opc1, opc2, r->name); } } } -- 2.17.0 From: Abdallah Bouassida <abdallah.bouassida@lauterbach.com> Generate an XML description for the cp-regs. Register these regs with the gdb_register_coprocessor(). Add arm_gdb_get_sysreg() to use it as a callback to read those regs. Add a dummy arm_gdb_set_sysreg(). Signed-off-by: Abdallah Bouassida <abdallah.bouassida@lauterbach.com> Tested-by: Alex Bennée <alex.bennee@linaro.org> Message-id: 1524153386-3550-4-git-send-email-abdallah.bouassida@lauterbach.com Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- include/qom/cpu.h | 5 ++- target/arm/cpu.h | 26 +++++++++++++++ gdbstub.c | 10 ++++++ target/arm/cpu.c | 1 + target/arm/gdbstub.c | 76 ++++++++++++++++++++++++++++++++++++++++++++ target/arm/helper.c | 26 +++++++++++++++ 6 files changed, 143 insertions(+), 1 deletion(-) diff --git a/include/qom/cpu.h b/include/qom/cpu.h index XXXXXXX..XXXXXXX 100644 --- a/include/qom/cpu.h +++ b/include/qom/cpu.h @@ -XXX,XX +XXX,XX @@ struct TranslationBlock; * before the insn which triggers a watchpoint rather than after it. * @gdb_arch_name: Optional callback that returns the architecture name known * to GDB. The caller must free the returned string with g_free. + * @gdb_get_dynamic_xml: Callback to return dynamically generated XML for the + * gdb stub. Returns a pointer to the XML contents for the specified XML file + * or NULL if the CPU doesn't have a dynamically generated content for it. * @cpu_exec_enter: Callback for cpu_exec preparation. * @cpu_exec_exit: Callback for cpu_exec cleanup. * @cpu_exec_interrupt: Callback for processing interrupts in cpu_exec. @@ -XXX,XX +XXX,XX @@ typedef struct CPUClass { const struct VMStateDescription *vmsd; const char *gdb_core_xml_file; gchar * (*gdb_arch_name)(CPUState *cpu); - + const char * (*gdb_get_dynamic_xml)(CPUState *cpu, const char *xmlname); void (*cpu_exec_enter)(CPUState *cpu); void (*cpu_exec_exit)(CPUState *cpu); bool (*cpu_exec_interrupt)(CPUState *cpu, int interrupt_request); diff --git a/target/arm/cpu.h b/target/arm/cpu.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -XXX,XX +XXX,XX @@ enum { s<2n+1> maps to the most significant half of d<n> */ +/** + * DynamicGDBXMLInfo: + * @desc: Contains the XML descriptions. + * @num_cpregs: Number of the Coprocessor registers seen by GDB. + * @cpregs_keys: Array that contains the corresponding Key of + * a given cpreg with the same order of the cpreg in the XML description. + */ +typedef struct DynamicGDBXMLInfo { + char *desc; + int num_cpregs; + uint32_t *cpregs_keys; +} DynamicGDBXMLInfo; + /* CPU state for each instance of a generic timer (in cp15 c14) */ typedef struct ARMGenericTimer { uint64_t cval; /* Timer CompareValue register */ @@ -XXX,XX +XXX,XX @@ struct ARMCPU { uint64_t *cpreg_vmstate_values; int32_t cpreg_vmstate_array_len; + DynamicGDBXMLInfo dyn_xml; + /* Timers used by the generic (architected) timer */ QEMUTimer *gt_timer[NUM_GTIMERS]; /* GPIO outputs for generic timer */ @@ -XXX,XX +XXX,XX @@ hwaddr arm_cpu_get_phys_page_attrs_debug(CPUState *cpu, vaddr addr, int arm_cpu_gdb_read_register(CPUState *cpu, uint8_t *buf, int reg); int arm_cpu_gdb_write_register(CPUState *cpu, uint8_t *buf, int reg); +/* Dynamically generates for gdb stub an XML description of the sysregs from + * the cp_regs hashtable. Returns the registered sysregs number. + */ +int arm_gen_dynamic_xml(CPUState *cpu); + +/* Returns the dynamically generated XML for the gdb stub. + * Returns a pointer to the XML contents for the specified XML file or NULL + * if the XML name doesn't match the predefined one. + */ +const char *arm_gdb_get_dynamic_xml(CPUState *cpu, const char *xmlname); + int arm_cpu_write_elf64_note(WriteCoreDumpFunction f, CPUState *cs, int cpuid, void *opaque); int arm_cpu_write_elf32_note(WriteCoreDumpFunction f, CPUState *cs, diff --git a/gdbstub.c b/gdbstub.c index XXXXXXX..XXXXXXX 100644 --- a/gdbstub.c +++ b/gdbstub.c @@ -XXX,XX +XXX,XX @@ static const char *get_feature_xml(const char *p, const char **newp, } return target_xml; } + if (cc->gdb_get_dynamic_xml) { + CPUState *cpu = first_cpu; + char *xmlname = g_strndup(p, len); + const char *xml = cc->gdb_get_dynamic_xml(cpu, xmlname); + + g_free(xmlname); + if (xml) { + return xml; + } + } for (i = 0; ; i++) { name = xml_builtin[i][0]; if (!name || (strncmp(name, p, len) == 0 && strlen(name) == len)) diff --git a/target/arm/cpu.c b/target/arm/cpu.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpu.c +++ b/target/arm/cpu.c @@ -XXX,XX +XXX,XX @@ static void arm_cpu_class_init(ObjectClass *oc, void *data) cc->gdb_num_core_regs = 26; cc->gdb_core_xml_file = "arm-core.xml"; cc->gdb_arch_name = arm_gdb_arch_name; + cc->gdb_get_dynamic_xml = arm_gdb_get_dynamic_xml; cc->gdb_stop_before_watchpoint = true; cc->debug_excp_handler = arm_debug_excp_handler; cc->debug_check_watchpoint = arm_debug_check_watchpoint; diff --git a/target/arm/gdbstub.c b/target/arm/gdbstub.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/gdbstub.c +++ b/target/arm/gdbstub.c @@ -XXX,XX +XXX,XX @@ #include "cpu.h" #include "exec/gdbstub.h" +typedef struct RegisterSysregXmlParam { + CPUState *cs; + GString *s; +} RegisterSysregXmlParam; + /* Old gdb always expect FPA registers. Newer (xml-aware) gdb only expect whatever the target description contains. Due to a historical mishap the FPA registers appear in between core integer regs and the CPSR. @@ -XXX,XX +XXX,XX @@ int arm_cpu_gdb_write_register(CPUState *cs, uint8_t *mem_buf, int n) /* Unknown register. */ return 0; } + +static void arm_gen_one_xml_reg_tag(GString *s, DynamicGDBXMLInfo *dyn_xml, + ARMCPRegInfo *ri, uint32_t ri_key, + int bitsize) +{ + g_string_append_printf(s, "<reg name=\"%s\"", ri->name); + g_string_append_printf(s, " bitsize=\"%d\"", bitsize); + g_string_append_printf(s, " group=\"cp_regs\"/>"); + dyn_xml->num_cpregs++; + dyn_xml->cpregs_keys[dyn_xml->num_cpregs - 1] = ri_key; +} + +static void arm_register_sysreg_for_xml(gpointer key, gpointer value, + gpointer p) +{ + uint32_t ri_key = *(uint32_t *)key; + ARMCPRegInfo *ri = value; + RegisterSysregXmlParam *param = (RegisterSysregXmlParam *)p; + GString *s = param->s; + ARMCPU *cpu = ARM_CPU(param->cs); + CPUARMState *env = &cpu->env; + DynamicGDBXMLInfo *dyn_xml = &cpu->dyn_xml; + + if (!(ri->type & (ARM_CP_NO_RAW | ARM_CP_NO_GDB))) { + if (arm_feature(env, ARM_FEATURE_AARCH64)) { + if (ri->state == ARM_CP_STATE_AA64) { + arm_gen_one_xml_reg_tag(s , dyn_xml, ri, ri_key, 64); + } + } else { + if (ri->state == ARM_CP_STATE_AA32) { + if (!arm_feature(env, ARM_FEATURE_EL3) && + (ri->secure & ARM_CP_SECSTATE_S)) { + return; + } + if (ri->type & ARM_CP_64BIT) { + arm_gen_one_xml_reg_tag(s , dyn_xml, ri, ri_key, 64); + } else { + arm_gen_one_xml_reg_tag(s , dyn_xml, ri, ri_key, 32); + } + } + } + } +} + +int arm_gen_dynamic_xml(CPUState *cs) +{ + ARMCPU *cpu = ARM_CPU(cs); + GString *s = g_string_new(NULL); + RegisterSysregXmlParam param = {cs, s}; + + cpu->dyn_xml.num_cpregs = 0; + cpu->dyn_xml.cpregs_keys = g_malloc(sizeof(uint32_t *) * + g_hash_table_size(cpu->cp_regs)); + g_string_printf(s, "<?xml version=\"1.0\"?>"); + g_string_append_printf(s, "<!DOCTYPE target SYSTEM \"gdb-target.dtd\">"); + g_string_append_printf(s, "<feature name=\"org.qemu.gdb.arm.sys.regs\">"); + g_hash_table_foreach(cpu->cp_regs, arm_register_sysreg_for_xml, &param); + g_string_append_printf(s, "</feature>"); + cpu->dyn_xml.desc = g_string_free(s, false); + return cpu->dyn_xml.num_cpregs; +} + +const char *arm_gdb_get_dynamic_xml(CPUState *cs, const char *xmlname) +{ + ARMCPU *cpu = ARM_CPU(cs); + + if (strcmp(xmlname, "system-registers.xml") == 0) { + return cpu->dyn_xml.desc; + } + return NULL; +} diff --git a/target/arm/helper.c b/target/arm/helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -XXX,XX +XXX,XX @@ static void write_raw_cp_reg(CPUARMState *env, const ARMCPRegInfo *ri, } } +static int arm_gdb_get_sysreg(CPUARMState *env, uint8_t *buf, int reg) +{ + ARMCPU *cpu = arm_env_get_cpu(env); + const ARMCPRegInfo *ri; + uint32_t key; + + key = cpu->dyn_xml.cpregs_keys[reg]; + ri = get_arm_cp_reginfo(cpu->cp_regs, key); + if (ri) { + if (cpreg_field_is_64bit(ri)) { + return gdb_get_reg64(buf, (uint64_t)read_raw_cp_reg(env, ri)); + } else { + return gdb_get_reg32(buf, (uint32_t)read_raw_cp_reg(env, ri)); + } + } + return 0; +} + +static int arm_gdb_set_sysreg(CPUARMState *env, uint8_t *buf, int reg) +{ + return 0; +} + static bool raw_accessors_invalid(const ARMCPRegInfo *ri) { /* Return true if the regdef would cause an assertion if you called @@ -XXX,XX +XXX,XX @@ void arm_cpu_register_gdb_regs_for_features(ARMCPU *cpu) gdb_register_coprocessor(cs, vfp_gdb_get_reg, vfp_gdb_set_reg, 19, "arm-vfp.xml", 0); } + gdb_register_coprocessor(cs, arm_gdb_get_sysreg, arm_gdb_set_sysreg, + arm_gen_dynamic_xml(cs), + "system-registers.xml", 0); } /* Sort alphabetically by type name, except for "any". */ -- 2.17.0 From: Francisco Iglesias <frasse.iglesias@gmail.com> Add a model of the generic DMA found on Xilinx ZynqMP. Signed-off-by: Francisco Iglesias <frasse.iglesias@gmail.com> Signed-off-by: Edgar E. Iglesias <edgar.iglesias@xilinx.com> Reviewed-by: Edgar E. Iglesias <edgar.iglesias@xilinx.com> Message-id: 20180503214201.29082-2-frasse.iglesias@gmail.com Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- hw/dma/Makefile.objs | 1 + include/hw/dma/xlnx-zdma.h | 84 ++++ hw/dma/xlnx-zdma.c | 832 +++++++++++++++++++++++++++++++++++++ 3 files changed, 917 insertions(+) create mode 100644 include/hw/dma/xlnx-zdma.h create mode 100644 hw/dma/xlnx-zdma.c diff --git a/hw/dma/Makefile.objs b/hw/dma/Makefile.objs index XXXXXXX..XXXXXXX 100644 --- a/hw/dma/Makefile.objs +++ b/hw/dma/Makefile.objs @@ -XXX,XX +XXX,XX @@ common-obj-$(CONFIG_ETRAXFS) += etraxfs_dma.o common-obj-$(CONFIG_STP2000) += sparc32_dma.o obj-$(CONFIG_XLNX_ZYNQMP) += xlnx_dpdma.o obj-$(CONFIG_XLNX_ZYNQMP_ARM) += xlnx_dpdma.o +common-obj-$(CONFIG_XLNX_ZYNQMP_ARM) += xlnx-zdma.o obj-$(CONFIG_OMAP) += omap_dma.o soc_dma.o obj-$(CONFIG_PXA2XX) += pxa2xx_dma.o diff --git a/include/hw/dma/xlnx-zdma.h b/include/hw/dma/xlnx-zdma.h new file mode 100644 index XXXXXXX..XXXXXXX --- /dev/null +++ b/include/hw/dma/xlnx-zdma.h @@ -XXX,XX +XXX,XX @@ +/* + * QEMU model of the ZynqMP generic DMA + * + * Copyright (c) 2014 Xilinx Inc. + * Copyright (c) 2018 FEIMTECH AB + * + * Written by Edgar E. Iglesias <edgar.iglesias@xilinx.com>, + * Francisco Iglesias <francisco.iglesias@feimtech.se> + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL + * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + */ + +#ifndef XLNX_ZDMA_H +#define XLNX_ZDMA_H + +#include "hw/sysbus.h" +#include "hw/register.h" +#include "sysemu/dma.h" + +#define ZDMA_R_MAX (0x204 / 4) + +typedef enum { + DISABLED = 0, + ENABLED = 1, + PAUSED = 2, +} XlnxZDMAState; + +typedef union { + struct { + uint64_t addr; + uint32_t size; + uint32_t attr; + }; + uint32_t words[4]; +} XlnxZDMADescr; + +typedef struct XlnxZDMA { + SysBusDevice parent_obj; + MemoryRegion iomem; + MemTxAttrs attr; + MemoryRegion *dma_mr; + AddressSpace *dma_as; + qemu_irq irq_zdma_ch_imr; + + struct { + uint32_t bus_width; + } cfg; + + XlnxZDMAState state; + bool error; + + XlnxZDMADescr dsc_src; + XlnxZDMADescr dsc_dst; + + uint32_t regs[ZDMA_R_MAX]; + RegisterInfo regs_info[ZDMA_R_MAX]; + + /* We don't model the common bufs. Must be at least 16 bytes + to model write only mode. */ + uint8_t buf[2048]; +} XlnxZDMA; + +#define TYPE_XLNX_ZDMA "xlnx.zdma" + +#define XLNX_ZDMA(obj) \ + OBJECT_CHECK(XlnxZDMA, (obj), TYPE_XLNX_ZDMA) + +#endif /* XLNX_ZDMA_H */ diff --git a/hw/dma/xlnx-zdma.c b/hw/dma/xlnx-zdma.c new file mode 100644 index XXXXXXX..XXXXXXX --- /dev/null +++ b/hw/dma/xlnx-zdma.c @@ -XXX,XX +XXX,XX @@ +/* + * QEMU model of the ZynqMP generic DMA + * + * Copyright (c) 2014 Xilinx Inc. + * Copyright (c) 2018 FEIMTECH AB + * + * Written by Edgar E. Iglesias <edgar.iglesias@xilinx.com>, + * Francisco Iglesias <francisco.iglesias@feimtech.se> + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL + * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + */ + +#include "qemu/osdep.h" +#include "hw/dma/xlnx-zdma.h" +#include "qemu/bitops.h" +#include "qemu/log.h" +#include "qapi/error.h" + +#ifndef XLNX_ZDMA_ERR_DEBUG +#define XLNX_ZDMA_ERR_DEBUG 0 +#endif + +REG32(ZDMA_ERR_CTRL, 0x0) + FIELD(ZDMA_ERR_CTRL, APB_ERR_RES, 0, 1) +REG32(ZDMA_CH_ISR, 0x100) + FIELD(ZDMA_CH_ISR, DMA_PAUSE, 11, 1) + FIELD(ZDMA_CH_ISR, DMA_DONE, 10, 1) + FIELD(ZDMA_CH_ISR, AXI_WR_DATA, 9, 1) + FIELD(ZDMA_CH_ISR, AXI_RD_DATA, 8, 1) + FIELD(ZDMA_CH_ISR, AXI_RD_DST_DSCR, 7, 1) + FIELD(ZDMA_CH_ISR, AXI_RD_SRC_DSCR, 6, 1) + FIELD(ZDMA_CH_ISR, IRQ_DST_ACCT_ERR, 5, 1) + FIELD(ZDMA_CH_ISR, IRQ_SRC_ACCT_ERR, 4, 1) + FIELD(ZDMA_CH_ISR, BYTE_CNT_OVRFL, 3, 1) + FIELD(ZDMA_CH_ISR, DST_DSCR_DONE, 2, 1) + FIELD(ZDMA_CH_ISR, SRC_DSCR_DONE, 1, 1) + FIELD(ZDMA_CH_ISR, INV_APB, 0, 1) +REG32(ZDMA_CH_IMR, 0x104) + FIELD(ZDMA_CH_IMR, DMA_PAUSE, 11, 1) + FIELD(ZDMA_CH_IMR, DMA_DONE, 10, 1) + FIELD(ZDMA_CH_IMR, AXI_WR_DATA, 9, 1) + FIELD(ZDMA_CH_IMR, AXI_RD_DATA, 8, 1) + FIELD(ZDMA_CH_IMR, AXI_RD_DST_DSCR, 7, 1) + FIELD(ZDMA_CH_IMR, AXI_RD_SRC_DSCR, 6, 1) + FIELD(ZDMA_CH_IMR, IRQ_DST_ACCT_ERR, 5, 1) + FIELD(ZDMA_CH_IMR, IRQ_SRC_ACCT_ERR, 4, 1) + FIELD(ZDMA_CH_IMR, BYTE_CNT_OVRFL, 3, 1) + FIELD(ZDMA_CH_IMR, DST_DSCR_DONE, 2, 1) + FIELD(ZDMA_CH_IMR, SRC_DSCR_DONE, 1, 1) + FIELD(ZDMA_CH_IMR, INV_APB, 0, 1) +REG32(ZDMA_CH_IEN, 0x108) + FIELD(ZDMA_CH_IEN, DMA_PAUSE, 11, 1) + FIELD(ZDMA_CH_IEN, DMA_DONE, 10, 1) + FIELD(ZDMA_CH_IEN, AXI_WR_DATA, 9, 1) + FIELD(ZDMA_CH_IEN, AXI_RD_DATA, 8, 1) + FIELD(ZDMA_CH_IEN, AXI_RD_DST_DSCR, 7, 1) + FIELD(ZDMA_CH_IEN, AXI_RD_SRC_DSCR, 6, 1) + FIELD(ZDMA_CH_IEN, IRQ_DST_ACCT_ERR, 5, 1) + FIELD(ZDMA_CH_IEN, IRQ_SRC_ACCT_ERR, 4, 1) + FIELD(ZDMA_CH_IEN, BYTE_CNT_OVRFL, 3, 1) + FIELD(ZDMA_CH_IEN, DST_DSCR_DONE, 2, 1) + FIELD(ZDMA_CH_IEN, SRC_DSCR_DONE, 1, 1) + FIELD(ZDMA_CH_IEN, INV_APB, 0, 1) +REG32(ZDMA_CH_IDS, 0x10c) + FIELD(ZDMA_CH_IDS, DMA_PAUSE, 11, 1) + FIELD(ZDMA_CH_IDS, DMA_DONE, 10, 1) + FIELD(ZDMA_CH_IDS, AXI_WR_DATA, 9, 1) + FIELD(ZDMA_CH_IDS, AXI_RD_DATA, 8, 1) + FIELD(ZDMA_CH_IDS, AXI_RD_DST_DSCR, 7, 1) + FIELD(ZDMA_CH_IDS, AXI_RD_SRC_DSCR, 6, 1) + FIELD(ZDMA_CH_IDS, IRQ_DST_ACCT_ERR, 5, 1) + FIELD(ZDMA_CH_IDS, IRQ_SRC_ACCT_ERR, 4, 1) + FIELD(ZDMA_CH_IDS, BYTE_CNT_OVRFL, 3, 1) + FIELD(ZDMA_CH_IDS, DST_DSCR_DONE, 2, 1) + FIELD(ZDMA_CH_IDS, SRC_DSCR_DONE, 1, 1) + FIELD(ZDMA_CH_IDS, INV_APB, 0, 1) +REG32(ZDMA_CH_CTRL0, 0x110) + FIELD(ZDMA_CH_CTRL0, OVR_FETCH, 7, 1) + FIELD(ZDMA_CH_CTRL0, POINT_TYPE, 6, 1) + FIELD(ZDMA_CH_CTRL0, MODE, 4, 2) + FIELD(ZDMA_CH_CTRL0, RATE_CTRL, 3, 1) + FIELD(ZDMA_CH_CTRL0, CONT_ADDR, 2, 1) + FIELD(ZDMA_CH_CTRL0, CONT, 1, 1) +REG32(ZDMA_CH_CTRL1, 0x114) + FIELD(ZDMA_CH_CTRL1, DST_ISSUE, 5, 5) + FIELD(ZDMA_CH_CTRL1, SRC_ISSUE, 0, 5) +REG32(ZDMA_CH_FCI, 0x118) + FIELD(ZDMA_CH_FCI, PROG_CELL_CNT, 2, 2) + FIELD(ZDMA_CH_FCI, SIDE, 1, 1) + FIELD(ZDMA_CH_FCI, EN, 0, 1) +REG32(ZDMA_CH_STATUS, 0x11c) + FIELD(ZDMA_CH_STATUS, STATE, 0, 2) +REG32(ZDMA_CH_DATA_ATTR, 0x120) + FIELD(ZDMA_CH_DATA_ATTR, ARBURST, 26, 2) + FIELD(ZDMA_CH_DATA_ATTR, ARCACHE, 22, 4) + FIELD(ZDMA_CH_DATA_ATTR, ARQOS, 18, 4) + FIELD(ZDMA_CH_DATA_ATTR, ARLEN, 14, 4) + FIELD(ZDMA_CH_DATA_ATTR, AWBURST, 12, 2) + FIELD(ZDMA_CH_DATA_ATTR, AWCACHE, 8, 4) + FIELD(ZDMA_CH_DATA_ATTR, AWQOS, 4, 4) + FIELD(ZDMA_CH_DATA_ATTR, AWLEN, 0, 4) +REG32(ZDMA_CH_DSCR_ATTR, 0x124) + FIELD(ZDMA_CH_DSCR_ATTR, AXCOHRNT, 8, 1) + FIELD(ZDMA_CH_DSCR_ATTR, AXCACHE, 4, 4) + FIELD(ZDMA_CH_DSCR_ATTR, AXQOS, 0, 4) +REG32(ZDMA_CH_SRC_DSCR_WORD0, 0x128) +REG32(ZDMA_CH_SRC_DSCR_WORD1, 0x12c) + FIELD(ZDMA_CH_SRC_DSCR_WORD1, MSB, 0, 17) +REG32(ZDMA_CH_SRC_DSCR_WORD2, 0x130) + FIELD(ZDMA_CH_SRC_DSCR_WORD2, SIZE, 0, 30) +REG32(ZDMA_CH_SRC_DSCR_WORD3, 0x134) + FIELD(ZDMA_CH_SRC_DSCR_WORD3, CMD, 3, 2) + FIELD(ZDMA_CH_SRC_DSCR_WORD3, INTR, 2, 1) + FIELD(ZDMA_CH_SRC_DSCR_WORD3, TYPE, 1, 1) + FIELD(ZDMA_CH_SRC_DSCR_WORD3, COHRNT, 0, 1) +REG32(ZDMA_CH_DST_DSCR_WORD0, 0x138) +REG32(ZDMA_CH_DST_DSCR_WORD1, 0x13c) + FIELD(ZDMA_CH_DST_DSCR_WORD1, MSB, 0, 17) +REG32(ZDMA_CH_DST_DSCR_WORD2, 0x140) + FIELD(ZDMA_CH_DST_DSCR_WORD2, SIZE, 0, 30) +REG32(ZDMA_CH_DST_DSCR_WORD3, 0x144) + FIELD(ZDMA_CH_DST_DSCR_WORD3, INTR, 2, 1) + FIELD(ZDMA_CH_DST_DSCR_WORD3, TYPE, 1, 1) + FIELD(ZDMA_CH_DST_DSCR_WORD3, COHRNT, 0, 1) +REG32(ZDMA_CH_WR_ONLY_WORD0, 0x148) +REG32(ZDMA_CH_WR_ONLY_WORD1, 0x14c) +REG32(ZDMA_CH_WR_ONLY_WORD2, 0x150) +REG32(ZDMA_CH_WR_ONLY_WORD3, 0x154) +REG32(ZDMA_CH_SRC_START_LSB, 0x158) +REG32(ZDMA_CH_SRC_START_MSB, 0x15c) + FIELD(ZDMA_CH_SRC_START_MSB, ADDR, 0, 17) +REG32(ZDMA_CH_DST_START_LSB, 0x160) +REG32(ZDMA_CH_DST_START_MSB, 0x164) + FIELD(ZDMA_CH_DST_START_MSB, ADDR, 0, 17) +REG32(ZDMA_CH_RATE_CTRL, 0x18c) + FIELD(ZDMA_CH_RATE_CTRL, CNT, 0, 12) +REG32(ZDMA_CH_SRC_CUR_PYLD_LSB, 0x168) +REG32(ZDMA_CH_SRC_CUR_PYLD_MSB, 0x16c) + FIELD(ZDMA_CH_SRC_CUR_PYLD_MSB, ADDR, 0, 17) +REG32(ZDMA_CH_DST_CUR_PYLD_LSB, 0x170) +REG32(ZDMA_CH_DST_CUR_PYLD_MSB, 0x174) + FIELD(ZDMA_CH_DST_CUR_PYLD_MSB, ADDR, 0, 17) +REG32(ZDMA_CH_SRC_CUR_DSCR_LSB, 0x178) +REG32(ZDMA_CH_SRC_CUR_DSCR_MSB, 0x17c) + FIELD(ZDMA_CH_SRC_CUR_DSCR_MSB, ADDR, 0, 17) +REG32(ZDMA_CH_DST_CUR_DSCR_LSB, 0x180) +REG32(ZDMA_CH_DST_CUR_DSCR_MSB, 0x184) + FIELD(ZDMA_CH_DST_CUR_DSCR_MSB, ADDR, 0, 17) +REG32(ZDMA_CH_TOTAL_BYTE, 0x188) +REG32(ZDMA_CH_RATE_CNTL, 0x18c) + FIELD(ZDMA_CH_RATE_CNTL, CNT, 0, 12) +REG32(ZDMA_CH_IRQ_SRC_ACCT, 0x190) + FIELD(ZDMA_CH_IRQ_SRC_ACCT, CNT, 0, 8) +REG32(ZDMA_CH_IRQ_DST_ACCT, 0x194) + FIELD(ZDMA_CH_IRQ_DST_ACCT, CNT, 0, 8) +REG32(ZDMA_CH_DBG0, 0x198) + FIELD(ZDMA_CH_DBG0, CMN_BUF_FREE, 0, 9) +REG32(ZDMA_CH_DBG1, 0x19c) + FIELD(ZDMA_CH_DBG1, CMN_BUF_OCC, 0, 9) +REG32(ZDMA_CH_CTRL2, 0x200) + FIELD(ZDMA_CH_CTRL2, EN, 0, 1) + +enum { + PT_REG = 0, + PT_MEM = 1, +}; + +enum { + CMD_HALT = 1, + CMD_STOP = 2, +}; + +enum { + RW_MODE_RW = 0, + RW_MODE_WO = 1, + RW_MODE_RO = 2, +}; + +enum { + DTYPE_LINEAR = 0, + DTYPE_LINKED = 1, +}; + +enum { + AXI_BURST_FIXED = 0, + AXI_BURST_INCR = 1, +}; + +static void zdma_ch_imr_update_irq(XlnxZDMA *s) +{ + bool pending; + + pending = s->regs[R_ZDMA_CH_ISR] & ~s->regs[R_ZDMA_CH_IMR]; + + qemu_set_irq(s->irq_zdma_ch_imr, pending); +} + +static void zdma_ch_isr_postw(RegisterInfo *reg, uint64_t val64) +{ + XlnxZDMA *s = XLNX_ZDMA(reg->opaque); + zdma_ch_imr_update_irq(s); +} + +static uint64_t zdma_ch_ien_prew(RegisterInfo *reg, uint64_t val64) +{ + XlnxZDMA *s = XLNX_ZDMA(reg->opaque); + uint32_t val = val64; + + s->regs[R_ZDMA_CH_IMR] &= ~val; + zdma_ch_imr_update_irq(s); + return 0; +} + +static uint64_t zdma_ch_ids_prew(RegisterInfo *reg, uint64_t val64) +{ + XlnxZDMA *s = XLNX_ZDMA(reg->opaque); + uint32_t val = val64; + + s->regs[R_ZDMA_CH_IMR] |= val; + zdma_ch_imr_update_irq(s); + return 0; +} + +static void zdma_set_state(XlnxZDMA *s, XlnxZDMAState state) +{ + s->state = state; + ARRAY_FIELD_DP32(s->regs, ZDMA_CH_STATUS, STATE, state); + + /* Signal error if we have an error condition. */ + if (s->error) { + ARRAY_FIELD_DP32(s->regs, ZDMA_CH_STATUS, STATE, 3); + } +} + +static void zdma_src_done(XlnxZDMA *s) +{ + unsigned int cnt; + cnt = ARRAY_FIELD_EX32(s->regs, ZDMA_CH_IRQ_SRC_ACCT, CNT); + cnt++; + ARRAY_FIELD_DP32(s->regs, ZDMA_CH_IRQ_SRC_ACCT, CNT, cnt); + ARRAY_FIELD_DP32(s->regs, ZDMA_CH_ISR, SRC_DSCR_DONE, true); + + /* Did we overflow? */ + if (cnt != ARRAY_FIELD_EX32(s->regs, ZDMA_CH_IRQ_SRC_ACCT, CNT)) { + ARRAY_FIELD_DP32(s->regs, ZDMA_CH_ISR, IRQ_SRC_ACCT_ERR, true); + } + zdma_ch_imr_update_irq(s); +} + +static void zdma_dst_done(XlnxZDMA *s) +{ + unsigned int cnt; + cnt = ARRAY_FIELD_EX32(s->regs, ZDMA_CH_IRQ_DST_ACCT, CNT); + cnt++; + ARRAY_FIELD_DP32(s->regs, ZDMA_CH_IRQ_DST_ACCT, CNT, cnt); + ARRAY_FIELD_DP32(s->regs, ZDMA_CH_ISR, DST_DSCR_DONE, true); + + /* Did we overflow? */ + if (cnt != ARRAY_FIELD_EX32(s->regs, ZDMA_CH_IRQ_DST_ACCT, CNT)) { + ARRAY_FIELD_DP32(s->regs, ZDMA_CH_ISR, IRQ_DST_ACCT_ERR, true); + } + zdma_ch_imr_update_irq(s); +} + +static uint64_t zdma_get_regaddr64(XlnxZDMA *s, unsigned int basereg) +{ + uint64_t addr; + + addr = s->regs[basereg + 1]; + addr <<= 32; + addr |= s->regs[basereg]; + + return addr; +} + +static void zdma_put_regaddr64(XlnxZDMA *s, unsigned int basereg, uint64_t addr) +{ + s->regs[basereg] = addr; + s->regs[basereg + 1] = addr >> 32; +} + +static bool zdma_load_descriptor(XlnxZDMA *s, uint64_t addr, void *buf) +{ + /* ZDMA descriptors must be aligned to their own size. */ + if (addr % sizeof(XlnxZDMADescr)) { + qemu_log_mask(LOG_GUEST_ERROR, + "zdma: unaligned descriptor at %" PRIx64, + addr); + memset(buf, 0xdeadbeef, sizeof(XlnxZDMADescr)); + s->error = true; + return false; + } + + address_space_rw(s->dma_as, addr, s->attr, + buf, sizeof(XlnxZDMADescr), false); + return true; +} + +static void zdma_load_src_descriptor(XlnxZDMA *s) +{ + uint64_t src_addr; + unsigned int ptype = ARRAY_FIELD_EX32(s->regs, ZDMA_CH_CTRL0, POINT_TYPE); + + if (ptype == PT_REG) { + memcpy(&s->dsc_src, &s->regs[R_ZDMA_CH_SRC_DSCR_WORD0], + sizeof(s->dsc_src)); + return; + } + + src_addr = zdma_get_regaddr64(s, R_ZDMA_CH_SRC_CUR_DSCR_LSB); + + if (!zdma_load_descriptor(s, src_addr, &s->dsc_src)) { + ARRAY_FIELD_DP32(s->regs, ZDMA_CH_ISR, AXI_RD_SRC_DSCR, true); + } +} + +static void zdma_load_dst_descriptor(XlnxZDMA *s) +{ + uint64_t dst_addr; + unsigned int ptype = ARRAY_FIELD_EX32(s->regs, ZDMA_CH_CTRL0, POINT_TYPE); + + if (ptype == PT_REG) { + memcpy(&s->dsc_dst, &s->regs[R_ZDMA_CH_DST_DSCR_WORD0], + sizeof(s->dsc_dst)); + return; + } + + dst_addr = zdma_get_regaddr64(s, R_ZDMA_CH_DST_CUR_DSCR_LSB); + + if (!zdma_load_descriptor(s, dst_addr, &s->dsc_dst)) { + ARRAY_FIELD_DP32(s->regs, ZDMA_CH_ISR, AXI_RD_DST_DSCR, true); + } +} + +static uint64_t zdma_update_descr_addr(XlnxZDMA *s, bool type, + unsigned int basereg) +{ + uint64_t addr, next; + + if (type == DTYPE_LINEAR) { + next = zdma_get_regaddr64(s, basereg); + next += sizeof(s->dsc_dst); + zdma_put_regaddr64(s, basereg, next); + } else { + addr = zdma_get_regaddr64(s, basereg); + addr += sizeof(s->dsc_dst); + address_space_rw(s->dma_as, addr, s->attr, (void *) &next, 8, false); + zdma_put_regaddr64(s, basereg, next); + } + return next; +} + +static void zdma_write_dst(XlnxZDMA *s, uint8_t *buf, uint32_t len) +{ + uint32_t dst_size, dlen; + bool dst_intr, dst_type; + unsigned int ptype = ARRAY_FIELD_EX32(s->regs, ZDMA_CH_CTRL0, POINT_TYPE); + unsigned int rw_mode = ARRAY_FIELD_EX32(s->regs, ZDMA_CH_CTRL0, MODE); + unsigned int burst_type = ARRAY_FIELD_EX32(s->regs, ZDMA_CH_DATA_ATTR, + AWBURST); + + /* FIXED burst types are only supported in simple dma mode. */ + if (ptype != PT_REG) { + burst_type = AXI_BURST_INCR; + } + + while (len) { + dst_size = FIELD_EX32(s->dsc_dst.words[2], ZDMA_CH_DST_DSCR_WORD2, + SIZE); + dst_type = FIELD_EX32(s->dsc_dst.words[3], ZDMA_CH_DST_DSCR_WORD3, + TYPE); + if (dst_size == 0 && ptype == PT_MEM) { + uint64_t next; + next = zdma_update_descr_addr(s, dst_type, + R_ZDMA_CH_DST_CUR_DSCR_LSB); + zdma_load_descriptor(s, next, &s->dsc_dst); + dst_size = FIELD_EX32(s->dsc_dst.words[2], ZDMA_CH_DST_DSCR_WORD2, + SIZE); + dst_type = FIELD_EX32(s->dsc_dst.words[3], ZDMA_CH_DST_DSCR_WORD3, + TYPE); + } + + /* Match what hardware does by ignoring the dst_size and only using + * the src size for Simple register mode. */ + if (ptype == PT_REG && rw_mode != RW_MODE_WO) { + dst_size = len; + } + + dst_intr = FIELD_EX32(s->dsc_dst.words[3], ZDMA_CH_DST_DSCR_WORD3, + INTR); + + dlen = len > dst_size ? dst_size : len; + if (burst_type == AXI_BURST_FIXED) { + if (dlen > (s->cfg.bus_width / 8)) { + dlen = s->cfg.bus_width / 8; + } + } + + address_space_rw(s->dma_as, s->dsc_dst.addr, s->attr, buf, dlen, + true); + if (burst_type == AXI_BURST_INCR) { + s->dsc_dst.addr += dlen; + } + dst_size -= dlen; + buf += dlen; + len -= dlen; + + if (dst_size == 0 && dst_intr) { + zdma_dst_done(s); + } + + /* Write back to buffered descriptor. */ + s->dsc_dst.words[2] = FIELD_DP32(s->dsc_dst.words[2], + ZDMA_CH_DST_DSCR_WORD2, + SIZE, + dst_size); + } +} + +static void zdma_process_descr(XlnxZDMA *s) +{ + uint64_t src_addr; + uint32_t src_size, len; + unsigned int src_cmd; + bool src_intr, src_type; + unsigned int ptype = ARRAY_FIELD_EX32(s->regs, ZDMA_CH_CTRL0, POINT_TYPE); + unsigned int rw_mode = ARRAY_FIELD_EX32(s->regs, ZDMA_CH_CTRL0, MODE); + unsigned int burst_type = ARRAY_FIELD_EX32(s->regs, ZDMA_CH_DATA_ATTR, + ARBURST); + + src_addr = s->dsc_src.addr; + src_size = FIELD_EX32(s->dsc_src.words[2], ZDMA_CH_SRC_DSCR_WORD2, SIZE); + src_cmd = FIELD_EX32(s->dsc_src.words[3], ZDMA_CH_SRC_DSCR_WORD3, CMD); + src_type = FIELD_EX32(s->dsc_src.words[3], ZDMA_CH_SRC_DSCR_WORD3, TYPE); + src_intr = FIELD_EX32(s->dsc_src.words[3], ZDMA_CH_SRC_DSCR_WORD3, INTR); + + /* FIXED burst types and non-rw modes are only supported in + * simple dma mode. + */ + if (ptype != PT_REG) { + if (rw_mode != RW_MODE_RW) { + qemu_log_mask(LOG_GUEST_ERROR, + "zDMA: rw-mode=%d but not simple DMA mode.\n", + rw_mode); + } + if (burst_type != AXI_BURST_INCR) { + qemu_log_mask(LOG_GUEST_ERROR, + "zDMA: burst_type=%d but not simple DMA mode.\n", + burst_type); + } + burst_type = AXI_BURST_INCR; + rw_mode = RW_MODE_RW; + } + + if (rw_mode == RW_MODE_WO) { + /* In Simple DMA Write-Only, we need to push DST size bytes + * regardless of what SRC size is set to. */ + src_size = FIELD_EX32(s->dsc_dst.words[2], ZDMA_CH_DST_DSCR_WORD2, + SIZE); + memcpy(s->buf, &s->regs[R_ZDMA_CH_WR_ONLY_WORD0], s->cfg.bus_width / 8); + } + + while (src_size) { + len = src_size > ARRAY_SIZE(s->buf) ? ARRAY_SIZE(s->buf) : src_size; + if (burst_type == AXI_BURST_FIXED) { + if (len > (s->cfg.bus_width / 8)) { + len = s->cfg.bus_width / 8; + } + } + + if (rw_mode == RW_MODE_WO) { + if (len > s->cfg.bus_width / 8) { + len = s->cfg.bus_width / 8; + } + } else { + address_space_rw(s->dma_as, src_addr, s->attr, s->buf, len, + false); + if (burst_type == AXI_BURST_INCR) { + src_addr += len; + } + } + + if (rw_mode != RW_MODE_RO) { + zdma_write_dst(s, s->buf, len); + } + + s->regs[R_ZDMA_CH_TOTAL_BYTE] += len; + src_size -= len; + } + + ARRAY_FIELD_DP32(s->regs, ZDMA_CH_ISR, DMA_DONE, true); + + if (src_intr) { + zdma_src_done(s); + } + + /* Load next descriptor. */ + if (ptype == PT_REG || src_cmd == CMD_STOP) { + ARRAY_FIELD_DP32(s->regs, ZDMA_CH_CTRL2, EN, 0); + zdma_set_state(s, DISABLED); + return; + } + + if (src_cmd == CMD_HALT) { + zdma_set_state(s, PAUSED); + ARRAY_FIELD_DP32(s->regs, ZDMA_CH_ISR, DMA_PAUSE, 1); + zdma_ch_imr_update_irq(s); + return; + } + + zdma_update_descr_addr(s, src_type, R_ZDMA_CH_SRC_CUR_DSCR_LSB); +} + +static void zdma_run(XlnxZDMA *s) +{ + while (s->state == ENABLED && !s->error) { + zdma_load_src_descriptor(s); + + if (s->error) { + zdma_set_state(s, DISABLED); + } else { + zdma_process_descr(s); + } + } + + zdma_ch_imr_update_irq(s); +} + +static void zdma_update_descr_addr_from_start(XlnxZDMA *s) +{ + uint64_t src_addr, dst_addr; + + src_addr = zdma_get_regaddr64(s, R_ZDMA_CH_SRC_START_LSB); + zdma_put_regaddr64(s, R_ZDMA_CH_SRC_CUR_DSCR_LSB, src_addr); + dst_addr = zdma_get_regaddr64(s, R_ZDMA_CH_DST_START_LSB); + zdma_put_regaddr64(s, R_ZDMA_CH_DST_CUR_DSCR_LSB, dst_addr); + zdma_load_dst_descriptor(s); +} + +static void zdma_ch_ctrlx_postw(RegisterInfo *reg, uint64_t val64) +{ + XlnxZDMA *s = XLNX_ZDMA(reg->opaque); + + if (ARRAY_FIELD_EX32(s->regs, ZDMA_CH_CTRL2, EN)) { + s->error = false; + + if (s->state == PAUSED && + ARRAY_FIELD_EX32(s->regs, ZDMA_CH_CTRL0, CONT)) { + if (ARRAY_FIELD_EX32(s->regs, ZDMA_CH_CTRL0, CONT_ADDR) == 1) { + zdma_update_descr_addr_from_start(s); + } else { + bool src_type = FIELD_EX32(s->dsc_src.words[3], + ZDMA_CH_SRC_DSCR_WORD3, TYPE); + zdma_update_descr_addr(s, src_type, + R_ZDMA_CH_SRC_CUR_DSCR_LSB); + } + ARRAY_FIELD_DP32(s->regs, ZDMA_CH_CTRL0, CONT, false); + zdma_set_state(s, ENABLED); + } else if (s->state == DISABLED) { + zdma_update_descr_addr_from_start(s); + zdma_set_state(s, ENABLED); + } + } else { + /* Leave Paused state? */ + if (s->state == PAUSED && + ARRAY_FIELD_EX32(s->regs, ZDMA_CH_CTRL0, CONT)) { + zdma_set_state(s, DISABLED); + } + } + + zdma_run(s); +} + +static RegisterAccessInfo zdma_regs_info[] = { + { .name = "ZDMA_ERR_CTRL", .addr = A_ZDMA_ERR_CTRL, + .rsvd = 0xfffffffe, + },{ .name = "ZDMA_CH_ISR", .addr = A_ZDMA_CH_ISR, + .rsvd = 0xfffff000, + .w1c = 0xfff, + .post_write = zdma_ch_isr_postw, + },{ .name = "ZDMA_CH_IMR", .addr = A_ZDMA_CH_IMR, + .reset = 0xfff, + .rsvd = 0xfffff000, + .ro = 0xfff, + },{ .name = "ZDMA_CH_IEN", .addr = A_ZDMA_CH_IEN, + .rsvd = 0xfffff000, + .pre_write = zdma_ch_ien_prew, + },{ .name = "ZDMA_CH_IDS", .addr = A_ZDMA_CH_IDS, + .rsvd = 0xfffff000, + .pre_write = zdma_ch_ids_prew, + },{ .name = "ZDMA_CH_CTRL0", .addr = A_ZDMA_CH_CTRL0, + .reset = 0x80, + .rsvd = 0xffffff01, + .post_write = zdma_ch_ctrlx_postw, + },{ .name = "ZDMA_CH_CTRL1", .addr = A_ZDMA_CH_CTRL1, + .reset = 0x3ff, + .rsvd = 0xfffffc00, + },{ .name = "ZDMA_CH_FCI", .addr = A_ZDMA_CH_FCI, + .rsvd = 0xffffffc0, + },{ .name = "ZDMA_CH_STATUS", .addr = A_ZDMA_CH_STATUS, + .rsvd = 0xfffffffc, + .ro = 0x3, + },{ .name = "ZDMA_CH_DATA_ATTR", .addr = A_ZDMA_CH_DATA_ATTR, + .reset = 0x483d20f, + .rsvd = 0xf0000000, + },{ .name = "ZDMA_CH_DSCR_ATTR", .addr = A_ZDMA_CH_DSCR_ATTR, + .rsvd = 0xfffffe00, + },{ .name = "ZDMA_CH_SRC_DSCR_WORD0", .addr = A_ZDMA_CH_SRC_DSCR_WORD0, + },{ .name = "ZDMA_CH_SRC_DSCR_WORD1", .addr = A_ZDMA_CH_SRC_DSCR_WORD1, + .rsvd = 0xfffe0000, + },{ .name = "ZDMA_CH_SRC_DSCR_WORD2", .addr = A_ZDMA_CH_SRC_DSCR_WORD2, + .rsvd = 0xc0000000, + },{ .name = "ZDMA_CH_SRC_DSCR_WORD3", .addr = A_ZDMA_CH_SRC_DSCR_WORD3, + .rsvd = 0xffffffe0, + },{ .name = "ZDMA_CH_DST_DSCR_WORD0", .addr = A_ZDMA_CH_DST_DSCR_WORD0, + },{ .name = "ZDMA_CH_DST_DSCR_WORD1", .addr = A_ZDMA_CH_DST_DSCR_WORD1, + .rsvd = 0xfffe0000, + },{ .name = "ZDMA_CH_DST_DSCR_WORD2", .addr = A_ZDMA_CH_DST_DSCR_WORD2, + .rsvd = 0xc0000000, + },{ .name = "ZDMA_CH_DST_DSCR_WORD3", .addr = A_ZDMA_CH_DST_DSCR_WORD3, + .rsvd = 0xfffffffa, + },{ .name = "ZDMA_CH_WR_ONLY_WORD0", .addr = A_ZDMA_CH_WR_ONLY_WORD0, + },{ .name = "ZDMA_CH_WR_ONLY_WORD1", .addr = A_ZDMA_CH_WR_ONLY_WORD1, + },{ .name = "ZDMA_CH_WR_ONLY_WORD2", .addr = A_ZDMA_CH_WR_ONLY_WORD2, + },{ .name = "ZDMA_CH_WR_ONLY_WORD3", .addr = A_ZDMA_CH_WR_ONLY_WORD3, + },{ .name = "ZDMA_CH_SRC_START_LSB", .addr = A_ZDMA_CH_SRC_START_LSB, + },{ .name = "ZDMA_CH_SRC_START_MSB", .addr = A_ZDMA_CH_SRC_START_MSB, + .rsvd = 0xfffe0000, + },{ .name = "ZDMA_CH_DST_START_LSB", .addr = A_ZDMA_CH_DST_START_LSB, + },{ .name = "ZDMA_CH_DST_START_MSB", .addr = A_ZDMA_CH_DST_START_MSB, + .rsvd = 0xfffe0000, + },{ .name = "ZDMA_CH_SRC_CUR_PYLD_LSB", .addr = A_ZDMA_CH_SRC_CUR_PYLD_LSB, + .ro = 0xffffffff, + },{ .name = "ZDMA_CH_SRC_CUR_PYLD_MSB", .addr = A_ZDMA_CH_SRC_CUR_PYLD_MSB, + .rsvd = 0xfffe0000, + .ro = 0x1ffff, + },{ .name = "ZDMA_CH_DST_CUR_PYLD_LSB", .addr = A_ZDMA_CH_DST_CUR_PYLD_LSB, + .ro = 0xffffffff, + },{ .name = "ZDMA_CH_DST_CUR_PYLD_MSB", .addr = A_ZDMA_CH_DST_CUR_PYLD_MSB, + .rsvd = 0xfffe0000, + .ro = 0x1ffff, + },{ .name = "ZDMA_CH_SRC_CUR_DSCR_LSB", .addr = A_ZDMA_CH_SRC_CUR_DSCR_LSB, + .ro = 0xffffffff, + },{ .name = "ZDMA_CH_SRC_CUR_DSCR_MSB", .addr = A_ZDMA_CH_SRC_CUR_DSCR_MSB, + .rsvd = 0xfffe0000, + .ro = 0x1ffff, + },{ .name = "ZDMA_CH_DST_CUR_DSCR_LSB", .addr = A_ZDMA_CH_DST_CUR_DSCR_LSB, + .ro = 0xffffffff, + },{ .name = "ZDMA_CH_DST_CUR_DSCR_MSB", .addr = A_ZDMA_CH_DST_CUR_DSCR_MSB, + .rsvd = 0xfffe0000, + .ro = 0x1ffff, + },{ .name = "ZDMA_CH_TOTAL_BYTE", .addr = A_ZDMA_CH_TOTAL_BYTE, + .w1c = 0xffffffff, + },{ .name = "ZDMA_CH_RATE_CNTL", .addr = A_ZDMA_CH_RATE_CNTL, + .rsvd = 0xfffff000, + },{ .name = "ZDMA_CH_IRQ_SRC_ACCT", .addr = A_ZDMA_CH_IRQ_SRC_ACCT, + .rsvd = 0xffffff00, + .ro = 0xff, + .cor = 0xff, + },{ .name = "ZDMA_CH_IRQ_DST_ACCT", .addr = A_ZDMA_CH_IRQ_DST_ACCT, + .rsvd = 0xffffff00, + .ro = 0xff, + .cor = 0xff, + },{ .name = "ZDMA_CH_DBG0", .addr = A_ZDMA_CH_DBG0, + .rsvd = 0xfffffe00, + .ro = 0x1ff, + },{ .name = "ZDMA_CH_DBG1", .addr = A_ZDMA_CH_DBG1, + .rsvd = 0xfffffe00, + .ro = 0x1ff, + },{ .name = "ZDMA_CH_CTRL2", .addr = A_ZDMA_CH_CTRL2, + .rsvd = 0xfffffffe, + .post_write = zdma_ch_ctrlx_postw, + } +}; + +static void zdma_reset(DeviceState *dev) +{ + XlnxZDMA *s = XLNX_ZDMA(dev); + unsigned int i; + + for (i = 0; i < ARRAY_SIZE(s->regs_info); ++i) { + register_reset(&s->regs_info[i]); + } + + zdma_ch_imr_update_irq(s); +} + +static uint64_t zdma_read(void *opaque, hwaddr addr, unsigned size) +{ + XlnxZDMA *s = XLNX_ZDMA(opaque); + RegisterInfo *r = &s->regs_info[addr / 4]; + + if (!r->data) { + qemu_log("%s: Decode error: read from %" HWADDR_PRIx "\n", + object_get_canonical_path(OBJECT(s)), + addr); + ARRAY_FIELD_DP32(s->regs, ZDMA_CH_ISR, INV_APB, true); + zdma_ch_imr_update_irq(s); + return 0; + } + return register_read(r, ~0, NULL, false); +} + +static void zdma_write(void *opaque, hwaddr addr, uint64_t value, + unsigned size) +{ + XlnxZDMA *s = XLNX_ZDMA(opaque); + RegisterInfo *r = &s->regs_info[addr / 4]; + + if (!r->data) { + qemu_log("%s: Decode error: write to %" HWADDR_PRIx "=%" PRIx64 "\n", + object_get_canonical_path(OBJECT(s)), + addr, value); + ARRAY_FIELD_DP32(s->regs, ZDMA_CH_ISR, INV_APB, true); + zdma_ch_imr_update_irq(s); + return; + } + register_write(r, value, ~0, NULL, false); +} + +static const MemoryRegionOps zdma_ops = { + .read = zdma_read, + .write = zdma_write, + .endianness = DEVICE_LITTLE_ENDIAN, + .valid = { + .min_access_size = 4, + .max_access_size = 4, + }, +}; + +static void zdma_realize(DeviceState *dev, Error **errp) +{ + XlnxZDMA *s = XLNX_ZDMA(dev); + unsigned int i; + + for (i = 0; i < ARRAY_SIZE(zdma_regs_info); ++i) { + RegisterInfo *r = &s->regs_info[zdma_regs_info[i].addr / 4]; + + *r = (RegisterInfo) { + .data = (uint8_t *)&s->regs[ + zdma_regs_info[i].addr / 4], + .data_size = sizeof(uint32_t), + .access = &zdma_regs_info[i], + .opaque = s, + }; + } + + if (s->dma_mr) { + s->dma_as = g_malloc0(sizeof(AddressSpace)); + address_space_init(s->dma_as, s->dma_mr, NULL); + } else { + s->dma_as = &address_space_memory; + } + s->attr = MEMTXATTRS_UNSPECIFIED; +} + +static void zdma_init(Object *obj) +{ + XlnxZDMA *s = XLNX_ZDMA(obj); + SysBusDevice *sbd = SYS_BUS_DEVICE(obj); + + memory_region_init_io(&s->iomem, obj, &zdma_ops, s, + TYPE_XLNX_ZDMA, ZDMA_R_MAX * 4); + sysbus_init_mmio(sbd, &s->iomem); + sysbus_init_irq(sbd, &s->irq_zdma_ch_imr); + + object_property_add_link(obj, "dma", TYPE_MEMORY_REGION, + (Object **)&s->dma_mr, + qdev_prop_allow_set_link_before_realize, + OBJ_PROP_LINK_UNREF_ON_RELEASE, + &error_abort); +} + +static const VMStateDescription vmstate_zdma = { + .name = TYPE_XLNX_ZDMA, + .version_id = 1, + .minimum_version_id = 1, + .minimum_version_id_old = 1, + .fields = (VMStateField[]) { + VMSTATE_UINT32_ARRAY(regs, XlnxZDMA, ZDMA_R_MAX), + VMSTATE_UINT32(state, XlnxZDMA), + VMSTATE_UINT32_ARRAY(dsc_src.words, XlnxZDMA, 4), + VMSTATE_UINT32_ARRAY(dsc_dst.words, XlnxZDMA, 4), + VMSTATE_END_OF_LIST(), + } +}; + +static Property zdma_props[] = { + DEFINE_PROP_UINT32("bus-width", XlnxZDMA, cfg.bus_width, 64), + DEFINE_PROP_END_OF_LIST(), +}; + +static void zdma_class_init(ObjectClass *klass, void *data) +{ + DeviceClass *dc = DEVICE_CLASS(klass); + + dc->reset = zdma_reset; + dc->realize = zdma_realize; + dc->props = zdma_props; + dc->vmsd = &vmstate_zdma; +} + +static const TypeInfo zdma_info = { + .name = TYPE_XLNX_ZDMA, + .parent = TYPE_SYS_BUS_DEVICE, + .instance_size = sizeof(XlnxZDMA), + .class_init = zdma_class_init, + .instance_init = zdma_init, +}; + +static void zdma_register_types(void) +{ + type_register_static(&zdma_info); +} + +type_init(zdma_register_types) -- 2.17.0 From: Francisco Iglesias <frasse.iglesias@gmail.com> The ZynqMP contains two instances of a generic DMA, the GDMA, located in the FPD (full power domain), and the ADMA, located in LPD (low power domain). This patch adds these two DMAs to the ZynqMP board. Signed-off-by: Francisco Iglesias <frasse.iglesias@gmail.com> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> Reviewed-by: Edgar E. Iglesias <edgar.iglesias@xilinx.com> Message-id: 20180503214201.29082-3-frasse.iglesias@gmail.com Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- include/hw/arm/xlnx-zynqmp.h | 5 ++++ hw/arm/xlnx-zynqmp.c | 53 ++++++++++++++++++++++++++++++++++++ 2 files changed, 58 insertions(+) diff --git a/include/hw/arm/xlnx-zynqmp.h b/include/hw/arm/xlnx-zynqmp.h index XXXXXXX..XXXXXXX 100644 --- a/include/hw/arm/xlnx-zynqmp.h +++ b/include/hw/arm/xlnx-zynqmp.h @@ -XXX,XX +XXX,XX @@ #include "hw/sd/sdhci.h" #include "hw/ssi/xilinx_spips.h" #include "hw/dma/xlnx_dpdma.h" +#include "hw/dma/xlnx-zdma.h" #include "hw/display/xlnx_dp.h" #include "hw/intc/xlnx-zynqmp-ipi.h" #include "hw/timer/xlnx-zynqmp-rtc.h" @@ -XXX,XX +XXX,XX @@ #define XLNX_ZYNQMP_NUM_UARTS 2 #define XLNX_ZYNQMP_NUM_SDHCI 2 #define XLNX_ZYNQMP_NUM_SPIS 2 +#define XLNX_ZYNQMP_NUM_GDMA_CH 8 +#define XLNX_ZYNQMP_NUM_ADMA_CH 8 #define XLNX_ZYNQMP_NUM_QSPI_BUS 2 #define XLNX_ZYNQMP_NUM_QSPI_BUS_CS 2 @@ -XXX,XX +XXX,XX @@ typedef struct XlnxZynqMPState { XlnxDPDMAState dpdma; XlnxZynqMPIPI ipi; XlnxZynqMPRTC rtc; + XlnxZDMA gdma[XLNX_ZYNQMP_NUM_GDMA_CH]; + XlnxZDMA adma[XLNX_ZYNQMP_NUM_ADMA_CH]; char *boot_cpu; ARMCPU *boot_cpu_ptr; diff --git a/hw/arm/xlnx-zynqmp.c b/hw/arm/xlnx-zynqmp.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/xlnx-zynqmp.c +++ b/hw/arm/xlnx-zynqmp.c @@ -XXX,XX +XXX,XX @@ static const int spi_intr[XLNX_ZYNQMP_NUM_SPIS] = { 19, 20, }; +static const uint64_t gdma_ch_addr[XLNX_ZYNQMP_NUM_GDMA_CH] = { + 0xFD500000, 0xFD510000, 0xFD520000, 0xFD530000, + 0xFD540000, 0xFD550000, 0xFD560000, 0xFD570000 +}; + +static const int gdma_ch_intr[XLNX_ZYNQMP_NUM_GDMA_CH] = { + 124, 125, 126, 127, 128, 129, 130, 131 +}; + +static const uint64_t adma_ch_addr[XLNX_ZYNQMP_NUM_ADMA_CH] = { + 0xFFA80000, 0xFFA90000, 0xFFAA0000, 0xFFAB0000, + 0xFFAC0000, 0xFFAD0000, 0xFFAE0000, 0xFFAF0000 +}; + +static const int adma_ch_intr[XLNX_ZYNQMP_NUM_ADMA_CH] = { + 77, 78, 79, 80, 81, 82, 83, 84 +}; + typedef struct XlnxZynqMPGICRegion { int region_index; uint32_t address; @@ -XXX,XX +XXX,XX @@ static void xlnx_zynqmp_init(Object *obj) object_initialize(&s->rtc, sizeof(s->rtc), TYPE_XLNX_ZYNQMP_RTC); qdev_set_parent_bus(DEVICE(&s->rtc), sysbus_get_default()); + + for (i = 0; i < XLNX_ZYNQMP_NUM_GDMA_CH; i++) { + object_initialize(&s->gdma[i], sizeof(s->gdma[i]), TYPE_XLNX_ZDMA); + qdev_set_parent_bus(DEVICE(&s->gdma[i]), sysbus_get_default()); + } + + for (i = 0; i < XLNX_ZYNQMP_NUM_ADMA_CH; i++) { + object_initialize(&s->adma[i], sizeof(s->adma[i]), TYPE_XLNX_ZDMA); + qdev_set_parent_bus(DEVICE(&s->adma[i]), sysbus_get_default()); + } } static void xlnx_zynqmp_realize(DeviceState *dev, Error **errp) @@ -XXX,XX +XXX,XX @@ static void xlnx_zynqmp_realize(DeviceState *dev, Error **errp) } sysbus_mmio_map(SYS_BUS_DEVICE(&s->rtc), 0, RTC_ADDR); sysbus_connect_irq(SYS_BUS_DEVICE(&s->rtc), 0, gic_spi[RTC_IRQ]); + + for (i = 0; i < XLNX_ZYNQMP_NUM_GDMA_CH; i++) { + object_property_set_uint(OBJECT(&s->gdma[i]), 128, "bus-width", &err); + object_property_set_bool(OBJECT(&s->gdma[i]), true, "realized", &err); + if (err) { + error_propagate(errp, err); + return; + } + + sysbus_mmio_map(SYS_BUS_DEVICE(&s->gdma[i]), 0, gdma_ch_addr[i]); + sysbus_connect_irq(SYS_BUS_DEVICE(&s->gdma[i]), 0, + gic_spi[gdma_ch_intr[i]]); + } + + for (i = 0; i < XLNX_ZYNQMP_NUM_ADMA_CH; i++) { + object_property_set_bool(OBJECT(&s->adma[i]), true, "realized", &err); + if (err) { + error_propagate(errp, err); + return; + } + + sysbus_mmio_map(SYS_BUS_DEVICE(&s->adma[i]), 0, adma_ch_addr[i]); + sysbus_connect_irq(SYS_BUS_DEVICE(&s->adma[i]), 0, + gic_spi[adma_ch_intr[i]]); + } } static Property xlnx_zynqmp_props[] = { -- 2.17.0 From: Eric Auger <eric.auger@redhat.com> Coverity complains about use of uninitialized Evt struct. The EVT_SET_TYPE and similar setters use deposit32() on fields in the struct, so they read the uninitialized existing values. In cases where we don't set all the fields in the event struct we'll end up leaking random uninitialized data from QEMU's stack into the guest. Initializing the struct with "Evt evt = {};" ought to satisfy Coverity and fix the data leak. Signed-off-by: Eric Auger <eric.auger@redhat.com> Reported-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Message-id: 1526493784-25328-2-git-send-email-eric.auger@redhat.com Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- hw/arm/smmuv3.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/hw/arm/smmuv3.c b/hw/arm/smmuv3.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/smmuv3.c +++ b/hw/arm/smmuv3.c @@ -XXX,XX +XXX,XX @@ static MemTxResult smmuv3_write_eventq(SMMUv3State *s, Evt *evt) void smmuv3_record_event(SMMUv3State *s, SMMUEventInfo *info) { - Evt evt; + Evt evt = {}; MemTxResult r; if (!smmuv3_eventq_enabled(s)) { -- 2.17.0 From: Eric Auger <eric.auger@redhat.com> Coverity points out that this can overflow if n > 31, because it's only doing 32-bit arithmetic. Let's use 1ULL instead of 1. Also the formulae used to compute n can be replaced by the level_shift() macro. Reported-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Eric Auger <eric.auger@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Message-id: 1526493784-25328-3-git-send-email-eric.auger@redhat.com Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- hw/arm/smmu-common.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/hw/arm/smmu-common.c b/hw/arm/smmu-common.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/smmu-common.c +++ b/hw/arm/smmu-common.c @@ -XXX,XX +XXX,XX @@ static inline hwaddr get_table_pte_address(uint64_t pte, int granule_sz) static inline hwaddr get_block_pte_address(uint64_t pte, int level, int granule_sz, uint64_t *bsz) { - int n = (granule_sz - 3) * (4 - level) + 3; + int n = level_shift(level, granule_sz); - *bsz = 1 << n; + *bsz = 1ULL << n; return PTE_ADDRESS(pte, n); } -- 2.17.0 From: Richard Henderson <richard.henderson@linaro.org> Move some stuff that will be common to both translate-a64.c and translate-sve.c. Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20180516223007.10256-2-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-a64.h | 118 +++++++++++++++++++++++++++++++++++++ target/arm/translate-a64.c | 112 +++++------------------------------ 2 files changed, 133 insertions(+), 97 deletions(-) create mode 100644 target/arm/translate-a64.h diff --git a/target/arm/translate-a64.h b/target/arm/translate-a64.h new file mode 100644 index XXXXXXX..XXXXXXX --- /dev/null +++ b/target/arm/translate-a64.h @@ -XXX,XX +XXX,XX @@ +/* + * AArch64 translation, common definitions. + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library; if not, see <http://www.gnu.org/licenses/>. + */ + +#ifndef TARGET_ARM_TRANSLATE_A64_H +#define TARGET_ARM_TRANSLATE_A64_H + +void unallocated_encoding(DisasContext *s); + +#define unsupported_encoding(s, insn) \ + do { \ + qemu_log_mask(LOG_UNIMP, \ + "%s:%d: unsupported instruction encoding 0x%08x " \ + "at pc=%016" PRIx64 "\n", \ + __FILE__, __LINE__, insn, s->pc - 4); \ + unallocated_encoding(s); \ + } while (0) + +TCGv_i64 new_tmp_a64(DisasContext *s); +TCGv_i64 new_tmp_a64_zero(DisasContext *s); +TCGv_i64 cpu_reg(DisasContext *s, int reg); +TCGv_i64 cpu_reg_sp(DisasContext *s, int reg); +TCGv_i64 read_cpu_reg(DisasContext *s, int reg, int sf); +TCGv_i64 read_cpu_reg_sp(DisasContext *s, int reg, int sf); +void write_fp_dreg(DisasContext *s, int reg, TCGv_i64 v); +TCGv_ptr get_fpstatus_ptr(bool); +bool logic_imm_decode_wmask(uint64_t *result, unsigned int immn, + unsigned int imms, unsigned int immr); +uint64_t vfp_expand_imm(int size, uint8_t imm8); +bool sve_access_check(DisasContext *s); + +/* We should have at some point before trying to access an FP register + * done the necessary access check, so assert that + * (a) we did the check and + * (b) we didn't then just plough ahead anyway if it failed. + * Print the instruction pattern in the abort message so we can figure + * out what we need to fix if a user encounters this problem in the wild. + */ +static inline void assert_fp_access_checked(DisasContext *s) +{ +#ifdef CONFIG_DEBUG_TCG + if (unlikely(!s->fp_access_checked || s->fp_excp_el)) { + fprintf(stderr, "target-arm: FP access check missing for " + "instruction 0x%08x\n", s->insn); + abort(); + } +#endif +} + +/* Return the offset into CPUARMState of an element of specified + * size, 'element' places in from the least significant end of + * the FP/vector register Qn. + */ +static inline int vec_reg_offset(DisasContext *s, int regno, + int element, TCGMemOp size) +{ + int offs = 0; +#ifdef HOST_WORDS_BIGENDIAN + /* This is complicated slightly because vfp.zregs[n].d[0] is + * still the low half and vfp.zregs[n].d[1] the high half + * of the 128 bit vector, even on big endian systems. + * Calculate the offset assuming a fully bigendian 128 bits, + * then XOR to account for the order of the two 64 bit halves. + */ + offs += (16 - ((element + 1) * (1 << size))); + offs ^= 8; +#else + offs += element * (1 << size); +#endif + offs += offsetof(CPUARMState, vfp.zregs[regno]); + assert_fp_access_checked(s); + return offs; +} + +/* Return the offset info CPUARMState of the "whole" vector register Qn. */ +static inline int vec_full_reg_offset(DisasContext *s, int regno) +{ + assert_fp_access_checked(s); + return offsetof(CPUARMState, vfp.zregs[regno]); +} + +/* Return a newly allocated pointer to the vector register. */ +static inline TCGv_ptr vec_full_reg_ptr(DisasContext *s, int regno) +{ + TCGv_ptr ret = tcg_temp_new_ptr(); + tcg_gen_addi_ptr(ret, cpu_env, vec_full_reg_offset(s, regno)); + return ret; +} + +/* Return the byte size of the "whole" vector register, VL / 8. */ +static inline int vec_full_reg_size(DisasContext *s) +{ + return s->sve_len; +} + +bool disas_sve(DisasContext *, uint32_t); + +/* Note that the gvec expanders operate on offsets + sizes. */ +typedef void GVecGen2Fn(unsigned, uint32_t, uint32_t, uint32_t, uint32_t); +typedef void GVecGen2iFn(unsigned, uint32_t, uint32_t, int64_t, + uint32_t, uint32_t); +typedef void GVecGen3Fn(unsigned, uint32_t, uint32_t, + uint32_t, uint32_t, uint32_t); + +#endif /* TARGET_ARM_TRANSLATE_A64_H */ diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-a64.c +++ b/target/arm/translate-a64.c @@ -XXX,XX +XXX,XX @@ #include "exec/log.h" #include "trace-tcg.h" +#include "translate-a64.h" static TCGv_i64 cpu_X[32]; static TCGv_i64 cpu_pc; /* Load/store exclusive handling */ static TCGv_i64 cpu_exclusive_high; -static TCGv_i64 cpu_reg(DisasContext *s, int reg); static const char *regnames[] = { "x0", "x1", "x2", "x3", "x4", "x5", "x6", "x7", @@ -XXX,XX +XXX,XX @@ typedef void CryptoThreeOpIntFn(TCGv_ptr, TCGv_ptr, TCGv_i32); typedef void CryptoThreeOpFn(TCGv_ptr, TCGv_ptr, TCGv_ptr); typedef void AtomicThreeOpFn(TCGv_i64, TCGv_i64, TCGv_i64, TCGArg, TCGMemOp); -/* Note that the gvec expanders operate on offsets + sizes. */ -typedef void GVecGen2Fn(unsigned, uint32_t, uint32_t, uint32_t, uint32_t); -typedef void GVecGen2iFn(unsigned, uint32_t, uint32_t, int64_t, - uint32_t, uint32_t); -typedef void GVecGen3Fn(unsigned, uint32_t, uint32_t, - uint32_t, uint32_t, uint32_t); - /* initialize TCG globals. */ void a64_translate_init(void) { @@ -XXX,XX +XXX,XX @@ static inline void gen_goto_tb(DisasContext *s, int n, uint64_t dest) } } -static void unallocated_encoding(DisasContext *s) +void unallocated_encoding(DisasContext *s) { /* Unallocated and reserved encodings are uncategorized */ gen_exception_insn(s, 4, EXCP_UDEF, syn_uncategorized(), default_exception_el(s)); } -#define unsupported_encoding(s, insn) \ - do { \ - qemu_log_mask(LOG_UNIMP, \ - "%s:%d: unsupported instruction encoding 0x%08x " \ - "at pc=%016" PRIx64 "\n", \ - __FILE__, __LINE__, insn, s->pc - 4); \ - unallocated_encoding(s); \ - } while (0) - static void init_tmp_a64_array(DisasContext *s) { #ifdef CONFIG_DEBUG_TCG @@ -XXX,XX +XXX,XX @@ static void free_tmp_a64(DisasContext *s) init_tmp_a64_array(s); } -static TCGv_i64 new_tmp_a64(DisasContext *s) +TCGv_i64 new_tmp_a64(DisasContext *s) { assert(s->tmp_a64_count < TMP_A64_MAX); return s->tmp_a64[s->tmp_a64_count++] = tcg_temp_new_i64(); } -static TCGv_i64 new_tmp_a64_zero(DisasContext *s) +TCGv_i64 new_tmp_a64_zero(DisasContext *s) { TCGv_i64 t = new_tmp_a64(s); tcg_gen_movi_i64(t, 0); @@ -XXX,XX +XXX,XX @@ static TCGv_i64 new_tmp_a64_zero(DisasContext *s) * to cpu_X[31] and ZR accesses to a temporary which can be discarded. * This is the point of the _sp forms. */ -static TCGv_i64 cpu_reg(DisasContext *s, int reg) +TCGv_i64 cpu_reg(DisasContext *s, int reg) { if (reg == 31) { return new_tmp_a64_zero(s); @@ -XXX,XX +XXX,XX @@ static TCGv_i64 cpu_reg(DisasContext *s, int reg) } /* register access for when 31 == SP */ -static TCGv_i64 cpu_reg_sp(DisasContext *s, int reg) +TCGv_i64 cpu_reg_sp(DisasContext *s, int reg) { return cpu_X[reg]; } @@ -XXX,XX +XXX,XX @@ static TCGv_i64 cpu_reg_sp(DisasContext *s, int reg) * representing the register contents. This TCGv is an auto-freed * temporary so it need not be explicitly freed, and may be modified. */ -static TCGv_i64 read_cpu_reg(DisasContext *s, int reg, int sf) +TCGv_i64 read_cpu_reg(DisasContext *s, int reg, int sf) { TCGv_i64 v = new_tmp_a64(s); if (reg != 31) { @@ -XXX,XX +XXX,XX @@ static TCGv_i64 read_cpu_reg(DisasContext *s, int reg, int sf) return v; } -static TCGv_i64 read_cpu_reg_sp(DisasContext *s, int reg, int sf) +TCGv_i64 read_cpu_reg_sp(DisasContext *s, int reg, int sf) { TCGv_i64 v = new_tmp_a64(s); if (sf) { @@ -XXX,XX +XXX,XX @@ static TCGv_i64 read_cpu_reg_sp(DisasContext *s, int reg, int sf) return v; } -/* We should have at some point before trying to access an FP register - * done the necessary access check, so assert that - * (a) we did the check and - * (b) we didn't then just plough ahead anyway if it failed. - * Print the instruction pattern in the abort message so we can figure - * out what we need to fix if a user encounters this problem in the wild. - */ -static inline void assert_fp_access_checked(DisasContext *s) -{ -#ifdef CONFIG_DEBUG_TCG - if (unlikely(!s->fp_access_checked || s->fp_excp_el)) { - fprintf(stderr, "target-arm: FP access check missing for " - "instruction 0x%08x\n", s->insn); - abort(); - } -#endif -} - -/* Return the offset into CPUARMState of an element of specified - * size, 'element' places in from the least significant end of - * the FP/vector register Qn. - */ -static inline int vec_reg_offset(DisasContext *s, int regno, - int element, TCGMemOp size) -{ - int offs = 0; -#ifdef HOST_WORDS_BIGENDIAN - /* This is complicated slightly because vfp.zregs[n].d[0] is - * still the low half and vfp.zregs[n].d[1] the high half - * of the 128 bit vector, even on big endian systems. - * Calculate the offset assuming a fully bigendian 128 bits, - * then XOR to account for the order of the two 64 bit halves. - */ - offs += (16 - ((element + 1) * (1 << size))); - offs ^= 8; -#else - offs += element * (1 << size); -#endif - offs += offsetof(CPUARMState, vfp.zregs[regno]); - assert_fp_access_checked(s); - return offs; -} - -/* Return the offset info CPUARMState of the "whole" vector register Qn. */ -static inline int vec_full_reg_offset(DisasContext *s, int regno) -{ - assert_fp_access_checked(s); - return offsetof(CPUARMState, vfp.zregs[regno]); -} - -/* Return a newly allocated pointer to the vector register. */ -static TCGv_ptr vec_full_reg_ptr(DisasContext *s, int regno) -{ - TCGv_ptr ret = tcg_temp_new_ptr(); - tcg_gen_addi_ptr(ret, cpu_env, vec_full_reg_offset(s, regno)); - return ret; -} - -/* Return the byte size of the "whole" vector register, VL / 8. */ -static inline int vec_full_reg_size(DisasContext *s) -{ - /* FIXME SVE: We should put the composite ZCR_EL* value into tb->flags. - In the meantime this is just the AdvSIMD length of 128. */ - return 128 / 8; -} - /* Return the offset into CPUARMState of a slice (from * the least significant end) of FP register Qn (ie * Dn, Sn, Hn or Bn). @@ -XXX,XX +XXX,XX @@ static void clear_vec_high(DisasContext *s, bool is_q, int rd) } } -static void write_fp_dreg(DisasContext *s, int reg, TCGv_i64 v) +void write_fp_dreg(DisasContext *s, int reg, TCGv_i64 v) { unsigned ofs = fp_reg_offset(s, reg, MO_64); @@ -XXX,XX +XXX,XX @@ static void write_fp_sreg(DisasContext *s, int reg, TCGv_i32 v) tcg_temp_free_i64(tmp); } -static TCGv_ptr get_fpstatus_ptr(bool is_f16) +TCGv_ptr get_fpstatus_ptr(bool is_f16) { TCGv_ptr statusptr = tcg_temp_new_ptr(); int offset; @@ -XXX,XX +XXX,XX @@ static inline bool fp_access_check(DisasContext *s) /* Check that SVE access is enabled. If it is, return true. * If not, emit code to generate an appropriate exception and return false. */ -static inline bool sve_access_check(DisasContext *s) +bool sve_access_check(DisasContext *s) { if (s->sve_excp_el) { gen_exception_insn(s, 4, EXCP_UDEF, syn_sve_access_trap(), s->sve_excp_el); return false; } - return true; + return fp_access_check(s); } /* @@ -XXX,XX +XXX,XX @@ static inline uint64_t bitmask64(unsigned int length) * value (ie should cause a guest UNDEF exception), and true if they are * valid, in which case the decoded bit pattern is written to result. */ -static bool logic_imm_decode_wmask(uint64_t *result, unsigned int immn, - unsigned int imms, unsigned int immr) +bool logic_imm_decode_wmask(uint64_t *result, unsigned int immn, + unsigned int imms, unsigned int immr) { uint64_t mask; unsigned e, levels, s, r; @@ -XXX,XX +XXX,XX @@ static void disas_fp_3src(DisasContext *s, uint32_t insn) * the range 01....1xx to 10....0xx, and the most significant 4 bits of * the mantissa; see VFPExpandImm() in the v8 ARM ARM. */ -static uint64_t vfp_expand_imm(int size, uint8_t imm8) +uint64_t vfp_expand_imm(int size, uint8_t imm8) { uint64_t imm; -- 2.17.0 From: Richard Henderson <richard.henderson@linaro.org> Including only 4, as-yet unimplemented, instruction patterns so that the whole thing compiles. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20180516223007.10256-3-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/Makefile.objs | 10 ++++++ target/arm/translate-a64.c | 7 ++++- target/arm/translate-sve.c | 63 ++++++++++++++++++++++++++++++++++++++ .gitignore | 1 + target/arm/sve.decode | 45 +++++++++++++++++++++++++++ 5 files changed, 125 insertions(+), 1 deletion(-) create mode 100644 target/arm/translate-sve.c create mode 100644 target/arm/sve.decode diff --git a/target/arm/Makefile.objs b/target/arm/Makefile.objs index XXXXXXX..XXXXXXX 100644 --- a/target/arm/Makefile.objs +++ b/target/arm/Makefile.objs @@ -XXX,XX +XXX,XX @@ obj-y += gdbstub.o obj-$(TARGET_AARCH64) += cpu64.o translate-a64.o helper-a64.o gdbstub64.o obj-y += crypto_helper.o obj-$(CONFIG_SOFTMMU) += arm-powerctl.o + +DECODETREE = $(SRC_PATH)/scripts/decodetree.py + +target/arm/decode-sve.inc.c: $(SRC_PATH)/target/arm/sve.decode $(DECODETREE) + $(call quiet-command,\ + $(PYTHON) $(DECODETREE) --decode disas_sve -o $@ $<,\ + "GEN", $(TARGET_DIR)$@) + +target/arm/translate-sve.o: target/arm/decode-sve.inc.c +obj-$(TARGET_AARCH64) += translate-sve.o diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-a64.c +++ b/target/arm/translate-a64.c @@ -XXX,XX +XXX,XX @@ static void disas_a64_insn(CPUARMState *env, DisasContext *s) s->fp_access_checked = false; switch (extract32(insn, 25, 4)) { - case 0x0: case 0x1: case 0x2: case 0x3: /* UNALLOCATED */ + case 0x0: case 0x1: case 0x3: /* UNALLOCATED */ unallocated_encoding(s); break; + case 0x2: + if (!arm_dc_feature(s, ARM_FEATURE_SVE) || !disas_sve(s, insn)) { + unallocated_encoding(s); + } + break; case 0x8: case 0x9: /* Data processing - immediate */ disas_data_proc_imm(s, insn); break; diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c new file mode 100644 index XXXXXXX..XXXXXXX --- /dev/null +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ +/* + * AArch64 SVE translation + * + * Copyright (c) 2018 Linaro, Ltd + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library; if not, see <http://www.gnu.org/licenses/>. + */ + +#include "qemu/osdep.h" +#include "cpu.h" +#include "exec/exec-all.h" +#include "tcg-op.h" +#include "tcg-op-gvec.h" +#include "qemu/log.h" +#include "arm_ldst.h" +#include "translate.h" +#include "internals.h" +#include "exec/helper-proto.h" +#include "exec/helper-gen.h" +#include "exec/log.h" +#include "trace-tcg.h" +#include "translate-a64.h" + +/* + * Include the generated decoder. + */ + +#include "decode-sve.inc.c" + +/* + * Implement all of the translator functions referenced by the decoder. + */ + +static bool trans_AND_zzz(DisasContext *s, arg_AND_zzz *a, uint32_t insn) +{ + return false; +} + +static bool trans_ORR_zzz(DisasContext *s, arg_ORR_zzz *a, uint32_t insn) +{ + return false; +} + +static bool trans_EOR_zzz(DisasContext *s, arg_EOR_zzz *a, uint32_t insn) +{ + return false; +} + +static bool trans_BIC_zzz(DisasContext *s, arg_BIC_zzz *a, uint32_t insn) +{ + return false; +} diff --git a/.gitignore b/.gitignore index XXXXXXX..XXXXXXX 100644 --- a/.gitignore +++ b/.gitignore @@ -XXX,XX +XXX,XX @@ trace-dtrace-root.h trace-dtrace-root.dtrace trace-ust-all.h trace-ust-all.c +/target/arm/decode-sve.inc.c diff --git a/target/arm/sve.decode b/target/arm/sve.decode new file mode 100644 index XXXXXXX..XXXXXXX --- /dev/null +++ b/target/arm/sve.decode @@ -XXX,XX +XXX,XX @@ +# AArch64 SVE instruction descriptions +# +# Copyright (c) 2017 Linaro, Ltd +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library; if not, see <http://www.gnu.org/licenses/>. + +# +# This file is processed by scripts/decodetree.py +# + +########################################################################### +# Named attribute sets. These are used to make nice(er) names +# when creating helpers common to those for the individual +# instruction patterns. + +&rrr_esz rd rn rm esz + +########################################################################### +# Named instruction formats. These are generally used to +# reduce the amount of duplication between instruction patterns. + +# Three operand with unused vector element size +@rd_rn_rm_e0 ........ ... rm:5 ... ... rn:5 rd:5 &rrr_esz esz=0 + +########################################################################### +# Instruction patterns. Grouped according to the SVE encodingindex.xhtml. + +### SVE Logical - Unpredicated Group + +# SVE bitwise logical operations (unpredicated) +AND_zzz 00000100 00 1 ..... 001 100 ..... ..... @rd_rn_rm_e0 +ORR_zzz 00000100 01 1 ..... 001 100 ..... ..... @rd_rn_rm_e0 +EOR_zzz 00000100 10 1 ..... 001 100 ..... ..... @rd_rn_rm_e0 +BIC_zzz 00000100 11 1 ..... 001 100 ..... ..... @rd_rn_rm_e0 -- 2.17.0 From: Richard Henderson <richard.henderson@linaro.org> These were the instructions that were stubbed out when introducing the decode skeleton. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20180516223007.10256-4-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 55 ++++++++++++++++++++++++++++++++------ 1 file changed, 47 insertions(+), 8 deletions(-) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ * Implement all of the translator functions referenced by the decoder. */ -static bool trans_AND_zzz(DisasContext *s, arg_AND_zzz *a, uint32_t insn) +/* Invoke a vector expander on two Zregs. */ +static bool do_vector2_z(DisasContext *s, GVecGen2Fn *gvec_fn, + int esz, int rd, int rn) { - return false; + if (sve_access_check(s)) { + unsigned vsz = vec_full_reg_size(s); + gvec_fn(esz, vec_full_reg_offset(s, rd), + vec_full_reg_offset(s, rn), vsz, vsz); + } + return true; } -static bool trans_ORR_zzz(DisasContext *s, arg_ORR_zzz *a, uint32_t insn) +/* Invoke a vector expander on three Zregs. */ +static bool do_vector3_z(DisasContext *s, GVecGen3Fn *gvec_fn, + int esz, int rd, int rn, int rm) { - return false; + if (sve_access_check(s)) { + unsigned vsz = vec_full_reg_size(s); + gvec_fn(esz, vec_full_reg_offset(s, rd), + vec_full_reg_offset(s, rn), + vec_full_reg_offset(s, rm), vsz, vsz); + } + return true; } -static bool trans_EOR_zzz(DisasContext *s, arg_EOR_zzz *a, uint32_t insn) +/* Invoke a vector move on two Zregs. */ +static bool do_mov_z(DisasContext *s, int rd, int rn) { - return false; + return do_vector2_z(s, tcg_gen_gvec_mov, 0, rd, rn); } -static bool trans_BIC_zzz(DisasContext *s, arg_BIC_zzz *a, uint32_t insn) +/* + *** SVE Logical - Unpredicated Group + */ + +static bool trans_AND_zzz(DisasContext *s, arg_rrr_esz *a, uint32_t insn) { - return false; + return do_vector3_z(s, tcg_gen_gvec_and, 0, a->rd, a->rn, a->rm); +} + +static bool trans_ORR_zzz(DisasContext *s, arg_rrr_esz *a, uint32_t insn) +{ + if (a->rn == a->rm) { /* MOV */ + return do_mov_z(s, a->rd, a->rn); + } else { + return do_vector3_z(s, tcg_gen_gvec_or, 0, a->rd, a->rn, a->rm); + } +} + +static bool trans_EOR_zzz(DisasContext *s, arg_rrr_esz *a, uint32_t insn) +{ + return do_vector3_z(s, tcg_gen_gvec_xor, 0, a->rd, a->rn, a->rm); +} + +static bool trans_BIC_zzz(DisasContext *s, arg_rrr_esz *a, uint32_t insn) +{ + return do_vector3_z(s, tcg_gen_gvec_andc, 0, a->rd, a->rn, a->rm); } -- 2.17.0 From: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20180516223007.10256-5-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 127 +++++++++++++++++++++++++++++++++++++ target/arm/sve.decode | 20 ++++++ 2 files changed, 147 insertions(+) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ * Implement all of the translator functions referenced by the decoder. */ +/* Return the offset info CPUARMState of the predicate vector register Pn. + * Note for this purpose, FFR is P16. + */ +static inline int pred_full_reg_offset(DisasContext *s, int regno) +{ + return offsetof(CPUARMState, vfp.pregs[regno]); +} + +/* Return the byte size of the whole predicate register, VL / 64. */ +static inline int pred_full_reg_size(DisasContext *s) +{ + return s->sve_len >> 3; +} + /* Invoke a vector expander on two Zregs. */ static bool do_vector2_z(DisasContext *s, GVecGen2Fn *gvec_fn, int esz, int rd, int rn) @@ -XXX,XX +XXX,XX @@ static bool trans_BIC_zzz(DisasContext *s, arg_rrr_esz *a, uint32_t insn) { return do_vector3_z(s, tcg_gen_gvec_andc, 0, a->rd, a->rn, a->rm); } + +/* + *** SVE Memory - 32-bit Gather and Unsized Contiguous Group + */ + +/* Subroutine loading a vector register at VOFS of LEN bytes. + * The load should begin at the address Rn + IMM. + */ + +static void do_ldr(DisasContext *s, uint32_t vofs, uint32_t len, + int rn, int imm) +{ + uint32_t len_align = QEMU_ALIGN_DOWN(len, 8); + uint32_t len_remain = len % 8; + uint32_t nparts = len / 8 + ctpop8(len_remain); + int midx = get_mem_index(s); + TCGv_i64 addr, t0, t1; + + addr = tcg_temp_new_i64(); + t0 = tcg_temp_new_i64(); + + /* Note that unpredicated load/store of vector/predicate registers + * are defined as a stream of bytes, which equates to little-endian + * operations on larger quantities. There is no nice way to force + * a little-endian load for aarch64_be-linux-user out of line. + * + * Attempt to keep code expansion to a minimum by limiting the + * amount of unrolling done. + */ + if (nparts <= 4) { + int i; + + for (i = 0; i < len_align; i += 8) { + tcg_gen_addi_i64(addr, cpu_reg_sp(s, rn), imm + i); + tcg_gen_qemu_ld_i64(t0, addr, midx, MO_LEQ); + tcg_gen_st_i64(t0, cpu_env, vofs + i); + } + } else { + TCGLabel *loop = gen_new_label(); + TCGv_ptr tp, i = tcg_const_local_ptr(0); + + gen_set_label(loop); + + /* Minimize the number of local temps that must be re-read from + * the stack each iteration. Instead, re-compute values other + * than the loop counter. + */ + tp = tcg_temp_new_ptr(); + tcg_gen_addi_ptr(tp, i, imm); + tcg_gen_extu_ptr_i64(addr, tp); + tcg_gen_add_i64(addr, addr, cpu_reg_sp(s, rn)); + + tcg_gen_qemu_ld_i64(t0, addr, midx, MO_LEQ); + + tcg_gen_add_ptr(tp, cpu_env, i); + tcg_gen_addi_ptr(i, i, 8); + tcg_gen_st_i64(t0, tp, vofs); + tcg_temp_free_ptr(tp); + + tcg_gen_brcondi_ptr(TCG_COND_LTU, i, len_align, loop); + tcg_temp_free_ptr(i); + } + + /* Predicate register loads can be any multiple of 2. + * Note that we still store the entire 64-bit unit into cpu_env. + */ + if (len_remain) { + tcg_gen_addi_i64(addr, cpu_reg_sp(s, rn), imm + len_align); + + switch (len_remain) { + case 2: + case 4: + case 8: + tcg_gen_qemu_ld_i64(t0, addr, midx, MO_LE | ctz32(len_remain)); + break; + + case 6: + t1 = tcg_temp_new_i64(); + tcg_gen_qemu_ld_i64(t0, addr, midx, MO_LEUL); + tcg_gen_addi_i64(addr, addr, 4); + tcg_gen_qemu_ld_i64(t1, addr, midx, MO_LEUW); + tcg_gen_deposit_i64(t0, t0, t1, 32, 32); + tcg_temp_free_i64(t1); + break; + + default: + g_assert_not_reached(); + } + tcg_gen_st_i64(t0, cpu_env, vofs + len_align); + } + tcg_temp_free_i64(addr); + tcg_temp_free_i64(t0); +} + +static bool trans_LDR_zri(DisasContext *s, arg_rri *a, uint32_t insn) +{ + if (sve_access_check(s)) { + int size = vec_full_reg_size(s); + int off = vec_full_reg_offset(s, a->rd); + do_ldr(s, off, size, a->rn, a->imm * size); + } + return true; +} + +static bool trans_LDR_pri(DisasContext *s, arg_rri *a, uint32_t insn) +{ + if (sve_access_check(s)) { + int size = pred_full_reg_size(s); + int off = pred_full_reg_offset(s, a->rd); + do_ldr(s, off, size, a->rn, a->imm * size); + } + return true; +} diff --git a/target/arm/sve.decode b/target/arm/sve.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -XXX,XX +XXX,XX @@ # This file is processed by scripts/decodetree.py # +########################################################################### +# Named fields. These are primarily for disjoint fields. + +%imm9_16_10 16:s6 10:3 + ########################################################################### # Named attribute sets. These are used to make nice(er) names # when creating helpers common to those for the individual # instruction patterns. +&rri rd rn imm &rrr_esz rd rn rm esz ########################################################################### @@ -XXX,XX +XXX,XX @@ # Three operand with unused vector element size @rd_rn_rm_e0 ........ ... rm:5 ... ... rn:5 rd:5 &rrr_esz esz=0 +# Basic Load/Store with 9-bit immediate offset +@pd_rn_i9 ........ ........ ...... rn:5 . rd:4 \ + &rri imm=%imm9_16_10 +@rd_rn_i9 ........ ........ ...... rn:5 rd:5 \ + &rri imm=%imm9_16_10 + ########################################################################### # Instruction patterns. Grouped according to the SVE encodingindex.xhtml. @@ -XXX,XX +XXX,XX @@ AND_zzz 00000100 00 1 ..... 001 100 ..... ..... @rd_rn_rm_e0 ORR_zzz 00000100 01 1 ..... 001 100 ..... ..... @rd_rn_rm_e0 EOR_zzz 00000100 10 1 ..... 001 100 ..... ..... @rd_rn_rm_e0 BIC_zzz 00000100 11 1 ..... 001 100 ..... ..... @rd_rn_rm_e0 + +### SVE Memory - 32-bit Gather and Unsized Contiguous Group + +# SVE load predicate register +LDR_pri 10000101 10 ...... 000 ... ..... 0 .... @pd_rn_i9 + +# SVE load vector register +LDR_zri 10000101 10 ...... 010 ... ..... ..... @rd_rn_i9 -- 2.17.0 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20180516223007.10256-6-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/Makefile.objs | 2 +- target/arm/helper-sve.h | 21 ++++++++++ target/arm/helper.h | 1 + target/arm/sve_helper.c | 78 ++++++++++++++++++++++++++++++++++++++ target/arm/translate-sve.c | 65 +++++++++++++++++++++++++++++++ target/arm/sve.decode | 5 +++ 6 files changed, 171 insertions(+), 1 deletion(-) create mode 100644 target/arm/helper-sve.h create mode 100644 target/arm/sve_helper.c diff --git a/target/arm/Makefile.objs b/target/arm/Makefile.objs index XXXXXXX..XXXXXXX 100644 --- a/target/arm/Makefile.objs +++ b/target/arm/Makefile.objs @@ -XXX,XX +XXX,XX @@ target/arm/decode-sve.inc.c: $(SRC_PATH)/target/arm/sve.decode $(DECODETREE) "GEN", $(TARGET_DIR)$@) target/arm/translate-sve.o: target/arm/decode-sve.inc.c -obj-$(TARGET_AARCH64) += translate-sve.o +obj-$(TARGET_AARCH64) += translate-sve.o sve_helper.o diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h new file mode 100644 index XXXXXXX..XXXXXXX --- /dev/null +++ b/target/arm/helper-sve.h @@ -XXX,XX +XXX,XX @@ +/* + * AArch64 SVE specific helper definitions + * + * Copyright (c) 2018 Linaro, Ltd + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library; if not, see <http://www.gnu.org/licenses/>. + */ + +DEF_HELPER_FLAGS_2(sve_predtest1, TCG_CALL_NO_WG, i32, i64, i64) +DEF_HELPER_FLAGS_3(sve_predtest, TCG_CALL_NO_WG, i32, ptr, ptr, i32) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_5(gvec_fcmlad, TCG_CALL_NO_RWG, #ifdef TARGET_AARCH64 #include "helper-a64.h" +#include "helper-sve.h" #endif diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c new file mode 100644 index XXXXXXX..XXXXXXX --- /dev/null +++ b/target/arm/sve_helper.c @@ -XXX,XX +XXX,XX @@ +/* + * ARM SVE Operations + * + * Copyright (c) 2018 Linaro, Ltd. + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library; if not, see <http://www.gnu.org/licenses/>. + */ + +#include "qemu/osdep.h" +#include "cpu.h" +#include "exec/exec-all.h" +#include "exec/cpu_ldst.h" +#include "exec/helper-proto.h" +#include "tcg/tcg-gvec-desc.h" + + +/* Return a value for NZCV as per the ARM PredTest pseudofunction. + * + * The return value has bit 31 set if N is set, bit 1 set if Z is clear, + * and bit 0 set if C is set. Compare the definitions of these variables + * within CPUARMState. + */ + +/* For no G bits set, NZCV = C. */ +#define PREDTEST_INIT 1 + +/* This is an iterative function, called for each Pd and Pg word + * moving forward. + */ +static uint32_t iter_predtest_fwd(uint64_t d, uint64_t g, uint32_t flags) +{ + if (likely(g)) { + /* Compute N from first D & G. + Use bit 2 to signal first G bit seen. */ + if (!(flags & 4)) { + flags |= ((d & (g & -g)) != 0) << 31; + flags |= 4; + } + + /* Accumulate Z from each D & G. */ + flags |= ((d & g) != 0) << 1; + + /* Compute C from last !(D & G). Replace previous. */ + flags = deposit32(flags, 0, 1, (d & pow2floor(g)) == 0); + } + return flags; +} + +/* The same for a single word predicate. */ +uint32_t HELPER(sve_predtest1)(uint64_t d, uint64_t g) +{ + return iter_predtest_fwd(d, g, PREDTEST_INIT); +} + +/* The same for a multi-word predicate. */ +uint32_t HELPER(sve_predtest)(void *vd, void *vg, uint32_t words) +{ + uint32_t flags = PREDTEST_INIT; + uint64_t *d = vd, *g = vg; + uintptr_t i = 0; + + do { + flags = iter_predtest_fwd(d[i], g[i], flags); + } while (++i < words); + + return flags; +} diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool do_mov_z(DisasContext *s, int rd, int rn) return do_vector2_z(s, tcg_gen_gvec_mov, 0, rd, rn); } +/* Set the cpu flags as per a return from an SVE helper. */ +static void do_pred_flags(TCGv_i32 t) +{ + tcg_gen_mov_i32(cpu_NF, t); + tcg_gen_andi_i32(cpu_ZF, t, 2); + tcg_gen_andi_i32(cpu_CF, t, 1); + tcg_gen_movi_i32(cpu_VF, 0); +} + +/* Subroutines computing the ARM PredTest psuedofunction. */ +static void do_predtest1(TCGv_i64 d, TCGv_i64 g) +{ + TCGv_i32 t = tcg_temp_new_i32(); + + gen_helper_sve_predtest1(t, d, g); + do_pred_flags(t); + tcg_temp_free_i32(t); +} + +static void do_predtest(DisasContext *s, int dofs, int gofs, int words) +{ + TCGv_ptr dptr = tcg_temp_new_ptr(); + TCGv_ptr gptr = tcg_temp_new_ptr(); + TCGv_i32 t; + + tcg_gen_addi_ptr(dptr, cpu_env, dofs); + tcg_gen_addi_ptr(gptr, cpu_env, gofs); + t = tcg_const_i32(words); + + gen_helper_sve_predtest(t, dptr, gptr, t); + tcg_temp_free_ptr(dptr); + tcg_temp_free_ptr(gptr); + + do_pred_flags(t); + tcg_temp_free_i32(t); +} + /* *** SVE Logical - Unpredicated Group */ @@ -XXX,XX +XXX,XX @@ static bool trans_BIC_zzz(DisasContext *s, arg_rrr_esz *a, uint32_t insn) return do_vector3_z(s, tcg_gen_gvec_andc, 0, a->rd, a->rn, a->rm); } +/* + *** SVE Predicate Misc Group + */ + +static bool trans_PTEST(DisasContext *s, arg_PTEST *a, uint32_t insn) +{ + if (sve_access_check(s)) { + int nofs = pred_full_reg_offset(s, a->rn); + int gofs = pred_full_reg_offset(s, a->pg); + int words = DIV_ROUND_UP(pred_full_reg_size(s), 8); + + if (words == 1) { + TCGv_i64 pn = tcg_temp_new_i64(); + TCGv_i64 pg = tcg_temp_new_i64(); + + tcg_gen_ld_i64(pn, cpu_env, nofs); + tcg_gen_ld_i64(pg, cpu_env, gofs); + do_predtest1(pn, pg); + + tcg_temp_free_i64(pn); + tcg_temp_free_i64(pg); + } else { + do_predtest(s, nofs, gofs, words); + } + } + return true; +} + /* *** SVE Memory - 32-bit Gather and Unsized Contiguous Group */ diff --git a/target/arm/sve.decode b/target/arm/sve.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -XXX,XX +XXX,XX @@ ORR_zzz 00000100 01 1 ..... 001 100 ..... ..... @rd_rn_rm_e0 EOR_zzz 00000100 10 1 ..... 001 100 ..... ..... @rd_rn_rm_e0 BIC_zzz 00000100 11 1 ..... 001 100 ..... ..... @rd_rn_rm_e0 +### SVE Predicate Misc Group + +# SVE predicate test +PTEST 00100101 01 010000 11 pg:4 0 rn:4 0 0000 + ### SVE Memory - 32-bit Gather and Unsized Contiguous Group # SVE load predicate register -- 2.17.0 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20180516223007.10256-7-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/cpu.h | 4 +- target/arm/helper-sve.h | 10 + target/arm/sve_helper.c | 39 ++++ target/arm/translate-sve.c | 361 +++++++++++++++++++++++++++++++++++++ target/arm/sve.decode | 16 ++ 5 files changed, 429 insertions(+), 1 deletion(-) diff --git a/target/arm/cpu.h b/target/arm/cpu.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -XXX,XX +XXX,XX @@ typedef struct CPUARMState { #ifdef TARGET_AARCH64 /* Store FFR as pregs[16] to make it easier to treat as any other. */ ARMPredicateReg pregs[17]; + /* Scratch space for aa64 sve predicate temporary. */ + ARMPredicateReg preg_tmp; #endif uint32_t xregs[16]; @@ -XXX,XX +XXX,XX @@ typedef struct CPUARMState { int vec_len; int vec_stride; - /* scratch space when Tn are not sufficient. */ + /* Scratch space for aa32 neon expansion. */ uint32_t scratch[8]; /* There are a number of distinct float control structures: diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_2(sve_predtest1, TCG_CALL_NO_WG, i32, i64, i64) DEF_HELPER_FLAGS_3(sve_predtest, TCG_CALL_NO_WG, i32, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve_and_pppp, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve_bic_pppp, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve_eor_pppp, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve_sel_pppp, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve_orr_pppp, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve_orn_pppp, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve_nor_pppp, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve_nand_pppp, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -XXX,XX +XXX,XX @@ uint32_t HELPER(sve_predtest)(void *vd, void *vg, uint32_t words) return flags; } + +#define LOGICAL_PPPP(NAME, FUNC) \ +void HELPER(NAME)(void *vd, void *vn, void *vm, void *vg, uint32_t desc) \ +{ \ + uintptr_t opr_sz = simd_oprsz(desc); \ + uint64_t *d = vd, *n = vn, *m = vm, *g = vg; \ + uintptr_t i; \ + for (i = 0; i < opr_sz / 8; ++i) { \ + d[i] = FUNC(n[i], m[i], g[i]); \ + } \ +} + +#define DO_AND(N, M, G) (((N) & (M)) & (G)) +#define DO_BIC(N, M, G) (((N) & ~(M)) & (G)) +#define DO_EOR(N, M, G) (((N) ^ (M)) & (G)) +#define DO_ORR(N, M, G) (((N) | (M)) & (G)) +#define DO_ORN(N, M, G) (((N) | ~(M)) & (G)) +#define DO_NOR(N, M, G) (~((N) | (M)) & (G)) +#define DO_NAND(N, M, G) (~((N) & (M)) & (G)) +#define DO_SEL(N, M, G) (((N) & (G)) | ((M) & ~(G))) + +LOGICAL_PPPP(sve_and_pppp, DO_AND) +LOGICAL_PPPP(sve_bic_pppp, DO_BIC) +LOGICAL_PPPP(sve_eor_pppp, DO_EOR) +LOGICAL_PPPP(sve_sel_pppp, DO_SEL) +LOGICAL_PPPP(sve_orr_pppp, DO_ORR) +LOGICAL_PPPP(sve_orn_pppp, DO_ORN) +LOGICAL_PPPP(sve_nor_pppp, DO_NOR) +LOGICAL_PPPP(sve_nand_pppp, DO_NAND) + +#undef DO_AND +#undef DO_BIC +#undef DO_EOR +#undef DO_ORR +#undef DO_ORN +#undef DO_NOR +#undef DO_NAND +#undef DO_SEL +#undef LOGICAL_PPPP diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static inline int pred_full_reg_size(DisasContext *s) return s->sve_len >> 3; } +/* Round up the size of a register to a size allowed by + * the tcg vector infrastructure. Any operation which uses this + * size may assume that the bits above pred_full_reg_size are zero, + * and must leave them the same way. + * + * Note that this is not needed for the vector registers as they + * are always properly sized for tcg vectors. + */ +static int size_for_gvec(int size) +{ + if (size <= 8) { + return 8; + } else { + return QEMU_ALIGN_UP(size, 16); + } +} + +static int pred_gvec_reg_size(DisasContext *s) +{ + return size_for_gvec(pred_full_reg_size(s)); +} + /* Invoke a vector expander on two Zregs. */ static bool do_vector2_z(DisasContext *s, GVecGen2Fn *gvec_fn, int esz, int rd, int rn) @@ -XXX,XX +XXX,XX @@ static bool do_mov_z(DisasContext *s, int rd, int rn) return do_vector2_z(s, tcg_gen_gvec_mov, 0, rd, rn); } +/* Invoke a vector expander on two Pregs. */ +static bool do_vector2_p(DisasContext *s, GVecGen2Fn *gvec_fn, + int esz, int rd, int rn) +{ + if (sve_access_check(s)) { + unsigned psz = pred_gvec_reg_size(s); + gvec_fn(esz, pred_full_reg_offset(s, rd), + pred_full_reg_offset(s, rn), psz, psz); + } + return true; +} + +/* Invoke a vector expander on three Pregs. */ +static bool do_vector3_p(DisasContext *s, GVecGen3Fn *gvec_fn, + int esz, int rd, int rn, int rm) +{ + if (sve_access_check(s)) { + unsigned psz = pred_gvec_reg_size(s); + gvec_fn(esz, pred_full_reg_offset(s, rd), + pred_full_reg_offset(s, rn), + pred_full_reg_offset(s, rm), psz, psz); + } + return true; +} + +/* Invoke a vector operation on four Pregs. */ +static bool do_vecop4_p(DisasContext *s, const GVecGen4 *gvec_op, + int rd, int rn, int rm, int rg) +{ + if (sve_access_check(s)) { + unsigned psz = pred_gvec_reg_size(s); + tcg_gen_gvec_4(pred_full_reg_offset(s, rd), + pred_full_reg_offset(s, rn), + pred_full_reg_offset(s, rm), + pred_full_reg_offset(s, rg), + psz, psz, gvec_op); + } + return true; +} + +/* Invoke a vector move on two Pregs. */ +static bool do_mov_p(DisasContext *s, int rd, int rn) +{ + return do_vector2_p(s, tcg_gen_gvec_mov, 0, rd, rn); +} + /* Set the cpu flags as per a return from an SVE helper. */ static void do_pred_flags(TCGv_i32 t) { @@ -XXX,XX +XXX,XX @@ static bool trans_BIC_zzz(DisasContext *s, arg_rrr_esz *a, uint32_t insn) return do_vector3_z(s, tcg_gen_gvec_andc, 0, a->rd, a->rn, a->rm); } +/* + *** SVE Predicate Logical Operations Group + */ + +static bool do_pppp_flags(DisasContext *s, arg_rprr_s *a, + const GVecGen4 *gvec_op) +{ + if (!sve_access_check(s)) { + return true; + } + + unsigned psz = pred_gvec_reg_size(s); + int dofs = pred_full_reg_offset(s, a->rd); + int nofs = pred_full_reg_offset(s, a->rn); + int mofs = pred_full_reg_offset(s, a->rm); + int gofs = pred_full_reg_offset(s, a->pg); + + if (psz == 8) { + /* Do the operation and the flags generation in temps. */ + TCGv_i64 pd = tcg_temp_new_i64(); + TCGv_i64 pn = tcg_temp_new_i64(); + TCGv_i64 pm = tcg_temp_new_i64(); + TCGv_i64 pg = tcg_temp_new_i64(); + + tcg_gen_ld_i64(pn, cpu_env, nofs); + tcg_gen_ld_i64(pm, cpu_env, mofs); + tcg_gen_ld_i64(pg, cpu_env, gofs); + + gvec_op->fni8(pd, pn, pm, pg); + tcg_gen_st_i64(pd, cpu_env, dofs); + + do_predtest1(pd, pg); + + tcg_temp_free_i64(pd); + tcg_temp_free_i64(pn); + tcg_temp_free_i64(pm); + tcg_temp_free_i64(pg); + } else { + /* The operation and flags generation is large. The computation + * of the flags depends on the original contents of the guarding + * predicate. If the destination overwrites the guarding predicate, + * then the easiest way to get this right is to save a copy. + */ + int tofs = gofs; + if (a->rd == a->pg) { + tofs = offsetof(CPUARMState, vfp.preg_tmp); + tcg_gen_gvec_mov(0, tofs, gofs, psz, psz); + } + + tcg_gen_gvec_4(dofs, nofs, mofs, gofs, psz, psz, gvec_op); + do_predtest(s, dofs, tofs, psz / 8); + } + return true; +} + +static void gen_and_pg_i64(TCGv_i64 pd, TCGv_i64 pn, TCGv_i64 pm, TCGv_i64 pg) +{ + tcg_gen_and_i64(pd, pn, pm); + tcg_gen_and_i64(pd, pd, pg); +} + +static void gen_and_pg_vec(unsigned vece, TCGv_vec pd, TCGv_vec pn, + TCGv_vec pm, TCGv_vec pg) +{ + tcg_gen_and_vec(vece, pd, pn, pm); + tcg_gen_and_vec(vece, pd, pd, pg); +} + +static bool trans_AND_pppp(DisasContext *s, arg_rprr_s *a, uint32_t insn) +{ + static const GVecGen4 op = { + .fni8 = gen_and_pg_i64, + .fniv = gen_and_pg_vec, + .fno = gen_helper_sve_and_pppp, + .prefer_i64 = TCG_TARGET_REG_BITS == 64, + }; + if (a->s) { + return do_pppp_flags(s, a, &op); + } else if (a->rn == a->rm) { + if (a->pg == a->rn) { + return do_mov_p(s, a->rd, a->rn); + } else { + return do_vector3_p(s, tcg_gen_gvec_and, 0, a->rd, a->rn, a->pg); + } + } else if (a->pg == a->rn || a->pg == a->rm) { + return do_vector3_p(s, tcg_gen_gvec_and, 0, a->rd, a->rn, a->rm); + } else { + return do_vecop4_p(s, &op, a->rd, a->rn, a->rm, a->pg); + } +} + +static void gen_bic_pg_i64(TCGv_i64 pd, TCGv_i64 pn, TCGv_i64 pm, TCGv_i64 pg) +{ + tcg_gen_andc_i64(pd, pn, pm); + tcg_gen_and_i64(pd, pd, pg); +} + +static void gen_bic_pg_vec(unsigned vece, TCGv_vec pd, TCGv_vec pn, + TCGv_vec pm, TCGv_vec pg) +{ + tcg_gen_andc_vec(vece, pd, pn, pm); + tcg_gen_and_vec(vece, pd, pd, pg); +} + +static bool trans_BIC_pppp(DisasContext *s, arg_rprr_s *a, uint32_t insn) +{ + static const GVecGen4 op = { + .fni8 = gen_bic_pg_i64, + .fniv = gen_bic_pg_vec, + .fno = gen_helper_sve_bic_pppp, + .prefer_i64 = TCG_TARGET_REG_BITS == 64, + }; + if (a->s) { + return do_pppp_flags(s, a, &op); + } else if (a->pg == a->rn) { + return do_vector3_p(s, tcg_gen_gvec_andc, 0, a->rd, a->rn, a->rm); + } else { + return do_vecop4_p(s, &op, a->rd, a->rn, a->rm, a->pg); + } +} + +static void gen_eor_pg_i64(TCGv_i64 pd, TCGv_i64 pn, TCGv_i64 pm, TCGv_i64 pg) +{ + tcg_gen_xor_i64(pd, pn, pm); + tcg_gen_and_i64(pd, pd, pg); +} + +static void gen_eor_pg_vec(unsigned vece, TCGv_vec pd, TCGv_vec pn, + TCGv_vec pm, TCGv_vec pg) +{ + tcg_gen_xor_vec(vece, pd, pn, pm); + tcg_gen_and_vec(vece, pd, pd, pg); +} + +static bool trans_EOR_pppp(DisasContext *s, arg_rprr_s *a, uint32_t insn) +{ + static const GVecGen4 op = { + .fni8 = gen_eor_pg_i64, + .fniv = gen_eor_pg_vec, + .fno = gen_helper_sve_eor_pppp, + .prefer_i64 = TCG_TARGET_REG_BITS == 64, + }; + if (a->s) { + return do_pppp_flags(s, a, &op); + } else { + return do_vecop4_p(s, &op, a->rd, a->rn, a->rm, a->pg); + } +} + +static void gen_sel_pg_i64(TCGv_i64 pd, TCGv_i64 pn, TCGv_i64 pm, TCGv_i64 pg) +{ + tcg_gen_and_i64(pn, pn, pg); + tcg_gen_andc_i64(pm, pm, pg); + tcg_gen_or_i64(pd, pn, pm); +} + +static void gen_sel_pg_vec(unsigned vece, TCGv_vec pd, TCGv_vec pn, + TCGv_vec pm, TCGv_vec pg) +{ + tcg_gen_and_vec(vece, pn, pn, pg); + tcg_gen_andc_vec(vece, pm, pm, pg); + tcg_gen_or_vec(vece, pd, pn, pm); +} + +static bool trans_SEL_pppp(DisasContext *s, arg_rprr_s *a, uint32_t insn) +{ + static const GVecGen4 op = { + .fni8 = gen_sel_pg_i64, + .fniv = gen_sel_pg_vec, + .fno = gen_helper_sve_sel_pppp, + .prefer_i64 = TCG_TARGET_REG_BITS == 64, + }; + if (a->s) { + return false; + } else { + return do_vecop4_p(s, &op, a->rd, a->rn, a->rm, a->pg); + } +} + +static void gen_orr_pg_i64(TCGv_i64 pd, TCGv_i64 pn, TCGv_i64 pm, TCGv_i64 pg) +{ + tcg_gen_or_i64(pd, pn, pm); + tcg_gen_and_i64(pd, pd, pg); +} + +static void gen_orr_pg_vec(unsigned vece, TCGv_vec pd, TCGv_vec pn, + TCGv_vec pm, TCGv_vec pg) +{ + tcg_gen_or_vec(vece, pd, pn, pm); + tcg_gen_and_vec(vece, pd, pd, pg); +} + +static bool trans_ORR_pppp(DisasContext *s, arg_rprr_s *a, uint32_t insn) +{ + static const GVecGen4 op = { + .fni8 = gen_orr_pg_i64, + .fniv = gen_orr_pg_vec, + .fno = gen_helper_sve_orr_pppp, + .prefer_i64 = TCG_TARGET_REG_BITS == 64, + }; + if (a->s) { + return do_pppp_flags(s, a, &op); + } else if (a->pg == a->rn && a->rn == a->rm) { + return do_mov_p(s, a->rd, a->rn); + } else { + return do_vecop4_p(s, &op, a->rd, a->rn, a->rm, a->pg); + } +} + +static void gen_orn_pg_i64(TCGv_i64 pd, TCGv_i64 pn, TCGv_i64 pm, TCGv_i64 pg) +{ + tcg_gen_orc_i64(pd, pn, pm); + tcg_gen_and_i64(pd, pd, pg); +} + +static void gen_orn_pg_vec(unsigned vece, TCGv_vec pd, TCGv_vec pn, + TCGv_vec pm, TCGv_vec pg) +{ + tcg_gen_orc_vec(vece, pd, pn, pm); + tcg_gen_and_vec(vece, pd, pd, pg); +} + +static bool trans_ORN_pppp(DisasContext *s, arg_rprr_s *a, uint32_t insn) +{ + static const GVecGen4 op = { + .fni8 = gen_orn_pg_i64, + .fniv = gen_orn_pg_vec, + .fno = gen_helper_sve_orn_pppp, + .prefer_i64 = TCG_TARGET_REG_BITS == 64, + }; + if (a->s) { + return do_pppp_flags(s, a, &op); + } else { + return do_vecop4_p(s, &op, a->rd, a->rn, a->rm, a->pg); + } +} + +static void gen_nor_pg_i64(TCGv_i64 pd, TCGv_i64 pn, TCGv_i64 pm, TCGv_i64 pg) +{ + tcg_gen_or_i64(pd, pn, pm); + tcg_gen_andc_i64(pd, pg, pd); +} + +static void gen_nor_pg_vec(unsigned vece, TCGv_vec pd, TCGv_vec pn, + TCGv_vec pm, TCGv_vec pg) +{ + tcg_gen_or_vec(vece, pd, pn, pm); + tcg_gen_andc_vec(vece, pd, pg, pd); +} + +static bool trans_NOR_pppp(DisasContext *s, arg_rprr_s *a, uint32_t insn) +{ + static const GVecGen4 op = { + .fni8 = gen_nor_pg_i64, + .fniv = gen_nor_pg_vec, + .fno = gen_helper_sve_nor_pppp, + .prefer_i64 = TCG_TARGET_REG_BITS == 64, + }; + if (a->s) { + return do_pppp_flags(s, a, &op); + } else { + return do_vecop4_p(s, &op, a->rd, a->rn, a->rm, a->pg); + } +} + +static void gen_nand_pg_i64(TCGv_i64 pd, TCGv_i64 pn, TCGv_i64 pm, TCGv_i64 pg) +{ + tcg_gen_and_i64(pd, pn, pm); + tcg_gen_andc_i64(pd, pg, pd); +} + +static void gen_nand_pg_vec(unsigned vece, TCGv_vec pd, TCGv_vec pn, + TCGv_vec pm, TCGv_vec pg) +{ + tcg_gen_and_vec(vece, pd, pn, pm); + tcg_gen_andc_vec(vece, pd, pg, pd); +} + +static bool trans_NAND_pppp(DisasContext *s, arg_rprr_s *a, uint32_t insn) +{ + static const GVecGen4 op = { + .fni8 = gen_nand_pg_i64, + .fniv = gen_nand_pg_vec, + .fno = gen_helper_sve_nand_pppp, + .prefer_i64 = TCG_TARGET_REG_BITS == 64, + }; + if (a->s) { + return do_pppp_flags(s, a, &op); + } else { + return do_vecop4_p(s, &op, a->rd, a->rn, a->rm, a->pg); + } +} + /* *** SVE Predicate Misc Group */ diff --git a/target/arm/sve.decode b/target/arm/sve.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -XXX,XX +XXX,XX @@ &rri rd rn imm &rrr_esz rd rn rm esz +&rprr_s rd pg rn rm s ########################################################################### # Named instruction formats. These are generally used to @@ -XXX,XX +XXX,XX @@ # Three operand with unused vector element size @rd_rn_rm_e0 ........ ... rm:5 ... ... rn:5 rd:5 &rrr_esz esz=0 +# Three predicate operand, with governing predicate, flag setting +@pd_pg_pn_pm_s ........ . s:1 .. rm:4 .. pg:4 . rn:4 . rd:4 &rprr_s + # Basic Load/Store with 9-bit immediate offset @pd_rn_i9 ........ ........ ...... rn:5 . rd:4 \ &rri imm=%imm9_16_10 @@ -XXX,XX +XXX,XX @@ ORR_zzz 00000100 01 1 ..... 001 100 ..... ..... @rd_rn_rm_e0 EOR_zzz 00000100 10 1 ..... 001 100 ..... ..... @rd_rn_rm_e0 BIC_zzz 00000100 11 1 ..... 001 100 ..... ..... @rd_rn_rm_e0 +### SVE Predicate Logical Operations Group + +# SVE predicate logical operations +AND_pppp 00100101 0. 00 .... 01 .... 0 .... 0 .... @pd_pg_pn_pm_s +BIC_pppp 00100101 0. 00 .... 01 .... 0 .... 1 .... @pd_pg_pn_pm_s +EOR_pppp 00100101 0. 00 .... 01 .... 1 .... 0 .... @pd_pg_pn_pm_s +SEL_pppp 00100101 0. 00 .... 01 .... 1 .... 1 .... @pd_pg_pn_pm_s +ORR_pppp 00100101 1. 00 .... 01 .... 0 .... 0 .... @pd_pg_pn_pm_s +ORN_pppp 00100101 1. 00 .... 01 .... 0 .... 1 .... @pd_pg_pn_pm_s +NOR_pppp 00100101 1. 00 .... 01 .... 1 .... 0 .... @pd_pg_pn_pm_s +NAND_pppp 00100101 1. 00 .... 01 .... 1 .... 1 .... @pd_pg_pn_pm_s + ### SVE Predicate Misc Group # SVE predicate test -- 2.17.0 From: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20180516223007.10256-8-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/cpu.h | 4 + target/arm/helper-sve.h | 3 + target/arm/sve_helper.c | 84 +++++++++++++++ target/arm/translate-sve.c | 209 +++++++++++++++++++++++++++++++++++++ target/arm/sve.decode | 31 ++++++ 5 files changed, 331 insertions(+) diff --git a/target/arm/cpu.h b/target/arm/cpu.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -XXX,XX +XXX,XX @@ typedef struct CPUARMState { #ifdef TARGET_AARCH64 /* Store FFR as pregs[16] to make it easier to treat as any other. */ +#define FFR_PRED_NUM 16 ARMPredicateReg pregs[17]; /* Scratch space for aa64 sve predicate temporary. */ ARMPredicateReg preg_tmp; @@ -XXX,XX +XXX,XX @@ static inline uint64_t *aa64_vfp_qreg(CPUARMState *env, unsigned regno) return &env->vfp.zregs[regno].d[0]; } +/* Shared between translate-sve.c and sve_helper.c. */ +extern const uint64_t pred_esz_masks[4]; + #endif diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_2(sve_predtest1, TCG_CALL_NO_WG, i32, i64, i64) DEF_HELPER_FLAGS_3(sve_predtest, TCG_CALL_NO_WG, i32, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve_pfirst, TCG_CALL_NO_WG, i32, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve_pnext, TCG_CALL_NO_WG, i32, ptr, ptr, i32) + DEF_HELPER_FLAGS_5(sve_and_pppp, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve_bic_pppp, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve_eor_pppp, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -XXX,XX +XXX,XX @@ LOGICAL_PPPP(sve_nand_pppp, DO_NAND) #undef DO_NAND #undef DO_SEL #undef LOGICAL_PPPP + +/* Similar to the ARM LastActiveElement pseudocode function, except the + result is multiplied by the element size. This includes the not found + indication; e.g. not found for esz=3 is -8. */ +static intptr_t last_active_element(uint64_t *g, intptr_t words, intptr_t esz) +{ + uint64_t mask = pred_esz_masks[esz]; + intptr_t i = words; + + do { + uint64_t this_g = g[--i] & mask; + if (this_g) { + return i * 64 + (63 - clz64(this_g)); + } + } while (i > 0); + return (intptr_t)-1 << esz; +} + +uint32_t HELPER(sve_pfirst)(void *vd, void *vg, uint32_t words) +{ + uint32_t flags = PREDTEST_INIT; + uint64_t *d = vd, *g = vg; + intptr_t i = 0; + + do { + uint64_t this_d = d[i]; + uint64_t this_g = g[i]; + + if (this_g) { + if (!(flags & 4)) { + /* Set in D the first bit of G. */ + this_d |= this_g & -this_g; + d[i] = this_d; + } + flags = iter_predtest_fwd(this_d, this_g, flags); + } + } while (++i < words); + + return flags; +} + +uint32_t HELPER(sve_pnext)(void *vd, void *vg, uint32_t pred_desc) +{ + intptr_t words = extract32(pred_desc, 0, SIMD_OPRSZ_BITS); + intptr_t esz = extract32(pred_desc, SIMD_DATA_SHIFT, 2); + uint32_t flags = PREDTEST_INIT; + uint64_t *d = vd, *g = vg, esz_mask; + intptr_t i, next; + + next = last_active_element(vd, words, esz) + (1 << esz); + esz_mask = pred_esz_masks[esz]; + + /* Similar to the pseudocode for pnext, but scaled by ESZ + so that we find the correct bit. */ + if (next < words * 64) { + uint64_t mask = -1; + + if (next & 63) { + mask = ~((1ull << (next & 63)) - 1); + next &= -64; + } + do { + uint64_t this_g = g[next / 64] & esz_mask & mask; + if (this_g != 0) { + next = (next & -64) + ctz64(this_g); + break; + } + next += 64; + mask = -1; + } while (next < words * 64); + } + + i = 0; + do { + uint64_t this_d = 0; + if (i == next / 64) { + this_d = 1ull << (next & 63); + } + d[i] = this_d; + flags = iter_predtest_fwd(this_d, g[i] & esz_mask, flags); + } while (++i < words); + + return flags; +} diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ #include "exec/exec-all.h" #include "tcg-op.h" #include "tcg-op-gvec.h" +#include "tcg-gvec-desc.h" #include "qemu/log.h" #include "arm_ldst.h" #include "translate.h" @@ -XXX,XX +XXX,XX @@ static void do_predtest(DisasContext *s, int dofs, int gofs, int words) tcg_temp_free_i32(t); } +/* For each element size, the bits within a predicate word that are active. */ +const uint64_t pred_esz_masks[4] = { + 0xffffffffffffffffull, 0x5555555555555555ull, + 0x1111111111111111ull, 0x0101010101010101ull +}; + /* *** SVE Logical - Unpredicated Group */ @@ -XXX,XX +XXX,XX @@ static bool trans_PTEST(DisasContext *s, arg_PTEST *a, uint32_t insn) return true; } +/* See the ARM pseudocode DecodePredCount. */ +static unsigned decode_pred_count(unsigned fullsz, int pattern, int esz) +{ + unsigned elements = fullsz >> esz; + unsigned bound; + + switch (pattern) { + case 0x0: /* POW2 */ + return pow2floor(elements); + case 0x1: /* VL1 */ + case 0x2: /* VL2 */ + case 0x3: /* VL3 */ + case 0x4: /* VL4 */ + case 0x5: /* VL5 */ + case 0x6: /* VL6 */ + case 0x7: /* VL7 */ + case 0x8: /* VL8 */ + bound = pattern; + break; + case 0x9: /* VL16 */ + case 0xa: /* VL32 */ + case 0xb: /* VL64 */ + case 0xc: /* VL128 */ + case 0xd: /* VL256 */ + bound = 16 << (pattern - 9); + break; + case 0x1d: /* MUL4 */ + return elements - elements % 4; + case 0x1e: /* MUL3 */ + return elements - elements % 3; + case 0x1f: /* ALL */ + return elements; + default: /* #uimm5 */ + return 0; + } + return elements >= bound ? bound : 0; +} + +/* This handles all of the predicate initialization instructions, + * PTRUE, PFALSE, SETFFR. For PFALSE, we will have set PAT == 32 + * so that decode_pred_count returns 0. For SETFFR, we will have + * set RD == 16 == FFR. + */ +static bool do_predset(DisasContext *s, int esz, int rd, int pat, bool setflag) +{ + if (!sve_access_check(s)) { + return true; + } + + unsigned fullsz = vec_full_reg_size(s); + unsigned ofs = pred_full_reg_offset(s, rd); + unsigned numelem, setsz, i; + uint64_t word, lastword; + TCGv_i64 t; + + numelem = decode_pred_count(fullsz, pat, esz); + + /* Determine what we must store into each bit, and how many. */ + if (numelem == 0) { + lastword = word = 0; + setsz = fullsz; + } else { + setsz = numelem << esz; + lastword = word = pred_esz_masks[esz]; + if (setsz % 64) { + lastword &= ~(-1ull << (setsz % 64)); + } + } + + t = tcg_temp_new_i64(); + if (fullsz <= 64) { + tcg_gen_movi_i64(t, lastword); + tcg_gen_st_i64(t, cpu_env, ofs); + goto done; + } + + if (word == lastword) { + unsigned maxsz = size_for_gvec(fullsz / 8); + unsigned oprsz = size_for_gvec(setsz / 8); + + if (oprsz * 8 == setsz) { + tcg_gen_gvec_dup64i(ofs, oprsz, maxsz, word); + goto done; + } + if (oprsz * 8 == setsz + 8) { + tcg_gen_gvec_dup64i(ofs, oprsz, maxsz, word); + tcg_gen_movi_i64(t, 0); + tcg_gen_st_i64(t, cpu_env, ofs + oprsz - 8); + goto done; + } + } + + setsz /= 8; + fullsz /= 8; + + tcg_gen_movi_i64(t, word); + for (i = 0; i < setsz; i += 8) { + tcg_gen_st_i64(t, cpu_env, ofs + i); + } + if (lastword != word) { + tcg_gen_movi_i64(t, lastword); + tcg_gen_st_i64(t, cpu_env, ofs + i); + i += 8; + } + if (i < fullsz) { + tcg_gen_movi_i64(t, 0); + for (; i < fullsz; i += 8) { + tcg_gen_st_i64(t, cpu_env, ofs + i); + } + } + + done: + tcg_temp_free_i64(t); + + /* PTRUES */ + if (setflag) { + tcg_gen_movi_i32(cpu_NF, -(word != 0)); + tcg_gen_movi_i32(cpu_CF, word == 0); + tcg_gen_movi_i32(cpu_VF, 0); + tcg_gen_mov_i32(cpu_ZF, cpu_NF); + } + return true; +} + +static bool trans_PTRUE(DisasContext *s, arg_PTRUE *a, uint32_t insn) +{ + return do_predset(s, a->esz, a->rd, a->pat, a->s); +} + +static bool trans_SETFFR(DisasContext *s, arg_SETFFR *a, uint32_t insn) +{ + /* Note pat == 31 is #all, to set all elements. */ + return do_predset(s, 0, FFR_PRED_NUM, 31, false); +} + +static bool trans_PFALSE(DisasContext *s, arg_PFALSE *a, uint32_t insn) +{ + /* Note pat == 32 is #unimp, to set no elements. */ + return do_predset(s, 0, a->rd, 32, false); +} + +static bool trans_RDFFR_p(DisasContext *s, arg_RDFFR_p *a, uint32_t insn) +{ + /* The path through do_pppp_flags is complicated enough to want to avoid + * duplication. Frob the arguments into the form of a predicated AND. + */ + arg_rprr_s alt_a = { + .rd = a->rd, .pg = a->pg, .s = a->s, + .rn = FFR_PRED_NUM, .rm = FFR_PRED_NUM, + }; + return trans_AND_pppp(s, &alt_a, insn); +} + +static bool trans_RDFFR(DisasContext *s, arg_RDFFR *a, uint32_t insn) +{ + return do_mov_p(s, a->rd, FFR_PRED_NUM); +} + +static bool trans_WRFFR(DisasContext *s, arg_WRFFR *a, uint32_t insn) +{ + return do_mov_p(s, FFR_PRED_NUM, a->rn); +} + +static bool do_pfirst_pnext(DisasContext *s, arg_rr_esz *a, + void (*gen_fn)(TCGv_i32, TCGv_ptr, + TCGv_ptr, TCGv_i32)) +{ + if (!sve_access_check(s)) { + return true; + } + + TCGv_ptr t_pd = tcg_temp_new_ptr(); + TCGv_ptr t_pg = tcg_temp_new_ptr(); + TCGv_i32 t; + unsigned desc; + + desc = DIV_ROUND_UP(pred_full_reg_size(s), 8); + desc = deposit32(desc, SIMD_DATA_SHIFT, 2, a->esz); + + tcg_gen_addi_ptr(t_pd, cpu_env, pred_full_reg_offset(s, a->rd)); + tcg_gen_addi_ptr(t_pg, cpu_env, pred_full_reg_offset(s, a->rn)); + t = tcg_const_i32(desc); + + gen_fn(t, t_pd, t_pg, t); + tcg_temp_free_ptr(t_pd); + tcg_temp_free_ptr(t_pg); + + do_pred_flags(t); + tcg_temp_free_i32(t); + return true; +} + +static bool trans_PFIRST(DisasContext *s, arg_rr_esz *a, uint32_t insn) +{ + return do_pfirst_pnext(s, a, gen_helper_sve_pfirst); +} + +static bool trans_PNEXT(DisasContext *s, arg_rr_esz *a, uint32_t insn) +{ + return do_pfirst_pnext(s, a, gen_helper_sve_pnext); +} + /* *** SVE Memory - 32-bit Gather and Unsized Contiguous Group */ diff --git a/target/arm/sve.decode b/target/arm/sve.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -XXX,XX +XXX,XX @@ # when creating helpers common to those for the individual # instruction patterns. +&rr_esz rd rn esz &rri rd rn imm &rrr_esz rd rn rm esz &rprr_s rd pg rn rm s @@ -XXX,XX +XXX,XX @@ # Named instruction formats. These are generally used to # reduce the amount of duplication between instruction patterns. +# Two operand with unused vector element size +@pd_pn_e0 ........ ........ ....... rn:4 . rd:4 &rr_esz esz=0 + +# Two operand +@pd_pn ........ esz:2 .. .... ....... rn:4 . rd:4 &rr_esz + # Three operand with unused vector element size @rd_rn_rm_e0 ........ ... rm:5 ... ... rn:5 rd:5 &rrr_esz esz=0 @@ -XXX,XX +XXX,XX @@ NAND_pppp 00100101 1. 00 .... 01 .... 1 .... 1 .... @pd_pg_pn_pm_s # SVE predicate test PTEST 00100101 01 010000 11 pg:4 0 rn:4 0 0000 +# SVE predicate initialize +PTRUE 00100101 esz:2 01100 s:1 111000 pat:5 0 rd:4 + +# SVE initialize FFR +SETFFR 00100101 0010 1100 1001 0000 0000 0000 + +# SVE zero predicate register +PFALSE 00100101 0001 1000 1110 0100 0000 rd:4 + +# SVE predicate read from FFR (predicated) +RDFFR_p 00100101 0 s:1 0110001111000 pg:4 0 rd:4 + +# SVE predicate read from FFR (unpredicated) +RDFFR 00100101 0001 1001 1111 0000 0000 rd:4 + +# SVE FFR write from predicate (WRFFR) +WRFFR 00100101 0010 1000 1001 000 rn:4 00000 + +# SVE predicate first active +PFIRST 00100101 01 011 000 11000 00 .... 0 .... @pd_pn_e0 + +# SVE predicate next active +PNEXT 00100101 .. 011 001 11000 10 .... 0 .... @pd_pn + ### SVE Memory - 32-bit Gather and Unsized Contiguous Group # SVE load predicate register -- 2.17.0 From: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20180516223007.10256-9-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/helper-sve.h | 145 +++++++++++++++++++++++++++ target/arm/sve_helper.c | 194 +++++++++++++++++++++++++++++++++++++ target/arm/translate-sve.c | 68 +++++++++++++ target/arm/sve.decode | 42 ++++++++ 4 files changed, 449 insertions(+) diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_3(sve_predtest, TCG_CALL_NO_WG, i32, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve_pfirst, TCG_CALL_NO_WG, i32, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve_pnext, TCG_CALL_NO_WG, i32, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve_and_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve_and_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve_and_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve_and_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve_eor_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve_eor_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve_eor_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve_eor_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve_orr_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve_orr_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve_orr_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve_orr_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve_bic_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve_bic_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve_bic_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve_bic_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve_add_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve_add_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve_add_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve_add_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve_sub_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve_sub_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve_sub_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve_sub_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve_smax_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve_smax_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve_smax_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve_smax_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve_umax_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve_umax_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve_umax_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve_umax_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve_smin_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve_smin_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve_smin_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve_smin_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve_umin_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve_umin_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve_umin_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve_umin_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve_sabd_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve_sabd_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve_sabd_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve_sabd_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve_uabd_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve_uabd_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve_uabd_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve_uabd_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve_mul_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve_mul_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve_mul_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve_mul_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve_smulh_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve_smulh_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve_smulh_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve_smulh_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve_umulh_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve_umulh_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve_umulh_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve_umulh_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve_sdiv_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve_sdiv_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve_udiv_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve_udiv_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_5(sve_and_pppp, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve_bic_pppp, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve_eor_pppp, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -XXX,XX +XXX,XX @@ #include "tcg/tcg-gvec-desc.h" +/* Note that vector data is stored in host-endian 64-bit chunks, + so addressing units smaller than that needs a host-endian fixup. */ +#ifdef HOST_WORDS_BIGENDIAN +#define H1(x) ((x) ^ 7) +#define H1_2(x) ((x) ^ 6) +#define H1_4(x) ((x) ^ 4) +#define H2(x) ((x) ^ 3) +#define H4(x) ((x) ^ 1) +#else +#define H1(x) (x) +#define H1_2(x) (x) +#define H1_4(x) (x) +#define H2(x) (x) +#define H4(x) (x) +#endif + /* Return a value for NZCV as per the ARM PredTest pseudofunction. * * The return value has bit 31 set if N is set, bit 1 set if Z is clear, @@ -XXX,XX +XXX,XX @@ LOGICAL_PPPP(sve_nand_pppp, DO_NAND) #undef DO_SEL #undef LOGICAL_PPPP +/* Fully general three-operand expander, controlled by a predicate. + * This is complicated by the host-endian storage of the register file. + */ +/* ??? I don't expect the compiler could ever vectorize this itself. + * With some tables we can convert bit masks to byte masks, and with + * extra care wrt byte/word ordering we could use gcc generic vectors + * and do 16 bytes at a time. + */ +#define DO_ZPZZ(NAME, TYPE, H, OP) \ +void HELPER(NAME)(void *vd, void *vn, void *vm, void *vg, uint32_t desc) \ +{ \ + intptr_t i, opr_sz = simd_oprsz(desc); \ + for (i = 0; i < opr_sz; ) { \ + uint16_t pg = *(uint16_t *)(vg + H1_2(i >> 3)); \ + do { \ + if (pg & 1) { \ + TYPE nn = *(TYPE *)(vn + H(i)); \ + TYPE mm = *(TYPE *)(vm + H(i)); \ + *(TYPE *)(vd + H(i)) = OP(nn, mm); \ + } \ + i += sizeof(TYPE), pg >>= sizeof(TYPE); \ + } while (i & 15); \ + } \ +} + +/* Similarly, specialized for 64-bit operands. */ +#define DO_ZPZZ_D(NAME, TYPE, OP) \ +void HELPER(NAME)(void *vd, void *vn, void *vm, void *vg, uint32_t desc) \ +{ \ + intptr_t i, opr_sz = simd_oprsz(desc) / 8; \ + TYPE *d = vd, *n = vn, *m = vm; \ + uint8_t *pg = vg; \ + for (i = 0; i < opr_sz; i += 1) { \ + if (pg[H1(i)] & 1) { \ + TYPE nn = n[i], mm = m[i]; \ + d[i] = OP(nn, mm); \ + } \ + } \ +} + +#define DO_AND(N, M) (N & M) +#define DO_EOR(N, M) (N ^ M) +#define DO_ORR(N, M) (N | M) +#define DO_BIC(N, M) (N & ~M) +#define DO_ADD(N, M) (N + M) +#define DO_SUB(N, M) (N - M) +#define DO_MAX(N, M) ((N) >= (M) ? (N) : (M)) +#define DO_MIN(N, M) ((N) >= (M) ? (M) : (N)) +#define DO_ABD(N, M) ((N) >= (M) ? (N) - (M) : (M) - (N)) +#define DO_MUL(N, M) (N * M) +#define DO_DIV(N, M) (M ? N / M : 0) + +DO_ZPZZ(sve_and_zpzz_b, uint8_t, H1, DO_AND) +DO_ZPZZ(sve_and_zpzz_h, uint16_t, H1_2, DO_AND) +DO_ZPZZ(sve_and_zpzz_s, uint32_t, H1_4, DO_AND) +DO_ZPZZ_D(sve_and_zpzz_d, uint64_t, DO_AND) + +DO_ZPZZ(sve_orr_zpzz_b, uint8_t, H1, DO_ORR) +DO_ZPZZ(sve_orr_zpzz_h, uint16_t, H1_2, DO_ORR) +DO_ZPZZ(sve_orr_zpzz_s, uint32_t, H1_4, DO_ORR) +DO_ZPZZ_D(sve_orr_zpzz_d, uint64_t, DO_ORR) + +DO_ZPZZ(sve_eor_zpzz_b, uint8_t, H1, DO_EOR) +DO_ZPZZ(sve_eor_zpzz_h, uint16_t, H1_2, DO_EOR) +DO_ZPZZ(sve_eor_zpzz_s, uint32_t, H1_4, DO_EOR) +DO_ZPZZ_D(sve_eor_zpzz_d, uint64_t, DO_EOR) + +DO_ZPZZ(sve_bic_zpzz_b, uint8_t, H1, DO_BIC) +DO_ZPZZ(sve_bic_zpzz_h, uint16_t, H1_2, DO_BIC) +DO_ZPZZ(sve_bic_zpzz_s, uint32_t, H1_4, DO_BIC) +DO_ZPZZ_D(sve_bic_zpzz_d, uint64_t, DO_BIC) + +DO_ZPZZ(sve_add_zpzz_b, uint8_t, H1, DO_ADD) +DO_ZPZZ(sve_add_zpzz_h, uint16_t, H1_2, DO_ADD) +DO_ZPZZ(sve_add_zpzz_s, uint32_t, H1_4, DO_ADD) +DO_ZPZZ_D(sve_add_zpzz_d, uint64_t, DO_ADD) + +DO_ZPZZ(sve_sub_zpzz_b, uint8_t, H1, DO_SUB) +DO_ZPZZ(sve_sub_zpzz_h, uint16_t, H1_2, DO_SUB) +DO_ZPZZ(sve_sub_zpzz_s, uint32_t, H1_4, DO_SUB) +DO_ZPZZ_D(sve_sub_zpzz_d, uint64_t, DO_SUB) + +DO_ZPZZ(sve_smax_zpzz_b, int8_t, H1, DO_MAX) +DO_ZPZZ(sve_smax_zpzz_h, int16_t, H1_2, DO_MAX) +DO_ZPZZ(sve_smax_zpzz_s, int32_t, H1_4, DO_MAX) +DO_ZPZZ_D(sve_smax_zpzz_d, int64_t, DO_MAX) + +DO_ZPZZ(sve_umax_zpzz_b, uint8_t, H1, DO_MAX) +DO_ZPZZ(sve_umax_zpzz_h, uint16_t, H1_2, DO_MAX) +DO_ZPZZ(sve_umax_zpzz_s, uint32_t, H1_4, DO_MAX) +DO_ZPZZ_D(sve_umax_zpzz_d, uint64_t, DO_MAX) + +DO_ZPZZ(sve_smin_zpzz_b, int8_t, H1, DO_MIN) +DO_ZPZZ(sve_smin_zpzz_h, int16_t, H1_2, DO_MIN) +DO_ZPZZ(sve_smin_zpzz_s, int32_t, H1_4, DO_MIN) +DO_ZPZZ_D(sve_smin_zpzz_d, int64_t, DO_MIN) + +DO_ZPZZ(sve_umin_zpzz_b, uint8_t, H1, DO_MIN) +DO_ZPZZ(sve_umin_zpzz_h, uint16_t, H1_2, DO_MIN) +DO_ZPZZ(sve_umin_zpzz_s, uint32_t, H1_4, DO_MIN) +DO_ZPZZ_D(sve_umin_zpzz_d, uint64_t, DO_MIN) + +DO_ZPZZ(sve_sabd_zpzz_b, int8_t, H1, DO_ABD) +DO_ZPZZ(sve_sabd_zpzz_h, int16_t, H1_2, DO_ABD) +DO_ZPZZ(sve_sabd_zpzz_s, int32_t, H1_4, DO_ABD) +DO_ZPZZ_D(sve_sabd_zpzz_d, int64_t, DO_ABD) + +DO_ZPZZ(sve_uabd_zpzz_b, uint8_t, H1, DO_ABD) +DO_ZPZZ(sve_uabd_zpzz_h, uint16_t, H1_2, DO_ABD) +DO_ZPZZ(sve_uabd_zpzz_s, uint32_t, H1_4, DO_ABD) +DO_ZPZZ_D(sve_uabd_zpzz_d, uint64_t, DO_ABD) + +/* Because the computation type is at least twice as large as required, + these work for both signed and unsigned source types. */ +static inline uint8_t do_mulh_b(int32_t n, int32_t m) +{ + return (n * m) >> 8; +} + +static inline uint16_t do_mulh_h(int32_t n, int32_t m) +{ + return (n * m) >> 16; +} + +static inline uint32_t do_mulh_s(int64_t n, int64_t m) +{ + return (n * m) >> 32; +} + +static inline uint64_t do_smulh_d(uint64_t n, uint64_t m) +{ + uint64_t lo, hi; + muls64(&lo, &hi, n, m); + return hi; +} + +static inline uint64_t do_umulh_d(uint64_t n, uint64_t m) +{ + uint64_t lo, hi; + mulu64(&lo, &hi, n, m); + return hi; +} + +DO_ZPZZ(sve_mul_zpzz_b, uint8_t, H1, DO_MUL) +DO_ZPZZ(sve_mul_zpzz_h, uint16_t, H1_2, DO_MUL) +DO_ZPZZ(sve_mul_zpzz_s, uint32_t, H1_4, DO_MUL) +DO_ZPZZ_D(sve_mul_zpzz_d, uint64_t, DO_MUL) + +DO_ZPZZ(sve_smulh_zpzz_b, int8_t, H1, do_mulh_b) +DO_ZPZZ(sve_smulh_zpzz_h, int16_t, H1_2, do_mulh_h) +DO_ZPZZ(sve_smulh_zpzz_s, int32_t, H1_4, do_mulh_s) +DO_ZPZZ_D(sve_smulh_zpzz_d, uint64_t, do_smulh_d) + +DO_ZPZZ(sve_umulh_zpzz_b, uint8_t, H1, do_mulh_b) +DO_ZPZZ(sve_umulh_zpzz_h, uint16_t, H1_2, do_mulh_h) +DO_ZPZZ(sve_umulh_zpzz_s, uint32_t, H1_4, do_mulh_s) +DO_ZPZZ_D(sve_umulh_zpzz_d, uint64_t, do_umulh_d) + +DO_ZPZZ(sve_sdiv_zpzz_s, int32_t, H1_4, DO_DIV) +DO_ZPZZ_D(sve_sdiv_zpzz_d, int64_t, DO_DIV) + +DO_ZPZZ(sve_udiv_zpzz_s, uint32_t, H1_4, DO_DIV) +DO_ZPZZ_D(sve_udiv_zpzz_d, uint64_t, DO_DIV) + +#undef DO_ZPZZ +#undef DO_ZPZZ_D +#undef DO_AND +#undef DO_ORR +#undef DO_EOR +#undef DO_BIC +#undef DO_ADD +#undef DO_SUB +#undef DO_MAX +#undef DO_MIN +#undef DO_ABD +#undef DO_MUL +#undef DO_DIV + /* Similar to the ARM LastActiveElement pseudocode function, except the result is multiplied by the element size. This includes the not found indication; e.g. not found for esz=3 is -8. */ diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool trans_BIC_zzz(DisasContext *s, arg_rrr_esz *a, uint32_t insn) return do_vector3_z(s, tcg_gen_gvec_andc, 0, a->rd, a->rn, a->rm); } +/* + *** SVE Integer Arithmetic - Binary Predicated Group + */ + +static bool do_zpzz_ool(DisasContext *s, arg_rprr_esz *a, gen_helper_gvec_4 *fn) +{ + unsigned vsz = vec_full_reg_size(s); + if (fn == NULL) { + return false; + } + if (sve_access_check(s)) { + tcg_gen_gvec_4_ool(vec_full_reg_offset(s, a->rd), + vec_full_reg_offset(s, a->rn), + vec_full_reg_offset(s, a->rm), + pred_full_reg_offset(s, a->pg), + vsz, vsz, 0, fn); + } + return true; +} + +#define DO_ZPZZ(NAME, name) \ +static bool trans_##NAME##_zpzz(DisasContext *s, arg_rprr_esz *a, \ + uint32_t insn) \ +{ \ + static gen_helper_gvec_4 * const fns[4] = { \ + gen_helper_sve_##name##_zpzz_b, gen_helper_sve_##name##_zpzz_h, \ + gen_helper_sve_##name##_zpzz_s, gen_helper_sve_##name##_zpzz_d, \ + }; \ + return do_zpzz_ool(s, a, fns[a->esz]); \ +} + +DO_ZPZZ(AND, and) +DO_ZPZZ(EOR, eor) +DO_ZPZZ(ORR, orr) +DO_ZPZZ(BIC, bic) + +DO_ZPZZ(ADD, add) +DO_ZPZZ(SUB, sub) + +DO_ZPZZ(SMAX, smax) +DO_ZPZZ(UMAX, umax) +DO_ZPZZ(SMIN, smin) +DO_ZPZZ(UMIN, umin) +DO_ZPZZ(SABD, sabd) +DO_ZPZZ(UABD, uabd) + +DO_ZPZZ(MUL, mul) +DO_ZPZZ(SMULH, smulh) +DO_ZPZZ(UMULH, umulh) + +static bool trans_SDIV_zpzz(DisasContext *s, arg_rprr_esz *a, uint32_t insn) +{ + static gen_helper_gvec_4 * const fns[4] = { + NULL, NULL, gen_helper_sve_sdiv_zpzz_s, gen_helper_sve_sdiv_zpzz_d + }; + return do_zpzz_ool(s, a, fns[a->esz]); +} + +static bool trans_UDIV_zpzz(DisasContext *s, arg_rprr_esz *a, uint32_t insn) +{ + static gen_helper_gvec_4 * const fns[4] = { + NULL, NULL, gen_helper_sve_udiv_zpzz_s, gen_helper_sve_udiv_zpzz_d + }; + return do_zpzz_ool(s, a, fns[a->esz]); +} + +#undef DO_ZPZZ + /* *** SVE Predicate Logical Operations Group */ diff --git a/target/arm/sve.decode b/target/arm/sve.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -XXX,XX +XXX,XX @@ %imm9_16_10 16:s6 10:3 +# Either a copy of rd (at bit 0), or a different source +# as propagated via the MOVPRFX instruction. +%reg_movprfx 0:5 + ########################################################################### # Named attribute sets. These are used to make nice(er) names # when creating helpers common to those for the individual @@ -XXX,XX +XXX,XX @@ &rri rd rn imm &rrr_esz rd rn rm esz &rprr_s rd pg rn rm s +&rprr_esz rd pg rn rm esz ########################################################################### # Named instruction formats. These are generally used to @@ -XXX,XX +XXX,XX @@ # Three predicate operand, with governing predicate, flag setting @pd_pg_pn_pm_s ........ . s:1 .. rm:4 .. pg:4 . rn:4 . rd:4 &rprr_s +# Two register operand, with governing predicate, vector element size +@rdn_pg_rm ........ esz:2 ... ... ... pg:3 rm:5 rd:5 \ + &rprr_esz rn=%reg_movprfx +@rdm_pg_rn ........ esz:2 ... ... ... pg:3 rn:5 rd:5 \ + &rprr_esz rm=%reg_movprfx + # Basic Load/Store with 9-bit immediate offset @pd_rn_i9 ........ ........ ...... rn:5 . rd:4 \ &rri imm=%imm9_16_10 @@ -XXX,XX +XXX,XX @@ ########################################################################### # Instruction patterns. Grouped according to the SVE encodingindex.xhtml. +### SVE Integer Arithmetic - Binary Predicated Group + +# SVE bitwise logical vector operations (predicated) +ORR_zpzz 00000100 .. 011 000 000 ... ..... ..... @rdn_pg_rm +EOR_zpzz 00000100 .. 011 001 000 ... ..... ..... @rdn_pg_rm +AND_zpzz 00000100 .. 011 010 000 ... ..... ..... @rdn_pg_rm +BIC_zpzz 00000100 .. 011 011 000 ... ..... ..... @rdn_pg_rm + +# SVE integer add/subtract vectors (predicated) +ADD_zpzz 00000100 .. 000 000 000 ... ..... ..... @rdn_pg_rm +SUB_zpzz 00000100 .. 000 001 000 ... ..... ..... @rdn_pg_rm +SUB_zpzz 00000100 .. 000 011 000 ... ..... ..... @rdm_pg_rn # SUBR + +# SVE integer min/max/difference (predicated) +SMAX_zpzz 00000100 .. 001 000 000 ... ..... ..... @rdn_pg_rm +UMAX_zpzz 00000100 .. 001 001 000 ... ..... ..... @rdn_pg_rm +SMIN_zpzz 00000100 .. 001 010 000 ... ..... ..... @rdn_pg_rm +UMIN_zpzz 00000100 .. 001 011 000 ... ..... ..... @rdn_pg_rm +SABD_zpzz 00000100 .. 001 100 000 ... ..... ..... @rdn_pg_rm +UABD_zpzz 00000100 .. 001 101 000 ... ..... ..... @rdn_pg_rm + +# SVE integer multiply/divide (predicated) +MUL_zpzz 00000100 .. 010 000 000 ... ..... ..... @rdn_pg_rm +SMULH_zpzz 00000100 .. 010 010 000 ... ..... ..... @rdn_pg_rm +UMULH_zpzz 00000100 .. 010 011 000 ... ..... ..... @rdn_pg_rm +# Note that divide requires size >= 2; below 2 is unallocated. +SDIV_zpzz 00000100 .. 010 100 000 ... ..... ..... @rdn_pg_rm +UDIV_zpzz 00000100 .. 010 101 000 ... ..... ..... @rdn_pg_rm +SDIV_zpzz 00000100 .. 010 110 000 ... ..... ..... @rdm_pg_rn # SDIVR +UDIV_zpzz 00000100 .. 010 111 000 ... ..... ..... @rdm_pg_rn # UDIVR + ### SVE Logical - Unpredicated Group # SVE bitwise logical operations (unpredicated) -- 2.17.0 From: Richard Henderson <richard.henderson@linaro.org> Excepting MOVPRFX, which isn't a reduction. Presumably it is placed within the group because of its encoding. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20180516223007.10256-10-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/helper-sve.h | 44 ++++++++++++++++++ target/arm/sve_helper.c | 91 ++++++++++++++++++++++++++++++++++++++ target/arm/translate-sve.c | 68 ++++++++++++++++++++++++++++ target/arm/sve.decode | 22 +++++++++ 4 files changed, 225 insertions(+) diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_5(sve_udiv_zpzz_s, TCG_CALL_NO_RWG, DEF_HELPER_FLAGS_5(sve_udiv_zpzz_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve_orv_b, TCG_CALL_NO_RWG, i64, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve_orv_h, TCG_CALL_NO_RWG, i64, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve_orv_s, TCG_CALL_NO_RWG, i64, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve_orv_d, TCG_CALL_NO_RWG, i64, ptr, ptr, i32) + +DEF_HELPER_FLAGS_3(sve_eorv_b, TCG_CALL_NO_RWG, i64, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve_eorv_h, TCG_CALL_NO_RWG, i64, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve_eorv_s, TCG_CALL_NO_RWG, i64, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve_eorv_d, TCG_CALL_NO_RWG, i64, ptr, ptr, i32) + +DEF_HELPER_FLAGS_3(sve_andv_b, TCG_CALL_NO_RWG, i64, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve_andv_h, TCG_CALL_NO_RWG, i64, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve_andv_s, TCG_CALL_NO_RWG, i64, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve_andv_d, TCG_CALL_NO_RWG, i64, ptr, ptr, i32) + +DEF_HELPER_FLAGS_3(sve_saddv_b, TCG_CALL_NO_RWG, i64, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve_saddv_h, TCG_CALL_NO_RWG, i64, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve_saddv_s, TCG_CALL_NO_RWG, i64, ptr, ptr, i32) + +DEF_HELPER_FLAGS_3(sve_uaddv_b, TCG_CALL_NO_RWG, i64, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve_uaddv_h, TCG_CALL_NO_RWG, i64, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve_uaddv_s, TCG_CALL_NO_RWG, i64, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve_uaddv_d, TCG_CALL_NO_RWG, i64, ptr, ptr, i32) + +DEF_HELPER_FLAGS_3(sve_smaxv_b, TCG_CALL_NO_RWG, i64, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve_smaxv_h, TCG_CALL_NO_RWG, i64, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve_smaxv_s, TCG_CALL_NO_RWG, i64, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve_smaxv_d, TCG_CALL_NO_RWG, i64, ptr, ptr, i32) + +DEF_HELPER_FLAGS_3(sve_umaxv_b, TCG_CALL_NO_RWG, i64, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve_umaxv_h, TCG_CALL_NO_RWG, i64, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve_umaxv_s, TCG_CALL_NO_RWG, i64, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve_umaxv_d, TCG_CALL_NO_RWG, i64, ptr, ptr, i32) + +DEF_HELPER_FLAGS_3(sve_sminv_b, TCG_CALL_NO_RWG, i64, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve_sminv_h, TCG_CALL_NO_RWG, i64, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve_sminv_s, TCG_CALL_NO_RWG, i64, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve_sminv_d, TCG_CALL_NO_RWG, i64, ptr, ptr, i32) + +DEF_HELPER_FLAGS_3(sve_uminv_b, TCG_CALL_NO_RWG, i64, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve_uminv_h, TCG_CALL_NO_RWG, i64, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve_uminv_s, TCG_CALL_NO_RWG, i64, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve_uminv_d, TCG_CALL_NO_RWG, i64, ptr, ptr, i32) + DEF_HELPER_FLAGS_5(sve_and_pppp, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve_bic_pppp, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve_eor_pppp, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -XXX,XX +XXX,XX @@ DO_ZPZZ_D(sve_udiv_zpzz_d, uint64_t, DO_DIV) #undef DO_ZPZZ #undef DO_ZPZZ_D + +/* Two-operand reduction expander, controlled by a predicate. + * The difference between TYPERED and TYPERET has to do with + * sign-extension. E.g. for SMAX, TYPERED must be signed, + * but TYPERET must be unsigned so that e.g. a 32-bit value + * is not sign-extended to the ABI uint64_t return type. + */ +/* ??? If we were to vectorize this by hand the reduction ordering + * would change. For integer operands, this is perfectly fine. + */ +#define DO_VPZ(NAME, TYPEELT, TYPERED, TYPERET, H, INIT, OP) \ +uint64_t HELPER(NAME)(void *vn, void *vg, uint32_t desc) \ +{ \ + intptr_t i, opr_sz = simd_oprsz(desc); \ + TYPERED ret = INIT; \ + for (i = 0; i < opr_sz; ) { \ + uint16_t pg = *(uint16_t *)(vg + H1_2(i >> 3)); \ + do { \ + if (pg & 1) { \ + TYPEELT nn = *(TYPEELT *)(vn + H(i)); \ + ret = OP(ret, nn); \ + } \ + i += sizeof(TYPEELT), pg >>= sizeof(TYPEELT); \ + } while (i & 15); \ + } \ + return (TYPERET)ret; \ +} + +#define DO_VPZ_D(NAME, TYPEE, TYPER, INIT, OP) \ +uint64_t HELPER(NAME)(void *vn, void *vg, uint32_t desc) \ +{ \ + intptr_t i, opr_sz = simd_oprsz(desc) / 8; \ + TYPEE *n = vn; \ + uint8_t *pg = vg; \ + TYPER ret = INIT; \ + for (i = 0; i < opr_sz; i += 1) { \ + if (pg[H1(i)] & 1) { \ + TYPEE nn = n[i]; \ + ret = OP(ret, nn); \ + } \ + } \ + return ret; \ +} + +DO_VPZ(sve_orv_b, uint8_t, uint8_t, uint8_t, H1, 0, DO_ORR) +DO_VPZ(sve_orv_h, uint16_t, uint16_t, uint16_t, H1_2, 0, DO_ORR) +DO_VPZ(sve_orv_s, uint32_t, uint32_t, uint32_t, H1_4, 0, DO_ORR) +DO_VPZ_D(sve_orv_d, uint64_t, uint64_t, 0, DO_ORR) + +DO_VPZ(sve_eorv_b, uint8_t, uint8_t, uint8_t, H1, 0, DO_EOR) +DO_VPZ(sve_eorv_h, uint16_t, uint16_t, uint16_t, H1_2, 0, DO_EOR) +DO_VPZ(sve_eorv_s, uint32_t, uint32_t, uint32_t, H1_4, 0, DO_EOR) +DO_VPZ_D(sve_eorv_d, uint64_t, uint64_t, 0, DO_EOR) + +DO_VPZ(sve_andv_b, uint8_t, uint8_t, uint8_t, H1, -1, DO_AND) +DO_VPZ(sve_andv_h, uint16_t, uint16_t, uint16_t, H1_2, -1, DO_AND) +DO_VPZ(sve_andv_s, uint32_t, uint32_t, uint32_t, H1_4, -1, DO_AND) +DO_VPZ_D(sve_andv_d, uint64_t, uint64_t, -1, DO_AND) + +DO_VPZ(sve_saddv_b, int8_t, uint64_t, uint64_t, H1, 0, DO_ADD) +DO_VPZ(sve_saddv_h, int16_t, uint64_t, uint64_t, H1_2, 0, DO_ADD) +DO_VPZ(sve_saddv_s, int32_t, uint64_t, uint64_t, H1_4, 0, DO_ADD) + +DO_VPZ(sve_uaddv_b, uint8_t, uint64_t, uint64_t, H1, 0, DO_ADD) +DO_VPZ(sve_uaddv_h, uint16_t, uint64_t, uint64_t, H1_2, 0, DO_ADD) +DO_VPZ(sve_uaddv_s, uint32_t, uint64_t, uint64_t, H1_4, 0, DO_ADD) +DO_VPZ_D(sve_uaddv_d, uint64_t, uint64_t, 0, DO_ADD) + +DO_VPZ(sve_smaxv_b, int8_t, int8_t, uint8_t, H1, INT8_MIN, DO_MAX) +DO_VPZ(sve_smaxv_h, int16_t, int16_t, uint16_t, H1_2, INT16_MIN, DO_MAX) +DO_VPZ(sve_smaxv_s, int32_t, int32_t, uint32_t, H1_4, INT32_MIN, DO_MAX) +DO_VPZ_D(sve_smaxv_d, int64_t, int64_t, INT64_MIN, DO_MAX) + +DO_VPZ(sve_umaxv_b, uint8_t, uint8_t, uint8_t, H1, 0, DO_MAX) +DO_VPZ(sve_umaxv_h, uint16_t, uint16_t, uint16_t, H1_2, 0, DO_MAX) +DO_VPZ(sve_umaxv_s, uint32_t, uint32_t, uint32_t, H1_4, 0, DO_MAX) +DO_VPZ_D(sve_umaxv_d, uint64_t, uint64_t, 0, DO_MAX) + +DO_VPZ(sve_sminv_b, int8_t, int8_t, uint8_t, H1, INT8_MAX, DO_MIN) +DO_VPZ(sve_sminv_h, int16_t, int16_t, uint16_t, H1_2, INT16_MAX, DO_MIN) +DO_VPZ(sve_sminv_s, int32_t, int32_t, uint32_t, H1_4, INT32_MAX, DO_MIN) +DO_VPZ_D(sve_sminv_d, int64_t, int64_t, INT64_MAX, DO_MIN) + +DO_VPZ(sve_uminv_b, uint8_t, uint8_t, uint8_t, H1, -1, DO_MIN) +DO_VPZ(sve_uminv_h, uint16_t, uint16_t, uint16_t, H1_2, -1, DO_MIN) +DO_VPZ(sve_uminv_s, uint32_t, uint32_t, uint32_t, H1_4, -1, DO_MIN) +DO_VPZ_D(sve_uminv_d, uint64_t, uint64_t, -1, DO_MIN) + +#undef DO_VPZ +#undef DO_VPZ_D + #undef DO_AND #undef DO_ORR #undef DO_EOR diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool trans_UDIV_zpzz(DisasContext *s, arg_rprr_esz *a, uint32_t insn) #undef DO_ZPZZ +/* + *** SVE Integer Reduction Group + */ + +typedef void gen_helper_gvec_reduc(TCGv_i64, TCGv_ptr, TCGv_ptr, TCGv_i32); +static bool do_vpz_ool(DisasContext *s, arg_rpr_esz *a, + gen_helper_gvec_reduc *fn) +{ + unsigned vsz = vec_full_reg_size(s); + TCGv_ptr t_zn, t_pg; + TCGv_i32 desc; + TCGv_i64 temp; + + if (fn == NULL) { + return false; + } + if (!sve_access_check(s)) { + return true; + } + + desc = tcg_const_i32(simd_desc(vsz, vsz, 0)); + temp = tcg_temp_new_i64(); + t_zn = tcg_temp_new_ptr(); + t_pg = tcg_temp_new_ptr(); + + tcg_gen_addi_ptr(t_zn, cpu_env, vec_full_reg_offset(s, a->rn)); + tcg_gen_addi_ptr(t_pg, cpu_env, pred_full_reg_offset(s, a->pg)); + fn(temp, t_zn, t_pg, desc); + tcg_temp_free_ptr(t_zn); + tcg_temp_free_ptr(t_pg); + tcg_temp_free_i32(desc); + + write_fp_dreg(s, a->rd, temp); + tcg_temp_free_i64(temp); + return true; +} + +#define DO_VPZ(NAME, name) \ +static bool trans_##NAME(DisasContext *s, arg_rpr_esz *a, uint32_t insn) \ +{ \ + static gen_helper_gvec_reduc * const fns[4] = { \ + gen_helper_sve_##name##_b, gen_helper_sve_##name##_h, \ + gen_helper_sve_##name##_s, gen_helper_sve_##name##_d, \ + }; \ + return do_vpz_ool(s, a, fns[a->esz]); \ +} + +DO_VPZ(ORV, orv) +DO_VPZ(ANDV, andv) +DO_VPZ(EORV, eorv) + +DO_VPZ(UADDV, uaddv) +DO_VPZ(SMAXV, smaxv) +DO_VPZ(UMAXV, umaxv) +DO_VPZ(SMINV, sminv) +DO_VPZ(UMINV, uminv) + +static bool trans_SADDV(DisasContext *s, arg_rpr_esz *a, uint32_t insn) +{ + static gen_helper_gvec_reduc * const fns[4] = { + gen_helper_sve_saddv_b, gen_helper_sve_saddv_h, + gen_helper_sve_saddv_s, NULL + }; + return do_vpz_ool(s, a, fns[a->esz]); +} + +#undef DO_VPZ + /* *** SVE Predicate Logical Operations Group */ diff --git a/target/arm/sve.decode b/target/arm/sve.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -XXX,XX +XXX,XX @@ &rr_esz rd rn esz &rri rd rn imm &rrr_esz rd rn rm esz +&rpr_esz rd pg rn esz &rprr_s rd pg rn rm s &rprr_esz rd pg rn rm esz @@ -XXX,XX +XXX,XX @@ @rdm_pg_rn ........ esz:2 ... ... ... pg:3 rn:5 rd:5 \ &rprr_esz rm=%reg_movprfx +# One register operand, with governing predicate, vector element size +@rd_pg_rn ........ esz:2 ... ... ... pg:3 rn:5 rd:5 &rpr_esz + # Basic Load/Store with 9-bit immediate offset @pd_rn_i9 ........ ........ ...... rn:5 . rd:4 \ &rri imm=%imm9_16_10 @@ -XXX,XX +XXX,XX @@ UDIV_zpzz 00000100 .. 010 101 000 ... ..... ..... @rdn_pg_rm SDIV_zpzz 00000100 .. 010 110 000 ... ..... ..... @rdm_pg_rn # SDIVR UDIV_zpzz 00000100 .. 010 111 000 ... ..... ..... @rdm_pg_rn # UDIVR +### SVE Integer Reduction Group + +# SVE bitwise logical reduction (predicated) +ORV 00000100 .. 011 000 001 ... ..... ..... @rd_pg_rn +EORV 00000100 .. 011 001 001 ... ..... ..... @rd_pg_rn +ANDV 00000100 .. 011 010 001 ... ..... ..... @rd_pg_rn + +# SVE integer add reduction (predicated) +# Note that saddv requires size != 3. +UADDV 00000100 .. 000 001 001 ... ..... ..... @rd_pg_rn +SADDV 00000100 .. 000 000 001 ... ..... ..... @rd_pg_rn + +# SVE integer min/max reduction (predicated) +SMAXV 00000100 .. 001 000 001 ... ..... ..... @rd_pg_rn +UMAXV 00000100 .. 001 001 001 ... ..... ..... @rd_pg_rn +SMINV 00000100 .. 001 010 001 ... ..... ..... @rd_pg_rn +UMINV 00000100 .. 001 011 001 ... ..... ..... @rd_pg_rn + ### SVE Logical - Unpredicated Group # SVE bitwise logical operations (unpredicated) -- 2.17.0 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20180516223007.10256-11-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/helper-sve.h | 25 ++++ target/arm/sve_helper.c | 264 +++++++++++++++++++++++++++++++++++++ target/arm/translate-sve.c | 130 ++++++++++++++++++ target/arm/sve.decode | 26 ++++ 4 files changed, 445 insertions(+) diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_3(sve_uminv_h, TCG_CALL_NO_RWG, i64, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve_uminv_s, TCG_CALL_NO_RWG, i64, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve_uminv_d, TCG_CALL_NO_RWG, i64, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve_clr_b, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve_clr_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve_clr_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve_clr_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve_asr_zpzi_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve_asr_zpzi_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve_asr_zpzi_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve_asr_zpzi_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve_lsr_zpzi_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve_lsr_zpzi_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve_lsr_zpzi_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve_lsr_zpzi_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve_lsl_zpzi_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve_lsl_zpzi_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve_lsl_zpzi_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve_lsl_zpzi_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve_asrd_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve_asrd_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve_asrd_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve_asrd_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_5(sve_and_pppp, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve_bic_pppp, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve_eor_pppp, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -XXX,XX +XXX,XX @@ uint32_t HELPER(sve_predtest)(void *vd, void *vg, uint32_t words) return flags; } +/* Expand active predicate bits to bytes, for byte elements. + * for (i = 0; i < 256; ++i) { + * unsigned long m = 0; + * for (j = 0; j < 8; j++) { + * if ((i >> j) & 1) { + * m |= 0xfful << (j << 3); + * } + * } + * printf("0x%016lx,\n", m); + * } + */ +static inline uint64_t expand_pred_b(uint8_t byte) +{ + static const uint64_t word[256] = { + 0x0000000000000000, 0x00000000000000ff, 0x000000000000ff00, + 0x000000000000ffff, 0x0000000000ff0000, 0x0000000000ff00ff, + 0x0000000000ffff00, 0x0000000000ffffff, 0x00000000ff000000, + 0x00000000ff0000ff, 0x00000000ff00ff00, 0x00000000ff00ffff, + 0x00000000ffff0000, 0x00000000ffff00ff, 0x00000000ffffff00, + 0x00000000ffffffff, 0x000000ff00000000, 0x000000ff000000ff, + 0x000000ff0000ff00, 0x000000ff0000ffff, 0x000000ff00ff0000, + 0x000000ff00ff00ff, 0x000000ff00ffff00, 0x000000ff00ffffff, + 0x000000ffff000000, 0x000000ffff0000ff, 0x000000ffff00ff00, + 0x000000ffff00ffff, 0x000000ffffff0000, 0x000000ffffff00ff, + 0x000000ffffffff00, 0x000000ffffffffff, 0x0000ff0000000000, + 0x0000ff00000000ff, 0x0000ff000000ff00, 0x0000ff000000ffff, + 0x0000ff0000ff0000, 0x0000ff0000ff00ff, 0x0000ff0000ffff00, + 0x0000ff0000ffffff, 0x0000ff00ff000000, 0x0000ff00ff0000ff, + 0x0000ff00ff00ff00, 0x0000ff00ff00ffff, 0x0000ff00ffff0000, + 0x0000ff00ffff00ff, 0x0000ff00ffffff00, 0x0000ff00ffffffff, + 0x0000ffff00000000, 0x0000ffff000000ff, 0x0000ffff0000ff00, + 0x0000ffff0000ffff, 0x0000ffff00ff0000, 0x0000ffff00ff00ff, + 0x0000ffff00ffff00, 0x0000ffff00ffffff, 0x0000ffffff000000, + 0x0000ffffff0000ff, 0x0000ffffff00ff00, 0x0000ffffff00ffff, + 0x0000ffffffff0000, 0x0000ffffffff00ff, 0x0000ffffffffff00, + 0x0000ffffffffffff, 0x00ff000000000000, 0x00ff0000000000ff, + 0x00ff00000000ff00, 0x00ff00000000ffff, 0x00ff000000ff0000, + 0x00ff000000ff00ff, 0x00ff000000ffff00, 0x00ff000000ffffff, + 0x00ff0000ff000000, 0x00ff0000ff0000ff, 0x00ff0000ff00ff00, + 0x00ff0000ff00ffff, 0x00ff0000ffff0000, 0x00ff0000ffff00ff, + 0x00ff0000ffffff00, 0x00ff0000ffffffff, 0x00ff00ff00000000, + 0x00ff00ff000000ff, 0x00ff00ff0000ff00, 0x00ff00ff0000ffff, + 0x00ff00ff00ff0000, 0x00ff00ff00ff00ff, 0x00ff00ff00ffff00, + 0x00ff00ff00ffffff, 0x00ff00ffff000000, 0x00ff00ffff0000ff, + 0x00ff00ffff00ff00, 0x00ff00ffff00ffff, 0x00ff00ffffff0000, + 0x00ff00ffffff00ff, 0x00ff00ffffffff00, 0x00ff00ffffffffff, + 0x00ffff0000000000, 0x00ffff00000000ff, 0x00ffff000000ff00, + 0x00ffff000000ffff, 0x00ffff0000ff0000, 0x00ffff0000ff00ff, + 0x00ffff0000ffff00, 0x00ffff0000ffffff, 0x00ffff00ff000000, + 0x00ffff00ff0000ff, 0x00ffff00ff00ff00, 0x00ffff00ff00ffff, + 0x00ffff00ffff0000, 0x00ffff00ffff00ff, 0x00ffff00ffffff00, + 0x00ffff00ffffffff, 0x00ffffff00000000, 0x00ffffff000000ff, + 0x00ffffff0000ff00, 0x00ffffff0000ffff, 0x00ffffff00ff0000, + 0x00ffffff00ff00ff, 0x00ffffff00ffff00, 0x00ffffff00ffffff, + 0x00ffffffff000000, 0x00ffffffff0000ff, 0x00ffffffff00ff00, + 0x00ffffffff00ffff, 0x00ffffffffff0000, 0x00ffffffffff00ff, + 0x00ffffffffffff00, 0x00ffffffffffffff, 0xff00000000000000, + 0xff000000000000ff, 0xff0000000000ff00, 0xff0000000000ffff, + 0xff00000000ff0000, 0xff00000000ff00ff, 0xff00000000ffff00, + 0xff00000000ffffff, 0xff000000ff000000, 0xff000000ff0000ff, + 0xff000000ff00ff00, 0xff000000ff00ffff, 0xff000000ffff0000, + 0xff000000ffff00ff, 0xff000000ffffff00, 0xff000000ffffffff, + 0xff0000ff00000000, 0xff0000ff000000ff, 0xff0000ff0000ff00, + 0xff0000ff0000ffff, 0xff0000ff00ff0000, 0xff0000ff00ff00ff, + 0xff0000ff00ffff00, 0xff0000ff00ffffff, 0xff0000ffff000000, + 0xff0000ffff0000ff, 0xff0000ffff00ff00, 0xff0000ffff00ffff, + 0xff0000ffffff0000, 0xff0000ffffff00ff, 0xff0000ffffffff00, + 0xff0000ffffffffff, 0xff00ff0000000000, 0xff00ff00000000ff, + 0xff00ff000000ff00, 0xff00ff000000ffff, 0xff00ff0000ff0000, + 0xff00ff0000ff00ff, 0xff00ff0000ffff00, 0xff00ff0000ffffff, + 0xff00ff00ff000000, 0xff00ff00ff0000ff, 0xff00ff00ff00ff00, + 0xff00ff00ff00ffff, 0xff00ff00ffff0000, 0xff00ff00ffff00ff, + 0xff00ff00ffffff00, 0xff00ff00ffffffff, 0xff00ffff00000000, + 0xff00ffff000000ff, 0xff00ffff0000ff00, 0xff00ffff0000ffff, + 0xff00ffff00ff0000, 0xff00ffff00ff00ff, 0xff00ffff00ffff00, + 0xff00ffff00ffffff, 0xff00ffffff000000, 0xff00ffffff0000ff, + 0xff00ffffff00ff00, 0xff00ffffff00ffff, 0xff00ffffffff0000, + 0xff00ffffffff00ff, 0xff00ffffffffff00, 0xff00ffffffffffff, + 0xffff000000000000, 0xffff0000000000ff, 0xffff00000000ff00, + 0xffff00000000ffff, 0xffff000000ff0000, 0xffff000000ff00ff, + 0xffff000000ffff00, 0xffff000000ffffff, 0xffff0000ff000000, + 0xffff0000ff0000ff, 0xffff0000ff00ff00, 0xffff0000ff00ffff, + 0xffff0000ffff0000, 0xffff0000ffff00ff, 0xffff0000ffffff00, + 0xffff0000ffffffff, 0xffff00ff00000000, 0xffff00ff000000ff, + 0xffff00ff0000ff00, 0xffff00ff0000ffff, 0xffff00ff00ff0000, + 0xffff00ff00ff00ff, 0xffff00ff00ffff00, 0xffff00ff00ffffff, + 0xffff00ffff000000, 0xffff00ffff0000ff, 0xffff00ffff00ff00, + 0xffff00ffff00ffff, 0xffff00ffffff0000, 0xffff00ffffff00ff, + 0xffff00ffffffff00, 0xffff00ffffffffff, 0xffffff0000000000, + 0xffffff00000000ff, 0xffffff000000ff00, 0xffffff000000ffff, + 0xffffff0000ff0000, 0xffffff0000ff00ff, 0xffffff0000ffff00, + 0xffffff0000ffffff, 0xffffff00ff000000, 0xffffff00ff0000ff, + 0xffffff00ff00ff00, 0xffffff00ff00ffff, 0xffffff00ffff0000, + 0xffffff00ffff00ff, 0xffffff00ffffff00, 0xffffff00ffffffff, + 0xffffffff00000000, 0xffffffff000000ff, 0xffffffff0000ff00, + 0xffffffff0000ffff, 0xffffffff00ff0000, 0xffffffff00ff00ff, + 0xffffffff00ffff00, 0xffffffff00ffffff, 0xffffffffff000000, + 0xffffffffff0000ff, 0xffffffffff00ff00, 0xffffffffff00ffff, + 0xffffffffffff0000, 0xffffffffffff00ff, 0xffffffffffffff00, + 0xffffffffffffffff, + }; + return word[byte]; +} + +/* Similarly for half-word elements. + * for (i = 0; i < 256; ++i) { + * unsigned long m = 0; + * if (i & 0xaa) { + * continue; + * } + * for (j = 0; j < 8; j += 2) { + * if ((i >> j) & 1) { + * m |= 0xfffful << (j << 3); + * } + * } + * printf("[0x%x] = 0x%016lx,\n", i, m); + * } + */ +static inline uint64_t expand_pred_h(uint8_t byte) +{ + static const uint64_t word[] = { + [0x01] = 0x000000000000ffff, [0x04] = 0x00000000ffff0000, + [0x05] = 0x00000000ffffffff, [0x10] = 0x0000ffff00000000, + [0x11] = 0x0000ffff0000ffff, [0x14] = 0x0000ffffffff0000, + [0x15] = 0x0000ffffffffffff, [0x40] = 0xffff000000000000, + [0x41] = 0xffff00000000ffff, [0x44] = 0xffff0000ffff0000, + [0x45] = 0xffff0000ffffffff, [0x50] = 0xffffffff00000000, + [0x51] = 0xffffffff0000ffff, [0x54] = 0xffffffffffff0000, + [0x55] = 0xffffffffffffffff, + }; + return word[byte & 0x55]; +} + +/* Similarly for single word elements. */ +static inline uint64_t expand_pred_s(uint8_t byte) +{ + static const uint64_t word[] = { + [0x01] = 0x00000000ffffffffull, + [0x10] = 0xffffffff00000000ull, + [0x11] = 0xffffffffffffffffull, + }; + return word[byte & 0x11]; +} + #define LOGICAL_PPPP(NAME, FUNC) \ void HELPER(NAME)(void *vd, void *vn, void *vm, void *vg, uint32_t desc) \ { \ @@ -XXX,XX +XXX,XX @@ uint32_t HELPER(sve_pnext)(void *vd, void *vg, uint32_t pred_desc) return flags; } + +/* Store zero into every active element of Zd. We will use this for two + * and three-operand predicated instructions for which logic dictates a + * zero result. In particular, logical shift by element size, which is + * otherwise undefined on the host. + * + * For element sizes smaller than uint64_t, we use tables to expand + * the N bits of the controlling predicate to a byte mask, and clear + * those bytes. + */ +void HELPER(sve_clr_b)(void *vd, void *vg, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc) / 8; + uint64_t *d = vd; + uint8_t *pg = vg; + for (i = 0; i < opr_sz; i += 1) { + d[i] &= ~expand_pred_b(pg[H1(i)]); + } +} + +void HELPER(sve_clr_h)(void *vd, void *vg, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc) / 8; + uint64_t *d = vd; + uint8_t *pg = vg; + for (i = 0; i < opr_sz; i += 1) { + d[i] &= ~expand_pred_h(pg[H1(i)]); + } +} + +void HELPER(sve_clr_s)(void *vd, void *vg, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc) / 8; + uint64_t *d = vd; + uint8_t *pg = vg; + for (i = 0; i < opr_sz; i += 1) { + d[i] &= ~expand_pred_s(pg[H1(i)]); + } +} + +void HELPER(sve_clr_d)(void *vd, void *vg, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc) / 8; + uint64_t *d = vd; + uint8_t *pg = vg; + for (i = 0; i < opr_sz; i += 1) { + if (pg[H1(i)] & 1) { + d[i] = 0; + } + } +} + +/* Three-operand expander, immediate operand, controlled by a predicate. + */ +#define DO_ZPZI(NAME, TYPE, H, OP) \ +void HELPER(NAME)(void *vd, void *vn, void *vg, uint32_t desc) \ +{ \ + intptr_t i, opr_sz = simd_oprsz(desc); \ + TYPE imm = simd_data(desc); \ + for (i = 0; i < opr_sz; ) { \ + uint16_t pg = *(uint16_t *)(vg + H1_2(i >> 3)); \ + do { \ + if (pg & 1) { \ + TYPE nn = *(TYPE *)(vn + H(i)); \ + *(TYPE *)(vd + H(i)) = OP(nn, imm); \ + } \ + i += sizeof(TYPE), pg >>= sizeof(TYPE); \ + } while (i & 15); \ + } \ +} + +/* Similarly, specialized for 64-bit operands. */ +#define DO_ZPZI_D(NAME, TYPE, OP) \ +void HELPER(NAME)(void *vd, void *vn, void *vg, uint32_t desc) \ +{ \ + intptr_t i, opr_sz = simd_oprsz(desc) / 8; \ + TYPE *d = vd, *n = vn; \ + TYPE imm = simd_data(desc); \ + uint8_t *pg = vg; \ + for (i = 0; i < opr_sz; i += 1) { \ + if (pg[H1(i)] & 1) { \ + TYPE nn = n[i]; \ + d[i] = OP(nn, imm); \ + } \ + } \ +} + +#define DO_SHR(N, M) (N >> M) +#define DO_SHL(N, M) (N << M) + +/* Arithmetic shift right for division. This rounds negative numbers + toward zero as per signed division. Therefore before shifting, + when N is negative, add 2**M-1. */ +#define DO_ASRD(N, M) ((N + (N < 0 ? ((__typeof(N))1 << M) - 1 : 0)) >> M) + +DO_ZPZI(sve_asr_zpzi_b, int8_t, H1, DO_SHR) +DO_ZPZI(sve_asr_zpzi_h, int16_t, H1_2, DO_SHR) +DO_ZPZI(sve_asr_zpzi_s, int32_t, H1_4, DO_SHR) +DO_ZPZI_D(sve_asr_zpzi_d, int64_t, DO_SHR) + +DO_ZPZI(sve_lsr_zpzi_b, uint8_t, H1, DO_SHR) +DO_ZPZI(sve_lsr_zpzi_h, uint16_t, H1_2, DO_SHR) +DO_ZPZI(sve_lsr_zpzi_s, uint32_t, H1_4, DO_SHR) +DO_ZPZI_D(sve_lsr_zpzi_d, uint64_t, DO_SHR) + +DO_ZPZI(sve_lsl_zpzi_b, uint8_t, H1, DO_SHL) +DO_ZPZI(sve_lsl_zpzi_h, uint16_t, H1_2, DO_SHL) +DO_ZPZI(sve_lsl_zpzi_s, uint32_t, H1_4, DO_SHL) +DO_ZPZI_D(sve_lsl_zpzi_d, uint64_t, DO_SHL) + +DO_ZPZI(sve_asrd_b, int8_t, H1, DO_ASRD) +DO_ZPZI(sve_asrd_h, int16_t, H1_2, DO_ASRD) +DO_ZPZI(sve_asrd_s, int32_t, H1_4, DO_ASRD) +DO_ZPZI_D(sve_asrd_d, int64_t, DO_ASRD) + +#undef DO_SHR +#undef DO_SHL +#undef DO_ASRD +#undef DO_ZPZI +#undef DO_ZPZI_D diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ #include "trace-tcg.h" #include "translate-a64.h" +/* + * Helpers for extracting complex instruction fields. + */ + +/* See e.g. ASR (immediate, predicated). + * Returns -1 for unallocated encoding; diagnose later. + */ +static int tszimm_esz(int x) +{ + x >>= 3; /* discard imm3 */ + return 31 - clz32(x); +} + +static int tszimm_shr(int x) +{ + return (16 << tszimm_esz(x)) - x; +} + +/* See e.g. LSL (immediate, predicated). */ +static int tszimm_shl(int x) +{ + return x - (8 << tszimm_esz(x)); +} + /* * Include the generated decoder. */ @@ -XXX,XX +XXX,XX @@ static bool trans_SADDV(DisasContext *s, arg_rpr_esz *a, uint32_t insn) #undef DO_VPZ +/* + *** SVE Shift by Immediate - Predicated Group + */ + +/* Store zero into every active element of Zd. We will use this for two + * and three-operand predicated instructions for which logic dictates a + * zero result. + */ +static bool do_clr_zp(DisasContext *s, int rd, int pg, int esz) +{ + static gen_helper_gvec_2 * const fns[4] = { + gen_helper_sve_clr_b, gen_helper_sve_clr_h, + gen_helper_sve_clr_s, gen_helper_sve_clr_d, + }; + if (sve_access_check(s)) { + unsigned vsz = vec_full_reg_size(s); + tcg_gen_gvec_2_ool(vec_full_reg_offset(s, rd), + pred_full_reg_offset(s, pg), + vsz, vsz, 0, fns[esz]); + } + return true; +} + +static bool do_zpzi_ool(DisasContext *s, arg_rpri_esz *a, + gen_helper_gvec_3 *fn) +{ + if (sve_access_check(s)) { + unsigned vsz = vec_full_reg_size(s); + tcg_gen_gvec_3_ool(vec_full_reg_offset(s, a->rd), + vec_full_reg_offset(s, a->rn), + pred_full_reg_offset(s, a->pg), + vsz, vsz, a->imm, fn); + } + return true; +} + +static bool trans_ASR_zpzi(DisasContext *s, arg_rpri_esz *a, uint32_t insn) +{ + static gen_helper_gvec_3 * const fns[4] = { + gen_helper_sve_asr_zpzi_b, gen_helper_sve_asr_zpzi_h, + gen_helper_sve_asr_zpzi_s, gen_helper_sve_asr_zpzi_d, + }; + if (a->esz < 0) { + /* Invalid tsz encoding -- see tszimm_esz. */ + return false; + } + /* Shift by element size is architecturally valid. For + arithmetic right-shift, it's the same as by one less. */ + a->imm = MIN(a->imm, (8 << a->esz) - 1); + return do_zpzi_ool(s, a, fns[a->esz]); +} + +static bool trans_LSR_zpzi(DisasContext *s, arg_rpri_esz *a, uint32_t insn) +{ + static gen_helper_gvec_3 * const fns[4] = { + gen_helper_sve_lsr_zpzi_b, gen_helper_sve_lsr_zpzi_h, + gen_helper_sve_lsr_zpzi_s, gen_helper_sve_lsr_zpzi_d, + }; + if (a->esz < 0) { + return false; + } + /* Shift by element size is architecturally valid. + For logical shifts, it is a zeroing operation. */ + if (a->imm >= (8 << a->esz)) { + return do_clr_zp(s, a->rd, a->pg, a->esz); + } else { + return do_zpzi_ool(s, a, fns[a->esz]); + } +} + +static bool trans_LSL_zpzi(DisasContext *s, arg_rpri_esz *a, uint32_t insn) +{ + static gen_helper_gvec_3 * const fns[4] = { + gen_helper_sve_lsl_zpzi_b, gen_helper_sve_lsl_zpzi_h, + gen_helper_sve_lsl_zpzi_s, gen_helper_sve_lsl_zpzi_d, + }; + if (a->esz < 0) { + return false; + } + /* Shift by element size is architecturally valid. + For logical shifts, it is a zeroing operation. */ + if (a->imm >= (8 << a->esz)) { + return do_clr_zp(s, a->rd, a->pg, a->esz); + } else { + return do_zpzi_ool(s, a, fns[a->esz]); + } +} + +static bool trans_ASRD(DisasContext *s, arg_rpri_esz *a, uint32_t insn) +{ + static gen_helper_gvec_3 * const fns[4] = { + gen_helper_sve_asrd_b, gen_helper_sve_asrd_h, + gen_helper_sve_asrd_s, gen_helper_sve_asrd_d, + }; + if (a->esz < 0) { + return false; + } + /* Shift by element size is architecturally valid. For arithmetic + right shift for division, it is a zeroing operation. */ + if (a->imm >= (8 << a->esz)) { + return do_clr_zp(s, a->rd, a->pg, a->esz); + } else { + return do_zpzi_ool(s, a, fns[a->esz]); + } +} + /* *** SVE Predicate Logical Operations Group */ diff --git a/target/arm/sve.decode b/target/arm/sve.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -XXX,XX +XXX,XX @@ ########################################################################### # Named fields. These are primarily for disjoint fields. +%imm6_22_5 22:1 5:5 %imm9_16_10 16:s6 10:3 +# A combination of tsz:imm3 -- extract esize. +%tszimm_esz 22:2 5:5 !function=tszimm_esz +# A combination of tsz:imm3 -- extract (2 * esize) - (tsz:imm3) +%tszimm_shr 22:2 5:5 !function=tszimm_shr +# A combination of tsz:imm3 -- extract (tsz:imm3) - esize +%tszimm_shl 22:2 5:5 !function=tszimm_shl + # Either a copy of rd (at bit 0), or a different source # as propagated via the MOVPRFX instruction. %reg_movprfx 0:5 @@ -XXX,XX +XXX,XX @@ &rpr_esz rd pg rn esz &rprr_s rd pg rn rm s &rprr_esz rd pg rn rm esz +&rpri_esz rd pg rn imm esz ########################################################################### # Named instruction formats. These are generally used to @@ -XXX,XX +XXX,XX @@ # One register operand, with governing predicate, vector element size @rd_pg_rn ........ esz:2 ... ... ... pg:3 rn:5 rd:5 &rpr_esz +# Two register operand, one immediate operand, with predicate, +# element size encoded as TSZHL. User must fill in imm. +@rdn_pg_tszimm ........ .. ... ... ... pg:3 ..... rd:5 \ + &rpri_esz rn=%reg_movprfx esz=%tszimm_esz + # Basic Load/Store with 9-bit immediate offset @pd_rn_i9 ........ ........ ...... rn:5 . rd:4 \ &rri imm=%imm9_16_10 @@ -XXX,XX +XXX,XX @@ UMAXV 00000100 .. 001 001 001 ... ..... ..... @rd_pg_rn SMINV 00000100 .. 001 010 001 ... ..... ..... @rd_pg_rn UMINV 00000100 .. 001 011 001 ... ..... ..... @rd_pg_rn +### SVE Shift by Immediate - Predicated Group + +# SVE bitwise shift by immediate (predicated) +ASR_zpzi 00000100 .. 000 000 100 ... .. ... ..... \ + @rdn_pg_tszimm imm=%tszimm_shr +LSR_zpzi 00000100 .. 000 001 100 ... .. ... ..... \ + @rdn_pg_tszimm imm=%tszimm_shr +LSL_zpzi 00000100 .. 000 011 100 ... .. ... ..... \ + @rdn_pg_tszimm imm=%tszimm_shl +ASRD 00000100 .. 000 100 100 ... .. ... ..... \ + @rdn_pg_tszimm imm=%tszimm_shr + ### SVE Logical - Unpredicated Group # SVE bitwise logical operations (unpredicated) -- 2.17.0 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20180516223007.10256-12-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/helper-sve.h | 27 +++++++++++++++++++++++++++ target/arm/sve_helper.c | 25 +++++++++++++++++++++++++ target/arm/translate-sve.c | 4 ++++ target/arm/sve.decode | 8 ++++++++ 4 files changed, 64 insertions(+) diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_5(sve_udiv_zpzz_s, TCG_CALL_NO_RWG, DEF_HELPER_FLAGS_5(sve_udiv_zpzz_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve_asr_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve_asr_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve_asr_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve_asr_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve_lsr_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve_lsr_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve_lsr_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve_lsr_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve_lsl_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve_lsl_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve_lsl_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve_lsl_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_3(sve_orv_b, TCG_CALL_NO_RWG, i64, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve_orv_h, TCG_CALL_NO_RWG, i64, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve_orv_s, TCG_CALL_NO_RWG, i64, ptr, ptr, i32) diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -XXX,XX +XXX,XX @@ DO_ZPZZ_D(sve_sdiv_zpzz_d, int64_t, DO_DIV) DO_ZPZZ(sve_udiv_zpzz_s, uint32_t, H1_4, DO_DIV) DO_ZPZZ_D(sve_udiv_zpzz_d, uint64_t, DO_DIV) +/* Note that all bits of the shift are significant + and not modulo the element size. */ +#define DO_ASR(N, M) (N >> MIN(M, sizeof(N) * 8 - 1)) +#define DO_LSR(N, M) (M < sizeof(N) * 8 ? N >> M : 0) +#define DO_LSL(N, M) (M < sizeof(N) * 8 ? N << M : 0) + +DO_ZPZZ(sve_asr_zpzz_b, int8_t, H1, DO_ASR) +DO_ZPZZ(sve_lsr_zpzz_b, uint8_t, H1_2, DO_LSR) +DO_ZPZZ(sve_lsl_zpzz_b, uint8_t, H1_4, DO_LSL) + +DO_ZPZZ(sve_asr_zpzz_h, int16_t, H1, DO_ASR) +DO_ZPZZ(sve_lsr_zpzz_h, uint16_t, H1_2, DO_LSR) +DO_ZPZZ(sve_lsl_zpzz_h, uint16_t, H1_4, DO_LSL) + +DO_ZPZZ(sve_asr_zpzz_s, int32_t, H1, DO_ASR) +DO_ZPZZ(sve_lsr_zpzz_s, uint32_t, H1_2, DO_LSR) +DO_ZPZZ(sve_lsl_zpzz_s, uint32_t, H1_4, DO_LSL) + +DO_ZPZZ_D(sve_asr_zpzz_d, int64_t, DO_ASR) +DO_ZPZZ_D(sve_lsr_zpzz_d, uint64_t, DO_LSR) +DO_ZPZZ_D(sve_lsl_zpzz_d, uint64_t, DO_LSL) + #undef DO_ZPZZ #undef DO_ZPZZ_D @@ -XXX,XX +XXX,XX @@ DO_VPZ_D(sve_uminv_d, uint64_t, uint64_t, -1, DO_MIN) #undef DO_ABD #undef DO_MUL #undef DO_DIV +#undef DO_ASR +#undef DO_LSR +#undef DO_LSL /* Similar to the ARM LastActiveElement pseudocode function, except the result is multiplied by the element size. This includes the not found diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ DO_ZPZZ(MUL, mul) DO_ZPZZ(SMULH, smulh) DO_ZPZZ(UMULH, umulh) +DO_ZPZZ(ASR, asr) +DO_ZPZZ(LSR, lsr) +DO_ZPZZ(LSL, lsl) + static bool trans_SDIV_zpzz(DisasContext *s, arg_rprr_esz *a, uint32_t insn) { static gen_helper_gvec_4 * const fns[4] = { diff --git a/target/arm/sve.decode b/target/arm/sve.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -XXX,XX +XXX,XX @@ LSL_zpzi 00000100 .. 000 011 100 ... .. ... ..... \ ASRD 00000100 .. 000 100 100 ... .. ... ..... \ @rdn_pg_tszimm imm=%tszimm_shr +# SVE bitwise shift by vector (predicated) +ASR_zpzz 00000100 .. 010 000 100 ... ..... ..... @rdn_pg_rm +LSR_zpzz 00000100 .. 010 001 100 ... ..... ..... @rdn_pg_rm +LSL_zpzz 00000100 .. 010 011 100 ... ..... ..... @rdn_pg_rm +ASR_zpzz 00000100 .. 010 100 100 ... ..... ..... @rdm_pg_rn # ASRR +LSR_zpzz 00000100 .. 010 101 100 ... ..... ..... @rdm_pg_rn # LSRR +LSL_zpzz 00000100 .. 010 111 100 ... ..... ..... @rdm_pg_rn # LSLR + ### SVE Logical - Unpredicated Group # SVE bitwise logical operations (unpredicated) -- 2.17.0 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20180516223007.10256-13-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/helper-sve.h | 21 +++++++++++++++++++++ target/arm/sve_helper.c | 35 +++++++++++++++++++++++++++++++++++ target/arm/translate-sve.c | 24 ++++++++++++++++++++++++ target/arm/sve.decode | 6 ++++++ 4 files changed, 86 insertions(+) diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_5(sve_lsl_zpzz_s, TCG_CALL_NO_RWG, DEF_HELPER_FLAGS_5(sve_lsl_zpzz_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve_asr_zpzw_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve_asr_zpzw_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve_asr_zpzw_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve_lsr_zpzw_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve_lsr_zpzw_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve_lsr_zpzw_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve_lsl_zpzw_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve_lsl_zpzw_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve_lsl_zpzw_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_3(sve_orv_b, TCG_CALL_NO_RWG, i64, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve_orv_h, TCG_CALL_NO_RWG, i64, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve_orv_s, TCG_CALL_NO_RWG, i64, ptr, ptr, i32) diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -XXX,XX +XXX,XX @@ DO_ZPZZ_D(sve_lsl_zpzz_d, uint64_t, DO_LSL) #undef DO_ZPZZ #undef DO_ZPZZ_D +/* Three-operand expander, controlled by a predicate, in which the + * third operand is "wide". That is, for D = N op M, the same 64-bit + * value of M is used with all of the narrower values of N. + */ +#define DO_ZPZW(NAME, TYPE, TYPEW, H, OP) \ +void HELPER(NAME)(void *vd, void *vn, void *vm, void *vg, uint32_t desc) \ +{ \ + intptr_t i, opr_sz = simd_oprsz(desc); \ + for (i = 0; i < opr_sz; ) { \ + uint8_t pg = *(uint8_t *)(vg + H1(i >> 3)); \ + TYPEW mm = *(TYPEW *)(vm + i); \ + do { \ + if (pg & 1) { \ + TYPE nn = *(TYPE *)(vn + H(i)); \ + *(TYPE *)(vd + H(i)) = OP(nn, mm); \ + } \ + i += sizeof(TYPE), pg >>= sizeof(TYPE); \ + } while (i & 7); \ + } \ +} + +DO_ZPZW(sve_asr_zpzw_b, int8_t, uint64_t, H1, DO_ASR) +DO_ZPZW(sve_lsr_zpzw_b, uint8_t, uint64_t, H1, DO_LSR) +DO_ZPZW(sve_lsl_zpzw_b, uint8_t, uint64_t, H1, DO_LSL) + +DO_ZPZW(sve_asr_zpzw_h, int16_t, uint64_t, H1_2, DO_ASR) +DO_ZPZW(sve_lsr_zpzw_h, uint16_t, uint64_t, H1_2, DO_LSR) +DO_ZPZW(sve_lsl_zpzw_h, uint16_t, uint64_t, H1_2, DO_LSL) + +DO_ZPZW(sve_asr_zpzw_s, int32_t, uint64_t, H1_4, DO_ASR) +DO_ZPZW(sve_lsr_zpzw_s, uint32_t, uint64_t, H1_4, DO_LSR) +DO_ZPZW(sve_lsl_zpzw_s, uint32_t, uint64_t, H1_4, DO_LSL) + +#undef DO_ZPZW + /* Two-operand reduction expander, controlled by a predicate. * The difference between TYPERED and TYPERET has to do with * sign-extension. E.g. for SMAX, TYPERED must be signed, diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool trans_ASRD(DisasContext *s, arg_rpri_esz *a, uint32_t insn) } } +/* + *** SVE Bitwise Shift - Predicated Group + */ + +#define DO_ZPZW(NAME, name) \ +static bool trans_##NAME##_zpzw(DisasContext *s, arg_rprr_esz *a, \ + uint32_t insn) \ +{ \ + static gen_helper_gvec_4 * const fns[3] = { \ + gen_helper_sve_##name##_zpzw_b, gen_helper_sve_##name##_zpzw_h, \ + gen_helper_sve_##name##_zpzw_s, \ + }; \ + if (a->esz < 0 || a->esz >= 3) { \ + return false; \ + } \ + return do_zpzz_ool(s, a, fns[a->esz]); \ +} + +DO_ZPZW(ASR, asr) +DO_ZPZW(LSR, lsr) +DO_ZPZW(LSL, lsl) + +#undef DO_ZPZW + /* *** SVE Predicate Logical Operations Group */ diff --git a/target/arm/sve.decode b/target/arm/sve.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -XXX,XX +XXX,XX @@ ASR_zpzz 00000100 .. 010 100 100 ... ..... ..... @rdm_pg_rn # ASRR LSR_zpzz 00000100 .. 010 101 100 ... ..... ..... @rdm_pg_rn # LSRR LSL_zpzz 00000100 .. 010 111 100 ... ..... ..... @rdm_pg_rn # LSLR +# SVE bitwise shift by wide elements (predicated) +# Note these require size != 3. +ASR_zpzw 00000100 .. 011 000 100 ... ..... ..... @rdn_pg_rm +LSR_zpzw 00000100 .. 011 001 100 ... ..... ..... @rdn_pg_rm +LSL_zpzw 00000100 .. 011 011 100 ... ..... ..... @rdn_pg_rm + ### SVE Logical - Unpredicated Group # SVE bitwise logical operations (unpredicated) -- 2.17.0 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20180516223007.10256-14-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/helper-sve.h | 60 ++++++++++++++++++ target/arm/sve_helper.c | 127 +++++++++++++++++++++++++++++++++++++ target/arm/translate-sve.c | 113 +++++++++++++++++++++++++++++++++ target/arm/sve.decode | 23 +++++++ 4 files changed, 323 insertions(+) diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_4(sve_asrd_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve_asrd_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve_asrd_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve_cls_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve_cls_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve_cls_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve_cls_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve_clz_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve_clz_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve_clz_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve_clz_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve_cnt_zpz_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve_cnt_zpz_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve_cnt_zpz_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve_cnt_zpz_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve_cnot_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve_cnot_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve_cnot_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve_cnot_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve_fabs_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve_fabs_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve_fabs_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve_fneg_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve_fneg_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve_fneg_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve_not_zpz_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve_not_zpz_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve_not_zpz_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve_not_zpz_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve_sxtb_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve_sxtb_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve_sxtb_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve_uxtb_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve_uxtb_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve_uxtb_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve_sxth_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve_sxth_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve_uxth_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve_uxth_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve_sxtw_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve_uxtw_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve_abs_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve_abs_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve_abs_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve_abs_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve_neg_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve_neg_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve_neg_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve_neg_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_5(sve_and_pppp, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve_bic_pppp, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve_eor_pppp, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -XXX,XX +XXX,XX @@ DO_ZPZW(sve_lsl_zpzw_s, uint32_t, uint64_t, H1_4, DO_LSL) #undef DO_ZPZW +/* Fully general two-operand expander, controlled by a predicate. + */ +#define DO_ZPZ(NAME, TYPE, H, OP) \ +void HELPER(NAME)(void *vd, void *vn, void *vg, uint32_t desc) \ +{ \ + intptr_t i, opr_sz = simd_oprsz(desc); \ + for (i = 0; i < opr_sz; ) { \ + uint16_t pg = *(uint16_t *)(vg + H1_2(i >> 3)); \ + do { \ + if (pg & 1) { \ + TYPE nn = *(TYPE *)(vn + H(i)); \ + *(TYPE *)(vd + H(i)) = OP(nn); \ + } \ + i += sizeof(TYPE), pg >>= sizeof(TYPE); \ + } while (i & 15); \ + } \ +} + +/* Similarly, specialized for 64-bit operands. */ +#define DO_ZPZ_D(NAME, TYPE, OP) \ +void HELPER(NAME)(void *vd, void *vn, void *vg, uint32_t desc) \ +{ \ + intptr_t i, opr_sz = simd_oprsz(desc) / 8; \ + TYPE *d = vd, *n = vn; \ + uint8_t *pg = vg; \ + for (i = 0; i < opr_sz; i += 1) { \ + if (pg[H1(i)] & 1) { \ + TYPE nn = n[i]; \ + d[i] = OP(nn); \ + } \ + } \ +} + +#define DO_CLS_B(N) (clrsb32(N) - 24) +#define DO_CLS_H(N) (clrsb32(N) - 16) + +DO_ZPZ(sve_cls_b, int8_t, H1, DO_CLS_B) +DO_ZPZ(sve_cls_h, int16_t, H1_2, DO_CLS_H) +DO_ZPZ(sve_cls_s, int32_t, H1_4, clrsb32) +DO_ZPZ_D(sve_cls_d, int64_t, clrsb64) + +#define DO_CLZ_B(N) (clz32(N) - 24) +#define DO_CLZ_H(N) (clz32(N) - 16) + +DO_ZPZ(sve_clz_b, uint8_t, H1, DO_CLZ_B) +DO_ZPZ(sve_clz_h, uint16_t, H1_2, DO_CLZ_H) +DO_ZPZ(sve_clz_s, uint32_t, H1_4, clz32) +DO_ZPZ_D(sve_clz_d, uint64_t, clz64) + +DO_ZPZ(sve_cnt_zpz_b, uint8_t, H1, ctpop8) +DO_ZPZ(sve_cnt_zpz_h, uint16_t, H1_2, ctpop16) +DO_ZPZ(sve_cnt_zpz_s, uint32_t, H1_4, ctpop32) +DO_ZPZ_D(sve_cnt_zpz_d, uint64_t, ctpop64) + +#define DO_CNOT(N) (N == 0) + +DO_ZPZ(sve_cnot_b, uint8_t, H1, DO_CNOT) +DO_ZPZ(sve_cnot_h, uint16_t, H1_2, DO_CNOT) +DO_ZPZ(sve_cnot_s, uint32_t, H1_4, DO_CNOT) +DO_ZPZ_D(sve_cnot_d, uint64_t, DO_CNOT) + +#define DO_FABS(N) (N & ((__typeof(N))-1 >> 1)) + +DO_ZPZ(sve_fabs_h, uint16_t, H1_2, DO_FABS) +DO_ZPZ(sve_fabs_s, uint32_t, H1_4, DO_FABS) +DO_ZPZ_D(sve_fabs_d, uint64_t, DO_FABS) + +#define DO_FNEG(N) (N ^ ~((__typeof(N))-1 >> 1)) + +DO_ZPZ(sve_fneg_h, uint16_t, H1_2, DO_FNEG) +DO_ZPZ(sve_fneg_s, uint32_t, H1_4, DO_FNEG) +DO_ZPZ_D(sve_fneg_d, uint64_t, DO_FNEG) + +#define DO_NOT(N) (~N) + +DO_ZPZ(sve_not_zpz_b, uint8_t, H1, DO_NOT) +DO_ZPZ(sve_not_zpz_h, uint16_t, H1_2, DO_NOT) +DO_ZPZ(sve_not_zpz_s, uint32_t, H1_4, DO_NOT) +DO_ZPZ_D(sve_not_zpz_d, uint64_t, DO_NOT) + +#define DO_SXTB(N) ((int8_t)N) +#define DO_SXTH(N) ((int16_t)N) +#define DO_SXTS(N) ((int32_t)N) +#define DO_UXTB(N) ((uint8_t)N) +#define DO_UXTH(N) ((uint16_t)N) +#define DO_UXTS(N) ((uint32_t)N) + +DO_ZPZ(sve_sxtb_h, uint16_t, H1_2, DO_SXTB) +DO_ZPZ(sve_sxtb_s, uint32_t, H1_4, DO_SXTB) +DO_ZPZ(sve_sxth_s, uint32_t, H1_4, DO_SXTH) +DO_ZPZ_D(sve_sxtb_d, uint64_t, DO_SXTB) +DO_ZPZ_D(sve_sxth_d, uint64_t, DO_SXTH) +DO_ZPZ_D(sve_sxtw_d, uint64_t, DO_SXTS) + +DO_ZPZ(sve_uxtb_h, uint16_t, H1_2, DO_UXTB) +DO_ZPZ(sve_uxtb_s, uint32_t, H1_4, DO_UXTB) +DO_ZPZ(sve_uxth_s, uint32_t, H1_4, DO_UXTH) +DO_ZPZ_D(sve_uxtb_d, uint64_t, DO_UXTB) +DO_ZPZ_D(sve_uxth_d, uint64_t, DO_UXTH) +DO_ZPZ_D(sve_uxtw_d, uint64_t, DO_UXTS) + +#define DO_ABS(N) (N < 0 ? -N : N) + +DO_ZPZ(sve_abs_b, int8_t, H1, DO_ABS) +DO_ZPZ(sve_abs_h, int16_t, H1_2, DO_ABS) +DO_ZPZ(sve_abs_s, int32_t, H1_4, DO_ABS) +DO_ZPZ_D(sve_abs_d, int64_t, DO_ABS) + +#define DO_NEG(N) (-N) + +DO_ZPZ(sve_neg_b, uint8_t, H1, DO_NEG) +DO_ZPZ(sve_neg_h, uint16_t, H1_2, DO_NEG) +DO_ZPZ(sve_neg_s, uint32_t, H1_4, DO_NEG) +DO_ZPZ_D(sve_neg_d, uint64_t, DO_NEG) + +#undef DO_CLS_B +#undef DO_CLS_H +#undef DO_CLZ_B +#undef DO_CLZ_H +#undef DO_CNOT +#undef DO_FABS +#undef DO_FNEG +#undef DO_ABS +#undef DO_NEG +#undef DO_ZPZ +#undef DO_ZPZ_D + /* Two-operand reduction expander, controlled by a predicate. * The difference between TYPERED and TYPERET has to do with * sign-extension. E.g. for SMAX, TYPERED must be signed, diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool trans_UDIV_zpzz(DisasContext *s, arg_rprr_esz *a, uint32_t insn) #undef DO_ZPZZ +/* + *** SVE Integer Arithmetic - Unary Predicated Group + */ + +static bool do_zpz_ool(DisasContext *s, arg_rpr_esz *a, gen_helper_gvec_3 *fn) +{ + if (fn == NULL) { + return false; + } + if (sve_access_check(s)) { + unsigned vsz = vec_full_reg_size(s); + tcg_gen_gvec_3_ool(vec_full_reg_offset(s, a->rd), + vec_full_reg_offset(s, a->rn), + pred_full_reg_offset(s, a->pg), + vsz, vsz, 0, fn); + } + return true; +} + +#define DO_ZPZ(NAME, name) \ +static bool trans_##NAME(DisasContext *s, arg_rpr_esz *a, uint32_t insn) \ +{ \ + static gen_helper_gvec_3 * const fns[4] = { \ + gen_helper_sve_##name##_b, gen_helper_sve_##name##_h, \ + gen_helper_sve_##name##_s, gen_helper_sve_##name##_d, \ + }; \ + return do_zpz_ool(s, a, fns[a->esz]); \ +} + +DO_ZPZ(CLS, cls) +DO_ZPZ(CLZ, clz) +DO_ZPZ(CNT_zpz, cnt_zpz) +DO_ZPZ(CNOT, cnot) +DO_ZPZ(NOT_zpz, not_zpz) +DO_ZPZ(ABS, abs) +DO_ZPZ(NEG, neg) + +static bool trans_FABS(DisasContext *s, arg_rpr_esz *a, uint32_t insn) +{ + static gen_helper_gvec_3 * const fns[4] = { + NULL, + gen_helper_sve_fabs_h, + gen_helper_sve_fabs_s, + gen_helper_sve_fabs_d + }; + return do_zpz_ool(s, a, fns[a->esz]); +} + +static bool trans_FNEG(DisasContext *s, arg_rpr_esz *a, uint32_t insn) +{ + static gen_helper_gvec_3 * const fns[4] = { + NULL, + gen_helper_sve_fneg_h, + gen_helper_sve_fneg_s, + gen_helper_sve_fneg_d + }; + return do_zpz_ool(s, a, fns[a->esz]); +} + +static bool trans_SXTB(DisasContext *s, arg_rpr_esz *a, uint32_t insn) +{ + static gen_helper_gvec_3 * const fns[4] = { + NULL, + gen_helper_sve_sxtb_h, + gen_helper_sve_sxtb_s, + gen_helper_sve_sxtb_d + }; + return do_zpz_ool(s, a, fns[a->esz]); +} + +static bool trans_UXTB(DisasContext *s, arg_rpr_esz *a, uint32_t insn) +{ + static gen_helper_gvec_3 * const fns[4] = { + NULL, + gen_helper_sve_uxtb_h, + gen_helper_sve_uxtb_s, + gen_helper_sve_uxtb_d + }; + return do_zpz_ool(s, a, fns[a->esz]); +} + +static bool trans_SXTH(DisasContext *s, arg_rpr_esz *a, uint32_t insn) +{ + static gen_helper_gvec_3 * const fns[4] = { + NULL, NULL, + gen_helper_sve_sxth_s, + gen_helper_sve_sxth_d + }; + return do_zpz_ool(s, a, fns[a->esz]); +} + +static bool trans_UXTH(DisasContext *s, arg_rpr_esz *a, uint32_t insn) +{ + static gen_helper_gvec_3 * const fns[4] = { + NULL, NULL, + gen_helper_sve_uxth_s, + gen_helper_sve_uxth_d + }; + return do_zpz_ool(s, a, fns[a->esz]); +} + +static bool trans_SXTW(DisasContext *s, arg_rpr_esz *a, uint32_t insn) +{ + return do_zpz_ool(s, a, a->esz == 3 ? gen_helper_sve_sxtw_d : NULL); +} + +static bool trans_UXTW(DisasContext *s, arg_rpr_esz *a, uint32_t insn) +{ + return do_zpz_ool(s, a, a->esz == 3 ? gen_helper_sve_uxtw_d : NULL); +} + +#undef DO_ZPZ + /* *** SVE Integer Reduction Group */ diff --git a/target/arm/sve.decode b/target/arm/sve.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -XXX,XX +XXX,XX @@ ASR_zpzw 00000100 .. 011 000 100 ... ..... ..... @rdn_pg_rm LSR_zpzw 00000100 .. 011 001 100 ... ..... ..... @rdn_pg_rm LSL_zpzw 00000100 .. 011 011 100 ... ..... ..... @rdn_pg_rm +### SVE Integer Arithmetic - Unary Predicated Group + +# SVE unary bit operations (predicated) +# Note esz != 0 for FABS and FNEG. +CLS 00000100 .. 011 000 101 ... ..... ..... @rd_pg_rn +CLZ 00000100 .. 011 001 101 ... ..... ..... @rd_pg_rn +CNT_zpz 00000100 .. 011 010 101 ... ..... ..... @rd_pg_rn +CNOT 00000100 .. 011 011 101 ... ..... ..... @rd_pg_rn +NOT_zpz 00000100 .. 011 110 101 ... ..... ..... @rd_pg_rn +FABS 00000100 .. 011 100 101 ... ..... ..... @rd_pg_rn +FNEG 00000100 .. 011 101 101 ... ..... ..... @rd_pg_rn + +# SVE integer unary operations (predicated) +# Note esz > original size for extensions. +ABS 00000100 .. 010 110 101 ... ..... ..... @rd_pg_rn +NEG 00000100 .. 010 111 101 ... ..... ..... @rd_pg_rn +SXTB 00000100 .. 010 000 101 ... ..... ..... @rd_pg_rn +UXTB 00000100 .. 010 001 101 ... ..... ..... @rd_pg_rn +SXTH 00000100 .. 010 010 101 ... ..... ..... @rd_pg_rn +UXTH 00000100 .. 010 011 101 ... ..... ..... @rd_pg_rn +SXTW 00000100 .. 010 100 101 ... ..... ..... @rd_pg_rn +UXTW 00000100 .. 010 101 101 ... ..... ..... @rd_pg_rn + ### SVE Logical - Unpredicated Group # SVE bitwise logical operations (unpredicated) -- 2.17.0 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20180516223007.10256-15-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/helper-sve.h | 18 ++++++++++++ target/arm/sve_helper.c | 57 ++++++++++++++++++++++++++++++++++++++ target/arm/translate-sve.c | 34 +++++++++++++++++++++++ target/arm/sve.decode | 17 ++++++++++++ 4 files changed, 126 insertions(+) diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_4(sve_neg_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve_neg_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve_neg_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_6(sve_mla_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_6(sve_mla_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_6(sve_mla_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_6(sve_mla_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_6(sve_mls_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_6(sve_mls_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_6(sve_mls_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_6(sve_mls_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_5(sve_and_pppp, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve_bic_pppp, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve_eor_pppp, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -XXX,XX +XXX,XX @@ DO_ZPZI_D(sve_asrd_d, int64_t, DO_ASRD) #undef DO_ASRD #undef DO_ZPZI #undef DO_ZPZI_D + +/* Fully general four-operand expander, controlled by a predicate. + */ +#define DO_ZPZZZ(NAME, TYPE, H, OP) \ +void HELPER(NAME)(void *vd, void *va, void *vn, void *vm, \ + void *vg, uint32_t desc) \ +{ \ + intptr_t i, opr_sz = simd_oprsz(desc); \ + for (i = 0; i < opr_sz; ) { \ + uint16_t pg = *(uint16_t *)(vg + H1_2(i >> 3)); \ + do { \ + if (pg & 1) { \ + TYPE nn = *(TYPE *)(vn + H(i)); \ + TYPE mm = *(TYPE *)(vm + H(i)); \ + TYPE aa = *(TYPE *)(va + H(i)); \ + *(TYPE *)(vd + H(i)) = OP(aa, nn, mm); \ + } \ + i += sizeof(TYPE), pg >>= sizeof(TYPE); \ + } while (i & 15); \ + } \ +} + +/* Similarly, specialized for 64-bit operands. */ +#define DO_ZPZZZ_D(NAME, TYPE, OP) \ +void HELPER(NAME)(void *vd, void *va, void *vn, void *vm, \ + void *vg, uint32_t desc) \ +{ \ + intptr_t i, opr_sz = simd_oprsz(desc) / 8; \ + TYPE *d = vd, *a = va, *n = vn, *m = vm; \ + uint8_t *pg = vg; \ + for (i = 0; i < opr_sz; i += 1) { \ + if (pg[H1(i)] & 1) { \ + TYPE aa = a[i], nn = n[i], mm = m[i]; \ + d[i] = OP(aa, nn, mm); \ + } \ + } \ +} + +#define DO_MLA(A, N, M) (A + N * M) +#define DO_MLS(A, N, M) (A - N * M) + +DO_ZPZZZ(sve_mla_b, uint8_t, H1, DO_MLA) +DO_ZPZZZ(sve_mls_b, uint8_t, H1, DO_MLS) + +DO_ZPZZZ(sve_mla_h, uint16_t, H1_2, DO_MLA) +DO_ZPZZZ(sve_mls_h, uint16_t, H1_2, DO_MLS) + +DO_ZPZZZ(sve_mla_s, uint32_t, H1_4, DO_MLA) +DO_ZPZZZ(sve_mls_s, uint32_t, H1_4, DO_MLS) + +DO_ZPZZZ_D(sve_mla_d, uint64_t, DO_MLA) +DO_ZPZZZ_D(sve_mls_d, uint64_t, DO_MLS) + +#undef DO_MLA +#undef DO_MLS +#undef DO_ZPZZZ +#undef DO_ZPZZZ_D diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ DO_ZPZW(LSL, lsl) #undef DO_ZPZW +/* + *** SVE Integer Multiply-Add Group + */ + +static bool do_zpzzz_ool(DisasContext *s, arg_rprrr_esz *a, + gen_helper_gvec_5 *fn) +{ + if (sve_access_check(s)) { + unsigned vsz = vec_full_reg_size(s); + tcg_gen_gvec_5_ool(vec_full_reg_offset(s, a->rd), + vec_full_reg_offset(s, a->ra), + vec_full_reg_offset(s, a->rn), + vec_full_reg_offset(s, a->rm), + pred_full_reg_offset(s, a->pg), + vsz, vsz, 0, fn); + } + return true; +} + +#define DO_ZPZZZ(NAME, name) \ +static bool trans_##NAME(DisasContext *s, arg_rprrr_esz *a, uint32_t insn) \ +{ \ + static gen_helper_gvec_5 * const fns[4] = { \ + gen_helper_sve_##name##_b, gen_helper_sve_##name##_h, \ + gen_helper_sve_##name##_s, gen_helper_sve_##name##_d, \ + }; \ + return do_zpzzz_ool(s, a, fns[a->esz]); \ +} + +DO_ZPZZZ(MLA, mla) +DO_ZPZZZ(MLS, mls) + +#undef DO_ZPZZZ + /* *** SVE Predicate Logical Operations Group */ diff --git a/target/arm/sve.decode b/target/arm/sve.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -XXX,XX +XXX,XX @@ &rpr_esz rd pg rn esz &rprr_s rd pg rn rm s &rprr_esz rd pg rn rm esz +&rprrr_esz rd pg rn rm ra esz &rpri_esz rd pg rn imm esz ########################################################################### @@ -XXX,XX +XXX,XX @@ @rdm_pg_rn ........ esz:2 ... ... ... pg:3 rn:5 rd:5 \ &rprr_esz rm=%reg_movprfx +# Three register operand, with governing predicate, vector element size +@rda_pg_rn_rm ........ esz:2 . rm:5 ... pg:3 rn:5 rd:5 \ + &rprrr_esz ra=%reg_movprfx +@rdn_pg_ra_rm ........ esz:2 . rm:5 ... pg:3 ra:5 rd:5 \ + &rprrr_esz rn=%reg_movprfx + # One register operand, with governing predicate, vector element size @rd_pg_rn ........ esz:2 ... ... ... pg:3 rn:5 rd:5 &rpr_esz @@ -XXX,XX +XXX,XX @@ UXTH 00000100 .. 010 011 101 ... ..... ..... @rd_pg_rn SXTW 00000100 .. 010 100 101 ... ..... ..... @rd_pg_rn UXTW 00000100 .. 010 101 101 ... ..... ..... @rd_pg_rn +### SVE Integer Multiply-Add Group + +# SVE integer multiply-add writing addend (predicated) +MLA 00000100 .. 0 ..... 010 ... ..... ..... @rda_pg_rn_rm +MLS 00000100 .. 0 ..... 011 ... ..... ..... @rda_pg_rn_rm + +# SVE integer multiply-add writing multiplicand (predicated) +MLA 00000100 .. 0 ..... 110 ... ..... ..... @rdn_pg_ra_rm # MAD +MLS 00000100 .. 0 ..... 111 ... ..... ..... @rdn_pg_ra_rm # MSB + ### SVE Logical - Unpredicated Group # SVE bitwise logical operations (unpredicated) -- 2.17.0 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20180516223007.10256-16-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 34 ++++++++++++++++++++++++++++++++++ target/arm/sve.decode | 13 +++++++++++++ 2 files changed, 47 insertions(+) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool trans_BIC_zzz(DisasContext *s, arg_rrr_esz *a, uint32_t insn) return do_vector3_z(s, tcg_gen_gvec_andc, 0, a->rd, a->rn, a->rm); } +/* + *** SVE Integer Arithmetic - Unpredicated Group + */ + +static bool trans_ADD_zzz(DisasContext *s, arg_rrr_esz *a, uint32_t insn) +{ + return do_vector3_z(s, tcg_gen_gvec_add, a->esz, a->rd, a->rn, a->rm); +} + +static bool trans_SUB_zzz(DisasContext *s, arg_rrr_esz *a, uint32_t insn) +{ + return do_vector3_z(s, tcg_gen_gvec_sub, a->esz, a->rd, a->rn, a->rm); +} + +static bool trans_SQADD_zzz(DisasContext *s, arg_rrr_esz *a, uint32_t insn) +{ + return do_vector3_z(s, tcg_gen_gvec_ssadd, a->esz, a->rd, a->rn, a->rm); +} + +static bool trans_SQSUB_zzz(DisasContext *s, arg_rrr_esz *a, uint32_t insn) +{ + return do_vector3_z(s, tcg_gen_gvec_sssub, a->esz, a->rd, a->rn, a->rm); +} + +static bool trans_UQADD_zzz(DisasContext *s, arg_rrr_esz *a, uint32_t insn) +{ + return do_vector3_z(s, tcg_gen_gvec_usadd, a->esz, a->rd, a->rn, a->rm); +} + +static bool trans_UQSUB_zzz(DisasContext *s, arg_rrr_esz *a, uint32_t insn) +{ + return do_vector3_z(s, tcg_gen_gvec_ussub, a->esz, a->rd, a->rn, a->rm); +} + /* *** SVE Integer Arithmetic - Binary Predicated Group */ diff --git a/target/arm/sve.decode b/target/arm/sve.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -XXX,XX +XXX,XX @@ # Three predicate operand, with governing predicate, flag setting @pd_pg_pn_pm_s ........ . s:1 .. rm:4 .. pg:4 . rn:4 . rd:4 &rprr_s +# Three operand, vector element size +@rd_rn_rm ........ esz:2 . rm:5 ... ... rn:5 rd:5 &rrr_esz + # Two register operand, with governing predicate, vector element size @rdn_pg_rm ........ esz:2 ... ... ... pg:3 rm:5 rd:5 \ &rprr_esz rn=%reg_movprfx @@ -XXX,XX +XXX,XX @@ MLS 00000100 .. 0 ..... 011 ... ..... ..... @rda_pg_rn_rm MLA 00000100 .. 0 ..... 110 ... ..... ..... @rdn_pg_ra_rm # MAD MLS 00000100 .. 0 ..... 111 ... ..... ..... @rdn_pg_ra_rm # MSB +### SVE Integer Arithmetic - Unpredicated Group + +# SVE integer add/subtract vectors (unpredicated) +ADD_zzz 00000100 .. 1 ..... 000 000 ..... ..... @rd_rn_rm +SUB_zzz 00000100 .. 1 ..... 000 001 ..... ..... @rd_rn_rm +SQADD_zzz 00000100 .. 1 ..... 000 100 ..... ..... @rd_rn_rm +UQADD_zzz 00000100 .. 1 ..... 000 101 ..... ..... @rd_rn_rm +SQSUB_zzz 00000100 .. 1 ..... 000 110 ..... ..... @rd_rn_rm +UQSUB_zzz 00000100 .. 1 ..... 000 111 ..... ..... @rd_rn_rm + ### SVE Logical - Unpredicated Group # SVE bitwise logical operations (unpredicated) -- 2.17.0 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20180516223007.10256-17-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/helper-sve.h | 5 +++ target/arm/sve_helper.c | 40 +++++++++++++++++++ target/arm/translate-sve.c | 79 ++++++++++++++++++++++++++++++++++++++ target/arm/sve.decode | 14 +++++++ 4 files changed, 138 insertions(+) diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_6(sve_mls_s, TCG_CALL_NO_RWG, DEF_HELPER_FLAGS_6(sve_mls_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve_index_b, TCG_CALL_NO_RWG, void, ptr, i32, i32, i32) +DEF_HELPER_FLAGS_4(sve_index_h, TCG_CALL_NO_RWG, void, ptr, i32, i32, i32) +DEF_HELPER_FLAGS_4(sve_index_s, TCG_CALL_NO_RWG, void, ptr, i32, i32, i32) +DEF_HELPER_FLAGS_4(sve_index_d, TCG_CALL_NO_RWG, void, ptr, i64, i64, i32) + DEF_HELPER_FLAGS_5(sve_and_pppp, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve_bic_pppp, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve_eor_pppp, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -XXX,XX +XXX,XX @@ DO_ZPZZZ_D(sve_mls_d, uint64_t, DO_MLS) #undef DO_MLS #undef DO_ZPZZZ #undef DO_ZPZZZ_D + +void HELPER(sve_index_b)(void *vd, uint32_t start, + uint32_t incr, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + uint8_t *d = vd; + for (i = 0; i < opr_sz; i += 1) { + d[H1(i)] = start + i * incr; + } +} + +void HELPER(sve_index_h)(void *vd, uint32_t start, + uint32_t incr, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc) / 2; + uint16_t *d = vd; + for (i = 0; i < opr_sz; i += 1) { + d[H2(i)] = start + i * incr; + } +} + +void HELPER(sve_index_s)(void *vd, uint32_t start, + uint32_t incr, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc) / 4; + uint32_t *d = vd; + for (i = 0; i < opr_sz; i += 1) { + d[H4(i)] = start + i * incr; + } +} + +void HELPER(sve_index_d)(void *vd, uint64_t start, + uint64_t incr, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc) / 8; + uint64_t *d = vd; + for (i = 0; i < opr_sz; i += 1) { + d[i] = start + i * incr; + } +} diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ DO_ZPZZZ(MLS, mls) #undef DO_ZPZZZ +/* + *** SVE Index Generation Group + */ + +static void do_index(DisasContext *s, int esz, int rd, + TCGv_i64 start, TCGv_i64 incr) +{ + unsigned vsz = vec_full_reg_size(s); + TCGv_i32 desc = tcg_const_i32(simd_desc(vsz, vsz, 0)); + TCGv_ptr t_zd = tcg_temp_new_ptr(); + + tcg_gen_addi_ptr(t_zd, cpu_env, vec_full_reg_offset(s, rd)); + if (esz == 3) { + gen_helper_sve_index_d(t_zd, start, incr, desc); + } else { + typedef void index_fn(TCGv_ptr, TCGv_i32, TCGv_i32, TCGv_i32); + static index_fn * const fns[3] = { + gen_helper_sve_index_b, + gen_helper_sve_index_h, + gen_helper_sve_index_s, + }; + TCGv_i32 s32 = tcg_temp_new_i32(); + TCGv_i32 i32 = tcg_temp_new_i32(); + + tcg_gen_extrl_i64_i32(s32, start); + tcg_gen_extrl_i64_i32(i32, incr); + fns[esz](t_zd, s32, i32, desc); + + tcg_temp_free_i32(s32); + tcg_temp_free_i32(i32); + } + tcg_temp_free_ptr(t_zd); + tcg_temp_free_i32(desc); +} + +static bool trans_INDEX_ii(DisasContext *s, arg_INDEX_ii *a, uint32_t insn) +{ + if (sve_access_check(s)) { + TCGv_i64 start = tcg_const_i64(a->imm1); + TCGv_i64 incr = tcg_const_i64(a->imm2); + do_index(s, a->esz, a->rd, start, incr); + tcg_temp_free_i64(start); + tcg_temp_free_i64(incr); + } + return true; +} + +static bool trans_INDEX_ir(DisasContext *s, arg_INDEX_ir *a, uint32_t insn) +{ + if (sve_access_check(s)) { + TCGv_i64 start = tcg_const_i64(a->imm); + TCGv_i64 incr = cpu_reg(s, a->rm); + do_index(s, a->esz, a->rd, start, incr); + tcg_temp_free_i64(start); + } + return true; +} + +static bool trans_INDEX_ri(DisasContext *s, arg_INDEX_ri *a, uint32_t insn) +{ + if (sve_access_check(s)) { + TCGv_i64 start = cpu_reg(s, a->rn); + TCGv_i64 incr = tcg_const_i64(a->imm); + do_index(s, a->esz, a->rd, start, incr); + tcg_temp_free_i64(incr); + } + return true; +} + +static bool trans_INDEX_rr(DisasContext *s, arg_INDEX_rr *a, uint32_t insn) +{ + if (sve_access_check(s)) { + TCGv_i64 start = cpu_reg(s, a->rn); + TCGv_i64 incr = cpu_reg(s, a->rm); + do_index(s, a->esz, a->rd, start, incr); + } + return true; +} + /* *** SVE Predicate Logical Operations Group */ diff --git a/target/arm/sve.decode b/target/arm/sve.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -XXX,XX +XXX,XX @@ ORR_zzz 00000100 01 1 ..... 001 100 ..... ..... @rd_rn_rm_e0 EOR_zzz 00000100 10 1 ..... 001 100 ..... ..... @rd_rn_rm_e0 BIC_zzz 00000100 11 1 ..... 001 100 ..... ..... @rd_rn_rm_e0 +### SVE Index Generation Group + +# SVE index generation (immediate start, immediate increment) +INDEX_ii 00000100 esz:2 1 imm2:s5 010000 imm1:s5 rd:5 + +# SVE index generation (immediate start, register increment) +INDEX_ir 00000100 esz:2 1 rm:5 010010 imm:s5 rd:5 + +# SVE index generation (register start, immediate increment) +INDEX_ri 00000100 esz:2 1 imm:s5 010001 rn:5 rd:5 + +# SVE index generation (register start, register increment) +INDEX_rr 00000100 .. 1 ..... 010011 ..... ..... @rd_rn_rm + ### SVE Predicate Logical Operations Group # SVE predicate logical operations -- 2.17.0 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20180516223007.10256-18-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 27 +++++++++++++++++++++++++++ target/arm/sve.decode | 12 ++++++++++++ 2 files changed, 39 insertions(+) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool trans_INDEX_rr(DisasContext *s, arg_INDEX_rr *a, uint32_t insn) return true; } +/* + *** SVE Stack Allocation Group + */ + +static bool trans_ADDVL(DisasContext *s, arg_ADDVL *a, uint32_t insn) +{ + TCGv_i64 rd = cpu_reg_sp(s, a->rd); + TCGv_i64 rn = cpu_reg_sp(s, a->rn); + tcg_gen_addi_i64(rd, rn, a->imm * vec_full_reg_size(s)); + return true; +} + +static bool trans_ADDPL(DisasContext *s, arg_ADDPL *a, uint32_t insn) +{ + TCGv_i64 rd = cpu_reg_sp(s, a->rd); + TCGv_i64 rn = cpu_reg_sp(s, a->rn); + tcg_gen_addi_i64(rd, rn, a->imm * pred_full_reg_size(s)); + return true; +} + +static bool trans_RDVL(DisasContext *s, arg_RDVL *a, uint32_t insn) +{ + TCGv_i64 reg = cpu_reg(s, a->rd); + tcg_gen_movi_i64(reg, a->imm * vec_full_reg_size(s)); + return true; +} + /* *** SVE Predicate Logical Operations Group */ diff --git a/target/arm/sve.decode b/target/arm/sve.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -XXX,XX +XXX,XX @@ # One register operand, with governing predicate, vector element size @rd_pg_rn ........ esz:2 ... ... ... pg:3 rn:5 rd:5 &rpr_esz +# Two register operands with a 6-bit signed immediate. +@rd_rn_i6 ........ ... rn:5 ..... imm:s6 rd:5 &rri + # Two register operand, one immediate operand, with predicate, # element size encoded as TSZHL. User must fill in imm. @rdn_pg_tszimm ........ .. ... ... ... pg:3 ..... rd:5 \ @@ -XXX,XX +XXX,XX @@ INDEX_ri 00000100 esz:2 1 imm:s5 010001 rn:5 rd:5 # SVE index generation (register start, register increment) INDEX_rr 00000100 .. 1 ..... 010011 ..... ..... @rd_rn_rm +### SVE Stack Allocation Group + +# SVE stack frame adjustment +ADDVL 00000100 001 ..... 01010 ...... ..... @rd_rn_i6 +ADDPL 00000100 011 ..... 01010 ...... ..... @rd_rn_i6 + +# SVE stack frame size +RDVL 00000100 101 11111 01010 imm:s6 rd:5 + ### SVE Predicate Logical Operations Group # SVE predicate logical operations -- 2.17.0 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20180516223007.10256-19-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/helper-sve.h | 12 ++++++ target/arm/sve_helper.c | 30 ++++++++++++++ target/arm/translate-sve.c | 85 ++++++++++++++++++++++++++++++++++++++ target/arm/sve.decode | 26 ++++++++++++ 4 files changed, 153 insertions(+) diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_4(sve_index_h, TCG_CALL_NO_RWG, void, ptr, i32, i32, i32) DEF_HELPER_FLAGS_4(sve_index_s, TCG_CALL_NO_RWG, void, ptr, i32, i32, i32) DEF_HELPER_FLAGS_4(sve_index_d, TCG_CALL_NO_RWG, void, ptr, i64, i64, i32) +DEF_HELPER_FLAGS_4(sve_asr_zzw_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve_asr_zzw_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve_asr_zzw_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve_lsr_zzw_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve_lsr_zzw_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve_lsr_zzw_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve_lsl_zzw_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve_lsl_zzw_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve_lsl_zzw_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_5(sve_and_pppp, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve_bic_pppp, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve_eor_pppp, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -XXX,XX +XXX,XX @@ DO_ZPZ(sve_neg_h, uint16_t, H1_2, DO_NEG) DO_ZPZ(sve_neg_s, uint32_t, H1_4, DO_NEG) DO_ZPZ_D(sve_neg_d, uint64_t, DO_NEG) +/* Three-operand expander, unpredicated, in which the third operand is "wide". + */ +#define DO_ZZW(NAME, TYPE, TYPEW, H, OP) \ +void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc) \ +{ \ + intptr_t i, opr_sz = simd_oprsz(desc); \ + for (i = 0; i < opr_sz; ) { \ + TYPEW mm = *(TYPEW *)(vm + i); \ + do { \ + TYPE nn = *(TYPE *)(vn + H(i)); \ + *(TYPE *)(vd + H(i)) = OP(nn, mm); \ + i += sizeof(TYPE); \ + } while (i & 7); \ + } \ +} + +DO_ZZW(sve_asr_zzw_b, int8_t, uint64_t, H1, DO_ASR) +DO_ZZW(sve_lsr_zzw_b, uint8_t, uint64_t, H1, DO_LSR) +DO_ZZW(sve_lsl_zzw_b, uint8_t, uint64_t, H1, DO_LSL) + +DO_ZZW(sve_asr_zzw_h, int16_t, uint64_t, H1_2, DO_ASR) +DO_ZZW(sve_lsr_zzw_h, uint16_t, uint64_t, H1_2, DO_LSR) +DO_ZZW(sve_lsl_zzw_h, uint16_t, uint64_t, H1_2, DO_LSL) + +DO_ZZW(sve_asr_zzw_s, int32_t, uint64_t, H1_4, DO_ASR) +DO_ZZW(sve_lsr_zzw_s, uint32_t, uint64_t, H1_4, DO_LSR) +DO_ZZW(sve_lsl_zzw_s, uint32_t, uint64_t, H1_4, DO_LSL) + +#undef DO_ZZW + #undef DO_CLS_B #undef DO_CLS_H #undef DO_CLZ_B diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool do_mov_z(DisasContext *s, int rd, int rn) return do_vector2_z(s, tcg_gen_gvec_mov, 0, rd, rn); } +/* Initialize a Zreg with replications of a 64-bit immediate. */ +static void do_dupi_z(DisasContext *s, int rd, uint64_t word) +{ + unsigned vsz = vec_full_reg_size(s); + tcg_gen_gvec_dup64i(vec_full_reg_offset(s, rd), vsz, vsz, word); +} + /* Invoke a vector expander on two Pregs. */ static bool do_vector2_p(DisasContext *s, GVecGen2Fn *gvec_fn, int esz, int rd, int rn) @@ -XXX,XX +XXX,XX @@ DO_ZPZW(LSL, lsl) #undef DO_ZPZW +/* + *** SVE Bitwise Shift - Unpredicated Group + */ + +static bool do_shift_imm(DisasContext *s, arg_rri_esz *a, bool asr, + void (*gvec_fn)(unsigned, uint32_t, uint32_t, + int64_t, uint32_t, uint32_t)) +{ + if (a->esz < 0) { + /* Invalid tsz encoding -- see tszimm_esz. */ + return false; + } + if (sve_access_check(s)) { + unsigned vsz = vec_full_reg_size(s); + /* Shift by element size is architecturally valid. For + arithmetic right-shift, it's the same as by one less. + Otherwise it is a zeroing operation. */ + if (a->imm >= 8 << a->esz) { + if (asr) { + a->imm = (8 << a->esz) - 1; + } else { + do_dupi_z(s, a->rd, 0); + return true; + } + } + gvec_fn(a->esz, vec_full_reg_offset(s, a->rd), + vec_full_reg_offset(s, a->rn), a->imm, vsz, vsz); + } + return true; +} + +static bool trans_ASR_zzi(DisasContext *s, arg_rri_esz *a, uint32_t insn) +{ + return do_shift_imm(s, a, true, tcg_gen_gvec_sari); +} + +static bool trans_LSR_zzi(DisasContext *s, arg_rri_esz *a, uint32_t insn) +{ + return do_shift_imm(s, a, false, tcg_gen_gvec_shri); +} + +static bool trans_LSL_zzi(DisasContext *s, arg_rri_esz *a, uint32_t insn) +{ + return do_shift_imm(s, a, false, tcg_gen_gvec_shli); +} + +static bool do_zzw_ool(DisasContext *s, arg_rrr_esz *a, gen_helper_gvec_3 *fn) +{ + if (fn == NULL) { + return false; + } + if (sve_access_check(s)) { + unsigned vsz = vec_full_reg_size(s); + tcg_gen_gvec_3_ool(vec_full_reg_offset(s, a->rd), + vec_full_reg_offset(s, a->rn), + vec_full_reg_offset(s, a->rm), + vsz, vsz, 0, fn); + } + return true; +} + +#define DO_ZZW(NAME, name) \ +static bool trans_##NAME##_zzw(DisasContext *s, arg_rrr_esz *a, \ + uint32_t insn) \ +{ \ + static gen_helper_gvec_3 * const fns[4] = { \ + gen_helper_sve_##name##_zzw_b, gen_helper_sve_##name##_zzw_h, \ + gen_helper_sve_##name##_zzw_s, NULL \ + }; \ + return do_zzw_ool(s, a, fns[a->esz]); \ +} + +DO_ZZW(ASR, asr) +DO_ZZW(LSR, lsr) +DO_ZZW(LSL, lsl) + +#undef DO_ZZW + /* *** SVE Integer Multiply-Add Group */ diff --git a/target/arm/sve.decode b/target/arm/sve.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -XXX,XX +XXX,XX @@ # A combination of tsz:imm3 -- extract (tsz:imm3) - esize %tszimm_shl 22:2 5:5 !function=tszimm_shl +# Similarly for the tszh/tszl pair at 22/16 for zzi +%tszimm16_esz 22:2 16:5 !function=tszimm_esz +%tszimm16_shr 22:2 16:5 !function=tszimm_shr +%tszimm16_shl 22:2 16:5 !function=tszimm_shl + # Either a copy of rd (at bit 0), or a different source # as propagated via the MOVPRFX instruction. %reg_movprfx 0:5 @@ -XXX,XX +XXX,XX @@ &rr_esz rd rn esz &rri rd rn imm +&rri_esz rd rn imm esz &rrr_esz rd rn rm esz &rpr_esz rd pg rn esz &rprr_s rd pg rn rm s @@ -XXX,XX +XXX,XX @@ @rdn_pg_tszimm ........ .. ... ... ... pg:3 ..... rd:5 \ &rpri_esz rn=%reg_movprfx esz=%tszimm_esz +# Similarly without predicate. +@rd_rn_tszimm ........ .. ... ... ...... rn:5 rd:5 \ + &rri_esz esz=%tszimm16_esz + # Basic Load/Store with 9-bit immediate offset @pd_rn_i9 ........ ........ ...... rn:5 . rd:4 \ &rri imm=%imm9_16_10 @@ -XXX,XX +XXX,XX @@ ADDPL 00000100 011 ..... 01010 ...... ..... @rd_rn_i6 # SVE stack frame size RDVL 00000100 101 11111 01010 imm:s6 rd:5 +### SVE Bitwise Shift - Unpredicated Group + +# SVE bitwise shift by immediate (unpredicated) +ASR_zzi 00000100 .. 1 ..... 1001 00 ..... ..... \ + @rd_rn_tszimm imm=%tszimm16_shr +LSR_zzi 00000100 .. 1 ..... 1001 01 ..... ..... \ + @rd_rn_tszimm imm=%tszimm16_shr +LSL_zzi 00000100 .. 1 ..... 1001 11 ..... ..... \ + @rd_rn_tszimm imm=%tszimm16_shl + +# SVE bitwise shift by wide elements (unpredicated) +# Note esz != 3 +ASR_zzw 00000100 .. 1 ..... 1000 00 ..... ..... @rd_rn_rm +LSR_zzw 00000100 .. 1 ..... 1000 01 ..... ..... @rd_rn_rm +LSL_zzw 00000100 .. 1 ..... 1000 11 ..... ..... @rd_rn_rm + ### SVE Predicate Logical Operations Group # SVE predicate logical operations -- 2.17.0 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20180516223007.10256-20-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/helper-sve.h | 5 +++++ target/arm/sve_helper.c | 40 ++++++++++++++++++++++++++++++++++++++ target/arm/translate-sve.c | 36 ++++++++++++++++++++++++++++++++++ target/arm/sve.decode | 12 ++++++++++++ 4 files changed, 93 insertions(+) diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_4(sve_lsl_zzw_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve_lsl_zzw_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve_lsl_zzw_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve_adr_p32, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve_adr_p64, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve_adr_s32, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve_adr_u32, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_5(sve_and_pppp, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve_bic_pppp, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve_eor_pppp, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -XXX,XX +XXX,XX @@ void HELPER(sve_index_d)(void *vd, uint64_t start, d[i] = start + i * incr; } } + +void HELPER(sve_adr_p32)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc) / 4; + uint32_t sh = simd_data(desc); + uint32_t *d = vd, *n = vn, *m = vm; + for (i = 0; i < opr_sz; i += 1) { + d[i] = n[i] + (m[i] << sh); + } +} + +void HELPER(sve_adr_p64)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc) / 8; + uint64_t sh = simd_data(desc); + uint64_t *d = vd, *n = vn, *m = vm; + for (i = 0; i < opr_sz; i += 1) { + d[i] = n[i] + (m[i] << sh); + } +} + +void HELPER(sve_adr_s32)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc) / 8; + uint64_t sh = simd_data(desc); + uint64_t *d = vd, *n = vn, *m = vm; + for (i = 0; i < opr_sz; i += 1) { + d[i] = n[i] + ((uint64_t)(int32_t)m[i] << sh); + } +} + +void HELPER(sve_adr_u32)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc) / 8; + uint64_t sh = simd_data(desc); + uint64_t *d = vd, *n = vn, *m = vm; + for (i = 0; i < opr_sz; i += 1) { + d[i] = n[i] + ((uint64_t)(uint32_t)m[i] << sh); + } +} diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool trans_RDVL(DisasContext *s, arg_RDVL *a, uint32_t insn) return true; } +/* + *** SVE Compute Vector Address Group + */ + +static bool do_adr(DisasContext *s, arg_rrri *a, gen_helper_gvec_3 *fn) +{ + if (sve_access_check(s)) { + unsigned vsz = vec_full_reg_size(s); + tcg_gen_gvec_3_ool(vec_full_reg_offset(s, a->rd), + vec_full_reg_offset(s, a->rn), + vec_full_reg_offset(s, a->rm), + vsz, vsz, a->imm, fn); + } + return true; +} + +static bool trans_ADR_p32(DisasContext *s, arg_rrri *a, uint32_t insn) +{ + return do_adr(s, a, gen_helper_sve_adr_p32); +} + +static bool trans_ADR_p64(DisasContext *s, arg_rrri *a, uint32_t insn) +{ + return do_adr(s, a, gen_helper_sve_adr_p64); +} + +static bool trans_ADR_s32(DisasContext *s, arg_rrri *a, uint32_t insn) +{ + return do_adr(s, a, gen_helper_sve_adr_s32); +} + +static bool trans_ADR_u32(DisasContext *s, arg_rrri *a, uint32_t insn) +{ + return do_adr(s, a, gen_helper_sve_adr_u32); +} + /* *** SVE Predicate Logical Operations Group */ diff --git a/target/arm/sve.decode b/target/arm/sve.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -XXX,XX +XXX,XX @@ &rr_esz rd rn esz &rri rd rn imm +&rrri rd rn rm imm &rri_esz rd rn imm esz &rrr_esz rd rn rm esz &rpr_esz rd pg rn esz @@ -XXX,XX +XXX,XX @@ # Three operand, vector element size @rd_rn_rm ........ esz:2 . rm:5 ... ... rn:5 rd:5 &rrr_esz +# Three operand with "memory" size, aka immediate left shift +@rd_rn_msz_rm ........ ... rm:5 .... imm:2 rn:5 rd:5 &rrri + # Two register operand, with governing predicate, vector element size @rdn_pg_rm ........ esz:2 ... ... ... pg:3 rm:5 rd:5 \ &rprr_esz rn=%reg_movprfx @@ -XXX,XX +XXX,XX @@ ASR_zzw 00000100 .. 1 ..... 1000 00 ..... ..... @rd_rn_rm LSR_zzw 00000100 .. 1 ..... 1000 01 ..... ..... @rd_rn_rm LSL_zzw 00000100 .. 1 ..... 1000 11 ..... ..... @rd_rn_rm +### SVE Compute Vector Address Group + +# SVE vector address generation +ADR_s32 00000100 00 1 ..... 1010 .. ..... ..... @rd_rn_msz_rm +ADR_u32 00000100 01 1 ..... 1010 .. ..... ..... @rd_rn_msz_rm +ADR_p32 00000100 10 1 ..... 1010 .. ..... ..... @rd_rn_msz_rm +ADR_p64 00000100 11 1 ..... 1010 .. ..... ..... @rd_rn_msz_rm + ### SVE Predicate Logical Operations Group # SVE predicate logical operations -- 2.17.0 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20180516223007.10256-21-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/helper-sve.h | 4 ++ target/arm/sve_helper.c | 90 ++++++++++++++++++++++++++++++++++++++ target/arm/translate-sve.c | 24 ++++++++++ target/arm/sve.decode | 7 +++ 4 files changed, 125 insertions(+) diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_4(sve_adr_p64, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve_adr_s32, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve_adr_u32, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve_fexpa_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve_fexpa_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve_fexpa_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) + DEF_HELPER_FLAGS_5(sve_and_pppp, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve_bic_pppp, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve_eor_pppp, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -XXX,XX +XXX,XX @@ void HELPER(sve_adr_u32)(void *vd, void *vn, void *vm, uint32_t desc) d[i] = n[i] + ((uint64_t)(uint32_t)m[i] << sh); } } + +void HELPER(sve_fexpa_h)(void *vd, void *vn, uint32_t desc) +{ + /* These constants are cut-and-paste directly from the ARM pseudocode. */ + static const uint16_t coeff[] = { + 0x0000, 0x0016, 0x002d, 0x0045, 0x005d, 0x0075, 0x008e, 0x00a8, + 0x00c2, 0x00dc, 0x00f8, 0x0114, 0x0130, 0x014d, 0x016b, 0x0189, + 0x01a8, 0x01c8, 0x01e8, 0x0209, 0x022b, 0x024e, 0x0271, 0x0295, + 0x02ba, 0x02e0, 0x0306, 0x032e, 0x0356, 0x037f, 0x03a9, 0x03d4, + }; + intptr_t i, opr_sz = simd_oprsz(desc) / 2; + uint16_t *d = vd, *n = vn; + + for (i = 0; i < opr_sz; i++) { + uint16_t nn = n[i]; + intptr_t idx = extract32(nn, 0, 5); + uint16_t exp = extract32(nn, 5, 5); + d[i] = coeff[idx] | (exp << 10); + } +} + +void HELPER(sve_fexpa_s)(void *vd, void *vn, uint32_t desc) +{ + /* These constants are cut-and-paste directly from the ARM pseudocode. */ + static const uint32_t coeff[] = { + 0x000000, 0x0164d2, 0x02cd87, 0x043a29, + 0x05aac3, 0x071f62, 0x08980f, 0x0a14d5, + 0x0b95c2, 0x0d1adf, 0x0ea43a, 0x1031dc, + 0x11c3d3, 0x135a2b, 0x14f4f0, 0x16942d, + 0x1837f0, 0x19e046, 0x1b8d3a, 0x1d3eda, + 0x1ef532, 0x20b051, 0x227043, 0x243516, + 0x25fed7, 0x27cd94, 0x29a15b, 0x2b7a3a, + 0x2d583f, 0x2f3b79, 0x3123f6, 0x3311c4, + 0x3504f3, 0x36fd92, 0x38fbaf, 0x3aff5b, + 0x3d08a4, 0x3f179a, 0x412c4d, 0x4346cd, + 0x45672a, 0x478d75, 0x49b9be, 0x4bec15, + 0x4e248c, 0x506334, 0x52a81e, 0x54f35b, + 0x5744fd, 0x599d16, 0x5bfbb8, 0x5e60f5, + 0x60ccdf, 0x633f89, 0x65b907, 0x68396a, + 0x6ac0c7, 0x6d4f30, 0x6fe4ba, 0x728177, + 0x75257d, 0x77d0df, 0x7a83b3, 0x7d3e0c, + }; + intptr_t i, opr_sz = simd_oprsz(desc) / 4; + uint32_t *d = vd, *n = vn; + + for (i = 0; i < opr_sz; i++) { + uint32_t nn = n[i]; + intptr_t idx = extract32(nn, 0, 6); + uint32_t exp = extract32(nn, 6, 8); + d[i] = coeff[idx] | (exp << 23); + } +} + +void HELPER(sve_fexpa_d)(void *vd, void *vn, uint32_t desc) +{ + /* These constants are cut-and-paste directly from the ARM pseudocode. */ + static const uint64_t coeff[] = { + 0x0000000000000ull, 0x02C9A3E778061ull, 0x059B0D3158574ull, + 0x0874518759BC8ull, 0x0B5586CF9890Full, 0x0E3EC32D3D1A2ull, + 0x11301D0125B51ull, 0x1429AAEA92DE0ull, 0x172B83C7D517Bull, + 0x1A35BEB6FCB75ull, 0x1D4873168B9AAull, 0x2063B88628CD6ull, + 0x2387A6E756238ull, 0x26B4565E27CDDull, 0x29E9DF51FDEE1ull, + 0x2D285A6E4030Bull, 0x306FE0A31B715ull, 0x33C08B26416FFull, + 0x371A7373AA9CBull, 0x3A7DB34E59FF7ull, 0x3DEA64C123422ull, + 0x4160A21F72E2Aull, 0x44E086061892Dull, 0x486A2B5C13CD0ull, + 0x4BFDAD5362A27ull, 0x4F9B2769D2CA7ull, 0x5342B569D4F82ull, + 0x56F4736B527DAull, 0x5AB07DD485429ull, 0x5E76F15AD2148ull, + 0x6247EB03A5585ull, 0x6623882552225ull, 0x6A09E667F3BCDull, + 0x6DFB23C651A2Full, 0x71F75E8EC5F74ull, 0x75FEB564267C9ull, + 0x7A11473EB0187ull, 0x7E2F336CF4E62ull, 0x82589994CCE13ull, + 0x868D99B4492EDull, 0x8ACE5422AA0DBull, 0x8F1AE99157736ull, + 0x93737B0CDC5E5ull, 0x97D829FDE4E50ull, 0x9C49182A3F090ull, + 0xA0C667B5DE565ull, 0xA5503B23E255Dull, 0xA9E6B5579FDBFull, + 0xAE89F995AD3ADull, 0xB33A2B84F15FBull, 0xB7F76F2FB5E47ull, + 0xBCC1E904BC1D2ull, 0xC199BDD85529Cull, 0xC67F12E57D14Bull, + 0xCB720DCEF9069ull, 0xD072D4A07897Cull, 0xD5818DCFBA487ull, + 0xDA9E603DB3285ull, 0xDFC97337B9B5Full, 0xE502EE78B3FF6ull, + 0xEA4AFA2A490DAull, 0xEFA1BEE615A27ull, 0xF50765B6E4540ull, + 0xFA7C1819E90D8ull, + }; + intptr_t i, opr_sz = simd_oprsz(desc) / 8; + uint64_t *d = vd, *n = vn; + + for (i = 0; i < opr_sz; i++) { + uint64_t nn = n[i]; + intptr_t idx = extract32(nn, 0, 6); + uint64_t exp = extract32(nn, 6, 11); + d[i] = coeff[idx] | (exp << 52); + } +} diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool trans_ADR_u32(DisasContext *s, arg_rrri *a, uint32_t insn) return do_adr(s, a, gen_helper_sve_adr_u32); } +/* + *** SVE Integer Misc - Unpredicated Group + */ + +static bool trans_FEXPA(DisasContext *s, arg_rr_esz *a, uint32_t insn) +{ + static gen_helper_gvec_2 * const fns[4] = { + NULL, + gen_helper_sve_fexpa_h, + gen_helper_sve_fexpa_s, + gen_helper_sve_fexpa_d, + }; + if (a->esz == 0) { + return false; + } + if (sve_access_check(s)) { + unsigned vsz = vec_full_reg_size(s); + tcg_gen_gvec_2_ool(vec_full_reg_offset(s, a->rd), + vec_full_reg_offset(s, a->rn), + vsz, vsz, 0, fns[a->esz]); + } + return true; +} + /* *** SVE Predicate Logical Operations Group */ diff --git a/target/arm/sve.decode b/target/arm/sve.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -XXX,XX +XXX,XX @@ # Two operand @pd_pn ........ esz:2 .. .... ....... rn:4 . rd:4 &rr_esz +@rd_rn ........ esz:2 ...... ...... rn:5 rd:5 &rr_esz # Three operand with unused vector element size @rd_rn_rm_e0 ........ ... rm:5 ... ... rn:5 rd:5 &rrr_esz esz=0 @@ -XXX,XX +XXX,XX @@ ADR_u32 00000100 01 1 ..... 1010 .. ..... ..... @rd_rn_msz_rm ADR_p32 00000100 10 1 ..... 1010 .. ..... ..... @rd_rn_msz_rm ADR_p64 00000100 11 1 ..... 1010 .. ..... ..... @rd_rn_msz_rm +### SVE Integer Misc - Unpredicated Group + +# SVE floating-point exponential accelerator +# Note esz != 0 +FEXPA 00000100 .. 1 00000 101110 ..... ..... @rd_rn + ### SVE Predicate Logical Operations Group # SVE predicate logical operations -- 2.17.0 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20180516223007.10256-22-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/helper-sve.h | 4 ++++ target/arm/sve_helper.c | 43 ++++++++++++++++++++++++++++++++++++++ target/arm/translate-sve.c | 21 +++++++++++++++++++ target/arm/sve.decode | 4 ++++ 4 files changed, 72 insertions(+) diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_3(sve_fexpa_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve_fexpa_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve_fexpa_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve_ftssel_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve_ftssel_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve_ftssel_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_5(sve_and_pppp, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve_bic_pppp, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve_eor_pppp, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -XXX,XX +XXX,XX @@ #include "exec/cpu_ldst.h" #include "exec/helper-proto.h" #include "tcg/tcg-gvec-desc.h" +#include "fpu/softfloat.h" /* Note that vector data is stored in host-endian 64-bit chunks, @@ -XXX,XX +XXX,XX @@ void HELPER(sve_fexpa_d)(void *vd, void *vn, uint32_t desc) d[i] = coeff[idx] | (exp << 52); } } + +void HELPER(sve_ftssel_h)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc) / 2; + uint16_t *d = vd, *n = vn, *m = vm; + for (i = 0; i < opr_sz; i += 1) { + uint16_t nn = n[i]; + uint16_t mm = m[i]; + if (mm & 1) { + nn = float16_one; + } + d[i] = nn ^ (mm & 2) << 14; + } +} + +void HELPER(sve_ftssel_s)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc) / 4; + uint32_t *d = vd, *n = vn, *m = vm; + for (i = 0; i < opr_sz; i += 1) { + uint32_t nn = n[i]; + uint32_t mm = m[i]; + if (mm & 1) { + nn = float32_one; + } + d[i] = nn ^ (mm & 2) << 30; + } +} + +void HELPER(sve_ftssel_d)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc) / 8; + uint64_t *d = vd, *n = vn, *m = vm; + for (i = 0; i < opr_sz; i += 1) { + uint64_t nn = n[i]; + uint64_t mm = m[i]; + if (mm & 1) { + nn = float64_one; + } + d[i] = nn ^ (mm & 2) << 62; + } +} diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool trans_FEXPA(DisasContext *s, arg_rr_esz *a, uint32_t insn) return true; } +static bool trans_FTSSEL(DisasContext *s, arg_rrr_esz *a, uint32_t insn) +{ + static gen_helper_gvec_3 * const fns[4] = { + NULL, + gen_helper_sve_ftssel_h, + gen_helper_sve_ftssel_s, + gen_helper_sve_ftssel_d, + }; + if (a->esz == 0) { + return false; + } + if (sve_access_check(s)) { + unsigned vsz = vec_full_reg_size(s); + tcg_gen_gvec_3_ool(vec_full_reg_offset(s, a->rd), + vec_full_reg_offset(s, a->rn), + vec_full_reg_offset(s, a->rm), + vsz, vsz, 0, fns[a->esz]); + } + return true; +} + /* *** SVE Predicate Logical Operations Group */ diff --git a/target/arm/sve.decode b/target/arm/sve.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -XXX,XX +XXX,XX @@ ADR_p64 00000100 11 1 ..... 1010 .. ..... ..... @rd_rn_msz_rm # Note esz != 0 FEXPA 00000100 .. 1 00000 101110 ..... ..... @rd_rn +# SVE floating-point trig select coefficient +# Note esz != 0 +FTSSEL 00000100 .. 1 ..... 101100 ..... ..... @rd_rn_rm + ### SVE Predicate Logical Operations Group # SVE predicate logical operations -- 2.17.0 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20180516223007.10256-23-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/helper-sve.h | 11 ++ target/arm/sve_helper.c | 136 ++++++++++++++++++ target/arm/translate-sve.c | 288 +++++++++++++++++++++++++++++++++++++ target/arm/sve.decode | 31 +++- 4 files changed, 465 insertions(+), 1 deletion(-) diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_4(sve_ftssel_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve_ftssel_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve_ftssel_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve_sqaddi_b, TCG_CALL_NO_RWG, void, ptr, ptr, s32, i32) +DEF_HELPER_FLAGS_4(sve_sqaddi_h, TCG_CALL_NO_RWG, void, ptr, ptr, s32, i32) +DEF_HELPER_FLAGS_4(sve_sqaddi_s, TCG_CALL_NO_RWG, void, ptr, ptr, s64, i32) +DEF_HELPER_FLAGS_4(sve_sqaddi_d, TCG_CALL_NO_RWG, void, ptr, ptr, s64, i32) + +DEF_HELPER_FLAGS_4(sve_uqaddi_b, TCG_CALL_NO_RWG, void, ptr, ptr, s32, i32) +DEF_HELPER_FLAGS_4(sve_uqaddi_h, TCG_CALL_NO_RWG, void, ptr, ptr, s32, i32) +DEF_HELPER_FLAGS_4(sve_uqaddi_s, TCG_CALL_NO_RWG, void, ptr, ptr, s64, i32) +DEF_HELPER_FLAGS_4(sve_uqaddi_d, TCG_CALL_NO_RWG, void, ptr, ptr, i64, i32) +DEF_HELPER_FLAGS_4(sve_uqsubi_d, TCG_CALL_NO_RWG, void, ptr, ptr, i64, i32) + DEF_HELPER_FLAGS_5(sve_and_pppp, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve_bic_pppp, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve_eor_pppp, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -XXX,XX +XXX,XX @@ void HELPER(sve_ftssel_d)(void *vd, void *vn, void *vm, uint32_t desc) d[i] = nn ^ (mm & 2) << 62; } } + +/* + * Signed saturating addition with scalar operand. + */ + +void HELPER(sve_sqaddi_b)(void *d, void *a, int32_t b, uint32_t desc) +{ + intptr_t i, oprsz = simd_oprsz(desc); + + for (i = 0; i < oprsz; i += sizeof(int8_t)) { + int r = *(int8_t *)(a + i) + b; + if (r > INT8_MAX) { + r = INT8_MAX; + } else if (r < INT8_MIN) { + r = INT8_MIN; + } + *(int8_t *)(d + i) = r; + } +} + +void HELPER(sve_sqaddi_h)(void *d, void *a, int32_t b, uint32_t desc) +{ + intptr_t i, oprsz = simd_oprsz(desc); + + for (i = 0; i < oprsz; i += sizeof(int16_t)) { + int r = *(int16_t *)(a + i) + b; + if (r > INT16_MAX) { + r = INT16_MAX; + } else if (r < INT16_MIN) { + r = INT16_MIN; + } + *(int16_t *)(d + i) = r; + } +} + +void HELPER(sve_sqaddi_s)(void *d, void *a, int64_t b, uint32_t desc) +{ + intptr_t i, oprsz = simd_oprsz(desc); + + for (i = 0; i < oprsz; i += sizeof(int32_t)) { + int64_t r = *(int32_t *)(a + i) + b; + if (r > INT32_MAX) { + r = INT32_MAX; + } else if (r < INT32_MIN) { + r = INT32_MIN; + } + *(int32_t *)(d + i) = r; + } +} + +void HELPER(sve_sqaddi_d)(void *d, void *a, int64_t b, uint32_t desc) +{ + intptr_t i, oprsz = simd_oprsz(desc); + + for (i = 0; i < oprsz; i += sizeof(int64_t)) { + int64_t ai = *(int64_t *)(a + i); + int64_t r = ai + b; + if (((r ^ ai) & ~(ai ^ b)) < 0) { + /* Signed overflow. */ + r = (r < 0 ? INT64_MAX : INT64_MIN); + } + *(int64_t *)(d + i) = r; + } +} + +/* + * Unsigned saturating addition with scalar operand. + */ + +void HELPER(sve_uqaddi_b)(void *d, void *a, int32_t b, uint32_t desc) +{ + intptr_t i, oprsz = simd_oprsz(desc); + + for (i = 0; i < oprsz; i += sizeof(uint8_t)) { + int r = *(uint8_t *)(a + i) + b; + if (r > UINT8_MAX) { + r = UINT8_MAX; + } else if (r < 0) { + r = 0; + } + *(uint8_t *)(d + i) = r; + } +} + +void HELPER(sve_uqaddi_h)(void *d, void *a, int32_t b, uint32_t desc) +{ + intptr_t i, oprsz = simd_oprsz(desc); + + for (i = 0; i < oprsz; i += sizeof(uint16_t)) { + int r = *(uint16_t *)(a + i) + b; + if (r > UINT16_MAX) { + r = UINT16_MAX; + } else if (r < 0) { + r = 0; + } + *(uint16_t *)(d + i) = r; + } +} + +void HELPER(sve_uqaddi_s)(void *d, void *a, int64_t b, uint32_t desc) +{ + intptr_t i, oprsz = simd_oprsz(desc); + + for (i = 0; i < oprsz; i += sizeof(uint32_t)) { + int64_t r = *(uint32_t *)(a + i) + b; + if (r > UINT32_MAX) { + r = UINT32_MAX; + } else if (r < 0) { + r = 0; + } + *(uint32_t *)(d + i) = r; + } +} + +void HELPER(sve_uqaddi_d)(void *d, void *a, uint64_t b, uint32_t desc) +{ + intptr_t i, oprsz = simd_oprsz(desc); + + for (i = 0; i < oprsz; i += sizeof(uint64_t)) { + uint64_t r = *(uint64_t *)(a + i) + b; + if (r < b) { + r = UINT64_MAX; + } + *(uint64_t *)(d + i) = r; + } +} + +void HELPER(sve_uqsubi_d)(void *d, void *a, uint64_t b, uint32_t desc) +{ + intptr_t i, oprsz = simd_oprsz(desc); + + for (i = 0; i < oprsz; i += sizeof(uint64_t)) { + uint64_t ai = *(uint64_t *)(a + i); + *(uint64_t *)(d + i) = (ai < b ? 0 : ai - b); + } +} diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static int tszimm_shl(int x) return x - (8 << tszimm_esz(x)); } +static inline int plus1(int x) +{ + return x + 1; +} + /* * Include the generated decoder. */ @@ -XXX,XX +XXX,XX @@ static bool trans_PNEXT(DisasContext *s, arg_rr_esz *a, uint32_t insn) return do_pfirst_pnext(s, a, gen_helper_sve_pnext); } +/* + *** SVE Element Count Group + */ + +/* Perform an inline saturating addition of a 32-bit value within + * a 64-bit register. The second operand is known to be positive, + * which halves the comparisions we must perform to bound the result. + */ +static void do_sat_addsub_32(TCGv_i64 reg, TCGv_i64 val, bool u, bool d) +{ + int64_t ibound; + TCGv_i64 bound; + TCGCond cond; + + /* Use normal 64-bit arithmetic to detect 32-bit overflow. */ + if (u) { + tcg_gen_ext32u_i64(reg, reg); + } else { + tcg_gen_ext32s_i64(reg, reg); + } + if (d) { + tcg_gen_sub_i64(reg, reg, val); + ibound = (u ? 0 : INT32_MIN); + cond = TCG_COND_LT; + } else { + tcg_gen_add_i64(reg, reg, val); + ibound = (u ? UINT32_MAX : INT32_MAX); + cond = TCG_COND_GT; + } + bound = tcg_const_i64(ibound); + tcg_gen_movcond_i64(cond, reg, reg, bound, bound, reg); + tcg_temp_free_i64(bound); +} + +/* Similarly with 64-bit values. */ +static void do_sat_addsub_64(TCGv_i64 reg, TCGv_i64 val, bool u, bool d) +{ + TCGv_i64 t0 = tcg_temp_new_i64(); + TCGv_i64 t1 = tcg_temp_new_i64(); + TCGv_i64 t2; + + if (u) { + if (d) { + tcg_gen_sub_i64(t0, reg, val); + tcg_gen_movi_i64(t1, 0); + tcg_gen_movcond_i64(TCG_COND_LTU, reg, reg, val, t1, t0); + } else { + tcg_gen_add_i64(t0, reg, val); + tcg_gen_movi_i64(t1, -1); + tcg_gen_movcond_i64(TCG_COND_LTU, reg, t0, reg, t1, t0); + } + } else { + if (d) { + /* Detect signed overflow for subtraction. */ + tcg_gen_xor_i64(t0, reg, val); + tcg_gen_sub_i64(t1, reg, val); + tcg_gen_xor_i64(reg, reg, t0); + tcg_gen_and_i64(t0, t0, reg); + + /* Bound the result. */ + tcg_gen_movi_i64(reg, INT64_MIN); + t2 = tcg_const_i64(0); + tcg_gen_movcond_i64(TCG_COND_LT, reg, t0, t2, reg, t1); + } else { + /* Detect signed overflow for addition. */ + tcg_gen_xor_i64(t0, reg, val); + tcg_gen_add_i64(reg, reg, val); + tcg_gen_xor_i64(t1, reg, val); + tcg_gen_andc_i64(t0, t1, t0); + + /* Bound the result. */ + tcg_gen_movi_i64(t1, INT64_MAX); + t2 = tcg_const_i64(0); + tcg_gen_movcond_i64(TCG_COND_LT, reg, t0, t2, t1, reg); + } + tcg_temp_free_i64(t2); + } + tcg_temp_free_i64(t0); + tcg_temp_free_i64(t1); +} + +/* Similarly with a vector and a scalar operand. */ +static void do_sat_addsub_vec(DisasContext *s, int esz, int rd, int rn, + TCGv_i64 val, bool u, bool d) +{ + unsigned vsz = vec_full_reg_size(s); + TCGv_ptr dptr, nptr; + TCGv_i32 t32, desc; + TCGv_i64 t64; + + dptr = tcg_temp_new_ptr(); + nptr = tcg_temp_new_ptr(); + tcg_gen_addi_ptr(dptr, cpu_env, vec_full_reg_offset(s, rd)); + tcg_gen_addi_ptr(nptr, cpu_env, vec_full_reg_offset(s, rn)); + desc = tcg_const_i32(simd_desc(vsz, vsz, 0)); + + switch (esz) { + case MO_8: + t32 = tcg_temp_new_i32(); + tcg_gen_extrl_i64_i32(t32, val); + if (d) { + tcg_gen_neg_i32(t32, t32); + } + if (u) { + gen_helper_sve_uqaddi_b(dptr, nptr, t32, desc); + } else { + gen_helper_sve_sqaddi_b(dptr, nptr, t32, desc); + } + tcg_temp_free_i32(t32); + break; + + case MO_16: + t32 = tcg_temp_new_i32(); + tcg_gen_extrl_i64_i32(t32, val); + if (d) { + tcg_gen_neg_i32(t32, t32); + } + if (u) { + gen_helper_sve_uqaddi_h(dptr, nptr, t32, desc); + } else { + gen_helper_sve_sqaddi_h(dptr, nptr, t32, desc); + } + tcg_temp_free_i32(t32); + break; + + case MO_32: + t64 = tcg_temp_new_i64(); + if (d) { + tcg_gen_neg_i64(t64, val); + } else { + tcg_gen_mov_i64(t64, val); + } + if (u) { + gen_helper_sve_uqaddi_s(dptr, nptr, t64, desc); + } else { + gen_helper_sve_sqaddi_s(dptr, nptr, t64, desc); + } + tcg_temp_free_i64(t64); + break; + + case MO_64: + if (u) { + if (d) { + gen_helper_sve_uqsubi_d(dptr, nptr, val, desc); + } else { + gen_helper_sve_uqaddi_d(dptr, nptr, val, desc); + } + } else if (d) { + t64 = tcg_temp_new_i64(); + tcg_gen_neg_i64(t64, val); + gen_helper_sve_sqaddi_d(dptr, nptr, t64, desc); + tcg_temp_free_i64(t64); + } else { + gen_helper_sve_sqaddi_d(dptr, nptr, val, desc); + } + break; + + default: + g_assert_not_reached(); + } + + tcg_temp_free_ptr(dptr); + tcg_temp_free_ptr(nptr); + tcg_temp_free_i32(desc); +} + +static bool trans_CNT_r(DisasContext *s, arg_CNT_r *a, uint32_t insn) +{ + if (sve_access_check(s)) { + unsigned fullsz = vec_full_reg_size(s); + unsigned numelem = decode_pred_count(fullsz, a->pat, a->esz); + tcg_gen_movi_i64(cpu_reg(s, a->rd), numelem * a->imm); + } + return true; +} + +static bool trans_INCDEC_r(DisasContext *s, arg_incdec_cnt *a, uint32_t insn) +{ + if (sve_access_check(s)) { + unsigned fullsz = vec_full_reg_size(s); + unsigned numelem = decode_pred_count(fullsz, a->pat, a->esz); + int inc = numelem * a->imm * (a->d ? -1 : 1); + TCGv_i64 reg = cpu_reg(s, a->rd); + + tcg_gen_addi_i64(reg, reg, inc); + } + return true; +} + +static bool trans_SINCDEC_r_32(DisasContext *s, arg_incdec_cnt *a, + uint32_t insn) +{ + if (!sve_access_check(s)) { + return true; + } + + unsigned fullsz = vec_full_reg_size(s); + unsigned numelem = decode_pred_count(fullsz, a->pat, a->esz); + int inc = numelem * a->imm; + TCGv_i64 reg = cpu_reg(s, a->rd); + + /* Use normal 64-bit arithmetic to detect 32-bit overflow. */ + if (inc == 0) { + if (a->u) { + tcg_gen_ext32u_i64(reg, reg); + } else { + tcg_gen_ext32s_i64(reg, reg); + } + } else { + TCGv_i64 t = tcg_const_i64(inc); + do_sat_addsub_32(reg, t, a->u, a->d); + tcg_temp_free_i64(t); + } + return true; +} + +static bool trans_SINCDEC_r_64(DisasContext *s, arg_incdec_cnt *a, + uint32_t insn) +{ + if (!sve_access_check(s)) { + return true; + } + + unsigned fullsz = vec_full_reg_size(s); + unsigned numelem = decode_pred_count(fullsz, a->pat, a->esz); + int inc = numelem * a->imm; + TCGv_i64 reg = cpu_reg(s, a->rd); + + if (inc != 0) { + TCGv_i64 t = tcg_const_i64(inc); + do_sat_addsub_64(reg, t, a->u, a->d); + tcg_temp_free_i64(t); + } + return true; +} + +static bool trans_INCDEC_v(DisasContext *s, arg_incdec2_cnt *a, uint32_t insn) +{ + if (a->esz == 0) { + return false; + } + + unsigned fullsz = vec_full_reg_size(s); + unsigned numelem = decode_pred_count(fullsz, a->pat, a->esz); + int inc = numelem * a->imm; + + if (inc != 0) { + if (sve_access_check(s)) { + TCGv_i64 t = tcg_const_i64(a->d ? -inc : inc); + tcg_gen_gvec_adds(a->esz, vec_full_reg_offset(s, a->rd), + vec_full_reg_offset(s, a->rn), + t, fullsz, fullsz); + tcg_temp_free_i64(t); + } + } else { + do_mov_z(s, a->rd, a->rn); + } + return true; +} + +static bool trans_SINCDEC_v(DisasContext *s, arg_incdec2_cnt *a, + uint32_t insn) +{ + if (a->esz == 0) { + return false; + } + + unsigned fullsz = vec_full_reg_size(s); + unsigned numelem = decode_pred_count(fullsz, a->pat, a->esz); + int inc = numelem * a->imm; + + if (inc != 0) { + if (sve_access_check(s)) { + TCGv_i64 t = tcg_const_i64(inc); + do_sat_addsub_vec(s, a->esz, a->rd, a->rn, t, a->u, a->d); + tcg_temp_free_i64(t); + } + } else { + do_mov_z(s, a->rd, a->rn); + } + return true; +} + /* *** SVE Memory - 32-bit Gather and Unsized Contiguous Group */ diff --git a/target/arm/sve.decode b/target/arm/sve.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -XXX,XX +XXX,XX @@ ########################################################################### # Named fields. These are primarily for disjoint fields. +%imm4_16_p1 16:4 !function=plus1 %imm6_22_5 22:1 5:5 %imm9_16_10 16:s6 10:3 @@ -XXX,XX +XXX,XX @@ &rprr_esz rd pg rn rm esz &rprrr_esz rd pg rn rm ra esz &rpri_esz rd pg rn imm esz +&ptrue rd esz pat s +&incdec_cnt rd pat esz imm d u +&incdec2_cnt rd rn pat esz imm d u ########################################################################### # Named instruction formats. These are generally used to @@ -XXX,XX +XXX,XX @@ @rd_rn_i9 ........ ........ ...... rn:5 rd:5 \ &rri imm=%imm9_16_10 +# One register, pattern, and uint4+1. +# User must fill in U and D. +@incdec_cnt ........ esz:2 .. .... ...... pat:5 rd:5 \ + &incdec_cnt imm=%imm4_16_p1 +@incdec2_cnt ........ esz:2 .. .... ...... pat:5 rd:5 \ + &incdec2_cnt imm=%imm4_16_p1 rn=%reg_movprfx + ########################################################################### # Instruction patterns. Grouped according to the SVE encodingindex.xhtml. @@ -XXX,XX +XXX,XX @@ FEXPA 00000100 .. 1 00000 101110 ..... ..... @rd_rn # Note esz != 0 FTSSEL 00000100 .. 1 ..... 101100 ..... ..... @rd_rn_rm -### SVE Predicate Logical Operations Group +### SVE Element Count Group + +# SVE element count +CNT_r 00000100 .. 10 .... 1110 0 0 ..... ..... @incdec_cnt d=0 u=1 + +# SVE inc/dec register by element count +INCDEC_r 00000100 .. 11 .... 1110 0 d:1 ..... ..... @incdec_cnt u=1 + +# SVE saturating inc/dec register by element count +SINCDEC_r_32 00000100 .. 10 .... 1111 d:1 u:1 ..... ..... @incdec_cnt +SINCDEC_r_64 00000100 .. 11 .... 1111 d:1 u:1 ..... ..... @incdec_cnt + +# SVE inc/dec vector by element count +# Note this requires esz != 0. +INCDEC_v 00000100 .. 1 1 .... 1100 0 d:1 ..... ..... @incdec2_cnt u=1 + +# SVE saturating inc/dec vector by element count +# Note these require esz != 0. +SINCDEC_v 00000100 .. 1 0 .... 1100 d:1 u:1 ..... ..... @incdec2_cnt # SVE predicate logical operations AND_pppp 00100101 0. 00 .... 01 .... 0 .... 0 .... @pd_pg_pn_pm_s -- 2.17.0 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20180516223007.10256-24-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/translate-sve.c | 49 ++++++++++++++++++++++++++++++++++++++ target/arm/sve.decode | 17 +++++++++++++ 2 files changed, 66 insertions(+) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool trans_SINCDEC_v(DisasContext *s, arg_incdec2_cnt *a, return true; } +/* + *** SVE Bitwise Immediate Group + */ + +static bool do_zz_dbm(DisasContext *s, arg_rr_dbm *a, GVecGen2iFn *gvec_fn) +{ + uint64_t imm; + if (!logic_imm_decode_wmask(&imm, extract32(a->dbm, 12, 1), + extract32(a->dbm, 0, 6), + extract32(a->dbm, 6, 6))) { + return false; + } + if (sve_access_check(s)) { + unsigned vsz = vec_full_reg_size(s); + gvec_fn(MO_64, vec_full_reg_offset(s, a->rd), + vec_full_reg_offset(s, a->rn), imm, vsz, vsz); + } + return true; +} + +static bool trans_AND_zzi(DisasContext *s, arg_rr_dbm *a, uint32_t insn) +{ + return do_zz_dbm(s, a, tcg_gen_gvec_andi); +} + +static bool trans_ORR_zzi(DisasContext *s, arg_rr_dbm *a, uint32_t insn) +{ + return do_zz_dbm(s, a, tcg_gen_gvec_ori); +} + +static bool trans_EOR_zzi(DisasContext *s, arg_rr_dbm *a, uint32_t insn) +{ + return do_zz_dbm(s, a, tcg_gen_gvec_xori); +} + +static bool trans_DUPM(DisasContext *s, arg_DUPM *a, uint32_t insn) +{ + uint64_t imm; + if (!logic_imm_decode_wmask(&imm, extract32(a->dbm, 12, 1), + extract32(a->dbm, 0, 6), + extract32(a->dbm, 6, 6))) { + return false; + } + if (sve_access_check(s)) { + do_dupi_z(s, a->rd, imm); + } + return true; +} + /* *** SVE Memory - 32-bit Gather and Unsized Contiguous Group */ diff --git a/target/arm/sve.decode b/target/arm/sve.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -XXX,XX +XXX,XX @@ &rr_esz rd rn esz &rri rd rn imm +&rr_dbm rd rn dbm &rrri rd rn rm imm &rri_esz rd rn imm esz &rrr_esz rd rn rm esz @@ -XXX,XX +XXX,XX @@ @rd_rn_tszimm ........ .. ... ... ...... rn:5 rd:5 \ &rri_esz esz=%tszimm16_esz +# Two register operand, one encoded bitmask. +@rdn_dbm ........ .. .... dbm:13 rd:5 \ + &rr_dbm rn=%reg_movprfx + # Basic Load/Store with 9-bit immediate offset @pd_rn_i9 ........ ........ ...... rn:5 . rd:4 \ &rri imm=%imm9_16_10 @@ -XXX,XX +XXX,XX @@ INCDEC_v 00000100 .. 1 1 .... 1100 0 d:1 ..... ..... @incdec2_cnt u=1 # Note these require esz != 0. SINCDEC_v 00000100 .. 1 0 .... 1100 d:1 u:1 ..... ..... @incdec2_cnt +### SVE Bitwise Immediate Group + +# SVE bitwise logical with immediate (unpredicated) +ORR_zzi 00000101 00 0000 ............. ..... @rdn_dbm +EOR_zzi 00000101 01 0000 ............. ..... @rdn_dbm +AND_zzi 00000101 10 0000 ............. ..... @rdn_dbm + +# SVE broadcast bitmask immediate +DUPM 00000101 11 0000 dbm:13 rd:5 + +### SVE Predicate Logical Operations Group + # SVE predicate logical operations AND_pppp 00100101 0. 00 .... 01 .... 0 .... 0 .... @pd_pg_pn_pm_s BIC_pppp 00100101 0. 00 .... 01 .... 0 .... 1 .... @pd_pg_pn_pm_s -- 2.17.0 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20180516223007.10256-25-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/helper-sve.h | 10 ++++ target/arm/sve_helper.c | 108 +++++++++++++++++++++++++++++++++++++ target/arm/translate-sve.c | 88 ++++++++++++++++++++++++++++++ target/arm/sve.decode | 19 ++++++- 4 files changed, 224 insertions(+), 1 deletion(-) diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_4(sve_uqaddi_s, TCG_CALL_NO_RWG, void, ptr, ptr, s64, i32) DEF_HELPER_FLAGS_4(sve_uqaddi_d, TCG_CALL_NO_RWG, void, ptr, ptr, i64, i32) DEF_HELPER_FLAGS_4(sve_uqsubi_d, TCG_CALL_NO_RWG, void, ptr, ptr, i64, i32) +DEF_HELPER_FLAGS_5(sve_cpy_m_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i64, i32) +DEF_HELPER_FLAGS_5(sve_cpy_m_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i64, i32) +DEF_HELPER_FLAGS_5(sve_cpy_m_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i64, i32) +DEF_HELPER_FLAGS_5(sve_cpy_m_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i64, i32) + +DEF_HELPER_FLAGS_4(sve_cpy_z_b, TCG_CALL_NO_RWG, void, ptr, ptr, i64, i32) +DEF_HELPER_FLAGS_4(sve_cpy_z_h, TCG_CALL_NO_RWG, void, ptr, ptr, i64, i32) +DEF_HELPER_FLAGS_4(sve_cpy_z_s, TCG_CALL_NO_RWG, void, ptr, ptr, i64, i32) +DEF_HELPER_FLAGS_4(sve_cpy_z_d, TCG_CALL_NO_RWG, void, ptr, ptr, i64, i32) + DEF_HELPER_FLAGS_5(sve_and_pppp, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve_bic_pppp, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve_eor_pppp, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -XXX,XX +XXX,XX @@ void HELPER(sve_uqsubi_d)(void *d, void *a, uint64_t b, uint32_t desc) *(uint64_t *)(d + i) = (ai < b ? 0 : ai - b); } } + +/* Two operand predicated copy immediate with merge. All valid immediates + * can fit within 17 signed bits in the simd_data field. + */ +void HELPER(sve_cpy_m_b)(void *vd, void *vn, void *vg, + uint64_t mm, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc) / 8; + uint64_t *d = vd, *n = vn; + uint8_t *pg = vg; + + mm = dup_const(MO_8, mm); + for (i = 0; i < opr_sz; i += 1) { + uint64_t nn = n[i]; + uint64_t pp = expand_pred_b(pg[H1(i)]); + d[i] = (mm & pp) | (nn & ~pp); + } +} + +void HELPER(sve_cpy_m_h)(void *vd, void *vn, void *vg, + uint64_t mm, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc) / 8; + uint64_t *d = vd, *n = vn; + uint8_t *pg = vg; + + mm = dup_const(MO_16, mm); + for (i = 0; i < opr_sz; i += 1) { + uint64_t nn = n[i]; + uint64_t pp = expand_pred_h(pg[H1(i)]); + d[i] = (mm & pp) | (nn & ~pp); + } +} + +void HELPER(sve_cpy_m_s)(void *vd, void *vn, void *vg, + uint64_t mm, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc) / 8; + uint64_t *d = vd, *n = vn; + uint8_t *pg = vg; + + mm = dup_const(MO_32, mm); + for (i = 0; i < opr_sz; i += 1) { + uint64_t nn = n[i]; + uint64_t pp = expand_pred_s(pg[H1(i)]); + d[i] = (mm & pp) | (nn & ~pp); + } +} + +void HELPER(sve_cpy_m_d)(void *vd, void *vn, void *vg, + uint64_t mm, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc) / 8; + uint64_t *d = vd, *n = vn; + uint8_t *pg = vg; + + for (i = 0; i < opr_sz; i += 1) { + uint64_t nn = n[i]; + d[i] = (pg[H1(i)] & 1 ? mm : nn); + } +} + +void HELPER(sve_cpy_z_b)(void *vd, void *vg, uint64_t val, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc) / 8; + uint64_t *d = vd; + uint8_t *pg = vg; + + val = dup_const(MO_8, val); + for (i = 0; i < opr_sz; i += 1) { + d[i] = val & expand_pred_b(pg[H1(i)]); + } +} + +void HELPER(sve_cpy_z_h)(void *vd, void *vg, uint64_t val, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc) / 8; + uint64_t *d = vd; + uint8_t *pg = vg; + + val = dup_const(MO_16, val); + for (i = 0; i < opr_sz; i += 1) { + d[i] = val & expand_pred_h(pg[H1(i)]); + } +} + +void HELPER(sve_cpy_z_s)(void *vd, void *vg, uint64_t val, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc) / 8; + uint64_t *d = vd; + uint8_t *pg = vg; + + val = dup_const(MO_32, val); + for (i = 0; i < opr_sz; i += 1) { + d[i] = val & expand_pred_s(pg[H1(i)]); + } +} + +void HELPER(sve_cpy_z_d)(void *vd, void *vg, uint64_t val, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc) / 8; + uint64_t *d = vd; + uint8_t *pg = vg; + + for (i = 0; i < opr_sz; i += 1) { + d[i] = (pg[H1(i)] & 1 ? val : 0); + } +} diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static inline int plus1(int x) return x + 1; } +/* The SH bit is in bit 8. Extract the low 8 and shift. */ +static inline int expand_imm_sh8s(int x) +{ + return (int8_t)x << (x & 0x100 ? 8 : 0); +} + /* * Include the generated decoder. */ @@ -XXX,XX +XXX,XX @@ static bool trans_DUPM(DisasContext *s, arg_DUPM *a, uint32_t insn) return true; } +/* + *** SVE Integer Wide Immediate - Predicated Group + */ + +/* Implement all merging copies. This is used for CPY (immediate), + * FCPY, CPY (scalar), CPY (SIMD&FP scalar). + */ +static void do_cpy_m(DisasContext *s, int esz, int rd, int rn, int pg, + TCGv_i64 val) +{ + typedef void gen_cpy(TCGv_ptr, TCGv_ptr, TCGv_ptr, TCGv_i64, TCGv_i32); + static gen_cpy * const fns[4] = { + gen_helper_sve_cpy_m_b, gen_helper_sve_cpy_m_h, + gen_helper_sve_cpy_m_s, gen_helper_sve_cpy_m_d, + }; + unsigned vsz = vec_full_reg_size(s); + TCGv_i32 desc = tcg_const_i32(simd_desc(vsz, vsz, 0)); + TCGv_ptr t_zd = tcg_temp_new_ptr(); + TCGv_ptr t_zn = tcg_temp_new_ptr(); + TCGv_ptr t_pg = tcg_temp_new_ptr(); + + tcg_gen_addi_ptr(t_zd, cpu_env, vec_full_reg_offset(s, rd)); + tcg_gen_addi_ptr(t_zn, cpu_env, vec_full_reg_offset(s, rn)); + tcg_gen_addi_ptr(t_pg, cpu_env, pred_full_reg_offset(s, pg)); + + fns[esz](t_zd, t_zn, t_pg, val, desc); + + tcg_temp_free_ptr(t_zd); + tcg_temp_free_ptr(t_zn); + tcg_temp_free_ptr(t_pg); + tcg_temp_free_i32(desc); +} + +static bool trans_FCPY(DisasContext *s, arg_FCPY *a, uint32_t insn) +{ + if (a->esz == 0) { + return false; + } + if (sve_access_check(s)) { + /* Decode the VFP immediate. */ + uint64_t imm = vfp_expand_imm(a->esz, a->imm); + TCGv_i64 t_imm = tcg_const_i64(imm); + do_cpy_m(s, a->esz, a->rd, a->rn, a->pg, t_imm); + tcg_temp_free_i64(t_imm); + } + return true; +} + +static bool trans_CPY_m_i(DisasContext *s, arg_rpri_esz *a, uint32_t insn) +{ + if (a->esz == 0 && extract32(insn, 13, 1)) { + return false; + } + if (sve_access_check(s)) { + TCGv_i64 t_imm = tcg_const_i64(a->imm); + do_cpy_m(s, a->esz, a->rd, a->rn, a->pg, t_imm); + tcg_temp_free_i64(t_imm); + } + return true; +} + +static bool trans_CPY_z_i(DisasContext *s, arg_CPY_z_i *a, uint32_t insn) +{ + static gen_helper_gvec_2i * const fns[4] = { + gen_helper_sve_cpy_z_b, gen_helper_sve_cpy_z_h, + gen_helper_sve_cpy_z_s, gen_helper_sve_cpy_z_d, + }; + + if (a->esz == 0 && extract32(insn, 13, 1)) { + return false; + } + if (sve_access_check(s)) { + unsigned vsz = vec_full_reg_size(s); + TCGv_i64 t_imm = tcg_const_i64(a->imm); + tcg_gen_gvec_2i_ool(vec_full_reg_offset(s, a->rd), + pred_full_reg_offset(s, a->pg), + t_imm, vsz, vsz, 0, fns[a->esz]); + tcg_temp_free_i64(t_imm); + } + return true; +} + /* *** SVE Memory - 32-bit Gather and Unsized Contiguous Group */ diff --git a/target/arm/sve.decode b/target/arm/sve.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -XXX,XX +XXX,XX @@ ########################################################################### # Named fields. These are primarily for disjoint fields. -%imm4_16_p1 16:4 !function=plus1 +%imm4_16_p1 16:4 !function=plus1 %imm6_22_5 22:1 5:5 %imm9_16_10 16:s6 10:3 @@ -XXX,XX +XXX,XX @@ %tszimm16_shr 22:2 16:5 !function=tszimm_shr %tszimm16_shl 22:2 16:5 !function=tszimm_shl +# Signed 8-bit immediate, optionally shifted left by 8. +%sh8_i8s 5:9 !function=expand_imm_sh8s + # Either a copy of rd (at bit 0), or a different source # as propagated via the MOVPRFX instruction. %reg_movprfx 0:5 @@ -XXX,XX +XXX,XX @@ @rd_rn_tszimm ........ .. ... ... ...... rn:5 rd:5 \ &rri_esz esz=%tszimm16_esz +# Two register operand, one immediate operand, with 4-bit predicate. +# User must fill in imm. +@rdn_pg4 ........ esz:2 .. pg:4 ... ........ rd:5 \ + &rpri_esz rn=%reg_movprfx + # Two register operand, one encoded bitmask. @rdn_dbm ........ .. .... dbm:13 rd:5 \ &rr_dbm rn=%reg_movprfx @@ -XXX,XX +XXX,XX @@ AND_zzi 00000101 10 0000 ............. ..... @rdn_dbm # SVE broadcast bitmask immediate DUPM 00000101 11 0000 dbm:13 rd:5 +### SVE Integer Wide Immediate - Predicated Group + +# SVE copy floating-point immediate (predicated) +FCPY 00000101 .. 01 .... 110 imm:8 ..... @rdn_pg4 + +# SVE copy integer immediate (predicated) +CPY_m_i 00000101 .. 01 .... 01 . ........ ..... @rdn_pg4 imm=%sh8_i8s +CPY_z_i 00000101 .. 01 .... 00 . ........ ..... @rdn_pg4 imm=%sh8_i8s + ### SVE Predicate Logical Operations Group # SVE predicate logical operations -- 2.17.0 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20180516223007.10256-26-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/helper-sve.h | 2 + target/arm/sve_helper.c | 81 ++++++++++++++++++++++++++++++++++++++ target/arm/translate-sve.c | 34 ++++++++++++++++ target/arm/sve.decode | 7 ++++ 4 files changed, 124 insertions(+) diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_4(sve_cpy_z_h, TCG_CALL_NO_RWG, void, ptr, ptr, i64, i32) DEF_HELPER_FLAGS_4(sve_cpy_z_s, TCG_CALL_NO_RWG, void, ptr, ptr, i64, i32) DEF_HELPER_FLAGS_4(sve_cpy_z_d, TCG_CALL_NO_RWG, void, ptr, ptr, i64, i32) +DEF_HELPER_FLAGS_4(sve_ext, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_5(sve_and_pppp, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve_bic_pppp, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve_eor_pppp, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -XXX,XX +XXX,XX @@ void HELPER(sve_cpy_z_d)(void *vd, void *vg, uint64_t val, uint32_t desc) d[i] = (pg[H1(i)] & 1 ? val : 0); } } + +/* Big-endian hosts need to frob the byte indicies. If the copy + * happens to be 8-byte aligned, then no frobbing necessary. + */ +static void swap_memmove(void *vd, void *vs, size_t n) +{ + uintptr_t d = (uintptr_t)vd; + uintptr_t s = (uintptr_t)vs; + uintptr_t o = (d | s | n) & 7; + size_t i; + +#ifndef HOST_WORDS_BIGENDIAN + o = 0; +#endif + switch (o) { + case 0: + memmove(vd, vs, n); + break; + + case 4: + if (d < s || d >= s + n) { + for (i = 0; i < n; i += 4) { + *(uint32_t *)H1_4(d + i) = *(uint32_t *)H1_4(s + i); + } + } else { + for (i = n; i > 0; ) { + i -= 4; + *(uint32_t *)H1_4(d + i) = *(uint32_t *)H1_4(s + i); + } + } + break; + + case 2: + case 6: + if (d < s || d >= s + n) { + for (i = 0; i < n; i += 2) { + *(uint16_t *)H1_2(d + i) = *(uint16_t *)H1_2(s + i); + } + } else { + for (i = n; i > 0; ) { + i -= 2; + *(uint16_t *)H1_2(d + i) = *(uint16_t *)H1_2(s + i); + } + } + break; + + default: + if (d < s || d >= s + n) { + for (i = 0; i < n; i++) { + *(uint8_t *)H1(d + i) = *(uint8_t *)H1(s + i); + } + } else { + for (i = n; i > 0; ) { + i -= 1; + *(uint8_t *)H1(d + i) = *(uint8_t *)H1(s + i); + } + } + break; + } +} + +void HELPER(sve_ext)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t opr_sz = simd_oprsz(desc); + size_t n_ofs = simd_data(desc); + size_t n_siz = opr_sz - n_ofs; + + if (vd != vm) { + swap_memmove(vd, vn + n_ofs, n_siz); + swap_memmove(vd + n_siz, vm, n_ofs); + } else if (vd != vn) { + swap_memmove(vd + n_siz, vd, n_ofs); + swap_memmove(vd, vn + n_ofs, n_siz); + } else { + /* vd == vn == vm. Need temp space. */ + ARMVectorReg tmp; + swap_memmove(&tmp, vm, n_ofs); + swap_memmove(vd, vd + n_ofs, n_siz); + memcpy(vd + n_siz, &tmp, n_ofs); + } +} diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool trans_CPY_z_i(DisasContext *s, arg_CPY_z_i *a, uint32_t insn) return true; } +/* + *** SVE Permute Extract Group + */ + +static bool trans_EXT(DisasContext *s, arg_EXT *a, uint32_t insn) +{ + if (!sve_access_check(s)) { + return true; + } + + unsigned vsz = vec_full_reg_size(s); + unsigned n_ofs = a->imm >= vsz ? 0 : a->imm; + unsigned n_siz = vsz - n_ofs; + unsigned d = vec_full_reg_offset(s, a->rd); + unsigned n = vec_full_reg_offset(s, a->rn); + unsigned m = vec_full_reg_offset(s, a->rm); + + /* Use host vector move insns if we have appropriate sizes + * and no unfortunate overlap. + */ + if (m != d + && n_ofs == size_for_gvec(n_ofs) + && n_siz == size_for_gvec(n_siz) + && (d != n || n_siz <= n_ofs)) { + tcg_gen_gvec_mov(0, d, n + n_ofs, n_siz, n_siz); + if (n_ofs != 0) { + tcg_gen_gvec_mov(0, d + n_siz, m, n_ofs, n_ofs); + } + } else { + tcg_gen_gvec_3_ool(d, n, m, vsz, vsz, n_ofs, gen_helper_sve_ext); + } + return true; +} + /* *** SVE Memory - 32-bit Gather and Unsized Contiguous Group */ diff --git a/target/arm/sve.decode b/target/arm/sve.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -XXX,XX +XXX,XX @@ %imm4_16_p1 16:4 !function=plus1 %imm6_22_5 22:1 5:5 +%imm8_16_10 16:5 10:3 %imm9_16_10 16:s6 10:3 # A combination of tsz:imm3 -- extract esize. @@ -XXX,XX +XXX,XX @@ FCPY 00000101 .. 01 .... 110 imm:8 ..... @rdn_pg4 CPY_m_i 00000101 .. 01 .... 01 . ........ ..... @rdn_pg4 imm=%sh8_i8s CPY_z_i 00000101 .. 01 .... 00 . ........ ..... @rdn_pg4 imm=%sh8_i8s +### SVE Permute - Extract Group + +# SVE extract vector (immediate offset) +EXT 00000101 001 ..... 000 ... rm:5 rd:5 \ + &rrri rn=%reg_movprfx imm=%imm8_16_10 + ### SVE Predicate Logical Operations Group # SVE predicate logical operations -- 2.17.0 Big pullreq this week, since it's got RTH's PAN/UAO/ATS1E1 implementation in it, and also Philippe's raspi board model cleanup patchset, as well as a scattering of smaller stuff. -- PMM The following changes since commit 7ce9ce89930ce260af839fb3e3e5f9101f5c69a0: Merge remote-tracking branch 'remotes/kraxel/tags/ui-20200212-pull-request' into staging (2020-02-13 11:06:32 +0000) are available in the Git repository at: https://git.linaro.org/people/pmaydell/qemu-arm.git tags/pull-target-arm-20200213 for you to fetch changes up to dc7a88d0810ad272bdcd2e0869359af78fdd9114: target/arm: Implement ARMv8.1-VMID16 extension (2020-02-13 14:30:51 +0000) ---------------------------------------------------------------- target-arm queue: * i.MX: Fix inverted sense of register bits in watchdog timer * i.MX: Add support for WDT on i.MX6 * arm/virt: cleanups to ACPI tables * Implement ARMv8.1-VMID16 extension * Implement ARMv8.1-PAN * Implement ARMv8.2-UAO * Implement ARMv8.2-ATS1E1 * ast2400/2500/2600: Wire up EHCI controllers * hw/char/exynos4210_uart: Fix memleaks in exynos4210_uart_init * hw/arm/raspi: Clean up the board code ---------------------------------------------------------------- Chen Qun (1): hw/char/exynos4210_uart: Fix memleaks in exynos4210_uart_init Guenter Roeck (2): hw/arm: ast2400/ast2500: Wire up EHCI controllers hw/arm: ast2600: Wire up EHCI controllers Heyi Guo (7): bios-tables-test: prepare to change ARM virt ACPI DSDT arm/virt/acpi: remove meaningless sub device "RP0" from PCI0 arm/virt/acpi: remove _ADR from devices identified by _HID arm/acpi: fix PCI _PRT definition arm/acpi: fix duplicated _UID of PCI interrupt link devices arm/acpi: simplify the description of PCI _CRS virt/acpi: update golden masters for DSDT update Peter Maydell (1): target/arm: Implement ARMv8.1-VMID16 extension Philippe Mathieu-Daudé (13): hw/arm/raspi: Use BCM2708 machine type with pre Device Tree kernels hw/arm/raspi: Correct the board descriptions hw/arm/raspi: Extract the version from the board revision hw/arm/raspi: Extract the RAM size from the board revision hw/arm/raspi: Extract the processor type from the board revision hw/arm/raspi: Trivial code movement hw/arm/raspi: Make machines children of abstract RaspiMachineClass hw/arm/raspi: Make board_rev a field of RaspiMachineClass hw/arm/raspi: Let class_init() directly call raspi_machine_init() hw/arm/raspi: Set default RAM size to size encoded in board revision hw/arm/raspi: Extract the board model from the board revision hw/arm/raspi: Use a unique raspi_machine_class_init() method hw/arm/raspi: Extract the cores count from the board revision Richard Henderson (20): target/arm: Add arm_mmu_idx_is_stage1_of_2 target/arm: Add mmu_idx for EL1 and EL2 w/ PAN enabled target/arm: Add isar_feature tests for PAN + ATS1E1 target/arm: Move LOR regdefs to file scope target/arm: Split out aarch32_cpsr_valid_mask target/arm: Mask CPSR_J when Jazelle is not enabled target/arm: Replace CPSR_ERET_MASK with aarch32_cpsr_valid_mask target/arm: Use aarch32_cpsr_valid_mask in helper_exception_return target/arm: Remove CPSR_RESERVED target/arm: Introduce aarch64_pstate_valid_mask target/arm: Update MSR access for PAN target/arm: Update arm_mmu_idx_el for PAN target/arm: Enforce PAN semantics in get_S1prot target/arm: Set PAN bit as required on exception entry target/arm: Implement ATS1E1 system registers target/arm: Enable ARMv8.2-ATS1E1 in -cpu max target/arm: Add ID_AA64MMFR2_EL1 target/arm: Update MSR access to UAO target/arm: Implement UAO semantics target/arm: Enable ARMv8.2-UAO in -cpu max Roman Kapl (2): i.MX: Fix inverted register bits in wdt code. i.MX: Add support for WDT on i.MX6 include/hw/arm/aspeed_soc.h | 6 + include/hw/arm/fsl-imx6.h | 3 + target/arm/cpu-param.h | 2 +- target/arm/cpu.h | 95 ++++++++--- target/arm/internals.h | 85 ++++++++++ hw/arm/aspeed_ast2600.c | 23 +++ hw/arm/aspeed_soc.c | 25 +++ hw/arm/fsl-imx6.c | 21 +++ hw/arm/raspi.c | 190 ++++++++++++++++------ hw/arm/virt-acpi-build.c | 25 +-- hw/char/exynos4210_uart.c | 5 +- hw/misc/imx2_wdt.c | 2 +- target/arm/cpu.c | 4 + target/arm/cpu64.c | 10 ++ target/arm/helper-a64.c | 6 +- target/arm/helper.c | 327 +++++++++++++++++++++++++++++--------- target/arm/kvm64.c | 2 + target/arm/op_helper.c | 14 +- target/arm/translate-a64.c | 31 ++++ target/arm/translate.c | 42 +++-- tests/data/acpi/virt/DSDT | Bin 18462 -> 5307 bytes tests/data/acpi/virt/DSDT.memhp | Bin 19799 -> 6644 bytes tests/data/acpi/virt/DSDT.numamem | Bin 18462 -> 5307 bytes 23 files changed, 731 insertions(+), 187 deletions(-) From: Roman Kapl <rka@sysgo.com> Documentation says for WDA '0: Assert WDOG output.' and for SRS '0: Assert system reset signal.'. Signed-off-by: Roman Kapl <rka@sysgo.com> Message-id: 20200207095409.11227-1-rka@sysgo.com Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- hw/misc/imx2_wdt.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/hw/misc/imx2_wdt.c b/hw/misc/imx2_wdt.c index XXXXXXX..XXXXXXX 100644 --- a/hw/misc/imx2_wdt.c +++ b/hw/misc/imx2_wdt.c @@ -XXX,XX +XXX,XX @@ static void imx2_wdt_write(void *opaque, hwaddr addr, uint64_t value, unsigned int size) { if (addr == IMX2_WDT_WCR && - (value & (IMX2_WDT_WCR_WDA | IMX2_WDT_WCR_SRS))) { + (~value & (IMX2_WDT_WCR_WDA | IMX2_WDT_WCR_SRS))) { watchdog_perform_action(); } } -- 2.20.1 From: Roman Kapl <rka@sysgo.com> Uses the i.MX2 rudimentary watchdog driver. Signed-off-by: Roman Kapl <rka@sysgo.com> Message-id: 20200207095529.11309-1-rka@sysgo.com Reviewed-by: Peter Maydell <peter.maydell@linaro.org> [PMM: removed accidental duplicate #include line] Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- include/hw/arm/fsl-imx6.h | 3 +++ hw/arm/fsl-imx6.c | 21 +++++++++++++++++++++ 2 files changed, 24 insertions(+) diff --git a/include/hw/arm/fsl-imx6.h b/include/hw/arm/fsl-imx6.h index XXXXXXX..XXXXXXX 100644 --- a/include/hw/arm/fsl-imx6.h +++ b/include/hw/arm/fsl-imx6.h @@ -XXX,XX +XXX,XX @@ #include "hw/cpu/a9mpcore.h" #include "hw/misc/imx6_ccm.h" #include "hw/misc/imx6_src.h" +#include "hw/misc/imx2_wdt.h" #include "hw/char/imx_serial.h" #include "hw/timer/imx_gpt.h" #include "hw/timer/imx_epit.h" @@ -XXX,XX +XXX,XX @@ #define FSL_IMX6_NUM_GPIOS 7 #define FSL_IMX6_NUM_ESDHCS 4 #define FSL_IMX6_NUM_ECSPIS 5 +#define FSL_IMX6_NUM_WDTS 2 typedef struct FslIMX6State { /*< private >*/ @@ -XXX,XX +XXX,XX @@ typedef struct FslIMX6State { IMXGPIOState gpio[FSL_IMX6_NUM_GPIOS]; SDHCIState esdhc[FSL_IMX6_NUM_ESDHCS]; IMXSPIState spi[FSL_IMX6_NUM_ECSPIS]; + IMX2WdtState wdt[FSL_IMX6_NUM_WDTS]; IMXFECState eth; MemoryRegion rom; MemoryRegion caam; diff --git a/hw/arm/fsl-imx6.c b/hw/arm/fsl-imx6.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/fsl-imx6.c +++ b/hw/arm/fsl-imx6.c @@ -XXX,XX +XXX,XX @@ static void fsl_imx6_init(Object *obj) sysbus_init_child_obj(obj, name, &s->spi[i], sizeof(s->spi[i]), TYPE_IMX_SPI); } + for (i = 0; i < FSL_IMX6_NUM_WDTS; i++) { + snprintf(name, NAME_SIZE, "wdt%d", i); + sysbus_init_child_obj(obj, name, &s->wdt[i], sizeof(s->wdt[i]), + TYPE_IMX2_WDT); + } + sysbus_init_child_obj(obj, "eth", &s->eth, sizeof(s->eth), TYPE_IMX_ENET); } @@ -XXX,XX +XXX,XX @@ static void fsl_imx6_realize(DeviceState *dev, Error **errp) qdev_get_gpio_in(DEVICE(&s->a9mpcore), FSL_IMX6_ENET_MAC_1588_IRQ)); + /* + * Watchdog + */ + for (i = 0; i < FSL_IMX6_NUM_WDTS; i++) { + static const hwaddr FSL_IMX6_WDOGn_ADDR[FSL_IMX6_NUM_WDTS] = { + FSL_IMX6_WDOG1_ADDR, + FSL_IMX6_WDOG2_ADDR, + }; + + object_property_set_bool(OBJECT(&s->wdt[i]), true, "realized", + &error_abort); + + sysbus_mmio_map(SYS_BUS_DEVICE(&s->wdt[i]), 0, FSL_IMX6_WDOGn_ADDR[i]); + } + /* ROM memory */ memory_region_init_rom(&s->rom, NULL, "imx6.rom", FSL_IMX6_ROM_SIZE, &err); -- 2.20.1 From: Heyi Guo <guoheyi@huawei.com> We are going to change ARM virt ACPI DSDT table, which will cause make check to fail, so temporarily add related golden masters to ignore list. Signed-off-by: Heyi Guo <guoheyi@huawei.com> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Message-id: 20200204014325.16279-2-guoheyi@huawei.com Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- tests/qtest/bios-tables-test-allowed-diff.h | 3 +++ 1 file changed, 3 insertions(+) diff --git a/tests/qtest/bios-tables-test-allowed-diff.h b/tests/qtest/bios-tables-test-allowed-diff.h index XXXXXXX..XXXXXXX 100644 --- a/tests/qtest/bios-tables-test-allowed-diff.h +++ b/tests/qtest/bios-tables-test-allowed-diff.h @@ -1 +1,4 @@ /* List of comma-separated changed AML files to ignore */ +"tests/data/acpi/virt/DSDT", +"tests/data/acpi/virt/DSDT.memhp", +"tests/data/acpi/virt/DSDT.numamem", -- 2.20.1 From: Heyi Guo <guoheyi@huawei.com> The sub device "RP0" under PCI0 in ACPI/DSDT does not contain any method or property other than "_ADR", so it is safe to remove it. Signed-off-by: Heyi Guo <guoheyi@huawei.com> Acked-by: "Michael S. Tsirkin" <mst@redhat.com> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Message-id: 20200204014325.16279-3-guoheyi@huawei.com Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- hw/arm/virt-acpi-build.c | 4 ---- 1 file changed, 4 deletions(-) diff --git a/hw/arm/virt-acpi-build.c b/hw/arm/virt-acpi-build.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/virt-acpi-build.c +++ b/hw/arm/virt-acpi-build.c @@ -XXX,XX +XXX,XX @@ static void acpi_dsdt_add_pci(Aml *scope, const MemMapEntry *memmap, aml_append(method, aml_return(buf)); aml_append(dev, method); - Aml *dev_rp0 = aml_device("%s", "RP0"); - aml_append(dev_rp0, aml_name_decl("_ADR", aml_int(0))); - aml_append(dev, dev_rp0); - Aml *dev_res0 = aml_device("%s", "RES0"); aml_append(dev_res0, aml_name_decl("_HID", aml_string("PNP0C02"))); crs = aml_resource_template(); -- 2.20.1 From: Heyi Guo <guoheyi@huawei.com> According to ACPI spec, _ADR should be used for device on a bus that has a standard enumeration algorithm, but not for device which is on system bus and must be enumerated by OSPM. And it is not recommended to contain both _HID and _ADR in a single device. See ACPI 6.3, section 6.1, top of page 343: A device object must contain either an _HID object or an _ADR object, but should not contain both. (https://uefi.org/sites/default/files/resources/ACPI_6_3_May16.pdf) Signed-off-by: Heyi Guo <guoheyi@huawei.com> Acked-by: Igor Mammedov <imammedo@redhat.com> Acked-by: Michael S. Tsirkin <mst@redhat.com> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Message-id: 20200204014325.16279-4-guoheyi@huawei.com Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- hw/arm/virt-acpi-build.c | 8 -------- 1 file changed, 8 deletions(-) diff --git a/hw/arm/virt-acpi-build.c b/hw/arm/virt-acpi-build.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/virt-acpi-build.c +++ b/hw/arm/virt-acpi-build.c @@ -XXX,XX +XXX,XX @@ static void acpi_dsdt_add_uart(Aml *scope, const MemMapEntry *uart_memmap, AML_EXCLUSIVE, &uart_irq, 1)); aml_append(dev, aml_name_decl("_CRS", crs)); - /* The _ADR entry is used to link this device to the UART described - * in the SPCR table, i.e. SPCR.base_address.address == _ADR. - */ - aml_append(dev, aml_name_decl("_ADR", aml_int(uart_memmap->base))); - aml_append(scope, dev); } @@ -XXX,XX +XXX,XX @@ static void acpi_dsdt_add_pci(Aml *scope, const MemMapEntry *memmap, aml_append(dev, aml_name_decl("_CID", aml_string("PNP0A03"))); aml_append(dev, aml_name_decl("_SEG", aml_int(0))); aml_append(dev, aml_name_decl("_BBN", aml_int(0))); - aml_append(dev, aml_name_decl("_ADR", aml_int(0))); aml_append(dev, aml_name_decl("_UID", aml_string("PCI0"))); aml_append(dev, aml_name_decl("_STR", aml_unicode("PCIe 0 Device"))); aml_append(dev, aml_name_decl("_CCA", aml_int(1))); @@ -XXX,XX +XXX,XX @@ static void acpi_dsdt_add_gpio(Aml *scope, const MemMapEntry *gpio_memmap, { Aml *dev = aml_device("GPO0"); aml_append(dev, aml_name_decl("_HID", aml_string("ARMH0061"))); - aml_append(dev, aml_name_decl("_ADR", aml_int(0))); aml_append(dev, aml_name_decl("_UID", aml_int(0))); Aml *crs = aml_resource_template(); @@ -XXX,XX +XXX,XX @@ static void acpi_dsdt_add_power_button(Aml *scope) { Aml *dev = aml_device(ACPI_POWER_BUTTON_DEVICE); aml_append(dev, aml_name_decl("_HID", aml_string("PNP0C0C"))); - aml_append(dev, aml_name_decl("_ADR", aml_int(0))); aml_append(dev, aml_name_decl("_UID", aml_int(0))); aml_append(scope, dev); } -- 2.20.1 From: Heyi Guo <guoheyi@huawei.com> The address field in each _PRT mapping package should be constructed with high word for device# and low word for function#, so it is wrong to use bus_no as the high word. The existing code adds a bunch useless entries with device #s above 31. Enumerate all possible slots (i.e. PCI_SLOT_MAX) instead. Signed-off-by: Heyi Guo <guoheyi@huawei.com> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Message-id: 20200204014325.16279-5-guoheyi@huawei.com Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- hw/arm/virt-acpi-build.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/hw/arm/virt-acpi-build.c b/hw/arm/virt-acpi-build.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/virt-acpi-build.c +++ b/hw/arm/virt-acpi-build.c @@ -XXX,XX +XXX,XX @@ static void acpi_dsdt_add_pci(Aml *scope, const MemMapEntry *memmap, { int ecam_id = VIRT_ECAM_ID(highmem_ecam); Aml *method, *crs, *ifctx, *UUID, *ifctx1, *elsectx, *buf; - int i, bus_no; + int i, slot_no; hwaddr base_mmio = memmap[VIRT_PCIE_MMIO].base; hwaddr size_mmio = memmap[VIRT_PCIE_MMIO].size; hwaddr base_pio = memmap[VIRT_PCIE_PIO].base; @@ -XXX,XX +XXX,XX @@ static void acpi_dsdt_add_pci(Aml *scope, const MemMapEntry *memmap, aml_append(dev, aml_name_decl("_CCA", aml_int(1))); /* Declare the PCI Routing Table. */ - Aml *rt_pkg = aml_varpackage(nr_pcie_buses * PCI_NUM_PINS); - for (bus_no = 0; bus_no < nr_pcie_buses; bus_no++) { + Aml *rt_pkg = aml_varpackage(PCI_SLOT_MAX * PCI_NUM_PINS); + for (slot_no = 0; slot_no < PCI_SLOT_MAX; slot_no++) { for (i = 0; i < PCI_NUM_PINS; i++) { - int gsi = (i + bus_no) % PCI_NUM_PINS; + int gsi = (i + slot_no) % PCI_NUM_PINS; Aml *pkg = aml_package(4); - aml_append(pkg, aml_int((bus_no << 16) | 0xFFFF)); + aml_append(pkg, aml_int((slot_no << 16) | 0xFFFF)); aml_append(pkg, aml_int(i)); aml_append(pkg, aml_name("GSI%d", gsi)); aml_append(pkg, aml_int(0)); -- 2.20.1 From: Heyi Guo <guoheyi@huawei.com> Using _UID of 0 for all PCI interrupt link devices absolutely violates the spec. Simply increase one by one. Signed-off-by: Heyi Guo <guoheyi@huawei.com> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Message-id: 20200204014325.16279-6-guoheyi@huawei.com Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- hw/arm/virt-acpi-build.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/hw/arm/virt-acpi-build.c b/hw/arm/virt-acpi-build.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/virt-acpi-build.c +++ b/hw/arm/virt-acpi-build.c @@ -XXX,XX +XXX,XX @@ static void acpi_dsdt_add_pci(Aml *scope, const MemMapEntry *memmap, uint32_t irqs = irq + i; Aml *dev_gsi = aml_device("GSI%d", i); aml_append(dev_gsi, aml_name_decl("_HID", aml_string("PNP0C0F"))); - aml_append(dev_gsi, aml_name_decl("_UID", aml_int(0))); + aml_append(dev_gsi, aml_name_decl("_UID", aml_int(i))); crs = aml_resource_template(); aml_append(crs, aml_interrupt(AML_CONSUMER, AML_LEVEL, AML_ACTIVE_HIGH, -- 2.20.1 From: Heyi Guo <guoheyi@huawei.com> The original code defines a named object for the resource template but then returns the resource template object itself; the resulted output is like below: Method (_CRS, 0, NotSerialized) // _CRS: Current Resource Settings { Name (RBUF, ResourceTemplate () { WordBusNumber (ResourceProducer, MinFixed, MaxFixed, PosDecode, 0x0000, // Granularity 0x0000, // Range Minimum 0x00FF, // Range Maximum 0x0000, // Translation Offset 0x0100, // Length ,, ) ...... }) Return (ResourceTemplate () { WordBusNumber (ResourceProducer, MinFixed, MaxFixed, PosDecode, 0x0000, // Granularity 0x0000, // Range Minimum 0x00FF, // Range Maximum 0x0000, // Translation Offset 0x0100, // Length ,, ) ...... }) } So the named object "RBUF" is actually useless. The more natural way is to return RBUF instead, or simply drop RBUF definition. Choose the latter one to simplify the code. Signed-off-by: Heyi Guo <guoheyi@huawei.com> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Message-id: 20200204014325.16279-7-guoheyi@huawei.com Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- hw/arm/virt-acpi-build.c | 1 - 1 file changed, 1 deletion(-) diff --git a/hw/arm/virt-acpi-build.c b/hw/arm/virt-acpi-build.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/virt-acpi-build.c +++ b/hw/arm/virt-acpi-build.c @@ -XXX,XX +XXX,XX @@ static void acpi_dsdt_add_pci(Aml *scope, const MemMapEntry *memmap, size_mmio_high)); } - aml_append(method, aml_name_decl("RBUF", rbuf)); aml_append(method, aml_return(rbuf)); aml_append(dev, method); -- 2.20.1 From: Heyi Guo <guoheyi@huawei.com> Differences between disassembled ASL files: @@ -XXX,XX +XXX,XX @@ * * Disassembling to symbolic ASL+ operators * - * Disassembly of DSDT, Thu Jan 23 16:00:04 2020 + * Disassembly of DSDT.new, Thu Jan 23 16:47:12 2020 * * Original Table Header: * Signature "DSDT" - * Length 0x0000481E (18462) + * Length 0x000014BB (5307) * Revision 0x02 - * Checksum 0x60 + * Checksum 0xD1 * OEM ID "BOCHS " * OEM Table ID "BXPCDSDT" * OEM Revision 0x00000001 (1) @@ -XXX,XX +XXX,XX @@ DefinitionBlock ("", "DSDT", 2, "BOCHS ", "BXPCDSDT", 0x00000001) 0x00000021, } }) - Name (_ADR, 0x09000000) // _ADR: Address } Device (FLS0) @@ -XXX,XX +XXX,XX @@ DefinitionBlock ("", "DSDT", 2, "BOCHS ", "BXPCDSDT", 0x00000001) Name (_CID, "PNP0A03" /* PCI Bus */) // _CID: Compatible ID Name (_SEG, Zero) // _SEG: PCI Segment Name (_BBN, Zero) // _BBN: BIOS Bus Number - Name (_ADR, Zero) // _ADR: Address Name (_UID, "PCI0") // _UID: Unique ID Name (_STR, Unicode ("PCIe 0 Device")) // _STR: Description String Name (_CCA, One) // _CCA: Cache Coherency Attribute - Name (_PRT, Package (0x0400) // _PRT: PCI Routing Table + Name (_PRT, Package (0x80) // _PRT: PCI Routing Table { Package (0x04) { @@ -XXX,XX +XXX,XX @@ DefinitionBlock ("", "DSDT", 2, "BOCHS ", "BXPCDSDT", 0x00000001) 0x03, GSI2, Zero - }, - - Package (0x04) - { - 0x0020FFFF, - Zero, - GSI0, - Zero - }, - - *Omit the other (4 * (256 - 32) - 2) packages* - - Package (0x04) - { - 0x00FFFFFF, - 0x03, - GSI2, - Zero } }) Device (GSI0) @@ -XXX,XX +XXX,XX @@ DefinitionBlock ("", "DSDT", 2, "BOCHS ", "BXPCDSDT", 0x00000001) Device (GSI1) { Name (_HID, "PNP0C0F" /* PCI Interrupt Link Device */) // _HID: Hardware ID - Name (_UID, Zero) // _UID: Unique ID + Name (_UID, One) // _UID: Unique ID Name (_PRS, ResourceTemplate () // _PRS: Possible Resource Settings { Interrupt (ResourceConsumer, Level, ActiveHigh, Exclusive, ,, ) @@ -XXX,XX +XXX,XX @@ DefinitionBlock ("", "DSDT", 2, "BOCHS ", "BXPCDSDT", 0x00000001) Device (GSI2) { Name (_HID, "PNP0C0F" /* PCI Interrupt Link Device */) // _HID: Hardware ID - Name (_UID, Zero) // _UID: Unique ID + Name (_UID, 0x02) // _UID: Unique ID Name (_PRS, ResourceTemplate () // _PRS: Possible Resource Settings { Interrupt (ResourceConsumer, Level, ActiveHigh, Exclusive, ,, ) @@ -XXX,XX +XXX,XX @@ DefinitionBlock ("", "DSDT", 2, "BOCHS ", "BXPCDSDT", 0x00000001) Device (GSI3) { Name (_HID, "PNP0C0F" /* PCI Interrupt Link Device */) // _HID: Hardware ID - Name (_UID, Zero) // _UID: Unique ID + Name (_UID, 0x03) // _UID: Unique ID Name (_PRS, ResourceTemplate () // _PRS: Possible Resource Settings { Interrupt (ResourceConsumer, Level, ActiveHigh, Exclusive, ,, ) @@ -XXX,XX +XXX,XX @@ DefinitionBlock ("", "DSDT", 2, "BOCHS ", "BXPCDSDT", 0x00000001) Method (_CRS, 0, NotSerialized) // _CRS: Current Resource Settings { - Name (RBUF, ResourceTemplate () - { - WordBusNumber (ResourceProducer, MinFixed, MaxFixed, PosDecode, - 0x0000, // Granularity - 0x0000, // Range Minimum - 0x00FF, // Range Maximum - 0x0000, // Translation Offset - 0x0100, // Length - ,, ) - DWordMemory (ResourceProducer, PosDecode, MinFixed, MaxFixed, NonCacheable, ReadWrite, - 0x00000000, // Granularity - 0x10000000, // Range Minimum - 0x3EFEFFFF, // Range Maximum - 0x00000000, // Translation Offset - 0x2EFF0000, // Length - ,, , AddressRangeMemory, TypeStatic) - DWordIO (ResourceProducer, MinFixed, MaxFixed, PosDecode, EntireRange, - 0x00000000, // Granularity - 0x00000000, // Range Minimum - 0x0000FFFF, // Range Maximum - 0x3EFF0000, // Translation Offset - 0x00010000, // Length - ,, , TypeStatic, DenseTranslation) - QWordMemory (ResourceProducer, PosDecode, MinFixed, MaxFixed, NonCacheable, ReadWrite, - 0x0000000000000000, // Granularity - 0x0000008000000000, // Range Minimum - 0x000000FFFFFFFFFF, // Range Maximum - 0x0000000000000000, // Translation Offset - 0x0000008000000000, // Length - ,, , AddressRangeMemory, TypeStatic) - }) Return (ResourceTemplate () { WordBusNumber (ResourceProducer, MinFixed, MaxFixed, PosDecode, @@ -XXX,XX +XXX,XX @@ DefinitionBlock ("", "DSDT", 2, "BOCHS ", "BXPCDSDT", 0x00000001) }) } - Device (RP0) - { - Name (_ADR, Zero) // _ADR: Address - } - Device (RES0) { Name (_HID, "PNP0C02" /* PNP Motherboard Resources */) // _HID: Hardware ID @@ -XXX,XX +XXX,XX @@ DefinitionBlock ("", "DSDT", 2, "BOCHS ", "BXPCDSDT", 0x00000001) Device (PWRB) { Name (_HID, "PNP0C0C" /* Power Button Device */) // _HID: Hardware ID - Name (_ADR, Zero) // _ADR: Address Name (_UID, Zero) // _UID: Unique ID } } The differences between the two versions of DSDT.memhp are almost the same as the above, except for total length and checksum. DSDT.numamem binary is just the same with DSDT on virt machine, so we don't show the differences again. Signed-off-by: Heyi Guo <guoheyi@huawei.com> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Message-id: 20200204014325.16279-8-guoheyi@huawei.com Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- tests/qtest/bios-tables-test-allowed-diff.h | 3 --- tests/data/acpi/virt/DSDT | Bin 18462 -> 5307 bytes tests/data/acpi/virt/DSDT.memhp | Bin 19799 -> 6644 bytes tests/data/acpi/virt/DSDT.numamem | Bin 18462 -> 5307 bytes 4 files changed, 3 deletions(-) diff --git a/tests/qtest/bios-tables-test-allowed-diff.h b/tests/qtest/bios-tables-test-allowed-diff.h index XXXXXXX..XXXXXXX 100644 --- a/tests/qtest/bios-tables-test-allowed-diff.h +++ b/tests/qtest/bios-tables-test-allowed-diff.h @@ -1,4 +1 @@ /* List of comma-separated changed AML files to ignore */ -"tests/data/acpi/virt/DSDT", -"tests/data/acpi/virt/DSDT.memhp", -"tests/data/acpi/virt/DSDT.numamem", diff --git a/tests/data/acpi/virt/DSDT b/tests/data/acpi/virt/DSDT index XXXXXXX..XXXXXXX 100644 GIT binary patch delta 156 zcmbO?fpNDcmrJlq$Zin^2BwP>xulufJQ*iyC^K43^tIeLL4lLWeZ}O>oO+X=b6T<Z z6mvCfR_C%{pDgc^#>hCi&BajKi^V<I(}*M9!_$Q~z%RhS*}#o~BR<sAg^OwOMVP!X lHhJdBGOmtnjvVpMLBX3Jy81CrwsAkqC^^YPgaxRb0RT`TDiQzy literal 18462 zcmc)ScXSkm8iw%+36N|;NFdS#0*VC-rifrC*(4Ap5OxEoL4yrNET~uz6^x349TdAp z#ol{Y6npQeh`smTHTRwDuD;K8uK!-nakJ0v%s2Z>CNML{-I`=g)4(x7&}nM*`1qLQ zpz7@!<28CLD+q${e)zR$!Q7lFEy?PZ=GK1kva+(=mNE4;-Kye^^@<TeZp*~_nxMJ0 zHYYy5A@gLSVN6+Bd3pND+?IGES==wydwyOJPRt96f?z?HAS-LIYPOcDs!0@tPc*ld z*Nsi4r;Ht!7_TYAF{L<Gn4Y5LgPhsga=1!)>Q!--tkj18UL_~9%E-FO@w(J16KWeK z3R0o1B%7*Y`C2Dl_1|lD%Il+5!;MwtOiE<F2dS-<*$ez@&A+j+pi>%K<|FWeGb6&y z{$oU^;O`OT=@Hf8tEg~uW<;!0)QlXPQQ<QxBWGks&FEq?Dt*Srku!3lX5`w8jeW-O z$QhlZX2fj9aG$YB<cuy+GYV|RCO%_C<czLSGYW0S2%j-Baz<{{j3S#c(r0WMIU_G> zMh}~@sm&<IuhC!oM=WYaiOtx|XGHF%{3Xfk>b-2n<~}2OKP`xQ9er%Z7Cs|-KkXJZ zqo2*#(q}~Pr-e~7`rC}Hd`9$s+C6H<K%23(&xqbni=t)>vKga%M)ZDK95rJNn^EO6 zqW9AtQ8R|vjB1||y`T1snz6Rch}(>c=>4=LYR0-YqsC`M@29Ip%~;Q7)cTC*{j^uq zj16o?ozIBgPkTqr7-lm@`;6%Qv`^HGGMk})#ykFn3jb}Wh~7{8M$M?O8TCFRdOz(K zHDhC&v5n7&-cMJLnz4z^*w$x6@2CBvW{k8MV|_;SemWp(#%4C7!DduO@23N!W^7?I z#`%os{j@Y{##T0Ce0s)$RoRX4`t%EF9M@P@RW?!wE^!@@rK&PKjHw;1+v@6Zy48V| zZgqs#EnF{rvMEtq8tdN}#Dn@^_h3*^rvGYm@8Dp1u&cfXS}1i8(wJ!KdGdwX`9V&P z{G9yu_F!~UBU1OXbiX|4Q4l^J>!hg2M7E+b=+P~wpuIgS2-nea=?d4<f`zH)I@Z&o znGDy|{ElUH%#>O-UE!LUMRh<FZc&SNtf>sw%xopQW6jJf$PU6aGmB=Y*3_aMYbwJv z^@=*SqNBsqvgt}2I~LUeR9cxycXo!ebH_F_&d#YdGcR80&Mt83kXWxEv#1WZ+^KYD zS2(-E*_BSEJ9FX8?N~GOEztp*JC*LtgHs3dsqbFLw<M2Fr8{GA#^BTeojaB8%!e}{ z&U`wR?$jBD{X7fc)B&J7mG0~YXE!*z(W!K2A)JM9>VVOmN_TdLvpbyK=~TK?CsXDu zf>Q^W?o_(77|voii|JIlvj?0#;M4)BJC*M231?3@d(x?NX9=7oaOwcool1AE2Ip#U zu12TQoxR}f1*Z;p-KlhEZ#a9y*_%$KJ9VmQKhHjJ>HyiDN_X~!voD-|=~TM2ADsQ* z)B&|SmF`>}&eh>uold1Y`@`8EP91=|Q|ZnDa1MZT0G&#A4uo?coH}54r_!CJaF)VZ zN~hADgWwzlrw;JlsdVRHI0wTym`<fT*MM^kICVncPNh57gmX<e*Q8VF&LMCPfm0_C z?o_&SEjZVLb1gcR?pzzrwc*qWhdY(-TnEl|;9Q4Jr90P!b6q%fg5pl4JBPwK6waY^ zD&4sroa@1<6B>6a-MK!T>%+M|ol19Z0Otm9>IBH0N_TDu=Z0`@NT<@B!{8hSr%ssM zsdVQ?aBc+WMszCOsq1p~_iY)RI>B<M(w({-r!_N2p5<_s)2Vc)F2ZTe%#mjWoH`+M zr_!Ce^rkg4$G4SmR??|-r!Kgeb7MGl0_RSpJ9U}OoWtQ9PN&kHy0~V}P2kiCpF5TA z)Fm}@j(~Fnol1A=LYg^8!l@HPcPibf%V*}?6wXcQRJv0a&CIzOoI0U&r_!CeRA$c2 z;oO`~r8{+j%$!@msS{9lD&48eV&>cu&MoOwx>Fa!%()euI$?FE(w({lX3nkQ+?r0M zJ4eAe3eHh<D&1KHXBC`PbSmAci(KZchO?SZr8{+L%bamI<8&(BsS8?KGjj%917{7L zN_Xlqme$PtJyZ*4EuBht>f)6->)@=TQ|V4!vNGprI7ic|bf+#<nR5)BW9U@6Q<taA zSr2DDol1A=qLewefpZ%=mG0D~D06NL=eBez-Kh&u<{S&>SUQ#N)MY1gHo)0Hr_!Ce z*ksOeaE_x>=}ui@GUs?W$J42FXCs`Aa5mDZbms&(C%`#@PNh2&a3<hP(5ZB163!%? zNjjD8Y=W~1&L%pQ?wkncL^vnXsdVQgI48k5iB6?EC&M`z&dGEt-MJl{+rhaVol19Z z59ju9ZcnGuojbs}1Dre1sdQ&EoXv1H)2Vdl6ga2AIfYK8J9mV0M>uz+Q|Zo~;M@t$ zo#<4$vjxr;I9upcx^pU=Q{kLSr_!A}!?`n@JJYFj=Pq#W0_QGtD&4s&oV&ugE1gPr z?gr;>aPCH@(w(~MZGYeH4(IN4D&08^&S`K?qf_b5J>c8}&OPW<x^quB_k?p#I+gC6 z4(D_@r_-r)=U#B`1?OILD&08)&KYpdpi}A2z2V#&&b{eWx^o{m_knXCI+gC+7tVd* z+?P(JJNJWgKREZJQ|Zo`aL$BtCY?%m?hohwaPCj1(wzsuc>tUT(5ZCifp8uO=Ye!8 z-8l=+S#ZvxQ|Zow;5-P<gXmPc^I$j+hVx)LmF_$Q&O_ilgifV94~6qkI1i;$>CV}3 z&W3X~ol18e2IpaL9!96qopa!v1Lqt%mF_$o&coq6oKB@XkAU+CIFF!H>CPkJJQB_$ z=~TLNE}V1WoJ*(Do%7(F2j@IGmF_$W&ZFQwicY0FTj6Ylvz1PzJCBC*XgH6iQ|ZoQ z;5-J-W9U@6^H?~Kh4WZCmF}Dm=X^Nl)2Vdlac~|7=W%o@-FZBm$HRF%ol18ufO7$y z3+Pn3b0M4y;ao_k(w!&3c><g#(5ZCiiEy3>=ZSPG-FXt6C&76Vol19}4Cl#ko=m6G zou|Ng3Y@3VsdVS5aGnb1sdOscc^aIj!Fd{;N_Q@Ta}k`2=v2CMF`SFxTui6Zou|Wj zI-IA|sdVQVaGn9@8FVV$xdhH7a4w-!>CQ9ZJQL0{=~TM&EI7}C^DH`*?mQdLv*A3O zPNh4~f%6<V&!JQ4&U4{B7tV9(RJ!v#IM0LgJUW%`JRi>U;XI#Cr8_Tx^8z?8pi}A2 z3*o#F&I{>Oy0Z<=HaOepRJ!vbI4^?pB081syco`l;k=kmr8_Ty^Ab2Op;PJ3rEo5V zb19ulcU}tTrEp$Kr_!C5!Fd^+m(i(o=jCu-4(H``D&2VnoL9hk1)WNFUJ2)wa9&BL z(w$eqc@>;j(W!Lj)o@-7=hbv7-FXe1*T8uVol19J3+J_PUQ4Iao!7y69h}$EsdVS{ za9$7R^>ixTc>|m`z<C3mN_XA}=Z$dQNT<@BH^F%moHx;_bmz@*-VEo>bSm9>3!JyW zc?+FNcisx;t#IB-r_!Cb!Fd~;x6!F|=k0Lb4(IK3D&2VpoOi%^2c1fH-U;WOaNbF$ z(w%p~c^8~_(W!Lj-EiIw=iPKF-FXk3_rQ4%ol1A!3+KIX-b<&_o%g|cADs8msdVT4 zaNZB+{d6kbxeU%_a4w@$>COk>d;rb|=v2D%K{y|T^Fcb5?pzM%ayXaMsdVQ<a6Sa* zLv$+L`7oRh!}&0sN_Rd2=Ob`FLZ{N5kHYyVoR89}bmwDmJ_hGwbSmBXIGm5e`8b_Q zcRm5<6L3C3r_!BI!uceePtvJ$=TmS#1?N+AD&6@soKM5~G@VL!J_F}7a6Ut)(w)!3 z`7E5z(y4Ukb8tQf=W}!_-T6G6&%^mVol1AU0Ot#EzCfqaoiD=qBAhSMsdVQ{aJ~fR zOLQvT`7)d@!}&6uN_V~j=PPi&LZ{N5E8tuK=L$NN?tB%_SK)k>PNh3vgYz{wU!zm$ z&e!359nRP3RJwB|oGam6NvG1CZ@~EmoNv&nbmuBKSHZc8PNh5Fg!4@}-=tIN&bQ!v z3(mLbRJ!wRINyfzZ90|id<V{V;CzQpr90n+^IbUKrBmt7_uzaF&iCk4y7PTF--q*k zI+gDH0L~BK{D4lSJ3oZ;LpVRAQ|Zo+;QR>AkLXmo^J6$ahVx@OmG1lm&QIX{gifV9 zKZWyCI6tLR>CVsK{0z>|=v2D%b2vYT^K&|t?)(DIFW~%wPNh4)g!4-{zob*?&adG7 z3eK<SRJ!wPIKPJTYdV$g{07c%;QWS8r8~cc^IJH-rBmt7@8J9n&hO|{y7PNDzlZaC zI+gDH0nQ)b{DDrTJAZ`pM>v0^Q|Zp1;QR^BpXgM&^Jh4JhVy4SmG1ln&R^jCg-)eA ze}(f`IDe&6>CWHa{0+|E=v2D%cQ}8C^LILx?)(GJKj8d>PNh5lg!4~0|D;pt&UQH4 z;cTZ<nQ}I_*5~MdjIsBd#>?tb?<du5qdwH5FqYr(K^|)csSol9Kj?#xm2_!ICX!j{ zQR(-;hHqB=U!#UZj7mMmQR%m9|J$gwB1WYi<EZqzw*PI^+7Y8tkEKVI6t%>wtAeG4 zTCix8Zc4^?4?p)L$W2sFtScVVH8$(`Zb7F4Jre}_VFW?ealM0}AS=A9KSk~Be{Pk! z+dfRsWEEtmN=tVv-mYh}f`#kbIvoql(`|eBC$o6^Yxwx=VCnyD%el#kjg3KWyeTm@ zD5=Y98J~>jESwR<YbKYsjp@30&*Gl3qUMH`l|PmCAGKuitg2;Ou9&uPMl44QROoB2 zzE;i*Bb*c7sSHQW32$Ph;cZ*dqQ%p*j?gpZ9ZQ$D^;)zzvs~)oqVUO?;lknLOJ`hE zn0h?iNcqwkB^$QXBpY(t2B%)lb0Z%AAUXW7hSPd~+R%4-yrC^`@m~4{W@lxEH~R3G z{6u3}OX^M4&8-bNiQ3FZ)ui^E@H1q>Ux3P3**|_v9lL~nNTs9FKc4iLqVQ|@!7|ld zrwj`}WoLA4jW+T3N9>e`Z|M%-z^y0J^HaZI*;zwVtIn%U=pEnMv2ycbIn77qhZ(O; z){Y%iGN7e)Qd8c{Fs8N@EuJ$q)=9tW^BX58s$=t-TT8<`sg0!sac$wRw~Pn>0Sn~0 A00000 diff --git a/tests/data/acpi/virt/DSDT.memhp b/tests/data/acpi/virt/DSDT.memhp index XXXXXXX..XXXXXXX 100644 GIT binary patch delta 173 zcmcaUi}8ywmrJlq$QMZl2ByY|T++<_a~LOTC^K43^tIeLL4lLWeZ}O>oO+X=b6T<Z z6mvCfR_C%{pDgc^#>hCi&BajKi^V<I(}*M9!_$Q~z%RhS*}#o~BR<sAg^OwOMVP!X pHhJdBGOmtnjvVpMLBX3Jy81D0wsHT%Dk&Kd9^{0q-Wg&Z0|3#tFC_o~ literal 19799 zcmc)ScX$+4-^TG-5+Q6T0U{ux#NIU_h}cOs!9=qVlK^VeU_n7F5wTzeR8;K7UZcj| zd+)vX-h1!8=DE)nxW78@^WSs5T$h`<XU_S}e0EPJGv{)rrn#nNayp87jHsTFs%tK* z-l8#8qjiZWio$aESu*d1!mZnytJ_-V4NH}mmlw6w)z|c`N;TFitP>TrO{}kpTIbak zrY5BG8=KN~<>eI>xs63_six)u!;(Yh_l`ov-cd;u9n~{RB$iQ{tyWbvO?|?K)_E1< z8k%!e8pbzGP?fb&Wk9lDu8P`6g|oHi(4``KRP2(-?s!p`!hDx8<0hxZWxH%%o1Q4h zNbRM$r7BshKB=mI_UzGnsJe!oRTWNZ%D)HMy_MSmF6_Aon~Zwou;pF?2b?bvcKfdq zJ)%V=Dsm;N!%>WMbG}5fM_i3Ut1;4RRL0gQh^x`lYE*iSQL!}&<7yOHjZt1>?bsTN z#nnh!jkUeT=-3)P<7)J=8l%0&I<Yl+#ntF%HP-POV`6I*#nmXW8e_c1y0JBS$JH2U zHP*En75Ft;I6jhbHA=0<dR`-TnDUn-Khy_XjrF}o{5b6scN{~l#s*#^ew_A=t1--K zZ0I%O$7#Q~8pEx|MqVR+oc52av9#6L*lWa((~`IvWmaRX*N7jd1LA5dYc;C8M*KJ( z7*}I?t5NMW;>YQrxEd>3jg-|G89z=-<7%vIHEO&@{5V}auEwfXqt<K0kJG_%HCDG8 z<Ge=vI2{sKV@<2EiPwl9r$ggvlv|BDuMs~^mx!w|(rVOujregoEUv~VtFfuqh##j* z#?=^YH8%4a@#A!OT#YeSqrq#$kJF{%YOH59bZM0Ns~`T}R>qIhrQ>RBU^T{jjrego zBCf_pR%1f0#!+t-G^Xlv2hupcvAn8$j4oVKx**G@%5WJ|yNO<_tH<kA7gl=J6<XiW zFKldS8ZaU`yfI2e_0iO*EGpN3HCoyynjY<-pOG46FG3odS_U0=UO{nGIIy^|xVt;r zq**1h%Ly%4L<9ST^~oLzBlDp^p)wlSx3EulcOnXX)Gn7oFE#9!-InR6rui6ps(z-e zu9>oJb1C%9H`N7E*rS?edMbvV`MnfmdghOyAPPP6O$)L;)il#nG4#|CW%i=0!))7J z${90Fbpe%A=A%0ogLARWKJ(7SvOV($ujtO6aO#p+N04u-3odsmy0aIYz2NLcr=mNH z;4I4Y%)ceNKy#;}JA1>ai}~!kmw8Ki)2ZmrB%Db&bwTG&MRyj%Sqx_}or><%9ff_K zec;pupgR@a*%!{faQ3BB(VhL^><6bV7~QGp&i-)rhqFJOitg0SlsQY_)CHzH72P=i z&H-=^pi|ME1K}J9r!Gj{sp!r@a1MfV5S@zdEQPZaPF=vdQ_-D^!?`$|i_@v-&cSdF zhEo^3?o@Q=5IBdxIfPC{cj{KtKF^_W>H^uFitbzj&L!Ynf=)$u4uf+ToVuWPr=mNT zgmXzam!wnCox|ZA4yP`_-Kpr#rQlo&&ZX#7bm!7=E)AzH*xjk<&Jl2qfO7<$ita3f zvkXpM;JZ`Noy)+v44li*sp!sS;anC@-4M7_(Vfe|xg4C!(W&Up<>6c&PTfGbQ_-C( zz_|jPE6}Ov&K2QY5l-E3xKq)cE5W%EoGa0(=+2elTp3Q?ptw`fovXmP3Y@FZsp!sC z;an9?-O#vG(VeToxf-0S(W&Up)!|$nPTc^xQ_-Dkz_|vTYtX6a&Nbm&6HeVQxl_@d zYr(k|oNLjk=uXYc+4;5{PTgR+Q_-E8j#JP4m1hN<6?7`PQ<HG&nZNQJ38!wz+^OhJ zO}(jS{`$5O&PqBJ-KhyTbB=;jH*oG$bf>1-%(*t4YtyOdPED?vb2OZ~;d7^=J2j<d z&UN5ihfYOzY9h^?W8l;cqB|Acsp&Iwt_$b7bSk=2lV;{z4^G`sx>M1enkqBr`f#pK zr=mMGL1xYk;M5JMI~CojX)$wd2<L`$D!NmXVdmTjPTjD&Q_-E80yF2vaBfVeqC3aJ zITp^bbSk>D3eGAxtLRj8rzW|~Sq*13or><%)Rs9@aHi-~bf+e?)H8nvTmxqfor><% zG?sei&qK9v*3zlyPEB5!a~z!G=u~v4rmW1l37nhIspw8kRGG65&N@03-Kps*bJoLI zPp6_gH7RAzP2t>>PDOWWD$1Oj!MPcoitf||lsOyVY@k!ootkzsXBy5lor><%WRp3^ z!#SQ#MR#h7$($45oIt0dI~(C_gtL)OMR#rv=jL#3PN$+fo8WAMvx!bccQ(V>3}-W) zitcQIvjxr;Iu+eH5zdKlPNY-Oos;041m`3=72P=*&dG32rc=?KTfn&moLkVT=*}(S z+!D?$=~Q&*R&Z_w=T>woy0aC|RybSfRCMRoaBdCf)^sYma~n9ffpZ%=72UZloZG^= zEuD()Y=g56&Ney~-8lu$DR54qQ_-E<!MPor+tI1$&h6pc9?tFQRCMPKaP9!-4s<HI zb4NILgmXta72T;>Z#&;kg>x#MitgMA&Yj@giB3g#?hNP7aPCZ}qC2O-IStNfbSk=Y zI-JwtoKB~rJ9mL|7dUsJQ_-C>;G6;H3_2CvxhtHz!nrG*itgME&fVbLjZQ^(?hfbf zaPCg0qC5A1a}PN8pi|ME?Qpik*-odTJNJZhPdN9aQ_-C>;hYKQOga_axfh&!!MPWm zitd~R=PWp9(W&Upz2V#&&b{eWbmu;B?gQsObSk=YUpV)Lb6+|Y-MJr}`@y*%or><9 z4d-k)XVa<Z&i&!sAI|;hRCMP7a2^2X0dy+5^FTNcg!4c;72SCdoCm>q5S@zdJQ&V{ z;XIg5MRy(o=OJ(&LZ_lT=fF7!&N*}{x^pg^bK#szr=mOO!8s4kd2}ke^H4Ysh4WB4 z72SCloQJ`A7@dmlJRHu$;XIs9MR(4Jb3UB&=~Q&*5pW&>=Mi)&y7NdlkA(9`Iu+e{ z6r4xFc@&+B?py%p0yr1Ysp!t5;XE47qv=$1=P_^|1LrYxD!TJnIFE(%SUMHmc^sU_ z!Fe2=itaof&g0=co=!z~o&e_waGpS?qB~E7^F%mLq*KwIC&76VoF~z#=*|u}JK*e~ zQ_-C#!+A2CC)26u&Qst#1<q6GRCMR5aGnb1sdOs3^E5b5gYz^x72SC{oTtNiI-QE{ zJOj=%;5>s)MR%SF=b3PxNvEPa&w}$TIM1R}(Vb_*c{ZG9)2ZmrbKpD&&U5HgbmzHn zo(t!>bSk>@JUGvT^E^5g-Ps9eC!C#hD!TK0IM0Xkd^#1~c>$akz<B|kitfA+&I{qZ zkWNK+cEQ;NXBVA{?z{-ji{QM7PDOWK4ClpgUQDN=J1>Fr5;!lRQ_-E5!g(p2m(r=| z&dcDu49?5wRCMR%a9$4Q<#Z~#^9nexfb$AE72SCyoL9nmC7p`yyb8{%;Jk`XMR#5e z=hbjtO{bzeuYvO#IIp2o(Vf@Ac`cmR(y8dq>)^Z&&g<w@bm#SOUJvK>bSk>@1~_kk z^9DK<-FYLNH^O-%or><f3C^3~yopXlcis%=&2ZjKr=mM=f%6tPZ=qAsowveyE1b8| zsp!tz;Jgja+vrqu=k0Lb4(IK3D!TIyIPZY-4muUxc_*BA!g(j1itfA%&b#2ei%vy% z-VNv7aNbR)qC4+_^By?wp;OVF_riHEocGeH=+67#ybsR%=u~v){czq7=lygly7K`z zAAs`#Iu+gdAe;}v`5>K&?tBQ&hv0mOPDOV<4CljeK1`>gJ0F4b5jY>AQ_-D|!ucqi zkJ72=&d1<<49>^sRCMR#a6S&_<8&&z^9eYgfb$7D72WwHoKM2}B%O-xd<xE|;CzZs zMRz_8=hJXLO{bzepMmokIG>?Y(Vfr2`7E5z(y8dq=iqz}&gbY<bm#MMJ`d;fbSk>@ z1vp=T^94E;-T5M%FT(jEor>;!3C@?`e2Gp)cfJhg%W%F-r=mMwf%6qOU!hacov*_A zDx9y<sp!tv;Cv0v*XUGq=j(934(IE1D!TIxINyNt4LTLw`6iri!uckhitc<1&bQ!v zi%vy%z76NwaK25aqC4M#^Bp+fp;OVF@51>mobS@9=+5`xd=Jj|=u~v)`*6Mw=lgUj zy7L1#KY;TCIu+gdA)Ft=`5~Q(?)(VOkKp`>PDOWq4ClvieoUvLJ3oQ*6F5JiQ_-ED z!uctjpVF!5&d=cd49?H!RCMR(aDEQw=X5H%^9wk?fb$DF72WwIoL|EEC7p`y{0h#m z;QWeCMR$G;=htw4O{bzezk%}`IKQD&(VgGI`7NB^(y8dq@8J9n&hO|{bm#YQeh=sO zbSk>@2RMI#^9MQ=-T5P&Kf?JVor>=K3C^G3{E1FQcm53L&v5=sr=mN5f%6wQf1y*+ zoxj5QE1bX5sp!t%;QS5F-{@3y=kIX-4(IQ5D!TIzIRAk24>}dy`6rxz!ucniithXi z&cERNi%vy%{tf5faQ;oFqC30c?1r<OPQ|RVbzXg;{>K>mzG<p_T=x5<dTrE0J^Ce! zGY|4uF3LX0BRuGX>q>jJH8(XUa;0+Le+^$&{l7{rA5$v3j-_&6*Zyy%R){H;UB^<n zZ*Bj#QY*%k%C5_mDlKVCRaZq_{nW5ztX@hd^bgNHiHe%4CypCX*DE>e$i7jJKH3sR z`Y@s>am0`)>XQhI`d8B3{r5)M#qKq=CDErKo76hfyjxon(Sp^iPo}{fy>^Fx`R2Kw zVg2l=>;G-fMa>f%8>6CBOH)HsI<9xygyvM?f*Db&W^zSmU9XO50|q5aTGMY-{xV|t z*i$FZs=9Z>S9V%3BUz{hBlWXLKP%fq2zA0jWhiw(cu9^3ubm|)bxcnjq%9Sh))k$D zPwL3G%dRd78{0$Uu)b@?`Ter%!%ix?W|XecR@0m=>|7>$G|#T{*hkH4@1H(#$mi)L z9!RA-dw1-jH?Sa)2rqj0OL0?Ud0X~N)vfc=g-x~jN7ZCUPI!h)_ywp;mjCNx$_xp8 zNF&DPKAzl<lJGM;Sf;*c>ovnub~dT4(JmG}Vy7Z}r8|6qTN`rqv%g>kiB+;)=hQao z8{ZmOMZJ@St#gY*Ow~5mHk6f)YO9^p(z`u`DV>8m4w#aN5?ilT5cZR<YfPn^Q{y{J zv^v>CXMp*QlbfoOb6V$(Uo3klYEqLul;Eo>ADugr^wiz<m^r&_+8(pocinsEvYAu& z+GFpTJ51dxcX$h*x6>>C)SNkWjp?RvguQwvhqlfe+pw^HDz1J_eL7vX<J5hoXKJQv z#y2GkTIa61=)h}2y@GU8bwin6>h#hOmKwCwxJ6r8>)e(9(Y7iYn@ra>w<Xf`<C_!d zru3vlx^~mHMC;tLf3z$}hc_-gp>~p9np0(^gTiBhA`OLb98p))l`L*eq#IJ3MWt0e z!y9PZ+M_A0%Y|*--4|?d%9PNfeM%_UIjAGM=az9>PD!+94&(g6ouwVwLkVk>1zN2G zn>i&hrFErHq$^RnM!KoFC1GXT&zKfv3Kn-{JnhN;`PNfPtA?gE{CejPA>Xzr-86dC z!a=$4tdbjm(chVq$D-3mE_Tn37KDz;eme4o?BYNz)@2u0Y^cqzT~&*@wS&`DTjbgf z(_&qF_;^C6u+R_+X`!JmbO;L#p~%dzIxK{~A!Ig(-kqhVgmq#2%#ahAl>;&>6SEes z2}=!OTSI3}ua2-f61j3@c+NrQ6uXcdsDT}b8D8bcWK!kZWYS_k_025~)&aG(hdqbQ z?V)(s*dC5EY|4E?q1(d6(W6S2*Z4~({`mp4hf%rcV_I1QtEKQ?ji!e|*S<>_b=i`o z%W904_xM-C%+Sp?(OIZxx-mSDDx4w8_bU&Nc+k0{Pt~)1=9Fgt{&a;w5w<Ibq1+Y5 zR4(gimGzp*19XmVDF{aw;<V|zsE3Xq?5{ktCbv8N5zp-|JmKqqzB~P)&+RUpVE=c! zD_uFQU&J1r$&P8!{P3<$4~vPgSTVh`xMP}5ky;)(y>;G*aH?E%>PnTTbYu&kwGsUX DDbP3` diff --git a/tests/data/acpi/virt/DSDT.numamem b/tests/data/acpi/virt/DSDT.numamem index XXXXXXX..XXXXXXX 100644 GIT binary patch delta 156 zcmbO?fpNDcmrJlq$Zin^2BwP>xulufJQ*iyC^K43^tIeLL4lLWeZ}O>oO+X=b6T<Z z6mvCfR_C%{pDgc^#>hCi&BajKi^V<I(}*M9!_$Q~z%RhS*}#o~BR<sAg^OwOMVP!X lHhJdBGOmtnjvVpMLBX3Jy81CrwsAkqC^^YPgaxRb0RT`TDiQzy literal 18462 zcmc)ScXSkm8iw%+36N|;NFdS#0*VC-rifrC*(4Ap5OxEoL4yrNET~uz6^x349TdAp z#ol{Y6npQeh`smTHTRwDuD;K8uK!-nakJ0v%s2Z>CNML{-I`=g)4(x7&}nM*`1qLQ zpz7@!<28CLD+q${e)zR$!Q7lFEy?PZ=GK1kva+(=mNE4;-Kye^^@<TeZp*~_nxMJ0 zHYYy5A@gLSVN6+Bd3pND+?IGES==wydwyOJPRt96f?z?HAS-LIYPOcDs!0@tPc*ld z*Nsi4r;Ht!7_TYAF{L<Gn4Y5LgPhsga=1!)>Q!--tkj18UL_~9%E-FO@w(J16KWeK z3R0o1B%7*Y`C2Dl_1|lD%Il+5!;MwtOiE<F2dS-<*$ez@&A+j+pi>%K<|FWeGb6&y z{$oU^;O`OT=@Hf8tEg~uW<;!0)QlXPQQ<QxBWGks&FEq?Dt*Srku!3lX5`w8jeW-O z$QhlZX2fj9aG$YB<cuy+GYV|RCO%_C<czLSGYW0S2%j-Baz<{{j3S#c(r0WMIU_G> zMh}~@sm&<IuhC!oM=WYaiOtx|XGHF%{3Xfk>b-2n<~}2OKP`xQ9er%Z7Cs|-KkXJZ zqo2*#(q}~Pr-e~7`rC}Hd`9$s+C6H<K%23(&xqbni=t)>vKga%M)ZDK95rJNn^EO6 zqW9AtQ8R|vjB1||y`T1snz6Rch}(>c=>4=LYR0-YqsC`M@29Ip%~;Q7)cTC*{j^uq zj16o?ozIBgPkTqr7-lm@`;6%Qv`^HGGMk})#ykFn3jb}Wh~7{8M$M?O8TCFRdOz(K zHDhC&v5n7&-cMJLnz4z^*w$x6@2CBvW{k8MV|_;SemWp(#%4C7!DduO@23N!W^7?I z#`%os{j@Y{##T0Ce0s)$RoRX4`t%EF9M@P@RW?!wE^!@@rK&PKjHw;1+v@6Zy48V| zZgqs#EnF{rvMEtq8tdN}#Dn@^_h3*^rvGYm@8Dp1u&cfXS}1i8(wJ!KdGdwX`9V&P z{G9yu_F!~UBU1OXbiX|4Q4l^J>!hg2M7E+b=+P~wpuIgS2-nea=?d4<f`zH)I@Z&o znGDy|{ElUH%#>O-UE!LUMRh<FZc&SNtf>sw%xopQW6jJf$PU6aGmB=Y*3_aMYbwJv z^@=*SqNBsqvgt}2I~LUeR9cxycXo!ebH_F_&d#YdGcR80&Mt83kXWxEv#1WZ+^KYD zS2(-E*_BSEJ9FX8?N~GOEztp*JC*LtgHs3dsqbFLw<M2Fr8{GA#^BTeojaB8%!e}{ z&U`wR?$jBD{X7fc)B&J7mG0~YXE!*z(W!K2A)JM9>VVOmN_TdLvpbyK=~TK?CsXDu zf>Q^W?o_(77|voii|JIlvj?0#;M4)BJC*M231?3@d(x?NX9=7oaOwcool1AE2Ip#U zu12TQoxR}f1*Z;p-KlhEZ#a9y*_%$KJ9VmQKhHjJ>HyiDN_X~!voD-|=~TM2ADsQ* z)B&|SmF`>}&eh>uold1Y`@`8EP91=|Q|ZnDa1MZT0G&#A4uo?coH}54r_!CJaF)VZ zN~hADgWwzlrw;JlsdVRHI0wTym`<fT*MM^kICVncPNh57gmX<e*Q8VF&LMCPfm0_C z?o_&SEjZVLb1gcR?pzzrwc*qWhdY(-TnEl|;9Q4Jr90P!b6q%fg5pl4JBPwK6waY^ zD&4sroa@1<6B>6a-MK!T>%+M|ol19Z0Otm9>IBH0N_TDu=Z0`@NT<@B!{8hSr%ssM zsdVQ?aBc+WMszCOsq1p~_iY)RI>B<M(w({-r!_N2p5<_s)2Vc)F2ZTe%#mjWoH`+M zr_!Ce^rkg4$G4SmR??|-r!Kgeb7MGl0_RSpJ9U}OoWtQ9PN&kHy0~V}P2kiCpF5TA z)Fm}@j(~Fnol1A=LYg^8!l@HPcPibf%V*}?6wXcQRJv0a&CIzOoI0U&r_!CeRA$c2 z;oO`~r8{+j%$!@msS{9lD&48eV&>cu&MoOwx>Fa!%()euI$?FE(w({lX3nkQ+?r0M zJ4eAe3eHh<D&1KHXBC`PbSmAci(KZchO?SZr8{+L%bamI<8&(BsS8?KGjj%917{7L zN_Xlqme$PtJyZ*4EuBht>f)6->)@=TQ|V4!vNGprI7ic|bf+#<nR5)BW9U@6Q<taA zSr2DDol1A=qLewefpZ%=mG0D~D06NL=eBez-Kh&u<{S&>SUQ#N)MY1gHo)0Hr_!Ce z*ksOeaE_x>=}ui@GUs?W$J42FXCs`Aa5mDZbms&(C%`#@PNh2&a3<hP(5ZB163!%? zNjjD8Y=W~1&L%pQ?wkncL^vnXsdVQgI48k5iB6?EC&M`z&dGEt-MJl{+rhaVol19Z z59ju9ZcnGuojbs}1Dre1sdQ&EoXv1H)2Vdl6ga2AIfYK8J9mV0M>uz+Q|Zo~;M@t$ zo#<4$vjxr;I9upcx^pU=Q{kLSr_!A}!?`n@JJYFj=Pq#W0_QGtD&4s&oV&ugE1gPr z?gr;>aPCH@(w(~MZGYeH4(IN4D&08^&S`K?qf_b5J>c8}&OPW<x^quB_k?p#I+gC6 z4(D_@r_-r)=U#B`1?OILD&08)&KYpdpi}A2z2V#&&b{eWx^o{m_knXCI+gC+7tVd* z+?P(JJNJWgKREZJQ|Zo`aL$BtCY?%m?hohwaPCj1(wzsuc>tUT(5ZCifp8uO=Ye!8 z-8l=+S#ZvxQ|Zow;5-P<gXmPc^I$j+hVx)LmF_$Q&O_ilgifV94~6qkI1i;$>CV}3 z&W3X~ol18e2IpaL9!96qopa!v1Lqt%mF_$o&coq6oKB@XkAU+CIFF!H>CPkJJQB_$ z=~TLNE}V1WoJ*(Do%7(F2j@IGmF_$W&ZFQwicY0FTj6Ylvz1PzJCBC*XgH6iQ|ZoQ z;5-J-W9U@6^H?~Kh4WZCmF}Dm=X^Nl)2Vdlac~|7=W%o@-FZBm$HRF%ol18ufO7$y z3+Pn3b0M4y;ao_k(w!&3c><g#(5ZCiiEy3>=ZSPG-FXt6C&76Vol19}4Cl#ko=m6G zou|Ng3Y@3VsdVS5aGnb1sdOscc^aIj!Fd{;N_Q@Ta}k`2=v2CMF`SFxTui6Zou|Wj zI-IA|sdVQVaGn9@8FVV$xdhH7a4w-!>CQ9ZJQL0{=~TM&EI7}C^DH`*?mQdLv*A3O zPNh4~f%6<V&!JQ4&U4{B7tV9(RJ!v#IM0LgJUW%`JRi>U;XI#Cr8_Tx^8z?8pi}A2 z3*o#F&I{>Oy0Z<=HaOepRJ!vbI4^?pB081syco`l;k=kmr8_Ty^Ab2Op;PJ3rEo5V zb19ulcU}tTrEp$Kr_!C5!Fd^+m(i(o=jCu-4(H``D&2VnoL9hk1)WNFUJ2)wa9&BL z(w$eqc@>;j(W!Lj)o@-7=hbv7-FXe1*T8uVol19J3+J_PUQ4Iao!7y69h}$EsdVS{ za9$7R^>ixTc>|m`z<C3mN_XA}=Z$dQNT<@BH^F%moHx;_bmz@*-VEo>bSm9>3!JyW zc?+FNcisx;t#IB-r_!Cb!Fd~;x6!F|=k0Lb4(IK3D&2VpoOi%^2c1fH-U;WOaNbF$ z(w%p~c^8~_(W!Lj-EiIw=iPKF-FXk3_rQ4%ol1A!3+KIX-b<&_o%g|cADs8msdVT4 zaNZB+{d6kbxeU%_a4w@$>COk>d;rb|=v2D%K{y|T^Fcb5?pzM%ayXaMsdVQ<a6Sa* zLv$+L`7oRh!}&0sN_Rd2=Ob`FLZ{N5kHYyVoR89}bmwDmJ_hGwbSmBXIGm5e`8b_Q zcRm5<6L3C3r_!BI!uceePtvJ$=TmS#1?N+AD&6@soKM5~G@VL!J_F}7a6Ut)(w)!3 z`7E5z(y4Ukb8tQf=W}!_-T6G6&%^mVol1AU0Ot#EzCfqaoiD=qBAhSMsdVQ{aJ~fR zOLQvT`7)d@!}&6uN_V~j=PPi&LZ{N5E8tuK=L$NN?tB%_SK)k>PNh3vgYz{wU!zm$ z&e!359nRP3RJwB|oGam6NvG1CZ@~EmoNv&nbmuBKSHZc8PNh5Fg!4@}-=tIN&bQ!v z3(mLbRJ!wRINyfzZ90|id<V{V;CzQpr90n+^IbUKrBmt7_uzaF&iCk4y7PTF--q*k zI+gDH0L~BK{D4lSJ3oZ;LpVRAQ|Zo+;QR>AkLXmo^J6$ahVx@OmG1lm&QIX{gifV9 zKZWyCI6tLR>CVsK{0z>|=v2D%b2vYT^K&|t?)(DIFW~%wPNh4)g!4-{zob*?&adG7 z3eK<SRJ!wPIKPJTYdV$g{07c%;QWS8r8~cc^IJH-rBmt7@8J9n&hO|{y7PNDzlZaC zI+gDH0nQ)b{DDrTJAZ`pM>v0^Q|Zp1;QR^BpXgM&^Jh4JhVy4SmG1ln&R^jCg-)eA ze}(f`IDe&6>CWHa{0+|E=v2D%cQ}8C^LILx?)(GJKj8d>PNh5lg!4~0|D;pt&UQH4 z;cTZ<nQ}I_*5~MdjIsBd#>?tb?<du5qdwH5FqYr(K^|)csSol9Kj?#xm2_!ICX!j{ zQR(-;hHqB=U!#UZj7mMmQR%m9|J$gwB1WYi<EZqzw*PI^+7Y8tkEKVI6t%>wtAeG4 zTCix8Zc4^?4?p)L$W2sFtScVVH8$(`Zb7F4Jre}_VFW?ealM0}AS=A9KSk~Be{Pk! z+dfRsWEEtmN=tVv-mYh}f`#kbIvoql(`|eBC$o6^Yxwx=VCnyD%el#kjg3KWyeTm@ zD5=Y98J~>jESwR<YbKYsjp@30&*Gl3qUMH`l|PmCAGKuitg2;Ou9&uPMl44QROoB2 zzE;i*Bb*c7sSHQW32$Ph;cZ*dqQ%p*j?gpZ9ZQ$D^;)zzvs~)oqVUO?;lknLOJ`hE zn0h?iNcqwkB^$QXBpY(t2B%)lb0Z%AAUXW7hSPd~+R%4-yrC^`@m~4{W@lxEH~R3G z{6u3}OX^M4&8-bNiQ3FZ)ui^E@H1q>Ux3P3**|_v9lL~nNTs9FKc4iLqVQ|@!7|ld zrwj`}WoLA4jW+T3N9>e`Z|M%-z^y0J^HaZI*;zwVtIn%U=pEnMv2ycbIn77qhZ(O; z){Y%iGN7e)Qd8c{Fs8N@EuJ$q)=9tW^BX58s$=t-TT8<`sg0!sac$wRw~Pn>0Sn~0 A00000 -- 2.20.1 From: Richard Henderson <richard.henderson@linaro.org> Use a common predicate for querying stage1-ness. Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200208125816.14954-2-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/internals.h | 18 ++++++++++++++++++ target/arm/helper.c | 8 +++----- 2 files changed, 21 insertions(+), 5 deletions(-) diff --git a/target/arm/internals.h b/target/arm/internals.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/internals.h +++ b/target/arm/internals.h @@ -XXX,XX +XXX,XX @@ static inline ARMMMUIdx arm_stage1_mmu_idx(CPUARMState *env) ARMMMUIdx arm_stage1_mmu_idx(CPUARMState *env); #endif +/** + * arm_mmu_idx_is_stage1_of_2: + * @mmu_idx: The ARMMMUIdx to test + * + * Return true if @mmu_idx is a NOTLB mmu_idx that is the + * first stage of a two stage regime. + */ +static inline bool arm_mmu_idx_is_stage1_of_2(ARMMMUIdx mmu_idx) +{ + switch (mmu_idx) { + case ARMMMUIdx_Stage1_E0: + case ARMMMUIdx_Stage1_E1: + return true; + default: + return false; + } +} + /* * Parameters of a given virtual address, as extracted from the * translation control register (TCR) for a given regime. diff --git a/target/arm/helper.c b/target/arm/helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -XXX,XX +XXX,XX @@ static uint64_t do_ats_write(CPUARMState *env, uint64_t value, bool take_exc = false; if (fi.s1ptw && current_el == 1 && !arm_is_secure(env) - && (mmu_idx == ARMMMUIdx_Stage1_E1 || - mmu_idx == ARMMMUIdx_Stage1_E0)) { + && arm_mmu_idx_is_stage1_of_2(mmu_idx)) { /* * Synchronous stage 2 fault on an access made as part of the * translation table walk for AT S1E0* or AT S1E1* insn @@ -XXX,XX +XXX,XX @@ static inline bool regime_translation_disabled(CPUARMState *env, } } - if ((env->cp15.hcr_el2 & HCR_DC) && - (mmu_idx == ARMMMUIdx_Stage1_E0 || mmu_idx == ARMMMUIdx_Stage1_E1)) { + if ((env->cp15.hcr_el2 & HCR_DC) && arm_mmu_idx_is_stage1_of_2(mmu_idx)) { /* HCR.DC means SCTLR_EL1.M behaves as 0 */ return true; } @@ -XXX,XX +XXX,XX @@ static hwaddr S1_ptw_translate(CPUARMState *env, ARMMMUIdx mmu_idx, hwaddr addr, MemTxAttrs txattrs, ARMMMUFaultInfo *fi) { - if ((mmu_idx == ARMMMUIdx_Stage1_E0 || mmu_idx == ARMMMUIdx_Stage1_E1) && + if (arm_mmu_idx_is_stage1_of_2(mmu_idx) && !regime_translation_disabled(env, ARMMMUIdx_Stage2)) { target_ulong s2size; hwaddr s2pa; -- 2.20.1 From: Richard Henderson <richard.henderson@linaro.org> To implement PAN, we will want to swap, for short periods of time, to a different privileged mmu_idx. In addition, we cannot do this with flushing alone, because the AT* instructions have both PAN and PAN-less versions. Add the ARMMMUIdx*_PAN constants where necessary next to the corresponding ARMMMUIdx* constant. Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200208125816.14954-3-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/cpu-param.h | 2 +- target/arm/cpu.h | 33 ++++++++++++++------- target/arm/internals.h | 9 ++++++ target/arm/helper.c | 60 +++++++++++++++++++++++++++++++------- target/arm/translate-a64.c | 3 ++ target/arm/translate.c | 2 ++ 6 files changed, 87 insertions(+), 22 deletions(-) diff --git a/target/arm/cpu-param.h b/target/arm/cpu-param.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpu-param.h +++ b/target/arm/cpu-param.h @@ -XXX,XX +XXX,XX @@ # define TARGET_PAGE_BITS_MIN 10 #endif -#define NB_MMU_MODES 9 +#define NB_MMU_MODES 12 #endif diff --git a/target/arm/cpu.h b/target/arm/cpu.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -XXX,XX +XXX,XX @@ bool write_cpustate_to_list(ARMCPU *cpu, bool kvm_sync); * 5. we want to be able to use the TLB for accesses done as part of a * stage1 page table walk, rather than having to walk the stage2 page * table over and over. + * 6. we need separate EL1/EL2 mmu_idx for handling the Privileged Access + * Never (PAN) bit within PSTATE. * * This gives us the following list of cases: * * NS EL0 EL1&0 stage 1+2 (aka NS PL0) * NS EL1 EL1&0 stage 1+2 (aka NS PL1) + * NS EL1 EL1&0 stage 1+2 +PAN * NS EL0 EL2&0 - * NS EL2 EL2&0 + * NS EL2 EL2&0 +PAN * NS EL2 (aka NS PL2) * S EL0 EL1&0 (aka S PL0) * S EL1 EL1&0 (not used if EL3 is 32 bit) + * S EL1 EL1&0 +PAN * S EL3 (aka S PL1) * NS EL1&0 stage 2 * - * for a total of 9 different mmu_idx. + * for a total of 12 different mmu_idx. * * R profile CPUs have an MPU, but can use the same set of MMU indexes * as A profile. They only need to distinguish NS EL0 and NS EL1 (and @@ -XXX,XX +XXX,XX @@ typedef enum ARMMMUIdx { /* * A-profile. */ - ARMMMUIdx_E10_0 = 0 | ARM_MMU_IDX_A, - ARMMMUIdx_E20_0 = 1 | ARM_MMU_IDX_A, + ARMMMUIdx_E10_0 = 0 | ARM_MMU_IDX_A, + ARMMMUIdx_E20_0 = 1 | ARM_MMU_IDX_A, - ARMMMUIdx_E10_1 = 2 | ARM_MMU_IDX_A, + ARMMMUIdx_E10_1 = 2 | ARM_MMU_IDX_A, + ARMMMUIdx_E10_1_PAN = 3 | ARM_MMU_IDX_A, - ARMMMUIdx_E2 = 3 | ARM_MMU_IDX_A, - ARMMMUIdx_E20_2 = 4 | ARM_MMU_IDX_A, + ARMMMUIdx_E2 = 4 | ARM_MMU_IDX_A, + ARMMMUIdx_E20_2 = 5 | ARM_MMU_IDX_A, + ARMMMUIdx_E20_2_PAN = 6 | ARM_MMU_IDX_A, - ARMMMUIdx_SE10_0 = 5 | ARM_MMU_IDX_A, - ARMMMUIdx_SE10_1 = 6 | ARM_MMU_IDX_A, - ARMMMUIdx_SE3 = 7 | ARM_MMU_IDX_A, + ARMMMUIdx_SE10_0 = 7 | ARM_MMU_IDX_A, + ARMMMUIdx_SE10_1 = 8 | ARM_MMU_IDX_A, + ARMMMUIdx_SE10_1_PAN = 9 | ARM_MMU_IDX_A, + ARMMMUIdx_SE3 = 10 | ARM_MMU_IDX_A, - ARMMMUIdx_Stage2 = 8 | ARM_MMU_IDX_A, + ARMMMUIdx_Stage2 = 11 | ARM_MMU_IDX_A, /* * These are not allocated TLBs and are used only for AT system @@ -XXX,XX +XXX,XX @@ typedef enum ARMMMUIdx { */ ARMMMUIdx_Stage1_E0 = 0 | ARM_MMU_IDX_NOTLB, ARMMMUIdx_Stage1_E1 = 1 | ARM_MMU_IDX_NOTLB, + ARMMMUIdx_Stage1_E1_PAN = 2 | ARM_MMU_IDX_NOTLB, /* * M-profile. @@ -XXX,XX +XXX,XX @@ typedef enum ARMMMUIdxBit { TO_CORE_BIT(E10_0), TO_CORE_BIT(E20_0), TO_CORE_BIT(E10_1), + TO_CORE_BIT(E10_1_PAN), TO_CORE_BIT(E2), TO_CORE_BIT(E20_2), + TO_CORE_BIT(E20_2_PAN), TO_CORE_BIT(SE10_0), TO_CORE_BIT(SE10_1), + TO_CORE_BIT(SE10_1_PAN), TO_CORE_BIT(SE3), TO_CORE_BIT(Stage2), diff --git a/target/arm/internals.h b/target/arm/internals.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/internals.h +++ b/target/arm/internals.h @@ -XXX,XX +XXX,XX @@ static inline bool regime_has_2_ranges(ARMMMUIdx mmu_idx) switch (mmu_idx) { case ARMMMUIdx_Stage1_E0: case ARMMMUIdx_Stage1_E1: + case ARMMMUIdx_Stage1_E1_PAN: case ARMMMUIdx_E10_0: case ARMMMUIdx_E10_1: + case ARMMMUIdx_E10_1_PAN: case ARMMMUIdx_E20_0: case ARMMMUIdx_E20_2: + case ARMMMUIdx_E20_2_PAN: case ARMMMUIdx_SE10_0: case ARMMMUIdx_SE10_1: + case ARMMMUIdx_SE10_1_PAN: return true; default: return false; @@ -XXX,XX +XXX,XX @@ static inline bool regime_is_secure(CPUARMState *env, ARMMMUIdx mmu_idx) switch (mmu_idx) { case ARMMMUIdx_E10_0: case ARMMMUIdx_E10_1: + case ARMMMUIdx_E10_1_PAN: case ARMMMUIdx_E20_0: case ARMMMUIdx_E20_2: + case ARMMMUIdx_E20_2_PAN: case ARMMMUIdx_Stage1_E0: case ARMMMUIdx_Stage1_E1: + case ARMMMUIdx_Stage1_E1_PAN: case ARMMMUIdx_E2: case ARMMMUIdx_Stage2: case ARMMMUIdx_MPrivNegPri: @@ -XXX,XX +XXX,XX @@ static inline bool regime_is_secure(CPUARMState *env, ARMMMUIdx mmu_idx) case ARMMMUIdx_SE3: case ARMMMUIdx_SE10_0: case ARMMMUIdx_SE10_1: + case ARMMMUIdx_SE10_1_PAN: case ARMMMUIdx_MSPrivNegPri: case ARMMMUIdx_MSUserNegPri: case ARMMMUIdx_MSPriv: @@ -XXX,XX +XXX,XX @@ static inline bool arm_mmu_idx_is_stage1_of_2(ARMMMUIdx mmu_idx) switch (mmu_idx) { case ARMMMUIdx_Stage1_E0: case ARMMMUIdx_Stage1_E1: + case ARMMMUIdx_Stage1_E1_PAN: return true; default: return false; diff --git a/target/arm/helper.c b/target/arm/helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -XXX,XX +XXX,XX @@ static void tlbiall_nsnh_write(CPUARMState *env, const ARMCPRegInfo *ri, tlb_flush_by_mmuidx(cs, ARMMMUIdxBit_E10_1 | + ARMMMUIdxBit_E10_1_PAN | ARMMMUIdxBit_E10_0 | ARMMMUIdxBit_Stage2); } @@ -XXX,XX +XXX,XX @@ static void tlbiall_nsnh_is_write(CPUARMState *env, const ARMCPRegInfo *ri, tlb_flush_by_mmuidx_all_cpus_synced(cs, ARMMMUIdxBit_E10_1 | + ARMMMUIdxBit_E10_1_PAN | ARMMMUIdxBit_E10_0 | ARMMMUIdxBit_Stage2); } @@ -XXX,XX +XXX,XX @@ static int gt_phys_redir_timeridx(CPUARMState *env) switch (arm_mmu_idx(env)) { case ARMMMUIdx_E20_0: case ARMMMUIdx_E20_2: + case ARMMMUIdx_E20_2_PAN: return GTIMER_HYP; default: return GTIMER_PHYS; @@ -XXX,XX +XXX,XX @@ static int gt_virt_redir_timeridx(CPUARMState *env) switch (arm_mmu_idx(env)) { case ARMMMUIdx_E20_0: case ARMMMUIdx_E20_2: + case ARMMMUIdx_E20_2_PAN: return GTIMER_HYPVIRT; default: return GTIMER_VIRT; @@ -XXX,XX +XXX,XX @@ static uint64_t do_ats_write(CPUARMState *env, uint64_t value, format64 = arm_s1_regime_using_lpae_format(env, mmu_idx); if (arm_feature(env, ARM_FEATURE_EL2)) { - if (mmu_idx == ARMMMUIdx_E10_0 || mmu_idx == ARMMMUIdx_E10_1) { + if (mmu_idx == ARMMMUIdx_E10_0 || + mmu_idx == ARMMMUIdx_E10_1 || + mmu_idx == ARMMMUIdx_E10_1_PAN) { format64 |= env->cp15.hcr_el2 & (HCR_VM | HCR_DC); } else { format64 |= arm_current_el(env) == 2; @@ -XXX,XX +XXX,XX @@ static void vmsa_tcr_ttbr_el2_write(CPUARMState *env, const ARMCPRegInfo *ri, if (extract64(raw_read(env, ri) ^ value, 48, 16) && (arm_hcr_el2_eff(env) & HCR_E2H)) { tlb_flush_by_mmuidx(env_cpu(env), - ARMMMUIdxBit_E20_2 | ARMMMUIdxBit_E20_0); + ARMMMUIdxBit_E20_2 | + ARMMMUIdxBit_E20_2_PAN | + ARMMMUIdxBit_E20_0); } raw_write(env, ri, value); } @@ -XXX,XX +XXX,XX @@ static void vttbr_write(CPUARMState *env, const ARMCPRegInfo *ri, if (raw_read(env, ri) != value) { tlb_flush_by_mmuidx(cs, ARMMMUIdxBit_E10_1 | + ARMMMUIdxBit_E10_1_PAN | ARMMMUIdxBit_E10_0 | ARMMMUIdxBit_Stage2); raw_write(env, ri, value); @@ -XXX,XX +XXX,XX @@ static int vae1_tlbmask(CPUARMState *env) { /* Since we exclude secure first, we may read HCR_EL2 directly. */ if (arm_is_secure_below_el3(env)) { - return ARMMMUIdxBit_SE10_1 | ARMMMUIdxBit_SE10_0; + return ARMMMUIdxBit_SE10_1 | + ARMMMUIdxBit_SE10_1_PAN | + ARMMMUIdxBit_SE10_0; } else if ((env->cp15.hcr_el2 & (HCR_E2H | HCR_TGE)) == (HCR_E2H | HCR_TGE)) { - return ARMMMUIdxBit_E20_2 | ARMMMUIdxBit_E20_0; + return ARMMMUIdxBit_E20_2 | + ARMMMUIdxBit_E20_2_PAN | + ARMMMUIdxBit_E20_0; } else { - return ARMMMUIdxBit_E10_1 | ARMMMUIdxBit_E10_0; + return ARMMMUIdxBit_E10_1 | + ARMMMUIdxBit_E10_1_PAN | + ARMMMUIdxBit_E10_0; } } @@ -XXX,XX +XXX,XX @@ static int alle1_tlbmask(CPUARMState *env) * stage 1 translations. */ if (arm_is_secure_below_el3(env)) { - return ARMMMUIdxBit_SE10_1 | ARMMMUIdxBit_SE10_0; + return ARMMMUIdxBit_SE10_1 | + ARMMMUIdxBit_SE10_1_PAN | + ARMMMUIdxBit_SE10_0; } else if (arm_feature(env, ARM_FEATURE_EL2)) { - return ARMMMUIdxBit_E10_1 | ARMMMUIdxBit_E10_0 | ARMMMUIdxBit_Stage2; + return ARMMMUIdxBit_E10_1 | + ARMMMUIdxBit_E10_1_PAN | + ARMMMUIdxBit_E10_0 | + ARMMMUIdxBit_Stage2; } else { - return ARMMMUIdxBit_E10_1 | ARMMMUIdxBit_E10_0; + return ARMMMUIdxBit_E10_1 | + ARMMMUIdxBit_E10_1_PAN | + ARMMMUIdxBit_E10_0; } } static int e2_tlbmask(CPUARMState *env) { /* TODO: ARMv8.4-SecEL2 */ - return ARMMMUIdxBit_E20_0 | ARMMMUIdxBit_E20_2 | ARMMMUIdxBit_E2; + return ARMMMUIdxBit_E20_0 | + ARMMMUIdxBit_E20_2 | + ARMMMUIdxBit_E20_2_PAN | + ARMMMUIdxBit_E2; } static void tlbi_aa64_alle1_write(CPUARMState *env, const ARMCPRegInfo *ri, @@ -XXX,XX +XXX,XX @@ static uint32_t regime_el(CPUARMState *env, ARMMMUIdx mmu_idx) switch (mmu_idx) { case ARMMMUIdx_E20_0: case ARMMMUIdx_E20_2: + case ARMMMUIdx_E20_2_PAN: case ARMMMUIdx_Stage2: case ARMMMUIdx_E2: return 2; @@ -XXX,XX +XXX,XX @@ static uint32_t regime_el(CPUARMState *env, ARMMMUIdx mmu_idx) case ARMMMUIdx_SE10_0: return arm_el_is_aa64(env, 3) ? 1 : 3; case ARMMMUIdx_SE10_1: + case ARMMMUIdx_SE10_1_PAN: case ARMMMUIdx_Stage1_E0: case ARMMMUIdx_Stage1_E1: + case ARMMMUIdx_Stage1_E1_PAN: case ARMMMUIdx_E10_0: case ARMMMUIdx_E10_1: + case ARMMMUIdx_E10_1_PAN: case ARMMMUIdx_MPrivNegPri: case ARMMMUIdx_MUserNegPri: case ARMMMUIdx_MPriv: @@ -XXX,XX +XXX,XX @@ static inline ARMMMUIdx stage_1_mmu_idx(ARMMMUIdx mmu_idx) return ARMMMUIdx_Stage1_E0; case ARMMMUIdx_E10_1: return ARMMMUIdx_Stage1_E1; + case ARMMMUIdx_E10_1_PAN: + return ARMMMUIdx_Stage1_E1_PAN; default: return mmu_idx; } @@ -XXX,XX +XXX,XX @@ static inline bool regime_is_user(CPUARMState *env, ARMMMUIdx mmu_idx) return false; case ARMMMUIdx_E10_0: case ARMMMUIdx_E10_1: + case ARMMMUIdx_E10_1_PAN: g_assert_not_reached(); } } @@ -XXX,XX +XXX,XX @@ bool get_phys_addr(CPUARMState *env, target_ulong address, target_ulong *page_size, ARMMMUFaultInfo *fi, ARMCacheAttrs *cacheattrs) { - if (mmu_idx == ARMMMUIdx_E10_0 || mmu_idx == ARMMMUIdx_E10_1) { + if (mmu_idx == ARMMMUIdx_E10_0 || + mmu_idx == ARMMMUIdx_E10_1 || + mmu_idx == ARMMMUIdx_E10_1_PAN) { /* Call ourselves recursively to do the stage 1 and then stage 2 * translations. */ @@ -XXX,XX +XXX,XX @@ int arm_mmu_idx_to_el(ARMMMUIdx mmu_idx) case ARMMMUIdx_SE10_0: return 0; case ARMMMUIdx_E10_1: + case ARMMMUIdx_E10_1_PAN: case ARMMMUIdx_SE10_1: + case ARMMMUIdx_SE10_1_PAN: return 1; case ARMMMUIdx_E2: case ARMMMUIdx_E20_2: + case ARMMMUIdx_E20_2_PAN: return 2; case ARMMMUIdx_SE3: return 3; @@ -XXX,XX +XXX,XX @@ static uint32_t rebuild_hflags_a64(CPUARMState *env, int el, int fp_el, /* TODO: ARMv8.2-UAO */ switch (mmu_idx) { case ARMMMUIdx_E10_1: + case ARMMMUIdx_E10_1_PAN: case ARMMMUIdx_SE10_1: + case ARMMMUIdx_SE10_1_PAN: /* TODO: ARMv8.3-NV */ flags = FIELD_DP32(flags, TBFLAG_A64, UNPRIV, 1); break; case ARMMMUIdx_E20_2: + case ARMMMUIdx_E20_2_PAN: /* TODO: ARMv8.4-SecEL2 */ /* * Note that E20_2 is gated by HCR_EL2.E2H == 1, but E20_0 is diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-a64.c +++ b/target/arm/translate-a64.c @@ -XXX,XX +XXX,XX @@ static int get_a64_user_mem_index(DisasContext *s) */ switch (useridx) { case ARMMMUIdx_E10_1: + case ARMMMUIdx_E10_1_PAN: useridx = ARMMMUIdx_E10_0; break; case ARMMMUIdx_E20_2: + case ARMMMUIdx_E20_2_PAN: useridx = ARMMMUIdx_E20_0; break; case ARMMMUIdx_SE10_1: + case ARMMMUIdx_SE10_1_PAN: useridx = ARMMMUIdx_SE10_0; break; default: diff --git a/target/arm/translate.c b/target/arm/translate.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate.c +++ b/target/arm/translate.c @@ -XXX,XX +XXX,XX @@ static inline int get_a32_user_mem_index(DisasContext *s) case ARMMMUIdx_E2: /* this one is UNPREDICTABLE */ case ARMMMUIdx_E10_0: case ARMMMUIdx_E10_1: + case ARMMMUIdx_E10_1_PAN: return arm_to_core_mmu_idx(ARMMMUIdx_E10_0); case ARMMMUIdx_SE3: case ARMMMUIdx_SE10_0: case ARMMMUIdx_SE10_1: + case ARMMMUIdx_SE10_1_PAN: return arm_to_core_mmu_idx(ARMMMUIdx_SE10_0); case ARMMMUIdx_MUser: case ARMMMUIdx_MPriv: -- 2.20.1 From: Richard Henderson <richard.henderson@linaro.org> Include definitions for all of the bits in ID_MMFR3. We already have a definition for ID_AA64MMFR1.PAN. Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200208125816.14954-4-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/cpu.h | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+) diff --git a/target/arm/cpu.h b/target/arm/cpu.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -XXX,XX +XXX,XX @@ FIELD(ID_ISAR6, FHM, 8, 4) FIELD(ID_ISAR6, SB, 12, 4) FIELD(ID_ISAR6, SPECRES, 16, 4) +FIELD(ID_MMFR3, CMAINTVA, 0, 4) +FIELD(ID_MMFR3, CMAINTSW, 4, 4) +FIELD(ID_MMFR3, BPMAINT, 8, 4) +FIELD(ID_MMFR3, MAINTBCST, 12, 4) +FIELD(ID_MMFR3, PAN, 16, 4) +FIELD(ID_MMFR3, COHWALK, 20, 4) +FIELD(ID_MMFR3, CMEMSZ, 24, 4) +FIELD(ID_MMFR3, SUPERSEC, 28, 4) + FIELD(ID_MMFR4, SPECSEI, 0, 4) FIELD(ID_MMFR4, AC2, 4, 4) FIELD(ID_MMFR4, XNX, 8, 4) @@ -XXX,XX +XXX,XX @@ static inline bool isar_feature_aa32_vminmaxnm(const ARMISARegisters *id) return FIELD_EX64(id->mvfr2, MVFR2, FPMISC) >= 4; } +static inline bool isar_feature_aa32_pan(const ARMISARegisters *id) +{ + return FIELD_EX64(id->mvfr0, ID_MMFR3, PAN) != 0; +} + +static inline bool isar_feature_aa32_ats1e1(const ARMISARegisters *id) +{ + return FIELD_EX64(id->mvfr0, ID_MMFR3, PAN) >= 2; +} + /* * 64-bit feature tests via id registers. */ @@ -XXX,XX +XXX,XX @@ static inline bool isar_feature_aa64_lor(const ARMISARegisters *id) return FIELD_EX64(id->id_aa64mmfr1, ID_AA64MMFR1, LO) != 0; } +static inline bool isar_feature_aa64_pan(const ARMISARegisters *id) +{ + return FIELD_EX64(id->id_aa64mmfr1, ID_AA64MMFR1, PAN) != 0; +} + +static inline bool isar_feature_aa64_ats1e1(const ARMISARegisters *id) +{ + return FIELD_EX64(id->id_aa64mmfr1, ID_AA64MMFR1, PAN) >= 2; +} + static inline bool isar_feature_aa64_bti(const ARMISARegisters *id) { return FIELD_EX64(id->id_aa64pfr1, ID_AA64PFR1, BT) != 0; -- 2.20.1 From: Richard Henderson <richard.henderson@linaro.org> For static const regdefs, file scope is preferred. Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200208125816.14954-5-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/helper.c | 57 +++++++++++++++++++++++---------------------- 1 file changed, 29 insertions(+), 28 deletions(-) diff --git a/target/arm/helper.c b/target/arm/helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -XXX,XX +XXX,XX @@ static CPAccessResult access_lor_other(CPUARMState *env, return access_lor_ns(env); } +/* + * A trivial implementation of ARMv8.1-LOR leaves all of these + * registers fixed at 0, which indicates that there are zero + * supported Limited Ordering regions. + */ +static const ARMCPRegInfo lor_reginfo[] = { + { .name = "LORSA_EL1", .state = ARM_CP_STATE_AA64, + .opc0 = 3, .opc1 = 0, .crn = 10, .crm = 4, .opc2 = 0, + .access = PL1_RW, .accessfn = access_lor_other, + .type = ARM_CP_CONST, .resetvalue = 0 }, + { .name = "LOREA_EL1", .state = ARM_CP_STATE_AA64, + .opc0 = 3, .opc1 = 0, .crn = 10, .crm = 4, .opc2 = 1, + .access = PL1_RW, .accessfn = access_lor_other, + .type = ARM_CP_CONST, .resetvalue = 0 }, + { .name = "LORN_EL1", .state = ARM_CP_STATE_AA64, + .opc0 = 3, .opc1 = 0, .crn = 10, .crm = 4, .opc2 = 2, + .access = PL1_RW, .accessfn = access_lor_other, + .type = ARM_CP_CONST, .resetvalue = 0 }, + { .name = "LORC_EL1", .state = ARM_CP_STATE_AA64, + .opc0 = 3, .opc1 = 0, .crn = 10, .crm = 4, .opc2 = 3, + .access = PL1_RW, .accessfn = access_lor_other, + .type = ARM_CP_CONST, .resetvalue = 0 }, + { .name = "LORID_EL1", .state = ARM_CP_STATE_AA64, + .opc0 = 3, .opc1 = 0, .crn = 10, .crm = 4, .opc2 = 7, + .access = PL1_R, .accessfn = access_lorid, + .type = ARM_CP_CONST, .resetvalue = 0 }, + REGINFO_SENTINEL +}; + #ifdef TARGET_AARCH64 static CPAccessResult access_pauth(CPUARMState *env, const ARMCPRegInfo *ri, bool isread) @@ -XXX,XX +XXX,XX @@ void register_cp_regs_for_features(ARMCPU *cpu) } if (cpu_isar_feature(aa64_lor, cpu)) { - /* - * A trivial implementation of ARMv8.1-LOR leaves all of these - * registers fixed at 0, which indicates that there are zero - * supported Limited Ordering regions. - */ - static const ARMCPRegInfo lor_reginfo[] = { - { .name = "LORSA_EL1", .state = ARM_CP_STATE_AA64, - .opc0 = 3, .opc1 = 0, .crn = 10, .crm = 4, .opc2 = 0, - .access = PL1_RW, .accessfn = access_lor_other, - .type = ARM_CP_CONST, .resetvalue = 0 }, - { .name = "LOREA_EL1", .state = ARM_CP_STATE_AA64, - .opc0 = 3, .opc1 = 0, .crn = 10, .crm = 4, .opc2 = 1, - .access = PL1_RW, .accessfn = access_lor_other, - .type = ARM_CP_CONST, .resetvalue = 0 }, - { .name = "LORN_EL1", .state = ARM_CP_STATE_AA64, - .opc0 = 3, .opc1 = 0, .crn = 10, .crm = 4, .opc2 = 2, - .access = PL1_RW, .accessfn = access_lor_other, - .type = ARM_CP_CONST, .resetvalue = 0 }, - { .name = "LORC_EL1", .state = ARM_CP_STATE_AA64, - .opc0 = 3, .opc1 = 0, .crn = 10, .crm = 4, .opc2 = 3, - .access = PL1_RW, .accessfn = access_lor_other, - .type = ARM_CP_CONST, .resetvalue = 0 }, - { .name = "LORID_EL1", .state = ARM_CP_STATE_AA64, - .opc0 = 3, .opc1 = 0, .crn = 10, .crm = 4, .opc2 = 7, - .access = PL1_R, .accessfn = access_lorid, - .type = ARM_CP_CONST, .resetvalue = 0 }, - REGINFO_SENTINEL - }; define_arm_cp_regs(cpu, lor_reginfo); } -- 2.20.1 From: Richard Henderson <richard.henderson@linaro.org> Split this helper out of msr_mask in translate.c. At the same time, transform the negative reductive logic to positive accumulative logic. It will be usable along the exception paths. While touching msr_mask, fix up formatting. Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Message-id: 20200208125816.14954-6-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/internals.h | 21 +++++++++++++++++++++ target/arm/translate.c | 40 +++++++++++++++++----------------------- 2 files changed, 38 insertions(+), 23 deletions(-) diff --git a/target/arm/internals.h b/target/arm/internals.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/internals.h +++ b/target/arm/internals.h @@ -XXX,XX +XXX,XX @@ static inline bool arm_mmu_idx_is_stage1_of_2(ARMMMUIdx mmu_idx) } } +static inline uint32_t aarch32_cpsr_valid_mask(uint64_t features, + const ARMISARegisters *id) +{ + uint32_t valid = CPSR_M | CPSR_AIF | CPSR_IL | CPSR_NZCV | CPSR_J; + + if ((features >> ARM_FEATURE_V4T) & 1) { + valid |= CPSR_T; + } + if ((features >> ARM_FEATURE_V5) & 1) { + valid |= CPSR_Q; /* V5TE in reality*/ + } + if ((features >> ARM_FEATURE_V6) & 1) { + valid |= CPSR_E | CPSR_GE; + } + if ((features >> ARM_FEATURE_THUMB2) & 1) { + valid |= CPSR_IT; + } + + return valid; +} + /* * Parameters of a given virtual address, as extracted from the * translation control register (TCR) for a given regime. diff --git a/target/arm/translate.c b/target/arm/translate.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate.c +++ b/target/arm/translate.c @@ -XXX,XX +XXX,XX @@ static inline void gen_mulxy(TCGv_i32 t0, TCGv_i32 t1, int x, int y) /* Return the mask of PSR bits set by a MSR instruction. */ static uint32_t msr_mask(DisasContext *s, int flags, int spsr) { - uint32_t mask; + uint32_t mask = 0; - mask = 0; - if (flags & (1 << 0)) + if (flags & (1 << 0)) { mask |= 0xff; - if (flags & (1 << 1)) + } + if (flags & (1 << 1)) { mask |= 0xff00; - if (flags & (1 << 2)) + } + if (flags & (1 << 2)) { mask |= 0xff0000; - if (flags & (1 << 3)) + } + if (flags & (1 << 3)) { mask |= 0xff000000; + } - /* Mask out undefined bits. */ - mask &= ~CPSR_RESERVED; - if (!arm_dc_feature(s, ARM_FEATURE_V4T)) { - mask &= ~CPSR_T; - } - if (!arm_dc_feature(s, ARM_FEATURE_V5)) { - mask &= ~CPSR_Q; /* V5TE in reality*/ - } - if (!arm_dc_feature(s, ARM_FEATURE_V6)) { - mask &= ~(CPSR_E | CPSR_GE); - } - if (!arm_dc_feature(s, ARM_FEATURE_THUMB2)) { - mask &= ~CPSR_IT; - } - /* Mask out execution state and reserved bits. */ + /* Mask out undefined and reserved bits. */ + mask &= aarch32_cpsr_valid_mask(s->features, s->isar); + + /* Mask out execution state. */ if (!spsr) { - mask &= ~(CPSR_EXEC | CPSR_RESERVED); + mask &= ~CPSR_EXEC; } + /* Mask out privileged bits. */ - if (IS_USER(s)) + if (IS_USER(s)) { mask &= CPSR_USER; + } return mask; } -- 2.20.1 From: Richard Henderson <richard.henderson@linaro.org> The J bit signals Jazelle mode, and so of course is RES0 when the feature is not enabled. Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Message-id: 20200208125816.14954-7-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/internals.h | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/target/arm/internals.h b/target/arm/internals.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/internals.h +++ b/target/arm/internals.h @@ -XXX,XX +XXX,XX @@ static inline bool arm_mmu_idx_is_stage1_of_2(ARMMMUIdx mmu_idx) static inline uint32_t aarch32_cpsr_valid_mask(uint64_t features, const ARMISARegisters *id) { - uint32_t valid = CPSR_M | CPSR_AIF | CPSR_IL | CPSR_NZCV | CPSR_J; + uint32_t valid = CPSR_M | CPSR_AIF | CPSR_IL | CPSR_NZCV; if ((features >> ARM_FEATURE_V4T) & 1) { valid |= CPSR_T; @@ -XXX,XX +XXX,XX @@ static inline uint32_t aarch32_cpsr_valid_mask(uint64_t features, if ((features >> ARM_FEATURE_THUMB2) & 1) { valid |= CPSR_IT; } + if (isar_feature_jazelle(id)) { + valid |= CPSR_J; + } return valid; } -- 2.20.1 From: Richard Henderson <richard.henderson@linaro.org> CPSR_ERET_MASK was a useless renaming of CPSR_RESERVED. The function also takes into account bits that the cpu does not support. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200208125816.14954-8-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/cpu.h | 2 -- target/arm/op_helper.c | 5 ++++- 2 files changed, 4 insertions(+), 3 deletions(-) diff --git a/target/arm/cpu.h b/target/arm/cpu.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -XXX,XX +XXX,XX @@ void pmu_init(ARMCPU *cpu); #define CPSR_USER (CPSR_NZCV | CPSR_Q | CPSR_GE) /* Execution state bits. MRS read as zero, MSR writes ignored. */ #define CPSR_EXEC (CPSR_T | CPSR_IT | CPSR_J | CPSR_IL) -/* Mask of bits which may be set by exception return copying them from SPSR */ -#define CPSR_ERET_MASK (~CPSR_RESERVED) /* Bit definitions for M profile XPSR. Most are the same as CPSR. */ #define XPSR_EXCP 0x1ffU diff --git a/target/arm/op_helper.c b/target/arm/op_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/op_helper.c +++ b/target/arm/op_helper.c @@ -XXX,XX +XXX,XX @@ void HELPER(cpsr_write)(CPUARMState *env, uint32_t val, uint32_t mask) /* Write the CPSR for a 32-bit exception return */ void HELPER(cpsr_write_eret)(CPUARMState *env, uint32_t val) { + uint32_t mask; + qemu_mutex_lock_iothread(); arm_call_pre_el_change_hook(env_archcpu(env)); qemu_mutex_unlock_iothread(); - cpsr_write(env, val, CPSR_ERET_MASK, CPSRWriteExceptionReturn); + mask = aarch32_cpsr_valid_mask(env->features, &env_archcpu(env)->isar); + cpsr_write(env, val, mask, CPSRWriteExceptionReturn); /* Generated code has already stored the new PC value, but * without masking out its low bits, because which bits need -- 2.20.1 From: Richard Henderson <richard.henderson@linaro.org> Using ~0 as the mask on the aarch64->aarch32 exception return was not even as correct as the CPSR_ERET_MASK that we had used on the aarch32->aarch32 exception return. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200208125816.14954-9-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/helper-a64.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/target/arm/helper-a64.c b/target/arm/helper-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper-a64.c +++ b/target/arm/helper-a64.c @@ -XXX,XX +XXX,XX @@ void HELPER(exception_return)(CPUARMState *env, uint64_t new_pc) { int cur_el = arm_current_el(env); unsigned int spsr_idx = aarch64_banked_spsr_index(cur_el); - uint32_t spsr = env->banked_spsr[spsr_idx]; + uint32_t mask, spsr = env->banked_spsr[spsr_idx]; int new_el; bool return_to_aa64 = (spsr & PSTATE_nRW) == 0; @@ -XXX,XX +XXX,XX @@ void HELPER(exception_return)(CPUARMState *env, uint64_t new_pc) * will sort the register banks out for us, and we've already * caught all the bad-mode cases in el_from_spsr(). */ - cpsr_write(env, spsr, ~0, CPSRWriteRaw); + mask = aarch32_cpsr_valid_mask(env->features, &env_archcpu(env)->isar); + cpsr_write(env, spsr, mask, CPSRWriteRaw); if (!arm_singlestep_active(env)) { env->uncached_cpsr &= ~PSTATE_SS; } -- 2.20.1 From: Richard Henderson <richard.henderson@linaro.org> The only remaining use was in op_helper.c. Use PSTATE_SS directly, and move the commentary so that it is more obvious what is going on. Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Message-id: 20200208125816.14954-10-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/cpu.h | 6 ------ target/arm/op_helper.c | 9 ++++++++- 2 files changed, 8 insertions(+), 7 deletions(-) diff --git a/target/arm/cpu.h b/target/arm/cpu.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -XXX,XX +XXX,XX @@ void pmu_init(ARMCPU *cpu); #define CPSR_IT_2_7 (0xfc00U) #define CPSR_GE (0xfU << 16) #define CPSR_IL (1U << 20) -/* Note that the RESERVED bits include bit 21, which is PSTATE_SS in - * an AArch64 SPSR but RES0 in AArch32 SPSR and CPSR. In QEMU we use - * env->uncached_cpsr bit 21 to store PSTATE.SS when executing in AArch32, - * where it is live state but not accessible to the AArch32 code. - */ -#define CPSR_RESERVED (0x7U << 21) #define CPSR_J (1U << 24) #define CPSR_IT_0_1 (3U << 25) #define CPSR_Q (1U << 27) diff --git a/target/arm/op_helper.c b/target/arm/op_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/op_helper.c +++ b/target/arm/op_helper.c @@ -XXX,XX +XXX,XX @@ void HELPER(exception_bkpt_insn)(CPUARMState *env, uint32_t syndrome) uint32_t HELPER(cpsr_read)(CPUARMState *env) { - return cpsr_read(env) & ~(CPSR_EXEC | CPSR_RESERVED); + /* + * We store the ARMv8 PSTATE.SS bit in env->uncached_cpsr. + * This is convenient for populating SPSR_ELx, but must be + * hidden from aarch32 mode, where it is not visible. + * + * TODO: ARMv8.4-DIT -- need to move SS somewhere else. + */ + return cpsr_read(env) & ~(CPSR_EXEC | PSTATE_SS); } void HELPER(cpsr_write)(CPUARMState *env, uint32_t val, uint32_t mask) -- 2.20.1 From: Richard Henderson <richard.henderson@linaro.org> Use this along the exception return path, where we previously accepted any values. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200208125816.14954-11-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/internals.h | 12 ++++++++++++ target/arm/helper-a64.c | 1 + 2 files changed, 13 insertions(+) diff --git a/target/arm/internals.h b/target/arm/internals.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/internals.h +++ b/target/arm/internals.h @@ -XXX,XX +XXX,XX @@ static inline uint32_t aarch32_cpsr_valid_mask(uint64_t features, return valid; } +static inline uint32_t aarch64_pstate_valid_mask(const ARMISARegisters *id) +{ + uint32_t valid; + + valid = PSTATE_M | PSTATE_DAIF | PSTATE_IL | PSTATE_SS | PSTATE_NZCV; + if (isar_feature_aa64_bti(id)) { + valid |= PSTATE_BTYPE; + } + + return valid; +} + /* * Parameters of a given virtual address, as extracted from the * translation control register (TCR) for a given regime. diff --git a/target/arm/helper-a64.c b/target/arm/helper-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper-a64.c +++ b/target/arm/helper-a64.c @@ -XXX,XX +XXX,XX @@ void HELPER(exception_return)(CPUARMState *env, uint64_t new_pc) cur_el, new_el, env->regs[15]); } else { env->aarch64 = 1; + spsr &= aarch64_pstate_valid_mask(&env_archcpu(env)->isar); pstate_write(env, spsr); if (!arm_singlestep_active(env)) { env->pstate &= ~PSTATE_SS; -- 2.20.1 From: Richard Henderson <richard.henderson@linaro.org> For aarch64, there's a dedicated msr (imm, reg) insn. For aarch32, this is done via msr to cpsr. Writes from el0 are ignored, which is already handled by the CPSR_USER mask. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200208125816.14954-12-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/cpu.h | 2 ++ target/arm/internals.h | 6 ++++++ target/arm/helper.c | 21 +++++++++++++++++++++ target/arm/translate-a64.c | 14 ++++++++++++++ 4 files changed, 43 insertions(+) diff --git a/target/arm/cpu.h b/target/arm/cpu.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -XXX,XX +XXX,XX @@ void pmu_init(ARMCPU *cpu); #define CPSR_IT_2_7 (0xfc00U) #define CPSR_GE (0xfU << 16) #define CPSR_IL (1U << 20) +#define CPSR_PAN (1U << 22) #define CPSR_J (1U << 24) #define CPSR_IT_0_1 (3U << 25) #define CPSR_Q (1U << 27) @@ -XXX,XX +XXX,XX @@ void pmu_init(ARMCPU *cpu); #define PSTATE_BTYPE (3U << 10) #define PSTATE_IL (1U << 20) #define PSTATE_SS (1U << 21) +#define PSTATE_PAN (1U << 22) #define PSTATE_V (1U << 28) #define PSTATE_C (1U << 29) #define PSTATE_Z (1U << 30) diff --git a/target/arm/internals.h b/target/arm/internals.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/internals.h +++ b/target/arm/internals.h @@ -XXX,XX +XXX,XX @@ static inline uint32_t aarch32_cpsr_valid_mask(uint64_t features, if (isar_feature_jazelle(id)) { valid |= CPSR_J; } + if (isar_feature_aa32_pan(id)) { + valid |= CPSR_PAN; + } return valid; } @@ -XXX,XX +XXX,XX @@ static inline uint32_t aarch64_pstate_valid_mask(const ARMISARegisters *id) if (isar_feature_aa64_bti(id)) { valid |= PSTATE_BTYPE; } + if (isar_feature_aa64_pan(id)) { + valid |= PSTATE_PAN; + } return valid; } diff --git a/target/arm/helper.c b/target/arm/helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -XXX,XX +XXX,XX @@ static void aa64_daif_write(CPUARMState *env, const ARMCPRegInfo *ri, env->daif = value & PSTATE_DAIF; } +static uint64_t aa64_pan_read(CPUARMState *env, const ARMCPRegInfo *ri) +{ + return env->pstate & PSTATE_PAN; +} + +static void aa64_pan_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) +{ + env->pstate = (env->pstate & ~PSTATE_PAN) | (value & PSTATE_PAN); +} + +static const ARMCPRegInfo pan_reginfo = { + .name = "PAN", .state = ARM_CP_STATE_AA64, + .opc0 = 3, .opc1 = 0, .crn = 4, .crm = 2, .opc2 = 3, + .type = ARM_CP_NO_RAW, .access = PL1_RW, + .readfn = aa64_pan_read, .writefn = aa64_pan_write +}; + static CPAccessResult aa64_cacheop_access(CPUARMState *env, const ARMCPRegInfo *ri, bool isread) @@ -XXX,XX +XXX,XX @@ void register_cp_regs_for_features(ARMCPU *cpu) if (cpu_isar_feature(aa64_lor, cpu)) { define_arm_cp_regs(cpu, lor_reginfo); } + if (cpu_isar_feature(aa64_pan, cpu)) { + define_one_arm_cp_reg(cpu, &pan_reginfo); + } if (arm_feature(env, ARM_FEATURE_EL2) && cpu_isar_feature(aa64_vh, cpu)) { define_arm_cp_regs(cpu, vhe_reginfo); diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-a64.c +++ b/target/arm/translate-a64.c @@ -XXX,XX +XXX,XX @@ static void handle_msr_i(DisasContext *s, uint32_t insn, s->base.is_jmp = DISAS_NEXT; break; + case 0x04: /* PAN */ + if (!dc_isar_feature(aa64_pan, s) || s->current_el == 0) { + goto do_unallocated; + } + if (crm & 1) { + set_pstate_bits(PSTATE_PAN); + } else { + clear_pstate_bits(PSTATE_PAN); + } + t1 = tcg_const_i32(s->current_el); + gen_helper_rebuild_hflags_a64(cpu_env, t1); + tcg_temp_free_i32(t1); + break; + case 0x05: /* SPSel */ if (s->current_el == 0) { goto do_unallocated; -- 2.20.1 From: Richard Henderson <richard.henderson@linaro.org> Examine the PAN bit for EL1, EL2, and Secure EL1 to determine if it applies. Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200208125816.14954-13-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/helper.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/target/arm/helper.c b/target/arm/helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -XXX,XX +XXX,XX @@ ARMMMUIdx arm_mmu_idx_el(CPUARMState *env, int el) return ARMMMUIdx_E10_0; case 1: if (arm_is_secure_below_el3(env)) { + if (env->pstate & PSTATE_PAN) { + return ARMMMUIdx_SE10_1_PAN; + } return ARMMMUIdx_SE10_1; } + if (env->pstate & PSTATE_PAN) { + return ARMMMUIdx_E10_1_PAN; + } return ARMMMUIdx_E10_1; case 2: /* TODO: ARMv8.4-SecEL2 */ /* Note that TGE does not apply at EL2. */ if ((env->cp15.hcr_el2 & HCR_E2H) && arm_el_is_aa64(env, 2)) { + if (env->pstate & PSTATE_PAN) { + return ARMMMUIdx_E20_2_PAN; + } return ARMMMUIdx_E20_2; } return ARMMMUIdx_E2; -- 2.20.1 From: Richard Henderson <richard.henderson@linaro.org> If we have a PAN-enforcing mmu_idx, set prot == 0 if user_rw != 0. Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200208125816.14954-14-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/internals.h | 13 +++++++++++++ target/arm/helper.c | 3 +++ 2 files changed, 16 insertions(+) diff --git a/target/arm/internals.h b/target/arm/internals.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/internals.h +++ b/target/arm/internals.h @@ -XXX,XX +XXX,XX @@ static inline bool regime_is_secure(CPUARMState *env, ARMMMUIdx mmu_idx) } } +static inline bool regime_is_pan(CPUARMState *env, ARMMMUIdx mmu_idx) +{ + switch (mmu_idx) { + case ARMMMUIdx_Stage1_E1_PAN: + case ARMMMUIdx_E10_1_PAN: + case ARMMMUIdx_E20_2_PAN: + case ARMMMUIdx_SE10_1_PAN: + return true; + default: + return false; + } +} + /* Return the FSR value for a debug exception (watchpoint, hardware * breakpoint or BKPT insn) targeting the specified exception level. */ diff --git a/target/arm/helper.c b/target/arm/helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -XXX,XX +XXX,XX @@ static int get_S1prot(CPUARMState *env, ARMMMUIdx mmu_idx, bool is_aa64, if (is_user) { prot_rw = user_rw; } else { + if (user_rw && regime_is_pan(env, mmu_idx)) { + return 0; + } prot_rw = simple_ap_to_rw_prot_is_user(ap, false); } -- 2.20.1 From: Richard Henderson <richard.henderson@linaro.org> The PAN bit is preserved, or set as per SCTLR_ELx.SPAN, plus several other conditions listed in the ARM ARM. Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Message-id: 20200208125816.14954-15-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/helper.c | 53 ++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 50 insertions(+), 3 deletions(-) diff --git a/target/arm/helper.c b/target/arm/helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -XXX,XX +XXX,XX @@ static void take_aarch32_exception(CPUARMState *env, int new_mode, uint32_t mask, uint32_t offset, uint32_t newpc) { + int new_el; + /* Change the CPU state so as to actually take the exception. */ switch_mode(env, new_mode); + new_el = arm_current_el(env); + /* * For exceptions taken to AArch32 we must clear the SS bit in both * PSTATE and in the old-state value we save to SPSR_<mode>, so zero it now. @@ -XXX,XX +XXX,XX @@ static void take_aarch32_exception(CPUARMState *env, int new_mode, env->uncached_cpsr = (env->uncached_cpsr & ~CPSR_M) | new_mode; /* Set new mode endianness */ env->uncached_cpsr &= ~CPSR_E; - if (env->cp15.sctlr_el[arm_current_el(env)] & SCTLR_EE) { + if (env->cp15.sctlr_el[new_el] & SCTLR_EE) { env->uncached_cpsr |= CPSR_E; } /* J and IL must always be cleared for exception entry */ @@ -XXX,XX +XXX,XX @@ static void take_aarch32_exception(CPUARMState *env, int new_mode, env->thumb = (env->cp15.sctlr_el[2] & SCTLR_TE) != 0; env->elr_el[2] = env->regs[15]; } else { + /* CPSR.PAN is normally preserved preserved unless... */ + if (cpu_isar_feature(aa64_pan, env_archcpu(env))) { + switch (new_el) { + case 3: + if (!arm_is_secure_below_el3(env)) { + /* ... the target is EL3, from non-secure state. */ + env->uncached_cpsr &= ~CPSR_PAN; + break; + } + /* ... the target is EL3, from secure state ... */ + /* fall through */ + case 1: + /* ... the target is EL1 and SCTLR.SPAN is 0. */ + if (!(env->cp15.sctlr_el[new_el] & SCTLR_SPAN)) { + env->uncached_cpsr |= CPSR_PAN; + } + break; + } + } /* * this is a lie, as there was no c1_sys on V4T/V5, but who cares * and we should just guard the thumb mode on V4 @@ -XXX,XX +XXX,XX @@ static void arm_cpu_do_interrupt_aarch64(CPUState *cs) unsigned int new_el = env->exception.target_el; target_ulong addr = env->cp15.vbar_el[new_el]; unsigned int new_mode = aarch64_pstate_mode(new_el, true); + unsigned int old_mode; unsigned int cur_el = arm_current_el(env); /* @@ -XXX,XX +XXX,XX @@ static void arm_cpu_do_interrupt_aarch64(CPUState *cs) } if (is_a64(env)) { - env->banked_spsr[aarch64_banked_spsr_index(new_el)] = pstate_read(env); + old_mode = pstate_read(env); aarch64_save_sp(env, arm_current_el(env)); env->elr_el[new_el] = env->pc; } else { - env->banked_spsr[aarch64_banked_spsr_index(new_el)] = cpsr_read(env); + old_mode = cpsr_read(env); env->elr_el[new_el] = env->regs[15]; aarch64_sync_32_to_64(env); env->condexec_bits = 0; } + env->banked_spsr[aarch64_banked_spsr_index(new_el)] = old_mode; + qemu_log_mask(CPU_LOG_INT, "...with ELR 0x%" PRIx64 "\n", env->elr_el[new_el]); + if (cpu_isar_feature(aa64_pan, cpu)) { + /* The value of PSTATE.PAN is normally preserved, except when ... */ + new_mode |= old_mode & PSTATE_PAN; + switch (new_el) { + case 2: + /* ... the target is EL2 with HCR_EL2.{E2H,TGE} == '11' ... */ + if ((arm_hcr_el2_eff(env) & (HCR_E2H | HCR_TGE)) + != (HCR_E2H | HCR_TGE)) { + break; + } + /* fall through */ + case 1: + /* ... the target is EL1 ... */ + /* ... and SCTLR_ELx.SPAN == 0, then set to 1. */ + if ((env->cp15.sctlr_el[new_el] & SCTLR_SPAN) == 0) { + new_mode |= PSTATE_PAN; + } + break; + } + } + pstate_write(env, PSTATE_DAIF | new_mode); env->aarch64 = 1; aarch64_restore_sp(env, new_el); -- 2.20.1 From: Richard Henderson <richard.henderson@linaro.org> This is a minor enhancement over ARMv8.1-PAN. The *_PAN mmu_idx are used with the existing do_ats_write. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200208125816.14954-16-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/helper.c | 56 ++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 50 insertions(+), 6 deletions(-) diff --git a/target/arm/helper.c b/target/arm/helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -XXX,XX +XXX,XX @@ static void ats_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value) switch (ri->opc2 & 6) { case 0: - /* stage 1 current state PL1: ATS1CPR, ATS1CPW */ + /* stage 1 current state PL1: ATS1CPR, ATS1CPW, ATS1CPRP, ATS1CPWP */ switch (el) { case 3: mmu_idx = ARMMMUIdx_SE3; break; case 2: - mmu_idx = ARMMMUIdx_Stage1_E1; - break; + g_assert(!secure); /* TODO: ARMv8.4-SecEL2 */ + /* fall through */ case 1: - mmu_idx = secure ? ARMMMUIdx_SE10_1 : ARMMMUIdx_Stage1_E1; + if (ri->crm == 9 && (env->uncached_cpsr & CPSR_PAN)) { + mmu_idx = (secure ? ARMMMUIdx_SE10_1_PAN + : ARMMMUIdx_Stage1_E1_PAN); + } else { + mmu_idx = secure ? ARMMMUIdx_SE10_1 : ARMMMUIdx_Stage1_E1; + } break; default: g_assert_not_reached(); @@ -XXX,XX +XXX,XX @@ static void ats_write64(CPUARMState *env, const ARMCPRegInfo *ri, switch (ri->opc2 & 6) { case 0: switch (ri->opc1) { - case 0: /* AT S1E1R, AT S1E1W */ - mmu_idx = secure ? ARMMMUIdx_SE10_1 : ARMMMUIdx_Stage1_E1; + case 0: /* AT S1E1R, AT S1E1W, AT S1E1RP, AT S1E1WP */ + if (ri->crm == 9 && (env->pstate & PSTATE_PAN)) { + mmu_idx = (secure ? ARMMMUIdx_SE10_1_PAN + : ARMMMUIdx_Stage1_E1_PAN); + } else { + mmu_idx = secure ? ARMMMUIdx_SE10_1 : ARMMMUIdx_Stage1_E1; + } break; case 4: /* AT S1E2R, AT S1E2W */ mmu_idx = ARMMMUIdx_E2; @@ -XXX,XX +XXX,XX @@ static const ARMCPRegInfo vhe_reginfo[] = { REGINFO_SENTINEL }; +#ifndef CONFIG_USER_ONLY +static const ARMCPRegInfo ats1e1_reginfo[] = { + { .name = "AT_S1E1R", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 0, .crn = 7, .crm = 9, .opc2 = 0, + .access = PL1_W, .type = ARM_CP_NO_RAW | ARM_CP_RAISES_EXC, + .writefn = ats_write64 }, + { .name = "AT_S1E1W", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 0, .crn = 7, .crm = 9, .opc2 = 1, + .access = PL1_W, .type = ARM_CP_NO_RAW | ARM_CP_RAISES_EXC, + .writefn = ats_write64 }, + REGINFO_SENTINEL +}; + +static const ARMCPRegInfo ats1cp_reginfo[] = { + { .name = "ATS1CPRP", + .cp = 15, .opc1 = 0, .crn = 7, .crm = 9, .opc2 = 0, + .access = PL1_W, .type = ARM_CP_NO_RAW | ARM_CP_RAISES_EXC, + .writefn = ats_write }, + { .name = "ATS1CPWP", + .cp = 15, .opc1 = 0, .crn = 7, .crm = 9, .opc2 = 1, + .access = PL1_W, .type = ARM_CP_NO_RAW | ARM_CP_RAISES_EXC, + .writefn = ats_write }, + REGINFO_SENTINEL +}; +#endif + void register_cp_regs_for_features(ARMCPU *cpu) { /* Register all the coprocessor registers based on feature bits */ @@ -XXX,XX +XXX,XX @@ void register_cp_regs_for_features(ARMCPU *cpu) if (cpu_isar_feature(aa64_pan, cpu)) { define_one_arm_cp_reg(cpu, &pan_reginfo); } +#ifndef CONFIG_USER_ONLY + if (cpu_isar_feature(aa64_ats1e1, cpu)) { + define_arm_cp_regs(cpu, ats1e1_reginfo); + } + if (cpu_isar_feature(aa32_ats1e1, cpu)) { + define_arm_cp_regs(cpu, ats1cp_reginfo); + } +#endif if (arm_feature(env, ARM_FEATURE_EL2) && cpu_isar_feature(aa64_vh, cpu)) { define_arm_cp_regs(cpu, vhe_reginfo); -- 2.20.1 From: Richard Henderson <richard.henderson@linaro.org> This includes enablement of ARMv8.1-PAN. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200208125816.14954-17-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/cpu.c | 4 ++++ target/arm/cpu64.c | 5 +++++ 2 files changed, 9 insertions(+) diff --git a/target/arm/cpu.c b/target/arm/cpu.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpu.c +++ b/target/arm/cpu.c @@ -XXX,XX +XXX,XX @@ static void arm_max_initfn(Object *obj) t = FIELD_DP32(t, MVFR2, FPMISC, 4); /* FP MaxNum */ cpu->isar.mvfr2 = t; + t = cpu->id_mmfr3; + t = FIELD_DP32(t, ID_MMFR3, PAN, 2); /* ATS1E1 */ + cpu->id_mmfr3 = t; + t = cpu->id_mmfr4; t = FIELD_DP32(t, ID_MMFR4, HPDS, 1); /* AA32HPD */ cpu->id_mmfr4 = t; diff --git a/target/arm/cpu64.c b/target/arm/cpu64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpu64.c +++ b/target/arm/cpu64.c @@ -XXX,XX +XXX,XX @@ static void aarch64_max_initfn(Object *obj) t = FIELD_DP64(t, ID_AA64MMFR1, HPDS, 1); /* HPD */ t = FIELD_DP64(t, ID_AA64MMFR1, LO, 1); t = FIELD_DP64(t, ID_AA64MMFR1, VH, 1); + t = FIELD_DP64(t, ID_AA64MMFR1, PAN, 2); /* ATS1E1 */ cpu->isar.id_aa64mmfr1 = t; /* Replicate the same data to the 32-bit id registers. */ @@ -XXX,XX +XXX,XX @@ static void aarch64_max_initfn(Object *obj) u = FIELD_DP32(u, ID_ISAR6, SPECRES, 1); cpu->isar.id_isar6 = u; + u = cpu->id_mmfr3; + u = FIELD_DP32(u, ID_MMFR3, PAN, 2); /* ATS1E1 */ + cpu->id_mmfr3 = u; + /* * FIXME: We do not yet support ARMv8.2-fp16 for AArch32 yet, * so do not set MVFR1.FPHP. Strictly speaking this is not legal, -- 2.20.1 From: Richard Henderson <richard.henderson@linaro.org> Add definitions for all of the fields, up to ARMv8.5. Convert the existing RESERVED register to a full register. Query KVM for the value of the register for the host. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200208125816.14954-18-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/cpu.h | 17 +++++++++++++++++ target/arm/helper.c | 4 ++-- target/arm/kvm64.c | 2 ++ 3 files changed, 21 insertions(+), 2 deletions(-) diff --git a/target/arm/cpu.h b/target/arm/cpu.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -XXX,XX +XXX,XX @@ struct ARMCPU { uint64_t id_aa64pfr1; uint64_t id_aa64mmfr0; uint64_t id_aa64mmfr1; + uint64_t id_aa64mmfr2; } isar; uint32_t midr; uint32_t revidr; @@ -XXX,XX +XXX,XX @@ FIELD(ID_AA64MMFR1, PAN, 20, 4) FIELD(ID_AA64MMFR1, SPECSEI, 24, 4) FIELD(ID_AA64MMFR1, XNX, 28, 4) +FIELD(ID_AA64MMFR2, CNP, 0, 4) +FIELD(ID_AA64MMFR2, UAO, 4, 4) +FIELD(ID_AA64MMFR2, LSM, 8, 4) +FIELD(ID_AA64MMFR2, IESB, 12, 4) +FIELD(ID_AA64MMFR2, VARANGE, 16, 4) +FIELD(ID_AA64MMFR2, CCIDX, 20, 4) +FIELD(ID_AA64MMFR2, NV, 24, 4) +FIELD(ID_AA64MMFR2, ST, 28, 4) +FIELD(ID_AA64MMFR2, AT, 32, 4) +FIELD(ID_AA64MMFR2, IDS, 36, 4) +FIELD(ID_AA64MMFR2, FWB, 40, 4) +FIELD(ID_AA64MMFR2, TTL, 48, 4) +FIELD(ID_AA64MMFR2, BBM, 52, 4) +FIELD(ID_AA64MMFR2, EVT, 56, 4) +FIELD(ID_AA64MMFR2, E0PD, 60, 4) + FIELD(ID_DFR0, COPDBG, 0, 4) FIELD(ID_DFR0, COPSDBG, 4, 4) FIELD(ID_DFR0, MMAPDBG, 8, 4) diff --git a/target/arm/helper.c b/target/arm/helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -XXX,XX +XXX,XX @@ void register_cp_regs_for_features(ARMCPU *cpu) .access = PL1_R, .type = ARM_CP_CONST, .accessfn = access_aa64_tid3, .resetvalue = cpu->isar.id_aa64mmfr1 }, - { .name = "ID_AA64MMFR2_EL1_RESERVED", .state = ARM_CP_STATE_AA64, + { .name = "ID_AA64MMFR2_EL1", .state = ARM_CP_STATE_AA64, .opc0 = 3, .opc1 = 0, .crn = 0, .crm = 7, .opc2 = 2, .access = PL1_R, .type = ARM_CP_CONST, .accessfn = access_aa64_tid3, - .resetvalue = 0 }, + .resetvalue = cpu->isar.id_aa64mmfr2 }, { .name = "ID_AA64MMFR3_EL1_RESERVED", .state = ARM_CP_STATE_AA64, .opc0 = 3, .opc1 = 0, .crn = 0, .crm = 7, .opc2 = 3, .access = PL1_R, .type = ARM_CP_CONST, diff --git a/target/arm/kvm64.c b/target/arm/kvm64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/kvm64.c +++ b/target/arm/kvm64.c @@ -XXX,XX +XXX,XX @@ bool kvm_arm_get_host_cpu_features(ARMHostCPUFeatures *ahcf) ARM64_SYS_REG(3, 0, 0, 7, 0)); err |= read_sys_reg64(fdarray[2], &ahcf->isar.id_aa64mmfr1, ARM64_SYS_REG(3, 0, 0, 7, 1)); + err |= read_sys_reg64(fdarray[2], &ahcf->isar.id_aa64mmfr2, + ARM64_SYS_REG(3, 0, 0, 7, 2)); /* * Note that if AArch32 support is not present in the host, -- 2.20.1 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200208125816.14954-19-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/cpu.h | 6 ++++++ target/arm/internals.h | 3 +++ target/arm/helper.c | 21 +++++++++++++++++++++ target/arm/translate-a64.c | 14 ++++++++++++++ 4 files changed, 44 insertions(+) diff --git a/target/arm/cpu.h b/target/arm/cpu.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -XXX,XX +XXX,XX @@ void pmu_init(ARMCPU *cpu); #define PSTATE_IL (1U << 20) #define PSTATE_SS (1U << 21) #define PSTATE_PAN (1U << 22) +#define PSTATE_UAO (1U << 23) #define PSTATE_V (1U << 28) #define PSTATE_C (1U << 29) #define PSTATE_Z (1U << 30) @@ -XXX,XX +XXX,XX @@ static inline bool isar_feature_aa64_ats1e1(const ARMISARegisters *id) return FIELD_EX64(id->id_aa64mmfr1, ID_AA64MMFR1, PAN) >= 2; } +static inline bool isar_feature_aa64_uao(const ARMISARegisters *id) +{ + return FIELD_EX64(id->id_aa64mmfr2, ID_AA64MMFR2, UAO) != 0; +} + static inline bool isar_feature_aa64_bti(const ARMISARegisters *id) { return FIELD_EX64(id->id_aa64pfr1, ID_AA64PFR1, BT) != 0; diff --git a/target/arm/internals.h b/target/arm/internals.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/internals.h +++ b/target/arm/internals.h @@ -XXX,XX +XXX,XX @@ static inline uint32_t aarch64_pstate_valid_mask(const ARMISARegisters *id) if (isar_feature_aa64_pan(id)) { valid |= PSTATE_PAN; } + if (isar_feature_aa64_uao(id)) { + valid |= PSTATE_UAO; + } return valid; } diff --git a/target/arm/helper.c b/target/arm/helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -XXX,XX +XXX,XX @@ static const ARMCPRegInfo pan_reginfo = { .readfn = aa64_pan_read, .writefn = aa64_pan_write }; +static uint64_t aa64_uao_read(CPUARMState *env, const ARMCPRegInfo *ri) +{ + return env->pstate & PSTATE_UAO; +} + +static void aa64_uao_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) +{ + env->pstate = (env->pstate & ~PSTATE_UAO) | (value & PSTATE_UAO); +} + +static const ARMCPRegInfo uao_reginfo = { + .name = "UAO", .state = ARM_CP_STATE_AA64, + .opc0 = 3, .opc1 = 0, .crn = 4, .crm = 2, .opc2 = 4, + .type = ARM_CP_NO_RAW, .access = PL1_RW, + .readfn = aa64_uao_read, .writefn = aa64_uao_write +}; + static CPAccessResult aa64_cacheop_access(CPUARMState *env, const ARMCPRegInfo *ri, bool isread) @@ -XXX,XX +XXX,XX @@ void register_cp_regs_for_features(ARMCPU *cpu) define_arm_cp_regs(cpu, ats1cp_reginfo); } #endif + if (cpu_isar_feature(aa64_uao, cpu)) { + define_one_arm_cp_reg(cpu, &uao_reginfo); + } if (arm_feature(env, ARM_FEATURE_EL2) && cpu_isar_feature(aa64_vh, cpu)) { define_arm_cp_regs(cpu, vhe_reginfo); diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-a64.c +++ b/target/arm/translate-a64.c @@ -XXX,XX +XXX,XX @@ static void handle_msr_i(DisasContext *s, uint32_t insn, s->base.is_jmp = DISAS_NEXT; break; + case 0x03: /* UAO */ + if (!dc_isar_feature(aa64_uao, s) || s->current_el == 0) { + goto do_unallocated; + } + if (crm & 1) { + set_pstate_bits(PSTATE_UAO); + } else { + clear_pstate_bits(PSTATE_UAO); + } + t1 = tcg_const_i32(s->current_el); + gen_helper_rebuild_hflags_a64(cpu_env, t1); + tcg_temp_free_i32(t1); + break; + case 0x04: /* PAN */ if (!dc_isar_feature(aa64_pan, s) || s->current_el == 0) { goto do_unallocated; -- 2.20.1 From: Richard Henderson <richard.henderson@linaro.org> We need only override the current condition under which TBFLAG_A64.UNPRIV is set. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200208125816.14954-20-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/helper.c | 41 +++++++++++++++++++++-------------------- 1 file changed, 21 insertions(+), 20 deletions(-) diff --git a/target/arm/helper.c b/target/arm/helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -XXX,XX +XXX,XX @@ static uint32_t rebuild_hflags_a64(CPUARMState *env, int el, int fp_el, } /* Compute the condition for using AccType_UNPRIV for LDTR et al. */ - /* TODO: ARMv8.2-UAO */ - switch (mmu_idx) { - case ARMMMUIdx_E10_1: - case ARMMMUIdx_E10_1_PAN: - case ARMMMUIdx_SE10_1: - case ARMMMUIdx_SE10_1_PAN: - /* TODO: ARMv8.3-NV */ - flags = FIELD_DP32(flags, TBFLAG_A64, UNPRIV, 1); - break; - case ARMMMUIdx_E20_2: - case ARMMMUIdx_E20_2_PAN: - /* TODO: ARMv8.4-SecEL2 */ - /* - * Note that E20_2 is gated by HCR_EL2.E2H == 1, but E20_0 is - * gated by HCR_EL2.<E2H,TGE> == '11', and so is LDTR. - */ - if (env->cp15.hcr_el2 & HCR_TGE) { + if (!(env->pstate & PSTATE_UAO)) { + switch (mmu_idx) { + case ARMMMUIdx_E10_1: + case ARMMMUIdx_E10_1_PAN: + case ARMMMUIdx_SE10_1: + case ARMMMUIdx_SE10_1_PAN: + /* TODO: ARMv8.3-NV */ flags = FIELD_DP32(flags, TBFLAG_A64, UNPRIV, 1); + break; + case ARMMMUIdx_E20_2: + case ARMMMUIdx_E20_2_PAN: + /* TODO: ARMv8.4-SecEL2 */ + /* + * Note that EL20_2 is gated by HCR_EL2.E2H == 1, but EL20_0 is + * gated by HCR_EL2.<E2H,TGE> == '11', and so is LDTR. + */ + if (env->cp15.hcr_el2 & HCR_TGE) { + flags = FIELD_DP32(flags, TBFLAG_A64, UNPRIV, 1); + } + break; + default: + break; } - break; - default: - break; } return rebuild_hflags_common(env, fp_el, mmu_idx, flags); -- 2.20.1 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200208125816.14954-21-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/cpu64.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/target/arm/cpu64.c b/target/arm/cpu64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpu64.c +++ b/target/arm/cpu64.c @@ -XXX,XX +XXX,XX @@ static void aarch64_max_initfn(Object *obj) t = FIELD_DP64(t, ID_AA64MMFR1, PAN, 2); /* ATS1E1 */ cpu->isar.id_aa64mmfr1 = t; + t = cpu->isar.id_aa64mmfr2; + t = FIELD_DP64(t, ID_AA64MMFR2, UAO, 1); + cpu->isar.id_aa64mmfr2 = t; + /* Replicate the same data to the 32-bit id registers. */ u = cpu->isar.id_isar5; u = FIELD_DP32(u, ID_ISAR5, AES, 2); /* AES + PMULL */ -- 2.20.1 From: Guenter Roeck <linux@roeck-us.net> Initialize EHCI controllers on AST2400 and AST2500 using the existing TYPE_PLATFORM_EHCI. After this change, booting ast2500-evb into Linux successfully instantiates a USB interface. ehci-platform 1e6a3000.usb: EHCI Host Controller ehci-platform 1e6a3000.usb: new USB bus registered, assigned bus number 1 ehci-platform 1e6a3000.usb: irq 21, io mem 0x1e6a3000 ehci-platform 1e6a3000.usb: USB 2.0 started, EHCI 1.00 usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 5.05 usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 usb usb1: Product: EHCI Host Controller Signed-off-by: Guenter Roeck <linux@roeck-us.net> Reviewed-by: Cédric Le Goater <clg@kaod.org> Reviewed-by: Joel Stanley <joel@jms.id.au> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Message-id: 20200206183437.3979-1-linux@roeck-us.net Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- include/hw/arm/aspeed_soc.h | 6 ++++++ hw/arm/aspeed_soc.c | 25 +++++++++++++++++++++++++ 2 files changed, 31 insertions(+) diff --git a/include/hw/arm/aspeed_soc.h b/include/hw/arm/aspeed_soc.h index XXXXXXX..XXXXXXX 100644 --- a/include/hw/arm/aspeed_soc.h +++ b/include/hw/arm/aspeed_soc.h @@ -XXX,XX +XXX,XX @@ #include "target/arm/cpu.h" #include "hw/gpio/aspeed_gpio.h" #include "hw/sd/aspeed_sdhci.h" +#include "hw/usb/hcd-ehci.h" #define ASPEED_SPIS_NUM 2 +#define ASPEED_EHCIS_NUM 2 #define ASPEED_WDTS_NUM 4 #define ASPEED_CPUS_NUM 2 #define ASPEED_MACS_NUM 4 @@ -XXX,XX +XXX,XX @@ typedef struct AspeedSoCState { AspeedXDMAState xdma; AspeedSMCState fmc; AspeedSMCState spi[ASPEED_SPIS_NUM]; + EHCISysBusState ehci[ASPEED_EHCIS_NUM]; AspeedSDMCState sdmc; AspeedWDTState wdt[ASPEED_WDTS_NUM]; FTGMAC100State ftgmac100[ASPEED_MACS_NUM]; @@ -XXX,XX +XXX,XX @@ typedef struct AspeedSoCClass { uint32_t silicon_rev; uint64_t sram_size; int spis_num; + int ehcis_num; int wdts_num; int macs_num; const int *irqmap; @@ -XXX,XX +XXX,XX @@ enum { ASPEED_FMC, ASPEED_SPI1, ASPEED_SPI2, + ASPEED_EHCI1, + ASPEED_EHCI2, ASPEED_VIC, ASPEED_SDMC, ASPEED_SCU, diff --git a/hw/arm/aspeed_soc.c b/hw/arm/aspeed_soc.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/aspeed_soc.c +++ b/hw/arm/aspeed_soc.c @@ -XXX,XX +XXX,XX @@ static const hwaddr aspeed_soc_ast2400_memmap[] = { [ASPEED_IOMEM] = 0x1E600000, [ASPEED_FMC] = 0x1E620000, [ASPEED_SPI1] = 0x1E630000, + [ASPEED_EHCI1] = 0x1E6A1000, [ASPEED_VIC] = 0x1E6C0000, [ASPEED_SDMC] = 0x1E6E0000, [ASPEED_SCU] = 0x1E6E2000, @@ -XXX,XX +XXX,XX @@ static const hwaddr aspeed_soc_ast2500_memmap[] = { [ASPEED_FMC] = 0x1E620000, [ASPEED_SPI1] = 0x1E630000, [ASPEED_SPI2] = 0x1E631000, + [ASPEED_EHCI1] = 0x1E6A1000, + [ASPEED_EHCI2] = 0x1E6A3000, [ASPEED_VIC] = 0x1E6C0000, [ASPEED_SDMC] = 0x1E6E0000, [ASPEED_SCU] = 0x1E6E2000, @@ -XXX,XX +XXX,XX @@ static const int aspeed_soc_ast2400_irqmap[] = { [ASPEED_UART5] = 10, [ASPEED_VUART] = 8, [ASPEED_FMC] = 19, + [ASPEED_EHCI1] = 5, + [ASPEED_EHCI2] = 13, [ASPEED_SDMC] = 0, [ASPEED_SCU] = 21, [ASPEED_ADC] = 31, @@ -XXX,XX +XXX,XX @@ static void aspeed_soc_init(Object *obj) sizeof(s->spi[i]), typename); } + for (i = 0; i < sc->ehcis_num; i++) { + sysbus_init_child_obj(obj, "ehci[*]", OBJECT(&s->ehci[i]), + sizeof(s->ehci[i]), TYPE_PLATFORM_EHCI); + } + snprintf(typename, sizeof(typename), "aspeed.sdmc-%s", socname); sysbus_init_child_obj(obj, "sdmc", OBJECT(&s->sdmc), sizeof(s->sdmc), typename); @@ -XXX,XX +XXX,XX @@ static void aspeed_soc_realize(DeviceState *dev, Error **errp) s->spi[i].ctrl->flash_window_base); } + /* EHCI */ + for (i = 0; i < sc->ehcis_num; i++) { + object_property_set_bool(OBJECT(&s->ehci[i]), true, "realized", &err); + if (err) { + error_propagate(errp, err); + return; + } + sysbus_mmio_map(SYS_BUS_DEVICE(&s->ehci[i]), 0, + sc->memmap[ASPEED_EHCI1 + i]); + sysbus_connect_irq(SYS_BUS_DEVICE(&s->ehci[i]), 0, + aspeed_soc_get_irq(s, ASPEED_EHCI1 + i)); + } + /* SDMC - SDRAM Memory Controller */ object_property_set_bool(OBJECT(&s->sdmc), true, "realized", &err); if (err) { @@ -XXX,XX +XXX,XX @@ static void aspeed_soc_ast2400_class_init(ObjectClass *oc, void *data) sc->silicon_rev = AST2400_A1_SILICON_REV; sc->sram_size = 0x8000; sc->spis_num = 1; + sc->ehcis_num = 1; sc->wdts_num = 2; sc->macs_num = 2; sc->irqmap = aspeed_soc_ast2400_irqmap; @@ -XXX,XX +XXX,XX @@ static void aspeed_soc_ast2500_class_init(ObjectClass *oc, void *data) sc->silicon_rev = AST2500_A1_SILICON_REV; sc->sram_size = 0x9000; sc->spis_num = 2; + sc->ehcis_num = 2; sc->wdts_num = 3; sc->macs_num = 2; sc->irqmap = aspeed_soc_ast2500_irqmap; -- 2.20.1 From: Guenter Roeck <linux@roeck-us.net> Initialize EHCI controllers on AST2600 using the existing TYPE_PLATFORM_EHCI. After this change, booting ast2600-evb into Linux successfully instantiates a USB interface after the necessary changes are made to its devicetree files. ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver ehci-platform: EHCI generic platform driver ehci-platform 1e6a3000.usb: EHCI Host Controller ehci-platform 1e6a3000.usb: new USB bus registered, assigned bus number 1 ehci-platform 1e6a3000.usb: irq 25, io mem 0x1e6a3000 ehci-platform 1e6a3000.usb: USB 2.0 started, EHCI 1.00 usb usb1: Manufacturer: Linux 5.5.0-09825-ga0802f2d0ef5-dirty ehci_hcd usb 1-1: new high-speed USB device number 2 using ehci-platform Reviewed-by: Cédric Le Goater <clg@kaod.org> Signed-off-by: Guenter Roeck <linux@roeck-us.net> Reviewed-by: Niek Linnenbank <nieklinnenbank@gmail.com> Message-id: 20200207174548.9087-1-linux@roeck-us.net Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- hw/arm/aspeed_ast2600.c | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) diff --git a/hw/arm/aspeed_ast2600.c b/hw/arm/aspeed_ast2600.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/aspeed_ast2600.c +++ b/hw/arm/aspeed_ast2600.c @@ -XXX,XX +XXX,XX @@ static const hwaddr aspeed_soc_ast2600_memmap[] = { [ASPEED_FMC] = 0x1E620000, [ASPEED_SPI1] = 0x1E630000, [ASPEED_SPI2] = 0x1E641000, + [ASPEED_EHCI1] = 0x1E6A1000, + [ASPEED_EHCI2] = 0x1E6A3000, [ASPEED_MII1] = 0x1E650000, [ASPEED_MII2] = 0x1E650008, [ASPEED_MII3] = 0x1E650010, @@ -XXX,XX +XXX,XX @@ static const int aspeed_soc_ast2600_irqmap[] = { [ASPEED_ADC] = 78, [ASPEED_XDMA] = 6, [ASPEED_SDHCI] = 43, + [ASPEED_EHCI1] = 5, + [ASPEED_EHCI2] = 9, [ASPEED_EMMC] = 15, [ASPEED_GPIO] = 40, [ASPEED_GPIO_1_8V] = 11, @@ -XXX,XX +XXX,XX @@ static void aspeed_soc_ast2600_init(Object *obj) sizeof(s->spi[i]), typename); } + for (i = 0; i < sc->ehcis_num; i++) { + sysbus_init_child_obj(obj, "ehci[*]", OBJECT(&s->ehci[i]), + sizeof(s->ehci[i]), TYPE_PLATFORM_EHCI); + } + snprintf(typename, sizeof(typename), "aspeed.sdmc-%s", socname); sysbus_init_child_obj(obj, "sdmc", OBJECT(&s->sdmc), sizeof(s->sdmc), typename); @@ -XXX,XX +XXX,XX @@ static void aspeed_soc_ast2600_realize(DeviceState *dev, Error **errp) s->spi[i].ctrl->flash_window_base); } + /* EHCI */ + for (i = 0; i < sc->ehcis_num; i++) { + object_property_set_bool(OBJECT(&s->ehci[i]), true, "realized", &err); + if (err) { + error_propagate(errp, err); + return; + } + sysbus_mmio_map(SYS_BUS_DEVICE(&s->ehci[i]), 0, + sc->memmap[ASPEED_EHCI1 + i]); + sysbus_connect_irq(SYS_BUS_DEVICE(&s->ehci[i]), 0, + aspeed_soc_get_irq(s, ASPEED_EHCI1 + i)); + } + /* SDMC - SDRAM Memory Controller */ object_property_set_bool(OBJECT(&s->sdmc), true, "realized", &err); if (err) { @@ -XXX,XX +XXX,XX @@ static void aspeed_soc_ast2600_class_init(ObjectClass *oc, void *data) sc->silicon_rev = AST2600_A0_SILICON_REV; sc->sram_size = 0x10000; sc->spis_num = 2; + sc->ehcis_num = 2; sc->wdts_num = 4; sc->macs_num = 4; sc->irqmap = aspeed_soc_ast2600_irqmap; -- 2.20.1 From: Chen Qun <kuhn.chenqun@huawei.com> It's easy to reproduce as follow: virsh qemu-monitor-command vm1 --pretty '{"execute": "device-list-properties", "arguments":{"typename":"exynos4210.uart"}}' ASAN shows memory leak stack: #1 0xfffd896d71cb in g_malloc0 (/lib64/libglib-2.0.so.0+0x571cb) #2 0xaaad270beee3 in timer_new_full /qemu/include/qemu/timer.h:530 #3 0xaaad270beee3 in timer_new /qemu/include/qemu/timer.h:551 #4 0xaaad270beee3 in timer_new_ns /qemu/include/qemu/timer.h:569 #5 0xaaad270beee3 in exynos4210_uart_init /qemu/hw/char/exynos4210_uart.c:677 #6 0xaaad275c8f4f in object_initialize_with_type /qemu/qom/object.c:516 #7 0xaaad275c91bb in object_new_with_type /qemu/qom/object.c:684 #8 0xaaad2755df2f in qmp_device_list_properties /qemu/qom/qom-qmp-cmds.c:152 Reported-by: Euler Robot <euler.robot@huawei.com> Signed-off-by: Chen Qun <kuhn.chenqun@huawei.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Message-id: 20200213025603.149432-1-kuhn.chenqun@huawei.com Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- hw/char/exynos4210_uart.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/hw/char/exynos4210_uart.c b/hw/char/exynos4210_uart.c index XXXXXXX..XXXXXXX 100644 --- a/hw/char/exynos4210_uart.c +++ b/hw/char/exynos4210_uart.c @@ -XXX,XX +XXX,XX @@ static void exynos4210_uart_init(Object *obj) SysBusDevice *dev = SYS_BUS_DEVICE(obj); Exynos4210UartState *s = EXYNOS4210_UART(dev); - s->fifo_timeout_timer = timer_new_ns(QEMU_CLOCK_VIRTUAL, - exynos4210_uart_timeout_int, s); s->wordtime = NANOSECONDS_PER_SECOND * 10 / 9600; /* memory mapping */ @@ -XXX,XX +XXX,XX @@ static void exynos4210_uart_realize(DeviceState *dev, Error **errp) { Exynos4210UartState *s = EXYNOS4210_UART(dev); + s->fifo_timeout_timer = timer_new_ns(QEMU_CLOCK_VIRTUAL, + exynos4210_uart_timeout_int, s); + qemu_chr_fe_set_handlers(&s->chr, exynos4210_uart_can_receive, exynos4210_uart_receive, exynos4210_uart_event, NULL, s, NULL, true); -- 2.20.1 From: Philippe Mathieu-Daudé <f4bug@amsat.org> When booting without device tree, the Linux kernels uses the $R1 register to determine the machine type. The list of values is registered at [1]. There are two entries for the Raspberry Pi: - https://www.arm.linux.org.uk/developer/machines/list.php?mid=3138 name: MACH_TYPE_BCM2708 value: 0xc42 (3138) status: Active, not mainlined date: 15 Oct 2010 - https://www.arm.linux.org.uk/developer/machines/list.php?mid=4828 name: MACH_TYPE_BCM2835 value: 4828 status: Active, mainlined date: 6 Dec 2013 QEMU always used the non-mainlined type MACH_TYPE_BCM2708. The value 0xc43 is registered to 'MX51_GGC' (processor i.MX51), and 0xc44 to 'Western Digital Sharespace NAS' (processor Marvell 88F5182). The Raspberry Pi foundation bootloader only sets the BCM2708 machine type, see [2] or [3]: 133 9: 134 mov r0, #0 135 ldr r1, =3138 @ BCM2708 machine id 136 ldr r2, atags @ ATAGS 137 bx r4 U-Boot only uses MACH_TYPE_BCM2708 (see [4]): 25 /* 26 * 2835 is a SKU in a series for which the 2708 is the first or primary SoC, 27 * so 2708 has historically been used rather than a dedicated 2835 ID. 28 * 29 * We don't define a machine type for bcm2709/bcm2836 since the RPi Foundation 30 * chose to use someone else's previously registered machine ID (3139, MX51_GGC) 31 * rather than obtaining a valid ID:-/ 32 * 33 * For the bcm2837, hopefully a machine type is not needed, since everything 34 * is DT. 35 */ While the definition MACH_BCM2709 with value 0xc43 was introduced in a commit described "Add 2709 platform for Raspberry Pi 2" out of the mainline Linux kernel, it does not seem used, and the platform is introduced with Device Tree support anyway (see [5] and [6]). Remove the unused values (0xc43 introduced in commit 1df7d1f9303aef "raspi: add raspberry pi 2 machine" and 0xc44 in commit bade58166f4 "raspi: Raspberry Pi 3 support"), keeping only MACH_TYPE_BCM2708. [1] https://www.arm.linux.org.uk/developer/machines/ [2] https://github.com/raspberrypi/tools/blob/920c7ed2e/armstubs/armstub7.S#L135 [3] https://github.com/raspberrypi/tools/blob/49719d554/armstubs/armstub7.S#L64 [4] https://gitlab.denx.de/u-boot/u-boot/blob/v2015.04/include/configs/rpi-common.h#L18 [5] https://github.com/raspberrypi/linux/commit/d9fac63adac#diff-6722037d79570df5b392a49e0e006573R526 [6] http://lists.infradead.org/pipermail/linux-rpi-kernel/2015-February/001268.html Cc: Zoltán Baldaszti <bztemail@gmail.com> Cc: Pekka Enberg <penberg@iki.fi> Cc: Stephen Warren <swarren@nvidia.com> Cc: Kshitij Soni <kshitij.soni@broadcom.com> Cc: Michael Chan <michael.chan@broadcom.com> Cc: Andrew Baumann <Andrew.Baumann@microsoft.com> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Reviewed-by: Niek Linnenbank <nieklinnenbank@gmail.com> Message-id: 20200208165645.15657-2-f4bug@amsat.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- hw/arm/raspi.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/hw/arm/raspi.c b/hw/arm/raspi.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/raspi.c +++ b/hw/arm/raspi.c @@ -XXX,XX +XXX,XX @@ #define FIRMWARE_ADDR_3 0x80000 /* Pi 3 loads kernel.img here by default */ #define SPINTABLE_ADDR 0xd8 /* Pi 3 bootloader spintable */ -/* Table of Linux board IDs for different Pi versions */ -static const int raspi_boardid[] = {[1] = 0xc42, [2] = 0xc43, [3] = 0xc44}; +/* Registered machine type (matches RPi Foundation bootloader and U-Boot) */ +#define MACH_TYPE_BCM2708 3138 typedef struct RasPiState { BCM283XState soc; @@ -XXX,XX +XXX,XX @@ static void setup_boot(MachineState *machine, int version, size_t ram_size) static struct arm_boot_info binfo; int r; - binfo.board_id = raspi_boardid[version]; + binfo.board_id = MACH_TYPE_BCM2708; binfo.ram_size = ram_size; binfo.nb_cpus = machine->smp.cpus; -- 2.20.1 From: Philippe Mathieu-Daudé <f4bug@amsat.org> We hardcode the board revision as 0xa21041 for the raspi2, and 0xa02082 for the raspi3: 166 static void raspi_init(MachineState *machine, int version) 167 { ... 194 int board_rev = version == 3 ? 0xa02082 : 0xa21041; These revision codes are for the 2B and 3B models, see: https://www.raspberrypi.org/documentation/hardware/raspberrypi/revision-codes/README.md Correct the board description. Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Message-id: 20200208165645.15657-3-f4bug@amsat.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- hw/arm/raspi.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/hw/arm/raspi.c b/hw/arm/raspi.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/raspi.c +++ b/hw/arm/raspi.c @@ -XXX,XX +XXX,XX @@ static void raspi2_init(MachineState *machine) static void raspi2_machine_init(MachineClass *mc) { - mc->desc = "Raspberry Pi 2"; + mc->desc = "Raspberry Pi 2B"; mc->init = raspi2_init; mc->block_default_type = IF_SD; mc->no_parallel = 1; @@ -XXX,XX +XXX,XX @@ static void raspi3_init(MachineState *machine) static void raspi3_machine_init(MachineClass *mc) { - mc->desc = "Raspberry Pi 3"; + mc->desc = "Raspberry Pi 3B"; mc->init = raspi3_init; mc->block_default_type = IF_SD; mc->no_parallel = 1; -- 2.20.1 From: Philippe Mathieu-Daudé <f4bug@amsat.org> The board revision encode the board version. Add a helper to extract the version, and use it. Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Message-id: 20200208165645.15657-4-f4bug@amsat.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- hw/arm/raspi.c | 31 +++++++++++++++++++++++++++---- 1 file changed, 27 insertions(+), 4 deletions(-) diff --git a/hw/arm/raspi.c b/hw/arm/raspi.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/raspi.c +++ b/hw/arm/raspi.c @@ -XXX,XX +XXX,XX @@ #include "qapi/error.h" #include "cpu.h" #include "hw/arm/bcm2836.h" +#include "hw/registerfields.h" #include "qemu/error-report.h" #include "hw/boards.h" #include "hw/loader.h" @@ -XXX,XX +XXX,XX @@ typedef struct RasPiState { MemoryRegion ram; } RasPiState; +/* + * Board revision codes: + * www.raspberrypi.org/documentation/hardware/raspberrypi/revision-codes/ + */ +FIELD(REV_CODE, REVISION, 0, 4); +FIELD(REV_CODE, TYPE, 4, 8); +FIELD(REV_CODE, PROCESSOR, 12, 4); +FIELD(REV_CODE, MANUFACTURER, 16, 4); +FIELD(REV_CODE, MEMORY_SIZE, 20, 3); +FIELD(REV_CODE, STYLE, 23, 1); + +static int board_processor_id(uint32_t board_rev) +{ + assert(FIELD_EX32(board_rev, REV_CODE, STYLE)); /* Only new style */ + return FIELD_EX32(board_rev, REV_CODE, PROCESSOR); +} + +static int board_version(uint32_t board_rev) +{ + return board_processor_id(board_rev) + 1; +} + static void write_smpboot(ARMCPU *cpu, const struct arm_boot_info *info) { static const uint32_t smpboot[] = { @@ -XXX,XX +XXX,XX @@ static void setup_boot(MachineState *machine, int version, size_t ram_size) arm_load_kernel(ARM_CPU(first_cpu), machine, &binfo); } -static void raspi_init(MachineState *machine, int version) +static void raspi_init(MachineState *machine, uint32_t board_rev) { RasPiState *s = g_new0(RasPiState, 1); + int version = board_version(board_rev); uint32_t vcram_size; DriveInfo *di; BlockBackend *blk; @@ -XXX,XX +XXX,XX @@ static void raspi_init(MachineState *machine, int version) /* Setup the SOC */ object_property_add_const_link(OBJECT(&s->soc), "ram", OBJECT(&s->ram), &error_abort); - int board_rev = version == 3 ? 0xa02082 : 0xa21041; object_property_set_int(OBJECT(&s->soc), board_rev, "board-rev", &error_abort); object_property_set_bool(OBJECT(&s->soc), true, "realized", &error_abort); @@ -XXX,XX +XXX,XX @@ static void raspi_init(MachineState *machine, int version) static void raspi2_init(MachineState *machine) { - raspi_init(machine, 2); + raspi_init(machine, 0xa21041); } static void raspi2_machine_init(MachineClass *mc) @@ -XXX,XX +XXX,XX @@ DEFINE_MACHINE("raspi2", raspi2_machine_init) #ifdef TARGET_AARCH64 static void raspi3_init(MachineState *machine) { - raspi_init(machine, 3); + raspi_init(machine, 0xa02082); } static void raspi3_machine_init(MachineClass *mc) -- 2.20.1 From: Philippe Mathieu-Daudé <f4bug@amsat.org> The board revision encode the amount of RAM. Add a helper to extract the RAM size, and use it. Since the amount of RAM is fixed (it is impossible to physically modify to have more or less RAM), do not allow sizes different than the one anounced by the manufacturer. Acked-by: Igor Mammedov <imammedo@redhat.com> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Message-id: 20200208165645.15657-5-f4bug@amsat.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- hw/arm/raspi.c | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-) diff --git a/hw/arm/raspi.c b/hw/arm/raspi.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/raspi.c +++ b/hw/arm/raspi.c @@ -XXX,XX +XXX,XX @@ #include "qemu/osdep.h" #include "qemu/units.h" +#include "qemu/cutils.h" #include "qapi/error.h" #include "cpu.h" #include "hw/arm/bcm2836.h" @@ -XXX,XX +XXX,XX @@ FIELD(REV_CODE, MANUFACTURER, 16, 4); FIELD(REV_CODE, MEMORY_SIZE, 20, 3); FIELD(REV_CODE, STYLE, 23, 1); +static uint64_t board_ram_size(uint32_t board_rev) +{ + assert(FIELD_EX32(board_rev, REV_CODE, STYLE)); /* Only new style */ + return 256 * MiB << FIELD_EX32(board_rev, REV_CODE, MEMORY_SIZE); +} + static int board_processor_id(uint32_t board_rev) { assert(FIELD_EX32(board_rev, REV_CODE, STYLE)); /* Only new style */ @@ -XXX,XX +XXX,XX @@ static void raspi_init(MachineState *machine, uint32_t board_rev) { RasPiState *s = g_new0(RasPiState, 1); int version = board_version(board_rev); + uint64_t ram_size = board_ram_size(board_rev); uint32_t vcram_size; DriveInfo *di; BlockBackend *blk; BusState *bus; DeviceState *carddev; - if (machine->ram_size > 1 * GiB) { - error_report("Requested ram size is too large for this machine: " - "maximum is 1GB"); + if (machine->ram_size != ram_size) { + char *size_str = size_to_str(ram_size); + error_report("Invalid RAM size, should be %s", size_str); + g_free(size_str); exit(1); } -- 2.20.1 From: Philippe Mathieu-Daudé <f4bug@amsat.org> The board revision encode the processor type. Add a helper to extract the type, and use it. Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Message-id: 20200208165645.15657-6-f4bug@amsat.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- hw/arm/raspi.c | 18 ++++++++++++++++-- 1 file changed, 16 insertions(+), 2 deletions(-) diff --git a/hw/arm/raspi.c b/hw/arm/raspi.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/raspi.c +++ b/hw/arm/raspi.c @@ -XXX,XX +XXX,XX @@ static int board_version(uint32_t board_rev) return board_processor_id(board_rev) + 1; } +static const char *board_soc_type(uint32_t board_rev) +{ + static const char *soc_types[] = { + NULL, TYPE_BCM2836, TYPE_BCM2837, + }; + int proc_id = board_processor_id(board_rev); + + if (proc_id >= ARRAY_SIZE(soc_types) || !soc_types[proc_id]) { + error_report("Unsupported processor id '%d' (board revision: 0x%x)", + proc_id, board_rev); + exit(1); + } + return soc_types[proc_id]; +} + static void write_smpboot(ARMCPU *cpu, const struct arm_boot_info *info) { static const uint32_t smpboot[] = { @@ -XXX,XX +XXX,XX @@ static void raspi_init(MachineState *machine, uint32_t board_rev) } object_initialize_child(OBJECT(machine), "soc", &s->soc, sizeof(s->soc), - version == 3 ? TYPE_BCM2837 : TYPE_BCM2836, - &error_abort, NULL); + board_soc_type(board_rev), &error_abort, NULL); /* Allocate and map RAM */ memory_region_allocate_system_memory(&s->ram, OBJECT(machine), "ram", -- 2.20.1 From: Philippe Mathieu-Daudé <f4bug@amsat.org> There is no point in creating the SoC object before allocating the RAM. Move the call to keep all the SoC-related calls together. Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Acked-by: Igor Mammedov <imammedo@redhat.com> Message-id: 20200208165645.15657-7-f4bug@amsat.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- hw/arm/raspi.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/hw/arm/raspi.c b/hw/arm/raspi.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/raspi.c +++ b/hw/arm/raspi.c @@ -XXX,XX +XXX,XX @@ static void raspi_init(MachineState *machine, uint32_t board_rev) exit(1); } - object_initialize_child(OBJECT(machine), "soc", &s->soc, sizeof(s->soc), - board_soc_type(board_rev), &error_abort, NULL); - /* Allocate and map RAM */ memory_region_allocate_system_memory(&s->ram, OBJECT(machine), "ram", machine->ram_size); @@ -XXX,XX +XXX,XX @@ static void raspi_init(MachineState *machine, uint32_t board_rev) memory_region_add_subregion_overlap(get_system_memory(), 0, &s->ram, 0); /* Setup the SOC */ + object_initialize_child(OBJECT(machine), "soc", &s->soc, sizeof(s->soc), + board_soc_type(board_rev), &error_abort, NULL); object_property_add_const_link(OBJECT(&s->soc), "ram", OBJECT(&s->ram), &error_abort); object_property_set_int(OBJECT(&s->soc), board_rev, "board-rev", -- 2.20.1 From: Philippe Mathieu-Daudé <f4bug@amsat.org> QOM'ify RaspiMachineState. Now machines inherit of RaspiMachineClass. Cc: Igor Mammedov <imammedo@redhat.com> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Acked-by: Igor Mammedov <imammedo@redhat.com> Message-id: 20200208165645.15657-8-f4bug@amsat.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- hw/arm/raspi.c | 56 +++++++++++++++++++++++++++++++++++++++++++------- 1 file changed, 49 insertions(+), 7 deletions(-) diff --git a/hw/arm/raspi.c b/hw/arm/raspi.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/raspi.c +++ b/hw/arm/raspi.c @@ -XXX,XX +XXX,XX @@ /* Registered machine type (matches RPi Foundation bootloader and U-Boot) */ #define MACH_TYPE_BCM2708 3138 -typedef struct RasPiState { +typedef struct RaspiMachineState { + /*< private >*/ + MachineState parent_obj; + /*< public >*/ BCM283XState soc; MemoryRegion ram; -} RasPiState; +} RaspiMachineState; + +typedef struct RaspiMachineClass { + /*< private >*/ + MachineClass parent_obj; + /*< public >*/ +} RaspiMachineClass; + +#define TYPE_RASPI_MACHINE MACHINE_TYPE_NAME("raspi-common") +#define RASPI_MACHINE(obj) \ + OBJECT_CHECK(RaspiMachineState, (obj), TYPE_RASPI_MACHINE) + +#define RASPI_MACHINE_CLASS(klass) \ + OBJECT_CLASS_CHECK(RaspiMachineClass, (klass), TYPE_RASPI_MACHINE) +#define RASPI_MACHINE_GET_CLASS(obj) \ + OBJECT_GET_CLASS(RaspiMachineClass, (obj), TYPE_RASPI_MACHINE) /* * Board revision codes: @@ -XXX,XX +XXX,XX @@ static void setup_boot(MachineState *machine, int version, size_t ram_size) static void raspi_init(MachineState *machine, uint32_t board_rev) { - RasPiState *s = g_new0(RasPiState, 1); + RaspiMachineState *s = RASPI_MACHINE(machine); int version = board_version(board_rev); uint64_t ram_size = board_ram_size(board_rev); uint32_t vcram_size; @@ -XXX,XX +XXX,XX @@ static void raspi2_init(MachineState *machine) raspi_init(machine, 0xa21041); } -static void raspi2_machine_init(MachineClass *mc) +static void raspi2_machine_class_init(ObjectClass *oc, void *data) { + MachineClass *mc = MACHINE_CLASS(oc); + mc->desc = "Raspberry Pi 2B"; mc->init = raspi2_init; mc->block_default_type = IF_SD; @@ -XXX,XX +XXX,XX @@ static void raspi2_machine_init(MachineClass *mc) mc->default_ram_size = 1 * GiB; mc->ignore_memory_transaction_failures = true; }; -DEFINE_MACHINE("raspi2", raspi2_machine_init) #ifdef TARGET_AARCH64 static void raspi3_init(MachineState *machine) @@ -XXX,XX +XXX,XX @@ static void raspi3_init(MachineState *machine) raspi_init(machine, 0xa02082); } -static void raspi3_machine_init(MachineClass *mc) +static void raspi3_machine_class_init(ObjectClass *oc, void *data) { + MachineClass *mc = MACHINE_CLASS(oc); + mc->desc = "Raspberry Pi 3B"; mc->init = raspi3_init; mc->block_default_type = IF_SD; @@ -XXX,XX +XXX,XX @@ static void raspi3_machine_init(MachineClass *mc) mc->default_cpus = BCM283X_NCPUS; mc->default_ram_size = 1 * GiB; } -DEFINE_MACHINE("raspi3", raspi3_machine_init) #endif + +static const TypeInfo raspi_machine_types[] = { + { + .name = MACHINE_TYPE_NAME("raspi2"), + .parent = TYPE_RASPI_MACHINE, + .class_init = raspi2_machine_class_init, +#ifdef TARGET_AARCH64 + }, { + .name = MACHINE_TYPE_NAME("raspi3"), + .parent = TYPE_RASPI_MACHINE, + .class_init = raspi3_machine_class_init, +#endif + }, { + .name = TYPE_RASPI_MACHINE, + .parent = TYPE_MACHINE, + .instance_size = sizeof(RaspiMachineState), + .class_size = sizeof(RaspiMachineClass), + .abstract = true, + } +}; + +DEFINE_TYPES(raspi_machine_types) -- 2.20.1 From: Philippe Mathieu-Daudé <f4bug@amsat.org> We want to have a common class_init(). The only value that matters (and changes) is the board revision. Pass the board_rev as class_data to class_init(). Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Message-id: 20200208165645.15657-9-f4bug@amsat.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- hw/arm/raspi.c | 17 ++++++++++++++--- 1 file changed, 14 insertions(+), 3 deletions(-) diff --git a/hw/arm/raspi.c b/hw/arm/raspi.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/raspi.c +++ b/hw/arm/raspi.c @@ -XXX,XX +XXX,XX @@ typedef struct RaspiMachineClass { /*< private >*/ MachineClass parent_obj; /*< public >*/ + uint32_t board_rev; } RaspiMachineClass; #define TYPE_RASPI_MACHINE MACHINE_TYPE_NAME("raspi-common") @@ -XXX,XX +XXX,XX @@ static void setup_boot(MachineState *machine, int version, size_t ram_size) arm_load_kernel(ARM_CPU(first_cpu), machine, &binfo); } -static void raspi_init(MachineState *machine, uint32_t board_rev) +static void raspi_init(MachineState *machine) { + RaspiMachineClass *mc = RASPI_MACHINE_GET_CLASS(machine); RaspiMachineState *s = RASPI_MACHINE(machine); + uint32_t board_rev = mc->board_rev; int version = board_version(board_rev); uint64_t ram_size = board_ram_size(board_rev); uint32_t vcram_size; @@ -XXX,XX +XXX,XX @@ static void raspi_init(MachineState *machine, uint32_t board_rev) static void raspi2_init(MachineState *machine) { - raspi_init(machine, 0xa21041); + raspi_init(machine); } static void raspi2_machine_class_init(ObjectClass *oc, void *data) { MachineClass *mc = MACHINE_CLASS(oc); + RaspiMachineClass *rmc = RASPI_MACHINE_CLASS(oc); + uint32_t board_rev = (uint32_t)(uintptr_t)data; + rmc->board_rev = board_rev; mc->desc = "Raspberry Pi 2B"; mc->init = raspi2_init; mc->block_default_type = IF_SD; @@ -XXX,XX +XXX,XX @@ static void raspi2_machine_class_init(ObjectClass *oc, void *data) #ifdef TARGET_AARCH64 static void raspi3_init(MachineState *machine) { - raspi_init(machine, 0xa02082); + raspi_init(machine); } static void raspi3_machine_class_init(ObjectClass *oc, void *data) { MachineClass *mc = MACHINE_CLASS(oc); + RaspiMachineClass *rmc = RASPI_MACHINE_CLASS(oc); + uint32_t board_rev = (uint32_t)(uintptr_t)data; + rmc->board_rev = board_rev; mc->desc = "Raspberry Pi 3B"; mc->init = raspi3_init; mc->block_default_type = IF_SD; @@ -XXX,XX +XXX,XX @@ static const TypeInfo raspi_machine_types[] = { .name = MACHINE_TYPE_NAME("raspi2"), .parent = TYPE_RASPI_MACHINE, .class_init = raspi2_machine_class_init, + .class_data = (void *)0xa21041, #ifdef TARGET_AARCH64 }, { .name = MACHINE_TYPE_NAME("raspi3"), .parent = TYPE_RASPI_MACHINE, .class_init = raspi3_machine_class_init, + .class_data = (void *)0xa02082, #endif }, { .name = TYPE_RASPI_MACHINE, -- 2.20.1 From: Philippe Mathieu-Daudé <f4bug@amsat.org> raspi_machine_init() access to board_rev via RaspiMachineClass. raspi2_init() and raspi3_init() do nothing. Call raspi_machine_init directly. Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Reviewed-by: Igor Mammedov <imammedo@redhat.com> Message-id: 20200208165645.15657-10-f4bug@amsat.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- hw/arm/raspi.c | 16 +++------------- 1 file changed, 3 insertions(+), 13 deletions(-) diff --git a/hw/arm/raspi.c b/hw/arm/raspi.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/raspi.c +++ b/hw/arm/raspi.c @@ -XXX,XX +XXX,XX @@ static void setup_boot(MachineState *machine, int version, size_t ram_size) arm_load_kernel(ARM_CPU(first_cpu), machine, &binfo); } -static void raspi_init(MachineState *machine) +static void raspi_machine_init(MachineState *machine) { RaspiMachineClass *mc = RASPI_MACHINE_GET_CLASS(machine); RaspiMachineState *s = RASPI_MACHINE(machine); @@ -XXX,XX +XXX,XX @@ static void raspi_init(MachineState *machine) setup_boot(machine, version, machine->ram_size - vcram_size); } -static void raspi2_init(MachineState *machine) -{ - raspi_init(machine); -} - static void raspi2_machine_class_init(ObjectClass *oc, void *data) { MachineClass *mc = MACHINE_CLASS(oc); @@ -XXX,XX +XXX,XX @@ static void raspi2_machine_class_init(ObjectClass *oc, void *data) rmc->board_rev = board_rev; mc->desc = "Raspberry Pi 2B"; - mc->init = raspi2_init; + mc->init = raspi_machine_init; mc->block_default_type = IF_SD; mc->no_parallel = 1; mc->no_floppy = 1; @@ -XXX,XX +XXX,XX @@ static void raspi2_machine_class_init(ObjectClass *oc, void *data) }; #ifdef TARGET_AARCH64 -static void raspi3_init(MachineState *machine) -{ - raspi_init(machine); -} - static void raspi3_machine_class_init(ObjectClass *oc, void *data) { MachineClass *mc = MACHINE_CLASS(oc); @@ -XXX,XX +XXX,XX @@ static void raspi3_machine_class_init(ObjectClass *oc, void *data) rmc->board_rev = board_rev; mc->desc = "Raspberry Pi 3B"; - mc->init = raspi3_init; + mc->init = raspi_machine_init; mc->block_default_type = IF_SD; mc->no_parallel = 1; mc->no_floppy = 1; -- 2.20.1 From: Philippe Mathieu-Daudé <f4bug@amsat.org> We added a helper to extract the RAM size from the board revision, and made board_rev a field of RaspiMachineClass. The class_init() can now use the helper to extract from the board revision the board-specific amount of RAM. Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Message-id: 20200208165645.15657-11-f4bug@amsat.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- hw/arm/raspi.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/hw/arm/raspi.c b/hw/arm/raspi.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/raspi.c +++ b/hw/arm/raspi.c @@ -XXX,XX +XXX,XX @@ static void raspi2_machine_class_init(ObjectClass *oc, void *data) mc->max_cpus = BCM283X_NCPUS; mc->min_cpus = BCM283X_NCPUS; mc->default_cpus = BCM283X_NCPUS; - mc->default_ram_size = 1 * GiB; + mc->default_ram_size = board_ram_size(board_rev); mc->ignore_memory_transaction_failures = true; }; @@ -XXX,XX +XXX,XX @@ static void raspi3_machine_class_init(ObjectClass *oc, void *data) mc->max_cpus = BCM283X_NCPUS; mc->min_cpus = BCM283X_NCPUS; mc->default_cpus = BCM283X_NCPUS; - mc->default_ram_size = 1 * GiB; + mc->default_ram_size = board_ram_size(board_rev); } #endif -- 2.20.1 From: Philippe Mathieu-Daudé <f4bug@amsat.org> The board revision encode the model type. Add a helper to extract the model, and use it. Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Message-id: 20200208165645.15657-12-f4bug@amsat.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- hw/arm/raspi.c | 18 ++++++++++++++++-- 1 file changed, 16 insertions(+), 2 deletions(-) diff --git a/hw/arm/raspi.c b/hw/arm/raspi.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/raspi.c +++ b/hw/arm/raspi.c @@ -XXX,XX +XXX,XX @@ static const char *board_soc_type(uint32_t board_rev) return soc_types[proc_id]; } +static const char *board_type(uint32_t board_rev) +{ + static const char *types[] = { + "A", "B", "A+", "B+", "2B", "Alpha", "CM1", NULL, "3B", "Zero", + "CM3", NULL, "Zero W", "3B+", "3A+", NULL, "CM3+", "4B", + }; + assert(FIELD_EX32(board_rev, REV_CODE, STYLE)); /* Only new style */ + int bt = FIELD_EX32(board_rev, REV_CODE, TYPE); + if (bt >= ARRAY_SIZE(types) || !types[bt]) { + return "Unknown"; + } + return types[bt]; +} + static void write_smpboot(ARMCPU *cpu, const struct arm_boot_info *info) { static const uint32_t smpboot[] = { @@ -XXX,XX +XXX,XX @@ static void raspi2_machine_class_init(ObjectClass *oc, void *data) uint32_t board_rev = (uint32_t)(uintptr_t)data; rmc->board_rev = board_rev; - mc->desc = "Raspberry Pi 2B"; + mc->desc = g_strdup_printf("Raspberry Pi %s", board_type(board_rev)); mc->init = raspi_machine_init; mc->block_default_type = IF_SD; mc->no_parallel = 1; @@ -XXX,XX +XXX,XX @@ static void raspi3_machine_class_init(ObjectClass *oc, void *data) uint32_t board_rev = (uint32_t)(uintptr_t)data; rmc->board_rev = board_rev; - mc->desc = "Raspberry Pi 3B"; + mc->desc = g_strdup_printf("Raspberry Pi %s", board_type(board_rev)); mc->init = raspi_machine_init; mc->block_default_type = IF_SD; mc->no_parallel = 1; -- 2.20.1 From: Philippe Mathieu-Daudé <f4bug@amsat.org> With the exception of the ignore_memory_transaction_failures flag set for the raspi2, both machine_class_init() methods are now identical. Merge them to keep a unique method. Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com> Reviewed-by: Igor Mammedov <imammedo@redhat.com> Message-id: 20200208165645.15657-13-f4bug@amsat.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- hw/arm/raspi.c | 31 ++++++------------------------- 1 file changed, 6 insertions(+), 25 deletions(-) diff --git a/hw/arm/raspi.c b/hw/arm/raspi.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/raspi.c +++ b/hw/arm/raspi.c @@ -XXX,XX +XXX,XX @@ static void raspi_machine_init(MachineState *machine) setup_boot(machine, version, machine->ram_size - vcram_size); } -static void raspi2_machine_class_init(ObjectClass *oc, void *data) +static void raspi_machine_class_init(ObjectClass *oc, void *data) { MachineClass *mc = MACHINE_CLASS(oc); RaspiMachineClass *rmc = RASPI_MACHINE_CLASS(oc); @@ -XXX,XX +XXX,XX @@ static void raspi2_machine_class_init(ObjectClass *oc, void *data) mc->min_cpus = BCM283X_NCPUS; mc->default_cpus = BCM283X_NCPUS; mc->default_ram_size = board_ram_size(board_rev); - mc->ignore_memory_transaction_failures = true; + if (board_version(board_rev) == 2) { + mc->ignore_memory_transaction_failures = true; + } }; -#ifdef TARGET_AARCH64 -static void raspi3_machine_class_init(ObjectClass *oc, void *data) -{ - MachineClass *mc = MACHINE_CLASS(oc); - RaspiMachineClass *rmc = RASPI_MACHINE_CLASS(oc); - uint32_t board_rev = (uint32_t)(uintptr_t)data; - - rmc->board_rev = board_rev; - mc->desc = g_strdup_printf("Raspberry Pi %s", board_type(board_rev)); - mc->init = raspi_machine_init; - mc->block_default_type = IF_SD; - mc->no_parallel = 1; - mc->no_floppy = 1; - mc->no_cdrom = 1; - mc->max_cpus = BCM283X_NCPUS; - mc->min_cpus = BCM283X_NCPUS; - mc->default_cpus = BCM283X_NCPUS; - mc->default_ram_size = board_ram_size(board_rev); -} -#endif - static const TypeInfo raspi_machine_types[] = { { .name = MACHINE_TYPE_NAME("raspi2"), .parent = TYPE_RASPI_MACHINE, - .class_init = raspi2_machine_class_init, + .class_init = raspi_machine_class_init, .class_data = (void *)0xa21041, #ifdef TARGET_AARCH64 }, { .name = MACHINE_TYPE_NAME("raspi3"), .parent = TYPE_RASPI_MACHINE, - .class_init = raspi3_machine_class_init, + .class_init = raspi_machine_class_init, .class_data = (void *)0xa02082, #endif }, { -- 2.20.1 From: Philippe Mathieu-Daudé <f4bug@amsat.org> The count of ARM cores is encoded in the board revision. Add a helper to extract the number of cores, and use it. This will be helpful when we add the Raspi0/1 that have a single core. Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Message-id: 20200208165645.15657-14-f4bug@amsat.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> [PMM: tweaked commit message as suggested by Igor] Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- hw/arm/raspi.c | 19 ++++++++++++++++--- 1 file changed, 16 insertions(+), 3 deletions(-) diff --git a/hw/arm/raspi.c b/hw/arm/raspi.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/raspi.c +++ b/hw/arm/raspi.c @@ -XXX,XX +XXX,XX @@ static const char *board_soc_type(uint32_t board_rev) return soc_types[proc_id]; } +static int cores_count(uint32_t board_rev) +{ + static const int soc_cores_count[] = { + 0, BCM283X_NCPUS, BCM283X_NCPUS, + }; + int proc_id = board_processor_id(board_rev); + + if (proc_id >= ARRAY_SIZE(soc_cores_count) || !soc_cores_count[proc_id]) { + error_report("Unsupported processor id '%d' (board revision: 0x%x)", + proc_id, board_rev); + exit(1); + } + return soc_cores_count[proc_id]; +} + static const char *board_type(uint32_t board_rev) { static const char *types[] = { @@ -XXX,XX +XXX,XX @@ static void raspi_machine_class_init(ObjectClass *oc, void *data) mc->no_parallel = 1; mc->no_floppy = 1; mc->no_cdrom = 1; - mc->max_cpus = BCM283X_NCPUS; - mc->min_cpus = BCM283X_NCPUS; - mc->default_cpus = BCM283X_NCPUS; + mc->default_cpus = mc->min_cpus = mc->max_cpus = cores_count(board_rev); mc->default_ram_size = board_ram_size(board_rev); if (board_version(board_rev) == 2) { mc->ignore_memory_transaction_failures = true; -- 2.20.1 The ARMv8.1-VMID16 extension extends the VMID from 8 bits to 16 bits: * the ID_AA64MMFR1_EL1.VMIDBits field specifies whether the VMID is 8 or 16 bits * the VMID field in VTTBR_EL2 is extended to 16 bits * VTCR_EL2.VS lets the guest specify whether to use the full 16 bits, or use the backwards-compatible 8 bits For QEMU implementing this is trivial: * we do not track VMIDs in TLB entries, so we never use the VMID field * we treat any write to VTTBR_EL2, not just a change to the VMID field bits, as a "possible VMID change" that causes us to throw away TLB entries, so that code doesn't need changing * we allow the guest to read/write the VTCR_EL2.VS bit already So all that's missing is the ID register part: report that we support VMID16 in our 'max' CPU. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200210120146.17631-1-peter.maydell@linaro.org --- target/arm/cpu64.c | 1 + 1 file changed, 1 insertion(+) diff --git a/target/arm/cpu64.c b/target/arm/cpu64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpu64.c +++ b/target/arm/cpu64.c @@ -XXX,XX +XXX,XX @@ static void aarch64_max_initfn(Object *obj) t = FIELD_DP64(t, ID_AA64MMFR1, LO, 1); t = FIELD_DP64(t, ID_AA64MMFR1, VH, 1); t = FIELD_DP64(t, ID_AA64MMFR1, PAN, 2); /* ATS1E1 */ + t = FIELD_DP64(t, ID_AA64MMFR1, VMIDBITS, 2); /* VMID16 */ cpu->isar.id_aa64mmfr1 = t; t = cpu->isar.id_aa64mmfr2; -- 2.20.1