First pullreq for arm of the 4.1 series, since I'm back from holiday now. This is mostly my M-profile FPU series and Philippe's devices.h cleanup. I have a pile of other patchsets to work through in my to-review folder, but 42 patches is definitely quite big enough to send now... thanks -- PMM The following changes since commit 413a99a92c13ec408dcf2adaa87918dc81e890c8: Add Nios II semihosting support. (2019-04-29 16:09:51 +0100) are available in the Git repository at: https://git.linaro.org/people/pmaydell/qemu-arm.git tags/pull-target-arm-20190429 for you to fetch changes up to 437cc27ddfded3bbab6afd5ac1761e0e195edba7: hw/devices: Move SMSC 91C111 declaration into a new header (2019-04-29 17:57:21 +0100) ---------------------------------------------------------------- target-arm queue: * remove "bag of random stuff" hw/devices.h header * implement FPU for Cortex-M and enable it for Cortex-M4 and -M33 * hw/dma: Compile the bcm2835_dma device as common object * configure: Remove --source-path option * hw/ssi/xilinx_spips: Avoid variable length array * hw/arm/smmuv3: Remove SMMUNotifierNode ---------------------------------------------------------------- Eric Auger (1): hw/arm/smmuv3: Remove SMMUNotifierNode Peter Maydell (28): hw/ssi/xilinx_spips: Avoid variable length array configure: Remove --source-path option target/arm: Make sure M-profile FPSCR RES0 bits are not settable hw/intc/armv7m_nvic: Allow reading of M-profile MVFR* registers target/arm: Implement dummy versions of M-profile FP-related registers target/arm: Disable most VFP sysregs for M-profile target/arm: Honour M-profile FP enable bits target/arm: Decode FP instructions for M profile target/arm: Clear CONTROL_S.SFPA in SG insn if FPU present target/arm: Handle SFPA and FPCA bits in reads and writes of CONTROL target/arm/helper: don't return early for STKOF faults during stacking target/arm: Handle floating point registers in exception entry target/arm: Implement v7m_update_fpccr() target/arm: Clear CONTROL.SFPA in BXNS and BLXNS target/arm: Clean excReturn bits when tail chaining target/arm: Allow for floating point in callee stack integrity check target/arm: Handle floating point registers in exception return target/arm: Move NS TBFLAG from bit 19 to bit 6 target/arm: Overlap VECSTRIDE and XSCALE_CPAR TB flags target/arm: Set FPCCR.S when executing M-profile floating point insns target/arm: Activate M-profile floating point context when FPCCR.ASPEN is set target/arm: New helper function arm_v7m_mmu_idx_all() target/arm: New function armv7m_nvic_set_pending_lazyfp() target/arm: Add lazy-FP-stacking support to v7m_stack_write() target/arm: Implement M-profile lazy FP state preservation target/arm: Implement VLSTM for v7M CPUs with an FPU target/arm: Implement VLLDM for v7M CPUs with an FPU target/arm: Enable FPU for Cortex-M4 and Cortex-M33 Philippe Mathieu-Daudé (13): hw/dma: Compile the bcm2835_dma device as common object hw/arm/aspeed: Use TYPE_TMP105/TYPE_PCA9552 instead of hardcoded string hw/arm/nseries: Use TYPE_TMP105 instead of hardcoded string hw/display/tc6393xb: Remove unused functions hw/devices: Move TC6393XB declarations into a new header hw/devices: Move Blizzard declarations into a new header hw/devices: Move CBus declarations into a new header hw/devices: Move Gamepad declarations into a new header hw/devices: Move TI touchscreen declarations into a new header hw/devices: Move LAN9118 declarations into a new header hw/net/ne2000-isa: Add guards to the header hw/net/lan9118: Export TYPE_LAN9118 and use it instead of hardcoded string hw/devices: Move SMSC 91C111 declaration into a new header configure | 10 +- hw/dma/Makefile.objs | 2 +- include/hw/arm/omap.h | 6 +- include/hw/arm/smmu-common.h | 8 +- include/hw/devices.h | 62 --- include/hw/display/blizzard.h | 22 ++ include/hw/display/tc6393xb.h | 24 ++ include/hw/input/gamepad.h | 19 + include/hw/input/tsc2xxx.h | 36 ++ include/hw/misc/cbus.h | 32 ++ include/hw/net/lan9118.h | 21 + include/hw/net/ne2000-isa.h | 6 + include/hw/net/smc91c111.h | 19 + include/qemu/typedefs.h | 1 - target/arm/cpu.h | 95 ++++- target/arm/helper.h | 5 + target/arm/translate.h | 3 + hw/arm/aspeed.c | 13 +- hw/arm/exynos4_boards.c | 3 +- hw/arm/gumstix.c | 2 +- hw/arm/integratorcp.c | 2 +- hw/arm/kzm.c | 2 +- hw/arm/mainstone.c | 2 +- hw/arm/mps2-tz.c | 3 +- hw/arm/mps2.c | 2 +- hw/arm/nseries.c | 7 +- hw/arm/palm.c | 2 +- hw/arm/realview.c | 3 +- hw/arm/smmu-common.c | 6 +- hw/arm/smmuv3.c | 28 +- hw/arm/stellaris.c | 2 +- hw/arm/tosa.c | 2 +- hw/arm/versatilepb.c | 2 +- hw/arm/vexpress.c | 2 +- hw/display/blizzard.c | 2 +- hw/display/tc6393xb.c | 18 +- hw/input/stellaris_input.c | 2 +- hw/input/tsc2005.c | 2 +- hw/input/tsc210x.c | 4 +- hw/intc/armv7m_nvic.c | 261 +++++++++++++ hw/misc/cbus.c | 2 +- hw/net/lan9118.c | 3 +- hw/net/smc91c111.c | 2 +- hw/ssi/xilinx_spips.c | 6 +- target/arm/cpu.c | 20 + target/arm/helper.c | 873 +++++++++++++++++++++++++++++++++++++++--- target/arm/machine.c | 16 + target/arm/translate.c | 150 +++++++- target/arm/vfp_helper.c | 8 + MAINTAINERS | 7 + 50 files changed, 1595 insertions(+), 235 deletions(-) delete mode 100644 include/hw/devices.h create mode 100644 include/hw/display/blizzard.h create mode 100644 include/hw/display/tc6393xb.h create mode 100644 include/hw/input/gamepad.h create mode 100644 include/hw/input/tsc2xxx.h create mode 100644 include/hw/misc/cbus.h create mode 100644 include/hw/net/lan9118.h create mode 100644 include/hw/net/smc91c111.h From: Eric Auger <eric.auger@redhat.com> The SMMUNotifierNode struct is not necessary and brings extra complexity so let's remove it. We now directly track the SMMUDevices which have registered IOMMU MR notifiers. This is inspired from the same transformation on intel-iommu done in commit b4a4ba0d68f50f218ee3957b6638dbee32a5eeef ("intel-iommu: remove IntelIOMMUNotifierNode") Signed-off-by: Eric Auger <eric.auger@redhat.com> Reviewed-by: Peter Xu <peterx@redhat.com> Message-id: 20190409160219.19026-1-eric.auger@redhat.com Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- include/hw/arm/smmu-common.h | 8 ++------ hw/arm/smmu-common.c | 6 +++--- hw/arm/smmuv3.c | 28 +++++++--------------------- 3 files changed, 12 insertions(+), 30 deletions(-) diff --git a/include/hw/arm/smmu-common.h b/include/hw/arm/smmu-common.h index XXXXXXX..XXXXXXX 100644 --- a/include/hw/arm/smmu-common.h +++ b/include/hw/arm/smmu-common.h @@ -XXX,XX +XXX,XX @@ typedef struct SMMUDevice { AddressSpace as; uint32_t cfg_cache_hits; uint32_t cfg_cache_misses; + QLIST_ENTRY(SMMUDevice) next; } SMMUDevice; -typedef struct SMMUNotifierNode { - SMMUDevice *sdev; - QLIST_ENTRY(SMMUNotifierNode) next; -} SMMUNotifierNode; - typedef struct SMMUPciBus { PCIBus *bus; SMMUDevice *pbdev[0]; /* Parent array is sparse, so dynamically alloc */ @@ -XXX,XX +XXX,XX @@ typedef struct SMMUState { GHashTable *iotlb; SMMUPciBus *smmu_pcibus_by_bus_num[SMMU_PCI_BUS_MAX]; PCIBus *pci_bus; - QLIST_HEAD(, SMMUNotifierNode) notifiers_list; + QLIST_HEAD(, SMMUDevice) devices_with_notifiers; uint8_t bus_num; PCIBus *primary_bus; } SMMUState; diff --git a/hw/arm/smmu-common.c b/hw/arm/smmu-common.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/smmu-common.c +++ b/hw/arm/smmu-common.c @@ -XXX,XX +XXX,XX @@ inline void smmu_inv_notifiers_mr(IOMMUMemoryRegion *mr) /* Unmap all notifiers of all mr's */ void smmu_inv_notifiers_all(SMMUState *s) { - SMMUNotifierNode *node; + SMMUDevice *sdev; - QLIST_FOREACH(node, &s->notifiers_list, next) { - smmu_inv_notifiers_mr(&node->sdev->iommu); + QLIST_FOREACH(sdev, &s->devices_with_notifiers, next) { + smmu_inv_notifiers_mr(&sdev->iommu); } } diff --git a/hw/arm/smmuv3.c b/hw/arm/smmuv3.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/smmuv3.c +++ b/hw/arm/smmuv3.c @@ -XXX,XX +XXX,XX @@ static void smmuv3_notify_iova(IOMMUMemoryRegion *mr, /* invalidate an asid/iova tuple in all mr's */ static void smmuv3_inv_notifiers_iova(SMMUState *s, int asid, dma_addr_t iova) { - SMMUNotifierNode *node; + SMMUDevice *sdev; - QLIST_FOREACH(node, &s->notifiers_list, next) { - IOMMUMemoryRegion *mr = &node->sdev->iommu; + QLIST_FOREACH(sdev, &s->devices_with_notifiers, next) { + IOMMUMemoryRegion *mr = &sdev->iommu; IOMMUNotifier *n; trace_smmuv3_inv_notifiers_iova(mr->parent_obj.name, asid, iova); @@ -XXX,XX +XXX,XX @@ static void smmuv3_notify_flag_changed(IOMMUMemoryRegion *iommu, SMMUDevice *sdev = container_of(iommu, SMMUDevice, iommu); SMMUv3State *s3 = sdev->smmu; SMMUState *s = &(s3->smmu_state); - SMMUNotifierNode *node = NULL; - SMMUNotifierNode *next_node = NULL; if (new & IOMMU_NOTIFIER_MAP) { int bus_num = pci_bus_num(sdev->bus); @@ -XXX,XX +XXX,XX @@ static void smmuv3_notify_flag_changed(IOMMUMemoryRegion *iommu, if (old == IOMMU_NOTIFIER_NONE) { trace_smmuv3_notify_flag_add(iommu->parent_obj.name); - node = g_malloc0(sizeof(*node)); - node->sdev = sdev; - QLIST_INSERT_HEAD(&s->notifiers_list, node, next); - return; - } - - /* update notifier node with new flags */ - QLIST_FOREACH_SAFE(node, &s->notifiers_list, next, next_node) { - if (node->sdev == sdev) { - if (new == IOMMU_NOTIFIER_NONE) { - trace_smmuv3_notify_flag_del(iommu->parent_obj.name); - QLIST_REMOVE(node, next); - g_free(node); - } - return; - } + QLIST_INSERT_HEAD(&s->devices_with_notifiers, sdev, next); + } else if (new == IOMMU_NOTIFIER_NONE) { + trace_smmuv3_notify_flag_del(iommu->parent_obj.name); + QLIST_REMOVE(sdev, next); } } -- 2.20.1 In the stripe8() function we use a variable length array; however we know that the maximum length required is MAX_NUM_BUSSES. Use a fixed-length array and an assert instead. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Reviewed-by: Francisco Iglesias <frasse.iglesias@gmail.com> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> Reviewed-by: Stefano Garzarella <sgarzare@redhat.com> Message-id: 20190328152635.2794-1-peter.maydell@linaro.org --- hw/ssi/xilinx_spips.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/hw/ssi/xilinx_spips.c b/hw/ssi/xilinx_spips.c index XXXXXXX..XXXXXXX 100644 --- a/hw/ssi/xilinx_spips.c +++ b/hw/ssi/xilinx_spips.c @@ -XXX,XX +XXX,XX @@ static void xlnx_zynqmp_qspips_reset(DeviceState *d) static inline void stripe8(uint8_t *x, int num, bool dir) { - uint8_t r[num]; - memset(r, 0, sizeof(uint8_t) * num); + uint8_t r[MAX_NUM_BUSSES]; int idx[2] = {0, 0}; int bit[2] = {0, 7}; int d = dir; + assert(num <= MAX_NUM_BUSSES); + memset(r, 0, sizeof(uint8_t) * num); + for (idx[0] = 0; idx[0] < num; ++idx[0]) { for (bit[0] = 7; bit[0] >= 0; bit[0]--) { r[idx[!d]] |= x[idx[d]] & 1 << bit[d] ? 1 << bit[!d] : 0; -- 2.20.1 Normally configure identifies the source path by looking at the location where the configure script itself exists. We also provide a --source-path option which lets the user manually override this. There isn't really an obvious use case for the --source-path option, and in commit 927128222b0a91f56c13a in 2017 we accidentally added some logic that looks at $source_path before the command line option that overrides it has been processed. The fact that nobody complained suggests that there isn't any use of this option and we aren't testing it either; remove it. This allows us to move the "make $source_path absolute" logic up so that there is no window in the script where $source_path is set but not yet absolute. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Message-id: 20190318134019.23729-1-peter.maydell@linaro.org --- configure | 10 ++-------- 1 file changed, 2 insertions(+), 8 deletions(-) diff --git a/configure b/configure index XXXXXXX..XXXXXXX 100755 --- a/configure +++ b/configure @@ -XXX,XX +XXX,XX @@ ld_has() { # default parameters source_path=$(dirname "$0") +# make source path absolute +source_path=$(cd "$source_path"; pwd) cpu="" iasl="iasl" interp_prefix="/usr/gnemul/qemu-%M" @@ -XXX,XX +XXX,XX @@ for opt do ;; --cxx=*) CXX="$optarg" ;; - --source-path=*) source_path="$optarg" - ;; --cpu=*) cpu="$optarg" ;; --extra-cflags=*) QEMU_CFLAGS="$QEMU_CFLAGS $optarg" @@ -XXX,XX +XXX,XX @@ if test "$debug_info" = "yes"; then LDFLAGS="-g $LDFLAGS" fi -# make source path absolute -source_path=$(cd "$source_path"; pwd) - # running configure in the source tree? # we know that's the case if configure is there. if test -f "./configure"; then @@ -XXX,XX +XXX,XX @@ for opt do ;; --interp-prefix=*) interp_prefix="$optarg" ;; - --source-path=*) - ;; --cross-prefix=*) ;; --cc=*) @@ -XXX,XX +XXX,XX @@ $(echo Available targets: $default_target_list | \ --target-list-exclude=LIST exclude a set of targets from the default target-list Advanced options (experts only): - --source-path=PATH path of source code [$source_path] --cross-prefix=PREFIX use PREFIX for compile tools [$cross_prefix] --cc=CC use C compiler CC [$cc] --iasl=IASL use ACPI compiler IASL [$iasl] -- 2.20.1 Enforce that for M-profile various FPSCR bits which are RES0 there but have defined meanings on A-profile are never settable. This ensures that M-profile code can't enable the A-profile behaviour (notably vector length/stride handling) by accident. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20190416125744.27770-2-peter.maydell@linaro.org --- target/arm/vfp_helper.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/target/arm/vfp_helper.c b/target/arm/vfp_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vfp_helper.c +++ b/target/arm/vfp_helper.c @@ -XXX,XX +XXX,XX @@ void HELPER(vfp_set_fpscr)(CPUARMState *env, uint32_t val) val &= ~FPCR_FZ16; } + if (arm_feature(env, ARM_FEATURE_M)) { + /* + * M profile FPSCR is RES0 for the QC, STRIDE, FZ16, LEN bits + * and also for the trapped-exception-handling bits IxE. + */ + val &= 0xf7c0009f; + } + /* * We don't implement trapped exception handling, so the * trap enable bits, IDE|IXE|UFE|OFE|DZE|IOE are all RAZ/WI (not RES0!) -- 2.20.1 For M-profile the MVFR* ID registers are memory mapped, in the range we implement via the NVIC. Allow them to be read. (If the CPU has no FPU, these registers are defined to be RAZ.) Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20190416125744.27770-3-peter.maydell@linaro.org --- hw/intc/armv7m_nvic.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/hw/intc/armv7m_nvic.c b/hw/intc/armv7m_nvic.c index XXXXXXX..XXXXXXX 100644 --- a/hw/intc/armv7m_nvic.c +++ b/hw/intc/armv7m_nvic.c @@ -XXX,XX +XXX,XX @@ static uint32_t nvic_readl(NVICState *s, uint32_t offset, MemTxAttrs attrs) return 0; } return cpu->env.v7m.sfar; + case 0xf40: /* MVFR0 */ + return cpu->isar.mvfr0; + case 0xf44: /* MVFR1 */ + return cpu->isar.mvfr1; + case 0xf48: /* MVFR2 */ + return cpu->isar.mvfr2; default: bad_offset: qemu_log_mask(LOG_GUEST_ERROR, "NVIC: Bad read offset 0x%x\n", offset); -- 2.20.1 The M-profile floating point support has three associated config registers: FPCAR, FPCCR and FPDSCR. It also makes the registers CPACR and NSACR have behaviour other than reads-as-zero. Add support for all of these as simple reads-as-written registers. We will hook up actual functionality later. The main complexity here is handling the FPCCR register, which has a mix of banked and unbanked bits. Note that we don't share storage with the A-profile cpu->cp15.nsacr and cpu->cp15.cpacr_el1, though the behaviour is quite similar, for two reasons: * the M profile CPACR is banked between security states * it preserves the invariant that M profile uses no state inside the cp15 substruct Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20190416125744.27770-4-peter.maydell@linaro.org --- target/arm/cpu.h | 34 ++++++++++++ hw/intc/armv7m_nvic.c | 125 ++++++++++++++++++++++++++++++++++++++++++ target/arm/cpu.c | 5 ++ target/arm/machine.c | 16 ++++++ 4 files changed, 180 insertions(+) diff --git a/target/arm/cpu.h b/target/arm/cpu.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -XXX,XX +XXX,XX @@ typedef struct CPUARMState { uint32_t scr[M_REG_NUM_BANKS]; uint32_t msplim[M_REG_NUM_BANKS]; uint32_t psplim[M_REG_NUM_BANKS]; + uint32_t fpcar[M_REG_NUM_BANKS]; + uint32_t fpccr[M_REG_NUM_BANKS]; + uint32_t fpdscr[M_REG_NUM_BANKS]; + uint32_t cpacr[M_REG_NUM_BANKS]; + uint32_t nsacr; } v7m; /* Information associated with an exception about to be taken: @@ -XXX,XX +XXX,XX @@ FIELD(V7M_CSSELR, LEVEL, 1, 3) */ FIELD(V7M_CSSELR, INDEX, 0, 4) +/* v7M FPCCR bits */ +FIELD(V7M_FPCCR, LSPACT, 0, 1) +FIELD(V7M_FPCCR, USER, 1, 1) +FIELD(V7M_FPCCR, S, 2, 1) +FIELD(V7M_FPCCR, THREAD, 3, 1) +FIELD(V7M_FPCCR, HFRDY, 4, 1) +FIELD(V7M_FPCCR, MMRDY, 5, 1) +FIELD(V7M_FPCCR, BFRDY, 6, 1) +FIELD(V7M_FPCCR, SFRDY, 7, 1) +FIELD(V7M_FPCCR, MONRDY, 8, 1) +FIELD(V7M_FPCCR, SPLIMVIOL, 9, 1) +FIELD(V7M_FPCCR, UFRDY, 10, 1) +FIELD(V7M_FPCCR, RES0, 11, 15) +FIELD(V7M_FPCCR, TS, 26, 1) +FIELD(V7M_FPCCR, CLRONRETS, 27, 1) +FIELD(V7M_FPCCR, CLRONRET, 28, 1) +FIELD(V7M_FPCCR, LSPENS, 29, 1) +FIELD(V7M_FPCCR, LSPEN, 30, 1) +FIELD(V7M_FPCCR, ASPEN, 31, 1) +/* These bits are banked. Others are non-banked and live in the M_REG_S bank */ +#define R_V7M_FPCCR_BANKED_MASK \ + (R_V7M_FPCCR_LSPACT_MASK | \ + R_V7M_FPCCR_USER_MASK | \ + R_V7M_FPCCR_THREAD_MASK | \ + R_V7M_FPCCR_MMRDY_MASK | \ + R_V7M_FPCCR_SPLIMVIOL_MASK | \ + R_V7M_FPCCR_UFRDY_MASK | \ + R_V7M_FPCCR_ASPEN_MASK) + /* * System register ID fields. */ diff --git a/hw/intc/armv7m_nvic.c b/hw/intc/armv7m_nvic.c index XXXXXXX..XXXXXXX 100644 --- a/hw/intc/armv7m_nvic.c +++ b/hw/intc/armv7m_nvic.c @@ -XXX,XX +XXX,XX @@ static uint32_t nvic_readl(NVICState *s, uint32_t offset, MemTxAttrs attrs) } case 0xd84: /* CSSELR */ return cpu->env.v7m.csselr[attrs.secure]; + case 0xd88: /* CPACR */ + if (!arm_feature(&cpu->env, ARM_FEATURE_VFP)) { + return 0; + } + return cpu->env.v7m.cpacr[attrs.secure]; + case 0xd8c: /* NSACR */ + if (!attrs.secure || !arm_feature(&cpu->env, ARM_FEATURE_VFP)) { + return 0; + } + return cpu->env.v7m.nsacr; /* TODO: Implement debug registers. */ case 0xd90: /* MPU_TYPE */ /* Unified MPU; if the MPU is not present this value is zero */ @@ -XXX,XX +XXX,XX @@ static uint32_t nvic_readl(NVICState *s, uint32_t offset, MemTxAttrs attrs) return 0; } return cpu->env.v7m.sfar; + case 0xf34: /* FPCCR */ + if (!arm_feature(&cpu->env, ARM_FEATURE_VFP)) { + return 0; + } + if (attrs.secure) { + return cpu->env.v7m.fpccr[M_REG_S]; + } else { + /* + * NS can read LSPEN, CLRONRET and MONRDY. It can read + * BFRDY and HFRDY if AIRCR.BFHFNMINS != 0; + * other non-banked bits RAZ. + * TODO: MONRDY should RAZ/WI if DEMCR.SDME is set. + */ + uint32_t value = cpu->env.v7m.fpccr[M_REG_S]; + uint32_t mask = R_V7M_FPCCR_LSPEN_MASK | + R_V7M_FPCCR_CLRONRET_MASK | + R_V7M_FPCCR_MONRDY_MASK; + + if (s->cpu->env.v7m.aircr & R_V7M_AIRCR_BFHFNMINS_MASK) { + mask |= R_V7M_FPCCR_BFRDY_MASK | R_V7M_FPCCR_HFRDY_MASK; + } + + value &= mask; + + value |= cpu->env.v7m.fpccr[M_REG_NS]; + return value; + } + case 0xf38: /* FPCAR */ + if (!arm_feature(&cpu->env, ARM_FEATURE_VFP)) { + return 0; + } + return cpu->env.v7m.fpcar[attrs.secure]; + case 0xf3c: /* FPDSCR */ + if (!arm_feature(&cpu->env, ARM_FEATURE_VFP)) { + return 0; + } + return cpu->env.v7m.fpdscr[attrs.secure]; case 0xf40: /* MVFR0 */ return cpu->isar.mvfr0; case 0xf44: /* MVFR1 */ @@ -XXX,XX +XXX,XX @@ static void nvic_writel(NVICState *s, uint32_t offset, uint32_t value, cpu->env.v7m.csselr[attrs.secure] = value & R_V7M_CSSELR_INDEX_MASK; } break; + case 0xd88: /* CPACR */ + if (arm_feature(&cpu->env, ARM_FEATURE_VFP)) { + /* We implement only the Floating Point extension's CP10/CP11 */ + cpu->env.v7m.cpacr[attrs.secure] = value & (0xf << 20); + } + break; + case 0xd8c: /* NSACR */ + if (attrs.secure && arm_feature(&cpu->env, ARM_FEATURE_VFP)) { + /* We implement only the Floating Point extension's CP10/CP11 */ + cpu->env.v7m.nsacr = value & (3 << 10); + } + break; case 0xd90: /* MPU_TYPE */ return; /* RO */ case 0xd94: /* MPU_CTRL */ @@ -XXX,XX +XXX,XX @@ static void nvic_writel(NVICState *s, uint32_t offset, uint32_t value, } break; } + case 0xf34: /* FPCCR */ + if (arm_feature(&cpu->env, ARM_FEATURE_VFP)) { + /* Not all bits here are banked. */ + uint32_t fpccr_s; + + if (!arm_feature(&cpu->env, ARM_FEATURE_V8)) { + /* Don't allow setting of bits not present in v7M */ + value &= (R_V7M_FPCCR_LSPACT_MASK | + R_V7M_FPCCR_USER_MASK | + R_V7M_FPCCR_THREAD_MASK | + R_V7M_FPCCR_HFRDY_MASK | + R_V7M_FPCCR_MMRDY_MASK | + R_V7M_FPCCR_BFRDY_MASK | + R_V7M_FPCCR_MONRDY_MASK | + R_V7M_FPCCR_LSPEN_MASK | + R_V7M_FPCCR_ASPEN_MASK); + } + value &= ~R_V7M_FPCCR_RES0_MASK; + + if (!attrs.secure) { + /* Some non-banked bits are configurably writable by NS */ + fpccr_s = cpu->env.v7m.fpccr[M_REG_S]; + if (!(fpccr_s & R_V7M_FPCCR_LSPENS_MASK)) { + uint32_t lspen = FIELD_EX32(value, V7M_FPCCR, LSPEN); + fpccr_s = FIELD_DP32(fpccr_s, V7M_FPCCR, LSPEN, lspen); + } + if (!(fpccr_s & R_V7M_FPCCR_CLRONRETS_MASK)) { + uint32_t cor = FIELD_EX32(value, V7M_FPCCR, CLRONRET); + fpccr_s = FIELD_DP32(fpccr_s, V7M_FPCCR, CLRONRET, cor); + } + if ((s->cpu->env.v7m.aircr & R_V7M_AIRCR_BFHFNMINS_MASK)) { + uint32_t hfrdy = FIELD_EX32(value, V7M_FPCCR, HFRDY); + uint32_t bfrdy = FIELD_EX32(value, V7M_FPCCR, BFRDY); + fpccr_s = FIELD_DP32(fpccr_s, V7M_FPCCR, HFRDY, hfrdy); + fpccr_s = FIELD_DP32(fpccr_s, V7M_FPCCR, BFRDY, bfrdy); + } + /* TODO MONRDY should RAZ/WI if DEMCR.SDME is set */ + { + uint32_t monrdy = FIELD_EX32(value, V7M_FPCCR, MONRDY); + fpccr_s = FIELD_DP32(fpccr_s, V7M_FPCCR, MONRDY, monrdy); + } + + /* + * All other non-banked bits are RAZ/WI from NS; write + * just the banked bits to fpccr[M_REG_NS]. + */ + value &= R_V7M_FPCCR_BANKED_MASK; + cpu->env.v7m.fpccr[M_REG_NS] = value; + } else { + fpccr_s = value; + } + cpu->env.v7m.fpccr[M_REG_S] = fpccr_s; + } + break; + case 0xf38: /* FPCAR */ + if (arm_feature(&cpu->env, ARM_FEATURE_VFP)) { + value &= ~7; + cpu->env.v7m.fpcar[attrs.secure] = value; + } + break; + case 0xf3c: /* FPDSCR */ + if (arm_feature(&cpu->env, ARM_FEATURE_VFP)) { + value &= 0x07c00000; + cpu->env.v7m.fpdscr[attrs.secure] = value; + } + break; case 0xf50: /* ICIALLU */ case 0xf58: /* ICIMVAU */ case 0xf5c: /* DCIMVAC */ diff --git a/target/arm/cpu.c b/target/arm/cpu.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpu.c +++ b/target/arm/cpu.c @@ -XXX,XX +XXX,XX @@ static void arm_cpu_reset(CPUState *s) env->v7m.ccr[M_REG_S] |= R_V7M_CCR_UNALIGN_TRP_MASK; } + if (arm_feature(env, ARM_FEATURE_VFP)) { + env->v7m.fpccr[M_REG_NS] = R_V7M_FPCCR_ASPEN_MASK; + env->v7m.fpccr[M_REG_S] = R_V7M_FPCCR_ASPEN_MASK | + R_V7M_FPCCR_LSPEN_MASK | R_V7M_FPCCR_S_MASK; + } /* Unlike A/R profile, M profile defines the reset LR value */ env->regs[14] = 0xffffffff; diff --git a/target/arm/machine.c b/target/arm/machine.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/machine.c +++ b/target/arm/machine.c @@ -XXX,XX +XXX,XX @@ static const VMStateDescription vmstate_m_v8m = { } }; +static const VMStateDescription vmstate_m_fp = { + .name = "cpu/m/fp", + .version_id = 1, + .minimum_version_id = 1, + .needed = vfp_needed, + .fields = (VMStateField[]) { + VMSTATE_UINT32_ARRAY(env.v7m.fpcar, ARMCPU, M_REG_NUM_BANKS), + VMSTATE_UINT32_ARRAY(env.v7m.fpccr, ARMCPU, M_REG_NUM_BANKS), + VMSTATE_UINT32_ARRAY(env.v7m.fpdscr, ARMCPU, M_REG_NUM_BANKS), + VMSTATE_UINT32_ARRAY(env.v7m.cpacr, ARMCPU, M_REG_NUM_BANKS), + VMSTATE_UINT32(env.v7m.nsacr, ARMCPU), + VMSTATE_END_OF_LIST() + } +}; + static const VMStateDescription vmstate_m = { .name = "cpu/m", .version_id = 4, @@ -XXX,XX +XXX,XX @@ static const VMStateDescription vmstate_m = { &vmstate_m_scr, &vmstate_m_other_sp, &vmstate_m_v8m, + &vmstate_m_fp, NULL } }; -- 2.20.1 The only "system register" that M-profile floating point exposes via the VMRS/VMRS instructions is FPSCR, and it does not have the odd special case for rd==15. Add a check to ensure we only expose FPSCR. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20190416125744.27770-5-peter.maydell@linaro.org --- target/arm/translate.c | 19 +++++++++++++++++-- 1 file changed, 17 insertions(+), 2 deletions(-) diff --git a/target/arm/translate.c b/target/arm/translate.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate.c +++ b/target/arm/translate.c @@ -XXX,XX +XXX,XX @@ static int disas_vfp_insn(DisasContext *s, uint32_t insn) } } } else { /* !dp */ + bool is_sysreg; + if ((insn & 0x6f) != 0x00) return 1; rn = VFP_SREG_N(insn); + + is_sysreg = extract32(insn, 21, 1); + + if (arm_dc_feature(s, ARM_FEATURE_M)) { + /* + * The only M-profile VFP vmrs/vmsr sysreg is FPSCR. + * Writes to R15 are UNPREDICTABLE; we choose to undef. + */ + if (is_sysreg && (rd == 15 || (rn >> 1) != ARM_VFP_FPSCR)) { + return 1; + } + } + if (insn & ARM_CP_RW_BIT) { /* vfp->arm */ - if (insn & (1 << 21)) { + if (is_sysreg) { /* system register */ rn >>= 1; @@ -XXX,XX +XXX,XX @@ static int disas_vfp_insn(DisasContext *s, uint32_t insn) } } else { /* arm->vfp */ - if (insn & (1 << 21)) { + if (is_sysreg) { rn >>= 1; /* system register */ switch (rn) { -- 2.20.1 Like AArch64, M-profile floating point has no FPEXC enable bit to gate floating point; so always set the VFPEN TB flag. M-profile also has CPACR and NSACR similar to A-profile; they behave slightly differently: * the CPACR is banked between Secure and Non-Secure * if the NSACR forces a trap then this is taken to the Secure state, not the Non-Secure state Honour the CPACR and NSACR settings. The NSACR handling requires us to borrow the exception.target_el field (usually meaningless for M profile) to distinguish the NOCP UsageFault taken to Secure state from the more usual fault taken to the current security state. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20190416125744.27770-6-peter.maydell@linaro.org --- target/arm/helper.c | 55 +++++++++++++++++++++++++++++++++++++++--- target/arm/translate.c | 10 ++++++-- 2 files changed, 60 insertions(+), 5 deletions(-) diff --git a/target/arm/helper.c b/target/arm/helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -XXX,XX +XXX,XX @@ uint32_t arm_phys_excp_target_el(CPUState *cs, uint32_t excp_idx, return target_el; } +/* + * Return true if the v7M CPACR permits access to the FPU for the specified + * security state and privilege level. + */ +static bool v7m_cpacr_pass(CPUARMState *env, bool is_secure, bool is_priv) +{ + switch (extract32(env->v7m.cpacr[is_secure], 20, 2)) { + case 0: + case 2: /* UNPREDICTABLE: we treat like 0 */ + return false; + case 1: + return is_priv; + case 3: + return true; + default: + g_assert_not_reached(); + } +} + static bool v7m_stack_write(ARMCPU *cpu, uint32_t addr, uint32_t value, ARMMMUIdx mmu_idx, bool ignfault) { @@ -XXX,XX +XXX,XX @@ void arm_v7m_cpu_do_interrupt(CPUState *cs) env->v7m.cfsr[env->v7m.secure] |= R_V7M_CFSR_UNDEFINSTR_MASK; break; case EXCP_NOCP: - armv7m_nvic_set_pending(env->nvic, ARMV7M_EXCP_USAGE, env->v7m.secure); - env->v7m.cfsr[env->v7m.secure] |= R_V7M_CFSR_NOCP_MASK; + { + /* + * NOCP might be directed to something other than the current + * security state if this fault is because of NSACR; we indicate + * the target security state using exception.target_el. + */ + int target_secstate; + + if (env->exception.target_el == 3) { + target_secstate = M_REG_S; + } else { + target_secstate = env->v7m.secure; + } + armv7m_nvic_set_pending(env->nvic, ARMV7M_EXCP_USAGE, target_secstate); + env->v7m.cfsr[target_secstate] |= R_V7M_CFSR_NOCP_MASK; break; + } case EXCP_INVSTATE: armv7m_nvic_set_pending(env->nvic, ARMV7M_EXCP_USAGE, env->v7m.secure); env->v7m.cfsr[env->v7m.secure] |= R_V7M_CFSR_INVSTATE_MASK; @@ -XXX,XX +XXX,XX @@ int fp_exception_el(CPUARMState *env, int cur_el) return 0; } + if (arm_feature(env, ARM_FEATURE_M)) { + /* CPACR can cause a NOCP UsageFault taken to current security state */ + if (!v7m_cpacr_pass(env, env->v7m.secure, cur_el != 0)) { + return 1; + } + + if (arm_feature(env, ARM_FEATURE_M_SECURITY) && !env->v7m.secure) { + if (!extract32(env->v7m.nsacr, 10, 1)) { + /* FP insns cause a NOCP UsageFault taken to Secure */ + return 3; + } + } + + return 0; + } + /* The CPACR controls traps to EL1, or PL1 if we're 32 bit: * 0, 2 : trap EL0 and EL1/PL1 accesses * 1 : trap only EL0 accesses @@ -XXX,XX +XXX,XX @@ void cpu_get_tb_cpu_state(CPUARMState *env, target_ulong *pc, flags = FIELD_DP32(flags, TBFLAG_A32, SCTLR_B, arm_sctlr_b(env)); flags = FIELD_DP32(flags, TBFLAG_A32, NS, !access_secure_reg(env)); if (env->vfp.xregs[ARM_VFP_FPEXC] & (1 << 30) - || arm_el_is_aa64(env, 1)) { + || arm_el_is_aa64(env, 1) || arm_feature(env, ARM_FEATURE_M)) { flags = FIELD_DP32(flags, TBFLAG_A32, VFPEN, 1); } flags = FIELD_DP32(flags, TBFLAG_A32, XSCALE_CPAR, env->cp15.c15_cpar); diff --git a/target/arm/translate.c b/target/arm/translate.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate.c +++ b/target/arm/translate.c @@ -XXX,XX +XXX,XX @@ static int disas_vfp_insn(DisasContext *s, uint32_t insn) * for attempts to execute invalid vfp/neon encodings with FP disabled. */ if (s->fp_excp_el) { - gen_exception_insn(s, 4, EXCP_UDEF, - syn_fp_access_trap(1, 0xe, false), s->fp_excp_el); + if (arm_dc_feature(s, ARM_FEATURE_M)) { + gen_exception_insn(s, 4, EXCP_NOCP, syn_uncategorized(), + s->fp_excp_el); + } else { + gen_exception_insn(s, 4, EXCP_UDEF, + syn_fp_access_trap(1, 0xe, false), + s->fp_excp_el); + } return 0; } -- 2.20.1 Correct the decode of the M-profile "coprocessor and floating-point instructions" space: * op0 == 0b11 is always unallocated * if the CPU has an FPU then all insns with op1 == 0b101 are floating point and go to disas_vfp_insn() For the moment we leave VLLDM and VLSTM as NOPs; in a later commit we will fill in the proper implementation for the case where an FPU is present. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20190416125744.27770-7-peter.maydell@linaro.org --- target/arm/translate.c | 26 ++++++++++++++++++++++---- 1 file changed, 22 insertions(+), 4 deletions(-) diff --git a/target/arm/translate.c b/target/arm/translate.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate.c +++ b/target/arm/translate.c @@ -XXX,XX +XXX,XX @@ static void disas_thumb2_insn(DisasContext *s, uint32_t insn) case 6: case 7: case 14: case 15: /* Coprocessor. */ if (arm_dc_feature(s, ARM_FEATURE_M)) { - /* We don't currently implement M profile FP support, - * so this entire space should give a NOCP fault, with - * the exception of the v8M VLLDM and VLSTM insns, which - * must be NOPs in Secure state and UNDEF in Nonsecure state. + /* 0b111x_11xx_xxxx_xxxx_xxxx_xxxx_xxxx_xxxx */ + if (extract32(insn, 24, 2) == 3) { + goto illegal_op; /* op0 = 0b11 : unallocated */ + } + + /* + * Decode VLLDM and VLSTM first: these are nonstandard because: + * * if there is no FPU then these insns must NOP in + * Secure state and UNDEF in Nonsecure state + * * if there is an FPU then these insns do not have + * the usual behaviour that disas_vfp_insn() provides of + * being controlled by CPACR/NSACR enable bits or the + * lazy-stacking logic. */ if (arm_dc_feature(s, ARM_FEATURE_V8) && (insn & 0xffa00f00) == 0xec200a00) { @@ -XXX,XX +XXX,XX @@ static void disas_thumb2_insn(DisasContext *s, uint32_t insn) /* Just NOP since FP support is not implemented */ break; } + if (arm_dc_feature(s, ARM_FEATURE_VFP) && + ((insn >> 8) & 0xe) == 10) { + /* FP, and the CPU supports it */ + if (disas_vfp_insn(s, insn)) { + goto illegal_op; + } + break; + } + /* All other insns: NOCP */ gen_exception_insn(s, 4, EXCP_NOCP, syn_uncategorized(), default_exception_el(s)); -- 2.20.1 If the floating point extension is present, then the SG instruction must clear the CONTROL_S.SFPA bit. Implement this. (On a no-FPU system the bit will always be zero, so we don't need to make the clearing of the bit conditional on ARM_FEATURE_VFP.) Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20190416125744.27770-8-peter.maydell@linaro.org --- target/arm/helper.c | 1 + 1 file changed, 1 insertion(+) diff --git a/target/arm/helper.c b/target/arm/helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -XXX,XX +XXX,XX @@ static bool v7m_handle_execute_nsc(ARMCPU *cpu) qemu_log_mask(CPU_LOG_INT, "...really an SG instruction at 0x%08" PRIx32 ", executing it\n", env->regs[15]); env->regs[14] &= ~1; + env->v7m.control[M_REG_S] &= ~R_V7M_CONTROL_SFPA_MASK; switch_v7m_security_state(env, true); xpsr_write(env, 0, XPSR_IT); env->regs[15] += 4; -- 2.20.1 The M-profile CONTROL register has two bits -- SFPA and FPCA -- which relate to floating-point support, and should be RES0 otherwise. Handle them correctly in the MSR/MRS register access code. Neither is banked between security states, so they are stored in v7m.control[M_REG_S] regardless of current security state. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20190416125744.27770-9-peter.maydell@linaro.org --- target/arm/helper.c | 57 ++++++++++++++++++++++++++++++++++++++------- 1 file changed, 49 insertions(+), 8 deletions(-) diff --git a/target/arm/helper.c b/target/arm/helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -XXX,XX +XXX,XX @@ uint32_t HELPER(v7m_mrs)(CPUARMState *env, uint32_t reg) return xpsr_read(env) & mask; break; case 20: /* CONTROL */ - return env->v7m.control[env->v7m.secure]; + { + uint32_t value = env->v7m.control[env->v7m.secure]; + if (!env->v7m.secure) { + /* SFPA is RAZ/WI from NS; FPCA is stored in the M_REG_S bank */ + value |= env->v7m.control[M_REG_S] & R_V7M_CONTROL_FPCA_MASK; + } + return value; + } case 0x94: /* CONTROL_NS */ /* We have to handle this here because unprivileged Secure code * can read the NS CONTROL register. @@ -XXX,XX +XXX,XX @@ uint32_t HELPER(v7m_mrs)(CPUARMState *env, uint32_t reg) if (!env->v7m.secure) { return 0; } - return env->v7m.control[M_REG_NS]; + return env->v7m.control[M_REG_NS] | + (env->v7m.control[M_REG_S] & R_V7M_CONTROL_FPCA_MASK); } if (el == 0) { @@ -XXX,XX +XXX,XX @@ void HELPER(v7m_msr)(CPUARMState *env, uint32_t maskreg, uint32_t val) */ uint32_t mask = extract32(maskreg, 8, 4); uint32_t reg = extract32(maskreg, 0, 8); + int cur_el = arm_current_el(env); - if (arm_current_el(env) == 0 && reg > 7) { - /* only xPSR sub-fields may be written by unprivileged */ + if (cur_el == 0 && reg > 7 && reg != 20) { + /* + * only xPSR sub-fields and CONTROL.SFPA may be written by + * unprivileged code + */ return; } @@ -XXX,XX +XXX,XX @@ void HELPER(v7m_msr)(CPUARMState *env, uint32_t maskreg, uint32_t val) env->v7m.control[M_REG_NS] &= ~R_V7M_CONTROL_NPRIV_MASK; env->v7m.control[M_REG_NS] |= val & R_V7M_CONTROL_NPRIV_MASK; } + /* + * SFPA is RAZ/WI from NS. FPCA is RO if NSACR.CP10 == 0, + * RES0 if the FPU is not present, and is stored in the S bank + */ + if (arm_feature(env, ARM_FEATURE_VFP) && + extract32(env->v7m.nsacr, 10, 1)) { + env->v7m.control[M_REG_S] &= ~R_V7M_CONTROL_FPCA_MASK; + env->v7m.control[M_REG_S] |= val & R_V7M_CONTROL_FPCA_MASK; + } return; case 0x98: /* SP_NS */ { @@ -XXX,XX +XXX,XX @@ void HELPER(v7m_msr)(CPUARMState *env, uint32_t maskreg, uint32_t val) env->v7m.faultmask[env->v7m.secure] = val & 1; break; case 20: /* CONTROL */ - /* Writing to the SPSEL bit only has an effect if we are in + /* + * Writing to the SPSEL bit only has an effect if we are in * thread mode; other bits can be updated by any privileged code. * write_v7m_control_spsel() deals with updating the SPSEL bit in * env->v7m.control, so we only need update the others. * For v7M, we must just ignore explicit writes to SPSEL in handler * mode; for v8M the write is permitted but will have no effect. + * All these bits are writes-ignored from non-privileged code, + * except for SFPA. */ - if (arm_feature(env, ARM_FEATURE_V8) || - !arm_v7m_is_handler_mode(env)) { + if (cur_el > 0 && (arm_feature(env, ARM_FEATURE_V8) || + !arm_v7m_is_handler_mode(env))) { write_v7m_control_spsel(env, (val & R_V7M_CONTROL_SPSEL_MASK) != 0); } - if (arm_feature(env, ARM_FEATURE_M_MAIN)) { + if (cur_el > 0 && arm_feature(env, ARM_FEATURE_M_MAIN)) { env->v7m.control[env->v7m.secure] &= ~R_V7M_CONTROL_NPRIV_MASK; env->v7m.control[env->v7m.secure] |= val & R_V7M_CONTROL_NPRIV_MASK; } + if (arm_feature(env, ARM_FEATURE_VFP)) { + /* + * SFPA is RAZ/WI from NS or if no FPU. + * FPCA is RO if NSACR.CP10 == 0, RES0 if the FPU is not present. + * Both are stored in the S bank. + */ + if (env->v7m.secure) { + env->v7m.control[M_REG_S] &= ~R_V7M_CONTROL_SFPA_MASK; + env->v7m.control[M_REG_S] |= val & R_V7M_CONTROL_SFPA_MASK; + } + if (cur_el > 0 && + (env->v7m.secure || !arm_feature(env, ARM_FEATURE_M_SECURITY) || + extract32(env->v7m.nsacr, 10, 1))) { + env->v7m.control[M_REG_S] &= ~R_V7M_CONTROL_FPCA_MASK; + env->v7m.control[M_REG_S] |= val & R_V7M_CONTROL_FPCA_MASK; + } + } break; default: bad_reg: -- 2.20.1 Currently the code in v7m_push_stack() which detects a violation of the v8M stack limit simply returns early if it does so. This is OK for the current integer-only code, but won't work for the floating point handling we're about to add. We need to continue executing the rest of the function so that we check for other exceptions like not having permission to use the FPU and so that we correctly set the FPCCR state if we are doing lazy stacking. Refactor to avoid the early return. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20190416125744.27770-10-peter.maydell@linaro.org --- target/arm/helper.c | 23 ++++++++++++++++++----- 1 file changed, 18 insertions(+), 5 deletions(-) diff --git a/target/arm/helper.c b/target/arm/helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -XXX,XX +XXX,XX @@ static bool v7m_push_stack(ARMCPU *cpu) * should ignore further stack faults trying to process * that derived exception.) */ - bool stacked_ok; + bool stacked_ok = true, limitviol = false; CPUARMState *env = &cpu->env; uint32_t xpsr = xpsr_read(env); uint32_t frameptr = env->regs[13]; @@ -XXX,XX +XXX,XX @@ static bool v7m_push_stack(ARMCPU *cpu) armv7m_nvic_set_pending(env->nvic, ARMV7M_EXCP_USAGE, env->v7m.secure); env->regs[13] = limit; - return true; + /* + * We won't try to perform any further memory accesses but + * we must continue through the following code to check for + * permission faults during FPU state preservation, and we + * must update FPCCR if lazy stacking is enabled. + */ + limitviol = true; + stacked_ok = false; } } @@ -XXX,XX +XXX,XX @@ static bool v7m_push_stack(ARMCPU *cpu) * (which may be taken in preference to the one we started with * if it has higher priority). */ - stacked_ok = + stacked_ok = stacked_ok && v7m_stack_write(cpu, frameptr, env->regs[0], mmu_idx, false) && v7m_stack_write(cpu, frameptr + 4, env->regs[1], mmu_idx, false) && v7m_stack_write(cpu, frameptr + 8, env->regs[2], mmu_idx, false) && @@ -XXX,XX +XXX,XX @@ static bool v7m_push_stack(ARMCPU *cpu) v7m_stack_write(cpu, frameptr + 24, env->regs[15], mmu_idx, false) && v7m_stack_write(cpu, frameptr + 28, xpsr, mmu_idx, false); - /* Update SP regardless of whether any of the stack accesses failed. */ - env->regs[13] = frameptr; + /* + * If we broke a stack limit then SP was already updated earlier; + * otherwise we update SP regardless of whether any of the stack + * accesses failed or we took some other kind of fault. + */ + if (!limitviol) { + env->regs[13] = frameptr; + } return !stacked_ok; } -- 2.20.1 Handle floating point registers in exception entry. This corresponds to the FP-specific parts of the pseudocode functions ActivateException() and PushStack(). We defer the code corresponding to UpdateFPCCR() to a later patch. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20190416125744.27770-11-peter.maydell@linaro.org --- target/arm/helper.c | 98 +++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 95 insertions(+), 3 deletions(-) diff --git a/target/arm/helper.c b/target/arm/helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -XXX,XX +XXX,XX @@ static void v7m_exception_taken(ARMCPU *cpu, uint32_t lr, bool dotailchain, switch_v7m_security_state(env, targets_secure); write_v7m_control_spsel(env, 0); arm_clear_exclusive(env); + /* Clear SFPA and FPCA (has no effect if no FPU) */ + env->v7m.control[M_REG_S] &= + ~(R_V7M_CONTROL_FPCA_MASK | R_V7M_CONTROL_SFPA_MASK); /* Clear IT bits */ env->condexec_bits = 0; env->regs[14] = lr; @@ -XXX,XX +XXX,XX @@ static bool v7m_push_stack(ARMCPU *cpu) uint32_t xpsr = xpsr_read(env); uint32_t frameptr = env->regs[13]; ARMMMUIdx mmu_idx = arm_mmu_idx(env); + uint32_t framesize; + bool nsacr_cp10 = extract32(env->v7m.nsacr, 10, 1); + + if ((env->v7m.control[M_REG_S] & R_V7M_CONTROL_FPCA_MASK) && + (env->v7m.secure || nsacr_cp10)) { + if (env->v7m.secure && + env->v7m.fpccr[M_REG_S] & R_V7M_FPCCR_TS_MASK) { + framesize = 0xa8; + } else { + framesize = 0x68; + } + } else { + framesize = 0x20; + } /* Align stack pointer if the guest wants that */ if ((frameptr & 4) && @@ -XXX,XX +XXX,XX @@ static bool v7m_push_stack(ARMCPU *cpu) xpsr |= XPSR_SPREALIGN; } - frameptr -= 0x20; + xpsr &= ~XPSR_SFPA; + if (env->v7m.secure && + (env->v7m.control[M_REG_S] & R_V7M_CONTROL_SFPA_MASK)) { + xpsr |= XPSR_SFPA; + } + + frameptr -= framesize; if (arm_feature(env, ARM_FEATURE_V8)) { uint32_t limit = v7m_sp_limit(env); @@ -XXX,XX +XXX,XX @@ static bool v7m_push_stack(ARMCPU *cpu) v7m_stack_write(cpu, frameptr + 24, env->regs[15], mmu_idx, false) && v7m_stack_write(cpu, frameptr + 28, xpsr, mmu_idx, false); + if (env->v7m.control[M_REG_S] & R_V7M_CONTROL_FPCA_MASK) { + /* FPU is active, try to save its registers */ + bool fpccr_s = env->v7m.fpccr[M_REG_S] & R_V7M_FPCCR_S_MASK; + bool lspact = env->v7m.fpccr[fpccr_s] & R_V7M_FPCCR_LSPACT_MASK; + + if (lspact && arm_feature(env, ARM_FEATURE_M_SECURITY)) { + qemu_log_mask(CPU_LOG_INT, + "...SecureFault because LSPACT and FPCA both set\n"); + env->v7m.sfsr |= R_V7M_SFSR_LSERR_MASK; + armv7m_nvic_set_pending(env->nvic, ARMV7M_EXCP_SECURE, false); + } else if (!env->v7m.secure && !nsacr_cp10) { + qemu_log_mask(CPU_LOG_INT, + "...Secure UsageFault with CFSR.NOCP because " + "NSACR.CP10 prevents stacking FP regs\n"); + armv7m_nvic_set_pending(env->nvic, ARMV7M_EXCP_USAGE, M_REG_S); + env->v7m.cfsr[M_REG_S] |= R_V7M_CFSR_NOCP_MASK; + } else { + if (!(env->v7m.fpccr[M_REG_S] & R_V7M_FPCCR_LSPEN_MASK)) { + /* Lazy stacking disabled, save registers now */ + int i; + bool cpacr_pass = v7m_cpacr_pass(env, env->v7m.secure, + arm_current_el(env) != 0); + + if (stacked_ok && !cpacr_pass) { + /* + * Take UsageFault if CPACR forbids access. The pseudocode + * here does a full CheckCPEnabled() but we know the NSACR + * check can never fail as we have already handled that. + */ + qemu_log_mask(CPU_LOG_INT, + "...UsageFault with CFSR.NOCP because " + "CPACR.CP10 prevents stacking FP regs\n"); + armv7m_nvic_set_pending(env->nvic, ARMV7M_EXCP_USAGE, + env->v7m.secure); + env->v7m.cfsr[env->v7m.secure] |= R_V7M_CFSR_NOCP_MASK; + stacked_ok = false; + } + + for (i = 0; i < ((framesize == 0xa8) ? 32 : 16); i += 2) { + uint64_t dn = *aa32_vfp_dreg(env, i / 2); + uint32_t faddr = frameptr + 0x20 + 4 * i; + uint32_t slo = extract64(dn, 0, 32); + uint32_t shi = extract64(dn, 32, 32); + + if (i >= 16) { + faddr += 8; /* skip the slot for the FPSCR */ + } + stacked_ok = stacked_ok && + v7m_stack_write(cpu, faddr, slo, mmu_idx, false) && + v7m_stack_write(cpu, faddr + 4, shi, mmu_idx, false); + } + stacked_ok = stacked_ok && + v7m_stack_write(cpu, frameptr + 0x60, + vfp_get_fpscr(env), mmu_idx, false); + if (cpacr_pass) { + for (i = 0; i < ((framesize == 0xa8) ? 32 : 16); i += 2) { + *aa32_vfp_dreg(env, i / 2) = 0; + } + vfp_set_fpscr(env, 0); + } + } else { + /* Lazy stacking enabled, save necessary info to stack later */ + /* TODO : equivalent of UpdateFPCCR() pseudocode */ + } + } + } + /* * If we broke a stack limit then SP was already updated earlier; * otherwise we update SP regardless of whether any of the stack @@ -XXX,XX +XXX,XX @@ void arm_v7m_cpu_do_interrupt(CPUState *cs) if (arm_feature(env, ARM_FEATURE_V8)) { lr = R_V7M_EXCRET_RES1_MASK | - R_V7M_EXCRET_DCRS_MASK | - R_V7M_EXCRET_FTYPE_MASK; + R_V7M_EXCRET_DCRS_MASK; /* The S bit indicates whether we should return to Secure * or NonSecure (ie our current state). * The ES bit indicates whether we're taking this exception @@ -XXX,XX +XXX,XX @@ void arm_v7m_cpu_do_interrupt(CPUState *cs) if (env->v7m.secure) { lr |= R_V7M_EXCRET_S_MASK; } + if (!(env->v7m.control[M_REG_S] & R_V7M_CONTROL_FPCA_MASK)) { + lr |= R_V7M_EXCRET_FTYPE_MASK; + } } else { lr = R_V7M_EXCRET_RES1_MASK | R_V7M_EXCRET_S_MASK | -- 2.20.1 Implement the code which updates the FPCCR register on an exception entry where we are going to use lazy FP stacking. We have to defer to the NVIC to determine whether the various exceptions are currently ready or not. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Message-id: 20190416125744.27770-12-peter.maydell@linaro.org --- target/arm/cpu.h | 14 +++++++++ hw/intc/armv7m_nvic.c | 34 ++++++++++++++++++++++ target/arm/helper.c | 67 ++++++++++++++++++++++++++++++++++++++++++- 3 files changed, 114 insertions(+), 1 deletion(-) diff --git a/target/arm/cpu.h b/target/arm/cpu.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -XXX,XX +XXX,XX @@ void armv7m_nvic_acknowledge_irq(void *opaque); * (Ignoring -1, this is the same as the RETTOBASE value before completion.) */ int armv7m_nvic_complete_irq(void *opaque, int irq, bool secure); +/** + * armv7m_nvic_get_ready_status(void *opaque, int irq, bool secure) + * @opaque: the NVIC + * @irq: the exception number to mark pending + * @secure: false for non-banked exceptions or for the nonsecure + * version of a banked exception, true for the secure version of a banked + * exception. + * + * Return whether an exception is "ready", i.e. whether the exception is + * enabled and is configured at a priority which would allow it to + * interrupt the current execution priority. This controls whether the + * RDY bit for it in the FPCCR is set. + */ +bool armv7m_nvic_get_ready_status(void *opaque, int irq, bool secure); /** * armv7m_nvic_raw_execution_priority: return the raw execution priority * @opaque: the NVIC diff --git a/hw/intc/armv7m_nvic.c b/hw/intc/armv7m_nvic.c index XXXXXXX..XXXXXXX 100644 --- a/hw/intc/armv7m_nvic.c +++ b/hw/intc/armv7m_nvic.c @@ -XXX,XX +XXX,XX @@ int armv7m_nvic_complete_irq(void *opaque, int irq, bool secure) return ret; } +bool armv7m_nvic_get_ready_status(void *opaque, int irq, bool secure) +{ + /* + * Return whether an exception is "ready", i.e. it is enabled and is + * configured at a priority which would allow it to interrupt the + * current execution priority. + * + * irq and secure have the same semantics as for armv7m_nvic_set_pending(): + * for non-banked exceptions secure is always false; for banked exceptions + * it indicates which of the exceptions is required. + */ + NVICState *s = (NVICState *)opaque; + bool banked = exc_is_banked(irq); + VecInfo *vec; + int running = nvic_exec_prio(s); + + assert(irq > ARMV7M_EXCP_RESET && irq < s->num_irq); + assert(!secure || banked); + + /* + * HardFault is an odd special case: we always check against -1, + * even if we're secure and HardFault has priority -3; we never + * need to check for enabled state. + */ + if (irq == ARMV7M_EXCP_HARD) { + return running > -1; + } + + vec = (banked && secure) ? &s->sec_vectors[irq] : &s->vectors[irq]; + + return vec->enabled && + exc_group_prio(s, vec->prio, secure) < running; +} + /* callback when external interrupt line is changed */ static void set_irq_level(void *opaque, int n, int level) { diff --git a/target/arm/helper.c b/target/arm/helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -XXX,XX +XXX,XX @@ static void v7m_exception_taken(ARMCPU *cpu, uint32_t lr, bool dotailchain, env->thumb = addr & 1; } +static void v7m_update_fpccr(CPUARMState *env, uint32_t frameptr, + bool apply_splim) +{ + /* + * Like the pseudocode UpdateFPCCR: save state in FPCAR and FPCCR + * that we will need later in order to do lazy FP reg stacking. + */ + bool is_secure = env->v7m.secure; + void *nvic = env->nvic; + /* + * Some bits are unbanked and live always in fpccr[M_REG_S]; some bits + * are banked and we want to update the bit in the bank for the + * current security state; and in one case we want to specifically + * update the NS banked version of a bit even if we are secure. + */ + uint32_t *fpccr_s = &env->v7m.fpccr[M_REG_S]; + uint32_t *fpccr_ns = &env->v7m.fpccr[M_REG_NS]; + uint32_t *fpccr = &env->v7m.fpccr[is_secure]; + bool hfrdy, bfrdy, mmrdy, ns_ufrdy, s_ufrdy, sfrdy, monrdy; + + env->v7m.fpcar[is_secure] = frameptr & ~0x7; + + if (apply_splim && arm_feature(env, ARM_FEATURE_V8)) { + bool splimviol; + uint32_t splim = v7m_sp_limit(env); + bool ign = armv7m_nvic_neg_prio_requested(nvic, is_secure) && + (env->v7m.ccr[is_secure] & R_V7M_CCR_STKOFHFNMIGN_MASK); + + splimviol = !ign && frameptr < splim; + *fpccr = FIELD_DP32(*fpccr, V7M_FPCCR, SPLIMVIOL, splimviol); + } + + *fpccr = FIELD_DP32(*fpccr, V7M_FPCCR, LSPACT, 1); + + *fpccr_s = FIELD_DP32(*fpccr_s, V7M_FPCCR, S, is_secure); + + *fpccr = FIELD_DP32(*fpccr, V7M_FPCCR, USER, arm_current_el(env) == 0); + + *fpccr = FIELD_DP32(*fpccr, V7M_FPCCR, THREAD, + !arm_v7m_is_handler_mode(env)); + + hfrdy = armv7m_nvic_get_ready_status(nvic, ARMV7M_EXCP_HARD, false); + *fpccr_s = FIELD_DP32(*fpccr_s, V7M_FPCCR, HFRDY, hfrdy); + + bfrdy = armv7m_nvic_get_ready_status(nvic, ARMV7M_EXCP_BUS, false); + *fpccr_s = FIELD_DP32(*fpccr_s, V7M_FPCCR, BFRDY, bfrdy); + + mmrdy = armv7m_nvic_get_ready_status(nvic, ARMV7M_EXCP_MEM, is_secure); + *fpccr = FIELD_DP32(*fpccr, V7M_FPCCR, MMRDY, mmrdy); + + ns_ufrdy = armv7m_nvic_get_ready_status(nvic, ARMV7M_EXCP_USAGE, false); + *fpccr_ns = FIELD_DP32(*fpccr_ns, V7M_FPCCR, UFRDY, ns_ufrdy); + + monrdy = armv7m_nvic_get_ready_status(nvic, ARMV7M_EXCP_DEBUG, false); + *fpccr_s = FIELD_DP32(*fpccr_s, V7M_FPCCR, MONRDY, monrdy); + + if (arm_feature(env, ARM_FEATURE_M_SECURITY)) { + s_ufrdy = armv7m_nvic_get_ready_status(nvic, ARMV7M_EXCP_USAGE, true); + *fpccr_s = FIELD_DP32(*fpccr_s, V7M_FPCCR, UFRDY, s_ufrdy); + + sfrdy = armv7m_nvic_get_ready_status(nvic, ARMV7M_EXCP_SECURE, false); + *fpccr_s = FIELD_DP32(*fpccr_s, V7M_FPCCR, SFRDY, sfrdy); + } +} + static bool v7m_push_stack(ARMCPU *cpu) { /* Do the "set up stack frame" part of exception entry, @@ -XXX,XX +XXX,XX @@ static bool v7m_push_stack(ARMCPU *cpu) } } else { /* Lazy stacking enabled, save necessary info to stack later */ - /* TODO : equivalent of UpdateFPCCR() pseudocode */ + v7m_update_fpccr(env, frameptr + 0x20, true); } } } -- 2.20.1 For v8M floating point support, transitions from Secure to Non-secure state via BLNS and BLXNS must clear the CONTROL.SFPA bit. (This corresponds to the pseudocode BranchToNS() function.) Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20190416125744.27770-13-peter.maydell@linaro.org --- target/arm/helper.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/target/arm/helper.c b/target/arm/helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -XXX,XX +XXX,XX @@ void HELPER(v7m_bxns)(CPUARMState *env, uint32_t dest) /* translate.c should have made BXNS UNDEF unless we're secure */ assert(env->v7m.secure); + if (!(dest & 1)) { + env->v7m.control[M_REG_S] &= ~R_V7M_CONTROL_SFPA_MASK; + } switch_v7m_security_state(env, dest & 1); env->thumb = 1; env->regs[15] = dest & ~1; @@ -XXX,XX +XXX,XX @@ void HELPER(v7m_blxns)(CPUARMState *env, uint32_t dest) */ write_v7m_exception(env, 1); } + env->v7m.control[M_REG_S] &= ~R_V7M_CONTROL_SFPA_MASK; switch_v7m_security_state(env, 0); env->thumb = 1; env->regs[15] = dest; -- 2.20.1 The TailChain() pseudocode specifies that a tail chaining exception should sanitize the excReturn all-ones bits and (if there is no FPU) the excReturn FType bits; we weren't doing this. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20190416125744.27770-14-peter.maydell@linaro.org --- target/arm/helper.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/target/arm/helper.c b/target/arm/helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -XXX,XX +XXX,XX @@ static void v7m_exception_taken(ARMCPU *cpu, uint32_t lr, bool dotailchain, qemu_log_mask(CPU_LOG_INT, "...taking pending %s exception %d\n", targets_secure ? "secure" : "nonsecure", exc); + if (dotailchain) { + /* Sanitize LR FType and PREFIX bits */ + if (!arm_feature(env, ARM_FEATURE_VFP)) { + lr |= R_V7M_EXCRET_FTYPE_MASK; + } + lr = deposit32(lr, 24, 8, 0xff); + } + if (arm_feature(env, ARM_FEATURE_V8)) { if (arm_feature(env, ARM_FEATURE_M_SECURITY) && (lr & R_V7M_EXCRET_S_MASK)) { -- 2.20.1 The magic value pushed onto the callee stack as an integrity check is different if floating point is present. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20190416125744.27770-15-peter.maydell@linaro.org --- target/arm/helper.c | 22 +++++++++++++++++++--- 1 file changed, 19 insertions(+), 3 deletions(-) diff --git a/target/arm/helper.c b/target/arm/helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -XXX,XX +XXX,XX @@ load_fail: return false; } +static uint32_t v7m_integrity_sig(CPUARMState *env, uint32_t lr) +{ + /* + * Return the integrity signature value for the callee-saves + * stack frame section. @lr is the exception return payload/LR value + * whose FType bit forms bit 0 of the signature if FP is present. + */ + uint32_t sig = 0xfefa125a; + + if (!arm_feature(env, ARM_FEATURE_VFP) || (lr & R_V7M_EXCRET_FTYPE_MASK)) { + sig |= 1; + } + return sig; +} + static bool v7m_push_callee_stack(ARMCPU *cpu, uint32_t lr, bool dotailchain, bool ignore_faults) { @@ -XXX,XX +XXX,XX @@ static bool v7m_push_callee_stack(ARMCPU *cpu, uint32_t lr, bool dotailchain, bool stacked_ok; uint32_t limit; bool want_psp; + uint32_t sig; if (dotailchain) { bool mode = lr & R_V7M_EXCRET_MODE_MASK; @@ -XXX,XX +XXX,XX @@ static bool v7m_push_callee_stack(ARMCPU *cpu, uint32_t lr, bool dotailchain, /* Write as much of the stack frame as we can. A write failure may * cause us to pend a derived exception. */ + sig = v7m_integrity_sig(env, lr); stacked_ok = - v7m_stack_write(cpu, frameptr, 0xfefa125b, mmu_idx, ignore_faults) && + v7m_stack_write(cpu, frameptr, sig, mmu_idx, ignore_faults) && v7m_stack_write(cpu, frameptr + 0x8, env->regs[4], mmu_idx, ignore_faults) && v7m_stack_write(cpu, frameptr + 0xc, env->regs[5], mmu_idx, @@ -XXX,XX +XXX,XX @@ static void do_v7m_exception_exit(ARMCPU *cpu) if (return_to_secure && ((excret & R_V7M_EXCRET_ES_MASK) == 0 || (excret & R_V7M_EXCRET_DCRS_MASK) == 0)) { - uint32_t expected_sig = 0xfefa125b; uint32_t actual_sig; pop_ok = v7m_stack_read(cpu, &actual_sig, frameptr, mmu_idx); - if (pop_ok && expected_sig != actual_sig) { + if (pop_ok && v7m_integrity_sig(env, excret) != actual_sig) { /* Take a SecureFault on the current stack */ env->v7m.sfsr |= R_V7M_SFSR_INVIS_MASK; armv7m_nvic_set_pending(env->nvic, ARMV7M_EXCP_SECURE, false); -- 2.20.1 Handle floating point registers in exception return. This corresponds to pseudocode functions ValidateExceptionReturn(), ExceptionReturn(), PopStack() and ConsumeExcStackFrame(). Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20190416125744.27770-16-peter.maydell@linaro.org --- target/arm/helper.c | 142 +++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 141 insertions(+), 1 deletion(-) diff --git a/target/arm/helper.c b/target/arm/helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -XXX,XX +XXX,XX @@ static void do_v7m_exception_exit(ARMCPU *cpu) bool rettobase = false; bool exc_secure = false; bool return_to_secure; + bool ftype; + bool restore_s16_s31; /* If we're not in Handler mode then jumps to magic exception-exit * addresses don't have magic behaviour. However for the v8M @@ -XXX,XX +XXX,XX @@ static void do_v7m_exception_exit(ARMCPU *cpu) excret); } + ftype = excret & R_V7M_EXCRET_FTYPE_MASK; + + if (!arm_feature(env, ARM_FEATURE_VFP) && !ftype) { + qemu_log_mask(LOG_GUEST_ERROR, "M profile: zero FTYPE in exception " + "exit PC value 0x%" PRIx32 " is UNPREDICTABLE " + "if FPU not present\n", + excret); + ftype = true; + } + if (arm_feature(env, ARM_FEATURE_M_SECURITY)) { /* EXC_RETURN.ES validation check (R_SMFL). We must do this before * we pick which FAULTMASK to clear. @@ -XXX,XX +XXX,XX @@ static void do_v7m_exception_exit(ARMCPU *cpu) */ write_v7m_control_spsel_for_secstate(env, return_to_sp_process, exc_secure); + /* + * Clear scratch FP values left in caller saved registers; this + * must happen before any kind of tail chaining. + */ + if ((env->v7m.fpccr[M_REG_S] & R_V7M_FPCCR_CLRONRET_MASK) && + (env->v7m.control[M_REG_S] & R_V7M_CONTROL_FPCA_MASK)) { + if (env->v7m.fpccr[M_REG_S] & R_V7M_FPCCR_LSPACT_MASK) { + env->v7m.sfsr |= R_V7M_SFSR_LSERR_MASK; + armv7m_nvic_set_pending(env->nvic, ARMV7M_EXCP_SECURE, false); + qemu_log_mask(CPU_LOG_INT, "...taking SecureFault on existing " + "stackframe: error during lazy state deactivation\n"); + v7m_exception_taken(cpu, excret, true, false); + return; + } else { + /* Clear s0..s15 and FPSCR */ + int i; + + for (i = 0; i < 16; i += 2) { + *aa32_vfp_dreg(env, i / 2) = 0; + } + vfp_set_fpscr(env, 0); + } + } + if (sfault) { env->v7m.sfsr |= R_V7M_SFSR_INVER_MASK; armv7m_nvic_set_pending(env->nvic, ARMV7M_EXCP_SECURE, false); @@ -XXX,XX +XXX,XX @@ static void do_v7m_exception_exit(ARMCPU *cpu) } } + if (!ftype) { + /* FP present and we need to handle it */ + if (!return_to_secure && + (env->v7m.fpccr[M_REG_S] & R_V7M_FPCCR_LSPACT_MASK)) { + armv7m_nvic_set_pending(env->nvic, ARMV7M_EXCP_SECURE, false); + env->v7m.sfsr |= R_V7M_SFSR_LSERR_MASK; + qemu_log_mask(CPU_LOG_INT, + "...taking SecureFault on existing stackframe: " + "Secure LSPACT set but exception return is " + "not to secure state\n"); + v7m_exception_taken(cpu, excret, true, false); + return; + } + + restore_s16_s31 = return_to_secure && + (env->v7m.fpccr[M_REG_S] & R_V7M_FPCCR_TS_MASK); + + if (env->v7m.fpccr[return_to_secure] & R_V7M_FPCCR_LSPACT_MASK) { + /* State in FPU is still valid, just clear LSPACT */ + env->v7m.fpccr[return_to_secure] &= ~R_V7M_FPCCR_LSPACT_MASK; + } else { + int i; + uint32_t fpscr; + bool cpacr_pass, nsacr_pass; + + cpacr_pass = v7m_cpacr_pass(env, return_to_secure, + return_to_priv); + nsacr_pass = return_to_secure || + extract32(env->v7m.nsacr, 10, 1); + + if (!cpacr_pass) { + armv7m_nvic_set_pending(env->nvic, ARMV7M_EXCP_USAGE, + return_to_secure); + env->v7m.cfsr[return_to_secure] |= R_V7M_CFSR_NOCP_MASK; + qemu_log_mask(CPU_LOG_INT, + "...taking UsageFault on existing " + "stackframe: CPACR.CP10 prevents unstacking " + "FP regs\n"); + v7m_exception_taken(cpu, excret, true, false); + return; + } else if (!nsacr_pass) { + armv7m_nvic_set_pending(env->nvic, ARMV7M_EXCP_USAGE, true); + env->v7m.cfsr[M_REG_S] |= R_V7M_CFSR_INVPC_MASK; + qemu_log_mask(CPU_LOG_INT, + "...taking Secure UsageFault on existing " + "stackframe: NSACR.CP10 prevents unstacking " + "FP regs\n"); + v7m_exception_taken(cpu, excret, true, false); + return; + } + + for (i = 0; i < (restore_s16_s31 ? 32 : 16); i += 2) { + uint32_t slo, shi; + uint64_t dn; + uint32_t faddr = frameptr + 0x20 + 4 * i; + + if (i >= 16) { + faddr += 8; /* Skip the slot for the FPSCR */ + } + + pop_ok = pop_ok && + v7m_stack_read(cpu, &slo, faddr, mmu_idx) && + v7m_stack_read(cpu, &shi, faddr + 4, mmu_idx); + + if (!pop_ok) { + break; + } + + dn = (uint64_t)shi << 32 | slo; + *aa32_vfp_dreg(env, i / 2) = dn; + } + pop_ok = pop_ok && + v7m_stack_read(cpu, &fpscr, frameptr + 0x60, mmu_idx); + if (pop_ok) { + vfp_set_fpscr(env, fpscr); + } + if (!pop_ok) { + /* + * These regs are 0 if security extension present; + * otherwise merely UNKNOWN. We zero always. + */ + for (i = 0; i < (restore_s16_s31 ? 32 : 16); i += 2) { + *aa32_vfp_dreg(env, i / 2) = 0; + } + vfp_set_fpscr(env, 0); + } + } + } + env->v7m.control[M_REG_S] = FIELD_DP32(env->v7m.control[M_REG_S], + V7M_CONTROL, FPCA, !ftype); + /* Commit to consuming the stack frame */ frameptr += 0x20; + if (!ftype) { + frameptr += 0x48; + if (restore_s16_s31) { + frameptr += 0x40; + } + } /* Undo stack alignment (the SPREALIGN bit indicates that the original * pre-exception SP was not 8-aligned and we added a padding word to * align it, so we undo this by ORing in the bit that increases it @@ -XXX,XX +XXX,XX @@ static void do_v7m_exception_exit(ARMCPU *cpu) *frame_sp_p = frameptr; } /* This xpsr_write() will invalidate frame_sp_p as it may switch stack */ - xpsr_write(env, xpsr, ~XPSR_SPREALIGN); + xpsr_write(env, xpsr, ~(XPSR_SPREALIGN | XPSR_SFPA)); + + if (env->v7m.secure) { + bool sfpa = xpsr & XPSR_SFPA; + + env->v7m.control[M_REG_S] = FIELD_DP32(env->v7m.control[M_REG_S], + V7M_CONTROL, SFPA, sfpa); + } /* The restored xPSR exception field will be zero if we're * resuming in Thread mode. If that doesn't match what the -- 2.20.1 Move the NS TBFLAG down from bit 19 to bit 6, which has not been used since commit c1e3781090b9d36c60 in 2015, when we started passing the entire MMU index in the TB flags rather than just a 'privilege level' bit. This rearrangement is not strictly necessary, but means that we can put M-profile-only bits next to each other rather than scattered across the flag word. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20190416125744.27770-17-peter.maydell@linaro.org --- target/arm/cpu.h | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/target/arm/cpu.h b/target/arm/cpu.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -XXX,XX +XXX,XX @@ FIELD(TBFLAG_ANY, BE_DATA, 23, 1) FIELD(TBFLAG_A32, THUMB, 0, 1) FIELD(TBFLAG_A32, VECLEN, 1, 3) FIELD(TBFLAG_A32, VECSTRIDE, 4, 2) +/* + * Indicates whether cp register reads and writes by guest code should access + * the secure or nonsecure bank of banked registers; note that this is not + * the same thing as the current security state of the processor! + */ +FIELD(TBFLAG_A32, NS, 6, 1) FIELD(TBFLAG_A32, VFPEN, 7, 1) FIELD(TBFLAG_A32, CONDEXEC, 8, 8) FIELD(TBFLAG_A32, SCTLR_B, 16, 1) @@ -XXX,XX +XXX,XX @@ FIELD(TBFLAG_A32, SCTLR_B, 16, 1) * checks on the other bits at runtime */ FIELD(TBFLAG_A32, XSCALE_CPAR, 17, 2) -/* Indicates whether cp register reads and writes by guest code should access - * the secure or nonsecure bank of banked registers; note that this is not - * the same thing as the current security state of the processor! - */ -FIELD(TBFLAG_A32, NS, 19, 1) /* For M profile only, Handler (ie not Thread) mode */ FIELD(TBFLAG_A32, HANDLER, 21, 1) /* For M profile only, whether we should generate stack-limit checks */ -- 2.20.1 We are close to running out of TB flags for AArch32; we could start using the cs_base word, but before we do that we can economise on our usage by sharing the same bits for the VFP VECSTRIDE field and the XScale XSCALE_CPAR field. This works because no XScale CPU ever had VFP. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20190416125744.27770-18-peter.maydell@linaro.org --- target/arm/cpu.h | 10 ++++++---- target/arm/cpu.c | 7 +++++++ target/arm/helper.c | 6 +++++- target/arm/translate.c | 9 +++++++-- 4 files changed, 25 insertions(+), 7 deletions(-) diff --git a/target/arm/cpu.h b/target/arm/cpu.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -XXX,XX +XXX,XX @@ FIELD(TBFLAG_ANY, BE_DATA, 23, 1) FIELD(TBFLAG_A32, THUMB, 0, 1) FIELD(TBFLAG_A32, VECLEN, 1, 3) FIELD(TBFLAG_A32, VECSTRIDE, 4, 2) +/* + * We store the bottom two bits of the CPAR as TB flags and handle + * checks on the other bits at runtime. This shares the same bits as + * VECSTRIDE, which is OK as no XScale CPU has VFP. + */ +FIELD(TBFLAG_A32, XSCALE_CPAR, 4, 2) /* * Indicates whether cp register reads and writes by guest code should access * the secure or nonsecure bank of banked registers; note that this is not @@ -XXX,XX +XXX,XX @@ FIELD(TBFLAG_A32, NS, 6, 1) FIELD(TBFLAG_A32, VFPEN, 7, 1) FIELD(TBFLAG_A32, CONDEXEC, 8, 8) FIELD(TBFLAG_A32, SCTLR_B, 16, 1) -/* We store the bottom two bits of the CPAR as TB flags and handle - * checks on the other bits at runtime - */ -FIELD(TBFLAG_A32, XSCALE_CPAR, 17, 2) /* For M profile only, Handler (ie not Thread) mode */ FIELD(TBFLAG_A32, HANDLER, 21, 1) /* For M profile only, whether we should generate stack-limit checks */ diff --git a/target/arm/cpu.c b/target/arm/cpu.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpu.c +++ b/target/arm/cpu.c @@ -XXX,XX +XXX,XX @@ static void arm_cpu_realizefn(DeviceState *dev, Error **errp) set_feature(env, ARM_FEATURE_THUMB_DSP); } + /* + * We rely on no XScale CPU having VFP so we can use the same bits in the + * TB flags field for VECSTRIDE and XSCALE_CPAR. + */ + assert(!(arm_feature(env, ARM_FEATURE_VFP) && + arm_feature(env, ARM_FEATURE_XSCALE))); + if (arm_feature(env, ARM_FEATURE_V7) && !arm_feature(env, ARM_FEATURE_M) && !arm_feature(env, ARM_FEATURE_PMSA)) { diff --git a/target/arm/helper.c b/target/arm/helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -XXX,XX +XXX,XX @@ void cpu_get_tb_cpu_state(CPUARMState *env, target_ulong *pc, || arm_el_is_aa64(env, 1) || arm_feature(env, ARM_FEATURE_M)) { flags = FIELD_DP32(flags, TBFLAG_A32, VFPEN, 1); } - flags = FIELD_DP32(flags, TBFLAG_A32, XSCALE_CPAR, env->cp15.c15_cpar); + /* Note that XSCALE_CPAR shares bits with VECSTRIDE */ + if (arm_feature(env, ARM_FEATURE_XSCALE)) { + flags = FIELD_DP32(flags, TBFLAG_A32, + XSCALE_CPAR, env->cp15.c15_cpar); + } } flags = FIELD_DP32(flags, TBFLAG_ANY, MMUIDX, arm_to_core_mmu_idx(mmu_idx)); diff --git a/target/arm/translate.c b/target/arm/translate.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate.c +++ b/target/arm/translate.c @@ -XXX,XX +XXX,XX @@ static void arm_tr_init_disas_context(DisasContextBase *dcbase, CPUState *cs) dc->fp_excp_el = FIELD_EX32(tb_flags, TBFLAG_ANY, FPEXC_EL); dc->vfp_enabled = FIELD_EX32(tb_flags, TBFLAG_A32, VFPEN); dc->vec_len = FIELD_EX32(tb_flags, TBFLAG_A32, VECLEN); - dc->vec_stride = FIELD_EX32(tb_flags, TBFLAG_A32, VECSTRIDE); - dc->c15_cpar = FIELD_EX32(tb_flags, TBFLAG_A32, XSCALE_CPAR); + if (arm_feature(env, ARM_FEATURE_XSCALE)) { + dc->c15_cpar = FIELD_EX32(tb_flags, TBFLAG_A32, XSCALE_CPAR); + dc->vec_stride = 0; + } else { + dc->vec_stride = FIELD_EX32(tb_flags, TBFLAG_A32, VECSTRIDE); + dc->c15_cpar = 0; + } dc->v7m_handler_mode = FIELD_EX32(tb_flags, TBFLAG_A32, HANDLER); dc->v8m_secure = arm_feature(env, ARM_FEATURE_M_SECURITY) && regime_is_secure(env, dc->mmu_idx); -- 2.20.1 The M-profile FPCCR.S bit indicates the security status of the floating point context. In the pseudocode ExecuteFPCheck() function it is unconditionally set to match the current security state whenever a floating point instruction is executed. Implement this by adding a new TB flag which tracks whether FPCCR.S is different from the current security state, so that we only need to emit the code to update it in the less-common case when it is not already set correctly. Note that we will add the handling for the other work done by ExecuteFPCheck() in later commits. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20190416125744.27770-19-peter.maydell@linaro.org --- target/arm/cpu.h | 2 ++ target/arm/translate.h | 1 + target/arm/helper.c | 5 +++++ target/arm/translate.c | 20 ++++++++++++++++++++ 4 files changed, 28 insertions(+) diff --git a/target/arm/cpu.h b/target/arm/cpu.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -XXX,XX +XXX,XX @@ FIELD(TBFLAG_A32, NS, 6, 1) FIELD(TBFLAG_A32, VFPEN, 7, 1) FIELD(TBFLAG_A32, CONDEXEC, 8, 8) FIELD(TBFLAG_A32, SCTLR_B, 16, 1) +/* For M profile only, set if FPCCR.S does not match current security state */ +FIELD(TBFLAG_A32, FPCCR_S_WRONG, 20, 1) /* For M profile only, Handler (ie not Thread) mode */ FIELD(TBFLAG_A32, HANDLER, 21, 1) /* For M profile only, whether we should generate stack-limit checks */ diff --git a/target/arm/translate.h b/target/arm/translate.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate.h +++ b/target/arm/translate.h @@ -XXX,XX +XXX,XX @@ typedef struct DisasContext { bool v7m_handler_mode; bool v8m_secure; /* true if v8M and we're in Secure mode */ bool v8m_stackcheck; /* true if we need to perform v8M stack limit checks */ + bool v8m_fpccr_s_wrong; /* true if v8M FPCCR.S != v8m_secure */ /* Immediate value in AArch32 SVC insn; must be set if is_jmp == DISAS_SWI * so that top level loop can generate correct syndrome information. */ diff --git a/target/arm/helper.c b/target/arm/helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -XXX,XX +XXX,XX @@ void cpu_get_tb_cpu_state(CPUARMState *env, target_ulong *pc, flags = FIELD_DP32(flags, TBFLAG_A32, STACKCHECK, 1); } + if (arm_feature(env, ARM_FEATURE_M_SECURITY) && + FIELD_EX32(env->v7m.fpccr[M_REG_S], V7M_FPCCR, S) != env->v7m.secure) { + flags = FIELD_DP32(flags, TBFLAG_A32, FPCCR_S_WRONG, 1); + } + *pflags = flags; *cs_base = 0; } diff --git a/target/arm/translate.c b/target/arm/translate.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate.c +++ b/target/arm/translate.c @@ -XXX,XX +XXX,XX @@ static int disas_vfp_insn(DisasContext *s, uint32_t insn) } } + if (arm_dc_feature(s, ARM_FEATURE_M)) { + /* Handle M-profile lazy FP state mechanics */ + + /* Update ownership of FP context: set FPCCR.S to match current state */ + if (s->v8m_fpccr_s_wrong) { + TCGv_i32 tmp; + + tmp = load_cpu_field(v7m.fpccr[M_REG_S]); + if (s->v8m_secure) { + tcg_gen_ori_i32(tmp, tmp, R_V7M_FPCCR_S_MASK); + } else { + tcg_gen_andi_i32(tmp, tmp, ~R_V7M_FPCCR_S_MASK); + } + store_cpu_field(tmp, v7m.fpccr[M_REG_S]); + /* Don't need to do this for any further FP insns in this TB */ + s->v8m_fpccr_s_wrong = false; + } + } + if (extract32(insn, 28, 4) == 0xf) { /* * Encodings with T=1 (Thumb) or unconditional (ARM): @@ -XXX,XX +XXX,XX @@ static void arm_tr_init_disas_context(DisasContextBase *dcbase, CPUState *cs) dc->v8m_secure = arm_feature(env, ARM_FEATURE_M_SECURITY) && regime_is_secure(env, dc->mmu_idx); dc->v8m_stackcheck = FIELD_EX32(tb_flags, TBFLAG_A32, STACKCHECK); + dc->v8m_fpccr_s_wrong = FIELD_EX32(tb_flags, TBFLAG_A32, FPCCR_S_WRONG); dc->cp_regs = cpu->cp_regs; dc->features = env->features; -- 2.20.1 The M-profile FPCCR.ASPEN bit indicates that automatic floating-point context preservation is enabled. Before executing any floating-point instruction, if FPCCR.ASPEN is set and the CONTROL FPCA/SFPA bits indicate that there is no active floating point context then we must create a new context (by initializing FPSCR and setting FPCA/SFPA to indicate that the context is now active). In the pseudocode this is handled by ExecuteFPCheck(). Implement this with a new TB flag which tracks whether we need to create a new FP context. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20190416125744.27770-20-peter.maydell@linaro.org --- target/arm/cpu.h | 2 ++ target/arm/translate.h | 1 + target/arm/helper.c | 13 +++++++++++++ target/arm/translate.c | 29 +++++++++++++++++++++++++++++ 4 files changed, 45 insertions(+) diff --git a/target/arm/cpu.h b/target/arm/cpu.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -XXX,XX +XXX,XX @@ FIELD(TBFLAG_A32, NS, 6, 1) FIELD(TBFLAG_A32, VFPEN, 7, 1) FIELD(TBFLAG_A32, CONDEXEC, 8, 8) FIELD(TBFLAG_A32, SCTLR_B, 16, 1) +/* For M profile only, set if we must create a new FP context */ +FIELD(TBFLAG_A32, NEW_FP_CTXT_NEEDED, 19, 1) /* For M profile only, set if FPCCR.S does not match current security state */ FIELD(TBFLAG_A32, FPCCR_S_WRONG, 20, 1) /* For M profile only, Handler (ie not Thread) mode */ diff --git a/target/arm/translate.h b/target/arm/translate.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate.h +++ b/target/arm/translate.h @@ -XXX,XX +XXX,XX @@ typedef struct DisasContext { bool v8m_secure; /* true if v8M and we're in Secure mode */ bool v8m_stackcheck; /* true if we need to perform v8M stack limit checks */ bool v8m_fpccr_s_wrong; /* true if v8M FPCCR.S != v8m_secure */ + bool v7m_new_fp_ctxt_needed; /* ASPEN set but no active FP context */ /* Immediate value in AArch32 SVC insn; must be set if is_jmp == DISAS_SWI * so that top level loop can generate correct syndrome information. */ diff --git a/target/arm/helper.c b/target/arm/helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -XXX,XX +XXX,XX @@ void cpu_get_tb_cpu_state(CPUARMState *env, target_ulong *pc, flags = FIELD_DP32(flags, TBFLAG_A32, FPCCR_S_WRONG, 1); } + if (arm_feature(env, ARM_FEATURE_M) && + (env->v7m.fpccr[env->v7m.secure] & R_V7M_FPCCR_ASPEN_MASK) && + (!(env->v7m.control[M_REG_S] & R_V7M_CONTROL_FPCA_MASK) || + (env->v7m.secure && + !(env->v7m.control[M_REG_S] & R_V7M_CONTROL_SFPA_MASK)))) { + /* + * ASPEN is set, but FPCA/SFPA indicate that there is no active + * FP context; we must create a new FP context before executing + * any FP insn. + */ + flags = FIELD_DP32(flags, TBFLAG_A32, NEW_FP_CTXT_NEEDED, 1); + } + *pflags = flags; *cs_base = 0; } diff --git a/target/arm/translate.c b/target/arm/translate.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate.c +++ b/target/arm/translate.c @@ -XXX,XX +XXX,XX @@ static int disas_vfp_insn(DisasContext *s, uint32_t insn) /* Don't need to do this for any further FP insns in this TB */ s->v8m_fpccr_s_wrong = false; } + + if (s->v7m_new_fp_ctxt_needed) { + /* + * Create new FP context by updating CONTROL.FPCA, CONTROL.SFPA + * and the FPSCR. + */ + TCGv_i32 control, fpscr; + uint32_t bits = R_V7M_CONTROL_FPCA_MASK; + + fpscr = load_cpu_field(v7m.fpdscr[s->v8m_secure]); + gen_helper_vfp_set_fpscr(cpu_env, fpscr); + tcg_temp_free_i32(fpscr); + /* + * We don't need to arrange to end the TB, because the only + * parts of FPSCR which we cache in the TB flags are the VECLEN + * and VECSTRIDE, and those don't exist for M-profile. + */ + + if (s->v8m_secure) { + bits |= R_V7M_CONTROL_SFPA_MASK; + } + control = load_cpu_field(v7m.control[M_REG_S]); + tcg_gen_ori_i32(control, control, bits); + store_cpu_field(control, v7m.control[M_REG_S]); + /* Don't need to do this for any further FP insns in this TB */ + s->v7m_new_fp_ctxt_needed = false; + } } if (extract32(insn, 28, 4) == 0xf) { @@ -XXX,XX +XXX,XX @@ static void arm_tr_init_disas_context(DisasContextBase *dcbase, CPUState *cs) regime_is_secure(env, dc->mmu_idx); dc->v8m_stackcheck = FIELD_EX32(tb_flags, TBFLAG_A32, STACKCHECK); dc->v8m_fpccr_s_wrong = FIELD_EX32(tb_flags, TBFLAG_A32, FPCCR_S_WRONG); + dc->v7m_new_fp_ctxt_needed = + FIELD_EX32(tb_flags, TBFLAG_A32, NEW_FP_CTXT_NEEDED); dc->cp_regs = cpu->cp_regs; dc->features = env->features; -- 2.20.1 Add a new helper function which returns the MMU index to use for v7M, where the caller specifies all of the security state, privilege level and whether the execution priority is negative, and reimplement the existing arm_v7m_mmu_idx_for_secstate_and_priv() in terms of it. We are going to need this for the lazy-FP-stacking code. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20190416125744.27770-21-peter.maydell@linaro.org --- target/arm/cpu.h | 7 +++++++ target/arm/helper.c | 14 +++++++++++--- 2 files changed, 18 insertions(+), 3 deletions(-) diff --git a/target/arm/cpu.h b/target/arm/cpu.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -XXX,XX +XXX,XX @@ static inline int arm_mmu_idx_to_el(ARMMMUIdx mmu_idx) } } +/* + * Return the MMU index for a v7M CPU with all relevant information + * manually specified. + */ +ARMMMUIdx arm_v7m_mmu_idx_all(CPUARMState *env, + bool secstate, bool priv, bool negpri); + /* Return the MMU index for a v7M CPU in the specified security and * privilege state. */ diff --git a/target/arm/helper.c b/target/arm/helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -XXX,XX +XXX,XX @@ int fp_exception_el(CPUARMState *env, int cur_el) return 0; } -ARMMMUIdx arm_v7m_mmu_idx_for_secstate_and_priv(CPUARMState *env, - bool secstate, bool priv) +ARMMMUIdx arm_v7m_mmu_idx_all(CPUARMState *env, + bool secstate, bool priv, bool negpri) { ARMMMUIdx mmu_idx = ARM_MMU_IDX_M; @@ -XXX,XX +XXX,XX @@ ARMMMUIdx arm_v7m_mmu_idx_for_secstate_and_priv(CPUARMState *env, mmu_idx |= ARM_MMU_IDX_M_PRIV; } - if (armv7m_nvic_neg_prio_requested(env->nvic, secstate)) { + if (negpri) { mmu_idx |= ARM_MMU_IDX_M_NEGPRI; } @@ -XXX,XX +XXX,XX @@ ARMMMUIdx arm_v7m_mmu_idx_for_secstate_and_priv(CPUARMState *env, return mmu_idx; } +ARMMMUIdx arm_v7m_mmu_idx_for_secstate_and_priv(CPUARMState *env, + bool secstate, bool priv) +{ + bool negpri = armv7m_nvic_neg_prio_requested(env->nvic, secstate); + + return arm_v7m_mmu_idx_all(env, secstate, priv, negpri); +} + /* Return the MMU index for a v7M CPU in the specified security state */ ARMMMUIdx arm_v7m_mmu_idx_for_secstate(CPUARMState *env, bool secstate) { -- 2.20.1 In the v7M architecture, if an exception is generated in the process of doing the lazy stacking of FP registers, the handling of possible escalation to HardFault is treated differently to the normal approach: it works based on the saved information about exception readiness that was stored in the FPCCR when the stack frame was created. Provide a new function armv7m_nvic_set_pending_lazyfp() which pends exceptions during lazy stacking, and implements this logic. This corresponds to the pseudocode TakePreserveFPException(). Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20190416125744.27770-22-peter.maydell@linaro.org --- target/arm/cpu.h | 12 ++++++ hw/intc/armv7m_nvic.c | 96 +++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 108 insertions(+) diff --git a/target/arm/cpu.h b/target/arm/cpu.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -XXX,XX +XXX,XX @@ void armv7m_nvic_set_pending(void *opaque, int irq, bool secure); * a different exception). */ void armv7m_nvic_set_pending_derived(void *opaque, int irq, bool secure); +/** + * armv7m_nvic_set_pending_lazyfp: mark this lazy FP exception as pending + * @opaque: the NVIC + * @irq: the exception number to mark pending + * @secure: false for non-banked exceptions or for the nonsecure + * version of a banked exception, true for the secure version of a banked + * exception. + * + * Similar to armv7m_nvic_set_pending(), but specifically for exceptions + * generated in the course of lazy stacking of FP registers. + */ +void armv7m_nvic_set_pending_lazyfp(void *opaque, int irq, bool secure); /** * armv7m_nvic_get_pending_irq_info: return highest priority pending * exception, and whether it targets Secure state diff --git a/hw/intc/armv7m_nvic.c b/hw/intc/armv7m_nvic.c index XXXXXXX..XXXXXXX 100644 --- a/hw/intc/armv7m_nvic.c +++ b/hw/intc/armv7m_nvic.c @@ -XXX,XX +XXX,XX @@ void armv7m_nvic_set_pending_derived(void *opaque, int irq, bool secure) do_armv7m_nvic_set_pending(opaque, irq, secure, true); } +void armv7m_nvic_set_pending_lazyfp(void *opaque, int irq, bool secure) +{ + /* + * Pend an exception during lazy FP stacking. This differs + * from the usual exception pending because the logic for + * whether we should escalate depends on the saved context + * in the FPCCR register, not on the current state of the CPU/NVIC. + */ + NVICState *s = (NVICState *)opaque; + bool banked = exc_is_banked(irq); + VecInfo *vec; + bool targets_secure; + bool escalate = false; + /* + * We will only look at bits in fpccr if this is a banked exception + * (in which case 'secure' tells us whether it is the S or NS version). + * All the bits for the non-banked exceptions are in fpccr_s. + */ + uint32_t fpccr_s = s->cpu->env.v7m.fpccr[M_REG_S]; + uint32_t fpccr = s->cpu->env.v7m.fpccr[secure]; + + assert(irq > ARMV7M_EXCP_RESET && irq < s->num_irq); + assert(!secure || banked); + + vec = (banked && secure) ? &s->sec_vectors[irq] : &s->vectors[irq]; + + targets_secure = banked ? secure : exc_targets_secure(s, irq); + + switch (irq) { + case ARMV7M_EXCP_DEBUG: + if (!(fpccr_s & R_V7M_FPCCR_MONRDY_MASK)) { + /* Ignore DebugMonitor exception */ + return; + } + break; + case ARMV7M_EXCP_MEM: + escalate = !(fpccr & R_V7M_FPCCR_MMRDY_MASK); + break; + case ARMV7M_EXCP_USAGE: + escalate = !(fpccr & R_V7M_FPCCR_UFRDY_MASK); + break; + case ARMV7M_EXCP_BUS: + escalate = !(fpccr_s & R_V7M_FPCCR_BFRDY_MASK); + break; + case ARMV7M_EXCP_SECURE: + escalate = !(fpccr_s & R_V7M_FPCCR_SFRDY_MASK); + break; + default: + g_assert_not_reached(); + } + + if (escalate) { + /* + * Escalate to HardFault: faults that initially targeted Secure + * continue to do so, even if HF normally targets NonSecure. + */ + irq = ARMV7M_EXCP_HARD; + if (arm_feature(&s->cpu->env, ARM_FEATURE_M_SECURITY) && + (targets_secure || + !(s->cpu->env.v7m.aircr & R_V7M_AIRCR_BFHFNMINS_MASK))) { + vec = &s->sec_vectors[irq]; + } else { + vec = &s->vectors[irq]; + } + } + + if (!vec->enabled || + nvic_exec_prio(s) <= exc_group_prio(s, vec->prio, secure)) { + if (!(fpccr_s & R_V7M_FPCCR_HFRDY_MASK)) { + /* + * We want to escalate to HardFault but the context the + * FP state belongs to prevents the exception pre-empting. + */ + cpu_abort(&s->cpu->parent_obj, + "Lockup: can't escalate to HardFault during " + "lazy FP register stacking\n"); + } + } + + if (escalate) { + s->cpu->env.v7m.hfsr |= R_V7M_HFSR_FORCED_MASK; + } + if (!vec->pending) { + vec->pending = 1; + /* + * We do not call nvic_irq_update(), because we know our caller + * is going to handle causing us to take the exception by + * raising EXCP_LAZYFP, so raising the IRQ line would be + * pointless extra work. We just need to recompute the + * priorities so that armv7m_nvic_can_take_pending_exception() + * returns the right answer. + */ + nvic_recompute_state(s); + } +} + /* Make pending IRQ active. */ void armv7m_nvic_acknowledge_irq(void *opaque) { -- 2.20.1 Pushing registers to the stack for v7M needs to handle three cases: * the "normal" case where we pend exceptions * an "ignore faults" case where we set FSR bits but do not pend exceptions (this is used when we are handling some kinds of derived exception on exception entry) * a "lazy FP stacking" case, where different FSR bits are set and the exception is pended differently Implement this by changing the existing flag argument that tells us whether to ignore faults or not into an enum that specifies which of the 3 modes we should handle. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20190416125744.27770-23-peter.maydell@linaro.org --- target/arm/helper.c | 118 +++++++++++++++++++++++++++++--------------- 1 file changed, 79 insertions(+), 39 deletions(-) diff --git a/target/arm/helper.c b/target/arm/helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -XXX,XX +XXX,XX @@ static bool v7m_cpacr_pass(CPUARMState *env, bool is_secure, bool is_priv) } } +/* + * What kind of stack write are we doing? This affects how exceptions + * generated during the stacking are treated. + */ +typedef enum StackingMode { + STACK_NORMAL, + STACK_IGNFAULTS, + STACK_LAZYFP, +} StackingMode; + static bool v7m_stack_write(ARMCPU *cpu, uint32_t addr, uint32_t value, - ARMMMUIdx mmu_idx, bool ignfault) + ARMMMUIdx mmu_idx, StackingMode mode) { CPUState *cs = CPU(cpu); CPUARMState *env = &cpu->env; @@ -XXX,XX +XXX,XX @@ static bool v7m_stack_write(ARMCPU *cpu, uint32_t addr, uint32_t value, &attrs, &prot, &page_size, &fi, NULL)) { /* MPU/SAU lookup failed */ if (fi.type == ARMFault_QEMU_SFault) { - qemu_log_mask(CPU_LOG_INT, - "...SecureFault with SFSR.AUVIOL during stacking\n"); - env->v7m.sfsr |= R_V7M_SFSR_AUVIOL_MASK | R_V7M_SFSR_SFARVALID_MASK; + if (mode == STACK_LAZYFP) { + qemu_log_mask(CPU_LOG_INT, + "...SecureFault with SFSR.LSPERR " + "during lazy stacking\n"); + env->v7m.sfsr |= R_V7M_SFSR_LSPERR_MASK; + } else { + qemu_log_mask(CPU_LOG_INT, + "...SecureFault with SFSR.AUVIOL " + "during stacking\n"); + env->v7m.sfsr |= R_V7M_SFSR_AUVIOL_MASK; + } + env->v7m.sfsr |= R_V7M_SFSR_SFARVALID_MASK; env->v7m.sfar = addr; exc = ARMV7M_EXCP_SECURE; exc_secure = false; } else { - qemu_log_mask(CPU_LOG_INT, "...MemManageFault with CFSR.MSTKERR\n"); - env->v7m.cfsr[secure] |= R_V7M_CFSR_MSTKERR_MASK; + if (mode == STACK_LAZYFP) { + qemu_log_mask(CPU_LOG_INT, + "...MemManageFault with CFSR.MLSPERR\n"); + env->v7m.cfsr[secure] |= R_V7M_CFSR_MLSPERR_MASK; + } else { + qemu_log_mask(CPU_LOG_INT, + "...MemManageFault with CFSR.MSTKERR\n"); + env->v7m.cfsr[secure] |= R_V7M_CFSR_MSTKERR_MASK; + } exc = ARMV7M_EXCP_MEM; exc_secure = secure; } @@ -XXX,XX +XXX,XX @@ static bool v7m_stack_write(ARMCPU *cpu, uint32_t addr, uint32_t value, attrs, &txres); if (txres != MEMTX_OK) { /* BusFault trying to write the data */ - qemu_log_mask(CPU_LOG_INT, "...BusFault with BFSR.STKERR\n"); - env->v7m.cfsr[M_REG_NS] |= R_V7M_CFSR_STKERR_MASK; + if (mode == STACK_LAZYFP) { + qemu_log_mask(CPU_LOG_INT, "...BusFault with BFSR.LSPERR\n"); + env->v7m.cfsr[M_REG_NS] |= R_V7M_CFSR_LSPERR_MASK; + } else { + qemu_log_mask(CPU_LOG_INT, "...BusFault with BFSR.STKERR\n"); + env->v7m.cfsr[M_REG_NS] |= R_V7M_CFSR_STKERR_MASK; + } exc = ARMV7M_EXCP_BUS; exc_secure = false; goto pend_fault; @@ -XXX,XX +XXX,XX @@ pend_fault: * later if we have two derived exceptions. * The only case when we must not pend the exception but instead * throw it away is if we are doing the push of the callee registers - * and we've already generated a derived exception. Even in this - * case we will still update the fault status registers. + * and we've already generated a derived exception (this is indicated + * by the caller passing STACK_IGNFAULTS). Even in this case we will + * still update the fault status registers. */ - if (!ignfault) { + switch (mode) { + case STACK_NORMAL: armv7m_nvic_set_pending_derived(env->nvic, exc, exc_secure); + break; + case STACK_LAZYFP: + armv7m_nvic_set_pending_lazyfp(env->nvic, exc, exc_secure); + break; + case STACK_IGNFAULTS: + break; } return false; } @@ -XXX,XX +XXX,XX @@ static bool v7m_push_callee_stack(ARMCPU *cpu, uint32_t lr, bool dotailchain, uint32_t limit; bool want_psp; uint32_t sig; + StackingMode smode = ignore_faults ? STACK_IGNFAULTS : STACK_NORMAL; if (dotailchain) { bool mode = lr & R_V7M_EXCRET_MODE_MASK; @@ -XXX,XX +XXX,XX @@ static bool v7m_push_callee_stack(ARMCPU *cpu, uint32_t lr, bool dotailchain, */ sig = v7m_integrity_sig(env, lr); stacked_ok = - v7m_stack_write(cpu, frameptr, sig, mmu_idx, ignore_faults) && - v7m_stack_write(cpu, frameptr + 0x8, env->regs[4], mmu_idx, - ignore_faults) && - v7m_stack_write(cpu, frameptr + 0xc, env->regs[5], mmu_idx, - ignore_faults) && - v7m_stack_write(cpu, frameptr + 0x10, env->regs[6], mmu_idx, - ignore_faults) && - v7m_stack_write(cpu, frameptr + 0x14, env->regs[7], mmu_idx, - ignore_faults) && - v7m_stack_write(cpu, frameptr + 0x18, env->regs[8], mmu_idx, - ignore_faults) && - v7m_stack_write(cpu, frameptr + 0x1c, env->regs[9], mmu_idx, - ignore_faults) && - v7m_stack_write(cpu, frameptr + 0x20, env->regs[10], mmu_idx, - ignore_faults) && - v7m_stack_write(cpu, frameptr + 0x24, env->regs[11], mmu_idx, - ignore_faults); + v7m_stack_write(cpu, frameptr, sig, mmu_idx, smode) && + v7m_stack_write(cpu, frameptr + 0x8, env->regs[4], mmu_idx, smode) && + v7m_stack_write(cpu, frameptr + 0xc, env->regs[5], mmu_idx, smode) && + v7m_stack_write(cpu, frameptr + 0x10, env->regs[6], mmu_idx, smode) && + v7m_stack_write(cpu, frameptr + 0x14, env->regs[7], mmu_idx, smode) && + v7m_stack_write(cpu, frameptr + 0x18, env->regs[8], mmu_idx, smode) && + v7m_stack_write(cpu, frameptr + 0x1c, env->regs[9], mmu_idx, smode) && + v7m_stack_write(cpu, frameptr + 0x20, env->regs[10], mmu_idx, smode) && + v7m_stack_write(cpu, frameptr + 0x24, env->regs[11], mmu_idx, smode); /* Update SP regardless of whether any of the stack accesses failed. */ *frame_sp_p = frameptr; @@ -XXX,XX +XXX,XX @@ static bool v7m_push_stack(ARMCPU *cpu) * if it has higher priority). */ stacked_ok = stacked_ok && - v7m_stack_write(cpu, frameptr, env->regs[0], mmu_idx, false) && - v7m_stack_write(cpu, frameptr + 4, env->regs[1], mmu_idx, false) && - v7m_stack_write(cpu, frameptr + 8, env->regs[2], mmu_idx, false) && - v7m_stack_write(cpu, frameptr + 12, env->regs[3], mmu_idx, false) && - v7m_stack_write(cpu, frameptr + 16, env->regs[12], mmu_idx, false) && - v7m_stack_write(cpu, frameptr + 20, env->regs[14], mmu_idx, false) && - v7m_stack_write(cpu, frameptr + 24, env->regs[15], mmu_idx, false) && - v7m_stack_write(cpu, frameptr + 28, xpsr, mmu_idx, false); + v7m_stack_write(cpu, frameptr, env->regs[0], mmu_idx, STACK_NORMAL) && + v7m_stack_write(cpu, frameptr + 4, env->regs[1], + mmu_idx, STACK_NORMAL) && + v7m_stack_write(cpu, frameptr + 8, env->regs[2], + mmu_idx, STACK_NORMAL) && + v7m_stack_write(cpu, frameptr + 12, env->regs[3], + mmu_idx, STACK_NORMAL) && + v7m_stack_write(cpu, frameptr + 16, env->regs[12], + mmu_idx, STACK_NORMAL) && + v7m_stack_write(cpu, frameptr + 20, env->regs[14], + mmu_idx, STACK_NORMAL) && + v7m_stack_write(cpu, frameptr + 24, env->regs[15], + mmu_idx, STACK_NORMAL) && + v7m_stack_write(cpu, frameptr + 28, xpsr, mmu_idx, STACK_NORMAL); if (env->v7m.control[M_REG_S] & R_V7M_CONTROL_FPCA_MASK) { /* FPU is active, try to save its registers */ @@ -XXX,XX +XXX,XX @@ static bool v7m_push_stack(ARMCPU *cpu) faddr += 8; /* skip the slot for the FPSCR */ } stacked_ok = stacked_ok && - v7m_stack_write(cpu, faddr, slo, mmu_idx, false) && - v7m_stack_write(cpu, faddr + 4, shi, mmu_idx, false); + v7m_stack_write(cpu, faddr, slo, + mmu_idx, STACK_NORMAL) && + v7m_stack_write(cpu, faddr + 4, shi, + mmu_idx, STACK_NORMAL); } stacked_ok = stacked_ok && v7m_stack_write(cpu, frameptr + 0x60, - vfp_get_fpscr(env), mmu_idx, false); + vfp_get_fpscr(env), mmu_idx, STACK_NORMAL); if (cpacr_pass) { for (i = 0; i < ((framesize == 0xa8) ? 32 : 16); i += 2) { *aa32_vfp_dreg(env, i / 2) = 0; -- 2.20.1 The M-profile architecture floating point system supports lazy FP state preservation, where FP registers are not pushed to the stack when an exception occurs but are instead only saved if and when the first FP instruction in the exception handler is executed. Implement this in QEMU, corresponding to the check of LSPACT in the pseudocode ExecuteFPCheck(). Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20190416125744.27770-24-peter.maydell@linaro.org --- target/arm/cpu.h | 3 ++ target/arm/helper.h | 2 + target/arm/translate.h | 1 + target/arm/helper.c | 112 +++++++++++++++++++++++++++++++++++++++++ target/arm/translate.c | 22 ++++++++ 5 files changed, 140 insertions(+) diff --git a/target/arm/cpu.h b/target/arm/cpu.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -XXX,XX +XXX,XX @@ #define EXCP_NOCP 17 /* v7M NOCP UsageFault */ #define EXCP_INVSTATE 18 /* v7M INVSTATE UsageFault */ #define EXCP_STKOF 19 /* v8M STKOF UsageFault */ +#define EXCP_LAZYFP 20 /* v7M fault during lazy FP stacking */ /* NB: add new EXCP_ defines to the array in arm_log_exception() too */ #define ARMV7M_EXCP_RESET 1 @@ -XXX,XX +XXX,XX @@ FIELD(TBFLAG_A32, NS, 6, 1) FIELD(TBFLAG_A32, VFPEN, 7, 1) FIELD(TBFLAG_A32, CONDEXEC, 8, 8) FIELD(TBFLAG_A32, SCTLR_B, 16, 1) +/* For M profile only, set if FPCCR.LSPACT is set */ +FIELD(TBFLAG_A32, LSPACT, 18, 1) /* For M profile only, set if we must create a new FP context */ FIELD(TBFLAG_A32, NEW_FP_CTXT_NEEDED, 19, 1) /* For M profile only, set if FPCCR.S does not match current security state */ diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_2(v7m_blxns, void, env, i32) DEF_HELPER_3(v7m_tt, i32, env, i32, i32) +DEF_HELPER_1(v7m_preserve_fp_state, void, env) + DEF_HELPER_2(v8m_stackcheck, void, env, i32) DEF_HELPER_4(access_check_cp_reg, void, env, ptr, i32, i32) diff --git a/target/arm/translate.h b/target/arm/translate.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate.h +++ b/target/arm/translate.h @@ -XXX,XX +XXX,XX @@ typedef struct DisasContext { bool v8m_stackcheck; /* true if we need to perform v8M stack limit checks */ bool v8m_fpccr_s_wrong; /* true if v8M FPCCR.S != v8m_secure */ bool v7m_new_fp_ctxt_needed; /* ASPEN set but no active FP context */ + bool v7m_lspact; /* FPCCR.LSPACT set */ /* Immediate value in AArch32 SVC insn; must be set if is_jmp == DISAS_SWI * so that top level loop can generate correct syndrome information. */ diff --git a/target/arm/helper.c b/target/arm/helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -XXX,XX +XXX,XX @@ void HELPER(v7m_blxns)(CPUARMState *env, uint32_t dest) g_assert_not_reached(); } +void HELPER(v7m_preserve_fp_state)(CPUARMState *env) +{ + /* translate.c should never generate calls here in user-only mode */ + g_assert_not_reached(); +} + uint32_t HELPER(v7m_tt)(CPUARMState *env, uint32_t addr, uint32_t op) { /* The TT instructions can be used by unprivileged code, but in @@ -XXX,XX +XXX,XX @@ pend_fault: return false; } +void HELPER(v7m_preserve_fp_state)(CPUARMState *env) +{ + /* + * Preserve FP state (because LSPACT was set and we are about + * to execute an FP instruction). This corresponds to the + * PreserveFPState() pseudocode. + * We may throw an exception if the stacking fails. + */ + ARMCPU *cpu = arm_env_get_cpu(env); + bool is_secure = env->v7m.fpccr[M_REG_S] & R_V7M_FPCCR_S_MASK; + bool negpri = !(env->v7m.fpccr[M_REG_S] & R_V7M_FPCCR_HFRDY_MASK); + bool is_priv = !(env->v7m.fpccr[is_secure] & R_V7M_FPCCR_USER_MASK); + bool splimviol = env->v7m.fpccr[is_secure] & R_V7M_FPCCR_SPLIMVIOL_MASK; + uint32_t fpcar = env->v7m.fpcar[is_secure]; + bool stacked_ok = true; + bool ts = is_secure && (env->v7m.fpccr[M_REG_S] & R_V7M_FPCCR_TS_MASK); + bool take_exception; + + /* Take the iothread lock as we are going to touch the NVIC */ + qemu_mutex_lock_iothread(); + + /* Check the background context had access to the FPU */ + if (!v7m_cpacr_pass(env, is_secure, is_priv)) { + armv7m_nvic_set_pending_lazyfp(env->nvic, ARMV7M_EXCP_USAGE, is_secure); + env->v7m.cfsr[is_secure] |= R_V7M_CFSR_NOCP_MASK; + stacked_ok = false; + } else if (!is_secure && !extract32(env->v7m.nsacr, 10, 1)) { + armv7m_nvic_set_pending_lazyfp(env->nvic, ARMV7M_EXCP_USAGE, M_REG_S); + env->v7m.cfsr[M_REG_S] |= R_V7M_CFSR_NOCP_MASK; + stacked_ok = false; + } + + if (!splimviol && stacked_ok) { + /* We only stack if the stack limit wasn't violated */ + int i; + ARMMMUIdx mmu_idx; + + mmu_idx = arm_v7m_mmu_idx_all(env, is_secure, is_priv, negpri); + for (i = 0; i < (ts ? 32 : 16); i += 2) { + uint64_t dn = *aa32_vfp_dreg(env, i / 2); + uint32_t faddr = fpcar + 4 * i; + uint32_t slo = extract64(dn, 0, 32); + uint32_t shi = extract64(dn, 32, 32); + + if (i >= 16) { + faddr += 8; /* skip the slot for the FPSCR */ + } + stacked_ok = stacked_ok && + v7m_stack_write(cpu, faddr, slo, mmu_idx, STACK_LAZYFP) && + v7m_stack_write(cpu, faddr + 4, shi, mmu_idx, STACK_LAZYFP); + } + + stacked_ok = stacked_ok && + v7m_stack_write(cpu, fpcar + 0x40, + vfp_get_fpscr(env), mmu_idx, STACK_LAZYFP); + } + + /* + * We definitely pended an exception, but it's possible that it + * might not be able to be taken now. If its priority permits us + * to take it now, then we must not update the LSPACT or FP regs, + * but instead jump out to take the exception immediately. + * If it's just pending and won't be taken until the current + * handler exits, then we do update LSPACT and the FP regs. + */ + take_exception = !stacked_ok && + armv7m_nvic_can_take_pending_exception(env->nvic); + + qemu_mutex_unlock_iothread(); + + if (take_exception) { + raise_exception_ra(env, EXCP_LAZYFP, 0, 1, GETPC()); + } + + env->v7m.fpccr[is_secure] &= ~R_V7M_FPCCR_LSPACT_MASK; + + if (ts) { + /* Clear s0 to s31 and the FPSCR */ + int i; + + for (i = 0; i < 32; i += 2) { + *aa32_vfp_dreg(env, i / 2) = 0; + } + vfp_set_fpscr(env, 0); + } + /* + * Otherwise s0 to s15 and FPSCR are UNKNOWN; we choose to leave them + * unchanged. + */ +} + /* Write to v7M CONTROL.SPSEL bit for the specified security bank. * This may change the current stack pointer between Main and Process * stack pointers if it is done for the CONTROL register for the current @@ -XXX,XX +XXX,XX @@ static void arm_log_exception(int idx) [EXCP_NOCP] = "v7M NOCP UsageFault", [EXCP_INVSTATE] = "v7M INVSTATE UsageFault", [EXCP_STKOF] = "v8M STKOF UsageFault", + [EXCP_LAZYFP] = "v7M exception during lazy FP stacking", }; if (idx >= 0 && idx < ARRAY_SIZE(excnames)) { @@ -XXX,XX +XXX,XX @@ void arm_v7m_cpu_do_interrupt(CPUState *cs) return; } break; + case EXCP_LAZYFP: + /* + * We already pended the specific exception in the NVIC in the + * v7m_preserve_fp_state() helper function. + */ + break; default: cpu_abort(cs, "Unhandled exception 0x%x\n", cs->exception_index); return; /* Never happens. Keep compiler happy. */ @@ -XXX,XX +XXX,XX @@ void cpu_get_tb_cpu_state(CPUARMState *env, target_ulong *pc, flags = FIELD_DP32(flags, TBFLAG_A32, NEW_FP_CTXT_NEEDED, 1); } + if (arm_feature(env, ARM_FEATURE_M)) { + bool is_secure = env->v7m.fpccr[M_REG_S] & R_V7M_FPCCR_S_MASK; + + if (env->v7m.fpccr[is_secure] & R_V7M_FPCCR_LSPACT_MASK) { + flags = FIELD_DP32(flags, TBFLAG_A32, LSPACT, 1); + } + } + *pflags = flags; *cs_base = 0; } diff --git a/target/arm/translate.c b/target/arm/translate.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate.c +++ b/target/arm/translate.c @@ -XXX,XX +XXX,XX @@ static int disas_vfp_insn(DisasContext *s, uint32_t insn) if (arm_dc_feature(s, ARM_FEATURE_M)) { /* Handle M-profile lazy FP state mechanics */ + /* Trigger lazy-state preservation if necessary */ + if (s->v7m_lspact) { + /* + * Lazy state saving affects external memory and also the NVIC, + * so we must mark it as an IO operation for icount. + */ + if (tb_cflags(s->base.tb) & CF_USE_ICOUNT) { + gen_io_start(); + } + gen_helper_v7m_preserve_fp_state(cpu_env); + if (tb_cflags(s->base.tb) & CF_USE_ICOUNT) { + gen_io_end(); + } + /* + * If the preserve_fp_state helper doesn't throw an exception + * then it will clear LSPACT; we don't need to repeat this for + * any further FP insns in this TB. + */ + s->v7m_lspact = false; + } + /* Update ownership of FP context: set FPCCR.S to match current state */ if (s->v8m_fpccr_s_wrong) { TCGv_i32 tmp; @@ -XXX,XX +XXX,XX @@ static void arm_tr_init_disas_context(DisasContextBase *dcbase, CPUState *cs) dc->v8m_fpccr_s_wrong = FIELD_EX32(tb_flags, TBFLAG_A32, FPCCR_S_WRONG); dc->v7m_new_fp_ctxt_needed = FIELD_EX32(tb_flags, TBFLAG_A32, NEW_FP_CTXT_NEEDED); + dc->v7m_lspact = FIELD_EX32(tb_flags, TBFLAG_A32, LSPACT); dc->cp_regs = cpu->cp_regs; dc->features = env->features; -- 2.20.1 Implement the VLSTM instruction for v7M for the FPU present case. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20190416125744.27770-25-peter.maydell@linaro.org --- target/arm/cpu.h | 2 + target/arm/helper.h | 2 + target/arm/helper.c | 84 ++++++++++++++++++++++++++++++++++++++++++ target/arm/translate.c | 15 +++++++- 4 files changed, 102 insertions(+), 1 deletion(-) diff --git a/target/arm/cpu.h b/target/arm/cpu.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -XXX,XX +XXX,XX @@ #define EXCP_INVSTATE 18 /* v7M INVSTATE UsageFault */ #define EXCP_STKOF 19 /* v8M STKOF UsageFault */ #define EXCP_LAZYFP 20 /* v7M fault during lazy FP stacking */ +#define EXCP_LSERR 21 /* v8M LSERR SecureFault */ +#define EXCP_UNALIGNED 22 /* v7M UNALIGNED UsageFault */ /* NB: add new EXCP_ defines to the array in arm_log_exception() too */ #define ARMV7M_EXCP_RESET 1 diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_3(v7m_tt, i32, env, i32, i32) DEF_HELPER_1(v7m_preserve_fp_state, void, env) +DEF_HELPER_2(v7m_vlstm, void, env, i32) + DEF_HELPER_2(v8m_stackcheck, void, env, i32) DEF_HELPER_4(access_check_cp_reg, void, env, ptr, i32, i32) diff --git a/target/arm/helper.c b/target/arm/helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -XXX,XX +XXX,XX @@ void HELPER(v7m_preserve_fp_state)(CPUARMState *env) g_assert_not_reached(); } +void HELPER(v7m_vlstm)(CPUARMState *env, uint32_t fptr) +{ + /* translate.c should never generate calls here in user-only mode */ + g_assert_not_reached(); +} + uint32_t HELPER(v7m_tt)(CPUARMState *env, uint32_t addr, uint32_t op) { /* The TT instructions can be used by unprivileged code, but in @@ -XXX,XX +XXX,XX @@ static void v7m_update_fpccr(CPUARMState *env, uint32_t frameptr, } } +void HELPER(v7m_vlstm)(CPUARMState *env, uint32_t fptr) +{ + /* fptr is the value of Rn, the frame pointer we store the FP regs to */ + bool s = env->v7m.fpccr[M_REG_S] & R_V7M_FPCCR_S_MASK; + bool lspact = env->v7m.fpccr[s] & R_V7M_FPCCR_LSPACT_MASK; + + assert(env->v7m.secure); + + if (!(env->v7m.control[M_REG_S] & R_V7M_CONTROL_SFPA_MASK)) { + return; + } + + /* Check access to the coprocessor is permitted */ + if (!v7m_cpacr_pass(env, true, arm_current_el(env) != 0)) { + raise_exception_ra(env, EXCP_NOCP, 0, 1, GETPC()); + } + + if (lspact) { + /* LSPACT should not be active when there is active FP state */ + raise_exception_ra(env, EXCP_LSERR, 0, 1, GETPC()); + } + + if (fptr & 7) { + raise_exception_ra(env, EXCP_UNALIGNED, 0, 1, GETPC()); + } + + /* + * Note that we do not use v7m_stack_write() here, because the + * accesses should not set the FSR bits for stacking errors if they + * fail. (In pseudocode terms, they are AccType_NORMAL, not AccType_STACK + * or AccType_LAZYFP). Faults in cpu_stl_data() will throw exceptions + * and longjmp out. + */ + if (!(env->v7m.fpccr[M_REG_S] & R_V7M_FPCCR_LSPEN_MASK)) { + bool ts = env->v7m.fpccr[M_REG_S] & R_V7M_FPCCR_TS_MASK; + int i; + + for (i = 0; i < (ts ? 32 : 16); i += 2) { + uint64_t dn = *aa32_vfp_dreg(env, i / 2); + uint32_t faddr = fptr + 4 * i; + uint32_t slo = extract64(dn, 0, 32); + uint32_t shi = extract64(dn, 32, 32); + + if (i >= 16) { + faddr += 8; /* skip the slot for the FPSCR */ + } + cpu_stl_data(env, faddr, slo); + cpu_stl_data(env, faddr + 4, shi); + } + cpu_stl_data(env, fptr + 0x40, vfp_get_fpscr(env)); + + /* + * If TS is 0 then s0 to s15 and FPSCR are UNKNOWN; we choose to + * leave them unchanged, matching our choice in v7m_preserve_fp_state. + */ + if (ts) { + for (i = 0; i < 32; i += 2) { + *aa32_vfp_dreg(env, i / 2) = 0; + } + vfp_set_fpscr(env, 0); + } + } else { + v7m_update_fpccr(env, fptr, false); + } + + env->v7m.control[M_REG_S] &= ~R_V7M_CONTROL_FPCA_MASK; +} + static bool v7m_push_stack(ARMCPU *cpu) { /* Do the "set up stack frame" part of exception entry, @@ -XXX,XX +XXX,XX @@ static void arm_log_exception(int idx) [EXCP_INVSTATE] = "v7M INVSTATE UsageFault", [EXCP_STKOF] = "v8M STKOF UsageFault", [EXCP_LAZYFP] = "v7M exception during lazy FP stacking", + [EXCP_LSERR] = "v8M LSERR UsageFault", + [EXCP_UNALIGNED] = "v7M UNALIGNED UsageFault", }; if (idx >= 0 && idx < ARRAY_SIZE(excnames)) { @@ -XXX,XX +XXX,XX @@ void arm_v7m_cpu_do_interrupt(CPUState *cs) armv7m_nvic_set_pending(env->nvic, ARMV7M_EXCP_USAGE, env->v7m.secure); env->v7m.cfsr[env->v7m.secure] |= R_V7M_CFSR_STKOF_MASK; break; + case EXCP_LSERR: + armv7m_nvic_set_pending(env->nvic, ARMV7M_EXCP_SECURE, false); + env->v7m.sfsr |= R_V7M_SFSR_LSERR_MASK; + break; + case EXCP_UNALIGNED: + armv7m_nvic_set_pending(env->nvic, ARMV7M_EXCP_USAGE, env->v7m.secure); + env->v7m.cfsr[env->v7m.secure] |= R_V7M_CFSR_UNALIGNED_MASK; + break; case EXCP_SWI: /* The PC already points to the next instruction. */ armv7m_nvic_set_pending(env->nvic, ARMV7M_EXCP_SVC, env->v7m.secure); diff --git a/target/arm/translate.c b/target/arm/translate.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate.c +++ b/target/arm/translate.c @@ -XXX,XX +XXX,XX @@ static void disas_thumb2_insn(DisasContext *s, uint32_t insn) if (!s->v8m_secure || (insn & 0x0040f0ff)) { goto illegal_op; } - /* Just NOP since FP support is not implemented */ + + if (arm_dc_feature(s, ARM_FEATURE_VFP)) { + TCGv_i32 fptr = load_reg(s, rn); + + if (extract32(insn, 20, 1)) { + /* VLLDM */ + } else { + gen_helper_v7m_vlstm(cpu_env, fptr); + } + tcg_temp_free_i32(fptr); + + /* End the TB, because we have updated FP control bits */ + s->base.is_jmp = DISAS_UPDATE; + } break; } if (arm_dc_feature(s, ARM_FEATURE_VFP) && -- 2.20.1 Implement the VLLDM instruction for v7M for the FPU present cas. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20190416125744.27770-26-peter.maydell@linaro.org --- target/arm/helper.h | 1 + target/arm/helper.c | 54 ++++++++++++++++++++++++++++++++++++++++++ target/arm/translate.c | 2 +- 3 files changed, 56 insertions(+), 1 deletion(-) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_3(v7m_tt, i32, env, i32, i32) DEF_HELPER_1(v7m_preserve_fp_state, void, env) DEF_HELPER_2(v7m_vlstm, void, env, i32) +DEF_HELPER_2(v7m_vlldm, void, env, i32) DEF_HELPER_2(v8m_stackcheck, void, env, i32) diff --git a/target/arm/helper.c b/target/arm/helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -XXX,XX +XXX,XX @@ void HELPER(v7m_vlstm)(CPUARMState *env, uint32_t fptr) g_assert_not_reached(); } +void HELPER(v7m_vlldm)(CPUARMState *env, uint32_t fptr) +{ + /* translate.c should never generate calls here in user-only mode */ + g_assert_not_reached(); +} + uint32_t HELPER(v7m_tt)(CPUARMState *env, uint32_t addr, uint32_t op) { /* The TT instructions can be used by unprivileged code, but in @@ -XXX,XX +XXX,XX @@ void HELPER(v7m_vlstm)(CPUARMState *env, uint32_t fptr) env->v7m.control[M_REG_S] &= ~R_V7M_CONTROL_FPCA_MASK; } +void HELPER(v7m_vlldm)(CPUARMState *env, uint32_t fptr) +{ + /* fptr is the value of Rn, the frame pointer we load the FP regs from */ + assert(env->v7m.secure); + + if (!(env->v7m.control[M_REG_S] & R_V7M_CONTROL_SFPA_MASK)) { + return; + } + + /* Check access to the coprocessor is permitted */ + if (!v7m_cpacr_pass(env, true, arm_current_el(env) != 0)) { + raise_exception_ra(env, EXCP_NOCP, 0, 1, GETPC()); + } + + if (env->v7m.fpccr[M_REG_S] & R_V7M_FPCCR_LSPACT_MASK) { + /* State in FP is still valid */ + env->v7m.fpccr[M_REG_S] &= ~R_V7M_FPCCR_LSPACT_MASK; + } else { + bool ts = env->v7m.fpccr[M_REG_S] & R_V7M_FPCCR_TS_MASK; + int i; + uint32_t fpscr; + + if (fptr & 7) { + raise_exception_ra(env, EXCP_UNALIGNED, 0, 1, GETPC()); + } + + for (i = 0; i < (ts ? 32 : 16); i += 2) { + uint32_t slo, shi; + uint64_t dn; + uint32_t faddr = fptr + 4 * i; + + if (i >= 16) { + faddr += 8; /* skip the slot for the FPSCR */ + } + + slo = cpu_ldl_data(env, faddr); + shi = cpu_ldl_data(env, faddr + 4); + + dn = (uint64_t) shi << 32 | slo; + *aa32_vfp_dreg(env, i / 2) = dn; + } + fpscr = cpu_ldl_data(env, fptr + 0x40); + vfp_set_fpscr(env, fpscr); + } + + env->v7m.control[M_REG_S] |= R_V7M_CONTROL_FPCA_MASK; +} + static bool v7m_push_stack(ARMCPU *cpu) { /* Do the "set up stack frame" part of exception entry, diff --git a/target/arm/translate.c b/target/arm/translate.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate.c +++ b/target/arm/translate.c @@ -XXX,XX +XXX,XX @@ static void disas_thumb2_insn(DisasContext *s, uint32_t insn) TCGv_i32 fptr = load_reg(s, rn); if (extract32(insn, 20, 1)) { - /* VLLDM */ + gen_helper_v7m_vlldm(cpu_env, fptr); } else { gen_helper_v7m_vlstm(cpu_env, fptr); } -- 2.20.1 Enable the FPU by default for the Cortex-M4 and Cortex-M33. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20190416125744.27770-27-peter.maydell@linaro.org --- target/arm/cpu.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/target/arm/cpu.c b/target/arm/cpu.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpu.c +++ b/target/arm/cpu.c @@ -XXX,XX +XXX,XX @@ static void cortex_m4_initfn(Object *obj) set_feature(&cpu->env, ARM_FEATURE_M); set_feature(&cpu->env, ARM_FEATURE_M_MAIN); set_feature(&cpu->env, ARM_FEATURE_THUMB_DSP); + set_feature(&cpu->env, ARM_FEATURE_VFP4); cpu->midr = 0x410fc240; /* r0p0 */ cpu->pmsav7_dregion = 8; + cpu->isar.mvfr0 = 0x10110021; + cpu->isar.mvfr1 = 0x11000011; + cpu->isar.mvfr2 = 0x00000000; cpu->id_pfr0 = 0x00000030; cpu->id_pfr1 = 0x00000200; cpu->id_dfr0 = 0x00100000; @@ -XXX,XX +XXX,XX @@ static void cortex_m33_initfn(Object *obj) set_feature(&cpu->env, ARM_FEATURE_M_MAIN); set_feature(&cpu->env, ARM_FEATURE_M_SECURITY); set_feature(&cpu->env, ARM_FEATURE_THUMB_DSP); + set_feature(&cpu->env, ARM_FEATURE_VFP4); cpu->midr = 0x410fd213; /* r0p3 */ cpu->pmsav7_dregion = 16; cpu->sau_sregion = 8; + cpu->isar.mvfr0 = 0x10110021; + cpu->isar.mvfr1 = 0x11000011; + cpu->isar.mvfr2 = 0x00000040; cpu->id_pfr0 = 0x00000030; cpu->id_pfr1 = 0x00000210; cpu->id_dfr0 = 0x00200000; -- 2.20.1 From: Philippe Mathieu-Daudé <philmd@redhat.com> This device is used by both ARM (BCM2836, for raspi2) and AArch64 (BCM2837, for raspi3) targets, and is not CPU-specific. Move it to common object, so we build it once for all targets. Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com> Message-id: 20190427133028.12874-1-philmd@redhat.com Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- hw/dma/Makefile.objs | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/hw/dma/Makefile.objs b/hw/dma/Makefile.objs index XXXXXXX..XXXXXXX 100644 --- a/hw/dma/Makefile.objs +++ b/hw/dma/Makefile.objs @@ -XXX,XX +XXX,XX @@ common-obj-$(CONFIG_XLNX_ZYNQMP_ARM) += xlnx-zdma.o obj-$(CONFIG_OMAP) += omap_dma.o soc_dma.o obj-$(CONFIG_PXA2XX) += pxa2xx_dma.o -obj-$(CONFIG_RASPI) += bcm2835_dma.o +common-obj-$(CONFIG_RASPI) += bcm2835_dma.o -- 2.20.1 From: Philippe Mathieu-Daudé <philmd@redhat.com> Reviewed-by: Thomas Huth <thuth@redhat.com> Reviewed-by: Cédric Le Goater <clg@kaod.org> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com> Message-id: 20190412165416.7977-2-philmd@redhat.com Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- hw/arm/aspeed.c | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/hw/arm/aspeed.c b/hw/arm/aspeed.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/aspeed.c +++ b/hw/arm/aspeed.c @@ -XXX,XX +XXX,XX @@ #include "hw/arm/aspeed_soc.h" #include "hw/boards.h" #include "hw/i2c/smbus_eeprom.h" +#include "hw/misc/pca9552.h" +#include "hw/misc/tmp105.h" #include "qemu/log.h" #include "sysemu/block-backend.h" #include "hw/loader.h" @@ -XXX,XX +XXX,XX @@ static void ast2500_evb_i2c_init(AspeedBoardState *bmc) eeprom_buf); /* The AST2500 EVB expects a LM75 but a TMP105 is compatible */ - i2c_create_slave(aspeed_i2c_get_bus(DEVICE(&soc->i2c), 7), "tmp105", 0x4d); + i2c_create_slave(aspeed_i2c_get_bus(DEVICE(&soc->i2c), 7), + TYPE_TMP105, 0x4d); /* The AST2500 EVB does not have an RTC. Let's pretend that one is * plugged on the I2C bus header */ @@ -XXX,XX +XXX,XX @@ static void witherspoon_bmc_i2c_init(AspeedBoardState *bmc) AspeedSoCState *soc = &bmc->soc; uint8_t *eeprom_buf = g_malloc0(8 * 1024); - i2c_create_slave(aspeed_i2c_get_bus(DEVICE(&soc->i2c), 3), "pca9552", 0x60); + i2c_create_slave(aspeed_i2c_get_bus(DEVICE(&soc->i2c), 3), TYPE_PCA9552, + 0x60); i2c_create_slave(aspeed_i2c_get_bus(DEVICE(&soc->i2c), 4), "tmp423", 0x4c); i2c_create_slave(aspeed_i2c_get_bus(DEVICE(&soc->i2c), 5), "tmp423", 0x4c); /* The Witherspoon expects a TMP275 but a TMP105 is compatible */ - i2c_create_slave(aspeed_i2c_get_bus(DEVICE(&soc->i2c), 9), "tmp105", 0x4a); + i2c_create_slave(aspeed_i2c_get_bus(DEVICE(&soc->i2c), 9), TYPE_TMP105, + 0x4a); /* The witherspoon board expects Epson RX8900 I2C RTC but a ds1338 is * good enough */ @@ -XXX,XX +XXX,XX @@ static void witherspoon_bmc_i2c_init(AspeedBoardState *bmc) smbus_eeprom_init_one(aspeed_i2c_get_bus(DEVICE(&soc->i2c), 11), 0x51, eeprom_buf); - i2c_create_slave(aspeed_i2c_get_bus(DEVICE(&soc->i2c), 11), "pca9552", + i2c_create_slave(aspeed_i2c_get_bus(DEVICE(&soc->i2c), 11), TYPE_PCA9552, 0x60); } -- 2.20.1 From: Philippe Mathieu-Daudé <philmd@redhat.com> Suggested-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com> Message-id: 20190412165416.7977-3-philmd@redhat.com Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- hw/arm/nseries.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/hw/arm/nseries.c b/hw/arm/nseries.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/nseries.c +++ b/hw/arm/nseries.c @@ -XXX,XX +XXX,XX @@ #include "hw/boards.h" #include "hw/i2c/i2c.h" #include "hw/devices.h" +#include "hw/misc/tmp105.h" #include "hw/block/flash.h" #include "hw/hw.h" #include "hw/bt.h" @@ -XXX,XX +XXX,XX @@ static void n8x0_i2c_setup(struct n800_s *s) qemu_register_powerdown_notifier(&n8x0_system_powerdown_notifier); /* Attach a TMP105 PM chip (A0 wired to ground) */ - dev = i2c_create_slave(i2c, "tmp105", N8X0_TMP105_ADDR); + dev = i2c_create_slave(i2c, TYPE_TMP105, N8X0_TMP105_ADDR); qdev_connect_gpio_out(dev, 0, tmp_irq); } -- 2.20.1 From: Philippe Mathieu-Daudé <philmd@redhat.com> No code used the tc6393xb_gpio_in_get() and tc6393xb_gpio_out_set() functions since their introduction in commit 88d2c950b002. Time to remove them. Suggested-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com> Message-id: 20190412165416.7977-4-philmd@redhat.com Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- include/hw/devices.h | 3 --- hw/display/tc6393xb.c | 16 ---------------- 2 files changed, 19 deletions(-) diff --git a/include/hw/devices.h b/include/hw/devices.h index XXXXXXX..XXXXXXX 100644 --- a/include/hw/devices.h +++ b/include/hw/devices.h @@ -XXX,XX +XXX,XX @@ void retu_key_event(void *retu, int state); typedef struct TC6393xbState TC6393xbState; TC6393xbState *tc6393xb_init(struct MemoryRegion *sysmem, uint32_t base, qemu_irq irq); -void tc6393xb_gpio_out_set(TC6393xbState *s, int line, - qemu_irq handler); -qemu_irq *tc6393xb_gpio_in_get(TC6393xbState *s); qemu_irq tc6393xb_l3v_get(TC6393xbState *s); #endif diff --git a/hw/display/tc6393xb.c b/hw/display/tc6393xb.c index XXXXXXX..XXXXXXX 100644 --- a/hw/display/tc6393xb.c +++ b/hw/display/tc6393xb.c @@ -XXX,XX +XXX,XX @@ struct TC6393xbState { blanked : 1; }; -qemu_irq *tc6393xb_gpio_in_get(TC6393xbState *s) -{ - return s->gpio_in; -} - static void tc6393xb_gpio_set(void *opaque, int line, int level) { // TC6393xbState *s = opaque; @@ -XXX,XX +XXX,XX @@ static void tc6393xb_gpio_set(void *opaque, int line, int level) // FIXME: how does the chip reflect the GPIO input level change? } -void tc6393xb_gpio_out_set(TC6393xbState *s, int line, - qemu_irq handler) -{ - if (line >= TC6393XB_GPIOS) { - fprintf(stderr, "TC6393xb: no GPIO pin %d\n", line); - return; - } - - s->handler[line] = handler; -} - static void tc6393xb_gpio_handler_update(TC6393xbState *s) { uint32_t level, diff; -- 2.20.1 From: Philippe Mathieu-Daudé <philmd@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com> Message-id: 20190412165416.7977-5-philmd@redhat.com Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- include/hw/devices.h | 6 ------ include/hw/display/tc6393xb.h | 24 ++++++++++++++++++++++++ hw/arm/tosa.c | 2 +- hw/display/tc6393xb.c | 2 +- MAINTAINERS | 1 + 5 files changed, 27 insertions(+), 8 deletions(-) create mode 100644 include/hw/display/tc6393xb.h diff --git a/include/hw/devices.h b/include/hw/devices.h index XXXXXXX..XXXXXXX 100644 --- a/include/hw/devices.h +++ b/include/hw/devices.h @@ -XXX,XX +XXX,XX @@ void *tahvo_init(qemu_irq irq, int betty); void retu_key_event(void *retu, int state); -/* tc6393xb.c */ -typedef struct TC6393xbState TC6393xbState; -TC6393xbState *tc6393xb_init(struct MemoryRegion *sysmem, - uint32_t base, qemu_irq irq); -qemu_irq tc6393xb_l3v_get(TC6393xbState *s); - #endif diff --git a/include/hw/display/tc6393xb.h b/include/hw/display/tc6393xb.h new file mode 100644 index XXXXXXX..XXXXXXX --- /dev/null +++ b/include/hw/display/tc6393xb.h @@ -XXX,XX +XXX,XX @@ +/* + * Toshiba TC6393XB I/O Controller. + * Found in Sharp Zaurus SL-6000 (tosa) or some + * Toshiba e-Series PDAs. + * + * Copyright (c) 2007 Hervé Poussineau + * + * This work is licensed under the terms of the GNU GPL, version 2 or later. + * See the COPYING file in the top-level directory. + */ + +#ifndef HW_DISPLAY_TC6393XB_H +#define HW_DISPLAY_TC6393XB_H + +#include "exec/memory.h" +#include "hw/irq.h" + +typedef struct TC6393xbState TC6393xbState; + +TC6393xbState *tc6393xb_init(struct MemoryRegion *sysmem, + uint32_t base, qemu_irq irq); +qemu_irq tc6393xb_l3v_get(TC6393xbState *s); + +#endif diff --git a/hw/arm/tosa.c b/hw/arm/tosa.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/tosa.c +++ b/hw/arm/tosa.c @@ -XXX,XX +XXX,XX @@ #include "hw/hw.h" #include "hw/arm/pxa.h" #include "hw/arm/arm.h" -#include "hw/devices.h" #include "hw/arm/sharpsl.h" #include "hw/pcmcia.h" #include "hw/boards.h" +#include "hw/display/tc6393xb.h" #include "hw/i2c/i2c.h" #include "hw/ssi/ssi.h" #include "hw/sysbus.h" diff --git a/hw/display/tc6393xb.c b/hw/display/tc6393xb.c index XXXXXXX..XXXXXXX 100644 --- a/hw/display/tc6393xb.c +++ b/hw/display/tc6393xb.c @@ -XXX,XX +XXX,XX @@ #include "qapi/error.h" #include "qemu/host-utils.h" #include "hw/hw.h" -#include "hw/devices.h" +#include "hw/display/tc6393xb.h" #include "hw/block/flash.h" #include "ui/console.h" #include "ui/pixel_ops.h" diff --git a/MAINTAINERS b/MAINTAINERS index XXXXXXX..XXXXXXX 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -XXX,XX +XXX,XX @@ F: hw/misc/mst_fpga.c F: hw/misc/max111x.c F: include/hw/arm/pxa.h F: include/hw/arm/sharpsl.h +F: include/hw/display/tc6393xb.h SABRELITE / i.MX6 M: Peter Maydell <peter.maydell@linaro.org> -- 2.20.1 From: Philippe Mathieu-Daudé <philmd@redhat.com> Add an entries the Blizzard device in MAINTAINERS. Reviewed-by: Thomas Huth <thuth@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com> Message-id: 20190412165416.7977-6-philmd@redhat.com Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- include/hw/devices.h | 7 ------- include/hw/display/blizzard.h | 22 ++++++++++++++++++++++ hw/arm/nseries.c | 1 + hw/display/blizzard.c | 2 +- MAINTAINERS | 2 ++ 5 files changed, 26 insertions(+), 8 deletions(-) create mode 100644 include/hw/display/blizzard.h diff --git a/include/hw/devices.h b/include/hw/devices.h index XXXXXXX..XXXXXXX 100644 --- a/include/hw/devices.h +++ b/include/hw/devices.h @@ -XXX,XX +XXX,XX @@ void tsc2005_set_transform(void *opaque, MouseTransformInfo *info); /* stellaris_input.c */ void stellaris_gamepad_init(int n, qemu_irq *irq, const int *keycode); -/* blizzard.c */ -void *s1d13745_init(qemu_irq gpio_int); -void s1d13745_write(void *opaque, int dc, uint16_t value); -void s1d13745_write_block(void *opaque, int dc, - void *buf, size_t len, int pitch); -uint16_t s1d13745_read(void *opaque, int dc); - /* cbus.c */ typedef struct { qemu_irq clk; diff --git a/include/hw/display/blizzard.h b/include/hw/display/blizzard.h new file mode 100644 index XXXXXXX..XXXXXXX --- /dev/null +++ b/include/hw/display/blizzard.h @@ -XXX,XX +XXX,XX @@ +/* + * Epson S1D13744/S1D13745 (Blizzard/Hailstorm/Tornado) LCD/TV controller. + * + * Copyright (C) 2008 Nokia Corporation + * Written by Andrzej Zaborowski + * + * This work is licensed under the terms of the GNU GPL, version 2 or later. + * See the COPYING file in the top-level directory. + */ + +#ifndef HW_DISPLAY_BLIZZARD_H +#define HW_DISPLAY_BLIZZARD_H + +#include "hw/irq.h" + +void *s1d13745_init(qemu_irq gpio_int); +void s1d13745_write(void *opaque, int dc, uint16_t value); +void s1d13745_write_block(void *opaque, int dc, + void *buf, size_t len, int pitch); +uint16_t s1d13745_read(void *opaque, int dc); + +#endif diff --git a/hw/arm/nseries.c b/hw/arm/nseries.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/nseries.c +++ b/hw/arm/nseries.c @@ -XXX,XX +XXX,XX @@ #include "hw/boards.h" #include "hw/i2c/i2c.h" #include "hw/devices.h" +#include "hw/display/blizzard.h" #include "hw/misc/tmp105.h" #include "hw/block/flash.h" #include "hw/hw.h" diff --git a/hw/display/blizzard.c b/hw/display/blizzard.c index XXXXXXX..XXXXXXX 100644 --- a/hw/display/blizzard.c +++ b/hw/display/blizzard.c @@ -XXX,XX +XXX,XX @@ #include "qemu/osdep.h" #include "qemu-common.h" #include "ui/console.h" -#include "hw/devices.h" +#include "hw/display/blizzard.h" #include "ui/pixel_ops.h" typedef void (*blizzard_fn_t)(uint8_t *, const uint8_t *, unsigned int); diff --git a/MAINTAINERS b/MAINTAINERS index XXXXXXX..XXXXXXX 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -XXX,XX +XXX,XX @@ M: Peter Maydell <peter.maydell@linaro.org> L: qemu-arm@nongnu.org S: Odd Fixes F: hw/arm/nseries.c +F: hw/display/blizzard.c F: hw/input/lm832x.c F: hw/input/tsc2005.c F: hw/misc/cbus.c F: hw/timer/twl92230.c +F: include/hw/display/blizzard.h Palm M: Andrzej Zaborowski <balrogg@gmail.com> -- 2.20.1 From: Philippe Mathieu-Daudé <philmd@redhat.com> Reviewed-by: Thomas Huth <thuth@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com> Message-id: 20190412165416.7977-7-philmd@redhat.com Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- include/hw/devices.h | 14 -------------- include/hw/misc/cbus.h | 32 ++++++++++++++++++++++++++++++++ hw/arm/nseries.c | 1 + hw/misc/cbus.c | 2 +- MAINTAINERS | 1 + 5 files changed, 35 insertions(+), 15 deletions(-) create mode 100644 include/hw/misc/cbus.h diff --git a/include/hw/devices.h b/include/hw/devices.h index XXXXXXX..XXXXXXX 100644 --- a/include/hw/devices.h +++ b/include/hw/devices.h @@ -XXX,XX +XXX,XX @@ void tsc2005_set_transform(void *opaque, MouseTransformInfo *info); /* stellaris_input.c */ void stellaris_gamepad_init(int n, qemu_irq *irq, const int *keycode); -/* cbus.c */ -typedef struct { - qemu_irq clk; - qemu_irq dat; - qemu_irq sel; -} CBus; -CBus *cbus_init(qemu_irq dat_out); -void cbus_attach(CBus *bus, void *slave_opaque); - -void *retu_init(qemu_irq irq, int vilma); -void *tahvo_init(qemu_irq irq, int betty); - -void retu_key_event(void *retu, int state); - #endif diff --git a/include/hw/misc/cbus.h b/include/hw/misc/cbus.h new file mode 100644 index XXXXXXX..XXXXXXX --- /dev/null +++ b/include/hw/misc/cbus.h @@ -XXX,XX +XXX,XX @@ +/* + * CBUS three-pin bus and the Retu / Betty / Tahvo / Vilma / Avilma / + * Hinku / Vinku / Ahne / Pihi chips used in various Nokia platforms. + * Based on reverse-engineering of a linux driver. + * + * Copyright (C) 2008 Nokia Corporation + * Written by Andrzej Zaborowski + * + * This work is licensed under the terms of the GNU GPL, version 2 or later. + * See the COPYING file in the top-level directory. + */ + +#ifndef HW_MISC_CBUS_H +#define HW_MISC_CBUS_H + +#include "hw/irq.h" + +typedef struct { + qemu_irq clk; + qemu_irq dat; + qemu_irq sel; +} CBus; + +CBus *cbus_init(qemu_irq dat_out); +void cbus_attach(CBus *bus, void *slave_opaque); + +void *retu_init(qemu_irq irq, int vilma); +void *tahvo_init(qemu_irq irq, int betty); + +void retu_key_event(void *retu, int state); + +#endif diff --git a/hw/arm/nseries.c b/hw/arm/nseries.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/nseries.c +++ b/hw/arm/nseries.c @@ -XXX,XX +XXX,XX @@ #include "hw/i2c/i2c.h" #include "hw/devices.h" #include "hw/display/blizzard.h" +#include "hw/misc/cbus.h" #include "hw/misc/tmp105.h" #include "hw/block/flash.h" #include "hw/hw.h" diff --git a/hw/misc/cbus.c b/hw/misc/cbus.c index XXXXXXX..XXXXXXX 100644 --- a/hw/misc/cbus.c +++ b/hw/misc/cbus.c @@ -XXX,XX +XXX,XX @@ #include "qemu/osdep.h" #include "hw/hw.h" #include "hw/irq.h" -#include "hw/devices.h" +#include "hw/misc/cbus.h" #include "sysemu/sysemu.h" //#define DEBUG diff --git a/MAINTAINERS b/MAINTAINERS index XXXXXXX..XXXXXXX 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -XXX,XX +XXX,XX @@ F: hw/input/tsc2005.c F: hw/misc/cbus.c F: hw/timer/twl92230.c F: include/hw/display/blizzard.h +F: include/hw/misc/cbus.h Palm M: Andrzej Zaborowski <balrogg@gmail.com> -- 2.20.1 From: Philippe Mathieu-Daudé <philmd@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com> Message-id: 20190412165416.7977-8-philmd@redhat.com Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- include/hw/devices.h | 3 --- include/hw/input/gamepad.h | 19 +++++++++++++++++++ hw/arm/stellaris.c | 2 +- hw/input/stellaris_input.c | 2 +- MAINTAINERS | 1 + 5 files changed, 22 insertions(+), 5 deletions(-) create mode 100644 include/hw/input/gamepad.h diff --git a/include/hw/devices.h b/include/hw/devices.h index XXXXXXX..XXXXXXX 100644 --- a/include/hw/devices.h +++ b/include/hw/devices.h @@ -XXX,XX +XXX,XX @@ void *tsc2005_init(qemu_irq pintdav); uint32_t tsc2005_txrx(void *opaque, uint32_t value, int len); void tsc2005_set_transform(void *opaque, MouseTransformInfo *info); -/* stellaris_input.c */ -void stellaris_gamepad_init(int n, qemu_irq *irq, const int *keycode); - #endif diff --git a/include/hw/input/gamepad.h b/include/hw/input/gamepad.h new file mode 100644 index XXXXXXX..XXXXXXX --- /dev/null +++ b/include/hw/input/gamepad.h @@ -XXX,XX +XXX,XX @@ +/* + * Gamepad style buttons connected to IRQ/GPIO lines + * + * Copyright (c) 2007 CodeSourcery. + * Written by Paul Brook + * + * This work is licensed under the terms of the GNU GPL, version 2 or later. + * See the COPYING file in the top-level directory. + */ + +#ifndef HW_INPUT_GAMEPAD_H +#define HW_INPUT_GAMEPAD_H + +#include "hw/irq.h" + +/* stellaris_input.c */ +void stellaris_gamepad_init(int n, qemu_irq *irq, const int *keycode); + +#endif diff --git a/hw/arm/stellaris.c b/hw/arm/stellaris.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/stellaris.c +++ b/hw/arm/stellaris.c @@ -XXX,XX +XXX,XX @@ #include "hw/sysbus.h" #include "hw/ssi/ssi.h" #include "hw/arm/arm.h" -#include "hw/devices.h" #include "qemu/timer.h" #include "hw/i2c/i2c.h" #include "net/net.h" @@ -XXX,XX +XXX,XX @@ #include "sysemu/sysemu.h" #include "hw/arm/armv7m.h" #include "hw/char/pl011.h" +#include "hw/input/gamepad.h" #include "hw/watchdog/cmsdk-apb-watchdog.h" #include "hw/misc/unimp.h" #include "cpu.h" diff --git a/hw/input/stellaris_input.c b/hw/input/stellaris_input.c index XXXXXXX..XXXXXXX 100644 --- a/hw/input/stellaris_input.c +++ b/hw/input/stellaris_input.c @@ -XXX,XX +XXX,XX @@ */ #include "qemu/osdep.h" #include "hw/hw.h" -#include "hw/devices.h" +#include "hw/input/gamepad.h" #include "ui/console.h" typedef struct { diff --git a/MAINTAINERS b/MAINTAINERS index XXXXXXX..XXXXXXX 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -XXX,XX +XXX,XX @@ M: Peter Maydell <peter.maydell@linaro.org> L: qemu-arm@nongnu.org S: Maintained F: hw/*/stellaris* +F: include/hw/input/gamepad.h Versatile Express M: Peter Maydell <peter.maydell@linaro.org> -- 2.20.1 From: Philippe Mathieu-Daudé <philmd@redhat.com> Since uWireSlave is only used in this new header, there is no need to expose it via "qemu/typedefs.h". Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com> Message-id: 20190412165416.7977-9-philmd@redhat.com Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- include/hw/arm/omap.h | 6 +----- include/hw/devices.h | 15 --------------- include/hw/input/tsc2xxx.h | 36 ++++++++++++++++++++++++++++++++++++ include/qemu/typedefs.h | 1 - hw/arm/nseries.c | 2 +- hw/arm/palm.c | 2 +- hw/input/tsc2005.c | 2 +- hw/input/tsc210x.c | 4 ++-- MAINTAINERS | 2 ++ 9 files changed, 44 insertions(+), 26 deletions(-) create mode 100644 include/hw/input/tsc2xxx.h diff --git a/include/hw/arm/omap.h b/include/hw/arm/omap.h index XXXXXXX..XXXXXXX 100644 --- a/include/hw/arm/omap.h +++ b/include/hw/arm/omap.h @@ -XXX,XX +XXX,XX @@ #include "exec/memory.h" # define hw_omap_h "omap.h" #include "hw/irq.h" +#include "hw/input/tsc2xxx.h" #include "target/arm/cpu-qom.h" #include "qemu/log.h" @@ -XXX,XX +XXX,XX @@ qemu_irq *omap_mpuio_in_get(struct omap_mpuio_s *s); void omap_mpuio_out_set(struct omap_mpuio_s *s, int line, qemu_irq handler); void omap_mpuio_key(struct omap_mpuio_s *s, int row, int col, int down); -struct uWireSlave { - uint16_t (*receive)(void *opaque); - void (*send)(void *opaque, uint16_t data); - void *opaque; -}; struct omap_uwire_s; void omap_uwire_attach(struct omap_uwire_s *s, uWireSlave *slave, int chipselect); diff --git a/include/hw/devices.h b/include/hw/devices.h index XXXXXXX..XXXXXXX 100644 --- a/include/hw/devices.h +++ b/include/hw/devices.h @@ -XXX,XX +XXX,XX @@ /* Devices that have nowhere better to go. */ #include "hw/hw.h" -#include "ui/console.h" /* smc91c111.c */ void smc91c111_init(NICInfo *, uint32_t, qemu_irq); @@ -XXX,XX +XXX,XX @@ void smc91c111_init(NICInfo *, uint32_t, qemu_irq); /* lan9118.c */ void lan9118_init(NICInfo *, uint32_t, qemu_irq); -/* tsc210x.c */ -uWireSlave *tsc2102_init(qemu_irq pint); -uWireSlave *tsc2301_init(qemu_irq penirq, qemu_irq kbirq, qemu_irq dav); -I2SCodec *tsc210x_codec(uWireSlave *chip); -uint32_t tsc210x_txrx(void *opaque, uint32_t value, int len); -void tsc210x_set_transform(uWireSlave *chip, - MouseTransformInfo *info); -void tsc210x_key_event(uWireSlave *chip, int key, int down); - -/* tsc2005.c */ -void *tsc2005_init(qemu_irq pintdav); -uint32_t tsc2005_txrx(void *opaque, uint32_t value, int len); -void tsc2005_set_transform(void *opaque, MouseTransformInfo *info); - #endif diff --git a/include/hw/input/tsc2xxx.h b/include/hw/input/tsc2xxx.h new file mode 100644 index XXXXXXX..XXXXXXX --- /dev/null +++ b/include/hw/input/tsc2xxx.h @@ -XXX,XX +XXX,XX @@ +/* + * TI touchscreen controller + * + * Copyright (c) 2006 Andrzej Zaborowski + * Copyright (C) 2008 Nokia Corporation + * + * This work is licensed under the terms of the GNU GPL, version 2 or later. + * See the COPYING file in the top-level directory. + */ + +#ifndef HW_INPUT_TSC2XXX_H +#define HW_INPUT_TSC2XXX_H + +#include "hw/irq.h" +#include "ui/console.h" + +typedef struct uWireSlave { + uint16_t (*receive)(void *opaque); + void (*send)(void *opaque, uint16_t data); + void *opaque; +} uWireSlave; + +/* tsc210x.c */ +uWireSlave *tsc2102_init(qemu_irq pint); +uWireSlave *tsc2301_init(qemu_irq penirq, qemu_irq kbirq, qemu_irq dav); +I2SCodec *tsc210x_codec(uWireSlave *chip); +uint32_t tsc210x_txrx(void *opaque, uint32_t value, int len); +void tsc210x_set_transform(uWireSlave *chip, MouseTransformInfo *info); +void tsc210x_key_event(uWireSlave *chip, int key, int down); + +/* tsc2005.c */ +void *tsc2005_init(qemu_irq pintdav); +uint32_t tsc2005_txrx(void *opaque, uint32_t value, int len); +void tsc2005_set_transform(void *opaque, MouseTransformInfo *info); + +#endif diff --git a/include/qemu/typedefs.h b/include/qemu/typedefs.h index XXXXXXX..XXXXXXX 100644 --- a/include/qemu/typedefs.h +++ b/include/qemu/typedefs.h @@ -XXX,XX +XXX,XX @@ typedef struct RAMBlock RAMBlock; typedef struct Range Range; typedef struct SHPCDevice SHPCDevice; typedef struct SSIBus SSIBus; -typedef struct uWireSlave uWireSlave; typedef struct VirtIODevice VirtIODevice; typedef struct Visitor Visitor; typedef void SaveStateHandler(QEMUFile *f, void *opaque); diff --git a/hw/arm/nseries.c b/hw/arm/nseries.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/nseries.c +++ b/hw/arm/nseries.c @@ -XXX,XX +XXX,XX @@ #include "ui/console.h" #include "hw/boards.h" #include "hw/i2c/i2c.h" -#include "hw/devices.h" #include "hw/display/blizzard.h" +#include "hw/input/tsc2xxx.h" #include "hw/misc/cbus.h" #include "hw/misc/tmp105.h" #include "hw/block/flash.h" diff --git a/hw/arm/palm.c b/hw/arm/palm.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/palm.c +++ b/hw/arm/palm.c @@ -XXX,XX +XXX,XX @@ #include "hw/arm/omap.h" #include "hw/boards.h" #include "hw/arm/arm.h" -#include "hw/devices.h" +#include "hw/input/tsc2xxx.h" #include "hw/loader.h" #include "exec/address-spaces.h" #include "cpu.h" diff --git a/hw/input/tsc2005.c b/hw/input/tsc2005.c index XXXXXXX..XXXXXXX 100644 --- a/hw/input/tsc2005.c +++ b/hw/input/tsc2005.c @@ -XXX,XX +XXX,XX @@ #include "hw/hw.h" #include "qemu/timer.h" #include "ui/console.h" -#include "hw/devices.h" +#include "hw/input/tsc2xxx.h" #include "trace.h" #define TSC_CUT_RESOLUTION(value, p) ((value) >> (16 - (p ? 12 : 10))) diff --git a/hw/input/tsc210x.c b/hw/input/tsc210x.c index XXXXXXX..XXXXXXX 100644 --- a/hw/input/tsc210x.c +++ b/hw/input/tsc210x.c @@ -XXX,XX +XXX,XX @@ #include "audio/audio.h" #include "qemu/timer.h" #include "ui/console.h" -#include "hw/arm/omap.h" /* For I2SCodec and uWireSlave */ -#include "hw/devices.h" +#include "hw/arm/omap.h" /* For I2SCodec */ +#include "hw/input/tsc2xxx.h" #define TSC_DATA_REGISTERS_PAGE 0x0 #define TSC_CONTROL_REGISTERS_PAGE 0x1 diff --git a/MAINTAINERS b/MAINTAINERS index XXXXXXX..XXXXXXX 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -XXX,XX +XXX,XX @@ F: hw/input/tsc2005.c F: hw/misc/cbus.c F: hw/timer/twl92230.c F: include/hw/display/blizzard.h +F: include/hw/input/tsc2xxx.h F: include/hw/misc/cbus.h Palm @@ -XXX,XX +XXX,XX @@ L: qemu-arm@nongnu.org S: Odd Fixes F: hw/arm/palm.c F: hw/input/tsc210x.c +F: include/hw/input/tsc2xxx.h Raspberry Pi M: Peter Maydell <peter.maydell@linaro.org> -- 2.20.1 From: Philippe Mathieu-Daudé <philmd@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com> Message-id: 20190412165416.7977-10-philmd@redhat.com Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- include/hw/devices.h | 3 --- include/hw/net/lan9118.h | 19 +++++++++++++++++++ hw/arm/kzm.c | 2 +- hw/arm/mps2.c | 2 +- hw/arm/realview.c | 1 + hw/arm/vexpress.c | 2 +- hw/net/lan9118.c | 2 +- 7 files changed, 24 insertions(+), 7 deletions(-) create mode 100644 include/hw/net/lan9118.h diff --git a/include/hw/devices.h b/include/hw/devices.h index XXXXXXX..XXXXXXX 100644 --- a/include/hw/devices.h +++ b/include/hw/devices.h @@ -XXX,XX +XXX,XX @@ /* smc91c111.c */ void smc91c111_init(NICInfo *, uint32_t, qemu_irq); -/* lan9118.c */ -void lan9118_init(NICInfo *, uint32_t, qemu_irq); - #endif diff --git a/include/hw/net/lan9118.h b/include/hw/net/lan9118.h new file mode 100644 index XXXXXXX..XXXXXXX --- /dev/null +++ b/include/hw/net/lan9118.h @@ -XXX,XX +XXX,XX @@ +/* + * SMSC LAN9118 Ethernet interface emulation + * + * Copyright (c) 2009 CodeSourcery, LLC. + * Written by Paul Brook + * + * This work is licensed under the terms of the GNU GPL, version 2 or later. + * See the COPYING file in the top-level directory. + */ + +#ifndef HW_NET_LAN9118_H +#define HW_NET_LAN9118_H + +#include "hw/irq.h" +#include "net/net.h" + +void lan9118_init(NICInfo *, uint32_t, qemu_irq); + +#endif diff --git a/hw/arm/kzm.c b/hw/arm/kzm.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/kzm.c +++ b/hw/arm/kzm.c @@ -XXX,XX +XXX,XX @@ #include "qemu/error-report.h" #include "exec/address-spaces.h" #include "net/net.h" -#include "hw/devices.h" +#include "hw/net/lan9118.h" #include "hw/char/serial.h" #include "sysemu/qtest.h" diff --git a/hw/arm/mps2.c b/hw/arm/mps2.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/mps2.c +++ b/hw/arm/mps2.c @@ -XXX,XX +XXX,XX @@ #include "hw/timer/cmsdk-apb-timer.h" #include "hw/timer/cmsdk-apb-dualtimer.h" #include "hw/misc/mps2-scc.h" -#include "hw/devices.h" +#include "hw/net/lan9118.h" #include "net/net.h" typedef enum MPS2FPGAType { diff --git a/hw/arm/realview.c b/hw/arm/realview.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/realview.c +++ b/hw/arm/realview.c @@ -XXX,XX +XXX,XX @@ #include "hw/arm/arm.h" #include "hw/arm/primecell.h" #include "hw/devices.h" +#include "hw/net/lan9118.h" #include "hw/pci/pci.h" #include "net/net.h" #include "sysemu/sysemu.h" diff --git a/hw/arm/vexpress.c b/hw/arm/vexpress.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/vexpress.c +++ b/hw/arm/vexpress.c @@ -XXX,XX +XXX,XX @@ #include "hw/sysbus.h" #include "hw/arm/arm.h" #include "hw/arm/primecell.h" -#include "hw/devices.h" +#include "hw/net/lan9118.h" #include "hw/i2c/i2c.h" #include "net/net.h" #include "sysemu/sysemu.h" diff --git a/hw/net/lan9118.c b/hw/net/lan9118.c index XXXXXXX..XXXXXXX 100644 --- a/hw/net/lan9118.c +++ b/hw/net/lan9118.c @@ -XXX,XX +XXX,XX @@ #include "hw/sysbus.h" #include "net/net.h" #include "net/eth.h" -#include "hw/devices.h" +#include "hw/net/lan9118.h" #include "sysemu/sysemu.h" #include "hw/ptimer.h" #include "qemu/log.h" -- 2.20.1 From: Philippe Mathieu-Daudé <philmd@redhat.com> Reviewed-by: Thomas Huth <thuth@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com> Message-id: 20190412165416.7977-11-philmd@redhat.com Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- include/hw/net/ne2000-isa.h | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/include/hw/net/ne2000-isa.h b/include/hw/net/ne2000-isa.h index XXXXXXX..XXXXXXX 100644 --- a/include/hw/net/ne2000-isa.h +++ b/include/hw/net/ne2000-isa.h @@ -XXX,XX +XXX,XX @@ * This work is licensed under the terms of the GNU GPL, version 2 or later. * See the COPYING file in the top-level directory. */ + +#ifndef HW_NET_NE2K_ISA_H +#define HW_NET_NE2K_ISA_H + #include "hw/hw.h" #include "hw/qdev.h" #include "hw/isa/isa.h" @@ -XXX,XX +XXX,XX @@ static inline ISADevice *isa_ne2000_init(ISABus *bus, int base, int irq, } return d; } + +#endif -- 2.20.1 From: Philippe Mathieu-Daudé <philmd@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com> Message-id: 20190412165416.7977-12-philmd@redhat.com Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- include/hw/net/lan9118.h | 2 ++ hw/arm/exynos4_boards.c | 3 ++- hw/arm/mps2-tz.c | 3 ++- hw/net/lan9118.c | 1 - 4 files changed, 6 insertions(+), 3 deletions(-) diff --git a/include/hw/net/lan9118.h b/include/hw/net/lan9118.h index XXXXXXX..XXXXXXX 100644 --- a/include/hw/net/lan9118.h +++ b/include/hw/net/lan9118.h @@ -XXX,XX +XXX,XX @@ #include "hw/irq.h" #include "net/net.h" +#define TYPE_LAN9118 "lan9118" + void lan9118_init(NICInfo *, uint32_t, qemu_irq); #endif diff --git a/hw/arm/exynos4_boards.c b/hw/arm/exynos4_boards.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/exynos4_boards.c +++ b/hw/arm/exynos4_boards.c @@ -XXX,XX +XXX,XX @@ #include "hw/arm/arm.h" #include "exec/address-spaces.h" #include "hw/arm/exynos4210.h" +#include "hw/net/lan9118.h" #include "hw/boards.h" #undef DEBUG @@ -XXX,XX +XXX,XX @@ static void lan9215_init(uint32_t base, qemu_irq irq) /* This should be a 9215 but the 9118 is close enough */ if (nd_table[0].used) { qemu_check_nic_model(&nd_table[0], "lan9118"); - dev = qdev_create(NULL, "lan9118"); + dev = qdev_create(NULL, TYPE_LAN9118); qdev_set_nic_properties(dev, &nd_table[0]); qdev_prop_set_uint32(dev, "mode_16bit", 1); qdev_init_nofail(dev); diff --git a/hw/arm/mps2-tz.c b/hw/arm/mps2-tz.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/mps2-tz.c +++ b/hw/arm/mps2-tz.c @@ -XXX,XX +XXX,XX @@ #include "hw/arm/armsse.h" #include "hw/dma/pl080.h" #include "hw/ssi/pl022.h" +#include "hw/net/lan9118.h" #include "net/net.h" #include "hw/core/split-irq.h" @@ -XXX,XX +XXX,XX @@ static MemoryRegion *make_eth_dev(MPS2TZMachineState *mms, void *opaque, * except that it doesn't support the checksum-offload feature. */ qemu_check_nic_model(nd, "lan9118"); - mms->lan9118 = qdev_create(NULL, "lan9118"); + mms->lan9118 = qdev_create(NULL, TYPE_LAN9118); qdev_set_nic_properties(mms->lan9118, nd); qdev_init_nofail(mms->lan9118); diff --git a/hw/net/lan9118.c b/hw/net/lan9118.c index XXXXXXX..XXXXXXX 100644 --- a/hw/net/lan9118.c +++ b/hw/net/lan9118.c @@ -XXX,XX +XXX,XX @@ static const VMStateDescription vmstate_lan9118_packet = { } }; -#define TYPE_LAN9118 "lan9118" #define LAN9118(obj) OBJECT_CHECK(lan9118_state, (obj), TYPE_LAN9118) typedef struct { -- 2.20.1 From: Philippe Mathieu-Daudé <philmd@redhat.com> This commit finally deletes "hw/devices.h". Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com> Message-id: 20190412165416.7977-13-philmd@redhat.com Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- include/hw/devices.h | 11 ----------- include/hw/net/smc91c111.h | 19 +++++++++++++++++++ hw/arm/gumstix.c | 2 +- hw/arm/integratorcp.c | 2 +- hw/arm/mainstone.c | 2 +- hw/arm/realview.c | 2 +- hw/arm/versatilepb.c | 2 +- hw/net/smc91c111.c | 2 +- 8 files changed, 25 insertions(+), 17 deletions(-) delete mode 100644 include/hw/devices.h create mode 100644 include/hw/net/smc91c111.h diff --git a/include/hw/devices.h b/include/hw/devices.h deleted file mode 100644 index XXXXXXX..XXXXXXX --- a/include/hw/devices.h +++ /dev/null @@ -XXX,XX +XXX,XX @@ -#ifndef QEMU_DEVICES_H -#define QEMU_DEVICES_H - -/* Devices that have nowhere better to go. */ - -#include "hw/hw.h" - -/* smc91c111.c */ -void smc91c111_init(NICInfo *, uint32_t, qemu_irq); - -#endif diff --git a/include/hw/net/smc91c111.h b/include/hw/net/smc91c111.h new file mode 100644 index XXXXXXX..XXXXXXX --- /dev/null +++ b/include/hw/net/smc91c111.h @@ -XXX,XX +XXX,XX @@ +/* + * SMSC 91C111 Ethernet interface emulation + * + * Copyright (c) 2005 CodeSourcery, LLC. + * Written by Paul Brook + * + * This work is licensed under the terms of the GNU GPL, version 2 or later. + * See the COPYING file in the top-level directory. + */ + +#ifndef HW_NET_SMC91C111_H +#define HW_NET_SMC91C111_H + +#include "hw/irq.h" +#include "net/net.h" + +void smc91c111_init(NICInfo *, uint32_t, qemu_irq); + +#endif diff --git a/hw/arm/gumstix.c b/hw/arm/gumstix.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/gumstix.c +++ b/hw/arm/gumstix.c @@ -XXX,XX +XXX,XX @@ #include "hw/arm/pxa.h" #include "net/net.h" #include "hw/block/flash.h" -#include "hw/devices.h" +#include "hw/net/smc91c111.h" #include "hw/boards.h" #include "exec/address-spaces.h" #include "sysemu/qtest.h" diff --git a/hw/arm/integratorcp.c b/hw/arm/integratorcp.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/integratorcp.c +++ b/hw/arm/integratorcp.c @@ -XXX,XX +XXX,XX @@ #include "qemu-common.h" #include "cpu.h" #include "hw/sysbus.h" -#include "hw/devices.h" #include "hw/boards.h" #include "hw/arm/arm.h" #include "hw/misc/arm_integrator_debug.h" +#include "hw/net/smc91c111.h" #include "net/net.h" #include "exec/address-spaces.h" #include "sysemu/sysemu.h" diff --git a/hw/arm/mainstone.c b/hw/arm/mainstone.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/mainstone.c +++ b/hw/arm/mainstone.c @@ -XXX,XX +XXX,XX @@ #include "hw/arm/pxa.h" #include "hw/arm/arm.h" #include "net/net.h" -#include "hw/devices.h" +#include "hw/net/smc91c111.h" #include "hw/boards.h" #include "hw/block/flash.h" #include "hw/sysbus.h" diff --git a/hw/arm/realview.c b/hw/arm/realview.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/realview.c +++ b/hw/arm/realview.c @@ -XXX,XX +XXX,XX @@ #include "hw/sysbus.h" #include "hw/arm/arm.h" #include "hw/arm/primecell.h" -#include "hw/devices.h" #include "hw/net/lan9118.h" +#include "hw/net/smc91c111.h" #include "hw/pci/pci.h" #include "net/net.h" #include "sysemu/sysemu.h" diff --git a/hw/arm/versatilepb.c b/hw/arm/versatilepb.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/versatilepb.c +++ b/hw/arm/versatilepb.c @@ -XXX,XX +XXX,XX @@ #include "cpu.h" #include "hw/sysbus.h" #include "hw/arm/arm.h" -#include "hw/devices.h" +#include "hw/net/smc91c111.h" #include "net/net.h" #include "sysemu/sysemu.h" #include "hw/pci/pci.h" diff --git a/hw/net/smc91c111.c b/hw/net/smc91c111.c index XXXXXXX..XXXXXXX 100644 --- a/hw/net/smc91c111.c +++ b/hw/net/smc91c111.c @@ -XXX,XX +XXX,XX @@ #include "qemu/osdep.h" #include "hw/sysbus.h" #include "net/net.h" -#include "hw/devices.h" +#include "hw/net/smc91c111.h" #include "qemu/log.h" /* For crc32 */ #include <zlib.h> -- 2.20.1 Just my fp16 work, plus some small stuff for the sbsa-ref board; but my rule of thumb is to send a pullreq once I get over about 30 patches... -- PMM The following changes since commit 2f4c51c0f384d7888a04b4815861e6d5fd244d75: Merge remote-tracking branch 'remotes/kraxel/tags/usb-20200831-pull-request' into staging (2020-08-31 19:39:13 +0100) are available in the Git repository at: https://git.linaro.org/people/pmaydell/qemu-arm.git tags/pull-target-arm-20200901 for you to fetch changes up to 3f462bf0f6ea6382dd1502d4eb1fcd33c8e774f5: hw/arm/sbsa-ref : Add embedded controller in secure memory (2020-09-01 14:01:34 +0100) ---------------------------------------------------------------- target-arm queue: * Implement fp16 support for AArch32 VFP and Neon * hw/arm/sbsa-ref: add "reg" property to DT cpu nodes * hw/arm/sbsa-ref : Add embedded controller in secure memory ---------------------------------------------------------------- Graeme Gregory (2): hw/misc/sbsa_ec : Add an embedded controller for sbsa-ref hw/arm/sbsa-ref : Add embedded controller in secure memory Leif Lindholm (1): hw/arm/sbsa-ref: add "reg" property to DT cpu nodes Peter Maydell (44): target/arm: Remove local definitions of float constants target/arm: Use correct ID register check for aa32_fp16_arith target/arm: Implement VFP fp16 for VFP_BINOP operations target/arm: Implement VFP fp16 VMLA, VMLS, VNMLS, VNMLA, VNMUL target/arm: Macroify trans functions for VFMA, VFMS, VFNMA, VFNMS target/arm: Implement VFP fp16 for fused-multiply-add target/arm: Macroify uses of do_vfp_2op_sp() and do_vfp_2op_dp() target/arm: Implement VFP fp16 for VABS, VNEG, VSQRT target/arm: Implement VFP fp16 for VMOV immediate target/arm: Implement VFP fp16 VCMP target/arm: Implement VFP fp16 VLDR and VSTR target/arm: Implement VFP fp16 VCVT between float and integer target/arm: Make VFP_CONV_FIX macros take separate float type and float size target/arm: Use macros instead of open-coding fp16 conversion helpers target/arm: Implement VFP fp16 VCVT between float and fixed-point target/arm: Implement VFP vp16 VCVT-with-specified-rounding-mode target/arm: Implement VFP fp16 VSEL target/arm: Implement VFP fp16 VRINT* target/arm: Implement new VFP fp16 insn VINS target/arm: Implement new VFP fp16 insn VMOVX target/arm: Implement VFP fp16 VMOV between gp and halfprec registers target/arm: Implement FP16 for Neon VADD, VSUB, VABD, VMUL target/arm: Implement fp16 for Neon VRECPE, VRSQRTE using gvec target/arm: Implement fp16 for Neon VABS, VNEG of floats target/arm: Implement fp16 for VCEQ, VCGE, VCGT comparisons target/arm: Implement fp16 for VACGE, VACGT target/arm: Implement fp16 for Neon VMAX, VMIN target/arm: Implement fp16 for Neon VMAXNM, VMINNM target/arm: Implement fp16 for Neon VMLA, VMLS operations target/arm: Implement fp16 for Neon VFMA, VMFS target/arm: Implement fp16 for Neon fp compare-vs-0 target/arm: Implement fp16 for Neon VRECPS target/arm: Implement fp16 for Neon VRSQRTS target/arm: Implement fp16 for Neon pairwise fp ops target/arm: Implement fp16 for Neon float-integer VCVT target/arm: Convert Neon VCVT fixed-point to gvec target/arm: Implement fp16 for Neon VCVT fixed-point target/arm: Implement fp16 for Neon VCVT with rounding modes target/arm: Implement fp16 for Neon VRINT-with-specified-rounding-mode target/arm: Implement fp16 for Neon VRINTX target/arm/vec_helper: Handle oprsz less than 16 bytes in indexed operations target/arm/vec_helper: Add gvec fp indexed multiply-and-add operations target/arm: Implement fp16 for Neon VMUL, VMLA, VMLS target/arm: Enable FP16 in '-cpu max' target/arm/cpu.h | 7 +- target/arm/helper.h | 133 ++++++- target/arm/neon-dp.decode | 8 +- target/arm/vfp-uncond.decode | 27 +- target/arm/vfp.decode | 34 +- hw/arm/sbsa-ref.c | 43 ++- hw/misc/sbsa_ec.c | 98 +++++ target/arm/cpu.c | 3 +- target/arm/cpu64.c | 10 +- target/arm/helper-a64.c | 11 - target/arm/translate-sve.c | 4 - target/arm/vec_helper.c | 431 ++++++++++++++++++++- target/arm/vfp_helper.c | 244 +++++------- hw/misc/meson.build | 2 + target/arm/translate-neon.c.inc | 755 +++++++++++++------------------------ target/arm/translate-vfp.c.inc | 810 ++++++++++++++++++++++++++++++++++++---- 16 files changed, 1819 insertions(+), 801 deletions(-) create mode 100644 hw/misc/sbsa_ec.c In several places the target/arm code defines local float constants for 2, 3 and 1.5, which are also provided by include/fpu/softfloat.h. Remove the unnecessary local duplicate versions. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-2-peter.maydell@linaro.org --- target/arm/helper-a64.c | 11 ----------- target/arm/translate-sve.c | 4 ---- target/arm/vfp_helper.c | 4 ---- 3 files changed, 19 deletions(-) diff --git a/target/arm/helper-a64.c b/target/arm/helper-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper-a64.c +++ b/target/arm/helper-a64.c @@ -XXX,XX +XXX,XX @@ uint64_t HELPER(neon_cgt_f64)(float64 a, float64 b, void *fpstp) * versions, these do a fully fused multiply-add or * multiply-add-and-halve. */ -#define float16_two make_float16(0x4000) -#define float16_three make_float16(0x4200) -#define float16_one_point_five make_float16(0x3e00) - -#define float32_two make_float32(0x40000000) -#define float32_three make_float32(0x40400000) -#define float32_one_point_five make_float32(0x3fc00000) - -#define float64_two make_float64(0x4000000000000000ULL) -#define float64_three make_float64(0x4008000000000000ULL) -#define float64_one_point_five make_float64(0x3FF8000000000000ULL) uint32_t HELPER(recpsf_f16)(uint32_t a, uint32_t b, void *fpstp) { diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -XXX,XX +XXX,XX @@ static bool trans_##NAME##_zpzi(DisasContext *s, arg_rpri_esz *a) \ return true; \ } -#define float16_two make_float16(0x4000) -#define float32_two make_float32(0x40000000) -#define float64_two make_float64(0x4000000000000000ULL) - DO_FP_IMM(FADD, fadds, half, one) DO_FP_IMM(FSUB, fsubs, half, one) DO_FP_IMM(FMUL, fmuls, half, two) diff --git a/target/arm/vfp_helper.c b/target/arm/vfp_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vfp_helper.c +++ b/target/arm/vfp_helper.c @@ -XXX,XX +XXX,XX @@ uint32_t HELPER(vfp_fcvt_f64_to_f16)(float64 a, void *fpstp, uint32_t ahp_mode) return r; } -#define float32_two make_float32(0x40000000) -#define float32_three make_float32(0x40400000) -#define float32_one_point_five make_float32(0x3fc00000) - float32 HELPER(recps_f32)(CPUARMState *env, float32 a, float32 b) { float_status *s = &env->vfp.standard_fp_status; -- 2.20.1 The aa32_fp16_arith feature check function currently looks at the AArch64 ID_AA64PFR0 register. This is (as the comment notes) not correct. The bogus check was put in mostly to allow testing of the fp16 variants of the VCMLA instructions and it was something of a mistake that we allowed them to exist in master. Switch the feature check function to testing VMFR1.FPHP, which is what it ought to be. This will remove emulation of the VCMLA and VCADD insns from AArch32 code running on an AArch64 '-cpu max' using system emulation. (They were never enabled for aarch32 linux-user and system-emulation.) Since we weren't advertising their existence via the AArch32 ID register, well-behaved guests wouldn't have been using them anyway. Once we have implemented all the AArch32 support for the FP16 extension we will advertise it in the MVFR1 ID register field, which will reenable these insns along with all the others. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-3-peter.maydell@linaro.org --- target/arm/cpu.h | 7 +------ 1 file changed, 1 insertion(+), 6 deletions(-) diff --git a/target/arm/cpu.h b/target/arm/cpu.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -XXX,XX +XXX,XX @@ static inline bool isar_feature_aa32_predinv(const ARMISARegisters *id) static inline bool isar_feature_aa32_fp16_arith(const ARMISARegisters *id) { - /* - * This is a placeholder for use by VCMA until the rest of - * the ARMv8.2-FP16 extension is implemented for aa32 mode. - * At which point we can properly set and check MVFR1.FPHP. - */ - return FIELD_EX64(id->id_aa64pfr0, ID_AA64PFR0, FP) == 1; + return FIELD_EX32(id->mvfr1, MVFR1, FPHP) >= 3; } static inline bool isar_feature_aa32_vfp_simd(const ARMISARegisters *id) -- 2.20.1 Implmeent VFP fp16 support for simple binary-operator VFP insns VADD, VSUB, VMUL, VDIV, VMINNM and VMAXNM: * make the VFP_BINOP() macro generate float16 helpers as well as float32 and float64 * implement a do_vfp_3op_hp() function similar to the existing do_vfp_3op_sp() * add decode for the half-precision insn patterns Note that the VFP_BINOP macro use creates a couple of unused helper functions vfp_maxh and vfp_minh, but they're small so it's not worth splitting the BINOP operations into "needs halfprec" and "no halfprec" groups. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-4-peter.maydell@linaro.org --- target/arm/helper.h | 8 ++++ target/arm/vfp-uncond.decode | 3 ++ target/arm/vfp.decode | 4 ++ target/arm/vfp_helper.c | 5 ++ target/arm/translate-vfp.c.inc | 86 ++++++++++++++++++++++++++++++++++ 5 files changed, 106 insertions(+) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_5(probe_access, TCG_CALL_NO_WG, void, env, tl, i32, i32, i32) DEF_HELPER_1(vfp_get_fpscr, i32, env) DEF_HELPER_2(vfp_set_fpscr, void, env, i32) +DEF_HELPER_3(vfp_addh, f16, f16, f16, ptr) DEF_HELPER_3(vfp_adds, f32, f32, f32, ptr) DEF_HELPER_3(vfp_addd, f64, f64, f64, ptr) +DEF_HELPER_3(vfp_subh, f16, f16, f16, ptr) DEF_HELPER_3(vfp_subs, f32, f32, f32, ptr) DEF_HELPER_3(vfp_subd, f64, f64, f64, ptr) +DEF_HELPER_3(vfp_mulh, f16, f16, f16, ptr) DEF_HELPER_3(vfp_muls, f32, f32, f32, ptr) DEF_HELPER_3(vfp_muld, f64, f64, f64, ptr) +DEF_HELPER_3(vfp_divh, f16, f16, f16, ptr) DEF_HELPER_3(vfp_divs, f32, f32, f32, ptr) DEF_HELPER_3(vfp_divd, f64, f64, f64, ptr) +DEF_HELPER_3(vfp_maxh, f16, f16, f16, ptr) DEF_HELPER_3(vfp_maxs, f32, f32, f32, ptr) DEF_HELPER_3(vfp_maxd, f64, f64, f64, ptr) +DEF_HELPER_3(vfp_minh, f16, f16, f16, ptr) DEF_HELPER_3(vfp_mins, f32, f32, f32, ptr) DEF_HELPER_3(vfp_mind, f64, f64, f64, ptr) +DEF_HELPER_3(vfp_maxnumh, f16, f16, f16, ptr) DEF_HELPER_3(vfp_maxnums, f32, f32, f32, ptr) DEF_HELPER_3(vfp_maxnumd, f64, f64, f64, ptr) +DEF_HELPER_3(vfp_minnumh, f16, f16, f16, ptr) DEF_HELPER_3(vfp_minnums, f32, f32, f32, ptr) DEF_HELPER_3(vfp_minnumd, f64, f64, f64, ptr) DEF_HELPER_1(vfp_negs, f32, f32) diff --git a/target/arm/vfp-uncond.decode b/target/arm/vfp-uncond.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vfp-uncond.decode +++ b/target/arm/vfp-uncond.decode @@ -XXX,XX +XXX,XX @@ VSEL 1111 1110 0. cc:2 .... .... 1010 .0.0 .... \ VSEL 1111 1110 0. cc:2 .... .... 1011 .0.0 .... \ vm=%vm_dp vn=%vn_dp vd=%vd_dp dp=1 +VMAXNM_hp 1111 1110 1.00 .... .... 1001 .0.0 .... @vfp_dnm_s +VMINNM_hp 1111 1110 1.00 .... .... 1001 .1.0 .... @vfp_dnm_s + VMAXNM_sp 1111 1110 1.00 .... .... 1010 .0.0 .... @vfp_dnm_s VMINNM_sp 1111 1110 1.00 .... .... 1010 .1.0 .... @vfp_dnm_s diff --git a/target/arm/vfp.decode b/target/arm/vfp.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vfp.decode +++ b/target/arm/vfp.decode @@ -XXX,XX +XXX,XX @@ VNMLS_dp ---- 1110 0.01 .... .... 1011 .0.0 .... @vfp_dnm_d VNMLA_sp ---- 1110 0.01 .... .... 1010 .1.0 .... @vfp_dnm_s VNMLA_dp ---- 1110 0.01 .... .... 1011 .1.0 .... @vfp_dnm_d +VMUL_hp ---- 1110 0.10 .... .... 1001 .0.0 .... @vfp_dnm_s VMUL_sp ---- 1110 0.10 .... .... 1010 .0.0 .... @vfp_dnm_s VMUL_dp ---- 1110 0.10 .... .... 1011 .0.0 .... @vfp_dnm_d VNMUL_sp ---- 1110 0.10 .... .... 1010 .1.0 .... @vfp_dnm_s VNMUL_dp ---- 1110 0.10 .... .... 1011 .1.0 .... @vfp_dnm_d +VADD_hp ---- 1110 0.11 .... .... 1001 .0.0 .... @vfp_dnm_s VADD_sp ---- 1110 0.11 .... .... 1010 .0.0 .... @vfp_dnm_s VADD_dp ---- 1110 0.11 .... .... 1011 .0.0 .... @vfp_dnm_d +VSUB_hp ---- 1110 0.11 .... .... 1001 .1.0 .... @vfp_dnm_s VSUB_sp ---- 1110 0.11 .... .... 1010 .1.0 .... @vfp_dnm_s VSUB_dp ---- 1110 0.11 .... .... 1011 .1.0 .... @vfp_dnm_d +VDIV_hp ---- 1110 1.00 .... .... 1001 .0.0 .... @vfp_dnm_s VDIV_sp ---- 1110 1.00 .... .... 1010 .0.0 .... @vfp_dnm_s VDIV_dp ---- 1110 1.00 .... .... 1011 .0.0 .... @vfp_dnm_d diff --git a/target/arm/vfp_helper.c b/target/arm/vfp_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vfp_helper.c +++ b/target/arm/vfp_helper.c @@ -XXX,XX +XXX,XX @@ void vfp_set_fpscr(CPUARMState *env, uint32_t val) #define VFP_HELPER(name, p) HELPER(glue(glue(vfp_,name),p)) #define VFP_BINOP(name) \ +dh_ctype_f16 VFP_HELPER(name, h)(dh_ctype_f16 a, dh_ctype_f16 b, void *fpstp) \ +{ \ + float_status *fpst = fpstp; \ + return float16_ ## name(a, b, fpst); \ +} \ float32 VFP_HELPER(name, s)(float32 a, float32 b, void *fpstp) \ { \ float_status *fpst = fpstp; \ diff --git a/target/arm/translate-vfp.c.inc b/target/arm/translate-vfp.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-vfp.c.inc +++ b/target/arm/translate-vfp.c.inc @@ -XXX,XX +XXX,XX @@ static bool do_vfp_3op_sp(DisasContext *s, VFPGen3OpSPFn *fn, return true; } +static bool do_vfp_3op_hp(DisasContext *s, VFPGen3OpSPFn *fn, + int vd, int vn, int vm, bool reads_vd) +{ + /* + * Do a half-precision operation. Functionally this is + * the same as do_vfp_3op_sp(), except: + * - it uses the FPST_FPCR_F16 + * - it doesn't need the VFP vector handling (fp16 is a + * v8 feature, and in v8 VFP vectors don't exist) + * - it does the aa32_fp16_arith feature test + */ + TCGv_i32 f0, f1, fd; + TCGv_ptr fpst; + + if (!dc_isar_feature(aa32_fp16_arith, s)) { + return false; + } + + if (s->vec_len != 0 || s->vec_stride != 0) { + return false; + } + + if (!vfp_access_check(s)) { + return true; + } + + f0 = tcg_temp_new_i32(); + f1 = tcg_temp_new_i32(); + fd = tcg_temp_new_i32(); + fpst = fpstatus_ptr(FPST_FPCR_F16); + + neon_load_reg32(f0, vn); + neon_load_reg32(f1, vm); + + if (reads_vd) { + neon_load_reg32(fd, vd); + } + fn(fd, f0, f1, fpst); + neon_store_reg32(fd, vd); + + tcg_temp_free_i32(f0); + tcg_temp_free_i32(f1); + tcg_temp_free_i32(fd); + tcg_temp_free_ptr(fpst); + + return true; +} + static bool do_vfp_3op_dp(DisasContext *s, VFPGen3OpDPFn *fn, int vd, int vn, int vm, bool reads_vd) { @@ -XXX,XX +XXX,XX @@ static bool trans_VNMLA_dp(DisasContext *s, arg_VNMLA_dp *a) return do_vfp_3op_dp(s, gen_VNMLA_dp, a->vd, a->vn, a->vm, true); } +static bool trans_VMUL_hp(DisasContext *s, arg_VMUL_sp *a) +{ + return do_vfp_3op_hp(s, gen_helper_vfp_mulh, a->vd, a->vn, a->vm, false); +} + static bool trans_VMUL_sp(DisasContext *s, arg_VMUL_sp *a) { return do_vfp_3op_sp(s, gen_helper_vfp_muls, a->vd, a->vn, a->vm, false); @@ -XXX,XX +XXX,XX @@ static bool trans_VNMUL_dp(DisasContext *s, arg_VNMUL_dp *a) return do_vfp_3op_dp(s, gen_VNMUL_dp, a->vd, a->vn, a->vm, false); } +static bool trans_VADD_hp(DisasContext *s, arg_VADD_sp *a) +{ + return do_vfp_3op_hp(s, gen_helper_vfp_addh, a->vd, a->vn, a->vm, false); +} + static bool trans_VADD_sp(DisasContext *s, arg_VADD_sp *a) { return do_vfp_3op_sp(s, gen_helper_vfp_adds, a->vd, a->vn, a->vm, false); @@ -XXX,XX +XXX,XX @@ static bool trans_VADD_dp(DisasContext *s, arg_VADD_dp *a) return do_vfp_3op_dp(s, gen_helper_vfp_addd, a->vd, a->vn, a->vm, false); } +static bool trans_VSUB_hp(DisasContext *s, arg_VSUB_sp *a) +{ + return do_vfp_3op_hp(s, gen_helper_vfp_subh, a->vd, a->vn, a->vm, false); +} + static bool trans_VSUB_sp(DisasContext *s, arg_VSUB_sp *a) { return do_vfp_3op_sp(s, gen_helper_vfp_subs, a->vd, a->vn, a->vm, false); @@ -XXX,XX +XXX,XX @@ static bool trans_VSUB_dp(DisasContext *s, arg_VSUB_dp *a) return do_vfp_3op_dp(s, gen_helper_vfp_subd, a->vd, a->vn, a->vm, false); } +static bool trans_VDIV_hp(DisasContext *s, arg_VDIV_sp *a) +{ + return do_vfp_3op_hp(s, gen_helper_vfp_divh, a->vd, a->vn, a->vm, false); +} + static bool trans_VDIV_sp(DisasContext *s, arg_VDIV_sp *a) { return do_vfp_3op_sp(s, gen_helper_vfp_divs, a->vd, a->vn, a->vm, false); @@ -XXX,XX +XXX,XX @@ static bool trans_VDIV_dp(DisasContext *s, arg_VDIV_dp *a) return do_vfp_3op_dp(s, gen_helper_vfp_divd, a->vd, a->vn, a->vm, false); } +static bool trans_VMINNM_hp(DisasContext *s, arg_VMINNM_sp *a) +{ + if (!dc_isar_feature(aa32_vminmaxnm, s)) { + return false; + } + return do_vfp_3op_hp(s, gen_helper_vfp_minnumh, + a->vd, a->vn, a->vm, false); +} + +static bool trans_VMAXNM_hp(DisasContext *s, arg_VMAXNM_sp *a) +{ + if (!dc_isar_feature(aa32_vminmaxnm, s)) { + return false; + } + return do_vfp_3op_hp(s, gen_helper_vfp_maxnumh, + a->vd, a->vn, a->vm, false); +} + static bool trans_VMINNM_sp(DisasContext *s, arg_VMINNM_sp *a) { if (!dc_isar_feature(aa32_vminmaxnm, s)) { -- 2.20.1 Implement fp16 versions of the VFP VMLA, VMLS, VNMLS, VNMLA, VNMUL instructions. (These are all the remaining ones which we implement via do_vfp_3op_[hsd]p().) Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-5-peter.maydell@linaro.org --- target/arm/helper.h | 1 + target/arm/vfp.decode | 5 ++ target/arm/vfp_helper.c | 5 ++ target/arm/translate-vfp.c.inc | 84 ++++++++++++++++++++++++++++++++++ 4 files changed, 95 insertions(+) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_3(vfp_maxnumd, f64, f64, f64, ptr) DEF_HELPER_3(vfp_minnumh, f16, f16, f16, ptr) DEF_HELPER_3(vfp_minnums, f32, f32, f32, ptr) DEF_HELPER_3(vfp_minnumd, f64, f64, f64, ptr) +DEF_HELPER_1(vfp_negh, f16, f16) DEF_HELPER_1(vfp_negs, f32, f32) DEF_HELPER_1(vfp_negd, f64, f64) DEF_HELPER_1(vfp_abss, f32, f32) diff --git a/target/arm/vfp.decode b/target/arm/vfp.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vfp.decode +++ b/target/arm/vfp.decode @@ -XXX,XX +XXX,XX @@ VLDM_VSTM_dp ---- 1101 0.1 l:1 rn:4 .... 1011 imm:8 \ vd=%vd_dp p=1 u=0 w=1 # 3-register VFP data-processing; bits [23,21:20,6] identify the operation. +VMLA_hp ---- 1110 0.00 .... .... 1001 .0.0 .... @vfp_dnm_s VMLA_sp ---- 1110 0.00 .... .... 1010 .0.0 .... @vfp_dnm_s VMLA_dp ---- 1110 0.00 .... .... 1011 .0.0 .... @vfp_dnm_d +VMLS_hp ---- 1110 0.00 .... .... 1001 .1.0 .... @vfp_dnm_s VMLS_sp ---- 1110 0.00 .... .... 1010 .1.0 .... @vfp_dnm_s VMLS_dp ---- 1110 0.00 .... .... 1011 .1.0 .... @vfp_dnm_d +VNMLS_hp ---- 1110 0.01 .... .... 1001 .0.0 .... @vfp_dnm_s VNMLS_sp ---- 1110 0.01 .... .... 1010 .0.0 .... @vfp_dnm_s VNMLS_dp ---- 1110 0.01 .... .... 1011 .0.0 .... @vfp_dnm_d +VNMLA_hp ---- 1110 0.01 .... .... 1001 .1.0 .... @vfp_dnm_s VNMLA_sp ---- 1110 0.01 .... .... 1010 .1.0 .... @vfp_dnm_s VNMLA_dp ---- 1110 0.01 .... .... 1011 .1.0 .... @vfp_dnm_d @@ -XXX,XX +XXX,XX @@ VMUL_hp ---- 1110 0.10 .... .... 1001 .0.0 .... @vfp_dnm_s VMUL_sp ---- 1110 0.10 .... .... 1010 .0.0 .... @vfp_dnm_s VMUL_dp ---- 1110 0.10 .... .... 1011 .0.0 .... @vfp_dnm_d +VNMUL_hp ---- 1110 0.10 .... .... 1001 .1.0 .... @vfp_dnm_s VNMUL_sp ---- 1110 0.10 .... .... 1010 .1.0 .... @vfp_dnm_s VNMUL_dp ---- 1110 0.10 .... .... 1011 .1.0 .... @vfp_dnm_d diff --git a/target/arm/vfp_helper.c b/target/arm/vfp_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vfp_helper.c +++ b/target/arm/vfp_helper.c @@ -XXX,XX +XXX,XX @@ VFP_BINOP(minnum) VFP_BINOP(maxnum) #undef VFP_BINOP +dh_ctype_f16 VFP_HELPER(neg, h)(dh_ctype_f16 a) +{ + return float16_chs(a); +} + float32 VFP_HELPER(neg, s)(float32 a) { return float32_chs(a); diff --git a/target/arm/translate-vfp.c.inc b/target/arm/translate-vfp.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-vfp.c.inc +++ b/target/arm/translate-vfp.c.inc @@ -XXX,XX +XXX,XX @@ static bool do_vfp_2op_dp(DisasContext *s, VFPGen2OpDPFn *fn, int vd, int vm) return true; } +static void gen_VMLA_hp(TCGv_i32 vd, TCGv_i32 vn, TCGv_i32 vm, TCGv_ptr fpst) +{ + /* Note that order of inputs to the add matters for NaNs */ + TCGv_i32 tmp = tcg_temp_new_i32(); + + gen_helper_vfp_mulh(tmp, vn, vm, fpst); + gen_helper_vfp_addh(vd, vd, tmp, fpst); + tcg_temp_free_i32(tmp); +} + +static bool trans_VMLA_hp(DisasContext *s, arg_VMLA_sp *a) +{ + return do_vfp_3op_hp(s, gen_VMLA_hp, a->vd, a->vn, a->vm, true); +} + static void gen_VMLA_sp(TCGv_i32 vd, TCGv_i32 vn, TCGv_i32 vm, TCGv_ptr fpst) { /* Note that order of inputs to the add matters for NaNs */ @@ -XXX,XX +XXX,XX @@ static bool trans_VMLA_dp(DisasContext *s, arg_VMLA_dp *a) return do_vfp_3op_dp(s, gen_VMLA_dp, a->vd, a->vn, a->vm, true); } +static void gen_VMLS_hp(TCGv_i32 vd, TCGv_i32 vn, TCGv_i32 vm, TCGv_ptr fpst) +{ + /* + * VMLS: vd = vd + -(vn * vm) + * Note that order of inputs to the add matters for NaNs. + */ + TCGv_i32 tmp = tcg_temp_new_i32(); + + gen_helper_vfp_mulh(tmp, vn, vm, fpst); + gen_helper_vfp_negh(tmp, tmp); + gen_helper_vfp_addh(vd, vd, tmp, fpst); + tcg_temp_free_i32(tmp); +} + +static bool trans_VMLS_hp(DisasContext *s, arg_VMLS_sp *a) +{ + return do_vfp_3op_hp(s, gen_VMLS_hp, a->vd, a->vn, a->vm, true); +} + static void gen_VMLS_sp(TCGv_i32 vd, TCGv_i32 vn, TCGv_i32 vm, TCGv_ptr fpst) { /* @@ -XXX,XX +XXX,XX @@ static bool trans_VMLS_dp(DisasContext *s, arg_VMLS_dp *a) return do_vfp_3op_dp(s, gen_VMLS_dp, a->vd, a->vn, a->vm, true); } +static void gen_VNMLS_hp(TCGv_i32 vd, TCGv_i32 vn, TCGv_i32 vm, TCGv_ptr fpst) +{ + /* + * VNMLS: -fd + (fn * fm) + * Note that it isn't valid to replace (-A + B) with (B - A) or similar + * plausible looking simplifications because this will give wrong results + * for NaNs. + */ + TCGv_i32 tmp = tcg_temp_new_i32(); + + gen_helper_vfp_mulh(tmp, vn, vm, fpst); + gen_helper_vfp_negh(vd, vd); + gen_helper_vfp_addh(vd, vd, tmp, fpst); + tcg_temp_free_i32(tmp); +} + +static bool trans_VNMLS_hp(DisasContext *s, arg_VNMLS_sp *a) +{ + return do_vfp_3op_hp(s, gen_VNMLS_hp, a->vd, a->vn, a->vm, true); +} + static void gen_VNMLS_sp(TCGv_i32 vd, TCGv_i32 vn, TCGv_i32 vm, TCGv_ptr fpst) { /* @@ -XXX,XX +XXX,XX @@ static bool trans_VNMLS_dp(DisasContext *s, arg_VNMLS_dp *a) return do_vfp_3op_dp(s, gen_VNMLS_dp, a->vd, a->vn, a->vm, true); } +static void gen_VNMLA_hp(TCGv_i32 vd, TCGv_i32 vn, TCGv_i32 vm, TCGv_ptr fpst) +{ + /* VNMLA: -fd + -(fn * fm) */ + TCGv_i32 tmp = tcg_temp_new_i32(); + + gen_helper_vfp_mulh(tmp, vn, vm, fpst); + gen_helper_vfp_negh(tmp, tmp); + gen_helper_vfp_negh(vd, vd); + gen_helper_vfp_addh(vd, vd, tmp, fpst); + tcg_temp_free_i32(tmp); +} + +static bool trans_VNMLA_hp(DisasContext *s, arg_VNMLA_sp *a) +{ + return do_vfp_3op_hp(s, gen_VNMLA_hp, a->vd, a->vn, a->vm, true); +} + static void gen_VNMLA_sp(TCGv_i32 vd, TCGv_i32 vn, TCGv_i32 vm, TCGv_ptr fpst) { /* VNMLA: -fd + -(fn * fm) */ @@ -XXX,XX +XXX,XX @@ static bool trans_VMUL_dp(DisasContext *s, arg_VMUL_dp *a) return do_vfp_3op_dp(s, gen_helper_vfp_muld, a->vd, a->vn, a->vm, false); } +static void gen_VNMUL_hp(TCGv_i32 vd, TCGv_i32 vn, TCGv_i32 vm, TCGv_ptr fpst) +{ + /* VNMUL: -(fn * fm) */ + gen_helper_vfp_mulh(vd, vn, vm, fpst); + gen_helper_vfp_negh(vd, vd); +} + +static bool trans_VNMUL_hp(DisasContext *s, arg_VNMUL_sp *a) +{ + return do_vfp_3op_hp(s, gen_VNMUL_hp, a->vd, a->vn, a->vm, false); +} + static void gen_VNMUL_sp(TCGv_i32 vd, TCGv_i32 vn, TCGv_i32 vm, TCGv_ptr fpst) { /* VNMUL: -(fn * fm) */ -- 2.20.1 Macroify creation of the trans functions for single and double precision VFMA, VFMS, VFNMA, VFNMS. The repetition was OK for two sizes, but we're about to add halfprec and it will get a bit more than seems reasonable. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-6-peter.maydell@linaro.org --- target/arm/translate-vfp.c.inc | 50 +++++++++------------------------- 1 file changed, 13 insertions(+), 37 deletions(-) diff --git a/target/arm/translate-vfp.c.inc b/target/arm/translate-vfp.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-vfp.c.inc +++ b/target/arm/translate-vfp.c.inc @@ -XXX,XX +XXX,XX @@ static bool do_vfm_sp(DisasContext *s, arg_VFMA_sp *a, bool neg_n, bool neg_d) return true; } -static bool trans_VFMA_sp(DisasContext *s, arg_VFMA_sp *a) -{ - return do_vfm_sp(s, a, false, false); -} - -static bool trans_VFMS_sp(DisasContext *s, arg_VFMS_sp *a) -{ - return do_vfm_sp(s, a, true, false); -} - -static bool trans_VFNMA_sp(DisasContext *s, arg_VFNMA_sp *a) -{ - return do_vfm_sp(s, a, false, true); -} - -static bool trans_VFNMS_sp(DisasContext *s, arg_VFNMS_sp *a) -{ - return do_vfm_sp(s, a, true, true); -} - static bool do_vfm_dp(DisasContext *s, arg_VFMA_dp *a, bool neg_n, bool neg_d) { /* @@ -XXX,XX +XXX,XX @@ static bool do_vfm_dp(DisasContext *s, arg_VFMA_dp *a, bool neg_n, bool neg_d) return true; } -static bool trans_VFMA_dp(DisasContext *s, arg_VFMA_dp *a) -{ - return do_vfm_dp(s, a, false, false); -} +#define MAKE_ONE_VFM_TRANS_FN(INSN, PREC, NEGN, NEGD) \ + static bool trans_##INSN##_##PREC(DisasContext *s, \ + arg_##INSN##_##PREC *a) \ + { \ + return do_vfm_##PREC(s, a, NEGN, NEGD); \ + } -static bool trans_VFMS_dp(DisasContext *s, arg_VFMS_dp *a) -{ - return do_vfm_dp(s, a, true, false); -} +#define MAKE_VFM_TRANS_FNS(PREC) \ + MAKE_ONE_VFM_TRANS_FN(VFMA, PREC, false, false) \ + MAKE_ONE_VFM_TRANS_FN(VFMS, PREC, true, false) \ + MAKE_ONE_VFM_TRANS_FN(VFNMA, PREC, false, true) \ + MAKE_ONE_VFM_TRANS_FN(VFNMS, PREC, true, true) -static bool trans_VFNMA_dp(DisasContext *s, arg_VFNMA_dp *a) -{ - return do_vfm_dp(s, a, false, true); -} - -static bool trans_VFNMS_dp(DisasContext *s, arg_VFNMS_dp *a) -{ - return do_vfm_dp(s, a, true, true); -} +MAKE_VFM_TRANS_FNS(sp) +MAKE_VFM_TRANS_FNS(dp) static bool trans_VMOV_imm_sp(DisasContext *s, arg_VMOV_imm_sp *a) { -- 2.20.1 Implement VFP fp16 support for fused multiply-add insns VFNMA, VFNMS, VFMA, VFMS. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-7-peter.maydell@linaro.org --- target/arm/helper.h | 1 + target/arm/vfp.decode | 5 +++ target/arm/vfp_helper.c | 7 ++++ target/arm/translate-vfp.c.inc | 64 ++++++++++++++++++++++++++++++++++ 4 files changed, 77 insertions(+) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_3(vfp_fcvt_f64_to_f16, TCG_CALL_NO_RWG, f16, f64, ptr, i32) DEF_HELPER_4(vfp_muladdd, f64, f64, f64, f64, ptr) DEF_HELPER_4(vfp_muladds, f32, f32, f32, f32, ptr) +DEF_HELPER_4(vfp_muladdh, f16, f16, f16, f16, ptr) DEF_HELPER_3(recps_f32, f32, env, f32, f32) DEF_HELPER_3(rsqrts_f32, f32, env, f32, f32) diff --git a/target/arm/vfp.decode b/target/arm/vfp.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vfp.decode +++ b/target/arm/vfp.decode @@ -XXX,XX +XXX,XX @@ VDIV_hp ---- 1110 1.00 .... .... 1001 .0.0 .... @vfp_dnm_s VDIV_sp ---- 1110 1.00 .... .... 1010 .0.0 .... @vfp_dnm_s VDIV_dp ---- 1110 1.00 .... .... 1011 .0.0 .... @vfp_dnm_d +VFMA_hp ---- 1110 1.10 .... .... 1001 .0. 0 .... @vfp_dnm_s +VFMS_hp ---- 1110 1.10 .... .... 1001 .1. 0 .... @vfp_dnm_s +VFNMA_hp ---- 1110 1.01 .... .... 1001 .0. 0 .... @vfp_dnm_s +VFNMS_hp ---- 1110 1.01 .... .... 1001 .1. 0 .... @vfp_dnm_s + VFMA_sp ---- 1110 1.10 .... .... 1010 .0. 0 .... @vfp_dnm_s VFMS_sp ---- 1110 1.10 .... .... 1010 .1. 0 .... @vfp_dnm_s VFNMA_sp ---- 1110 1.01 .... .... 1010 .0. 0 .... @vfp_dnm_s diff --git a/target/arm/vfp_helper.c b/target/arm/vfp_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vfp_helper.c +++ b/target/arm/vfp_helper.c @@ -XXX,XX +XXX,XX @@ uint32_t HELPER(rsqrte_u32)(uint32_t a) } /* VFPv4 fused multiply-accumulate */ +dh_ctype_f16 VFP_HELPER(muladd, h)(dh_ctype_f16 a, dh_ctype_f16 b, + dh_ctype_f16 c, void *fpstp) +{ + float_status *fpst = fpstp; + return float16_muladd(a, b, c, 0, fpst); +} + float32 VFP_HELPER(muladd, s)(float32 a, float32 b, float32 c, void *fpstp) { float_status *fpst = fpstp; diff --git a/target/arm/translate-vfp.c.inc b/target/arm/translate-vfp.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-vfp.c.inc +++ b/target/arm/translate-vfp.c.inc @@ -XXX,XX +XXX,XX @@ static bool trans_VMAXNM_dp(DisasContext *s, arg_VMAXNM_dp *a) a->vd, a->vn, a->vm, false); } +static bool do_vfm_hp(DisasContext *s, arg_VFMA_sp *a, bool neg_n, bool neg_d) +{ + /* + * VFNMA : fd = muladd(-fd, fn, fm) + * VFNMS : fd = muladd(-fd, -fn, fm) + * VFMA : fd = muladd( fd, fn, fm) + * VFMS : fd = muladd( fd, -fn, fm) + * + * These are fused multiply-add, and must be done as one floating + * point operation with no rounding between the multiplication and + * addition steps. NB that doing the negations here as separate + * steps is correct : an input NaN should come out with its sign + * bit flipped if it is a negated-input. + */ + TCGv_ptr fpst; + TCGv_i32 vn, vm, vd; + + /* + * Present in VFPv4 only, and only with the FP16 extension. + * Note that we can't rely on the SIMDFMAC check alone, because + * in a Neon-no-VFP core that ID register field will be non-zero. + */ + if (!dc_isar_feature(aa32_fp16_arith, s) || + !dc_isar_feature(aa32_simdfmac, s) || + !dc_isar_feature(aa32_fpsp_v2, s)) { + return false; + } + + if (s->vec_len != 0 || s->vec_stride != 0) { + return false; + } + + if (!vfp_access_check(s)) { + return true; + } + + vn = tcg_temp_new_i32(); + vm = tcg_temp_new_i32(); + vd = tcg_temp_new_i32(); + + neon_load_reg32(vn, a->vn); + neon_load_reg32(vm, a->vm); + if (neg_n) { + /* VFNMS, VFMS */ + gen_helper_vfp_negh(vn, vn); + } + neon_load_reg32(vd, a->vd); + if (neg_d) { + /* VFNMA, VFNMS */ + gen_helper_vfp_negh(vd, vd); + } + fpst = fpstatus_ptr(FPST_FPCR_F16); + gen_helper_vfp_muladdh(vd, vn, vm, vd, fpst); + neon_store_reg32(vd, a->vd); + + tcg_temp_free_ptr(fpst); + tcg_temp_free_i32(vn); + tcg_temp_free_i32(vm); + tcg_temp_free_i32(vd); + + return true; +} + static bool do_vfm_sp(DisasContext *s, arg_VFMA_sp *a, bool neg_n, bool neg_d) { /* @@ -XXX,XX +XXX,XX @@ static bool do_vfm_dp(DisasContext *s, arg_VFMA_dp *a, bool neg_n, bool neg_d) MAKE_ONE_VFM_TRANS_FN(VFNMA, PREC, false, true) \ MAKE_ONE_VFM_TRANS_FN(VFNMS, PREC, true, true) +MAKE_VFM_TRANS_FNS(hp) MAKE_VFM_TRANS_FNS(sp) MAKE_VFM_TRANS_FNS(dp) -- 2.20.1 Macroify the uses of do_vfp_2op_sp() and do_vfp_2op_dp(); this will make it easier to add the halfprec support. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-8-peter.maydell@linaro.org --- target/arm/translate-vfp.c.inc | 49 ++++++++++------------------------ 1 file changed, 14 insertions(+), 35 deletions(-) diff --git a/target/arm/translate-vfp.c.inc b/target/arm/translate-vfp.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-vfp.c.inc +++ b/target/arm/translate-vfp.c.inc @@ -XXX,XX +XXX,XX @@ static bool trans_VMOV_imm_dp(DisasContext *s, arg_VMOV_imm_dp *a) return true; } -static bool trans_VMOV_reg_sp(DisasContext *s, arg_VMOV_reg_sp *a) -{ - return do_vfp_2op_sp(s, tcg_gen_mov_i32, a->vd, a->vm); -} +#define DO_VFP_2OP(INSN, PREC, FN) \ + static bool trans_##INSN##_##PREC(DisasContext *s, \ + arg_##INSN##_##PREC *a) \ + { \ + return do_vfp_2op_##PREC(s, FN, a->vd, a->vm); \ + } -static bool trans_VMOV_reg_dp(DisasContext *s, arg_VMOV_reg_dp *a) -{ - return do_vfp_2op_dp(s, tcg_gen_mov_i64, a->vd, a->vm); -} +DO_VFP_2OP(VMOV_reg, sp, tcg_gen_mov_i32) +DO_VFP_2OP(VMOV_reg, dp, tcg_gen_mov_i64) -static bool trans_VABS_sp(DisasContext *s, arg_VABS_sp *a) -{ - return do_vfp_2op_sp(s, gen_helper_vfp_abss, a->vd, a->vm); -} +DO_VFP_2OP(VABS, sp, gen_helper_vfp_abss) +DO_VFP_2OP(VABS, dp, gen_helper_vfp_absd) -static bool trans_VABS_dp(DisasContext *s, arg_VABS_dp *a) -{ - return do_vfp_2op_dp(s, gen_helper_vfp_absd, a->vd, a->vm); -} - -static bool trans_VNEG_sp(DisasContext *s, arg_VNEG_sp *a) -{ - return do_vfp_2op_sp(s, gen_helper_vfp_negs, a->vd, a->vm); -} - -static bool trans_VNEG_dp(DisasContext *s, arg_VNEG_dp *a) -{ - return do_vfp_2op_dp(s, gen_helper_vfp_negd, a->vd, a->vm); -} +DO_VFP_2OP(VNEG, sp, gen_helper_vfp_negs) +DO_VFP_2OP(VNEG, dp, gen_helper_vfp_negd) static void gen_VSQRT_sp(TCGv_i32 vd, TCGv_i32 vm) { gen_helper_vfp_sqrts(vd, vm, cpu_env); } -static bool trans_VSQRT_sp(DisasContext *s, arg_VSQRT_sp *a) -{ - return do_vfp_2op_sp(s, gen_VSQRT_sp, a->vd, a->vm); -} - static void gen_VSQRT_dp(TCGv_i64 vd, TCGv_i64 vm) { gen_helper_vfp_sqrtd(vd, vm, cpu_env); } -static bool trans_VSQRT_dp(DisasContext *s, arg_VSQRT_dp *a) -{ - return do_vfp_2op_dp(s, gen_VSQRT_dp, a->vd, a->vm); -} +DO_VFP_2OP(VSQRT, sp, gen_VSQRT_sp) +DO_VFP_2OP(VSQRT, dp, gen_VSQRT_dp) static bool trans_VCMP_sp(DisasContext *s, arg_VCMP_sp *a) { -- 2.20.1 Implement VFP fp16 for VABS, VNEG and VSQRT. This is all the fp16 insns that use the DO_VFP_2OP macro, because there is no fp16 version of VMOV_reg. Notes: * the gen_helper_vfp_negh already exists as we needed to create it for the fp16 multiply-add insns * as usual we need to use the f16 version of the fp_status; this is only relevant for VSQRT Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-9-peter.maydell@linaro.org --- target/arm/helper.h | 2 ++ target/arm/vfp.decode | 3 +++ target/arm/vfp_helper.c | 10 +++++++++ target/arm/translate-vfp.c.inc | 40 ++++++++++++++++++++++++++++++++++ 4 files changed, 55 insertions(+) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_3(vfp_minnumd, f64, f64, f64, ptr) DEF_HELPER_1(vfp_negh, f16, f16) DEF_HELPER_1(vfp_negs, f32, f32) DEF_HELPER_1(vfp_negd, f64, f64) +DEF_HELPER_1(vfp_absh, f16, f16) DEF_HELPER_1(vfp_abss, f32, f32) DEF_HELPER_1(vfp_absd, f64, f64) +DEF_HELPER_2(vfp_sqrth, f16, f16, env) DEF_HELPER_2(vfp_sqrts, f32, f32, env) DEF_HELPER_2(vfp_sqrtd, f64, f64, env) DEF_HELPER_3(vfp_cmps, void, f32, f32, env) diff --git a/target/arm/vfp.decode b/target/arm/vfp.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vfp.decode +++ b/target/arm/vfp.decode @@ -XXX,XX +XXX,XX @@ VMOV_imm_dp ---- 1110 1.11 .... .... 1011 0000 .... \ VMOV_reg_sp ---- 1110 1.11 0000 .... 1010 01.0 .... @vfp_dm_ss VMOV_reg_dp ---- 1110 1.11 0000 .... 1011 01.0 .... @vfp_dm_dd +VABS_hp ---- 1110 1.11 0000 .... 1001 11.0 .... @vfp_dm_ss VABS_sp ---- 1110 1.11 0000 .... 1010 11.0 .... @vfp_dm_ss VABS_dp ---- 1110 1.11 0000 .... 1011 11.0 .... @vfp_dm_dd +VNEG_hp ---- 1110 1.11 0001 .... 1001 01.0 .... @vfp_dm_ss VNEG_sp ---- 1110 1.11 0001 .... 1010 01.0 .... @vfp_dm_ss VNEG_dp ---- 1110 1.11 0001 .... 1011 01.0 .... @vfp_dm_dd +VSQRT_hp ---- 1110 1.11 0001 .... 1001 11.0 .... @vfp_dm_ss VSQRT_sp ---- 1110 1.11 0001 .... 1010 11.0 .... @vfp_dm_ss VSQRT_dp ---- 1110 1.11 0001 .... 1011 11.0 .... @vfp_dm_dd diff --git a/target/arm/vfp_helper.c b/target/arm/vfp_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vfp_helper.c +++ b/target/arm/vfp_helper.c @@ -XXX,XX +XXX,XX @@ float64 VFP_HELPER(neg, d)(float64 a) return float64_chs(a); } +dh_ctype_f16 VFP_HELPER(abs, h)(dh_ctype_f16 a) +{ + return float16_abs(a); +} + float32 VFP_HELPER(abs, s)(float32 a) { return float32_abs(a); @@ -XXX,XX +XXX,XX @@ float64 VFP_HELPER(abs, d)(float64 a) return float64_abs(a); } +dh_ctype_f16 VFP_HELPER(sqrt, h)(dh_ctype_f16 a, CPUARMState *env) +{ + return float16_sqrt(a, &env->vfp.fp_status_f16); +} + float32 VFP_HELPER(sqrt, s)(float32 a, CPUARMState *env) { return float32_sqrt(a, &env->vfp.fp_status); diff --git a/target/arm/translate-vfp.c.inc b/target/arm/translate-vfp.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-vfp.c.inc +++ b/target/arm/translate-vfp.c.inc @@ -XXX,XX +XXX,XX @@ static bool do_vfp_2op_sp(DisasContext *s, VFPGen2OpSPFn *fn, int vd, int vm) return true; } +static bool do_vfp_2op_hp(DisasContext *s, VFPGen2OpSPFn *fn, int vd, int vm) +{ + /* + * Do a half-precision operation. Functionally this is + * the same as do_vfp_2op_sp(), except: + * - it doesn't need the VFP vector handling (fp16 is a + * v8 feature, and in v8 VFP vectors don't exist) + * - it does the aa32_fp16_arith feature test + */ + TCGv_i32 f0; + + if (!dc_isar_feature(aa32_fp16_arith, s)) { + return false; + } + + if (s->vec_len != 0 || s->vec_stride != 0) { + return false; + } + + if (!vfp_access_check(s)) { + return true; + } + + f0 = tcg_temp_new_i32(); + neon_load_reg32(f0, vm); + fn(f0, f0); + neon_store_reg32(f0, vd); + tcg_temp_free_i32(f0); + + return true; +} + static bool do_vfp_2op_dp(DisasContext *s, VFPGen2OpDPFn *fn, int vd, int vm) { uint32_t delta_m = 0; @@ -XXX,XX +XXX,XX @@ static bool trans_VMOV_imm_dp(DisasContext *s, arg_VMOV_imm_dp *a) DO_VFP_2OP(VMOV_reg, sp, tcg_gen_mov_i32) DO_VFP_2OP(VMOV_reg, dp, tcg_gen_mov_i64) +DO_VFP_2OP(VABS, hp, gen_helper_vfp_absh) DO_VFP_2OP(VABS, sp, gen_helper_vfp_abss) DO_VFP_2OP(VABS, dp, gen_helper_vfp_absd) +DO_VFP_2OP(VNEG, hp, gen_helper_vfp_negh) DO_VFP_2OP(VNEG, sp, gen_helper_vfp_negs) DO_VFP_2OP(VNEG, dp, gen_helper_vfp_negd) +static void gen_VSQRT_hp(TCGv_i32 vd, TCGv_i32 vm) +{ + gen_helper_vfp_sqrth(vd, vm, cpu_env); +} + static void gen_VSQRT_sp(TCGv_i32 vd, TCGv_i32 vm) { gen_helper_vfp_sqrts(vd, vm, cpu_env); @@ -XXX,XX +XXX,XX @@ static void gen_VSQRT_dp(TCGv_i64 vd, TCGv_i64 vm) gen_helper_vfp_sqrtd(vd, vm, cpu_env); } +DO_VFP_2OP(VSQRT, hp, gen_VSQRT_hp) DO_VFP_2OP(VSQRT, sp, gen_VSQRT_sp) DO_VFP_2OP(VSQRT, dp, gen_VSQRT_dp) -- 2.20.1 Implement VFP fp16 support for the VMOV immediate insn. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-10-peter.maydell@linaro.org --- target/arm/vfp.decode | 2 ++ target/arm/translate-vfp.c.inc | 22 ++++++++++++++++++++++ 2 files changed, 24 insertions(+) diff --git a/target/arm/vfp.decode b/target/arm/vfp.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vfp.decode +++ b/target/arm/vfp.decode @@ -XXX,XX +XXX,XX @@ VFMS_dp ---- 1110 1.10 .... .... 1011 .1.0 .... @vfp_dnm_d VFNMA_dp ---- 1110 1.01 .... .... 1011 .0.0 .... @vfp_dnm_d VFNMS_dp ---- 1110 1.01 .... .... 1011 .1.0 .... @vfp_dnm_d +VMOV_imm_hp ---- 1110 1.11 .... .... 1001 0000 .... \ + vd=%vd_sp imm=%vmov_imm VMOV_imm_sp ---- 1110 1.11 .... .... 1010 0000 .... \ vd=%vd_sp imm=%vmov_imm VMOV_imm_dp ---- 1110 1.11 .... .... 1011 0000 .... \ diff --git a/target/arm/translate-vfp.c.inc b/target/arm/translate-vfp.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-vfp.c.inc +++ b/target/arm/translate-vfp.c.inc @@ -XXX,XX +XXX,XX @@ MAKE_VFM_TRANS_FNS(hp) MAKE_VFM_TRANS_FNS(sp) MAKE_VFM_TRANS_FNS(dp) +static bool trans_VMOV_imm_hp(DisasContext *s, arg_VMOV_imm_sp *a) +{ + TCGv_i32 fd; + + if (!dc_isar_feature(aa32_fp16_arith, s)) { + return false; + } + + if (s->vec_len != 0 || s->vec_stride != 0) { + return false; + } + + if (!vfp_access_check(s)) { + return true; + } + + fd = tcg_const_i32(vfp_expand_imm(MO_16, a->imm)); + neon_store_reg32(fd, a->vd); + tcg_temp_free_i32(fd); + return true; +} + static bool trans_VMOV_imm_sp(DisasContext *s, arg_VMOV_imm_sp *a) { uint32_t delta_d = 0; -- 2.20.1 Implement fp16 version of VCMP. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-11-peter.maydell@linaro.org --- target/arm/helper.h | 2 ++ target/arm/vfp.decode | 2 ++ target/arm/vfp_helper.c | 15 +++++++------ target/arm/translate-vfp.c.inc | 39 ++++++++++++++++++++++++++++++++++ 4 files changed, 51 insertions(+), 7 deletions(-) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_1(vfp_absd, f64, f64) DEF_HELPER_2(vfp_sqrth, f16, f16, env) DEF_HELPER_2(vfp_sqrts, f32, f32, env) DEF_HELPER_2(vfp_sqrtd, f64, f64, env) +DEF_HELPER_3(vfp_cmph, void, f16, f16, env) DEF_HELPER_3(vfp_cmps, void, f32, f32, env) DEF_HELPER_3(vfp_cmpd, void, f64, f64, env) +DEF_HELPER_3(vfp_cmpeh, void, f16, f16, env) DEF_HELPER_3(vfp_cmpes, void, f32, f32, env) DEF_HELPER_3(vfp_cmped, void, f64, f64, env) diff --git a/target/arm/vfp.decode b/target/arm/vfp.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vfp.decode +++ b/target/arm/vfp.decode @@ -XXX,XX +XXX,XX @@ VSQRT_hp ---- 1110 1.11 0001 .... 1001 11.0 .... @vfp_dm_ss VSQRT_sp ---- 1110 1.11 0001 .... 1010 11.0 .... @vfp_dm_ss VSQRT_dp ---- 1110 1.11 0001 .... 1011 11.0 .... @vfp_dm_dd +VCMP_hp ---- 1110 1.11 010 z:1 .... 1001 e:1 1.0 .... \ + vd=%vd_sp vm=%vm_sp VCMP_sp ---- 1110 1.11 010 z:1 .... 1010 e:1 1.0 .... \ vd=%vd_sp vm=%vm_sp VCMP_dp ---- 1110 1.11 010 z:1 .... 1011 e:1 1.0 .... \ diff --git a/target/arm/vfp_helper.c b/target/arm/vfp_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vfp_helper.c +++ b/target/arm/vfp_helper.c @@ -XXX,XX +XXX,XX @@ static void softfloat_to_vfp_compare(CPUARMState *env, FloatRelation cmp) } /* XXX: check quiet/signaling case */ -#define DO_VFP_cmp(p, type) \ -void VFP_HELPER(cmp, p)(type a, type b, CPUARMState *env) \ +#define DO_VFP_cmp(P, FLOATTYPE, ARGTYPE, FPST) \ +void VFP_HELPER(cmp, P)(ARGTYPE a, ARGTYPE b, CPUARMState *env) \ { \ softfloat_to_vfp_compare(env, \ - type ## _compare_quiet(a, b, &env->vfp.fp_status)); \ + FLOATTYPE ## _compare_quiet(a, b, &env->vfp.FPST)); \ } \ -void VFP_HELPER(cmpe, p)(type a, type b, CPUARMState *env) \ +void VFP_HELPER(cmpe, P)(ARGTYPE a, ARGTYPE b, CPUARMState *env) \ { \ softfloat_to_vfp_compare(env, \ - type ## _compare(a, b, &env->vfp.fp_status)); \ + FLOATTYPE ## _compare(a, b, &env->vfp.FPST)); \ } -DO_VFP_cmp(s, float32) -DO_VFP_cmp(d, float64) +DO_VFP_cmp(h, float16, dh_ctype_f16, fp_status_f16) +DO_VFP_cmp(s, float32, float32, fp_status) +DO_VFP_cmp(d, float64, float64, fp_status) #undef DO_VFP_cmp /* Integer to float and float to integer conversions */ diff --git a/target/arm/translate-vfp.c.inc b/target/arm/translate-vfp.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-vfp.c.inc +++ b/target/arm/translate-vfp.c.inc @@ -XXX,XX +XXX,XX @@ DO_VFP_2OP(VSQRT, hp, gen_VSQRT_hp) DO_VFP_2OP(VSQRT, sp, gen_VSQRT_sp) DO_VFP_2OP(VSQRT, dp, gen_VSQRT_dp) +static bool trans_VCMP_hp(DisasContext *s, arg_VCMP_sp *a) +{ + TCGv_i32 vd, vm; + + if (!dc_isar_feature(aa32_fp16_arith, s)) { + return false; + } + + /* Vm/M bits must be zero for the Z variant */ + if (a->z && a->vm != 0) { + return false; + } + + if (!vfp_access_check(s)) { + return true; + } + + vd = tcg_temp_new_i32(); + vm = tcg_temp_new_i32(); + + neon_load_reg32(vd, a->vd); + if (a->z) { + tcg_gen_movi_i32(vm, 0); + } else { + neon_load_reg32(vm, a->vm); + } + + if (a->e) { + gen_helper_vfp_cmpeh(vd, vm, cpu_env); + } else { + gen_helper_vfp_cmph(vd, vm, cpu_env); + } + + tcg_temp_free_i32(vd); + tcg_temp_free_i32(vm); + + return true; +} + static bool trans_VCMP_sp(DisasContext *s, arg_VCMP_sp *a) { TCGv_i32 vd, vm; -- 2.20.1 Implement the fp16 versions of the VFP VLDR/VSTR (immediate). Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-12-peter.maydell@linaro.org --- target/arm/vfp.decode | 3 +-- target/arm/translate-vfp.c.inc | 35 ++++++++++++++++++++++++++++++++++ 2 files changed, 36 insertions(+), 2 deletions(-) diff --git a/target/arm/vfp.decode b/target/arm/vfp.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vfp.decode +++ b/target/arm/vfp.decode @@ -XXX,XX +XXX,XX @@ VMOV_single ---- 1110 000 l:1 .... rt:4 1010 . 001 0000 vn=%vn_sp VMOV_64_sp ---- 1100 010 op:1 rt2:4 rt:4 1010 00.1 .... vm=%vm_sp VMOV_64_dp ---- 1100 010 op:1 rt2:4 rt:4 1011 00.1 .... vm=%vm_dp -# Note that the half-precision variants of VLDR and VSTR are -# not part of this decodetree at all because they have bits [9:8] == 0b01 +VLDR_VSTR_hp ---- 1101 u:1 .0 l:1 rn:4 .... 1001 imm:8 vd=%vd_sp VLDR_VSTR_sp ---- 1101 u:1 .0 l:1 rn:4 .... 1010 imm:8 vd=%vd_sp VLDR_VSTR_dp ---- 1101 u:1 .0 l:1 rn:4 .... 1011 imm:8 vd=%vd_dp diff --git a/target/arm/translate-vfp.c.inc b/target/arm/translate-vfp.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-vfp.c.inc +++ b/target/arm/translate-vfp.c.inc @@ -XXX,XX +XXX,XX @@ static bool trans_VMOV_64_dp(DisasContext *s, arg_VMOV_64_dp *a) return true; } +static bool trans_VLDR_VSTR_hp(DisasContext *s, arg_VLDR_VSTR_sp *a) +{ + uint32_t offset; + TCGv_i32 addr, tmp; + + if (!dc_isar_feature(aa32_fp16_arith, s)) { + return false; + } + + if (!vfp_access_check(s)) { + return true; + } + + /* imm8 field is offset/2 for fp16, unlike fp32 and fp64 */ + offset = a->imm << 1; + if (!a->u) { + offset = -offset; + } + + /* For thumb, use of PC is UNPREDICTABLE. */ + addr = add_reg_for_lit(s, a->rn, offset); + tmp = tcg_temp_new_i32(); + if (a->l) { + gen_aa32_ld16u(s, tmp, addr, get_mem_index(s)); + neon_store_reg32(tmp, a->vd); + } else { + neon_load_reg32(tmp, a->vd); + gen_aa32_st16(s, tmp, addr, get_mem_index(s)); + } + tcg_temp_free_i32(tmp); + tcg_temp_free_i32(addr); + + return true; +} + static bool trans_VLDR_VSTR_sp(DisasContext *s, arg_VLDR_VSTR_sp *a) { uint32_t offset; -- 2.20.1 Implement the fp16 versions of the VFP VCVT instruction forms which convert between floating point and integer. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-13-peter.maydell@linaro.org --- target/arm/vfp.decode | 4 +++ target/arm/translate-vfp.c.inc | 65 ++++++++++++++++++++++++++++++++++ 2 files changed, 69 insertions(+) diff --git a/target/arm/vfp.decode b/target/arm/vfp.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vfp.decode +++ b/target/arm/vfp.decode @@ -XXX,XX +XXX,XX @@ VCVT_sp ---- 1110 1.11 0111 .... 1010 11.0 .... @vfp_dm_ds VCVT_dp ---- 1110 1.11 0111 .... 1011 11.0 .... @vfp_dm_sd # VCVT from integer to floating point: Vm always single; Vd depends on size +VCVT_int_hp ---- 1110 1.11 1000 .... 1001 s:1 1.0 .... \ + vd=%vd_sp vm=%vm_sp VCVT_int_sp ---- 1110 1.11 1000 .... 1010 s:1 1.0 .... \ vd=%vd_sp vm=%vm_sp VCVT_int_dp ---- 1110 1.11 1000 .... 1011 s:1 1.0 .... \ @@ -XXX,XX +XXX,XX @@ VCVT_fix_dp ---- 1110 1.11 1.1. .... 1011 .1.0 .... \ vd=%vd_dp imm=%vm_sp opc=%vcvt_fix_op # VCVT float to integer (VCVT and VCVTR): Vd always single; Vd depends on size +VCVT_hp_int ---- 1110 1.11 110 s:1 .... 1001 rz:1 1.0 .... \ + vd=%vd_sp vm=%vm_sp VCVT_sp_int ---- 1110 1.11 110 s:1 .... 1010 rz:1 1.0 .... \ vd=%vd_sp vm=%vm_sp VCVT_dp_int ---- 1110 1.11 110 s:1 .... 1011 rz:1 1.0 .... \ diff --git a/target/arm/translate-vfp.c.inc b/target/arm/translate-vfp.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-vfp.c.inc +++ b/target/arm/translate-vfp.c.inc @@ -XXX,XX +XXX,XX @@ static bool trans_VCVT_dp(DisasContext *s, arg_VCVT_dp *a) return true; } +static bool trans_VCVT_int_hp(DisasContext *s, arg_VCVT_int_sp *a) +{ + TCGv_i32 vm; + TCGv_ptr fpst; + + if (!dc_isar_feature(aa32_fp16_arith, s)) { + return false; + } + + if (!vfp_access_check(s)) { + return true; + } + + vm = tcg_temp_new_i32(); + neon_load_reg32(vm, a->vm); + fpst = fpstatus_ptr(FPST_FPCR_F16); + if (a->s) { + /* i32 -> f16 */ + gen_helper_vfp_sitoh(vm, vm, fpst); + } else { + /* u32 -> f16 */ + gen_helper_vfp_uitoh(vm, vm, fpst); + } + neon_store_reg32(vm, a->vd); + tcg_temp_free_i32(vm); + tcg_temp_free_ptr(fpst); + return true; +} + static bool trans_VCVT_int_sp(DisasContext *s, arg_VCVT_int_sp *a) { TCGv_i32 vm; @@ -XXX,XX +XXX,XX @@ static bool trans_VCVT_fix_dp(DisasContext *s, arg_VCVT_fix_dp *a) return true; } +static bool trans_VCVT_hp_int(DisasContext *s, arg_VCVT_sp_int *a) +{ + TCGv_i32 vm; + TCGv_ptr fpst; + + if (!dc_isar_feature(aa32_fp16_arith, s)) { + return false; + } + + if (!vfp_access_check(s)) { + return true; + } + + fpst = fpstatus_ptr(FPST_FPCR_F16); + vm = tcg_temp_new_i32(); + neon_load_reg32(vm, a->vm); + + if (a->s) { + if (a->rz) { + gen_helper_vfp_tosizh(vm, vm, fpst); + } else { + gen_helper_vfp_tosih(vm, vm, fpst); + } + } else { + if (a->rz) { + gen_helper_vfp_touizh(vm, vm, fpst); + } else { + gen_helper_vfp_touih(vm, vm, fpst); + } + } + neon_store_reg32(vm, a->vd); + tcg_temp_free_i32(vm); + tcg_temp_free_ptr(fpst); + return true; +} + static bool trans_VCVT_sp_int(DisasContext *s, arg_VCVT_sp_int *a) { TCGv_i32 vm; -- 2.20.1 Currently the VFP_CONV_FIX macros take a single fsz argument for the size of the float type, which is used both to select the name of the functions to call (eg float32_is_any_nan()) and also for the type to use for the float inputs and outputs (eg float32). Separate these into fsz and ftype arguments, so that we can use them for fp16, which uses 'float16' in the function names but is still passing inputs and outputs in a 32-bit sized type. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-14-peter.maydell@linaro.org --- target/arm/vfp_helper.c | 46 ++++++++++++++++++++--------------------- 1 file changed, 23 insertions(+), 23 deletions(-) diff --git a/target/arm/vfp_helper.c b/target/arm/vfp_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vfp_helper.c +++ b/target/arm/vfp_helper.c @@ -XXX,XX +XXX,XX @@ float32 VFP_HELPER(fcvts, d)(float64 x, CPUARMState *env) } /* VFP3 fixed point conversion. */ -#define VFP_CONV_FIX_FLOAT(name, p, fsz, isz, itype) \ -float##fsz HELPER(vfp_##name##to##p)(uint##isz##_t x, uint32_t shift, \ +#define VFP_CONV_FIX_FLOAT(name, p, fsz, ftype, isz, itype) \ +ftype HELPER(vfp_##name##to##p)(uint##isz##_t x, uint32_t shift, \ void *fpstp) \ { return itype##_to_##float##fsz##_scalbn(x, -shift, fpstp); } -#define VFP_CONV_FLOAT_FIX_ROUND(name, p, fsz, isz, itype, ROUND, suff) \ -uint##isz##_t HELPER(vfp_to##name##p##suff)(float##fsz x, uint32_t shift, \ +#define VFP_CONV_FLOAT_FIX_ROUND(name, p, fsz, ftype, isz, itype, ROUND, suff) \ +uint##isz##_t HELPER(vfp_to##name##p##suff)(ftype x, uint32_t shift, \ void *fpst) \ { \ if (unlikely(float##fsz##_is_any_nan(x))) { \ @@ -XXX,XX +XXX,XX @@ uint##isz##_t HELPER(vfp_to##name##p##suff)(float##fsz x, uint32_t shift, \ return float##fsz##_to_##itype##_scalbn(x, ROUND, shift, fpst); \ } -#define VFP_CONV_FIX(name, p, fsz, isz, itype) \ -VFP_CONV_FIX_FLOAT(name, p, fsz, isz, itype) \ -VFP_CONV_FLOAT_FIX_ROUND(name, p, fsz, isz, itype, \ +#define VFP_CONV_FIX(name, p, fsz, ftype, isz, itype) \ +VFP_CONV_FIX_FLOAT(name, p, fsz, ftype, isz, itype) \ +VFP_CONV_FLOAT_FIX_ROUND(name, p, fsz, ftype, isz, itype, \ float_round_to_zero, _round_to_zero) \ -VFP_CONV_FLOAT_FIX_ROUND(name, p, fsz, isz, itype, \ +VFP_CONV_FLOAT_FIX_ROUND(name, p, fsz, ftype, isz, itype, \ get_float_rounding_mode(fpst), ) -#define VFP_CONV_FIX_A64(name, p, fsz, isz, itype) \ -VFP_CONV_FIX_FLOAT(name, p, fsz, isz, itype) \ -VFP_CONV_FLOAT_FIX_ROUND(name, p, fsz, isz, itype, \ +#define VFP_CONV_FIX_A64(name, p, fsz, ftype, isz, itype) \ +VFP_CONV_FIX_FLOAT(name, p, fsz, ftype, isz, itype) \ +VFP_CONV_FLOAT_FIX_ROUND(name, p, fsz, ftype, isz, itype, \ get_float_rounding_mode(fpst), ) -VFP_CONV_FIX(sh, d, 64, 64, int16) -VFP_CONV_FIX(sl, d, 64, 64, int32) -VFP_CONV_FIX_A64(sq, d, 64, 64, int64) -VFP_CONV_FIX(uh, d, 64, 64, uint16) -VFP_CONV_FIX(ul, d, 64, 64, uint32) -VFP_CONV_FIX_A64(uq, d, 64, 64, uint64) -VFP_CONV_FIX(sh, s, 32, 32, int16) -VFP_CONV_FIX(sl, s, 32, 32, int32) -VFP_CONV_FIX_A64(sq, s, 32, 64, int64) -VFP_CONV_FIX(uh, s, 32, 32, uint16) -VFP_CONV_FIX(ul, s, 32, 32, uint32) -VFP_CONV_FIX_A64(uq, s, 32, 64, uint64) +VFP_CONV_FIX(sh, d, 64, float64, 64, int16) +VFP_CONV_FIX(sl, d, 64, float64, 64, int32) +VFP_CONV_FIX_A64(sq, d, 64, float64, 64, int64) +VFP_CONV_FIX(uh, d, 64, float64, 64, uint16) +VFP_CONV_FIX(ul, d, 64, float64, 64, uint32) +VFP_CONV_FIX_A64(uq, d, 64, float64, 64, uint64) +VFP_CONV_FIX(sh, s, 32, float32, 32, int16) +VFP_CONV_FIX(sl, s, 32, float32, 32, int32) +VFP_CONV_FIX_A64(sq, s, 32, float32, 64, int64) +VFP_CONV_FIX(uh, s, 32, float32, 32, uint16) +VFP_CONV_FIX(ul, s, 32, float32, 32, uint32) +VFP_CONV_FIX_A64(uq, s, 32, float32, 64, uint64) #undef VFP_CONV_FIX #undef VFP_CONV_FIX_FLOAT -- 2.20.1 Now the VFP_CONV_FIX macros can handle fp16's distinction between the width of the operation and the width of the type used to pass operands, use the macros rather than the open-coded functions. This creates an extra six helper functions, all of which we are going to need for the AArch32 VFP fp16 instructions. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-15-peter.maydell@linaro.org --- target/arm/helper.h | 6 +++ target/arm/vfp_helper.c | 86 +++-------------------------------------- 2 files changed, 12 insertions(+), 80 deletions(-) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_2(vfp_tosizh, s32, f16, ptr) DEF_HELPER_2(vfp_tosizs, s32, f32, ptr) DEF_HELPER_2(vfp_tosizd, s32, f64, ptr) +DEF_HELPER_3(vfp_toshh_round_to_zero, i32, f16, i32, ptr) +DEF_HELPER_3(vfp_toslh_round_to_zero, i32, f16, i32, ptr) +DEF_HELPER_3(vfp_touhh_round_to_zero, i32, f16, i32, ptr) +DEF_HELPER_3(vfp_toulh_round_to_zero, i32, f16, i32, ptr) DEF_HELPER_3(vfp_toshs_round_to_zero, i32, f32, i32, ptr) DEF_HELPER_3(vfp_tosls_round_to_zero, i32, f32, i32, ptr) DEF_HELPER_3(vfp_touhs_round_to_zero, i32, f32, i32, ptr) @@ -XXX,XX +XXX,XX @@ DEF_HELPER_3(vfp_sqtod, f64, i64, i32, ptr) DEF_HELPER_3(vfp_uhtod, f64, i64, i32, ptr) DEF_HELPER_3(vfp_ultod, f64, i64, i32, ptr) DEF_HELPER_3(vfp_uqtod, f64, i64, i32, ptr) +DEF_HELPER_3(vfp_shtoh, f16, i32, i32, ptr) +DEF_HELPER_3(vfp_uhtoh, f16, i32, i32, ptr) DEF_HELPER_3(vfp_sltoh, f16, i32, i32, ptr) DEF_HELPER_3(vfp_ultoh, f16, i32, i32, ptr) DEF_HELPER_3(vfp_sqtoh, f16, i64, i32, ptr) diff --git a/target/arm/vfp_helper.c b/target/arm/vfp_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vfp_helper.c +++ b/target/arm/vfp_helper.c @@ -XXX,XX +XXX,XX @@ VFP_CONV_FIX_A64(sq, s, 32, float32, 64, int64) VFP_CONV_FIX(uh, s, 32, float32, 32, uint16) VFP_CONV_FIX(ul, s, 32, float32, 32, uint32) VFP_CONV_FIX_A64(uq, s, 32, float32, 64, uint64) +VFP_CONV_FIX(sh, h, 16, dh_ctype_f16, 32, int16) +VFP_CONV_FIX(sl, h, 16, dh_ctype_f16, 32, int32) +VFP_CONV_FIX_A64(sq, h, 16, dh_ctype_f16, 64, int64) +VFP_CONV_FIX(uh, h, 16, dh_ctype_f16, 32, uint16) +VFP_CONV_FIX(ul, h, 16, dh_ctype_f16, 32, uint32) +VFP_CONV_FIX_A64(uq, h, 16, dh_ctype_f16, 64, uint64) #undef VFP_CONV_FIX #undef VFP_CONV_FIX_FLOAT #undef VFP_CONV_FLOAT_FIX_ROUND #undef VFP_CONV_FIX_A64 -uint32_t HELPER(vfp_sltoh)(uint32_t x, uint32_t shift, void *fpst) -{ - return int32_to_float16_scalbn(x, -shift, fpst); -} - -uint32_t HELPER(vfp_ultoh)(uint32_t x, uint32_t shift, void *fpst) -{ - return uint32_to_float16_scalbn(x, -shift, fpst); -} - -uint32_t HELPER(vfp_sqtoh)(uint64_t x, uint32_t shift, void *fpst) -{ - return int64_to_float16_scalbn(x, -shift, fpst); -} - -uint32_t HELPER(vfp_uqtoh)(uint64_t x, uint32_t shift, void *fpst) -{ - return uint64_to_float16_scalbn(x, -shift, fpst); -} - -uint32_t HELPER(vfp_toshh)(uint32_t x, uint32_t shift, void *fpst) -{ - if (unlikely(float16_is_any_nan(x))) { - float_raise(float_flag_invalid, fpst); - return 0; - } - return float16_to_int16_scalbn(x, get_float_rounding_mode(fpst), - shift, fpst); -} - -uint32_t HELPER(vfp_touhh)(uint32_t x, uint32_t shift, void *fpst) -{ - if (unlikely(float16_is_any_nan(x))) { - float_raise(float_flag_invalid, fpst); - return 0; - } - return float16_to_uint16_scalbn(x, get_float_rounding_mode(fpst), - shift, fpst); -} - -uint32_t HELPER(vfp_toslh)(uint32_t x, uint32_t shift, void *fpst) -{ - if (unlikely(float16_is_any_nan(x))) { - float_raise(float_flag_invalid, fpst); - return 0; - } - return float16_to_int32_scalbn(x, get_float_rounding_mode(fpst), - shift, fpst); -} - -uint32_t HELPER(vfp_toulh)(uint32_t x, uint32_t shift, void *fpst) -{ - if (unlikely(float16_is_any_nan(x))) { - float_raise(float_flag_invalid, fpst); - return 0; - } - return float16_to_uint32_scalbn(x, get_float_rounding_mode(fpst), - shift, fpst); -} - -uint64_t HELPER(vfp_tosqh)(uint32_t x, uint32_t shift, void *fpst) -{ - if (unlikely(float16_is_any_nan(x))) { - float_raise(float_flag_invalid, fpst); - return 0; - } - return float16_to_int64_scalbn(x, get_float_rounding_mode(fpst), - shift, fpst); -} - -uint64_t HELPER(vfp_touqh)(uint32_t x, uint32_t shift, void *fpst) -{ - if (unlikely(float16_is_any_nan(x))) { - float_raise(float_flag_invalid, fpst); - return 0; - } - return float16_to_uint64_scalbn(x, get_float_rounding_mode(fpst), - shift, fpst); -} - /* Set the current fp rounding mode and return the old one. * The argument is a softfloat float_round_ value. */ -- 2.20.1 Implement the fp16 versions of the VFP VCVT instruction forms which convert between floating point and fixed-point. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-16-peter.maydell@linaro.org --- target/arm/vfp.decode | 2 ++ target/arm/translate-vfp.c.inc | 59 ++++++++++++++++++++++++++++++++++ 2 files changed, 61 insertions(+) diff --git a/target/arm/vfp.decode b/target/arm/vfp.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vfp.decode +++ b/target/arm/vfp.decode @@ -XXX,XX +XXX,XX @@ VJCVT ---- 1110 1.11 1001 .... 1011 11.0 .... @vfp_dm_sd # We assemble bits 18 (op), 16 (u) and 7 (sx) into a single opc field # for the convenience of the trans_VCVT_fix functions. %vcvt_fix_op 18:1 16:1 7:1 +VCVT_fix_hp ---- 1110 1.11 1.1. .... 1001 .1.0 .... \ + vd=%vd_sp imm=%vm_sp opc=%vcvt_fix_op VCVT_fix_sp ---- 1110 1.11 1.1. .... 1010 .1.0 .... \ vd=%vd_sp imm=%vm_sp opc=%vcvt_fix_op VCVT_fix_dp ---- 1110 1.11 1.1. .... 1011 .1.0 .... \ diff --git a/target/arm/translate-vfp.c.inc b/target/arm/translate-vfp.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-vfp.c.inc +++ b/target/arm/translate-vfp.c.inc @@ -XXX,XX +XXX,XX @@ static bool trans_VJCVT(DisasContext *s, arg_VJCVT *a) return true; } +static bool trans_VCVT_fix_hp(DisasContext *s, arg_VCVT_fix_sp *a) +{ + TCGv_i32 vd, shift; + TCGv_ptr fpst; + int frac_bits; + + if (!dc_isar_feature(aa32_fp16_arith, s)) { + return false; + } + + if (!vfp_access_check(s)) { + return true; + } + + frac_bits = (a->opc & 1) ? (32 - a->imm) : (16 - a->imm); + + vd = tcg_temp_new_i32(); + neon_load_reg32(vd, a->vd); + + fpst = fpstatus_ptr(FPST_FPCR_F16); + shift = tcg_const_i32(frac_bits); + + /* Switch on op:U:sx bits */ + switch (a->opc) { + case 0: + gen_helper_vfp_shtoh(vd, vd, shift, fpst); + break; + case 1: + gen_helper_vfp_sltoh(vd, vd, shift, fpst); + break; + case 2: + gen_helper_vfp_uhtoh(vd, vd, shift, fpst); + break; + case 3: + gen_helper_vfp_ultoh(vd, vd, shift, fpst); + break; + case 4: + gen_helper_vfp_toshh_round_to_zero(vd, vd, shift, fpst); + break; + case 5: + gen_helper_vfp_toslh_round_to_zero(vd, vd, shift, fpst); + break; + case 6: + gen_helper_vfp_touhh_round_to_zero(vd, vd, shift, fpst); + break; + case 7: + gen_helper_vfp_toulh_round_to_zero(vd, vd, shift, fpst); + break; + default: + g_assert_not_reached(); + } + + neon_store_reg32(vd, a->vd); + tcg_temp_free_i32(vd); + tcg_temp_free_i32(shift); + tcg_temp_free_ptr(fpst); + return true; +} + static bool trans_VCVT_fix_sp(DisasContext *s, arg_VCVT_fix_sp *a) { TCGv_i32 vd, shift; -- 2.20.1 Implement the fp16 versions of the VFP VCVT instruction forms which convert between floating point and integer with a specified rounding mode. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-17-peter.maydell@linaro.org --- target/arm/vfp-uncond.decode | 6 ++++-- target/arm/translate-vfp.c.inc | 32 ++++++++++++++++++++++++-------- 2 files changed, 28 insertions(+), 10 deletions(-) diff --git a/target/arm/vfp-uncond.decode b/target/arm/vfp-uncond.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vfp-uncond.decode +++ b/target/arm/vfp-uncond.decode @@ -XXX,XX +XXX,XX @@ VRINT 1111 1110 1.11 10 rm:2 .... 1011 01.0 .... \ vm=%vm_dp vd=%vd_dp dp=1 # VCVT float to int with specified rounding mode; Vd is always single-precision +VCVT 1111 1110 1.11 11 rm:2 .... 1001 op:1 1.0 .... \ + vm=%vm_sp vd=%vd_sp sz=1 VCVT 1111 1110 1.11 11 rm:2 .... 1010 op:1 1.0 .... \ - vm=%vm_sp vd=%vd_sp dp=0 + vm=%vm_sp vd=%vd_sp sz=2 VCVT 1111 1110 1.11 11 rm:2 .... 1011 op:1 1.0 .... \ - vm=%vm_dp vd=%vd_sp dp=1 + vm=%vm_dp vd=%vd_sp sz=3 diff --git a/target/arm/translate-vfp.c.inc b/target/arm/translate-vfp.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-vfp.c.inc +++ b/target/arm/translate-vfp.c.inc @@ -XXX,XX +XXX,XX @@ static bool trans_VRINT(DisasContext *s, arg_VRINT *a) static bool trans_VCVT(DisasContext *s, arg_VCVT *a) { uint32_t rd, rm; - bool dp = a->dp; + int sz = a->sz; TCGv_ptr fpst; TCGv_i32 tcg_rmode, tcg_shift; int rounding = fp_decode_rm[a->rm]; @@ -XXX,XX +XXX,XX @@ static bool trans_VCVT(DisasContext *s, arg_VCVT *a) return false; } - if (dp && !dc_isar_feature(aa32_fpdp_v2, s)) { + if (sz == 3 && !dc_isar_feature(aa32_fpdp_v2, s)) { + return false; + } + + if (sz == 1 && !dc_isar_feature(aa32_fp16_arith, s)) { return false; } /* UNDEF accesses to D16-D31 if they don't exist */ - if (dp && !dc_isar_feature(aa32_simd_r32, s) && (a->vm & 0x10)) { + if (sz == 3 && !dc_isar_feature(aa32_simd_r32, s) && (a->vm & 0x10)) { return false; } @@ -XXX,XX +XXX,XX @@ static bool trans_VCVT(DisasContext *s, arg_VCVT *a) return true; } - fpst = fpstatus_ptr(FPST_FPCR); + if (sz == 1) { + fpst = fpstatus_ptr(FPST_FPCR_F16); + } else { + fpst = fpstatus_ptr(FPST_FPCR); + } tcg_shift = tcg_const_i32(0); tcg_rmode = tcg_const_i32(arm_rmode_to_sf(rounding)); gen_helper_set_rmode(tcg_rmode, tcg_rmode, fpst); - if (dp) { + if (sz == 3) { TCGv_i64 tcg_double, tcg_res; TCGv_i32 tcg_tmp; tcg_double = tcg_temp_new_i64(); @@ -XXX,XX +XXX,XX @@ static bool trans_VCVT(DisasContext *s, arg_VCVT *a) tcg_single = tcg_temp_new_i32(); tcg_res = tcg_temp_new_i32(); neon_load_reg32(tcg_single, rm); - if (is_signed) { - gen_helper_vfp_tosls(tcg_res, tcg_single, tcg_shift, fpst); + if (sz == 1) { + if (is_signed) { + gen_helper_vfp_toslh(tcg_res, tcg_single, tcg_shift, fpst); + } else { + gen_helper_vfp_toulh(tcg_res, tcg_single, tcg_shift, fpst); + } } else { - gen_helper_vfp_touls(tcg_res, tcg_single, tcg_shift, fpst); + if (is_signed) { + gen_helper_vfp_tosls(tcg_res, tcg_single, tcg_shift, fpst); + } else { + gen_helper_vfp_touls(tcg_res, tcg_single, tcg_shift, fpst); + } } neon_store_reg32(tcg_res, rd); tcg_temp_free_i32(tcg_res); -- 2.20.1 Implement the fp16 versions of the VFP VSEL instruction. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-18-peter.maydell@linaro.org --- target/arm/vfp-uncond.decode | 6 ++++-- target/arm/translate-vfp.c.inc | 16 ++++++++++++---- 2 files changed, 16 insertions(+), 6 deletions(-) diff --git a/target/arm/vfp-uncond.decode b/target/arm/vfp-uncond.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vfp-uncond.decode +++ b/target/arm/vfp-uncond.decode @@ -XXX,XX +XXX,XX @@ @vfp_dnm_s ................................ vm=%vm_sp vn=%vn_sp vd=%vd_sp @vfp_dnm_d ................................ vm=%vm_dp vn=%vn_dp vd=%vd_dp +VSEL 1111 1110 0. cc:2 .... .... 1001 .0.0 .... \ + vm=%vm_sp vn=%vn_sp vd=%vd_sp sz=1 VSEL 1111 1110 0. cc:2 .... .... 1010 .0.0 .... \ - vm=%vm_sp vn=%vn_sp vd=%vd_sp dp=0 + vm=%vm_sp vn=%vn_sp vd=%vd_sp sz=2 VSEL 1111 1110 0. cc:2 .... .... 1011 .0.0 .... \ - vm=%vm_dp vn=%vn_dp vd=%vd_dp dp=1 + vm=%vm_dp vn=%vn_dp vd=%vd_dp sz=3 VMAXNM_hp 1111 1110 1.00 .... .... 1001 .0.0 .... @vfp_dnm_s VMINNM_hp 1111 1110 1.00 .... .... 1001 .1.0 .... @vfp_dnm_s diff --git a/target/arm/translate-vfp.c.inc b/target/arm/translate-vfp.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-vfp.c.inc +++ b/target/arm/translate-vfp.c.inc @@ -XXX,XX +XXX,XX @@ static bool vfp_access_check(DisasContext *s) static bool trans_VSEL(DisasContext *s, arg_VSEL *a) { uint32_t rd, rn, rm; - bool dp = a->dp; + int sz = a->sz; if (!dc_isar_feature(aa32_vsel, s)) { return false; } - if (dp && !dc_isar_feature(aa32_fpdp_v2, s)) { + if (sz == 3 && !dc_isar_feature(aa32_fpdp_v2, s)) { + return false; + } + + if (sz == 1 && !dc_isar_feature(aa32_fp16_arith, s)) { return false; } /* UNDEF accesses to D16-D31 if they don't exist */ - if (dp && !dc_isar_feature(aa32_simd_r32, s) && + if (sz == 3 && !dc_isar_feature(aa32_simd_r32, s) && ((a->vm | a->vn | a->vd) & 0x10)) { return false; } @@ -XXX,XX +XXX,XX @@ static bool trans_VSEL(DisasContext *s, arg_VSEL *a) return true; } - if (dp) { + if (sz == 3) { TCGv_i64 frn, frm, dest; TCGv_i64 tmp, zero, zf, nf, vf; @@ -XXX,XX +XXX,XX @@ static bool trans_VSEL(DisasContext *s, arg_VSEL *a) tcg_temp_free_i32(tmp); break; } + /* For fp16 the top half is always zeroes */ + if (sz == 1) { + tcg_gen_andi_i32(dest, dest, 0xffff); + } neon_store_reg32(dest, rd); tcg_temp_free_i32(frn); tcg_temp_free_i32(frm); -- 2.20.1 Implement the fp16 version of the VFP VRINT* insns. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-19-peter.maydell@linaro.org --- target/arm/helper.h | 2 + target/arm/vfp-uncond.decode | 6 ++- target/arm/vfp.decode | 3 ++ target/arm/vfp_helper.c | 21 ++++++++ target/arm/translate-vfp.c.inc | 98 +++++++++++++++++++++++++++++++--- 5 files changed, 122 insertions(+), 8 deletions(-) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_3(shr_cc, i32, env, i32, i32) DEF_HELPER_3(sar_cc, i32, env, i32, i32) DEF_HELPER_3(ror_cc, i32, env, i32, i32) +DEF_HELPER_FLAGS_2(rinth_exact, TCG_CALL_NO_RWG, f16, f16, ptr) DEF_HELPER_FLAGS_2(rints_exact, TCG_CALL_NO_RWG, f32, f32, ptr) DEF_HELPER_FLAGS_2(rintd_exact, TCG_CALL_NO_RWG, f64, f64, ptr) +DEF_HELPER_FLAGS_2(rinth, TCG_CALL_NO_RWG, f16, f16, ptr) DEF_HELPER_FLAGS_2(rints, TCG_CALL_NO_RWG, f32, f32, ptr) DEF_HELPER_FLAGS_2(rintd, TCG_CALL_NO_RWG, f64, f64, ptr) diff --git a/target/arm/vfp-uncond.decode b/target/arm/vfp-uncond.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vfp-uncond.decode +++ b/target/arm/vfp-uncond.decode @@ -XXX,XX +XXX,XX @@ VMINNM_sp 1111 1110 1.00 .... .... 1010 .1.0 .... @vfp_dnm_s VMAXNM_dp 1111 1110 1.00 .... .... 1011 .0.0 .... @vfp_dnm_d VMINNM_dp 1111 1110 1.00 .... .... 1011 .1.0 .... @vfp_dnm_d +VRINT 1111 1110 1.11 10 rm:2 .... 1001 01.0 .... \ + vm=%vm_sp vd=%vd_sp sz=1 VRINT 1111 1110 1.11 10 rm:2 .... 1010 01.0 .... \ - vm=%vm_sp vd=%vd_sp dp=0 + vm=%vm_sp vd=%vd_sp sz=2 VRINT 1111 1110 1.11 10 rm:2 .... 1011 01.0 .... \ - vm=%vm_dp vd=%vd_dp dp=1 + vm=%vm_dp vd=%vd_dp sz=3 # VCVT float to int with specified rounding mode; Vd is always single-precision VCVT 1111 1110 1.11 11 rm:2 .... 1001 op:1 1.0 .... \ diff --git a/target/arm/vfp.decode b/target/arm/vfp.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vfp.decode +++ b/target/arm/vfp.decode @@ -XXX,XX +XXX,XX @@ VCVT_f16_f32 ---- 1110 1.11 0011 .... 1010 t:1 1.0 .... \ VCVT_f16_f64 ---- 1110 1.11 0011 .... 1011 t:1 1.0 .... \ vd=%vd_sp vm=%vm_dp +VRINTR_hp ---- 1110 1.11 0110 .... 1001 01.0 .... @vfp_dm_ss VRINTR_sp ---- 1110 1.11 0110 .... 1010 01.0 .... @vfp_dm_ss VRINTR_dp ---- 1110 1.11 0110 .... 1011 01.0 .... @vfp_dm_dd +VRINTZ_hp ---- 1110 1.11 0110 .... 1001 11.0 .... @vfp_dm_ss VRINTZ_sp ---- 1110 1.11 0110 .... 1010 11.0 .... @vfp_dm_ss VRINTZ_dp ---- 1110 1.11 0110 .... 1011 11.0 .... @vfp_dm_dd +VRINTX_hp ---- 1110 1.11 0111 .... 1001 01.0 .... @vfp_dm_ss VRINTX_sp ---- 1110 1.11 0111 .... 1010 01.0 .... @vfp_dm_ss VRINTX_dp ---- 1110 1.11 0111 .... 1011 01.0 .... @vfp_dm_dd diff --git a/target/arm/vfp_helper.c b/target/arm/vfp_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vfp_helper.c +++ b/target/arm/vfp_helper.c @@ -XXX,XX +XXX,XX @@ float64 VFP_HELPER(muladd, d)(float64 a, float64 b, float64 c, void *fpstp) } /* ARMv8 round to integral */ +dh_ctype_f16 HELPER(rinth_exact)(dh_ctype_f16 x, void *fp_status) +{ + return float16_round_to_int(x, fp_status); +} + float32 HELPER(rints_exact)(float32 x, void *fp_status) { return float32_round_to_int(x, fp_status); @@ -XXX,XX +XXX,XX @@ float64 HELPER(rintd_exact)(float64 x, void *fp_status) return float64_round_to_int(x, fp_status); } +dh_ctype_f16 HELPER(rinth)(dh_ctype_f16 x, void *fp_status) +{ + int old_flags = get_float_exception_flags(fp_status), new_flags; + float16 ret; + + ret = float16_round_to_int(x, fp_status); + + /* Suppress any inexact exceptions the conversion produced */ + if (!(old_flags & float_flag_inexact)) { + new_flags = get_float_exception_flags(fp_status); + set_float_exception_flags(new_flags & ~float_flag_inexact, fp_status); + } + + return ret; +} + float32 HELPER(rints)(float32 x, void *fp_status) { int old_flags = get_float_exception_flags(fp_status), new_flags; diff --git a/target/arm/translate-vfp.c.inc b/target/arm/translate-vfp.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-vfp.c.inc +++ b/target/arm/translate-vfp.c.inc @@ -XXX,XX +XXX,XX @@ static const uint8_t fp_decode_rm[] = { static bool trans_VRINT(DisasContext *s, arg_VRINT *a) { uint32_t rd, rm; - bool dp = a->dp; + int sz = a->sz; TCGv_ptr fpst; TCGv_i32 tcg_rmode; int rounding = fp_decode_rm[a->rm]; @@ -XXX,XX +XXX,XX @@ static bool trans_VRINT(DisasContext *s, arg_VRINT *a) return false; } - if (dp && !dc_isar_feature(aa32_fpdp_v2, s)) { + if (sz == 3 && !dc_isar_feature(aa32_fpdp_v2, s)) { + return false; + } + + if (sz == 1 && !dc_isar_feature(aa32_fp16_arith, s)) { return false; } /* UNDEF accesses to D16-D31 if they don't exist */ - if (dp && !dc_isar_feature(aa32_simd_r32, s) && + if (sz == 3 && !dc_isar_feature(aa32_simd_r32, s) && ((a->vm | a->vd) & 0x10)) { return false; } @@ -XXX,XX +XXX,XX @@ static bool trans_VRINT(DisasContext *s, arg_VRINT *a) return true; } - fpst = fpstatus_ptr(FPST_FPCR); + if (sz == 1) { + fpst = fpstatus_ptr(FPST_FPCR_F16); + } else { + fpst = fpstatus_ptr(FPST_FPCR); + } tcg_rmode = tcg_const_i32(arm_rmode_to_sf(rounding)); gen_helper_set_rmode(tcg_rmode, tcg_rmode, fpst); - if (dp) { + if (sz == 3) { TCGv_i64 tcg_op; TCGv_i64 tcg_res; tcg_op = tcg_temp_new_i64(); @@ -XXX,XX +XXX,XX @@ static bool trans_VRINT(DisasContext *s, arg_VRINT *a) tcg_op = tcg_temp_new_i32(); tcg_res = tcg_temp_new_i32(); neon_load_reg32(tcg_op, rm); - gen_helper_rints(tcg_res, tcg_op, fpst); + if (sz == 1) { + gen_helper_rinth(tcg_res, tcg_op, fpst); + } else { + gen_helper_rints(tcg_res, tcg_op, fpst); + } neon_store_reg32(tcg_res, rd); tcg_temp_free_i32(tcg_op); tcg_temp_free_i32(tcg_res); @@ -XXX,XX +XXX,XX @@ static bool trans_VCVT_f16_f64(DisasContext *s, arg_VCVT_f16_f64 *a) return true; } +static bool trans_VRINTR_hp(DisasContext *s, arg_VRINTR_sp *a) +{ + TCGv_ptr fpst; + TCGv_i32 tmp; + + if (!dc_isar_feature(aa32_fp16_arith, s)) { + return false; + } + + if (!vfp_access_check(s)) { + return true; + } + + tmp = tcg_temp_new_i32(); + neon_load_reg32(tmp, a->vm); + fpst = fpstatus_ptr(FPST_FPCR_F16); + gen_helper_rinth(tmp, tmp, fpst); + neon_store_reg32(tmp, a->vd); + tcg_temp_free_ptr(fpst); + tcg_temp_free_i32(tmp); + return true; +} + static bool trans_VRINTR_sp(DisasContext *s, arg_VRINTR_sp *a) { TCGv_ptr fpst; @@ -XXX,XX +XXX,XX @@ static bool trans_VRINTR_dp(DisasContext *s, arg_VRINTR_dp *a) return true; } +static bool trans_VRINTZ_hp(DisasContext *s, arg_VRINTZ_sp *a) +{ + TCGv_ptr fpst; + TCGv_i32 tmp; + TCGv_i32 tcg_rmode; + + if (!dc_isar_feature(aa32_fp16_arith, s)) { + return false; + } + + if (!vfp_access_check(s)) { + return true; + } + + tmp = tcg_temp_new_i32(); + neon_load_reg32(tmp, a->vm); + fpst = fpstatus_ptr(FPST_FPCR_F16); + tcg_rmode = tcg_const_i32(float_round_to_zero); + gen_helper_set_rmode(tcg_rmode, tcg_rmode, fpst); + gen_helper_rinth(tmp, tmp, fpst); + gen_helper_set_rmode(tcg_rmode, tcg_rmode, fpst); + neon_store_reg32(tmp, a->vd); + tcg_temp_free_ptr(fpst); + tcg_temp_free_i32(tcg_rmode); + tcg_temp_free_i32(tmp); + return true; +} + static bool trans_VRINTZ_sp(DisasContext *s, arg_VRINTZ_sp *a) { TCGv_ptr fpst; @@ -XXX,XX +XXX,XX @@ static bool trans_VRINTZ_dp(DisasContext *s, arg_VRINTZ_dp *a) return true; } +static bool trans_VRINTX_hp(DisasContext *s, arg_VRINTX_sp *a) +{ + TCGv_ptr fpst; + TCGv_i32 tmp; + + if (!dc_isar_feature(aa32_fp16_arith, s)) { + return false; + } + + if (!vfp_access_check(s)) { + return true; + } + + tmp = tcg_temp_new_i32(); + neon_load_reg32(tmp, a->vm); + fpst = fpstatus_ptr(FPST_FPCR_F16); + gen_helper_rinth_exact(tmp, tmp, fpst); + neon_store_reg32(tmp, a->vd); + tcg_temp_free_ptr(fpst); + tcg_temp_free_i32(tmp); + return true; +} + static bool trans_VRINTX_sp(DisasContext *s, arg_VRINTX_sp *a) { TCGv_ptr fpst; -- 2.20.1 The fp16 extension includes a new instruction VINS, which copies the lower 16 bits of a 32-bit source VFP register into the upper 16 bits of the destination. Implement it. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-20-peter.maydell@linaro.org --- target/arm/vfp-uncond.decode | 3 +++ target/arm/translate-vfp.c.inc | 28 ++++++++++++++++++++++++++++ 2 files changed, 31 insertions(+) diff --git a/target/arm/vfp-uncond.decode b/target/arm/vfp-uncond.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vfp-uncond.decode +++ b/target/arm/vfp-uncond.decode @@ -XXX,XX +XXX,XX @@ VCVT 1111 1110 1.11 11 rm:2 .... 1010 op:1 1.0 .... \ vm=%vm_sp vd=%vd_sp sz=2 VCVT 1111 1110 1.11 11 rm:2 .... 1011 op:1 1.0 .... \ vm=%vm_dp vd=%vd_sp sz=3 + +VINS 1111 1110 1.11 0000 .... 1010 11 . 0 .... \ + vd=%vd_sp vm=%vm_sp diff --git a/target/arm/translate-vfp.c.inc b/target/arm/translate-vfp.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-vfp.c.inc +++ b/target/arm/translate-vfp.c.inc @@ -XXX,XX +XXX,XX @@ static bool trans_NOCP(DisasContext *s, arg_NOCP *a) return false; } + +static bool trans_VINS(DisasContext *s, arg_VINS *a) +{ + TCGv_i32 rd, rm; + + if (!dc_isar_feature(aa32_fp16_arith, s)) { + return false; + } + + if (s->vec_len != 0 || s->vec_stride != 0) { + return false; + } + + if (!vfp_access_check(s)) { + return true; + } + + /* Insert low half of Vm into high half of Vd */ + rm = tcg_temp_new_i32(); + rd = tcg_temp_new_i32(); + neon_load_reg32(rm, a->vm); + neon_load_reg32(rd, a->vd); + tcg_gen_deposit_i32(rd, rd, rm, 16, 16); + neon_store_reg32(rd, a->vd); + tcg_temp_free_i32(rm); + tcg_temp_free_i32(rd); + return true; +} -- 2.20.1 The fp16 extension includes a new instruction VMOVX, which copies the upper 16 bits of a 32-bit source VFP register into the lower 16 bits of the destination and zeroes the high half of the destination. Implement it. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-21-peter.maydell@linaro.org --- target/arm/vfp-uncond.decode | 3 +++ target/arm/translate-vfp.c.inc | 25 +++++++++++++++++++++++++ 2 files changed, 28 insertions(+) diff --git a/target/arm/vfp-uncond.decode b/target/arm/vfp-uncond.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vfp-uncond.decode +++ b/target/arm/vfp-uncond.decode @@ -XXX,XX +XXX,XX @@ VCVT 1111 1110 1.11 11 rm:2 .... 1010 op:1 1.0 .... \ VCVT 1111 1110 1.11 11 rm:2 .... 1011 op:1 1.0 .... \ vm=%vm_dp vd=%vd_sp sz=3 +VMOVX 1111 1110 1.11 0000 .... 1010 01 . 0 .... \ + vd=%vd_sp vm=%vm_sp + VINS 1111 1110 1.11 0000 .... 1010 11 . 0 .... \ vd=%vd_sp vm=%vm_sp diff --git a/target/arm/translate-vfp.c.inc b/target/arm/translate-vfp.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-vfp.c.inc +++ b/target/arm/translate-vfp.c.inc @@ -XXX,XX +XXX,XX @@ static bool trans_VINS(DisasContext *s, arg_VINS *a) tcg_temp_free_i32(rd); return true; } + +static bool trans_VMOVX(DisasContext *s, arg_VINS *a) +{ + TCGv_i32 rm; + + if (!dc_isar_feature(aa32_fp16_arith, s)) { + return false; + } + + if (s->vec_len != 0 || s->vec_stride != 0) { + return false; + } + + if (!vfp_access_check(s)) { + return true; + } + + /* Set Vd to high half of Vm */ + rm = tcg_temp_new_i32(); + neon_load_reg32(rm, a->vm); + tcg_gen_shri_i32(rm, rm, 16); + neon_store_reg32(rm, a->vd); + tcg_temp_free_i32(rm); + return true; +} -- 2.20.1 Implement the VFP fp16 variant of VMOV that transfers a 16-bit value between a general purpose register and a VFP register. Note that Rt == 15 is UNPREDICTABLE; since this insn is v8 and later only we have no need to replicate the old "updates CPSR.NZCV" behaviour that the singleprec version of this insn does. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-22-peter.maydell@linaro.org --- target/arm/vfp.decode | 1 + target/arm/translate-vfp.c.inc | 34 ++++++++++++++++++++++++++++++++++ 2 files changed, 35 insertions(+) diff --git a/target/arm/vfp.decode b/target/arm/vfp.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vfp.decode +++ b/target/arm/vfp.decode @@ -XXX,XX +XXX,XX @@ VDUP ---- 1110 1 b:1 q:1 0 .... rt:4 1011 . 0 e:1 1 0000 \ vn=%vn_dp VMSR_VMRS ---- 1110 111 l:1 reg:4 rt:4 1010 0001 0000 +VMOV_half ---- 1110 000 l:1 .... rt:4 1001 . 001 0000 vn=%vn_sp VMOV_single ---- 1110 000 l:1 .... rt:4 1010 . 001 0000 vn=%vn_sp VMOV_64_sp ---- 1100 010 op:1 rt2:4 rt:4 1010 00.1 .... vm=%vm_sp diff --git a/target/arm/translate-vfp.c.inc b/target/arm/translate-vfp.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-vfp.c.inc +++ b/target/arm/translate-vfp.c.inc @@ -XXX,XX +XXX,XX @@ static bool trans_VMSR_VMRS(DisasContext *s, arg_VMSR_VMRS *a) return true; } +static bool trans_VMOV_half(DisasContext *s, arg_VMOV_single *a) +{ + TCGv_i32 tmp; + + if (!dc_isar_feature(aa32_fp16_arith, s)) { + return false; + } + + if (a->rt == 15) { + /* UNPREDICTABLE; we choose to UNDEF */ + return false; + } + + if (!vfp_access_check(s)) { + return true; + } + + if (a->l) { + /* VFP to general purpose register */ + tmp = tcg_temp_new_i32(); + neon_load_reg32(tmp, a->vn); + tcg_gen_andi_i32(tmp, tmp, 0xffff); + store_reg(s, a->rt, tmp); + } else { + /* general purpose register to VFP */ + tmp = load_reg(s, a->rt); + tcg_gen_andi_i32(tmp, tmp, 0xffff); + neon_store_reg32(tmp, a->vn); + tcg_temp_free_i32(tmp); + } + + return true; +} + static bool trans_VMOV_single(DisasContext *s, arg_VMOV_single *a) { TCGv_i32 tmp; -- 2.20.1 Implement FP16 support for the Neon insns which use the DO_3S_FP_GVEC macro: VADD, VSUB, VABD, VMUL. For VABD this requires us to implement a new gvec_fabd_h helper using the machinery we have already for the other helpers. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-24-peter.maydell@linaro.org --- target/arm/helper.h | 1 + target/arm/vec_helper.c | 6 ++++++ target/arm/translate-neon.c.inc | 36 +++++++++++++++++---------------- 3 files changed, 26 insertions(+), 17 deletions(-) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_5(gvec_fmul_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_fmul_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_fmul_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_fabd_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_fabd_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_ftsmul_h, TCG_CALL_NO_RWG, diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -XXX,XX +XXX,XX @@ static float64 float64_ftsmul(float64 op1, uint64_t op2, float_status *stat) return result; } +static float16 float16_abd(float16 op1, float16 op2, float_status *stat) +{ + return float16_abs(float16_sub(op1, op2, stat)); +} + static float32 float32_abd(float32 op1, float32 op2, float_status *stat) { return float32_abs(float32_sub(op1, op2, stat)); @@ -XXX,XX +XXX,XX @@ DO_3OP(gvec_ftsmul_h, float16_ftsmul, float16) DO_3OP(gvec_ftsmul_s, float32_ftsmul, float32) DO_3OP(gvec_ftsmul_d, float64_ftsmul, float64) +DO_3OP(gvec_fabd_h, float16_abd, float16) DO_3OP(gvec_fabd_s, float32_abd, float32) #ifdef TARGET_AARCH64 diff --git a/target/arm/translate-neon.c.inc b/target/arm/translate-neon.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-neon.c.inc +++ b/target/arm/translate-neon.c.inc @@ -XXX,XX +XXX,XX @@ static bool do_3same_fp(DisasContext *s, arg_3same *a, VFPGen3OpSPFn *fn, return true; } -/* - * For all the functions using this macro, size == 1 means fp16, - * which is an architecture extension we don't implement yet. - */ -#define DO_3S_FP_GVEC(INSN,FUNC) \ - static void gen_##INSN##_3s(unsigned vece, uint32_t rd_ofs, \ - uint32_t rn_ofs, uint32_t rm_ofs, \ - uint32_t oprsz, uint32_t maxsz) \ +#define WRAP_FP_GVEC(WRAPNAME, FPST, FUNC) \ + static void WRAPNAME(unsigned vece, uint32_t rd_ofs, \ + uint32_t rn_ofs, uint32_t rm_ofs, \ + uint32_t oprsz, uint32_t maxsz) \ { \ - TCGv_ptr fpst = fpstatus_ptr(FPST_STD); \ + TCGv_ptr fpst = fpstatus_ptr(FPST); \ tcg_gen_gvec_3_ptr(rd_ofs, rn_ofs, rm_ofs, fpst, \ oprsz, maxsz, 0, FUNC); \ tcg_temp_free_ptr(fpst); \ - } \ + } + +#define DO_3S_FP_GVEC(INSN,SFUNC,HFUNC) \ + WRAP_FP_GVEC(gen_##INSN##_fp32_3s, FPST_STD, SFUNC) \ + WRAP_FP_GVEC(gen_##INSN##_fp16_3s, FPST_STD_F16, HFUNC) \ static bool trans_##INSN##_fp_3s(DisasContext *s, arg_3same *a) \ { \ if (a->size != 0) { \ - /* TODO fp16 support */ \ - return false; \ + if (!dc_isar_feature(aa32_fp16_arith, s)) { \ + return false; \ + } \ + return do_3same(s, a, gen_##INSN##_fp16_3s); \ } \ - return do_3same(s, a, gen_##INSN##_3s); \ + return do_3same(s, a, gen_##INSN##_fp32_3s); \ } -DO_3S_FP_GVEC(VADD, gen_helper_gvec_fadd_s) -DO_3S_FP_GVEC(VSUB, gen_helper_gvec_fsub_s) -DO_3S_FP_GVEC(VABD, gen_helper_gvec_fabd_s) -DO_3S_FP_GVEC(VMUL, gen_helper_gvec_fmul_s) +DO_3S_FP_GVEC(VADD, gen_helper_gvec_fadd_s, gen_helper_gvec_fadd_h) +DO_3S_FP_GVEC(VSUB, gen_helper_gvec_fsub_s, gen_helper_gvec_fsub_h) +DO_3S_FP_GVEC(VABD, gen_helper_gvec_fabd_s, gen_helper_gvec_fabd_h) +DO_3S_FP_GVEC(VMUL, gen_helper_gvec_fmul_s, gen_helper_gvec_fmul_h) /* * For all the functions using this macro, size == 1 means fp16, -- 2.20.1 We already have gvec helpers for floating point VRECPE and VRQSRTE, so convert the Neon decoder to use them and add the fp16 support. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-25-peter.maydell@linaro.org --- target/arm/translate-neon.c.inc | 31 +++++++++++++++++++++++++++++-- 1 file changed, 29 insertions(+), 2 deletions(-) diff --git a/target/arm/translate-neon.c.inc b/target/arm/translate-neon.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-neon.c.inc +++ b/target/arm/translate-neon.c.inc @@ -XXX,XX +XXX,XX @@ static bool do_2misc_fp(DisasContext *s, arg_2misc *a, return do_2misc_fp(s, a, FUNC); \ } -DO_2MISC_FP(VRECPE_F, gen_helper_recpe_f32) -DO_2MISC_FP(VRSQRTE_F, gen_helper_rsqrte_f32) DO_2MISC_FP(VCVT_FS, gen_helper_vfp_sitos) DO_2MISC_FP(VCVT_FU, gen_helper_vfp_uitos) DO_2MISC_FP(VCVT_SF, gen_helper_vfp_tosizs) DO_2MISC_FP(VCVT_UF, gen_helper_vfp_touizs) +#define DO_2MISC_FP_VEC(INSN, HFUNC, SFUNC) \ + static void gen_##INSN(unsigned vece, uint32_t rd_ofs, \ + uint32_t rm_ofs, \ + uint32_t oprsz, uint32_t maxsz) \ + { \ + static gen_helper_gvec_2_ptr * const fns[4] = { \ + NULL, HFUNC, SFUNC, NULL, \ + }; \ + TCGv_ptr fpst; \ + fpst = fpstatus_ptr(vece == MO_16 ? FPST_STD_F16 : FPST_STD); \ + tcg_gen_gvec_2_ptr(rd_ofs, rm_ofs, fpst, oprsz, maxsz, 0, \ + fns[vece]); \ + tcg_temp_free_ptr(fpst); \ + } \ + static bool trans_##INSN(DisasContext *s, arg_2misc *a) \ + { \ + if (a->size == MO_16) { \ + if (!dc_isar_feature(aa32_fp16_arith, s)) { \ + return false; \ + } \ + } else if (a->size != MO_32) { \ + return false; \ + } \ + return do_2misc_vec(s, a, gen_##INSN); \ + } + +DO_2MISC_FP_VEC(VRECPE_F, gen_helper_gvec_frecpe_h, gen_helper_gvec_frecpe_s) +DO_2MISC_FP_VEC(VRSQRTE_F, gen_helper_gvec_frsqrte_h, gen_helper_gvec_frsqrte_s) + static bool trans_VRINTX(DisasContext *s, arg_2misc *a) { if (!arm_dc_feature(s, ARM_FEATURE_V8)) { -- 2.20.1 Rewrite Neon VABS/VNEG of floats to use gvec logical AND and XOR, so that we can implement the fp16 version of the insns. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-26-peter.maydell@linaro.org --- target/arm/translate-neon.c.inc | 34 +++++++++++++++++++++++++++------ 1 file changed, 28 insertions(+), 6 deletions(-) diff --git a/target/arm/translate-neon.c.inc b/target/arm/translate-neon.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-neon.c.inc +++ b/target/arm/translate-neon.c.inc @@ -XXX,XX +XXX,XX @@ static bool trans_VCNT(DisasContext *s, arg_2misc *a) return do_2misc(s, a, gen_helper_neon_cnt_u8); } +static void gen_VABS_F(unsigned vece, uint32_t rd_ofs, uint32_t rm_ofs, + uint32_t oprsz, uint32_t maxsz) +{ + tcg_gen_gvec_andi(vece, rd_ofs, rm_ofs, + vece == MO_16 ? 0x7fff : 0x7fffffff, + oprsz, maxsz); +} + static bool trans_VABS_F(DisasContext *s, arg_2misc *a) { - if (a->size != 2) { + if (a->size == MO_16) { + if (!dc_isar_feature(aa32_fp16_arith, s)) { + return false; + } + } else if (a->size != MO_32) { return false; } - /* TODO: FP16 : size == 1 */ - return do_2misc(s, a, gen_helper_vfp_abss); + return do_2misc_vec(s, a, gen_VABS_F); +} + +static void gen_VNEG_F(unsigned vece, uint32_t rd_ofs, uint32_t rm_ofs, + uint32_t oprsz, uint32_t maxsz) +{ + tcg_gen_gvec_xori(vece, rd_ofs, rm_ofs, + vece == MO_16 ? 0x8000 : 0x80000000, + oprsz, maxsz); } static bool trans_VNEG_F(DisasContext *s, arg_2misc *a) { - if (a->size != 2) { + if (a->size == MO_16) { + if (!dc_isar_feature(aa32_fp16_arith, s)) { + return false; + } + } else if (a->size != MO_32) { return false; } - /* TODO: FP16 : size == 1 */ - return do_2misc(s, a, gen_helper_vfp_negs); + return do_2misc_vec(s, a, gen_VNEG_F); } static bool trans_VRECPE(DisasContext *s, arg_2misc *a) -- 2.20.1 Convert the Neon floating-point vector comparison ops VCEQ, VCGE and VCGT over to using a gvec helper and use this to implement the fp16 case. (We put the float16_ceq() etc functions above the DO_2OP() macro definition because later when we convert the compare-against-zero instructions we'll want their definitions to be visible at that point in the source file.) Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-27-peter.maydell@linaro.org --- target/arm/helper.h | 9 +++++++ target/arm/vec_helper.c | 44 +++++++++++++++++++++++++++++++++ target/arm/translate-neon.c.inc | 6 ++--- 3 files changed, 56 insertions(+), 3 deletions(-) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_5(gvec_fmul_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_fabd_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_fabd_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_fceq_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_fceq_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(gvec_fcge_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_fcge_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(gvec_fcgt_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_fcgt_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_5(gvec_ftsmul_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_ftsmul_s, TCG_CALL_NO_RWG, diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -XXX,XX +XXX,XX @@ void HELPER(gvec_fcmlad)(void *vd, void *vn, void *vm, clear_tail(d, opr_sz, simd_maxsz(desc)); } +/* + * Floating point comparisons producing an integer result (all 1s or all 0s). + * Note that EQ doesn't signal InvalidOp for QNaNs but GE and GT do. + * Softfloat routines return 0/1, which we convert to the 0/-1 Neon requires. + */ +static uint16_t float16_ceq(float16 op1, float16 op2, float_status *stat) +{ + return -float16_eq_quiet(op1, op2, stat); +} + +static uint32_t float32_ceq(float32 op1, float32 op2, float_status *stat) +{ + return -float32_eq_quiet(op1, op2, stat); +} + +static uint16_t float16_cge(float16 op1, float16 op2, float_status *stat) +{ + return -float16_le(op2, op1, stat); +} + +static uint32_t float32_cge(float32 op1, float32 op2, float_status *stat) +{ + return -float32_le(op2, op1, stat); +} + +static uint16_t float16_cgt(float16 op1, float16 op2, float_status *stat) +{ + return -float16_lt(op2, op1, stat); +} + +static uint32_t float32_cgt(float32 op1, float32 op2, float_status *stat) +{ + return -float32_lt(op2, op1, stat); +} + #define DO_2OP(NAME, FUNC, TYPE) \ void HELPER(NAME)(void *vd, void *vn, void *stat, uint32_t desc) \ { \ @@ -XXX,XX +XXX,XX @@ DO_3OP(gvec_ftsmul_d, float64_ftsmul, float64) DO_3OP(gvec_fabd_h, float16_abd, float16) DO_3OP(gvec_fabd_s, float32_abd, float32) +DO_3OP(gvec_fceq_h, float16_ceq, float16) +DO_3OP(gvec_fceq_s, float32_ceq, float32) + +DO_3OP(gvec_fcge_h, float16_cge, float16) +DO_3OP(gvec_fcge_s, float32_cge, float32) + +DO_3OP(gvec_fcgt_h, float16_cgt, float16) +DO_3OP(gvec_fcgt_s, float32_cgt, float32) + #ifdef TARGET_AARCH64 DO_3OP(gvec_recps_h, helper_recpsf_f16, float16) diff --git a/target/arm/translate-neon.c.inc b/target/arm/translate-neon.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-neon.c.inc +++ b/target/arm/translate-neon.c.inc @@ -XXX,XX +XXX,XX @@ DO_3S_FP_GVEC(VADD, gen_helper_gvec_fadd_s, gen_helper_gvec_fadd_h) DO_3S_FP_GVEC(VSUB, gen_helper_gvec_fsub_s, gen_helper_gvec_fsub_h) DO_3S_FP_GVEC(VABD, gen_helper_gvec_fabd_s, gen_helper_gvec_fabd_h) DO_3S_FP_GVEC(VMUL, gen_helper_gvec_fmul_s, gen_helper_gvec_fmul_h) +DO_3S_FP_GVEC(VCEQ, gen_helper_gvec_fceq_s, gen_helper_gvec_fceq_h) +DO_3S_FP_GVEC(VCGE, gen_helper_gvec_fcge_s, gen_helper_gvec_fcge_h) +DO_3S_FP_GVEC(VCGT, gen_helper_gvec_fcgt_s, gen_helper_gvec_fcgt_h) /* * For all the functions using this macro, size == 1 means fp16, @@ -XXX,XX +XXX,XX @@ DO_3S_FP_GVEC(VMUL, gen_helper_gvec_fmul_s, gen_helper_gvec_fmul_h) return do_3same_fp(s, a, FUNC, READS_VD); \ } -DO_3S_FP(VCEQ, gen_helper_neon_ceq_f32, false) -DO_3S_FP(VCGE, gen_helper_neon_cge_f32, false) -DO_3S_FP(VCGT, gen_helper_neon_cgt_f32, false) DO_3S_FP(VACGE, gen_helper_neon_acge_f32, false) DO_3S_FP(VACGT, gen_helper_neon_acgt_f32, false) DO_3S_FP(VMAX, gen_helper_vfp_maxs, false) -- 2.20.1 Convert the neon floating-point vector absolute comparison ops VACGE and VACGT over to using a gvec hepler and use this to implement the fp16 case. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-28-peter.maydell@linaro.org --- target/arm/helper.h | 6 ++++++ target/arm/vec_helper.c | 26 ++++++++++++++++++++++++++ target/arm/translate-neon.c.inc | 4 ++-- 3 files changed, 34 insertions(+), 2 deletions(-) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_5(gvec_fcge_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_fcgt_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_fcgt_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_facge_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_facge_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(gvec_facgt_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_facgt_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_5(gvec_ftsmul_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_ftsmul_s, TCG_CALL_NO_RWG, diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -XXX,XX +XXX,XX @@ static uint32_t float32_cgt(float32 op1, float32 op2, float_status *stat) return -float32_lt(op2, op1, stat); } +static uint16_t float16_acge(float16 op1, float16 op2, float_status *stat) +{ + return -float16_le(float16_abs(op2), float16_abs(op1), stat); +} + +static uint32_t float32_acge(float32 op1, float32 op2, float_status *stat) +{ + return -float32_le(float32_abs(op2), float32_abs(op1), stat); +} + +static uint16_t float16_acgt(float16 op1, float16 op2, float_status *stat) +{ + return -float16_lt(float16_abs(op2), float16_abs(op1), stat); +} + +static uint32_t float32_acgt(float32 op1, float32 op2, float_status *stat) +{ + return -float32_lt(float32_abs(op2), float32_abs(op1), stat); +} + #define DO_2OP(NAME, FUNC, TYPE) \ void HELPER(NAME)(void *vd, void *vn, void *stat, uint32_t desc) \ { \ @@ -XXX,XX +XXX,XX @@ DO_3OP(gvec_fcge_s, float32_cge, float32) DO_3OP(gvec_fcgt_h, float16_cgt, float16) DO_3OP(gvec_fcgt_s, float32_cgt, float32) +DO_3OP(gvec_facge_h, float16_acge, float16) +DO_3OP(gvec_facge_s, float32_acge, float32) + +DO_3OP(gvec_facgt_h, float16_acgt, float16) +DO_3OP(gvec_facgt_s, float32_acgt, float32) + #ifdef TARGET_AARCH64 DO_3OP(gvec_recps_h, helper_recpsf_f16, float16) diff --git a/target/arm/translate-neon.c.inc b/target/arm/translate-neon.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-neon.c.inc +++ b/target/arm/translate-neon.c.inc @@ -XXX,XX +XXX,XX @@ DO_3S_FP_GVEC(VMUL, gen_helper_gvec_fmul_s, gen_helper_gvec_fmul_h) DO_3S_FP_GVEC(VCEQ, gen_helper_gvec_fceq_s, gen_helper_gvec_fceq_h) DO_3S_FP_GVEC(VCGE, gen_helper_gvec_fcge_s, gen_helper_gvec_fcge_h) DO_3S_FP_GVEC(VCGT, gen_helper_gvec_fcgt_s, gen_helper_gvec_fcgt_h) +DO_3S_FP_GVEC(VACGE, gen_helper_gvec_facge_s, gen_helper_gvec_facge_h) +DO_3S_FP_GVEC(VACGT, gen_helper_gvec_facgt_s, gen_helper_gvec_facgt_h) /* * For all the functions using this macro, size == 1 means fp16, @@ -XXX,XX +XXX,XX @@ DO_3S_FP_GVEC(VCGT, gen_helper_gvec_fcgt_s, gen_helper_gvec_fcgt_h) return do_3same_fp(s, a, FUNC, READS_VD); \ } -DO_3S_FP(VACGE, gen_helper_neon_acge_f32, false) -DO_3S_FP(VACGT, gen_helper_neon_acgt_f32, false) DO_3S_FP(VMAX, gen_helper_vfp_maxs, false) DO_3S_FP(VMIN, gen_helper_vfp_mins, false) -- 2.20.1 Convert the Neon float-point VMAX and VMIN insns over to using a gvec helper, and use this to implement the fp16 case. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-29-peter.maydell@linaro.org --- target/arm/helper.h | 6 ++++++ target/arm/vec_helper.c | 6 ++++++ target/arm/translate-neon.c.inc | 5 ++--- 3 files changed, 14 insertions(+), 3 deletions(-) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_5(gvec_facge_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_facgt_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_facgt_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_fmax_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_fmax_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(gvec_fmin_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_fmin_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_5(gvec_ftsmul_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_ftsmul_s, TCG_CALL_NO_RWG, diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -XXX,XX +XXX,XX @@ DO_3OP(gvec_facge_s, float32_acge, float32) DO_3OP(gvec_facgt_h, float16_acgt, float16) DO_3OP(gvec_facgt_s, float32_acgt, float32) +DO_3OP(gvec_fmax_h, float16_max, float16) +DO_3OP(gvec_fmax_s, float32_max, float32) + +DO_3OP(gvec_fmin_h, float16_min, float16) +DO_3OP(gvec_fmin_s, float32_min, float32) + #ifdef TARGET_AARCH64 DO_3OP(gvec_recps_h, helper_recpsf_f16, float16) diff --git a/target/arm/translate-neon.c.inc b/target/arm/translate-neon.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-neon.c.inc +++ b/target/arm/translate-neon.c.inc @@ -XXX,XX +XXX,XX @@ DO_3S_FP_GVEC(VCGE, gen_helper_gvec_fcge_s, gen_helper_gvec_fcge_h) DO_3S_FP_GVEC(VCGT, gen_helper_gvec_fcgt_s, gen_helper_gvec_fcgt_h) DO_3S_FP_GVEC(VACGE, gen_helper_gvec_facge_s, gen_helper_gvec_facge_h) DO_3S_FP_GVEC(VACGT, gen_helper_gvec_facgt_s, gen_helper_gvec_facgt_h) +DO_3S_FP_GVEC(VMAX, gen_helper_gvec_fmax_s, gen_helper_gvec_fmax_h) +DO_3S_FP_GVEC(VMIN, gen_helper_gvec_fmin_s, gen_helper_gvec_fmin_h) /* * For all the functions using this macro, size == 1 means fp16, @@ -XXX,XX +XXX,XX @@ DO_3S_FP_GVEC(VACGT, gen_helper_gvec_facgt_s, gen_helper_gvec_facgt_h) return do_3same_fp(s, a, FUNC, READS_VD); \ } -DO_3S_FP(VMAX, gen_helper_vfp_maxs, false) -DO_3S_FP(VMIN, gen_helper_vfp_mins, false) - static void gen_VMLA_fp_3s(TCGv_i32 vd, TCGv_i32 vn, TCGv_i32 vm, TCGv_ptr fpstatus) { -- 2.20.1 Convert the Neon floating point VMAXNM and VMINNM insns to using a gvec helper and use this to implement the fp16 case. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-30-peter.maydell@linaro.org --- target/arm/helper.h | 6 ++++++ target/arm/vec_helper.c | 6 ++++++ target/arm/translate-neon.c.inc | 23 +++++++++++++++-------- 3 files changed, 27 insertions(+), 8 deletions(-) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_5(gvec_fmax_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_fmin_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_fmin_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_fmaxnum_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_fmaxnum_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(gvec_fminnum_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_fminnum_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_5(gvec_ftsmul_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_ftsmul_s, TCG_CALL_NO_RWG, diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -XXX,XX +XXX,XX @@ DO_3OP(gvec_fmax_s, float32_max, float32) DO_3OP(gvec_fmin_h, float16_min, float16) DO_3OP(gvec_fmin_s, float32_min, float32) +DO_3OP(gvec_fmaxnum_h, float16_maxnum, float16) +DO_3OP(gvec_fmaxnum_s, float32_maxnum, float32) + +DO_3OP(gvec_fminnum_h, float16_minnum, float16) +DO_3OP(gvec_fminnum_s, float32_minnum, float32) + #ifdef TARGET_AARCH64 DO_3OP(gvec_recps_h, helper_recpsf_f16, float16) diff --git a/target/arm/translate-neon.c.inc b/target/arm/translate-neon.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-neon.c.inc +++ b/target/arm/translate-neon.c.inc @@ -XXX,XX +XXX,XX @@ static void gen_VMLS_fp_3s(TCGv_i32 vd, TCGv_i32 vn, TCGv_i32 vm, DO_3S_FP(VMLA, gen_VMLA_fp_3s, true) DO_3S_FP(VMLS, gen_VMLS_fp_3s, true) +WRAP_FP_GVEC(gen_VMAXNM_fp32_3s, FPST_STD, gen_helper_gvec_fmaxnum_s) +WRAP_FP_GVEC(gen_VMAXNM_fp16_3s, FPST_STD_F16, gen_helper_gvec_fmaxnum_h) +WRAP_FP_GVEC(gen_VMINNM_fp32_3s, FPST_STD, gen_helper_gvec_fminnum_s) +WRAP_FP_GVEC(gen_VMINNM_fp16_3s, FPST_STD_F16, gen_helper_gvec_fminnum_h) + static bool trans_VMAXNM_fp_3s(DisasContext *s, arg_3same *a) { if (!arm_dc_feature(s, ARM_FEATURE_V8)) { @@ -XXX,XX +XXX,XX @@ static bool trans_VMAXNM_fp_3s(DisasContext *s, arg_3same *a) } if (a->size != 0) { - /* TODO fp16 support */ - return false; + if (!dc_isar_feature(aa32_fp16_arith, s)) { + return false; + } + return do_3same(s, a, gen_VMAXNM_fp16_3s); } - - return do_3same_fp(s, a, gen_helper_vfp_maxnums, false); + return do_3same(s, a, gen_VMAXNM_fp32_3s); } static bool trans_VMINNM_fp_3s(DisasContext *s, arg_3same *a) @@ -XXX,XX +XXX,XX @@ static bool trans_VMINNM_fp_3s(DisasContext *s, arg_3same *a) } if (a->size != 0) { - /* TODO fp16 support */ - return false; + if (!dc_isar_feature(aa32_fp16_arith, s)) { + return false; + } + return do_3same(s, a, gen_VMINNM_fp16_3s); } - - return do_3same_fp(s, a, gen_helper_vfp_minnums, false); + return do_3same(s, a, gen_VMINNM_fp32_3s); } WRAP_ENV_FN(gen_VRECPS_tramp, gen_helper_recps_f32) -- 2.20.1 Convert the Neon floating-point VMLA and VMLS insns over to using a gvec helper, and use this to implement the fp16 case. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-31-peter.maydell@linaro.org --- target/arm/helper.h | 6 +++++ target/arm/vec_helper.c | 42 +++++++++++++++++++++++++++++++++ target/arm/translate-neon.c.inc | 33 ++------------------------ 3 files changed, 50 insertions(+), 31 deletions(-) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_5(gvec_fmaxnum_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i3 DEF_HELPER_FLAGS_5(gvec_fminnum_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_fminnum_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_fmla_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_fmla_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(gvec_fmls_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_fmls_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_5(gvec_ftsmul_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_ftsmul_s, TCG_CALL_NO_RWG, diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -XXX,XX +XXX,XX @@ DO_3OP(gvec_rsqrts_d, helper_rsqrtsf_f64, float64) #endif #undef DO_3OP +/* Non-fused multiply-add (unlike float16_muladd etc, which are fused) */ +static float16 float16_muladd_nf(float16 dest, float16 op1, float16 op2, + float_status *stat) +{ + return float16_add(dest, float16_mul(op1, op2, stat), stat); +} + +static float32 float32_muladd_nf(float32 dest, float32 op1, float32 op2, + float_status *stat) +{ + return float32_add(dest, float32_mul(op1, op2, stat), stat); +} + +static float16 float16_mulsub_nf(float16 dest, float16 op1, float16 op2, + float_status *stat) +{ + return float16_sub(dest, float16_mul(op1, op2, stat), stat); +} + +static float32 float32_mulsub_nf(float32 dest, float32 op1, float32 op2, + float_status *stat) +{ + return float32_sub(dest, float32_mul(op1, op2, stat), stat); +} + +#define DO_MULADD(NAME, FUNC, TYPE) \ +void HELPER(NAME)(void *vd, void *vn, void *vm, void *stat, uint32_t desc) \ +{ \ + intptr_t i, oprsz = simd_oprsz(desc); \ + TYPE *d = vd, *n = vn, *m = vm; \ + for (i = 0; i < oprsz / sizeof(TYPE); i++) { \ + d[i] = FUNC(d[i], n[i], m[i], stat); \ + } \ + clear_tail(d, oprsz, simd_maxsz(desc)); \ +} + +DO_MULADD(gvec_fmla_h, float16_muladd_nf, float16) +DO_MULADD(gvec_fmla_s, float32_muladd_nf, float32) + +DO_MULADD(gvec_fmls_h, float16_mulsub_nf, float16) +DO_MULADD(gvec_fmls_s, float32_mulsub_nf, float32) + /* For the indexed ops, SVE applies the index per 128-bit vector segment. * For AdvSIMD, there is of course only one such vector segment. */ diff --git a/target/arm/translate-neon.c.inc b/target/arm/translate-neon.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-neon.c.inc +++ b/target/arm/translate-neon.c.inc @@ -XXX,XX +XXX,XX @@ DO_3S_FP_GVEC(VACGE, gen_helper_gvec_facge_s, gen_helper_gvec_facge_h) DO_3S_FP_GVEC(VACGT, gen_helper_gvec_facgt_s, gen_helper_gvec_facgt_h) DO_3S_FP_GVEC(VMAX, gen_helper_gvec_fmax_s, gen_helper_gvec_fmax_h) DO_3S_FP_GVEC(VMIN, gen_helper_gvec_fmin_s, gen_helper_gvec_fmin_h) - -/* - * For all the functions using this macro, size == 1 means fp16, - * which is an architecture extension we don't implement yet. - */ -#define DO_3S_FP(INSN,FUNC,READS_VD) \ - static bool trans_##INSN##_fp_3s(DisasContext *s, arg_3same *a) \ - { \ - if (a->size != 0) { \ - /* TODO fp16 support */ \ - return false; \ - } \ - return do_3same_fp(s, a, FUNC, READS_VD); \ - } - -static void gen_VMLA_fp_3s(TCGv_i32 vd, TCGv_i32 vn, TCGv_i32 vm, - TCGv_ptr fpstatus) -{ - gen_helper_vfp_muls(vn, vn, vm, fpstatus); - gen_helper_vfp_adds(vd, vd, vn, fpstatus); -} - -static void gen_VMLS_fp_3s(TCGv_i32 vd, TCGv_i32 vn, TCGv_i32 vm, - TCGv_ptr fpstatus) -{ - gen_helper_vfp_muls(vn, vn, vm, fpstatus); - gen_helper_vfp_subs(vd, vd, vn, fpstatus); -} - -DO_3S_FP(VMLA, gen_VMLA_fp_3s, true) -DO_3S_FP(VMLS, gen_VMLS_fp_3s, true) +DO_3S_FP_GVEC(VMLA, gen_helper_gvec_fmla_s, gen_helper_gvec_fmla_h) +DO_3S_FP_GVEC(VMLS, gen_helper_gvec_fmls_s, gen_helper_gvec_fmls_h) WRAP_FP_GVEC(gen_VMAXNM_fp32_3s, FPST_STD, gen_helper_gvec_fmaxnum_s) WRAP_FP_GVEC(gen_VMAXNM_fp16_3s, FPST_STD_F16, gen_helper_gvec_fmaxnum_h) -- 2.20.1 Convert the neon floating-point vector operations VFMA and VFMS to use a gvec helper, and use this to implement the fp16 case. This is the last use of do_3same_fp() so we can now delete that function. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-32-peter.maydell@linaro.org --- target/arm/helper.h | 6 +++ target/arm/vec_helper.c | 33 +++++++++++- target/arm/translate-neon.c.inc | 92 +-------------------------------- 3 files changed, 40 insertions(+), 91 deletions(-) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_5(gvec_fmla_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_fmls_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_fmls_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_vfma_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_vfma_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(gvec_vfms_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_vfms_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_5(gvec_ftsmul_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_ftsmul_s, TCG_CALL_NO_RWG, diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -XXX,XX +XXX,XX @@ static float32 float32_mulsub_nf(float32 dest, float32 op1, float32 op2, return float32_sub(dest, float32_mul(op1, op2, stat), stat); } -#define DO_MULADD(NAME, FUNC, TYPE) \ +/* Fused versions; these have the semantics Neon VFMA/VFMS want */ +static float16 float16_muladd_f(float16 dest, float16 op1, float16 op2, + float_status *stat) +{ + return float16_muladd(op1, op2, dest, 0, stat); +} + +static float32 float32_muladd_f(float32 dest, float32 op1, float32 op2, + float_status *stat) +{ + return float32_muladd(op1, op2, dest, 0, stat); +} + +static float16 float16_mulsub_f(float16 dest, float16 op1, float16 op2, + float_status *stat) +{ + return float16_muladd(float16_chs(op1), op2, dest, 0, stat); +} + +static float32 float32_mulsub_f(float32 dest, float32 op1, float32 op2, + float_status *stat) +{ + return float32_muladd(float32_chs(op1), op2, dest, 0, stat); +} + +#define DO_MULADD(NAME, FUNC, TYPE) \ void HELPER(NAME)(void *vd, void *vn, void *vm, void *stat, uint32_t desc) \ { \ intptr_t i, oprsz = simd_oprsz(desc); \ @@ -XXX,XX +XXX,XX @@ DO_MULADD(gvec_fmla_s, float32_muladd_nf, float32) DO_MULADD(gvec_fmls_h, float16_mulsub_nf, float16) DO_MULADD(gvec_fmls_s, float32_mulsub_nf, float32) +DO_MULADD(gvec_vfma_h, float16_muladd_f, float16) +DO_MULADD(gvec_vfma_s, float32_muladd_f, float32) + +DO_MULADD(gvec_vfms_h, float16_mulsub_f, float16) +DO_MULADD(gvec_vfms_s, float32_mulsub_f, float32) + /* For the indexed ops, SVE applies the index per 128-bit vector segment. * For AdvSIMD, there is of course only one such vector segment. */ diff --git a/target/arm/translate-neon.c.inc b/target/arm/translate-neon.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-neon.c.inc +++ b/target/arm/translate-neon.c.inc @@ -XXX,XX +XXX,XX @@ DO_3SAME_PAIR(VPADD, padd_u) DO_3SAME_VQDMULH(VQDMULH, qdmulh) DO_3SAME_VQDMULH(VQRDMULH, qrdmulh) -static bool do_3same_fp(DisasContext *s, arg_3same *a, VFPGen3OpSPFn *fn, - bool reads_vd) -{ - /* - * FP operations handled elementwise 32 bits at a time. - * If reads_vd is true then the old value of Vd will be - * loaded before calling the callback function. This is - * used for multiply-accumulate type operations. - */ - TCGv_i32 tmp, tmp2; - int pass; - - if (!arm_dc_feature(s, ARM_FEATURE_NEON)) { - return false; - } - - /* UNDEF accesses to D16-D31 if they don't exist. */ - if (!dc_isar_feature(aa32_simd_r32, s) && - ((a->vd | a->vn | a->vm) & 0x10)) { - return false; - } - - if ((a->vn | a->vm | a->vd) & a->q) { - return false; - } - - if (!vfp_access_check(s)) { - return true; - } - - TCGv_ptr fpstatus = fpstatus_ptr(FPST_STD); - for (pass = 0; pass < (a->q ? 4 : 2); pass++) { - tmp = neon_load_reg(a->vn, pass); - tmp2 = neon_load_reg(a->vm, pass); - if (reads_vd) { - TCGv_i32 tmp_rd = neon_load_reg(a->vd, pass); - fn(tmp_rd, tmp, tmp2, fpstatus); - neon_store_reg(a->vd, pass, tmp_rd); - tcg_temp_free_i32(tmp); - } else { - fn(tmp, tmp, tmp2, fpstatus); - neon_store_reg(a->vd, pass, tmp); - } - tcg_temp_free_i32(tmp2); - } - tcg_temp_free_ptr(fpstatus); - return true; -} - #define WRAP_FP_GVEC(WRAPNAME, FPST, FUNC) \ static void WRAPNAME(unsigned vece, uint32_t rd_ofs, \ uint32_t rn_ofs, uint32_t rm_ofs, \ @@ -XXX,XX +XXX,XX @@ DO_3S_FP_GVEC(VMAX, gen_helper_gvec_fmax_s, gen_helper_gvec_fmax_h) DO_3S_FP_GVEC(VMIN, gen_helper_gvec_fmin_s, gen_helper_gvec_fmin_h) DO_3S_FP_GVEC(VMLA, gen_helper_gvec_fmla_s, gen_helper_gvec_fmla_h) DO_3S_FP_GVEC(VMLS, gen_helper_gvec_fmls_s, gen_helper_gvec_fmls_h) +DO_3S_FP_GVEC(VFMA, gen_helper_gvec_vfma_s, gen_helper_gvec_vfma_h) +DO_3S_FP_GVEC(VFMS, gen_helper_gvec_vfms_s, gen_helper_gvec_vfms_h) WRAP_FP_GVEC(gen_VMAXNM_fp32_3s, FPST_STD, gen_helper_gvec_fmaxnum_s) WRAP_FP_GVEC(gen_VMAXNM_fp16_3s, FPST_STD_F16, gen_helper_gvec_fmaxnum_h) @@ -XXX,XX +XXX,XX @@ static bool trans_VRSQRTS_fp_3s(DisasContext *s, arg_3same *a) return do_3same(s, a, gen_VRSQRTS_fp_3s); } -static void gen_VFMA_fp_3s(TCGv_i32 vd, TCGv_i32 vn, TCGv_i32 vm, - TCGv_ptr fpstatus) -{ - gen_helper_vfp_muladds(vd, vn, vm, vd, fpstatus); -} - -static bool trans_VFMA_fp_3s(DisasContext *s, arg_3same *a) -{ - if (!dc_isar_feature(aa32_simdfmac, s)) { - return false; - } - - if (a->size != 0) { - /* TODO fp16 support */ - return false; - } - - return do_3same_fp(s, a, gen_VFMA_fp_3s, true); -} - -static void gen_VFMS_fp_3s(TCGv_i32 vd, TCGv_i32 vn, TCGv_i32 vm, - TCGv_ptr fpstatus) -{ - gen_helper_vfp_negs(vn, vn); - gen_helper_vfp_muladds(vd, vn, vm, vd, fpstatus); -} - -static bool trans_VFMS_fp_3s(DisasContext *s, arg_3same *a) -{ - if (!dc_isar_feature(aa32_simdfmac, s)) { - return false; - } - - if (a->size != 0) { - /* TODO fp16 support */ - return false; - } - - return do_3same_fp(s, a, gen_VFMS_fp_3s, true); -} - static bool do_3same_fp_pair(DisasContext *s, arg_3same *a, VFPGen3OpSPFn *fn) { /* FP operations handled pairwise 32 bits at a time */ -- 2.20.1 Convert the neon floating-point vector compare-vs-0 insns VCEQ0, VCGT0, VCLE0, VCGE0 and VCLT0 to use a gvec helper, and use this to implement the fp16 case. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-33-peter.maydell@linaro.org --- target/arm/helper.h | 15 +++++++++++++++ target/arm/vec_helper.c | 25 +++++++++++++++++++++++++ target/arm/translate-neon.c.inc | 33 +++++---------------------------- 3 files changed, 45 insertions(+), 28 deletions(-) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_4(gvec_frsqrte_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_frsqrte_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_frsqrte_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_fcgt0_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_fcgt0_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(gvec_fcge0_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_fcge0_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(gvec_fceq0_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_fceq0_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(gvec_fcle0_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_fcle0_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(gvec_fclt0_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_fclt0_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_5(gvec_fadd_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_fadd_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_fadd_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -XXX,XX +XXX,XX @@ DO_2OP(gvec_frsqrte_h, helper_rsqrte_f16, float16) DO_2OP(gvec_frsqrte_s, helper_rsqrte_f32, float32) DO_2OP(gvec_frsqrte_d, helper_rsqrte_f64, float64) +#define WRAP_CMP0_FWD(FN, CMPOP, TYPE) \ + static TYPE TYPE##_##FN##0(TYPE op, float_status *stat) \ + { \ + return TYPE##_##CMPOP(op, TYPE##_zero, stat); \ + } + +#define WRAP_CMP0_REV(FN, CMPOP, TYPE) \ + static TYPE TYPE##_##FN##0(TYPE op, float_status *stat) \ + { \ + return TYPE##_##CMPOP(TYPE##_zero, op, stat); \ + } + +#define DO_2OP_CMP0(FN, CMPOP, DIRN) \ + WRAP_CMP0_##DIRN(FN, CMPOP, float16) \ + WRAP_CMP0_##DIRN(FN, CMPOP, float32) \ + DO_2OP(gvec_f##FN##0_h, float16_##FN##0, float16) \ + DO_2OP(gvec_f##FN##0_s, float32_##FN##0, float32) + +DO_2OP_CMP0(cgt, cgt, FWD) +DO_2OP_CMP0(cge, cge, FWD) +DO_2OP_CMP0(ceq, ceq, FWD) +DO_2OP_CMP0(clt, cgt, REV) +DO_2OP_CMP0(cle, cge, REV) + #undef DO_2OP +#undef DO_2OP_CMP0 /* Floating-point trigonometric starting value. * See the ARM ARM pseudocode function FPTrigSMul. diff --git a/target/arm/translate-neon.c.inc b/target/arm/translate-neon.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-neon.c.inc +++ b/target/arm/translate-neon.c.inc @@ -XXX,XX +XXX,XX @@ DO_2MISC_FP(VCVT_UF, gen_helper_vfp_touizs) DO_2MISC_FP_VEC(VRECPE_F, gen_helper_gvec_frecpe_h, gen_helper_gvec_frecpe_s) DO_2MISC_FP_VEC(VRSQRTE_F, gen_helper_gvec_frsqrte_h, gen_helper_gvec_frsqrte_s) +DO_2MISC_FP_VEC(VCGT0_F, gen_helper_gvec_fcgt0_h, gen_helper_gvec_fcgt0_s) +DO_2MISC_FP_VEC(VCGE0_F, gen_helper_gvec_fcge0_h, gen_helper_gvec_fcge0_s) +DO_2MISC_FP_VEC(VCEQ0_F, gen_helper_gvec_fceq0_h, gen_helper_gvec_fceq0_s) +DO_2MISC_FP_VEC(VCLT0_F, gen_helper_gvec_fclt0_h, gen_helper_gvec_fclt0_s) +DO_2MISC_FP_VEC(VCLE0_F, gen_helper_gvec_fcle0_h, gen_helper_gvec_fcle0_s) static bool trans_VRINTX(DisasContext *s, arg_2misc *a) { @@ -XXX,XX +XXX,XX @@ static bool trans_VRINTX(DisasContext *s, arg_2misc *a) return do_2misc_fp(s, a, gen_helper_rints_exact); } -#define WRAP_FP_CMP0_FWD(WRAPNAME, FUNC) \ - static void WRAPNAME(TCGv_i32 d, TCGv_i32 m, TCGv_ptr fpst) \ - { \ - TCGv_i32 zero = tcg_const_i32(0); \ - FUNC(d, m, zero, fpst); \ - tcg_temp_free_i32(zero); \ - } -#define WRAP_FP_CMP0_REV(WRAPNAME, FUNC) \ - static void WRAPNAME(TCGv_i32 d, TCGv_i32 m, TCGv_ptr fpst) \ - { \ - TCGv_i32 zero = tcg_const_i32(0); \ - FUNC(d, zero, m, fpst); \ - tcg_temp_free_i32(zero); \ - } - -#define DO_FP_CMP0(INSN, FUNC, REV) \ - WRAP_FP_CMP0_##REV(gen_##INSN, FUNC) \ - static bool trans_##INSN(DisasContext *s, arg_2misc *a) \ - { \ - return do_2misc_fp(s, a, gen_##INSN); \ - } - -DO_FP_CMP0(VCGT0_F, gen_helper_neon_cgt_f32, FWD) -DO_FP_CMP0(VCGE0_F, gen_helper_neon_cge_f32, FWD) -DO_FP_CMP0(VCEQ0_F, gen_helper_neon_ceq_f32, FWD) -DO_FP_CMP0(VCLE0_F, gen_helper_neon_cge_f32, REV) -DO_FP_CMP0(VCLT0_F, gen_helper_neon_cgt_f32, REV) - static bool do_vrint(DisasContext *s, arg_2misc *a, int rmode) { /* -- 2.20.1 Convert the Neon VRECPS insn to using a gvec helper, and use this to implement the fp16 case. The phrasing of the new float32_recps_nf() is slightly different from the old recps_f32() so that it parallels the f16 version; for f16 we can't assume that flush-to-zero is always enabled. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-34-peter.maydell@linaro.org --- target/arm/helper.h | 4 +++- target/arm/vec_helper.c | 31 +++++++++++++++++++++++++++++++ target/arm/vfp_helper.c | 13 ------------- target/arm/translate-neon.c.inc | 21 +-------------------- 4 files changed, 35 insertions(+), 34 deletions(-) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_4(vfp_muladdd, f64, f64, f64, f64, ptr) DEF_HELPER_4(vfp_muladds, f32, f32, f32, f32, ptr) DEF_HELPER_4(vfp_muladdh, f16, f16, f16, f16, ptr) -DEF_HELPER_3(recps_f32, f32, env, f32, f32) DEF_HELPER_3(rsqrts_f32, f32, env, f32, f32) DEF_HELPER_FLAGS_2(recpe_f16, TCG_CALL_NO_RWG, f16, f16, ptr) DEF_HELPER_FLAGS_2(recpe_f32, TCG_CALL_NO_RWG, f32, f32, ptr) @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_5(gvec_fmaxnum_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i3 DEF_HELPER_FLAGS_5(gvec_fminnum_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_fminnum_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_recps_nf_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_recps_nf_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_5(gvec_fmla_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_fmla_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -XXX,XX +XXX,XX @@ static float32 float32_abd(float32 op1, float32 op2, float_status *stat) return float32_abs(float32_sub(op1, op2, stat)); } +/* + * Reciprocal step. These are the AArch32 version which uses a + * non-fused multiply-and-subtract. + */ +static float16 float16_recps_nf(float16 op1, float16 op2, float_status *stat) +{ + op1 = float16_squash_input_denormal(op1, stat); + op2 = float16_squash_input_denormal(op2, stat); + + if ((float16_is_infinity(op1) && float16_is_zero(op2)) || + (float16_is_infinity(op2) && float16_is_zero(op1))) { + return float16_two; + } + return float16_sub(float16_two, float16_mul(op1, op2, stat), stat); +} + +static float32 float32_recps_nf(float32 op1, float32 op2, float_status *stat) +{ + op1 = float32_squash_input_denormal(op1, stat); + op2 = float32_squash_input_denormal(op2, stat); + + if ((float32_is_infinity(op1) && float32_is_zero(op2)) || + (float32_is_infinity(op2) && float32_is_zero(op1))) { + return float32_two; + } + return float32_sub(float32_two, float32_mul(op1, op2, stat), stat); +} + #define DO_3OP(NAME, FUNC, TYPE) \ void HELPER(NAME)(void *vd, void *vn, void *vm, void *stat, uint32_t desc) \ { \ @@ -XXX,XX +XXX,XX @@ DO_3OP(gvec_fmaxnum_s, float32_maxnum, float32) DO_3OP(gvec_fminnum_h, float16_minnum, float16) DO_3OP(gvec_fminnum_s, float32_minnum, float32) +DO_3OP(gvec_recps_nf_h, float16_recps_nf, float16) +DO_3OP(gvec_recps_nf_s, float32_recps_nf, float32) + #ifdef TARGET_AARCH64 DO_3OP(gvec_recps_h, helper_recpsf_f16, float16) diff --git a/target/arm/vfp_helper.c b/target/arm/vfp_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vfp_helper.c +++ b/target/arm/vfp_helper.c @@ -XXX,XX +XXX,XX @@ uint32_t HELPER(vfp_fcvt_f64_to_f16)(float64 a, void *fpstp, uint32_t ahp_mode) return r; } -float32 HELPER(recps_f32)(CPUARMState *env, float32 a, float32 b) -{ - float_status *s = &env->vfp.standard_fp_status; - if ((float32_is_infinity(a) && float32_is_zero_or_denormal(b)) || - (float32_is_infinity(b) && float32_is_zero_or_denormal(a))) { - if (!(float32_is_zero(a) || float32_is_zero(b))) { - float_raise(float_flag_input_denormal, s); - } - return float32_two; - } - return float32_sub(float32_two, float32_mul(a, b, s), s); -} - float32 HELPER(rsqrts_f32)(CPUARMState *env, float32 a, float32 b) { float_status *s = &env->vfp.standard_fp_status; diff --git a/target/arm/translate-neon.c.inc b/target/arm/translate-neon.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-neon.c.inc +++ b/target/arm/translate-neon.c.inc @@ -XXX,XX +XXX,XX @@ DO_3S_FP_GVEC(VMLA, gen_helper_gvec_fmla_s, gen_helper_gvec_fmla_h) DO_3S_FP_GVEC(VMLS, gen_helper_gvec_fmls_s, gen_helper_gvec_fmls_h) DO_3S_FP_GVEC(VFMA, gen_helper_gvec_vfma_s, gen_helper_gvec_vfma_h) DO_3S_FP_GVEC(VFMS, gen_helper_gvec_vfms_s, gen_helper_gvec_vfms_h) +DO_3S_FP_GVEC(VRECPS, gen_helper_gvec_recps_nf_s, gen_helper_gvec_recps_nf_h) WRAP_FP_GVEC(gen_VMAXNM_fp32_3s, FPST_STD, gen_helper_gvec_fmaxnum_s) WRAP_FP_GVEC(gen_VMAXNM_fp16_3s, FPST_STD_F16, gen_helper_gvec_fmaxnum_h) @@ -XXX,XX +XXX,XX @@ static bool trans_VMINNM_fp_3s(DisasContext *s, arg_3same *a) return do_3same(s, a, gen_VMINNM_fp32_3s); } -WRAP_ENV_FN(gen_VRECPS_tramp, gen_helper_recps_f32) - -static void gen_VRECPS_fp_3s(unsigned vece, uint32_t rd_ofs, - uint32_t rn_ofs, uint32_t rm_ofs, - uint32_t oprsz, uint32_t maxsz) -{ - static const GVecGen3 ops = { .fni4 = gen_VRECPS_tramp }; - tcg_gen_gvec_3(rd_ofs, rn_ofs, rm_ofs, oprsz, maxsz, &ops); -} - -static bool trans_VRECPS_fp_3s(DisasContext *s, arg_3same *a) -{ - if (a->size != 0) { - /* TODO fp16 support */ - return false; - } - - return do_3same(s, a, gen_VRECPS_fp_3s); -} - WRAP_ENV_FN(gen_VRSQRTS_tramp, gen_helper_rsqrts_f32) static void gen_VRSQRTS_fp_3s(unsigned vece, uint32_t rd_ofs, -- 2.20.1 Convert the Neon VRSQRTS insn to using a gvec helper, and use this to implement the fp16 case. As with VRECPS, we adjust the phrasing of the new implementation slightly so that the fp32 version parallels the fp16 one. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-35-peter.maydell@linaro.org --- target/arm/helper.h | 4 +++- target/arm/vec_helper.c | 30 ++++++++++++++++++++++++++++++ target/arm/vfp_helper.c | 15 --------------- target/arm/translate-neon.c.inc | 21 +-------------------- 4 files changed, 34 insertions(+), 36 deletions(-) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_4(vfp_muladdd, f64, f64, f64, f64, ptr) DEF_HELPER_4(vfp_muladds, f32, f32, f32, f32, ptr) DEF_HELPER_4(vfp_muladdh, f16, f16, f16, f16, ptr) -DEF_HELPER_3(rsqrts_f32, f32, env, f32, f32) DEF_HELPER_FLAGS_2(recpe_f16, TCG_CALL_NO_RWG, f16, f16, ptr) DEF_HELPER_FLAGS_2(recpe_f32, TCG_CALL_NO_RWG, f32, f32, ptr) DEF_HELPER_FLAGS_2(recpe_f64, TCG_CALL_NO_RWG, f64, f64, ptr) @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_5(gvec_fminnum_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i3 DEF_HELPER_FLAGS_5(gvec_recps_nf_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_recps_nf_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_rsqrts_nf_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_rsqrts_nf_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_5(gvec_fmla_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_fmla_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -XXX,XX +XXX,XX @@ static float32 float32_recps_nf(float32 op1, float32 op2, float_status *stat) return float32_sub(float32_two, float32_mul(op1, op2, stat), stat); } +/* Reciprocal square-root step. AArch32 non-fused semantics. */ +static float16 float16_rsqrts_nf(float16 op1, float16 op2, float_status *stat) +{ + op1 = float16_squash_input_denormal(op1, stat); + op2 = float16_squash_input_denormal(op2, stat); + + if ((float16_is_infinity(op1) && float16_is_zero(op2)) || + (float16_is_infinity(op2) && float16_is_zero(op1))) { + return float16_one_point_five; + } + op1 = float16_sub(float16_three, float16_mul(op1, op2, stat), stat); + return float16_div(op1, float16_two, stat); +} + +static float32 float32_rsqrts_nf(float32 op1, float32 op2, float_status *stat) +{ + op1 = float32_squash_input_denormal(op1, stat); + op2 = float32_squash_input_denormal(op2, stat); + + if ((float32_is_infinity(op1) && float32_is_zero(op2)) || + (float32_is_infinity(op2) && float32_is_zero(op1))) { + return float32_one_point_five; + } + op1 = float32_sub(float32_three, float32_mul(op1, op2, stat), stat); + return float32_div(op1, float32_two, stat); +} + #define DO_3OP(NAME, FUNC, TYPE) \ void HELPER(NAME)(void *vd, void *vn, void *vm, void *stat, uint32_t desc) \ { \ @@ -XXX,XX +XXX,XX @@ DO_3OP(gvec_fminnum_s, float32_minnum, float32) DO_3OP(gvec_recps_nf_h, float16_recps_nf, float16) DO_3OP(gvec_recps_nf_s, float32_recps_nf, float32) +DO_3OP(gvec_rsqrts_nf_h, float16_rsqrts_nf, float16) +DO_3OP(gvec_rsqrts_nf_s, float32_rsqrts_nf, float32) + #ifdef TARGET_AARCH64 DO_3OP(gvec_recps_h, helper_recpsf_f16, float16) diff --git a/target/arm/vfp_helper.c b/target/arm/vfp_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vfp_helper.c +++ b/target/arm/vfp_helper.c @@ -XXX,XX +XXX,XX @@ uint32_t HELPER(vfp_fcvt_f64_to_f16)(float64 a, void *fpstp, uint32_t ahp_mode) return r; } -float32 HELPER(rsqrts_f32)(CPUARMState *env, float32 a, float32 b) -{ - float_status *s = &env->vfp.standard_fp_status; - float32 product; - if ((float32_is_infinity(a) && float32_is_zero_or_denormal(b)) || - (float32_is_infinity(b) && float32_is_zero_or_denormal(a))) { - if (!(float32_is_zero(a) || float32_is_zero(b))) { - float_raise(float_flag_input_denormal, s); - } - return float32_one_point_five; - } - product = float32_mul(a, b, s); - return float32_div(float32_sub(float32_three, product, s), float32_two, s); -} - /* NEON helpers. */ /* Constants 256 and 512 are used in some helpers; we avoid relying on diff --git a/target/arm/translate-neon.c.inc b/target/arm/translate-neon.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-neon.c.inc +++ b/target/arm/translate-neon.c.inc @@ -XXX,XX +XXX,XX @@ DO_3S_FP_GVEC(VMLS, gen_helper_gvec_fmls_s, gen_helper_gvec_fmls_h) DO_3S_FP_GVEC(VFMA, gen_helper_gvec_vfma_s, gen_helper_gvec_vfma_h) DO_3S_FP_GVEC(VFMS, gen_helper_gvec_vfms_s, gen_helper_gvec_vfms_h) DO_3S_FP_GVEC(VRECPS, gen_helper_gvec_recps_nf_s, gen_helper_gvec_recps_nf_h) +DO_3S_FP_GVEC(VRSQRTS, gen_helper_gvec_rsqrts_nf_s, gen_helper_gvec_rsqrts_nf_h) WRAP_FP_GVEC(gen_VMAXNM_fp32_3s, FPST_STD, gen_helper_gvec_fmaxnum_s) WRAP_FP_GVEC(gen_VMAXNM_fp16_3s, FPST_STD_F16, gen_helper_gvec_fmaxnum_h) @@ -XXX,XX +XXX,XX @@ static bool trans_VMINNM_fp_3s(DisasContext *s, arg_3same *a) return do_3same(s, a, gen_VMINNM_fp32_3s); } -WRAP_ENV_FN(gen_VRSQRTS_tramp, gen_helper_rsqrts_f32) - -static void gen_VRSQRTS_fp_3s(unsigned vece, uint32_t rd_ofs, - uint32_t rn_ofs, uint32_t rm_ofs, - uint32_t oprsz, uint32_t maxsz) -{ - static const GVecGen3 ops = { .fni4 = gen_VRSQRTS_tramp }; - tcg_gen_gvec_3(rd_ofs, rn_ofs, rm_ofs, oprsz, maxsz, &ops); -} - -static bool trans_VRSQRTS_fp_3s(DisasContext *s, arg_3same *a) -{ - if (a->size != 0) { - /* TODO fp16 support */ - return false; - } - - return do_3same(s, a, gen_VRSQRTS_fp_3s); -} - static bool do_3same_fp_pair(DisasContext *s, arg_3same *a, VFPGen3OpSPFn *fn) { /* FP operations handled pairwise 32 bits at a time */ -- 2.20.1 Convert the Neon pairwise fp ops to use a single gvic-style helper to do the full operation instead of one helper call for each 32-bit part. This allows us to use the same framework to implement the fp16. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-36-peter.maydell@linaro.org --- target/arm/helper.h | 7 +++++ target/arm/vec_helper.c | 45 +++++++++++++++++++++++++++++++++ target/arm/translate-neon.c.inc | 42 ++++++++++++------------------ 3 files changed, 68 insertions(+), 26 deletions(-) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_5(gvec_fcmlas_idx, TCG_CALL_NO_RWG, DEF_HELPER_FLAGS_5(gvec_fcmlad, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(neon_paddh, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(neon_pmaxh, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(neon_pminh, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(neon_padds, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(neon_pmaxs, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(neon_pmins, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_4(gvec_frecpe_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_frecpe_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_frecpe_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -XXX,XX +XXX,XX @@ DO_ABA(gvec_uaba_s, uint32_t) DO_ABA(gvec_uaba_d, uint64_t) #undef DO_ABA + +#define DO_NEON_PAIRWISE(NAME, OP) \ + void HELPER(NAME##s)(void *vd, void *vn, void *vm, \ + void *stat, uint32_t oprsz) \ + { \ + float_status *fpst = stat; \ + float32 *d = vd; \ + float32 *n = vn; \ + float32 *m = vm; \ + float32 r0, r1; \ + \ + /* Read all inputs before writing outputs in case vm == vd */ \ + r0 = float32_##OP(n[H4(0)], n[H4(1)], fpst); \ + r1 = float32_##OP(m[H4(0)], m[H4(1)], fpst); \ + \ + d[H4(0)] = r0; \ + d[H4(1)] = r1; \ + } \ + \ + void HELPER(NAME##h)(void *vd, void *vn, void *vm, \ + void *stat, uint32_t oprsz) \ + { \ + float_status *fpst = stat; \ + float16 *d = vd; \ + float16 *n = vn; \ + float16 *m = vm; \ + float16 r0, r1, r2, r3; \ + \ + /* Read all inputs before writing outputs in case vm == vd */ \ + r0 = float16_##OP(n[H2(0)], n[H2(1)], fpst); \ + r1 = float16_##OP(n[H2(2)], n[H2(3)], fpst); \ + r2 = float16_##OP(m[H2(0)], m[H2(1)], fpst); \ + r3 = float16_##OP(m[H2(2)], m[H2(3)], fpst); \ + \ + d[H4(0)] = r0; \ + d[H4(1)] = r1; \ + d[H4(2)] = r2; \ + d[H4(3)] = r3; \ + } + +DO_NEON_PAIRWISE(neon_padd, add) +DO_NEON_PAIRWISE(neon_pmax, max) +DO_NEON_PAIRWISE(neon_pmin, min) + +#undef DO_NEON_PAIRWISE diff --git a/target/arm/translate-neon.c.inc b/target/arm/translate-neon.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-neon.c.inc +++ b/target/arm/translate-neon.c.inc @@ -XXX,XX +XXX,XX @@ static bool trans_VMINNM_fp_3s(DisasContext *s, arg_3same *a) return do_3same(s, a, gen_VMINNM_fp32_3s); } -static bool do_3same_fp_pair(DisasContext *s, arg_3same *a, VFPGen3OpSPFn *fn) +static bool do_3same_fp_pair(DisasContext *s, arg_3same *a, + gen_helper_gvec_3_ptr *fn) { - /* FP operations handled pairwise 32 bits at a time */ - TCGv_i32 tmp, tmp2, tmp3; + /* FP pairwise operations */ TCGv_ptr fpstatus; if (!arm_dc_feature(s, ARM_FEATURE_NEON)) { @@ -XXX,XX +XXX,XX @@ static bool do_3same_fp_pair(DisasContext *s, arg_3same *a, VFPGen3OpSPFn *fn) assert(a->q == 0); /* enforced by decode patterns */ - /* - * Note that we have to be careful not to clobber the source operands - * in the "vm == vd" case by storing the result of the first pass too - * early. Since Q is 0 there are always just two passes, so instead - * of a complicated loop over each pass we just unroll. - */ - fpstatus = fpstatus_ptr(FPST_STD); - tmp = neon_load_reg(a->vn, 0); - tmp2 = neon_load_reg(a->vn, 1); - fn(tmp, tmp, tmp2, fpstatus); - tcg_temp_free_i32(tmp2); - tmp3 = neon_load_reg(a->vm, 0); - tmp2 = neon_load_reg(a->vm, 1); - fn(tmp3, tmp3, tmp2, fpstatus); - tcg_temp_free_i32(tmp2); + fpstatus = fpstatus_ptr(a->size != 0 ? FPST_STD_F16 : FPST_STD); + tcg_gen_gvec_3_ptr(vfp_reg_offset(1, a->vd), + vfp_reg_offset(1, a->vn), + vfp_reg_offset(1, a->vm), + fpstatus, 8, 8, 0, fn); tcg_temp_free_ptr(fpstatus); - neon_store_reg(a->vd, 0, tmp); - neon_store_reg(a->vd, 1, tmp3); return true; } @@ -XXX,XX +XXX,XX @@ static bool do_3same_fp_pair(DisasContext *s, arg_3same *a, VFPGen3OpSPFn *fn) static bool trans_##INSN##_fp_3s(DisasContext *s, arg_3same *a) \ { \ if (a->size != 0) { \ - /* TODO fp16 support */ \ - return false; \ + if (!dc_isar_feature(aa32_fp16_arith, s)) { \ + return false; \ + } \ + return do_3same_fp_pair(s, a, FUNC##h); \ } \ - return do_3same_fp_pair(s, a, FUNC); \ + return do_3same_fp_pair(s, a, FUNC##s); \ } -DO_3S_FP_PAIR(VPADD, gen_helper_vfp_adds) -DO_3S_FP_PAIR(VPMAX, gen_helper_vfp_maxs) -DO_3S_FP_PAIR(VPMIN, gen_helper_vfp_mins) +DO_3S_FP_PAIR(VPADD, gen_helper_neon_padd) +DO_3S_FP_PAIR(VPMAX, gen_helper_neon_pmax) +DO_3S_FP_PAIR(VPMIN, gen_helper_neon_pmin) static bool do_vector_2sh(DisasContext *s, arg_2reg_shift *a, GVecGen2iFn *fn) { -- 2.20.1 Convert the Neon float-integer VCVT insns to gvec, and use this to implement fp16 support for them. Note that unlike the VFP int<->fp16 VCVT insns we converted earlier and which convert to/from a 32-bit integer, these Neon insns convert to/from 16-bit integers. So we can use the existing vfp conversion helpers for the f32<->u32/i32 case but need to provide our own for f16<->u16/i16. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-37-peter.maydell@linaro.org --- target/arm/helper.h | 9 +++++++++ target/arm/vec_helper.c | 29 +++++++++++++++++++++++++++++ target/arm/translate-neon.c.inc | 15 ++++----------- 3 files changed, 42 insertions(+), 11 deletions(-) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_5(neon_padds, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(neon_pmaxs, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(neon_pmins, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_sstoh, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_sitos, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_ustoh, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_uitos, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_tosszh, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_tosizs, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_touszh, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_touizs, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_4(gvec_frecpe_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_frecpe_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_frecpe_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -XXX,XX +XXX,XX @@ static uint32_t float32_acgt(float32 op1, float32 op2, float_status *stat) return -float32_lt(float32_abs(op2), float32_abs(op1), stat); } +static int16_t vfp_tosszh(float16 x, void *fpstp) +{ + float_status *fpst = fpstp; + if (float16_is_any_nan(x)) { + float_raise(float_flag_invalid, fpst); + return 0; + } + return float16_to_int16_round_to_zero(x, fpst); +} + +static uint16_t vfp_touszh(float16 x, void *fpstp) +{ + float_status *fpst = fpstp; + if (float16_is_any_nan(x)) { + float_raise(float_flag_invalid, fpst); + return 0; + } + return float16_to_uint16_round_to_zero(x, fpst); +} + #define DO_2OP(NAME, FUNC, TYPE) \ void HELPER(NAME)(void *vd, void *vn, void *stat, uint32_t desc) \ { \ @@ -XXX,XX +XXX,XX @@ DO_2OP(gvec_frsqrte_h, helper_rsqrte_f16, float16) DO_2OP(gvec_frsqrte_s, helper_rsqrte_f32, float32) DO_2OP(gvec_frsqrte_d, helper_rsqrte_f64, float64) +DO_2OP(gvec_sitos, helper_vfp_sitos, int32_t) +DO_2OP(gvec_uitos, helper_vfp_uitos, uint32_t) +DO_2OP(gvec_tosizs, helper_vfp_tosizs, float32) +DO_2OP(gvec_touizs, helper_vfp_touizs, float32) +DO_2OP(gvec_sstoh, int16_to_float16, int16_t) +DO_2OP(gvec_ustoh, uint16_to_float16, uint16_t) +DO_2OP(gvec_tosszh, vfp_tosszh, float16) +DO_2OP(gvec_touszh, vfp_touszh, float16) + #define WRAP_CMP0_FWD(FN, CMPOP, TYPE) \ static TYPE TYPE##_##FN##0(TYPE op, float_status *stat) \ { \ diff --git a/target/arm/translate-neon.c.inc b/target/arm/translate-neon.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-neon.c.inc +++ b/target/arm/translate-neon.c.inc @@ -XXX,XX +XXX,XX @@ static bool do_2misc_fp(DisasContext *s, arg_2misc *a, return true; } -#define DO_2MISC_FP(INSN, FUNC) \ - static bool trans_##INSN(DisasContext *s, arg_2misc *a) \ - { \ - return do_2misc_fp(s, a, FUNC); \ - } - -DO_2MISC_FP(VCVT_FS, gen_helper_vfp_sitos) -DO_2MISC_FP(VCVT_FU, gen_helper_vfp_uitos) -DO_2MISC_FP(VCVT_SF, gen_helper_vfp_tosizs) -DO_2MISC_FP(VCVT_UF, gen_helper_vfp_touizs) - #define DO_2MISC_FP_VEC(INSN, HFUNC, SFUNC) \ static void gen_##INSN(unsigned vece, uint32_t rd_ofs, \ uint32_t rm_ofs, \ @@ -XXX,XX +XXX,XX @@ DO_2MISC_FP_VEC(VCGE0_F, gen_helper_gvec_fcge0_h, gen_helper_gvec_fcge0_s) DO_2MISC_FP_VEC(VCEQ0_F, gen_helper_gvec_fceq0_h, gen_helper_gvec_fceq0_s) DO_2MISC_FP_VEC(VCLT0_F, gen_helper_gvec_fclt0_h, gen_helper_gvec_fclt0_s) DO_2MISC_FP_VEC(VCLE0_F, gen_helper_gvec_fcle0_h, gen_helper_gvec_fcle0_s) +DO_2MISC_FP_VEC(VCVT_FS, gen_helper_gvec_sstoh, gen_helper_gvec_sitos) +DO_2MISC_FP_VEC(VCVT_FU, gen_helper_gvec_ustoh, gen_helper_gvec_uitos) +DO_2MISC_FP_VEC(VCVT_SF, gen_helper_gvec_tosszh, gen_helper_gvec_tosizs) +DO_2MISC_FP_VEC(VCVT_UF, gen_helper_gvec_touszh, gen_helper_gvec_touizs) static bool trans_VRINTX(DisasContext *s, arg_2misc *a) { -- 2.20.1 Convert the Neon VCVT float<->fixed-point insns to a gvec style, in preparation for adding fp16 support. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-38-peter.maydell@linaro.org --- target/arm/helper.h | 5 +++++ target/arm/vec_helper.c | 20 +++++++++++++++++++ target/arm/translate-neon.c.inc | 35 +++++++++++++++++---------------- 3 files changed, 43 insertions(+), 17 deletions(-) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_4(gvec_tosizs, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_touszh, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_touizs, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_vcvt_sf, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_vcvt_uf, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_vcvt_fs, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_vcvt_fu, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_4(gvec_frecpe_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_frecpe_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_frecpe_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -XXX,XX +XXX,XX @@ DO_NEON_PAIRWISE(neon_pmax, max) DO_NEON_PAIRWISE(neon_pmin, min) #undef DO_NEON_PAIRWISE + +#define DO_VCVT_FIXED(NAME, FUNC, TYPE) \ + void HELPER(NAME)(void *vd, void *vn, void *stat, uint32_t desc) \ + { \ + intptr_t i, oprsz = simd_oprsz(desc); \ + int shift = simd_data(desc); \ + TYPE *d = vd, *n = vn; \ + float_status *fpst = stat; \ + for (i = 0; i < oprsz / sizeof(TYPE); i++) { \ + d[i] = FUNC(n[i], shift, fpst); \ + } \ + clear_tail(d, oprsz, simd_maxsz(desc)); \ + } + +DO_VCVT_FIXED(gvec_vcvt_sf, helper_vfp_sltos, uint32_t) +DO_VCVT_FIXED(gvec_vcvt_uf, helper_vfp_ultos, uint32_t) +DO_VCVT_FIXED(gvec_vcvt_fs, helper_vfp_tosls_round_to_zero, uint32_t) +DO_VCVT_FIXED(gvec_vcvt_fu, helper_vfp_touls_round_to_zero, uint32_t) + +#undef DO_VCVT_FIXED diff --git a/target/arm/translate-neon.c.inc b/target/arm/translate-neon.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-neon.c.inc +++ b/target/arm/translate-neon.c.inc @@ -XXX,XX +XXX,XX @@ static bool trans_VSHLL_U_2sh(DisasContext *s, arg_2reg_shift *a) } static bool do_fp_2sh(DisasContext *s, arg_2reg_shift *a, - NeonGenTwoSingleOpFn *fn) + gen_helper_gvec_2_ptr *fn) { /* FP operations in 2-reg-and-shift group */ - TCGv_i32 tmp, shiftv; - TCGv_ptr fpstatus; - int pass; + int vec_size = a->q ? 16 : 8; + int rd_ofs = neon_reg_offset(a->vd, 0); + int rm_ofs = neon_reg_offset(a->vm, 0); + TCGv_ptr fpst; if (!arm_dc_feature(s, ARM_FEATURE_NEON)) { return false; } + if (a->size != 0) { + if (!dc_isar_feature(aa32_fp16_arith, s)) { + return false; + } + } + /* UNDEF accesses to D16-D31 if they don't exist. */ if (!dc_isar_feature(aa32_simd_r32, s) && ((a->vd | a->vm) & 0x10)) { @@ -XXX,XX +XXX,XX @@ static bool do_fp_2sh(DisasContext *s, arg_2reg_shift *a, return true; } - fpstatus = fpstatus_ptr(FPST_STD); - shiftv = tcg_const_i32(a->shift); - for (pass = 0; pass < (a->q ? 4 : 2); pass++) { - tmp = neon_load_reg(a->vm, pass); - fn(tmp, tmp, shiftv, fpstatus); - neon_store_reg(a->vd, pass, tmp); - } - tcg_temp_free_ptr(fpstatus); - tcg_temp_free_i32(shiftv); + fpst = fpstatus_ptr(a->size ? FPST_STD_F16 : FPST_STD); + tcg_gen_gvec_2_ptr(rd_ofs, rm_ofs, fpst, vec_size, vec_size, a->shift, fn); + tcg_temp_free_ptr(fpst); return true; } @@ -XXX,XX +XXX,XX @@ static bool do_fp_2sh(DisasContext *s, arg_2reg_shift *a, return do_fp_2sh(s, a, FUNC); \ } -DO_FP_2SH(VCVT_SF, gen_helper_vfp_sltos) -DO_FP_2SH(VCVT_UF, gen_helper_vfp_ultos) -DO_FP_2SH(VCVT_FS, gen_helper_vfp_tosls_round_to_zero) -DO_FP_2SH(VCVT_FU, gen_helper_vfp_touls_round_to_zero) +DO_FP_2SH(VCVT_SF, gen_helper_gvec_vcvt_sf) +DO_FP_2SH(VCVT_UF, gen_helper_gvec_vcvt_uf) +DO_FP_2SH(VCVT_FS, gen_helper_gvec_vcvt_fs) +DO_FP_2SH(VCVT_FU, gen_helper_gvec_vcvt_fu) static uint64_t asimd_imm_const(uint32_t imm, int cmode, int op) { -- 2.20.1 Implement fp16 for the Neon VCVT insns which convert between float and fixed-point. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-39-peter.maydell@linaro.org --- target/arm/helper.h | 5 +++++ target/arm/neon-dp.decode | 8 +++++++- target/arm/vec_helper.c | 4 ++++ target/arm/translate-neon.c.inc | 5 +++++ 4 files changed, 21 insertions(+), 1 deletion(-) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_4(gvec_vcvt_uf, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_vcvt_fs, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_vcvt_fu, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_vcvt_sh, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_vcvt_uh, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_vcvt_hs, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_vcvt_hu, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_4(gvec_frecpe_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_frecpe_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_frecpe_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) diff --git a/target/arm/neon-dp.decode b/target/arm/neon-dp.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/neon-dp.decode +++ b/target/arm/neon-dp.decode @@ -XXX,XX +XXX,XX @@ VMINNM_fp_3s 1111 001 1 0 . 1 . .... .... 1111 ... 1 .... @3same_fp # We use size=0 for fp32 and size=1 for fp16 to match the 3-same encodings. @2reg_vcvt .... ... . . . 1 ..... .... .... . q:1 . . .... \ &2reg_shift vm=%vm_dp vd=%vd_dp size=0 shift=%neon_rshift_i5 +@2reg_vcvt_f16 .... ... . . . 11 .... .... .... . q:1 . . .... \ + &2reg_shift vm=%vm_dp vd=%vd_dp size=1 shift=%neon_rshift_i4 VSHR_S_2sh 1111 001 0 1 . ...... .... 0000 . . . 1 .... @2reg_shr_d VSHR_S_2sh 1111 001 0 1 . ...... .... 0000 . . . 1 .... @2reg_shr_s @@ -XXX,XX +XXX,XX @@ VSHLL_U_2sh 1111 001 1 1 . ...... .... 1010 . 0 . 1 .... @2reg_shll_h VSHLL_U_2sh 1111 001 1 1 . ...... .... 1010 . 0 . 1 .... @2reg_shll_b # VCVT fixed<->float conversions -# TODO: FP16 fixed<->float conversions are opc==0b1100 and 0b1101 +VCVT_SH_2sh 1111 001 0 1 . ...... .... 1100 0 . . 1 .... @2reg_vcvt_f16 +VCVT_UH_2sh 1111 001 1 1 . ...... .... 1100 0 . . 1 .... @2reg_vcvt_f16 +VCVT_HS_2sh 1111 001 0 1 . ...... .... 1101 0 . . 1 .... @2reg_vcvt_f16 +VCVT_HU_2sh 1111 001 1 1 . ...... .... 1101 0 . . 1 .... @2reg_vcvt_f16 + VCVT_SF_2sh 1111 001 0 1 . ...... .... 1110 0 . . 1 .... @2reg_vcvt VCVT_UF_2sh 1111 001 1 1 . ...... .... 1110 0 . . 1 .... @2reg_vcvt VCVT_FS_2sh 1111 001 0 1 . ...... .... 1111 0 . . 1 .... @2reg_vcvt diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -XXX,XX +XXX,XX @@ DO_VCVT_FIXED(gvec_vcvt_sf, helper_vfp_sltos, uint32_t) DO_VCVT_FIXED(gvec_vcvt_uf, helper_vfp_ultos, uint32_t) DO_VCVT_FIXED(gvec_vcvt_fs, helper_vfp_tosls_round_to_zero, uint32_t) DO_VCVT_FIXED(gvec_vcvt_fu, helper_vfp_touls_round_to_zero, uint32_t) +DO_VCVT_FIXED(gvec_vcvt_sh, helper_vfp_shtoh, uint16_t) +DO_VCVT_FIXED(gvec_vcvt_uh, helper_vfp_uhtoh, uint16_t) +DO_VCVT_FIXED(gvec_vcvt_hs, helper_vfp_toshh_round_to_zero, uint16_t) +DO_VCVT_FIXED(gvec_vcvt_hu, helper_vfp_touhh_round_to_zero, uint16_t) #undef DO_VCVT_FIXED diff --git a/target/arm/translate-neon.c.inc b/target/arm/translate-neon.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-neon.c.inc +++ b/target/arm/translate-neon.c.inc @@ -XXX,XX +XXX,XX @@ DO_FP_2SH(VCVT_UF, gen_helper_gvec_vcvt_uf) DO_FP_2SH(VCVT_FS, gen_helper_gvec_vcvt_fs) DO_FP_2SH(VCVT_FU, gen_helper_gvec_vcvt_fu) +DO_FP_2SH(VCVT_SH, gen_helper_gvec_vcvt_sh) +DO_FP_2SH(VCVT_UH, gen_helper_gvec_vcvt_uh) +DO_FP_2SH(VCVT_HS, gen_helper_gvec_vcvt_hs) +DO_FP_2SH(VCVT_HU, gen_helper_gvec_vcvt_hu) + static uint64_t asimd_imm_const(uint32_t imm, int cmode, int op) { /* -- 2.20.1 Convert the Neon VCVT with-specified-rounding-mode instructions to gvec, and use this to implement fp16 support for them. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-40-peter.maydell@linaro.org --- target/arm/helper.h | 5 ++ target/arm/vec_helper.c | 23 +++++++ target/arm/translate-neon.c.inc | 105 ++++++++++++-------------------- 3 files changed, 66 insertions(+), 67 deletions(-) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_4(gvec_vcvt_uh, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_vcvt_hs, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_vcvt_hu, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_vcvt_rm_ss, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_vcvt_rm_us, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_vcvt_rm_sh, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_vcvt_rm_uh, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_4(gvec_frecpe_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_frecpe_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_frecpe_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -XXX,XX +XXX,XX @@ DO_VCVT_FIXED(gvec_vcvt_hs, helper_vfp_toshh_round_to_zero, uint16_t) DO_VCVT_FIXED(gvec_vcvt_hu, helper_vfp_touhh_round_to_zero, uint16_t) #undef DO_VCVT_FIXED + +#define DO_VCVT_RMODE(NAME, FUNC, TYPE) \ + void HELPER(NAME)(void *vd, void *vn, void *stat, uint32_t desc) \ + { \ + float_status *fpst = stat; \ + intptr_t i, oprsz = simd_oprsz(desc); \ + uint32_t rmode = simd_data(desc); \ + uint32_t prev_rmode = get_float_rounding_mode(fpst); \ + TYPE *d = vd, *n = vn; \ + set_float_rounding_mode(rmode, fpst); \ + for (i = 0; i < oprsz / sizeof(TYPE); i++) { \ + d[i] = FUNC(n[i], 0, fpst); \ + } \ + set_float_rounding_mode(prev_rmode, fpst); \ + clear_tail(d, oprsz, simd_maxsz(desc)); \ + } + +DO_VCVT_RMODE(gvec_vcvt_rm_ss, helper_vfp_tosls, uint32_t) +DO_VCVT_RMODE(gvec_vcvt_rm_us, helper_vfp_touls, uint32_t) +DO_VCVT_RMODE(gvec_vcvt_rm_sh, helper_vfp_toshh, uint16_t) +DO_VCVT_RMODE(gvec_vcvt_rm_uh, helper_vfp_touhh, uint16_t) + +#undef DO_VCVT_RMODE diff --git a/target/arm/translate-neon.c.inc b/target/arm/translate-neon.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-neon.c.inc +++ b/target/arm/translate-neon.c.inc @@ -XXX,XX +XXX,XX @@ DO_VRINT(VRINTZ, FPROUNDING_ZERO) DO_VRINT(VRINTM, FPROUNDING_NEGINF) DO_VRINT(VRINTP, FPROUNDING_POSINF) -static bool do_vcvt(DisasContext *s, arg_2misc *a, int rmode, bool is_signed) -{ - /* - * Handle a VCVT* operation by iterating 32 bits at a time, - * with a specified rounding mode in operation. - */ - int pass; - TCGv_ptr fpst; - TCGv_i32 tcg_rmode, tcg_shift; - - if (!arm_dc_feature(s, ARM_FEATURE_NEON) || - !arm_dc_feature(s, ARM_FEATURE_V8)) { - return false; +#define DO_VEC_RMODE(INSN, RMODE, OP) \ + static void gen_##INSN(unsigned vece, uint32_t rd_ofs, \ + uint32_t rm_ofs, \ + uint32_t oprsz, uint32_t maxsz) \ + { \ + static gen_helper_gvec_2_ptr * const fns[4] = { \ + NULL, \ + gen_helper_gvec_##OP##h, \ + gen_helper_gvec_##OP##s, \ + NULL, \ + }; \ + TCGv_ptr fpst; \ + fpst = fpstatus_ptr(vece == 1 ? FPST_STD_F16 : FPST_STD); \ + tcg_gen_gvec_2_ptr(rd_ofs, rm_ofs, fpst, oprsz, maxsz, \ + arm_rmode_to_sf(RMODE), fns[vece]); \ + tcg_temp_free_ptr(fpst); \ + } \ + static bool trans_##INSN(DisasContext *s, arg_2misc *a) \ + { \ + if (!arm_dc_feature(s, ARM_FEATURE_V8)) { \ + return false; \ + } \ + if (a->size == MO_16) { \ + if (!dc_isar_feature(aa32_fp16_arith, s)) { \ + return false; \ + } \ + } else if (a->size != MO_32) { \ + return false; \ + } \ + return do_2misc_vec(s, a, gen_##INSN); \ } - /* UNDEF accesses to D16-D31 if they don't exist. */ - if (!dc_isar_feature(aa32_simd_r32, s) && - ((a->vd | a->vm) & 0x10)) { - return false; - } - - if (a->size != 2) { - /* TODO: FP16 will be the size == 1 case */ - return false; - } - - if ((a->vd | a->vm) & a->q) { - return false; - } - - if (!vfp_access_check(s)) { - return true; - } - - fpst = fpstatus_ptr(FPST_STD); - tcg_shift = tcg_const_i32(0); - tcg_rmode = tcg_const_i32(arm_rmode_to_sf(rmode)); - gen_helper_set_neon_rmode(tcg_rmode, tcg_rmode, cpu_env); - for (pass = 0; pass < (a->q ? 4 : 2); pass++) { - TCGv_i32 tmp = neon_load_reg(a->vm, pass); - if (is_signed) { - gen_helper_vfp_tosls(tmp, tmp, tcg_shift, fpst); - } else { - gen_helper_vfp_touls(tmp, tmp, tcg_shift, fpst); - } - neon_store_reg(a->vd, pass, tmp); - } - gen_helper_set_neon_rmode(tcg_rmode, tcg_rmode, cpu_env); - tcg_temp_free_i32(tcg_rmode); - tcg_temp_free_i32(tcg_shift); - tcg_temp_free_ptr(fpst); - - return true; -} - -#define DO_VCVT(INSN, RMODE, SIGNED) \ - static bool trans_##INSN(DisasContext *s, arg_2misc *a) \ - { \ - return do_vcvt(s, a, RMODE, SIGNED); \ - } - -DO_VCVT(VCVTAU, FPROUNDING_TIEAWAY, false) -DO_VCVT(VCVTAS, FPROUNDING_TIEAWAY, true) -DO_VCVT(VCVTNU, FPROUNDING_TIEEVEN, false) -DO_VCVT(VCVTNS, FPROUNDING_TIEEVEN, true) -DO_VCVT(VCVTPU, FPROUNDING_POSINF, false) -DO_VCVT(VCVTPS, FPROUNDING_POSINF, true) -DO_VCVT(VCVTMU, FPROUNDING_NEGINF, false) -DO_VCVT(VCVTMS, FPROUNDING_NEGINF, true) +DO_VEC_RMODE(VCVTAU, FPROUNDING_TIEAWAY, vcvt_rm_u) +DO_VEC_RMODE(VCVTAS, FPROUNDING_TIEAWAY, vcvt_rm_s) +DO_VEC_RMODE(VCVTNU, FPROUNDING_TIEEVEN, vcvt_rm_u) +DO_VEC_RMODE(VCVTNS, FPROUNDING_TIEEVEN, vcvt_rm_s) +DO_VEC_RMODE(VCVTPU, FPROUNDING_POSINF, vcvt_rm_u) +DO_VEC_RMODE(VCVTPS, FPROUNDING_POSINF, vcvt_rm_s) +DO_VEC_RMODE(VCVTMU, FPROUNDING_NEGINF, vcvt_rm_u) +DO_VEC_RMODE(VCVTMS, FPROUNDING_NEGINF, vcvt_rm_s) static bool trans_VSWP(DisasContext *s, arg_2misc *a) { -- 2.20.1 Convert the Neon VRINT-with-specified-rounding-mode insns to gvec, and use this to implement the fp16 versions. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-41-peter.maydell@linaro.org --- target/arm/helper.h | 4 +- target/arm/vec_helper.c | 21 +++++++++++ target/arm/vfp_helper.c | 17 --------- target/arm/translate-neon.c.inc | 67 +++------------------------------ 4 files changed, 30 insertions(+), 79 deletions(-) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_3(vfp_sqtoh, f16, i64, i32, ptr) DEF_HELPER_3(vfp_uqtoh, f16, i64, i32, ptr) DEF_HELPER_FLAGS_2(set_rmode, TCG_CALL_NO_RWG, i32, i32, ptr) -DEF_HELPER_FLAGS_2(set_neon_rmode, TCG_CALL_NO_RWG, i32, i32, env) DEF_HELPER_FLAGS_3(vfp_fcvt_f16_to_f32, TCG_CALL_NO_RWG, f32, f16, ptr, i32) DEF_HELPER_FLAGS_3(vfp_fcvt_f32_to_f16, TCG_CALL_NO_RWG, f16, f32, ptr, i32) @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_4(gvec_vcvt_rm_us, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_vcvt_rm_sh, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_vcvt_rm_uh, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_vrint_rm_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_vrint_rm_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_4(gvec_frecpe_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_frecpe_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_frecpe_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -XXX,XX +XXX,XX @@ DO_VCVT_RMODE(gvec_vcvt_rm_sh, helper_vfp_toshh, uint16_t) DO_VCVT_RMODE(gvec_vcvt_rm_uh, helper_vfp_touhh, uint16_t) #undef DO_VCVT_RMODE + +#define DO_VRINT_RMODE(NAME, FUNC, TYPE) \ + void HELPER(NAME)(void *vd, void *vn, void *stat, uint32_t desc) \ + { \ + float_status *fpst = stat; \ + intptr_t i, oprsz = simd_oprsz(desc); \ + uint32_t rmode = simd_data(desc); \ + uint32_t prev_rmode = get_float_rounding_mode(fpst); \ + TYPE *d = vd, *n = vn; \ + set_float_rounding_mode(rmode, fpst); \ + for (i = 0; i < oprsz / sizeof(TYPE); i++) { \ + d[i] = FUNC(n[i], fpst); \ + } \ + set_float_rounding_mode(prev_rmode, fpst); \ + clear_tail(d, oprsz, simd_maxsz(desc)); \ + } + +DO_VRINT_RMODE(gvec_vrint_rm_h, helper_rinth, uint16_t) +DO_VRINT_RMODE(gvec_vrint_rm_s, helper_rints, uint32_t) + +#undef DO_VRINT_RMODE diff --git a/target/arm/vfp_helper.c b/target/arm/vfp_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vfp_helper.c +++ b/target/arm/vfp_helper.c @@ -XXX,XX +XXX,XX @@ uint32_t HELPER(set_rmode)(uint32_t rmode, void *fpstp) return prev_rmode; } -/* Set the current fp rounding mode in the standard fp status and return - * the old one. This is for NEON instructions that need to change the - * rounding mode but wish to use the standard FPSCR values for everything - * else. Always set the rounding mode back to the correct value after - * modifying it. - * The argument is a softfloat float_round_ value. - */ -uint32_t HELPER(set_neon_rmode)(uint32_t rmode, CPUARMState *env) -{ - float_status *fp_status = &env->vfp.standard_fp_status; - - uint32_t prev_rmode = get_float_rounding_mode(fp_status); - set_float_rounding_mode(rmode, fp_status); - - return prev_rmode; -} - /* Half precision conversions. */ float32 HELPER(vfp_fcvt_f16_to_f32)(uint32_t a, void *fpstp, uint32_t ahp_mode) { diff --git a/target/arm/translate-neon.c.inc b/target/arm/translate-neon.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-neon.c.inc +++ b/target/arm/translate-neon.c.inc @@ -XXX,XX +XXX,XX @@ static bool trans_VRINTX(DisasContext *s, arg_2misc *a) return do_2misc_fp(s, a, gen_helper_rints_exact); } -static bool do_vrint(DisasContext *s, arg_2misc *a, int rmode) -{ - /* - * Handle a VRINT* operation by iterating 32 bits at a time, - * with a specified rounding mode in operation. - */ - int pass; - TCGv_ptr fpst; - TCGv_i32 tcg_rmode; - - if (!arm_dc_feature(s, ARM_FEATURE_NEON) || - !arm_dc_feature(s, ARM_FEATURE_V8)) { - return false; - } - - /* UNDEF accesses to D16-D31 if they don't exist. */ - if (!dc_isar_feature(aa32_simd_r32, s) && - ((a->vd | a->vm) & 0x10)) { - return false; - } - - if (a->size != 2) { - /* TODO: FP16 will be the size == 1 case */ - return false; - } - - if ((a->vd | a->vm) & a->q) { - return false; - } - - if (!vfp_access_check(s)) { - return true; - } - - fpst = fpstatus_ptr(FPST_STD); - tcg_rmode = tcg_const_i32(arm_rmode_to_sf(rmode)); - gen_helper_set_neon_rmode(tcg_rmode, tcg_rmode, cpu_env); - for (pass = 0; pass < (a->q ? 4 : 2); pass++) { - TCGv_i32 tmp = neon_load_reg(a->vm, pass); - gen_helper_rints(tmp, tmp, fpst); - neon_store_reg(a->vd, pass, tmp); - } - gen_helper_set_neon_rmode(tcg_rmode, tcg_rmode, cpu_env); - tcg_temp_free_i32(tcg_rmode); - tcg_temp_free_ptr(fpst); - - return true; -} - -#define DO_VRINT(INSN, RMODE) \ - static bool trans_##INSN(DisasContext *s, arg_2misc *a) \ - { \ - return do_vrint(s, a, RMODE); \ - } - -DO_VRINT(VRINTN, FPROUNDING_TIEEVEN) -DO_VRINT(VRINTA, FPROUNDING_TIEAWAY) -DO_VRINT(VRINTZ, FPROUNDING_ZERO) -DO_VRINT(VRINTM, FPROUNDING_NEGINF) -DO_VRINT(VRINTP, FPROUNDING_POSINF) - #define DO_VEC_RMODE(INSN, RMODE, OP) \ static void gen_##INSN(unsigned vece, uint32_t rd_ofs, \ uint32_t rm_ofs, \ @@ -XXX,XX +XXX,XX @@ DO_VEC_RMODE(VCVTPS, FPROUNDING_POSINF, vcvt_rm_s) DO_VEC_RMODE(VCVTMU, FPROUNDING_NEGINF, vcvt_rm_u) DO_VEC_RMODE(VCVTMS, FPROUNDING_NEGINF, vcvt_rm_s) +DO_VEC_RMODE(VRINTN, FPROUNDING_TIEEVEN, vrint_rm_) +DO_VEC_RMODE(VRINTA, FPROUNDING_TIEAWAY, vrint_rm_) +DO_VEC_RMODE(VRINTZ, FPROUNDING_ZERO, vrint_rm_) +DO_VEC_RMODE(VRINTM, FPROUNDING_NEGINF, vrint_rm_) +DO_VEC_RMODE(VRINTP, FPROUNDING_POSINF, vrint_rm_) + static bool trans_VSWP(DisasContext *s, arg_2misc *a) { TCGv_i64 rm, rd; -- 2.20.1 Convert the Neon VRINTX insn to use gvec, and use this to implement fp16 support for it. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-42-peter.maydell@linaro.org --- target/arm/helper.h | 3 +++ target/arm/vec_helper.c | 3 +++ target/arm/translate-neon.c.inc | 45 +++------------------------------ 3 files changed, 9 insertions(+), 42 deletions(-) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_4(gvec_vcvt_rm_uh, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_vrint_rm_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_vrint_rm_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_vrintx_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_vrintx_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_4(gvec_frecpe_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_frecpe_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_frecpe_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -XXX,XX +XXX,XX @@ DO_2OP(gvec_frsqrte_h, helper_rsqrte_f16, float16) DO_2OP(gvec_frsqrte_s, helper_rsqrte_f32, float32) DO_2OP(gvec_frsqrte_d, helper_rsqrte_f64, float64) +DO_2OP(gvec_vrintx_h, float16_round_to_int, float16) +DO_2OP(gvec_vrintx_s, float32_round_to_int, float32) + DO_2OP(gvec_sitos, helper_vfp_sitos, int32_t) DO_2OP(gvec_uitos, helper_vfp_uitos, uint32_t) DO_2OP(gvec_tosizs, helper_vfp_tosizs, float32) diff --git a/target/arm/translate-neon.c.inc b/target/arm/translate-neon.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-neon.c.inc +++ b/target/arm/translate-neon.c.inc @@ -XXX,XX +XXX,XX @@ static bool trans_VQNEG(DisasContext *s, arg_2misc *a) return do_2misc(s, a, fn[a->size]); } -static bool do_2misc_fp(DisasContext *s, arg_2misc *a, - NeonGenOneSingleOpFn *fn) -{ - int pass; - TCGv_ptr fpst; - - /* Handle a 2-reg-misc operation by iterating 32 bits at a time */ - if (!arm_dc_feature(s, ARM_FEATURE_NEON)) { - return false; - } - - /* UNDEF accesses to D16-D31 if they don't exist. */ - if (!dc_isar_feature(aa32_simd_r32, s) && - ((a->vd | a->vm) & 0x10)) { - return false; - } - - if (a->size != 2) { - /* TODO: FP16 will be the size == 1 case */ - return false; - } - - if ((a->vd | a->vm) & a->q) { - return false; - } - - if (!vfp_access_check(s)) { - return true; - } - - fpst = fpstatus_ptr(FPST_STD); - for (pass = 0; pass < (a->q ? 4 : 2); pass++) { - TCGv_i32 tmp = neon_load_reg(a->vm, pass); - fn(tmp, tmp, fpst); - neon_store_reg(a->vd, pass, tmp); - } - tcg_temp_free_ptr(fpst); - - return true; -} - #define DO_2MISC_FP_VEC(INSN, HFUNC, SFUNC) \ static void gen_##INSN(unsigned vece, uint32_t rd_ofs, \ uint32_t rm_ofs, \ @@ -XXX,XX +XXX,XX @@ DO_2MISC_FP_VEC(VCVT_FU, gen_helper_gvec_ustoh, gen_helper_gvec_uitos) DO_2MISC_FP_VEC(VCVT_SF, gen_helper_gvec_tosszh, gen_helper_gvec_tosizs) DO_2MISC_FP_VEC(VCVT_UF, gen_helper_gvec_touszh, gen_helper_gvec_touizs) +DO_2MISC_FP_VEC(VRINTX_impl, gen_helper_gvec_vrintx_h, gen_helper_gvec_vrintx_s) + static bool trans_VRINTX(DisasContext *s, arg_2misc *a) { if (!arm_dc_feature(s, ARM_FEATURE_V8)) { return false; } - return do_2misc_fp(s, a, gen_helper_rints_exact); + return trans_VRINTX_impl(s, a); } #define DO_VEC_RMODE(INSN, RMODE, OP) \ -- 2.20.1 In the gvec helper functions for indexed operations, for AArch32 Neon the oprsz (total size of the vector) can be less than 16 bytes if the operation is on a D reg. Since the inner loop in these helpers always goes from 0 to segment, we must clamp it based on oprsz to avoid processing a full 16 byte segment when asked to handle an 8 byte wide vector. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-43-peter.maydell@linaro.org --- target/arm/vec_helper.c | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -XXX,XX +XXX,XX @@ DO_MULADD(gvec_vfms_s, float32_mulsub_f, float32) #define DO_MUL_IDX(NAME, TYPE, H) \ void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc) \ { \ - intptr_t i, j, oprsz = simd_oprsz(desc), segment = 16 / sizeof(TYPE); \ + intptr_t i, j, oprsz = simd_oprsz(desc); \ + intptr_t segment = MIN(16, oprsz) / sizeof(TYPE); \ intptr_t idx = simd_data(desc); \ TYPE *d = vd, *n = vn, *m = vm; \ for (i = 0; i < oprsz / sizeof(TYPE); i += segment) { \ @@ -XXX,XX +XXX,XX @@ DO_MUL_IDX(gvec_mul_idx_d, uint64_t, ) #define DO_MLA_IDX(NAME, TYPE, OP, H) \ void HELPER(NAME)(void *vd, void *vn, void *vm, void *va, uint32_t desc) \ { \ - intptr_t i, j, oprsz = simd_oprsz(desc), segment = 16 / sizeof(TYPE); \ + intptr_t i, j, oprsz = simd_oprsz(desc); \ + intptr_t segment = MIN(16, oprsz) / sizeof(TYPE); \ intptr_t idx = simd_data(desc); \ TYPE *d = vd, *n = vn, *m = vm, *a = va; \ for (i = 0; i < oprsz / sizeof(TYPE); i += segment) { \ @@ -XXX,XX +XXX,XX @@ DO_MLA_IDX(gvec_mls_idx_d, uint64_t, -, ) #define DO_FMUL_IDX(NAME, TYPE, H) \ void HELPER(NAME)(void *vd, void *vn, void *vm, void *stat, uint32_t desc) \ { \ - intptr_t i, j, oprsz = simd_oprsz(desc), segment = 16 / sizeof(TYPE); \ + intptr_t i, j, oprsz = simd_oprsz(desc); \ + intptr_t segment = MIN(16, oprsz) / sizeof(TYPE); \ intptr_t idx = simd_data(desc); \ TYPE *d = vd, *n = vn, *m = vm; \ for (i = 0; i < oprsz / sizeof(TYPE); i += segment) { \ @@ -XXX,XX +XXX,XX @@ DO_FMUL_IDX(gvec_fmul_idx_d, float64, ) void HELPER(NAME)(void *vd, void *vn, void *vm, void *va, \ void *stat, uint32_t desc) \ { \ - intptr_t i, j, oprsz = simd_oprsz(desc), segment = 16 / sizeof(TYPE); \ + intptr_t i, j, oprsz = simd_oprsz(desc); \ + intptr_t segment = MIN(16, oprsz) / sizeof(TYPE); \ TYPE op1_neg = extract32(desc, SIMD_DATA_SHIFT, 1); \ intptr_t idx = desc >> (SIMD_DATA_SHIFT + 1); \ TYPE *d = vd, *n = vn, *m = vm, *a = va; \ -- 2.20.1 Add gvec helpers for doing Neon-style indexed non-fused fp multiply-and-accumulate operations. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Message-id: 20200828183354.27913-44-peter.maydell@linaro.org --- target/arm/helper.h | 10 ++++++++++ target/arm/vec_helper.c | 27 ++++++++++++++++++++++----- 2 files changed, 32 insertions(+), 5 deletions(-) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_5(gvec_fmul_idx_s, TCG_CALL_NO_RWG, DEF_HELPER_FLAGS_5(gvec_fmul_idx_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_fmla_nf_idx_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_fmla_nf_idx_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(gvec_fmls_nf_idx_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_fmls_nf_idx_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_6(gvec_fmla_idx_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_6(gvec_fmla_idx_s, TCG_CALL_NO_RWG, diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -XXX,XX +XXX,XX @@ DO_MLA_IDX(gvec_mls_idx_d, uint64_t, -, ) #undef DO_MLA_IDX -#define DO_FMUL_IDX(NAME, TYPE, H) \ +#define DO_FMUL_IDX(NAME, ADD, TYPE, H) \ void HELPER(NAME)(void *vd, void *vn, void *vm, void *stat, uint32_t desc) \ { \ intptr_t i, j, oprsz = simd_oprsz(desc); \ @@ -XXX,XX +XXX,XX @@ void HELPER(NAME)(void *vd, void *vn, void *vm, void *stat, uint32_t desc) \ for (i = 0; i < oprsz / sizeof(TYPE); i += segment) { \ TYPE mm = m[H(i + idx)]; \ for (j = 0; j < segment; j++) { \ - d[i + j] = TYPE##_mul(n[i + j], mm, stat); \ + d[i + j] = TYPE##_##ADD(d[i + j], \ + TYPE##_mul(n[i + j], mm, stat), stat); \ } \ } \ clear_tail(d, oprsz, simd_maxsz(desc)); \ } -DO_FMUL_IDX(gvec_fmul_idx_h, float16, H2) -DO_FMUL_IDX(gvec_fmul_idx_s, float32, H4) -DO_FMUL_IDX(gvec_fmul_idx_d, float64, ) +#define float16_nop(N, M, S) (M) +#define float32_nop(N, M, S) (M) +#define float64_nop(N, M, S) (M) +DO_FMUL_IDX(gvec_fmul_idx_h, nop, float16, H2) +DO_FMUL_IDX(gvec_fmul_idx_s, nop, float32, H4) +DO_FMUL_IDX(gvec_fmul_idx_d, nop, float64, ) + +/* + * Non-fused multiply-accumulate operations, for Neon. NB that unlike + * the fused ops below they assume accumulate both from and into Vd. + */ +DO_FMUL_IDX(gvec_fmla_nf_idx_h, add, float16, H2) +DO_FMUL_IDX(gvec_fmla_nf_idx_s, add, float32, H4) +DO_FMUL_IDX(gvec_fmls_nf_idx_h, sub, float16, H2) +DO_FMUL_IDX(gvec_fmls_nf_idx_s, sub, float32, H4) + +#undef float16_nop +#undef float32_nop +#undef float64_nop #undef DO_FMUL_IDX #define DO_FMLA_IDX(NAME, TYPE, H) \ -- 2.20.1 Convert the Neon floating-point VMUL, VMLA and VMLS to use gvec, and use this to implement fp16 support. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-45-peter.maydell@linaro.org --- target/arm/translate-neon.c.inc | 114 ++++++++++++++++---------------- 1 file changed, 57 insertions(+), 57 deletions(-) diff --git a/target/arm/translate-neon.c.inc b/target/arm/translate-neon.c.inc index XXXXXXX..XXXXXXX 100644 --- a/target/arm/translate-neon.c.inc +++ b/target/arm/translate-neon.c.inc @@ -XXX,XX +XXX,XX @@ static bool trans_VMLS_2sc(DisasContext *s, arg_2scalar *a) return do_2scalar(s, a, opfn[a->size], accfn[a->size]); } -/* - * Rather than have a float-specific version of do_2scalar just for - * three insns, we wrap a NeonGenTwoSingleOpFn to turn it into - * a NeonGenTwoOpFn. - */ -#define WRAP_FP_FN(WRAPNAME, FUNC) \ - static void WRAPNAME(TCGv_i32 rd, TCGv_i32 rn, TCGv_i32 rm) \ - { \ - TCGv_ptr fpstatus = fpstatus_ptr(FPST_STD); \ - FUNC(rd, rn, rm, fpstatus); \ - tcg_temp_free_ptr(fpstatus); \ +static bool do_2scalar_fp_vec(DisasContext *s, arg_2scalar *a, + gen_helper_gvec_3_ptr *fn) +{ + /* Two registers and a scalar, using gvec */ + int vec_size = a->q ? 16 : 8; + int rd_ofs = neon_reg_offset(a->vd, 0); + int rn_ofs = neon_reg_offset(a->vn, 0); + int rm_ofs; + int idx; + TCGv_ptr fpstatus; + + if (!arm_dc_feature(s, ARM_FEATURE_NEON)) { + return false; } -WRAP_FP_FN(gen_VMUL_F_mul, gen_helper_vfp_muls) -WRAP_FP_FN(gen_VMUL_F_add, gen_helper_vfp_adds) -WRAP_FP_FN(gen_VMUL_F_sub, gen_helper_vfp_subs) + /* UNDEF accesses to D16-D31 if they don't exist. */ + if (!dc_isar_feature(aa32_simd_r32, s) && + ((a->vd | a->vn | a->vm) & 0x10)) { + return false; + } -static bool trans_VMUL_F_2sc(DisasContext *s, arg_2scalar *a) -{ - static NeonGenTwoOpFn * const opfn[] = { - NULL, - NULL, /* TODO: fp16 support */ - gen_VMUL_F_mul, - NULL, - }; + if (!fn) { + /* Bad size (including size == 3, which is a different insn group) */ + return false; + } - return do_2scalar(s, a, opfn[a->size], NULL); + if (a->q && ((a->vd | a->vn) & 1)) { + return false; + } + + if (!vfp_access_check(s)) { + return true; + } + + /* a->vm is M:Vm, which encodes both register and index */ + idx = extract32(a->vm, a->size + 2, 2); + a->vm = extract32(a->vm, 0, a->size + 2); + rm_ofs = neon_reg_offset(a->vm, 0); + + fpstatus = fpstatus_ptr(a->size == 1 ? FPST_STD_F16 : FPST_STD); + tcg_gen_gvec_3_ptr(rd_ofs, rn_ofs, rm_ofs, fpstatus, + vec_size, vec_size, idx, fn); + tcg_temp_free_ptr(fpstatus); + return true; } -static bool trans_VMLA_F_2sc(DisasContext *s, arg_2scalar *a) -{ - static NeonGenTwoOpFn * const opfn[] = { - NULL, - NULL, /* TODO: fp16 support */ - gen_VMUL_F_mul, - NULL, - }; - static NeonGenTwoOpFn * const accfn[] = { - NULL, - NULL, /* TODO: fp16 support */ - gen_VMUL_F_add, - NULL, - }; +#define DO_VMUL_F_2sc(NAME, FUNC) \ + static bool trans_##NAME##_F_2sc(DisasContext *s, arg_2scalar *a) \ + { \ + static gen_helper_gvec_3_ptr * const opfn[] = { \ + NULL, \ + gen_helper_##FUNC##_h, \ + gen_helper_##FUNC##_s, \ + NULL, \ + }; \ + if (a->size == MO_16 && !dc_isar_feature(aa32_fp16_arith, s)) { \ + return false; \ + } \ + return do_2scalar_fp_vec(s, a, opfn[a->size]); \ + } - return do_2scalar(s, a, opfn[a->size], accfn[a->size]); -} - -static bool trans_VMLS_F_2sc(DisasContext *s, arg_2scalar *a) -{ - static NeonGenTwoOpFn * const opfn[] = { - NULL, - NULL, /* TODO: fp16 support */ - gen_VMUL_F_mul, - NULL, - }; - static NeonGenTwoOpFn * const accfn[] = { - NULL, - NULL, /* TODO: fp16 support */ - gen_VMUL_F_sub, - NULL, - }; - - return do_2scalar(s, a, opfn[a->size], accfn[a->size]); -} +DO_VMUL_F_2sc(VMUL, gvec_fmul_idx) +DO_VMUL_F_2sc(VMLA, gvec_fmla_nf_idx) +DO_VMUL_F_2sc(VMLS, gvec_fmls_nf_idx) WRAP_ENV_FN(gen_VQDMULH_16, gen_helper_neon_qdmulh_s16) WRAP_ENV_FN(gen_VQDMULH_32, gen_helper_neon_qdmulh_s32) -- 2.20.1 Set the MVFR1 ID register FPHP and SIMDHP fields to indicate that our "-cpu max" has v8.2-FP16. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200828183354.27913-46-peter.maydell@linaro.org --- target/arm/cpu.c | 3 ++- target/arm/cpu64.c | 10 ++++------ 2 files changed, 6 insertions(+), 7 deletions(-) diff --git a/target/arm/cpu.c b/target/arm/cpu.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpu.c +++ b/target/arm/cpu.c @@ -XXX,XX +XXX,XX @@ static void arm_max_initfn(Object *obj) cpu->isar.id_isar6 = t; t = cpu->isar.mvfr1; - t = FIELD_DP32(t, MVFR1, FPHP, 2); /* v8.0 FP support */ + t = FIELD_DP32(t, MVFR1, FPHP, 3); /* v8.2-FP16 */ + t = FIELD_DP32(t, MVFR1, SIMDHP, 2); /* v8.2-FP16 */ cpu->isar.mvfr1 = t; t = cpu->isar.mvfr2; diff --git a/target/arm/cpu64.c b/target/arm/cpu64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpu64.c +++ b/target/arm/cpu64.c @@ -XXX,XX +XXX,XX @@ static void aarch64_max_initfn(Object *obj) u = FIELD_DP32(u, ID_DFR0, PERFMON, 5); /* v8.4-PMU */ cpu->isar.id_dfr0 = u; - /* - * FIXME: We do not yet support ARMv8.2-fp16 for AArch32 yet, - * so do not set MVFR1.FPHP. Strictly speaking this is not legal, - * but it is also not legal to enable SVE without support for FP16, - * and enabling SVE in system mode is more useful in the short term. - */ + u = cpu->isar.mvfr1; + u = FIELD_DP32(u, MVFR1, FPHP, 3); /* v8.2-FP16 */ + u = FIELD_DP32(u, MVFR1, SIMDHP, 2); /* v8.2-FP16 */ + cpu->isar.mvfr1 = u; #ifdef CONFIG_USER_ONLY /* For usermode -cpu max we can use a larger and more efficient DCZ -- 2.20.1 From: Leif Lindholm <leif@nuviainc.com> The sbsa-ref platform uses a minimal device tree to pass amount of memory as well as number of cpus to the firmware. However, when dumping that minimal dtb (with -M sbsa-virt,dumpdtb=<file>), the resulting blob generates a warning when decompiled by dtc due to lack of reg property. Add a simple reg property per cpu, representing a 64-bit MPIDR_EL1. This also ends up being cleaner than having the firmware calculating its own IDs for generating APCI. Signed-off-by: Leif Lindholm <leif@nuviainc.com> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Message-id: 20200827124335.30586-1-leif@nuviainc.com Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- hw/arm/sbsa-ref.c | 29 +++++++++++++++++++++++------ 1 file changed, 23 insertions(+), 6 deletions(-) diff --git a/hw/arm/sbsa-ref.c b/hw/arm/sbsa-ref.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/sbsa-ref.c +++ b/hw/arm/sbsa-ref.c @@ -XXX,XX +XXX,XX @@ static const int sbsa_ref_irqmap[] = { [SBSA_EHCI] = 11, }; +static uint64_t sbsa_ref_cpu_mp_affinity(SBSAMachineState *sms, int idx) +{ + uint8_t clustersz = ARM_DEFAULT_CPUS_PER_CLUSTER; + return arm_cpu_mp_affinity(idx, clustersz); +} + /* * Firmware on this machine only uses ACPI table to load OS, these limited * device tree nodes are just to let firmware know the info which varies from @@ -XXX,XX +XXX,XX @@ static void create_fdt(SBSAMachineState *sms) g_free(matrix); } + /* + * From Documentation/devicetree/bindings/arm/cpus.yaml + * On ARM v8 64-bit systems this property is required + * and matches the MPIDR_EL1 register affinity bits. + * + * * If cpus node's #address-cells property is set to 2 + * + * The first reg cell bits [7:0] must be set to + * bits [39:32] of MPIDR_EL1. + * + * The second reg cell bits [23:0] must be set to + * bits [23:0] of MPIDR_EL1. + */ qemu_fdt_add_subnode(sms->fdt, "/cpus"); + qemu_fdt_setprop_cell(sms->fdt, "/cpus", "#address-cells", 2); + qemu_fdt_setprop_cell(sms->fdt, "/cpus", "#size-cells", 0x0); for (cpu = sms->smp_cpus - 1; cpu >= 0; cpu--) { char *nodename = g_strdup_printf("/cpus/cpu@%d", cpu); ARMCPU *armcpu = ARM_CPU(qemu_get_cpu(cpu)); CPUState *cs = CPU(armcpu); + uint64_t mpidr = sbsa_ref_cpu_mp_affinity(sms, cpu); qemu_fdt_add_subnode(sms->fdt, nodename); + qemu_fdt_setprop_u64(sms->fdt, nodename, "reg", mpidr); if (ms->possible_cpus->cpus[cs->cpu_index].props.has_node_id) { qemu_fdt_setprop_cell(sms->fdt, nodename, "numa-node-id", @@ -XXX,XX +XXX,XX @@ static void sbsa_ref_init(MachineState *machine) arm_load_kernel(ARM_CPU(first_cpu), machine, &sms->bootinfo); } -static uint64_t sbsa_ref_cpu_mp_affinity(SBSAMachineState *sms, int idx) -{ - uint8_t clustersz = ARM_DEFAULT_CPUS_PER_CLUSTER; - return arm_cpu_mp_affinity(idx, clustersz); -} - static const CPUArchIdList *sbsa_ref_possible_cpu_arch_ids(MachineState *ms) { unsigned int max_cpus = ms->smp.max_cpus; -- 2.20.1 From: Graeme Gregory <graeme@nuviainc.com> A difference between sbsa platform and the virt platform is PSCI is handled by ARM-TF in the sbsa platform. This means that the PSCI code there needs to communicate some of the platform power changes down to the qemu code for things like shutdown/reset control. Space has been left to extend the EC if we find other use cases in future where ARM-TF and qemu need to communicate. Signed-off-by: Graeme Gregory <graeme@nuviainc.com> Reviewed-by: Leif Lindholm <leif@nuviainc.com> Tested-by: Leif Lindholm <leif@nuviainc.com> Message-id: 20200826141952.136164-2-graeme@nuviainc.com Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- hw/misc/sbsa_ec.c | 98 +++++++++++++++++++++++++++++++++++++++++++++ hw/misc/meson.build | 2 + 2 files changed, 100 insertions(+) create mode 100644 hw/misc/sbsa_ec.c diff --git a/hw/misc/sbsa_ec.c b/hw/misc/sbsa_ec.c new file mode 100644 index XXXXXXX..XXXXXXX --- /dev/null +++ b/hw/misc/sbsa_ec.c @@ -XXX,XX +XXX,XX @@ +/* + * ARM SBSA Reference Platform Embedded Controller + * + * A device to allow PSCI running in the secure side of sbsa-ref machine + * to communicate platform power states to qemu. + * + * Copyright (c) 2020 Nuvia Inc + * Written by Graeme Gregory <graeme@nuviainc.com> + * + * SPDX-License-Identifer: GPL-2.0-or-later + */ + +#include "qemu/osdep.h" +#include "qemu-common.h" +#include "qemu/log.h" +#include "hw/sysbus.h" +#include "sysemu/runstate.h" + +typedef struct { + SysBusDevice parent_obj; + MemoryRegion iomem; +} SECUREECState; + +#define TYPE_SBSA_EC "sbsa-ec" +#define SECURE_EC(obj) OBJECT_CHECK(SECUREECState, (obj), TYPE_SBSA_EC) + +enum sbsa_ec_powerstates { + SBSA_EC_CMD_POWEROFF = 0x01, + SBSA_EC_CMD_REBOOT = 0x02, +}; + +static uint64_t sbsa_ec_read(void *opaque, hwaddr offset, unsigned size) +{ + /* No use for this currently */ + qemu_log_mask(LOG_GUEST_ERROR, "sbsa-ec: no readable registers"); + return 0; +} + +static void sbsa_ec_write(void *opaque, hwaddr offset, + uint64_t value, unsigned size) +{ + if (offset == 0) { /* PSCI machine power command register */ + switch (value) { + case SBSA_EC_CMD_POWEROFF: + qemu_system_shutdown_request(SHUTDOWN_CAUSE_GUEST_SHUTDOWN); + break; + case SBSA_EC_CMD_REBOOT: + qemu_system_reset_request(SHUTDOWN_CAUSE_GUEST_RESET); + break; + default: + qemu_log_mask(LOG_GUEST_ERROR, + "sbsa-ec: unknown power command"); + } + } else { + qemu_log_mask(LOG_GUEST_ERROR, "sbsa-ec: unknown EC register"); + } +} + +static const MemoryRegionOps sbsa_ec_ops = { + .read = sbsa_ec_read, + .write = sbsa_ec_write, + .endianness = DEVICE_NATIVE_ENDIAN, + .valid.min_access_size = 4, + .valid.max_access_size = 4, +}; + +static void sbsa_ec_init(Object *obj) +{ + SECUREECState *s = SECURE_EC(obj); + SysBusDevice *dev = SYS_BUS_DEVICE(obj); + + memory_region_init_io(&s->iomem, obj, &sbsa_ec_ops, s, "sbsa-ec", + 0x1000); + sysbus_init_mmio(dev, &s->iomem); +} + +static void sbsa_ec_class_init(ObjectClass *klass, void *data) +{ + DeviceClass *dc = DEVICE_CLASS(klass); + + /* No vmstate or reset required: device has no internal state */ + dc->user_creatable = false; +} + +static const TypeInfo sbsa_ec_info = { + .name = TYPE_SBSA_EC, + .parent = TYPE_SYS_BUS_DEVICE, + .instance_size = sizeof(SECUREECState), + .instance_init = sbsa_ec_init, + .class_init = sbsa_ec_class_init, +}; + +static void sbsa_ec_register_type(void) +{ + type_register_static(&sbsa_ec_info); +} + +type_init(sbsa_ec_register_type); diff --git a/hw/misc/meson.build b/hw/misc/meson.build index XXXXXXX..XXXXXXX 100644 --- a/hw/misc/meson.build +++ b/hw/misc/meson.build @@ -XXX,XX +XXX,XX @@ specific_ss.add(when: 'CONFIG_MAC_VIA', if_true: files('mac_via.c')) specific_ss.add(when: 'CONFIG_MIPS_CPS', if_true: files('mips_cmgcr.c', 'mips_cpc.c')) specific_ss.add(when: 'CONFIG_MIPS_ITU', if_true: files('mips_itu.c')) + +specific_ss.add(when: 'CONFIG_SBSA_REF', if_true: files('sbsa_ec.c')) -- 2.20.1 From: Graeme Gregory <graeme@nuviainc.com> Add the previously created sbsa-ec device to the sbsa-ref machine in secure memory so the PSCI implementation in ARM-TF can access it, but not expose it to non secure firmware or OS except by via ARM-TF. Signed-off-by: Graeme Gregory <graeme@nuviainc.com> Reviewed-by: Leif Lindholm <leif@nuviainc.com> Tested-by: Leif Lindholm <leif@nuviainc.com> Message-id: 20200826141952.136164-3-graeme@nuviainc.com Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- hw/arm/sbsa-ref.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/hw/arm/sbsa-ref.c b/hw/arm/sbsa-ref.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/sbsa-ref.c +++ b/hw/arm/sbsa-ref.c @@ -XXX,XX +XXX,XX @@ enum { SBSA_CPUPERIPHS, SBSA_GIC_DIST, SBSA_GIC_REDIST, + SBSA_SECURE_EC, SBSA_SMMU, SBSA_UART, SBSA_RTC, @@ -XXX,XX +XXX,XX @@ static const MemMapEntry sbsa_ref_memmap[] = { [SBSA_CPUPERIPHS] = { 0x40000000, 0x00040000 }, [SBSA_GIC_DIST] = { 0x40060000, 0x00010000 }, [SBSA_GIC_REDIST] = { 0x40080000, 0x04000000 }, + [SBSA_SECURE_EC] = { 0x50000000, 0x00001000 }, [SBSA_UART] = { 0x60000000, 0x00001000 }, [SBSA_RTC] = { 0x60010000, 0x00001000 }, [SBSA_GPIO] = { 0x60020000, 0x00001000 }, @@ -XXX,XX +XXX,XX @@ static void *sbsa_ref_dtb(const struct arm_boot_info *binfo, int *fdt_size) return board->fdt; } +static void create_secure_ec(MemoryRegion *mem) +{ + hwaddr base = sbsa_ref_memmap[SBSA_SECURE_EC].base; + DeviceState *dev = qdev_new("sbsa-ec"); + SysBusDevice *s = SYS_BUS_DEVICE(dev); + + memory_region_add_subregion(mem, base, + sysbus_mmio_get_region(s, 0)); +} + static void sbsa_ref_init(MachineState *machine) { unsigned int smp_cpus = machine->smp.cpus; @@ -XXX,XX +XXX,XX @@ static void sbsa_ref_init(MachineState *machine) create_pcie(sms); + create_secure_ec(secure_sysmem); + sms->bootinfo.ram_size = machine->ram_size; sms->bootinfo.nb_cpus = smp_cpus; sms->bootinfo.board_id = -1; -- 2.20.1