Hi; here's the latest batch of arm changes. The big thing in here is the SMMUv3 changes to add stage-2 translation support. thanks -- PMM The following changes since commit aa9bbd865502ed517624ab6fe7d4b5d89ca95e43: Merge tag 'pull-ppc-20230528' of https://gitlab.com/danielhb/qemu into staging (2023-05-29 14:31:52 -0700) are available in the Git repository at: https://git.linaro.org/people/pmaydell/qemu-arm.git tags/pull-target-arm-20230530 for you to fetch changes up to b03d0d4f531a8b867e0aac1fab0b876903015680: docs: sbsa: correct graphics card name (2023-05-30 13:32:46 +0100) ---------------------------------------------------------------- target-arm queue: * fsl-imx6: Add SNVS support for i.MX6 boards * smmuv3: Add support for stage 2 translations * hw/dma/xilinx_axidma: Check DMASR.HALTED to prevent infinite loop * hw/arm/xlnx-zynqmp: fix unsigned error when checking the RPUs number * cleanups for recent Kconfig changes * target/arm: Explicitly select short-format FSR for M-profile * tests/qtest: Run arm-specific tests only if the required machine is available * hw/arm/sbsa-ref: add GIC node into DT * docs: sbsa: correct graphics card name * Update copyright dates to 2023 ---------------------------------------------------------------- Clément Chigot (1): hw/arm/xlnx-zynqmp: fix unsigned error when checking the RPUs number Enze Li (1): Update copyright dates to 2023 Fabiano Rosas (3): target/arm: Explain why we need to select ARM_V7M arm/Kconfig: Keep Kconfig default entries in default.mak as documentation arm/Kconfig: Make TCG dependence explicit Marcin Juszkiewicz (2): hw/arm/sbsa-ref: add GIC node into DT docs: sbsa: correct graphics card name Mostafa Saleh (10): hw/arm/smmuv3: Add missing fields for IDR0 hw/arm/smmuv3: Update translation config to hold stage-2 hw/arm/smmuv3: Refactor stage-1 PTW hw/arm/smmuv3: Add page table walk for stage-2 hw/arm/smmuv3: Parse STE config for stage-2 hw/arm/smmuv3: Make TLB lookup work for stage-2 hw/arm/smmuv3: Add VMID to TLB tagging hw/arm/smmuv3: Add CMDs related to stage-2 hw/arm/smmuv3: Add stage-2 support in iova notifier hw/arm/smmuv3: Add knob to choose translation stage and enable stage-2 Peter Maydell (1): target/arm: Explicitly select short-format FSR for M-profile Thomas Huth (1): tests/qtest: Run arm-specific tests only if the required machine is available Tommy Wu (1): hw/dma/xilinx_axidma: Check DMASR.HALTED to prevent infinite loop. Vitaly Cheptsov (1): fsl-imx6: Add SNVS support for i.MX6 boards docs/conf.py | 2 +- docs/system/arm/sbsa.rst | 2 +- configs/devices/aarch64-softmmu/default.mak | 6 + configs/devices/arm-softmmu/default.mak | 40 ++++ hw/arm/smmu-internal.h | 37 +++ hw/arm/smmuv3-internal.h | 12 +- include/hw/arm/fsl-imx6.h | 2 + include/hw/arm/smmu-common.h | 45 +++- include/hw/arm/smmuv3.h | 4 + include/qemu/help-texts.h | 2 +- hw/arm/fsl-imx6.c | 8 + hw/arm/sbsa-ref.c | 19 +- hw/arm/smmu-common.c | 209 ++++++++++++++-- hw/arm/smmuv3.c | 357 ++++++++++++++++++++++++---- hw/arm/xlnx-zynqmp.c | 2 +- hw/dma/xilinx_axidma.c | 11 +- target/arm/tcg/tlb_helper.c | 13 +- hw/arm/Kconfig | 123 ++++++---- hw/arm/trace-events | 14 +- target/arm/Kconfig | 3 + tests/qtest/meson.build | 7 +- 21 files changed, 773 insertions(+), 145 deletions(-) From: Vitaly Cheptsov <cheptsov@ispras.ru> SNVS is supported on both i.MX6 and i.MX6UL and is needed to support shutdown on the board. Cc: Peter Maydell <peter.maydell@linaro.org> (odd fixer:SABRELITE / i.MX6) Cc: Jean-Christophe Dubois <jcd@tribudubois.net> (reviewer:SABRELITE / i.MX6) Cc: qemu-arm@nongnu.org (open list:SABRELITE / i.MX6) Cc: qemu-devel@nongnu.org (open list:All patches CC here) Signed-off-by: Vitaly Cheptsov <cheptsov@ispras.ru> Message-id: 20230515095015.66860-1-cheptsov@ispras.ru Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- include/hw/arm/fsl-imx6.h | 2 ++ hw/arm/fsl-imx6.c | 8 ++++++++ 2 files changed, 10 insertions(+) diff --git a/include/hw/arm/fsl-imx6.h b/include/hw/arm/fsl-imx6.h index XXXXXXX..XXXXXXX 100644 --- a/include/hw/arm/fsl-imx6.h +++ b/include/hw/arm/fsl-imx6.h @@ -XXX,XX +XXX,XX @@ #include "hw/cpu/a9mpcore.h" #include "hw/misc/imx6_ccm.h" #include "hw/misc/imx6_src.h" +#include "hw/misc/imx7_snvs.h" #include "hw/watchdog/wdt_imx2.h" #include "hw/char/imx_serial.h" #include "hw/timer/imx_gpt.h" @@ -XXX,XX +XXX,XX @@ struct FslIMX6State { A9MPPrivState a9mpcore; IMX6CCMState ccm; IMX6SRCState src; + IMX7SNVSState snvs; IMXSerialState uart[FSL_IMX6_NUM_UARTS]; IMXGPTState gpt; IMXEPITState epit[FSL_IMX6_NUM_EPITS]; diff --git a/hw/arm/fsl-imx6.c b/hw/arm/fsl-imx6.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/fsl-imx6.c +++ b/hw/arm/fsl-imx6.c @@ -XXX,XX +XXX,XX @@ static void fsl_imx6_init(Object *obj) object_initialize_child(obj, "src", &s->src, TYPE_IMX6_SRC); + object_initialize_child(obj, "snvs", &s->snvs, TYPE_IMX7_SNVS); + for (i = 0; i < FSL_IMX6_NUM_UARTS; i++) { snprintf(name, NAME_SIZE, "uart%d", i + 1); object_initialize_child(obj, name, &s->uart[i], TYPE_IMX_SERIAL); @@ -XXX,XX +XXX,XX @@ static void fsl_imx6_realize(DeviceState *dev, Error **errp) qdev_get_gpio_in(DEVICE(&s->a9mpcore), FSL_IMX6_ENET_MAC_1588_IRQ)); + /* + * SNVS + */ + sysbus_realize(SYS_BUS_DEVICE(&s->snvs), &error_abort); + sysbus_mmio_map(SYS_BUS_DEVICE(&s->snvs), 0, FSL_IMX6_SNVSHP_ADDR); + /* * Watchdog */ -- 2.34.1 From: Mostafa Saleh <smostafa@google.com> In preparation for adding stage-2 support. Add IDR0 fields related to stage-2. VMID16: 16-bit VMID supported. S2P: Stage-2 translation supported. They are described in 6.3.1 SMMU_IDR0. No functional change intended. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Eric Auger <eric.auger@redhat.com> Signed-off-by: Mostafa Saleh <smostafa@google.com> Tested-by: Eric Auger <eric.auger@redhat.com> Tested-by: Jean-Philippe Brucker <jean-philippe@linaro.org> Message-id: 20230516203327.2051088-2-smostafa@google.com Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- hw/arm/smmuv3-internal.h | 2 ++ 1 file changed, 2 insertions(+) diff --git a/hw/arm/smmuv3-internal.h b/hw/arm/smmuv3-internal.h index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/smmuv3-internal.h +++ b/hw/arm/smmuv3-internal.h @@ -XXX,XX +XXX,XX @@ typedef enum SMMUTranslationStatus { /* MMIO Registers */ REG32(IDR0, 0x0) + FIELD(IDR0, S2P, 0 , 1) FIELD(IDR0, S1P, 1 , 1) FIELD(IDR0, TTF, 2 , 2) FIELD(IDR0, COHACC, 4 , 1) FIELD(IDR0, ASID16, 12, 1) + FIELD(IDR0, VMID16, 18, 1) FIELD(IDR0, TTENDIAN, 21, 2) FIELD(IDR0, STALL_MODEL, 24, 2) FIELD(IDR0, TERM_MODEL, 26, 1) -- 2.34.1 From: Mostafa Saleh <smostafa@google.com> In preparation for adding stage-2 support, add a S2 config struct(SMMUS2Cfg), composed of the following fields and embedded in the main SMMUTransCfg: -tsz: Size of IPA input region (S2T0SZ) -sl0: Start level of translation (S2SL0) -affd: AF Fault Disable (S2AFFD) -record_faults: Record fault events (S2R) -granule_sz: Granule page shift (based on S2TG) -vmid: Virtual Machine ID (S2VMID) -vttb: Address of translation table base (S2TTB) -eff_ps: Effective PA output range (based on S2PS) They will be used in the next patches in stage-2 address translation. The fields in SMMUS2Cfg, are reordered to make the shared and stage-1 fields next to each other, this reordering didn't change the struct size (104 bytes before and after). Stage-1 only fields: aa64, asid, tt, ttb, tbi, record_faults, oas. oas is stage-1 output address size. However, it is used to check input address in case stage-1 is unimplemented or bypassed according to SMMUv3 manual IHI0070.E "3.4. Address sizes" Shared fields: stage, disabled, bypassed, aborted, iotlb_*. No functional change intended. Reviewed-by: Eric Auger <eric.auger@redhat.com> Signed-off-by: Mostafa Saleh <smostafa@google.com> Tested-by: Eric Auger <eric.auger@redhat.com> Tested-by: Jean-Philippe Brucker <jean-philippe@linaro.org> Message-id: 20230516203327.2051088-3-smostafa@google.com Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- include/hw/arm/smmu-common.h | 22 +++++++++++++++++++--- 1 file changed, 19 insertions(+), 3 deletions(-) diff --git a/include/hw/arm/smmu-common.h b/include/hw/arm/smmu-common.h index XXXXXXX..XXXXXXX 100644 --- a/include/hw/arm/smmu-common.h +++ b/include/hw/arm/smmu-common.h @@ -XXX,XX +XXX,XX @@ typedef struct SMMUTLBEntry { uint8_t granule; } SMMUTLBEntry; +/* Stage-2 configuration. */ +typedef struct SMMUS2Cfg { + uint8_t tsz; /* Size of IPA input region (S2T0SZ) */ + uint8_t sl0; /* Start level of translation (S2SL0) */ + bool affd; /* AF Fault Disable (S2AFFD) */ + bool record_faults; /* Record fault events (S2R) */ + uint8_t granule_sz; /* Granule page shift (based on S2TG) */ + uint8_t eff_ps; /* Effective PA output range (based on S2PS) */ + uint16_t vmid; /* Virtual Machine ID (S2VMID) */ + uint64_t vttb; /* Address of translation table base (S2TTB) */ +} SMMUS2Cfg; + /* * Generic structure populated by derived SMMU devices * after decoding the configuration information and used as * input to the page table walk */ typedef struct SMMUTransCfg { + /* Shared fields between stage-1 and stage-2. */ int stage; /* translation stage */ - bool aa64; /* arch64 or aarch32 translation table */ bool disabled; /* smmu is disabled */ bool bypassed; /* translation is bypassed */ bool aborted; /* translation is aborted */ + uint32_t iotlb_hits; /* counts IOTLB hits */ + uint32_t iotlb_misses; /* counts IOTLB misses*/ + /* Used by stage-1 only. */ + bool aa64; /* arch64 or aarch32 translation table */ bool record_faults; /* record fault events */ uint64_t ttb; /* TT base address */ uint8_t oas; /* output address width */ uint8_t tbi; /* Top Byte Ignore */ uint16_t asid; SMMUTransTableInfo tt[2]; - uint32_t iotlb_hits; /* counts IOTLB hits for this asid */ - uint32_t iotlb_misses; /* counts IOTLB misses for this asid */ + /* Used by stage-2 only. */ + struct SMMUS2Cfg s2cfg; } SMMUTransCfg; typedef struct SMMUDevice { -- 2.34.1 From: Mostafa Saleh <smostafa@google.com> In preparation for adding stage-2 support, rename smmu_ptw_64 to smmu_ptw_64_s1 and refactor some of the code so it can be reused in stage-2 page table walk. Remove AA64 check from PTW as decode_cd already ensures that AA64 is used, otherwise it faults with C_BAD_CD. A stage member is added to SMMUPTWEventInfo to differentiate between stage-1 and stage-2 ptw faults. Add stage argument to trace_smmu_ptw_level be consistent with other trace events. Signed-off-by: Mostafa Saleh <smostafa@google.com> Reviewed-by: Eric Auger <eric.auger@redhat.com> Tested-by: Eric Auger <eric.auger@redhat.com> Tested-by: Jean-Philippe Brucker <jean-philippe@linaro.org> Message-id: 20230516203327.2051088-4-smostafa@google.com Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- include/hw/arm/smmu-common.h | 16 +++++++++++++--- hw/arm/smmu-common.c | 27 ++++++++++----------------- hw/arm/smmuv3.c | 2 ++ hw/arm/trace-events | 2 +- 4 files changed, 26 insertions(+), 21 deletions(-) diff --git a/include/hw/arm/smmu-common.h b/include/hw/arm/smmu-common.h index XXXXXXX..XXXXXXX 100644 --- a/include/hw/arm/smmu-common.h +++ b/include/hw/arm/smmu-common.h @@ -XXX,XX +XXX,XX @@ #include "hw/pci/pci.h" #include "qom/object.h" -#define SMMU_PCI_BUS_MAX 256 -#define SMMU_PCI_DEVFN_MAX 256 -#define SMMU_PCI_DEVFN(sid) (sid & 0xFF) +#define SMMU_PCI_BUS_MAX 256 +#define SMMU_PCI_DEVFN_MAX 256 +#define SMMU_PCI_DEVFN(sid) (sid & 0xFF) + +/* VMSAv8-64 Translation constants and functions */ +#define VMSA_LEVELS 4 + +#define VMSA_STRIDE(gran) ((gran) - VMSA_LEVELS + 1) +#define VMSA_BIT_LVL(isz, strd, lvl) ((isz) - (strd) * \ + (VMSA_LEVELS - (lvl))) +#define VMSA_IDXMSK(isz, strd, lvl) ((1ULL << \ + VMSA_BIT_LVL(isz, strd, lvl)) - 1) /* * Page table walk error types @@ -XXX,XX +XXX,XX @@ typedef enum { } SMMUPTWEventType; typedef struct SMMUPTWEventInfo { + int stage; SMMUPTWEventType type; dma_addr_t addr; /* fetched address that induced an abort, if any */ } SMMUPTWEventInfo; diff --git a/hw/arm/smmu-common.c b/hw/arm/smmu-common.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/smmu-common.c +++ b/hw/arm/smmu-common.c @@ -XXX,XX +XXX,XX @@ SMMUTransTableInfo *select_tt(SMMUTransCfg *cfg, dma_addr_t iova) } /** - * smmu_ptw_64 - VMSAv8-64 Walk of the page tables for a given IOVA + * smmu_ptw_64_s1 - VMSAv8-64 Walk of the page tables for a given IOVA * @cfg: translation config * @iova: iova to translate * @perm: access type @@ -XXX,XX +XXX,XX @@ SMMUTransTableInfo *select_tt(SMMUTransCfg *cfg, dma_addr_t iova) * Upon success, @tlbe is filled with translated_addr and entry * permission rights. */ -static int smmu_ptw_64(SMMUTransCfg *cfg, - dma_addr_t iova, IOMMUAccessFlags perm, - SMMUTLBEntry *tlbe, SMMUPTWEventInfo *info) +static int smmu_ptw_64_s1(SMMUTransCfg *cfg, + dma_addr_t iova, IOMMUAccessFlags perm, + SMMUTLBEntry *tlbe, SMMUPTWEventInfo *info) { dma_addr_t baseaddr, indexmask; int stage = cfg->stage; @@ -XXX,XX +XXX,XX @@ static int smmu_ptw_64(SMMUTransCfg *cfg, } granule_sz = tt->granule_sz; - stride = granule_sz - 3; + stride = VMSA_STRIDE(granule_sz); inputsize = 64 - tt->tsz; level = 4 - (inputsize - 4) / stride; - indexmask = (1ULL << (inputsize - (stride * (4 - level)))) - 1; + indexmask = VMSA_IDXMSK(inputsize, stride, level); baseaddr = extract64(tt->ttb, 0, 48); baseaddr &= ~indexmask; - while (level <= 3) { + while (level < VMSA_LEVELS) { uint64_t subpage_size = 1ULL << level_shift(level, granule_sz); uint64_t mask = subpage_size - 1; uint32_t offset = iova_level_offset(iova, inputsize, level, granule_sz); @@ -XXX,XX +XXX,XX @@ static int smmu_ptw_64(SMMUTransCfg *cfg, if (get_pte(baseaddr, offset, &pte, info)) { goto error; } - trace_smmu_ptw_level(level, iova, subpage_size, + trace_smmu_ptw_level(stage, level, iova, subpage_size, baseaddr, offset, pte); if (is_invalid_pte(pte) || is_reserved_pte(pte, level)) { @@ -XXX,XX +XXX,XX @@ static int smmu_ptw_64(SMMUTransCfg *cfg, info->type = SMMU_PTW_ERR_TRANSLATION; error: + info->stage = 1; tlbe->entry.perm = IOMMU_NONE; return -EINVAL; } @@ -XXX,XX +XXX,XX @@ error: int smmu_ptw(SMMUTransCfg *cfg, dma_addr_t iova, IOMMUAccessFlags perm, SMMUTLBEntry *tlbe, SMMUPTWEventInfo *info) { - if (!cfg->aa64) { - /* - * This code path is not entered as we check this while decoding - * the configuration data in the derived SMMU model. - */ - g_assert_not_reached(); - } - - return smmu_ptw_64(cfg, iova, perm, tlbe, info); + return smmu_ptw_64_s1(cfg, iova, perm, tlbe, info); } /** diff --git a/hw/arm/smmuv3.c b/hw/arm/smmuv3.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/smmuv3.c +++ b/hw/arm/smmuv3.c @@ -XXX,XX +XXX,XX @@ static IOMMUTLBEntry smmuv3_translate(IOMMUMemoryRegion *mr, hwaddr addr, cached_entry = g_new0(SMMUTLBEntry, 1); if (smmu_ptw(cfg, aligned_addr, flag, cached_entry, &ptw_info)) { + /* All faults from PTW has S2 field. */ + event.u.f_walk_eabt.s2 = (ptw_info.stage == 2); g_free(cached_entry); switch (ptw_info.type) { case SMMU_PTW_ERR_WALK_EABT: diff --git a/hw/arm/trace-events b/hw/arm/trace-events index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/trace-events +++ b/hw/arm/trace-events @@ -XXX,XX +XXX,XX @@ virt_acpi_setup(void) "No fw cfg or ACPI disabled. Bailing out." # smmu-common.c smmu_add_mr(const char *name) "%s" -smmu_ptw_level(int level, uint64_t iova, size_t subpage_size, uint64_t baseaddr, uint32_t offset, uint64_t pte) "level=%d iova=0x%"PRIx64" subpage_sz=0x%zx baseaddr=0x%"PRIx64" offset=%d => pte=0x%"PRIx64 +smmu_ptw_level(int stage, int level, uint64_t iova, size_t subpage_size, uint64_t baseaddr, uint32_t offset, uint64_t pte) "stage=%d level=%d iova=0x%"PRIx64" subpage_sz=0x%zx baseaddr=0x%"PRIx64" offset=%d => pte=0x%"PRIx64 smmu_ptw_invalid_pte(int stage, int level, uint64_t baseaddr, uint64_t pteaddr, uint32_t offset, uint64_t pte) "stage=%d level=%d base@=0x%"PRIx64" pte@=0x%"PRIx64" offset=%d pte=0x%"PRIx64 smmu_ptw_page_pte(int stage, int level, uint64_t iova, uint64_t baseaddr, uint64_t pteaddr, uint64_t pte, uint64_t address) "stage=%d level=%d iova=0x%"PRIx64" base@=0x%"PRIx64" pte@=0x%"PRIx64" pte=0x%"PRIx64" page address = 0x%"PRIx64 smmu_ptw_block_pte(int stage, int level, uint64_t baseaddr, uint64_t pteaddr, uint64_t pte, uint64_t iova, uint64_t gpa, int bsize_mb) "stage=%d level=%d base@=0x%"PRIx64" pte@=0x%"PRIx64" pte=0x%"PRIx64" iova=0x%"PRIx64" block address = 0x%"PRIx64" block size = %d MiB" -- 2.34.1 From: Mostafa Saleh <smostafa@google.com> In preparation for adding stage-2 support, add Stage-2 PTW code. Only Aarch64 format is supported as stage-1. Nesting stage-1 and stage-2 is not supported right now. HTTU is not supported, SW is expected to maintain the Access flag. This is described in the SMMUv3 manual(IHI 0070.E.a) "5.2. Stream Table Entry" in "[181] S2AFFD". This flag determines the behavior on access of a stage-2 page whose descriptor has AF == 0: - 0b0: An Access flag fault occurs (stall not supported). - 0b1: An Access flag fault never occurs. An Access fault takes priority over a Permission fault. There are 3 address size checks for stage-2 according to (IHI 0070.E.a) in "3.4. Address sizes". - As nesting is not supported, input address is passed directly to stage-2, and is checked against IAS. We use cfg->oas to hold the OAS when stage-1 is not used, this is set in the next patch. This check is done outside of smmu_ptw_64_s2 as it is not part of stage-2(it throws stage-1 fault), and the stage-2 function shouldn't change it's behavior when nesting is supported. When nesting is supported and we figure out how to combine TLB for stage-1 and stage-2 we can move this check into the stage-1 function as described in ARM DDI0487I.a in pseudocode aarch64/translation/vmsa_translation/AArch64.S1Translate aarch64/translation/vmsa_translation/AArch64.S1DisabledOutput - Input to stage-2 is checked against s2t0sz, and throws stage-2 transaltion fault if exceeds it. - Output of stage-2 is checked against effective PA output range. Reviewed-by: Eric Auger <eric.auger@redhat.com> Signed-off-by: Mostafa Saleh <smostafa@google.com> Tested-by: Eric Auger <eric.auger@redhat.com> Tested-by: Jean-Philippe Brucker <jean-philippe@linaro.org> Message-id: 20230516203327.2051088-5-smostafa@google.com Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- hw/arm/smmu-internal.h | 35 ++++++++++ hw/arm/smmu-common.c | 142 ++++++++++++++++++++++++++++++++++++++++- 2 files changed, 176 insertions(+), 1 deletion(-) diff --git a/hw/arm/smmu-internal.h b/hw/arm/smmu-internal.h index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/smmu-internal.h +++ b/hw/arm/smmu-internal.h @@ -XXX,XX +XXX,XX @@ #define PTE_APTABLE(pte) \ (extract64(pte, 61, 2)) +#define PTE_AF(pte) \ + (extract64(pte, 10, 1)) /* * TODO: At the moment all transactions are considered as privileged (EL1) * as IOMMU translation callback does not pass user/priv attributes. @@ -XXX,XX +XXX,XX @@ #define is_permission_fault(ap, perm) \ (((perm) & IOMMU_WO) && ((ap) & 0x2)) +#define is_permission_fault_s2(s2ap, perm) \ + (!(((s2ap) & (perm)) == (perm))) + #define PTE_AP_TO_PERM(ap) \ (IOMMU_ACCESS_FLAG(true, !((ap) & 0x2))) @@ -XXX,XX +XXX,XX @@ uint64_t iova_level_offset(uint64_t iova, int inputsize, MAKE_64BIT_MASK(0, gsz - 3); } +/* FEAT_LPA2 and FEAT_TTST are not implemented. */ +static inline int get_start_level(int sl0 , int granule_sz) +{ + /* ARM DDI0487I.a: Table D8-12. */ + if (granule_sz == 12) { + return 2 - sl0; + } + /* ARM DDI0487I.a: Table D8-22 and Table D8-31. */ + return 3 - sl0; +} + +/* + * Index in a concatenated first level stage-2 page table. + * ARM DDI0487I.a: D8.2.2 Concatenated translation tables. + */ +static inline int pgd_concat_idx(int start_level, int granule_sz, + dma_addr_t ipa) +{ + uint64_t ret; + /* + * Get the number of bits handled by next levels, then any extra bits in + * the address should index the concatenated tables. This relation can be + * deduced from tables in ARM DDI0487I.a: D8.2.7-9 + */ + int shift = level_shift(start_level - 1, granule_sz); + + ret = ipa >> shift; + return ret; +} + #define SMMU_IOTLB_ASID(key) ((key).asid) typedef struct SMMUIOTLBPageInvInfo { diff --git a/hw/arm/smmu-common.c b/hw/arm/smmu-common.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/smmu-common.c +++ b/hw/arm/smmu-common.c @@ -XXX,XX +XXX,XX @@ error: return -EINVAL; } +/** + * smmu_ptw_64_s2 - VMSAv8-64 Walk of the page tables for a given ipa + * for stage-2. + * @cfg: translation config + * @ipa: ipa to translate + * @perm: access type + * @tlbe: SMMUTLBEntry (out) + * @info: handle to an error info + * + * Return 0 on success, < 0 on error. In case of error, @info is filled + * and tlbe->perm is set to IOMMU_NONE. + * Upon success, @tlbe is filled with translated_addr and entry + * permission rights. + */ +static int smmu_ptw_64_s2(SMMUTransCfg *cfg, + dma_addr_t ipa, IOMMUAccessFlags perm, + SMMUTLBEntry *tlbe, SMMUPTWEventInfo *info) +{ + const int stage = 2; + int granule_sz = cfg->s2cfg.granule_sz; + /* ARM DDI0487I.a: Table D8-7. */ + int inputsize = 64 - cfg->s2cfg.tsz; + int level = get_start_level(cfg->s2cfg.sl0, granule_sz); + int stride = VMSA_STRIDE(granule_sz); + int idx = pgd_concat_idx(level, granule_sz, ipa); + /* + * Get the ttb from concatenated structure. + * The offset is the idx * size of each ttb(number of ptes * (sizeof(pte)) + */ + uint64_t baseaddr = extract64(cfg->s2cfg.vttb, 0, 48) + (1 << stride) * + idx * sizeof(uint64_t); + dma_addr_t indexmask = VMSA_IDXMSK(inputsize, stride, level); + + baseaddr &= ~indexmask; + + /* + * On input, a stage 2 Translation fault occurs if the IPA is outside the + * range configured by the relevant S2T0SZ field of the STE. + */ + if (ipa >= (1ULL << inputsize)) { + info->type = SMMU_PTW_ERR_TRANSLATION; + goto error; + } + + while (level < VMSA_LEVELS) { + uint64_t subpage_size = 1ULL << level_shift(level, granule_sz); + uint64_t mask = subpage_size - 1; + uint32_t offset = iova_level_offset(ipa, inputsize, level, granule_sz); + uint64_t pte, gpa; + dma_addr_t pte_addr = baseaddr + offset * sizeof(pte); + uint8_t s2ap; + + if (get_pte(baseaddr, offset, &pte, info)) { + goto error; + } + trace_smmu_ptw_level(stage, level, ipa, subpage_size, + baseaddr, offset, pte); + if (is_invalid_pte(pte) || is_reserved_pte(pte, level)) { + trace_smmu_ptw_invalid_pte(stage, level, baseaddr, + pte_addr, offset, pte); + break; + } + + if (is_table_pte(pte, level)) { + baseaddr = get_table_pte_address(pte, granule_sz); + level++; + continue; + } else if (is_page_pte(pte, level)) { + gpa = get_page_pte_address(pte, granule_sz); + trace_smmu_ptw_page_pte(stage, level, ipa, + baseaddr, pte_addr, pte, gpa); + } else { + uint64_t block_size; + + gpa = get_block_pte_address(pte, level, granule_sz, + &block_size); + trace_smmu_ptw_block_pte(stage, level, baseaddr, + pte_addr, pte, ipa, gpa, + block_size >> 20); + } + + /* + * If S2AFFD and PTE.AF are 0 => fault. (5.2. Stream Table Entry) + * An Access fault takes priority over a Permission fault. + */ + if (!PTE_AF(pte) && !cfg->s2cfg.affd) { + info->type = SMMU_PTW_ERR_ACCESS; + goto error; + } + + s2ap = PTE_AP(pte); + if (is_permission_fault_s2(s2ap, perm)) { + info->type = SMMU_PTW_ERR_PERMISSION; + goto error; + } + + /* + * The address output from the translation causes a stage 2 Address + * Size fault if it exceeds the effective PA output range. + */ + if (gpa >= (1ULL << cfg->s2cfg.eff_ps)) { + info->type = SMMU_PTW_ERR_ADDR_SIZE; + goto error; + } + + tlbe->entry.translated_addr = gpa; + tlbe->entry.iova = ipa & ~mask; + tlbe->entry.addr_mask = mask; + tlbe->entry.perm = s2ap; + tlbe->level = level; + tlbe->granule = granule_sz; + return 0; + } + info->type = SMMU_PTW_ERR_TRANSLATION; + +error: + info->stage = 2; + tlbe->entry.perm = IOMMU_NONE; + return -EINVAL; +} + /** * smmu_ptw - Walk the page tables for an IOVA, according to @cfg * @@ -XXX,XX +XXX,XX @@ error: int smmu_ptw(SMMUTransCfg *cfg, dma_addr_t iova, IOMMUAccessFlags perm, SMMUTLBEntry *tlbe, SMMUPTWEventInfo *info) { - return smmu_ptw_64_s1(cfg, iova, perm, tlbe, info); + if (cfg->stage == 1) { + return smmu_ptw_64_s1(cfg, iova, perm, tlbe, info); + } else if (cfg->stage == 2) { + /* + * If bypassing stage 1(or unimplemented), the input address is passed + * directly to stage 2 as IPA. If the input address of a transaction + * exceeds the size of the IAS, a stage 1 Address Size fault occurs. + * For AA64, IAS = OAS according to (IHI 0070.E.a) "3.4 Address sizes" + */ + if (iova >= (1ULL << cfg->oas)) { + info->type = SMMU_PTW_ERR_ADDR_SIZE; + info->stage = 1; + tlbe->entry.perm = IOMMU_NONE; + return -EINVAL; + } + + return smmu_ptw_64_s2(cfg, iova, perm, tlbe, info); + } + + g_assert_not_reached(); } /** -- 2.34.1 From: Mostafa Saleh <smostafa@google.com> Parse stage-2 configuration from STE and populate it in SMMUS2Cfg. Validity of field values are checked when possible. Only AA64 tables are supported and Small Translation Tables (STT) are not supported. According to SMMUv3 UM(IHI0070E) "5.2 Stream Table Entry": All fields with an S2 prefix (with the exception of S2VMID) are IGNORED when stage-2 bypasses translation (Config[1] == 0). Which means that VMID can be used(for TLB tagging) even if stage-2 is bypassed, so we parse it unconditionally when S2P exists. Otherwise it is set to -1.(only S1P) As stall is not supported, if S2S is set the translation would abort. For S2R, we reuse the same code used for stage-1 with flag record_faults. However when nested translation is supported we would need to separate stage-1 and stage-2 faults. Fix wrong shift in STE_S2HD, STE_S2HA, STE_S2S. Signed-off-by: Mostafa Saleh <smostafa@google.com> Tested-by: Eric Auger <eric.auger@redhat.com> Tested-by: Jean-Philippe Brucker <jean-philippe@linaro.org> Reviewed-by: Eric Auger <eric.auger@redhat.com> Message-id: 20230516203327.2051088-6-smostafa@google.com Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- hw/arm/smmuv3-internal.h | 10 +- include/hw/arm/smmu-common.h | 1 + include/hw/arm/smmuv3.h | 3 + hw/arm/smmuv3.c | 181 +++++++++++++++++++++++++++++++++-- 4 files changed, 185 insertions(+), 10 deletions(-) diff --git a/hw/arm/smmuv3-internal.h b/hw/arm/smmuv3-internal.h index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/smmuv3-internal.h +++ b/hw/arm/smmuv3-internal.h @@ -XXX,XX +XXX,XX @@ typedef struct CD { #define STE_S2TG(x) extract32((x)->word[5], 14, 2) #define STE_S2PS(x) extract32((x)->word[5], 16, 3) #define STE_S2AA64(x) extract32((x)->word[5], 19, 1) -#define STE_S2HD(x) extract32((x)->word[5], 24, 1) -#define STE_S2HA(x) extract32((x)->word[5], 25, 1) -#define STE_S2S(x) extract32((x)->word[5], 26, 1) +#define STE_S2ENDI(x) extract32((x)->word[5], 20, 1) +#define STE_S2AFFD(x) extract32((x)->word[5], 21, 1) +#define STE_S2HD(x) extract32((x)->word[5], 23, 1) +#define STE_S2HA(x) extract32((x)->word[5], 24, 1) +#define STE_S2S(x) extract32((x)->word[5], 25, 1) +#define STE_S2R(x) extract32((x)->word[5], 26, 1) + #define STE_CTXPTR(x) \ ({ \ unsigned long addr; \ diff --git a/include/hw/arm/smmu-common.h b/include/hw/arm/smmu-common.h index XXXXXXX..XXXXXXX 100644 --- a/include/hw/arm/smmu-common.h +++ b/include/hw/arm/smmu-common.h @@ -XXX,XX +XXX,XX @@ /* VMSAv8-64 Translation constants and functions */ #define VMSA_LEVELS 4 +#define VMSA_MAX_S2_CONCAT 16 #define VMSA_STRIDE(gran) ((gran) - VMSA_LEVELS + 1) #define VMSA_BIT_LVL(isz, strd, lvl) ((isz) - (strd) * \ diff --git a/include/hw/arm/smmuv3.h b/include/hw/arm/smmuv3.h index XXXXXXX..XXXXXXX 100644 --- a/include/hw/arm/smmuv3.h +++ b/include/hw/arm/smmuv3.h @@ -XXX,XX +XXX,XX @@ struct SMMUv3Class { #define TYPE_ARM_SMMUV3 "arm-smmuv3" OBJECT_DECLARE_TYPE(SMMUv3State, SMMUv3Class, ARM_SMMUV3) +#define STAGE1_SUPPORTED(s) FIELD_EX32(s->idr[0], IDR0, S1P) +#define STAGE2_SUPPORTED(s) FIELD_EX32(s->idr[0], IDR0, S2P) + #endif diff --git a/hw/arm/smmuv3.c b/hw/arm/smmuv3.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/smmuv3.c +++ b/hw/arm/smmuv3.c @@ -XXX,XX +XXX,XX @@ #include "smmuv3-internal.h" #include "smmu-internal.h" +#define PTW_RECORD_FAULT(cfg) (((cfg)->stage == 1) ? (cfg)->record_faults : \ + (cfg)->s2cfg.record_faults) + /** * smmuv3_trigger_irq - pulse @irq if enabled and update * GERROR register in case of GERROR interrupt @@ -XXX,XX +XXX,XX @@ static int smmu_get_cd(SMMUv3State *s, STE *ste, uint32_t ssid, return 0; } +/* + * Max valid value is 39 when SMMU_IDR3.STT == 0. + * In architectures after SMMUv3.0: + * - If STE.S2TG selects a 4KB or 16KB granule, the minimum valid value for this + * field is MAX(16, 64-IAS) + * - If STE.S2TG selects a 64KB granule, the minimum valid value for this field + * is (64-IAS). + * As we only support AA64, IAS = OAS. + */ +static bool s2t0sz_valid(SMMUTransCfg *cfg) +{ + if (cfg->s2cfg.tsz > 39) { + return false; + } + + if (cfg->s2cfg.granule_sz == 16) { + return (cfg->s2cfg.tsz >= 64 - oas2bits(SMMU_IDR5_OAS)); + } + + return (cfg->s2cfg.tsz >= MAX(64 - oas2bits(SMMU_IDR5_OAS), 16)); +} + +/* + * Return true if s2 page table config is valid. + * This checks with the configured start level, ias_bits and granularity we can + * have a valid page table as described in ARM ARM D8.2 Translation process. + * The idea here is to see for the highest possible number of IPA bits, how + * many concatenated tables we would need, if it is more than 16, then this is + * not possible. + */ +static bool s2_pgtable_config_valid(uint8_t sl0, uint8_t t0sz, uint8_t gran) +{ + int level = get_start_level(sl0, gran); + uint64_t ipa_bits = 64 - t0sz; + uint64_t max_ipa = (1ULL << ipa_bits) - 1; + int nr_concat = pgd_concat_idx(level, gran, max_ipa) + 1; + + return nr_concat <= VMSA_MAX_S2_CONCAT; +} + +static int decode_ste_s2_cfg(SMMUTransCfg *cfg, STE *ste) +{ + cfg->stage = 2; + + if (STE_S2AA64(ste) == 0x0) { + qemu_log_mask(LOG_UNIMP, + "SMMUv3 AArch32 tables not supported\n"); + g_assert_not_reached(); + } + + switch (STE_S2TG(ste)) { + case 0x0: /* 4KB */ + cfg->s2cfg.granule_sz = 12; + break; + case 0x1: /* 64KB */ + cfg->s2cfg.granule_sz = 16; + break; + case 0x2: /* 16KB */ + cfg->s2cfg.granule_sz = 14; + break; + default: + qemu_log_mask(LOG_GUEST_ERROR, + "SMMUv3 bad STE S2TG: %x\n", STE_S2TG(ste)); + goto bad_ste; + } + + cfg->s2cfg.vttb = STE_S2TTB(ste); + + cfg->s2cfg.sl0 = STE_S2SL0(ste); + /* FEAT_TTST not supported. */ + if (cfg->s2cfg.sl0 == 0x3) { + qemu_log_mask(LOG_UNIMP, "SMMUv3 S2SL0 = 0x3 has no meaning!\n"); + goto bad_ste; + } + + /* For AA64, The effective S2PS size is capped to the OAS. */ + cfg->s2cfg.eff_ps = oas2bits(MIN(STE_S2PS(ste), SMMU_IDR5_OAS)); + /* + * It is ILLEGAL for the address in S2TTB to be outside the range + * described by the effective S2PS value. + */ + if (cfg->s2cfg.vttb & ~(MAKE_64BIT_MASK(0, cfg->s2cfg.eff_ps))) { + qemu_log_mask(LOG_GUEST_ERROR, + "SMMUv3 S2TTB too large 0x%lx, effective PS %d bits\n", + cfg->s2cfg.vttb, cfg->s2cfg.eff_ps); + goto bad_ste; + } + + cfg->s2cfg.tsz = STE_S2T0SZ(ste); + + if (!s2t0sz_valid(cfg)) { + qemu_log_mask(LOG_GUEST_ERROR, "SMMUv3 bad STE S2T0SZ = %d\n", + cfg->s2cfg.tsz); + goto bad_ste; + } + + if (!s2_pgtable_config_valid(cfg->s2cfg.sl0, cfg->s2cfg.tsz, + cfg->s2cfg.granule_sz)) { + qemu_log_mask(LOG_GUEST_ERROR, + "SMMUv3 STE stage 2 config not valid!\n"); + goto bad_ste; + } + + /* Only LE supported(IDR0.TTENDIAN). */ + if (STE_S2ENDI(ste)) { + qemu_log_mask(LOG_GUEST_ERROR, + "SMMUv3 STE_S2ENDI only supports LE!\n"); + goto bad_ste; + } + + cfg->s2cfg.affd = STE_S2AFFD(ste); + + cfg->s2cfg.record_faults = STE_S2R(ste); + /* As stall is not supported. */ + if (STE_S2S(ste)) { + qemu_log_mask(LOG_UNIMP, "SMMUv3 Stall not implemented!\n"); + goto bad_ste; + } + + /* This is still here as stage 2 has not been fully enabled yet. */ + qemu_log_mask(LOG_UNIMP, "SMMUv3 does not support stage 2 yet\n"); + goto bad_ste; + + return 0; + +bad_ste: + return -EINVAL; +} + /* Returns < 0 in case of invalid STE, 0 otherwise */ static int decode_ste(SMMUv3State *s, SMMUTransCfg *cfg, STE *ste, SMMUEventInfo *event) { uint32_t config; + int ret; if (!STE_VALID(ste)) { if (!event->inval_ste_allowed) { @@ -XXX,XX +XXX,XX @@ static int decode_ste(SMMUv3State *s, SMMUTransCfg *cfg, return 0; } - if (STE_CFG_S2_ENABLED(config)) { - qemu_log_mask(LOG_UNIMP, "SMMUv3 does not support stage 2 yet\n"); + /* + * If a stage is enabled in SW while not advertised, throw bad ste + * according to user manual(IHI0070E) "5.2 Stream Table Entry". + */ + if (!STAGE1_SUPPORTED(s) && STE_CFG_S1_ENABLED(config)) { + qemu_log_mask(LOG_GUEST_ERROR, "SMMUv3 S1 used but not supported.\n"); goto bad_ste; } + if (!STAGE2_SUPPORTED(s) && STE_CFG_S2_ENABLED(config)) { + qemu_log_mask(LOG_GUEST_ERROR, "SMMUv3 S2 used but not supported.\n"); + goto bad_ste; + } + + if (STAGE2_SUPPORTED(s)) { + /* VMID is considered even if s2 is disabled. */ + cfg->s2cfg.vmid = STE_S2VMID(ste); + } else { + /* Default to -1 */ + cfg->s2cfg.vmid = -1; + } + + if (STE_CFG_S2_ENABLED(config)) { + /* + * Stage-1 OAS defaults to OAS even if not enabled as it would be used + * in input address check for stage-2. + */ + cfg->oas = oas2bits(SMMU_IDR5_OAS); + ret = decode_ste_s2_cfg(cfg, ste); + if (ret) { + goto bad_ste; + } + } if (STE_S1CDMAX(ste) != 0) { qemu_log_mask(LOG_UNIMP, @@ -XXX,XX +XXX,XX @@ static IOMMUTLBEntry smmuv3_translate(IOMMUMemoryRegion *mr, hwaddr addr, if (cached_entry) { if ((flag & IOMMU_WO) && !(cached_entry->entry.perm & IOMMU_WO)) { status = SMMU_TRANS_ERROR; - if (cfg->record_faults) { + /* + * We know that the TLB only contains either stage-1 or stage-2 as + * nesting is not supported. So it is sufficient to check the + * translation stage to know the TLB stage for now. + */ + event.u.f_walk_eabt.s2 = (cfg->stage == 2); + if (PTW_RECORD_FAULT(cfg)) { event.type = SMMU_EVT_F_PERMISSION; event.u.f_permission.addr = addr; event.u.f_permission.rnw = flag & 0x1; @@ -XXX,XX +XXX,XX @@ static IOMMUTLBEntry smmuv3_translate(IOMMUMemoryRegion *mr, hwaddr addr, event.u.f_walk_eabt.addr2 = ptw_info.addr; break; case SMMU_PTW_ERR_TRANSLATION: - if (cfg->record_faults) { + if (PTW_RECORD_FAULT(cfg)) { event.type = SMMU_EVT_F_TRANSLATION; event.u.f_translation.addr = addr; event.u.f_translation.rnw = flag & 0x1; } break; case SMMU_PTW_ERR_ADDR_SIZE: - if (cfg->record_faults) { + if (PTW_RECORD_FAULT(cfg)) { event.type = SMMU_EVT_F_ADDR_SIZE; event.u.f_addr_size.addr = addr; event.u.f_addr_size.rnw = flag & 0x1; } break; case SMMU_PTW_ERR_ACCESS: - if (cfg->record_faults) { + if (PTW_RECORD_FAULT(cfg)) { event.type = SMMU_EVT_F_ACCESS; event.u.f_access.addr = addr; event.u.f_access.rnw = flag & 0x1; } break; case SMMU_PTW_ERR_PERMISSION: - if (cfg->record_faults) { + if (PTW_RECORD_FAULT(cfg)) { event.type = SMMU_EVT_F_PERMISSION; event.u.f_permission.addr = addr; event.u.f_permission.rnw = flag & 0x1; -- 2.34.1 From: Mostafa Saleh <smostafa@google.com> Right now, either stage-1 or stage-2 are supported, this simplifies how we can deal with TLBs. This patch makes TLB lookup work if stage-2 is enabled instead of stage-1. TLB lookup is done before a PTW, if a valid entry is found we won't do the PTW. To be able to do TLB lookup, we need the correct tagging info, as granularity and input size, so we get this based on the supported translation stage. The TLB entries are added correctly from each stage PTW. When nested translation is supported, this would need to change, for example if we go with a combined TLB implementation, we would need to use the min of the granularities in TLB. As stage-2 shouldn't be tagged by ASID, it will be set to -1 if S1P is not enabled. Signed-off-by: Mostafa Saleh <smostafa@google.com> Reviewed-by: Eric Auger <eric.auger@redhat.com> Tested-by: Eric Auger <eric.auger@redhat.com> Tested-by: Jean-Philippe Brucker <jean-philippe@linaro.org> Message-id: 20230516203327.2051088-7-smostafa@google.com Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- hw/arm/smmuv3.c | 44 +++++++++++++++++++++++++++++++++----------- 1 file changed, 33 insertions(+), 11 deletions(-) diff --git a/hw/arm/smmuv3.c b/hw/arm/smmuv3.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/smmuv3.c +++ b/hw/arm/smmuv3.c @@ -XXX,XX +XXX,XX @@ static int smmuv3_decode_config(IOMMUMemoryRegion *mr, SMMUTransCfg *cfg, STE ste; CD cd; + /* ASID defaults to -1 (if s1 is not supported). */ + cfg->asid = -1; + ret = smmu_find_ste(s, sid, &ste, event); if (ret) { return ret; @@ -XXX,XX +XXX,XX @@ static IOMMUTLBEntry smmuv3_translate(IOMMUMemoryRegion *mr, hwaddr addr, .addr_mask = ~(hwaddr)0, .perm = IOMMU_NONE, }; + /* + * Combined attributes used for TLB lookup, as only one stage is supported, + * it will hold attributes based on the enabled stage. + */ + SMMUTransTableInfo tt_combined; qemu_mutex_lock(&s->mutex); @@ -XXX,XX +XXX,XX @@ static IOMMUTLBEntry smmuv3_translate(IOMMUMemoryRegion *mr, hwaddr addr, goto epilogue; } - tt = select_tt(cfg, addr); - if (!tt) { - if (cfg->record_faults) { - event.type = SMMU_EVT_F_TRANSLATION; - event.u.f_translation.addr = addr; - event.u.f_translation.rnw = flag & 0x1; + if (cfg->stage == 1) { + /* Select stage1 translation table. */ + tt = select_tt(cfg, addr); + if (!tt) { + if (cfg->record_faults) { + event.type = SMMU_EVT_F_TRANSLATION; + event.u.f_translation.addr = addr; + event.u.f_translation.rnw = flag & 0x1; + } + status = SMMU_TRANS_ERROR; + goto epilogue; } - status = SMMU_TRANS_ERROR; - goto epilogue; - } + tt_combined.granule_sz = tt->granule_sz; + tt_combined.tsz = tt->tsz; - page_mask = (1ULL << (tt->granule_sz)) - 1; + } else { + /* Stage2. */ + tt_combined.granule_sz = cfg->s2cfg.granule_sz; + tt_combined.tsz = cfg->s2cfg.tsz; + } + /* + * TLB lookup looks for granule and input size for a translation stage, + * as only one stage is supported right now, choose the right values + * from the configuration. + */ + page_mask = (1ULL << tt_combined.granule_sz) - 1; aligned_addr = addr & ~page_mask; - cached_entry = smmu_iotlb_lookup(bs, cfg, tt, aligned_addr); + cached_entry = smmu_iotlb_lookup(bs, cfg, &tt_combined, aligned_addr); if (cached_entry) { if ((flag & IOMMU_WO) && !(cached_entry->entry.perm & IOMMU_WO)) { status = SMMU_TRANS_ERROR; -- 2.34.1 From: Mostafa Saleh <smostafa@google.com> Allow TLB to be tagged with VMID. If stage-1 is only supported, VMID is set to -1 and ignored from STE and CMD_TLBI_NH* cmds. Update smmu_iotlb_insert trace event to have vmid. Signed-off-by: Mostafa Saleh <smostafa@google.com> Reviewed-by: Eric Auger <eric.auger@redhat.com> Tested-by: Eric Auger <eric.auger@redhat.com> Tested-by: Jean-Philippe Brucker <jean-philippe@linaro.org> Message-id: 20230516203327.2051088-8-smostafa@google.com Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- hw/arm/smmu-internal.h | 2 ++ include/hw/arm/smmu-common.h | 5 +++-- hw/arm/smmu-common.c | 36 ++++++++++++++++++++++-------------- hw/arm/smmuv3.c | 12 +++++++++--- hw/arm/trace-events | 6 +++--- 5 files changed, 39 insertions(+), 22 deletions(-) diff --git a/hw/arm/smmu-internal.h b/hw/arm/smmu-internal.h index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/smmu-internal.h +++ b/hw/arm/smmu-internal.h @@ -XXX,XX +XXX,XX @@ static inline int pgd_concat_idx(int start_level, int granule_sz, } #define SMMU_IOTLB_ASID(key) ((key).asid) +#define SMMU_IOTLB_VMID(key) ((key).vmid) typedef struct SMMUIOTLBPageInvInfo { int asid; + int vmid; uint64_t iova; uint64_t mask; } SMMUIOTLBPageInvInfo; diff --git a/include/hw/arm/smmu-common.h b/include/hw/arm/smmu-common.h index XXXXXXX..XXXXXXX 100644 --- a/include/hw/arm/smmu-common.h +++ b/include/hw/arm/smmu-common.h @@ -XXX,XX +XXX,XX @@ typedef struct SMMUPciBus { typedef struct SMMUIOTLBKey { uint64_t iova; uint16_t asid; + uint16_t vmid; uint8_t tg; uint8_t level; } SMMUIOTLBKey; @@ -XXX,XX +XXX,XX @@ IOMMUMemoryRegion *smmu_iommu_mr(SMMUState *s, uint32_t sid); SMMUTLBEntry *smmu_iotlb_lookup(SMMUState *bs, SMMUTransCfg *cfg, SMMUTransTableInfo *tt, hwaddr iova); void smmu_iotlb_insert(SMMUState *bs, SMMUTransCfg *cfg, SMMUTLBEntry *entry); -SMMUIOTLBKey smmu_get_iotlb_key(uint16_t asid, uint64_t iova, +SMMUIOTLBKey smmu_get_iotlb_key(uint16_t asid, uint16_t vmid, uint64_t iova, uint8_t tg, uint8_t level); void smmu_iotlb_inv_all(SMMUState *s); void smmu_iotlb_inv_asid(SMMUState *s, uint16_t asid); -void smmu_iotlb_inv_iova(SMMUState *s, int asid, dma_addr_t iova, +void smmu_iotlb_inv_iova(SMMUState *s, int asid, int vmid, dma_addr_t iova, uint8_t tg, uint64_t num_pages, uint8_t ttl); /* Unmap the range of all the notifiers registered to any IOMMU mr */ diff --git a/hw/arm/smmu-common.c b/hw/arm/smmu-common.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/smmu-common.c +++ b/hw/arm/smmu-common.c @@ -XXX,XX +XXX,XX @@ static guint smmu_iotlb_key_hash(gconstpointer v) /* Jenkins hash */ a = b = c = JHASH_INITVAL + sizeof(*key); - a += key->asid + key->level + key->tg; + a += key->asid + key->vmid + key->level + key->tg; b += extract64(key->iova, 0, 32); c += extract64(key->iova, 32, 32); @@ -XXX,XX +XXX,XX @@ static gboolean smmu_iotlb_key_equal(gconstpointer v1, gconstpointer v2) SMMUIOTLBKey *k1 = (SMMUIOTLBKey *)v1, *k2 = (SMMUIOTLBKey *)v2; return (k1->asid == k2->asid) && (k1->iova == k2->iova) && - (k1->level == k2->level) && (k1->tg == k2->tg); + (k1->level == k2->level) && (k1->tg == k2->tg) && + (k1->vmid == k2->vmid); } -SMMUIOTLBKey smmu_get_iotlb_key(uint16_t asid, uint64_t iova, +SMMUIOTLBKey smmu_get_iotlb_key(uint16_t asid, uint16_t vmid, uint64_t iova, uint8_t tg, uint8_t level) { - SMMUIOTLBKey key = {.asid = asid, .iova = iova, .tg = tg, .level = level}; + SMMUIOTLBKey key = {.asid = asid, .vmid = vmid, .iova = iova, + .tg = tg, .level = level}; return key; } @@ -XXX,XX +XXX,XX @@ SMMUTLBEntry *smmu_iotlb_lookup(SMMUState *bs, SMMUTransCfg *cfg, uint64_t mask = subpage_size - 1; SMMUIOTLBKey key; - key = smmu_get_iotlb_key(cfg->asid, iova & ~mask, tg, level); + key = smmu_get_iotlb_key(cfg->asid, cfg->s2cfg.vmid, + iova & ~mask, tg, level); entry = g_hash_table_lookup(bs->iotlb, &key); if (entry) { break; @@ -XXX,XX +XXX,XX @@ SMMUTLBEntry *smmu_iotlb_lookup(SMMUState *bs, SMMUTransCfg *cfg, if (entry) { cfg->iotlb_hits++; - trace_smmu_iotlb_lookup_hit(cfg->asid, iova, + trace_smmu_iotlb_lookup_hit(cfg->asid, cfg->s2cfg.vmid, iova, cfg->iotlb_hits, cfg->iotlb_misses, 100 * cfg->iotlb_hits / (cfg->iotlb_hits + cfg->iotlb_misses)); } else { cfg->iotlb_misses++; - trace_smmu_iotlb_lookup_miss(cfg->asid, iova, + trace_smmu_iotlb_lookup_miss(cfg->asid, cfg->s2cfg.vmid, iova, cfg->iotlb_hits, cfg->iotlb_misses, 100 * cfg->iotlb_hits / (cfg->iotlb_hits + cfg->iotlb_misses)); @@ -XXX,XX +XXX,XX @@ void smmu_iotlb_insert(SMMUState *bs, SMMUTransCfg *cfg, SMMUTLBEntry *new) smmu_iotlb_inv_all(bs); } - *key = smmu_get_iotlb_key(cfg->asid, new->entry.iova, tg, new->level); - trace_smmu_iotlb_insert(cfg->asid, new->entry.iova, tg, new->level); + *key = smmu_get_iotlb_key(cfg->asid, cfg->s2cfg.vmid, new->entry.iova, + tg, new->level); + trace_smmu_iotlb_insert(cfg->asid, cfg->s2cfg.vmid, new->entry.iova, + tg, new->level); g_hash_table_insert(bs->iotlb, key, new); } @@ -XXX,XX +XXX,XX @@ static gboolean smmu_hash_remove_by_asid(gpointer key, gpointer value, return SMMU_IOTLB_ASID(*iotlb_key) == asid; } - -static gboolean smmu_hash_remove_by_asid_iova(gpointer key, gpointer value, +static gboolean smmu_hash_remove_by_asid_vmid_iova(gpointer key, gpointer value, gpointer user_data) { SMMUTLBEntry *iter = (SMMUTLBEntry *)value; @@ -XXX,XX +XXX,XX @@ static gboolean smmu_hash_remove_by_asid_iova(gpointer key, gpointer value, if (info->asid >= 0 && info->asid != SMMU_IOTLB_ASID(iotlb_key)) { return false; } + if (info->vmid >= 0 && info->vmid != SMMU_IOTLB_VMID(iotlb_key)) { + return false; + } return ((info->iova & ~entry->addr_mask) == entry->iova) || ((entry->iova & ~info->mask) == info->iova); } -void smmu_iotlb_inv_iova(SMMUState *s, int asid, dma_addr_t iova, +void smmu_iotlb_inv_iova(SMMUState *s, int asid, int vmid, dma_addr_t iova, uint8_t tg, uint64_t num_pages, uint8_t ttl) { /* if tg is not set we use 4KB range invalidation */ uint8_t granule = tg ? tg * 2 + 10 : 12; if (ttl && (num_pages == 1) && (asid >= 0)) { - SMMUIOTLBKey key = smmu_get_iotlb_key(asid, iova, tg, ttl); + SMMUIOTLBKey key = smmu_get_iotlb_key(asid, vmid, iova, tg, ttl); if (g_hash_table_remove(s->iotlb, &key)) { return; @@ -XXX,XX +XXX,XX @@ void smmu_iotlb_inv_iova(SMMUState *s, int asid, dma_addr_t iova, SMMUIOTLBPageInvInfo info = { .asid = asid, .iova = iova, + .vmid = vmid, .mask = (num_pages * 1 << granule) - 1}; g_hash_table_foreach_remove(s->iotlb, - smmu_hash_remove_by_asid_iova, + smmu_hash_remove_by_asid_vmid_iova, &info); } diff --git a/hw/arm/smmuv3.c b/hw/arm/smmuv3.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/smmuv3.c +++ b/hw/arm/smmuv3.c @@ -XXX,XX +XXX,XX @@ static void smmuv3_s1_range_inval(SMMUState *s, Cmd *cmd) { dma_addr_t end, addr = CMD_ADDR(cmd); uint8_t type = CMD_TYPE(cmd); - uint16_t vmid = CMD_VMID(cmd); + int vmid = -1; uint8_t scale = CMD_SCALE(cmd); uint8_t num = CMD_NUM(cmd); uint8_t ttl = CMD_TTL(cmd); @@ -XXX,XX +XXX,XX @@ static void smmuv3_s1_range_inval(SMMUState *s, Cmd *cmd) uint64_t num_pages; uint8_t granule; int asid = -1; + SMMUv3State *smmuv3 = ARM_SMMUV3(s); + + /* Only consider VMID if stage-2 is supported. */ + if (STAGE2_SUPPORTED(smmuv3)) { + vmid = CMD_VMID(cmd); + } if (type == SMMU_CMD_TLBI_NH_VA) { asid = CMD_ASID(cmd); @@ -XXX,XX +XXX,XX @@ static void smmuv3_s1_range_inval(SMMUState *s, Cmd *cmd) if (!tg) { trace_smmuv3_s1_range_inval(vmid, asid, addr, tg, 1, ttl, leaf); smmuv3_inv_notifiers_iova(s, asid, addr, tg, 1); - smmu_iotlb_inv_iova(s, asid, addr, tg, 1, ttl); + smmu_iotlb_inv_iova(s, asid, vmid, addr, tg, 1, ttl); return; } @@ -XXX,XX +XXX,XX @@ static void smmuv3_s1_range_inval(SMMUState *s, Cmd *cmd) num_pages = (mask + 1) >> granule; trace_smmuv3_s1_range_inval(vmid, asid, addr, tg, num_pages, ttl, leaf); smmuv3_inv_notifiers_iova(s, asid, addr, tg, num_pages); - smmu_iotlb_inv_iova(s, asid, addr, tg, num_pages, ttl); + smmu_iotlb_inv_iova(s, asid, vmid, addr, tg, num_pages, ttl); addr += mask + 1; } } diff --git a/hw/arm/trace-events b/hw/arm/trace-events index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/trace-events +++ b/hw/arm/trace-events @@ -XXX,XX +XXX,XX @@ smmu_iotlb_inv_all(void) "IOTLB invalidate all" smmu_iotlb_inv_asid(uint16_t asid) "IOTLB invalidate asid=%d" smmu_iotlb_inv_iova(uint16_t asid, uint64_t addr) "IOTLB invalidate asid=%d addr=0x%"PRIx64 smmu_inv_notifiers_mr(const char *name) "iommu mr=%s" -smmu_iotlb_lookup_hit(uint16_t asid, uint64_t addr, uint32_t hit, uint32_t miss, uint32_t p) "IOTLB cache HIT asid=%d addr=0x%"PRIx64" hit=%d miss=%d hit rate=%d" -smmu_iotlb_lookup_miss(uint16_t asid, uint64_t addr, uint32_t hit, uint32_t miss, uint32_t p) "IOTLB cache MISS asid=%d addr=0x%"PRIx64" hit=%d miss=%d hit rate=%d" -smmu_iotlb_insert(uint16_t asid, uint64_t addr, uint8_t tg, uint8_t level) "IOTLB ++ asid=%d addr=0x%"PRIx64" tg=%d level=%d" +smmu_iotlb_lookup_hit(uint16_t asid, uint16_t vmid, uint64_t addr, uint32_t hit, uint32_t miss, uint32_t p) "IOTLB cache HIT asid=%d vmid=%d addr=0x%"PRIx64" hit=%d miss=%d hit rate=%d" +smmu_iotlb_lookup_miss(uint16_t asid, uint16_t vmid, uint64_t addr, uint32_t hit, uint32_t miss, uint32_t p) "IOTLB cache MISS asid=%d vmid=%d addr=0x%"PRIx64" hit=%d miss=%d hit rate=%d" +smmu_iotlb_insert(uint16_t asid, uint16_t vmid, uint64_t addr, uint8_t tg, uint8_t level) "IOTLB ++ asid=%d vmid=%d addr=0x%"PRIx64" tg=%d level=%d" # smmuv3.c smmuv3_read_mmio(uint64_t addr, uint64_t val, unsigned size, uint32_t r) "addr: 0x%"PRIx64" val:0x%"PRIx64" size: 0x%x(%d)" -- 2.34.1 From: Mostafa Saleh <smostafa@google.com> CMD_TLBI_S2_IPA: As S1+S2 is not enabled, for now this can be the same as CMD_TLBI_NH_VAA. CMD_TLBI_S12_VMALL: Added new function to invalidate TLB by VMID. For stage-1 only commands, add a check to throw CERROR_ILL if used when stage-1 is not supported. Reviewed-by: Eric Auger <eric.auger@redhat.com> Signed-off-by: Mostafa Saleh <smostafa@google.com> Tested-by: Eric Auger <eric.auger@redhat.com> Tested-by: Jean-Philippe Brucker <jean-philippe@linaro.org> Message-id: 20230516203327.2051088-9-smostafa@google.com Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- include/hw/arm/smmu-common.h | 1 + hw/arm/smmu-common.c | 16 +++++++++++ hw/arm/smmuv3.c | 55 ++++++++++++++++++++++++++++++------ hw/arm/trace-events | 4 ++- 4 files changed, 67 insertions(+), 9 deletions(-) diff --git a/include/hw/arm/smmu-common.h b/include/hw/arm/smmu-common.h index XXXXXXX..XXXXXXX 100644 --- a/include/hw/arm/smmu-common.h +++ b/include/hw/arm/smmu-common.h @@ -XXX,XX +XXX,XX @@ SMMUIOTLBKey smmu_get_iotlb_key(uint16_t asid, uint16_t vmid, uint64_t iova, uint8_t tg, uint8_t level); void smmu_iotlb_inv_all(SMMUState *s); void smmu_iotlb_inv_asid(SMMUState *s, uint16_t asid); +void smmu_iotlb_inv_vmid(SMMUState *s, uint16_t vmid); void smmu_iotlb_inv_iova(SMMUState *s, int asid, int vmid, dma_addr_t iova, uint8_t tg, uint64_t num_pages, uint8_t ttl); diff --git a/hw/arm/smmu-common.c b/hw/arm/smmu-common.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/smmu-common.c +++ b/hw/arm/smmu-common.c @@ -XXX,XX +XXX,XX @@ static gboolean smmu_hash_remove_by_asid(gpointer key, gpointer value, return SMMU_IOTLB_ASID(*iotlb_key) == asid; } + +static gboolean smmu_hash_remove_by_vmid(gpointer key, gpointer value, + gpointer user_data) +{ + uint16_t vmid = *(uint16_t *)user_data; + SMMUIOTLBKey *iotlb_key = (SMMUIOTLBKey *)key; + + return SMMU_IOTLB_VMID(*iotlb_key) == vmid; +} + static gboolean smmu_hash_remove_by_asid_vmid_iova(gpointer key, gpointer value, gpointer user_data) { @@ -XXX,XX +XXX,XX @@ void smmu_iotlb_inv_asid(SMMUState *s, uint16_t asid) g_hash_table_foreach_remove(s->iotlb, smmu_hash_remove_by_asid, &asid); } +inline void smmu_iotlb_inv_vmid(SMMUState *s, uint16_t vmid) +{ + trace_smmu_iotlb_inv_vmid(vmid); + g_hash_table_foreach_remove(s->iotlb, smmu_hash_remove_by_vmid, &vmid); +} + /* VMSAv8-64 Translation */ /** diff --git a/hw/arm/smmuv3.c b/hw/arm/smmuv3.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/smmuv3.c +++ b/hw/arm/smmuv3.c @@ -XXX,XX +XXX,XX @@ static void smmuv3_inv_notifiers_iova(SMMUState *s, int asid, dma_addr_t iova, } } -static void smmuv3_s1_range_inval(SMMUState *s, Cmd *cmd) +static void smmuv3_range_inval(SMMUState *s, Cmd *cmd) { dma_addr_t end, addr = CMD_ADDR(cmd); uint8_t type = CMD_TYPE(cmd); @@ -XXX,XX +XXX,XX @@ static void smmuv3_s1_range_inval(SMMUState *s, Cmd *cmd) } if (!tg) { - trace_smmuv3_s1_range_inval(vmid, asid, addr, tg, 1, ttl, leaf); + trace_smmuv3_range_inval(vmid, asid, addr, tg, 1, ttl, leaf); smmuv3_inv_notifiers_iova(s, asid, addr, tg, 1); smmu_iotlb_inv_iova(s, asid, vmid, addr, tg, 1, ttl); return; @@ -XXX,XX +XXX,XX @@ static void smmuv3_s1_range_inval(SMMUState *s, Cmd *cmd) uint64_t mask = dma_aligned_pow2_mask(addr, end, 64); num_pages = (mask + 1) >> granule; - trace_smmuv3_s1_range_inval(vmid, asid, addr, tg, num_pages, ttl, leaf); + trace_smmuv3_range_inval(vmid, asid, addr, tg, num_pages, ttl, leaf); smmuv3_inv_notifiers_iova(s, asid, addr, tg, num_pages); smmu_iotlb_inv_iova(s, asid, vmid, addr, tg, num_pages, ttl); addr += mask + 1; @@ -XXX,XX +XXX,XX @@ static int smmuv3_cmdq_consume(SMMUv3State *s) { uint16_t asid = CMD_ASID(&cmd); + if (!STAGE1_SUPPORTED(s)) { + cmd_error = SMMU_CERROR_ILL; + break; + } + trace_smmuv3_cmdq_tlbi_nh_asid(asid); smmu_inv_notifiers_all(&s->smmu_state); smmu_iotlb_inv_asid(bs, asid); break; } case SMMU_CMD_TLBI_NH_ALL: + if (!STAGE1_SUPPORTED(s)) { + cmd_error = SMMU_CERROR_ILL; + break; + } + QEMU_FALLTHROUGH; case SMMU_CMD_TLBI_NSNH_ALL: trace_smmuv3_cmdq_tlbi_nh(); smmu_inv_notifiers_all(&s->smmu_state); @@ -XXX,XX +XXX,XX @@ static int smmuv3_cmdq_consume(SMMUv3State *s) break; case SMMU_CMD_TLBI_NH_VAA: case SMMU_CMD_TLBI_NH_VA: - smmuv3_s1_range_inval(bs, &cmd); + if (!STAGE1_SUPPORTED(s)) { + cmd_error = SMMU_CERROR_ILL; + break; + } + smmuv3_range_inval(bs, &cmd); + break; + case SMMU_CMD_TLBI_S12_VMALL: + { + uint16_t vmid = CMD_VMID(&cmd); + + if (!STAGE2_SUPPORTED(s)) { + cmd_error = SMMU_CERROR_ILL; + break; + } + + trace_smmuv3_cmdq_tlbi_s12_vmid(vmid); + smmu_inv_notifiers_all(&s->smmu_state); + smmu_iotlb_inv_vmid(bs, vmid); + break; + } + case SMMU_CMD_TLBI_S2_IPA: + if (!STAGE2_SUPPORTED(s)) { + cmd_error = SMMU_CERROR_ILL; + break; + } + /* + * As currently only either s1 or s2 are supported + * we can reuse same function for s2. + */ + smmuv3_range_inval(bs, &cmd); break; case SMMU_CMD_TLBI_EL3_ALL: case SMMU_CMD_TLBI_EL3_VA: @@ -XXX,XX +XXX,XX @@ static int smmuv3_cmdq_consume(SMMUv3State *s) case SMMU_CMD_TLBI_EL2_ASID: case SMMU_CMD_TLBI_EL2_VA: case SMMU_CMD_TLBI_EL2_VAA: - case SMMU_CMD_TLBI_S12_VMALL: - case SMMU_CMD_TLBI_S2_IPA: case SMMU_CMD_ATC_INV: case SMMU_CMD_PRI_RESP: case SMMU_CMD_RESUME: @@ -XXX,XX +XXX,XX @@ static int smmuv3_cmdq_consume(SMMUv3State *s) break; default: cmd_error = SMMU_CERROR_ILL; - qemu_log_mask(LOG_GUEST_ERROR, - "Illegal command type: %d\n", CMD_TYPE(&cmd)); break; } qemu_mutex_unlock(&s->mutex); if (cmd_error) { + if (cmd_error == SMMU_CERROR_ILL) { + qemu_log_mask(LOG_GUEST_ERROR, + "Illegal command type: %d\n", CMD_TYPE(&cmd)); + } break; } /* diff --git a/hw/arm/trace-events b/hw/arm/trace-events index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/trace-events +++ b/hw/arm/trace-events @@ -XXX,XX +XXX,XX @@ smmu_ptw_block_pte(int stage, int level, uint64_t baseaddr, uint64_t pteaddr, ui smmu_get_pte(uint64_t baseaddr, int index, uint64_t pteaddr, uint64_t pte) "baseaddr=0x%"PRIx64" index=0x%x, pteaddr=0x%"PRIx64", pte=0x%"PRIx64 smmu_iotlb_inv_all(void) "IOTLB invalidate all" smmu_iotlb_inv_asid(uint16_t asid) "IOTLB invalidate asid=%d" +smmu_iotlb_inv_vmid(uint16_t vmid) "IOTLB invalidate vmid=%d" smmu_iotlb_inv_iova(uint16_t asid, uint64_t addr) "IOTLB invalidate asid=%d addr=0x%"PRIx64 smmu_inv_notifiers_mr(const char *name) "iommu mr=%s" smmu_iotlb_lookup_hit(uint16_t asid, uint16_t vmid, uint64_t addr, uint32_t hit, uint32_t miss, uint32_t p) "IOTLB cache HIT asid=%d vmid=%d addr=0x%"PRIx64" hit=%d miss=%d hit rate=%d" @@ -XXX,XX +XXX,XX @@ smmuv3_cmdq_cfgi_ste_range(int start, int end) "start=0x%x - end=0x%x" smmuv3_cmdq_cfgi_cd(uint32_t sid) "sid=0x%x" smmuv3_config_cache_hit(uint32_t sid, uint32_t hits, uint32_t misses, uint32_t perc) "Config cache HIT for sid=0x%x (hits=%d, misses=%d, hit rate=%d)" smmuv3_config_cache_miss(uint32_t sid, uint32_t hits, uint32_t misses, uint32_t perc) "Config cache MISS for sid=0x%x (hits=%d, misses=%d, hit rate=%d)" -smmuv3_s1_range_inval(int vmid, int asid, uint64_t addr, uint8_t tg, uint64_t num_pages, uint8_t ttl, bool leaf) "vmid=%d asid=%d addr=0x%"PRIx64" tg=%d num_pages=0x%"PRIx64" ttl=%d leaf=%d" +smmuv3_range_inval(int vmid, int asid, uint64_t addr, uint8_t tg, uint64_t num_pages, uint8_t ttl, bool leaf) "vmid=%d asid=%d addr=0x%"PRIx64" tg=%d num_pages=0x%"PRIx64" ttl=%d leaf=%d" smmuv3_cmdq_tlbi_nh(void) "" smmuv3_cmdq_tlbi_nh_asid(uint16_t asid) "asid=%d" +smmuv3_cmdq_tlbi_s12_vmid(uint16_t vmid) "vmid=%d" smmuv3_config_cache_inv(uint32_t sid) "Config cache INV for sid=0x%x" smmuv3_notify_flag_add(const char *iommu) "ADD SMMUNotifier node for iommu mr=%s" smmuv3_notify_flag_del(const char *iommu) "DEL SMMUNotifier node for iommu mr=%s" -- 2.34.1 From: Mostafa Saleh <smostafa@google.com> In smmuv3_notify_iova, read the granule based on translation stage and use VMID if valid value is sent. Signed-off-by: Mostafa Saleh <smostafa@google.com> Reviewed-by: Eric Auger <eric.auger@redhat.com> Tested-by: Eric Auger <eric.auger@redhat.com> Tested-by: Jean-Philippe Brucker <jean-philippe@linaro.org> Message-id: 20230516203327.2051088-10-smostafa@google.com Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- hw/arm/smmuv3.c | 39 ++++++++++++++++++++++++++------------- hw/arm/trace-events | 2 +- 2 files changed, 27 insertions(+), 14 deletions(-) diff --git a/hw/arm/smmuv3.c b/hw/arm/smmuv3.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/smmuv3.c +++ b/hw/arm/smmuv3.c @@ -XXX,XX +XXX,XX @@ epilogue: * @mr: IOMMU mr region handle * @n: notifier to be called * @asid: address space ID or negative value if we don't care + * @vmid: virtual machine ID or negative value if we don't care * @iova: iova * @tg: translation granule (if communicated through range invalidation) * @num_pages: number of @granule sized pages (if tg != 0), otherwise 1 */ static void smmuv3_notify_iova(IOMMUMemoryRegion *mr, IOMMUNotifier *n, - int asid, dma_addr_t iova, - uint8_t tg, uint64_t num_pages) + int asid, int vmid, + dma_addr_t iova, uint8_t tg, + uint64_t num_pages) { SMMUDevice *sdev = container_of(mr, SMMUDevice, iommu); IOMMUTLBEvent event; uint8_t granule; + SMMUv3State *s = sdev->smmu; if (!tg) { SMMUEventInfo event = {.inval_ste_allowed = true}; @@ -XXX,XX +XXX,XX @@ static void smmuv3_notify_iova(IOMMUMemoryRegion *mr, return; } - tt = select_tt(cfg, iova); - if (!tt) { + if (vmid >= 0 && cfg->s2cfg.vmid != vmid) { return; } - granule = tt->granule_sz; + + if (STAGE1_SUPPORTED(s)) { + tt = select_tt(cfg, iova); + if (!tt) { + return; + } + granule = tt->granule_sz; + } else { + granule = cfg->s2cfg.granule_sz; + } + } else { granule = tg * 2 + 10; } @@ -XXX,XX +XXX,XX @@ static void smmuv3_notify_iova(IOMMUMemoryRegion *mr, memory_region_notify_iommu_one(n, &event); } -/* invalidate an asid/iova range tuple in all mr's */ -static void smmuv3_inv_notifiers_iova(SMMUState *s, int asid, dma_addr_t iova, - uint8_t tg, uint64_t num_pages) +/* invalidate an asid/vmid/iova range tuple in all mr's */ +static void smmuv3_inv_notifiers_iova(SMMUState *s, int asid, int vmid, + dma_addr_t iova, uint8_t tg, + uint64_t num_pages) { SMMUDevice *sdev; @@ -XXX,XX +XXX,XX @@ static void smmuv3_inv_notifiers_iova(SMMUState *s, int asid, dma_addr_t iova, IOMMUMemoryRegion *mr = &sdev->iommu; IOMMUNotifier *n; - trace_smmuv3_inv_notifiers_iova(mr->parent_obj.name, asid, iova, - tg, num_pages); + trace_smmuv3_inv_notifiers_iova(mr->parent_obj.name, asid, vmid, + iova, tg, num_pages); IOMMU_NOTIFIER_FOREACH(n, mr) { - smmuv3_notify_iova(mr, n, asid, iova, tg, num_pages); + smmuv3_notify_iova(mr, n, asid, vmid, iova, tg, num_pages); } } } @@ -XXX,XX +XXX,XX @@ static void smmuv3_range_inval(SMMUState *s, Cmd *cmd) if (!tg) { trace_smmuv3_range_inval(vmid, asid, addr, tg, 1, ttl, leaf); - smmuv3_inv_notifiers_iova(s, asid, addr, tg, 1); + smmuv3_inv_notifiers_iova(s, asid, vmid, addr, tg, 1); smmu_iotlb_inv_iova(s, asid, vmid, addr, tg, 1, ttl); return; } @@ -XXX,XX +XXX,XX @@ static void smmuv3_range_inval(SMMUState *s, Cmd *cmd) num_pages = (mask + 1) >> granule; trace_smmuv3_range_inval(vmid, asid, addr, tg, num_pages, ttl, leaf); - smmuv3_inv_notifiers_iova(s, asid, addr, tg, num_pages); + smmuv3_inv_notifiers_iova(s, asid, vmid, addr, tg, num_pages); smmu_iotlb_inv_iova(s, asid, vmid, addr, tg, num_pages, ttl); addr += mask + 1; } diff --git a/hw/arm/trace-events b/hw/arm/trace-events index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/trace-events +++ b/hw/arm/trace-events @@ -XXX,XX +XXX,XX @@ smmuv3_cmdq_tlbi_s12_vmid(uint16_t vmid) "vmid=%d" smmuv3_config_cache_inv(uint32_t sid) "Config cache INV for sid=0x%x" smmuv3_notify_flag_add(const char *iommu) "ADD SMMUNotifier node for iommu mr=%s" smmuv3_notify_flag_del(const char *iommu) "DEL SMMUNotifier node for iommu mr=%s" -smmuv3_inv_notifiers_iova(const char *name, uint16_t asid, uint64_t iova, uint8_t tg, uint64_t num_pages) "iommu mr=%s asid=%d iova=0x%"PRIx64" tg=%d num_pages=0x%"PRIx64 +smmuv3_inv_notifiers_iova(const char *name, uint16_t asid, uint16_t vmid, uint64_t iova, uint8_t tg, uint64_t num_pages) "iommu mr=%s asid=%d vmid=%d iova=0x%"PRIx64" tg=%d num_pages=0x%"PRIx64 -- 2.34.1 From: Mostafa Saleh <smostafa@google.com> As everything is in place, we can use a new system property to advertise which stage is supported and remove bad_ste from STE stage2 config. The property added arm-smmuv3.stage can have 3 values: - "1": Stage-1 only is advertised. - "2": Stage-2 only is advertised. If not passed or an unsupported value is passed, it will default to stage-1. Advertise VMID16. Don't try to decode CD, if stage-2 is configured. Reviewed-by: Eric Auger <eric.auger@redhat.com> Signed-off-by: Mostafa Saleh <smostafa@google.com> Tested-by: Eric Auger <eric.auger@redhat.com> Tested-by: Jean-Philippe Brucker <jean-philippe@linaro.org> Message-id: 20230516203327.2051088-11-smostafa@google.com Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- include/hw/arm/smmuv3.h | 1 + hw/arm/smmuv3.c | 32 ++++++++++++++++++++++---------- 2 files changed, 23 insertions(+), 10 deletions(-) diff --git a/include/hw/arm/smmuv3.h b/include/hw/arm/smmuv3.h index XXXXXXX..XXXXXXX 100644 --- a/include/hw/arm/smmuv3.h +++ b/include/hw/arm/smmuv3.h @@ -XXX,XX +XXX,XX @@ struct SMMUv3State { qemu_irq irq[4]; QemuMutex mutex; + char *stage; }; typedef enum { diff --git a/hw/arm/smmuv3.c b/hw/arm/smmuv3.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/smmuv3.c +++ b/hw/arm/smmuv3.c @@ -XXX,XX +XXX,XX @@ #include "hw/irq.h" #include "hw/sysbus.h" #include "migration/vmstate.h" +#include "hw/qdev-properties.h" #include "hw/qdev-core.h" #include "hw/pci/pci.h" #include "cpu.h" @@ -XXX,XX +XXX,XX @@ void smmuv3_record_event(SMMUv3State *s, SMMUEventInfo *info) static void smmuv3_init_regs(SMMUv3State *s) { - /** - * IDR0: stage1 only, AArch64 only, coherent access, 16b ASID, - * multi-level stream table - */ - s->idr[0] = FIELD_DP32(s->idr[0], IDR0, S1P, 1); /* stage 1 supported */ + /* Based on sys property, the stages supported in smmu will be advertised.*/ + if (s->stage && !strcmp("2", s->stage)) { + s->idr[0] = FIELD_DP32(s->idr[0], IDR0, S2P, 1); + } else { + s->idr[0] = FIELD_DP32(s->idr[0], IDR0, S1P, 1); + } + s->idr[0] = FIELD_DP32(s->idr[0], IDR0, TTF, 2); /* AArch64 PTW only */ s->idr[0] = FIELD_DP32(s->idr[0], IDR0, COHACC, 1); /* IO coherent */ s->idr[0] = FIELD_DP32(s->idr[0], IDR0, ASID16, 1); /* 16-bit ASID */ + s->idr[0] = FIELD_DP32(s->idr[0], IDR0, VMID16, 1); /* 16-bit VMID */ s->idr[0] = FIELD_DP32(s->idr[0], IDR0, TTENDIAN, 2); /* little endian */ s->idr[0] = FIELD_DP32(s->idr[0], IDR0, STALL_MODEL, 1); /* No stall */ /* terminated transaction will always be aborted/error returned */ @@ -XXX,XX +XXX,XX @@ static int decode_ste_s2_cfg(SMMUTransCfg *cfg, STE *ste) goto bad_ste; } - /* This is still here as stage 2 has not been fully enabled yet. */ - qemu_log_mask(LOG_UNIMP, "SMMUv3 does not support stage 2 yet\n"); - goto bad_ste; - return 0; bad_ste: @@ -XXX,XX +XXX,XX @@ static int smmuv3_decode_config(IOMMUMemoryRegion *mr, SMMUTransCfg *cfg, return ret; } - if (cfg->aborted || cfg->bypassed) { + if (cfg->aborted || cfg->bypassed || (cfg->stage == 2)) { return 0; } @@ -XXX,XX +XXX,XX @@ static const VMStateDescription vmstate_smmuv3 = { } }; +static Property smmuv3_properties[] = { + /* + * Stages of translation advertised. + * "1": Stage 1 + * "2": Stage 2 + * Defaults to stage 1 + */ + DEFINE_PROP_STRING("stage", SMMUv3State, stage), + DEFINE_PROP_END_OF_LIST() +}; + static void smmuv3_instance_init(Object *obj) { /* Nothing much to do here as of now */ @@ -XXX,XX +XXX,XX @@ static void smmuv3_class_init(ObjectClass *klass, void *data) &c->parent_phases); c->parent_realize = dc->realize; dc->realize = smmu_realize; + device_class_set_props(dc, smmuv3_properties); } static int smmuv3_notify_flag_changed(IOMMUMemoryRegion *iommu, -- 2.34.1 From: Tommy Wu <tommy.wu@sifive.com> When we receive a packet from the xilinx_axienet and then try to s2mem through the xilinx_axidma, if the descriptor ring buffer is full in the xilinx axidma driver, we’ll assert the DMASR.HALTED in the function : stream_process_s2mem and return 0. In the end, we’ll be stuck in an infinite loop in axienet_eth_rx_notify. This patch checks the DMASR.HALTED state when we try to push data from xilinx axi-enet to xilinx axi-dma. When the DMASR.HALTED is asserted, we will not keep pushing the data and then prevent the infinte loop. Signed-off-by: Tommy Wu <tommy.wu@sifive.com> Reviewed-by: Edgar E. Iglesias <edgar@zeroasic.com> Reviewed-by: Frank Chang <frank.chang@sifive.com> Message-id: 20230519062137.1251741-1-tommy.wu@sifive.com Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- hw/dma/xilinx_axidma.c | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/hw/dma/xilinx_axidma.c b/hw/dma/xilinx_axidma.c index XXXXXXX..XXXXXXX 100644 --- a/hw/dma/xilinx_axidma.c +++ b/hw/dma/xilinx_axidma.c @@ -XXX,XX +XXX,XX @@ static inline int stream_idle(struct Stream *s) return !!(s->regs[R_DMASR] & DMASR_IDLE); } +static inline int stream_halted(struct Stream *s) +{ + return !!(s->regs[R_DMASR] & DMASR_HALTED); +} + static void stream_reset(struct Stream *s) { s->regs[R_DMASR] = DMASR_HALTED; /* starts up halted. */ @@ -XXX,XX +XXX,XX @@ static void stream_process_mem2s(struct Stream *s, StreamSink *tx_data_dev, uint64_t addr; bool eop; - if (!stream_running(s) || stream_idle(s)) { + if (!stream_running(s) || stream_idle(s) || stream_halted(s)) { return; } @@ -XXX,XX +XXX,XX @@ static size_t stream_process_s2mem(struct Stream *s, unsigned char *buf, unsigned int rxlen; size_t pos = 0; - if (!stream_running(s) || stream_idle(s)) { + if (!stream_running(s) || stream_idle(s) || stream_halted(s)) { return 0; } @@ -XXX,XX +XXX,XX @@ xilinx_axidma_data_stream_can_push(StreamSink *obj, XilinxAXIDMAStreamSink *ds = XILINX_AXI_DMA_DATA_STREAM(obj); struct Stream *s = &ds->dma->streams[1]; - if (!stream_running(s) || stream_idle(s)) { + if (!stream_running(s) || stream_idle(s) || stream_halted(s)) { ds->dma->notify = notify; ds->dma->notify_opaque = notify_opaque; return false; -- 2.34.1 From: Clément Chigot <chigot@adacore.com> When passing --smp with a number lower than XLNX_ZYNQMP_NUM_APU_CPUS, the expression (ms->smp.cpus - XLNX_ZYNQMP_NUM_APU_CPUS) will result in a positive number as ms->smp.cpus is a unsigned int. This will raise the following error afterwards, as Qemu will try to instantiate some additional RPUs. | $ qemu-system-aarch64 --smp 1 -M xlnx-zcu102 | ** | ERROR:../src/tcg/tcg.c:777:tcg_register_thread: | assertion failed: (n < tcg_max_ctxs) Signed-off-by: Clément Chigot <chigot@adacore.com> Reviewed-by: Francisco Iglesias <frasse.iglesias@gmail.com> Tested-by: Francisco Iglesias <frasse.iglesias@gmail.com> Message-id: 20230524143714.565792-1-chigot@adacore.com Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- hw/arm/xlnx-zynqmp.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/hw/arm/xlnx-zynqmp.c b/hw/arm/xlnx-zynqmp.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/xlnx-zynqmp.c +++ b/hw/arm/xlnx-zynqmp.c @@ -XXX,XX +XXX,XX @@ static void xlnx_zynqmp_create_rpu(MachineState *ms, XlnxZynqMPState *s, const char *boot_cpu, Error **errp) { int i; - int num_rpus = MIN(ms->smp.cpus - XLNX_ZYNQMP_NUM_APU_CPUS, + int num_rpus = MIN((int)(ms->smp.cpus - XLNX_ZYNQMP_NUM_APU_CPUS), XLNX_ZYNQMP_NUM_RPU_CPUS); if (num_rpus <= 0) { -- 2.34.1 From: Thomas Huth <thuth@redhat.com> pflash-cfi02-test.c always uses the "musicpal" machine for testing, test-arm-mptimer.c always uses the "vexpress-a9" machine, and microbit-test.c requires the "microbit" machine, so we should only run these tests if the machines have been enabled in the configuration. Signed-off-by: Thomas Huth <thuth@redhat.com> Reviewed-by: Fabiano Rosas <farosas@suse.de> Message-id: 20230524080600.1618137-1-thuth@redhat.com Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- tests/qtest/meson.build | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/tests/qtest/meson.build b/tests/qtest/meson.build index XXXXXXX..XXXXXXX 100644 --- a/tests/qtest/meson.build +++ b/tests/qtest/meson.build @@ -XXX,XX +XXX,XX @@ qtests_arm = \ (config_all_devices.has_key('CONFIG_CMSDK_APB_DUALTIMER') ? ['cmsdk-apb-dualtimer-test'] : []) + \ (config_all_devices.has_key('CONFIG_CMSDK_APB_TIMER') ? ['cmsdk-apb-timer-test'] : []) + \ (config_all_devices.has_key('CONFIG_CMSDK_APB_WATCHDOG') ? ['cmsdk-apb-watchdog-test'] : []) + \ - (config_all_devices.has_key('CONFIG_PFLASH_CFI02') ? ['pflash-cfi02-test'] : []) + \ + (config_all_devices.has_key('CONFIG_PFLASH_CFI02') and + config_all_devices.has_key('CONFIG_MUSICPAL') ? ['pflash-cfi02-test'] : []) + \ (config_all_devices.has_key('CONFIG_ASPEED_SOC') ? qtests_aspeed : []) + \ (config_all_devices.has_key('CONFIG_NPCM7XX') ? qtests_npcm7xx : []) + \ (config_all_devices.has_key('CONFIG_GENERIC_LOADER') ? ['hexloader-test'] : []) + \ (config_all_devices.has_key('CONFIG_TPM_TIS_I2C') ? ['tpm-tis-i2c-test'] : []) + \ + (config_all_devices.has_key('CONFIG_VEXPRESS') ? ['test-arm-mptimer'] : []) + \ + (config_all_devices.has_key('CONFIG_MICROBIT') ? ['microbit-test'] : []) + \ ['arm-cpu-features', - 'microbit-test', - 'test-arm-mptimer', 'boot-serial-test'] # TODO: once aarch64 TCG is fixed on ARM 32 bit host, make bios-tables-test unconditional -- 2.34.1 For M-profile, there is no guest-facing A-profile format FSR, but we still use the env->exception.fsr field to pass fault information from the point where a fault is raised to the code in arm_v7m_cpu_do_interrupt() which interprets it and sets the M-profile specific fault status registers. So it doesn't matter whether we fill in env->exception.fsr in the short format or the LPAE format, as long as both sides agree. As it happens arm_v7m_cpu_do_interrupt() assumes short-form. In compute_fsr_fsc() we weren't explicitly choosing short-form for M-profile, but instead relied on it falling out in the wash because arm_s1_regime_using_lpae_format() would be false. This was broken in commit 452c67a4 when we added v8R support, because we said "PMSAv8 is always LPAE format" (as it is for v8R), forgetting that we were implicitly using this code path on M-profile. At that point we would hit a g_assert_not_reached(): ERROR:../../target/arm/internals.h:549:arm_fi_to_lfsc: code should not be reached #7 0x0000555555e055f7 in arm_fi_to_lfsc (fi=0x7fffecff9a90) at ../../target/arm/internals.h:549 #8 0x0000555555e05a27 in compute_fsr_fsc (env=0x555557356670, fi=0x7fffecff9a90, target_el=1, mmu_idx=1, ret_fsc=0x7fffecff9a1c) at ../../target/arm/tlb_helper.c:95 #9 0x0000555555e05b62 in arm_deliver_fault (cpu=0x555557354800, addr=268961344, access_type=MMU_INST_FETCH, mmu_idx=1, fi=0x7fffecff9a90) at ../../target/arm/tlb_helper.c:132 #10 0x0000555555e06095 in arm_cpu_tlb_fill (cs=0x555557354800, address=268961344, size=1, access_type=MMU_INST_FETCH, mmu_idx=1, probe=false, retaddr=0) at ../../target/arm/tlb_helper.c:260 The specific assertion changed when commit fcc7404eff24b4c added "assert not M-profile" to arm_is_secure_below_el3(), because the conditions being checked in compute_fsr_fsc() include arm_el_is_aa64(), which will end up calling arm_is_secure_below_el3() and asserting before we try to call arm_fi_to_lfsc(): #7 0x0000555555efaf43 in arm_is_secure_below_el3 (env=0x5555574665a0) at ../../target/arm/cpu.h:2396 #8 0x0000555555efb103 in arm_is_el2_enabled (env=0x5555574665a0) at ../../target/arm/cpu.h:2448 #9 0x0000555555efb204 in arm_el_is_aa64 (env=0x5555574665a0, el=1) at ../../target/arm/cpu.h:2509 #10 0x0000555555efbdfd in compute_fsr_fsc (env=0x5555574665a0, fi=0x7fffecff99e0, target_el=1, mmu_idx=1, ret_fsc=0x7fffecff996c) Avoid the assertion and the incorrect FSR format selection by explicitly making M-profile use the short-format in this function. Fixes: 452c67a42704 ("target/arm: Enable TTBCR_EAE for ARMv8-R AArch32")a Resolves: https://gitlab.com/qemu-project/qemu/-/issues/1658 Cc: qemu-stable@nongnu.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20230523131726.866635-1-peter.maydell@linaro.org --- target/arm/tcg/tlb_helper.c | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-) diff --git a/target/arm/tcg/tlb_helper.c b/target/arm/tcg/tlb_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/tlb_helper.c +++ b/target/arm/tcg/tlb_helper.c @@ -XXX,XX +XXX,XX @@ static uint32_t compute_fsr_fsc(CPUARMState *env, ARMMMUFaultInfo *fi, ARMMMUIdx arm_mmu_idx = core_to_arm_mmu_idx(env, mmu_idx); uint32_t fsr, fsc; - if (target_el == 2 || arm_el_is_aa64(env, target_el) || - arm_s1_regime_using_lpae_format(env, arm_mmu_idx)) { + /* + * For M-profile there is no guest-facing FSR. We compute a + * short-form value for env->exception.fsr which we will then + * examine in arm_v7m_cpu_do_interrupt(). In theory we could + * use the LPAE format instead as long as both bits of code agree + * (and arm_fi_to_lfsc() handled the M-profile specific + * ARMFault_QEMU_NSCExec and ARMFault_QEMU_SFault cases). + */ + if (!arm_feature(env, ARM_FEATURE_M) && + (target_el == 2 || arm_el_is_aa64(env, target_el) || + arm_s1_regime_using_lpae_format(env, arm_mmu_idx))) { /* * LPAE format fault status register : bottom 6 bits are * status code in the same form as needed for syndrome -- 2.34.1 From: Fabiano Rosas <farosas@suse.de> We currently need to select ARM_V7M unconditionally when TCG is present in the build because some translate.c helpers and the whole of m_helpers.c are not yet under CONFIG_ARM_V7M. Suggested-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Fabiano Rosas <farosas@suse.de> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Message-id: 20230523180525.29994-2-farosas@suse.de Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/Kconfig | 3 +++ 1 file changed, 3 insertions(+) diff --git a/target/arm/Kconfig b/target/arm/Kconfig index XXXXXXX..XXXXXXX 100644 --- a/target/arm/Kconfig +++ b/target/arm/Kconfig @@ -XXX,XX +XXX,XX @@ config ARM bool select ARM_COMPATIBLE_SEMIHOSTING if TCG + + # We need to select this until we move m_helper.c and the + # translate.c v7m helpers under ARM_V7M. select ARM_V7M if TCG config AARCH64 -- 2.34.1 From: Fabiano Rosas <farosas@suse.de> When we moved the arm default CONFIGs into Kconfig and removed them from default.mak, we made it harder to identify which CONFIGs are selected by default in case users want to disable them. Bring back the default entries into default.mak, but keep them commented out. This way users can keep their workflows of editing default.mak to remove build options without needing to search through Kconfig. Reported-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Fabiano Rosas <farosas@suse.de> Reviewed-by: Thomas Huth <thuth@redhat.com> Message-id: 20230523180525.29994-3-farosas@suse.de Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- configs/devices/aarch64-softmmu/default.mak | 6 ++++ configs/devices/arm-softmmu/default.mak | 40 +++++++++++++++++++++ 2 files changed, 46 insertions(+) diff --git a/configs/devices/aarch64-softmmu/default.mak b/configs/devices/aarch64-softmmu/default.mak index XXXXXXX..XXXXXXX 100644 --- a/configs/devices/aarch64-softmmu/default.mak +++ b/configs/devices/aarch64-softmmu/default.mak @@ -XXX,XX +XXX,XX @@ # We support all the 32 bit boards so need all their config include ../arm-softmmu/default.mak + +# These are selected by default when TCG is enabled, uncomment them to +# keep out of the build. +# CONFIG_XLNX_ZYNQMP_ARM=n +# CONFIG_XLNX_VERSAL=n +# CONFIG_SBSA_REF=n diff --git a/configs/devices/arm-softmmu/default.mak b/configs/devices/arm-softmmu/default.mak index XXXXXXX..XXXXXXX 100644 --- a/configs/devices/arm-softmmu/default.mak +++ b/configs/devices/arm-softmmu/default.mak @@ -XXX,XX +XXX,XX @@ # CONFIG_TEST_DEVICES=n CONFIG_ARM_VIRT=y + +# These are selected by default when TCG is enabled, uncomment them to +# keep out of the build. +# CONFIG_CUBIEBOARD=n +# CONFIG_EXYNOS4=n +# CONFIG_HIGHBANK=n +# CONFIG_INTEGRATOR=n +# CONFIG_FSL_IMX31=n +# CONFIG_MUSICPAL=n +# CONFIG_MUSCA=n +# CONFIG_CHEETAH=n +# CONFIG_SX1=n +# CONFIG_NSERIES=n +# CONFIG_STELLARIS=n +# CONFIG_STM32VLDISCOVERY=n +# CONFIG_REALVIEW=n +# CONFIG_VERSATILE=n +# CONFIG_VEXPRESS=n +# CONFIG_ZYNQ=n +# CONFIG_MAINSTONE=n +# CONFIG_GUMSTIX=n +# CONFIG_SPITZ=n +# CONFIG_TOSA=n +# CONFIG_Z2=n +# CONFIG_NPCM7XX=n +# CONFIG_COLLIE=n +# CONFIG_ASPEED_SOC=n +# CONFIG_NETDUINO2=n +# CONFIG_NETDUINOPLUS2=n +# CONFIG_OLIMEX_STM32_H405=n +# CONFIG_MPS2=n +# CONFIG_RASPI=n +# CONFIG_DIGIC=n +# CONFIG_SABRELITE=n +# CONFIG_EMCRAFT_SF2=n +# CONFIG_MICROBIT=n +# CONFIG_FSL_IMX25=n +# CONFIG_FSL_IMX7=n +# CONFIG_FSL_IMX6UL=n +# CONFIG_ALLWINNER_H3=n -- 2.34.1 From: Fabiano Rosas <farosas@suse.de> Replace the 'default y if TCG' pattern with 'default y; depends on TCG'. That makes explict that there is a dependence on TCG and enabling these CONFIGs via .mak files without TCG present will fail earlier. Suggested-by: Paolo Bonzini <pbonzini@redhat.com> Signed-off-by: Fabiano Rosas <farosas@suse.de> Reviewed-by: Thomas Huth <thuth@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Message-id: 20230523180525.29994-4-farosas@suse.de Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- hw/arm/Kconfig | 123 ++++++++++++++++++++++++++++++++----------------- 1 file changed, 82 insertions(+), 41 deletions(-) diff --git a/hw/arm/Kconfig b/hw/arm/Kconfig index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/Kconfig +++ b/hw/arm/Kconfig @@ -XXX,XX +XXX,XX @@ config ARM_VIRT config CHEETAH bool - default y if TCG && ARM + default y + depends on TCG && ARM select OMAP select TSC210X config CUBIEBOARD bool - default y if TCG && ARM + default y + depends on TCG && ARM select ALLWINNER_A10 config DIGIC bool - default y if TCG && ARM + default y + depends on TCG && ARM select PTIMER select PFLASH_CFI02 config EXYNOS4 bool - default y if TCG && ARM + default y + depends on TCG && ARM imply I2C_DEVICES select A9MPCORE select I2C @@ -XXX,XX +XXX,XX @@ config EXYNOS4 config HIGHBANK bool - default y if TCG && ARM + default y + depends on TCG && ARM select A9MPCORE select A15MPCORE select AHCI @@ -XXX,XX +XXX,XX @@ config HIGHBANK config INTEGRATOR bool - default y if TCG && ARM + default y + depends on TCG && ARM select ARM_TIMER select INTEGRATOR_DEBUG select PL011 # UART @@ -XXX,XX +XXX,XX @@ config INTEGRATOR config MAINSTONE bool - default y if TCG && ARM + default y + depends on TCG && ARM select PXA2XX select PFLASH_CFI01 select SMC91C111 config MUSCA bool - default y if TCG && ARM + default y + depends on TCG && ARM select ARMSSE select PL011 select PL031 @@ -XXX,XX +XXX,XX @@ config MARVELL_88W8618 config MUSICPAL bool - default y if TCG && ARM + default y + depends on TCG && ARM select OR_IRQ select BITBANG_I2C select MARVELL_88W8618 @@ -XXX,XX +XXX,XX @@ config MUSICPAL config NETDUINO2 bool - default y if TCG && ARM + default y + depends on TCG && ARM select STM32F205_SOC config NETDUINOPLUS2 bool - default y if TCG && ARM + default y + depends on TCG && ARM select STM32F405_SOC config OLIMEX_STM32_H405 bool - default y if TCG && ARM + default y + depends on TCG && ARM select STM32F405_SOC config NSERIES bool - default y if TCG && ARM + default y + depends on TCG && ARM select OMAP select TMP105 # temperature sensor select BLIZZARD # LCD/TV controller @@ -XXX,XX +XXX,XX @@ config PXA2XX config GUMSTIX bool - default y if TCG && ARM + default y + depends on TCG && ARM select PFLASH_CFI01 select SMC91C111 select PXA2XX config TOSA bool - default y if TCG && ARM + default y + depends on TCG && ARM select ZAURUS # scoop select MICRODRIVE select PXA2XX @@ -XXX,XX +XXX,XX @@ config TOSA config SPITZ bool - default y if TCG && ARM + default y + depends on TCG && ARM select ADS7846 # touch-screen controller select MAX111X # A/D converter select WM8750 # audio codec @@ -XXX,XX +XXX,XX @@ config SPITZ config Z2 bool - default y if TCG && ARM + default y + depends on TCG && ARM select PFLASH_CFI01 select WM8750 select PL011 # UART @@ -XXX,XX +XXX,XX @@ config Z2 config REALVIEW bool - default y if TCG && ARM + default y + depends on TCG && ARM imply PCI_DEVICES imply PCI_TESTDEV imply I2C_DEVICES @@ -XXX,XX +XXX,XX @@ config REALVIEW config SBSA_REF bool - default y if TCG && AARCH64 + default y + depends on TCG && AARCH64 imply PCI_DEVICES select AHCI select ARM_SMMUV3 @@ -XXX,XX +XXX,XX @@ config SBSA_REF config SABRELITE bool - default y if TCG && ARM + default y + depends on TCG && ARM select FSL_IMX6 select SSI_M25P80 config STELLARIS bool - default y if TCG && ARM + default y + depends on TCG && ARM imply I2C_DEVICES select ARM_V7M select CMSDK_APB_WATCHDOG @@ -XXX,XX +XXX,XX @@ config STELLARIS config STM32VLDISCOVERY bool - default y if TCG && ARM + default y + depends on TCG && ARM select STM32F100_SOC config STRONGARM @@ -XXX,XX +XXX,XX @@ config STRONGARM config COLLIE bool - default y if TCG && ARM + default y + depends on TCG && ARM select PFLASH_CFI01 select ZAURUS # scoop select STRONGARM config SX1 bool - default y if TCG && ARM + default y + depends on TCG && ARM select OMAP config VERSATILE bool - default y if TCG && ARM + default y + depends on TCG && ARM select ARM_TIMER # sp804 select PFLASH_CFI01 select LSI_SCSI_PCI @@ -XXX,XX +XXX,XX @@ config VERSATILE config VEXPRESS bool - default y if TCG && ARM + default y + depends on TCG && ARM select A9MPCORE select A15MPCORE select ARM_MPTIMER @@ -XXX,XX +XXX,XX @@ config VEXPRESS config ZYNQ bool - default y if TCG && ARM + default y + depends on TCG && ARM select A9MPCORE select CADENCE # UART select PFLASH_CFI02 @@ -XXX,XX +XXX,XX @@ config ZYNQ config ARM_V7M bool # currently v7M must be included in a TCG build due to translate.c - default y if TCG && ARM + default y + depends on TCG && ARM select PTIMER config ALLWINNER_A10 @@ -XXX,XX +XXX,XX @@ config ALLWINNER_A10 config ALLWINNER_H3 bool - default y if TCG && ARM + default y + depends on TCG && ARM select ALLWINNER_A10_PIT select ALLWINNER_SUN8I_EMAC select ALLWINNER_I2C @@ -XXX,XX +XXX,XX @@ config ALLWINNER_H3 config RASPI bool - default y if TCG && ARM + default y + depends on TCG && ARM select FRAMEBUFFER select PL011 # UART select SDHCI @@ -XXX,XX +XXX,XX @@ config STM32F405_SOC config XLNX_ZYNQMP_ARM bool - default y if TCG && AARCH64 + default y + depends on TCG && AARCH64 select AHCI select ARM_GIC select CADENCE @@ -XXX,XX +XXX,XX @@ config XLNX_ZYNQMP_ARM config XLNX_VERSAL bool - default y if TCG && AARCH64 + default y + depends on TCG && AARCH64 select ARM_GIC select PL011 select CADENCE @@ -XXX,XX +XXX,XX @@ config XLNX_VERSAL config NPCM7XX bool - default y if TCG && ARM + default y + depends on TCG && ARM select A9MPCORE select ADM1272 select ARM_GIC @@ -XXX,XX +XXX,XX @@ config NPCM7XX config FSL_IMX25 bool - default y if TCG && ARM + default y + depends on TCG && ARM imply I2C_DEVICES select IMX select IMX_FEC @@ -XXX,XX +XXX,XX @@ config FSL_IMX25 config FSL_IMX31 bool - default y if TCG && ARM + default y + depends on TCG && ARM imply I2C_DEVICES select SERIAL select IMX @@ -XXX,XX +XXX,XX @@ config FSL_IMX6 config ASPEED_SOC bool - default y if TCG && ARM + default y + depends on TCG && ARM select DS1338 select FTGMAC100 select I2C @@ -XXX,XX +XXX,XX @@ config ASPEED_SOC config MPS2 bool - default y if TCG && ARM + default y + depends on TCG && ARM imply I2C_DEVICES select ARMSSE select LAN9118 @@ -XXX,XX +XXX,XX @@ config MPS2 config FSL_IMX7 bool - default y if TCG && ARM + default y + depends on TCG && ARM imply PCI_DEVICES imply TEST_DEVICES imply I2C_DEVICES @@ -XXX,XX +XXX,XX @@ config ARM_SMMUV3 config FSL_IMX6UL bool - default y if TCG && ARM + default y + depends on TCG && ARM imply I2C_DEVICES select A15MPCORE select IMX @@ -XXX,XX +XXX,XX @@ config FSL_IMX6UL config MICROBIT bool - default y if TCG && ARM + default y + depends on TCG && ARM select NRF51_SOC config NRF51_SOC @@ -XXX,XX +XXX,XX @@ config NRF51_SOC config EMCRAFT_SF2 bool - default y if TCG && ARM + default y + depends on TCG && ARM select MSF2 select SSI_M25P80 -- 2.34.1 From: Enze Li <lienze@kylinos.cn> I noticed that in the latest version, the copyright string is still 2022, even though 2023 is halfway through. This patch fixes that and fixes the documentation along with it. Signed-off-by: Enze Li <lienze@kylinos.cn> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Message-id: 20230525064345.1152801-1-lienze@kylinos.cn Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- docs/conf.py | 2 +- include/qemu/help-texts.h | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/conf.py b/docs/conf.py index XXXXXXX..XXXXXXX 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -XXX,XX +XXX,XX @@ # General information about the project. project = u'QEMU' -copyright = u'2022, The QEMU Project Developers' +copyright = u'2023, The QEMU Project Developers' author = u'The QEMU Project Developers' # The version info for the project you're documenting, acts as replacement for diff --git a/include/qemu/help-texts.h b/include/qemu/help-texts.h index XXXXXXX..XXXXXXX 100644 --- a/include/qemu/help-texts.h +++ b/include/qemu/help-texts.h @@ -XXX,XX +XXX,XX @@ #define QEMU_HELP_TEXTS_H /* Copyright string for -version arguments, About dialogs, etc */ -#define QEMU_COPYRIGHT "Copyright (c) 2003-2022 " \ +#define QEMU_COPYRIGHT "Copyright (c) 2003-2023 " \ "Fabrice Bellard and the QEMU Project developers" /* Bug reporting information for --help arguments, About dialogs, etc */ -- 2.34.1 From: Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org> Let add GIC information into DeviceTree as part of SBSA-REF versioning. Trusted Firmware will read it and provide to next firmware level. Bumps platform version to 0.1 one so we can check is node is present. Signed-off-by: Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- hw/arm/sbsa-ref.c | 19 ++++++++++++++++++- 1 file changed, 18 insertions(+), 1 deletion(-) diff --git a/hw/arm/sbsa-ref.c b/hw/arm/sbsa-ref.c index XXXXXXX..XXXXXXX 100644 --- a/hw/arm/sbsa-ref.c +++ b/hw/arm/sbsa-ref.c @@ -XXX,XX +XXX,XX @@ #include "exec/hwaddr.h" #include "kvm_arm.h" #include "hw/arm/boot.h" +#include "hw/arm/fdt.h" #include "hw/arm/smmuv3.h" #include "hw/block/flash.h" #include "hw/boards.h" @@ -XXX,XX +XXX,XX @@ static uint64_t sbsa_ref_cpu_mp_affinity(SBSAMachineState *sms, int idx) return arm_cpu_mp_affinity(idx, clustersz); } +static void sbsa_fdt_add_gic_node(SBSAMachineState *sms) +{ + char *nodename; + + nodename = g_strdup_printf("/intc"); + qemu_fdt_add_subnode(sms->fdt, nodename); + qemu_fdt_setprop_sized_cells(sms->fdt, nodename, "reg", + 2, sbsa_ref_memmap[SBSA_GIC_DIST].base, + 2, sbsa_ref_memmap[SBSA_GIC_DIST].size, + 2, sbsa_ref_memmap[SBSA_GIC_REDIST].base, + 2, sbsa_ref_memmap[SBSA_GIC_REDIST].size); + + g_free(nodename); +} /* * Firmware on this machine only uses ACPI table to load OS, these limited * device tree nodes are just to let firmware know the info which varies from @@ -XXX,XX +XXX,XX @@ static void create_fdt(SBSAMachineState *sms) * fw compatibility. */ qemu_fdt_setprop_cell(fdt, "/", "machine-version-major", 0); - qemu_fdt_setprop_cell(fdt, "/", "machine-version-minor", 0); + qemu_fdt_setprop_cell(fdt, "/", "machine-version-minor", 1); if (ms->numa_state->have_numa_distance) { int size = nb_numa_nodes * nb_numa_nodes * 3 * sizeof(uint32_t); @@ -XXX,XX +XXX,XX @@ static void create_fdt(SBSAMachineState *sms) g_free(nodename); } + + sbsa_fdt_add_gic_node(sms); } #define SBSA_FLASH_SECTOR_SIZE (256 * KiB) -- 2.34.1 From: Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org> We moved from VGA to Bochs to have PCIe card. Signed-off-by: Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- docs/system/arm/sbsa.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/system/arm/sbsa.rst b/docs/system/arm/sbsa.rst index XXXXXXX..XXXXXXX 100644 --- a/docs/system/arm/sbsa.rst +++ b/docs/system/arm/sbsa.rst @@ -XXX,XX +XXX,XX @@ The sbsa-ref board supports: - System bus EHCI controller - CDROM and hard disc on AHCI bus - E1000E ethernet card on PCIe bus - - VGA display adaptor on PCIe bus + - Bochs display adapter on PCIe bus - A generic SBSA watchdog device -- 2.34.1 Another very large pullreq (this one mostly because it has RTH's decodetree conversion series in it), but this should be the last of the really large things in my to-review queue... thanks -- PMM The following changes since commit 83aaec1d5a49f158abaa31797a0f976b3c07e5ca: Merge tag 'pull-tcg-20241212' of https://gitlab.com/rth7680/qemu into staging (2024-12-12 18:45:39 -0500) are available in the Git repository at: https://git.linaro.org/people/pmaydell/qemu-arm.git tags/pull-target-arm-20241213 for you to fetch changes up to 48e652c4bd9570f6f24def25355cb3009a7300f8: target/arm: Simplify condition for tlbi_el2_cp_reginfo[] (2024-12-13 15:41:09 +0000) ---------------------------------------------------------------- target-arm queue: * Finish conversion of A64 decoder to decodetree * Use float_round_to_odd in helper_fcvtx_f64_to_f32 * Move TLBI insn emulation code out to its own source file * docs/system/arm: fix broken links, document undocumented properties * MAINTAINERS: correct an email address ---------------------------------------------------------------- Brian Cain (1): MAINTAINERS: correct my email address Peter Maydell (10): target/arm: Move some TLBI insns to their own source file target/arm: Move TLBI insns for AArch32 EL2 to tlbi_insn_helper.c target/arm: Move AArch64 TLBI insns from v8_cp_reginfo[] target/arm: Move the AArch64 EL2 TLBI insns target/arm: Move AArch64 EL3 TLBI insns target/arm: Move TLBI range insns target/arm: Move the TLBI OS insns to tlb-insns.c. target/arm: Move small helper functions to tlb-insns.c target/arm: Move RME TLB insns to tlb-insns.c target/arm: Simplify condition for tlbi_el2_cp_reginfo[] Pierrick Bouvier (4): docs/system/arm/orangepi: update links docs/system/arm/fby35: document execute-in-place property docs/system/arm/xlnx-versal-virt: document ospi-flash property docs/system/arm/virt: document missing properties Richard Henderson (70): target/arm: Add section labels for "Data Processing (register)" target/arm: Convert UDIV, SDIV to decodetree target/arm: Convert LSLV, LSRV, ASRV, RORV to decodetree target/arm: Convert CRC32, CRC32C to decodetree target/arm: Convert SUBP, IRG, GMI to decodetree target/arm: Convert PACGA to decodetree target/arm: Convert RBIT, REV16, REV32, REV64 to decodetree target/arm: Convert CLZ, CLS to decodetree target/arm: Convert PAC[ID]*, AUT[ID]* to decodetree target/arm: Convert XPAC[ID] to decodetree target/arm: Convert disas_logic_reg to decodetree target/arm: Convert disas_add_sub_ext_reg to decodetree target/arm: Convert disas_add_sub_reg to decodetree target/arm: Convert disas_data_proc_3src to decodetree target/arm: Convert disas_adc_sbc to decodetree target/arm: Convert RMIF to decodetree target/arm: Convert SETF8, SETF16 to decodetree target/arm: Convert CCMP, CCMN to decodetree target/arm: Convert disas_cond_select to decodetree target/arm: Introduce fp_access_check_scalar_hsd target/arm: Introduce fp_access_check_vector_hsd target/arm: Convert FCMP, FCMPE, FCCMP, FCCMPE to decodetree target/arm: Fix decode of fp16 vector fabs, fneg, fsqrt target/arm: Convert FMOV, FABS, FNEG (scalar) to decodetree target/arm: Pass fpstatus to vfp_sqrt* target/arm: Remove helper_sqrt_f16 target/arm: Convert FSQRT (scalar) to decodetree target/arm: Convert FRINT[NPMSAXI] (scalar) to decodetree target/arm: Convert BFCVT to decodetree target/arm: Convert FRINT{32, 64}[ZX] (scalar) to decodetree target/arm: Convert FCVT (scalar) to decodetree target/arm: Convert handle_fpfpcvt to decodetree target/arm: Convert FJCVTZS to decodetree target/arm: Convert handle_fmov to decodetree target/arm: Convert SQABS, SQNEG to decodetree target/arm: Convert ABS, NEG to decodetree target/arm: Introduce gen_gvec_cls, gen_gvec_clz target/arm: Convert CLS, CLZ (vector) to decodetree target/arm: Introduce gen_gvec_cnt, gen_gvec_rbit target/arm: Convert CNT, NOT, RBIT (vector) to decodetree target/arm: Convert CMGT, CMGE, GMLT, GMLE, CMEQ (zero) to decodetree target/arm: Introduce gen_gvec_rev{16,32,64} target/arm: Convert handle_rev to decodetree target/arm: Move helper_neon_addlp_{s8, s16} to neon_helper.c target/arm: Introduce gen_gvec_{s,u}{add,ada}lp target/arm: Convert handle_2misc_pairwise to decodetree target/arm: Remove helper_neon_{add,sub}l_u{16,32} target/arm: Introduce clear_vec target/arm: Convert XTN, SQXTUN, SQXTN, UQXTN to decodetree target/arm: Convert FCVTN, BFCVTN to decodetree target/arm: Convert FCVTXN to decodetree target/arm: Convert SHLL to decodetree target/arm: Implement gen_gvec_fabs, gen_gvec_fneg target/arm: Convert FABS, FNEG (vector) to decodetree target/arm: Convert FSQRT (vector) to decodetree target/arm: Convert FRINT* (vector) to decodetree target/arm: Convert FCVT* (vector, integer) scalar to decodetree target/arm: Convert FCVT* (vector, fixed-point) scalar to decodetree target/arm: Convert [US]CVTF (vector, integer) scalar to decodetree target/arm: Convert [US]CVTF (vector, fixed-point) scalar to decodetree target/arm: Rename helper_gvec_vcvt_[hf][su] with _rz target/arm: Convert [US]CVTF (vector) to decodetree target/arm: Convert FCVTZ[SU] (vector, fixed-point) to decodetree target/arm: Convert FCVT* (vector, integer) to decodetree target/arm: Convert handle_2misc_fcmp_zero to decodetree target/arm: Convert FRECPE, FRECPX, FRSQRTE to decodetree target/arm: Introduce gen_gvec_urecpe, gen_gvec_ursqrte target/arm: Convert URECPE and URSQRTE to decodetree target/arm: Convert FCVTL to decodetree target/arm: Use float_round_to_odd in helper_fcvtx_f64_to_f32 MAINTAINERS | 2 +- docs/system/arm/fby35.rst | 5 + docs/system/arm/orangepi.rst | 4 +- docs/system/arm/virt.rst | 16 + docs/system/arm/xlnx-versal-virt.rst | 3 + target/arm/helper.h | 43 +- target/arm/internals.h | 9 + target/arm/tcg/helper-a64.h | 7 - target/arm/tcg/translate.h | 35 + target/arm/tcg/a64.decode | 502 ++- target/arm/helper.c | 1208 +------- target/arm/tcg-stubs.c | 5 + target/arm/tcg/gengvec.c | 369 +++ target/arm/tcg/helper-a64.c | 122 +- target/arm/tcg/neon_helper.c | 106 +- target/arm/tcg/tlb-insns.c | 1266 ++++++++ target/arm/tcg/translate-a64.c | 5670 +++++++++++----------------------- target/arm/tcg/translate-neon.c | 337 +- target/arm/tcg/translate-vfp.c | 6 +- target/arm/tcg/vec_helper.c | 65 +- target/arm/vfp_helper.c | 16 +- target/arm/tcg/meson.build | 1 + 22 files changed, 4203 insertions(+), 5594 deletions(-) create mode 100644 target/arm/tcg/tlb-insns.c From: Richard Henderson <richard.henderson@linaro.org> At the same time, use ### to separate 3rd-level sections. We already use ### for 4.1.92 Data Processing (immediate), but not the two following two third-level sections: 4.1.93 Branches, and 4.1.94 Loads and stores. Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-2-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 19 +++++++++++++++++-- 1 file changed, 17 insertions(+), 2 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ UBFM . 10 100110 . ...... ...... ..... ..... @bitfield_32 EXTR 1 00 100111 1 0 rm:5 imm:6 rn:5 rd:5 &extract sf=1 EXTR 0 00 100111 0 0 rm:5 0 imm:5 rn:5 rd:5 &extract sf=0 -# Branches +### Branches %imm26 0:s26 !function=times_4 @branch . ..... .......................... &i imm=%imm26 @@ -XXX,XX +XXX,XX @@ HLT 1101 0100 010 ................ 000 00 @i16 # DCPS2 1101 0100 101 ................ 000 10 @i16 # DCPS3 1101 0100 101 ................ 000 11 @i16 -# Loads and stores +### Loads and stores &stxr rn rt rt2 rs sz lasr &stlr rn rt sz lasr @@ -XXX,XX +XXX,XX @@ CPYP 00 011 1 01000 ..... .... 01 ..... ..... @cpy CPYM 00 011 1 01010 ..... .... 01 ..... ..... @cpy CPYE 00 011 1 01100 ..... .... 01 ..... ..... @cpy +### Data Processing (register) + +# Data Processing (2-source) +# Data Processing (1-source) +# Logical (shifted reg) +# Add/subtract (shifted reg) +# Add/subtract (extended reg) +# Add/subtract (carry) +# Rotate right into flags +# Evaluate into flags +# Conditional compare (regster) +# Conditional compare (immediate) +# Conditional select +# Data Processing (3-source) + ### Cryptographic AES AESE 01001110 00 10100 00100 10 ..... ..... @r2r_q1e0 -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-3-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 7 ++++ target/arm/tcg/translate-a64.c | 64 +++++++++++++++++----------------- 2 files changed, 39 insertions(+), 32 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ &r rn &ri rd imm &rri_sf rd rn imm sf +&rrr_sf rd rn rm sf &i imm &rr_e rd rn esz &rri_e rd rn imm esz @@ -XXX,XX +XXX,XX @@ CPYE 00 011 1 01100 ..... .... 01 ..... ..... @cpy ### Data Processing (register) # Data Processing (2-source) + +@rrr_sf sf:1 .......... rm:5 ...... rn:5 rd:5 &rrr_sf + +UDIV . 00 11010110 ..... 00001 0 ..... ..... @rrr_sf +SDIV . 00 11010110 ..... 00001 1 ..... ..... @rrr_sf + # Data Processing (1-source) # Logical (shifted reg) # Add/subtract (shifted reg) diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ TRANS(UQRSHRN_si, do_scalar_shift_imm_narrow, a, uqrshrn_fns, 0, false) TRANS(SQSHRUN_si, do_scalar_shift_imm_narrow, a, sqshrun_fns, MO_SIGN, false) TRANS(SQRSHRUN_si, do_scalar_shift_imm_narrow, a, sqrshrun_fns, MO_SIGN, false) +static bool do_div(DisasContext *s, arg_rrr_sf *a, bool is_signed) +{ + TCGv_i64 tcg_n, tcg_m, tcg_rd; + tcg_rd = cpu_reg(s, a->rd); + + if (!a->sf && is_signed) { + tcg_n = tcg_temp_new_i64(); + tcg_m = tcg_temp_new_i64(); + tcg_gen_ext32s_i64(tcg_n, cpu_reg(s, a->rn)); + tcg_gen_ext32s_i64(tcg_m, cpu_reg(s, a->rm)); + } else { + tcg_n = read_cpu_reg(s, a->rn, a->sf); + tcg_m = read_cpu_reg(s, a->rm, a->sf); + } + + if (is_signed) { + gen_helper_sdiv64(tcg_rd, tcg_n, tcg_m); + } else { + gen_helper_udiv64(tcg_rd, tcg_n, tcg_m); + } + + if (!a->sf) { /* zero extend final result */ + tcg_gen_ext32u_i64(tcg_rd, tcg_rd); + } + return true; +} + +TRANS(SDIV, do_div, a, true) +TRANS(UDIV, do_div, a, false) + /* Shift a TCGv src by TCGv shift_amount, put result in dst. * Note that it is the caller's responsibility to ensure that the * shift amount is in range (ie 0..31 or 0..63) and provide the ARM @@ -XXX,XX +XXX,XX @@ static void disas_data_proc_1src(DisasContext *s, uint32_t insn) #undef MAP } -static void handle_div(DisasContext *s, bool is_signed, unsigned int sf, - unsigned int rm, unsigned int rn, unsigned int rd) -{ - TCGv_i64 tcg_n, tcg_m, tcg_rd; - tcg_rd = cpu_reg(s, rd); - - if (!sf && is_signed) { - tcg_n = tcg_temp_new_i64(); - tcg_m = tcg_temp_new_i64(); - tcg_gen_ext32s_i64(tcg_n, cpu_reg(s, rn)); - tcg_gen_ext32s_i64(tcg_m, cpu_reg(s, rm)); - } else { - tcg_n = read_cpu_reg(s, rn, sf); - tcg_m = read_cpu_reg(s, rm, sf); - } - - if (is_signed) { - gen_helper_sdiv64(tcg_rd, tcg_n, tcg_m); - } else { - gen_helper_udiv64(tcg_rd, tcg_n, tcg_m); - } - - if (!sf) { /* zero extend final result */ - tcg_gen_ext32u_i64(tcg_rd, tcg_rd); - } -} /* LSLV, LSRV, ASRV, RORV */ static void handle_shift_reg(DisasContext *s, @@ -XXX,XX +XXX,XX @@ static void disas_data_proc_2src(DisasContext *s, uint32_t insn) } } break; - case 2: /* UDIV */ - handle_div(s, false, sf, rm, rn, rd); - break; - case 3: /* SDIV */ - handle_div(s, true, sf, rm, rn, rd); - break; case 4: /* IRG */ if (sf == 0 || !dc_isar_feature(aa64_mte_insn_reg, s)) { goto do_unallocated; @@ -XXX,XX +XXX,XX @@ static void disas_data_proc_2src(DisasContext *s, uint32_t insn) } default: do_unallocated: + case 2: /* UDIV */ + case 3: /* SDIV */ unallocated_encoding(s); break; } -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-4-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 4 +++ target/arm/tcg/translate-a64.c | 46 ++++++++++++++++------------------ 2 files changed, 25 insertions(+), 25 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ CPYE 00 011 1 01100 ..... .... 01 ..... ..... @cpy UDIV . 00 11010110 ..... 00001 0 ..... ..... @rrr_sf SDIV . 00 11010110 ..... 00001 1 ..... ..... @rrr_sf +LSLV . 00 11010110 ..... 00100 0 ..... ..... @rrr_sf +LSRV . 00 11010110 ..... 00100 1 ..... ..... @rrr_sf +ASRV . 00 11010110 ..... 00101 0 ..... ..... @rrr_sf +RORV . 00 11010110 ..... 00101 1 ..... ..... @rrr_sf # Data Processing (1-source) # Logical (shifted reg) diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static void shift_reg_imm(TCGv_i64 dst, TCGv_i64 src, int sf, } } +static bool do_shift_reg(DisasContext *s, arg_rrr_sf *a, + enum a64_shift_type shift_type) +{ + TCGv_i64 tcg_shift = tcg_temp_new_i64(); + TCGv_i64 tcg_rd = cpu_reg(s, a->rd); + TCGv_i64 tcg_rn = read_cpu_reg(s, a->rn, a->sf); + + tcg_gen_andi_i64(tcg_shift, cpu_reg(s, a->rm), a->sf ? 63 : 31); + shift_reg(tcg_rd, tcg_rn, a->sf, shift_type, tcg_shift); + return true; +} + +TRANS(LSLV, do_shift_reg, a, A64_SHIFT_TYPE_LSL) +TRANS(LSRV, do_shift_reg, a, A64_SHIFT_TYPE_LSR) +TRANS(ASRV, do_shift_reg, a, A64_SHIFT_TYPE_ASR) +TRANS(RORV, do_shift_reg, a, A64_SHIFT_TYPE_ROR) + /* Logical (shifted register) * 31 30 29 28 24 23 22 21 20 16 15 10 9 5 4 0 * +----+-----+-----------+-------+---+------+--------+------+------+ @@ -XXX,XX +XXX,XX @@ static void disas_data_proc_1src(DisasContext *s, uint32_t insn) } -/* LSLV, LSRV, ASRV, RORV */ -static void handle_shift_reg(DisasContext *s, - enum a64_shift_type shift_type, unsigned int sf, - unsigned int rm, unsigned int rn, unsigned int rd) -{ - TCGv_i64 tcg_shift = tcg_temp_new_i64(); - TCGv_i64 tcg_rd = cpu_reg(s, rd); - TCGv_i64 tcg_rn = read_cpu_reg(s, rn, sf); - - tcg_gen_andi_i64(tcg_shift, cpu_reg(s, rm), sf ? 63 : 31); - shift_reg(tcg_rd, tcg_rn, sf, shift_type, tcg_shift); -} - /* CRC32[BHWX], CRC32C[BHWX] */ static void handle_crc32(DisasContext *s, unsigned int sf, unsigned int sz, bool crc32c, @@ -XXX,XX +XXX,XX @@ static void disas_data_proc_2src(DisasContext *s, uint32_t insn) tcg_gen_or_i64(cpu_reg(s, rd), cpu_reg(s, rm), t); } break; - case 8: /* LSLV */ - handle_shift_reg(s, A64_SHIFT_TYPE_LSL, sf, rm, rn, rd); - break; - case 9: /* LSRV */ - handle_shift_reg(s, A64_SHIFT_TYPE_LSR, sf, rm, rn, rd); - break; - case 10: /* ASRV */ - handle_shift_reg(s, A64_SHIFT_TYPE_ASR, sf, rm, rn, rd); - break; - case 11: /* RORV */ - handle_shift_reg(s, A64_SHIFT_TYPE_ROR, sf, rm, rn, rd); - break; case 12: /* PACGA */ if (sf == 0 || !dc_isar_feature(aa64_pauth, s)) { goto do_unallocated; @@ -XXX,XX +XXX,XX @@ static void disas_data_proc_2src(DisasContext *s, uint32_t insn) do_unallocated: case 2: /* UDIV */ case 3: /* SDIV */ + case 8: /* LSLV */ + case 9: /* LSRV */ + case 10: /* ASRV */ + case 11: /* RORV */ unallocated_encoding(s); break; } -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-5-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 12 ++++ target/arm/tcg/translate-a64.c | 101 +++++++++++++-------------------- 2 files changed, 53 insertions(+), 60 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ @rr_d ........ ... ..... ...... rn:5 rd:5 &rr_e esz=3 @rr_sd ........ ... ..... ...... rn:5 rd:5 &rr_e esz=%esz_sd +@rrr_b ........ ... rm:5 ...... rn:5 rd:5 &rrr_e esz=0 @rrr_h ........ ... rm:5 ...... rn:5 rd:5 &rrr_e esz=1 +@rrr_s ........ ... rm:5 ...... rn:5 rd:5 &rrr_e esz=2 @rrr_d ........ ... rm:5 ...... rn:5 rd:5 &rrr_e esz=3 @rrr_sd ........ ... rm:5 ...... rn:5 rd:5 &rrr_e esz=%esz_sd @rrr_hsd ........ ... rm:5 ...... rn:5 rd:5 &rrr_e esz=%esz_hsd @@ -XXX,XX +XXX,XX @@ LSRV . 00 11010110 ..... 00100 1 ..... ..... @rrr_sf ASRV . 00 11010110 ..... 00101 0 ..... ..... @rrr_sf RORV . 00 11010110 ..... 00101 1 ..... ..... @rrr_sf +CRC32 0 00 11010110 ..... 0100 00 ..... ..... @rrr_b +CRC32 0 00 11010110 ..... 0100 01 ..... ..... @rrr_h +CRC32 0 00 11010110 ..... 0100 10 ..... ..... @rrr_s +CRC32 1 00 11010110 ..... 0100 11 ..... ..... @rrr_d + +CRC32C 0 00 11010110 ..... 0101 00 ..... ..... @rrr_b +CRC32C 0 00 11010110 ..... 0101 01 ..... ..... @rrr_h +CRC32C 0 00 11010110 ..... 0101 10 ..... ..... @rrr_s +CRC32C 1 00 11010110 ..... 0101 11 ..... ..... @rrr_d + # Data Processing (1-source) # Logical (shifted reg) # Add/subtract (shifted reg) diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ TRANS(LSRV, do_shift_reg, a, A64_SHIFT_TYPE_LSR) TRANS(ASRV, do_shift_reg, a, A64_SHIFT_TYPE_ASR) TRANS(RORV, do_shift_reg, a, A64_SHIFT_TYPE_ROR) +static bool do_crc32(DisasContext *s, arg_rrr_e *a, bool crc32c) +{ + TCGv_i64 tcg_acc, tcg_val, tcg_rd; + TCGv_i32 tcg_bytes; + + switch (a->esz) { + case MO_8: + case MO_16: + case MO_32: + tcg_val = tcg_temp_new_i64(); + tcg_gen_extract_i64(tcg_val, cpu_reg(s, a->rm), 0, 8 << a->esz); + break; + case MO_64: + tcg_val = cpu_reg(s, a->rm); + break; + default: + g_assert_not_reached(); + } + tcg_acc = cpu_reg(s, a->rn); + tcg_bytes = tcg_constant_i32(1 << a->esz); + tcg_rd = cpu_reg(s, a->rd); + + if (crc32c) { + gen_helper_crc32c_64(tcg_rd, tcg_acc, tcg_val, tcg_bytes); + } else { + gen_helper_crc32_64(tcg_rd, tcg_acc, tcg_val, tcg_bytes); + } + return true; +} + +TRANS_FEAT(CRC32, aa64_crc32, do_crc32, a, false) +TRANS_FEAT(CRC32C, aa64_crc32, do_crc32, a, true) + /* Logical (shifted register) * 31 30 29 28 24 23 22 21 20 16 15 10 9 5 4 0 * +----+-----+-----------+-------+---+------+--------+------+------+ @@ -XXX,XX +XXX,XX @@ static void disas_data_proc_1src(DisasContext *s, uint32_t insn) } -/* CRC32[BHWX], CRC32C[BHWX] */ -static void handle_crc32(DisasContext *s, - unsigned int sf, unsigned int sz, bool crc32c, - unsigned int rm, unsigned int rn, unsigned int rd) -{ - TCGv_i64 tcg_acc, tcg_val; - TCGv_i32 tcg_bytes; - - if (!dc_isar_feature(aa64_crc32, s) - || (sf == 1 && sz != 3) - || (sf == 0 && sz == 3)) { - unallocated_encoding(s); - return; - } - - if (sz == 3) { - tcg_val = cpu_reg(s, rm); - } else { - uint64_t mask; - switch (sz) { - case 0: - mask = 0xFF; - break; - case 1: - mask = 0xFFFF; - break; - case 2: - mask = 0xFFFFFFFF; - break; - default: - g_assert_not_reached(); - } - tcg_val = tcg_temp_new_i64(); - tcg_gen_andi_i64(tcg_val, cpu_reg(s, rm), mask); - } - - tcg_acc = cpu_reg(s, rn); - tcg_bytes = tcg_constant_i32(1 << sz); - - if (crc32c) { - gen_helper_crc32c_64(cpu_reg(s, rd), tcg_acc, tcg_val, tcg_bytes); - } else { - gen_helper_crc32_64(cpu_reg(s, rd), tcg_acc, tcg_val, tcg_bytes); - } -} - /* Data-processing (2 source) * 31 30 29 28 21 20 16 15 10 9 5 4 0 * +----+---+---+-----------------+------+--------+------+------+ @@ -XXX,XX +XXX,XX @@ static void disas_data_proc_2src(DisasContext *s, uint32_t insn) gen_helper_pacga(cpu_reg(s, rd), tcg_env, cpu_reg(s, rn), cpu_reg_sp(s, rm)); break; - case 16: - case 17: - case 18: - case 19: - case 20: - case 21: - case 22: - case 23: /* CRC32 */ - { - int sz = extract32(opcode, 0, 2); - bool crc32c = extract32(opcode, 2, 1); - handle_crc32(s, sf, sz, crc32c, rm, rn, rd); - break; - } default: do_unallocated: case 2: /* UDIV */ @@ -XXX,XX +XXX,XX @@ static void disas_data_proc_2src(DisasContext *s, uint32_t insn) case 9: /* LSRV */ case 10: /* ASRV */ case 11: /* RORV */ + case 16: + case 17: + case 18: + case 19: + case 20: + case 21: + case 22: + case 23: /* CRC32 */ unallocated_encoding(s); break; } -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-6-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 7 +++ target/arm/tcg/translate-a64.c | 94 +++++++++++++++++++--------------- 2 files changed, 59 insertions(+), 42 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ %hlm 11:1 20:2 &r rn +&rrr rd rn rm &ri rd imm &rri_sf rd rn imm sf &rrr_sf rd rn rm sf @@ -XXX,XX +XXX,XX @@ CPYE 00 011 1 01100 ..... .... 01 ..... ..... @cpy # Data Processing (2-source) +@rrr . .......... rm:5 ...... rn:5 rd:5 &rrr @rrr_sf sf:1 .......... rm:5 ...... rn:5 rd:5 &rrr_sf UDIV . 00 11010110 ..... 00001 0 ..... ..... @rrr_sf @@ -XXX,XX +XXX,XX @@ CRC32C 0 00 11010110 ..... 0101 01 ..... ..... @rrr_h CRC32C 0 00 11010110 ..... 0101 10 ..... ..... @rrr_s CRC32C 1 00 11010110 ..... 0101 11 ..... ..... @rrr_d +SUBP 1 00 11010110 ..... 000000 ..... ..... @rrr +SUBPS 1 01 11010110 ..... 000000 ..... ..... @rrr +IRG 1 00 11010110 ..... 000100 ..... ..... @rrr +GMI 1 00 11010110 ..... 000101 ..... ..... @rrr + # Data Processing (1-source) # Logical (shifted reg) # Add/subtract (shifted reg) diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static bool do_crc32(DisasContext *s, arg_rrr_e *a, bool crc32c) TRANS_FEAT(CRC32, aa64_crc32, do_crc32, a, false) TRANS_FEAT(CRC32C, aa64_crc32, do_crc32, a, true) +static bool do_subp(DisasContext *s, arg_rrr *a, bool setflag) +{ + TCGv_i64 tcg_n = read_cpu_reg_sp(s, a->rn, true); + TCGv_i64 tcg_m = read_cpu_reg_sp(s, a->rm, true); + TCGv_i64 tcg_d = cpu_reg(s, a->rd); + + tcg_gen_sextract_i64(tcg_n, tcg_n, 0, 56); + tcg_gen_sextract_i64(tcg_m, tcg_m, 0, 56); + + if (setflag) { + gen_sub_CC(true, tcg_d, tcg_n, tcg_m); + } else { + tcg_gen_sub_i64(tcg_d, tcg_n, tcg_m); + } + return true; +} + +TRANS_FEAT(SUBP, aa64_mte_insn_reg, do_subp, a, false) +TRANS_FEAT(SUBPS, aa64_mte_insn_reg, do_subp, a, true) + +static bool trans_IRG(DisasContext *s, arg_rrr *a) +{ + if (dc_isar_feature(aa64_mte_insn_reg, s)) { + TCGv_i64 tcg_rd = cpu_reg_sp(s, a->rd); + TCGv_i64 tcg_rn = cpu_reg_sp(s, a->rn); + + if (s->ata[0]) { + gen_helper_irg(tcg_rd, tcg_env, tcg_rn, cpu_reg(s, a->rm)); + } else { + gen_address_with_allocation_tag0(tcg_rd, tcg_rn); + } + return true; + } + return false; +} + +static bool trans_GMI(DisasContext *s, arg_rrr *a) +{ + if (dc_isar_feature(aa64_mte_insn_reg, s)) { + TCGv_i64 t = tcg_temp_new_i64(); + + tcg_gen_extract_i64(t, cpu_reg_sp(s, a->rn), 56, 4); + tcg_gen_shl_i64(t, tcg_constant_i64(1), t); + tcg_gen_or_i64(cpu_reg(s, a->rd), cpu_reg(s, a->rm), t); + return true; + } + return false; +} + /* Logical (shifted register) * 31 30 29 28 24 23 22 21 20 16 15 10 9 5 4 0 * +----+-----+-----------+-------+---+------+--------+------+------+ @@ -XXX,XX +XXX,XX @@ static void disas_data_proc_2src(DisasContext *s, uint32_t insn) } switch (opcode) { - case 0: /* SUBP(S) */ - if (sf == 0 || !dc_isar_feature(aa64_mte_insn_reg, s)) { - goto do_unallocated; - } else { - TCGv_i64 tcg_n, tcg_m, tcg_d; - - tcg_n = read_cpu_reg_sp(s, rn, true); - tcg_m = read_cpu_reg_sp(s, rm, true); - tcg_gen_sextract_i64(tcg_n, tcg_n, 0, 56); - tcg_gen_sextract_i64(tcg_m, tcg_m, 0, 56); - tcg_d = cpu_reg(s, rd); - - if (setflag) { - gen_sub_CC(true, tcg_d, tcg_n, tcg_m); - } else { - tcg_gen_sub_i64(tcg_d, tcg_n, tcg_m); - } - } - break; - case 4: /* IRG */ - if (sf == 0 || !dc_isar_feature(aa64_mte_insn_reg, s)) { - goto do_unallocated; - } - if (s->ata[0]) { - gen_helper_irg(cpu_reg_sp(s, rd), tcg_env, - cpu_reg_sp(s, rn), cpu_reg(s, rm)); - } else { - gen_address_with_allocation_tag0(cpu_reg_sp(s, rd), - cpu_reg_sp(s, rn)); - } - break; - case 5: /* GMI */ - if (sf == 0 || !dc_isar_feature(aa64_mte_insn_reg, s)) { - goto do_unallocated; - } else { - TCGv_i64 t = tcg_temp_new_i64(); - - tcg_gen_extract_i64(t, cpu_reg_sp(s, rn), 56, 4); - tcg_gen_shl_i64(t, tcg_constant_i64(1), t); - tcg_gen_or_i64(cpu_reg(s, rd), cpu_reg(s, rm), t); - } - break; case 12: /* PACGA */ if (sf == 0 || !dc_isar_feature(aa64_pauth, s)) { goto do_unallocated; @@ -XXX,XX +XXX,XX @@ static void disas_data_proc_2src(DisasContext *s, uint32_t insn) break; default: do_unallocated: + case 0: /* SUBP(S) */ case 2: /* UDIV */ case 3: /* SDIV */ + case 4: /* IRG */ + case 5: /* GMI */ case 8: /* LSLV */ case 9: /* LSRV */ case 10: /* ASRV */ -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Remove disas_data_proc_2src, as this was the last insn decoded by that function. Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-7-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 2 ++ target/arm/tcg/translate-a64.c | 65 ++++++---------------------------- 2 files changed, 13 insertions(+), 54 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ SUBPS 1 01 11010110 ..... 000000 ..... ..... @rrr IRG 1 00 11010110 ..... 000100 ..... ..... @rrr GMI 1 00 11010110 ..... 000101 ..... ..... @rrr +PACGA 1 00 11010110 ..... 001100 ..... ..... @rrr + # Data Processing (1-source) # Logical (shifted reg) # Add/subtract (shifted reg) diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static bool trans_GMI(DisasContext *s, arg_rrr *a) return false; } +static bool trans_PACGA(DisasContext *s, arg_rrr *a) +{ + if (dc_isar_feature(aa64_pauth, s)) { + gen_helper_pacga(cpu_reg(s, a->rd), tcg_env, + cpu_reg(s, a->rn), cpu_reg_sp(s, a->rm)); + return true; + } + return false; +} + /* Logical (shifted register) * 31 30 29 28 24 23 22 21 20 16 15 10 9 5 4 0 * +----+-----+-----------+-------+---+------+--------+------+------+ @@ -XXX,XX +XXX,XX @@ static void disas_data_proc_1src(DisasContext *s, uint32_t insn) } -/* Data-processing (2 source) - * 31 30 29 28 21 20 16 15 10 9 5 4 0 - * +----+---+---+-----------------+------+--------+------+------+ - * | sf | 0 | S | 1 1 0 1 0 1 1 0 | Rm | opcode | Rn | Rd | - * +----+---+---+-----------------+------+--------+------+------+ - */ -static void disas_data_proc_2src(DisasContext *s, uint32_t insn) -{ - unsigned int sf, rm, opcode, rn, rd, setflag; - sf = extract32(insn, 31, 1); - setflag = extract32(insn, 29, 1); - rm = extract32(insn, 16, 5); - opcode = extract32(insn, 10, 6); - rn = extract32(insn, 5, 5); - rd = extract32(insn, 0, 5); - - if (setflag && opcode != 0) { - unallocated_encoding(s); - return; - } - - switch (opcode) { - case 12: /* PACGA */ - if (sf == 0 || !dc_isar_feature(aa64_pauth, s)) { - goto do_unallocated; - } - gen_helper_pacga(cpu_reg(s, rd), tcg_env, - cpu_reg(s, rn), cpu_reg_sp(s, rm)); - break; - default: - do_unallocated: - case 0: /* SUBP(S) */ - case 2: /* UDIV */ - case 3: /* SDIV */ - case 4: /* IRG */ - case 5: /* GMI */ - case 8: /* LSLV */ - case 9: /* LSRV */ - case 10: /* ASRV */ - case 11: /* RORV */ - case 16: - case 17: - case 18: - case 19: - case 20: - case 21: - case 22: - case 23: /* CRC32 */ - unallocated_encoding(s); - break; - } -} - /* * Data processing - register * 31 30 29 28 25 21 20 16 10 0 @@ -XXX,XX +XXX,XX @@ static void disas_data_proc_reg(DisasContext *s, uint32_t insn) if (op0) { /* (1 source) */ disas_data_proc_1src(s, insn); } else { /* (2 source) */ - disas_data_proc_2src(s, insn); + goto do_unallocated; } break; case 0x8 ... 0xf: /* (3 source) */ -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-8-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 11 +++ target/arm/tcg/translate-a64.c | 137 +++++++++++++++------------------ 2 files changed, 72 insertions(+), 76 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ &r rn &rrr rd rn rm &ri rd imm +&rr rd rn +&rr_sf rd rn sf &rri_sf rd rn imm sf &rrr_sf rd rn rm sf &i imm @@ -XXX,XX +XXX,XX @@ GMI 1 00 11010110 ..... 000101 ..... ..... @rrr PACGA 1 00 11010110 ..... 001100 ..... ..... @rrr # Data Processing (1-source) + +@rr . .......... ..... ...... rn:5 rd:5 &rr +@rr_sf sf:1 .......... ..... ...... rn:5 rd:5 &rr_sf + +RBIT . 10 11010110 00000 000000 ..... ..... @rr_sf +REV16 . 10 11010110 00000 000001 ..... ..... @rr_sf +REV32 . 10 11010110 00000 000010 ..... ..... @rr_sf +REV64 1 10 11010110 00000 000011 ..... ..... @rr + # Logical (shifted reg) # Add/subtract (shifted reg) # Add/subtract (extended reg) diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static bool trans_PACGA(DisasContext *s, arg_rrr *a) return false; } +typedef void ArithOneOp(TCGv_i64, TCGv_i64); + +static bool gen_rr(DisasContext *s, int rd, int rn, ArithOneOp fn) +{ + fn(cpu_reg(s, rd), cpu_reg(s, rn)); + return true; +} + +static void gen_rbit32(TCGv_i64 tcg_rd, TCGv_i64 tcg_rn) +{ + TCGv_i32 t32 = tcg_temp_new_i32(); + + tcg_gen_extrl_i64_i32(t32, tcg_rn); + gen_helper_rbit(t32, t32); + tcg_gen_extu_i32_i64(tcg_rd, t32); +} + +static void gen_rev16_xx(TCGv_i64 tcg_rd, TCGv_i64 tcg_rn, TCGv_i64 mask) +{ + TCGv_i64 tcg_tmp = tcg_temp_new_i64(); + + tcg_gen_shri_i64(tcg_tmp, tcg_rn, 8); + tcg_gen_and_i64(tcg_rd, tcg_rn, mask); + tcg_gen_and_i64(tcg_tmp, tcg_tmp, mask); + tcg_gen_shli_i64(tcg_rd, tcg_rd, 8); + tcg_gen_or_i64(tcg_rd, tcg_rd, tcg_tmp); +} + +static void gen_rev16_32(TCGv_i64 tcg_rd, TCGv_i64 tcg_rn) +{ + gen_rev16_xx(tcg_rd, tcg_rn, tcg_constant_i64(0x00ff00ff)); +} + +static void gen_rev16_64(TCGv_i64 tcg_rd, TCGv_i64 tcg_rn) +{ + gen_rev16_xx(tcg_rd, tcg_rn, tcg_constant_i64(0x00ff00ff00ff00ffull)); +} + +static void gen_rev_32(TCGv_i64 tcg_rd, TCGv_i64 tcg_rn) +{ + tcg_gen_bswap32_i64(tcg_rd, tcg_rn, TCG_BSWAP_OZ); +} + +static void gen_rev32(TCGv_i64 tcg_rd, TCGv_i64 tcg_rn) +{ + tcg_gen_bswap64_i64(tcg_rd, tcg_rn); + tcg_gen_rotri_i64(tcg_rd, tcg_rd, 32); +} + +TRANS(RBIT, gen_rr, a->rd, a->rn, a->sf ? gen_helper_rbit64 : gen_rbit32) +TRANS(REV16, gen_rr, a->rd, a->rn, a->sf ? gen_rev16_64 : gen_rev16_32) +TRANS(REV32, gen_rr, a->rd, a->rn, a->sf ? gen_rev32 : gen_rev_32) +TRANS(REV64, gen_rr, a->rd, a->rn, tcg_gen_bswap64_i64) + /* Logical (shifted register) * 31 30 29 28 24 23 22 21 20 16 15 10 9 5 4 0 * +----+-----+-----------+-------+---+------+--------+------+------+ @@ -XXX,XX +XXX,XX @@ static void handle_cls(DisasContext *s, unsigned int sf, } } -static void handle_rbit(DisasContext *s, unsigned int sf, - unsigned int rn, unsigned int rd) -{ - TCGv_i64 tcg_rd, tcg_rn; - tcg_rd = cpu_reg(s, rd); - tcg_rn = cpu_reg(s, rn); - - if (sf) { - gen_helper_rbit64(tcg_rd, tcg_rn); - } else { - TCGv_i32 tcg_tmp32 = tcg_temp_new_i32(); - tcg_gen_extrl_i64_i32(tcg_tmp32, tcg_rn); - gen_helper_rbit(tcg_tmp32, tcg_tmp32); - tcg_gen_extu_i32_i64(tcg_rd, tcg_tmp32); - } -} - -/* REV with sf==1, opcode==3 ("REV64") */ -static void handle_rev64(DisasContext *s, unsigned int sf, - unsigned int rn, unsigned int rd) -{ - if (!sf) { - unallocated_encoding(s); - return; - } - tcg_gen_bswap64_i64(cpu_reg(s, rd), cpu_reg(s, rn)); -} - -/* REV with sf==0, opcode==2 - * REV32 (sf==1, opcode==2) - */ -static void handle_rev32(DisasContext *s, unsigned int sf, - unsigned int rn, unsigned int rd) -{ - TCGv_i64 tcg_rd = cpu_reg(s, rd); - TCGv_i64 tcg_rn = cpu_reg(s, rn); - - if (sf) { - tcg_gen_bswap64_i64(tcg_rd, tcg_rn); - tcg_gen_rotri_i64(tcg_rd, tcg_rd, 32); - } else { - tcg_gen_bswap32_i64(tcg_rd, tcg_rn, TCG_BSWAP_OZ); - } -} - -/* REV16 (opcode==1) */ -static void handle_rev16(DisasContext *s, unsigned int sf, - unsigned int rn, unsigned int rd) -{ - TCGv_i64 tcg_rd = cpu_reg(s, rd); - TCGv_i64 tcg_tmp = tcg_temp_new_i64(); - TCGv_i64 tcg_rn = read_cpu_reg(s, rn, sf); - TCGv_i64 mask = tcg_constant_i64(sf ? 0x00ff00ff00ff00ffull : 0x00ff00ff); - - tcg_gen_shri_i64(tcg_tmp, tcg_rn, 8); - tcg_gen_and_i64(tcg_rd, tcg_rn, mask); - tcg_gen_and_i64(tcg_tmp, tcg_tmp, mask); - tcg_gen_shli_i64(tcg_rd, tcg_rd, 8); - tcg_gen_or_i64(tcg_rd, tcg_rd, tcg_tmp); -} - /* Data-processing (1 source) * 31 30 29 28 21 20 16 15 10 9 5 4 0 * +----+---+---+-----------------+---------+--------+------+------+ @@ -XXX,XX +XXX,XX @@ static void disas_data_proc_1src(DisasContext *s, uint32_t insn) #define MAP(SF, O2, O1) ((SF) | (O1 << 1) | (O2 << 7)) switch (MAP(sf, opcode2, opcode)) { - case MAP(0, 0x00, 0x00): /* RBIT */ - case MAP(1, 0x00, 0x00): - handle_rbit(s, sf, rn, rd); - break; - case MAP(0, 0x00, 0x01): /* REV16 */ - case MAP(1, 0x00, 0x01): - handle_rev16(s, sf, rn, rd); - break; - case MAP(0, 0x00, 0x02): /* REV/REV32 */ - case MAP(1, 0x00, 0x02): - handle_rev32(s, sf, rn, rd); - break; - case MAP(1, 0x00, 0x03): /* REV64 */ - handle_rev64(s, sf, rn, rd); - break; case MAP(0, 0x00, 0x04): /* CLZ */ case MAP(1, 0x00, 0x04): handle_clz(s, sf, rn, rd); @@ -XXX,XX +XXX,XX @@ static void disas_data_proc_1src(DisasContext *s, uint32_t insn) break; default: do_unallocated: + case MAP(0, 0x00, 0x00): /* RBIT */ + case MAP(1, 0x00, 0x00): + case MAP(0, 0x00, 0x01): /* REV16 */ + case MAP(1, 0x00, 0x01): + case MAP(0, 0x00, 0x02): /* REV/REV32 */ + case MAP(1, 0x00, 0x02): + case MAP(1, 0x00, 0x03): /* REV64 */ unallocated_encoding(s); break; } -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-9-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 3 ++ target/arm/tcg/translate-a64.c | 72 ++++++++++++++-------------------- 2 files changed, 33 insertions(+), 42 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ REV16 . 10 11010110 00000 000001 ..... ..... @rr_sf REV32 . 10 11010110 00000 000010 ..... ..... @rr_sf REV64 1 10 11010110 00000 000011 ..... ..... @rr +CLZ . 10 11010110 00000 000100 ..... ..... @rr_sf +CLS . 10 11010110 00000 000101 ..... ..... @rr_sf + # Logical (shifted reg) # Add/subtract (shifted reg) # Add/subtract (extended reg) diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ TRANS(REV16, gen_rr, a->rd, a->rn, a->sf ? gen_rev16_64 : gen_rev16_32) TRANS(REV32, gen_rr, a->rd, a->rn, a->sf ? gen_rev32 : gen_rev_32) TRANS(REV64, gen_rr, a->rd, a->rn, tcg_gen_bswap64_i64) +static void gen_clz32(TCGv_i64 tcg_rd, TCGv_i64 tcg_rn) +{ + TCGv_i32 t32 = tcg_temp_new_i32(); + + tcg_gen_extrl_i64_i32(t32, tcg_rn); + tcg_gen_clzi_i32(t32, t32, 32); + tcg_gen_extu_i32_i64(tcg_rd, t32); +} + +static void gen_clz64(TCGv_i64 tcg_rd, TCGv_i64 tcg_rn) +{ + tcg_gen_clzi_i64(tcg_rd, tcg_rn, 64); +} + +static void gen_cls32(TCGv_i64 tcg_rd, TCGv_i64 tcg_rn) +{ + TCGv_i32 t32 = tcg_temp_new_i32(); + + tcg_gen_extrl_i64_i32(t32, tcg_rn); + tcg_gen_clrsb_i32(t32, t32); + tcg_gen_extu_i32_i64(tcg_rd, t32); +} + +TRANS(CLZ, gen_rr, a->rd, a->rn, a->sf ? gen_clz64 : gen_clz32) +TRANS(CLS, gen_rr, a->rd, a->rn, a->sf ? tcg_gen_clrsb_i64 : gen_cls32) + /* Logical (shifted register) * 31 30 29 28 24 23 22 21 20 16 15 10 9 5 4 0 * +----+-----+-----------+-------+---+------+--------+------+------+ @@ -XXX,XX +XXX,XX @@ static void disas_cond_select(DisasContext *s, uint32_t insn) } } -static void handle_clz(DisasContext *s, unsigned int sf, - unsigned int rn, unsigned int rd) -{ - TCGv_i64 tcg_rd, tcg_rn; - tcg_rd = cpu_reg(s, rd); - tcg_rn = cpu_reg(s, rn); - - if (sf) { - tcg_gen_clzi_i64(tcg_rd, tcg_rn, 64); - } else { - TCGv_i32 tcg_tmp32 = tcg_temp_new_i32(); - tcg_gen_extrl_i64_i32(tcg_tmp32, tcg_rn); - tcg_gen_clzi_i32(tcg_tmp32, tcg_tmp32, 32); - tcg_gen_extu_i32_i64(tcg_rd, tcg_tmp32); - } -} - -static void handle_cls(DisasContext *s, unsigned int sf, - unsigned int rn, unsigned int rd) -{ - TCGv_i64 tcg_rd, tcg_rn; - tcg_rd = cpu_reg(s, rd); - tcg_rn = cpu_reg(s, rn); - - if (sf) { - tcg_gen_clrsb_i64(tcg_rd, tcg_rn); - } else { - TCGv_i32 tcg_tmp32 = tcg_temp_new_i32(); - tcg_gen_extrl_i64_i32(tcg_tmp32, tcg_rn); - tcg_gen_clrsb_i32(tcg_tmp32, tcg_tmp32); - tcg_gen_extu_i32_i64(tcg_rd, tcg_tmp32); - } -} - /* Data-processing (1 source) * 31 30 29 28 21 20 16 15 10 9 5 4 0 * +----+---+---+-----------------+---------+--------+------+------+ @@ -XXX,XX +XXX,XX @@ static void disas_data_proc_1src(DisasContext *s, uint32_t insn) #define MAP(SF, O2, O1) ((SF) | (O1 << 1) | (O2 << 7)) switch (MAP(sf, opcode2, opcode)) { - case MAP(0, 0x00, 0x04): /* CLZ */ - case MAP(1, 0x00, 0x04): - handle_clz(s, sf, rn, rd); - break; - case MAP(0, 0x00, 0x05): /* CLS */ - case MAP(1, 0x00, 0x05): - handle_cls(s, sf, rn, rd); - break; case MAP(1, 0x01, 0x00): /* PACIA */ if (s->pauth_active) { tcg_rd = cpu_reg(s, rd); @@ -XXX,XX +XXX,XX @@ static void disas_data_proc_1src(DisasContext *s, uint32_t insn) case MAP(0, 0x00, 0x02): /* REV/REV32 */ case MAP(1, 0x00, 0x02): case MAP(1, 0x00, 0x03): /* REV64 */ + case MAP(0, 0x00, 0x04): /* CLZ */ + case MAP(1, 0x00, 0x04): + case MAP(0, 0x00, 0x05): /* CLS */ + case MAP(1, 0x00, 0x05): unallocated_encoding(s); break; } -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> This includes PACIA, PACIZA, PACIB, PACIZB, PACDA, PACDZA, PACDB, PACDZB, AUTIA, AUTIZA, AUTIB, AUTIZB, AUTDA, AUTDZA, AUTDB, AUTDZB. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-10-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 13 +++ target/arm/tcg/translate-a64.c | 173 +++++++++------------------------ 2 files changed, 58 insertions(+), 128 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ REV64 1 10 11010110 00000 000011 ..... ..... @rr CLZ . 10 11010110 00000 000100 ..... ..... @rr_sf CLS . 10 11010110 00000 000101 ..... ..... @rr_sf +&pacaut rd rn z +@pacaut . .. ........ ..... .. z:1 ... rn:5 rd:5 &pacaut + +PACIA 1 10 11010110 00001 00.000 ..... ..... @pacaut +PACIB 1 10 11010110 00001 00.001 ..... ..... @pacaut +PACDA 1 10 11010110 00001 00.010 ..... ..... @pacaut +PACDB 1 10 11010110 00001 00.011 ..... ..... @pacaut + +AUTIA 1 10 11010110 00001 00.100 ..... ..... @pacaut +AUTIB 1 10 11010110 00001 00.101 ..... ..... @pacaut +AUTDA 1 10 11010110 00001 00.110 ..... ..... @pacaut +AUTDB 1 10 11010110 00001 00.111 ..... ..... @pacaut + # Logical (shifted reg) # Add/subtract (shifted reg) # Add/subtract (extended reg) diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static void gen_cls32(TCGv_i64 tcg_rd, TCGv_i64 tcg_rn) TRANS(CLZ, gen_rr, a->rd, a->rn, a->sf ? gen_clz64 : gen_clz32) TRANS(CLS, gen_rr, a->rd, a->rn, a->sf ? tcg_gen_clrsb_i64 : gen_cls32) +static bool gen_pacaut(DisasContext *s, arg_pacaut *a, NeonGenTwo64OpEnvFn fn) +{ + TCGv_i64 tcg_rd, tcg_rn; + + if (a->z) { + if (a->rn != 31) { + return false; + } + tcg_rn = tcg_constant_i64(0); + } else { + tcg_rn = cpu_reg_sp(s, a->rn); + } + if (s->pauth_active) { + tcg_rd = cpu_reg(s, a->rd); + fn(tcg_rd, tcg_env, tcg_rd, tcg_rn); + } + return true; +} + +TRANS_FEAT(PACIA, aa64_pauth, gen_pacaut, a, gen_helper_pacia) +TRANS_FEAT(PACIB, aa64_pauth, gen_pacaut, a, gen_helper_pacib) +TRANS_FEAT(PACDA, aa64_pauth, gen_pacaut, a, gen_helper_pacda) +TRANS_FEAT(PACDB, aa64_pauth, gen_pacaut, a, gen_helper_pacdb) + +TRANS_FEAT(AUTIA, aa64_pauth, gen_pacaut, a, gen_helper_autia) +TRANS_FEAT(AUTIB, aa64_pauth, gen_pacaut, a, gen_helper_autib) +TRANS_FEAT(AUTDA, aa64_pauth, gen_pacaut, a, gen_helper_autda) +TRANS_FEAT(AUTDB, aa64_pauth, gen_pacaut, a, gen_helper_autdb) + /* Logical (shifted register) * 31 30 29 28 24 23 22 21 20 16 15 10 9 5 4 0 * +----+-----+-----------+-------+---+------+--------+------+------+ @@ -XXX,XX +XXX,XX @@ static void disas_data_proc_1src(DisasContext *s, uint32_t insn) #define MAP(SF, O2, O1) ((SF) | (O1 << 1) | (O2 << 7)) switch (MAP(sf, opcode2, opcode)) { - case MAP(1, 0x01, 0x00): /* PACIA */ - if (s->pauth_active) { - tcg_rd = cpu_reg(s, rd); - gen_helper_pacia(tcg_rd, tcg_env, tcg_rd, cpu_reg_sp(s, rn)); - } else if (!dc_isar_feature(aa64_pauth, s)) { - goto do_unallocated; - } - break; - case MAP(1, 0x01, 0x01): /* PACIB */ - if (s->pauth_active) { - tcg_rd = cpu_reg(s, rd); - gen_helper_pacib(tcg_rd, tcg_env, tcg_rd, cpu_reg_sp(s, rn)); - } else if (!dc_isar_feature(aa64_pauth, s)) { - goto do_unallocated; - } - break; - case MAP(1, 0x01, 0x02): /* PACDA */ - if (s->pauth_active) { - tcg_rd = cpu_reg(s, rd); - gen_helper_pacda(tcg_rd, tcg_env, tcg_rd, cpu_reg_sp(s, rn)); - } else if (!dc_isar_feature(aa64_pauth, s)) { - goto do_unallocated; - } - break; - case MAP(1, 0x01, 0x03): /* PACDB */ - if (s->pauth_active) { - tcg_rd = cpu_reg(s, rd); - gen_helper_pacdb(tcg_rd, tcg_env, tcg_rd, cpu_reg_sp(s, rn)); - } else if (!dc_isar_feature(aa64_pauth, s)) { - goto do_unallocated; - } - break; - case MAP(1, 0x01, 0x04): /* AUTIA */ - if (s->pauth_active) { - tcg_rd = cpu_reg(s, rd); - gen_helper_autia(tcg_rd, tcg_env, tcg_rd, cpu_reg_sp(s, rn)); - } else if (!dc_isar_feature(aa64_pauth, s)) { - goto do_unallocated; - } - break; - case MAP(1, 0x01, 0x05): /* AUTIB */ - if (s->pauth_active) { - tcg_rd = cpu_reg(s, rd); - gen_helper_autib(tcg_rd, tcg_env, tcg_rd, cpu_reg_sp(s, rn)); - } else if (!dc_isar_feature(aa64_pauth, s)) { - goto do_unallocated; - } - break; - case MAP(1, 0x01, 0x06): /* AUTDA */ - if (s->pauth_active) { - tcg_rd = cpu_reg(s, rd); - gen_helper_autda(tcg_rd, tcg_env, tcg_rd, cpu_reg_sp(s, rn)); - } else if (!dc_isar_feature(aa64_pauth, s)) { - goto do_unallocated; - } - break; - case MAP(1, 0x01, 0x07): /* AUTDB */ - if (s->pauth_active) { - tcg_rd = cpu_reg(s, rd); - gen_helper_autdb(tcg_rd, tcg_env, tcg_rd, cpu_reg_sp(s, rn)); - } else if (!dc_isar_feature(aa64_pauth, s)) { - goto do_unallocated; - } - break; - case MAP(1, 0x01, 0x08): /* PACIZA */ - if (!dc_isar_feature(aa64_pauth, s) || rn != 31) { - goto do_unallocated; - } else if (s->pauth_active) { - tcg_rd = cpu_reg(s, rd); - gen_helper_pacia(tcg_rd, tcg_env, tcg_rd, tcg_constant_i64(0)); - } - break; - case MAP(1, 0x01, 0x09): /* PACIZB */ - if (!dc_isar_feature(aa64_pauth, s) || rn != 31) { - goto do_unallocated; - } else if (s->pauth_active) { - tcg_rd = cpu_reg(s, rd); - gen_helper_pacib(tcg_rd, tcg_env, tcg_rd, tcg_constant_i64(0)); - } - break; - case MAP(1, 0x01, 0x0a): /* PACDZA */ - if (!dc_isar_feature(aa64_pauth, s) || rn != 31) { - goto do_unallocated; - } else if (s->pauth_active) { - tcg_rd = cpu_reg(s, rd); - gen_helper_pacda(tcg_rd, tcg_env, tcg_rd, tcg_constant_i64(0)); - } - break; - case MAP(1, 0x01, 0x0b): /* PACDZB */ - if (!dc_isar_feature(aa64_pauth, s) || rn != 31) { - goto do_unallocated; - } else if (s->pauth_active) { - tcg_rd = cpu_reg(s, rd); - gen_helper_pacdb(tcg_rd, tcg_env, tcg_rd, tcg_constant_i64(0)); - } - break; - case MAP(1, 0x01, 0x0c): /* AUTIZA */ - if (!dc_isar_feature(aa64_pauth, s) || rn != 31) { - goto do_unallocated; - } else if (s->pauth_active) { - tcg_rd = cpu_reg(s, rd); - gen_helper_autia(tcg_rd, tcg_env, tcg_rd, tcg_constant_i64(0)); - } - break; - case MAP(1, 0x01, 0x0d): /* AUTIZB */ - if (!dc_isar_feature(aa64_pauth, s) || rn != 31) { - goto do_unallocated; - } else if (s->pauth_active) { - tcg_rd = cpu_reg(s, rd); - gen_helper_autib(tcg_rd, tcg_env, tcg_rd, tcg_constant_i64(0)); - } - break; - case MAP(1, 0x01, 0x0e): /* AUTDZA */ - if (!dc_isar_feature(aa64_pauth, s) || rn != 31) { - goto do_unallocated; - } else if (s->pauth_active) { - tcg_rd = cpu_reg(s, rd); - gen_helper_autda(tcg_rd, tcg_env, tcg_rd, tcg_constant_i64(0)); - } - break; - case MAP(1, 0x01, 0x0f): /* AUTDZB */ - if (!dc_isar_feature(aa64_pauth, s) || rn != 31) { - goto do_unallocated; - } else if (s->pauth_active) { - tcg_rd = cpu_reg(s, rd); - gen_helper_autdb(tcg_rd, tcg_env, tcg_rd, tcg_constant_i64(0)); - } - break; case MAP(1, 0x01, 0x10): /* XPACI */ if (!dc_isar_feature(aa64_pauth, s) || rn != 31) { goto do_unallocated; @@ -XXX,XX +XXX,XX @@ static void disas_data_proc_1src(DisasContext *s, uint32_t insn) case MAP(1, 0x00, 0x04): case MAP(0, 0x00, 0x05): /* CLS */ case MAP(1, 0x00, 0x05): + case MAP(1, 0x01, 0x00): /* PACIA */ + case MAP(1, 0x01, 0x01): /* PACIB */ + case MAP(1, 0x01, 0x02): /* PACDA */ + case MAP(1, 0x01, 0x03): /* PACDB */ + case MAP(1, 0x01, 0x04): /* AUTIA */ + case MAP(1, 0x01, 0x05): /* AUTIB */ + case MAP(1, 0x01, 0x06): /* AUTDA */ + case MAP(1, 0x01, 0x07): /* AUTDB */ + case MAP(1, 0x01, 0x08): /* PACIZA */ + case MAP(1, 0x01, 0x09): /* PACIZB */ + case MAP(1, 0x01, 0x0a): /* PACDZA */ + case MAP(1, 0x01, 0x0b): /* PACDZB */ + case MAP(1, 0x01, 0x0c): /* AUTIZA */ + case MAP(1, 0x01, 0x0d): /* AUTIZB */ + case MAP(1, 0x01, 0x0e): /* AUTDZA */ + case MAP(1, 0x01, 0x0f): /* AUTDZB */ unallocated_encoding(s); break; } -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Remove disas_data_proc_1src, as these were the last insns decoded by that function. Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-11-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 3 ++ target/arm/tcg/translate-a64.c | 99 +++++----------------------------- 2 files changed, 16 insertions(+), 86 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ AUTIB 1 10 11010110 00001 00.101 ..... ..... @pacaut AUTDA 1 10 11010110 00001 00.110 ..... ..... @pacaut AUTDB 1 10 11010110 00001 00.111 ..... ..... @pacaut +XPACI 1 10 11010110 00001 010000 11111 rd:5 +XPACD 1 10 11010110 00001 010001 11111 rd:5 + # Logical (shifted reg) # Add/subtract (shifted reg) # Add/subtract (extended reg) diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ TRANS_FEAT(AUTIB, aa64_pauth, gen_pacaut, a, gen_helper_autib) TRANS_FEAT(AUTDA, aa64_pauth, gen_pacaut, a, gen_helper_autda) TRANS_FEAT(AUTDB, aa64_pauth, gen_pacaut, a, gen_helper_autdb) +static bool do_xpac(DisasContext *s, int rd, NeonGenOne64OpEnvFn *fn) +{ + if (s->pauth_active) { + TCGv_i64 tcg_rd = cpu_reg(s, rd); + fn(tcg_rd, tcg_env, tcg_rd); + } + return true; +} + +TRANS_FEAT(XPACI, aa64_pauth, do_xpac, a->rd, gen_helper_xpaci) +TRANS_FEAT(XPACD, aa64_pauth, do_xpac, a->rd, gen_helper_xpacd) + /* Logical (shifted register) * 31 30 29 28 24 23 22 21 20 16 15 10 9 5 4 0 * +----+-----+-----------+-------+---+------+--------+------+------+ @@ -XXX,XX +XXX,XX @@ static void disas_cond_select(DisasContext *s, uint32_t insn) } } -/* Data-processing (1 source) - * 31 30 29 28 21 20 16 15 10 9 5 4 0 - * +----+---+---+-----------------+---------+--------+------+------+ - * | sf | 1 | S | 1 1 0 1 0 1 1 0 | opcode2 | opcode | Rn | Rd | - * +----+---+---+-----------------+---------+--------+------+------+ - */ -static void disas_data_proc_1src(DisasContext *s, uint32_t insn) -{ - unsigned int sf, opcode, opcode2, rn, rd; - TCGv_i64 tcg_rd; - - if (extract32(insn, 29, 1)) { - unallocated_encoding(s); - return; - } - - sf = extract32(insn, 31, 1); - opcode = extract32(insn, 10, 6); - opcode2 = extract32(insn, 16, 5); - rn = extract32(insn, 5, 5); - rd = extract32(insn, 0, 5); - -#define MAP(SF, O2, O1) ((SF) | (O1 << 1) | (O2 << 7)) - - switch (MAP(sf, opcode2, opcode)) { - case MAP(1, 0x01, 0x10): /* XPACI */ - if (!dc_isar_feature(aa64_pauth, s) || rn != 31) { - goto do_unallocated; - } else if (s->pauth_active) { - tcg_rd = cpu_reg(s, rd); - gen_helper_xpaci(tcg_rd, tcg_env, tcg_rd); - } - break; - case MAP(1, 0x01, 0x11): /* XPACD */ - if (!dc_isar_feature(aa64_pauth, s) || rn != 31) { - goto do_unallocated; - } else if (s->pauth_active) { - tcg_rd = cpu_reg(s, rd); - gen_helper_xpacd(tcg_rd, tcg_env, tcg_rd); - } - break; - default: - do_unallocated: - case MAP(0, 0x00, 0x00): /* RBIT */ - case MAP(1, 0x00, 0x00): - case MAP(0, 0x00, 0x01): /* REV16 */ - case MAP(1, 0x00, 0x01): - case MAP(0, 0x00, 0x02): /* REV/REV32 */ - case MAP(1, 0x00, 0x02): - case MAP(1, 0x00, 0x03): /* REV64 */ - case MAP(0, 0x00, 0x04): /* CLZ */ - case MAP(1, 0x00, 0x04): - case MAP(0, 0x00, 0x05): /* CLS */ - case MAP(1, 0x00, 0x05): - case MAP(1, 0x01, 0x00): /* PACIA */ - case MAP(1, 0x01, 0x01): /* PACIB */ - case MAP(1, 0x01, 0x02): /* PACDA */ - case MAP(1, 0x01, 0x03): /* PACDB */ - case MAP(1, 0x01, 0x04): /* AUTIA */ - case MAP(1, 0x01, 0x05): /* AUTIB */ - case MAP(1, 0x01, 0x06): /* AUTDA */ - case MAP(1, 0x01, 0x07): /* AUTDB */ - case MAP(1, 0x01, 0x08): /* PACIZA */ - case MAP(1, 0x01, 0x09): /* PACIZB */ - case MAP(1, 0x01, 0x0a): /* PACDZA */ - case MAP(1, 0x01, 0x0b): /* PACDZB */ - case MAP(1, 0x01, 0x0c): /* AUTIZA */ - case MAP(1, 0x01, 0x0d): /* AUTIZB */ - case MAP(1, 0x01, 0x0e): /* AUTDZA */ - case MAP(1, 0x01, 0x0f): /* AUTDZB */ - unallocated_encoding(s); - break; - } - -#undef MAP -} - - /* * Data processing - register * 31 30 29 28 25 21 20 16 10 0 @@ -XXX,XX +XXX,XX @@ static void disas_data_proc_1src(DisasContext *s, uint32_t insn) */ static void disas_data_proc_reg(DisasContext *s, uint32_t insn) { - int op0 = extract32(insn, 30, 1); int op1 = extract32(insn, 28, 1); int op2 = extract32(insn, 21, 4); int op3 = extract32(insn, 10, 6); @@ -XXX,XX +XXX,XX @@ static void disas_data_proc_reg(DisasContext *s, uint32_t insn) disas_cond_select(s, insn); break; - case 0x6: /* Data-processing */ - if (op0) { /* (1 source) */ - disas_data_proc_1src(s, insn); - } else { /* (2 source) */ - goto do_unallocated; - } - break; case 0x8 ... 0xf: /* (3 source) */ disas_data_proc_3src(s, insn); break; default: do_unallocated: + case 0x6: /* Data-processing */ unallocated_encoding(s); break; } -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> This includes AND, BIC, ORR, ORN, EOR, EON, ANDS, BICS (shifted reg). Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-12-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 9 +++ target/arm/tcg/translate-a64.c | 117 ++++++++++++--------------------- 2 files changed, 51 insertions(+), 75 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ XPACI 1 10 11010110 00001 010000 11111 rd:5 XPACD 1 10 11010110 00001 010001 11111 rd:5 # Logical (shifted reg) + +&logic_shift rd rn rm sf sa st n +@logic_shift sf:1 .. ..... st:2 n:1 rm:5 sa:6 rn:5 rd:5 &logic_shift + +AND_r . 00 01010 .. . ..... ...... ..... ..... @logic_shift +ORR_r . 01 01010 .. . ..... ...... ..... ..... @logic_shift +EOR_r . 10 01010 .. . ..... ...... ..... ..... @logic_shift +ANDS_r . 11 01010 .. . ..... ...... ..... ..... @logic_shift + # Add/subtract (shifted reg) # Add/subtract (extended reg) # Add/subtract (carry) diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static bool do_xpac(DisasContext *s, int rd, NeonGenOne64OpEnvFn *fn) TRANS_FEAT(XPACI, aa64_pauth, do_xpac, a->rd, gen_helper_xpaci) TRANS_FEAT(XPACD, aa64_pauth, do_xpac, a->rd, gen_helper_xpacd) -/* Logical (shifted register) - * 31 30 29 28 24 23 22 21 20 16 15 10 9 5 4 0 - * +----+-----+-----------+-------+---+------+--------+------+------+ - * | sf | opc | 0 1 0 1 0 | shift | N | Rm | imm6 | Rn | Rd | - * +----+-----+-----------+-------+---+------+--------+------+------+ - */ -static void disas_logic_reg(DisasContext *s, uint32_t insn) +static bool do_logic_reg(DisasContext *s, arg_logic_shift *a, + ArithTwoOp *fn, ArithTwoOp *inv_fn, bool setflags) { TCGv_i64 tcg_rd, tcg_rn, tcg_rm; - unsigned int sf, opc, shift_type, invert, rm, shift_amount, rn, rd; - sf = extract32(insn, 31, 1); - opc = extract32(insn, 29, 2); - shift_type = extract32(insn, 22, 2); - invert = extract32(insn, 21, 1); - rm = extract32(insn, 16, 5); - shift_amount = extract32(insn, 10, 6); - rn = extract32(insn, 5, 5); - rd = extract32(insn, 0, 5); - - if (!sf && (shift_amount & (1 << 5))) { - unallocated_encoding(s); - return; + if (!a->sf && (a->sa & (1 << 5))) { + return false; } - tcg_rd = cpu_reg(s, rd); + tcg_rd = cpu_reg(s, a->rd); + tcg_rn = cpu_reg(s, a->rn); - if (opc == 1 && shift_amount == 0 && shift_type == 0 && rn == 31) { - /* Unshifted ORR and ORN with WZR/XZR is the standard encoding for - * register-register MOV and MVN, so it is worth special casing. - */ - tcg_rm = cpu_reg(s, rm); - if (invert) { + tcg_rm = read_cpu_reg(s, a->rm, a->sf); + if (a->sa) { + shift_reg_imm(tcg_rm, tcg_rm, a->sf, a->st, a->sa); + } + + (a->n ? inv_fn : fn)(tcg_rd, tcg_rn, tcg_rm); + if (!a->sf) { + tcg_gen_ext32u_i64(tcg_rd, tcg_rd); + } + if (setflags) { + gen_logic_CC(a->sf, tcg_rd); + } + return true; +} + +static bool trans_ORR_r(DisasContext *s, arg_logic_shift *a) +{ + /* + * Unshifted ORR and ORN with WZR/XZR is the standard encoding for + * register-register MOV and MVN, so it is worth special casing. + */ + if (a->sa == 0 && a->st == 0 && a->rn == 31) { + TCGv_i64 tcg_rd = cpu_reg(s, a->rd); + TCGv_i64 tcg_rm = cpu_reg(s, a->rm); + + if (a->n) { tcg_gen_not_i64(tcg_rd, tcg_rm); - if (!sf) { + if (!a->sf) { tcg_gen_ext32u_i64(tcg_rd, tcg_rd); } } else { - if (sf) { + if (a->sf) { tcg_gen_mov_i64(tcg_rd, tcg_rm); } else { tcg_gen_ext32u_i64(tcg_rd, tcg_rm); } } - return; + return true; } - tcg_rm = read_cpu_reg(s, rm, sf); - - if (shift_amount) { - shift_reg_imm(tcg_rm, tcg_rm, sf, shift_type, shift_amount); - } - - tcg_rn = cpu_reg(s, rn); - - switch (opc | (invert << 2)) { - case 0: /* AND */ - case 3: /* ANDS */ - tcg_gen_and_i64(tcg_rd, tcg_rn, tcg_rm); - break; - case 1: /* ORR */ - tcg_gen_or_i64(tcg_rd, tcg_rn, tcg_rm); - break; - case 2: /* EOR */ - tcg_gen_xor_i64(tcg_rd, tcg_rn, tcg_rm); - break; - case 4: /* BIC */ - case 7: /* BICS */ - tcg_gen_andc_i64(tcg_rd, tcg_rn, tcg_rm); - break; - case 5: /* ORN */ - tcg_gen_orc_i64(tcg_rd, tcg_rn, tcg_rm); - break; - case 6: /* EON */ - tcg_gen_eqv_i64(tcg_rd, tcg_rn, tcg_rm); - break; - default: - assert(FALSE); - break; - } - - if (!sf) { - tcg_gen_ext32u_i64(tcg_rd, tcg_rd); - } - - if (opc == 3) { - gen_logic_CC(sf, tcg_rd); - } + return do_logic_reg(s, a, tcg_gen_or_i64, tcg_gen_orc_i64, false); } +TRANS(AND_r, do_logic_reg, a, tcg_gen_and_i64, tcg_gen_andc_i64, false) +TRANS(ANDS_r, do_logic_reg, a, tcg_gen_and_i64, tcg_gen_andc_i64, true) +TRANS(EOR_r, do_logic_reg, a, tcg_gen_xor_i64, tcg_gen_eqv_i64, false) + /* * Add/subtract (extended register) * @@ -XXX,XX +XXX,XX @@ static void disas_data_proc_reg(DisasContext *s, uint32_t insn) /* Add/sub (shifted register) */ disas_add_sub_reg(s, insn); } - } else { - /* Logical (shifted register) */ - disas_logic_reg(s, insn); + return; } - return; + goto do_unallocated; } switch (op2) { -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> This includes ADD, SUB, ADDS, SUBS (extended register). Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-13-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 9 +++++ target/arm/tcg/translate-a64.c | 65 +++++++++++----------------------- 2 files changed, 29 insertions(+), 45 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ ANDS_r . 11 01010 .. . ..... ...... ..... ..... @logic_shift # Add/subtract (shifted reg) # Add/subtract (extended reg) + +&addsub_ext rd rn rm sf sa st +@addsub_ext sf:1 .. ........ rm:5 st:3 sa:3 rn:5 rd:5 &addsub_ext + +ADD_ext . 00 01011001 ..... ... ... ..... ..... @addsub_ext +SUB_ext . 10 01011001 ..... ... ... ..... ..... @addsub_ext +ADDS_ext . 01 01011001 ..... ... ... ..... ..... @addsub_ext +SUBS_ext . 11 01011001 ..... ... ... ..... ..... @addsub_ext + # Add/subtract (carry) # Rotate right into flags # Evaluate into flags diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ TRANS(AND_r, do_logic_reg, a, tcg_gen_and_i64, tcg_gen_andc_i64, false) TRANS(ANDS_r, do_logic_reg, a, tcg_gen_and_i64, tcg_gen_andc_i64, true) TRANS(EOR_r, do_logic_reg, a, tcg_gen_xor_i64, tcg_gen_eqv_i64, false) -/* - * Add/subtract (extended register) - * - * 31|30|29|28 24|23 22|21|20 16|15 13|12 10|9 5|4 0| - * +--+--+--+-----------+-----+--+-------+------+------+----+----+ - * |sf|op| S| 0 1 0 1 1 | opt | 1| Rm |option| imm3 | Rn | Rd | - * +--+--+--+-----------+-----+--+-------+------+------+----+----+ - * - * sf: 0 -> 32bit, 1 -> 64bit - * op: 0 -> add , 1 -> sub - * S: 1 -> set flags - * opt: 00 - * option: extension type (see DecodeRegExtend) - * imm3: optional shift to Rm - * - * Rd = Rn + LSL(extend(Rm), amount) - */ -static void disas_add_sub_ext_reg(DisasContext *s, uint32_t insn) +static bool do_addsub_ext(DisasContext *s, arg_addsub_ext *a, + bool sub_op, bool setflags) { - int rd = extract32(insn, 0, 5); - int rn = extract32(insn, 5, 5); - int imm3 = extract32(insn, 10, 3); - int option = extract32(insn, 13, 3); - int rm = extract32(insn, 16, 5); - int opt = extract32(insn, 22, 2); - bool setflags = extract32(insn, 29, 1); - bool sub_op = extract32(insn, 30, 1); - bool sf = extract32(insn, 31, 1); + TCGv_i64 tcg_rm, tcg_rn, tcg_rd, tcg_result; - TCGv_i64 tcg_rm, tcg_rn; /* temps */ - TCGv_i64 tcg_rd; - TCGv_i64 tcg_result; - - if (imm3 > 4 || opt != 0) { - unallocated_encoding(s); - return; + if (a->sa > 4) { + return false; } /* non-flag setting ops may use SP */ if (!setflags) { - tcg_rd = cpu_reg_sp(s, rd); + tcg_rd = cpu_reg_sp(s, a->rd); } else { - tcg_rd = cpu_reg(s, rd); + tcg_rd = cpu_reg(s, a->rd); } - tcg_rn = read_cpu_reg_sp(s, rn, sf); + tcg_rn = read_cpu_reg_sp(s, a->rn, a->sf); - tcg_rm = read_cpu_reg(s, rm, sf); - ext_and_shift_reg(tcg_rm, tcg_rm, option, imm3); + tcg_rm = read_cpu_reg(s, a->rm, a->sf); + ext_and_shift_reg(tcg_rm, tcg_rm, a->st, a->sa); tcg_result = tcg_temp_new_i64(); - if (!setflags) { if (sub_op) { tcg_gen_sub_i64(tcg_result, tcg_rn, tcg_rm); @@ -XXX,XX +XXX,XX @@ static void disas_add_sub_ext_reg(DisasContext *s, uint32_t insn) } } else { if (sub_op) { - gen_sub_CC(sf, tcg_result, tcg_rn, tcg_rm); + gen_sub_CC(a->sf, tcg_result, tcg_rn, tcg_rm); } else { - gen_add_CC(sf, tcg_result, tcg_rn, tcg_rm); + gen_add_CC(a->sf, tcg_result, tcg_rn, tcg_rm); } } - if (sf) { + if (a->sf) { tcg_gen_mov_i64(tcg_rd, tcg_result); } else { tcg_gen_ext32u_i64(tcg_rd, tcg_result); } + return true; } +TRANS(ADD_ext, do_addsub_ext, a, false, false) +TRANS(SUB_ext, do_addsub_ext, a, true, false) +TRANS(ADDS_ext, do_addsub_ext, a, false, true) +TRANS(SUBS_ext, do_addsub_ext, a, true, true) + /* * Add/subtract (shifted register) * @@ -XXX,XX +XXX,XX @@ static void disas_data_proc_reg(DisasContext *s, uint32_t insn) if (!op1) { if (op2 & 8) { if (op2 & 1) { - /* Add/sub (extended register) */ - disas_add_sub_ext_reg(s, insn); + goto do_unallocated; } else { /* Add/sub (shifted register) */ disas_add_sub_reg(s, insn); -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> This includes ADD, SUB, ADDS, SUBS (shifted register). Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-14-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 9 +++++ target/arm/tcg/translate-a64.c | 64 ++++++++++------------------------ 2 files changed, 27 insertions(+), 46 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ EOR_r . 10 01010 .. . ..... ...... ..... ..... @logic_shift ANDS_r . 11 01010 .. . ..... ...... ..... ..... @logic_shift # Add/subtract (shifted reg) + +&addsub_shift rd rn rm sf sa st +@addsub_shift sf:1 .. ..... st:2 . rm:5 sa:6 rn:5 rd:5 &addsub_shift + +ADD_r . 00 01011 .. 0 ..... ...... ..... ..... @addsub_shift +SUB_r . 10 01011 .. 0 ..... ...... ..... ..... @addsub_shift +ADDS_r . 01 01011 .. 0 ..... ...... ..... ..... @addsub_shift +SUBS_r . 11 01011 .. 0 ..... ...... ..... ..... @addsub_shift + # Add/subtract (extended reg) &addsub_ext rd rn rm sf sa st diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ TRANS(SUB_ext, do_addsub_ext, a, true, false) TRANS(ADDS_ext, do_addsub_ext, a, false, true) TRANS(SUBS_ext, do_addsub_ext, a, true, true) -/* - * Add/subtract (shifted register) - * - * 31 30 29 28 24 23 22 21 20 16 15 10 9 5 4 0 - * +--+--+--+-----------+-----+--+-------+---------+------+------+ - * |sf|op| S| 0 1 0 1 1 |shift| 0| Rm | imm6 | Rn | Rd | - * +--+--+--+-----------+-----+--+-------+---------+------+------+ - * - * sf: 0 -> 32bit, 1 -> 64bit - * op: 0 -> add , 1 -> sub - * S: 1 -> set flags - * shift: 00 -> LSL, 01 -> LSR, 10 -> ASR, 11 -> RESERVED - * imm6: Shift amount to apply to Rm before the add/sub - */ -static void disas_add_sub_reg(DisasContext *s, uint32_t insn) +static bool do_addsub_reg(DisasContext *s, arg_addsub_shift *a, + bool sub_op, bool setflags) { - int rd = extract32(insn, 0, 5); - int rn = extract32(insn, 5, 5); - int imm6 = extract32(insn, 10, 6); - int rm = extract32(insn, 16, 5); - int shift_type = extract32(insn, 22, 2); - bool setflags = extract32(insn, 29, 1); - bool sub_op = extract32(insn, 30, 1); - bool sf = extract32(insn, 31, 1); + TCGv_i64 tcg_rd, tcg_rn, tcg_rm, tcg_result; - TCGv_i64 tcg_rd = cpu_reg(s, rd); - TCGv_i64 tcg_rn, tcg_rm; - TCGv_i64 tcg_result; - - if ((shift_type == 3) || (!sf && (imm6 > 31))) { - unallocated_encoding(s); - return; + if (a->st == 3 || (!a->sf && (a->sa & 32))) { + return false; } - tcg_rn = read_cpu_reg(s, rn, sf); - tcg_rm = read_cpu_reg(s, rm, sf); + tcg_rd = cpu_reg(s, a->rd); + tcg_rn = read_cpu_reg(s, a->rn, a->sf); + tcg_rm = read_cpu_reg(s, a->rm, a->sf); - shift_reg_imm(tcg_rm, tcg_rm, sf, shift_type, imm6); + shift_reg_imm(tcg_rm, tcg_rm, a->sf, a->st, a->sa); tcg_result = tcg_temp_new_i64(); - if (!setflags) { if (sub_op) { tcg_gen_sub_i64(tcg_result, tcg_rn, tcg_rm); @@ -XXX,XX +XXX,XX @@ static void disas_add_sub_reg(DisasContext *s, uint32_t insn) } } else { if (sub_op) { - gen_sub_CC(sf, tcg_result, tcg_rn, tcg_rm); + gen_sub_CC(a->sf, tcg_result, tcg_rn, tcg_rm); } else { - gen_add_CC(sf, tcg_result, tcg_rn, tcg_rm); + gen_add_CC(a->sf, tcg_result, tcg_rn, tcg_rm); } } - if (sf) { + if (a->sf) { tcg_gen_mov_i64(tcg_rd, tcg_result); } else { tcg_gen_ext32u_i64(tcg_rd, tcg_result); } + return true; } +TRANS(ADD_r, do_addsub_reg, a, false, false) +TRANS(SUB_r, do_addsub_reg, a, true, false) +TRANS(ADDS_r, do_addsub_reg, a, false, true) +TRANS(SUBS_r, do_addsub_reg, a, true, true) + /* Data-processing (3 source) * * 31 30 29 28 24 23 21 20 16 15 14 10 9 5 4 0 @@ -XXX,XX +XXX,XX @@ static void disas_data_proc_reg(DisasContext *s, uint32_t insn) int op3 = extract32(insn, 10, 6); if (!op1) { - if (op2 & 8) { - if (op2 & 1) { - goto do_unallocated; - } else { - /* Add/sub (shifted register) */ - disas_add_sub_reg(s, insn); - } - return; - } goto do_unallocated; } -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> This includes MADD, MSUB, SMADDL, SMSUBL, UMADDL, UMSUBL, SMULH, UMULH. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-15-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 16 +++++ target/arm/tcg/translate-a64.c | 119 ++++++++++++--------------------- 2 files changed, 59 insertions(+), 76 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ SUBS_ext . 11 01011001 ..... ... ... ..... ..... @addsub_ext # Conditional select # Data Processing (3-source) +&rrrr rd rn rm ra +@rrrr . .. ........ rm:5 . ra:5 rn:5 rd:5 &rrrr + +MADD_w 0 00 11011000 ..... 0 ..... ..... ..... @rrrr +MSUB_w 0 00 11011000 ..... 1 ..... ..... ..... @rrrr +MADD_x 1 00 11011000 ..... 0 ..... ..... ..... @rrrr +MSUB_x 1 00 11011000 ..... 1 ..... ..... ..... @rrrr + +SMADDL 1 00 11011001 ..... 0 ..... ..... ..... @rrrr +SMSUBL 1 00 11011001 ..... 1 ..... ..... ..... @rrrr +UMADDL 1 00 11011101 ..... 0 ..... ..... ..... @rrrr +UMSUBL 1 00 11011101 ..... 1 ..... ..... ..... @rrrr + +SMULH 1 00 11011010 ..... 0 11111 ..... ..... @rrr +UMULH 1 00 11011110 ..... 0 11111 ..... ..... @rrr + ### Cryptographic AES AESE 01001110 00 10100 00100 10 ..... ..... @r2r_q1e0 diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ TRANS(SUB_r, do_addsub_reg, a, true, false) TRANS(ADDS_r, do_addsub_reg, a, false, true) TRANS(SUBS_r, do_addsub_reg, a, true, true) -/* Data-processing (3 source) - * - * 31 30 29 28 24 23 21 20 16 15 14 10 9 5 4 0 - * +--+------+-----------+------+------+----+------+------+------+ - * |sf| op54 | 1 1 0 1 1 | op31 | Rm | o0 | Ra | Rn | Rd | - * +--+------+-----------+------+------+----+------+------+------+ - */ -static void disas_data_proc_3src(DisasContext *s, uint32_t insn) +static bool do_mulh(DisasContext *s, arg_rrr *a, + void (*fn)(TCGv_i64, TCGv_i64, TCGv_i64, TCGv_i64)) { - int rd = extract32(insn, 0, 5); - int rn = extract32(insn, 5, 5); - int ra = extract32(insn, 10, 5); - int rm = extract32(insn, 16, 5); - int op_id = (extract32(insn, 29, 3) << 4) | - (extract32(insn, 21, 3) << 1) | - extract32(insn, 15, 1); - bool sf = extract32(insn, 31, 1); - bool is_sub = extract32(op_id, 0, 1); - bool is_high = extract32(op_id, 2, 1); - bool is_signed = false; - TCGv_i64 tcg_op1; - TCGv_i64 tcg_op2; - TCGv_i64 tcg_tmp; + TCGv_i64 discard = tcg_temp_new_i64(); + TCGv_i64 tcg_rd = cpu_reg(s, a->rd); + TCGv_i64 tcg_rn = cpu_reg(s, a->rn); + TCGv_i64 tcg_rm = cpu_reg(s, a->rm); - /* Note that op_id is sf:op54:op31:o0 so it includes the 32/64 size flag */ - switch (op_id) { - case 0x42: /* SMADDL */ - case 0x43: /* SMSUBL */ - case 0x44: /* SMULH */ - is_signed = true; - break; - case 0x0: /* MADD (32bit) */ - case 0x1: /* MSUB (32bit) */ - case 0x40: /* MADD (64bit) */ - case 0x41: /* MSUB (64bit) */ - case 0x4a: /* UMADDL */ - case 0x4b: /* UMSUBL */ - case 0x4c: /* UMULH */ - break; - default: - unallocated_encoding(s); - return; - } + fn(discard, tcg_rd, tcg_rn, tcg_rm); + return true; +} - if (is_high) { - TCGv_i64 low_bits = tcg_temp_new_i64(); /* low bits discarded */ - TCGv_i64 tcg_rd = cpu_reg(s, rd); - TCGv_i64 tcg_rn = cpu_reg(s, rn); - TCGv_i64 tcg_rm = cpu_reg(s, rm); +TRANS(SMULH, do_mulh, a, tcg_gen_muls2_i64) +TRANS(UMULH, do_mulh, a, tcg_gen_mulu2_i64) - if (is_signed) { - tcg_gen_muls2_i64(low_bits, tcg_rd, tcg_rn, tcg_rm); - } else { - tcg_gen_mulu2_i64(low_bits, tcg_rd, tcg_rn, tcg_rm); - } - return; - } +static bool do_muladd(DisasContext *s, arg_rrrr *a, + bool sf, bool is_sub, MemOp mop) +{ + TCGv_i64 tcg_rd = cpu_reg(s, a->rd); + TCGv_i64 tcg_op1, tcg_op2; - tcg_op1 = tcg_temp_new_i64(); - tcg_op2 = tcg_temp_new_i64(); - tcg_tmp = tcg_temp_new_i64(); - - if (op_id < 0x42) { - tcg_gen_mov_i64(tcg_op1, cpu_reg(s, rn)); - tcg_gen_mov_i64(tcg_op2, cpu_reg(s, rm)); + if (mop == MO_64) { + tcg_op1 = cpu_reg(s, a->rn); + tcg_op2 = cpu_reg(s, a->rm); } else { - if (is_signed) { - tcg_gen_ext32s_i64(tcg_op1, cpu_reg(s, rn)); - tcg_gen_ext32s_i64(tcg_op2, cpu_reg(s, rm)); - } else { - tcg_gen_ext32u_i64(tcg_op1, cpu_reg(s, rn)); - tcg_gen_ext32u_i64(tcg_op2, cpu_reg(s, rm)); - } + tcg_op1 = tcg_temp_new_i64(); + tcg_op2 = tcg_temp_new_i64(); + tcg_gen_ext_i64(tcg_op1, cpu_reg(s, a->rn), mop); + tcg_gen_ext_i64(tcg_op2, cpu_reg(s, a->rm), mop); } - if (ra == 31 && !is_sub) { + if (a->ra == 31 && !is_sub) { /* Special-case MADD with rA == XZR; it is the standard MUL alias */ - tcg_gen_mul_i64(cpu_reg(s, rd), tcg_op1, tcg_op2); + tcg_gen_mul_i64(tcg_rd, tcg_op1, tcg_op2); } else { + TCGv_i64 tcg_tmp = tcg_temp_new_i64(); + TCGv_i64 tcg_ra = cpu_reg(s, a->ra); + tcg_gen_mul_i64(tcg_tmp, tcg_op1, tcg_op2); if (is_sub) { - tcg_gen_sub_i64(cpu_reg(s, rd), cpu_reg(s, ra), tcg_tmp); + tcg_gen_sub_i64(tcg_rd, tcg_ra, tcg_tmp); } else { - tcg_gen_add_i64(cpu_reg(s, rd), cpu_reg(s, ra), tcg_tmp); + tcg_gen_add_i64(tcg_rd, tcg_ra, tcg_tmp); } } if (!sf) { - tcg_gen_ext32u_i64(cpu_reg(s, rd), cpu_reg(s, rd)); + tcg_gen_ext32u_i64(tcg_rd, tcg_rd); } + return true; } +TRANS(MADD_w, do_muladd, a, false, false, MO_64) +TRANS(MSUB_w, do_muladd, a, false, true, MO_64) +TRANS(MADD_x, do_muladd, a, true, false, MO_64) +TRANS(MSUB_x, do_muladd, a, true, true, MO_64) + +TRANS(SMADDL, do_muladd, a, true, false, MO_SL) +TRANS(SMSUBL, do_muladd, a, true, true, MO_SL) +TRANS(UMADDL, do_muladd, a, true, false, MO_UL) +TRANS(UMSUBL, do_muladd, a, true, true, MO_UL) + /* Add/subtract (with carry) * 31 30 29 28 27 26 25 24 23 22 21 20 16 15 10 9 5 4 0 * +--+--+--+------------------------+------+-------------+------+-----+ @@ -XXX,XX +XXX,XX @@ static void disas_data_proc_reg(DisasContext *s, uint32_t insn) disas_cond_select(s, insn); break; - case 0x8 ... 0xf: /* (3 source) */ - disas_data_proc_3src(s, insn); - break; - default: do_unallocated: case 0x6: /* Data-processing */ + case 0x8 ... 0xf: /* (3 source) */ unallocated_encoding(s); break; } -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> This includes ADC, SBC, ADCS, SBCS. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-16-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 6 +++++ target/arm/tcg/translate-a64.c | 43 +++++++++++++--------------------- 2 files changed, 22 insertions(+), 27 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ ADDS_ext . 01 01011001 ..... ... ... ..... ..... @addsub_ext SUBS_ext . 11 01011001 ..... ... ... ..... ..... @addsub_ext # Add/subtract (carry) + +ADC . 00 11010000 ..... 000000 ..... ..... @rrr_sf +ADCS . 01 11010000 ..... 000000 ..... ..... @rrr_sf +SBC . 10 11010000 ..... 000000 ..... ..... @rrr_sf +SBCS . 11 11010000 ..... 000000 ..... ..... @rrr_sf + # Rotate right into flags # Evaluate into flags # Conditional compare (regster) diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ TRANS(SMSUBL, do_muladd, a, true, true, MO_SL) TRANS(UMADDL, do_muladd, a, true, false, MO_UL) TRANS(UMSUBL, do_muladd, a, true, true, MO_UL) -/* Add/subtract (with carry) - * 31 30 29 28 27 26 25 24 23 22 21 20 16 15 10 9 5 4 0 - * +--+--+--+------------------------+------+-------------+------+-----+ - * |sf|op| S| 1 1 0 1 0 0 0 0 | rm | 0 0 0 0 0 0 | Rn | Rd | - * +--+--+--+------------------------+------+-------------+------+-----+ - */ - -static void disas_adc_sbc(DisasContext *s, uint32_t insn) +static bool do_adc_sbc(DisasContext *s, arg_rrr_sf *a, + bool is_sub, bool setflags) { - unsigned int sf, op, setflags, rm, rn, rd; TCGv_i64 tcg_y, tcg_rn, tcg_rd; - sf = extract32(insn, 31, 1); - op = extract32(insn, 30, 1); - setflags = extract32(insn, 29, 1); - rm = extract32(insn, 16, 5); - rn = extract32(insn, 5, 5); - rd = extract32(insn, 0, 5); + tcg_rd = cpu_reg(s, a->rd); + tcg_rn = cpu_reg(s, a->rn); - tcg_rd = cpu_reg(s, rd); - tcg_rn = cpu_reg(s, rn); - - if (op) { + if (is_sub) { tcg_y = tcg_temp_new_i64(); - tcg_gen_not_i64(tcg_y, cpu_reg(s, rm)); + tcg_gen_not_i64(tcg_y, cpu_reg(s, a->rm)); } else { - tcg_y = cpu_reg(s, rm); + tcg_y = cpu_reg(s, a->rm); } if (setflags) { - gen_adc_CC(sf, tcg_rd, tcg_rn, tcg_y); + gen_adc_CC(a->sf, tcg_rd, tcg_rn, tcg_y); } else { - gen_adc(sf, tcg_rd, tcg_rn, tcg_y); + gen_adc(a->sf, tcg_rd, tcg_rn, tcg_y); } + return true; } +TRANS(ADC, do_adc_sbc, a, false, false) +TRANS(SBC, do_adc_sbc, a, true, false) +TRANS(ADCS, do_adc_sbc, a, false, true) +TRANS(SBCS, do_adc_sbc, a, true, true) + /* * Rotate right into flags * 31 30 29 21 15 10 5 4 0 @@ -XXX,XX +XXX,XX @@ static void disas_data_proc_reg(DisasContext *s, uint32_t insn) switch (op2) { case 0x0: switch (op3) { - case 0x00: /* Add/subtract (with carry) */ - disas_adc_sbc(s, insn); - break; - case 0x01: /* Rotate right into flags */ case 0x21: disas_rotate_right_into_flags(s, insn); @@ -XXX,XX +XXX,XX @@ static void disas_data_proc_reg(DisasContext *s, uint32_t insn) break; default: + case 0x00: /* Add/subtract (with carry) */ goto do_unallocated; } break; -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-17-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 3 +++ target/arm/tcg/translate-a64.c | 32 +++++++++----------------------- 2 files changed, 12 insertions(+), 23 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ SBC . 10 11010000 ..... 000000 ..... ..... @rrr_sf SBCS . 11 11010000 ..... 000000 ..... ..... @rrr_sf # Rotate right into flags + +RMIF 1 01 11010000 imm:6 00001 rn:5 0 mask:4 + # Evaluate into flags # Conditional compare (regster) # Conditional compare (immediate) diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ TRANS(SBC, do_adc_sbc, a, true, false) TRANS(ADCS, do_adc_sbc, a, false, true) TRANS(SBCS, do_adc_sbc, a, true, true) -/* - * Rotate right into flags - * 31 30 29 21 15 10 5 4 0 - * +--+--+--+-----------------+--------+-----------+------+--+------+ - * |sf|op| S| 1 1 0 1 0 0 0 0 | imm6 | 0 0 0 0 1 | Rn |o2| mask | - * +--+--+--+-----------------+--------+-----------+------+--+------+ - */ -static void disas_rotate_right_into_flags(DisasContext *s, uint32_t insn) +static bool trans_RMIF(DisasContext *s, arg_RMIF *a) { - int mask = extract32(insn, 0, 4); - int o2 = extract32(insn, 4, 1); - int rn = extract32(insn, 5, 5); - int imm6 = extract32(insn, 15, 6); - int sf_op_s = extract32(insn, 29, 3); + int mask = a->mask; TCGv_i64 tcg_rn; TCGv_i32 nzcv; - if (sf_op_s != 5 || o2 != 0 || !dc_isar_feature(aa64_condm_4, s)) { - unallocated_encoding(s); - return; + if (!dc_isar_feature(aa64_condm_4, s)) { + return false; } - tcg_rn = read_cpu_reg(s, rn, 1); - tcg_gen_rotri_i64(tcg_rn, tcg_rn, imm6); + tcg_rn = read_cpu_reg(s, a->rn, 1); + tcg_gen_rotri_i64(tcg_rn, tcg_rn, a->imm); nzcv = tcg_temp_new_i32(); tcg_gen_extrl_i64_i32(nzcv, tcg_rn); @@ -XXX,XX +XXX,XX @@ static void disas_rotate_right_into_flags(DisasContext *s, uint32_t insn) if (mask & 1) { /* V */ tcg_gen_shli_i32(cpu_VF, nzcv, 31 - 0); } + return true; } /* @@ -XXX,XX +XXX,XX @@ static void disas_data_proc_reg(DisasContext *s, uint32_t insn) switch (op2) { case 0x0: switch (op3) { - case 0x01: /* Rotate right into flags */ - case 0x21: - disas_rotate_right_into_flags(s, insn); - break; - case 0x02: /* Evaluate into flags */ case 0x12: case 0x22: @@ -XXX,XX +XXX,XX @@ static void disas_data_proc_reg(DisasContext *s, uint32_t insn) default: case 0x00: /* Add/subtract (with carry) */ + case 0x01: /* Rotate right into flags */ + case 0x21: goto do_unallocated; } break; -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-18-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 4 +++ target/arm/tcg/translate-a64.c | 48 +++++----------------------------- 2 files changed, 11 insertions(+), 41 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ SBCS . 11 11010000 ..... 000000 ..... ..... @rrr_sf RMIF 1 01 11010000 imm:6 00001 rn:5 0 mask:4 # Evaluate into flags + +SETF8 0 01 11010000 00000 000010 rn:5 01101 +SETF16 0 01 11010000 00000 010010 rn:5 01101 + # Conditional compare (regster) # Conditional compare (immediate) # Conditional select diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static bool trans_RMIF(DisasContext *s, arg_RMIF *a) return true; } -/* - * Evaluate into flags - * 31 30 29 21 15 14 10 5 4 0 - * +--+--+--+-----------------+---------+----+---------+------+--+------+ - * |sf|op| S| 1 1 0 1 0 0 0 0 | opcode2 | sz | 0 0 1 0 | Rn |o3| mask | - * +--+--+--+-----------------+---------+----+---------+------+--+------+ - */ -static void disas_evaluate_into_flags(DisasContext *s, uint32_t insn) +static bool do_setf(DisasContext *s, int rn, int shift) { - int o3_mask = extract32(insn, 0, 5); - int rn = extract32(insn, 5, 5); - int o2 = extract32(insn, 15, 6); - int sz = extract32(insn, 14, 1); - int sf_op_s = extract32(insn, 29, 3); - TCGv_i32 tmp; - int shift; + TCGv_i32 tmp = tcg_temp_new_i32(); - if (sf_op_s != 1 || o2 != 0 || o3_mask != 0xd || - !dc_isar_feature(aa64_condm_4, s)) { - unallocated_encoding(s); - return; - } - shift = sz ? 16 : 24; /* SETF16 or SETF8 */ - - tmp = tcg_temp_new_i32(); tcg_gen_extrl_i64_i32(tmp, cpu_reg(s, rn)); tcg_gen_shli_i32(cpu_NF, tmp, shift); tcg_gen_shli_i32(cpu_VF, tmp, shift - 1); tcg_gen_mov_i32(cpu_ZF, cpu_NF); tcg_gen_xor_i32(cpu_VF, cpu_VF, cpu_NF); + return true; } +TRANS_FEAT(SETF8, aa64_condm_4, do_setf, a->rn, 24) +TRANS_FEAT(SETF16, aa64_condm_4, do_setf, a->rn, 16) + /* Conditional compare (immediate / register) * 31 30 29 28 27 26 25 24 23 22 21 20 16 15 12 11 10 9 5 4 3 0 * +--+--+--+------------------------+--------+------+----+--+------+--+-----+ @@ -XXX,XX +XXX,XX @@ static void disas_data_proc_reg(DisasContext *s, uint32_t insn) { int op1 = extract32(insn, 28, 1); int op2 = extract32(insn, 21, 4); - int op3 = extract32(insn, 10, 6); if (!op1) { goto do_unallocated; } switch (op2) { - case 0x0: - switch (op3) { - case 0x02: /* Evaluate into flags */ - case 0x12: - case 0x22: - case 0x32: - disas_evaluate_into_flags(s, insn); - break; - - default: - case 0x00: /* Add/subtract (with carry) */ - case 0x01: /* Rotate right into flags */ - case 0x21: - goto do_unallocated; - } - break; - case 0x2: /* Conditional compare */ disas_cc(s, insn); /* both imm and reg forms */ break; @@ -XXX,XX +XXX,XX @@ static void disas_data_proc_reg(DisasContext *s, uint32_t insn) default: do_unallocated: + case 0x0: case 0x6: /* Data-processing */ case 0x8 ... 0xf: /* (3 source) */ unallocated_encoding(s); -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-19-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 6 ++-- target/arm/tcg/translate-a64.c | 66 +++++++++++----------------------- 2 files changed, 25 insertions(+), 47 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ RMIF 1 01 11010000 imm:6 00001 rn:5 0 mask:4 SETF8 0 01 11010000 00000 000010 rn:5 01101 SETF16 0 01 11010000 00000 010010 rn:5 01101 -# Conditional compare (regster) -# Conditional compare (immediate) +# Conditional compare + +CCMP sf:1 op:1 1 11010010 y:5 cond:4 imm:1 0 rn:5 0 nzcv:4 + # Conditional select # Data Processing (3-source) diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static bool do_setf(DisasContext *s, int rn, int shift) TRANS_FEAT(SETF8, aa64_condm_4, do_setf, a->rn, 24) TRANS_FEAT(SETF16, aa64_condm_4, do_setf, a->rn, 16) -/* Conditional compare (immediate / register) - * 31 30 29 28 27 26 25 24 23 22 21 20 16 15 12 11 10 9 5 4 3 0 - * +--+--+--+------------------------+--------+------+----+--+------+--+-----+ - * |sf|op| S| 1 1 0 1 0 0 1 0 |imm5/rm | cond |i/r |o2| Rn |o3|nzcv | - * +--+--+--+------------------------+--------+------+----+--+------+--+-----+ - * [1] y [0] [0] - */ -static void disas_cc(DisasContext *s, uint32_t insn) +/* CCMP, CCMN */ +static bool trans_CCMP(DisasContext *s, arg_CCMP *a) { - unsigned int sf, op, y, cond, rn, nzcv, is_imm; - TCGv_i32 tcg_t0, tcg_t1, tcg_t2; - TCGv_i64 tcg_tmp, tcg_y, tcg_rn; + TCGv_i32 tcg_t0 = tcg_temp_new_i32(); + TCGv_i32 tcg_t1 = tcg_temp_new_i32(); + TCGv_i32 tcg_t2 = tcg_temp_new_i32(); + TCGv_i64 tcg_tmp = tcg_temp_new_i64(); + TCGv_i64 tcg_rn, tcg_y; DisasCompare c; - - if (!extract32(insn, 29, 1)) { - unallocated_encoding(s); - return; - } - if (insn & (1 << 10 | 1 << 4)) { - unallocated_encoding(s); - return; - } - sf = extract32(insn, 31, 1); - op = extract32(insn, 30, 1); - is_imm = extract32(insn, 11, 1); - y = extract32(insn, 16, 5); /* y = rm (reg) or imm5 (imm) */ - cond = extract32(insn, 12, 4); - rn = extract32(insn, 5, 5); - nzcv = extract32(insn, 0, 4); + unsigned nzcv; /* Set T0 = !COND. */ - tcg_t0 = tcg_temp_new_i32(); - arm_test_cc(&c, cond); + arm_test_cc(&c, a->cond); tcg_gen_setcondi_i32(tcg_invert_cond(c.cond), tcg_t0, c.value, 0); /* Load the arguments for the new comparison. */ - if (is_imm) { - tcg_y = tcg_temp_new_i64(); - tcg_gen_movi_i64(tcg_y, y); + if (a->imm) { + tcg_y = tcg_constant_i64(a->y); } else { - tcg_y = cpu_reg(s, y); + tcg_y = cpu_reg(s, a->y); } - tcg_rn = cpu_reg(s, rn); + tcg_rn = cpu_reg(s, a->rn); /* Set the flags for the new comparison. */ - tcg_tmp = tcg_temp_new_i64(); - if (op) { - gen_sub_CC(sf, tcg_tmp, tcg_rn, tcg_y); + if (a->op) { + gen_sub_CC(a->sf, tcg_tmp, tcg_rn, tcg_y); } else { - gen_add_CC(sf, tcg_tmp, tcg_rn, tcg_y); + gen_add_CC(a->sf, tcg_tmp, tcg_rn, tcg_y); } - /* If COND was false, force the flags to #nzcv. Compute two masks + /* + * If COND was false, force the flags to #nzcv. Compute two masks * to help with this: T1 = (COND ? 0 : -1), T2 = (COND ? -1 : 0). * For tcg hosts that support ANDC, we can make do with just T1. * In either case, allow the tcg optimizer to delete any unused mask. */ - tcg_t1 = tcg_temp_new_i32(); - tcg_t2 = tcg_temp_new_i32(); tcg_gen_neg_i32(tcg_t1, tcg_t0); tcg_gen_subi_i32(tcg_t2, tcg_t0, 1); + nzcv = a->nzcv; if (nzcv & 8) { /* N */ tcg_gen_or_i32(cpu_NF, cpu_NF, tcg_t1); } else { @@ -XXX,XX +XXX,XX @@ static void disas_cc(DisasContext *s, uint32_t insn) tcg_gen_and_i32(cpu_VF, cpu_VF, tcg_t2); } } + return true; } /* Conditional select @@ -XXX,XX +XXX,XX @@ static void disas_data_proc_reg(DisasContext *s, uint32_t insn) } switch (op2) { - case 0x2: /* Conditional compare */ - disas_cc(s, insn); /* both imm and reg forms */ - break; - case 0x4: /* Conditional select */ disas_cond_select(s, insn); break; @@ -XXX,XX +XXX,XX @@ static void disas_data_proc_reg(DisasContext *s, uint32_t insn) default: do_unallocated: case 0x0: + case 0x2: /* Conditional compare */ case 0x6: /* Data-processing */ case 0x8 ... 0xf: /* (3 source) */ unallocated_encoding(s); -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> This includes CSEL, CSINC, CSINV, CSNEG. Remove disas_data_proc_reg, as these were the last insns decoded by that function. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-20-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 3 ++ target/arm/tcg/translate-a64.c | 84 ++++++---------------------------- 2 files changed, 17 insertions(+), 70 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ SETF16 0 01 11010000 00000 010010 rn:5 01101 CCMP sf:1 op:1 1 11010010 y:5 cond:4 imm:1 0 rn:5 0 nzcv:4 # Conditional select + +CSEL sf:1 else_inv:1 011010100 rm:5 cond:4 0 else_inc:1 rn:5 rd:5 + # Data Processing (3-source) &rrrr rd rn rm ra diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static bool trans_CCMP(DisasContext *s, arg_CCMP *a) return true; } -/* Conditional select - * 31 30 29 28 21 20 16 15 12 11 10 9 5 4 0 - * +----+----+---+-----------------+------+------+-----+------+------+ - * | sf | op | S | 1 1 0 1 0 1 0 0 | Rm | cond | op2 | Rn | Rd | - * +----+----+---+-----------------+------+------+-----+------+------+ - */ -static void disas_cond_select(DisasContext *s, uint32_t insn) +static bool trans_CSEL(DisasContext *s, arg_CSEL *a) { - unsigned int sf, else_inv, rm, cond, else_inc, rn, rd; - TCGv_i64 tcg_rd, zero; + TCGv_i64 tcg_rd = cpu_reg(s, a->rd); + TCGv_i64 zero = tcg_constant_i64(0); DisasCompare64 c; - if (extract32(insn, 29, 1) || extract32(insn, 11, 1)) { - /* S == 1 or op2<1> == 1 */ - unallocated_encoding(s); - return; - } - sf = extract32(insn, 31, 1); - else_inv = extract32(insn, 30, 1); - rm = extract32(insn, 16, 5); - cond = extract32(insn, 12, 4); - else_inc = extract32(insn, 10, 1); - rn = extract32(insn, 5, 5); - rd = extract32(insn, 0, 5); + a64_test_cc(&c, a->cond); - tcg_rd = cpu_reg(s, rd); - - a64_test_cc(&c, cond); - zero = tcg_constant_i64(0); - - if (rn == 31 && rm == 31 && (else_inc ^ else_inv)) { + if (a->rn == 31 && a->rm == 31 && (a->else_inc ^ a->else_inv)) { /* CSET & CSETM. */ - if (else_inv) { + if (a->else_inv) { tcg_gen_negsetcond_i64(tcg_invert_cond(c.cond), tcg_rd, c.value, zero); } else { @@ -XXX,XX +XXX,XX @@ static void disas_cond_select(DisasContext *s, uint32_t insn) tcg_rd, c.value, zero); } } else { - TCGv_i64 t_true = cpu_reg(s, rn); - TCGv_i64 t_false = read_cpu_reg(s, rm, 1); - if (else_inv && else_inc) { + TCGv_i64 t_true = cpu_reg(s, a->rn); + TCGv_i64 t_false = read_cpu_reg(s, a->rm, 1); + + if (a->else_inv && a->else_inc) { tcg_gen_neg_i64(t_false, t_false); - } else if (else_inv) { + } else if (a->else_inv) { tcg_gen_not_i64(t_false, t_false); - } else if (else_inc) { + } else if (a->else_inc) { tcg_gen_addi_i64(t_false, t_false, 1); } tcg_gen_movcond_i64(c.cond, tcg_rd, c.value, zero, t_true, t_false); } - if (!sf) { + if (!a->sf) { tcg_gen_ext32u_i64(tcg_rd, tcg_rd); } -} - -/* - * Data processing - register - * 31 30 29 28 25 21 20 16 10 0 - * +--+---+--+---+-------+-----+-------+-------+---------+ - * | |op0| |op1| 1 0 1 | op2 | | op3 | | - * +--+---+--+---+-------+-----+-------+-------+---------+ - */ -static void disas_data_proc_reg(DisasContext *s, uint32_t insn) -{ - int op1 = extract32(insn, 28, 1); - int op2 = extract32(insn, 21, 4); - - if (!op1) { - goto do_unallocated; - } - - switch (op2) { - case 0x4: /* Conditional select */ - disas_cond_select(s, insn); - break; - - default: - do_unallocated: - case 0x0: - case 0x2: /* Conditional compare */ - case 0x6: /* Data-processing */ - case 0x8 ... 0xf: /* (3 source) */ - unallocated_encoding(s); - break; - } + return true; } static void handle_fp_compare(DisasContext *s, int size, @@ -XXX,XX +XXX,XX @@ static bool btype_destination_ok(uint32_t insn, bool bt, int btype) static void disas_a64_legacy(DisasContext *s, uint32_t insn) { switch (extract32(insn, 25, 4)) { - case 0x5: - case 0xd: /* Data processing - register */ - disas_data_proc_reg(s, insn); - break; case 0x7: case 0xf: /* Data processing - SIMD and floating point */ disas_data_proc_simd_fp(s, insn); -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Provide a simple way to check for float64, float32, and float16 support, as well as the fpu enabled. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-21-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/translate-a64.c | 62 ++++++++++++++++++---------------- 1 file changed, 32 insertions(+), 30 deletions(-) diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static bool fp_access_check(DisasContext *s) return true; } +/* + * Return <0 for non-supported element sizes, with MO_16 controlled by + * FEAT_FP16; return 0 for fp disabled; otherwise return >0 for success. + */ +static int fp_access_check_scalar_hsd(DisasContext *s, MemOp esz) +{ + switch (esz) { + case MO_64: + case MO_32: + break; + case MO_16: + if (!dc_isar_feature(aa64_fp16, s)) { + return -1; + } + break; + default: + return -1; + } + return fp_access_check(s); +} + /* * Check that SVE access is enabled. If it is, return true. * If not, emit code to generate an appropriate exception and return false. @@ -XXX,XX +XXX,XX @@ static bool trans_FCSEL(DisasContext *s, arg_FCSEL *a) { TCGv_i64 t_true, t_false; DisasCompare64 c; + int check = fp_access_check_scalar_hsd(s, a->esz); - switch (a->esz) { - case MO_32: - case MO_64: - break; - case MO_16: - if (!dc_isar_feature(aa64_fp16, s)) { - return false; - } - break; - default: - return false; - } - - if (!fp_access_check(s)) { - return true; + if (check <= 0) { + return check == 0; } /* Zero extend sreg & hreg inputs to 64 bits now. */ @@ -XXX,XX +XXX,XX @@ TRANS(FMINV_s, do_fp_reduction, a, gen_helper_vfp_mins) static bool trans_FMOVI_s(DisasContext *s, arg_FMOVI_s *a) { - switch (a->esz) { - case MO_32: - case MO_64: - break; - case MO_16: - if (!dc_isar_feature(aa64_fp16, s)) { - return false; - } - break; - default: - return false; - } - if (fp_access_check(s)) { - uint64_t imm = vfp_expand_imm(a->esz, a->imm); - write_fp_dreg(s, a->rd, tcg_constant_i64(imm)); + int check = fp_access_check_scalar_hsd(s, a->esz); + uint64_t imm; + + if (check <= 0) { + return check == 0; } + + imm = vfp_expand_imm(a->esz, a->imm); + write_fp_dreg(s, a->rd, tcg_constant_i64(imm)); return true; } -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Provide a simple way to check for float64, float32, and float16 support vs vector width, as well as the fpu enabled. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-22-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/translate-a64.c | 135 +++++++++++++-------------------- 1 file changed, 54 insertions(+), 81 deletions(-) diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static int fp_access_check_scalar_hsd(DisasContext *s, MemOp esz) return fp_access_check(s); } +/* Likewise, but vector MO_64 must have two elements. */ +static int fp_access_check_vector_hsd(DisasContext *s, bool is_q, MemOp esz) +{ + switch (esz) { + case MO_64: + if (!is_q) { + return -1; + } + break; + case MO_32: + break; + case MO_16: + if (!dc_isar_feature(aa64_fp16, s)) { + return -1; + } + break; + default: + return -1; + } + return fp_access_check(s); +} + /* * Check that SVE access is enabled. If it is, return true. * If not, emit code to generate an appropriate exception and return false. @@ -XXX,XX +XXX,XX @@ static bool do_fp3_vector(DisasContext *s, arg_qrrr_e *a, int data, gen_helper_gvec_3_ptr * const fns[3]) { MemOp esz = a->esz; + int check = fp_access_check_vector_hsd(s, a->q, esz); - switch (esz) { - case MO_64: - if (!a->q) { - return false; - } - break; - case MO_32: - break; - case MO_16: - if (!dc_isar_feature(aa64_fp16, s)) { - return false; - } - break; - default: - return false; - } - if (fp_access_check(s)) { - gen_gvec_op3_fpst(s, a->q, a->rd, a->rn, a->rm, - esz == MO_16, data, fns[esz - 1]); + if (check <= 0) { + return check == 0; } + + gen_gvec_op3_fpst(s, a->q, a->rd, a->rn, a->rm, + esz == MO_16, data, fns[esz - 1]); return true; } @@ -XXX,XX +XXX,XX @@ TRANS_FEAT(FCADD_270, aa64_fcma, do_fp3_vector, a, 1, f_vector_fcadd) static bool trans_FCMLA_v(DisasContext *s, arg_FCMLA_v *a) { - gen_helper_gvec_4_ptr *fn; + static gen_helper_gvec_4_ptr * const fn[] = { + [MO_16] = gen_helper_gvec_fcmlah, + [MO_32] = gen_helper_gvec_fcmlas, + [MO_64] = gen_helper_gvec_fcmlad, + }; + int check; if (!dc_isar_feature(aa64_fcma, s)) { return false; } - switch (a->esz) { - case MO_64: - if (!a->q) { - return false; - } - fn = gen_helper_gvec_fcmlad; - break; - case MO_32: - fn = gen_helper_gvec_fcmlas; - break; - case MO_16: - if (!dc_isar_feature(aa64_fp16, s)) { - return false; - } - fn = gen_helper_gvec_fcmlah; - break; - default: - return false; - } - if (fp_access_check(s)) { - gen_gvec_op4_fpst(s, a->q, a->rd, a->rn, a->rm, a->rd, - a->esz == MO_16, a->rot, fn); + + check = fp_access_check_vector_hsd(s, a->q, a->esz); + if (check <= 0) { + return check == 0; } + + gen_gvec_op4_fpst(s, a->q, a->rd, a->rn, a->rm, a->rd, + a->esz == MO_16, a->rot, fn[a->esz]); return true; } @@ -XXX,XX +XXX,XX @@ static bool do_fp3_vector_idx(DisasContext *s, arg_qrrx_e *a, gen_helper_gvec_3_ptr * const fns[3]) { MemOp esz = a->esz; + int check = fp_access_check_vector_hsd(s, a->q, esz); - switch (esz) { - case MO_64: - if (!a->q) { - return false; - } - break; - case MO_32: - break; - case MO_16: - if (!dc_isar_feature(aa64_fp16, s)) { - return false; - } - break; - default: - g_assert_not_reached(); - } - if (fp_access_check(s)) { - gen_gvec_op3_fpst(s, a->q, a->rd, a->rn, a->rm, - esz == MO_16, a->idx, fns[esz - 1]); + if (check <= 0) { + return check == 0; } + + gen_gvec_op3_fpst(s, a->q, a->rd, a->rn, a->rm, + esz == MO_16, a->idx, fns[esz - 1]); return true; } @@ -XXX,XX +XXX,XX @@ static bool do_fmla_vector_idx(DisasContext *s, arg_qrrx_e *a, bool neg) gen_helper_gvec_fmla_idx_d, }; MemOp esz = a->esz; + int check = fp_access_check_vector_hsd(s, a->q, esz); - switch (esz) { - case MO_64: - if (!a->q) { - return false; - } - break; - case MO_32: - break; - case MO_16: - if (!dc_isar_feature(aa64_fp16, s)) { - return false; - } - break; - default: - g_assert_not_reached(); - } - if (fp_access_check(s)) { - gen_gvec_op4_fpst(s, a->q, a->rd, a->rn, a->rm, a->rd, - esz == MO_16, (a->idx << 1) | neg, - fns[esz - 1]); + if (check <= 0) { + return check == 0; } + + gen_gvec_op4_fpst(s, a->q, a->rd, a->rn, a->rm, a->rd, + esz == MO_16, (a->idx << 1) | neg, + fns[esz - 1]); return true; } -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-23-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 8 + target/arm/tcg/translate-a64.c | 283 ++++++++++++--------------------- 2 files changed, 112 insertions(+), 179 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ FMINV_s 0110 1110 10 11000 01111 10 ..... ..... @rr_q1e2 FMOVI_s 0001 1110 .. 1 imm:8 100 00000 rd:5 esz=%esz_hsd +# Floating-point Compare + +FCMP 00011110 .. 1 rm:5 001000 rn:5 e:1 z:1 000 esz=%esz_hsd + +# Floating-point Conditional Compare + +FCCMP 00011110 .. 1 rm:5 cond:4 01 rn:5 e:1 nzcv:4 esz=%esz_hsd + # Advanced SIMD Modified Immediate / Shift by Immediate %abcdefgh 16:3 5:5 diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static bool trans_FMOVI_s(DisasContext *s, arg_FMOVI_s *a) return true; } +/* + * Floating point compare, conditional compare + */ + +static void handle_fp_compare(DisasContext *s, int size, + unsigned int rn, unsigned int rm, + bool cmp_with_zero, bool signal_all_nans) +{ + TCGv_i64 tcg_flags = tcg_temp_new_i64(); + TCGv_ptr fpst = fpstatus_ptr(size == MO_16 ? FPST_FPCR_F16 : FPST_FPCR); + + if (size == MO_64) { + TCGv_i64 tcg_vn, tcg_vm; + + tcg_vn = read_fp_dreg(s, rn); + if (cmp_with_zero) { + tcg_vm = tcg_constant_i64(0); + } else { + tcg_vm = read_fp_dreg(s, rm); + } + if (signal_all_nans) { + gen_helper_vfp_cmped_a64(tcg_flags, tcg_vn, tcg_vm, fpst); + } else { + gen_helper_vfp_cmpd_a64(tcg_flags, tcg_vn, tcg_vm, fpst); + } + } else { + TCGv_i32 tcg_vn = tcg_temp_new_i32(); + TCGv_i32 tcg_vm = tcg_temp_new_i32(); + + read_vec_element_i32(s, tcg_vn, rn, 0, size); + if (cmp_with_zero) { + tcg_gen_movi_i32(tcg_vm, 0); + } else { + read_vec_element_i32(s, tcg_vm, rm, 0, size); + } + + switch (size) { + case MO_32: + if (signal_all_nans) { + gen_helper_vfp_cmpes_a64(tcg_flags, tcg_vn, tcg_vm, fpst); + } else { + gen_helper_vfp_cmps_a64(tcg_flags, tcg_vn, tcg_vm, fpst); + } + break; + case MO_16: + if (signal_all_nans) { + gen_helper_vfp_cmpeh_a64(tcg_flags, tcg_vn, tcg_vm, fpst); + } else { + gen_helper_vfp_cmph_a64(tcg_flags, tcg_vn, tcg_vm, fpst); + } + break; + default: + g_assert_not_reached(); + } + } + + gen_set_nzcv(tcg_flags); +} + +/* FCMP, FCMPE */ +static bool trans_FCMP(DisasContext *s, arg_FCMP *a) +{ + int check = fp_access_check_scalar_hsd(s, a->esz); + + if (check <= 0) { + return check == 0; + } + + handle_fp_compare(s, a->esz, a->rn, a->rm, a->z, a->e); + return true; +} + +/* FCCMP, FCCMPE */ +static bool trans_FCCMP(DisasContext *s, arg_FCCMP *a) +{ + TCGLabel *label_continue = NULL; + int check = fp_access_check_scalar_hsd(s, a->esz); + + if (check <= 0) { + return check == 0; + } + + if (a->cond < 0x0e) { /* not always */ + TCGLabel *label_match = gen_new_label(); + label_continue = gen_new_label(); + arm_gen_test_cc(a->cond, label_match); + /* nomatch: */ + gen_set_nzcv(tcg_constant_i64(a->nzcv << 28)); + tcg_gen_br(label_continue); + gen_set_label(label_match); + } + + handle_fp_compare(s, a->esz, a->rn, a->rm, false, a->e); + + if (label_continue) { + gen_set_label(label_continue); + } + return true; +} + /* * Advanced SIMD Modified Immediate */ @@ -XXX,XX +XXX,XX @@ static bool trans_CSEL(DisasContext *s, arg_CSEL *a) return true; } -static void handle_fp_compare(DisasContext *s, int size, - unsigned int rn, unsigned int rm, - bool cmp_with_zero, bool signal_all_nans) -{ - TCGv_i64 tcg_flags = tcg_temp_new_i64(); - TCGv_ptr fpst = fpstatus_ptr(size == MO_16 ? FPST_FPCR_F16 : FPST_FPCR); - - if (size == MO_64) { - TCGv_i64 tcg_vn, tcg_vm; - - tcg_vn = read_fp_dreg(s, rn); - if (cmp_with_zero) { - tcg_vm = tcg_constant_i64(0); - } else { - tcg_vm = read_fp_dreg(s, rm); - } - if (signal_all_nans) { - gen_helper_vfp_cmped_a64(tcg_flags, tcg_vn, tcg_vm, fpst); - } else { - gen_helper_vfp_cmpd_a64(tcg_flags, tcg_vn, tcg_vm, fpst); - } - } else { - TCGv_i32 tcg_vn = tcg_temp_new_i32(); - TCGv_i32 tcg_vm = tcg_temp_new_i32(); - - read_vec_element_i32(s, tcg_vn, rn, 0, size); - if (cmp_with_zero) { - tcg_gen_movi_i32(tcg_vm, 0); - } else { - read_vec_element_i32(s, tcg_vm, rm, 0, size); - } - - switch (size) { - case MO_32: - if (signal_all_nans) { - gen_helper_vfp_cmpes_a64(tcg_flags, tcg_vn, tcg_vm, fpst); - } else { - gen_helper_vfp_cmps_a64(tcg_flags, tcg_vn, tcg_vm, fpst); - } - break; - case MO_16: - if (signal_all_nans) { - gen_helper_vfp_cmpeh_a64(tcg_flags, tcg_vn, tcg_vm, fpst); - } else { - gen_helper_vfp_cmph_a64(tcg_flags, tcg_vn, tcg_vm, fpst); - } - break; - default: - g_assert_not_reached(); - } - } - - gen_set_nzcv(tcg_flags); -} - -/* Floating point compare - * 31 30 29 28 24 23 22 21 20 16 15 14 13 10 9 5 4 0 - * +---+---+---+-----------+------+---+------+-----+---------+------+-------+ - * | M | 0 | S | 1 1 1 1 0 | type | 1 | Rm | op | 1 0 0 0 | Rn | op2 | - * +---+---+---+-----------+------+---+------+-----+---------+------+-------+ - */ -static void disas_fp_compare(DisasContext *s, uint32_t insn) -{ - unsigned int mos, type, rm, op, rn, opc, op2r; - int size; - - mos = extract32(insn, 29, 3); - type = extract32(insn, 22, 2); - rm = extract32(insn, 16, 5); - op = extract32(insn, 14, 2); - rn = extract32(insn, 5, 5); - opc = extract32(insn, 3, 2); - op2r = extract32(insn, 0, 3); - - if (mos || op || op2r) { - unallocated_encoding(s); - return; - } - - switch (type) { - case 0: - size = MO_32; - break; - case 1: - size = MO_64; - break; - case 3: - size = MO_16; - if (dc_isar_feature(aa64_fp16, s)) { - break; - } - /* fallthru */ - default: - unallocated_encoding(s); - return; - } - - if (!fp_access_check(s)) { - return; - } - - handle_fp_compare(s, size, rn, rm, opc & 1, opc & 2); -} - -/* Floating point conditional compare - * 31 30 29 28 24 23 22 21 20 16 15 12 11 10 9 5 4 3 0 - * +---+---+---+-----------+------+---+------+------+-----+------+----+------+ - * | M | 0 | S | 1 1 1 1 0 | type | 1 | Rm | cond | 0 1 | Rn | op | nzcv | - * +---+---+---+-----------+------+---+------+------+-----+------+----+------+ - */ -static void disas_fp_ccomp(DisasContext *s, uint32_t insn) -{ - unsigned int mos, type, rm, cond, rn, op, nzcv; - TCGLabel *label_continue = NULL; - int size; - - mos = extract32(insn, 29, 3); - type = extract32(insn, 22, 2); - rm = extract32(insn, 16, 5); - cond = extract32(insn, 12, 4); - rn = extract32(insn, 5, 5); - op = extract32(insn, 4, 1); - nzcv = extract32(insn, 0, 4); - - if (mos) { - unallocated_encoding(s); - return; - } - - switch (type) { - case 0: - size = MO_32; - break; - case 1: - size = MO_64; - break; - case 3: - size = MO_16; - if (dc_isar_feature(aa64_fp16, s)) { - break; - } - /* fallthru */ - default: - unallocated_encoding(s); - return; - } - - if (!fp_access_check(s)) { - return; - } - - if (cond < 0x0e) { /* not always */ - TCGLabel *label_match = gen_new_label(); - label_continue = gen_new_label(); - arm_gen_test_cc(cond, label_match); - /* nomatch: */ - gen_set_nzcv(tcg_constant_i64(nzcv << 28)); - tcg_gen_br(label_continue); - gen_set_label(label_match); - } - - handle_fp_compare(s, size, rn, rm, false, op); - - if (cond < 0x0e) { - gen_set_label(label_continue); - } -} - /* Floating-point data-processing (1 source) - half precision */ static void handle_fp_1src_half(DisasContext *s, int opcode, int rd, int rn) { @@ -XXX,XX +XXX,XX @@ static void disas_data_proc_fp(DisasContext *s, uint32_t insn) disas_fp_fixed_conv(s, insn); } else { switch (extract32(insn, 10, 2)) { - case 1: - /* Floating point conditional compare */ - disas_fp_ccomp(s, insn); - break; - case 2: - /* Floating point data-processing (2 source) */ - unallocated_encoding(s); /* in decodetree */ - break; - case 3: - /* Floating point conditional select */ + case 1: /* Floating point conditional compare */ + case 2: /* Floating point data-processing (2 source) */ + case 3: /* Floating point conditional select */ unallocated_encoding(s); /* in decodetree */ break; case 0: @@ -XXX,XX +XXX,XX @@ static void disas_data_proc_fp(DisasContext *s, uint32_t insn) break; case 1: /* [15:12] == xx10 */ /* Floating point compare */ - disas_fp_compare(s, insn); + unallocated_encoding(s); /* in decodetree */ break; case 2: /* [15:12] == x100 */ /* Floating point data-processing (1 source) */ -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> These opcodes are only supported as vector operations, not as advsimd scalar. Set only_in_vector, and remove the unreachable implementation of scalar fneg. Reported-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Message-id: 20241211163036.2297116-24-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/translate-a64.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc_fp16(DisasContext *s, uint32_t insn) break; case 0x2f: /* FABS */ case 0x6f: /* FNEG */ + only_in_vector = true; need_fpst = false; break; case 0x7d: /* FRSQRTE */ + break; case 0x7f: /* FSQRT (vector) */ + only_in_vector = true; break; default: unallocated_encoding(s); @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc_fp16(DisasContext *s, uint32_t insn) case 0x7b: /* FCVTZU */ gen_helper_advsimd_f16touinth(tcg_res, tcg_op, tcg_fpstatus); break; - case 0x6f: /* FNEG */ - tcg_gen_xori_i32(tcg_res, tcg_op, 0x8000); - break; case 0x7d: /* FRSQRTE */ gen_helper_rsqrte_f16(tcg_res, tcg_op, tcg_fpstatus); break; -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-25-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 7 +++ target/arm/tcg/translate-a64.c | 105 +++++++++++++++++++++++---------- 2 files changed, 81 insertions(+), 31 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ @rr_h ........ ... ..... ...... rn:5 rd:5 &rr_e esz=1 @rr_d ........ ... ..... ...... rn:5 rd:5 &rr_e esz=3 @rr_sd ........ ... ..... ...... rn:5 rd:5 &rr_e esz=%esz_sd +@rr_hsd ........ ... ..... ...... rn:5 rd:5 &rr_e esz=%esz_hsd @rrr_b ........ ... rm:5 ...... rn:5 rd:5 &rrr_e esz=0 @rrr_h ........ ... rm:5 ...... rn:5 rd:5 &rrr_e esz=1 @@ -XXX,XX +XXX,XX @@ FMAXV_s 0110 1110 00 11000 01111 10 ..... ..... @rr_q1e2 FMINV_h 0.00 1110 10 11000 01111 10 ..... ..... @qrr_h FMINV_s 0110 1110 10 11000 01111 10 ..... ..... @rr_q1e2 +# Floating-point data processing (1 source) + +FMOV_s 00011110 .. 1 000000 10000 ..... ..... @rr_hsd +FABS_s 00011110 .. 1 000001 10000 ..... ..... @rr_hsd +FNEG_s 00011110 .. 1 000010 10000 ..... ..... @rr_hsd + # Floating-point Immediate FMOVI_s 0001 1110 .. 1 imm:8 100 00000 rd:5 esz=%esz_hsd diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static bool trans_CSEL(DisasContext *s, arg_CSEL *a) return true; } +typedef struct FPScalar1Int { + void (*gen_h)(TCGv_i32, TCGv_i32); + void (*gen_s)(TCGv_i32, TCGv_i32); + void (*gen_d)(TCGv_i64, TCGv_i64); +} FPScalar1Int; + +static bool do_fp1_scalar_int(DisasContext *s, arg_rr_e *a, + const FPScalar1Int *f) +{ + switch (a->esz) { + case MO_64: + if (fp_access_check(s)) { + TCGv_i64 t = read_fp_dreg(s, a->rn); + f->gen_d(t, t); + write_fp_dreg(s, a->rd, t); + } + break; + case MO_32: + if (fp_access_check(s)) { + TCGv_i32 t = read_fp_sreg(s, a->rn); + f->gen_s(t, t); + write_fp_sreg(s, a->rd, t); + } + break; + case MO_16: + if (!dc_isar_feature(aa64_fp16, s)) { + return false; + } + if (fp_access_check(s)) { + TCGv_i32 t = read_fp_hreg(s, a->rn); + f->gen_h(t, t); + write_fp_sreg(s, a->rd, t); + } + break; + default: + return false; + } + return true; +} + +static const FPScalar1Int f_scalar_fmov = { + tcg_gen_mov_i32, + tcg_gen_mov_i32, + tcg_gen_mov_i64, +}; +TRANS(FMOV_s, do_fp1_scalar_int, a, &f_scalar_fmov) + +static const FPScalar1Int f_scalar_fabs = { + gen_vfp_absh, + gen_vfp_abss, + gen_vfp_absd, +}; +TRANS(FABS_s, do_fp1_scalar_int, a, &f_scalar_fabs) + +static const FPScalar1Int f_scalar_fneg = { + gen_vfp_negh, + gen_vfp_negs, + gen_vfp_negd, +}; +TRANS(FNEG_s, do_fp1_scalar_int, a, &f_scalar_fneg) + /* Floating-point data-processing (1 source) - half precision */ static void handle_fp_1src_half(DisasContext *s, int opcode, int rd, int rn) { @@ -XXX,XX +XXX,XX @@ static void handle_fp_1src_half(DisasContext *s, int opcode, int rd, int rn) TCGv_i32 tcg_res = tcg_temp_new_i32(); switch (opcode) { - case 0x0: /* FMOV */ - tcg_gen_mov_i32(tcg_res, tcg_op); - break; - case 0x1: /* FABS */ - gen_vfp_absh(tcg_res, tcg_op); - break; - case 0x2: /* FNEG */ - gen_vfp_negh(tcg_res, tcg_op); - break; case 0x3: /* FSQRT */ fpst = fpstatus_ptr(FPST_FPCR_F16); gen_helper_sqrt_f16(tcg_res, tcg_op, fpst); @@ -XXX,XX +XXX,XX @@ static void handle_fp_1src_half(DisasContext *s, int opcode, int rd, int rn) gen_helper_advsimd_rinth(tcg_res, tcg_op, fpst); break; default: + case 0x0: /* FMOV */ + case 0x1: /* FABS */ + case 0x2: /* FNEG */ g_assert_not_reached(); } @@ -XXX,XX +XXX,XX @@ static void handle_fp_1src_single(DisasContext *s, int opcode, int rd, int rn) tcg_res = tcg_temp_new_i32(); switch (opcode) { - case 0x0: /* FMOV */ - tcg_gen_mov_i32(tcg_res, tcg_op); - goto done; - case 0x1: /* FABS */ - gen_vfp_abss(tcg_res, tcg_op); - goto done; - case 0x2: /* FNEG */ - gen_vfp_negs(tcg_res, tcg_op); - goto done; case 0x3: /* FSQRT */ gen_helper_vfp_sqrts(tcg_res, tcg_op, tcg_env); goto done; @@ -XXX,XX +XXX,XX @@ static void handle_fp_1src_single(DisasContext *s, int opcode, int rd, int rn) gen_fpst = gen_helper_frint64_s; break; default: + case 0x0: /* FMOV */ + case 0x1: /* FABS */ + case 0x2: /* FNEG */ g_assert_not_reached(); } @@ -XXX,XX +XXX,XX @@ static void handle_fp_1src_double(DisasContext *s, int opcode, int rd, int rn) TCGv_ptr fpst; int rmode = -1; - switch (opcode) { - case 0x0: /* FMOV */ - gen_gvec_fn2(s, false, rd, rn, tcg_gen_gvec_mov, 0); - return; - } - tcg_op = read_fp_dreg(s, rn); tcg_res = tcg_temp_new_i64(); switch (opcode) { - case 0x1: /* FABS */ - gen_vfp_absd(tcg_res, tcg_op); - goto done; - case 0x2: /* FNEG */ - gen_vfp_negd(tcg_res, tcg_op); - goto done; case 0x3: /* FSQRT */ gen_helper_vfp_sqrtd(tcg_res, tcg_op, tcg_env); goto done; @@ -XXX,XX +XXX,XX @@ static void handle_fp_1src_double(DisasContext *s, int opcode, int rd, int rn) gen_fpst = gen_helper_frint64_d; break; default: + case 0x0: /* FMOV */ + case 0x1: /* FABS */ + case 0x2: /* FNEG */ g_assert_not_reached(); } @@ -XXX,XX +XXX,XX @@ static void disas_fp_1src(DisasContext *s, uint32_t insn) goto do_unallocated; } /* fall through */ - case 0x0 ... 0x3: + case 0x3: case 0x8 ... 0xc: case 0xe ... 0xf: /* 32-to-32 and 64-to-64 ops */ @@ -XXX,XX +XXX,XX @@ static void disas_fp_1src(DisasContext *s, uint32_t insn) default: do_unallocated: + case 0x0: /* FMOV */ + case 0x1: /* FABS */ + case 0x2: /* FNEG */ unallocated_encoding(s); break; } -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Pass fpstatus not env, like most other fp helpers. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-26-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/helper.h | 6 +++--- target/arm/tcg/translate-a64.c | 15 +++++++-------- target/arm/tcg/translate-vfp.c | 6 +++--- target/arm/vfp_helper.c | 12 ++++++------ 4 files changed, 19 insertions(+), 20 deletions(-) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_3(vfp_maxnumd, f64, f64, f64, ptr) DEF_HELPER_3(vfp_minnumh, f16, f16, f16, ptr) DEF_HELPER_3(vfp_minnums, f32, f32, f32, ptr) DEF_HELPER_3(vfp_minnumd, f64, f64, f64, ptr) -DEF_HELPER_2(vfp_sqrth, f16, f16, env) -DEF_HELPER_2(vfp_sqrts, f32, f32, env) -DEF_HELPER_2(vfp_sqrtd, f64, f64, env) +DEF_HELPER_2(vfp_sqrth, f16, f16, ptr) +DEF_HELPER_2(vfp_sqrts, f32, f32, ptr) +DEF_HELPER_2(vfp_sqrtd, f64, f64, ptr) DEF_HELPER_3(vfp_cmph, void, f16, f16, env) DEF_HELPER_3(vfp_cmps, void, f32, f32, env) DEF_HELPER_3(vfp_cmpd, void, f64, f64, env) diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static void handle_fp_1src_single(DisasContext *s, int opcode, int rd, int rn) switch (opcode) { case 0x3: /* FSQRT */ - gen_helper_vfp_sqrts(tcg_res, tcg_op, tcg_env); - goto done; + gen_fpst = gen_helper_vfp_sqrts; + break; case 0x6: /* BFCVT */ gen_fpst = gen_helper_bfcvt; break; @@ -XXX,XX +XXX,XX @@ static void handle_fp_1src_single(DisasContext *s, int opcode, int rd, int rn) gen_fpst(tcg_res, tcg_op, fpst); } - done: write_fp_sreg(s, rd, tcg_res); } @@ -XXX,XX +XXX,XX @@ static void handle_fp_1src_double(DisasContext *s, int opcode, int rd, int rn) switch (opcode) { case 0x3: /* FSQRT */ - gen_helper_vfp_sqrtd(tcg_res, tcg_op, tcg_env); - goto done; + gen_fpst = gen_helper_vfp_sqrtd; + break; case 0x8: /* FRINTN */ case 0x9: /* FRINTP */ case 0xa: /* FRINTM */ @@ -XXX,XX +XXX,XX @@ static void handle_fp_1src_double(DisasContext *s, int opcode, int rd, int rn) gen_fpst(tcg_res, tcg_op, fpst); } - done: write_fp_dreg(s, rd, tcg_res); } @@ -XXX,XX +XXX,XX @@ static void handle_2misc_64(DisasContext *s, int opcode, bool u, gen_vfp_negd(tcg_rd, tcg_rn); break; case 0x7f: /* FSQRT */ - gen_helper_vfp_sqrtd(tcg_rd, tcg_rn, tcg_env); + gen_helper_vfp_sqrtd(tcg_rd, tcg_rn, tcg_fpstatus); break; case 0x1a: /* FCVTNS */ case 0x1b: /* FCVTMS */ @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) handle_2misc_fcmp_zero(s, opcode, false, u, is_q, size, rn, rd); return; case 0x7f: /* FSQRT */ + need_fpstatus = true; if (size == 3 && !is_q) { unallocated_encoding(s); return; @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) gen_vfp_negs(tcg_res, tcg_op); break; case 0x7f: /* FSQRT */ - gen_helper_vfp_sqrts(tcg_res, tcg_op, tcg_env); + gen_helper_vfp_sqrts(tcg_res, tcg_op, tcg_fpstatus); break; case 0x1a: /* FCVTNS */ case 0x1b: /* FCVTMS */ diff --git a/target/arm/tcg/translate-vfp.c b/target/arm/tcg/translate-vfp.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-vfp.c +++ b/target/arm/tcg/translate-vfp.c @@ -XXX,XX +XXX,XX @@ DO_VFP_2OP(VNEG, dp, gen_vfp_negd, aa32_fpdp_v2) static void gen_VSQRT_hp(TCGv_i32 vd, TCGv_i32 vm) { - gen_helper_vfp_sqrth(vd, vm, tcg_env); + gen_helper_vfp_sqrth(vd, vm, fpstatus_ptr(FPST_FPCR_F16)); } static void gen_VSQRT_sp(TCGv_i32 vd, TCGv_i32 vm) { - gen_helper_vfp_sqrts(vd, vm, tcg_env); + gen_helper_vfp_sqrts(vd, vm, fpstatus_ptr(FPST_FPCR)); } static void gen_VSQRT_dp(TCGv_i64 vd, TCGv_i64 vm) { - gen_helper_vfp_sqrtd(vd, vm, tcg_env); + gen_helper_vfp_sqrtd(vd, vm, fpstatus_ptr(FPST_FPCR)); } DO_VFP_2OP(VSQRT, hp, gen_VSQRT_hp, aa32_fp16_arith) diff --git a/target/arm/vfp_helper.c b/target/arm/vfp_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vfp_helper.c +++ b/target/arm/vfp_helper.c @@ -XXX,XX +XXX,XX @@ VFP_BINOP(minnum) VFP_BINOP(maxnum) #undef VFP_BINOP -dh_ctype_f16 VFP_HELPER(sqrt, h)(dh_ctype_f16 a, CPUARMState *env) +dh_ctype_f16 VFP_HELPER(sqrt, h)(dh_ctype_f16 a, void *fpstp) { - return float16_sqrt(a, &env->vfp.fp_status_f16); + return float16_sqrt(a, fpstp); } -float32 VFP_HELPER(sqrt, s)(float32 a, CPUARMState *env) +float32 VFP_HELPER(sqrt, s)(float32 a, void *fpstp) { - return float32_sqrt(a, &env->vfp.fp_status); + return float32_sqrt(a, fpstp); } -float64 VFP_HELPER(sqrt, d)(float64 a, CPUARMState *env) +float64 VFP_HELPER(sqrt, d)(float64 a, void *fpstp) { - return float64_sqrt(a, &env->vfp.fp_status); + return float64_sqrt(a, fpstp); } static void softfloat_to_vfp_compare(CPUARMState *env, FloatRelation cmp) -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> This function is identical with helper_vfp_sqrth. Replace all uses. Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-27-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/helper-a64.h | 1 - target/arm/tcg/helper-a64.c | 11 ----------- target/arm/tcg/translate-a64.c | 4 ++-- 3 files changed, 2 insertions(+), 14 deletions(-) diff --git a/target/arm/tcg/helper-a64.h b/target/arm/tcg/helper-a64.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/helper-a64.h +++ b/target/arm/tcg/helper-a64.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_2(advsimd_rinth_exact, f16, f16, ptr) DEF_HELPER_2(advsimd_rinth, f16, f16, ptr) DEF_HELPER_2(advsimd_f16tosinth, i32, f16, ptr) DEF_HELPER_2(advsimd_f16touinth, i32, f16, ptr) -DEF_HELPER_2(sqrt_f16, f16, f16, ptr) DEF_HELPER_2(exception_return, void, env, i64) DEF_HELPER_FLAGS_2(dc_zva, TCG_CALL_NO_WG, void, env, i64) diff --git a/target/arm/tcg/helper-a64.c b/target/arm/tcg/helper-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/helper-a64.c +++ b/target/arm/tcg/helper-a64.c @@ -XXX,XX +XXX,XX @@ illegal_return: "resuming execution at 0x%" PRIx64 "\n", cur_el, env->pc); } -/* - * Square Root and Reciprocal square root - */ - -uint32_t HELPER(sqrt_f16)(uint32_t a, void *fpstp) -{ - float_status *s = fpstp; - - return float16_sqrt(a, s); -} - void HELPER(dc_zva)(CPUARMState *env, uint64_t vaddr_in) { uintptr_t ra = GETPC(); diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static void handle_fp_1src_half(DisasContext *s, int opcode, int rd, int rn) switch (opcode) { case 0x3: /* FSQRT */ fpst = fpstatus_ptr(FPST_FPCR_F16); - gen_helper_sqrt_f16(tcg_res, tcg_op, fpst); + gen_helper_vfp_sqrth(tcg_res, tcg_op, fpst); break; case 0x8: /* FRINTN */ case 0x9: /* FRINTP */ @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc_fp16(DisasContext *s, uint32_t insn) gen_helper_rsqrte_f16(tcg_res, tcg_op, tcg_fpstatus); break; case 0x7f: /* FSQRT */ - gen_helper_sqrt_f16(tcg_res, tcg_op, tcg_fpstatus); + gen_helper_vfp_sqrth(tcg_res, tcg_op, tcg_fpstatus); break; default: g_assert_not_reached(); -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-28-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 1 + target/arm/tcg/translate-a64.c | 72 ++++++++++++++++++++++++++++------ 2 files changed, 62 insertions(+), 11 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ FMINV_s 0110 1110 10 11000 01111 10 ..... ..... @rr_q1e2 FMOV_s 00011110 .. 1 000000 10000 ..... ..... @rr_hsd FABS_s 00011110 .. 1 000001 10000 ..... ..... @rr_hsd FNEG_s 00011110 .. 1 000010 10000 ..... ..... @rr_hsd +FSQRT_s 00011110 .. 1 000011 10000 ..... ..... @rr_hsd # Floating-point Immediate diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static const FPScalar1Int f_scalar_fneg = { }; TRANS(FNEG_s, do_fp1_scalar_int, a, &f_scalar_fneg) +typedef struct FPScalar1 { + void (*gen_h)(TCGv_i32, TCGv_i32, TCGv_ptr); + void (*gen_s)(TCGv_i32, TCGv_i32, TCGv_ptr); + void (*gen_d)(TCGv_i64, TCGv_i64, TCGv_ptr); +} FPScalar1; + +static bool do_fp1_scalar(DisasContext *s, arg_rr_e *a, + const FPScalar1 *f, int rmode) +{ + TCGv_i32 tcg_rmode = NULL; + TCGv_ptr fpst; + TCGv_i64 t64; + TCGv_i32 t32; + int check = fp_access_check_scalar_hsd(s, a->esz); + + if (check <= 0) { + return check == 0; + } + + fpst = fpstatus_ptr(a->esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR); + if (rmode >= 0) { + tcg_rmode = gen_set_rmode(rmode, fpst); + } + + switch (a->esz) { + case MO_64: + t64 = read_fp_dreg(s, a->rn); + f->gen_d(t64, t64, fpst); + write_fp_dreg(s, a->rd, t64); + break; + case MO_32: + t32 = read_fp_sreg(s, a->rn); + f->gen_s(t32, t32, fpst); + write_fp_sreg(s, a->rd, t32); + break; + case MO_16: + t32 = read_fp_hreg(s, a->rn); + f->gen_h(t32, t32, fpst); + write_fp_sreg(s, a->rd, t32); + break; + default: + g_assert_not_reached(); + } + + if (rmode >= 0) { + gen_restore_rmode(tcg_rmode, fpst); + } + return true; +} + +static const FPScalar1 f_scalar_fsqrt = { + gen_helper_vfp_sqrth, + gen_helper_vfp_sqrts, + gen_helper_vfp_sqrtd, +}; +TRANS(FSQRT_s, do_fp1_scalar, a, &f_scalar_fsqrt, -1) + /* Floating-point data-processing (1 source) - half precision */ static void handle_fp_1src_half(DisasContext *s, int opcode, int rd, int rn) { @@ -XXX,XX +XXX,XX @@ static void handle_fp_1src_half(DisasContext *s, int opcode, int rd, int rn) TCGv_i32 tcg_res = tcg_temp_new_i32(); switch (opcode) { - case 0x3: /* FSQRT */ - fpst = fpstatus_ptr(FPST_FPCR_F16); - gen_helper_vfp_sqrth(tcg_res, tcg_op, fpst); - break; case 0x8: /* FRINTN */ case 0x9: /* FRINTP */ case 0xa: /* FRINTM */ @@ -XXX,XX +XXX,XX @@ static void handle_fp_1src_half(DisasContext *s, int opcode, int rd, int rn) case 0x0: /* FMOV */ case 0x1: /* FABS */ case 0x2: /* FNEG */ + case 0x3: /* FSQRT */ g_assert_not_reached(); } @@ -XXX,XX +XXX,XX @@ static void handle_fp_1src_single(DisasContext *s, int opcode, int rd, int rn) tcg_res = tcg_temp_new_i32(); switch (opcode) { - case 0x3: /* FSQRT */ - gen_fpst = gen_helper_vfp_sqrts; - break; case 0x6: /* BFCVT */ gen_fpst = gen_helper_bfcvt; break; @@ -XXX,XX +XXX,XX @@ static void handle_fp_1src_single(DisasContext *s, int opcode, int rd, int rn) case 0x0: /* FMOV */ case 0x1: /* FABS */ case 0x2: /* FNEG */ + case 0x3: /* FSQRT */ g_assert_not_reached(); } @@ -XXX,XX +XXX,XX @@ static void handle_fp_1src_double(DisasContext *s, int opcode, int rd, int rn) tcg_res = tcg_temp_new_i64(); switch (opcode) { - case 0x3: /* FSQRT */ - gen_fpst = gen_helper_vfp_sqrtd; - break; case 0x8: /* FRINTN */ case 0x9: /* FRINTP */ case 0xa: /* FRINTM */ @@ -XXX,XX +XXX,XX @@ static void handle_fp_1src_double(DisasContext *s, int opcode, int rd, int rn) case 0x0: /* FMOV */ case 0x1: /* FABS */ case 0x2: /* FNEG */ + case 0x3: /* FSQRT */ g_assert_not_reached(); } @@ -XXX,XX +XXX,XX @@ static void disas_fp_1src(DisasContext *s, uint32_t insn) goto do_unallocated; } /* fall through */ - case 0x3: case 0x8 ... 0xc: case 0xe ... 0xf: /* 32-to-32 and 64-to-64 ops */ @@ -XXX,XX +XXX,XX @@ static void disas_fp_1src(DisasContext *s, uint32_t insn) case 0x0: /* FMOV */ case 0x1: /* FABS */ case 0x2: /* FNEG */ + case 0x3: /* FSQRT */ unallocated_encoding(s); break; } -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Remove handle_fp_1src_half as these were the last insns decoded by that function. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-29-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 8 +++ target/arm/tcg/translate-a64.c | 117 +++++++++++---------------------- 2 files changed, 46 insertions(+), 79 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ FABS_s 00011110 .. 1 000001 10000 ..... ..... @rr_hsd FNEG_s 00011110 .. 1 000010 10000 ..... ..... @rr_hsd FSQRT_s 00011110 .. 1 000011 10000 ..... ..... @rr_hsd +FRINTN_s 00011110 .. 1 001000 10000 ..... ..... @rr_hsd +FRINTP_s 00011110 .. 1 001001 10000 ..... ..... @rr_hsd +FRINTM_s 00011110 .. 1 001010 10000 ..... ..... @rr_hsd +FRINTZ_s 00011110 .. 1 001011 10000 ..... ..... @rr_hsd +FRINTA_s 00011110 .. 1 001100 10000 ..... ..... @rr_hsd +FRINTX_s 00011110 .. 1 001110 10000 ..... ..... @rr_hsd +FRINTI_s 00011110 .. 1 001111 10000 ..... ..... @rr_hsd + # Floating-point Immediate FMOVI_s 0001 1110 .. 1 imm:8 100 00000 rd:5 esz=%esz_hsd diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static const FPScalar1 f_scalar_fsqrt = { }; TRANS(FSQRT_s, do_fp1_scalar, a, &f_scalar_fsqrt, -1) -/* Floating-point data-processing (1 source) - half precision */ -static void handle_fp_1src_half(DisasContext *s, int opcode, int rd, int rn) -{ - TCGv_ptr fpst = NULL; - TCGv_i32 tcg_op = read_fp_hreg(s, rn); - TCGv_i32 tcg_res = tcg_temp_new_i32(); +static const FPScalar1 f_scalar_frint = { + gen_helper_advsimd_rinth, + gen_helper_rints, + gen_helper_rintd, +}; +TRANS(FRINTN_s, do_fp1_scalar, a, &f_scalar_frint, FPROUNDING_TIEEVEN) +TRANS(FRINTP_s, do_fp1_scalar, a, &f_scalar_frint, FPROUNDING_POSINF) +TRANS(FRINTM_s, do_fp1_scalar, a, &f_scalar_frint, FPROUNDING_NEGINF) +TRANS(FRINTZ_s, do_fp1_scalar, a, &f_scalar_frint, FPROUNDING_ZERO) +TRANS(FRINTA_s, do_fp1_scalar, a, &f_scalar_frint, FPROUNDING_TIEAWAY) +TRANS(FRINTI_s, do_fp1_scalar, a, &f_scalar_frint, -1) - switch (opcode) { - case 0x8: /* FRINTN */ - case 0x9: /* FRINTP */ - case 0xa: /* FRINTM */ - case 0xb: /* FRINTZ */ - case 0xc: /* FRINTA */ - { - TCGv_i32 tcg_rmode; - - fpst = fpstatus_ptr(FPST_FPCR_F16); - tcg_rmode = gen_set_rmode(opcode & 7, fpst); - gen_helper_advsimd_rinth(tcg_res, tcg_op, fpst); - gen_restore_rmode(tcg_rmode, fpst); - break; - } - case 0xe: /* FRINTX */ - fpst = fpstatus_ptr(FPST_FPCR_F16); - gen_helper_advsimd_rinth_exact(tcg_res, tcg_op, fpst); - break; - case 0xf: /* FRINTI */ - fpst = fpstatus_ptr(FPST_FPCR_F16); - gen_helper_advsimd_rinth(tcg_res, tcg_op, fpst); - break; - default: - case 0x0: /* FMOV */ - case 0x1: /* FABS */ - case 0x2: /* FNEG */ - case 0x3: /* FSQRT */ - g_assert_not_reached(); - } - - write_fp_sreg(s, rd, tcg_res); -} +static const FPScalar1 f_scalar_frintx = { + gen_helper_advsimd_rinth_exact, + gen_helper_rints_exact, + gen_helper_rintd_exact, +}; +TRANS(FRINTX_s, do_fp1_scalar, a, &f_scalar_frintx, -1) /* Floating-point data-processing (1 source) - single precision */ static void handle_fp_1src_single(DisasContext *s, int opcode, int rd, int rn) @@ -XXX,XX +XXX,XX @@ static void handle_fp_1src_single(DisasContext *s, int opcode, int rd, int rn) case 0x6: /* BFCVT */ gen_fpst = gen_helper_bfcvt; break; - case 0x8: /* FRINTN */ - case 0x9: /* FRINTP */ - case 0xa: /* FRINTM */ - case 0xb: /* FRINTZ */ - case 0xc: /* FRINTA */ - rmode = opcode & 7; - gen_fpst = gen_helper_rints; - break; - case 0xe: /* FRINTX */ - gen_fpst = gen_helper_rints_exact; - break; - case 0xf: /* FRINTI */ - gen_fpst = gen_helper_rints; - break; case 0x10: /* FRINT32Z */ rmode = FPROUNDING_ZERO; gen_fpst = gen_helper_frint32_s; @@ -XXX,XX +XXX,XX @@ static void handle_fp_1src_single(DisasContext *s, int opcode, int rd, int rn) case 0x1: /* FABS */ case 0x2: /* FNEG */ case 0x3: /* FSQRT */ + case 0x8: /* FRINTN */ + case 0x9: /* FRINTP */ + case 0xa: /* FRINTM */ + case 0xb: /* FRINTZ */ + case 0xc: /* FRINTA */ + case 0xe: /* FRINTX */ + case 0xf: /* FRINTI */ g_assert_not_reached(); } @@ -XXX,XX +XXX,XX @@ static void handle_fp_1src_double(DisasContext *s, int opcode, int rd, int rn) tcg_res = tcg_temp_new_i64(); switch (opcode) { - case 0x8: /* FRINTN */ - case 0x9: /* FRINTP */ - case 0xa: /* FRINTM */ - case 0xb: /* FRINTZ */ - case 0xc: /* FRINTA */ - rmode = opcode & 7; - gen_fpst = gen_helper_rintd; - break; - case 0xe: /* FRINTX */ - gen_fpst = gen_helper_rintd_exact; - break; - case 0xf: /* FRINTI */ - gen_fpst = gen_helper_rintd; - break; case 0x10: /* FRINT32Z */ rmode = FPROUNDING_ZERO; gen_fpst = gen_helper_frint32_d; @@ -XXX,XX +XXX,XX @@ static void handle_fp_1src_double(DisasContext *s, int opcode, int rd, int rn) case 0x1: /* FABS */ case 0x2: /* FNEG */ case 0x3: /* FSQRT */ + case 0x8: /* FRINTN */ + case 0x9: /* FRINTP */ + case 0xa: /* FRINTM */ + case 0xb: /* FRINTZ */ + case 0xc: /* FRINTA */ + case 0xe: /* FRINTX */ + case 0xf: /* FRINTI */ g_assert_not_reached(); } @@ -XXX,XX +XXX,XX @@ static void disas_fp_1src(DisasContext *s, uint32_t insn) if (type > 1 || !dc_isar_feature(aa64_frint, s)) { goto do_unallocated; } - /* fall through */ - case 0x8 ... 0xc: - case 0xe ... 0xf: /* 32-to-32 and 64-to-64 ops */ switch (type) { case 0: @@ -XXX,XX +XXX,XX @@ static void disas_fp_1src(DisasContext *s, uint32_t insn) handle_fp_1src_double(s, opcode, rd, rn); break; case 3: - if (!dc_isar_feature(aa64_fp16, s)) { - goto do_unallocated; - } - - if (!fp_access_check(s)) { - return; - } - handle_fp_1src_half(s, opcode, rd, rn); - break; default: goto do_unallocated; } @@ -XXX,XX +XXX,XX @@ static void disas_fp_1src(DisasContext *s, uint32_t insn) case 0x1: /* FABS */ case 0x2: /* FNEG */ case 0x3: /* FSQRT */ + case 0x8: /* FRINTN */ + case 0x9: /* FRINTP */ + case 0xa: /* FRINTM */ + case 0xb: /* FRINTZ */ + case 0xc: /* FRINTA */ + case 0xe: /* FRINTX */ + case 0xf: /* FRINTI */ unallocated_encoding(s); break; } -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-30-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 3 +++ target/arm/tcg/translate-a64.c | 26 +++++++------------------- 2 files changed, 10 insertions(+), 19 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ &qrrrr_e q rd rn rm ra esz @rr_h ........ ... ..... ...... rn:5 rd:5 &rr_e esz=1 +@rr_s ........ ... ..... ...... rn:5 rd:5 &rr_e esz=2 @rr_d ........ ... ..... ...... rn:5 rd:5 &rr_e esz=3 @rr_sd ........ ... ..... ...... rn:5 rd:5 &rr_e esz=%esz_sd @rr_hsd ........ ... ..... ...... rn:5 rd:5 &rr_e esz=%esz_hsd @@ -XXX,XX +XXX,XX @@ FRINTA_s 00011110 .. 1 001100 10000 ..... ..... @rr_hsd FRINTX_s 00011110 .. 1 001110 10000 ..... ..... @rr_hsd FRINTI_s 00011110 .. 1 001111 10000 ..... ..... @rr_hsd +BFCVT_s 00011110 01 1 000110 10000 ..... ..... @rr_s + # Floating-point Immediate FMOVI_s 0001 1110 .. 1 imm:8 100 00000 rd:5 esz=%esz_hsd diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static const FPScalar1 f_scalar_frintx = { }; TRANS(FRINTX_s, do_fp1_scalar, a, &f_scalar_frintx, -1) +static const FPScalar1 f_scalar_bfcvt = { + .gen_s = gen_helper_bfcvt, +}; +TRANS_FEAT(BFCVT_s, aa64_bf16, do_fp1_scalar, a, &f_scalar_bfcvt, -1) + /* Floating-point data-processing (1 source) - single precision */ static void handle_fp_1src_single(DisasContext *s, int opcode, int rd, int rn) { @@ -XXX,XX +XXX,XX @@ static void handle_fp_1src_single(DisasContext *s, int opcode, int rd, int rn) tcg_res = tcg_temp_new_i32(); switch (opcode) { - case 0x6: /* BFCVT */ - gen_fpst = gen_helper_bfcvt; - break; case 0x10: /* FRINT32Z */ rmode = FPROUNDING_ZERO; gen_fpst = gen_helper_frint32_s; @@ -XXX,XX +XXX,XX @@ static void handle_fp_1src_single(DisasContext *s, int opcode, int rd, int rn) case 0x1: /* FABS */ case 0x2: /* FNEG */ case 0x3: /* FSQRT */ + case 0x6: /* BFCVT */ case 0x8: /* FRINTN */ case 0x9: /* FRINTP */ case 0xa: /* FRINTM */ @@ -XXX,XX +XXX,XX @@ static void disas_fp_1src(DisasContext *s, uint32_t insn) } break; - case 0x6: - switch (type) { - case 1: /* BFCVT */ - if (!dc_isar_feature(aa64_bf16, s)) { - goto do_unallocated; - } - if (!fp_access_check(s)) { - return; - } - handle_fp_1src_single(s, opcode, rd, rn); - break; - default: - goto do_unallocated; - } - break; - default: do_unallocated: case 0x0: /* FMOV */ case 0x1: /* FABS */ case 0x2: /* FNEG */ case 0x3: /* FSQRT */ + case 0x6: /* BFCVT */ case 0x8: /* FRINTN */ case 0x9: /* FRINTP */ case 0xa: /* FRINTM */ -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Remove handle_fp_1src_single and handle_fp_1src_double as these were the last insns decoded by those functions. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-31-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 5 ++ target/arm/tcg/translate-a64.c | 146 ++++----------------------------- 2 files changed, 22 insertions(+), 129 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ FRINTI_s 00011110 .. 1 001111 10000 ..... ..... @rr_hsd BFCVT_s 00011110 01 1 000110 10000 ..... ..... @rr_s +FRINT32Z_s 00011110 0. 1 010000 10000 ..... ..... @rr_sd +FRINT32X_s 00011110 0. 1 010001 10000 ..... ..... @rr_sd +FRINT64Z_s 00011110 0. 1 010010 10000 ..... ..... @rr_sd +FRINT64X_s 00011110 0. 1 010011 10000 ..... ..... @rr_sd + # Floating-point Immediate FMOVI_s 0001 1110 .. 1 imm:8 100 00000 rd:5 esz=%esz_hsd diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static const FPScalar1 f_scalar_bfcvt = { }; TRANS_FEAT(BFCVT_s, aa64_bf16, do_fp1_scalar, a, &f_scalar_bfcvt, -1) -/* Floating-point data-processing (1 source) - single precision */ -static void handle_fp_1src_single(DisasContext *s, int opcode, int rd, int rn) -{ - void (*gen_fpst)(TCGv_i32, TCGv_i32, TCGv_ptr); - TCGv_i32 tcg_op, tcg_res; - TCGv_ptr fpst; - int rmode = -1; +static const FPScalar1 f_scalar_frint32 = { + NULL, + gen_helper_frint32_s, + gen_helper_frint32_d, +}; +TRANS_FEAT(FRINT32Z_s, aa64_frint, do_fp1_scalar, a, + &f_scalar_frint32, FPROUNDING_ZERO) +TRANS_FEAT(FRINT32X_s, aa64_frint, do_fp1_scalar, a, &f_scalar_frint32, -1) - tcg_op = read_fp_sreg(s, rn); - tcg_res = tcg_temp_new_i32(); - - switch (opcode) { - case 0x10: /* FRINT32Z */ - rmode = FPROUNDING_ZERO; - gen_fpst = gen_helper_frint32_s; - break; - case 0x11: /* FRINT32X */ - gen_fpst = gen_helper_frint32_s; - break; - case 0x12: /* FRINT64Z */ - rmode = FPROUNDING_ZERO; - gen_fpst = gen_helper_frint64_s; - break; - case 0x13: /* FRINT64X */ - gen_fpst = gen_helper_frint64_s; - break; - default: - case 0x0: /* FMOV */ - case 0x1: /* FABS */ - case 0x2: /* FNEG */ - case 0x3: /* FSQRT */ - case 0x6: /* BFCVT */ - case 0x8: /* FRINTN */ - case 0x9: /* FRINTP */ - case 0xa: /* FRINTM */ - case 0xb: /* FRINTZ */ - case 0xc: /* FRINTA */ - case 0xe: /* FRINTX */ - case 0xf: /* FRINTI */ - g_assert_not_reached(); - } - - fpst = fpstatus_ptr(FPST_FPCR); - if (rmode >= 0) { - TCGv_i32 tcg_rmode = gen_set_rmode(rmode, fpst); - gen_fpst(tcg_res, tcg_op, fpst); - gen_restore_rmode(tcg_rmode, fpst); - } else { - gen_fpst(tcg_res, tcg_op, fpst); - } - - write_fp_sreg(s, rd, tcg_res); -} - -/* Floating-point data-processing (1 source) - double precision */ -static void handle_fp_1src_double(DisasContext *s, int opcode, int rd, int rn) -{ - void (*gen_fpst)(TCGv_i64, TCGv_i64, TCGv_ptr); - TCGv_i64 tcg_op, tcg_res; - TCGv_ptr fpst; - int rmode = -1; - - tcg_op = read_fp_dreg(s, rn); - tcg_res = tcg_temp_new_i64(); - - switch (opcode) { - case 0x10: /* FRINT32Z */ - rmode = FPROUNDING_ZERO; - gen_fpst = gen_helper_frint32_d; - break; - case 0x11: /* FRINT32X */ - gen_fpst = gen_helper_frint32_d; - break; - case 0x12: /* FRINT64Z */ - rmode = FPROUNDING_ZERO; - gen_fpst = gen_helper_frint64_d; - break; - case 0x13: /* FRINT64X */ - gen_fpst = gen_helper_frint64_d; - break; - default: - case 0x0: /* FMOV */ - case 0x1: /* FABS */ - case 0x2: /* FNEG */ - case 0x3: /* FSQRT */ - case 0x8: /* FRINTN */ - case 0x9: /* FRINTP */ - case 0xa: /* FRINTM */ - case 0xb: /* FRINTZ */ - case 0xc: /* FRINTA */ - case 0xe: /* FRINTX */ - case 0xf: /* FRINTI */ - g_assert_not_reached(); - } - - fpst = fpstatus_ptr(FPST_FPCR); - if (rmode >= 0) { - TCGv_i32 tcg_rmode = gen_set_rmode(rmode, fpst); - gen_fpst(tcg_res, tcg_op, fpst); - gen_restore_rmode(tcg_rmode, fpst); - } else { - gen_fpst(tcg_res, tcg_op, fpst); - } - - write_fp_dreg(s, rd, tcg_res); -} +static const FPScalar1 f_scalar_frint64 = { + NULL, + gen_helper_frint64_s, + gen_helper_frint64_d, +}; +TRANS_FEAT(FRINT64Z_s, aa64_frint, do_fp1_scalar, a, + &f_scalar_frint64, FPROUNDING_ZERO) +TRANS_FEAT(FRINT64X_s, aa64_frint, do_fp1_scalar, a, &f_scalar_frint64, -1) static void handle_fp_fcvt(DisasContext *s, int opcode, int rd, int rn, int dtype, int ntype) @@ -XXX,XX +XXX,XX @@ static void disas_fp_1src(DisasContext *s, uint32_t insn) break; } - case 0x10 ... 0x13: /* FRINT{32,64}{X,Z} */ - if (type > 1 || !dc_isar_feature(aa64_frint, s)) { - goto do_unallocated; - } - /* 32-to-32 and 64-to-64 ops */ - switch (type) { - case 0: - if (!fp_access_check(s)) { - return; - } - handle_fp_1src_single(s, opcode, rd, rn); - break; - case 1: - if (!fp_access_check(s)) { - return; - } - handle_fp_1src_double(s, opcode, rd, rn); - break; - case 3: - default: - goto do_unallocated; - } - break; - default: do_unallocated: case 0x0: /* FMOV */ @@ -XXX,XX +XXX,XX @@ static void disas_fp_1src(DisasContext *s, uint32_t insn) case 0xc: /* FRINTA */ case 0xe: /* FRINTX */ case 0xf: /* FRINTI */ + case 0x10 ... 0x13: /* FRINT{32,64}{X,Z} */ unallocated_encoding(s); break; } -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Remove handle_fp_fcvt and disas_fp_1src as these were the last insns decoded by those functions. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-32-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 7 ++ target/arm/tcg/translate-a64.c | 172 +++++++++++++-------------------- 2 files changed, 74 insertions(+), 105 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ FRINT32X_s 00011110 0. 1 010001 10000 ..... ..... @rr_sd FRINT64Z_s 00011110 0. 1 010010 10000 ..... ..... @rr_sd FRINT64X_s 00011110 0. 1 010011 10000 ..... ..... @rr_sd +FCVT_s_ds 00011110 00 1 000101 10000 ..... ..... @rr +FCVT_s_hs 00011110 00 1 000111 10000 ..... ..... @rr +FCVT_s_sd 00011110 01 1 000100 10000 ..... ..... @rr +FCVT_s_hd 00011110 01 1 000111 10000 ..... ..... @rr +FCVT_s_sh 00011110 11 1 000100 10000 ..... ..... @rr +FCVT_s_dh 00011110 11 1 000101 10000 ..... ..... @rr + # Floating-point Immediate FMOVI_s 0001 1110 .. 1 imm:8 100 00000 rd:5 esz=%esz_hsd diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ TRANS_FEAT(FRINT64Z_s, aa64_frint, do_fp1_scalar, a, &f_scalar_frint64, FPROUNDING_ZERO) TRANS_FEAT(FRINT64X_s, aa64_frint, do_fp1_scalar, a, &f_scalar_frint64, -1) -static void handle_fp_fcvt(DisasContext *s, int opcode, - int rd, int rn, int dtype, int ntype) +static bool trans_FCVT_s_ds(DisasContext *s, arg_rr *a) { - switch (ntype) { - case 0x0: - { - TCGv_i32 tcg_rn = read_fp_sreg(s, rn); - if (dtype == 1) { - /* Single to double */ - TCGv_i64 tcg_rd = tcg_temp_new_i64(); - gen_helper_vfp_fcvtds(tcg_rd, tcg_rn, tcg_env); - write_fp_dreg(s, rd, tcg_rd); - } else { - /* Single to half */ - TCGv_i32 tcg_rd = tcg_temp_new_i32(); - TCGv_i32 ahp = get_ahp_flag(); - TCGv_ptr fpst = fpstatus_ptr(FPST_FPCR); + if (fp_access_check(s)) { + TCGv_i32 tcg_rn = read_fp_sreg(s, a->rn); + TCGv_i64 tcg_rd = tcg_temp_new_i64(); - gen_helper_vfp_fcvt_f32_to_f16(tcg_rd, tcg_rn, fpst, ahp); - /* write_fp_sreg is OK here because top half of tcg_rd is zero */ - write_fp_sreg(s, rd, tcg_rd); - } - break; - } - case 0x1: - { - TCGv_i64 tcg_rn = read_fp_dreg(s, rn); - TCGv_i32 tcg_rd = tcg_temp_new_i32(); - if (dtype == 0) { - /* Double to single */ - gen_helper_vfp_fcvtsd(tcg_rd, tcg_rn, tcg_env); - } else { - TCGv_ptr fpst = fpstatus_ptr(FPST_FPCR); - TCGv_i32 ahp = get_ahp_flag(); - /* Double to half */ - gen_helper_vfp_fcvt_f64_to_f16(tcg_rd, tcg_rn, fpst, ahp); - /* write_fp_sreg is OK here because top half of tcg_rd is zero */ - } - write_fp_sreg(s, rd, tcg_rd); - break; - } - case 0x3: - { - TCGv_i32 tcg_rn = read_fp_sreg(s, rn); - TCGv_ptr tcg_fpst = fpstatus_ptr(FPST_FPCR); - TCGv_i32 tcg_ahp = get_ahp_flag(); - tcg_gen_ext16u_i32(tcg_rn, tcg_rn); - if (dtype == 0) { - /* Half to single */ - TCGv_i32 tcg_rd = tcg_temp_new_i32(); - gen_helper_vfp_fcvt_f16_to_f32(tcg_rd, tcg_rn, tcg_fpst, tcg_ahp); - write_fp_sreg(s, rd, tcg_rd); - } else { - /* Half to double */ - TCGv_i64 tcg_rd = tcg_temp_new_i64(); - gen_helper_vfp_fcvt_f16_to_f64(tcg_rd, tcg_rn, tcg_fpst, tcg_ahp); - write_fp_dreg(s, rd, tcg_rd); - } - break; - } - default: - g_assert_not_reached(); + gen_helper_vfp_fcvtds(tcg_rd, tcg_rn, tcg_env); + write_fp_dreg(s, a->rd, tcg_rd); } + return true; } -/* Floating point data-processing (1 source) - * 31 30 29 28 24 23 22 21 20 15 14 10 9 5 4 0 - * +---+---+---+-----------+------+---+--------+-----------+------+------+ - * | M | 0 | S | 1 1 1 1 0 | type | 1 | opcode | 1 0 0 0 0 | Rn | Rd | - * +---+---+---+-----------+------+---+--------+-----------+------+------+ - */ -static void disas_fp_1src(DisasContext *s, uint32_t insn) +static bool trans_FCVT_s_hs(DisasContext *s, arg_rr *a) { - int mos = extract32(insn, 29, 3); - int type = extract32(insn, 22, 2); - int opcode = extract32(insn, 15, 6); - int rn = extract32(insn, 5, 5); - int rd = extract32(insn, 0, 5); + if (fp_access_check(s)) { + TCGv_i32 tmp = read_fp_sreg(s, a->rn); + TCGv_i32 ahp = get_ahp_flag(); + TCGv_ptr fpst = fpstatus_ptr(FPST_FPCR); - if (mos) { - goto do_unallocated; + gen_helper_vfp_fcvt_f32_to_f16(tmp, tmp, fpst, ahp); + /* write_fp_sreg is OK here because top half of result is zero */ + write_fp_sreg(s, a->rd, tmp); } + return true; +} - switch (opcode) { - case 0x4: case 0x5: case 0x7: - { - /* FCVT between half, single and double precision */ - int dtype = extract32(opcode, 0, 2); - if (type == 2 || dtype == type) { - goto do_unallocated; - } - if (!fp_access_check(s)) { - return; - } +static bool trans_FCVT_s_sd(DisasContext *s, arg_rr *a) +{ + if (fp_access_check(s)) { + TCGv_i64 tcg_rn = read_fp_dreg(s, a->rn); + TCGv_i32 tcg_rd = tcg_temp_new_i32(); - handle_fp_fcvt(s, opcode, rd, rn, dtype, type); - break; + gen_helper_vfp_fcvtsd(tcg_rd, tcg_rn, tcg_env); + write_fp_sreg(s, a->rd, tcg_rd); } + return true; +} - default: - do_unallocated: - case 0x0: /* FMOV */ - case 0x1: /* FABS */ - case 0x2: /* FNEG */ - case 0x3: /* FSQRT */ - case 0x6: /* BFCVT */ - case 0x8: /* FRINTN */ - case 0x9: /* FRINTP */ - case 0xa: /* FRINTM */ - case 0xb: /* FRINTZ */ - case 0xc: /* FRINTA */ - case 0xe: /* FRINTX */ - case 0xf: /* FRINTI */ - case 0x10 ... 0x13: /* FRINT{32,64}{X,Z} */ - unallocated_encoding(s); - break; +static bool trans_FCVT_s_hd(DisasContext *s, arg_rr *a) +{ + if (fp_access_check(s)) { + TCGv_i64 tcg_rn = read_fp_dreg(s, a->rn); + TCGv_i32 tcg_rd = tcg_temp_new_i32(); + TCGv_i32 ahp = get_ahp_flag(); + TCGv_ptr fpst = fpstatus_ptr(FPST_FPCR); + + gen_helper_vfp_fcvt_f64_to_f16(tcg_rd, tcg_rn, fpst, ahp); + /* write_fp_sreg is OK here because top half of tcg_rd is zero */ + write_fp_sreg(s, a->rd, tcg_rd); } + return true; +} + +static bool trans_FCVT_s_sh(DisasContext *s, arg_rr *a) +{ + if (fp_access_check(s)) { + TCGv_i32 tcg_rn = read_fp_hreg(s, a->rn); + TCGv_i32 tcg_rd = tcg_temp_new_i32(); + TCGv_ptr tcg_fpst = fpstatus_ptr(FPST_FPCR); + TCGv_i32 tcg_ahp = get_ahp_flag(); + + gen_helper_vfp_fcvt_f16_to_f32(tcg_rd, tcg_rn, tcg_fpst, tcg_ahp); + write_fp_sreg(s, a->rd, tcg_rd); + } + return true; +} + +static bool trans_FCVT_s_dh(DisasContext *s, arg_rr *a) +{ + if (fp_access_check(s)) { + TCGv_i32 tcg_rn = read_fp_hreg(s, a->rn); + TCGv_i64 tcg_rd = tcg_temp_new_i64(); + TCGv_ptr tcg_fpst = fpstatus_ptr(FPST_FPCR); + TCGv_i32 tcg_ahp = get_ahp_flag(); + + gen_helper_vfp_fcvt_f16_to_f64(tcg_rd, tcg_rn, tcg_fpst, tcg_ahp); + write_fp_dreg(s, a->rd, tcg_rd); + } + return true; } /* Handle floating point <=> fixed point conversions. Note that we can @@ -XXX,XX +XXX,XX @@ static void disas_data_proc_fp(DisasContext *s, uint32_t insn) break; case 2: /* [15:12] == x100 */ /* Floating point data-processing (1 source) */ - disas_fp_1src(s, insn); + unallocated_encoding(s); /* in decodetree */ break; case 3: /* [15:12] == 1000 */ unallocated_encoding(s); -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> This includes SCVTF, UCVTF, FCVT{N,P,M,Z,A}{S,U}. Remove disas_fp_fixed_conv as those were the last insns decoded by that function. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-33-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 40 ++++ target/arm/tcg/translate-a64.c | 391 ++++++++++++++------------------- 2 files changed, 209 insertions(+), 222 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ FMAXV_s 0110 1110 00 11000 01111 10 ..... ..... @rr_q1e2 FMINV_h 0.00 1110 10 11000 01111 10 ..... ..... @qrr_h FMINV_s 0110 1110 10 11000 01111 10 ..... ..... @rr_q1e2 +# Conversion between floating-point and fixed-point (general register) + +&fcvt rd rn esz sf shift +%fcvt_shift32 10:5 !function=rsub_32 +%fcvt_shift64 10:6 !function=rsub_64 + +@fcvt32 0 ....... .. ...... 1..... rn:5 rd:5 \ + &fcvt sf=0 esz=%esz_hsd shift=%fcvt_shift32 +@fcvt64 1 ....... .. ...... ...... rn:5 rd:5 \ + &fcvt sf=1 esz=%esz_hsd shift=%fcvt_shift64 + +SCVTF_g . 0011110 .. 000010 ...... ..... ..... @fcvt32 +SCVTF_g . 0011110 .. 000010 ...... ..... ..... @fcvt64 +UCVTF_g . 0011110 .. 000011 ...... ..... ..... @fcvt32 +UCVTF_g . 0011110 .. 000011 ...... ..... ..... @fcvt64 + +FCVTZS_g . 0011110 .. 011000 ...... ..... ..... @fcvt32 +FCVTZS_g . 0011110 .. 011000 ...... ..... ..... @fcvt64 +FCVTZU_g . 0011110 .. 011001 ...... ..... ..... @fcvt32 +FCVTZU_g . 0011110 .. 011001 ...... ..... ..... @fcvt64 + +# Conversion between floating-point and integer (general register) + +@icvt sf:1 ....... .. ...... ...... rn:5 rd:5 \ + &fcvt esz=%esz_hsd shift=0 + +SCVTF_g . 0011110 .. 100010 000000 ..... ..... @icvt +UCVTF_g . 0011110 .. 100011 000000 ..... ..... @icvt + +FCVTNS_g . 0011110 .. 100000 000000 ..... ..... @icvt +FCVTNU_g . 0011110 .. 100001 000000 ..... ..... @icvt +FCVTPS_g . 0011110 .. 101000 000000 ..... ..... @icvt +FCVTPU_g . 0011110 .. 101001 000000 ..... ..... @icvt +FCVTMS_g . 0011110 .. 110000 000000 ..... ..... @icvt +FCVTMU_g . 0011110 .. 110001 000000 ..... ..... @icvt +FCVTZS_g . 0011110 .. 111000 000000 ..... ..... @icvt +FCVTZU_g . 0011110 .. 111001 000000 ..... ..... @icvt +FCVTAS_g . 0011110 .. 100100 000000 ..... ..... @icvt +FCVTAU_g . 0011110 .. 100101 000000 ..... ..... @icvt + # Floating-point data processing (1 source) FMOV_s 00011110 .. 1 000000 10000 ..... ..... @rr_hsd diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static bool trans_FCVT_s_dh(DisasContext *s, arg_rr *a) return true; } -/* Handle floating point <=> fixed point conversions. Note that we can - * also deal with fp <=> integer conversions as a special case (scale == 64) - * OPTME: consider handling that special case specially or at least skipping - * the call to scalbn in the helpers for zero shifts. - */ -static void handle_fpfpcvt(DisasContext *s, int rd, int rn, int opcode, - bool itof, int rmode, int scale, int sf, int type) +static bool do_cvtf_scalar(DisasContext *s, MemOp esz, int rd, int shift, + TCGv_i64 tcg_int, bool is_signed) { - bool is_signed = !(opcode & 1); TCGv_ptr tcg_fpstatus; TCGv_i32 tcg_shift, tcg_single; TCGv_i64 tcg_double; - tcg_fpstatus = fpstatus_ptr(type == 3 ? FPST_FPCR_F16 : FPST_FPCR); + tcg_fpstatus = fpstatus_ptr(esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR); + tcg_shift = tcg_constant_i32(shift); - tcg_shift = tcg_constant_i32(64 - scale); - - if (itof) { - TCGv_i64 tcg_int = cpu_reg(s, rn); - if (!sf) { - TCGv_i64 tcg_extend = tcg_temp_new_i64(); - - if (is_signed) { - tcg_gen_ext32s_i64(tcg_extend, tcg_int); - } else { - tcg_gen_ext32u_i64(tcg_extend, tcg_int); - } - - tcg_int = tcg_extend; + switch (esz) { + case MO_64: + tcg_double = tcg_temp_new_i64(); + if (is_signed) { + gen_helper_vfp_sqtod(tcg_double, tcg_int, tcg_shift, tcg_fpstatus); + } else { + gen_helper_vfp_uqtod(tcg_double, tcg_int, tcg_shift, tcg_fpstatus); } + write_fp_dreg(s, rd, tcg_double); + break; - switch (type) { - case 1: /* float64 */ - tcg_double = tcg_temp_new_i64(); - if (is_signed) { - gen_helper_vfp_sqtod(tcg_double, tcg_int, - tcg_shift, tcg_fpstatus); - } else { - gen_helper_vfp_uqtod(tcg_double, tcg_int, - tcg_shift, tcg_fpstatus); - } - write_fp_dreg(s, rd, tcg_double); - break; - - case 0: /* float32 */ - tcg_single = tcg_temp_new_i32(); - if (is_signed) { - gen_helper_vfp_sqtos(tcg_single, tcg_int, - tcg_shift, tcg_fpstatus); - } else { - gen_helper_vfp_uqtos(tcg_single, tcg_int, - tcg_shift, tcg_fpstatus); - } - write_fp_sreg(s, rd, tcg_single); - break; - - case 3: /* float16 */ - tcg_single = tcg_temp_new_i32(); - if (is_signed) { - gen_helper_vfp_sqtoh(tcg_single, tcg_int, - tcg_shift, tcg_fpstatus); - } else { - gen_helper_vfp_uqtoh(tcg_single, tcg_int, - tcg_shift, tcg_fpstatus); - } - write_fp_sreg(s, rd, tcg_single); - break; - - default: - g_assert_not_reached(); + case MO_32: + tcg_single = tcg_temp_new_i32(); + if (is_signed) { + gen_helper_vfp_sqtos(tcg_single, tcg_int, tcg_shift, tcg_fpstatus); + } else { + gen_helper_vfp_uqtos(tcg_single, tcg_int, tcg_shift, tcg_fpstatus); } - } else { - TCGv_i64 tcg_int = cpu_reg(s, rd); - TCGv_i32 tcg_rmode; + write_fp_sreg(s, rd, tcg_single); + break; - if (extract32(opcode, 2, 1)) { - /* There are too many rounding modes to all fit into rmode, - * so FCVTA[US] is a special case. - */ - rmode = FPROUNDING_TIEAWAY; + case MO_16: + tcg_single = tcg_temp_new_i32(); + if (is_signed) { + gen_helper_vfp_sqtoh(tcg_single, tcg_int, tcg_shift, tcg_fpstatus); + } else { + gen_helper_vfp_uqtoh(tcg_single, tcg_int, tcg_shift, tcg_fpstatus); } + write_fp_sreg(s, rd, tcg_single); + break; - tcg_rmode = gen_set_rmode(rmode, tcg_fpstatus); - - switch (type) { - case 1: /* float64 */ - tcg_double = read_fp_dreg(s, rn); - if (is_signed) { - if (!sf) { - gen_helper_vfp_tosld(tcg_int, tcg_double, - tcg_shift, tcg_fpstatus); - } else { - gen_helper_vfp_tosqd(tcg_int, tcg_double, - tcg_shift, tcg_fpstatus); - } - } else { - if (!sf) { - gen_helper_vfp_tould(tcg_int, tcg_double, - tcg_shift, tcg_fpstatus); - } else { - gen_helper_vfp_touqd(tcg_int, tcg_double, - tcg_shift, tcg_fpstatus); - } - } - if (!sf) { - tcg_gen_ext32u_i64(tcg_int, tcg_int); - } - break; - - case 0: /* float32 */ - tcg_single = read_fp_sreg(s, rn); - if (sf) { - if (is_signed) { - gen_helper_vfp_tosqs(tcg_int, tcg_single, - tcg_shift, tcg_fpstatus); - } else { - gen_helper_vfp_touqs(tcg_int, tcg_single, - tcg_shift, tcg_fpstatus); - } - } else { - TCGv_i32 tcg_dest = tcg_temp_new_i32(); - if (is_signed) { - gen_helper_vfp_tosls(tcg_dest, tcg_single, - tcg_shift, tcg_fpstatus); - } else { - gen_helper_vfp_touls(tcg_dest, tcg_single, - tcg_shift, tcg_fpstatus); - } - tcg_gen_extu_i32_i64(tcg_int, tcg_dest); - } - break; - - case 3: /* float16 */ - tcg_single = read_fp_sreg(s, rn); - if (sf) { - if (is_signed) { - gen_helper_vfp_tosqh(tcg_int, tcg_single, - tcg_shift, tcg_fpstatus); - } else { - gen_helper_vfp_touqh(tcg_int, tcg_single, - tcg_shift, tcg_fpstatus); - } - } else { - TCGv_i32 tcg_dest = tcg_temp_new_i32(); - if (is_signed) { - gen_helper_vfp_toslh(tcg_dest, tcg_single, - tcg_shift, tcg_fpstatus); - } else { - gen_helper_vfp_toulh(tcg_dest, tcg_single, - tcg_shift, tcg_fpstatus); - } - tcg_gen_extu_i32_i64(tcg_int, tcg_dest); - } - break; - - default: - g_assert_not_reached(); - } - - gen_restore_rmode(tcg_rmode, tcg_fpstatus); + default: + g_assert_not_reached(); } + return true; } -/* Floating point <-> fixed point conversions - * 31 30 29 28 24 23 22 21 20 19 18 16 15 10 9 5 4 0 - * +----+---+---+-----------+------+---+-------+--------+-------+------+------+ - * | sf | 0 | S | 1 1 1 1 0 | type | 0 | rmode | opcode | scale | Rn | Rd | - * +----+---+---+-----------+------+---+-------+--------+-------+------+------+ - */ -static void disas_fp_fixed_conv(DisasContext *s, uint32_t insn) +static bool do_cvtf_g(DisasContext *s, arg_fcvt *a, bool is_signed) { - int rd = extract32(insn, 0, 5); - int rn = extract32(insn, 5, 5); - int scale = extract32(insn, 10, 6); - int opcode = extract32(insn, 16, 3); - int rmode = extract32(insn, 19, 2); - int type = extract32(insn, 22, 2); - bool sbit = extract32(insn, 29, 1); - bool sf = extract32(insn, 31, 1); - bool itof; + TCGv_i64 tcg_int; + int check = fp_access_check_scalar_hsd(s, a->esz); - if (sbit || (!sf && scale < 32)) { - unallocated_encoding(s); - return; + if (check <= 0) { + return check == 0; } - switch (type) { - case 0: /* float32 */ - case 1: /* float64 */ - break; - case 3: /* float16 */ - if (dc_isar_feature(aa64_fp16, s)) { - break; + if (a->sf) { + tcg_int = cpu_reg(s, a->rn); + } else { + tcg_int = read_cpu_reg(s, a->rn, true); + if (is_signed) { + tcg_gen_ext32s_i64(tcg_int, tcg_int); + } else { + tcg_gen_ext32u_i64(tcg_int, tcg_int); } - /* fallthru */ - default: - unallocated_encoding(s); - return; } - - switch ((rmode << 3) | opcode) { - case 0x2: /* SCVTF */ - case 0x3: /* UCVTF */ - itof = true; - break; - case 0x18: /* FCVTZS */ - case 0x19: /* FCVTZU */ - itof = false; - break; - default: - unallocated_encoding(s); - return; - } - - if (!fp_access_check(s)) { - return; - } - - handle_fpfpcvt(s, rd, rn, opcode, itof, FPROUNDING_ZERO, scale, sf, type); + return do_cvtf_scalar(s, a->esz, a->rd, a->shift, tcg_int, is_signed); } +TRANS(SCVTF_g, do_cvtf_g, a, true) +TRANS(UCVTF_g, do_cvtf_g, a, false) + +static void do_fcvt_scalar(DisasContext *s, MemOp out, MemOp esz, + TCGv_i64 tcg_out, int shift, int rn, + ARMFPRounding rmode) +{ + TCGv_ptr tcg_fpstatus; + TCGv_i32 tcg_shift, tcg_rmode, tcg_single; + + tcg_fpstatus = fpstatus_ptr(esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR); + tcg_shift = tcg_constant_i32(shift); + tcg_rmode = gen_set_rmode(rmode, tcg_fpstatus); + + switch (esz) { + case MO_64: + read_vec_element(s, tcg_out, rn, 0, MO_64); + switch (out) { + case MO_64 | MO_SIGN: + gen_helper_vfp_tosqd(tcg_out, tcg_out, tcg_shift, tcg_fpstatus); + break; + case MO_64: + gen_helper_vfp_touqd(tcg_out, tcg_out, tcg_shift, tcg_fpstatus); + break; + case MO_32 | MO_SIGN: + gen_helper_vfp_tosld(tcg_out, tcg_out, tcg_shift, tcg_fpstatus); + break; + case MO_32: + gen_helper_vfp_tould(tcg_out, tcg_out, tcg_shift, tcg_fpstatus); + break; + default: + g_assert_not_reached(); + } + break; + + case MO_32: + tcg_single = read_fp_sreg(s, rn); + switch (out) { + case MO_64 | MO_SIGN: + gen_helper_vfp_tosqs(tcg_out, tcg_single, tcg_shift, tcg_fpstatus); + break; + case MO_64: + gen_helper_vfp_touqs(tcg_out, tcg_single, tcg_shift, tcg_fpstatus); + break; + case MO_32 | MO_SIGN: + gen_helper_vfp_tosls(tcg_single, tcg_single, + tcg_shift, tcg_fpstatus); + tcg_gen_extu_i32_i64(tcg_out, tcg_single); + break; + case MO_32: + gen_helper_vfp_touls(tcg_single, tcg_single, + tcg_shift, tcg_fpstatus); + tcg_gen_extu_i32_i64(tcg_out, tcg_single); + break; + default: + g_assert_not_reached(); + } + break; + + case MO_16: + tcg_single = read_fp_hreg(s, rn); + switch (out) { + case MO_64 | MO_SIGN: + gen_helper_vfp_tosqh(tcg_out, tcg_single, tcg_shift, tcg_fpstatus); + break; + case MO_64: + gen_helper_vfp_touqh(tcg_out, tcg_single, tcg_shift, tcg_fpstatus); + break; + case MO_32 | MO_SIGN: + gen_helper_vfp_toslh(tcg_single, tcg_single, + tcg_shift, tcg_fpstatus); + tcg_gen_extu_i32_i64(tcg_out, tcg_single); + break; + case MO_32: + gen_helper_vfp_toulh(tcg_single, tcg_single, + tcg_shift, tcg_fpstatus); + tcg_gen_extu_i32_i64(tcg_out, tcg_single); + break; + default: + g_assert_not_reached(); + } + break; + + default: + g_assert_not_reached(); + } + + gen_restore_rmode(tcg_rmode, tcg_fpstatus); +} + +static bool do_fcvt_g(DisasContext *s, arg_fcvt *a, + ARMFPRounding rmode, bool is_signed) +{ + TCGv_i64 tcg_int; + int check = fp_access_check_scalar_hsd(s, a->esz); + + if (check <= 0) { + return check == 0; + } + + tcg_int = cpu_reg(s, a->rd); + do_fcvt_scalar(s, (a->sf ? MO_64 : MO_32) | (is_signed ? MO_SIGN : 0), + a->esz, tcg_int, a->shift, a->rn, rmode); + + if (!a->sf) { + tcg_gen_ext32u_i64(tcg_int, tcg_int); + } + return true; +} + +TRANS(FCVTNS_g, do_fcvt_g, a, FPROUNDING_TIEEVEN, true) +TRANS(FCVTNU_g, do_fcvt_g, a, FPROUNDING_TIEEVEN, false) +TRANS(FCVTPS_g, do_fcvt_g, a, FPROUNDING_POSINF, true) +TRANS(FCVTPU_g, do_fcvt_g, a, FPROUNDING_POSINF, false) +TRANS(FCVTMS_g, do_fcvt_g, a, FPROUNDING_NEGINF, true) +TRANS(FCVTMU_g, do_fcvt_g, a, FPROUNDING_NEGINF, false) +TRANS(FCVTZS_g, do_fcvt_g, a, FPROUNDING_ZERO, true) +TRANS(FCVTZU_g, do_fcvt_g, a, FPROUNDING_ZERO, false) +TRANS(FCVTAS_g, do_fcvt_g, a, FPROUNDING_TIEAWAY, true) +TRANS(FCVTAU_g, do_fcvt_g, a, FPROUNDING_TIEAWAY, false) + static void handle_fmov(DisasContext *s, int rd, int rn, int type, bool itof) { /* FMOV: gpr to or from float, double, or top half of quad fp reg, @@ -XXX,XX +XXX,XX @@ static void disas_fp_int_conv(DisasContext *s, uint32_t insn) switch (opcode) { case 2: /* SCVTF */ case 3: /* UCVTF */ - itof = true; - /* fallthru */ case 4: /* FCVTAS */ case 5: /* FCVTAU */ - if (rmode != 0) { - goto do_unallocated; - } - /* fallthru */ case 0: /* FCVT[NPMZ]S */ case 1: /* FCVT[NPMZ]U */ - switch (type) { - case 0: /* float32 */ - case 1: /* float64 */ - break; - case 3: /* float16 */ - if (!dc_isar_feature(aa64_fp16, s)) { - goto do_unallocated; - } - break; - default: - goto do_unallocated; - } - if (!fp_access_check(s)) { - return; - } - handle_fpfpcvt(s, rd, rn, opcode, itof, rmode, 64, sf, type); - break; + goto do_unallocated; default: switch (sf << 7 | type << 5 | rmode << 3 | opcode) { @@ -XXX,XX +XXX,XX @@ static void disas_data_proc_fp(DisasContext *s, uint32_t insn) unallocated_encoding(s); /* in decodetree */ } else if (extract32(insn, 21, 1) == 0) { /* Floating point to fixed point conversions */ - disas_fp_fixed_conv(s, insn); + unallocated_encoding(s); /* in decodetree */ } else { switch (extract32(insn, 10, 2)) { case 1: /* Floating point conditional compare */ -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-34-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 2 ++ target/arm/tcg/translate-a64.c | 41 +++++++++++++++++----------------- 2 files changed, 22 insertions(+), 21 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ FCVTZU_g . 0011110 .. 111001 000000 ..... ..... @icvt FCVTAS_g . 0011110 .. 100100 000000 ..... ..... @icvt FCVTAU_g . 0011110 .. 100101 000000 ..... ..... @icvt +FJCVTZS 0 0011110 01 111110 000000 ..... ..... @rr + # Floating-point data processing (1 source) FMOV_s 00011110 .. 1 000000 10000 ..... ..... @rr_hsd diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ TRANS(FCVTZU_g, do_fcvt_g, a, FPROUNDING_ZERO, false) TRANS(FCVTAS_g, do_fcvt_g, a, FPROUNDING_TIEAWAY, true) TRANS(FCVTAU_g, do_fcvt_g, a, FPROUNDING_TIEAWAY, false) +static bool trans_FJCVTZS(DisasContext *s, arg_FJCVTZS *a) +{ + if (!dc_isar_feature(aa64_jscvt, s)) { + return false; + } + if (fp_access_check(s)) { + TCGv_i64 t = read_fp_dreg(s, a->rn); + TCGv_ptr fpstatus = fpstatus_ptr(FPST_FPCR); + + gen_helper_fjcvtzs(t, t, fpstatus); + + tcg_gen_ext32u_i64(cpu_reg(s, a->rd), t); + tcg_gen_extrh_i64_i32(cpu_ZF, t); + tcg_gen_movi_i32(cpu_CF, 0); + tcg_gen_movi_i32(cpu_NF, 0); + tcg_gen_movi_i32(cpu_VF, 0); + } + return true; +} + static void handle_fmov(DisasContext *s, int rd, int rn, int type, bool itof) { /* FMOV: gpr to or from float, double, or top half of quad fp reg, @@ -XXX,XX +XXX,XX @@ static void handle_fmov(DisasContext *s, int rd, int rn, int type, bool itof) } } -static void handle_fjcvtzs(DisasContext *s, int rd, int rn) -{ - TCGv_i64 t = read_fp_dreg(s, rn); - TCGv_ptr fpstatus = fpstatus_ptr(FPST_FPCR); - - gen_helper_fjcvtzs(t, t, fpstatus); - - tcg_gen_ext32u_i64(cpu_reg(s, rd), t); - tcg_gen_extrh_i64_i32(cpu_ZF, t); - tcg_gen_movi_i32(cpu_CF, 0); - tcg_gen_movi_i32(cpu_NF, 0); - tcg_gen_movi_i32(cpu_VF, 0); -} - /* Floating point <-> integer conversions * 31 30 29 28 24 23 22 21 20 19 18 16 15 10 9 5 4 0 * +----+---+---+-----------+------+---+-------+-----+-------------+----+----+ @@ -XXX,XX +XXX,XX @@ static void disas_fp_int_conv(DisasContext *s, uint32_t insn) break; case 0b00111110: /* FJCVTZS */ - if (!dc_isar_feature(aa64_jscvt, s)) { - goto do_unallocated; - } else if (fp_access_check(s)) { - handle_fjcvtzs(s, rd, rn); - } - break; - default: do_unallocated: unallocated_encoding(s); -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Remove disas_fp_int_conv and disas_data_proc_fp as these were the last insns decoded by those functions. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-35-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 14 ++ target/arm/tcg/translate-a64.c | 232 ++++++++++----------------------- 2 files changed, 86 insertions(+), 160 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ FCVTAU_g . 0011110 .. 100101 000000 ..... ..... @icvt FJCVTZS 0 0011110 01 111110 000000 ..... ..... @rr +FMOV_ws 0 0011110 00 100110 000000 ..... ..... @rr +FMOV_sw 0 0011110 00 100111 000000 ..... ..... @rr + +FMOV_xd 1 0011110 01 100110 000000 ..... ..... @rr +FMOV_dx 1 0011110 01 100111 000000 ..... ..... @rr + +# Move to/from upper half of 128-bit +FMOV_xu 1 0011110 10 101110 000000 ..... ..... @rr +FMOV_ux 1 0011110 10 101111 000000 ..... ..... @rr + +# Half-precision allows both sf=0 and sf=1 with identical results +FMOV_xh - 0011110 11 100110 000000 ..... ..... @rr +FMOV_hx - 0011110 11 100111 000000 ..... ..... @rr + # Floating-point data processing (1 source) FMOV_s 00011110 .. 1 000000 10000 ..... ..... @rr_hsd diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static bool trans_FJCVTZS(DisasContext *s, arg_FJCVTZS *a) return true; } -static void handle_fmov(DisasContext *s, int rd, int rn, int type, bool itof) +static bool trans_FMOV_hx(DisasContext *s, arg_rr *a) { - /* FMOV: gpr to or from float, double, or top half of quad fp reg, - * without conversion. - */ - - if (itof) { - TCGv_i64 tcg_rn = cpu_reg(s, rn); - TCGv_i64 tmp; - - switch (type) { - case 0: - /* 32 bit */ - tmp = tcg_temp_new_i64(); - tcg_gen_ext32u_i64(tmp, tcg_rn); - write_fp_dreg(s, rd, tmp); - break; - case 1: - /* 64 bit */ - write_fp_dreg(s, rd, tcg_rn); - break; - case 2: - /* 64 bit to top half. */ - tcg_gen_st_i64(tcg_rn, tcg_env, fp_reg_hi_offset(s, rd)); - clear_vec_high(s, true, rd); - break; - case 3: - /* 16 bit */ - tmp = tcg_temp_new_i64(); - tcg_gen_ext16u_i64(tmp, tcg_rn); - write_fp_dreg(s, rd, tmp); - break; - default: - g_assert_not_reached(); - } - } else { - TCGv_i64 tcg_rd = cpu_reg(s, rd); - - switch (type) { - case 0: - /* 32 bit */ - tcg_gen_ld32u_i64(tcg_rd, tcg_env, fp_reg_offset(s, rn, MO_32)); - break; - case 1: - /* 64 bit */ - tcg_gen_ld_i64(tcg_rd, tcg_env, fp_reg_offset(s, rn, MO_64)); - break; - case 2: - /* 64 bits from top half */ - tcg_gen_ld_i64(tcg_rd, tcg_env, fp_reg_hi_offset(s, rn)); - break; - case 3: - /* 16 bit */ - tcg_gen_ld16u_i64(tcg_rd, tcg_env, fp_reg_offset(s, rn, MO_16)); - break; - default: - g_assert_not_reached(); - } + if (!dc_isar_feature(aa64_fp16, s)) { + return false; } + if (fp_access_check(s)) { + TCGv_i64 tcg_rn = cpu_reg(s, a->rn); + TCGv_i64 tmp = tcg_temp_new_i64(); + tcg_gen_ext16u_i64(tmp, tcg_rn); + write_fp_dreg(s, a->rd, tmp); + } + return true; } -/* Floating point <-> integer conversions - * 31 30 29 28 24 23 22 21 20 19 18 16 15 10 9 5 4 0 - * +----+---+---+-----------+------+---+-------+-----+-------------+----+----+ - * | sf | 0 | S | 1 1 1 1 0 | type | 1 | rmode | opc | 0 0 0 0 0 0 | Rn | Rd | - * +----+---+---+-----------+------+---+-------+-----+-------------+----+----+ - */ -static void disas_fp_int_conv(DisasContext *s, uint32_t insn) +static bool trans_FMOV_sw(DisasContext *s, arg_rr *a) { - int rd = extract32(insn, 0, 5); - int rn = extract32(insn, 5, 5); - int opcode = extract32(insn, 16, 3); - int rmode = extract32(insn, 19, 2); - int type = extract32(insn, 22, 2); - bool sbit = extract32(insn, 29, 1); - bool sf = extract32(insn, 31, 1); - bool itof = false; - - if (sbit) { - goto do_unallocated; - } - - switch (opcode) { - case 2: /* SCVTF */ - case 3: /* UCVTF */ - case 4: /* FCVTAS */ - case 5: /* FCVTAU */ - case 0: /* FCVT[NPMZ]S */ - case 1: /* FCVT[NPMZ]U */ - goto do_unallocated; - - default: - switch (sf << 7 | type << 5 | rmode << 3 | opcode) { - case 0b01100110: /* FMOV half <-> 32-bit int */ - case 0b01100111: - case 0b11100110: /* FMOV half <-> 64-bit int */ - case 0b11100111: - if (!dc_isar_feature(aa64_fp16, s)) { - goto do_unallocated; - } - /* fallthru */ - case 0b00000110: /* FMOV 32-bit */ - case 0b00000111: - case 0b10100110: /* FMOV 64-bit */ - case 0b10100111: - case 0b11001110: /* FMOV top half of 128-bit */ - case 0b11001111: - if (!fp_access_check(s)) { - return; - } - itof = opcode & 1; - handle_fmov(s, rd, rn, type, itof); - break; - - case 0b00111110: /* FJCVTZS */ - default: - do_unallocated: - unallocated_encoding(s); - return; - } - break; + if (fp_access_check(s)) { + TCGv_i64 tcg_rn = cpu_reg(s, a->rn); + TCGv_i64 tmp = tcg_temp_new_i64(); + tcg_gen_ext32u_i64(tmp, tcg_rn); + write_fp_dreg(s, a->rd, tmp); } + return true; } -/* FP-specific subcases of table C3-6 (SIMD and FP data processing) - * 31 30 29 28 25 24 0 - * +---+---+---+---------+-----------------------------+ - * | | 0 | | 1 1 1 1 | | - * +---+---+---+---------+-----------------------------+ - */ -static void disas_data_proc_fp(DisasContext *s, uint32_t insn) +static bool trans_FMOV_dx(DisasContext *s, arg_rr *a) { - if (extract32(insn, 24, 1)) { - unallocated_encoding(s); /* in decodetree */ - } else if (extract32(insn, 21, 1) == 0) { - /* Floating point to fixed point conversions */ - unallocated_encoding(s); /* in decodetree */ - } else { - switch (extract32(insn, 10, 2)) { - case 1: /* Floating point conditional compare */ - case 2: /* Floating point data-processing (2 source) */ - case 3: /* Floating point conditional select */ - unallocated_encoding(s); /* in decodetree */ - break; - case 0: - switch (ctz32(extract32(insn, 12, 4))) { - case 0: /* [15:12] == xxx1 */ - /* Floating point immediate */ - unallocated_encoding(s); /* in decodetree */ - break; - case 1: /* [15:12] == xx10 */ - /* Floating point compare */ - unallocated_encoding(s); /* in decodetree */ - break; - case 2: /* [15:12] == x100 */ - /* Floating point data-processing (1 source) */ - unallocated_encoding(s); /* in decodetree */ - break; - case 3: /* [15:12] == 1000 */ - unallocated_encoding(s); - break; - default: /* [15:12] == 0000 */ - /* Floating point <-> integer conversions */ - disas_fp_int_conv(s, insn); - break; - } - break; - } + if (fp_access_check(s)) { + TCGv_i64 tcg_rn = cpu_reg(s, a->rn); + write_fp_dreg(s, a->rd, tcg_rn); } + return true; +} + +static bool trans_FMOV_ux(DisasContext *s, arg_rr *a) +{ + if (fp_access_check(s)) { + TCGv_i64 tcg_rn = cpu_reg(s, a->rn); + tcg_gen_st_i64(tcg_rn, tcg_env, fp_reg_hi_offset(s, a->rd)); + clear_vec_high(s, true, a->rd); + } + return true; +} + +static bool trans_FMOV_xh(DisasContext *s, arg_rr *a) +{ + if (!dc_isar_feature(aa64_fp16, s)) { + return false; + } + if (fp_access_check(s)) { + TCGv_i64 tcg_rd = cpu_reg(s, a->rd); + tcg_gen_ld16u_i64(tcg_rd, tcg_env, fp_reg_offset(s, a->rn, MO_16)); + } + return true; +} + +static bool trans_FMOV_ws(DisasContext *s, arg_rr *a) +{ + if (fp_access_check(s)) { + TCGv_i64 tcg_rd = cpu_reg(s, a->rd); + tcg_gen_ld32u_i64(tcg_rd, tcg_env, fp_reg_offset(s, a->rn, MO_32)); + } + return true; +} + +static bool trans_FMOV_xd(DisasContext *s, arg_rr *a) +{ + if (fp_access_check(s)) { + TCGv_i64 tcg_rd = cpu_reg(s, a->rd); + tcg_gen_ld_i64(tcg_rd, tcg_env, fp_reg_offset(s, a->rn, MO_64)); + } + return true; +} + +static bool trans_FMOV_xu(DisasContext *s, arg_rr *a) +{ + if (fp_access_check(s)) { + TCGv_i64 tcg_rd = cpu_reg(s, a->rd); + tcg_gen_ld_i64(tcg_rd, tcg_env, fp_reg_hi_offset(s, a->rn)); + } + return true; } /* Common vector code for handling integer to FP conversion */ @@ -XXX,XX +XXX,XX @@ static void disas_data_proc_simd(DisasContext *s, uint32_t insn) static void disas_data_proc_simd_fp(DisasContext *s, uint32_t insn) { if (extract32(insn, 28, 1) == 1 && extract32(insn, 30, 1) == 0) { - disas_data_proc_fp(s, insn); + unallocated_encoding(s); /* in decodetree */ } else { /* SIMD, including crypto */ disas_data_proc_simd(s, insn); -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-36-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 11 +++ target/arm/tcg/translate-a64.c | 123 +++++++++++++++++++++------------ 2 files changed, 89 insertions(+), 45 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ @rr_h ........ ... ..... ...... rn:5 rd:5 &rr_e esz=1 @rr_s ........ ... ..... ...... rn:5 rd:5 &rr_e esz=2 @rr_d ........ ... ..... ...... rn:5 rd:5 &rr_e esz=3 +@rr_e ........ esz:2 . ..... ...... rn:5 rd:5 &rr_e @rr_sd ........ ... ..... ...... rn:5 rd:5 &rr_e esz=%esz_sd @rr_hsd ........ ... ..... ...... rn:5 rd:5 &rr_e esz=%esz_hsd @@ -XXX,XX +XXX,XX @@ UQRSHRN_si 0111 11110 .... ... 10011 1 ..... ..... @shri_s SQRSHRUN_si 0111 11110 .... ... 10001 1 ..... ..... @shri_b SQRSHRUN_si 0111 11110 .... ... 10001 1 ..... ..... @shri_h SQRSHRUN_si 0111 11110 .... ... 10001 1 ..... ..... @shri_s + +# Advanced SIMD scalar two-register miscellaneous + +SQABS_s 0101 1110 ..1 00000 01111 0 ..... ..... @rr_e +SQNEG_s 0111 1110 ..1 00000 01111 0 ..... ..... @rr_e + +# Advanced SIMD two-register miscellaneous + +SQABS_v 0.00 1110 ..1 00000 01111 0 ..... ..... @qrr_e +SQNEG_v 0.10 1110 ..1 00000 01111 0 ..... ..... @qrr_e diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static bool trans_FMOV_xu(DisasContext *s, arg_rr *a) return true; } +typedef struct ENVScalar1 { + NeonGenOneOpEnvFn *gen_bhs[3]; + NeonGenOne64OpEnvFn *gen_d; +} ENVScalar1; + +static bool do_env_scalar1(DisasContext *s, arg_rr_e *a, const ENVScalar1 *f) +{ + if (!fp_access_check(s)) { + return true; + } + if (a->esz == MO_64) { + TCGv_i64 t = read_fp_dreg(s, a->rn); + f->gen_d(t, tcg_env, t); + write_fp_dreg(s, a->rd, t); + } else { + TCGv_i32 t = tcg_temp_new_i32(); + + read_vec_element_i32(s, t, a->rn, 0, a->esz); + f->gen_bhs[a->esz](t, tcg_env, t); + write_fp_sreg(s, a->rd, t); + } + return true; +} + +static bool do_env_vector1(DisasContext *s, arg_qrr_e *a, const ENVScalar1 *f) +{ + if (a->esz == MO_64 && !a->q) { + return false; + } + if (!fp_access_check(s)) { + return true; + } + if (a->esz == MO_64) { + TCGv_i64 t = tcg_temp_new_i64(); + + for (int i = 0; i < 2; ++i) { + read_vec_element(s, t, a->rn, i, MO_64); + f->gen_d(t, tcg_env, t); + write_vec_element(s, t, a->rd, i, MO_64); + } + } else { + TCGv_i32 t = tcg_temp_new_i32(); + int n = (a->q ? 16 : 8) >> a->esz; + + for (int i = 0; i < n; ++i) { + read_vec_element_i32(s, t, a->rn, i, a->esz); + f->gen_bhs[a->esz](t, tcg_env, t); + write_vec_element_i32(s, t, a->rd, i, a->esz); + } + } + clear_vec_high(s, a->q, a->rd); + return true; +} + +static const ENVScalar1 f_scalar_sqabs = { + { gen_helper_neon_qabs_s8, + gen_helper_neon_qabs_s16, + gen_helper_neon_qabs_s32 }, + gen_helper_neon_qabs_s64, +}; +TRANS(SQABS_s, do_env_scalar1, a, &f_scalar_sqabs) +TRANS(SQABS_v, do_env_vector1, a, &f_scalar_sqabs) + +static const ENVScalar1 f_scalar_sqneg = { + { gen_helper_neon_qneg_s8, + gen_helper_neon_qneg_s16, + gen_helper_neon_qneg_s32 }, + gen_helper_neon_qneg_s64, +}; +TRANS(SQNEG_s, do_env_scalar1, a, &f_scalar_sqneg) +TRANS(SQNEG_v, do_env_vector1, a, &f_scalar_sqneg) + /* Common vector code for handling integer to FP conversion */ static void handle_simd_intfp_conv(DisasContext *s, int rd, int rn, int elements, int is_signed, @@ -XXX,XX +XXX,XX @@ static void handle_2misc_64(DisasContext *s, int opcode, bool u, */ tcg_gen_not_i64(tcg_rd, tcg_rn); break; - case 0x7: /* SQABS, SQNEG */ - if (u) { - gen_helper_neon_qneg_s64(tcg_rd, tcg_env, tcg_rn); - } else { - gen_helper_neon_qabs_s64(tcg_rd, tcg_env, tcg_rn); - } - break; case 0xa: /* CMLT */ cond = TCG_COND_LT; do_cmop: @@ -XXX,XX +XXX,XX @@ static void handle_2misc_64(DisasContext *s, int opcode, bool u, gen_helper_frint64_d(tcg_rd, tcg_rn, tcg_fpstatus); break; default: + case 0x7: /* SQABS, SQNEG */ g_assert_not_reached(); } } @@ -XXX,XX +XXX,XX @@ static void disas_simd_scalar_two_reg_misc(DisasContext *s, uint32_t insn) TCGv_ptr tcg_fpstatus; switch (opcode) { - case 0x7: /* SQABS / SQNEG */ - break; case 0xa: /* CMLT */ if (u) { unallocated_encoding(s); @@ -XXX,XX +XXX,XX @@ static void disas_simd_scalar_two_reg_misc(DisasContext *s, uint32_t insn) break; default: case 0x3: /* USQADD / SUQADD */ + case 0x7: /* SQABS / SQNEG */ unallocated_encoding(s); return; } @@ -XXX,XX +XXX,XX @@ static void disas_simd_scalar_two_reg_misc(DisasContext *s, uint32_t insn) read_vec_element_i32(s, tcg_rn, rn, 0, size); switch (opcode) { - case 0x7: /* SQABS, SQNEG */ - { - NeonGenOneOpEnvFn *genfn; - static NeonGenOneOpEnvFn * const fns[3][2] = { - { gen_helper_neon_qabs_s8, gen_helper_neon_qneg_s8 }, - { gen_helper_neon_qabs_s16, gen_helper_neon_qneg_s16 }, - { gen_helper_neon_qabs_s32, gen_helper_neon_qneg_s32 }, - }; - genfn = fns[size][u]; - genfn(tcg_rd, tcg_env, tcg_rn); - break; - } case 0x1a: /* FCVTNS */ case 0x1b: /* FCVTMS */ case 0x1c: /* FCVTAS */ @@ -XXX,XX +XXX,XX @@ static void disas_simd_scalar_two_reg_misc(DisasContext *s, uint32_t insn) tcg_fpstatus); break; default: + case 0x7: /* SQABS, SQNEG */ g_assert_not_reached(); } @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) return; } break; - case 0x7: /* SQABS, SQNEG */ - if (size == 3 && !is_q) { - unallocated_encoding(s); - return; - } - break; case 0xc ... 0xf: case 0x16 ... 0x1f: { @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) } default: case 0x3: /* SUQADD, USQADD */ + case 0x7: /* SQABS, SQNEG */ unallocated_encoding(s); return; } @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) tcg_gen_clrsb_i32(tcg_res, tcg_op); } break; - case 0x7: /* SQABS, SQNEG */ - if (u) { - gen_helper_neon_qneg_s32(tcg_res, tcg_env, tcg_op); - } else { - gen_helper_neon_qabs_s32(tcg_res, tcg_env, tcg_op); - } - break; case 0x2f: /* FABS */ gen_vfp_abss(tcg_res, tcg_op); break; @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) gen_helper_frint64_s(tcg_res, tcg_op, tcg_fpstatus); break; default: + case 0x7: /* SQABS, SQNEG */ g_assert_not_reached(); } } else { @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) gen_helper_neon_cnt_u8(tcg_res, tcg_op); } break; - case 0x7: /* SQABS, SQNEG */ - { - NeonGenOneOpEnvFn *genfn; - static NeonGenOneOpEnvFn * const fns[2][2] = { - { gen_helper_neon_qabs_s8, gen_helper_neon_qneg_s8 }, - { gen_helper_neon_qabs_s16, gen_helper_neon_qneg_s16 }, - }; - genfn = fns[size][u]; - genfn(tcg_res, tcg_env, tcg_op); - break; - } case 0x4: /* CLS, CLZ */ if (u) { if (size == 0) { @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) } break; default: + case 0x7: /* SQABS, SQNEG */ g_assert_not_reached(); } } -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-37-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 4 +++ target/arm/tcg/translate-a64.c | 46 +++++++++++++++++++++++----------- 2 files changed, 35 insertions(+), 15 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ SQRSHRUN_si 0111 11110 .... ... 10001 1 ..... ..... @shri_s SQABS_s 0101 1110 ..1 00000 01111 0 ..... ..... @rr_e SQNEG_s 0111 1110 ..1 00000 01111 0 ..... ..... @rr_e +ABS_s 0101 1110 111 00000 10111 0 ..... ..... @rr +NEG_s 0111 1110 111 00000 10111 0 ..... ..... @rr # Advanced SIMD two-register miscellaneous SQABS_v 0.00 1110 ..1 00000 01111 0 ..... ..... @qrr_e SQNEG_v 0.10 1110 ..1 00000 01111 0 ..... ..... @qrr_e +ABS_v 0.00 1110 ..1 00000 10111 0 ..... ..... @qrr_e +NEG_v 0.10 1110 ..1 00000 10111 0 ..... ..... @qrr_e diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static const ENVScalar1 f_scalar_sqneg = { TRANS(SQNEG_s, do_env_scalar1, a, &f_scalar_sqneg) TRANS(SQNEG_v, do_env_vector1, a, &f_scalar_sqneg) +static bool do_scalar1_d(DisasContext *s, arg_rr *a, ArithOneOp *f) +{ + if (fp_access_check(s)) { + TCGv_i64 t = read_fp_dreg(s, a->rn); + f(t, t); + write_fp_dreg(s, a->rd, t); + } + return true; +} + +TRANS(ABS_s, do_scalar1_d, a, tcg_gen_abs_i64) +TRANS(NEG_s, do_scalar1_d, a, tcg_gen_neg_i64) + +static bool do_gvec_fn2(DisasContext *s, arg_qrr_e *a, GVecGen2Fn *fn) +{ + if (!a->q && a->esz == MO_64) { + return false; + } + if (fp_access_check(s)) { + gen_gvec_fn2(s, a->q, a->rd, a->rn, fn, a->esz); + } + return true; +} + +TRANS(ABS_v, do_gvec_fn2, a, tcg_gen_gvec_abs) +TRANS(NEG_v, do_gvec_fn2, a, tcg_gen_gvec_neg) + /* Common vector code for handling integer to FP conversion */ static void handle_simd_intfp_conv(DisasContext *s, int rd, int rn, int elements, int is_signed, @@ -XXX,XX +XXX,XX @@ static void handle_2misc_64(DisasContext *s, int opcode, bool u, case 0x9: /* CMEQ, CMLE */ cond = u ? TCG_COND_LE : TCG_COND_EQ; goto do_cmop; - case 0xb: /* ABS, NEG */ - if (u) { - tcg_gen_neg_i64(tcg_rd, tcg_rn); - } else { - tcg_gen_abs_i64(tcg_rd, tcg_rn); - } - break; case 0x2f: /* FABS */ gen_vfp_absd(tcg_rd, tcg_rn); break; @@ -XXX,XX +XXX,XX @@ static void handle_2misc_64(DisasContext *s, int opcode, bool u, break; default: case 0x7: /* SQABS, SQNEG */ + case 0xb: /* ABS, NEG */ g_assert_not_reached(); } } @@ -XXX,XX +XXX,XX @@ static void disas_simd_scalar_two_reg_misc(DisasContext *s, uint32_t insn) /* fall through */ case 0x8: /* CMGT, CMGE */ case 0x9: /* CMEQ, CMLE */ - case 0xb: /* ABS, NEG */ if (size != 3) { unallocated_encoding(s); return; @@ -XXX,XX +XXX,XX @@ static void disas_simd_scalar_two_reg_misc(DisasContext *s, uint32_t insn) default: case 0x3: /* USQADD / SUQADD */ case 0x7: /* SQABS / SQNEG */ + case 0xb: /* ABS, NEG */ unallocated_encoding(s); return; } @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) /* fall through */ case 0x8: /* CMGT, CMGE */ case 0x9: /* CMEQ, CMLE */ - case 0xb: /* ABS, NEG */ if (size == 3 && !is_q) { unallocated_encoding(s); return; @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) default: case 0x3: /* SUQADD, USQADD */ case 0x7: /* SQABS, SQNEG */ + case 0xb: /* ABS, NEG */ unallocated_encoding(s); return; } @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) gen_gvec_fn2(s, is_q, rd, rn, gen_gvec_clt0, size); return; case 0xb: - if (u) { /* ABS, NEG */ - gen_gvec_fn2(s, is_q, rd, rn, tcg_gen_gvec_neg, size); - } else { - gen_gvec_fn2(s, is_q, rd, rn, tcg_gen_gvec_abs, size); - } - return; + g_assert_not_reached(); } if (size == 3) { -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Add gvec interfaces for CLS and CLZ operations. Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-38-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/translate.h | 5 +++++ target/arm/tcg/gengvec.c | 35 +++++++++++++++++++++++++++++++++ target/arm/tcg/translate-a64.c | 29 +++++++-------------------- target/arm/tcg/translate-neon.c | 29 ++------------------------- 4 files changed, 49 insertions(+), 49 deletions(-) diff --git a/target/arm/tcg/translate.h b/target/arm/tcg/translate.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate.h +++ b/target/arm/tcg/translate.h @@ -XXX,XX +XXX,XX @@ void gen_gvec_umaxp(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, void gen_gvec_uminp(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz); +void gen_gvec_cls(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, + uint32_t opr_sz, uint32_t max_sz); +void gen_gvec_clz(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, + uint32_t opr_sz, uint32_t max_sz); + /* * Forward to the isar_feature_* tests given a DisasContext pointer. */ diff --git a/target/arm/tcg/gengvec.c b/target/arm/tcg/gengvec.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/gengvec.c +++ b/target/arm/tcg/gengvec.c @@ -XXX,XX +XXX,XX @@ void gen_gvec_urhadd(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, assert(vece <= MO_32); tcg_gen_gvec_3(rd_ofs, rn_ofs, rm_ofs, opr_sz, max_sz, &g[vece]); } + +void gen_gvec_cls(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, + uint32_t opr_sz, uint32_t max_sz) +{ + static const GVecGen2 g[] = { + { .fni4 = gen_helper_neon_cls_s8, + .vece = MO_8 }, + { .fni4 = gen_helper_neon_cls_s16, + .vece = MO_16 }, + { .fni4 = tcg_gen_clrsb_i32, + .vece = MO_32 }, + }; + assert(vece <= MO_32); + tcg_gen_gvec_2(rd_ofs, rn_ofs, opr_sz, max_sz, &g[vece]); +} + +static void gen_clz32_i32(TCGv_i32 d, TCGv_i32 n) +{ + tcg_gen_clzi_i32(d, n, 32); +} + +void gen_gvec_clz(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, + uint32_t opr_sz, uint32_t max_sz) +{ + static const GVecGen2 g[] = { + { .fni4 = gen_helper_neon_clz_u8, + .vece = MO_8 }, + { .fni4 = gen_helper_neon_clz_u16, + .vece = MO_16 }, + { .fni4 = gen_clz32_i32, + .vece = MO_32 }, + }; + assert(vece <= MO_32); + tcg_gen_gvec_2(rd_ofs, rn_ofs, opr_sz, max_sz, &g[vece]); +} diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) } switch (opcode) { + case 0x4: /* CLZ, CLS */ + if (u) { + gen_gvec_fn2(s, is_q, rd, rn, gen_gvec_clz, size); + } else { + gen_gvec_fn2(s, is_q, rd, rn, gen_gvec_cls, size); + } + return; case 0x5: if (u && size == 0) { /* NOT */ gen_gvec_fn2(s, is_q, rd, rn, tcg_gen_gvec_not, 0); @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) if (size == 2) { /* Special cases for 32 bit elements */ switch (opcode) { - case 0x4: /* CLS */ - if (u) { - tcg_gen_clzi_i32(tcg_res, tcg_op, 32); - } else { - tcg_gen_clrsb_i32(tcg_res, tcg_op); - } - break; case 0x2f: /* FABS */ gen_vfp_abss(tcg_res, tcg_op); break; @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) gen_helper_neon_cnt_u8(tcg_res, tcg_op); } break; - case 0x4: /* CLS, CLZ */ - if (u) { - if (size == 0) { - gen_helper_neon_clz_u8(tcg_res, tcg_op); - } else { - gen_helper_neon_clz_u16(tcg_res, tcg_op); - } - } else { - if (size == 0) { - gen_helper_neon_cls_s8(tcg_res, tcg_op); - } else { - gen_helper_neon_cls_s16(tcg_res, tcg_op); - } - } - break; default: case 0x7: /* SQABS, SQNEG */ g_assert_not_reached(); diff --git a/target/arm/tcg/translate-neon.c b/target/arm/tcg/translate-neon.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-neon.c +++ b/target/arm/tcg/translate-neon.c @@ -XXX,XX +XXX,XX @@ DO_2MISC_VEC(VCGT0, gen_gvec_cgt0) DO_2MISC_VEC(VCLE0, gen_gvec_cle0) DO_2MISC_VEC(VCGE0, gen_gvec_cge0) DO_2MISC_VEC(VCLT0, gen_gvec_clt0) +DO_2MISC_VEC(VCLS, gen_gvec_cls) +DO_2MISC_VEC(VCLZ, gen_gvec_clz) static bool trans_VMVN(DisasContext *s, arg_2misc *a) { @@ -XXX,XX +XXX,XX @@ static bool trans_VREV16(DisasContext *s, arg_2misc *a) return do_2misc(s, a, gen_rev16); } -static bool trans_VCLS(DisasContext *s, arg_2misc *a) -{ - static NeonGenOneOpFn * const fn[] = { - gen_helper_neon_cls_s8, - gen_helper_neon_cls_s16, - gen_helper_neon_cls_s32, - NULL, - }; - return do_2misc(s, a, fn[a->size]); -} - -static void do_VCLZ_32(TCGv_i32 rd, TCGv_i32 rm) -{ - tcg_gen_clzi_i32(rd, rm, 32); -} - -static bool trans_VCLZ(DisasContext *s, arg_2misc *a) -{ - static NeonGenOneOpFn * const fn[] = { - gen_helper_neon_clz_u8, - gen_helper_neon_clz_u16, - do_VCLZ_32, - NULL, - }; - return do_2misc(s, a, fn[a->size]); -} - static bool trans_VCNT(DisasContext *s, arg_2misc *a) { if (a->size != 0) { -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-39-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 2 ++ target/arm/tcg/translate-a64.c | 37 ++++++++++++++++------------------ 2 files changed, 19 insertions(+), 20 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ SQABS_v 0.00 1110 ..1 00000 01111 0 ..... ..... @qrr_e SQNEG_v 0.10 1110 ..1 00000 01111 0 ..... ..... @qrr_e ABS_v 0.00 1110 ..1 00000 10111 0 ..... ..... @qrr_e NEG_v 0.10 1110 ..1 00000 10111 0 ..... ..... @qrr_e +CLS_v 0.00 1110 ..1 00000 01001 0 ..... ..... @qrr_e +CLZ_v 0.10 1110 ..1 00000 01001 0 ..... ..... @qrr_e diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static bool do_gvec_fn2(DisasContext *s, arg_qrr_e *a, GVecGen2Fn *fn) TRANS(ABS_v, do_gvec_fn2, a, tcg_gen_gvec_abs) TRANS(NEG_v, do_gvec_fn2, a, tcg_gen_gvec_neg) +static bool do_gvec_fn2_bhs(DisasContext *s, arg_qrr_e *a, GVecGen2Fn *fn) +{ + if (a->esz == MO_64) { + return false; + } + if (fp_access_check(s)) { + gen_gvec_fn2(s, a->q, a->rd, a->rn, fn, a->esz); + } + return true; +} + +TRANS(CLS_v, do_gvec_fn2_bhs, a, gen_gvec_cls) +TRANS(CLZ_v, do_gvec_fn2_bhs, a, gen_gvec_clz) + /* Common vector code for handling integer to FP conversion */ static void handle_simd_intfp_conv(DisasContext *s, int rd, int rn, int elements, int is_signed, @@ -XXX,XX +XXX,XX @@ static void handle_2misc_64(DisasContext *s, int opcode, bool u, TCGCond cond; switch (opcode) { - case 0x4: /* CLS, CLZ */ - if (u) { - tcg_gen_clzi_i64(tcg_rd, tcg_rn, 64); - } else { - tcg_gen_clrsb_i64(tcg_rd, tcg_rn); - } - break; case 0x5: /* NOT */ /* This opcode is shared with CNT and RBIT but we have earlier * enforced that size == 3 if and only if this is the NOT insn. @@ -XXX,XX +XXX,XX @@ static void handle_2misc_64(DisasContext *s, int opcode, bool u, gen_helper_frint64_d(tcg_rd, tcg_rn, tcg_fpstatus); break; default: + case 0x4: /* CLS, CLZ */ case 0x7: /* SQABS, SQNEG */ case 0xb: /* ABS, NEG */ g_assert_not_reached(); @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) handle_2misc_narrow(s, false, opcode, u, is_q, size, rn, rd); return; - case 0x4: /* CLS, CLZ */ - if (size == 3) { - unallocated_encoding(s); - return; - } - break; case 0x2: /* SADDLP, UADDLP */ case 0x6: /* SADALP, UADALP */ if (size == 3) { @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) } default: case 0x3: /* SUQADD, USQADD */ + case 0x4: /* CLS, CLZ */ case 0x7: /* SQABS, SQNEG */ case 0xb: /* ABS, NEG */ unallocated_encoding(s); @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) } switch (opcode) { - case 0x4: /* CLZ, CLS */ - if (u) { - gen_gvec_fn2(s, is_q, rd, rn, gen_gvec_clz, size); - } else { - gen_gvec_fn2(s, is_q, rd, rn, gen_gvec_cls, size); - } - return; case 0x5: if (u && size == 0) { /* NOT */ gen_gvec_fn2(s, is_q, rd, rn, tcg_gen_gvec_not, 0); @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) case 0xa: /* CMLT */ gen_gvec_fn2(s, is_q, rd, rn, gen_gvec_clt0, size); return; + case 0x4: /* CLZ, CLS */ case 0xb: g_assert_not_reached(); } -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Add gvec interfaces for CNT and RBIT operations. Use ctpop8 for CNT and revbit+bswap for RBIT. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-40-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/helper.h | 4 ++-- target/arm/tcg/translate.h | 4 ++++ target/arm/tcg/gengvec.c | 16 ++++++++++++++++ target/arm/tcg/neon_helper.c | 21 --------------------- target/arm/tcg/translate-a64.c | 32 +++++++++----------------------- target/arm/tcg/translate-neon.c | 16 ++++++++-------- target/arm/tcg/vec_helper.c | 24 ++++++++++++++++++++++++ 7 files changed, 63 insertions(+), 54 deletions(-) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_1(neon_clz_u16, i32, i32) DEF_HELPER_1(neon_cls_s8, i32, i32) DEF_HELPER_1(neon_cls_s16, i32, i32) DEF_HELPER_1(neon_cls_s32, i32, i32) -DEF_HELPER_1(neon_cnt_u8, i32, i32) -DEF_HELPER_FLAGS_1(neon_rbit_u8, TCG_CALL_NO_RWG_SE, i32, i32) +DEF_HELPER_FLAGS_3(gvec_cnt_b, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(gvec_rbit_b, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_3(neon_qdmulh_s16, i32, env, i32, i32) DEF_HELPER_3(neon_qrdmulh_s16, i32, env, i32, i32) diff --git a/target/arm/tcg/translate.h b/target/arm/tcg/translate.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate.h +++ b/target/arm/tcg/translate.h @@ -XXX,XX +XXX,XX @@ void gen_gvec_cls(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, uint32_t opr_sz, uint32_t max_sz); void gen_gvec_clz(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, uint32_t opr_sz, uint32_t max_sz); +void gen_gvec_cnt(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, + uint32_t opr_sz, uint32_t max_sz); +void gen_gvec_rbit(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, + uint32_t opr_sz, uint32_t max_sz); /* * Forward to the isar_feature_* tests given a DisasContext pointer. diff --git a/target/arm/tcg/gengvec.c b/target/arm/tcg/gengvec.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/gengvec.c +++ b/target/arm/tcg/gengvec.c @@ -XXX,XX +XXX,XX @@ void gen_gvec_clz(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, assert(vece <= MO_32); tcg_gen_gvec_2(rd_ofs, rn_ofs, opr_sz, max_sz, &g[vece]); } + +void gen_gvec_cnt(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, + uint32_t opr_sz, uint32_t max_sz) +{ + assert(vece == MO_8); + tcg_gen_gvec_2_ool(rd_ofs, rn_ofs, opr_sz, max_sz, 0, + gen_helper_gvec_cnt_b); +} + +void gen_gvec_rbit(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, + uint32_t opr_sz, uint32_t max_sz) +{ + assert(vece == MO_8); + tcg_gen_gvec_2_ool(rd_ofs, rn_ofs, opr_sz, max_sz, 0, + gen_helper_gvec_rbit_b); +} diff --git a/target/arm/tcg/neon_helper.c b/target/arm/tcg/neon_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/neon_helper.c +++ b/target/arm/tcg/neon_helper.c @@ -XXX,XX +XXX,XX @@ uint32_t HELPER(neon_cls_s32)(uint32_t x) return count - 1; } -/* Bit count. */ -uint32_t HELPER(neon_cnt_u8)(uint32_t x) -{ - x = (x & 0x55555555) + ((x >> 1) & 0x55555555); - x = (x & 0x33333333) + ((x >> 2) & 0x33333333); - x = (x & 0x0f0f0f0f) + ((x >> 4) & 0x0f0f0f0f); - return x; -} - -/* Reverse bits in each 8 bit word */ -uint32_t HELPER(neon_rbit_u8)(uint32_t x) -{ - x = ((x & 0xf0f0f0f0) >> 4) - | ((x & 0x0f0f0f0f) << 4); - x = ((x & 0x88888888) >> 3) - | ((x & 0x44444444) >> 1) - | ((x & 0x22222222) << 1) - | ((x & 0x11111111) << 3); - return x; -} - #define NEON_QDMULH16(dest, src1, src2, round) do { \ uint32_t tmp = (int32_t)(int16_t) src1 * (int16_t) src2; \ if ((tmp ^ (tmp << 1)) & SIGNBIT) { \ diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) } switch (opcode) { - case 0x5: - if (u && size == 0) { /* NOT */ + case 0x5: /* CNT, NOT, RBIT */ + if (!u) { + gen_gvec_fn2(s, is_q, rd, rn, gen_gvec_cnt, 0); + } else if (size) { + gen_gvec_fn2(s, is_q, rd, rn, gen_gvec_rbit, 0); + } else { gen_gvec_fn2(s, is_q, rd, rn, tcg_gen_gvec_not, 0); - return; } - break; + return; case 0x8: /* CMGT, CMGE */ if (u) { gen_gvec_fn2(s, is_q, rd, rn, gen_gvec_cge0, size); @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) } else { int pass; + assert(size == 2); for (pass = 0; pass < (is_q ? 4 : 2); pass++) { TCGv_i32 tcg_op = tcg_temp_new_i32(); TCGv_i32 tcg_res = tcg_temp_new_i32(); read_vec_element_i32(s, tcg_op, rn, pass, MO_32); - if (size == 2) { + { /* Special cases for 32 bit elements */ switch (opcode) { case 0x2f: /* FABS */ @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) case 0x7: /* SQABS, SQNEG */ g_assert_not_reached(); } - } else { - /* Use helpers for 8 and 16 bit elements */ - switch (opcode) { - case 0x5: /* CNT, RBIT */ - /* For these two insns size is part of the opcode specifier - * (handled earlier); they always operate on byte elements. - */ - if (u) { - gen_helper_neon_rbit_u8(tcg_res, tcg_op); - } else { - gen_helper_neon_cnt_u8(tcg_res, tcg_op); - } - break; - default: - case 0x7: /* SQABS, SQNEG */ - g_assert_not_reached(); - } } - write_vec_element_i32(s, tcg_res, rd, pass, MO_32); } } diff --git a/target/arm/tcg/translate-neon.c b/target/arm/tcg/translate-neon.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-neon.c +++ b/target/arm/tcg/translate-neon.c @@ -XXX,XX +XXX,XX @@ static bool trans_VMVN(DisasContext *s, arg_2misc *a) return do_2misc_vec(s, a, tcg_gen_gvec_not); } +static bool trans_VCNT(DisasContext *s, arg_2misc *a) +{ + if (a->size != 0) { + return false; + } + return do_2misc_vec(s, a, gen_gvec_cnt); +} + #define WRAP_2M_3_OOL_FN(WRAPNAME, FUNC, DATA) \ static void WRAPNAME(unsigned vece, uint32_t rd_ofs, \ uint32_t rm_ofs, uint32_t oprsz, \ @@ -XXX,XX +XXX,XX @@ static bool trans_VREV16(DisasContext *s, arg_2misc *a) return do_2misc(s, a, gen_rev16); } -static bool trans_VCNT(DisasContext *s, arg_2misc *a) -{ - if (a->size != 0) { - return false; - } - return do_2misc(s, a, gen_helper_neon_cnt_u8); -} - static void gen_VABS_F(unsigned vece, uint32_t rd_ofs, uint32_t rm_ofs, uint32_t oprsz, uint32_t maxsz) { diff --git a/target/arm/tcg/vec_helper.c b/target/arm/tcg/vec_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/vec_helper.c +++ b/target/arm/tcg/vec_helper.c @@ -XXX,XX +XXX,XX @@ DO_CLAMP(gvec_uclamp_b, uint8_t) DO_CLAMP(gvec_uclamp_h, uint16_t) DO_CLAMP(gvec_uclamp_s, uint32_t) DO_CLAMP(gvec_uclamp_d, uint64_t) + +/* Bit count in each 8-bit word. */ +void HELPER(gvec_cnt_b)(void *vd, void *vn, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + uint8_t *d = vd, *n = vn; + + for (i = 0; i < opr_sz; ++i) { + d[i] = ctpop8(n[i]); + } + clear_tail(d, opr_sz, simd_maxsz(desc)); +} + +/* Reverse bits in each 8 bit word */ +void HELPER(gvec_rbit_b)(void *vd, void *vn, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + uint64_t *d = vd, *n = vn; + + for (i = 0; i < opr_sz / 8; ++i) { + d[i] = revbit64(bswap64(n[i])); + } + clear_tail(d, opr_sz, simd_maxsz(desc)); +} -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-41-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 4 ++++ target/arm/tcg/translate-a64.c | 34 ++++++---------------------------- 2 files changed, 10 insertions(+), 28 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ @rrr_q1e3 ........ ... rm:5 ...... rn:5 rd:5 &qrrr_e q=1 esz=3 @rrrr_q1e3 ........ ... rm:5 . ra:5 rn:5 rd:5 &qrrrr_e q=1 esz=3 +@qrr_b . q:1 ...... .. ...... ...... rn:5 rd:5 &qrr_e esz=0 @qrr_h . q:1 ...... .. ...... ...... rn:5 rd:5 &qrr_e esz=1 @qrr_e . q:1 ...... esz:2 ...... ...... rn:5 rd:5 &qrr_e @@ -XXX,XX +XXX,XX @@ ABS_v 0.00 1110 ..1 00000 10111 0 ..... ..... @qrr_e NEG_v 0.10 1110 ..1 00000 10111 0 ..... ..... @qrr_e CLS_v 0.00 1110 ..1 00000 01001 0 ..... ..... @qrr_e CLZ_v 0.10 1110 ..1 00000 01001 0 ..... ..... @qrr_e +CNT_v 0.00 1110 001 00000 01011 0 ..... ..... @qrr_b +NOT_v 0.10 1110 001 00000 01011 0 ..... ..... @qrr_b +RBIT_v 0.10 1110 011 00000 01011 0 ..... ..... @qrr_b diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static bool do_gvec_fn2(DisasContext *s, arg_qrr_e *a, GVecGen2Fn *fn) TRANS(ABS_v, do_gvec_fn2, a, tcg_gen_gvec_abs) TRANS(NEG_v, do_gvec_fn2, a, tcg_gen_gvec_neg) +TRANS(NOT_v, do_gvec_fn2, a, tcg_gen_gvec_not) +TRANS(CNT_v, do_gvec_fn2, a, gen_gvec_cnt) +TRANS(RBIT_v, do_gvec_fn2, a, gen_gvec_rbit) static bool do_gvec_fn2_bhs(DisasContext *s, arg_qrr_e *a, GVecGen2Fn *fn) { @@ -XXX,XX +XXX,XX @@ static void handle_2misc_64(DisasContext *s, int opcode, bool u, TCGCond cond; switch (opcode) { - case 0x5: /* NOT */ - /* This opcode is shared with CNT and RBIT but we have earlier - * enforced that size == 3 if and only if this is the NOT insn. - */ - tcg_gen_not_i64(tcg_rd, tcg_rn); - break; case 0xa: /* CMLT */ cond = TCG_COND_LT; do_cmop: @@ -XXX,XX +XXX,XX @@ static void handle_2misc_64(DisasContext *s, int opcode, bool u, break; default: case 0x4: /* CLS, CLZ */ + case 0x5: /* NOT */ case 0x7: /* SQABS, SQNEG */ case 0xb: /* ABS, NEG */ g_assert_not_reached(); @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) case 0x1: /* REV16 */ handle_rev(s, opcode, u, is_q, size, rn, rd); return; - case 0x5: /* CNT, NOT, RBIT */ - if (u && size == 0) { - /* NOT */ - break; - } else if (u && size == 1) { - /* RBIT */ - break; - } else if (!u && size == 0) { - /* CNT */ - break; - } - unallocated_encoding(s); - return; case 0x12: /* XTN, XTN2, SQXTUN, SQXTUN2 */ case 0x14: /* SQXTN, SQXTN2, UQXTN, UQXTN2 */ if (size == 3) { @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) default: case 0x3: /* SUQADD, USQADD */ case 0x4: /* CLS, CLZ */ + case 0x5: /* CNT, NOT, RBIT */ case 0x7: /* SQABS, SQNEG */ case 0xb: /* ABS, NEG */ unallocated_encoding(s); @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) } switch (opcode) { - case 0x5: /* CNT, NOT, RBIT */ - if (!u) { - gen_gvec_fn2(s, is_q, rd, rn, gen_gvec_cnt, 0); - } else if (size) { - gen_gvec_fn2(s, is_q, rd, rn, gen_gvec_rbit, 0); - } else { - gen_gvec_fn2(s, is_q, rd, rn, tcg_gen_gvec_not, 0); - } - return; case 0x8: /* CMGT, CMGE */ if (u) { gen_gvec_fn2(s, is_q, rd, rn, gen_gvec_cge0, size); @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) gen_gvec_fn2(s, is_q, rd, rn, gen_gvec_clt0, size); return; case 0x4: /* CLZ, CLS */ + case 0x5: /* CNT, NOT, RBIT */ case 0xb: g_assert_not_reached(); } -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-42-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 10 ++++ target/arm/tcg/translate-a64.c | 94 +++++++++++----------------------- 2 files changed, 40 insertions(+), 64 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ SQABS_s 0101 1110 ..1 00000 01111 0 ..... ..... @rr_e SQNEG_s 0111 1110 ..1 00000 01111 0 ..... ..... @rr_e ABS_s 0101 1110 111 00000 10111 0 ..... ..... @rr NEG_s 0111 1110 111 00000 10111 0 ..... ..... @rr +CMGT0_s 0101 1110 111 00000 10001 0 ..... ..... @rr +CMGE0_s 0111 1110 111 00000 10001 0 ..... ..... @rr +CMEQ0_s 0101 1110 111 00000 10011 0 ..... ..... @rr +CMLE0_s 0111 1110 111 00000 10011 0 ..... ..... @rr +CMLT0_s 0101 1110 111 00000 10101 0 ..... ..... @rr # Advanced SIMD two-register miscellaneous @@ -XXX,XX +XXX,XX @@ CLZ_v 0.10 1110 ..1 00000 01001 0 ..... ..... @qrr_e CNT_v 0.00 1110 001 00000 01011 0 ..... ..... @qrr_b NOT_v 0.10 1110 001 00000 01011 0 ..... ..... @qrr_b RBIT_v 0.10 1110 011 00000 01011 0 ..... ..... @qrr_b +CMGT0_v 0.00 1110 ..1 00000 10001 0 ..... ..... @qrr_e +CMGE0_v 0.10 1110 ..1 00000 10001 0 ..... ..... @qrr_e +CMEQ0_v 0.00 1110 ..1 00000 10011 0 ..... ..... @qrr_e +CMLE0_v 0.10 1110 ..1 00000 10011 0 ..... ..... @qrr_e +CMLT0_v 0.00 1110 ..1 00000 10101 0 ..... ..... @qrr_e diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static bool do_scalar1_d(DisasContext *s, arg_rr *a, ArithOneOp *f) TRANS(ABS_s, do_scalar1_d, a, tcg_gen_abs_i64) TRANS(NEG_s, do_scalar1_d, a, tcg_gen_neg_i64) +static bool do_cmop0_d(DisasContext *s, arg_rr *a, TCGCond cond) +{ + if (fp_access_check(s)) { + TCGv_i64 t = read_fp_dreg(s, a->rn); + tcg_gen_negsetcond_i64(cond, t, t, tcg_constant_i64(0)); + write_fp_dreg(s, a->rd, t); + } + return true; +} + +TRANS(CMGT0_s, do_cmop0_d, a, TCG_COND_GT) +TRANS(CMGE0_s, do_cmop0_d, a, TCG_COND_GE) +TRANS(CMLE0_s, do_cmop0_d, a, TCG_COND_LE) +TRANS(CMLT0_s, do_cmop0_d, a, TCG_COND_LT) +TRANS(CMEQ0_s, do_cmop0_d, a, TCG_COND_EQ) + static bool do_gvec_fn2(DisasContext *s, arg_qrr_e *a, GVecGen2Fn *fn) { if (!a->q && a->esz == MO_64) { @@ -XXX,XX +XXX,XX @@ TRANS(NEG_v, do_gvec_fn2, a, tcg_gen_gvec_neg) TRANS(NOT_v, do_gvec_fn2, a, tcg_gen_gvec_not) TRANS(CNT_v, do_gvec_fn2, a, gen_gvec_cnt) TRANS(RBIT_v, do_gvec_fn2, a, gen_gvec_rbit) +TRANS(CMGT0_v, do_gvec_fn2, a, gen_gvec_cgt0) +TRANS(CMGE0_v, do_gvec_fn2, a, gen_gvec_cge0) +TRANS(CMLT0_v, do_gvec_fn2, a, gen_gvec_clt0) +TRANS(CMLE0_v, do_gvec_fn2, a, gen_gvec_cle0) +TRANS(CMEQ0_v, do_gvec_fn2, a, gen_gvec_ceq0) static bool do_gvec_fn2_bhs(DisasContext *s, arg_qrr_e *a, GVecGen2Fn *fn) { @@ -XXX,XX +XXX,XX @@ static void handle_2misc_64(DisasContext *s, int opcode, bool u, * The caller only need provide tcg_rmode and tcg_fpstatus if the op * requires them. */ - TCGCond cond; - switch (opcode) { - case 0xa: /* CMLT */ - cond = TCG_COND_LT; - do_cmop: - /* 64 bit integer comparison against zero, result is test ? -1 : 0. */ - tcg_gen_negsetcond_i64(cond, tcg_rd, tcg_rn, tcg_constant_i64(0)); - break; - case 0x8: /* CMGT, CMGE */ - cond = u ? TCG_COND_GE : TCG_COND_GT; - goto do_cmop; - case 0x9: /* CMEQ, CMLE */ - cond = u ? TCG_COND_LE : TCG_COND_EQ; - goto do_cmop; case 0x2f: /* FABS */ gen_vfp_absd(tcg_rd, tcg_rn); break; @@ -XXX,XX +XXX,XX @@ static void handle_2misc_64(DisasContext *s, int opcode, bool u, case 0x4: /* CLS, CLZ */ case 0x5: /* NOT */ case 0x7: /* SQABS, SQNEG */ + case 0x8: /* CMGT, CMGE */ + case 0x9: /* CMEQ, CMLE */ + case 0xa: /* CMLT */ case 0xb: /* ABS, NEG */ g_assert_not_reached(); } @@ -XXX,XX +XXX,XX @@ static void disas_simd_scalar_two_reg_misc(DisasContext *s, uint32_t insn) TCGv_ptr tcg_fpstatus; switch (opcode) { - case 0xa: /* CMLT */ - if (u) { - unallocated_encoding(s); - return; - } - /* fall through */ - case 0x8: /* CMGT, CMGE */ - case 0x9: /* CMEQ, CMLE */ - if (size != 3) { - unallocated_encoding(s); - return; - } - break; case 0x12: /* SQXTUN */ if (!u) { unallocated_encoding(s); @@ -XXX,XX +XXX,XX @@ static void disas_simd_scalar_two_reg_misc(DisasContext *s, uint32_t insn) default: case 0x3: /* USQADD / SUQADD */ case 0x7: /* SQABS / SQNEG */ + case 0x8: /* CMGT, CMGE */ + case 0x9: /* CMEQ, CMLE */ + case 0xa: /* CMLT */ case 0xb: /* ABS, NEG */ unallocated_encoding(s); return; @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) } handle_shll(s, is_q, size, rn, rd); return; - case 0xa: /* CMLT */ - if (u == 1) { - unallocated_encoding(s); - return; - } - /* fall through */ - case 0x8: /* CMGT, CMGE */ - case 0x9: /* CMEQ, CMLE */ - if (size == 3 && !is_q) { - unallocated_encoding(s); - return; - } - break; case 0xc ... 0xf: case 0x16 ... 0x1f: { @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) case 0x4: /* CLS, CLZ */ case 0x5: /* CNT, NOT, RBIT */ case 0x7: /* SQABS, SQNEG */ + case 0x8: /* CMGT, CMGE */ + case 0x9: /* CMEQ, CMLE */ + case 0xa: /* CMLT */ case 0xb: /* ABS, NEG */ unallocated_encoding(s); return; @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) tcg_rmode = NULL; } - switch (opcode) { - case 0x8: /* CMGT, CMGE */ - if (u) { - gen_gvec_fn2(s, is_q, rd, rn, gen_gvec_cge0, size); - } else { - gen_gvec_fn2(s, is_q, rd, rn, gen_gvec_cgt0, size); - } - return; - case 0x9: /* CMEQ, CMLE */ - if (u) { - gen_gvec_fn2(s, is_q, rd, rn, gen_gvec_cle0, size); - } else { - gen_gvec_fn2(s, is_q, rd, rn, gen_gvec_ceq0, size); - } - return; - case 0xa: /* CMLT */ - gen_gvec_fn2(s, is_q, rd, rn, gen_gvec_clt0, size); - return; - case 0x4: /* CLZ, CLS */ - case 0x5: /* CNT, NOT, RBIT */ - case 0xb: - g_assert_not_reached(); - } - if (size == 3) { /* All 64-bit element operations can be shared with scalar 2misc */ int pass; -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-43-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/translate.h | 6 +++ target/arm/tcg/gengvec.c | 58 ++++++++++++++++++++++ target/arm/tcg/translate-neon.c | 88 +++++++-------------------------- 3 files changed, 81 insertions(+), 71 deletions(-) diff --git a/target/arm/tcg/translate.h b/target/arm/tcg/translate.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate.h +++ b/target/arm/tcg/translate.h @@ -XXX,XX +XXX,XX @@ void gen_gvec_cnt(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, uint32_t opr_sz, uint32_t max_sz); void gen_gvec_rbit(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, uint32_t opr_sz, uint32_t max_sz); +void gen_gvec_rev16(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, + uint32_t opr_sz, uint32_t max_sz); +void gen_gvec_rev32(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, + uint32_t opr_sz, uint32_t max_sz); +void gen_gvec_rev64(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, + uint32_t opr_sz, uint32_t max_sz); /* * Forward to the isar_feature_* tests given a DisasContext pointer. diff --git a/target/arm/tcg/gengvec.c b/target/arm/tcg/gengvec.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/gengvec.c +++ b/target/arm/tcg/gengvec.c @@ -XXX,XX +XXX,XX @@ void gen_gvec_rbit(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, tcg_gen_gvec_2_ool(rd_ofs, rn_ofs, opr_sz, max_sz, 0, gen_helper_gvec_rbit_b); } + +void gen_gvec_rev16(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, + uint32_t opr_sz, uint32_t max_sz) +{ + assert(vece == MO_8); + tcg_gen_gvec_rotli(MO_16, rd_ofs, rn_ofs, 8, opr_sz, max_sz); +} + +static void gen_bswap32_i64(TCGv_i64 d, TCGv_i64 n) +{ + tcg_gen_bswap64_i64(d, n); + tcg_gen_rotli_i64(d, d, 32); +} + +void gen_gvec_rev32(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, + uint32_t opr_sz, uint32_t max_sz) +{ + static const GVecGen2 g = { + .fni8 = gen_bswap32_i64, + .fni4 = tcg_gen_bswap32_i32, + .prefer_i64 = TCG_TARGET_REG_BITS == 64, + .vece = MO_32 + }; + + switch (vece) { + case MO_16: + tcg_gen_gvec_rotli(MO_32, rd_ofs, rn_ofs, 16, opr_sz, max_sz); + break; + case MO_8: + tcg_gen_gvec_2(rd_ofs, rn_ofs, opr_sz, max_sz, &g); + break; + default: + g_assert_not_reached(); + } +} + +void gen_gvec_rev64(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, + uint32_t opr_sz, uint32_t max_sz) +{ + static const GVecGen2 g[] = { + { .fni8 = tcg_gen_bswap64_i64, + .vece = MO_64 }, + { .fni8 = tcg_gen_hswap_i64, + .vece = MO_64 }, + }; + + switch (vece) { + case MO_32: + tcg_gen_gvec_rotli(MO_64, rd_ofs, rn_ofs, 32, opr_sz, max_sz); + break; + case MO_8: + case MO_16: + tcg_gen_gvec_2(rd_ofs, rn_ofs, opr_sz, max_sz, &g[vece]); + break; + default: + g_assert_not_reached(); + } +} diff --git a/target/arm/tcg/translate-neon.c b/target/arm/tcg/translate-neon.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-neon.c +++ b/target/arm/tcg/translate-neon.c @@ -XXX,XX +XXX,XX @@ static bool trans_VDUP_scalar(DisasContext *s, arg_VDUP_scalar *a) return true; } -static bool trans_VREV64(DisasContext *s, arg_VREV64 *a) -{ - int pass, half; - TCGv_i32 tmp[2]; - - if (!arm_dc_feature(s, ARM_FEATURE_NEON)) { - return false; - } - - /* UNDEF accesses to D16-D31 if they don't exist. */ - if (!dc_isar_feature(aa32_simd_r32, s) && - ((a->vd | a->vm) & 0x10)) { - return false; - } - - if ((a->vd | a->vm) & a->q) { - return false; - } - - if (a->size == 3) { - return false; - } - - if (!vfp_access_check(s)) { - return true; - } - - tmp[0] = tcg_temp_new_i32(); - tmp[1] = tcg_temp_new_i32(); - - for (pass = 0; pass < (a->q ? 2 : 1); pass++) { - for (half = 0; half < 2; half++) { - read_neon_element32(tmp[half], a->vm, pass * 2 + half, MO_32); - switch (a->size) { - case 0: - tcg_gen_bswap32_i32(tmp[half], tmp[half]); - break; - case 1: - gen_swap_half(tmp[half], tmp[half]); - break; - case 2: - break; - default: - g_assert_not_reached(); - } - } - write_neon_element32(tmp[1], a->vd, pass * 2, MO_32); - write_neon_element32(tmp[0], a->vd, pass * 2 + 1, MO_32); - } - return true; -} - static bool do_2misc_pairwise(DisasContext *s, arg_2misc *a, NeonGenWidenFn *widenfn, NeonGenTwo64OpFn *opfn, @@ -XXX,XX +XXX,XX @@ DO_2MISC_VEC(VCGE0, gen_gvec_cge0) DO_2MISC_VEC(VCLT0, gen_gvec_clt0) DO_2MISC_VEC(VCLS, gen_gvec_cls) DO_2MISC_VEC(VCLZ, gen_gvec_clz) +DO_2MISC_VEC(VREV64, gen_gvec_rev64) static bool trans_VMVN(DisasContext *s, arg_2misc *a) { @@ -XXX,XX +XXX,XX @@ static bool trans_VCNT(DisasContext *s, arg_2misc *a) return do_2misc_vec(s, a, gen_gvec_cnt); } +static bool trans_VREV16(DisasContext *s, arg_2misc *a) +{ + if (a->size != 0) { + return false; + } + return do_2misc_vec(s, a, gen_gvec_rev16); +} + +static bool trans_VREV32(DisasContext *s, arg_2misc *a) +{ + if (a->size != 0 && a->size != 1) { + return false; + } + return do_2misc_vec(s, a, gen_gvec_rev32); +} + #define WRAP_2M_3_OOL_FN(WRAPNAME, FUNC, DATA) \ static void WRAPNAME(unsigned vece, uint32_t rd_ofs, \ uint32_t rm_ofs, uint32_t oprsz, \ @@ -XXX,XX +XXX,XX @@ static bool do_2misc(DisasContext *s, arg_2misc *a, NeonGenOneOpFn *fn) return true; } -static bool trans_VREV32(DisasContext *s, arg_2misc *a) -{ - static NeonGenOneOpFn * const fn[] = { - tcg_gen_bswap32_i32, - gen_swap_half, - NULL, - NULL, - }; - return do_2misc(s, a, fn[a->size]); -} - -static bool trans_VREV16(DisasContext *s, arg_2misc *a) -{ - if (a->size != 0) { - return false; - } - return do_2misc(s, a, gen_rev16); -} - static void gen_VABS_F(unsigned vece, uint32_t rd_ofs, uint32_t rm_ofs, uint32_t oprsz, uint32_t maxsz) { -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> This includes REV16, REV32, REV64. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-44-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 5 +++ target/arm/tcg/translate-a64.c | 79 +++------------------------------- 2 files changed, 10 insertions(+), 74 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ @qrr_b . q:1 ...... .. ...... ...... rn:5 rd:5 &qrr_e esz=0 @qrr_h . q:1 ...... .. ...... ...... rn:5 rd:5 &qrr_e esz=1 +@qrr_bh . q:1 ...... . esz:1 ...... ...... rn:5 rd:5 &qrr_e @qrr_e . q:1 ...... esz:2 ...... ...... rn:5 rd:5 &qrr_e @qrrr_b . q:1 ...... ... rm:5 ...... rn:5 rd:5 &qrrr_e esz=0 @@ -XXX,XX +XXX,XX @@ CMGE0_v 0.10 1110 ..1 00000 10001 0 ..... ..... @qrr_e CMEQ0_v 0.00 1110 ..1 00000 10011 0 ..... ..... @qrr_e CMLE0_v 0.10 1110 ..1 00000 10011 0 ..... ..... @qrr_e CMLT0_v 0.00 1110 ..1 00000 10101 0 ..... ..... @qrr_e + +REV16_v 0.00 1110 001 00000 00011 0 ..... ..... @qrr_b +REV32_v 0.10 1110 0.1 00000 00001 0 ..... ..... @qrr_bh +REV64_v 0.00 1110 ..1 00000 00001 0 ..... ..... @qrr_e diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ TRANS(CMGE0_v, do_gvec_fn2, a, gen_gvec_cge0) TRANS(CMLT0_v, do_gvec_fn2, a, gen_gvec_clt0) TRANS(CMLE0_v, do_gvec_fn2, a, gen_gvec_cle0) TRANS(CMEQ0_v, do_gvec_fn2, a, gen_gvec_ceq0) +TRANS(REV16_v, do_gvec_fn2, a, gen_gvec_rev16) +TRANS(REV32_v, do_gvec_fn2, a, gen_gvec_rev32) static bool do_gvec_fn2_bhs(DisasContext *s, arg_qrr_e *a, GVecGen2Fn *fn) { @@ -XXX,XX +XXX,XX @@ static bool do_gvec_fn2_bhs(DisasContext *s, arg_qrr_e *a, GVecGen2Fn *fn) TRANS(CLS_v, do_gvec_fn2_bhs, a, gen_gvec_cls) TRANS(CLZ_v, do_gvec_fn2_bhs, a, gen_gvec_clz) +TRANS(REV64_v, do_gvec_fn2_bhs, a, gen_gvec_rev64) /* Common vector code for handling integer to FP conversion */ static void handle_simd_intfp_conv(DisasContext *s, int rd, int rn, @@ -XXX,XX +XXX,XX @@ static void handle_2misc_widening(DisasContext *s, int opcode, bool is_q, } } -static void handle_rev(DisasContext *s, int opcode, bool u, - bool is_q, int size, int rn, int rd) -{ - int op = (opcode << 1) | u; - int opsz = op + size; - int grp_size = 3 - opsz; - int dsize = is_q ? 128 : 64; - int i; - - if (opsz >= 3) { - unallocated_encoding(s); - return; - } - - if (!fp_access_check(s)) { - return; - } - - if (size == 0) { - /* Special case bytes, use bswap op on each group of elements */ - int groups = dsize / (8 << grp_size); - - for (i = 0; i < groups; i++) { - TCGv_i64 tcg_tmp = tcg_temp_new_i64(); - - read_vec_element(s, tcg_tmp, rn, i, grp_size); - switch (grp_size) { - case MO_16: - tcg_gen_bswap16_i64(tcg_tmp, tcg_tmp, TCG_BSWAP_IZ); - break; - case MO_32: - tcg_gen_bswap32_i64(tcg_tmp, tcg_tmp, TCG_BSWAP_IZ); - break; - case MO_64: - tcg_gen_bswap64_i64(tcg_tmp, tcg_tmp); - break; - default: - g_assert_not_reached(); - } - write_vec_element(s, tcg_tmp, rd, i, grp_size); - } - clear_vec_high(s, is_q, rd); - } else { - int revmask = (1 << grp_size) - 1; - int esize = 8 << size; - int elements = dsize / esize; - TCGv_i64 tcg_rn = tcg_temp_new_i64(); - TCGv_i64 tcg_rd[2]; - - for (i = 0; i < 2; i++) { - tcg_rd[i] = tcg_temp_new_i64(); - tcg_gen_movi_i64(tcg_rd[i], 0); - } - - for (i = 0; i < elements; i++) { - int e_rev = (i & 0xf) ^ revmask; - int w = (e_rev * esize) / 64; - int o = (e_rev * esize) % 64; - - read_vec_element(s, tcg_rn, rn, i, size); - tcg_gen_deposit_i64(tcg_rd[w], tcg_rd[w], tcg_rn, o, esize); - } - - for (i = 0; i < 2; i++) { - write_vec_element(s, tcg_rd[i], rd, i, MO_64); - } - clear_vec_high(s, true, rd); - } -} - static void handle_2misc_pairwise(DisasContext *s, int opcode, bool u, bool is_q, int size, int rn, int rd) { @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) TCGv_ptr tcg_fpstatus; switch (opcode) { - case 0x0: /* REV64, REV32 */ - case 0x1: /* REV16 */ - handle_rev(s, opcode, u, is_q, size, rn, rd); - return; case 0x12: /* XTN, XTN2, SQXTUN, SQXTUN2 */ case 0x14: /* SQXTN, SQXTN2, UQXTN, UQXTN2 */ if (size == 3) { @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) break; } default: + case 0x0: /* REV64, REV32 */ + case 0x1: /* REV16 */ case 0x3: /* SUQADD, USQADD */ case 0x4: /* CLS, CLZ */ case 0x5: /* CNT, NOT, RBIT */ -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Move from helper-a64.c to neon_helper.c so that these functions are available for arm32 code as well. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-45-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/helper.h | 2 ++ target/arm/tcg/helper-a64.h | 2 -- target/arm/tcg/helper-a64.c | 43 ------------------------------------ target/arm/tcg/neon_helper.c | 43 ++++++++++++++++++++++++++++++++++++ 4 files changed, 45 insertions(+), 45 deletions(-) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_2(neon_addl_u16, i64, i64, i64) DEF_HELPER_2(neon_addl_u32, i64, i64, i64) DEF_HELPER_2(neon_paddl_u16, i64, i64, i64) DEF_HELPER_2(neon_paddl_u32, i64, i64, i64) +DEF_HELPER_FLAGS_1(neon_addlp_s8, TCG_CALL_NO_RWG_SE, i64, i64) +DEF_HELPER_FLAGS_1(neon_addlp_s16, TCG_CALL_NO_RWG_SE, i64, i64) DEF_HELPER_2(neon_subl_u16, i64, i64, i64) DEF_HELPER_2(neon_subl_u32, i64, i64, i64) DEF_HELPER_3(neon_addl_saturate_s32, i64, env, i64, i64) diff --git a/target/arm/tcg/helper-a64.h b/target/arm/tcg/helper-a64.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/helper-a64.h +++ b/target/arm/tcg/helper-a64.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_3(recpsf_f64, TCG_CALL_NO_RWG, f64, f64, f64, ptr) DEF_HELPER_FLAGS_3(rsqrtsf_f16, TCG_CALL_NO_RWG, f16, f16, f16, ptr) DEF_HELPER_FLAGS_3(rsqrtsf_f32, TCG_CALL_NO_RWG, f32, f32, f32, ptr) DEF_HELPER_FLAGS_3(rsqrtsf_f64, TCG_CALL_NO_RWG, f64, f64, f64, ptr) -DEF_HELPER_FLAGS_1(neon_addlp_s8, TCG_CALL_NO_RWG_SE, i64, i64) DEF_HELPER_FLAGS_1(neon_addlp_u8, TCG_CALL_NO_RWG_SE, i64, i64) -DEF_HELPER_FLAGS_1(neon_addlp_s16, TCG_CALL_NO_RWG_SE, i64, i64) DEF_HELPER_FLAGS_1(neon_addlp_u16, TCG_CALL_NO_RWG_SE, i64, i64) DEF_HELPER_FLAGS_2(frecpx_f64, TCG_CALL_NO_RWG, f64, f64, ptr) DEF_HELPER_FLAGS_2(frecpx_f32, TCG_CALL_NO_RWG, f32, f32, ptr) diff --git a/target/arm/tcg/helper-a64.c b/target/arm/tcg/helper-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/helper-a64.c +++ b/target/arm/tcg/helper-a64.c @@ -XXX,XX +XXX,XX @@ float64 HELPER(rsqrtsf_f64)(float64 a, float64 b, void *fpstp) return float64_muladd(a, b, float64_three, float_muladd_halve_result, fpst); } -/* Pairwise long add: add pairs of adjacent elements into - * double-width elements in the result (eg _s8 is an 8x8->16 op) - */ -uint64_t HELPER(neon_addlp_s8)(uint64_t a) -{ - uint64_t nsignmask = 0x0080008000800080ULL; - uint64_t wsignmask = 0x8000800080008000ULL; - uint64_t elementmask = 0x00ff00ff00ff00ffULL; - uint64_t tmp1, tmp2; - uint64_t res, signres; - - /* Extract odd elements, sign extend each to a 16 bit field */ - tmp1 = a & elementmask; - tmp1 ^= nsignmask; - tmp1 |= wsignmask; - tmp1 = (tmp1 - nsignmask) ^ wsignmask; - /* Ditto for the even elements */ - tmp2 = (a >> 8) & elementmask; - tmp2 ^= nsignmask; - tmp2 |= wsignmask; - tmp2 = (tmp2 - nsignmask) ^ wsignmask; - - /* calculate the result by summing bits 0..14, 16..22, etc, - * and then adjusting the sign bits 15, 23, etc manually. - * This ensures the addition can't overflow the 16 bit field. - */ - signres = (tmp1 ^ tmp2) & wsignmask; - res = (tmp1 & ~wsignmask) + (tmp2 & ~wsignmask); - res ^= signres; - - return res; -} - uint64_t HELPER(neon_addlp_u8)(uint64_t a) { uint64_t tmp; @@ -XXX,XX +XXX,XX @@ uint64_t HELPER(neon_addlp_u8)(uint64_t a) return tmp; } -uint64_t HELPER(neon_addlp_s16)(uint64_t a) -{ - int32_t reslo, reshi; - - reslo = (int32_t)(int16_t)a + (int32_t)(int16_t)(a >> 16); - reshi = (int32_t)(int16_t)(a >> 32) + (int32_t)(int16_t)(a >> 48); - - return (uint32_t)reslo | (((uint64_t)reshi) << 32); -} - uint64_t HELPER(neon_addlp_u16)(uint64_t a) { uint64_t tmp; diff --git a/target/arm/tcg/neon_helper.c b/target/arm/tcg/neon_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/neon_helper.c +++ b/target/arm/tcg/neon_helper.c @@ -XXX,XX +XXX,XX @@ uint64_t HELPER(neon_paddl_u32)(uint64_t a, uint64_t b) return low + ((uint64_t)high << 32); } +/* Pairwise long add: add pairs of adjacent elements into + * double-width elements in the result (eg _s8 is an 8x8->16 op) + */ +uint64_t HELPER(neon_addlp_s8)(uint64_t a) +{ + uint64_t nsignmask = 0x0080008000800080ULL; + uint64_t wsignmask = 0x8000800080008000ULL; + uint64_t elementmask = 0x00ff00ff00ff00ffULL; + uint64_t tmp1, tmp2; + uint64_t res, signres; + + /* Extract odd elements, sign extend each to a 16 bit field */ + tmp1 = a & elementmask; + tmp1 ^= nsignmask; + tmp1 |= wsignmask; + tmp1 = (tmp1 - nsignmask) ^ wsignmask; + /* Ditto for the even elements */ + tmp2 = (a >> 8) & elementmask; + tmp2 ^= nsignmask; + tmp2 |= wsignmask; + tmp2 = (tmp2 - nsignmask) ^ wsignmask; + + /* calculate the result by summing bits 0..14, 16..22, etc, + * and then adjusting the sign bits 15, 23, etc manually. + * This ensures the addition can't overflow the 16 bit field. + */ + signres = (tmp1 ^ tmp2) & wsignmask; + res = (tmp1 & ~wsignmask) + (tmp2 & ~wsignmask); + res ^= signres; + + return res; +} + +uint64_t HELPER(neon_addlp_s16)(uint64_t a) +{ + int32_t reslo, reshi; + + reslo = (int32_t)(int16_t)a + (int32_t)(int16_t)(a >> 16); + reshi = (int32_t)(int16_t)(a >> 32) + (int32_t)(int16_t)(a >> 48); + + return (uint32_t)reslo | (((uint64_t)reshi) << 32); +} + uint64_t HELPER(neon_subl_u16)(uint64_t a, uint64_t b) { uint64_t mask; -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Pairwise addition with and without accumulation. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-46-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/helper.h | 2 - target/arm/tcg/translate.h | 9 ++ target/arm/tcg/gengvec.c | 230 ++++++++++++++++++++++++++++++++ target/arm/tcg/neon_helper.c | 22 --- target/arm/tcg/translate-neon.c | 150 +-------------------- 5 files changed, 243 insertions(+), 170 deletions(-) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_1(neon_widen_s16, i64, i32) DEF_HELPER_2(neon_addl_u16, i64, i64, i64) DEF_HELPER_2(neon_addl_u32, i64, i64, i64) -DEF_HELPER_2(neon_paddl_u16, i64, i64, i64) -DEF_HELPER_2(neon_paddl_u32, i64, i64, i64) DEF_HELPER_FLAGS_1(neon_addlp_s8, TCG_CALL_NO_RWG_SE, i64, i64) DEF_HELPER_FLAGS_1(neon_addlp_s16, TCG_CALL_NO_RWG_SE, i64, i64) DEF_HELPER_2(neon_subl_u16, i64, i64, i64) diff --git a/target/arm/tcg/translate.h b/target/arm/tcg/translate.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate.h +++ b/target/arm/tcg/translate.h @@ -XXX,XX +XXX,XX @@ void gen_gvec_rev32(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, void gen_gvec_rev64(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, uint32_t opr_sz, uint32_t max_sz); +void gen_gvec_saddlp(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, + uint32_t opr_sz, uint32_t max_sz); +void gen_gvec_sadalp(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, + uint32_t opr_sz, uint32_t max_sz); +void gen_gvec_uaddlp(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, + uint32_t opr_sz, uint32_t max_sz); +void gen_gvec_uadalp(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, + uint32_t opr_sz, uint32_t max_sz); + /* * Forward to the isar_feature_* tests given a DisasContext pointer. */ diff --git a/target/arm/tcg/gengvec.c b/target/arm/tcg/gengvec.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/gengvec.c +++ b/target/arm/tcg/gengvec.c @@ -XXX,XX +XXX,XX @@ void gen_gvec_rev64(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, g_assert_not_reached(); } } + +static void gen_saddlp_vec(unsigned vece, TCGv_vec d, TCGv_vec n) +{ + int half = 4 << vece; + TCGv_vec t = tcg_temp_new_vec_matching(d); + + tcg_gen_shli_vec(vece, t, n, half); + tcg_gen_sari_vec(vece, d, n, half); + tcg_gen_sari_vec(vece, t, t, half); + tcg_gen_add_vec(vece, d, d, t); +} + +static void gen_saddlp_s_i64(TCGv_i64 d, TCGv_i64 n) +{ + TCGv_i64 t = tcg_temp_new_i64(); + + tcg_gen_ext32s_i64(t, n); + tcg_gen_sari_i64(d, n, 32); + tcg_gen_add_i64(d, d, t); +} + +void gen_gvec_saddlp(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, + uint32_t opr_sz, uint32_t max_sz) +{ + static const TCGOpcode vecop_list[] = { + INDEX_op_sari_vec, INDEX_op_shli_vec, INDEX_op_add_vec, 0 + }; + static const GVecGen2 g[] = { + { .fniv = gen_saddlp_vec, + .fni8 = gen_helper_neon_addlp_s8, + .opt_opc = vecop_list, + .vece = MO_16 }, + { .fniv = gen_saddlp_vec, + .fni8 = gen_helper_neon_addlp_s16, + .opt_opc = vecop_list, + .vece = MO_32 }, + { .fniv = gen_saddlp_vec, + .fni8 = gen_saddlp_s_i64, + .opt_opc = vecop_list, + .vece = MO_64 }, + }; + assert(vece <= MO_32); + tcg_gen_gvec_2(rd_ofs, rn_ofs, opr_sz, max_sz, &g[vece]); +} + +static void gen_sadalp_vec(unsigned vece, TCGv_vec d, TCGv_vec n) +{ + TCGv_vec t = tcg_temp_new_vec_matching(d); + + gen_saddlp_vec(vece, t, n); + tcg_gen_add_vec(vece, d, d, t); +} + +static void gen_sadalp_b_i64(TCGv_i64 d, TCGv_i64 n) +{ + TCGv_i64 t = tcg_temp_new_i64(); + + gen_helper_neon_addlp_s8(t, n); + tcg_gen_vec_add16_i64(d, d, t); +} + +static void gen_sadalp_h_i64(TCGv_i64 d, TCGv_i64 n) +{ + TCGv_i64 t = tcg_temp_new_i64(); + + gen_helper_neon_addlp_s16(t, n); + tcg_gen_vec_add32_i64(d, d, t); +} + +static void gen_sadalp_s_i64(TCGv_i64 d, TCGv_i64 n) +{ + TCGv_i64 t = tcg_temp_new_i64(); + + gen_saddlp_s_i64(t, n); + tcg_gen_add_i64(d, d, t); +} + +void gen_gvec_sadalp(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, + uint32_t opr_sz, uint32_t max_sz) +{ + static const TCGOpcode vecop_list[] = { + INDEX_op_sari_vec, INDEX_op_shli_vec, INDEX_op_add_vec, 0 + }; + static const GVecGen2 g[] = { + { .fniv = gen_sadalp_vec, + .fni8 = gen_sadalp_b_i64, + .opt_opc = vecop_list, + .load_dest = true, + .vece = MO_16 }, + { .fniv = gen_sadalp_vec, + .fni8 = gen_sadalp_h_i64, + .opt_opc = vecop_list, + .load_dest = true, + .vece = MO_32 }, + { .fniv = gen_sadalp_vec, + .fni8 = gen_sadalp_s_i64, + .opt_opc = vecop_list, + .load_dest = true, + .vece = MO_64 }, + }; + assert(vece <= MO_32); + tcg_gen_gvec_2(rd_ofs, rn_ofs, opr_sz, max_sz, &g[vece]); +} + +static void gen_uaddlp_vec(unsigned vece, TCGv_vec d, TCGv_vec n) +{ + int half = 4 << vece; + TCGv_vec t = tcg_temp_new_vec_matching(d); + TCGv_vec m = tcg_constant_vec_matching(d, vece, MAKE_64BIT_MASK(0, half)); + + tcg_gen_shri_vec(vece, t, n, half); + tcg_gen_and_vec(vece, d, n, m); + tcg_gen_add_vec(vece, d, d, t); +} + +static void gen_uaddlp_b_i64(TCGv_i64 d, TCGv_i64 n) +{ + TCGv_i64 t = tcg_temp_new_i64(); + TCGv_i64 m = tcg_constant_i64(dup_const(MO_16, 0xff)); + + tcg_gen_shri_i64(t, n, 8); + tcg_gen_and_i64(d, n, m); + tcg_gen_and_i64(t, t, m); + /* No carry between widened unsigned elements. */ + tcg_gen_add_i64(d, d, t); +} + +static void gen_uaddlp_h_i64(TCGv_i64 d, TCGv_i64 n) +{ + TCGv_i64 t = tcg_temp_new_i64(); + TCGv_i64 m = tcg_constant_i64(dup_const(MO_32, 0xffff)); + + tcg_gen_shri_i64(t, n, 16); + tcg_gen_and_i64(d, n, m); + tcg_gen_and_i64(t, t, m); + /* No carry between widened unsigned elements. */ + tcg_gen_add_i64(d, d, t); +} + +static void gen_uaddlp_s_i64(TCGv_i64 d, TCGv_i64 n) +{ + TCGv_i64 t = tcg_temp_new_i64(); + + tcg_gen_ext32u_i64(t, n); + tcg_gen_shri_i64(d, n, 32); + tcg_gen_add_i64(d, d, t); +} + +void gen_gvec_uaddlp(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, + uint32_t opr_sz, uint32_t max_sz) +{ + static const TCGOpcode vecop_list[] = { + INDEX_op_shri_vec, INDEX_op_add_vec, 0 + }; + static const GVecGen2 g[] = { + { .fniv = gen_uaddlp_vec, + .fni8 = gen_uaddlp_b_i64, + .opt_opc = vecop_list, + .vece = MO_16 }, + { .fniv = gen_uaddlp_vec, + .fni8 = gen_uaddlp_h_i64, + .opt_opc = vecop_list, + .vece = MO_32 }, + { .fniv = gen_uaddlp_vec, + .fni8 = gen_uaddlp_s_i64, + .opt_opc = vecop_list, + .vece = MO_64 }, + }; + assert(vece <= MO_32); + tcg_gen_gvec_2(rd_ofs, rn_ofs, opr_sz, max_sz, &g[vece]); +} + +static void gen_uadalp_vec(unsigned vece, TCGv_vec d, TCGv_vec n) +{ + TCGv_vec t = tcg_temp_new_vec_matching(d); + + gen_uaddlp_vec(vece, t, n); + tcg_gen_add_vec(vece, d, d, t); +} + +static void gen_uadalp_b_i64(TCGv_i64 d, TCGv_i64 n) +{ + TCGv_i64 t = tcg_temp_new_i64(); + + gen_uaddlp_b_i64(t, n); + tcg_gen_vec_add16_i64(d, d, t); +} + +static void gen_uadalp_h_i64(TCGv_i64 d, TCGv_i64 n) +{ + TCGv_i64 t = tcg_temp_new_i64(); + + gen_uaddlp_h_i64(t, n); + tcg_gen_vec_add32_i64(d, d, t); +} + +static void gen_uadalp_s_i64(TCGv_i64 d, TCGv_i64 n) +{ + TCGv_i64 t = tcg_temp_new_i64(); + + gen_uaddlp_s_i64(t, n); + tcg_gen_add_i64(d, d, t); +} + +void gen_gvec_uadalp(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, + uint32_t opr_sz, uint32_t max_sz) +{ + static const TCGOpcode vecop_list[] = { + INDEX_op_shri_vec, INDEX_op_add_vec, 0 + }; + static const GVecGen2 g[] = { + { .fniv = gen_uadalp_vec, + .fni8 = gen_uadalp_b_i64, + .load_dest = true, + .opt_opc = vecop_list, + .vece = MO_16 }, + { .fniv = gen_uadalp_vec, + .fni8 = gen_uadalp_h_i64, + .load_dest = true, + .opt_opc = vecop_list, + .vece = MO_32 }, + { .fniv = gen_uadalp_vec, + .fni8 = gen_uadalp_s_i64, + .load_dest = true, + .opt_opc = vecop_list, + .vece = MO_64 }, + }; + assert(vece <= MO_32); + tcg_gen_gvec_2(rd_ofs, rn_ofs, opr_sz, max_sz, &g[vece]); +} diff --git a/target/arm/tcg/neon_helper.c b/target/arm/tcg/neon_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/neon_helper.c +++ b/target/arm/tcg/neon_helper.c @@ -XXX,XX +XXX,XX @@ uint64_t HELPER(neon_addl_u32)(uint64_t a, uint64_t b) return (a + b) ^ mask; } -uint64_t HELPER(neon_paddl_u16)(uint64_t a, uint64_t b) -{ - uint64_t tmp; - uint64_t tmp2; - - tmp = a & 0x0000ffff0000ffffull; - tmp += (a >> 16) & 0x0000ffff0000ffffull; - tmp2 = b & 0xffff0000ffff0000ull; - tmp2 += (b << 16) & 0xffff0000ffff0000ull; - return ( tmp & 0xffff) - | ((tmp >> 16) & 0xffff0000ull) - | ((tmp2 << 16) & 0xffff00000000ull) - | ( tmp2 & 0xffff000000000000ull); -} - -uint64_t HELPER(neon_paddl_u32)(uint64_t a, uint64_t b) -{ - uint32_t low = a + (a >> 32); - uint32_t high = b + (b >> 32); - return low + ((uint64_t)high << 32); -} - /* Pairwise long add: add pairs of adjacent elements into * double-width elements in the result (eg _s8 is an 8x8->16 op) */ diff --git a/target/arm/tcg/translate-neon.c b/target/arm/tcg/translate-neon.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-neon.c +++ b/target/arm/tcg/translate-neon.c @@ -XXX,XX +XXX,XX @@ static bool trans_VDUP_scalar(DisasContext *s, arg_VDUP_scalar *a) return true; } -static bool do_2misc_pairwise(DisasContext *s, arg_2misc *a, - NeonGenWidenFn *widenfn, - NeonGenTwo64OpFn *opfn, - NeonGenTwo64OpFn *accfn) -{ - /* - * Pairwise long operations: widen both halves of the pair, - * combine the pairs with the opfn, and then possibly accumulate - * into the destination with the accfn. - */ - int pass; - - if (!arm_dc_feature(s, ARM_FEATURE_NEON)) { - return false; - } - - /* UNDEF accesses to D16-D31 if they don't exist. */ - if (!dc_isar_feature(aa32_simd_r32, s) && - ((a->vd | a->vm) & 0x10)) { - return false; - } - - if ((a->vd | a->vm) & a->q) { - return false; - } - - if (!widenfn) { - return false; - } - - if (!vfp_access_check(s)) { - return true; - } - - for (pass = 0; pass < a->q + 1; pass++) { - TCGv_i32 tmp; - TCGv_i64 rm0_64, rm1_64, rd_64; - - rm0_64 = tcg_temp_new_i64(); - rm1_64 = tcg_temp_new_i64(); - rd_64 = tcg_temp_new_i64(); - - tmp = tcg_temp_new_i32(); - read_neon_element32(tmp, a->vm, pass * 2, MO_32); - widenfn(rm0_64, tmp); - read_neon_element32(tmp, a->vm, pass * 2 + 1, MO_32); - widenfn(rm1_64, tmp); - - opfn(rd_64, rm0_64, rm1_64); - - if (accfn) { - TCGv_i64 tmp64 = tcg_temp_new_i64(); - read_neon_element64(tmp64, a->vd, pass, MO_64); - accfn(rd_64, tmp64, rd_64); - } - write_neon_element64(rd_64, a->vd, pass, MO_64); - } - return true; -} - -static bool trans_VPADDL_S(DisasContext *s, arg_2misc *a) -{ - static NeonGenWidenFn * const widenfn[] = { - gen_helper_neon_widen_s8, - gen_helper_neon_widen_s16, - tcg_gen_ext_i32_i64, - NULL, - }; - static NeonGenTwo64OpFn * const opfn[] = { - gen_helper_neon_paddl_u16, - gen_helper_neon_paddl_u32, - tcg_gen_add_i64, - NULL, - }; - - return do_2misc_pairwise(s, a, widenfn[a->size], opfn[a->size], NULL); -} - -static bool trans_VPADDL_U(DisasContext *s, arg_2misc *a) -{ - static NeonGenWidenFn * const widenfn[] = { - gen_helper_neon_widen_u8, - gen_helper_neon_widen_u16, - tcg_gen_extu_i32_i64, - NULL, - }; - static NeonGenTwo64OpFn * const opfn[] = { - gen_helper_neon_paddl_u16, - gen_helper_neon_paddl_u32, - tcg_gen_add_i64, - NULL, - }; - - return do_2misc_pairwise(s, a, widenfn[a->size], opfn[a->size], NULL); -} - -static bool trans_VPADAL_S(DisasContext *s, arg_2misc *a) -{ - static NeonGenWidenFn * const widenfn[] = { - gen_helper_neon_widen_s8, - gen_helper_neon_widen_s16, - tcg_gen_ext_i32_i64, - NULL, - }; - static NeonGenTwo64OpFn * const opfn[] = { - gen_helper_neon_paddl_u16, - gen_helper_neon_paddl_u32, - tcg_gen_add_i64, - NULL, - }; - static NeonGenTwo64OpFn * const accfn[] = { - gen_helper_neon_addl_u16, - gen_helper_neon_addl_u32, - tcg_gen_add_i64, - NULL, - }; - - return do_2misc_pairwise(s, a, widenfn[a->size], opfn[a->size], - accfn[a->size]); -} - -static bool trans_VPADAL_U(DisasContext *s, arg_2misc *a) -{ - static NeonGenWidenFn * const widenfn[] = { - gen_helper_neon_widen_u8, - gen_helper_neon_widen_u16, - tcg_gen_extu_i32_i64, - NULL, - }; - static NeonGenTwo64OpFn * const opfn[] = { - gen_helper_neon_paddl_u16, - gen_helper_neon_paddl_u32, - tcg_gen_add_i64, - NULL, - }; - static NeonGenTwo64OpFn * const accfn[] = { - gen_helper_neon_addl_u16, - gen_helper_neon_addl_u32, - tcg_gen_add_i64, - NULL, - }; - - return do_2misc_pairwise(s, a, widenfn[a->size], opfn[a->size], - accfn[a->size]); -} - typedef void ZipFn(TCGv_ptr, TCGv_ptr); static bool do_zip_uzp(DisasContext *s, arg_2misc *a, @@ -XXX,XX +XXX,XX @@ DO_2MISC_VEC(VCLT0, gen_gvec_clt0) DO_2MISC_VEC(VCLS, gen_gvec_cls) DO_2MISC_VEC(VCLZ, gen_gvec_clz) DO_2MISC_VEC(VREV64, gen_gvec_rev64) +DO_2MISC_VEC(VPADDL_S, gen_gvec_saddlp) +DO_2MISC_VEC(VPADDL_U, gen_gvec_uaddlp) +DO_2MISC_VEC(VPADAL_S, gen_gvec_sadalp) +DO_2MISC_VEC(VPADAL_U, gen_gvec_uadalp) static bool trans_VMVN(DisasContext *s, arg_2misc *a) { -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> This includes SADDLP, UADDLP, SADALP, UADALP. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-47-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/helper-a64.h | 2 - target/arm/tcg/a64.decode | 5 ++ target/arm/tcg/helper-a64.c | 18 -------- target/arm/tcg/translate-a64.c | 84 +++------------------------------- 4 files changed, 11 insertions(+), 98 deletions(-) diff --git a/target/arm/tcg/helper-a64.h b/target/arm/tcg/helper-a64.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/helper-a64.h +++ b/target/arm/tcg/helper-a64.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_3(recpsf_f64, TCG_CALL_NO_RWG, f64, f64, f64, ptr) DEF_HELPER_FLAGS_3(rsqrtsf_f16, TCG_CALL_NO_RWG, f16, f16, f16, ptr) DEF_HELPER_FLAGS_3(rsqrtsf_f32, TCG_CALL_NO_RWG, f32, f32, f32, ptr) DEF_HELPER_FLAGS_3(rsqrtsf_f64, TCG_CALL_NO_RWG, f64, f64, f64, ptr) -DEF_HELPER_FLAGS_1(neon_addlp_u8, TCG_CALL_NO_RWG_SE, i64, i64) -DEF_HELPER_FLAGS_1(neon_addlp_u16, TCG_CALL_NO_RWG_SE, i64, i64) DEF_HELPER_FLAGS_2(frecpx_f64, TCG_CALL_NO_RWG, f64, f64, ptr) DEF_HELPER_FLAGS_2(frecpx_f32, TCG_CALL_NO_RWG, f32, f32, ptr) DEF_HELPER_FLAGS_2(frecpx_f16, TCG_CALL_NO_RWG, f16, f16, ptr) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ CMLT0_v 0.00 1110 ..1 00000 10101 0 ..... ..... @qrr_e REV16_v 0.00 1110 001 00000 00011 0 ..... ..... @qrr_b REV32_v 0.10 1110 0.1 00000 00001 0 ..... ..... @qrr_bh REV64_v 0.00 1110 ..1 00000 00001 0 ..... ..... @qrr_e + +SADDLP_v 0.00 1110 ..1 00000 00101 0 ..... ..... @qrr_e +UADDLP_v 0.10 1110 ..1 00000 00101 0 ..... ..... @qrr_e +SADALP_v 0.00 1110 ..1 00000 01101 0 ..... ..... @qrr_e +UADALP_v 0.10 1110 ..1 00000 01101 0 ..... ..... @qrr_e diff --git a/target/arm/tcg/helper-a64.c b/target/arm/tcg/helper-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/helper-a64.c +++ b/target/arm/tcg/helper-a64.c @@ -XXX,XX +XXX,XX @@ float64 HELPER(rsqrtsf_f64)(float64 a, float64 b, void *fpstp) return float64_muladd(a, b, float64_three, float_muladd_halve_result, fpst); } -uint64_t HELPER(neon_addlp_u8)(uint64_t a) -{ - uint64_t tmp; - - tmp = a & 0x00ff00ff00ff00ffULL; - tmp += (a >> 8) & 0x00ff00ff00ff00ffULL; - return tmp; -} - -uint64_t HELPER(neon_addlp_u16)(uint64_t a) -{ - uint64_t tmp; - - tmp = a & 0x0000ffff0000ffffULL; - tmp += (a >> 16) & 0x0000ffff0000ffffULL; - return tmp; -} - /* Floating-point reciprocal exponent - see FPRecpX in ARM ARM */ uint32_t HELPER(frecpx_f16)(uint32_t a, void *fpstp) { diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static bool do_gvec_fn2_bhs(DisasContext *s, arg_qrr_e *a, GVecGen2Fn *fn) TRANS(CLS_v, do_gvec_fn2_bhs, a, gen_gvec_cls) TRANS(CLZ_v, do_gvec_fn2_bhs, a, gen_gvec_clz) TRANS(REV64_v, do_gvec_fn2_bhs, a, gen_gvec_rev64) +TRANS(SADDLP_v, do_gvec_fn2_bhs, a, gen_gvec_saddlp) +TRANS(UADDLP_v, do_gvec_fn2_bhs, a, gen_gvec_uaddlp) +TRANS(SADALP_v, do_gvec_fn2_bhs, a, gen_gvec_sadalp) +TRANS(UADALP_v, do_gvec_fn2_bhs, a, gen_gvec_uadalp) /* Common vector code for handling integer to FP conversion */ static void handle_simd_intfp_conv(DisasContext *s, int rd, int rn, @@ -XXX,XX +XXX,XX @@ static void handle_2misc_widening(DisasContext *s, int opcode, bool is_q, } } -static void handle_2misc_pairwise(DisasContext *s, int opcode, bool u, - bool is_q, int size, int rn, int rd) -{ - /* Implement the pairwise operations from 2-misc: - * SADDLP, UADDLP, SADALP, UADALP. - * These all add pairs of elements in the input to produce a - * double-width result element in the output (possibly accumulating). - */ - bool accum = (opcode == 0x6); - int maxpass = is_q ? 2 : 1; - int pass; - TCGv_i64 tcg_res[2]; - - if (size == 2) { - /* 32 + 32 -> 64 op */ - MemOp memop = size + (u ? 0 : MO_SIGN); - - for (pass = 0; pass < maxpass; pass++) { - TCGv_i64 tcg_op1 = tcg_temp_new_i64(); - TCGv_i64 tcg_op2 = tcg_temp_new_i64(); - - tcg_res[pass] = tcg_temp_new_i64(); - - read_vec_element(s, tcg_op1, rn, pass * 2, memop); - read_vec_element(s, tcg_op2, rn, pass * 2 + 1, memop); - tcg_gen_add_i64(tcg_res[pass], tcg_op1, tcg_op2); - if (accum) { - read_vec_element(s, tcg_op1, rd, pass, MO_64); - tcg_gen_add_i64(tcg_res[pass], tcg_res[pass], tcg_op1); - } - } - } else { - for (pass = 0; pass < maxpass; pass++) { - TCGv_i64 tcg_op = tcg_temp_new_i64(); - NeonGenOne64OpFn *genfn; - static NeonGenOne64OpFn * const fns[2][2] = { - { gen_helper_neon_addlp_s8, gen_helper_neon_addlp_u8 }, - { gen_helper_neon_addlp_s16, gen_helper_neon_addlp_u16 }, - }; - - genfn = fns[size][u]; - - tcg_res[pass] = tcg_temp_new_i64(); - - read_vec_element(s, tcg_op, rn, pass, MO_64); - genfn(tcg_res[pass], tcg_op); - - if (accum) { - read_vec_element(s, tcg_op, rd, pass, MO_64); - if (size == 0) { - gen_helper_neon_addl_u16(tcg_res[pass], - tcg_res[pass], tcg_op); - } else { - gen_helper_neon_addl_u32(tcg_res[pass], - tcg_res[pass], tcg_op); - } - } - } - } - if (!is_q) { - tcg_res[1] = tcg_constant_i64(0); - } - for (pass = 0; pass < 2; pass++) { - write_vec_element(s, tcg_res[pass], rd, pass, MO_64); - } -} - static void handle_shll(DisasContext *s, bool is_q, int size, int rn, int rd) { /* Implement SHLL and SHLL2 */ @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) handle_2misc_narrow(s, false, opcode, u, is_q, size, rn, rd); return; - case 0x2: /* SADDLP, UADDLP */ - case 0x6: /* SADALP, UADALP */ - if (size == 3) { - unallocated_encoding(s); - return; - } - if (!fp_access_check(s)) { - return; - } - handle_2misc_pairwise(s, opcode, u, is_q, size, rn, rd); - return; case 0x13: /* SHLL, SHLL2 */ if (u == 0 || size == 3) { unallocated_encoding(s); @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) default: case 0x0: /* REV64, REV32 */ case 0x1: /* REV16 */ + case 0x2: /* SADDLP, UADDLP */ case 0x3: /* SUQADD, USQADD */ case 0x4: /* CLS, CLZ */ case 0x5: /* CNT, NOT, RBIT */ + case 0x6: /* SADALP, UADALP */ case 0x7: /* SQABS, SQNEG */ case 0x8: /* CMGT, CMGE */ case 0x9: /* CMEQ, CMLE */ -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> These have generic equivalents: tcg_gen_vec_{add,sub}{16,32}_i64. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-48-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/helper.h | 4 ---- target/arm/tcg/neon_helper.c | 36 --------------------------------- target/arm/tcg/translate-neon.c | 22 ++++++++++---------- 3 files changed, 11 insertions(+), 51 deletions(-) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_1(neon_widen_s8, i64, i32) DEF_HELPER_1(neon_widen_u16, i64, i32) DEF_HELPER_1(neon_widen_s16, i64, i32) -DEF_HELPER_2(neon_addl_u16, i64, i64, i64) -DEF_HELPER_2(neon_addl_u32, i64, i64, i64) DEF_HELPER_FLAGS_1(neon_addlp_s8, TCG_CALL_NO_RWG_SE, i64, i64) DEF_HELPER_FLAGS_1(neon_addlp_s16, TCG_CALL_NO_RWG_SE, i64, i64) -DEF_HELPER_2(neon_subl_u16, i64, i64, i64) -DEF_HELPER_2(neon_subl_u32, i64, i64, i64) DEF_HELPER_3(neon_addl_saturate_s32, i64, env, i64, i64) DEF_HELPER_3(neon_addl_saturate_s64, i64, env, i64, i64) DEF_HELPER_2(neon_abdl_u16, i64, i32, i32) diff --git a/target/arm/tcg/neon_helper.c b/target/arm/tcg/neon_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/neon_helper.c +++ b/target/arm/tcg/neon_helper.c @@ -XXX,XX +XXX,XX @@ uint64_t HELPER(neon_widen_s16)(uint32_t x) return ((uint32_t)(int16_t)x) | (high << 32); } -uint64_t HELPER(neon_addl_u16)(uint64_t a, uint64_t b) -{ - uint64_t mask; - mask = (a ^ b) & 0x8000800080008000ull; - a &= ~0x8000800080008000ull; - b &= ~0x8000800080008000ull; - return (a + b) ^ mask; -} - -uint64_t HELPER(neon_addl_u32)(uint64_t a, uint64_t b) -{ - uint64_t mask; - mask = (a ^ b) & 0x8000000080000000ull; - a &= ~0x8000000080000000ull; - b &= ~0x8000000080000000ull; - return (a + b) ^ mask; -} - /* Pairwise long add: add pairs of adjacent elements into * double-width elements in the result (eg _s8 is an 8x8->16 op) */ @@ -XXX,XX +XXX,XX @@ uint64_t HELPER(neon_addlp_s16)(uint64_t a) return (uint32_t)reslo | (((uint64_t)reshi) << 32); } -uint64_t HELPER(neon_subl_u16)(uint64_t a, uint64_t b) -{ - uint64_t mask; - mask = (a ^ ~b) & 0x8000800080008000ull; - a |= 0x8000800080008000ull; - b &= ~0x8000800080008000ull; - return (a - b) ^ mask; -} - -uint64_t HELPER(neon_subl_u32)(uint64_t a, uint64_t b) -{ - uint64_t mask; - mask = (a ^ ~b) & 0x8000000080000000ull; - a |= 0x8000000080000000ull; - b &= ~0x8000000080000000ull; - return (a - b) ^ mask; -} - uint64_t HELPER(neon_addl_saturate_s32)(CPUARMState *env, uint64_t a, uint64_t b) { uint32_t x, y; diff --git a/target/arm/tcg/translate-neon.c b/target/arm/tcg/translate-neon.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-neon.c +++ b/target/arm/tcg/translate-neon.c @@ -XXX,XX +XXX,XX @@ static bool do_prewiden_3d(DisasContext *s, arg_3diff *a, NULL, NULL, \ }; \ static NeonGenTwo64OpFn * const addfn[] = { \ - gen_helper_neon_##OP##l_u16, \ - gen_helper_neon_##OP##l_u32, \ + tcg_gen_vec_##OP##16_i64, \ + tcg_gen_vec_##OP##32_i64, \ tcg_gen_##OP##_i64, \ NULL, \ }; \ @@ -XXX,XX +XXX,XX @@ static bool do_narrow_3d(DisasContext *s, arg_3diff *a, static bool trans_##INSN##_3d(DisasContext *s, arg_3diff *a) \ { \ static NeonGenTwo64OpFn * const addfn[] = { \ - gen_helper_neon_##OP##l_u16, \ - gen_helper_neon_##OP##l_u32, \ + tcg_gen_vec_##OP##16_i64, \ + tcg_gen_vec_##OP##32_i64, \ tcg_gen_##OP##_i64, \ NULL, \ }; \ @@ -XXX,XX +XXX,XX @@ static bool trans_VABAL_S_3d(DisasContext *s, arg_3diff *a) NULL, }; static NeonGenTwo64OpFn * const addfn[] = { - gen_helper_neon_addl_u16, - gen_helper_neon_addl_u32, + tcg_gen_vec_add16_i64, + tcg_gen_vec_add32_i64, tcg_gen_add_i64, NULL, }; @@ -XXX,XX +XXX,XX @@ static bool trans_VABAL_U_3d(DisasContext *s, arg_3diff *a) NULL, }; static NeonGenTwo64OpFn * const addfn[] = { - gen_helper_neon_addl_u16, - gen_helper_neon_addl_u32, + tcg_gen_vec_add16_i64, + tcg_gen_vec_add32_i64, tcg_gen_add_i64, NULL, }; @@ -XXX,XX +XXX,XX @@ static bool trans_VMULL_U_3d(DisasContext *s, arg_3diff *a) NULL, \ }; \ static NeonGenTwo64OpFn * const accfn[] = { \ - gen_helper_neon_##ACC##l_u16, \ - gen_helper_neon_##ACC##l_u32, \ + tcg_gen_vec_##ACC##16_i64, \ + tcg_gen_vec_##ACC##32_i64, \ tcg_gen_##ACC##_i64, \ NULL, \ }; \ @@ -XXX,XX +XXX,XX @@ static bool trans_VMULL_U_2sc(DisasContext *s, arg_2scalar *a) }; \ static NeonGenTwo64OpFn * const accfn[] = { \ NULL, \ - gen_helper_neon_##ACC##l_u32, \ + tcg_gen_vec_##ACC##32_i64, \ tcg_gen_##ACC##_i64, \ NULL, \ }; \ -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> In a couple of places, clearing the entire vector before storing one element is the easiest solution. Wrap that into a helper function. Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-49-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/translate-a64.c | 21 ++++++++++++--------- 1 file changed, 12 insertions(+), 9 deletions(-) diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static TCGv_i32 read_fp_hreg(DisasContext *s, int reg) return v; } -/* Clear the bits above an N-bit vector, for N = (is_q ? 128 : 64). +static void clear_vec(DisasContext *s, int rd) +{ + unsigned ofs = fp_reg_offset(s, rd, MO_64); + unsigned vsz = vec_full_reg_size(s); + + tcg_gen_gvec_dup_imm(MO_64, ofs, vsz, vsz, 0); +} + +/* + * Clear the bits above an N-bit vector, for N = (is_q ? 128 : 64). * If SVE is not enabled, then there are only 128 bits in the vector. */ static void clear_vec_high(DisasContext *s, bool is_q, int rd) @@ -XXX,XX +XXX,XX @@ static bool trans_SM3SS1(DisasContext *s, arg_SM3SS1 *a) TCGv_i32 tcg_op2 = tcg_temp_new_i32(); TCGv_i32 tcg_op3 = tcg_temp_new_i32(); TCGv_i32 tcg_res = tcg_temp_new_i32(); - unsigned vsz, dofs; read_vec_element_i32(s, tcg_op1, a->rn, 3, MO_32); read_vec_element_i32(s, tcg_op2, a->rm, 3, MO_32); @@ -XXX,XX +XXX,XX @@ static bool trans_SM3SS1(DisasContext *s, arg_SM3SS1 *a) tcg_gen_rotri_i32(tcg_res, tcg_res, 25); /* Clear the whole register first, then store bits [127:96]. */ - vsz = vec_full_reg_size(s); - dofs = vec_full_reg_offset(s, a->rd); - tcg_gen_gvec_dup_imm(MO_64, dofs, vsz, vsz, 0); + clear_vec(s, a->rd); write_vec_element_i32(s, tcg_res, a->rd, 3, MO_32); } return true; @@ -XXX,XX +XXX,XX @@ static bool do_scalar_muladd_widening_idx(DisasContext *s, arg_rrx_e *a, TCGv_i64 t0 = tcg_temp_new_i64(); TCGv_i64 t1 = tcg_temp_new_i64(); TCGv_i64 t2 = tcg_temp_new_i64(); - unsigned vsz, dofs; if (acc) { read_vec_element(s, t0, a->rd, 0, a->esz + 1); @@ -XXX,XX +XXX,XX @@ static bool do_scalar_muladd_widening_idx(DisasContext *s, arg_rrx_e *a, fn(t0, t1, t2); /* Clear the whole register first, then store scalar. */ - vsz = vec_full_reg_size(s); - dofs = vec_full_reg_offset(s, a->rd); - tcg_gen_gvec_dup_imm(MO_64, dofs, vsz, vsz, 0); + clear_vec(s, a->rd); write_vec_element(s, t0, a->rd, 0, a->esz + 1); } return true; -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-50-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 9 ++ target/arm/tcg/translate-a64.c | 153 ++++++++++++++++++++------------- 2 files changed, 102 insertions(+), 60 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ CMEQ0_s 0101 1110 111 00000 10011 0 ..... ..... @rr CMLE0_s 0111 1110 111 00000 10011 0 ..... ..... @rr CMLT0_s 0101 1110 111 00000 10101 0 ..... ..... @rr +SQXTUN_s 0111 1110 ..1 00001 00101 0 ..... ..... @rr_e +SQXTN_s 0101 1110 ..1 00001 01001 0 ..... ..... @rr_e +UQXTN_s 0111 1110 ..1 00001 01001 0 ..... ..... @rr_e + # Advanced SIMD two-register miscellaneous SQABS_v 0.00 1110 ..1 00000 01111 0 ..... ..... @qrr_e @@ -XXX,XX +XXX,XX @@ SADDLP_v 0.00 1110 ..1 00000 00101 0 ..... ..... @qrr_e UADDLP_v 0.10 1110 ..1 00000 00101 0 ..... ..... @qrr_e SADALP_v 0.00 1110 ..1 00000 01101 0 ..... ..... @qrr_e UADALP_v 0.10 1110 ..1 00000 01101 0 ..... ..... @qrr_e + +XTN 0.00 1110 ..1 00001 00101 0 ..... ..... @qrr_e +SQXTUN_v 0.10 1110 ..1 00001 00101 0 ..... ..... @qrr_e +SQXTN_v 0.00 1110 ..1 00001 01001 0 ..... ..... @qrr_e +UQXTN_v 0.10 1110 ..1 00001 01001 0 ..... ..... @qrr_e diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ TRANS(CMLE0_s, do_cmop0_d, a, TCG_COND_LE) TRANS(CMLT0_s, do_cmop0_d, a, TCG_COND_LT) TRANS(CMEQ0_s, do_cmop0_d, a, TCG_COND_EQ) +static bool do_2misc_narrow_scalar(DisasContext *s, arg_rr_e *a, + ArithOneOp * const fn[3]) +{ + if (a->esz == MO_64) { + return false; + } + if (fp_access_check(s)) { + TCGv_i64 t = tcg_temp_new_i64(); + + read_vec_element(s, t, a->rn, 0, a->esz + 1); + fn[a->esz](t, t); + clear_vec(s, a->rd); + write_vec_element(s, t, a->rd, 0, a->esz); + } + return true; +} + +#define WRAP_ENV(NAME) \ + static void gen_##NAME(TCGv_i64 d, TCGv_i64 n) \ + { gen_helper_##NAME(d, tcg_env, n); } + +WRAP_ENV(neon_unarrow_sat8) +WRAP_ENV(neon_unarrow_sat16) +WRAP_ENV(neon_unarrow_sat32) + +static ArithOneOp * const f_scalar_sqxtun[] = { + gen_neon_unarrow_sat8, + gen_neon_unarrow_sat16, + gen_neon_unarrow_sat32, +}; +TRANS(SQXTUN_s, do_2misc_narrow_scalar, a, f_scalar_sqxtun) + +WRAP_ENV(neon_narrow_sat_s8) +WRAP_ENV(neon_narrow_sat_s16) +WRAP_ENV(neon_narrow_sat_s32) + +static ArithOneOp * const f_scalar_sqxtn[] = { + gen_neon_narrow_sat_s8, + gen_neon_narrow_sat_s16, + gen_neon_narrow_sat_s32, +}; +TRANS(SQXTN_s, do_2misc_narrow_scalar, a, f_scalar_sqxtn) + +WRAP_ENV(neon_narrow_sat_u8) +WRAP_ENV(neon_narrow_sat_u16) +WRAP_ENV(neon_narrow_sat_u32) + +static ArithOneOp * const f_scalar_uqxtn[] = { + gen_neon_narrow_sat_u8, + gen_neon_narrow_sat_u16, + gen_neon_narrow_sat_u32, +}; +TRANS(UQXTN_s, do_2misc_narrow_scalar, a, f_scalar_uqxtn) + +#undef WRAP_ENV + static bool do_gvec_fn2(DisasContext *s, arg_qrr_e *a, GVecGen2Fn *fn) { if (!a->q && a->esz == MO_64) { @@ -XXX,XX +XXX,XX @@ TRANS(UADDLP_v, do_gvec_fn2_bhs, a, gen_gvec_uaddlp) TRANS(SADALP_v, do_gvec_fn2_bhs, a, gen_gvec_sadalp) TRANS(UADALP_v, do_gvec_fn2_bhs, a, gen_gvec_uadalp) +static bool do_2misc_narrow_vector(DisasContext *s, arg_qrr_e *a, + ArithOneOp * const fn[3]) +{ + if (a->esz == MO_64) { + return false; + } + if (fp_access_check(s)) { + TCGv_i64 t0 = tcg_temp_new_i64(); + TCGv_i64 t1 = tcg_temp_new_i64(); + + read_vec_element(s, t0, a->rn, 0, MO_64); + read_vec_element(s, t1, a->rn, 1, MO_64); + fn[a->esz](t0, t0); + fn[a->esz](t1, t1); + write_vec_element(s, t0, a->rd, a->q ? 2 : 0, MO_32); + write_vec_element(s, t1, a->rd, a->q ? 3 : 1, MO_32); + clear_vec_high(s, a->q, a->rd); + } + return true; +} + +static ArithOneOp * const f_scalar_xtn[] = { + gen_helper_neon_narrow_u8, + gen_helper_neon_narrow_u16, + tcg_gen_ext32u_i64, +}; +TRANS(XTN, do_2misc_narrow_vector, a, f_scalar_xtn) +TRANS(SQXTUN_v, do_2misc_narrow_vector, a, f_scalar_sqxtun) +TRANS(SQXTN_v, do_2misc_narrow_vector, a, f_scalar_sqxtn) +TRANS(UQXTN_v, do_2misc_narrow_vector, a, f_scalar_uqxtn) + /* Common vector code for handling integer to FP conversion */ static void handle_simd_intfp_conv(DisasContext *s, int rd, int rn, int elements, int is_signed, @@ -XXX,XX +XXX,XX @@ static void handle_2misc_narrow(DisasContext *s, bool scalar, tcg_res[pass] = tcg_temp_new_i64(); switch (opcode) { - case 0x12: /* XTN, SQXTUN */ - { - static NeonGenOne64OpFn * const xtnfns[3] = { - gen_helper_neon_narrow_u8, - gen_helper_neon_narrow_u16, - tcg_gen_ext32u_i64, - }; - static NeonGenOne64OpEnvFn * const sqxtunfns[3] = { - gen_helper_neon_unarrow_sat8, - gen_helper_neon_unarrow_sat16, - gen_helper_neon_unarrow_sat32, - }; - if (u) { - genenvfn = sqxtunfns[size]; - } else { - genfn = xtnfns[size]; - } - break; - } - case 0x14: /* SQXTN, UQXTN */ - { - static NeonGenOne64OpEnvFn * const fns[3][2] = { - { gen_helper_neon_narrow_sat_s8, - gen_helper_neon_narrow_sat_u8 }, - { gen_helper_neon_narrow_sat_s16, - gen_helper_neon_narrow_sat_u16 }, - { gen_helper_neon_narrow_sat_s32, - gen_helper_neon_narrow_sat_u32 }, - }; - genenvfn = fns[size][u]; - break; - } case 0x16: /* FCVTN, FCVTN2 */ /* 32 bit to 16 bit or 64 bit to 32 bit float conversion */ if (size == 2) { @@ -XXX,XX +XXX,XX @@ static void handle_2misc_narrow(DisasContext *s, bool scalar, } break; default: + case 0x12: /* XTN, SQXTUN */ + case 0x14: /* SQXTN, UQXTN */ g_assert_not_reached(); } @@ -XXX,XX +XXX,XX @@ static void disas_simd_scalar_two_reg_misc(DisasContext *s, uint32_t insn) TCGv_ptr tcg_fpstatus; switch (opcode) { - case 0x12: /* SQXTUN */ - if (!u) { - unallocated_encoding(s); - return; - } - /* fall through */ - case 0x14: /* SQXTN, UQXTN */ - if (size == 3) { - unallocated_encoding(s); - return; - } - if (!fp_access_check(s)) { - return; - } - handle_2misc_narrow(s, true, opcode, u, false, size, rn, rd); - return; case 0xc ... 0xf: case 0x16 ... 0x1d: case 0x1f: @@ -XXX,XX +XXX,XX @@ static void disas_simd_scalar_two_reg_misc(DisasContext *s, uint32_t insn) case 0x9: /* CMEQ, CMLE */ case 0xa: /* CMLT */ case 0xb: /* ABS, NEG */ + case 0x12: /* SQXTUN */ + case 0x14: /* SQXTN, UQXTN */ unallocated_encoding(s); return; } @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) TCGv_ptr tcg_fpstatus; switch (opcode) { - case 0x12: /* XTN, XTN2, SQXTUN, SQXTUN2 */ - case 0x14: /* SQXTN, SQXTN2, UQXTN, UQXTN2 */ - if (size == 3) { - unallocated_encoding(s); - return; - } - if (!fp_access_check(s)) { - return; - } - - handle_2misc_narrow(s, false, opcode, u, is_q, size, rn, rd); - return; case 0x13: /* SHLL, SHLL2 */ if (u == 0 || size == 3) { unallocated_encoding(s); @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) case 0x9: /* CMEQ, CMLE */ case 0xa: /* CMLT */ case 0xb: /* ABS, NEG */ + case 0x12: /* XTN, XTN2, SQXTUN, SQXTUN2 */ + case 0x14: /* SQXTN, SQXTN2, UQXTN, UQXTN2 */ unallocated_encoding(s); return; } -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-51-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 5 ++ target/arm/tcg/translate-a64.c | 89 ++++++++++++++++++---------------- 2 files changed, 52 insertions(+), 42 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ %rd 0:5 %esz_sd 22:1 !function=plus_2 +%esz_hs 22:1 !function=plus_1 %esz_hsd 22:2 !function=xor_2 %hl 11:1 21:1 %hlm 11:1 20:2 @@ -XXX,XX +XXX,XX @@ @qrr_b . q:1 ...... .. ...... ...... rn:5 rd:5 &qrr_e esz=0 @qrr_h . q:1 ...... .. ...... ...... rn:5 rd:5 &qrr_e esz=1 @qrr_bh . q:1 ...... . esz:1 ...... ...... rn:5 rd:5 &qrr_e +@qrr_hs . q:1 ...... .. ...... ...... rn:5 rd:5 &qrr_e esz=%esz_hs @qrr_e . q:1 ...... esz:2 ...... ...... rn:5 rd:5 &qrr_e @qrrr_b . q:1 ...... ... rm:5 ...... rn:5 rd:5 &qrrr_e esz=0 @@ -XXX,XX +XXX,XX @@ XTN 0.00 1110 ..1 00001 00101 0 ..... ..... @qrr_e SQXTUN_v 0.10 1110 ..1 00001 00101 0 ..... ..... @qrr_e SQXTN_v 0.00 1110 ..1 00001 01001 0 ..... ..... @qrr_e UQXTN_v 0.10 1110 ..1 00001 01001 0 ..... ..... @qrr_e + +FCVTN_v 0.00 1110 0.1 00001 01101 0 ..... ..... @qrr_hs +BFCVTN_v 0.00 1110 101 00001 01101 0 ..... ..... @qrr_h diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ TRANS(SQXTUN_v, do_2misc_narrow_vector, a, f_scalar_sqxtun) TRANS(SQXTN_v, do_2misc_narrow_vector, a, f_scalar_sqxtn) TRANS(UQXTN_v, do_2misc_narrow_vector, a, f_scalar_uqxtn) +static void gen_fcvtn_hs(TCGv_i64 d, TCGv_i64 n) +{ + TCGv_i32 tcg_lo = tcg_temp_new_i32(); + TCGv_i32 tcg_hi = tcg_temp_new_i32(); + TCGv_ptr fpst = fpstatus_ptr(FPST_FPCR); + TCGv_i32 ahp = get_ahp_flag(); + + tcg_gen_extr_i64_i32(tcg_lo, tcg_hi, n); + gen_helper_vfp_fcvt_f32_to_f16(tcg_lo, tcg_lo, fpst, ahp); + gen_helper_vfp_fcvt_f32_to_f16(tcg_hi, tcg_hi, fpst, ahp); + tcg_gen_deposit_i32(tcg_lo, tcg_lo, tcg_hi, 16, 16); + tcg_gen_extu_i32_i64(d, tcg_lo); +} + +static void gen_fcvtn_sd(TCGv_i64 d, TCGv_i64 n) +{ + TCGv_i32 tmp = tcg_temp_new_i32(); + gen_helper_vfp_fcvtsd(tmp, n, tcg_env); + tcg_gen_extu_i32_i64(d, tmp); +} + +static ArithOneOp * const f_vector_fcvtn[] = { + NULL, + gen_fcvtn_hs, + gen_fcvtn_sd, +}; +TRANS(FCVTN_v, do_2misc_narrow_vector, a, f_vector_fcvtn) + +static void gen_bfcvtn_hs(TCGv_i64 d, TCGv_i64 n) +{ + TCGv_ptr fpst = fpstatus_ptr(FPST_FPCR); + TCGv_i32 tmp = tcg_temp_new_i32(); + gen_helper_bfcvt_pair(tmp, n, fpst); + tcg_gen_extu_i32_i64(d, tmp); +} + +static ArithOneOp * const f_vector_bfcvtn[] = { + NULL, + gen_bfcvtn_hs, + NULL, +}; +TRANS_FEAT(BFCVTN_v, aa64_bf16, do_2misc_narrow_vector, a, f_vector_bfcvtn) + /* Common vector code for handling integer to FP conversion */ static void handle_simd_intfp_conv(DisasContext *s, int rd, int rn, int elements, int is_signed, @@ -XXX,XX +XXX,XX @@ static void handle_2misc_narrow(DisasContext *s, bool scalar, tcg_res[pass] = tcg_temp_new_i64(); switch (opcode) { - case 0x16: /* FCVTN, FCVTN2 */ - /* 32 bit to 16 bit or 64 bit to 32 bit float conversion */ - if (size == 2) { - TCGv_i32 tmp = tcg_temp_new_i32(); - gen_helper_vfp_fcvtsd(tmp, tcg_op, tcg_env); - tcg_gen_extu_i32_i64(tcg_res[pass], tmp); - } else { - TCGv_i32 tcg_lo = tcg_temp_new_i32(); - TCGv_i32 tcg_hi = tcg_temp_new_i32(); - TCGv_ptr fpst = fpstatus_ptr(FPST_FPCR); - TCGv_i32 ahp = get_ahp_flag(); - - tcg_gen_extr_i64_i32(tcg_lo, tcg_hi, tcg_op); - gen_helper_vfp_fcvt_f32_to_f16(tcg_lo, tcg_lo, fpst, ahp); - gen_helper_vfp_fcvt_f32_to_f16(tcg_hi, tcg_hi, fpst, ahp); - tcg_gen_deposit_i32(tcg_lo, tcg_lo, tcg_hi, 16, 16); - tcg_gen_extu_i32_i64(tcg_res[pass], tcg_lo); - } - break; - case 0x36: /* BFCVTN, BFCVTN2 */ - { - TCGv_ptr fpst = fpstatus_ptr(FPST_FPCR); - TCGv_i32 tmp = tcg_temp_new_i32(); - gen_helper_bfcvt_pair(tmp, tcg_op, fpst); - tcg_gen_extu_i32_i64(tcg_res[pass], tmp); - } - break; case 0x56: /* FCVTXN, FCVTXN2 */ { /* @@ -XXX,XX +XXX,XX @@ static void handle_2misc_narrow(DisasContext *s, bool scalar, default: case 0x12: /* XTN, SQXTUN */ case 0x14: /* SQXTN, UQXTN */ + case 0x16: /* FCVTN, FCVTN2 */ + case 0x36: /* BFCVTN, BFCVTN2 */ g_assert_not_reached(); } @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) unallocated_encoding(s); return; } - /* fall through */ - case 0x16: /* FCVTN, FCVTN2 */ - /* handle_2misc_narrow does a 2*size -> size operation, but these - * instructions encode the source size rather than dest size. - */ - if (!fp_access_check(s)) { - return; - } - handle_2misc_narrow(s, false, opcode, 0, is_q, size - 1, rn, rd); - return; - case 0x36: /* BFCVTN, BFCVTN2 */ - if (!dc_isar_feature(aa64_bf16, s) || size != 2) { - unallocated_encoding(s); - return; - } if (!fp_access_check(s)) { return; } @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) } break; default: + case 0x16: /* FCVTN, FCVTN2 */ + case 0x36: /* BFCVTN, BFCVTN2 */ unallocated_encoding(s); return; } -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Remove handle_2misc_narrow as this was the last insn decoded by that function. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-52-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 4 ++ target/arm/tcg/translate-a64.c | 101 +++++++-------------------------- 2 files changed, 24 insertions(+), 81 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ @qrr_b . q:1 ...... .. ...... ...... rn:5 rd:5 &qrr_e esz=0 @qrr_h . q:1 ...... .. ...... ...... rn:5 rd:5 &qrr_e esz=1 +@qrr_s . q:1 ...... .. ...... ...... rn:5 rd:5 &qrr_e esz=2 @qrr_bh . q:1 ...... . esz:1 ...... ...... rn:5 rd:5 &qrr_e @qrr_hs . q:1 ...... .. ...... ...... rn:5 rd:5 &qrr_e esz=%esz_hs @qrr_e . q:1 ...... esz:2 ...... ...... rn:5 rd:5 &qrr_e @@ -XXX,XX +XXX,XX @@ SQXTUN_s 0111 1110 ..1 00001 00101 0 ..... ..... @rr_e SQXTN_s 0101 1110 ..1 00001 01001 0 ..... ..... @rr_e UQXTN_s 0111 1110 ..1 00001 01001 0 ..... ..... @rr_e +FCVTXN_s 0111 1110 011 00001 01101 0 ..... ..... @rr_s + # Advanced SIMD two-register miscellaneous SQABS_v 0.00 1110 ..1 00000 01111 0 ..... ..... @qrr_e @@ -XXX,XX +XXX,XX @@ SQXTN_v 0.00 1110 ..1 00001 01001 0 ..... ..... @qrr_e UQXTN_v 0.10 1110 ..1 00001 01001 0 ..... ..... @qrr_e FCVTN_v 0.00 1110 0.1 00001 01101 0 ..... ..... @qrr_hs +FCVTXN_v 0.10 1110 011 00001 01101 0 ..... ..... @qrr_s BFCVTN_v 0.00 1110 101 00001 01101 0 ..... ..... @qrr_h diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static ArithOneOp * const f_scalar_uqxtn[] = { }; TRANS(UQXTN_s, do_2misc_narrow_scalar, a, f_scalar_uqxtn) +static void gen_fcvtxn_sd(TCGv_i64 d, TCGv_i64 n) +{ + /* + * 64 bit to 32 bit float conversion + * with von Neumann rounding (round to odd) + */ + TCGv_i32 tmp = tcg_temp_new_i32(); + gen_helper_fcvtx_f64_to_f32(tmp, n, tcg_env); + tcg_gen_extu_i32_i64(d, tmp); +} + +static ArithOneOp * const f_scalar_fcvtxn[] = { + NULL, + NULL, + gen_fcvtxn_sd, +}; +TRANS(FCVTXN_s, do_2misc_narrow_scalar, a, f_scalar_fcvtxn) + #undef WRAP_ENV static bool do_gvec_fn2(DisasContext *s, arg_qrr_e *a, GVecGen2Fn *fn) @@ -XXX,XX +XXX,XX @@ static ArithOneOp * const f_vector_fcvtn[] = { gen_fcvtn_sd, }; TRANS(FCVTN_v, do_2misc_narrow_vector, a, f_vector_fcvtn) +TRANS(FCVTXN_v, do_2misc_narrow_vector, a, f_scalar_fcvtxn) static void gen_bfcvtn_hs(TCGv_i64 d, TCGv_i64 n) { @@ -XXX,XX +XXX,XX @@ static void handle_2misc_reciprocal(DisasContext *s, int opcode, } } -static void handle_2misc_narrow(DisasContext *s, bool scalar, - int opcode, bool u, bool is_q, - int size, int rn, int rd) -{ - /* Handle 2-reg-misc ops which are narrowing (so each 2*size element - * in the source becomes a size element in the destination). - */ - int pass; - TCGv_i64 tcg_res[2]; - int destelt = is_q ? 2 : 0; - int passes = scalar ? 1 : 2; - - if (scalar) { - tcg_res[1] = tcg_constant_i64(0); - } - - for (pass = 0; pass < passes; pass++) { - TCGv_i64 tcg_op = tcg_temp_new_i64(); - NeonGenOne64OpFn *genfn = NULL; - NeonGenOne64OpEnvFn *genenvfn = NULL; - - if (scalar) { - read_vec_element(s, tcg_op, rn, pass, size + 1); - } else { - read_vec_element(s, tcg_op, rn, pass, MO_64); - } - tcg_res[pass] = tcg_temp_new_i64(); - - switch (opcode) { - case 0x56: /* FCVTXN, FCVTXN2 */ - { - /* - * 64 bit to 32 bit float conversion - * with von Neumann rounding (round to odd) - */ - TCGv_i32 tmp = tcg_temp_new_i32(); - assert(size == 2); - gen_helper_fcvtx_f64_to_f32(tmp, tcg_op, tcg_env); - tcg_gen_extu_i32_i64(tcg_res[pass], tmp); - } - break; - default: - case 0x12: /* XTN, SQXTUN */ - case 0x14: /* SQXTN, UQXTN */ - case 0x16: /* FCVTN, FCVTN2 */ - case 0x36: /* BFCVTN, BFCVTN2 */ - g_assert_not_reached(); - } - - if (genfn) { - genfn(tcg_res[pass], tcg_op); - } else if (genenvfn) { - genenvfn(tcg_res[pass], tcg_env, tcg_op); - } - } - - for (pass = 0; pass < 2; pass++) { - write_vec_element(s, tcg_res[pass], rd, destelt + pass, MO_32); - } - clear_vec_high(s, is_q, rd); -} - /* AdvSIMD scalar two reg misc * 31 30 29 28 24 23 22 21 17 16 12 11 10 9 5 4 0 * +-----+---+-----------+------+-----------+--------+-----+------+------+ @@ -XXX,XX +XXX,XX @@ static void disas_simd_scalar_two_reg_misc(DisasContext *s, uint32_t insn) rmode = FPROUNDING_TIEAWAY; break; case 0x56: /* FCVTXN, FCVTXN2 */ - if (size == 2) { - unallocated_encoding(s); - return; - } - if (!fp_access_check(s)) { - return; - } - handle_2misc_narrow(s, true, opcode, u, false, size - 1, rn, rd); - return; default: unallocated_encoding(s); return; @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) } handle_2misc_reciprocal(s, opcode, false, u, is_q, size, rn, rd); return; - case 0x56: /* FCVTXN, FCVTXN2 */ - if (size == 2) { - unallocated_encoding(s); - return; - } - if (!fp_access_check(s)) { - return; - } - handle_2misc_narrow(s, false, opcode, 0, is_q, size - 1, rn, rd); - return; case 0x17: /* FCVTL, FCVTL2 */ if (!fp_access_check(s)) { return; @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) default: case 0x16: /* FCVTN, FCVTN2 */ case 0x36: /* BFCVTN, BFCVTN2 */ + case 0x56: /* FCVTXN, FCVTXN2 */ unallocated_encoding(s); return; } -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-53-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 2 + target/arm/tcg/translate-a64.c | 75 +++++++++++++++++----------------- 2 files changed, 40 insertions(+), 37 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ UQXTN_v 0.10 1110 ..1 00001 01001 0 ..... ..... @qrr_e FCVTN_v 0.00 1110 0.1 00001 01101 0 ..... ..... @qrr_hs FCVTXN_v 0.10 1110 011 00001 01101 0 ..... ..... @qrr_s BFCVTN_v 0.00 1110 101 00001 01101 0 ..... ..... @qrr_h + +SHLL_v 0.10 1110 ..1 00001 00111 0 ..... ..... @qrr_e diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static ArithOneOp * const f_vector_bfcvtn[] = { }; TRANS_FEAT(BFCVTN_v, aa64_bf16, do_2misc_narrow_vector, a, f_vector_bfcvtn) +static bool trans_SHLL_v(DisasContext *s, arg_qrr_e *a) +{ + static NeonGenWidenFn * const widenfns[3] = { + gen_helper_neon_widen_u8, + gen_helper_neon_widen_u16, + tcg_gen_extu_i32_i64, + }; + NeonGenWidenFn *widenfn; + TCGv_i64 tcg_res[2]; + TCGv_i32 tcg_op; + int part, pass; + + if (a->esz == MO_64) { + return false; + } + if (!fp_access_check(s)) { + return true; + } + + tcg_op = tcg_temp_new_i32(); + widenfn = widenfns[a->esz]; + part = a->q ? 2 : 0; + + for (pass = 0; pass < 2; pass++) { + read_vec_element_i32(s, tcg_op, a->rn, part + pass, MO_32); + tcg_res[pass] = tcg_temp_new_i64(); + widenfn(tcg_res[pass], tcg_op); + tcg_gen_shli_i64(tcg_res[pass], tcg_res[pass], 8 << a->esz); + } + + for (pass = 0; pass < 2; pass++) { + write_vec_element(s, tcg_res[pass], a->rd, pass, MO_64); + } + return true; +} + + /* Common vector code for handling integer to FP conversion */ static void handle_simd_intfp_conv(DisasContext *s, int rd, int rn, int elements, int is_signed, @@ -XXX,XX +XXX,XX @@ static void handle_2misc_widening(DisasContext *s, int opcode, bool is_q, } } -static void handle_shll(DisasContext *s, bool is_q, int size, int rn, int rd) -{ - /* Implement SHLL and SHLL2 */ - int pass; - int part = is_q ? 2 : 0; - TCGv_i64 tcg_res[2]; - - for (pass = 0; pass < 2; pass++) { - static NeonGenWidenFn * const widenfns[3] = { - gen_helper_neon_widen_u8, - gen_helper_neon_widen_u16, - tcg_gen_extu_i32_i64, - }; - NeonGenWidenFn *widenfn = widenfns[size]; - TCGv_i32 tcg_op = tcg_temp_new_i32(); - - read_vec_element_i32(s, tcg_op, rn, part + pass, MO_32); - tcg_res[pass] = tcg_temp_new_i64(); - widenfn(tcg_res[pass], tcg_op); - tcg_gen_shli_i64(tcg_res[pass], tcg_res[pass], 8 << size); - } - - for (pass = 0; pass < 2; pass++) { - write_vec_element(s, tcg_res[pass], rd, pass, MO_64); - } -} - /* AdvSIMD two reg misc * 31 30 29 28 24 23 22 21 17 16 12 11 10 9 5 4 0 * +---+---+---+-----------+------+-----------+--------+-----+------+------+ @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) TCGv_ptr tcg_fpstatus; switch (opcode) { - case 0x13: /* SHLL, SHLL2 */ - if (u == 0 || size == 3) { - unallocated_encoding(s); - return; - } - if (!fp_access_check(s)) { - return; - } - handle_shll(s, is_q, size, rn, rd); - return; case 0xc ... 0xf: case 0x16 ... 0x1f: { @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) case 0xa: /* CMLT */ case 0xb: /* ABS, NEG */ case 0x12: /* XTN, XTN2, SQXTUN, SQXTUN2 */ + case 0x13: /* SHLL, SHLL2 */ case 0x14: /* SQXTN, SQXTN2, UQXTN, UQXTN2 */ unallocated_encoding(s); return; -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Move the current implementation out of translate-neon.c, and extend to handle all element sizes. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-54-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/translate.h | 6 ++++++ target/arm/tcg/gengvec.c | 14 ++++++++++++++ target/arm/tcg/translate-neon.c | 20 ++------------------ 3 files changed, 22 insertions(+), 18 deletions(-) diff --git a/target/arm/tcg/translate.h b/target/arm/tcg/translate.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate.h +++ b/target/arm/tcg/translate.h @@ -XXX,XX +XXX,XX @@ void gen_gvec_uaddlp(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, void gen_gvec_uadalp(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, uint32_t opr_sz, uint32_t max_sz); +/* These exclusively manipulate the sign bit. */ +void gen_gvec_fabs(unsigned vece, uint32_t dofs, uint32_t aofs, + uint32_t oprsz, uint32_t maxsz); +void gen_gvec_fneg(unsigned vece, uint32_t dofs, uint32_t aofs, + uint32_t oprsz, uint32_t maxsz); + /* * Forward to the isar_feature_* tests given a DisasContext pointer. */ diff --git a/target/arm/tcg/gengvec.c b/target/arm/tcg/gengvec.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/gengvec.c +++ b/target/arm/tcg/gengvec.c @@ -XXX,XX +XXX,XX @@ void gen_gvec_uadalp(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, assert(vece <= MO_32); tcg_gen_gvec_2(rd_ofs, rn_ofs, opr_sz, max_sz, &g[vece]); } + +void gen_gvec_fabs(unsigned vece, uint32_t dofs, uint32_t aofs, + uint32_t oprsz, uint32_t maxsz) +{ + uint64_t s_bit = 1ull << ((8 << vece) - 1); + tcg_gen_gvec_andi(vece, dofs, aofs, s_bit - 1, oprsz, maxsz); +} + +void gen_gvec_fneg(unsigned vece, uint32_t dofs, uint32_t aofs, + uint32_t oprsz, uint32_t maxsz) +{ + uint64_t s_bit = 1ull << ((8 << vece) - 1); + tcg_gen_gvec_xori(vece, dofs, aofs, s_bit, oprsz, maxsz); +} diff --git a/target/arm/tcg/translate-neon.c b/target/arm/tcg/translate-neon.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-neon.c +++ b/target/arm/tcg/translate-neon.c @@ -XXX,XX +XXX,XX @@ static bool do_2misc(DisasContext *s, arg_2misc *a, NeonGenOneOpFn *fn) return true; } -static void gen_VABS_F(unsigned vece, uint32_t rd_ofs, uint32_t rm_ofs, - uint32_t oprsz, uint32_t maxsz) -{ - tcg_gen_gvec_andi(vece, rd_ofs, rm_ofs, - vece == MO_16 ? 0x7fff : 0x7fffffff, - oprsz, maxsz); -} - static bool trans_VABS_F(DisasContext *s, arg_2misc *a) { if (a->size == MO_16) { @@ -XXX,XX +XXX,XX @@ static bool trans_VABS_F(DisasContext *s, arg_2misc *a) } else if (a->size != MO_32) { return false; } - return do_2misc_vec(s, a, gen_VABS_F); -} - -static void gen_VNEG_F(unsigned vece, uint32_t rd_ofs, uint32_t rm_ofs, - uint32_t oprsz, uint32_t maxsz) -{ - tcg_gen_gvec_xori(vece, rd_ofs, rm_ofs, - vece == MO_16 ? 0x8000 : 0x80000000, - oprsz, maxsz); + return do_2misc_vec(s, a, gen_gvec_fabs); } static bool trans_VNEG_F(DisasContext *s, arg_2misc *a) @@ -XXX,XX +XXX,XX @@ static bool trans_VNEG_F(DisasContext *s, arg_2misc *a) } else if (a->size != MO_32) { return false; } - return do_2misc_vec(s, a, gen_VNEG_F); + return do_2misc_vec(s, a, gen_gvec_fneg); } static bool trans_VRECPE(DisasContext *s, arg_2misc *a) -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-55-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 7 +++++ target/arm/tcg/translate-a64.c | 54 +++++++++++++++------------------- 2 files changed, 31 insertions(+), 30 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ @qrr_s . q:1 ...... .. ...... ...... rn:5 rd:5 &qrr_e esz=2 @qrr_bh . q:1 ...... . esz:1 ...... ...... rn:5 rd:5 &qrr_e @qrr_hs . q:1 ...... .. ...... ...... rn:5 rd:5 &qrr_e esz=%esz_hs +@qrr_sd . q:1 ...... .. ...... ...... rn:5 rd:5 &qrr_e esz=%esz_sd @qrr_e . q:1 ...... esz:2 ...... ...... rn:5 rd:5 &qrr_e @qrrr_b . q:1 ...... ... rm:5 ...... rn:5 rd:5 &qrrr_e esz=0 @@ -XXX,XX +XXX,XX @@ FCVTXN_v 0.10 1110 011 00001 01101 0 ..... ..... @qrr_s BFCVTN_v 0.00 1110 101 00001 01101 0 ..... ..... @qrr_h SHLL_v 0.10 1110 ..1 00001 00111 0 ..... ..... @qrr_e + +FABS_v 0.00 1110 111 11000 11111 0 ..... ..... @qrr_h +FABS_v 0.00 1110 1.1 00000 11111 0 ..... ..... @qrr_sd + +FNEG_v 0.10 1110 111 11000 11111 0 ..... ..... @qrr_h +FNEG_v 0.10 1110 1.1 00000 11111 0 ..... ..... @qrr_sd diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static bool trans_SHLL_v(DisasContext *s, arg_qrr_e *a) return true; } +static bool do_fabs_fneg_v(DisasContext *s, arg_qrr_e *a, GVecGen2Fn *fn) +{ + int check = fp_access_check_vector_hsd(s, a->q, a->esz); + + if (check <= 0) { + return check == 0; + } + + gen_gvec_fn2(s, a->q, a->rd, a->rn, fn, a->esz); + return true; +} + +TRANS(FABS_v, do_fabs_fneg_v, a, gen_gvec_fabs) +TRANS(FNEG_v, do_fabs_fneg_v, a, gen_gvec_fneg) /* Common vector code for handling integer to FP conversion */ static void handle_simd_intfp_conv(DisasContext *s, int rd, int rn, @@ -XXX,XX +XXX,XX @@ static void handle_2misc_64(DisasContext *s, int opcode, bool u, * requires them. */ switch (opcode) { - case 0x2f: /* FABS */ - gen_vfp_absd(tcg_rd, tcg_rn); - break; - case 0x6f: /* FNEG */ - gen_vfp_negd(tcg_rd, tcg_rn); - break; case 0x7f: /* FSQRT */ gen_helper_vfp_sqrtd(tcg_rd, tcg_rn, tcg_fpstatus); break; @@ -XXX,XX +XXX,XX @@ static void handle_2misc_64(DisasContext *s, int opcode, bool u, case 0x9: /* CMEQ, CMLE */ case 0xa: /* CMLT */ case 0xb: /* ABS, NEG */ + case 0x2f: /* FABS */ + case 0x6f: /* FNEG */ g_assert_not_reached(); } } @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) opcode |= (extract32(size, 1, 1) << 5) | (u << 6); size = is_double ? 3 : 2; switch (opcode) { - case 0x2f: /* FABS */ - case 0x6f: /* FNEG */ - if (size == 3 && !is_q) { - unallocated_encoding(s); - return; - } - break; case 0x1d: /* SCVTF */ case 0x5d: /* UCVTF */ { @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) case 0x16: /* FCVTN, FCVTN2 */ case 0x36: /* BFCVTN, BFCVTN2 */ case 0x56: /* FCVTXN, FCVTXN2 */ + case 0x2f: /* FABS */ + case 0x6f: /* FNEG */ unallocated_encoding(s); return; } @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) { /* Special cases for 32 bit elements */ switch (opcode) { - case 0x2f: /* FABS */ - gen_vfp_abss(tcg_res, tcg_op); - break; - case 0x6f: /* FNEG */ - gen_vfp_negs(tcg_res, tcg_op); - break; case 0x7f: /* FSQRT */ gen_helper_vfp_sqrts(tcg_res, tcg_op, tcg_fpstatus); break; @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) break; default: case 0x7: /* SQABS, SQNEG */ + case 0x2f: /* FABS */ + case 0x6f: /* FNEG */ g_assert_not_reached(); } } @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc_fp16(DisasContext *s, uint32_t insn) case 0x7b: /* FCVTZU */ rmode = FPROUNDING_ZERO; break; - case 0x2f: /* FABS */ - case 0x6f: /* FNEG */ - only_in_vector = true; - need_fpst = false; - break; case 0x7d: /* FRSQRTE */ break; case 0x7f: /* FSQRT (vector) */ only_in_vector = true; break; default: + case 0x2f: /* FABS */ + case 0x6f: /* FNEG */ unallocated_encoding(s); return; } @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc_fp16(DisasContext *s, uint32_t insn) case 0x59: /* FRINTX */ gen_helper_advsimd_rinth_exact(tcg_res, tcg_op, tcg_fpstatus); break; - case 0x2f: /* FABS */ - tcg_gen_andi_i32(tcg_res, tcg_op, 0x7fff); - break; - case 0x6f: /* FNEG */ - tcg_gen_xori_i32(tcg_res, tcg_op, 0x8000); - break; case 0x7d: /* FRSQRTE */ gen_helper_rsqrte_f16(tcg_res, tcg_op, tcg_fpstatus); break; @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc_fp16(DisasContext *s, uint32_t insn) gen_helper_vfp_sqrth(tcg_res, tcg_op, tcg_fpstatus); break; default: + case 0x2f: /* FABS */ + case 0x6f: /* FNEG */ g_assert_not_reached(); } -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-56-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 3 ++ target/arm/tcg/translate-a64.c | 69 ++++++++++++++++++++++++---------- 2 files changed, 53 insertions(+), 19 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ FABS_v 0.00 1110 1.1 00000 11111 0 ..... ..... @qrr_sd FNEG_v 0.10 1110 111 11000 11111 0 ..... ..... @qrr_h FNEG_v 0.10 1110 1.1 00000 11111 0 ..... ..... @qrr_sd + +FSQRT_v 0.10 1110 111 11001 11111 0 ..... ..... @qrr_h +FSQRT_v 0.10 1110 1.1 00001 11111 0 ..... ..... @qrr_sd diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static bool do_fabs_fneg_v(DisasContext *s, arg_qrr_e *a, GVecGen2Fn *fn) TRANS(FABS_v, do_fabs_fneg_v, a, gen_gvec_fabs) TRANS(FNEG_v, do_fabs_fneg_v, a, gen_gvec_fneg) +static bool do_fp1_vector(DisasContext *s, arg_qrr_e *a, + const FPScalar1 *f, int rmode) +{ + TCGv_i32 tcg_rmode = NULL; + TCGv_ptr fpst; + int check = fp_access_check_vector_hsd(s, a->q, a->esz); + + if (check <= 0) { + return check == 0; + } + + fpst = fpstatus_ptr(a->esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR); + if (rmode >= 0) { + tcg_rmode = gen_set_rmode(rmode, fpst); + } + + if (a->esz == MO_64) { + TCGv_i64 t64 = tcg_temp_new_i64(); + + for (int pass = 0; pass < 2; ++pass) { + read_vec_element(s, t64, a->rn, pass, MO_64); + f->gen_d(t64, t64, fpst); + write_vec_element(s, t64, a->rd, pass, MO_64); + } + } else { + TCGv_i32 t32 = tcg_temp_new_i32(); + void (*gen)(TCGv_i32, TCGv_i32, TCGv_ptr) + = (a->esz == MO_16 ? f->gen_h : f->gen_s); + + for (int pass = 0, n = (a->q ? 16 : 8) >> a->esz; pass < n; ++pass) { + read_vec_element_i32(s, t32, a->rn, pass, a->esz); + gen(t32, t32, fpst); + write_vec_element_i32(s, t32, a->rd, pass, a->esz); + } + } + clear_vec_high(s, a->q, a->rd); + + if (rmode >= 0) { + gen_restore_rmode(tcg_rmode, fpst); + } + return true; +} + +TRANS(FSQRT_v, do_fp1_vector, a, &f_scalar_fsqrt, -1) + /* Common vector code for handling integer to FP conversion */ static void handle_simd_intfp_conv(DisasContext *s, int rd, int rn, int elements, int is_signed, @@ -XXX,XX +XXX,XX @@ static void handle_2misc_64(DisasContext *s, int opcode, bool u, * requires them. */ switch (opcode) { - case 0x7f: /* FSQRT */ - gen_helper_vfp_sqrtd(tcg_rd, tcg_rn, tcg_fpstatus); - break; case 0x1a: /* FCVTNS */ case 0x1b: /* FCVTMS */ case 0x1c: /* FCVTAS */ @@ -XXX,XX +XXX,XX @@ static void handle_2misc_64(DisasContext *s, int opcode, bool u, case 0xb: /* ABS, NEG */ case 0x2f: /* FABS */ case 0x6f: /* FNEG */ + case 0x7f: /* FSQRT */ g_assert_not_reached(); } } @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) } handle_2misc_fcmp_zero(s, opcode, false, u, is_q, size, rn, rd); return; - case 0x7f: /* FSQRT */ - need_fpstatus = true; - if (size == 3 && !is_q) { - unallocated_encoding(s); - return; - } - break; case 0x1a: /* FCVTNS */ case 0x1b: /* FCVTMS */ case 0x3a: /* FCVTPS */ @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) case 0x56: /* FCVTXN, FCVTXN2 */ case 0x2f: /* FABS */ case 0x6f: /* FNEG */ + case 0x7f: /* FSQRT */ unallocated_encoding(s); return; } @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) { /* Special cases for 32 bit elements */ switch (opcode) { - case 0x7f: /* FSQRT */ - gen_helper_vfp_sqrts(tcg_res, tcg_op, tcg_fpstatus); - break; case 0x1a: /* FCVTNS */ case 0x1b: /* FCVTMS */ case 0x1c: /* FCVTAS */ @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) case 0x7: /* SQABS, SQNEG */ case 0x2f: /* FABS */ case 0x6f: /* FNEG */ + case 0x7f: /* FSQRT */ g_assert_not_reached(); } } @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc_fp16(DisasContext *s, uint32_t insn) break; case 0x7d: /* FRSQRTE */ break; - case 0x7f: /* FSQRT (vector) */ - only_in_vector = true; - break; default: case 0x2f: /* FABS */ case 0x6f: /* FNEG */ + case 0x7f: /* FSQRT (vector) */ unallocated_encoding(s); return; } @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc_fp16(DisasContext *s, uint32_t insn) case 0x7d: /* FRSQRTE */ gen_helper_rsqrte_f16(tcg_res, tcg_op, tcg_fpstatus); break; - case 0x7f: /* FSQRT */ - gen_helper_vfp_sqrth(tcg_res, tcg_op, tcg_fpstatus); - break; default: case 0x2f: /* FABS */ case 0x6f: /* FNEG */ + case 0x7f: /* FSQRT */ g_assert_not_reached(); } -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-57-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 26 +++++ target/arm/tcg/translate-a64.c | 176 ++++++++++++--------------------- 2 files changed, 88 insertions(+), 114 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ FNEG_v 0.10 1110 1.1 00000 11111 0 ..... ..... @qrr_sd FSQRT_v 0.10 1110 111 11001 11111 0 ..... ..... @qrr_h FSQRT_v 0.10 1110 1.1 00001 11111 0 ..... ..... @qrr_sd + +FRINTN_v 0.00 1110 011 11001 10001 0 ..... ..... @qrr_h +FRINTN_v 0.00 1110 0.1 00001 10001 0 ..... ..... @qrr_sd + +FRINTM_v 0.00 1110 011 11001 10011 0 ..... ..... @qrr_h +FRINTM_v 0.00 1110 0.1 00001 10011 0 ..... ..... @qrr_sd + +FRINTP_v 0.00 1110 111 11001 10001 0 ..... ..... @qrr_h +FRINTP_v 0.00 1110 1.1 00001 10001 0 ..... ..... @qrr_sd + +FRINTZ_v 0.00 1110 111 11001 10011 0 ..... ..... @qrr_h +FRINTZ_v 0.00 1110 1.1 00001 10011 0 ..... ..... @qrr_sd + +FRINTA_v 0.10 1110 011 11001 10001 0 ..... ..... @qrr_h +FRINTA_v 0.10 1110 0.1 00001 10001 0 ..... ..... @qrr_sd + +FRINTX_v 0.10 1110 011 11001 10011 0 ..... ..... @qrr_h +FRINTX_v 0.10 1110 0.1 00001 10011 0 ..... ..... @qrr_sd + +FRINTI_v 0.10 1110 111 11001 10011 0 ..... ..... @qrr_h +FRINTI_v 0.10 1110 1.1 00001 10011 0 ..... ..... @qrr_sd + +FRINT32Z_v 0.00 1110 0.1 00001 11101 0 ..... ..... @qrr_sd +FRINT32X_v 0.10 1110 0.1 00001 11101 0 ..... ..... @qrr_sd +FRINT64Z_v 0.00 1110 0.1 00001 11111 0 ..... ..... @qrr_sd +FRINT64X_v 0.10 1110 0.1 00001 11111 0 ..... ..... @qrr_sd diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static bool do_fp1_vector(DisasContext *s, arg_qrr_e *a, TRANS(FSQRT_v, do_fp1_vector, a, &f_scalar_fsqrt, -1) +TRANS(FRINTN_v, do_fp1_vector, a, &f_scalar_frint, FPROUNDING_TIEEVEN) +TRANS(FRINTP_v, do_fp1_vector, a, &f_scalar_frint, FPROUNDING_POSINF) +TRANS(FRINTM_v, do_fp1_vector, a, &f_scalar_frint, FPROUNDING_NEGINF) +TRANS(FRINTZ_v, do_fp1_vector, a, &f_scalar_frint, FPROUNDING_ZERO) +TRANS(FRINTA_v, do_fp1_vector, a, &f_scalar_frint, FPROUNDING_TIEAWAY) +TRANS(FRINTI_v, do_fp1_vector, a, &f_scalar_frint, -1) +TRANS(FRINTX_v, do_fp1_vector, a, &f_scalar_frintx, -1) + +TRANS_FEAT(FRINT32Z_v, aa64_frint, do_fp1_vector, a, + &f_scalar_frint32, FPROUNDING_ZERO) +TRANS_FEAT(FRINT32X_v, aa64_frint, do_fp1_vector, a, &f_scalar_frint32, -1) +TRANS_FEAT(FRINT64Z_v, aa64_frint, do_fp1_vector, a, + &f_scalar_frint64, FPROUNDING_ZERO) +TRANS_FEAT(FRINT64X_v, aa64_frint, do_fp1_vector, a, &f_scalar_frint64, -1) + /* Common vector code for handling integer to FP conversion */ static void handle_simd_intfp_conv(DisasContext *s, int rd, int rn, int elements, int is_signed, @@ -XXX,XX +XXX,XX @@ static void handle_2misc_64(DisasContext *s, int opcode, bool u, case 0x7b: /* FCVTZU */ gen_helper_vfp_touqd(tcg_rd, tcg_rn, tcg_constant_i32(0), tcg_fpstatus); break; - case 0x18: /* FRINTN */ - case 0x19: /* FRINTM */ - case 0x38: /* FRINTP */ - case 0x39: /* FRINTZ */ - case 0x58: /* FRINTA */ - case 0x79: /* FRINTI */ - gen_helper_rintd(tcg_rd, tcg_rn, tcg_fpstatus); - break; - case 0x59: /* FRINTX */ - gen_helper_rintd_exact(tcg_rd, tcg_rn, tcg_fpstatus); - break; - case 0x1e: /* FRINT32Z */ - case 0x5e: /* FRINT32X */ - gen_helper_frint32_d(tcg_rd, tcg_rn, tcg_fpstatus); - break; - case 0x1f: /* FRINT64Z */ - case 0x5f: /* FRINT64X */ - gen_helper_frint64_d(tcg_rd, tcg_rn, tcg_fpstatus); - break; default: case 0x4: /* CLS, CLZ */ case 0x5: /* NOT */ @@ -XXX,XX +XXX,XX @@ static void handle_2misc_64(DisasContext *s, int opcode, bool u, case 0x2f: /* FABS */ case 0x6f: /* FNEG */ case 0x7f: /* FSQRT */ + case 0x18: /* FRINTN */ + case 0x19: /* FRINTM */ + case 0x38: /* FRINTP */ + case 0x39: /* FRINTZ */ + case 0x58: /* FRINTA */ + case 0x79: /* FRINTI */ + case 0x59: /* FRINTX */ + case 0x1e: /* FRINT32Z */ + case 0x5e: /* FRINT32X */ + case 0x1f: /* FRINT64Z */ + case 0x5f: /* FRINT64X */ g_assert_not_reached(); } } @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) } handle_2misc_widening(s, opcode, is_q, size, rn, rd); return; - case 0x18: /* FRINTN */ - case 0x19: /* FRINTM */ - case 0x38: /* FRINTP */ - case 0x39: /* FRINTZ */ - rmode = extract32(opcode, 5, 1) | (extract32(opcode, 0, 1) << 1); - /* fall through */ - case 0x59: /* FRINTX */ - case 0x79: /* FRINTI */ - need_fpstatus = true; - if (size == 3 && !is_q) { - unallocated_encoding(s); - return; - } - break; - case 0x58: /* FRINTA */ - rmode = FPROUNDING_TIEAWAY; - need_fpstatus = true; - if (size == 3 && !is_q) { - unallocated_encoding(s); - return; - } - break; case 0x7c: /* URSQRTE */ if (size == 3) { unallocated_encoding(s); return; } break; - case 0x1e: /* FRINT32Z */ - case 0x1f: /* FRINT64Z */ - rmode = FPROUNDING_ZERO; - /* fall through */ - case 0x5e: /* FRINT32X */ - case 0x5f: /* FRINT64X */ - need_fpstatus = true; - if ((size == 3 && !is_q) || !dc_isar_feature(aa64_frint, s)) { - unallocated_encoding(s); - return; - } - break; default: case 0x16: /* FCVTN, FCVTN2 */ case 0x36: /* BFCVTN, BFCVTN2 */ @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) case 0x2f: /* FABS */ case 0x6f: /* FNEG */ case 0x7f: /* FSQRT */ + case 0x18: /* FRINTN */ + case 0x19: /* FRINTM */ + case 0x38: /* FRINTP */ + case 0x39: /* FRINTZ */ + case 0x59: /* FRINTX */ + case 0x79: /* FRINTI */ + case 0x58: /* FRINTA */ + case 0x1e: /* FRINT32Z */ + case 0x1f: /* FRINT64Z */ + case 0x5e: /* FRINT32X */ + case 0x5f: /* FRINT64X */ unallocated_encoding(s); return; } @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) gen_helper_vfp_touls(tcg_res, tcg_op, tcg_constant_i32(0), tcg_fpstatus); break; - case 0x18: /* FRINTN */ - case 0x19: /* FRINTM */ - case 0x38: /* FRINTP */ - case 0x39: /* FRINTZ */ - case 0x58: /* FRINTA */ - case 0x79: /* FRINTI */ - gen_helper_rints(tcg_res, tcg_op, tcg_fpstatus); - break; - case 0x59: /* FRINTX */ - gen_helper_rints_exact(tcg_res, tcg_op, tcg_fpstatus); - break; case 0x7c: /* URSQRTE */ gen_helper_rsqrte_u32(tcg_res, tcg_op); break; - case 0x1e: /* FRINT32Z */ - case 0x5e: /* FRINT32X */ - gen_helper_frint32_s(tcg_res, tcg_op, tcg_fpstatus); - break; - case 0x1f: /* FRINT64Z */ - case 0x5f: /* FRINT64X */ - gen_helper_frint64_s(tcg_res, tcg_op, tcg_fpstatus); - break; default: case 0x7: /* SQABS, SQNEG */ case 0x2f: /* FABS */ case 0x6f: /* FNEG */ case 0x7f: /* FSQRT */ + case 0x18: /* FRINTN */ + case 0x19: /* FRINTM */ + case 0x38: /* FRINTP */ + case 0x39: /* FRINTZ */ + case 0x58: /* FRINTA */ + case 0x79: /* FRINTI */ + case 0x59: /* FRINTX */ + case 0x1e: /* FRINT32Z */ + case 0x5e: /* FRINT32X */ + case 0x1f: /* FRINT64Z */ + case 0x5f: /* FRINT64X */ g_assert_not_reached(); } } @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc_fp16(DisasContext *s, uint32_t insn) int rn, rd; bool is_q; bool is_scalar; - bool only_in_vector = false; int pass; TCGv_i32 tcg_rmode = NULL; @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc_fp16(DisasContext *s, uint32_t insn) case 0x3d: /* FRECPE */ case 0x3f: /* FRECPX */ break; - case 0x18: /* FRINTN */ - only_in_vector = true; - rmode = FPROUNDING_TIEEVEN; - break; - case 0x19: /* FRINTM */ - only_in_vector = true; - rmode = FPROUNDING_NEGINF; - break; - case 0x38: /* FRINTP */ - only_in_vector = true; - rmode = FPROUNDING_POSINF; - break; - case 0x39: /* FRINTZ */ - only_in_vector = true; - rmode = FPROUNDING_ZERO; - break; - case 0x58: /* FRINTA */ - only_in_vector = true; - rmode = FPROUNDING_TIEAWAY; - break; - case 0x59: /* FRINTX */ - case 0x79: /* FRINTI */ - only_in_vector = true; - /* current rounding mode */ - break; case 0x1a: /* FCVTNS */ rmode = FPROUNDING_TIEEVEN; break; @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc_fp16(DisasContext *s, uint32_t insn) case 0x2f: /* FABS */ case 0x6f: /* FNEG */ case 0x7f: /* FSQRT (vector) */ + case 0x18: /* FRINTN */ + case 0x19: /* FRINTM */ + case 0x38: /* FRINTP */ + case 0x39: /* FRINTZ */ + case 0x58: /* FRINTA */ + case 0x59: /* FRINTX */ + case 0x79: /* FRINTI */ unallocated_encoding(s); return; } @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc_fp16(DisasContext *s, uint32_t insn) unallocated_encoding(s); return; } - /* FRINTxx is only in the vector form */ - if (only_in_vector) { - unallocated_encoding(s); - return; - } } if (!fp_access_check(s)) { @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc_fp16(DisasContext *s, uint32_t insn) case 0x7b: /* FCVTZU */ gen_helper_advsimd_f16touinth(tcg_res, tcg_op, tcg_fpstatus); break; - case 0x18: /* FRINTN */ - case 0x19: /* FRINTM */ - case 0x38: /* FRINTP */ - case 0x39: /* FRINTZ */ - case 0x58: /* FRINTA */ - case 0x79: /* FRINTI */ - gen_helper_advsimd_rinth(tcg_res, tcg_op, tcg_fpstatus); - break; - case 0x59: /* FRINTX */ - gen_helper_advsimd_rinth_exact(tcg_res, tcg_op, tcg_fpstatus); - break; case 0x7d: /* FRSQRTE */ gen_helper_rsqrte_f16(tcg_res, tcg_op, tcg_fpstatus); break; @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc_fp16(DisasContext *s, uint32_t insn) case 0x2f: /* FABS */ case 0x6f: /* FNEG */ case 0x7f: /* FSQRT */ + case 0x18: /* FRINTN */ + case 0x19: /* FRINTM */ + case 0x38: /* FRINTP */ + case 0x39: /* FRINTZ */ + case 0x58: /* FRINTA */ + case 0x79: /* FRINTI */ + case 0x59: /* FRINTX */ g_assert_not_reached(); } -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Arm silliness with naming, the scalar insns described as part of the vector instructions, as separate from the "regular" scalar insns which output to general registers. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-58-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 30 ++++++++ target/arm/tcg/translate-a64.c | 133 ++++++++++++++------------------- 2 files changed, 86 insertions(+), 77 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ UQXTN_s 0111 1110 ..1 00001 01001 0 ..... ..... @rr_e FCVTXN_s 0111 1110 011 00001 01101 0 ..... ..... @rr_s +@icvt_h . ....... .. ...... ...... rn:5 rd:5 \ + &fcvt sf=0 esz=1 shift=0 +@icvt_sd . ....... .. ...... ...... rn:5 rd:5 \ + &fcvt sf=0 esz=%esz_sd shift=0 + +FCVTNS_f 0101 1110 011 11001 10101 0 ..... ..... @icvt_h +FCVTNS_f 0101 1110 0.1 00001 10101 0 ..... ..... @icvt_sd +FCVTNU_f 0111 1110 011 11001 10101 0 ..... ..... @icvt_h +FCVTNU_f 0111 1110 0.1 00001 10101 0 ..... ..... @icvt_sd + +FCVTPS_f 0101 1110 111 11001 10101 0 ..... ..... @icvt_h +FCVTPS_f 0101 1110 1.1 00001 10101 0 ..... ..... @icvt_sd +FCVTPU_f 0111 1110 111 11001 10101 0 ..... ..... @icvt_h +FCVTPU_f 0111 1110 1.1 00001 10101 0 ..... ..... @icvt_sd + +FCVTMS_f 0101 1110 011 11001 10111 0 ..... ..... @icvt_h +FCVTMS_f 0101 1110 0.1 00001 10111 0 ..... ..... @icvt_sd +FCVTMU_f 0111 1110 011 11001 10111 0 ..... ..... @icvt_h +FCVTMU_f 0111 1110 0.1 00001 10111 0 ..... ..... @icvt_sd + +FCVTZS_f 0101 1110 111 11001 10111 0 ..... ..... @icvt_h +FCVTZS_f 0101 1110 1.1 00001 10111 0 ..... ..... @icvt_sd +FCVTZU_f 0111 1110 111 11001 10111 0 ..... ..... @icvt_h +FCVTZU_f 0111 1110 1.1 00001 10111 0 ..... ..... @icvt_sd + +FCVTAS_f 0101 1110 011 11001 11001 0 ..... ..... @icvt_h +FCVTAS_f 0101 1110 0.1 00001 11001 0 ..... ..... @icvt_sd +FCVTAU_f 0111 1110 011 11001 11001 0 ..... ..... @icvt_h +FCVTAU_f 0111 1110 0.1 00001 11001 0 ..... ..... @icvt_sd + # Advanced SIMD two-register miscellaneous SQABS_v 0.00 1110 ..1 00000 01111 0 ..... ..... @qrr_e diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static void do_fcvt_scalar(DisasContext *s, MemOp out, MemOp esz, tcg_shift, tcg_fpstatus); tcg_gen_extu_i32_i64(tcg_out, tcg_single); break; + case MO_16 | MO_SIGN: + gen_helper_vfp_toshh(tcg_single, tcg_single, + tcg_shift, tcg_fpstatus); + tcg_gen_extu_i32_i64(tcg_out, tcg_single); + break; + case MO_16: + gen_helper_vfp_touhh(tcg_single, tcg_single, + tcg_shift, tcg_fpstatus); + tcg_gen_extu_i32_i64(tcg_out, tcg_single); + break; default: g_assert_not_reached(); } @@ -XXX,XX +XXX,XX @@ TRANS(FCVTZU_g, do_fcvt_g, a, FPROUNDING_ZERO, false) TRANS(FCVTAS_g, do_fcvt_g, a, FPROUNDING_TIEAWAY, true) TRANS(FCVTAU_g, do_fcvt_g, a, FPROUNDING_TIEAWAY, false) +/* + * FCVT* (vector), scalar version. + * Which sounds weird, but really just means output to fp register + * instead of output to general register. Input and output element + * size are always equal. + */ +static bool do_fcvt_f(DisasContext *s, arg_fcvt *a, + ARMFPRounding rmode, bool is_signed) +{ + TCGv_i64 tcg_int; + int check = fp_access_check_scalar_hsd(s, a->esz); + + if (check <= 0) { + return check == 0; + } + + tcg_int = tcg_temp_new_i64(); + do_fcvt_scalar(s, a->esz | (is_signed ? MO_SIGN : 0), + a->esz, tcg_int, a->shift, a->rn, rmode); + + clear_vec(s, a->rd); + write_vec_element(s, tcg_int, a->rd, 0, a->esz); + return true; +} + +TRANS(FCVTNS_f, do_fcvt_f, a, FPROUNDING_TIEEVEN, true) +TRANS(FCVTNU_f, do_fcvt_f, a, FPROUNDING_TIEEVEN, false) +TRANS(FCVTPS_f, do_fcvt_f, a, FPROUNDING_POSINF, true) +TRANS(FCVTPU_f, do_fcvt_f, a, FPROUNDING_POSINF, false) +TRANS(FCVTMS_f, do_fcvt_f, a, FPROUNDING_NEGINF, true) +TRANS(FCVTMU_f, do_fcvt_f, a, FPROUNDING_NEGINF, false) +TRANS(FCVTZS_f, do_fcvt_f, a, FPROUNDING_ZERO, true) +TRANS(FCVTZU_f, do_fcvt_f, a, FPROUNDING_ZERO, false) +TRANS(FCVTAS_f, do_fcvt_f, a, FPROUNDING_TIEAWAY, true) +TRANS(FCVTAU_f, do_fcvt_f, a, FPROUNDING_TIEAWAY, false) + static bool trans_FJCVTZS(DisasContext *s, arg_FJCVTZS *a) { if (!dc_isar_feature(aa64_jscvt, s)) { @@ -XXX,XX +XXX,XX @@ static void disas_simd_scalar_two_reg_misc(DisasContext *s, uint32_t insn) int opcode = extract32(insn, 12, 5); int size = extract32(insn, 22, 2); bool u = extract32(insn, 29, 1); - bool is_fcvt = false; - int rmode; - TCGv_i32 tcg_rmode; - TCGv_ptr tcg_fpstatus; switch (opcode) { case 0xc ... 0xf: @@ -XXX,XX +XXX,XX @@ static void disas_simd_scalar_two_reg_misc(DisasContext *s, uint32_t insn) case 0x5b: /* FCVTMU */ case 0x7a: /* FCVTPU */ case 0x7b: /* FCVTZU */ - is_fcvt = true; - rmode = extract32(opcode, 5, 1) | (extract32(opcode, 0, 1) << 1); - break; case 0x1c: /* FCVTAS */ case 0x5c: /* FCVTAU */ - /* TIEAWAY doesn't fit in the usual rounding mode encoding */ - is_fcvt = true; - rmode = FPROUNDING_TIEAWAY; - break; case 0x56: /* FCVTXN, FCVTXN2 */ default: unallocated_encoding(s); @@ -XXX,XX +XXX,XX @@ static void disas_simd_scalar_two_reg_misc(DisasContext *s, uint32_t insn) unallocated_encoding(s); return; } - - if (!fp_access_check(s)) { - return; - } - - if (is_fcvt) { - tcg_fpstatus = fpstatus_ptr(FPST_FPCR); - tcg_rmode = gen_set_rmode(rmode, tcg_fpstatus); - } else { - tcg_fpstatus = NULL; - tcg_rmode = NULL; - } - - if (size == 3) { - TCGv_i64 tcg_rn = read_fp_dreg(s, rn); - TCGv_i64 tcg_rd = tcg_temp_new_i64(); - - handle_2misc_64(s, opcode, u, tcg_rd, tcg_rn, tcg_rmode, tcg_fpstatus); - write_fp_dreg(s, rd, tcg_rd); - } else { - TCGv_i32 tcg_rn = tcg_temp_new_i32(); - TCGv_i32 tcg_rd = tcg_temp_new_i32(); - - read_vec_element_i32(s, tcg_rn, rn, 0, size); - - switch (opcode) { - case 0x1a: /* FCVTNS */ - case 0x1b: /* FCVTMS */ - case 0x1c: /* FCVTAS */ - case 0x3a: /* FCVTPS */ - case 0x3b: /* FCVTZS */ - gen_helper_vfp_tosls(tcg_rd, tcg_rn, tcg_constant_i32(0), - tcg_fpstatus); - break; - case 0x5a: /* FCVTNU */ - case 0x5b: /* FCVTMU */ - case 0x5c: /* FCVTAU */ - case 0x7a: /* FCVTPU */ - case 0x7b: /* FCVTZU */ - gen_helper_vfp_touls(tcg_rd, tcg_rn, tcg_constant_i32(0), - tcg_fpstatus); - break; - default: - case 0x7: /* SQABS, SQNEG */ - g_assert_not_reached(); - } - - write_fp_sreg(s, rd, tcg_rd); - } - - if (is_fcvt) { - gen_restore_rmode(tcg_rmode, tcg_fpstatus); - } + g_assert_not_reached(); } /* AdvSIMD shift by immediate @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc_fp16(DisasContext *s, uint32_t insn) TCGv_i32 tcg_res = tcg_temp_new_i32(); switch (fpop) { - case 0x1a: /* FCVTNS */ - case 0x1b: /* FCVTMS */ - case 0x1c: /* FCVTAS */ - case 0x3a: /* FCVTPS */ - case 0x3b: /* FCVTZS */ - gen_helper_advsimd_f16tosinth(tcg_res, tcg_op, tcg_fpstatus); - break; case 0x3d: /* FRECPE */ gen_helper_recpe_f16(tcg_res, tcg_op, tcg_fpstatus); break; case 0x3f: /* FRECPX */ gen_helper_frecpx_f16(tcg_res, tcg_op, tcg_fpstatus); break; + case 0x7d: /* FRSQRTE */ + gen_helper_rsqrte_f16(tcg_res, tcg_op, tcg_fpstatus); + break; + default: + case 0x1a: /* FCVTNS */ + case 0x1b: /* FCVTMS */ + case 0x1c: /* FCVTAS */ + case 0x3a: /* FCVTPS */ + case 0x3b: /* FCVTZS */ case 0x5a: /* FCVTNU */ case 0x5b: /* FCVTMU */ case 0x5c: /* FCVTAU */ case 0x7a: /* FCVTPU */ case 0x7b: /* FCVTZU */ - gen_helper_advsimd_f16touinth(tcg_res, tcg_op, tcg_fpstatus); - break; - case 0x7d: /* FRSQRTE */ - gen_helper_rsqrte_f16(tcg_res, tcg_op, tcg_fpstatus); - break; - default: g_assert_not_reached(); } -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-59-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 19 +++++++++++++++++++ target/arm/tcg/translate-a64.c | 4 +--- 2 files changed, 20 insertions(+), 3 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ FCVTAS_f 0101 1110 0.1 00001 11001 0 ..... ..... @icvt_sd FCVTAU_f 0111 1110 011 11001 11001 0 ..... ..... @icvt_h FCVTAU_f 0111 1110 0.1 00001 11001 0 ..... ..... @icvt_sd +%fcvt_f_sh_h 16:4 !function=rsub_16 +%fcvt_f_sh_s 16:5 !function=rsub_32 +%fcvt_f_sh_d 16:6 !function=rsub_64 + +@fcvt_fixed_h .... .... . 001 .... ...... rn:5 rd:5 \ + &fcvt sf=0 esz=1 shift=%fcvt_f_sh_h +@fcvt_fixed_s .... .... . 01 ..... ...... rn:5 rd:5 \ + &fcvt sf=0 esz=2 shift=%fcvt_f_sh_s +@fcvt_fixed_d .... .... . 1 ...... ...... rn:5 rd:5 \ + &fcvt sf=0 esz=3 shift=%fcvt_f_sh_d + +FCVTZS_f 0101 1111 0 ....... 111111 ..... ..... @fcvt_fixed_h +FCVTZS_f 0101 1111 0 ....... 111111 ..... ..... @fcvt_fixed_s +FCVTZS_f 0101 1111 0 ....... 111111 ..... ..... @fcvt_fixed_d + +FCVTZU_f 0111 1111 0 ....... 111111 ..... ..... @fcvt_fixed_h +FCVTZU_f 0111 1111 0 ....... 111111 ..... ..... @fcvt_fixed_s +FCVTZU_f 0111 1111 0 ....... 111111 ..... ..... @fcvt_fixed_d + # Advanced SIMD two-register miscellaneous SQABS_v 0.00 1110 ..1 00000 01111 0 ..... ..... @qrr_e diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static void disas_simd_scalar_shift_imm(DisasContext *s, uint32_t insn) handle_simd_shift_intfp_conv(s, true, false, is_u, immh, immb, opcode, rn, rd); break; - case 0x1f: /* FCVTZS, FCVTZU */ - handle_simd_shift_fpint_conv(s, true, false, is_u, immh, immb, rn, rd); - break; default: case 0x00: /* SSHR / USHR */ case 0x02: /* SSRA / USRA */ @@ -XXX,XX +XXX,XX @@ static void disas_simd_scalar_shift_imm(DisasContext *s, uint32_t insn) case 0x11: /* SQRSHRUN */ case 0x12: /* SQSHRN, UQSHRN */ case 0x13: /* SQRSHRN, UQRSHRN */ + case 0x1f: /* FCVTZS, FCVTZU */ unallocated_encoding(s); break; } -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-60-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 6 ++++++ target/arm/tcg/translate-a64.c | 35 ++++++++++++++++++++++++---------- 2 files changed, 31 insertions(+), 10 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ FCVTXN_s 0111 1110 011 00001 01101 0 ..... ..... @rr_s @icvt_sd . ....... .. ...... ...... rn:5 rd:5 \ &fcvt sf=0 esz=%esz_sd shift=0 +SCVTF_f 0101 1110 011 11001 11011 0 ..... ..... @icvt_h +SCVTF_f 0101 1110 0.1 00001 11011 0 ..... ..... @icvt_sd + +UCVTF_f 0111 1110 011 11001 11011 0 ..... ..... @icvt_h +UCVTF_f 0111 1110 0.1 00001 11011 0 ..... ..... @icvt_sd + FCVTNS_f 0101 1110 011 11001 10101 0 ..... ..... @icvt_h FCVTNS_f 0101 1110 0.1 00001 10101 0 ..... ..... @icvt_sd FCVTNU_f 0111 1110 011 11001 10101 0 ..... ..... @icvt_h diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static bool do_cvtf_g(DisasContext *s, arg_fcvt *a, bool is_signed) TRANS(SCVTF_g, do_cvtf_g, a, true) TRANS(UCVTF_g, do_cvtf_g, a, false) +/* + * [US]CVTF (vector), scalar version. + * Which sounds weird, but really just means input from fp register + * instead of input from general register. Input and output element + * size are always equal. + */ +static bool do_cvtf_f(DisasContext *s, arg_fcvt *a, bool is_signed) +{ + TCGv_i64 tcg_int; + int check = fp_access_check_scalar_hsd(s, a->esz); + + if (check <= 0) { + return check == 0; + } + + tcg_int = tcg_temp_new_i64(); + read_vec_element(s, tcg_int, a->rn, 0, a->esz | (is_signed ? MO_SIGN : 0)); + return do_cvtf_scalar(s, a->esz, a->rd, a->shift, tcg_int, is_signed); +} + +TRANS(SCVTF_f, do_cvtf_f, a, true) +TRANS(UCVTF_f, do_cvtf_f, a, false) + static void do_fcvt_scalar(DisasContext *s, MemOp out, MemOp esz, TCGv_i64 tcg_out, int shift, int rn, ARMFPRounding rmode) @@ -XXX,XX +XXX,XX @@ static void disas_simd_scalar_two_reg_misc(DisasContext *s, uint32_t insn) case 0x6d: /* FCMLE (zero) */ handle_2misc_fcmp_zero(s, opcode, true, u, true, size, rn, rd); return; - case 0x1d: /* SCVTF */ - case 0x5d: /* UCVTF */ - { - bool is_signed = (opcode == 0x1d); - if (!fp_access_check(s)) { - return; - } - handle_simd_intfp_conv(s, rd, rn, 1, is_signed, 0, size); - return; - } case 0x3d: /* FRECPE */ case 0x3f: /* FRECPX */ case 0x7d: /* FRSQRTE */ @@ -XXX,XX +XXX,XX @@ static void disas_simd_scalar_two_reg_misc(DisasContext *s, uint32_t insn) case 0x1c: /* FCVTAS */ case 0x5c: /* FCVTAU */ case 0x56: /* FCVTXN, FCVTXN2 */ + case 0x1d: /* SCVTF */ + case 0x5d: /* UCVTF */ default: unallocated_encoding(s); return; -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Remove disas_simd_scalar_shift_imm as these were the last insns decoded by that function. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-61-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 8 ++++++ target/arm/tcg/translate-a64.c | 47 ---------------------------------- 2 files changed, 8 insertions(+), 47 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ FCVTAU_f 0111 1110 0.1 00001 11001 0 ..... ..... @icvt_sd @fcvt_fixed_d .... .... . 1 ...... ...... rn:5 rd:5 \ &fcvt sf=0 esz=3 shift=%fcvt_f_sh_d +SCVTF_f 0101 1111 0 ....... 111001 ..... ..... @fcvt_fixed_h +SCVTF_f 0101 1111 0 ....... 111001 ..... ..... @fcvt_fixed_s +SCVTF_f 0101 1111 0 ....... 111001 ..... ..... @fcvt_fixed_d + +UCVTF_f 0111 1111 0 ....... 111001 ..... ..... @fcvt_fixed_h +UCVTF_f 0111 1111 0 ....... 111001 ..... ..... @fcvt_fixed_s +UCVTF_f 0111 1111 0 ....... 111001 ..... ..... @fcvt_fixed_d + FCVTZS_f 0101 1111 0 ....... 111111 ..... ..... @fcvt_fixed_h FCVTZS_f 0101 1111 0 ....... 111111 ..... ..... @fcvt_fixed_s FCVTZS_f 0101 1111 0 ....... 111111 ..... ..... @fcvt_fixed_d diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static void handle_simd_shift_fpint_conv(DisasContext *s, bool is_scalar, gen_restore_rmode(tcg_rmode, tcg_fpstatus); } -/* AdvSIMD scalar shift by immediate - * 31 30 29 28 23 22 19 18 16 15 11 10 9 5 4 0 - * +-----+---+-------------+------+------+--------+---+------+------+ - * | 0 1 | U | 1 1 1 1 1 0 | immh | immb | opcode | 1 | Rn | Rd | - * +-----+---+-------------+------+------+--------+---+------+------+ - * - * This is the scalar version so it works on a fixed sized registers - */ -static void disas_simd_scalar_shift_imm(DisasContext *s, uint32_t insn) -{ - int rd = extract32(insn, 0, 5); - int rn = extract32(insn, 5, 5); - int opcode = extract32(insn, 11, 5); - int immb = extract32(insn, 16, 3); - int immh = extract32(insn, 19, 4); - bool is_u = extract32(insn, 29, 1); - - if (immh == 0) { - unallocated_encoding(s); - return; - } - - switch (opcode) { - case 0x1c: /* SCVTF, UCVTF */ - handle_simd_shift_intfp_conv(s, true, false, is_u, immh, immb, - opcode, rn, rd); - break; - default: - case 0x00: /* SSHR / USHR */ - case 0x02: /* SSRA / USRA */ - case 0x04: /* SRSHR / URSHR */ - case 0x06: /* SRSRA / URSRA */ - case 0x08: /* SRI */ - case 0x0a: /* SHL / SLI */ - case 0x0c: /* SQSHLU */ - case 0x0e: /* SQSHL, UQSHL */ - case 0x10: /* SQSHRUN */ - case 0x11: /* SQRSHRUN */ - case 0x12: /* SQSHRN, UQSHRN */ - case 0x13: /* SQRSHRN, UQRSHRN */ - case 0x1f: /* FCVTZS, FCVTZU */ - unallocated_encoding(s); - break; - } -} - static void handle_2misc_64(DisasContext *s, int opcode, bool u, TCGv_i64 tcg_rd, TCGv_i64 tcg_rn, TCGv_i32 tcg_rmode, TCGv_ptr tcg_fpstatus) @@ -XXX,XX +XXX,XX @@ static const AArch64DecodeTable data_proc_simd[] = { { 0x0e200800, 0x9f3e0c00, disas_simd_two_reg_misc }, { 0x0f000400, 0x9f800400, disas_simd_shift_imm }, { 0x5e200800, 0xdf3e0c00, disas_simd_scalar_two_reg_misc }, - { 0x5f000400, 0xdf800400, disas_simd_scalar_shift_imm }, { 0x0e780800, 0x8f7e0c00, disas_simd_two_reg_misc_fp16 }, { 0x00000000, 0x00000000, NULL } }; -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Emphasize that these functions use round-to-zero mode. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-62-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/helper.h | 8 ++++---- target/arm/tcg/translate-neon.c | 8 ++++---- target/arm/tcg/vec_helper.c | 8 ++++---- 3 files changed, 12 insertions(+), 12 deletions(-) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_4(gvec_touizs, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_vcvt_sf, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_vcvt_uf, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(gvec_vcvt_fs, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(gvec_vcvt_fu, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_vcvt_rz_fs, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_vcvt_rz_fu, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_vcvt_sh, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_vcvt_uh, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(gvec_vcvt_hs, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(gvec_vcvt_hu, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_vcvt_rz_hs, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_vcvt_rz_hu, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_vcvt_rm_ss, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_vcvt_rm_us, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) diff --git a/target/arm/tcg/translate-neon.c b/target/arm/tcg/translate-neon.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-neon.c +++ b/target/arm/tcg/translate-neon.c @@ -XXX,XX +XXX,XX @@ static bool do_fp_2sh(DisasContext *s, arg_2reg_shift *a, DO_FP_2SH(VCVT_SF, gen_helper_gvec_vcvt_sf) DO_FP_2SH(VCVT_UF, gen_helper_gvec_vcvt_uf) -DO_FP_2SH(VCVT_FS, gen_helper_gvec_vcvt_fs) -DO_FP_2SH(VCVT_FU, gen_helper_gvec_vcvt_fu) +DO_FP_2SH(VCVT_FS, gen_helper_gvec_vcvt_rz_fs) +DO_FP_2SH(VCVT_FU, gen_helper_gvec_vcvt_rz_fu) DO_FP_2SH(VCVT_SH, gen_helper_gvec_vcvt_sh) DO_FP_2SH(VCVT_UH, gen_helper_gvec_vcvt_uh) -DO_FP_2SH(VCVT_HS, gen_helper_gvec_vcvt_hs) -DO_FP_2SH(VCVT_HU, gen_helper_gvec_vcvt_hu) +DO_FP_2SH(VCVT_HS, gen_helper_gvec_vcvt_rz_hs) +DO_FP_2SH(VCVT_HU, gen_helper_gvec_vcvt_rz_hu) static bool do_1reg_imm(DisasContext *s, arg_1reg_imm *a, GVecGen2iFn *fn) diff --git a/target/arm/tcg/vec_helper.c b/target/arm/tcg/vec_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/vec_helper.c +++ b/target/arm/tcg/vec_helper.c @@ -XXX,XX +XXX,XX @@ DO_3OP_PAIR(gvec_uminp_s, MIN, uint32_t, H4) DO_VCVT_FIXED(gvec_vcvt_sf, helper_vfp_sltos, uint32_t) DO_VCVT_FIXED(gvec_vcvt_uf, helper_vfp_ultos, uint32_t) -DO_VCVT_FIXED(gvec_vcvt_fs, helper_vfp_tosls_round_to_zero, uint32_t) -DO_VCVT_FIXED(gvec_vcvt_fu, helper_vfp_touls_round_to_zero, uint32_t) +DO_VCVT_FIXED(gvec_vcvt_rz_fs, helper_vfp_tosls_round_to_zero, uint32_t) +DO_VCVT_FIXED(gvec_vcvt_rz_fu, helper_vfp_touls_round_to_zero, uint32_t) DO_VCVT_FIXED(gvec_vcvt_sh, helper_vfp_shtoh, uint16_t) DO_VCVT_FIXED(gvec_vcvt_uh, helper_vfp_uhtoh, uint16_t) -DO_VCVT_FIXED(gvec_vcvt_hs, helper_vfp_toshh_round_to_zero, uint16_t) -DO_VCVT_FIXED(gvec_vcvt_hu, helper_vfp_touhh_round_to_zero, uint16_t) +DO_VCVT_FIXED(gvec_vcvt_rz_hs, helper_vfp_toshh_round_to_zero, uint16_t) +DO_VCVT_FIXED(gvec_vcvt_rz_hu, helper_vfp_touhh_round_to_zero, uint16_t) #undef DO_VCVT_FIXED -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Remove handle_simd_intfp_conv and handle_simd_shift_intfp_conv as these were the last insns decoded by those functions. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-63-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/helper.h | 3 + target/arm/tcg/a64.decode | 22 ++++ target/arm/tcg/translate-a64.c | 201 ++++++--------------------------- target/arm/tcg/vec_helper.c | 7 +- 4 files changed, 66 insertions(+), 167 deletions(-) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_4(gvec_vcvt_uh, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_vcvt_rz_hs, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_vcvt_rz_hu, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_vcvt_sd, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_vcvt_ud, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_4(gvec_vcvt_rm_ss, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_vcvt_rm_us, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_vcvt_rm_sh, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ FRINT32Z_v 0.00 1110 0.1 00001 11101 0 ..... ..... @qrr_sd FRINT32X_v 0.10 1110 0.1 00001 11101 0 ..... ..... @qrr_sd FRINT64Z_v 0.00 1110 0.1 00001 11111 0 ..... ..... @qrr_sd FRINT64X_v 0.10 1110 0.1 00001 11111 0 ..... ..... @qrr_sd + +SCVTF_vi 0.00 1110 011 11001 11011 0 ..... ..... @qrr_h +SCVTF_vi 0.00 1110 0.1 00001 11011 0 ..... ..... @qrr_sd + +UCVTF_vi 0.10 1110 011 11001 11011 0 ..... ..... @qrr_h +UCVTF_vi 0.10 1110 0.1 00001 11011 0 ..... ..... @qrr_sd + +&fcvt_q rd rn esz q shift +@fcvtq_h . q:1 . ...... 001 .... ...... rn:5 rd:5 \ + &fcvt_q esz=1 shift=%fcvt_f_sh_h +@fcvtq_s . q:1 . ...... 01 ..... ...... rn:5 rd:5 \ + &fcvt_q esz=2 shift=%fcvt_f_sh_s +@fcvtq_d . q:1 . ...... 1 ...... ...... rn:5 rd:5 \ + &fcvt_q esz=3 shift=%fcvt_f_sh_d + +SCVTF_vf 0.00 11110 ....... 111001 ..... ..... @fcvtq_h +SCVTF_vf 0.00 11110 ....... 111001 ..... ..... @fcvtq_s +SCVTF_vf 0.00 11110 ....... 111001 ..... ..... @fcvtq_d + +UCVTF_vf 0.10 11110 ....... 111001 ..... ..... @fcvtq_h +UCVTF_vf 0.10 11110 ....... 111001 ..... ..... @fcvtq_s +UCVTF_vf 0.10 11110 ....... 111001 ..... ..... @fcvtq_d diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ TRANS_FEAT(FRINT64Z_v, aa64_frint, do_fp1_vector, a, &f_scalar_frint64, FPROUNDING_ZERO) TRANS_FEAT(FRINT64X_v, aa64_frint, do_fp1_vector, a, &f_scalar_frint64, -1) -/* Common vector code for handling integer to FP conversion */ -static void handle_simd_intfp_conv(DisasContext *s, int rd, int rn, - int elements, int is_signed, - int fracbits, int size) +static bool do_gvec_op2_fpst(DisasContext *s, MemOp esz, bool is_q, + int rd, int rn, int data, + gen_helper_gvec_2_ptr * const fns[3]) { - TCGv_ptr tcg_fpst = fpstatus_ptr(size == MO_16 ? FPST_FPCR_F16 : FPST_FPCR); - TCGv_i32 tcg_shift = NULL; + int check = fp_access_check_vector_hsd(s, is_q, esz); + TCGv_ptr fpst; - MemOp mop = size | (is_signed ? MO_SIGN : 0); - int pass; - - if (fracbits || size == MO_64) { - tcg_shift = tcg_constant_i32(fracbits); + if (check <= 0) { + return check == 0; } - if (size == MO_64) { - TCGv_i64 tcg_int64 = tcg_temp_new_i64(); - TCGv_i64 tcg_double = tcg_temp_new_i64(); - - for (pass = 0; pass < elements; pass++) { - read_vec_element(s, tcg_int64, rn, pass, mop); - - if (is_signed) { - gen_helper_vfp_sqtod(tcg_double, tcg_int64, - tcg_shift, tcg_fpst); - } else { - gen_helper_vfp_uqtod(tcg_double, tcg_int64, - tcg_shift, tcg_fpst); - } - if (elements == 1) { - write_fp_dreg(s, rd, tcg_double); - } else { - write_vec_element(s, tcg_double, rd, pass, MO_64); - } - } - } else { - TCGv_i32 tcg_int32 = tcg_temp_new_i32(); - TCGv_i32 tcg_float = tcg_temp_new_i32(); - - for (pass = 0; pass < elements; pass++) { - read_vec_element_i32(s, tcg_int32, rn, pass, mop); - - switch (size) { - case MO_32: - if (fracbits) { - if (is_signed) { - gen_helper_vfp_sltos(tcg_float, tcg_int32, - tcg_shift, tcg_fpst); - } else { - gen_helper_vfp_ultos(tcg_float, tcg_int32, - tcg_shift, tcg_fpst); - } - } else { - if (is_signed) { - gen_helper_vfp_sitos(tcg_float, tcg_int32, tcg_fpst); - } else { - gen_helper_vfp_uitos(tcg_float, tcg_int32, tcg_fpst); - } - } - break; - case MO_16: - if (fracbits) { - if (is_signed) { - gen_helper_vfp_sltoh(tcg_float, tcg_int32, - tcg_shift, tcg_fpst); - } else { - gen_helper_vfp_ultoh(tcg_float, tcg_int32, - tcg_shift, tcg_fpst); - } - } else { - if (is_signed) { - gen_helper_vfp_sitoh(tcg_float, tcg_int32, tcg_fpst); - } else { - gen_helper_vfp_uitoh(tcg_float, tcg_int32, tcg_fpst); - } - } - break; - default: - g_assert_not_reached(); - } - - if (elements == 1) { - write_fp_sreg(s, rd, tcg_float); - } else { - write_vec_element_i32(s, tcg_float, rd, pass, size); - } - } - } - - clear_vec_high(s, elements << size == 16, rd); + fpst = fpstatus_ptr(esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR); + tcg_gen_gvec_2_ptr(vec_full_reg_offset(s, rd), + vec_full_reg_offset(s, rn), fpst, + is_q ? 16 : 8, vec_full_reg_size(s), + data, fns[esz - 1]); + return true; } -/* UCVTF/SCVTF - Integer to FP conversion */ -static void handle_simd_shift_intfp_conv(DisasContext *s, bool is_scalar, - bool is_q, bool is_u, - int immh, int immb, int opcode, - int rn, int rd) -{ - int size, elements, fracbits; - int immhb = immh << 3 | immb; +static gen_helper_gvec_2_ptr * const f_scvtf_v[] = { + gen_helper_gvec_vcvt_sh, + gen_helper_gvec_vcvt_sf, + gen_helper_gvec_vcvt_sd, +}; +TRANS(SCVTF_vi, do_gvec_op2_fpst, + a->esz, a->q, a->rd, a->rn, 0, f_scvtf_v) +TRANS(SCVTF_vf, do_gvec_op2_fpst, + a->esz, a->q, a->rd, a->rn, a->shift, f_scvtf_v) - if (immh & 8) { - size = MO_64; - if (!is_scalar && !is_q) { - unallocated_encoding(s); - return; - } - } else if (immh & 4) { - size = MO_32; - } else if (immh & 2) { - size = MO_16; - if (!dc_isar_feature(aa64_fp16, s)) { - unallocated_encoding(s); - return; - } - } else { - /* immh == 0 would be a failure of the decode logic */ - g_assert(immh == 1); - unallocated_encoding(s); - return; - } - - if (is_scalar) { - elements = 1; - } else { - elements = (8 << is_q) >> size; - } - fracbits = (16 << size) - immhb; - - if (!fp_access_check(s)) { - return; - } - - handle_simd_intfp_conv(s, rd, rn, elements, !is_u, fracbits, size); -} +static gen_helper_gvec_2_ptr * const f_ucvtf_v[] = { + gen_helper_gvec_vcvt_uh, + gen_helper_gvec_vcvt_uf, + gen_helper_gvec_vcvt_ud, +}; +TRANS(UCVTF_vi, do_gvec_op2_fpst, + a->esz, a->q, a->rd, a->rn, 0, f_ucvtf_v) +TRANS(UCVTF_vf, do_gvec_op2_fpst, + a->esz, a->q, a->rd, a->rn, a->shift, f_ucvtf_v) /* FCVTZS, FVCVTZU - FP to fixedpoint conversion */ static void handle_simd_shift_fpint_conv(DisasContext *s, bool is_scalar, @@ -XXX,XX +XXX,XX @@ static void disas_simd_shift_imm(DisasContext *s, uint32_t insn) } switch (opcode) { - case 0x1c: /* SCVTF / UCVTF */ - handle_simd_shift_intfp_conv(s, false, is_q, is_u, immh, immb, - opcode, rn, rd); - break; case 0x1f: /* FCVTZS/ FCVTZU */ handle_simd_shift_fpint_conv(s, false, is_q, is_u, immh, immb, rn, rd); return; @@ -XXX,XX +XXX,XX @@ static void disas_simd_shift_imm(DisasContext *s, uint32_t insn) case 0x12: /* SQSHRN / UQSHRN */ case 0x13: /* SQRSHRN / UQRSHRN */ case 0x14: /* SSHLL / USHLL */ + case 0x1c: /* SCVTF / UCVTF */ unallocated_encoding(s); return; } @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) opcode |= (extract32(size, 1, 1) << 5) | (u << 6); size = is_double ? 3 : 2; switch (opcode) { - case 0x1d: /* SCVTF */ - case 0x5d: /* UCVTF */ - { - bool is_signed = (opcode == 0x1d) ? true : false; - int elements = is_double ? 2 : is_q ? 4 : 2; - if (is_double && !is_q) { - unallocated_encoding(s); - return; - } - if (!fp_access_check(s)) { - return; - } - handle_simd_intfp_conv(s, rd, rn, elements, is_signed, 0, size); - return; - } case 0x2c: /* FCMGT (zero) */ case 0x2d: /* FCMEQ (zero) */ case 0x2e: /* FCMLT (zero) */ @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) case 0x1f: /* FRINT64Z */ case 0x5e: /* FRINT32X */ case 0x5f: /* FRINT64X */ + case 0x1d: /* SCVTF */ + case 0x5d: /* UCVTF */ unallocated_encoding(s); return; } @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc_fp16(DisasContext *s, uint32_t insn) fpop = deposit32(fpop, 6, 1, u); switch (fpop) { - case 0x1d: /* SCVTF */ - case 0x5d: /* UCVTF */ - { - int elements; - - if (is_scalar) { - elements = 1; - } else { - elements = (is_q ? 8 : 4); - } - - if (!fp_access_check(s)) { - return; - } - handle_simd_intfp_conv(s, rd, rn, elements, !u, 0, MO_16); - return; - } - break; case 0x2c: /* FCMGT (zero) */ case 0x2d: /* FCMEQ (zero) */ case 0x2e: /* FCMLT (zero) */ @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc_fp16(DisasContext *s, uint32_t insn) case 0x58: /* FRINTA */ case 0x59: /* FRINTX */ case 0x79: /* FRINTI */ + case 0x1d: /* SCVTF */ + case 0x5d: /* UCVTF */ unallocated_encoding(s); return; } diff --git a/target/arm/tcg/vec_helper.c b/target/arm/tcg/vec_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/vec_helper.c +++ b/target/arm/tcg/vec_helper.c @@ -XXX,XX +XXX,XX @@ DO_3OP_PAIR(gvec_uminp_s, MIN, uint32_t, H4) clear_tail(d, oprsz, simd_maxsz(desc)); \ } +DO_VCVT_FIXED(gvec_vcvt_sd, helper_vfp_sqtod, uint64_t) +DO_VCVT_FIXED(gvec_vcvt_ud, helper_vfp_uqtod, uint64_t) DO_VCVT_FIXED(gvec_vcvt_sf, helper_vfp_sltos, uint32_t) DO_VCVT_FIXED(gvec_vcvt_uf, helper_vfp_ultos, uint32_t) -DO_VCVT_FIXED(gvec_vcvt_rz_fs, helper_vfp_tosls_round_to_zero, uint32_t) -DO_VCVT_FIXED(gvec_vcvt_rz_fu, helper_vfp_touls_round_to_zero, uint32_t) DO_VCVT_FIXED(gvec_vcvt_sh, helper_vfp_shtoh, uint16_t) DO_VCVT_FIXED(gvec_vcvt_uh, helper_vfp_uhtoh, uint16_t) + +DO_VCVT_FIXED(gvec_vcvt_rz_fs, helper_vfp_tosls_round_to_zero, uint32_t) +DO_VCVT_FIXED(gvec_vcvt_rz_fu, helper_vfp_touls_round_to_zero, uint32_t) DO_VCVT_FIXED(gvec_vcvt_rz_hs, helper_vfp_toshh_round_to_zero, uint16_t) DO_VCVT_FIXED(gvec_vcvt_rz_hu, helper_vfp_touhh_round_to_zero, uint16_t) -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Remove handle_simd_shift_fpint_conv and disas_simd_shift_imm as these were the last insns decoded by those functions. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-64-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/helper.h | 4 + target/arm/tcg/a64.decode | 8 ++ target/arm/tcg/translate-a64.c | 160 +++------------------------------ target/arm/tcg/vec_helper.c | 2 + target/arm/vfp_helper.c | 4 + 5 files changed, 32 insertions(+), 146 deletions(-) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_3(vfp_touhs_round_to_zero, i32, f32, i32, ptr) DEF_HELPER_3(vfp_touls_round_to_zero, i32, f32, i32, ptr) DEF_HELPER_3(vfp_toshd_round_to_zero, i64, f64, i32, ptr) DEF_HELPER_3(vfp_tosld_round_to_zero, i64, f64, i32, ptr) +DEF_HELPER_3(vfp_tosqd_round_to_zero, i64, f64, i32, ptr) DEF_HELPER_3(vfp_touhd_round_to_zero, i64, f64, i32, ptr) DEF_HELPER_3(vfp_tould_round_to_zero, i64, f64, i32, ptr) +DEF_HELPER_3(vfp_touqd_round_to_zero, i64, f64, i32, ptr) DEF_HELPER_3(vfp_touhh, i32, f16, i32, ptr) DEF_HELPER_3(vfp_toshh, i32, f16, i32, ptr) DEF_HELPER_3(vfp_toulh, i32, f16, i32, ptr) @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_4(gvec_vcvt_rz_hu, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_vcvt_sd, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_vcvt_ud, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_vcvt_rz_ds, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_vcvt_rz_du, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_vcvt_rm_ss, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_vcvt_rm_us, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ SCVTF_vf 0.00 11110 ....... 111001 ..... ..... @fcvtq_d UCVTF_vf 0.10 11110 ....... 111001 ..... ..... @fcvtq_h UCVTF_vf 0.10 11110 ....... 111001 ..... ..... @fcvtq_s UCVTF_vf 0.10 11110 ....... 111001 ..... ..... @fcvtq_d + +FCVTZS_vf 0.00 11110 ....... 111111 ..... ..... @fcvtq_h +FCVTZS_vf 0.00 11110 ....... 111111 ..... ..... @fcvtq_s +FCVTZS_vf 0.00 11110 ....... 111111 ..... ..... @fcvtq_d + +FCVTZU_vf 0.10 11110 ....... 111111 ..... ..... @fcvtq_h +FCVTZU_vf 0.10 11110 ....... 111111 ..... ..... @fcvtq_s +FCVTZU_vf 0.10 11110 ....... 111111 ..... ..... @fcvtq_d diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ TRANS(UCVTF_vi, do_gvec_op2_fpst, TRANS(UCVTF_vf, do_gvec_op2_fpst, a->esz, a->q, a->rd, a->rn, a->shift, f_ucvtf_v) -/* FCVTZS, FVCVTZU - FP to fixedpoint conversion */ -static void handle_simd_shift_fpint_conv(DisasContext *s, bool is_scalar, - bool is_q, bool is_u, - int immh, int immb, int rn, int rd) -{ - int immhb = immh << 3 | immb; - int pass, size, fracbits; - TCGv_ptr tcg_fpstatus; - TCGv_i32 tcg_rmode, tcg_shift; +static gen_helper_gvec_2_ptr * const f_fcvtzs_vf[] = { + gen_helper_gvec_vcvt_rz_hs, + gen_helper_gvec_vcvt_rz_fs, + gen_helper_gvec_vcvt_rz_ds, +}; +TRANS(FCVTZS_vf, do_gvec_op2_fpst, + a->esz, a->q, a->rd, a->rn, a->shift, f_fcvtzs_vf) - if (immh & 0x8) { - size = MO_64; - if (!is_scalar && !is_q) { - unallocated_encoding(s); - return; - } - } else if (immh & 0x4) { - size = MO_32; - } else if (immh & 0x2) { - size = MO_16; - if (!dc_isar_feature(aa64_fp16, s)) { - unallocated_encoding(s); - return; - } - } else { - /* Should have split out AdvSIMD modified immediate earlier. */ - assert(immh == 1); - unallocated_encoding(s); - return; - } - - if (!fp_access_check(s)) { - return; - } - - assert(!(is_scalar && is_q)); - - tcg_fpstatus = fpstatus_ptr(size == MO_16 ? FPST_FPCR_F16 : FPST_FPCR); - tcg_rmode = gen_set_rmode(FPROUNDING_ZERO, tcg_fpstatus); - fracbits = (16 << size) - immhb; - tcg_shift = tcg_constant_i32(fracbits); - - if (size == MO_64) { - int maxpass = is_scalar ? 1 : 2; - - for (pass = 0; pass < maxpass; pass++) { - TCGv_i64 tcg_op = tcg_temp_new_i64(); - - read_vec_element(s, tcg_op, rn, pass, MO_64); - if (is_u) { - gen_helper_vfp_touqd(tcg_op, tcg_op, tcg_shift, tcg_fpstatus); - } else { - gen_helper_vfp_tosqd(tcg_op, tcg_op, tcg_shift, tcg_fpstatus); - } - write_vec_element(s, tcg_op, rd, pass, MO_64); - } - clear_vec_high(s, is_q, rd); - } else { - void (*fn)(TCGv_i32, TCGv_i32, TCGv_i32, TCGv_ptr); - int maxpass = is_scalar ? 1 : ((8 << is_q) >> size); - - switch (size) { - case MO_16: - if (is_u) { - fn = gen_helper_vfp_touhh; - } else { - fn = gen_helper_vfp_toshh; - } - break; - case MO_32: - if (is_u) { - fn = gen_helper_vfp_touls; - } else { - fn = gen_helper_vfp_tosls; - } - break; - default: - g_assert_not_reached(); - } - - for (pass = 0; pass < maxpass; pass++) { - TCGv_i32 tcg_op = tcg_temp_new_i32(); - - read_vec_element_i32(s, tcg_op, rn, pass, size); - fn(tcg_op, tcg_op, tcg_shift, tcg_fpstatus); - if (is_scalar) { - if (size == MO_16 && !is_u) { - tcg_gen_ext16u_i32(tcg_op, tcg_op); - } - write_fp_sreg(s, rd, tcg_op); - } else { - write_vec_element_i32(s, tcg_op, rd, pass, size); - } - } - if (!is_scalar) { - clear_vec_high(s, is_q, rd); - } - } - - gen_restore_rmode(tcg_rmode, tcg_fpstatus); -} +static gen_helper_gvec_2_ptr * const f_fcvtzu_vf[] = { + gen_helper_gvec_vcvt_rz_hu, + gen_helper_gvec_vcvt_rz_fu, + gen_helper_gvec_vcvt_rz_du, +}; +TRANS(FCVTZU_vf, do_gvec_op2_fpst, + a->esz, a->q, a->rd, a->rn, a->shift, f_fcvtzu_vf) static void handle_2misc_64(DisasContext *s, int opcode, bool u, TCGv_i64 tcg_rd, TCGv_i64 tcg_rn, @@ -XXX,XX +XXX,XX @@ static void disas_simd_scalar_two_reg_misc(DisasContext *s, uint32_t insn) g_assert_not_reached(); } -/* AdvSIMD shift by immediate - * 31 30 29 28 23 22 19 18 16 15 11 10 9 5 4 0 - * +---+---+---+-------------+------+------+--------+---+------+------+ - * | 0 | Q | U | 0 1 1 1 1 0 | immh | immb | opcode | 1 | Rn | Rd | - * +---+---+---+-------------+------+------+--------+---+------+------+ - */ -static void disas_simd_shift_imm(DisasContext *s, uint32_t insn) -{ - int rd = extract32(insn, 0, 5); - int rn = extract32(insn, 5, 5); - int opcode = extract32(insn, 11, 5); - int immb = extract32(insn, 16, 3); - int immh = extract32(insn, 19, 4); - bool is_u = extract32(insn, 29, 1); - bool is_q = extract32(insn, 30, 1); - - if (immh == 0) { - unallocated_encoding(s); - return; - } - - switch (opcode) { - case 0x1f: /* FCVTZS/ FCVTZU */ - handle_simd_shift_fpint_conv(s, false, is_q, is_u, immh, immb, rn, rd); - return; - default: - case 0x00: /* SSHR / USHR */ - case 0x02: /* SSRA / USRA (accumulate) */ - case 0x04: /* SRSHR / URSHR (rounding) */ - case 0x06: /* SRSRA / URSRA (accum + rounding) */ - case 0x08: /* SRI */ - case 0x0a: /* SHL / SLI */ - case 0x0c: /* SQSHLU */ - case 0x0e: /* SQSHL, UQSHL */ - case 0x10: /* SHRN / SQSHRUN */ - case 0x11: /* RSHRN / SQRSHRUN */ - case 0x12: /* SQSHRN / UQSHRN */ - case 0x13: /* SQRSHRN / UQRSHRN */ - case 0x14: /* SSHLL / USHLL */ - case 0x1c: /* SCVTF / UCVTF */ - unallocated_encoding(s); - return; - } -} - static void handle_2misc_widening(DisasContext *s, int opcode, bool is_q, int size, int rn, int rd) { @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc_fp16(DisasContext *s, uint32_t insn) static const AArch64DecodeTable data_proc_simd[] = { /* pattern , mask , fn */ { 0x0e200800, 0x9f3e0c00, disas_simd_two_reg_misc }, - { 0x0f000400, 0x9f800400, disas_simd_shift_imm }, { 0x5e200800, 0xdf3e0c00, disas_simd_scalar_two_reg_misc }, { 0x0e780800, 0x8f7e0c00, disas_simd_two_reg_misc_fp16 }, { 0x00000000, 0x00000000, NULL } diff --git a/target/arm/tcg/vec_helper.c b/target/arm/tcg/vec_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/vec_helper.c +++ b/target/arm/tcg/vec_helper.c @@ -XXX,XX +XXX,XX @@ DO_VCVT_FIXED(gvec_vcvt_uf, helper_vfp_ultos, uint32_t) DO_VCVT_FIXED(gvec_vcvt_sh, helper_vfp_shtoh, uint16_t) DO_VCVT_FIXED(gvec_vcvt_uh, helper_vfp_uhtoh, uint16_t) +DO_VCVT_FIXED(gvec_vcvt_rz_ds, helper_vfp_tosqd_round_to_zero, uint64_t) +DO_VCVT_FIXED(gvec_vcvt_rz_du, helper_vfp_touqd_round_to_zero, uint64_t) DO_VCVT_FIXED(gvec_vcvt_rz_fs, helper_vfp_tosls_round_to_zero, uint32_t) DO_VCVT_FIXED(gvec_vcvt_rz_fu, helper_vfp_touls_round_to_zero, uint32_t) DO_VCVT_FIXED(gvec_vcvt_rz_hs, helper_vfp_toshh_round_to_zero, uint16_t) diff --git a/target/arm/vfp_helper.c b/target/arm/vfp_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/vfp_helper.c +++ b/target/arm/vfp_helper.c @@ -XXX,XX +XXX,XX @@ VFP_CONV_FIX_A64(sq, h, 16, dh_ctype_f16, 64, int64) VFP_CONV_FIX(uh, h, 16, dh_ctype_f16, 32, uint16) VFP_CONV_FIX(ul, h, 16, dh_ctype_f16, 32, uint32) VFP_CONV_FIX_A64(uq, h, 16, dh_ctype_f16, 64, uint64) +VFP_CONV_FLOAT_FIX_ROUND(sq, d, 64, float64, 64, int64, + float_round_to_zero, _round_to_zero) +VFP_CONV_FLOAT_FIX_ROUND(uq, d, 64, float64, 64, uint64, + float_round_to_zero, _round_to_zero) #undef VFP_CONV_FIX #undef VFP_CONV_FIX_FLOAT -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Remove handle_2misc_64 as these were the last insns decoded by that function. Remove helper_advsimd_f16to[su]inth as unused; we now always go through helper_vfp_to[su]hh or a specialized vector function instead. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-65-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/helper.h | 2 + target/arm/tcg/helper-a64.h | 2 - target/arm/tcg/a64.decode | 25 ++++ target/arm/tcg/helper-a64.c | 32 ----- target/arm/tcg/translate-a64.c | 227 +++++++++++---------------------- target/arm/tcg/vec_helper.c | 2 + 6 files changed, 102 insertions(+), 188 deletions(-) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_4(gvec_vcvt_ud, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_vcvt_rz_ds, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_vcvt_rz_du, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_vcvt_rm_sd, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_vcvt_rm_ud, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_vcvt_rm_ss, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_vcvt_rm_us, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_vcvt_rm_sh, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) diff --git a/target/arm/tcg/helper-a64.h b/target/arm/tcg/helper-a64.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/helper-a64.h +++ b/target/arm/tcg/helper-a64.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_3(advsimd_mulx2h, i32, i32, i32, ptr) DEF_HELPER_4(advsimd_muladd2h, i32, i32, i32, i32, ptr) DEF_HELPER_2(advsimd_rinth_exact, f16, f16, ptr) DEF_HELPER_2(advsimd_rinth, f16, f16, ptr) -DEF_HELPER_2(advsimd_f16tosinth, i32, f16, ptr) -DEF_HELPER_2(advsimd_f16touinth, i32, f16, ptr) DEF_HELPER_2(exception_return, void, env, i64) DEF_HELPER_FLAGS_2(dc_zva, TCG_CALL_NO_WG, void, env, i64) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ SCVTF_vi 0.00 1110 0.1 00001 11011 0 ..... ..... @qrr_sd UCVTF_vi 0.10 1110 011 11001 11011 0 ..... ..... @qrr_h UCVTF_vi 0.10 1110 0.1 00001 11011 0 ..... ..... @qrr_sd +FCVTNS_vi 0.00 1110 011 11001 10101 0 ..... ..... @qrr_h +FCVTNS_vi 0.00 1110 0.1 00001 10101 0 ..... ..... @qrr_sd +FCVTNU_vi 0.10 1110 011 11001 10101 0 ..... ..... @qrr_h +FCVTNU_vi 0.10 1110 0.1 00001 10101 0 ..... ..... @qrr_sd + +FCVTPS_vi 0.00 1110 111 11001 10101 0 ..... ..... @qrr_h +FCVTPS_vi 0.00 1110 1.1 00001 10101 0 ..... ..... @qrr_sd +FCVTPU_vi 0.10 1110 111 11001 10101 0 ..... ..... @qrr_h +FCVTPU_vi 0.10 1110 1.1 00001 10101 0 ..... ..... @qrr_sd + +FCVTMS_vi 0.00 1110 011 11001 10111 0 ..... ..... @qrr_h +FCVTMS_vi 0.00 1110 0.1 00001 10111 0 ..... ..... @qrr_sd +FCVTMU_vi 0.10 1110 011 11001 10111 0 ..... ..... @qrr_h +FCVTMU_vi 0.10 1110 0.1 00001 10111 0 ..... ..... @qrr_sd + +FCVTZS_vi 0.00 1110 111 11001 10111 0 ..... ..... @qrr_h +FCVTZS_vi 0.00 1110 1.1 00001 10111 0 ..... ..... @qrr_sd +FCVTZU_vi 0.10 1110 111 11001 10111 0 ..... ..... @qrr_h +FCVTZU_vi 0.10 1110 1.1 00001 10111 0 ..... ..... @qrr_sd + +FCVTAS_vi 0.00 1110 011 11001 11001 0 ..... ..... @qrr_h +FCVTAS_vi 0.00 1110 0.1 00001 11001 0 ..... ..... @qrr_sd +FCVTAU_vi 0.10 1110 011 11001 11001 0 ..... ..... @qrr_h +FCVTAU_vi 0.10 1110 0.1 00001 11001 0 ..... ..... @qrr_sd + &fcvt_q rd rn esz q shift @fcvtq_h . q:1 . ...... 001 .... ...... rn:5 rd:5 \ &fcvt_q esz=1 shift=%fcvt_f_sh_h diff --git a/target/arm/tcg/helper-a64.c b/target/arm/tcg/helper-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/helper-a64.c +++ b/target/arm/tcg/helper-a64.c @@ -XXX,XX +XXX,XX @@ uint32_t HELPER(advsimd_rinth)(uint32_t x, void *fp_status) return ret; } -/* - * Half-precision floating point conversion functions - * - * There are a multitude of conversion functions with various - * different rounding modes. This is dealt with by the calling code - * setting the mode appropriately before calling the helper. - */ - -uint32_t HELPER(advsimd_f16tosinth)(uint32_t a, void *fpstp) -{ - float_status *fpst = fpstp; - - /* Invalid if we are passed a NaN */ - if (float16_is_any_nan(a)) { - float_raise(float_flag_invalid, fpst); - return 0; - } - return float16_to_int16(a, fpst); -} - -uint32_t HELPER(advsimd_f16touinth)(uint32_t a, void *fpstp) -{ - float_status *fpst = fpstp; - - /* Invalid if we are passed a NaN */ - if (float16_is_any_nan(a)) { - float_raise(float_flag_invalid, fpst); - return 0; - } - return float16_to_uint16(a, fpst); -} - static int el_from_spsr(uint32_t spsr) { /* Return the exception level that this SPSR is requesting a return to, diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static gen_helper_gvec_2_ptr * const f_fcvtzu_vf[] = { TRANS(FCVTZU_vf, do_gvec_op2_fpst, a->esz, a->q, a->rd, a->rn, a->shift, f_fcvtzu_vf) -static void handle_2misc_64(DisasContext *s, int opcode, bool u, - TCGv_i64 tcg_rd, TCGv_i64 tcg_rn, - TCGv_i32 tcg_rmode, TCGv_ptr tcg_fpstatus) -{ - /* Handle 64->64 opcodes which are shared between the scalar and - * vector 2-reg-misc groups. We cover every integer opcode where size == 3 - * is valid in either group and also the double-precision fp ops. - * The caller only need provide tcg_rmode and tcg_fpstatus if the op - * requires them. - */ - switch (opcode) { - case 0x1a: /* FCVTNS */ - case 0x1b: /* FCVTMS */ - case 0x1c: /* FCVTAS */ - case 0x3a: /* FCVTPS */ - case 0x3b: /* FCVTZS */ - gen_helper_vfp_tosqd(tcg_rd, tcg_rn, tcg_constant_i32(0), tcg_fpstatus); - break; - case 0x5a: /* FCVTNU */ - case 0x5b: /* FCVTMU */ - case 0x5c: /* FCVTAU */ - case 0x7a: /* FCVTPU */ - case 0x7b: /* FCVTZU */ - gen_helper_vfp_touqd(tcg_rd, tcg_rn, tcg_constant_i32(0), tcg_fpstatus); - break; - default: - case 0x4: /* CLS, CLZ */ - case 0x5: /* NOT */ - case 0x7: /* SQABS, SQNEG */ - case 0x8: /* CMGT, CMGE */ - case 0x9: /* CMEQ, CMLE */ - case 0xa: /* CMLT */ - case 0xb: /* ABS, NEG */ - case 0x2f: /* FABS */ - case 0x6f: /* FNEG */ - case 0x7f: /* FSQRT */ - case 0x18: /* FRINTN */ - case 0x19: /* FRINTM */ - case 0x38: /* FRINTP */ - case 0x39: /* FRINTZ */ - case 0x58: /* FRINTA */ - case 0x79: /* FRINTI */ - case 0x59: /* FRINTX */ - case 0x1e: /* FRINT32Z */ - case 0x5e: /* FRINT32X */ - case 0x1f: /* FRINT64Z */ - case 0x5f: /* FRINT64X */ - g_assert_not_reached(); - } -} +static gen_helper_gvec_2_ptr * const f_fcvt_s_vi[] = { + gen_helper_gvec_vcvt_rm_sh, + gen_helper_gvec_vcvt_rm_ss, + gen_helper_gvec_vcvt_rm_sd, +}; + +static gen_helper_gvec_2_ptr * const f_fcvt_u_vi[] = { + gen_helper_gvec_vcvt_rm_uh, + gen_helper_gvec_vcvt_rm_us, + gen_helper_gvec_vcvt_rm_ud, +}; + +TRANS(FCVTNS_vi, do_gvec_op2_fpst, + a->esz, a->q, a->rd, a->rn, float_round_nearest_even, f_fcvt_s_vi) +TRANS(FCVTNU_vi, do_gvec_op2_fpst, + a->esz, a->q, a->rd, a->rn, float_round_nearest_even, f_fcvt_u_vi) +TRANS(FCVTPS_vi, do_gvec_op2_fpst, + a->esz, a->q, a->rd, a->rn, float_round_up, f_fcvt_s_vi) +TRANS(FCVTPU_vi, do_gvec_op2_fpst, + a->esz, a->q, a->rd, a->rn, float_round_up, f_fcvt_u_vi) +TRANS(FCVTMS_vi, do_gvec_op2_fpst, + a->esz, a->q, a->rd, a->rn, float_round_down, f_fcvt_s_vi) +TRANS(FCVTMU_vi, do_gvec_op2_fpst, + a->esz, a->q, a->rd, a->rn, float_round_down, f_fcvt_u_vi) +TRANS(FCVTZS_vi, do_gvec_op2_fpst, + a->esz, a->q, a->rd, a->rn, float_round_to_zero, f_fcvt_s_vi) +TRANS(FCVTZU_vi, do_gvec_op2_fpst, + a->esz, a->q, a->rd, a->rn, float_round_to_zero, f_fcvt_u_vi) +TRANS(FCVTAS_vi, do_gvec_op2_fpst, + a->esz, a->q, a->rd, a->rn, float_round_ties_away, f_fcvt_s_vi) +TRANS(FCVTAU_vi, do_gvec_op2_fpst, + a->esz, a->q, a->rd, a->rn, float_round_ties_away, f_fcvt_u_vi) static void handle_2misc_fcmp_zero(DisasContext *s, int opcode, bool is_scalar, bool is_u, bool is_q, @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) } handle_2misc_fcmp_zero(s, opcode, false, u, is_q, size, rn, rd); return; - case 0x1a: /* FCVTNS */ - case 0x1b: /* FCVTMS */ - case 0x3a: /* FCVTPS */ - case 0x3b: /* FCVTZS */ - case 0x5a: /* FCVTNU */ - case 0x5b: /* FCVTMU */ - case 0x7a: /* FCVTPU */ - case 0x7b: /* FCVTZU */ - need_fpstatus = true; - rmode = extract32(opcode, 5, 1) | (extract32(opcode, 0, 1) << 1); - if (size == 3 && !is_q) { - unallocated_encoding(s); - return; - } - break; - case 0x5c: /* FCVTAU */ - case 0x1c: /* FCVTAS */ - need_fpstatus = true; - rmode = FPROUNDING_TIEAWAY; - if (size == 3 && !is_q) { - unallocated_encoding(s); - return; - } - break; case 0x3c: /* URECPE */ if (size == 3) { unallocated_encoding(s); @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) case 0x5f: /* FRINT64X */ case 0x1d: /* SCVTF */ case 0x5d: /* UCVTF */ + case 0x1a: /* FCVTNS */ + case 0x1b: /* FCVTMS */ + case 0x3a: /* FCVTPS */ + case 0x3b: /* FCVTZS */ + case 0x5a: /* FCVTNU */ + case 0x5b: /* FCVTMU */ + case 0x7a: /* FCVTPU */ + case 0x7b: /* FCVTZU */ + case 0x5c: /* FCVTAU */ + case 0x1c: /* FCVTAS */ unallocated_encoding(s); return; } @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) tcg_rmode = NULL; } - if (size == 3) { - /* All 64-bit element operations can be shared with scalar 2misc */ - int pass; - - /* Coverity claims (size == 3 && !is_q) has been eliminated - * from all paths leading to here. - */ - tcg_debug_assert(is_q); - for (pass = 0; pass < 2; pass++) { - TCGv_i64 tcg_op = tcg_temp_new_i64(); - TCGv_i64 tcg_res = tcg_temp_new_i64(); - - read_vec_element(s, tcg_op, rn, pass, MO_64); - - handle_2misc_64(s, opcode, u, tcg_res, tcg_op, - tcg_rmode, tcg_fpstatus); - - write_vec_element(s, tcg_res, rd, pass, MO_64); - } - } else { + { int pass; assert(size == 2); @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) { /* Special cases for 32 bit elements */ switch (opcode) { - case 0x1a: /* FCVTNS */ - case 0x1b: /* FCVTMS */ - case 0x1c: /* FCVTAS */ - case 0x3a: /* FCVTPS */ - case 0x3b: /* FCVTZS */ - gen_helper_vfp_tosls(tcg_res, tcg_op, - tcg_constant_i32(0), tcg_fpstatus); - break; - case 0x5a: /* FCVTNU */ - case 0x5b: /* FCVTMU */ - case 0x5c: /* FCVTAU */ - case 0x7a: /* FCVTPU */ - case 0x7b: /* FCVTZU */ - gen_helper_vfp_touls(tcg_res, tcg_op, - tcg_constant_i32(0), tcg_fpstatus); - break; case 0x7c: /* URSQRTE */ gen_helper_rsqrte_u32(tcg_res, tcg_op); break; @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) case 0x5e: /* FRINT32X */ case 0x1f: /* FRINT64Z */ case 0x5f: /* FRINT64X */ + case 0x1a: /* FCVTNS */ + case 0x1b: /* FCVTMS */ + case 0x1c: /* FCVTAS */ + case 0x3a: /* FCVTPS */ + case 0x3b: /* FCVTZS */ + case 0x5a: /* FCVTNU */ + case 0x5b: /* FCVTMU */ + case 0x5c: /* FCVTAU */ + case 0x7a: /* FCVTPU */ + case 0x7b: /* FCVTZU */ g_assert_not_reached(); } } @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc_fp16(DisasContext *s, uint32_t insn) case 0x3d: /* FRECPE */ case 0x3f: /* FRECPX */ break; - case 0x1a: /* FCVTNS */ - rmode = FPROUNDING_TIEEVEN; - break; - case 0x1b: /* FCVTMS */ - rmode = FPROUNDING_NEGINF; - break; - case 0x1c: /* FCVTAS */ - rmode = FPROUNDING_TIEAWAY; - break; - case 0x3a: /* FCVTPS */ - rmode = FPROUNDING_POSINF; - break; - case 0x3b: /* FCVTZS */ - rmode = FPROUNDING_ZERO; - break; - case 0x5a: /* FCVTNU */ - rmode = FPROUNDING_TIEEVEN; - break; - case 0x5b: /* FCVTMU */ - rmode = FPROUNDING_NEGINF; - break; - case 0x5c: /* FCVTAU */ - rmode = FPROUNDING_TIEAWAY; - break; - case 0x7a: /* FCVTPU */ - rmode = FPROUNDING_POSINF; - break; - case 0x7b: /* FCVTZU */ - rmode = FPROUNDING_ZERO; - break; case 0x7d: /* FRSQRTE */ break; default: @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc_fp16(DisasContext *s, uint32_t insn) case 0x79: /* FRINTI */ case 0x1d: /* SCVTF */ case 0x5d: /* UCVTF */ + case 0x1a: /* FCVTNS */ + case 0x1b: /* FCVTMS */ + case 0x1c: /* FCVTAS */ + case 0x3a: /* FCVTPS */ + case 0x3b: /* FCVTZS */ + case 0x5a: /* FCVTNU */ + case 0x5b: /* FCVTMU */ + case 0x5c: /* FCVTAU */ + case 0x7a: /* FCVTPU */ + case 0x7b: /* FCVTZU */ unallocated_encoding(s); return; } @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc_fp16(DisasContext *s, uint32_t insn) read_vec_element_i32(s, tcg_op, rn, pass, MO_16); switch (fpop) { - case 0x1a: /* FCVTNS */ - case 0x1b: /* FCVTMS */ - case 0x1c: /* FCVTAS */ - case 0x3a: /* FCVTPS */ - case 0x3b: /* FCVTZS */ - gen_helper_advsimd_f16tosinth(tcg_res, tcg_op, tcg_fpstatus); - break; case 0x3d: /* FRECPE */ gen_helper_recpe_f16(tcg_res, tcg_op, tcg_fpstatus); break; - case 0x5a: /* FCVTNU */ - case 0x5b: /* FCVTMU */ - case 0x5c: /* FCVTAU */ - case 0x7a: /* FCVTPU */ - case 0x7b: /* FCVTZU */ - gen_helper_advsimd_f16touinth(tcg_res, tcg_op, tcg_fpstatus); - break; case 0x7d: /* FRSQRTE */ gen_helper_rsqrte_f16(tcg_res, tcg_op, tcg_fpstatus); break; @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc_fp16(DisasContext *s, uint32_t insn) case 0x58: /* FRINTA */ case 0x79: /* FRINTI */ case 0x59: /* FRINTX */ + case 0x1a: /* FCVTNS */ + case 0x1b: /* FCVTMS */ + case 0x1c: /* FCVTAS */ + case 0x3a: /* FCVTPS */ + case 0x3b: /* FCVTZS */ + case 0x5a: /* FCVTNU */ + case 0x5b: /* FCVTMU */ + case 0x5c: /* FCVTAU */ + case 0x7a: /* FCVTPU */ + case 0x7b: /* FCVTZU */ g_assert_not_reached(); } diff --git a/target/arm/tcg/vec_helper.c b/target/arm/tcg/vec_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/vec_helper.c +++ b/target/arm/tcg/vec_helper.c @@ -XXX,XX +XXX,XX @@ DO_VCVT_FIXED(gvec_vcvt_rz_hu, helper_vfp_touhh_round_to_zero, uint16_t) clear_tail(d, oprsz, simd_maxsz(desc)); \ } +DO_VCVT_RMODE(gvec_vcvt_rm_sd, helper_vfp_tosqd, uint64_t) +DO_VCVT_RMODE(gvec_vcvt_rm_ud, helper_vfp_touqd, uint64_t) DO_VCVT_RMODE(gvec_vcvt_rm_ss, helper_vfp_tosls, uint32_t) DO_VCVT_RMODE(gvec_vcvt_rm_us, helper_vfp_touls, uint32_t) DO_VCVT_RMODE(gvec_vcvt_rm_sh, helper_vfp_toshh, uint16_t) -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> This includes FCMEQ, FCMGT, FCMGE, FCMLT, FCMLE. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-66-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/helper.h | 5 + target/arm/tcg/a64.decode | 30 ++++ target/arm/tcg/translate-a64.c | 249 +++++++++++++-------------------- target/arm/tcg/vec_helper.c | 4 +- 4 files changed, 138 insertions(+), 150 deletions(-) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_4(gvec_frsqrte_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_fcgt0_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_fcgt0_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_fcgt0_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_fcge0_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_fcge0_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_fcge0_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_fceq0_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_fceq0_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_fceq0_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_fcle0_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_fcle0_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_fcle0_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_fclt0_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_fclt0_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_fclt0_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_fadd_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_fadd_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ UQXTN_s 0111 1110 ..1 00001 01001 0 ..... ..... @rr_e FCVTXN_s 0111 1110 011 00001 01101 0 ..... ..... @rr_s +FCMGT0_s 0101 1110 111 11000 11001 0 ..... ..... @rr_h +FCMGT0_s 0101 1110 1.1 00000 11001 0 ..... ..... @rr_sd + +FCMGE0_s 0111 1110 111 11000 11001 0 ..... ..... @rr_h +FCMGE0_s 0111 1110 1.1 00000 11001 0 ..... ..... @rr_sd + +FCMEQ0_s 0101 1110 111 11000 11011 0 ..... ..... @rr_h +FCMEQ0_s 0101 1110 1.1 00000 11011 0 ..... ..... @rr_sd + +FCMLE0_s 0111 1110 111 11000 11011 0 ..... ..... @rr_h +FCMLE0_s 0111 1110 1.1 00000 11011 0 ..... ..... @rr_sd + +FCMLT0_s 0101 1110 111 11000 11101 0 ..... ..... @rr_h +FCMLT0_s 0101 1110 1.1 00000 11101 0 ..... ..... @rr_sd + @icvt_h . ....... .. ...... ...... rn:5 rd:5 \ &fcvt sf=0 esz=1 shift=0 @icvt_sd . ....... .. ...... ...... rn:5 rd:5 \ @@ -XXX,XX +XXX,XX @@ FCVTAS_vi 0.00 1110 0.1 00001 11001 0 ..... ..... @qrr_sd FCVTAU_vi 0.10 1110 011 11001 11001 0 ..... ..... @qrr_h FCVTAU_vi 0.10 1110 0.1 00001 11001 0 ..... ..... @qrr_sd +FCMGT0_v 0.00 1110 111 11000 11001 0 ..... ..... @qrr_h +FCMGT0_v 0.00 1110 1.1 00000 11001 0 ..... ..... @qrr_sd + +FCMGE0_v 0.10 1110 111 11000 11001 0 ..... ..... @qrr_h +FCMGE0_v 0.10 1110 1.1 00000 11001 0 ..... ..... @qrr_sd + +FCMEQ0_v 0.00 1110 111 11000 11011 0 ..... ..... @qrr_h +FCMEQ0_v 0.00 1110 1.1 00000 11011 0 ..... ..... @qrr_sd + +FCMLE0_v 0.10 1110 111 11000 11011 0 ..... ..... @qrr_h +FCMLE0_v 0.10 1110 1.1 00000 11011 0 ..... ..... @qrr_sd + +FCMLT0_v 0.00 1110 111 11000 11101 0 ..... ..... @qrr_h +FCMLT0_v 0.00 1110 1.1 00000 11101 0 ..... ..... @qrr_sd + &fcvt_q rd rn esz q shift @fcvtq_h . q:1 . ...... 001 .... ...... rn:5 rd:5 \ &fcvt_q esz=1 shift=%fcvt_f_sh_h diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static const FPScalar f_scalar_frsqrts = { }; TRANS(FRSQRTS_s, do_fp3_scalar, a, &f_scalar_frsqrts) +static bool do_fcmp0_s(DisasContext *s, arg_rr_e *a, + const FPScalar *f, bool swap) +{ + switch (a->esz) { + case MO_64: + if (fp_access_check(s)) { + TCGv_i64 t0 = read_fp_dreg(s, a->rn); + TCGv_i64 t1 = tcg_constant_i64(0); + if (swap) { + f->gen_d(t0, t1, t0, fpstatus_ptr(FPST_FPCR)); + } else { + f->gen_d(t0, t0, t1, fpstatus_ptr(FPST_FPCR)); + } + write_fp_dreg(s, a->rd, t0); + } + break; + case MO_32: + if (fp_access_check(s)) { + TCGv_i32 t0 = read_fp_sreg(s, a->rn); + TCGv_i32 t1 = tcg_constant_i32(0); + if (swap) { + f->gen_s(t0, t1, t0, fpstatus_ptr(FPST_FPCR)); + } else { + f->gen_s(t0, t0, t1, fpstatus_ptr(FPST_FPCR)); + } + write_fp_sreg(s, a->rd, t0); + } + break; + case MO_16: + if (!dc_isar_feature(aa64_fp16, s)) { + return false; + } + if (fp_access_check(s)) { + TCGv_i32 t0 = read_fp_hreg(s, a->rn); + TCGv_i32 t1 = tcg_constant_i32(0); + if (swap) { + f->gen_h(t0, t1, t0, fpstatus_ptr(FPST_FPCR_F16)); + } else { + f->gen_h(t0, t0, t1, fpstatus_ptr(FPST_FPCR_F16)); + } + write_fp_sreg(s, a->rd, t0); + } + break; + default: + return false; + } + return true; +} + +TRANS(FCMEQ0_s, do_fcmp0_s, a, &f_scalar_fcmeq, false) +TRANS(FCMGT0_s, do_fcmp0_s, a, &f_scalar_fcmgt, false) +TRANS(FCMGE0_s, do_fcmp0_s, a, &f_scalar_fcmge, false) +TRANS(FCMLT0_s, do_fcmp0_s, a, &f_scalar_fcmgt, true) +TRANS(FCMLE0_s, do_fcmp0_s, a, &f_scalar_fcmge, true) + static bool do_satacc_s(DisasContext *s, arg_rrr_e *a, MemOp sgn_n, MemOp sgn_m, void (*gen_bhs)(TCGv_i64, TCGv_i64, TCGv_i64, TCGv_i64, MemOp), @@ -XXX,XX +XXX,XX @@ TRANS(FCVTAS_vi, do_gvec_op2_fpst, TRANS(FCVTAU_vi, do_gvec_op2_fpst, a->esz, a->q, a->rd, a->rn, float_round_ties_away, f_fcvt_u_vi) -static void handle_2misc_fcmp_zero(DisasContext *s, int opcode, - bool is_scalar, bool is_u, bool is_q, - int size, int rn, int rd) -{ - bool is_double = (size == MO_64); - TCGv_ptr fpst; +static gen_helper_gvec_2_ptr * const f_fceq0[] = { + gen_helper_gvec_fceq0_h, + gen_helper_gvec_fceq0_s, + gen_helper_gvec_fceq0_d, +}; +TRANS(FCMEQ0_v, do_gvec_op2_fpst, a->esz, a->q, a->rd, a->rn, 0, f_fceq0) - if (!fp_access_check(s)) { - return; - } +static gen_helper_gvec_2_ptr * const f_fcgt0[] = { + gen_helper_gvec_fcgt0_h, + gen_helper_gvec_fcgt0_s, + gen_helper_gvec_fcgt0_d, +}; +TRANS(FCMGT0_v, do_gvec_op2_fpst, a->esz, a->q, a->rd, a->rn, 0, f_fcgt0) - fpst = fpstatus_ptr(size == MO_16 ? FPST_FPCR_F16 : FPST_FPCR); +static gen_helper_gvec_2_ptr * const f_fcge0[] = { + gen_helper_gvec_fcge0_h, + gen_helper_gvec_fcge0_s, + gen_helper_gvec_fcge0_d, +}; +TRANS(FCMGE0_v, do_gvec_op2_fpst, a->esz, a->q, a->rd, a->rn, 0, f_fcge0) - if (is_double) { - TCGv_i64 tcg_op = tcg_temp_new_i64(); - TCGv_i64 tcg_zero = tcg_constant_i64(0); - TCGv_i64 tcg_res = tcg_temp_new_i64(); - NeonGenTwoDoubleOpFn *genfn; - bool swap = false; - int pass; +static gen_helper_gvec_2_ptr * const f_fclt0[] = { + gen_helper_gvec_fclt0_h, + gen_helper_gvec_fclt0_s, + gen_helper_gvec_fclt0_d, +}; +TRANS(FCMLT0_v, do_gvec_op2_fpst, a->esz, a->q, a->rd, a->rn, 0, f_fclt0) - switch (opcode) { - case 0x2e: /* FCMLT (zero) */ - swap = true; - /* fallthrough */ - case 0x2c: /* FCMGT (zero) */ - genfn = gen_helper_neon_cgt_f64; - break; - case 0x2d: /* FCMEQ (zero) */ - genfn = gen_helper_neon_ceq_f64; - break; - case 0x6d: /* FCMLE (zero) */ - swap = true; - /* fall through */ - case 0x6c: /* FCMGE (zero) */ - genfn = gen_helper_neon_cge_f64; - break; - default: - g_assert_not_reached(); - } - - for (pass = 0; pass < (is_scalar ? 1 : 2); pass++) { - read_vec_element(s, tcg_op, rn, pass, MO_64); - if (swap) { - genfn(tcg_res, tcg_zero, tcg_op, fpst); - } else { - genfn(tcg_res, tcg_op, tcg_zero, fpst); - } - write_vec_element(s, tcg_res, rd, pass, MO_64); - } - - clear_vec_high(s, !is_scalar, rd); - } else { - TCGv_i32 tcg_op = tcg_temp_new_i32(); - TCGv_i32 tcg_zero = tcg_constant_i32(0); - TCGv_i32 tcg_res = tcg_temp_new_i32(); - NeonGenTwoSingleOpFn *genfn; - bool swap = false; - int pass, maxpasses; - - if (size == MO_16) { - switch (opcode) { - case 0x2e: /* FCMLT (zero) */ - swap = true; - /* fall through */ - case 0x2c: /* FCMGT (zero) */ - genfn = gen_helper_advsimd_cgt_f16; - break; - case 0x2d: /* FCMEQ (zero) */ - genfn = gen_helper_advsimd_ceq_f16; - break; - case 0x6d: /* FCMLE (zero) */ - swap = true; - /* fall through */ - case 0x6c: /* FCMGE (zero) */ - genfn = gen_helper_advsimd_cge_f16; - break; - default: - g_assert_not_reached(); - } - } else { - switch (opcode) { - case 0x2e: /* FCMLT (zero) */ - swap = true; - /* fall through */ - case 0x2c: /* FCMGT (zero) */ - genfn = gen_helper_neon_cgt_f32; - break; - case 0x2d: /* FCMEQ (zero) */ - genfn = gen_helper_neon_ceq_f32; - break; - case 0x6d: /* FCMLE (zero) */ - swap = true; - /* fall through */ - case 0x6c: /* FCMGE (zero) */ - genfn = gen_helper_neon_cge_f32; - break; - default: - g_assert_not_reached(); - } - } - - if (is_scalar) { - maxpasses = 1; - } else { - int vector_size = 8 << is_q; - maxpasses = vector_size >> size; - } - - for (pass = 0; pass < maxpasses; pass++) { - read_vec_element_i32(s, tcg_op, rn, pass, size); - if (swap) { - genfn(tcg_res, tcg_zero, tcg_op, fpst); - } else { - genfn(tcg_res, tcg_op, tcg_zero, fpst); - } - if (is_scalar) { - write_fp_sreg(s, rd, tcg_res); - } else { - write_vec_element_i32(s, tcg_res, rd, pass, size); - } - } - - if (!is_scalar) { - clear_vec_high(s, is_q, rd); - } - } -} +static gen_helper_gvec_2_ptr * const f_fcle0[] = { + gen_helper_gvec_fcle0_h, + gen_helper_gvec_fcle0_s, + gen_helper_gvec_fcle0_d, +}; +TRANS(FCMLE0_v, do_gvec_op2_fpst, a->esz, a->q, a->rd, a->rn, 0, f_fcle0) static void handle_2misc_reciprocal(DisasContext *s, int opcode, bool is_scalar, bool is_u, bool is_q, @@ -XXX,XX +XXX,XX @@ static void disas_simd_scalar_two_reg_misc(DisasContext *s, uint32_t insn) opcode |= (extract32(size, 1, 1) << 5) | (u << 6); size = extract32(size, 0, 1) ? 3 : 2; switch (opcode) { - case 0x2c: /* FCMGT (zero) */ - case 0x2d: /* FCMEQ (zero) */ - case 0x2e: /* FCMLT (zero) */ - case 0x6c: /* FCMGE (zero) */ - case 0x6d: /* FCMLE (zero) */ - handle_2misc_fcmp_zero(s, opcode, true, u, true, size, rn, rd); - return; case 0x3d: /* FRECPE */ case 0x3f: /* FRECPX */ case 0x7d: /* FRSQRTE */ @@ -XXX,XX +XXX,XX @@ static void disas_simd_scalar_two_reg_misc(DisasContext *s, uint32_t insn) case 0x56: /* FCVTXN, FCVTXN2 */ case 0x1d: /* SCVTF */ case 0x5d: /* UCVTF */ + case 0x2c: /* FCMGT (zero) */ + case 0x2d: /* FCMEQ (zero) */ + case 0x2e: /* FCMLT (zero) */ + case 0x6c: /* FCMGE (zero) */ + case 0x6d: /* FCMLE (zero) */ default: unallocated_encoding(s); return; @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) opcode |= (extract32(size, 1, 1) << 5) | (u << 6); size = is_double ? 3 : 2; switch (opcode) { - case 0x2c: /* FCMGT (zero) */ - case 0x2d: /* FCMEQ (zero) */ - case 0x2e: /* FCMLT (zero) */ - case 0x6c: /* FCMGE (zero) */ - case 0x6d: /* FCMLE (zero) */ - if (size == 3 && !is_q) { - unallocated_encoding(s); - return; - } - handle_2misc_fcmp_zero(s, opcode, false, u, is_q, size, rn, rd); - return; case 0x3c: /* URECPE */ if (size == 3) { unallocated_encoding(s); @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) case 0x7b: /* FCVTZU */ case 0x5c: /* FCVTAU */ case 0x1c: /* FCVTAS */ + case 0x2c: /* FCMGT (zero) */ + case 0x2d: /* FCMEQ (zero) */ + case 0x2e: /* FCMLT (zero) */ + case 0x6c: /* FCMGE (zero) */ + case 0x6d: /* FCMLE (zero) */ unallocated_encoding(s); return; } @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc_fp16(DisasContext *s, uint32_t insn) fpop = deposit32(fpop, 6, 1, u); switch (fpop) { - case 0x2c: /* FCMGT (zero) */ - case 0x2d: /* FCMEQ (zero) */ - case 0x2e: /* FCMLT (zero) */ - case 0x6c: /* FCMGE (zero) */ - case 0x6d: /* FCMLE (zero) */ - handle_2misc_fcmp_zero(s, fpop, is_scalar, 0, is_q, MO_16, rn, rd); - return; case 0x3d: /* FRECPE */ case 0x3f: /* FRECPX */ break; @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc_fp16(DisasContext *s, uint32_t insn) case 0x5c: /* FCVTAU */ case 0x7a: /* FCVTPU */ case 0x7b: /* FCVTZU */ + case 0x2c: /* FCMGT (zero) */ + case 0x2d: /* FCMEQ (zero) */ + case 0x2e: /* FCMLT (zero) */ + case 0x6c: /* FCMGE (zero) */ + case 0x6d: /* FCMLE (zero) */ unallocated_encoding(s); return; } diff --git a/target/arm/tcg/vec_helper.c b/target/arm/tcg/vec_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/vec_helper.c +++ b/target/arm/tcg/vec_helper.c @@ -XXX,XX +XXX,XX @@ DO_2OP(gvec_touszh, vfp_touszh, float16) #define DO_2OP_CMP0(FN, CMPOP, DIRN) \ WRAP_CMP0_##DIRN(FN, CMPOP, float16) \ WRAP_CMP0_##DIRN(FN, CMPOP, float32) \ + WRAP_CMP0_##DIRN(FN, CMPOP, float64) \ DO_2OP(gvec_f##FN##0_h, float16_##FN##0, float16) \ - DO_2OP(gvec_f##FN##0_s, float32_##FN##0, float32) + DO_2OP(gvec_f##FN##0_s, float32_##FN##0, float32) \ + DO_2OP(gvec_f##FN##0_d, float64_##FN##0, float64) DO_2OP_CMP0(cgt, cgt, FWD) DO_2OP_CMP0(cge, cge, FWD) -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Remove disas_simd_scalar_two_reg_misc and disas_simd_two_reg_misc_fp16 as these were the last insns decoded by those functions. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-67-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 15 ++ target/arm/tcg/translate-a64.c | 329 ++++----------------------------- 2 files changed, 53 insertions(+), 291 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ FCMLE0_s 0111 1110 1.1 00000 11011 0 ..... ..... @rr_sd FCMLT0_s 0101 1110 111 11000 11101 0 ..... ..... @rr_h FCMLT0_s 0101 1110 1.1 00000 11101 0 ..... ..... @rr_sd +FRECPE_s 0101 1110 111 11001 11011 0 ..... ..... @rr_h +FRECPE_s 0101 1110 1.1 00001 11011 0 ..... ..... @rr_sd + +FRECPX_s 0101 1110 111 11001 11111 0 ..... ..... @rr_h +FRECPX_s 0101 1110 1.1 00001 11111 0 ..... ..... @rr_sd + +FRSQRTE_s 0111 1110 111 11001 11011 0 ..... ..... @rr_h +FRSQRTE_s 0111 1110 1.1 00001 11011 0 ..... ..... @rr_sd + @icvt_h . ....... .. ...... ...... rn:5 rd:5 \ &fcvt sf=0 esz=1 shift=0 @icvt_sd . ....... .. ...... ...... rn:5 rd:5 \ @@ -XXX,XX +XXX,XX @@ FCMLE0_v 0.10 1110 1.1 00000 11011 0 ..... ..... @qrr_sd FCMLT0_v 0.00 1110 111 11000 11101 0 ..... ..... @qrr_h FCMLT0_v 0.00 1110 1.1 00000 11101 0 ..... ..... @qrr_sd +FRECPE_v 0.00 1110 111 11001 11011 0 ..... ..... @qrr_h +FRECPE_v 0.00 1110 1.1 00001 11011 0 ..... ..... @qrr_sd + +FRSQRTE_v 0.10 1110 111 11001 11011 0 ..... ..... @qrr_h +FRSQRTE_v 0.10 1110 1.1 00001 11011 0 ..... ..... @qrr_sd + &fcvt_q rd rn esz q shift @fcvtq_h . q:1 . ...... 001 .... ...... rn:5 rd:5 \ &fcvt_q esz=1 shift=%fcvt_f_sh_h diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ TRANS_FEAT(FRINT64Z_s, aa64_frint, do_fp1_scalar, a, &f_scalar_frint64, FPROUNDING_ZERO) TRANS_FEAT(FRINT64X_s, aa64_frint, do_fp1_scalar, a, &f_scalar_frint64, -1) +static const FPScalar1 f_scalar_frecpe = { + gen_helper_recpe_f16, + gen_helper_recpe_f32, + gen_helper_recpe_f64, +}; +TRANS(FRECPE_s, do_fp1_scalar, a, &f_scalar_frecpe, -1) + +static const FPScalar1 f_scalar_frecpx = { + gen_helper_frecpx_f16, + gen_helper_frecpx_f32, + gen_helper_frecpx_f64, +}; +TRANS(FRECPX_s, do_fp1_scalar, a, &f_scalar_frecpx, -1) + +static const FPScalar1 f_scalar_frsqrte = { + gen_helper_rsqrte_f16, + gen_helper_rsqrte_f32, + gen_helper_rsqrte_f64, +}; +TRANS(FRSQRTE_s, do_fp1_scalar, a, &f_scalar_frsqrte, -1) + static bool trans_FCVT_s_ds(DisasContext *s, arg_rr *a) { if (fp_access_check(s)) { @@ -XXX,XX +XXX,XX @@ static gen_helper_gvec_2_ptr * const f_fcle0[] = { }; TRANS(FCMLE0_v, do_gvec_op2_fpst, a->esz, a->q, a->rd, a->rn, 0, f_fcle0) +static gen_helper_gvec_2_ptr * const f_frecpe[] = { + gen_helper_gvec_frecpe_h, + gen_helper_gvec_frecpe_s, + gen_helper_gvec_frecpe_d, +}; +TRANS(FRECPE_v, do_gvec_op2_fpst, a->esz, a->q, a->rd, a->rn, 0, f_frecpe) + +static gen_helper_gvec_2_ptr * const f_frsqrte[] = { + gen_helper_gvec_frsqrte_h, + gen_helper_gvec_frsqrte_s, + gen_helper_gvec_frsqrte_d, +}; +TRANS(FRSQRTE_v, do_gvec_op2_fpst, a->esz, a->q, a->rd, a->rn, 0, f_frsqrte) + static void handle_2misc_reciprocal(DisasContext *s, int opcode, bool is_scalar, bool is_u, bool is_q, int size, int rn, int rd) { bool is_double = (size == 3); - TCGv_ptr fpst = fpstatus_ptr(FPST_FPCR); if (is_double) { - TCGv_i64 tcg_op = tcg_temp_new_i64(); - TCGv_i64 tcg_res = tcg_temp_new_i64(); - int pass; - - for (pass = 0; pass < (is_scalar ? 1 : 2); pass++) { - read_vec_element(s, tcg_op, rn, pass, MO_64); - switch (opcode) { - case 0x3d: /* FRECPE */ - gen_helper_recpe_f64(tcg_res, tcg_op, fpst); - break; - case 0x3f: /* FRECPX */ - gen_helper_frecpx_f64(tcg_res, tcg_op, fpst); - break; - case 0x7d: /* FRSQRTE */ - gen_helper_rsqrte_f64(tcg_res, tcg_op, fpst); - break; - default: - g_assert_not_reached(); - } - write_vec_element(s, tcg_res, rd, pass, MO_64); - } - clear_vec_high(s, !is_scalar, rd); + g_assert_not_reached(); } else { TCGv_i32 tcg_op = tcg_temp_new_i32(); TCGv_i32 tcg_res = tcg_temp_new_i32(); @@ -XXX,XX +XXX,XX @@ static void handle_2misc_reciprocal(DisasContext *s, int opcode, gen_helper_recpe_u32(tcg_res, tcg_op); break; case 0x3d: /* FRECPE */ - gen_helper_recpe_f32(tcg_res, tcg_op, fpst); - break; case 0x3f: /* FRECPX */ - gen_helper_frecpx_f32(tcg_res, tcg_op, fpst); - break; case 0x7d: /* FRSQRTE */ - gen_helper_rsqrte_f32(tcg_res, tcg_op, fpst); - break; default: g_assert_not_reached(); } @@ -XXX,XX +XXX,XX @@ static void handle_2misc_reciprocal(DisasContext *s, int opcode, } } -/* AdvSIMD scalar two reg misc - * 31 30 29 28 24 23 22 21 17 16 12 11 10 9 5 4 0 - * +-----+---+-----------+------+-----------+--------+-----+------+------+ - * | 0 1 | U | 1 1 1 1 0 | size | 1 0 0 0 0 | opcode | 1 0 | Rn | Rd | - * +-----+---+-----------+------+-----------+--------+-----+------+------+ - */ -static void disas_simd_scalar_two_reg_misc(DisasContext *s, uint32_t insn) -{ - int rd = extract32(insn, 0, 5); - int rn = extract32(insn, 5, 5); - int opcode = extract32(insn, 12, 5); - int size = extract32(insn, 22, 2); - bool u = extract32(insn, 29, 1); - - switch (opcode) { - case 0xc ... 0xf: - case 0x16 ... 0x1d: - case 0x1f: - /* Floating point: U, size[1] and opcode indicate operation; - * size[0] indicates single or double precision. - */ - opcode |= (extract32(size, 1, 1) << 5) | (u << 6); - size = extract32(size, 0, 1) ? 3 : 2; - switch (opcode) { - case 0x3d: /* FRECPE */ - case 0x3f: /* FRECPX */ - case 0x7d: /* FRSQRTE */ - if (!fp_access_check(s)) { - return; - } - handle_2misc_reciprocal(s, opcode, true, u, true, size, rn, rd); - return; - case 0x1a: /* FCVTNS */ - case 0x1b: /* FCVTMS */ - case 0x3a: /* FCVTPS */ - case 0x3b: /* FCVTZS */ - case 0x5a: /* FCVTNU */ - case 0x5b: /* FCVTMU */ - case 0x7a: /* FCVTPU */ - case 0x7b: /* FCVTZU */ - case 0x1c: /* FCVTAS */ - case 0x5c: /* FCVTAU */ - case 0x56: /* FCVTXN, FCVTXN2 */ - case 0x1d: /* SCVTF */ - case 0x5d: /* UCVTF */ - case 0x2c: /* FCMGT (zero) */ - case 0x2d: /* FCMEQ (zero) */ - case 0x2e: /* FCMLT (zero) */ - case 0x6c: /* FCMGE (zero) */ - case 0x6d: /* FCMLE (zero) */ - default: - unallocated_encoding(s); - return; - } - break; - default: - case 0x3: /* USQADD / SUQADD */ - case 0x7: /* SQABS / SQNEG */ - case 0x8: /* CMGT, CMGE */ - case 0x9: /* CMEQ, CMLE */ - case 0xa: /* CMLT */ - case 0xb: /* ABS, NEG */ - case 0x12: /* SQXTUN */ - case 0x14: /* SQXTN, UQXTN */ - unallocated_encoding(s); - return; - } - g_assert_not_reached(); -} - static void handle_2misc_widening(DisasContext *s, int opcode, bool is_q, int size, int rn, int rd) { @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) unallocated_encoding(s); return; } - /* fall through */ - case 0x3d: /* FRECPE */ - case 0x7d: /* FRSQRTE */ - if (size == 3 && !is_q) { - unallocated_encoding(s); - return; - } if (!fp_access_check(s)) { return; } @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) case 0x2e: /* FCMLT (zero) */ case 0x6c: /* FCMGE (zero) */ case 0x6d: /* FCMLE (zero) */ + case 0x3d: /* FRECPE */ + case 0x7d: /* FRSQRTE */ unallocated_encoding(s); return; } @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) } } -/* AdvSIMD [scalar] two register miscellaneous (FP16) - * - * 31 30 29 28 27 24 23 22 21 17 16 12 11 10 9 5 4 0 - * +---+---+---+---+---------+---+-------------+--------+-----+------+------+ - * | 0 | Q | U | S | 1 1 1 0 | a | 1 1 1 1 0 0 | opcode | 1 0 | Rn | Rd | - * +---+---+---+---+---------+---+-------------+--------+-----+------+------+ - * mask: 1000 1111 0111 1110 0000 1100 0000 0000 0x8f7e 0c00 - * val: 0000 1110 0111 1000 0000 1000 0000 0000 0x0e78 0800 - * - * This actually covers two groups where scalar access is governed by - * bit 28. A bunch of the instructions (float to integral) only exist - * in the vector form and are un-allocated for the scalar decode. Also - * in the scalar decode Q is always 1. - */ -static void disas_simd_two_reg_misc_fp16(DisasContext *s, uint32_t insn) -{ - int fpop, opcode, a, u; - int rn, rd; - bool is_q; - bool is_scalar; - - int pass; - TCGv_i32 tcg_rmode = NULL; - TCGv_ptr tcg_fpstatus = NULL; - bool need_fpst = true; - int rmode = -1; - - if (!dc_isar_feature(aa64_fp16, s)) { - unallocated_encoding(s); - return; - } - - rd = extract32(insn, 0, 5); - rn = extract32(insn, 5, 5); - - a = extract32(insn, 23, 1); - u = extract32(insn, 29, 1); - is_scalar = extract32(insn, 28, 1); - is_q = extract32(insn, 30, 1); - - opcode = extract32(insn, 12, 5); - fpop = deposit32(opcode, 5, 1, a); - fpop = deposit32(fpop, 6, 1, u); - - switch (fpop) { - case 0x3d: /* FRECPE */ - case 0x3f: /* FRECPX */ - break; - case 0x7d: /* FRSQRTE */ - break; - default: - case 0x2f: /* FABS */ - case 0x6f: /* FNEG */ - case 0x7f: /* FSQRT (vector) */ - case 0x18: /* FRINTN */ - case 0x19: /* FRINTM */ - case 0x38: /* FRINTP */ - case 0x39: /* FRINTZ */ - case 0x58: /* FRINTA */ - case 0x59: /* FRINTX */ - case 0x79: /* FRINTI */ - case 0x1d: /* SCVTF */ - case 0x5d: /* UCVTF */ - case 0x1a: /* FCVTNS */ - case 0x1b: /* FCVTMS */ - case 0x1c: /* FCVTAS */ - case 0x3a: /* FCVTPS */ - case 0x3b: /* FCVTZS */ - case 0x5a: /* FCVTNU */ - case 0x5b: /* FCVTMU */ - case 0x5c: /* FCVTAU */ - case 0x7a: /* FCVTPU */ - case 0x7b: /* FCVTZU */ - case 0x2c: /* FCMGT (zero) */ - case 0x2d: /* FCMEQ (zero) */ - case 0x2e: /* FCMLT (zero) */ - case 0x6c: /* FCMGE (zero) */ - case 0x6d: /* FCMLE (zero) */ - unallocated_encoding(s); - return; - } - - - /* Check additional constraints for the scalar encoding */ - if (is_scalar) { - if (!is_q) { - unallocated_encoding(s); - return; - } - } - - if (!fp_access_check(s)) { - return; - } - - if (rmode >= 0 || need_fpst) { - tcg_fpstatus = fpstatus_ptr(FPST_FPCR_F16); - } - - if (rmode >= 0) { - tcg_rmode = gen_set_rmode(rmode, tcg_fpstatus); - } - - if (is_scalar) { - TCGv_i32 tcg_op = read_fp_hreg(s, rn); - TCGv_i32 tcg_res = tcg_temp_new_i32(); - - switch (fpop) { - case 0x3d: /* FRECPE */ - gen_helper_recpe_f16(tcg_res, tcg_op, tcg_fpstatus); - break; - case 0x3f: /* FRECPX */ - gen_helper_frecpx_f16(tcg_res, tcg_op, tcg_fpstatus); - break; - case 0x7d: /* FRSQRTE */ - gen_helper_rsqrte_f16(tcg_res, tcg_op, tcg_fpstatus); - break; - default: - case 0x1a: /* FCVTNS */ - case 0x1b: /* FCVTMS */ - case 0x1c: /* FCVTAS */ - case 0x3a: /* FCVTPS */ - case 0x3b: /* FCVTZS */ - case 0x5a: /* FCVTNU */ - case 0x5b: /* FCVTMU */ - case 0x5c: /* FCVTAU */ - case 0x7a: /* FCVTPU */ - case 0x7b: /* FCVTZU */ - g_assert_not_reached(); - } - - /* limit any sign extension going on */ - tcg_gen_andi_i32(tcg_res, tcg_res, 0xffff); - write_fp_sreg(s, rd, tcg_res); - } else { - for (pass = 0; pass < (is_q ? 8 : 4); pass++) { - TCGv_i32 tcg_op = tcg_temp_new_i32(); - TCGv_i32 tcg_res = tcg_temp_new_i32(); - - read_vec_element_i32(s, tcg_op, rn, pass, MO_16); - - switch (fpop) { - case 0x3d: /* FRECPE */ - gen_helper_recpe_f16(tcg_res, tcg_op, tcg_fpstatus); - break; - case 0x7d: /* FRSQRTE */ - gen_helper_rsqrte_f16(tcg_res, tcg_op, tcg_fpstatus); - break; - default: - case 0x2f: /* FABS */ - case 0x6f: /* FNEG */ - case 0x7f: /* FSQRT */ - case 0x18: /* FRINTN */ - case 0x19: /* FRINTM */ - case 0x38: /* FRINTP */ - case 0x39: /* FRINTZ */ - case 0x58: /* FRINTA */ - case 0x79: /* FRINTI */ - case 0x59: /* FRINTX */ - case 0x1a: /* FCVTNS */ - case 0x1b: /* FCVTMS */ - case 0x1c: /* FCVTAS */ - case 0x3a: /* FCVTPS */ - case 0x3b: /* FCVTZS */ - case 0x5a: /* FCVTNU */ - case 0x5b: /* FCVTMU */ - case 0x5c: /* FCVTAU */ - case 0x7a: /* FCVTPU */ - case 0x7b: /* FCVTZU */ - g_assert_not_reached(); - } - - write_vec_element_i32(s, tcg_res, rd, pass, MO_16); - } - - clear_vec_high(s, is_q, rd); - } - - if (tcg_rmode) { - gen_restore_rmode(tcg_rmode, tcg_fpstatus); - } -} - /* C3.6 Data processing - SIMD, inc Crypto * * As the decode gets a little complex we are using a table based @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc_fp16(DisasContext *s, uint32_t insn) static const AArch64DecodeTable data_proc_simd[] = { /* pattern , mask , fn */ { 0x0e200800, 0x9f3e0c00, disas_simd_two_reg_misc }, - { 0x5e200800, 0xdf3e0c00, disas_simd_scalar_two_reg_misc }, - { 0x0e780800, 0x8f7e0c00, disas_simd_two_reg_misc_fp16 }, { 0x00000000, 0x00000000, NULL } }; -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-68-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/helper.h | 3 +++ target/arm/tcg/translate.h | 5 +++++ target/arm/tcg/gengvec.c | 16 ++++++++++++++++ target/arm/tcg/translate-neon.c | 4 ++-- target/arm/tcg/vec_helper.c | 22 ++++++++++++++++++++++ 5 files changed, 48 insertions(+), 2 deletions(-) diff --git a/target/arm/helper.h b/target/arm/helper.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_4(gvec_uminp_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_uminp_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_uminp_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(gvec_urecpe_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(gvec_ursqrte_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) + #ifdef TARGET_AARCH64 #include "tcg/helper-a64.h" #include "tcg/helper-sve.h" diff --git a/target/arm/tcg/translate.h b/target/arm/tcg/translate.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate.h +++ b/target/arm/tcg/translate.h @@ -XXX,XX +XXX,XX @@ void gen_gvec_fabs(unsigned vece, uint32_t dofs, uint32_t aofs, void gen_gvec_fneg(unsigned vece, uint32_t dofs, uint32_t aofs, uint32_t oprsz, uint32_t maxsz); +void gen_gvec_urecpe(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, + uint32_t opr_sz, uint32_t max_sz); +void gen_gvec_ursqrte(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, + uint32_t opr_sz, uint32_t max_sz); + /* * Forward to the isar_feature_* tests given a DisasContext pointer. */ diff --git a/target/arm/tcg/gengvec.c b/target/arm/tcg/gengvec.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/gengvec.c +++ b/target/arm/tcg/gengvec.c @@ -XXX,XX +XXX,XX @@ void gen_gvec_fneg(unsigned vece, uint32_t dofs, uint32_t aofs, uint64_t s_bit = 1ull << ((8 << vece) - 1); tcg_gen_gvec_xori(vece, dofs, aofs, s_bit, oprsz, maxsz); } + +void gen_gvec_urecpe(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, + uint32_t opr_sz, uint32_t max_sz) +{ + assert(vece == MO_32); + tcg_gen_gvec_2_ool(rd_ofs, rn_ofs, opr_sz, max_sz, 0, + gen_helper_gvec_urecpe_s); +} + +void gen_gvec_ursqrte(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, + uint32_t opr_sz, uint32_t max_sz) +{ + assert(vece == MO_32); + tcg_gen_gvec_2_ool(rd_ofs, rn_ofs, opr_sz, max_sz, 0, + gen_helper_gvec_ursqrte_s); +} diff --git a/target/arm/tcg/translate-neon.c b/target/arm/tcg/translate-neon.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-neon.c +++ b/target/arm/tcg/translate-neon.c @@ -XXX,XX +XXX,XX @@ static bool trans_VRECPE(DisasContext *s, arg_2misc *a) if (a->size != 2) { return false; } - return do_2misc(s, a, gen_helper_recpe_u32); + return do_2misc_vec(s, a, gen_gvec_urecpe); } static bool trans_VRSQRTE(DisasContext *s, arg_2misc *a) @@ -XXX,XX +XXX,XX @@ static bool trans_VRSQRTE(DisasContext *s, arg_2misc *a) if (a->size != 2) { return false; } - return do_2misc(s, a, gen_helper_rsqrte_u32); + return do_2misc_vec(s, a, gen_gvec_ursqrte); } #define WRAP_1OP_ENV_FN(WRAPNAME, FUNC) \ diff --git a/target/arm/tcg/vec_helper.c b/target/arm/tcg/vec_helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/vec_helper.c +++ b/target/arm/tcg/vec_helper.c @@ -XXX,XX +XXX,XX @@ void HELPER(gvec_rbit_b)(void *vd, void *vn, uint32_t desc) } clear_tail(d, opr_sz, simd_maxsz(desc)); } + +void HELPER(gvec_urecpe_s)(void *vd, void *vn, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + uint32_t *d = vd, *n = vn; + + for (i = 0; i < opr_sz / 4; ++i) { + d[i] = helper_recpe_u32(n[i]); + } + clear_tail(d, opr_sz, simd_maxsz(desc)); +} + +void HELPER(gvec_ursqrte_s)(void *vd, void *vn, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + uint32_t *d = vd, *n = vn; + + for (i = 0; i < opr_sz / 4; ++i) { + d[i] = helper_rsqrte_u32(n[i]); + } + clear_tail(d, opr_sz, simd_maxsz(desc)); +} -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Remove handle_2misc_reciprocal as these were the last insns decoded by that function. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-69-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 3 + target/arm/tcg/translate-a64.c | 139 ++------------------------------- 2 files changed, 8 insertions(+), 134 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ FRECPE_v 0.00 1110 1.1 00001 11011 0 ..... ..... @qrr_sd FRSQRTE_v 0.10 1110 111 11001 11011 0 ..... ..... @qrr_h FRSQRTE_v 0.10 1110 1.1 00001 11011 0 ..... ..... @qrr_sd +URECPE_v 0.00 1110 101 00001 11001 0 ..... ..... @qrr_s +URSQRTE_v 0.10 1110 101 00001 11001 0 ..... ..... @qrr_s + &fcvt_q rd rn esz q shift @fcvtq_h . q:1 . ...... 001 .... ...... rn:5 rd:5 \ &fcvt_q esz=1 shift=%fcvt_f_sh_h diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ TRANS(CMLE0_v, do_gvec_fn2, a, gen_gvec_cle0) TRANS(CMEQ0_v, do_gvec_fn2, a, gen_gvec_ceq0) TRANS(REV16_v, do_gvec_fn2, a, gen_gvec_rev16) TRANS(REV32_v, do_gvec_fn2, a, gen_gvec_rev32) +TRANS(URECPE_v, do_gvec_fn2, a, gen_gvec_urecpe) +TRANS(URSQRTE_v, do_gvec_fn2, a, gen_gvec_ursqrte) static bool do_gvec_fn2_bhs(DisasContext *s, arg_qrr_e *a, GVecGen2Fn *fn) { @@ -XXX,XX +XXX,XX @@ static gen_helper_gvec_2_ptr * const f_frsqrte[] = { }; TRANS(FRSQRTE_v, do_gvec_op2_fpst, a->esz, a->q, a->rd, a->rn, 0, f_frsqrte) -static void handle_2misc_reciprocal(DisasContext *s, int opcode, - bool is_scalar, bool is_u, bool is_q, - int size, int rn, int rd) -{ - bool is_double = (size == 3); - - if (is_double) { - g_assert_not_reached(); - } else { - TCGv_i32 tcg_op = tcg_temp_new_i32(); - TCGv_i32 tcg_res = tcg_temp_new_i32(); - int pass, maxpasses; - - if (is_scalar) { - maxpasses = 1; - } else { - maxpasses = is_q ? 4 : 2; - } - - for (pass = 0; pass < maxpasses; pass++) { - read_vec_element_i32(s, tcg_op, rn, pass, MO_32); - - switch (opcode) { - case 0x3c: /* URECPE */ - gen_helper_recpe_u32(tcg_res, tcg_op); - break; - case 0x3d: /* FRECPE */ - case 0x3f: /* FRECPX */ - case 0x7d: /* FRSQRTE */ - default: - g_assert_not_reached(); - } - - if (is_scalar) { - write_fp_sreg(s, rd, tcg_res); - } else { - write_vec_element_i32(s, tcg_res, rd, pass, MO_32); - } - } - if (!is_scalar) { - clear_vec_high(s, is_q, rd); - } - } -} - static void handle_2misc_widening(DisasContext *s, int opcode, bool is_q, int size, int rn, int rd) { @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) bool is_q = extract32(insn, 30, 1); int rn = extract32(insn, 5, 5); int rd = extract32(insn, 0, 5); - bool need_fpstatus = false; - int rmode = -1; - TCGv_i32 tcg_rmode; - TCGv_ptr tcg_fpstatus; switch (opcode) { case 0xc ... 0xf: @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) opcode |= (extract32(size, 1, 1) << 5) | (u << 6); size = is_double ? 3 : 2; switch (opcode) { - case 0x3c: /* URECPE */ - if (size == 3) { - unallocated_encoding(s); - return; - } - if (!fp_access_check(s)) { - return; - } - handle_2misc_reciprocal(s, opcode, false, u, is_q, size, rn, rd); - return; case 0x17: /* FCVTL, FCVTL2 */ if (!fp_access_check(s)) { return; } handle_2misc_widening(s, opcode, is_q, size, rn, rd); return; - case 0x7c: /* URSQRTE */ - if (size == 3) { - unallocated_encoding(s); - return; - } - break; default: case 0x16: /* FCVTN, FCVTN2 */ case 0x36: /* BFCVTN, BFCVTN2 */ @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) case 0x6d: /* FCMLE (zero) */ case 0x3d: /* FRECPE */ case 0x7d: /* FRSQRTE */ + case 0x3c: /* URECPE */ + case 0x7c: /* URSQRTE */ unallocated_encoding(s); return; } @@ -XXX,XX +XXX,XX @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) unallocated_encoding(s); return; } - - if (!fp_access_check(s)) { - return; - } - - if (need_fpstatus || rmode >= 0) { - tcg_fpstatus = fpstatus_ptr(FPST_FPCR); - } else { - tcg_fpstatus = NULL; - } - if (rmode >= 0) { - tcg_rmode = gen_set_rmode(rmode, tcg_fpstatus); - } else { - tcg_rmode = NULL; - } - - { - int pass; - - assert(size == 2); - for (pass = 0; pass < (is_q ? 4 : 2); pass++) { - TCGv_i32 tcg_op = tcg_temp_new_i32(); - TCGv_i32 tcg_res = tcg_temp_new_i32(); - - read_vec_element_i32(s, tcg_op, rn, pass, MO_32); - - { - /* Special cases for 32 bit elements */ - switch (opcode) { - case 0x7c: /* URSQRTE */ - gen_helper_rsqrte_u32(tcg_res, tcg_op); - break; - default: - case 0x7: /* SQABS, SQNEG */ - case 0x2f: /* FABS */ - case 0x6f: /* FNEG */ - case 0x7f: /* FSQRT */ - case 0x18: /* FRINTN */ - case 0x19: /* FRINTM */ - case 0x38: /* FRINTP */ - case 0x39: /* FRINTZ */ - case 0x58: /* FRINTA */ - case 0x79: /* FRINTI */ - case 0x59: /* FRINTX */ - case 0x1e: /* FRINT32Z */ - case 0x5e: /* FRINT32X */ - case 0x1f: /* FRINT64Z */ - case 0x5f: /* FRINT64X */ - case 0x1a: /* FCVTNS */ - case 0x1b: /* FCVTMS */ - case 0x1c: /* FCVTAS */ - case 0x3a: /* FCVTPS */ - case 0x3b: /* FCVTZS */ - case 0x5a: /* FCVTNU */ - case 0x5b: /* FCVTMU */ - case 0x5c: /* FCVTAU */ - case 0x7a: /* FCVTPU */ - case 0x7b: /* FCVTZU */ - g_assert_not_reached(); - } - } - write_vec_element_i32(s, tcg_res, rd, pass, MO_32); - } - } - clear_vec_high(s, is_q, rd); - - if (tcg_rmode) { - gen_restore_rmode(tcg_rmode, tcg_fpstatus); - } + g_assert_not_reached(); } /* C3.6 Data processing - SIMD, inc Crypto -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Remove lookup_disas_fn, handle_2misc_widening, disas_simd_two_reg_misc, disas_data_proc_simd, disas_data_proc_simd_fp, disas_a64_legacy, as this is the final insn to be converted. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241211163036.2297116-70-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/a64.decode | 2 + target/arm/tcg/translate-a64.c | 202 +++------------------------------ 2 files changed, 18 insertions(+), 186 deletions(-) diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/a64.decode +++ b/target/arm/tcg/a64.decode @@ -XXX,XX +XXX,XX @@ FRSQRTE_v 0.10 1110 1.1 00001 11011 0 ..... ..... @qrr_sd URECPE_v 0.00 1110 101 00001 11001 0 ..... ..... @qrr_s URSQRTE_v 0.10 1110 101 00001 11001 0 ..... ..... @qrr_s +FCVTL_v 0.00 1110 0.1 00001 01111 0 ..... ..... @qrr_sd + &fcvt_q rd rn esz q shift @fcvtq_h . q:1 . ...... 001 .... ...... rn:5 rd:5 \ &fcvt_q esz=1 shift=%fcvt_f_sh_h diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/translate-a64.c +++ b/target/arm/tcg/translate-a64.c @@ -XXX,XX +XXX,XX @@ static inline void gen_check_sp_alignment(DisasContext *s) */ } -/* - * This provides a simple table based table lookup decoder. It is - * intended to be used when the relevant bits for decode are too - * awkwardly placed and switch/if based logic would be confusing and - * deeply nested. Since it's a linear search through the table, tables - * should be kept small. - * - * It returns the first handler where insn & mask == pattern, or - * NULL if there is no match. - * The table is terminated by an empty mask (i.e. 0) - */ -static inline AArch64DecodeFn *lookup_disas_fn(const AArch64DecodeTable *table, - uint32_t insn) -{ - const AArch64DecodeTable *tptr = table; - - while (tptr->mask) { - if ((insn & tptr->mask) == tptr->pattern) { - return tptr->disas_fn; - } - tptr++; - } - return NULL; -} - /* * The instruction disassembly implemented here matches * the instruction encoding classifications in chapter C4 @@ -XXX,XX +XXX,XX @@ static gen_helper_gvec_2_ptr * const f_frsqrte[] = { }; TRANS(FRSQRTE_v, do_gvec_op2_fpst, a->esz, a->q, a->rd, a->rn, 0, f_frsqrte) -static void handle_2misc_widening(DisasContext *s, int opcode, bool is_q, - int size, int rn, int rd) +static bool trans_FCVTL_v(DisasContext *s, arg_qrr_e *a) { /* Handle 2-reg-misc ops which are widening (so each size element * in the source becomes a 2*size element in the destination. @@ -XXX,XX +XXX,XX @@ static void handle_2misc_widening(DisasContext *s, int opcode, bool is_q, */ int pass; - if (size == 3) { + if (!fp_access_check(s)) { + return true; + } + + if (a->esz == MO_64) { /* 32 -> 64 bit fp conversion */ TCGv_i64 tcg_res[2]; - int srcelt = is_q ? 2 : 0; + TCGv_i32 tcg_op = tcg_temp_new_i32(); + int srcelt = a->q ? 2 : 0; for (pass = 0; pass < 2; pass++) { - TCGv_i32 tcg_op = tcg_temp_new_i32(); tcg_res[pass] = tcg_temp_new_i64(); - - read_vec_element_i32(s, tcg_op, rn, srcelt + pass, MO_32); + read_vec_element_i32(s, tcg_op, a->rn, srcelt + pass, MO_32); gen_helper_vfp_fcvtds(tcg_res[pass], tcg_op, tcg_env); } for (pass = 0; pass < 2; pass++) { - write_vec_element(s, tcg_res[pass], rd, pass, MO_64); + write_vec_element(s, tcg_res[pass], a->rd, pass, MO_64); } } else { /* 16 -> 32 bit fp conversion */ - int srcelt = is_q ? 4 : 0; + int srcelt = a->q ? 4 : 0; TCGv_i32 tcg_res[4]; TCGv_ptr fpst = fpstatus_ptr(FPST_FPCR); TCGv_i32 ahp = get_ahp_flag(); for (pass = 0; pass < 4; pass++) { tcg_res[pass] = tcg_temp_new_i32(); - - read_vec_element_i32(s, tcg_res[pass], rn, srcelt + pass, MO_16); + read_vec_element_i32(s, tcg_res[pass], a->rn, srcelt + pass, MO_16); gen_helper_vfp_fcvt_f16_to_f32(tcg_res[pass], tcg_res[pass], fpst, ahp); } for (pass = 0; pass < 4; pass++) { - write_vec_element_i32(s, tcg_res[pass], rd, pass, MO_32); + write_vec_element_i32(s, tcg_res[pass], a->rd, pass, MO_32); } } -} - -/* AdvSIMD two reg misc - * 31 30 29 28 24 23 22 21 17 16 12 11 10 9 5 4 0 - * +---+---+---+-----------+------+-----------+--------+-----+------+------+ - * | 0 | Q | U | 0 1 1 1 0 | size | 1 0 0 0 0 | opcode | 1 0 | Rn | Rd | - * +---+---+---+-----------+------+-----------+--------+-----+------+------+ - */ -static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn) -{ - int size = extract32(insn, 22, 2); - int opcode = extract32(insn, 12, 5); - bool u = extract32(insn, 29, 1); - bool is_q = extract32(insn, 30, 1); - int rn = extract32(insn, 5, 5); - int rd = extract32(insn, 0, 5); - - switch (opcode) { - case 0xc ... 0xf: - case 0x16 ... 0x1f: - { - /* Floating point: U, size[1] and opcode indicate operation; - * size[0] indicates single or double precision. - */ - int is_double = extract32(size, 0, 1); - opcode |= (extract32(size, 1, 1) << 5) | (u << 6); - size = is_double ? 3 : 2; - switch (opcode) { - case 0x17: /* FCVTL, FCVTL2 */ - if (!fp_access_check(s)) { - return; - } - handle_2misc_widening(s, opcode, is_q, size, rn, rd); - return; - default: - case 0x16: /* FCVTN, FCVTN2 */ - case 0x36: /* BFCVTN, BFCVTN2 */ - case 0x56: /* FCVTXN, FCVTXN2 */ - case 0x2f: /* FABS */ - case 0x6f: /* FNEG */ - case 0x7f: /* FSQRT */ - case 0x18: /* FRINTN */ - case 0x19: /* FRINTM */ - case 0x38: /* FRINTP */ - case 0x39: /* FRINTZ */ - case 0x59: /* FRINTX */ - case 0x79: /* FRINTI */ - case 0x58: /* FRINTA */ - case 0x1e: /* FRINT32Z */ - case 0x1f: /* FRINT64Z */ - case 0x5e: /* FRINT32X */ - case 0x5f: /* FRINT64X */ - case 0x1d: /* SCVTF */ - case 0x5d: /* UCVTF */ - case 0x1a: /* FCVTNS */ - case 0x1b: /* FCVTMS */ - case 0x3a: /* FCVTPS */ - case 0x3b: /* FCVTZS */ - case 0x5a: /* FCVTNU */ - case 0x5b: /* FCVTMU */ - case 0x7a: /* FCVTPU */ - case 0x7b: /* FCVTZU */ - case 0x5c: /* FCVTAU */ - case 0x1c: /* FCVTAS */ - case 0x2c: /* FCMGT (zero) */ - case 0x2d: /* FCMEQ (zero) */ - case 0x2e: /* FCMLT (zero) */ - case 0x6c: /* FCMGE (zero) */ - case 0x6d: /* FCMLE (zero) */ - case 0x3d: /* FRECPE */ - case 0x7d: /* FRSQRTE */ - case 0x3c: /* URECPE */ - case 0x7c: /* URSQRTE */ - unallocated_encoding(s); - return; - } - break; - } - default: - case 0x0: /* REV64, REV32 */ - case 0x1: /* REV16 */ - case 0x2: /* SADDLP, UADDLP */ - case 0x3: /* SUQADD, USQADD */ - case 0x4: /* CLS, CLZ */ - case 0x5: /* CNT, NOT, RBIT */ - case 0x6: /* SADALP, UADALP */ - case 0x7: /* SQABS, SQNEG */ - case 0x8: /* CMGT, CMGE */ - case 0x9: /* CMEQ, CMLE */ - case 0xa: /* CMLT */ - case 0xb: /* ABS, NEG */ - case 0x12: /* XTN, XTN2, SQXTUN, SQXTUN2 */ - case 0x13: /* SHLL, SHLL2 */ - case 0x14: /* SQXTN, SQXTN2, UQXTN, UQXTN2 */ - unallocated_encoding(s); - return; - } - g_assert_not_reached(); -} - -/* C3.6 Data processing - SIMD, inc Crypto - * - * As the decode gets a little complex we are using a table based - * approach for this part of the decode. - */ -static const AArch64DecodeTable data_proc_simd[] = { - /* pattern , mask , fn */ - { 0x0e200800, 0x9f3e0c00, disas_simd_two_reg_misc }, - { 0x00000000, 0x00000000, NULL } -}; - -static void disas_data_proc_simd(DisasContext *s, uint32_t insn) -{ - /* Note that this is called with all non-FP cases from - * table C3-6 so it must UNDEF for entries not specifically - * allocated to instructions in that table. - */ - AArch64DecodeFn *fn = lookup_disas_fn(&data_proc_simd[0], insn); - if (fn) { - fn(s, insn); - } else { - unallocated_encoding(s); - } -} - -/* C3.6 Data processing - SIMD and floating point */ -static void disas_data_proc_simd_fp(DisasContext *s, uint32_t insn) -{ - if (extract32(insn, 28, 1) == 1 && extract32(insn, 30, 1) == 0) { - unallocated_encoding(s); /* in decodetree */ - } else { - /* SIMD, including crypto */ - disas_data_proc_simd(s, insn); - } + clear_vec_high(s, true, a->rd); + return true; } static bool trans_OK(DisasContext *s, arg_OK *a) @@ -XXX,XX +XXX,XX @@ static bool btype_destination_ok(uint32_t insn, bool bt, int btype) return false; } -/* C3.1 A64 instruction index by encoding */ -static void disas_a64_legacy(DisasContext *s, uint32_t insn) -{ - switch (extract32(insn, 25, 4)) { - case 0x7: - case 0xf: /* Data processing - SIMD and floating point */ - disas_data_proc_simd_fp(s, insn); - break; - default: - unallocated_encoding(s); - break; - } -} - static void aarch64_tr_init_disas_context(DisasContextBase *dcbase, CPUState *cpu) { @@ -XXX,XX +XXX,XX @@ static void aarch64_tr_translate_insn(DisasContextBase *dcbase, CPUState *cpu) if (!disas_a64(s, insn) && !disas_sme(s, insn) && !disas_sve(s, insn)) { - disas_a64_legacy(s, insn); + unallocated_encoding(s); } /* -- 2.34.1 From: Richard Henderson <richard.henderson@linaro.org> Softfloat has native support for round-to-odd. Use it. Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241206031428.78634-1-richard.henderson@linaro.org Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- target/arm/tcg/helper-a64.c | 18 ++++-------------- 1 file changed, 4 insertions(+), 14 deletions(-) diff --git a/target/arm/tcg/helper-a64.c b/target/arm/tcg/helper-a64.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/helper-a64.c +++ b/target/arm/tcg/helper-a64.c @@ -XXX,XX +XXX,XX @@ float64 HELPER(frecpx_f64)(float64 a, void *fpstp) float32 HELPER(fcvtx_f64_to_f32)(float64 a, CPUARMState *env) { - /* Von Neumann rounding is implemented by using round-to-zero - * and then setting the LSB of the result if Inexact was raised. - */ float32 r; float_status *fpst = &env->vfp.fp_status; - float_status tstat = *fpst; - int exflags; + int old = get_float_rounding_mode(fpst); - set_float_rounding_mode(float_round_to_zero, &tstat); - set_float_exception_flags(0, &tstat); - r = float64_to_float32(a, &tstat); - exflags = get_float_exception_flags(&tstat); - if (exflags & float_flag_inexact) { - r = make_float32(float32_val(r) | 1); - } - exflags |= get_float_exception_flags(fpst); - set_float_exception_flags(exflags, fpst); + set_float_rounding_mode(float_round_to_odd, fpst); + r = float64_to_float32(a, fpst); + set_float_rounding_mode(old, fpst); return r; } -- 2.34.1 From: Pierrick Bouvier <pierrick.bouvier@linaro.org> www.orangepi.org does not support https, it's expected to stick to http. Reviewed-by: Niek Linnenbank <nieklinnenbank@gmail.com> Signed-off-by: Pierrick Bouvier <pierrick.bouvier@linaro.org> Message-id: 20241206192254.3889131-2-pierrick.bouvier@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- docs/system/arm/orangepi.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/system/arm/orangepi.rst b/docs/system/arm/orangepi.rst index XXXXXXX..XXXXXXX 100644 --- a/docs/system/arm/orangepi.rst +++ b/docs/system/arm/orangepi.rst @@ -XXX,XX +XXX,XX @@ Orange Pi PC images Note that the mainline kernel does not have a root filesystem. You may provide it with an official Orange Pi PC image from the official website: - http://www.orangepi.org/downloadresources/ + http://www.orangepi.org/html/serviceAndSupport/index.html Another possibility is to run an Armbian image for Orange Pi PC which can be downloaded from: @@ -XXX,XX +XXX,XX @@ including the Orange Pi PC. NetBSD 9.0 is known to work best for the Orange Pi P board and provides a fully working system with serial console, networking and storage. For the Orange Pi PC machine, get the 'evbarm-earmv7hf' based image from: - https://cdn.netbsd.org/pub/NetBSD/NetBSD-9.0/evbarm-earmv7hf/binary/gzimg/armv7.img.gz + https://archive.netbsd.org/pub/NetBSD-archive/NetBSD-9.0/evbarm-earmv7hf/binary/gzimg/armv7.img.gz The image requires manually installing U-Boot in the image. Build U-Boot with the orangepi_pc_defconfig configuration as described in the previous section. -- 2.34.1 From: Pierrick Bouvier <pierrick.bouvier@linaro.org> Reviewed-by: Cédric Le Goater <clg@redhat.com> Signed-off-by: Pierrick Bouvier <pierrick.bouvier@linaro.org> Message-id: 20241206192254.3889131-3-pierrick.bouvier@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- docs/system/arm/fby35.rst | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/docs/system/arm/fby35.rst b/docs/system/arm/fby35.rst index XXXXXXX..XXXXXXX 100644 --- a/docs/system/arm/fby35.rst +++ b/docs/system/arm/fby35.rst @@ -XXX,XX +XXX,XX @@ process starts. $ screen /dev/tty0 # In a separate TMUX pane, terminal window, etc. $ screen /dev/tty1 $ (qemu) c # Start the boot process once screen is setup. + +This machine model supports emulation of the boot from the CE0 flash device by +setting option ``execute-in-place``. When using this option, the CPU fetches +instructions to execute by reading CE0 and not from a preloaded ROM +initialized at machine init time. As a result, execution will be slower. -- 2.34.1 From: Pierrick Bouvier <pierrick.bouvier@linaro.org> Signed-off-by: Pierrick Bouvier <pierrick.bouvier@linaro.org> Message-id: 20241206192254.3889131-4-pierrick.bouvier@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> --- docs/system/arm/xlnx-versal-virt.rst | 3 +++ 1 file changed, 3 insertions(+) diff --git a/docs/system/arm/xlnx-versal-virt.rst b/docs/system/arm/xlnx-versal-virt.rst index XXXXXXX..XXXXXXX 100644 --- a/docs/system/arm/xlnx-versal-virt.rst +++ b/docs/system/arm/xlnx-versal-virt.rst @@ -XXX,XX +XXX,XX @@ Run the following at the U-Boot prompt: fdt set /chosen/dom0 reg <0x00000000 0x40000000 0x0 0x03100000> booti 30000000 - 20000000 +It's possible to change the OSPI flash model emulated by using the machine model +option ``ospi-flash``. + BBRAM File Backend """""""""""""""""" BBRAM can have an optional file backend, which must be a seekable -- 2.34.1 From: Pierrick Bouvier <pierrick.bouvier@linaro.org> Signed-off-by: Pierrick Bouvier <pierrick.bouvier@linaro.org> Message-id: 20241206192254.3889131-5-pierrick.bouvier@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> --- docs/system/arm/virt.rst | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/docs/system/arm/virt.rst b/docs/system/arm/virt.rst index XXXXXXX..XXXXXXX 100644 --- a/docs/system/arm/virt.rst +++ b/docs/system/arm/virt.rst @@ -XXX,XX +XXX,XX @@ iommu ``smmuv3`` Create an SMMUv3 +default-bus-bypass-iommu + Set ``on``/``off`` to enable/disable `bypass_iommu + <https://gitlab.com/qemu-project/qemu/-/blob/master/docs/bypass-iommu.txt>`_ + for default root bus. + ras Set ``on``/``off`` to enable/disable reporting host memory errors to a guest using ACPI and guest external abort exceptions. The default is off. +acpi + Set ``on``/``off``/``auto`` to enable/disable ACPI. + dtb-randomness Set ``on``/``off`` to pass random seeds via the guest DTB rng-seed and kaslr-seed nodes (in both "/chosen" and @@ -XXX,XX +XXX,XX @@ dtb-randomness dtb-kaslr-seed A deprecated synonym for dtb-randomness. +x-oem-id + Set string (up to 6 bytes) to override the default value of field OEMID in ACPI + table header. + +x-oem-table-id + Set string (up to 8 bytes) to override the default value of field OEM Table ID + in ACPI table header. + Linux guest kernel configuration """""""""""""""""""""""""""""""" -- 2.34.1 From: Brian Cain <brian.cain@oss.qualcomm.com> Mea culpa, I don't know how I got this wrong in 2dfe93699c. Still getting used to the new address, I suppose. Somehow I got it right in the mailmap, though. Signed-off-by: Brian Cain <brian.cain@oss.qualcomm.com> Message-id: 20241209181242.1434231-1-brian.cain@oss.qualcomm.com Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> --- MAINTAINERS | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/MAINTAINERS b/MAINTAINERS index XXXXXXX..XXXXXXX 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -XXX,XX +XXX,XX @@ F: target/avr/ F: tests/functional/test_avr_mega2560.py Hexagon TCG CPUs -M: Brian Cain <bcain@oss.qualcomm.com> +M: Brian Cain <brian.cain@oss.qualcomm.com> S: Supported F: target/hexagon/ X: target/hexagon/idef-parser/ -- 2.34.1 target/arm/helper.c is very large and unwieldy. One subset of code that we can pull out into its own file is the cpreg arrays and corresponding functions for the TLBI instructions. Because these are instructions they are only relevant for TCG and we can make the new file only be built for CONFIG_TCG. In this commit we move the AArch32 instructions from: not_v7_cp_reginfo[] v7_cp_reginfo[] v7mp_cp_reginfo[] v8_cp_reginfo[] into a new file target/arm/tcg/tlb-insns.c. A few small functions are used both by functions we haven't yet moved across and by functions we have already moved. We temporarily make these global with a prototype in cpregs.h; when the move of all TLBI insns is complete these will return to being file-local. For CONFIG_TCG, this is just moving code around. For a KVM only build, these cpregs will no longer be added to the cpregs hashtable for the CPU. However this should not be a behaviour change, because: * we never try to migration sync or otherwise include ARM_CP_NO_RAW cpregs * for migration we treat the kernel's list of system registers as the authoritative one, so these TLBI insns were never in it anyway The no-tcg stub of define_tlb_insn_regs() therefore does nothing. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241210160452.2427965-2-peter.maydell@linaro.org --- target/arm/cpregs.h | 14 +++ target/arm/internals.h | 3 + target/arm/helper.c | 231 ++-------------------------------- target/arm/tcg-stubs.c | 5 + target/arm/tcg/tlb-insns.c | 246 +++++++++++++++++++++++++++++++++++++ target/arm/tcg/meson.build | 1 + 6 files changed, 280 insertions(+), 220 deletions(-) create mode 100644 target/arm/tcg/tlb-insns.c diff --git a/target/arm/cpregs.h b/target/arm/cpregs.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpregs.h +++ b/target/arm/cpregs.h @@ -XXX,XX +XXX,XX @@ static inline bool arm_cpreg_traps_in_nv(const ARMCPRegInfo *ri) return ri->opc1 == 4 || ri->opc1 == 5; } +/* + * Temporary declarations of functions until the move to tlb_insn_helper.c + * is complete and we can make the functions static again + */ +CPAccessResult access_ttlb(CPUARMState *env, const ARMCPRegInfo *ri, + bool isread); +CPAccessResult access_ttlbis(CPUARMState *env, const ARMCPRegInfo *ri, + bool isread); +bool tlb_force_broadcast(CPUARMState *env); +void tlbimva_hyp_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value); +void tlbimva_hyp_is_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value); + #endif /* TARGET_ARM_CPREGS_H */ diff --git a/target/arm/internals.h b/target/arm/internals.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/internals.h +++ b/target/arm/internals.h @@ -XXX,XX +XXX,XX @@ static inline uint64_t pauth_ptr_mask(ARMVAParameters param) /* Add the cpreg definitions for debug related system registers */ void define_debug_regs(ARMCPU *cpu); +/* Add the cpreg definitions for TLBI instructions */ +void define_tlb_insn_regs(ARMCPU *cpu); + /* Effective value of MDCR_EL2 */ static inline uint64_t arm_mdcr_el2_eff(CPUARMState *env) { diff --git a/target/arm/helper.c b/target/arm/helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -XXX,XX +XXX,XX @@ static CPAccessResult access_tacr(CPUARMState *env, const ARMCPRegInfo *ri, } /* Check for traps from EL1 due to HCR_EL2.TTLB. */ -static CPAccessResult access_ttlb(CPUARMState *env, const ARMCPRegInfo *ri, - bool isread) +CPAccessResult access_ttlb(CPUARMState *env, const ARMCPRegInfo *ri, + bool isread) { if (arm_current_el(env) == 1 && (arm_hcr_el2_eff(env) & HCR_TTLB)) { return CP_ACCESS_TRAP_EL2; @@ -XXX,XX +XXX,XX @@ static CPAccessResult access_ttlb(CPUARMState *env, const ARMCPRegInfo *ri, } /* Check for traps from EL1 due to HCR_EL2.TTLB or TTLBIS. */ -static CPAccessResult access_ttlbis(CPUARMState *env, const ARMCPRegInfo *ri, - bool isread) +CPAccessResult access_ttlbis(CPUARMState *env, const ARMCPRegInfo *ri, + bool isread) { if (arm_current_el(env) == 1 && (arm_hcr_el2_eff(env) & (HCR_TTLB | HCR_TTLBIS))) { @@ -XXX,XX +XXX,XX @@ static int alle1_tlbmask(CPUARMState *env) ARMMMUIdxBit_Stage2_S); } - -/* IS variants of TLB operations must affect all cores */ -static void tlbiall_is_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) -{ - CPUState *cs = env_cpu(env); - - tlb_flush_all_cpus_synced(cs); -} - -static void tlbiasid_is_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) -{ - CPUState *cs = env_cpu(env); - - tlb_flush_all_cpus_synced(cs); -} - -static void tlbimva_is_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) -{ - CPUState *cs = env_cpu(env); - - tlb_flush_page_all_cpus_synced(cs, value & TARGET_PAGE_MASK); -} - -static void tlbimvaa_is_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) -{ - CPUState *cs = env_cpu(env); - - tlb_flush_page_all_cpus_synced(cs, value & TARGET_PAGE_MASK); -} - /* * Non-IS variants of TLB operations are upgraded to * IS versions if we are at EL1 and HCR_EL2.FB is effectively set to * force broadcast of these operations. */ -static bool tlb_force_broadcast(CPUARMState *env) +bool tlb_force_broadcast(CPUARMState *env) { return arm_current_el(env) == 1 && (arm_hcr_el2_eff(env) & HCR_FB); } -static void tlbiall_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) -{ - /* Invalidate all (TLBIALL) */ - CPUState *cs = env_cpu(env); - - if (tlb_force_broadcast(env)) { - tlb_flush_all_cpus_synced(cs); - } else { - tlb_flush(cs); - } -} - -static void tlbimva_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) -{ - /* Invalidate single TLB entry by MVA and ASID (TLBIMVA) */ - CPUState *cs = env_cpu(env); - - value &= TARGET_PAGE_MASK; - if (tlb_force_broadcast(env)) { - tlb_flush_page_all_cpus_synced(cs, value); - } else { - tlb_flush_page(cs, value); - } -} - -static void tlbiasid_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) -{ - /* Invalidate by ASID (TLBIASID) */ - CPUState *cs = env_cpu(env); - - if (tlb_force_broadcast(env)) { - tlb_flush_all_cpus_synced(cs); - } else { - tlb_flush(cs); - } -} - -static void tlbimvaa_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) -{ - /* Invalidate single entry by MVA, all ASIDs (TLBIMVAA) */ - CPUState *cs = env_cpu(env); - - value &= TARGET_PAGE_MASK; - if (tlb_force_broadcast(env)) { - tlb_flush_page_all_cpus_synced(cs, value); - } else { - tlb_flush_page(cs, value); - } -} - static void tlbiall_nsnh_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value) { @@ -XXX,XX +XXX,XX @@ static void tlbiall_hyp_is_write(CPUARMState *env, const ARMCPRegInfo *ri, tlb_flush_by_mmuidx_all_cpus_synced(cs, ARMMMUIdxBit_E2); } -static void tlbimva_hyp_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) +void tlbimva_hyp_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) { CPUState *cs = env_cpu(env); uint64_t pageaddr = value & ~MAKE_64BIT_MASK(0, 12); @@ -XXX,XX +XXX,XX @@ static void tlbimva_hyp_write(CPUARMState *env, const ARMCPRegInfo *ri, tlb_flush_page_by_mmuidx(cs, pageaddr, ARMMMUIdxBit_E2); } -static void tlbimva_hyp_is_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) +void tlbimva_hyp_is_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) { CPUState *cs = env_cpu(env); uint64_t pageaddr = value & ~MAKE_64BIT_MASK(0, 12); @@ -XXX,XX +XXX,XX @@ static void tlbimva_hyp_is_write(CPUARMState *env, const ARMCPRegInfo *ri, ARMMMUIdxBit_E2); } -static void tlbiipas2_hyp_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) -{ - CPUState *cs = env_cpu(env); - uint64_t pageaddr = (value & MAKE_64BIT_MASK(0, 28)) << 12; - - tlb_flush_page_by_mmuidx(cs, pageaddr, ARMMMUIdxBit_Stage2); -} - -static void tlbiipas2is_hyp_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) -{ - CPUState *cs = env_cpu(env); - uint64_t pageaddr = (value & MAKE_64BIT_MASK(0, 28)) << 12; - - tlb_flush_page_by_mmuidx_all_cpus_synced(cs, pageaddr, ARMMMUIdxBit_Stage2); -} - static const ARMCPRegInfo cp_reginfo[] = { /* * Define the secure and non-secure FCSE identifier CP registers @@ -XXX,XX +XXX,XX @@ static const ARMCPRegInfo not_v7_cp_reginfo[] = { */ { .name = "DBGDIDR", .cp = 14, .crn = 0, .crm = 0, .opc1 = 0, .opc2 = 0, .access = PL0_R, .type = ARM_CP_CONST, .resetvalue = 0 }, - /* - * MMU TLB control. Note that the wildcarding means we cover not just - * the unified TLB ops but also the dside/iside/inner-shareable variants. - */ - { .name = "TLBIALL", .cp = 15, .crn = 8, .crm = CP_ANY, - .opc1 = CP_ANY, .opc2 = 0, .access = PL1_W, .writefn = tlbiall_write, - .type = ARM_CP_NO_RAW }, - { .name = "TLBIMVA", .cp = 15, .crn = 8, .crm = CP_ANY, - .opc1 = CP_ANY, .opc2 = 1, .access = PL1_W, .writefn = tlbimva_write, - .type = ARM_CP_NO_RAW }, - { .name = "TLBIASID", .cp = 15, .crn = 8, .crm = CP_ANY, - .opc1 = CP_ANY, .opc2 = 2, .access = PL1_W, .writefn = tlbiasid_write, - .type = ARM_CP_NO_RAW }, - { .name = "TLBIMVAA", .cp = 15, .crn = 8, .crm = CP_ANY, - .opc1 = CP_ANY, .opc2 = 3, .access = PL1_W, .writefn = tlbimvaa_write, - .type = ARM_CP_NO_RAW }, { .name = "PRRR", .cp = 15, .crn = 10, .crm = 2, .opc1 = 0, .opc2 = 0, .access = PL1_RW, .type = ARM_CP_NOP }, { .name = "NMRR", .cp = 15, .crn = 10, .crm = 2, @@ -XXX,XX +XXX,XX @@ static const ARMCPRegInfo v7_cp_reginfo[] = { .opc0 = 3, .opc1 = 0, .crn = 12, .crm = 1, .opc2 = 0, .fgt = FGT_ISR_EL1, .type = ARM_CP_NO_RAW, .access = PL1_R, .readfn = isr_read }, - /* 32 bit ITLB invalidates */ - { .name = "ITLBIALL", .cp = 15, .opc1 = 0, .crn = 8, .crm = 5, .opc2 = 0, - .type = ARM_CP_NO_RAW, .access = PL1_W, .accessfn = access_ttlb, - .writefn = tlbiall_write }, - { .name = "ITLBIMVA", .cp = 15, .opc1 = 0, .crn = 8, .crm = 5, .opc2 = 1, - .type = ARM_CP_NO_RAW, .access = PL1_W, .accessfn = access_ttlb, - .writefn = tlbimva_write }, - { .name = "ITLBIASID", .cp = 15, .opc1 = 0, .crn = 8, .crm = 5, .opc2 = 2, - .type = ARM_CP_NO_RAW, .access = PL1_W, .accessfn = access_ttlb, - .writefn = tlbiasid_write }, - /* 32 bit DTLB invalidates */ - { .name = "DTLBIALL", .cp = 15, .opc1 = 0, .crn = 8, .crm = 6, .opc2 = 0, - .type = ARM_CP_NO_RAW, .access = PL1_W, .accessfn = access_ttlb, - .writefn = tlbiall_write }, - { .name = "DTLBIMVA", .cp = 15, .opc1 = 0, .crn = 8, .crm = 6, .opc2 = 1, - .type = ARM_CP_NO_RAW, .access = PL1_W, .accessfn = access_ttlb, - .writefn = tlbimva_write }, - { .name = "DTLBIASID", .cp = 15, .opc1 = 0, .crn = 8, .crm = 6, .opc2 = 2, - .type = ARM_CP_NO_RAW, .access = PL1_W, .accessfn = access_ttlb, - .writefn = tlbiasid_write }, - /* 32 bit TLB invalidates */ - { .name = "TLBIALL", .cp = 15, .opc1 = 0, .crn = 8, .crm = 7, .opc2 = 0, - .type = ARM_CP_NO_RAW, .access = PL1_W, .accessfn = access_ttlb, - .writefn = tlbiall_write }, - { .name = "TLBIMVA", .cp = 15, .opc1 = 0, .crn = 8, .crm = 7, .opc2 = 1, - .type = ARM_CP_NO_RAW, .access = PL1_W, .accessfn = access_ttlb, - .writefn = tlbimva_write }, - { .name = "TLBIASID", .cp = 15, .opc1 = 0, .crn = 8, .crm = 7, .opc2 = 2, - .type = ARM_CP_NO_RAW, .access = PL1_W, .accessfn = access_ttlb, - .writefn = tlbiasid_write }, - { .name = "TLBIMVAA", .cp = 15, .opc1 = 0, .crn = 8, .crm = 7, .opc2 = 3, - .type = ARM_CP_NO_RAW, .access = PL1_W, .accessfn = access_ttlb, - .writefn = tlbimvaa_write }, -}; - -static const ARMCPRegInfo v7mp_cp_reginfo[] = { - /* 32 bit TLB invalidates, Inner Shareable */ - { .name = "TLBIALLIS", .cp = 15, .opc1 = 0, .crn = 8, .crm = 3, .opc2 = 0, - .type = ARM_CP_NO_RAW, .access = PL1_W, .accessfn = access_ttlbis, - .writefn = tlbiall_is_write }, - { .name = "TLBIMVAIS", .cp = 15, .opc1 = 0, .crn = 8, .crm = 3, .opc2 = 1, - .type = ARM_CP_NO_RAW, .access = PL1_W, .accessfn = access_ttlbis, - .writefn = tlbimva_is_write }, - { .name = "TLBIASIDIS", .cp = 15, .opc1 = 0, .crn = 8, .crm = 3, .opc2 = 2, - .type = ARM_CP_NO_RAW, .access = PL1_W, .accessfn = access_ttlbis, - .writefn = tlbiasid_is_write }, - { .name = "TLBIMVAAIS", .cp = 15, .opc1 = 0, .crn = 8, .crm = 3, .opc2 = 3, - .type = ARM_CP_NO_RAW, .access = PL1_W, .accessfn = access_ttlbis, - .writefn = tlbimvaa_is_write }, }; static const ARMCPRegInfo pmovsset_cp_reginfo[] = { @@ -XXX,XX +XXX,XX @@ static const ARMCPRegInfo v8_cp_reginfo[] = { .fieldoffset = offsetof(CPUARMState, cp15.par_el[1]), .writefn = par_write }, #endif - /* TLB invalidate last level of translation table walk */ - { .name = "TLBIMVALIS", .cp = 15, .opc1 = 0, .crn = 8, .crm = 3, .opc2 = 5, - .type = ARM_CP_NO_RAW, .access = PL1_W, .accessfn = access_ttlbis, - .writefn = tlbimva_is_write }, - { .name = "TLBIMVAALIS", .cp = 15, .opc1 = 0, .crn = 8, .crm = 3, .opc2 = 7, - .type = ARM_CP_NO_RAW, .access = PL1_W, .accessfn = access_ttlbis, - .writefn = tlbimvaa_is_write }, - { .name = "TLBIMVAL", .cp = 15, .opc1 = 0, .crn = 8, .crm = 7, .opc2 = 5, - .type = ARM_CP_NO_RAW, .access = PL1_W, .accessfn = access_ttlb, - .writefn = tlbimva_write }, - { .name = "TLBIMVAAL", .cp = 15, .opc1 = 0, .crn = 8, .crm = 7, .opc2 = 7, - .type = ARM_CP_NO_RAW, .access = PL1_W, .accessfn = access_ttlb, - .writefn = tlbimvaa_write }, - { .name = "TLBIMVALH", .cp = 15, .opc1 = 4, .crn = 8, .crm = 7, .opc2 = 5, - .type = ARM_CP_NO_RAW, .access = PL2_W, - .writefn = tlbimva_hyp_write }, - { .name = "TLBIMVALHIS", - .cp = 15, .opc1 = 4, .crn = 8, .crm = 3, .opc2 = 5, - .type = ARM_CP_NO_RAW, .access = PL2_W, - .writefn = tlbimva_hyp_is_write }, - { .name = "TLBIIPAS2", - .cp = 15, .opc1 = 4, .crn = 8, .crm = 4, .opc2 = 1, - .type = ARM_CP_NO_RAW, .access = PL2_W, - .writefn = tlbiipas2_hyp_write }, - { .name = "TLBIIPAS2IS", - .cp = 15, .opc1 = 4, .crn = 8, .crm = 0, .opc2 = 1, - .type = ARM_CP_NO_RAW, .access = PL2_W, - .writefn = tlbiipas2is_hyp_write }, - { .name = "TLBIIPAS2L", - .cp = 15, .opc1 = 4, .crn = 8, .crm = 4, .opc2 = 5, - .type = ARM_CP_NO_RAW, .access = PL2_W, - .writefn = tlbiipas2_hyp_write }, - { .name = "TLBIIPAS2LIS", - .cp = 15, .opc1 = 4, .crn = 8, .crm = 0, .opc2 = 5, - .type = ARM_CP_NO_RAW, .access = PL2_W, - .writefn = tlbiipas2is_hyp_write }, /* 32 bit cache operations */ { .name = "ICIALLUIS", .cp = 15, .opc1 = 0, .crn = 7, .crm = 1, .opc2 = 0, .type = ARM_CP_NOP, .access = PL1_W, .accessfn = access_ticab }, @@ -XXX,XX +XXX,XX @@ void register_cp_regs_for_features(ARMCPU *cpu) define_arm_cp_regs(cpu, not_v8_cp_reginfo); } + define_tlb_insn_regs(cpu); + if (arm_feature(env, ARM_FEATURE_V6)) { /* The ID registers all have impdef reset values */ ARMCPRegInfo v6_idregs[] = { @@ -XXX,XX +XXX,XX @@ void register_cp_regs_for_features(ARMCPU *cpu) if (arm_feature(env, ARM_FEATURE_V6K)) { define_arm_cp_regs(cpu, v6k_cp_reginfo); } - if (arm_feature(env, ARM_FEATURE_V7MP) && - !arm_feature(env, ARM_FEATURE_PMSA)) { - define_arm_cp_regs(cpu, v7mp_cp_reginfo); - } if (arm_feature(env, ARM_FEATURE_V7VE)) { define_arm_cp_regs(cpu, pmovsset_cp_reginfo); } diff --git a/target/arm/tcg-stubs.c b/target/arm/tcg-stubs.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg-stubs.c +++ b/target/arm/tcg-stubs.c @@ -XXX,XX +XXX,XX @@ void raise_exception_ra(CPUARMState *env, uint32_t excp, uint32_t syndrome, void assert_hflags_rebuild_correctly(CPUARMState *env) { } + +/* TLBI insns are only used by TCG, so we don't need to do anything for KVM */ +void define_tlb_insn_regs(ARMCPU *cpu) +{ +} diff --git a/target/arm/tcg/tlb-insns.c b/target/arm/tcg/tlb-insns.c new file mode 100644 index XXXXXXX..XXXXXXX --- /dev/null +++ b/target/arm/tcg/tlb-insns.c @@ -XXX,XX +XXX,XX @@ +/* + * Helpers for TLBI insns + * + * This code is licensed under the GNU GPL v2 or later. + * + * SPDX-License-Identifier: GPL-2.0-or-later + */ +#include "qemu/osdep.h" +#include "exec/exec-all.h" +#include "cpu.h" +#include "internals.h" +#include "cpu-features.h" +#include "cpregs.h" + +/* IS variants of TLB operations must affect all cores */ +static void tlbiall_is_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) +{ + CPUState *cs = env_cpu(env); + + tlb_flush_all_cpus_synced(cs); +} + +static void tlbiasid_is_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) +{ + CPUState *cs = env_cpu(env); + + tlb_flush_all_cpus_synced(cs); +} + +static void tlbimva_is_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) +{ + CPUState *cs = env_cpu(env); + + tlb_flush_page_all_cpus_synced(cs, value & TARGET_PAGE_MASK); +} + +static void tlbimvaa_is_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) +{ + CPUState *cs = env_cpu(env); + + tlb_flush_page_all_cpus_synced(cs, value & TARGET_PAGE_MASK); +} + +static void tlbiall_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) +{ + /* Invalidate all (TLBIALL) */ + CPUState *cs = env_cpu(env); + + if (tlb_force_broadcast(env)) { + tlb_flush_all_cpus_synced(cs); + } else { + tlb_flush(cs); + } +} + +static void tlbimva_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) +{ + /* Invalidate single TLB entry by MVA and ASID (TLBIMVA) */ + CPUState *cs = env_cpu(env); + + value &= TARGET_PAGE_MASK; + if (tlb_force_broadcast(env)) { + tlb_flush_page_all_cpus_synced(cs, value); + } else { + tlb_flush_page(cs, value); + } +} + +static void tlbiasid_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) +{ + /* Invalidate by ASID (TLBIASID) */ + CPUState *cs = env_cpu(env); + + if (tlb_force_broadcast(env)) { + tlb_flush_all_cpus_synced(cs); + } else { + tlb_flush(cs); + } +} + +static void tlbimvaa_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) +{ + /* Invalidate single entry by MVA, all ASIDs (TLBIMVAA) */ + CPUState *cs = env_cpu(env); + + value &= TARGET_PAGE_MASK; + if (tlb_force_broadcast(env)) { + tlb_flush_page_all_cpus_synced(cs, value); + } else { + tlb_flush_page(cs, value); + } +} + +static void tlbiipas2_hyp_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) +{ + CPUState *cs = env_cpu(env); + uint64_t pageaddr = (value & MAKE_64BIT_MASK(0, 28)) << 12; + + tlb_flush_page_by_mmuidx(cs, pageaddr, ARMMMUIdxBit_Stage2); +} + +static void tlbiipas2is_hyp_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) +{ + CPUState *cs = env_cpu(env); + uint64_t pageaddr = (value & MAKE_64BIT_MASK(0, 28)) << 12; + + tlb_flush_page_by_mmuidx_all_cpus_synced(cs, pageaddr, ARMMMUIdxBit_Stage2); +} + +static const ARMCPRegInfo tlbi_not_v7_cp_reginfo[] = { + /* + * MMU TLB control. Note that the wildcarding means we cover not just + * the unified TLB ops but also the dside/iside/inner-shareable variants. + */ + { .name = "TLBIALL", .cp = 15, .crn = 8, .crm = CP_ANY, + .opc1 = CP_ANY, .opc2 = 0, .access = PL1_W, .writefn = tlbiall_write, + .type = ARM_CP_NO_RAW }, + { .name = "TLBIMVA", .cp = 15, .crn = 8, .crm = CP_ANY, + .opc1 = CP_ANY, .opc2 = 1, .access = PL1_W, .writefn = tlbimva_write, + .type = ARM_CP_NO_RAW }, + { .name = "TLBIASID", .cp = 15, .crn = 8, .crm = CP_ANY, + .opc1 = CP_ANY, .opc2 = 2, .access = PL1_W, .writefn = tlbiasid_write, + .type = ARM_CP_NO_RAW }, + { .name = "TLBIMVAA", .cp = 15, .crn = 8, .crm = CP_ANY, + .opc1 = CP_ANY, .opc2 = 3, .access = PL1_W, .writefn = tlbimvaa_write, + .type = ARM_CP_NO_RAW }, +}; + +static const ARMCPRegInfo tlbi_v7_cp_reginfo[] = { + /* 32 bit ITLB invalidates */ + { .name = "ITLBIALL", .cp = 15, .opc1 = 0, .crn = 8, .crm = 5, .opc2 = 0, + .type = ARM_CP_NO_RAW, .access = PL1_W, .accessfn = access_ttlb, + .writefn = tlbiall_write }, + { .name = "ITLBIMVA", .cp = 15, .opc1 = 0, .crn = 8, .crm = 5, .opc2 = 1, + .type = ARM_CP_NO_RAW, .access = PL1_W, .accessfn = access_ttlb, + .writefn = tlbimva_write }, + { .name = "ITLBIASID", .cp = 15, .opc1 = 0, .crn = 8, .crm = 5, .opc2 = 2, + .type = ARM_CP_NO_RAW, .access = PL1_W, .accessfn = access_ttlb, + .writefn = tlbiasid_write }, + /* 32 bit DTLB invalidates */ + { .name = "DTLBIALL", .cp = 15, .opc1 = 0, .crn = 8, .crm = 6, .opc2 = 0, + .type = ARM_CP_NO_RAW, .access = PL1_W, .accessfn = access_ttlb, + .writefn = tlbiall_write }, + { .name = "DTLBIMVA", .cp = 15, .opc1 = 0, .crn = 8, .crm = 6, .opc2 = 1, + .type = ARM_CP_NO_RAW, .access = PL1_W, .accessfn = access_ttlb, + .writefn = tlbimva_write }, + { .name = "DTLBIASID", .cp = 15, .opc1 = 0, .crn = 8, .crm = 6, .opc2 = 2, + .type = ARM_CP_NO_RAW, .access = PL1_W, .accessfn = access_ttlb, + .writefn = tlbiasid_write }, + /* 32 bit TLB invalidates */ + { .name = "TLBIALL", .cp = 15, .opc1 = 0, .crn = 8, .crm = 7, .opc2 = 0, + .type = ARM_CP_NO_RAW, .access = PL1_W, .accessfn = access_ttlb, + .writefn = tlbiall_write }, + { .name = "TLBIMVA", .cp = 15, .opc1 = 0, .crn = 8, .crm = 7, .opc2 = 1, + .type = ARM_CP_NO_RAW, .access = PL1_W, .accessfn = access_ttlb, + .writefn = tlbimva_write }, + { .name = "TLBIASID", .cp = 15, .opc1 = 0, .crn = 8, .crm = 7, .opc2 = 2, + .type = ARM_CP_NO_RAW, .access = PL1_W, .accessfn = access_ttlb, + .writefn = tlbiasid_write }, + { .name = "TLBIMVAA", .cp = 15, .opc1 = 0, .crn = 8, .crm = 7, .opc2 = 3, + .type = ARM_CP_NO_RAW, .access = PL1_W, .accessfn = access_ttlb, + .writefn = tlbimvaa_write }, +}; + +static const ARMCPRegInfo tlbi_v7mp_cp_reginfo[] = { + /* 32 bit TLB invalidates, Inner Shareable */ + { .name = "TLBIALLIS", .cp = 15, .opc1 = 0, .crn = 8, .crm = 3, .opc2 = 0, + .type = ARM_CP_NO_RAW, .access = PL1_W, .accessfn = access_ttlbis, + .writefn = tlbiall_is_write }, + { .name = "TLBIMVAIS", .cp = 15, .opc1 = 0, .crn = 8, .crm = 3, .opc2 = 1, + .type = ARM_CP_NO_RAW, .access = PL1_W, .accessfn = access_ttlbis, + .writefn = tlbimva_is_write }, + { .name = "TLBIASIDIS", .cp = 15, .opc1 = 0, .crn = 8, .crm = 3, .opc2 = 2, + .type = ARM_CP_NO_RAW, .access = PL1_W, .accessfn = access_ttlbis, + .writefn = tlbiasid_is_write }, + { .name = "TLBIMVAAIS", .cp = 15, .opc1 = 0, .crn = 8, .crm = 3, .opc2 = 3, + .type = ARM_CP_NO_RAW, .access = PL1_W, .accessfn = access_ttlbis, + .writefn = tlbimvaa_is_write }, +}; + +static const ARMCPRegInfo tlbi_v8_cp_reginfo[] = { + /* AArch32 TLB invalidate last level of translation table walk */ + { .name = "TLBIMVALIS", .cp = 15, .opc1 = 0, .crn = 8, .crm = 3, .opc2 = 5, + .type = ARM_CP_NO_RAW, .access = PL1_W, .accessfn = access_ttlbis, + .writefn = tlbimva_is_write }, + { .name = "TLBIMVAALIS", .cp = 15, .opc1 = 0, .crn = 8, .crm = 3, .opc2 = 7, + .type = ARM_CP_NO_RAW, .access = PL1_W, .accessfn = access_ttlbis, + .writefn = tlbimvaa_is_write }, + { .name = "TLBIMVAL", .cp = 15, .opc1 = 0, .crn = 8, .crm = 7, .opc2 = 5, + .type = ARM_CP_NO_RAW, .access = PL1_W, .accessfn = access_ttlb, + .writefn = tlbimva_write }, + { .name = "TLBIMVAAL", .cp = 15, .opc1 = 0, .crn = 8, .crm = 7, .opc2 = 7, + .type = ARM_CP_NO_RAW, .access = PL1_W, .accessfn = access_ttlb, + .writefn = tlbimvaa_write }, + { .name = "TLBIMVALH", .cp = 15, .opc1 = 4, .crn = 8, .crm = 7, .opc2 = 5, + .type = ARM_CP_NO_RAW, .access = PL2_W, + .writefn = tlbimva_hyp_write }, + { .name = "TLBIMVALHIS", + .cp = 15, .opc1 = 4, .crn = 8, .crm = 3, .opc2 = 5, + .type = ARM_CP_NO_RAW, .access = PL2_W, + .writefn = tlbimva_hyp_is_write }, + { .name = "TLBIIPAS2", + .cp = 15, .opc1 = 4, .crn = 8, .crm = 4, .opc2 = 1, + .type = ARM_CP_NO_RAW, .access = PL2_W, + .writefn = tlbiipas2_hyp_write }, + { .name = "TLBIIPAS2IS", + .cp = 15, .opc1 = 4, .crn = 8, .crm = 0, .opc2 = 1, + .type = ARM_CP_NO_RAW, .access = PL2_W, + .writefn = tlbiipas2is_hyp_write }, + { .name = "TLBIIPAS2L", + .cp = 15, .opc1 = 4, .crn = 8, .crm = 4, .opc2 = 5, + .type = ARM_CP_NO_RAW, .access = PL2_W, + .writefn = tlbiipas2_hyp_write }, + { .name = "TLBIIPAS2LIS", + .cp = 15, .opc1 = 4, .crn = 8, .crm = 0, .opc2 = 5, + .type = ARM_CP_NO_RAW, .access = PL2_W, + .writefn = tlbiipas2is_hyp_write }, +}; + +void define_tlb_insn_regs(ARMCPU *cpu) +{ + CPUARMState *env = &cpu->env; + + if (!arm_feature(env, ARM_FEATURE_V7)) { + define_arm_cp_regs(cpu, tlbi_not_v7_cp_reginfo); + } else { + define_arm_cp_regs(cpu, tlbi_v7_cp_reginfo); + } + if (arm_feature(env, ARM_FEATURE_V7MP) && + !arm_feature(env, ARM_FEATURE_PMSA)) { + define_arm_cp_regs(cpu, tlbi_v7mp_cp_reginfo); + } + if (arm_feature(env, ARM_FEATURE_V8)) { + define_arm_cp_regs(cpu, tlbi_v8_cp_reginfo); + } +} diff --git a/target/arm/tcg/meson.build b/target/arm/tcg/meson.build index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/meson.build +++ b/target/arm/tcg/meson.build @@ -XXX,XX +XXX,XX @@ arm_ss.add(files( 'op_helper.c', 'tlb_helper.c', 'vec_helper.c', + 'tlb-insns.c', )) arm_ss.add(when: 'TARGET_AARCH64', if_true: files( -- 2.34.1 Move the AArch32 TLBI insns for AArch32 EL2 to tlbi_insn_helper.c. To keep this as an obviously pure code-movement, we retain the same condition for registering tlbi_el2_cp_reginfo that we use for el2_cp_reginfo. We'll be able to simplify this condition later, since the need to define the reginfo for EL3-without-EL2 doesn't apply for the TLBI ops specifically. This move brings all the uses of tlbimva_hyp_write() and tlbimva_hyp_is_write() back into a single file, so we can move those also, and make them file-local again. The helper alle1_tlbmask() is an exception to the pattern that we only need to make these functions global temporarily, because once this refactoring is complete it will be called by both code in helper.c (vttbr_write()) and by code in tlb-insns.c. We therefore put its prototype in a permanent home in internals.h. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241210160452.2427965-3-peter.maydell@linaro.org --- target/arm/cpregs.h | 4 -- target/arm/internals.h | 6 +++ target/arm/helper.c | 74 +-------------------------------- target/arm/tcg/tlb-insns.c | 85 ++++++++++++++++++++++++++++++++++++++ 4 files changed, 92 insertions(+), 77 deletions(-) diff --git a/target/arm/cpregs.h b/target/arm/cpregs.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpregs.h +++ b/target/arm/cpregs.h @@ -XXX,XX +XXX,XX @@ CPAccessResult access_ttlb(CPUARMState *env, const ARMCPRegInfo *ri, CPAccessResult access_ttlbis(CPUARMState *env, const ARMCPRegInfo *ri, bool isread); bool tlb_force_broadcast(CPUARMState *env); -void tlbimva_hyp_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value); -void tlbimva_hyp_is_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value); #endif /* TARGET_ARM_CPREGS_H */ diff --git a/target/arm/internals.h b/target/arm/internals.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/internals.h +++ b/target/arm/internals.h @@ -XXX,XX +XXX,XX @@ uint64_t gt_get_countervalue(CPUARMState *env); * and CNTVCT_EL0 (this will be either 0 or the value of CNTVOFF_EL2). */ uint64_t gt_virt_cnt_offset(CPUARMState *env); + +/* + * Return mask of ARMMMUIdxBit values corresponding to an "invalidate + * all EL1" scope; this covers stage 1 and stage 2. + */ +int alle1_tlbmask(CPUARMState *env); #endif diff --git a/target/arm/helper.c b/target/arm/helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -XXX,XX +XXX,XX @@ static void contextidr_write(CPUARMState *env, const ARMCPRegInfo *ri, raw_write(env, ri, value); } -static int alle1_tlbmask(CPUARMState *env) +int alle1_tlbmask(CPUARMState *env) { /* * Note that the 'ALL' scope must invalidate both stage 1 and @@ -XXX,XX +XXX,XX @@ bool tlb_force_broadcast(CPUARMState *env) return arm_current_el(env) == 1 && (arm_hcr_el2_eff(env) & HCR_FB); } -static void tlbiall_nsnh_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) -{ - CPUState *cs = env_cpu(env); - - tlb_flush_by_mmuidx(cs, alle1_tlbmask(env)); -} - -static void tlbiall_nsnh_is_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) -{ - CPUState *cs = env_cpu(env); - - tlb_flush_by_mmuidx_all_cpus_synced(cs, alle1_tlbmask(env)); -} - - -static void tlbiall_hyp_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) -{ - CPUState *cs = env_cpu(env); - - tlb_flush_by_mmuidx(cs, ARMMMUIdxBit_E2); -} - -static void tlbiall_hyp_is_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) -{ - CPUState *cs = env_cpu(env); - - tlb_flush_by_mmuidx_all_cpus_synced(cs, ARMMMUIdxBit_E2); -} - -void tlbimva_hyp_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) -{ - CPUState *cs = env_cpu(env); - uint64_t pageaddr = value & ~MAKE_64BIT_MASK(0, 12); - - tlb_flush_page_by_mmuidx(cs, pageaddr, ARMMMUIdxBit_E2); -} - -void tlbimva_hyp_is_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) -{ - CPUState *cs = env_cpu(env); - uint64_t pageaddr = value & ~MAKE_64BIT_MASK(0, 12); - - tlb_flush_page_by_mmuidx_all_cpus_synced(cs, pageaddr, - ARMMMUIdxBit_E2); -} - static const ARMCPRegInfo cp_reginfo[] = { /* * Define the secure and non-secure FCSE identifier CP registers @@ -XXX,XX +XXX,XX @@ static const ARMCPRegInfo el2_cp_reginfo[] = { { .name = "HTTBR", .cp = 15, .opc1 = 4, .crm = 2, .access = PL2_RW, .type = ARM_CP_64BIT | ARM_CP_ALIAS, .fieldoffset = offsetof(CPUARMState, cp15.ttbr0_el[2]) }, - { .name = "TLBIALLNSNH", - .cp = 15, .opc1 = 4, .crn = 8, .crm = 7, .opc2 = 4, - .type = ARM_CP_NO_RAW, .access = PL2_W, - .writefn = tlbiall_nsnh_write }, - { .name = "TLBIALLNSNHIS", - .cp = 15, .opc1 = 4, .crn = 8, .crm = 3, .opc2 = 4, - .type = ARM_CP_NO_RAW, .access = PL2_W, - .writefn = tlbiall_nsnh_is_write }, - { .name = "TLBIALLH", .cp = 15, .opc1 = 4, .crn = 8, .crm = 7, .opc2 = 0, - .type = ARM_CP_NO_RAW, .access = PL2_W, - .writefn = tlbiall_hyp_write }, - { .name = "TLBIALLHIS", .cp = 15, .opc1 = 4, .crn = 8, .crm = 3, .opc2 = 0, - .type = ARM_CP_NO_RAW, .access = PL2_W, - .writefn = tlbiall_hyp_is_write }, - { .name = "TLBIMVAH", .cp = 15, .opc1 = 4, .crn = 8, .crm = 7, .opc2 = 1, - .type = ARM_CP_NO_RAW, .access = PL2_W, - .writefn = tlbimva_hyp_write }, - { .name = "TLBIMVAHIS", .cp = 15, .opc1 = 4, .crn = 8, .crm = 3, .opc2 = 1, - .type = ARM_CP_NO_RAW, .access = PL2_W, - .writefn = tlbimva_hyp_is_write }, { .name = "TLBI_ALLE2", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 7, .opc2 = 0, .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF, diff --git a/target/arm/tcg/tlb-insns.c b/target/arm/tcg/tlb-insns.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/tlb-insns.c +++ b/target/arm/tcg/tlb-insns.c @@ -XXX,XX +XXX,XX @@ static void tlbimvaa_write(CPUARMState *env, const ARMCPRegInfo *ri, } } +static void tlbimva_hyp_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) +{ + CPUState *cs = env_cpu(env); + uint64_t pageaddr = value & ~MAKE_64BIT_MASK(0, 12); + + tlb_flush_page_by_mmuidx(cs, pageaddr, ARMMMUIdxBit_E2); +} + +static void tlbimva_hyp_is_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) +{ + CPUState *cs = env_cpu(env); + uint64_t pageaddr = value & ~MAKE_64BIT_MASK(0, 12); + + tlb_flush_page_by_mmuidx_all_cpus_synced(cs, pageaddr, + ARMMMUIdxBit_E2); +} + static void tlbiipas2_hyp_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value) { @@ -XXX,XX +XXX,XX @@ static void tlbiipas2is_hyp_write(CPUARMState *env, const ARMCPRegInfo *ri, tlb_flush_page_by_mmuidx_all_cpus_synced(cs, pageaddr, ARMMMUIdxBit_Stage2); } +static void tlbiall_nsnh_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) +{ + CPUState *cs = env_cpu(env); + + tlb_flush_by_mmuidx(cs, alle1_tlbmask(env)); +} + +static void tlbiall_nsnh_is_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) +{ + CPUState *cs = env_cpu(env); + + tlb_flush_by_mmuidx_all_cpus_synced(cs, alle1_tlbmask(env)); +} + + +static void tlbiall_hyp_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) +{ + CPUState *cs = env_cpu(env); + + tlb_flush_by_mmuidx(cs, ARMMMUIdxBit_E2); +} + +static void tlbiall_hyp_is_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) +{ + CPUState *cs = env_cpu(env); + + tlb_flush_by_mmuidx_all_cpus_synced(cs, ARMMMUIdxBit_E2); +} + static const ARMCPRegInfo tlbi_not_v7_cp_reginfo[] = { /* * MMU TLB control. Note that the wildcarding means we cover not just @@ -XXX,XX +XXX,XX @@ static const ARMCPRegInfo tlbi_v8_cp_reginfo[] = { .writefn = tlbiipas2is_hyp_write }, }; +static const ARMCPRegInfo tlbi_el2_cp_reginfo[] = { + { .name = "TLBIALLNSNH", + .cp = 15, .opc1 = 4, .crn = 8, .crm = 7, .opc2 = 4, + .type = ARM_CP_NO_RAW, .access = PL2_W, + .writefn = tlbiall_nsnh_write }, + { .name = "TLBIALLNSNHIS", + .cp = 15, .opc1 = 4, .crn = 8, .crm = 3, .opc2 = 4, + .type = ARM_CP_NO_RAW, .access = PL2_W, + .writefn = tlbiall_nsnh_is_write }, + { .name = "TLBIALLH", .cp = 15, .opc1 = 4, .crn = 8, .crm = 7, .opc2 = 0, + .type = ARM_CP_NO_RAW, .access = PL2_W, + .writefn = tlbiall_hyp_write }, + { .name = "TLBIALLHIS", .cp = 15, .opc1 = 4, .crn = 8, .crm = 3, .opc2 = 0, + .type = ARM_CP_NO_RAW, .access = PL2_W, + .writefn = tlbiall_hyp_is_write }, + { .name = "TLBIMVAH", .cp = 15, .opc1 = 4, .crn = 8, .crm = 7, .opc2 = 1, + .type = ARM_CP_NO_RAW, .access = PL2_W, + .writefn = tlbimva_hyp_write }, + { .name = "TLBIMVAHIS", .cp = 15, .opc1 = 4, .crn = 8, .crm = 3, .opc2 = 1, + .type = ARM_CP_NO_RAW, .access = PL2_W, + .writefn = tlbimva_hyp_is_write }, +}; + void define_tlb_insn_regs(ARMCPU *cpu) { CPUARMState *env = &cpu->env; @@ -XXX,XX +XXX,XX @@ void define_tlb_insn_regs(ARMCPU *cpu) if (arm_feature(env, ARM_FEATURE_V8)) { define_arm_cp_regs(cpu, tlbi_v8_cp_reginfo); } + /* + * We retain the existing logic for when to register these TLBI + * ops (i.e. matching the condition for el2_cp_reginfo[] in + * helper.c), but we will be able to simplify this later. + */ + if (arm_feature(env, ARM_FEATURE_EL2) + || (arm_feature(env, ARM_FEATURE_EL3) + && arm_feature(env, ARM_FEATURE_V8))) { + define_arm_cp_regs(cpu, tlbi_el2_cp_reginfo); + } } -- 2.34.1 Move the AArch64 TLBI insns that are declared in v8_cp_reginfo[] into tlb-insns.c. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241210160452.2427965-4-peter.maydell@linaro.org --- target/arm/cpregs.h | 11 +++ target/arm/helper.c | 182 +++---------------------------------- target/arm/tcg/tlb-insns.c | 160 ++++++++++++++++++++++++++++++++ 3 files changed, 182 insertions(+), 171 deletions(-) diff --git a/target/arm/cpregs.h b/target/arm/cpregs.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpregs.h +++ b/target/arm/cpregs.h @@ -XXX,XX +XXX,XX @@ CPAccessResult access_ttlb(CPUARMState *env, const ARMCPRegInfo *ri, CPAccessResult access_ttlbis(CPUARMState *env, const ARMCPRegInfo *ri, bool isread); bool tlb_force_broadcast(CPUARMState *env); +int tlbbits_for_regime(CPUARMState *env, ARMMMUIdx mmu_idx, + uint64_t addr); +int vae1_tlbbits(CPUARMState *env, uint64_t addr); +int vae1_tlbmask(CPUARMState *env); +int ipas2e1_tlbmask(CPUARMState *env, int64_t value); +void tlbi_aa64_vmalle1is_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value); +void tlbi_aa64_alle1is_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value); +void tlbi_aa64_vae1is_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value); #endif /* TARGET_ARM_CPREGS_H */ diff --git a/target/arm/helper.c b/target/arm/helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -XXX,XX +XXX,XX @@ static CPAccessResult access_tocu(CPUARMState *env, const ARMCPRegInfo *ri, * Page D4-1736 (DDI0487A.b) */ -static int vae1_tlbmask(CPUARMState *env) +int vae1_tlbmask(CPUARMState *env) { uint64_t hcr = arm_hcr_el2_eff(env); uint16_t mask; @@ -XXX,XX +XXX,XX @@ static int vae2_tlbmask(CPUARMState *env) } /* Return 56 if TBI is enabled, 64 otherwise. */ -static int tlbbits_for_regime(CPUARMState *env, ARMMMUIdx mmu_idx, - uint64_t addr) +int tlbbits_for_regime(CPUARMState *env, ARMMMUIdx mmu_idx, + uint64_t addr) { uint64_t tcr = regime_tcr(env, mmu_idx); int tbi = aa64_va_parameter_tbi(tcr, mmu_idx); @@ -XXX,XX +XXX,XX @@ static int tlbbits_for_regime(CPUARMState *env, ARMMMUIdx mmu_idx, return (tbi >> select) & 1 ? 56 : 64; } -static int vae1_tlbbits(CPUARMState *env, uint64_t addr) +int vae1_tlbbits(CPUARMState *env, uint64_t addr) { uint64_t hcr = arm_hcr_el2_eff(env); ARMMMUIdx mmu_idx; @@ -XXX,XX +XXX,XX @@ static int vae2_tlbbits(CPUARMState *env, uint64_t addr) return tlbbits_for_regime(env, mmu_idx, addr); } -static void tlbi_aa64_vmalle1is_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) +void tlbi_aa64_vmalle1is_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) { CPUState *cs = env_cpu(env); int mask = vae1_tlbmask(env); @@ -XXX,XX +XXX,XX @@ static void tlbi_aa64_vmalle1is_write(CPUARMState *env, const ARMCPRegInfo *ri, tlb_flush_by_mmuidx_all_cpus_synced(cs, mask); } -static void tlbi_aa64_vmalle1_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) -{ - CPUState *cs = env_cpu(env); - int mask = vae1_tlbmask(env); - - if (tlb_force_broadcast(env)) { - tlb_flush_by_mmuidx_all_cpus_synced(cs, mask); - } else { - tlb_flush_by_mmuidx(cs, mask); - } -} - static int e2_tlbmask(CPUARMState *env) { return (ARMMMUIdxBit_E20_0 | @@ -XXX,XX +XXX,XX @@ static int e2_tlbmask(CPUARMState *env) ARMMMUIdxBit_E2); } -static void tlbi_aa64_alle1_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) -{ - CPUState *cs = env_cpu(env); - int mask = alle1_tlbmask(env); - - tlb_flush_by_mmuidx(cs, mask); -} - static void tlbi_aa64_alle2_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value) { @@ -XXX,XX +XXX,XX @@ static void tlbi_aa64_alle3_write(CPUARMState *env, const ARMCPRegInfo *ri, tlb_flush_by_mmuidx(cs, ARMMMUIdxBit_E3); } -static void tlbi_aa64_alle1is_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) +void tlbi_aa64_alle1is_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) { CPUState *cs = env_cpu(env); int mask = alle1_tlbmask(env); @@ -XXX,XX +XXX,XX @@ static void tlbi_aa64_vae3_write(CPUARMState *env, const ARMCPRegInfo *ri, tlb_flush_page_by_mmuidx(cs, pageaddr, ARMMMUIdxBit_E3); } -static void tlbi_aa64_vae1is_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) +void tlbi_aa64_vae1is_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) { CPUState *cs = env_cpu(env); int mask = vae1_tlbmask(env); @@ -XXX,XX +XXX,XX @@ static void tlbi_aa64_vae1is_write(CPUARMState *env, const ARMCPRegInfo *ri, tlb_flush_page_bits_by_mmuidx_all_cpus_synced(cs, pageaddr, mask, bits); } -static void tlbi_aa64_vae1_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) -{ - /* - * Invalidate by VA, EL1&0 (AArch64 version). - * Currently handles all of VAE1, VAAE1, VAALE1 and VALE1, - * since we don't support flush-for-specific-ASID-only or - * flush-last-level-only. - */ - CPUState *cs = env_cpu(env); - int mask = vae1_tlbmask(env); - uint64_t pageaddr = sextract64(value << 12, 0, 56); - int bits = vae1_tlbbits(env, pageaddr); - - if (tlb_force_broadcast(env)) { - tlb_flush_page_bits_by_mmuidx_all_cpus_synced(cs, pageaddr, mask, bits); - } else { - tlb_flush_page_bits_by_mmuidx(cs, pageaddr, mask, bits); - } -} - static void tlbi_aa64_vae2is_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value) { @@ -XXX,XX +XXX,XX @@ static void tlbi_aa64_vae3is_write(CPUARMState *env, const ARMCPRegInfo *ri, ARMMMUIdxBit_E3, bits); } -static int ipas2e1_tlbmask(CPUARMState *env, int64_t value) +int ipas2e1_tlbmask(CPUARMState *env, int64_t value) { /* * The MSB of value is the NS field, which only applies if SEL2 @@ -XXX,XX +XXX,XX @@ static int ipas2e1_tlbmask(CPUARMState *env, int64_t value) : ARMMMUIdxBit_Stage2); } -static void tlbi_aa64_ipas2e1_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) -{ - CPUState *cs = env_cpu(env); - int mask = ipas2e1_tlbmask(env, value); - uint64_t pageaddr = sextract64(value << 12, 0, 56); - - if (tlb_force_broadcast(env)) { - tlb_flush_page_by_mmuidx_all_cpus_synced(cs, pageaddr, mask); - } else { - tlb_flush_page_by_mmuidx(cs, pageaddr, mask); - } -} - -static void tlbi_aa64_ipas2e1is_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) -{ - CPUState *cs = env_cpu(env); - int mask = ipas2e1_tlbmask(env, value); - uint64_t pageaddr = sextract64(value << 12, 0, 56); - - tlb_flush_page_by_mmuidx_all_cpus_synced(cs, pageaddr, mask); -} - #ifdef TARGET_AARCH64 typedef struct { uint64_t base; @@ -XXX,XX +XXX,XX @@ static const ARMCPRegInfo v8_cp_reginfo[] = { .opc0 = 1, .opc1 = 0, .crn = 7, .crm = 14, .opc2 = 2, .fgt = FGT_DCCISW, .access = PL1_W, .accessfn = access_tsw, .type = ARM_CP_NOP }, - /* TLBI operations */ - { .name = "TLBI_VMALLE1IS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 3, .opc2 = 0, - .access = PL1_W, .accessfn = access_ttlbis, .type = ARM_CP_NO_RAW, - .fgt = FGT_TLBIVMALLE1IS, - .writefn = tlbi_aa64_vmalle1is_write }, - { .name = "TLBI_VAE1IS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 3, .opc2 = 1, - .access = PL1_W, .accessfn = access_ttlbis, .type = ARM_CP_NO_RAW, - .fgt = FGT_TLBIVAE1IS, - .writefn = tlbi_aa64_vae1is_write }, - { .name = "TLBI_ASIDE1IS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 3, .opc2 = 2, - .access = PL1_W, .accessfn = access_ttlbis, .type = ARM_CP_NO_RAW, - .fgt = FGT_TLBIASIDE1IS, - .writefn = tlbi_aa64_vmalle1is_write }, - { .name = "TLBI_VAAE1IS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 3, .opc2 = 3, - .access = PL1_W, .accessfn = access_ttlbis, .type = ARM_CP_NO_RAW, - .fgt = FGT_TLBIVAAE1IS, - .writefn = tlbi_aa64_vae1is_write }, - { .name = "TLBI_VALE1IS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 3, .opc2 = 5, - .access = PL1_W, .accessfn = access_ttlbis, .type = ARM_CP_NO_RAW, - .fgt = FGT_TLBIVALE1IS, - .writefn = tlbi_aa64_vae1is_write }, - { .name = "TLBI_VAALE1IS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 3, .opc2 = 7, - .access = PL1_W, .accessfn = access_ttlbis, .type = ARM_CP_NO_RAW, - .fgt = FGT_TLBIVAALE1IS, - .writefn = tlbi_aa64_vae1is_write }, - { .name = "TLBI_VMALLE1", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 7, .opc2 = 0, - .access = PL1_W, .accessfn = access_ttlb, .type = ARM_CP_NO_RAW, - .fgt = FGT_TLBIVMALLE1, - .writefn = tlbi_aa64_vmalle1_write }, - { .name = "TLBI_VAE1", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 7, .opc2 = 1, - .access = PL1_W, .accessfn = access_ttlb, .type = ARM_CP_NO_RAW, - .fgt = FGT_TLBIVAE1, - .writefn = tlbi_aa64_vae1_write }, - { .name = "TLBI_ASIDE1", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 7, .opc2 = 2, - .access = PL1_W, .accessfn = access_ttlb, .type = ARM_CP_NO_RAW, - .fgt = FGT_TLBIASIDE1, - .writefn = tlbi_aa64_vmalle1_write }, - { .name = "TLBI_VAAE1", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 7, .opc2 = 3, - .access = PL1_W, .accessfn = access_ttlb, .type = ARM_CP_NO_RAW, - .fgt = FGT_TLBIVAAE1, - .writefn = tlbi_aa64_vae1_write }, - { .name = "TLBI_VALE1", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 7, .opc2 = 5, - .access = PL1_W, .accessfn = access_ttlb, .type = ARM_CP_NO_RAW, - .fgt = FGT_TLBIVALE1, - .writefn = tlbi_aa64_vae1_write }, - { .name = "TLBI_VAALE1", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 7, .opc2 = 7, - .access = PL1_W, .accessfn = access_ttlb, .type = ARM_CP_NO_RAW, - .fgt = FGT_TLBIVAALE1, - .writefn = tlbi_aa64_vae1_write }, - { .name = "TLBI_IPAS2E1IS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 0, .opc2 = 1, - .access = PL2_W, .type = ARM_CP_NO_RAW, - .writefn = tlbi_aa64_ipas2e1is_write }, - { .name = "TLBI_IPAS2LE1IS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 0, .opc2 = 5, - .access = PL2_W, .type = ARM_CP_NO_RAW, - .writefn = tlbi_aa64_ipas2e1is_write }, - { .name = "TLBI_ALLE1IS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 3, .opc2 = 4, - .access = PL2_W, .type = ARM_CP_NO_RAW, - .writefn = tlbi_aa64_alle1is_write }, - { .name = "TLBI_VMALLS12E1IS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 3, .opc2 = 6, - .access = PL2_W, .type = ARM_CP_NO_RAW, - .writefn = tlbi_aa64_alle1is_write }, - { .name = "TLBI_IPAS2E1", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 4, .opc2 = 1, - .access = PL2_W, .type = ARM_CP_NO_RAW, - .writefn = tlbi_aa64_ipas2e1_write }, - { .name = "TLBI_IPAS2LE1", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 4, .opc2 = 5, - .access = PL2_W, .type = ARM_CP_NO_RAW, - .writefn = tlbi_aa64_ipas2e1_write }, - { .name = "TLBI_ALLE1", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 7, .opc2 = 4, - .access = PL2_W, .type = ARM_CP_NO_RAW, - .writefn = tlbi_aa64_alle1_write }, - { .name = "TLBI_VMALLS12E1", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 7, .opc2 = 6, - .access = PL2_W, .type = ARM_CP_NO_RAW, - .writefn = tlbi_aa64_alle1is_write }, #ifndef CONFIG_USER_ONLY /* 64 bit address translation operations */ { .name = "AT_S1E1R", .state = ARM_CP_STATE_AA64, diff --git a/target/arm/tcg/tlb-insns.c b/target/arm/tcg/tlb-insns.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/tlb-insns.c +++ b/target/arm/tcg/tlb-insns.c @@ -XXX,XX +XXX,XX @@ static void tlbiall_hyp_is_write(CPUARMState *env, const ARMCPRegInfo *ri, tlb_flush_by_mmuidx_all_cpus_synced(cs, ARMMMUIdxBit_E2); } +static void tlbi_aa64_vmalle1_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) +{ + CPUState *cs = env_cpu(env); + int mask = vae1_tlbmask(env); + + if (tlb_force_broadcast(env)) { + tlb_flush_by_mmuidx_all_cpus_synced(cs, mask); + } else { + tlb_flush_by_mmuidx(cs, mask); + } +} + +static void tlbi_aa64_alle1_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) +{ + CPUState *cs = env_cpu(env); + int mask = alle1_tlbmask(env); + + tlb_flush_by_mmuidx(cs, mask); +} + +static void tlbi_aa64_vae1_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) +{ + /* + * Invalidate by VA, EL1&0 (AArch64 version). + * Currently handles all of VAE1, VAAE1, VAALE1 and VALE1, + * since we don't support flush-for-specific-ASID-only or + * flush-last-level-only. + */ + CPUState *cs = env_cpu(env); + int mask = vae1_tlbmask(env); + uint64_t pageaddr = sextract64(value << 12, 0, 56); + int bits = vae1_tlbbits(env, pageaddr); + + if (tlb_force_broadcast(env)) { + tlb_flush_page_bits_by_mmuidx_all_cpus_synced(cs, pageaddr, mask, bits); + } else { + tlb_flush_page_bits_by_mmuidx(cs, pageaddr, mask, bits); + } +} + +static void tlbi_aa64_ipas2e1_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) +{ + CPUState *cs = env_cpu(env); + int mask = ipas2e1_tlbmask(env, value); + uint64_t pageaddr = sextract64(value << 12, 0, 56); + + if (tlb_force_broadcast(env)) { + tlb_flush_page_by_mmuidx_all_cpus_synced(cs, pageaddr, mask); + } else { + tlb_flush_page_by_mmuidx(cs, pageaddr, mask); + } +} + +static void tlbi_aa64_ipas2e1is_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) +{ + CPUState *cs = env_cpu(env); + int mask = ipas2e1_tlbmask(env, value); + uint64_t pageaddr = sextract64(value << 12, 0, 56); + + tlb_flush_page_by_mmuidx_all_cpus_synced(cs, pageaddr, mask); +} + static const ARMCPRegInfo tlbi_not_v7_cp_reginfo[] = { /* * MMU TLB control. Note that the wildcarding means we cover not just @@ -XXX,XX +XXX,XX @@ static const ARMCPRegInfo tlbi_v8_cp_reginfo[] = { .cp = 15, .opc1 = 4, .crn = 8, .crm = 0, .opc2 = 5, .type = ARM_CP_NO_RAW, .access = PL2_W, .writefn = tlbiipas2is_hyp_write }, + /* AArch64 TLBI operations */ + { .name = "TLBI_VMALLE1IS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 3, .opc2 = 0, + .access = PL1_W, .accessfn = access_ttlbis, .type = ARM_CP_NO_RAW, + .fgt = FGT_TLBIVMALLE1IS, + .writefn = tlbi_aa64_vmalle1is_write }, + { .name = "TLBI_VAE1IS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 3, .opc2 = 1, + .access = PL1_W, .accessfn = access_ttlbis, .type = ARM_CP_NO_RAW, + .fgt = FGT_TLBIVAE1IS, + .writefn = tlbi_aa64_vae1is_write }, + { .name = "TLBI_ASIDE1IS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 3, .opc2 = 2, + .access = PL1_W, .accessfn = access_ttlbis, .type = ARM_CP_NO_RAW, + .fgt = FGT_TLBIASIDE1IS, + .writefn = tlbi_aa64_vmalle1is_write }, + { .name = "TLBI_VAAE1IS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 3, .opc2 = 3, + .access = PL1_W, .accessfn = access_ttlbis, .type = ARM_CP_NO_RAW, + .fgt = FGT_TLBIVAAE1IS, + .writefn = tlbi_aa64_vae1is_write }, + { .name = "TLBI_VALE1IS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 3, .opc2 = 5, + .access = PL1_W, .accessfn = access_ttlbis, .type = ARM_CP_NO_RAW, + .fgt = FGT_TLBIVALE1IS, + .writefn = tlbi_aa64_vae1is_write }, + { .name = "TLBI_VAALE1IS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 3, .opc2 = 7, + .access = PL1_W, .accessfn = access_ttlbis, .type = ARM_CP_NO_RAW, + .fgt = FGT_TLBIVAALE1IS, + .writefn = tlbi_aa64_vae1is_write }, + { .name = "TLBI_VMALLE1", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 7, .opc2 = 0, + .access = PL1_W, .accessfn = access_ttlb, .type = ARM_CP_NO_RAW, + .fgt = FGT_TLBIVMALLE1, + .writefn = tlbi_aa64_vmalle1_write }, + { .name = "TLBI_VAE1", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 7, .opc2 = 1, + .access = PL1_W, .accessfn = access_ttlb, .type = ARM_CP_NO_RAW, + .fgt = FGT_TLBIVAE1, + .writefn = tlbi_aa64_vae1_write }, + { .name = "TLBI_ASIDE1", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 7, .opc2 = 2, + .access = PL1_W, .accessfn = access_ttlb, .type = ARM_CP_NO_RAW, + .fgt = FGT_TLBIASIDE1, + .writefn = tlbi_aa64_vmalle1_write }, + { .name = "TLBI_VAAE1", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 7, .opc2 = 3, + .access = PL1_W, .accessfn = access_ttlb, .type = ARM_CP_NO_RAW, + .fgt = FGT_TLBIVAAE1, + .writefn = tlbi_aa64_vae1_write }, + { .name = "TLBI_VALE1", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 7, .opc2 = 5, + .access = PL1_W, .accessfn = access_ttlb, .type = ARM_CP_NO_RAW, + .fgt = FGT_TLBIVALE1, + .writefn = tlbi_aa64_vae1_write }, + { .name = "TLBI_VAALE1", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 7, .opc2 = 7, + .access = PL1_W, .accessfn = access_ttlb, .type = ARM_CP_NO_RAW, + .fgt = FGT_TLBIVAALE1, + .writefn = tlbi_aa64_vae1_write }, + { .name = "TLBI_IPAS2E1IS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 0, .opc2 = 1, + .access = PL2_W, .type = ARM_CP_NO_RAW, + .writefn = tlbi_aa64_ipas2e1is_write }, + { .name = "TLBI_IPAS2LE1IS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 0, .opc2 = 5, + .access = PL2_W, .type = ARM_CP_NO_RAW, + .writefn = tlbi_aa64_ipas2e1is_write }, + { .name = "TLBI_ALLE1IS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 3, .opc2 = 4, + .access = PL2_W, .type = ARM_CP_NO_RAW, + .writefn = tlbi_aa64_alle1is_write }, + { .name = "TLBI_VMALLS12E1IS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 3, .opc2 = 6, + .access = PL2_W, .type = ARM_CP_NO_RAW, + .writefn = tlbi_aa64_alle1is_write }, + { .name = "TLBI_IPAS2E1", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 4, .opc2 = 1, + .access = PL2_W, .type = ARM_CP_NO_RAW, + .writefn = tlbi_aa64_ipas2e1_write }, + { .name = "TLBI_IPAS2LE1", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 4, .opc2 = 5, + .access = PL2_W, .type = ARM_CP_NO_RAW, + .writefn = tlbi_aa64_ipas2e1_write }, + { .name = "TLBI_ALLE1", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 7, .opc2 = 4, + .access = PL2_W, .type = ARM_CP_NO_RAW, + .writefn = tlbi_aa64_alle1_write }, + { .name = "TLBI_VMALLS12E1", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 7, .opc2 = 6, + .access = PL2_W, .type = ARM_CP_NO_RAW, + .writefn = tlbi_aa64_alle1is_write }, }; static const ARMCPRegInfo tlbi_el2_cp_reginfo[] = { -- 2.34.1 Move the AArch64 EL2 TLBI insn definitions that were in el2_cp_reginfo[] across to tlb-insns.c. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241210160452.2427965-5-peter.maydell@linaro.org --- target/arm/cpregs.h | 7 +++++ target/arm/helper.c | 61 ++++---------------------------------- target/arm/tcg/tlb-insns.c | 49 ++++++++++++++++++++++++++++++ 3 files changed, 62 insertions(+), 55 deletions(-) diff --git a/target/arm/cpregs.h b/target/arm/cpregs.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpregs.h +++ b/target/arm/cpregs.h @@ -XXX,XX +XXX,XX @@ bool tlb_force_broadcast(CPUARMState *env); int tlbbits_for_regime(CPUARMState *env, ARMMMUIdx mmu_idx, uint64_t addr); int vae1_tlbbits(CPUARMState *env, uint64_t addr); +int vae2_tlbbits(CPUARMState *env, uint64_t addr); int vae1_tlbmask(CPUARMState *env); +int vae2_tlbmask(CPUARMState *env); int ipas2e1_tlbmask(CPUARMState *env, int64_t value); +int e2_tlbmask(CPUARMState *env); void tlbi_aa64_vmalle1is_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value); void tlbi_aa64_alle1is_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value); void tlbi_aa64_vae1is_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value); +void tlbi_aa64_alle2is_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value); +void tlbi_aa64_vae2is_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value); #endif /* TARGET_ARM_CPREGS_H */ diff --git a/target/arm/helper.c b/target/arm/helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -XXX,XX +XXX,XX @@ int vae1_tlbmask(CPUARMState *env) return mask; } -static int vae2_tlbmask(CPUARMState *env) +int vae2_tlbmask(CPUARMState *env) { uint64_t hcr = arm_hcr_el2_eff(env); uint16_t mask; @@ -XXX,XX +XXX,XX @@ int vae1_tlbbits(CPUARMState *env, uint64_t addr) return tlbbits_for_regime(env, mmu_idx, addr); } -static int vae2_tlbbits(CPUARMState *env, uint64_t addr) +int vae2_tlbbits(CPUARMState *env, uint64_t addr) { uint64_t hcr = arm_hcr_el2_eff(env); ARMMMUIdx mmu_idx; @@ -XXX,XX +XXX,XX @@ void tlbi_aa64_vmalle1is_write(CPUARMState *env, const ARMCPRegInfo *ri, tlb_flush_by_mmuidx_all_cpus_synced(cs, mask); } -static int e2_tlbmask(CPUARMState *env) +int e2_tlbmask(CPUARMState *env) { return (ARMMMUIdxBit_E20_0 | ARMMMUIdxBit_E20_2 | @@ -XXX,XX +XXX,XX @@ static int e2_tlbmask(CPUARMState *env) ARMMMUIdxBit_E2); } -static void tlbi_aa64_alle2_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) -{ - CPUState *cs = env_cpu(env); - int mask = e2_tlbmask(env); - - tlb_flush_by_mmuidx(cs, mask); -} - static void tlbi_aa64_alle3_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value) { @@ -XXX,XX +XXX,XX @@ void tlbi_aa64_alle1is_write(CPUARMState *env, const ARMCPRegInfo *ri, tlb_flush_by_mmuidx_all_cpus_synced(cs, mask); } -static void tlbi_aa64_alle2is_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) +void tlbi_aa64_alle2is_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) { CPUState *cs = env_cpu(env); int mask = e2_tlbmask(env); @@ -XXX,XX +XXX,XX @@ static void tlbi_aa64_alle3is_write(CPUARMState *env, const ARMCPRegInfo *ri, tlb_flush_by_mmuidx_all_cpus_synced(cs, ARMMMUIdxBit_E3); } -static void tlbi_aa64_vae2_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) -{ - /* - * Invalidate by VA, EL2 - * Currently handles both VAE2 and VALE2, since we don't support - * flush-last-level-only. - */ - CPUState *cs = env_cpu(env); - int mask = vae2_tlbmask(env); - uint64_t pageaddr = sextract64(value << 12, 0, 56); - int bits = vae2_tlbbits(env, pageaddr); - - tlb_flush_page_bits_by_mmuidx(cs, pageaddr, mask, bits); -} - static void tlbi_aa64_vae3_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value) { @@ -XXX,XX +XXX,XX @@ void tlbi_aa64_vae1is_write(CPUARMState *env, const ARMCPRegInfo *ri, tlb_flush_page_bits_by_mmuidx_all_cpus_synced(cs, pageaddr, mask, bits); } -static void tlbi_aa64_vae2is_write(CPUARMState *env, const ARMCPRegInfo *ri, +void tlbi_aa64_vae2is_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value) { CPUState *cs = env_cpu(env); @@ -XXX,XX +XXX,XX @@ static const ARMCPRegInfo el2_cp_reginfo[] = { { .name = "HTTBR", .cp = 15, .opc1 = 4, .crm = 2, .access = PL2_RW, .type = ARM_CP_64BIT | ARM_CP_ALIAS, .fieldoffset = offsetof(CPUARMState, cp15.ttbr0_el[2]) }, - { .name = "TLBI_ALLE2", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 7, .opc2 = 0, - .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF, - .writefn = tlbi_aa64_alle2_write }, - { .name = "TLBI_VAE2", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 7, .opc2 = 1, - .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF, - .writefn = tlbi_aa64_vae2_write }, - { .name = "TLBI_VALE2", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 7, .opc2 = 5, - .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF, - .writefn = tlbi_aa64_vae2_write }, - { .name = "TLBI_ALLE2IS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 3, .opc2 = 0, - .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF, - .writefn = tlbi_aa64_alle2is_write }, - { .name = "TLBI_VAE2IS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 3, .opc2 = 1, - .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF, - .writefn = tlbi_aa64_vae2is_write }, - { .name = "TLBI_VALE2IS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 3, .opc2 = 5, - .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF, - .writefn = tlbi_aa64_vae2is_write }, #ifndef CONFIG_USER_ONLY /* * Unlike the other EL2-related AT operations, these must diff --git a/target/arm/tcg/tlb-insns.c b/target/arm/tcg/tlb-insns.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/tlb-insns.c +++ b/target/arm/tcg/tlb-insns.c @@ -XXX,XX +XXX,XX @@ static void tlbi_aa64_alle1_write(CPUARMState *env, const ARMCPRegInfo *ri, tlb_flush_by_mmuidx(cs, mask); } +static void tlbi_aa64_alle2_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) +{ + CPUState *cs = env_cpu(env); + int mask = e2_tlbmask(env); + + tlb_flush_by_mmuidx(cs, mask); +} + +static void tlbi_aa64_vae2_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) +{ + /* + * Invalidate by VA, EL2 + * Currently handles both VAE2 and VALE2, since we don't support + * flush-last-level-only. + */ + CPUState *cs = env_cpu(env); + int mask = vae2_tlbmask(env); + uint64_t pageaddr = sextract64(value << 12, 0, 56); + int bits = vae2_tlbbits(env, pageaddr); + + tlb_flush_page_bits_by_mmuidx(cs, pageaddr, mask, bits); +} + static void tlbi_aa64_vae1_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value) { @@ -XXX,XX +XXX,XX @@ static const ARMCPRegInfo tlbi_el2_cp_reginfo[] = { { .name = "TLBIMVAHIS", .cp = 15, .opc1 = 4, .crn = 8, .crm = 3, .opc2 = 1, .type = ARM_CP_NO_RAW, .access = PL2_W, .writefn = tlbimva_hyp_is_write }, + { .name = "TLBI_ALLE2", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 7, .opc2 = 0, + .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF, + .writefn = tlbi_aa64_alle2_write }, + { .name = "TLBI_VAE2", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 7, .opc2 = 1, + .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF, + .writefn = tlbi_aa64_vae2_write }, + { .name = "TLBI_VALE2", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 7, .opc2 = 5, + .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF, + .writefn = tlbi_aa64_vae2_write }, + { .name = "TLBI_ALLE2IS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 3, .opc2 = 0, + .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF, + .writefn = tlbi_aa64_alle2is_write }, + { .name = "TLBI_VAE2IS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 3, .opc2 = 1, + .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF, + .writefn = tlbi_aa64_vae2is_write }, + { .name = "TLBI_VALE2IS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 3, .opc2 = 5, + .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF, + .writefn = tlbi_aa64_vae2is_write }, }; void define_tlb_insn_regs(ARMCPU *cpu) -- 2.34.1 Move the AArch64 EL3 TLBI insns from el3_cp_reginfo[] across to tlb-insns.c. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241210160452.2427965-6-peter.maydell@linaro.org --- target/arm/cpregs.h | 4 +++ target/arm/helper.c | 56 +++----------------------------------- target/arm/tcg/tlb-insns.c | 54 ++++++++++++++++++++++++++++++++++++ 3 files changed, 62 insertions(+), 52 deletions(-) diff --git a/target/arm/cpregs.h b/target/arm/cpregs.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpregs.h +++ b/target/arm/cpregs.h @@ -XXX,XX +XXX,XX @@ void tlbi_aa64_alle2is_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value); void tlbi_aa64_vae2is_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value); +void tlbi_aa64_vae3is_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value); +void tlbi_aa64_alle3is_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value); #endif /* TARGET_ARM_CPREGS_H */ diff --git a/target/arm/helper.c b/target/arm/helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -XXX,XX +XXX,XX @@ int e2_tlbmask(CPUARMState *env) ARMMMUIdxBit_E2); } -static void tlbi_aa64_alle3_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) -{ - ARMCPU *cpu = env_archcpu(env); - CPUState *cs = CPU(cpu); - - tlb_flush_by_mmuidx(cs, ARMMMUIdxBit_E3); -} - void tlbi_aa64_alle1is_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value) { @@ -XXX,XX +XXX,XX @@ void tlbi_aa64_alle2is_write(CPUARMState *env, const ARMCPRegInfo *ri, tlb_flush_by_mmuidx_all_cpus_synced(cs, mask); } -static void tlbi_aa64_alle3is_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) +void tlbi_aa64_alle3is_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) { CPUState *cs = env_cpu(env); tlb_flush_by_mmuidx_all_cpus_synced(cs, ARMMMUIdxBit_E3); } -static void tlbi_aa64_vae3_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) -{ - /* - * Invalidate by VA, EL3 - * Currently handles both VAE3 and VALE3, since we don't support - * flush-last-level-only. - */ - ARMCPU *cpu = env_archcpu(env); - CPUState *cs = CPU(cpu); - uint64_t pageaddr = sextract64(value << 12, 0, 56); - - tlb_flush_page_by_mmuidx(cs, pageaddr, ARMMMUIdxBit_E3); -} - void tlbi_aa64_vae1is_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value) { @@ -XXX,XX +XXX,XX @@ void tlbi_aa64_vae2is_write(CPUARMState *env, const ARMCPRegInfo *ri, tlb_flush_page_bits_by_mmuidx_all_cpus_synced(cs, pageaddr, mask, bits); } -static void tlbi_aa64_vae3is_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) +void tlbi_aa64_vae3is_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) { CPUState *cs = env_cpu(env); uint64_t pageaddr = sextract64(value << 12, 0, 56); @@ -XXX,XX +XXX,XX @@ static const ARMCPRegInfo el3_cp_reginfo[] = { .opc0 = 3, .opc1 = 6, .crn = 5, .crm = 1, .opc2 = 1, .access = PL3_RW, .type = ARM_CP_CONST, .resetvalue = 0 }, - { .name = "TLBI_ALLE3IS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 3, .opc2 = 0, - .access = PL3_W, .type = ARM_CP_NO_RAW, - .writefn = tlbi_aa64_alle3is_write }, - { .name = "TLBI_VAE3IS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 3, .opc2 = 1, - .access = PL3_W, .type = ARM_CP_NO_RAW, - .writefn = tlbi_aa64_vae3is_write }, - { .name = "TLBI_VALE3IS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 3, .opc2 = 5, - .access = PL3_W, .type = ARM_CP_NO_RAW, - .writefn = tlbi_aa64_vae3is_write }, - { .name = "TLBI_ALLE3", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 7, .opc2 = 0, - .access = PL3_W, .type = ARM_CP_NO_RAW, - .writefn = tlbi_aa64_alle3_write }, - { .name = "TLBI_VAE3", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 7, .opc2 = 1, - .access = PL3_W, .type = ARM_CP_NO_RAW, - .writefn = tlbi_aa64_vae3_write }, - { .name = "TLBI_VALE3", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 7, .opc2 = 5, - .access = PL3_W, .type = ARM_CP_NO_RAW, - .writefn = tlbi_aa64_vae3_write }, }; #ifndef CONFIG_USER_ONLY diff --git a/target/arm/tcg/tlb-insns.c b/target/arm/tcg/tlb-insns.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/tlb-insns.c +++ b/target/arm/tcg/tlb-insns.c @@ -XXX,XX +XXX,XX @@ static void tlbi_aa64_alle2_write(CPUARMState *env, const ARMCPRegInfo *ri, tlb_flush_by_mmuidx(cs, mask); } +static void tlbi_aa64_alle3_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) +{ + ARMCPU *cpu = env_archcpu(env); + CPUState *cs = CPU(cpu); + + tlb_flush_by_mmuidx(cs, ARMMMUIdxBit_E3); +} + static void tlbi_aa64_vae2_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value) { @@ -XXX,XX +XXX,XX @@ static void tlbi_aa64_vae2_write(CPUARMState *env, const ARMCPRegInfo *ri, tlb_flush_page_bits_by_mmuidx(cs, pageaddr, mask, bits); } +static void tlbi_aa64_vae3_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) +{ + /* + * Invalidate by VA, EL3 + * Currently handles both VAE3 and VALE3, since we don't support + * flush-last-level-only. + */ + ARMCPU *cpu = env_archcpu(env); + CPUState *cs = CPU(cpu); + uint64_t pageaddr = sextract64(value << 12, 0, 56); + + tlb_flush_page_by_mmuidx(cs, pageaddr, ARMMMUIdxBit_E3); +} + static void tlbi_aa64_vae1_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value) { @@ -XXX,XX +XXX,XX @@ static const ARMCPRegInfo tlbi_el2_cp_reginfo[] = { .writefn = tlbi_aa64_vae2is_write }, }; +static const ARMCPRegInfo tlbi_el3_cp_reginfo[] = { + { .name = "TLBI_ALLE3IS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 3, .opc2 = 0, + .access = PL3_W, .type = ARM_CP_NO_RAW, + .writefn = tlbi_aa64_alle3is_write }, + { .name = "TLBI_VAE3IS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 3, .opc2 = 1, + .access = PL3_W, .type = ARM_CP_NO_RAW, + .writefn = tlbi_aa64_vae3is_write }, + { .name = "TLBI_VALE3IS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 3, .opc2 = 5, + .access = PL3_W, .type = ARM_CP_NO_RAW, + .writefn = tlbi_aa64_vae3is_write }, + { .name = "TLBI_ALLE3", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 7, .opc2 = 0, + .access = PL3_W, .type = ARM_CP_NO_RAW, + .writefn = tlbi_aa64_alle3_write }, + { .name = "TLBI_VAE3", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 7, .opc2 = 1, + .access = PL3_W, .type = ARM_CP_NO_RAW, + .writefn = tlbi_aa64_vae3_write }, + { .name = "TLBI_VALE3", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 7, .opc2 = 5, + .access = PL3_W, .type = ARM_CP_NO_RAW, + .writefn = tlbi_aa64_vae3_write }, +}; + void define_tlb_insn_regs(ARMCPU *cpu) { CPUARMState *env = &cpu->env; @@ -XXX,XX +XXX,XX @@ void define_tlb_insn_regs(ARMCPU *cpu) && arm_feature(env, ARM_FEATURE_V8))) { define_arm_cp_regs(cpu, tlbi_el2_cp_reginfo); } + if (arm_feature(env, ARM_FEATURE_EL3)) { + define_arm_cp_regs(cpu, tlbi_el3_cp_reginfo); + } } -- 2.34.1 Move the TLBI invalidate-range insns across to tlb-insns.c. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241210160452.2427965-7-peter.maydell@linaro.org --- target/arm/cpregs.h | 2 + target/arm/helper.c | 330 +------------------------------------ target/arm/tcg/tlb-insns.c | 329 ++++++++++++++++++++++++++++++++++++ 3 files changed, 333 insertions(+), 328 deletions(-) diff --git a/target/arm/cpregs.h b/target/arm/cpregs.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpregs.h +++ b/target/arm/cpregs.h @@ -XXX,XX +XXX,XX @@ CPAccessResult access_ttlb(CPUARMState *env, const ARMCPRegInfo *ri, bool isread); CPAccessResult access_ttlbis(CPUARMState *env, const ARMCPRegInfo *ri, bool isread); +CPAccessResult access_ttlbos(CPUARMState *env, const ARMCPRegInfo *ri, + bool isread); bool tlb_force_broadcast(CPUARMState *env); int tlbbits_for_regime(CPUARMState *env, ARMMMUIdx mmu_idx, uint64_t addr); diff --git a/target/arm/helper.c b/target/arm/helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -XXX,XX +XXX,XX @@ CPAccessResult access_ttlbis(CPUARMState *env, const ARMCPRegInfo *ri, #ifdef TARGET_AARCH64 /* Check for traps from EL1 due to HCR_EL2.TTLB or TTLBOS. */ -static CPAccessResult access_ttlbos(CPUARMState *env, const ARMCPRegInfo *ri, - bool isread) +CPAccessResult access_ttlbos(CPUARMState *env, const ARMCPRegInfo *ri, + bool isread) { if (arm_current_el(env) == 1 && (arm_hcr_el2_eff(env) & (HCR_TTLB | HCR_TTLBOS))) { @@ -XXX,XX +XXX,XX @@ int ipas2e1_tlbmask(CPUARMState *env, int64_t value) : ARMMMUIdxBit_Stage2); } -#ifdef TARGET_AARCH64 -typedef struct { - uint64_t base; - uint64_t length; -} TLBIRange; - -static ARMGranuleSize tlbi_range_tg_to_gran_size(int tg) -{ - /* - * Note that the TLBI range TG field encoding differs from both - * TG0 and TG1 encodings. - */ - switch (tg) { - case 1: - return Gran4K; - case 2: - return Gran16K; - case 3: - return Gran64K; - default: - return GranInvalid; - } -} - -static TLBIRange tlbi_aa64_get_range(CPUARMState *env, ARMMMUIdx mmuidx, - uint64_t value) -{ - unsigned int page_size_granule, page_shift, num, scale, exponent; - /* Extract one bit to represent the va selector in use. */ - uint64_t select = sextract64(value, 36, 1); - ARMVAParameters param = aa64_va_parameters(env, select, mmuidx, true, false); - TLBIRange ret = { }; - ARMGranuleSize gran; - - page_size_granule = extract64(value, 46, 2); - gran = tlbi_range_tg_to_gran_size(page_size_granule); - - /* The granule encoded in value must match the granule in use. */ - if (gran != param.gran) { - qemu_log_mask(LOG_GUEST_ERROR, "Invalid tlbi page size granule %d\n", - page_size_granule); - return ret; - } - - page_shift = arm_granule_bits(gran); - num = extract64(value, 39, 5); - scale = extract64(value, 44, 2); - exponent = (5 * scale) + 1; - - ret.length = (num + 1) << (exponent + page_shift); - - if (param.select) { - ret.base = sextract64(value, 0, 37); - } else { - ret.base = extract64(value, 0, 37); - } - if (param.ds) { - /* - * With DS=1, BaseADDR is always shifted 16 so that it is able - * to address all 52 va bits. The input address is perforce - * aligned on a 64k boundary regardless of translation granule. - */ - page_shift = 16; - } - ret.base <<= page_shift; - - return ret; -} - -static void do_rvae_write(CPUARMState *env, uint64_t value, - int idxmap, bool synced) -{ - ARMMMUIdx one_idx = ARM_MMU_IDX_A | ctz32(idxmap); - TLBIRange range; - int bits; - - range = tlbi_aa64_get_range(env, one_idx, value); - bits = tlbbits_for_regime(env, one_idx, range.base); - - if (synced) { - tlb_flush_range_by_mmuidx_all_cpus_synced(env_cpu(env), - range.base, - range.length, - idxmap, - bits); - } else { - tlb_flush_range_by_mmuidx(env_cpu(env), range.base, - range.length, idxmap, bits); - } -} - -static void tlbi_aa64_rvae1_write(CPUARMState *env, - const ARMCPRegInfo *ri, - uint64_t value) -{ - /* - * Invalidate by VA range, EL1&0. - * Currently handles all of RVAE1, RVAAE1, RVAALE1 and RVALE1, - * since we don't support flush-for-specific-ASID-only or - * flush-last-level-only. - */ - - do_rvae_write(env, value, vae1_tlbmask(env), - tlb_force_broadcast(env)); -} - -static void tlbi_aa64_rvae1is_write(CPUARMState *env, - const ARMCPRegInfo *ri, - uint64_t value) -{ - /* - * Invalidate by VA range, Inner/Outer Shareable EL1&0. - * Currently handles all of RVAE1IS, RVAE1OS, RVAAE1IS, RVAAE1OS, - * RVAALE1IS, RVAALE1OS, RVALE1IS and RVALE1OS, since we don't support - * flush-for-specific-ASID-only, flush-last-level-only or inner/outer - * shareable specific flushes. - */ - - do_rvae_write(env, value, vae1_tlbmask(env), true); -} - -static void tlbi_aa64_rvae2_write(CPUARMState *env, - const ARMCPRegInfo *ri, - uint64_t value) -{ - /* - * Invalidate by VA range, EL2. - * Currently handles all of RVAE2 and RVALE2, - * since we don't support flush-for-specific-ASID-only or - * flush-last-level-only. - */ - - do_rvae_write(env, value, vae2_tlbmask(env), - tlb_force_broadcast(env)); - - -} - -static void tlbi_aa64_rvae2is_write(CPUARMState *env, - const ARMCPRegInfo *ri, - uint64_t value) -{ - /* - * Invalidate by VA range, Inner/Outer Shareable, EL2. - * Currently handles all of RVAE2IS, RVAE2OS, RVALE2IS and RVALE2OS, - * since we don't support flush-for-specific-ASID-only, - * flush-last-level-only or inner/outer shareable specific flushes. - */ - - do_rvae_write(env, value, vae2_tlbmask(env), true); - -} - -static void tlbi_aa64_rvae3_write(CPUARMState *env, - const ARMCPRegInfo *ri, - uint64_t value) -{ - /* - * Invalidate by VA range, EL3. - * Currently handles all of RVAE3 and RVALE3, - * since we don't support flush-for-specific-ASID-only or - * flush-last-level-only. - */ - - do_rvae_write(env, value, ARMMMUIdxBit_E3, tlb_force_broadcast(env)); -} - -static void tlbi_aa64_rvae3is_write(CPUARMState *env, - const ARMCPRegInfo *ri, - uint64_t value) -{ - /* - * Invalidate by VA range, EL3, Inner/Outer Shareable. - * Currently handles all of RVAE3IS, RVAE3OS, RVALE3IS and RVALE3OS, - * since we don't support flush-for-specific-ASID-only, - * flush-last-level-only or inner/outer specific flushes. - */ - - do_rvae_write(env, value, ARMMMUIdxBit_E3, true); -} - -static void tlbi_aa64_ripas2e1_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) -{ - do_rvae_write(env, value, ipas2e1_tlbmask(env, value), - tlb_force_broadcast(env)); -} - -static void tlbi_aa64_ripas2e1is_write(CPUARMState *env, - const ARMCPRegInfo *ri, - uint64_t value) -{ - do_rvae_write(env, value, ipas2e1_tlbmask(env, value), true); -} -#endif - static CPAccessResult aa64_zva_access(CPUARMState *env, const ARMCPRegInfo *ri, bool isread) { @@ -XXX,XX +XXX,XX @@ static const ARMCPRegInfo pauth_reginfo[] = { .fieldoffset = offsetof(CPUARMState, keys.apib.hi) }, }; -static const ARMCPRegInfo tlbirange_reginfo[] = { - { .name = "TLBI_RVAE1IS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 2, .opc2 = 1, - .access = PL1_W, .accessfn = access_ttlbis, .type = ARM_CP_NO_RAW, - .fgt = FGT_TLBIRVAE1IS, - .writefn = tlbi_aa64_rvae1is_write }, - { .name = "TLBI_RVAAE1IS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 2, .opc2 = 3, - .access = PL1_W, .accessfn = access_ttlbis, .type = ARM_CP_NO_RAW, - .fgt = FGT_TLBIRVAAE1IS, - .writefn = tlbi_aa64_rvae1is_write }, - { .name = "TLBI_RVALE1IS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 2, .opc2 = 5, - .access = PL1_W, .accessfn = access_ttlbis, .type = ARM_CP_NO_RAW, - .fgt = FGT_TLBIRVALE1IS, - .writefn = tlbi_aa64_rvae1is_write }, - { .name = "TLBI_RVAALE1IS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 2, .opc2 = 7, - .access = PL1_W, .accessfn = access_ttlbis, .type = ARM_CP_NO_RAW, - .fgt = FGT_TLBIRVAALE1IS, - .writefn = tlbi_aa64_rvae1is_write }, - { .name = "TLBI_RVAE1OS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 5, .opc2 = 1, - .access = PL1_W, .accessfn = access_ttlbos, .type = ARM_CP_NO_RAW, - .fgt = FGT_TLBIRVAE1OS, - .writefn = tlbi_aa64_rvae1is_write }, - { .name = "TLBI_RVAAE1OS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 5, .opc2 = 3, - .access = PL1_W, .accessfn = access_ttlbos, .type = ARM_CP_NO_RAW, - .fgt = FGT_TLBIRVAAE1OS, - .writefn = tlbi_aa64_rvae1is_write }, - { .name = "TLBI_RVALE1OS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 5, .opc2 = 5, - .access = PL1_W, .accessfn = access_ttlbos, .type = ARM_CP_NO_RAW, - .fgt = FGT_TLBIRVALE1OS, - .writefn = tlbi_aa64_rvae1is_write }, - { .name = "TLBI_RVAALE1OS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 5, .opc2 = 7, - .access = PL1_W, .accessfn = access_ttlbos, .type = ARM_CP_NO_RAW, - .fgt = FGT_TLBIRVAALE1OS, - .writefn = tlbi_aa64_rvae1is_write }, - { .name = "TLBI_RVAE1", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 6, .opc2 = 1, - .access = PL1_W, .accessfn = access_ttlb, .type = ARM_CP_NO_RAW, - .fgt = FGT_TLBIRVAE1, - .writefn = tlbi_aa64_rvae1_write }, - { .name = "TLBI_RVAAE1", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 6, .opc2 = 3, - .access = PL1_W, .accessfn = access_ttlb, .type = ARM_CP_NO_RAW, - .fgt = FGT_TLBIRVAAE1, - .writefn = tlbi_aa64_rvae1_write }, - { .name = "TLBI_RVALE1", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 6, .opc2 = 5, - .access = PL1_W, .accessfn = access_ttlb, .type = ARM_CP_NO_RAW, - .fgt = FGT_TLBIRVALE1, - .writefn = tlbi_aa64_rvae1_write }, - { .name = "TLBI_RVAALE1", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 6, .opc2 = 7, - .access = PL1_W, .accessfn = access_ttlb, .type = ARM_CP_NO_RAW, - .fgt = FGT_TLBIRVAALE1, - .writefn = tlbi_aa64_rvae1_write }, - { .name = "TLBI_RIPAS2E1IS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 0, .opc2 = 2, - .access = PL2_W, .type = ARM_CP_NO_RAW, - .writefn = tlbi_aa64_ripas2e1is_write }, - { .name = "TLBI_RIPAS2LE1IS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 0, .opc2 = 6, - .access = PL2_W, .type = ARM_CP_NO_RAW, - .writefn = tlbi_aa64_ripas2e1is_write }, - { .name = "TLBI_RVAE2IS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 2, .opc2 = 1, - .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF, - .writefn = tlbi_aa64_rvae2is_write }, - { .name = "TLBI_RVALE2IS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 2, .opc2 = 5, - .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF, - .writefn = tlbi_aa64_rvae2is_write }, - { .name = "TLBI_RIPAS2E1", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 4, .opc2 = 2, - .access = PL2_W, .type = ARM_CP_NO_RAW, - .writefn = tlbi_aa64_ripas2e1_write }, - { .name = "TLBI_RIPAS2LE1", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 4, .opc2 = 6, - .access = PL2_W, .type = ARM_CP_NO_RAW, - .writefn = tlbi_aa64_ripas2e1_write }, - { .name = "TLBI_RVAE2OS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 5, .opc2 = 1, - .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF, - .writefn = tlbi_aa64_rvae2is_write }, - { .name = "TLBI_RVALE2OS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 5, .opc2 = 5, - .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF, - .writefn = tlbi_aa64_rvae2is_write }, - { .name = "TLBI_RVAE2", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 6, .opc2 = 1, - .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF, - .writefn = tlbi_aa64_rvae2_write }, - { .name = "TLBI_RVALE2", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 6, .opc2 = 5, - .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF, - .writefn = tlbi_aa64_rvae2_write }, - { .name = "TLBI_RVAE3IS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 2, .opc2 = 1, - .access = PL3_W, .type = ARM_CP_NO_RAW, - .writefn = tlbi_aa64_rvae3is_write }, - { .name = "TLBI_RVALE3IS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 2, .opc2 = 5, - .access = PL3_W, .type = ARM_CP_NO_RAW, - .writefn = tlbi_aa64_rvae3is_write }, - { .name = "TLBI_RVAE3OS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 5, .opc2 = 1, - .access = PL3_W, .type = ARM_CP_NO_RAW, - .writefn = tlbi_aa64_rvae3is_write }, - { .name = "TLBI_RVALE3OS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 5, .opc2 = 5, - .access = PL3_W, .type = ARM_CP_NO_RAW, - .writefn = tlbi_aa64_rvae3is_write }, - { .name = "TLBI_RVAE3", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 6, .opc2 = 1, - .access = PL3_W, .type = ARM_CP_NO_RAW, - .writefn = tlbi_aa64_rvae3_write }, - { .name = "TLBI_RVALE3", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 6, .opc2 = 5, - .access = PL3_W, .type = ARM_CP_NO_RAW, - .writefn = tlbi_aa64_rvae3_write }, -}; - static const ARMCPRegInfo tlbios_reginfo[] = { { .name = "TLBI_VMALLE1OS", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 1, .opc2 = 0, @@ -XXX,XX +XXX,XX @@ void register_cp_regs_for_features(ARMCPU *cpu) if (cpu_isar_feature(aa64_rndr, cpu)) { define_arm_cp_regs(cpu, rndr_reginfo); } - if (cpu_isar_feature(aa64_tlbirange, cpu)) { - define_arm_cp_regs(cpu, tlbirange_reginfo); - } if (cpu_isar_feature(aa64_tlbios, cpu)) { define_arm_cp_regs(cpu, tlbios_reginfo); } diff --git a/target/arm/tcg/tlb-insns.c b/target/arm/tcg/tlb-insns.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/tlb-insns.c +++ b/target/arm/tcg/tlb-insns.c @@ -XXX,XX +XXX,XX @@ * SPDX-License-Identifier: GPL-2.0-or-later */ #include "qemu/osdep.h" +#include "qemu/log.h" #include "exec/exec-all.h" #include "cpu.h" #include "internals.h" @@ -XXX,XX +XXX,XX @@ static const ARMCPRegInfo tlbi_el3_cp_reginfo[] = { .writefn = tlbi_aa64_vae3_write }, }; +#ifdef TARGET_AARCH64 +typedef struct { + uint64_t base; + uint64_t length; +} TLBIRange; + +static ARMGranuleSize tlbi_range_tg_to_gran_size(int tg) +{ + /* + * Note that the TLBI range TG field encoding differs from both + * TG0 and TG1 encodings. + */ + switch (tg) { + case 1: + return Gran4K; + case 2: + return Gran16K; + case 3: + return Gran64K; + default: + return GranInvalid; + } +} + +static TLBIRange tlbi_aa64_get_range(CPUARMState *env, ARMMMUIdx mmuidx, + uint64_t value) +{ + unsigned int page_size_granule, page_shift, num, scale, exponent; + /* Extract one bit to represent the va selector in use. */ + uint64_t select = sextract64(value, 36, 1); + ARMVAParameters param = aa64_va_parameters(env, select, mmuidx, true, false); + TLBIRange ret = { }; + ARMGranuleSize gran; + + page_size_granule = extract64(value, 46, 2); + gran = tlbi_range_tg_to_gran_size(page_size_granule); + + /* The granule encoded in value must match the granule in use. */ + if (gran != param.gran) { + qemu_log_mask(LOG_GUEST_ERROR, "Invalid tlbi page size granule %d\n", + page_size_granule); + return ret; + } + + page_shift = arm_granule_bits(gran); + num = extract64(value, 39, 5); + scale = extract64(value, 44, 2); + exponent = (5 * scale) + 1; + + ret.length = (num + 1) << (exponent + page_shift); + + if (param.select) { + ret.base = sextract64(value, 0, 37); + } else { + ret.base = extract64(value, 0, 37); + } + if (param.ds) { + /* + * With DS=1, BaseADDR is always shifted 16 so that it is able + * to address all 52 va bits. The input address is perforce + * aligned on a 64k boundary regardless of translation granule. + */ + page_shift = 16; + } + ret.base <<= page_shift; + + return ret; +} + +static void do_rvae_write(CPUARMState *env, uint64_t value, + int idxmap, bool synced) +{ + ARMMMUIdx one_idx = ARM_MMU_IDX_A | ctz32(idxmap); + TLBIRange range; + int bits; + + range = tlbi_aa64_get_range(env, one_idx, value); + bits = tlbbits_for_regime(env, one_idx, range.base); + + if (synced) { + tlb_flush_range_by_mmuidx_all_cpus_synced(env_cpu(env), + range.base, + range.length, + idxmap, + bits); + } else { + tlb_flush_range_by_mmuidx(env_cpu(env), range.base, + range.length, idxmap, bits); + } +} + +static void tlbi_aa64_rvae1_write(CPUARMState *env, + const ARMCPRegInfo *ri, + uint64_t value) +{ + /* + * Invalidate by VA range, EL1&0. + * Currently handles all of RVAE1, RVAAE1, RVAALE1 and RVALE1, + * since we don't support flush-for-specific-ASID-only or + * flush-last-level-only. + */ + + do_rvae_write(env, value, vae1_tlbmask(env), + tlb_force_broadcast(env)); +} + +static void tlbi_aa64_rvae1is_write(CPUARMState *env, + const ARMCPRegInfo *ri, + uint64_t value) +{ + /* + * Invalidate by VA range, Inner/Outer Shareable EL1&0. + * Currently handles all of RVAE1IS, RVAE1OS, RVAAE1IS, RVAAE1OS, + * RVAALE1IS, RVAALE1OS, RVALE1IS and RVALE1OS, since we don't support + * flush-for-specific-ASID-only, flush-last-level-only or inner/outer + * shareable specific flushes. + */ + + do_rvae_write(env, value, vae1_tlbmask(env), true); +} + +static void tlbi_aa64_rvae2_write(CPUARMState *env, + const ARMCPRegInfo *ri, + uint64_t value) +{ + /* + * Invalidate by VA range, EL2. + * Currently handles all of RVAE2 and RVALE2, + * since we don't support flush-for-specific-ASID-only or + * flush-last-level-only. + */ + + do_rvae_write(env, value, vae2_tlbmask(env), + tlb_force_broadcast(env)); + + +} + +static void tlbi_aa64_rvae2is_write(CPUARMState *env, + const ARMCPRegInfo *ri, + uint64_t value) +{ + /* + * Invalidate by VA range, Inner/Outer Shareable, EL2. + * Currently handles all of RVAE2IS, RVAE2OS, RVALE2IS and RVALE2OS, + * since we don't support flush-for-specific-ASID-only, + * flush-last-level-only or inner/outer shareable specific flushes. + */ + + do_rvae_write(env, value, vae2_tlbmask(env), true); + +} + +static void tlbi_aa64_rvae3_write(CPUARMState *env, + const ARMCPRegInfo *ri, + uint64_t value) +{ + /* + * Invalidate by VA range, EL3. + * Currently handles all of RVAE3 and RVALE3, + * since we don't support flush-for-specific-ASID-only or + * flush-last-level-only. + */ + + do_rvae_write(env, value, ARMMMUIdxBit_E3, tlb_force_broadcast(env)); +} + +static void tlbi_aa64_rvae3is_write(CPUARMState *env, + const ARMCPRegInfo *ri, + uint64_t value) +{ + /* + * Invalidate by VA range, EL3, Inner/Outer Shareable. + * Currently handles all of RVAE3IS, RVAE3OS, RVALE3IS and RVALE3OS, + * since we don't support flush-for-specific-ASID-only, + * flush-last-level-only or inner/outer specific flushes. + */ + + do_rvae_write(env, value, ARMMMUIdxBit_E3, true); +} + +static void tlbi_aa64_ripas2e1_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) +{ + do_rvae_write(env, value, ipas2e1_tlbmask(env, value), + tlb_force_broadcast(env)); +} + +static void tlbi_aa64_ripas2e1is_write(CPUARMState *env, + const ARMCPRegInfo *ri, + uint64_t value) +{ + do_rvae_write(env, value, ipas2e1_tlbmask(env, value), true); +} + +static const ARMCPRegInfo tlbirange_reginfo[] = { + { .name = "TLBI_RVAE1IS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 2, .opc2 = 1, + .access = PL1_W, .accessfn = access_ttlbis, .type = ARM_CP_NO_RAW, + .fgt = FGT_TLBIRVAE1IS, + .writefn = tlbi_aa64_rvae1is_write }, + { .name = "TLBI_RVAAE1IS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 2, .opc2 = 3, + .access = PL1_W, .accessfn = access_ttlbis, .type = ARM_CP_NO_RAW, + .fgt = FGT_TLBIRVAAE1IS, + .writefn = tlbi_aa64_rvae1is_write }, + { .name = "TLBI_RVALE1IS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 2, .opc2 = 5, + .access = PL1_W, .accessfn = access_ttlbis, .type = ARM_CP_NO_RAW, + .fgt = FGT_TLBIRVALE1IS, + .writefn = tlbi_aa64_rvae1is_write }, + { .name = "TLBI_RVAALE1IS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 2, .opc2 = 7, + .access = PL1_W, .accessfn = access_ttlbis, .type = ARM_CP_NO_RAW, + .fgt = FGT_TLBIRVAALE1IS, + .writefn = tlbi_aa64_rvae1is_write }, + { .name = "TLBI_RVAE1OS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 5, .opc2 = 1, + .access = PL1_W, .accessfn = access_ttlbos, .type = ARM_CP_NO_RAW, + .fgt = FGT_TLBIRVAE1OS, + .writefn = tlbi_aa64_rvae1is_write }, + { .name = "TLBI_RVAAE1OS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 5, .opc2 = 3, + .access = PL1_W, .accessfn = access_ttlbos, .type = ARM_CP_NO_RAW, + .fgt = FGT_TLBIRVAAE1OS, + .writefn = tlbi_aa64_rvae1is_write }, + { .name = "TLBI_RVALE1OS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 5, .opc2 = 5, + .access = PL1_W, .accessfn = access_ttlbos, .type = ARM_CP_NO_RAW, + .fgt = FGT_TLBIRVALE1OS, + .writefn = tlbi_aa64_rvae1is_write }, + { .name = "TLBI_RVAALE1OS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 5, .opc2 = 7, + .access = PL1_W, .accessfn = access_ttlbos, .type = ARM_CP_NO_RAW, + .fgt = FGT_TLBIRVAALE1OS, + .writefn = tlbi_aa64_rvae1is_write }, + { .name = "TLBI_RVAE1", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 6, .opc2 = 1, + .access = PL1_W, .accessfn = access_ttlb, .type = ARM_CP_NO_RAW, + .fgt = FGT_TLBIRVAE1, + .writefn = tlbi_aa64_rvae1_write }, + { .name = "TLBI_RVAAE1", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 6, .opc2 = 3, + .access = PL1_W, .accessfn = access_ttlb, .type = ARM_CP_NO_RAW, + .fgt = FGT_TLBIRVAAE1, + .writefn = tlbi_aa64_rvae1_write }, + { .name = "TLBI_RVALE1", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 6, .opc2 = 5, + .access = PL1_W, .accessfn = access_ttlb, .type = ARM_CP_NO_RAW, + .fgt = FGT_TLBIRVALE1, + .writefn = tlbi_aa64_rvae1_write }, + { .name = "TLBI_RVAALE1", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 6, .opc2 = 7, + .access = PL1_W, .accessfn = access_ttlb, .type = ARM_CP_NO_RAW, + .fgt = FGT_TLBIRVAALE1, + .writefn = tlbi_aa64_rvae1_write }, + { .name = "TLBI_RIPAS2E1IS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 0, .opc2 = 2, + .access = PL2_W, .type = ARM_CP_NO_RAW, + .writefn = tlbi_aa64_ripas2e1is_write }, + { .name = "TLBI_RIPAS2LE1IS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 0, .opc2 = 6, + .access = PL2_W, .type = ARM_CP_NO_RAW, + .writefn = tlbi_aa64_ripas2e1is_write }, + { .name = "TLBI_RVAE2IS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 2, .opc2 = 1, + .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF, + .writefn = tlbi_aa64_rvae2is_write }, + { .name = "TLBI_RVALE2IS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 2, .opc2 = 5, + .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF, + .writefn = tlbi_aa64_rvae2is_write }, + { .name = "TLBI_RIPAS2E1", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 4, .opc2 = 2, + .access = PL2_W, .type = ARM_CP_NO_RAW, + .writefn = tlbi_aa64_ripas2e1_write }, + { .name = "TLBI_RIPAS2LE1", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 4, .opc2 = 6, + .access = PL2_W, .type = ARM_CP_NO_RAW, + .writefn = tlbi_aa64_ripas2e1_write }, + { .name = "TLBI_RVAE2OS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 5, .opc2 = 1, + .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF, + .writefn = tlbi_aa64_rvae2is_write }, + { .name = "TLBI_RVALE2OS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 5, .opc2 = 5, + .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF, + .writefn = tlbi_aa64_rvae2is_write }, + { .name = "TLBI_RVAE2", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 6, .opc2 = 1, + .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF, + .writefn = tlbi_aa64_rvae2_write }, + { .name = "TLBI_RVALE2", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 6, .opc2 = 5, + .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF, + .writefn = tlbi_aa64_rvae2_write }, + { .name = "TLBI_RVAE3IS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 2, .opc2 = 1, + .access = PL3_W, .type = ARM_CP_NO_RAW, + .writefn = tlbi_aa64_rvae3is_write }, + { .name = "TLBI_RVALE3IS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 2, .opc2 = 5, + .access = PL3_W, .type = ARM_CP_NO_RAW, + .writefn = tlbi_aa64_rvae3is_write }, + { .name = "TLBI_RVAE3OS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 5, .opc2 = 1, + .access = PL3_W, .type = ARM_CP_NO_RAW, + .writefn = tlbi_aa64_rvae3is_write }, + { .name = "TLBI_RVALE3OS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 5, .opc2 = 5, + .access = PL3_W, .type = ARM_CP_NO_RAW, + .writefn = tlbi_aa64_rvae3is_write }, + { .name = "TLBI_RVAE3", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 6, .opc2 = 1, + .access = PL3_W, .type = ARM_CP_NO_RAW, + .writefn = tlbi_aa64_rvae3_write }, + { .name = "TLBI_RVALE3", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 6, .opc2 = 5, + .access = PL3_W, .type = ARM_CP_NO_RAW, + .writefn = tlbi_aa64_rvae3_write }, +}; +#endif + void define_tlb_insn_regs(ARMCPU *cpu) { CPUARMState *env = &cpu->env; @@ -XXX,XX +XXX,XX @@ void define_tlb_insn_regs(ARMCPU *cpu) if (arm_feature(env, ARM_FEATURE_EL3)) { define_arm_cp_regs(cpu, tlbi_el3_cp_reginfo); } +#ifdef TARGET_AARCH64 + if (cpu_isar_feature(aa64_tlbirange, cpu)) { + define_arm_cp_regs(cpu, tlbirange_reginfo); + } +#endif } -- 2.34.1 Move the TLBI OS insns across to tlb-insns.c. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241210160452.2427965-8-peter.maydell@linaro.org --- target/arm/helper.c | 80 -------------------------------------- target/arm/tcg/tlb-insns.c | 80 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 80 insertions(+), 80 deletions(-) diff --git a/target/arm/helper.c b/target/arm/helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -XXX,XX +XXX,XX @@ static const ARMCPRegInfo pauth_reginfo[] = { .fieldoffset = offsetof(CPUARMState, keys.apib.hi) }, }; -static const ARMCPRegInfo tlbios_reginfo[] = { - { .name = "TLBI_VMALLE1OS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 1, .opc2 = 0, - .access = PL1_W, .accessfn = access_ttlbos, .type = ARM_CP_NO_RAW, - .fgt = FGT_TLBIVMALLE1OS, - .writefn = tlbi_aa64_vmalle1is_write }, - { .name = "TLBI_VAE1OS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 1, .opc2 = 1, - .fgt = FGT_TLBIVAE1OS, - .access = PL1_W, .accessfn = access_ttlbos, .type = ARM_CP_NO_RAW, - .writefn = tlbi_aa64_vae1is_write }, - { .name = "TLBI_ASIDE1OS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 1, .opc2 = 2, - .access = PL1_W, .accessfn = access_ttlbos, .type = ARM_CP_NO_RAW, - .fgt = FGT_TLBIASIDE1OS, - .writefn = tlbi_aa64_vmalle1is_write }, - { .name = "TLBI_VAAE1OS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 1, .opc2 = 3, - .access = PL1_W, .accessfn = access_ttlbos, .type = ARM_CP_NO_RAW, - .fgt = FGT_TLBIVAAE1OS, - .writefn = tlbi_aa64_vae1is_write }, - { .name = "TLBI_VALE1OS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 1, .opc2 = 5, - .access = PL1_W, .accessfn = access_ttlbos, .type = ARM_CP_NO_RAW, - .fgt = FGT_TLBIVALE1OS, - .writefn = tlbi_aa64_vae1is_write }, - { .name = "TLBI_VAALE1OS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 1, .opc2 = 7, - .access = PL1_W, .accessfn = access_ttlbos, .type = ARM_CP_NO_RAW, - .fgt = FGT_TLBIVAALE1OS, - .writefn = tlbi_aa64_vae1is_write }, - { .name = "TLBI_ALLE2OS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 1, .opc2 = 0, - .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF, - .writefn = tlbi_aa64_alle2is_write }, - { .name = "TLBI_VAE2OS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 1, .opc2 = 1, - .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF, - .writefn = tlbi_aa64_vae2is_write }, - { .name = "TLBI_ALLE1OS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 1, .opc2 = 4, - .access = PL2_W, .type = ARM_CP_NO_RAW, - .writefn = tlbi_aa64_alle1is_write }, - { .name = "TLBI_VALE2OS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 1, .opc2 = 5, - .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF, - .writefn = tlbi_aa64_vae2is_write }, - { .name = "TLBI_VMALLS12E1OS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 1, .opc2 = 6, - .access = PL2_W, .type = ARM_CP_NO_RAW, - .writefn = tlbi_aa64_alle1is_write }, - { .name = "TLBI_IPAS2E1OS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 4, .opc2 = 0, - .access = PL2_W, .type = ARM_CP_NOP }, - { .name = "TLBI_RIPAS2E1OS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 4, .opc2 = 3, - .access = PL2_W, .type = ARM_CP_NOP }, - { .name = "TLBI_IPAS2LE1OS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 4, .opc2 = 4, - .access = PL2_W, .type = ARM_CP_NOP }, - { .name = "TLBI_RIPAS2LE1OS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 4, .opc2 = 7, - .access = PL2_W, .type = ARM_CP_NOP }, - { .name = "TLBI_ALLE3OS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 1, .opc2 = 0, - .access = PL3_W, .type = ARM_CP_NO_RAW, - .writefn = tlbi_aa64_alle3is_write }, - { .name = "TLBI_VAE3OS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 1, .opc2 = 1, - .access = PL3_W, .type = ARM_CP_NO_RAW, - .writefn = tlbi_aa64_vae3is_write }, - { .name = "TLBI_VALE3OS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 1, .opc2 = 5, - .access = PL3_W, .type = ARM_CP_NO_RAW, - .writefn = tlbi_aa64_vae3is_write }, -}; - static uint64_t rndr_readfn(CPUARMState *env, const ARMCPRegInfo *ri) { Error *err = NULL; @@ -XXX,XX +XXX,XX @@ void register_cp_regs_for_features(ARMCPU *cpu) if (cpu_isar_feature(aa64_rndr, cpu)) { define_arm_cp_regs(cpu, rndr_reginfo); } - if (cpu_isar_feature(aa64_tlbios, cpu)) { - define_arm_cp_regs(cpu, tlbios_reginfo); - } /* Data Cache clean instructions up to PoP */ if (cpu_isar_feature(aa64_dcpop, cpu)) { define_one_arm_cp_reg(cpu, dcpop_reg); diff --git a/target/arm/tcg/tlb-insns.c b/target/arm/tcg/tlb-insns.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/tlb-insns.c +++ b/target/arm/tcg/tlb-insns.c @@ -XXX,XX +XXX,XX @@ static const ARMCPRegInfo tlbirange_reginfo[] = { .access = PL3_W, .type = ARM_CP_NO_RAW, .writefn = tlbi_aa64_rvae3_write }, }; + +static const ARMCPRegInfo tlbios_reginfo[] = { + { .name = "TLBI_VMALLE1OS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 1, .opc2 = 0, + .access = PL1_W, .accessfn = access_ttlbos, .type = ARM_CP_NO_RAW, + .fgt = FGT_TLBIVMALLE1OS, + .writefn = tlbi_aa64_vmalle1is_write }, + { .name = "TLBI_VAE1OS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 1, .opc2 = 1, + .fgt = FGT_TLBIVAE1OS, + .access = PL1_W, .accessfn = access_ttlbos, .type = ARM_CP_NO_RAW, + .writefn = tlbi_aa64_vae1is_write }, + { .name = "TLBI_ASIDE1OS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 1, .opc2 = 2, + .access = PL1_W, .accessfn = access_ttlbos, .type = ARM_CP_NO_RAW, + .fgt = FGT_TLBIASIDE1OS, + .writefn = tlbi_aa64_vmalle1is_write }, + { .name = "TLBI_VAAE1OS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 1, .opc2 = 3, + .access = PL1_W, .accessfn = access_ttlbos, .type = ARM_CP_NO_RAW, + .fgt = FGT_TLBIVAAE1OS, + .writefn = tlbi_aa64_vae1is_write }, + { .name = "TLBI_VALE1OS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 1, .opc2 = 5, + .access = PL1_W, .accessfn = access_ttlbos, .type = ARM_CP_NO_RAW, + .fgt = FGT_TLBIVALE1OS, + .writefn = tlbi_aa64_vae1is_write }, + { .name = "TLBI_VAALE1OS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 1, .opc2 = 7, + .access = PL1_W, .accessfn = access_ttlbos, .type = ARM_CP_NO_RAW, + .fgt = FGT_TLBIVAALE1OS, + .writefn = tlbi_aa64_vae1is_write }, + { .name = "TLBI_ALLE2OS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 1, .opc2 = 0, + .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF, + .writefn = tlbi_aa64_alle2is_write }, + { .name = "TLBI_VAE2OS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 1, .opc2 = 1, + .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF, + .writefn = tlbi_aa64_vae2is_write }, + { .name = "TLBI_ALLE1OS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 1, .opc2 = 4, + .access = PL2_W, .type = ARM_CP_NO_RAW, + .writefn = tlbi_aa64_alle1is_write }, + { .name = "TLBI_VALE2OS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 1, .opc2 = 5, + .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF, + .writefn = tlbi_aa64_vae2is_write }, + { .name = "TLBI_VMALLS12E1OS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 1, .opc2 = 6, + .access = PL2_W, .type = ARM_CP_NO_RAW, + .writefn = tlbi_aa64_alle1is_write }, + { .name = "TLBI_IPAS2E1OS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 4, .opc2 = 0, + .access = PL2_W, .type = ARM_CP_NOP }, + { .name = "TLBI_RIPAS2E1OS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 4, .opc2 = 3, + .access = PL2_W, .type = ARM_CP_NOP }, + { .name = "TLBI_IPAS2LE1OS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 4, .opc2 = 4, + .access = PL2_W, .type = ARM_CP_NOP }, + { .name = "TLBI_RIPAS2LE1OS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 4, .opc2 = 7, + .access = PL2_W, .type = ARM_CP_NOP }, + { .name = "TLBI_ALLE3OS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 1, .opc2 = 0, + .access = PL3_W, .type = ARM_CP_NO_RAW, + .writefn = tlbi_aa64_alle3is_write }, + { .name = "TLBI_VAE3OS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 1, .opc2 = 1, + .access = PL3_W, .type = ARM_CP_NO_RAW, + .writefn = tlbi_aa64_vae3is_write }, + { .name = "TLBI_VALE3OS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 1, .opc2 = 5, + .access = PL3_W, .type = ARM_CP_NO_RAW, + .writefn = tlbi_aa64_vae3is_write }, +}; #endif void define_tlb_insn_regs(ARMCPU *cpu) @@ -XXX,XX +XXX,XX @@ void define_tlb_insn_regs(ARMCPU *cpu) if (cpu_isar_feature(aa64_tlbirange, cpu)) { define_arm_cp_regs(cpu, tlbirange_reginfo); } + if (cpu_isar_feature(aa64_tlbios, cpu)) { + define_arm_cp_regs(cpu, tlbios_reginfo); + } #endif } -- 2.34.1 The remaining functions that we temporarily made global are now used only from callsits in tlb-insns.c; move them across and make them file-local again. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241210160452.2427965-9-peter.maydell@linaro.org --- target/arm/cpregs.h | 34 ------ target/arm/helper.c | 220 ------------------------------------- target/arm/tcg/tlb-insns.c | 220 +++++++++++++++++++++++++++++++++++++ 3 files changed, 220 insertions(+), 254 deletions(-) diff --git a/target/arm/cpregs.h b/target/arm/cpregs.h index XXXXXXX..XXXXXXX 100644 --- a/target/arm/cpregs.h +++ b/target/arm/cpregs.h @@ -XXX,XX +XXX,XX @@ static inline bool arm_cpreg_traps_in_nv(const ARMCPRegInfo *ri) return ri->opc1 == 4 || ri->opc1 == 5; } -/* - * Temporary declarations of functions until the move to tlb_insn_helper.c - * is complete and we can make the functions static again - */ -CPAccessResult access_ttlb(CPUARMState *env, const ARMCPRegInfo *ri, - bool isread); -CPAccessResult access_ttlbis(CPUARMState *env, const ARMCPRegInfo *ri, - bool isread); -CPAccessResult access_ttlbos(CPUARMState *env, const ARMCPRegInfo *ri, - bool isread); -bool tlb_force_broadcast(CPUARMState *env); -int tlbbits_for_regime(CPUARMState *env, ARMMMUIdx mmu_idx, - uint64_t addr); -int vae1_tlbbits(CPUARMState *env, uint64_t addr); -int vae2_tlbbits(CPUARMState *env, uint64_t addr); -int vae1_tlbmask(CPUARMState *env); -int vae2_tlbmask(CPUARMState *env); -int ipas2e1_tlbmask(CPUARMState *env, int64_t value); -int e2_tlbmask(CPUARMState *env); -void tlbi_aa64_vmalle1is_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value); -void tlbi_aa64_alle1is_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value); -void tlbi_aa64_vae1is_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value); -void tlbi_aa64_alle2is_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value); -void tlbi_aa64_vae2is_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value); -void tlbi_aa64_vae3is_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value); -void tlbi_aa64_alle3is_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value); - #endif /* TARGET_ARM_CPREGS_H */ diff --git a/target/arm/helper.c b/target/arm/helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -XXX,XX +XXX,XX @@ static CPAccessResult access_tacr(CPUARMState *env, const ARMCPRegInfo *ri, return CP_ACCESS_OK; } -/* Check for traps from EL1 due to HCR_EL2.TTLB. */ -CPAccessResult access_ttlb(CPUARMState *env, const ARMCPRegInfo *ri, - bool isread) -{ - if (arm_current_el(env) == 1 && (arm_hcr_el2_eff(env) & HCR_TTLB)) { - return CP_ACCESS_TRAP_EL2; - } - return CP_ACCESS_OK; -} - -/* Check for traps from EL1 due to HCR_EL2.TTLB or TTLBIS. */ -CPAccessResult access_ttlbis(CPUARMState *env, const ARMCPRegInfo *ri, - bool isread) -{ - if (arm_current_el(env) == 1 && - (arm_hcr_el2_eff(env) & (HCR_TTLB | HCR_TTLBIS))) { - return CP_ACCESS_TRAP_EL2; - } - return CP_ACCESS_OK; -} - -#ifdef TARGET_AARCH64 -/* Check for traps from EL1 due to HCR_EL2.TTLB or TTLBOS. */ -CPAccessResult access_ttlbos(CPUARMState *env, const ARMCPRegInfo *ri, - bool isread) -{ - if (arm_current_el(env) == 1 && - (arm_hcr_el2_eff(env) & (HCR_TTLB | HCR_TTLBOS))) { - return CP_ACCESS_TRAP_EL2; - } - return CP_ACCESS_OK; -} -#endif - static void dacr_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value) { ARMCPU *cpu = env_archcpu(env); @@ -XXX,XX +XXX,XX @@ int alle1_tlbmask(CPUARMState *env) ARMMMUIdxBit_Stage2_S); } -/* - * Non-IS variants of TLB operations are upgraded to - * IS versions if we are at EL1 and HCR_EL2.FB is effectively set to - * force broadcast of these operations. - */ -bool tlb_force_broadcast(CPUARMState *env) -{ - return arm_current_el(env) == 1 && (arm_hcr_el2_eff(env) & HCR_FB); -} - static const ARMCPRegInfo cp_reginfo[] = { /* * Define the secure and non-secure FCSE identifier CP registers @@ -XXX,XX +XXX,XX @@ static CPAccessResult access_tocu(CPUARMState *env, const ARMCPRegInfo *ri, return do_cacheop_pou_access(env, HCR_TOCU | HCR_TPU); } -/* - * See: D4.7.2 TLB maintenance requirements and the TLB maintenance instructions - * Page D4-1736 (DDI0487A.b) - */ - -int vae1_tlbmask(CPUARMState *env) -{ - uint64_t hcr = arm_hcr_el2_eff(env); - uint16_t mask; - - assert(arm_feature(env, ARM_FEATURE_AARCH64)); - - if ((hcr & (HCR_E2H | HCR_TGE)) == (HCR_E2H | HCR_TGE)) { - mask = ARMMMUIdxBit_E20_2 | - ARMMMUIdxBit_E20_2_PAN | - ARMMMUIdxBit_E20_0; - } else { - /* This is AArch64 only, so we don't need to touch the EL30_x TLBs */ - mask = ARMMMUIdxBit_E10_1 | - ARMMMUIdxBit_E10_1_PAN | - ARMMMUIdxBit_E10_0; - } - return mask; -} - -int vae2_tlbmask(CPUARMState *env) -{ - uint64_t hcr = arm_hcr_el2_eff(env); - uint16_t mask; - - if (hcr & HCR_E2H) { - mask = ARMMMUIdxBit_E20_2 | - ARMMMUIdxBit_E20_2_PAN | - ARMMMUIdxBit_E20_0; - } else { - mask = ARMMMUIdxBit_E2; - } - return mask; -} - -/* Return 56 if TBI is enabled, 64 otherwise. */ -int tlbbits_for_regime(CPUARMState *env, ARMMMUIdx mmu_idx, - uint64_t addr) -{ - uint64_t tcr = regime_tcr(env, mmu_idx); - int tbi = aa64_va_parameter_tbi(tcr, mmu_idx); - int select = extract64(addr, 55, 1); - - return (tbi >> select) & 1 ? 56 : 64; -} - -int vae1_tlbbits(CPUARMState *env, uint64_t addr) -{ - uint64_t hcr = arm_hcr_el2_eff(env); - ARMMMUIdx mmu_idx; - - assert(arm_feature(env, ARM_FEATURE_AARCH64)); - - /* Only the regime of the mmu_idx below is significant. */ - if ((hcr & (HCR_E2H | HCR_TGE)) == (HCR_E2H | HCR_TGE)) { - mmu_idx = ARMMMUIdx_E20_0; - } else { - mmu_idx = ARMMMUIdx_E10_0; - } - - return tlbbits_for_regime(env, mmu_idx, addr); -} - -int vae2_tlbbits(CPUARMState *env, uint64_t addr) -{ - uint64_t hcr = arm_hcr_el2_eff(env); - ARMMMUIdx mmu_idx; - - /* - * Only the regime of the mmu_idx below is significant. - * Regime EL2&0 has two ranges with separate TBI configuration, while EL2 - * only has one. - */ - if (hcr & HCR_E2H) { - mmu_idx = ARMMMUIdx_E20_2; - } else { - mmu_idx = ARMMMUIdx_E2; - } - - return tlbbits_for_regime(env, mmu_idx, addr); -} - -void tlbi_aa64_vmalle1is_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) -{ - CPUState *cs = env_cpu(env); - int mask = vae1_tlbmask(env); - - tlb_flush_by_mmuidx_all_cpus_synced(cs, mask); -} - -int e2_tlbmask(CPUARMState *env) -{ - return (ARMMMUIdxBit_E20_0 | - ARMMMUIdxBit_E20_2 | - ARMMMUIdxBit_E20_2_PAN | - ARMMMUIdxBit_E2); -} - -void tlbi_aa64_alle1is_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) -{ - CPUState *cs = env_cpu(env); - int mask = alle1_tlbmask(env); - - tlb_flush_by_mmuidx_all_cpus_synced(cs, mask); -} - -void tlbi_aa64_alle2is_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) -{ - CPUState *cs = env_cpu(env); - int mask = e2_tlbmask(env); - - tlb_flush_by_mmuidx_all_cpus_synced(cs, mask); -} - -void tlbi_aa64_alle3is_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) -{ - CPUState *cs = env_cpu(env); - - tlb_flush_by_mmuidx_all_cpus_synced(cs, ARMMMUIdxBit_E3); -} - -void tlbi_aa64_vae1is_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) -{ - CPUState *cs = env_cpu(env); - int mask = vae1_tlbmask(env); - uint64_t pageaddr = sextract64(value << 12, 0, 56); - int bits = vae1_tlbbits(env, pageaddr); - - tlb_flush_page_bits_by_mmuidx_all_cpus_synced(cs, pageaddr, mask, bits); -} - -void tlbi_aa64_vae2is_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) -{ - CPUState *cs = env_cpu(env); - int mask = vae2_tlbmask(env); - uint64_t pageaddr = sextract64(value << 12, 0, 56); - int bits = vae2_tlbbits(env, pageaddr); - - tlb_flush_page_bits_by_mmuidx_all_cpus_synced(cs, pageaddr, mask, bits); -} - -void tlbi_aa64_vae3is_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) -{ - CPUState *cs = env_cpu(env); - uint64_t pageaddr = sextract64(value << 12, 0, 56); - int bits = tlbbits_for_regime(env, ARMMMUIdx_E3, pageaddr); - - tlb_flush_page_bits_by_mmuidx_all_cpus_synced(cs, pageaddr, - ARMMMUIdxBit_E3, bits); -} - -int ipas2e1_tlbmask(CPUARMState *env, int64_t value) -{ - /* - * The MSB of value is the NS field, which only applies if SEL2 - * is implemented and SCR_EL3.NS is not set (i.e. in secure mode). - */ - return (value >= 0 - && cpu_isar_feature(aa64_sel2, env_archcpu(env)) - && arm_is_secure_below_el3(env) - ? ARMMMUIdxBit_Stage2_S - : ARMMMUIdxBit_Stage2); -} - static CPAccessResult aa64_zva_access(CPUARMState *env, const ARMCPRegInfo *ri, bool isread) { diff --git a/target/arm/tcg/tlb-insns.c b/target/arm/tcg/tlb-insns.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/tlb-insns.c +++ b/target/arm/tcg/tlb-insns.c @@ -XXX,XX +XXX,XX @@ #include "cpu-features.h" #include "cpregs.h" +/* Check for traps from EL1 due to HCR_EL2.TTLB. */ +static CPAccessResult access_ttlb(CPUARMState *env, const ARMCPRegInfo *ri, + bool isread) +{ + if (arm_current_el(env) == 1 && (arm_hcr_el2_eff(env) & HCR_TTLB)) { + return CP_ACCESS_TRAP_EL2; + } + return CP_ACCESS_OK; +} + +/* Check for traps from EL1 due to HCR_EL2.TTLB or TTLBIS. */ +static CPAccessResult access_ttlbis(CPUARMState *env, const ARMCPRegInfo *ri, + bool isread) +{ + if (arm_current_el(env) == 1 && + (arm_hcr_el2_eff(env) & (HCR_TTLB | HCR_TTLBIS))) { + return CP_ACCESS_TRAP_EL2; + } + return CP_ACCESS_OK; +} + +#ifdef TARGET_AARCH64 +/* Check for traps from EL1 due to HCR_EL2.TTLB or TTLBOS. */ +static CPAccessResult access_ttlbos(CPUARMState *env, const ARMCPRegInfo *ri, + bool isread) +{ + if (arm_current_el(env) == 1 && + (arm_hcr_el2_eff(env) & (HCR_TTLB | HCR_TTLBOS))) { + return CP_ACCESS_TRAP_EL2; + } + return CP_ACCESS_OK; +} +#endif + /* IS variants of TLB operations must affect all cores */ static void tlbiall_is_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value) @@ -XXX,XX +XXX,XX @@ static void tlbimvaa_is_write(CPUARMState *env, const ARMCPRegInfo *ri, tlb_flush_page_all_cpus_synced(cs, value & TARGET_PAGE_MASK); } +/* + * Non-IS variants of TLB operations are upgraded to + * IS versions if we are at EL1 and HCR_EL2.FB is effectively set to + * force broadcast of these operations. + */ +static bool tlb_force_broadcast(CPUARMState *env) +{ + return arm_current_el(env) == 1 && (arm_hcr_el2_eff(env) & HCR_FB); +} + static void tlbiall_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value) { @@ -XXX,XX +XXX,XX @@ static void tlbiall_hyp_is_write(CPUARMState *env, const ARMCPRegInfo *ri, tlb_flush_by_mmuidx_all_cpus_synced(cs, ARMMMUIdxBit_E2); } +/* + * See: D4.7.2 TLB maintenance requirements and the TLB maintenance instructions + * Page D4-1736 (DDI0487A.b) + */ + +static int vae1_tlbmask(CPUARMState *env) +{ + uint64_t hcr = arm_hcr_el2_eff(env); + uint16_t mask; + + assert(arm_feature(env, ARM_FEATURE_AARCH64)); + + if ((hcr & (HCR_E2H | HCR_TGE)) == (HCR_E2H | HCR_TGE)) { + mask = ARMMMUIdxBit_E20_2 | + ARMMMUIdxBit_E20_2_PAN | + ARMMMUIdxBit_E20_0; + } else { + /* This is AArch64 only, so we don't need to touch the EL30_x TLBs */ + mask = ARMMMUIdxBit_E10_1 | + ARMMMUIdxBit_E10_1_PAN | + ARMMMUIdxBit_E10_0; + } + return mask; +} + +static int vae2_tlbmask(CPUARMState *env) +{ + uint64_t hcr = arm_hcr_el2_eff(env); + uint16_t mask; + + if (hcr & HCR_E2H) { + mask = ARMMMUIdxBit_E20_2 | + ARMMMUIdxBit_E20_2_PAN | + ARMMMUIdxBit_E20_0; + } else { + mask = ARMMMUIdxBit_E2; + } + return mask; +} + +/* Return 56 if TBI is enabled, 64 otherwise. */ +static int tlbbits_for_regime(CPUARMState *env, ARMMMUIdx mmu_idx, + uint64_t addr) +{ + uint64_t tcr = regime_tcr(env, mmu_idx); + int tbi = aa64_va_parameter_tbi(tcr, mmu_idx); + int select = extract64(addr, 55, 1); + + return (tbi >> select) & 1 ? 56 : 64; +} + +static int vae1_tlbbits(CPUARMState *env, uint64_t addr) +{ + uint64_t hcr = arm_hcr_el2_eff(env); + ARMMMUIdx mmu_idx; + + assert(arm_feature(env, ARM_FEATURE_AARCH64)); + + /* Only the regime of the mmu_idx below is significant. */ + if ((hcr & (HCR_E2H | HCR_TGE)) == (HCR_E2H | HCR_TGE)) { + mmu_idx = ARMMMUIdx_E20_0; + } else { + mmu_idx = ARMMMUIdx_E10_0; + } + + return tlbbits_for_regime(env, mmu_idx, addr); +} + +static int vae2_tlbbits(CPUARMState *env, uint64_t addr) +{ + uint64_t hcr = arm_hcr_el2_eff(env); + ARMMMUIdx mmu_idx; + + /* + * Only the regime of the mmu_idx below is significant. + * Regime EL2&0 has two ranges with separate TBI configuration, while EL2 + * only has one. + */ + if (hcr & HCR_E2H) { + mmu_idx = ARMMMUIdx_E20_2; + } else { + mmu_idx = ARMMMUIdx_E2; + } + + return tlbbits_for_regime(env, mmu_idx, addr); +} + +static void tlbi_aa64_vmalle1is_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) +{ + CPUState *cs = env_cpu(env); + int mask = vae1_tlbmask(env); + + tlb_flush_by_mmuidx_all_cpus_synced(cs, mask); +} + static void tlbi_aa64_vmalle1_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value) { @@ -XXX,XX +XXX,XX @@ static void tlbi_aa64_vmalle1_write(CPUARMState *env, const ARMCPRegInfo *ri, } } +static int e2_tlbmask(CPUARMState *env) +{ + return (ARMMMUIdxBit_E20_0 | + ARMMMUIdxBit_E20_2 | + ARMMMUIdxBit_E20_2_PAN | + ARMMMUIdxBit_E2); +} + static void tlbi_aa64_alle1_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value) { @@ -XXX,XX +XXX,XX @@ static void tlbi_aa64_alle3_write(CPUARMState *env, const ARMCPRegInfo *ri, tlb_flush_by_mmuidx(cs, ARMMMUIdxBit_E3); } +static void tlbi_aa64_alle1is_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) +{ + CPUState *cs = env_cpu(env); + int mask = alle1_tlbmask(env); + + tlb_flush_by_mmuidx_all_cpus_synced(cs, mask); +} + +static void tlbi_aa64_alle2is_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) +{ + CPUState *cs = env_cpu(env); + int mask = e2_tlbmask(env); + + tlb_flush_by_mmuidx_all_cpus_synced(cs, mask); +} + +static void tlbi_aa64_alle3is_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) +{ + CPUState *cs = env_cpu(env); + + tlb_flush_by_mmuidx_all_cpus_synced(cs, ARMMMUIdxBit_E3); +} + static void tlbi_aa64_vae2_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value) { @@ -XXX,XX +XXX,XX @@ static void tlbi_aa64_vae3_write(CPUARMState *env, const ARMCPRegInfo *ri, tlb_flush_page_by_mmuidx(cs, pageaddr, ARMMMUIdxBit_E3); } +static void tlbi_aa64_vae1is_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) +{ + CPUState *cs = env_cpu(env); + int mask = vae1_tlbmask(env); + uint64_t pageaddr = sextract64(value << 12, 0, 56); + int bits = vae1_tlbbits(env, pageaddr); + + tlb_flush_page_bits_by_mmuidx_all_cpus_synced(cs, pageaddr, mask, bits); +} + static void tlbi_aa64_vae1_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value) { @@ -XXX,XX +XXX,XX @@ static void tlbi_aa64_vae1_write(CPUARMState *env, const ARMCPRegInfo *ri, } } +static void tlbi_aa64_vae2is_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) +{ + CPUState *cs = env_cpu(env); + int mask = vae2_tlbmask(env); + uint64_t pageaddr = sextract64(value << 12, 0, 56); + int bits = vae2_tlbbits(env, pageaddr); + + tlb_flush_page_bits_by_mmuidx_all_cpus_synced(cs, pageaddr, mask, bits); +} + +static void tlbi_aa64_vae3is_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) +{ + CPUState *cs = env_cpu(env); + uint64_t pageaddr = sextract64(value << 12, 0, 56); + int bits = tlbbits_for_regime(env, ARMMMUIdx_E3, pageaddr); + + tlb_flush_page_bits_by_mmuidx_all_cpus_synced(cs, pageaddr, + ARMMMUIdxBit_E3, bits); +} + +static int ipas2e1_tlbmask(CPUARMState *env, int64_t value) +{ + /* + * The MSB of value is the NS field, which only applies if SEL2 + * is implemented and SCR_EL3.NS is not set (i.e. in secure mode). + */ + return (value >= 0 + && cpu_isar_feature(aa64_sel2, env_archcpu(env)) + && arm_is_secure_below_el3(env) + ? ARMMMUIdxBit_Stage2_S + : ARMMMUIdxBit_Stage2); +} + static void tlbi_aa64_ipas2e1_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value) { -- 2.34.1 Move the FEAT_RME specific TLB insns across to tlb-insns.c. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241210160452.2427965-10-peter.maydell@linaro.org --- target/arm/helper.c | 38 -------------------------------- target/arm/tcg/tlb-insns.c | 45 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 45 insertions(+), 38 deletions(-) diff --git a/target/arm/helper.c b/target/arm/helper.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -XXX,XX +XXX,XX @@ static const ARMCPRegInfo sme_reginfo[] = { .type = ARM_CP_CONST, .resetvalue = 0 }, }; -static void tlbi_aa64_paall_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) -{ - CPUState *cs = env_cpu(env); - - tlb_flush(cs); -} - static void gpccr_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value) { @@ -XXX,XX +XXX,XX @@ static void gpccr_reset(CPUARMState *env, const ARMCPRegInfo *ri) env_archcpu(env)->reset_l0gptsz); } -static void tlbi_aa64_paallos_write(CPUARMState *env, const ARMCPRegInfo *ri, - uint64_t value) -{ - CPUState *cs = env_cpu(env); - - tlb_flush_all_cpus_synced(cs); -} - static const ARMCPRegInfo rme_reginfo[] = { { .name = "GPCCR_EL3", .state = ARM_CP_STATE_AA64, .opc0 = 3, .opc1 = 6, .crn = 2, .crm = 1, .opc2 = 6, @@ -XXX,XX +XXX,XX @@ static const ARMCPRegInfo rme_reginfo[] = { { .name = "MFAR_EL3", .state = ARM_CP_STATE_AA64, .opc0 = 3, .opc1 = 6, .crn = 6, .crm = 0, .opc2 = 5, .access = PL3_RW, .fieldoffset = offsetof(CPUARMState, cp15.mfar_el3) }, - { .name = "TLBI_PAALL", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 7, .opc2 = 4, - .access = PL3_W, .type = ARM_CP_NO_RAW, - .writefn = tlbi_aa64_paall_write }, - { .name = "TLBI_PAALLOS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 1, .opc2 = 4, - .access = PL3_W, .type = ARM_CP_NO_RAW, - .writefn = tlbi_aa64_paallos_write }, - /* - * QEMU does not have a way to invalidate by physical address, thus - * invalidating a range of physical addresses is accomplished by - * flushing all tlb entries in the outer shareable domain, - * just like PAALLOS. - */ - { .name = "TLBI_RPALOS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 4, .opc2 = 7, - .access = PL3_W, .type = ARM_CP_NO_RAW, - .writefn = tlbi_aa64_paallos_write }, - { .name = "TLBI_RPAOS", .state = ARM_CP_STATE_AA64, - .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 4, .opc2 = 3, - .access = PL3_W, .type = ARM_CP_NO_RAW, - .writefn = tlbi_aa64_paallos_write }, { .name = "DC_CIPAPA", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 6, .crn = 7, .crm = 14, .opc2 = 1, .access = PL3_W, .type = ARM_CP_NOP }, diff --git a/target/arm/tcg/tlb-insns.c b/target/arm/tcg/tlb-insns.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/tlb-insns.c +++ b/target/arm/tcg/tlb-insns.c @@ -XXX,XX +XXX,XX @@ static const ARMCPRegInfo tlbios_reginfo[] = { .access = PL3_W, .type = ARM_CP_NO_RAW, .writefn = tlbi_aa64_vae3is_write }, }; + +static void tlbi_aa64_paall_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) +{ + CPUState *cs = env_cpu(env); + + tlb_flush(cs); +} + +static void tlbi_aa64_paallos_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) +{ + CPUState *cs = env_cpu(env); + + tlb_flush_all_cpus_synced(cs); +} + +static const ARMCPRegInfo tlbi_rme_reginfo[] = { + { .name = "TLBI_PAALL", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 7, .opc2 = 4, + .access = PL3_W, .type = ARM_CP_NO_RAW, + .writefn = tlbi_aa64_paall_write }, + { .name = "TLBI_PAALLOS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 1, .opc2 = 4, + .access = PL3_W, .type = ARM_CP_NO_RAW, + .writefn = tlbi_aa64_paallos_write }, + /* + * QEMU does not have a way to invalidate by physical address, thus + * invalidating a range of physical addresses is accomplished by + * flushing all tlb entries in the outer shareable domain, + * just like PAALLOS. + */ + { .name = "TLBI_RPALOS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 4, .opc2 = 7, + .access = PL3_W, .type = ARM_CP_NO_RAW, + .writefn = tlbi_aa64_paallos_write }, + { .name = "TLBI_RPAOS", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 4, .opc2 = 3, + .access = PL3_W, .type = ARM_CP_NO_RAW, + .writefn = tlbi_aa64_paallos_write }, +}; + #endif void define_tlb_insn_regs(ARMCPU *cpu) @@ -XXX,XX +XXX,XX @@ void define_tlb_insn_regs(ARMCPU *cpu) if (cpu_isar_feature(aa64_tlbios, cpu)) { define_arm_cp_regs(cpu, tlbios_reginfo); } + if (cpu_isar_feature(aa64_rme, cpu)) { + define_arm_cp_regs(cpu, tlbi_rme_reginfo); + } #endif } -- 2.34.1 We currently register the tlbi_el2_cp_reginfo[] TLBI insns if EL2 is implemented, or if EL3 and v8 is implemented. This is a copy of the logic used for el2_cp_reginfo[], but for the specific case of the TLBI insns we can simplify it. This is because we do not need the "if EL2 does not exist but EL3 does then EL2 registers should exist and be RAZ/WI" handling here: all our cpregs are for instructions, which UNDEF when EL3 exists and EL2 does not. Simplify the condition down to just "if EL2 exists". This is not a behaviour change because: * for AArch64 insns we marked them with ARM_CP_EL3_NO_EL2_UNDEF, which meant that define_arm_cp_regs() would ignore them if EL2 wasn't present * for AArch32 insns, the .access = PL2_W meant that if EL2 was not present the only way to get at them was from AArch32 EL3; but we have no CPUs which have ARM_FEATURE_V8 but start in AArch32 Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20241210160452.2427965-11-peter.maydell@linaro.org --- target/arm/tcg/tlb-insns.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/target/arm/tcg/tlb-insns.c b/target/arm/tcg/tlb-insns.c index XXXXXXX..XXXXXXX 100644 --- a/target/arm/tcg/tlb-insns.c +++ b/target/arm/tcg/tlb-insns.c @@ -XXX,XX +XXX,XX @@ void define_tlb_insn_regs(ARMCPU *cpu) * ops (i.e. matching the condition for el2_cp_reginfo[] in * helper.c), but we will be able to simplify this later. */ - if (arm_feature(env, ARM_FEATURE_EL2) - || (arm_feature(env, ARM_FEATURE_EL3) - && arm_feature(env, ARM_FEATURE_V8))) { + if (arm_feature(env, ARM_FEATURE_EL2)) { define_arm_cp_regs(cpu, tlbi_el2_cp_reginfo); } if (arm_feature(env, ARM_FEATURE_EL3)) { -- 2.34.1