From nobody Fri Dec 19 20:53:16 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1530075933227100.48462384036065; Tue, 26 Jun 2018 22:05:33 -0700 (PDT) Received: from localhost ([::1]:56690 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fY2e1-0004GQ-D9 for importer@patchew.org; Wed, 27 Jun 2018 01:05:29 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:32804) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fY29n-0004dr-8f for qemu-devel@nongnu.org; Wed, 27 Jun 2018 00:34:18 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fY29l-00010b-El for qemu-devel@nongnu.org; Wed, 27 Jun 2018 00:34:15 -0400 Received: from mail-pl0-x241.google.com ([2607:f8b0:400e:c01::241]:42165) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fY29l-0000zp-72 for qemu-devel@nongnu.org; Wed, 27 Jun 2018 00:34:13 -0400 Received: by mail-pl0-x241.google.com with SMTP id w17-v6so411779pll.9 for ; Tue, 26 Jun 2018 21:34:13 -0700 (PDT) Received: from cloudburst.twiddle.net (97-126-112-211.tukw.qwest.net. [97.126.112.211]) by smtp.gmail.com with ESMTPSA id p20-v6sm4577638pff.90.2018.06.26.21.34.10 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 26 Jun 2018 21:34:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=jAT3MbivOAuh2cXOphZuTzwQ/j70NHuupQD49/hoOCQ=; b=Bvdz6coePgMV9aY0yCHnfbjN+wjvXlOGI3HX+feoE4GZPLoGgdCgmDHC2S5bVlZmJN e/f65YmpnBwtuAhIH0iNM3N5FCAFNjup4Aol4lDKdADuJyXCAvS1ZlGo8LrJwC1UZ4Al mYNPkwsBgMKoZbB439NY62H2T9EIKKt9Ibg4Q= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=jAT3MbivOAuh2cXOphZuTzwQ/j70NHuupQD49/hoOCQ=; b=s68GPBNB/AuleelkGgS3o1vr7tbLXD84bEbNUCM/wvsMOtC9JTDSSZUPqyRcivsYVT ix6M3ykGqzC4ZvtNv4qIzapPRelO94zAeQ0tfwd/q4Sops3bRvlGSUNkTGi5B/mWJWmR mu4oWombBs+tevsWu9xcB1kiz1ntgVKvj/TeL5iGw+DTPl+FfQ/7dCiUH9YGbxqV3mgw nK9M5qd47e6Ir2Zq+otDeAlfL8AXZgKw8n5phHJTkwHKP65xxUclejIlaR2HF871WHlw rhUj2dY3iqJF3fQ2h8myPDsTrOWuKbuZxDkuf+h2GBeJvdzjdKHoTvb69iBwfmYyxL5q F1cA== X-Gm-Message-State: APt69E0VsUbVXJsd3mYzhpYfISSX3y+tZwz2Q3uiIlKp3vgKxMHum8Ff H8hnN0clgx6oiJM1HqTZR58tfNrlQyU= X-Google-Smtp-Source: ADUXVKKXw0TYBplr5d1PZYnwk68imk0K8U0/3vTbR63jb73SLYbRKvvfALHqX4d4RXS0Xst7amx82A== X-Received: by 2002:a17:902:3c5:: with SMTP id d63-v6mr4518297pld.163.1530074051944; Tue, 26 Jun 2018 21:34:11 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Date: Tue, 26 Jun 2018 21:33:22 -0700 Message-Id: <20180627043328.11531-30-richard.henderson@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180627043328.11531-1-richard.henderson@linaro.org> References: <20180627043328.11531-1-richard.henderson@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400e:c01::241 Subject: [Qemu-devel] [PATCH v6 29/35] target/arm: Implement SVE fp complex multiply add X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZohoMail: RDKM_2 RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Reviewed-by: Peter Maydell Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 4 + target/arm/sve_helper.c | 162 +++++++++++++++++++++++++++++++++++++ target/arm/translate-sve.c | 37 +++++++++ target/arm/sve.decode | 4 + 4 files changed, 207 insertions(+) diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 0bd9fe2f28..023952a9a4 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -1115,6 +1115,10 @@ DEF_HELPER_FLAGS_3(sve_fnmls_zpzzz_h, TCG_CALL_NO_RW= G, void, env, ptr, i32) DEF_HELPER_FLAGS_3(sve_fnmls_zpzzz_s, TCG_CALL_NO_RWG, void, env, ptr, i32) DEF_HELPER_FLAGS_3(sve_fnmls_zpzzz_d, TCG_CALL_NO_RWG, void, env, ptr, i32) =20 +DEF_HELPER_FLAGS_3(sve_fcmla_zpzzz_h, TCG_CALL_NO_RWG, void, env, ptr, i32) +DEF_HELPER_FLAGS_3(sve_fcmla_zpzzz_s, TCG_CALL_NO_RWG, void, env, ptr, i32) +DEF_HELPER_FLAGS_3(sve_fcmla_zpzzz_d, TCG_CALL_NO_RWG, void, env, ptr, i32) + DEF_HELPER_FLAGS_5(sve_ftmad_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr,= i32) DEF_HELPER_FLAGS_5(sve_ftmad_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr,= i32) DEF_HELPER_FLAGS_5(sve_ftmad_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr,= i32) diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index bdb7565779..790cbacd14 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -3757,6 +3757,168 @@ void HELPER(sve_fcadd_d)(void *vd, void *vn, void *= vm, void *vg, } while (i !=3D 0); } =20 +/* + * FP Complex Multiply + */ + +QEMU_BUILD_BUG_ON(SIMD_DATA_SHIFT + 22 > 32); + +void HELPER(sve_fcmla_zpzzz_h)(CPUARMState *env, void *vg, uint32_t desc) +{ + intptr_t j, i =3D simd_oprsz(desc); + unsigned rd =3D extract32(desc, SIMD_DATA_SHIFT, 5); + unsigned rn =3D extract32(desc, SIMD_DATA_SHIFT + 5, 5); + unsigned rm =3D extract32(desc, SIMD_DATA_SHIFT + 10, 5); + unsigned ra =3D extract32(desc, SIMD_DATA_SHIFT + 15, 5); + unsigned rot =3D extract32(desc, SIMD_DATA_SHIFT + 20, 2); + bool flip =3D rot & 1; + float16 neg_imag, neg_real; + void *vd =3D &env->vfp.zregs[rd]; + void *vn =3D &env->vfp.zregs[rn]; + void *vm =3D &env->vfp.zregs[rm]; + void *va =3D &env->vfp.zregs[ra]; + uint64_t *g =3D vg; + + neg_imag =3D float16_set_sign(0, (rot & 2) !=3D 0); + neg_real =3D float16_set_sign(0, rot =3D=3D 1 || rot =3D=3D 2); + + do { + uint64_t pg =3D g[(i - 1) >> 6]; + do { + float16 e1, e2, e3, e4, nr, ni, mr, mi, d; + + /* I holds the real index; J holds the imag index. */ + j =3D i - sizeof(float16); + i -=3D 2 * sizeof(float16); + + nr =3D *(float16 *)(vn + H1_2(i)); + ni =3D *(float16 *)(vn + H1_2(j)); + mr =3D *(float16 *)(vm + H1_2(i)); + mi =3D *(float16 *)(vm + H1_2(j)); + + e2 =3D (flip ? ni : nr); + e1 =3D (flip ? mi : mr) ^ neg_real; + e4 =3D e2; + e3 =3D (flip ? mr : mi) ^ neg_imag; + + if (likely((pg >> (i & 63)) & 1)) { + d =3D *(float16 *)(va + H1_2(i)); + d =3D float16_muladd(e2, e1, d, 0, &env->vfp.fp_status_f16= ); + *(float16 *)(vd + H1_2(i)) =3D d; + } + if (likely((pg >> (j & 63)) & 1)) { + d =3D *(float16 *)(va + H1_2(j)); + d =3D float16_muladd(e4, e3, d, 0, &env->vfp.fp_status_f16= ); + *(float16 *)(vd + H1_2(j)) =3D d; + } + } while (i & 63); + } while (i !=3D 0); +} + +void HELPER(sve_fcmla_zpzzz_s)(CPUARMState *env, void *vg, uint32_t desc) +{ + intptr_t j, i =3D simd_oprsz(desc); + unsigned rd =3D extract32(desc, SIMD_DATA_SHIFT, 5); + unsigned rn =3D extract32(desc, SIMD_DATA_SHIFT + 5, 5); + unsigned rm =3D extract32(desc, SIMD_DATA_SHIFT + 10, 5); + unsigned ra =3D extract32(desc, SIMD_DATA_SHIFT + 15, 5); + unsigned rot =3D extract32(desc, SIMD_DATA_SHIFT + 20, 2); + bool flip =3D rot & 1; + float32 neg_imag, neg_real; + void *vd =3D &env->vfp.zregs[rd]; + void *vn =3D &env->vfp.zregs[rn]; + void *vm =3D &env->vfp.zregs[rm]; + void *va =3D &env->vfp.zregs[ra]; + uint64_t *g =3D vg; + + neg_imag =3D float32_set_sign(0, (rot & 2) !=3D 0); + neg_real =3D float32_set_sign(0, rot =3D=3D 1 || rot =3D=3D 2); + + do { + uint64_t pg =3D g[(i - 1) >> 6]; + do { + float32 e1, e2, e3, e4, nr, ni, mr, mi, d; + + /* I holds the real index; J holds the imag index. */ + j =3D i - sizeof(float32); + i -=3D 2 * sizeof(float32); + + nr =3D *(float32 *)(vn + H1_2(i)); + ni =3D *(float32 *)(vn + H1_2(j)); + mr =3D *(float32 *)(vm + H1_2(i)); + mi =3D *(float32 *)(vm + H1_2(j)); + + e2 =3D (flip ? ni : nr); + e1 =3D (flip ? mi : mr) ^ neg_real; + e4 =3D e2; + e3 =3D (flip ? mr : mi) ^ neg_imag; + + if (likely((pg >> (i & 63)) & 1)) { + d =3D *(float32 *)(va + H1_2(i)); + d =3D float32_muladd(e2, e1, d, 0, &env->vfp.fp_status); + *(float32 *)(vd + H1_2(i)) =3D d; + } + if (likely((pg >> (j & 63)) & 1)) { + d =3D *(float32 *)(va + H1_2(j)); + d =3D float32_muladd(e4, e3, d, 0, &env->vfp.fp_status); + *(float32 *)(vd + H1_2(j)) =3D d; + } + } while (i & 63); + } while (i !=3D 0); +} + +void HELPER(sve_fcmla_zpzzz_d)(CPUARMState *env, void *vg, uint32_t desc) +{ + intptr_t j, i =3D simd_oprsz(desc); + unsigned rd =3D extract32(desc, SIMD_DATA_SHIFT, 5); + unsigned rn =3D extract32(desc, SIMD_DATA_SHIFT + 5, 5); + unsigned rm =3D extract32(desc, SIMD_DATA_SHIFT + 10, 5); + unsigned ra =3D extract32(desc, SIMD_DATA_SHIFT + 15, 5); + unsigned rot =3D extract32(desc, SIMD_DATA_SHIFT + 20, 2); + bool flip =3D rot & 1; + float64 neg_imag, neg_real; + void *vd =3D &env->vfp.zregs[rd]; + void *vn =3D &env->vfp.zregs[rn]; + void *vm =3D &env->vfp.zregs[rm]; + void *va =3D &env->vfp.zregs[ra]; + uint64_t *g =3D vg; + + neg_imag =3D float64_set_sign(0, (rot & 2) !=3D 0); + neg_real =3D float64_set_sign(0, rot =3D=3D 1 || rot =3D=3D 2); + + do { + uint64_t pg =3D g[(i - 1) >> 6]; + do { + float64 e1, e2, e3, e4, nr, ni, mr, mi, d; + + /* I holds the real index; J holds the imag index. */ + j =3D i - sizeof(float64); + i -=3D 2 * sizeof(float64); + + nr =3D *(float64 *)(vn + H1_2(i)); + ni =3D *(float64 *)(vn + H1_2(j)); + mr =3D *(float64 *)(vm + H1_2(i)); + mi =3D *(float64 *)(vm + H1_2(j)); + + e2 =3D (flip ? ni : nr); + e1 =3D (flip ? mi : mr) ^ neg_real; + e4 =3D e2; + e3 =3D (flip ? mr : mi) ^ neg_imag; + + if (likely((pg >> (i & 63)) & 1)) { + d =3D *(float64 *)(va + H1_2(i)); + d =3D float64_muladd(e2, e1, d, 0, &env->vfp.fp_status); + *(float64 *)(vd + H1_2(i)) =3D d; + } + if (likely((pg >> (j & 63)) & 1)) { + d =3D *(float64 *)(va + H1_2(j)); + d =3D float64_muladd(e4, e3, d, 0, &env->vfp.fp_status); + *(float64 *)(vd + H1_2(j)) =3D d; + } + } while (i & 63); + } while (i !=3D 0); +} + /* * Load contiguous data, protected by a governing predicate. */ diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index b1764f099b..7ce3222158 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -3968,6 +3968,43 @@ DO_FMLA(FNMLS_zpzzz, fnmls_zpzzz) =20 #undef DO_FMLA =20 +static bool trans_FCMLA_zpzzz(DisasContext *s, + arg_FCMLA_zpzzz *a, uint32_t insn) +{ + static gen_helper_sve_fmla * const fns[3] =3D { + gen_helper_sve_fcmla_zpzzz_h, + gen_helper_sve_fcmla_zpzzz_s, + gen_helper_sve_fcmla_zpzzz_d, + }; + + if (a->esz =3D=3D 0) { + return false; + } + if (sve_access_check(s)) { + unsigned vsz =3D vec_full_reg_size(s); + unsigned desc; + TCGv_i32 t_desc; + TCGv_ptr pg =3D tcg_temp_new_ptr(); + + /* We would need 7 operands to pass these arguments "properly". + * So we encode all the register numbers into the descriptor. + */ + desc =3D deposit32(a->rd, 5, 5, a->rn); + desc =3D deposit32(desc, 10, 5, a->rm); + desc =3D deposit32(desc, 15, 5, a->ra); + desc =3D deposit32(desc, 20, 2, a->rot); + desc =3D sextract32(desc, 0, 22); + desc =3D simd_desc(vsz, vsz, desc); + + t_desc =3D tcg_const_i32(desc); + tcg_gen_addi_ptr(pg, cpu_env, pred_full_reg_offset(s, a->pg)); + fns[a->esz - 1](cpu_env, pg, t_desc); + tcg_temp_free_i32(t_desc); + tcg_temp_free_ptr(pg); + } + return true; +} + /* *** SVE Floating Point Unary Operations Prediated Group */ diff --git a/target/arm/sve.decode b/target/arm/sve.decode index e5f8f43254..e342cfdf14 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -729,6 +729,10 @@ MUL_zzi 00100101 .. 110 000 110 ........ .....= @rdn_i8s FCADD 01100100 esz:2 00000 rot:1 100 pg:3 rm:5 rd:5 \ rn=3D%reg_movprfx =20 +# SVE floating-point complex multiply-add (predicated) +FCMLA_zpzzz 01100100 esz:2 0 rm:5 0 rot:2 pg:3 rn:5 rd:5 \ + ra=3D%reg_movprfx + ### SVE FP Multiply-Add Indexed Group =20 # SVE floating-point multiply-add (indexed) --=20 2.17.1