From nobody Wed Apr 16 04:27:01 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=linaro.org ARC-Seal: i=1; a=rsa-sha256; t=1560429132; cv=none; d=zoho.com; s=zohoarc; b=C0d9uvxzHdoViDRnPIaEycyG+y5xOtVbBAZ7Zwh/rumC71yvgfwelu2FhIsL1trrOYVdPadK4sQplau2kriyXudM2nvaMznxQYfbhc6OBCjVEmhZkFBmkw9UXq8yEgAHHYK0ySzFGfXX/L/GE4u9TaeVfgHebkHFf67T7xUg4fI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1560429132; h=Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=Dn7KxRgCvtYAfGaacJQP6WpAQ39HiMNEc3qaxDs7q9A=; b=OacwmQcNmJ663/0cTuPL7y2qcdXoTPsirr5l6FnpBo5273DoYjPHoFQ/aLxRIlGEviZFVo5OTHgu5W1ps1gezczPUVBTS1ZwEnha26LP1dv+HgGqV31Mz/oiOXvftaq/ypcp00dz+q7++31IIAbXVKt1F3ebBV/m0JG7umLYuas= ARC-Authentication-Results: i=1; mx.zoho.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1560429132163193.9303664421078; Thu, 13 Jun 2019 05:32:12 -0700 (PDT) Received: from localhost ([::1]:39402 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hbOtm-0007Yj-V0 for importer@patchew.org; Thu, 13 Jun 2019 08:32:10 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:58721) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hbOcr-0007tJ-Ih for qemu-devel@nongnu.org; Thu, 13 Jun 2019 08:14:44 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hbOco-0003sX-Ll for qemu-devel@nongnu.org; Thu, 13 Jun 2019 08:14:41 -0400 Received: from mail-wr1-x42a.google.com ([2a00:1450:4864:20::42a]:43657) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1hbOco-0003rf-A1 for qemu-devel@nongnu.org; Thu, 13 Jun 2019 08:14:38 -0400 Received: by mail-wr1-x42a.google.com with SMTP id p13so10425245wru.10 for ; Thu, 13 Jun 2019 05:14:38 -0700 (PDT) Received: from orth.archaic.org.uk (orth.archaic.org.uk. [81.2.115.148]) by smtp.gmail.com with ESMTPSA id y6sm2010576wrp.12.2019.06.13.05.14.35 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 13 Jun 2019 05:14:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=Dn7KxRgCvtYAfGaacJQP6WpAQ39HiMNEc3qaxDs7q9A=; b=w9USfq8SpZVDbWWHMlnVyLn2X7L7GT73hbIwUn2nCPulqc0YFhk+aaqCioCgMIGyal A8xqPKOsiom7UJN3VIIQ6BW66dF5ZC1z3WkugACyVpOLinv8gHteulBR824yIP0niOOc TDdcSs7o2+icBqC0SaztB90wd560tBpXz8jE74whsimDkhot/4nd3c6y+vp9LiLWl5Fa 4MrZ6TOcue/E+Yq3n5LTEzX6Esxl9/r3QDYDnl+LlgOeZJS6CtouWzUV+pgPhKfRgTV4 cd2y+/UxtYA9zRTA/2vkgPDuHYSaAaQLjK/KlZWFFngegBS4yVtNJCA5MFw20FLJXG01 4Uyg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Dn7KxRgCvtYAfGaacJQP6WpAQ39HiMNEc3qaxDs7q9A=; b=c0eXY6ChSooqdSXNGMUywJTXD3ci0bFRyk6AQ+Yk6gUQ0u+2ySdVd8wflueizduizj B2Y3/U6kLc8yl0EoZNnYSNCZ2Ose4LxUbFSyCwpmfViuY2JFJ2raY3IF7VKxQsE2GWRa NoDodtj61sGV56Aqn4rUTMJ2qLzWdTb5nllFzBDpucCTbyJrL3rP5pqmoTrFpGyNACy2 DUKQKSRnlb0WHl8CQPzT4lEhKH8hXvFM7SgsHSG34lOvcNeqk2RqQ193/zh6A2zBgDh7 hWCbP5FmKeXIGg7J5DRJy3RPeeM46KlhYWVU8Y/GNR2hWsk+CcLETOyE+TzXjnXZDMsv 2oaA== X-Gm-Message-State: APjAAAW6yWWHBgY9sNetvRUBpPJWL22AFHtBJLvU7RbRTAA/92mX0SYe P6BKR2b0AdsPATiDXpMnfqPS6+8Pev723Q== X-Google-Smtp-Source: APXvYqzDWL5qgquu7ZAMjBbA4/DeF8mvuR7qQ+AvZfTdNrPq5jq6P8DNU5N00/U+YZtjtqa6jjq1Jw== X-Received: by 2002:adf:f181:: with SMTP id h1mr1580777wro.86.1560428076670; Thu, 13 Jun 2019 05:14:36 -0700 (PDT) From: Peter Maydell To: qemu-devel@nongnu.org Date: Thu, 13 Jun 2019 13:13:46 +0100 Message-Id: <20190613121433.5246-2-peter.maydell@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190613121433.5246-1-peter.maydell@linaro.org> References: <20190613121433.5246-1-peter.maydell@linaro.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:4864:20::42a Subject: [Qemu-devel] [PULL 01/48] target/arm: Vectorize USHL and SSHL X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" From: Richard Henderson These instructions shift left or right depending on the sign of the input, and 7 bits are significant to the shift. This requires several masks and selects in addition to the actual shifts to form the complete answer. That said, the operation is still a small improvement even for two 64-bit elements -- 13 vector operations instead of 2 * 7 integer operations. Signed-off-by: Richard Henderson Message-id: 20190603232209.20704-1-richard.henderson@linaro.org Reviewed-by: Peter Maydell Signed-off-by: Peter Maydell --- target/arm/helper.h | 11 +- target/arm/translate.h | 6 + target/arm/neon_helper.c | 33 ---- target/arm/translate-a64.c | 18 +-- target/arm/translate.c | 300 +++++++++++++++++++++++++++++++++++-- target/arm/vec_helper.c | 88 +++++++++++ 6 files changed, 390 insertions(+), 66 deletions(-) diff --git a/target/arm/helper.h b/target/arm/helper.h index 132aa1682e8..1ab0fe597a9 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -297,14 +297,8 @@ DEF_HELPER_2(neon_abd_s16, i32, i32, i32) DEF_HELPER_2(neon_abd_u32, i32, i32, i32) DEF_HELPER_2(neon_abd_s32, i32, i32, i32) =20 -DEF_HELPER_2(neon_shl_u8, i32, i32, i32) -DEF_HELPER_2(neon_shl_s8, i32, i32, i32) DEF_HELPER_2(neon_shl_u16, i32, i32, i32) DEF_HELPER_2(neon_shl_s16, i32, i32, i32) -DEF_HELPER_2(neon_shl_u32, i32, i32, i32) -DEF_HELPER_2(neon_shl_s32, i32, i32, i32) -DEF_HELPER_2(neon_shl_u64, i64, i64, i64) -DEF_HELPER_2(neon_shl_s64, i64, i64, i64) DEF_HELPER_2(neon_rshl_u8, i32, i32, i32) DEF_HELPER_2(neon_rshl_s8, i32, i32, i32) DEF_HELPER_2(neon_rshl_u16, i32, i32, i32) @@ -691,6 +685,11 @@ DEF_HELPER_FLAGS_2(frint64_s, TCG_CALL_NO_RWG, f32, f3= 2, ptr) DEF_HELPER_FLAGS_2(frint32_d, TCG_CALL_NO_RWG, f64, f64, ptr) DEF_HELPER_FLAGS_2(frint64_d, TCG_CALL_NO_RWG, f64, f64, ptr) =20 +DEF_HELPER_FLAGS_4(gvec_sshl_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_sshl_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_ushl_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_ushl_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + #ifdef TARGET_AARCH64 #include "helper-a64.h" #include "helper-sve.h" diff --git a/target/arm/translate.h b/target/arm/translate.h index c2348def0d1..f357b767cb9 100644 --- a/target/arm/translate.h +++ b/target/arm/translate.h @@ -244,6 +244,8 @@ extern const GVecGen3 bif_op; extern const GVecGen3 mla_op[4]; extern const GVecGen3 mls_op[4]; extern const GVecGen3 cmtst_op[4]; +extern const GVecGen3 sshl_op[4]; +extern const GVecGen3 ushl_op[4]; extern const GVecGen2i ssra_op[4]; extern const GVecGen2i usra_op[4]; extern const GVecGen2i sri_op[4]; @@ -253,6 +255,10 @@ extern const GVecGen4 sqadd_op[4]; extern const GVecGen4 uqsub_op[4]; extern const GVecGen4 sqsub_op[4]; void gen_cmtst_i64(TCGv_i64 d, TCGv_i64 a, TCGv_i64 b); +void gen_ushl_i32(TCGv_i32 d, TCGv_i32 a, TCGv_i32 b); +void gen_sshl_i32(TCGv_i32 d, TCGv_i32 a, TCGv_i32 b); +void gen_ushl_i64(TCGv_i64 d, TCGv_i64 a, TCGv_i64 b); +void gen_sshl_i64(TCGv_i64 d, TCGv_i64 a, TCGv_i64 b); =20 /* * Forward to the isar_feature_* tests given a DisasContext pointer. diff --git a/target/arm/neon_helper.c b/target/arm/neon_helper.c index 42590567236..c581ffb7d3c 100644 --- a/target/arm/neon_helper.c +++ b/target/arm/neon_helper.c @@ -615,24 +615,9 @@ NEON_VOP(abd_u32, neon_u32, 1) } else { \ dest =3D src1 << tmp; \ }} while (0) -NEON_VOP(shl_u8, neon_u8, 4) NEON_VOP(shl_u16, neon_u16, 2) -NEON_VOP(shl_u32, neon_u32, 1) #undef NEON_FN =20 -uint64_t HELPER(neon_shl_u64)(uint64_t val, uint64_t shiftop) -{ - int8_t shift =3D (int8_t)shiftop; - if (shift >=3D 64 || shift <=3D -64) { - val =3D 0; - } else if (shift < 0) { - val >>=3D -shift; - } else { - val <<=3D shift; - } - return val; -} - #define NEON_FN(dest, src1, src2) do { \ int8_t tmp; \ tmp =3D (int8_t)src2; \ @@ -645,27 +630,9 @@ uint64_t HELPER(neon_shl_u64)(uint64_t val, uint64_t s= hiftop) } else { \ dest =3D src1 << tmp; \ }} while (0) -NEON_VOP(shl_s8, neon_s8, 4) NEON_VOP(shl_s16, neon_s16, 2) -NEON_VOP(shl_s32, neon_s32, 1) #undef NEON_FN =20 -uint64_t HELPER(neon_shl_s64)(uint64_t valop, uint64_t shiftop) -{ - int8_t shift =3D (int8_t)shiftop; - int64_t val =3D valop; - if (shift >=3D 64) { - val =3D 0; - } else if (shift <=3D -64) { - val >>=3D 63; - } else if (shift < 0) { - val >>=3D -shift; - } else { - val <<=3D shift; - } - return val; -} - #define NEON_FN(dest, src1, src2) do { \ int8_t tmp; \ tmp =3D (int8_t)src2; \ diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c index 8a3bf204d3d..f53cd288158 100644 --- a/target/arm/translate-a64.c +++ b/target/arm/translate-a64.c @@ -8847,9 +8847,9 @@ static void handle_3same_64(DisasContext *s, int opco= de, bool u, break; case 0x8: /* SSHL, USHL */ if (u) { - gen_helper_neon_shl_u64(tcg_rd, tcg_rn, tcg_rm); + gen_ushl_i64(tcg_rd, tcg_rn, tcg_rm); } else { - gen_helper_neon_shl_s64(tcg_rd, tcg_rn, tcg_rm); + gen_sshl_i64(tcg_rd, tcg_rn, tcg_rm); } break; case 0x9: /* SQSHL, UQSHL */ @@ -11244,6 +11244,10 @@ static void disas_simd_3same_int(DisasContext *s, = uint32_t insn) is_q ? 16 : 8, vec_full_reg_size(s), (u ? uqsub_op : sqsub_op) + size); return; + case 0x08: /* SSHL, USHL */ + gen_gvec_op3(s, is_q, rd, rn, rm, + u ? &ushl_op[size] : &sshl_op[size]); + return; case 0x0c: /* SMAX, UMAX */ if (u) { gen_gvec_fn3(s, is_q, rd, rn, rm, tcg_gen_gvec_umax, size); @@ -11359,16 +11363,6 @@ static void disas_simd_3same_int(DisasContext *s, = uint32_t insn) genfn =3D fns[size][u]; break; } - case 0x8: /* SSHL, USHL */ - { - static NeonGenTwoOpFn * const fns[3][2] =3D { - { gen_helper_neon_shl_s8, gen_helper_neon_shl_u8 }, - { gen_helper_neon_shl_s16, gen_helper_neon_shl_u16 }, - { gen_helper_neon_shl_s32, gen_helper_neon_shl_u32 }, - }; - genfn =3D fns[size][u]; - break; - } case 0x9: /* SQSHL, UQSHL */ { static NeonGenTwoOpEnvFn * const fns[3][2] =3D { diff --git a/target/arm/translate.c b/target/arm/translate.c index d25e19ef113..096e338bf06 100644 --- a/target/arm/translate.c +++ b/target/arm/translate.c @@ -5374,13 +5374,13 @@ static inline void gen_neon_shift_narrow(int size, = TCGv_i32 var, TCGv_i32 shift, if (u) { switch (size) { case 1: gen_helper_neon_shl_u16(var, var, shift); break; - case 2: gen_helper_neon_shl_u32(var, var, shift); break; + case 2: gen_ushl_i32(var, var, shift); break; default: abort(); } } else { switch (size) { case 1: gen_helper_neon_shl_s16(var, var, shift); break; - case 2: gen_helper_neon_shl_s32(var, var, shift); break; + case 2: gen_sshl_i32(var, var, shift); break; default: abort(); } } @@ -6259,6 +6259,282 @@ const GVecGen3 cmtst_op[4] =3D { .vece =3D MO_64 }, }; =20 +void gen_ushl_i32(TCGv_i32 d, TCGv_i32 a, TCGv_i32 b) +{ + TCGv_i32 lval =3D tcg_temp_new_i32(); + TCGv_i32 rval =3D tcg_temp_new_i32(); + TCGv_i32 lsh =3D tcg_temp_new_i32(); + TCGv_i32 rsh =3D tcg_temp_new_i32(); + TCGv_i32 zero =3D tcg_const_i32(0); + TCGv_i32 max =3D tcg_const_i32(32); + + /* + * Rely on the TCG guarantee that out of range shifts produce + * unspecified results, not undefined behaviour (i.e. no trap). + * Discard out-of-range results after the fact. + */ + tcg_gen_ext8s_i32(lsh, b); + tcg_gen_neg_i32(rsh, lsh); + tcg_gen_shl_i32(lval, a, lsh); + tcg_gen_shr_i32(rval, a, rsh); + tcg_gen_movcond_i32(TCG_COND_LTU, d, lsh, max, lval, zero); + tcg_gen_movcond_i32(TCG_COND_LTU, d, rsh, max, rval, d); + + tcg_temp_free_i32(lval); + tcg_temp_free_i32(rval); + tcg_temp_free_i32(lsh); + tcg_temp_free_i32(rsh); + tcg_temp_free_i32(zero); + tcg_temp_free_i32(max); +} + +void gen_ushl_i64(TCGv_i64 d, TCGv_i64 a, TCGv_i64 b) +{ + TCGv_i64 lval =3D tcg_temp_new_i64(); + TCGv_i64 rval =3D tcg_temp_new_i64(); + TCGv_i64 lsh =3D tcg_temp_new_i64(); + TCGv_i64 rsh =3D tcg_temp_new_i64(); + TCGv_i64 zero =3D tcg_const_i64(0); + TCGv_i64 max =3D tcg_const_i64(64); + + /* + * Rely on the TCG guarantee that out of range shifts produce + * unspecified results, not undefined behaviour (i.e. no trap). + * Discard out-of-range results after the fact. + */ + tcg_gen_ext8s_i64(lsh, b); + tcg_gen_neg_i64(rsh, lsh); + tcg_gen_shl_i64(lval, a, lsh); + tcg_gen_shr_i64(rval, a, rsh); + tcg_gen_movcond_i64(TCG_COND_LTU, d, lsh, max, lval, zero); + tcg_gen_movcond_i64(TCG_COND_LTU, d, rsh, max, rval, d); + + tcg_temp_free_i64(lval); + tcg_temp_free_i64(rval); + tcg_temp_free_i64(lsh); + tcg_temp_free_i64(rsh); + tcg_temp_free_i64(zero); + tcg_temp_free_i64(max); +} + +static void gen_ushl_vec(unsigned vece, TCGv_vec d, TCGv_vec a, TCGv_vec b) +{ + TCGv_vec lval =3D tcg_temp_new_vec_matching(d); + TCGv_vec rval =3D tcg_temp_new_vec_matching(d); + TCGv_vec lsh =3D tcg_temp_new_vec_matching(d); + TCGv_vec rsh =3D tcg_temp_new_vec_matching(d); + TCGv_vec msk, max; + + /* + * Rely on the TCG guarantee that out of range shifts produce + * unspecified results, not undefined behaviour (i.e. no trap). + * Discard out-of-range results after the fact. + */ + tcg_gen_neg_vec(vece, rsh, b); + if (vece =3D=3D MO_8) { + tcg_gen_mov_vec(lsh, b); + } else { + msk =3D tcg_temp_new_vec_matching(d); + tcg_gen_dupi_vec(vece, msk, 0xff); + tcg_gen_and_vec(vece, lsh, b, msk); + tcg_gen_and_vec(vece, rsh, rsh, msk); + tcg_temp_free_vec(msk); + } + + /* + * Perform possibly out of range shifts, trusting that the operation + * does not trap. Discard unused results after the fact. + */ + tcg_gen_shlv_vec(vece, lval, a, lsh); + tcg_gen_shrv_vec(vece, rval, a, rsh); + + max =3D tcg_temp_new_vec_matching(d); + tcg_gen_dupi_vec(vece, max, 8 << vece); + + /* + * The choice of LT (signed) and GEU (unsigned) are biased toward + * the instructions of the x86_64 host. For MO_8, the whole byte + * is significant so we must use an unsigned compare; otherwise we + * have already masked to a byte and so a signed compare works. + * Other tcg hosts have a full set of comparisons and do not care. + */ + if (vece =3D=3D MO_8) { + tcg_gen_cmp_vec(TCG_COND_GEU, vece, lsh, lsh, max); + tcg_gen_cmp_vec(TCG_COND_GEU, vece, rsh, rsh, max); + tcg_gen_andc_vec(vece, lval, lval, lsh); + tcg_gen_andc_vec(vece, rval, rval, rsh); + } else { + tcg_gen_cmp_vec(TCG_COND_LT, vece, lsh, lsh, max); + tcg_gen_cmp_vec(TCG_COND_LT, vece, rsh, rsh, max); + tcg_gen_and_vec(vece, lval, lval, lsh); + tcg_gen_and_vec(vece, rval, rval, rsh); + } + tcg_gen_or_vec(vece, d, lval, rval); + + tcg_temp_free_vec(max); + tcg_temp_free_vec(lval); + tcg_temp_free_vec(rval); + tcg_temp_free_vec(lsh); + tcg_temp_free_vec(rsh); +} + +static const TCGOpcode ushl_list[] =3D { + INDEX_op_neg_vec, INDEX_op_shlv_vec, + INDEX_op_shrv_vec, INDEX_op_cmp_vec, 0 +}; + +const GVecGen3 ushl_op[4] =3D { + { .fniv =3D gen_ushl_vec, + .fno =3D gen_helper_gvec_ushl_b, + .opt_opc =3D ushl_list, + .vece =3D MO_8 }, + { .fniv =3D gen_ushl_vec, + .fno =3D gen_helper_gvec_ushl_h, + .opt_opc =3D ushl_list, + .vece =3D MO_16 }, + { .fni4 =3D gen_ushl_i32, + .fniv =3D gen_ushl_vec, + .opt_opc =3D ushl_list, + .vece =3D MO_32 }, + { .fni8 =3D gen_ushl_i64, + .fniv =3D gen_ushl_vec, + .opt_opc =3D ushl_list, + .vece =3D MO_64 }, +}; + +void gen_sshl_i32(TCGv_i32 d, TCGv_i32 a, TCGv_i32 b) +{ + TCGv_i32 lval =3D tcg_temp_new_i32(); + TCGv_i32 rval =3D tcg_temp_new_i32(); + TCGv_i32 lsh =3D tcg_temp_new_i32(); + TCGv_i32 rsh =3D tcg_temp_new_i32(); + TCGv_i32 zero =3D tcg_const_i32(0); + TCGv_i32 max =3D tcg_const_i32(31); + + /* + * Rely on the TCG guarantee that out of range shifts produce + * unspecified results, not undefined behaviour (i.e. no trap). + * Discard out-of-range results after the fact. + */ + tcg_gen_ext8s_i32(lsh, b); + tcg_gen_neg_i32(rsh, lsh); + tcg_gen_shl_i32(lval, a, lsh); + tcg_gen_umin_i32(rsh, rsh, max); + tcg_gen_sar_i32(rval, a, rsh); + tcg_gen_movcond_i32(TCG_COND_LEU, lval, lsh, max, lval, zero); + tcg_gen_movcond_i32(TCG_COND_LT, d, lsh, zero, rval, lval); + + tcg_temp_free_i32(lval); + tcg_temp_free_i32(rval); + tcg_temp_free_i32(lsh); + tcg_temp_free_i32(rsh); + tcg_temp_free_i32(zero); + tcg_temp_free_i32(max); +} + +void gen_sshl_i64(TCGv_i64 d, TCGv_i64 a, TCGv_i64 b) +{ + TCGv_i64 lval =3D tcg_temp_new_i64(); + TCGv_i64 rval =3D tcg_temp_new_i64(); + TCGv_i64 lsh =3D tcg_temp_new_i64(); + TCGv_i64 rsh =3D tcg_temp_new_i64(); + TCGv_i64 zero =3D tcg_const_i64(0); + TCGv_i64 max =3D tcg_const_i64(63); + + /* + * Rely on the TCG guarantee that out of range shifts produce + * unspecified results, not undefined behaviour (i.e. no trap). + * Discard out-of-range results after the fact. + */ + tcg_gen_ext8s_i64(lsh, b); + tcg_gen_neg_i64(rsh, lsh); + tcg_gen_shl_i64(lval, a, lsh); + tcg_gen_umin_i64(rsh, rsh, max); + tcg_gen_sar_i64(rval, a, rsh); + tcg_gen_movcond_i64(TCG_COND_LEU, lval, lsh, max, lval, zero); + tcg_gen_movcond_i64(TCG_COND_LT, d, lsh, zero, rval, lval); + + tcg_temp_free_i64(lval); + tcg_temp_free_i64(rval); + tcg_temp_free_i64(lsh); + tcg_temp_free_i64(rsh); + tcg_temp_free_i64(zero); + tcg_temp_free_i64(max); +} + +static void gen_sshl_vec(unsigned vece, TCGv_vec d, TCGv_vec a, TCGv_vec b) +{ + TCGv_vec lval =3D tcg_temp_new_vec_matching(d); + TCGv_vec rval =3D tcg_temp_new_vec_matching(d); + TCGv_vec lsh =3D tcg_temp_new_vec_matching(d); + TCGv_vec rsh =3D tcg_temp_new_vec_matching(d); + TCGv_vec tmp =3D tcg_temp_new_vec_matching(d); + + /* + * Rely on the TCG guarantee that out of range shifts produce + * unspecified results, not undefined behaviour (i.e. no trap). + * Discard out-of-range results after the fact. + */ + tcg_gen_neg_vec(vece, rsh, b); + if (vece =3D=3D MO_8) { + tcg_gen_mov_vec(lsh, b); + } else { + tcg_gen_dupi_vec(vece, tmp, 0xff); + tcg_gen_and_vec(vece, lsh, b, tmp); + tcg_gen_and_vec(vece, rsh, rsh, tmp); + } + + /* Bound rsh so out of bound right shift gets -1. */ + tcg_gen_dupi_vec(vece, tmp, (8 << vece) - 1); + tcg_gen_umin_vec(vece, rsh, rsh, tmp); + tcg_gen_cmp_vec(TCG_COND_GT, vece, tmp, lsh, tmp); + + tcg_gen_shlv_vec(vece, lval, a, lsh); + tcg_gen_sarv_vec(vece, rval, a, rsh); + + /* Select in-bound left shift. */ + tcg_gen_andc_vec(vece, lval, lval, tmp); + + /* Select between left and right shift. */ + if (vece =3D=3D MO_8) { + tcg_gen_dupi_vec(vece, tmp, 0); + tcg_gen_cmpsel_vec(TCG_COND_LT, vece, d, lsh, tmp, rval, lval); + } else { + tcg_gen_dupi_vec(vece, tmp, 0x80); + tcg_gen_cmpsel_vec(TCG_COND_LT, vece, d, lsh, tmp, lval, rval); + } + + tcg_temp_free_vec(lval); + tcg_temp_free_vec(rval); + tcg_temp_free_vec(lsh); + tcg_temp_free_vec(rsh); + tcg_temp_free_vec(tmp); +} + +static const TCGOpcode sshl_list[] =3D { + INDEX_op_neg_vec, INDEX_op_umin_vec, INDEX_op_shlv_vec, + INDEX_op_sarv_vec, INDEX_op_cmp_vec, INDEX_op_cmpsel_vec, 0 +}; + +const GVecGen3 sshl_op[4] =3D { + { .fniv =3D gen_sshl_vec, + .fno =3D gen_helper_gvec_sshl_b, + .opt_opc =3D sshl_list, + .vece =3D MO_8 }, + { .fniv =3D gen_sshl_vec, + .fno =3D gen_helper_gvec_sshl_h, + .opt_opc =3D sshl_list, + .vece =3D MO_16 }, + { .fni4 =3D gen_sshl_i32, + .fniv =3D gen_sshl_vec, + .opt_opc =3D sshl_list, + .vece =3D MO_32 }, + { .fni8 =3D gen_sshl_i64, + .fniv =3D gen_sshl_vec, + .opt_opc =3D sshl_list, + .vece =3D MO_64 }, +}; + static void gen_uqadd_vec(unsigned vece, TCGv_vec t, TCGv_vec sat, TCGv_vec a, TCGv_vec b) { @@ -6662,6 +6938,11 @@ static int disas_neon_data_insn(DisasContext *s, uin= t32_t insn) vec_size, vec_size); } return 0; + + case NEON_3R_VSHL: + tcg_gen_gvec_3(rd_ofs, rn_ofs, rm_ofs, vec_size, vec_size, + u ? &ushl_op[size] : &sshl_op[size]); + return 0; } =20 if (size =3D=3D 3) { @@ -6670,13 +6951,6 @@ static int disas_neon_data_insn(DisasContext *s, uin= t32_t insn) neon_load_reg64(cpu_V0, rn + pass); neon_load_reg64(cpu_V1, rm + pass); switch (op) { - case NEON_3R_VSHL: - if (u) { - gen_helper_neon_shl_u64(cpu_V0, cpu_V1, cpu_V0); - } else { - gen_helper_neon_shl_s64(cpu_V0, cpu_V1, cpu_V0); - } - break; case NEON_3R_VQSHL: if (u) { gen_helper_neon_qshl_u64(cpu_V0, cpu_env, @@ -6711,7 +6985,6 @@ static int disas_neon_data_insn(DisasContext *s, uint= 32_t insn) } pairwise =3D 0; switch (op) { - case NEON_3R_VSHL: case NEON_3R_VQSHL: case NEON_3R_VRSHL: case NEON_3R_VQRSHL: @@ -6791,9 +7064,6 @@ static int disas_neon_data_insn(DisasContext *s, uint= 32_t insn) case NEON_3R_VHSUB: GEN_NEON_INTEGER_OP(hsub); break; - case NEON_3R_VSHL: - GEN_NEON_INTEGER_OP(shl); - break; case NEON_3R_VQSHL: GEN_NEON_INTEGER_OP_ENV(qshl); break; @@ -7202,9 +7472,9 @@ static int disas_neon_data_insn(DisasContext *s, uint= 32_t insn) } } else { if (input_unsigned) { - gen_helper_neon_shl_u64(cpu_V0, in, tmp64); + gen_ushl_i64(cpu_V0, in, tmp64); } else { - gen_helper_neon_shl_s64(cpu_V0, in, tmp64); + gen_sshl_i64(cpu_V0, in, tmp64); } } tmp =3D tcg_temp_new_i32(); diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index dedef62403a..fcb36639036 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -1046,3 +1046,91 @@ void HELPER(gvec_fmlal_idx_a64)(void *vd, void *vn, = void *vm, do_fmlal_idx(vd, vn, vm, &env->vfp.fp_status, desc, get_flush_inputs_to_zero(&env->vfp.fp_status_f16)); } + +void HELPER(gvec_sshl_b)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, opr_sz =3D simd_oprsz(desc); + int8_t *d =3D vd, *n =3D vn, *m =3D vm; + + for (i =3D 0; i < opr_sz; ++i) { + int8_t mm =3D m[i]; + int8_t nn =3D n[i]; + int8_t res =3D 0; + if (mm >=3D 0) { + if (mm < 8) { + res =3D nn << mm; + } + } else { + res =3D nn >> (mm > -8 ? -mm : 7); + } + d[i] =3D res; + } + clear_tail(d, opr_sz, simd_maxsz(desc)); +} + +void HELPER(gvec_sshl_h)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, opr_sz =3D simd_oprsz(desc); + int16_t *d =3D vd, *n =3D vn, *m =3D vm; + + for (i =3D 0; i < opr_sz / 2; ++i) { + int8_t mm =3D m[i]; /* only 8 bits of shift are significant */ + int16_t nn =3D n[i]; + int16_t res =3D 0; + if (mm >=3D 0) { + if (mm < 16) { + res =3D nn << mm; + } + } else { + res =3D nn >> (mm > -16 ? -mm : 15); + } + d[i] =3D res; + } + clear_tail(d, opr_sz, simd_maxsz(desc)); +} + +void HELPER(gvec_ushl_b)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, opr_sz =3D simd_oprsz(desc); + uint8_t *d =3D vd, *n =3D vn, *m =3D vm; + + for (i =3D 0; i < opr_sz; ++i) { + int8_t mm =3D m[i]; + uint8_t nn =3D n[i]; + uint8_t res =3D 0; + if (mm >=3D 0) { + if (mm < 8) { + res =3D nn << mm; + } + } else { + if (mm > -8) { + res =3D nn >> -mm; + } + } + d[i] =3D res; + } + clear_tail(d, opr_sz, simd_maxsz(desc)); +} + +void HELPER(gvec_ushl_h)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, opr_sz =3D simd_oprsz(desc); + uint16_t *d =3D vd, *n =3D vn, *m =3D vm; + + for (i =3D 0; i < opr_sz / 2; ++i) { + int8_t mm =3D m[i]; /* only 8 bits of shift are significant */ + uint16_t nn =3D n[i]; + uint16_t res =3D 0; + if (mm >=3D 0) { + if (mm < 16) { + res =3D nn << mm; + } + } else { + if (mm > -16) { + res =3D nn >> -mm; + } + } + d[i] =3D res; + } + clear_tail(d, opr_sz, simd_maxsz(desc)); +} --=20 2.20.1