From nobody Tue Jul 28 15:16:25 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=linaro.org ARC-Seal: i=1; a=rsa-sha256; t=1777211755; cv=none; d=zohomail.com; s=zohoarc; b=nEfWsRRCDp/PK5hLvG7bOIFEOo4RQZcgb5yv5+KFoyekQIvauqDldVkxS5h169yvEEkUTV+6CdCN8PKW7bKRU4+7i5GUqqHRnurVSf5aSTNlvcflUSobC8Hgvf3kvtXrl11MftysfnGvvkIUj1gYz+J5VofkkZsfLYcEedMGv20= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1777211755; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=p0uQob+N5O8wbOKZuJ2cl6QGt2btZMLnp6lMLH+YdM8=; b=dvigQng4ULJnmShRuefVEG0mYlkcwgeXyfaI44Lmm7gdIIwknQI3Xft3SAl29pdaTxJooR5i23n6ioKVLz6U06CJuDT73IHjO/mZWRXs8M1jF+jphCW+QL35Vwt76xszdd/2KUlpptC1L8XO7BjgoGD/K7F0aaQ2EB9xw5mlbbs= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1777211755275902.0768380047737; Sun, 26 Apr 2026 06:55:55 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wGzsh-0008Ez-DR; Sun, 26 Apr 2026 09:50:43 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wGzs5-0007ED-W2 for qemu-devel@nongnu.org; Sun, 26 Apr 2026 09:50:08 -0400 Received: from mail-pf1-x42d.google.com ([2607:f8b0:4864:20::42d]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1wGzs3-0002ui-Fb for qemu-devel@nongnu.org; Sun, 26 Apr 2026 09:50:05 -0400 Received: by mail-pf1-x42d.google.com with SMTP id d2e1a72fcca58-82f9fdfc965so3680033b3a.1 for ; Sun, 26 Apr 2026 06:50:03 -0700 (PDT) Received: from stoup.. ([180.233.125.15]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-82f8ebba485sm36307102b3a.38.2026.04.26.06.49.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 26 Apr 2026 06:50:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1777211402; x=1777816202; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=p0uQob+N5O8wbOKZuJ2cl6QGt2btZMLnp6lMLH+YdM8=; b=WAYmfVvk9iC1JFCcZFJSiiCydk7qp2AmVUV0ZDF6GruI1+dSnu7689JhTgrnisbMIl tk/xTA2wuDA180ok0UU+2nmUaO2tA+uQUqmPBdrYUFmxCtz7LlX3ka45qA5SlkKYBeLd 8UQp6SXcL2buIEMorisLMGlPFlNFmRc5+bAh0VgJ2DlYDjSp+4qvt2Vs0lch/d/EFvhm 5phUga8ADIUESzyMTnSCTNBSfxm5dhy42oi413K5/dXD4UPBb0W8hFCNtdYNcccI55Hd V7u0esqkLN3rUnoy5+6ObhCUWT7jy9PjsJonw+WXz26thivFwxNB19k5pYY9ZMHalzQ+ gpvw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1777211402; x=1777816202; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=p0uQob+N5O8wbOKZuJ2cl6QGt2btZMLnp6lMLH+YdM8=; b=oMgXhEfszovTgMnXf14v9hN9Z2mKSWNTMSXHpRBGwDhbzYtAsgWSNrsmMXP8SRDsX7 4VYBm4l18YLzwdT/UsjuGgQkSFqf5TWFaU6s1IM+gLRNrAuXy2rqCUe9iaerTc5D5oyw DmmSbO4UvJjm1sRamQJziCaozpoKNaxaR68rvj5y5PjAqCj9q8LivwwDeJPokIZli5di dgnTH7PgxmI/VXGjRMohpyoO4xDa1aXg9KDcV6u7XURCdOJjD/WMJtgDm1FEswLjFyk0 2O2YqvmDGo6SW0nEULdFRHtqo0m7H1mf9s5xnQPnFhyDMAkEfDUayK0jraQAQ8X/YMpF sCRA== X-Gm-Message-State: AOJu0YwAOiTQ2NmAvybi4oMy3hdoSd/03WMC5XbNUQjihAJZ5VOoA/kB w1qVHowwTgTmfeQOr53y2BgK/lEGIZo4two6s97QNaWW/B/YkRSjD6t116K5KXhkUuWSN8+bOi2 ozov9wXA= X-Gm-Gg: AeBDieuemHiGHP74vvPKBuZ9jK5aDR1KKdFcWtkcfy7qIO5bwd3pU2zMAqHQKf8iUkK 0FBj9ci/2ZOt9he9TIyFr1fiiwVfrxeU8HV8wkLPo8nVYmzBg4ugYB4RTUtO2hPJ9GZB2DnaHm+ j8+IOy7PTr9FGiYVWUDccY/hZNPNvHMMOO3g4m6gBTfFTJqPn/l/6it77Z3rRd+rz8qxj488iST dhYyr1fJ9WzXCtek0EeqO9Ay1ZR5nCyreQOc4RNqjrPVh1TLOaTbtYXxwekBBPQfqqeVCOSNGsF edQtbu0W1mQ5NrKqK4ApkGZuw+x5evMFIuEN1Sr84MOh8yHpX1hw7Cij0M+aShv00Q0yBoXQ/GO LfN1MmLozBch7ls7WJZX5FuxqrCRZIs7APwVvNlEkf9y4Q53uWdwLxA6pvdL7FmB3cl54QW7+87 mU16ozry4VVYK0Dofx6muIQfIfVOAyr4hSKOZ4kqsx X-Received: by 2002:a05:6a00:6c8d:b0:82c:7f08:8826 with SMTP id d2e1a72fcca58-82f8c8b3dbemr35029369b3a.17.1777211401888; Sun, 26 Apr 2026 06:50:01 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: qemu-arm@nongnu.org, qemu-s390x@nongnu.org Subject: [PATCH 84/84] target/arm: Use FloatParts64 in f16_dotadd Date: Sun, 26 Apr 2026 23:40:01 +1000 Message-ID: <20260426134002.865628-85-richard.henderson@linaro.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260426134002.865628-1-richard.henderson@linaro.org> References: <20260426134002.865628-1-richard.henderson@linaro.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::42d; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x42d.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @linaro.org) X-ZM-MESSAGEID: 1777211756812158500 Content-Type: text/plain; charset="utf-8" Use softfloat-parts.h so that we can more naturally perform the required operations witha single rounding step. This happens to also simplify the NaN detection step. Signed-off-by: Richard Henderson --- target/arm/tcg/sme_helper.c | 96 ++++++++++++++++--------------------- 1 file changed, 40 insertions(+), 56 deletions(-) diff --git a/target/arm/tcg/sme_helper.c b/target/arm/tcg/sme_helper.c index 0702a1b129..71fe44445d 100644 --- a/target/arm/tcg/sme_helper.c +++ b/target/arm/tcg/sme_helper.c @@ -27,6 +27,7 @@ #include "accel/tcg/helper-retaddr.h" #include "qemu/int128.h" #include "fpu/softfloat.h" +#include "fpu/softfloat-parts.h" #include "vec_internal.h" #include "sve_ldst_internal.h" =20 @@ -1227,18 +1228,15 @@ static inline uint32_t bf16mop_ah_neg_adj_pair(uint= 32_t pair, uint32_t pg) } =20 static float32 f16_dotadd(float32 sum, uint32_t e1, uint32_t e2, - float_status *s_f16, float_status *s_std, - float_status *s_odd) + float_status *s_f16, float_status *s_std) { /* - * We need three different float_status for different parts of this + * We need two different float_status for different parts of this * operation: * - the input conversion of the float16 values must use the * f16-specific float_status, so that the FPCR.FZ16 control is appl= ied * - operations on float32 including the final accumulation must use * the normal float_status, so that FPCR.FZ is applied - * - we have pre-set-up copy of s_std which is set to round-to-odd, - * for the multiply (see below) */ float16 h1r =3D e1 & 0xffff; float16 h1c =3D e1 >> 16; @@ -1246,48 +1244,49 @@ static float32 f16_dotadd(float32 sum, uint32_t e1,= uint32_t e2, float16 h2c =3D e2 >> 16; float32 t32; =20 + FloatParts64 p1r =3D float16_unpack_canonical(h1r, s_f16); + FloatParts64 p1c =3D float16_unpack_canonical(h1c, s_f16); + FloatParts64 p2r =3D float16_unpack_canonical(h2r, s_f16); + FloatParts64 p2c =3D float16_unpack_canonical(h2c, s_f16); + + int all_mask =3D (float_cmask(p1r.cls) | float_cmask(p1c.cls) | + float_cmask(p1r.cls) | float_cmask(p1c.cls)); + /* C.f. FPProcessNaNs4 */ - if (float16_is_any_nan(h1r) || float16_is_any_nan(h1c) || - float16_is_any_nan(h2r) || float16_is_any_nan(h2c)) { + if (unlikely(all_mask & float_cmask_anynan)) { float16 t16; =20 - if (float16_is_signaling_nan(h1r, s_f16)) { - t16 =3D h1r; - } else if (float16_is_signaling_nan(h1c, s_f16)) { - t16 =3D h1c; - } else if (float16_is_signaling_nan(h2r, s_f16)) { - t16 =3D h2r; - } else if (float16_is_signaling_nan(h2c, s_f16)) { - t16 =3D h2c; - } else if (float16_is_any_nan(h1r)) { - t16 =3D h1r; - } else if (float16_is_any_nan(h1c)) { - t16 =3D h1c; - } else if (float16_is_any_nan(h2r)) { - t16 =3D h2r; + if (unlikely(all_mask & float_cmask_snan)) { + if (p1r.cls =3D=3D float_class_snan) { + t16 =3D h1r; + } else if (p1c.cls =3D=3D float_class_snan) { + t16 =3D h1c; + } else if (p2r.cls =3D=3D float_class_snan) { + t16 =3D h2r; + } else { + t16 =3D h2c; + } } else { - t16 =3D h2c; + if (p1r.cls =3D=3D float_class_qnan) { + t16 =3D h1r; + } else if (p1c.cls =3D=3D float_class_qnan) { + t16 =3D h1c; + } else if (p2r.cls =3D=3D float_class_qnan) { + t16 =3D h2r; + } else { + t16 =3D h2c; + } } t32 =3D float16_to_float32(t16, true, s_f16); } else { - float64 e1r =3D float16_to_float64(h1r, true, s_f16); - float64 e1c =3D float16_to_float64(h1c, true, s_f16); - float64 e2r =3D float16_to_float64(h2r, true, s_f16); - float64 e2c =3D float16_to_float64(h2c, true, s_f16); - float64 t64; - /* * The ARM pseudocode function FPDot performs both multiplies - * and the add with a single rounding operation. Emulate this - * by performing the first multiply in round-to-odd, then doing - * the second multiply as fused multiply-add, and rounding to - * float32 all in one step. + * and the add with a single rounding operation. */ - t64 =3D float64_mul(e1r, e2r, s_odd); - t64 =3D float64r32_muladd(e1c, e2c, t64, 0, s_std); + FloatParts64 tmp =3D parts64_mul(&p1r, &p2r, s_std); + tmp =3D parts64_muladd_scalbn(&p1c, &p2c, &tmp, 0, 0, s_std); =20 - /* This conversion is exact, because we've already rounded. */ - t32 =3D float64_to_float32(t64, s_std); + t32 =3D float32_round_pack_canonical(&tmp, s_std); } =20 /* The final accumulation step is not fused. */ @@ -1299,9 +1298,6 @@ static void do_fmopa_w_h(void *vza, void *vzn, void *= vzm, uint16_t *pn, uint32_t negx, bool ah_neg) { intptr_t row, col, oprsz =3D simd_maxsz(desc); - float_status fpst_odd =3D env->vfp.fp_status[FPST_ZA]; - - set_float_rounding_mode(float_round_to_odd, &fpst_odd); =20 for (row =3D 0; row < oprsz; ) { uint16_t prow =3D pn[H2(row >> 4)]; @@ -1325,8 +1321,7 @@ static void do_fmopa_w_h(void *vza, void *vzn, void *= vzm, uint16_t *pn, m =3D f16mop_adj_pair(m, pcol, 0); *a =3D f16_dotadd(*a, n, m, &env->vfp.fp_status[FPST_ZA_F16], - &env->vfp.fp_status[FPST_ZA], - &fpst_odd); + &env->vfp.fp_status[FPST_ZA]); } col +=3D 4; pcol >>=3D 4; @@ -1363,15 +1358,12 @@ void HELPER(sme2_fdot_h)(void *vd, void *vn, void *= vm, void *va, bool za =3D extract32(desc, SIMD_DATA_SHIFT, 1); float_status *fpst_std =3D &env->vfp.fp_status[za ? FPST_ZA : FPST_A64= ]; float_status *fpst_f16 =3D &env->vfp.fp_status[za ? FPST_ZA_F16 : FPST= _A64_F16]; - float_status fpst_odd =3D *fpst_std; float32 *d =3D vd, *a =3D va; uint32_t *n =3D vn, *m =3D vm; =20 - set_float_rounding_mode(float_round_to_odd, &fpst_odd); - for (i =3D 0; i < oprsz / sizeof(float32); ++i) { d[H4(i)] =3D f16_dotadd(a[H4(i)], n[H4(i)], m[H4(i)], - fpst_f16, fpst_std, &fpst_odd); + fpst_f16, fpst_std); } } =20 @@ -1385,17 +1377,14 @@ void HELPER(sme2_fdot_idx_h)(void *vd, void *vn, vo= id *vm, void *va, bool za =3D extract32(desc, SIMD_DATA_SHIFT + 2, 1); float_status *fpst_std =3D &env->vfp.fp_status[za ? FPST_ZA : FPST_A64= ]; float_status *fpst_f16 =3D &env->vfp.fp_status[za ? FPST_ZA_F16 : FPST= _A64_F16]; - float_status fpst_odd =3D *fpst_std; float32 *d =3D vd, *a =3D va; uint32_t *n =3D vn, *m =3D (uint32_t *)vm + H4(idx); =20 - set_float_rounding_mode(float_round_to_odd, &fpst_odd); - for (i =3D 0; i < elements; i +=3D eltspersegment) { uint32_t mm =3D m[i]; for (j =3D 0; j < eltspersegment; ++j) { d[H4(i + j)] =3D f16_dotadd(a[H4(i + j)], n[H4(i + j)], mm, - fpst_f16, fpst_std, &fpst_odd); + fpst_f16, fpst_std); } } } @@ -1408,24 +1397,19 @@ void HELPER(sme2_fvdot_idx_h)(void *vd, void *vn, v= oid *vm, void *va, intptr_t eltspersegment =3D MIN(4, elements); int idx =3D extract32(desc, SIMD_DATA_SHIFT, 2); int sel =3D extract32(desc, SIMD_DATA_SHIFT + 2, 1); - float_status fpst_odd, *fpst_std, *fpst_f16; float32 *d =3D vd, *a =3D va; uint16_t *n0 =3D vn; uint16_t *n1 =3D vn + sizeof(ARMVectorReg); uint32_t *m =3D (uint32_t *)vm + H4(idx); =20 - fpst_std =3D &env->vfp.fp_status[FPST_ZA]; - fpst_f16 =3D &env->vfp.fp_status[FPST_ZA_F16]; - fpst_odd =3D *fpst_std; - set_float_rounding_mode(float_round_to_odd, &fpst_odd); - for (i =3D 0; i < elements; i +=3D eltspersegment) { uint32_t mm =3D m[i]; for (j =3D 0; j < eltspersegment; ++j) { uint32_t nn =3D (n0[H2(2 * (i + j) + sel)]) | (n1[H2(2 * (i + j) + sel)] << 16); d[i + H4(j)] =3D f16_dotadd(a[i + H4(j)], nn, mm, - fpst_f16, fpst_std, &fpst_odd); + &env->vfp.fp_status[FPST_ZA_F16], + &env->vfp.fp_status[FPST_ZA]); } } } --=20 2.43.0