From nobody Fri Dec 19 18:53:21 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=linaro.org ARC-Seal: i=1; a=rsa-sha256; t=1588953321; cv=none; d=zohomail.com; s=zohoarc; b=PKRj5rbTomQBk2GJQtf/FKdG/jrzi0iGGxf/GVoR7GLHdjkGpwOc5gYsV0N3BJaerOPCytmDo85w28j8bFlcjH+mQctFlWiPI6PzyeLmk3DTPx47yrGO7Y+sGzDjc5lTNk6RdJW+bLQo73FBscvPdJNg//xN9ZCvh7qGln8KAgg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1588953321; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=suwrl9RFQOuVoDr4Q1KFh+iDEpTs3nZzUnaA8dPRNoc=; b=KK89TpncbEArUlrdvnXdUfnB1JzP8wDpwjjIU9uab4/1zlHgmZ06wGTJtLkyUoZCmqNjQ5qjoW9LmMElNGSo++xE0KfjUecTZnTfgreTaffAG79ZVgHNkYQVWXs6x9o6ELyRTeqjwdyJM4j8rlJxc0DAZRquwLCnnO6YbPOL5Ho= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1588953321845705.0812141513275; Fri, 8 May 2020 08:55:21 -0700 (PDT) Received: from localhost ([::1]:57046 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jX5LM-0003NO-HQ for importer@patchew.org; Fri, 08 May 2020 11:55:20 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:41346) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jX5As-0007sm-6Y for qemu-devel@nongnu.org; Fri, 08 May 2020 11:44:30 -0400 Received: from mail-pf1-x442.google.com ([2607:f8b0:4864:20::442]:38244) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jX5Al-0005by-Om for qemu-devel@nongnu.org; Fri, 08 May 2020 11:44:29 -0400 Received: by mail-pf1-x442.google.com with SMTP id y25so1131456pfn.5 for ; Fri, 08 May 2020 08:44:21 -0700 (PDT) Received: from localhost.localdomain (174-21-149-226.tukw.qwest.net. [174.21.149.226]) by smtp.gmail.com with ESMTPSA id o99sm2727662pjo.8.2020.05.08.08.44.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 08 May 2020 08:44:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=suwrl9RFQOuVoDr4Q1KFh+iDEpTs3nZzUnaA8dPRNoc=; b=myHPN7GQc+G596z4C8ExclUNOOuHTCAg6CBr7FbS0XQc0pB9kyHB1RyRj/SlDUEpmL B7p1/YwfQUT0GmihHP4S7pz3ZJZDfYS/x5yCPfMVbZ1005k5VQe3eA80oQXDhlcNo1Id mCEFhpgiaVAtNS1+swJU16Xhg6+ORLTKdPovJf4LtFScnc8y7zSwaRlFDA4GK9JCy+/F +HEeu4vu//D/TdoN5nfMMy/hDFY18bOqvMNV4nX2i43bDiCmntUuA8B0XZO7iCeUYM6e DoBnMp7W2XKRiZxPzNdPoJzVy4PrfpcZao+VxKUTJiTR7DSlpofBLa6NhflvXrykhiL+ 8kWQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=suwrl9RFQOuVoDr4Q1KFh+iDEpTs3nZzUnaA8dPRNoc=; b=NSC0WLrJ8ttlkrRnn9zSOmmpYtz2ri1LwCxPMxwEqkv3uKJYT1Cj/D4YAd8j6Yucle 0bgIUQnT1qR0vC0VIlo+abRU2SKO4b6CYKa98Xa0yZ+7i/XqemeftNSzMumbXnCORL9Q EflviBYlSxpDHv8+Op/KNlwUCtxNNQv1hV88QkEgRDx7A65Qjvrbxf6dWWvVXUwV4D6x 6BgN6e5s5wWBEYqelOEyGtb2hjTGSZ4R8VKDh1HyBFXbpqFygxOPcXiZU8VZ4SXvHMfo TG92mMnC4ORHybOU9rlhehEfFlNkEn6ZaQE5H/Axdo2HiibcsDPNu75enj1DqlY+TXFL 8uCw== X-Gm-Message-State: AGi0PubO7183cSg9RCm5gKbN9OvFnYbqCJmd6lf6MePd5YvvykEcbRVw gp1I4l6C3YjNYAvj3W5VrJ9qXZPkrmk= X-Google-Smtp-Source: APiQypIBSo6DtFdI7XJ2pBIxPL6rtJX5wLRFbJX58N1FTH4mZAKvIvvZKGAJ1RAaMGg06Yh7neP0lQ== X-Received: by 2002:a65:62d5:: with SMTP id m21mr2592997pgv.75.1588952660064; Fri, 08 May 2020 08:44:20 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 10/19] target/arm: Adjust interface of sve_ld1_host_fn Date: Fri, 8 May 2020 08:43:50 -0700 Message-Id: <20200508154359.7494-11-richard.henderson@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200508154359.7494-1-richard.henderson@linaro.org> References: <20200508154359.7494-1-richard.henderson@linaro.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::442; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x442.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @linaro.org) Content-Type: text/plain; charset="utf-8" The current interface includes a loop; change it to load a single element. We will then be able to use the function for ld{2,3,4} where individual vector elements are not adjacent. Replace each call with the simplest possible loop over active elements. Reviewed-by: Peter Maydell Signed-off-by: Richard Henderson --- target/arm/sve_helper.c | 124 ++++++++++++++++++++-------------------- 1 file changed, 63 insertions(+), 61 deletions(-) diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 2f053a9152..d007137735 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -3972,20 +3972,10 @@ void HELPER(sve_fcmla_zpzzz_d)(CPUARMState *env, vo= id *vg, uint32_t desc) */ =20 /* - * Load elements into @vd, controlled by @vg, from @host + @mem_ofs. - * Memory is valid through @host + @mem_max. The register element - * indices are inferred from @mem_ofs, as modified by the types for - * which the helper is built. Return the @mem_ofs of the first element - * not loaded (which is @mem_max if they are all loaded). - * - * For softmmu, we have fully validated the guest page. For user-only, - * we cannot fully validate without taking the mmap lock, but since we - * know the access is within one host page, if any access is valid they - * all must be valid. However, when @vg is all false, it may be that - * no access is valid. + * Load one element into @vd + @reg_off from @host. + * The controlling predicate is known to be true. */ -typedef intptr_t sve_ld1_host_fn(void *vd, void *vg, void *host, - intptr_t mem_ofs, intptr_t mem_max); +typedef void sve_ldst1_host_fn(void *vd, intptr_t reg_off, void *host); =20 /* * Load one element into @vd + @reg_off from (@env, @vaddr, @ra). @@ -3999,20 +3989,10 @@ typedef void sve_ldst1_tlb_fn(CPUARMState *env, voi= d *vd, intptr_t reg_off, */ =20 #define DO_LD_HOST(NAME, H, TYPEE, TYPEM, HOST) \ -static intptr_t sve_##NAME##_host(void *vd, void *vg, void *host, = \ - intptr_t mem_off, const intptr_t mem_max= ) \ -{ = \ - intptr_t reg_off =3D mem_off * (sizeof(TYPEE) / sizeof(TYPEM)); = \ - uint64_t *pg =3D vg; = \ - while (mem_off + sizeof(TYPEM) <=3D mem_max) { = \ - TYPEM val =3D 0; = \ - if (likely((pg[reg_off >> 6] >> (reg_off & 63)) & 1)) { = \ - val =3D HOST(host + mem_off); = \ - } = \ - *(TYPEE *)(vd + H(reg_off)) =3D val; = \ - mem_off +=3D sizeof(TYPEM), reg_off +=3D sizeof(TYPEE); = \ - } = \ - return mem_off; = \ +static void sve_##NAME##_host(void *vd, intptr_t reg_off, void *host) \ +{ \ + TYPEM val =3D HOST(host); \ + *(TYPEE *)(vd + H(reg_off)) =3D val; \ } =20 #define DO_LD_TLB(NAME, H, TYPEE, TYPEM, TLB) \ @@ -4411,7 +4391,7 @@ static inline bool test_host_page(void *host) static void sve_ld1_r(CPUARMState *env, void *vg, const target_ulong addr, uint32_t desc, const uintptr_t retaddr, const int esz, const int msz, - sve_ld1_host_fn *host_fn, + sve_ldst1_host_fn *host_fn, sve_ldst1_tlb_fn *tlb_fn) { const TCGMemOpIdx oi =3D extract32(desc, SIMD_DATA_SHIFT, MEMOPIDX_SHI= FT); @@ -4445,8 +4425,12 @@ static void sve_ld1_r(CPUARMState *env, void *vg, co= nst target_ulong addr, if (likely(split =3D=3D mem_max)) { host =3D tlb_vaddr_to_host(env, addr + mem_off, MMU_DATA_LOAD, mmu= _idx); if (test_host_page(host)) { - mem_off =3D host_fn(vd, vg, host - mem_off, mem_off, mem_max); - tcg_debug_assert(mem_off =3D=3D mem_max); + intptr_t i =3D reg_off; + host -=3D mem_off; + do { + host_fn(vd, i, host + (i >> diffsz)); + i =3D find_next_active(vg, i + (1 << esz), reg_max, esz); + } while (i < reg_max); /* After having taken any fault, zero leading inactive element= s. */ swap_memzero(vd, reg_off); return; @@ -4459,7 +4443,12 @@ static void sve_ld1_r(CPUARMState *env, void *vg, co= nst target_ulong addr, */ #ifdef CONFIG_USER_ONLY swap_memzero(&scratch, reg_off); - host_fn(&scratch, vg, g2h(addr), mem_off, mem_max); + host =3D g2h(addr); + do { + host_fn(&scratch, reg_off, host + (reg_off >> diffsz)); + reg_off +=3D 1 << esz; + reg_off =3D find_next_active(vg, reg_off, reg_max, esz); + } while (reg_off < reg_max); #else memset(&scratch, 0, reg_max); goto start; @@ -4477,9 +4466,13 @@ static void sve_ld1_r(CPUARMState *env, void *vg, co= nst target_ulong addr, host =3D tlb_vaddr_to_host(env, addr + mem_off, MMU_DATA_LOAD, mmu_idx); if (host) { - mem_off =3D host_fn(&scratch, vg, host - mem_off, - mem_off, split); - reg_off =3D mem_off << diffsz; + host -=3D mem_off; + do { + host_fn(&scratch, reg_off, host + mem_off); + reg_off +=3D 1 << esz; + reg_off =3D find_next_active(vg, reg_off, reg_max, esz= ); + mem_off =3D reg_off >> diffsz; + } while (split - mem_off >=3D (1 << msz)); continue; } } @@ -4706,7 +4699,7 @@ static void record_fault(CPUARMState *env, uintptr_t = i, uintptr_t oprsz) static void sve_ldff1_r(CPUARMState *env, void *vg, const target_ulong add= r, uint32_t desc, const uintptr_t retaddr, const int esz, const int msz, - sve_ld1_host_fn *host_fn, + sve_ldst1_host_fn *host_fn, sve_ldst1_tlb_fn *tlb_fn) { const TCGMemOpIdx oi =3D extract32(desc, SIMD_DATA_SHIFT, MEMOPIDX_SHI= FT); @@ -4716,7 +4709,7 @@ static void sve_ldff1_r(CPUARMState *env, void *vg, c= onst target_ulong addr, const int diffsz =3D esz - msz; const intptr_t reg_max =3D simd_oprsz(desc); const intptr_t mem_max =3D reg_max >> diffsz; - intptr_t split, reg_off, mem_off; + intptr_t split, reg_off, mem_off, i; void *host; =20 /* Skip to the first active element. */ @@ -4739,28 +4732,18 @@ static void sve_ldff1_r(CPUARMState *env, void *vg,= const target_ulong addr, if (likely(split =3D=3D mem_max)) { host =3D tlb_vaddr_to_host(env, addr + mem_off, MMU_DATA_LOAD, mmu= _idx); if (test_host_page(host)) { - mem_off =3D host_fn(vd, vg, host - mem_off, mem_off, mem_max); - tcg_debug_assert(mem_off =3D=3D mem_max); + i =3D reg_off; + host -=3D mem_off; + do { + host_fn(vd, i, host + (i >> diffsz)); + i =3D find_next_active(vg, i + (1 << esz), reg_max, esz); + } while (i < reg_max); /* After any fault, zero any leading inactive elements. */ swap_memzero(vd, reg_off); return; } } =20 -#ifdef CONFIG_USER_ONLY - /* - * The page(s) containing this first element at ADDR+MEM_OFF must - * be valid. Considering that this first element may be misaligned - * and cross a page boundary itself, take the rest of the page from - * the last byte of the element. - */ - split =3D max_for_page(addr, mem_off + (1 << msz) - 1, mem_max); - mem_off =3D host_fn(vd, vg, g2h(addr), mem_off, split); - - /* After any fault, zero any leading inactive elements. */ - swap_memzero(vd, reg_off); - reg_off =3D mem_off << diffsz; -#else /* * Perform one normal read, which will fault or not. * But it is likely to bring the page into the tlb. @@ -4777,11 +4760,15 @@ static void sve_ldff1_r(CPUARMState *env, void *vg,= const target_ulong addr, if (split >=3D (1 << msz)) { host =3D tlb_vaddr_to_host(env, addr + mem_off, MMU_DATA_LOAD, mmu= _idx); if (host) { - mem_off =3D host_fn(vd, vg, host - mem_off, mem_off, split); - reg_off =3D mem_off << diffsz; + host -=3D mem_off; + do { + host_fn(vd, reg_off, host + mem_off); + reg_off +=3D 1 << esz; + reg_off =3D find_next_active(vg, reg_off, reg_max, esz); + mem_off =3D reg_off >> diffsz; + } while (split - mem_off >=3D (1 << msz)); } } -#endif =20 record_fault(env, reg_off, reg_max); } @@ -4791,7 +4778,7 @@ static void sve_ldff1_r(CPUARMState *env, void *vg, c= onst target_ulong addr, */ static void sve_ldnf1_r(CPUARMState *env, void *vg, const target_ulong add= r, uint32_t desc, const int esz, const int msz, - sve_ld1_host_fn *host_fn) + sve_ldst1_host_fn *host_fn) { const unsigned rd =3D extract32(desc, SIMD_DATA_SHIFT + MEMOPIDX_SHIFT= , 5); void *vd =3D &env->vfp.zregs[rd]; @@ -4806,7 +4793,13 @@ static void sve_ldnf1_r(CPUARMState *env, void *vg, = const target_ulong addr, host =3D tlb_vaddr_to_host(env, addr, MMU_DATA_LOAD, mmu_idx); if (likely(page_check_range(addr, mem_max, PAGE_READ) =3D=3D 0)) { /* The entire operation is valid and will not fault. */ - host_fn(vd, vg, host, 0, mem_max); + reg_off =3D 0; + do { + mem_off =3D reg_off >> diffsz; + host_fn(vd, reg_off, host + mem_off); + reg_off +=3D 1 << esz; + reg_off =3D find_next_active(vg, reg_off, reg_max, esz); + } while (reg_off < reg_max); return; } #endif @@ -4826,8 +4819,12 @@ static void sve_ldnf1_r(CPUARMState *env, void *vg, = const target_ulong addr, if (page_check_range(addr + mem_off, 1 << msz, PAGE_READ) =3D=3D 0) { /* At least one load is valid; take the rest of the page. */ split =3D max_for_page(addr, mem_off + (1 << msz) - 1, mem_max); - mem_off =3D host_fn(vd, vg, host, mem_off, split); - reg_off =3D mem_off << diffsz; + do { + host_fn(vd, reg_off, host + mem_off); + reg_off +=3D 1 << esz; + reg_off =3D find_next_active(vg, reg_off, reg_max, esz); + mem_off =3D reg_off >> diffsz; + } while (split - mem_off >=3D (1 << msz)); } #else /* @@ -4848,8 +4845,13 @@ static void sve_ldnf1_r(CPUARMState *env, void *vg, = const target_ulong addr, host =3D tlb_vaddr_to_host(env, addr + mem_off, MMU_DATA_LOAD, mmu_idx= ); split =3D max_for_page(addr, mem_off, mem_max); if (host && split >=3D (1 << msz)) { - mem_off =3D host_fn(vd, vg, host - mem_off, mem_off, split); - reg_off =3D mem_off << diffsz; + host -=3D mem_off; + do { + host_fn(vd, reg_off, host + mem_off); + reg_off +=3D 1 << esz; + reg_off =3D find_next_active(vg, reg_off, reg_max, esz); + mem_off =3D reg_off >> diffsz; + } while (split - mem_off >=3D (1 << msz)); } #endif =20 --=20 2.20.1