From nobody Tue Feb 10 07:42:18 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=linaro.org ARC-Seal: i=1; a=rsa-sha256; t=1588264835; cv=none; d=zohomail.com; s=zohoarc; b=NmzFQ73xRfAmUvqxXvx1V2C9e/lNwDn+T5C+8OLlT9K38oBmErpXheNhUespxO/mPCyJhllORPm8WZX3fKmO5lnYFTPRBhoY3At3qdtP7jSRhEFf8Bjcfph/Jg/Th8lHtOjIqGTUd44xwoRW5xhwIrhEK1vVJpRtWcCmukwJBTg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1588264835; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=suwrl9RFQOuVoDr4Q1KFh+iDEpTs3nZzUnaA8dPRNoc=; b=b8+S9v6R1qnmzS+fSkniACcQuekxijL9HdI6qQl6C+m8ETxUVJd/a7HRT5TaWMGun27mcb0gxg7wK0GoyMLCw2SVzEUJ3r1GYZqSt/oDIP3Z7gUxqDE/O33IF59wAh7wpUJWm1JvyOk1cRABI4zOUEcb0E13aQswEbhVVADIpqc= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1588264835092336.12565317865085; Thu, 30 Apr 2020 09:40:35 -0700 (PDT) Received: from localhost ([::1]:35724 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jUCEj-0001ns-O2 for importer@patchew.org; Thu, 30 Apr 2020 12:40:33 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49370) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jUC3A-0001pl-2r for qemu-devel@nongnu.org; Thu, 30 Apr 2020 12:28:40 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.90_1) (envelope-from ) id 1jUC34-0006Xj-Bn for qemu-devel@nongnu.org; Thu, 30 Apr 2020 12:28:35 -0400 Received: from mail-pg1-x541.google.com ([2607:f8b0:4864:20::541]:38207) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jUC33-0006TW-TS for qemu-devel@nongnu.org; Thu, 30 Apr 2020 12:28:29 -0400 Received: by mail-pg1-x541.google.com with SMTP id l25so645641pgc.5 for ; Thu, 30 Apr 2020 09:28:29 -0700 (PDT) Received: from localhost.localdomain (174-21-149-226.tukw.qwest.net. [174.21.149.226]) by smtp.gmail.com with ESMTPSA id j5sm243514pfh.58.2020.04.30.09.28.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 30 Apr 2020 09:28:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=suwrl9RFQOuVoDr4Q1KFh+iDEpTs3nZzUnaA8dPRNoc=; b=JLix+IVuTMnjIs/ulxnD9IEp6TfTl7q2sFcBbtsQ6EI4sTOScilDE5oiA/PfGnXBZl 0OYCW9aBvlRPS2EZV3hYYSIotA6VdchjVUtdMOSvDi1Lz0t1QFZy2azsBx/2EORt4Btq CVBXKhjlnxHqiuGMvNQf5lDlq+P+qo8e6ZQKuDIpD8qCpZPARvXRk3noaBZ80Fp/T346 wvHo2QGf7peF9AqKZ2AgNM2Fbc2O9B+r1zsBQRhXWsF/ZtkQbswe7R77sWiKWY0QtS6j RGMKKG7zMORZbt14FYthbfFrCym97Oqqqis2Yg8cT6jPnxZOFU3jvu2sUuk2n1Oz8E6t VdjA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=suwrl9RFQOuVoDr4Q1KFh+iDEpTs3nZzUnaA8dPRNoc=; b=a71DDDcg7KRO6vqlFtQgiFLOH4Bf+FPJYOmw3eyTIhEjKKEa+YTVjDik9XfaoHB3PZ 7LI/bgmEZnIIuEhn7JwzRfehw0F63mi2CHRxLC5CAA/Jqdp7C2Im9dKmHy7kz/Pf1fxQ Z5u0SelVnSh63MRTreeHONpidJ+G1N/x3IoUcys7pSNdYK256d0hsTG6pSpiaV9aGIP0 jUNxhkRk5Lm9A5B3dm0WWEqC2S1KS6+SCph1vn/KOKJmnY6/vhMlNa7moZX5iT9h4GMi VYFqprg/apUWeIAZrWlkdXOCMAYHZsOVncTSdP3soDhuROLpS7VdIc0klzbydyXwAuOH LIfw== X-Gm-Message-State: AGi0PuaN8bsHFudFkTqF/SLBP/gSwe2P2O08vJf2aWg/w6EKEu+nrs9z Nxc8IrebzjLk2ZcNtZz1jllNgEar1RU= X-Google-Smtp-Source: APiQypI1fybAuARIEMORrUQpBtSDe4fxXSFf1LArZ2qxE4+Yx6isEnzCRhtfOVq5JiWifxItYLjXuQ== X-Received: by 2002:a63:4665:: with SMTP id v37mr3780795pgk.297.1588264107940; Thu, 30 Apr 2020 09:28:27 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 09/18] target/arm: Adjust interface of sve_ld1_host_fn Date: Thu, 30 Apr 2020 09:28:04 -0700 Message-Id: <20200430162813.17671-10-richard.henderson@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200430162813.17671-1-richard.henderson@linaro.org> References: <20200430162813.17671-1-richard.henderson@linaro.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::541; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x541.google.com X-detected-operating-system: by eggs.gnu.org: Error: [-] PROGRAM ABORT : Malformed IPv6 address (bad octet value). Location : parse_addr6(), p0f-client.c:67 X-Received-From: 2607:f8b0:4864:20::541 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @linaro.org) Content-Type: text/plain; charset="utf-8" The current interface includes a loop; change it to load a single element. We will then be able to use the function for ld{2,3,4} where individual vector elements are not adjacent. Replace each call with the simplest possible loop over active elements. Reviewed-by: Peter Maydell Signed-off-by: Richard Henderson --- target/arm/sve_helper.c | 124 ++++++++++++++++++++-------------------- 1 file changed, 63 insertions(+), 61 deletions(-) diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 2f053a9152..d007137735 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -3972,20 +3972,10 @@ void HELPER(sve_fcmla_zpzzz_d)(CPUARMState *env, vo= id *vg, uint32_t desc) */ =20 /* - * Load elements into @vd, controlled by @vg, from @host + @mem_ofs. - * Memory is valid through @host + @mem_max. The register element - * indices are inferred from @mem_ofs, as modified by the types for - * which the helper is built. Return the @mem_ofs of the first element - * not loaded (which is @mem_max if they are all loaded). - * - * For softmmu, we have fully validated the guest page. For user-only, - * we cannot fully validate without taking the mmap lock, but since we - * know the access is within one host page, if any access is valid they - * all must be valid. However, when @vg is all false, it may be that - * no access is valid. + * Load one element into @vd + @reg_off from @host. + * The controlling predicate is known to be true. */ -typedef intptr_t sve_ld1_host_fn(void *vd, void *vg, void *host, - intptr_t mem_ofs, intptr_t mem_max); +typedef void sve_ldst1_host_fn(void *vd, intptr_t reg_off, void *host); =20 /* * Load one element into @vd + @reg_off from (@env, @vaddr, @ra). @@ -3999,20 +3989,10 @@ typedef void sve_ldst1_tlb_fn(CPUARMState *env, voi= d *vd, intptr_t reg_off, */ =20 #define DO_LD_HOST(NAME, H, TYPEE, TYPEM, HOST) \ -static intptr_t sve_##NAME##_host(void *vd, void *vg, void *host, = \ - intptr_t mem_off, const intptr_t mem_max= ) \ -{ = \ - intptr_t reg_off =3D mem_off * (sizeof(TYPEE) / sizeof(TYPEM)); = \ - uint64_t *pg =3D vg; = \ - while (mem_off + sizeof(TYPEM) <=3D mem_max) { = \ - TYPEM val =3D 0; = \ - if (likely((pg[reg_off >> 6] >> (reg_off & 63)) & 1)) { = \ - val =3D HOST(host + mem_off); = \ - } = \ - *(TYPEE *)(vd + H(reg_off)) =3D val; = \ - mem_off +=3D sizeof(TYPEM), reg_off +=3D sizeof(TYPEE); = \ - } = \ - return mem_off; = \ +static void sve_##NAME##_host(void *vd, intptr_t reg_off, void *host) \ +{ \ + TYPEM val =3D HOST(host); \ + *(TYPEE *)(vd + H(reg_off)) =3D val; \ } =20 #define DO_LD_TLB(NAME, H, TYPEE, TYPEM, TLB) \ @@ -4411,7 +4391,7 @@ static inline bool test_host_page(void *host) static void sve_ld1_r(CPUARMState *env, void *vg, const target_ulong addr, uint32_t desc, const uintptr_t retaddr, const int esz, const int msz, - sve_ld1_host_fn *host_fn, + sve_ldst1_host_fn *host_fn, sve_ldst1_tlb_fn *tlb_fn) { const TCGMemOpIdx oi =3D extract32(desc, SIMD_DATA_SHIFT, MEMOPIDX_SHI= FT); @@ -4445,8 +4425,12 @@ static void sve_ld1_r(CPUARMState *env, void *vg, co= nst target_ulong addr, if (likely(split =3D=3D mem_max)) { host =3D tlb_vaddr_to_host(env, addr + mem_off, MMU_DATA_LOAD, mmu= _idx); if (test_host_page(host)) { - mem_off =3D host_fn(vd, vg, host - mem_off, mem_off, mem_max); - tcg_debug_assert(mem_off =3D=3D mem_max); + intptr_t i =3D reg_off; + host -=3D mem_off; + do { + host_fn(vd, i, host + (i >> diffsz)); + i =3D find_next_active(vg, i + (1 << esz), reg_max, esz); + } while (i < reg_max); /* After having taken any fault, zero leading inactive element= s. */ swap_memzero(vd, reg_off); return; @@ -4459,7 +4443,12 @@ static void sve_ld1_r(CPUARMState *env, void *vg, co= nst target_ulong addr, */ #ifdef CONFIG_USER_ONLY swap_memzero(&scratch, reg_off); - host_fn(&scratch, vg, g2h(addr), mem_off, mem_max); + host =3D g2h(addr); + do { + host_fn(&scratch, reg_off, host + (reg_off >> diffsz)); + reg_off +=3D 1 << esz; + reg_off =3D find_next_active(vg, reg_off, reg_max, esz); + } while (reg_off < reg_max); #else memset(&scratch, 0, reg_max); goto start; @@ -4477,9 +4466,13 @@ static void sve_ld1_r(CPUARMState *env, void *vg, co= nst target_ulong addr, host =3D tlb_vaddr_to_host(env, addr + mem_off, MMU_DATA_LOAD, mmu_idx); if (host) { - mem_off =3D host_fn(&scratch, vg, host - mem_off, - mem_off, split); - reg_off =3D mem_off << diffsz; + host -=3D mem_off; + do { + host_fn(&scratch, reg_off, host + mem_off); + reg_off +=3D 1 << esz; + reg_off =3D find_next_active(vg, reg_off, reg_max, esz= ); + mem_off =3D reg_off >> diffsz; + } while (split - mem_off >=3D (1 << msz)); continue; } } @@ -4706,7 +4699,7 @@ static void record_fault(CPUARMState *env, uintptr_t = i, uintptr_t oprsz) static void sve_ldff1_r(CPUARMState *env, void *vg, const target_ulong add= r, uint32_t desc, const uintptr_t retaddr, const int esz, const int msz, - sve_ld1_host_fn *host_fn, + sve_ldst1_host_fn *host_fn, sve_ldst1_tlb_fn *tlb_fn) { const TCGMemOpIdx oi =3D extract32(desc, SIMD_DATA_SHIFT, MEMOPIDX_SHI= FT); @@ -4716,7 +4709,7 @@ static void sve_ldff1_r(CPUARMState *env, void *vg, c= onst target_ulong addr, const int diffsz =3D esz - msz; const intptr_t reg_max =3D simd_oprsz(desc); const intptr_t mem_max =3D reg_max >> diffsz; - intptr_t split, reg_off, mem_off; + intptr_t split, reg_off, mem_off, i; void *host; =20 /* Skip to the first active element. */ @@ -4739,28 +4732,18 @@ static void sve_ldff1_r(CPUARMState *env, void *vg,= const target_ulong addr, if (likely(split =3D=3D mem_max)) { host =3D tlb_vaddr_to_host(env, addr + mem_off, MMU_DATA_LOAD, mmu= _idx); if (test_host_page(host)) { - mem_off =3D host_fn(vd, vg, host - mem_off, mem_off, mem_max); - tcg_debug_assert(mem_off =3D=3D mem_max); + i =3D reg_off; + host -=3D mem_off; + do { + host_fn(vd, i, host + (i >> diffsz)); + i =3D find_next_active(vg, i + (1 << esz), reg_max, esz); + } while (i < reg_max); /* After any fault, zero any leading inactive elements. */ swap_memzero(vd, reg_off); return; } } =20 -#ifdef CONFIG_USER_ONLY - /* - * The page(s) containing this first element at ADDR+MEM_OFF must - * be valid. Considering that this first element may be misaligned - * and cross a page boundary itself, take the rest of the page from - * the last byte of the element. - */ - split =3D max_for_page(addr, mem_off + (1 << msz) - 1, mem_max); - mem_off =3D host_fn(vd, vg, g2h(addr), mem_off, split); - - /* After any fault, zero any leading inactive elements. */ - swap_memzero(vd, reg_off); - reg_off =3D mem_off << diffsz; -#else /* * Perform one normal read, which will fault or not. * But it is likely to bring the page into the tlb. @@ -4777,11 +4760,15 @@ static void sve_ldff1_r(CPUARMState *env, void *vg,= const target_ulong addr, if (split >=3D (1 << msz)) { host =3D tlb_vaddr_to_host(env, addr + mem_off, MMU_DATA_LOAD, mmu= _idx); if (host) { - mem_off =3D host_fn(vd, vg, host - mem_off, mem_off, split); - reg_off =3D mem_off << diffsz; + host -=3D mem_off; + do { + host_fn(vd, reg_off, host + mem_off); + reg_off +=3D 1 << esz; + reg_off =3D find_next_active(vg, reg_off, reg_max, esz); + mem_off =3D reg_off >> diffsz; + } while (split - mem_off >=3D (1 << msz)); } } -#endif =20 record_fault(env, reg_off, reg_max); } @@ -4791,7 +4778,7 @@ static void sve_ldff1_r(CPUARMState *env, void *vg, c= onst target_ulong addr, */ static void sve_ldnf1_r(CPUARMState *env, void *vg, const target_ulong add= r, uint32_t desc, const int esz, const int msz, - sve_ld1_host_fn *host_fn) + sve_ldst1_host_fn *host_fn) { const unsigned rd =3D extract32(desc, SIMD_DATA_SHIFT + MEMOPIDX_SHIFT= , 5); void *vd =3D &env->vfp.zregs[rd]; @@ -4806,7 +4793,13 @@ static void sve_ldnf1_r(CPUARMState *env, void *vg, = const target_ulong addr, host =3D tlb_vaddr_to_host(env, addr, MMU_DATA_LOAD, mmu_idx); if (likely(page_check_range(addr, mem_max, PAGE_READ) =3D=3D 0)) { /* The entire operation is valid and will not fault. */ - host_fn(vd, vg, host, 0, mem_max); + reg_off =3D 0; + do { + mem_off =3D reg_off >> diffsz; + host_fn(vd, reg_off, host + mem_off); + reg_off +=3D 1 << esz; + reg_off =3D find_next_active(vg, reg_off, reg_max, esz); + } while (reg_off < reg_max); return; } #endif @@ -4826,8 +4819,12 @@ static void sve_ldnf1_r(CPUARMState *env, void *vg, = const target_ulong addr, if (page_check_range(addr + mem_off, 1 << msz, PAGE_READ) =3D=3D 0) { /* At least one load is valid; take the rest of the page. */ split =3D max_for_page(addr, mem_off + (1 << msz) - 1, mem_max); - mem_off =3D host_fn(vd, vg, host, mem_off, split); - reg_off =3D mem_off << diffsz; + do { + host_fn(vd, reg_off, host + mem_off); + reg_off +=3D 1 << esz; + reg_off =3D find_next_active(vg, reg_off, reg_max, esz); + mem_off =3D reg_off >> diffsz; + } while (split - mem_off >=3D (1 << msz)); } #else /* @@ -4848,8 +4845,13 @@ static void sve_ldnf1_r(CPUARMState *env, void *vg, = const target_ulong addr, host =3D tlb_vaddr_to_host(env, addr + mem_off, MMU_DATA_LOAD, mmu_idx= ); split =3D max_for_page(addr, mem_off, mem_max); if (host && split >=3D (1 << msz)) { - mem_off =3D host_fn(vd, vg, host - mem_off, mem_off, split); - reg_off =3D mem_off << diffsz; + host -=3D mem_off; + do { + host_fn(vd, reg_off, host + mem_off); + reg_off +=3D 1 << esz; + reg_off =3D find_next_active(vg, reg_off, reg_max, esz); + mem_off =3D reg_off >> diffsz; + } while (split - mem_off >=3D (1 << msz)); } #endif =20 --=20 2.20.1