From nobody Fri Dec 19 04:38:28 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=linaro.org ARC-Seal: i=1; a=rsa-sha256; t=1589204343; cv=none; d=zohomail.com; s=zohoarc; b=QYnqM27Ao/lS9p19BROY+9zUh250BzsTpKrcX1TEqnMa0+BuykH+PBGS+Ci6x+uKT9lU7wxxuSnSxv6NPYCQZbw32BreYQyXomq/TyBmpNq+9NtezC5Jua5U1aw0aNRS5Pya1MLp22B7RAOELT77WTkNSpjq7bBbFnEhvMvqxRw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1589204343; h=Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=Mu/irtNrmgEpmvQIk5i6zytZKlEaEPyqeCdRpUMAy+s=; b=XdyatLz6I6u7AcyTwLHrBx3KKUHvpC+hs7FzwoBOJGJtcvnHVZd/9lzj09rwI3xb4PaWfGOwDGV1RpYE1sIlXd75MuaW7E26fVY4HSQVG4dM2ApSfvispdSjOzM3DDIJLVGadkgq9oN9JWIhMwOqssUerJgXAuzgh4dg3OL/iyE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1589204343615251.97444487228472; Mon, 11 May 2020 06:39:03 -0700 (PDT) Received: from localhost ([::1]:33034 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jY8e6-00034g-8f for importer@patchew.org; Mon, 11 May 2020 09:39:02 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48976) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jY8Zd-0001tl-4Z for qemu-devel@nongnu.org; Mon, 11 May 2020 09:34:25 -0400 Received: from mail-wm1-x344.google.com ([2a00:1450:4864:20::344]:40445) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jY8Zb-00071b-1X for qemu-devel@nongnu.org; Mon, 11 May 2020 09:34:24 -0400 Received: by mail-wm1-x344.google.com with SMTP id u16so19265508wmc.5 for ; Mon, 11 May 2020 06:34:22 -0700 (PDT) Received: from orth.archaic.org.uk (orth.archaic.org.uk. [81.2.115.148]) by smtp.gmail.com with ESMTPSA id m3sm2154818wrn.96.2020.05.11.06.34.19 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 May 2020 06:34:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=Mu/irtNrmgEpmvQIk5i6zytZKlEaEPyqeCdRpUMAy+s=; b=i5hRyZEbtS4HONy8j3hTYlL1QKSTfYGbqdpPx1nlGkWPlp9BBhzXN8bltiK9eM5ebA WhG/w2Hvaxv37ZK6ABgFXv8eh4weIjS/TWOdvEU3WQR/lR1bYulD/radQufY2/JibZ0M UKpqaoCRG9326vbVFW57JJglhxXsjhFKWPQlKKcJMDNDJ3tpaHSBCd9UksppkVMCegXA wLF7EszVohb10oAmp7hjqpt/QY/e26FQNPG4E7iqRfV9kVE7m/fjywRYr27k1kaP7KsP B5SPRGVBxcihAsSWVmUkeCqLFFgqChedZ1tEWlx0wFMezbTJe/OYvEymWOQ12TVjou8N hfzg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Mu/irtNrmgEpmvQIk5i6zytZKlEaEPyqeCdRpUMAy+s=; b=DZQDEGJmsS4LDUiHzeiCFDr9X40OcGjLIWMa35VNDL6z1qJMHDWzErkOCPEtEzk0p0 FudoysGRpZfVOLwwo/9oMmEmhXXWoiR7s/PNUfQlphha1Ay8HD2av6jr696DHiDL6Mm0 RXhIMDalPk5vfBs6bminWzzBCrQKgRlIVia8sTMh8GWX1oEmpiOYBL5vJ2KXfuJeBNsB PMyAaQMjm9GlZpxT/rqrjt3f36uSI3MgYruqEvZxLuDHMGGwsnht/fxy+cNk7WqnLd5b gbAGu/4J2OLslZ9/UrbixU8//ZwUScFIyzIQgC06iKHnq8pULLdiSZlXwrZtuD72P7Em u4xw== X-Gm-Message-State: AGi0PuboER7Vyq5222BgPKQqdleGnJPCXlJoSpxSrhtnOl6x2/zAeUcl 8SqE1Kuc6PeGoGA76jJR4Ye7w0V/Qp/KUg== X-Google-Smtp-Source: APiQypJthHjE2YYC8fYUmTMWEc2sUUNETKwZKxKoBtvMxcXpNgc1r6SaDkhB7xeJzzfL305cCwb4vA== X-Received: by 2002:a05:600c:243:: with SMTP id 3mr27707510wmj.73.1589204061072; Mon, 11 May 2020 06:34:21 -0700 (PDT) From: Peter Maydell To: qemu-devel@nongnu.org Subject: [PULL 12/34] accel/tcg: Add probe_access_flags Date: Mon, 11 May 2020 14:33:43 +0100 Message-Id: <20200511133405.5275-13-peter.maydell@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200511133405.5275-1-peter.maydell@linaro.org> References: <20200511133405.5275-1-peter.maydell@linaro.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::344; envelope-from=peter.maydell@linaro.org; helo=mail-wm1-x344.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @linaro.org) Content-Type: text/plain; charset="utf-8" From: Richard Henderson This new interface will allow targets to probe for a page and then handle watchpoints themselves. This will be most useful for vector predicated memory operations, where one page lookup can be used for many operations, and one test can avoid many watchpoint checks. Signed-off-by: Richard Henderson Message-id: 20200508154359.7494-6-richard.henderson@linaro.org Reviewed-by: Peter Maydell Signed-off-by: Peter Maydell --- include/exec/cpu-all.h | 13 ++- include/exec/exec-all.h | 22 +++++ accel/tcg/cputlb.c | 177 ++++++++++++++++++++-------------------- accel/tcg/user-exec.c | 43 ++++++++-- 4 files changed, 158 insertions(+), 97 deletions(-) diff --git a/include/exec/cpu-all.h b/include/exec/cpu-all.h index 49384bb66a5..43ddcf024c6 100644 --- a/include/exec/cpu-all.h +++ b/include/exec/cpu-all.h @@ -328,7 +328,18 @@ CPUArchState *cpu_copy(CPUArchState *env); | CPU_INTERRUPT_TGT_EXT_3 \ | CPU_INTERRUPT_TGT_EXT_4) =20 -#if !defined(CONFIG_USER_ONLY) +#ifdef CONFIG_USER_ONLY + +/* + * Allow some level of source compatibility with softmmu. We do not + * support any of the more exotic features, so only invalid pages may + * be signaled by probe_access_flags(). + */ +#define TLB_INVALID_MASK (1 << (TARGET_PAGE_BITS_MIN - 1)) +#define TLB_MMIO 0 +#define TLB_WATCHPOINT 0 + +#else =20 /* * Flags stored in the low bits of the TLB virtual address. diff --git a/include/exec/exec-all.h b/include/exec/exec-all.h index d656a1f05c0..8792bea07ab 100644 --- a/include/exec/exec-all.h +++ b/include/exec/exec-all.h @@ -362,6 +362,28 @@ static inline void *probe_read(CPUArchState *env, targ= et_ulong addr, int size, return probe_access(env, addr, size, MMU_DATA_LOAD, mmu_idx, retaddr); } =20 +/** + * probe_access_flags: + * @env: CPUArchState + * @addr: guest virtual address to look up + * @access_type: read, write or execute permission + * @mmu_idx: MMU index to use for lookup + * @nonfault: suppress the fault + * @phost: return value for host address + * @retaddr: return address for unwinding + * + * Similar to probe_access, loosely returning the TLB_FLAGS_MASK for + * the page, and storing the host address for RAM in @phost. + * + * If @nonfault is set, do not raise an exception but return TLB_INVALID_M= ASK. + * Do not handle watchpoints, but include TLB_WATCHPOINT in the returned f= lags. + * Do handle clean pages, so exclude TLB_NOTDIRY from the returned flags. + * For simplicity, all "mmio-like" flags are folded to TLB_MMIO. + */ +int probe_access_flags(CPUArchState *env, target_ulong addr, + MMUAccessType access_type, int mmu_idx, + bool nonfault, void **phost, uintptr_t retaddr); + #define CODE_GEN_ALIGN 16 /* must be >=3D of the size of a icach= e line */ =20 /* Estimated block size for TB allocation. */ diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index e3b5750c3b1..c708e9785f7 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -1231,86 +1231,16 @@ static void notdirty_write(CPUState *cpu, vaddr mem= _vaddr, unsigned size, } } =20 -/* - * Probe for whether the specified guest access is permitted. If it is not - * permitted then an exception will be taken in the same way as if this - * were a real access (and we will not return). - * If the size is 0 or the page requires I/O access, returns NULL; otherwi= se, - * returns the address of the host page similar to tlb_vaddr_to_host(). - */ -void *probe_access(CPUArchState *env, target_ulong addr, int size, - MMUAccessType access_type, int mmu_idx, uintptr_t retad= dr) +static int probe_access_internal(CPUArchState *env, target_ulong addr, + int fault_size, MMUAccessType access_type, + int mmu_idx, bool nonfault, + void **phost, uintptr_t retaddr) { uintptr_t index =3D tlb_index(env, mmu_idx, addr); CPUTLBEntry *entry =3D tlb_entry(env, mmu_idx, addr); - target_ulong tlb_addr; - size_t elt_ofs; - int wp_access; - - g_assert(-(addr | TARGET_PAGE_MASK) >=3D size); - - switch (access_type) { - case MMU_DATA_LOAD: - elt_ofs =3D offsetof(CPUTLBEntry, addr_read); - wp_access =3D BP_MEM_READ; - break; - case MMU_DATA_STORE: - elt_ofs =3D offsetof(CPUTLBEntry, addr_write); - wp_access =3D BP_MEM_WRITE; - break; - case MMU_INST_FETCH: - elt_ofs =3D offsetof(CPUTLBEntry, addr_code); - wp_access =3D BP_MEM_READ; - break; - default: - g_assert_not_reached(); - } - tlb_addr =3D tlb_read_ofs(entry, elt_ofs); - - if (unlikely(!tlb_hit(tlb_addr, addr))) { - if (!victim_tlb_hit(env, mmu_idx, index, elt_ofs, - addr & TARGET_PAGE_MASK)) { - tlb_fill(env_cpu(env), addr, size, access_type, mmu_idx, retad= dr); - /* TLB resize via tlb_fill may have moved the entry. */ - index =3D tlb_index(env, mmu_idx, addr); - entry =3D tlb_entry(env, mmu_idx, addr); - } - tlb_addr =3D tlb_read_ofs(entry, elt_ofs); - } - - if (!size) { - return NULL; - } - - if (unlikely(tlb_addr & TLB_FLAGS_MASK)) { - CPUIOTLBEntry *iotlbentry =3D &env_tlb(env)->d[mmu_idx].iotlb[inde= x]; - - /* Reject I/O access, or other required slow-path. */ - if (tlb_addr & (TLB_MMIO | TLB_BSWAP | TLB_DISCARD_WRITE)) { - return NULL; - } - - /* Handle watchpoints. */ - if (tlb_addr & TLB_WATCHPOINT) { - cpu_check_watchpoint(env_cpu(env), addr, size, - iotlbentry->attrs, wp_access, retaddr); - } - - /* Handle clean RAM pages. */ - if (tlb_addr & TLB_NOTDIRTY) { - notdirty_write(env_cpu(env), addr, size, iotlbentry, retaddr); - } - } - - return (void *)((uintptr_t)addr + entry->addend); -} - -void *tlb_vaddr_to_host(CPUArchState *env, abi_ptr addr, - MMUAccessType access_type, int mmu_idx) -{ - CPUTLBEntry *entry =3D tlb_entry(env, mmu_idx, addr); - target_ulong tlb_addr, page; + target_ulong tlb_addr, page_addr; size_t elt_ofs; + int flags; =20 switch (access_type) { case MMU_DATA_LOAD: @@ -1325,20 +1255,19 @@ void *tlb_vaddr_to_host(CPUArchState *env, abi_ptr = addr, default: g_assert_not_reached(); } - - page =3D addr & TARGET_PAGE_MASK; tlb_addr =3D tlb_read_ofs(entry, elt_ofs); =20 - if (!tlb_hit_page(tlb_addr, page)) { - uintptr_t index =3D tlb_index(env, mmu_idx, addr); - - if (!victim_tlb_hit(env, mmu_idx, index, elt_ofs, page)) { + page_addr =3D addr & TARGET_PAGE_MASK; + if (!tlb_hit_page(tlb_addr, page_addr)) { + if (!victim_tlb_hit(env, mmu_idx, index, elt_ofs, page_addr)) { CPUState *cs =3D env_cpu(env); CPUClass *cc =3D CPU_GET_CLASS(cs); =20 - if (!cc->tlb_fill(cs, addr, 0, access_type, mmu_idx, true, 0))= { + if (!cc->tlb_fill(cs, addr, fault_size, access_type, + mmu_idx, nonfault, retaddr)) { /* Non-faulting page table read failed. */ - return NULL; + *phost =3D NULL; + return TLB_INVALID_MASK; } =20 /* TLB resize via tlb_fill may have moved the entry. */ @@ -1346,15 +1275,89 @@ void *tlb_vaddr_to_host(CPUArchState *env, abi_ptr = addr, } tlb_addr =3D tlb_read_ofs(entry, elt_ofs); } + flags =3D tlb_addr & TLB_FLAGS_MASK; =20 - if (tlb_addr & ~TARGET_PAGE_MASK) { - /* IO access */ + /* Fold all "mmio-like" bits into TLB_MMIO. This is not RAM. */ + if (unlikely(flags & ~(TLB_WATCHPOINT | TLB_NOTDIRTY))) { + *phost =3D NULL; + return TLB_MMIO; + } + + /* Everything else is RAM. */ + *phost =3D (void *)((uintptr_t)addr + entry->addend); + return flags; +} + +int probe_access_flags(CPUArchState *env, target_ulong addr, + MMUAccessType access_type, int mmu_idx, + bool nonfault, void **phost, uintptr_t retaddr) +{ + int flags; + + flags =3D probe_access_internal(env, addr, 0, access_type, mmu_idx, + nonfault, phost, retaddr); + + /* Handle clean RAM pages. */ + if (unlikely(flags & TLB_NOTDIRTY)) { + uintptr_t index =3D tlb_index(env, mmu_idx, addr); + CPUIOTLBEntry *iotlbentry =3D &env_tlb(env)->d[mmu_idx].iotlb[inde= x]; + + notdirty_write(env_cpu(env), addr, 1, iotlbentry, retaddr); + flags &=3D ~TLB_NOTDIRTY; + } + + return flags; +} + +void *probe_access(CPUArchState *env, target_ulong addr, int size, + MMUAccessType access_type, int mmu_idx, uintptr_t retad= dr) +{ + void *host; + int flags; + + g_assert(-(addr | TARGET_PAGE_MASK) >=3D size); + + flags =3D probe_access_internal(env, addr, size, access_type, mmu_idx, + false, &host, retaddr); + + /* Per the interface, size =3D=3D 0 merely faults the access. */ + if (size =3D=3D 0) { return NULL; } =20 - return (void *)((uintptr_t)addr + entry->addend); + if (unlikely(flags & (TLB_NOTDIRTY | TLB_WATCHPOINT))) { + uintptr_t index =3D tlb_index(env, mmu_idx, addr); + CPUIOTLBEntry *iotlbentry =3D &env_tlb(env)->d[mmu_idx].iotlb[inde= x]; + + /* Handle watchpoints. */ + if (flags & TLB_WATCHPOINT) { + int wp_access =3D (access_type =3D=3D MMU_DATA_STORE + ? BP_MEM_WRITE : BP_MEM_READ); + cpu_check_watchpoint(env_cpu(env), addr, size, + iotlbentry->attrs, wp_access, retaddr); + } + + /* Handle clean RAM pages. */ + if (flags & TLB_NOTDIRTY) { + notdirty_write(env_cpu(env), addr, 1, iotlbentry, retaddr); + } + } + + return host; } =20 +void *tlb_vaddr_to_host(CPUArchState *env, abi_ptr addr, + MMUAccessType access_type, int mmu_idx) +{ + void *host; + int flags; + + flags =3D probe_access_internal(env, addr, 0, access_type, + mmu_idx, true, &host, 0); + + /* No combination of flags are expected by the caller. */ + return flags ? NULL : host; +} =20 #ifdef CONFIG_PLUGIN /* diff --git a/accel/tcg/user-exec.c b/accel/tcg/user-exec.c index 03538e2a389..987342c50c9 100644 --- a/accel/tcg/user-exec.c +++ b/accel/tcg/user-exec.c @@ -190,13 +190,12 @@ static inline int handle_cpu_signal(uintptr_t pc, sig= info_t *info, g_assert_not_reached(); } =20 -void *probe_access(CPUArchState *env, target_ulong addr, int size, - MMUAccessType access_type, int mmu_idx, uintptr_t retad= dr) +static int probe_access_internal(CPUArchState *env, target_ulong addr, + int fault_size, MMUAccessType access_type, + bool nonfault, uintptr_t ra) { int flags; =20 - g_assert(-(addr | TARGET_PAGE_MASK) >=3D size); - switch (access_type) { case MMU_DATA_STORE: flags =3D PAGE_WRITE; @@ -212,12 +211,38 @@ void *probe_access(CPUArchState *env, target_ulong ad= dr, int size, } =20 if (!guest_addr_valid(addr) || page_check_range(addr, 1, flags) < 0) { - CPUState *cpu =3D env_cpu(env); - CPUClass *cc =3D CPU_GET_CLASS(cpu); - cc->tlb_fill(cpu, addr, size, access_type, MMU_USER_IDX, false, - retaddr); - g_assert_not_reached(); + if (nonfault) { + return TLB_INVALID_MASK; + } else { + CPUState *cpu =3D env_cpu(env); + CPUClass *cc =3D CPU_GET_CLASS(cpu); + cc->tlb_fill(cpu, addr, fault_size, access_type, + MMU_USER_IDX, false, ra); + g_assert_not_reached(); + } } + return 0; +} + +int probe_access_flags(CPUArchState *env, target_ulong addr, + MMUAccessType access_type, int mmu_idx, + bool nonfault, void **phost, uintptr_t ra) +{ + int flags; + + flags =3D probe_access_internal(env, addr, 0, access_type, nonfault, r= a); + *phost =3D flags ? NULL : g2h(addr); + return flags; +} + +void *probe_access(CPUArchState *env, target_ulong addr, int size, + MMUAccessType access_type, int mmu_idx, uintptr_t ra) +{ + int flags; + + g_assert(-(addr | TARGET_PAGE_MASK) >=3D size); + flags =3D probe_access_internal(env, addr, size, access_type, false, r= a); + g_assert(flags =3D=3D 0); =20 return size ? g2h(addr) : NULL; } --=20 2.20.1