From nobody Mon Apr 29 10:20:02 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=linaro.org ARC-Seal: i=1; a=rsa-sha256; t=1568829965; cv=none; d=zoho.com; s=zohoarc; b=jw9lX9DRWtz4vKEhCApAdhkR0dT4CXGcO0kLiwtik0k9iRLqKUbiPo2dQKeQwEaj3ICkhIO/TsDKWyEYmi09v7R79YB8eg8ng8H7Fn4M5HHuEQPVHmzq+6yE6UlR4xE8l41bUIwSWeLroxx4es/nU4rBb8VSmHlgyH77kTiKHxk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1568829965; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=YucIarDiughcNjItqw4GzM0WuzRRGrmYnyWJbXHK1Xc=; b=f+3rWd1Yth9535sRV1xks545AXULlKInRr0BC1wb/RxdM18xKQhJn82D+NjQJ9GcDOj4b/V2RU4kbca0oeGwKw3n4T2wlbVRjzMMHCOzfQuBOtSIh6YzTZF5j5GTjYYIOQaJX0rd+dkZjISYDq6PikAT8/rIlCA5NlPL6kIiO/8= ARC-Authentication-Results: i=1; mx.zoho.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1568829965529604.6252966915808; Wed, 18 Sep 2019 11:06:05 -0700 (PDT) Received: from localhost ([::1]:33954 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iAeL0-0003da-8Q for importer@patchew.org; Wed, 18 Sep 2019 14:05:58 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45835) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iAeI5-0001y5-P3 for qemu-devel@nongnu.org; Wed, 18 Sep 2019 14:02:58 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iAeI4-0000RE-HV for qemu-devel@nongnu.org; Wed, 18 Sep 2019 14:02:57 -0400 Received: from mail-pf1-x436.google.com ([2607:f8b0:4864:20::436]:41085) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1iAeI4-0000QG-C4 for qemu-devel@nongnu.org; Wed, 18 Sep 2019 14:02:56 -0400 Received: by mail-pf1-x436.google.com with SMTP id q7so487625pfh.8 for ; Wed, 18 Sep 2019 11:02:56 -0700 (PDT) Received: from localhost.localdomain (97-113-7-119.tukw.qwest.net. [97.113.7.119]) by smtp.gmail.com with ESMTPSA id b16sm13319518pfb.54.2019.09.18.11.02.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 18 Sep 2019 11:02:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=YucIarDiughcNjItqw4GzM0WuzRRGrmYnyWJbXHK1Xc=; b=uFGlCSOWaQ9pQd3SvbCiO9C6FlvhlqZSoeFpvmgEgvp3iwl1XnrYNWgeu3zb5nUqd3 g2P5J2ghYZUwV2iS7KA3lfqAyOq3Vb/oDz7V7zb3BFWQ1du1w89Q73UzS8ZYRmrUJPdk VkAzEkZ9XcoWj0kmsHR+bNH3f9jxmA8jAftVbTBItI+2Ia+C9Qq6IYqrpxzHIfs7atiX 3n4c+MGZ6TWyd9BnYtbIkr5T1i1dfBvHjhWA8T43KTW8jP2AUKnmFFKi7aCRSNwh2/xq jt8xd9JMbz/WFG+97tLpYltfNdmfkVSARZqo1Or9dADhbKdSBM0BsbZ4uBm+f0nKQiVL tcow== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=YucIarDiughcNjItqw4GzM0WuzRRGrmYnyWJbXHK1Xc=; b=dPZOaePNTQuy1AyR2mvzCJahFgOJFKAYCnWmzxpRTOoa9MyjQaH/XvYQI59A7TLHJE jGP9vNHK5U2OJ/BZuni+U5NQlVpvxZ0Gh7Q0CDQRf8TKn8rzKAaY5QXUeR3Paeus5I8W Ep4IalCshhwKlXIw56mWZq8pCUm3oC91JLvAcrkNIgSxmGTVZFgqObE1gcgkd3PMpyYI zze8Uzm1urnVRtO5XeXTs58bFKCnHNB1ANGWHX8tPRebr4+JWX5zzS+BCJmXweYHyXU+ GCjfBiRcimNkXmOqj1vKzOG63rTp6g95+ll+7vjaDJ4OWtiQBJB/I0XAjk/1LSFpH/xj GHNg== X-Gm-Message-State: APjAAAX8m3hGz2lPSo//HS2VvGPeZ+rda0yXKFwWPhXrvmaJTtO7dH9F n4blPaAG6i1r+ZaII59/HwKzx+yZcEI= X-Google-Smtp-Source: APXvYqxjB8+UqTp0iZwFqovSRuG/3lZJcaCgJvnTQ3VSJlNBpkCkpqAFr1ZivhXzc7i/x0VPM8gBJQ== X-Received: by 2002:a63:cf0a:: with SMTP id j10mr5032746pgg.388.1568829774491; Wed, 18 Sep 2019 11:02:54 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Date: Wed, 18 Sep 2019 11:02:47 -0700 Message-Id: <20190918180251.32003-2-richard.henderson@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190918180251.32003-1-richard.henderson@linaro.org> References: <20190918180251.32003-1-richard.henderson@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::436 Subject: [Qemu-devel] [RFC v2 1/5] cputlb: Disable __always_inline__ without optimization X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: pbonzini@redhat.com, alex.bennee@linaro.org, stefanha@redhat.com, david@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" This forced inlining can result in missing symbols, which makes a debugging build harder to follow. Reported-by: Peter Maydell Signed-off-by: Richard Henderson Reviewed-by: David Hildenbrand --- include/qemu/compiler.h | 11 +++++++++++ accel/tcg/cputlb.c | 4 ++-- 2 files changed, 13 insertions(+), 2 deletions(-) diff --git a/include/qemu/compiler.h b/include/qemu/compiler.h index 09fc44cca4..d6d400c523 100644 --- a/include/qemu/compiler.h +++ b/include/qemu/compiler.h @@ -170,6 +170,17 @@ # define QEMU_NONSTRING #endif =20 +/* + * Forced inlining may be desired to encourage constant propagation + * of function parameters. However, it can also make debugging harder, + * so disable it for a non-optimizing build. + */ +#if defined(__OPTIMIZE__) && __has_attribute(always_inline) +#define QEMU_ALWAYS_INLINE __attribute__((always_inline)) +#else +#define QEMU_ALWAYS_INLINE +#endif + /* Implement C11 _Generic via GCC builtins. Example: * * QEMU_GENERIC(x, (float, sinf), (long double, sinl), sin) (x) diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index abae79650c..2222b87764 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -1281,7 +1281,7 @@ static void *atomic_mmu_lookup(CPUArchState *env, tar= get_ulong addr, typedef uint64_t FullLoadHelper(CPUArchState *env, target_ulong addr, TCGMemOpIdx oi, uintptr_t retaddr); =20 -static inline uint64_t __attribute__((always_inline)) +static inline uint64_t QEMU_ALWAYS_INLINE load_helper(CPUArchState *env, target_ulong addr, TCGMemOpIdx oi, uintptr_t retaddr, MemOp op, bool code_read, FullLoadHelper *full_load) @@ -1530,7 +1530,7 @@ tcg_target_ulong helper_be_ldsl_mmu(CPUArchState *env= , target_ulong addr, * Store Helpers */ =20 -static inline void __attribute__((always_inline)) +static inline void QEMU_ALWAYS_INLINE store_helper(CPUArchState *env, target_ulong addr, uint64_t val, TCGMemOpIdx oi, uintptr_t retaddr, MemOp op) { --=20 2.17.1 From nobody Mon Apr 29 10:20:02 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=linaro.org ARC-Seal: i=1; a=rsa-sha256; t=1568830175; cv=none; d=zoho.com; s=zohoarc; b=IXpGC7MRR9RyBDQMOffRCxTy6mdqUngAeD8H6Ghzov5JSkKqZvAuyb13ynd6d0f5KTCDIHrsG3AUB6A2/uJdVA1keZXFFHBmQm9f04WBrPiDVfieTQtPpdcnq53a6FjvVb1szaQ7oAcNU8RhsoRD9ZmB4kqbl8eOIhZ9eVPGmcw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1568830175; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=3cfrqeC7LyJB/ROXN1SmwKtiOawGwvTqMSrbdq/eHoM=; b=LXRxKAQfE81I4pA5BNlkxdMmV26ZTzJ05jHVyBhkSGXgdfEKVlr67HJfBth4a6JyIsPPlTX+u26r7wbDYXK3apXrRDOO8U1iala1Jydp5HpbBPznqmREimETZcVTsSSYwmXjuGoTtc6AdE17vwS5evFf42fAEhsodeJEhSukJUU= ARC-Authentication-Results: i=1; mx.zoho.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1568830175169350.62397095575227; Wed, 18 Sep 2019 11:09:35 -0700 (PDT) Received: from localhost ([::1]:34020 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iAeOT-0007DM-Mr for importer@patchew.org; Wed, 18 Sep 2019 14:09:33 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45852) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iAeI7-0001zg-Ox for qemu-devel@nongnu.org; Wed, 18 Sep 2019 14:03:02 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iAeI5-0000Rn-Up for qemu-devel@nongnu.org; Wed, 18 Sep 2019 14:02:59 -0400 Received: from mail-pf1-x442.google.com ([2607:f8b0:4864:20::442]:45460) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1iAeI5-0000RO-Lk for qemu-devel@nongnu.org; Wed, 18 Sep 2019 14:02:57 -0400 Received: by mail-pf1-x442.google.com with SMTP id y72so470757pfb.12 for ; Wed, 18 Sep 2019 11:02:57 -0700 (PDT) Received: from localhost.localdomain (97-113-7-119.tukw.qwest.net. [97.113.7.119]) by smtp.gmail.com with ESMTPSA id b16sm13319518pfb.54.2019.09.18.11.02.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 18 Sep 2019 11:02:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=3cfrqeC7LyJB/ROXN1SmwKtiOawGwvTqMSrbdq/eHoM=; b=rMDmqTEgtsk7zVSCslbphhy6+8evakesTNdLdxDMljXF9O24DEjuHvazZGHJLRMZSm Ss3pD5Sz3T2a9vHlpT/zytApIz1+f5/qnJY8CUVLBgsBV23oEetbrMmF1gp4mlkFpOiL pRCuN4NrTxYEu54HQrJp5CzUOzNiye+dxumMid5vUcw7PXDy3QPxyMaxlC459vU5xZMK YK/e1l8X1vGM8TD/hY/kzWEFaPbjf3Y52R0j1H9u7ZYb/p185nhNZljuVCUJim2YkxMO z9mVxTtvBq6AH+7ZlNlRcsjJ1Zn7qLZs0e2Ez88n/N8gXVjxabzJZIfCpyJMpz4Jy8VY T42A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=3cfrqeC7LyJB/ROXN1SmwKtiOawGwvTqMSrbdq/eHoM=; b=HpMIc7LiLSZDDw7FqHBiht1X/oDtQZPyPH78KLjHe7mY0GgPdDRIwTS+sVaUFuHw02 yHCsEhRvQ5WiUBQE0T4ch4tVuN4+a5EkKmbt0H8wfyHvLTLZPCqRWLQ+zr2Ml2eXn6b0 CZR7u3ZtDP1c6Y8DSzRCgC9Ho48EykuF1TCmRTpyidRbawXxv2HXHySSmrqgO03ZGC7Z 0E9xK90RIyNMARJFvEq4xcD2C9wBWG1h5Ppj8cGuFRHyNjcZAnipR3D2rEtAVPFCfm6V R2W6gsxDEj4d5+Yvy9UVQ/ObjGk8o+ycu9j8suuCchqNlR0XiBcvl372++spf726fln+ rRfg== X-Gm-Message-State: APjAAAXIGnQ+ZShuAAIyAn4ab1ndpcnzhQOfuq1tX2B0l1UIKktS/mq4 ndZiQ01QDsthzPN3g257S04LesYKS08= X-Google-Smtp-Source: APXvYqzsoQCugEJErcw3TD9yOUmLcITA53WIVuVhoFiHvqoOaHqDE72+qCXQg4UrRiS1S0ExEBPUsA== X-Received: by 2002:a63:2b0c:: with SMTP id r12mr5159576pgr.206.1568829776074; Wed, 18 Sep 2019 11:02:56 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Date: Wed, 18 Sep 2019 11:02:48 -0700 Message-Id: <20190918180251.32003-3-richard.henderson@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190918180251.32003-1-richard.henderson@linaro.org> References: <20190918180251.32003-1-richard.henderson@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::442 Subject: [Qemu-devel] [RFC v2 2/5] cputlb: Replace switches in load/store_helper with callback X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: pbonzini@redhat.com, alex.bennee@linaro.org, stefanha@redhat.com, david@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Add a function parameter to perform the actual load/store to ram. With optimization, this results in identical code. Signed-off-by: Richard Henderson --- accel/tcg/cputlb.c | 159 +++++++++++++++++++++++---------------------- 1 file changed, 83 insertions(+), 76 deletions(-) diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index 2222b87764..b4a63d3928 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -1280,11 +1280,38 @@ static void *atomic_mmu_lookup(CPUArchState *env, t= arget_ulong addr, =20 typedef uint64_t FullLoadHelper(CPUArchState *env, target_ulong addr, TCGMemOpIdx oi, uintptr_t retaddr); +typedef uint64_t LoadHelper(const void *); + +/* Wrap the unaligned load helpers to that they have a common signature. = */ +static inline uint64_t wrap_ldub(const void *haddr) +{ + return ldub_p(haddr); +} + +static inline uint64_t wrap_lduw_be(const void *haddr) +{ + return lduw_be_p(haddr); +} + +static inline uint64_t wrap_lduw_le(const void *haddr) +{ + return lduw_le_p(haddr); +} + +static inline uint64_t wrap_ldul_be(const void *haddr) +{ + return (uint32_t)ldl_be_p(haddr); +} + +static inline uint64_t wrap_ldul_le(const void *haddr) +{ + return (uint32_t)ldl_le_p(haddr); +} =20 static inline uint64_t QEMU_ALWAYS_INLINE load_helper(CPUArchState *env, target_ulong addr, TCGMemOpIdx oi, uintptr_t retaddr, MemOp op, bool code_read, - FullLoadHelper *full_load) + FullLoadHelper *full_load, LoadHelper *direct) { uintptr_t mmu_idx =3D get_mmuidx(oi); uintptr_t index =3D tlb_index(env, mmu_idx, addr); @@ -1373,33 +1400,7 @@ load_helper(CPUArchState *env, target_ulong addr, TC= GMemOpIdx oi, =20 do_aligned_access: haddr =3D (void *)((uintptr_t)addr + entry->addend); - switch (op) { - case MO_UB: - res =3D ldub_p(haddr); - break; - case MO_BEUW: - res =3D lduw_be_p(haddr); - break; - case MO_LEUW: - res =3D lduw_le_p(haddr); - break; - case MO_BEUL: - res =3D (uint32_t)ldl_be_p(haddr); - break; - case MO_LEUL: - res =3D (uint32_t)ldl_le_p(haddr); - break; - case MO_BEQ: - res =3D ldq_be_p(haddr); - break; - case MO_LEQ: - res =3D ldq_le_p(haddr); - break; - default: - g_assert_not_reached(); - } - - return res; + return direct(haddr); } =20 /* @@ -1415,7 +1416,8 @@ load_helper(CPUArchState *env, target_ulong addr, TCG= MemOpIdx oi, static uint64_t full_ldub_mmu(CPUArchState *env, target_ulong addr, TCGMemOpIdx oi, uintptr_t retaddr) { - return load_helper(env, addr, oi, retaddr, MO_UB, false, full_ldub_mmu= ); + return load_helper(env, addr, oi, retaddr, MO_UB, false, + full_ldub_mmu, wrap_ldub); } =20 tcg_target_ulong helper_ret_ldub_mmu(CPUArchState *env, target_ulong addr, @@ -1428,7 +1430,7 @@ static uint64_t full_le_lduw_mmu(CPUArchState *env, t= arget_ulong addr, TCGMemOpIdx oi, uintptr_t retaddr) { return load_helper(env, addr, oi, retaddr, MO_LEUW, false, - full_le_lduw_mmu); + full_le_lduw_mmu, wrap_lduw_le); } =20 tcg_target_ulong helper_le_lduw_mmu(CPUArchState *env, target_ulong addr, @@ -1441,7 +1443,7 @@ static uint64_t full_be_lduw_mmu(CPUArchState *env, t= arget_ulong addr, TCGMemOpIdx oi, uintptr_t retaddr) { return load_helper(env, addr, oi, retaddr, MO_BEUW, false, - full_be_lduw_mmu); + full_be_lduw_mmu, wrap_lduw_be); } =20 tcg_target_ulong helper_be_lduw_mmu(CPUArchState *env, target_ulong addr, @@ -1454,7 +1456,7 @@ static uint64_t full_le_ldul_mmu(CPUArchState *env, t= arget_ulong addr, TCGMemOpIdx oi, uintptr_t retaddr) { return load_helper(env, addr, oi, retaddr, MO_LEUL, false, - full_le_ldul_mmu); + full_le_ldul_mmu, wrap_ldul_le); } =20 tcg_target_ulong helper_le_ldul_mmu(CPUArchState *env, target_ulong addr, @@ -1467,7 +1469,7 @@ static uint64_t full_be_ldul_mmu(CPUArchState *env, t= arget_ulong addr, TCGMemOpIdx oi, uintptr_t retaddr) { return load_helper(env, addr, oi, retaddr, MO_BEUL, false, - full_be_ldul_mmu); + full_be_ldul_mmu, wrap_ldul_be); } =20 tcg_target_ulong helper_be_ldul_mmu(CPUArchState *env, target_ulong addr, @@ -1480,14 +1482,14 @@ uint64_t helper_le_ldq_mmu(CPUArchState *env, targe= t_ulong addr, TCGMemOpIdx oi, uintptr_t retaddr) { return load_helper(env, addr, oi, retaddr, MO_LEQ, false, - helper_le_ldq_mmu); + helper_le_ldq_mmu, ldq_le_p); } =20 uint64_t helper_be_ldq_mmu(CPUArchState *env, target_ulong addr, TCGMemOpIdx oi, uintptr_t retaddr) { return load_helper(env, addr, oi, retaddr, MO_BEQ, false, - helper_be_ldq_mmu); + helper_be_ldq_mmu, ldq_be_p); } =20 /* @@ -1530,9 +1532,38 @@ tcg_target_ulong helper_be_ldsl_mmu(CPUArchState *en= v, target_ulong addr, * Store Helpers */ =20 +typedef void StoreHelper(void *, uint64_t); + +/* Wrap the unaligned store helpers to that they have a common signature. = */ +static inline void wrap_stb(void *haddr, uint64_t val) +{ + stb_p(haddr, val); +} + +static inline void wrap_stw_be(void *haddr, uint64_t val) +{ + stw_be_p(haddr, val); +} + +static inline void wrap_stw_le(void *haddr, uint64_t val) +{ + stw_le_p(haddr, val); +} + +static inline void wrap_stl_be(void *haddr, uint64_t val) +{ + stl_be_p(haddr, val); +} + +static inline void wrap_stl_le(void *haddr, uint64_t val) +{ + stl_le_p(haddr, val); +} + static inline void QEMU_ALWAYS_INLINE store_helper(CPUArchState *env, target_ulong addr, uint64_t val, - TCGMemOpIdx oi, uintptr_t retaddr, MemOp op) + TCGMemOpIdx oi, uintptr_t retaddr, MemOp op, + StoreHelper *direct) { uintptr_t mmu_idx =3D get_mmuidx(oi); uintptr_t index =3D tlb_index(env, mmu_idx, addr); @@ -1657,74 +1688,49 @@ store_helper(CPUArchState *env, target_ulong addr, = uint64_t val, =20 do_aligned_access: haddr =3D (void *)((uintptr_t)addr + entry->addend); - switch (op) { - case MO_UB: - stb_p(haddr, val); - break; - case MO_BEUW: - stw_be_p(haddr, val); - break; - case MO_LEUW: - stw_le_p(haddr, val); - break; - case MO_BEUL: - stl_be_p(haddr, val); - break; - case MO_LEUL: - stl_le_p(haddr, val); - break; - case MO_BEQ: - stq_be_p(haddr, val); - break; - case MO_LEQ: - stq_le_p(haddr, val); - break; - default: - g_assert_not_reached(); - break; - } + direct(haddr, val); } =20 void helper_ret_stb_mmu(CPUArchState *env, target_ulong addr, uint8_t val, TCGMemOpIdx oi, uintptr_t retaddr) { - store_helper(env, addr, val, oi, retaddr, MO_UB); + store_helper(env, addr, val, oi, retaddr, MO_UB, wrap_stb); } =20 void helper_le_stw_mmu(CPUArchState *env, target_ulong addr, uint16_t val, TCGMemOpIdx oi, uintptr_t retaddr) { - store_helper(env, addr, val, oi, retaddr, MO_LEUW); + store_helper(env, addr, val, oi, retaddr, MO_LEUW, wrap_stw_le); } =20 void helper_be_stw_mmu(CPUArchState *env, target_ulong addr, uint16_t val, TCGMemOpIdx oi, uintptr_t retaddr) { - store_helper(env, addr, val, oi, retaddr, MO_BEUW); + store_helper(env, addr, val, oi, retaddr, MO_BEUW, wrap_stw_be); } =20 void helper_le_stl_mmu(CPUArchState *env, target_ulong addr, uint32_t val, TCGMemOpIdx oi, uintptr_t retaddr) { - store_helper(env, addr, val, oi, retaddr, MO_LEUL); + store_helper(env, addr, val, oi, retaddr, MO_LEUL, wrap_stl_le); } =20 void helper_be_stl_mmu(CPUArchState *env, target_ulong addr, uint32_t val, TCGMemOpIdx oi, uintptr_t retaddr) { - store_helper(env, addr, val, oi, retaddr, MO_BEUL); + store_helper(env, addr, val, oi, retaddr, MO_BEUL, wrap_stl_be); } =20 void helper_le_stq_mmu(CPUArchState *env, target_ulong addr, uint64_t val, TCGMemOpIdx oi, uintptr_t retaddr) { - store_helper(env, addr, val, oi, retaddr, MO_LEQ); + store_helper(env, addr, val, oi, retaddr, MO_LEQ, stq_le_p); } =20 void helper_be_stq_mmu(CPUArchState *env, target_ulong addr, uint64_t val, TCGMemOpIdx oi, uintptr_t retaddr) { - store_helper(env, addr, val, oi, retaddr, MO_BEQ); + store_helper(env, addr, val, oi, retaddr, MO_BEQ, stq_be_p); } =20 /* First set of helpers allows passing in of OI and RETADDR. This makes @@ -1789,7 +1795,8 @@ void helper_be_stq_mmu(CPUArchState *env, target_ulon= g addr, uint64_t val, static uint64_t full_ldub_cmmu(CPUArchState *env, target_ulong addr, TCGMemOpIdx oi, uintptr_t retaddr) { - return load_helper(env, addr, oi, retaddr, MO_8, true, full_ldub_cmmu); + return load_helper(env, addr, oi, retaddr, MO_8, true, + full_ldub_cmmu, wrap_ldub); } =20 uint8_t helper_ret_ldb_cmmu(CPUArchState *env, target_ulong addr, @@ -1802,7 +1809,7 @@ static uint64_t full_le_lduw_cmmu(CPUArchState *env, = target_ulong addr, TCGMemOpIdx oi, uintptr_t retaddr) { return load_helper(env, addr, oi, retaddr, MO_LEUW, true, - full_le_lduw_cmmu); + full_le_lduw_cmmu, wrap_lduw_le); } =20 uint16_t helper_le_ldw_cmmu(CPUArchState *env, target_ulong addr, @@ -1815,7 +1822,7 @@ static uint64_t full_be_lduw_cmmu(CPUArchState *env, = target_ulong addr, TCGMemOpIdx oi, uintptr_t retaddr) { return load_helper(env, addr, oi, retaddr, MO_BEUW, true, - full_be_lduw_cmmu); + full_be_lduw_cmmu, wrap_lduw_be); } =20 uint16_t helper_be_ldw_cmmu(CPUArchState *env, target_ulong addr, @@ -1828,7 +1835,7 @@ static uint64_t full_le_ldul_cmmu(CPUArchState *env, = target_ulong addr, TCGMemOpIdx oi, uintptr_t retaddr) { return load_helper(env, addr, oi, retaddr, MO_LEUL, true, - full_le_ldul_cmmu); + full_le_ldul_cmmu, wrap_ldul_le); } =20 uint32_t helper_le_ldl_cmmu(CPUArchState *env, target_ulong addr, @@ -1841,7 +1848,7 @@ static uint64_t full_be_ldul_cmmu(CPUArchState *env, = target_ulong addr, TCGMemOpIdx oi, uintptr_t retaddr) { return load_helper(env, addr, oi, retaddr, MO_BEUL, true, - full_be_ldul_cmmu); + full_be_ldul_cmmu, wrap_ldul_be); } =20 uint32_t helper_be_ldl_cmmu(CPUArchState *env, target_ulong addr, @@ -1854,12 +1861,12 @@ uint64_t helper_le_ldq_cmmu(CPUArchState *env, targ= et_ulong addr, TCGMemOpIdx oi, uintptr_t retaddr) { return load_helper(env, addr, oi, retaddr, MO_LEQ, true, - helper_le_ldq_cmmu); + helper_le_ldq_cmmu, ldq_le_p); } =20 uint64_t helper_be_ldq_cmmu(CPUArchState *env, target_ulong addr, TCGMemOpIdx oi, uintptr_t retaddr) { return load_helper(env, addr, oi, retaddr, MO_BEQ, true, - helper_be_ldq_cmmu); + helper_be_ldq_cmmu, ldq_be_p); } --=20 2.17.1 From nobody Mon Apr 29 10:20:02 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=linaro.org ARC-Seal: i=1; a=rsa-sha256; t=1568830442; cv=none; d=zoho.com; s=zohoarc; b=kmaQFj/3GQpoOiVbvj/VUG4Gnl5UAtXAnd0+qkLQRKizHI2SHhowNG+Qk0KSr0/aFMBSrVlDgSWbDdzGmNKWz3xokXKyLN4C26zDl8ulgmEevbFYax1n6P/YZjJkvDlSRHtENpZkS0/RpQJL/9sh1k+WzKlUxdIg/NxcikS6ISc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1568830442; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=6ZfbVwaLbCiGgpEw6rDTymD03FTM4V/YWYRYGaFUxL8=; b=X1R9N8ctkAToKH37anu7dhIYDR9r2K70x4ICOWrdv40IGUa3yFKNLVvn6bdpX1KovcxGNBEKXq9dP3inx+tZ2iv4uSdQ5qrnBLbFndgSSOpBJs/B1M9+BtqjnPimsGYC0ueCuevbb7DAudhf61bBzMud4xaCq8pmcaEkoId3H3c= ARC-Authentication-Results: i=1; mx.zoho.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1568830442803477.24694910595326; Wed, 18 Sep 2019 11:14:02 -0700 (PDT) Received: from localhost ([::1]:34104 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iAeSn-00024m-6w for importer@patchew.org; Wed, 18 Sep 2019 14:14:01 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45881) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iAeIC-00022R-MN for qemu-devel@nongnu.org; Wed, 18 Sep 2019 14:03:06 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iAeI7-0000SF-5T for qemu-devel@nongnu.org; Wed, 18 Sep 2019 14:03:02 -0400 Received: from mail-pg1-x542.google.com ([2607:f8b0:4864:20::542]:41358) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1iAeI6-0000S5-Ts for qemu-devel@nongnu.org; Wed, 18 Sep 2019 14:02:59 -0400 Received: by mail-pg1-x542.google.com with SMTP id x15so283375pgg.8 for ; Wed, 18 Sep 2019 11:02:58 -0700 (PDT) Received: from localhost.localdomain (97-113-7-119.tukw.qwest.net. [97.113.7.119]) by smtp.gmail.com with ESMTPSA id b16sm13319518pfb.54.2019.09.18.11.02.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 18 Sep 2019 11:02:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=6ZfbVwaLbCiGgpEw6rDTymD03FTM4V/YWYRYGaFUxL8=; b=xW6Pxapb4zhFSPriqaOFq5+JoB/3MzVaCDoku904vsV6YBnLN4Qv0r459FXGfSz74z +sj6VkGcrFNoipwbiVCwIeJiqrrpX/hd+bV8Mg0pLT5m6/s919OqRaItLUK4tFyqEIzj Q4lCorWDxtOsgikaC59FpsOD8lVAkgHirHReT3r8Otv21nQ8HIffZSqrUeNdX81/Gjsf +frDnS3OpQGYxoX3StqVLEDBa9C1aVBE6xLf9V92EP3VMXEjBnX4D2sTDUkDmQh0yjdf Go+AlUUBxseBbg+1c4bUBi/9xrU+Y4lNEhkr4AxksEeC042tqmEWn/8DA6FdFyAN4jKc TGQA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=6ZfbVwaLbCiGgpEw6rDTymD03FTM4V/YWYRYGaFUxL8=; b=T4VkzmvitVaEd+eUCX3MqUSNQ7IwWhgtzLwi2dV6P94yJ9f+SHXJrBry+C8a6kyHIq sY/1TAai8Al0ECSA+aCQaMaS9cBSAJDZv9Xq1WNfSNPsIBK7bqxecNdVcDO7BsAeVFmY 9PO35F6NfhiBwePXuQdWi4H5YgjM6+AdFcEnnLCH1Lzr1/FaB9FhbB9KJLkFJ16y6VNv ad24yZAURPzwyvkChxxanmQvmFMhqOG5S6tlNMmJdoIl3dgaeLdmDcENFXAOMxOStXoB FKtJnhMYZhSzhOuQlpIjR9yg/g8zBSK8VxNs4pZ0YdPiW6E562pw3JtkDveTbQPx/RVl lXKw== X-Gm-Message-State: APjAAAVCsebKkPb4ioSSbk4ZBtBhq6K+KOpZFM17hzv8A0Jna/VnGJ3/ VXyJnzbEPEkmxEXYi7SRzBrJM9d4My8= X-Google-Smtp-Source: APXvYqyCj1ZISnK93TdUFK72LfcW6pUr7T+8PFJV86fBe/XtHtZLcoeU9TYhJn4PpPEtzNnMSSHsMw== X-Received: by 2002:a63:eb52:: with SMTP id b18mr2716710pgk.71.1568829777336; Wed, 18 Sep 2019 11:02:57 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Date: Wed, 18 Sep 2019 11:02:49 -0700 Message-Id: <20190918180251.32003-4-richard.henderson@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190918180251.32003-1-richard.henderson@linaro.org> References: <20190918180251.32003-1-richard.henderson@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::542 Subject: [Qemu-devel] [RFC v2 3/5] cputlb: Introduce TLB_BSWAP X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: pbonzini@redhat.com, alex.bennee@linaro.org, stefanha@redhat.com, david@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Handle bswap on ram directly in load/store_helper. This fixes a bug with the previous implementation in that one cannot use the I/O path for RAM. Fixes: a26fc6f5152b47f1 Signed-off-by: Richard Henderson --- include/exec/cpu-all.h | 2 + accel/tcg/cputlb.c | 105 +++++++++++++++++++++-------------------- 2 files changed, 57 insertions(+), 50 deletions(-) diff --git a/include/exec/cpu-all.h b/include/exec/cpu-all.h index d2d443c4f9..3928edab9a 100644 --- a/include/exec/cpu-all.h +++ b/include/exec/cpu-all.h @@ -331,6 +331,8 @@ CPUArchState *cpu_copy(CPUArchState *env); #define TLB_MMIO (1 << (TARGET_PAGE_BITS - 3)) /* Set if TLB entry contains a watchpoint. */ #define TLB_WATCHPOINT (1 << (TARGET_PAGE_BITS - 4)) +/* Set if TLB entry requires byte swap. */ +#define TLB_BSWAP (1 << (TARGET_PAGE_BITS - 5)) =20 /* Use this mask to check interception with an alignment mask * in a TCG backend. diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index b4a63d3928..7301f9e699 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -737,8 +737,7 @@ void tlb_set_page_with_attrs(CPUState *cpu, target_ulon= g vaddr, address |=3D TLB_INVALID_MASK; } if (attrs.byte_swap) { - /* Force the access through the I/O slow path. */ - address |=3D TLB_MMIO; + address |=3D TLB_BSWAP; } if (!memory_region_is_ram(section->mr) && !memory_region_is_romd(section->mr)) { @@ -901,10 +900,6 @@ static uint64_t io_readx(CPUArchState *env, CPUIOTLBEn= try *iotlbentry, bool locked =3D false; MemTxResult r; =20 - if (iotlbentry->attrs.byte_swap) { - op ^=3D MO_BSWAP; - } - section =3D iotlb_to_section(cpu, iotlbentry->addr, iotlbentry->attrs); mr =3D section->mr; mr_offset =3D (iotlbentry->addr & TARGET_PAGE_MASK) + addr; @@ -947,10 +942,6 @@ static void io_writex(CPUArchState *env, CPUIOTLBEntry= *iotlbentry, bool locked =3D false; MemTxResult r; =20 - if (iotlbentry->attrs.byte_swap) { - op ^=3D MO_BSWAP; - } - section =3D iotlb_to_section(cpu, iotlbentry->addr, iotlbentry->attrs); mr =3D section->mr; mr_offset =3D (iotlbentry->addr & TARGET_PAGE_MASK) + addr; @@ -1311,7 +1302,8 @@ static inline uint64_t wrap_ldul_le(const void *haddr) static inline uint64_t QEMU_ALWAYS_INLINE load_helper(CPUArchState *env, target_ulong addr, TCGMemOpIdx oi, uintptr_t retaddr, MemOp op, bool code_read, - FullLoadHelper *full_load, LoadHelper *direct) + FullLoadHelper *full_load, LoadHelper *direct, + LoadHelper *direct_swap) { uintptr_t mmu_idx =3D get_mmuidx(oi); uintptr_t index =3D tlb_index(env, mmu_idx, addr); @@ -1361,17 +1353,22 @@ load_helper(CPUArchState *env, target_ulong addr, T= CGMemOpIdx oi, /* On watchpoint hit, this will longjmp out. */ cpu_check_watchpoint(env_cpu(env), addr, size, iotlbentry->attrs, BP_MEM_READ, retaddr); - - /* The backing page may or may not require I/O. */ - tlb_addr &=3D ~TLB_WATCHPOINT; - if ((tlb_addr & ~TARGET_PAGE_MASK) =3D=3D 0) { - goto do_aligned_access; - } } =20 /* Handle I/O access. */ - return io_readx(env, iotlbentry, mmu_idx, addr, - retaddr, access_type, op); + if (likely(tlb_addr & TLB_MMIO)) { + return io_readx(env, iotlbentry, mmu_idx, addr, + retaddr, access_type, + op ^ (tlb_addr & TLB_BSWAP ? MO_BSWAP : 0)); + } + + haddr =3D (void *)((uintptr_t)addr + entry->addend); + + if (unlikely(tlb_addr & TLB_BSWAP)) { + return direct_swap(haddr); + } else { + return direct(haddr); + } } =20 /* Handle slow unaligned access (it spans two pages or IO). */ @@ -1398,7 +1395,6 @@ load_helper(CPUArchState *env, target_ulong addr, TCG= MemOpIdx oi, return res & MAKE_64BIT_MASK(0, size * 8); } =20 - do_aligned_access: haddr =3D (void *)((uintptr_t)addr + entry->addend); return direct(haddr); } @@ -1417,7 +1413,7 @@ static uint64_t full_ldub_mmu(CPUArchState *env, targ= et_ulong addr, TCGMemOpIdx oi, uintptr_t retaddr) { return load_helper(env, addr, oi, retaddr, MO_UB, false, - full_ldub_mmu, wrap_ldub); + full_ldub_mmu, wrap_ldub, wrap_ldub); } =20 tcg_target_ulong helper_ret_ldub_mmu(CPUArchState *env, target_ulong addr, @@ -1430,7 +1426,7 @@ static uint64_t full_le_lduw_mmu(CPUArchState *env, t= arget_ulong addr, TCGMemOpIdx oi, uintptr_t retaddr) { return load_helper(env, addr, oi, retaddr, MO_LEUW, false, - full_le_lduw_mmu, wrap_lduw_le); + full_le_lduw_mmu, wrap_lduw_le, wrap_lduw_be); } =20 tcg_target_ulong helper_le_lduw_mmu(CPUArchState *env, target_ulong addr, @@ -1443,7 +1439,7 @@ static uint64_t full_be_lduw_mmu(CPUArchState *env, t= arget_ulong addr, TCGMemOpIdx oi, uintptr_t retaddr) { return load_helper(env, addr, oi, retaddr, MO_BEUW, false, - full_be_lduw_mmu, wrap_lduw_be); + full_be_lduw_mmu, wrap_lduw_be, wrap_lduw_le); } =20 tcg_target_ulong helper_be_lduw_mmu(CPUArchState *env, target_ulong addr, @@ -1456,7 +1452,7 @@ static uint64_t full_le_ldul_mmu(CPUArchState *env, t= arget_ulong addr, TCGMemOpIdx oi, uintptr_t retaddr) { return load_helper(env, addr, oi, retaddr, MO_LEUL, false, - full_le_ldul_mmu, wrap_ldul_le); + full_le_ldul_mmu, wrap_ldul_le, wrap_ldul_be); } =20 tcg_target_ulong helper_le_ldul_mmu(CPUArchState *env, target_ulong addr, @@ -1469,7 +1465,7 @@ static uint64_t full_be_ldul_mmu(CPUArchState *env, t= arget_ulong addr, TCGMemOpIdx oi, uintptr_t retaddr) { return load_helper(env, addr, oi, retaddr, MO_BEUL, false, - full_be_ldul_mmu, wrap_ldul_be); + full_be_ldul_mmu, wrap_ldul_be, wrap_ldul_le); } =20 tcg_target_ulong helper_be_ldul_mmu(CPUArchState *env, target_ulong addr, @@ -1482,14 +1478,14 @@ uint64_t helper_le_ldq_mmu(CPUArchState *env, targe= t_ulong addr, TCGMemOpIdx oi, uintptr_t retaddr) { return load_helper(env, addr, oi, retaddr, MO_LEQ, false, - helper_le_ldq_mmu, ldq_le_p); + helper_le_ldq_mmu, ldq_le_p, ldq_be_p); } =20 uint64_t helper_be_ldq_mmu(CPUArchState *env, target_ulong addr, TCGMemOpIdx oi, uintptr_t retaddr) { return load_helper(env, addr, oi, retaddr, MO_BEQ, false, - helper_be_ldq_mmu, ldq_be_p); + helper_be_ldq_mmu, ldq_be_p, ldq_le_p); } =20 /* @@ -1563,7 +1559,7 @@ static inline void wrap_stl_le(void *haddr, uint64_t = val) static inline void QEMU_ALWAYS_INLINE store_helper(CPUArchState *env, target_ulong addr, uint64_t val, TCGMemOpIdx oi, uintptr_t retaddr, MemOp op, - StoreHelper *direct) + StoreHelper *direct, StoreHelper *direct_swap) { uintptr_t mmu_idx =3D get_mmuidx(oi); uintptr_t index =3D tlb_index(env, mmu_idx, addr); @@ -1608,16 +1604,22 @@ store_helper(CPUArchState *env, target_ulong addr, = uint64_t val, /* On watchpoint hit, this will longjmp out. */ cpu_check_watchpoint(env_cpu(env), addr, size, iotlbentry->attrs, BP_MEM_WRITE, retaddr); - - /* The backing page may or may not require I/O. */ - tlb_addr &=3D ~TLB_WATCHPOINT; - if ((tlb_addr & ~TARGET_PAGE_MASK) =3D=3D 0) { - goto do_aligned_access; - } } =20 /* Handle I/O access. */ - io_writex(env, iotlbentry, mmu_idx, val, addr, retaddr, op); + if (likely(tlb_addr & (TLB_MMIO | TLB_NOTDIRTY))) { + io_writex(env, iotlbentry, mmu_idx, val, addr, retaddr, + op ^ (tlb_addr & TLB_BSWAP ? MO_BSWAP : 0)); + return; + } + + haddr =3D (void *)((uintptr_t)addr + entry->addend); + + if (unlikely(tlb_addr & TLB_BSWAP)) { + direct_swap(haddr, val); + } else { + direct(haddr, val); + } return; } =20 @@ -1686,7 +1688,6 @@ store_helper(CPUArchState *env, target_ulong addr, ui= nt64_t val, return; } =20 - do_aligned_access: haddr =3D (void *)((uintptr_t)addr + entry->addend); direct(haddr, val); } @@ -1694,43 +1695,47 @@ store_helper(CPUArchState *env, target_ulong addr, = uint64_t val, void helper_ret_stb_mmu(CPUArchState *env, target_ulong addr, uint8_t val, TCGMemOpIdx oi, uintptr_t retaddr) { - store_helper(env, addr, val, oi, retaddr, MO_UB, wrap_stb); + store_helper(env, addr, val, oi, retaddr, MO_UB, wrap_stb, wrap_stb); } =20 void helper_le_stw_mmu(CPUArchState *env, target_ulong addr, uint16_t val, TCGMemOpIdx oi, uintptr_t retaddr) { - store_helper(env, addr, val, oi, retaddr, MO_LEUW, wrap_stw_le); + store_helper(env, addr, val, oi, retaddr, MO_LEUW, + wrap_stw_le, wrap_stw_be); } =20 void helper_be_stw_mmu(CPUArchState *env, target_ulong addr, uint16_t val, TCGMemOpIdx oi, uintptr_t retaddr) { - store_helper(env, addr, val, oi, retaddr, MO_BEUW, wrap_stw_be); + store_helper(env, addr, val, oi, retaddr, MO_BEUW, + wrap_stw_be, wrap_stw_le); } =20 void helper_le_stl_mmu(CPUArchState *env, target_ulong addr, uint32_t val, TCGMemOpIdx oi, uintptr_t retaddr) { - store_helper(env, addr, val, oi, retaddr, MO_LEUL, wrap_stl_le); + store_helper(env, addr, val, oi, retaddr, MO_LEUL, + wrap_stl_le, wrap_stl_be); } =20 void helper_be_stl_mmu(CPUArchState *env, target_ulong addr, uint32_t val, TCGMemOpIdx oi, uintptr_t retaddr) { - store_helper(env, addr, val, oi, retaddr, MO_BEUL, wrap_stl_be); + store_helper(env, addr, val, oi, retaddr, MO_BEUL, + wrap_stl_be, wrap_stl_le); } =20 void helper_le_stq_mmu(CPUArchState *env, target_ulong addr, uint64_t val, TCGMemOpIdx oi, uintptr_t retaddr) { - store_helper(env, addr, val, oi, retaddr, MO_LEQ, stq_le_p); + store_helper(env, addr, val, oi, retaddr, MO_LEQ, stq_le_p, stq_be_p); } =20 void helper_be_stq_mmu(CPUArchState *env, target_ulong addr, uint64_t val, TCGMemOpIdx oi, uintptr_t retaddr) { - store_helper(env, addr, val, oi, retaddr, MO_BEQ, stq_be_p); + store_helper(env, addr, val, oi, retaddr, MO_BEQ, stq_be_p, stq_le_p); } =20 /* First set of helpers allows passing in of OI and RETADDR. This makes @@ -1796,7 +1801,7 @@ static uint64_t full_ldub_cmmu(CPUArchState *env, tar= get_ulong addr, TCGMemOpIdx oi, uintptr_t retaddr) { return load_helper(env, addr, oi, retaddr, MO_8, true, - full_ldub_cmmu, wrap_ldub); + full_ldub_cmmu, wrap_ldub, wrap_ldub); } =20 uint8_t helper_ret_ldb_cmmu(CPUArchState *env, target_ulong addr, @@ -1809,7 +1814,7 @@ static uint64_t full_le_lduw_cmmu(CPUArchState *env, = target_ulong addr, TCGMemOpIdx oi, uintptr_t retaddr) { return load_helper(env, addr, oi, retaddr, MO_LEUW, true, - full_le_lduw_cmmu, wrap_lduw_le); + full_le_lduw_cmmu, wrap_lduw_le, wrap_lduw_be); } =20 uint16_t helper_le_ldw_cmmu(CPUArchState *env, target_ulong addr, @@ -1822,7 +1827,7 @@ static uint64_t full_be_lduw_cmmu(CPUArchState *env, = target_ulong addr, TCGMemOpIdx oi, uintptr_t retaddr) { return load_helper(env, addr, oi, retaddr, MO_BEUW, true, - full_be_lduw_cmmu, wrap_lduw_be); + full_be_lduw_cmmu, wrap_lduw_be, wrap_lduw_le); } =20 uint16_t helper_be_ldw_cmmu(CPUArchState *env, target_ulong addr, @@ -1835,7 +1840,7 @@ static uint64_t full_le_ldul_cmmu(CPUArchState *env, = target_ulong addr, TCGMemOpIdx oi, uintptr_t retaddr) { return load_helper(env, addr, oi, retaddr, MO_LEUL, true, - full_le_ldul_cmmu, wrap_ldul_le); + full_le_ldul_cmmu, wrap_ldul_le, wrap_ldul_be); } =20 uint32_t helper_le_ldl_cmmu(CPUArchState *env, target_ulong addr, @@ -1848,7 +1853,7 @@ static uint64_t full_be_ldul_cmmu(CPUArchState *env, = target_ulong addr, TCGMemOpIdx oi, uintptr_t retaddr) { return load_helper(env, addr, oi, retaddr, MO_BEUL, true, - full_be_ldul_cmmu, wrap_ldul_be); + full_be_ldul_cmmu, wrap_ldul_be, wrap_ldul_le); } =20 uint32_t helper_be_ldl_cmmu(CPUArchState *env, target_ulong addr, @@ -1861,12 +1866,12 @@ uint64_t helper_le_ldq_cmmu(CPUArchState *env, targ= et_ulong addr, TCGMemOpIdx oi, uintptr_t retaddr) { return load_helper(env, addr, oi, retaddr, MO_LEQ, true, - helper_le_ldq_cmmu, ldq_le_p); + helper_le_ldq_cmmu, ldq_le_p, ldq_be_p); } =20 uint64_t helper_be_ldq_cmmu(CPUArchState *env, target_ulong addr, TCGMemOpIdx oi, uintptr_t retaddr) { return load_helper(env, addr, oi, retaddr, MO_BEQ, true, - helper_be_ldq_cmmu, ldq_be_p); + helper_be_ldq_cmmu, ldq_be_p, ldq_le_p); } --=20 2.17.1 From nobody Mon Apr 29 10:20:02 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=linaro.org ARC-Seal: i=1; a=rsa-sha256; t=1568830142; cv=none; d=zoho.com; s=zohoarc; b=bjlXLEG7nF3PXzMNaUKdA98K4OV5uC951pGMdQR3luRtMG7ScyQ7ty51XVcRQYUbD7T+6NWsD88ZL+pXstkvipsM+Xx0y9kJqmCTnABTEdqZG0zEg2yWvrszAS7wD3IJNkEpQt8s/X3imYY81S0kzDdRuc/sBUNQYgmOALvuUBk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1568830142; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=0As7gB6KKU9NL64qrnl7+jEEiK3R3QPZ3EzF2mVrRe8=; b=OCMSLMfvXWXvPwglNlH56O/UjA+nK7UMoK8Ya4wgmBZtMUSQdPkWg4gYCR5hsDkysf6f9aFDvYD8gC1v2oWdXANJ4MEwNTmQq5HQ2eJwe8QXu5EgAvuCjTrjX7Z8SUk5fPhvurDzex80fCfw7zPCVSF5tOolUgGvuqgjL6i/sDs= ARC-Authentication-Results: i=1; mx.zoho.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1568830142354445.99476543533603; Wed, 18 Sep 2019 11:09:02 -0700 (PDT) Received: from localhost ([::1]:34012 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iAeNw-0006iz-Ov for importer@patchew.org; Wed, 18 Sep 2019 14:09:00 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45886) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iAeIC-00022S-Rs for qemu-devel@nongnu.org; Wed, 18 Sep 2019 14:03:06 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iAeI8-0000T1-I9 for qemu-devel@nongnu.org; Wed, 18 Sep 2019 14:03:02 -0400 Received: from mail-pl1-x641.google.com ([2607:f8b0:4864:20::641]:44257) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1iAeI8-0000Sb-CL for qemu-devel@nongnu.org; Wed, 18 Sep 2019 14:03:00 -0400 Received: by mail-pl1-x641.google.com with SMTP id q15so303191pll.11 for ; Wed, 18 Sep 2019 11:03:00 -0700 (PDT) Received: from localhost.localdomain (97-113-7-119.tukw.qwest.net. [97.113.7.119]) by smtp.gmail.com with ESMTPSA id b16sm13319518pfb.54.2019.09.18.11.02.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 18 Sep 2019 11:02:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=0As7gB6KKU9NL64qrnl7+jEEiK3R3QPZ3EzF2mVrRe8=; b=AFAPrM0pPH8ESex+gy/AqU0FMEmGTp3q4xlC75z7dg/HBbHpV+jHURa5omqxuNmQqs mRT/8AH+kApx+llQ37YUijo87FqkandZlwIRLwhzB02az24ybzFMvXlajlSNE5qeTnQO KD/p7ibsGV27JHWa1xOT6z55LMYKCQb9e3r3dOWFeF24FcTBDZ38fPVGGgdhVjWnVv2c 0vmXXANGYR0Ks/j8uimqYidQLmG0QYdcsjn8JNxiLa/M60jyPp/5c7Cvg9OF+KnLVjN0 jixp1qc7bDuvT6EtrzqSTZhPM4ojvSr7kBVIzH89Fdax4Sg5Srkc3crK1+ZUogMCYios Ui2g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=0As7gB6KKU9NL64qrnl7+jEEiK3R3QPZ3EzF2mVrRe8=; b=thu64Nr3ifDU9irw9fa+RwGGKNGFKGt4pGe13uplO/LCLMWEe8VDFFGmmBHUohgGct Hl+q6zhmHmS9FGIRiAF0eydcUP0UhuKInTQT2WKUCQj0/t1Af6yOwIgxfkaH2CrBMzxD /+9ggt/f6Wv6XW7naJZsIqV5Wn43g1fbw1xoTY3vHzaJaP2gf3VZnDrP5KcGL+VE++2T AJjI2Ly3Gt1OalfzsRFRPTVlT9CKkkQlj2IfeQva9XUrIbGroCzJrO/DlyaB38Z+AhfL GPddKr6qDG6x324UM9YXl+CLmkDyND5G1/hzXLpfnkVstvwnB5zO0a9DYjH4Wv+SQ3in 14xQ== X-Gm-Message-State: APjAAAWDA2zWwMOY+imN8EPiuWtdyMWWctJaAq5BDCqLh96a7kbph+K1 +kZiJsTuYtb5f9qjPDQGZIqVVM9juB0= X-Google-Smtp-Source: APXvYqwGkcvzRFfL+7l5sIWLFjnqNZV14Bgg7OQ7ISw4p/Meq2vlAQlA6zMYs/6Rxjeo2e8MIeiJNQ== X-Received: by 2002:a17:902:aa43:: with SMTP id c3mr5290659plr.11.1568829778950; Wed, 18 Sep 2019 11:02:58 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Date: Wed, 18 Sep 2019 11:02:50 -0700 Message-Id: <20190918180251.32003-5-richard.henderson@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190918180251.32003-1-richard.henderson@linaro.org> References: <20190918180251.32003-1-richard.henderson@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::641 Subject: [Qemu-devel] [RFC v2 4/5] exec: Adjust notdirty tracing X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: pbonzini@redhat.com, alex.bennee@linaro.org, stefanha@redhat.com, david@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" The memory_region_tb_read tracepoint is unreachable, since notdirty is supposed to apply only to writes. The memory_region_tb_write tracepoint is mis-named, because notdirty is not only used for TB invalidation. It is also used for e.g. VGA RAM updates and migration. Replace memory_region_tb_write with memory_notdirty_write_access, and place it in memory_notdirty_write_prepare where it can catch all of the instances. Add memory_notdirty_set_dirty to log when we no longer intercept writes to a page. Signed-off-by: Richard Henderson Reviewed-by: David Hildenbrand --- v2: Rename new tracepoints as suggested (David) --- exec.c | 3 +++ memory.c | 4 ---- trace-events | 4 ++-- 3 files changed, 5 insertions(+), 6 deletions(-) diff --git a/exec.c b/exec.c index 8b998974f8..5f2587b621 100644 --- a/exec.c +++ b/exec.c @@ -2755,6 +2755,8 @@ void memory_notdirty_write_prepare(NotDirtyInfo *ndi, ndi->size =3D size; ndi->pages =3D NULL; =20 + trace_memory_notdirty_write_access(mem_vaddr, ram_addr, size); + assert(tcg_enabled()); if (!cpu_physical_memory_get_dirty_flag(ram_addr, DIRTY_MEMORY_CODE)) { ndi->pages =3D page_collection_lock(ram_addr, ram_addr + size); @@ -2779,6 +2781,7 @@ void memory_notdirty_write_complete(NotDirtyInfo *ndi) /* we remove the notdirty callback only if the code has been flushed */ if (!cpu_physical_memory_is_clean(ndi->ram_addr)) { + trace_memory_notdirty_set_dirty(ndi->mem_vaddr); tlb_set_dirty(ndi->cpu, ndi->mem_vaddr); } } diff --git a/memory.c b/memory.c index b9dd6b94ca..57c44c97db 100644 --- a/memory.c +++ b/memory.c @@ -438,7 +438,6 @@ static MemTxResult memory_region_read_accessor(MemoryR= egion *mr, /* Accesses to code which has previously been translated into a TB= show * up in the MMIO path, as accesses to the io_mem_notdirty * MemoryRegion. */ - trace_memory_region_tb_read(get_cpu_index(), addr, tmp, size); } else if (TRACE_MEMORY_REGION_OPS_READ_ENABLED) { hwaddr abs_addr =3D memory_region_to_absolute_addr(mr, addr); trace_memory_region_ops_read(get_cpu_index(), mr, abs_addr, tmp, s= ize); @@ -465,7 +464,6 @@ static MemTxResult memory_region_read_with_attrs_access= or(MemoryRegion *mr, /* Accesses to code which has previously been translated into a TB= show * up in the MMIO path, as accesses to the io_mem_notdirty * MemoryRegion. */ - trace_memory_region_tb_read(get_cpu_index(), addr, tmp, size); } else if (TRACE_MEMORY_REGION_OPS_READ_ENABLED) { hwaddr abs_addr =3D memory_region_to_absolute_addr(mr, addr); trace_memory_region_ops_read(get_cpu_index(), mr, abs_addr, tmp, s= ize); @@ -490,7 +488,6 @@ static MemTxResult memory_region_write_accessor(MemoryR= egion *mr, /* Accesses to code which has previously been translated into a TB= show * up in the MMIO path, as accesses to the io_mem_notdirty * MemoryRegion. */ - trace_memory_region_tb_write(get_cpu_index(), addr, tmp, size); } else if (TRACE_MEMORY_REGION_OPS_WRITE_ENABLED) { hwaddr abs_addr =3D memory_region_to_absolute_addr(mr, addr); trace_memory_region_ops_write(get_cpu_index(), mr, abs_addr, tmp, = size); @@ -515,7 +512,6 @@ static MemTxResult memory_region_write_with_attrs_acces= sor(MemoryRegion *mr, /* Accesses to code which has previously been translated into a TB= show * up in the MMIO path, as accesses to the io_mem_notdirty * MemoryRegion. */ - trace_memory_region_tb_write(get_cpu_index(), addr, tmp, size); } else if (TRACE_MEMORY_REGION_OPS_WRITE_ENABLED) { hwaddr abs_addr =3D memory_region_to_absolute_addr(mr, addr); trace_memory_region_ops_write(get_cpu_index(), mr, abs_addr, tmp, = size); diff --git a/trace-events b/trace-events index 823a4ae64e..20821ba545 100644 --- a/trace-events +++ b/trace-events @@ -52,14 +52,14 @@ dma_map_wait(void *dbs) "dbs=3D%p" find_ram_offset(uint64_t size, uint64_t offset) "size: 0x%" PRIx64 " @ 0x%= " PRIx64 find_ram_offset_loop(uint64_t size, uint64_t candidate, uint64_t offset, u= int64_t next, uint64_t mingap) "trying size: 0x%" PRIx64 " @ 0x%" PRIx64 ",= offset: 0x%" PRIx64" next: 0x%" PRIx64 " mingap: 0x%" PRIx64 ram_block_discard_range(const char *rbname, void *hva, size_t length, bool= need_madvise, bool need_fallocate, int ret) "%s@%p + 0x%zx: madvise: %d fa= llocate: %d ret: %d" +memory_notdirty_write_access(uint64_t vaddr, uint64_t ram_addr, unsigned s= ize) "0x%" PRIx64 " ram_addr 0x%" PRIx64 " size %u" +memory_notdirty_set_dirty(uint64_t vaddr) "0x%" PRIx64 =20 # memory.c memory_region_ops_read(int cpu_index, void *mr, uint64_t addr, uint64_t va= lue, unsigned size) "cpu %d mr %p addr 0x%"PRIx64" value 0x%"PRIx64" size %= u" memory_region_ops_write(int cpu_index, void *mr, uint64_t addr, uint64_t v= alue, unsigned size) "cpu %d mr %p addr 0x%"PRIx64" value 0x%"PRIx64" size = %u" memory_region_subpage_read(int cpu_index, void *mr, uint64_t offset, uint6= 4_t value, unsigned size) "cpu %d mr %p offset 0x%"PRIx64" value 0x%"PRIx64= " size %u" memory_region_subpage_write(int cpu_index, void *mr, uint64_t offset, uint= 64_t value, unsigned size) "cpu %d mr %p offset 0x%"PRIx64" value 0x%"PRIx6= 4" size %u" -memory_region_tb_read(int cpu_index, uint64_t addr, uint64_t value, unsign= ed size) "cpu %d addr 0x%"PRIx64" value 0x%"PRIx64" size %u" -memory_region_tb_write(int cpu_index, uint64_t addr, uint64_t value, unsig= ned size) "cpu %d addr 0x%"PRIx64" value 0x%"PRIx64" size %u" memory_region_ram_device_read(int cpu_index, void *mr, uint64_t addr, uint= 64_t value, unsigned size) "cpu %d mr %p addr 0x%"PRIx64" value 0x%"PRIx64"= size %u" memory_region_ram_device_write(int cpu_index, void *mr, uint64_t addr, uin= t64_t value, unsigned size) "cpu %d mr %p addr 0x%"PRIx64" value 0x%"PRIx64= " size %u" flatview_new(void *view, void *root) "%p (root %p)" --=20 2.17.1 From nobody Mon Apr 29 10:20:02 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=linaro.org ARC-Seal: i=1; a=rsa-sha256; t=1568830338; cv=none; d=zoho.com; s=zohoarc; b=ZiYHSyX+AV0AkAwYwy6FeLXtpcX3Hx6ntnSDR8KB+n5DCd0Q5nEM+8ppkNvrghDGLkRAeGyFeYvjNALtZV8U8NSiVw1cfF2HZ8ldWOdyGWl31VCLLnxc21Yo6wU3ugFen4j6CeC/BjQsJOAPSIDarMCP5FJFlYpaJ4RGOJkGjfI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1568830338; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=XLbwjWXO2o10cL5F/+EEPP36UeeSrQvwCcEMZEHp8Do=; b=cuPh5ahapIS6QBss+HTgZjPuwpUb/tH+28AGMQi6DEfErJzKPLtvmGp6ieLCN80bUxGMVt+0Q33w3uR2ZG/ky3Rum7CRREvDuUn2i6s+vsLxWCiN7/EQ2DPocCM0TX8T/FVw7E/kfDP+CXTdiOPXEp41GooXef8oW0++54NIubk= ARC-Authentication-Results: i=1; mx.zoho.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1568830338233780.2735029251244; Wed, 18 Sep 2019 11:12:18 -0700 (PDT) Received: from localhost ([::1]:34098 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iAeR3-0001Xq-PB for importer@patchew.org; Wed, 18 Sep 2019 14:12:13 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45914) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iAeIG-000240-Va for qemu-devel@nongnu.org; Wed, 18 Sep 2019 14:03:10 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iAeIE-0000Ui-Ct for qemu-devel@nongnu.org; Wed, 18 Sep 2019 14:03:08 -0400 Received: from mail-pl1-x644.google.com ([2607:f8b0:4864:20::644]:43955) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1iAeIA-0000TB-Ut for qemu-devel@nongnu.org; Wed, 18 Sep 2019 14:03:04 -0400 Received: by mail-pl1-x644.google.com with SMTP id 4so308112pld.10 for ; Wed, 18 Sep 2019 11:03:01 -0700 (PDT) Received: from localhost.localdomain (97-113-7-119.tukw.qwest.net. [97.113.7.119]) by smtp.gmail.com with ESMTPSA id b16sm13319518pfb.54.2019.09.18.11.02.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 18 Sep 2019 11:02:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=XLbwjWXO2o10cL5F/+EEPP36UeeSrQvwCcEMZEHp8Do=; b=nTU/ED6P7tx9oR8mjY6GuoWqAtdeLxB+hvbHKfkaBosbes8Q2JsZ9P27HwAzIOUB9P c5oC+NZDJn1QnwOkApyWRoX8lVkH5PW7Iggct9LXEah6C+V6UYHTt9/TUR9PY8HwAoTS RDzkr2aZDNsp3a1YjLSGDcMi5G8LsSi5qmdFG100qe0Q0o/tL8GBt3B5hXqeOcRRP7nx HAJdLVQUFnGKbIxOyTp7M3JVTVkbWjXrbqOJ5E3B6ctiKZQ0ZSu6Jezb6cS2lVCQ0fnT qM2s5QPtIlCDZtnVf71qmk/oTQTjV80HdfidOyRYjBkgB3mV/YcAJzlRJBVhWmOqadRg uwfw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=XLbwjWXO2o10cL5F/+EEPP36UeeSrQvwCcEMZEHp8Do=; b=WVKKa7fXaLLom6QGYcpEsHga2HswrjL4C6I+Efn8LhWU0kh+yVp0G5VsF8meQXRT0E Ri5ZERlodtC0UGHf6CE9cjrGPXgVnWNeMWRmhhQ6hICRidLlPzeE3PIKuN7mO8T0fRsQ qhUY6iHCy8YfvuW8CcZuCJcz1tzl/UL52FqNpYbETjXoqf1M7CI/dXb0NFX/+OmAP0/p uXCsF0vFryqjsBbGxCuGzmW3qViqpvbkTPMDOXH+/unXDuS2YSrBdiyQL+zSGBbHxyOM lKokD38OgS/NDS7Ivw9mK+Y0DsBMsvaLz/e4BEfZG5CWRG4t998s4CK02ZpJaileSFnW 6rww== X-Gm-Message-State: APjAAAU4SF9OnaJ88q+AnSPvZHccTKj/OJuL/GpAmauBVtZGQXpBif8n h4tfhlRChgqIy0ytB2sYmZFnnSww6v8= X-Google-Smtp-Source: APXvYqzFpuHokLmlPmCjcVcGyxXxURMwvXngdcmMAohomqXq7tY/eDteA3cEmzCzc+LRRunsQegtiw== X-Received: by 2002:a17:902:8bca:: with SMTP id r10mr5407494plo.338.1568829780263; Wed, 18 Sep 2019 11:03:00 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Date: Wed, 18 Sep 2019 11:02:51 -0700 Message-Id: <20190918180251.32003-6-richard.henderson@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190918180251.32003-1-richard.henderson@linaro.org> References: <20190918180251.32003-1-richard.henderson@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::644 Subject: [Qemu-devel] [RFC v2 5/5] cputlb: Move NOTDIRTY handling from I/O path to TLB path X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: pbonzini@redhat.com, alex.bennee@linaro.org, stefanha@redhat.com, david@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Pages that we want to track for NOTDIRTY are RAM. We do not really need to go through the I/O path to handle them. Signed-off-by: Richard Henderson --- include/exec/cpu-common.h | 1 - accel/tcg/cputlb.c | 23 ++++++++++++++--- exec.c | 54 +++------------------------------------ memory.c | 16 ------------ 4 files changed, 23 insertions(+), 71 deletions(-) diff --git a/include/exec/cpu-common.h b/include/exec/cpu-common.h index f7dbe75fbc..06c60c82be 100644 --- a/include/exec/cpu-common.h +++ b/include/exec/cpu-common.h @@ -101,7 +101,6 @@ void qemu_flush_coalesced_mmio_buffer(void); void cpu_flush_icache_range(hwaddr start, hwaddr len); =20 extern struct MemoryRegion io_mem_rom; -extern struct MemoryRegion io_mem_notdirty; =20 typedef int (RAMBlockIterFunc)(RAMBlock *rb, void *opaque); =20 diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index 7301f9e699..c6c66b40bb 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -904,7 +904,7 @@ static uint64_t io_readx(CPUArchState *env, CPUIOTLBEnt= ry *iotlbentry, mr =3D section->mr; mr_offset =3D (iotlbentry->addr & TARGET_PAGE_MASK) + addr; cpu->mem_io_pc =3D retaddr; - if (mr !=3D &io_mem_rom && mr !=3D &io_mem_notdirty && !cpu->can_do_io= ) { + if (mr !=3D &io_mem_rom && !cpu->can_do_io) { cpu_io_recompile(cpu, retaddr); } =20 @@ -945,7 +945,7 @@ static void io_writex(CPUArchState *env, CPUIOTLBEntry = *iotlbentry, section =3D iotlb_to_section(cpu, iotlbentry->addr, iotlbentry->attrs); mr =3D section->mr; mr_offset =3D (iotlbentry->addr & TARGET_PAGE_MASK) + addr; - if (mr !=3D &io_mem_rom && mr !=3D &io_mem_notdirty && !cpu->can_do_io= ) { + if (mr !=3D &io_mem_rom && !cpu->can_do_io) { cpu_io_recompile(cpu, retaddr); } cpu->mem_io_vaddr =3D addr; @@ -1607,7 +1607,7 @@ store_helper(CPUArchState *env, target_ulong addr, ui= nt64_t val, } =20 /* Handle I/O access. */ - if (likely(tlb_addr & (TLB_MMIO | TLB_NOTDIRTY))) { + if (tlb_addr & TLB_MMIO) { io_writex(env, iotlbentry, mmu_idx, val, addr, retaddr, op ^ (tlb_addr & TLB_BSWAP ? MO_BSWAP : 0)); return; @@ -1615,6 +1615,23 @@ store_helper(CPUArchState *env, target_ulong addr, u= int64_t val, =20 haddr =3D (void *)((uintptr_t)addr + entry->addend); =20 + /* Handle clean RAM pages. */ + if (tlb_addr & TLB_NOTDIRTY) { + NotDirtyInfo ndi; + + memory_notdirty_write_prepare(&ndi, env_cpu(env), addr, + addr + iotlbentry->addr, size); + + if (unlikely(tlb_addr & TLB_BSWAP)) { + direct_swap(haddr, val); + } else { + direct(haddr, val); + } + + memory_notdirty_write_complete(&ndi); + return; + } + if (unlikely(tlb_addr & TLB_BSWAP)) { direct_swap(haddr, val); } else { diff --git a/exec.c b/exec.c index 5f2587b621..4600292ee2 100644 --- a/exec.c +++ b/exec.c @@ -88,7 +88,7 @@ static MemoryRegion *system_io; AddressSpace address_space_io; AddressSpace address_space_memory; =20 -MemoryRegion io_mem_rom, io_mem_notdirty; +MemoryRegion io_mem_rom; static MemoryRegion io_mem_unassigned; #endif =20 @@ -191,8 +191,7 @@ typedef struct subpage_t { } subpage_t; =20 #define PHYS_SECTION_UNASSIGNED 0 -#define PHYS_SECTION_NOTDIRTY 1 -#define PHYS_SECTION_ROM 2 +#define PHYS_SECTION_ROM 1 =20 static void io_mem_init(void); static void memory_map_init(void); @@ -1473,9 +1472,7 @@ hwaddr memory_region_section_get_iotlb(CPUState *cpu, if (memory_region_is_ram(section->mr)) { /* Normal RAM. */ iotlb =3D memory_region_get_ram_addr(section->mr) + xlat; - if (!section->readonly) { - iotlb |=3D PHYS_SECTION_NOTDIRTY; - } else { + if (section->readonly) { iotlb |=3D PHYS_SECTION_ROM; } } else { @@ -2786,42 +2783,6 @@ void memory_notdirty_write_complete(NotDirtyInfo *nd= i) } } =20 -/* Called within RCU critical section. */ -static void notdirty_mem_write(void *opaque, hwaddr ram_addr, - uint64_t val, unsigned size) -{ - NotDirtyInfo ndi; - - memory_notdirty_write_prepare(&ndi, current_cpu, current_cpu->mem_io_v= addr, - ram_addr, size); - - stn_p(qemu_map_ram_ptr(NULL, ram_addr), size, val); - memory_notdirty_write_complete(&ndi); -} - -static bool notdirty_mem_accepts(void *opaque, hwaddr addr, - unsigned size, bool is_write, - MemTxAttrs attrs) -{ - return is_write; -} - -static const MemoryRegionOps notdirty_mem_ops =3D { - .write =3D notdirty_mem_write, - .valid.accepts =3D notdirty_mem_accepts, - .endianness =3D DEVICE_NATIVE_ENDIAN, - .valid =3D { - .min_access_size =3D 1, - .max_access_size =3D 8, - .unaligned =3D false, - }, - .impl =3D { - .min_access_size =3D 1, - .max_access_size =3D 8, - .unaligned =3D false, - }, -}; - /* Generate a debug exception if a watchpoint has been hit. */ void cpu_check_watchpoint(CPUState *cpu, vaddr addr, vaddr len, MemTxAttrs attrs, int flags, uintptr_t ra) @@ -3051,13 +3012,6 @@ static void io_mem_init(void) NULL, NULL, UINT64_MAX); memory_region_init_io(&io_mem_unassigned, NULL, &unassigned_mem_ops, N= ULL, NULL, UINT64_MAX); - - /* io_mem_notdirty calls tb_invalidate_phys_page_fast, - * which can be called without the iothread mutex. - */ - memory_region_init_io(&io_mem_notdirty, NULL, ¬dirty_mem_ops, NULL, - NULL, UINT64_MAX); - memory_region_clear_global_locking(&io_mem_notdirty); } =20 AddressSpaceDispatch *address_space_dispatch_new(FlatView *fv) @@ -3067,8 +3021,6 @@ AddressSpaceDispatch *address_space_dispatch_new(Flat= View *fv) =20 n =3D dummy_section(&d->map, fv, &io_mem_unassigned); assert(n =3D=3D PHYS_SECTION_UNASSIGNED); - n =3D dummy_section(&d->map, fv, &io_mem_notdirty); - assert(n =3D=3D PHYS_SECTION_NOTDIRTY); n =3D dummy_section(&d->map, fv, &io_mem_rom); assert(n =3D=3D PHYS_SECTION_ROM); =20 diff --git a/memory.c b/memory.c index 57c44c97db..a99b8c0767 100644 --- a/memory.c +++ b/memory.c @@ -434,10 +434,6 @@ static MemTxResult memory_region_read_accessor(Memory= Region *mr, tmp =3D mr->ops->read(mr->opaque, addr, size); if (mr->subpage) { trace_memory_region_subpage_read(get_cpu_index(), mr, addr, tmp, s= ize); - } else if (mr =3D=3D &io_mem_notdirty) { - /* Accesses to code which has previously been translated into a TB= show - * up in the MMIO path, as accesses to the io_mem_notdirty - * MemoryRegion. */ } else if (TRACE_MEMORY_REGION_OPS_READ_ENABLED) { hwaddr abs_addr =3D memory_region_to_absolute_addr(mr, addr); trace_memory_region_ops_read(get_cpu_index(), mr, abs_addr, tmp, s= ize); @@ -460,10 +456,6 @@ static MemTxResult memory_region_read_with_attrs_acces= sor(MemoryRegion *mr, r =3D mr->ops->read_with_attrs(mr->opaque, addr, &tmp, size, attrs); if (mr->subpage) { trace_memory_region_subpage_read(get_cpu_index(), mr, addr, tmp, s= ize); - } else if (mr =3D=3D &io_mem_notdirty) { - /* Accesses to code which has previously been translated into a TB= show - * up in the MMIO path, as accesses to the io_mem_notdirty - * MemoryRegion. */ } else if (TRACE_MEMORY_REGION_OPS_READ_ENABLED) { hwaddr abs_addr =3D memory_region_to_absolute_addr(mr, addr); trace_memory_region_ops_read(get_cpu_index(), mr, abs_addr, tmp, s= ize); @@ -484,10 +476,6 @@ static MemTxResult memory_region_write_accessor(Memory= Region *mr, =20 if (mr->subpage) { trace_memory_region_subpage_write(get_cpu_index(), mr, addr, tmp, = size); - } else if (mr =3D=3D &io_mem_notdirty) { - /* Accesses to code which has previously been translated into a TB= show - * up in the MMIO path, as accesses to the io_mem_notdirty - * MemoryRegion. */ } else if (TRACE_MEMORY_REGION_OPS_WRITE_ENABLED) { hwaddr abs_addr =3D memory_region_to_absolute_addr(mr, addr); trace_memory_region_ops_write(get_cpu_index(), mr, abs_addr, tmp, = size); @@ -508,10 +496,6 @@ static MemTxResult memory_region_write_with_attrs_acce= ssor(MemoryRegion *mr, =20 if (mr->subpage) { trace_memory_region_subpage_write(get_cpu_index(), mr, addr, tmp, = size); - } else if (mr =3D=3D &io_mem_notdirty) { - /* Accesses to code which has previously been translated into a TB= show - * up in the MMIO path, as accesses to the io_mem_notdirty - * MemoryRegion. */ } else if (TRACE_MEMORY_REGION_OPS_WRITE_ENABLED) { hwaddr abs_addr =3D memory_region_to_absolute_addr(mr, addr); trace_memory_region_ops_write(get_cpu_index(), mr, abs_addr, tmp, = size); --=20 2.17.1