From nobody Tue Feb 10 10:20:42 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=linaro.org ARC-Seal: i=1; a=rsa-sha256; t=1666305992; cv=none; d=zohomail.com; s=zohoarc; b=DZfM5XLaQaTZqYIqoabyFEhrVKQeS/31Bfgw0mS8DgbDPdHcI4mFTiWVs4PRk7QbDE7avdUPoz4RJj+3lI92+6ec/cvlY5Jd5TIzokB93a7KnkTcJ6T5fz84b7UO3PWeitpgN/+yjl7df0IvTHpmOTJ1N3dqWY3uWnDzEtOBoRU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1666305992; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=DfngjCYW3yhM7GTZnvGecJb6bmu3XmLkK+TAKSRW4rQ=; b=EuL0abpzJflnPYAiSJKwaxi5iHFTC5sqZoUeypLdOwqQ96kxe1+zCH9WqASKNxa0LPdqI01r97J9BLK3xNRPWpykd+kUm5yvGeLK2JZkWv2xY/vlgjKRtDez6/blUMwyufhx3quVasuOw4Ht1HBUAbvb4tE2IhK12h6k47gD4JI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1666305992333159.58331403915497; Thu, 20 Oct 2022 15:46:32 -0700 (PDT) Received: from localhost ([::1] helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oleJ5-0003In-4t for importer@patchew.org; Thu, 20 Oct 2022 18:46:31 -0400 Received: from [::1] (helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oleA9-0008LF-Ra for importer@patchew.org; Thu, 20 Oct 2022 18:37:17 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oleA3-0007xZ-AB for qemu-devel@nongnu.org; Thu, 20 Oct 2022 18:37:11 -0400 Received: from mail-pj1-x102c.google.com ([2607:f8b0:4864:20::102c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1ole9z-00016a-CD for qemu-devel@nongnu.org; Thu, 20 Oct 2022 18:37:11 -0400 Received: by mail-pj1-x102c.google.com with SMTP id pb15so734653pjb.5 for ; Thu, 20 Oct 2022 15:37:06 -0700 (PDT) Received: from localhost.localdomain ([149.135.10.35]) by smtp.gmail.com with ESMTPSA id r10-20020aa7988a000000b0054cd16c9f6bsm13753841pfl.200.2022.10.20.15.37.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 20 Oct 2022 15:37:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=DfngjCYW3yhM7GTZnvGecJb6bmu3XmLkK+TAKSRW4rQ=; b=Oowj5dC8FvOA10U/QDzHzPHfRLms3fjyFXA5oAsJXCys5YHx7PDyGBErl5y0tcE0Qs BrVV+pSje8pQPQhWM+zyU8HZvxQe01iFRJ2p8Z7tYrKYFqbGoLpRNceu3DWlyfUuAK7S bHRqjWFnTPSkyT7jQ7vPKkCXxMDaX39C9iTGG2OGTuLewEKJDv7kcn3bvB2APglZAXJ7 Dmm+O9xJNy88uWVW1OadohKFEL2BXYK0sTmr7wstEo/qAWBmJrJQjr4Ku8gCAL9FqGde H4YXo6kNqsKyDUzpkhWMG+kAqXsVHvY/zdOEM6b/NJY1cKsaCUbpWOXPlsP2g6nuabfG 7Z8w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=DfngjCYW3yhM7GTZnvGecJb6bmu3XmLkK+TAKSRW4rQ=; b=S6auWlqe/7cgoXVxhh2uT38KPfo1xdK1pJ8gIXfbWswtJVZ/5wbCJ0BWPUgy+hnQ4e nkim1Zg3EZQNP3LaTPaNvIEw0KK8XbleuqS0cupWPK908UcQEmni4jquZF+l0kRe9u5Q dGiTuH/r0xV1FHcM3AoUwoxVYj7XgY9grI3FaEdSNN1CGxh5r2Dv0JJ0/tFmE+dZNeQ4 s6w2ilYSWieb5GStVtzAce9ADNg/xFjkI9k34Ufx6mdu5g/3b8v3ac73D+54hxC0+djZ laCqEUUkmKqHnpaG3LrflpmA6vMldglki1ZBG2ac8Uvo1u5plcwFQhiY6eY9ds9tWwz9 w02w== X-Gm-Message-State: ACrzQf2rK5bwzDqZmk3Sx04PKtLBP5Xdqt3zdWHhOHskDybsUFECYLcD CqNAFp0+FHoYim1WQwCn1pUlH6UYFHOhvByI X-Google-Smtp-Source: AMsMyM4jhBgGj2JA4jufNezut8qWb/tZ4SqSKzXFIMgzOU5p/pSlKSzE0CcyivnuPbn649q0eiPdnQ== X-Received: by 2002:a17:90b:4d90:b0:20d:9ded:4144 with SMTP id oj16-20020a17090b4d9000b0020d9ded4144mr18364403pjb.90.1666305425967; Thu, 20 Oct 2022 15:37:05 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: qemu-arm@nongnu.org, Peter Maydell Subject: [PATCH v5 12/14] target/arm: Implement FEAT_HAFDBS, access flag portion Date: Fri, 21 Oct 2022 08:35:46 +1000 Message-Id: <20221020223548.2310496-13-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221020223548.2310496-1-richard.henderson@linaro.org> References: <20221020223548.2310496-1-richard.henderson@linaro.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::102c; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x102c.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @linaro.org) X-ZM-MESSAGEID: 1666305994120100003 Content-Type: text/plain; charset="utf-8" Perform the atomic update for hardware management of the access flag. Reviewed-by: Peter Maydell Signed-off-by: Richard Henderson --- v4: Raise permission fault if pte read-only and atomic update reqd. Split out dirty bit portion. Prepare for a single update for AF + DB. v5: Fix s1ns typo; incorporate i_yzsvv comment. Move the AF update before attributes are extracted and merged. Do permission check before pte writeback, per AArch64.S1Translate. --- docs/system/arm/emulation.rst | 1 + target/arm/cpu64.c | 1 + target/arm/ptw.c | 176 +++++++++++++++++++++++++++++----- 3 files changed, 156 insertions(+), 22 deletions(-) diff --git a/docs/system/arm/emulation.rst b/docs/system/arm/emulation.rst index cfb4b0768b..580e67b190 100644 --- a/docs/system/arm/emulation.rst +++ b/docs/system/arm/emulation.rst @@ -32,6 +32,7 @@ the following architecture extensions: - FEAT_FlagM (Flag manipulation instructions v2) - FEAT_FlagM2 (Enhancements to flag manipulation instructions) - FEAT_GTG (Guest translation granule size) +- FEAT_HAFDBS (Hardware management of the access flag and dirty bit state) - FEAT_HCX (Support for the HCRX_EL2 register) - FEAT_HPDS (Hierarchical permission disables) - FEAT_I8MM (AArch64 Int8 matrix multiplication instructions) diff --git a/target/arm/cpu64.c b/target/arm/cpu64.c index 85e0d1daf1..fe1369fe96 100644 --- a/target/arm/cpu64.c +++ b/target/arm/cpu64.c @@ -1165,6 +1165,7 @@ static void aarch64_max_initfn(Object *obj) cpu->isar.id_aa64mmfr0 =3D t; =20 t =3D cpu->isar.id_aa64mmfr1; + t =3D FIELD_DP64(t, ID_AA64MMFR1, HAFDBS, 1); /* FEAT_HAFDBS, AF onl= y */ t =3D FIELD_DP64(t, ID_AA64MMFR1, VMIDBITS, 2); /* FEAT_VMID16 */ t =3D FIELD_DP64(t, ID_AA64MMFR1, VH, 1); /* FEAT_VHE */ t =3D FIELD_DP64(t, ID_AA64MMFR1, HPDS, 1); /* FEAT_HPDS */ diff --git a/target/arm/ptw.c b/target/arm/ptw.c index 165e70d044..209ea7024b 100644 --- a/target/arm/ptw.c +++ b/target/arm/ptw.c @@ -21,7 +21,9 @@ typedef struct S1Translate { bool in_secure; bool in_debug; bool out_secure; + bool out_rw; bool out_be; + hwaddr out_virt; hwaddr out_phys; void *out_host; } S1Translate; @@ -238,6 +240,8 @@ static bool S1_ptw_translate(CPUARMState *env, S1Transl= ate *ptw, uint8_t pte_attrs; bool pte_secure; =20 + ptw->out_virt =3D addr; + if (unlikely(ptw->in_debug)) { /* * From gdbstub, do not use softmmu so that we don't modify the @@ -266,6 +270,7 @@ static bool S1_ptw_translate(CPUARMState *env, S1Transl= ate *ptw, pte_secure =3D is_secure; } ptw->out_host =3D NULL; + ptw->out_rw =3D false; } else { CPUTLBEntryFull *full; int flags; @@ -280,6 +285,7 @@ static bool S1_ptw_translate(CPUARMState *env, S1Transl= ate *ptw, goto fail; } ptw->out_phys =3D full->phys_addr; + ptw->out_rw =3D full->prot & PROT_WRITE; pte_attrs =3D full->pte_attrs; pte_secure =3D full->attrs.secure; } @@ -323,14 +329,16 @@ static uint32_t arm_ldl_ptw(CPUARMState *env, S1Trans= late *ptw, ARMMMUFaultInfo *fi) { CPUState *cs =3D env_cpu(env); + void *host =3D ptw->out_host; uint32_t data; =20 - if (likely(ptw->out_host)) { + if (likely(host)) { /* Page tables are in RAM, and we have the host address. */ + data =3D qatomic_read((uint32_t *)host); if (ptw->out_be) { - data =3D ldl_be_p(ptw->out_host); + data =3D be32_to_cpu(data); } else { - data =3D ldl_le_p(ptw->out_host); + data =3D le32_to_cpu(data); } } else { /* Page tables are in MMIO. */ @@ -356,15 +364,25 @@ static uint64_t arm_ldq_ptw(CPUARMState *env, S1Trans= late *ptw, ARMMMUFaultInfo *fi) { CPUState *cs =3D env_cpu(env); + void *host =3D ptw->out_host; uint64_t data; =20 - if (likely(ptw->out_host)) { + if (likely(host)) { /* Page tables are in RAM, and we have the host address. */ +#ifdef CONFIG_ATOMIC64 + data =3D qatomic_read__nocheck((uint64_t *)host); if (ptw->out_be) { - data =3D ldq_be_p(ptw->out_host); + data =3D be64_to_cpu(data); } else { - data =3D ldq_le_p(ptw->out_host); + data =3D le64_to_cpu(data); } +#else + if (ptw->out_be) { + data =3D ldq_be_p(host); + } else { + data =3D ldq_le_p(host); + } +#endif } else { /* Page tables are in MMIO. */ MemTxAttrs attrs =3D { .secure =3D ptw->out_secure }; @@ -385,6 +403,91 @@ static uint64_t arm_ldq_ptw(CPUARMState *env, S1Transl= ate *ptw, return data; } =20 +static uint64_t arm_casq_ptw(CPUARMState *env, uint64_t old_val, + uint64_t new_val, S1Translate *ptw, + ARMMMUFaultInfo *fi) +{ + uint64_t cur_val; + void *host =3D ptw->out_host; + + if (unlikely(!host)) { + fi->type =3D ARMFault_UnsuppAtomicUpdate; + fi->s1ptw =3D true; + return 0; + } + + /* + * Raising a stage2 Protection fault for an atomic update to a read-on= ly + * page is delayed until it is certain that there is a change to make. + */ + if (unlikely(!ptw->out_rw)) { + int flags; + void *discard; + + env->tlb_fi =3D fi; + flags =3D probe_access_flags(env, ptw->out_virt, MMU_DATA_STORE, + arm_to_core_mmu_idx(ptw->in_ptw_idx), + true, &discard, 0); + env->tlb_fi =3D NULL; + + if (unlikely(flags & TLB_INVALID_MASK)) { + assert(fi->type !=3D ARMFault_None); + fi->s2addr =3D ptw->out_virt; + fi->stage2 =3D true; + fi->s1ptw =3D true; + fi->s1ns =3D !ptw->in_secure; + return 0; + } + + /* In case CAS mismatches and we loop, remember writability. */ + ptw->out_rw =3D true; + } + +#ifdef CONFIG_ATOMIC64 + if (ptw->out_be) { + old_val =3D cpu_to_be64(old_val); + new_val =3D cpu_to_be64(new_val); + cur_val =3D qatomic_cmpxchg__nocheck((uint64_t *)host, old_val, ne= w_val); + cur_val =3D be64_to_cpu(cur_val); + } else { + old_val =3D cpu_to_le64(old_val); + new_val =3D cpu_to_le64(new_val); + cur_val =3D qatomic_cmpxchg__nocheck((uint64_t *)host, old_val, ne= w_val); + cur_val =3D le64_to_cpu(cur_val); + } +#else + /* + * We can't support the full 64-bit atomic cmpxchg on the host. + * Because this is only used for FEAT_HAFDBS, which is only for AA64, + * we know that TCG_OVERSIZED_GUEST is set, which means that we are + * running in round-robin mode and could only race with dma i/o. + */ +#ifndef TCG_OVERSIZED_GUEST +# error "Unexpected configuration" +#endif + bool locked =3D qemu_mutex_iothread_locked(); + if (!locked) { + qemu_mutex_lock_iothread(); + } + if (ptw->out_be) { + cur_val =3D ldq_be_p(host); + if (cur_val =3D=3D old_val) { + stq_be_p(host, new_val); + } + } else { + cur_val =3D ldq_le_p(host); + if (cur_val =3D=3D old_val) { + stq_le_p(host, new_val); + } + } + if (!locked) { + qemu_mutex_unlock_iothread(); + } +#endif + + return cur_val; +} + static bool get_level1_table_address(CPUARMState *env, ARMMMUIdx mmu_idx, uint32_t *table, uint32_t address) { @@ -1077,7 +1180,7 @@ static bool get_phys_addr_lpae(CPUARMState *env, S1Tr= anslate *ptw, uint32_t el =3D regime_el(env, mmu_idx); uint64_t descaddrmask; bool aarch64 =3D arm_el_is_aa64(env, el); - uint64_t descriptor; + uint64_t descriptor, new_descriptor; bool nstable; =20 /* TODO: This code does not support shareability levels. */ @@ -1291,7 +1394,9 @@ static bool get_phys_addr_lpae(CPUARMState *env, S1Tr= anslate *ptw, if (fi->type !=3D ARMFault_None) { goto do_fault; } + new_descriptor =3D descriptor; =20 + restart_atomic_update: if (!(descriptor & 1) || (!(descriptor & 2) && (level =3D=3D 3))) { /* Invalid, or the Reserved level 3 encoding */ goto do_translation_fault; @@ -1337,17 +1442,36 @@ static bool get_phys_addr_lpae(CPUARMState *env, S1= Translate *ptw, * to give a correct page or table address, the address field * in a block descriptor is smaller; so we need to explicitly * clear the lower bits here before ORing in the low vaddr bits. + * + * Afterward, descaddr is the final physical address. */ page_size =3D (1ULL << ((stride * (4 - level)) + 3)); descaddr &=3D ~(hwaddr)(page_size - 1); descaddr |=3D (address & (page_size - 1)); =20 + if (likely(!ptw->in_debug)) { + /* + * Access flag. + * If HA is enabled, prepare to update the descriptor below. + * Otherwise, pass the access fault on to software. + */ + if (!(descriptor & (1 << 10))) { + if (param.ha) { + new_descriptor |=3D 1 << 10; /* AF */ + } else { + fi->type =3D ARMFault_AccessFlag; + goto do_fault; + } + } + } + /* - * Extract attributes from the descriptor, and apply table descriptors. - * Stage 2 table descriptors do not include any attribute fields. - * HPD disables all the table attributes except NSTable. + * Extract attributes from the (modified) descriptor, and apply + * table descriptors. Stage 2 table descriptors do not include + * any attribute fields. HPD disables all the table attributes + * except NSTable. */ - attrs =3D descriptor & (MAKE_64BIT_MASK(2, 10) | MAKE_64BIT_MASK(50, 1= 4)); + attrs =3D new_descriptor & (MAKE_64BIT_MASK(2, 10) | MAKE_64BIT_MASK(5= 0, 14)); if (!regime_is_stage2(mmu_idx)) { attrs |=3D nstable << 5; /* NS */ if (!param.hpd) { @@ -1361,18 +1485,7 @@ static bool get_phys_addr_lpae(CPUARMState *env, S1T= ranslate *ptw, } } =20 - /* - * Here descaddr is the final physical address, and attributes - * are all in attrs. - */ - if ((attrs & (1 << 10)) =3D=3D 0) { - /* Access flag */ - fi->type =3D ARMFault_AccessFlag; - goto do_fault; - } - ap =3D extract32(attrs, 6, 2); - if (regime_is_stage2(mmu_idx)) { ns =3D mmu_idx =3D=3D ARMMMUIdx_Stage2; xn =3D extract64(attrs, 54, 2); @@ -1389,6 +1502,25 @@ static bool get_phys_addr_lpae(CPUARMState *env, S1T= ranslate *ptw, goto do_fault; } =20 + /* If FEAT_HAFDBS has made changes, update the PTE. */ + if (new_descriptor !=3D descriptor) { + new_descriptor =3D arm_casq_ptw(env, descriptor, new_descriptor, p= tw, fi); + if (fi->type !=3D ARMFault_None) { + goto do_fault; + } + /* + * I_YZSVV says that if the in-memory descriptor has changed, + * then we must use the information in that new value + * (which might include a different output address, different + * attributes, or generate a fault). + * Restart the handling of the descriptor value from scratch. + */ + if (new_descriptor !=3D descriptor) { + descriptor =3D new_descriptor; + goto restart_atomic_update; + } + } + if (ns) { /* * The NS bit will (as required by the architecture) have no effec= t if --=20 2.34.1