From nobody Fri Apr 10 20:16:10 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3DB0EC00140 for ; Fri, 19 Aug 2022 01:35:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229498AbiHSBfm (ORCPT ); Thu, 18 Aug 2022 21:35:42 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51784 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242028AbiHSBfk (ORCPT ); Thu, 18 Aug 2022 21:35:40 -0400 Received: from mail-yw1-x114a.google.com (mail-yw1-x114a.google.com [IPv6:2607:f8b0:4864:20::114a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6964C77552 for ; Thu, 18 Aug 2022 18:35:39 -0700 (PDT) Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-333b218f2cbso54238767b3.0 for ; Thu, 18 Aug 2022 18:35:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:from:subject:mime-version:message-id:date:from:to:cc; bh=oZFtvhwwLycAg9JyRv4nQ5Y9An2VOnZjO51MLHku7jw=; b=qWGzRc+qtsOIR7eIcMor/2en6BehPZakmRgIKsISqzKA4aylG3Bx9uK0d24R2TG9qj 2l0T9Y30KER+NWerKlR32bTm5EW+BZbJHwTH7TJRKdeJWlFcCc0FLSkdjX2SomsKjOcH 0qYwnOvPFBMcOqejrGxRVIzBVO8+86pfHGfKtCibKY2fVZcd8IGDdgmmZ7LP8rFTuymE iPipzJFLfO2oYU9mkFwK4ny7HT05ckPyX4QjdduOyGDhX3X7Ufk/AJUGIi1xy00CTTyg aqxQ4ylzEF7doA7Bts+CqAGEpUm9o5ycEkamvu58rCQdJkZvscIY1i9nnJKFCXizcONg H7ag== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:from:subject:mime-version:message-id:date:x-gm-message-state :from:to:cc; bh=oZFtvhwwLycAg9JyRv4nQ5Y9An2VOnZjO51MLHku7jw=; b=Ts4pOOMcWFvB+clu+Khjn6lsys+VnrBeqhpvv7JRXtNMunq4At29dGrFRdh3tedjzH bnIjMm+tt5ddKXlY8aN52AewJxgKztirUefb46RLI0SSmoCTbnGMDHi3RAKiCNHx3Efy T0DfiFXn0oUstK0nB5SbPpM8zmS/DWUriGVhhHCSlGJy66/5SssCg2R7diAziPgllbvn /chKkMHhLk862LYvAkMP91Ua1SbzCAOdDERRFOFN4WKGF6gzh5vxXU5DDdbyyjYe/X15 16u06Sp1oNmtMUgByrRgH5RazbGD0gAVzon4IAjyA5JT9GpnLgZIFZNzcvOkPjC1dFsD YiwA== X-Gm-Message-State: ACgBeo3M5CAXzupMwJSyVfKCjic62XoFegMo/6EuYxv0iNZJeIGGS8S3 OhGa/JZQXNy39ORWS2s4QefXDPI= X-Google-Smtp-Source: AA6agR52vBcrAJzOGMPlCvhkn6sZjkVI03YVJkFfWqvf2yfHvpgx9riawPB8cBu92xm/Eppc3jgdQUc= X-Received: from pcc-desktop.svl.corp.google.com ([2620:15c:2ce:200:c686:ab0e:7472:becb]) (user=pcc job=sendgmr) by 2002:a25:c3c6:0:b0:67b:5458:c19b with SMTP id t189-20020a25c3c6000000b0067b5458c19bmr5361708ybf.408.1660872938706; Thu, 18 Aug 2022 18:35:38 -0700 (PDT) Date: Thu, 18 Aug 2022 18:35:26 -0700 Message-Id: <20220819013526.2682765-1-pcc@google.com> Mime-Version: 1.0 X-Mailer: git-send-email 2.37.1.595.g718a3a8f04-goog Subject: [PATCH] arm64: mte: move register initialization to C From: Peter Collingbourne To: Catalin Marinas Cc: Peter Collingbourne , Evgenii Stepanov , Marc Zyngier , Will Deacon , Vincenzo Frascino , Andrey Konovalov , Mark Brown , Linux ARM , LKML Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" If FEAT_MTE2 is disabled via the arm64.nomte command line argument on a CPU that claims to support FEAT_MTE2, the kernel will use Tagged Normal in the MAIR. If we interpret arm64.nomte to mean that the CPU does not in fact implement FEAT_MTE2, setting the system register like this may lead to UNSPECIFIED behavior. Fix it by arranging for MAIR to be set in the C function cpu_enable_mte which is called based on the sanitized version of the system register. There is no need for the rest of the MTE-related system register initialization to happen from assembly, with the exception of TCR_EL1, which must be set to include at least TBI1 because the secondary CPUs access KASan-allocated data structures early. Therefore, make the TCR_EL1 initialization unconditional and move the rest of the initialization to cpu_enable_mte so that we no longer have a dependency on the unsanitized ID register value. Signed-off-by: Peter Collingbourne Suggested-by: Catalin Marinas Link: https://linux-review.googlesource.com/id/I2c7df6bd4ea2dfc59376a8b9b5d= 3562b015c7198 --- arch/arm64/kernel/cpufeature.c | 39 +++++++++++++++++++++++++++++++ arch/arm64/mm/proc.S | 42 ++-------------------------------- 2 files changed, 41 insertions(+), 40 deletions(-) diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index 907401e4fffb..3554ff869f4b 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -2030,8 +2030,47 @@ static void bti_enable(const struct arm64_cpu_capabi= lities *__unused) #ifdef CONFIG_ARM64_MTE static void cpu_enable_mte(struct arm64_cpu_capabilities const *cap) { + u64 rgsr; + sysreg_clear_set(sctlr_el1, 0, SCTLR_ELx_ATA | SCTLR_EL1_ATA0); + + /* + * CnP must be enabled only after the MAIR_EL1 register has been set + * up. Inconsistent MAIR_EL1 between CPUs sharing the same TLB may + * lead to the wrong memory type being used for a brief window during + * CPU power-up. + * + * CnP is not a boot feature so MTE gets enabled before CnP, but let's + * make sure that is the case. + */ + BUG_ON(read_sysreg(ttbr0_el1) & TTBR_CNP_BIT); + BUG_ON(read_sysreg(ttbr1_el1) & TTBR_CNP_BIT); + + /* Normal Tagged memory type at the corresponding MAIR index */ + sysreg_clear_set( + mair_el1, MAIR_ATTRIDX(MAIR_ATTR_MASK, MT_NORMAL_TAGGED), + MAIR_ATTRIDX(MAIR_ATTR_NORMAL_TAGGED, MT_NORMAL_TAGGED)); + + write_sysreg_s(KERNEL_GCR_EL1, SYS_GCR_EL1); + + /* + * If GCR_EL1.RRND=3D1 is implemented the same way as RRND=3D0, then + * RGSR_EL1.SEED must be non-zero for IRG to produce + * pseudorandom numbers. As RGSR_EL1 is UNKNOWN out of reset, we + * must initialize it. + */ + rgsr =3D (read_sysreg(CNTVCT_EL0) << SYS_RGSR_EL1_SEED_SHIFT) & + SYS_RGSR_EL1_SEED_MASK; + if (rgsr =3D=3D 0) + rgsr =3D 1 << SYS_RGSR_EL1_SEED_SHIFT; + write_sysreg_s(rgsr, SYS_RGSR_EL1); + + /* clear any pending tag check faults in TFSR*_EL1 */ + write_sysreg_s(0, SYS_TFSR_EL1); + write_sysreg_s(0, SYS_TFSRE0_EL1); + isb(); + local_flush_tlb_all(); =20 /* * Clear the tags in the zero page. This needs to be done via the diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S index 7837a69524c5..6f01f5a54cc2 100644 --- a/arch/arm64/mm/proc.S +++ b/arch/arm64/mm/proc.S @@ -58,7 +58,7 @@ =20 /* * Default MAIR_EL1. MT_NORMAL_TAGGED is initially mapped as Normal memory= and - * changed during __cpu_setup to Normal Tagged if the system supports MTE. + * changed during cpu_enable_mte to Normal Tagged if the system supports M= TE. */ #define MAIR_EL1_SET \ (MAIR_ATTRIDX(MAIR_ATTR_DEVICE_nGnRnE, MT_DEVICE_nGnRnE) | \ @@ -426,46 +426,8 @@ SYM_FUNC_START(__cpu_setup) mov_q mair, MAIR_EL1_SET mov_q tcr, TCR_TxSZ(VA_BITS) | TCR_CACHE_FLAGS | TCR_SMP_FLAGS | \ TCR_TG_FLAGS | TCR_KASLR_FLAGS | TCR_ASID16 | \ - TCR_TBI0 | TCR_A1 | TCR_KASAN_SW_FLAGS + TCR_TBI0 | TCR_A1 | TCR_KASAN_SW_FLAGS | TCR_MTE_FLAGS =20 -#ifdef CONFIG_ARM64_MTE - /* - * Update MAIR_EL1, GCR_EL1 and TFSR*_EL1 if MTE is supported - * (ID_AA64PFR1_EL1[11:8] > 1). - */ - mrs x10, ID_AA64PFR1_EL1 - ubfx x10, x10, #ID_AA64PFR1_MTE_SHIFT, #4 - cmp x10, #ID_AA64PFR1_MTE - b.lt 1f - - /* Normal Tagged memory type at the corresponding MAIR index */ - mov x10, #MAIR_ATTR_NORMAL_TAGGED - bfi mair, x10, #(8 * MT_NORMAL_TAGGED), #8 - - mov x10, #KERNEL_GCR_EL1 - msr_s SYS_GCR_EL1, x10 - - /* - * If GCR_EL1.RRND=3D1 is implemented the same way as RRND=3D0, then - * RGSR_EL1.SEED must be non-zero for IRG to produce - * pseudorandom numbers. As RGSR_EL1 is UNKNOWN out of reset, we - * must initialize it. - */ - mrs x10, CNTVCT_EL0 - ands x10, x10, #SYS_RGSR_EL1_SEED_MASK - csinc x10, x10, xzr, ne - lsl x10, x10, #SYS_RGSR_EL1_SEED_SHIFT - msr_s SYS_RGSR_EL1, x10 - - /* clear any pending tag check faults in TFSR*_EL1 */ - msr_s SYS_TFSR_EL1, xzr - msr_s SYS_TFSRE0_EL1, xzr - - /* set the TCR_EL1 bits */ - mov_q x10, TCR_MTE_FLAGS - orr tcr, tcr, x10 -1: -#endif tcr_clear_errata_bits tcr, x9, x5 =20 #ifdef CONFIG_ARM64_VA_BITS_52 --=20 2.37.1.595.g718a3a8f04-goog