From nobody Mon Jun 8 08:27:58 2026 Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1E5273E008F for ; Wed, 3 Jun 2026 16:11:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780503082; cv=none; b=G0kxbtvlojospRKLQE/5kfPFh+RDtpQX5zlm8uHXdgcxH866i0E53mltMlCjFQKGHR487rY3w7c9HlYBMwWl7WwiLcfV9JO692OUe5d6YT9OtJYpucI2MDD05z/mFTvK5I+JbvpqhP7fd0BU9ZIN51Falek1Dax5Ply03DzpB0s= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780503082; c=relaxed/simple; bh=eaNupvnA0Lqw/MOrFohHOEsrKK+Vfr9jUBx2HbnfPRI=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=lasdT5vOeTzb6cmSxgEnMaFknwXTM8EkwGjFU5UwdjERuycG3AS3BJcQcK9sZG8JlWucE+uN5a8x4rZ4wyyM+8aEoyHq8pAU432mYcQ9JUGFluETBfI/5BnRIR+0RKzkkEQrO6kMG/537ZKfVIFkd4fbtayZ8SMdjQYhE6sn+Cc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=tIShb6Qe; arc=none smtp.client-ip=209.85.128.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="tIShb6Qe" Received: by mail-wm1-f74.google.com with SMTP id 5b1f17b1804b1-490a767c7dcso31983485e9.2 for ; Wed, 03 Jun 2026 09:11:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1780503079; x=1781107879; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=WbgvnoAHr1VZwp/TGprKI6jwxgm2Tuk9bwrWQUAuGn8=; b=tIShb6Qe294qeaOGuxPopMUjQjFP6wrZ92gIbwYjL23K3MJZOcllb9rLANe7S7wc+j rsK7lHTgEOtL2zDJd0O94C4z2anbx+3xtSbZi/av5UaYTMjaJsjH+/lw8BfCRxwVkb7v YT5NUGYyZqWlS7bVfc4mAbx8de7fCnZl5zakaHOoxMaU8tHjgPRfWR4vwSacQXdclsg4 StMXdiQOlq5VRuxvydeZxBMjg015zgvujCHPGUwjHKreBwTLDQ0kfCTM+hoBiEcwfzt0 fZgHzFfejYg/UiG3Xp2eXckocu5iXNVEQ7TjtZv6D2SDeWrDLv4Nl2UM3tWajHkxB2GC R/YQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780503079; x=1781107879; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=WbgvnoAHr1VZwp/TGprKI6jwxgm2Tuk9bwrWQUAuGn8=; b=RIk0UdULBODsUH+gBiZpulKmxi45JYgNZz8VPRzPYINtA1zWQ8XD5rxn7WrNAYD31S UnzQ4MzN6HKwYvPQXAv7aLp9RYmz/zfFUIoUsYzI89VYA1chKAaX7BRfX0wu338o5CsA xAWoGwXQmMXZ0TcLesu0g2wKXy4qEf/zqy5fg9o0R1nSEKZnOiyxhxbHNVNfNEUxFW0A h0sXRW9w7Rc/kV31tubL4b07hldDjT4TkD8Mj1167wtopjBDNWfjpgD7j7HZphLTZSfo G2HQbFls+ITzB54LYJpiVb7uSJyznTz68rjKY8CxgxsIGTXRLYqrAJU6Rjq0sMey4vFC v6MQ== X-Gm-Message-State: AOJu0Yz4vzsD4qEmYD6VXPHx8tByKSU36b9PWo1kg/Ms58r4LYtS+LVZ f1vIkmceFpm6hax7XyothQHB8jBXbUh+35Uib7izBQSyZ7kB+f4dC01iWWZjVn6C921fMN1vtA= = X-Received: from wruh4.prod.google.com ([2002:a5d:6884:0:b0:43d:7e53:c76a]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:414f:b0:490:b645:3213 with SMTP id 5b1f17b1804b1-490b645343bmr42462745e9.19.1780503079435; Wed, 03 Jun 2026 09:11:19 -0700 (PDT) Date: Wed, 3 Jun 2026 18:09:51 +0200 In-Reply-To: <20260603160949.3372482-6-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260603160949.3372482-6-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=5738; i=ardb@kernel.org; h=from:subject; bh=H9XZxM8XBIS3dkH+KdJkynk6oFDR1zV1Vb5pNRNFzRU=; b=owGbwMvMwCVmkMcZplerG8N4Wi2JIUsh8DzDwfV/5vOnvllwZIk0Y+nb+pOhh2KSuA8vEr+Rz qBsVavdUcrCIMbFICumyCIw+++7nacnStU6z5KFmcPKBDaEi1MAJnLmIcN/rw1nuH8kfxSp4ReY xBokMWvuuo68ZWwbW60S2H7vzZvoxshwgW3XwzULLl7eGtkw+d2zAtWQBxclN1Qqb9kQ/pKdb1M 6FwA= X-Mailer: git-send-email 2.54.0.1013.g208068f2d8-goog Message-ID: <20260603160949.3372482-7-ardb+git@google.com> Subject: [PATCH 1/4] arm64: Move generic KASAN page tables out of BSS too From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, will@kernel.org, catalin.marinas@arm.com, maz@kernel.org, Ard Biesheuvel , Kevin Brodsky , Mark Brown Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel Make sure that all KASAN page tables are emitted into the .pgdir section (provided that the arch has one - otherwise, fall back to page aligned BSS) This is needed because BSS itself is no longer accessible via the linear map. Signed-off-by: Ard Biesheuvel --- arch/arm64/include/asm/linkage.h | 2 ++ arch/arm64/include/asm/mmu.h | 2 -- arch/arm64/kernel/vmlinux.lds.S | 8 ++++---- arch/arm64/mm/fixmap.c | 6 +++--- arch/arm64/mm/kasan_init.c | 2 +- include/linux/linkage.h | 4 ++++ mm/kasan/init.c | 10 +++++----- 7 files changed, 19 insertions(+), 15 deletions(-) diff --git a/arch/arm64/include/asm/linkage.h b/arch/arm64/include/asm/link= age.h index 40bd17add539..8637f667667c 100644 --- a/arch/arm64/include/asm/linkage.h +++ b/arch/arm64/include/asm/linkage.h @@ -43,4 +43,6 @@ SYM_TYPED_START(name, SYM_L_GLOBAL, SYM_A_ALIGN) \ bti c ; =20 +#define __bss_pgtbl __section(".bss..pgtbl") __aligned(PAGE_SIZE) + #endif diff --git a/arch/arm64/include/asm/mmu.h b/arch/arm64/include/asm/mmu.h index fb95754f2876..5e1211c540ab 100644 --- a/arch/arm64/include/asm/mmu.h +++ b/arch/arm64/include/asm/mmu.h @@ -13,8 +13,6 @@ =20 #ifndef __ASSEMBLER__ =20 -#define __pgtbl_bss __section(".pgdir.bss") __aligned(PAGE_SIZE) - #include #include =20 diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.ld= s.S index 2b0ebfb30c63..d3ed59abab38 100644 --- a/arch/arm64/kernel/vmlinux.lds.S +++ b/arch/arm64/kernel/vmlinux.lds.S @@ -352,11 +352,11 @@ SECTIONS BSS_SECTION(SBSS_ALIGN, 0, PAGE_SIZE) __pi___bss_start =3D __bss_start; =20 - /* fixmap BSS starts here - preceding data/BSS is omitted from the linear= map */ - .pgdir.bss (NOLOAD) : ALIGN(PAGE_SIZE) { - *(.pgdir.bss) + /* page table BSS starts here - preceding data/BSS is omitted from the li= near map */ + .pgtbl : ALIGN(PAGE_SIZE) { + *(.bss..pgtbl) } - ASSERT(ADDR(.pgdir.bss) =3D=3D __bss_stop, ".pgdir.bss must follow BSS") + ASSERT(ADDR(.pgtbl) =3D=3D __bss_stop, ".pgtbl must follow BSS") =20 . =3D ALIGN(PAGE_SIZE); __pi_init_pg_dir =3D .; diff --git a/arch/arm64/mm/fixmap.c b/arch/arm64/mm/fixmap.c index 1a3bbd67dd76..f66a0016dd02 100644 --- a/arch/arm64/mm/fixmap.c +++ b/arch/arm64/mm/fixmap.c @@ -31,9 +31,9 @@ static_assert(NR_BM_PMD_TABLES =3D=3D 1); =20 #define BM_PTE_TABLE_IDX(addr) __BM_TABLE_IDX(addr, PMD_SHIFT) =20 -static pte_t bm_pte[NR_BM_PTE_TABLES][PTRS_PER_PTE] __pgtbl_bss; -static pmd_t bm_pmd[PTRS_PER_PMD] __pgtbl_bss __maybe_unused; -static pud_t bm_pud[PTRS_PER_PUD] __pgtbl_bss __maybe_unused; +static pte_t bm_pte[NR_BM_PTE_TABLES][PTRS_PER_PTE] __bss_pgtbl; +static pmd_t bm_pmd[PTRS_PER_PMD] __bss_pgtbl __maybe_unused; +static pud_t bm_pud[PTRS_PER_PUD] __bss_pgtbl __maybe_unused; =20 static inline pte_t *fixmap_pte(unsigned long addr) { diff --git a/arch/arm64/mm/kasan_init.c b/arch/arm64/mm/kasan_init.c index dbf22cae82ee..3fcad956fdf7 100644 --- a/arch/arm64/mm/kasan_init.c +++ b/arch/arm64/mm/kasan_init.c @@ -214,7 +214,7 @@ asmlinkage void __init kasan_early_init(void) * shadow pud_t[]/p4d_t[], which could end up getting corrupted * when the linear region is mapped. */ - static pte_t tbl[PTRS_PER_PTE] __pgtbl_bss; + static pte_t tbl[PTRS_PER_PTE] __bss_pgtbl; pgd_t *pgdp =3D pgd_offset_k(KASAN_SHADOW_START); =20 set_pgd(pgdp, __pgd(__pa_symbol(tbl) | PGD_TYPE_TABLE)); diff --git a/include/linux/linkage.h b/include/linux/linkage.h index b11660b706c5..53fe1f48fd28 100644 --- a/include/linux/linkage.h +++ b/include/linux/linkage.h @@ -39,6 +39,10 @@ #define __page_aligned_data __section(".data..page_aligned") __aligned(PAG= E_SIZE) #define __page_aligned_bss __section(".bss..page_aligned") __aligned(PAGE_= SIZE) =20 +#ifndef __bss_pgtbl +#define __bss_pgtbl __page_aligned_bss +#endif + /* * For assembly routines. * diff --git a/mm/kasan/init.c b/mm/kasan/init.c index 9c880f607c6a..66a883887987 100644 --- a/mm/kasan/init.c +++ b/mm/kasan/init.c @@ -26,10 +26,10 @@ * - Latter it reused it as zero shadow to cover large ranges of memory * that allowed to access, but not handled by kasan (vmalloc/vmemmap .= ..). */ -unsigned char kasan_early_shadow_page[PAGE_SIZE] __page_aligned_bss; +unsigned char kasan_early_shadow_page[PAGE_SIZE] __bss_pgtbl; =20 #if CONFIG_PGTABLE_LEVELS > 4 -p4d_t kasan_early_shadow_p4d[MAX_PTRS_PER_P4D] __page_aligned_bss; +p4d_t kasan_early_shadow_p4d[MAX_PTRS_PER_P4D] __bss_pgtbl; static inline bool kasan_p4d_table(pgd_t pgd) { return pgd_page(pgd) =3D=3D virt_to_page(lm_alias(kasan_early_shadow_p4d)= ); @@ -41,7 +41,7 @@ static inline bool kasan_p4d_table(pgd_t pgd) } #endif #if CONFIG_PGTABLE_LEVELS > 3 -pud_t kasan_early_shadow_pud[MAX_PTRS_PER_PUD] __page_aligned_bss; +pud_t kasan_early_shadow_pud[MAX_PTRS_PER_PUD] __bss_pgtbl; static inline bool kasan_pud_table(p4d_t p4d) { return p4d_page(p4d) =3D=3D virt_to_page(lm_alias(kasan_early_shadow_pud)= ); @@ -53,7 +53,7 @@ static inline bool kasan_pud_table(p4d_t p4d) } #endif #if CONFIG_PGTABLE_LEVELS > 2 -pmd_t kasan_early_shadow_pmd[MAX_PTRS_PER_PMD] __page_aligned_bss; +pmd_t kasan_early_shadow_pmd[MAX_PTRS_PER_PMD] __bss_pgtbl; static inline bool kasan_pmd_table(pud_t pud) { return pud_page(pud) =3D=3D virt_to_page(lm_alias(kasan_early_shadow_pmd)= ); @@ -65,7 +65,7 @@ static inline bool kasan_pmd_table(pud_t pud) } #endif pte_t kasan_early_shadow_pte[MAX_PTRS_PER_PTE + PTE_HWTABLE_PTRS] - __page_aligned_bss; + __bss_pgtbl; =20 static inline bool kasan_pte_table(pmd_t pmd) { --=20 2.54.0.1013.g208068f2d8-goog From nobody Mon Jun 8 08:27:58 2026 Received: from mail-wm1-f73.google.com (mail-wm1-f73.google.com [209.85.128.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7332D3E275F for ; Wed, 3 Jun 2026 16:11:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780503083; cv=none; b=We2cFQ8DVA/yGi1GZM4MSQKRbiDf/ltDH4ZDEPmdaHp2ctFxuEYxpwEnbMlB6LBil7zWfJzX6UuFmNtWQce1qlHt/8H+jKDh7etOn3mCwv6xcEu/cGxorXX1X4oBK/gpe6GwADYd20H26yKGyiV/Jd/gQPzkfK4LFaFKjMj6rj8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780503083; c=relaxed/simple; bh=dEmc5sV0UNjyYz4TuMmIs0aQ6UcAq2tyWsOVeh5s+Us=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=uAW/zYAQq4zVC34HwYudg6pD/V3IiI3xZYQQgWLdhBHbQZyUXpuL3/oLuTQfq5prCf/6e9AvYzdw2mRfnr+j9qnWw8IHTv7ePbat2eoJ0F0vscY7io/c1IjRDAC4lUA+N62pfXS028rUC7OoE4diYLKfcULKHfd35W1hEUyAcrk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=SJehfnNZ; arc=none smtp.client-ip=209.85.128.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="SJehfnNZ" Received: by mail-wm1-f73.google.com with SMTP id 5b1f17b1804b1-490a786f987so36909965e9.3 for ; Wed, 03 Jun 2026 09:11:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1780503081; x=1781107881; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=TL4hLQqtTeIYj6fVsZXeAVFV7Zv0bhA39dqz5T4VWsY=; b=SJehfnNZoroL500wolBX28Emc0OCQkPa318+dCTOa1i3E2YugIj5CKy6nGIUH3IKe7 jTPGtowKqdtm0LS2J0PACmXK36jn4APbLktLasa+gnKiuLPxo21EYLROhojZJUvv6AHU 9ATWz74i+33mJb8I7+RhzcmuQZf/oSHrf92ik0M9Ao6Zyiae04i3tyexs8TzP3JfijuW F4ofCiPGAhXx/+W+xL4PzR7TtlM8pgGOLzmnA3mpd8WWL2IUFRcpyK2YgOM6g/Kiitml ERbXKrFNh0vR6GtAfuJFVmC3RboBpKfRF7PlOjHOrkej1jABk1AnDmgKJJLl4a2Ji9qa iK0g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780503081; x=1781107881; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=TL4hLQqtTeIYj6fVsZXeAVFV7Zv0bhA39dqz5T4VWsY=; b=ULl/H/uqiPBfz+yvU/6b/VtQGx9wnt5zkQUAvXvpTR/LijEkLZ3FAekovXrwRXU+tK NYWh+Mm51ewM+0ftBsFe2LjsF1tIPLfUQV4t1/firSTczgKUkJjCTqkt/xdYZR4Lr+72 /ecfDDknNf4Tf0fIh7GfTj8+KchHdSTK7Wr07jwXO15BIA0fdLgHSjChMUmIBSXQ1tku NTVlKHRyA+V9EAUckSTQDVRxbpSyyh904xE406TuGfezP50a6jUVZ/csZX0hVfMAy2vT QIruvOJQJrjnztJZDRKM199farwwBScow2nu0JeMKW4DGsCrYavVIoPXs/BwnYwvA9ZS D7KA== X-Gm-Message-State: AOJu0YxmjdTvMTKxfJAD64b61VDvaE/akMuSEPEArcoRH+AD3I0mxlFT t/5vvUC/YRWiDhf0tRgPsGnoRWu9rby5ZxY45+i3O4xkTnK8P/C+uU3hhYr16XdCDDkNNVnTpg= = X-Received: from wmoz18.prod.google.com ([2002:a05:600c:792:b0:490:b58a:2c80]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:820a:b0:490:9782:3eb8 with SMTP id 5b1f17b1804b1-490b5fd7378mr70201975e9.25.1780503080726; Wed, 03 Jun 2026 09:11:20 -0700 (PDT) Date: Wed, 3 Jun 2026 18:09:52 +0200 In-Reply-To: <20260603160949.3372482-6-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260603160949.3372482-6-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=2222; i=ardb@kernel.org; h=from:subject; bh=klTYuM9nrZaRSnTcs7Uaq38kYQvUy5mIsHZz1sf0jcw=; b=owGbwMvMwCVmkMcZplerG8N4Wi2JIUsh8KL6g/daO9pnK2+X3DhXPXKbsllExfMJs+ocW2KSW Y0fh0zoKGVhEONikBVTZBGY/ffdztMTpWqdZ8nCzGFlAhnCwMUpABOximD4p7Hz1NRnV46F1Zft DDxXyWkZcXf2ml3G5jNyAwOueKik6zAyHJklGVT4TXPNy8qLuzxbxG89qbH06y1pzigNu5+XEP2 THQA= X-Mailer: git-send-email 2.54.0.1013.g208068f2d8-goog Message-ID: <20260603160949.3372482-8-ardb+git@google.com> Subject: [PATCH 2/4] arm64: Avoid double evaluation of __ptep_get() From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, will@kernel.org, catalin.marinas@arm.com, maz@kernel.org, Ard Biesheuvel , Kevin Brodsky , Mark Brown Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel Sashiko warns that the new pte_valid_noncont() macro is used in a manner where the argument (which performs a READ_ONCE() of the descriptor) is evaluated twice. Drop the macro that we just added, and move the check into the newly added users. Signed-off-by: Ard Biesheuvel --- arch/arm64/include/asm/pgtable.h | 4 ---- arch/arm64/mm/mmu.c | 14 ++++++++++---- 2 files changed, 10 insertions(+), 8 deletions(-) diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgta= ble.h index 491ba0a6492d..c9e4e00a9af2 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -181,10 +181,6 @@ static inline pteval_t __phys_to_pte_val(phys_addr_t p= hys) * Returns true if the pte is valid and has the contiguous bit set. */ #define pte_valid_cont(pte) (pte_valid(pte) && pte_cont(pte)) -/* - * Returns true if the pte is valid and has the contiguous bit cleared. - */ -#define pte_valid_noncont(pte) (pte_valid(pte) && !pte_cont(pte)) /* * Could the pte be present in the TLB? We must check mm_tlb_flush_pending * so that we don't erroneously return false for pages that have been diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index be51f6cac86f..d68e691c093a 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -186,9 +186,12 @@ static void init_pte(pte_t *ptep, unsigned long addr, = unsigned long end, =20 static bool pte_range_has_valid_noncont(pte_t *ptep) { - for (int i =3D 0; i < CONT_PTES; i++) - if (pte_valid_noncont(__ptep_get(&ptep[i]))) + for (int i =3D 0; i < CONT_PTES; i++) { + pte_t pte =3D __ptep_get(&ptep[i]); + + if (pte_valid(pte) && !pte_cont(pte)) return true; + } return false; } =20 @@ -291,9 +294,12 @@ static int init_pmd(pmd_t *pmdp, unsigned long addr, u= nsigned long end, =20 static bool pmd_range_has_valid_noncont(pmd_t *pmdp) { - for (int i =3D 0; i < CONT_PMDS; i++) - if (pte_valid_noncont(pmd_pte(READ_ONCE(pmdp[i])))) + for (int i =3D 0; i < CONT_PMDS; i++) { + pte_t pte =3D pmd_pte(READ_ONCE(pmdp[i])); + + if (pte_valid(pte) && !pte_cont(pte)) return true; + } return false; } =20 --=20 2.54.0.1013.g208068f2d8-goog From nobody Mon Jun 8 08:27:58 2026 Received: from mail-wm1-f73.google.com (mail-wm1-f73.google.com [209.85.128.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CDDE13E3D98 for ; Wed, 3 Jun 2026 16:11:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780503085; cv=none; b=pYEmzmPK//0Jxy5isrFIU3MGM7A3kkPT+qXD707NshpTQdf6YtgLnkZYrZPxLZKhscJdnlJrWQO3EVHJTodKw+KYJr4PpWb7oMUvk1H7z8uWCfWJ+83/LMyen6R3v1WnjyvwjmvnpA/hXQPeyZ5Gp3DtAd2dQuMBdyf2MNnN/PY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780503085; c=relaxed/simple; bh=fJGNcMlz+SxsA+jLRLlCKD046oRYRmRlCUIrYDn356w=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=G9ErU4x7Jia0Omc9tzUraVyzr8Tu0CvPFt+HnlgVAvToMkjQ7dSoZvxlX9QCh4eYLrgBJaZCDBSnP/8G9pNrM8i55+w/93rSZV2oIKKSxwTm/VHQs4BQD85P5P5yvWx+PE/gu/Gyx+nOtbEekDUdVXQl2JQQOWtd93PacJ5Cjow= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=W7OKzSMi; arc=none smtp.client-ip=209.85.128.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="W7OKzSMi" Received: by mail-wm1-f73.google.com with SMTP id 5b1f17b1804b1-490b7959b9fso9885365e9.2 for ; Wed, 03 Jun 2026 09:11:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1780503082; x=1781107882; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=S0uwFj2oVK8Ym93JzitEXh3ZnJKfgcyu+cm0Jby8VRA=; b=W7OKzSMiHvLyuiG6KWbQjKU1Pn7cIXE/Gm2q7bnSNNH4eTWtdDs16Xxvv/NVDnQ4Is EDRihyNe1ioOLl1WDAfBEQ6b52X4i2U8pv5Smbu5Aa3kBsbFZoypwNtZW4aisOJj5OB1 WEy6uDx0sMk8TSx1bCqUiEWnfoVJMumjh55C7ghyjrEaRzqpdITLEeeKeazdkTniVv4h Vrvp473gRcXXB95FzoccLnsCb+Joiga1E69488m2HqckppZJD7JvxHI8psomcPIWPBOD QKP3Q4k5irw9bxT+Tt8e0W7d1GFIDbhv2F0oRzlCsZP6ZLr7s2fp+TvjesurYD+uyJNx fI3w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780503082; x=1781107882; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=S0uwFj2oVK8Ym93JzitEXh3ZnJKfgcyu+cm0Jby8VRA=; b=Q+NoSzzdWhbYXFzpXIzFlyRYK1ivtY0t3IcGZXS02om/JINvFqJ6A9swsEyNd2yhWl bufItRSK74d0SPiSYpDoyM/MkooGqFAX4IClQeILv8/6Tj/buqegz5YhrhhUuwm/r6ib WBpajvkaIs3M2nqRWjZpHO7RIWzEGvYGpBJobSjzVKQf4I0PPIGj7mCHCKQlzkTYvNYv clMSGTqMeQ5RdxTL9GdWvuiAqxKElNDbk5DfQk0Z3P2mxrgqK/4tR8mm8/z8tC6JBTYD qxbWV9OF3AefSwdTYRTgiRUMyp5eOx1biCsHKQWi3yrYjG30cZpOANbQgZGJWz5rCgvv QcBQ== X-Gm-Message-State: AOJu0YxZkq5xFywkVx564WxO6fHbuz03wUV2ibohb5sN6f+EfwaOlaDa Oa878SuUglgs+QYIM+zCazbpUeuBe8F/fSRjVPkR9fpI7ZsTujz39JKNCM9OdZc3KUxZRZGLxg= = X-Received: from wmog10.prod.google.com ([2002:a05:600c:310a:b0:490:b360:5841]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:4e87:b0:490:9d1b:f05c with SMTP id 5b1f17b1804b1-490b60e3f91mr74583475e9.33.1780503082036; Wed, 03 Jun 2026 09:11:22 -0700 (PDT) Date: Wed, 3 Jun 2026 18:09:53 +0200 In-Reply-To: <20260603160949.3372482-6-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260603160949.3372482-6-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=1037; i=ardb@kernel.org; h=from:subject; bh=SJHawBMrvWNXo+6oa7fD+p3VxoLw7RWBCVzA/QDwCOg=; b=owGbwMvMwCVmkMcZplerG8N4Wi2JIUsh8FLd1w3h24XNTyeYGLFxHm72u+/DcGNVcbKX3jXWH UnCv092lLIwiHExyIopsgjM/vtu5+mJUrXOs2Rh5rAygQxh4OIUgIn01TIyvHN4t/fK3iV7N0yb H3rcwJP5rHPc4/MJQYtmbpwYdK+Ut4vhr+SPVTMyHFw14pUUEyfzzTXYGGSy5frpGwsWtSz5UpT uwAEA X-Mailer: git-send-email 2.54.0.1013.g208068f2d8-goog Message-ID: <20260603160949.3372482-9-ardb+git@google.com> Subject: [PATCH 3/4] arm64: mte: Disregard the zero page explicitly for manipulating tags From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, will@kernel.org, catalin.marinas@arm.com, maz@kernel.org, Ard Biesheuvel , Kevin Brodsky , Mark Brown Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel The zero page is conceptually immutable, and will be moved into .rodata to prevent inadvertent corruption. Prepare the MTE code for this, by ensuring that the zero page is never taken into account for tag manipulation, given that those actions will no longer be permitted on the read-only alias of .rodata in the linear map. Signed-off-by: Ard Biesheuvel --- arch/arm64/include/asm/mte.h | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/arch/arm64/include/asm/mte.h b/arch/arm64/include/asm/mte.h index 7f7b97e09996..093b34944aee 100644 --- a/arch/arm64/include/asm/mte.h +++ b/arch/arm64/include/asm/mte.h @@ -80,6 +80,11 @@ static inline bool page_mte_tagged(struct page *page) */ static inline bool try_page_mte_tagging(struct page *page) { + extern struct page *__zero_page; + + if (page =3D=3D __zero_page) + return false; + VM_WARN_ON_ONCE(folio_test_hugetlb(page_folio(page))); =20 if (!test_and_set_bit(PG_mte_lock, &page->flags.f)) --=20 2.54.0.1013.g208068f2d8-goog From nobody Mon Jun 8 08:27:58 2026 Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D7B713E51E8 for ; Wed, 3 Jun 2026 16:11:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780503086; cv=none; b=gZxb0g6yMy6UGaKo1fROHGIEXWwomSkxywMtRQ7TzlnZyGfbZSPdwf38lnf9aWhnvyLbUT1GgYoZvRgc5WVRx17He0jMIlaceuM5qljsMNZ1SwM8lhBZ28IaJjgx5X7gaYJ5r3s9lm6MrvMFuhaoD0pHQ7mLUwj5ddfDrU2lRus= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780503086; c=relaxed/simple; bh=Sxq8KhfQah2sz1OaM1MlEZAstsAQ3MB/PdaH812A9kI=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=MzHQwnX4BZJ8N4TOxJcWvrBGwMOvYFqlI545GjRSfkp0ybwkFxd10Gbqa1ULWlsrctSpRVGCA8vgaXmwB1R5gbhq3KQ9YWFlGuu6A4wKppctMWcfWT9Cs+9CIoH+vkxWrO02agk9cx2XKStkCFzngJdBbL4gFUNcozvOAg2zOTs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=FLy6AbVM; arc=none smtp.client-ip=209.85.128.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="FLy6AbVM" Received: by mail-wm1-f74.google.com with SMTP id 5b1f17b1804b1-4904ee02e72so99911445e9.1 for ; Wed, 03 Jun 2026 09:11:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1780503083; x=1781107883; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=oimI3tmTeudUzVEIRzgp5TsbqR86loAKNzvaAFIXh5A=; b=FLy6AbVMAU8lWVNtevG8xlCK+eTTZDshLekmcC/KNLxZZfYieqyuXbVS1Z7+7fOxeB h2qa7XkoOgvdH0ed59SCHuSm0RoSM2VUHl37x4teeAoHWN2jnOSspluJ/DO+aDvPxNxt m7LkyplNsdb+XrgeRDVTtCZc/lBgV2zCgUs72zFGelYUYXk0EJEeXQ5g0PTrnlYNfL0W H6nPrBueVrOyDvQk8bx3xiapRsB/rwiTumEaKV6TqFr/eED2oDFx8G8P+rYJG3s7ZKY/ 4wF1SBmBu3WQ/04PTGEtkA/OTebhQj2kZ4z51SAG1bfvtHQ/eZ6S0nKAQ5Kxmqss7Gn6 lBRA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780503083; x=1781107883; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=oimI3tmTeudUzVEIRzgp5TsbqR86loAKNzvaAFIXh5A=; b=X7c4IynXZQf9nVBLJk3zzoydACcp3pq8eG7pFUGrOZGdbY1z/XQRXDVH1EvV/68MOZ CgcOvaWqj7xXeWMCxFchLpoD9iHQn1UmR024/3wubbMERr72Xjk01Ch0evzlfFs6V0Vv XqfOcauDJ0qGKXIrP2xNwDnSTRVNeZK2FsLM4y7eZ3VuxI25aMZxCQUsHkw8yR+RyZ4A o9B5yPPhsVJh3zbAjRFe48dJSEbjuQQFCnUxirCA0XsX7sg5uV4HaJLZPiR240SP8938 Zq89bTR5APzXo0qevtNJWKae/IyZs0Fd2ywz8FFVQzPH/hSGY1bBm7TFB20wE3/w/ldE F/Xg== X-Gm-Message-State: AOJu0YyasyHH36rsCWbzEWBApQO10i3bsEoOFBu1ZcNJKnitO+8YIIn5 IHQ/UJ9yVYYaTknVQwkVjYtvW53OLtsb8F3SzkmIWa7kb+ogneQuuG33MtYn2Kr36Ufv72xB7w= = X-Received: from wmpy16.prod.google.com ([2002:a05:600c:3410:b0:490:ac8b:3a74]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:3b09:b0:490:b475:af32 with SMTP id 5b1f17b1804b1-490b5e5da1amr70363605e9.23.1780503083096; Wed, 03 Jun 2026 09:11:23 -0700 (PDT) Date: Wed, 3 Jun 2026 18:09:54 +0200 In-Reply-To: <20260603160949.3372482-6-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260603160949.3372482-6-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=2090; i=ardb@kernel.org; h=from:subject; bh=kKVLW1q4ohISCLzJXLX0SgEDWExD4Na9wQ8ubZPYl8I=; b=owGbwMvMwCVmkMcZplerG8N4Wi2JIUsh8Mofu9i66rN/dCOM8tobT3uXTsizYVojeUT/yoE4j ainrLIdpSwMYlwMsmKKLAKz/77beXqiVK3zLFmYOaxMIEMYuDgFYCJ7XzP8M/gj2uC0dkLiVZeP rrlLLOYbiD5v1l/KN+3Svtr9jP+vvmdkWCXPzaSt1VYk1WmqKr5xw5Tnq5nE3OJeKPcssJr9jzG AAwA= X-Mailer: git-send-email 2.54.0.1013.g208068f2d8-goog Message-ID: <20260603160949.3372482-10-ardb+git@google.com> Subject: [PATCH 4/4] arm64: mm: Defer remap of linear alias of data/bss From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, will@kernel.org, catalin.marinas@arm.com, maz@kernel.org, Ard Biesheuvel , Kevin Brodsky , Mark Brown Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel Marking the linear alias of data/bss invalid involves calling set_memory_valid(), which calls split_kernel_leaf_mapping() under the hood. On BBML2_NOABORT capable systems, this may result in the need to allocate page tables at a time when the generic memory allocation APIs are not yet available, resulting in a splat like WARNING: arch/arm64/mm/mmu.c:821 at split_kernel_leaf_mapping+0x15c/0x17= 0, CPU#0: swapper/0 Modules linked in: CPU: 0 UID: 0 PID: 0 Comm: swapper Not tainted 7.1.0-rc6 #1 PREEMPT(unde= f) pstate: a04000c9 (NzCv daIF +PAN -UAO -TCO -DIT -SSBS BTYPE=3D--) pc : split_kernel_leaf_mapping+0x15c/0x170 lr : update_range_prot+0x40/0x128 sp : ffffc99ad3863c80 ... Call trace: split_kernel_leaf_mapping+0x15c/0x170 (P) update_range_prot+0x40/0x128 set_memory_valid+0x94/0xe0 mark_linear_data_alias_valid+0x54/0x68 map_mem+0x1fc/0x240 paging_init+0x48/0x210 setup_arch+0x274/0x338 start_kernel+0x98/0x538 __primary_switched+0x88/0x98 as reported by CKI automated testing. Signed-off-by: Ard Biesheuvel --- arch/arm64/mm/mmu.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index d68e691c093a..3134f1c1097c 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -1095,6 +1095,9 @@ void __init mark_linear_text_alias_ro(void) (unsigned long)__init_begin - (unsigned long)_text, PAGE_KERNEL_RO); =20 + /* Map the kernel data/bss as invalid in the linear map */ + mark_linear_data_alias_valid(false); + /* * Register a PM notifier to remap the linear alias of data/bss as * valid read-only before hibernation. This is needed because the @@ -1237,9 +1240,6 @@ static void __init map_mem(void) __map_memblock(start, end, pgprot_tagged(PAGE_KERNEL), flags); } - - /* Map the kernel data/bss as invalid in the linear map */ - mark_linear_data_alias_valid(false); } =20 void mark_rodata_ro(void) --=20 2.54.0.1013.g208068f2d8-goog