From nobody Mon Jun 8 20:41:28 2026 Received: from mail-wm1-f73.google.com (mail-wm1-f73.google.com [209.85.128.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0F29D3B8BCF for ; Tue, 26 May 2026 17:59:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779818364; cv=none; b=AeL2yG3owWxQfGxP3S8pvb7CWYrCQSS1wwhS6nciR5z93r47bKHZ6hXmxKrgu0iapEeo8daALglwl0FsshBwcV9Mf+krjWZajjnZTadzuYSoBcGQxzt1QR+ZXCZY6qcaQ28e11rzwZiP3/7t5p9ww+rXIlWlM3uJ1xIwXBAIPo0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779818364; c=relaxed/simple; bh=FcYdFUFDbYIAdFyZlCTNmSVC8QDMBmfYYdOjNopJ730=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=lJOgX2rBLYGH1WTmNiUsktOwTqf9rFfMY3Ga/pCGmen9STHwOAOeM1cwNoklFbKQhZ5Rt65+EW9ae9UVVIWQ13aH0Vgc+XLDNYivyK6Dnw2mrlFwL+VE2tFyZYG8SVmOV931C3Q5M7hCGfT0shDy7ojxZb+sd1dKmjjtQ2m8Vc4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=ek+RZk/S; arc=none smtp.client-ip=209.85.128.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="ek+RZk/S" Received: by mail-wm1-f73.google.com with SMTP id 5b1f17b1804b1-490261c79fcso37463235e9.0 for ; Tue, 26 May 2026 10:59:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1779818361; x=1780423161; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=+RFflol+bsWEcr4KlK98zfj4SqOIgycSAYTOMgzlgw0=; b=ek+RZk/SvuyLoWh2fNp249lo3CeBqrG6tOgGVPX9BuzLKKLnkuDxAsEWZr0SLGKklb XJ21kUwz24VbHPEZIi1haVholViaa3Xwa7am1knB2UqPwTsJ97bdZJP2sTM7uQQ/ljvF 70qBdqbYnI2fcHhUdD070oCjtKIgrGpFYGU5s7R0qg1SRCuv/O/iWfU4heeZepJx9VNn MTxWpyGahRE5tQoMv8emS5xukNc74UIb0mtZhonqQZPo70ifGA6vQz9EG4kEeuvp0fiO NLol9Dw4Unt3hBoODHQNxIQwqMgMi8qzm6uo0jZijDodwFahDSjTYIBkK3aCuIP/3Vyb gDyg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1779818361; x=1780423161; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=+RFflol+bsWEcr4KlK98zfj4SqOIgycSAYTOMgzlgw0=; b=nIafhS3ybBo9WYwIOE/R0JXPVh2s2YugXkoyegFrCE2icAfVpk+CMu9vvAIDyhVFzf Nlnv6wTgFK2GT+cOEeCM8OAWX06Y7ImuPC0qquSI4g7fW8Uo82ANIPeVjgP43fKlcIIL aEdoLzyimIts5gaMLvlDse5kxvGYmTv9PNFIwUlHVDXVyrNnN/AzsmKDoTeJq62fVvxA cRIa9bFbpTnJny7x2oZC7fWHbRHMgE1vFt1q9jwCrDD7BGqIJ9kGTfVoA0QgYtHmQJAR KPTjn039ASAbwm+zPtW+YMtBJ/lsS1ssn9nYBdSd79Xw/7KYCj+iWebxuPPnbFCclpEa Ef0A== X-Gm-Message-State: AOJu0Yx6UpIdYFCSE/QLupQnmFbd+oEPAv5KAebWKc6Wcdmh2BGlGD5r UH1IiKM/s0Fgkw44al36SL7KFwV7cg7t5TWJ9Et5CwMPl34Q8cOAC/KjFs/uwtQDD8vhLSbtaw= = X-Received: from wmgb20.prod.google.com ([2002:a05:600c:1514:b0:48f:de5b:469b]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:8599:b0:48f:e230:29f4 with SMTP id 5b1f17b1804b1-490426ade02mr205371775e9.15.1779818361151; Tue, 26 May 2026 10:59:21 -0700 (PDT) Date: Tue, 26 May 2026 19:58:48 +0200 In-Reply-To: <20260526175846.2694125-17-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260526175846.2694125-17-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=1021; i=ardb@kernel.org; h=from:subject; bh=Kh7cxWXkNrSTDvznUsNThby14Lf6cpInZSVAIOoSUVU=; b=owGbwMvMwCVmkMcZplerG8N4Wi2JIUv0fsQvk8viPq9V2vx6dMWennJ+dkQqIjT+cV/VLZF1b R7nhSZ2lLIwiHExyIopsgjM/vtu5+mJUrXOs2Rh5rAygQxh4OIUgImsLWZk2HK0ii1yonQZe7ZV +ApXXrWOD0eLlqwV53zN/y7vkpHmYUaGZx8+meyLTM/2WSaWZW95P1zc+UOGxrXah+2Pw7MWVjW xAgA= X-Mailer: git-send-email 2.54.0.794.g4f17f83d09-goog Message-ID: <20260526175846.2694125-18-ardb+git@google.com> Subject: [PATCH v6 01/15] arm64: mm: Remove bogus stop condition from map_mem() loop From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, Ard Biesheuvel , Ryan Roberts , Anshuman Khandual , Liz Prucka , Seth Jenkins , Kees Cook , Mike Rapoport , David Hildenbrand , Andrew Morton , Jann Horn , linux-mm@kvack.org, linux-hardening@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-sh@vger.kernel.org, Kevin Brodsky Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel The memblock API guarantees that start is not greater than or equal to end, so there is no need to test it. And if it were, it is doubtful that breaking out of the loop would be a reasonable course of action here (rather than attempting to map the remaining regions) So let's drop this check. Reviewed-by: Ryan Roberts Reviewed-by: Kevin Brodsky Signed-off-by: Ard Biesheuvel --- arch/arm64/mm/mmu.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index dd85e093ffdb..112fa4a3b0eb 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -1173,8 +1173,6 @@ static void __init map_mem(pgd_t *pgdp) =20 /* map all the memory banks */ for_each_mem_range(i, &start, &end) { - if (start >=3D end) - break; /* * The linear map must allow allocation tags reading/writing * if MTE is present. Otherwise, it has the same attributes as --=20 2.54.0.794.g4f17f83d09-goog From nobody Mon Jun 8 20:41:28 2026 Received: from mail-wr1-f73.google.com (mail-wr1-f73.google.com [209.85.221.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6057B4218AA for ; Tue, 26 May 2026 17:59:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779818365; cv=none; b=rYrBnxzzTNlBDB2Qj0WKuGTdG0Eh0xvJUqgkawF8vlN0dZEFK8/sUr875HiNs+qO1Jz6qF6a/zbHwfCcf9h3smjDoHtAlYz1qTLpBjJRCYewcaWMJYUuZ3iYl+2UAoNBeRrnbIyf1CwdVo3V8Grv+SAh6EFD7oLiJFcMx28e+T0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779818365; c=relaxed/simple; bh=R1m6k39Kd5OLBqsE8GC497+ARMHkcaNzcA58gMuGmBQ=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=lPVezY2GC5gxBhr4XomgiH6CBdJXIhPqT5A9cbnuKVIBgnkT3sRBsGvjhlEYeaCAl/cWeQXkmv0O81g1j/48JwG2tVi1BFYZBojir7tJK80ZXQNeElOZwFUwEicGY3wORGALPYB20DGq6OAmztsmYLn9GjsmgHVRB0uC5nvceGY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=Dca1DNw9; arc=none smtp.client-ip=209.85.221.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="Dca1DNw9" Received: by mail-wr1-f73.google.com with SMTP id ffacd0b85a97d-44f56d5523eso9666006f8f.1 for ; Tue, 26 May 2026 10:59:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1779818363; x=1780423163; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=yyd3IudRVUHVyeM6Sgx7GyCvuriuV9w8zEGgNDuF+gw=; b=Dca1DNw9jTpNmoIXPfNF0hs9fT8HMj0eW9HN4w3Ad9sS6/w8hwM3TFwRH/3TjM+79b 0M80DJAFEJ071CTxxB9cZDHyIzClrBxd68g83I2RzzvqPwj0F2SMX3JC+2L3pbf7bzNT t1plu8UBlX/n4o5P/4oEQYcUZ9SMGxS4aXqc9WiHO6iO2JwDnMPjTDU6JX83zPZDEtkt sZbmZWgkYfiUuw7idnwsr/gVDvYUrL2UoJSIBVGpztNokG4UcgCfQ5fX+UhvTFbbF7iO ou8QdWuXsKlDVGImLt/HIhBLdK9goptRTdZEBi0jD3gCI2ZXI8xZagOd/qijqfqXuKXN Xkqg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1779818363; x=1780423163; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=yyd3IudRVUHVyeM6Sgx7GyCvuriuV9w8zEGgNDuF+gw=; b=E3oDwW0v2/Gp9TPnCnsW8OQ3EZnOix+3AU/2SlC/xSNsCl9SQkzylmq5uA9ZSqsrML NTxTegiUVazeo2jC25t44ylob7r/Hs5eq34GTXZgN5aufnmbVZT5KgLhvbft84YAqqXr wFM6TQIjveMqat3lPTrajEzurw1oM+sd/W+rN+n9X870ymCYA0i/CYMzKofQFWor+DJ/ xWyMBzlDZZHJS86NQkqvkK0CML96QBvVULrpU1ikw4Z8fIY5Q/dLdSUZ6yuSXtCwXg3t uoJr8Pc2waxU9EZ8NYjcMb6DzYjWkIMpn99rRKZrE+WTyYWXaD6dQ0I5e2NH1HhXCe89 0DPA== X-Gm-Message-State: AOJu0YzlfXkuFQ9wfytH6i7VKDO5p0vwUHbTyDTwL0UTpWfJfvmkMUqk lzphvvvaxCrkWLIdFzs/HzIMkkx9FvFKsvL5oVCA252AR8QgQbj/kDsh1k6W5x1e07CaIfAJHQ= = X-Received: from wrwy1.prod.google.com ([2002:a05:6000:1081:b0:44c:3df5:500e]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6000:4917:b0:43d:dd:8ca4 with SMTP id ffacd0b85a97d-45eb36ab5c9mr33376312f8f.14.1779818362409; Tue, 26 May 2026 10:59:22 -0700 (PDT) Date: Tue, 26 May 2026 19:58:49 +0200 In-Reply-To: <20260526175846.2694125-17-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260526175846.2694125-17-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=3439; i=ardb@kernel.org; h=from:subject; bh=vIIm6COov+NQzX2nIS8cvw7su4mTKmzskcBcPmulqnI=; b=owGbwMvMwCVmkMcZplerG8N4Wi2JIUv0fpS02zOfRUy3961i3vC1b7n+Evm+/R38NgG/DlrXZ Qm+/ezWUcrCIMbFICumyCIw+++7nacnStU6z5KFmcPKBDKEgYtTACYyZREjw8TXaqzPetiWhH24 cp5p0vNHbzoXsV7Pm+XWwVYqvcHR9SUjw6wVekazft8qKTufNy9Z5jRL5boX02v9d1fyMKbXF/Q c4gAA X-Mailer: git-send-email 2.54.0.794.g4f17f83d09-goog Message-ID: <20260526175846.2694125-19-ardb+git@google.com> Subject: [PATCH v6 02/15] arm64: mm: Drop redundant pgd_t* argument from map_mem() From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, Ard Biesheuvel , Ryan Roberts , Anshuman Khandual , Liz Prucka , Seth Jenkins , Kees Cook , Mike Rapoport , David Hildenbrand , Andrew Morton , Jann Horn , linux-mm@kvack.org, linux-hardening@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-sh@vger.kernel.org, Kevin Brodsky Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel __map_memblock() and map_mem() always operate on swapper_pg_dir, so there is no need to pass around a pgd_t pointer between them. Reviewed-by: Ryan Roberts Reviewed-by: Kevin Brodsky Signed-off-by: Ard Biesheuvel --- arch/arm64/mm/mmu.c | 25 ++++++++++---------- 1 file changed, 12 insertions(+), 13 deletions(-) diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index 112fa4a3b0eb..aa0e2c6435f7 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -1035,11 +1035,11 @@ static void update_mapping_prot(phys_addr_t phys, u= nsigned long virt, flush_tlb_kernel_range(virt, virt + size); } =20 -static void __init __map_memblock(pgd_t *pgdp, phys_addr_t start, - phys_addr_t end, pgprot_t prot, int flags) +static void __init __map_memblock(phys_addr_t start, phys_addr_t end, + pgprot_t prot, int flags) { - early_create_pgd_mapping(pgdp, start, __phys_to_virt(start), end - start, - prot, early_pgtable_alloc, flags); + early_create_pgd_mapping(swapper_pg_dir, start, __phys_to_virt(start), + end - start, prot, early_pgtable_alloc, flags); } =20 void __init mark_linear_text_alias_ro(void) @@ -1087,13 +1087,13 @@ static phys_addr_t __init arm64_kfence_alloc_pool(v= oid) return kfence_pool; } =20 -static void __init arm64_kfence_map_pool(phys_addr_t kfence_pool, pgd_t *p= gdp) +static void __init arm64_kfence_map_pool(phys_addr_t kfence_pool) { if (!kfence_pool) return; =20 /* KFENCE pool needs page-level mapping. */ - __map_memblock(pgdp, kfence_pool, kfence_pool + KFENCE_POOL_SIZE, + __map_memblock(kfence_pool, kfence_pool + KFENCE_POOL_SIZE, pgprot_tagged(PAGE_KERNEL), NO_BLOCK_MAPPINGS | NO_CONT_MAPPINGS); memblock_clear_nomap(kfence_pool, KFENCE_POOL_SIZE); @@ -1129,11 +1129,11 @@ bool arch_kfence_init_pool(void) #else /* CONFIG_KFENCE */ =20 static inline phys_addr_t arm64_kfence_alloc_pool(void) { return 0; } -static inline void arm64_kfence_map_pool(phys_addr_t kfence_pool, pgd_t *p= gdp) { } +static inline void arm64_kfence_map_pool(phys_addr_t kfence_pool) { } =20 #endif /* CONFIG_KFENCE */ =20 -static void __init map_mem(pgd_t *pgdp) +static void __init map_mem(void) { static const u64 direct_map_end =3D _PAGE_END(VA_BITS_MIN); phys_addr_t kernel_start =3D __pa_symbol(_text); @@ -1178,7 +1178,7 @@ static void __init map_mem(pgd_t *pgdp) * if MTE is present. Otherwise, it has the same attributes as * PAGE_KERNEL. */ - __map_memblock(pgdp, start, end, pgprot_tagged(PAGE_KERNEL), + __map_memblock(start, end, pgprot_tagged(PAGE_KERNEL), flags); } =20 @@ -1192,10 +1192,9 @@ static void __init map_mem(pgd_t *pgdp) * Note that contiguous mappings cannot be remapped in this way, * so we should avoid them here. */ - __map_memblock(pgdp, kernel_start, kernel_end, - PAGE_KERNEL, NO_CONT_MAPPINGS); + __map_memblock(kernel_start, kernel_end, PAGE_KERNEL, NO_CONT_MAPPINGS); memblock_clear_nomap(kernel_start, kernel_end - kernel_start); - arm64_kfence_map_pool(early_kfence_pool, pgdp); + arm64_kfence_map_pool(early_kfence_pool); } =20 void mark_rodata_ro(void) @@ -1417,7 +1416,7 @@ static void __init create_idmap(void) =20 void __init paging_init(void) { - map_mem(swapper_pg_dir); + map_mem(); =20 memblock_allow_resize(); =20 --=20 2.54.0.794.g4f17f83d09-goog From nobody Mon Jun 8 20:41:28 2026 Received: from mail-ed1-f74.google.com (mail-ed1-f74.google.com [209.85.208.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B580B421EF3 for ; Tue, 26 May 2026 17:59:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779818367; cv=none; b=a+E0naYrZoWvrMGnRYC0BfeqiIIThk1BYgBJf6/AlrxYLCjfamav3Dnm0Nsm64PW50rdx3Jb0g4fdhrinfVs5oT93m4D/ZkDyaCJ2Lnm7jaaGKdMVLyV5LQK5n2BbP7LeD7sO8xb9OJiX3tbNbglHTjEuBaZyxFOZluWfSzIyG4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779818367; c=relaxed/simple; bh=4/+Et9+8M/9XRBpvIOadBWxJIjJHwMIcMP8jy/wEGGU=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=r1k/AM9GeETlo9eqcb6RQHmiZCGFigkJSe+WMjpwMLYSoTps6SKNxfSh+NrTJmd2kxKrPKx6l96p3HyFW1tK5wQSQbzJOCPykZdnNytKE1CDaGQ9TmlKrDGoYCYyD/ucpcfVxvlm7DrZclw6IUAN67oVGzXn/Ry2WQABgOHN93g= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=YXkJ0zFh; arc=none smtp.client-ip=209.85.208.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="YXkJ0zFh" Received: by mail-ed1-f74.google.com with SMTP id 4fb4d7f45d1cf-68751570301so6069321a12.3 for ; Tue, 26 May 2026 10:59:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1779818364; x=1780423164; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=LtBeFQlOADPYK4gJNp1okGIhaF4i6t2Ouexf2+2MrWs=; b=YXkJ0zFhtozYojxxx7riz+o5C/AtqKMm55r9+yQVH7MxFbEHK8AbPlr1ivjFRJd2i7 XIjDqdwZHFCZO7vdcI0XKwUDkbvp2FAOYFbBpHdbTTrf0CWfo03SyZZNmZcmR3pAvjm4 PF6bHtEf7VB6RX+tW+fjnbJcfKr+YTGy35JpI1fp0k9NY/bKxvjhKOpin4cUqxsexhYa CuxWcL48AuH95IAlHib377zJgAn/YbccToova5zCPSpmLrOm65XTyaL3Twq6Jc3A2cva RXk2cD34cNdKg60YbgQtZRTiV+mEgYMMxSz+PoF/8hJ0hJNad/DtyvzW8lNbgf1vTyQ0 lGZA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1779818364; x=1780423164; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=LtBeFQlOADPYK4gJNp1okGIhaF4i6t2Ouexf2+2MrWs=; b=G4otgmGD2zER3PQPW0uW1OtNU8bTB/fKBkqzeHBbRhNwOPeEmu1WrxBBu2HRlVIN3N BETHe9XcJe43ujzEBtmIW3XWlVDaLFRfHhq4FN+w8VUG9Q/33nvgAWRN4qNTdHIzQUMk E2qMN5sUYQJCJACIBozMcK6ke3fC7nbhJb/+FITiet5Xh9MseEHUIRy6R/jkqAOYDsNr Q3dDt31wd1kjhpmXd2nU2K1Os6jnC/NvagPhhS+jij4x4nF+YgX5chxk002F4KhVWCJ3 +RQLv2pAHk9cCu2syVjrKL3Bi59/aZQwbkQIcbGf9yGAZMcZ+l49Ex7rqD0BCPPuHkh3 M+Ag== X-Gm-Message-State: AOJu0YzYXxoh8ta5OGLefNqQcP2TvX6/j6LnU1Z0in1g8Wpz5eRA2fuT pSAnZh9SHgEQzfqBekOYWqUg14hzfFnl6W8fEUkCoy8zhrtHuMn8gY6dAXADPOgL4rY6S6d1wA= = X-Received: from edye18.prod.google.com ([2002:a05:6402:892:b0:67c:573d:d3a0]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6402:40d4:b0:67b:cd1f:9cc1 with SMTP id 4fb4d7f45d1cf-6889c445088mr10125279a12.6.1779818363764; Tue, 26 May 2026 10:59:23 -0700 (PDT) Date: Tue, 26 May 2026 19:58:50 +0200 In-Reply-To: <20260526175846.2694125-17-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260526175846.2694125-17-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=2121; i=ardb@kernel.org; h=from:subject; bh=UYuabKAGVqw/38U/k7XQfrNPG5ThRXq/7mqnsjcGnSk=; b=owGbwMvMwCVmkMcZplerG8N4Wi2JIUv0fvTm0BsvTTTaUheFf26uWyada7zXR7A5ui2rSTZi1 o2FJrwdpSwMYlwMsmKKLAKz/77beXqiVK3zLFmYOaxMIEMYuDgFYCKHcxj+Cl+U+L3mz31fZxuh tHcrzdsEq/i1War/p2x78yRMa8USUUaGzw8WujlZRJw7v672d9asvI8f57TITLjpqnwn3Gp2wO5 +NgA= X-Mailer: git-send-email 2.54.0.794.g4f17f83d09-goog Message-ID: <20260526175846.2694125-20-ardb+git@google.com> Subject: [PATCH v6 03/15] arm64: mm: Check for pud_/pmd_set_huge() failures on kernel mappings From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, Ard Biesheuvel , Ryan Roberts , Anshuman Khandual , Liz Prucka , Seth Jenkins , Kees Cook , Mike Rapoport , David Hildenbrand , Andrew Morton , Jann Horn , linux-mm@kvack.org, linux-hardening@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-sh@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel Sashiko reports: | If pmd_set_huge() rejects an unsafe page table transition (such as | mapping a different physical address over an existing block mapping), | it returns 0 and leaves the page table entry unmodified. | | Because *pmdp remains unmodified, READ_ONCE(pmd_val(*pmdp)) will equal | pmd_val(old_pmd). The transition from old_pmd to old_pmd is evaluated | as safe by pgattr_change_is_safe(), so the BUG_ON never triggers. | | This allows invalid and unsafe mapping updates to be silently dropped | instead of panicking, leaving stale memory mappings active while the | caller assumes the update was successful. The same applies to pud_set_huge() in alloc_init_pud(). Given how it is generally preferred to limp on rather than blow up the system if an unexpected condition such as this one occurs, and the fact that there are no known cases where this disparity results in real problems, let's WARN on these failures rather than BUG, allowing the system to survive to the point where it can actually report them. Signed-off-by: Ard Biesheuvel --- arch/arm64/mm/mmu.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index aa0e2c6435f7..b2ba5b35c35f 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -257,7 +257,7 @@ static int init_pmd(pmd_t *pmdp, unsigned long addr, un= signed long end, /* try section mapping first */ if (((addr | next | phys) & ~PMD_MASK) =3D=3D 0 && (flags & NO_BLOCK_MAPPINGS) =3D=3D 0) { - pmd_set_huge(pmdp, phys, prot); + WARN_ON(!pmd_set_huge(pmdp, phys, prot)); =20 /* * After the PMD entry has been populated once, we @@ -380,7 +380,7 @@ static int alloc_init_pud(p4d_t *p4dp, unsigned long ad= dr, unsigned long end, if (pud_sect_supported() && ((addr | next | phys) & ~PUD_MASK) =3D=3D 0 && (flags & NO_BLOCK_MAPPINGS) =3D=3D 0) { - pud_set_huge(pudp, phys, prot); + WARN_ON(!pud_set_huge(pudp, phys, prot)); =20 /* * After the PUD entry has been populated once, we --=20 2.54.0.794.g4f17f83d09-goog From nobody Mon Jun 8 20:41:28 2026 Received: from mail-wm1-f73.google.com (mail-wm1-f73.google.com [209.85.128.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 27004423A68 for ; Tue, 26 May 2026 17:59:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779818368; cv=none; b=nRP+kvSIMI/PksiFdJBSUfx3dDHPspj2aj/pu2SttMyJQf9TPDrIh93R2kb5vR1/1Ge/YCIfF5covgIQOMdxp+CeGR3emdKOc8UnbT37kcYoAygHn5T8b64Rz8K7IBIM+6aYMlnM2MXmTWdUdw7noC53eVnFY6pYYsEtn/OxdN0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779818368; c=relaxed/simple; bh=V2TOu05l3UzOFtNXRBGFhB28KhfoYTIhb2a8yVyUWJ4=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=SVUhZK09GIuMmOwaB5MHju40bRdjiv9j0ZGA3WIvHgMI4TpBuZpazEk9MwLrwPMC/CpisVoIMRtXku2F/COqdwR/o20lEPOsJQ1+mre6tI4gNLdnLRo73YtokFOfsZxm+Gsvpp17h/fAN4DexiubQHOs9ZWKdZQuRhYQKtiVg40= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=lNevinoE; arc=none smtp.client-ip=209.85.128.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="lNevinoE" Received: by mail-wm1-f73.google.com with SMTP id 5b1f17b1804b1-49048e21ea7so20745575e9.1 for ; Tue, 26 May 2026 10:59:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1779818365; x=1780423165; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=3mHEuoeqgeJwwKsR5vVxh3j7NIc/RKdm0H7T5NxNZG0=; b=lNevinoEY4bvcQ8lEo1Y4FGFq5erahRUD8NrbawyUWXfnGJQuvjLaGZpoLLzHcAH4P fdKe7f88EBZGiazEkGQgojvk/15y6eqqmYlBz6HECx2tjUpIHKYMM1MtquZD+LCpJ22V lHqiODgB5yoslmeO3UluJVowpsy3RKWp+P8ooiBzpuxhnA61uvMiK4Zs1DKa5MSH9kYX MrqUzQ4i9FceSYU44yg3pC7oalu8GaQahP1J6kqWlErkrkoCmCEldCnuujObsNZNJGxw hm2N85zV3yYabVqrQFZ7pw/Vde8YYrIJH19A1XEjWsJ/tC5niKQmd5swqrTM0p+gmjt4 sPIQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1779818365; x=1780423165; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=3mHEuoeqgeJwwKsR5vVxh3j7NIc/RKdm0H7T5NxNZG0=; b=FhAeQtWva5fVnqheJ4tHNoTHmhpzkJNGOIREALiqGDwtsuvXJedqFEDLkXqzH6Km26 bLjY8zHFPeqCmJ7PEKcRLCnPdPCuOITdsGijQjUoNSDkEYatdDeUQPkGTunp4AI7s0YY fsjXJkI0ATNq9CBAN6idRqpqi7Ek0PsLWskw2lzwuBPAxquCo0copO/G4OYHoXuuin6A EUOoqLE4LenDO7eYF1oNUWwFDGgaDDqE6OA2M1CfflVPDkEVluI/8eflZXEzaFvvXk5c 9TZC9bqG9mWC4OSfFsnH3TqKFLpSn/a+u5WssCwAEbYbNRobC6Hh1wddCi1ZDDnearkm d2Bw== X-Gm-Message-State: AOJu0YxN35mq1gdse4xC7c3DGQXnOmr2wEZcJ3cgOLyC/YgQvbpBQ6KA sxufpgI4iS/Ml0V/3OSLzaMNG+cDE6O3Swy/CJptSbrVx6WZzo+lbIi+zVN7HsxSLHl5jtNHKA= = X-Received: from wmbh6.prod.google.com ([2002:a05:600c:a106:b0:48f:de29:3f52]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:a402:b0:48f:e230:29f5 with SMTP id 5b1f17b1804b1-490426b0c92mr233953415e9.16.1779818365325; Tue, 26 May 2026 10:59:25 -0700 (PDT) Date: Tue, 26 May 2026 19:58:51 +0200 In-Reply-To: <20260526175846.2694125-17-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260526175846.2694125-17-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=1360; i=ardb@kernel.org; h=from:subject; bh=HTVCAgvGTLScrJ2CZV4D8VSId3uhva+Scq8FVpoE17Y=; b=owGbwMvMwCVmkMcZplerG8N4Wi2JIUv0fgzPcSP+Z1uPSc089cXOMuHXyr/rHy2TE5mx/o6dx X9ece7OjlIWBjEuBlkxRRaB2X/f7Tw9UarWeZYszBxWJpAhDFycAjARxViG/1GnxVK2TMh4vX6H yFPbSV03ey81Ml2OaYnZZ/vlkGzNih2MDIuvZ3AHycz03hNo9ypyg7OB0C2/DLkSk46sXolZjBl hfAA= X-Mailer: git-send-email 2.54.0.794.g4f17f83d09-goog Message-ID: <20260526175846.2694125-21-ardb+git@google.com> Subject: [PATCH v6 04/15] arm64: mm: Preserve existing table mappings when mapping DRAM From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, Ard Biesheuvel , Ryan Roberts , Anshuman Khandual , Liz Prucka , Seth Jenkins , Kees Cook , Mike Rapoport , David Hildenbrand , Andrew Morton , Jann Horn , linux-mm@kvack.org, linux-hardening@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-sh@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel Instead of blindly overwriting an existing table entry when mapping DRAM regions, take care not to replace a pre-existing table entry with a block entry. This permits the logic of mapping the kernel's linear alias to be simplified in a subsequent patch. Reviewed-by: Ryan Roberts Signed-off-by: Ard Biesheuvel --- arch/arm64/mm/mmu.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index b2ba5b35c35f..5c827fa3cd38 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -256,7 +256,8 @@ static int init_pmd(pmd_t *pmdp, unsigned long addr, un= signed long end, =20 /* try section mapping first */ if (((addr | next | phys) & ~PMD_MASK) =3D=3D 0 && - (flags & NO_BLOCK_MAPPINGS) =3D=3D 0) { + (flags & NO_BLOCK_MAPPINGS) =3D=3D 0 && + !pmd_table(old_pmd)) { WARN_ON(!pmd_set_huge(pmdp, phys, prot)); =20 /* @@ -379,7 +380,8 @@ static int alloc_init_pud(p4d_t *p4dp, unsigned long ad= dr, unsigned long end, */ if (pud_sect_supported() && ((addr | next | phys) & ~PUD_MASK) =3D=3D 0 && - (flags & NO_BLOCK_MAPPINGS) =3D=3D 0) { + (flags & NO_BLOCK_MAPPINGS) =3D=3D 0 && + !pud_table(old_pud)) { WARN_ON(!pud_set_huge(pudp, phys, prot)); =20 /* --=20 2.54.0.794.g4f17f83d09-goog From nobody Mon Jun 8 20:41:28 2026 Received: from mail-ed1-f73.google.com (mail-ed1-f73.google.com [209.85.208.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7A25B426680 for ; Tue, 26 May 2026 17:59:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779818370; cv=none; b=COdd2jcNAcxfO4SGXx0yOQFlCb3Md/2L/TN88CnZVlq3NTE0XAO4Xs+kWrBDe9aEVYuyL86NW+wVV26TvNNestqgHZyPadPwj0coKVMLAZDLNsHjO4K+rM28COlrEnptp4oGX2c/NGwnAeEllsLvb0izE3grd6yGAX4TV5AIIqQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779818370; c=relaxed/simple; bh=bTvTY7THc5d7WTIDs3aH2OJ7lspQKDu8wsfUcID9dlw=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Geur7aRyZU+g+Nqw6hpOW1bQVm0GHoZaFsNECYynIzVDBqt7KCzyObcYMqmUJWo4yl2IlRs9bJ8KsYZL7Ytfh2GLXBUDjY3YXKL/p0+e20vlh+aI8CFB/Y5dA07SAdYF2QAsK3YGtK4jv5hfU5sHL5UnLBHM3QjlkQVUfVyt5j4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=OJuTOyhC; arc=none smtp.client-ip=209.85.208.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="OJuTOyhC" Received: by mail-ed1-f73.google.com with SMTP id 4fb4d7f45d1cf-67f7c414c58so10068244a12.1 for ; Tue, 26 May 2026 10:59:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1779818367; x=1780423167; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=cVtRZrGktzRjCdUnH3kN7Vy8R0F1sW5LmttIAYz/k8A=; b=OJuTOyhC+6j9IsXOWQecMnCypGZohsJ8ODZBl9/d5oI0kBn2OeqkSbRu2OVoZBznqu gRr+wRnmUseXHodoy6Ip8B5DKRnBb+hBwPi8oTBaNR8f5wye7Bhehl/RQh+/q7HRBL7N ewHYzSzjh3zH6rKVCaI0eYgYrEZd6rm7zJTw9pGZZbdj/rzqPA3IajTPKtCI19QrKLfh g9C3LvtACsvJMp844W61jSQPKCuF0KCRFdyZJFO/cSsA5RinexdIOotnL7L2IXZ6vxn9 u+ewOeSvfruXAf66QHZwVZydXY/PBnAM9X5AaEso4u/Revq/aE3udFHuzNncYSmWzqDR jKug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1779818367; x=1780423167; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=cVtRZrGktzRjCdUnH3kN7Vy8R0F1sW5LmttIAYz/k8A=; b=RI0KuQq60R4F9wDHDAhA9ygCktQxHGOZrZl80UJxj577a0BKgMAcwGeDyKeSo8NCIO GJI86jDtgBS6g3v+C3N6H391HyVNajmlALkDAjDPeuGKeIKTwaO1cupTTfgQLGso2YA1 Gmfnn3Dvm5geGmJ9XdC+z/XzuhWjSfnGrIi7VdTNmXbAZLkZ0O8XWyhVs50uvdDq52p5 TqR6lPEvpIowZAHvmVGF9g6HURwgVfbRjKR5BhC6nEHji0LfZMf5eeksxsXgZpwJQKj7 +rhsO/jn39yciS0fv6Cf01c5aR1oLXINXcU0/sH5k4HDZTwtzqW4yH6nXNRzrhE33PS7 dwuw== X-Gm-Message-State: AOJu0YxPb4h3tnaoSrN8eEzd0MewS7pPRdoQYk4TSCrpPi94ggilhLDj pz44xYJypy8SjxNtJYeDUFJPl/7ocs2LeQkFkj0zH3wv0tYxNcpVEvMhbQ0DABu0zjXegRZVgA= = X-Received: from edgg7.prod.google.com ([2002:a05:6402:1ec7:b0:687:a46:a91a]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6402:3891:b0:683:1cc8:84b0 with SMTP id 4fb4d7f45d1cf-6889c41ece7mr9857461a12.1.1779818366685; Tue, 26 May 2026 10:59:26 -0700 (PDT) Date: Tue, 26 May 2026 19:58:52 +0200 In-Reply-To: <20260526175846.2694125-17-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260526175846.2694125-17-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=3700; i=ardb@kernel.org; h=from:subject; bh=z6XqclO2VktRYngIWmFbVHfouJpd6sLXvnE8xwKtYlk=; b=owGbwMvMwCVmkMcZplerG8N4Wi2JIUv0flxH4ekTmyZ0HhJ9JL0wwCSxxnH9nteLX//dwO4dq 58gueBoRykLgxgXg6yYIovA7L/vdp6eKFXrPEsWZg4rE8gQBi5OAZhI3R2G36zF1sq/DnmIFKhu n6+5tapE4TVTQeM123nvlwhbO97ZVc/wV4jD64xshEfNl9JbBTcNtzWsVTgTNfuZcwu3tFeOvPJ UZgA= X-Mailer: git-send-email 2.54.0.794.g4f17f83d09-goog Message-ID: <20260526175846.2694125-22-ardb+git@google.com> Subject: [PATCH v6 05/15] arm64: mm: Preserve non-contiguous descriptors when mapping DRAM From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, Ard Biesheuvel , Ryan Roberts , Anshuman Khandual , Liz Prucka , Seth Jenkins , Kees Cook , Mike Rapoport , David Hildenbrand , Andrew Morton , Jann Horn , linux-mm@kvack.org, linux-hardening@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-sh@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel Instead of blindly overwriting existing live entries regardless of the value of their contiguous bit when mapping DRAM regions at contiguous-hint granularity, check whether the contiguous region in question contains any valid descriptors that have the contiguous bit cleared, and in that case, leave the contiguous bit unset on the entire region. This permits the logic of mapping the kernel's linear alias to be simplified in a subsequent patch. Note that this can only result in a misprogrammed contiguous bit (as per ARM ARM RNGLXZ) if the region in question already contains a mix of valid contiguous and valid non-contiguous descriptors, in which case it was already misprogrammed to begin with. Signed-off-by: Ard Biesheuvel --- arch/arm64/include/asm/pgtable.h | 4 ++++ arch/arm64/mm/mmu.c | 22 ++++++++++++++++++-- 2 files changed, 24 insertions(+), 2 deletions(-) diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgta= ble.h index 4dfa42b7d053..a1c5894332d9 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -181,6 +181,10 @@ static inline pteval_t __phys_to_pte_val(phys_addr_t p= hys) * Returns true if the pte is valid and has the contiguous bit set. */ #define pte_valid_cont(pte) (pte_valid(pte) && pte_cont(pte)) +/* + * Returns true if the pte is valid and has the contiguous bit cleared. + */ +#define pte_valid_noncont(pte) (pte_valid(pte) && !pte_cont(pte)) /* * Could the pte be present in the TLB? We must check mm_tlb_flush_pending * so that we don't erroneously return false for pages that have been diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index 5c827fa3cd38..6b42d724bd1b 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -187,6 +187,14 @@ static void init_pte(pte_t *ptep, unsigned long addr, = unsigned long end, } while (ptep++, addr +=3D PAGE_SIZE, addr !=3D end); } =20 +static bool pte_range_has_valid_noncont(pte_t *ptep) +{ + for (int i =3D 0; i < CONT_PTES; i++) + if (pte_valid_noncont(__ptep_get(&ptep[i]))) + return true; + return false; +} + static int alloc_init_cont_pte(pmd_t *pmdp, unsigned long addr, unsigned long end, phys_addr_t phys, pgprot_t prot, @@ -224,7 +232,8 @@ static int alloc_init_cont_pte(pmd_t *pmdp, unsigned lo= ng addr, =20 /* use a contiguous mapping if the range is suitably aligned */ if ((((addr | next | phys) & ~CONT_PTE_MASK) =3D=3D 0) && - (flags & NO_CONT_MAPPINGS) =3D=3D 0) + (flags & NO_CONT_MAPPINGS) =3D=3D 0 && + !pte_range_has_valid_noncont(ptep)) __prot =3D __pgprot(pgprot_val(prot) | PTE_CONT); =20 init_pte(ptep, addr, next, phys, __prot); @@ -283,6 +292,14 @@ static int init_pmd(pmd_t *pmdp, unsigned long addr, u= nsigned long end, return 0; } =20 +static bool pmd_range_has_valid_noncont(pmd_t *pmdp) +{ + for (int i =3D 0; i < CONT_PMDS; i++) + if (pte_valid_noncont(pmd_pte(READ_ONCE(pmdp[i])))) + return true; + return false; +} + static int alloc_init_cont_pmd(pud_t *pudp, unsigned long addr, unsigned long end, phys_addr_t phys, pgprot_t prot, @@ -324,7 +341,8 @@ static int alloc_init_cont_pmd(pud_t *pudp, unsigned lo= ng addr, =20 /* use a contiguous mapping if the range is suitably aligned */ if ((((addr | next | phys) & ~CONT_PMD_MASK) =3D=3D 0) && - (flags & NO_CONT_MAPPINGS) =3D=3D 0) + (flags & NO_CONT_MAPPINGS) =3D=3D 0 && + !pmd_range_has_valid_noncont(pmdp)) __prot =3D __pgprot(pgprot_val(prot) | PTE_CONT); =20 ret =3D init_pmd(pmdp, addr, next, phys, __prot, pgtable_alloc, flags); --=20 2.54.0.794.g4f17f83d09-goog From nobody Mon Jun 8 20:41:28 2026 Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A9A324218AA for ; Tue, 26 May 2026 17:59:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779818371; cv=none; b=UN/FflbVe0gKlTbdD2c5X1ac61apVGuu5Y7y/6JH4StrWFVMIW7rXQbkueCVtYV42UJ9a5/OylFE8pxfNjQQFTsi8dtnnCHlTX/f56J6YwNc8qRFcZogpAs6AJrKGQr8SoL/6eayp9BkUsbhJSCYVrd8l7LFlxGFKPV+L1VpXGQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779818371; c=relaxed/simple; bh=X2BtPkz0m6On3LJePhaEPBMybWT3ReRxX78S/Z3E1Tc=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=iHfLE0sQ+15axVeDNfmaMwx+hW6R8r3TRiPF+PQElqJp8Y7fjGEl271iP1FyP8F9P4J6MKw6lZV085FEutU/FEnrCwOZUJy6Cm+YwhWirDagBNbqN9HQKMuspjoR+MVUQDhXXMTS081KONK8qmE8Dy3Is1GjGfyfOqWexjJ9iGQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=NMWU1BpP; arc=none smtp.client-ip=209.85.128.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="NMWU1BpP" Received: by mail-wm1-f74.google.com with SMTP id 5b1f17b1804b1-49050c44bcdso44502135e9.3 for ; Tue, 26 May 2026 10:59:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1779818368; x=1780423168; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=cNZ6xElfqK3A4SE3F9P829uWscll1GihYVBHckeq6A0=; b=NMWU1BpPKkvro0jmCWXTu386hXiF08MauOubrfPycNwgjYYL96IGYoRjF1UTuNwgIz N0tyY7OUdOCNUA+Oz/WstbIzM+qqQD+83EoqYDx5t95fh08w0GNgKlOg7YlLXL+fc9ZH SHeB+HGKyTDik4dD/gN3JHj6LbY/OPosKxbL0HElkBzfdJiXVoB/OWZBcpVVc5ZGHjkT otaKRDnbOL8vo8pHDcthInV+/+tWXqLJwlRx7ZsVGimiV27bn1YL7e9USQlBLFVZmBYH CbvCPgrH0FZMhluFvXqQ4STBDzbdJhnS3aok/XIqha34YwmXGqvbHQrDtgNebONY2Amn leXg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1779818368; x=1780423168; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=cNZ6xElfqK3A4SE3F9P829uWscll1GihYVBHckeq6A0=; b=Mq3rGtyGv6TGu+TCHycYWMEZHscpkwWifdlEMxzXIj8j2chKlOipvkmYkhK1R0GxUj 5oM9iB4k0+pjpPP1hIogWCigs1CHXmGiA8lb/fmj7E+PiyoEKphgiUC3Hn101t6YxRGZ HjE3C4OlY+8TCARJSw7DXYdkMvyDxArbUqrgSHj6yxjTqCmVjD+E5kF5dvr0V673Lmjq 55TTPkyMOjzDDugZpBLQnonoJd0wKPkzgqBuy/8F8C2kesrxxIcxr/9d/kP4Z0MZRPB2 mthxCqval6JRAik0hqZO27S4JCcsYs0E6+3cgP1fYjrtVQVFcob3Ol/bZ8t0O9O3aunr Csjw== X-Gm-Message-State: AOJu0YySk2x4Moeuh29aKsPzOi+4b6VPx/sxsgekt0K8Cny5HkfMKqqh 5bARC2a8X0mtYTKEg7ocd6czVI5pboq/eNSpFuXjz6XAeySBDwhFkBJb6gM9HK+O16j//fAp6A= = X-Received: from wmbje7.prod.google.com ([2002:a05:600c:1f87:b0:48a:5970:2007]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:3b97:b0:490:52fb:12dd with SMTP id 5b1f17b1804b1-49052fb14fdmr292012705e9.10.1779818367951; Tue, 26 May 2026 10:59:27 -0700 (PDT) Date: Tue, 26 May 2026 19:58:53 +0200 In-Reply-To: <20260526175846.2694125-17-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260526175846.2694125-17-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=1543; i=ardb@kernel.org; h=from:subject; bh=0Wnsvtw0Z2vu8jTM+plTHkuWXyq+RnVgc0hID3nHEQ0=; b=owGbwMvMwCVmkMcZplerG8N4Wi2JIUv0fnxSZsWx12f761qXqC0qS65U2ZVyr3Slf87jmSwTV KRvvbncUcrCIMbFICumyCIw+++7nacnStU6z5KFmcPKBDKEgYtTACbSLsjIsFvkWXjX0hVFJxZW Ldt97vPXdU3PRc8XH9GZ8UZlg9pB7T+MDA9i+Q/xn5Hz+Zl1U29m5LMYieUfPCfvSXROOpI/3bA ngBUA X-Mailer: git-send-email 2.54.0.794.g4f17f83d09-goog Message-ID: <20260526175846.2694125-23-ardb+git@google.com> Subject: [PATCH v6 06/15] arm64: mm: Permit contiguous descriptors to be manipulated From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, Ard Biesheuvel , Ryan Roberts , Anshuman Khandual , Liz Prucka , Seth Jenkins , Kees Cook , Mike Rapoport , David Hildenbrand , Andrew Morton , Jann Horn , linux-mm@kvack.org, linux-hardening@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-sh@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel Currently, pgattr_change_is_safe() is overly pedantic when it comes to descriptors with the contiguous hint attribute set, as it rejects assignments even if the old and the new value are the same. In fact, as per ARM ARM RJQQTC, manipulating descriptors with the contiguous bit set is safe as long as the bit itself does not change value, in the sense that no TLB conflict aborts or other exceptions may be raised as a result. Inconsistent permission attributes within the contiguous region may result in any of the alternatives to be taken to apply to the entire region, which might be a programming error, but it does not constitute an unsafe manipulation in terms of what pgattr_change_is_safe() is intended to detect. So drop the special PTE_CONT check, but still omit PTE_CONT from 'mask' so that modifying the bit is still regarded as unsafe. Signed-off-by: Ard Biesheuvel --- arch/arm64/mm/mmu.c | 4 ---- 1 file changed, 4 deletions(-) diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index 6b42d724bd1b..d7a6991e1844 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -134,10 +134,6 @@ bool pgattr_change_is_safe(pteval_t old, pteval_t new) if (pte_pfn(__pte(old)) !=3D pte_pfn(__pte(new))) return false; =20 - /* live contiguous mappings may not be manipulated at all */ - if ((old | new) & PTE_CONT) - return false; - /* Transitioning from Non-Global to Global is unsafe */ if (old & ~new & PTE_NG) return false; --=20 2.54.0.794.g4f17f83d09-goog From nobody Mon Jun 8 20:41:28 2026 Received: from mail-wr1-f74.google.com (mail-wr1-f74.google.com [209.85.221.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 628D1426D2B for ; Tue, 26 May 2026 17:59:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779818373; cv=none; b=sdi61P3dMZnvkrSf1LxdeiRUWseEJeVzVoOVUNkCtV6MAeXkJBpqZLbxDbXI0NgQTMrTvUjdmMTEzDSPvgCVxz7zTvzhZaKC0jL07cVdzFO4QEHiUC+JeWI1yhOOHVt5Hkr+0eBMmz/mAAHEfJs1bUvjhlBUSjb1biU4iA7AQMY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779818373; c=relaxed/simple; bh=cVEpJWPlb9+3lxdcLCIklwwAAIRNQ16kMW0XcS1SelU=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=pc2YdB9aTj7Co84duCzn6K9lNa2kSQx8+TT/NQD44nTdzSADYtgrHPRYeZuBZMPCmWS1yvSm3IpxfUv9QjC1PlSlFBqHs59S3YvDSv9ypVMGgyXR5TiQhY/susoeX/TqNodmF1I6+EF4BNbVZFl0kamtLqmSUejbp2Y98Ndt4jw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=BNYgCLkv; arc=none smtp.client-ip=209.85.221.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="BNYgCLkv" Received: by mail-wr1-f74.google.com with SMTP id ffacd0b85a97d-44a71109b94so7381053f8f.3 for ; Tue, 26 May 2026 10:59:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1779818370; x=1780423170; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=uVwVNe3cqI0RCHG1xJv3LRo5BwxDkOUdlieXkffr7Nw=; b=BNYgCLkvbZizC5WJEa2VPQqjW/L9o4X6Rh7oYvari0Gy+iuCnhWt6sEQtAjvFVXvwx toNO7AiwHMnU+cV/EAlX76wKZkV52xTdOyf9Cif4YijSHswaXkBiuZiWxQOLLSc1VPxE +HMgGT54VdW51VXenrh1Pfimau6cXFKNhefiuTP3atg1HXiB5u+Uo6Dtljp+mXlkLQnU LTzNPQ0MZYGfNHIMaCKcKdCQFeC962IR6fuP++kJFkK/3TCaGh1MrVD4mVzFZVaXtm53 pOLUneRVo9LopfiXjuoKRe1LYbTJq4ltMBuCQkqD7CwwN4mg4JgKYO6XaaX06GHqo8Fj zlWg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1779818370; x=1780423170; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=uVwVNe3cqI0RCHG1xJv3LRo5BwxDkOUdlieXkffr7Nw=; b=lA6HTWuA0Qm35fAoqc16N5pY68MOgsf5q+Bs6JW1Z7g+l0mJ6E5MxQtngsoM2rGcAF 2Us7ZwUoSyD/uz+yR1gYn5gBBYZ/44A5bKzx3RWB6ddezIv/O26afLH0tlE/DNrM8Qhc wtesDpqpifke2ssag8FoZKPvaYJlK1rva7akApd1oQg6qzb8y+66l6WUzrTtk7UtxfYf 3f5UHOOxJy0dFDci2tdhMG5KYKCzRUsfSRHRh9m0246UgL17k/deq3WGVs8bCloUNzXh tPonrt6S/kFw9aPzHJPHxsxjOm6cNyWABMCNddIlTEwVCTvFHgkrcA063RlD8E5zbfGx 09yw== X-Gm-Message-State: AOJu0Yx/iVByh7xIEP9zmTZp1G4FoOYudCNDfh2yy7qHOfZyCVyXhh5V GNe5GEI8/i9K2P0UI3JLOhrqNWYnM9LqNe8g/uz8TCEkLmB349k1U5AocOL14jKcR21Xeo0ElQ= = X-Received: from wmxb15-n2.prod.google.com ([2002:a05:600d:844f:20b0:48f:d328:60c]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:4e14:b0:490:5cb3:e936 with SMTP id 5b1f17b1804b1-4905cb3eb00mr223400195e9.4.1779818369475; Tue, 26 May 2026 10:59:29 -0700 (PDT) Date: Tue, 26 May 2026 19:58:54 +0200 In-Reply-To: <20260526175846.2694125-17-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260526175846.2694125-17-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=3123; i=ardb@kernel.org; h=from:subject; bh=p/GjcIFRscd8SARDtYvyFpPt1lS/DKkRLXn2dxWNXF4=; b=owGbwMvMwCVmkMcZplerG8N4Wi2JIUv0fsKpx8vWnXp9PDbK7d7pQ5f7tV6IZt198fXxgbl3+ RdMT7Zr6ShlYRDjYpAVU2QRmP333c7TE6VqnWfJwsxhZQIZwsDFKQAT0bvEyHDiTj+j5qWAr0Y/ aoUNp/5XU7S1WV4v/ue8Dkfu9tf7llszMqzS+fykyFm2qtwylU02r291afDEKR3P9d4q/j84eW/ EbjYA X-Mailer: git-send-email 2.54.0.794.g4f17f83d09-goog Message-ID: <20260526175846.2694125-24-ardb+git@google.com> Subject: [PATCH v6 07/15] arm64: kfence: Avoid NOMAP tricks when mapping the early pool From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, Ard Biesheuvel , Ryan Roberts , Anshuman Khandual , Liz Prucka , Seth Jenkins , Kees Cook , Mike Rapoport , David Hildenbrand , Andrew Morton , Jann Horn , linux-mm@kvack.org, linux-hardening@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-sh@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel Now that the map_mem() routines respect existing page mappings and contiguous granule sized blocks with the contiguous bit cleared, there is no longer a reason to play tricks with the memblock NOMAP attribute. Instead, the kfence pool can be allocated and mapped with page granularity first, and this granularity will be respected when the rest of DRAM is mapped later, even if block and contiguous mappings are allowed for the remainder of those mappings. Signed-off-by: Ard Biesheuvel Reviewed-by: Kevin Brodsky --- arch/arm64/mm/mmu.c | 25 ++++---------------- 1 file changed, 5 insertions(+), 20 deletions(-) diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index d7a6991e1844..55bb40348a47 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -1083,36 +1083,24 @@ static int __init parse_kfence_early_init(char *arg) } early_param("kfence.sample_interval", parse_kfence_early_init); =20 -static phys_addr_t __init arm64_kfence_alloc_pool(void) +static void __init arm64_kfence_map_pool(void) { phys_addr_t kfence_pool; =20 if (!kfence_early_init) - return 0; + return; =20 kfence_pool =3D memblock_phys_alloc(KFENCE_POOL_SIZE, PAGE_SIZE); if (!kfence_pool) { pr_err("failed to allocate kfence pool\n"); kfence_early_init =3D false; - return 0; - } - - /* Temporarily mark as NOMAP. */ - memblock_mark_nomap(kfence_pool, KFENCE_POOL_SIZE); - - return kfence_pool; -} - -static void __init arm64_kfence_map_pool(phys_addr_t kfence_pool) -{ - if (!kfence_pool) return; + } =20 /* KFENCE pool needs page-level mapping. */ __map_memblock(kfence_pool, kfence_pool + KFENCE_POOL_SIZE, pgprot_tagged(PAGE_KERNEL), NO_BLOCK_MAPPINGS | NO_CONT_MAPPINGS); - memblock_clear_nomap(kfence_pool, KFENCE_POOL_SIZE); __kfence_pool =3D phys_to_virt(kfence_pool); } =20 @@ -1144,8 +1132,7 @@ bool arch_kfence_init_pool(void) } #else /* CONFIG_KFENCE */ =20 -static inline phys_addr_t arm64_kfence_alloc_pool(void) { return 0; } -static inline void arm64_kfence_map_pool(phys_addr_t kfence_pool) { } +static inline void arm64_kfence_map_pool(void) { } =20 #endif /* CONFIG_KFENCE */ =20 @@ -1155,7 +1142,6 @@ static void __init map_mem(void) phys_addr_t kernel_start =3D __pa_symbol(_text); phys_addr_t kernel_end =3D __pa_symbol(__init_begin); phys_addr_t start, end; - phys_addr_t early_kfence_pool; int flags =3D NO_EXEC_MAPPINGS; u64 i; =20 @@ -1172,7 +1158,7 @@ static void __init map_mem(void) BUILD_BUG_ON(pgd_index(direct_map_end - 1) =3D=3D pgd_index(direct_map_en= d) && pgd_index(_PAGE_OFFSET(VA_BITS_MIN)) !=3D PTRS_PER_PGD - 1); =20 - early_kfence_pool =3D arm64_kfence_alloc_pool(); + arm64_kfence_map_pool(); =20 linear_map_requires_bbml2 =3D !force_pte_mapping() && can_set_direct_map(= ); =20 @@ -1210,7 +1196,6 @@ static void __init map_mem(void) */ __map_memblock(kernel_start, kernel_end, PAGE_KERNEL, NO_CONT_MAPPINGS); memblock_clear_nomap(kernel_start, kernel_end - kernel_start); - arm64_kfence_map_pool(early_kfence_pool); } =20 void mark_rodata_ro(void) --=20 2.54.0.794.g4f17f83d09-goog From nobody Mon Jun 8 20:41:28 2026 Received: from mail-ej1-f74.google.com (mail-ej1-f74.google.com [209.85.218.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 77674426EB2 for ; Tue, 26 May 2026 17:59:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779818374; cv=none; b=FVfBAz+PDGFYcwvJrYqufB28oVX2fJB0l8MvcrX9bsmU3ti091TP4UZmDveuGflIJrPbITOkjestrTvekPHOFvtx0OH4YYUrNPyc9TUxp1ky/CdHaFyskODIcoGEUlAfvDNvxBjIDLgOzk9JuriF8FEBmKw9nBFbTXVEX3iP460= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779818374; c=relaxed/simple; bh=L5SP3ynx6v29Pg3poBNMmeI60r8Qx4J8iiZZMS51BC8=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=fpnrfxfDQyLBLgj7u0+8zd3ZXpuI+GehiTlf6Dp4RUReDyx/SwcvgsUOJxJ8teCmXBHMXL1aiCHbi6VL7p2wnm489SZvW1h0UNNvqzB5OGLu1srcu3/+68BG/g3UDocMnjbc9fWfugfvc4A6Q4nsYJ6rL2e5QKR1zGWm3SYSRNU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=RAt/t0kr; arc=none smtp.client-ip=209.85.218.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="RAt/t0kr" Received: by mail-ej1-f74.google.com with SMTP id a640c23a62f3a-bdad545342aso138901566b.0 for ; Tue, 26 May 2026 10:59:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1779818371; x=1780423171; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=8QGcTo9f84jQQ3RURqBOJp1vLQa/oONqz3vOx7niDYc=; b=RAt/t0krw9In7Rl1yhBiwVXkZN7xVVmXZVQJ5hhnU+xjsW7mS7Xc/VzJEdD9M82gYm M4s/GUEKLDXdxd6AneqSJBRwjwEzLfmfxoLDZOeWOi9gKCsqr3t9K3p89AQvOzmXJgPD AOWVSDx3uK7Jrt0T6qkXzyxDKEVwymsZEWgHCVNjXlQrhavZUI0xP19CWlMHbEKWWFyB IB+msfoKc4MSNoOM/JtKItQdEEhWIMRFQCsZDhTY5ed4M+1E89fxovzJLfUADiYM7ZWT 3u58vcU+B0SMANN5n0s2LGPFTLy3Pm6ZcGKathNdbEYkx7jL1INSG0Zy4lGQ+CmrNJDq fqwg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1779818371; x=1780423171; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=8QGcTo9f84jQQ3RURqBOJp1vLQa/oONqz3vOx7niDYc=; b=fa+wk+IDZMDwiNw06LaB3ZtDz9AzPOy27T5AhLfI/Lf5UhobYsM7LX8JTnUKeuyatg HHkyBQsE5/j8pTcT8fD1Nij15qvCE3pjj520i3++GIRznE0F6zQtbRoOJi1prwvkpS1B BBnSG5Xs6yOEUBEOHsQYdfNYfrRJBPS3LRK4CVDOETPPHYOEXD9jXPFI0yAKN4LV01vY YjLVnrUR3NQIVyBghpmi8XF+x+smC9eXFQ5N2iF2wkXpSzXQgb9w+HNEcfuCxth21V6/ zPUKbvbw2YHSVkKKUPg0jnDz//Ou5AUXFSv40LqPxUY/XTDhALVuyqy8im6MS0IF4pQt llgQ== X-Gm-Message-State: AOJu0YwXOn/mXaIM5UC/1r/XHSAwTLO8oafA4Hb6yqQpe+f/DYK+UBqZ EAZuicAAVK1TKsTPXh4lHD6tXcw8ZPIWfwymoQOgB2D6/lVQArvpuJukB8klj0NXPnRrMtbbNw= = X-Received: from edyn13.prod.google.com ([2002:a05:6402:6cd:b0:687:e30e:62a9]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a17:907:ca84:b0:bd2:e940:db25 with SMTP id a640c23a62f3a-bdd22e30cc2mr1206557666b.13.1779818370675; Tue, 26 May 2026 10:59:30 -0700 (PDT) Date: Tue, 26 May 2026 19:58:55 +0200 In-Reply-To: <20260526175846.2694125-17-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260526175846.2694125-17-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=2937; i=ardb@kernel.org; h=from:subject; bh=xB9oKBi8uLt+1POTvwyFN4ig92BBqhDsXz5+cc7USWw=; b=owGbwMvMwCVmkMcZplerG8N4Wi2JIUv0fhKj4MmKF6L9vM+ex9Rs9b0+/d9Nq+XeXuXTJwrky 3esF1nfUcrCIMbFICumyCIw+++7nacnStU6z5KFmcPKBDKEgYtTACby6AzD/9Dd3yeIi89KKpkZ tS7DjyPAYY6GxZyveRIHIpY9rS1/zc3wV6BhbVqFXb6I0JNF9ifsanP6XEOzznS/b/x27JQXe9l 0VgA= X-Mailer: git-send-email 2.54.0.794.g4f17f83d09-goog Message-ID: <20260526175846.2694125-25-ardb+git@google.com> Subject: [PATCH v6 08/15] arm64: mm: Permit contiguous attribute for preliminary mappings From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, Ard Biesheuvel , Ryan Roberts , Anshuman Khandual , Liz Prucka , Seth Jenkins , Kees Cook , Mike Rapoport , David Hildenbrand , Andrew Morton , Jann Horn , linux-mm@kvack.org, linux-hardening@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-sh@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel There are a few cases where we omit the contiguous hint for mappings that start out as read-write and are remapped read-only later, on the basis that manipulating live descriptors with the PTE_CONT attribute set is unsafe. When support for the contiguous hint was added to the code, the ARM ARM was ambiguous about this, and so we erred on the side of caution. In the meantime, this has been clarified [0], and regions that will be remapped in their entirety, retaining the contiguous bit on all entries, can use the contiguous hint both in the initial mapping as well as the one that replaces it. Note that this requires that the logic that may be called to remap overlapping regions respects existing valid descriptors that have the contiguous bit cleared. So omit the NO_CONT_MAPPINGS flag in places where it is unneeded. Thanks to Ryan for the reference. [0] RJQQTC For a TLB lookup in a contiguous region mapped by translation table entries= that have consistent values for the Contiguous bit, but have the OA, attributes,= or permissions misprogrammed, that TLB lookup is permitted to produce an OA, a= ccess permissions, and memory attributes that are consistent with any one of the programmed translation table values. Signed-off-by: Ard Biesheuvel Reviewed-by: Kevin Brodsky --- arch/arm64/mm/mmu.c | 10 +++------- 1 file changed, 3 insertions(+), 7 deletions(-) diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index 55bb40348a47..04cc579c7a15 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -1016,8 +1016,7 @@ void __init create_mapping_noalloc(phys_addr_t phys, = unsigned long virt, &phys, virt); return; } - early_create_pgd_mapping(init_mm.pgd, phys, virt, size, prot, NULL, - NO_CONT_MAPPINGS); + early_create_pgd_mapping(init_mm.pgd, phys, virt, size, prot, NULL, 0); } =20 void __init create_pgd_mapping(struct mm_struct *mm, phys_addr_t phys, @@ -1044,8 +1043,7 @@ static void update_mapping_prot(phys_addr_t phys, uns= igned long virt, return; } =20 - early_create_pgd_mapping(init_mm.pgd, phys, virt, size, prot, NULL, - NO_CONT_MAPPINGS); + early_create_pgd_mapping(init_mm.pgd, phys, virt, size, prot, NULL, 0); =20 /* flush the TLBs after updating live kernel mappings */ flush_tlb_kernel_range(virt, virt + size); @@ -1191,10 +1189,8 @@ static void __init map_mem(void) * alternative patching has completed). This makes the contents * of the region accessible to subsystems such as hibernate, * but protects it from inadvertent modification or execution. - * Note that contiguous mappings cannot be remapped in this way, - * so we should avoid them here. */ - __map_memblock(kernel_start, kernel_end, PAGE_KERNEL, NO_CONT_MAPPINGS); + __map_memblock(kernel_start, kernel_end, PAGE_KERNEL, 0); memblock_clear_nomap(kernel_start, kernel_end - kernel_start); } =20 --=20 2.54.0.794.g4f17f83d09-goog From nobody Mon Jun 8 20:41:28 2026 Received: from mail-wr1-f73.google.com (mail-wr1-f73.google.com [209.85.221.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 14C03421F05 for ; Tue, 26 May 2026 17:59:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779818376; cv=none; b=Qa27IptujmLuPgHOrAU6/oQ5Nj1jrh5Rjp5U3zgHFexbiXBaGHoNgELEQAj+o2KieC8H2GVJ7fHHaBvtQVam45YhJbIIduAB3Pl+usTJ/ekG2jhOmkSYf9gUGc+TmOPICT/YUIsTd/qSy/q2gNeFp/iE1Aiv9IOtZtzm7lR7oqE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779818376; c=relaxed/simple; bh=ZmR71NqyGJG4EJRqxzWirNcZmBaGhL9AzpoJZLkCgUI=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=A8nx2Q9Han+6kEbKAklvQ2ofuf5857tvkTZM4dd0wAfXk4u71thhyvLw6q9EexOCfhVfEsdQmwQUwU4C4nVRJkYsrmhKUkA1wMI+ERuHT5Ei3iH3Q16kUed5hRWJREsB/8Fwbs2RUJZyQ1upFkgfpHOYESdFy7CVFQwOKInT6Qc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=nxuJtLmA; arc=none smtp.client-ip=209.85.221.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="nxuJtLmA" Received: by mail-wr1-f73.google.com with SMTP id ffacd0b85a97d-45e78dd7baaso8774892f8f.0 for ; Tue, 26 May 2026 10:59:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1779818372; x=1780423172; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=TJx2KhHmtgY53AkMvOyg5uK/J3luFDMcMijObo9+4vo=; b=nxuJtLmAQTr49ypdI6gtMNRPvShm5nLWpzWqBVpnlR9i5HBWj5NpbnBuJEyEm2tIam JCVVEsPMSLXI5s6jY1UROY1qUSbPn93qVRxG1q+CPVThczwmKI5IZfA2mjSvBgcaWtHq cooEovDDvjhiLfSCeXcdPtn5I30+zwViMx4kviK/INqfxCrFi5HVyrZF3tNQ8k7bYgar KOlnzvPsW2FnUQX2rPdr5UuqwFRJaVP4lMvDyh2bbJouH1dqXz/gxSH0WgKvwzF5dtAX WI80bN1s2t8Q/d7W8SavQwErh+k1Z74fQwP4JEvwQb3huix/s3Dfu1fI200UuSbEU4IS R5uQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1779818372; x=1780423172; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=TJx2KhHmtgY53AkMvOyg5uK/J3luFDMcMijObo9+4vo=; b=gwvNr+vCxxrB6bN1xyo6ufYHzjEPWNToxwmjIzv7iLX+pIIwf8/uhQ5QV0OwjVsLnp 4iWrLmwkMiDU5JQ1pKPoiPM6MWo1fqMix2uGgxoV2UDs4ffeNBDq3aSBrRw7JzGhkJET DMbgUaQdpHkdzOULlxVGDGeAFeZt5gVjI0Z7heLQmyRZh3y3tLl58FNqxRrcvaFdiU7z L94hHq9izS7g3rp7KnN9qWxmOrVho+5NVrKiRcuF5IHKvUPWAR7j6HRgI9fCv4pfJYEj 0n2lwG9H529IvcvN15IfQZ9QtSXOZO182YW4EUYqZOG53qjs3RreR6vWL1g58Kv7nAD3 BTjw== X-Gm-Message-State: AOJu0YzrU6oepfQaEkesbmEwiEXx2N8+nCh3YsAb8EsvcSCDLxCHrFI7 Pz1hwEFRqu4N8yNqdxBQE7rktthKbmAKnsayJFCu53F90aaQ+tPq1zyG7HdG2WUR2EeInGh4Rw= = X-Received: from wrjn10.prod.google.com ([2002:adf:e34a:0:b0:43d:6f5f:8c52]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:35c8:b0:490:44eb:c1e7 with SMTP id 5b1f17b1804b1-49044ebc2d5mr360486815e9.30.1779818372100; Tue, 26 May 2026 10:59:32 -0700 (PDT) Date: Tue, 26 May 2026 19:58:56 +0200 In-Reply-To: <20260526175846.2694125-17-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260526175846.2694125-17-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=3242; i=ardb@kernel.org; h=from:subject; bh=Qta0PMsGcvlQhpXqhjldDBt900x2YBq541ys84w8Z6A=; b=owGbwMvMwCVmkMcZplerG8N4Wi2JIUv0frL6+VN/jnUf5F5tPjtywfpJk9QUy2MKCjk0NTccj r7PUqfVUcrCIMbFICumyCIw+++7nacnStU6z5KFmcPKBDKEgYtTACayl53hf8n7L8U7Dq/XtJM8 aaNT5xu9zmX9+a6ICfks3cp+rZ7mixj+Z6226zYNmPCvKn7q8tmXSzeumHZtU+IR1bAruyazqkp WMAIA X-Mailer: git-send-email 2.54.0.794.g4f17f83d09-goog Message-ID: <20260526175846.2694125-26-ardb+git@google.com> Subject: [PATCH v6 09/15] arm64: Move fixmap and kasan page tables to end of kernel image From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, Ard Biesheuvel , Ryan Roberts , Anshuman Khandual , Liz Prucka , Seth Jenkins , Kees Cook , Mike Rapoport , David Hildenbrand , Andrew Morton , Jann Horn , linux-mm@kvack.org, linux-hardening@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-sh@vger.kernel.org, Kevin Brodsky Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel Move the fixmap and kasan page tables out of the BSS section, and place them at the end of the image, right before the init_pg_dir section where some of the other statically allocated page tables live. These page tables are currently the only data objects in vmlinux that are meant to be accessed via the kernel image's linear alias, and so placing them together allows the remainder of the data/bss section to be remapped read-only or unmapped entirely. Reviewed-by: Kevin Brodsky Signed-off-by: Ard Biesheuvel --- arch/arm64/include/asm/mmu.h | 2 ++ arch/arm64/kernel/vmlinux.lds.S | 8 +++++++- arch/arm64/mm/fixmap.c | 6 +++--- arch/arm64/mm/kasan_init.c | 2 +- 4 files changed, 13 insertions(+), 5 deletions(-) diff --git a/arch/arm64/include/asm/mmu.h b/arch/arm64/include/asm/mmu.h index 5e1211c540ab..fb95754f2876 100644 --- a/arch/arm64/include/asm/mmu.h +++ b/arch/arm64/include/asm/mmu.h @@ -13,6 +13,8 @@ =20 #ifndef __ASSEMBLER__ =20 +#define __pgtbl_bss __section(".pgdir.bss") __aligned(PAGE_SIZE) + #include #include =20 diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.ld= s.S index e1ac876200a3..2b0ebfb30c63 100644 --- a/arch/arm64/kernel/vmlinux.lds.S +++ b/arch/arm64/kernel/vmlinux.lds.S @@ -349,9 +349,15 @@ SECTIONS _edata =3D .; =20 /* start of zero-init region */ - BSS_SECTION(SBSS_ALIGN, 0, 0) + BSS_SECTION(SBSS_ALIGN, 0, PAGE_SIZE) __pi___bss_start =3D __bss_start; =20 + /* fixmap BSS starts here - preceding data/BSS is omitted from the linear= map */ + .pgdir.bss (NOLOAD) : ALIGN(PAGE_SIZE) { + *(.pgdir.bss) + } + ASSERT(ADDR(.pgdir.bss) =3D=3D __bss_stop, ".pgdir.bss must follow BSS") + . =3D ALIGN(PAGE_SIZE); __pi_init_pg_dir =3D .; . +=3D INIT_DIR_SIZE; diff --git a/arch/arm64/mm/fixmap.c b/arch/arm64/mm/fixmap.c index c5c5425791da..1a3bbd67dd76 100644 --- a/arch/arm64/mm/fixmap.c +++ b/arch/arm64/mm/fixmap.c @@ -31,9 +31,9 @@ static_assert(NR_BM_PMD_TABLES =3D=3D 1); =20 #define BM_PTE_TABLE_IDX(addr) __BM_TABLE_IDX(addr, PMD_SHIFT) =20 -static pte_t bm_pte[NR_BM_PTE_TABLES][PTRS_PER_PTE] __page_aligned_bss; -static pmd_t bm_pmd[PTRS_PER_PMD] __page_aligned_bss __maybe_unused; -static pud_t bm_pud[PTRS_PER_PUD] __page_aligned_bss __maybe_unused; +static pte_t bm_pte[NR_BM_PTE_TABLES][PTRS_PER_PTE] __pgtbl_bss; +static pmd_t bm_pmd[PTRS_PER_PMD] __pgtbl_bss __maybe_unused; +static pud_t bm_pud[PTRS_PER_PUD] __pgtbl_bss __maybe_unused; =20 static inline pte_t *fixmap_pte(unsigned long addr) { diff --git a/arch/arm64/mm/kasan_init.c b/arch/arm64/mm/kasan_init.c index abeb81bf6ebd..dbf22cae82ee 100644 --- a/arch/arm64/mm/kasan_init.c +++ b/arch/arm64/mm/kasan_init.c @@ -214,7 +214,7 @@ asmlinkage void __init kasan_early_init(void) * shadow pud_t[]/p4d_t[], which could end up getting corrupted * when the linear region is mapped. */ - static pte_t tbl[PTRS_PER_PTE] __page_aligned_bss; + static pte_t tbl[PTRS_PER_PTE] __pgtbl_bss; pgd_t *pgdp =3D pgd_offset_k(KASAN_SHADOW_START); =20 set_pgd(pgdp, __pgd(__pa_symbol(tbl) | PGD_TYPE_TABLE)); --=20 2.54.0.794.g4f17f83d09-goog From nobody Mon Jun 8 20:41:28 2026 Received: from mail-wm1-f73.google.com (mail-wm1-f73.google.com [209.85.128.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 78B76427A16 for ; Tue, 26 May 2026 17:59:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779818377; cv=none; b=YeAfC+pIZuvQZGnjV5KHtMl6wiNr4p7mCqvtcmdwn6xoMURZkG0CTIy4xCQ9jvOT5gcJr+Ao9cn/EK32AgVv0KyCsXxoOlcK3PcVvLhK9B3Ek01Erl4tfbxPU5fUDT1d3hwpqpVdQKDvjCYtu+IteuzQcsNw1lg5LpQ+mo2Avxw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779818377; c=relaxed/simple; bh=ev685/CJw382iK9z08ELcRr7fLfPN9V7kdOx6hU1hW0=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=PLBhyx3rTTlh0xgZoOtiuKvM7qkrqi3UT1ZX3Y1699xuV3CwAfStZosJ7nusH+Y/E6ZkP0VqkddRA/NQh/GRD7lEbwbeLG8qU2+2ZAPmgUtI9NZmfuEBbsmDksP/NYNfmEn7XDl+G7tV2y5rTEyvrKdr5e+yN2TFtkLhzUG6eRk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=Bxzhk6iW; arc=none smtp.client-ip=209.85.128.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="Bxzhk6iW" Received: by mail-wm1-f73.google.com with SMTP id 5b1f17b1804b1-49058e91639so23202215e9.3 for ; Tue, 26 May 2026 10:59:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1779818374; x=1780423174; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=/YEs+3ha0YZ0GL65qiPfv2bavm0rO2bwvFUt4jkd+Vg=; b=Bxzhk6iWBaI6lCd87AuEqX+Pj9CMyxoolSzXMXMKq+qvnAklYTn8fTIpOZw23co12c qzqBZF8RjWrRE0hfoYFyto+1my244bRfdSQIB1w/iB+e0QWEtDw0oCsELa3AmcGyx+VT U9jpxwWqcRf9XGewjUyo9CLE3zigv56SMkQrIUVPWYSwhaLVSRFAF6rMfQmVkJJ/rbG7 DuMGC/Pql9w9NQnEzcnu2r/nDJUxaiLdoA7Kb2rg9Hr0NK+pvpArX7YywUZKnqqgDFbq 7PYCaatch8nKWSJm71uJAt0AOCWPXUiSi3dNiTPektdADBQeyNuonvN0K57g3HXkEcc+ SHdA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1779818374; x=1780423174; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=/YEs+3ha0YZ0GL65qiPfv2bavm0rO2bwvFUt4jkd+Vg=; b=aLuANM8pRqSbKfC5Kp7C5mk3+xarFxMWW/w18ctt8jXsp5jYnJi1Ps9RzEsVwqmMVA LqlNQMPONq8NMjKYIwu+6WAfPNAXD5ryekajT/xuy+IQdqFlDPcFQwPcmoAyucs0bPPk xGQzlDNefKQzK4N2XxBUQY7voddTh4+P5CG+P4pTQLSQmCGirA/g5xRFTpyEcVz5qg6/ K1f7b5KRn7qheTdFXl2hXPugw7F9SyLD3RldpKr8WIgz8BFTlYnWxsYcI5yyZMVrOIsl nv8ZusMEiuTWRaHky3fPfA4lqdyMUtLOyPUPEvTQa8fHB3GdLmsw0bE0eow5i2GeSAi7 OOqQ== X-Gm-Message-State: AOJu0YxE5C1sl6mqNSIDSLf6ji7B2AasafDUm6Kknjw2lnEvUebtutNh vpPf2UKfRLg3LOiXqnE24zTQtQ3K5ug15c/Re+TYWY4TEdDZXeOJtOl1/JH16x41ApSsu2c95w= = X-Received: from wmri26-n1.prod.google.com ([2002:a05:600c:8a1a:10b0:490:49db:2263]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:5298:b0:48f:e230:1d12 with SMTP id 5b1f17b1804b1-490428dd63emr346556585e9.31.1779818373602; Tue, 26 May 2026 10:59:33 -0700 (PDT) Date: Tue, 26 May 2026 19:58:57 +0200 In-Reply-To: <20260526175846.2694125-17-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260526175846.2694125-17-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=2521; i=ardb@kernel.org; h=from:subject; bh=IVFHANRX1Y5B7sO2czLQgDyXUEkQiTSy8G1FQDCgg/E=; b=owGbwMvMwCVmkMcZplerG8N4Wi2JIUv0fsoLbrWLu7gSw7a4He9yccnfFaz2Zfnzl9euX9kmw 5F4//u9jlIWBjEuBlkxRRaB2X/f7Tw9UarWeZYszBxWJpAhDFycAjCR2HhGhum3KqfUzpnosfjs O4PZz65tu/Cten/afJ4dmhfPiNRN+BDP8D8myOOB5p+Lm9QF963bJLbl3IvwqSc/fj98Oa1ZZH3 U5m2sAA== X-Mailer: git-send-email 2.54.0.794.g4f17f83d09-goog Message-ID: <20260526175846.2694125-27-ardb+git@google.com> Subject: [PATCH v6 10/15] arm64: mm: Don't abuse memblock NOMAP to check for overlaps From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, Ard Biesheuvel , Ryan Roberts , Anshuman Khandual , Liz Prucka , Seth Jenkins , Kees Cook , Mike Rapoport , David Hildenbrand , Andrew Morton , Jann Horn , linux-mm@kvack.org, linux-hardening@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-sh@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel Now that the linear region mapping routines respect existing table mappings and contiguous block and page mappings, it is no longer needed to fiddle with the memblock tables to set and clear the NOMAP attribute in order to omit text and rodata when creating the linear map. Instead, map the kernel text and rodata alias first with the desired initial attributes and granularity, so that the loop iterating over the memblocks will not remap it in a manner that prevents it from being remapped with updated attributes later. Signed-off-by: Ard Biesheuvel --- arch/arm64/mm/mmu.c | 23 ++++++-------------- 1 file changed, 7 insertions(+), 16 deletions(-) diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index 04cc579c7a15..b20c76b8381d 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -1164,12 +1164,14 @@ static void __init map_mem(void) flags |=3D NO_BLOCK_MAPPINGS | NO_CONT_MAPPINGS; =20 /* - * Take care not to create a writable alias for the - * read-only text and rodata sections of the kernel image. - * So temporarily mark them as NOMAP to skip mappings in - * the following for-loop + * Map the linear alias of the [_text, __init_begin) interval + * as non-executable now, and remove the write permission in + * mark_linear_text_alias_ro() above (which will be called after + * alternative patching has completed). This makes the contents + * of the region accessible to subsystems such as hibernate, + * but protects it from inadvertent modification or execution. */ - memblock_mark_nomap(kernel_start, kernel_end - kernel_start); + __map_memblock(kernel_start, kernel_end, PAGE_KERNEL, flags); =20 /* map all the memory banks */ for_each_mem_range(i, &start, &end) { @@ -1181,17 +1183,6 @@ static void __init map_mem(void) __map_memblock(start, end, pgprot_tagged(PAGE_KERNEL), flags); } - - /* - * Map the linear alias of the [_text, __init_begin) interval - * as non-executable now, and remove the write permission in - * mark_linear_text_alias_ro() below (which will be called after - * alternative patching has completed). This makes the contents - * of the region accessible to subsystems such as hibernate, - * but protects it from inadvertent modification or execution. - */ - __map_memblock(kernel_start, kernel_end, PAGE_KERNEL, 0); - memblock_clear_nomap(kernel_start, kernel_end - kernel_start); } =20 void mark_rodata_ro(void) --=20 2.54.0.794.g4f17f83d09-goog From nobody Mon Jun 8 20:41:28 2026 Received: from mail-wm1-f73.google.com (mail-wm1-f73.google.com [209.85.128.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D4BB6423160 for ; Tue, 26 May 2026 17:59:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779818379; cv=none; b=PFCQndLOXy/kP9Ox8qu15hAruOnTkm3u0/ijcSYGpsHXzacd9DZkHzae0ZvwySuX/uGfMiWIltuEdj2CYd3HvEtKDKgSHc6M8RQSW88nbE660dkAYtkTJ4Lpwx7FPOtZNvSIvqM1abOTb5fs31xq2KML6ebhchx0nZodnKc4aQY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779818379; c=relaxed/simple; bh=knJFoS+4h4KHdDV3SxOHBSwzIjeDpJAyubTc8rLv7Hs=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=gwMXTI0FJ4x2tQJMs1vRa7sj4+GFlz5mrqP/gIFezWcJiO23vO8A5MJ1ZvluFq6DlWFhsUsvU/IV+RdblzmtL65jUypzANPX6c6TtfuXvvPLVSjy5kgw5QsRpRSR+76jIPGc/lt8HL2h5Hwj//nD97Y97QhpJItUOrGxpXnllpk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=e+JRnHmB; arc=none smtp.client-ip=209.85.128.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="e+JRnHmB" Received: by mail-wm1-f73.google.com with SMTP id 5b1f17b1804b1-48fd33b4921so70242825e9.2 for ; Tue, 26 May 2026 10:59:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1779818375; x=1780423175; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=hN7dKQgvYd9nbZsuBceB/G3FfRSci9s4lIdLcy6NsRQ=; b=e+JRnHmBuZ7l0QZ2mEHqmd8YODMGS/hrtWHFgQQ/C8IXTVspbdhSvPlZEFrQlFUwpQ ZbPZg0FTIBMUkcb/0Io8nrieOGlEhbYSbx/feepu7UYU80dmyIcdJvOJnKJ6X3sMpOan /UwlGWH7WxFgWj/nBPL4E8T6r0ylMfkgQe4OL0HuHg4oR9zVGj04uY8UZwhgr7K0R/AH AMhpkZ3gTmotAflnya1cIKDiSKZf408LBWxHDedDShP19AzEhJ7zBkftuEErHm6eLDAP jBY3XXGObk5Ps+qHKoIy9iM6NvuWBIKXr3dSAMzUxVf72U+zY4qWqKAH/u/+n3WXOUT4 CyLA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1779818375; x=1780423175; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=hN7dKQgvYd9nbZsuBceB/G3FfRSci9s4lIdLcy6NsRQ=; b=YrvOFHDsrAbYf3m4CgZkReicVv3aD15loeIUlOiNzHu6N9oz80UCXVTfoZeUhmEfe5 UVTf/9cf1xTIQak7T+HRGjhyxK1XdRArHC4GHG0rPEW2Y2gtCOfNQQX0Z/3FErNU/NKn srR9ROyny0kdEkMIroZGgFwi7azhgIWE9QlVgJMqI2kyXhiyfaIVzevJoVq2tbiq6n6f vebJO4uzpbaVySKdJ7CQe/SD+SXPSdxgp08XVS2IngR8KhHiRRCtF7ng4a64jPs3qkOi kIz0Id2zZKXqN2QVVgKfqXzPXe2z1BZmJqnWj568Jk9puU3fkxCtwBZBYuxfIEqRExhH pbUw== X-Gm-Message-State: AOJu0Yz2KjevSB2onOFZeFSIs1+yykOfuE+RnCvx6Gk9sol0IHD72YrB wikVWjMEeyNGkIRkjot4avq5lHJnWC7eAOWyWsApxSJkga5mdNBowPgBu24auR+l5ltL/8SeGA= = X-Received: from wmrk9.prod.google.com ([2002:a05:600c:b49:b0:490:7e29:9cea]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:a402:b0:490:3f7a:108b with SMTP id 5b1f17b1804b1-490426c5be8mr278139655e9.16.1779818375139; Tue, 26 May 2026 10:59:35 -0700 (PDT) Date: Tue, 26 May 2026 19:58:58 +0200 In-Reply-To: <20260526175846.2694125-17-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260526175846.2694125-17-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=2428; i=ardb@kernel.org; h=from:subject; bh=bcrl0Dq8YQBZ2y6oSLTxv/uZS/MgfYTkD9gOirfShVU=; b=owGbwMvMwCVmkMcZplerG8N4Wi2JIUv0fppn5XvTONsranx8M7mVQs+XJQdo2RuuSWq7JaMf9 7fh7PqOUhYGMS4GWTFFFoHZf9/tPD1RqtZ5lizMHFYmkCEMXJwCMJEXcowMtw6WZR/exrN7YplZ UMhusfI9n1W/fnxWF5Br8bb/o5j6XUaG9kh5ycXzfXi7yhR03u2bYfF33raUQN5POWLfbh9Jdln ECgA= X-Mailer: git-send-email 2.54.0.794.g4f17f83d09-goog Message-ID: <20260526175846.2694125-28-ardb+git@google.com> Subject: [PATCH v6 11/15] arm64: mm: Map the kernel data/bss read-only in the linear map From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, Ard Biesheuvel , Ryan Roberts , Anshuman Khandual , Liz Prucka , Seth Jenkins , Kees Cook , Mike Rapoport , David Hildenbrand , Andrew Morton , Jann Horn , linux-mm@kvack.org, linux-hardening@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-sh@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel On systems where the bootloader adheres to the original arm64 boot protocol, the placement of the kernel in the physical address space is highly predictable, and this makes the placement of its linear alias in the kernel virtual address space equally predictable, given the lack of randomization of the linear map. The linear aliases of the kernel text and rodata regions are already mapped read-only, but the kernel data and bss are mapped read-write in this region. This is not needed, so map them read-only as well. Note that the statically allocated kernel page tables do need to be modifiable via the linear map, so leave these mapped read-write. Signed-off-by: Ard Biesheuvel Reviewed-by: Kevin Brodsky --- arch/arm64/mm/mmu.c | 15 +++++++++++++-- 1 file changed, 13 insertions(+), 2 deletions(-) diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index b20c76b8381d..e7ca53d20b87 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -1138,7 +1138,9 @@ static void __init map_mem(void) { static const u64 direct_map_end =3D _PAGE_END(VA_BITS_MIN); phys_addr_t kernel_start =3D __pa_symbol(_text); - phys_addr_t kernel_end =3D __pa_symbol(__init_begin); + phys_addr_t init_begin =3D __pa_symbol(__init_begin); + phys_addr_t init_end =3D __pa_symbol(__init_end); + phys_addr_t kernel_end =3D __pa_symbol(__bss_stop); phys_addr_t start, end; int flags =3D NO_EXEC_MAPPINGS; u64 i; @@ -1171,7 +1173,11 @@ static void __init map_mem(void) * of the region accessible to subsystems such as hibernate, * but protects it from inadvertent modification or execution. */ - __map_memblock(kernel_start, kernel_end, PAGE_KERNEL, flags); + __map_memblock(kernel_start, init_begin, PAGE_KERNEL, flags); + + /* Map the kernel data/bss so it can be remapped later */ + __map_memblock(init_end, kernel_end, pgprot_tagged(PAGE_KERNEL), + flags); =20 /* map all the memory banks */ for_each_mem_range(i, &start, &end) { @@ -1183,6 +1189,11 @@ static void __init map_mem(void) __map_memblock(start, end, pgprot_tagged(PAGE_KERNEL), flags); } + + /* Map the kernel data/bss read-only in the linear map */ + __map_memblock(init_end, kernel_end, PAGE_KERNEL_RO, flags); + flush_tlb_kernel_range((unsigned long)lm_alias(__init_end), + (unsigned long)lm_alias(__bss_stop)); } =20 void mark_rodata_ro(void) --=20 2.54.0.794.g4f17f83d09-goog From nobody Mon Jun 8 20:41:28 2026 Received: from mail-ed1-f74.google.com (mail-ed1-f74.google.com [209.85.208.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0DD7742885D for ; Tue, 26 May 2026 17:59:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779818379; cv=none; b=CxZXvPhhedM32S5jaYIrT+piRKOXFBL1ZsVEJ0pQMdep7isv/OKqDcxHa24jCkSaOWOS/cEzC7IOKGJpq+vbQVYyMrjy3m0fcD9fpdU/jGnXx0gsNJCHwVCEVY789CgokkKg+hN81CWuSd1TLn/KCTdz8SGrfugETpozhxXMXY8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779818379; c=relaxed/simple; bh=SPk/NWH8coVOt0tso4SER9c0QEdFsfo4KSqPkXnf6iA=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=peTE29rblN988gcRJzJlx/69do45LzGutqGTQCH6VgJCd1hRZes033LbUkE9yjxnaKhzirHTbZe9mAHhlrU7/wAwxOxh/O51XFyVwjegLqCa+rgZ6Qk0rDxzhKK8Aqk3P8lWZjPec+xqATuWIrnYYJlUG1QNSqo44JFNlZpwrEI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=ZIqFG50k; arc=none smtp.client-ip=209.85.208.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="ZIqFG50k" Received: by mail-ed1-f74.google.com with SMTP id 4fb4d7f45d1cf-688b055e937so3002012a12.3 for ; Tue, 26 May 2026 10:59:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1779818376; x=1780423176; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=ZqWf3aIh9jwoRDW7ej6OaYtUXPKSIApXmlGfcanpMS4=; b=ZIqFG50kSU6Ao0R4CWm24LIUGo13l6VOTLkLphI4vn+YUTGpVGlbzRosuz41OSxAtY nzxnhjZamDmDpN1lMsY5is91EuDEMp0T/ybV4gTHdK0BpkT8kfVqxhuUd1NoZicx7oeA 6O94W3OA0IclPSO9PuTrbicoTkjKHCedvOrUQjjMHIb7WY5Gh3CO5TTgwO0mNz/zC9sV sbqe9TOzsGvchHm+m4SSGeKcweGSu7zdM6FfIjcTopTBGZL4WBYRMmQJHHlF0s33wwnL wQUWD2PcVNS9/gMPxchQRo9pXirYKFNj3YZuZdLJpErCRzHQFE1gWxSqgADgE9TBN8ve GYkA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1779818376; x=1780423176; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=ZqWf3aIh9jwoRDW7ej6OaYtUXPKSIApXmlGfcanpMS4=; b=fHDaM1fVShdYvDFH5cW+3EnWI26fIzuKyuJf2Iu3INNc5PHYDX1d8s+Qpxx2iPDWiM OEWhCCeWQ2LGN4GdkOlufJMd2QrAvyeQy6DoF3Al4ZuRfWolwCp/XHl724osCD71nYDg jpfmq0ZnaSNQYP6mOiGwtLHFnyAcdR9/qANMgSOsCgbcsYQRxvY4+H9Izg2X++AXsH6o S01w5zvWS69z7+ZIzgye7EI+URmD/45pwvYBjR591sa0J1Px3YgHxy8upu3ajnxGt3xg /V+y+FJwJJWvCME6woFnLGTNyIDKvnf4xxzceRqOSoFLIocsX1AgBQFMKMUTCTdY38EW +VYg== X-Gm-Message-State: AOJu0YwMeUNxkMV/K967w57x15V97mBSa8+mYIbPFzaxcMT/uiaTes+j r8kLSkkeqGQDpiechs+KBf6jXpc4vD8iSbRyWOGMOJLBCaehBvd0KFnN3U8KR71U/PMTHLQllg= = X-Received: from edqv21.prod.google.com ([2002:aa7:d815:0:b0:67b:7c67:1fa3]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6402:4348:b0:674:5b2c:d42c with SMTP id 4fb4d7f45d1cf-6889c406b3cmr9914567a12.3.1779818376178; Tue, 26 May 2026 10:59:36 -0700 (PDT) Date: Tue, 26 May 2026 19:58:59 +0200 In-Reply-To: <20260526175846.2694125-17-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260526175846.2694125-17-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=3562; i=ardb@kernel.org; h=from:subject; bh=UEzDYeArIhcNSC2frPncytSQGHg972cpOAqaWBBxbIk=; b=owGbwMvMwCVmkMcZplerG8N4Wi2JIUv0fnq9X+pUBo2wreeSOJ9dXiQeemVxk13k2m0KUYbfu fQXzIvuKGVhEONikBVTZBGY/ffdztMTpWqdZ8nCzGFlAhnCwMUpABPRzGRkWNkdW18r/eqia85v tUMNMXXf3V+K+oQl+nh/av96UvTZBEaGFv6biY3/V70q4frOvkbMYcPfR2skZm6tkJiyR9TynUs ZOwA= X-Mailer: git-send-email 2.54.0.794.g4f17f83d09-goog Message-ID: <20260526175846.2694125-29-ardb+git@google.com> Subject: [PATCH v6 12/15] powerpc/code-patching: Avoid r/w mapping of the zero page From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, Ard Biesheuvel , Ryan Roberts , Anshuman Khandual , Liz Prucka , Seth Jenkins , Kees Cook , Mike Rapoport , David Hildenbrand , Andrew Morton , Jann Horn , linux-mm@kvack.org, linux-hardening@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-sh@vger.kernel.org, Madhavan Srinivasan , Michael Ellerman , Nicholas Piggin , "Christophe Leroy (CS GROUP)" Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel The only remaining use of map_patch_area() is mapping the zero page, and immediately unmapping it again so that the intermediate page table levels are all guaranteed to be populated. The use of the zero page here is completely arbitrary, and not harmful per se, but currently, it creates a writable mapping, and does so in a manner that requires that the empty_zero_page[] symbol is not const-qualified. Given that this is about to change, and that map_patch_area() now never maps anything other than the zero page, let's simplify the code and - remove the helpers and call [un]map_kernel_page() directly - take the PA of empty_zero_page directly - create a read-only temporary mapping. This allows empty_zero_page[] to be repainted as const u8[] in a subsequent patch, without making substantial changes to this code patching logic. Cc: Madhavan Srinivasan Cc: Michael Ellerman Cc: Nicholas Piggin Cc: "Christophe Leroy (CS GROUP)" Link: https://lore.kernel.org/all/20260520085423.485402-1-ardb@kernel.org/ Signed-off-by: Ard Biesheuvel --- arch/powerpc/lib/code-patching.c | 52 +------------------- 1 file changed, 2 insertions(+), 50 deletions(-) diff --git a/arch/powerpc/lib/code-patching.c b/arch/powerpc/lib/code-patch= ing.c index f84e0337cc02..44ff9f684bef 100644 --- a/arch/powerpc/lib/code-patching.c +++ b/arch/powerpc/lib/code-patching.c @@ -60,9 +60,6 @@ struct patch_context { =20 static DEFINE_PER_CPU(struct patch_context, cpu_patching_context); =20 -static int map_patch_area(void *addr, unsigned long text_poke_addr); -static void unmap_patch_area(unsigned long addr); - static bool mm_patch_enabled(void) { return IS_ENABLED(CONFIG_SMP) && radix_enabled(); @@ -117,11 +114,11 @@ static int text_area_cpu_up(unsigned int cpu) =20 // Map/unmap the area to ensure all page tables are pre-allocated addr =3D (unsigned long)area->addr; - err =3D map_patch_area(empty_zero_page, addr); + err =3D map_kernel_page(addr, __pa_symbol(empty_zero_page), PAGE_KERNEL_R= O); if (err) return err; =20 - unmap_patch_area(addr); + unmap_kernel_page(addr); =20 this_cpu_write(cpu_patching_context.area, area); this_cpu_write(cpu_patching_context.addr, addr); @@ -233,51 +230,6 @@ static unsigned long get_patch_pfn(void *addr) return __pa_symbol(addr) >> PAGE_SHIFT; } =20 -/* - * This can be called for kernel text or a module. - */ -static int map_patch_area(void *addr, unsigned long text_poke_addr) -{ - unsigned long pfn =3D get_patch_pfn(addr); - - return map_kernel_page(text_poke_addr, (pfn << PAGE_SHIFT), PAGE_KERNEL); -} - -static void unmap_patch_area(unsigned long addr) -{ - pte_t *ptep; - pmd_t *pmdp; - pud_t *pudp; - p4d_t *p4dp; - pgd_t *pgdp; - - pgdp =3D pgd_offset_k(addr); - if (WARN_ON(pgd_none(*pgdp))) - return; - - p4dp =3D p4d_offset(pgdp, addr); - if (WARN_ON(p4d_none(*p4dp))) - return; - - pudp =3D pud_offset(p4dp, addr); - if (WARN_ON(pud_none(*pudp))) - return; - - pmdp =3D pmd_offset(pudp, addr); - if (WARN_ON(pmd_none(*pmdp))) - return; - - ptep =3D pte_offset_kernel(pmdp, addr); - if (WARN_ON(pte_none(*ptep))) - return; - - /* - * In hash, pte_clear flushes the tlb, in radix, we have to - */ - pte_clear(&init_mm, addr, ptep); - flush_tlb_kernel_range(addr, addr + PAGE_SIZE); -} - static int __do_patch_mem_mm(void *addr, unsigned long val, bool is_dword) { int err; --=20 2.54.0.794.g4f17f83d09-goog From nobody Mon Jun 8 20:41:28 2026 Received: from mail-wr1-f73.google.com (mail-wr1-f73.google.com [209.85.221.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5B3FF426ED6 for ; Tue, 26 May 2026 17:59:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779818381; cv=none; b=dbqwFgjI+D24F+nccv7IttlXu6fgz2Th92RS1fQBy9xnXICVKBMVdSdv1gSGKoVcPmQnKbVPGegegyC3Pfv8ZbyRgPCUfHqNdglxbzhKkx/RrWv8j6Jh4rtPf3gvVHHA7xFTiTr7XW6e7vix5rygY+BNRTi9BqhFrYwOvaL6BOY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779818381; c=relaxed/simple; bh=Dv+z+rs70M0xjYisYL9XokmSShn9jQLiJBvjCaKzxRg=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=UuvJWc3xyo0TInpJ392Px1TWdKZbMnWjE6V6L5B2eQqtFtRGVavg0vxQrKxONBrP7CflizxBFMqUyuobQ/5KUsJZ67Qst9qx1bK6nKWghmUDjboCe8RamN4Wp7ggjB+1IhnlXgesmynqtPvdH2sLDCK2Dbpjc/ywmre5jyjHiIM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=O8q4TckL; arc=none smtp.client-ip=209.85.221.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="O8q4TckL" Received: by mail-wr1-f73.google.com with SMTP id ffacd0b85a97d-43d7730e9e3so6597205f8f.2 for ; Tue, 26 May 2026 10:59:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1779818378; x=1780423178; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=pjw03D6SHFyZLBNhPne++YEq9ou7VA2sQlGz9HbrowM=; b=O8q4TckLnxHv7C5X/pJXidT5rlBtOqeBDdjao1Seu1Z0eFPL9doj6XQG1JE1MJI+Uw f0gWZYb22xKhGlmEB5U49acPfnAODCZ4tNdsYbotSpxkO7CysJrHp1Bti3OF2DMCZuYR lKNDbtiTdyhWgI57+Kkyux9FAquHl5C5Dtkd1zpqxu/MoX8bmw5nqLznLDmJ6HcyFyO4 UEY8iDK9QH6K+K2f7gpy2uLqBt0ds+Mix9N+lbC79U+9BoSDeNZx9MyjSoYQe2zWD+KE QkQGys4i7yvTGPVGhVXlVQBG5EWf4Twofv8omymt5vZd5DWA28WuNMOjfTt634hedt/l a0mQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1779818378; x=1780423178; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=pjw03D6SHFyZLBNhPne++YEq9ou7VA2sQlGz9HbrowM=; b=Q4dZk2kP9GnsLAqxnxoZSxmDyXSMaVXY6A07y4sbxr49dgJQsOuEnW1aKMx8UcGc7l xYMTllTlkTnamBmSacfMwXSdG1zlJyag5ExlfwsUqg+hJ60sCgpQS7UF3jwAPpREew65 svqw7vtkfZdltM1YfBFAgDO0CnPyNvq5vmVpC7IF8YZVdMfHvF7BhCS1p/oJjfrLzH5N Su+Z5jjXJcghacHFKTIFUHoN5EoORQfFI3ZFVvOpAMl3Xe8FIX0WdDTtVfqsOVvVQbKL gS4SEpocMfOs+YWdecxR/3KsZ7RYsqKFUrGkUUNaq1OD6aSnNaRhpul42vQl7bARcNQf hTZQ== X-Gm-Message-State: AOJu0YxIS1mbRcx6eDI20TgNSYoOqk11gDlYFd5Br7y2RQhi5vs4zpZE W5AR15cq898camc1G958L/G9CMGDU+DU+ohYyx3itJrBXhpA0KjJIuWr4avvMn3bFeXU5XbIKQ= = X-Received: from wmog11.prod.google.com ([2002:a05:600c:310b:b0:488:7f73:f59f]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:4f88:b0:490:50eb:b777 with SMTP id 5b1f17b1804b1-49050ebb9e0mr289239055e9.5.1779818377750; Tue, 26 May 2026 10:59:37 -0700 (PDT) Date: Tue, 26 May 2026 19:59:00 +0200 In-Reply-To: <20260526175846.2694125-17-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260526175846.2694125-17-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=1330; i=ardb@kernel.org; h=from:subject; bh=P6+9rEICJJwtkUC1X6hejRp2Q1Pr3xumYehGC6YytMU=; b=owGbwMvMwCVmkMcZplerG8N4Wi2JIUv0fobew29CRUaVxU/ZtLw+1lwTWmzi/Jj1qSv70V6Hy z8YV03qKGVhEONikBVTZBGY/ffdztMTpWqdZ8nCzGFlAhnCwMUpABNpOcfI8HBnlRjjxeVTqlfn ZXLs3ZJ7bJOj+aZIhVt/dgfO0n5tlsrwz8LsvO1Fp6MbfH7/PnJz9VaPp6kP950W7qm/NTevLfh YHjcA X-Mailer: git-send-email 2.54.0.794.g4f17f83d09-goog Message-ID: <20260526175846.2694125-30-ardb+git@google.com> Subject: [PATCH v6 13/15] sh: cast away constness from the zero page when flushing it from the cache From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, Ard Biesheuvel , Ryan Roberts , Anshuman Khandual , Liz Prucka , Seth Jenkins , Kees Cook , Mike Rapoport , David Hildenbrand , Andrew Morton , Jann Horn , linux-mm@kvack.org, linux-hardening@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-sh@vger.kernel.org, Yoshinori Sato , Rich Felker , John Paul Adrian Glaubitz Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel SH performs cache maintenance on the zero page during boot, presumably to ensure that any clearing of BSS that has occurred at startup is visible to other CPUs and DMA devices. The __flush_wback_region() function takes a void* argument, which is conceptually sound, but given that empty_zero_page[] must never be modified, it is being repainted as const, making it incompatible with a void* formal parameter. Given the above, and the fact that __flush_wback_region() is in fact a function pointer variable with multiple implementations, take the easy way out, and cast away the constness in this particular invocation. Cc: Yoshinori Sato Cc: Rich Felker Cc: John Paul Adrian Glaubitz Signed-off-by: Ard Biesheuvel Reviewed-by: Geert Uytterhoeven --- arch/sh/mm/init.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/sh/mm/init.c b/arch/sh/mm/init.c index 4e40d5e96be9..acbb481cdbfe 100644 --- a/arch/sh/mm/init.c +++ b/arch/sh/mm/init.c @@ -332,7 +332,7 @@ void __init mem_init(void) cpu_cache_init(); =20 /* clear the zero-page */ - __flush_wback_region(empty_zero_page, PAGE_SIZE); + __flush_wback_region((void *)empty_zero_page, PAGE_SIZE); =20 vsyscall_init(); =20 --=20 2.54.0.794.g4f17f83d09-goog From nobody Mon Jun 8 20:41:28 2026 Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6A38A42B72D for ; Tue, 26 May 2026 17:59:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779818382; cv=none; b=Yr28PytM1SkRYo5oqCnPgpN/4/dWwqwR2VmbNI90xgpzQADfLzwabIdUeDdsYkZJFNLv0TSZBV22iptF63gtZoqjDnP/WANSBXe44Bc+jtZedEvjKjw+FIBHEOK6heV+LyINaS15lW6ce/lz2hBbqjvtia02rFpl/xsfZrvomII= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779818382; c=relaxed/simple; bh=kx6uPXi5Ha8BFgjXpuzN0TMPj4yTatVChoNZQ3O8Wjw=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=aT24xF4zoEifvTJfJUk9rLnH6T4niEPbjDESqkCGnIEMI3nHde1ER/s+7CQAeO1EBTl0l9zD8JPf+dixwggu0MJYrlo65v8Ob07ETA6M1zSp19WKt5N/PgrPOwYpM2qSgHvNmoGj7PTIkpj/k4Gut4c1dj1Oz2/EJluxSA0aHcE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=u/4dY+Rz; arc=none smtp.client-ip=209.85.128.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="u/4dY+Rz" Received: by mail-wm1-f74.google.com with SMTP id 5b1f17b1804b1-48fd33b4921so70243085e9.2 for ; Tue, 26 May 2026 10:59:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1779818379; x=1780423179; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=Bg9HdLIrtfTsrLcuMDzN3Kt0yw7TgIGpz80e87aySNQ=; b=u/4dY+Rz3VwT32/uv/cg5+eIUKD3Y2GiYnwJ4ujGUGZ62gwCkzX7szI5RKe0pjPbPI TpkOfwOZ4VgMNn2Zud2BxENjI2UA2CyW7CioBGChbANUKhXKpC1NJR/DcmDOh+mRz8iD zsAslbiNP6mQ9CVoxolg6Ki1EjSt/LrqwJQytm+5rVINPe3TMaHWUSBWnPI+scKszMa1 GqpGmqZ4Ljsg8BcKC/hyYl4uVD120jx/stsL2MP66WJ4XahVIRD6QBheTXP6G4HSdvlr zKVV9aEnPIibUfFB/mvuGcifbsWJ42qyIDmK7IH858Le4C1ROFecv6g1RDpjhpQ1Qtqp f0Og== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1779818379; x=1780423179; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Bg9HdLIrtfTsrLcuMDzN3Kt0yw7TgIGpz80e87aySNQ=; b=Ps996ibavuJ7z/5hhAcuRQfjXMx5txdWjmnKXg187ta83bwE9/OKb6X9QcexSvVBoS z3SpQ+N1umEswZSMW2f1uCCfjy6/kxO6kn6k+2Vlz2FaBkddHAbomuBH/jvJ1/PSeqt1 xjyYh8Ul+yu40ci9c27TweeKB/OVjSxz68J4Rik6XgPIbgrhF2buTkU3x49awtnEZIB6 wPCOrUi3GowDF1RyqEN9ozO/r441SAo/DSc4kmN+T0xNAo08V7alOR1aColXWVlk+5lj 84ZXl2Dp6RsM5KtQegjN02Fkjv9j9rUWf6+VwMy3p4EMiNCmaXdVA1UZPIDmb0Yo2xNU 2ZUQ== X-Gm-Message-State: AOJu0YxR9y6wwzBcepsB3VagoW118XNYxyvL4wbKCmVr9LPyPvActpLF kW/MWEo9LwzMVwwIvPtDNt1VlXuy0UYaAyWqDOVsXJf4CgpqQjEhKCd3lVn1Ov1aUqtHU9FCkQ= = X-Received: from wmfv6.prod.google.com ([2002:a05:600c:15c6:b0:485:3f38:3dd2]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:35c8:b0:490:45bb:8dd9 with SMTP id 5b1f17b1804b1-49045bb8efdmr348556125e9.8.1779818378710; Tue, 26 May 2026 10:59:38 -0700 (PDT) Date: Tue, 26 May 2026 19:59:01 +0200 In-Reply-To: <20260526175846.2694125-17-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260526175846.2694125-17-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=1898; i=ardb@kernel.org; h=from:subject; bh=mYtVYiSzRogY9cxOoJbyywYkU254ucBHoAXG9B3cPO8=; b=owGbwMvMwCVmkMcZplerG8N4Wi2JIUv0fiZrSsLEz85Vxb/brc+ssGD6r5/EdCw1zi66VivRb 5JYlGdHKQuDGBeDrJgii8Dsv+92np4oVes8SxZmDisTyBAGLk4BmEhzByPD0ky+XTe7StYuXrtA se7PPWVX9mMHpG+dm3iu/EHwtpNeExj+Cv4Q0JnTu+ZI3cG3t5OPxTdavzl3yvBbYsCGnriEHY2 z2AE= X-Mailer: git-send-email 2.54.0.794.g4f17f83d09-goog Message-ID: <20260526175846.2694125-31-ardb+git@google.com> Subject: [PATCH v6 14/15] mm: Make empty_zero_page[] const From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, Ard Biesheuvel , Ryan Roberts , Anshuman Khandual , Liz Prucka , Seth Jenkins , Kees Cook , Mike Rapoport , David Hildenbrand , Andrew Morton , Jann Horn , linux-mm@kvack.org, linux-hardening@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-sh@vger.kernel.org, Kevin Brodsky , Feng Tang Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel The empty zero page is used to back any kernel or user space mapping that is supposed to remain cleared, and so the page itself is never supposed to be modified. So mark it as const, which moves it into .rodata rather than .bss: on most architectures, this ensures that both the kernel's mapping of it and any aliases that are accessible via the kernel direct (linear) map are mapped read-only, and cannot be used (inadvertently or maliciously) to corrupt the contents of the zero page. Reviewed-by: Mike Rapoport (Microsoft) Reviewed-by: Kevin Brodsky Acked-by: David Hildenbrand (Arm) Reviewed-by: Jann Horn Reviewed-by: Feng Tang Signed-off-by: Ard Biesheuvel --- include/linux/pgtable.h | 2 +- mm/mm_init.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/include/linux/pgtable.h b/include/linux/pgtable.h index cdd68ed3ae1a..67aa23814010 100644 --- a/include/linux/pgtable.h +++ b/include/linux/pgtable.h @@ -1993,7 +1993,7 @@ static inline unsigned long zero_pfn(unsigned long ad= dr) return zero_page_pfn; } =20 -extern uint8_t empty_zero_page[PAGE_SIZE]; +extern const uint8_t empty_zero_page[PAGE_SIZE]; extern struct page *__zero_page; =20 static inline struct page *_zero_page(unsigned long addr) diff --git a/mm/mm_init.c b/mm/mm_init.c index f9f8e1af921c..46cf001238c5 100644 --- a/mm/mm_init.c +++ b/mm/mm_init.c @@ -57,7 +57,7 @@ unsigned long zero_page_pfn __ro_after_init; EXPORT_SYMBOL(zero_page_pfn); =20 #ifndef __HAVE_COLOR_ZERO_PAGE -uint8_t empty_zero_page[PAGE_SIZE] __page_aligned_bss; +const uint8_t empty_zero_page[PAGE_SIZE] __aligned(PAGE_SIZE); EXPORT_SYMBOL(empty_zero_page); =20 struct page *__zero_page __ro_after_init; --=20 2.54.0.794.g4f17f83d09-goog From nobody Mon Jun 8 20:41:28 2026 Received: from mail-wm1-f73.google.com (mail-wm1-f73.google.com [209.85.128.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 93D32425CEF for ; Tue, 26 May 2026 17:59:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779818383; cv=none; b=NFEHkhsATQjtob7pTHz5QxLYq9mOsziu6HPySS5QbEGXDrmdLuEPNq+ISAMxfY28oaLEk2/8T8ZC1FGz3gOy4RbV/xM73B2kHAfhQ5WtQNPGA4uU5a6abI7CUAO7VjVyA9DVC0CogcXx/8K1bAZs9sMlE8ysA5PL0y6QTi0olTY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779818383; c=relaxed/simple; bh=Bix0lMESrdQfOPiyFEdbITdZ4dbSHfscFezrIl5cdik=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=blNVFyWeYk9kJpJYD3qam4sulr8A1zMvYC7BmzBJXwuQAtdKymP5DVjb2FtynwvMpsxYuJlhGsPnQysKSRTVqON2Cs9dPq9dkBYsgEKHHY4ptCt44o2LFQ3woRZCH2LGlkJsIEWUJnWtTsdZpR7NLm4y6ZAo9w93zcG8Mrfxh1A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=usXleILj; arc=none smtp.client-ip=209.85.128.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="usXleILj" Received: by mail-wm1-f73.google.com with SMTP id 5b1f17b1804b1-48e89faa62eso64055295e9.1 for ; Tue, 26 May 2026 10:59:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1779818380; x=1780423180; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=YW83U9WXzmuOgO+qiU2xntn4xO4oQspRqeeRLd3+Ago=; b=usXleILjLezU0hyAGD3fJPqWLRDq5Ip5L8meUtXpx5eLoRlJ+tAmp2bGGxRENJ4xI8 a0hcBSKQl8oohcCv1jpOdCuFs3u+LUw5RrhkKfj8Vo7e4SiWQUA/JzAVjpQRK4lprQnM 5TaH0GJ0yfFLPIPfYK3wmevq6mJ+65ZUeRHOfI5A7bheezvA/boKTUrrDq9RBcxuxAqz ytasL4wIcEsxfZvZ4aqWe0xMTk0RleO8RlAqmu9kXPi62H29SPCNTaaDBL0vHmRL6wJF T5B/K0XFJGbV/MTjt9obPJcdhag/t0CjlmvepOyDzfWp2w1Ang7uTYDb0EGVfDiW3LvW xzvQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1779818380; x=1780423180; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=YW83U9WXzmuOgO+qiU2xntn4xO4oQspRqeeRLd3+Ago=; b=Q4XI5XkRPpGdy495RFokspro7S8YOMaflNqO2CjDtk4Sk6De1xeCcnYgLP8KNHeTJ/ hOdtqq8U5rMoKOikICs4vxVb8+oh0M7eT/iO8ZaZGOgG3t83D5EcIUcKSR+qVOCK5Mmf K1WN6bkp0YFGUZzR6sP/FP1/LPNhNOA5ywaVAwOd2wnDNW4RprIUezVJKmbQ+kwa0T7R LmSAvdxQoV73AlnQDDPRFRBDnRQxy2Yo1LAc40/WqmVQOx3ftuJwm7hssRAR9KfdD6mK Yle++a7lJtku8hWUDoWmiTg5LIhbS4R+euiOuAtvREub6oluWmYCFt2/64GXTnpEbt36 WBHA== X-Gm-Message-State: AOJu0YwI31etiUADzjTNOO+7YqaTnqZKn/6aAjQ8xkosTV+ZV73uE7Qi bgkURcCtmgfjSE9FA0dv7rUSbJKPguZkm7XHINGOmfW2HT30QPRBICCQHx3QBTCtskpg6fLHAw= = X-Received: from wmbet15.prod.google.com ([2002:a05:600c:818f:b0:490:3dc3:e5bb]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:c4a1:b0:490:3fd9:e78b with SMTP id 5b1f17b1804b1-490426cef8bmr337552265e9.17.1779818379722; Tue, 26 May 2026 10:59:39 -0700 (PDT) Date: Tue, 26 May 2026 19:59:02 +0200 In-Reply-To: <20260526175846.2694125-17-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260526175846.2694125-17-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=3044; i=ardb@kernel.org; h=from:subject; bh=yy+qAKwWsZGAcScio201064FyPP//0qmouvkE8IJvkQ=; b=owGbwMvMwCVmkMcZplerG8N4Wi2JIUv0fnaITtnLMqdFLhxOPsKdSl/fzf8vyMu45VaardtDA beLD6d2lLIwiHExyIopsgjM/vtu5+mJUrXOs2Rh5rAygQxh4OIUgInYcDP8lc9eXJtj5vLqXm5w sP0Cd7YzPcsCnzf0M7wXrV0bcOh0CsN/58OffBc4fZ3FmC48Z1Xn1KPu098u1d/xmnenxq61z05 +ZQEA X-Mailer: git-send-email 2.54.0.794.g4f17f83d09-goog Message-ID: <20260526175846.2694125-32-ardb+git@google.com> Subject: [PATCH v6 15/15] arm64: mm: Unmap kernel data/bss entirely from the linear map From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, Ard Biesheuvel , Ryan Roberts , Anshuman Khandual , Liz Prucka , Seth Jenkins , Kees Cook , Mike Rapoport , David Hildenbrand , Andrew Morton , Jann Horn , linux-mm@kvack.org, linux-hardening@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-sh@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel The linear aliases of the kernel text and rodata are mapped read-only in the linear map as well. Given that the contents of these regions are mostly identical to the version in the loadable image, mapping them read-only and leaving their contents visible is a reasonable hardening measure. Data and bss, however, are now also mapped read-only but the contents of these regions are more likely to contain data that we'd rather not leak. So let's unmap these entirely in the linear map when the kernel is running normally. When going into hibernation or waking up from it, these regions need to be mapped, so map the region initially, and toggle the valid bit so map/unmap the region as needed. (While the hibernation snapshot logic seems able to map inaccessible pages as needed, it currently disregards non-present pages entirely.) Signed-off-by: Ard Biesheuvel --- arch/arm64/mm/mmu.c | 39 +++++++++++++++++--- 1 file changed, 34 insertions(+), 5 deletions(-) diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index e7ca53d20b87..cb00e42abbe1 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -24,6 +24,7 @@ #include #include #include +#include #include #include #include @@ -1056,6 +1057,29 @@ static void __init __map_memblock(phys_addr_t start,= phys_addr_t end, end - start, prot, early_pgtable_alloc, flags); } =20 +static void remap_linear_data_alias(bool unmap) +{ + set_memory_valid((unsigned long)lm_alias(__init_end), + (unsigned long)(__bss_stop - __init_end) / PAGE_SIZE, + !unmap); +} + +static int arm64_hibernate_pm_notify(struct notifier_block *nb, + unsigned long mode, void *unused) +{ + switch (mode) { + default: + break; + case PM_POST_HIBERNATION: + remap_linear_data_alias(true); + break; + case PM_HIBERNATION_PREPARE: + remap_linear_data_alias(false); + break; + } + return 0; +} + void __init mark_linear_text_alias_ro(void) { /* @@ -1064,6 +1088,16 @@ void __init mark_linear_text_alias_ro(void) update_mapping_prot(__pa_symbol(_text), (unsigned long)lm_alias(_text), (unsigned long)__init_begin - (unsigned long)_text, PAGE_KERNEL_RO); + + remap_linear_data_alias(true); + + if (IS_ENABLED(CONFIG_HIBERNATION)) { + static struct notifier_block nb =3D { + .notifier_call =3D arm64_hibernate_pm_notify + }; + + register_pm_notifier(&nb); + } } =20 #ifdef CONFIG_KFENCE @@ -1189,11 +1223,6 @@ static void __init map_mem(void) __map_memblock(start, end, pgprot_tagged(PAGE_KERNEL), flags); } - - /* Map the kernel data/bss read-only in the linear map */ - __map_memblock(init_end, kernel_end, PAGE_KERNEL_RO, flags); - flush_tlb_kernel_range((unsigned long)lm_alias(__init_end), - (unsigned long)lm_alias(__bss_stop)); } =20 void mark_rodata_ro(void) --=20 2.54.0.794.g4f17f83d09-goog