From nobody Fri Apr 3 08:09:59 2026 Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DAE173A875E for ; Fri, 20 Mar 2026 15:00:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774018806; cv=none; b=AxBYnYTbRIy2gvLG01yjRDBqmGqyDgSSQ+b2cfK8NnAyrAjNZexkGDz/7Wp1Rsc24ggAqGsJqw+uwV2X+NNHV+gz+MwiZzAFy+cYlrOsYjNEScn3qL35Fj5jCA6W3wIgnvYBC9o+kWnwWCYE5etQWtzbvt+splm/xX//WM4leOw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774018806; c=relaxed/simple; bh=hSf9Tqj+M66MUxvWzisQlO7n4l0uBaFj5MijTyH9wpQ=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=SMe7aELKImdPx6zVlokNC2DxGhUOE6knUZANnTZzBNDsGzCbul3ztLG2GaqoTMXEI7qLAEI3NHsZvGO/AUHPYXvLtxusHPayIs96FRr3clhEYDqCDBQmjhRqzOA9sDtYaJIdy6P0h9eMZdB70pUMvr0TrU2lIJpEz857IPXDGWY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=irmfGjzw; arc=none smtp.client-ip=209.85.128.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="irmfGjzw" Received: by mail-wm1-f74.google.com with SMTP id 5b1f17b1804b1-486fc42c83aso14286915e9.0 for ; Fri, 20 Mar 2026 08:00:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1774018799; x=1774623599; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=r5a6/TT9mH0HWBhHsirKfDsnDlXwFBJ2biY25rPtLGE=; b=irmfGjzwx1lgjlyezDBpXjyUnJNqV7EbJRQ3zyGXM2Y+lY0Fr7ok9q3FNSz2iWj6xi vxB9esjobpAhSIoQxrtjlnuGtvJLeAQ5GuE79lk3WSDkIA2cTt/oRx3WF9OdVV7teQTj oLQD0f8svPggo02+b2LAw8iH0S0K1JWbTCgApRhuX2ZViecwin6fg4aWdFsE4fVeswWn 0Rg/xHhJ4hnl/63CxexRfEjIk8sBoZ9vlyc9oRu9yreQk79VYx3BvcuErinGadtmX2E0 fnQvfbdyLTU1enniUJ2evgzO/w+XbD7I9XHkOgdxBPcmR/qkfyMqO63JWbqxQfUEhP8R 8/Pg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1774018799; x=1774623599; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=r5a6/TT9mH0HWBhHsirKfDsnDlXwFBJ2biY25rPtLGE=; b=pficGc/OABIZFiZCOye++6v+9wa7EZ7so8+zLU7G0yZbkqGz/tZMFumg8DbUSXfxYI sIBNOtt5VSHB4e2ljFsKQvHpe3iXL+B/tfWpwe3qa9MB4PRnaedx5b63IcwMsCLA/MCY gYSElGePqJc3VrC3+4xzgld3NCu3dqRqDv+0RgAe5g5TgXrio5/SCdTPjJmDxl40pHWJ ptPFkn+wpPi6nz1Kc5dAJg8Ppit22syJIByBNq+wq83FkGTISzNJzFF49b2USjdiz4mX S1FwnsGLW3nSUXd/3huAioHCs2u6UOtJfmddZi20eqVRXOmHjakyfAx6fJ2hObbqzN8E Eggw== X-Gm-Message-State: AOJu0YzJgSyj7SXOvbwLSfTfq51aKi6kMa9CkbLD41EgVu1AvuOvpOf5 /aH9Clo1/zUBRhEsCBhPlsaF0FZPuVrogYERbK03AmgMP2VahTbBwSF5w8QIWWro9Bc3baoGuzd iajaFgOxJwR0aEPY7/4v779YNO5sZhvlm7g994xrZh1pw/8Ls9cXFt65gD8dsA9I6b2b8Dy1eqZ fcKp2bWkfc/0oe3EaP5eA9Kr2ab3Mm9qn6wQ== X-Received: from wmbha26.prod.google.com ([2002:a05:600c:861a:b0:486:fbd8:2af6]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:4e49:b0:486:f9a4:d487 with SMTP id 5b1f17b1804b1-486feb6253dmr47950535e9.0.1774018798971; Fri, 20 Mar 2026 07:59:58 -0700 (PDT) Date: Fri, 20 Mar 2026 15:59:36 +0100 In-Reply-To: <20260320145934.2349881-15-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260320145934.2349881-15-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=1378; i=ardb@kernel.org; h=from:subject; bh=RibIB1GLYG+hdvnhodqrMTwRdLcWjhU+FIfxTN8QGOE=; b=owGbwMvMwCVmkMcZplerG8N4Wi2JIXNvwg0NRZfn2x1sFK1v1Pz9rqZ67tSf5pz7BSbbI45lq r6JCj/SUcrCIMbFICumyCIw+++7nacnStU6z5KFmcPKBDKEgYtTACYSe4Lhf4av44tX7XM2z5jP tiQteavvlcf7rP4+8/f/HWmS78DCrcjwT7fZMPfNT6PIiPcsKSeFWxb9ELqz5sr8wg1Z953NNuz 9zAcA X-Mailer: git-send-email 2.53.0.959.g497ff81fa9-goog Message-ID: <20260320145934.2349881-16-ardb+git@google.com> Subject: [PATCH v3 01/13] arm64: Move the zero page to rodata From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: linux-arm-kernel@lists.infradead.org, will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, Ard Biesheuvel , Ryan Roberts , Anshuman Khandual , Liz Prucka , Seth Jenkins , Kees Cook , linux-hardening@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel The zero page should contain only zero bytes, and so mapping it read-write is unnecessary. Combine it with reserved_pg_dir, which lives in the read-only region of the kernel, and already serves a similar purpose. Reviewed-by: Ryan Roberts Signed-off-by: Ard Biesheuvel --- arch/arm64/kernel/vmlinux.lds.S | 1 + arch/arm64/mm/mmu.c | 3 +-- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.ld= s.S index 2964aad0362e..2d021a576e50 100644 --- a/arch/arm64/kernel/vmlinux.lds.S +++ b/arch/arm64/kernel/vmlinux.lds.S @@ -229,6 +229,7 @@ SECTIONS #endif =20 reserved_pg_dir =3D .; + empty_zero_page =3D .; . +=3D PAGE_SIZE; =20 swapper_pg_dir =3D .; diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index a6a00accf4f9..795743913ce5 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -66,9 +66,8 @@ long __section(".mmuoff.data.write") __early_cpu_boot_sta= tus; =20 /* * Empty_zero_page is a special page that is used for zero-initialized data - * and COW. + * and COW. Defined in the linker script. */ -unsigned long empty_zero_page[PAGE_SIZE / sizeof(unsigned long)] __page_al= igned_bss; EXPORT_SYMBOL(empty_zero_page); =20 static DEFINE_SPINLOCK(swapper_pgdir_lock); --=20 2.53.0.959.g497ff81fa9-goog From nobody Fri Apr 3 08:09:59 2026 Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4F8413C4542 for ; Fri, 20 Mar 2026 15:00:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774018806; cv=none; b=gbiII8RSkvCFDoH1fla8b27inxq+VAuAe9Sg6I+lZ3+AIxLK204N/XUTwbs/0lU44brs4UI8IT44nhe5mCNRTdXWuRCmBVtei5QiBAzE5aXVS/AURKhHth04ZBtQQpwwH9q1KnBe2e/u/HdUCImVgjIE449uTZU0K6vxs2nyxxQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774018806; c=relaxed/simple; bh=vQN4I39KPpSfNgcpJLnoBqAKyQ/8ns2ksOX2q4RXfa8=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Mjq/pZjbBF1eQ4/e+C8v7RW8AR3D3PBDvK20y5NaGOGHBMsQ0iEQrvVIMrr2VfKElb+10lFrqXZkiCfa5qhkjIjgWIslJwm4KFBNHC1b6ORSXV3HeTiTGMc4fnh5J8nmV0JRsC2vZyjCEBYoNolW78EXbslMOsQCvZcoER1Xvmo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=AVkO7KpO; arc=none smtp.client-ip=209.85.128.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="AVkO7KpO" Received: by mail-wm1-f74.google.com with SMTP id 5b1f17b1804b1-4852ccff333so19604485e9.2 for ; Fri, 20 Mar 2026 08:00:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1774018800; x=1774623600; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=Bpulp9qaDSG0AQBX1jMfHDsyJIuHHS4dbSktfX7imgM=; b=AVkO7KpODbQtmAyn9aCEXh87OfaBC/5f0TGGOla35eIDkGb8+xygB1NDs7ht/UMtDa 4agP6Feu329d8sL7V0WmVWgSnosdI6MSSihHT1hRXS24Xfhl+HEYnOKC9TGzWWcb5iBJ 636+sty3G1Z26VhXCpTdPie2smdM7kghWQFe8kxDNg/UL8eLfewiN9wV5mb7MWJZwOkL cyJoD2ODqL8YDvTrYuIjv+4fXwGsPBFbDOakiqzVlT+QVEj7FFgbvV4Qaj74V1sW7Uqi nGEREG6cCNDNQAtAamW0jyvBEBmsplr0RWMrwyEQ+AICrRsawou6ssq/xeWBsxo1Oteb ahmQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1774018800; x=1774623600; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Bpulp9qaDSG0AQBX1jMfHDsyJIuHHS4dbSktfX7imgM=; b=LmwEOsHnNQ0CglRq8svJKgwChiOBVypDkQ05H39igbdZeAaiU0tnISbwjJNc6ytByb 3tXPXvL3T3NiJKvcBR7MQhzE9CwQUOX30kUKwRO5pDNjUdpwz7VGhlAGklJbcjv8lr+p Zcw3juMXaEZtfNTtlON61eUvDxsaSMFCTKj6aYT7dY8UUH7400w1pqLi5qQMMeCJeBSo 3E2eaXUyzNFjSSsegsapjkHDMyjz3ePjpAsLX8PhNcCcxkzjPbg9D31lbWr/362xTpAB xUGYCeIihr9IVndqMd9NSzcNef+YPoyi2NIUQ6jIxtQx6PTbV653qLCoe+NSuznFGvXo Pl/A== X-Gm-Message-State: AOJu0Ywh7PWuwRCQrvZs9il6tFu4mEnfKHPfMGOVu6vxVcq/H/8Zqmv8 HppMfns7+2pZru8LBOIUO+0kHbJZ06mxor9oHKcGuAM2e4lUEJfFjvQTNWhCX7Q+8vRePvb3Hw/ TvbqiIdqFMbH4dP7V1Vd6osBaB59+xW08UMSipmvbMhjcBiGlz64+7v8TNzKi38kmLBGZ7/InrP 8EvOAwzbr/woBN0fA8TQwcr7RDn5Q+WTJJAA== X-Received: from wmhf4.prod.google.com ([2002:a7b:cc04:0:b0:485:4553:1a97]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:4e8e:b0:485:3e6c:aacb with SMTP id 5b1f17b1804b1-486feda8c69mr48236065e9.4.1774018799996; Fri, 20 Mar 2026 07:59:59 -0700 (PDT) Date: Fri, 20 Mar 2026 15:59:37 +0100 In-Reply-To: <20260320145934.2349881-15-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260320145934.2349881-15-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=1340; i=ardb@kernel.org; h=from:subject; bh=RNseX4mRXVBWe3ByFtnF7vDF/KyNT++5b73PmV91ziY=; b=owGbwMvMwCVmkMcZplerG8N4Wi2JIXNvwq2EdddL5C1WvNOwK94xj3Pq40+BMyeKKn7+bJ8Qu 499zbLpHaUsDGJcDLJiiiwCs/++23l6olSt8yxZmDmsTCBDGLg4BWAim9cx/Hfgi/mWLLjsk9uN F+teaatVvZ0RbGgj9CG4gPVB1aX0S3wMf7hu3pPafatOKSa/zt88y3a55KMrpRVVNqt75DvYwwq v8QEA X-Mailer: git-send-email 2.53.0.959.g497ff81fa9-goog Message-ID: <20260320145934.2349881-17-ardb+git@google.com> Subject: [PATCH v3 02/13] arm64: mm: Preserve existing table mappings when mapping DRAM From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: linux-arm-kernel@lists.infradead.org, will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, Ard Biesheuvel , Ryan Roberts , Anshuman Khandual , Liz Prucka , Seth Jenkins , Kees Cook , linux-hardening@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel Instead of blindly overwriting an existing table entry when mapping DRAM regions, take care not to replace a pre-existing table entry with a block entry. This permits the logic of mapping the kernel's linear alias to be simplified in a subsequent patch. Reviewed-by: Ryan Roberts Signed-off-by: Ard Biesheuvel --- arch/arm64/mm/mmu.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index 795743913ce5..9927b55022d8 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -262,7 +262,8 @@ static int init_pmd(pmd_t *pmdp, unsigned long addr, un= signed long end, =20 /* try section mapping first */ if (((addr | next | phys) & ~PMD_MASK) =3D=3D 0 && - (flags & NO_BLOCK_MAPPINGS) =3D=3D 0) { + (flags & NO_BLOCK_MAPPINGS) =3D=3D 0 && + !pmd_table(old_pmd)) { pmd_set_huge(pmdp, phys, prot); =20 /* @@ -385,7 +386,8 @@ static int alloc_init_pud(p4d_t *p4dp, unsigned long ad= dr, unsigned long end, */ if (pud_sect_supported() && ((addr | next | phys) & ~PUD_MASK) =3D=3D 0 && - (flags & NO_BLOCK_MAPPINGS) =3D=3D 0) { + (flags & NO_BLOCK_MAPPINGS) =3D=3D 0 && + !pud_table(old_pud)) { pud_set_huge(pudp, phys, prot); =20 /* --=20 2.53.0.959.g497ff81fa9-goog From nobody Fri Apr 3 08:09:59 2026 Received: from mail-wm1-f73.google.com (mail-wm1-f73.google.com [209.85.128.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 500DE3C4565 for ; Fri, 20 Mar 2026 15:00:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774018808; cv=none; b=t9mCyW3cE0w2YYQq4YMqcI7rD/Mm6+I0x3XJHEFlECkYQFPfilXpJFudVrit/YVnj2ti8b4DZXTIbMcFJNtf6tkIiYQYxxxIhGLBxpqOMPcLgOzymoKfAqBWmyKutaSXnHryalyaJASZ+zp8x0fxVONHfmRJbRl4s7pHBvTJCpI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774018808; c=relaxed/simple; bh=lPdFgqEE8oXtZZQFrMk9kg0bR0AvOvClQoLr8bca0gQ=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=orlPC3TomoCIiPEoSTbOCE1mmU4u9EmptWZFtMa0yMoNv3hEzk1J20+bwnj78nMtcJrw8ERPOy+8ewPr7UBK6hinc238/ug6Iz4QCJZaT2xdgH1fEzhEnL7Y8OOqwz2DOGTuhXeLChxvyjoySU6zRn0eDoRgV8K0BrGcvyhMUTU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=nTaW9sce; arc=none smtp.client-ip=209.85.128.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="nTaW9sce" Received: by mail-wm1-f73.google.com with SMTP id 5b1f17b1804b1-485c45885e6so30905175e9.0 for ; Fri, 20 Mar 2026 08:00:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1774018801; x=1774623601; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=neRKNeSqpvJnLheaedaHzSD+GAHs81oovoUYepc/lXo=; b=nTaW9sceDNi5sBhx9ePkdCY1jF6JS3TEFAaC+gmRnb0TUKgMHg0PSh4W0a7/Bl/ctk ismke/OvyabB9ImX6GId6zRJpzWzttmZslnYKhV3M1a9dL07b1loas20I+34Gp4uuXwi Ap2VnHFTMoCIh8ACn0llXQaojVUBl4h+AZWgRREqoaXjfo2w1e40m7URxOFtpYVWPAuz e5reEy9MGv+bvUZUwz4zjfOwv8SC6d9gIMZTmHazQSbx7OXA4E9ZHMlfHjmxdsr3YBgg x01mz/gvSgsV7HT5In692s7MP8DGeG2LJJloEkK6x3ieA57TF+5B8FFYWK0K7E/UkwEp kXPg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1774018801; x=1774623601; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=neRKNeSqpvJnLheaedaHzSD+GAHs81oovoUYepc/lXo=; b=aO4wedPTCqRdBLz80x6KQDHbk+SkLPgLYPJxgVszFmnCoIaJYZG6pCx6dI9XM0e9i7 KfRGgDNWEnGBHubraT/N5g1GIYmzp6ZOG5RaU7Vc32HIG+jEHWJRCEBCxMNk6jrgTRkn ht2a3qqO8PgbUmPZBOD2OLeUdXRtU6Dzc4Rl7XWg2rek2qnMxbzcOFdG8xi4qmuAclJ2 XmMyu/9F05l/IIfxj180PxA4r98vfHc+4nfEnbxFim+a4SEkc59kzgYSblwuE6cOh6HU N8NGGF/V99xUTPtyPTff8e1NngOMBxhI2q9T1AQUwt7oRmlUhoFRR3R7MXDkkHU0G49I 5uxg== X-Gm-Message-State: AOJu0YzBZeA1yTeKtlNuN5rrYB2Vc+t7k83y6D5NIbqcuZwd4B1VLYnF RMpWsGpOGxdW8vQOiRTCFokG+CLg7fmejk975E2gs1Y3X7dfy8TC5NyGCfGSzyI/TEfR8glfzXS xLTwvAN4DvnolG56rJeW6KRWkfPugtGok6VCFjsSQNaPJYxkg3f+rL7DD0EDI2lwu1YZphOOjCO AaGfeSnaZPfu9+lCQoCqt2g21gnV4zOLyRaQ== X-Received: from wmlf18.prod.google.com ([2002:a7b:c8d2:0:b0:483:7a98:f072]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:c48f:b0:485:b6dd:5066 with SMTP id 5b1f17b1804b1-486febb6014mr50607075e9.7.1774018800895; Fri, 20 Mar 2026 08:00:00 -0700 (PDT) Date: Fri, 20 Mar 2026 15:59:38 +0100 In-Reply-To: <20260320145934.2349881-15-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260320145934.2349881-15-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=2697; i=ardb@kernel.org; h=from:subject; bh=TuswSFvpHM3Mr71nHK8iSkeQ2cBVxUsQsRsYd8yRYuI=; b=owGbwMvMwCVmkMcZplerG8N4Wi2JIXNvwu1QGc17/9r21omerLN+XHuGaVdo3enIhfHzJrH+i Or3aT3YUcrCIMbFICumyCIw+++7nacnStU6z5KFmcPKBDKEgYtTACYi6sTIcHtC4bWNE784v+yf /Pguw5T9ay//LWk3CFO9K+rmvuEK60ZGhoXzPlp/tZjoyPd26e7G850npbKnlJ10WfX05sOIqul HbzIAAA== X-Mailer: git-send-email 2.53.0.959.g497ff81fa9-goog Message-ID: <20260320145934.2349881-18-ardb+git@google.com> Subject: [PATCH v3 03/13] arm64: mm: Preserve non-contiguous descriptors when mapping DRAM From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: linux-arm-kernel@lists.infradead.org, will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, Ard Biesheuvel , Ryan Roberts , Anshuman Khandual , Liz Prucka , Seth Jenkins , Kees Cook , linux-hardening@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel Instead of blindly overwriting existing live entries with the contiguous bit cleared when mapping DRAM regions, check whether the contiguous region in question starts with a descriptor that has the valid bit set and the contiguous bit cleared, and in that case, leave the contiguous bit unset on the entire region. This permits the logic of mapping the kernel's linear alias to be simplified in a subsequent patch. Note that not setting the contiguous bit on any of the descriptors in the contiguous region can only result in an invalid configuration if it was already invalid to begin with. Reviewed-by: Ryan Roberts Signed-off-by: Ard Biesheuvel --- arch/arm64/include/asm/pgtable.h | 4 ++++ arch/arm64/mm/mmu.c | 6 ++++-- 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgta= ble.h index b3e58735c49b..dc007043d86b 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -187,6 +187,10 @@ static inline pteval_t __phys_to_pte_val(phys_addr_t p= hys) * Returns true if the pte is valid and has the contiguous bit set. */ #define pte_valid_cont(pte) (pte_valid(pte) && pte_cont(pte)) +/* + * Returns true if the pte is valid and has the contiguous bit cleared. + */ +#define pte_valid_noncont(pte) (pte_valid(pte) && !pte_cont(pte)) /* * Could the pte be present in the TLB? We must check mm_tlb_flush_pending * so that we don't erroneously return false for pages that have been diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index 9927b55022d8..7f7d63009440 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -230,7 +230,8 @@ static int alloc_init_cont_pte(pmd_t *pmdp, unsigned lo= ng addr, =20 /* use a contiguous mapping if the range is suitably aligned */ if ((((addr | next | phys) & ~CONT_PTE_MASK) =3D=3D 0) && - (flags & NO_CONT_MAPPINGS) =3D=3D 0) + (flags & NO_CONT_MAPPINGS) =3D=3D 0 && + !pte_valid_noncont(__ptep_get(ptep))) __prot =3D __pgprot(pgprot_val(prot) | PTE_CONT); =20 init_pte(ptep, addr, next, phys, __prot); @@ -330,7 +331,8 @@ static int alloc_init_cont_pmd(pud_t *pudp, unsigned lo= ng addr, =20 /* use a contiguous mapping if the range is suitably aligned */ if ((((addr | next | phys) & ~CONT_PMD_MASK) =3D=3D 0) && - (flags & NO_CONT_MAPPINGS) =3D=3D 0) + (flags & NO_CONT_MAPPINGS) =3D=3D 0 && + !pte_valid_noncont(pmd_pte(READ_ONCE(*pmdp)))) __prot =3D __pgprot(pgprot_val(prot) | PTE_CONT); =20 ret =3D init_pmd(pmdp, addr, next, phys, __prot, pgtable_alloc, flags); --=20 2.53.0.959.g497ff81fa9-goog From nobody Fri Apr 3 08:09:59 2026 Received: from mail-wm1-f73.google.com (mail-wm1-f73.google.com [209.85.128.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 50423301704 for ; Fri, 20 Mar 2026 15:00:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774018808; cv=none; b=NUNFDMuRcHgcAn57dF0/Saplyt8vso6n+IO52P0F6zw9bA9NxUJ/4WYxcPJnrx9XZfJUtmDFkp92xp9ndayUOdJZjT035WGij1gNXGxh3fTTFseS3/r+3wo0/dsqBxQ4/zr70Rf1DZEu8ebFAesqvICepFWCxO+gaABOOcX6wak= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774018808; c=relaxed/simple; bh=N12j8EIF8rQgy0go0YZV4EDhJgbdVWCWVwT0HY9CkXc=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=RZPOXycL0P+e6ULAnTNQJ9zsC2etPnZIxRZomu+sCiTyO6v3m8LLUy2wbUL9vpk+xUGQz6BIIauIeFM1NrbozdSiKfAycCu9LO33omPrtlkgQY5oyfyv2IZPZedKLMAn1zvDUybatxyJKPesVi3/0l24ROTlYNxZp40OyBsskeM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=YUzPIQCV; arc=none smtp.client-ip=209.85.128.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="YUzPIQCV" Received: by mail-wm1-f73.google.com with SMTP id 5b1f17b1804b1-485345e2fdfso3274265e9.2 for ; Fri, 20 Mar 2026 08:00:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1774018802; x=1774623602; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=Ry+R/ZaKd/jQkpCgufXe2UFucnecXTxYuVWEy5sRefU=; b=YUzPIQCV9jfrB1P7Z8kjBS6yyyJEJ8RS/qU/xKTvJrrsE24HHNJM88EaxUvsytlI0/ OHBYrX42S/Dcwr6Ttozlm4dRuOrvuFLCiSIhwFIBOkf+0+tLvFT1apu41DprB1CI+siU yOK2RdGu9mkHLYyrd16ox3qUY6+k7hToaRlzDeiYKYco00ON9D4HhrwcqW8g4jXWHooI j9cMcavpuBV/5CML+Xoc/ncavRdv45ir8F1NAFvg/Hc2d/QEjnvh62VmXYhVSWPFVh+s qZE6fqejDzVCLU4jc6XMwh74uetFy2Yz8X//KHS8Lc3rSLdSbIqIGJTSlZGYA1Icr60T TF2w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1774018802; x=1774623602; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Ry+R/ZaKd/jQkpCgufXe2UFucnecXTxYuVWEy5sRefU=; b=iQ82xwiPF2VMZgeG08aiyZkeMYfsOTb+4QK0K6NNW7GwDqtLPKLPSMmfBvSFhYtIde YnD/2cdIhbNVnp//a0G5OAOcjC9E17iLRFUdX1/VvfFugFgYBB9XR81pG2B6aHu6RBp9 ofyDoNhClrc/PQ3rrOIcpnm3OMQj55Y+93PnavaiUO70XW3FCrbapCHjQpHEbaBxa53U myHry3I1wIATw0Dsgd+6hdDvOh4RM2t06LQWj5pyvPBNfaVGMopKJDU1HIPQUqAkb78S ZsmpYUsxqCX4WEiB4S/pjZNV75vg9DYRTzgW2Xybtxa9ZXtrDvmjuKsWg5y4m5Zr/TBE yCXA== X-Gm-Message-State: AOJu0Yyk+5CAGgu+gGsxscyY64KqUzeRLflcc1UZs9gPbK7ep7qI6thJ 1txmlFyu68qzOHFeVHBs3DMzHZMF/1w5rmVRSalYnmROIjBej6g2MND0bB1Uxig/TUsNzNOaUPB K9b6Ul3uKa5WlYLMsJfb+0yVlBLYoe9tyqiQcYAc//HPnPFQOm10nKEFjHeIRIS/JQ14G4nVglf EAeGCYlBYHc8hB+fQKqqfePRUNWdHziLzopg== X-Received: from wmow21.prod.google.com ([2002:a05:600c:4755:b0:485:6976:d608]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:3b07:b0:485:3abe:ab86 with SMTP id 5b1f17b1804b1-486fedab424mr52152315e9.4.1774018802093; Fri, 20 Mar 2026 08:00:02 -0700 (PDT) Date: Fri, 20 Mar 2026 15:59:39 +0100 In-Reply-To: <20260320145934.2349881-15-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260320145934.2349881-15-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=966; i=ardb@kernel.org; h=from:subject; bh=xcKv0iq0nxsp6X5JaqW8GiHsewOrGWIHuRHfq8YpCzU=; b=owGbwMvMwCVmkMcZplerG8N4Wi2JIXNvwp0GtsN3TltOPrsrt2BdV3b6jJx5clvuzr38q3frW 8FZxnvfdpSyMIhxMciKKbIIzP77bufpiVK1zrNkYeawMoEMYeDiFICJPI5mZLgfK623XG9/YMbb zItORtx3L8xmzvydcsxi1tbtaxmY5YsY/uk8vmXq83RThfOLXVyap3xeyITYpEfoRW/XPHlqWdE nE14A X-Mailer: git-send-email 2.53.0.959.g497ff81fa9-goog Message-ID: <20260320145934.2349881-19-ardb+git@google.com> Subject: [PATCH v3 04/13] arm64: mm: Remove bogus stop condition from map_mem() loop From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: linux-arm-kernel@lists.infradead.org, will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, Ard Biesheuvel , Ryan Roberts , Anshuman Khandual , Liz Prucka , Seth Jenkins , Kees Cook , linux-hardening@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel The memblock API guarantees that start is not greater than or equal to end, so there is no need to test it. And if were, it is doubtful that breaking out of the loop would be a reasonable course of action here (rather than attempting to map the remaining regions) So let's drop this check. Reviewed-by: Ryan Roberts Signed-off-by: Ard Biesheuvel --- arch/arm64/mm/mmu.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index 7f7d63009440..652fe2c52b5a 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -1157,8 +1157,6 @@ static void __init map_mem(pgd_t *pgdp) =20 /* map all the memory banks */ for_each_mem_range(i, &start, &end) { - if (start >=3D end) - break; /* * The linear map must allow allocation tags reading/writing * if MTE is present. Otherwise, it has the same attributes as --=20 2.53.0.959.g497ff81fa9-goog From nobody Fri Apr 3 08:09:59 2026 Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 88DFE3C5525 for ; Fri, 20 Mar 2026 15:00:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774018809; cv=none; b=V1UOqD0snqVeHCeJRNwXzWXYmmGtDD2VvAm4jILE+HjEAAh0h8qTYJWzfFX34dD1ZYb58hakS5Rf7o+fXJdy/PMAU1KVwTI2jvCZDRtruqRJ+t+Qm1l/QdPUP03qG+XOL33twG5khAGx348FahTDKYU2WvrqLP4kG+uZj3cLEPo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774018809; c=relaxed/simple; bh=u1362gPTWsQEK3Ox41VXDUHVwbj9c9rnh1coDsbboU8=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=kDAVxhHN1wKpwZUw3E09ysjW7PIh5UECj/qqGyykfw5Aj6uIUrzTql1i5tgIdxNJrob9yDVxIMxKXU0U1gueDrYQMV5TE7S93C4qCiYzmndZdHZyB9OQXENwsH3hqsG/c2htllfWwQMna0hUh1Z2JbAkntBvr78QJhF+hFumzZ4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=YdMEfUft; arc=none smtp.client-ip=209.85.128.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="YdMEfUft" Received: by mail-wm1-f74.google.com with SMTP id 5b1f17b1804b1-48542d5aa9eso5235445e9.0 for ; Fri, 20 Mar 2026 08:00:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1774018803; x=1774623603; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=5Jh/oU+9W3ai+MQNGkutnA3eVYC0Uphnsk6tMJ4kF8k=; b=YdMEfUftvnlRTZ5AiiysRUU4eO/oBf9JuKnnH6vWwy1384qpVoLOjdthMwj89+tS1L kziYcfsInzflamTw9CTdQhM7HjILYFEsKss4+S1pS6g1WPQlvJl2Y4ir00ThjyHBlmE2 Gu8GC3azIKXYibeeun/t3nxiUf9tv8feck482PPbW70cFtTiGXzmYEjLgYbWKYGQL0YS TPcNAoO7qMZPPgghKBJI2zsndt55FVdgI/Y97q3xaYtvUl/P8H6ooJ+DLX9YiwrScBAR IgPCSJZAFutxKjsLTZkMWPwiGX3+nADCu9ZBFoSTkm3GGSwQL+Yj07HPvQR92CevnQkI U+BA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1774018803; x=1774623603; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=5Jh/oU+9W3ai+MQNGkutnA3eVYC0Uphnsk6tMJ4kF8k=; b=oSZGHHTizp3fG0KuVVEHQDJJE+7unbuoUKrBd3UOaxQB7CNIFJhU8460XUHQmSvp7k 9iQhObUiaSjxM6hCFepgo+Hrav3Y6nVI6wrPCVaoTRl78XearxetJcejghwh0zaYxd1M 0z8ljf4B1F7eNrV2KULGet4TCRCGaSVxHF9k1ReJ84sREZb2/BrwFqtYBk64wRSd7BeR +gl75bFYZSGKRHQw0Wccqt908Fc6ckzHWm7SfPzV1u4JAgs7bk+XgAuae7eHvHi67i4l JglCz2QPxcnnX4vLs/d56GCOWgFdmwhpn1iOntSqVDTVcYmFjKCHrVL3JEXDDOiaMFYY Ljww== X-Gm-Message-State: AOJu0YxgMgFh8PvFOmMfjCPCVqbCKKgwldrBuX5k1KKNgP2j2Xi+WZZi iytxEHb1PNHOayupzQw3KBIJt4XnaLB+HY8X2O0jEEix7TbBpRFr9XGaZLt55to0EoWDFT0ULRt umV88sY22pzND7VE/YQ7kHIuJgCl6PYIJNSftw8lVgKU4TQel8CPCqY5x8Xs03pH2jqq3Ui5hX6 LdboX6rgBO8Wq/rKYdC8UzbPxXFYfogcMuNw== X-Received: from wmi11.prod.google.com ([2002:a05:600c:20b:b0:477:5a4b:d57f]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:5299:b0:486:fe46:b647 with SMTP id 5b1f17b1804b1-486fedbfd67mr48551215e9.10.1774018803211; Fri, 20 Mar 2026 08:00:03 -0700 (PDT) Date: Fri, 20 Mar 2026 15:59:40 +0100 In-Reply-To: <20260320145934.2349881-15-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260320145934.2349881-15-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=3387; i=ardb@kernel.org; h=from:subject; bh=5dRiPzrD0xyZ8rzbnMvrrnQQbnRprbflRHsa9Z++mUs=; b=owGbwMvMwCVmkMcZplerG8N4Wi2JIXNvwr17PDktXg+PWEvusPPc9vGt+LUQ2ZsvgwX3cE77/ WzZLsaDHaUsDGJcDLJiiiwCs/++23l6olSt8yxZmDmsTCBDGLg4BWAioYcZ/lkbHt4863555up0 h9ylYX8m9e1cPyPO4pT6kuUv8xPXz9vL8M/OvY/Tar30Hws7Q/357jxH3lzkNbDyZuJKvJ9wrEA gkwkA X-Mailer: git-send-email 2.53.0.959.g497ff81fa9-goog Message-ID: <20260320145934.2349881-20-ardb+git@google.com> Subject: [PATCH v3 05/13] arm64: mm: Drop redundant pgd_t* argument from map_mem() From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: linux-arm-kernel@lists.infradead.org, will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, Ard Biesheuvel , Ryan Roberts , Anshuman Khandual , Liz Prucka , Seth Jenkins , Kees Cook , linux-hardening@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel __map_memblock() and map_mem() always operate on swapper_pg_dir, so there is no need to pass around a pgd_t pointer between them. Reviewed-by: Ryan Roberts Signed-off-by: Ard Biesheuvel --- arch/arm64/mm/mmu.c | 25 ++++++++++---------- 1 file changed, 12 insertions(+), 13 deletions(-) diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index 652fe2c52b5a..744cf76f25aa 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -1019,11 +1019,11 @@ static void update_mapping_prot(phys_addr_t phys, u= nsigned long virt, flush_tlb_kernel_range(virt, virt + size); } =20 -static void __init __map_memblock(pgd_t *pgdp, phys_addr_t start, - phys_addr_t end, pgprot_t prot, int flags) +static void __init __map_memblock(phys_addr_t start, phys_addr_t end, + pgprot_t prot, int flags) { - early_create_pgd_mapping(pgdp, start, __phys_to_virt(start), end - start, - prot, early_pgtable_alloc, flags); + early_create_pgd_mapping(swapper_pg_dir, start, __phys_to_virt(start), + end - start, prot, early_pgtable_alloc, flags); } =20 void __init mark_linear_text_alias_ro(void) @@ -1071,13 +1071,13 @@ static phys_addr_t __init arm64_kfence_alloc_pool(v= oid) return kfence_pool; } =20 -static void __init arm64_kfence_map_pool(phys_addr_t kfence_pool, pgd_t *p= gdp) +static void __init arm64_kfence_map_pool(phys_addr_t kfence_pool) { if (!kfence_pool) return; =20 /* KFENCE pool needs page-level mapping. */ - __map_memblock(pgdp, kfence_pool, kfence_pool + KFENCE_POOL_SIZE, + __map_memblock(kfence_pool, kfence_pool + KFENCE_POOL_SIZE, pgprot_tagged(PAGE_KERNEL), NO_BLOCK_MAPPINGS | NO_CONT_MAPPINGS); memblock_clear_nomap(kfence_pool, KFENCE_POOL_SIZE); @@ -1113,11 +1113,11 @@ bool arch_kfence_init_pool(void) #else /* CONFIG_KFENCE */ =20 static inline phys_addr_t arm64_kfence_alloc_pool(void) { return 0; } -static inline void arm64_kfence_map_pool(phys_addr_t kfence_pool, pgd_t *p= gdp) { } +static inline void arm64_kfence_map_pool(phys_addr_t kfence_pool) { } =20 #endif /* CONFIG_KFENCE */ =20 -static void __init map_mem(pgd_t *pgdp) +static void __init map_mem(void) { static const u64 direct_map_end =3D _PAGE_END(VA_BITS_MIN); phys_addr_t kernel_start =3D __pa_symbol(_text); @@ -1162,7 +1162,7 @@ static void __init map_mem(pgd_t *pgdp) * if MTE is present. Otherwise, it has the same attributes as * PAGE_KERNEL. */ - __map_memblock(pgdp, start, end, pgprot_tagged(PAGE_KERNEL), + __map_memblock(start, end, pgprot_tagged(PAGE_KERNEL), flags); } =20 @@ -1176,10 +1176,9 @@ static void __init map_mem(pgd_t *pgdp) * Note that contiguous mappings cannot be remapped in this way, * so we should avoid them here. */ - __map_memblock(pgdp, kernel_start, kernel_end, - PAGE_KERNEL, NO_CONT_MAPPINGS); + __map_memblock(kernel_start, kernel_end, PAGE_KERNEL, NO_CONT_MAPPINGS); memblock_clear_nomap(kernel_start, kernel_end - kernel_start); - arm64_kfence_map_pool(early_kfence_pool, pgdp); + arm64_kfence_map_pool(early_kfence_pool); } =20 void mark_rodata_ro(void) @@ -1401,7 +1400,7 @@ static void __init create_idmap(void) =20 void __init paging_init(void) { - map_mem(swapper_pg_dir); + map_mem(); =20 memblock_allow_resize(); =20 --=20 2.53.0.959.g497ff81fa9-goog From nobody Fri Apr 3 08:09:59 2026 Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 82A263C5524 for ; Fri, 20 Mar 2026 15:00:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774018810; cv=none; b=MB+hwIKmLd1x96vbK+YPpLgnDTL0DTSY6QLiG3HO1iEmm3eGZv0eq3cDq2UMWOKEgW5BvNyoDR+QyatpBdV4Cw5O56RKWcuvxE6kYurs5ajuiIQlO2cSpv4ETm4OekZqb2SrD8nmI7nYZ9M5u4CAxaDZNoo0jVdjsaefxJ9nz2w= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774018810; c=relaxed/simple; bh=VqpVj+W0FM6b9m8WDPY3YbvaCmG28PnmLS9IjR3RHKo=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=cZwW6phvd0vOVKvlCIqviQOwLH+anQA4xsthH+iG8KFfcs/nPIpbAty++o51q/zduYeDZhFrGgHmjo1euVqmRPvcVXChvfsV+UNY8iQCmIXq1Y8I+vWVvUXTSDBfEECnra9s6HKXhWv7C9Vt81AQ5TkVTxjKdcT7tMHoUJc7+4g= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=fi5D94oP; arc=none smtp.client-ip=209.85.128.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="fi5D94oP" Received: by mail-wm1-f74.google.com with SMTP id 5b1f17b1804b1-486fe36cf73so7101955e9.1 for ; Fri, 20 Mar 2026 08:00:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1774018805; x=1774623605; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=FREiFmsRIXiOQcSDf+g4S+t23y0B3LOgQe0t9T04+rU=; b=fi5D94oPSe/zlJhcU/9OwaY5KG/DDr6G/M1edabSkl2bDsKI5zXg2ycGNyyJ3cr1zV 12cAcoM9pd0IKVwdwiRpupU0Qo4vQ17grQgH+5hz3MOYn119sjsErvv00TXfpJDbJ86X w0EHu0ZTcVMsyOP+F0PoNxf6ET2eOzNyc8afUSmFcbFcc7J0Zr4maUj7rmkePHr78oOs 8+BfHcqtATLnVz/IpwpI6z4zH6slhJ4CZZvcWpHDSfjlIWIstwrnLq7m7374G78dOar8 sPsuqTIAYvOw+oyT8TAprGI0N5pcHdfCGsGOqrdJWg8piD6VsDrxja0pR5Hslte/xi9N Z7rg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1774018805; x=1774623605; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=FREiFmsRIXiOQcSDf+g4S+t23y0B3LOgQe0t9T04+rU=; b=gdIqbp2pKSAtl1EvF23p5Uhkv1y8ojEYYH1pI6ZQKWNEcIlHgqSj+AWlchtDVkGI8w hBrTscWkqIXo3NIAhz5tkyibjINGw8nrFIHU3RBQUaVKaM9dGFHEN+7tTfg2nntnUGZU YZuVNWVoujWN8RI1g5Se4VceoxjvBUt0dCpZUj0aC7VEVW3V1QFz3baFphyy5NVxPiP4 Ag+/SUITQRUtzXsBTnEjdxLqJ+QRaaC4CJxrSbpkTnpdwiRC0yfmMgulJlmTuGNhmihL 3/DV8snGY7tDoKFLKK/6fxDVActjBEDjN7lbzZefSiVQPBFueZXqlb/D47pNn4gtwWoR eQpQ== X-Gm-Message-State: AOJu0YzzkIgkFzfdVc1vEcyuWAeE9rXub7KDGnJv+cd4eeGQ3mGYumKF Zn3qmj+1m+BpFCKeHiiT78rPN0O/WQnecGG7NPGgtEsYAObxRoyg11RBJftscHERfkOsYIs5FIe Unm1Bmo3tdL54zhXKZOhQy9szD4FqdmjKCRAOGPopi6pKNDh3MPfF3/37oZgHUUKMycyR8PXbxX QKUUy0iM9TedmTxn7qSruCrXuIggeKD+RKyA== X-Received: from wrwp1.prod.google.com ([2002:a5d:68c1:0:b0:43b:443c:fc00]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:1f0e:b0:486:5f71:5829 with SMTP id 5b1f17b1804b1-486fedab740mr49836565e9.5.1774018804352; Fri, 20 Mar 2026 08:00:04 -0700 (PDT) Date: Fri, 20 Mar 2026 15:59:41 +0100 In-Reply-To: <20260320145934.2349881-15-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260320145934.2349881-15-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=914; i=ardb@kernel.org; h=from:subject; bh=E+pO5/eHkkh2sajPGACS/nW0kkP4G/kfrd6uShLIRX4=; b=owGbwMvMwCVmkMcZplerG8N4Wi2JIXNvwn1xAS6VoMiOyA2+FtGnPNwL5JMFZkScquZs1dh6u t1XqKejlIVBjItBVkyRRWD233c7T0+UqnWeJQszh5UJZAgDF6cATCR+GSPDoSuMq7lCWb9aveeY 7Bn0TcF2hdf55QUZssayBlVT/HlnMjKs/79HqyM9/M2u1Qa/OsRvb2R5csxUmk3+gI7K2Rfx28T 4AQ== X-Mailer: git-send-email 2.53.0.959.g497ff81fa9-goog Message-ID: <20260320145934.2349881-21-ardb+git@google.com> Subject: [PATCH v3 06/13] arm64: mm: Permit contiguous descriptors to be rewritten From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: linux-arm-kernel@lists.infradead.org, will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, Ard Biesheuvel , Ryan Roberts , Anshuman Khandual , Liz Prucka , Seth Jenkins , Kees Cook , linux-hardening@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel Currently, pgattr_change_is_safe() is overly pedantic when it comes to descriptors with the contiguous hint attribute set, as it rejects assignments even if the old and the new value are the same. So relax the check to allow that. Signed-off-by: Ard Biesheuvel --- arch/arm64/mm/mmu.c | 4 ---- 1 file changed, 4 deletions(-) diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index 744cf76f25aa..6780236b6cf8 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -140,10 +140,6 @@ bool pgattr_change_is_safe(pteval_t old, pteval_t new) if (pte_pfn(__pte(old)) !=3D pte_pfn(__pte(new))) return false; =20 - /* live contiguous mappings may not be manipulated at all */ - if ((old | new) & PTE_CONT) - return false; - /* Transitioning from Non-Global to Global is unsafe */ if (old & ~new & PTE_NG) return false; --=20 2.53.0.959.g497ff81fa9-goog From nobody Fri Apr 3 08:09:59 2026 Received: from mail-wm1-f73.google.com (mail-wm1-f73.google.com [209.85.128.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 16D273C6611 for ; Fri, 20 Mar 2026 15:00:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774018812; cv=none; b=VoyuPNDg+XmBg9XtOWk53rMIEPvz9XgytLZFDeEIimkbTWu0Ax7oS8jZ0URADQL5tKLNZWqbFs/eNMV8u51stAvYjFO6WhxF7pg5FDrOp6/yAI8IVzvnh/kjr0jyELb1xnQeAH9e+VeXS34tuYyV0Qr50IeB8JdBW4pW1CIEE+E= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774018812; c=relaxed/simple; bh=canmmlI35APlaWJNQK0BwtfwphNlOC33tKxAIxRLctg=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=XUHAvBW4HjR9shfaA11RTlhYvxrF8OuSqwU94krtMQsu+LwaI8aExp2J2y49QuuPVqSJLY6YUX6AmF9PzstpuNB52q8iJT9SlN9rTOtBRwCQ8Js2naK0xbMVyyMUoGiTDwBgPoK6fqkToAtmBSI21eTOjv3zf9vkLJUFXGsWC74= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=e0EZ689I; arc=none smtp.client-ip=209.85.128.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="e0EZ689I" Received: by mail-wm1-f73.google.com with SMTP id 5b1f17b1804b1-485397788b3so15939405e9.2 for ; Fri, 20 Mar 2026 08:00:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1774018806; x=1774623606; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=+nHE9g4Iled9/dsrfh2bAJinc1yfJyvV2ys78zWRafI=; b=e0EZ689IBdRnk4PHf+07tz6ZUhlC890vRag/fppf5ejLUJMV1fVzqzC5U32oywIoht jLr1eddQwtybnwLUTRzTcAkn83+68oJtKoGnQz6zOl7p8fiyqwSgQPIOObuMDBNW09Rr zeKvOB4z2Nnm2RMdOnJ4KC190t+zBr72ZZZFKVpRy2khjzt5IqYUNVV7lPafkYSB1J/v TCQJ+qEbsH6yJuoiHqtS5A6//IrFhw4Pnv8IjO2QCCUrXd710f6ESYtbsxpOj0ABmZcj GmTzJjCiIazdQU7P+tIL/a3TYYd91Ib0BE0p4tB25QzGO8WgcjvnRf25AN57ONBpI3Sb Drbw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1774018806; x=1774623606; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=+nHE9g4Iled9/dsrfh2bAJinc1yfJyvV2ys78zWRafI=; b=adsRKsQQbYTVJNb2mpS2TGNljLx9vOdosUEYQhv4ECGzGCODKYvhlMstSuuh0OkL/f Xj+XqsJtA6UWp5+X4hPjjOqz3C2Cf/bIn0dZgHiK0rSWVV6gPVOH9ClQK+gkegKMdCZM aHIQUaNYN7m/eyEjFP4ZbznWQZDpzT7C1O0HQCPLmrtSJIJn+QF0+vuwvalRl+wnGmgl 8rpkwP9F8n1ukRxk7LhzISJhmOiEjEst1s7ucONtyjXjNa4SyXgp4sjbyYzMfvLBIKJf XN91XHCgN+5xveBWSGeWhk6v7zOEk08i9IaacXXZvBXXkYz03s3PMu8IM5Hu26Ffz/xH Ftig== X-Gm-Message-State: AOJu0YzLRs21fBy9Vm5w/seMgwxlLIZBtRmuK1VIdLDuONf6PHJ5D9um qqlmxnTuwN2vAKXaiwu2yorDDYp1ooqmzDi8K5T+MU6QNPQmp4OA+I3doA6CRqaEfl43lNan238 VZgPPXerkd0k7AK8DktXTrFnkyYtqcFDg4mIAgvzBvXSHXoHqx+Z0KjTpicDTGJHEtKJwc62IaJ y4CZ6zDbZ0hSNc8jY33K5IqphPk4yd4fnwgw== X-Received: from wmaj7.prod.google.com ([2002:a05:600c:6c07:b0:480:4a03:7b6f]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:4e8e:b0:483:c35d:3659 with SMTP id 5b1f17b1804b1-486fee04a48mr47773445e9.18.1774018805524; Fri, 20 Mar 2026 08:00:05 -0700 (PDT) Date: Fri, 20 Mar 2026 15:59:42 +0100 In-Reply-To: <20260320145934.2349881-15-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260320145934.2349881-15-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=1198; i=ardb@kernel.org; h=from:subject; bh=dhCmT0xhgO1wynakhoxtMTULOKMtch1e7OuwxF75oGw=; b=owGbwMvMwCVmkMcZplerG8N4Wi2JIXNvwoMI8+i12Utr1y67d7vnI7fbJp4TO31Wf/7f9fSZp X3oO/cfHaUsDGJcDLJiiiwCs/++23l6olSt8yxZmDmsTCBDGLg4BWAigkyMDGscNhv8F7vWM/FC Br+T9vd3Pc+uP/MPefnRf/d5MatA3WeMDPcY8y/NOCEmXsvdHe7z5c29JU1ceyKN9p2IN9+S579 /KTsA X-Mailer: git-send-email 2.53.0.959.g497ff81fa9-goog Message-ID: <20260320145934.2349881-22-ardb+git@google.com> Subject: [PATCH v3 07/13] arm64: mm: Use hierarchical XN mapping for the fixmap From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: linux-arm-kernel@lists.infradead.org, will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, Ard Biesheuvel , Ryan Roberts , Anshuman Khandual , Liz Prucka , Seth Jenkins , Kees Cook , linux-hardening@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel Nothing in the fixmap or in its vicinity requires executable permissions, and given that it is placed at exactly 1 GiB from the end of the virtual address space, we can safely set the hierarchical XN attributes on the level 2 table entries covering the fixmap, without running the risk of inadvertently taking away the executable permissions on an adjacent mappings. This is a hardening measure that reduces the risk of the fixmap being abused to create executable mappings in the kernel address space. Signed-off-by: Ard Biesheuvel --- arch/arm64/mm/fixmap.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/arch/arm64/mm/fixmap.c b/arch/arm64/mm/fixmap.c index c5c5425791da..c3dd3c868cf5 100644 --- a/arch/arm64/mm/fixmap.c +++ b/arch/arm64/mm/fixmap.c @@ -48,7 +48,8 @@ static void __init early_fixmap_init_pte(pmd_t *pmdp, uns= igned long addr) if (pmd_none(pmd)) { ptep =3D bm_pte[BM_PTE_TABLE_IDX(addr)]; __pmd_populate(pmdp, __pa_symbol(ptep), - PMD_TYPE_TABLE | PMD_TABLE_AF); + PMD_TYPE_TABLE | PMD_TABLE_AF | + PMD_TABLE_PXN | PMD_TABLE_UXN); } } =20 --=20 2.53.0.959.g497ff81fa9-goog From nobody Fri Apr 3 08:09:59 2026 Received: from mail-wm1-f73.google.com (mail-wm1-f73.google.com [209.85.128.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4ECB53C6A45 for ; Fri, 20 Mar 2026 15:00:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774018814; cv=none; b=NP2tLtVF/GqSv1XvczjH+pBq8EQD89Hhpiyf+1krDkjFl9lksLhAAyoHktpQFsshr0KaBxvXrtkE8NZu5pujgnStKridTVylWvxHoHc5jvCn/7UP41lWqQV3fDdVUlKaa3GI9ypAehZJI7HcYJ/ijVyuN73/R7MarHGPKTV2Mqw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774018814; c=relaxed/simple; bh=SnMvYXzc/W5zYVDsnz04o+AoFXRo83ibLCHafQE6S54=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=UomwNdO+WqW4l4IjBZuHjXrAWQh6EDkkOMkPpPg3st1+pHad/7qjmSWoRdnplQ1kHjPQYX4Z/707HSbH3c1PmWXR8d1VwaLeJCu8hau+l9+MGC7CDHNzFxGQhZ2LJOpOo8ZYQD0OpcpQUbMGs2btCezKlE/B23JHNbMMo6Thru8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=C/pSL+qB; arc=none smtp.client-ip=209.85.128.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="C/pSL+qB" Received: by mail-wm1-f73.google.com with SMTP id 5b1f17b1804b1-486f89b7f69so12728445e9.0 for ; Fri, 20 Mar 2026 08:00:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1774018807; x=1774623607; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=d5AT0J1SzBmKDTRI+MPd7NHNRTY3KrP/EHQWSy92u3Y=; b=C/pSL+qB6v0YHTnWHntgY8IeBRuvFQ/iXQO/8RqwhYoNfQrp/Uq+bZpdXcLqE9Y+Tw V+CspFH2SooDdZcT1E1XmFDN4m7Adbb8tTwoPkp6G1Di1HUVoYhuc9NFkNQ/TixmCmLw uJ4bsQGSoggjIj6Ke5KuzKsDNfHdryi7gQJp45eBpiGoP39Iqvb8qgXy6vOpojrEZjXH wF+FPrbTg9W9kRi1+WLKPqe/tYp39/pIfusqVisKERFL+atUpPiHtLe+ctHVsW0iDLQU T6CXRPrwiPsiDKpApudDkDbwrM+Kjh2Psf0eYQqhoZj5tp0B6XT30fBhiikwbScjJzPK 3fdg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1774018807; x=1774623607; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=d5AT0J1SzBmKDTRI+MPd7NHNRTY3KrP/EHQWSy92u3Y=; b=XiLN52Jw8NfQDhPIwv8qYcHniEUJAaICsBoLwW6YDAvZ8baECfyoGYSBtNMPGHCeV6 D0eL1isGSwnoY+tiekh63IslhVdmMH9odiPlgp8yr+2e2tFN1sDpnz+IV5gSQ6rOF8NW zmvWIepjq9rg82/2VwypwTmIQxE8DI9PfCx3L1H2eGzu5qzEX1Dk1xynSpEkkn6Vo+rw kCM7vzEyJAhzcUwlauWMh9kGT+tzarhS7hF+rIn1yBeq4mmnMSgFJkbQ4jNGgkupmEi9 pbI/u5PuePp1hQWmqjiZu4Ed5ZyI3QWJeyMn4mA9vNr6MC6U2ertcWxYqkwu0ebRlmnX IQaQ== X-Gm-Message-State: AOJu0YyW6K96p1Mpvc1uwJlrEKqALPOgrPVqz95KAn6mvLmVGMZjHMny 56jXjc8rhivF1e4Vy+huRIOkXUuygze3irp42lJAL7R6J46H8H//38Cr/sGeiovy/Ig1H5EBuE/ yfioD66UiK8Bb00jnEVkIqqtnO4CF0nVEaVOtt2eRbhHbORkWj+kOKg7pjfYaYAB9/pPGAW4+uj 2Ak45pBzAViOrxNudr/RJs7pt91pR52exmeA== X-Received: from wmbhc20.prod.google.com ([2002:a05:600c:8714:b0:486:fe34:ca4f]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600d:8401:b0:483:709e:f238 with SMTP id 5b1f17b1804b1-486fee297demr40658355e9.29.1774018807146; Fri, 20 Mar 2026 08:00:07 -0700 (PDT) Date: Fri, 20 Mar 2026 15:59:43 +0100 In-Reply-To: <20260320145934.2349881-15-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260320145934.2349881-15-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=3121; i=ardb@kernel.org; h=from:subject; bh=VnAD88hkLu5Up6Du4sjF1AveOX8gYjjnmNe0JbrOG+0=; b=owGbwMvMwCVmkMcZplerG8N4Wi2JIXNvwqMIO70jyYE/p2/KNi920Lz0X/ugW5nOKo/Md52XA 26Hz1vaUcrCIMbFICumyCIw+++7nacnStU6z5KFmcPKBDKEgYtTACZiO5WR4d+tzgX3g9Jerc87 E3eRxd664Iji4VvFHa9cJBU57ZbkBzMyPChvTPatULg0M6nmzqWtzAf+Zx1N+sL29ENg1lJm71n 5nAA= X-Mailer: git-send-email 2.53.0.959.g497ff81fa9-goog Message-ID: <20260320145934.2349881-23-ardb+git@google.com> Subject: [PATCH v3 08/13] arm64: kfence: Avoid NOMAP tricks when mapping the early pool From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: linux-arm-kernel@lists.infradead.org, will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, Ard Biesheuvel , Ryan Roberts , Anshuman Khandual , Liz Prucka , Seth Jenkins , Kees Cook , linux-hardening@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel Now that the map_mem() routines respect existing page mappings and contiguous granule sized blocks with the contiguous bit cleared, there is no longer a reason to play tricks with the memblock NOMAP attribute. Instead, the kfence pool can be allocated and mapped with page granularity first, and this granularity will be respected when the rest of DRAM is mapped later, even if block and contiguous mappings are allowed for the remainder of those mappings. Signed-off-by: Ard Biesheuvel --- arch/arm64/mm/mmu.c | 25 ++++---------------- 1 file changed, 5 insertions(+), 20 deletions(-) diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index 6780236b6cf8..1c434c242641 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -1047,36 +1047,24 @@ static int __init parse_kfence_early_init(char *arg) } early_param("kfence.sample_interval", parse_kfence_early_init); =20 -static phys_addr_t __init arm64_kfence_alloc_pool(void) +static void __init arm64_kfence_map_pool(void) { phys_addr_t kfence_pool; =20 if (!kfence_early_init) - return 0; + return; =20 kfence_pool =3D memblock_phys_alloc(KFENCE_POOL_SIZE, PAGE_SIZE); if (!kfence_pool) { pr_err("failed to allocate kfence pool\n"); kfence_early_init =3D false; - return 0; - } - - /* Temporarily mark as NOMAP. */ - memblock_mark_nomap(kfence_pool, KFENCE_POOL_SIZE); - - return kfence_pool; -} - -static void __init arm64_kfence_map_pool(phys_addr_t kfence_pool) -{ - if (!kfence_pool) return; + } =20 /* KFENCE pool needs page-level mapping. */ __map_memblock(kfence_pool, kfence_pool + KFENCE_POOL_SIZE, pgprot_tagged(PAGE_KERNEL), NO_BLOCK_MAPPINGS | NO_CONT_MAPPINGS); - memblock_clear_nomap(kfence_pool, KFENCE_POOL_SIZE); __kfence_pool =3D phys_to_virt(kfence_pool); } =20 @@ -1108,8 +1096,7 @@ bool arch_kfence_init_pool(void) } #else /* CONFIG_KFENCE */ =20 -static inline phys_addr_t arm64_kfence_alloc_pool(void) { return 0; } -static inline void arm64_kfence_map_pool(phys_addr_t kfence_pool) { } +static inline void arm64_kfence_map_pool(void) { } =20 #endif /* CONFIG_KFENCE */ =20 @@ -1119,7 +1106,6 @@ static void __init map_mem(void) phys_addr_t kernel_start =3D __pa_symbol(_text); phys_addr_t kernel_end =3D __pa_symbol(__init_begin); phys_addr_t start, end; - phys_addr_t early_kfence_pool; int flags =3D NO_EXEC_MAPPINGS; u64 i; =20 @@ -1136,7 +1122,7 @@ static void __init map_mem(void) BUILD_BUG_ON(pgd_index(direct_map_end - 1) =3D=3D pgd_index(direct_map_en= d) && pgd_index(_PAGE_OFFSET(VA_BITS_MIN)) !=3D PTRS_PER_PGD - 1); =20 - early_kfence_pool =3D arm64_kfence_alloc_pool(); + arm64_kfence_map_pool(); =20 linear_map_requires_bbml2 =3D !force_pte_mapping() && can_set_direct_map(= ); =20 @@ -1174,7 +1160,6 @@ static void __init map_mem(void) */ __map_memblock(kernel_start, kernel_end, PAGE_KERNEL, NO_CONT_MAPPINGS); memblock_clear_nomap(kernel_start, kernel_end - kernel_start); - arm64_kfence_map_pool(early_kfence_pool); } =20 void mark_rodata_ro(void) --=20 2.53.0.959.g497ff81fa9-goog From nobody Fri Apr 3 08:09:59 2026 Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6051C3C73ED for ; Fri, 20 Mar 2026 15:00:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774018815; cv=none; b=j0QvN9rXDmtbS86lEVBZ+RHoa3QR62c9gWU190AdCFEMicCz1/tIHumAh/bzVz3+fbBCiuxnHSvMXnq3FTJrlQGYvDejJKPQi+X1cs8HkvcmXsgy7tfD7HYw2u5XteoG12IRAnVTM4dz3UFXNZERws2D6KU4ixA3jLl6qcgfffg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774018815; c=relaxed/simple; bh=9xYyHjmXPOhumsnBnYpVBRzZTGFurfLp1nmOTwbhhEg=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=NkXpkDQ3dm1P7R2oHLdHQF9jET659c6FEo+Yr3s5L2Icg9bwPzy4yYNvf2DXtkcvYIQvipFWBLU4QY9lhGJprCrF8o3iqcqeLjKCdeL8tL7051ozJY7CoBepC0d8Igbo8OwrRGqyu/5n8d3uV8ZNQn4tuPok5hHR4NaV828Anw0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=wNa8Q8Ge; arc=none smtp.client-ip=209.85.128.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="wNa8Q8Ge" Received: by mail-wm1-f74.google.com with SMTP id 5b1f17b1804b1-486fe3b9441so7200075e9.3 for ; Fri, 20 Mar 2026 08:00:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1774018808; x=1774623608; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=22ZPIRvlYCrswgtwd4fGeLcC3pajXKmPw3uCdobj4b4=; b=wNa8Q8Ge8Frf0gcPOm+jzzGvBVeehsIKI7a4hSZ6MiZUBfP01fwYmLi4mUKeJeA8Pc t2M207mnp4a2r4rglI5Hfi+EmsGEIyOzubZFfO0ZGDVIm82TSbRigAxzrTwWqNsqgy3N o1ixQMkWAefA1A8XaT/0ho4eLmlLb4e4Zqudd7FDjm1zbsn29Ja+Xf9GAm7DJ2LOWcJK EE+IxGOCe8lqx4BwAjalBHipnnMPcw6Z3geg1IvgC5i7TI5UPJGMWecSXi2gIDFIYjsa Jovu/OvWvc4GyrAqHnb3RKweJuMe8O86cfonIVouiPZcyBIwXDTl7L3TiPolDJJt31Dk X/6g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1774018808; x=1774623608; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=22ZPIRvlYCrswgtwd4fGeLcC3pajXKmPw3uCdobj4b4=; b=PzEi2NtfWQ/lSAkR8yiLQbcTKnJQ2lQ7PVU5ukG5O+rM+1fpTwVu94u5GkLGSvTm/F ARscanbjMuf04bzQf+Px+6sgD+QkF+4peWLu+RUdJPzT9V3PfAgio8DGgkVJjmVMGk1F 95d3Rf6l/1SpUTyWDuzCWn/dj8EhNy4Z9ZeBQXUN/JD5m02MRDsf9VBd4kYI0I4zjGwI tB3HB/0ZpNHtB9jGGzzxIDfBQwlf26Abs/PMFudh56Tt5nsUMqWu0HpXdYEccsYov0Hx plAqcQKQqe/fpwyqbOlw4N7Jty39arZ3ZPjLgCLR1IUoaMfR108TzbR17kfXtahWIRJd QzmQ== X-Gm-Message-State: AOJu0Ywo60Hr/fWLDlIAh4xKXe7MI86Y9P/Wl+Dh0XkgzL36cIN9TNs+ 6PYTJg8Va8Cn6lBf5jv0CRSaW3pCXObUktifUThBNQm8A8T3kb1T0K5LQmd07WKlesh0xix+13e TwD3Ay+KHuoEoMklEMv1phxnkMnC6Bjum0MaL8Lg4VzAKbTmiVrvN0GY7WTE3sYW5tcMYqkZ6N+ yFqzKIkbzQ84AXlNV9McM5PVkAgjFZapc7QA== X-Received: from wmej5.prod.google.com ([2002:a05:600c:42c5:b0:483:6f09:9913]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:1f86:b0:485:17a7:b9c7 with SMTP id 5b1f17b1804b1-486fedb551dmr52900775e9.10.1774018808233; Fri, 20 Mar 2026 08:00:08 -0700 (PDT) Date: Fri, 20 Mar 2026 15:59:44 +0100 In-Reply-To: <20260320145934.2349881-15-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260320145934.2349881-15-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=2889; i=ardb@kernel.org; h=from:subject; bh=X4PynXinCgtnG4ju4w4ZcwweYSkVMNJ0BLV72qdGaw0=; b=owGbwMvMwCVmkMcZplerG8N4Wi2JIXNvwuOqFzV7PEyd+lTWXMvNXC+b5vGz/NNK1o2nmQ0Mf l1mtbHqKGVhEONikBVTZBGY/ffdztMTpWqdZ8nCzGFlAhnCwMUpABNpl2BkeD/30OcX73f4uuzi npme+cGe9cL318K/OkU+3+fdMynBcxIjw2Y7lsv3p6UZMvgV9xStK7dMnnivY/qvWRF22h3urZc 28wMA X-Mailer: git-send-email 2.53.0.959.g497ff81fa9-goog Message-ID: <20260320145934.2349881-24-ardb+git@google.com> Subject: [PATCH v3 09/13] arm64: mm: Permit contiguous attribute for preliminary mappings From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: linux-arm-kernel@lists.infradead.org, will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, Ard Biesheuvel , Ryan Roberts , Anshuman Khandual , Liz Prucka , Seth Jenkins , Kees Cook , linux-hardening@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel There are a few cases where we omit the contiguous hint for mappings that start out as read-write and are remapped read-only later, on the basis that manipulating live descriptors with the PTE_CONT attribute set is unsafe. When support for the contiguous hint was added to the code, the ARM ARM was ambiguous about this, and so we erred on the side of caution. In the meantime, this has been clarified [0], and regions that will be remapped in their entirety can use the contiguous hint both in the initial mapping as well as the one that replaces it. Note that this requires that the logic that may be called to remap overlapping regions respects existing valid descriptors that have the contiguous bit cleared. So omit the NO_CONT_MAPPINGS flag in places where it is unneeded. Thanks to Ryan for the reference. [0] RJQQTC For a TLB lookup in a contiguous region mapped by translation table entries= that have consistent values for the Contiguous bit, but have the OA, attributes,= or permissions misprogrammed, that TLB lookup is permitted to produce an OA, a= ccess permissions, and memory attributes that are consistent with any one of the programmed translation table values. Signed-off-by: Ard Biesheuvel --- arch/arm64/mm/mmu.c | 10 +++------- 1 file changed, 3 insertions(+), 7 deletions(-) diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index 1c434c242641..b52254790fda 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -980,8 +980,7 @@ void __init create_mapping_noalloc(phys_addr_t phys, un= signed long virt, &phys, virt); return; } - early_create_pgd_mapping(init_mm.pgd, phys, virt, size, prot, NULL, - NO_CONT_MAPPINGS); + early_create_pgd_mapping(init_mm.pgd, phys, virt, size, prot, NULL, 0); } =20 void __init create_pgd_mapping(struct mm_struct *mm, phys_addr_t phys, @@ -1008,8 +1007,7 @@ static void update_mapping_prot(phys_addr_t phys, uns= igned long virt, return; } =20 - early_create_pgd_mapping(init_mm.pgd, phys, virt, size, prot, NULL, - NO_CONT_MAPPINGS); + early_create_pgd_mapping(init_mm.pgd, phys, virt, size, prot, NULL, 0); =20 /* flush the TLBs after updating live kernel mappings */ flush_tlb_kernel_range(virt, virt + size); @@ -1155,10 +1153,8 @@ static void __init map_mem(void) * alternative patching has completed). This makes the contents * of the region accessible to subsystems such as hibernate, * but protects it from inadvertent modification or execution. - * Note that contiguous mappings cannot be remapped in this way, - * so we should avoid them here. */ - __map_memblock(kernel_start, kernel_end, PAGE_KERNEL, NO_CONT_MAPPINGS); + __map_memblock(kernel_start, kernel_end, PAGE_KERNEL, 0); memblock_clear_nomap(kernel_start, kernel_end - kernel_start); } =20 --=20 2.53.0.959.g497ff81fa9-goog From nobody Fri Apr 3 08:09:59 2026 Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 45DC43C456B for ; Fri, 20 Mar 2026 15:00:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774018815; cv=none; b=Su1CW8fQffkxfngE9uI95CAX7Uk9ZekPdGr0wm/kaYrU95So8RXnZBD9lHVgqJIEQ06E6PgIIPgNKSrrduwiW/Yt3mXpxZtcGos62+pHZNQaoQ0ieMbmnvWqWbw8eeLf8/PlcS8nr6xvQPSjsxYHZDRHZCKImYb9ShaUPaeEj28= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774018815; c=relaxed/simple; bh=HoCluaTNl9G0txdMgRZVXDTrF1dgE/9ZbkNP4CHKFxs=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=BotV6dY4habH0i87KsgNdcDiHlM8l0UREJCZ1STGO/AavXrqsBh8aBeHls+t+GB8VCd/AhWw5fpwDoTqjVJPnmvHr+VvjfkrfWo7wmiAl3q3+QdKLrgo7Mx3eORNjlkZRv1WlhRcfMBlPolL1U0fuo+MjyfcQ5qi4zGDbPUoNuI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=fSQBrSnS; arc=none smtp.client-ip=209.85.128.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="fSQBrSnS" Received: by mail-wm1-f74.google.com with SMTP id 5b1f17b1804b1-48531e6012bso35462495e9.1 for ; Fri, 20 Mar 2026 08:00:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1774018809; x=1774623609; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=CjGy6iI8Lh1+RRlATGg4ihT8EoGsIDJChLyr1UIJtV4=; b=fSQBrSnSJ2Inqlvki+Ny1DziYrYOFP6nvAnQJE8BTMqfCuQQCDsrn35vgKUMZokyJN 5+DEHKZKe6nI4Qf9fXORnlH41XMBYy/Ryemskfda3Vte8b4lNfbvUTe31l1sSdYIV/T7 pfa26sSbvPPqfiVJRfU121veheghrARDBDhAi/2EqoOqxLk3d9lgG0BDOhjJhG69tSqy dK9ekAdapQCePYMVX0++hIpjbDRKwOt1zNRLe4hJA4Mip0ok8Zc8cyVUfB5bOj9bFLub bZEStEs0iCp3Na5AwymKQCdCsR5PfDCvG6NKCi12b0tt46Q1vy/svshXbXcuCILOmofZ Nybw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1774018809; x=1774623609; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=CjGy6iI8Lh1+RRlATGg4ihT8EoGsIDJChLyr1UIJtV4=; b=N/+OyIa3Ewb00/QjXDzgPdMG5+ZckGPk23SW+MhWHDO0D78xCN7L96/73sTHEz6yZi 18uN6LntMHmv2+hpRL7Vah4NpbNK0pDiD0ICrni8bWpWBg/wMZgPoM3DSWX85oq6gm28 V7/vvqaTv5Dl1+cVy9FS7jI6lbgjASdHXxpYlU9+ZxEH5PMV+yLNdceZTevolRb1/erz QyiCoEjP7qEQr4K4dVLJqTPAIWEh0IDoh/rzqah9oTR1ludkL9UgWxDoyuaGhfwXTrxb Qbx/Lx5L77qPgUjEnra4lfd0O7YCdFWi1bAFDpRSXx2Sle/d95irulKY5B+wqg01DMOC vFnw== X-Gm-Message-State: AOJu0YwZlb9CQ7ozGa9iH91tZsSCfXliEWF4rXmgyopmjd+tTsIvmq3c DB3RbI+VH0BTEhuSyjZH2v0KF35NcYKoiZIhQRANzlRMeVzy0hfDxTHCKKSq4DnqvrIwbbywfSK hTUlsWnf9KwxVWmoHw+7xJDlS5gOiIeOzCGFI19iZ8vyXZ1yWDiXFULyXSTnV6NN4baOjs58R1U zxQuw2Zmja+lY2MEo9JbR+EpMnBCOKuTkRmw== X-Received: from wmro18.prod.google.com ([2002:a05:600c:3792:b0:485:3f57:3523]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:c48f:b0:485:2a85:e5ec with SMTP id 5b1f17b1804b1-486febb60c6mr50654835e9.2.1774018809101; Fri, 20 Mar 2026 08:00:09 -0700 (PDT) Date: Fri, 20 Mar 2026 15:59:45 +0100 In-Reply-To: <20260320145934.2349881-15-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260320145934.2349881-15-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=1883; i=ardb@kernel.org; h=from:subject; bh=Tdi7XhvZoK6DyplxF9vlkCNhMDHkkf+zkyeuXcEumuM=; b=owGbwMvMwCVmkMcZplerG8N4Wi2JIXNvwhMJ6eNTvln2M575eF1IP7qYb3vbFKfHNqy3N/jId d+5fOdzRykLgxgXg6yYIovA7L/vdp6eKFXrPEsWZg4rE8gQBi5OAZiIrQbD/7LHP8Je2fJcfpVk dpgv73/XeeYJrRf97U0O6DhwrL3CspThf3Fi7Lk6pYKFkV9/vA9W28FQfbjy7fHJoh8antZMv5h mygIA X-Mailer: git-send-email 2.53.0.959.g497ff81fa9-goog Message-ID: <20260320145934.2349881-25-ardb+git@google.com> Subject: [PATCH v3 10/13] arm64: Move fixmap page tables to end of kernel image From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: linux-arm-kernel@lists.infradead.org, will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, Ard Biesheuvel , Ryan Roberts , Anshuman Khandual , Liz Prucka , Seth Jenkins , Kees Cook , linux-hardening@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel Move the fixmap page tables out of the BSS section, and place them at the end of the image, right before the init_pg_dir section where some of the other statically allocated page tables live. These page tables are currently the only data objects in vmlinux that are meant to be accessed via the kernel image's linear alias, and so placing them together allows the remainder of the data/bss section to be remapped read-only or unmapped entirely. Signed-off-by: Ard Biesheuvel --- arch/arm64/kernel/vmlinux.lds.S | 5 +++++ arch/arm64/mm/fixmap.c | 5 +++-- 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.ld= s.S index 2d021a576e50..282516def39c 100644 --- a/arch/arm64/kernel/vmlinux.lds.S +++ b/arch/arm64/kernel/vmlinux.lds.S @@ -335,6 +335,11 @@ SECTIONS __pi___bss_start =3D __bss_start; =20 . =3D ALIGN(PAGE_SIZE); + .pgdir : { + __pgdir_start =3D .; + *(.fixmap_bss) + } + __pi_init_pg_dir =3D .; . +=3D INIT_DIR_SIZE; __pi_init_pg_end =3D .; diff --git a/arch/arm64/mm/fixmap.c b/arch/arm64/mm/fixmap.c index c3dd3c868cf5..30aba998cf38 100644 --- a/arch/arm64/mm/fixmap.c +++ b/arch/arm64/mm/fixmap.c @@ -31,9 +31,10 @@ static_assert(NR_BM_PMD_TABLES =3D=3D 1); =20 #define BM_PTE_TABLE_IDX(addr) __BM_TABLE_IDX(addr, PMD_SHIFT) =20 +#define __fixmap_bss __section(".fixmap_bss") __aligned(PAGE_SIZE) static pte_t bm_pte[NR_BM_PTE_TABLES][PTRS_PER_PTE] __page_aligned_bss; -static pmd_t bm_pmd[PTRS_PER_PMD] __page_aligned_bss __maybe_unused; -static pud_t bm_pud[PTRS_PER_PUD] __page_aligned_bss __maybe_unused; +static pmd_t bm_pmd[PTRS_PER_PMD] __fixmap_bss __maybe_unused; +static pud_t bm_pud[PTRS_PER_PUD] __fixmap_bss __maybe_unused; =20 static inline pte_t *fixmap_pte(unsigned long addr) { --=20 2.53.0.959.g497ff81fa9-goog From nobody Fri Apr 3 08:09:59 2026 Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BA20A3C9442 for ; Fri, 20 Mar 2026 15:00:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774018818; cv=none; b=IHepgbdz3E7UV8I+b2aj8ysZpWRB5OGDRBMZGDmQHbB86D933FPXYiDYk2VrAYO/JDeFGtu+Q4cwZcUx/6pD7D5PEdh+KzE3JsKWhFmzJ32AHtY7CF6dmmUBAvgyjkkNXxSWOCdNWKG6Xjw79n1G7yvMWEkKmWZv02GDQgujPK0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774018818; c=relaxed/simple; bh=HVhid55KAMA64LXH8K7EKWVDpjjCZUptBd/c4Vbba8U=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=mEgdQn3lSbPM5SlwZri9clBlKYtnSCZSVQf8oSc+eu1BfRj8lqVwTEqf6lLcH+s6+S0IADDuCUu8QPFgKc+sdNVO1TtXvZHh3ZbZS+4zbDhf8veRcfZGvSyMNOq6+JTyozYeV3rclKjrADJ25cbrdRe7jzv6x4ABNt9crEwOgrQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=Uz4RZ9Ox; arc=none smtp.client-ip=209.85.128.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="Uz4RZ9Ox" Received: by mail-wm1-f74.google.com with SMTP id 5b1f17b1804b1-486ff4498b2so5344615e9.0 for ; Fri, 20 Mar 2026 08:00:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1774018812; x=1774623612; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=mjzcF7K4l6kuyuHp5hOrQQC0yTx4zgF02dd3Je0OTIw=; b=Uz4RZ9Ox+VSuV/A/HrOVowMzXNWven71FWTwP8diFNGmlmGIJn1eLuEdPhQXl0ZCuW Y6JmEquTMpqR79BGg1AeSkYOgHvD/U8zBZxItn8R5Ptlc6lXPFBpAMIukz66yb1k7VeN tFY31UjCBUb3UxHLGpz0ui4L+IFsSjFdAPXRJSusW5RKRQBTEkwdvS4KSFPZ2warE4VA zvegqZqetbMvYmYOHqV4FmikXZC0qLM1kSE1nCVJ6a1CC/7BS48wnlZeaiTwxewHtMm2 T90+CeJWf8LouDqL7DKiCIzx+f6CZZYKvXbmhDKH72+Cn8Z+wt1QjsbCQQdB7lguqIbJ Z1fQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1774018812; x=1774623612; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=mjzcF7K4l6kuyuHp5hOrQQC0yTx4zgF02dd3Je0OTIw=; b=nhig558mYKj9a7X7+7ARdivNx/d6XoDGV2GFK8dLKu31CJ2xi5oanv3ck0vaPvXF8t O+4+wSRketf/gz8UEN6w79C9gqwJW1V4P2vI3dS0I3Ns0rlRR9X9pQfLWiZQYADPMGIs k3dY2bxGXmJvWBKECW0DAnOx3Pc5BTzOJYnef5cv1++jIx5cB0xdtnLvNnK4PUHxjUzu 1sXVUUgxh57lsc6dahbUT1kKWKRc/YBQsV8lHOojgaxuVyTEbXYdGkrz77p+XrEP/pqw swXMA7sFgT9nDRA18vKtbQqdl00F/0ZVjNpqKsnfQbG3HN6Q0Lvsqa9uXxUtxPC3OpRw 8iTg== X-Gm-Message-State: AOJu0YwKZeg6BA0z+hrAAQBs1BI+Yust3RetGO69K8hppOWCu5g9yDOr shT5fTL4vZtczHEtlUzNPEMuEh5KJnljnqcobMLE/UCrS5QbulTGNKqqUe+W0mLPc559PAQwcDl ywOU1j+10ZyyeGf+6qKPqQagiyJyJLk7SDtMBE/zhi80wllR+8UptaXrnApJ1vMYK7Trzghlxq0 kplGygcC2rDZM41ITWSFxAnMp4C+jNB/8sQA== X-Received: from wrbcp39.prod.google.com ([2002:a05:6000:4027:b0:43b:5b92:aa63]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:a402:b0:485:5981:1423 with SMTP id 5b1f17b1804b1-486fede720fmr41862035e9.3.1774018810476; Fri, 20 Mar 2026 08:00:10 -0700 (PDT) Date: Fri, 20 Mar 2026 15:59:46 +0100 In-Reply-To: <20260320145934.2349881-15-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260320145934.2349881-15-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=2314; i=ardb@kernel.org; h=from:subject; bh=e+g1FyvCjZvn4UCkoUHoA+5i42in8tqyf6pjup/50iM=; b=owGbwMvMwCVmkMcZplerG8N4Wi2JIXNvwrPN+XcrfM5u2LGPW+l33FZpAYFdgfsmi8y9Msn/u Okuo4KajlIWBjEuBlkxRRaB2X/f7Tw9UarWeZYszBxWJpAhDFycAjCRpS8Y/juyPTwoeO9PblKs 9p6b8z4Y8ecpv3vDdOhIwerefXKx7vIM/wMC9nOH+KRE51akbLd6u845sLdNp/q7lFtNWsibC2F cnAA= X-Mailer: git-send-email 2.53.0.959.g497ff81fa9-goog Message-ID: <20260320145934.2349881-26-ardb+git@google.com> Subject: [PATCH v3 11/13] arm64: mm: Don't abuse memblock NOMAP to check for overlaps From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: linux-arm-kernel@lists.infradead.org, will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, Ard Biesheuvel , Ryan Roberts , Anshuman Khandual , Liz Prucka , Seth Jenkins , Kees Cook , linux-hardening@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel Now that the DRAM mapping routines respect existing table mappings and contiguous block and page mappings, it is no longer needed to fiddle with the memblock tables to set and clear the NOMAP attribute. Instead, map the kernel text and rodata alias first, so that they will not be added later when mapping the memblocks. Signed-off-by: Ard Biesheuvel --- arch/arm64/mm/mmu.c | 23 ++++++-------------- 1 file changed, 7 insertions(+), 16 deletions(-) diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index b52254790fda..34ad45a2d95f 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -1128,12 +1128,14 @@ static void __init map_mem(void) flags |=3D NO_BLOCK_MAPPINGS | NO_CONT_MAPPINGS; =20 /* - * Take care not to create a writable alias for the - * read-only text and rodata sections of the kernel image. - * So temporarily mark them as NOMAP to skip mappings in - * the following for-loop + * Map the linear alias of the [_text, __init_begin) interval + * as non-executable now, and remove the write permission in + * mark_linear_text_alias_ro() above (which will be called after + * alternative patching has completed). This makes the contents + * of the region accessible to subsystems such as hibernate, + * but protects it from inadvertent modification or execution. */ - memblock_mark_nomap(kernel_start, kernel_end - kernel_start); + __map_memblock(kernel_start, kernel_end, PAGE_KERNEL, flags); =20 /* map all the memory banks */ for_each_mem_range(i, &start, &end) { @@ -1145,17 +1147,6 @@ static void __init map_mem(void) __map_memblock(start, end, pgprot_tagged(PAGE_KERNEL), flags); } - - /* - * Map the linear alias of the [_text, __init_begin) interval - * as non-executable now, and remove the write permission in - * mark_linear_text_alias_ro() below (which will be called after - * alternative patching has completed). This makes the contents - * of the region accessible to subsystems such as hibernate, - * but protects it from inadvertent modification or execution. - */ - __map_memblock(kernel_start, kernel_end, PAGE_KERNEL, 0); - memblock_clear_nomap(kernel_start, kernel_end - kernel_start); } =20 void mark_rodata_ro(void) --=20 2.53.0.959.g497ff81fa9-goog From nobody Fri Apr 3 08:09:59 2026 Received: from mail-wm1-f73.google.com (mail-wm1-f73.google.com [209.85.128.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1770D3C9EEB for ; Fri, 20 Mar 2026 15:00:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774018818; cv=none; b=LRBPy8smNzrlOgp2/grWcH7OXnAMobjTNBqs/29tEAePP+5FFCUTCnxQk24gVhXPEvK+FFODFJIA7RmS0XLPN791sniq6/eeQpMixgtB8z1gj7awWf4TdCU8UNlpPRMdt26sCokWLiTfwb5TRyG3bLVg6mivBOBFAY06y7F8Q9M= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774018818; c=relaxed/simple; bh=y2fq/B0Ste8ZL4LB615vCRbVd9h9AjaQG4TJxVVO0rQ=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=GxM3/GqLbnpL704cIbjWAtkADWFeOa0MG8tWzfqN1juhXUjPipjkrU3j/jqRlTo6Kwza4l6VlKzf4zg3/iOC98K62G+VrfccwAOqf1bVTLXQ0LQ5SqSqSS1oWG/0xFXn2hn4yCfwT5m2vSL4snqdnjSpAuYY9PoBnoNn/a0fzUM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=tZd3AXuP; arc=none smtp.client-ip=209.85.128.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="tZd3AXuP" Received: by mail-wm1-f73.google.com with SMTP id 5b1f17b1804b1-486fe36cf73so7103295e9.1 for ; Fri, 20 Mar 2026 08:00:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1774018813; x=1774623613; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=AllHBw18cmDC2YVJSjzRxyh9hUFulIuf1ZVRU3fSw0M=; b=tZd3AXuPHFCVKiP5LrHfPv0BDfUfKFh3CzalBGra2ykIjkYfasx4aUgIzv0O9SvMlw 8GManECpLghmXzTNENUo8N/L/oRjucVmMJCL7HGpmL63elJh70s7R5FRhPeElS3ITPEC 1V0fw2a3WnDPsrtPamZbE+Fdf/aBNNhkZG1QQ+GGRJ8tcvrX9tAZAUWc3kScLTdG5xVi bjv3pEGUSELAV9tmNVsGYTBV/KIdZdZW7tezTezIjmeLTLd0lzD8AX08MH2XdsIImH7f Xin5g0q9m94KbHOeGLA3ghdETGLaym5Da4SRvf3m2tFXePiGdLTBQ4kpwtU98BOINkh7 c7Uw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1774018813; x=1774623613; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=AllHBw18cmDC2YVJSjzRxyh9hUFulIuf1ZVRU3fSw0M=; b=oP0qJL7TJ9CJ/s7qZ0wma8RZ9RusPYJtHlMCJlbymhuWMH7Xy6z0WOWFStCEso8Ks6 uYgUAzwluZFDGEhLpTpEq8aj/m0/fxSVkI69iIAaoygTr2pxxQ7Hiej3YFMSNn4Zjedn WFEYCyA2iTaIyQzQJXIIx38gIy9+1xGVaSRo/yWe+mutVvVzioBNyzMrlD8F547o1Zza zme4nJY6OSUVYW2+saBYHO4uEL2TDYMircwFqK1z1TZcLy0DCVIFqylyapiHs+P241So 8p6FQMe7jOPByyp3iWbCxXKT6q4xG4AKhVmO6RdqHQ54vm50jTx/owDwdTVeiM/lD6x9 EKAw== X-Gm-Message-State: AOJu0YxVRw0+9nkm+8R2K/C6uKChiNk4z8jchsqIx1Aj4rUwurzjiVZJ 3btDvhMaCRx9FIibv/3JhbHxxpjfUqBXEy0qsLeFpqR0RzC4rYg3KRV4h4uPK2EkcpDQOgyxEyK 16a56U/RZ8VzTBt+awbckFPOikNWdyMHjT08o3igyP834WY0s9E5kpB0EbMqzgSWwNh/9U4cz88 HOuKZ4whmE3M0vdbHBikuXI3SFpax7YDtV6Q== X-Received: from wmaj7.prod.google.com ([2002:a05:600c:6c07:b0:480:4a03:7b6f]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:8215:b0:485:4388:348b with SMTP id 5b1f17b1804b1-486feb5a3camr48030045e9.0.1774018812807; Fri, 20 Mar 2026 08:00:12 -0700 (PDT) Date: Fri, 20 Mar 2026 15:59:47 +0100 In-Reply-To: <20260320145934.2349881-15-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260320145934.2349881-15-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=3040; i=ardb@kernel.org; h=from:subject; bh=mMdF4nB0Sfg+/JaqMNkV+NJRkgmrh0vw+xoaE44MJaI=; b=owGbwMvMwCVmkMcZplerG8N4Wi2JIXNvwvMJ98weNU5ptLIJkv/+Oj2Ay5hDUO1NXFvd89uqx 9r0Hgl1lLIwiHExyIopsgjM/vtu5+mJUrXOs2Rh5rAygQxh4OIUgImUzWP4H7t84yyZKcV7+2Zt ypvonf/98fKYjb9+hDk/FNIVvnnampvhn0WH35Jm2fqL8y/5SM/0LvQuyv63O988PprzjlyX3p0 nfAA= X-Mailer: git-send-email 2.53.0.959.g497ff81fa9-goog Message-ID: <20260320145934.2349881-27-ardb+git@google.com> Subject: [PATCH v3 12/13] arm64: mm: Map the kernel data/bss read-only in the linear map From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: linux-arm-kernel@lists.infradead.org, will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, Ard Biesheuvel , Ryan Roberts , Anshuman Khandual , Liz Prucka , Seth Jenkins , Kees Cook , linux-hardening@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel On systems where the bootloader adheres to the original arm64 boot protocol, the placement of the kernel in the physical address space is highly predictable, and this makes the placement of its linear alias in the kernel virtual address space equally predictable, given the lack of randomization of the linear map. The linear aliases of the kernel text and rodata regions are already mapped read-only, but the kernel data and bss are mapped read-write in this region. This is not needed, so map them read-only as well. Note that the statically allocated kernel page tables do need to be modifiable via the linear map, so leave these mapped read-write. Signed-off-by: Ard Biesheuvel --- arch/arm64/include/asm/sections.h | 1 + arch/arm64/mm/mmu.c | 14 ++++++++++++-- 2 files changed, 13 insertions(+), 2 deletions(-) diff --git a/arch/arm64/include/asm/sections.h b/arch/arm64/include/asm/sec= tions.h index 51b0d594239e..f7fe2bcbfd03 100644 --- a/arch/arm64/include/asm/sections.h +++ b/arch/arm64/include/asm/sections.h @@ -23,6 +23,7 @@ extern char __irqentry_text_start[], __irqentry_text_end[= ]; extern char __mmuoff_data_start[], __mmuoff_data_end[]; extern char __entry_tramp_text_start[], __entry_tramp_text_end[]; extern char __relocate_new_kernel_start[], __relocate_new_kernel_end[]; +extern char __pgdir_start[]; =20 static inline size_t entry_tramp_text_size(void) { diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index 34ad45a2d95f..5332f4ec743e 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -1102,7 +1102,9 @@ static void __init map_mem(void) { static const u64 direct_map_end =3D _PAGE_END(VA_BITS_MIN); phys_addr_t kernel_start =3D __pa_symbol(_text); - phys_addr_t kernel_end =3D __pa_symbol(__init_begin); + phys_addr_t init_begin =3D __pa_symbol(__init_begin); + phys_addr_t init_end =3D __pa_symbol(__init_end); + phys_addr_t kernel_end =3D __pa_symbol(__pgdir_start); phys_addr_t start, end; int flags =3D NO_EXEC_MAPPINGS; u64 i; @@ -1135,7 +1137,10 @@ static void __init map_mem(void) * of the region accessible to subsystems such as hibernate, * but protects it from inadvertent modification or execution. */ - __map_memblock(kernel_start, kernel_end, PAGE_KERNEL, flags); + __map_memblock(kernel_start, init_begin, PAGE_KERNEL, flags); + + /* Map the kernel data/bss so it can be remapped later */ + __map_memblock(init_end, kernel_end, PAGE_KERNEL, flags); =20 /* map all the memory banks */ for_each_mem_range(i, &start, &end) { @@ -1147,6 +1152,11 @@ static void __init map_mem(void) __map_memblock(start, end, pgprot_tagged(PAGE_KERNEL), flags); } + + /* Map the kernel data/bss read-only in the linear map */ + __map_memblock(init_end, kernel_end, PAGE_KERNEL_RO, flags); + flush_tlb_kernel_range((unsigned long)lm_alias(__init_end), + (unsigned long)lm_alias(__pgdir_start)); } =20 void mark_rodata_ro(void) --=20 2.53.0.959.g497ff81fa9-goog From nobody Fri Apr 3 08:09:59 2026 Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 306AC3C553D for ; Fri, 20 Mar 2026 15:00:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774018822; cv=none; b=R7lFH7WxpGlMyVPOp5BEjIio7m4184bO5pvKqDHuchvG9qLfV2Zuw4VzGBGRDybBEA0gilaDeZOA9PJu/BNtFmjIiqlft1Tqc/KbylXVlWb40eeiKGPlvBP9RUokjVSNuPzQl8NjpbN+NwoiI7FZigzhSCBRj2iJeLNf4ggkC9M= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774018822; c=relaxed/simple; bh=AqLFJZeKWYgkpI7s4CDm2jdoXLVsXgEja2BsVoD4rdw=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=lkWRdS4jY0I5o3Id1vTwepHUPbqOETpWO0+s/4hXfJbPfFOO8e6lTsM1FCzHPQBbC50FwNYg2MLqNFdhvgCR9BChRQ5sOFugdc4we4sqXmSDB0dEwlbSzzb8S8tBY7X+vmqSgZeTEj/lhZCC43i1/SjryFDZgZhCCn669pluP50= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=aSYB+0op; arc=none smtp.client-ip=209.85.128.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="aSYB+0op" Received: by mail-wm1-f74.google.com with SMTP id 5b1f17b1804b1-4836abfc742so5538595e9.0 for ; Fri, 20 Mar 2026 08:00:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1774018814; x=1774623614; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=UrZacB4uB0kaaolZjrInY8IvgEW43dbywNDGay4OvCg=; b=aSYB+0opzuXC6Edw52NR5+eNhuIYkxHzRusoSOZh/PDwGpqptZRxko1zbS6q98xQSX CrjOF8k6vCdoJ97dq6lqTa+vpNRUI/CQua6PifRA4qS2mslJO4umt5x//LBApUB7tc+M aj3lI2hvhk2qwgOzmxvTk9wVvxu+UduwnqU1SmF0Vv2eeXKxcLGgXG3DSvE0dXMLeHzR QPz6V7epDc9HfJX1ez9CqfRO+iWjlgLwL8zxH6N8oD13MyIauCCIPJpT+FFXYNOnon6y xdTmObgM03OgZvNY+LF5Iwtw3LAcW7Zs/CuvcmnljVpMNXxMjDURq5OSJmGTAfFlpEwR sCyQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1774018814; x=1774623614; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=UrZacB4uB0kaaolZjrInY8IvgEW43dbywNDGay4OvCg=; b=G4r7dkk5GO8PREENWz50OdganlP5oGh3HkNxRITvAMe8vwb88U+SD84oS9NR/8GsG7 JvkHWybm+SayHuTK8LkAzCP56nVx/wCjR8omm3ZKEfNdIisJVp1kvYcmy/HDM1T+qU/Z 2yAeMdffeK7zRgFCLls/FZ/enhGfm1gLEbAfqxeXRzgarBdHwAUs7Jlp52codCPbGrmo d+fmgQW1QPR/xpkyBYwIZPe/zJDHcALrIfSjMbuDi++qd7yeXCBP3zyk9zdla89wFUsx XLXUzKzZNJ0JrfowaGxCBt45y8KXnqdJhUjexAzN8vBYZqXlzfJKzp1bS93zT4cNkR/X hKxg== X-Gm-Message-State: AOJu0Yw00RRN8nyah+j2aAog1GjQPelEmh3V1hpySnl+/Dn09mvIoAJX +Dw5W6AqnoO46AUhzXiC1rGjQ1rWTevrxIbGVmS8mG3Dmb1SLxHWZ6ZZ5PF/uzXQT2jujn0SIHe URDFfakBrI4KcVZ1AifXdKQgd2hO+DEdzqKIgXTOsnD6/KOAoBaR9db52u2lE9ORB3cIefz09mp 49mPyTOY5L5TkCgdrP2syOHfQ1ZqHHGUbuJw== X-Received: from wmlu26.prod.google.com ([2002:a05:600c:211a:b0:486:f89b:7f29]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:4e8f:b0:483:badb:618b with SMTP id 5b1f17b1804b1-486fee1e062mr49167685e9.24.1774018814045; Fri, 20 Mar 2026 08:00:14 -0700 (PDT) Date: Fri, 20 Mar 2026 15:59:48 +0100 In-Reply-To: <20260320145934.2349881-15-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260320145934.2349881-15-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=3378; i=ardb@kernel.org; h=from:subject; bh=iQp7irbDz/c9Axp3wqFfYmhO5jDbkoyXPLaKPJQN80w=; b=owGbwMvMwCVmkMcZplerG8N4Wi2JIXNvwouWK0ItWr1ZT/UYDA/sXPgq8H3H7KqWf9pttyVTc 3ovvuXqKGVhEONikBVTZBGY/ffdztMTpWqdZ8nCzGFlAhnCwMUpABOJEmJkOPJtLVeORLTLrXWT 2fw+vNT/Z7hj09/UgmMr6jeWFU5QO8DI8EpJ/2Vg8LUJ/84589p8Ong8uCLRuXOT0Pyr095e7vg 3jxkA X-Mailer: git-send-email 2.53.0.959.g497ff81fa9-goog Message-ID: <20260320145934.2349881-28-ardb+git@google.com> Subject: [PATCH v3 13/13] arm64: mm: Unmap kernel data/bss entirely from the linear map From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: linux-arm-kernel@lists.infradead.org, will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, Ard Biesheuvel , Ryan Roberts , Anshuman Khandual , Liz Prucka , Seth Jenkins , Kees Cook , linux-hardening@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel The linear aliases of the kernel text and rodata are mapped read-only in the linear map as well. Given that the contents of these regions are mostly identical to the version in the loadable image, mapping them read-only and leaving their contents visible is a reasonable hardening measure. Data and bss, however, are now also mapped read-only but the contents of these regions are more likely to contain data that we'd rather not leak. So let's unmap these entirely in the linear map when the kernel is running normally. When going into hibernation or waking up from it, these regions need to be mapped, so map the region initially, and toggle the valid bit so map/unmap the region as needed. Signed-off-by: Ard Biesheuvel --- arch/arm64/mm/mmu.c | 44 +++++++++++++++++--- 1 file changed, 38 insertions(+), 6 deletions(-) diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index 5332f4ec743e..82a495563b60 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -24,6 +24,7 @@ #include #include #include +#include #include #include #include @@ -1020,6 +1021,31 @@ static void __init __map_memblock(phys_addr_t start,= phys_addr_t end, end - start, prot, early_pgtable_alloc, flags); } =20 +static void remap_linear_data_alias(bool unmap) +{ + set_memory_valid((unsigned long)lm_alias(__init_end), + (unsigned long)(__pgdir_start - __init_end) / PAGE_SIZE, + !unmap); +} + +static int arm64_hibernate_pm_notify(struct notifier_block *nb, + unsigned long mode, void *unused) +{ + switch (mode) { + default: + break; + case PM_POST_HIBERNATION: + case PM_POST_RESTORE: + remap_linear_data_alias(true); + break; + case PM_HIBERNATION_PREPARE: + case PM_RESTORE_PREPARE: + remap_linear_data_alias(false); + break; + } + return 0; +} + void __init mark_linear_text_alias_ro(void) { /* @@ -1028,6 +1054,16 @@ void __init mark_linear_text_alias_ro(void) update_mapping_prot(__pa_symbol(_text), (unsigned long)lm_alias(_text), (unsigned long)__init_begin - (unsigned long)_text, PAGE_KERNEL_RO); + + remap_linear_data_alias(true); + + if (IS_ENABLED(CONFIG_HIBERNATION)) { + static struct notifier_block nb =3D { + .notifier_call =3D arm64_hibernate_pm_notify + }; + + register_pm_notifier(&nb); + } } =20 #ifdef CONFIG_KFENCE @@ -1140,7 +1176,8 @@ static void __init map_mem(void) __map_memblock(kernel_start, init_begin, PAGE_KERNEL, flags); =20 /* Map the kernel data/bss so it can be remapped later */ - __map_memblock(init_end, kernel_end, PAGE_KERNEL, flags); + __map_memblock(init_end, kernel_end, PAGE_KERNEL, + flags | NO_BLOCK_MAPPINGS); =20 /* map all the memory banks */ for_each_mem_range(i, &start, &end) { @@ -1152,11 +1189,6 @@ static void __init map_mem(void) __map_memblock(start, end, pgprot_tagged(PAGE_KERNEL), flags); } - - /* Map the kernel data/bss read-only in the linear map */ - __map_memblock(init_end, kernel_end, PAGE_KERNEL_RO, flags); - flush_tlb_kernel_range((unsigned long)lm_alias(__init_end), - (unsigned long)lm_alias(__pgdir_start)); } =20 void mark_rodata_ro(void) --=20 2.53.0.959.g497ff81fa9-goog