From nobody Wed Jun 17 06:02:47 2026 Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D8408340A46 for ; Mon, 27 Apr 2026 15:35:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777304138; cv=none; b=q2uYRepGgyhsNTlzsseSp+DXc0K5NXKg1YEnpZv4QZYL3dm9MaxEq8aaBvPa+nQIxmpRttQp4p5jTn7QkbTBHgnVlQ3Q/Qcsfsj8eNCn39AmBSEK1Nf0+aK+s6YUUsgTT+QlF3a5/jTQ9kkBAW8BlfWSDzFW/1rdiZaaz4cGtjQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777304138; c=relaxed/simple; bh=Sl0OBQBWIK0zh6lHXps5IqXvZm9y+3UkjWufXvijumM=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Yt8DUYpmmGPPVNnEf8Uc3uQS2e/2RCgIy405Y2sfWDJwd/6xIeRfFaRQu/BJ6Bnwj11mXNLf289DS5EjN+W3PPDXwSAQPQm1CLDZibPMO5gEmZMtJMAIbVXVAoNR7YRbEx8IvVY071k5/DmBgd2T8Qx+6x18VUB/08LHT66RZu8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=NH+E9lRw; arc=none smtp.client-ip=209.85.128.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="NH+E9lRw" Received: by mail-wm1-f74.google.com with SMTP id 5b1f17b1804b1-488f973ddfeso76714545e9.3 for ; Mon, 27 Apr 2026 08:35:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1777304135; x=1777908935; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=8F3kASMqf5BHYl2gzcA0DgWEOwzoo2uBq3thfdXqJMo=; b=NH+E9lRwKjOCEzgfYbeuSSQsUlncTtFpQ7d0aV59HERZvL67lDcVTFkLfgR2PEa6p7 irdw3W9Xi96W9FufSW0+7SYY/8H5UVcTTYROrkr0fToNQKOZUkaezDPHWgeYHGgpANFz z9V1apwiDDMnTFRtCRrVnp4oR+9JqYiw5WRLNiKs96ZLI2qJIcgwXWnJapyKLw4uzgZs tvynBFbqWKfJDscJgGEJS+E9RwVeGAu1b5pU3blf6hwj6OtNH3zfvU8KmPEJ5CjPG7qq qKzYyLYXY+sC60Qsm/BNDy+qPp52tHWbRkSEvS0Wp4EDknE1rn4w0BPt4Uak4M0FfhEA K6Ww== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1777304135; x=1777908935; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=8F3kASMqf5BHYl2gzcA0DgWEOwzoo2uBq3thfdXqJMo=; b=jZErYjjC44mykvH73Cvc8cWKf1vC2SR+8i84XBkXf4mKLr8KCEjAMS1vw5oHv6KNkl KcO8llUnlD2X03DfaxpUkaioRUVV/iVI+TNlWiufywG3ieVEFyvlkCVJK2kI5RN19Xv/ MqUOaccdsefO2uMO39giI8hOerPwZEJ08B04r5rhz9VOq82LtxRrCql29XA64cTFYwx/ bEzNw5+Be5dp4OXZrh0KyZEj5Qem+mJCnilLNz/Bv5cRBaQrJl02Qs/6SXg+lfv8Uui9 O6UFzjGCjkWnHhn4a4CTWSA+VOfRlSxuXJpe9AaSDj8jqXBp9K2GEvWTty07pI59Sfgu /fmQ== X-Gm-Message-State: AOJu0YzhBEJKHJREEnJizlrymUL4hz0Iz9Ttrcdv8B3kbi4VDJvDrIaw TJqm0+LnwTHR8vtDqi8FaMSbXohVH+UdCc+YM2r0bVL5nZOCMuLbKUG0dce/QsTSYESozbzA/w= = X-Received: from wmbh12.prod.google.com ([2002:a05:600c:a10c:b0:489:1946:b592]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:1e28:b0:483:7903:c3b1 with SMTP id 5b1f17b1804b1-488fb77fbf3mr626097515e9.20.1777304135240; Mon, 27 Apr 2026 08:35:35 -0700 (PDT) Date: Mon, 27 Apr 2026 17:34:18 +0200 In-Reply-To: <20260427153416.2103979-17-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260427153416.2103979-17-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=837; i=ardb@kernel.org; h=from:subject; bh=FjvXC4Bvv/morAQPM0ElYAlwegkzMza4j/YwiIL/yF4=; b=owGbwMvMwCVmkMcZplerG8N4Wi2JIfN94++NX3ll/ZiXVZap93TO00rsSY3MY/ndJRGotvX3m Y9v23Q6SlkYxLgYZMUUWQRm/3238/REqVrnWbIwc1iZQIYwcHEKwEQu/mX4xXRjUvDxfwUKzSeC Qp+sfOS5rfhfFteH21tOH9i1KfXn+g6G/9lLOeQeqzgJu68KXXDhuUvujolfrnt0PsxjVi7vjpH 14gcA X-Mailer: git-send-email 2.54.0.rc2.544.gc7ae2d5bb8-goog Message-ID: <20260427153416.2103979-18-ardb+git@google.com> Subject: [PATCH v4 01/15] arm64: mm: Map the linear alias of text/rodata as tagged From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, Ard Biesheuvel , Ryan Roberts , Anshuman Khandual , Liz Prucka , Seth Jenkins , Kees Cook , Mike Rapoport , David Hildenbrand , Andrew Morton , linux-mm@kvack.org, linux-hardening@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel Before moving the empty_zero_page into the __ro_after_init section, make sure it has the memory-tagged type. This is needed to ensure that cpu_enable_mte() will be able to initialize the tags correctly. Signed-off-by: Ard Biesheuvel --- arch/arm64/mm/mmu.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index dd85e093ffdb..f084993024ab 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -1049,7 +1049,7 @@ void __init mark_linear_text_alias_ro(void) */ update_mapping_prot(__pa_symbol(_text), (unsigned long)lm_alias(_text), (unsigned long)__init_begin - (unsigned long)_text, - PAGE_KERNEL_RO); + pgprot_tagged(PAGE_KERNEL_RO)); } =20 #ifdef CONFIG_KFENCE --=20 2.54.0.rc2.544.gc7ae2d5bb8-goog From nobody Wed Jun 17 06:02:47 2026 Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1D34D346E47 for ; Mon, 27 Apr 2026 15:35:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777304139; cv=none; b=SizyuCd1aP0VuPbe8rJ9egyWq4vY3+YfUN3iCdHfchnJs6WCUOgkMSxaQd1BA1ZNI2yLQWRM3ZGJFQaeIX6vuK1eQzkuuPsb2kIWcLYLrM5MR9BTqUocAbBNaeNwzVaDpo+QGj1GsIvAv/yeVhfbyNIFUzXW8SWEJ3c+Qp8Lavg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777304139; c=relaxed/simple; bh=DEtCdbcQASaDQouORSj9oUIdAtG/cwaStohTOB0WxM4=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=URiOu47jXsTMv+Z5xa20x5jtaEowsmVdM7u9WMCE8wTftP4U92KlDpaOgX2GePNTYmI8IhFlgc4JNrSqB2ZVTXt7iwIc7wfgKiuiftg/649DztJO4vu1YbU8QGhE7cc2K6hHoawiF/ykaGJGqX3yjvQUugkC6clw7VeM2DIIG3k= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=ujMJvyru; arc=none smtp.client-ip=209.85.128.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="ujMJvyru" Received: by mail-wm1-f74.google.com with SMTP id 5b1f17b1804b1-488cc31ea57so80247765e9.3 for ; Mon, 27 Apr 2026 08:35:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1777304136; x=1777908936; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=etH/ucf0ROVAevEBkHReKesjYkPK8m2/xS+mJi9/Az0=; b=ujMJvyruB43A0JqS4KmdDUuXrwKbIMJRo51BwoUBY2BkO/60c5EIBcY66uPKYZVcab hC2Rxw2qOwTkqIEwJdZsdtgt6Jt76NjvSjfTuvPu9GQp7BjAXP/SyLDUYpspByTWbSGL Zd+G2LqXUI6mmpyd8SqLe1oS7pJzjp93g3hpXMkdoMRv88PqI9IPuHV5UvdBr81FIoBf 5V3fDn9RmvRl5Uv3VlYKzphat9SWDt0gfxOcFhNjc9kX9ePxqwqlim4f5JGUtXlcFRc+ Dr5C+WmHMcSELnrPW+OPdDyVPyLARYdL6l2c5S94YIB83SCZ5gWQsCRRDIXTg6AOMq31 20eQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1777304136; x=1777908936; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=etH/ucf0ROVAevEBkHReKesjYkPK8m2/xS+mJi9/Az0=; b=UbXPZJvCi/xbSnmpQRWAtqXwH2ZbWegYl96Vli5im2V5cVaaz7qi6k8LpGm7z885cX QU7vxGmgGiI1t00AsXTJQWYlg6lfZ3f7hL/hzfloulWTZjV6/UHI/VN8TdYmoqKvWCAs Kze7PWKvDBgdhW/JQZDJvROM1+VWGxAr9x7xIx5Nhfeab8LcJ0ohRLJOG6Z/7PAGcIcs fbGk0JTbN4iN8y8Qq5WIsIjkocG6kgxggv/KnM2wEtL81ctxwQ4owSrwTba4jhIX7SrA Wbs3yu6nKDfy86n1jf0xSW6EoRVTcaquWiH90kIEm+Uc/KfdkXZ87vWcYXntx4VwS3Y8 y4rw== X-Gm-Message-State: AOJu0YxZEuiE0wbB5SLJ0dCNTywnYKr+mwDQ8QcZS+9WtXbR0WuxmEzQ HtB8FixyCuB6bM2onVezXE8XgLkfewjj3sZqoLDSGiFK/+rFRR6K7FyOw5ZAIamWifblF6/ctw= = X-Received: from wmcn9.prod.google.com ([2002:a05:600c:c0c9:b0:485:fb9c:ffa5]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:3e0d:b0:489:6c28:dbc6 with SMTP id 5b1f17b1804b1-4896c28dd4emr454740575e9.31.1777304136468; Mon, 27 Apr 2026 08:35:36 -0700 (PDT) Date: Mon, 27 Apr 2026 17:34:19 +0200 In-Reply-To: <20260427153416.2103979-17-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260427153416.2103979-17-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=1123; i=ardb@kernel.org; h=from:subject; bh=Xui4zHPMyqNAfMojX3d72dmyPg9HBiaeXXlaXljmfJY=; b=owGbwMvMwCVmkMcZplerG8N4Wi2JIfN945+SRZJXH5vdEE+o7Yvo32RxfuaV3S/5HSeeLwufe mV64bHIjlIWBjEuBlkxRRaB2X/f7Tw9UarWeZYszBxWJpAhDFycAjARpwUM/6tzU1eeXBAlq+Z5 +VDlfPmd8pIhfj95ZQ8bhFlenPP7LBvD/7yyGqPIczsnJ5sZWSR+cbBZf/Xwp98hlSGvVEzOB0r eZAIA X-Mailer: git-send-email 2.54.0.rc2.544.gc7ae2d5bb8-goog Message-ID: <20260427153416.2103979-19-ardb+git@google.com> Subject: [PATCH v4 02/15] mm: Make empty_zero_page __ro_after_init From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, Ard Biesheuvel , Ryan Roberts , Anshuman Khandual , Liz Prucka , Seth Jenkins , Kees Cook , Mike Rapoport , David Hildenbrand , Andrew Morton , linux-mm@kvack.org, linux-hardening@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel The empty zero page is used to back any kernel or user space mapping that is supposed to remain cleared, and so the page itself is never supposed to be modified. So make it __ro_after_init rather than __page_aligned_bss: on most architectures, this ensures that both the kernel's mapping of it and any aliases that are accessible via the kernel direct (linear) map are mapped read-only, and cannot be used (inadvertently or maliciously) to corrupt the contents of the zero page. Signed-off-by: Ard Biesheuvel Acked-by: David Hildenbrand (Arm) Reviewed-by: Feng Tang Reviewed-by: Jann Horn Reviewed-by: Kevin Brodsky Reviewed-by: Mike Rapoport (Microsoft) --- mm/mm_init.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/mm_init.c b/mm/mm_init.c index f9f8e1af921c..6ca01ed2a5a4 100644 --- a/mm/mm_init.c +++ b/mm/mm_init.c @@ -57,7 +57,7 @@ unsigned long zero_page_pfn __ro_after_init; EXPORT_SYMBOL(zero_page_pfn); =20 #ifndef __HAVE_COLOR_ZERO_PAGE -uint8_t empty_zero_page[PAGE_SIZE] __page_aligned_bss; +uint8_t empty_zero_page[PAGE_SIZE] __ro_after_init __aligned(PAGE_SIZE); EXPORT_SYMBOL(empty_zero_page); =20 struct page *__zero_page __ro_after_init; --=20 2.54.0.rc2.544.gc7ae2d5bb8-goog From nobody Wed Jun 17 06:02:47 2026 Received: from mail-wm1-f73.google.com (mail-wm1-f73.google.com [209.85.128.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4A3E7349B16 for ; Mon, 27 Apr 2026 15:35:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777304142; cv=none; b=QCTiFZm1tPX2VT7Bc0kemv3twrlW8xkJJnVkAWdSuEiqRkiNeQN/1p635ZfsBA4uWQVj8uPk3mSfmqXeZ8NUU6Q4IL2IqIWCEu+VxHaUe+TleVPRP1kMZ0UfisMkKKbIM6KWG9uUxgnCEWUJIoH7A2curzxqU+LwnorUFJtfZnc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777304142; c=relaxed/simple; bh=dtc12+NMVIXo5ice1yzML7hq7wO8bZW1G1K/irYhX/g=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=D4acGW2MPQhF8kHJkoJ41HGLO4xFtR9U9X+vch/0ZUbpyj+i7PfhGU57EXU2X7cB+AjnMn21ltz8IXUzE3YrjkCktPL3U7BHoWiPtvGHK+fozHAwF98TVO/GEzsA6Kf4bnJKLOPe3fTVjhoj2qicwgqNfjh/SOzLQnNtc8k+hTo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=uhOwhZXD; arc=none smtp.client-ip=209.85.128.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="uhOwhZXD" Received: by mail-wm1-f73.google.com with SMTP id 5b1f17b1804b1-486fa07f2bbso72562225e9.2 for ; Mon, 27 Apr 2026 08:35:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1777304137; x=1777908937; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=Wc51wPzk67h/KJWuIdvgej4c5wVXm+tSlcwLrREqPXw=; b=uhOwhZXDvEW9MAb+baj06krBE2oliep+BSSwvPcagHtmnftY0UvBDz8h5JZG3ZQttH YlibFwHVummJt6jVvAucwWKEboUeWvz96aXxInErc6+RLWVeWXIP2XkI1LZB8fo3gMV9 hmA9oPfsmwpXtoLxqtupaavJC+ahMwI57S1/6L37PVg9vP7Fspxon6UsLt1edH4HzafY l9ZIWWeA3SejmCt6bJuR3QLas8cTlCfy2CGW+gy7CLiKFtgyK3BrkFRVh7nUNMPP67r0 dUuYxbsbrmMAIOzWrpquu/jEmC7USnCrpjykRxAl3Yq09EiQkqLgiMz/FXncLgH26H8B RfFw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1777304137; x=1777908937; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Wc51wPzk67h/KJWuIdvgej4c5wVXm+tSlcwLrREqPXw=; b=EISUJKOPnNAXovezUwdC9Omc0FSb3OixTxYYDInjzyqpi9gG6BB2ySfiMHQ3M0VFXX W06j+OnfFsT78k62KBaMDgZ3aLRMMi/CEnp5KJp67i35dvq3IyKPZtsUW7rEGRoSG8Sr LRRJb7OfmqL7UxwR6PdS5c6Xi8K8kaKP48inmTopnUO4jXa4lTdQ8A2tIJz1aTNDGPu5 2jWaxjDRTPUrXcLixcbm+m90x6xajEQa82EqmH9zYMnfpAmMkTO4YNYb8oU9EgQ4jl1F 1vMM36V4Q2Us02BjrVky1jgAcp7MR1HpUUiMND/T9jGC2FdTDDXfDHyfLezHt99p0BSc Nmjw== X-Gm-Message-State: AOJu0YyI8tHfm12ZgyLjml1YwLXIUT1njUbtXzPCOlQcU9XdK0UYju2D wFsPbkT3K8aH0UNe+9k0tuQzB+1XNvzd3Ijf8hD2l3R+CyJOndIUl0ZtlwNQ3EbwoRIO5IBXjg= = X-Received: from wrrf15.prod.google.com ([2002:adf:f98f:0:b0:445:9b34:32eb]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:3b14:b0:48a:52d9:54ab with SMTP id 5b1f17b1804b1-48a52d95604mr460176645e9.30.1777304137397; Mon, 27 Apr 2026 08:35:37 -0700 (PDT) Date: Mon, 27 Apr 2026 17:34:20 +0200 In-Reply-To: <20260427153416.2103979-17-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260427153416.2103979-17-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=1344; i=ardb@kernel.org; h=from:subject; bh=GHwWYq60lqNuozERz0S434bqDfMxYbnUraaeSVA3Tn0=; b=owGbwMvMwCVmkMcZplerG8N4Wi2JIfN947/7X8wOXuLePiOp7MONnvXiUl5NBamL377hk804c tjhpENeRykLgxgXg6yYIovA7L/vdp6eKFXrPEsWZg4rE8gQBi5OAZjI+npGhoWiYueTlCv3njc5 eKI4u3WiW9EuwdunmA+vma0g5vHDOoKRYavEAeGovTdWuwnO7aw0j1t2v8qU20TqZU3o42gvDsX t/AA= X-Mailer: git-send-email 2.54.0.rc2.544.gc7ae2d5bb8-goog Message-ID: <20260427153416.2103979-20-ardb+git@google.com> Subject: [PATCH v4 03/15] arm64: mm: Preserve existing table mappings when mapping DRAM From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, Ard Biesheuvel , Ryan Roberts , Anshuman Khandual , Liz Prucka , Seth Jenkins , Kees Cook , Mike Rapoport , David Hildenbrand , Andrew Morton , linux-mm@kvack.org, linux-hardening@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel Instead of blindly overwriting an existing table entry when mapping DRAM regions, take care not to replace a pre-existing table entry with a block entry. This permits the logic of mapping the kernel's linear alias to be simplified in a subsequent patch. Reviewed-by: Ryan Roberts Signed-off-by: Ard Biesheuvel --- arch/arm64/mm/mmu.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index f084993024ab..801a54ade76f 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -256,7 +256,8 @@ static int init_pmd(pmd_t *pmdp, unsigned long addr, un= signed long end, =20 /* try section mapping first */ if (((addr | next | phys) & ~PMD_MASK) =3D=3D 0 && - (flags & NO_BLOCK_MAPPINGS) =3D=3D 0) { + (flags & NO_BLOCK_MAPPINGS) =3D=3D 0 && + !pmd_table(old_pmd)) { pmd_set_huge(pmdp, phys, prot); =20 /* @@ -379,7 +380,8 @@ static int alloc_init_pud(p4d_t *p4dp, unsigned long ad= dr, unsigned long end, */ if (pud_sect_supported() && ((addr | next | phys) & ~PUD_MASK) =3D=3D 0 && - (flags & NO_BLOCK_MAPPINGS) =3D=3D 0) { + (flags & NO_BLOCK_MAPPINGS) =3D=3D 0 && + !pud_table(old_pud)) { pud_set_huge(pudp, phys, prot); =20 /* --=20 2.54.0.rc2.544.gc7ae2d5bb8-goog From nobody Wed Jun 17 06:02:47 2026 Received: from mail-wr1-f74.google.com (mail-wr1-f74.google.com [209.85.221.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6850534750F for ; Mon, 27 Apr 2026 15:35:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777304142; cv=none; b=NmKBuj1hz5edFa8ZvZ46FOAL2Nm1pcxqs1DnZjSgViSCESi7lbodadhV4Lhez8AGisI3BfyLwCFzM+kpN/DF2S1OwNbmVwV6S0dNCW3/8+P9SwNVREUfrhm2E+f00bNGwXEYJSHTWnra5bst9PlGox2lb9HSmu6paOmAzWrJN6I= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777304142; c=relaxed/simple; bh=gSm6M1lFtROo4rxGtvr0diruKjpMo8ltJrOdz1rGszA=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=n+1hs2XbZTAaNLto3o7rBj4JMhuiTioO1caH/PiY/zfc5iIYnY+v+6hnLSMhTjEn7CBRh0nbH+Se8o3mKAv+d1pUwldwhHGn6tLLTtM+Z6pA/0D6JNDFs+J+EhJCZlUUfiGORCV2JI6r6arQk/BEIHNffJ19yYSJqSLNiy2L/3g= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=HGBw9skk; arc=none smtp.client-ip=209.85.221.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="HGBw9skk" Received: by mail-wr1-f74.google.com with SMTP id ffacd0b85a97d-43d7a5b9678so8097075f8f.2 for ; Mon, 27 Apr 2026 08:35:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1777304138; x=1777908938; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=prYizMoqeWKaDZjQ0nuvo6tbUP8AzcnCOIOw0eqIeeQ=; b=HGBw9skkCjDnRfPDQcWiy+PANf/FeNfO4QzLLTYFz1aITbmI0erstYcNUeWqk8JtIF 0sxwbpXYYvyHKuW9HWKTbr4uonvyYlNCV6eYlOVkOZgW5/zqpgchLveXF8AAPI1QO7yC sAxaaton8WCGdN+AUzMpBYWQ6vcZWWqMIoBooIIR1aMciwgTzsJiQiaGQtRmLJ3d3D76 2qym/6pFywzAYiHPogvdoseaaZOcxij48o8EoQE76oPYuH0SQ5IQzPRqzjJYPuJfmSi9 84UyoEqKI8ZFTmdQIlUVlxSMs8KXOJmux5AeDpwh91HW82fWr4PyixwQrLNelVH1QZQV 96mA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1777304138; x=1777908938; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=prYizMoqeWKaDZjQ0nuvo6tbUP8AzcnCOIOw0eqIeeQ=; b=gH5KGl0lYzcJz/cjPIufMqnejzh0R/Rrkd/nNEKzwQzK6aN3sQGZTS3X1G8wuUniO/ qUoLMGU1yuJbW7wCRH2M4UB1tdjJVdcFFGJ7bWM3oy+9swXCch4UX3VU3I2mqtKQAwCU ROGtEgBxKlda4Px0zqI7bvSh2A+fX4e2p3mC+/IQ0lOh7e/nkX5fAlq2oIQmpsfsSn5/ qDGc2mbdQ9aa2Cb2RYq3kbMTDW8atMd60VJfprR+KdRwGbsc2tMvX8RwRzwu9bMbdI7o 3i8I9KcuNHoCP0VJw2uQ/F4GxTavZ2xb2TNhwbzaJD0q8VyjPt2e9fZ6QoLgmdC1LGGS vc3A== X-Gm-Message-State: AOJu0YzgQ7OFeRGkCZEeUsn/PVqZGjQRhXtBwwXw/d+ZA//D5/NtR8IV s1DirhsuDphQiQSa1QS5fHvYXySbIG/m9XpPFszirb5j56/XGN8xWmKruGxiHWCaYvHcESRNtQ= = X-Received: from wrpx17.prod.google.com ([2002:adf:f651:0:b0:441:2eb5:f2f3]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:621a:b0:48a:6268:18a9 with SMTP id 5b1f17b1804b1-48a626818cdmr232216125e9.13.1777304138328; Mon, 27 Apr 2026 08:35:38 -0700 (PDT) Date: Mon, 27 Apr 2026 17:34:21 +0200 In-Reply-To: <20260427153416.2103979-17-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260427153416.2103979-17-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=2701; i=ardb@kernel.org; h=from:subject; bh=aEcnYdyzs+Jl6x1LXyRc2wvdSGWl6fC+aTg1LsbHjwQ=; b=owGbwMvMwCVmkMcZplerG8N4Wi2JIfN94//XXIcrTPk7rYUCN2cm+KhM2sqjKyt3IviukIL3z nmlYokdpSwMYlwMsmKKLAKz/77beXqiVK3zLFmYOaxMIEMYuDgFYCITmhn+h+c0/z7TXLNLwT+5 2H668O/amKwlTWfv3NYJ/6xb++rrQYb/3i4e3oYrxG//mbFf4HQ3M8eGnkiV+PfzOz49Fdq1VO4 XGwA= X-Mailer: git-send-email 2.54.0.rc2.544.gc7ae2d5bb8-goog Message-ID: <20260427153416.2103979-21-ardb+git@google.com> Subject: [PATCH v4 04/15] arm64: mm: Preserve non-contiguous descriptors when mapping DRAM From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, Ard Biesheuvel , Ryan Roberts , Anshuman Khandual , Liz Prucka , Seth Jenkins , Kees Cook , Mike Rapoport , David Hildenbrand , Andrew Morton , linux-mm@kvack.org, linux-hardening@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel Instead of blindly overwriting existing live entries with the contiguous bit cleared when mapping DRAM regions, check whether the contiguous region in question starts with a descriptor that has the valid bit set and the contiguous bit cleared, and in that case, leave the contiguous bit unset on the entire region. This permits the logic of mapping the kernel's linear alias to be simplified in a subsequent patch. Note that not setting the contiguous bit on any of the descriptors in the contiguous region can only result in an invalid configuration if it was already invalid to begin with. Reviewed-by: Ryan Roberts Signed-off-by: Ard Biesheuvel --- arch/arm64/include/asm/pgtable.h | 4 ++++ arch/arm64/mm/mmu.c | 6 ++++-- 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgta= ble.h index 4dfa42b7d053..a1c5894332d9 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -181,6 +181,10 @@ static inline pteval_t __phys_to_pte_val(phys_addr_t p= hys) * Returns true if the pte is valid and has the contiguous bit set. */ #define pte_valid_cont(pte) (pte_valid(pte) && pte_cont(pte)) +/* + * Returns true if the pte is valid and has the contiguous bit cleared. + */ +#define pte_valid_noncont(pte) (pte_valid(pte) && !pte_cont(pte)) /* * Could the pte be present in the TLB? We must check mm_tlb_flush_pending * so that we don't erroneously return false for pages that have been diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index 801a54ade76f..005844e521bd 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -224,7 +224,8 @@ static int alloc_init_cont_pte(pmd_t *pmdp, unsigned lo= ng addr, =20 /* use a contiguous mapping if the range is suitably aligned */ if ((((addr | next | phys) & ~CONT_PTE_MASK) =3D=3D 0) && - (flags & NO_CONT_MAPPINGS) =3D=3D 0) + (flags & NO_CONT_MAPPINGS) =3D=3D 0 && + !pte_valid_noncont(__ptep_get(ptep))) __prot =3D __pgprot(pgprot_val(prot) | PTE_CONT); =20 init_pte(ptep, addr, next, phys, __prot); @@ -324,7 +325,8 @@ static int alloc_init_cont_pmd(pud_t *pudp, unsigned lo= ng addr, =20 /* use a contiguous mapping if the range is suitably aligned */ if ((((addr | next | phys) & ~CONT_PMD_MASK) =3D=3D 0) && - (flags & NO_CONT_MAPPINGS) =3D=3D 0) + (flags & NO_CONT_MAPPINGS) =3D=3D 0 && + !pte_valid_noncont(pmd_pte(READ_ONCE(*pmdp)))) __prot =3D __pgprot(pgprot_val(prot) | PTE_CONT); =20 ret =3D init_pmd(pmdp, addr, next, phys, __prot, pgtable_alloc, flags); --=20 2.54.0.rc2.544.gc7ae2d5bb8-goog From nobody Wed Jun 17 06:02:47 2026 Received: from mail-wm1-f73.google.com (mail-wm1-f73.google.com [209.85.128.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 25BE134D4F9 for ; Mon, 27 Apr 2026 15:35:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777304143; cv=none; b=KymKpWNG7oQ1Z4t5K93kbbHA8vOkct0/8ocYzr3lYIe73mk7L51LPP+TH39sF359kWQKosQ3xtSc1S5pSFj+qFExTVyRMeM5ojEH/hGKPeU71Ht0CecgiZZRWJB150DRRvlFoJ43Qpse5V6PUqf66QczN5znxPGzALF8aah4Nqk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777304143; c=relaxed/simple; bh=iOBV/ragVWNZA9cpwk/m5In9oe0bnruniT9MVmdSMu4=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=IBAjGh4/LS4v4qemQKaniCSF8Hs4FCwJwnvwSgAyePrdzRNyULFZiZGM/aNF7aU6hl7gZkMp6dwDSBy0WGbqzJriwRDdQQ01W8O6mTICIcwbAYXznAN2useleyZdC882ARaCNLCfPpxLwndL1lcDvc7ASbu+QKB3qjDkaRJbueg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=AZelkoK8; arc=none smtp.client-ip=209.85.128.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="AZelkoK8" Received: by mail-wm1-f73.google.com with SMTP id 5b1f17b1804b1-488f973ddfeso76715085e9.3 for ; Mon, 27 Apr 2026 08:35:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1777304139; x=1777908939; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=Fih3i4e18PfvVXsOqO+51UGmMLaCoh2lR+Igb4mepcc=; b=AZelkoK8lV4B6imC/9IdUO0WNNoVWz8A7iy1WwR8OPBjBRaf7qpBiAiGhRhKSJU/CR +7MfgYWJ0dr1RsQJJOsKpSthlG/8ygEV/xPiTscuTesFW2W7Zm2kxD5fLGpjB3C6ket7 A6bcKjMvWhpa5Wai25MmKUwuoc8ajl3Q2zWbdWiAha153RGh8E4r72kVkjUkqHf4fJPL qaruTFeSuhwliUd9OPSKSqdZq94IjAs9rbhVJ+cmpw3n9zKsL5N53Lx4MMiTn2cARAJu dvwzn+RpXrI1j+26SIySvpuggTKR02Hd1WDSQiD+RHRKr0fNOz3rZcYbSQsI1sGqoT6L usDA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1777304139; x=1777908939; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Fih3i4e18PfvVXsOqO+51UGmMLaCoh2lR+Igb4mepcc=; b=lCd0sYaBxY0tZEnvkaTmMnr3x/0BvKH1rhoB1LJhKXWEmuEIGwDRHuKInEMlN5BmIR o2Oc4wx49TdFUynALEOe+fRSprNUXmkTqnbkO3N31dFnKxuPOd4qxKie0vlOS8l0dOiF /DyQDTgaC/y5HH4ZXfNx8ZlOTmcH8uDTxHNyubd5Svzwyva1cFUsPFbyhEKLfyD4dgR1 jW9+OJ4M59TLsE0wN4EbQqKC4BtMCyyLPlD+Q+QfnYhlloBGnai+OhWUnwvlyOBqtRBV RXQdKXWQ/HCsrInj6im0hndGuCsWQlsrczFQjB1f/XRcE/d8sMdVDXBiHE7e1h9nqc2W ydmA== X-Gm-Message-State: AOJu0YwQvY7pHpl1sswNSKN0XyyJH+t1W4+N5SGSSGWcMHT/PUXheMW1 zBOfdbWfHP7SnNvAc6PUkpLhrhztsUOnCi7LHNFiG3d7lhDyodGkx8+kSRvbE3trrVTub2b8sw= = X-Received: from wmok24.prod.google.com ([2002:a05:600c:4798:b0:485:3a48:d6c5]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:a30b:b0:487:4eb:d125 with SMTP id 5b1f17b1804b1-488fb74dff1mr437765065e9.9.1777304139320; Mon, 27 Apr 2026 08:35:39 -0700 (PDT) Date: Mon, 27 Apr 2026 17:34:22 +0200 In-Reply-To: <20260427153416.2103979-17-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260427153416.2103979-17-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=973; i=ardb@kernel.org; h=from:subject; bh=h7ZYPXsk7x8qiYN5Bb4ArPGJm788B+DXEeUwDjC4Hlg=; b=owGbwMvMwCVmkMcZplerG8N4Wi2JIfN9E2OMr//DSScOyewrtDKSTj2pNjs54uXG3/nW1/QLW jXW+t/rKGVhEONikBVTZBGY/ffdztMTpWqdZ8nCzGFlAhnCwMUpABPxecnwP+a5BK/ZucVOeU81 7jYvnbjm7Q/9FoVaDcW+23MP86w91sLwi+mK+4lU2YQl89dJGK4S/TllWwHHxeqs1PkRZyUC700 14gQA X-Mailer: git-send-email 2.54.0.rc2.544.gc7ae2d5bb8-goog Message-ID: <20260427153416.2103979-22-ardb+git@google.com> Subject: [PATCH v4 05/15] arm64: mm: Remove bogus stop condition from map_mem() loop From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, Ard Biesheuvel , Ryan Roberts , Anshuman Khandual , Liz Prucka , Seth Jenkins , Kees Cook , Mike Rapoport , David Hildenbrand , Andrew Morton , linux-mm@kvack.org, linux-hardening@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel The memblock API guarantees that start is not greater than or equal to end, so there is no need to test it. And if it were, it is doubtful that breaking out of the loop would be a reasonable course of action here (rather than attempting to map the remaining regions) So let's drop this check. Reviewed-by: Ryan Roberts Signed-off-by: Ard Biesheuvel Reviewed-by: Kevin Brodsky --- arch/arm64/mm/mmu.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index 005844e521bd..bfbf3fe0d1be 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -1177,8 +1177,6 @@ static void __init map_mem(pgd_t *pgdp) =20 /* map all the memory banks */ for_each_mem_range(i, &start, &end) { - if (start >=3D end) - break; /* * The linear map must allow allocation tags reading/writing * if MTE is present. Otherwise, it has the same attributes as --=20 2.54.0.rc2.544.gc7ae2d5bb8-goog From nobody Wed Jun 17 06:02:47 2026 Received: from mail-wm1-f73.google.com (mail-wm1-f73.google.com [209.85.128.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F3A9C3845CF for ; Mon, 27 Apr 2026 15:35:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777304143; cv=none; b=rjQhImRNQSeIwp1G6Du+XWbWTNVyA2Qs/t4IPQfVNbek0MbI734tQ7jfgNZntETvAROj7Y6QhqxiNBfx9QmSLvcyWmdMyZgzQxgz7IgmuOE7PRZTWjRVJwksWoxKRJUbnpa9T9eX4Z1XzGnq/fReyxAsUe25vKMl1NsvbsU/NDw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777304143; c=relaxed/simple; bh=YSn5OY2igzYqsigHvr+U2TYxlmwUZPpRqN93TFEfT3I=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=ZFEQYEd/oc2VTShzizfUp3fJTDM2Rzp7ChWZuveDOkrUvxSXKIEP19j+gRLykps4pVj6EvWfmmmOof2aqBNN3h8CzY+Rq1jfY0zDcwJcX0x1G6vk/Axf0g+rCAwtc5giGn+W5hSRqyge3osr6Qif97k9KrR7gllcqpp4sDiJeKE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=KJrczqzr; arc=none smtp.client-ip=209.85.128.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="KJrczqzr" Received: by mail-wm1-f73.google.com with SMTP id 5b1f17b1804b1-4837b6f6b93so111412755e9.3 for ; Mon, 27 Apr 2026 08:35:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1777304140; x=1777908940; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=XPGO5NkKZbwBF89donUpo2WBB4phkcPD6/ayIGThuKs=; b=KJrczqzrhwUrQT/5+h4Qi/Oj0GPy8e4BraXIOjUU5gt9KccJTeuj/6NDpicmEKO6xp 5Uw1h/Fg+F2KqSXl0FoB8OuQ/moqklegdKv+Z7klBdvKDIhlCf2J9lhOyjJt1triMblG 6aFpsvminyb1n7bMlLMdU0OBemsXEgma3E3K6LaKwDcOrz3zOx8EKcxONA2xoHjjmrFW 82Wh6wjMhU9YMLMz3uU7KYvf6VHWlSwsD1i/A6s2Fdygoc602JLNfHXM8QBGxbz/iyux IFH2f01+cbZiLXw8quMCUwCuCfzJDYp8A810b4Lqwc+rynM++0QO1mD81iUG3Q3TYVTI iHzQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1777304140; x=1777908940; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=XPGO5NkKZbwBF89donUpo2WBB4phkcPD6/ayIGThuKs=; b=CduTDsaH++Xnn0UIM+7fyZyRLLhfwl1z1W+rNIhfeQoPojzXijHvb4iVw0adD6GdXe lk5SF/SUVCSP8dkW1FyL2tHGW87/tMpAgI1kGgKABrAUC8cKW+de6jSfoM+Idp12mGGb N8PJIe9n7VRkFCHUrs9Ld0gWlW9C99Fey2PjR27S9e1zBR9ydwwBidXHAB/yrsa2AP07 gU8B433btxX6mjSI2F0ZRECC1r+olyKpPRL3R8o+OoTaCuCa0yfEqa4QViJAkT8fVgyR wnh//fkOx3m2+xpzcD9vf3F+TgFSAS1l9v1b0/6iEP8+3Xmbvqt/sT2TWfiQnYuRu8W0 WnCQ== X-Gm-Message-State: AOJu0YywY+ZFK3/wks7IefsSzf4+F//mM+Kocu0IQgwWXKmaVrK+ndH2 vN/QozZamSBKL4vKM7sjhPZcHFFSoem+JMcXZ8M8m/3peK3Mf78TUq4AVf+6BKdK4bJIHIdRww= = X-Received: from wmbjs21.prod.google.com ([2002:a05:600c:5655:b0:489:690:d5dd]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:c04a:b0:487:12c:e7e1 with SMTP id 5b1f17b1804b1-488fb750840mr434963185e9.11.1777304140187; Mon, 27 Apr 2026 08:35:40 -0700 (PDT) Date: Mon, 27 Apr 2026 17:34:23 +0200 In-Reply-To: <20260427153416.2103979-17-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260427153416.2103979-17-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=3391; i=ardb@kernel.org; h=from:subject; bh=I2DZ4/UqDVyZEyJTtirGLkm2tY5dVwLgkN0d0TthFiQ=; b=owGbwMvMwCVmkMcZplerG8N4Wi2JIfN9E5Oz4cnQFbZz/ubvO6+x9N2e9y2fv818/K0644BF2 uqMO09WdZSyMIhxMciKKbIIzP77bufpiVK1zrNkYeawMoEMYeDiFICJfP3FyNCv9vzcT50tV2ab tKs6rdmtmP+ZSdS8q2CHk5bVhmD/I6sZGe6dv55h+qD/YWdJV464V9y9yPhfLStb8rbUHZdcsCT 6KCcA X-Mailer: git-send-email 2.54.0.rc2.544.gc7ae2d5bb8-goog Message-ID: <20260427153416.2103979-23-ardb+git@google.com> Subject: [PATCH v4 06/15] arm64: mm: Drop redundant pgd_t* argument from map_mem() From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, Ard Biesheuvel , Ryan Roberts , Anshuman Khandual , Liz Prucka , Seth Jenkins , Kees Cook , Mike Rapoport , David Hildenbrand , Andrew Morton , linux-mm@kvack.org, linux-hardening@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel __map_memblock() and map_mem() always operate on swapper_pg_dir, so there is no need to pass around a pgd_t pointer between them. Reviewed-by: Ryan Roberts Signed-off-by: Ard Biesheuvel Reviewed-by: Kevin Brodsky --- arch/arm64/mm/mmu.c | 25 ++++++++++---------- 1 file changed, 12 insertions(+), 13 deletions(-) diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index bfbf3fe0d1be..9610dd2d7bd9 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -1039,11 +1039,11 @@ static void update_mapping_prot(phys_addr_t phys, u= nsigned long virt, flush_tlb_kernel_range(virt, virt + size); } =20 -static void __init __map_memblock(pgd_t *pgdp, phys_addr_t start, - phys_addr_t end, pgprot_t prot, int flags) +static void __init __map_memblock(phys_addr_t start, phys_addr_t end, + pgprot_t prot, int flags) { - early_create_pgd_mapping(pgdp, start, __phys_to_virt(start), end - start, - prot, early_pgtable_alloc, flags); + early_create_pgd_mapping(swapper_pg_dir, start, __phys_to_virt(start), + end - start, prot, early_pgtable_alloc, flags); } =20 void __init mark_linear_text_alias_ro(void) @@ -1091,13 +1091,13 @@ static phys_addr_t __init arm64_kfence_alloc_pool(v= oid) return kfence_pool; } =20 -static void __init arm64_kfence_map_pool(phys_addr_t kfence_pool, pgd_t *p= gdp) +static void __init arm64_kfence_map_pool(phys_addr_t kfence_pool) { if (!kfence_pool) return; =20 /* KFENCE pool needs page-level mapping. */ - __map_memblock(pgdp, kfence_pool, kfence_pool + KFENCE_POOL_SIZE, + __map_memblock(kfence_pool, kfence_pool + KFENCE_POOL_SIZE, pgprot_tagged(PAGE_KERNEL), NO_BLOCK_MAPPINGS | NO_CONT_MAPPINGS); memblock_clear_nomap(kfence_pool, KFENCE_POOL_SIZE); @@ -1133,11 +1133,11 @@ bool arch_kfence_init_pool(void) #else /* CONFIG_KFENCE */ =20 static inline phys_addr_t arm64_kfence_alloc_pool(void) { return 0; } -static inline void arm64_kfence_map_pool(phys_addr_t kfence_pool, pgd_t *p= gdp) { } +static inline void arm64_kfence_map_pool(phys_addr_t kfence_pool) { } =20 #endif /* CONFIG_KFENCE */ =20 -static void __init map_mem(pgd_t *pgdp) +static void __init map_mem(void) { static const u64 direct_map_end =3D _PAGE_END(VA_BITS_MIN); phys_addr_t kernel_start =3D __pa_symbol(_text); @@ -1182,7 +1182,7 @@ static void __init map_mem(pgd_t *pgdp) * if MTE is present. Otherwise, it has the same attributes as * PAGE_KERNEL. */ - __map_memblock(pgdp, start, end, pgprot_tagged(PAGE_KERNEL), + __map_memblock(start, end, pgprot_tagged(PAGE_KERNEL), flags); } =20 @@ -1196,10 +1196,9 @@ static void __init map_mem(pgd_t *pgdp) * Note that contiguous mappings cannot be remapped in this way, * so we should avoid them here. */ - __map_memblock(pgdp, kernel_start, kernel_end, - PAGE_KERNEL, NO_CONT_MAPPINGS); + __map_memblock(kernel_start, kernel_end, PAGE_KERNEL, NO_CONT_MAPPINGS); memblock_clear_nomap(kernel_start, kernel_end - kernel_start); - arm64_kfence_map_pool(early_kfence_pool, pgdp); + arm64_kfence_map_pool(early_kfence_pool); } =20 void mark_rodata_ro(void) @@ -1421,7 +1420,7 @@ static void __init create_idmap(void) =20 void __init paging_init(void) { - map_mem(swapper_pg_dir); + map_mem(); =20 memblock_allow_resize(); =20 --=20 2.54.0.rc2.544.gc7ae2d5bb8-goog From nobody Wed Jun 17 06:02:47 2026 Received: from mail-wr1-f74.google.com (mail-wr1-f74.google.com [209.85.221.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B77763B6343 for ; Mon, 27 Apr 2026 15:35:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777304145; cv=none; b=SPyXts9lzdMizPBo8qM7MGS2W8wVgCZwbFuE8oywjq4XTscklJV7TQihUEmMh4sH8xGo0TqnIMB66x+WchFnm1ZuWGM/NHsLm1HrT3gMkTSWi/JclDofaUFlQvZe70n9xQdC0kuEflTLHgAaSrDp8ymFHTO98JY58dcAQeX9J6E= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777304145; c=relaxed/simple; bh=BDMt9Te/HD4/FmFEC5d4cBAlbQOmsC59W6/7DsC1lyY=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=jXmcp8GXbwLIYG3BI1paNBpArfCq13n38gFrxUdx8wuB1OL+9r9e1VAchw3MLlUjyO4FDxwGIsKWbs1JKT9tIKhQmgNOdC1Xqb8a2DKbY59zYJuZg2NzpSSdt9AqI/dTET/HV32ZIOp2JI/07aI99h5uN/FT32GO/TlwbfaeitM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=qUwv47yP; arc=none smtp.client-ip=209.85.221.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="qUwv47yP" Received: by mail-wr1-f74.google.com with SMTP id ffacd0b85a97d-440d12a472eso7493969f8f.3 for ; Mon, 27 Apr 2026 08:35:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1777304142; x=1777908942; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=+WwzihijZssqSOtMqDRFEFZS96kFccDT7AOSNModvwc=; b=qUwv47yPbLdle3jNwWHGnD+P29y2hvQnXGyTGRVtGZ0ndIso5/NKzSw6IDodkuNp80 r+xITZZEi6mgHGRYeLHnryn3HSRgQD+FocBBFlm02w+cT8vbbMCMBpZNkJyzl/cD9zp3 4hvpHSHNzYIvirWttiaqmLX5457VHPk1GFBL4UEyzk62MXmpI6nioa1sd51HnK2pW1W2 in6llznOSwm6lcyFsUy1FpVTEbzLkWxRHN5q86yoeFjlAQIWuhQCbmF11Qz0W10o8MPO CmubSsWK5a2+MiuVZ4Yoaz2DxOvTNc+oiR+O/nuZ9NCKp59Yuy95cFakz5kIlb9YcJln bD/A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1777304142; x=1777908942; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=+WwzihijZssqSOtMqDRFEFZS96kFccDT7AOSNModvwc=; b=mcYgNvZFX7he80kSPastgEHrZU1UbtO0yWmB4ci+QD47EcyX9oqEVrhuGopoEY37q+ ws9SAll63dh6e6vlah+5QjHYzZHz/coRLWM3/pQ1EykXbyXfszFwdxRf5le06fhlbmdu 9WGFgw9VKIycL4BVpOs74L6Kpe8h2X+gcjtZE8lwD3xUHrN+aHzmpjQzQsS89IMrOQ0x XSEPRv/ci2q3UckZg+eRsGKP1M1ZYtKzexkOgTZXqKMPCGGgUHK2DK7yHYUzbnc5T/QR 7HlJ1QLUhd2IFag5pqhk8I1yUyaufYV3vDn8t4w/7vytOdpfVB3peBG7ucsQylz+fVuj tdWg== X-Gm-Message-State: AOJu0YxoxjzhBEThKl9sePsYnOh6yTwNzUf4w7k3wiFieyRqH18Iv2Vl 0esQU5/TNMu5fAepzjAoUgpgFIVc/qlmzEIkWkIauGHxGgMF8RX+tHTCbTbUCTUDQDlbtATXuw= = X-Received: from wrbdy16.prod.google.com ([2002:a05:6000:e50:b0:43d:729a:b525]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a5d:5d05:0:b0:441:247a:e98e with SMTP id ffacd0b85a97d-441247ae9ebmr36895346f8f.24.1777304141472; Mon, 27 Apr 2026 08:35:41 -0700 (PDT) Date: Mon, 27 Apr 2026 17:34:24 +0200 In-Reply-To: <20260427153416.2103979-17-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260427153416.2103979-17-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=918; i=ardb@kernel.org; h=from:subject; bh=2M+2R/xfGuHti5Wurk6r5E9xqbGluJyDsIl7VYMOsyI=; b=owGbwMvMwCVmkMcZplerG8N4Wi2JIfN9E8tUw49az8LPeSafvHDx3bajk3bfqxCf4/ElO9n+i EqbmadXRykLgxgXg6yYIovA7L/vdp6eKFXrPEsWZg4rE8gQBi5OAZhIxHGGf2rT5kc6JZw8EfTc dtq0B6b/C6uZdnxyULeaKj/79Ou3L1gZGd7fr5d6L8ZyoL7vypL3Bx1P/wzxNdu7JOz5R5H7c65 9f80EAA== X-Mailer: git-send-email 2.54.0.rc2.544.gc7ae2d5bb8-goog Message-ID: <20260427153416.2103979-24-ardb+git@google.com> Subject: [PATCH v4 07/15] arm64: mm: Permit contiguous descriptors to be rewritten From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, Ard Biesheuvel , Ryan Roberts , Anshuman Khandual , Liz Prucka , Seth Jenkins , Kees Cook , Mike Rapoport , David Hildenbrand , Andrew Morton , linux-mm@kvack.org, linux-hardening@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel Currently, pgattr_change_is_safe() is overly pedantic when it comes to descriptors with the contiguous hint attribute set, as it rejects assignments even if the old and the new value are the same. So relax the check to allow that. Signed-off-by: Ard Biesheuvel --- arch/arm64/mm/mmu.c | 4 ---- 1 file changed, 4 deletions(-) diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index 9610dd2d7bd9..bfb2f1cae724 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -134,10 +134,6 @@ bool pgattr_change_is_safe(pteval_t old, pteval_t new) if (pte_pfn(__pte(old)) !=3D pte_pfn(__pte(new))) return false; =20 - /* live contiguous mappings may not be manipulated at all */ - if ((old | new) & PTE_CONT) - return false; - /* Transitioning from Non-Global to Global is unsafe */ if (old & ~new & PTE_NG) return false; --=20 2.54.0.rc2.544.gc7ae2d5bb8-goog From nobody Wed Jun 17 06:02:47 2026 Received: from mail-wm1-f73.google.com (mail-wm1-f73.google.com [209.85.128.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 77F523C13E5 for ; Mon, 27 Apr 2026 15:35:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777304146; cv=none; b=MEQypGrcf0EWRl9QAJRzHX705UdY+LhHYQB543lU66maq6O+e4t0CrZX+8md0Hwsm4S13HPcg+am3Pqg1QdqdelVzDigp9bPOqtYjv0hYadaPsYXdZCEzkdqWLz0bK8AVNXNDc4Ctwtnd3wrhAYcf4N1F3wSVGN0Pb3mPxdi4K8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777304146; c=relaxed/simple; bh=XQTAyphR1+vWloMkYQ0Xfuz2LlYAScTF25LGihxHCes=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Rj2SOxioTwplJfKtweq66B8+wokF1tIPRytuu2J8F+WUsT4UZTM9cluwJQ7PuaIkbkiRiPRvvhw/AvWzBVmhhc7cuCpc3GZh8ueT+HpcIfGb7UUabbo972K4j771jEBegavpQqGWWNAN6GgxVti1KlRMbdR3cgZd4IS+yoOuGoo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=bH+cgSn9; arc=none smtp.client-ip=209.85.128.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="bH+cgSn9" Received: by mail-wm1-f73.google.com with SMTP id 5b1f17b1804b1-488d1b5bca0so61924985e9.2 for ; Mon, 27 Apr 2026 08:35:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1777304143; x=1777908943; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=XRW0Zv+9eSRHC2Z8bSw9fwUsmK+GqZsuc2Bm/mr5lio=; b=bH+cgSn9XqmZ2RJjPxrk1VL3v4kg+eNBH+J7fQsb5GvUfqCzcqAB1mTR+WN3TZsHLq h6aH92Ufh/x8nwiGW4iPO5mvpPrw+F9+Dt1cBYLfg5ofIypXT3bRMWxLzuGFEbRvap4N 6Q77+M2YlgS+U723/3wVb3Au66c5p2HpoY4yCsWGL0nVbBOvKqR4upwK3hoc2f+qu96L 729tNxWJUFQeAORavPjM8bepZXYcGUXOq6IJM1ik/90SQLc+Xcy3a4OB8hZIX9no8UP4 1batBezw9q/fl/YdxretNp9X4Smi9Fy9GXAqPP1xHjP7mkk4eyNhzmLTBKSMvni5qnHY UzVg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1777304143; x=1777908943; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=XRW0Zv+9eSRHC2Z8bSw9fwUsmK+GqZsuc2Bm/mr5lio=; b=ExdcFOk571vaXJQ+9IdZM46rzzgfSOuO7GOmSULOrIHbbq38TIgyEXDWZz+7sDriLp Eb7SPrpQKdX5jJ1kKmQHGlFAO80BzANks3QTN7eoMUGSrZE+lzYkPGLbPBk1IFAglCiE YhlAAFjLT6L3d4adNHU7wKFyuObKT26n3NRRLqAboQ+lxjegCGuN0vaYnkqCuWXx4Brl GSYC4JR69iOXSLaegJkwHFqOxbJO7ZXwoKsqVi7q+XVxsl1sZmdJ6ytLQ8eiepXvyfos ttJBz/B9IRPwZPMA2mGQ1FVXXfVe8VpsYZToCdFwAGQkqRdr0HstOadtaOKRybQgWb18 T/bw== X-Gm-Message-State: AOJu0Yy1dXMjHiG6B5rQenyqUS0TlPi0OychorcP69jNs5TxHCz/9YJa WFAUKXTjPg4ngjIR7a+MYup+cqsWth7yorIVv2XBs37yCJzoZGaE6ikkKnzq5j/12SGXvqGsIw= = X-Received: from wmi26.prod.google.com ([2002:a05:600c:21a:b0:48a:591a:7a7f]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:6085:b0:488:936a:6220 with SMTP id 5b1f17b1804b1-488fb782618mr560638815e9.21.1777304142771; Mon, 27 Apr 2026 08:35:42 -0700 (PDT) Date: Mon, 27 Apr 2026 17:34:25 +0200 In-Reply-To: <20260427153416.2103979-17-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260427153416.2103979-17-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=3127; i=ardb@kernel.org; h=from:subject; bh=j6jBQGBWye927YKYtEIXGLmbEkQZrSxTUWO+ZEqsDOA=; b=owGbwMvMwCVmkMcZplerG8N4Wi2JIfN9E6tZbZSRzXSOd4Kz7RcrfTl71zvelVlQ7kjb6c7+6 +/LvbZ3lLIwiHExyIopsgjM/vtu5+mJUrXOs2Rh5rAygQxh4OIUgImcesvIMHmRmfnsqu7t5Yc4 f/yT6uhbkVjgOtPMoPXVglXVv166rmL4H7fu82/15c+rukSuf9ApCKp0mWdk7517vtF+nva/d2u e8gAA X-Mailer: git-send-email 2.54.0.rc2.544.gc7ae2d5bb8-goog Message-ID: <20260427153416.2103979-25-ardb+git@google.com> Subject: [PATCH v4 08/15] arm64: kfence: Avoid NOMAP tricks when mapping the early pool From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, Ard Biesheuvel , Ryan Roberts , Anshuman Khandual , Liz Prucka , Seth Jenkins , Kees Cook , Mike Rapoport , David Hildenbrand , Andrew Morton , linux-mm@kvack.org, linux-hardening@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel Now that the map_mem() routines respect existing page mappings and contiguous granule sized blocks with the contiguous bit cleared, there is no longer a reason to play tricks with the memblock NOMAP attribute. Instead, the kfence pool can be allocated and mapped with page granularity first, and this granularity will be respected when the rest of DRAM is mapped later, even if block and contiguous mappings are allowed for the remainder of those mappings. Signed-off-by: Ard Biesheuvel --- arch/arm64/mm/mmu.c | 25 ++++---------------- 1 file changed, 5 insertions(+), 20 deletions(-) diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index bfb2f1cae724..4eab40f4aa6f 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -1067,36 +1067,24 @@ static int __init parse_kfence_early_init(char *arg) } early_param("kfence.sample_interval", parse_kfence_early_init); =20 -static phys_addr_t __init arm64_kfence_alloc_pool(void) +static void __init arm64_kfence_map_pool(void) { phys_addr_t kfence_pool; =20 if (!kfence_early_init) - return 0; + return; =20 kfence_pool =3D memblock_phys_alloc(KFENCE_POOL_SIZE, PAGE_SIZE); if (!kfence_pool) { pr_err("failed to allocate kfence pool\n"); kfence_early_init =3D false; - return 0; - } - - /* Temporarily mark as NOMAP. */ - memblock_mark_nomap(kfence_pool, KFENCE_POOL_SIZE); - - return kfence_pool; -} - -static void __init arm64_kfence_map_pool(phys_addr_t kfence_pool) -{ - if (!kfence_pool) return; + } =20 /* KFENCE pool needs page-level mapping. */ __map_memblock(kfence_pool, kfence_pool + KFENCE_POOL_SIZE, pgprot_tagged(PAGE_KERNEL), NO_BLOCK_MAPPINGS | NO_CONT_MAPPINGS); - memblock_clear_nomap(kfence_pool, KFENCE_POOL_SIZE); __kfence_pool =3D phys_to_virt(kfence_pool); } =20 @@ -1128,8 +1116,7 @@ bool arch_kfence_init_pool(void) } #else /* CONFIG_KFENCE */ =20 -static inline phys_addr_t arm64_kfence_alloc_pool(void) { return 0; } -static inline void arm64_kfence_map_pool(phys_addr_t kfence_pool) { } +static inline void arm64_kfence_map_pool(void) { } =20 #endif /* CONFIG_KFENCE */ =20 @@ -1139,7 +1126,6 @@ static void __init map_mem(void) phys_addr_t kernel_start =3D __pa_symbol(_text); phys_addr_t kernel_end =3D __pa_symbol(__init_begin); phys_addr_t start, end; - phys_addr_t early_kfence_pool; int flags =3D NO_EXEC_MAPPINGS; u64 i; =20 @@ -1156,7 +1142,7 @@ static void __init map_mem(void) BUILD_BUG_ON(pgd_index(direct_map_end - 1) =3D=3D pgd_index(direct_map_en= d) && pgd_index(_PAGE_OFFSET(VA_BITS_MIN)) !=3D PTRS_PER_PGD - 1); =20 - early_kfence_pool =3D arm64_kfence_alloc_pool(); + arm64_kfence_map_pool(); =20 linear_map_requires_bbml2 =3D !force_pte_mapping() && can_set_direct_map(= ); =20 @@ -1194,7 +1180,6 @@ static void __init map_mem(void) */ __map_memblock(kernel_start, kernel_end, PAGE_KERNEL, NO_CONT_MAPPINGS); memblock_clear_nomap(kernel_start, kernel_end - kernel_start); - arm64_kfence_map_pool(early_kfence_pool); } =20 void mark_rodata_ro(void) --=20 2.54.0.rc2.544.gc7ae2d5bb8-goog From nobody Wed Jun 17 06:02:47 2026 Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C51F73D6674 for ; Mon, 27 Apr 2026 15:35:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777304147; cv=none; b=YK6hCSehO0SbmvotNOf1Dox/A1WaWFPknj8NfCIaQvV9pIZhNbz6tIKYQb+/C2sPvh538U95/82XcdlO1NBpagdoxjrfedYOf+PlDPV88IFHD/xg2W4NI0fO59fU+TqCu1216EvY4XMPyY34y6uOpV8b+LG0ZRHKmW5oGopv8Qk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777304147; c=relaxed/simple; bh=cnTzZwtV6wM4iTLgIq3xKSD1jkOxl7u/+eyHiVvz3to=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=rgO/RPoE0YkYGW/jf7Ws20Ovs4208zo4S/XHICVotPY8JFjM89ZVv6wpBfh7uv+5IEU4JQz4226OX+ktYWvhOfTjm3IDlgyL/oL4JurM3BHlQ4IVw6kNYl0l2hTcauIwS14zMsMJ3suoaiUqhvIp0TSzXZqEBeRjWWkWDSUl1Wo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=CKeg2180; arc=none smtp.client-ip=209.85.128.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="CKeg2180" Received: by mail-wm1-f74.google.com with SMTP id 5b1f17b1804b1-488c2aa6becso94072645e9.2 for ; Mon, 27 Apr 2026 08:35:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1777304144; x=1777908944; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=1BA/E6WicDwezK3gyiAo3eqUE5pDQu9c0JrSuIiaUTE=; b=CKeg2180Eycn76dTsW0ZLOFNX32fAOUIxWMUB3Bm1289kgHoV/c/2HM6aLwtH5YfyX Klm+E07iOM4B3l7uKA+JA6e+1Zbsn7D7OiQcbwN6nWFFFlTnlXMbADSLh4BxdimTOmWI iGrzozOK5iz1X7yt5iHeCl/di/m+Er4Gfn2rmX/OtS34zsU0xLcwAETnxYaG95LkB6+k GZ2/M5U5fi3037cHj8WWgagRThbHORHaLHwMdMiSrVsiMeskq5dbhto9+Zp+Kkd8Xmy+ 4rzGj5ACJt5WuSUle1q8EDkRug1CiYeqGdI35FCmk2F+O8YgIcKeX2/CdQkh5FvzW27L P1kQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1777304144; x=1777908944; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=1BA/E6WicDwezK3gyiAo3eqUE5pDQu9c0JrSuIiaUTE=; b=Ew2tZMqvKrYlQW0r+T+HDRf3h10KPsUZhPqoVOZXrY+6OyrQ0hcF+5E99f0xm8ecpw olQwiryZWoYp2/yy+TwbCFUF1faC3NftYA9HRs3t8K/7IgJJOCisZ7vO5DJNj3Vik9Lv gaIKHbyD8NJzAkjkd39I2bCfg53LneEvaca5wAYf1HDCNITQM6Zhx1IgJ9Jiizn3Q+Dg IMZ/R/q70yYD2Wy5Fs+w2CnQa5vJgD+6xLTYrsQfVX0ldUOhARJjuYuHufIFqbTaxNyG lyz7dmaEQuRSj5DLfjF/TYlR2f+wulcnu9CiVegBH75WWYx6DZ1BlzKo+8Qa3nIHwUJ9 Qpyw== X-Gm-Message-State: AOJu0YzF113MJ+8Ni0evA3EGZQMQ/ckO1FdzlTC8EPk7HfFriTOMpVPA scQmnve3ArQDywqyrcRNio48NnJulWJoDEkzko/MjNw6oYUO9gbtptemkhfgri0IPo0a61yO7Q= = X-Received: from wmsl15.prod.google.com ([2002:a05:600c:1d0f:b0:488:7f4d:30f4]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:8115:b0:488:8577:d9cc with SMTP id 5b1f17b1804b1-488fb77faacmr552964815e9.20.1777304143994; Mon, 27 Apr 2026 08:35:43 -0700 (PDT) Date: Mon, 27 Apr 2026 17:34:26 +0200 In-Reply-To: <20260427153416.2103979-17-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260427153416.2103979-17-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=2895; i=ardb@kernel.org; h=from:subject; bh=KZuPOnzhvCTYIkGg+7u5wsbMT7w/fODSH3EL/0/zagg=; b=owGbwMvMwCVmkMcZplerG8N4Wi2JIfN9E5vLgafBpVHCxz1SUmelxP6Kd3qkdfpFsMXLg9sPF WoserK1o5SFQYyLQVZMkUVg9t93O09PlKp1niULM4eVCWQIAxenAEykTJOR4TUXe41Hw4vaRzxT mGLXWVx6se1hW1uOb3v81vdLAn5m5jP84ZJm0rae256UuMVmRe7sInHmbY93p/Jwyz966BT+h1O dCQA= X-Mailer: git-send-email 2.54.0.rc2.544.gc7ae2d5bb8-goog Message-ID: <20260427153416.2103979-26-ardb+git@google.com> Subject: [PATCH v4 09/15] arm64: mm: Permit contiguous attribute for preliminary mappings From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, Ard Biesheuvel , Ryan Roberts , Anshuman Khandual , Liz Prucka , Seth Jenkins , Kees Cook , Mike Rapoport , David Hildenbrand , Andrew Morton , linux-mm@kvack.org, linux-hardening@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel There are a few cases where we omit the contiguous hint for mappings that start out as read-write and are remapped read-only later, on the basis that manipulating live descriptors with the PTE_CONT attribute set is unsafe. When support for the contiguous hint was added to the code, the ARM ARM was ambiguous about this, and so we erred on the side of caution. In the meantime, this has been clarified [0], and regions that will be remapped in their entirety can use the contiguous hint both in the initial mapping as well as the one that replaces it. Note that this requires that the logic that may be called to remap overlapping regions respects existing valid descriptors that have the contiguous bit cleared. So omit the NO_CONT_MAPPINGS flag in places where it is unneeded. Thanks to Ryan for the reference. [0] RJQQTC For a TLB lookup in a contiguous region mapped by translation table entries= that have consistent values for the Contiguous bit, but have the OA, attributes,= or permissions misprogrammed, that TLB lookup is permitted to produce an OA, a= ccess permissions, and memory attributes that are consistent with any one of the programmed translation table values. Signed-off-by: Ard Biesheuvel --- arch/arm64/mm/mmu.c | 10 +++------- 1 file changed, 3 insertions(+), 7 deletions(-) diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index 4eab40f4aa6f..5e2348b15783 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -1000,8 +1000,7 @@ void __init create_mapping_noalloc(phys_addr_t phys, = unsigned long virt, &phys, virt); return; } - early_create_pgd_mapping(init_mm.pgd, phys, virt, size, prot, NULL, - NO_CONT_MAPPINGS); + early_create_pgd_mapping(init_mm.pgd, phys, virt, size, prot, NULL, 0); } =20 void __init create_pgd_mapping(struct mm_struct *mm, phys_addr_t phys, @@ -1028,8 +1027,7 @@ static void update_mapping_prot(phys_addr_t phys, uns= igned long virt, return; } =20 - early_create_pgd_mapping(init_mm.pgd, phys, virt, size, prot, NULL, - NO_CONT_MAPPINGS); + early_create_pgd_mapping(init_mm.pgd, phys, virt, size, prot, NULL, 0); =20 /* flush the TLBs after updating live kernel mappings */ flush_tlb_kernel_range(virt, virt + size); @@ -1175,10 +1173,8 @@ static void __init map_mem(void) * alternative patching has completed). This makes the contents * of the region accessible to subsystems such as hibernate, * but protects it from inadvertent modification or execution. - * Note that contiguous mappings cannot be remapped in this way, - * so we should avoid them here. */ - __map_memblock(kernel_start, kernel_end, PAGE_KERNEL, NO_CONT_MAPPINGS); + __map_memblock(kernel_start, kernel_end, PAGE_KERNEL, 0); memblock_clear_nomap(kernel_start, kernel_end - kernel_start); } =20 --=20 2.54.0.rc2.544.gc7ae2d5bb8-goog From nobody Wed Jun 17 06:02:47 2026 Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 913553D6CDE for ; Mon, 27 Apr 2026 15:35:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777304148; cv=none; b=C2HllwStFKrB0+zLMySdXoQMVuWlFlF24GmmlparKSQjy7Bt8EB9p3Abx+UJ4lueGYawrqJ2kyYWcFLW+TviJVD3DRuMsXCqODFSu7nz/GJrj0Jxyf6AZ9S9Fanjyx70DjcSLmjJTmz05VZ7VYB9iGh1AbWgOdDLJ4nt0osNCZc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777304148; c=relaxed/simple; bh=5aJqD3YjV78A3aSXyWLniC+6Eqi71MBuIePvROYkngA=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=ogUxfPkwzHSWvtI5N/QUcW8Fhnvb1DHP7BM8yT2DlOESaCLNFALlcT28konQod/wezP9TIAMpzsStu7t9dFoF6bmjxdWYp0Lfv+n+c9paLN3a87M0JwxdxLM6AWBUUPJW6c5IWXy4zupgJF6tLvnREq3w3CRVV3989K1W0zn2O8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=UHsXUKXc; arc=none smtp.client-ip=209.85.128.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="UHsXUKXc" Received: by mail-wm1-f74.google.com with SMTP id 5b1f17b1804b1-488bd1ee9e7so91794655e9.1 for ; Mon, 27 Apr 2026 08:35:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1777304145; x=1777908945; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=pmP6R09Q9nWqYYDZQR+Duu/8tNuRQcuIH0M5Nuu7R3A=; b=UHsXUKXcCkd96A2T3fVhJWyy2gGMy2hmkeqcticoBJ1u4ls79WF/bz+dEEg0tcHcPq wF2L32k9NaV4CiAiylnAjXks4MGpsjy5CRvo4lCYHPKY/MjWgnA68Ea95KFSnn8N6zSK lt7i9jGi9Vy0VaUda41zAXMxm9D9x2yZcHquMe4SvU/TMcByUdzaeAcFxAD1SgC6UAtx RGKS15rwN2ken6f6Ry3VaKFxk1wgX7vE1zYNDbmAYPGdMvTaYvH8xYS1jNtwomNnwiPX PKLSqPs4KAEc/LU+4HeZffXdWaOFda5vPUDbJf8LI/EADoIwU33w24SFTo3oT8bWSkF+ J9Iw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1777304145; x=1777908945; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=pmP6R09Q9nWqYYDZQR+Duu/8tNuRQcuIH0M5Nuu7R3A=; b=WlCgNhgbAWy3a5q5SgKjTSYS9tQRmNcCDi6VMGKeIJCglnc5nw5D9nbVXTk2mqD7fZ XpzUD0AuOuRmo06mIjnWIkjS/K4p7GdIkj3BzooftkWc6TiAbkulvGFdEwC3VpfVR5+6 UORkcvY7D2/5UOY1DkGRDvRh8wySXWEIjqyAuPPNsVBWYr6A+krEiaFyj6KLMIEcSOxe RdfMaG8TPHbdd4sx7HSDOhfvGEAvvPfbx1o0JJQVYn4XdR03YV8g4OCmYgM4y3eNo1bh YJKJ4lJUNjoRS9jtjVBZbctvMvkqsCB88Y9inSw5M6jVrW1rn00/9SgHUJFSaTp6hTKE jamQ== X-Gm-Message-State: AOJu0YwMEAj7xvvraVX7mJQtRSPiG581ftQZ+NtmrSIPnraaxooRW4Ue 7EwO2aIwaKViQFJOpzPj0CA2ASQhHvMWA1/2AeuhrE8W3ejWvW7Hqq2fmlNmp+8fbCYIt0dlNg= = X-Received: from wmqo21.prod.google.com ([2002:a05:600c:4fd5:b0:486:7a8:4f4e]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:190f:b0:48a:6315:da26 with SMTP id 5b1f17b1804b1-48a6315da72mr216539215e9.26.1777304144872; Mon, 27 Apr 2026 08:35:44 -0700 (PDT) Date: Mon, 27 Apr 2026 17:34:27 +0200 In-Reply-To: <20260427153416.2103979-17-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260427153416.2103979-17-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=1971; i=ardb@kernel.org; h=from:subject; bh=TaqiODuFtOBPeOY7+zirlhCAHHbOKzDzJrx2Uvu6+0U=; b=owGbwMvMwCVmkMcZplerG8N4Wi2JIfN9E4fzFC72tqPtWjWHA8yaWU1Y72S85i48K/X2HmNcy pIdrxM7SlkYxLgYZMUUWQRm/3238/REqVrnWbIwc1iZQIYwcHEKwETSPjIybPkSXFT9mmXt+wyr Tw3/otuqXJ56eLXKyPRt3vTriPK1hYwM5w6/Xlfbco49iWf7XsYt/4uijGorz1hflb/6oC80o/Y yDwA= X-Mailer: git-send-email 2.54.0.rc2.544.gc7ae2d5bb8-goog Message-ID: <20260427153416.2103979-27-ardb+git@google.com> Subject: [PATCH v4 10/15] arm64: Move fixmap page tables to end of kernel image From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, Ard Biesheuvel , Ryan Roberts , Anshuman Khandual , Liz Prucka , Seth Jenkins , Kees Cook , Mike Rapoport , David Hildenbrand , Andrew Morton , linux-mm@kvack.org, linux-hardening@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel Move the fixmap page tables out of the BSS section, and place them at the end of the image, right before the init_pg_dir section where some of the other statically allocated page tables live. These page tables are currently the only data objects in vmlinux that are meant to be accessed via the kernel image's linear alias, and so placing them together allows the remainder of the data/bss section to be remapped read-only or unmapped entirely. Signed-off-by: Ard Biesheuvel Reviewed-by: Kevin Brodsky --- arch/arm64/kernel/vmlinux.lds.S | 5 +++++ arch/arm64/mm/fixmap.c | 7 ++++--- 2 files changed, 9 insertions(+), 3 deletions(-) diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.ld= s.S index e1ac876200a3..2dca18574619 100644 --- a/arch/arm64/kernel/vmlinux.lds.S +++ b/arch/arm64/kernel/vmlinux.lds.S @@ -353,6 +353,11 @@ SECTIONS __pi___bss_start =3D __bss_start; =20 . =3D ALIGN(PAGE_SIZE); + .fixmap_pgdir : { + __fixmap_pgdir_start =3D .; + *(.fixmap_bss) + } + __pi_init_pg_dir =3D .; . +=3D INIT_DIR_SIZE; __pi_init_pg_end =3D .; diff --git a/arch/arm64/mm/fixmap.c b/arch/arm64/mm/fixmap.c index c5c5425791da..b649ea1a46e4 100644 --- a/arch/arm64/mm/fixmap.c +++ b/arch/arm64/mm/fixmap.c @@ -31,9 +31,10 @@ static_assert(NR_BM_PMD_TABLES =3D=3D 1); =20 #define BM_PTE_TABLE_IDX(addr) __BM_TABLE_IDX(addr, PMD_SHIFT) =20 -static pte_t bm_pte[NR_BM_PTE_TABLES][PTRS_PER_PTE] __page_aligned_bss; -static pmd_t bm_pmd[PTRS_PER_PMD] __page_aligned_bss __maybe_unused; -static pud_t bm_pud[PTRS_PER_PUD] __page_aligned_bss __maybe_unused; +#define __fixmap_bss __section(".fixmap_bss") __aligned(PAGE_SIZE) +static pte_t bm_pte[NR_BM_PTE_TABLES][PTRS_PER_PTE] __fixmap_bss; +static pmd_t bm_pmd[PTRS_PER_PMD] __fixmap_bss __maybe_unused; +static pud_t bm_pud[PTRS_PER_PUD] __fixmap_bss __maybe_unused; =20 static inline pte_t *fixmap_pte(unsigned long addr) { --=20 2.54.0.rc2.544.gc7ae2d5bb8-goog From nobody Wed Jun 17 06:02:47 2026 Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C9D263D75DC for ; Mon, 27 Apr 2026 15:35:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777304149; cv=none; b=HS4FEON0tgFYOcMEh93gqvxxElHNXrJuowfLHmcoOfEiUN8s0T7r/s1kxp6FgvsAB2XuNubLNgvcHyqlyWkTyD3vL95arafIJWvI35QRhPIZEYKtRc9moIjGj4JrVcFJdHm+h+cAenmh8ciWG03v4ickeukebZrh3NMZa+PGPZo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777304149; c=relaxed/simple; bh=aVILoAGoZ8TJ1QFt3bYntAAPxvwsWTuUlPK7jfhYHJw=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=JyXdmRJAlfO8JT/KoduVQuEKMlxx7qiucuy4Fh0c2WJU7qYL1G7k26UNcicCS5FIhCtgEFJ0AQezgW4C+qMXfMGYx84pyqumtxo/BsGx+zC0iW4FTNdhJaGoKhvvaAkfQk6aE+IZYmQ7+o5UKhI/qPaXegTjc78MnCHWtFiV9tw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=IQ3r6tpv; arc=none smtp.client-ip=209.85.128.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="IQ3r6tpv" Received: by mail-wm1-f74.google.com with SMTP id 5b1f17b1804b1-48906aa28cbso66730615e9.0 for ; Mon, 27 Apr 2026 08:35:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1777304146; x=1777908946; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=wb69xevabgQmFZqaaKZbB6ba2WMM/rgvKxL0qThMEZ4=; b=IQ3r6tpvewLjnJRpRLrJI7aBbfHR++0AumWEjClCjecnfQOB0XcAC4QJjpzN2bhZ0q ndFMch0uXhfRtFayJbpvN3F3H31WZpOTH3Ka5iNOmz4H341eVpcQW/dOrjDhY0KiPl7G XbHoNJdTvDVRfOCj491GGtUqPMbLJOpsStkIPUeseQLqyICwFiqtjCPaaGkOmAyh7Rxd 5uTRQ/RzZklpUKtP6NwwxaXOXHP32zbuYY5vbabcB1ZRa7Tw28mTyPrdn+fI2oCheoUM 8fZBFDyWX85tj6CVziad7oB5PBerZeKc1kHyWqG+SakH0Ehzyi/hG1/FbkFWC6+khqe0 biUw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1777304146; x=1777908946; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=wb69xevabgQmFZqaaKZbB6ba2WMM/rgvKxL0qThMEZ4=; b=EUZZ9+KP7y8XjrH9eBCQ89rnHlFKo7rop4RK0jc6FFKgAi2OEJe4PYWKdSkUQd9B5j vE0oRbNbfqnm+pt5ez/tEI99ccuk7QB+ctJJNvDwwxemgLNHNokOiix1CyaQMM0pfotR CUHGRQSAkhBdA92hsxboZNCAd/0lvn9U674sbrw/GIoEtexWdBoC9VhZkWUkJGZbR3qM 0MjtyWMIx9e3IUvPwTdzSlr5XTy41DLZqOWje19mlvSWO36lpddb57vJuAD2zcNKv13w ES3kpo69F3arXOTUr0cm2JU950Qkeeoff/jTPSpraoyRgj/cTEnRrj6nCCzGGtJ/A0ax Msig== X-Gm-Message-State: AOJu0YzcWA6a7FqFjr+uPIZxxHmjtxYlZJLtsrLUzCiiiVUWBk+OcPvT dZcposnvBIBeVifT9wBULpyU9UYHQdSHm2EdoYOZQ45im7C2xSEgVSej9SyeKawkqaBs1f3tXA= = X-Received: from wmbd5.prod.google.com ([2002:a05:600c:58c5:b0:487:38f4:9550]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:870e:b0:488:aa33:dc8f with SMTP id 5b1f17b1804b1-488fb84ffb8mr585875005e9.0.1777304146208; Mon, 27 Apr 2026 08:35:46 -0700 (PDT) Date: Mon, 27 Apr 2026 17:34:28 +0200 In-Reply-To: <20260427153416.2103979-17-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260427153416.2103979-17-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=2467; i=ardb@kernel.org; h=from:subject; bh=TVDzWXCizjusDiJt30HNr7P0mL2kBIblXPS9xGhGOSo=; b=owGbwMvMwCVmkMcZplerG8N4Wi2JIfN9E+ehGfdF/QRq/FmuJKZkGvWc/iT0g6Gl2I5tubnAy Y9CG793lLIwiHExyIopsgjM/vtu5+mJUrXOs2Rh5rAygQxh4OIUgIlEqjAyvDcKPHlm+gTZr9FO zowHGPXW3UnPWRRZvm1iz+ZPz5uFVjMyPDzwccZLZ47pM1nk2/aq55zdOYtBz/fIlbs/+xIOXNz +mgUA X-Mailer: git-send-email 2.54.0.rc2.544.gc7ae2d5bb8-goog Message-ID: <20260427153416.2103979-28-ardb+git@google.com> Subject: [PATCH v4 11/15] arm64: mm: Don't abuse memblock NOMAP to check for overlaps From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, Ard Biesheuvel , Ryan Roberts , Anshuman Khandual , Liz Prucka , Seth Jenkins , Kees Cook , Mike Rapoport , David Hildenbrand , Andrew Morton , linux-mm@kvack.org, linux-hardening@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel Now that the DRAM mapping routines respect existing table mappings and contiguous block and page mappings, it is no longer needed to fiddle with the memblock tables to set and clear the NOMAP attribute in order to omit text and rodata when creating the linear map. Instead, map the kernel text and rodata alias first with the desired attributes, so that they will not be remapped later with different attributes when mapping the memblocks. Signed-off-by: Ard Biesheuvel --- arch/arm64/mm/mmu.c | 24 +++++++------------- 1 file changed, 8 insertions(+), 16 deletions(-) diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index 5e2348b15783..1a4b4337d29a 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -1148,12 +1148,15 @@ static void __init map_mem(void) flags |=3D NO_BLOCK_MAPPINGS | NO_CONT_MAPPINGS; =20 /* - * Take care not to create a writable alias for the - * read-only text and rodata sections of the kernel image. - * So temporarily mark them as NOMAP to skip mappings in - * the following for-loop + * Map the linear alias of the [_text, __init_begin) interval + * as non-executable now, and remove the write permission in + * mark_linear_text_alias_ro() above (which will be called after + * alternative patching has completed). This makes the contents + * of the region accessible to subsystems such as hibernate, + * but protects it from inadvertent modification or execution. */ - memblock_mark_nomap(kernel_start, kernel_end - kernel_start); + __map_memblock(kernel_start, kernel_end, pgprot_tagged(PAGE_KERNEL), + flags); =20 /* map all the memory banks */ for_each_mem_range(i, &start, &end) { @@ -1165,17 +1168,6 @@ static void __init map_mem(void) __map_memblock(start, end, pgprot_tagged(PAGE_KERNEL), flags); } - - /* - * Map the linear alias of the [_text, __init_begin) interval - * as non-executable now, and remove the write permission in - * mark_linear_text_alias_ro() below (which will be called after - * alternative patching has completed). This makes the contents - * of the region accessible to subsystems such as hibernate, - * but protects it from inadvertent modification or execution. - */ - __map_memblock(kernel_start, kernel_end, PAGE_KERNEL, 0); - memblock_clear_nomap(kernel_start, kernel_end - kernel_start); } =20 void mark_rodata_ro(void) --=20 2.54.0.rc2.544.gc7ae2d5bb8-goog From nobody Wed Jun 17 06:02:47 2026 Received: from mail-wr1-f73.google.com (mail-wr1-f73.google.com [209.85.221.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F2B5D3D8111 for ; Mon, 27 Apr 2026 15:35:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777304150; cv=none; b=peeK/Dtoat8w61AnOCfauA1QkcB6bBsHt7puDYbQPltzohe2R5Bdm1gBfA1A69pJN63ZjnSKx6NHKKeMpzumR0wzWJVl2Ec8RZnJEeoHzsaFFRfFqVeCTWGPynAb9M9Xr6fwqiUm46Z4nfoWRGJ08/rNTXK2eY8VOv2K4u63BCA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777304150; c=relaxed/simple; bh=xZChKJVKCOn2ZQHxZ9briXztPKj9FQZx9XMBSRmIWus=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=cwhEgYw3Ty6W24G4vFi+fbXTP9kD7E7u2TaUqzv6lsbYvmsgAhjLz7sq0U45Rvgi/KbwNENOkiOGkL00/06tTSw1O1V8jCq08pT9AkmF7yPZUjUOattpX9dEhITjrhbSMC3CW7p1xpZByE9blUmAkJ5d6aDx/o1rwuzvYbXrPz8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=VAATGf4Z; arc=none smtp.client-ip=209.85.221.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="VAATGf4Z" Received: by mail-wr1-f73.google.com with SMTP id ffacd0b85a97d-441243ba35fso5436411f8f.0 for ; Mon, 27 Apr 2026 08:35:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1777304147; x=1777908947; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=UxYPOVRWmcj1klY6tkKd45C/dDkcyiHdqjwOkqhsN/0=; b=VAATGf4ZKf0orp+AVPhpwBCMCjAf+JCA+k/LZtRxiBGjxhIs0Aa17J9NDVnnggjXS9 m0vQRSuEfF2xUkG8D+LGsvmcwt2Dm3tNjhwnFxvwjHlFdCCzQVqm1XfMAjpIW4BnWXji LjtHCTf0H/DUZkYPPXcIm5O0uAAfHde7vhE/Sy8A50mmApAbFM4pYVtqyp/p11UG7IYf 2a/+iJmSHut7eZICyDG97Uqb4Lp8nF4ojWznL2FY9l+eNxXuukjnWcKPgxf0c61RCagw Ex42aZcs8Qy3Q50PwuuwIOycNIj6chPTQPmesRJds7mN2DEIp3NwZqjy/W7Zt5LlGTc2 +LnA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1777304147; x=1777908947; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=UxYPOVRWmcj1klY6tkKd45C/dDkcyiHdqjwOkqhsN/0=; b=iKybA7JwD2wcEmVAC/oa/Oke0eFE8qtfh0h4oD3Cl6HywO8q/O65XNtDrfioVXUD2S w5/OaSdRkC6BAUqJL5vBzGiTUkSPNUBC9LmQWgY41K8dWzbSCsinwlHHkmSsZoeFKHEU ZZWwZbXg2j2M0rwl6Up+QyUb16zpuX42ZtXJUFGivHr8b254wvn0K95ZECXmJhy07Jsq r4BdvTOuKj+TVc3OeRjLVXH9F0CylOgJHTXKLIR00l8QmmRzGyHq5ML13GV6QowBBmtM BQtJq45dzEggUA8RCepFDvtAkXyidz/CCYRjiZzAtSFfx1AqlxJM3H61BjZCVc/g1oVT rLYg== X-Gm-Message-State: AOJu0Yx/vDdhes7VtNz4ouEziOEutCdHloR2GR2s8gJEzF4XRrgj9mX4 SWcbgC3agLnY1BJz0sCcbuDRbPpfIGqAe6TWZQQqVYkaT40fF/5My5o+nHuMWRV8ay5uDs1weQ= = X-Received: from wrsm7.prod.google.com ([2002:adf:fe47:0:b0:43f:e932:b48d]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a5d:5d88:0:b0:43d:6e0:9458 with SMTP id ffacd0b85a97d-43fe3e0c779mr67106901f8f.39.1777304147061; Mon, 27 Apr 2026 08:35:47 -0700 (PDT) Date: Mon, 27 Apr 2026 17:34:29 +0200 In-Reply-To: <20260427153416.2103979-17-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260427153416.2103979-17-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=3111; i=ardb@kernel.org; h=from:subject; bh=vnOPgzR9gwGy7RU3jbNwlQgfBXncMvaaP3oMs7gg07Y=; b=owGbwMvMwCVmkMcZplerG8N4Wi2JIfN9E7fhjbPcgs4XVKfeTnOd6X5d81L1wd6JdtmfgucZt Dn6GzB2lLIwiHExyIopsgjM/vtu5+mJUrXOs2Rh5rAygQxh4OIUgIlM/Mfwz1hYf8184+dKqf8u huwKYeCbfkN1Q8PZ5+f9Qlm8AtrWXmX4Z3804IBu667aItk0A8Wfbn81/Y/98HK+tuNd4pHLyrc 5WQA= X-Mailer: git-send-email 2.54.0.rc2.544.gc7ae2d5bb8-goog Message-ID: <20260427153416.2103979-29-ardb+git@google.com> Subject: [PATCH v4 12/15] arm64: mm: Map the kernel data/bss read-only in the linear map From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, Ard Biesheuvel , Ryan Roberts , Anshuman Khandual , Liz Prucka , Seth Jenkins , Kees Cook , Mike Rapoport , David Hildenbrand , Andrew Morton , linux-mm@kvack.org, linux-hardening@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel On systems where the bootloader adheres to the original arm64 boot protocol, the placement of the kernel in the physical address space is highly predictable, and this makes the placement of its linear alias in the kernel virtual address space equally predictable, given the lack of randomization of the linear map. The linear aliases of the kernel text and rodata regions are already mapped read-only, but the kernel data and bss are mapped read-write in this region. This is not needed, so map them read-only as well. Note that the statically allocated kernel page tables do need to be modifiable via the linear map, so leave these mapped read-write. Signed-off-by: Ard Biesheuvel --- arch/arm64/include/asm/sections.h | 1 + arch/arm64/mm/mmu.c | 16 ++++++++++++++-- 2 files changed, 15 insertions(+), 2 deletions(-) diff --git a/arch/arm64/include/asm/sections.h b/arch/arm64/include/asm/sec= tions.h index 51b0d594239e..32ec21af0823 100644 --- a/arch/arm64/include/asm/sections.h +++ b/arch/arm64/include/asm/sections.h @@ -23,6 +23,7 @@ extern char __irqentry_text_start[], __irqentry_text_end[= ]; extern char __mmuoff_data_start[], __mmuoff_data_end[]; extern char __entry_tramp_text_start[], __entry_tramp_text_end[]; extern char __relocate_new_kernel_start[], __relocate_new_kernel_end[]; +extern char __fixmap_pgdir_start[]; =20 static inline size_t entry_tramp_text_size(void) { diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index 1a4b4337d29a..9361b7efb848 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -1122,7 +1122,9 @@ static void __init map_mem(void) { static const u64 direct_map_end =3D _PAGE_END(VA_BITS_MIN); phys_addr_t kernel_start =3D __pa_symbol(_text); - phys_addr_t kernel_end =3D __pa_symbol(__init_begin); + phys_addr_t init_begin =3D __pa_symbol(__init_begin); + phys_addr_t init_end =3D __pa_symbol(__init_end); + phys_addr_t kernel_end =3D __pa_symbol(__fixmap_pgdir_start); phys_addr_t start, end; int flags =3D NO_EXEC_MAPPINGS; u64 i; @@ -1155,7 +1157,11 @@ static void __init map_mem(void) * of the region accessible to subsystems such as hibernate, * but protects it from inadvertent modification or execution. */ - __map_memblock(kernel_start, kernel_end, pgprot_tagged(PAGE_KERNEL), + __map_memblock(kernel_start, init_begin, pgprot_tagged(PAGE_KERNEL), + flags); + + /* Map the kernel data/bss so it can be remapped later */ + __map_memblock(init_end, kernel_end, pgprot_tagged(PAGE_KERNEL), flags); =20 /* map all the memory banks */ @@ -1168,6 +1174,12 @@ static void __init map_mem(void) __map_memblock(start, end, pgprot_tagged(PAGE_KERNEL), flags); } + + /* Map the kernel data/bss read-only in the linear map */ + __map_memblock(init_end, kernel_end, pgprot_tagged(PAGE_KERNEL_RO), + flags); + flush_tlb_kernel_range((unsigned long)lm_alias(__init_end), + (unsigned long)lm_alias(__fixmap_pgdir_start)); } =20 void mark_rodata_ro(void) --=20 2.54.0.rc2.544.gc7ae2d5bb8-goog From nobody Wed Jun 17 06:02:47 2026 Received: from mail-wm1-f73.google.com (mail-wm1-f73.google.com [209.85.128.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 505253385BE for ; Mon, 27 Apr 2026 15:35:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777304151; cv=none; b=RZzF3JftFO9oTt7T3K6cYUrM+of7X2LrgluGhUDsA7q7EjhDTvB2m5jEz/eqYzHTiEWh3TKimrgqlkaTswYinIaB1S4Ej5wB9wOrsuYpZwCQ1feF02iu7bXW0iruYf9F6R5ibZeHBa9wte5Csrv201FaxU+XhLf/YpbV5e7f1ZU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777304151; c=relaxed/simple; bh=O/Gz5Nv/ULAYPtMULzEkMfXxCcd0WOOKaB94QYkzs4Y=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=G59dJPQHjuSxarKAUNfF1Db/+9AAom9ccuwVUI63kXwQwODgLPkPrHCZwqWIhIDeQbQbZaiQ2/Isfxp6zdBXlbqy4Y/l72YOqb8LNzvUev6cejiWdPZV1Tw3cuMvlKFlOl/4YC+WgOEWbu8BTYR+V7H8pNVeJreRLYWSWkdLbZ4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=i3KQTKFv; arc=none smtp.client-ip=209.85.128.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="i3KQTKFv" Received: by mail-wm1-f73.google.com with SMTP id 5b1f17b1804b1-488d8deb75fso85321505e9.3 for ; Mon, 27 Apr 2026 08:35:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1777304149; x=1777908949; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=LHBfAOq62ddYo5eYH63kmRGKW2W5oXHrS23q0wSzvjk=; b=i3KQTKFve1kA8loUu+2Nm7QkZkI1w4XF99eaYcQo9JClsupShmwTtAjYSJw8tpnzQk ZDgRBCBtzGIcKS+tOXnjaYIYf02d0hi9T1kC9n4s2jYIjsDQUzQKG8LRz8Elg6Y0CYHV rwfZeLKwx3G3uHdDcKWgxBhNXBpFwflUMv/FS1rm3aLk/88Maxv90T4mBlEgIiLPy5/t EusDYULznB0csoVTNNzu463yidxmHO/lu3nz4U9vqJaOe6H1g9BbVW6hjeS3d1VXiisM dLYe7uVpGrpTu/77mPogEcAg3DqPZKvKhsW1iVjpUunNIMLkieQOGeXIJnvCeGXVVFNF Grpw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1777304149; x=1777908949; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=LHBfAOq62ddYo5eYH63kmRGKW2W5oXHrS23q0wSzvjk=; b=jSbR8xIt1GjEjA0wQAaaKthKBXzfS3UA0IR3ESaRDNDxPHz1MFh8pKRaOfrQpAKO/9 fGwy8N6Z4VIEYgyixTO31gvrP8GTWaeaTRhcsMxPI3GrN2sBtO/HtnuzbZJjPqKqBAeD TeUXMv8Psab5Q/eFOE2UeFSfOcWYNa0qkY0clwYGRL4TpXTz0I5C9miB+o3jgXRG6yYZ cKmFSnGoEJTJBuS9nEKrTAK+n0vDMZOo3hVVFXUvfgMz7UgltXdFzjNvlPZdau6cFuMh 3SB0K+uVkJD8TY5udCJdH0Gl0YDJZgxwcJHQnVpyUPq4H4my3I6EwBAGZtW8Ik4NbglF 7K3A== X-Gm-Message-State: AOJu0YxU6Ma7OC3TAXpFx13Dolh3l75Ax+EhJQuLUNmJSUFUVCxivXJ4 TsSVUlcmfMEPlF1qwhRhBRtFllJ7EHcrqLMNzaqSv1IEwiRCUvqrsnKKZoFkk4oS4CLa4GqpyQ= = X-Received: from wmim14.prod.google.com ([2002:a7b:cb8e:0:b0:485:4f4a:bd84]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:628c:b0:48a:52d4:888c with SMTP id 5b1f17b1804b1-48a52d48985mr436500725e9.3.1777304148352; Mon, 27 Apr 2026 08:35:48 -0700 (PDT) Date: Mon, 27 Apr 2026 17:34:30 +0200 In-Reply-To: <20260427153416.2103979-17-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260427153416.2103979-17-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=3345; i=ardb@kernel.org; h=from:subject; bh=wQjR+F+1V7sUzLN4TnEphIKngbWEIlFdlV4UM9366h0=; b=owGbwMvMwCVmkMcZplerG8N4Wi2JIfN9E8/zzp9rzJvuTrrssME56tgdeZGiCxu1/hnmHFEw2 Kx3/PTcjlIWBjEuBlkxRRaB2X/f7Tw9UarWeZYszBxWJpAhDFycAjARdX2Gf7qTeGeyv/WQ3m+5 YJN2SVKJmjhP0Cq12slhs9JOff29JYeR4fLhIoXXDu2TA92k7l89733+pdS0gHk/wh3rGVYs+M3 wjgkA X-Mailer: git-send-email 2.54.0.rc2.544.gc7ae2d5bb8-goog Message-ID: <20260427153416.2103979-30-ardb+git@google.com> Subject: [PATCH v4 13/15] arm64: mm: Unmap kernel data/bss entirely from the linear map From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, Ard Biesheuvel , Ryan Roberts , Anshuman Khandual , Liz Prucka , Seth Jenkins , Kees Cook , Mike Rapoport , David Hildenbrand , Andrew Morton , linux-mm@kvack.org, linux-hardening@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel The linear aliases of the kernel text and rodata are mapped read-only in the linear map as well. Given that the contents of these regions are mostly identical to the version in the loadable image, mapping them read-only and leaving their contents visible is a reasonable hardening measure. Data and bss, however, are now also mapped read-only but the contents of these regions are more likely to contain data that we'd rather not leak. So let's unmap these entirely in the linear map when the kernel is running normally. When going into hibernation or waking up from it, these regions need to be mapped, so map the region initially, and toggle the valid bit so map/unmap the region as needed. Signed-off-by: Ard Biesheuvel --- arch/arm64/mm/mmu.c | 44 ++++++++++++++++---- 1 file changed, 37 insertions(+), 7 deletions(-) diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index 9361b7efb848..a464f3d2d2df 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -24,6 +24,7 @@ #include #include #include +#include #include #include #include @@ -1040,6 +1041,31 @@ static void __init __map_memblock(phys_addr_t start,= phys_addr_t end, end - start, prot, early_pgtable_alloc, flags); } =20 +static void remap_linear_data_alias(bool unmap) +{ + set_memory_valid((unsigned long)lm_alias(__init_end), + (unsigned long)(__fixmap_pgdir_start - __init_end) / PAGE_SIZE, + !unmap); +} + +static int arm64_hibernate_pm_notify(struct notifier_block *nb, + unsigned long mode, void *unused) +{ + switch (mode) { + default: + break; + case PM_POST_HIBERNATION: + case PM_POST_RESTORE: + remap_linear_data_alias(true); + break; + case PM_HIBERNATION_PREPARE: + case PM_RESTORE_PREPARE: + remap_linear_data_alias(false); + break; + } + return 0; +} + void __init mark_linear_text_alias_ro(void) { /* @@ -1048,6 +1074,16 @@ void __init mark_linear_text_alias_ro(void) update_mapping_prot(__pa_symbol(_text), (unsigned long)lm_alias(_text), (unsigned long)__init_begin - (unsigned long)_text, pgprot_tagged(PAGE_KERNEL_RO)); + + remap_linear_data_alias(true); + + if (IS_ENABLED(CONFIG_HIBERNATION)) { + static struct notifier_block nb =3D { + .notifier_call =3D arm64_hibernate_pm_notify + }; + + register_pm_notifier(&nb); + } } =20 #ifdef CONFIG_KFENCE @@ -1162,7 +1198,7 @@ static void __init map_mem(void) =20 /* Map the kernel data/bss so it can be remapped later */ __map_memblock(init_end, kernel_end, pgprot_tagged(PAGE_KERNEL), - flags); + flags | NO_BLOCK_MAPPINGS); =20 /* map all the memory banks */ for_each_mem_range(i, &start, &end) { @@ -1174,12 +1210,6 @@ static void __init map_mem(void) __map_memblock(start, end, pgprot_tagged(PAGE_KERNEL), flags); } - - /* Map the kernel data/bss read-only in the linear map */ - __map_memblock(init_end, kernel_end, pgprot_tagged(PAGE_KERNEL_RO), - flags); - flush_tlb_kernel_range((unsigned long)lm_alias(__init_end), - (unsigned long)lm_alias(__fixmap_pgdir_start)); } =20 void mark_rodata_ro(void) --=20 2.54.0.rc2.544.gc7ae2d5bb8-goog From nobody Wed Jun 17 06:02:47 2026 Received: from mail-wr1-f74.google.com (mail-wr1-f74.google.com [209.85.221.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2A3D43D9044 for ; Mon, 27 Apr 2026 15:35:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777304152; cv=none; b=aWGuAsc0y1mqHinKiv4ZgMMZn+uiKmmCkZX6w1Ag09wWk/JIPEYJ8KJGV/RCgnooa5JSeYn1FgzjOXvub/mTTFw/2yqkwDFiA6oOg3i3nEYMGU5UYiZxhbVeW/Gf/Pqpt7yFL2eHQVYMs491eefMvmF05DkTBQFLHayKPjeX7S4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777304152; c=relaxed/simple; bh=KF0mgBdYxfDGv1ypJG02sC7+sHIEf/dO3mdr8ffAPqg=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=f9Ag/dzBcoITTk4u1EUWADVFR3rJmfDpDMeod5mXzuQbCETZjQyw6yurq9thMOVO75iZy8ovWK3ieIcmd4cAL9eeNhSQGoHBNmGTRq74T0KvS5In3OrEzXHib6kyG11qtQAPgzk0ZJypGXc1H2/LvA08fFFCehHhoahH0DVKfNo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=pC5xsVVM; arc=none smtp.client-ip=209.85.221.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="pC5xsVVM" Received: by mail-wr1-f74.google.com with SMTP id ffacd0b85a97d-440d12a472eso7494130f8f.3 for ; Mon, 27 Apr 2026 08:35:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1777304150; x=1777908950; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=XTH9nHu8rDITPKAJB2BnahRrdCbOr3wfwwaQ1ZOWZK8=; b=pC5xsVVMXaTFdYGEYgAABeSROKlluH0hpFscImBOtva97t3NFf92R2t4B2pQw5e1+2 //C5FwI2nPOwOEWDfhHsHgc2dWx2Z4WDVu6Xmw+02t60SFBJkeK5pQvUuCsIThybL09r 40UviY5Fejt4flXeq+F+++G1YJ+wrz+ml36JK75WyEvFQOMsxj8d+iBoqmnKpR43Aqur rkedbeA9iHsb4G6OWddZ0u4C2BnGsg8n0U9g40bselFPIlYE0jt0qEgyE//CZkLASHQ1 1ozvVO7ZmlY7YjO4oEgrQkstVP1VUAyhjxQGuWcpgfbg1CeTMyRNGfOLaL+g78DiFG+l t4Gw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1777304150; x=1777908950; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=XTH9nHu8rDITPKAJB2BnahRrdCbOr3wfwwaQ1ZOWZK8=; b=MJJnXrRXnAH/4MSuT56rheI+GrzHa6nmmTSlqCqjLRs2EKPQnyrXeeoL1mBsXrfag/ nxxT/jA6/aa8V28fca0YjJtxmrjOCgOVdKulD1UPiSaXOCQ1h8Qg7BWuB8TUrG5vp+oE vYv0BVtDivgn+l9mvIaMh3at2gVRUi0ROLZmBYKleZeB9nZ0jLGZe0qeJL0Clz/2Prur wRvWr/AbB6nhsHhMShleeIfsczbbXgwXAV5IyzoZ8BQmo/yFWsQeQlBsDaEuvdQN/XiF ivIiFCMYVnXVs4VKe3aKsVo9jv+k8LxGqARZkdqCJzffu5gMognl0WoH9jlJUL/vnBGt ZVcg== X-Gm-Message-State: AOJu0Yw3t4/AK63MfyfeygADqYr3pKRBdMJ7an6kmgtyjCycqCO5HDhF SMilftFIxqkP2qHenc1YDm+GVWVftpJvSmHftOI5o6c3Jrm6t+TqMdsd/Defd1VaoLnKT7BOYA= = X-Received: from wmcn9.prod.google.com ([2002:a05:600c:c0c9:b0:485:fb9c:ffa5]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:3546:b0:488:81b1:ae36 with SMTP id 5b1f17b1804b1-488fb7880camr604813405e9.23.1777304149544; Mon, 27 Apr 2026 08:35:49 -0700 (PDT) Date: Mon, 27 Apr 2026 17:34:31 +0200 In-Reply-To: <20260427153416.2103979-17-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260427153416.2103979-17-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=5803; i=ardb@kernel.org; h=from:subject; bh=YSdu4q+PtPsInD53oLl+3uBw5SBzUVoUd7FzWVZ14jY=; b=owGbwMvMwCVmkMcZplerG8N4Wi2JIfN9E5+Lo4bOq9KZm6o4L8u/vb/hydxl+fVRaYE/zvc9Y VmglsHQUcrCIMbFICumyCIw+++7nacnStU6z5KFmcPKBDKEgYtTACYix8jIsKnEu41r6pMdV/e7 tUrq7Z357SWjW05Bks+rK6s47Vdea2ZkmLNng5zCxX8/rKfelTlnZcUXtfa71MttQVn9jCEdZr+ LeQA= X-Mailer: git-send-email 2.54.0.rc2.544.gc7ae2d5bb8-goog Message-ID: <20260427153416.2103979-31-ardb+git@google.com> Subject: [PATCH v4 14/15] arm64: mm: Generalize manipulation code of read-only descriptors From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, Ard Biesheuvel , Ryan Roberts , Anshuman Khandual , Liz Prucka , Seth Jenkins , Kees Cook , Mike Rapoport , David Hildenbrand , Andrew Morton , linux-mm@kvack.org, linux-hardening@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel Before moving the fixmap PUD/PMD tables into .rodata, update the existing descriptor manipulation code so it will fallback to the fixmap for any descriptor located in the .pgdir_rodata section. This is slightly more costly, as it evaluates whether or not a descriptor is in the kernel's rodata region at levels PMD and higher for any configuration, rather than only when the level in question is the root level. Signed-off-by: Ard Biesheuvel --- arch/arm64/include/asm/pgtable.h | 27 ++++++++++---------- arch/arm64/kernel/vmlinux.lds.S | 8 ++++-- arch/arm64/mm/mmu.c | 24 ++++++++--------- 3 files changed, 31 insertions(+), 28 deletions(-) diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgta= ble.h index a1c5894332d9..94235dd428be 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -816,23 +816,22 @@ extern pgd_t swapper_pg_dir[]; extern pgd_t idmap_pg_dir[]; extern pgd_t tramp_pg_dir[]; extern pgd_t reserved_pg_dir[]; +extern pgd_t __pgdir_rodata_start[], __pgdir_rodata_end[]; =20 -extern void set_swapper_pgd(pgd_t *pgdp, pgd_t pgd); +extern void set_rodata_pte(pte_t *ptep, pte_t pte); =20 -static inline bool in_swapper_pgdir(void *addr) +static inline bool in_pgdir_rodata(void *addr) { - return ((unsigned long)addr & PAGE_MASK) =3D=3D - ((unsigned long)swapper_pg_dir & PAGE_MASK); + return addr >=3D (void *)__pgdir_rodata_start && + addr < (void *)__pgdir_rodata_end; } =20 static inline void set_pmd(pmd_t *pmdp, pmd_t pmd) { -#ifdef __PAGETABLE_PMD_FOLDED - if (in_swapper_pgdir(pmdp)) { - set_swapper_pgd((pgd_t *)pmdp, __pgd(pmd_val(pmd))); + if (in_pgdir_rodata(pmdp)) { + set_rodata_pte((pte_t *)pmdp, __pte(pmd_val(pmd))); return; } -#endif /* __PAGETABLE_PMD_FOLDED */ =20 WRITE_ONCE(*pmdp, pmd); =20 @@ -893,8 +892,8 @@ static inline bool pgtable_l4_enabled(void); =20 static inline void set_pud(pud_t *pudp, pud_t pud) { - if (!pgtable_l4_enabled() && in_swapper_pgdir(pudp)) { - set_swapper_pgd((pgd_t *)pudp, __pgd(pud_val(pud))); + if (in_pgdir_rodata(pudp)) { + set_rodata_pte((pte_t *)pudp, __pte(pud_val(pud))); return; } =20 @@ -974,8 +973,8 @@ static inline bool mm_pud_folded(const struct mm_struct= *mm) =20 static inline void set_p4d(p4d_t *p4dp, p4d_t p4d) { - if (in_swapper_pgdir(p4dp)) { - set_swapper_pgd((pgd_t *)p4dp, __pgd(p4d_val(p4d))); + if (in_pgdir_rodata(p4dp)) { + set_rodata_pte((pte_t *)p4dp, __pte(p4d_val(p4d))); return; } =20 @@ -1102,8 +1101,8 @@ static inline bool mm_p4d_folded(const struct mm_stru= ct *mm) =20 static inline void set_pgd(pgd_t *pgdp, pgd_t pgd) { - if (in_swapper_pgdir(pgdp)) { - set_swapper_pgd(pgdp, __pgd(pgd_val(pgd))); + if (in_pgdir_rodata(pgdp)) { + set_rodata_pte((pte_t *)pgdp, __pte(pgd_val(pgd))); return; } =20 diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.ld= s.S index 2dca18574619..e5e1d0fd7f27 100644 --- a/arch/arm64/kernel/vmlinux.lds.S +++ b/arch/arm64/kernel/vmlinux.lds.S @@ -243,8 +243,12 @@ SECTIONS reserved_pg_dir =3D .; . +=3D PAGE_SIZE; =20 - swapper_pg_dir =3D .; - . +=3D PAGE_SIZE; + .pgdir_rodata : { + __pgdir_rodata_start =3D .; + swapper_pg_dir =3D .; + . +=3D PAGE_SIZE; + __pgdir_rodata_end =3D .; + } =20 . =3D ALIGN(SEGMENT_ALIGN); __init_begin =3D .; diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index a464f3d2d2df..84d81bae07a7 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -65,34 +65,34 @@ static bool rodata_is_rw __ro_after_init =3D true; */ long __section(".mmuoff.data.write") __early_cpu_boot_status; =20 -static DEFINE_SPINLOCK(swapper_pgdir_lock); +static DEFINE_SPINLOCK(rodata_pgdir_lock); static DEFINE_MUTEX(fixmap_lock); =20 -void noinstr set_swapper_pgd(pgd_t *pgdp, pgd_t pgd) +void noinstr set_rodata_pte(pte_t *ptep, pte_t pte) { - pgd_t *fixmap_pgdp; + pte_t *fixmap_ptep; =20 /* - * Don't bother with the fixmap if swapper_pg_dir is still mapped - * writable in the kernel mapping. + * Don't bother with the fixmap if rodata is still mapped + * writable in the kernel and linear mappings. */ if (rodata_is_rw) { - WRITE_ONCE(*pgdp, pgd); + WRITE_ONCE(*ptep, pte); dsb(ishst); isb(); return; } =20 - spin_lock(&swapper_pgdir_lock); - fixmap_pgdp =3D pgd_set_fixmap(__pa_symbol(pgdp)); - WRITE_ONCE(*fixmap_pgdp, pgd); + spin_lock(&rodata_pgdir_lock); + fixmap_ptep =3D pte_set_fixmap(__pa_nodebug(ptep)); + WRITE_ONCE(*fixmap_ptep, pte); /* * We need dsb(ishst) here to ensure the page-table-walker sees * our new entry before set_p?d() returns. The fixmap's * flush_tlb_kernel_range() via clear_fixmap() does this for us. */ - pgd_clear_fixmap(); - spin_unlock(&swapper_pgdir_lock); + pte_clear_fixmap(); + spin_unlock(&rodata_pgdir_lock); } =20 pgprot_t phys_mem_access_prot(struct file *file, unsigned long pfn, @@ -1071,6 +1071,7 @@ void __init mark_linear_text_alias_ro(void) /* * Remove the write permissions from the linear alias of .text/.rodata */ + WRITE_ONCE(rodata_is_rw, false); update_mapping_prot(__pa_symbol(_text), (unsigned long)lm_alias(_text), (unsigned long)__init_begin - (unsigned long)_text, pgprot_tagged(PAGE_KERNEL_RO)); @@ -1221,7 +1222,6 @@ void mark_rodata_ro(void) * to cover NOTES and EXCEPTION_TABLE. */ section_size =3D (unsigned long)__init_begin - (unsigned long)__start_rod= ata; - WRITE_ONCE(rodata_is_rw, false); update_mapping_prot(__pa_symbol(__start_rodata), (unsigned long)__start_r= odata, section_size, PAGE_KERNEL_RO); /* mark the range between _text and _stext as read only. */ --=20 2.54.0.rc2.544.gc7ae2d5bb8-goog From nobody Wed Jun 17 06:02:47 2026 Received: from mail-wm1-f73.google.com (mail-wm1-f73.google.com [209.85.128.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5B29B3D9DBD for ; Mon, 27 Apr 2026 15:35:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777304154; cv=none; b=u1AsOHor6sPOIg9PnAnKIH2s6MwcFtpcmStxz/U2eTaQXHkIIrt++ch1DuGIsDQQeeVbEONZywqwGeBe+02wnZO7KieebhoFg0KnfibRike5XcHrsAQ+DZRFky4yu88guD/WgLxKDZnVsepogUPPzJhuNJYceFritanCe20O4mU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777304154; c=relaxed/simple; bh=3xiY6SGTFT0XKPbS48ai+rC6RTmOW0yJHBsDIHf0M2Y=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=srLTo1qmA9liQKYdo8ynDrztzfi6eGNXtYpvrF4Gnr13i5jN28mWzRY6VdcIVIVkKLjfceeUxMZpjEVP2fg9p4/x5oeCgWb410QoSneJX75UZ6XaoTCVLgLyb3H9qjI+7OfrF9FOj/4MP12Gh9bHT8LOWeoTuJbG4oKusczlYek= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=j7bSlVgP; arc=none smtp.client-ip=209.85.128.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="j7bSlVgP" Received: by mail-wm1-f73.google.com with SMTP id 5b1f17b1804b1-4837b6f6b93so111416695e9.3 for ; Mon, 27 Apr 2026 08:35:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1777304151; x=1777908951; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=DT4zvpGacxHRk/WPHA3P347efnejtMNVhL3FmZpnqUQ=; b=j7bSlVgP/KuNbKNcqcWBHw4UauesNn53/2TzWNjnjM+7atfPbKfOO4g+fFi0SCKSFE lD9uUAwdmJ3NMB9KlQhKQt52O/e/OuYYeCM9H+mg6hNdvPDF1b50MXuarZlZxa35v88l IBf8Ngez32pknX6q4cEMsO81niQQ18s4uErR27mkn8bAjYWYXfGZtq2JaH4BIOU9sO3P wwn8QN6AbySH+bjaOSjMW7yy1LzlqjJaYbiS8Us8xBrPtzIdc7IKNJ27O4a52YRdDRC9 ovzgCVTMUmVHbDjnESSz66wFARdBq59WrpuHgh0uxIAsMehDZKnYsn3BI4EdcJ2oeoSy Cqxw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1777304151; x=1777908951; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=DT4zvpGacxHRk/WPHA3P347efnejtMNVhL3FmZpnqUQ=; b=rS71LHLU/xQK44IQ4BuyVt3g95OJ2eLqhR2bydcgeSkdpXivjf5qYGjjFXdPTw/izR xkFiJpYHaBZr8l/Frba1ptloUpWH0kmrI56L7obxs+cJPrdNgG9L50e4kulv5OugGkHQ DOXfc/tWajb2DbDGErzLKidc9Uax900h7EaOmpOEZ3XchBxKGDW4bvFfsE5JV5JHWp7V 8RPJwwcVqJrw/JFRBV80bTaNNEKfDgUh+d8xlnj9NBt9yYRhiXx5DI/EA2uHhA9qLcnM ZGAcgp0IjeGGOJCI/VDp0d6GIgs6Nq3Uun+mn6fnlC3ERr3UcbkynuEd35pBsxNlgIM4 Ltpw== X-Gm-Message-State: AOJu0YxGeObZfY/dsbdKbMbqnkJANk8sKhdY+aoISrXDZA33pO4QUAEH fbzKiTbITWnP/7rnAwuiZo+St9ucZxl9idyO9wYkOM39zfCOicAUg3l4I+1x59jm99SXlty2gA= = X-Received: from wmbdn19.prod.google.com ([2002:a05:600c:6553:b0:48a:55e6:d5c4]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:888b:b0:488:c40b:c8bf with SMTP id 5b1f17b1804b1-488fb73d234mr529149015e9.2.1777304150888; Mon, 27 Apr 2026 08:35:50 -0700 (PDT) Date: Mon, 27 Apr 2026 17:34:32 +0200 In-Reply-To: <20260427153416.2103979-17-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260427153416.2103979-17-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=3660; i=ardb@kernel.org; h=from:subject; bh=5/WaFTKUieGOECS5oMgCFK1ktF6MkiLzxXcp+K6DIKQ=; b=owGbwMvMwCVmkMcZplerG8N4Wi2JIfN9E3/1lYfbJ77dJSXx5Ty7SW+1FL9kwrFd8h9mz9lz8 P9LxbYTHaUsDGJcDLJiiiwCs/++23l6olSt8yxZmDmsTCBDGLg4BWAibycx/M9OuztxTuOemrh5 3P/U/VdeN3wn/f+WxmsXu7Ztgtv4VhYz/HfpvK1unhEare1hsj3hZmaG3YWulTsrSjxvCd3xnTH xNxMA X-Mailer: git-send-email 2.54.0.rc2.544.gc7ae2d5bb8-goog Message-ID: <20260427153416.2103979-32-ardb+git@google.com> Subject: [PATCH v4 15/15] arm64: mm: Remap linear aliases of the fixmap page tables read-only From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, Ard Biesheuvel , Ryan Roberts , Anshuman Khandual , Liz Prucka , Seth Jenkins , Kees Cook , Mike Rapoport , David Hildenbrand , Andrew Morton , linux-mm@kvack.org, linux-hardening@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel The fixmap page tables are statically allocated, and are currently mapped read-write both in the kernel mapping as well as its linear alias. Due to lack of randomization of the linear map, these tables will appear at a priori known offsets in the virtual address space when booting without physical randomization, which means that a single kernel write primitive is sufficient for an attacker to map memory of their own choosing with any permissions at a known virtual address in the kernel's address space. To harden against this, move the fixmap PUD and PMD tables to .pgdir_rodata, so that both their kernel mappings as well as their linear aliases are mapped read-only during ordinary execution. The PTE table needs to remain read-write accessible via the kernel mapping, but its linear alias can be remapped read-only as well. Signed-off-by: Ard Biesheuvel --- arch/arm64/include/asm/pgtable.h | 6 ++++-- arch/arm64/kernel/vmlinux.lds.S | 1 + arch/arm64/mm/fixmap.c | 5 +++-- arch/arm64/mm/mmu.c | 5 +++++ 4 files changed, 13 insertions(+), 4 deletions(-) diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgta= ble.h index 94235dd428be..21afe923cd71 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -822,8 +822,10 @@ extern void set_rodata_pte(pte_t *ptep, pte_t pte); =20 static inline bool in_pgdir_rodata(void *addr) { - return addr >=3D (void *)__pgdir_rodata_start && - addr < (void *)__pgdir_rodata_end; + phys_addr_t pa =3D __pa_nodebug(addr); + + return pa >=3D __pa_symbol_nodebug(__pgdir_rodata_start) && + pa < __pa_symbol_nodebug(__pgdir_rodata_end); } =20 static inline void set_pmd(pmd_t *pmdp, pmd_t pmd) diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.ld= s.S index e5e1d0fd7f27..9b346dd24d1c 100644 --- a/arch/arm64/kernel/vmlinux.lds.S +++ b/arch/arm64/kernel/vmlinux.lds.S @@ -247,6 +247,7 @@ SECTIONS __pgdir_rodata_start =3D .; swapper_pg_dir =3D .; . +=3D PAGE_SIZE; + *(.fixmap_rodata) __pgdir_rodata_end =3D .; } =20 diff --git a/arch/arm64/mm/fixmap.c b/arch/arm64/mm/fixmap.c index b649ea1a46e4..ad6d46e5c23e 100644 --- a/arch/arm64/mm/fixmap.c +++ b/arch/arm64/mm/fixmap.c @@ -32,9 +32,10 @@ static_assert(NR_BM_PMD_TABLES =3D=3D 1); #define BM_PTE_TABLE_IDX(addr) __BM_TABLE_IDX(addr, PMD_SHIFT) =20 #define __fixmap_bss __section(".fixmap_bss") __aligned(PAGE_SIZE) +#define __fixmap_rodata __section(".fixmap_rodata") __aligned(PAGE_SIZE) static pte_t bm_pte[NR_BM_PTE_TABLES][PTRS_PER_PTE] __fixmap_bss; -static pmd_t bm_pmd[PTRS_PER_PMD] __fixmap_bss __maybe_unused; -static pud_t bm_pud[PTRS_PER_PUD] __fixmap_bss __maybe_unused; +static pmd_t bm_pmd[PTRS_PER_PMD] __fixmap_rodata __maybe_unused; +static pud_t bm_pud[PTRS_PER_PUD] __fixmap_rodata __maybe_unused; =20 static inline pte_t *fixmap_pte(unsigned long addr) { diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index 84d81bae07a7..e76fe5b0c5fe 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -1076,6 +1076,11 @@ void __init mark_linear_text_alias_ro(void) (unsigned long)__init_begin - (unsigned long)_text, pgprot_tagged(PAGE_KERNEL_RO)); =20 + /* Map the fixmap PTE table at __fixmap_pgdir_start R/O in linear map too= */ + update_mapping_prot(__pa_symbol(__fixmap_pgdir_start), + (unsigned long)lm_alias(__fixmap_pgdir_start), + PAGE_SIZE, pgprot_tagged(PAGE_KERNEL_RO)); + remap_linear_data_alias(true); =20 if (IS_ENABLED(CONFIG_HIBERNATION)) { --=20 2.54.0.rc2.544.gc7ae2d5bb8-goog