From nobody Mon Sep 28 02:56:56 2026 Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B0D1A4503FA for ; Thu, 27 Aug 2026 16:44:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.70 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787849075; cv=none; b=uzF9YTJjx8uRx1nI31pmjLDC82RjTS5yUyxQeFCPIv24zjKX3OsKrGN4Ov9la28Y8oa/1X5idUsQdlqAAc7AIQNFOqJ8zbHcMU7xSliiBr/8U9yQiqH/D2JfGEPlE2XgCKDj3MJlU92K79Bhljq41p9hks01ODpk85ssAUXtTRQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787849075; c=relaxed/simple; bh=oNUu7Uy4qCZ7ywxZ27PZJHygQFjkapl5eRuKDQ/hMt0=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=XLyjnt6NSJ1th176+9F29zpPQ/pB13vK7d4gabN91ACQmWErA6B5+9P2A6XOxHShmRq/p7h0VZ4br0UraUjfCev4CYWcS66FMttHv5H6E+MQgZbGWp14saN0GvDVwYdKimkHAw5aQJQg3ZsTnlIaL6WeEM2gg/R80aNUock/GnQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=ni4rf7L/; arc=none smtp.client-ip=209.85.128.70 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="ni4rf7L/" Received: by mail-wm1-f70.google.com with SMTP id 5b1f17b1804b1-49b7c1dc61eso6311575e9.3 for ; Thu, 27 Aug 2026 09:44:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1787849072; x=1788453872; darn=vger.kernel.org; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:from:to:cc:subject:date:message-id:reply-to :content-type; bh=7CyCUbo8ckePbD8MMp0AVVWrvzOSsuGmEZvYz1btlf4=; b=ni4rf7L/PCHkxUaxiv3h9g1eFgH6vz3mU+ccXB13M7gyg0eatedBonTZfBBHZYTzu5 jIuN/HG4eClDHfdVIj6LXwzHuMH+Yvm+ehpM33VdoFOm6bjYNXI9KSiuOnZZv04w+ZWS DO2z9MCo0h7rzSoHz/envN41Hazl31wv4NOoPyXqf96dBcLRBV7Z6XvHcICYXAhnR6ZZ TTl66eb9OriC3o3t/rclufiDPzN5cXeLdYYd/EOt4Zd6ORf+X01j0chaiFJHb/7l5/Fd fe2NOGYaXTCRmNSaVLdAj3p4ECUIyM9uPH56hSAEz35ol6MLSXtNPD1AKY/quqxGbwdy GBTA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1787849072; x=1788453872; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to:content-type; bh=7CyCUbo8ckePbD8MMp0AVVWrvzOSsuGmEZvYz1btlf4=; b=R7c21777vkH1YgbN2f+ev1pmp6ncz8L/PRSfgSspXqRK+Ho3qY0MfRo9Ad8PozL/tP CIgRLgIg6W69BCD47HOt36Rt5ncvS2QjHqFQxkQ422MSnoZiK98DkyVAxBnIg7N+pS8f tcE3VJaFXVN2H/MW5mZWepAsgJHuIsdPl55OzyJnAZd/gWISg80No2uycv0qaGrx8vVk k9owJsdb1JOMicIr7R48h8OhoMPFVGb7OLyuPmH0Y5Nt4w/VBY/metkiNdpwO1JSJkhU +zwuxgTfMJPchu7RxU8gxhVt6lum/vUOqycxexGoj1o1vN7DievscDYrD7HLjMlH8jo6 OxPA== X-Gm-Message-State: AFuF++kpKprAcbeazM/CcRAj13rj6L/kcW7Y5am+D17nBQC7bDJ015n4 4TgwF4b+8zeFhWEWI44hZ7WGzce330lh0L3YVS8YFce9R6/ZAAM46kKQjYH2qv0Ibt9mZmYGaRk RhogbYYFMv/yU/AxGIU3A8PaeuQ9NoVmbcmoZoCaTMmcqndC/EsNn1HwyZFkSFvZZLVF8Pvpghw Yytb4gNwQqBlbiubpphR5wjLJTkXzYc7wY+Q== X-Received: from wmbjq10.prod.google.com ([2002:a05:600c:55ca:b0:499:c422:dc65]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:4445:b0:493:e451:a9e1 with SMTP id 5b1f17b1804b1-49b91c2e150mr4552335e9.2.1787849071663; Thu, 27 Aug 2026 09:44:31 -0700 (PDT) Date: Thu, 27 Aug 2026 18:44:11 +0200 In-Reply-To: <20260827164409.3421848-6-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260827164409.3421848-6-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=4080; i=ardb@kernel.org; h=from:subject; bh=Ad7xQcsBExdRIPTIoeDmAPV3aD58c+p2kqv58VFSn9Q=; b=owGbwMvMwCVmkMcZplerG8N4Wi2JIWtCZnTXg8WttnEv102d8l2FK3Rr/9F9MrOac9mmRRbMN zh05synjlIWBjEuBlkxRRaB2X/f7Tw9UarWeZYszBxWJpAhDFycAjCR3R8ZGV4HN6r+1Xnn9vzl 15/TlX6wzE6bVrRp4+Z9YhOcTDXmOQoyMhyK2fL8PvNT05AXrElL6hwOTl8j+v/Lg3WbrnpmpMh ttOEFAA== X-Mailer: git-send-email 2.55.0.897.gb25b4bd76c-goog Message-ID: <20260827164409.3421848-7-ardb+git@google.com> Subject: [RFC PATCH v2 1/4] arm64: mm: Map fixmap PTE tables r/o in the linear map From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: linux-arm-kernel@lists.infradead.org, Ard Biesheuvel Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel Without physical KASLR, the fixmap page tables will appear at an a priori known offset in the physical address space, and due to the lack of randomization, the linear map carries a writeable alias of the fixmap PTE pages, which appears at an offset in the kernel VA space that is also predictable. Given that the placement of the fixmap area is never randomized either, a single store to this linear alias region is sufficient to map any physical page with any permissions at a known offset in the kernel VA space, including on top of the PTI trampoline. Avoid this, by remapping the fixmap PTE pages read-only in the linear map. This is possible because all updates to bm_pte[] occur via the mapping of the kernel image in the vmap area. A read-only mapping is still needed for things like ptdump that walk the page tables. Signed-off-by: Ard Biesheuvel --- arch/arm64/include/asm/fixmap.h | 3 +++ arch/arm64/mm/fixmap.c | 8 +++++--- arch/arm64/mm/mmu.c | 8 ++++++++ 3 files changed, 16 insertions(+), 3 deletions(-) diff --git a/arch/arm64/include/asm/fixmap.h b/arch/arm64/include/asm/fixma= p.h index 170c3502d723..9191125738e9 100644 --- a/arch/arm64/include/asm/fixmap.h +++ b/arch/arm64/include/asm/fixmap.h @@ -112,6 +112,9 @@ enum fixed_addresses { =20 void __init early_fixmap_init(void); =20 +extern pte_t fixmap_bm_pte[][PTRS_PER_PTE]; +extern const size_t fixmap_bm_pte_size; + #define __early_set_fixmap __set_fixmap =20 extern void __set_fixmap(enum fixed_addresses idx, phys_addr_t phys, pgpro= t_t prot); diff --git a/arch/arm64/mm/fixmap.c b/arch/arm64/mm/fixmap.c index f66a0016dd02..3a8cf6de6a7d 100644 --- a/arch/arm64/mm/fixmap.c +++ b/arch/arm64/mm/fixmap.c @@ -31,13 +31,15 @@ static_assert(NR_BM_PMD_TABLES =3D=3D 1); =20 #define BM_PTE_TABLE_IDX(addr) __BM_TABLE_IDX(addr, PMD_SHIFT) =20 -static pte_t bm_pte[NR_BM_PTE_TABLES][PTRS_PER_PTE] __bss_pgtbl; +pte_t fixmap_bm_pte[NR_BM_PTE_TABLES][PTRS_PER_PTE] __bss_pgtbl; static pmd_t bm_pmd[PTRS_PER_PMD] __bss_pgtbl __maybe_unused; static pud_t bm_pud[PTRS_PER_PUD] __bss_pgtbl __maybe_unused; =20 +const size_t fixmap_bm_pte_size =3D sizeof(fixmap_bm_pte); + static inline pte_t *fixmap_pte(unsigned long addr) { - return &bm_pte[BM_PTE_TABLE_IDX(addr)][pte_index(addr)]; + return &fixmap_bm_pte[BM_PTE_TABLE_IDX(addr)][pte_index(addr)]; } =20 static void __init early_fixmap_init_pte(pmd_t *pmdp, unsigned long addr) @@ -46,7 +48,7 @@ static void __init early_fixmap_init_pte(pmd_t *pmdp, uns= igned long addr) pte_t *ptep; =20 if (pmd_none(pmd)) { - ptep =3D bm_pte[BM_PTE_TABLE_IDX(addr)]; + ptep =3D fixmap_bm_pte[BM_PTE_TABLE_IDX(addr)]; __pmd_populate(pmdp, __pa_symbol(ptep), PMD_TYPE_TABLE | PMD_TABLE_AF); } diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index 79d90226fd5d..9c1aa838e9d5 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -1184,6 +1184,7 @@ static void __init map_mem(void) phys_addr_t init_begin =3D __pa_symbol(__init_begin); phys_addr_t init_end =3D __pa_symbol(__init_end); phys_addr_t kernel_end =3D __pa_symbol(__bss_stop); + phys_addr_t fixmap_pte_base =3D __pa_symbol(fixmap_bm_pte); phys_addr_t start, end; int flags =3D NO_EXEC_MAPPINGS; u64 i; @@ -1225,6 +1226,9 @@ static void __init map_mem(void) __map_memblock(init_end, kernel_end, pgprot_tagged(PAGE_KERNEL), flags); =20 + __map_memblock(fixmap_pte_base, fixmap_pte_base + fixmap_bm_pte_size, + pgprot_tagged(PAGE_KERNEL), flags); + /* map all the memory banks */ for_each_mem_range(i, &start, &end) { /* @@ -1268,6 +1272,10 @@ void mark_rodata_ro(void) (unsigned long)_stext - (unsigned long)_text, PAGE_KERNEL_RO); =20 + update_mapping_prot(__pa_symbol(fixmap_bm_pte), + (unsigned long)lm_alias(fixmap_bm_pte), + fixmap_bm_pte_size, PAGE_KERNEL_RO); + /* Map the kernel data/bss as invalid in the linear map */ mark_linear_data_alias_valid(false); } --=20 2.55.0.887.g758fc8c411-goog From nobody Mon Sep 28 02:56:56 2026 Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BD5FE48C8D7 for ; Thu, 27 Aug 2026 16:44:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.72 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787849076; cv=none; b=tJNuhAAIzKWd5nwY1S3FrIJRUCqVwfng0jIIXcCn97RawENpRFvQMDyw5Mcs7rKRuR8W6zv9Hb+ECIELziDY2gF1sOrYjFcnTA0o7Z98SSwlCKXMUmJOzAqUCdOE5/7BThrB8878zDoIX8MfLvFi4rXK4fqJ+eNl+uA4lBjAigA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787849076; c=relaxed/simple; bh=mis7t5NiiX9Q0gCF98X4pR5PBaFhXL0ylYMufFT8C+Q=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=g9vK7SwNxUX0NQXjAAGlT2OPXjYWQRV11QK1MWrT6pdks9ROBbJbEFM1uZqePPgNOcE9C8Lj1Nbk2O5qeBxwREecsgzrIAqQIh6laujsZ+Uj5X6ljOPAvNJaJTFmgYJ4rjcVeGgbcELwSJ/DFsVtCRSBM5EINqb8Z5oyzH090jM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=nD1HIwLc; arc=none smtp.client-ip=209.85.221.72 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="nD1HIwLc" Received: by mail-wr1-f72.google.com with SMTP id ffacd0b85a97d-482f41ca436so20218f8f.1 for ; Thu, 27 Aug 2026 09:44:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1787849073; x=1788453873; darn=vger.kernel.org; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:from:to:cc:subject:date:message-id:reply-to :content-type; bh=unrQEc9LN1JZQh7e5YkkcFIaqqsfhvS1fmOT/8/PD+U=; b=nD1HIwLcMYg1PFAyi2WpKpScfuWujCkfNAyLu7Sxh1a55XCN+ePfDQsaDdb0GGRSDj wOFeRU2yxOytWVRQaEoyWLJFufRgRNlpU1c1M74jtq0QQMUZi6Sh2WjhDP9YJRV1MF7A 2IVJlxTENiDwQyk5dnS6Nj3lGnoMfysBZPk2wPvjJqh3Ont7TQN10VoAQkuYox+8Onus gKklUKhYnGsf/7h2AjrCRvD6wKzf7VDIX7BcKINTaYFcGBWq//8MeeUcBsROLqIssCih 66Fp7AnOkEz+uwGiVtegUubf+onloWu2ZnQzXpIPESoGsJYtxxMGB8xTq/6ja9fU3ziD sUSA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1787849073; x=1788453873; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to:content-type; bh=unrQEc9LN1JZQh7e5YkkcFIaqqsfhvS1fmOT/8/PD+U=; b=OE3yp99p2c+s34R5ME+GEo/1n3JasplmRCYu6bEzswk0Sv2al7fONpF0si/W4szKmm vA97fhyBuzFSUp3CeYrbSTwenWBVs48/ydQznk9/S9NkMXkLQ71+MBk52tqH6JUsZqmy Zaf+MGbyvBL8XTbXWu0dyOBn1FSuspymOqNJiJFFgZG3Wmz9mpXa5vBcRA1b7Z7W0jn0 bSbwHXPoaTiM7KJ29wYlR/OwOlswDovezZ3ki6aWQE7zY0TmGTU/SidgH8JP1kYaAPLg csg2qylW+JdtGa/TC6vBgTDSqtGfrC758+ED4hAg7VoCiVXUshicdem/wZacMJgWOwWf SqIA== X-Gm-Message-State: AFuF++kDopApnHJ9KOQU1aVteCOI3CoWNRGJY6IBDWulhVrH80Z5Txev LG49hNKCkjMKgBZ964D3hbXaNDhGgPk5sPCvt4Tam5ljqHNKvXbBBnPI/bb3YatySGLR1S8ZcOh mwAFLlNiYa7K75K47x98fEQ1GiqRhiTVUTGr4HdwfPMLGehpDWoa7JnvCUoSEg3cPrMF7B4nkhv phCuSFKhcA6t/BlzYa/9pUMItSOYg2N6hcYg== X-Received: from wmlc16.prod.google.com ([2002:a7b:c850:0:b0:495:5edc:795b]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:1f8d:b0:499:621a:2ec2 with SMTP id 5b1f17b1804b1-49b91c17c41mr4617635e9.3.1787849072790; Thu, 27 Aug 2026 09:44:32 -0700 (PDT) Date: Thu, 27 Aug 2026 18:44:12 +0200 In-Reply-To: <20260827164409.3421848-6-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260827164409.3421848-6-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=5363; i=ardb@kernel.org; h=from:subject; bh=IXwvq/N/tjoj1rBZtZdDN7mLt+hNS7LMJFa0rm3k0+Y=; b=owGbwMvMwCVmkMcZplerG8N4Wi2JIWtCZkzKm9Xbn65Wvf1HVXO2m+/z5fPaL7zrsHy43dJ5w ddHJXkzO0pZGMS4GGTFFFkEZv99t/P0RKla51myMHNYmUCGMHBxCsBEjEoY/qk95OHUKhcw3B27 Rl9VrelVyptFld+fWJ62rI/wquyJf8PIsKV41pJtfLPmrCkyO/FOQzDky5EoqdVnymPfdMWcS5s Uzg4A X-Mailer: git-send-email 2.55.0.897.gb25b4bd76c-goog Message-ID: <20260827164409.3421848-8-ardb+git@google.com> Subject: [RFC PATCH v2 2/4] arm64: mm: Use fault handler to permit swapper_pg_dir updates From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: linux-arm-kernel@lists.infradead.org, Ard Biesheuvel Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel Instead of checking the destination of a page table descriptor update against the virtual address of swapper_pg_dir in the kernel image, and taking a dedicated path to perform such updates via a fixmap alias, rely on a fault handler to do so if the plain store faulted. This is more efficient for the general case, and makes it more straight-forward to extend this logic to other page tables that may have been allocated statically in the .rodata region of the kernel image. Signed-off-by: Ard Biesheuvel --- arch/arm64/include/asm/fixmap.h | 1 + arch/arm64/include/asm/pgtable.h | 49 +++++++++----------- arch/arm64/mm/mmu.c | 18 +++---- 3 files changed, 32 insertions(+), 36 deletions(-) diff --git a/arch/arm64/include/asm/fixmap.h b/arch/arm64/include/asm/fixma= p.h index 9191125738e9..8db791dcb4cb 100644 --- a/arch/arm64/include/asm/fixmap.h +++ b/arch/arm64/include/asm/fixmap.h @@ -99,6 +99,7 @@ enum fixed_addresses { FIX_PUD, FIX_P4D, FIX_PGD, + FIX_PTVAL, =20 __end_of_fixed_addresses }; diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgta= ble.h index 6000905a2e86..799cd9d52c28 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -35,6 +35,7 @@ #include #include #include +#include #include #include #include @@ -814,7 +815,7 @@ extern pgd_t idmap_pg_dir[]; extern pgd_t tramp_pg_dir[]; extern pgd_t reserved_pg_dir[]; =20 -extern void set_swapper_pgd(pgd_t *pgdp, pgd_t pgd); +void try_set_readonly_ptval(ptval_t *ptvalp, ptval_t ptval); =20 static inline bool in_swapper_pgdir(void *addr) { @@ -824,17 +825,14 @@ static inline bool in_swapper_pgdir(void *addr) =20 static inline void set_pmd(pmd_t *pmdp, pmd_t pmd) { -#ifdef __PAGETABLE_PMD_FOLDED - if (in_swapper_pgdir(pmdp)) { - set_swapper_pgd((pgd_t *)pmdp, __pgd(pmd_val(pmd))); - return; - } -#endif /* __PAGETABLE_PMD_FOLDED */ - - WRITE_ONCE(*pmdp, pmd); + __put_kernel_nofault(pmdp, &pmd, pmd_t, fault); =20 if (pmd_valid(pmd)) queue_pte_barriers(); + return; + +fault: + try_set_readonly_ptval(&pmd_val(*pmdp), pmd_val(pmd)); } =20 static inline void pmd_clear(pmd_t *pmdp) @@ -890,15 +888,14 @@ static inline bool pgtable_l4_enabled(void); =20 static inline void set_pud(pud_t *pudp, pud_t pud) { - if (!pgtable_l4_enabled() && in_swapper_pgdir(pudp)) { - set_swapper_pgd((pgd_t *)pudp, __pgd(pud_val(pud))); - return; - } - - WRITE_ONCE(*pudp, pud); + __put_kernel_nofault(pudp, &pud, pud_t, fault); =20 if (pud_valid(pud)) queue_pte_barriers(); + return; + +fault: + try_set_readonly_ptval(&pud_val(*pudp), pud_val(pud)); } =20 static inline void pud_clear(pud_t *pudp) @@ -971,13 +968,12 @@ static inline bool mm_pud_folded(const struct mm_stru= ct *mm) =20 static inline void set_p4d(p4d_t *p4dp, p4d_t p4d) { - if (in_swapper_pgdir(p4dp)) { - set_swapper_pgd((pgd_t *)p4dp, __pgd(p4d_val(p4d))); - return; - } - - WRITE_ONCE(*p4dp, p4d); + __put_kernel_nofault(p4dp, &p4d, p4d_t, fault); queue_pte_barriers(); + return; + +fault: + try_set_readonly_ptval(&p4d_val(*p4dp), p4d_val(p4d)); } =20 static inline void p4d_clear(p4d_t *p4dp) @@ -1099,13 +1095,12 @@ static inline bool mm_p4d_folded(const struct mm_st= ruct *mm) =20 static inline void set_pgd(pgd_t *pgdp, pgd_t pgd) { - if (in_swapper_pgdir(pgdp)) { - set_swapper_pgd(pgdp, __pgd(pgd_val(pgd))); - return; - } - - WRITE_ONCE(*pgdp, pgd); + __put_kernel_nofault(pgdp, &pgd, pgd_t, fault); queue_pte_barriers(); + return; + +fault: + try_set_readonly_ptval(&pgd_val(*pgdp), pgd_val(pgd)); } =20 static inline void pgd_clear(pgd_t *pgdp) diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index 9c1aa838e9d5..b97e4bf99ca5 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -63,34 +63,34 @@ static bool rodata_is_rw __ro_after_init =3D true; */ long __section(".mmuoff.data.write") __early_cpu_boot_status; =20 -static DEFINE_SPINLOCK(swapper_pgdir_lock); static DEFINE_MUTEX(fixmap_lock); =20 -void noinstr set_swapper_pgd(pgd_t *pgdp, pgd_t pgd) +void noinstr try_set_readonly_ptval(ptval_t *ptvalp, ptval_t ptval) { - pgd_t *fixmap_pgdp; + static DEFINE_SPINLOCK(lock); + + BUG_ON(!in_swapper_pgdir(ptvalp)); =20 /* * Don't bother with the fixmap if swapper_pg_dir is still mapped * writable in the kernel mapping. */ if (rodata_is_rw) { - WRITE_ONCE(*pgdp, pgd); + WRITE_ONCE(*ptvalp, ptval); dsb(ishst); isb(); return; } =20 - spin_lock(&swapper_pgdir_lock); - fixmap_pgdp =3D pgd_set_fixmap(__pa_symbol(pgdp)); - WRITE_ONCE(*fixmap_pgdp, pgd); + guard(spinlock)(&lock); + ptvalp =3D (ptval_t *)set_fixmap_offset(FIX_PTVAL, __pa_symbol(ptvalp)); + WRITE_ONCE(*ptvalp, ptval); /* * We need dsb(ishst) here to ensure the page-table-walker sees * our new entry before set_p?d() returns. The fixmap's * flush_tlb_kernel_range() via clear_fixmap() does this for us. */ - pgd_clear_fixmap(); - spin_unlock(&swapper_pgdir_lock); + clear_fixmap(FIX_PTVAL); } =20 pgprot_t phys_mem_access_prot(struct file *file, unsigned long pfn, --=20 2.55.0.887.g758fc8c411-goog From nobody Mon Sep 28 02:56:56 2026 Received: from mail-wr1-f71.google.com (mail-wr1-f71.google.com [209.85.221.71]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D883A2D662F for ; Thu, 27 Aug 2026 16:44:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.71 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787849077; cv=none; b=J291GflR5mOTI3er2itw5NYe87NZ8RMSRDVWo9pYN0wjvoAWl1n13yHgcoU97k8nq8flb2T0yyS8AVzhkMjJM5wYYowAWpBBUttMBwSnkzBFkuSSBqw2sMynoNY09TtQ7dYMY/x+pZgg+YOdWwgvtVSLjeTww5ZTt3R0Os2cW3M= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787849077; c=relaxed/simple; bh=LFMNzqUI5j3XorhtaPRlaLPyBA8hl3XXmpLbCx2lnwU=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=irwBEuFM9TiM8RqUX5Vgz8ypUV7V45LvSSTlMueqaPJMMJOvLAiTCifTADcc68gLRAU23V4Dzk/ivCiSGmWOJWwFqoWgG3EY8PKj94yNJnxnhKJNzUhvtqVHKiuxrhmql9/3sOyz/z6ODv+er0LXONC7dN0CG+jZb5BGpJoxM/Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=pYup77Hg; arc=none smtp.client-ip=209.85.221.71 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="pYup77Hg" Received: by mail-wr1-f71.google.com with SMTP id ffacd0b85a97d-48000d75326so12802f8f.2 for ; Thu, 27 Aug 2026 09:44:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1787849074; x=1788453874; darn=vger.kernel.org; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:from:to:cc:subject:date:message-id:reply-to :content-type; bh=PJiY7FvuPSk4vmkG+0TxlMWylcjf3Tl2Za7dTctMfp4=; b=pYup77HgLIOL9y/DZz9Yui0kJovHA137bnK/kT8NX4dlltmcSCDvMri/rcAWeSrUVI 7lm5+GBQDbL2d+lCxxchIXv87JA9aC+JcpzlGTncHUiF3kbEzhmOEyCOG+nXvazaNGzk loRaA+l/XXdlQcOsFFiqzvf0r2lfMJL2UZD99xbTl5G2LA2rbcYLYYlkXAtmRUvhcfSb 6Fr1guqZq6f0KFbZcB9plEpdZMzTsMjWy89zudPXBq2ZYy/VDiKhVIl6qVGCkvjXrbbu RpmkC1XHOzPA3oGdS8tnzdwZOGxhfzXNm//81td/6eGzKYxk2E+wAstnhbsxUEfh9pEq HNaA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1787849074; x=1788453874; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to:content-type; bh=PJiY7FvuPSk4vmkG+0TxlMWylcjf3Tl2Za7dTctMfp4=; b=kEFs/TmJsClYuui+qiB3AW9mFf7pfnk7F3r3Be8RT01nJ9VxwoUyzumWsQKqBymXz+ M1FPFUbVeQfAdeEW9SWWsdqQkU3g9pao54/6vNFUOcDvFFexArjKfqZp6HckJR6LBgBo 8DLgf+fd+dYVa0hC6xkOyr94ZRBn5tJoJ2Pxl+bOU17sK0LY0ulk6zs0FHa623ULbz/a h35zgv6jJ24XzIgoDtBX+AVJTt0O7nM6wQzN0oJoLLzLphcUhPm7wbzMbBcncZdZBYGr mMMgNjISoWqoDy4A23TjZAp2rs4WIxYQdPBi5S1v53dcCdCqgdRNbzD0IChLntQzCphH LdLw== X-Gm-Message-State: AFuF++nO1rU60oqq75JFUHcV6neGwLux5ymKvFi/r/vSYdfQk1ta6Sm+ mxfcGJXSq5D8nGbh0Z9p+3CokRaagJklwFPRwSuqdF4riolwtOkWriMiJwNDiTLcJz7GQNLtew8 7KzK1v6agUR2GyinDfF/LUxkOx0Kw6C1y9NtgmlX+tLQNlhB8H0KKwsA4AIz5AF8PCNQKsKKW7A WwTdqdtfn6f+k6cuVxhVT8y0ScbfyGJtTvhw== X-Received: from wmbz8.prod.google.com ([2002:a05:600c:c088:b0:493:d1ba:af39]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:8581:b0:499:a760:722f with SMTP id 5b1f17b1804b1-49b91c47bf8mr2996215e9.13.1787849073998; Thu, 27 Aug 2026 09:44:33 -0700 (PDT) Date: Thu, 27 Aug 2026 18:44:13 +0200 In-Reply-To: <20260827164409.3421848-6-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260827164409.3421848-6-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=3860; i=ardb@kernel.org; h=from:subject; bh=99CV4XPf1Je/vZyhxqcfc6DlqZHXzUdS1hB8ZXFmVcM=; b=owGbwMvMwCVmkMcZplerG8N4Wi2JIWtCZtyKEpF9jaJhbA6KjNMV+X7sWlpw42m3Y2rLLr6CZ 65/VmV3lLIwiHExyIopsgjM/vtu5+mJUrXOs2Rh5rAygQxh4OIUgIlc2c7wP25nM5+wTNEvn7/T 0o0Vp62N/qPHaDNpi5/dLeX3l+zi4xn+Z2mUsya0F9QwvRRpyU87wZA/88cZ0YzQtSvq/mbLrJn CDQA= X-Mailer: git-send-email 2.55.0.897.gb25b4bd76c-goog Message-ID: <20260827164409.3421848-9-ardb+git@google.com> Subject: [RFC PATCH v2 3/4] arm64: mm: Create r/o page table region that permits updates From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: linux-arm-kernel@lists.infradead.org, Ard Biesheuvel Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel Generalize the handling of swapper_pg_dir, and apply the fixmap fallback for page table updates to all page tables in the .rodata..pgtbl section. Currently, this holds only swapper_pg_dir, but this will be expanded in subsequent patches. Signed-off-by: Ard Biesheuvel --- arch/arm64/include/asm/pgtable.h | 6 ----- arch/arm64/kernel/vmlinux.lds.S | 6 +++++ arch/arm64/mm/mmu.c | 26 +++++++++++++++++--- 3 files changed, 29 insertions(+), 9 deletions(-) diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgta= ble.h index 799cd9d52c28..de0a07dae83a 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -817,12 +817,6 @@ extern pgd_t reserved_pg_dir[]; =20 void try_set_readonly_ptval(ptval_t *ptvalp, ptval_t ptval); =20 -static inline bool in_swapper_pgdir(void *addr) -{ - return ((unsigned long)addr & PAGE_MASK) =3D=3D - ((unsigned long)swapper_pg_dir & PAGE_MASK); -} - static inline void set_pmd(pmd_t *pmdp, pmd_t pmd) { __put_kernel_nofault(pmdp, &pmd, pmd_t, fault); diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.ld= s.S index af1d72020976..2cda0b7c41d4 100644 --- a/arch/arm64/kernel/vmlinux.lds.S +++ b/arch/arm64/kernel/vmlinux.lds.S @@ -243,9 +243,15 @@ SECTIONS reserved_pg_dir =3D .; . +=3D PAGE_SIZE; =20 + __rodata_pgtbl_start =3D .; swapper_pg_dir =3D .; . +=3D PAGE_SIZE; =20 + .pgtbl.ro_after_init : ALIGN(PAGE_SIZE) { + *(.pgtbl.ro_after_init) + } + __rodata_pgtbl_end =3D .; + . =3D ALIGN(SEGMENT_ALIGN); __init_begin =3D .; __inittext_begin =3D .; diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index b97e4bf99ca5..dad482d66f17 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -65,17 +65,27 @@ long __section(".mmuoff.data.write") __early_cpu_boot_s= tatus; =20 static DEFINE_MUTEX(fixmap_lock); =20 +static struct range kimg_ropgtbl_range __ro_after_init; + void noinstr try_set_readonly_ptval(ptval_t *ptvalp, ptval_t ptval) { static DEFINE_SPINLOCK(lock); + bool is_lm =3D __is_lm_address(ptvalp); + u64 pa =3D is_lm ? __pa(ptvalp) : __pa_symbol(ptvalp); =20 - BUG_ON(!in_swapper_pgdir(ptvalp)); + if (!range_contains(&kimg_ropgtbl_range, + &DEFINE_RANGE(pa, pa + sizeof(ptval_t)))) + BUG(); =20 /* - * Don't bother with the fixmap if swapper_pg_dir is still mapped + * Don't bother with the fixmap if .rodata is still mapped * writable in the kernel mapping. */ if (rodata_is_rw) { + /* no fault should have occurred for a kimg address */ + BUG_ON(!is_lm); + + ptvalp =3D (ptval_t *)__phys_to_kimg(pa); WRITE_ONCE(*ptvalp, ptval); dsb(ishst); isb(); @@ -83,7 +93,7 @@ void noinstr try_set_readonly_ptval(ptval_t *ptvalp, ptva= l_t ptval) } =20 guard(spinlock)(&lock); - ptvalp =3D (ptval_t *)set_fixmap_offset(FIX_PTVAL, __pa_symbol(ptvalp)); + ptvalp =3D (ptval_t *)set_fixmap_offset(FIX_PTVAL, pa); WRITE_ONCE(*ptvalp, ptval); /* * We need dsb(ishst) here to ensure the page-table-walker sees @@ -1177,6 +1187,14 @@ static inline void arm64_kfence_map_pool(void) { } =20 #endif /* CONFIG_KFENCE */ =20 +static void __init record_ropgtbl_phys_range(void) +{ + extern const char __rodata_pgtbl_start[], __rodata_pgtbl_end[]; + + kimg_ropgtbl_range =3D DEFINE_RANGE(__pa_symbol(__rodata_pgtbl_start), + __pa_symbol(__rodata_pgtbl_end)); +} + static void __init map_mem(void) { static const u64 direct_map_end =3D _PAGE_END(VA_BITS_MIN); @@ -1189,6 +1207,8 @@ static void __init map_mem(void) int flags =3D NO_EXEC_MAPPINGS; u64 i; =20 + record_ropgtbl_phys_range(); + /* * Setting hierarchical PXNTable attributes on table entries covering * the linear region is only possible if it is guaranteed that no table --=20 2.55.0.887.g758fc8c411-goog From nobody Mon Sep 28 02:56:56 2026 Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 65519483BFF for ; Thu, 27 Aug 2026 16:44:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.71 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787849078; cv=none; b=FaHVcSznmeCFiCG7oOYzWNwHz/05wzcPY231sQq7bkouvdABhljMEbiHIttfL1s7oGJ1dSMgLFab4llXnRdGgaQrXptukjnkKXqE0fhH0/uM12fO6WFgh5HD/OLRtkJOP5iALvtU8fr52JeONCkggjJ93SnCw8CyozChof+c944= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787849078; c=relaxed/simple; bh=BHgZZrHxJq7P0OR82PRXWx26UIoot1+LWqzEJHh+qQw=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=o6buazB7w4Vv0MnGy/xcbgyDQpInirEFbJsb+SqqPu/Ii6YkqhQwVm59qLEhVKLXG3k7/mRDLDq6+zh2hQTJP7i/nbMFxXb1VNWMtR6jvtHVCfzfjTP3tfg12iSdeCosTtZbYrPXmxnKR7L5Y013YwJjM0njx6T4yOZwIVM281o= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=poMc5Ibz; arc=none smtp.client-ip=209.85.128.71 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="poMc5Ibz" Received: by mail-wm1-f71.google.com with SMTP id 5b1f17b1804b1-499a7993a9bso17934165e9.1 for ; Thu, 27 Aug 2026 09:44:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1787849075; x=1788453875; darn=vger.kernel.org; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:from:to:cc:subject:date:message-id:reply-to :content-type; bh=43iLPgD5M5R/GHwUlmehT6b7nY+XiShZxegudQN2h98=; b=poMc5IbzNL+BiFzbmROn8ZdmS+w2hz7jKZBSrucWmDIoi4UGh9eVHZh4eRvHnvTGQc vcV6ZpB/GCdv9b5BiiY/PhlBRKMrAmoNHCzwznny+shlfuuqFZ5diOfCFLgkCVCnD1IH oV6wE5CFelexV09VApa9+NQDMJvCnRhSHJtpFTwAHlBy5PRKN6w92bzpiDmbDCuHfEep aFQRFNinXa4rVUwaVFLbUqz1qC7DBNPRvHbAKiwPuffif7SJYLiMAvtZXqIg7/5GdXtb T2oKdYjFdOJ2ISDAzFkvy6uSupmNC1h5g8Z8bPqJq3WqyW4MY9lhskK5TI3xm/NUatWN RF1w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1787849075; x=1788453875; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to:content-type; bh=43iLPgD5M5R/GHwUlmehT6b7nY+XiShZxegudQN2h98=; b=Hk0R6JQ2rwsOT6UvH8mdyq6LvPA+dANJsxd4M63ibHSkGWDSCqJp402ZsFGA2XkGGk W/SHXcyn+07b6uJTY6Sdd88wq263zRXo3wPSR8T0ohlWLKl023s1ZqnDlXwiZ9fkoNOK JLcdVw7nvYUEGmHKFCuJpjFOx4uU+X5a+MAkx5c2zOvWsc5tbfKAyP4N1jh7bWNWkw2S SK7823C+8LEMxbwQFCPzZvjYF1LivRjfhg/zTgd8uvLo7L4FJJuC43vm8uaiMQYOZQ/b stpddbkfPz2zNCiFAE+590vBQbDuL6iY8brrPfxISj2IU0BZN1tWtInxiFhkYcgh/vQH riMA== X-Gm-Message-State: AFuF++nqefhmS3syid2Hr95DT0gIWWXvotkj5XF1/2CjewrIuhW2E/GA 93LO7G2EJIzh4MPBrQyBFans8fX9QWypkgEyi+gnhHVyAH6bE+lYdEyQRQTEhEfNkPpQuI4Whlu TRvRB/hKzrZVEiiOsuKQyDgQmQfE0euBukqyLn3TU9x5wQv5U9oZtndUVq0m0MTRzQGI4VNQb5F 2aImyoP/hM7Um3gotjinhNK9zZOUt7qLM+sg== X-Received: from wrod1.prod.google.com ([2002:adf:ef81:0:b0:47f:93e7:6c8c]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:c178:b0:499:872b:abd4 with SMTP id 5b1f17b1804b1-49b91c21b86mr3254375e9.6.1787849075235; Thu, 27 Aug 2026 09:44:35 -0700 (PDT) Date: Thu, 27 Aug 2026 18:44:14 +0200 In-Reply-To: <20260827164409.3421848-6-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260827164409.3421848-6-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=2014; i=ardb@kernel.org; h=from:subject; bh=iWvmUytrCQAfIl3uYQyfQdqp2tLg/jsjNdw7eZtuwd0=; b=owGbwMvMwCVmkMcZplerG8N4Wi2JIWtCZny0jfFXyRPr6kztSk5KTjo11/i5XetD2xOzdr/7G G+1bWJmRykLgxgXg6yYIovA7L/vdp6eKFXrPEsWZg4rE8gQBi5OAZjI0niG/yVp86bI9mTdPJ2/ 7slZ67LSlMfBb4xPn25vNBcuj2xLL2FkmLDqohFn5pQSo3qWIqvZt4S6f7iXrQ9NKtfmP9f4YX4 TFwA= X-Mailer: git-send-email 2.55.0.897.gb25b4bd76c-goog Message-ID: <20260827164409.3421848-10-ardb+git@google.com> Subject: [RFC PATCH v2 4/4] arm64: mm: Move fixmap intermediate page tables into .rodata From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: linux-arm-kernel@lists.infradead.org, Ard Biesheuvel Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel The fixmap intermediate page tables are allocated statically, are installed into the kernel's page table hierarchy early during boot, and control a slice of the kernel's virtual address space that is not subject to KASLR randomization. Combined with the lack of randomization of the linear map, and the tendency of some Android bootloaders to place the kernel image at the base of DRAM in the physical space, the placement of these page tables produces a vulnerability that is comparatively easy to exploit. Avoid this, by moving these intermediate page tables into .rodata, so that they cannot be manipulated directly via the linear map. Signed-off-by: Ard Biesheuvel --- arch/arm64/include/asm/linkage.h | 1 + arch/arm64/mm/fixmap.c | 4 ++-- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/arch/arm64/include/asm/linkage.h b/arch/arm64/include/asm/link= age.h index d1f7a16729d2..00963e11ebf0 100644 --- a/arch/arm64/include/asm/linkage.h +++ b/arch/arm64/include/asm/linkage.h @@ -45,6 +45,7 @@ =20 #define _THIS_IP_ ({ unsigned long __ip; asm volatile("adr %0, ." : "=3Dr"= (__ip)); __ip; }) =20 +#define __rodata_pgtbl __section(".pgtbl.ro_after_init") __aligned(PAGE_SI= ZE) #define __bss_pgtbl __section(".bss..pgtbl") __aligned(PAGE_SIZE) =20 #endif diff --git a/arch/arm64/mm/fixmap.c b/arch/arm64/mm/fixmap.c index 3a8cf6de6a7d..ab0f9ba7b712 100644 --- a/arch/arm64/mm/fixmap.c +++ b/arch/arm64/mm/fixmap.c @@ -32,8 +32,8 @@ static_assert(NR_BM_PMD_TABLES =3D=3D 1); #define BM_PTE_TABLE_IDX(addr) __BM_TABLE_IDX(addr, PMD_SHIFT) =20 pte_t fixmap_bm_pte[NR_BM_PTE_TABLES][PTRS_PER_PTE] __bss_pgtbl; -static pmd_t bm_pmd[PTRS_PER_PMD] __bss_pgtbl __maybe_unused; -static pud_t bm_pud[PTRS_PER_PUD] __bss_pgtbl __maybe_unused; +static pmd_t bm_pmd[PTRS_PER_PMD] __rodata_pgtbl; +static pud_t bm_pud[PTRS_PER_PUD] __rodata_pgtbl; =20 const size_t fixmap_bm_pte_size =3D sizeof(fixmap_bm_pte); =20 --=20 2.55.0.887.g758fc8c411-goog