From nobody Sat Sep 26 20:27:49 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7A2B93C98B5 for ; Sun, 30 Aug 2026 17:24:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788110642; cv=none; b=Am7BPT67ezGmhQtS+GF7KMMSEcMZWbq5/UiknRklyv17zgSP77lWi+dxTSs7SqsuszGOwTCdAqiJa2UrOGEuz1sz7PoBPw8Fv+eSWeW1Snl7b5iIX6xign5mJ8RalKYZxdH039J3qKMZAnpXOPReSZc3XLsXQHCQxSn3ji4B3iE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788110642; c=relaxed/simple; bh=XaLS4nfXqVy84eCnC7lh5/Z+hlX4pQRvnZIjNy3Ttnk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Z0CwPsFXXP/WIyjasdUHSVJSYVkk+DM4kYBkjyN/teim3fqJhMjXojaXhDhUA7sKnlWtOwtPPFYKkS+nJXc1IJOuUVCWFlsD9B5tCfup3ZyxCSNXEtBiQTwuCCeGhVEcrT//IVX99GnP+uYnKt1mtWrBj9uhNnptusm3FqRj9Qk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=fnuPJlXi; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="fnuPJlXi" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 01B681F00AC4; Sun, 30 Aug 2026 17:23:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1788110640; bh=r0Za95NxwjXTwT3tld/1tYydLeQeiEWwl1b59iKEheQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=fnuPJlXiuUQiluo/jnM9JTcIU8FveBGagrgFnEmZcQPYQmjUl5lf7+dNt1lc3Wp6w aGdLmxy2A7rGsPI1w4S2elodwv/3/Wy/0zUt8DBW28Cy44t6CGnCyKGdOGx9FOvfGw V8DeuHAQJW/6yHdyxilmRtjCt9aJdNsz3zsb8INjkR4di50Ujv6W1QoQonV/y9OkYF WLB8iMeJkaS3yaKZVQALcLpkCSDVdtv442uaEdHSOpPXzd6NqqGjn3lOS951+H94Zr JiPA+p6cS7eW2H2c4oaDw5h2f2tsYedJXR7S+quvpr7v0fRbrOfT+LTw77D5JbzdN9 wEBKaVAT0cs2Q== From: Jisheng Zhang To: Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexandre Ghiti Cc: linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH 1/3] riscv: remove RISCV_ALTERNATIVE Kconfig option Date: Mon, 31 Aug 2026 01:03:59 +0800 Message-ID: <20260830170414.4505-2-jszhang@kernel.org> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260830170414.4505-1-jszhang@kernel.org> References: <20260830170414.4505-1-jszhang@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" riscv always selects RISCV_ALTERNATIVE now, so we can remove this Kconfig option and enable RISCV_ALTERNATIVE code unconditionally. Signed-off-by: Jisheng Zhang Reviewed-by: Conor Dooley --- arch/riscv/Kconfig | 23 -------------------- arch/riscv/Kconfig.errata | 5 +---- arch/riscv/include/asm/alternative-macros.h | 24 --------------------- arch/riscv/include/asm/alternative.h | 10 --------- arch/riscv/include/asm/cpufeature-macros.h | 10 ++------- arch/riscv/include/asm/cpufeature.h | 6 ++---- arch/riscv/include/asm/vendor_extensions.h | 18 ++++++---------- arch/riscv/kernel/Makefile | 2 +- arch/riscv/kernel/cpufeature.c | 2 -- 9 files changed, 12 insertions(+), 88 deletions(-) diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index f8e26c4bed2b..13b7bb77087e 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -218,7 +218,6 @@ config RISCV select PCI_ECAM if (ACPI && PCI) select PCI_MSI if PCI select RELOCATABLE if !MMU && !PHYS_RAM_BASE_FIXED - select RISCV_ALTERNATIVE select RISCV_APLIC select RISCV_IMSIC select RISCV_INTC @@ -539,17 +538,8 @@ config RISCV_COMBO_SPINLOCKS =20 endchoice =20 -config RISCV_ALTERNATIVE - bool - help - This Kconfig allows the kernel to automatically patch the - erratum or cpufeature required by the execution platform at run - time. The code patching overhead is minimal, as it's only done - once at boot and once on each module load. - config RISCV_ALTERNATIVE_EARLY bool - depends on RISCV_ALTERNATIVE help Allows early patching of the kernel for special errata =20 @@ -578,7 +568,6 @@ config RISCV_ISA_SUPM config RISCV_ISA_SVNAPOT bool "Svnapot extension support for supervisor mode NAPOT pages" depends on 64BIT && MMU - depends on RISCV_ALTERNATIVE default y help Enable support for the Svnapot ISA-extension when it is detected @@ -615,7 +604,6 @@ config RISCV_ISA_SSQOSID config RISCV_ISA_SVPBMT bool "Svpbmt extension support for supervisor mode page-based memory type= s" depends on 64BIT && MMU - depends on RISCV_ALTERNATIVE default y help Add support for the Svpbmt ISA-extension (Supervisor-mode: @@ -685,7 +673,6 @@ config RISCV_ISA_V_PREEMPTIVE =20 config RISCV_ISA_ZAWRS bool "Zawrs extension support for more efficient busy waiting" - depends on RISCV_ALTERNATIVE default y help The Zawrs extension defines instructions to be used in polling loops @@ -704,7 +691,6 @@ config TOOLCHAIN_HAS_ZABHA config RISCV_ISA_ZABHA bool "Zabha extension support for atomic byte/halfword operations" depends on TOOLCHAIN_HAS_ZABHA - depends on RISCV_ALTERNATIVE default y help Enable the use of the Zabha ISA-extension to implement kernel @@ -721,7 +707,6 @@ config TOOLCHAIN_HAS_ZACAS =20 config RISCV_ISA_ZACAS bool "Zacas extension support for atomic CAS" - depends on RISCV_ALTERNATIVE default y help Enable the use of the Zacas ISA-extension to implement kernel atomic @@ -766,7 +751,6 @@ config RISCV_ISA_ZBA =20 config RISCV_ISA_ZBB bool "Zbb extension support for bit manipulation instructions" - depends on RISCV_ALTERNATIVE default y help Add support for enabling optimisations in the kernel when the @@ -791,7 +775,6 @@ config RISCV_ISA_ZBC bool "Zbc extension support for carry-less multiplication instructions" depends on TOOLCHAIN_HAS_ZBC depends on MMU - depends on RISCV_ALTERNATIVE default y help Adds support to dynamically detect the presence of the Zbc @@ -813,7 +796,6 @@ config TOOLCHAIN_HAS_ZBKB config RISCV_ISA_ZBKB bool "Zbkb extension support for bit manipulation instructions" depends on TOOLCHAIN_HAS_ZBKB - depends on RISCV_ALTERNATIVE default y help Adds support to dynamically detect the presence of the ZBKB @@ -827,7 +809,6 @@ config RISCV_ISA_ZBKB config RISCV_ISA_ZICBOM bool "Zicbom extension support for non-coherent DMA operation" depends on MMU - depends on RISCV_ALTERNATIVE default y select RISCV_DMA_NONCOHERENT select DMA_DIRECT_REMAP @@ -843,7 +824,6 @@ config RISCV_ISA_ZICBOM =20 config RISCV_ISA_ZICBOZ bool "Zicboz extension support for faster zeroing of memory" - depends on RISCV_ALTERNATIVE default y help Enable the use of the Zicboz extension (cbo.zero instruction) @@ -856,7 +836,6 @@ config RISCV_ISA_ZICBOZ config RISCV_ISA_ZICBOP bool "Zicbop extension support for cache block prefetch" depends on MMU - depends on RISCV_ALTERNATIVE default y help Adds support to dynamically detect the presence of the ZICBOP @@ -871,7 +850,6 @@ config RISCV_ISA_ZICBOP config RISCV_ISA_SVRSW60T59B bool "Svrsw60t59b extension support for using PTE bits 60 and 59" depends on MMU && 64BIT - depends on RISCV_ALTERNATIVE default y help Adds support to dynamically detect the presence of the Svrsw60t59b @@ -1204,7 +1182,6 @@ config RISCV_USER_CFI bool "riscv userspace control flow integrity" depends on 64BIT && MMU && \ $(cc-option,-mabi=3Dlp64 -march=3Drv64ima_zicfiss_zicfilp -fcf-protectio= n=3Dfull) - depends on RISCV_ALTERNATIVE select RISCV_SBI select ARCH_HAS_USER_SHADOW_STACK select ARCH_USES_HIGH_VMA_FLAGS diff --git a/arch/riscv/Kconfig.errata b/arch/riscv/Kconfig.errata index 3c945d086c7d..1a66e5b6f7d1 100644 --- a/arch/riscv/Kconfig.errata +++ b/arch/riscv/Kconfig.errata @@ -2,7 +2,7 @@ menu "CPU errata selection" =20 config ERRATA_ANDES bool "Andes AX45MP errata" - depends on RISCV_ALTERNATIVE && RISCV_SBI + depends on RISCV_SBI help All Andes errata Kconfig depend on this Kconfig. Disabling this Kconfig will disable all Andes errata. Please say "Y" @@ -23,7 +23,6 @@ config ERRATA_ANDES_CMO =20 config ERRATA_MIPS bool "MIPS errata" - depends on RISCV_ALTERNATIVE help All MIPS errata Kconfig depend on this Kconfig. Disabling this Kconfig will disable all MIPS errata. Please say "Y" @@ -46,7 +45,6 @@ config ERRATA_MIPS_P8700_PAUSE_OPCODE =20 config ERRATA_SIFIVE bool "SiFive errata" - depends on RISCV_ALTERNATIVE help All SiFive errata Kconfig depend on this Kconfig. Disabling this Kconfig will disable all SiFive errata. Please say "Y" @@ -98,7 +96,6 @@ config ERRATA_STARFIVE_JH7100 =20 config ERRATA_THEAD bool "T-HEAD errata" - depends on RISCV_ALTERNATIVE help All T-HEAD errata Kconfig depend on this Kconfig. Disabling this Kconfig will disable all T-HEAD errata. Please say "Y" diff --git a/arch/riscv/include/asm/alternative-macros.h b/arch/riscv/inclu= de/asm/alternative-macros.h index 9619bd5c8eba..629d884578da 100644 --- a/arch/riscv/include/asm/alternative-macros.h +++ b/arch/riscv/include/asm/alternative-macros.h @@ -2,8 +2,6 @@ #ifndef __ASM_ALTERNATIVE_MACROS_H #define __ASM_ALTERNATIVE_MACROS_H =20 -#ifdef CONFIG_RISCV_ALTERNATIVE - #ifdef __ASSEMBLER__ =20 .macro ALT_ENTRY oldptr newptr vendor_id patch_id new_len @@ -108,28 +106,6 @@ __ALTERNATIVE_CFG_2(old_c, new_c_1, vendor_id_1, patch_id_1, IS_ENABLED(C= ONFIG_k_1), \ new_c_2, vendor_id_2, patch_id_2, IS_ENABLED(CONFIG_k_2)) =20 -#else /* CONFIG_RISCV_ALTERNATIVE */ -#ifdef __ASSEMBLER__ - -.macro ALTERNATIVE_CFG old_c - \old_c -.endm - -#define __ALTERNATIVE_CFG(old_c, ...) ALTERNATIVE_CFG old_c -#define __ALTERNATIVE_CFG_2(old_c, ...) ALTERNATIVE_CFG old_c - -#else /* !__ASSEMBLER__ */ - -#define __ALTERNATIVE_CFG(old_c, ...) old_c "\n" -#define __ALTERNATIVE_CFG_2(old_c, ...) old_c "\n" - -#endif /* __ASSEMBLER__ */ - -#define _ALTERNATIVE_CFG(old_c, ...) __ALTERNATIVE_CFG(old_c) -#define _ALTERNATIVE_CFG_2(old_c, ...) __ALTERNATIVE_CFG_2(old_c) - -#endif /* CONFIG_RISCV_ALTERNATIVE */ - /* * Usage: * ALTERNATIVE(old_content, new_content, vendor_id, patch_id, CONFIG_k) diff --git a/arch/riscv/include/asm/alternative.h b/arch/riscv/include/asm/= alternative.h index 8407d1d535b8..688c7d1a9ae3 100644 --- a/arch/riscv/include/asm/alternative.h +++ b/arch/riscv/include/asm/alternative.h @@ -10,8 +10,6 @@ =20 #ifndef __ASSEMBLER__ =20 -#ifdef CONFIG_RISCV_ALTERNATIVE - #include #include #include @@ -61,13 +59,5 @@ void thead_errata_patch_func(struct alt_entry *begin, st= ruct alt_entry *end, void riscv_cpufeature_patch_func(struct alt_entry *begin, struct alt_entry= *end, unsigned int stage); =20 -#else /* CONFIG_RISCV_ALTERNATIVE */ - -static inline void apply_boot_alternatives(void) { } -static inline void apply_early_boot_alternatives(void) { } -static inline void apply_module_alternatives(void *start, size_t length) {= } - -#endif /* CONFIG_RISCV_ALTERNATIVE */ - #endif #endif diff --git a/arch/riscv/include/asm/cpufeature-macros.h b/arch/riscv/includ= e/asm/cpufeature-macros.h index a8103edbf51f..adaf9e3fb25c 100644 --- a/arch/riscv/include/asm/cpufeature-macros.h +++ b/arch/riscv/include/asm/cpufeature-macros.h @@ -47,20 +47,14 @@ static __always_inline bool riscv_has_extension_unlikel= y(const unsigned long ext { compiletime_assert(ext < RISCV_ISA_EXT_MAX, "ext must be < RISCV_ISA_EXT_= MAX"); =20 - if (IS_ENABLED(CONFIG_RISCV_ALTERNATIVE)) - return __riscv_has_extension_unlikely(STANDARD_EXT, ext); - - return __riscv_isa_extension_available(NULL, ext); + return __riscv_has_extension_unlikely(STANDARD_EXT, ext); } =20 static __always_inline bool riscv_has_extension_likely(const unsigned long= ext) { compiletime_assert(ext < RISCV_ISA_EXT_MAX, "ext must be < RISCV_ISA_EXT_= MAX"); =20 - if (IS_ENABLED(CONFIG_RISCV_ALTERNATIVE)) - return __riscv_has_extension_likely(STANDARD_EXT, ext); - - return __riscv_isa_extension_available(NULL, ext); + return __riscv_has_extension_likely(STANDARD_EXT, ext); } =20 #endif /* _ASM_CPUFEATURE_MACROS_H */ diff --git a/arch/riscv/include/asm/cpufeature.h b/arch/riscv/include/asm/c= pufeature.h index 739fcc84bf7b..37c9f2a0fb54 100644 --- a/arch/riscv/include/asm/cpufeature.h +++ b/arch/riscv/include/asm/cpufeature.h @@ -134,8 +134,7 @@ static __always_inline bool riscv_cpu_has_extension_lik= ely(int cpu, const unsign { compiletime_assert(ext < RISCV_ISA_EXT_MAX, "ext must be < RISCV_ISA_EXT_= MAX"); =20 - if (IS_ENABLED(CONFIG_RISCV_ALTERNATIVE) && - __riscv_has_extension_likely(STANDARD_EXT, ext)) + if (__riscv_has_extension_likely(STANDARD_EXT, ext)) return true; =20 return __riscv_isa_extension_available(hart_isa[cpu].isa, ext); @@ -145,8 +144,7 @@ static __always_inline bool riscv_cpu_has_extension_unl= ikely(int cpu, const unsi { compiletime_assert(ext < RISCV_ISA_EXT_MAX, "ext must be < RISCV_ISA_EXT_= MAX"); =20 - if (IS_ENABLED(CONFIG_RISCV_ALTERNATIVE) && - __riscv_has_extension_unlikely(STANDARD_EXT, ext)) + if (__riscv_has_extension_unlikely(STANDARD_EXT, ext)) return true; =20 return __riscv_isa_extension_available(hart_isa[cpu].isa, ext); diff --git a/arch/riscv/include/asm/vendor_extensions.h b/arch/riscv/includ= e/asm/vendor_extensions.h index 7437304a71b9..b00149be2627 100644 --- a/arch/riscv/include/asm/vendor_extensions.h +++ b/arch/riscv/include/asm/vendor_extensions.h @@ -54,11 +54,9 @@ static __always_inline bool riscv_has_vendor_extension_l= ikely(const unsigned lon if (!IS_ENABLED(CONFIG_RISCV_ISA_VENDOR_EXT)) return false; =20 - if (IS_ENABLED(CONFIG_RISCV_ALTERNATIVE)) - return __riscv_has_extension_likely(vendor, - ext + RISCV_VENDOR_EXT_ALTERNATIVES_BASE); + return __riscv_has_extension_likely(vendor, + ext + RISCV_VENDOR_EXT_ALTERNATIVES_BASE); =20 - return __riscv_isa_vendor_extension_available(VENDOR_EXT_ALL_CPUS, vendor= , ext); } =20 static __always_inline bool riscv_has_vendor_extension_unlikely(const unsi= gned long vendor, @@ -67,11 +65,9 @@ static __always_inline bool riscv_has_vendor_extension_u= nlikely(const unsigned l if (!IS_ENABLED(CONFIG_RISCV_ISA_VENDOR_EXT)) return false; =20 - if (IS_ENABLED(CONFIG_RISCV_ALTERNATIVE)) - return __riscv_has_extension_unlikely(vendor, - ext + RISCV_VENDOR_EXT_ALTERNATIVES_BASE); + return __riscv_has_extension_unlikely(vendor, + ext + RISCV_VENDOR_EXT_ALTERNATIVES_BASE); =20 - return __riscv_isa_vendor_extension_available(VENDOR_EXT_ALL_CPUS, vendor= , ext); } =20 static __always_inline bool riscv_cpu_has_vendor_extension_likely(const un= signed long vendor, @@ -80,8 +76,7 @@ static __always_inline bool riscv_cpu_has_vendor_extensio= n_likely(const unsigned if (!IS_ENABLED(CONFIG_RISCV_ISA_VENDOR_EXT)) return false; =20 - if (IS_ENABLED(CONFIG_RISCV_ALTERNATIVE) && - __riscv_has_extension_likely(vendor, ext + RISCV_VENDOR_EXT_ALTERNATI= VES_BASE)) + if (__riscv_has_extension_likely(vendor, ext + RISCV_VENDOR_EXT_ALTERNATI= VES_BASE)) return true; =20 return __riscv_isa_vendor_extension_available(cpu, vendor, ext); @@ -94,8 +89,7 @@ static __always_inline bool riscv_cpu_has_vendor_extensio= n_unlikely(const unsign if (!IS_ENABLED(CONFIG_RISCV_ISA_VENDOR_EXT)) return false; =20 - if (IS_ENABLED(CONFIG_RISCV_ALTERNATIVE) && - __riscv_has_extension_unlikely(vendor, ext + RISCV_VENDOR_EXT_ALTERNA= TIVES_BASE)) + if (__riscv_has_extension_unlikely(vendor, ext + RISCV_VENDOR_EXT_ALTERNA= TIVES_BASE)) return true; =20 return __riscv_isa_vendor_extension_available(cpu, vendor, ext); diff --git a/arch/riscv/kernel/Makefile b/arch/riscv/kernel/Makefile index ebe1c3588177..91353c94bfab 100644 --- a/arch/riscv/kernel/Makefile +++ b/arch/riscv/kernel/Makefile @@ -48,7 +48,7 @@ always-$(KBUILD_BUILTIN) +=3D vmlinux.lds =20 obj-y +=3D head.o obj-y +=3D soc.o -obj-$(CONFIG_RISCV_ALTERNATIVE) +=3D alternative.o +obj-y +=3D alternative.o obj-y +=3D cpu.o obj-y +=3D cpufeature.o obj-y +=3D entry.o diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c index d2ec96843456..9915121e9438 100644 --- a/arch/riscv/kernel/cpufeature.c +++ b/arch/riscv/kernel/cpufeature.c @@ -1217,7 +1217,6 @@ void __init riscv_user_isa_enable(void) pr_warn("Zicbop disabled as it is unavailable on some harts\n"); } =20 -#ifdef CONFIG_RISCV_ALTERNATIVE /* * Alternative patch sites consider 48 bits when determining when to patch * the old instruction sequence with the new. These bits are broken into a @@ -1306,4 +1305,3 @@ void __init_or_module riscv_cpufeature_patch_func(str= uct alt_entry *begin, mutex_unlock(&text_mutex); } } -#endif --=20 2.53.0 From nobody Sat Sep 26 20:27:49 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5FC203CC7DC for ; Sun, 30 Aug 2026 17:24:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788110646; cv=none; b=hckTopRWvW3w08GXCLsRwHg3zzGeesIdegEPalXfLzOnd/repiGl3eID6Eee6hV58IUf8BoAilKV2w2/jvtBfBf/ci4t2p2r1qqQaAipBNQnxk+hx8gmNYUfN3x3jQ0nrI02wcXk/x60vugTQnzZ0ENxoVO/pOd3Cqf/QtwO02g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788110646; c=relaxed/simple; bh=FS6iIFpgIRntBvke0ypecdBplBg+sST3pgVlfY83BiQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=giu+AsZqKPeEPFR2bhECHgc0gkO8kWOUeJEuE9aapc3MU6VVqJm7+tNzH/AroBSX/vLKz9IG0lJ2jirtxCk8eFAFufyoS+iw7hGw9pPsj2qgqysKsTWt50zqvTksffR4Oc1ridc1LwUqVDJRJpsD2Wd2eW1hwfogEX0EhzlyNow= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=WG417k9U; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="WG417k9U" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3790A1F00ADF; Sun, 30 Aug 2026 17:24:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1788110642; bh=b6AoAFWyY4iKujGv3HFd++zappWu8IRi6MxeUuLqKws=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=WG417k9UEPMGSAHaY0+/3GD63jo6EpdKn9mrj9WLyt70p1Z315wJEqvKqmzdiuf87 DtiHXQmfxOCnCaCqbF+e6N0VQvOI4wXbLjc5PlWADA2EqNDJfXvHWPsCCm1RfzSQXV LsLpDAO0SYT2r3ouQKQUqGSttodu34D6In97oHQD5cgLFqHApwOU04epI1VW0Q1KYb WI4LPhmTo4ZvTaufKQ/CLxhaHHHYZkJM/lXlTY62N8v+YtEUdNmrp9jovTOYntWC/L Mn98qL+obcpuGiwfyGEf0/KUxWSf346zABzyqAY8pH3AQOvGu+EKN2iQOmiImXf9RQ dAyGzUOfrTpAg== From: Jisheng Zhang To: Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexandre Ghiti Cc: linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH 2/3] riscv: convert pgtable_l4|l5_enabled to inline function Date: Mon, 31 Aug 2026 01:04:00 +0800 Message-ID: <20260830170414.4505-3-jszhang@kernel.org> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260830170414.4505-1-jszhang@kernel.org> References: <20260830170414.4505-1-jszhang@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" This is the preparation of optimizing pgtable_l4|l5_enabled(). No functionality change. Signed-off-by: Jisheng Zhang --- arch/riscv/include/asm/pgalloc.h | 14 +++---- arch/riscv/include/asm/pgtable-32.h | 4 +- arch/riscv/include/asm/pgtable-64.h | 50 +++++++++++++--------- arch/riscv/include/asm/pgtable.h | 4 +- arch/riscv/include/asm/vmalloc.h | 5 +-- arch/riscv/kernel/cpu.c | 4 +- arch/riscv/mm/init.c | 64 ++++++++++++++--------------- arch/riscv/mm/kasan_init.c | 20 ++++----- arch/riscv/mm/pgtable.c | 4 +- arch/riscv/mm/ptdump.c | 4 +- 10 files changed, 91 insertions(+), 82 deletions(-) diff --git a/arch/riscv/include/asm/pgalloc.h b/arch/riscv/include/asm/pgal= loc.h index 770ce18a7328..0133f7ba41d7 100644 --- a/arch/riscv/include/asm/pgalloc.h +++ b/arch/riscv/include/asm/pgalloc.h @@ -41,7 +41,7 @@ static inline void pud_populate(struct mm_struct *mm, pud= _t *pud, pmd_t *pmd) =20 static inline void p4d_populate(struct mm_struct *mm, p4d_t *p4d, pud_t *p= ud) { - if (pgtable_l4_enabled) { + if (pgtable_l4_enabled()) { unsigned long pfn =3D virt_to_pfn(pud); =20 set_p4d(p4d, __p4d((pfn << _PAGE_PFN_SHIFT) | _PAGE_TABLE)); @@ -51,7 +51,7 @@ static inline void p4d_populate(struct mm_struct *mm, p4d= _t *p4d, pud_t *pud) static inline void p4d_populate_safe(struct mm_struct *mm, p4d_t *p4d, pud_t *pud) { - if (pgtable_l4_enabled) { + if (pgtable_l4_enabled()) { unsigned long pfn =3D virt_to_pfn(pud); =20 set_p4d_safe(p4d, @@ -61,7 +61,7 @@ static inline void p4d_populate_safe(struct mm_struct *mm= , p4d_t *p4d, =20 static inline void pgd_populate(struct mm_struct *mm, pgd_t *pgd, p4d_t *p= 4d) { - if (pgtable_l5_enabled) { + if (pgtable_l5_enabled()) { unsigned long pfn =3D virt_to_pfn(p4d); =20 set_pgd(pgd, __pgd((pfn << _PAGE_PFN_SHIFT) | _PAGE_TABLE)); @@ -71,7 +71,7 @@ static inline void pgd_populate(struct mm_struct *mm, pgd= _t *pgd, p4d_t *p4d) static inline void pgd_populate_safe(struct mm_struct *mm, pgd_t *pgd, p4d_t *p4d) { - if (pgtable_l5_enabled) { + if (pgtable_l5_enabled()) { unsigned long pfn =3D virt_to_pfn(p4d); =20 set_pgd_safe(pgd, @@ -82,21 +82,21 @@ static inline void pgd_populate_safe(struct mm_struct *= mm, pgd_t *pgd, #define pud_free pud_free static inline void pud_free(struct mm_struct *mm, pud_t *pud) { - if (pgtable_l4_enabled) + if (pgtable_l4_enabled()) __pud_free(mm, pud); } =20 static inline void __pud_free_tlb(struct mmu_gather *tlb, pud_t *pud, unsigned long addr) { - if (pgtable_l4_enabled) + if (pgtable_l4_enabled()) tlb_remove_ptdesc(tlb, virt_to_ptdesc(pud)); } =20 static inline void __p4d_free_tlb(struct mmu_gather *tlb, p4d_t *p4d, unsigned long addr) { - if (pgtable_l5_enabled) + if (pgtable_l5_enabled()) tlb_remove_ptdesc(tlb, virt_to_ptdesc(p4d)); } #endif /* __PAGETABLE_PMD_FOLDED */ diff --git a/arch/riscv/include/asm/pgtable-32.h b/arch/riscv/include/asm/p= gtable-32.h index 00f3369570a8..483a87a4290b 100644 --- a/arch/riscv/include/asm/pgtable-32.h +++ b/arch/riscv/include/asm/pgtable-32.h @@ -33,7 +33,7 @@ _PAGE_WRITE | _PAGE_EXEC | \ _PAGE_USER | _PAGE_GLOBAL)) =20 -static const __maybe_unused int pgtable_l4_enabled; -static const __maybe_unused int pgtable_l5_enabled; +static inline bool pgtable_l4_enabled() { return false; } +static inline bool pgtable_l5_enabled() { return false; } =20 #endif /* _ASM_RISCV_PGTABLE_32_H */ diff --git a/arch/riscv/include/asm/pgtable-64.h b/arch/riscv/include/asm/p= gtable-64.h index 6e789fa58514..72b8c63469fa 100644 --- a/arch/riscv/include/asm/pgtable-64.h +++ b/arch/riscv/include/asm/pgtable-64.h @@ -10,14 +10,24 @@ #include #include =20 -extern bool pgtable_l4_enabled; -extern bool pgtable_l5_enabled; +extern bool _pgtable_l4_enabled; +extern bool _pgtable_l5_enabled; + +static __always_inline bool pgtable_l5_enabled(void) +{ + return _pgtable_l5_enabled; +} + +static __always_inline bool pgtable_l4_enabled(void) +{ + return _pgtable_l4_enabled; +} =20 #define PGDIR_SHIFT_L3 30 #define PGDIR_SHIFT_L4 39 #define PGDIR_SHIFT_L5 48 -#define PGDIR_SHIFT (pgtable_l5_enabled ? PGDIR_SHIFT_L5 : \ - (pgtable_l4_enabled ? PGDIR_SHIFT_L4 : PGDIR_SHIFT_L3)) +#define PGDIR_SHIFT (pgtable_l5_enabled() ? PGDIR_SHIFT_L5 : \ + (pgtable_l4_enabled() ? PGDIR_SHIFT_L4 : PGDIR_SHIFT_L3)) /* Size of region mapped by a page global directory */ #define PGDIR_SIZE (_AC(1, UL) << PGDIR_SHIFT) #define PGDIR_MASK (~(PGDIR_SIZE - 1)) @@ -26,8 +36,8 @@ extern bool pgtable_l5_enabled; #define P4D_SHIFT_L3 30 #define P4D_SHIFT_L4 39 #define P4D_SHIFT_L5 39 -#define P4D_SHIFT (pgtable_l5_enabled ? P4D_SHIFT_L5 : \ - (pgtable_l4_enabled ? P4D_SHIFT_L4 : P4D_SHIFT_L3)) +#define P4D_SHIFT (pgtable_l5_enabled() ? P4D_SHIFT_L5 : \ + (pgtable_l4_enabled() ? P4D_SHIFT_L4 : P4D_SHIFT_L3)) #define P4D_SIZE (_AC(1, UL) << P4D_SHIFT) #define P4D_MASK (~(P4D_SIZE - 1)) =20 @@ -233,7 +243,7 @@ static inline struct page *pud_page(pud_t pud) #define mm_p4d_folded mm_p4d_folded static inline bool mm_p4d_folded(struct mm_struct *mm) { - if (pgtable_l5_enabled) + if (pgtable_l5_enabled()) return false; =20 return true; @@ -242,7 +252,7 @@ static inline bool mm_p4d_folded(struct mm_struct *mm) #define mm_pud_folded mm_pud_folded static inline bool mm_pud_folded(struct mm_struct *mm) { - if (pgtable_l4_enabled) + if (pgtable_l4_enabled()) return false; =20 return true; @@ -275,7 +285,7 @@ static inline unsigned long _pmd_pfn(pmd_t pmd) =20 static inline void set_p4d(p4d_t *p4dp, p4d_t p4d) { - if (pgtable_l4_enabled) + if (pgtable_l4_enabled()) WRITE_ONCE(*p4dp, p4d); else set_pud((pud_t *)p4dp, (pud_t){ p4d_val(p4d) }); @@ -283,7 +293,7 @@ static inline void set_p4d(p4d_t *p4dp, p4d_t p4d) =20 static inline int p4d_none(p4d_t p4d) { - if (pgtable_l4_enabled) + if (pgtable_l4_enabled()) return (p4d_val(p4d) =3D=3D 0); =20 return 0; @@ -291,7 +301,7 @@ static inline int p4d_none(p4d_t p4d) =20 static inline int p4d_present(p4d_t p4d) { - if (pgtable_l4_enabled) + if (pgtable_l4_enabled()) return (p4d_val(p4d) & _PAGE_PRESENT); =20 return 1; @@ -299,7 +309,7 @@ static inline int p4d_present(p4d_t p4d) =20 static inline int p4d_bad(p4d_t p4d) { - if (pgtable_l4_enabled) + if (pgtable_l4_enabled()) return !p4d_present(p4d); =20 return 0; @@ -307,7 +317,7 @@ static inline int p4d_bad(p4d_t p4d) =20 static inline void p4d_clear(p4d_t *p4d) { - if (pgtable_l4_enabled) + if (pgtable_l4_enabled()) set_p4d(p4d, __p4d(0)); } =20 @@ -323,7 +333,7 @@ static inline unsigned long _p4d_pfn(p4d_t p4d) =20 static inline pud_t *p4d_pgtable(p4d_t p4d) { - if (pgtable_l4_enabled) + if (pgtable_l4_enabled()) return (pud_t *)pfn_to_virt(__page_val_to_pfn(p4d_val(p4d))); =20 return (pud_t *)pud_pgtable((pud_t) { p4d_val(p4d) }); @@ -342,7 +352,7 @@ pud_t *pud_offset(p4d_t *p4d, unsigned long address); =20 static inline void set_pgd(pgd_t *pgdp, pgd_t pgd) { - if (pgtable_l5_enabled) + if (pgtable_l5_enabled()) WRITE_ONCE(*pgdp, pgd); else set_p4d((p4d_t *)pgdp, (p4d_t){ pgd_val(pgd) }); @@ -350,7 +360,7 @@ static inline void set_pgd(pgd_t *pgdp, pgd_t pgd) =20 static inline int pgd_none(pgd_t pgd) { - if (pgtable_l5_enabled) + if (pgtable_l5_enabled()) return (pgd_val(pgd) =3D=3D 0); =20 return 0; @@ -358,7 +368,7 @@ static inline int pgd_none(pgd_t pgd) =20 static inline int pgd_present(pgd_t pgd) { - if (pgtable_l5_enabled) + if (pgtable_l5_enabled()) return (pgd_val(pgd) & _PAGE_PRESENT); =20 return 1; @@ -366,7 +376,7 @@ static inline int pgd_present(pgd_t pgd) =20 static inline int pgd_bad(pgd_t pgd) { - if (pgtable_l5_enabled) + if (pgtable_l5_enabled()) return !pgd_present(pgd); =20 return 0; @@ -374,13 +384,13 @@ static inline int pgd_bad(pgd_t pgd) =20 static inline void pgd_clear(pgd_t *pgd) { - if (pgtable_l5_enabled) + if (pgtable_l5_enabled()) set_pgd(pgd, __pgd(0)); } =20 static inline p4d_t *pgd_pgtable(pgd_t pgd) { - if (pgtable_l5_enabled) + if (pgtable_l5_enabled()) return (p4d_t *)pfn_to_virt(__page_val_to_pfn(pgd_val(pgd))); =20 return (p4d_t *)p4d_pgtable((p4d_t) { pgd_val(pgd) }); diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgta= ble.h index 40b1ed4f3ea8..a71671166852 100644 --- a/arch/riscv/include/asm/pgtable.h +++ b/arch/riscv/include/asm/pgtable.h @@ -75,8 +75,8 @@ #define VA_BITS_SV48 48 #define VA_BITS_SV57 57 =20 -#define VA_BITS (pgtable_l5_enabled ? \ - VA_BITS_SV57 : (pgtable_l4_enabled ? VA_BITS_SV48 : VA_BITS_SV39)) +#define VA_BITS (pgtable_l5_enabled() ? \ + VA_BITS_SV57 : (pgtable_l4_enabled() ? VA_BITS_SV48 : VA_BITS_SV39)) #else #define VA_BITS VA_BITS_SV32 #endif diff --git a/arch/riscv/include/asm/vmalloc.h b/arch/riscv/include/asm/vmal= loc.h index fefe94dc98e2..4d8b80ce570b 100644 --- a/arch/riscv/include/asm/vmalloc.h +++ b/arch/riscv/include/asm/vmalloc.h @@ -3,15 +3,14 @@ #define _ASM_RISCV_VMALLOC_H =20 #ifdef CONFIG_HAVE_ARCH_HUGE_VMAP - -extern bool pgtable_l4_enabled, pgtable_l5_enabled; +#include =20 #define IOREMAP_MAX_ORDER (PUD_SHIFT) =20 #define arch_vmap_pud_supported arch_vmap_pud_supported static inline bool arch_vmap_pud_supported(pgprot_t prot) { - return pgtable_l4_enabled || pgtable_l5_enabled; + return pgtable_l4_enabled() || pgtable_l5_enabled(); } =20 #define arch_vmap_pmd_supported arch_vmap_pmd_supported diff --git a/arch/riscv/kernel/cpu.c b/arch/riscv/kernel/cpu.c index 3dbc8cc557dd..af84b97af3a1 100644 --- a/arch/riscv/kernel/cpu.c +++ b/arch/riscv/kernel/cpu.c @@ -292,9 +292,9 @@ static void print_mmu(struct seq_file *f) #if defined(CONFIG_32BIT) sv_type =3D "sv32"; #elif defined(CONFIG_64BIT) - if (pgtable_l5_enabled) + if (pgtable_l5_enabled()) sv_type =3D "sv57"; - else if (pgtable_l4_enabled) + else if (pgtable_l4_enabled()) sv_type =3D "sv48"; else sv_type =3D "sv39"; diff --git a/arch/riscv/mm/init.c b/arch/riscv/mm/init.c index f8994caefc70..fc74142fe6e6 100644 --- a/arch/riscv/mm/init.c +++ b/arch/riscv/mm/init.c @@ -52,10 +52,10 @@ u64 satp_mode __ro_after_init =3D SATP_MODE_32; EXPORT_SYMBOL(satp_mode); =20 #ifdef CONFIG_64BIT -bool pgtable_l4_enabled __ro_after_init =3D true; -bool pgtable_l5_enabled __ro_after_init =3D true; -EXPORT_SYMBOL(pgtable_l4_enabled); -EXPORT_SYMBOL(pgtable_l5_enabled); +bool _pgtable_l4_enabled __ro_after_init =3D true; +bool _pgtable_l5_enabled __ro_after_init =3D true; +EXPORT_SYMBOL(_pgtable_l4_enabled); +EXPORT_SYMBOL(_pgtable_l5_enabled); #endif =20 phys_addr_t phys_ram_base __ro_after_init; @@ -652,23 +652,23 @@ static void __meminit create_p4d_mapping(p4d_t *p4dp,= uintptr_t va, phys_addr_t } =20 #define pgd_next_t p4d_t -#define alloc_pgd_next(__va) (pgtable_l5_enabled ? \ - pt_ops.alloc_p4d(__va) : (pgtable_l4_enabled ? \ +#define alloc_pgd_next(__va) (pgtable_l5_enabled() ? \ + pt_ops.alloc_p4d(__va) : (pgtable_l4_enabled() ? \ pt_ops.alloc_pud(__va) : pt_ops.alloc_pmd(__va))) -#define get_pgd_next_virt(__pa) (pgtable_l5_enabled ? \ - pt_ops.get_p4d_virt(__pa) : (pgd_next_t *)(pgtable_l4_enabled ? \ +#define get_pgd_next_virt(__pa) (pgtable_l5_enabled() ? \ + pt_ops.get_p4d_virt(__pa) : (pgd_next_t *)(pgtable_l4_enabled() ? \ pt_ops.get_pud_virt(__pa) : (pud_t *)pt_ops.get_pmd_virt(__pa))) #define create_pgd_next_mapping(__nextp, __va, __pa, __sz, __prot) \ - (pgtable_l5_enabled ? \ + (pgtable_l5_enabled() ? \ create_p4d_mapping(__nextp, __va, __pa, __sz, __prot) : \ - (pgtable_l4_enabled ? \ + (pgtable_l4_enabled() ? \ create_pud_mapping((pud_t *)__nextp, __va, __pa, __sz, __prot) : \ create_pmd_mapping((pmd_t *)__nextp, __va, __pa, __sz, __prot))) -#define fixmap_pgd_next (pgtable_l5_enabled ? \ - (uintptr_t)fixmap_p4d : (pgtable_l4_enabled ? \ +#define fixmap_pgd_next (pgtable_l5_enabled() ? \ + (uintptr_t)fixmap_p4d : (pgtable_l4_enabled() ? \ (uintptr_t)fixmap_pud : (uintptr_t)fixmap_pmd)) -#define trampoline_pgd_next (pgtable_l5_enabled ? \ - (uintptr_t)trampoline_p4d : (pgtable_l4_enabled ? \ +#define trampoline_pgd_next (pgtable_l5_enabled() ? \ + (uintptr_t)trampoline_p4d : (pgtable_l4_enabled() ? \ (uintptr_t)trampoline_pud : (uintptr_t)trampoline_pmd)) #else #define pgd_next_t pte_t @@ -713,11 +713,11 @@ static uintptr_t __meminit best_map_size(phys_addr_t = pa, uintptr_t va, phys_addr if (debug_pagealloc_enabled()) return PAGE_SIZE; =20 - if (pgtable_l5_enabled && + if (pgtable_l5_enabled() && !(pa & (P4D_SIZE - 1)) && !(va & (P4D_SIZE - 1)) && size >=3D P4D_SIZ= E) return P4D_SIZE; =20 - if (pgtable_l4_enabled && + if (pgtable_l4_enabled() && !(pa & (PUD_SIZE - 1)) && !(va & (PUD_SIZE - 1)) && size >=3D PUD_SIZ= E) return PUD_SIZE; =20 @@ -769,14 +769,14 @@ u64 __pi_set_satp_mode_from_fdt(uintptr_t dtb_pa); =20 static void __init disable_pgtable_l5(void) { - pgtable_l5_enabled =3D false; + _pgtable_l5_enabled =3D false; kernel_map.page_offset =3D PAGE_OFFSET_L4; satp_mode =3D SATP_MODE_48; } =20 static void __init disable_pgtable_l4(void) { - pgtable_l4_enabled =3D false; + _pgtable_l4_enabled =3D false; kernel_map.page_offset =3D PAGE_OFFSET_L3; satp_mode =3D SATP_MODE_39; } @@ -867,7 +867,7 @@ static __init void set_satp_mode(uintptr_t dtb_pa) =20 create_pgd_mapping(early_pg_dir, set_satp_mode_pmd, - pgtable_l5_enabled ? + pgtable_l5_enabled() ? (uintptr_t)early_p4d : (uintptr_t)early_pud, PGDIR_SIZE, PAGE_TABLE); =20 @@ -879,7 +879,7 @@ static __init void set_satp_mode(uintptr_t dtb_pa) local_flush_tlb_all(); =20 if (hw_satp !=3D identity_satp) { - if (pgtable_l5_enabled) { + if (pgtable_l5_enabled()) { disable_pgtable_l5(); memset(early_pg_dir, 0, PAGE_SIZE); goto retry; @@ -1120,11 +1120,11 @@ asmlinkage void __init setup_vm(uintptr_t dtb_pa) =20 #ifndef __PAGETABLE_PMD_FOLDED /* Setup fixmap P4D and PUD */ - if (pgtable_l5_enabled) + if (pgtable_l5_enabled()) create_p4d_mapping(fixmap_p4d, FIXADDR_START, (uintptr_t)fixmap_pud, P4D_SIZE, PAGE_TABLE); /* Setup fixmap PUD and PMD */ - if (pgtable_l4_enabled) + if (pgtable_l4_enabled()) create_pud_mapping(fixmap_pud, FIXADDR_START, (uintptr_t)fixmap_pmd, PUD_SIZE, PAGE_TABLE); create_pmd_mapping(fixmap_pmd, FIXADDR_START, @@ -1132,10 +1132,10 @@ asmlinkage void __init setup_vm(uintptr_t dtb_pa) /* Setup trampoline PGD and PMD */ create_pgd_mapping(trampoline_pg_dir, kernel_map.virt_addr, trampoline_pgd_next, PGDIR_SIZE, PAGE_TABLE); - if (pgtable_l5_enabled) + if (pgtable_l5_enabled()) create_p4d_mapping(trampoline_p4d, kernel_map.virt_addr, (uintptr_t)trampoline_pud, P4D_SIZE, PAGE_TABLE); - if (pgtable_l4_enabled) + if (pgtable_l4_enabled()) create_pud_mapping(trampoline_pud, kernel_map.virt_addr, (uintptr_t)trampoline_pmd, PUD_SIZE, PAGE_TABLE); create_pmd_mapping(trampoline_pmd, kernel_map.virt_addr, @@ -1406,7 +1406,7 @@ static void __init preallocate_pgd_pages_range(unsign= ed long start, unsigned lon if (!p4d) goto failed; =20 - if (pgtable_l5_enabled) + if (pgtable_l5_enabled()) continue; =20 lvl =3D "pud"; @@ -1414,7 +1414,7 @@ static void __init preallocate_pgd_pages_range(unsign= ed long start, unsigned lon if (!pud) goto failed; =20 - if (pgtable_l4_enabled) + if (pgtable_l4_enabled()) continue; =20 lvl =3D "pmd"; @@ -1624,13 +1624,13 @@ static void __meminit remove_pud_mapping(pud_t *pud= _base, unsigned long addr, un =20 for (; addr < end; addr =3D next) { next =3D pud_addr_end(addr, end); - pudp =3D pgtable_l4_enabled ? pud_base + pud_index(addr) : pud_base; + pudp =3D pgtable_l4_enabled() ? pud_base + pud_index(addr) : pud_base; pud =3D pudp_get(pudp); if (!pud_present(pud)) continue; =20 if (pud_leaf(pud)) { - if (pgtable_l4_enabled) { + if (pgtable_l4_enabled()) { pud_clear(pudp); if (is_vmemmap) free_vmemmap_storage(pud_page(pud), PUD_SIZE, altmap); @@ -1641,7 +1641,7 @@ static void __meminit remove_pud_mapping(pud_t *pud_b= ase, unsigned long addr, un pmd_base =3D pmd_offset(pudp, 0); remove_pmd_mapping(pmd_base, addr, next, is_vmemmap, altmap); =20 - if (pgtable_l4_enabled) + if (pgtable_l4_enabled()) free_pmd_table(pmd_base, pudp, is_vmemmap); } } @@ -1655,13 +1655,13 @@ static void __meminit remove_p4d_mapping(p4d_t *p4d= _base, unsigned long addr, un =20 for (; addr < end; addr =3D next) { next =3D p4d_addr_end(addr, end); - p4dp =3D pgtable_l5_enabled ? p4d_base + p4d_index(addr) : p4d_base; + p4dp =3D pgtable_l5_enabled() ? p4d_base + p4d_index(addr) : p4d_base; p4d =3D p4dp_get(p4dp); if (!p4d_present(p4d)) continue; =20 if (p4d_leaf(p4d)) { - if (pgtable_l5_enabled) { + if (pgtable_l5_enabled()) { p4d_clear(p4dp); if (is_vmemmap) free_vmemmap_storage(p4d_page(p4d), P4D_SIZE, altmap); @@ -1672,7 +1672,7 @@ static void __meminit remove_p4d_mapping(p4d_t *p4d_b= ase, unsigned long addr, un pud_base =3D pud_offset(p4dp, 0); remove_pud_mapping(pud_base, addr, next, is_vmemmap, altmap); =20 - if (pgtable_l5_enabled) + if (pgtable_l5_enabled()) free_pud_table(pud_base, p4dp); } } diff --git a/arch/riscv/mm/kasan_init.c b/arch/riscv/mm/kasan_init.c index 1f3aa9611187..84f87c8277c3 100644 --- a/arch/riscv/mm/kasan_init.c +++ b/arch/riscv/mm/kasan_init.c @@ -168,7 +168,7 @@ static void __init kasan_early_clear_pud(p4d_t *p4dp, pud_t *pudp, *base_pud; unsigned long next; =20 - if (!pgtable_l4_enabled) { + if (!pgtable_l4_enabled()) { pudp =3D (pud_t *)p4dp; } else { base_pud =3D pt_ops.get_pud_virt(pfn_to_phys(_p4d_pfn(p4dp_get(p4dp)))); @@ -193,7 +193,7 @@ static void __init kasan_early_clear_p4d(pgd_t *pgdp, p4d_t *p4dp, *base_p4d; unsigned long next; =20 - if (!pgtable_l5_enabled) { + if (!pgtable_l5_enabled()) { p4dp =3D (p4d_t *)pgdp; } else { base_p4d =3D pt_ops.get_p4d_virt(pfn_to_phys(_pgd_pfn(pgdp_get(pgdp)))); @@ -203,7 +203,7 @@ static void __init kasan_early_clear_p4d(pgd_t *pgdp, do { next =3D p4d_addr_end(vaddr, end); =20 - if (pgtable_l4_enabled && IS_ALIGNED(vaddr, P4D_SIZE) && + if (pgtable_l4_enabled() && IS_ALIGNED(vaddr, P4D_SIZE) && (next - vaddr) >=3D P4D_SIZE) { p4d_clear(p4dp); continue; @@ -221,7 +221,7 @@ static void __init kasan_early_clear_pgd(pgd_t *pgdp, do { next =3D pgd_addr_end(vaddr, end); =20 - if (pgtable_l5_enabled && IS_ALIGNED(vaddr, PGDIR_SIZE) && + if (pgtable_l5_enabled() && IS_ALIGNED(vaddr, PGDIR_SIZE) && (next - vaddr) >=3D PGDIR_SIZE) { pgd_clear(pgdp); continue; @@ -239,7 +239,7 @@ static void __init kasan_early_populate_pud(p4d_t *p4dp, phys_addr_t phys_addr; unsigned long next; =20 - if (!pgtable_l4_enabled) { + if (!pgtable_l4_enabled()) { pudp =3D (pud_t *)p4dp; } else { base_pud =3D pt_ops.get_pud_virt(pfn_to_phys(_p4d_pfn(p4dp_get(p4dp)))); @@ -277,7 +277,7 @@ static void __init kasan_early_populate_p4d(pgd_t *pgdp, * Note that this test is then completely equivalent to * p4dp =3D p4d_offset(pgdp, vaddr) */ - if (!pgtable_l5_enabled) { + if (!pgtable_l5_enabled()) { p4dp =3D (p4d_t *)pgdp; } else { base_p4d =3D pt_ops.get_p4d_virt(pfn_to_phys(_pgd_pfn(pgdp_get(pgdp)))); @@ -336,7 +336,7 @@ asmlinkage void __init kasan_early_init(void) (__pa((uintptr_t)kasan_early_shadow_pte)), PAGE_TABLE)); =20 - if (pgtable_l4_enabled) { + if (pgtable_l4_enabled()) { for (i =3D 0; i < PTRS_PER_PUD; ++i) set_pud(kasan_early_shadow_pud + i, pfn_pud(PFN_DOWN @@ -344,7 +344,7 @@ asmlinkage void __init kasan_early_init(void) PAGE_TABLE)); } =20 - if (pgtable_l5_enabled) { + if (pgtable_l5_enabled()) { for (i =3D 0; i < PTRS_PER_P4D; ++i) set_p4d(kasan_early_shadow_p4d + i, pfn_p4d(PFN_DOWN @@ -461,7 +461,7 @@ static void __init create_tmp_mapping(void) memcpy(tmp_pg_dir, swapper_pg_dir, sizeof(pgd_t) * PTRS_PER_PGD); =20 /* Copy the last p4d since it is shared with the kernel mapping. */ - if (pgtable_l5_enabled) { + if (pgtable_l5_enabled()) { ptr =3D (p4d_t *)pgd_page_vaddr(pgdp_get(pgd_offset_k(KASAN_SHADOW_END))= ); memcpy(tmp_p4d, ptr, sizeof(p4d_t) * PTRS_PER_P4D); set_pgd(&tmp_pg_dir[pgd_index(KASAN_SHADOW_END)], @@ -472,7 +472,7 @@ static void __init create_tmp_mapping(void) } =20 /* Copy the last pud since it is shared with the kernel mapping. */ - if (pgtable_l4_enabled) { + if (pgtable_l4_enabled()) { ptr =3D (pud_t *)p4d_page_vaddr(p4dp_get(base_p4d + p4d_index(KASAN_SHAD= OW_END))); memcpy(tmp_pud, ptr, sizeof(pud_t) * PTRS_PER_PUD); set_p4d(&base_p4d[p4d_index(KASAN_SHADOW_END)], diff --git a/arch/riscv/mm/pgtable.c b/arch/riscv/mm/pgtable.c index 9c4427d0b187..cd180f9abb5e 100644 --- a/arch/riscv/mm/pgtable.c +++ b/arch/riscv/mm/pgtable.c @@ -41,7 +41,7 @@ EXPORT_SYMBOL_GPL(ptep_test_and_clear_young); #ifdef CONFIG_64BIT pud_t *pud_offset(p4d_t *p4d, unsigned long address) { - if (pgtable_l4_enabled) + if (pgtable_l4_enabled()) return p4d_pgtable(p4dp_get(p4d)) + pud_index(address); =20 return (pud_t *)p4d; @@ -50,7 +50,7 @@ EXPORT_SYMBOL_GPL(pud_offset); =20 p4d_t *p4d_offset(pgd_t *pgd, unsigned long address) { - if (pgtable_l5_enabled) + if (pgtable_l5_enabled()) return pgd_pgtable(pgdp_get(pgd)) + p4d_index(address); =20 return (p4d_t *)pgd; diff --git a/arch/riscv/mm/ptdump.c b/arch/riscv/mm/ptdump.c index f4b4a9fcbbd8..f765a63e2ca7 100644 --- a/arch/riscv/mm/ptdump.c +++ b/arch/riscv/mm/ptdump.c @@ -446,8 +446,8 @@ static int __init ptdump_init(void) =20 kernel_ptd_info.base_addr =3D KERN_VIRT_START; =20 - pg_level[1].name =3D pgtable_l5_enabled ? "P4D" : "PGD"; - pg_level[2].name =3D pgtable_l4_enabled ? "PUD" : "PGD"; + pg_level[1].name =3D pgtable_l5_enabled() ? "P4D" : "PGD"; + pg_level[2].name =3D pgtable_l4_enabled() ? "PUD" : "PGD"; =20 for (i =3D 0; i < ARRAY_SIZE(pg_level); i++) for (j =3D 0; j < ARRAY_SIZE(pte_bits); j++) --=20 2.53.0 From nobody Sat Sep 26 20:27:49 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 87A0D3C1991 for ; Sun, 30 Aug 2026 17:24:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788110648; cv=none; b=reObmpvpMquBbd7gdu8RmYpG5Rc8ntSLgdDQ53Ll0VQDr+4zcufZLt7P145PnDusyAJGaSgZgfT/KSQaGMFD5+d4Vy3K6svWOn/EAQGuORrDgj8cYOtza//NNXdJp20tww8VQGf8G2AorWaQ88UcxGc/V3OS07hECxPUN50umGY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788110648; c=relaxed/simple; bh=iQcqNEQOT/K9fnrEGIQ6j5eVfV7bAE2OEaIYoDLpmk0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=R6QbuQ8HcPN7/DlO8+URitHi4oRXCzmo+Xa9E9dLTYuPnz8R+5kZhzwO+2HukE2wyEwu4eQoXtzWX9gseWMEz2QXVquwruw03KMKlrO77fwJgevXXWE93sMHDdSztqLMjWURHRiyvCfnFv4EBY6Coj1ssLN13PB3FEarGXsohJE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=mV9yI1Bw; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="mV9yI1Bw" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 363D81F00ACA; Sun, 30 Aug 2026 17:24:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1788110644; bh=7vEavxeDlD+SBN+P8+60a1sNGZK4Q3B0tECP75Nnt6M=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=mV9yI1BwtEwrWa9LcmSZHsTurdV6r7ojnxpfuPAsp/CNJr4pkVHvU89EVYoNR9QhJ iMeKqwwkuqd8DOoqnTtZaGdxXkGvc2OWeqJhuoUOZ7SOchLaPnHTRjqT1H9mw2VEVZ K2OLP60DBr1f+Q5ECQv7ML+cEPzqPVp0tOy5K8Q20W2U7ZJpQWXe5s56b4RKCKhZ1T 79Mfq5qLjbP0y2Ji6eZeTTfzHf2DF5DeE11yL9l1nsruPP8zDvnLoyM+PKjJj4Pgl8 /zwk6vY7J4MIg5KWrkP3JhVLqsmap7zdollvOpv6h9vE3TCrZn6Qey0ewrQXO+atI9 EQRBq9vLxaWPA== From: Jisheng Zhang To: Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexandre Ghiti Cc: linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH 3/3] riscv: introduce cap framework and use it to optimize pgtable_l4|l5_enabled Date: Mon, 31 Aug 2026 01:04:01 +0800 Message-ID: <20260830170414.4505-4-jszhang@kernel.org> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260830170414.4505-1-jszhang@kernel.org> References: <20260830170414.4505-1-jszhang@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable The pgtable_l4|[l5]_enabled check sits at hot code path, performance is impacted a lot. Since pgtable_l4|[l5]_enabled isn't changed after boot, we can use alternative mechanism to optimize them. So the question is whether we can add RISCV_ISA_EXT_SV48/SV5 and use riscv_has_extension_*() or not. Although, per [1] and [2], SV48 and SV57 are ISA exensions too, RISCV_ISA_EXT_SV48/SV57 are to describe hw supported extensions, while this doesn't mean the pgtable_l4|l5 is enabled, for example, we may pass no5lvl/no4lvl kernel boot args or explicitly ask for SV39 by setting dt mmu-type property as "riscv,sv39". If we clear RISCV_ISA_EXT_SV48|SV57, then internal extension queries and potentially userspace reporting can no longer distinguish =E2=80=9Cunsupported=E2=80=9D from =E2=80=9Csupported but disab= led.=E2=80=9D Introduce cap framework to describe the capabilities selected by kernel. It also uses similar alternatives mechanism as the riscv_has_extension_*() helpers. After that, use it to optimize pgtable_l4|l5_enabled. For the typical access_ok(addr, 1); before the patch: ... auipc a5,0xb43 lbu a5,100(a5) # ffffffff80b51f68 bnez a5,ffffffff8000ef46 auipc a5,0xb43 lbu a5,91(a5) # ffffffff80b51f69 beqz a5,ffffffff8000ef5a ... after the patch: there are only two j or nop instructions which avoid memory load and test branch. Initial test lmbench's lat_syscall write on TH1520 platforms shows that the write syscall latency is reduced by about 2.38%. Signed-off-by: Jisheng Zhang Link: https://github.com/riscv/riscv-isa-manual/blob/main/src/profiles/prof= iles.adoc [1] Link: https://riscv.atlassian.net/wiki/spaces/HOME/pages/16154732/Ratified+= ISA+Extensions [2] --- arch/riscv/Kconfig | 1 + arch/riscv/include/asm/alternative.h | 2 +- arch/riscv/include/asm/cpufeature-macros.h | 46 ++++++++++++++++++++++ arch/riscv/include/asm/cpufeature.h | 2 + arch/riscv/include/asm/hwcap.h | 6 +++ arch/riscv/include/asm/pgtable-64.h | 12 ++++++ arch/riscv/kernel/alternative.c | 24 +++++++---- arch/riscv/kernel/cpufeature.c | 41 +++++++++++++++---- arch/riscv/mm/init.c | 9 +++++ 9 files changed, 128 insertions(+), 15 deletions(-) diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index 13b7bb77087e..e9476b8cbeb0 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -218,6 +218,7 @@ config RISCV select PCI_ECAM if (ACPI && PCI) select PCI_MSI if PCI select RELOCATABLE if !MMU && !PHYS_RAM_BASE_FIXED + select RISCV_ALTERNATIVE_EARLY if 64BIT select RISCV_APLIC select RISCV_IMSIC select RISCV_INTC diff --git a/arch/riscv/include/asm/alternative.h b/arch/riscv/include/asm/= alternative.h index 688c7d1a9ae3..6be7b2b6ade9 100644 --- a/arch/riscv/include/asm/alternative.h +++ b/arch/riscv/include/asm/alternative.h @@ -33,7 +33,7 @@ void __init apply_early_boot_alternatives(void); void apply_module_alternatives(void *start, size_t length); =20 void riscv_alternative_fix_offsets(void *alt_ptr, unsigned int len, - int patch_offset); + int patch_offset, bool early); =20 struct alt_entry { s32 old_offset; /* offset relative to original instruction or data */ diff --git a/arch/riscv/include/asm/cpufeature-macros.h b/arch/riscv/includ= e/asm/cpufeature-macros.h index adaf9e3fb25c..dca320e7e88c 100644 --- a/arch/riscv/include/asm/cpufeature-macros.h +++ b/arch/riscv/include/asm/cpufeature-macros.h @@ -1,6 +1,7 @@ /* SPDX-License-Identifier: GPL-2.0-only */ /* * Copyright 2022-2024 Rivos, Inc + * Copyright 2026 Jisheng Zhang */ =20 #ifndef _ASM_CPUFEATURE_MACROS_H @@ -57,4 +58,49 @@ static __always_inline bool riscv_has_extension_likely(c= onst unsigned long ext) return __riscv_has_extension_likely(STANDARD_EXT, ext); } =20 +static __always_inline bool __riscv_has_cap_likely(const unsigned long cap) +{ + asm goto(ALTERNATIVE("j %l[l_no]", "nop", 0, %[cap], 1) + : + : [cap] "i" (cap) + : + : l_no); + + return true; +l_no: + return false; +} + +static __always_inline bool __riscv_has_cap_unlikely(const unsigned long c= ap) +{ + + asm goto(ALTERNATIVE("nop", "j %l[l_yes]", 0, %[cap], 1) + : + : [cap] "i" (cap) + : + : l_yes); + + return false; +l_yes: + return true; +} + +static __always_inline bool riscv_has_cap_unlikely(const unsigned long cap) +{ + compiletime_assert(cap >=3D RISCV_ISA_EXT_MAX && + cap < RISCV_CAP_MAX, + "cap must be >=3D RISCV_ISA_EXT_MAX and < RISCV_CAP_MAX"); + + return __riscv_has_cap_unlikely(cap); +} + +static __always_inline bool riscv_has_cap_likely(const unsigned long cap) +{ + compiletime_assert(cap >=3D RISCV_ISA_EXT_MAX && + cap < RISCV_CAP_MAX, + "cap must be >=3D RISCV_ISA_EXT_MAX and < RISCV_CAP_MAX"); + + return __riscv_has_cap_likely(cap); +} + #endif /* _ASM_CPUFEATURE_MACROS_H */ diff --git a/arch/riscv/include/asm/cpufeature.h b/arch/riscv/include/asm/c= pufeature.h index 37c9f2a0fb54..50453586558e 100644 --- a/arch/riscv/include/asm/cpufeature.h +++ b/arch/riscv/include/asm/cpufeature.h @@ -36,6 +36,8 @@ extern const struct seq_operations cpuinfo_op; /* Per-cpu ISA extensions. */ extern struct riscv_isainfo hart_isa[NR_CPUS]; =20 +extern DECLARE_BITMAP(riscv_cap, RISCV_CAP_MAX - RISCV_ISA_EXT_MAX); + extern u32 thead_vlenb_of; =20 void __init riscv_user_isa_enable(void); diff --git a/arch/riscv/include/asm/hwcap.h b/arch/riscv/include/asm/hwcap.h index f8db798b2654..4a3e2c43afc4 100644 --- a/arch/riscv/include/asm/hwcap.h +++ b/arch/riscv/include/asm/hwcap.h @@ -128,6 +128,12 @@ #define RISCV_ISA_EXT_MAX 128 #define RISCV_ISA_EXT_INVALID U32_MAX =20 +/* Kernel capabilities occupy the gap below vendor alternative IDs. */ +#define RISCV_CAP_PGTABLE_L4 RISCV_ISA_EXT_MAX +#define RISCV_CAP_PGTABLE_L5 (RISCV_ISA_EXT_MAX + 1) +#define RISCV_CAP_EARLY_MAX (RISCV_ISA_EXT_MAX + 2) +#define RISCV_CAP_MAX RISCV_CAP_EARLY_MAX + #ifdef CONFIG_RISCV_M_MODE #define RISCV_ISA_EXT_SxAIA RISCV_ISA_EXT_SMAIA #define RISCV_ISA_EXT_SUPM RISCV_ISA_EXT_SMNPM diff --git a/arch/riscv/include/asm/pgtable-64.h b/arch/riscv/include/asm/p= gtable-64.h index 72b8c63469fa..b0f59e3d5c31 100644 --- a/arch/riscv/include/asm/pgtable-64.h +++ b/arch/riscv/include/asm/pgtable-64.h @@ -13,6 +13,7 @@ extern bool _pgtable_l4_enabled; extern bool _pgtable_l5_enabled; =20 +#ifdef USE_EARLY_PGTABLE_LEVELS static __always_inline bool pgtable_l5_enabled(void) { return _pgtable_l5_enabled; @@ -22,6 +23,17 @@ static __always_inline bool pgtable_l4_enabled(void) { return _pgtable_l4_enabled; } +#else +static __always_inline bool pgtable_l4_enabled(void) +{ + return riscv_has_cap_likely(RISCV_CAP_PGTABLE_L4); +} + +static __always_inline bool pgtable_l5_enabled(void) +{ + return riscv_has_cap_likely(RISCV_CAP_PGTABLE_L5); +} +#endif =20 #define PGDIR_SHIFT_L3 30 #define PGDIR_SHIFT_L4 39 diff --git a/arch/riscv/kernel/alternative.c b/arch/riscv/kernel/alternativ= e.c index c0c9306022c5..bbb215349452 100644 --- a/arch/riscv/kernel/alternative.c +++ b/arch/riscv/kernel/alternative.c @@ -75,7 +75,8 @@ static u32 riscv_instruction_at(void *p) } =20 static void riscv_alternative_fix_auipc_jalr(void *ptr, u32 auipc_insn, - u32 jalr_insn, int patch_offset) + u32 jalr_insn, int patch_offset, + bool early) { u32 call[2] =3D { auipc_insn, jalr_insn }; s32 imm; @@ -88,10 +89,15 @@ static void riscv_alternative_fix_auipc_jalr(void *ptr,= u32 auipc_insn, riscv_insn_insert_utype_itype_imm(&call[0], &call[1], imm); =20 /* patch the call place again */ - patch_text_nosync(ptr, call, sizeof(u32) * 2); + if (early) { + memcpy(ptr, call, sizeof(call)); + } else { + patch_text_nosync(ptr, call, sizeof(call)); + } } =20 -static void riscv_alternative_fix_jal(void *ptr, u32 jal_insn, int patch_o= ffset) +static void riscv_alternative_fix_jal(void *ptr, u32 jal_insn, int patch_o= ffset, + bool early) { s32 imm; =20 @@ -103,11 +109,14 @@ static void riscv_alternative_fix_jal(void *ptr, u32 = jal_insn, int patch_offset) riscv_insn_insert_jtype_imm(&jal_insn, imm); =20 /* patch the call place again */ - patch_text_nosync(ptr, &jal_insn, sizeof(u32)); + if (early) + memcpy(ptr, &jal_insn, sizeof(u32)); + else + patch_text_nosync(ptr, &jal_insn, sizeof(u32)); } =20 void riscv_alternative_fix_offsets(void *alt_ptr, unsigned int len, - int patch_offset) + int patch_offset, bool early) { int num_insn =3D len / sizeof(u32); int i; @@ -131,7 +140,8 @@ void riscv_alternative_fix_offsets(void *alt_ptr, unsig= ned int len, continue; =20 riscv_alternative_fix_auipc_jalr(alt_ptr + i * sizeof(u32), - insn, insn2, patch_offset); + insn, insn2, patch_offset, + early); i++; } =20 @@ -144,7 +154,7 @@ void riscv_alternative_fix_offsets(void *alt_ptr, unsig= ned int len, continue; =20 riscv_alternative_fix_jal(alt_ptr + i * sizeof(u32), - insn, patch_offset); + insn, patch_offset, early); } } } diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c index 9915121e9438..9b47fea25ee4 100644 --- a/arch/riscv/kernel/cpufeature.c +++ b/arch/riscv/kernel/cpufeature.c @@ -40,6 +40,7 @@ unsigned long elf_hwcap __read_mostly; =20 /* Host ISA bitmap */ static DECLARE_BITMAP(riscv_isa, RISCV_ISA_EXT_MAX) __read_mostly; +DECLARE_BITMAP(riscv_cap, RISCV_CAP_MAX - RISCV_ISA_EXT_MAX) __read_mostly; =20 /* Per-cpu ISA extensions. */ struct riscv_isainfo hart_isa[NR_CPUS]; @@ -81,6 +82,14 @@ bool __riscv_isa_extension_available(const unsigned long= *isa_bitmap, unsigned i } EXPORT_SYMBOL_GPL(__riscv_isa_extension_available); =20 +static bool __riscv_cap_available(unsigned int bit) +{ + if (bit >=3D RISCV_CAP_MAX || bit < RISCV_ISA_EXT_MAX) + return false; + + return test_bit(bit - RISCV_ISA_EXT_MAX, riscv_cap); +} + static int riscv_ext_f_depends(const struct riscv_isa_ext_data *data, const unsigned long *isa_bitmap) { @@ -1257,9 +1266,7 @@ void __init_or_module riscv_cpufeature_patch_func(str= uct alt_entry *begin, struct alt_entry *alt; void *oldptr, *altptr; u16 id, value, vendor; - - if (stage =3D=3D RISCV_ALTERNATIVES_EARLY_BOOT) - return; + bool early =3D stage =3D=3D RISCV_ALTERNATIVES_EARLY_BOOT; =20 for (alt =3D begin; alt < end; alt++) { id =3D PATCH_ID_CPUFEATURE_ID(alt->patch_id); @@ -1274,6 +1281,8 @@ void __init_or_module riscv_cpufeature_patch_func(str= uct alt_entry *begin, * vendor extension. */ if (id < RISCV_ISA_EXT_MAX) { + if (early) + continue; /* * This patch should be treated as errata so skip * processing here. @@ -1287,7 +1296,14 @@ void __init_or_module riscv_cpufeature_patch_func(st= ruct alt_entry *begin, value =3D PATCH_ID_CPUFEATURE_VALUE(alt->patch_id); if (!riscv_cpufeature_patch_check(id, value)) continue; + } else if (id < RISCV_CAP_MAX) { + if (id >=3D RISCV_CAP_EARLY_MAX && early) + continue; + if (!__riscv_cap_available(id)) + continue; } else if (id >=3D RISCV_VENDOR_EXT_ALTERNATIVES_BASE) { + if (early) + continue; if (!__riscv_isa_vendor_extension_available(VENDOR_EXT_ALL_CPUS, vendor, id - RISCV_VENDOR_EXT_ALTERNATIVES_BASE)) continue; @@ -1299,9 +1315,20 @@ void __init_or_module riscv_cpufeature_patch_func(st= ruct alt_entry *begin, oldptr =3D ALT_OLD_PTR(alt); altptr =3D ALT_ALT_PTR(alt); =20 - mutex_lock(&text_mutex); - patch_text_nosync(oldptr, altptr, alt->alt_len); - riscv_alternative_fix_offsets(oldptr, alt->alt_len, oldptr - altptr); - mutex_unlock(&text_mutex); + if (early) { + /* oldptr is writable through the MMU-off kernel mapping. */ + memcpy(oldptr, altptr, alt->alt_len); + riscv_alternative_fix_offsets(oldptr, alt->alt_len, + oldptr - altptr, true); + } else { + mutex_lock(&text_mutex); + patch_text_nosync(oldptr, altptr, alt->alt_len); + riscv_alternative_fix_offsets(oldptr, alt->alt_len, + oldptr - altptr, false); + mutex_unlock(&text_mutex); + } } + + if (early) + local_flush_icache_all(); } diff --git a/arch/riscv/mm/init.c b/arch/riscv/mm/init.c index fc74142fe6e6..aca09992ef06 100644 --- a/arch/riscv/mm/init.c +++ b/arch/riscv/mm/init.c @@ -6,6 +6,11 @@ * Nick Kossifidis */ =20 +#ifdef CONFIG_64BIT +/* riscv_has_cap_likely() cannot be used this early */ +#define USE_EARLY_PGTABLE_LEVELS +#endif + #include #include #include @@ -892,6 +897,10 @@ static __init void set_satp_mode(uintptr_t dtb_pa) memset(early_p4d, 0, PAGE_SIZE); memset(early_pud, 0, PAGE_SIZE); memset(early_pmd, 0, PAGE_SIZE); + if (pgtable_l4_enabled()) + set_bit(RISCV_CAP_PGTABLE_L4 - RISCV_ISA_EXT_MAX, riscv_cap); + if (pgtable_l5_enabled()) + set_bit(RISCV_CAP_PGTABLE_L5 - RISCV_ISA_EXT_MAX, riscv_cap); } #endif =20 --=20 2.53.0