From nobody Fri Sep 25 23:08:55 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A7C7C50AC3E for ; Mon, 7 Sep 2026 15:34:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788795277; cv=none; b=iB7T0I24lz1ww/1QdJKLiWW7o3vzB+jZAHtWMqrp+kRhOcjq4pqziGTwEUaJiJIHvIVZxvLiJB6anfm2MdMAys3ywdHd/LmnqtPQpWVBoGKasoTucvyu7x+hTCqgjWR85QhNzi38qfit12OYPx9GMCtOoT8bSVg628brsvtvF9w= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788795277; c=relaxed/simple; bh=XaLS4nfXqVy84eCnC7lh5/Z+hlX4pQRvnZIjNy3Ttnk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=PPwwExrZh3bgc+NvBF1EMh/At0VDh7V/oQMcqig9PH6BhhAwDaqbNZZoZqHjkaETiQ5eVnAG9z0Wu87rIUMkTPM208rGpVqlBuGtEQDidmmR+rT/+HwxekRcHcMVwET9Nal4/ycbSOzRVEO7L/eQAkEBIYPRd+9mrQt9zG9otVY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=TURDlmqu; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="TURDlmqu" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9F5151F00A3D; Mon, 7 Sep 2026 15:34:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1788795272; bh=r0Za95NxwjXTwT3tld/1tYydLeQeiEWwl1b59iKEheQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=TURDlmquGsKNYqmsNPbCPkCcUDihn9823SLB+grDmMY7+P/TuKMZr/8IQ3aE/BMCo bug1vHbMRepKKndsrfHhcNAHlQJ7VQHn6dBYIIC4U6u7E9fVR//kOLkwRGAu6ynIOA AeJYnkx0y4H+2GhSCYrNLAWRvwMjfH83a1dE+OLavRmrCmI6i2PA/Ia1dO8yw+ir2o Wsb3r8JiFG112BrYS+3OCryc2ECNPtGzmT6upLxLwTHKCgXIqRu31JO+VtM7vkty1w aagGz7cEViBIcGQO1YfYVoiY6e5L6WEsQIegw1eJSzBJVELqEGK/5i75KlAKONDseV ZcTpkUWrldmbw== From: Jisheng Zhang To: Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexandre Ghiti , Andrey Ryabinin , Alexander Potapenko , Andrey Konovalov , Dmitry Vyukov , Vincenzo Frascino Cc: linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com Subject: [PATCH v2 1/5] riscv: remove RISCV_ALTERNATIVE Kconfig option Date: Mon, 7 Sep 2026 23:14:33 +0800 Message-ID: <20260907151437.7603-2-jszhang@kernel.org> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260907151437.7603-1-jszhang@kernel.org> References: <20260907151437.7603-1-jszhang@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" riscv always selects RISCV_ALTERNATIVE now, so we can remove this Kconfig option and enable RISCV_ALTERNATIVE code unconditionally. Signed-off-by: Jisheng Zhang --- arch/riscv/Kconfig | 23 -------------------- arch/riscv/Kconfig.errata | 5 +---- arch/riscv/include/asm/alternative-macros.h | 24 --------------------- arch/riscv/include/asm/alternative.h | 10 --------- arch/riscv/include/asm/cpufeature-macros.h | 10 ++------- arch/riscv/include/asm/cpufeature.h | 6 ++---- arch/riscv/include/asm/vendor_extensions.h | 18 ++++++---------- arch/riscv/kernel/Makefile | 2 +- arch/riscv/kernel/cpufeature.c | 2 -- 9 files changed, 12 insertions(+), 88 deletions(-) diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index f8e26c4bed2b..13b7bb77087e 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -218,7 +218,6 @@ config RISCV select PCI_ECAM if (ACPI && PCI) select PCI_MSI if PCI select RELOCATABLE if !MMU && !PHYS_RAM_BASE_FIXED - select RISCV_ALTERNATIVE select RISCV_APLIC select RISCV_IMSIC select RISCV_INTC @@ -539,17 +538,8 @@ config RISCV_COMBO_SPINLOCKS =20 endchoice =20 -config RISCV_ALTERNATIVE - bool - help - This Kconfig allows the kernel to automatically patch the - erratum or cpufeature required by the execution platform at run - time. The code patching overhead is minimal, as it's only done - once at boot and once on each module load. - config RISCV_ALTERNATIVE_EARLY bool - depends on RISCV_ALTERNATIVE help Allows early patching of the kernel for special errata =20 @@ -578,7 +568,6 @@ config RISCV_ISA_SUPM config RISCV_ISA_SVNAPOT bool "Svnapot extension support for supervisor mode NAPOT pages" depends on 64BIT && MMU - depends on RISCV_ALTERNATIVE default y help Enable support for the Svnapot ISA-extension when it is detected @@ -615,7 +604,6 @@ config RISCV_ISA_SSQOSID config RISCV_ISA_SVPBMT bool "Svpbmt extension support for supervisor mode page-based memory type= s" depends on 64BIT && MMU - depends on RISCV_ALTERNATIVE default y help Add support for the Svpbmt ISA-extension (Supervisor-mode: @@ -685,7 +673,6 @@ config RISCV_ISA_V_PREEMPTIVE =20 config RISCV_ISA_ZAWRS bool "Zawrs extension support for more efficient busy waiting" - depends on RISCV_ALTERNATIVE default y help The Zawrs extension defines instructions to be used in polling loops @@ -704,7 +691,6 @@ config TOOLCHAIN_HAS_ZABHA config RISCV_ISA_ZABHA bool "Zabha extension support for atomic byte/halfword operations" depends on TOOLCHAIN_HAS_ZABHA - depends on RISCV_ALTERNATIVE default y help Enable the use of the Zabha ISA-extension to implement kernel @@ -721,7 +707,6 @@ config TOOLCHAIN_HAS_ZACAS =20 config RISCV_ISA_ZACAS bool "Zacas extension support for atomic CAS" - depends on RISCV_ALTERNATIVE default y help Enable the use of the Zacas ISA-extension to implement kernel atomic @@ -766,7 +751,6 @@ config RISCV_ISA_ZBA =20 config RISCV_ISA_ZBB bool "Zbb extension support for bit manipulation instructions" - depends on RISCV_ALTERNATIVE default y help Add support for enabling optimisations in the kernel when the @@ -791,7 +775,6 @@ config RISCV_ISA_ZBC bool "Zbc extension support for carry-less multiplication instructions" depends on TOOLCHAIN_HAS_ZBC depends on MMU - depends on RISCV_ALTERNATIVE default y help Adds support to dynamically detect the presence of the Zbc @@ -813,7 +796,6 @@ config TOOLCHAIN_HAS_ZBKB config RISCV_ISA_ZBKB bool "Zbkb extension support for bit manipulation instructions" depends on TOOLCHAIN_HAS_ZBKB - depends on RISCV_ALTERNATIVE default y help Adds support to dynamically detect the presence of the ZBKB @@ -827,7 +809,6 @@ config RISCV_ISA_ZBKB config RISCV_ISA_ZICBOM bool "Zicbom extension support for non-coherent DMA operation" depends on MMU - depends on RISCV_ALTERNATIVE default y select RISCV_DMA_NONCOHERENT select DMA_DIRECT_REMAP @@ -843,7 +824,6 @@ config RISCV_ISA_ZICBOM =20 config RISCV_ISA_ZICBOZ bool "Zicboz extension support for faster zeroing of memory" - depends on RISCV_ALTERNATIVE default y help Enable the use of the Zicboz extension (cbo.zero instruction) @@ -856,7 +836,6 @@ config RISCV_ISA_ZICBOZ config RISCV_ISA_ZICBOP bool "Zicbop extension support for cache block prefetch" depends on MMU - depends on RISCV_ALTERNATIVE default y help Adds support to dynamically detect the presence of the ZICBOP @@ -871,7 +850,6 @@ config RISCV_ISA_ZICBOP config RISCV_ISA_SVRSW60T59B bool "Svrsw60t59b extension support for using PTE bits 60 and 59" depends on MMU && 64BIT - depends on RISCV_ALTERNATIVE default y help Adds support to dynamically detect the presence of the Svrsw60t59b @@ -1204,7 +1182,6 @@ config RISCV_USER_CFI bool "riscv userspace control flow integrity" depends on 64BIT && MMU && \ $(cc-option,-mabi=3Dlp64 -march=3Drv64ima_zicfiss_zicfilp -fcf-protectio= n=3Dfull) - depends on RISCV_ALTERNATIVE select RISCV_SBI select ARCH_HAS_USER_SHADOW_STACK select ARCH_USES_HIGH_VMA_FLAGS diff --git a/arch/riscv/Kconfig.errata b/arch/riscv/Kconfig.errata index 3c945d086c7d..1a66e5b6f7d1 100644 --- a/arch/riscv/Kconfig.errata +++ b/arch/riscv/Kconfig.errata @@ -2,7 +2,7 @@ menu "CPU errata selection" =20 config ERRATA_ANDES bool "Andes AX45MP errata" - depends on RISCV_ALTERNATIVE && RISCV_SBI + depends on RISCV_SBI help All Andes errata Kconfig depend on this Kconfig. Disabling this Kconfig will disable all Andes errata. Please say "Y" @@ -23,7 +23,6 @@ config ERRATA_ANDES_CMO =20 config ERRATA_MIPS bool "MIPS errata" - depends on RISCV_ALTERNATIVE help All MIPS errata Kconfig depend on this Kconfig. Disabling this Kconfig will disable all MIPS errata. Please say "Y" @@ -46,7 +45,6 @@ config ERRATA_MIPS_P8700_PAUSE_OPCODE =20 config ERRATA_SIFIVE bool "SiFive errata" - depends on RISCV_ALTERNATIVE help All SiFive errata Kconfig depend on this Kconfig. Disabling this Kconfig will disable all SiFive errata. Please say "Y" @@ -98,7 +96,6 @@ config ERRATA_STARFIVE_JH7100 =20 config ERRATA_THEAD bool "T-HEAD errata" - depends on RISCV_ALTERNATIVE help All T-HEAD errata Kconfig depend on this Kconfig. Disabling this Kconfig will disable all T-HEAD errata. Please say "Y" diff --git a/arch/riscv/include/asm/alternative-macros.h b/arch/riscv/inclu= de/asm/alternative-macros.h index 9619bd5c8eba..629d884578da 100644 --- a/arch/riscv/include/asm/alternative-macros.h +++ b/arch/riscv/include/asm/alternative-macros.h @@ -2,8 +2,6 @@ #ifndef __ASM_ALTERNATIVE_MACROS_H #define __ASM_ALTERNATIVE_MACROS_H =20 -#ifdef CONFIG_RISCV_ALTERNATIVE - #ifdef __ASSEMBLER__ =20 .macro ALT_ENTRY oldptr newptr vendor_id patch_id new_len @@ -108,28 +106,6 @@ __ALTERNATIVE_CFG_2(old_c, new_c_1, vendor_id_1, patch_id_1, IS_ENABLED(C= ONFIG_k_1), \ new_c_2, vendor_id_2, patch_id_2, IS_ENABLED(CONFIG_k_2)) =20 -#else /* CONFIG_RISCV_ALTERNATIVE */ -#ifdef __ASSEMBLER__ - -.macro ALTERNATIVE_CFG old_c - \old_c -.endm - -#define __ALTERNATIVE_CFG(old_c, ...) ALTERNATIVE_CFG old_c -#define __ALTERNATIVE_CFG_2(old_c, ...) ALTERNATIVE_CFG old_c - -#else /* !__ASSEMBLER__ */ - -#define __ALTERNATIVE_CFG(old_c, ...) old_c "\n" -#define __ALTERNATIVE_CFG_2(old_c, ...) old_c "\n" - -#endif /* __ASSEMBLER__ */ - -#define _ALTERNATIVE_CFG(old_c, ...) __ALTERNATIVE_CFG(old_c) -#define _ALTERNATIVE_CFG_2(old_c, ...) __ALTERNATIVE_CFG_2(old_c) - -#endif /* CONFIG_RISCV_ALTERNATIVE */ - /* * Usage: * ALTERNATIVE(old_content, new_content, vendor_id, patch_id, CONFIG_k) diff --git a/arch/riscv/include/asm/alternative.h b/arch/riscv/include/asm/= alternative.h index 8407d1d535b8..688c7d1a9ae3 100644 --- a/arch/riscv/include/asm/alternative.h +++ b/arch/riscv/include/asm/alternative.h @@ -10,8 +10,6 @@ =20 #ifndef __ASSEMBLER__ =20 -#ifdef CONFIG_RISCV_ALTERNATIVE - #include #include #include @@ -61,13 +59,5 @@ void thead_errata_patch_func(struct alt_entry *begin, st= ruct alt_entry *end, void riscv_cpufeature_patch_func(struct alt_entry *begin, struct alt_entry= *end, unsigned int stage); =20 -#else /* CONFIG_RISCV_ALTERNATIVE */ - -static inline void apply_boot_alternatives(void) { } -static inline void apply_early_boot_alternatives(void) { } -static inline void apply_module_alternatives(void *start, size_t length) {= } - -#endif /* CONFIG_RISCV_ALTERNATIVE */ - #endif #endif diff --git a/arch/riscv/include/asm/cpufeature-macros.h b/arch/riscv/includ= e/asm/cpufeature-macros.h index a8103edbf51f..adaf9e3fb25c 100644 --- a/arch/riscv/include/asm/cpufeature-macros.h +++ b/arch/riscv/include/asm/cpufeature-macros.h @@ -47,20 +47,14 @@ static __always_inline bool riscv_has_extension_unlikel= y(const unsigned long ext { compiletime_assert(ext < RISCV_ISA_EXT_MAX, "ext must be < RISCV_ISA_EXT_= MAX"); =20 - if (IS_ENABLED(CONFIG_RISCV_ALTERNATIVE)) - return __riscv_has_extension_unlikely(STANDARD_EXT, ext); - - return __riscv_isa_extension_available(NULL, ext); + return __riscv_has_extension_unlikely(STANDARD_EXT, ext); } =20 static __always_inline bool riscv_has_extension_likely(const unsigned long= ext) { compiletime_assert(ext < RISCV_ISA_EXT_MAX, "ext must be < RISCV_ISA_EXT_= MAX"); =20 - if (IS_ENABLED(CONFIG_RISCV_ALTERNATIVE)) - return __riscv_has_extension_likely(STANDARD_EXT, ext); - - return __riscv_isa_extension_available(NULL, ext); + return __riscv_has_extension_likely(STANDARD_EXT, ext); } =20 #endif /* _ASM_CPUFEATURE_MACROS_H */ diff --git a/arch/riscv/include/asm/cpufeature.h b/arch/riscv/include/asm/c= pufeature.h index 739fcc84bf7b..37c9f2a0fb54 100644 --- a/arch/riscv/include/asm/cpufeature.h +++ b/arch/riscv/include/asm/cpufeature.h @@ -134,8 +134,7 @@ static __always_inline bool riscv_cpu_has_extension_lik= ely(int cpu, const unsign { compiletime_assert(ext < RISCV_ISA_EXT_MAX, "ext must be < RISCV_ISA_EXT_= MAX"); =20 - if (IS_ENABLED(CONFIG_RISCV_ALTERNATIVE) && - __riscv_has_extension_likely(STANDARD_EXT, ext)) + if (__riscv_has_extension_likely(STANDARD_EXT, ext)) return true; =20 return __riscv_isa_extension_available(hart_isa[cpu].isa, ext); @@ -145,8 +144,7 @@ static __always_inline bool riscv_cpu_has_extension_unl= ikely(int cpu, const unsi { compiletime_assert(ext < RISCV_ISA_EXT_MAX, "ext must be < RISCV_ISA_EXT_= MAX"); =20 - if (IS_ENABLED(CONFIG_RISCV_ALTERNATIVE) && - __riscv_has_extension_unlikely(STANDARD_EXT, ext)) + if (__riscv_has_extension_unlikely(STANDARD_EXT, ext)) return true; =20 return __riscv_isa_extension_available(hart_isa[cpu].isa, ext); diff --git a/arch/riscv/include/asm/vendor_extensions.h b/arch/riscv/includ= e/asm/vendor_extensions.h index 7437304a71b9..b00149be2627 100644 --- a/arch/riscv/include/asm/vendor_extensions.h +++ b/arch/riscv/include/asm/vendor_extensions.h @@ -54,11 +54,9 @@ static __always_inline bool riscv_has_vendor_extension_l= ikely(const unsigned lon if (!IS_ENABLED(CONFIG_RISCV_ISA_VENDOR_EXT)) return false; =20 - if (IS_ENABLED(CONFIG_RISCV_ALTERNATIVE)) - return __riscv_has_extension_likely(vendor, - ext + RISCV_VENDOR_EXT_ALTERNATIVES_BASE); + return __riscv_has_extension_likely(vendor, + ext + RISCV_VENDOR_EXT_ALTERNATIVES_BASE); =20 - return __riscv_isa_vendor_extension_available(VENDOR_EXT_ALL_CPUS, vendor= , ext); } =20 static __always_inline bool riscv_has_vendor_extension_unlikely(const unsi= gned long vendor, @@ -67,11 +65,9 @@ static __always_inline bool riscv_has_vendor_extension_u= nlikely(const unsigned l if (!IS_ENABLED(CONFIG_RISCV_ISA_VENDOR_EXT)) return false; =20 - if (IS_ENABLED(CONFIG_RISCV_ALTERNATIVE)) - return __riscv_has_extension_unlikely(vendor, - ext + RISCV_VENDOR_EXT_ALTERNATIVES_BASE); + return __riscv_has_extension_unlikely(vendor, + ext + RISCV_VENDOR_EXT_ALTERNATIVES_BASE); =20 - return __riscv_isa_vendor_extension_available(VENDOR_EXT_ALL_CPUS, vendor= , ext); } =20 static __always_inline bool riscv_cpu_has_vendor_extension_likely(const un= signed long vendor, @@ -80,8 +76,7 @@ static __always_inline bool riscv_cpu_has_vendor_extensio= n_likely(const unsigned if (!IS_ENABLED(CONFIG_RISCV_ISA_VENDOR_EXT)) return false; =20 - if (IS_ENABLED(CONFIG_RISCV_ALTERNATIVE) && - __riscv_has_extension_likely(vendor, ext + RISCV_VENDOR_EXT_ALTERNATI= VES_BASE)) + if (__riscv_has_extension_likely(vendor, ext + RISCV_VENDOR_EXT_ALTERNATI= VES_BASE)) return true; =20 return __riscv_isa_vendor_extension_available(cpu, vendor, ext); @@ -94,8 +89,7 @@ static __always_inline bool riscv_cpu_has_vendor_extensio= n_unlikely(const unsign if (!IS_ENABLED(CONFIG_RISCV_ISA_VENDOR_EXT)) return false; =20 - if (IS_ENABLED(CONFIG_RISCV_ALTERNATIVE) && - __riscv_has_extension_unlikely(vendor, ext + RISCV_VENDOR_EXT_ALTERNA= TIVES_BASE)) + if (__riscv_has_extension_unlikely(vendor, ext + RISCV_VENDOR_EXT_ALTERNA= TIVES_BASE)) return true; =20 return __riscv_isa_vendor_extension_available(cpu, vendor, ext); diff --git a/arch/riscv/kernel/Makefile b/arch/riscv/kernel/Makefile index ebe1c3588177..91353c94bfab 100644 --- a/arch/riscv/kernel/Makefile +++ b/arch/riscv/kernel/Makefile @@ -48,7 +48,7 @@ always-$(KBUILD_BUILTIN) +=3D vmlinux.lds =20 obj-y +=3D head.o obj-y +=3D soc.o -obj-$(CONFIG_RISCV_ALTERNATIVE) +=3D alternative.o +obj-y +=3D alternative.o obj-y +=3D cpu.o obj-y +=3D cpufeature.o obj-y +=3D entry.o diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c index d2ec96843456..9915121e9438 100644 --- a/arch/riscv/kernel/cpufeature.c +++ b/arch/riscv/kernel/cpufeature.c @@ -1217,7 +1217,6 @@ void __init riscv_user_isa_enable(void) pr_warn("Zicbop disabled as it is unavailable on some harts\n"); } =20 -#ifdef CONFIG_RISCV_ALTERNATIVE /* * Alternative patch sites consider 48 bits when determining when to patch * the old instruction sequence with the new. These bits are broken into a @@ -1306,4 +1305,3 @@ void __init_or_module riscv_cpufeature_patch_func(str= uct alt_entry *begin, mutex_unlock(&text_mutex); } } -#endif --=20 2.53.0 From nobody Fri Sep 25 23:08:55 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id ADCDB3C0A14 for ; Mon, 7 Sep 2026 15:34:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788795283; cv=none; b=fIeoigPsU75ZOZPy4ThFzotzU1gIBuLPt6oSTIBvAA1tq/ce5jO1Pvw9XEhoajIkZ3BcMRp+Bvh+vnujNN+u6/JeYjTePKNoZ0pyBtg3OM/oUMkQ+Fg0t54k5HCo63FwX7MIaXVE0vukkNQ8oHjZM3es6x1d/MgKj3KP4ddu1Vc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788795283; c=relaxed/simple; bh=KGCrAR/T2WVSS15NlOOPgq4g/mbcRioGXInGYY/rJ0E=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=jmghgKzbkXvFEZs9+LhbkYlVk3K1+BFaQUjc5Y0OVnd+DBboVNhf4/WeNRORLQ5wNPfOFN4kkpaa1OqKhtrt66cqg9kx/3dBxnHbr31Uagd8dAYtvzI38yKfIDnAxc9aTdj3a54K/WCgeMIwPYjUafs8IkgOW6czwbxBElNnDeA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=EoElUDct; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="EoElUDct" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 969691F00A3A; Mon, 7 Sep 2026 15:34:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1788795276; bh=znvmixE620ZLsCnkyAYTT60sRef1h2y+RevyjPTXKGg=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=EoElUDctbswG2S0pdCcuH3mNf5iyHzvwoVFzhy1so7iH3XgH/HDALstZVu+XYWPrH +EhaQRkFvoBg2aA9XGFXjIUECtD4tvbSC497MGBT/HZE+s38oUZIxeyZgMTVNKFCwF KIrq6lU4/SGwlAksXRIZHKGRoUxT4CpoAS3lfXuF73tkU/PyvN/4qKIPCWTuYZT4Dk uWbaLs1F4ferJ/2ak/nGob4HfKUXqCYylAyMTn6uoWo3mOSIDJWO2VoZEbemYKjBiL RW5r8xUk9h22VoRtbTRmEjLMLgbU5TdK/ptG9HJT2Xv2DQHRf82BmE0sJ3bQvE+xyg xK0vkWCAKJR/A== From: Jisheng Zhang To: Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexandre Ghiti , Andrey Ryabinin , Alexander Potapenko , Andrey Konovalov , Dmitry Vyukov , Vincenzo Frascino Cc: linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com Subject: [PATCH v2 2/5] riscv: convert pgtable_l4|l5_enabled to inline function Date: Mon, 7 Sep 2026 23:14:34 +0800 Message-ID: <20260907151437.7603-3-jszhang@kernel.org> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260907151437.7603-1-jszhang@kernel.org> References: <20260907151437.7603-1-jszhang@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" This is the preparation of optimizing pgtable_l4|l5_enabled(). No functionality change. Signed-off-by: Jisheng Zhang --- arch/riscv/include/asm/pgalloc.h | 14 +++---- arch/riscv/include/asm/pgtable-32.h | 4 +- arch/riscv/include/asm/pgtable-64.h | 50 +++++++++++++--------- arch/riscv/include/asm/pgtable.h | 4 +- arch/riscv/include/asm/vmalloc.h | 5 +-- arch/riscv/kernel/cpu.c | 4 +- arch/riscv/mm/init.c | 64 ++++++++++++++--------------- arch/riscv/mm/kasan_init.c | 20 ++++----- arch/riscv/mm/pgtable.c | 4 +- arch/riscv/mm/ptdump.c | 4 +- 10 files changed, 91 insertions(+), 82 deletions(-) diff --git a/arch/riscv/include/asm/pgalloc.h b/arch/riscv/include/asm/pgal= loc.h index 770ce18a7328..0133f7ba41d7 100644 --- a/arch/riscv/include/asm/pgalloc.h +++ b/arch/riscv/include/asm/pgalloc.h @@ -41,7 +41,7 @@ static inline void pud_populate(struct mm_struct *mm, pud= _t *pud, pmd_t *pmd) =20 static inline void p4d_populate(struct mm_struct *mm, p4d_t *p4d, pud_t *p= ud) { - if (pgtable_l4_enabled) { + if (pgtable_l4_enabled()) { unsigned long pfn =3D virt_to_pfn(pud); =20 set_p4d(p4d, __p4d((pfn << _PAGE_PFN_SHIFT) | _PAGE_TABLE)); @@ -51,7 +51,7 @@ static inline void p4d_populate(struct mm_struct *mm, p4d= _t *p4d, pud_t *pud) static inline void p4d_populate_safe(struct mm_struct *mm, p4d_t *p4d, pud_t *pud) { - if (pgtable_l4_enabled) { + if (pgtable_l4_enabled()) { unsigned long pfn =3D virt_to_pfn(pud); =20 set_p4d_safe(p4d, @@ -61,7 +61,7 @@ static inline void p4d_populate_safe(struct mm_struct *mm= , p4d_t *p4d, =20 static inline void pgd_populate(struct mm_struct *mm, pgd_t *pgd, p4d_t *p= 4d) { - if (pgtable_l5_enabled) { + if (pgtable_l5_enabled()) { unsigned long pfn =3D virt_to_pfn(p4d); =20 set_pgd(pgd, __pgd((pfn << _PAGE_PFN_SHIFT) | _PAGE_TABLE)); @@ -71,7 +71,7 @@ static inline void pgd_populate(struct mm_struct *mm, pgd= _t *pgd, p4d_t *p4d) static inline void pgd_populate_safe(struct mm_struct *mm, pgd_t *pgd, p4d_t *p4d) { - if (pgtable_l5_enabled) { + if (pgtable_l5_enabled()) { unsigned long pfn =3D virt_to_pfn(p4d); =20 set_pgd_safe(pgd, @@ -82,21 +82,21 @@ static inline void pgd_populate_safe(struct mm_struct *= mm, pgd_t *pgd, #define pud_free pud_free static inline void pud_free(struct mm_struct *mm, pud_t *pud) { - if (pgtable_l4_enabled) + if (pgtable_l4_enabled()) __pud_free(mm, pud); } =20 static inline void __pud_free_tlb(struct mmu_gather *tlb, pud_t *pud, unsigned long addr) { - if (pgtable_l4_enabled) + if (pgtable_l4_enabled()) tlb_remove_ptdesc(tlb, virt_to_ptdesc(pud)); } =20 static inline void __p4d_free_tlb(struct mmu_gather *tlb, p4d_t *p4d, unsigned long addr) { - if (pgtable_l5_enabled) + if (pgtable_l5_enabled()) tlb_remove_ptdesc(tlb, virt_to_ptdesc(p4d)); } #endif /* __PAGETABLE_PMD_FOLDED */ diff --git a/arch/riscv/include/asm/pgtable-32.h b/arch/riscv/include/asm/p= gtable-32.h index 00f3369570a8..5003a7e0f0b0 100644 --- a/arch/riscv/include/asm/pgtable-32.h +++ b/arch/riscv/include/asm/pgtable-32.h @@ -33,7 +33,7 @@ _PAGE_WRITE | _PAGE_EXEC | \ _PAGE_USER | _PAGE_GLOBAL)) =20 -static const __maybe_unused int pgtable_l4_enabled; -static const __maybe_unused int pgtable_l5_enabled; +static inline bool pgtable_l4_enabled(void) { return false; } +static inline bool pgtable_l5_enabled(void) { return false; } =20 #endif /* _ASM_RISCV_PGTABLE_32_H */ diff --git a/arch/riscv/include/asm/pgtable-64.h b/arch/riscv/include/asm/p= gtable-64.h index 6e789fa58514..72b8c63469fa 100644 --- a/arch/riscv/include/asm/pgtable-64.h +++ b/arch/riscv/include/asm/pgtable-64.h @@ -10,14 +10,24 @@ #include #include =20 -extern bool pgtable_l4_enabled; -extern bool pgtable_l5_enabled; +extern bool _pgtable_l4_enabled; +extern bool _pgtable_l5_enabled; + +static __always_inline bool pgtable_l5_enabled(void) +{ + return _pgtable_l5_enabled; +} + +static __always_inline bool pgtable_l4_enabled(void) +{ + return _pgtable_l4_enabled; +} =20 #define PGDIR_SHIFT_L3 30 #define PGDIR_SHIFT_L4 39 #define PGDIR_SHIFT_L5 48 -#define PGDIR_SHIFT (pgtable_l5_enabled ? PGDIR_SHIFT_L5 : \ - (pgtable_l4_enabled ? PGDIR_SHIFT_L4 : PGDIR_SHIFT_L3)) +#define PGDIR_SHIFT (pgtable_l5_enabled() ? PGDIR_SHIFT_L5 : \ + (pgtable_l4_enabled() ? PGDIR_SHIFT_L4 : PGDIR_SHIFT_L3)) /* Size of region mapped by a page global directory */ #define PGDIR_SIZE (_AC(1, UL) << PGDIR_SHIFT) #define PGDIR_MASK (~(PGDIR_SIZE - 1)) @@ -26,8 +36,8 @@ extern bool pgtable_l5_enabled; #define P4D_SHIFT_L3 30 #define P4D_SHIFT_L4 39 #define P4D_SHIFT_L5 39 -#define P4D_SHIFT (pgtable_l5_enabled ? P4D_SHIFT_L5 : \ - (pgtable_l4_enabled ? P4D_SHIFT_L4 : P4D_SHIFT_L3)) +#define P4D_SHIFT (pgtable_l5_enabled() ? P4D_SHIFT_L5 : \ + (pgtable_l4_enabled() ? P4D_SHIFT_L4 : P4D_SHIFT_L3)) #define P4D_SIZE (_AC(1, UL) << P4D_SHIFT) #define P4D_MASK (~(P4D_SIZE - 1)) =20 @@ -233,7 +243,7 @@ static inline struct page *pud_page(pud_t pud) #define mm_p4d_folded mm_p4d_folded static inline bool mm_p4d_folded(struct mm_struct *mm) { - if (pgtable_l5_enabled) + if (pgtable_l5_enabled()) return false; =20 return true; @@ -242,7 +252,7 @@ static inline bool mm_p4d_folded(struct mm_struct *mm) #define mm_pud_folded mm_pud_folded static inline bool mm_pud_folded(struct mm_struct *mm) { - if (pgtable_l4_enabled) + if (pgtable_l4_enabled()) return false; =20 return true; @@ -275,7 +285,7 @@ static inline unsigned long _pmd_pfn(pmd_t pmd) =20 static inline void set_p4d(p4d_t *p4dp, p4d_t p4d) { - if (pgtable_l4_enabled) + if (pgtable_l4_enabled()) WRITE_ONCE(*p4dp, p4d); else set_pud((pud_t *)p4dp, (pud_t){ p4d_val(p4d) }); @@ -283,7 +293,7 @@ static inline void set_p4d(p4d_t *p4dp, p4d_t p4d) =20 static inline int p4d_none(p4d_t p4d) { - if (pgtable_l4_enabled) + if (pgtable_l4_enabled()) return (p4d_val(p4d) =3D=3D 0); =20 return 0; @@ -291,7 +301,7 @@ static inline int p4d_none(p4d_t p4d) =20 static inline int p4d_present(p4d_t p4d) { - if (pgtable_l4_enabled) + if (pgtable_l4_enabled()) return (p4d_val(p4d) & _PAGE_PRESENT); =20 return 1; @@ -299,7 +309,7 @@ static inline int p4d_present(p4d_t p4d) =20 static inline int p4d_bad(p4d_t p4d) { - if (pgtable_l4_enabled) + if (pgtable_l4_enabled()) return !p4d_present(p4d); =20 return 0; @@ -307,7 +317,7 @@ static inline int p4d_bad(p4d_t p4d) =20 static inline void p4d_clear(p4d_t *p4d) { - if (pgtable_l4_enabled) + if (pgtable_l4_enabled()) set_p4d(p4d, __p4d(0)); } =20 @@ -323,7 +333,7 @@ static inline unsigned long _p4d_pfn(p4d_t p4d) =20 static inline pud_t *p4d_pgtable(p4d_t p4d) { - if (pgtable_l4_enabled) + if (pgtable_l4_enabled()) return (pud_t *)pfn_to_virt(__page_val_to_pfn(p4d_val(p4d))); =20 return (pud_t *)pud_pgtable((pud_t) { p4d_val(p4d) }); @@ -342,7 +352,7 @@ pud_t *pud_offset(p4d_t *p4d, unsigned long address); =20 static inline void set_pgd(pgd_t *pgdp, pgd_t pgd) { - if (pgtable_l5_enabled) + if (pgtable_l5_enabled()) WRITE_ONCE(*pgdp, pgd); else set_p4d((p4d_t *)pgdp, (p4d_t){ pgd_val(pgd) }); @@ -350,7 +360,7 @@ static inline void set_pgd(pgd_t *pgdp, pgd_t pgd) =20 static inline int pgd_none(pgd_t pgd) { - if (pgtable_l5_enabled) + if (pgtable_l5_enabled()) return (pgd_val(pgd) =3D=3D 0); =20 return 0; @@ -358,7 +368,7 @@ static inline int pgd_none(pgd_t pgd) =20 static inline int pgd_present(pgd_t pgd) { - if (pgtable_l5_enabled) + if (pgtable_l5_enabled()) return (pgd_val(pgd) & _PAGE_PRESENT); =20 return 1; @@ -366,7 +376,7 @@ static inline int pgd_present(pgd_t pgd) =20 static inline int pgd_bad(pgd_t pgd) { - if (pgtable_l5_enabled) + if (pgtable_l5_enabled()) return !pgd_present(pgd); =20 return 0; @@ -374,13 +384,13 @@ static inline int pgd_bad(pgd_t pgd) =20 static inline void pgd_clear(pgd_t *pgd) { - if (pgtable_l5_enabled) + if (pgtable_l5_enabled()) set_pgd(pgd, __pgd(0)); } =20 static inline p4d_t *pgd_pgtable(pgd_t pgd) { - if (pgtable_l5_enabled) + if (pgtable_l5_enabled()) return (p4d_t *)pfn_to_virt(__page_val_to_pfn(pgd_val(pgd))); =20 return (p4d_t *)p4d_pgtable((p4d_t) { pgd_val(pgd) }); diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgta= ble.h index 40b1ed4f3ea8..a71671166852 100644 --- a/arch/riscv/include/asm/pgtable.h +++ b/arch/riscv/include/asm/pgtable.h @@ -75,8 +75,8 @@ #define VA_BITS_SV48 48 #define VA_BITS_SV57 57 =20 -#define VA_BITS (pgtable_l5_enabled ? \ - VA_BITS_SV57 : (pgtable_l4_enabled ? VA_BITS_SV48 : VA_BITS_SV39)) +#define VA_BITS (pgtable_l5_enabled() ? \ + VA_BITS_SV57 : (pgtable_l4_enabled() ? VA_BITS_SV48 : VA_BITS_SV39)) #else #define VA_BITS VA_BITS_SV32 #endif diff --git a/arch/riscv/include/asm/vmalloc.h b/arch/riscv/include/asm/vmal= loc.h index fefe94dc98e2..4d8b80ce570b 100644 --- a/arch/riscv/include/asm/vmalloc.h +++ b/arch/riscv/include/asm/vmalloc.h @@ -3,15 +3,14 @@ #define _ASM_RISCV_VMALLOC_H =20 #ifdef CONFIG_HAVE_ARCH_HUGE_VMAP - -extern bool pgtable_l4_enabled, pgtable_l5_enabled; +#include =20 #define IOREMAP_MAX_ORDER (PUD_SHIFT) =20 #define arch_vmap_pud_supported arch_vmap_pud_supported static inline bool arch_vmap_pud_supported(pgprot_t prot) { - return pgtable_l4_enabled || pgtable_l5_enabled; + return pgtable_l4_enabled() || pgtable_l5_enabled(); } =20 #define arch_vmap_pmd_supported arch_vmap_pmd_supported diff --git a/arch/riscv/kernel/cpu.c b/arch/riscv/kernel/cpu.c index 3dbc8cc557dd..af84b97af3a1 100644 --- a/arch/riscv/kernel/cpu.c +++ b/arch/riscv/kernel/cpu.c @@ -292,9 +292,9 @@ static void print_mmu(struct seq_file *f) #if defined(CONFIG_32BIT) sv_type =3D "sv32"; #elif defined(CONFIG_64BIT) - if (pgtable_l5_enabled) + if (pgtable_l5_enabled()) sv_type =3D "sv57"; - else if (pgtable_l4_enabled) + else if (pgtable_l4_enabled()) sv_type =3D "sv48"; else sv_type =3D "sv39"; diff --git a/arch/riscv/mm/init.c b/arch/riscv/mm/init.c index f8994caefc70..fc74142fe6e6 100644 --- a/arch/riscv/mm/init.c +++ b/arch/riscv/mm/init.c @@ -52,10 +52,10 @@ u64 satp_mode __ro_after_init =3D SATP_MODE_32; EXPORT_SYMBOL(satp_mode); =20 #ifdef CONFIG_64BIT -bool pgtable_l4_enabled __ro_after_init =3D true; -bool pgtable_l5_enabled __ro_after_init =3D true; -EXPORT_SYMBOL(pgtable_l4_enabled); -EXPORT_SYMBOL(pgtable_l5_enabled); +bool _pgtable_l4_enabled __ro_after_init =3D true; +bool _pgtable_l5_enabled __ro_after_init =3D true; +EXPORT_SYMBOL(_pgtable_l4_enabled); +EXPORT_SYMBOL(_pgtable_l5_enabled); #endif =20 phys_addr_t phys_ram_base __ro_after_init; @@ -652,23 +652,23 @@ static void __meminit create_p4d_mapping(p4d_t *p4dp,= uintptr_t va, phys_addr_t } =20 #define pgd_next_t p4d_t -#define alloc_pgd_next(__va) (pgtable_l5_enabled ? \ - pt_ops.alloc_p4d(__va) : (pgtable_l4_enabled ? \ +#define alloc_pgd_next(__va) (pgtable_l5_enabled() ? \ + pt_ops.alloc_p4d(__va) : (pgtable_l4_enabled() ? \ pt_ops.alloc_pud(__va) : pt_ops.alloc_pmd(__va))) -#define get_pgd_next_virt(__pa) (pgtable_l5_enabled ? \ - pt_ops.get_p4d_virt(__pa) : (pgd_next_t *)(pgtable_l4_enabled ? \ +#define get_pgd_next_virt(__pa) (pgtable_l5_enabled() ? \ + pt_ops.get_p4d_virt(__pa) : (pgd_next_t *)(pgtable_l4_enabled() ? \ pt_ops.get_pud_virt(__pa) : (pud_t *)pt_ops.get_pmd_virt(__pa))) #define create_pgd_next_mapping(__nextp, __va, __pa, __sz, __prot) \ - (pgtable_l5_enabled ? \ + (pgtable_l5_enabled() ? \ create_p4d_mapping(__nextp, __va, __pa, __sz, __prot) : \ - (pgtable_l4_enabled ? \ + (pgtable_l4_enabled() ? \ create_pud_mapping((pud_t *)__nextp, __va, __pa, __sz, __prot) : \ create_pmd_mapping((pmd_t *)__nextp, __va, __pa, __sz, __prot))) -#define fixmap_pgd_next (pgtable_l5_enabled ? \ - (uintptr_t)fixmap_p4d : (pgtable_l4_enabled ? \ +#define fixmap_pgd_next (pgtable_l5_enabled() ? \ + (uintptr_t)fixmap_p4d : (pgtable_l4_enabled() ? \ (uintptr_t)fixmap_pud : (uintptr_t)fixmap_pmd)) -#define trampoline_pgd_next (pgtable_l5_enabled ? \ - (uintptr_t)trampoline_p4d : (pgtable_l4_enabled ? \ +#define trampoline_pgd_next (pgtable_l5_enabled() ? \ + (uintptr_t)trampoline_p4d : (pgtable_l4_enabled() ? \ (uintptr_t)trampoline_pud : (uintptr_t)trampoline_pmd)) #else #define pgd_next_t pte_t @@ -713,11 +713,11 @@ static uintptr_t __meminit best_map_size(phys_addr_t = pa, uintptr_t va, phys_addr if (debug_pagealloc_enabled()) return PAGE_SIZE; =20 - if (pgtable_l5_enabled && + if (pgtable_l5_enabled() && !(pa & (P4D_SIZE - 1)) && !(va & (P4D_SIZE - 1)) && size >=3D P4D_SIZ= E) return P4D_SIZE; =20 - if (pgtable_l4_enabled && + if (pgtable_l4_enabled() && !(pa & (PUD_SIZE - 1)) && !(va & (PUD_SIZE - 1)) && size >=3D PUD_SIZ= E) return PUD_SIZE; =20 @@ -769,14 +769,14 @@ u64 __pi_set_satp_mode_from_fdt(uintptr_t dtb_pa); =20 static void __init disable_pgtable_l5(void) { - pgtable_l5_enabled =3D false; + _pgtable_l5_enabled =3D false; kernel_map.page_offset =3D PAGE_OFFSET_L4; satp_mode =3D SATP_MODE_48; } =20 static void __init disable_pgtable_l4(void) { - pgtable_l4_enabled =3D false; + _pgtable_l4_enabled =3D false; kernel_map.page_offset =3D PAGE_OFFSET_L3; satp_mode =3D SATP_MODE_39; } @@ -867,7 +867,7 @@ static __init void set_satp_mode(uintptr_t dtb_pa) =20 create_pgd_mapping(early_pg_dir, set_satp_mode_pmd, - pgtable_l5_enabled ? + pgtable_l5_enabled() ? (uintptr_t)early_p4d : (uintptr_t)early_pud, PGDIR_SIZE, PAGE_TABLE); =20 @@ -879,7 +879,7 @@ static __init void set_satp_mode(uintptr_t dtb_pa) local_flush_tlb_all(); =20 if (hw_satp !=3D identity_satp) { - if (pgtable_l5_enabled) { + if (pgtable_l5_enabled()) { disable_pgtable_l5(); memset(early_pg_dir, 0, PAGE_SIZE); goto retry; @@ -1120,11 +1120,11 @@ asmlinkage void __init setup_vm(uintptr_t dtb_pa) =20 #ifndef __PAGETABLE_PMD_FOLDED /* Setup fixmap P4D and PUD */ - if (pgtable_l5_enabled) + if (pgtable_l5_enabled()) create_p4d_mapping(fixmap_p4d, FIXADDR_START, (uintptr_t)fixmap_pud, P4D_SIZE, PAGE_TABLE); /* Setup fixmap PUD and PMD */ - if (pgtable_l4_enabled) + if (pgtable_l4_enabled()) create_pud_mapping(fixmap_pud, FIXADDR_START, (uintptr_t)fixmap_pmd, PUD_SIZE, PAGE_TABLE); create_pmd_mapping(fixmap_pmd, FIXADDR_START, @@ -1132,10 +1132,10 @@ asmlinkage void __init setup_vm(uintptr_t dtb_pa) /* Setup trampoline PGD and PMD */ create_pgd_mapping(trampoline_pg_dir, kernel_map.virt_addr, trampoline_pgd_next, PGDIR_SIZE, PAGE_TABLE); - if (pgtable_l5_enabled) + if (pgtable_l5_enabled()) create_p4d_mapping(trampoline_p4d, kernel_map.virt_addr, (uintptr_t)trampoline_pud, P4D_SIZE, PAGE_TABLE); - if (pgtable_l4_enabled) + if (pgtable_l4_enabled()) create_pud_mapping(trampoline_pud, kernel_map.virt_addr, (uintptr_t)trampoline_pmd, PUD_SIZE, PAGE_TABLE); create_pmd_mapping(trampoline_pmd, kernel_map.virt_addr, @@ -1406,7 +1406,7 @@ static void __init preallocate_pgd_pages_range(unsign= ed long start, unsigned lon if (!p4d) goto failed; =20 - if (pgtable_l5_enabled) + if (pgtable_l5_enabled()) continue; =20 lvl =3D "pud"; @@ -1414,7 +1414,7 @@ static void __init preallocate_pgd_pages_range(unsign= ed long start, unsigned lon if (!pud) goto failed; =20 - if (pgtable_l4_enabled) + if (pgtable_l4_enabled()) continue; =20 lvl =3D "pmd"; @@ -1624,13 +1624,13 @@ static void __meminit remove_pud_mapping(pud_t *pud= _base, unsigned long addr, un =20 for (; addr < end; addr =3D next) { next =3D pud_addr_end(addr, end); - pudp =3D pgtable_l4_enabled ? pud_base + pud_index(addr) : pud_base; + pudp =3D pgtable_l4_enabled() ? pud_base + pud_index(addr) : pud_base; pud =3D pudp_get(pudp); if (!pud_present(pud)) continue; =20 if (pud_leaf(pud)) { - if (pgtable_l4_enabled) { + if (pgtable_l4_enabled()) { pud_clear(pudp); if (is_vmemmap) free_vmemmap_storage(pud_page(pud), PUD_SIZE, altmap); @@ -1641,7 +1641,7 @@ static void __meminit remove_pud_mapping(pud_t *pud_b= ase, unsigned long addr, un pmd_base =3D pmd_offset(pudp, 0); remove_pmd_mapping(pmd_base, addr, next, is_vmemmap, altmap); =20 - if (pgtable_l4_enabled) + if (pgtable_l4_enabled()) free_pmd_table(pmd_base, pudp, is_vmemmap); } } @@ -1655,13 +1655,13 @@ static void __meminit remove_p4d_mapping(p4d_t *p4d= _base, unsigned long addr, un =20 for (; addr < end; addr =3D next) { next =3D p4d_addr_end(addr, end); - p4dp =3D pgtable_l5_enabled ? p4d_base + p4d_index(addr) : p4d_base; + p4dp =3D pgtable_l5_enabled() ? p4d_base + p4d_index(addr) : p4d_base; p4d =3D p4dp_get(p4dp); if (!p4d_present(p4d)) continue; =20 if (p4d_leaf(p4d)) { - if (pgtable_l5_enabled) { + if (pgtable_l5_enabled()) { p4d_clear(p4dp); if (is_vmemmap) free_vmemmap_storage(p4d_page(p4d), P4D_SIZE, altmap); @@ -1672,7 +1672,7 @@ static void __meminit remove_p4d_mapping(p4d_t *p4d_b= ase, unsigned long addr, un pud_base =3D pud_offset(p4dp, 0); remove_pud_mapping(pud_base, addr, next, is_vmemmap, altmap); =20 - if (pgtable_l5_enabled) + if (pgtable_l5_enabled()) free_pud_table(pud_base, p4dp); } } diff --git a/arch/riscv/mm/kasan_init.c b/arch/riscv/mm/kasan_init.c index 1f3aa9611187..84f87c8277c3 100644 --- a/arch/riscv/mm/kasan_init.c +++ b/arch/riscv/mm/kasan_init.c @@ -168,7 +168,7 @@ static void __init kasan_early_clear_pud(p4d_t *p4dp, pud_t *pudp, *base_pud; unsigned long next; =20 - if (!pgtable_l4_enabled) { + if (!pgtable_l4_enabled()) { pudp =3D (pud_t *)p4dp; } else { base_pud =3D pt_ops.get_pud_virt(pfn_to_phys(_p4d_pfn(p4dp_get(p4dp)))); @@ -193,7 +193,7 @@ static void __init kasan_early_clear_p4d(pgd_t *pgdp, p4d_t *p4dp, *base_p4d; unsigned long next; =20 - if (!pgtable_l5_enabled) { + if (!pgtable_l5_enabled()) { p4dp =3D (p4d_t *)pgdp; } else { base_p4d =3D pt_ops.get_p4d_virt(pfn_to_phys(_pgd_pfn(pgdp_get(pgdp)))); @@ -203,7 +203,7 @@ static void __init kasan_early_clear_p4d(pgd_t *pgdp, do { next =3D p4d_addr_end(vaddr, end); =20 - if (pgtable_l4_enabled && IS_ALIGNED(vaddr, P4D_SIZE) && + if (pgtable_l4_enabled() && IS_ALIGNED(vaddr, P4D_SIZE) && (next - vaddr) >=3D P4D_SIZE) { p4d_clear(p4dp); continue; @@ -221,7 +221,7 @@ static void __init kasan_early_clear_pgd(pgd_t *pgdp, do { next =3D pgd_addr_end(vaddr, end); =20 - if (pgtable_l5_enabled && IS_ALIGNED(vaddr, PGDIR_SIZE) && + if (pgtable_l5_enabled() && IS_ALIGNED(vaddr, PGDIR_SIZE) && (next - vaddr) >=3D PGDIR_SIZE) { pgd_clear(pgdp); continue; @@ -239,7 +239,7 @@ static void __init kasan_early_populate_pud(p4d_t *p4dp, phys_addr_t phys_addr; unsigned long next; =20 - if (!pgtable_l4_enabled) { + if (!pgtable_l4_enabled()) { pudp =3D (pud_t *)p4dp; } else { base_pud =3D pt_ops.get_pud_virt(pfn_to_phys(_p4d_pfn(p4dp_get(p4dp)))); @@ -277,7 +277,7 @@ static void __init kasan_early_populate_p4d(pgd_t *pgdp, * Note that this test is then completely equivalent to * p4dp =3D p4d_offset(pgdp, vaddr) */ - if (!pgtable_l5_enabled) { + if (!pgtable_l5_enabled()) { p4dp =3D (p4d_t *)pgdp; } else { base_p4d =3D pt_ops.get_p4d_virt(pfn_to_phys(_pgd_pfn(pgdp_get(pgdp)))); @@ -336,7 +336,7 @@ asmlinkage void __init kasan_early_init(void) (__pa((uintptr_t)kasan_early_shadow_pte)), PAGE_TABLE)); =20 - if (pgtable_l4_enabled) { + if (pgtable_l4_enabled()) { for (i =3D 0; i < PTRS_PER_PUD; ++i) set_pud(kasan_early_shadow_pud + i, pfn_pud(PFN_DOWN @@ -344,7 +344,7 @@ asmlinkage void __init kasan_early_init(void) PAGE_TABLE)); } =20 - if (pgtable_l5_enabled) { + if (pgtable_l5_enabled()) { for (i =3D 0; i < PTRS_PER_P4D; ++i) set_p4d(kasan_early_shadow_p4d + i, pfn_p4d(PFN_DOWN @@ -461,7 +461,7 @@ static void __init create_tmp_mapping(void) memcpy(tmp_pg_dir, swapper_pg_dir, sizeof(pgd_t) * PTRS_PER_PGD); =20 /* Copy the last p4d since it is shared with the kernel mapping. */ - if (pgtable_l5_enabled) { + if (pgtable_l5_enabled()) { ptr =3D (p4d_t *)pgd_page_vaddr(pgdp_get(pgd_offset_k(KASAN_SHADOW_END))= ); memcpy(tmp_p4d, ptr, sizeof(p4d_t) * PTRS_PER_P4D); set_pgd(&tmp_pg_dir[pgd_index(KASAN_SHADOW_END)], @@ -472,7 +472,7 @@ static void __init create_tmp_mapping(void) } =20 /* Copy the last pud since it is shared with the kernel mapping. */ - if (pgtable_l4_enabled) { + if (pgtable_l4_enabled()) { ptr =3D (pud_t *)p4d_page_vaddr(p4dp_get(base_p4d + p4d_index(KASAN_SHAD= OW_END))); memcpy(tmp_pud, ptr, sizeof(pud_t) * PTRS_PER_PUD); set_p4d(&base_p4d[p4d_index(KASAN_SHADOW_END)], diff --git a/arch/riscv/mm/pgtable.c b/arch/riscv/mm/pgtable.c index 9c4427d0b187..cd180f9abb5e 100644 --- a/arch/riscv/mm/pgtable.c +++ b/arch/riscv/mm/pgtable.c @@ -41,7 +41,7 @@ EXPORT_SYMBOL_GPL(ptep_test_and_clear_young); #ifdef CONFIG_64BIT pud_t *pud_offset(p4d_t *p4d, unsigned long address) { - if (pgtable_l4_enabled) + if (pgtable_l4_enabled()) return p4d_pgtable(p4dp_get(p4d)) + pud_index(address); =20 return (pud_t *)p4d; @@ -50,7 +50,7 @@ EXPORT_SYMBOL_GPL(pud_offset); =20 p4d_t *p4d_offset(pgd_t *pgd, unsigned long address) { - if (pgtable_l5_enabled) + if (pgtable_l5_enabled()) return pgd_pgtable(pgdp_get(pgd)) + p4d_index(address); =20 return (p4d_t *)pgd; diff --git a/arch/riscv/mm/ptdump.c b/arch/riscv/mm/ptdump.c index f4b4a9fcbbd8..f765a63e2ca7 100644 --- a/arch/riscv/mm/ptdump.c +++ b/arch/riscv/mm/ptdump.c @@ -446,8 +446,8 @@ static int __init ptdump_init(void) =20 kernel_ptd_info.base_addr =3D KERN_VIRT_START; =20 - pg_level[1].name =3D pgtable_l5_enabled ? "P4D" : "PGD"; - pg_level[2].name =3D pgtable_l4_enabled ? "PUD" : "PGD"; + pg_level[1].name =3D pgtable_l5_enabled() ? "P4D" : "PGD"; + pg_level[2].name =3D pgtable_l4_enabled() ? "PUD" : "PGD"; =20 for (i =3D 0; i < ARRAY_SIZE(pg_level); i++) for (j =3D 0; j < ARRAY_SIZE(pte_bits); j++) --=20 2.53.0 From nobody Fri Sep 25 23:08:55 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3465346C825 for ; Mon, 7 Sep 2026 15:34:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788795284; cv=none; b=N1xGYvQKMm6qGoN4EPYJmq5822V6vsnVhjtS7XD5Dghu3zMlzWCZTaPOYlGUTftAbKQGvOctj2Lzr586/iHjUy8UIRdlBZtbi8CMWQ27aS3eBu0lIDAUnWpWCad+kKTuYcmhJ0yD3UY1BQ8unNBUJ5z+gvXAhfJkMsl30ISqgrc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788795284; c=relaxed/simple; bh=qNRUlVvJmH0jxlXaKiyR9RFK8ibTL+EhHPEU+jYKlqc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Q0IpegRiFPEXGCPK4SryA1ZfHndip1UxlOv8zoLkraiEF2jGmm3ebKhVN0TT7wwg7IJmoIaCE4PoPxWwETmp/4/68OjQ2VOwqX/7at7OIfokMWKgTlG9D0v6Sc7DU1sUnncpQk8vscSX+uX/HioGSQCu/kkciIkEqMYODWLy9j8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=aT3U8a6Z; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="aT3U8a6Z" Received: by smtp.kernel.org (Postfix) with ESMTPSA id EF5A11F00A3E; Mon, 7 Sep 2026 15:34:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1788795279; bh=1Hh4Nf8Yl3K5FyycaNNg5vDmNNmDv1N4JqtXomGD8Ig=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=aT3U8a6ZznBlJfZfzFd202dq9tEv1ybkFGvCocW14OpbkaU2p5Zv8pA9AdYJHzkoF BbDkqHW7PCvvp/QO5fbppBvy7r3l/V5+HaDah6sOPnbsRvB4nov7ozk9BU8wTnUjuh +RzIHPM4/D9O6MwzvKZ+mBQ6UOAO4uY0fWfA6BHFzyIf28NUIH1sDmWU/6KtBXOcTe jwe6gcl/tuxH7EFOuVRPtumLYHQbOQsn+FlhPElckiUhG+SqwaWNRChHnnYSEo7iFq 07l6ZqOpmnKxVSvIAkAKLrS31Y20B0GfAxFxZNta/tL8mssdQ4FWJ7nJyCFbtNtgE0 +0qOSLoibxmOA== From: Jisheng Zhang To: Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexandre Ghiti , Andrey Ryabinin , Alexander Potapenko , Andrey Konovalov , Dmitry Vyukov , Vincenzo Frascino Cc: linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com Subject: [PATCH v2 3/5] riscv: support early isa ext and use it to optimize pgtable_l4|l5_enabled Date: Mon, 7 Sep 2026 23:14:35 +0800 Message-ID: <20260907151437.7603-4-jszhang@kernel.org> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260907151437.7603-1-jszhang@kernel.org> References: <20260907151437.7603-1-jszhang@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The pgtable_l4|[l5]_enabled check sits at hot code path, performance is impacted a lot. Since pgtable_l4|[l5]_enabled isn't changed after boot, we can use alternative mechanism to optimize them. So the question is whether we can add RISCV_ISA_EXT_SV48/SV5 and use riscv_has_extension_*() or not. Per [1] and [2], SV48 and SV57 are ISA exensions too. From another side, riscv_has_extension_[un]likely() and other related functions report whether the extension is supported and enabled on the platform. So SV48 and SV57 can be supported with current isa extension alternative mechanism. However, to use it to optimize pgtable_l4|l5_enabled, we have support the "early" characteristic, I.E besides risc_isa bitmap setting, we need to support appling alternative early before MMU on. After that, use it to optimize pgtable_l4|l5_enabled. For the typical access_ok(addr, 1); before the patch: ... auipc a5,0xb43 lbu a5,100(a5) # ffffffff80b51f68 bnez a5,ffffffff8000ef46 auipc a5,0xb43 lbu a5,91(a5) # ffffffff80b51f69 beqz a5,ffffffff8000ef5a ... after the patch: These memory load and test branch instructions are replaced with only two j or nop instructions. Initial test lmbench's lat_syscall write on TH1520 platforms shows that the write syscall latency is reduced by about 2.38%. Signed-off-by: Jisheng Zhang Link: https://github.com/riscv/riscv-isa-manual/blob/main/src/profiles/prof= iles.adoc [1] Link: https://riscv.atlassian.net/wiki/spaces/HOME/pages/16154732/Ratified+= ISA+Extensions [2] --- arch/riscv/Kconfig | 1 + arch/riscv/include/asm/alternative.h | 2 +- arch/riscv/include/asm/cpufeature.h | 2 + arch/riscv/include/asm/hwcap.h | 2 + arch/riscv/include/asm/pgtable-64.h | 12 ++++++ arch/riscv/kernel/alternative.c | 24 +++++++---- arch/riscv/kernel/cpufeature.c | 60 ++++++++++++++++++++++------ arch/riscv/mm/init.c | 9 +++++ 8 files changed, 91 insertions(+), 21 deletions(-) diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index 13b7bb77087e..e9476b8cbeb0 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -218,6 +218,7 @@ config RISCV select PCI_ECAM if (ACPI && PCI) select PCI_MSI if PCI select RELOCATABLE if !MMU && !PHYS_RAM_BASE_FIXED + select RISCV_ALTERNATIVE_EARLY if 64BIT select RISCV_APLIC select RISCV_IMSIC select RISCV_INTC diff --git a/arch/riscv/include/asm/alternative.h b/arch/riscv/include/asm/= alternative.h index 688c7d1a9ae3..6be7b2b6ade9 100644 --- a/arch/riscv/include/asm/alternative.h +++ b/arch/riscv/include/asm/alternative.h @@ -33,7 +33,7 @@ void __init apply_early_boot_alternatives(void); void apply_module_alternatives(void *start, size_t length); =20 void riscv_alternative_fix_offsets(void *alt_ptr, unsigned int len, - int patch_offset); + int patch_offset, bool early); =20 struct alt_entry { s32 old_offset; /* offset relative to original instruction or data */ diff --git a/arch/riscv/include/asm/cpufeature.h b/arch/riscv/include/asm/c= pufeature.h index 37c9f2a0fb54..af18ba93f580 100644 --- a/arch/riscv/include/asm/cpufeature.h +++ b/arch/riscv/include/asm/cpufeature.h @@ -36,6 +36,8 @@ extern const struct seq_operations cpuinfo_op; /* Per-cpu ISA extensions. */ extern struct riscv_isainfo hart_isa[NR_CPUS]; =20 +extern DECLARE_BITMAP(riscv_isa, RISCV_ISA_EXT_MAX); + extern u32 thead_vlenb_of; =20 void __init riscv_user_isa_enable(void); diff --git a/arch/riscv/include/asm/hwcap.h b/arch/riscv/include/asm/hwcap.h index f8db798b2654..b6d50025cb79 100644 --- a/arch/riscv/include/asm/hwcap.h +++ b/arch/riscv/include/asm/hwcap.h @@ -122,6 +122,8 @@ #define RISCV_ISA_EXT_ZICCAMOA 113 #define RISCV_ISA_EXT_ZICCIF 114 #define RISCV_ISA_EXT_ZA64RS 115 +#define RISCV_ISA_EXT_SV48 116 +#define RISCV_ISA_EXT_SV57 117 =20 #define RISCV_ISA_EXT_XLINUXENVCFG 127 =20 diff --git a/arch/riscv/include/asm/pgtable-64.h b/arch/riscv/include/asm/p= gtable-64.h index 72b8c63469fa..7e30ec0e9f6c 100644 --- a/arch/riscv/include/asm/pgtable-64.h +++ b/arch/riscv/include/asm/pgtable-64.h @@ -13,6 +13,7 @@ extern bool _pgtable_l4_enabled; extern bool _pgtable_l5_enabled; =20 +#ifdef USE_EARLY_PGTABLE_LEVELS static __always_inline bool pgtable_l5_enabled(void) { return _pgtable_l5_enabled; @@ -22,6 +23,17 @@ static __always_inline bool pgtable_l4_enabled(void) { return _pgtable_l4_enabled; } +#else +static __always_inline bool pgtable_l4_enabled(void) +{ + return riscv_has_extension_likely(RISCV_ISA_EXT_SV48); +} + +static __always_inline bool pgtable_l5_enabled(void) +{ + return riscv_has_extension_likely(RISCV_ISA_EXT_SV57); +} +#endif =20 #define PGDIR_SHIFT_L3 30 #define PGDIR_SHIFT_L4 39 diff --git a/arch/riscv/kernel/alternative.c b/arch/riscv/kernel/alternativ= e.c index c0c9306022c5..bbb215349452 100644 --- a/arch/riscv/kernel/alternative.c +++ b/arch/riscv/kernel/alternative.c @@ -75,7 +75,8 @@ static u32 riscv_instruction_at(void *p) } =20 static void riscv_alternative_fix_auipc_jalr(void *ptr, u32 auipc_insn, - u32 jalr_insn, int patch_offset) + u32 jalr_insn, int patch_offset, + bool early) { u32 call[2] =3D { auipc_insn, jalr_insn }; s32 imm; @@ -88,10 +89,15 @@ static void riscv_alternative_fix_auipc_jalr(void *ptr,= u32 auipc_insn, riscv_insn_insert_utype_itype_imm(&call[0], &call[1], imm); =20 /* patch the call place again */ - patch_text_nosync(ptr, call, sizeof(u32) * 2); + if (early) { + memcpy(ptr, call, sizeof(call)); + } else { + patch_text_nosync(ptr, call, sizeof(call)); + } } =20 -static void riscv_alternative_fix_jal(void *ptr, u32 jal_insn, int patch_o= ffset) +static void riscv_alternative_fix_jal(void *ptr, u32 jal_insn, int patch_o= ffset, + bool early) { s32 imm; =20 @@ -103,11 +109,14 @@ static void riscv_alternative_fix_jal(void *ptr, u32 = jal_insn, int patch_offset) riscv_insn_insert_jtype_imm(&jal_insn, imm); =20 /* patch the call place again */ - patch_text_nosync(ptr, &jal_insn, sizeof(u32)); + if (early) + memcpy(ptr, &jal_insn, sizeof(u32)); + else + patch_text_nosync(ptr, &jal_insn, sizeof(u32)); } =20 void riscv_alternative_fix_offsets(void *alt_ptr, unsigned int len, - int patch_offset) + int patch_offset, bool early) { int num_insn =3D len / sizeof(u32); int i; @@ -131,7 +140,8 @@ void riscv_alternative_fix_offsets(void *alt_ptr, unsig= ned int len, continue; =20 riscv_alternative_fix_auipc_jalr(alt_ptr + i * sizeof(u32), - insn, insn2, patch_offset); + insn, insn2, patch_offset, + early); i++; } =20 @@ -144,7 +154,7 @@ void riscv_alternative_fix_offsets(void *alt_ptr, unsig= ned int len, continue; =20 riscv_alternative_fix_jal(alt_ptr + i * sizeof(u32), - insn, patch_offset); + insn, patch_offset, early); } } } diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c index 9915121e9438..ce08e7f19e2d 100644 --- a/arch/riscv/kernel/cpufeature.c +++ b/arch/riscv/kernel/cpufeature.c @@ -39,7 +39,7 @@ static bool any_cpu_has_zicbom; unsigned long elf_hwcap __read_mostly; =20 /* Host ISA bitmap */ -static DECLARE_BITMAP(riscv_isa, RISCV_ISA_EXT_MAX) __read_mostly; +DECLARE_BITMAP(riscv_isa, RISCV_ISA_EXT_MAX) __read_mostly; =20 /* Per-cpu ISA extensions. */ struct riscv_isainfo hart_isa[NR_CPUS]; @@ -624,6 +624,8 @@ const struct riscv_isa_ext_data riscv_isa_ext[] =3D { __RISCV_ISA_EXT_DATA(svpbmt, RISCV_ISA_EXT_SVPBMT), __RISCV_ISA_EXT_DATA(svrsw60t59b, RISCV_ISA_EXT_SVRSW60T59B), __RISCV_ISA_EXT_DATA(svvptc, RISCV_ISA_EXT_SVVPTC), + __RISCV_ISA_EXT_DATA(sv48, RISCV_ISA_EXT_SV48), + __RISCV_ISA_EXT_DATA(sv57, RISCV_ISA_EXT_SV57), }; =20 const size_t riscv_isa_ext_count =3D ARRAY_SIZE(riscv_isa_ext); @@ -862,7 +864,8 @@ static void __init riscv_parse_isa_string(const char *i= sa, unsigned long *bitmap } } =20 -static void __init riscv_fill_hwcap_from_isa_string(unsigned long *isa2hwc= ap) +static void __init riscv_fill_hwcap_from_isa_string(unsigned long *isa2hwc= ap, + unsigned long *riscv_isa_early) { struct device_node *node; const char *isa; @@ -933,6 +936,7 @@ static void __init riscv_fill_hwcap_from_isa_string(uns= igned long *isa2hwcap) if (acpi_disabled && boot_vendorid =3D=3D THEAD_VENDOR_ID && boot_archid= =3D=3D 0x0) clear_bit(RISCV_ISA_EXT_V, source_isa); =20 + bitmap_or(source_isa, source_isa, riscv_isa_early, RISCV_ISA_EXT_MAX); riscv_resolve_isa(source_isa, isainfo->isa, &this_hwcap, isa2hwcap); =20 /* @@ -1050,7 +1054,8 @@ static int has_thead_homogeneous_vlenb(void) return 0; } =20 -static int __init riscv_fill_hwcap_from_ext_list(unsigned long *isa2hwcap) +static int __init riscv_fill_hwcap_from_ext_list(unsigned long *isa2hwcap, + unsigned long *riscv_isa_early) { unsigned int cpu; bool mitigated; @@ -1082,6 +1087,7 @@ static int __init riscv_fill_hwcap_from_ext_list(unsi= gned long *isa2hwcap) riscv_isa_set_ext(ext, source_isa); } =20 + bitmap_or(source_isa, source_isa, riscv_isa_early, RISCV_ISA_EXT_MAX); riscv_resolve_isa(source_isa, isainfo->isa, &this_hwcap, isa2hwcap); riscv_fill_cpu_vendor_ext(cpu_node, cpu); =20 @@ -1137,6 +1143,7 @@ void __init riscv_fill_hwcap(void) { char print_str[NUM_ALPHA_EXTS + 1]; unsigned long isa2hwcap[RISCV_ISA_EXT_BASE] =3D {0}; + DECLARE_BITMAP(riscv_isa_early, RISCV_ISA_EXT_MAX); int i, j; =20 isa2hwcap[RISCV_ISA_EXT_I] =3D COMPAT_HWCAP_ISA_I; @@ -1147,14 +1154,17 @@ void __init riscv_fill_hwcap(void) isa2hwcap[RISCV_ISA_EXT_C] =3D COMPAT_HWCAP_ISA_C; isa2hwcap[RISCV_ISA_EXT_V] =3D COMPAT_HWCAP_ISA_V; =20 + bitmap_copy(riscv_isa_early, riscv_isa, RISCV_ISA_EXT_MAX); + bitmap_zero(riscv_isa, RISCV_ISA_EXT_MAX); + if (!acpi_disabled) { - riscv_fill_hwcap_from_isa_string(isa2hwcap); + riscv_fill_hwcap_from_isa_string(isa2hwcap, riscv_isa_early); } else { - int ret =3D riscv_fill_hwcap_from_ext_list(isa2hwcap); + int ret =3D riscv_fill_hwcap_from_ext_list(isa2hwcap, riscv_isa_early); =20 if (ret && riscv_isa_fallback) { pr_info("Falling back to deprecated \"riscv,isa\"\n"); - riscv_fill_hwcap_from_isa_string(isa2hwcap); + riscv_fill_hwcap_from_isa_string(isa2hwcap, riscv_isa_early); } } =20 @@ -1250,6 +1260,17 @@ static bool riscv_cpufeature_patch_check(u16 id, u16= value) return false; } =20 +static bool __init_or_module riscv_is_isa_ext_early_id(u16 id) +{ + switch (id) { + case RISCV_ISA_EXT_SV48: + case RISCV_ISA_EXT_SV57: + return true; + } + + return false; +} + void __init_or_module riscv_cpufeature_patch_func(struct alt_entry *begin, struct alt_entry *end, unsigned int stage) @@ -1257,9 +1278,7 @@ void __init_or_module riscv_cpufeature_patch_func(str= uct alt_entry *begin, struct alt_entry *alt; void *oldptr, *altptr; u16 id, value, vendor; - - if (stage =3D=3D RISCV_ALTERNATIVES_EARLY_BOOT) - return; + bool early =3D stage =3D=3D RISCV_ALTERNATIVES_EARLY_BOOT; =20 for (alt =3D begin; alt < end; alt++) { id =3D PATCH_ID_CPUFEATURE_ID(alt->patch_id); @@ -1274,6 +1293,8 @@ void __init_or_module riscv_cpufeature_patch_func(str= uct alt_entry *begin, * vendor extension. */ if (id < RISCV_ISA_EXT_MAX) { + if (early && !riscv_is_isa_ext_early_id(id)) + continue; /* * This patch should be treated as errata so skip * processing here. @@ -1288,6 +1309,8 @@ void __init_or_module riscv_cpufeature_patch_func(str= uct alt_entry *begin, if (!riscv_cpufeature_patch_check(id, value)) continue; } else if (id >=3D RISCV_VENDOR_EXT_ALTERNATIVES_BASE) { + if (early) + continue; if (!__riscv_isa_vendor_extension_available(VENDOR_EXT_ALL_CPUS, vendor, id - RISCV_VENDOR_EXT_ALTERNATIVES_BASE)) continue; @@ -1299,9 +1322,20 @@ void __init_or_module riscv_cpufeature_patch_func(st= ruct alt_entry *begin, oldptr =3D ALT_OLD_PTR(alt); altptr =3D ALT_ALT_PTR(alt); =20 - mutex_lock(&text_mutex); - patch_text_nosync(oldptr, altptr, alt->alt_len); - riscv_alternative_fix_offsets(oldptr, alt->alt_len, oldptr - altptr); - mutex_unlock(&text_mutex); + if (early) { + /* oldptr is writable through the MMU-off kernel mapping. */ + memcpy(oldptr, altptr, alt->alt_len); + riscv_alternative_fix_offsets(oldptr, alt->alt_len, + oldptr - altptr, true); + } else { + mutex_lock(&text_mutex); + patch_text_nosync(oldptr, altptr, alt->alt_len); + riscv_alternative_fix_offsets(oldptr, alt->alt_len, + oldptr - altptr, false); + mutex_unlock(&text_mutex); + } } + + if (early) + local_flush_icache_all(); } diff --git a/arch/riscv/mm/init.c b/arch/riscv/mm/init.c index fc74142fe6e6..2a2402d79bc4 100644 --- a/arch/riscv/mm/init.c +++ b/arch/riscv/mm/init.c @@ -6,6 +6,11 @@ * Nick Kossifidis */ =20 +#ifdef CONFIG_64BIT +/* riscv_has_extension_likely() cannot be used this early */ +#define USE_EARLY_PGTABLE_LEVELS +#endif + #include #include #include @@ -892,6 +897,10 @@ static __init void set_satp_mode(uintptr_t dtb_pa) memset(early_p4d, 0, PAGE_SIZE); memset(early_pud, 0, PAGE_SIZE); memset(early_pmd, 0, PAGE_SIZE); + if (pgtable_l4_enabled()) + set_bit(RISCV_ISA_EXT_SV48, riscv_isa); + if (pgtable_l5_enabled()) + set_bit(RISCV_ISA_EXT_SV57, riscv_isa); } #endif =20 --=20 2.53.0 From nobody Fri Sep 25 23:08:55 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 49BA2509F16 for ; Mon, 7 Sep 2026 15:34:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788795288; cv=none; b=YlTOOqvGKAvDGNCRCkrCMgp7VSLNuywybXUGF6ZRF5QyJjanrgDGIfe2pr9Euqbo2wwkYOkcGMOQCKkGQdBaED8CSSQ2fJ7zQVdA+IL6rISxv69GwbluWoNGUZQl9X2NqdJ4sZCnsidzyZh4VPk4WIZLTMupGB9aJ9/NZC/8/YE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788795288; c=relaxed/simple; bh=aUqa+U0rvYDMuxwBfMOm7AffnEN6LwPYcMtdpT7cB1s=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=RfI8pOGkeQqsRoO2lXOtmXtRTDMWlcdE61ivy5XPKg8jmcaoIgitlSqZEi4pevG6M40yRat8Q1Aw82EiNehfgJJGsbf37Qt1ipA8Cpn23+w2c/dnwvbIbkwRZcos5vMBgC16ioNwAf+LosR6VtvUnXPGxQd+aVjpmLZevwc6X2M= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=biphSlJw; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="biphSlJw" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5F28F1F00A3F; Mon, 7 Sep 2026 15:34:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1788795284; bh=rUtjrYEbG1KraQAUaYquvt0ku0ZcfAlrQLb/m0sJffw=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=biphSlJwKgERHw41gbSpl409w9CPUrCx+oUvKD1kExC314T9bfsjIBHLcWAAJfggO fXgCo477UfIL4G2wpxQZytwZZvyDas9nlSXqdFA5rWg/9nmktjQ8+3cFoH1//xrlOd fnipba2ZH9L8MroTcdLfH2Akq/xYbH51tJosJKZK/zE1zwMChOQTy6rpS9cM52/wMU g8n35syniHXawoapW0+1ySHjhiMjl7ZenSh+0BwIeogDz9RRCx3UCJz/vP5eh2Pr7X rrmiMy8QBHgQrZxF4/7agvtnKeJZG1Te9UWwsb1TutArFBYYXMSmc7b2KSdyw44oS+ kX1Dg8wLJ9t6g== From: Jisheng Zhang To: Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexandre Ghiti , Andrey Ryabinin , Alexander Potapenko , Andrey Konovalov , Dmitry Vyukov , Vincenzo Frascino Cc: linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com Subject: [PATCH v2 4/5] riscv: RISCV_ISA_SV48 and RISCV_ISA_SV57 under NONPORTABLE Date: Mon, 7 Sep 2026 23:14:36 +0800 Message-ID: <20260907151437.7603-5-jszhang@kernel.org> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260907151437.7603-1-jszhang@kernel.org> References: <20260907151437.7603-1-jszhang@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" In last commit, we have optimized the pgtable_l4|l5_enabled with isa extension alternative mechanism, thus a typical access_ok(addr, 1); is compiled as: ffffffff8001dd30: 1141 addi sp,sp,-16 ffffffff8001dd32: e022 sd s0,0(sp) ffffffff8001dd34: e406 sd ra,8(sp) ffffffff8001dd36: 0800 addi s0,sp,16 ffffffff8001dd38: 00000013 nop ffffffff8001dd3c: 02a0006f j ffffffff8001dd66 ffffffff8001dd40: 60a2 ld ra,8(sp) ffffffff8001dd42: 6402 ld s0,0(sp) ffffffff8001dd44: 57fd li a5,-1 ffffffff8001dd46: 83a1 srli a5,a5,0x8 ffffffff8001dd48: 00a7b533 sltu a0,a5,a0 ffffffff8001dd4c: 00154513 xori a0,a0,1 ffffffff8001dd50: 0141 addi sp,sp,16 ffffffff8001dd52: 8082 ret ... ffffffff8001dd64: bfe1 j ffffffff8001dd3c ffffffff8001dd66: 0180006f j ffffffff8001dd7e ffffffff8001dd6a: 60a2 ld ra,8(sp) ffffffff8001dd6c: 6402 ld s0,0(sp) ffffffff8001dd6e: 57fd li a5,-1 ffffffff8001dd70: 83c5 srli a5,a5,0x11 ffffffff8001dd72: 00a7b533 sltu a0,a5,a0 ffffffff8001dd76: 00154513 xori a0,a0,1 ffffffff8001dd7a: 0141 addi sp,sp,16 ffffffff8001dd7c: 8082 ret ffffffff8001dd7e: 60a2 ld ra,8(sp) ffffffff8001dd80: 6402 ld s0,0(sp) ffffffff8001dd82: 57fd li a5,-1 ffffffff8001dd84: 83e9 srli a5,a5,0x1a ffffffff8001dd86: 00a7b533 sltu a0,a5,a0 ffffffff8001dd8a: 00154513 xori a0,a0,1 ffffffff8001dd8e: 0141 addi sp,sp,16 ffffffff8001dd90: 8082 ret As can be seen, different branches for SV39/SV48/SV57 are still there, since access_ok() sits at hot code path, why not reduce the code size to optimize to make the instruction cache happy? Introduce RISCV_ISA_SV48 and RISCV_ISA_SV57 under NONPORTABLE, so that the embedded platforms can choose the best option themselves, while still keep the feature of unified one kernel Image for all SV39, SV48 and SV57. Before the patch, vmlinux built with pure RISCV64 defconfig text data bss dec hex filename 13439437 6790446 483221 20713104 13c0e90 /tmp/old/vmlinux After the patch, vmlinux built with pure RISCV64 defconfig: text data bss dec hex filename 13314445 6787934 466837 20569216 139dc80 /tmp/new/vmlinux .text section is reduced by 122KB! And after the patch, typical access_ok(addr, 1) is compiled as: ffffffff8001809a: 1141 addi sp,sp,-16 ffffffff8001809c: e022 sd s0,0(sp) ffffffff8001809e: e406 sd ra,8(sp) ffffffff800180a0: 0800 addi s0,sp,16 ffffffff800180a2: 00000013 nop ffffffff800180a6: 60a2 ld ra,8(sp) ffffffff800180a8: 6402 ld s0,0(sp) ffffffff800180aa: 4785 li a5,1 ffffffff800180ac: 179a slli a5,a5,0x26 ffffffff800180ae: 00f53533 sltu a0,a0,a5 ffffffff800180b2: 0141 addi sp,sp,16 ffffffff800180b4: 8082 ret only 12 instruction!! For refernece, before the patch, the access(addr, 1) needs 32 instructions. Signed-off-by: Jisheng Zhang --- arch/riscv/Kconfig | 24 ++++++++++++++++++++++++ arch/riscv/include/asm/pgtable-64.h | 12 ++++++++++++ arch/riscv/mm/init.c | 10 ++++++++++ 3 files changed, 46 insertions(+) diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index e9476b8cbeb0..4cdb1e3f34be 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -602,6 +602,30 @@ config RISCV_ISA_SSQOSID cache partitioning scheme and use the MCID to track how much cache a process, or a group of processes, is using. =20 +config RISCV_ISA_SV48 + bool "Page-Based 48-bit Virtual-Memory System" + depends on 64BIT && NONPORTABLE + default y + help + Add support for the Page-Based 48-bit Virtual-Memory System. + + Enable this option for systems for which a 39-bit virtual address + space is insufficient. But if 39-bit is enough, disabling this + option will result in smaller kernel size and slighly better + performance. + +config RISCV_ISA_SV57 + bool "Page-Based 57-bit Virtual-Memory System" + depends on 64BIT && NONPORTABLE + default y + help + Add support for the Page-Based 57-bit Virtual-Memory System. + + Enable this option for systems for which a 48-bit virtual address + space is insufficient. But if 48-bit is enough, disabling this + option will result in smaller kernel size and slighly better + performance. + config RISCV_ISA_SVPBMT bool "Svpbmt extension support for supervisor mode page-based memory type= s" depends on 64BIT && MMU diff --git a/arch/riscv/include/asm/pgtable-64.h b/arch/riscv/include/asm/p= gtable-64.h index 7e30ec0e9f6c..f2d29da93eea 100644 --- a/arch/riscv/include/asm/pgtable-64.h +++ b/arch/riscv/include/asm/pgtable-64.h @@ -16,21 +16,33 @@ extern bool _pgtable_l5_enabled; #ifdef USE_EARLY_PGTABLE_LEVELS static __always_inline bool pgtable_l5_enabled(void) { + if (IS_ENABLED(CONFIG_NONPORTABLE) && !IS_ENABLED(CONFIG_RISCV_ISA_SV57)) + return false; + return _pgtable_l5_enabled; } =20 static __always_inline bool pgtable_l4_enabled(void) { + if (IS_ENABLED(CONFIG_NONPORTABLE) && !IS_ENABLED(CONFIG_RISCV_ISA_SV48)) + return false; + return _pgtable_l4_enabled; } #else static __always_inline bool pgtable_l4_enabled(void) { + if (IS_ENABLED(CONFIG_NONPORTABLE) && !IS_ENABLED(CONFIG_RISCV_ISA_SV48)) + return false; + return riscv_has_extension_likely(RISCV_ISA_EXT_SV48); } =20 static __always_inline bool pgtable_l5_enabled(void) { + if (IS_ENABLED(CONFIG_NONPORTABLE) && !IS_ENABLED(CONFIG_RISCV_ISA_SV57)) + return false; + return riscv_has_extension_likely(RISCV_ISA_EXT_SV57); } #endif diff --git a/arch/riscv/mm/init.c b/arch/riscv/mm/init.c index 2a2402d79bc4..90bb85c5f167 100644 --- a/arch/riscv/mm/init.c +++ b/arch/riscv/mm/init.c @@ -844,6 +844,16 @@ static __init void set_satp_mode(uintptr_t dtb_pa) =20 kernel_map.page_offset =3D PAGE_OFFSET_L5; =20 + /* Fall back to Sv48 if Sv57 is unsupported on non-portable systems */ + if (IS_ENABLED(CONFIG_NONPORTABLE) && !IS_ENABLED(CONFIG_RISCV_ISA_SV57) = && + satp_mode_limit =3D=3D 0) + satp_mode_limit =3D SATP_MODE_48; + + /* Fall back to Sv39 if Sv48 is unsupported on non-portable systems */ + if (IS_ENABLED(CONFIG_NONPORTABLE) && !IS_ENABLED(CONFIG_RISCV_ISA_SV48) = && + satp_mode_limit =3D=3D SATP_MODE_48) + satp_mode_limit =3D SATP_MODE_39; + if (satp_mode_limit =3D=3D SATP_MODE_48) { disable_pgtable_l5(); } else if (satp_mode_limit =3D=3D SATP_MODE_39) { --=20 2.53.0 From nobody Fri Sep 25 23:08:55 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C72B7471439 for ; Mon, 7 Sep 2026 15:34:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788795290; cv=none; b=gl8QDydu135B7iKHOvz9ghrVHVdpaH0JlQBm/dnIMmpdWanpUzNoWmy8M/8ERm28QynXu5eU8zVlE70L6S+2v9dL7V/E/YlqhiOg9IfRW30x4rNlnmCtgG9qpLVC30Ylc//yLhxhEQLEPj2a6M8NK5AP2kpH0VKmuQbmYCExDOg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788795290; c=relaxed/simple; bh=HRopG0SjAFWAqjDfgeMCYjhPxT9GEmqSvnctlvCkhM0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=YuScoLzE1CAAj8w6HLPSl49U5GKtR/AfhCQDxIJU1UwbDKlhauapGVmGsqyXtUvrO8pq7xZXobGEasGK2vW8/0eoQWT563RJpFj8UGj8H4t6A4IWZ6hJqA3Ef5cuF5CXNfxVRklezz78uSpUVpGAMmiaSht8vvJqKt3DHDhfilk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Jeq0YGq6; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Jeq0YGq6" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 88AB91F00A3A; Mon, 7 Sep 2026 15:34:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1788795287; bh=A1WMEWftQfe3FQuEVzWJAAShiADDVwkA0Mf0m9ZcbKs=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=Jeq0YGq6bhKq73IyNWav30rXrgwxrFcZlZSb8uJISuCW4zLxIjOaQSLDYNxzQvirr No9pe5mXFeW64/rURB9fjvzy6xyBusMU4YYu1W8oLsLoeD+aWH9OISsB8VHh5s/fXO OXDhk7Swbl2fxLB2gHF3jt34WhbUkbqUyNvkwEg9NwcMbuwSDymSKxe5K4RENv+faw Pzz2pG4f/8PAGm9VM18EXzp/o90TwRRuMrKfNQpyz0klys9+7zYNdX42Sxeqo33kmg pYbkBOYxbsIb+RsJJA728p1sfkRbOiBLXaVRbACUHyUS1xh8tEsFGMCzGxfCDuEYOJ Bi3jTt2+5E+9w== From: Jisheng Zhang To: Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexandre Ghiti , Andrey Ryabinin , Alexander Potapenko , Andrey Konovalov , Dmitry Vyukov , Vincenzo Frascino Cc: linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com Subject: [PATCH v2 5/5] riscv: mm: unexport _pgtable_l4_enabled and _pgtable_l5_enabled Date: Mon, 7 Sep 2026 23:14:37 +0800 Message-ID: <20260907151437.7603-6-jszhang@kernel.org> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260907151437.7603-1-jszhang@kernel.org> References: <20260907151437.7603-1-jszhang@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Now, these two vars are only used by kernel itself, especially the code before MMU on. No modules users any more, unexport them. Signed-off-by: Jisheng Zhang --- arch/riscv/mm/init.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/arch/riscv/mm/init.c b/arch/riscv/mm/init.c index 90bb85c5f167..ed800a4e0e8c 100644 --- a/arch/riscv/mm/init.c +++ b/arch/riscv/mm/init.c @@ -59,8 +59,6 @@ EXPORT_SYMBOL(satp_mode); #ifdef CONFIG_64BIT bool _pgtable_l4_enabled __ro_after_init =3D true; bool _pgtable_l5_enabled __ro_after_init =3D true; -EXPORT_SYMBOL(_pgtable_l4_enabled); -EXPORT_SYMBOL(_pgtable_l5_enabled); #endif =20 phys_addr_t phys_ram_base __ro_after_init; --=20 2.53.0