From nobody Tue Jun 23 05:03:54 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 10D1FC43217 for ; Thu, 10 Mar 2022 14:33:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238431AbiCJOed (ORCPT ); Thu, 10 Mar 2022 09:34:33 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51972 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S244458AbiCJO25 (ORCPT ); Thu, 10 Mar 2022 09:28:57 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 58327D2079; Thu, 10 Mar 2022 06:24:01 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id A4080B825A7; Thu, 10 Mar 2022 14:23:41 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id F1BB3C340E8; Thu, 10 Mar 2022 14:23:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1646922220; bh=LzEuzEp1gkK+XkCfZ9yJFcbKa4Vp7YQdCRdeK3/79vc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=VTRfuazlrjw9j2nbtDAjik/I5fF75VbYC9U0KyjhUI+3cprSY6i3Z+rSFMuLjBWVI R7nt32BNfQ5oAxZf+wv+F3AfbYv4SGUodOOQgen0F4Ys0WKWcU26jZbLrbvdgOd3Tk Q4Bv9hbH6FGjSlAeRaxCAIy7rlCp1BXgnKRp4VyU= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, "Peter Zijlstra (Intel)" , Borislav Petkov , Josh Poimboeuf , Alexei Starovoitov Subject: [PATCH 5.10 01/58] x86,bugs: Unconditionally allow spectre_v2=retpoline,amd Date: Thu, 10 Mar 2022 15:18:21 +0100 Message-Id: <20220310140812.913098399@linuxfoundation.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220310140812.869208747@linuxfoundation.org> References: <20220310140812.869208747@linuxfoundation.org> User-Agent: quilt/0.66 X-stable: review X-Patchwork-Hint: ignore MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Peter Zijlstra commit f8a66d608a3e471e1202778c2a36cbdc96bae73b upstream. Currently Linux prevents usage of retpoline,amd on !AMD hardware, this is unfriendly and gets in the way of testing. Remove this restriction. Signed-off-by: Peter Zijlstra (Intel) Reviewed-by: Borislav Petkov Acked-by: Josh Poimboeuf Tested-by: Alexei Starovoitov Link: https://lore.kernel.org/r/20211026120310.487348118@infradead.org Signed-off-by: Greg Kroah-Hartman Tested-by: Bagas Sanjaya Tested-by: Florian Fainelli Tested-by: Fox Chen Tested-by: Guenter Roeck Tested-by: Linux Kernel Functional Testing Tested-by: Pavel Machek (CIP) = =20 Tested-by: Shuah Khan Tested-by: Sudip Mukherjee --- arch/x86/kernel/cpu/bugs.c | 7 ------- 1 file changed, 7 deletions(-) --- a/arch/x86/kernel/cpu/bugs.c +++ b/arch/x86/kernel/cpu/bugs.c @@ -845,13 +845,6 @@ static enum spectre_v2_mitigation_cmd __ return SPECTRE_V2_CMD_AUTO; } =20 - if (cmd =3D=3D SPECTRE_V2_CMD_RETPOLINE_AMD && - boot_cpu_data.x86_vendor !=3D X86_VENDOR_HYGON && - boot_cpu_data.x86_vendor !=3D X86_VENDOR_AMD) { - pr_err("retpoline,amd selected but CPU is not AMD. Switching to AUTO sel= ect\n"); - return SPECTRE_V2_CMD_AUTO; - } - spec_v2_print_cond(mitigation_options[i].option, mitigation_options[i].secure); return cmd; From nobody Tue Jun 23 05:03:54 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4838DC433F5 for ; Thu, 10 Mar 2022 14:34:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236545AbiCJOfG (ORCPT ); Thu, 10 Mar 2022 09:35:06 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50134 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S244536AbiCJO3Q (ORCPT ); Thu, 10 Mar 2022 09:29:16 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 422A3D3AC7; Thu, 10 Mar 2022 06:24:09 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id DDA6061CEE; Thu, 10 Mar 2022 14:23:53 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id EE80BC340F3; Thu, 10 Mar 2022 14:23:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1646922233; bh=sHBaHcUjLV8fVZHhrO9dbiOsJRcm/xEqOES3gSji6w8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=hzut2IRJ81b5A2iLHpWEM1Opfz2bu9Wy+R8WiKVJncDl4lDB5W+i1/LRHaQH+ez14 zjqWCvMdwileBEF9/zJuIN9MZbNBmq+EKw2JGLCW81bBphXXKqHMJFr3dle3OVeCCS l5s2k1WoLuU7rWGYzST0DnJyHt2nr3AbI+t8pzlA= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Josh Poimboeuf , "Peter Zijlstra (Intel)" , Borislav Petkov , Thomas Gleixner , Frank van der Linden Subject: [PATCH 5.10 02/58] x86/speculation: Rename RETPOLINE_AMD to RETPOLINE_LFENCE Date: Thu, 10 Mar 2022 15:18:22 +0100 Message-Id: <20220310140812.941915745@linuxfoundation.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220310140812.869208747@linuxfoundation.org> References: <20220310140812.869208747@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: "Peter Zijlstra (Intel)" commit d45476d9832409371537013ebdd8dc1a7781f97a upstream. The RETPOLINE_AMD name is unfortunate since it isn't necessarily AMD only, in fact Hygon also uses it. Furthermore it will likely be sufficient for some Intel processors. Therefore rename the thing to RETPOLINE_LFENCE to better describe what it is. Add the spectre_v2=3Dretpoline,lfence option as an alias to spectre_v2=3Dretpoline,amd to preserve existing setups. However, the output of /sys/devices/system/cpu/vulnerabilities/spectre_v2 will be changed. [ bp: Fix typos, massage. ] Co-developed-by: Josh Poimboeuf Signed-off-by: Josh Poimboeuf Signed-off-by: Peter Zijlstra (Intel) Signed-off-by: Borislav Petkov Reviewed-by: Thomas Gleixner [fllinden@amazon.com: backported to 5.10] Signed-off-by: Frank van der Linden Signed-off-by: Greg Kroah-Hartman Tested-by: Bagas Sanjaya Tested-by: Florian Fainelli Tested-by: Fox Chen Tested-by: Guenter Roeck Tested-by: Linux Kernel Functional Testing Tested-by: Pavel Machek (CIP) = =20 Tested-by: Shuah Khan Tested-by: Sudip Mukherjee --- arch/x86/include/asm/cpufeatures.h | 2 +- arch/x86/include/asm/nospec-branch.h | 12 ++++++------ arch/x86/kernel/cpu/bugs.c | 29 ++++++++++++++++++--------= --- tools/arch/x86/include/asm/cpufeatures.h | 2 +- 4 files changed, 26 insertions(+), 19 deletions(-) --- a/arch/x86/include/asm/cpufeatures.h +++ b/arch/x86/include/asm/cpufeatures.h @@ -204,7 +204,7 @@ #define X86_FEATURE_SME ( 7*32+10) /* AMD Secure Memory Encryption */ #define X86_FEATURE_PTI ( 7*32+11) /* Kernel Page Table Isolation enable= d */ #define X86_FEATURE_RETPOLINE ( 7*32+12) /* "" Generic Retpoline mitigati= on for Spectre variant 2 */ -#define X86_FEATURE_RETPOLINE_AMD ( 7*32+13) /* "" AMD Retpoline mitigatio= n for Spectre variant 2 */ +#define X86_FEATURE_RETPOLINE_LFENCE ( 7*32+13) /* "" Use LFENCE for Spect= re variant 2 */ #define X86_FEATURE_INTEL_PPIN ( 7*32+14) /* Intel Processor Inventory Nu= mber */ #define X86_FEATURE_CDP_L2 ( 7*32+15) /* Code and Data Prioritization L2 = */ #define X86_FEATURE_MSR_SPEC_CTRL ( 7*32+16) /* "" MSR SPEC_CTRL is implem= ented */ --- a/arch/x86/include/asm/nospec-branch.h +++ b/arch/x86/include/asm/nospec-branch.h @@ -82,7 +82,7 @@ #ifdef CONFIG_RETPOLINE ALTERNATIVE_2 __stringify(ANNOTATE_RETPOLINE_SAFE; jmp *%\reg), \ __stringify(jmp __x86_retpoline_\reg), X86_FEATURE_RETPOLINE, \ - __stringify(lfence; ANNOTATE_RETPOLINE_SAFE; jmp *%\reg), X86_FEAT= URE_RETPOLINE_AMD + __stringify(lfence; ANNOTATE_RETPOLINE_SAFE; jmp *%\reg), X86_FEAT= URE_RETPOLINE_LFENCE #else jmp *%\reg #endif @@ -92,7 +92,7 @@ #ifdef CONFIG_RETPOLINE ALTERNATIVE_2 __stringify(ANNOTATE_RETPOLINE_SAFE; call *%\reg), \ __stringify(call __x86_retpoline_\reg), X86_FEATURE_RETPOLINE, \ - __stringify(lfence; ANNOTATE_RETPOLINE_SAFE; call *%\reg), X86_FEA= TURE_RETPOLINE_AMD + __stringify(lfence; ANNOTATE_RETPOLINE_SAFE; call *%\reg), X86_FEA= TURE_RETPOLINE_LFENCE #else call *%\reg #endif @@ -134,7 +134,7 @@ "lfence;\n" \ ANNOTATE_RETPOLINE_SAFE \ "call *%[thunk_target]\n", \ - X86_FEATURE_RETPOLINE_AMD) + X86_FEATURE_RETPOLINE_LFENCE) =20 # define THUNK_TARGET(addr) [thunk_target] "r" (addr) =20 @@ -164,7 +164,7 @@ "lfence;\n" \ ANNOTATE_RETPOLINE_SAFE \ "call *%[thunk_target]\n", \ - X86_FEATURE_RETPOLINE_AMD) + X86_FEATURE_RETPOLINE_LFENCE) =20 # define THUNK_TARGET(addr) [thunk_target] "rm" (addr) #endif @@ -176,8 +176,8 @@ /* The Spectre V2 mitigation variants */ enum spectre_v2_mitigation { SPECTRE_V2_NONE, - SPECTRE_V2_RETPOLINE_GENERIC, - SPECTRE_V2_RETPOLINE_AMD, + SPECTRE_V2_RETPOLINE, + SPECTRE_V2_LFENCE, SPECTRE_V2_IBRS_ENHANCED, }; =20 --- a/arch/x86/kernel/cpu/bugs.c +++ b/arch/x86/kernel/cpu/bugs.c @@ -627,7 +627,7 @@ enum spectre_v2_mitigation_cmd { SPECTRE_V2_CMD_FORCE, SPECTRE_V2_CMD_RETPOLINE, SPECTRE_V2_CMD_RETPOLINE_GENERIC, - SPECTRE_V2_CMD_RETPOLINE_AMD, + SPECTRE_V2_CMD_RETPOLINE_LFENCE, }; =20 enum spectre_v2_user_cmd { @@ -787,8 +787,8 @@ set_mode: =20 static const char * const spectre_v2_strings[] =3D { [SPECTRE_V2_NONE] =3D "Vulnerable", - [SPECTRE_V2_RETPOLINE_GENERIC] =3D "Mitigation: Full generic retpoline", - [SPECTRE_V2_RETPOLINE_AMD] =3D "Mitigation: Full AMD retpoline", + [SPECTRE_V2_RETPOLINE] =3D "Mitigation: Retpolines", + [SPECTRE_V2_LFENCE] =3D "Mitigation: LFENCE", [SPECTRE_V2_IBRS_ENHANCED] =3D "Mitigation: Enhanced IBRS", }; =20 @@ -800,7 +800,8 @@ static const struct { { "off", SPECTRE_V2_CMD_NONE, false }, { "on", SPECTRE_V2_CMD_FORCE, true }, { "retpoline", SPECTRE_V2_CMD_RETPOLINE, false }, - { "retpoline,amd", SPECTRE_V2_CMD_RETPOLINE_AMD, false }, + { "retpoline,amd", SPECTRE_V2_CMD_RETPOLINE_LFENCE, false }, + { "retpoline,lfence", SPECTRE_V2_CMD_RETPOLINE_LFENCE, false }, { "retpoline,generic", SPECTRE_V2_CMD_RETPOLINE_GENERIC, false }, { "auto", SPECTRE_V2_CMD_AUTO, false }, }; @@ -838,13 +839,19 @@ static enum spectre_v2_mitigation_cmd __ } =20 if ((cmd =3D=3D SPECTRE_V2_CMD_RETPOLINE || - cmd =3D=3D SPECTRE_V2_CMD_RETPOLINE_AMD || + cmd =3D=3D SPECTRE_V2_CMD_RETPOLINE_LFENCE || cmd =3D=3D SPECTRE_V2_CMD_RETPOLINE_GENERIC) && !IS_ENABLED(CONFIG_RETPOLINE)) { pr_err("%s selected but not compiled in. Switching to AUTO select\n", mi= tigation_options[i].option); return SPECTRE_V2_CMD_AUTO; } =20 + if ((cmd =3D=3D SPECTRE_V2_CMD_RETPOLINE_LFENCE) && + !boot_cpu_has(X86_FEATURE_LFENCE_RDTSC)) { + pr_err("%s selected, but CPU doesn't have a serializing LFENCE. Switchin= g to AUTO select\n", mitigation_options[i].option); + return SPECTRE_V2_CMD_AUTO; + } + spec_v2_print_cond(mitigation_options[i].option, mitigation_options[i].secure); return cmd; @@ -879,9 +886,9 @@ static void __init spectre_v2_select_mit if (IS_ENABLED(CONFIG_RETPOLINE)) goto retpoline_auto; break; - case SPECTRE_V2_CMD_RETPOLINE_AMD: + case SPECTRE_V2_CMD_RETPOLINE_LFENCE: if (IS_ENABLED(CONFIG_RETPOLINE)) - goto retpoline_amd; + goto retpoline_lfence; break; case SPECTRE_V2_CMD_RETPOLINE_GENERIC: if (IS_ENABLED(CONFIG_RETPOLINE)) @@ -898,17 +905,17 @@ static void __init spectre_v2_select_mit retpoline_auto: if (boot_cpu_data.x86_vendor =3D=3D X86_VENDOR_AMD || boot_cpu_data.x86_vendor =3D=3D X86_VENDOR_HYGON) { - retpoline_amd: + retpoline_lfence: if (!boot_cpu_has(X86_FEATURE_LFENCE_RDTSC)) { pr_err("Spectre mitigation: LFENCE not serializing, switching to generi= c retpoline\n"); goto retpoline_generic; } - mode =3D SPECTRE_V2_RETPOLINE_AMD; - setup_force_cpu_cap(X86_FEATURE_RETPOLINE_AMD); + mode =3D SPECTRE_V2_LFENCE; + setup_force_cpu_cap(X86_FEATURE_RETPOLINE_LFENCE); setup_force_cpu_cap(X86_FEATURE_RETPOLINE); } else { retpoline_generic: - mode =3D SPECTRE_V2_RETPOLINE_GENERIC; + mode =3D SPECTRE_V2_RETPOLINE; setup_force_cpu_cap(X86_FEATURE_RETPOLINE); } =20 --- a/tools/arch/x86/include/asm/cpufeatures.h +++ b/tools/arch/x86/include/asm/cpufeatures.h @@ -204,7 +204,7 @@ #define X86_FEATURE_SME ( 7*32+10) /* AMD Secure Memory Encryption */ #define X86_FEATURE_PTI ( 7*32+11) /* Kernel Page Table Isolation enable= d */ #define X86_FEATURE_RETPOLINE ( 7*32+12) /* "" Generic Retpoline mitigati= on for Spectre variant 2 */ -#define X86_FEATURE_RETPOLINE_AMD ( 7*32+13) /* "" AMD Retpoline mitigatio= n for Spectre variant 2 */ +#define X86_FEATURE_RETPOLINE_LFENCE ( 7*32+13) /* "" Use LFENCEs for Spec= tre variant 2 */ #define X86_FEATURE_INTEL_PPIN ( 7*32+14) /* Intel Processor Inventory Nu= mber */ #define X86_FEATURE_CDP_L2 ( 7*32+15) /* Code and Data Prioritization L2 = */ #define X86_FEATURE_MSR_SPEC_CTRL ( 7*32+16) /* "" MSR SPEC_CTRL is implem= ented */ From nobody Tue Jun 23 05:03:54 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 78B9AC4167E for ; Thu, 10 Mar 2022 14:42:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344715AbiCJOlI (ORCPT ); Thu, 10 Mar 2022 09:41:08 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51148 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S244548AbiCJO3Q (ORCPT ); Thu, 10 Mar 2022 09:29:16 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C9C16D4452; Thu, 10 Mar 2022 06:24:10 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id E971861CF0; Thu, 10 Mar 2022 14:23:56 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id C58BFC340EB; Thu, 10 Mar 2022 14:23:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1646922236; bh=Qx+KHgV/ydSEsi4ov7VXFZFQ7ycDkH0YTJVrmXMB47o=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=l2lVR3g0Vf6XxgGfxfp4+9JsSTqaXDAv7jP6M7FVY4FMypVPmiMV/Pu16y9a+vFRH jHGMshqjmegqld5/yl+L4ODuEXC/wMCva0hkUwXgSU4dHwC2TSyKK+S8TM1a48R/sU 9n7DFn6rrrncFRz+s4pesrisBofmInGyKeZSW6fI= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Josh Poimboeuf , "Peter Zijlstra (Intel)" , Borislav Petkov , Patrick Colp , Thomas Gleixner Subject: [PATCH 5.10 03/58] x86/speculation: Add eIBRS + Retpoline options Date: Thu, 10 Mar 2022 15:18:23 +0100 Message-Id: <20220310140812.969972842@linuxfoundation.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220310140812.869208747@linuxfoundation.org> References: <20220310140812.869208747@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Peter Zijlstra commit 1e19da8522c81bf46b335f84137165741e0d82b7 upstream. Thanks to the chaps at VUsec it is now clear that eIBRS is not sufficient, therefore allow enabling of retpolines along with eIBRS. Add spectre_v2=3Deibrs, spectre_v2=3Deibrs,lfence and spectre_v2=3Deibrs,retpoline options to explicitly pick your preferred means of mitigation. Since there's new mitigations there's also user visible changes in /sys/devices/system/cpu/vulnerabilities/spectre_v2 to reflect these new mitigations. [ bp: Massage commit message, trim error messages, do more precise eIBRS mode checking. ] Co-developed-by: Josh Poimboeuf Signed-off-by: Josh Poimboeuf Signed-off-by: Peter Zijlstra (Intel) Signed-off-by: Borislav Petkov Reviewed-by: Patrick Colp Reviewed-by: Thomas Gleixner Signed-off-by: Greg Kroah-Hartman Tested-by: Bagas Sanjaya Tested-by: Florian Fainelli Tested-by: Fox Chen Tested-by: Guenter Roeck Tested-by: Linux Kernel Functional Testing Tested-by: Pavel Machek (CIP) = =20 Tested-by: Shuah Khan Tested-by: Sudip Mukherjee --- arch/x86/include/asm/nospec-branch.h | 4 - arch/x86/kernel/cpu/bugs.c | 133 +++++++++++++++++++++++++-----= ----- 2 files changed, 99 insertions(+), 38 deletions(-) --- a/arch/x86/include/asm/nospec-branch.h +++ b/arch/x86/include/asm/nospec-branch.h @@ -178,7 +178,9 @@ enum spectre_v2_mitigation { SPECTRE_V2_NONE, SPECTRE_V2_RETPOLINE, SPECTRE_V2_LFENCE, - SPECTRE_V2_IBRS_ENHANCED, + SPECTRE_V2_EIBRS, + SPECTRE_V2_EIBRS_RETPOLINE, + SPECTRE_V2_EIBRS_LFENCE, }; =20 /* The indirect branch speculation control variants */ --- a/arch/x86/kernel/cpu/bugs.c +++ b/arch/x86/kernel/cpu/bugs.c @@ -628,6 +628,9 @@ enum spectre_v2_mitigation_cmd { SPECTRE_V2_CMD_RETPOLINE, SPECTRE_V2_CMD_RETPOLINE_GENERIC, SPECTRE_V2_CMD_RETPOLINE_LFENCE, + SPECTRE_V2_CMD_EIBRS, + SPECTRE_V2_CMD_EIBRS_RETPOLINE, + SPECTRE_V2_CMD_EIBRS_LFENCE, }; =20 enum spectre_v2_user_cmd { @@ -700,6 +703,13 @@ spectre_v2_parse_user_cmdline(enum spect return SPECTRE_V2_USER_CMD_AUTO; } =20 +static inline bool spectre_v2_in_eibrs_mode(enum spectre_v2_mitigation mod= e) +{ + return (mode =3D=3D SPECTRE_V2_EIBRS || + mode =3D=3D SPECTRE_V2_EIBRS_RETPOLINE || + mode =3D=3D SPECTRE_V2_EIBRS_LFENCE); +} + static void __init spectre_v2_user_select_mitigation(enum spectre_v2_mitigation_cmd v2_cmd) { @@ -767,7 +777,7 @@ spectre_v2_user_select_mitigation(enum s */ if (!boot_cpu_has(X86_FEATURE_STIBP) || !smt_possible || - spectre_v2_enabled =3D=3D SPECTRE_V2_IBRS_ENHANCED) + spectre_v2_in_eibrs_mode(spectre_v2_enabled)) return; =20 /* @@ -789,7 +799,9 @@ static const char * const spectre_v2_str [SPECTRE_V2_NONE] =3D "Vulnerable", [SPECTRE_V2_RETPOLINE] =3D "Mitigation: Retpolines", [SPECTRE_V2_LFENCE] =3D "Mitigation: LFENCE", - [SPECTRE_V2_IBRS_ENHANCED] =3D "Mitigation: Enhanced IBRS", + [SPECTRE_V2_EIBRS] =3D "Mitigation: Enhanced IBRS", + [SPECTRE_V2_EIBRS_LFENCE] =3D "Mitigation: Enhanced IBRS + LFENCE", + [SPECTRE_V2_EIBRS_RETPOLINE] =3D "Mitigation: Enhanced IBRS + Retpolines= ", }; =20 static const struct { @@ -803,6 +815,9 @@ static const struct { { "retpoline,amd", SPECTRE_V2_CMD_RETPOLINE_LFENCE, false }, { "retpoline,lfence", SPECTRE_V2_CMD_RETPOLINE_LFENCE, false }, { "retpoline,generic", SPECTRE_V2_CMD_RETPOLINE_GENERIC, false }, + { "eibrs", SPECTRE_V2_CMD_EIBRS, false }, + { "eibrs,lfence", SPECTRE_V2_CMD_EIBRS_LFENCE, false }, + { "eibrs,retpoline", SPECTRE_V2_CMD_EIBRS_RETPOLINE, false }, { "auto", SPECTRE_V2_CMD_AUTO, false }, }; =20 @@ -840,15 +855,29 @@ static enum spectre_v2_mitigation_cmd __ =20 if ((cmd =3D=3D SPECTRE_V2_CMD_RETPOLINE || cmd =3D=3D SPECTRE_V2_CMD_RETPOLINE_LFENCE || - cmd =3D=3D SPECTRE_V2_CMD_RETPOLINE_GENERIC) && + cmd =3D=3D SPECTRE_V2_CMD_RETPOLINE_GENERIC || + cmd =3D=3D SPECTRE_V2_CMD_EIBRS_LFENCE || + cmd =3D=3D SPECTRE_V2_CMD_EIBRS_RETPOLINE) && !IS_ENABLED(CONFIG_RETPOLINE)) { - pr_err("%s selected but not compiled in. Switching to AUTO select\n", mi= tigation_options[i].option); + pr_err("%s selected but not compiled in. Switching to AUTO select\n", + mitigation_options[i].option); + return SPECTRE_V2_CMD_AUTO; + } + + if ((cmd =3D=3D SPECTRE_V2_CMD_EIBRS || + cmd =3D=3D SPECTRE_V2_CMD_EIBRS_LFENCE || + cmd =3D=3D SPECTRE_V2_CMD_EIBRS_RETPOLINE) && + !boot_cpu_has(X86_FEATURE_IBRS_ENHANCED)) { + pr_err("%s selected but CPU doesn't have eIBRS. Switching to AUTO select= \n", + mitigation_options[i].option); return SPECTRE_V2_CMD_AUTO; } =20 - if ((cmd =3D=3D SPECTRE_V2_CMD_RETPOLINE_LFENCE) && + if ((cmd =3D=3D SPECTRE_V2_CMD_RETPOLINE_LFENCE || + cmd =3D=3D SPECTRE_V2_CMD_EIBRS_LFENCE) && !boot_cpu_has(X86_FEATURE_LFENCE_RDTSC)) { - pr_err("%s selected, but CPU doesn't have a serializing LFENCE. Switchin= g to AUTO select\n", mitigation_options[i].option); + pr_err("%s selected, but CPU doesn't have a serializing LFENCE. Switchin= g to AUTO select\n", + mitigation_options[i].option); return SPECTRE_V2_CMD_AUTO; } =20 @@ -857,6 +886,25 @@ static enum spectre_v2_mitigation_cmd __ return cmd; } =20 +static enum spectre_v2_mitigation __init spectre_v2_select_retpoline(void) +{ + if (!IS_ENABLED(CONFIG_RETPOLINE)) { + pr_err("Kernel not compiled with retpoline; no mitigation available!"); + return SPECTRE_V2_NONE; + } + + if (boot_cpu_data.x86_vendor =3D=3D X86_VENDOR_AMD || + boot_cpu_data.x86_vendor =3D=3D X86_VENDOR_HYGON) { + if (!boot_cpu_has(X86_FEATURE_LFENCE_RDTSC)) { + pr_err("LFENCE not serializing, switching to generic retpoline\n"); + return SPECTRE_V2_RETPOLINE; + } + return SPECTRE_V2_LFENCE; + } + + return SPECTRE_V2_RETPOLINE; +} + static void __init spectre_v2_select_mitigation(void) { enum spectre_v2_mitigation_cmd cmd =3D spectre_v2_parse_cmdline(); @@ -877,49 +925,60 @@ static void __init spectre_v2_select_mit case SPECTRE_V2_CMD_FORCE: case SPECTRE_V2_CMD_AUTO: if (boot_cpu_has(X86_FEATURE_IBRS_ENHANCED)) { - mode =3D SPECTRE_V2_IBRS_ENHANCED; - /* Force it so VMEXIT will restore correctly */ - x86_spec_ctrl_base |=3D SPEC_CTRL_IBRS; - wrmsrl(MSR_IA32_SPEC_CTRL, x86_spec_ctrl_base); - goto specv2_set_mode; + mode =3D SPECTRE_V2_EIBRS; + break; } - if (IS_ENABLED(CONFIG_RETPOLINE)) - goto retpoline_auto; + + mode =3D spectre_v2_select_retpoline(); break; + case SPECTRE_V2_CMD_RETPOLINE_LFENCE: - if (IS_ENABLED(CONFIG_RETPOLINE)) - goto retpoline_lfence; + mode =3D SPECTRE_V2_LFENCE; break; + case SPECTRE_V2_CMD_RETPOLINE_GENERIC: - if (IS_ENABLED(CONFIG_RETPOLINE)) - goto retpoline_generic; + mode =3D SPECTRE_V2_RETPOLINE; break; + case SPECTRE_V2_CMD_RETPOLINE: - if (IS_ENABLED(CONFIG_RETPOLINE)) - goto retpoline_auto; + mode =3D spectre_v2_select_retpoline(); + break; + + case SPECTRE_V2_CMD_EIBRS: + mode =3D SPECTRE_V2_EIBRS; + break; + + case SPECTRE_V2_CMD_EIBRS_LFENCE: + mode =3D SPECTRE_V2_EIBRS_LFENCE; + break; + + case SPECTRE_V2_CMD_EIBRS_RETPOLINE: + mode =3D SPECTRE_V2_EIBRS_RETPOLINE; break; } - pr_err("Spectre mitigation: kernel not compiled with retpoline; no mitiga= tion available!"); - return; =20 -retpoline_auto: - if (boot_cpu_data.x86_vendor =3D=3D X86_VENDOR_AMD || - boot_cpu_data.x86_vendor =3D=3D X86_VENDOR_HYGON) { - retpoline_lfence: - if (!boot_cpu_has(X86_FEATURE_LFENCE_RDTSC)) { - pr_err("Spectre mitigation: LFENCE not serializing, switching to generi= c retpoline\n"); - goto retpoline_generic; - } - mode =3D SPECTRE_V2_LFENCE; + if (spectre_v2_in_eibrs_mode(mode)) { + /* Force it so VMEXIT will restore correctly */ + x86_spec_ctrl_base |=3D SPEC_CTRL_IBRS; + wrmsrl(MSR_IA32_SPEC_CTRL, x86_spec_ctrl_base); + } + + switch (mode) { + case SPECTRE_V2_NONE: + case SPECTRE_V2_EIBRS: + break; + + case SPECTRE_V2_LFENCE: + case SPECTRE_V2_EIBRS_LFENCE: setup_force_cpu_cap(X86_FEATURE_RETPOLINE_LFENCE); + fallthrough; + + case SPECTRE_V2_RETPOLINE: + case SPECTRE_V2_EIBRS_RETPOLINE: setup_force_cpu_cap(X86_FEATURE_RETPOLINE); - } else { - retpoline_generic: - mode =3D SPECTRE_V2_RETPOLINE; - setup_force_cpu_cap(X86_FEATURE_RETPOLINE); + break; } =20 -specv2_set_mode: spectre_v2_enabled =3D mode; pr_info("%s\n", spectre_v2_strings[mode]); =20 @@ -945,7 +1004,7 @@ specv2_set_mode: * the CPU supports Enhanced IBRS, kernel might un-intentionally not * enable IBRS around firmware calls. */ - if (boot_cpu_has(X86_FEATURE_IBRS) && mode !=3D SPECTRE_V2_IBRS_ENHANCED)= { + if (boot_cpu_has(X86_FEATURE_IBRS) && !spectre_v2_in_eibrs_mode(mode)) { setup_force_cpu_cap(X86_FEATURE_USE_IBRS_FW); pr_info("Enabling Restricted Speculation for firmware calls\n"); } @@ -1621,7 +1680,7 @@ static ssize_t tsx_async_abort_show_stat =20 static char *stibp_state(void) { - if (spectre_v2_enabled =3D=3D SPECTRE_V2_IBRS_ENHANCED) + if (spectre_v2_in_eibrs_mode(spectre_v2_enabled)) return ""; =20 switch (spectre_v2_user_stibp) { From nobody Tue Jun 23 05:03:54 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AD9E2C4321E for ; Thu, 10 Mar 2022 14:33:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S245238AbiCJOei (ORCPT ); Thu, 10 Mar 2022 09:34:38 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49514 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S244475AbiCJO25 (ORCPT ); Thu, 10 Mar 2022 09:28:57 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7346BD21C6; Thu, 10 Mar 2022 06:24:02 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 9D27EB81E9E; Thu, 10 Mar 2022 14:24:00 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id E3E5EC340F3; Thu, 10 Mar 2022 14:23:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1646922239; bh=ek27sDL8nRpw3Td5eid2CXK/8pNndPVGVVLB+34CqUo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Nj7ohTmin9Uh+t1V8vCOd3KCln93YzPj9AOKGxw1eYNLLm+uqlCqLel/rur7CWgDu pbzdoeuyjDYvhHBOygGYRaeSTivr5Rbz3Kdo4L6+st73JBSYkgVZ3QzaQJWnu6K0Rn fkNMlGS/QUSO4/0eDIO0dSV3vuGHY1oO2PbJA1wY= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, "Peter Zijlstra (Intel)" , Borislav Petkov , Thomas Gleixner , Frank van der Linden Subject: [PATCH 5.10 04/58] Documentation/hw-vuln: Update spectre doc Date: Thu, 10 Mar 2022 15:18:24 +0100 Message-Id: <20220310140812.998038010@linuxfoundation.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220310140812.869208747@linuxfoundation.org> References: <20220310140812.869208747@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Peter Zijlstra commit 5ad3eb1132453b9795ce5fd4572b1c18b292cca9 upstream. Update the doc with the new fun. [ bp: Massage commit message. ] Signed-off-by: Peter Zijlstra (Intel) Signed-off-by: Borislav Petkov Reviewed-by: Thomas Gleixner [fllinden@amazon.com: backported to 5.10] Signed-off-by: Frank van der Linden Signed-off-by: Greg Kroah-Hartman Tested-by: Bagas Sanjaya Tested-by: Florian Fainelli Tested-by: Fox Chen Tested-by: Guenter Roeck Tested-by: Linux Kernel Functional Testing Tested-by: Pavel Machek (CIP) = =20 Tested-by: Shuah Khan Tested-by: Sudip Mukherjee --- Documentation/admin-guide/hw-vuln/spectre.rst | 42 ++++++++++++++++---= ----- Documentation/admin-guide/kernel-parameters.txt | 8 +++- 2 files changed, 35 insertions(+), 15 deletions(-) --- a/Documentation/admin-guide/hw-vuln/spectre.rst +++ b/Documentation/admin-guide/hw-vuln/spectre.rst @@ -131,6 +131,19 @@ steer its indirect branch speculations t speculative execution's side effects left in level 1 cache to infer the victim's data. =20 +Yet another variant 2 attack vector is for the attacker to poison the +Branch History Buffer (BHB) to speculatively steer an indirect branch +to a specific Branch Target Buffer (BTB) entry, even if the entry isn't +associated with the source address of the indirect branch. Specifically, +the BHB might be shared across privilege levels even in the presence of +Enhanced IBRS. + +Currently the only known real-world BHB attack vector is via +unprivileged eBPF. Therefore, it's highly recommended to not enable +unprivileged eBPF, especially when eIBRS is used (without retpolines). +For a full mitigation against BHB attacks, it's recommended to use +retpolines (or eIBRS combined with retpolines). + Attack scenarios ---------------- =20 @@ -364,13 +377,15 @@ The possible values in this file are: =20 - Kernel status: =20 - =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D - 'Not affected' The processor is not vulnerable - 'Vulnerable' Vulnerable, no mitigation - 'Mitigation: Full generic retpoline' Software-focused mitigation - 'Mitigation: Full AMD retpoline' AMD-specific software mitigation - 'Mitigation: Enhanced IBRS' Hardware-focused mitigation - =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + 'Not affected' The processor is not vulnerable + 'Mitigation: None' Vulnerable, no mitigation + 'Mitigation: Retpolines' Use Retpoline thunks + 'Mitigation: LFENCE' Use LFENCE instructions + 'Mitigation: Enhanced IBRS' Hardware-focused mitigation + 'Mitigation: Enhanced IBRS + Retpolines' Hardware-focused + Retpolines + 'Mitigation: Enhanced IBRS + LFENCE' Hardware-focused + LFENCE + =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =20 - Firmware status: Show if Indirect Branch Restricted Speculation (IBRS)= is used to protect against Spectre variant 2 attacks when calling firmwar= e (x86 only). @@ -584,12 +599,13 @@ kernel command line. =20 Specific mitigations can also be selected manually: =20 - retpoline - replace indirect branches - retpoline,generic - google's original retpoline - retpoline,amd - AMD-specific minimal thunk + retpoline auto pick between generic,lfence + retpoline,generic Retpolines + retpoline,lfence LFENCE; indirect branch + retpoline,amd alias for retpoline,lfence + eibrs enhanced IBRS + eibrs,retpoline enhanced IBRS + Retpolines + eibrs,lfence enhanced IBRS + LFENCE =20 Not specifying this option is equivalent to spectre_v2=3Dauto. --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -4957,8 +4957,12 @@ Specific mitigations can also be selected manually: =20 retpoline - replace indirect branches - retpoline,generic - google's original retpoline - retpoline,amd - AMD-specific minimal thunk + retpoline,generic - Retpolines + retpoline,lfence - LFENCE; indirect branch + retpoline,amd - alias for retpoline,lfence + eibrs - enhanced IBRS + eibrs,retpoline - enhanced IBRS + Retpolines + eibrs,lfence - enhanced IBRS + LFENCE =20 Not specifying this option is equivalent to spectre_v2=3Dauto. From nobody Tue Jun 23 05:03:54 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 66078C3A59E for ; Thu, 10 Mar 2022 14:33:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243449AbiCJOeq (ORCPT ); Thu, 10 Mar 2022 09:34:46 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49606 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S244511AbiCJO3O (ORCPT ); Thu, 10 Mar 2022 09:29:14 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DD41DCCC72; Thu, 10 Mar 2022 06:24:05 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id D49D8B8266E; Thu, 10 Mar 2022 14:24:03 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1D9C9C340E8; Thu, 10 Mar 2022 14:24:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1646922242; bh=SDpUFU1TJQ9FPEXZ/2tbgw+vF0p/dqGxte1p0E0q2Ow=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=aFoXuWpOIHgM0zrGWrHetoWumN95cYILeXy9lu6JUzicaoCwvrQ9Q+FMaKOJlKJuX PL3Q8JpZcnMaCkSw1lNo5M4IZHZ0g42RYc+2RlBjfVHPsVNTldPkQKbg5GPYgItie6 b9hdEt2stj68uxkPGb2mGeCG8k/sgMn1lrFUxT9Q= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Josh Poimboeuf , Borislav Petkov , Thomas Gleixner , Frank van der Linden Subject: [PATCH 5.10 05/58] x86/speculation: Include unprivileged eBPF status in Spectre v2 mitigation reporting Date: Thu, 10 Mar 2022 15:18:25 +0100 Message-Id: <20220310140813.026694138@linuxfoundation.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220310140812.869208747@linuxfoundation.org> References: <20220310140812.869208747@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Josh Poimboeuf commit 44a3918c8245ab10c6c9719dd12e7a8d291980d8 upstream. With unprivileged eBPF enabled, eIBRS (without retpoline) is vulnerable to Spectre v2 BHB-based attacks. When both are enabled, print a warning message and report it in the 'spectre_v2' sysfs vulnerabilities file. Signed-off-by: Josh Poimboeuf Signed-off-by: Borislav Petkov Reviewed-by: Thomas Gleixner [fllinden@amazon.com: backported to 5.10] Signed-off-by: Frank van der Linden Signed-off-by: Greg Kroah-Hartman Tested-by: Bagas Sanjaya Tested-by: Florian Fainelli Tested-by: Fox Chen Tested-by: Guenter Roeck Tested-by: Linux Kernel Functional Testing Tested-by: Pavel Machek (CIP) = =20 Tested-by: Shuah Khan Tested-by: Sudip Mukherjee --- arch/x86/kernel/cpu/bugs.c | 35 +++++++++++++++++++++++++++++------ include/linux/bpf.h | 12 ++++++++++++ kernel/sysctl.c | 7 +++++++ 3 files changed, 48 insertions(+), 6 deletions(-) --- a/arch/x86/kernel/cpu/bugs.c +++ b/arch/x86/kernel/cpu/bugs.c @@ -16,6 +16,7 @@ #include #include #include +#include =20 #include #include @@ -613,6 +614,16 @@ static inline const char *spectre_v2_mod static inline const char *spectre_v2_module_string(void) { return ""; } #endif =20 +#define SPECTRE_V2_EIBRS_EBPF_MSG "WARNING: Unprivileged eBPF is enabled w= ith eIBRS on, data leaks possible via Spectre v2 BHB attacks!\n" + +#ifdef CONFIG_BPF_SYSCALL +void unpriv_ebpf_notify(int new_state) +{ + if (spectre_v2_enabled =3D=3D SPECTRE_V2_EIBRS && !new_state) + pr_err(SPECTRE_V2_EIBRS_EBPF_MSG); +} +#endif + static inline bool match_option(const char *arg, int arglen, const char *o= pt) { int len =3D strlen(opt); @@ -957,6 +968,9 @@ static void __init spectre_v2_select_mit break; } =20 + if (mode =3D=3D SPECTRE_V2_EIBRS && unprivileged_ebpf_enabled()) + pr_err(SPECTRE_V2_EIBRS_EBPF_MSG); + if (spectre_v2_in_eibrs_mode(mode)) { /* Force it so VMEXIT will restore correctly */ x86_spec_ctrl_base |=3D SPEC_CTRL_IBRS; @@ -1710,6 +1724,20 @@ static char *ibpb_state(void) return ""; } =20 +static ssize_t spectre_v2_show_state(char *buf) +{ + if (spectre_v2_enabled =3D=3D SPECTRE_V2_EIBRS && unprivileged_ebpf_enabl= ed()) + return sprintf(buf, "Vulnerable: Unprivileged eBPF enabled\n"); + + return sprintf(buf, "%s%s%s%s%s%s\n", + spectre_v2_strings[spectre_v2_enabled], + ibpb_state(), + boot_cpu_has(X86_FEATURE_USE_IBRS_FW) ? ", IBRS_FW" : "", + stibp_state(), + boot_cpu_has(X86_FEATURE_RSB_CTXSW) ? ", RSB filling" : "", + spectre_v2_module_string()); +} + static ssize_t srbds_show_state(char *buf) { return sprintf(buf, "%s\n", srbds_strings[srbds_mitigation]); @@ -1735,12 +1763,7 @@ static ssize_t cpu_show_common(struct de return sprintf(buf, "%s\n", spectre_v1_strings[spectre_v1_mitigation]); =20 case X86_BUG_SPECTRE_V2: - return sprintf(buf, "%s%s%s%s%s%s\n", spectre_v2_strings[spectre_v2_enab= led], - ibpb_state(), - boot_cpu_has(X86_FEATURE_USE_IBRS_FW) ? ", IBRS_FW" : "", - stibp_state(), - boot_cpu_has(X86_FEATURE_RSB_CTXSW) ? ", RSB filling" : "", - spectre_v2_module_string()); + return spectre_v2_show_state(buf); =20 case X86_BUG_SPEC_STORE_BYPASS: return sprintf(buf, "%s\n", ssb_strings[ssb_mode]); --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -1485,6 +1485,12 @@ struct bpf_prog *bpf_prog_by_id(u32 id); struct bpf_link *bpf_link_by_id(u32 id); =20 const struct bpf_func_proto *bpf_base_func_proto(enum bpf_func_id func_id); + +static inline bool unprivileged_ebpf_enabled(void) +{ + return !sysctl_unprivileged_bpf_disabled; +} + #else /* !CONFIG_BPF_SYSCALL */ static inline struct bpf_prog *bpf_prog_get(u32 ufd) { @@ -1679,6 +1685,12 @@ bpf_base_func_proto(enum bpf_func_id fun { return NULL; } + +static inline bool unprivileged_ebpf_enabled(void) +{ + return false; +} + #endif /* CONFIG_BPF_SYSCALL */ =20 static inline struct bpf_prog *bpf_prog_get_type(u32 ufd, --- a/kernel/sysctl.c +++ b/kernel/sysctl.c @@ -234,6 +234,10 @@ static int bpf_stats_handler(struct ctl_ return ret; } =20 +void __weak unpriv_ebpf_notify(int new_state) +{ +} + static int bpf_unpriv_handler(struct ctl_table *table, int write, void *buffer, size_t *lenp, loff_t *ppos) { @@ -251,6 +255,9 @@ static int bpf_unpriv_handler(struct ctl return -EPERM; *(int *)table->data =3D unpriv_enable; } + + unpriv_ebpf_notify(unpriv_enable); + return ret; } #endif /* CONFIG_BPF_SYSCALL && CONFIG_SYSCTL */ From nobody Tue Jun 23 05:03:54 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5B82FC41535 for ; Thu, 10 Mar 2022 14:42:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344658AbiCJOlG (ORCPT ); Thu, 10 Mar 2022 09:41:06 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49730 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S244528AbiCJO3P (ORCPT ); Thu, 10 Mar 2022 09:29:15 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 36FEAD31F4; Thu, 10 Mar 2022 06:24:08 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 60CDC61D3B; Thu, 10 Mar 2022 14:24:06 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 638E5C36AE7; Thu, 10 Mar 2022 14:24:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1646922245; bh=CtdSGUhCx/XkfKoxOYRUMs1Huk9CIG095Jidwc20icI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=b1AwTeB2L/n4ARYu05Rc3rIqqNL4UvXVeBO5V2D9b6bchfx9jVn5w+wyCbAs9lZwZ UUb24TN3C/c0SBhV5DL9X2j/VnsXKxVLnsynP1d2eHLP5hqVCPs6g/0dKEjdJVqfMM uR98h3RCx+wSETxPKOH1hq4nmi1qc8y+hi9osRvk= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Kim Phillips , Borislav Petkov Subject: [PATCH 5.10 06/58] x86/speculation: Use generic retpoline by default on AMD Date: Thu, 10 Mar 2022 15:18:26 +0100 Message-Id: <20220310140813.055661892@linuxfoundation.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220310140812.869208747@linuxfoundation.org> References: <20220310140812.869208747@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Kim Phillips commit 244d00b5dd4755f8df892c86cab35fb2cfd4f14b upstream. AMD retpoline may be susceptible to speculation. The speculation execution window for an incorrect indirect branch prediction using LFENCE/JMP sequence may potentially be large enough to allow exploitation using Spectre V2. By default, don't use retpoline,lfence on AMD. Instead, use the generic retpoline. Signed-off-by: Kim Phillips Signed-off-by: Borislav Petkov Signed-off-by: Greg Kroah-Hartman Tested-by: Bagas Sanjaya Tested-by: Florian Fainelli Tested-by: Fox Chen Tested-by: Guenter Roeck Tested-by: Linux Kernel Functional Testing Tested-by: Pavel Machek (CIP) = =20 Tested-by: Shuah Khan Tested-by: Sudip Mukherjee --- arch/x86/kernel/cpu/bugs.c | 9 --------- 1 file changed, 9 deletions(-) --- a/arch/x86/kernel/cpu/bugs.c +++ b/arch/x86/kernel/cpu/bugs.c @@ -904,15 +904,6 @@ static enum spectre_v2_mitigation __init return SPECTRE_V2_NONE; } =20 - if (boot_cpu_data.x86_vendor =3D=3D X86_VENDOR_AMD || - boot_cpu_data.x86_vendor =3D=3D X86_VENDOR_HYGON) { - if (!boot_cpu_has(X86_FEATURE_LFENCE_RDTSC)) { - pr_err("LFENCE not serializing, switching to generic retpoline\n"); - return SPECTRE_V2_RETPOLINE; - } - return SPECTRE_V2_LFENCE; - } - return SPECTRE_V2_RETPOLINE; } From nobody Tue Jun 23 05:03:54 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 05F5DC433EF for ; Thu, 10 Mar 2022 14:34:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240477AbiCJOfT (ORCPT ); Thu, 10 Mar 2022 09:35:19 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51942 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S244562AbiCJO3R (ORCPT ); Thu, 10 Mar 2022 09:29:17 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 29340D4C81; Thu, 10 Mar 2022 06:24:12 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 341E2B82681; Thu, 10 Mar 2022 14:24:10 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 93302C340E8; Thu, 10 Mar 2022 14:24:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1646922249; bh=wBLb1JgeLGCmIlJRSZ+nBqsfFArPm7x3PghLmULm8JI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Rc1FedPavxYV4057rDk40Q0sf/FQSsRoNCJ3GEPVN63122mz8MKzykbYBqF0NZc+r J0Gxb74Jx9inFCKrBI/Ma7xdABkK1UhiwLLlx1B5S4x1RUG8rjLgrqsIxBs1Y+f+TM 7VNr7AYg1cN+VkQHfT93K9+Scj7JFQ8ef1ZEHFZ8= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Kim Phillips , Borislav Petkov Subject: [PATCH 5.10 07/58] x86/speculation: Update link to AMD speculation whitepaper Date: Thu, 10 Mar 2022 15:18:27 +0100 Message-Id: <20220310140813.083764120@linuxfoundation.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220310140812.869208747@linuxfoundation.org> References: <20220310140812.869208747@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Kim Phillips commit e9b6013a7ce31535b04b02ba99babefe8a8599fa upstream. Update the link to the "Software Techniques for Managing Speculation on AMD Processors" whitepaper. Signed-off-by: Kim Phillips Signed-off-by: Borislav Petkov Signed-off-by: Greg Kroah-Hartman Tested-by: Bagas Sanjaya Tested-by: Florian Fainelli Tested-by: Fox Chen Tested-by: Guenter Roeck Tested-by: Linux Kernel Functional Testing Tested-by: Pavel Machek (CIP) = =20 Tested-by: Shuah Khan Tested-by: Sudip Mukherjee --- Documentation/admin-guide/hw-vuln/spectre.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) --- a/Documentation/admin-guide/hw-vuln/spectre.rst +++ b/Documentation/admin-guide/hw-vuln/spectre.rst @@ -60,8 +60,8 @@ privileged data touched during the specu Spectre variant 1 attacks take advantage of speculative execution of conditional branches, while Spectre variant 2 attacks use speculative execution of indirect branches to leak privileged memory. -See :ref:`[1] ` :ref:`[5] ` :ref:`[7] ` -:ref:`[10] ` :ref:`[11] `. +See :ref:`[1] ` :ref:`[5] ` :ref:`[6] ` +:ref:`[7] ` :ref:`[10] ` :ref:`[11] `. =20 Spectre variant 1 (Bounds Check Bypass) --------------------------------------- @@ -746,7 +746,7 @@ AMD white papers: =20 .. _spec_ref6: =20 -[6] `Software techniques for managing speculation on AMD processors `_. +[6] `Software techniques for managing speculation on AMD processors `_. =20 ARM white papers: From nobody Tue Jun 23 05:03:54 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0EA39C433FE for ; Thu, 10 Mar 2022 14:34:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235412AbiCJOfh (ORCPT ); Thu, 10 Mar 2022 09:35:37 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50328 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S244755AbiCJO32 (ORCPT ); Thu, 10 Mar 2022 09:29:28 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 14E25DE2E6; Thu, 10 Mar 2022 06:24:38 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 2912F61D78; Thu, 10 Mar 2022 14:24:13 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2B150C36AE5; Thu, 10 Mar 2022 14:24:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1646922252; bh=jr+6+1MnJQNcfljlgg0AAAnKF+/2D+Wgaosbrook0EU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=VUQ4UbV+o+YSisb1l5MBI8zvA5mONzOJgitXbIYLdMytTnA8PSLPAgTb6CAAjO2uh nLJ2yaM+gXK96yEs/ZHD1t+aiv5UeB6F0Z+O615+qzA88Pru/Vyd61H9RnSGneXAI0 QErfzsjfVSUEjBvd2KsrH+Rqa0i984Pe+7Hbc03k= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Josh Poimboeuf , Borislav Petkov Subject: [PATCH 5.10 08/58] x86/speculation: Warn about Spectre v2 LFENCE mitigation Date: Thu, 10 Mar 2022 15:18:28 +0100 Message-Id: <20220310140813.112529706@linuxfoundation.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220310140812.869208747@linuxfoundation.org> References: <20220310140812.869208747@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Josh Poimboeuf commit eafd987d4a82c7bb5aa12f0e3b4f8f3dea93e678 upstream. With: f8a66d608a3e ("x86,bugs: Unconditionally allow spectre_v2=3Dretpoline,amd= ") it became possible to enable the LFENCE "retpoline" on Intel. However, Intel doesn't recommend it, as it has some weaknesses compared to retpoline. Now AMD doesn't recommend it either. It can still be left available as a cmdline option. It's faster than retpoline but is weaker in certain scenarios -- particularly SMT, but even non-SMT may be vulnerable in some cases. So just unconditionally warn if the user requests it on the cmdline. [ bp: Massage commit message. ] Signed-off-by: Josh Poimboeuf Signed-off-by: Borislav Petkov Signed-off-by: Greg Kroah-Hartman Tested-by: Bagas Sanjaya Tested-by: Florian Fainelli Tested-by: Fox Chen Tested-by: Guenter Roeck Tested-by: Linux Kernel Functional Testing Tested-by: Pavel Machek (CIP) = =20 Tested-by: Shuah Khan Tested-by: Sudip Mukherjee --- arch/x86/kernel/cpu/bugs.c | 5 +++++ 1 file changed, 5 insertions(+) --- a/arch/x86/kernel/cpu/bugs.c +++ b/arch/x86/kernel/cpu/bugs.c @@ -614,6 +614,7 @@ static inline const char *spectre_v2_mod static inline const char *spectre_v2_module_string(void) { return ""; } #endif =20 +#define SPECTRE_V2_LFENCE_MSG "WARNING: LFENCE mitigation is not recommend= ed for this CPU, data leaks possible!\n" #define SPECTRE_V2_EIBRS_EBPF_MSG "WARNING: Unprivileged eBPF is enabled w= ith eIBRS on, data leaks possible via Spectre v2 BHB attacks!\n" =20 #ifdef CONFIG_BPF_SYSCALL @@ -935,6 +936,7 @@ static void __init spectre_v2_select_mit break; =20 case SPECTRE_V2_CMD_RETPOLINE_LFENCE: + pr_err(SPECTRE_V2_LFENCE_MSG); mode =3D SPECTRE_V2_LFENCE; break; =20 @@ -1717,6 +1719,9 @@ static char *ibpb_state(void) =20 static ssize_t spectre_v2_show_state(char *buf) { + if (spectre_v2_enabled =3D=3D SPECTRE_V2_LFENCE) + return sprintf(buf, "Vulnerable: LFENCE\n"); + if (spectre_v2_enabled =3D=3D SPECTRE_V2_EIBRS && unprivileged_ebpf_enabl= ed()) return sprintf(buf, "Vulnerable: Unprivileged eBPF enabled\n"); From nobody Tue Jun 23 05:03:54 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C7A56C433F5 for ; Thu, 10 Mar 2022 14:34:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234483AbiCJOf0 (ORCPT ); Thu, 10 Mar 2022 09:35:26 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50046 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S244628AbiCJO3V (ORCPT ); Thu, 10 Mar 2022 09:29:21 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A788DD5F76; Thu, 10 Mar 2022 06:24:20 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 20914B825A7; Thu, 10 Mar 2022 14:24:17 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 53240C340EB; Thu, 10 Mar 2022 14:24:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1646922255; bh=PrzkUSJfjRah5bDqvxlns5Yo150NQElLN2igS09Mj4k=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=KK2IXeJu1cMawYdiePB5VjBRwha6C8hDUKlvZf0SaLfQgUaXm14DKsqlMfWgxo5Nh UHweMa0I9uE3AKVALBrT4e7OG6YHyt1cswDUXZpYlZBuuxqsgBjQ3NswOlfYJ3qqeJ NAQ9pKhiKoWAPt6evVilDixQgITh+MNqj2ARbv94= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Alyssa Milburn , Josh Poimboeuf , Borislav Petkov Subject: [PATCH 5.10 09/58] x86/speculation: Warn about eIBRS + LFENCE + Unprivileged eBPF + SMT Date: Thu, 10 Mar 2022 15:18:29 +0100 Message-Id: <20220310140813.140889219@linuxfoundation.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220310140812.869208747@linuxfoundation.org> References: <20220310140812.869208747@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Josh Poimboeuf commit 0de05d056afdb00eca8c7bbb0c79a3438daf700c upstream. The commit 44a3918c8245 ("x86/speculation: Include unprivileged eBPF status in Spec= tre v2 mitigation reporting") added a warning for the "eIBRS + unprivileged eBPF" combination, which has been shown to be vulnerable against Spectre v2 BHB-based attacks. However, there's no warning about the "eIBRS + LFENCE retpoline + unprivileged eBPF" combo. The LFENCE adds more protection by shortening the speculation window after a mispredicted branch. That makes an attack significantly more difficult, even with unprivileged eBPF. So at least for now the logic doesn't warn about that combination. But if you then add SMT into the mix, the SMT attack angle weakens the effectiveness of the LFENCE considerably. So extend the "eIBRS + unprivileged eBPF" warning to also include the "eIBRS + LFENCE + unprivileged eBPF + SMT" case. [ bp: Massage commit message. ] Suggested-by: Alyssa Milburn Signed-off-by: Josh Poimboeuf Signed-off-by: Borislav Petkov Signed-off-by: Greg Kroah-Hartman Tested-by: Bagas Sanjaya Tested-by: Florian Fainelli Tested-by: Fox Chen Tested-by: Guenter Roeck Tested-by: Linux Kernel Functional Testing Tested-by: Pavel Machek (CIP) = =20 Tested-by: Shuah Khan Tested-by: Sudip Mukherjee --- arch/x86/kernel/cpu/bugs.c | 27 +++++++++++++++++++++++++-- 1 file changed, 25 insertions(+), 2 deletions(-) --- a/arch/x86/kernel/cpu/bugs.c +++ b/arch/x86/kernel/cpu/bugs.c @@ -616,12 +616,27 @@ static inline const char *spectre_v2_mod =20 #define SPECTRE_V2_LFENCE_MSG "WARNING: LFENCE mitigation is not recommend= ed for this CPU, data leaks possible!\n" #define SPECTRE_V2_EIBRS_EBPF_MSG "WARNING: Unprivileged eBPF is enabled w= ith eIBRS on, data leaks possible via Spectre v2 BHB attacks!\n" +#define SPECTRE_V2_EIBRS_LFENCE_EBPF_SMT_MSG "WARNING: Unprivileged eBPF i= s enabled with eIBRS+LFENCE mitigation and SMT, data leaks possible via Spe= ctre v2 BHB attacks!\n" =20 #ifdef CONFIG_BPF_SYSCALL void unpriv_ebpf_notify(int new_state) { - if (spectre_v2_enabled =3D=3D SPECTRE_V2_EIBRS && !new_state) + if (new_state) + return; + + /* Unprivileged eBPF is enabled */ + + switch (spectre_v2_enabled) { + case SPECTRE_V2_EIBRS: pr_err(SPECTRE_V2_EIBRS_EBPF_MSG); + break; + case SPECTRE_V2_EIBRS_LFENCE: + if (sched_smt_active()) + pr_err(SPECTRE_V2_EIBRS_LFENCE_EBPF_SMT_MSG); + break; + default: + break; + } } #endif =20 @@ -1081,6 +1096,10 @@ void cpu_bugs_smt_update(void) { mutex_lock(&spec_ctrl_mutex); =20 + if (sched_smt_active() && unprivileged_ebpf_enabled() && + spectre_v2_enabled =3D=3D SPECTRE_V2_EIBRS_LFENCE) + pr_warn_once(SPECTRE_V2_EIBRS_LFENCE_EBPF_SMT_MSG); + switch (spectre_v2_user_stibp) { case SPECTRE_V2_USER_NONE: break; @@ -1723,7 +1742,11 @@ static ssize_t spectre_v2_show_state(cha return sprintf(buf, "Vulnerable: LFENCE\n"); =20 if (spectre_v2_enabled =3D=3D SPECTRE_V2_EIBRS && unprivileged_ebpf_enabl= ed()) - return sprintf(buf, "Vulnerable: Unprivileged eBPF enabled\n"); + return sprintf(buf, "Vulnerable: eIBRS with unprivileged eBPF\n"); + + if (sched_smt_active() && unprivileged_ebpf_enabled() && + spectre_v2_enabled =3D=3D SPECTRE_V2_EIBRS_LFENCE) + return sprintf(buf, "Vulnerable: eIBRS+LFENCE with unprivileged eBPF and= SMT\n"); =20 return sprintf(buf, "%s%s%s%s%s%s\n", spectre_v2_strings[spectre_v2_enabled], From nobody Tue Jun 23 05:03:54 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4D9ABC4167B for ; Thu, 10 Mar 2022 14:42:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344612AbiCJOlE (ORCPT ); Thu, 10 Mar 2022 09:41:04 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49432 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S244466AbiCJO25 (ORCPT ); Thu, 10 Mar 2022 09:28:57 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B8CE9D21C2; Thu, 10 Mar 2022 06:24:01 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 51F7561D33; Thu, 10 Mar 2022 14:23:44 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 42D9FC340F4; Thu, 10 Mar 2022 14:23:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1646922223; bh=Vq6Ncc2+RgGiJQUsLedhR21yXbke25YcWmkNbJNb698=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=H3a7PHZS3v2hDGlK+KeUkKA9tZT52C48RKSuQn3MbdypuwJN1tIRxKFXmBY7zmnj1 POek9geywpl0oQLNcE84xwtTqtdJUzsj32mcAP2AOw86kMLI492l/aSij9axpFm+JY LkmTuJunGcJM4SnjeLOXcvZk5TOQlw30fFTdJJwI= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Catalin Marinas , "Russell King (Oracle)" Subject: [PATCH 5.10 10/58] ARM: report Spectre v2 status through sysfs Date: Thu, 10 Mar 2022 15:18:30 +0100 Message-Id: <20220310140813.169295282@linuxfoundation.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220310140812.869208747@linuxfoundation.org> References: <20220310140812.869208747@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: "Russell King (Oracle)" commit 9dd78194a3722fa6712192cdd4f7032d45112a9a upstream. As per other architectures, add support for reporting the Spectre vulnerability status via sysfs CPU. Acked-by: Catalin Marinas Signed-off-by: Russell King (Oracle) Signed-off-by: Greg Kroah-Hartman Tested-by: Bagas Sanjaya Tested-by: Florian Fainelli Tested-by: Fox Chen Tested-by: Guenter Roeck Tested-by: Linux Kernel Functional Testing Tested-by: Pavel Machek (CIP) = =20 Tested-by: Shuah Khan Tested-by: Sudip Mukherjee --- arch/arm/include/asm/spectre.h | 28 ++++++++ arch/arm/kernel/Makefile | 2=20 arch/arm/kernel/spectre.c | 54 +++++++++++++++ arch/arm/mm/Kconfig | 1=20 arch/arm/mm/proc-v7-bugs.c | 141 +++++++++++++++++++++++++++++-------= ----- 5 files changed, 187 insertions(+), 39 deletions(-) create mode 100644 arch/arm/include/asm/spectre.h create mode 100644 arch/arm/kernel/spectre.c --- /dev/null +++ b/arch/arm/include/asm/spectre.h @@ -0,0 +1,28 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ + +#ifndef __ASM_SPECTRE_H +#define __ASM_SPECTRE_H + +enum { + SPECTRE_UNAFFECTED, + SPECTRE_MITIGATED, + SPECTRE_VULNERABLE, +}; + +enum { + __SPECTRE_V2_METHOD_BPIALL, + __SPECTRE_V2_METHOD_ICIALLU, + __SPECTRE_V2_METHOD_SMC, + __SPECTRE_V2_METHOD_HVC, +}; + +enum { + SPECTRE_V2_METHOD_BPIALL =3D BIT(__SPECTRE_V2_METHOD_BPIALL), + SPECTRE_V2_METHOD_ICIALLU =3D BIT(__SPECTRE_V2_METHOD_ICIALLU), + SPECTRE_V2_METHOD_SMC =3D BIT(__SPECTRE_V2_METHOD_SMC), + SPECTRE_V2_METHOD_HVC =3D BIT(__SPECTRE_V2_METHOD_HVC), +}; + +void spectre_v2_update_state(unsigned int state, unsigned int methods); + +#endif --- a/arch/arm/kernel/Makefile +++ b/arch/arm/kernel/Makefile @@ -106,4 +106,6 @@ endif =20 obj-$(CONFIG_HAVE_ARM_SMCCC) +=3D smccc-call.o =20 +obj-$(CONFIG_GENERIC_CPU_VULNERABILITIES) +=3D spectre.o + extra-y :=3D $(head-y) vmlinux.lds --- /dev/null +++ b/arch/arm/kernel/spectre.c @@ -0,0 +1,54 @@ +// SPDX-License-Identifier: GPL-2.0-only +#include +#include + +#include + +ssize_t cpu_show_spectre_v1(struct device *dev, struct device_attribute *a= ttr, + char *buf) +{ + return sprintf(buf, "Mitigation: __user pointer sanitization\n"); +} + +static unsigned int spectre_v2_state; +static unsigned int spectre_v2_methods; + +void spectre_v2_update_state(unsigned int state, unsigned int method) +{ + if (state > spectre_v2_state) + spectre_v2_state =3D state; + spectre_v2_methods |=3D method; +} + +ssize_t cpu_show_spectre_v2(struct device *dev, struct device_attribute *a= ttr, + char *buf) +{ + const char *method; + + if (spectre_v2_state =3D=3D SPECTRE_UNAFFECTED) + return sprintf(buf, "%s\n", "Not affected"); + + if (spectre_v2_state !=3D SPECTRE_MITIGATED) + return sprintf(buf, "%s\n", "Vulnerable"); + + switch (spectre_v2_methods) { + case SPECTRE_V2_METHOD_BPIALL: + method =3D "Branch predictor hardening"; + break; + + case SPECTRE_V2_METHOD_ICIALLU: + method =3D "I-cache invalidation"; + break; + + case SPECTRE_V2_METHOD_SMC: + case SPECTRE_V2_METHOD_HVC: + method =3D "Firmware call"; + break; + + default: + method =3D "Multiple mitigations"; + break; + } + + return sprintf(buf, "Mitigation: %s\n", method); +} --- a/arch/arm/mm/Kconfig +++ b/arch/arm/mm/Kconfig @@ -833,6 +833,7 @@ config CPU_BPREDICT_DISABLE =20 config CPU_SPECTRE bool + select GENERIC_CPU_VULNERABILITIES =20 config HARDEN_BRANCH_PREDICTOR bool "Harden the branch predictor against aliasing attacks" if EXPERT --- a/arch/arm/mm/proc-v7-bugs.c +++ b/arch/arm/mm/proc-v7-bugs.c @@ -6,8 +6,35 @@ #include #include #include +#include #include =20 +#ifdef CONFIG_ARM_PSCI +static int __maybe_unused spectre_v2_get_cpu_fw_mitigation_state(void) +{ + struct arm_smccc_res res; + + arm_smccc_1_1_invoke(ARM_SMCCC_ARCH_FEATURES_FUNC_ID, + ARM_SMCCC_ARCH_WORKAROUND_1, &res); + + switch ((int)res.a0) { + case SMCCC_RET_SUCCESS: + return SPECTRE_MITIGATED; + + case SMCCC_ARCH_WORKAROUND_RET_UNAFFECTED: + return SPECTRE_UNAFFECTED; + + default: + return SPECTRE_VULNERABLE; + } +} +#else +static int __maybe_unused spectre_v2_get_cpu_fw_mitigation_state(void) +{ + return SPECTRE_VULNERABLE; +} +#endif + #ifdef CONFIG_HARDEN_BRANCH_PREDICTOR DEFINE_PER_CPU(harden_branch_predictor_fn_t, harden_branch_predictor_fn); =20 @@ -36,13 +63,60 @@ static void __maybe_unused call_hvc_arch arm_smccc_1_1_hvc(ARM_SMCCC_ARCH_WORKAROUND_1, NULL); } =20 -static void cpu_v7_spectre_init(void) +static unsigned int spectre_v2_install_workaround(unsigned int method) { const char *spectre_v2_method =3D NULL; int cpu =3D smp_processor_id(); =20 if (per_cpu(harden_branch_predictor_fn, cpu)) - return; + return SPECTRE_MITIGATED; + + switch (method) { + case SPECTRE_V2_METHOD_BPIALL: + per_cpu(harden_branch_predictor_fn, cpu) =3D + harden_branch_predictor_bpiall; + spectre_v2_method =3D "BPIALL"; + break; + + case SPECTRE_V2_METHOD_ICIALLU: + per_cpu(harden_branch_predictor_fn, cpu) =3D + harden_branch_predictor_iciallu; + spectre_v2_method =3D "ICIALLU"; + break; + + case SPECTRE_V2_METHOD_HVC: + per_cpu(harden_branch_predictor_fn, cpu) =3D + call_hvc_arch_workaround_1; + cpu_do_switch_mm =3D cpu_v7_hvc_switch_mm; + spectre_v2_method =3D "hypervisor"; + break; + + case SPECTRE_V2_METHOD_SMC: + per_cpu(harden_branch_predictor_fn, cpu) =3D + call_smc_arch_workaround_1; + cpu_do_switch_mm =3D cpu_v7_smc_switch_mm; + spectre_v2_method =3D "firmware"; + break; + } + + if (spectre_v2_method) + pr_info("CPU%u: Spectre v2: using %s workaround\n", + smp_processor_id(), spectre_v2_method); + + return SPECTRE_MITIGATED; +} +#else +static unsigned int spectre_v2_install_workaround(unsigned int method) +{ + pr_info("CPU%u: Spectre V2: workarounds disabled by configuration\n"); + + return SPECTRE_VULNERABLE; +} +#endif + +static void cpu_v7_spectre_v2_init(void) +{ + unsigned int state, method =3D 0; =20 switch (read_cpuid_part()) { case ARM_CPU_PART_CORTEX_A8: @@ -51,68 +125,57 @@ static void cpu_v7_spectre_init(void) case ARM_CPU_PART_CORTEX_A17: case ARM_CPU_PART_CORTEX_A73: case ARM_CPU_PART_CORTEX_A75: - per_cpu(harden_branch_predictor_fn, cpu) =3D - harden_branch_predictor_bpiall; - spectre_v2_method =3D "BPIALL"; + state =3D SPECTRE_MITIGATED; + method =3D SPECTRE_V2_METHOD_BPIALL; break; =20 case ARM_CPU_PART_CORTEX_A15: case ARM_CPU_PART_BRAHMA_B15: - per_cpu(harden_branch_predictor_fn, cpu) =3D - harden_branch_predictor_iciallu; - spectre_v2_method =3D "ICIALLU"; + state =3D SPECTRE_MITIGATED; + method =3D SPECTRE_V2_METHOD_ICIALLU; break; =20 -#ifdef CONFIG_ARM_PSCI case ARM_CPU_PART_BRAHMA_B53: /* Requires no workaround */ + state =3D SPECTRE_UNAFFECTED; break; + default: /* Other ARM CPUs require no workaround */ - if (read_cpuid_implementor() =3D=3D ARM_CPU_IMP_ARM) + if (read_cpuid_implementor() =3D=3D ARM_CPU_IMP_ARM) { + state =3D SPECTRE_UNAFFECTED; break; + } + fallthrough; - /* Cortex A57/A72 require firmware workaround */ - case ARM_CPU_PART_CORTEX_A57: - case ARM_CPU_PART_CORTEX_A72: { - struct arm_smccc_res res; =20 - arm_smccc_1_1_invoke(ARM_SMCCC_ARCH_FEATURES_FUNC_ID, - ARM_SMCCC_ARCH_WORKAROUND_1, &res); - if ((int)res.a0 !=3D 0) - return; + /* Cortex A57/A72 require firmware workaround */ + case ARM_CPU_PART_CORTEX_A57: + case ARM_CPU_PART_CORTEX_A72: + state =3D spectre_v2_get_cpu_fw_mitigation_state(); + if (state !=3D SPECTRE_MITIGATED) + break; =20 switch (arm_smccc_1_1_get_conduit()) { case SMCCC_CONDUIT_HVC: - per_cpu(harden_branch_predictor_fn, cpu) =3D - call_hvc_arch_workaround_1; - cpu_do_switch_mm =3D cpu_v7_hvc_switch_mm; - spectre_v2_method =3D "hypervisor"; + method =3D SPECTRE_V2_METHOD_HVC; break; =20 case SMCCC_CONDUIT_SMC: - per_cpu(harden_branch_predictor_fn, cpu) =3D - call_smc_arch_workaround_1; - cpu_do_switch_mm =3D cpu_v7_smc_switch_mm; - spectre_v2_method =3D "firmware"; + method =3D SPECTRE_V2_METHOD_SMC; break; =20 default: + state =3D SPECTRE_VULNERABLE; break; } } -#endif - } =20 - if (spectre_v2_method) - pr_info("CPU%u: Spectre v2: using %s workaround\n", - smp_processor_id(), spectre_v2_method); -} -#else -static void cpu_v7_spectre_init(void) -{ + if (state =3D=3D SPECTRE_MITIGATED) + state =3D spectre_v2_install_workaround(method); + + spectre_v2_update_state(state, method); } -#endif =20 static __maybe_unused bool cpu_v7_check_auxcr_set(bool *warned, u32 mask, const char *msg) @@ -142,16 +205,16 @@ static bool check_spectre_auxcr(bool *wa void cpu_v7_ca8_ibe(void) { if (check_spectre_auxcr(this_cpu_ptr(&spectre_warned), BIT(6))) - cpu_v7_spectre_init(); + cpu_v7_spectre_v2_init(); } =20 void cpu_v7_ca15_ibe(void) { if (check_spectre_auxcr(this_cpu_ptr(&spectre_warned), BIT(0))) - cpu_v7_spectre_init(); + cpu_v7_spectre_v2_init(); } =20 void cpu_v7_bugs_init(void) { - cpu_v7_spectre_init(); + cpu_v7_spectre_v2_init(); } From nobody Tue Jun 23 05:03:54 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 05ECFC3A589 for ; Thu, 10 Mar 2022 14:33:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236668AbiCJOez (ORCPT ); Thu, 10 Mar 2022 09:34:55 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52068 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S244476AbiCJO25 (ORCPT ); Thu, 10 Mar 2022 09:28:57 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1B80BD2248; Thu, 10 Mar 2022 06:24:02 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 6C02361CFB; Thu, 10 Mar 2022 14:23:47 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 6D540C340E8; Thu, 10 Mar 2022 14:23:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1646922226; bh=C+Kjyydpel09kWKlp+dh4V69PO5a/M+1eW27lj06JFU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=0GIoMIImdZnYvN/zIgcu7970ZxqYnnqYBiHWCTQEi6t3BCKDrb/5vVpatp0SmJjmg q3fr8GR+wlLsiKQVcPdKtAkwbh1/XSLzB7s0Nvk+64nMCXlPh3xC/lZjn6ekyUlj6Y +cpbHRjMiuEwfdzG+z9wi+l20C+MYWKgTa6WQ5Dg= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Catalin Marinas , "Russell King (Oracle)" Subject: [PATCH 5.10 11/58] ARM: early traps initialisation Date: Thu, 10 Mar 2022 15:18:31 +0100 Message-Id: <20220310140813.197349320@linuxfoundation.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220310140812.869208747@linuxfoundation.org> References: <20220310140812.869208747@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: "Russell King (Oracle)" commit 04e91b7324760a377a725e218b5ee783826d30f5 upstream. Provide a couple of helpers to copy the vectors and stubs, and also to flush the copied vectors and stubs. Acked-by: Catalin Marinas Signed-off-by: Russell King (Oracle) Signed-off-by: Greg Kroah-Hartman Tested-by: Bagas Sanjaya Tested-by: Florian Fainelli Tested-by: Fox Chen Tested-by: Guenter Roeck Tested-by: Linux Kernel Functional Testing Tested-by: Pavel Machek (CIP) = =20 Tested-by: Shuah Khan Tested-by: Sudip Mukherjee --- arch/arm/kernel/traps.c | 27 +++++++++++++++++++++------ 1 file changed, 21 insertions(+), 6 deletions(-) --- a/arch/arm/kernel/traps.c +++ b/arch/arm/kernel/traps.c @@ -806,10 +806,22 @@ static inline void __init kuser_init(voi } #endif =20 +#ifndef CONFIG_CPU_V7M +static void copy_from_lma(void *vma, void *lma_start, void *lma_end) +{ + memcpy(vma, lma_start, lma_end - lma_start); +} + +static void flush_vectors(void *vma, size_t offset, size_t size) +{ + unsigned long start =3D (unsigned long)vma + offset; + unsigned long end =3D start + size; + + flush_icache_range(start, end); +} + void __init early_trap_init(void *vectors_base) { -#ifndef CONFIG_CPU_V7M - unsigned long vectors =3D (unsigned long)vectors_base; extern char __stubs_start[], __stubs_end[]; extern char __vectors_start[], __vectors_end[]; unsigned i; @@ -830,17 +842,20 @@ void __init early_trap_init(void *vector * into the vector page, mapped at 0xffff0000, and ensure these * are visible to the instruction stream. */ - memcpy((void *)vectors, __vectors_start, __vectors_end - __vectors_start); - memcpy((void *)vectors + 0x1000, __stubs_start, __stubs_end - __stubs_sta= rt); + copy_from_lma(vectors_base, __vectors_start, __vectors_end); + copy_from_lma(vectors_base + 0x1000, __stubs_start, __stubs_end); =20 kuser_init(vectors_base); =20 - flush_icache_range(vectors, vectors + PAGE_SIZE * 2); + flush_vectors(vectors_base, 0, PAGE_SIZE * 2); +} #else /* ifndef CONFIG_CPU_V7M */ +void __init early_trap_init(void *vectors_base) +{ /* * on V7-M there is no need to copy the vector table to a dedicated * memory area. The address is configurable and so a table in the kernel * image can be used. */ -#endif } +#endif From nobody Tue Jun 23 05:03:54 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CE868C3527E for ; Thu, 10 Mar 2022 14:33:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S245174AbiCJOeR (ORCPT ); Thu, 10 Mar 2022 09:34:17 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49730 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S244412AbiCJO2y (ORCPT ); Thu, 10 Mar 2022 09:28:54 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3BFE3D10A5; Thu, 10 Mar 2022 06:23:57 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 6DF55B82672; Thu, 10 Mar 2022 14:23:51 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id B29DCC340E8; Thu, 10 Mar 2022 14:23:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1646922230; bh=iG4dGyVQTtgev5VOsQ8d1I4SvBWcNj19Mn9JiAy+Rwo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=EACZd+laC1OYAHRa7MnhKtmtwsF/b1jPGjhKfyB1J3XDHpBwEXjFIPWz2SfDLgN3K b0asMn5jWG9libXqx0H6er3eJ1WePPDA7gc0T2eLrm5Oa81PkOd9vS+sHJXYWWGHsd L2OGUsvY+p7DgMBuy0O++kKj20eCBqTqy/UEYa0I= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Catalin Marinas , "Russell King (Oracle)" Subject: [PATCH 5.10 12/58] ARM: use LOADADDR() to get load address of sections Date: Thu, 10 Mar 2022 15:18:32 +0100 Message-Id: <20220310140813.225153043@linuxfoundation.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220310140812.869208747@linuxfoundation.org> References: <20220310140812.869208747@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: "Russell King (Oracle)" commit 8d9d651ff2270a632e9dc497b142db31e8911315 upstream. Use the linker's LOADADDR() macro to get the load address of the sections, and provide a macro to set the start and end symbols. Acked-by: Catalin Marinas Signed-off-by: Russell King (Oracle) Signed-off-by: Greg Kroah-Hartman Tested-by: Bagas Sanjaya Tested-by: Florian Fainelli Tested-by: Fox Chen Tested-by: Guenter Roeck Tested-by: Linux Kernel Functional Testing Tested-by: Pavel Machek (CIP) = =20 Tested-by: Shuah Khan Tested-by: Sudip Mukherjee --- arch/arm/include/asm/vmlinux.lds.h | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-) --- a/arch/arm/include/asm/vmlinux.lds.h +++ b/arch/arm/include/asm/vmlinux.lds.h @@ -26,6 +26,11 @@ #define ARM_MMU_DISCARD(x) x #endif =20 +/* Set start/end symbol names to the LMA for the section */ +#define ARM_LMA(sym, section) \ + sym##_start =3D LOADADDR(section); \ + sym##_end =3D LOADADDR(section) + SIZEOF(section) + #define PROC_INFO \ . =3D ALIGN(4); \ __proc_info_begin =3D .; \ @@ -110,19 +115,19 @@ * only thing that matters is their relative offsets */ #define ARM_VECTORS \ - __vectors_start =3D .; \ + __vectors_lma =3D .; \ .vectors 0xffff0000 : AT(__vectors_start) { \ *(.vectors) \ } \ - . =3D __vectors_start + SIZEOF(.vectors); \ - __vectors_end =3D .; \ + ARM_LMA(__vectors, .vectors); \ + . =3D __vectors_lma + SIZEOF(.vectors); \ \ - __stubs_start =3D .; \ - .stubs ADDR(.vectors) + 0x1000 : AT(__stubs_start) { \ + __stubs_lma =3D .; \ + .stubs ADDR(.vectors) + 0x1000 : AT(__stubs_lma) { \ *(.stubs) \ } \ - . =3D __stubs_start + SIZEOF(.stubs); \ - __stubs_end =3D .; \ + ARM_LMA(__stubs, .stubs); \ + . =3D __stubs_lma + SIZEOF(.stubs); \ \ PROVIDE(vector_fiq_offset =3D vector_fiq - ADDR(.vectors)); From nobody Tue Jun 23 05:03:54 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C3B01C43217 for ; Thu, 10 Mar 2022 14:41:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244822AbiCJOiQ (ORCPT ); Thu, 10 Mar 2022 09:38:16 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50046 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1343621AbiCJObG (ORCPT ); Thu, 10 Mar 2022 09:31:06 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 070F09BBB2; Thu, 10 Mar 2022 06:27:08 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 2E4E9B8267B; Thu, 10 Mar 2022 14:26:37 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 77D23C340EB; Thu, 10 Mar 2022 14:26:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1646922395; bh=baxK+JmlXBcmFeTDzMr6h4q4h6PlSxvLkBq2vh+AFVo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=1hh1XS3Mmn2PJODMHANLczqP8Dgd7KeAl8Em60iNsUKjO+N5YVHEXZTeF5UANGtKr WTqyHsDF+qOZHGQzwqAKaOH7EUGsPlOpyuTjXrHRv8k+8N40RAASZZmHR4lMJ8kUNu wjfRk5x87rxxogM45YLWtyCl7dAh1swbnS3ne+LU= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Catalin Marinas , "Russell King (Oracle)" Subject: [PATCH 5.10 13/58] ARM: Spectre-BHB workaround Date: Thu, 10 Mar 2022 15:18:33 +0100 Message-Id: <20220310140813.253467897@linuxfoundation.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220310140812.869208747@linuxfoundation.org> References: <20220310140812.869208747@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: "Russell King (Oracle)" commit b9baf5c8c5c356757f4f9d8180b5e9d234065bc3 upstream. Workaround the Spectre BHB issues for Cortex-A15, Cortex-A57, Cortex-A72, Cortex-A73 and Cortex-A75. We also include Brahma B15 as well to be safe, which is affected by Spectre V2 in the same ways as Cortex-A15. Reviewed-by: Catalin Marinas Signed-off-by: Russell King (Oracle) [changes due to lack of SYSTEM_FREEING_INITMEM - gregkh] Signed-off-by: Greg Kroah-Hartman Tested-by: Bagas Sanjaya Tested-by: Florian Fainelli Tested-by: Fox Chen Tested-by: Guenter Roeck Tested-by: Linux Kernel Functional Testing Tested-by: Pavel Machek (CIP) = =20 Tested-by: Shuah Khan Tested-by: Sudip Mukherjee --- arch/arm/include/asm/assembler.h | 10 ++++ arch/arm/include/asm/spectre.h | 4 + arch/arm/include/asm/vmlinux.lds.h | 18 +++++++- arch/arm/kernel/entry-armv.S | 79 ++++++++++++++++++++++++++++++++= ++--- arch/arm/kernel/entry-common.S | 24 +++++++++++ arch/arm/kernel/spectre.c | 4 + arch/arm/kernel/traps.c | 38 +++++++++++++++++ arch/arm/mm/Kconfig | 10 ++++ arch/arm/mm/proc-v7-bugs.c | 76 ++++++++++++++++++++++++++++++++= +++ 9 files changed, 254 insertions(+), 9 deletions(-) --- a/arch/arm/include/asm/assembler.h +++ b/arch/arm/include/asm/assembler.h @@ -107,6 +107,16 @@ .endm #endif =20 +#if __LINUX_ARM_ARCH__ < 7 + .macro dsb, args + mcr p15, 0, r0, c7, c10, 4 + .endm + + .macro isb, args + mcr p15, 0, r0, c7, r5, 4 + .endm +#endif + .macro asm_trace_hardirqs_off, save=3D1 #if defined(CONFIG_TRACE_IRQFLAGS) .if \save --- a/arch/arm/include/asm/spectre.h +++ b/arch/arm/include/asm/spectre.h @@ -14,6 +14,7 @@ enum { __SPECTRE_V2_METHOD_ICIALLU, __SPECTRE_V2_METHOD_SMC, __SPECTRE_V2_METHOD_HVC, + __SPECTRE_V2_METHOD_LOOP8, }; =20 enum { @@ -21,8 +22,11 @@ enum { SPECTRE_V2_METHOD_ICIALLU =3D BIT(__SPECTRE_V2_METHOD_ICIALLU), SPECTRE_V2_METHOD_SMC =3D BIT(__SPECTRE_V2_METHOD_SMC), SPECTRE_V2_METHOD_HVC =3D BIT(__SPECTRE_V2_METHOD_HVC), + SPECTRE_V2_METHOD_LOOP8 =3D BIT(__SPECTRE_V2_METHOD_LOOP8), }; =20 void spectre_v2_update_state(unsigned int state, unsigned int methods); =20 +int spectre_bhb_update_vectors(unsigned int method); + #endif --- a/arch/arm/include/asm/vmlinux.lds.h +++ b/arch/arm/include/asm/vmlinux.lds.h @@ -116,11 +116,23 @@ */ #define ARM_VECTORS \ __vectors_lma =3D .; \ - .vectors 0xffff0000 : AT(__vectors_start) { \ - *(.vectors) \ + OVERLAY 0xffff0000 : NOCROSSREFS AT(__vectors_lma) { \ + .vectors { \ + *(.vectors) \ + } \ + .vectors.bhb.loop8 { \ + *(.vectors.bhb.loop8) \ + } \ + .vectors.bhb.bpiall { \ + *(.vectors.bhb.bpiall) \ + } \ } \ ARM_LMA(__vectors, .vectors); \ - . =3D __vectors_lma + SIZEOF(.vectors); \ + ARM_LMA(__vectors_bhb_loop8, .vectors.bhb.loop8); \ + ARM_LMA(__vectors_bhb_bpiall, .vectors.bhb.bpiall); \ + . =3D __vectors_lma + SIZEOF(.vectors) + \ + SIZEOF(.vectors.bhb.loop8) + \ + SIZEOF(.vectors.bhb.bpiall); \ \ __stubs_lma =3D .; \ .stubs ADDR(.vectors) + 0x1000 : AT(__stubs_lma) { \ --- a/arch/arm/kernel/entry-armv.S +++ b/arch/arm/kernel/entry-armv.S @@ -1005,12 +1005,11 @@ vector_\name: sub lr, lr, #\correction .endif =20 - @ - @ Save r0, lr_ (parent PC) and spsr_ - @ (parent CPSR) - @ + @ Save r0, lr_ (parent PC) stmia sp, {r0, lr} @ save r0, lr - mrs lr, spsr + + @ Save spsr_ (parent CPSR) +2: mrs lr, spsr str lr, [sp, #8] @ save spsr =20 @ @@ -1031,6 +1030,44 @@ vector_\name: movs pc, lr @ branch to handler in SVC mode ENDPROC(vector_\name) =20 +#ifdef CONFIG_HARDEN_BRANCH_HISTORY + .subsection 1 + .align 5 +vector_bhb_loop8_\name: + .if \correction + sub lr, lr, #\correction + .endif + + @ Save r0, lr_ (parent PC) + stmia sp, {r0, lr} + + @ bhb workaround + mov r0, #8 +1: b . + 4 + subs r0, r0, #1 + bne 1b + dsb + isb + b 2b +ENDPROC(vector_bhb_loop8_\name) + +vector_bhb_bpiall_\name: + .if \correction + sub lr, lr, #\correction + .endif + + @ Save r0, lr_ (parent PC) + stmia sp, {r0, lr} + + @ bhb workaround + mcr p15, 0, r0, c7, c5, 6 @ BPIALL + @ isb not needed due to "movs pc, lr" in the vector stub + @ which gives a "context synchronisation". + b 2b +ENDPROC(vector_bhb_bpiall_\name) + .previous +#endif + .align 2 @ handler addresses follow this label 1: @@ -1039,6 +1076,10 @@ ENDPROC(vector_\name) .section .stubs, "ax", %progbits @ This must be the first word .word vector_swi +#ifdef CONFIG_HARDEN_BRANCH_HISTORY + .word vector_bhb_loop8_swi + .word vector_bhb_bpiall_swi +#endif =20 vector_rst: ARM( swi SYS_ERROR0 ) @@ -1153,8 +1194,10 @@ vector_addrexcptn: * FIQ "NMI" handler *------------------------------------------------------------------------= ----- * Handle a FIQ using the SVC stack allowing FIQ act like NMI on x86 - * systems. + * systems. This must be the last vector stub, so lets place it in its own + * subsection. */ + .subsection 2 vector_stub fiq, FIQ_MODE, 4 =20 .long __fiq_usr @ 0 (USR_26 / USR_32) @@ -1187,6 +1230,30 @@ vector_addrexcptn: W(b) vector_irq W(b) vector_fiq =20 +#ifdef CONFIG_HARDEN_BRANCH_HISTORY + .section .vectors.bhb.loop8, "ax", %progbits +.L__vectors_bhb_loop8_start: + W(b) vector_rst + W(b) vector_bhb_loop8_und + W(ldr) pc, .L__vectors_bhb_loop8_start + 0x1004 + W(b) vector_bhb_loop8_pabt + W(b) vector_bhb_loop8_dabt + W(b) vector_addrexcptn + W(b) vector_bhb_loop8_irq + W(b) vector_bhb_loop8_fiq + + .section .vectors.bhb.bpiall, "ax", %progbits +.L__vectors_bhb_bpiall_start: + W(b) vector_rst + W(b) vector_bhb_bpiall_und + W(ldr) pc, .L__vectors_bhb_bpiall_start + 0x1008 + W(b) vector_bhb_bpiall_pabt + W(b) vector_bhb_bpiall_dabt + W(b) vector_addrexcptn + W(b) vector_bhb_bpiall_irq + W(b) vector_bhb_bpiall_fiq +#endif + .data .align 2 =20 --- a/arch/arm/kernel/entry-common.S +++ b/arch/arm/kernel/entry-common.S @@ -163,12 +163,36 @@ ENDPROC(ret_from_fork) */ =20 .align 5 +#ifdef CONFIG_HARDEN_BRANCH_HISTORY +ENTRY(vector_bhb_loop8_swi) + sub sp, sp, #PT_REGS_SIZE + stmia sp, {r0 - r12} + mov r8, #8 +1: b 2f +2: subs r8, r8, #1 + bne 1b + dsb + isb + b 3f +ENDPROC(vector_bhb_loop8_swi) + + .align 5 +ENTRY(vector_bhb_bpiall_swi) + sub sp, sp, #PT_REGS_SIZE + stmia sp, {r0 - r12} + mcr p15, 0, r8, c7, c5, 6 @ BPIALL + isb + b 3f +ENDPROC(vector_bhb_bpiall_swi) +#endif + .align 5 ENTRY(vector_swi) #ifdef CONFIG_CPU_V7M v7m_exception_entry #else sub sp, sp, #PT_REGS_SIZE stmia sp, {r0 - r12} @ Calling r0 - r12 +3: ARM( add r8, sp, #S_PC ) ARM( stmdb r8, {sp, lr}^ ) @ Calling sp, lr THUMB( mov r8, sp ) --- a/arch/arm/kernel/spectre.c +++ b/arch/arm/kernel/spectre.c @@ -45,6 +45,10 @@ ssize_t cpu_show_spectre_v2(struct devic method =3D "Firmware call"; break; =20 + case SPECTRE_V2_METHOD_LOOP8: + method =3D "History overwrite"; + break; + default: method =3D "Multiple mitigations"; break; --- a/arch/arm/kernel/traps.c +++ b/arch/arm/kernel/traps.c @@ -30,6 +30,7 @@ #include #include #include +#include #include #include #include @@ -820,6 +821,43 @@ static void flush_vectors(void *vma, siz flush_icache_range(start, end); } =20 +#ifdef CONFIG_HARDEN_BRANCH_HISTORY +int spectre_bhb_update_vectors(unsigned int method) +{ + extern char __vectors_bhb_bpiall_start[], __vectors_bhb_bpiall_end[]; + extern char __vectors_bhb_loop8_start[], __vectors_bhb_loop8_end[]; + void *vec_start, *vec_end; + + if (system_state > SYSTEM_SCHEDULING) { + pr_err("CPU%u: Spectre BHB workaround too late - system vulnerable\n", + smp_processor_id()); + return SPECTRE_VULNERABLE; + } + + switch (method) { + case SPECTRE_V2_METHOD_LOOP8: + vec_start =3D __vectors_bhb_loop8_start; + vec_end =3D __vectors_bhb_loop8_end; + break; + + case SPECTRE_V2_METHOD_BPIALL: + vec_start =3D __vectors_bhb_bpiall_start; + vec_end =3D __vectors_bhb_bpiall_end; + break; + + default: + pr_err("CPU%u: unknown Spectre BHB state %d\n", + smp_processor_id(), method); + return SPECTRE_VULNERABLE; + } + + copy_from_lma(vectors_page, vec_start, vec_end); + flush_vectors(vectors_page, 0, vec_end - vec_start); + + return SPECTRE_MITIGATED; +} +#endif + void __init early_trap_init(void *vectors_base) { extern char __stubs_start[], __stubs_end[]; --- a/arch/arm/mm/Kconfig +++ b/arch/arm/mm/Kconfig @@ -854,6 +854,16 @@ config HARDEN_BRANCH_PREDICTOR =20 If unsure, say Y. =20 +config HARDEN_BRANCH_HISTORY + bool "Harden Spectre style attacks against branch history" if EXPERT + depends on CPU_SPECTRE + default y + help + Speculation attacks against some high-performance processors can + make use of branch history to influence future speculation. When + taking an exception, a sequence of branches overwrites the branch + history, or branch history is invalidated. + config TLS_REG_EMUL bool select NEED_KUSER_HELPERS --- a/arch/arm/mm/proc-v7-bugs.c +++ b/arch/arm/mm/proc-v7-bugs.c @@ -177,6 +177,81 @@ static void cpu_v7_spectre_v2_init(void) spectre_v2_update_state(state, method); } =20 +#ifdef CONFIG_HARDEN_BRANCH_HISTORY +static int spectre_bhb_method; + +static const char *spectre_bhb_method_name(int method) +{ + switch (method) { + case SPECTRE_V2_METHOD_LOOP8: + return "loop"; + + case SPECTRE_V2_METHOD_BPIALL: + return "BPIALL"; + + default: + return "unknown"; + } +} + +static int spectre_bhb_install_workaround(int method) +{ + if (spectre_bhb_method !=3D method) { + if (spectre_bhb_method) { + pr_err("CPU%u: Spectre BHB: method disagreement, system vulnerable\n", + smp_processor_id()); + + return SPECTRE_VULNERABLE; + } + + if (spectre_bhb_update_vectors(method) =3D=3D SPECTRE_VULNERABLE) + return SPECTRE_VULNERABLE; + + spectre_bhb_method =3D method; + } + + pr_info("CPU%u: Spectre BHB: using %s workaround\n", + smp_processor_id(), spectre_bhb_method_name(method)); + + return SPECTRE_MITIGATED; +} +#else +static int spectre_bhb_install_workaround(int method) +{ + return SPECTRE_VULNERABLE; +} +#endif + +static void cpu_v7_spectre_bhb_init(void) +{ + unsigned int state, method =3D 0; + + switch (read_cpuid_part()) { + case ARM_CPU_PART_CORTEX_A15: + case ARM_CPU_PART_BRAHMA_B15: + case ARM_CPU_PART_CORTEX_A57: + case ARM_CPU_PART_CORTEX_A72: + state =3D SPECTRE_MITIGATED; + method =3D SPECTRE_V2_METHOD_LOOP8; + break; + + case ARM_CPU_PART_CORTEX_A73: + case ARM_CPU_PART_CORTEX_A75: + state =3D SPECTRE_MITIGATED; + method =3D SPECTRE_V2_METHOD_BPIALL; + break; + + default: + state =3D SPECTRE_UNAFFECTED; + break; + } + + if (state =3D=3D SPECTRE_MITIGATED) + state =3D spectre_bhb_install_workaround(method); + + spectre_v2_update_state(state, method); +} + static __maybe_unused bool cpu_v7_check_auxcr_set(bool *warned, u32 mask, const char *msg) { @@ -217,4 +292,5 @@ void cpu_v7_ca15_ibe(void) void cpu_v7_bugs_init(void) { cpu_v7_spectre_v2_init(); + cpu_v7_spectre_bhb_init(); } From nobody Tue Jun 23 05:03:54 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 28526C4321E for ; Thu, 10 Mar 2022 14:34:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237537AbiCJOfe (ORCPT ); Thu, 10 Mar 2022 09:35:34 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50102 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S244645AbiCJO3W (ORCPT ); Thu, 10 Mar 2022 09:29:22 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 99B0BD7620; Thu, 10 Mar 2022 06:24:22 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 69E5361D34; Thu, 10 Mar 2022 14:24:22 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 72384C340EB; Thu, 10 Mar 2022 14:24:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1646922261; bh=YYTxn4Nwrg6CuOgCloYwoSlfT43YKOmCi0fRD30gcL8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=XQKhYMzHDmJWaDcUp6AjKy9NCzAVCjqSxC3CCnGbZrWQoF/wlG+k0GmVz336YX2fo HWQ6+2qa09MnB5gdTl1jg/wzxP8SGEv5f6yjqTMJa7AS4Z66FvE9Ox23VxD7zjAPAG 1DGgjukQwruFVwjdCAbRoJmoDos+P5E6xzwXxSvU= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, "Russell King (Oracle)" Subject: [PATCH 5.10 14/58] ARM: include unprivileged BPF status in Spectre V2 reporting Date: Thu, 10 Mar 2022 15:18:34 +0100 Message-Id: <20220310140813.281736162@linuxfoundation.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220310140812.869208747@linuxfoundation.org> References: <20220310140812.869208747@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Russell King (Oracle) commit 25875aa71dfefd1959f07e626c4d285b88b27ac2 upstream. The mitigations for Spectre-BHB are only applied when an exception is taken, but when unprivileged BPF is enabled, userspace can load BPF programs that can be used to exploit the problem. When unprivileged BPF is enabled, report the vulnerable status via the spectre_v2 sysfs file. Signed-off-by: Russell King (Oracle) Signed-off-by: Greg Kroah-Hartman Tested-by: Bagas Sanjaya Tested-by: Florian Fainelli Tested-by: Fox Chen Tested-by: Guenter Roeck Tested-by: Linux Kernel Functional Testing Tested-by: Pavel Machek (CIP) = =20 Tested-by: Shuah Khan Tested-by: Sudip Mukherjee --- arch/arm/kernel/spectre.c | 13 +++++++++++++ 1 file changed, 13 insertions(+) --- a/arch/arm/kernel/spectre.c +++ b/arch/arm/kernel/spectre.c @@ -1,9 +1,19 @@ // SPDX-License-Identifier: GPL-2.0-only +#include #include #include =20 #include =20 +static bool _unprivileged_ebpf_enabled(void) +{ +#ifdef CONFIG_BPF_SYSCALL + return !sysctl_unprivileged_bpf_disabled; +#else + return false +#endif +} + ssize_t cpu_show_spectre_v1(struct device *dev, struct device_attribute *a= ttr, char *buf) { @@ -31,6 +41,9 @@ ssize_t cpu_show_spectre_v2(struct devic if (spectre_v2_state !=3D SPECTRE_MITIGATED) return sprintf(buf, "%s\n", "Vulnerable"); =20 + if (_unprivileged_ebpf_enabled()) + return sprintf(buf, "Vulnerable: Unprivileged eBPF enabled\n"); + switch (spectre_v2_methods) { case SPECTRE_V2_METHOD_BPIALL: method =3D "Branch predictor hardening"; From nobody Tue Jun 23 05:03:54 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 58F86C433EF for ; Thu, 10 Mar 2022 14:35:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243275AbiCJOgS (ORCPT ); Thu, 10 Mar 2022 09:36:18 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50328 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S244952AbiCJO3l (ORCPT ); Thu, 10 Mar 2022 09:29:41 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B9826156963; Thu, 10 Mar 2022 06:25:08 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 4B64BB8267B; Thu, 10 Mar 2022 14:24:57 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id B1461C340E8; Thu, 10 Mar 2022 14:24:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1646922296; bh=BjhNuDswWhutw0Va1gA9M+7Vl6Iaj6ZmKGq889IWLoA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Ma3yPSCd2gHee7LBAY3vHmWZ38u4+yYbQsbD8Xz5En6PlAfpGWCRIVa44xzyAWuGY e6uIurMcZlzYh66dGILTw/Rs0mbYJD8oTiYwttKlZ1oo/H/Q7F1Y4Xnv8E5lORkRZn WGrdZRvdZQzz2Kvo0rm8J87K1Y9xCKr0IUkFDiNE= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Will Deacon , Hector Martin Subject: [PATCH 5.10 15/58] arm64: cputype: Add CPU implementor & types for the Apple M1 cores Date: Thu, 10 Mar 2022 15:18:35 +0100 Message-Id: <20220310140813.310046847@linuxfoundation.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220310140812.869208747@linuxfoundation.org> References: <20220310140812.869208747@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Hector Martin commit 11ecdad722daafcac09c4859dddf31b3d46449bc upstream. The implementor will be used to condition the FIQ support quirk. The specific CPU types are not used at the moment, but let's add them for documentation purposes. Acked-by: Will Deacon Signed-off-by: Hector Martin Signed-off-by: Greg Kroah-Hartman Tested-by: Bagas Sanjaya Tested-by: Florian Fainelli Tested-by: Fox Chen Tested-by: Guenter Roeck Tested-by: Linux Kernel Functional Testing Tested-by: Pavel Machek (CIP) = =20 Tested-by: Shuah Khan Tested-by: Sudip Mukherjee --- arch/arm64/include/asm/cputype.h | 6 ++++++ 1 file changed, 6 insertions(+) --- a/arch/arm64/include/asm/cputype.h +++ b/arch/arm64/include/asm/cputype.h @@ -59,6 +59,7 @@ #define ARM_CPU_IMP_NVIDIA 0x4E #define ARM_CPU_IMP_FUJITSU 0x46 #define ARM_CPU_IMP_HISI 0x48 +#define ARM_CPU_IMP_APPLE 0x61 =20 #define ARM_CPU_PART_AEM_V8 0xD0F #define ARM_CPU_PART_FOUNDATION 0xD00 @@ -99,6 +100,9 @@ =20 #define HISI_CPU_PART_TSV110 0xD01 =20 +#define APPLE_CPU_PART_M1_ICESTORM 0x022 +#define APPLE_CPU_PART_M1_FIRESTORM 0x023 + #define MIDR_CORTEX_A53 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTE= X_A53) #define MIDR_CORTEX_A57 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTE= X_A57) #define MIDR_CORTEX_A72 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTE= X_A72) @@ -127,6 +131,8 @@ #define MIDR_NVIDIA_CARMEL MIDR_CPU_MODEL(ARM_CPU_IMP_NVIDIA, NVIDIA_CPU_P= ART_CARMEL) #define MIDR_FUJITSU_A64FX MIDR_CPU_MODEL(ARM_CPU_IMP_FUJITSU, FUJITSU_CPU= _PART_A64FX) #define MIDR_HISI_TSV110 MIDR_CPU_MODEL(ARM_CPU_IMP_HISI, HISI_CPU_PART_TS= V110) +#define MIDR_APPLE_M1_ICESTORM MIDR_CPU_MODEL(ARM_CPU_IMP_APPLE, APPLE_CPU= _PART_M1_ICESTORM) +#define MIDR_APPLE_M1_FIRESTORM MIDR_CPU_MODEL(ARM_CPU_IMP_APPLE, APPLE_CP= U_PART_M1_FIRESTORM) =20 /* Fujitsu Erratum 010001 affects A64FX 1.0 and 1.1, (v0r0 and v1r0) */ #define MIDR_FUJITSU_ERRATUM_010001 MIDR_FUJITSU_A64FX From nobody Tue Jun 23 05:03:54 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 68CADC433EF for ; Thu, 10 Mar 2022 14:35:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244321AbiCJOgw (ORCPT ); Thu, 10 Mar 2022 09:36:52 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50356 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245119AbiCJOaI (ORCPT ); Thu, 10 Mar 2022 09:30:08 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DB7E6167FBE; Thu, 10 Mar 2022 06:25:31 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 7FE8BB82681; Thu, 10 Mar 2022 14:25:31 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id C1028C340E8; Thu, 10 Mar 2022 14:25:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1646922330; bh=VIeR+4wJVyYLfoUSGwRC4ht7eH6VTH3h63d5twbkx7s=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=fQmQZy/z4rtFa6ZBeXQJGCyPdqs8eWb7DhsFmoctv9iMgwiy8szShhkGAdYIfEOh7 lh2yQB9Tw3or4MxKg9CqpCQFacs2uffVBzLAV/CyMZSv+aM1Qj2iVnieFB7slNwp9y vYjcD4pXriy5WEQu2lDXQ4cf4eoWifPkyo/fxpaE= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Catalin Marinas , Mark Rutland , Will Deacon , Anshuman Khandual , Suzuki K Poulose Subject: [PATCH 5.10 16/58] arm64: Add Neoverse-N2, Cortex-A710 CPU part definition Date: Thu, 10 Mar 2022 15:18:36 +0100 Message-Id: <20220310140813.337478886@linuxfoundation.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220310140812.869208747@linuxfoundation.org> References: <20220310140812.869208747@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Suzuki K Poulose commit 2d0d656700d67239a57afaf617439143d8dac9be upstream. Add the CPU Partnumbers for the new Arm designs. Cc: Catalin Marinas Cc: Mark Rutland Cc: Will Deacon Acked-by: Catalin Marinas Reviewed-by: Anshuman Khandual Signed-off-by: Suzuki K Poulose Link: https://lore.kernel.org/r/20211019163153.3692640-2-suzuki.poulose@arm= .com Signed-off-by: Will Deacon Signed-off-by: Greg Kroah-Hartman Tested-by: Bagas Sanjaya Tested-by: Florian Fainelli Tested-by: Fox Chen Tested-by: Guenter Roeck Tested-by: Linux Kernel Functional Testing Tested-by: Pavel Machek (CIP) = =20 Tested-by: Shuah Khan Tested-by: Sudip Mukherjee --- arch/arm64/include/asm/cputype.h | 4 ++++ 1 file changed, 4 insertions(+) --- a/arch/arm64/include/asm/cputype.h +++ b/arch/arm64/include/asm/cputype.h @@ -73,6 +73,8 @@ #define ARM_CPU_PART_CORTEX_A76 0xD0B #define ARM_CPU_PART_NEOVERSE_N1 0xD0C #define ARM_CPU_PART_CORTEX_A77 0xD0D +#define ARM_CPU_PART_CORTEX_A710 0xD47 +#define ARM_CPU_PART_NEOVERSE_N2 0xD49 =20 #define APM_CPU_PART_POTENZA 0x000 =20 @@ -113,6 +115,8 @@ #define MIDR_CORTEX_A76 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTE= X_A76) #define MIDR_NEOVERSE_N1 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_NEOV= ERSE_N1) #define MIDR_CORTEX_A77 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTE= X_A77) +#define MIDR_CORTEX_A710 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORT= EX_A710) +#define MIDR_NEOVERSE_N2 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_NEOV= ERSE_N2) #define MIDR_THUNDERX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_T= HUNDERX) #define MIDR_THUNDERX_81XX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_P= ART_THUNDERX_81XX) #define MIDR_THUNDERX_83XX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_P= ART_THUNDERX_83XX) From nobody Tue Jun 23 05:03:54 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3EF8DC3527B for ; Thu, 10 Mar 2022 14:42:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1345361AbiCJOlu (ORCPT ); Thu, 10 Mar 2022 09:41:50 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49798 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245428AbiCJOa3 (ORCPT ); Thu, 10 Mar 2022 09:30:29 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3F5B517FD02; Thu, 10 Mar 2022 06:26:12 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 150D1B8267B; Thu, 10 Mar 2022 14:26:06 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 47450C340F4; Thu, 10 Mar 2022 14:26:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1646922364; bh=Z76mI4D7yx4KPGKf1EO1KkTg3gHcQGJDisT/RVaxLNo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=a4CyY0650Y3JjtSbusyJyWL79rrD4nTuRR66lRskg0iFTW1dnOhKD/iUeDNtxaKnm yvTNAx4615t3ABCIvUyLKjQ19l6B4L+lEV22Kj0L7/uZH/U5wsr+R4cJxpxy1VCBdJ ZHkQVisFDFsm52D9XUB6j5P0E2ipbDcdF2bB0J8M= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Will Deacon , Suzuki Poulose , linux-arm-kernel@lists.infradead.org, Anshuman Khandual , Catalin Marinas Subject: [PATCH 5.10 17/58] arm64: Add Cortex-X2 CPU part definition Date: Thu, 10 Mar 2022 15:18:37 +0100 Message-Id: <20220310140813.365438626@linuxfoundation.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220310140812.869208747@linuxfoundation.org> References: <20220310140812.869208747@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Anshuman Khandual commit 72bb9dcb6c33cfac80282713c2b4f2b254cd24d1 upstream. Add the CPU Partnumbers for the new Arm designs. Cc: Will Deacon Cc: Suzuki Poulose Cc: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Anshuman Khandual Reviewed-by: Suzuki K Poulose Link: https://lore.kernel.org/r/1642994138-25887-2-git-send-email-anshuman.= khandual@arm.com Signed-off-by: Catalin Marinas Signed-off-by: Greg Kroah-Hartman Tested-by: Bagas Sanjaya Tested-by: Florian Fainelli Tested-by: Fox Chen Tested-by: Guenter Roeck Tested-by: Linux Kernel Functional Testing Tested-by: Pavel Machek (CIP) = =20 Tested-by: Shuah Khan Tested-by: Sudip Mukherjee --- arch/arm64/include/asm/cputype.h | 2 ++ 1 file changed, 2 insertions(+) --- a/arch/arm64/include/asm/cputype.h +++ b/arch/arm64/include/asm/cputype.h @@ -74,6 +74,7 @@ #define ARM_CPU_PART_NEOVERSE_N1 0xD0C #define ARM_CPU_PART_CORTEX_A77 0xD0D #define ARM_CPU_PART_CORTEX_A710 0xD47 +#define ARM_CPU_PART_CORTEX_X2 0xD48 #define ARM_CPU_PART_NEOVERSE_N2 0xD49 =20 #define APM_CPU_PART_POTENZA 0x000 @@ -116,6 +117,7 @@ #define MIDR_NEOVERSE_N1 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_NEOV= ERSE_N1) #define MIDR_CORTEX_A77 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTE= X_A77) #define MIDR_CORTEX_A710 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORT= EX_A710) +#define MIDR_CORTEX_X2 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX= _X2) #define MIDR_NEOVERSE_N2 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_NEOV= ERSE_N2) #define MIDR_THUNDERX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_T= HUNDERX) #define MIDR_THUNDERX_81XX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_P= ART_THUNDERX_81XX) From nobody Tue Jun 23 05:03:54 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1A3A8C433F5 for ; Thu, 10 Mar 2022 14:36:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244588AbiCJOhl (ORCPT ); Thu, 10 Mar 2022 09:37:41 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49724 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245495AbiCJOad (ORCPT ); Thu, 10 Mar 2022 09:30:33 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 64A0C180D2B; Thu, 10 Mar 2022 06:26:21 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 82CDA61CFB; Thu, 10 Mar 2022 14:26:20 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 859FAC340EB; Thu, 10 Mar 2022 14:26:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1646922379; bh=UcH8Zzvo0iQZ5Rax9GLIymn0QqKccpjDo791IEfgOr8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=hSRFbNo+Vfezvzmok8C4n9rbcFIQ0qnX2K1sRmc9GEbAb0Pr7+33ZMKJgLiSfYVJM 1KwaPlXLT045XTcLTKGl2e3/aUj6JqE6rspjEuMvM9m0xajtO3Qz2dfL/JSbCCqUfn NcD0AnO9YBydhl8S6GCaBuJXxfJTNOP3DnJnPGUs= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Catalin Marinas , Will Deacon , Suzuki Poulose , linux-arm-kernel@lists.infradead.org, Anshuman Khandual , Mathieu Poirier Subject: [PATCH 5.10 18/58] arm64: Add Cortex-A510 CPU part definition Date: Thu, 10 Mar 2022 15:18:38 +0100 Message-Id: <20220310140813.393834618@linuxfoundation.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220310140812.869208747@linuxfoundation.org> References: <20220310140812.869208747@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Anshuman Khandual commit 53960faf2b731dd2f9ed6e1334634b8ba6286850 upstream. Add the CPU Partnumbers for the new Arm designs. Cc: Catalin Marinas Cc: Will Deacon Cc: Suzuki Poulose Cc: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org Reviewed-by: Suzuki K Poulose Acked-by: Catalin Marinas Signed-off-by: Anshuman Khandual Link: https://lore.kernel.org/r/1643120437-14352-2-git-send-email-anshuman.= khandual@arm.com Signed-off-by: Mathieu Poirier Signed-off-by: Greg Kroah-Hartman Tested-by: Bagas Sanjaya Tested-by: Florian Fainelli Tested-by: Fox Chen Tested-by: Guenter Roeck Tested-by: Linux Kernel Functional Testing Tested-by: Pavel Machek (CIP) = =20 Tested-by: Shuah Khan Tested-by: Sudip Mukherjee --- arch/arm64/include/asm/cputype.h | 2 ++ 1 file changed, 2 insertions(+) --- a/arch/arm64/include/asm/cputype.h +++ b/arch/arm64/include/asm/cputype.h @@ -73,6 +73,7 @@ #define ARM_CPU_PART_CORTEX_A76 0xD0B #define ARM_CPU_PART_NEOVERSE_N1 0xD0C #define ARM_CPU_PART_CORTEX_A77 0xD0D +#define ARM_CPU_PART_CORTEX_A510 0xD46 #define ARM_CPU_PART_CORTEX_A710 0xD47 #define ARM_CPU_PART_CORTEX_X2 0xD48 #define ARM_CPU_PART_NEOVERSE_N2 0xD49 @@ -116,6 +117,7 @@ #define MIDR_CORTEX_A76 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTE= X_A76) #define MIDR_NEOVERSE_N1 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_NEOV= ERSE_N1) #define MIDR_CORTEX_A77 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTE= X_A77) +#define MIDR_CORTEX_A510 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORT= EX_A510) #define MIDR_CORTEX_A710 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORT= EX_A710) #define MIDR_CORTEX_X2 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX= _X2) #define MIDR_NEOVERSE_N2 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_NEOV= ERSE_N2) From nobody Tue Jun 23 05:03:54 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A54D3C433EF for ; Thu, 10 Mar 2022 14:41:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244761AbiCJOiG (ORCPT ); Thu, 10 Mar 2022 09:38:06 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51970 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245754AbiCJOaw (ORCPT ); Thu, 10 Mar 2022 09:30:52 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 10572186BB5; Thu, 10 Mar 2022 06:26:51 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id DCE4D61CFE; Thu, 10 Mar 2022 14:26:23 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id CB319C340F3; Thu, 10 Mar 2022 14:26:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1646922383; bh=EiN5jOQma35TrgWid/JvsEci+kvH7N05UOAhG25jB0w=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=zTql1RaHDZh8TqQqPXDtyz2MVkIIWN2djcbvAuZEKUqo2+Dz0PWk4qhxrehtvlbaz hhrQxSAfoXMj3/UnN3tT4B9gBegqNkIPhGSvVe6kG0qvp8HGcigUZZoSNpLu7Qdw1L UK2WXpIke7uOoGdYX9l+gLcleq2OG5a/qnoAFyjA= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Will Deacon , Marc Zyngier Subject: [PATCH 5.10 19/58] arm64: Add HWCAP for self-synchronising virtual counter Date: Thu, 10 Mar 2022 15:18:39 +0100 Message-Id: <20220310140813.422534556@linuxfoundation.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220310140812.869208747@linuxfoundation.org> References: <20220310140812.869208747@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Marc Zyngier commit fee29f008aa3f2aff01117f28b57b1145d92cb9b upstream. Since userspace can make use of the CNTVSS_EL0 instruction, expose it via a HWCAP. Suggested-by: Will Deacon Acked-by: Will Deacon Signed-off-by: Marc Zyngier Link: https://lore.kernel.org/r/20211017124225.3018098-18-maz@kernel.org Signed-off-by: Will Deacon Signed-off-by: Greg Kroah-Hartman Tested-by: Bagas Sanjaya Tested-by: Florian Fainelli Tested-by: Fox Chen Tested-by: Guenter Roeck Tested-by: Linux Kernel Functional Testing Tested-by: Pavel Machek (CIP) = =20 Tested-by: Shuah Khan Tested-by: Sudip Mukherjee --- Documentation/arm64/cpu-feature-registers.rst | 12 ++++++++++-- Documentation/arm64/elf_hwcaps.rst | 4 ++++ arch/arm64/include/asm/hwcap.h | 1 + arch/arm64/include/uapi/asm/hwcap.h | 1 + arch/arm64/kernel/cpufeature.c | 3 ++- arch/arm64/kernel/cpuinfo.c | 1 + 6 files changed, 19 insertions(+), 3 deletions(-) --- a/Documentation/arm64/cpu-feature-registers.rst +++ b/Documentation/arm64/cpu-feature-registers.rst @@ -235,7 +235,15 @@ infrastructure: | DPB | [3-0] | y | +------------------------------+---------+---------+ =20 - 6) ID_AA64MMFR2_EL1 - Memory model feature register 2 + 6) ID_AA64MMFR0_EL1 - Memory model feature register 0 + + +------------------------------+---------+---------+ + | Name | bits | visible | + +------------------------------+---------+---------+ + | ECV | [63-60] | y | + +------------------------------+---------+---------+ + + 7) ID_AA64MMFR2_EL1 - Memory model feature register 2 =20 +------------------------------+---------+---------+ | Name | bits | visible | @@ -243,7 +251,7 @@ infrastructure: | AT | [35-32] | y | +------------------------------+---------+---------+ =20 - 7) ID_AA64ZFR0_EL1 - SVE feature ID register 0 + 8) ID_AA64ZFR0_EL1 - SVE feature ID register 0 =20 +------------------------------+---------+---------+ | Name | bits | visible | --- a/Documentation/arm64/elf_hwcaps.rst +++ b/Documentation/arm64/elf_hwcaps.rst @@ -245,6 +245,10 @@ HWCAP2_MTE Functionality implied by ID_AA64PFR1_EL1.MTE =3D=3D 0b0010, as describ= ed by Documentation/arm64/memory-tagging-extension.rst. =20 +HWCAP2_ECV + + Functionality implied by ID_AA64MMFR0_EL1.ECV =3D=3D 0b0001. + 4. Unused AT_HWCAP bits ----------------------- =20 --- a/arch/arm64/include/asm/hwcap.h +++ b/arch/arm64/include/asm/hwcap.h @@ -105,6 +105,7 @@ #define KERNEL_HWCAP_RNG __khwcap2_feature(RNG) #define KERNEL_HWCAP_BTI __khwcap2_feature(BTI) #define KERNEL_HWCAP_MTE __khwcap2_feature(MTE) +#define KERNEL_HWCAP_ECV __khwcap2_feature(ECV) =20 /* * This yields a mask that user programs can use to figure out what --- a/arch/arm64/include/uapi/asm/hwcap.h +++ b/arch/arm64/include/uapi/asm/hwcap.h @@ -75,5 +75,6 @@ #define HWCAP2_RNG (1 << 16) #define HWCAP2_BTI (1 << 17) #define HWCAP2_MTE (1 << 18) +#define HWCAP2_ECV (1 << 19) =20 #endif /* _UAPI__ASM_HWCAP_H */ --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -259,7 +259,7 @@ static const struct arm64_ftr_bits ftr_i }; =20 static const struct arm64_ftr_bits ftr_id_aa64mmfr0[] =3D { - ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR0_ECV_S= HIFT, 4, 0), + ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR0_ECV_= SHIFT, 4, 0), ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR0_FGT_S= HIFT, 4, 0), ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR0_EXS_S= HIFT, 4, 0), /* @@ -2252,6 +2252,7 @@ static const struct arm64_cpu_capabiliti #ifdef CONFIG_ARM64_MTE HWCAP_CAP(SYS_ID_AA64PFR1_EL1, ID_AA64PFR1_MTE_SHIFT, FTR_UNSIGNED, ID_AA= 64PFR1_MTE, CAP_HWCAP, KERNEL_HWCAP_MTE), #endif /* CONFIG_ARM64_MTE */ + HWCAP_CAP(SYS_ID_AA64MMFR0_EL1, ID_AA64MMFR0_ECV_SHIFT, FTR_UNSIGNED, 1, = CAP_HWCAP, KERNEL_HWCAP_ECV), {}, }; =20 --- a/arch/arm64/kernel/cpuinfo.c +++ b/arch/arm64/kernel/cpuinfo.c @@ -94,6 +94,7 @@ static const char *const hwcap_str[] =3D { [KERNEL_HWCAP_RNG] =3D "rng", [KERNEL_HWCAP_BTI] =3D "bti", [KERNEL_HWCAP_MTE] =3D "mte", + [KERNEL_HWCAP_ECV] =3D "ecv", }; =20 #ifdef CONFIG_COMPAT From nobody Tue Jun 23 05:03:54 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C4433C4332F for ; Thu, 10 Mar 2022 14:42:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1345466AbiCJOly (ORCPT ); Thu, 10 Mar 2022 09:41:54 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49514 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245565AbiCJOaj (ORCPT ); Thu, 10 Mar 2022 09:30:39 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BFE4E182BE7; Thu, 10 Mar 2022 06:26:27 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 5BE4FB825A7; Thu, 10 Mar 2022 14:26:27 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id C56EEC340F3; Thu, 10 Mar 2022 14:26:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1646922386; bh=l8kb+JyFkCFG12lhe+TB8NG8sV6PXySdPd9O6zdBpGk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=QWRUs7ELghAbsWeoFRR+sROWTW8IQAB5nRt4QD1is6bBNTbyU6GlIzdxj9CCDZLPm Ts5S+zuU9kAJRjp9yEkqg5DCFg+48FBdb+XgLx9jfoSagnyLQxKIcD4hpX6oqI9DKE EB5E5eNSnmexYD1W/ycsdPyjOr9mGD8x3V2cnahg= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Joey Gouly , Will Deacon , Marc Zyngier , James Morse , Alexandru Elisei , Suzuki K Poulose , Reiji Watanabe , Catalin Marinas Subject: [PATCH 5.10 20/58] arm64: add ID_AA64ISAR2_EL1 sys register Date: Thu, 10 Mar 2022 15:18:40 +0100 Message-Id: <20220310140813.450792361@linuxfoundation.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220310140812.869208747@linuxfoundation.org> References: <20220310140812.869208747@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Joey Gouly commit 9e45365f1469ef2b934f9d035975dbc9ad352116 upstream. This is a new ID register, introduced in 8.7. Signed-off-by: Joey Gouly Cc: Will Deacon Cc: Marc Zyngier Cc: James Morse Cc: Alexandru Elisei Cc: Suzuki K Poulose Cc: Reiji Watanabe Acked-by: Marc Zyngier Link: https://lore.kernel.org/r/20211210165432.8106-3-joey.gouly@arm.com Signed-off-by: Catalin Marinas Signed-off-by: Greg Kroah-Hartman Tested-by: Bagas Sanjaya Tested-by: Florian Fainelli Tested-by: Fox Chen Tested-by: Guenter Roeck Tested-by: Linux Kernel Functional Testing Tested-by: Pavel Machek (CIP) = =20 Tested-by: Shuah Khan Tested-by: Sudip Mukherjee --- arch/arm64/include/asm/cpu.h | 1 + arch/arm64/include/asm/sysreg.h | 15 +++++++++++++++ arch/arm64/kernel/cpufeature.c | 9 +++++++++ arch/arm64/kernel/cpuinfo.c | 1 + arch/arm64/kvm/sys_regs.c | 2 +- 5 files changed, 27 insertions(+), 1 deletion(-) --- a/arch/arm64/include/asm/cpu.h +++ b/arch/arm64/include/asm/cpu.h @@ -25,6 +25,7 @@ struct cpuinfo_arm64 { u64 reg_id_aa64dfr1; u64 reg_id_aa64isar0; u64 reg_id_aa64isar1; + u64 reg_id_aa64isar2; u64 reg_id_aa64mmfr0; u64 reg_id_aa64mmfr1; u64 reg_id_aa64mmfr2; --- a/arch/arm64/include/asm/sysreg.h +++ b/arch/arm64/include/asm/sysreg.h @@ -175,6 +175,7 @@ =20 #define SYS_ID_AA64ISAR0_EL1 sys_reg(3, 0, 0, 6, 0) #define SYS_ID_AA64ISAR1_EL1 sys_reg(3, 0, 0, 6, 1) +#define SYS_ID_AA64ISAR2_EL1 sys_reg(3, 0, 0, 6, 2) =20 #define SYS_ID_AA64MMFR0_EL1 sys_reg(3, 0, 0, 7, 0) #define SYS_ID_AA64MMFR1_EL1 sys_reg(3, 0, 0, 7, 1) @@ -687,6 +688,20 @@ #define ID_AA64ISAR1_GPI_NI 0x0 #define ID_AA64ISAR1_GPI_IMP_DEF 0x1 =20 +/* id_aa64isar2 */ +#define ID_AA64ISAR2_RPRES_SHIFT 4 +#define ID_AA64ISAR2_WFXT_SHIFT 0 + +#define ID_AA64ISAR2_RPRES_8BIT 0x0 +#define ID_AA64ISAR2_RPRES_12BIT 0x1 +/* + * Value 0x1 has been removed from the architecture, and is + * reserved, but has not yet been removed from the ARM ARM + * as of ARM DDI 0487G.b. + */ +#define ID_AA64ISAR2_WFXT_NI 0x0 +#define ID_AA64ISAR2_WFXT_SUPPORTED 0x2 + /* id_aa64pfr0 */ #define ID_AA64PFR0_CSV3_SHIFT 60 #define ID_AA64PFR0_CSV2_SHIFT 56 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -205,6 +205,10 @@ static const struct arm64_ftr_bits ftr_i ARM64_FTR_END, }; =20 +static const struct arm64_ftr_bits ftr_id_aa64isar2[] =3D { + ARM64_FTR_END, +}; + static const struct arm64_ftr_bits ftr_id_aa64pfr0[] =3D { ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR0_CSV= 3_SHIFT, 4, 0), ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR0_CSV= 2_SHIFT, 4, 0), @@ -596,6 +600,7 @@ static const struct __ftr_reg_entry { /* Op1 =3D 0, CRn =3D 0, CRm =3D 6 */ ARM64_FTR_REG(SYS_ID_AA64ISAR0_EL1, ftr_id_aa64isar0), ARM64_FTR_REG(SYS_ID_AA64ISAR1_EL1, ftr_id_aa64isar1), + ARM64_FTR_REG(SYS_ID_AA64ISAR2_EL1, ftr_id_aa64isar2), =20 /* Op1 =3D 0, CRn =3D 0, CRm =3D 7 */ ARM64_FTR_REG(SYS_ID_AA64MMFR0_EL1, ftr_id_aa64mmfr0), @@ -830,6 +835,7 @@ void __init init_cpu_features(struct cpu init_cpu_ftr_reg(SYS_ID_AA64DFR1_EL1, info->reg_id_aa64dfr1); init_cpu_ftr_reg(SYS_ID_AA64ISAR0_EL1, info->reg_id_aa64isar0); init_cpu_ftr_reg(SYS_ID_AA64ISAR1_EL1, info->reg_id_aa64isar1); + init_cpu_ftr_reg(SYS_ID_AA64ISAR2_EL1, info->reg_id_aa64isar2); init_cpu_ftr_reg(SYS_ID_AA64MMFR0_EL1, info->reg_id_aa64mmfr0); init_cpu_ftr_reg(SYS_ID_AA64MMFR1_EL1, info->reg_id_aa64mmfr1); init_cpu_ftr_reg(SYS_ID_AA64MMFR2_EL1, info->reg_id_aa64mmfr2); @@ -1058,6 +1064,8 @@ void update_cpu_features(int cpu, info->reg_id_aa64isar0, boot->reg_id_aa64isar0); taint |=3D check_update_ftr_reg(SYS_ID_AA64ISAR1_EL1, cpu, info->reg_id_aa64isar1, boot->reg_id_aa64isar1); + taint |=3D check_update_ftr_reg(SYS_ID_AA64ISAR2_EL1, cpu, + info->reg_id_aa64isar2, boot->reg_id_aa64isar2); =20 /* * Differing PARange support is fine as long as all peripherals and @@ -1157,6 +1165,7 @@ static u64 __read_sysreg_by_encoding(u32 read_sysreg_case(SYS_ID_AA64MMFR2_EL1); read_sysreg_case(SYS_ID_AA64ISAR0_EL1); read_sysreg_case(SYS_ID_AA64ISAR1_EL1); + read_sysreg_case(SYS_ID_AA64ISAR2_EL1); =20 read_sysreg_case(SYS_CNTFRQ_EL0); read_sysreg_case(SYS_CTR_EL0); --- a/arch/arm64/kernel/cpuinfo.c +++ b/arch/arm64/kernel/cpuinfo.c @@ -365,6 +365,7 @@ static void __cpuinfo_store_cpu(struct c info->reg_id_aa64dfr1 =3D read_cpuid(ID_AA64DFR1_EL1); info->reg_id_aa64isar0 =3D read_cpuid(ID_AA64ISAR0_EL1); info->reg_id_aa64isar1 =3D read_cpuid(ID_AA64ISAR1_EL1); + info->reg_id_aa64isar2 =3D read_cpuid(ID_AA64ISAR2_EL1); info->reg_id_aa64mmfr0 =3D read_cpuid(ID_AA64MMFR0_EL1); info->reg_id_aa64mmfr1 =3D read_cpuid(ID_AA64MMFR1_EL1); info->reg_id_aa64mmfr2 =3D read_cpuid(ID_AA64MMFR2_EL1); --- a/arch/arm64/kvm/sys_regs.c +++ b/arch/arm64/kvm/sys_regs.c @@ -1517,7 +1517,7 @@ static const struct sys_reg_desc sys_reg /* CRm=3D6 */ ID_SANITISED(ID_AA64ISAR0_EL1), ID_SANITISED(ID_AA64ISAR1_EL1), - ID_UNALLOCATED(6,2), + ID_SANITISED(ID_AA64ISAR2_EL1), ID_UNALLOCATED(6,3), ID_UNALLOCATED(6,4), ID_UNALLOCATED(6,5), From nobody Tue Jun 23 05:03:54 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6B152C3527C for ; Thu, 10 Mar 2022 14:42:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1345503AbiCJOl5 (ORCPT ); Thu, 10 Mar 2022 09:41:57 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49600 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245590AbiCJOal (ORCPT ); Thu, 10 Mar 2022 09:30:41 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8856D182D98; Thu, 10 Mar 2022 06:26:30 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 2720F61DE2; Thu, 10 Mar 2022 14:26:30 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id D464EC36AE2; Thu, 10 Mar 2022 14:26:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1646922389; bh=dqDpI/5xQ7Av+jQXodxQtvhlZ6BRecM6BgdEoxb4u4Y=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=tdwtNeOOjgo8MuggDvsHSZ/EU6eI8jSoV6TeXVQiCHgbfWxxGQD2etbWUoZXY2IFw JCs8B5l7thV12JW33uXFqKAauCrOGZJzY1aVvhx42Lw3OSAt7z1LTuPUO46U+cpvY2 WNe7CmUgHinrhMFHq9W6jbuX37iGT9zgHm2/b7To= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Joey Gouly , Will Deacon , Marc Zyngier , Catalin Marinas Subject: [PATCH 5.10 21/58] arm64: cpufeature: add HWCAP for FEAT_AFP Date: Thu, 10 Mar 2022 15:18:41 +0100 Message-Id: <20220310140813.479479299@linuxfoundation.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220310140812.869208747@linuxfoundation.org> References: <20220310140812.869208747@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Joey Gouly commit 5c13f042e73200b50573ace63e1a6b94e2917616 upstream. Add a new HWCAP to detect the Alternate Floating-point Behaviour feature (FEAT_AFP), introduced in Armv8.7. Also expose this to userspace in the ID_AA64MMFR1_EL1 feature register. Signed-off-by: Joey Gouly Cc: Will Deacon Acked-by: Marc Zyngier Link: https://lore.kernel.org/r/20211210165432.8106-2-joey.gouly@arm.com Signed-off-by: Catalin Marinas Signed-off-by: Greg Kroah-Hartman Tested-by: Bagas Sanjaya Tested-by: Florian Fainelli Tested-by: Fox Chen Tested-by: Guenter Roeck Tested-by: Linux Kernel Functional Testing Tested-by: Pavel Machek (CIP) = =20 Tested-by: Shuah Khan Tested-by: Sudip Mukherjee --- Documentation/arm64/cpu-feature-registers.rst | 9 +++++++++ Documentation/arm64/elf_hwcaps.rst | 4 ++++ arch/arm64/include/asm/hwcap.h | 1 + arch/arm64/include/asm/sysreg.h | 1 + arch/arm64/include/uapi/asm/hwcap.h | 1 + arch/arm64/kernel/cpufeature.c | 2 ++ arch/arm64/kernel/cpuinfo.c | 1 + 7 files changed, 19 insertions(+) --- a/Documentation/arm64/cpu-feature-registers.rst +++ b/Documentation/arm64/cpu-feature-registers.rst @@ -275,6 +275,15 @@ infrastructure: | SVEVer | [3-0] | y | +------------------------------+---------+---------+ =20 + 8) ID_AA64MMFR1_EL1 - Memory model feature register 1 + + +------------------------------+---------+---------+ + | Name | bits | visible | + +------------------------------+---------+---------+ + | AFP | [47-44] | y | + +------------------------------+---------+---------+ + + Appendix I: Example ------------------- =20 --- a/Documentation/arm64/elf_hwcaps.rst +++ b/Documentation/arm64/elf_hwcaps.rst @@ -249,6 +249,10 @@ HWCAP2_ECV =20 Functionality implied by ID_AA64MMFR0_EL1.ECV =3D=3D 0b0001. =20 +HWCAP2_AFP + + Functionality implied by ID_AA64MFR1_EL1.AFP =3D=3D 0b0001. + 4. Unused AT_HWCAP bits ----------------------- =20 --- a/arch/arm64/include/asm/hwcap.h +++ b/arch/arm64/include/asm/hwcap.h @@ -106,6 +106,7 @@ #define KERNEL_HWCAP_BTI __khwcap2_feature(BTI) #define KERNEL_HWCAP_MTE __khwcap2_feature(MTE) #define KERNEL_HWCAP_ECV __khwcap2_feature(ECV) +#define KERNEL_HWCAP_AFP __khwcap2_feature(AFP) =20 /* * This yields a mask that user programs can use to figure out what --- a/arch/arm64/include/asm/sysreg.h +++ b/arch/arm64/include/asm/sysreg.h @@ -801,6 +801,7 @@ #endif =20 /* id_aa64mmfr1 */ +#define ID_AA64MMFR1_AFP_SHIFT 44 #define ID_AA64MMFR1_ETS_SHIFT 36 #define ID_AA64MMFR1_TWED_SHIFT 32 #define ID_AA64MMFR1_XNX_SHIFT 28 --- a/arch/arm64/include/uapi/asm/hwcap.h +++ b/arch/arm64/include/uapi/asm/hwcap.h @@ -76,5 +76,6 @@ #define HWCAP2_BTI (1 << 17) #define HWCAP2_MTE (1 << 18) #define HWCAP2_ECV (1 << 19) +#define HWCAP2_AFP (1 << 20) =20 #endif /* _UAPI__ASM_HWCAP_H */ --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -309,6 +309,7 @@ static const struct arm64_ftr_bits ftr_i }; =20 static const struct arm64_ftr_bits ftr_id_aa64mmfr1[] =3D { + ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR1_AFP_= SHIFT, 4, 0), ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR1_ETS_S= HIFT, 4, 0), ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR1_TWED_= SHIFT, 4, 0), ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR1_XNX_S= HIFT, 4, 0), @@ -2262,6 +2263,7 @@ static const struct arm64_cpu_capabiliti HWCAP_CAP(SYS_ID_AA64PFR1_EL1, ID_AA64PFR1_MTE_SHIFT, FTR_UNSIGNED, ID_AA= 64PFR1_MTE, CAP_HWCAP, KERNEL_HWCAP_MTE), #endif /* CONFIG_ARM64_MTE */ HWCAP_CAP(SYS_ID_AA64MMFR0_EL1, ID_AA64MMFR0_ECV_SHIFT, FTR_UNSIGNED, 1, = CAP_HWCAP, KERNEL_HWCAP_ECV), + HWCAP_CAP(SYS_ID_AA64MMFR1_EL1, ID_AA64MMFR1_AFP_SHIFT, FTR_UNSIGNED, 1, = CAP_HWCAP, KERNEL_HWCAP_AFP), {}, }; =20 --- a/arch/arm64/kernel/cpuinfo.c +++ b/arch/arm64/kernel/cpuinfo.c @@ -95,6 +95,7 @@ static const char *const hwcap_str[] =3D { [KERNEL_HWCAP_BTI] =3D "bti", [KERNEL_HWCAP_MTE] =3D "mte", [KERNEL_HWCAP_ECV] =3D "ecv", + [KERNEL_HWCAP_AFP] =3D "afp", }; =20 #ifdef CONFIG_COMPAT From nobody Tue Jun 23 05:03:54 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 256C4C433EF for ; Thu, 10 Mar 2022 14:36:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243751AbiCJOhu (ORCPT ); Thu, 10 Mar 2022 09:37:50 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51990 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245650AbiCJOaq (ORCPT ); Thu, 10 Mar 2022 09:30:46 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 49E77184630; Thu, 10 Mar 2022 06:26:35 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 73BB461CFB; Thu, 10 Mar 2022 14:26:33 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 80C46C340E8; Thu, 10 Mar 2022 14:26:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1646922392; bh=i/sntCpNH5GroLhd61OVs50uRsZvFc/O/fHof6GetNI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=lpsY7IxN4ZIxRCXTp1VFQzeP0tYBWB7qTi5EqiWHrmaaZGlLYzX46sH2vW/iZbw3f NANz0uJQgLCvKUrvKfQdty3VESAVuVMPO/JJBweSoiLmz8ZPaL5Epem8Ne3kQA4SlF uLhE+GwbU3+8O3yuWufggtGEakIykbXIGpI0aeVg= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Joey Gouly , Will Deacon , Jonathan Corbet , Marc Zyngier , Catalin Marinas Subject: [PATCH 5.10 22/58] arm64: cpufeature: add HWCAP for FEAT_RPRES Date: Thu, 10 Mar 2022 15:18:42 +0100 Message-Id: <20220310140813.507388640@linuxfoundation.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220310140812.869208747@linuxfoundation.org> References: <20220310140812.869208747@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Joey Gouly commit 1175011a7d0030d49dc9c10bde36f08f26d0a8ee upstream. Add a new HWCAP to detect the Increased precision of Reciprocal Estimate and Reciprocal Square Root Estimate feature (FEAT_RPRES), introduced in Arm= v8.7. Also expose this to userspace in the ID_AA64ISAR2_EL1 feature register. Signed-off-by: Joey Gouly Cc: Will Deacon Cc: Jonathan Corbet Acked-by: Marc Zyngier Link: https://lore.kernel.org/r/20211210165432.8106-4-joey.gouly@arm.com Signed-off-by: Catalin Marinas Signed-off-by: Greg Kroah-Hartman Tested-by: Bagas Sanjaya Tested-by: Florian Fainelli Tested-by: Fox Chen Tested-by: Guenter Roeck Tested-by: Linux Kernel Functional Testing Tested-by: Pavel Machek (CIP) = =20 Tested-by: Shuah Khan Tested-by: Sudip Mukherjee --- Documentation/arm64/cpu-feature-registers.rst | 8 ++++++++ Documentation/arm64/elf_hwcaps.rst | 4 ++++ arch/arm64/include/asm/hwcap.h | 1 + arch/arm64/include/uapi/asm/hwcap.h | 1 + arch/arm64/kernel/cpufeature.c | 2 ++ arch/arm64/kernel/cpuinfo.c | 1 + 6 files changed, 17 insertions(+) --- a/Documentation/arm64/cpu-feature-registers.rst +++ b/Documentation/arm64/cpu-feature-registers.rst @@ -283,6 +283,14 @@ infrastructure: | AFP | [47-44] | y | +------------------------------+---------+---------+ =20 + 9) ID_AA64ISAR2_EL1 - Instruction set attribute register 2 + + +------------------------------+---------+---------+ + | Name | bits | visible | + +------------------------------+---------+---------+ + | RPRES | [7-4] | y | + +------------------------------+---------+---------+ + =20 Appendix I: Example ------------------- --- a/Documentation/arm64/elf_hwcaps.rst +++ b/Documentation/arm64/elf_hwcaps.rst @@ -253,6 +253,10 @@ HWCAP2_AFP =20 Functionality implied by ID_AA64MFR1_EL1.AFP =3D=3D 0b0001. =20 +HWCAP2_RPRES + + Functionality implied by ID_AA64ISAR2_EL1.RPRES =3D=3D 0b0001. + 4. Unused AT_HWCAP bits ----------------------- =20 --- a/arch/arm64/include/asm/hwcap.h +++ b/arch/arm64/include/asm/hwcap.h @@ -107,6 +107,7 @@ #define KERNEL_HWCAP_MTE __khwcap2_feature(MTE) #define KERNEL_HWCAP_ECV __khwcap2_feature(ECV) #define KERNEL_HWCAP_AFP __khwcap2_feature(AFP) +#define KERNEL_HWCAP_RPRES __khwcap2_feature(RPRES) =20 /* * This yields a mask that user programs can use to figure out what --- a/arch/arm64/include/uapi/asm/hwcap.h +++ b/arch/arm64/include/uapi/asm/hwcap.h @@ -77,5 +77,6 @@ #define HWCAP2_MTE (1 << 18) #define HWCAP2_ECV (1 << 19) #define HWCAP2_AFP (1 << 20) +#define HWCAP2_RPRES (1 << 21) =20 #endif /* _UAPI__ASM_HWCAP_H */ --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -206,6 +206,7 @@ static const struct arm64_ftr_bits ftr_i }; =20 static const struct arm64_ftr_bits ftr_id_aa64isar2[] =3D { + ARM64_FTR_BITS(FTR_VISIBLE, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64ISAR2_R= PRES_SHIFT, 4, 0), ARM64_FTR_END, }; =20 @@ -2264,6 +2265,7 @@ static const struct arm64_cpu_capabiliti #endif /* CONFIG_ARM64_MTE */ HWCAP_CAP(SYS_ID_AA64MMFR0_EL1, ID_AA64MMFR0_ECV_SHIFT, FTR_UNSIGNED, 1, = CAP_HWCAP, KERNEL_HWCAP_ECV), HWCAP_CAP(SYS_ID_AA64MMFR1_EL1, ID_AA64MMFR1_AFP_SHIFT, FTR_UNSIGNED, 1, = CAP_HWCAP, KERNEL_HWCAP_AFP), + HWCAP_CAP(SYS_ID_AA64ISAR2_EL1, ID_AA64ISAR2_RPRES_SHIFT, FTR_UNSIGNED, 1= , CAP_HWCAP, KERNEL_HWCAP_RPRES), {}, }; =20 --- a/arch/arm64/kernel/cpuinfo.c +++ b/arch/arm64/kernel/cpuinfo.c @@ -96,6 +96,7 @@ static const char *const hwcap_str[] =3D { [KERNEL_HWCAP_MTE] =3D "mte", [KERNEL_HWCAP_ECV] =3D "ecv", [KERNEL_HWCAP_AFP] =3D "afp", + [KERNEL_HWCAP_RPRES] =3D "rpres", }; =20 #ifdef CONFIG_COMPAT From nobody Tue Jun 23 05:03:54 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AA1D6C4321E for ; Thu, 10 Mar 2022 14:42:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S245289AbiCJOnK (ORCPT ); Thu, 10 Mar 2022 09:43:10 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50134 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1344027AbiCJObh (ORCPT ); Thu, 10 Mar 2022 09:31:37 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E4310BECFA; Thu, 10 Mar 2022 06:29:27 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 41ED5B81E9E; Thu, 10 Mar 2022 14:24:26 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 85ACEC340EB; Thu, 10 Mar 2022 14:24:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1646922264; bh=KehoIr6yz70C9INH4cXgl326hlvCsnsKZSidzApz9l8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=tFF2dmvw/ENX1mwHPBQLDyInG/X7UwAhB0dYvNzvZFaNHuIap/+5luek9lVQrBvIs CC95FiDUugcJLWaDt+z1bWV1yIYcogsNz/Co7qJGJvaFPscJi0XAx4mOSxAVFri5/Q eyv+5Y7F8mzBU/4dYPTjvJulvmYirK95+pEapDW8= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, "Russell King (Oracle)" , Catalin Marinas , James Morse Subject: [PATCH 5.10 23/58] arm64: entry.S: Add ventry overflow sanity checks Date: Thu, 10 Mar 2022 15:18:43 +0100 Message-Id: <20220310140813.536582570@linuxfoundation.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220310140812.869208747@linuxfoundation.org> References: <20220310140812.869208747@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: James Morse commit 4330e2c5c04c27bebf89d34e0bc14e6943413067 upstream. Subsequent patches add even more code to the ventry slots. Ensure kernels that overflow a ventry slot don't get built. Reviewed-by: Russell King (Oracle) Reviewed-by: Catalin Marinas Signed-off-by: James Morse Signed-off-by: Greg Kroah-Hartman Tested-by: Bagas Sanjaya Tested-by: Florian Fainelli Tested-by: Fox Chen Tested-by: Guenter Roeck Tested-by: Linux Kernel Functional Testing Tested-by: Pavel Machek (CIP) = =20 Tested-by: Shuah Khan Tested-by: Sudip Mukherjee --- arch/arm64/kernel/entry.S | 3 +++ 1 file changed, 3 insertions(+) --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -62,6 +62,7 @@ =20 .macro kernel_ventry, el, label, regsize =3D 64 .align 7 +.Lventry_start\@: #ifdef CONFIG_UNMAP_KERNEL_AT_EL0 .if \el =3D=3D 0 alternative_if ARM64_UNMAP_KERNEL_AT_EL0 @@ -120,6 +121,7 @@ alternative_else_nop_endif mrs x0, tpidrro_el0 #endif b el\()\el\()_\label +.org .Lventry_start\@ + 128 // Did we overflow the ventry slot? .endm =20 .macro tramp_alias, dst, sym @@ -832,6 +834,7 @@ alternative_else_nop_endif add x30, x30, #(1b - tramp_vectors) isb ret +.org 1b + 128 // Did we overflow the ventry slot? .endm =20 .macro tramp_exit, regsize =3D 64 From nobody Tue Jun 23 05:03:54 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9A83FC433EF for ; Thu, 10 Mar 2022 14:34:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234654AbiCJOfv (ORCPT ); Thu, 10 Mar 2022 09:35:51 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49556 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S244709AbiCJO3Z (ORCPT ); Thu, 10 Mar 2022 09:29:25 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DB3F0DB86B; Thu, 10 Mar 2022 06:24:30 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id ADF3761D38; Thu, 10 Mar 2022 14:24:28 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id B2DE7C340E8; Thu, 10 Mar 2022 14:24:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1646922268; bh=eNC2LoRSShxShPBQOu9Ekx31Np/dRyimoxlehhvsvm4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=KbmaiZH6UITUMjQOaYaOnblJpZD9Ur+w86WTviJMSGcflIMm9TS3St24DCm5MuCbs itBMdHN4cCWtHmB72A5QcJfXAfeINgbruuk93VQ0Lc4y5WsoM0UJfzbMJGP85IzClD SaC5Ji+r0TEc/S1SuuilI8OMSojqU/7+jGo+hx8o= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, "Russell King (Oracle)" , Catalin Marinas , James Morse Subject: [PATCH 5.10 24/58] arm64: spectre: Rename spectre_v4_patch_fw_mitigation_conduit Date: Thu, 10 Mar 2022 15:18:44 +0100 Message-Id: <20220310140813.565133100@linuxfoundation.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220310140812.869208747@linuxfoundation.org> References: <20220310140812.869208747@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: James Morse commit 1b33d4860deaecf1d8eec3061b7e7ed7ab0bae8d upstream. The spectre-v4 sequence includes an SMC from the assembly entry code. spectre_v4_patch_fw_mitigation_conduit is the patching callback that generates an HVC or SMC depending on the SMCCC conduit type. As this isn't specific to spectre-v4, rename it smccc_patch_fw_mitigation_conduit so it can be re-used. Reviewed-by: Russell King (Oracle) Reviewed-by: Catalin Marinas Signed-off-by: James Morse Signed-off-by: Greg Kroah-Hartman Tested-by: Bagas Sanjaya Tested-by: Florian Fainelli Tested-by: Fox Chen Tested-by: Guenter Roeck Tested-by: Linux Kernel Functional Testing Tested-by: Pavel Machek (CIP) = =20 Tested-by: Shuah Khan Tested-by: Sudip Mukherjee --- arch/arm64/kernel/entry.S | 2 +- arch/arm64/kernel/proton-pack.c | 6 +++--- 2 files changed, 4 insertions(+), 4 deletions(-) --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -143,7 +143,7 @@ alternative_cb_end tbnz \tmp2, #TIF_SSBD, .L__asm_ssbd_skip\@ mov w0, #ARM_SMCCC_ARCH_WORKAROUND_2 mov w1, #\state -alternative_cb spectre_v4_patch_fw_mitigation_conduit +alternative_cb smccc_patch_fw_mitigation_conduit nop // Patched to SMC/HVC #0 alternative_cb_end .L__asm_ssbd_skip\@: --- a/arch/arm64/kernel/proton-pack.c +++ b/arch/arm64/kernel/proton-pack.c @@ -571,9 +571,9 @@ void __init spectre_v4_patch_fw_mitigati * Patch a NOP in the Spectre-v4 mitigation code with an SMC/HVC instructi= on * to call into firmware to adjust the mitigation state. */ -void __init spectre_v4_patch_fw_mitigation_conduit(struct alt_instr *alt, - __le32 *origptr, - __le32 *updptr, int nr_inst) +void __init smccc_patch_fw_mitigation_conduit(struct alt_instr *alt, + __le32 *origptr, + __le32 *updptr, int nr_inst) { u32 insn; From nobody Tue Jun 23 05:03:54 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 87C14C3526E for ; Thu, 10 Mar 2022 14:42:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344777AbiCJOlL (ORCPT ); Thu, 10 Mar 2022 09:41:11 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49800 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S244733AbiCJO30 (ORCPT ); Thu, 10 Mar 2022 09:29:26 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 628D9DCE00; Thu, 10 Mar 2022 06:24:34 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 81F7BB82544; Thu, 10 Mar 2022 14:24:32 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id B31A4C340E8; Thu, 10 Mar 2022 14:24:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1646922271; bh=PD8YDBuRBOII3aVAAcFtCX2cx+HFLHuZPpAQYmq1x+Y=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=TtiwXKaqZ+uZBDt91hUaHlJDDA90eMjUYS2e7KJpX5D7py3Hn675r0Y4F4N2WYuZK Hr3fkM7qg6lCDcW4RBiFFVlkq5f4mIqBowekXE0d+uBpBK74h5QNwWCsEszZg7t/Dk bn2qT9mxzZjb1gkx10I+k/mQtVT72NCqzKCgpf5c= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, "Russell King (Oracle)" , Catalin Marinas , James Morse Subject: [PATCH 5.10 25/58] arm64: entry: Make the trampoline cleanup optional Date: Thu, 10 Mar 2022 15:18:45 +0100 Message-Id: <20220310140813.593477273@linuxfoundation.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220310140812.869208747@linuxfoundation.org> References: <20220310140812.869208747@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: James Morse commit d739da1694a0eaef0358a42b76904b611539b77b upstream. Subsequent patches will add additional sets of vectors that use the same tricks as the kpti vectors to reach the full-fat vectors. The full-fat vectors contain some cleanup for kpti that is patched in by alternatives when kpti is in use. Once there are additional vectors, the cleanup will be needed in more cases. But on big/little systems, the cleanup would be harmful if no trampoline vector were in use. Instead of forcing CPUs that don't need a trampoline vector to use one, make the trampoline cleanup optional. Entry at the top of the vectors will skip the cleanup. The trampoline vectors can then skip the first instruction, triggering the cleanup to run. Reviewed-by: Russell King (Oracle) Reviewed-by: Catalin Marinas Signed-off-by: James Morse Signed-off-by: Greg Kroah-Hartman Tested-by: Bagas Sanjaya Tested-by: Florian Fainelli Tested-by: Fox Chen Tested-by: Guenter Roeck Tested-by: Linux Kernel Functional Testing Tested-by: Pavel Machek (CIP) = =20 Tested-by: Shuah Khan Tested-by: Sudip Mukherjee --- arch/arm64/kernel/entry.S | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -65,14 +65,18 @@ .Lventry_start\@: #ifdef CONFIG_UNMAP_KERNEL_AT_EL0 .if \el =3D=3D 0 -alternative_if ARM64_UNMAP_KERNEL_AT_EL0 + /* + * This must be the first instruction of the EL0 vector entries. It is + * skipped by the trampoline vectors, to trigger the cleanup. + */ + b .Lskip_tramp_vectors_cleanup\@ .if \regsize =3D=3D 64 mrs x30, tpidrro_el0 msr tpidrro_el0, xzr .else mov x30, xzr .endif -alternative_else_nop_endif +.Lskip_tramp_vectors_cleanup\@: .endif #endif =20 @@ -831,7 +835,7 @@ alternative_if_not ARM64_WORKAROUND_CAVI prfm plil1strm, [x30, #(1b - tramp_vectors)] alternative_else_nop_endif msr vbar_el1, x30 - add x30, x30, #(1b - tramp_vectors) + add x30, x30, #(1b - tramp_vectors + 4) isb ret .org 1b + 128 // Did we overflow the ventry slot? From nobody Tue Jun 23 05:03:54 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 339CEC433F5 for ; Thu, 10 Mar 2022 14:34:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244315AbiCJOfp (ORCPT ); Thu, 10 Mar 2022 09:35:45 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49796 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S244746AbiCJO31 (ORCPT ); Thu, 10 Mar 2022 09:29:27 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9EBB0D64E2; Thu, 10 Mar 2022 06:24:34 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 7FFFD61CF0; Thu, 10 Mar 2022 14:24:34 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8F730C340E8; Thu, 10 Mar 2022 14:24:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1646922273; bh=A6oXWWdywzHZKFb4f9k7sQK6E39VRSRYq5LVYyv73hs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=yf0DiOWE32F6xRI62c/kBy0sho02GL+VjYWvF8Fy3wKgXymHrLvNRl5+uO9tETjqV f+JouvKR3v1Uc1L+s9qpTCCQGQAX4sKwQNLwJWisKfr4yeuTBnnTP0D9PsDoJDecOs rQq6fgAsL8HpdCfOmAHvl/UqDJGPPLl71jofcRmg= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, "Russell King (Oracle)" , Catalin Marinas , James Morse Subject: [PATCH 5.10 26/58] arm64: entry: Free up another register on kptis tramp_exit path Date: Thu, 10 Mar 2022 15:18:46 +0100 Message-Id: <20220310140813.622219780@linuxfoundation.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220310140812.869208747@linuxfoundation.org> References: <20220310140812.869208747@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: James Morse commit 03aff3a77a58b5b52a77e00537a42090ad57b80b upstream. Kpti stashes x30 in far_el1 while it uses x30 for all its work. Making the vectors a per-cpu data structure will require a second register. Allow tramp_exit two registers before it unmaps the kernel, by leaving x30 on the stack, and stashing x29 in far_el1. Reviewed-by: Russell King (Oracle) Reviewed-by: Catalin Marinas Signed-off-by: James Morse Signed-off-by: Greg Kroah-Hartman Tested-by: Bagas Sanjaya Tested-by: Florian Fainelli Tested-by: Fox Chen Tested-by: Guenter Roeck Tested-by: Linux Kernel Functional Testing Tested-by: Pavel Machek (CIP) = =20 Tested-by: Shuah Khan Tested-by: Sudip Mukherjee --- arch/arm64/kernel/entry.S | 19 +++++++++++++------ 1 file changed, 13 insertions(+), 6 deletions(-) --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -357,14 +357,16 @@ alternative_else_nop_endif ldp x24, x25, [sp, #16 * 12] ldp x26, x27, [sp, #16 * 13] ldp x28, x29, [sp, #16 * 14] - ldr lr, [sp, #S_LR] - add sp, sp, #S_FRAME_SIZE // restore sp =20 .if \el =3D=3D 0 -alternative_insn eret, nop, ARM64_UNMAP_KERNEL_AT_EL0 +alternative_if_not ARM64_UNMAP_KERNEL_AT_EL0 + ldr lr, [sp, #S_LR] + add sp, sp, #S_FRAME_SIZE // restore sp + eret +alternative_else_nop_endif #ifdef CONFIG_UNMAP_KERNEL_AT_EL0 bne 4f - msr far_el1, x30 + msr far_el1, x29 tramp_alias x30, tramp_exit_native br x30 4: @@ -372,6 +374,9 @@ alternative_insn eret, nop, ARM64_UNMAP_ br x30 #endif .else + ldr lr, [sp, #S_LR] + add sp, sp, #S_FRAME_SIZE // restore sp + /* Ensure any device/NC reads complete */ alternative_insn nop, "dmb sy", ARM64_WORKAROUND_1508412 =20 @@ -844,10 +849,12 @@ alternative_else_nop_endif .macro tramp_exit, regsize =3D 64 adr x30, tramp_vectors msr vbar_el1, x30 - tramp_unmap_kernel x30 + ldr lr, [sp, #S_LR] + tramp_unmap_kernel x29 .if \regsize =3D=3D 64 - mrs x30, far_el1 + mrs x29, far_el1 .endif + add sp, sp, #S_FRAME_SIZE // restore sp eret sb .endm From nobody Tue Jun 23 05:03:54 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 918E1C43219 for ; Thu, 10 Mar 2022 14:42:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344946AbiCJOlZ (ORCPT ); Thu, 10 Mar 2022 09:41:25 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50098 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245008AbiCJO3o (ORCPT ); Thu, 10 Mar 2022 09:29:44 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8609A158EBD; Thu, 10 Mar 2022 06:25:18 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 45C95B825F3; Thu, 10 Mar 2022 14:24:38 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 81637C36AE7; Thu, 10 Mar 2022 14:24:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1646922277; bh=WxwhphBYDTEtoXiz1Wh3kUHuvq885+/TfYmzRuOxkfg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=B/vbWSW52UmZlc+wh3djHRhMxcJtJNPBxAQRpwtY5v8h6Z1cmhh8j5tDeeyJL2Uoc QNqwazoQhuc6i8OT6cG8lUuGSRSKyopywxAznNuQcGpiNAYWERKibV5KR1qbJ71AlZ l5DmGr+8fOVHaztOTttHqWgOlq0rD3Cj+ZB5Uwl8= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, "Russell King (Oracle)" , Catalin Marinas , James Morse Subject: [PATCH 5.10 27/58] arm64: entry: Move the trampoline data page before the text page Date: Thu, 10 Mar 2022 15:18:47 +0100 Message-Id: <20220310140813.651015986@linuxfoundation.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220310140812.869208747@linuxfoundation.org> References: <20220310140812.869208747@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: James Morse commit c091fb6ae059cda563b2a4d93fdbc548ef34e1d6 upstream. The trampoline code has a data page that holds the address of the vectors, which is unmapped when running in user-space. This ensures that with CONFIG_RANDOMIZE_BASE, the randomised address of the kernel can't be discovered until after the kernel has been mapped. If the trampoline text page is extended to include multiple sets of vectors, it will be larger than a single page, making it tricky to find the data page without knowing the size of the trampoline text pages, which will vary with PAGE_SIZE. Move the data page to appear before the text page. This allows the data page to be found without knowing the size of the trampoline text pages. 'tramp_vectors' is used to refer to the beginning of the .entry.tramp.text section, do that explicitly. Reviewed-by: Russell King (Oracle) Reviewed-by: Catalin Marinas Signed-off-by: James Morse Signed-off-by: Greg Kroah-Hartman Tested-by: Bagas Sanjaya Tested-by: Florian Fainelli Tested-by: Fox Chen Tested-by: Guenter Roeck Tested-by: Linux Kernel Functional Testing Tested-by: Pavel Machek (CIP) = =20 Tested-by: Shuah Khan Tested-by: Sudip Mukherjee --- arch/arm64/include/asm/fixmap.h | 2 +- arch/arm64/kernel/entry.S | 9 +++++++-- 2 files changed, 8 insertions(+), 3 deletions(-) --- a/arch/arm64/include/asm/fixmap.h +++ b/arch/arm64/include/asm/fixmap.h @@ -62,8 +62,8 @@ enum fixed_addresses { #endif /* CONFIG_ACPI_APEI_GHES */ =20 #ifdef CONFIG_UNMAP_KERNEL_AT_EL0 - FIX_ENTRY_TRAMP_DATA, FIX_ENTRY_TRAMP_TEXT, + FIX_ENTRY_TRAMP_DATA, #define TRAMP_VALIAS (__fix_to_virt(FIX_ENTRY_TRAMP_TEXT)) #endif /* CONFIG_UNMAP_KERNEL_AT_EL0 */ __end_of_permanent_fixed_addresses, --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -814,6 +814,11 @@ alternative_else_nop_endif */ .endm =20 + .macro tramp_data_page dst + adr \dst, .entry.tramp.text + sub \dst, \dst, PAGE_SIZE + .endm + .macro tramp_ventry, regsize =3D 64 .align 7 1: @@ -830,7 +835,7 @@ alternative_else_nop_endif 2: tramp_map_kernel x30 #ifdef CONFIG_RANDOMIZE_BASE - adr x30, tramp_vectors + PAGE_SIZE + tramp_data_page x30 alternative_insn isb, nop, ARM64_WORKAROUND_QCOM_FALKOR_E1003 ldr x30, [x30] #else @@ -984,7 +989,7 @@ SYM_CODE_START(__sdei_asm_entry_trampoli 1: str x4, [x1, #(SDEI_EVENT_INTREGS + S_ORIG_ADDR_LIMIT)] =20 #ifdef CONFIG_RANDOMIZE_BASE - adr x4, tramp_vectors + PAGE_SIZE + tramp_data_page x4 add x4, x4, #:lo12:__sdei_asm_trampoline_next_handler ldr x4, [x4] #else From nobody Tue Jun 23 05:03:54 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D2722C433EF for ; Thu, 10 Mar 2022 14:35:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238270AbiCJOgZ (ORCPT ); Thu, 10 Mar 2022 09:36:25 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49798 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S244828AbiCJO3d (ORCPT ); Thu, 10 Mar 2022 09:29:33 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D04B71405C1; Thu, 10 Mar 2022 06:24:44 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id A005BB82683; Thu, 10 Mar 2022 14:24:41 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1A544C340E8; Thu, 10 Mar 2022 14:24:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1646922280; bh=4V4EDjYNF4xqOm3IhKKiac1VO/Sb5F21Yw8VdkksPQ4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=w3saS5Xj6cZUEJmIogVuASiA4vDsDsFN4TO3t8d5GL0HLAq+ymny/ykwUU6u/gsYC KhlPP33jdOAavEet9LKbsR8NQbtx6R7GNbJPYhg+hQ+CTt2dAqN4EiHOHbC4B1bqLu 7mt3pntsW0983THvTRDBKThOrMiadtbYJmKzaj34= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, "Russell King (Oracle)" , Catalin Marinas , James Morse Subject: [PATCH 5.10 28/58] arm64: entry: Allow tramp_alias to access symbols after the 4K boundary Date: Thu, 10 Mar 2022 15:18:48 +0100 Message-Id: <20220310140813.678468825@linuxfoundation.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220310140812.869208747@linuxfoundation.org> References: <20220310140812.869208747@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: James Morse commit 6c5bf79b69f911560fbf82214c0971af6e58e682 upstream. Systems using kpti enter and exit the kernel through a trampoline mapping that is always mapped, even when the kernel is not. tramp_valias is a macro to find the address of a symbol in the trampoline mapping. Adding extra sets of vectors will expand the size of the entry.tramp.text section to beyond 4K. tramp_valias will be unable to generate addresses for symbols beyond 4K as it uses the 12 bit immediate of the add instruction. As there are now two registers available when tramp_alias is called, use the extra register to avoid the 4K limit of the 12 bit immediate. Reviewed-by: Russell King (Oracle) Reviewed-by: Catalin Marinas Signed-off-by: James Morse Signed-off-by: Greg Kroah-Hartman Tested-by: Bagas Sanjaya Tested-by: Florian Fainelli Tested-by: Fox Chen Tested-by: Guenter Roeck Tested-by: Linux Kernel Functional Testing Tested-by: Pavel Machek (CIP) = =20 Tested-by: Shuah Khan Tested-by: Sudip Mukherjee --- arch/arm64/kernel/entry.S | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -128,9 +128,12 @@ .org .Lventry_start\@ + 128 // Did we overflow the ventry slot? .endm =20 - .macro tramp_alias, dst, sym + .macro tramp_alias, dst, sym, tmp mov_q \dst, TRAMP_VALIAS - add \dst, \dst, #(\sym - .entry.tramp.text) + adr_l \tmp, \sym + add \dst, \dst, \tmp + adr_l \tmp, .entry.tramp.text + sub \dst, \dst, \tmp .endm =20 /* @@ -367,10 +370,10 @@ alternative_else_nop_endif #ifdef CONFIG_UNMAP_KERNEL_AT_EL0 bne 4f msr far_el1, x29 - tramp_alias x30, tramp_exit_native + tramp_alias x30, tramp_exit_native, x29 br x30 4: - tramp_alias x30, tramp_exit_compat + tramp_alias x30, tramp_exit_compat, x29 br x30 #endif .else @@ -1131,7 +1134,7 @@ alternative_if_not ARM64_UNMAP_KERNEL_AT alternative_else_nop_endif =20 #ifdef CONFIG_UNMAP_KERNEL_AT_EL0 - tramp_alias dst=3Dx5, sym=3D__sdei_asm_exit_trampoline + tramp_alias dst=3Dx5, sym=3D__sdei_asm_exit_trampoline, tmp=3Dx3 br x5 #endif SYM_CODE_END(__sdei_asm_handler) From nobody Tue Jun 23 05:03:54 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1800DC433F5 for ; Thu, 10 Mar 2022 14:35:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244272AbiCJOgC (ORCPT ); Thu, 10 Mar 2022 09:36:02 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50108 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S244836AbiCJO3d (ORCPT ); Thu, 10 Mar 2022 09:29:33 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8CC7DDCE0D; Thu, 10 Mar 2022 06:24:46 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id DFD89B82544; Thu, 10 Mar 2022 14:24:44 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 58DAFC340EB; Thu, 10 Mar 2022 14:24:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1646922283; bh=Z7pnDDP4PixJazKkW3aaIvDxwShxt7Bmhfqwcoma7Ow=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=YH4ZFcpE5T63uJpCoScnVcnVck/Hu7uigm/BzJBvBP2clNE4LAfMVTPARm4gTcjEU Gha8JP5PvSsDUAU9rtHVta6bqjufbrnu4tWE8nYXuERRULRBgsYFP5MAR8aYr/nKm6 8M6yAuD7g5sWTllYHfyHzzzVYOVmMwqjCN8WLJqg= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, "Russell King (Oracle)" , Catalin Marinas , James Morse Subject: [PATCH 5.10 29/58] arm64: entry: Dont assume tramp_vectors is the start of the vectors Date: Thu, 10 Mar 2022 15:18:49 +0100 Message-Id: <20220310140813.706610992@linuxfoundation.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220310140812.869208747@linuxfoundation.org> References: <20220310140812.869208747@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: James Morse commit ed50da7764535f1e24432ded289974f2bf2b0c5a upstream. The tramp_ventry macro uses tramp_vectors as the address of the vectors when calculating which ventry in the 'full fat' vectors to branch to. While there is one set of tramp_vectors, this will be true. Adding multiple sets of vectors will break this assumption. Move the generation of the vectors to a macro, and pass the start of the vectors as an argument to tramp_ventry. Reviewed-by: Russell King (Oracle) Reviewed-by: Catalin Marinas Signed-off-by: James Morse Signed-off-by: Greg Kroah-Hartman Tested-by: Bagas Sanjaya Tested-by: Florian Fainelli Tested-by: Fox Chen Tested-by: Guenter Roeck Tested-by: Linux Kernel Functional Testing Tested-by: Pavel Machek (CIP) = =20 Tested-by: Shuah Khan Tested-by: Sudip Mukherjee --- arch/arm64/kernel/entry.S | 30 ++++++++++++++++-------------- 1 file changed, 16 insertions(+), 14 deletions(-) --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -822,7 +822,7 @@ alternative_else_nop_endif sub \dst, \dst, PAGE_SIZE .endm =20 - .macro tramp_ventry, regsize =3D 64 + .macro tramp_ventry, vector_start, regsize .align 7 1: .if \regsize =3D=3D 64 @@ -845,10 +845,10 @@ alternative_insn isb, nop, ARM64_WORKARO ldr x30, =3Dvectors #endif alternative_if_not ARM64_WORKAROUND_CAVIUM_TX2_219_PRFM - prfm plil1strm, [x30, #(1b - tramp_vectors)] + prfm plil1strm, [x30, #(1b - \vector_start)] alternative_else_nop_endif msr vbar_el1, x30 - add x30, x30, #(1b - tramp_vectors + 4) + add x30, x30, #(1b - \vector_start + 4) isb ret .org 1b + 128 // Did we overflow the ventry slot? @@ -867,19 +867,21 @@ alternative_else_nop_endif sb .endm =20 - .align 11 -SYM_CODE_START_NOALIGN(tramp_vectors) + .macro generate_tramp_vector +.Lvector_start\@: .space 0x400 =20 - tramp_ventry - tramp_ventry - tramp_ventry - tramp_ventry - - tramp_ventry 32 - tramp_ventry 32 - tramp_ventry 32 - tramp_ventry 32 + .rept 4 + tramp_ventry .Lvector_start\@, 64 + .endr + .rept 4 + tramp_ventry .Lvector_start\@, 32 + .endr + .endm + + .align 11 +SYM_CODE_START_NOALIGN(tramp_vectors) + generate_tramp_vector SYM_CODE_END(tramp_vectors) =20 SYM_CODE_START(tramp_exit_native) From nobody Tue Jun 23 05:03:54 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 120F6C433F5 for ; Thu, 10 Mar 2022 14:35:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236841AbiCJOgo (ORCPT ); Thu, 10 Mar 2022 09:36:44 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48198 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245076AbiCJOaG (ORCPT ); Thu, 10 Mar 2022 09:30:06 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EE8AA160FE5; Thu, 10 Mar 2022 06:25:24 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 7127761CF0; Thu, 10 Mar 2022 14:24:47 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 6EC78C340E8; Thu, 10 Mar 2022 14:24:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1646922286; bh=wl6HBnhpwKS+JR4VuT7rhMipb3Fa6aCKKhyiO/sl5rg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Ag0z5Jrx0YoPzdFOQhnYopl2t3PQMyhiJVnQYLbga+fNuf1T49mGV3/nNKXewRUUQ CpxrT7I0J2txEyQ8LdKiAQSwFM64FaFqZsfjTeHxxSJrjEbHWdz5rz1jqgYv0KeyVZ SyxJa76f+Wy2QSW3LWzP7GGPPrnVxLxiHfvbaReQ= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, "Russell King (Oracle)" , Catalin Marinas , James Morse Subject: [PATCH 5.10 30/58] arm64: entry: Move trampoline macros out of ifdefd section Date: Thu, 10 Mar 2022 15:18:50 +0100 Message-Id: <20220310140813.734626881@linuxfoundation.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220310140812.869208747@linuxfoundation.org> References: <20220310140812.869208747@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: James Morse commit 13d7a08352a83ef2252aeb464a5e08dfc06b5dfd upstream. The macros for building the kpti trampoline are all behind CONFIG_UNMAP_KERNEL_AT_EL0, and in a region that outputs to the .entry.tramp.text section. Move the macros out so they can be used to generate other kinds of trampoline. Only the symbols need to be guarded by CONFIG_UNMAP_KERNEL_AT_EL0 and appear in the .entry.tramp.text section. Reviewed-by: Russell King (Oracle) Reviewed-by: Catalin Marinas Signed-off-by: James Morse Signed-off-by: Greg Kroah-Hartman Tested-by: Bagas Sanjaya Tested-by: Florian Fainelli Tested-by: Fox Chen Tested-by: Guenter Roeck Tested-by: Linux Kernel Functional Testing Tested-by: Pavel Machek (CIP) = =20 Tested-by: Shuah Khan Tested-by: Sudip Mukherjee --- arch/arm64/kernel/entry.S | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -778,12 +778,6 @@ SYM_CODE_END(ret_to_user) =20 .popsection // .entry.text =20 -#ifdef CONFIG_UNMAP_KERNEL_AT_EL0 -/* - * Exception vectors trampoline. - */ - .pushsection ".entry.tramp.text", "ax" - // Move from tramp_pg_dir to swapper_pg_dir .macro tramp_map_kernel, tmp mrs \tmp, ttbr1_el1 @@ -879,6 +873,11 @@ alternative_else_nop_endif .endr .endm =20 +#ifdef CONFIG_UNMAP_KERNEL_AT_EL0 +/* + * Exception vectors trampoline. + */ + .pushsection ".entry.tramp.text", "ax" .align 11 SYM_CODE_START_NOALIGN(tramp_vectors) generate_tramp_vector From nobody Tue Jun 23 05:03:54 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BFBA7C433FE for ; Thu, 10 Mar 2022 14:36:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S242278AbiCJOhN (ORCPT ); Thu, 10 Mar 2022 09:37:13 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49734 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245208AbiCJOaO (ORCPT ); Thu, 10 Mar 2022 09:30:14 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 50D5516BCF1; Thu, 10 Mar 2022 06:25:39 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 70733B8267A; Thu, 10 Mar 2022 14:24:51 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id DDFD5C340E8; Thu, 10 Mar 2022 14:24:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1646922290; bh=9h2i3R524QpBTyblkHBTWgTjpMg6HyF7bRA77J2gV9o=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ry+LvXggiyv3rz4M6DQwC2muDpO41x5Sb/hUv1gcndVlmSZgPQLIILsdI7PjEMdjO bMPoCXmrtzc26spNAg7rni0jXyJj1hpMXFE1xch8qaXwmqBXXrwcVYpZhdhiSEluWW RzRlBYIuzS7deWCjQoQzgm4gwGQI9FML/KBylHmA= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, "Russell King (Oracle)" , Catalin Marinas , James Morse Subject: [PATCH 5.10 31/58] arm64: entry: Make the kpti trampolines kpti sequence optional Date: Thu, 10 Mar 2022 15:18:51 +0100 Message-Id: <20220310140813.762278227@linuxfoundation.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220310140812.869208747@linuxfoundation.org> References: <20220310140812.869208747@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: James Morse commit c47e4d04ba0f1ea17353d85d45f611277507e07a upstream. Spectre-BHB needs to add sequences to the vectors. Having one global set of vectors is a problem for big/little systems where the sequence is costly on cpus that are not vulnerable. Making the vectors per-cpu in the style of KVM's bh_harden_hyp_vecs requires the vectors to be generated by macros. Make the kpti re-mapping of the kernel optional, so the macros can be used without kpti. Reviewed-by: Russell King (Oracle) Reviewed-by: Catalin Marinas Signed-off-by: James Morse Signed-off-by: Greg Kroah-Hartman Tested-by: Bagas Sanjaya Tested-by: Florian Fainelli Tested-by: Fox Chen Tested-by: Guenter Roeck Tested-by: Linux Kernel Functional Testing Tested-by: Pavel Machek (CIP) = =20 Tested-by: Shuah Khan Tested-by: Sudip Mukherjee --- arch/arm64/kernel/entry.S | 18 ++++++++++++------ 1 file changed, 12 insertions(+), 6 deletions(-) --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -816,9 +816,10 @@ alternative_else_nop_endif sub \dst, \dst, PAGE_SIZE .endm =20 - .macro tramp_ventry, vector_start, regsize + .macro tramp_ventry, vector_start, regsize, kpti .align 7 1: + .if \kpti =3D=3D 1 .if \regsize =3D=3D 64 msr tpidrro_el0, x30 // Restored in kernel_ventry .endif @@ -841,9 +842,14 @@ alternative_insn isb, nop, ARM64_WORKARO alternative_if_not ARM64_WORKAROUND_CAVIUM_TX2_219_PRFM prfm plil1strm, [x30, #(1b - \vector_start)] alternative_else_nop_endif + msr vbar_el1, x30 - add x30, x30, #(1b - \vector_start + 4) isb + .else + ldr x30, =3Dvectors + .endif // \kpti =3D=3D 1 + + add x30, x30, #(1b - \vector_start + 4) ret .org 1b + 128 // Did we overflow the ventry slot? .endm @@ -861,15 +867,15 @@ alternative_else_nop_endif sb .endm =20 - .macro generate_tramp_vector + .macro generate_tramp_vector, kpti .Lvector_start\@: .space 0x400 =20 .rept 4 - tramp_ventry .Lvector_start\@, 64 + tramp_ventry .Lvector_start\@, 64, \kpti .endr .rept 4 - tramp_ventry .Lvector_start\@, 32 + tramp_ventry .Lvector_start\@, 32, \kpti .endr .endm =20 @@ -880,7 +886,7 @@ alternative_else_nop_endif .pushsection ".entry.tramp.text", "ax" .align 11 SYM_CODE_START_NOALIGN(tramp_vectors) - generate_tramp_vector + generate_tramp_vector kpti=3D1 SYM_CODE_END(tramp_vectors) =20 SYM_CODE_START(tramp_exit_native) From nobody Tue Jun 23 05:03:54 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8084CC4708B for ; Thu, 10 Mar 2022 14:42:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344842AbiCJOlQ (ORCPT ); Thu, 10 Mar 2022 09:41:16 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49798 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S244937AbiCJO3k (ORCPT ); Thu, 10 Mar 2022 09:29:40 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2CF5B1567A1; Thu, 10 Mar 2022 06:25:04 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id C231661B63; Thu, 10 Mar 2022 14:24:53 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id CE210C36AE2; Thu, 10 Mar 2022 14:24:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1646922293; bh=yX+SwqC0qS9bi3l69qLS3vC4ItdYeNobtBQQvskQajI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ycqw30zahoYtQphmb94GQvAoQXKrclBlxvCuD9yIeUCERtCRLdAqtvWBKTp/Ujux7 UI8R22vAR46JLcDwZpwV3IetUC+DinP2y32isaGRuxB7lCvY1dXztcfjpIcBKUNPIo WtUhAcQe/fwJLD3Wfm6dG9y96iCHH1rGkJvZi9uk= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Catalin Marinas , James Morse Subject: [PATCH 5.10 32/58] arm64: entry: Allow the trampoline text to occupy multiple pages Date: Thu, 10 Mar 2022 15:18:52 +0100 Message-Id: <20220310140813.789872383@linuxfoundation.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220310140812.869208747@linuxfoundation.org> References: <20220310140812.869208747@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: James Morse commit a9c406e6462ff14956d690de7bbe5131a5677dc9 upstream. Adding a second set of vectors to .entry.tramp.text will make it larger than a single 4K page. Allow the trampoline text to occupy up to three pages by adding two more fixmap slots. Previous changes to tramp_valias allowed it to reach beyond a single page. Reviewed-by: Catalin Marinas Signed-off-by: James Morse Signed-off-by: Greg Kroah-Hartman Tested-by: Bagas Sanjaya Tested-by: Florian Fainelli Tested-by: Fox Chen Tested-by: Guenter Roeck Tested-by: Linux Kernel Functional Testing Tested-by: Pavel Machek (CIP) = =20 Tested-by: Shuah Khan Tested-by: Sudip Mukherjee --- arch/arm64/include/asm/fixmap.h | 6 ++++-- arch/arm64/include/asm/sections.h | 5 +++++ arch/arm64/kernel/entry.S | 2 +- arch/arm64/kernel/vmlinux.lds.S | 2 +- arch/arm64/mm/mmu.c | 12 +++++++++--- 5 files changed, 20 insertions(+), 7 deletions(-) --- a/arch/arm64/include/asm/fixmap.h +++ b/arch/arm64/include/asm/fixmap.h @@ -62,9 +62,11 @@ enum fixed_addresses { #endif /* CONFIG_ACPI_APEI_GHES */ =20 #ifdef CONFIG_UNMAP_KERNEL_AT_EL0 - FIX_ENTRY_TRAMP_TEXT, + FIX_ENTRY_TRAMP_TEXT3, + FIX_ENTRY_TRAMP_TEXT2, + FIX_ENTRY_TRAMP_TEXT1, FIX_ENTRY_TRAMP_DATA, -#define TRAMP_VALIAS (__fix_to_virt(FIX_ENTRY_TRAMP_TEXT)) +#define TRAMP_VALIAS (__fix_to_virt(FIX_ENTRY_TRAMP_TEXT1)) #endif /* CONFIG_UNMAP_KERNEL_AT_EL0 */ __end_of_permanent_fixed_addresses, =20 --- a/arch/arm64/include/asm/sections.h +++ b/arch/arm64/include/asm/sections.h @@ -19,4 +19,9 @@ extern char __irqentry_text_start[], __i extern char __mmuoff_data_start[], __mmuoff_data_end[]; extern char __entry_tramp_text_start[], __entry_tramp_text_end[]; =20 +static inline size_t entry_tramp_text_size(void) +{ + return __entry_tramp_text_end - __entry_tramp_text_start; +} + #endif /* __ASM_SECTIONS_H */ --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -812,7 +812,7 @@ alternative_else_nop_endif .endm =20 .macro tramp_data_page dst - adr \dst, .entry.tramp.text + adr_l \dst, .entry.tramp.text sub \dst, \dst, PAGE_SIZE .endm =20 --- a/arch/arm64/kernel/vmlinux.lds.S +++ b/arch/arm64/kernel/vmlinux.lds.S @@ -299,7 +299,7 @@ ASSERT(__hibernate_exit_text_end - (__hi <=3D SZ_4K, "Hibernate exit text too big or misaligned") #endif #ifdef CONFIG_UNMAP_KERNEL_AT_EL0 -ASSERT((__entry_tramp_text_end - __entry_tramp_text_start) =3D=3D PAGE_SIZ= E, +ASSERT((__entry_tramp_text_end - __entry_tramp_text_start) <=3D 3*PAGE_SIZ= E, "Entry trampoline text too big") #endif /* --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -592,6 +592,8 @@ early_param("rodata", parse_rodata); #ifdef CONFIG_UNMAP_KERNEL_AT_EL0 static int __init map_entry_trampoline(void) { + int i; + pgprot_t prot =3D rodata_enabled ? PAGE_KERNEL_ROX : PAGE_KERNEL_EXEC; phys_addr_t pa_start =3D __pa_symbol(__entry_tramp_text_start); =20 @@ -600,11 +602,15 @@ static int __init map_entry_trampoline(v =20 /* Map only the text into the trampoline page table */ memset(tramp_pg_dir, 0, PGD_SIZE); - __create_pgd_mapping(tramp_pg_dir, pa_start, TRAMP_VALIAS, PAGE_SIZE, - prot, __pgd_pgtable_alloc, 0); + __create_pgd_mapping(tramp_pg_dir, pa_start, TRAMP_VALIAS, + entry_tramp_text_size(), prot, + __pgd_pgtable_alloc, NO_BLOCK_MAPPINGS); =20 /* Map both the text and data into the kernel page table */ - __set_fixmap(FIX_ENTRY_TRAMP_TEXT, pa_start, prot); + for (i =3D 0; i < DIV_ROUND_UP(entry_tramp_text_size(), PAGE_SIZE); i++) + __set_fixmap(FIX_ENTRY_TRAMP_TEXT1 - i, + pa_start + i * PAGE_SIZE, prot); + if (IS_ENABLED(CONFIG_RANDOMIZE_BASE)) { extern char __entry_tramp_data_start[]; From nobody Tue Jun 23 05:03:54 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 266DAC433FE for ; Thu, 10 Mar 2022 14:35:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240909AbiCJOgb (ORCPT ); Thu, 10 Mar 2022 09:36:31 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52068 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S244973AbiCJO3m (ORCPT ); Thu, 10 Mar 2022 09:29:42 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 32CB5158794; Thu, 10 Mar 2022 06:25:11 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id D4A7F61D7D; Thu, 10 Mar 2022 14:24:59 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id E4D4FC340E8; Thu, 10 Mar 2022 14:24:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1646922299; bh=rxrVtqzt8KzhHd7aJFQSaUe6TCVXuDiXrwAXtG1xse4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Iw/L+d4qL9xwlXYgqxofvJkticlbl0p76vrqLO4GoNnoDjOZfLHSEXKYp2B+TzBlC v/YmsDFmUTYp3s4wrgJW0fsBlzYRVGcE9F3W1DebuxbK8EMZ7j5NtPtvzBnZP8Lffo qgusHYZgd/fAuFy4tYefORNgqIMxOmgtUgPc1Zy8= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, "Russell King (Oracle)" , Catalin Marinas , James Morse Subject: [PATCH 5.10 33/58] arm64: entry: Add non-kpti __bp_harden_el1_vectors for mitigations Date: Thu, 10 Mar 2022 15:18:53 +0100 Message-Id: <20220310140813.817865923@linuxfoundation.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220310140812.869208747@linuxfoundation.org> References: <20220310140812.869208747@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: James Morse commit aff65393fa1401e034656e349abd655cfe272de0 upstream. kpti is an optional feature, for systems not using kpti a set of vectors for the spectre-bhb mitigations is needed. Add another set of vectors, __bp_harden_el1_vectors, that will be used if a mitigation is needed and kpti is not in use. The EL1 ventries are repeated verbatim as there is no additional work needed for entry from EL1. Reviewed-by: Russell King (Oracle) Reviewed-by: Catalin Marinas Signed-off-by: James Morse Signed-off-by: Greg Kroah-Hartman Tested-by: Bagas Sanjaya Tested-by: Florian Fainelli Tested-by: Fox Chen Tested-by: Guenter Roeck Tested-by: Linux Kernel Functional Testing Tested-by: Pavel Machek (CIP) = =20 Tested-by: Shuah Khan Tested-by: Sudip Mukherjee --- arch/arm64/kernel/entry.S | 35 ++++++++++++++++++++++++++++++++++- 1 file changed, 34 insertions(+), 1 deletion(-) --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -819,10 +819,11 @@ alternative_else_nop_endif .macro tramp_ventry, vector_start, regsize, kpti .align 7 1: - .if \kpti =3D=3D 1 .if \regsize =3D=3D 64 msr tpidrro_el0, x30 // Restored in kernel_ventry .endif + + .if \kpti =3D=3D 1 /* * Defend against branch aliasing attacks by pushing a dummy * entry onto the return stack and using a RET instruction to @@ -910,6 +911,38 @@ SYM_DATA_END(__entry_tramp_data_start) #endif /* CONFIG_UNMAP_KERNEL_AT_EL0 */ =20 /* + * Exception vectors for spectre mitigations on entry from EL1 when + * kpti is not in use. + */ + .macro generate_el1_vector +.Lvector_start\@: + kernel_ventry 1, sync_invalid // Synchronous EL1t + kernel_ventry 1, irq_invalid // IRQ EL1t + kernel_ventry 1, fiq_invalid // FIQ EL1t + kernel_ventry 1, error_invalid // Error EL1t + + kernel_ventry 1, sync // Synchronous EL1h + kernel_ventry 1, irq // IRQ EL1h + kernel_ventry 1, fiq_invalid // FIQ EL1h + kernel_ventry 1, error // Error EL1h + + .rept 4 + tramp_ventry .Lvector_start\@, 64, kpti=3D0 + .endr + .rept 4 + tramp_ventry .Lvector_start\@, 32, kpti=3D0 + .endr + .endm + + .pushsection ".entry.text", "ax" + .align 11 +SYM_CODE_START(__bp_harden_el1_vectors) + generate_el1_vector +SYM_CODE_END(__bp_harden_el1_vectors) + .popsection + + +/* * Register switch for AArch64. The callee-saved registers need to be saved * and restored. On entry: * x0 =3D previous task_struct (must be preserved across the switch) From nobody Tue Jun 23 05:03:54 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A54D2C433EF for ; Thu, 10 Mar 2022 14:35:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244474AbiCJOgP (ORCPT ); Thu, 10 Mar 2022 09:36:15 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49730 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S244926AbiCJO3k (ORCPT ); Thu, 10 Mar 2022 09:29:40 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BCADE156792; Thu, 10 Mar 2022 06:25:03 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 41C7CB82544; Thu, 10 Mar 2022 14:25:03 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id AE7F2C340E8; Thu, 10 Mar 2022 14:25:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1646922302; bh=Zy7ESnXJ/lsVEyziSCsKdmIWNOqS6ARmQC6cp9E55Qw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=rtPOTfwVeikkehEntsuVAGwtDBLfGCeyw30rIvGpvdwDgDrF/9mX2/WDo3z5n/eWj TKI2sV6uQM853KC+6ygEtvO0ZjSd5ce9Z6oNX9XrhFxTCQi3CSExmIE1x73aBgPsRK T9X+gIR1rcieX0PZdZGhWU09wzPKKA09gChzEti0= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Catalin Marinas , James Morse Subject: [PATCH 5.10 34/58] arm64: entry: Add vectors that have the bhb mitigation sequences Date: Thu, 10 Mar 2022 15:18:54 +0100 Message-Id: <20220310140813.845491188@linuxfoundation.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220310140812.869208747@linuxfoundation.org> References: <20220310140812.869208747@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: James Morse commit ba2689234be92024e5635d30fe744f4853ad97db upstream. Some CPUs affected by Spectre-BHB need a sequence of branches, or a firmware call to be run before any indirect branch. This needs to go in the vectors. No CPU needs both. While this can be patched in, it would run on all CPUs as there is a single set of vectors. If only one part of a big/little combination is affected, the unaffected CPUs have to run the mitigation too. Create extra vectors that include the sequence. Subsequent patches will allow affected CPUs to select this set of vectors. Later patches will modify the loop count to match what the CPU requires. Reviewed-by: Catalin Marinas Signed-off-by: James Morse Signed-off-by: Greg Kroah-Hartman Tested-by: Bagas Sanjaya Tested-by: Florian Fainelli Tested-by: Fox Chen Tested-by: Guenter Roeck Tested-by: Linux Kernel Functional Testing Tested-by: Pavel Machek (CIP) = =20 Tested-by: Shuah Khan Tested-by: Sudip Mukherjee --- arch/arm64/include/asm/assembler.h | 24 ++++++++++++++++ arch/arm64/include/asm/vectors.h | 34 +++++++++++++++++++++++ arch/arm64/kernel/entry.S | 53 ++++++++++++++++++++++++++++++--= ----- include/linux/arm-smccc.h | 5 +++ 4 files changed, 107 insertions(+), 9 deletions(-) create mode 100644 arch/arm64/include/asm/vectors.h --- a/arch/arm64/include/asm/assembler.h +++ b/arch/arm64/include/asm/assembler.h @@ -795,4 +795,28 @@ USER(\label, ic ivau, \tmp2) // invali =20 #endif /* GNU_PROPERTY_AARCH64_FEATURE_1_DEFAULT */ =20 + .macro __mitigate_spectre_bhb_loop tmp +#ifdef CONFIG_MITIGATE_SPECTRE_BRANCH_HISTORY + mov \tmp, #32 +.Lspectre_bhb_loop\@: + b . + 4 + subs \tmp, \tmp, #1 + b.ne .Lspectre_bhb_loop\@ + sb +#endif /* CONFIG_MITIGATE_SPECTRE_BRANCH_HISTORY */ + .endm + + /* Save/restores x0-x3 to the stack */ + .macro __mitigate_spectre_bhb_fw +#ifdef CONFIG_MITIGATE_SPECTRE_BRANCH_HISTORY + stp x0, x1, [sp, #-16]! + stp x2, x3, [sp, #-16]! + mov w0, #ARM_SMCCC_ARCH_WORKAROUND_3 +alternative_cb smccc_patch_fw_mitigation_conduit + nop // Patched to SMC/HVC #0 +alternative_cb_end + ldp x2, x3, [sp], #16 + ldp x0, x1, [sp], #16 +#endif /* CONFIG_MITIGATE_SPECTRE_BRANCH_HISTORY */ + .endm #endif /* __ASM_ASSEMBLER_H */ --- /dev/null +++ b/arch/arm64/include/asm/vectors.h @@ -0,0 +1,34 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2022 ARM Ltd. + */ +#ifndef __ASM_VECTORS_H +#define __ASM_VECTORS_H + +/* + * Note: the order of this enum corresponds to two arrays in entry.S: + * tramp_vecs and __bp_harden_el1_vectors. By default the canonical + * 'full fat' vectors are used directly. + */ +enum arm64_bp_harden_el1_vectors { +#ifdef CONFIG_MITIGATE_SPECTRE_BRANCH_HISTORY + /* + * Perform the BHB loop mitigation, before branching to the canonical + * vectors. + */ + EL1_VECTOR_BHB_LOOP, + + /* + * Make the SMC call for firmware mitigation, before branching to the + * canonical vectors. + */ + EL1_VECTOR_BHB_FW, +#endif /* CONFIG_MITIGATE_SPECTRE_BRANCH_HISTORY */ + + /* + * Remap the kernel before branching to the canonical vectors. + */ + EL1_VECTOR_KPTI, ++}; + +#endif /* __ASM_VECTORS_H */ --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -816,13 +816,26 @@ alternative_else_nop_endif sub \dst, \dst, PAGE_SIZE .endm =20 - .macro tramp_ventry, vector_start, regsize, kpti + +#define BHB_MITIGATION_NONE 0 +#define BHB_MITIGATION_LOOP 1 +#define BHB_MITIGATION_FW 2 + + .macro tramp_ventry, vector_start, regsize, kpti, bhb .align 7 1: .if \regsize =3D=3D 64 msr tpidrro_el0, x30 // Restored in kernel_ventry .endif =20 + .if \bhb =3D=3D BHB_MITIGATION_LOOP + /* + * This sequence must appear before the first indirect branch. i.e. the + * ret out of tramp_ventry. It appears here because x30 is free. + */ + __mitigate_spectre_bhb_loop x30 + .endif // \bhb =3D=3D BHB_MITIGATION_LOOP + .if \kpti =3D=3D 1 /* * Defend against branch aliasing attacks by pushing a dummy @@ -850,6 +863,15 @@ alternative_else_nop_endif ldr x30, =3Dvectors .endif // \kpti =3D=3D 1 =20 + .if \bhb =3D=3D BHB_MITIGATION_FW + /* + * The firmware sequence must appear before the first indirect branch. + * i.e. the ret out of tramp_ventry. But it also needs the stack to be + * mapped to save/restore the registers the SMC clobbers. + */ + __mitigate_spectre_bhb_fw + .endif // \bhb =3D=3D BHB_MITIGATION_FW + add x30, x30, #(1b - \vector_start + 4) ret .org 1b + 128 // Did we overflow the ventry slot? @@ -857,6 +879,9 @@ alternative_else_nop_endif =20 .macro tramp_exit, regsize =3D 64 adr x30, tramp_vectors +#ifdef CONFIG_MITIGATE_SPECTRE_BRANCH_HISTORY + add x30, x30, SZ_4K +#endif msr vbar_el1, x30 ldr lr, [sp, #S_LR] tramp_unmap_kernel x29 @@ -868,26 +893,32 @@ alternative_else_nop_endif sb .endm =20 - .macro generate_tramp_vector, kpti + .macro generate_tramp_vector, kpti, bhb .Lvector_start\@: .space 0x400 =20 .rept 4 - tramp_ventry .Lvector_start\@, 64, \kpti + tramp_ventry .Lvector_start\@, 64, \kpti, \bhb .endr .rept 4 - tramp_ventry .Lvector_start\@, 32, \kpti + tramp_ventry .Lvector_start\@, 32, \kpti, \bhb .endr .endm =20 #ifdef CONFIG_UNMAP_KERNEL_AT_EL0 /* * Exception vectors trampoline. + * The order must match __bp_harden_el1_vectors and the + * arm64_bp_harden_el1_vectors enum. */ .pushsection ".entry.tramp.text", "ax" .align 11 SYM_CODE_START_NOALIGN(tramp_vectors) - generate_tramp_vector kpti=3D1 +#ifdef CONFIG_MITIGATE_SPECTRE_BRANCH_HISTORY + generate_tramp_vector kpti=3D1, bhb=3DBHB_MITIGATION_LOOP + generate_tramp_vector kpti=3D1, bhb=3DBHB_MITIGATION_FW +#endif /* CONFIG_MITIGATE_SPECTRE_BRANCH_HISTORY */ + generate_tramp_vector kpti=3D1, bhb=3DBHB_MITIGATION_NONE SYM_CODE_END(tramp_vectors) =20 SYM_CODE_START(tramp_exit_native) @@ -914,7 +945,7 @@ SYM_DATA_END(__entry_tramp_data_start) * Exception vectors for spectre mitigations on entry from EL1 when * kpti is not in use. */ - .macro generate_el1_vector + .macro generate_el1_vector, bhb .Lvector_start\@: kernel_ventry 1, sync_invalid // Synchronous EL1t kernel_ventry 1, irq_invalid // IRQ EL1t @@ -927,17 +958,21 @@ SYM_DATA_END(__entry_tramp_data_start) kernel_ventry 1, error // Error EL1h =20 .rept 4 - tramp_ventry .Lvector_start\@, 64, kpti=3D0 + tramp_ventry .Lvector_start\@, 64, 0, \bhb .endr .rept 4 - tramp_ventry .Lvector_start\@, 32, kpti=3D0 + tramp_ventry .Lvector_start\@, 32, 0, \bhb .endr .endm =20 +/* The order must match tramp_vecs and the arm64_bp_harden_el1_vectors enu= m. */ .pushsection ".entry.text", "ax" .align 11 SYM_CODE_START(__bp_harden_el1_vectors) - generate_el1_vector +#ifdef CONFIG_MITIGATE_SPECTRE_BRANCH_HISTORY + generate_el1_vector bhb=3DBHB_MITIGATION_LOOP + generate_el1_vector bhb=3DBHB_MITIGATION_FW +#endif /* CONFIG_MITIGATE_SPECTRE_BRANCH_HISTORY */ SYM_CODE_END(__bp_harden_el1_vectors) .popsection =20 --- a/include/linux/arm-smccc.h +++ b/include/linux/arm-smccc.h @@ -87,6 +87,11 @@ ARM_SMCCC_SMC_32, \ 0, 0x7fff) =20 +#define ARM_SMCCC_ARCH_WORKAROUND_3 \ + ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, \ + ARM_SMCCC_SMC_32, \ + 0, 0x3fff) + #define SMCCC_ARCH_WORKAROUND_RET_UNAFFECTED 1 =20 /* Paravirtualised time calls (defined by ARM DEN0057A) */ From nobody Tue Jun 23 05:03:54 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9B8D0C46467 for ; Thu, 10 Mar 2022 14:42:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344904AbiCJOlT (ORCPT ); Thu, 10 Mar 2022 09:41:19 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49570 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S244987AbiCJO3n (ORCPT ); Thu, 10 Mar 2022 09:29:43 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 31CDDB8B6D; Thu, 10 Mar 2022 06:25:14 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 975BDB8266E; Thu, 10 Mar 2022 14:25:06 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id D97E7C36AF3; Thu, 10 Mar 2022 14:25:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1646922305; bh=Qz6t2IobnC8RCnxQbuaAbH5Pkqhth068M/VMGtTIl+E=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=OgIkZMd+Zi9prbMRn9JUD1X8ctFMVNY2Ql3bo4OOplJZTjJceF/i690jP9wnQ6utd fC5aKFXLHaP0txY4PqnZF6IDDM3f6me24PFLfDlQOAf0/LxkMVJxTBIM8DvI4/SWp+ eYK8ODCNPQHzJn2V72cX4ppRNNbQezP84LYXsptI= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Catalin Marinas , James Morse Subject: [PATCH 5.10 35/58] arm64: entry: Add macro for reading symbol addresses from the trampoline Date: Thu, 10 Mar 2022 15:18:55 +0100 Message-Id: <20220310140813.873354590@linuxfoundation.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220310140812.869208747@linuxfoundation.org> References: <20220310140812.869208747@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: James Morse commit b28a8eebe81c186fdb1a0078263b30576c8e1f42 upstream. The trampoline code needs to use the address of symbols in the wider kernel, e.g. vectors. PC-relative addressing wouldn't work as the trampoline code doesn't run at the address the linker expected. tramp_ventry uses a literal pool, unless CONFIG_RANDOMIZE_BASE is set, in which case it uses the data page as a literal pool because the data page can be unmapped when running in user-space, which is required for CPUs vulnerable to meltdown. Pull this logic out as a macro, instead of adding a third copy of it. Reviewed-by: Catalin Marinas Signed-off-by: James Morse Signed-off-by: Greg Kroah-Hartman Tested-by: Bagas Sanjaya Tested-by: Florian Fainelli Tested-by: Fox Chen Tested-by: Guenter Roeck Tested-by: Linux Kernel Functional Testing Tested-by: Pavel Machek (CIP) = =20 Tested-by: Shuah Khan Tested-by: Sudip Mukherjee --- arch/arm64/kernel/entry.S | 36 ++++++++++++++++-------------------- 1 file changed, 16 insertions(+), 20 deletions(-) --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -816,6 +816,15 @@ alternative_else_nop_endif sub \dst, \dst, PAGE_SIZE .endm =20 + .macro tramp_data_read_var dst, var +#ifdef CONFIG_RANDOMIZE_BASE + tramp_data_page \dst + add \dst, \dst, #:lo12:__entry_tramp_data_\var + ldr \dst, [\dst] +#else + ldr \dst, =3D\var +#endif + .endm =20 #define BHB_MITIGATION_NONE 0 #define BHB_MITIGATION_LOOP 1 @@ -846,13 +855,8 @@ alternative_else_nop_endif b . 2: tramp_map_kernel x30 -#ifdef CONFIG_RANDOMIZE_BASE - tramp_data_page x30 alternative_insn isb, nop, ARM64_WORKAROUND_QCOM_FALKOR_E1003 - ldr x30, [x30] -#else - ldr x30, =3Dvectors -#endif + tramp_data_read_var x30, vectors alternative_if_not ARM64_WORKAROUND_CAVIUM_TX2_219_PRFM prfm plil1strm, [x30, #(1b - \vector_start)] alternative_else_nop_endif @@ -935,7 +939,12 @@ SYM_CODE_END(tramp_exit_compat) .pushsection ".rodata", "a" .align PAGE_SHIFT SYM_DATA_START(__entry_tramp_data_start) +__entry_tramp_data_vectors: .quad vectors +#ifdef CONFIG_ARM_SDE_INTERFACE +__entry_tramp_data___sdei_asm_handler: + .quad __sdei_asm_handler +#endif /* CONFIG_ARM_SDE_INTERFACE */ SYM_DATA_END(__entry_tramp_data_start) .popsection // .rodata #endif /* CONFIG_RANDOMIZE_BASE */ @@ -1066,13 +1075,7 @@ SYM_CODE_START(__sdei_asm_entry_trampoli */ 1: str x4, [x1, #(SDEI_EVENT_INTREGS + S_ORIG_ADDR_LIMIT)] =20 -#ifdef CONFIG_RANDOMIZE_BASE - tramp_data_page x4 - add x4, x4, #:lo12:__sdei_asm_trampoline_next_handler - ldr x4, [x4] -#else - ldr x4, =3D__sdei_asm_handler -#endif + tramp_data_read_var x4, __sdei_asm_handler br x4 SYM_CODE_END(__sdei_asm_entry_trampoline) NOKPROBE(__sdei_asm_entry_trampoline) @@ -1095,13 +1098,6 @@ SYM_CODE_END(__sdei_asm_exit_trampoline) NOKPROBE(__sdei_asm_exit_trampoline) .ltorg .popsection // .entry.tramp.text -#ifdef CONFIG_RANDOMIZE_BASE -.pushsection ".rodata", "a" -SYM_DATA_START(__sdei_asm_trampoline_next_handler) - .quad __sdei_asm_handler -SYM_DATA_END(__sdei_asm_trampoline_next_handler) -.popsection // .rodata -#endif /* CONFIG_RANDOMIZE_BASE */ #endif /* CONFIG_UNMAP_KERNEL_AT_EL0 */ =20 /* From nobody Tue Jun 23 05:03:54 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1EB81C433FE for ; Thu, 10 Mar 2022 14:35:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244504AbiCJOgi (ORCPT ); Thu, 10 Mar 2022 09:36:38 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49714 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S244988AbiCJO3n (ORCPT ); Thu, 10 Mar 2022 09:29:43 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C8051B82FA; Thu, 10 Mar 2022 06:25:13 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 0E87961DC6; Thu, 10 Mar 2022 14:25:09 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1594FC340E8; Thu, 10 Mar 2022 14:25:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1646922308; bh=5MfjfJ6jp8YXTQIQcKBjVhF84Y+Y1og+PNsFWbJtkeM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=IaT1Cpd6XtkrVT1YxKeOZ4uAyNMjCyAKw3Jcbcw1oEg+1pWw8DTRLcV2Vi45HDPfw gmMhKvMwOcLtphXGNF1c+yorqtfPqSwX4hQv2K72fsl4HwXYoOcbYl/HlRZ1Inpzsy Z1LrOpNdZi6akWlmNrFgTAHO6UOK+TkkwjyKAP4A= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Catalin Marinas , James Morse Subject: [PATCH 5.10 36/58] arm64: Add percpu vectors for EL1 Date: Thu, 10 Mar 2022 15:18:56 +0100 Message-Id: <20220310140813.900663021@linuxfoundation.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220310140812.869208747@linuxfoundation.org> References: <20220310140812.869208747@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: James Morse commit bd09128d16fac3c34b80bd6a29088ac632e8ce09 upstream. The Spectre-BHB workaround adds a firmware call to the vectors. This is needed on some CPUs, but not others. To avoid the unaffected CPU in a big/little pair from making the firmware call, create per cpu vectors. The per-cpu vectors only apply when returning from EL0. Systems using KPTI can use the canonical 'full-fat' vectors directly at EL1, the trampoline exit code will switch to this_cpu_vector on exit to EL0. Systems not using KPTI should always use this_cpu_vector. this_cpu_vector will point at a vector in tramp_vecs or __bp_harden_el1_vectors, depending on whether KPTI is in use. Reviewed-by: Catalin Marinas Signed-off-by: James Morse Signed-off-by: Greg Kroah-Hartman Tested-by: Bagas Sanjaya Tested-by: Florian Fainelli Tested-by: Fox Chen Tested-by: Guenter Roeck Tested-by: Linux Kernel Functional Testing Tested-by: Pavel Machek (CIP) = =20 Tested-by: Shuah Khan Tested-by: Sudip Mukherjee --- arch/arm64/include/asm/vectors.h | 29 ++++++++++++++++++++++++++++- arch/arm64/kernel/cpufeature.c | 11 +++++++++++ arch/arm64/kernel/entry.S | 12 ++++++------ arch/arm64/kvm/hyp/vhe/switch.c | 9 +++++++-- 4 files changed, 52 insertions(+), 9 deletions(-) --- a/arch/arm64/include/asm/vectors.h +++ b/arch/arm64/include/asm/vectors.h @@ -5,6 +5,15 @@ #ifndef __ASM_VECTORS_H #define __ASM_VECTORS_H =20 +#include +#include + +#include + +extern char vectors[]; +extern char tramp_vectors[]; +extern char __bp_harden_el1_vectors[]; + /* * Note: the order of this enum corresponds to two arrays in entry.S: * tramp_vecs and __bp_harden_el1_vectors. By default the canonical @@ -29,6 +38,24 @@ enum arm64_bp_harden_el1_vectors { * Remap the kernel before branching to the canonical vectors. */ EL1_VECTOR_KPTI, -+}; +}; + +/* The vectors to use on return from EL0. e.g. to remap the kernel */ +DECLARE_PER_CPU_READ_MOSTLY(const char *, this_cpu_vector); + +#ifndef CONFIG_UNMAP_KERNEL_AT_EL0 +#define TRAMP_VALIAS 0 +#endif + +static inline const char * +arm64_get_bp_hardening_vector(enum arm64_bp_harden_el1_vectors slot) +{ + if (arm64_kernel_unmapped_at_el0()) + return (char *)TRAMP_VALIAS + SZ_2K * slot; + + WARN_ON_ONCE(slot =3D=3D EL1_VECTOR_KPTI); + + return __bp_harden_el1_vectors + SZ_2K * slot; +} =20 #endif /* __ASM_VECTORS_H */ --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -65,11 +65,13 @@ #include #include #include +#include #include #include #include #include #include + #include #include #include @@ -79,6 +81,7 @@ #include #include #include +#include #include =20 /* Kernel representation of AT_HWCAP and AT_HWCAP2 */ @@ -104,6 +107,8 @@ DECLARE_BITMAP(boot_capabilities, ARM64_ bool arm64_use_ng_mappings =3D false; EXPORT_SYMBOL(arm64_use_ng_mappings); =20 +DEFINE_PER_CPU_READ_MOSTLY(const char *, this_cpu_vector) =3D vectors; + /* * Flag to indicate if we have computed the system wide * capabilities based on the boot time active CPUs. This @@ -1413,6 +1418,12 @@ kpti_install_ng_mappings(const struct ar =20 int cpu =3D smp_processor_id(); =20 + if (__this_cpu_read(this_cpu_vector) =3D=3D vectors) { + const char *v =3D arm64_get_bp_hardening_vector(EL1_VECTOR_KPTI); + + __this_cpu_write(this_cpu_vector, v); + } + /* * We don't need to rewrite the page-tables if either we've done * it already or we have KASLR enabled and therefore have not --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -63,7 +63,6 @@ .macro kernel_ventry, el, label, regsize =3D 64 .align 7 .Lventry_start\@: -#ifdef CONFIG_UNMAP_KERNEL_AT_EL0 .if \el =3D=3D 0 /* * This must be the first instruction of the EL0 vector entries. It is @@ -78,7 +77,6 @@ .endif .Lskip_tramp_vectors_cleanup\@: .endif -#endif =20 sub sp, sp, #S_FRAME_SIZE #ifdef CONFIG_VMAP_STACK @@ -882,10 +880,10 @@ alternative_else_nop_endif .endm =20 .macro tramp_exit, regsize =3D 64 - adr x30, tramp_vectors -#ifdef CONFIG_MITIGATE_SPECTRE_BRANCH_HISTORY - add x30, x30, SZ_4K -#endif + tramp_data_read_var x30, this_cpu_vector + this_cpu_offset x29 + ldr x30, [x30, x29] + msr vbar_el1, x30 ldr lr, [sp, #S_LR] tramp_unmap_kernel x29 @@ -945,6 +943,8 @@ __entry_tramp_data_vectors: __entry_tramp_data___sdei_asm_handler: .quad __sdei_asm_handler #endif /* CONFIG_ARM_SDE_INTERFACE */ +__entry_tramp_data_this_cpu_vector: + .quad this_cpu_vector SYM_DATA_END(__entry_tramp_data_start) .popsection // .rodata #endif /* CONFIG_RANDOMIZE_BASE */ --- a/arch/arm64/kvm/hyp/vhe/switch.c +++ b/arch/arm64/kvm/hyp/vhe/switch.c @@ -10,6 +10,7 @@ #include #include #include +#include #include =20 #include @@ -25,6 +26,7 @@ #include #include #include +#include =20 const char __hyp_panic_string[] =3D "HYP panic:\nPS:%08llx PC:%016llx ESR:= %08llx\nFAR:%016llx HPFAR:%016llx PAR:%016llx\nVCPU:%p\n"; =20 @@ -70,7 +72,7 @@ NOKPROBE_SYMBOL(__activate_traps); =20 static void __deactivate_traps(struct kvm_vcpu *vcpu) { - extern char vectors[]; /* kernel exception vectors */ + const char *host_vectors =3D vectors; =20 ___deactivate_traps(vcpu); =20 @@ -84,7 +86,10 @@ static void __deactivate_traps(struct kv asm(ALTERNATIVE("nop", "isb", ARM64_WORKAROUND_SPECULATIVE_AT)); =20 write_sysreg(CPACR_EL1_DEFAULT, cpacr_el1); - write_sysreg(vectors, vbar_el1); + + if (!arm64_kernel_unmapped_at_el0()) + host_vectors =3D __this_cpu_read(this_cpu_vector); + write_sysreg(host_vectors, vbar_el1); } NOKPROBE_SYMBOL(__deactivate_traps); From nobody Tue Jun 23 05:03:54 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C83DCC4707A for ; Thu, 10 Mar 2022 14:42:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1345043AbiCJOlc (ORCPT ); Thu, 10 Mar 2022 09:41:32 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50106 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245233AbiCJOaQ (ORCPT ); Thu, 10 Mar 2022 09:30:16 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C9EC91704D0; Thu, 10 Mar 2022 06:25:44 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id D0638B82676; Thu, 10 Mar 2022 14:25:12 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 4080FC340F5; Thu, 10 Mar 2022 14:25:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1646922311; bh=RqzeAQhcKiZiQ5wOPHZeqLujTq7UN4gy/nboCZyWHbg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=hB2xNvgDCpTrumpjhINTKNtDYM0hJVXoKHUJc+ukIVVUrGH39yche5RNf4lfDteAJ T1JBo944bcTT4SW5uylgyEVCzFMJrKU4j/MeWuSXqNUx8WhKBYqUxDbeVg7AD962Og 52kZBwnx8/URmJeoOLrT8q2U7g35WlvIJmESQTTg= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Catalin Marinas , James Morse Subject: [PATCH 5.10 37/58] arm64: proton-pack: Report Spectre-BHB vulnerabilities as part of Spectre-v2 Date: Thu, 10 Mar 2022 15:18:57 +0100 Message-Id: <20220310140813.928124989@linuxfoundation.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220310140812.869208747@linuxfoundation.org> References: <20220310140812.869208747@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: James Morse commit dee435be76f4117410bbd90573a881fd33488f37 upstream. Speculation attacks against some high-performance processors can make use of branch history to influence future speculation as part of a spectre-v2 attack. This is not mitigated by CSV2, meaning CPUs that previously reported 'Not affected' are now moderately mitigated by CSV2. Update the value in /sys/devices/system/cpu/vulnerabilities/spectre_v2 to also show the state of the BHB mitigation. Reviewed-by: Catalin Marinas Signed-off-by: James Morse Signed-off-by: Greg Kroah-Hartman Tested-by: Bagas Sanjaya Tested-by: Florian Fainelli Tested-by: Fox Chen Tested-by: Guenter Roeck Tested-by: Linux Kernel Functional Testing Tested-by: Pavel Machek (CIP) = =20 Tested-by: Shuah Khan Tested-by: Sudip Mukherjee --- arch/arm64/include/asm/spectre.h | 2 ++ arch/arm64/kernel/proton-pack.c | 36 ++++++++++++++++++++++++++++++++++= -- 2 files changed, 36 insertions(+), 2 deletions(-) --- a/arch/arm64/include/asm/spectre.h +++ b/arch/arm64/include/asm/spectre.h @@ -29,4 +29,6 @@ bool has_spectre_v4(const struct arm64_c void spectre_v4_enable_mitigation(const struct arm64_cpu_capabilities *__u= nused); void spectre_v4_enable_task_mitigation(struct task_struct *tsk); =20 +enum mitigation_state arm64_get_spectre_bhb_state(void); + #endif /* __ASM_SPECTRE_H */ --- a/arch/arm64/kernel/proton-pack.c +++ b/arch/arm64/kernel/proton-pack.c @@ -94,14 +94,39 @@ static bool spectre_v2_mitigations_off(v return ret; } =20 +static const char *get_bhb_affected_string(enum mitigation_state bhb_state) +{ + switch (bhb_state) { + case SPECTRE_UNAFFECTED: + return ""; + default: + case SPECTRE_VULNERABLE: + return ", but not BHB"; + case SPECTRE_MITIGATED: + return ", BHB"; + } +} + ssize_t cpu_show_spectre_v2(struct device *dev, struct device_attribute *a= ttr, char *buf) { + enum mitigation_state bhb_state =3D arm64_get_spectre_bhb_state(); + const char *bhb_str =3D get_bhb_affected_string(bhb_state); + const char *v2_str =3D "Branch predictor hardening"; + switch (spectre_v2_state) { case SPECTRE_UNAFFECTED: - return sprintf(buf, "Not affected\n"); + if (bhb_state =3D=3D SPECTRE_UNAFFECTED) + return sprintf(buf, "Not affected\n"); + + /* + * Platforms affected by Spectre-BHB can't report + * "Not affected" for Spectre-v2. + */ + v2_str =3D "CSV2"; + fallthrough; case SPECTRE_MITIGATED: - return sprintf(buf, "Mitigation: Branch predictor hardening\n"); + return sprintf(buf, "Mitigation: %s%s\n", v2_str, bhb_str); case SPECTRE_VULNERABLE: fallthrough; default: @@ -787,3 +812,10 @@ int arch_prctl_spec_ctrl_get(struct task return -ENODEV; } } + +static enum mitigation_state spectre_bhb_state; + +enum mitigation_state arm64_get_spectre_bhb_state(void) +{ + return spectre_bhb_state; +} From nobody Tue Jun 23 05:03:54 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id F39D6C4707E for ; Thu, 10 Mar 2022 14:42:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1345145AbiCJOlj (ORCPT ); Thu, 10 Mar 2022 09:41:39 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48214 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245258AbiCJOaS (ORCPT ); Thu, 10 Mar 2022 09:30:18 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9CAF5172271; Thu, 10 Mar 2022 06:25:46 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 3C29961D23; Thu, 10 Mar 2022 14:25:15 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 24701C340F5; Thu, 10 Mar 2022 14:25:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1646922314; bh=J6wXw2OY/SEj7WgYtzb95YfVux64kBMGaFrb3rNLZiI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=bsVMsMkHBB06gIq1L+/d6oNjRHQvaXlce9iwUY168Y6J3hIMNtRfpN/TKz0y7Dcya PTK0TJ+5ctCg7aT0KLwqzz7IadcAaKipGk666W3F8aFPlMLMfe7pD71etQxHUNJuXU bPTjPn0Fj5A+KGQvgcSdHufgTFyygsBdWz4y+MR4= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Catalin Marinas , James Morse Subject: [PATCH 5.10 38/58] KVM: arm64: Allow indirect vectors to be used without SPECTRE_V3A Date: Thu, 10 Mar 2022 15:18:58 +0100 Message-Id: <20220310140813.956533242@linuxfoundation.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220310140812.869208747@linuxfoundation.org> References: <20220310140812.869208747@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: James Morse commit 5bdf3437603d4af87f9c7f424b0c8aeed2420745 upstream. CPUs vulnerable to Spectre-BHB either need to make an SMC-CC firmware call from the vectors, or run a sequence of branches. This gets added to the hyp vectors. If there is no support for arch-workaround-1 in firmware, the indirect vector will be used. kvm_init_vector_slots() only initialises the two indirect slots if the platform is vulnerable to Spectre-v3a. pKVM's hyp_map_vectors() only initialises __hyp_bp_vect_base if the platform is vulnerable to Spectre-v3a. As there are about to more users of the indirect vectors, ensure their entries in hyp_spectre_vector_selector[] are always initialised, and __hyp_bp_vect_base defaults to the regular VA mapping. The Spectre-v3a check is moved to a helper kvm_system_needs_idmapped_vectors(), and merged with the code that creates the hyp mappings. Reviewed-by: Catalin Marinas Signed-off-by: James Morse Signed-off-by: Greg Kroah-Hartman Tested-by: Bagas Sanjaya Tested-by: Florian Fainelli Tested-by: Fox Chen Tested-by: Guenter Roeck Tested-by: Linux Kernel Functional Testing Tested-by: Pavel Machek (CIP) = =20 Tested-by: Shuah Khan Tested-by: Sudip Mukherjee --- arch/arm64/include/asm/cpucaps.h | 3 + arch/arm64/include/asm/kvm_asm.h | 6 +++ arch/arm64/include/asm/kvm_mmu.h | 3 + arch/arm64/include/asm/mmu.h | 6 +++ arch/arm64/kernel/proton-pack.c | 47 +++++++++++++++++++++++++++ arch/arm64/kvm/arm.c | 3 + arch/arm64/kvm/hyp/smccc_wa.S | 66 ++++++++++++++++++++++++++++++++++= +++++ 7 files changed, 130 insertions(+), 4 deletions(-) --- a/arch/arm64/include/asm/cpucaps.h +++ b/arch/arm64/include/asm/cpucaps.h @@ -66,7 +66,8 @@ #define ARM64_HAS_TLB_RANGE 56 #define ARM64_MTE 57 #define ARM64_WORKAROUND_1508412 58 +#define ARM64_SPECTRE_BHB 59 =20 -#define ARM64_NCAPS 59 +#define ARM64_NCAPS 60 =20 #endif /* __ASM_CPUCAPS_H */ --- a/arch/arm64/include/asm/kvm_asm.h +++ b/arch/arm64/include/asm/kvm_asm.h @@ -35,6 +35,8 @@ #define KVM_VECTOR_PREAMBLE (2 * AARCH64_INSN_SIZE) =20 #define __SMCCC_WORKAROUND_1_SMC_SZ 36 +#define __SMCCC_WORKAROUND_3_SMC_SZ 36 +#define __SPECTRE_BHB_LOOP_SZ 44 =20 #define KVM_HOST_SMCCC_ID(id) \ ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, \ @@ -199,6 +201,10 @@ extern void __vgic_v3_init_lrs(void); extern u32 __kvm_get_mdcr_el2(void); =20 extern char __smccc_workaround_1_smc[__SMCCC_WORKAROUND_1_SMC_SZ]; +extern char __smccc_workaround_3_smc[__SMCCC_WORKAROUND_3_SMC_SZ]; +extern char __spectre_bhb_loop_k8[__SPECTRE_BHB_LOOP_SZ]; +extern char __spectre_bhb_loop_k24[__SPECTRE_BHB_LOOP_SZ]; +extern char __spectre_bhb_loop_k32[__SPECTRE_BHB_LOOP_SZ]; =20 /* * Obtain the PC-relative address of a kernel symbol --- a/arch/arm64/include/asm/kvm_mmu.h +++ b/arch/arm64/include/asm/kvm_mmu.h @@ -237,7 +237,8 @@ static inline void *kvm_get_hyp_vector(v void *vect =3D kern_hyp_va(kvm_ksym_ref(__kvm_hyp_vector)); int slot =3D -1; =20 - if (cpus_have_const_cap(ARM64_SPECTRE_V2) && data->fn) { + if ((cpus_have_const_cap(ARM64_SPECTRE_V2) || + cpus_have_const_cap(ARM64_SPECTRE_BHB)) && data->template_start) { vect =3D kern_hyp_va(kvm_ksym_ref(__bp_harden_hyp_vecs)); slot =3D data->hyp_vectors_slot; } --- a/arch/arm64/include/asm/mmu.h +++ b/arch/arm64/include/asm/mmu.h @@ -67,6 +67,12 @@ typedef void (*bp_hardening_cb_t)(void); struct bp_hardening_data { int hyp_vectors_slot; bp_hardening_cb_t fn; + + /* + * template_start is only used by the BHB mitigation to identify the + * hyp_vectors_slot sequence. + */ + const char *template_start; }; =20 DECLARE_PER_CPU_READ_MOSTLY(struct bp_hardening_data, bp_hardening_data); --- a/arch/arm64/kernel/proton-pack.c +++ b/arch/arm64/kernel/proton-pack.c @@ -220,9 +220,9 @@ static void __copy_hyp_vect_bpi(int slot __flush_icache_range((uintptr_t)dst, (uintptr_t)dst + SZ_2K); } =20 +static DEFINE_RAW_SPINLOCK(bp_lock); static void install_bp_hardening_cb(bp_hardening_cb_t fn) { - static DEFINE_RAW_SPINLOCK(bp_lock); int cpu, slot =3D -1; const char *hyp_vecs_start =3D __smccc_workaround_1_smc; const char *hyp_vecs_end =3D __smccc_workaround_1_smc + @@ -253,6 +253,7 @@ static void install_bp_hardening_cb(bp_h =20 __this_cpu_write(bp_hardening_data.hyp_vectors_slot, slot); __this_cpu_write(bp_hardening_data.fn, fn); + __this_cpu_write(bp_hardening_data.template_start, hyp_vecs_start); raw_spin_unlock(&bp_lock); } #else @@ -819,3 +820,47 @@ enum mitigation_state arm64_get_spectre_ { return spectre_bhb_state; } + +static int kvm_bhb_get_vecs_size(const char *start) +{ + if (start =3D=3D __smccc_workaround_3_smc) + return __SMCCC_WORKAROUND_3_SMC_SZ; + else if (start =3D=3D __spectre_bhb_loop_k8 || + start =3D=3D __spectre_bhb_loop_k24 || + start =3D=3D __spectre_bhb_loop_k32) + return __SPECTRE_BHB_LOOP_SZ; + + return 0; +} + +void kvm_setup_bhb_slot(const char *hyp_vecs_start) +{ + int cpu, slot =3D -1, size; + const char *hyp_vecs_end; + + if (!IS_ENABLED(CONFIG_KVM) || !is_hyp_mode_available()) + return; + + size =3D kvm_bhb_get_vecs_size(hyp_vecs_start); + if (WARN_ON_ONCE(!hyp_vecs_start || !size)) + return; + hyp_vecs_end =3D hyp_vecs_start + size; + + raw_spin_lock(&bp_lock); + for_each_possible_cpu(cpu) { + if (per_cpu(bp_hardening_data.template_start, cpu) =3D=3D hyp_vecs_start= ) { + slot =3D per_cpu(bp_hardening_data.hyp_vectors_slot, cpu); + break; + } + } + + if (slot =3D=3D -1) { + slot =3D atomic_inc_return(&arm64_el2_vector_last_slot); + BUG_ON(slot >=3D BP_HARDEN_EL2_SLOTS); + __copy_hyp_vect_bpi(slot, hyp_vecs_start, hyp_vecs_end); + } + + __this_cpu_write(bp_hardening_data.hyp_vectors_slot, slot); + __this_cpu_write(bp_hardening_data.template_start, hyp_vecs_start); + raw_spin_unlock(&bp_lock); +} --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -1337,7 +1337,8 @@ static int kvm_map_vectors(void) * !SV2 + HEL2 -> allocate one vector slot and use exec mapping * SV2 + HEL2 -> use hardened vectors and use exec mapping */ - if (cpus_have_const_cap(ARM64_SPECTRE_V2)) { + if (cpus_have_const_cap(ARM64_SPECTRE_V2) || + cpus_have_const_cap(ARM64_SPECTRE_BHB)) { __kvm_bp_vect_base =3D kvm_ksym_ref(__bp_harden_hyp_vecs); __kvm_bp_vect_base =3D kern_hyp_va(__kvm_bp_vect_base); } --- a/arch/arm64/kvm/hyp/smccc_wa.S +++ b/arch/arm64/kvm/hyp/smccc_wa.S @@ -30,3 +30,69 @@ SYM_DATA_START(__smccc_workaround_1_smc) 1: .org __smccc_workaround_1_smc + __SMCCC_WORKAROUND_1_SMC_SZ .org 1b SYM_DATA_END(__smccc_workaround_1_smc) + + .global __smccc_workaround_3_smc +SYM_DATA_START(__smccc_workaround_3_smc) + esb + sub sp, sp, #(8 * 4) + stp x2, x3, [sp, #(8 * 0)] + stp x0, x1, [sp, #(8 * 2)] + mov w0, #ARM_SMCCC_ARCH_WORKAROUND_3 + smc #0 + ldp x2, x3, [sp, #(8 * 0)] + ldp x0, x1, [sp, #(8 * 2)] + add sp, sp, #(8 * 4) +1: .org __smccc_workaround_3_smc + __SMCCC_WORKAROUND_3_SMC_SZ + .org 1b +SYM_DATA_END(__smccc_workaround_3_smc) + + .global __spectre_bhb_loop_k8 +SYM_DATA_START(__spectre_bhb_loop_k8) + esb + sub sp, sp, #(8 * 2) + stp x0, x1, [sp, #(8 * 0)] + mov x0, #8 +2: b . + 4 + subs x0, x0, #1 + b.ne 2b + dsb nsh + isb + ldp x0, x1, [sp, #(8 * 0)] + add sp, sp, #(8 * 2) +1: .org __spectre_bhb_loop_k8 + __SPECTRE_BHB_LOOP_SZ + .org 1b +SYM_DATA_END(__spectre_bhb_loop_k8) + + .global __spectre_bhb_loop_k24 +SYM_DATA_START(__spectre_bhb_loop_k24) + esb + sub sp, sp, #(8 * 2) + stp x0, x1, [sp, #(8 * 0)] + mov x0, #8 +2: b . + 4 + subs x0, x0, #1 + b.ne 2b + dsb nsh + isb + ldp x0, x1, [sp, #(8 * 0)] + add sp, sp, #(8 * 2) +1: .org __spectre_bhb_loop_k24 + __SPECTRE_BHB_LOOP_SZ + .org 1b +SYM_DATA_END(__spectre_bhb_loop_k24) + + .global __spectre_bhb_loop_k32 +SYM_DATA_START(__spectre_bhb_loop_k32) + esb + sub sp, sp, #(8 * 2) + stp x0, x1, [sp, #(8 * 0)] + mov x0, #8 +2: b . + 4 + subs x0, x0, #1 + b.ne 2b + dsb nsh + isb + ldp x0, x1, [sp, #(8 * 0)] + add sp, sp, #(8 * 2) +1: .org __spectre_bhb_loop_k32 + __SPECTRE_BHB_LOOP_SZ + .org 1b +SYM_DATA_END(__spectre_bhb_loop_k32) From nobody Tue Jun 23 05:03:54 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 52B2CC4321E for ; Thu, 10 Mar 2022 14:53:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344602AbiCJOxi (ORCPT ); Thu, 10 Mar 2022 09:53:38 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49558 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243024AbiCJObk (ORCPT ); Thu, 10 Mar 2022 09:31:40 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 173818E194; Thu, 10 Mar 2022 06:30:22 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 05A66B82672; Thu, 10 Mar 2022 14:25:19 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 69821C340E8; Thu, 10 Mar 2022 14:25:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1646922317; bh=sGZ8WQW5JZScPSyh92/G1jjpe6ZP/eHiJZhx/V6WF/k=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=aGFO38BxjML+m55mC8JNjM2ahSaRkoPmgU8HECoJSANlxaN83Ezo4UYEd1z8Kuy/W c0h3FOnXeDBSPS1D+o9sFU+qf7H6/v0aGloolJ+9TMKI4NnhzfipqJZ8XSDHAKTYo7 RnuMKexEr7KQczwHxp+8ICKmOwFq65tW3gQ2yWNI= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Catalin Marinas , James Morse Subject: [PATCH 5.10 39/58] arm64: Mitigate spectre style branch history side channels Date: Thu, 10 Mar 2022 15:18:59 +0100 Message-Id: <20220310140813.984380513@linuxfoundation.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220310140812.869208747@linuxfoundation.org> References: <20220310140812.869208747@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: James Morse commit 558c303c9734af5a813739cd284879227f7297d2 upstream. Speculation attacks against some high-performance processors can make use of branch history to influence future speculation. When taking an exception from user-space, a sequence of branches or a firmware call overwrites or invalidates the branch history. The sequence of branches is added to the vectors, and should appear before the first indirect branch. For systems using KPTI the sequence is added to the kpti trampoline where it has a free register as the exit from the trampoline is via a 'ret'. For systems not using KPTI, the same register tricks are used to free up a register in the vectors. For the firmware call, arch-workaround-3 clobbers 4 registers, so there is no choice but to save them to the EL1 stack. This only happens for entry from EL0, so if we take an exception due to the stack access, it will not become re-entrant. For KVM, the existing branch-predictor-hardening vectors are used. When a spectre version of these vectors is in use, the firmware call is sufficient to mitigate against Spectre-BHB. For the non-spectre versions, the sequence of branches is added to the indirect vector. Reviewed-by: Catalin Marinas [ modified for stable, removed bitmap of mitigations, use kvm template infrastructure ] Signed-off-by: James Morse Signed-off-by: Greg Kroah-Hartman Tested-by: Bagas Sanjaya Tested-by: Florian Fainelli Tested-by: Fox Chen Tested-by: Guenter Roeck Tested-by: Linux Kernel Functional Testing Tested-by: Pavel Machek (CIP) = =20 Tested-by: Shuah Khan Tested-by: Sudip Mukherjee --- arch/arm64/Kconfig | 9 + arch/arm64/include/asm/assembler.h | 4=20 arch/arm64/include/asm/cpufeature.h | 15 ++ arch/arm64/include/asm/cputype.h | 8 + arch/arm64/include/asm/spectre.h | 4=20 arch/arm64/include/asm/sysreg.h | 1=20 arch/arm64/include/asm/vectors.h | 5=20 arch/arm64/kernel/cpu_errata.c | 7 + arch/arm64/kernel/proton-pack.c | 234 +++++++++++++++++++++++++++++++= ++++- arch/arm64/kvm/hyp/hyp-entry.S | 4=20 10 files changed, 288 insertions(+), 3 deletions(-) --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -1184,6 +1184,15 @@ config UNMAP_KERNEL_AT_EL0 =20 If unsure, say Y. =20 +config MITIGATE_SPECTRE_BRANCH_HISTORY + bool "Mitigate Spectre style attacks against branch history" if EXPERT + default y + help + Speculation attacks against some high-performance processors can + make use of branch history to influence future speculation. + When taking an exception from user-space, a sequence of branches + or a firmware call overwrites the branch history. + config RODATA_FULL_DEFAULT_ENABLED bool "Apply r/o permissions of VM areas also to their linear aliases" default y --- a/arch/arm64/include/asm/assembler.h +++ b/arch/arm64/include/asm/assembler.h @@ -797,7 +797,9 @@ USER(\label, ic ivau, \tmp2) // invali =20 .macro __mitigate_spectre_bhb_loop tmp #ifdef CONFIG_MITIGATE_SPECTRE_BRANCH_HISTORY - mov \tmp, #32 +alternative_cb spectre_bhb_patch_loop_iter + mov \tmp, #32 // Patched to correct the immediate +alternative_cb_end .Lspectre_bhb_loop\@: b . + 4 subs \tmp, \tmp, #1 --- a/arch/arm64/include/asm/cpufeature.h +++ b/arch/arm64/include/asm/cpufeature.h @@ -606,6 +606,21 @@ static inline bool cpu_supports_mixed_en return id_aa64mmfr0_mixed_endian_el0(read_cpuid(ID_AA64MMFR0_EL1)); } =20 +static inline bool supports_csv2p3(int scope) +{ + u64 pfr0; + u8 csv2_val; + + if (scope =3D=3D SCOPE_LOCAL_CPU) + pfr0 =3D read_sysreg_s(SYS_ID_AA64PFR0_EL1); + else + pfr0 =3D read_sanitised_ftr_reg(SYS_ID_AA64PFR0_EL1); + + csv2_val =3D cpuid_feature_extract_unsigned_field(pfr0, + ID_AA64PFR0_CSV2_SHIFT); + return csv2_val =3D=3D 3; +} + static inline bool system_supports_32bit_el0(void) { return cpus_have_const_cap(ARM64_HAS_32BIT_EL0); --- a/arch/arm64/include/asm/cputype.h +++ b/arch/arm64/include/asm/cputype.h @@ -73,10 +73,14 @@ #define ARM_CPU_PART_CORTEX_A76 0xD0B #define ARM_CPU_PART_NEOVERSE_N1 0xD0C #define ARM_CPU_PART_CORTEX_A77 0xD0D +#define ARM_CPU_PART_NEOVERSE_V1 0xD40 +#define ARM_CPU_PART_CORTEX_A78 0xD41 +#define ARM_CPU_PART_CORTEX_X1 0xD44 #define ARM_CPU_PART_CORTEX_A510 0xD46 #define ARM_CPU_PART_CORTEX_A710 0xD47 #define ARM_CPU_PART_CORTEX_X2 0xD48 #define ARM_CPU_PART_NEOVERSE_N2 0xD49 +#define ARM_CPU_PART_CORTEX_A78C 0xD4B =20 #define APM_CPU_PART_POTENZA 0x000 =20 @@ -117,10 +121,14 @@ #define MIDR_CORTEX_A76 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTE= X_A76) #define MIDR_NEOVERSE_N1 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_NEOV= ERSE_N1) #define MIDR_CORTEX_A77 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTE= X_A77) +#define MIDR_NEOVERSE_V1 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_NEOV= ERSE_V1) +#define MIDR_CORTEX_A78 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTE= X_A78) +#define MIDR_CORTEX_X1 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX= _X1) #define MIDR_CORTEX_A510 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORT= EX_A510) #define MIDR_CORTEX_A710 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORT= EX_A710) #define MIDR_CORTEX_X2 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX= _X2) #define MIDR_NEOVERSE_N2 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_NEOV= ERSE_N2) +#define MIDR_CORTEX_A78C MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORT= EX_A78C) #define MIDR_THUNDERX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_T= HUNDERX) #define MIDR_THUNDERX_81XX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_P= ART_THUNDERX_81XX) #define MIDR_THUNDERX_83XX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_P= ART_THUNDERX_83XX) --- a/arch/arm64/include/asm/spectre.h +++ b/arch/arm64/include/asm/spectre.h @@ -30,5 +30,7 @@ void spectre_v4_enable_mitigation(const void spectre_v4_enable_task_mitigation(struct task_struct *tsk); =20 enum mitigation_state arm64_get_spectre_bhb_state(void); - +bool is_spectre_bhb_affected(const struct arm64_cpu_capabilities *entry, i= nt scope); +u8 spectre_bhb_loop_affected(int scope); +void spectre_bhb_enable_mitigation(const struct arm64_cpu_capabilities *__= unused); #endif /* __ASM_SPECTRE_H */ --- a/arch/arm64/include/asm/sysreg.h +++ b/arch/arm64/include/asm/sysreg.h @@ -801,6 +801,7 @@ #endif =20 /* id_aa64mmfr1 */ +#define ID_AA64MMFR1_ECBHB_SHIFT 60 #define ID_AA64MMFR1_AFP_SHIFT 44 #define ID_AA64MMFR1_ETS_SHIFT 36 #define ID_AA64MMFR1_TWED_SHIFT 32 --- a/arch/arm64/include/asm/vectors.h +++ b/arch/arm64/include/asm/vectors.h @@ -40,6 +40,11 @@ enum arm64_bp_harden_el1_vectors { EL1_VECTOR_KPTI, }; =20 +#ifndef CONFIG_MITIGATE_SPECTRE_BRANCH_HISTORY +#define EL1_VECTOR_BHB_LOOP -1 +#define EL1_VECTOR_BHB_FW -1 +#endif /* !CONFIG_MITIGATE_SPECTRE_BRANCH_HISTORY */ + /* The vectors to use on return from EL0. e.g. to remap the kernel */ DECLARE_PER_CPU_READ_MOSTLY(const char *, this_cpu_vector); =20 --- a/arch/arm64/kernel/cpu_errata.c +++ b/arch/arm64/kernel/cpu_errata.c @@ -473,6 +473,13 @@ const struct arm64_cpu_capabilities arm6 .matches =3D has_spectre_v4, .cpu_enable =3D spectre_v4_enable_mitigation, }, + { + .desc =3D "Spectre-BHB", + .capability =3D ARM64_SPECTRE_BHB, + .type =3D ARM64_CPUCAP_LOCAL_CPU_ERRATUM, + .matches =3D is_spectre_bhb_affected, + .cpu_enable =3D spectre_bhb_enable_mitigation, + }, #ifdef CONFIG_ARM64_ERRATUM_1418040 { .desc =3D "ARM erratum 1418040", --- a/arch/arm64/kernel/proton-pack.c +++ b/arch/arm64/kernel/proton-pack.c @@ -24,8 +24,11 @@ #include #include =20 +#include #include #include +#include +#include =20 /* * We try to ensure that the mitigation state can never change as the resu= lt of @@ -814,6 +817,17 @@ int arch_prctl_spec_ctrl_get(struct task } } =20 +/* + * Spectre BHB. + * + * A CPU is either: + * - Mitigated by a branchy loop a CPU specific number of times, and listed + * in our "loop mitigated list". + * - Mitigated in software by the firmware Spectre v2 call. + * - Has the 'Exception Clears Branch History Buffer' (ECBHB) feature, so = no + * software mitigation in the vectors is needed. + * - Has CSV2.3, so is unaffected. + */ static enum mitigation_state spectre_bhb_state; =20 enum mitigation_state arm64_get_spectre_bhb_state(void) @@ -821,6 +835,150 @@ enum mitigation_state arm64_get_spectre_ return spectre_bhb_state; } =20 +/* + * This must be called with SCOPE_LOCAL_CPU for each type of CPU, before a= ny + * SCOPE_SYSTEM call will give the right answer. + */ +u8 spectre_bhb_loop_affected(int scope) +{ + u8 k =3D 0; + static u8 max_bhb_k; + + if (scope =3D=3D SCOPE_LOCAL_CPU) { + static const struct midr_range spectre_bhb_k32_list[] =3D { + MIDR_ALL_VERSIONS(MIDR_CORTEX_A78), + MIDR_ALL_VERSIONS(MIDR_CORTEX_A78C), + MIDR_ALL_VERSIONS(MIDR_CORTEX_X1), + MIDR_ALL_VERSIONS(MIDR_CORTEX_A710), + MIDR_ALL_VERSIONS(MIDR_CORTEX_X2), + MIDR_ALL_VERSIONS(MIDR_NEOVERSE_N2), + MIDR_ALL_VERSIONS(MIDR_NEOVERSE_V1), + {}, + }; + static const struct midr_range spectre_bhb_k24_list[] =3D { + MIDR_ALL_VERSIONS(MIDR_CORTEX_A76), + MIDR_ALL_VERSIONS(MIDR_CORTEX_A77), + MIDR_ALL_VERSIONS(MIDR_NEOVERSE_N1), + {}, + }; + static const struct midr_range spectre_bhb_k8_list[] =3D { + MIDR_ALL_VERSIONS(MIDR_CORTEX_A72), + MIDR_ALL_VERSIONS(MIDR_CORTEX_A57), + {}, + }; + + if (is_midr_in_range_list(read_cpuid_id(), spectre_bhb_k32_list)) + k =3D 32; + else if (is_midr_in_range_list(read_cpuid_id(), spectre_bhb_k24_list)) + k =3D 24; + else if (is_midr_in_range_list(read_cpuid_id(), spectre_bhb_k8_list)) + k =3D 8; + + max_bhb_k =3D max(max_bhb_k, k); + } else { + k =3D max_bhb_k; + } + + return k; +} + +static enum mitigation_state spectre_bhb_get_cpu_fw_mitigation_state(void) +{ + int ret; + struct arm_smccc_res res; + + arm_smccc_1_1_invoke(ARM_SMCCC_ARCH_FEATURES_FUNC_ID, + ARM_SMCCC_ARCH_WORKAROUND_3, &res); + + ret =3D res.a0; + switch (ret) { + case SMCCC_RET_SUCCESS: + return SPECTRE_MITIGATED; + case SMCCC_ARCH_WORKAROUND_RET_UNAFFECTED: + return SPECTRE_UNAFFECTED; + default: + fallthrough; + case SMCCC_RET_NOT_SUPPORTED: + return SPECTRE_VULNERABLE; + } +} + +static bool is_spectre_bhb_fw_affected(int scope) +{ + static bool system_affected; + enum mitigation_state fw_state; + bool has_smccc =3D arm_smccc_1_1_get_conduit() !=3D SMCCC_CONDUIT_NONE; + static const struct midr_range spectre_bhb_firmware_mitigated_list[] =3D { + MIDR_ALL_VERSIONS(MIDR_CORTEX_A73), + MIDR_ALL_VERSIONS(MIDR_CORTEX_A75), + {}, + }; + bool cpu_in_list =3D is_midr_in_range_list(read_cpuid_id(), + spectre_bhb_firmware_mitigated_list); + + if (scope !=3D SCOPE_LOCAL_CPU) + return system_affected; + + fw_state =3D spectre_bhb_get_cpu_fw_mitigation_state(); + if (cpu_in_list || (has_smccc && fw_state =3D=3D SPECTRE_MITIGATED)) { + system_affected =3D true; + return true; + } + + return false; +} + +static bool supports_ecbhb(int scope) +{ + u64 mmfr1; + + if (scope =3D=3D SCOPE_LOCAL_CPU) + mmfr1 =3D read_sysreg_s(SYS_ID_AA64MMFR1_EL1); + else + mmfr1 =3D read_sanitised_ftr_reg(SYS_ID_AA64MMFR1_EL1); + + return cpuid_feature_extract_unsigned_field(mmfr1, + ID_AA64MMFR1_ECBHB_SHIFT); +} + +bool is_spectre_bhb_affected(const struct arm64_cpu_capabilities *entry, + int scope) +{ + WARN_ON(scope !=3D SCOPE_LOCAL_CPU || preemptible()); + + if (supports_csv2p3(scope)) + return false; + + if (spectre_bhb_loop_affected(scope)) + return true; + + if (is_spectre_bhb_fw_affected(scope)) + return true; + + return false; +} + +static void this_cpu_set_vectors(enum arm64_bp_harden_el1_vectors slot) +{ + const char *v =3D arm64_get_bp_hardening_vector(slot); + + if (slot < 0) + return; + + __this_cpu_write(this_cpu_vector, v); + + /* + * When KPTI is in use, the vectors are switched when exiting to + * user-space. + */ + if (arm64_kernel_unmapped_at_el0()) + return; + + write_sysreg(v, vbar_el1); + isb(); +} + +#ifdef CONFIG_KVM static int kvm_bhb_get_vecs_size(const char *start) { if (start =3D=3D __smccc_workaround_3_smc) @@ -833,7 +991,7 @@ static int kvm_bhb_get_vecs_size(const c return 0; } =20 -void kvm_setup_bhb_slot(const char *hyp_vecs_start) +static void kvm_setup_bhb_slot(const char *hyp_vecs_start) { int cpu, slot =3D -1, size; const char *hyp_vecs_end; @@ -864,3 +1022,77 @@ void kvm_setup_bhb_slot(const char *hyp_ __this_cpu_write(bp_hardening_data.template_start, hyp_vecs_start); raw_spin_unlock(&bp_lock); } +#else +#define __smccc_workaround_3_smc NULL +#define __spectre_bhb_loop_k8 NULL +#define __spectre_bhb_loop_k24 NULL +#define __spectre_bhb_loop_k32 NULL + +static void kvm_setup_bhb_slot(const char *hyp_vecs_start) { } +#endif /* CONFIG_KVM */ + +void spectre_bhb_enable_mitigation(const struct arm64_cpu_capabilities *en= try) +{ + enum mitigation_state fw_state, state =3D SPECTRE_VULNERABLE; + + if (!is_spectre_bhb_affected(entry, SCOPE_LOCAL_CPU)) + return; + + if (arm64_get_spectre_v2_state() =3D=3D SPECTRE_VULNERABLE) { + /* No point mitigating Spectre-BHB alone. */ + } else if (!IS_ENABLED(CONFIG_MITIGATE_SPECTRE_BRANCH_HISTORY)) { + pr_info_once("spectre-bhb mitigation disabled by compile time option\n"); + } else if (cpu_mitigations_off()) { + pr_info_once("spectre-bhb mitigation disabled by command line option\n"); + } else if (supports_ecbhb(SCOPE_LOCAL_CPU)) { + state =3D SPECTRE_MITIGATED; + } else if (spectre_bhb_loop_affected(SCOPE_LOCAL_CPU)) { + switch (spectre_bhb_loop_affected(SCOPE_SYSTEM)) { + case 8: + kvm_setup_bhb_slot(__spectre_bhb_loop_k8); + break; + case 24: + kvm_setup_bhb_slot(__spectre_bhb_loop_k24); + break; + case 32: + kvm_setup_bhb_slot(__spectre_bhb_loop_k32); + break; + default: + WARN_ON_ONCE(1); + } + this_cpu_set_vectors(EL1_VECTOR_BHB_LOOP); + + state =3D SPECTRE_MITIGATED; + } else if (is_spectre_bhb_fw_affected(SCOPE_LOCAL_CPU)) { + fw_state =3D spectre_bhb_get_cpu_fw_mitigation_state(); + if (fw_state =3D=3D SPECTRE_MITIGATED) { + kvm_setup_bhb_slot(__smccc_workaround_3_smc); + this_cpu_set_vectors(EL1_VECTOR_BHB_FW); + + state =3D SPECTRE_MITIGATED; + } + } + + update_mitigation_state(&spectre_bhb_state, state); +} + +/* Patched to correct the immediate */ +void noinstr spectre_bhb_patch_loop_iter(struct alt_instr *alt, + __le32 *origptr, __le32 *updptr, int nr_inst) +{ + u8 rd; + u32 insn; + u16 loop_count =3D spectre_bhb_loop_affected(SCOPE_SYSTEM); + + BUG_ON(nr_inst !=3D 1); /* MOV -> MOV */ + + if (!IS_ENABLED(CONFIG_MITIGATE_SPECTRE_BRANCH_HISTORY)) + return; + + insn =3D le32_to_cpu(*origptr); + rd =3D aarch64_insn_decode_register(AARCH64_INSN_REGTYPE_RD, insn); + insn =3D aarch64_insn_gen_movewide(rd, loop_count, 0, + AARCH64_INSN_VARIANT_64BIT, + AARCH64_INSN_MOVEWIDE_ZERO); + *updptr++ =3D cpu_to_le32(insn); +} --- a/arch/arm64/kvm/hyp/hyp-entry.S +++ b/arch/arm64/kvm/hyp/hyp-entry.S @@ -61,6 +61,10 @@ el1_sync: // Guest trapped into EL2 /* ARM_SMCCC_ARCH_WORKAROUND_2 handling */ eor w1, w1, #(ARM_SMCCC_ARCH_WORKAROUND_1 ^ \ ARM_SMCCC_ARCH_WORKAROUND_2) + cbz w1, wa_epilogue + + eor w1, w1, #(ARM_SMCCC_ARCH_WORKAROUND_2 ^ \ + ARM_SMCCC_ARCH_WORKAROUND_3) cbnz w1, el1_trap =20 wa_epilogue: From nobody Tue Jun 23 05:03:54 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A9DF6C35274 for ; Thu, 10 Mar 2022 14:42:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344990AbiCJOl1 (ORCPT ); Thu, 10 Mar 2022 09:41:27 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51166 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245041AbiCJO3u (ORCPT ); Thu, 10 Mar 2022 09:29:50 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A886915C9F2; Thu, 10 Mar 2022 06:25:22 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 43845B825A7; Thu, 10 Mar 2022 14:25:22 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8E60DC340E8; Thu, 10 Mar 2022 14:25:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1646922320; bh=CKv9siGuusoENM2DBRrXprlaYHpaPdR7ANqVe2dSUZk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=0Xi4CsSbief1BCa+XEBg2inAg3Z3XPrqKGCVFl9yVUj2DC2r+RKb17TzIlkK8IZ6x FmwQ/MIxLlvoQItSxcg6gucBliIpL9qvOUlsTvOutNJ6WNEcJNLHE1emAguJn+Yeow m7UrM9biVgU1R+gpAg6okRWLYLLBlzdS3QPwoFSo= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, "Russell King (Oracle)" , Catalin Marinas , James Morse Subject: [PATCH 5.10 40/58] KVM: arm64: Allow SMCCC_ARCH_WORKAROUND_3 to be discovered and migrated Date: Thu, 10 Mar 2022 15:19:00 +0100 Message-Id: <20220310140814.012332345@linuxfoundation.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220310140812.869208747@linuxfoundation.org> References: <20220310140812.869208747@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: James Morse commit a5905d6af492ee6a4a2205f0d550b3f931b03d03 upstream. KVM allows the guest to discover whether the ARCH_WORKAROUND SMCCC are implemented, and to preserve that state during migration through its firmware register interface. Add the necessary boiler plate for SMCCC_ARCH_WORKAROUND_3. Reviewed-by: Russell King (Oracle) Reviewed-by: Catalin Marinas Signed-off-by: James Morse Signed-off-by: Greg Kroah-Hartman Tested-by: Bagas Sanjaya Tested-by: Florian Fainelli Tested-by: Fox Chen Tested-by: Guenter Roeck Tested-by: Linux Kernel Functional Testing Tested-by: Pavel Machek (CIP) = =20 Tested-by: Shuah Khan Tested-by: Sudip Mukherjee --- arch/arm64/include/uapi/asm/kvm.h | 5 +++++ arch/arm64/kvm/hypercalls.c | 12 ++++++++++++ arch/arm64/kvm/psci.c | 18 +++++++++++++++++- 3 files changed, 34 insertions(+), 1 deletion(-) --- a/arch/arm64/include/uapi/asm/kvm.h +++ b/arch/arm64/include/uapi/asm/kvm.h @@ -273,6 +273,11 @@ struct kvm_vcpu_events { #define KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_2_NOT_REQUIRED 3 #define KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_2_ENABLED (1U << 4) =20 +#define KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_3 KVM_REG_ARM_FW_REG(3) +#define KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_3_NOT_AVAIL 0 +#define KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_3_AVAIL 1 +#define KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_3_NOT_REQUIRED 2 + /* SVE registers */ #define KVM_REG_ARM64_SVE (0x15 << KVM_REG_ARM_COPROC_SHIFT) =20 --- a/arch/arm64/kvm/hypercalls.c +++ b/arch/arm64/kvm/hypercalls.c @@ -58,6 +58,18 @@ int kvm_hvc_call_handler(struct kvm_vcpu break; } break; + case ARM_SMCCC_ARCH_WORKAROUND_3: + switch (arm64_get_spectre_bhb_state()) { + case SPECTRE_VULNERABLE: + break; + case SPECTRE_MITIGATED: + val =3D SMCCC_RET_SUCCESS; + break; + case SPECTRE_UNAFFECTED: + val =3D SMCCC_ARCH_WORKAROUND_RET_UNAFFECTED; + break; + } + break; case ARM_SMCCC_HV_PV_TIME_FEATURES: val =3D SMCCC_RET_SUCCESS; break; --- a/arch/arm64/kvm/psci.c +++ b/arch/arm64/kvm/psci.c @@ -397,7 +397,7 @@ int kvm_psci_call(struct kvm_vcpu *vcpu) =20 int kvm_arm_get_fw_num_regs(struct kvm_vcpu *vcpu) { - return 3; /* PSCI version and two workaround registers */ + return 4; /* PSCI version and three workaround registers */ } =20 int kvm_arm_copy_fw_reg_indices(struct kvm_vcpu *vcpu, u64 __user *uindice= s) @@ -411,6 +411,9 @@ int kvm_arm_copy_fw_reg_indices(struct k if (put_user(KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_2, uindices++)) return -EFAULT; =20 + if (put_user(KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_3, uindices++)) + return -EFAULT; + return 0; } =20 @@ -450,6 +453,17 @@ static int get_kernel_wa_level(u64 regid case SPECTRE_VULNERABLE: return KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_2_NOT_AVAIL; } + break; + case KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_3: + switch (arm64_get_spectre_bhb_state()) { + case SPECTRE_VULNERABLE: + return KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_3_NOT_AVAIL; + case SPECTRE_MITIGATED: + return KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_3_AVAIL; + case SPECTRE_UNAFFECTED: + return KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_3_NOT_REQUIRED; + } + return KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_3_NOT_AVAIL; } =20 return -EINVAL; @@ -466,6 +480,7 @@ int kvm_arm_get_fw_reg(struct kvm_vcpu * break; case KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_1: case KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_2: + case KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_3: val =3D get_kernel_wa_level(reg->id) & KVM_REG_FEATURE_LEVEL_MASK; break; default: @@ -511,6 +526,7 @@ int kvm_arm_set_fw_reg(struct kvm_vcpu * } =20 case KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_1: + case KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_3: if (val & ~KVM_REG_FEATURE_LEVEL_MASK) return -EINVAL; From nobody Tue Jun 23 05:03:54 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DABBDC433F5 for ; Thu, 10 Mar 2022 14:36:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231995AbiCJOg6 (ORCPT ); Thu, 10 Mar 2022 09:36:58 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50046 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245121AbiCJOaI (ORCPT ); Thu, 10 Mar 2022 09:30:08 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9A28B167F9C; Thu, 10 Mar 2022 06:25:31 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 193A6B82544; Thu, 10 Mar 2022 14:25:25 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 88066C36AE5; Thu, 10 Mar 2022 14:25:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1646922323; bh=dUq87xpJ+d6NqExqnKaLDlps4HA8AtfE6Mg9Q3ce9NY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=X8eMDNculFE2qXXPYBnRhRKAPdD83IWKEg+IQnQ0qDnnrW6wu7lL3C+A0Vw+WTDBd jV8efaeYjwcKA3aICiuBvG4IYWAM0ZeAs5UF3zKf0/rGVksB7Kq5JkZiVFkT2Svs6h phJ9HATLEx75dZuw2zO27Vh93M/TCGO/K+EFAx3Y= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, "Russell King (Oracle)" , Catalin Marinas , James Morse Subject: [PATCH 5.10 41/58] arm64: Use the clearbhb instruction in mitigations Date: Thu, 10 Mar 2022 15:19:01 +0100 Message-Id: <20220310140814.040787017@linuxfoundation.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220310140812.869208747@linuxfoundation.org> References: <20220310140812.869208747@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: James Morse commit 228a26b912287934789023b4132ba76065d9491c upstream. Future CPUs may implement a clearbhb instruction that is sufficient to mitigate SpectreBHB. CPUs that implement this instruction, but not CSV2.3 must be affected by Spectre-BHB. Add support to use this instruction as the BHB mitigation on CPUs that support it. The instruction is in the hint space, so it will be treated by a NOP as older CPUs. Reviewed-by: Russell King (Oracle) Reviewed-by: Catalin Marinas [ modified for stable: Use a KVM vector template instead of alternatives, removed bitmap of mitigations ] Signed-off-by: James Morse Signed-off-by: Greg Kroah-Hartman Tested-by: Bagas Sanjaya Tested-by: Florian Fainelli Tested-by: Fox Chen Tested-by: Guenter Roeck Tested-by: Linux Kernel Functional Testing Tested-by: Pavel Machek (CIP) = =20 Tested-by: Shuah Khan Tested-by: Sudip Mukherjee --- arch/arm64/include/asm/assembler.h | 7 +++++++ arch/arm64/include/asm/cpufeature.h | 13 +++++++++++++ arch/arm64/include/asm/insn.h | 1 + arch/arm64/include/asm/kvm_asm.h | 2 ++ arch/arm64/include/asm/sysreg.h | 1 + arch/arm64/include/asm/vectors.h | 7 +++++++ arch/arm64/kernel/cpufeature.c | 1 + arch/arm64/kernel/entry.S | 8 ++++++++ arch/arm64/kernel/proton-pack.c | 12 ++++++++++++ arch/arm64/kvm/hyp/smccc_wa.S | 9 +++++++++ 10 files changed, 61 insertions(+) --- a/arch/arm64/include/asm/assembler.h +++ b/arch/arm64/include/asm/assembler.h @@ -98,6 +98,13 @@ .endm =20 /* + * Clear Branch History instruction + */ + .macro clearbhb + hint #22 + .endm + +/* * Speculation barrier */ .macro sb --- a/arch/arm64/include/asm/cpufeature.h +++ b/arch/arm64/include/asm/cpufeature.h @@ -621,6 +621,19 @@ static inline bool supports_csv2p3(int s return csv2_val =3D=3D 3; } =20 +static inline bool supports_clearbhb(int scope) +{ + u64 isar2; + + if (scope =3D=3D SCOPE_LOCAL_CPU) + isar2 =3D read_sysreg_s(SYS_ID_AA64ISAR2_EL1); + else + isar2 =3D read_sanitised_ftr_reg(SYS_ID_AA64ISAR2_EL1); + + return cpuid_feature_extract_unsigned_field(isar2, + ID_AA64ISAR2_CLEARBHB_SHIFT); +} + static inline bool system_supports_32bit_el0(void) { return cpus_have_const_cap(ARM64_HAS_32BIT_EL0); --- a/arch/arm64/include/asm/insn.h +++ b/arch/arm64/include/asm/insn.h @@ -65,6 +65,7 @@ enum aarch64_insn_hint_cr_op { AARCH64_INSN_HINT_PSB =3D 0x11 << 5, AARCH64_INSN_HINT_TSB =3D 0x12 << 5, AARCH64_INSN_HINT_CSDB =3D 0x14 << 5, + AARCH64_INSN_HINT_CLEARBHB =3D 0x16 << 5, =20 AARCH64_INSN_HINT_BTI =3D 0x20 << 5, AARCH64_INSN_HINT_BTIC =3D 0x22 << 5, --- a/arch/arm64/include/asm/kvm_asm.h +++ b/arch/arm64/include/asm/kvm_asm.h @@ -37,6 +37,7 @@ #define __SMCCC_WORKAROUND_1_SMC_SZ 36 #define __SMCCC_WORKAROUND_3_SMC_SZ 36 #define __SPECTRE_BHB_LOOP_SZ 44 +#define __SPECTRE_BHB_CLEARBHB_SZ 12 =20 #define KVM_HOST_SMCCC_ID(id) \ ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, \ @@ -205,6 +206,7 @@ extern char __smccc_workaround_3_smc[__S extern char __spectre_bhb_loop_k8[__SPECTRE_BHB_LOOP_SZ]; extern char __spectre_bhb_loop_k24[__SPECTRE_BHB_LOOP_SZ]; extern char __spectre_bhb_loop_k32[__SPECTRE_BHB_LOOP_SZ]; +extern char __spectre_bhb_clearbhb[__SPECTRE_BHB_LOOP_SZ]; =20 /* * Obtain the PC-relative address of a kernel symbol --- a/arch/arm64/include/asm/sysreg.h +++ b/arch/arm64/include/asm/sysreg.h @@ -689,6 +689,7 @@ #define ID_AA64ISAR1_GPI_IMP_DEF 0x1 =20 /* id_aa64isar2 */ +#define ID_AA64ISAR2_CLEARBHB_SHIFT 28 #define ID_AA64ISAR2_RPRES_SHIFT 4 #define ID_AA64ISAR2_WFXT_SHIFT 0 =20 --- a/arch/arm64/include/asm/vectors.h +++ b/arch/arm64/include/asm/vectors.h @@ -32,6 +32,12 @@ enum arm64_bp_harden_el1_vectors { * canonical vectors. */ EL1_VECTOR_BHB_FW, + + /* + * Use the ClearBHB instruction, before branching to the canonical + * vectors. + */ + EL1_VECTOR_BHB_CLEAR_INSN, #endif /* CONFIG_MITIGATE_SPECTRE_BRANCH_HISTORY */ =20 /* @@ -43,6 +49,7 @@ enum arm64_bp_harden_el1_vectors { #ifndef CONFIG_MITIGATE_SPECTRE_BRANCH_HISTORY #define EL1_VECTOR_BHB_LOOP -1 #define EL1_VECTOR_BHB_FW -1 +#define EL1_VECTOR_BHB_CLEAR_INSN -1 #endif /* !CONFIG_MITIGATE_SPECTRE_BRANCH_HISTORY */ =20 /* The vectors to use on return from EL0. e.g. to remap the kernel */ --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -211,6 +211,7 @@ static const struct arm64_ftr_bits ftr_i }; =20 static const struct arm64_ftr_bits ftr_id_aa64isar2[] =3D { + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_HIGHER_SAFE, ID_AA64ISAR2_CLEA= RBHB_SHIFT, 4, 0), ARM64_FTR_BITS(FTR_VISIBLE, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64ISAR2_R= PRES_SHIFT, 4, 0), ARM64_FTR_END, }; --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -827,6 +827,7 @@ alternative_else_nop_endif #define BHB_MITIGATION_NONE 0 #define BHB_MITIGATION_LOOP 1 #define BHB_MITIGATION_FW 2 +#define BHB_MITIGATION_INSN 3 =20 .macro tramp_ventry, vector_start, regsize, kpti, bhb .align 7 @@ -843,6 +844,11 @@ alternative_else_nop_endif __mitigate_spectre_bhb_loop x30 .endif // \bhb =3D=3D BHB_MITIGATION_LOOP =20 + .if \bhb =3D=3D BHB_MITIGATION_INSN + clearbhb + isb + .endif // \bhb =3D=3D BHB_MITIGATION_INSN + .if \kpti =3D=3D 1 /* * Defend against branch aliasing attacks by pushing a dummy @@ -919,6 +925,7 @@ SYM_CODE_START_NOALIGN(tramp_vectors) #ifdef CONFIG_MITIGATE_SPECTRE_BRANCH_HISTORY generate_tramp_vector kpti=3D1, bhb=3DBHB_MITIGATION_LOOP generate_tramp_vector kpti=3D1, bhb=3DBHB_MITIGATION_FW + generate_tramp_vector kpti=3D1, bhb=3DBHB_MITIGATION_INSN #endif /* CONFIG_MITIGATE_SPECTRE_BRANCH_HISTORY */ generate_tramp_vector kpti=3D1, bhb=3DBHB_MITIGATION_NONE SYM_CODE_END(tramp_vectors) @@ -981,6 +988,7 @@ SYM_CODE_START(__bp_harden_el1_vectors) #ifdef CONFIG_MITIGATE_SPECTRE_BRANCH_HISTORY generate_el1_vector bhb=3DBHB_MITIGATION_LOOP generate_el1_vector bhb=3DBHB_MITIGATION_FW + generate_el1_vector bhb=3DBHB_MITIGATION_INSN #endif /* CONFIG_MITIGATE_SPECTRE_BRANCH_HISTORY */ SYM_CODE_END(__bp_harden_el1_vectors) .popsection --- a/arch/arm64/kernel/proton-pack.c +++ b/arch/arm64/kernel/proton-pack.c @@ -824,6 +824,7 @@ int arch_prctl_spec_ctrl_get(struct task * - Mitigated by a branchy loop a CPU specific number of times, and listed * in our "loop mitigated list". * - Mitigated in software by the firmware Spectre v2 call. + * - Has the ClearBHB instruction to perform the mitigation. * - Has the 'Exception Clears Branch History Buffer' (ECBHB) feature, so = no * software mitigation in the vectors is needed. * - Has CSV2.3, so is unaffected. @@ -949,6 +950,9 @@ bool is_spectre_bhb_affected(const struc if (supports_csv2p3(scope)) return false; =20 + if (supports_clearbhb(scope)) + return true; + if (spectre_bhb_loop_affected(scope)) return true; =20 @@ -987,6 +991,8 @@ static int kvm_bhb_get_vecs_size(const c start =3D=3D __spectre_bhb_loop_k24 || start =3D=3D __spectre_bhb_loop_k32) return __SPECTRE_BHB_LOOP_SZ; + else if (start =3D=3D __spectre_bhb_clearbhb) + return __SPECTRE_BHB_CLEARBHB_SZ; =20 return 0; } @@ -1027,6 +1033,7 @@ static void kvm_setup_bhb_slot(const cha #define __spectre_bhb_loop_k8 NULL #define __spectre_bhb_loop_k24 NULL #define __spectre_bhb_loop_k32 NULL +#define __spectre_bhb_clearbhb NULL =20 static void kvm_setup_bhb_slot(const char *hyp_vecs_start) { } #endif /* CONFIG_KVM */ @@ -1046,6 +1053,11 @@ void spectre_bhb_enable_mitigation(const pr_info_once("spectre-bhb mitigation disabled by command line option\n"); } else if (supports_ecbhb(SCOPE_LOCAL_CPU)) { state =3D SPECTRE_MITIGATED; + } else if (supports_clearbhb(SCOPE_LOCAL_CPU)) { + kvm_setup_bhb_slot(__spectre_bhb_clearbhb); + this_cpu_set_vectors(EL1_VECTOR_BHB_CLEAR_INSN); + + state =3D SPECTRE_MITIGATED; } else if (spectre_bhb_loop_affected(SCOPE_LOCAL_CPU)) { switch (spectre_bhb_loop_affected(SCOPE_SYSTEM)) { case 8: --- a/arch/arm64/kvm/hyp/smccc_wa.S +++ b/arch/arm64/kvm/hyp/smccc_wa.S @@ -96,3 +96,12 @@ SYM_DATA_START(__spectre_bhb_loop_k32) 1: .org __spectre_bhb_loop_k32 + __SPECTRE_BHB_LOOP_SZ .org 1b SYM_DATA_END(__spectre_bhb_loop_k32) + + .global __spectre_bhb_clearbhb +SYM_DATA_START(__spectre_bhb_clearbhb) + esb + clearbhb + isb +1: .org __spectre_bhb_clearbhb + __SPECTRE_BHB_CLEARBHB_SZ + .org 1b +SYM_DATA_END(__spectre_bhb_clearbhb) From nobody Tue Jun 23 05:03:54 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0290FC433F5 for ; Thu, 10 Mar 2022 14:36:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238403AbiCJOhF (ORCPT ); Thu, 10 Mar 2022 09:37:05 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48112 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245145AbiCJOaK (ORCPT ); Thu, 10 Mar 2022 09:30:10 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B20D41688EE; Thu, 10 Mar 2022 06:25:33 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 4941BB82615; Thu, 10 Mar 2022 14:25:28 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8230FC36AE2; Thu, 10 Mar 2022 14:25:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1646922327; bh=KQ/z7MJIjzydCdxJw2qt2ZWgIt4pm1aDURrKJrC2ub0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=b3FXbM1cS/tBmJP1An5meBuT9X6mo0UW+6W2A5dYUdDdPrdmvxPnXUNnoNtDw+MZi gdxmG0BIXUrxWuJ0Gh5fHQ326Chm/KdF7rQJLdJwA6Q6FrRelOx6YS659Oe+ZOtUlI 8OJ2egbacLkh4zudL3EEFx98OoViDcdDXYy1wT9c= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Catalin Marinas , James Morse Subject: [PATCH 5.10 42/58] arm64: proton-pack: Include unprivileged eBPF status in Spectre v2 mitigation reporting Date: Thu, 10 Mar 2022 15:19:02 +0100 Message-Id: <20220310140814.068974463@linuxfoundation.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220310140812.869208747@linuxfoundation.org> References: <20220310140812.869208747@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: James Morse commit 58c9a5060cb7cd529d49c93954cdafe81c1d642a upstream. The mitigations for Spectre-BHB are only applied when an exception is taken from user-space. The mitigation status is reported via the spectre_v2 sysfs vulnerabilities file. When unprivileged eBPF is enabled the mitigation in the exception vectors can be avoided by an eBPF program. When unprivileged eBPF is enabled, print a warning and report vulnerable via the sysfs vulnerabilities file. Acked-by: Catalin Marinas Signed-off-by: James Morse Signed-off-by: Greg Kroah-Hartman Tested-by: Bagas Sanjaya Tested-by: Florian Fainelli Tested-by: Fox Chen Tested-by: Guenter Roeck Tested-by: Linux Kernel Functional Testing Tested-by: Pavel Machek (CIP) = =20 Tested-by: Shuah Khan Tested-by: Sudip Mukherjee --- arch/arm64/kernel/proton-pack.c | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) --- a/arch/arm64/kernel/proton-pack.c +++ b/arch/arm64/kernel/proton-pack.c @@ -18,6 +18,7 @@ */ =20 #include +#include #include #include #include @@ -110,6 +111,15 @@ static const char *get_bhb_affected_stri } } =20 +static bool _unprivileged_ebpf_enabled(void) +{ +#ifdef CONFIG_BPF_SYSCALL + return !sysctl_unprivileged_bpf_disabled; +#else + return false; +#endif +} + ssize_t cpu_show_spectre_v2(struct device *dev, struct device_attribute *a= ttr, char *buf) { @@ -129,6 +139,9 @@ ssize_t cpu_show_spectre_v2(struct devic v2_str =3D "CSV2"; fallthrough; case SPECTRE_MITIGATED: + if (bhb_state =3D=3D SPECTRE_MITIGATED && _unprivileged_ebpf_enabled()) + return sprintf(buf, "Vulnerable: Unprivileged eBPF enabled\n"); + return sprintf(buf, "Mitigation: %s%s\n", v2_str, bhb_str); case SPECTRE_VULNERABLE: fallthrough; @@ -1108,3 +1121,16 @@ void noinstr spectre_bhb_patch_loop_iter AARCH64_INSN_MOVEWIDE_ZERO); *updptr++ =3D cpu_to_le32(insn); } + +#ifdef CONFIG_BPF_SYSCALL +#define EBPF_WARN "Unprivileged eBPF is enabled, data leaks possible via S= pectre v2 BHB attacks!\n" +void unpriv_ebpf_notify(int new_state) +{ + if (spectre_v2_state =3D=3D SPECTRE_VULNERABLE || + spectre_bhb_state !=3D SPECTRE_MITIGATED) + return; + + if (!new_state) + pr_err("WARNING: %s", EBPF_WARN); +} +#endif From nobody Tue Jun 23 05:03:54 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D685BC35278 for ; Thu, 10 Mar 2022 14:42:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1345076AbiCJOlf (ORCPT ); Thu, 10 Mar 2022 09:41:35 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51990 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245157AbiCJOaK (ORCPT ); Thu, 10 Mar 2022 09:30:10 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5ED811693B3; Thu, 10 Mar 2022 06:25:35 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id B7928B825F3; Thu, 10 Mar 2022 14:25:34 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2EF90C340E8; Thu, 10 Mar 2022 14:25:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1646922333; bh=dBM59trElM9mKdvh5X3TAf3lhidiLDUO/YzmBUoNg2U=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=lpMG6FxCnSS49LGw2rHSuxC4lGcFuR1X+coPbr8lHRovmOZb42XTI7UShVe7q6/B7 7hF6QKY1LlC4P31KNf0RCmOIdq1A9aaRCTNJ6L/yhEsgikpIX+frwm3OBbOhzh51q/ uc3N+7U+3UncP7o8l5nmgEaaNUSleBEq78wjVZrc= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Emmanuel Gil Peyrot , Nathan Chancellor , Linus Torvalds Subject: [PATCH 5.10 43/58] ARM: fix build error when BPF_SYSCALL is disabled Date: Thu, 10 Mar 2022 15:19:03 +0100 Message-Id: <20220310140814.097294178@linuxfoundation.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220310140812.869208747@linuxfoundation.org> References: <20220310140812.869208747@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Emmanuel Gil Peyrot commit 330f4c53d3c2d8b11d86ec03a964b86dc81452f5 upstream. It was missing a semicolon. Signed-off-by: Emmanuel Gil Peyrot Reviewed-by: Nathan Chancellor Fixes: 25875aa71dfe ("ARM: include unprivileged BPF status in Spectre V2 re= porting"). Signed-off-by: Linus Torvalds Signed-off-by: Greg Kroah-Hartman Tested-by: Bagas Sanjaya Tested-by: Florian Fainelli Tested-by: Fox Chen Tested-by: Guenter Roeck Tested-by: Linux Kernel Functional Testing Tested-by: Pavel Machek (CIP) = =20 Tested-by: Shuah Khan Tested-by: Sudip Mukherjee --- arch/arm/kernel/spectre.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --- a/arch/arm/kernel/spectre.c +++ b/arch/arm/kernel/spectre.c @@ -10,7 +10,7 @@ static bool _unprivileged_ebpf_enabled(v #ifdef CONFIG_BPF_SYSCALL return !sysctl_unprivileged_bpf_disabled; #else - return false + return false; #endif } From nobody Tue Jun 23 05:03:54 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 21EAEC4707F for ; Thu, 10 Mar 2022 14:42:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1345313AbiCJOlr (ORCPT ); Thu, 10 Mar 2022 09:41:47 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50048 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245414AbiCJOa3 (ORCPT ); Thu, 10 Mar 2022 09:30:29 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1ADE617EDA2; Thu, 10 Mar 2022 06:26:10 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 31F3461B63; Thu, 10 Mar 2022 14:25:37 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3E803C340E8; Thu, 10 Mar 2022 14:25:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1646922336; bh=X16joH9fZDCYNrDYqgmCgmFINl4xn8rfr8maBQfJ+UE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=wzQ90BWIP+f44xnqWwvDQgg5FuUmGN6XazQvPuyxfohmmJgoilM6u6WVzTORhalEU Wrcc9INtnr4rXiBysQQ4VcKNyORmoRhFLt7myDUeTQtyohRcBCCfwd6+nSTnmya7bD 1q1tk2Cd3q9DWIRGVpixelONXSZHoYYvxIxGTddA= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, kernel test robot , "Russell King (Oracle)" , Catalin Marinas , Linus Torvalds Subject: [PATCH 5.10 44/58] ARM: fix co-processor register typo Date: Thu, 10 Mar 2022 15:19:04 +0100 Message-Id: <20220310140814.125001802@linuxfoundation.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220310140812.869208747@linuxfoundation.org> References: <20220310140812.869208747@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Russell King (Oracle) commit 33970b031dc4653cc9dc80f2886976706c4c8ef1 upstream. In the recent Spectre BHB patches, there was a typo that is only exposed in certain configurations: mcr p15,0,XX,c7,r5,4 should have been mcr p15,0,XX,c7,c5,4 Reported-by: kernel test robot Fixes: b9baf5c8c5c3 ("ARM: Spectre-BHB workaround") Signed-off-by: Russell King (Oracle) Acked-by: Catalin Marinas Signed-off-by: Linus Torvalds Signed-off-by: Greg Kroah-Hartman Tested-by: Bagas Sanjaya Tested-by: Florian Fainelli Tested-by: Fox Chen Tested-by: Guenter Roeck Tested-by: Linux Kernel Functional Testing Tested-by: Pavel Machek (CIP) = =20 Tested-by: Shuah Khan Tested-by: Sudip Mukherjee --- arch/arm/include/asm/assembler.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --- a/arch/arm/include/asm/assembler.h +++ b/arch/arm/include/asm/assembler.h @@ -113,7 +113,7 @@ .endm =20 .macro isb, args - mcr p15, 0, r0, c7, r5, 4 + mcr p15, 0, r0, c7, c5, 4 .endm #endif From nobody Tue Jun 23 05:03:54 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 62392C433F5 for ; Thu, 10 Mar 2022 14:36:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244542AbiCJOhR (ORCPT ); Thu, 10 Mar 2022 09:37:17 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49716 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245209AbiCJOaO (ORCPT ); Thu, 10 Mar 2022 09:30:14 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 927DE16EA81; Thu, 10 Mar 2022 06:25:42 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 17629B8267E; Thu, 10 Mar 2022 14:25:41 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7F642C340E8; Thu, 10 Mar 2022 14:25:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1646922339; bh=ZwwrUYRY5dio8pDaWYDTlGOjhbR2smNpyr8q2xrywcw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=oGG3TC0SkxrcDdukpK8EkO6xPxBwdRnEQGCOvhpEHsaj+gyI+CBCLiypsOus2I+nI PtGp3OHgkQMdBiL3QChL4vbwcQ33L1CYzlAHdTWEjPjZqWAot/5mMVTq7s8+ZR7KFH g0Ykmm6AeS/YLrvtJH3VlKxynEAYu8kcZOzXdhkE= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Nathan Chancellor , Linus Torvalds Subject: [PATCH 5.10 45/58] ARM: Do not use NOCROSSREFS directive with ld.lld Date: Thu, 10 Mar 2022 15:19:05 +0100 Message-Id: <20220310140814.152792297@linuxfoundation.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220310140812.869208747@linuxfoundation.org> References: <20220310140812.869208747@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Nathan Chancellor commit 36168e387fa7d0f1fe0cd5cf76c8cea7aee714fa upstream. ld.lld does not support the NOCROSSREFS directive at the moment, which breaks the build after commit b9baf5c8c5c3 ("ARM: Spectre-BHB workaround"): ld.lld: error: ./arch/arm/kernel/vmlinux.lds:34: AT expected, but got NOC= ROSSREFS Support for this directive will eventually be implemented, at which point a version check can be added. To avoid breaking the build in the meantime, just define NOCROSSREFS to nothing when using ld.lld, with a link to the issue for tracking. Cc: stable@vger.kernel.org Fixes: b9baf5c8c5c3 ("ARM: Spectre-BHB workaround") Link: https://github.com/ClangBuiltLinux/linux/issues/1609 Signed-off-by: Nathan Chancellor Signed-off-by: Linus Torvalds Signed-off-by: Greg Kroah-Hartman Tested-by: Bagas Sanjaya Tested-by: Florian Fainelli Tested-by: Fox Chen Tested-by: Guenter Roeck Tested-by: Linux Kernel Functional Testing Tested-by: Pavel Machek (CIP) = =20 Tested-by: Shuah Khan Tested-by: Sudip Mukherjee --- arch/arm/include/asm/vmlinux.lds.h | 8 ++++++++ 1 file changed, 8 insertions(+) --- a/arch/arm/include/asm/vmlinux.lds.h +++ b/arch/arm/include/asm/vmlinux.lds.h @@ -26,6 +26,14 @@ #define ARM_MMU_DISCARD(x) x #endif =20 +/* + * ld.lld does not support NOCROSSREFS: + * https://github.com/ClangBuiltLinux/linux/issues/1609 + */ +#ifdef CONFIG_LD_IS_LLD +#define NOCROSSREFS +#endif + /* Set start/end symbol names to the LMA for the section */ #define ARM_LMA(sym, section) \ sym##_start =3D LOADADDR(section); \ From nobody Tue Jun 23 05:03:54 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8E53FC433F5 for ; Thu, 10 Mar 2022 14:36:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241255AbiCJOh3 (ORCPT ); Thu, 10 Mar 2022 09:37:29 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49564 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245480AbiCJOad (ORCPT ); Thu, 10 Mar 2022 09:30:33 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 600B9180235; Thu, 10 Mar 2022 06:26:18 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id D06A061CF0; Thu, 10 Mar 2022 14:25:43 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id C38F9C340E8; Thu, 10 Mar 2022 14:25:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1646922343; bh=rwW4tcI0CEy+l2tKmxy/vTKismrz0Bmt/3O2aj3M+UI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=hhr2F597ZSMHZXpfi9aRsdmwG8s1yua1IlrZ9PUr+e1gKIo0VXZNFEKAx826OfQ/u q/o3JTizC/0hYHFFL2kbZoVwlc/ouJfTGIHp5P/p+PcInOw80R19BBsUkLRdWA6AyD AJtMA5BUOfXp+6nCq084qRvtQzjKS514KNbPnbno= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, kernel test robot , "Russell King (Oracle)" , Linus Torvalds Subject: [PATCH 5.10 46/58] ARM: fix build warning in proc-v7-bugs.c Date: Thu, 10 Mar 2022 15:19:06 +0100 Message-Id: <20220310140814.181013692@linuxfoundation.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220310140812.869208747@linuxfoundation.org> References: <20220310140812.869208747@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Russell King (Oracle) commit b1a384d2cbccb1eb3f84765020d25e2c1929706e upstream. The kernel test robot discovered that building without HARDEN_BRANCH_PREDICTOR issues a warning due to a missing argument to pr_info(). Add the missing argument. Reported-by: kernel test robot Fixes: 9dd78194a372 ("ARM: report Spectre v2 status through sysfs") Signed-off-by: Russell King (Oracle) Signed-off-by: Linus Torvalds Signed-off-by: Greg Kroah-Hartman Tested-by: Bagas Sanjaya Tested-by: Florian Fainelli Tested-by: Fox Chen Tested-by: Guenter Roeck Tested-by: Linux Kernel Functional Testing Tested-by: Pavel Machek (CIP) = =20 Tested-by: Shuah Khan Tested-by: Sudip Mukherjee --- arch/arm/mm/proc-v7-bugs.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) --- a/arch/arm/mm/proc-v7-bugs.c +++ b/arch/arm/mm/proc-v7-bugs.c @@ -108,7 +108,8 @@ static unsigned int spectre_v2_install_w #else static unsigned int spectre_v2_install_workaround(unsigned int method) { - pr_info("CPU%u: Spectre V2: workarounds disabled by configuration\n"); + pr_info("CPU%u: Spectre V2: workarounds disabled by configuration\n", + smp_processor_id()); =20 return SPECTRE_VULNERABLE; } From nobody Tue Jun 23 05:03:54 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 989D7C433EF for ; Thu, 10 Mar 2022 14:36:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244219AbiCJOhr (ORCPT ); Thu, 10 Mar 2022 09:37:47 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50328 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245549AbiCJOag (ORCPT ); Thu, 10 Mar 2022 09:30:36 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 918E5182BDE; Thu, 10 Mar 2022 06:26:25 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 8FE3761D38; Thu, 10 Mar 2022 14:25:46 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id A0138C340E8; Thu, 10 Mar 2022 14:25:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1646922346; bh=Fx9vrl9Digl1jmM6i7NHjvlnbGVHe2w4pnEADaxx72U=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=d5fw9ezFnnZhgREtZOUHRamXGk/vVzOvjbXfRSFQ+/H+AD5ClxDNuhRtE5VLFbgnf Fj7/vw+I2wgZCd83YzUg0bRLSQPL8jFJajx7A2/vMSYl7KsfsvhYnu0nQIvxsIGXok ETNhXc1PAFzdWnYqr2oxK81F/XiXxbwdDr4+/hxo= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Demi Marie Obenour , Juergen Gross , Jan Beulich Subject: [PATCH 5.10 47/58] xen/xenbus: dont let xenbus_grant_ring() remove grants in error case Date: Thu, 10 Mar 2022 15:19:07 +0100 Message-Id: <20220310140814.208687707@linuxfoundation.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220310140812.869208747@linuxfoundation.org> References: <20220310140812.869208747@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Juergen Gross Commit 3777ea7bac3113005b7180e6b9dadf16d19a5827 upstream. Letting xenbus_grant_ring() tear down grants in the error case is problematic, as the other side could already have used these grants. Calling gnttab_end_foreign_access_ref() without checking success is resulting in an unclear situation for any caller of xenbus_grant_ring() as in the error case the memory pages of the ring page might be partially mapped. Freeing them would risk unwanted foreign access to them, while not freeing them would leak memory. In order to remove the need to undo any gnttab_grant_foreign_access() calls, use gnttab_alloc_grant_references() to make sure no further error can occur in the loop granting access to the ring pages. It should be noted that this way of handling removes leaking of grant entries in the error case, too. This is CVE-2022-23040 / part of XSA-396. Reported-by: Demi Marie Obenour Signed-off-by: Juergen Gross Reviewed-by: Jan Beulich Signed-off-by: Greg Kroah-Hartman Tested-by: Bagas Sanjaya Tested-by: Florian Fainelli Tested-by: Fox Chen Tested-by: Guenter Roeck Tested-by: Linux Kernel Functional Testing Tested-by: Pavel Machek (CIP) = =20 Tested-by: Shuah Khan Tested-by: Sudip Mukherjee --- drivers/xen/xenbus/xenbus_client.c | 24 +++++++++++------------- 1 file changed, 11 insertions(+), 13 deletions(-) --- a/drivers/xen/xenbus/xenbus_client.c +++ b/drivers/xen/xenbus/xenbus_client.c @@ -379,7 +379,14 @@ int xenbus_grant_ring(struct xenbus_devi unsigned int nr_pages, grant_ref_t *grefs) { int err; - int i, j; + unsigned int i; + grant_ref_t gref_head; + + err =3D gnttab_alloc_grant_references(nr_pages, &gref_head); + if (err) { + xenbus_dev_fatal(dev, err, "granting access to ring page"); + return err; + } =20 for (i =3D 0; i < nr_pages; i++) { unsigned long gfn; @@ -389,23 +396,14 @@ int xenbus_grant_ring(struct xenbus_devi else gfn =3D virt_to_gfn(vaddr); =20 - err =3D gnttab_grant_foreign_access(dev->otherend_id, gfn, 0); - if (err < 0) { - xenbus_dev_fatal(dev, err, - "granting access to ring page"); - goto fail; - } - grefs[i] =3D err; + grefs[i] =3D gnttab_claim_grant_reference(&gref_head); + gnttab_grant_foreign_access_ref(grefs[i], dev->otherend_id, + gfn, 0); =20 vaddr =3D vaddr + XEN_PAGE_SIZE; } =20 return 0; - -fail: - for (j =3D 0; j < i; j++) - gnttab_end_foreign_access_ref(grefs[j], 0); - return err; } EXPORT_SYMBOL_GPL(xenbus_grant_ring); From nobody Tue Jun 23 05:03:54 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1337FC3527A for ; Thu, 10 Mar 2022 14:42:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1345256AbiCJOlp (ORCPT ); Thu, 10 Mar 2022 09:41:45 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49728 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245289AbiCJOaU (ORCPT ); Thu, 10 Mar 2022 09:30:20 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 93C8E179257; Thu, 10 Mar 2022 06:25:53 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id A51F261CFB; Thu, 10 Mar 2022 14:25:49 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id B4EBAC340EB; Thu, 10 Mar 2022 14:25:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1646922349; bh=KXvlQRxJJINV8Dk/CHoR1FjXU3vHLhI/Xfy6EUeIGug=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=1c63Di8R5MuTX8n6jMDYtRCwoNUV4kj6uWc4kLrWLJU3550AHxUVh2SKMBQszjiQM hrWTkGNlIqi/HGDidJuf8n4GdkfdCNLP44Kg6Kypd3czmkqPp3UseT4Smv4cAjoWqY UNEG/AGuOdNWv2AXh2BoIJSpJIhlprtCrPrF42nE= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Demi Marie Obenour , Juergen Gross , Jan Beulich Subject: [PATCH 5.10 48/58] xen/grant-table: add gnttab_try_end_foreign_access() Date: Thu, 10 Mar 2022 15:19:08 +0100 Message-Id: <20220310140814.236353343@linuxfoundation.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220310140812.869208747@linuxfoundation.org> References: <20220310140812.869208747@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Juergen Gross Commit 6b1775f26a2da2b05a6dc8ec2b5d14e9a4701a1a upstream. Add a new grant table function gnttab_try_end_foreign_access(), which will remove and free a grant if it is not in use. Its main use case is to either free a grant if it is no longer in use, or to take some other action if it is still in use. This other action can be an error exit, or (e.g. in the case of blkfront persistent grant feature) some special handling. This is CVE-2022-23036, CVE-2022-23038 / part of XSA-396. Reported-by: Demi Marie Obenour Signed-off-by: Juergen Gross Reviewed-by: Jan Beulich Signed-off-by: Greg Kroah-Hartman Tested-by: Bagas Sanjaya Tested-by: Florian Fainelli Tested-by: Fox Chen Tested-by: Guenter Roeck Tested-by: Linux Kernel Functional Testing Tested-by: Pavel Machek (CIP) = =20 Tested-by: Shuah Khan Tested-by: Sudip Mukherjee --- drivers/xen/grant-table.c | 14 ++++++++++++-- include/xen/grant_table.h | 12 ++++++++++++ 2 files changed, 24 insertions(+), 2 deletions(-) --- a/drivers/xen/grant-table.c +++ b/drivers/xen/grant-table.c @@ -435,11 +435,21 @@ static void gnttab_add_deferred(grant_re what, ref, page ? page_to_pfn(page) : -1); } =20 +int gnttab_try_end_foreign_access(grant_ref_t ref) +{ + int ret =3D _gnttab_end_foreign_access_ref(ref, 0); + + if (ret) + put_free_entry(ref); + + return ret; +} +EXPORT_SYMBOL_GPL(gnttab_try_end_foreign_access); + void gnttab_end_foreign_access(grant_ref_t ref, int readonly, unsigned long page) { - if (gnttab_end_foreign_access_ref(ref, readonly)) { - put_free_entry(ref); + if (gnttab_try_end_foreign_access(ref)) { if (page !=3D 0) put_page(virt_to_page(page)); } else --- a/include/xen/grant_table.h +++ b/include/xen/grant_table.h @@ -97,10 +97,22 @@ int gnttab_end_foreign_access_ref(grant_ * access has been ended, free the given page too. Access will be ended * immediately iff the grant entry is not in use, otherwise it will happen * some time later. page may be 0, in which case no freeing will occur. + * Note that the granted page might still be accessed (read or write) by t= he + * other side after gnttab_end_foreign_access() returns, so even if page w= as + * specified as 0 it is not allowed to just reuse the page for other + * purposes immediately. */ void gnttab_end_foreign_access(grant_ref_t ref, int readonly, unsigned long page); =20 +/* + * End access through the given grant reference, iff the grant entry is + * no longer in use. In case of success ending foreign access, the + * grant reference is deallocated. + * Return 1 if the grant entry was freed, 0 if it is still in use. + */ +int gnttab_try_end_foreign_access(grant_ref_t ref); + int gnttab_grant_foreign_transfer(domid_t domid, unsigned long pfn); =20 unsigned long gnttab_end_foreign_transfer_ref(grant_ref_t ref); From nobody Tue Jun 23 05:03:54 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AB8B0C388F3 for ; Thu, 10 Mar 2022 14:42:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1345191AbiCJOln (ORCPT ); Thu, 10 Mar 2022 09:41:43 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50098 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245303AbiCJOaW (ORCPT ); Thu, 10 Mar 2022 09:30:22 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D53FEC24BB; Thu, 10 Mar 2022 06:25:54 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id F288661DAA; Thu, 10 Mar 2022 14:25:52 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 06301C340E8; Thu, 10 Mar 2022 14:25:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1646922352; bh=t5N0HYAinQmf9TSOz79Zbd1nDog2nkPaJh5KtODZ0MI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=jzje3w+eazQobkE1COkBWHj38HsRNgLrpk1OUBexV2DU/UL8G3D5+drwteAQjp3Go MS/SCObqugDKLnlLrqOWFFi6xVpZzY3AcnzE3Xjy7L4kBm0sxsa3But+XXwzc/l5rC wiv/F2blg1iAWL5OBMrRNsTIMK/WSe/7av5PknZU= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Demi Marie Obenour , Juergen Gross , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= Subject: [PATCH 5.10 49/58] xen/blkfront: dont use gnttab_query_foreign_access() for mapped status Date: Thu, 10 Mar 2022 15:19:09 +0100 Message-Id: <20220310140814.264466581@linuxfoundation.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220310140812.869208747@linuxfoundation.org> References: <20220310140812.869208747@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Juergen Gross Commit abf1fd5919d6238ee3bc5eb4a9b6c3947caa6638 upstream. It isn't enough to check whether a grant is still being in use by calling gnttab_query_foreign_access(), as a mapping could be realized by the other side just after having called that function. In case the call was done in preparation of revoking a grant it is better to do so via gnttab_end_foreign_access_ref() and check the success of that operation instead. For the ring allocation use alloc_pages_exact() in order to avoid high order pages in case of a multi-page ring. If a grant wasn't unmapped by the backend without persistent grants being used, set the device state to "error". This is CVE-2022-23036 / part of XSA-396. Reported-by: Demi Marie Obenour Signed-off-by: Juergen Gross Reviewed-by: Roger Pau Monn=C3=A9 Signed-off-by: Greg Kroah-Hartman Tested-by: Bagas Sanjaya Tested-by: Florian Fainelli Tested-by: Fox Chen Tested-by: Guenter Roeck Tested-by: Linux Kernel Functional Testing Tested-by: Pavel Machek (CIP) = =20 Tested-by: Shuah Khan Tested-by: Sudip Mukherjee --- drivers/block/xen-blkfront.c | 63 +++++++++++++++++++++++++-------------= ----- 1 file changed, 37 insertions(+), 26 deletions(-) --- a/drivers/block/xen-blkfront.c +++ b/drivers/block/xen-blkfront.c @@ -1352,7 +1352,8 @@ free_shadow: rinfo->ring_ref[i] =3D GRANT_INVALID_REF; } } - free_pages((unsigned long)rinfo->ring.sring, get_order(info->nr_ring_page= s * XEN_PAGE_SIZE)); + free_pages_exact(rinfo->ring.sring, + info->nr_ring_pages * XEN_PAGE_SIZE); rinfo->ring.sring =3D NULL; =20 if (rinfo->irq) @@ -1436,9 +1437,15 @@ static int blkif_get_final_status(enum b return BLKIF_RSP_OKAY; } =20 -static bool blkif_completion(unsigned long *id, - struct blkfront_ring_info *rinfo, - struct blkif_response *bret) +/* + * Return values: + * 1 response processed. + * 0 missing further responses. + * -1 error while processing. + */ +static int blkif_completion(unsigned long *id, + struct blkfront_ring_info *rinfo, + struct blkif_response *bret) { int i =3D 0; struct scatterlist *sg; @@ -1461,7 +1468,7 @@ static bool blkif_completion(unsigned lo =20 /* Wait the second response if not yet here. */ if (s2->status < REQ_DONE) - return false; + return 0; =20 bret->status =3D blkif_get_final_status(s->status, s2->status); @@ -1512,42 +1519,43 @@ static bool blkif_completion(unsigned lo } /* Add the persistent grant into the list of free grants */ for (i =3D 0; i < num_grant; i++) { - if (gnttab_query_foreign_access(s->grants_used[i]->gref)) { + if (!gnttab_try_end_foreign_access(s->grants_used[i]->gref)) { /* * If the grant is still mapped by the backend (the * backend has chosen to make this grant persistent) * we add it at the head of the list, so it will be * reused first. */ - if (!info->feature_persistent) - pr_alert_ratelimited("backed has not unmapped grant: %u\n", - s->grants_used[i]->gref); + if (!info->feature_persistent) { + pr_alert("backed has not unmapped grant: %u\n", + s->grants_used[i]->gref); + return -1; + } list_add(&s->grants_used[i]->node, &rinfo->grants); rinfo->persistent_gnts_c++; } else { /* - * If the grant is not mapped by the backend we end the - * foreign access and add it to the tail of the list, - * so it will not be picked again unless we run out of - * persistent grants. + * If the grant is not mapped by the backend we add it + * to the tail of the list, so it will not be picked + * again unless we run out of persistent grants. */ - gnttab_end_foreign_access(s->grants_used[i]->gref, 0, 0UL); s->grants_used[i]->gref =3D GRANT_INVALID_REF; list_add_tail(&s->grants_used[i]->node, &rinfo->grants); } } if (s->req.operation =3D=3D BLKIF_OP_INDIRECT) { for (i =3D 0; i < INDIRECT_GREFS(num_grant); i++) { - if (gnttab_query_foreign_access(s->indirect_grants[i]->gref)) { - if (!info->feature_persistent) - pr_alert_ratelimited("backed has not unmapped grant: %u\n", - s->indirect_grants[i]->gref); + if (!gnttab_try_end_foreign_access(s->indirect_grants[i]->gref)) { + if (!info->feature_persistent) { + pr_alert("backed has not unmapped grant: %u\n", + s->indirect_grants[i]->gref); + return -1; + } list_add(&s->indirect_grants[i]->node, &rinfo->grants); rinfo->persistent_gnts_c++; } else { struct page *indirect_page; =20 - gnttab_end_foreign_access(s->indirect_grants[i]->gref, 0, 0UL); /* * Add the used indirect page back to the list of * available pages for indirect grefs. @@ -1562,7 +1570,7 @@ static bool blkif_completion(unsigned lo } } =20 - return true; + return 1; } =20 static irqreturn_t blkif_interrupt(int irq, void *dev_id) @@ -1628,12 +1636,17 @@ static irqreturn_t blkif_interrupt(int i } =20 if (bret.operation !=3D BLKIF_OP_DISCARD) { + int ret; + /* * We may need to wait for an extra response if the * I/O request is split in 2 */ - if (!blkif_completion(&id, rinfo, &bret)) + ret =3D blkif_completion(&id, rinfo, &bret); + if (!ret) continue; + if (unlikely(ret < 0)) + goto err; } =20 if (add_id_to_freelist(rinfo, id)) { @@ -1740,8 +1753,7 @@ static int setup_blkring(struct xenbus_d for (i =3D 0; i < info->nr_ring_pages; i++) rinfo->ring_ref[i] =3D GRANT_INVALID_REF; =20 - sring =3D (struct blkif_sring *)__get_free_pages(GFP_NOIO | __GFP_HIGH, - get_order(ring_size)); + sring =3D alloc_pages_exact(ring_size, GFP_NOIO); if (!sring) { xenbus_dev_fatal(dev, -ENOMEM, "allocating shared ring"); return -ENOMEM; @@ -1751,7 +1763,7 @@ static int setup_blkring(struct xenbus_d =20 err =3D xenbus_grant_ring(dev, rinfo->ring.sring, info->nr_ring_pages, gr= ef); if (err < 0) { - free_pages((unsigned long)sring, get_order(ring_size)); + free_pages_exact(sring, ring_size); rinfo->ring.sring =3D NULL; goto fail; } @@ -2729,11 +2741,10 @@ static void purge_persistent_grants(stru list_for_each_entry_safe(gnt_list_entry, tmp, &rinfo->grants, node) { if (gnt_list_entry->gref =3D=3D GRANT_INVALID_REF || - gnttab_query_foreign_access(gnt_list_entry->gref)) + !gnttab_try_end_foreign_access(gnt_list_entry->gref)) continue; =20 list_del(&gnt_list_entry->node); - gnttab_end_foreign_access(gnt_list_entry->gref, 0, 0UL); rinfo->persistent_gnts_c--; gnt_list_entry->gref =3D GRANT_INVALID_REF; list_add_tail(&gnt_list_entry->node, &rinfo->grants); From nobody Tue Jun 23 05:03:54 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DB8E6C433EF for ; Thu, 10 Mar 2022 14:45:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S242630AbiCJOpu (ORCPT ); Thu, 10 Mar 2022 09:45:50 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42902 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S244547AbiCJOda (ORCPT ); Thu, 10 Mar 2022 09:33:30 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7CD3BEB159; Thu, 10 Mar 2022 06:31:05 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 8A8F0B8266E; Thu, 10 Mar 2022 14:25:56 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id D3112C340EB; Thu, 10 Mar 2022 14:25:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1646922355; bh=CDNfl4IGoyFCsE3oBTqgf51g83179xk1H8jlgFCafWo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=WEH2owP8emqzU/Ir+0nxArXZhWHEGPxp5h4aG9IJZyt51nNPHUGXvwFCig2h4nyo0 537rM5qsw0r9ND7ZB8gb5cwx0fPI28kWkgQJuHuS6M0g28kr75vVtwD/Nhjlf3XzyP RjKBvEWiFRA8nq/Ax7bEzTxzdF9bSn5GynhbZRG8= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Demi Marie Obenour , Juergen Gross , Jan Beulich Subject: [PATCH 5.10 50/58] xen/netfront: dont use gnttab_query_foreign_access() for mapped status Date: Thu, 10 Mar 2022 15:19:10 +0100 Message-Id: <20220310140814.292401122@linuxfoundation.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220310140812.869208747@linuxfoundation.org> References: <20220310140812.869208747@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Juergen Gross Commit 31185df7e2b1d2fa1de4900247a12d7b9c7087eb upstream. It isn't enough to check whether a grant is still being in use by calling gnttab_query_foreign_access(), as a mapping could be realized by the other side just after having called that function. In case the call was done in preparation of revoking a grant it is better to do so via gnttab_end_foreign_access_ref() and check the success of that operation instead. This is CVE-2022-23037 / part of XSA-396. Reported-by: Demi Marie Obenour Signed-off-by: Juergen Gross Reviewed-by: Jan Beulich Signed-off-by: Greg Kroah-Hartman Tested-by: Bagas Sanjaya Tested-by: Florian Fainelli Tested-by: Fox Chen Tested-by: Guenter Roeck Tested-by: Linux Kernel Functional Testing Tested-by: Pavel Machek (CIP) = =20 Tested-by: Shuah Khan Tested-by: Sudip Mukherjee --- drivers/net/xen-netfront.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) --- a/drivers/net/xen-netfront.c +++ b/drivers/net/xen-netfront.c @@ -424,14 +424,12 @@ static bool xennet_tx_buf_gc(struct netf queue->tx_link[id] =3D TX_LINK_NONE; skb =3D queue->tx_skbs[id]; queue->tx_skbs[id] =3D NULL; - if (unlikely(gnttab_query_foreign_access( - queue->grant_tx_ref[id]) !=3D 0)) { + if (unlikely(!gnttab_end_foreign_access_ref( + queue->grant_tx_ref[id], GNTMAP_readonly))) { dev_alert(dev, "Grant still in use by backend domain\n"); goto err; } - gnttab_end_foreign_access_ref( - queue->grant_tx_ref[id], GNTMAP_readonly); gnttab_release_grant_reference( &queue->gref_tx_head, queue->grant_tx_ref[id]); queue->grant_tx_ref[id] =3D GRANT_INVALID_REF; From nobody Tue Jun 23 05:03:54 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 87FC2C433FE for ; Thu, 10 Mar 2022 14:41:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244651AbiCJOiC (ORCPT ); Thu, 10 Mar 2022 09:38:02 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49606 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245576AbiCJOak (ORCPT ); Thu, 10 Mar 2022 09:30:40 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 61F96182BF0; Thu, 10 Mar 2022 06:26:29 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id E2A6561D22; Thu, 10 Mar 2022 14:25:58 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id EC134C340EB; Thu, 10 Mar 2022 14:25:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1646922358; bh=a//jtcByFiHXOCJ+/M4jiKQWdvoAGv/83F2/7HhgSnI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=yGVT5G0iETcXpobsHXZ2VGnOf6Jg28YuPLKP/oC4lCLPo6SoAyJYYNzosD7N6xXk4 OInQZ9vxxWIo35/NeC0u/4gJ2ECMBbnrNN5Umxu9SV7HGS9ZHVtZ28XcredMbreUBv Hg/cJkDQf5IDjZkbyigFitBozh58+qAjvOsgiMIE= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Demi Marie Obenour , Juergen Gross , Jan Beulich Subject: [PATCH 5.10 51/58] xen/scsifront: dont use gnttab_query_foreign_access() for mapped status Date: Thu, 10 Mar 2022 15:19:11 +0100 Message-Id: <20220310140814.320833025@linuxfoundation.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220310140812.869208747@linuxfoundation.org> References: <20220310140812.869208747@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Juergen Gross Commit 33172ab50a53578a95691310f49567c9266968b0 upstream. It isn't enough to check whether a grant is still being in use by calling gnttab_query_foreign_access(), as a mapping could be realized by the other side just after having called that function. In case the call was done in preparation of revoking a grant it is better to do so via gnttab_try_end_foreign_access() and check the success of that operation instead. This is CVE-2022-23038 / part of XSA-396. Reported-by: Demi Marie Obenour Signed-off-by: Juergen Gross Reviewed-by: Jan Beulich Signed-off-by: Greg Kroah-Hartman Tested-by: Bagas Sanjaya Tested-by: Florian Fainelli Tested-by: Fox Chen Tested-by: Guenter Roeck Tested-by: Linux Kernel Functional Testing Tested-by: Pavel Machek (CIP) = =20 Tested-by: Shuah Khan Tested-by: Sudip Mukherjee --- drivers/scsi/xen-scsifront.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) --- a/drivers/scsi/xen-scsifront.c +++ b/drivers/scsi/xen-scsifront.c @@ -233,12 +233,11 @@ static void scsifront_gnttab_done(struct return; =20 for (i =3D 0; i < shadow->nr_grants; i++) { - if (unlikely(gnttab_query_foreign_access(shadow->gref[i]))) { + if (unlikely(!gnttab_try_end_foreign_access(shadow->gref[i]))) { shost_printk(KERN_ALERT, info->host, KBUILD_MODNAME "grant still in use by backend\n"); BUG(); } - gnttab_end_foreign_access(shadow->gref[i], 0, 0UL); } =20 kfree(shadow->sg); From nobody Tue Jun 23 05:03:54 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id F340FC433EF for ; Thu, 10 Mar 2022 14:36:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244556AbiCJOhZ (ORCPT ); Thu, 10 Mar 2022 09:37:25 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49600 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245370AbiCJOa0 (ORCPT ); Thu, 10 Mar 2022 09:30:26 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7F65417DB87; Thu, 10 Mar 2022 06:26:04 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 247B061D78; Thu, 10 Mar 2022 14:26:02 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 17D9FC340F3; Thu, 10 Mar 2022 14:26:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1646922361; bh=T/mHRVpyB/klvNBKNmoZm1usKzPz4/NT8r60/gXuNsc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Cw2VWb3UVG+Sk+uIYyB2WTpbJmvxD+mAdDZWUPHuRicBGL3w92oU3cjPX9ytvZCKP j+Nk64OKLMxJtJ9JcQIho1UosZIXfoWgwRozTWQpt7Rtmz02o5TP+6C1HHFNTLWgbW wi4mJUO5/VP4vUHtuf8gRd4M2MR8VH602J3jfBRM= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Demi Marie Obenour , Juergen Gross , Jan Beulich Subject: [PATCH 5.10 52/58] xen/gntalloc: dont use gnttab_query_foreign_access() Date: Thu, 10 Mar 2022 15:19:12 +0100 Message-Id: <20220310140814.347969279@linuxfoundation.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220310140812.869208747@linuxfoundation.org> References: <20220310140812.869208747@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Juergen Gross Commit d3b6372c5881cb54925212abb62c521df8ba4809 upstream. Using gnttab_query_foreign_access() is unsafe, as it is racy by design. The use case in the gntalloc driver is not needed at all. While at it replace the call of gnttab_end_foreign_access_ref() with a call of gnttab_end_foreign_access(), which is what is really wanted there. In case the grant wasn't used due to an allocation failure, just free the grant via gnttab_free_grant_reference(). This is CVE-2022-23039 / part of XSA-396. Reported-by: Demi Marie Obenour Signed-off-by: Juergen Gross Reviewed-by: Jan Beulich Signed-off-by: Greg Kroah-Hartman Tested-by: Bagas Sanjaya Tested-by: Florian Fainelli Tested-by: Fox Chen Tested-by: Guenter Roeck Tested-by: Linux Kernel Functional Testing Tested-by: Pavel Machek (CIP) = =20 Tested-by: Shuah Khan Tested-by: Sudip Mukherjee --- drivers/xen/gntalloc.c | 25 +++++++------------------ 1 file changed, 7 insertions(+), 18 deletions(-) --- a/drivers/xen/gntalloc.c +++ b/drivers/xen/gntalloc.c @@ -169,20 +169,14 @@ undo: __del_gref(gref); } =20 - /* It's possible for the target domain to map the just-allocated grant - * references by blindly guessing their IDs; if this is done, then - * __del_gref will leave them in the queue_gref list. They need to be - * added to the global list so that we can free them when they are no - * longer referenced. - */ - if (unlikely(!list_empty(&queue_gref))) - list_splice_tail(&queue_gref, &gref_list); mutex_unlock(&gref_mutex); return rc; } =20 static void __del_gref(struct gntalloc_gref *gref) { + unsigned long addr; + if (gref->notify.flags & UNMAP_NOTIFY_CLEAR_BYTE) { uint8_t *tmp =3D kmap(gref->page); tmp[gref->notify.pgoff] =3D 0; @@ -196,21 +190,16 @@ static void __del_gref(struct gntalloc_g gref->notify.flags =3D 0; =20 if (gref->gref_id) { - if (gnttab_query_foreign_access(gref->gref_id)) - return; - - if (!gnttab_end_foreign_access_ref(gref->gref_id, 0)) - return; - - gnttab_free_grant_reference(gref->gref_id); + if (gref->page) { + addr =3D (unsigned long)page_to_virt(gref->page); + gnttab_end_foreign_access(gref->gref_id, 0, addr); + } else + gnttab_free_grant_reference(gref->gref_id); } =20 gref_size--; list_del(&gref->next_gref); =20 - if (gref->page) - __free_page(gref->page); - kfree(gref); } From nobody Tue Jun 23 05:03:54 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 78FA2C433F5 for ; Thu, 10 Mar 2022 14:41:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244078AbiCJOh6 (ORCPT ); Thu, 10 Mar 2022 09:37:58 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49802 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245676AbiCJOas (ORCPT ); Thu, 10 Mar 2022 09:30:48 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4E0A1184B7B; Thu, 10 Mar 2022 06:26:39 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 7F86FB82676; Thu, 10 Mar 2022 14:26:09 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id C4506C340EB; Thu, 10 Mar 2022 14:26:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1646922368; bh=gBicCRT9OVvgDAQ5hBhrDnX8cBLGvxfOndfcHGcsKz4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=oliB1+Jp8m8P1PB5LsyxbVDX3HOnsu0e8OyEi7HoU/8GJOTPNZw6IRvSNF+dVCsUY wiLV2+wkkztIfbzQZYSm4XFoAbcAXloZDlZx90GsJq1vXyZ9nJzGxE7s2JVrx+BMUj i+haHyui1by8Kb9JlcDQju51f3x5kCY04BGYopYE= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Juergen Gross , Jan Beulich Subject: [PATCH 5.10 53/58] xen: remove gnttab_query_foreign_access() Date: Thu, 10 Mar 2022 15:19:13 +0100 Message-Id: <20220310140814.376092737@linuxfoundation.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220310140812.869208747@linuxfoundation.org> References: <20220310140812.869208747@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Juergen Gross Commit 1dbd11ca75fe664d3e54607547771d021f531f59 upstream. Remove gnttab_query_foreign_access(), as it is unused and unsafe to use. All previous use cases assumed a grant would not be in use after gnttab_query_foreign_access() returned 0. This information is useless in best case, as it only refers to a situation in the past, which could have changed already. Signed-off-by: Juergen Gross Reviewed-by: Jan Beulich Signed-off-by: Greg Kroah-Hartman Tested-by: Bagas Sanjaya Tested-by: Florian Fainelli Tested-by: Fox Chen Tested-by: Guenter Roeck Tested-by: Linux Kernel Functional Testing Tested-by: Pavel Machek (CIP) = =20 Tested-by: Shuah Khan Tested-by: Sudip Mukherjee --- drivers/xen/grant-table.c | 25 ------------------------- include/xen/grant_table.h | 2 -- 2 files changed, 27 deletions(-) --- a/drivers/xen/grant-table.c +++ b/drivers/xen/grant-table.c @@ -133,13 +133,6 @@ struct gnttab_ops { * return the frame. */ unsigned long (*end_foreign_transfer_ref)(grant_ref_t ref); - /* - * Query the status of a grant entry. Ref parameter is reference of - * queried grant entry, return value is the status of queried entry. - * Detailed status(writing/reading) can be gotten from the return value - * by bit operations. - */ - int (*query_foreign_access)(grant_ref_t ref); }; =20 struct unmap_refs_callback_data { @@ -284,22 +277,6 @@ int gnttab_grant_foreign_access(domid_t } EXPORT_SYMBOL_GPL(gnttab_grant_foreign_access); =20 -static int gnttab_query_foreign_access_v1(grant_ref_t ref) -{ - return gnttab_shared.v1[ref].flags & (GTF_reading|GTF_writing); -} - -static int gnttab_query_foreign_access_v2(grant_ref_t ref) -{ - return grstatus[ref] & (GTF_reading|GTF_writing); -} - -int gnttab_query_foreign_access(grant_ref_t ref) -{ - return gnttab_interface->query_foreign_access(ref); -} -EXPORT_SYMBOL_GPL(gnttab_query_foreign_access); - static int gnttab_end_foreign_access_ref_v1(grant_ref_t ref, int readonly) { u16 flags, nflags; @@ -1427,7 +1404,6 @@ static const struct gnttab_ops gnttab_v1 .update_entry =3D gnttab_update_entry_v1, .end_foreign_access_ref =3D gnttab_end_foreign_access_ref_v1, .end_foreign_transfer_ref =3D gnttab_end_foreign_transfer_ref_v1, - .query_foreign_access =3D gnttab_query_foreign_access_v1, }; =20 static const struct gnttab_ops gnttab_v2_ops =3D { @@ -1439,7 +1415,6 @@ static const struct gnttab_ops gnttab_v2 .update_entry =3D gnttab_update_entry_v2, .end_foreign_access_ref =3D gnttab_end_foreign_access_ref_v2, .end_foreign_transfer_ref =3D gnttab_end_foreign_transfer_ref_v2, - .query_foreign_access =3D gnttab_query_foreign_access_v2, }; =20 static bool gnttab_need_v2(void) --- a/include/xen/grant_table.h +++ b/include/xen/grant_table.h @@ -118,8 +118,6 @@ int gnttab_grant_foreign_transfer(domid_ unsigned long gnttab_end_foreign_transfer_ref(grant_ref_t ref); unsigned long gnttab_end_foreign_transfer(grant_ref_t ref); =20 -int gnttab_query_foreign_access(grant_ref_t ref); - /* * operations on reserved batches of grant references */ From nobody Tue Jun 23 05:03:54 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 793FFC47081 for ; Thu, 10 Mar 2022 14:42:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1345539AbiCJOl7 (ORCPT ); Thu, 10 Mar 2022 09:41:59 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49562 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245684AbiCJOat (ORCPT ); Thu, 10 Mar 2022 09:30:49 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E2510186229; Thu, 10 Mar 2022 06:26:41 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 5FBEFB82544; Thu, 10 Mar 2022 14:26:12 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id A530DC340E8; Thu, 10 Mar 2022 14:26:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1646922371; bh=WW3DOoq91ik363zaWmSo3HAXnnIJnEoktcdQieptd8g=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=mwLt5TjtBc3XMoeOIRfuvAl8ZNj4QT6gcxuzoKbN3YbJ1q7Et1Tl/kvd441KCkRAi tupnjvjZpWa/8XCACLtyxssybqHiZLU2LgtjOYSN0KMYc4LnE6mJcJF2CNAIBQiAoD 4gGaKQB6Z87+xNNu+dcpXAJjAKALC64M9amvQWAk= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Simon Gaiser , Juergen Gross , Jan Beulich Subject: [PATCH 5.10 54/58] xen/9p: use alloc/free_pages_exact() Date: Thu, 10 Mar 2022 15:19:14 +0100 Message-Id: <20220310140814.404593818@linuxfoundation.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220310140812.869208747@linuxfoundation.org> References: <20220310140812.869208747@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Juergen Gross Commit 5cadd4bb1d7fc9ab201ac14620d1a478357e4ebd upstream. Instead of __get_free_pages() and free_pages() use alloc_pages_exact() and free_pages_exact(). This is in preparation of a change of gnttab_end_foreign_access() which will prohibit use of high-order pages. By using the local variable "order" instead of ring->intf->ring_order in the error path of xen_9pfs_front_alloc_dataring() another bug is fixed, as the error path can be entered before ring->intf->ring_order is being set. By using alloc_pages_exact() the size in bytes is specified for the allocation, which fixes another bug for the case of order < (PAGE_SHIFT - XEN_PAGE_SHIFT). This is part of CVE-2022-23041 / XSA-396. Reported-by: Simon Gaiser Signed-off-by: Juergen Gross Reviewed-by: Jan Beulich Signed-off-by: Greg Kroah-Hartman Tested-by: Bagas Sanjaya Tested-by: Florian Fainelli Tested-by: Fox Chen Tested-by: Guenter Roeck Tested-by: Linux Kernel Functional Testing Tested-by: Pavel Machek (CIP) = =20 Tested-by: Shuah Khan Tested-by: Sudip Mukherjee --- net/9p/trans_xen.c | 14 ++++++-------- 1 file changed, 6 insertions(+), 8 deletions(-) --- a/net/9p/trans_xen.c +++ b/net/9p/trans_xen.c @@ -304,9 +304,9 @@ static void xen_9pfs_front_free(struct x ref =3D priv->rings[i].intf->ref[j]; gnttab_end_foreign_access(ref, 0, 0); } - free_pages((unsigned long)priv->rings[i].data.in, - priv->rings[i].intf->ring_order - - (PAGE_SHIFT - XEN_PAGE_SHIFT)); + free_pages_exact(priv->rings[i].data.in, + 1UL << (priv->rings[i].intf->ring_order + + XEN_PAGE_SHIFT)); } gnttab_end_foreign_access(priv->rings[i].ref, 0, 0); free_page((unsigned long)priv->rings[i].intf); @@ -345,8 +345,8 @@ static int xen_9pfs_front_alloc_dataring if (ret < 0) goto out; ring->ref =3D ret; - bytes =3D (void *)__get_free_pages(GFP_KERNEL | __GFP_ZERO, - order - (PAGE_SHIFT - XEN_PAGE_SHIFT)); + bytes =3D alloc_pages_exact(1UL << (order + XEN_PAGE_SHIFT), + GFP_KERNEL | __GFP_ZERO); if (!bytes) { ret =3D -ENOMEM; goto out; @@ -377,9 +377,7 @@ out: if (bytes) { for (i--; i >=3D 0; i--) gnttab_end_foreign_access(ring->intf->ref[i], 0, 0); - free_pages((unsigned long)bytes, - ring->intf->ring_order - - (PAGE_SHIFT - XEN_PAGE_SHIFT)); + free_pages_exact(bytes, 1UL << (order + XEN_PAGE_SHIFT)); } gnttab_end_foreign_access(ring->ref, 0, 0); free_page((unsigned long)ring->intf); From nobody Tue Jun 23 05:03:54 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4D7A6C47080 for ; Thu, 10 Mar 2022 14:42:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1345419AbiCJOlw (ORCPT ); Thu, 10 Mar 2022 09:41:52 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49568 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245465AbiCJOac (ORCPT ); Thu, 10 Mar 2022 09:30:32 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A9E83180218; Thu, 10 Mar 2022 06:26:17 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 30BF1B82615; Thu, 10 Mar 2022 14:26:15 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9B6DEC340E8; Thu, 10 Mar 2022 14:26:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1646922374; bh=MfHU0Kwo0xA5jQW28E3BI2IOIWOZWlLewPFWs+uMccc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=vLNi7cVzw8Yv8g+1ZqEdppnF6OdCJlf4H21GOaEBd3mkHnJfUDAHE/d6MPsUTmBUO lXbyBZAWsmxY6IotBlYSSqaruwgb6hQnhzIyBEzLkNJs7tW0utAETRG/uYoPL1X4NS O0ElGum20UYtEZmRgz5WuXhfv27PvJ92aohEdtBs= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Simon Gaiser , Juergen Gross , Jan Beulich Subject: [PATCH 5.10 55/58] xen/pvcalls: use alloc/free_pages_exact() Date: Thu, 10 Mar 2022 15:19:15 +0100 Message-Id: <20220310140814.433062031@linuxfoundation.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220310140812.869208747@linuxfoundation.org> References: <20220310140812.869208747@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Juergen Gross Commit b0576cc9c6b843d99c6982888d59a56209341888 upstream. Instead of __get_free_pages() and free_pages() use alloc_pages_exact() and free_pages_exact(). This is in preparation of a change of gnttab_end_foreign_access() which will prohibit use of high-order pages. This is part of CVE-2022-23041 / XSA-396. Reported-by: Simon Gaiser Signed-off-by: Juergen Gross Reviewed-by: Jan Beulich Signed-off-by: Greg Kroah-Hartman Tested-by: Bagas Sanjaya Tested-by: Florian Fainelli Tested-by: Fox Chen Tested-by: Guenter Roeck Tested-by: Linux Kernel Functional Testing Tested-by: Pavel Machek (CIP) = =20 Tested-by: Shuah Khan Tested-by: Sudip Mukherjee --- drivers/xen/pvcalls-front.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) --- a/drivers/xen/pvcalls-front.c +++ b/drivers/xen/pvcalls-front.c @@ -337,8 +337,8 @@ static void free_active_ring(struct sock if (!map->active.ring) return; =20 - free_pages((unsigned long)map->active.data.in, - map->active.ring->ring_order); + free_pages_exact(map->active.data.in, + PAGE_SIZE << map->active.ring->ring_order); free_page((unsigned long)map->active.ring); } =20 @@ -352,8 +352,8 @@ static int alloc_active_ring(struct sock goto out; =20 map->active.ring->ring_order =3D PVCALLS_RING_ORDER; - bytes =3D (void *)__get_free_pages(GFP_KERNEL | __GFP_ZERO, - PVCALLS_RING_ORDER); + bytes =3D alloc_pages_exact(PAGE_SIZE << PVCALLS_RING_ORDER, + GFP_KERNEL | __GFP_ZERO); if (!bytes) goto out; From nobody Tue Jun 23 05:03:54 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DC517C433F5 for ; Thu, 10 Mar 2022 14:36:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244569AbiCJOhh (ORCPT ); Thu, 10 Mar 2022 09:37:37 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52064 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245483AbiCJOad (ORCPT ); Thu, 10 Mar 2022 09:30:33 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 64AE0151368; Thu, 10 Mar 2022 06:26:20 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 78DF561DCE; Thu, 10 Mar 2022 14:26:17 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 6B538C340E8; Thu, 10 Mar 2022 14:26:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1646922376; bh=GjlfoHMOkglagidzH3/LvdIMhfB/hvj/e1m4Bzl2uHw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=rCJFTZdjNODT2+z3Q1UUMz2fes/pqsw3snhO+Cw1vmTaYHrvF2MIQXLCTonkHRmhz oO7o//OQGBzySrbNPSQQoBHQGf3Ev2ZdrYY5e5ptJIQeF3JKvAUveXrDYKl9N8pPGY 15EZLpsZWYuS8TZrb4AJvSbPNQZoXXFDhXk47ndM= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Simon Gaiser , Juergen Gross , Jan Beulich Subject: [PATCH 5.10 56/58] xen/gnttab: fix gnttab_end_foreign_access() without page specified Date: Thu, 10 Mar 2022 15:19:16 +0100 Message-Id: <20220310140814.460961562@linuxfoundation.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220310140812.869208747@linuxfoundation.org> References: <20220310140812.869208747@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Juergen Gross Commit 42baefac638f06314298087394b982ead9ec444b upstream. gnttab_end_foreign_access() is used to free a grant reference and optionally to free the associated page. In case the grant is still in use by the other side processing is being deferred. This leads to a problem in case no page to be freed is specified by the caller: the caller doesn't know that the page is still mapped by the other side and thus should not be used for other purposes. The correct way to handle this situation is to take an additional reference to the granted page in case handling is being deferred and to drop that reference when the grant reference could be freed finally. This requires that there are no users of gnttab_end_foreign_access() left directly repurposing the granted page after the call, as this might result in clobbered data or information leaks via the not yet freed grant reference. This is part of CVE-2022-23041 / XSA-396. Reported-by: Simon Gaiser Signed-off-by: Juergen Gross Reviewed-by: Jan Beulich Signed-off-by: Greg Kroah-Hartman Tested-by: Bagas Sanjaya Tested-by: Florian Fainelli Tested-by: Fox Chen Tested-by: Guenter Roeck Tested-by: Linux Kernel Functional Testing Tested-by: Pavel Machek (CIP) = =20 Tested-by: Shuah Khan Tested-by: Sudip Mukherjee --- drivers/xen/grant-table.c | 36 +++++++++++++++++++++++++++++------- include/xen/grant_table.h | 7 ++++++- 2 files changed, 35 insertions(+), 8 deletions(-) --- a/drivers/xen/grant-table.c +++ b/drivers/xen/grant-table.c @@ -133,6 +133,10 @@ struct gnttab_ops { * return the frame. */ unsigned long (*end_foreign_transfer_ref)(grant_ref_t ref); + /* + * Read the frame number related to a given grant reference. + */ + unsigned long (*read_frame)(grant_ref_t ref); }; =20 struct unmap_refs_callback_data { @@ -330,6 +334,16 @@ int gnttab_end_foreign_access_ref(grant_ } EXPORT_SYMBOL_GPL(gnttab_end_foreign_access_ref); =20 +static unsigned long gnttab_read_frame_v1(grant_ref_t ref) +{ + return gnttab_shared.v1[ref].frame; +} + +static unsigned long gnttab_read_frame_v2(grant_ref_t ref) +{ + return gnttab_shared.v2[ref].full_page.frame; +} + struct deferred_entry { struct list_head list; grant_ref_t ref; @@ -359,12 +373,9 @@ static void gnttab_handle_deferred(struc spin_unlock_irqrestore(&gnttab_list_lock, flags); if (_gnttab_end_foreign_access_ref(entry->ref, entry->ro)) { put_free_entry(entry->ref); - if (entry->page) { - pr_debug("freeing g.e. %#x (pfn %#lx)\n", - entry->ref, page_to_pfn(entry->page)); - put_page(entry->page); - } else - pr_info("freeing g.e. %#x\n", entry->ref); + pr_debug("freeing g.e. %#x (pfn %#lx)\n", + entry->ref, page_to_pfn(entry->page)); + put_page(entry->page); kfree(entry); entry =3D NULL; } else { @@ -389,9 +400,18 @@ static void gnttab_handle_deferred(struc static void gnttab_add_deferred(grant_ref_t ref, bool readonly, struct page *page) { - struct deferred_entry *entry =3D kmalloc(sizeof(*entry), GFP_ATOMIC); + struct deferred_entry *entry; + gfp_t gfp =3D (in_atomic() || irqs_disabled()) ? GFP_ATOMIC : GFP_KERNEL; const char *what =3D KERN_WARNING "leaking"; =20 + entry =3D kmalloc(sizeof(*entry), gfp); + if (!page) { + unsigned long gfn =3D gnttab_interface->read_frame(ref); + + page =3D pfn_to_page(gfn_to_pfn(gfn)); + get_page(page); + } + if (entry) { unsigned long flags; =20 @@ -1404,6 +1424,7 @@ static const struct gnttab_ops gnttab_v1 .update_entry =3D gnttab_update_entry_v1, .end_foreign_access_ref =3D gnttab_end_foreign_access_ref_v1, .end_foreign_transfer_ref =3D gnttab_end_foreign_transfer_ref_v1, + .read_frame =3D gnttab_read_frame_v1, }; =20 static const struct gnttab_ops gnttab_v2_ops =3D { @@ -1415,6 +1436,7 @@ static const struct gnttab_ops gnttab_v2 .update_entry =3D gnttab_update_entry_v2, .end_foreign_access_ref =3D gnttab_end_foreign_access_ref_v2, .end_foreign_transfer_ref =3D gnttab_end_foreign_transfer_ref_v2, + .read_frame =3D gnttab_read_frame_v2, }; =20 static bool gnttab_need_v2(void) --- a/include/xen/grant_table.h +++ b/include/xen/grant_table.h @@ -100,7 +100,12 @@ int gnttab_end_foreign_access_ref(grant_ * Note that the granted page might still be accessed (read or write) by t= he * other side after gnttab_end_foreign_access() returns, so even if page w= as * specified as 0 it is not allowed to just reuse the page for other - * purposes immediately. + * purposes immediately. gnttab_end_foreign_access() will take an addition= al + * reference to the granted page in this case, which is dropped only after + * the grant is no longer in use. + * This requires that multi page allocations for areas subject to + * gnttab_end_foreign_access() are done via alloc_pages_exact() (and freei= ng + * via free_pages_exact()) in order to avoid high order pages. */ void gnttab_end_foreign_access(grant_ref_t ref, int readonly, unsigned long page); From nobody Tue Jun 23 05:03:54 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2CAD3C4167D for ; Thu, 10 Mar 2022 14:41:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S245015AbiCJOiq (ORCPT ); Thu, 10 Mar 2022 09:38:46 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49728 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1343618AbiCJObF (ORCPT ); Thu, 10 Mar 2022 09:31:05 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E7B7A97BB4; Thu, 10 Mar 2022 06:27:07 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 61C43B8267D; Thu, 10 Mar 2022 14:26:43 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id B235FC340EB; Thu, 10 Mar 2022 14:26:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1646922402; bh=PSfVrGpf3kK5IY8j2ssNafnPXsTlPyYKJIlMijG4Kj0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=gV/2ReFkKXwRIfuepRKdCjTop62LHWcRFymolpIFz2y3wg6THWXuC/RYwU2DUQdRV fdjz1lEk9a8I7z/9RYRkRitEbZdCScbtOTOKwfRRqRYi7nVknc+D+yr8hVf7FQrmyd D4iE0cpkvSgpoUOBFANACfcCm9dU9uWd8QsoatYo= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Demi Marie Obenour , Juergen Gross , Jan Beulich Subject: [PATCH 5.10 57/58] xen/netfront: react properly to failing gnttab_end_foreign_access_ref() Date: Thu, 10 Mar 2022 15:19:17 +0100 Message-Id: <20220310140814.489486454@linuxfoundation.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220310140812.869208747@linuxfoundation.org> References: <20220310140812.869208747@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Juergen Gross Commit 66e3531b33ee51dad17c463b4d9c9f52e341503d upstream. When calling gnttab_end_foreign_access_ref() the returned value must be tested and the reaction to that value should be appropriate. In case of failure in xennet_get_responses() the reaction should not be to crash the system, but to disable the network device. The calls in setup_netfront() can be replaced by calls of gnttab_end_foreign_access(). While at it avoid double free of ring pages and grant references via xennet_disconnect_backend() in this case. This is CVE-2022-23042 / part of XSA-396. Reported-by: Demi Marie Obenour Signed-off-by: Juergen Gross Reviewed-by: Jan Beulich Signed-off-by: Greg Kroah-Hartman Tested-by: Bagas Sanjaya Tested-by: Florian Fainelli Tested-by: Fox Chen Tested-by: Guenter Roeck Tested-by: Linux Kernel Functional Testing Tested-by: Pavel Machek (CIP) = =20 Tested-by: Shuah Khan Tested-by: Sudip Mukherjee --- drivers/net/xen-netfront.c | 48 +++++++++++++++++++++++++++++-----------= ----- 1 file changed, 31 insertions(+), 17 deletions(-) --- a/drivers/net/xen-netfront.c +++ b/drivers/net/xen-netfront.c @@ -990,7 +990,6 @@ static int xennet_get_responses(struct n struct device *dev =3D &queue->info->netdev->dev; struct bpf_prog *xdp_prog; struct xdp_buff xdp; - unsigned long ret; int slots =3D 1; int err =3D 0; u32 verdict; @@ -1032,8 +1031,13 @@ static int xennet_get_responses(struct n goto next; } =20 - ret =3D gnttab_end_foreign_access_ref(ref, 0); - BUG_ON(!ret); + if (!gnttab_end_foreign_access_ref(ref, 0)) { + dev_alert(dev, + "Grant still in use by backend domain\n"); + queue->info->broken =3D true; + dev_alert(dev, "Disabled for further use\n"); + return -EINVAL; + } =20 gnttab_release_grant_reference(&queue->gref_rx_head, ref); =20 @@ -1254,6 +1258,10 @@ static int xennet_poll(struct napi_struc &need_xdp_flush); =20 if (unlikely(err)) { + if (queue->info->broken) { + spin_unlock(&queue->rx_lock); + return 0; + } err: while ((skb =3D __skb_dequeue(&tmpq))) __skb_queue_tail(&errq, skb); @@ -1918,7 +1926,7 @@ static int setup_netfront(struct xenbus_ struct netfront_queue *queue, unsigned int feature_split_evtchn) { struct xen_netif_tx_sring *txs; - struct xen_netif_rx_sring *rxs; + struct xen_netif_rx_sring *rxs =3D NULL; grant_ref_t gref; int err; =20 @@ -1938,21 +1946,21 @@ static int setup_netfront(struct xenbus_ =20 err =3D xenbus_grant_ring(dev, txs, 1, &gref); if (err < 0) - goto grant_tx_ring_fail; + goto fail; queue->tx_ring_ref =3D gref; =20 rxs =3D (struct xen_netif_rx_sring *)get_zeroed_page(GFP_NOIO | __GFP_HIG= H); if (!rxs) { err =3D -ENOMEM; xenbus_dev_fatal(dev, err, "allocating rx ring page"); - goto alloc_rx_ring_fail; + goto fail; } SHARED_RING_INIT(rxs); FRONT_RING_INIT(&queue->rx, rxs, XEN_PAGE_SIZE); =20 err =3D xenbus_grant_ring(dev, rxs, 1, &gref); if (err < 0) - goto grant_rx_ring_fail; + goto fail; queue->rx_ring_ref =3D gref; =20 if (feature_split_evtchn) @@ -1965,22 +1973,28 @@ static int setup_netfront(struct xenbus_ err =3D setup_netfront_single(queue); =20 if (err) - goto alloc_evtchn_fail; + goto fail; =20 return 0; =20 /* If we fail to setup netfront, it is safe to just revoke access to * granted pages because backend is not accessing it at this point. */ -alloc_evtchn_fail: - gnttab_end_foreign_access_ref(queue->rx_ring_ref, 0); -grant_rx_ring_fail: - free_page((unsigned long)rxs); -alloc_rx_ring_fail: - gnttab_end_foreign_access_ref(queue->tx_ring_ref, 0); -grant_tx_ring_fail: - free_page((unsigned long)txs); -fail: + fail: + if (queue->rx_ring_ref !=3D GRANT_INVALID_REF) { + gnttab_end_foreign_access(queue->rx_ring_ref, 0, + (unsigned long)rxs); + queue->rx_ring_ref =3D GRANT_INVALID_REF; + } else { + free_page((unsigned long)rxs); + } + if (queue->tx_ring_ref !=3D GRANT_INVALID_REF) { + gnttab_end_foreign_access(queue->tx_ring_ref, 0, + (unsigned long)txs); + queue->tx_ring_ref =3D GRANT_INVALID_REF; + } else { + free_page((unsigned long)txs); + } return err; } From nobody Tue Jun 23 05:03:54 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B3A37C4332F for ; Thu, 10 Mar 2022 14:41:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244790AbiCJOiL (ORCPT ); Thu, 10 Mar 2022 09:38:11 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51216 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1343567AbiCJObD (ORCPT ); Thu, 10 Mar 2022 09:31:03 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 15B9F90260; Thu, 10 Mar 2022 06:27:00 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 738D8B8267A; Thu, 10 Mar 2022 14:26:40 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id B9C2EC340E8; Thu, 10 Mar 2022 14:26:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1646922399; bh=Onl24IHa01ENjY4Di4IqG6DGSCjo9r2zGSIqYKmut+c=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=jQdSSGrSShz3UvMZFr6ubNB1GhDvxfDuMItsYk7MCk5dDWEqUdgBtvAj/Wg0Kwpa2 +KgrdomSfy4TYZlEolgMNjSd3UdViYHuzwcmForAaXkBVzTnK478VS7p01eo6Rrf4X LplHu1zRxJK0ErUaq2HKe/jcf+MGAvqhLwIhY01o= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Hans de Goede , "Rafael J. Wysocki" , Sasha Levin , Justin Forbes , Mark Pearson Subject: [PATCH 5.10 58/58] Revert "ACPI: PM: s2idle: Cancel wakeup before dispatching EC GPE" Date: Thu, 10 Mar 2022 15:19:18 +0100 Message-Id: <20220310140814.516821164@linuxfoundation.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220310140812.869208747@linuxfoundation.org> References: <20220310140812.869208747@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Greg Kroah-Hartman This reverts commit 48e413087de1bc688ad732839efc0a5627f38d34 which is commit dc0075ba7f387fe4c48a8c674b11ab6f374a6acc upstream. It's been reported to cause problems with a number of Fedora and Arch Linux users, so drop it for now until that is resolved. Link: https://lore.kernel.org/r/CAJZ5v0gE52NT=3D4kN4MkhV3Gx=3DM5CeMGVHOF0jg= TXDb5WwAMs_Q@mail.gmail.com Link: https://lore.kernel.org/r/31b9d1cd-6a67-218b-4ada-12f72e6f00dc@redhat= .com Reported-by: Hans de Goede Reported-by: Rafael J. Wysocki Cc: Sasha Levin Cc: Justin Forbes Cc: Mark Pearson Signed-off-by: Greg Kroah-Hartman Tested-by: Bagas Sanjaya Tested-by: Florian Fainelli Tested-by: Fox Chen Tested-by: Guenter Roeck Tested-by: Linux Kernel Functional Testing Tested-by: Pavel Machek (CIP) = =20 Tested-by: Shuah Khan Tested-by: Sudip Mukherjee --- drivers/acpi/ec.c | 10 ---------- drivers/acpi/sleep.c | 14 ++++++++++---- 2 files changed, 10 insertions(+), 14 deletions(-) --- a/drivers/acpi/ec.c +++ b/drivers/acpi/ec.c @@ -2065,16 +2065,6 @@ bool acpi_ec_dispatch_gpe(void) return true; =20 /* - * Cancel the SCI wakeup and process all pending events in case there - * are any wakeup ones in there. - * - * Note that if any non-EC GPEs are active at this point, the SCI will - * retrigger after the rearming in acpi_s2idle_wake(), so no events - * should be missed by canceling the wakeup here. - */ - pm_system_cancel_wakeup(); - - /* * Dispatch the EC GPE in-band, but do not report wakeup in any case * to allow the caller to process events properly after that. */ --- a/drivers/acpi/sleep.c +++ b/drivers/acpi/sleep.c @@ -1012,15 +1012,21 @@ static bool acpi_s2idle_wake(void) return true; } =20 - /* - * Check non-EC GPE wakeups and if there are none, cancel the - * SCI-related wakeup and dispatch the EC GPE. - */ + /* Check non-EC GPE wakeups and dispatch the EC GPE. */ if (acpi_ec_dispatch_gpe()) { pm_pr_dbg("ACPI non-EC GPE wakeup\n"); return true; } =20 + /* + * Cancel the SCI wakeup and process all pending events in case + * there are any wakeup ones in there. + * + * Note that if any non-EC GPEs are active at this point, the + * SCI will retrigger after the rearming below, so no events + * should be missed by canceling the wakeup here. + */ + pm_system_cancel_wakeup(); acpi_os_wait_events_complete(); =20 /*