From nobody Fri Dec 19 19:04:52 2025 Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2051.outbound.protection.outlook.com [40.107.92.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1891A205AC8 for ; Wed, 8 Jan 2025 20:25:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.92.51 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736367958; cv=fail; b=WZhX/Di+UOqyZnhQUtXyo31dOVvnUNd4/RUSJRXUwNJdZmlfpQHy2fU+qgfz+AXibF85w00t644TrhgmTIlCOidVA709V/VAFs4fUr/LunRjXtzBjPHVzlQIF4xOVA7DkO1XZcHRelTkyQu9Y7AWtnsclrIAZAwDtPwE3pI/cnQ= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736367958; c=relaxed/simple; bh=OO0PwBsX15Su1hYR3HKp7iafG3IU7lnf5eN0iUMTPmg=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=TUtgxD31jaIrRvUESkOS6E7H3W43QXwcfYJfkB8kBFIJiqwvpmooALv/Kxh7BDpQp38XQGS7Mwjide8PFYS08R40VHKDbf/LiC5RS8p/e69zFyrszZXHuNLKcj8AB72Iy7G/2L+fi8PmRoKuCZZrdU15YKyrKXNMn0ysGmtabdQ= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=rLfxuiRA; arc=fail smtp.client-ip=40.107.92.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="rLfxuiRA" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=HQtaT26b2e8Uh6vGp5DbJbozEwDaeu4IQGQftDbdkQ5/GhBuct7p5bbQhS4ekq2XWLQQ5LzlIYFH9uodkRkNWodLLAGhN/ZwW8u6UiJCXqbB2nquS9plSitDsKL5RLNqr4wrZQLv5YbkB/1DgXAC34D+S/5MVrLysHtyCCKPZ3XtqNlHnjAjzFDamhO2KGDhRQYsFaqzGovAgFGa3SjSvJRdC+iMr7SSI19dX7jrYu4mDxMP1IJcG5SEVHk2ygGK/AfUgXxc+HNPn5AdqpnWe2pjJcFJPjkQIGjO5cjH9gUy59rVyR3+G0uU7VBBiqN1c1/pdKsscKnBuUExlbzPCA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=sIewzWF60ijT7ixtbT8qWwbMEbtFIs8rnhRJZYwksoA=; b=sGuhltv2Z27Dk3Qq22JYZI4qykpGC1z7BiYjGdJHC4Xhjmf3XadVQhd/kdFe0CW4skRvaB4IKSnO81gIpjEAUKvDw05UPQEbAoTzlHYFwiGuZ61D7o4fdAwYJpdUYHr5nOBo3bn/PJ3s4kGGHDqIMjHcebfG8W4o6Qdn+MQQEeemskXKL+GRjsn0YRWKeuxZEHSqcKfdLRfWtCs3gURRYSaS02+Ooo/Kfd8DJsLg3pgOQvGQ2oer7okHtEJl3HMs+LkYpx0mFJXTwsM8T8a6aK/FKr/5IZHxfYFsF/JhMksFXp+j2c2K4i3aVth1Tg2WjsVznQD7BG92WXCU8TdANg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=linutronix.de smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=sIewzWF60ijT7ixtbT8qWwbMEbtFIs8rnhRJZYwksoA=; b=rLfxuiRA+bbjC/fyKmcOC5mwAIBIvfHWLOATXwCvVXkf+BJ7/YfVqrPJvGW2AibO4iuT/5NC+roIG+0BLIf0yX8dfdtqHzsGJphzmhXcYoB0yNQ2sO+OyWly2OTn518HqkKJid+0ltAxFqRMSsCKi/lDmgaUcenOacE4DxVgbhY= Received: from SA9PR11CA0029.namprd11.prod.outlook.com (2603:10b6:806:6e::34) by SA3PR12MB7860.namprd12.prod.outlook.com (2603:10b6:806:307::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8314.17; Wed, 8 Jan 2025 20:25:46 +0000 Received: from DS3PEPF000099E2.namprd04.prod.outlook.com (2603:10b6:806:6e:cafe::91) by SA9PR11CA0029.outlook.office365.com (2603:10b6:806:6e::34) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8335.11 via Frontend Transport; Wed, 8 Jan 2025 20:25:45 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by DS3PEPF000099E2.mail.protection.outlook.com (10.167.17.201) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.8335.7 via Frontend Transport; Wed, 8 Jan 2025 20:25:45 +0000 Received: from tiny.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Wed, 8 Jan 2025 14:25:43 -0600 From: David Kaplan To: Thomas Gleixner , Borislav Petkov , Peter Zijlstra , Josh Poimboeuf , Pawan Gupta , Ingo Molnar , Dave Hansen , , "H . Peter Anvin" CC: Subject: [PATCH v3 15/35] x86/bugs: Restructure spectre_v2 mitigation Date: Wed, 8 Jan 2025 14:24:55 -0600 Message-ID: <20250108202515.385902-16-david.kaplan@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250108202515.385902-1-david.kaplan@amd.com> References: <20250108202515.385902-1-david.kaplan@amd.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: SATLEXMB04.amd.com (10.181.40.145) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS3PEPF000099E2:EE_|SA3PR12MB7860:EE_ X-MS-Office365-Filtering-Correlation-Id: b57492c1-c5e3-4caa-66af-08dd3022a1d0 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|7416014|1800799024|82310400026|36860700013; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?faPAELDuENkLlKg8EmiZ9EP8GhxQYKYJ0TkJw6Zy1I0iQlkRXcku4uQvkmI5?= =?us-ascii?Q?dH7cIBbTQrKEvDOEi1cI1pYivifvVbYdGVmtjI8BgE59/X2EFYnxJVHru1Kr?= =?us-ascii?Q?FRaY05g4tA1e4Yo5n9stt+8VGHbs+RbKemXxEfMJz6MZuKriu7FLieR7ajqu?= =?us-ascii?Q?F575v1tOMEImByezQkfPMimbK/gnuqdL1QufclerZWnLH1XodZ4uVyBA7h4p?= =?us-ascii?Q?bsQp/tRfOGY0tisUm7cI25akXfCxpOMojnaFEYUfVTObJq3r6uq5gRQGiaOO?= =?us-ascii?Q?eY+VuIij6rARdSP1tiC8Rv928GbjJKpwmX/RJgPxBk9uekrrhjjaBBfIoX1E?= =?us-ascii?Q?22rrUWEMUNZRAUMqdh9eJ4XERsb/WKwIzoQgf28v280FcnMnXVm5/Zxfq1/u?= =?us-ascii?Q?O2Oa4hheKxjbr7XLN+G92jPBh3CwvCixtKJLQ3thRAosXN5CTZmxuIKlhiZf?= =?us-ascii?Q?OnHea46+7F5TXJZIT1BEbnwNapPg4JFEVTOb4Swv53FTgGZvAn1YoyascqDz?= =?us-ascii?Q?Aey4EPzMXQLj5ClD/UrUuJKDTWUty/dw1nBYbNhSchY3JAoFgdRNxXBSrPIv?= =?us-ascii?Q?kcjcVCErqDfk54C9ed9Pl/DxAUi6IP4/8qSlaw9Ff5Cy2DUDbteG66ig4SWi?= =?us-ascii?Q?E23MqPn83xtQaun3LiAVfQ98lOJYTWaJw2piZT/uZY944NwEWaptAx//cnAI?= =?us-ascii?Q?4K5CXJMdUuqPlmKquGalRMpfHoJDJf8Tom2nvl4EHhxXf4UrkmolAkVVQFBl?= =?us-ascii?Q?mMBaty//FKfvfz9mG3TFPyH504Siug5JOZXfh1CljSUJS5SgBhKNmcyddkHv?= =?us-ascii?Q?tPZ45bwTY5N2wdtxOfqksAEkRieztCalhfjXHyqTXliqHLyDFj/WpslxGoGt?= =?us-ascii?Q?aFN1Hj0FGAUSJRgdot/O+1wHE/4GwEdvOVxhEJAYyj3a2tBbCVByNDXyYapd?= =?us-ascii?Q?iNJlzr6H9QzPjzT0H/DXqjjeXd1m4A5o+ugT9foPMJdCQcLCBH4jkQOCjbuk?= =?us-ascii?Q?rwD/t/Titf0yDMG6tSQJ6Msl/Eou4OBSDMaf1mb3Td4Hx2+jG9n+kp837x1a?= =?us-ascii?Q?/qpUjk1VnS9rmxdC70SkGB1v53QEqH5snbKrKsdut/hl/m3Be8OydW7jSJ2G?= =?us-ascii?Q?wQGwLJWb/+JVPTodhYrN8VnU2bM/uvYB1eB1ZkfBdxKktzjie812W37ue7Jx?= =?us-ascii?Q?frgv2Y1a1Ixm4DlBtsAlD9wMtXdBXJp820gVM2WHlINcPws1LUMdYGfcv4DY?= =?us-ascii?Q?oTn9PShJv5XyVbi/Yll4clAx2SlN10gtaQ6fzg9DclooGMhh7LTf1OY4mi+d?= =?us-ascii?Q?XEeGmNjzaPu+13Is6zrFjlBe7y9CrRRw4P34F7XmQYWhtfcfp4EEg85oEp1w?= =?us-ascii?Q?QRS2Mab+vivtzwSab4TITqHXsD+EvQef+scpFrBTvps5UAS0hKVY1H1ISuzQ?= =?us-ascii?Q?mJmtolFJs/VN1FRq/YMpWuITAJFpYlMZJAfomp18cc6AfNXvko6UKOk+OHEV?= =?us-ascii?Q?XtlOU2ExXX2zbHw=3D?= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(376014)(7416014)(1800799024)(82310400026)(36860700013);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Jan 2025 20:25:45.7195 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: b57492c1-c5e3-4caa-66af-08dd3022a1d0 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: DS3PEPF000099E2.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA3PR12MB7860 Content-Type: text/plain; charset="utf-8" Restructure spectre_v2 to use select/update/apply functions to create consistent vulnerability handling. The spectre_v2 mitigation may be updated based on the selected retbleed mitigation. Signed-off-by: David Kaplan --- arch/x86/kernel/cpu/bugs.c | 65 ++++++++++++++++++++++++++------------ 1 file changed, 44 insertions(+), 21 deletions(-) diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c index fb92344d63cd..440fe9ee1c63 100644 --- a/arch/x86/kernel/cpu/bugs.c +++ b/arch/x86/kernel/cpu/bugs.c @@ -56,6 +56,8 @@ static void __init spectre_v1_select_mitigation(void); static void __init spectre_v1_apply_mitigation(void); static void __init spectre_v2_select_mitigation(void); +static void __init spectre_v2_update_mitigation(void); +static void __init spectre_v2_apply_mitigation(void); static void __init retbleed_select_mitigation(void); static void __init retbleed_update_mitigation(void); static void __init retbleed_apply_mitigation(void); @@ -209,6 +211,7 @@ void __init cpu_select_mitigations(void) * After mitigations are selected, some may need to update their * choices. */ + spectre_v2_update_mitigation(); retbleed_update_mitigation(); spectre_v2_user_update_mitigation(); mds_update_mitigation(); @@ -217,6 +220,7 @@ void __init cpu_select_mitigations(void) rfds_update_mitigation(); =20 spectre_v1_apply_mitigation(); + spectre_v2_apply_mitigation(); retbleed_apply_mitigation(); spectre_v2_user_apply_mitigation(); mds_apply_mitigation(); @@ -1831,18 +1835,18 @@ static void __init bhi_apply_mitigation(void) =20 static void __init spectre_v2_select_mitigation(void) { - enum spectre_v2_mitigation_cmd cmd =3D spectre_v2_parse_cmdline(); enum spectre_v2_mitigation mode =3D SPECTRE_V2_NONE; + spectre_v2_cmd =3D spectre_v2_parse_cmdline(); =20 /* * If the CPU is not affected and the command line mode is NONE or AUTO * then nothing to do. */ if (!boot_cpu_has_bug(X86_BUG_SPECTRE_V2) && - (cmd =3D=3D SPECTRE_V2_CMD_NONE || cmd =3D=3D SPECTRE_V2_CMD_AUTO)) + (spectre_v2_cmd =3D=3D SPECTRE_V2_CMD_NONE || spectre_v2_cmd =3D=3D S= PECTRE_V2_CMD_AUTO)) return; =20 - switch (cmd) { + switch (spectre_v2_cmd) { case SPECTRE_V2_CMD_NONE: return; =20 @@ -1886,10 +1890,32 @@ static void __init spectre_v2_select_mitigation(voi= d) break; } =20 - if (mode =3D=3D SPECTRE_V2_EIBRS && unprivileged_ebpf_enabled()) + spectre_v2_enabled =3D mode; +} + +static void __init spectre_v2_update_mitigation(void) +{ + if (spectre_v2_cmd =3D=3D SPECTRE_V2_CMD_AUTO) { + if (IS_ENABLED(CONFIG_MITIGATION_IBRS_ENTRY) && + boot_cpu_has_bug(X86_BUG_RETBLEED) && + retbleed_mitigation !=3D RETBLEED_MITIGATION_NONE && + retbleed_mitigation !=3D RETBLEED_MITIGATION_STUFF && + boot_cpu_has(X86_FEATURE_IBRS) && + boot_cpu_data.x86_vendor =3D=3D X86_VENDOR_INTEL) { + spectre_v2_enabled =3D SPECTRE_V2_IBRS; + } + } + + if (boot_cpu_has_bug(X86_BUG_SPECTRE_V2) && !cpu_mitigations_off()) + pr_info("%s\n", spectre_v2_strings[spectre_v2_enabled]); +} + +static void __init spectre_v2_apply_mitigation(void) +{ + if (spectre_v2_enabled =3D=3D SPECTRE_V2_EIBRS && unprivileged_ebpf_enabl= ed()) pr_err(SPECTRE_V2_EIBRS_EBPF_MSG); =20 - if (spectre_v2_in_ibrs_mode(mode)) { + if (spectre_v2_in_ibrs_mode(spectre_v2_enabled)) { if (boot_cpu_has(X86_FEATURE_AUTOIBRS)) { msr_set_bit(MSR_EFER, _EFER_AUTOIBRS); } else { @@ -1898,8 +1924,10 @@ static void __init spectre_v2_select_mitigation(void) } } =20 - switch (mode) { + switch (spectre_v2_enabled) { case SPECTRE_V2_NONE: + return; + case SPECTRE_V2_EIBRS: break; =20 @@ -1925,14 +1953,11 @@ static void __init spectre_v2_select_mitigation(voi= d) * JMPs gets protection against BHI and Intramode-BTI, but RET * prediction from a non-RSB predictor is still a risk. */ - if (mode =3D=3D SPECTRE_V2_EIBRS_LFENCE || - mode =3D=3D SPECTRE_V2_EIBRS_RETPOLINE || - mode =3D=3D SPECTRE_V2_RETPOLINE) + if (spectre_v2_enabled =3D=3D SPECTRE_V2_EIBRS_LFENCE || + spectre_v2_enabled =3D=3D SPECTRE_V2_EIBRS_RETPOLINE || + spectre_v2_enabled =3D=3D SPECTRE_V2_RETPOLINE) spec_ctrl_disable_kernel_rrsba(); =20 - spectre_v2_enabled =3D mode; - pr_info("%s\n", spectre_v2_strings[mode]); - /* * If Spectre v2 protection has been enabled, fill the RSB during a * context switch. In general there are two types of RSB attacks @@ -1974,7 +1999,7 @@ static void __init spectre_v2_select_mitigation(void) setup_force_cpu_cap(X86_FEATURE_RSB_CTXSW); pr_info("Spectre v2 / SpectreRSB mitigation: Filling RSB on context switc= h\n"); =20 - spectre_v2_determine_rsb_fill_type_at_vmexit(mode); + spectre_v2_determine_rsb_fill_type_at_vmexit(spectre_v2_enabled); =20 /* * Retpoline protects the kernel, but doesn't protect firmware. IBRS @@ -1982,10 +2007,10 @@ static void __init spectre_v2_select_mitigation(voi= d) * firmware calls only when IBRS / Enhanced / Automatic IBRS aren't * otherwise enabled. * - * Use "mode" to check Enhanced IBRS instead of boot_cpu_has(), because - * the user might select retpoline on the kernel command line and if - * the CPU supports Enhanced IBRS, kernel might un-intentionally not - * enable IBRS around firmware calls. + * Use "spectre_v2_enabled" to check Enhanced IBRS instead of + * boot_cpu_has(), because the user might select retpoline on the kernel + * command line and if the CPU supports Enhanced IBRS, kernel might + * un-intentionally not enable IBRS around firmware calls. */ if (boot_cpu_has_bug(X86_BUG_RETBLEED) && boot_cpu_has(X86_FEATURE_IBPB) && @@ -1997,13 +2022,11 @@ static void __init spectre_v2_select_mitigation(voi= d) pr_info("Enabling Speculation Barrier for firmware calls\n"); } =20 - } else if (boot_cpu_has(X86_FEATURE_IBRS) && !spectre_v2_in_ibrs_mode(mod= e)) { + } else if (boot_cpu_has(X86_FEATURE_IBRS) && + !spectre_v2_in_ibrs_mode(spectre_v2_enabled)) { setup_force_cpu_cap(X86_FEATURE_USE_IBRS_FW); pr_info("Enabling Restricted Speculation for firmware calls\n"); } - - /* Set up IBPB and STIBP depending on the general spectre V2 command */ - spectre_v2_cmd =3D cmd; } =20 static void update_stibp_msr(void * __unused) --=20 2.34.1