From nobody Tue Dec 16 20:15:24 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3031BC07545 for ; Tue, 24 Oct 2023 13:21:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234534AbjJXNVh (ORCPT ); Tue, 24 Oct 2023 09:21:37 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44344 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234531AbjJXNU4 (ORCPT ); Tue, 24 Oct 2023 09:20:56 -0400 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 05C49D7A; Tue, 24 Oct 2023 06:20:51 -0700 (PDT) Date: Tue, 24 Oct 2023 13:20:49 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1698153650; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ubbKhCulsj2q4PUhz9A/DEc/a9AajgIXUfhHXnbqTao=; b=LrTJtL0myt+lhVJcBHTlDoDGcc3wQIvwd6Esz8fVNhKLPRyRbb1Ki/HHlfUvSldEoWHEaL ni+r80WGrTcUx7GgRXHz2EcO1U7Y+hJnvjBLgh8naRMS+1xlAw0QQhnZmODHFH6Ga1gZ1t OMtQD6fqrR/KD6lZ+Ytgw7j2KIAFF3UL9FaL7xBO38l7UVYGoofNTJf7WcstdIbyoR6Bv+ PKcWsMRioBWOYF7Z7kWgobb8aqfM5NXAQhU0t1PgnwAZzvyPLbUjUs89y49wdv4YainCLV jaCoFY0WZbuGRz0oqCgwe3F1rjyaTFKLUkvmPAqw0ss1Hu1ip/O9xk8Id8godQ== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1698153650; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ubbKhCulsj2q4PUhz9A/DEc/a9AajgIXUfhHXnbqTao=; b=g/TGauzLHbsqF/uGn0/J7SybTxW+nrFtzLCTZY6Fcta73AWool7eTWgUyjwhawFyugkYWT ptKed8n5mysihhCw== From: "tip-bot2 for Thomas Gleixner" Sender: tip-bot2@linutronix.de Reply-to: linux-kernel@vger.kernel.org To: linux-tip-commits@vger.kernel.org Subject: [tip: x86/microcode] x86/microcode: Handle "offline" CPUs correctly Cc: Thomas Gleixner , "Borislav Petkov (AMD)" , x86@kernel.org, linux-kernel@vger.kernel.org In-Reply-To: <20231002115903.660850472@linutronix.de> References: <20231002115903.660850472@linutronix.de> MIME-Version: 1.0 Message-ID: <169815364937.3135.892989407998967316.tip-bot2@tip-bot2> Robot-ID: Robot-Unsubscribe: Contact to get blacklisted from these emails Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org The following commit has been merged into the x86/microcode branch of tip: Commit-ID: 8f849ff63bcbc77670da03cb8f2b78b06257f455 Gitweb: https://git.kernel.org/tip/8f849ff63bcbc77670da03cb8f2b78b06= 257f455 Author: Thomas Gleixner AuthorDate: Mon, 02 Oct 2023 14:00:08 +02:00 Committer: Borislav Petkov (AMD) CommitterDate: Tue, 24 Oct 2023 15:05:55 +02:00 x86/microcode: Handle "offline" CPUs correctly Offline CPUs need to be parked in a safe loop when microcode update is in progress on the primary CPU. Currently, offline CPUs are parked in mwait_play_dead(), and for Intel CPUs, its not a safe instruction, because the MWAIT instruction can be patched in the new microcode update that can cause instability. - Add a new microcode state 'UCODE_OFFLINE' to report status on per-CPU basis. - Force NMI on the offline CPUs. Wake up offline CPUs while the update is in progress and then return them back to mwait_play_dead() after microcode update is complete. Signed-off-by: Thomas Gleixner Signed-off-by: Borislav Petkov (AMD) Link: https://lore.kernel.org/r/20231002115903.660850472@linutronix.de --- arch/x86/include/asm/microcode.h | 1 +- arch/x86/kernel/cpu/microcode/core.c | 112 +++++++++++++++++++++- arch/x86/kernel/cpu/microcode/internal.h | 1 +- arch/x86/kernel/nmi.c | 5 +- 4 files changed, 113 insertions(+), 6 deletions(-) diff --git a/arch/x86/include/asm/microcode.h b/arch/x86/include/asm/microc= ode.h index 8292482..0ee6ed0 100644 --- a/arch/x86/include/asm/microcode.h +++ b/arch/x86/include/asm/microcode.h @@ -73,6 +73,7 @@ static inline u32 intel_get_microcode_revision(void) #endif /* !CONFIG_CPU_SUP_INTEL */ =20 bool microcode_nmi_handler(void); +void microcode_offline_nmi_handler(void); =20 #ifdef CONFIG_MICROCODE_LATE_LOADING DECLARE_STATIC_KEY_FALSE(microcode_nmi_handler_enable); diff --git a/arch/x86/kernel/cpu/microcode/core.c b/arch/x86/kernel/cpu/mic= rocode/core.c index 6c90836..718d648 100644 --- a/arch/x86/kernel/cpu/microcode/core.c +++ b/arch/x86/kernel/cpu/microcode/core.c @@ -272,8 +272,9 @@ struct microcode_ctrl { =20 DEFINE_STATIC_KEY_FALSE(microcode_nmi_handler_enable); static DEFINE_PER_CPU(struct microcode_ctrl, ucode_ctrl); +static atomic_t late_cpus_in, offline_in_nmi; static unsigned int loops_per_usec; -static atomic_t late_cpus_in; +static cpumask_t cpu_offline_mask; =20 static noinstr bool wait_for_cpus(atomic_t *cnt) { @@ -381,7 +382,7 @@ static noinstr void load_secondary(unsigned int cpu) instrumentation_end(); } =20 -static void load_primary(unsigned int cpu) +static void __load_primary(unsigned int cpu) { struct cpumask *secondaries =3D topology_sibling_cpumask(cpu); enum sibling_ctrl ctrl; @@ -416,6 +417,67 @@ static void load_primary(unsigned int cpu) } } =20 +static bool kick_offline_cpus(unsigned int nr_offl) +{ + unsigned int cpu, timeout; + + for_each_cpu(cpu, &cpu_offline_mask) { + /* Enable the rendezvous handler and send NMI */ + per_cpu(ucode_ctrl.nmi_enabled, cpu) =3D true; + apic_send_nmi_to_offline_cpu(cpu); + } + + /* Wait for them to arrive */ + for (timeout =3D 0; timeout < (USEC_PER_SEC / 2); timeout++) { + if (atomic_read(&offline_in_nmi) =3D=3D nr_offl) + return true; + udelay(1); + } + /* Let the others time out */ + return false; +} + +static void release_offline_cpus(void) +{ + unsigned int cpu; + + for_each_cpu(cpu, &cpu_offline_mask) + per_cpu(ucode_ctrl.ctrl, cpu) =3D SCTRL_DONE; +} + +static void load_primary(unsigned int cpu) +{ + unsigned int nr_offl =3D cpumask_weight(&cpu_offline_mask); + bool proceed =3D true; + + /* Kick soft-offlined SMT siblings if required */ + if (!cpu && nr_offl) + proceed =3D kick_offline_cpus(nr_offl); + + /* If the soft-offlined CPUs did not respond, abort */ + if (proceed) + __load_primary(cpu); + + /* Unconditionally release soft-offlined SMT siblings if required */ + if (!cpu && nr_offl) + release_offline_cpus(); +} + +/* + * Minimal stub rendezvous handler for soft-offlined CPUs which participate + * in the NMI rendezvous to protect against a concurrent NMI on affected + * CPUs. + */ +void noinstr microcode_offline_nmi_handler(void) +{ + if (!raw_cpu_read(ucode_ctrl.nmi_enabled)) + return; + raw_cpu_write(ucode_ctrl.nmi_enabled, false); + raw_cpu_write(ucode_ctrl.result, UCODE_OFFLINE); + raw_atomic_inc(&offline_in_nmi); + wait_for_ctrl(); +} + static noinstr bool microcode_update_handler(void) { unsigned int cpu =3D raw_smp_processor_id(); @@ -472,6 +534,7 @@ static int load_cpus_stopped(void *unused) static int load_late_stop_cpus(void) { unsigned int cpu, updated =3D 0, failed =3D 0, timedout =3D 0, siblings = =3D 0; + unsigned int nr_offl, offline =3D 0; int old_rev =3D boot_cpu_data.microcode; struct cpuinfo_x86 prev_info; =20 @@ -479,6 +542,7 @@ static int load_late_stop_cpus(void) pr_err("You should switch to early loading, if possible.\n"); =20 atomic_set(&late_cpus_in, num_online_cpus()); + atomic_set(&offline_in_nmi, 0); loops_per_usec =3D loops_per_jiffy / (TICK_NSEC / 1000); =20 /* @@ -501,6 +565,7 @@ static int load_late_stop_cpus(void) case UCODE_UPDATED: updated++; break; case UCODE_TIMEOUT: timedout++; break; case UCODE_OK: siblings++; break; + case UCODE_OFFLINE: offline++; break; default: failed++; break; } } @@ -512,6 +577,13 @@ static int load_late_stop_cpus(void) /* Nothing changed. */ if (!failed && !timedout) return 0; + + nr_offl =3D cpumask_weight(&cpu_offline_mask); + if (offline < nr_offl) { + pr_warn("%u offline siblings did not respond.\n", + nr_offl - atomic_read(&offline_in_nmi)); + return -EIO; + } pr_err("update failed: %u CPUs failed %u CPUs timed out\n", failed, timedout); return -EIO; @@ -545,19 +617,49 @@ static int load_late_stop_cpus(void) * modern CPUs uses MWAIT, which is also not guaranteed to be safe * against a microcode update which affects MWAIT. * - * 2) Initialize the per CPU control structure + * As soft-offlined CPUs still react on NMIs, the SMT sibling + * restriction can be lifted when the vendor driver signals to use NMI + * for rendezvous and the APIC provides a mechanism to send an NMI to a + * soft-offlined CPU. The soft-offlined CPUs are then able to + * participate in the rendezvous in a trivial stub handler. + * + * 2) Initialize the per CPU control structure and create a cpumask + * which contains "offline"; secondary threads, so they can be handled + * correctly by a control CPU. */ static bool setup_cpus(void) { struct microcode_ctrl ctrl =3D { .ctrl =3D SCTRL_WAIT, .result =3D -1, }; + bool allow_smt_offline; unsigned int cpu; =20 + allow_smt_offline =3D microcode_ops->nmi_safe || + (microcode_ops->use_nmi && apic->nmi_to_offline_cpu); + + cpumask_clear(&cpu_offline_mask); + for_each_cpu_and(cpu, cpu_present_mask, &cpus_booted_once_mask) { + /* + * Offline CPUs sit in one of the play_dead() functions + * with interrupts disabled, but they still react on NMIs + * and execute arbitrary code. Also MWAIT being updated + * while the offline CPU sits there is not necessarily safe + * on all CPU variants. + * + * Mark them in the offline_cpus mask which will be handled + * by CPU0 later in the update process. + * + * Ensure that the primary thread is online so that it is + * guaranteed that all cores are updated. + */ if (!cpu_online(cpu)) { - if (topology_is_primary_thread(cpu) || !microcode_ops->nmi_safe) { - pr_err("CPU %u not online\n", cpu); + if (topology_is_primary_thread(cpu) || !allow_smt_offline) { + pr_err("CPU %u not online, loading aborted\n", cpu); return false; } + cpumask_set_cpu(cpu, &cpu_offline_mask); + per_cpu(ucode_ctrl, cpu) =3D ctrl; + continue; } =20 /* diff --git a/arch/x86/kernel/cpu/microcode/internal.h b/arch/x86/kernel/cpu= /microcode/internal.h index 627d238..1a396fc 100644 --- a/arch/x86/kernel/cpu/microcode/internal.h +++ b/arch/x86/kernel/cpu/microcode/internal.h @@ -17,6 +17,7 @@ enum ucode_state { UCODE_NFOUND, UCODE_ERROR, UCODE_TIMEOUT, + UCODE_OFFLINE, }; =20 struct microcode_ops { diff --git a/arch/x86/kernel/nmi.c b/arch/x86/kernel/nmi.c index a87d856..2c6ede4 100644 --- a/arch/x86/kernel/nmi.c +++ b/arch/x86/kernel/nmi.c @@ -502,8 +502,11 @@ DEFINE_IDTENTRY_RAW(exc_nmi) if (IS_ENABLED(CONFIG_NMI_CHECK_CPU)) raw_atomic_long_inc(&nsp->idt_calls); =20 - if (IS_ENABLED(CONFIG_SMP) && arch_cpu_is_offline(smp_processor_id())) + if (IS_ENABLED(CONFIG_SMP) && arch_cpu_is_offline(smp_processor_id())) { + if (microcode_nmi_handler_enabled()) + microcode_offline_nmi_handler(); return; + } =20 if (this_cpu_read(nmi_state) !=3D NMI_NOT_RUNNING) { this_cpu_write(nmi_state, NMI_LATCHED);