From nobody Thu Sep 18 20:23:28 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1E258C0015E for ; Fri, 21 Jul 2023 20:19:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230498AbjGUUT0 (ORCPT ); Fri, 21 Jul 2023 16:19:26 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37980 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230044AbjGUUTR (ORCPT ); Fri, 21 Jul 2023 16:19:17 -0400 Received: from mail-yb1-xb49.google.com (mail-yb1-xb49.google.com [IPv6:2607:f8b0:4864:20::b49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 81C79359F for ; Fri, 21 Jul 2023 13:19:06 -0700 (PDT) Received: by mail-yb1-xb49.google.com with SMTP id 3f1490d57ef6-ccabb20111dso2289578276.3 for ; Fri, 21 Jul 2023 13:19:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1689970746; x=1690575546; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=xY+3Pnlxin9cJMvMOS3842OXvOZxwo3V1XbzLrWDw/E=; b=Lqb3RehZeO7ntuc5ijbgqhf+wouAzLMMAFKmweKe4G70L9uc4Bw2Jl5XydLCvynLja MxugEfg74h0RFAs1g/mPDWIA6CQEQjP/1tCMXpzbUBwwHi1wUVOx7FGGl6cj3EKgt4DI 2serW/L/1xq/mdpALeeGTUFcMs8dGf7LsjjD0F7Qj7rNn0+spOMJVHUn7lKygW1pXfr6 V4l0GVwRDsM4qqERhw/rUjtXUDV9O3+sw+gB5y+qOA/uO7E4/u1wlbkArtHA17Pz8aqj lI6iYWVldo4NzF+NtwvTmo0m32Ap85XnNI96DTf2N43IXGAmmG5+ay/zpvN3HuCBAR7U b77Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689970746; x=1690575546; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=xY+3Pnlxin9cJMvMOS3842OXvOZxwo3V1XbzLrWDw/E=; b=bGuMXEiHTk4sUIkQOoQ6QsbddYs6SydKyUGk6tDW0mUzaXqOD4qeUkaXhIMiMmy3LV V0IWQ1YDtkBNHDdFRqtCQLybUxoPWG+M/bGzEDY+H6uIAq+TXWGUP9q6vggOQK+I43Q+ c7+RQ2Y3CS3VMwBR6VjtlyzRCefJfp2Jn9XQBF2LCE+wCEbEuksI27HI3cs3se5EmzuG hHm7iELHwn4pxWJ4+TnDYxdQacnB+R/yWr4WGaG22eBvspf4dncZ1CW8vH8j/gJgd9p5 LOGLKuQZnlesTkgMGICjRn40ddoOuzcpH5D2OT2phvUfuoHnCi7uiQSunaGgA9AMKECZ Vc6g== X-Gm-Message-State: ABy/qLaNQs13xtxAs8IxmpOdtb5f6b3kJ9SJ/S1Fq+QoXkWQbPz5WS6/ WfX4RKYj+AbDFyV20y1ZuPTqk67WpKw= X-Google-Smtp-Source: APBJJlGl5vZN06f98oGYYGpQC92l2Mt5+UVa2JPZ8raPJUCa9tUN62sdYqhFV6XvSia8+6L1I1Yz+z97vko= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a25:428d:0:b0:c65:7352:4b5 with SMTP id p135-20020a25428d000000b00c65735204b5mr21381yba.0.1689970745731; Fri, 21 Jul 2023 13:19:05 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 21 Jul 2023 13:18:41 -0700 In-Reply-To: <20230721201859.2307736-1-seanjc@google.com> Mime-Version: 1.0 References: <20230721201859.2307736-1-seanjc@google.com> X-Mailer: git-send-email 2.41.0.487.g6d72f3e995-goog Message-ID: <20230721201859.2307736-2-seanjc@google.com> Subject: [PATCH v4 01/19] x86/reboot: VMCLEAR active VMCSes before emergency reboot From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, Sean Christopherson , Paolo Bonzini Cc: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, Andrew Cooper , Kai Huang , Chao Gao Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" VMCLEAR active VMCSes before any emergency reboot, not just if the kernel may kexec into a new kernel after a crash. Per Intel's SDM, the VMX architecture doesn't require the CPU to flush the VMCS cache on INIT. If an emergency reboot doesn't RESET CPUs, cached VMCSes could theoretically be kept and only be written back to memory after the new kernel is booted, i.e. could effectively corrupt memory after reboot. Opportunistically remove the setting of the global pointer to NULL to make checkpatch happy. Cc: Andrew Cooper Signed-off-by: Sean Christopherson --- arch/x86/include/asm/kexec.h | 2 -- arch/x86/include/asm/reboot.h | 2 ++ arch/x86/kernel/crash.c | 31 ------------------------------- arch/x86/kernel/reboot.c | 22 ++++++++++++++++++++++ arch/x86/kvm/vmx/vmx.c | 10 +++------- 5 files changed, 27 insertions(+), 40 deletions(-) diff --git a/arch/x86/include/asm/kexec.h b/arch/x86/include/asm/kexec.h index 5b77bbc28f96..819046974b99 100644 --- a/arch/x86/include/asm/kexec.h +++ b/arch/x86/include/asm/kexec.h @@ -205,8 +205,6 @@ int arch_kimage_file_post_load_cleanup(struct kimage *i= mage); #endif #endif =20 -typedef void crash_vmclear_fn(void); -extern crash_vmclear_fn __rcu *crash_vmclear_loaded_vmcss; extern void kdump_nmi_shootdown_cpus(void); =20 #endif /* __ASSEMBLY__ */ diff --git a/arch/x86/include/asm/reboot.h b/arch/x86/include/asm/reboot.h index 9177b4354c3f..dc201724a643 100644 --- a/arch/x86/include/asm/reboot.h +++ b/arch/x86/include/asm/reboot.h @@ -25,6 +25,8 @@ void __noreturn machine_real_restart(unsigned int type); #define MRR_BIOS 0 #define MRR_APM 1 =20 +typedef void crash_vmclear_fn(void); +extern crash_vmclear_fn __rcu *crash_vmclear_loaded_vmcss; void cpu_emergency_disable_virtualization(void); =20 typedef void (*nmi_shootdown_cb)(int, struct pt_regs*); diff --git a/arch/x86/kernel/crash.c b/arch/x86/kernel/crash.c index cdd92ab43cda..54cd959cb316 100644 --- a/arch/x86/kernel/crash.c +++ b/arch/x86/kernel/crash.c @@ -48,38 +48,12 @@ struct crash_memmap_data { unsigned int type; }; =20 -/* - * This is used to VMCLEAR all VMCSs loaded on the - * processor. And when loading kvm_intel module, the - * callback function pointer will be assigned. - * - * protected by rcu. - */ -crash_vmclear_fn __rcu *crash_vmclear_loaded_vmcss =3D NULL; -EXPORT_SYMBOL_GPL(crash_vmclear_loaded_vmcss); - -static inline void cpu_crash_vmclear_loaded_vmcss(void) -{ - crash_vmclear_fn *do_vmclear_operation =3D NULL; - - rcu_read_lock(); - do_vmclear_operation =3D rcu_dereference(crash_vmclear_loaded_vmcss); - if (do_vmclear_operation) - do_vmclear_operation(); - rcu_read_unlock(); -} - #if defined(CONFIG_SMP) && defined(CONFIG_X86_LOCAL_APIC) =20 static void kdump_nmi_callback(int cpu, struct pt_regs *regs) { crash_save_cpu(regs, cpu); =20 - /* - * VMCLEAR VMCSs loaded on all cpus if needed. - */ - cpu_crash_vmclear_loaded_vmcss(); - /* * Disable Intel PT to stop its logging */ @@ -133,11 +107,6 @@ void native_machine_crash_shutdown(struct pt_regs *reg= s) =20 crash_smp_send_stop(); =20 - /* - * VMCLEAR VMCSs loaded on this cpu if needed. - */ - cpu_crash_vmclear_loaded_vmcss(); - cpu_emergency_disable_virtualization(); =20 /* diff --git a/arch/x86/kernel/reboot.c b/arch/x86/kernel/reboot.c index 3adbe97015c1..3fa4c6717a1d 100644 --- a/arch/x86/kernel/reboot.c +++ b/arch/x86/kernel/reboot.c @@ -787,6 +787,26 @@ void machine_crash_shutdown(struct pt_regs *regs) } #endif =20 +/* + * This is used to VMCLEAR all VMCSs loaded on the + * processor. And when loading kvm_intel module, the + * callback function pointer will be assigned. + * + * protected by rcu. + */ +crash_vmclear_fn __rcu *crash_vmclear_loaded_vmcss; +EXPORT_SYMBOL_GPL(crash_vmclear_loaded_vmcss); + +static inline void cpu_crash_vmclear_loaded_vmcss(void) +{ + crash_vmclear_fn *do_vmclear_operation =3D NULL; + + rcu_read_lock(); + do_vmclear_operation =3D rcu_dereference(crash_vmclear_loaded_vmcss); + if (do_vmclear_operation) + do_vmclear_operation(); + rcu_read_unlock(); +} =20 /* This is the CPU performing the emergency shutdown work. */ int crashing_cpu =3D -1; @@ -798,6 +818,8 @@ int crashing_cpu =3D -1; */ void cpu_emergency_disable_virtualization(void) { + cpu_crash_vmclear_loaded_vmcss(); + cpu_emergency_vmxoff(); cpu_emergency_svm_disable(); } diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 0ecf4be2c6af..7f692d97a821 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -41,7 +41,7 @@ #include #include #include -#include +#include #include #include #include @@ -744,7 +744,6 @@ static int vmx_set_guest_uret_msr(struct vcpu_vmx *vmx, return ret; } =20 -#ifdef CONFIG_KEXEC_CORE static void crash_vmclear_local_loaded_vmcss(void) { int cpu =3D raw_smp_processor_id(); @@ -754,7 +753,6 @@ static void crash_vmclear_local_loaded_vmcss(void) loaded_vmcss_on_cpu_link) vmcs_clear(v->vmcs); } -#endif /* CONFIG_KEXEC_CORE */ =20 static void __loaded_vmcs_clear(void *arg) { @@ -8592,10 +8590,9 @@ static void __vmx_exit(void) { allow_smaller_maxphyaddr =3D false; =20 -#ifdef CONFIG_KEXEC_CORE RCU_INIT_POINTER(crash_vmclear_loaded_vmcss, NULL); synchronize_rcu(); -#endif + vmx_cleanup_l1d_flush(); } =20 @@ -8644,10 +8641,9 @@ static int __init vmx_init(void) pi_init_cpu(cpu); } =20 -#ifdef CONFIG_KEXEC_CORE rcu_assign_pointer(crash_vmclear_loaded_vmcss, crash_vmclear_local_loaded_vmcss); -#endif + vmx_check_vmcs12_offsets(); =20 /* --=20 2.41.0.487.g6d72f3e995-goog From nobody Thu Sep 18 20:23:28 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D3E49C001DE for ; Fri, 21 Jul 2023 20:19:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231207AbjGUUT3 (ORCPT ); Fri, 21 Jul 2023 16:19:29 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38010 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229572AbjGUUTS (ORCPT ); Fri, 21 Jul 2023 16:19:18 -0400 Received: from mail-yb1-xb49.google.com (mail-yb1-xb49.google.com [IPv6:2607:f8b0:4864:20::b49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 811B835AC for ; Fri, 21 Jul 2023 13:19:08 -0700 (PDT) Received: by mail-yb1-xb49.google.com with SMTP id 3f1490d57ef6-d061f324d64so526865276.1 for ; Fri, 21 Jul 2023 13:19:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1689970748; x=1690575548; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=8mt2rx9ZLjLB1KoF/SJo016XQ9rXRtE6G9Sw+PvIk2E=; b=46uRXOuIfr0DhaSfjJ1gilpLeWI3SkY0gBhy4/JE8Ffy8aB+wmHHv0eZLbPKwxGf1W zRibu2Q+TOMsOsJJb4OHXHOPphrSRQVtzF5JSYhNBOCF7LUJtPtEWSRYc2JzZ4acAnor /zmJ7rgJese6SY0MTnK9LF9xzGYySIQj6WHl2lAxP+dEQMc/jN7H8FWycEgSVz1jPhiN kHTFjuNTxv4dt/QE31tPp7lm/S5hHzAiuf3QYbCoqwzmHIbVLyemLu2fOiOfokkTKl20 itFUL9M3X8lEkW6ABJkQLWfuSKa41xBM+UYU6HXoNTkiMFGmgDhcqWF7uCFWGdPxsAIv jV9w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689970748; x=1690575548; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=8mt2rx9ZLjLB1KoF/SJo016XQ9rXRtE6G9Sw+PvIk2E=; b=b376PQm6Xfdpm85rM0cqRpxcPkOJPQzw52zmr1Kc7Y1WdecvnFUKYEpf8mlHAbYrwI S2dv/RjwXTyTQJwDOeMaZDqQ/yuSiWyTVwzPYpz6ctKb426wcgBHD4IkFTtEDfQY9UDC +iwxBAxyiUoZobjlkl9keUrq+M8Og7qyXBC5GW/lXrKI/2VhgO1ITjKFFPeTbJYeK8R4 J8rfMy2N+a0ekC4ls3FWou8ct2pFrzfCCG/hHSmLsJxwLEa8eDJodde+/QLepM43eSq3 BT7RxHFb6B8rR21iogIVMw3KIvy2K+lXoxiFnePxwHA+GQZcPMbuU8EDJZCzdhwnhFO8 nXyg== X-Gm-Message-State: ABy/qLaqVcKgRXWgrV6tT4f3IFeDzBQbr9JOmBfa0Eyl9UsL+/Qnexf8 BCg0tG20scFroqpav7sX1erzIZwh2Do= X-Google-Smtp-Source: APBJJlF95+kSFxBs9d4pDOtehohhALNf2zhk0CRH0P4ZIcPOxfGtYaIR5ilvDhFJ1p2u7OpKqjT4MbBf20Q= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a25:8208:0:b0:cf2:9e82:a5b5 with SMTP id q8-20020a258208000000b00cf29e82a5b5mr18342ybk.8.1689970747834; Fri, 21 Jul 2023 13:19:07 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 21 Jul 2023 13:18:42 -0700 In-Reply-To: <20230721201859.2307736-1-seanjc@google.com> Mime-Version: 1.0 References: <20230721201859.2307736-1-seanjc@google.com> X-Mailer: git-send-email 2.41.0.487.g6d72f3e995-goog Message-ID: <20230721201859.2307736-3-seanjc@google.com> Subject: [PATCH v4 02/19] x86/reboot: Harden virtualization hooks for emergency reboot From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, Sean Christopherson , Paolo Bonzini Cc: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, Andrew Cooper , Kai Huang , Chao Gao Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Provide dedicated helpers to (un)register virt hooks used during an emergency crash/reboot, and WARN if there is an attempt to overwrite the registered callback, or an attempt to do an unpaired unregister. Opportunsitically use rcu_assign_pointer() instead of RCU_INIT_POINTER(), mainly so that the set/unset paths are more symmetrical, but also because any performance gains from using RCU_INIT_POINTER() are meaningless for this code. Reviewed-by: Kai Huang Signed-off-by: Sean Christopherson --- arch/x86/include/asm/reboot.h | 5 +++-- arch/x86/kernel/reboot.c | 30 ++++++++++++++++++++++++------ arch/x86/kvm/vmx/vmx.c | 6 ++---- 3 files changed, 29 insertions(+), 12 deletions(-) diff --git a/arch/x86/include/asm/reboot.h b/arch/x86/include/asm/reboot.h index dc201724a643..74c6a624d166 100644 --- a/arch/x86/include/asm/reboot.h +++ b/arch/x86/include/asm/reboot.h @@ -25,8 +25,9 @@ void __noreturn machine_real_restart(unsigned int type); #define MRR_BIOS 0 #define MRR_APM 1 =20 -typedef void crash_vmclear_fn(void); -extern crash_vmclear_fn __rcu *crash_vmclear_loaded_vmcss; +typedef void (cpu_emergency_virt_cb)(void); +void cpu_emergency_register_virt_callback(cpu_emergency_virt_cb *callback); +void cpu_emergency_unregister_virt_callback(cpu_emergency_virt_cb *callbac= k); void cpu_emergency_disable_virtualization(void); =20 typedef void (*nmi_shootdown_cb)(int, struct pt_regs*); diff --git a/arch/x86/kernel/reboot.c b/arch/x86/kernel/reboot.c index 3fa4c6717a1d..62ccedeb5e2b 100644 --- a/arch/x86/kernel/reboot.c +++ b/arch/x86/kernel/reboot.c @@ -794,17 +794,35 @@ void machine_crash_shutdown(struct pt_regs *regs) * * protected by rcu. */ -crash_vmclear_fn __rcu *crash_vmclear_loaded_vmcss; -EXPORT_SYMBOL_GPL(crash_vmclear_loaded_vmcss); +static cpu_emergency_virt_cb __rcu *cpu_emergency_virt_callback; + +void cpu_emergency_register_virt_callback(cpu_emergency_virt_cb *callback) +{ + if (WARN_ON_ONCE(rcu_access_pointer(cpu_emergency_virt_callback))) + return; + + rcu_assign_pointer(cpu_emergency_virt_callback, callback); +} +EXPORT_SYMBOL_GPL(cpu_emergency_register_virt_callback); + +void cpu_emergency_unregister_virt_callback(cpu_emergency_virt_cb *callbac= k) +{ + if (WARN_ON_ONCE(rcu_access_pointer(cpu_emergency_virt_callback) !=3D cal= lback)) + return; + + rcu_assign_pointer(cpu_emergency_virt_callback, NULL); + synchronize_rcu(); +} +EXPORT_SYMBOL_GPL(cpu_emergency_unregister_virt_callback); =20 static inline void cpu_crash_vmclear_loaded_vmcss(void) { - crash_vmclear_fn *do_vmclear_operation =3D NULL; + cpu_emergency_virt_cb *callback; =20 rcu_read_lock(); - do_vmclear_operation =3D rcu_dereference(crash_vmclear_loaded_vmcss); - if (do_vmclear_operation) - do_vmclear_operation(); + callback =3D rcu_dereference(cpu_emergency_virt_callback); + if (callback) + callback(); rcu_read_unlock(); } =20 diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 7f692d97a821..019cefc65142 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -8590,8 +8590,7 @@ static void __vmx_exit(void) { allow_smaller_maxphyaddr =3D false; =20 - RCU_INIT_POINTER(crash_vmclear_loaded_vmcss, NULL); - synchronize_rcu(); + cpu_emergency_unregister_virt_callback(crash_vmclear_local_loaded_vmcss); =20 vmx_cleanup_l1d_flush(); } @@ -8641,8 +8640,7 @@ static int __init vmx_init(void) pi_init_cpu(cpu); } =20 - rcu_assign_pointer(crash_vmclear_loaded_vmcss, - crash_vmclear_local_loaded_vmcss); + cpu_emergency_register_virt_callback(crash_vmclear_local_loaded_vmcss); =20 vmx_check_vmcs12_offsets(); =20 --=20 2.41.0.487.g6d72f3e995-goog From nobody Thu Sep 18 20:23:28 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A36C5EB64DD for ; Fri, 21 Jul 2023 20:19:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231251AbjGUUTc (ORCPT ); Fri, 21 Jul 2023 16:19:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37956 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229601AbjGUUTT (ORCPT ); Fri, 21 Jul 2023 16:19:19 -0400 Received: from mail-pl1-x649.google.com (mail-pl1-x649.google.com [IPv6:2607:f8b0:4864:20::649]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 101EB30CB for ; Fri, 21 Jul 2023 13:19:11 -0700 (PDT) Received: by mail-pl1-x649.google.com with SMTP id d9443c01a7336-1b9de7951easo13877795ad.0 for ; Fri, 21 Jul 2023 13:19:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1689970750; x=1690575550; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=IIqhdp8WqL0kWTvtPZo4YKSDQAXM01F7uHc3z46B11M=; b=N6+HAnOlKDhLqVNY9edBHUTpkSa+XY7oBMtLhV0W+2bPtkJJTDUxW1hvkMsrnQHj98 oiQUl+6svRNCcwf0uFTQJMji4LvzrNr2XGACfpM6JfC8YBqlliASriW/eTzwiz4Cj+qw Q7/LFi0tCAvBmJlOSW1stt1jDJo1Q/5KJupzQaFRZ4qAZtYc4HevNfBNLG3CVgQQlkBK bL97aMTEWyf0Y5VvSMMlEKaugqItpB+/9EF1SUMbEX1j6K5nBib+wPVofkxxhooK+5V1 rq7l+I5yiOcLjTL43F7br+tt0K4eb8nGYIQxsVTwHV54sBhWTWHF8rKe2hB8x2y2ZwjI A0wQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689970750; x=1690575550; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=IIqhdp8WqL0kWTvtPZo4YKSDQAXM01F7uHc3z46B11M=; b=agctSLOvUP/67+A8uWmWhor/2LrzwObkf/q5QkFG/1zPQ22RrIlwkDYW798FVe3t9i MuiB5RT5uKJBqFsijszvmYM/HXBVOCPPaYZIc5CrWHWeJ22nO/oTqFqLA3L1Y1XbsvIS WuT0Q69LH6DPz2/hfbKd7N3j8dwSOQhxk02IHXmcjK/68YyZJMZd7cM4MROMYVmoMedl 9Zn0qMwjXzRtnI5gLg3gMBV7tuPsoDSldIV2tFKBuKoSvK+OCDnxk6DmHnnHkW6y07hX tiihyoef3Akoy7aVNJmg5E+xDt1NFmpmCFjoqApHIMPHFogYir2zcePL7vBGbA3/gTmp 9QAA== X-Gm-Message-State: ABy/qLZwVD8+7ibW8P0q3WcgQdUa2lqCZwy5FFZVD9YmFriHBwfo6HQg upf7wXiQ1ed0CTkRNkbw598Q+1/Mw3U= X-Google-Smtp-Source: APBJJlE/7vXHTCq9jJ6EV6q2DuyaSpyg4++NHPOMxgndH9mm+1zOSzr4NdqmL9LH5P57nPMWJEhYoKBpUWY= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a17:903:2291:b0:1b8:866f:6fc1 with SMTP id b17-20020a170903229100b001b8866f6fc1mr11914plh.0.1689970750480; Fri, 21 Jul 2023 13:19:10 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 21 Jul 2023 13:18:43 -0700 In-Reply-To: <20230721201859.2307736-1-seanjc@google.com> Mime-Version: 1.0 References: <20230721201859.2307736-1-seanjc@google.com> X-Mailer: git-send-email 2.41.0.487.g6d72f3e995-goog Message-ID: <20230721201859.2307736-4-seanjc@google.com> Subject: [PATCH v4 03/19] x86/reboot: KVM: Handle VMXOFF in KVM's reboot callback From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, Sean Christopherson , Paolo Bonzini Cc: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, Andrew Cooper , Kai Huang , Chao Gao Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Use KVM VMX's reboot/crash callback to do VMXOFF in an emergency instead of manually and blindly doing VMXOFF. There's no need to attempt VMXOFF if a hypervisor, i.e. KVM, isn't loaded/active, i.e. if the CPU can't possibly be post-VMXON. Signed-off-by: Sean Christopherson Reviewed-by: Kai Huang --- arch/x86/include/asm/virtext.h | 10 ---------- arch/x86/kernel/reboot.c | 29 +++++++++-------------------- arch/x86/kvm/vmx/vmx.c | 8 +++++--- 3 files changed, 14 insertions(+), 33 deletions(-) diff --git a/arch/x86/include/asm/virtext.h b/arch/x86/include/asm/virtext.h index 3b12e6b99412..5bc29fab15da 100644 --- a/arch/x86/include/asm/virtext.h +++ b/arch/x86/include/asm/virtext.h @@ -70,16 +70,6 @@ static inline void __cpu_emergency_vmxoff(void) cpu_vmxoff(); } =20 -/** Disable VMX if it is supported and enabled on the current CPU - */ -static inline void cpu_emergency_vmxoff(void) -{ - if (cpu_has_vmx()) - __cpu_emergency_vmxoff(); -} - - - =20 /* * SVM functions: diff --git a/arch/x86/kernel/reboot.c b/arch/x86/kernel/reboot.c index 62ccedeb5e2b..d2d0f2672a64 100644 --- a/arch/x86/kernel/reboot.c +++ b/arch/x86/kernel/reboot.c @@ -787,13 +787,7 @@ void machine_crash_shutdown(struct pt_regs *regs) } #endif =20 -/* - * This is used to VMCLEAR all VMCSs loaded on the - * processor. And when loading kvm_intel module, the - * callback function pointer will be assigned. - * - * protected by rcu. - */ +/* RCU-protected callback to disable virtualization prior to reboot. */ static cpu_emergency_virt_cb __rcu *cpu_emergency_virt_callback; =20 void cpu_emergency_register_virt_callback(cpu_emergency_virt_cb *callback) @@ -815,17 +809,6 @@ void cpu_emergency_unregister_virt_callback(cpu_emerge= ncy_virt_cb *callback) } EXPORT_SYMBOL_GPL(cpu_emergency_unregister_virt_callback); =20 -static inline void cpu_crash_vmclear_loaded_vmcss(void) -{ - cpu_emergency_virt_cb *callback; - - rcu_read_lock(); - callback =3D rcu_dereference(cpu_emergency_virt_callback); - if (callback) - callback(); - rcu_read_unlock(); -} - /* This is the CPU performing the emergency shutdown work. */ int crashing_cpu =3D -1; =20 @@ -836,9 +819,15 @@ int crashing_cpu =3D -1; */ void cpu_emergency_disable_virtualization(void) { - cpu_crash_vmclear_loaded_vmcss(); + cpu_emergency_virt_cb *callback; =20 - cpu_emergency_vmxoff(); + rcu_read_lock(); + callback =3D rcu_dereference(cpu_emergency_virt_callback); + if (callback) + callback(); + rcu_read_unlock(); + + /* KVM_AMD doesn't yet utilize the common callback. */ cpu_emergency_svm_disable(); } =20 diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 019cefc65142..682c20b33a96 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -744,7 +744,7 @@ static int vmx_set_guest_uret_msr(struct vcpu_vmx *vmx, return ret; } =20 -static void crash_vmclear_local_loaded_vmcss(void) +static void vmx_emergency_disable(void) { int cpu =3D raw_smp_processor_id(); struct loaded_vmcs *v; @@ -752,6 +752,8 @@ static void crash_vmclear_local_loaded_vmcss(void) list_for_each_entry(v, &per_cpu(loaded_vmcss_on_cpu, cpu), loaded_vmcss_on_cpu_link) vmcs_clear(v->vmcs); + + __cpu_emergency_vmxoff(); } =20 static void __loaded_vmcs_clear(void *arg) @@ -8590,7 +8592,7 @@ static void __vmx_exit(void) { allow_smaller_maxphyaddr =3D false; =20 - cpu_emergency_unregister_virt_callback(crash_vmclear_local_loaded_vmcss); + cpu_emergency_unregister_virt_callback(vmx_emergency_disable); =20 vmx_cleanup_l1d_flush(); } @@ -8640,7 +8642,7 @@ static int __init vmx_init(void) pi_init_cpu(cpu); } =20 - cpu_emergency_register_virt_callback(crash_vmclear_local_loaded_vmcss); + cpu_emergency_register_virt_callback(vmx_emergency_disable); =20 vmx_check_vmcs12_offsets(); =20 --=20 2.41.0.487.g6d72f3e995-goog From nobody Thu Sep 18 20:23:28 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5CFA6EB64DD for ; Fri, 21 Jul 2023 20:19:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231295AbjGUUTk (ORCPT ); Fri, 21 Jul 2023 16:19:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38054 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230356AbjGUUTT (ORCPT ); Fri, 21 Jul 2023 16:19:19 -0400 Received: from mail-yw1-x1149.google.com (mail-yw1-x1149.google.com [IPv6:2607:f8b0:4864:20::1149]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DDBFDC6 for ; Fri, 21 Jul 2023 13:19:12 -0700 (PDT) Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-57059e6f9c7so39817167b3.0 for ; Fri, 21 Jul 2023 13:19:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1689970752; x=1690575552; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=KapJsJDXZSaTZ0KCMkqJA+w32osB4ivWYrQOtexabwQ=; b=TrXZbzXJvXjAFMtmdjbY6g43FEEUJ51e56dkvtg+kfgYdfON7FtAc+d4i+2limdYih Fjpb36idC7mIVH+9ApO7W85B+EBgNR1y6VZFqbg9bjcKv172M5172Tu0I1dzmE0TZhX3 D9a4xyqNrw76UX3aD/sp0MisPY3yUWWKy6fXVSHGN4RpngnPjxMpr1WT6B57I5nDMM9h IGtYIrkL6QnQtXLq3ifiyUfnkXFyNu8goNHHb5EOEskvSfgSpgVq/fQOHdiQZkOKHfsp pRjXnhp8+gJFBM/PT+CJZnzWIouDqwd+pTmAWoAH8rgQR31KG+5TVJ87+VibBetAjN4U 9KGQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689970752; x=1690575552; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=KapJsJDXZSaTZ0KCMkqJA+w32osB4ivWYrQOtexabwQ=; b=RUOv8jvbXetdtINDT/14WhPjhxxpX31Z1U95Omh5sR15DO+xxKTRbLLbrHtiG/X2C2 zltzAERAY6vY3jshWLtB0kjanCm3xNFfQB2b7A+GlxJWd5CNEZoVwqjyGWweiB82cVa3 5hSB2ox0jNwADFrrui1gU7V+/mXQwJ/zz8B66chTZEy2acwHDYNZ09ZnJfUnn9uAKCmj RX2izTEuwrpMviwDK/Vaw8JnsVysOkFlthz1SF562aCK/DVqSGElV08Ev4S7/jC5lDTs v0HPa/cUzD53U0wBDUcaDaSQI3yHaBtRZc6XKoeWIEkCvr5qF7hBKbxz8+6QNOvpfMKM 9IxQ== X-Gm-Message-State: ABy/qLZMD2lVe1yhT5uqmYVFeFRyFuZD0cA6mpE8lczuSCAXkErMh2pU J8XA7c7DYu0/2K9vom9UPNVa4pcT40c= X-Google-Smtp-Source: APBJJlGu5YOBwzaswNuu+hMFwXKYPw6WfFvR2MaNIQNXi7dan4IRvarFADNDlK9IoFXtUVAADyo+h6fqziI= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a81:ad24:0:b0:56c:e53d:ae90 with SMTP id l36-20020a81ad24000000b0056ce53dae90mr13963ywh.4.1689970752188; Fri, 21 Jul 2023 13:19:12 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 21 Jul 2023 13:18:44 -0700 In-Reply-To: <20230721201859.2307736-1-seanjc@google.com> Mime-Version: 1.0 References: <20230721201859.2307736-1-seanjc@google.com> X-Mailer: git-send-email 2.41.0.487.g6d72f3e995-goog Message-ID: <20230721201859.2307736-5-seanjc@google.com> Subject: [PATCH v4 04/19] x86/reboot: KVM: Disable SVM during reboot via virt/KVM reboot callback From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, Sean Christopherson , Paolo Bonzini Cc: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, Andrew Cooper , Kai Huang , Chao Gao Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Use the virt callback to disable SVM (and set GIF=3D1) during an emergency instead of blindly attempting to disable SVM. Like the VMX case, if a hypervisor, i.e. KVM, isn't loaded/active, SVM can't be in use. Acked-by: Kai Huang Signed-off-by: Sean Christopherson --- arch/x86/include/asm/virtext.h | 8 -------- arch/x86/kernel/reboot.c | 3 --- arch/x86/kvm/svm/svm.c | 19 +++++++++++++++++-- 3 files changed, 17 insertions(+), 13 deletions(-) diff --git a/arch/x86/include/asm/virtext.h b/arch/x86/include/asm/virtext.h index 5bc29fab15da..aaed66249ccf 100644 --- a/arch/x86/include/asm/virtext.h +++ b/arch/x86/include/asm/virtext.h @@ -133,12 +133,4 @@ static inline void cpu_svm_disable(void) } } =20 -/** Makes sure SVM is disabled, if it is supported on the CPU - */ -static inline void cpu_emergency_svm_disable(void) -{ - if (cpu_has_svm(NULL)) - cpu_svm_disable(); -} - #endif /* _ASM_X86_VIRTEX_H */ diff --git a/arch/x86/kernel/reboot.c b/arch/x86/kernel/reboot.c index d2d0f2672a64..48ad2d1ff83d 100644 --- a/arch/x86/kernel/reboot.c +++ b/arch/x86/kernel/reboot.c @@ -826,9 +826,6 @@ void cpu_emergency_disable_virtualization(void) if (callback) callback(); rcu_read_unlock(); - - /* KVM_AMD doesn't yet utilize the common callback. */ - cpu_emergency_svm_disable(); } =20 #if defined(CONFIG_SMP) diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index d381ad424554..1ae9c2c7eacb 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c @@ -39,6 +39,7 @@ #include #include #include +#include #include =20 #include @@ -563,6 +564,11 @@ void __svm_write_tsc_multiplier(u64 multiplier) preempt_enable(); } =20 +static void svm_emergency_disable(void) +{ + cpu_svm_disable(); +} + static void svm_hardware_disable(void) { /* Make sure we clean up behind us */ @@ -5209,6 +5215,13 @@ static struct kvm_x86_init_ops svm_init_ops __initda= ta =3D { .pmu_ops =3D &amd_pmu_ops, }; =20 +static void __svm_exit(void) +{ + kvm_x86_vendor_exit(); + + cpu_emergency_unregister_virt_callback(svm_emergency_disable); +} + static int __init svm_init(void) { int r; @@ -5222,6 +5235,8 @@ static int __init svm_init(void) if (r) return r; =20 + cpu_emergency_register_virt_callback(svm_emergency_disable); + /* * Common KVM initialization _must_ come last, after this, /dev/kvm is * exposed to userspace! @@ -5234,14 +5249,14 @@ static int __init svm_init(void) return 0; =20 err_kvm_init: - kvm_x86_vendor_exit(); + __svm_exit(); return r; } =20 static void __exit svm_exit(void) { kvm_exit(); - kvm_x86_vendor_exit(); + __svm_exit(); } =20 module_init(svm_init) --=20 2.41.0.487.g6d72f3e995-goog From nobody Thu Sep 18 20:23:28 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6973DEB64DD for ; Fri, 21 Jul 2023 20:19:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231304AbjGUUTp (ORCPT ); Fri, 21 Jul 2023 16:19:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37978 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230291AbjGUUTT (ORCPT ); Fri, 21 Jul 2023 16:19:19 -0400 Received: from mail-yb1-xb49.google.com (mail-yb1-xb49.google.com [IPv6:2607:f8b0:4864:20::b49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 064CD12F for ; Fri, 21 Jul 2023 13:19:15 -0700 (PDT) Received: by mail-yb1-xb49.google.com with SMTP id 3f1490d57ef6-bd69bb4507eso2275796276.2 for ; Fri, 21 Jul 2023 13:19:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1689970754; x=1690575554; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=RNIzxG+DyKxhv7/WUhcg42RyR1LzF+A7NEJm4Wy1bYU=; b=VLZQUdsQavq82AMjpjS9rdHI6jO9sP6xssXKUkaB7U/GQwL8nid7eG6wVk1JwgqkoT FFTkgw45ADeMUlu5ALDnr115Ixck3boFbgPyZpQreyFzl6yJ+S/V+dJuYxQ5XWvOE/eI 4kkQnZ9H9c5aWJmAayTqrjC0yCMJ1b22MPWz1FBwlIMfjWdFE90FsWNd70jKxCl95sG3 QXIGYOX/P4AermBbmroCFajzBcZIR2L3b4+vbW3tHy384n+r5uiD9o4Vvd532w5CUlex ObV2LDr84QRbQEztcWeYQ+Dwl6PU4ML/7junWpBh6RsYoPWUnLQw+9OhPN5uWwkg48ZQ mHIw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689970754; x=1690575554; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=RNIzxG+DyKxhv7/WUhcg42RyR1LzF+A7NEJm4Wy1bYU=; b=PRrY6mNAS1gij1NGKky9Fj3nfv0PIAejt7v+69kZOqNJdmdc2XyVU8MIr2HkxrGr/y A9KIUkvPeEwrEBnPqHq+ZmiGDgjomXF9xNtPRbaVWWBTnI5YQE//2ZWQ1xPjc+BzVqei ei6BnsgiLE6Ky5ZxA8SvP0omUd5UJ9Z6nHCOvH1v+MH0fVlzU4kf6UGMGu2JAx4BmzWa HgDYMXUOX3Hvg1l2QDSNBPdYWSuHThH13MoAvCpNhb/WhbuPdigmuxq0kWG/bxTG0Xk4 j5ym3y4WfSD9rKhAhJ699KfrsZznr0C44cLOQLLymwWZQ0/AbLzpKwm9XQtsSr4cONzP E9hA== X-Gm-Message-State: ABy/qLZYKzdUVBbCCgZtYnEvW0YrOpOXuimsnIdieeRatSxOz2yZ30jt 47iTK2ETIdAjMu254WnBo0tVjpKNxx0= X-Google-Smtp-Source: APBJJlF/B9hp1aDOjZGhKyt3iXYT71YIQNpTjE8kJ9mxksplOcD8TO2n3UiT2Q1QOPU47P+izOZbgiJ3zSE= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a25:b310:0:b0:cfd:d50f:df5b with SMTP id l16-20020a25b310000000b00cfdd50fdf5bmr17452ybj.13.1689970754335; Fri, 21 Jul 2023 13:19:14 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 21 Jul 2023 13:18:45 -0700 In-Reply-To: <20230721201859.2307736-1-seanjc@google.com> Mime-Version: 1.0 References: <20230721201859.2307736-1-seanjc@google.com> X-Mailer: git-send-email 2.41.0.487.g6d72f3e995-goog Message-ID: <20230721201859.2307736-6-seanjc@google.com> Subject: [PATCH v4 05/19] x86/reboot: Assert that IRQs are disabled when turning off virtualization From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, Sean Christopherson , Paolo Bonzini Cc: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, Andrew Cooper , Kai Huang , Chao Gao Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Assert that IRQs are disabled when turning off virtualization in an emergency. KVM enables hardware via on_each_cpu(), i.e. could re-enable hardware if a pending IPI were delivered after disabling virtualization. Remove a misleading comment from emergency_reboot_disable_virtualization() about "just" needing to guarantee the CPU is stable (see above). Reviewed-by: Kai Huang Signed-off-by: Sean Christopherson --- arch/x86/kernel/reboot.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/arch/x86/kernel/reboot.c b/arch/x86/kernel/reboot.c index 48ad2d1ff83d..4cad7183b89e 100644 --- a/arch/x86/kernel/reboot.c +++ b/arch/x86/kernel/reboot.c @@ -532,7 +532,6 @@ static inline void nmi_shootdown_cpus_on_restart(void); =20 static void emergency_reboot_disable_virtualization(void) { - /* Just make sure we won't change CPUs while doing this */ local_irq_disable(); =20 /* @@ -821,6 +820,13 @@ void cpu_emergency_disable_virtualization(void) { cpu_emergency_virt_cb *callback; =20 + /* + * IRQs must be disabled as KVM enables virtualization in hardware via + * function call IPIs, i.e. IRQs need to be disabled to guarantee + * virtualization stays disabled. + */ + lockdep_assert_irqs_disabled(); + rcu_read_lock(); callback =3D rcu_dereference(cpu_emergency_virt_callback); if (callback) --=20 2.41.0.487.g6d72f3e995-goog From nobody Thu Sep 18 20:23:28 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CB3DFC0015E for ; Fri, 21 Jul 2023 20:19:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231277AbjGUUTi (ORCPT ); Fri, 21 Jul 2023 16:19:38 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37980 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230375AbjGUUTU (ORCPT ); Fri, 21 Jul 2023 16:19:20 -0400 Received: from mail-pl1-x64a.google.com (mail-pl1-x64a.google.com [IPv6:2607:f8b0:4864:20::64a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0C48F30CA for ; Fri, 21 Jul 2023 13:19:17 -0700 (PDT) Received: by mail-pl1-x64a.google.com with SMTP id d9443c01a7336-1b888bdacbcso12652395ad.2 for ; Fri, 21 Jul 2023 13:19:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1689970756; x=1690575556; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=Mde8vJ2EtcoesCeVYOs5lIQ6ce5w4DlvdNVLFNYZgvw=; b=OG1xATxce96iuWnMzlgqDnDaZv2FXzskZA+y4BUtx5VmlP67RNa2QHKDFmey4WYDT7 zvNUXdMCZrwTkCSrECGSiGHB6pP4XPFVEmSyDGZDd8vLnmjzw2ELySlD1Ofsw6BwcCYW dTl3nD8gBC7wcs2gpf74X3VknyBBLWpQT/S+GlaYcetZzIGEyFZR0khZhhEpMpUsfHS/ tzCGxebOb956the72GUxF9v7PXeo3Wuf+ivVaHjOogwQNBQ3QYf+d5JiF2PYnWcl7EzX Y7QIP/wBx9b3lUz+RkVAiiY1YweC3T0ki70Hv16Hyb3COP5kSj4beryjARpodF6LnY6v Y/Jg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689970756; x=1690575556; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=Mde8vJ2EtcoesCeVYOs5lIQ6ce5w4DlvdNVLFNYZgvw=; b=KRsBdwcgJweWUQdK1iYvuU8tYVOIJgtBag4ozRTI7GD3NWHmaUxBv0qtaxvubTGOlb L10xambda66QXTwDFGaxBWYDYSAOoB9tTa/iMCrdlH+bO74yTBbb/QnzlQzRcyW1gBb8 EoVjiiTmWeYvpBGvor3EAdgFIm2TbLs0WwN8iwZW6mSixJ3AwJbuTDB1gTjdLiOY8TtY 12v+PaKBfBvzteTHd8lsJiN4OgQ9Gke478CVlVhf31USs15n2z7ZGiUGy6nGBltAOt65 HMcqUrf3pp5f1V4e0TDNXcQ2zLfRqFOMMzl0c0SM39GNdhLCzqvw6LFHLZ7wiROJIuxb A9qA== X-Gm-Message-State: ABy/qLZWCRJXlBFRKgTqTr8Urfd4tN8xZJjhPN0Q6ypeTByP6Y6OlBjp g1u+iNoeAwnlnciQPF6WZPW8wOW+sE0= X-Google-Smtp-Source: APBJJlGlHNXn19ATaXD6Ps+chy94B6SDHHnp2N5mtBL/FJxhcaQpXhy1QBTb+21BUjq1Om4DVC5jIj0WCFo= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a17:903:1c7:b0:1b5:2496:8c0d with SMTP id e7-20020a17090301c700b001b524968c0dmr11698plh.3.1689970756405; Fri, 21 Jul 2023 13:19:16 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 21 Jul 2023 13:18:46 -0700 In-Reply-To: <20230721201859.2307736-1-seanjc@google.com> Mime-Version: 1.0 References: <20230721201859.2307736-1-seanjc@google.com> X-Mailer: git-send-email 2.41.0.487.g6d72f3e995-goog Message-ID: <20230721201859.2307736-7-seanjc@google.com> Subject: [PATCH v4 06/19] x86/reboot: Hoist "disable virt" helpers above "emergency reboot" path From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, Sean Christopherson , Paolo Bonzini Cc: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, Andrew Cooper , Kai Huang , Chao Gao Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Move the various "disable virtualization" helpers above the emergency reboot path so that emergency_reboot_disable_virtualization() can be stubbed out in a future patch if neither KVM_INTEL nor KVM_AMD is enabled, i.e. if there is no in-tree user of CPU virtualization. No functional change intended. Reviewed-by: Kai Huang Signed-off-by: Sean Christopherson --- arch/x86/kernel/reboot.c | 90 ++++++++++++++++++++-------------------- 1 file changed, 45 insertions(+), 45 deletions(-) diff --git a/arch/x86/kernel/reboot.c b/arch/x86/kernel/reboot.c index 4cad7183b89e..85cb2dfcb67b 100644 --- a/arch/x86/kernel/reboot.c +++ b/arch/x86/kernel/reboot.c @@ -530,6 +530,51 @@ static inline void kb_wait(void) =20 static inline void nmi_shootdown_cpus_on_restart(void); =20 +/* RCU-protected callback to disable virtualization prior to reboot. */ +static cpu_emergency_virt_cb __rcu *cpu_emergency_virt_callback; + +void cpu_emergency_register_virt_callback(cpu_emergency_virt_cb *callback) +{ + if (WARN_ON_ONCE(rcu_access_pointer(cpu_emergency_virt_callback))) + return; + + rcu_assign_pointer(cpu_emergency_virt_callback, callback); +} +EXPORT_SYMBOL_GPL(cpu_emergency_register_virt_callback); + +void cpu_emergency_unregister_virt_callback(cpu_emergency_virt_cb *callbac= k) +{ + if (WARN_ON_ONCE(rcu_access_pointer(cpu_emergency_virt_callback) !=3D cal= lback)) + return; + + rcu_assign_pointer(cpu_emergency_virt_callback, NULL); + synchronize_rcu(); +} +EXPORT_SYMBOL_GPL(cpu_emergency_unregister_virt_callback); + +/* + * Disable virtualization, i.e. VMX or SVM, to ensure INIT is recognized d= uring + * reboot. VMX blocks INIT if the CPU is post-VMXON, and SVM blocks INIT = if + * GIF=3D0, i.e. if the crash occurred between CLGI and STGI. + */ +void cpu_emergency_disable_virtualization(void) +{ + cpu_emergency_virt_cb *callback; + + /* + * IRQs must be disabled as KVM enables virtualization in hardware via + * function call IPIs, i.e. IRQs need to be disabled to guarantee + * virtualization stays disabled. + */ + lockdep_assert_irqs_disabled(); + + rcu_read_lock(); + callback =3D rcu_dereference(cpu_emergency_virt_callback); + if (callback) + callback(); + rcu_read_unlock(); +} + static void emergency_reboot_disable_virtualization(void) { local_irq_disable(); @@ -786,54 +831,9 @@ void machine_crash_shutdown(struct pt_regs *regs) } #endif =20 -/* RCU-protected callback to disable virtualization prior to reboot. */ -static cpu_emergency_virt_cb __rcu *cpu_emergency_virt_callback; - -void cpu_emergency_register_virt_callback(cpu_emergency_virt_cb *callback) -{ - if (WARN_ON_ONCE(rcu_access_pointer(cpu_emergency_virt_callback))) - return; - - rcu_assign_pointer(cpu_emergency_virt_callback, callback); -} -EXPORT_SYMBOL_GPL(cpu_emergency_register_virt_callback); - -void cpu_emergency_unregister_virt_callback(cpu_emergency_virt_cb *callbac= k) -{ - if (WARN_ON_ONCE(rcu_access_pointer(cpu_emergency_virt_callback) !=3D cal= lback)) - return; - - rcu_assign_pointer(cpu_emergency_virt_callback, NULL); - synchronize_rcu(); -} -EXPORT_SYMBOL_GPL(cpu_emergency_unregister_virt_callback); - /* This is the CPU performing the emergency shutdown work. */ int crashing_cpu =3D -1; =20 -/* - * Disable virtualization, i.e. VMX or SVM, to ensure INIT is recognized d= uring - * reboot. VMX blocks INIT if the CPU is post-VMXON, and SVM blocks INIT = if - * GIF=3D0, i.e. if the crash occurred between CLGI and STGI. - */ -void cpu_emergency_disable_virtualization(void) -{ - cpu_emergency_virt_cb *callback; - - /* - * IRQs must be disabled as KVM enables virtualization in hardware via - * function call IPIs, i.e. IRQs need to be disabled to guarantee - * virtualization stays disabled. - */ - lockdep_assert_irqs_disabled(); - - rcu_read_lock(); - callback =3D rcu_dereference(cpu_emergency_virt_callback); - if (callback) - callback(); - rcu_read_unlock(); -} - #if defined(CONFIG_SMP) =20 static nmi_shootdown_cb shootdown_callback; --=20 2.41.0.487.g6d72f3e995-goog From nobody Thu Sep 18 20:23:28 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AA8A4EB64DD for ; Fri, 21 Jul 2023 20:19:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231269AbjGUUTf (ORCPT ); Fri, 21 Jul 2023 16:19:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37916 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230490AbjGUUTX (ORCPT ); Fri, 21 Jul 2023 16:19:23 -0400 Received: from mail-pl1-x64a.google.com (mail-pl1-x64a.google.com [IPv6:2607:f8b0:4864:20::64a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D32073A87 for ; Fri, 21 Jul 2023 13:19:18 -0700 (PDT) Received: by mail-pl1-x64a.google.com with SMTP id d9443c01a7336-1b8a4571c1aso12775545ad.0 for ; Fri, 21 Jul 2023 13:19:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1689970758; x=1690575558; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=Xl/21nVHjxj9H71udXldkW3KRDjTiiyAWxQB92FBmLk=; b=Z9kjVXVGPt914xe9fLgGWQ7njz6z0IpDO/S9kHemlYQXnKLvSITcv91xTIWcVJtv4m CbGJMGVWN0Jx7yI0SSDzh5WgyereyRUc/tEyDpGT6MLizJYYxihqtQBSefCIu/1aboBj 0pFG/ifvnhuy1V3QTj222F9L3dkjAm/fM+mZ5LvJsm2PGxEdi2yFpOUZ6DMSDNxMm4DI o/VS8q8UCReLs1GvOJfJFGFkGBUim2JZRxl8RuCfyBdOzDRbkrsJk1z5x/ZIP4GwZ4+o 3wsH96lRHH20QOA7RyxWEHtNxRZlo33VGObc3zEeEV5ho06ifM+g9OcDd0QadVn+P/E+ TuFg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689970758; x=1690575558; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=Xl/21nVHjxj9H71udXldkW3KRDjTiiyAWxQB92FBmLk=; b=LRlhr7uGpyTcSSOPpOeT2tovIR2oAM3uLriwHVozPXtE1kZSvoG8ynd02/B8PcdgmF /+a6F2cYCVb52Zkc9em4tma0tfy+KdgPFQhbAJUc3YgBs37tl7WxdAT41+g3yZI1ady0 NHDj7qPa5zmdzVm2/6twbCim3MqFSW2y7UB2BcqGozgvnAZmgtbUBeqqW/QgDg59NjDl WoKU7BIeksRxGw9PBCNMJ37auosCTgrR2HHcODx3/oRqjYqvsXmMIIuz9ML5qqtLl/ks PzQDqOeglgcBPtu1U0CKl3YcGzV1Pq7FJv3l7m3Yx2mheuKm5IuiX/9TxZBihOl+6/vp 9sGA== X-Gm-Message-State: ABy/qLYkSFYqbMj1vGbOPnVfQGnwmGaRvPYUlbR4a1ea/pXOnBjLtjjJ MeHBW13rElX/EziIBN2PRW/zjS9PwyY= X-Google-Smtp-Source: APBJJlEzwu5ChQyLGT+FTc9q1c52pU5AreTLyPoJzZDa1ixxgiv0ygl5+KZatUpkRn4shtsG7YxA8qtZsN8= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a17:902:ecc1:b0:1b8:95fc:d0f with SMTP id a1-20020a170902ecc100b001b895fc0d0fmr13723plh.7.1689970758296; Fri, 21 Jul 2023 13:19:18 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 21 Jul 2023 13:18:47 -0700 In-Reply-To: <20230721201859.2307736-1-seanjc@google.com> Mime-Version: 1.0 References: <20230721201859.2307736-1-seanjc@google.com> X-Mailer: git-send-email 2.41.0.487.g6d72f3e995-goog Message-ID: <20230721201859.2307736-8-seanjc@google.com> Subject: [PATCH v4 07/19] x86/reboot: Disable virtualization during reboot iff callback is registered From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, Sean Christopherson , Paolo Bonzini Cc: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, Andrew Cooper , Kai Huang , Chao Gao Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Attempt to disable virtualization during an emergency reboot if and only if there is a registered virt callback, i.e. iff a hypervisor (KVM) is active. If there's no active hypervisor, then the CPU can't be operating with VMX or SVM enabled (barring an egregious bug). Checking for a valid callback instead of simply for SVM or VMX support can also eliminates spurious NMIs by avoiding the unecessary call to nmi_shootdown_cpus_on_restart(). Note, IRQs are disabled, which prevents KVM from coming along and enabling virtualization after the fact. Signed-off-by: Sean Christopherson Reviewed-by: Kai Huang --- arch/x86/kernel/reboot.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/arch/x86/kernel/reboot.c b/arch/x86/kernel/reboot.c index 85cb2dfcb67b..98e5db3fd7f4 100644 --- a/arch/x86/kernel/reboot.c +++ b/arch/x86/kernel/reboot.c @@ -22,7 +22,6 @@ #include #include #include -#include #include #include #include @@ -589,7 +588,7 @@ static void emergency_reboot_disable_virtualization(voi= d) * Do the NMI shootdown even if virtualization is off on _this_ CPU, as * other CPUs may have virtualization enabled. */ - if (cpu_has_vmx() || cpu_has_svm(NULL)) { + if (rcu_access_pointer(cpu_emergency_virt_callback)) { /* Safely force _this_ CPU out of VMX/SVM operation. */ cpu_emergency_disable_virtualization(); =20 --=20 2.41.0.487.g6d72f3e995-goog From nobody Thu Sep 18 20:23:28 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C6A54C0015E for ; Fri, 21 Jul 2023 20:19:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231355AbjGUUTx (ORCPT ); Fri, 21 Jul 2023 16:19:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38246 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231179AbjGUUT1 (ORCPT ); Fri, 21 Jul 2023 16:19:27 -0400 Received: from mail-yw1-x1149.google.com (mail-yw1-x1149.google.com [IPv6:2607:f8b0:4864:20::1149]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D95083A8C for ; Fri, 21 Jul 2023 13:19:21 -0700 (PDT) Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-583994e0bc5so12901007b3.0 for ; Fri, 21 Jul 2023 13:19:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1689970760; x=1690575560; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=kp8wKDt0eMxNmmgxbfIQ59HsgNC9pjZ5TgTeqdMxU/I=; b=AAcenZvw756VBG7/uWjvcLB2GB6SaD9rHsGVP1ovtfQ7g3P1/K5Es/5cP7bQX4Cett TXIqUcFS8rwOAQTdffRrYdTVUeGqUN4Pk1zQVEpgyHZgM5CU4q7BAddWItw5Q7B/HUFX dm9BEq62JHh36Dyv0YCxgBb//gvHCL44aM4hNFvAWv9IYaRXhunR1CtejhbcQSL/3BKu jhhLx9apEs7AWteNUlsHKFY1npg1EGGfPoIogyCbJDG+NVcTCP8a4KpbcRzwBW7l7aEc fnDXtssi6A7cJBIy/mxf0JRinHLH1Q1C9G7n2rTaBE2AVxXbRmy7Sib5y3L9HVmp9h0H 1bJA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689970760; x=1690575560; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=kp8wKDt0eMxNmmgxbfIQ59HsgNC9pjZ5TgTeqdMxU/I=; b=jrwqRpOghM2r0mFoDborV9a9yuYvhTDdkiW68QJ6YHB04/z8nJ02oeBhqiAWznPmYE Uccj+Q4mVSzRf2A/5riwDi/8TnZqDL9p4Wjb5dEvPQNdvrp3GgvaCTe2+VNu89Xfuzlu b5nQYsN8Jm4nclM4vTz5eJvmxqt85eInm4/hPv4UQsvMifYB3AeqfBE+Q0SKR8hV3W2x 0n5rHXDkMvjC/NiQmrwBbUeq2HooFhJgzT4NUt5NGDrnuyk2VnhOeyzMev2OOmbYJQN/ Pza+NfgEaWV6BWuwcIx6welwiTmdzZDGlJ+OBptThOCvoz6A3js+HGiP8FGGmQr4I0wj 9aQQ== X-Gm-Message-State: ABy/qLbg8EfWdYR7UShtk6j/bNzGVq9IJCwFFmuvmg2di8iJ5jcDViCp DmDHkhgkW/qrm0GyXwNFPFHWzMKI0xk= X-Google-Smtp-Source: APBJJlGTfQ3Re85Ub4mDu2d7yq7yFxRBK4TrexqrcdueUqc8FNScDCwWwNnAZeh/fkd/oR5J515PtlELu78= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a81:ac14:0:b0:581:3899:91bc with SMTP id k20-20020a81ac14000000b00581389991bcmr9159ywh.6.1689970760558; Fri, 21 Jul 2023 13:19:20 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 21 Jul 2023 13:18:48 -0700 In-Reply-To: <20230721201859.2307736-1-seanjc@google.com> Mime-Version: 1.0 References: <20230721201859.2307736-1-seanjc@google.com> X-Mailer: git-send-email 2.41.0.487.g6d72f3e995-goog Message-ID: <20230721201859.2307736-9-seanjc@google.com> Subject: [PATCH v4 08/19] x86/reboot: Expose VMCS crash hooks if and only if KVM_{INTEL,AMD} is enabled From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, Sean Christopherson , Paolo Bonzini Cc: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, Andrew Cooper , Kai Huang , Chao Gao Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Expose the crash/reboot hooks used by KVM to disable virtualization in hardware and unblock INIT only if there's a potential in-tree user, i.e. either KVM_INTEL or KVM_AMD is enabled. Reviewed-by: Kai Huang Signed-off-by: Sean Christopherson --- arch/x86/include/asm/reboot.h | 4 ++++ arch/x86/kernel/reboot.c | 5 ++++- 2 files changed, 8 insertions(+), 1 deletion(-) diff --git a/arch/x86/include/asm/reboot.h b/arch/x86/include/asm/reboot.h index 74c6a624d166..6536873f8fc0 100644 --- a/arch/x86/include/asm/reboot.h +++ b/arch/x86/include/asm/reboot.h @@ -25,10 +25,14 @@ void __noreturn machine_real_restart(unsigned int type); #define MRR_BIOS 0 #define MRR_APM 1 =20 +#if IS_ENABLED(CONFIG_KVM_INTEL) || IS_ENABLED(CONFIG_KVM_AMD) typedef void (cpu_emergency_virt_cb)(void); void cpu_emergency_register_virt_callback(cpu_emergency_virt_cb *callback); void cpu_emergency_unregister_virt_callback(cpu_emergency_virt_cb *callbac= k); void cpu_emergency_disable_virtualization(void); +#else +static inline void cpu_emergency_disable_virtualization(void) {} +#endif /* CONFIG_KVM_INTEL || CONFIG_KVM_AMD */ =20 typedef void (*nmi_shootdown_cb)(int, struct pt_regs*); void nmi_shootdown_cpus(nmi_shootdown_cb callback); diff --git a/arch/x86/kernel/reboot.c b/arch/x86/kernel/reboot.c index 98e5db3fd7f4..830425e6d38e 100644 --- a/arch/x86/kernel/reboot.c +++ b/arch/x86/kernel/reboot.c @@ -529,6 +529,7 @@ static inline void kb_wait(void) =20 static inline void nmi_shootdown_cpus_on_restart(void); =20 +#if IS_ENABLED(CONFIG_KVM_INTEL) || IS_ENABLED(CONFIG_KVM_AMD) /* RCU-protected callback to disable virtualization prior to reboot. */ static cpu_emergency_virt_cb __rcu *cpu_emergency_virt_callback; =20 @@ -596,7 +597,9 @@ static void emergency_reboot_disable_virtualization(voi= d) nmi_shootdown_cpus_on_restart(); } } - +#else +static void emergency_reboot_disable_virtualization(void) { } +#endif /* CONFIG_KVM_INTEL || CONFIG_KVM_AMD */ =20 void __attribute__((weak)) mach_reboot_fixups(void) { --=20 2.41.0.487.g6d72f3e995-goog From nobody Thu Sep 18 20:23:28 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8DDFCEB64DD for ; Fri, 21 Jul 2023 20:19:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231312AbjGUUT4 (ORCPT ); Fri, 21 Jul 2023 16:19:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37962 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231228AbjGUUTb (ORCPT ); Fri, 21 Jul 2023 16:19:31 -0400 Received: from mail-yw1-x114a.google.com (mail-yw1-x114a.google.com [IPv6:2607:f8b0:4864:20::114a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AE35C35BB for ; Fri, 21 Jul 2023 13:19:23 -0700 (PDT) Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-5834d9ec5f7so55179147b3.1 for ; Fri, 21 Jul 2023 13:19:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1689970763; x=1690575563; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=BxIdNivQmBNQSOlXPPIQGt5Qzqr/3+s4YS24y1+Qiwc=; b=Gu6EPZofnedmVcu0puGF5uwGMb+h5xD1qVMdFfjhDXwf0ojqQzZcD1qu3BQJegbXvr KLDtNr2gxNXiIGG01q1nWzcC8TWglrMhxSng4NJhIYe0BpCdke22vxvfKTVbuomgvUAN S0b2LRn9a7StmxWvePSakXfo0f6VjA0axqDjd3pZE1xoRBMdZO313rBIR4eu9qDLo1wM usH/HLHPIkA/2/CyF4G08TZ7X9D3vVMe5uXA535W675kmFLdT8+6jOaYRnBeap6S3TBj xg8Pj92dho+Sxc3VU4JDANpOH3JDVZFimXcHgJArCZfVA+xuGI+5RJ5z3B3A6VLxOY+a DVPg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689970763; x=1690575563; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=BxIdNivQmBNQSOlXPPIQGt5Qzqr/3+s4YS24y1+Qiwc=; b=VgknbKuV9GgHZQ3qnwgEUYwTAMn1YVYPVIinZM/Xn3vij2MJAUkPnVUVGud7nn8bTh HfLEZ8u6VoQrTdp0IvQNDXN3RksyYj45Z/wB6yCOe2rY5lpeA89V7Prvm7jNdCM45PTK oyc8sXs2X354RoVOaPDSVlL4hGpvXKEA+iYZjp0Gf7ZkCvtCgVaYZEMwkPlbUOxjxA5g 7WsgFUaz1FEmdfTfuRYkUIOA/YFf76mwZ/GSzpJ8Tgn8V5guUwXVCbZDsbPTktOnFW8A cfHSrVYcflKw+4QUf1Nf2VptELdfrw6xqyNf0Py1wOVwQnnzdwWTmHcDTNWRxcp5VWtH tTNA== X-Gm-Message-State: ABy/qLbYj06gyFgavmWUB1EYKMM1mgDhcr1Ctkoy6XBXoYQfgYvFvxDq 2ZC6ONtGXVSm+dzK/+VNKig/OwCNzRg= X-Google-Smtp-Source: APBJJlH+JZBH+A2VRttCcgXplZCHJDHZLy6DgW6Igx6xsBqN18TpvDYqMpdpfHZnvHRh1rUMWWVU6dzjIzg= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a5b:a8d:0:b0:d04:6d5e:4ded with SMTP id h13-20020a5b0a8d000000b00d046d5e4dedmr25333ybq.6.1689970762838; Fri, 21 Jul 2023 13:19:22 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 21 Jul 2023 13:18:49 -0700 In-Reply-To: <20230721201859.2307736-1-seanjc@google.com> Mime-Version: 1.0 References: <20230721201859.2307736-1-seanjc@google.com> X-Mailer: git-send-email 2.41.0.487.g6d72f3e995-goog Message-ID: <20230721201859.2307736-10-seanjc@google.com> Subject: [PATCH v4 09/19] x86/virt: KVM: Open code cpu_has_vmx() in KVM VMX From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, Sean Christopherson , Paolo Bonzini Cc: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, Andrew Cooper , Kai Huang , Chao Gao Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Fold the raw CPUID check for VMX into kvm_is_vmx_supported(), its sole user. Keep the check even though KVM also checks X86_FEATURE_VMX, as the intent is to provide a unique error message if VMX is unsupported by hardware, whereas X86_FEATURE_VMX may be clear due to firmware and/or kernel actions. No functional change intended. Reviewed-by: Kai Huang Signed-off-by: Sean Christopherson --- arch/x86/include/asm/virtext.h | 10 ---------- arch/x86/kvm/vmx/vmx.c | 2 +- 2 files changed, 1 insertion(+), 11 deletions(-) diff --git a/arch/x86/include/asm/virtext.h b/arch/x86/include/asm/virtext.h index aaed66249ccf..b1171a5ad452 100644 --- a/arch/x86/include/asm/virtext.h +++ b/arch/x86/include/asm/virtext.h @@ -22,14 +22,6 @@ /* * VMX functions: */ - -static inline int cpu_has_vmx(void) -{ - unsigned long ecx =3D cpuid_ecx(1); - return test_bit(5, &ecx); /* CPUID.1:ECX.VMX[bit 5] -> VT */ -} - - /** * cpu_vmxoff() - Disable VMX on the current CPU * @@ -61,8 +53,6 @@ static inline int cpu_vmx_enabled(void) } =20 /** Disable VMX if it is enabled on the current CPU - * - * You shouldn't call this if cpu_has_vmx() returns 0. */ static inline void __cpu_emergency_vmxoff(void) { diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 682c20b33a96..71571cd9adbb 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -2718,7 +2718,7 @@ static bool kvm_is_vmx_supported(void) { int cpu =3D raw_smp_processor_id(); =20 - if (!cpu_has_vmx()) { + if (!(cpuid_ecx(1) & feature_bit(VMX))) { pr_err("VMX not supported by CPU %d\n", cpu); return false; } --=20 2.41.0.487.g6d72f3e995-goog From nobody Thu Sep 18 20:23:28 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id F12ACEB64DD for ; Fri, 21 Jul 2023 20:20:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231415AbjGUUT7 (ORCPT ); Fri, 21 Jul 2023 16:19:59 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38758 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231311AbjGUUTp (ORCPT ); Fri, 21 Jul 2023 16:19:45 -0400 Received: from mail-pl1-x649.google.com (mail-pl1-x649.google.com [IPv6:2607:f8b0:4864:20::649]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E04F6B7 for ; Fri, 21 Jul 2023 13:19:25 -0700 (PDT) Received: by mail-pl1-x649.google.com with SMTP id d9443c01a7336-1b9de7951easo13878525ad.0 for ; Fri, 21 Jul 2023 13:19:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1689970765; x=1690575565; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=BABBYG9T7Yn2sZTHqi5m2u4CXRQ2CmLvyTIjco5c63g=; b=L1pQH1k9NYEONtiByxTTURMw61jXy4L9Kk3ri6cUN6Z+iQ6l9paGL8Mts/CoI1ah8E p5sh/GxpWGNRTHR8gZR3aL7Z1eHd7efWK8WBAT9JhKhc/YN0RxnOWyQ2JXr/nsvMvRF8 T8UWXdr1r33MMP2c9F+tPUT0Vx9KzkmzaqkWHFTgRAcqdD3k8r+UJhpBa3ubw0HY3ib2 imDCqrcruiJZnhVO9V3DRNlFTLXF0AKlqECEJy2mZKhEppb21CSS06AmUP0RyBFxFEon XhNIiCfxnzxVY8X+aZED+NIEf02YSjU1tdYg+DMuihX7XrM45w+LABXt9Gv1lLloQIHt eExQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689970765; x=1690575565; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=BABBYG9T7Yn2sZTHqi5m2u4CXRQ2CmLvyTIjco5c63g=; b=QuplGhVHuzJvyuHnPRXEuLAPiKH17YK5t3cqVHkZB9HP/4/WRHdkSPs8hcGq5/hB2S +DEgePG4db1gnM9X+/2ePxJCml5s6LWe54B1qphymU1LWPEBwecLPkvSMUznayrMi7+A ktqV3QUPaKBrAWos1hjI04fBTGX9jaNEah1TZGp2MvKiiZ254TDYUzmOgqCafq92Ei/K Ii4D3A6mQznr+lOplFDhlMvjvdZP5GFm3v5zwXfIdlkzTzCM3tPmwDhMPDmjHKnX+/Ac jEcRQMvaJqfGWe+TcBYf4ub2L94arUcTY+pVS5zXDtyEWkN6OJd74R0G/Y3Uohx+f9qd okuA== X-Gm-Message-State: ABy/qLZ/5PRGxaaxvjyVzc8tB5iVf6Ar63k1UifQBrrfVN9Ch3ZurG4v CcY1k01969Ujx5jTFlvGuHIvcJfq4Kw= X-Google-Smtp-Source: APBJJlG7qyKs7dIq2bWnrS/RJELFnYwUdBBi7VqUqUs0BYMwPmd19ZF5uNKykA3UY+W5oKnx/mhTmA8GenQ= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a17:903:2308:b0:1b8:80c9:a98e with SMTP id d8-20020a170903230800b001b880c9a98emr11974plh.13.1689970764858; Fri, 21 Jul 2023 13:19:24 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 21 Jul 2023 13:18:50 -0700 In-Reply-To: <20230721201859.2307736-1-seanjc@google.com> Mime-Version: 1.0 References: <20230721201859.2307736-1-seanjc@google.com> X-Mailer: git-send-email 2.41.0.487.g6d72f3e995-goog Message-ID: <20230721201859.2307736-11-seanjc@google.com> Subject: [PATCH v4 10/19] x86/virt: KVM: Move VMXOFF helpers into KVM VMX From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, Sean Christopherson , Paolo Bonzini Cc: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, Andrew Cooper , Kai Huang , Chao Gao Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Now that VMX is disabled in emergencies via the virt callbacks, move the VMXOFF helpers into KVM, the only remaining user. No functional change intended. Reviewed-by: Kai Huang Signed-off-by: Sean Christopherson --- arch/x86/include/asm/virtext.h | 42 ---------------------------------- arch/x86/kvm/vmx/vmx.c | 29 ++++++++++++++++++++--- 2 files changed, 26 insertions(+), 45 deletions(-) diff --git a/arch/x86/include/asm/virtext.h b/arch/x86/include/asm/virtext.h index b1171a5ad452..a27801f2bc71 100644 --- a/arch/x86/include/asm/virtext.h +++ b/arch/x86/include/asm/virtext.h @@ -19,48 +19,6 @@ #include #include =20 -/* - * VMX functions: - */ -/** - * cpu_vmxoff() - Disable VMX on the current CPU - * - * Disable VMX and clear CR4.VMXE (even if VMXOFF faults) - * - * Note, VMXOFF causes a #UD if the CPU is !post-VMXON, but it's impossibl= e to - * atomically track post-VMXON state, e.g. this may be called in NMI conte= xt. - * Eat all faults as all other faults on VMXOFF faults are mode related, i= .e. - * faults are guaranteed to be due to the !post-VMXON check unless the CPU= is - * magically in RM, VM86, compat mode, or at CPL>0. - */ -static inline int cpu_vmxoff(void) -{ - asm_volatile_goto("1: vmxoff\n\t" - _ASM_EXTABLE(1b, %l[fault]) - ::: "cc", "memory" : fault); - - cr4_clear_bits(X86_CR4_VMXE); - return 0; - -fault: - cr4_clear_bits(X86_CR4_VMXE); - return -EIO; -} - -static inline int cpu_vmx_enabled(void) -{ - return __read_cr4() & X86_CR4_VMXE; -} - -/** Disable VMX if it is enabled on the current CPU - */ -static inline void __cpu_emergency_vmxoff(void) -{ - if (cpu_vmx_enabled()) - cpu_vmxoff(); -} - - /* * SVM functions: */ diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 71571cd9adbb..6f4fcd82fa6e 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -47,7 +47,6 @@ #include #include #include -#include #include =20 #include "capabilities.h" @@ -744,6 +743,29 @@ static int vmx_set_guest_uret_msr(struct vcpu_vmx *vmx, return ret; } =20 +/* + * Disable VMX and clear CR4.VMXE (even if VMXOFF faults) + * + * Note, VMXOFF causes a #UD if the CPU is !post-VMXON, but it's impossibl= e to + * atomically track post-VMXON state, e.g. this may be called in NMI conte= xt. + * Eat all faults as all other faults on VMXOFF faults are mode related, i= .e. + * faults are guaranteed to be due to the !post-VMXON check unless the CPU= is + * magically in RM, VM86, compat mode, or at CPL>0. + */ +static int kvm_cpu_vmxoff(void) +{ + asm_volatile_goto("1: vmxoff\n\t" + _ASM_EXTABLE(1b, %l[fault]) + ::: "cc", "memory" : fault); + + cr4_clear_bits(X86_CR4_VMXE); + return 0; + +fault: + cr4_clear_bits(X86_CR4_VMXE); + return -EIO; +} + static void vmx_emergency_disable(void) { int cpu =3D raw_smp_processor_id(); @@ -753,7 +775,8 @@ static void vmx_emergency_disable(void) loaded_vmcss_on_cpu_link) vmcs_clear(v->vmcs); =20 - __cpu_emergency_vmxoff(); + if (__read_cr4() & X86_CR4_VMXE) + kvm_cpu_vmxoff(); } =20 static void __loaded_vmcs_clear(void *arg) @@ -2818,7 +2841,7 @@ static void vmx_hardware_disable(void) { vmclear_local_loaded_vmcss(); =20 - if (cpu_vmxoff()) + if (kvm_cpu_vmxoff()) kvm_spurious_fault(); =20 hv_reset_evmcs(); --=20 2.41.0.487.g6d72f3e995-goog From nobody Thu Sep 18 20:23:28 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E1F8AC41513 for ; Fri, 21 Jul 2023 20:20:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231451AbjGUUUH (ORCPT ); Fri, 21 Jul 2023 16:20:07 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38862 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231172AbjGUUTt (ORCPT ); Fri, 21 Jul 2023 16:19:49 -0400 Received: from mail-yw1-x1149.google.com (mail-yw1-x1149.google.com [IPv6:2607:f8b0:4864:20::1149]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0471A3AAA for ; Fri, 21 Jul 2023 13:19:27 -0700 (PDT) Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-5704e551e8bso23665447b3.3 for ; Fri, 21 Jul 2023 13:19:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1689970767; x=1690575567; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=KuoRE5Yl/DRtsjNa9mcAjfrWHh9Er79Gr1x6j80L0wM=; b=zD0iPAvsDw8xZvpkb6VCX2DZ6JQWGPjSkgFkC8hr4z3ClAZZgFh1N6K8U9uOWj82dp 7NBT8FLyUndSgt9nq0w0GtFwlmH6y3aYd4KDVxwEGAqJDg+lEF5owwSgo/isG26leJl2 WSW/txg1FS72tYowQfx/2adEJfka7gRf1lcfiA6tv8sRmumeZcalqKkBWisH0xC3+eSb yl+7xT9TZ7iW0pRwKkHScttEu3f7Wtg+yiiUHWnhPj9o9hWIg78vyIwhZ2EbR4Ua63cn KcHhoZxuRL/UQxqcQF8/gpLWTsmMelRFXVaQb1XBGI14PiIlz3uQu2uIh2s5G4c7ZoKx clBw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689970767; x=1690575567; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=KuoRE5Yl/DRtsjNa9mcAjfrWHh9Er79Gr1x6j80L0wM=; b=XrP8PrsPEylaFQiCHlq9suAjveTR3e3D/MsZt4xoXo9FVVB5KnrMBA3HebcDALAoKt XU6CULr5DFm4VxFG+eyud6FeKt5QbqHw64weIFyn+rWmwN9twzoF5C8G1ZMhbZXLviYZ JsmIAaIwtPYhrHpoB9k2bT9+ib2d38yGFDwvR5yhaw9sYTw6f7bkukSQ8LZcr/JNFEpt j0jFWwKWKcRnf9Yljba2/B8cOM/t/LLdDXCBkDssIS0xXj4yO3G4OuSqmDITFiiuhn/i 0s/mbYqxmybKmT5dSxugPCCi/G5zaRw5L9o+kyrS793lK5Ea7CcdP8BKmadPFCTnV4hp /dew== X-Gm-Message-State: ABy/qLZZcDDY3z+modO6+q4753gGb4roQ5+Fo1648BAK7AsxjhxkB6fb aS0o/JP9/KNhV2g7/S4QK4LLTc+2xtg= X-Google-Smtp-Source: APBJJlEhYy425WCnIIBImM5L/iSd03oI8qew0VWqrx4AwdpTdIPIg5t9V5WyZH2ZcsgmbT1N58lAS9Y5pS8= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a05:6902:102f:b0:c22:38c2:43de with SMTP id x15-20020a056902102f00b00c2238c243demr21396ybt.11.1689970766940; Fri, 21 Jul 2023 13:19:26 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 21 Jul 2023 13:18:51 -0700 In-Reply-To: <20230721201859.2307736-1-seanjc@google.com> Mime-Version: 1.0 References: <20230721201859.2307736-1-seanjc@google.com> X-Mailer: git-send-email 2.41.0.487.g6d72f3e995-goog Message-ID: <20230721201859.2307736-12-seanjc@google.com> Subject: [PATCH v4 11/19] KVM: SVM: Make KVM_AMD depend on CPU_SUP_AMD or CPU_SUP_HYGON From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, Sean Christopherson , Paolo Bonzini Cc: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, Andrew Cooper , Kai Huang , Chao Gao Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Make building KVM SVM support depend on support for AMD or Hygon. KVM already effectively restricts SVM support to AMD and Hygon by virtue of the vendor string checks in cpu_has_svm(), and KVM VMX supports depends on one of its three known vendors (Intel, Centaur, or Zhaoxin). Add the CPU_SUP_HYGON clause even though CPU_SUP_HYGON selects CPU_SUP_AMD to document that KVM SVM support isn't just for AMD CPUs, and to prevent breakage should Hygon support ever become a standalone thing. Signed-off-by: Sean Christopherson --- arch/x86/kvm/Kconfig | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/kvm/Kconfig b/arch/x86/kvm/Kconfig index 89ca7f4c1464..66dbd1f4d57d 100644 --- a/arch/x86/kvm/Kconfig +++ b/arch/x86/kvm/Kconfig @@ -101,7 +101,7 @@ config X86_SGX_KVM =20 config KVM_AMD tristate "KVM for AMD processors support" - depends on KVM + depends on KVM && (CPU_SUP_AMD || CPU_SUP_HYGON) help Provides support for KVM on AMD processors equipped with the AMD-V (SVM) extensions. --=20 2.41.0.487.g6d72f3e995-goog From nobody Thu Sep 18 20:23:28 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8911AC41513 for ; Fri, 21 Jul 2023 20:20:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231245AbjGUUUL (ORCPT ); Fri, 21 Jul 2023 16:20:11 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38920 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231235AbjGUUTx (ORCPT ); Fri, 21 Jul 2023 16:19:53 -0400 Received: from mail-yb1-xb4a.google.com (mail-yb1-xb4a.google.com [IPv6:2607:f8b0:4864:20::b4a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5B2223C0F for ; Fri, 21 Jul 2023 13:19:30 -0700 (PDT) Received: by mail-yb1-xb4a.google.com with SMTP id 3f1490d57ef6-cbcffb18afeso2290121276.2 for ; Fri, 21 Jul 2023 13:19:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1689970769; x=1690575569; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=crVRYHKBiix+9k25sqJYwYQ0PSqShcL9j1oDpcsA6Dw=; b=RBMLDqiHzxUqzpaXDgFLDPjpgSNfDoJR5XmTJs6L39FY207ornlD4wCggqqn+VPvnX obC6S0TQlHfM4bAlkMqGJoQfHMjSkF5rRIvGYLw0fKnXaYAG4jlkOuczjYzK20mkYAdj 2YUb2v/3KZQC34lVsFniXRfYO+rQiS7vKyeoW2e8jLhiydA8oIrl9Lk83BpOAAzGHwsS yQz2W/AOA95nN24PfMg8quSvL9++qNmgUOm8rU4FPZqMM/ZKEf4dQ+KjsJIEJm7VODr3 hN10+m9kznfjnNbZSm7xnVwZJq9pQnA6bgdEHbuu2VCvgb2oKlx6c+qOmdjYfoKCXoAa vzwQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689970769; x=1690575569; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=crVRYHKBiix+9k25sqJYwYQ0PSqShcL9j1oDpcsA6Dw=; b=CPKSZsjCld4Q0qE63aXH/2q91ntZg5q+Uj8EtdLI6rSWV7rKUx8R6zUCNIpKdCUu+Q shuCBuwvM8YNumZ0OMpPJI1oKQ+5WoUCNQzdea4cfssc0hofbA5Peb2VShm6bYcflRsS //7UvyDGbsOFaQZ7zT1+SMh1RwzjrvJKHiFGtmLxNYC/4BFxst4LBSy9v+c9N1Rr7uo9 nVWZ2w1rotvUtoQxQkR2YgGhQgKwPvwQduU4znBf6iWW1m8B/SwpLD3gFIzfGzj7W/mF JMf0ryDl0VgHWJZBSiDzoglLjfUURwU/9/n7zo0dAC0omuE3KbNkYklQI50Ml0B/GDFm NGFQ== X-Gm-Message-State: ABy/qLYQLkWN19viOfmaHB3I4+UgnPmmMNq4x7nkvgdcVsjvwXgNRGl2 YvAOEuYZE4/e3K53x9JRv2Yy/P2PDJY= X-Google-Smtp-Source: APBJJlE6O9VWoBAfPXKasyFdCqfa4t7wQIuYDKMRbknWhJoZeoUSwoDVta8TzLzKwebv41u+QzA8IEe4eCk= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a25:188b:0:b0:cab:e42c:876b with SMTP id 133-20020a25188b000000b00cabe42c876bmr18130yby.3.1689970768563; Fri, 21 Jul 2023 13:19:28 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 21 Jul 2023 13:18:52 -0700 In-Reply-To: <20230721201859.2307736-1-seanjc@google.com> Mime-Version: 1.0 References: <20230721201859.2307736-1-seanjc@google.com> X-Mailer: git-send-email 2.41.0.487.g6d72f3e995-goog Message-ID: <20230721201859.2307736-13-seanjc@google.com> Subject: [PATCH v4 12/19] x86/virt: Drop unnecessary check on extended CPUID level in cpu_has_svm() From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, Sean Christopherson , Paolo Bonzini Cc: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, Andrew Cooper , Kai Huang , Chao Gao Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Drop the explicit check on the extended CPUID level in cpu_has_svm(), the kernel's cached CPUID info will leave the entire SVM leaf unset if said leaf is not supported by hardware. Prior to using cached information, the check was needed to avoid false positives due to Intel's rather crazy CPUID behavior of returning the values of the maximum supported leaf if the specified leaf is unsupported. Fixes: 682a8108872f ("x86/kvm/svm: Simplify cpu_has_svm()") Signed-off-by: Sean Christopherson --- arch/x86/include/asm/virtext.h | 6 ------ 1 file changed, 6 deletions(-) diff --git a/arch/x86/include/asm/virtext.h b/arch/x86/include/asm/virtext.h index a27801f2bc71..be50c414efe4 100644 --- a/arch/x86/include/asm/virtext.h +++ b/arch/x86/include/asm/virtext.h @@ -39,12 +39,6 @@ static inline int cpu_has_svm(const char **msg) return 0; } =20 - if (boot_cpu_data.extended_cpuid_level < SVM_CPUID_FUNC) { - if (msg) - *msg =3D "can't execute cpuid_8000000a"; - return 0; - } - if (!boot_cpu_has(X86_FEATURE_SVM)) { if (msg) *msg =3D "svm not available"; --=20 2.41.0.487.g6d72f3e995-goog From nobody Thu Sep 18 20:23:28 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1AD09C0015E for ; Fri, 21 Jul 2023 20:20:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231518AbjGUUUj (ORCPT ); Fri, 21 Jul 2023 16:20:39 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37916 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231460AbjGUUUH (ORCPT ); Fri, 21 Jul 2023 16:20:07 -0400 Received: from mail-yw1-x1149.google.com (mail-yw1-x1149.google.com [IPv6:2607:f8b0:4864:20::1149]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F303935AB for ; Fri, 21 Jul 2023 13:19:31 -0700 (PDT) Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-583a89cccf6so6221507b3.1 for ; Fri, 21 Jul 2023 13:19:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1689970770; x=1690575570; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=mHJaGcmVZZrx5eJzID4aSbivNsbcbnHCRBUFTHr46UU=; b=xFjEUGb0YeU1GuqO6TMYTw4S8pE3wSKHfAnKkjt80FS5sXpKILObv/a3O3ewHWsGTO /uzvI4F2rqO1kbuzWSa4MliHY9cF1H/Ac9QzYVtv+NpjhbtyPfoNrV+E8v7WZNB/bFw6 ZQ7omF8dzvvNGZgQWyEPIyPC2w4JP9kZNJKLQfZrbxFODjZi5nLgku8ikRETSpBGVBoX oTGOuajDeuXcYaiGo4yFKLgo/FiyRHoPoBqYGCiHsyTHKlkUVEYlJTNm2xlmLZ2ST0En IL9sBALfri08BXauA1hERqbg4OfcffWRTJh4RjhdbbAEe8+cFgFJdJ8ifVoxgauGAZCM iuuA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689970770; x=1690575570; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=mHJaGcmVZZrx5eJzID4aSbivNsbcbnHCRBUFTHr46UU=; b=ED78eCYn1HC4WemdjwIID/SpOYjC/blCWapItl7GwlWNsPSc/PA5B7uy+Y9DlqevML Vu1zrE214JxqP0XuX74JzyJn8cKWuJY0UWCbUw72Y+/V01eUeJjFJsrsPJW3BmdlFMN9 fdULHlq/gS2sppiLmdbaLWotEpU5jkhiNPAdXmRVgcMC4uVOO+ZGnRvnoALvhiyXKiJo e8OoLNGy/cTq02lwZTroGI1+jMQErpy8OY5MvsxFlJS69EgfJ5m8XyucFT+gQllfnD1+ 2NaBVSNcD4eulUMxiGesAOfn7aCkKKvoHYN72hs/8fbjqXmoVEaS0ZUzuPyFLjhXSFGZ j+hw== X-Gm-Message-State: ABy/qLb/I5X3HLYwgLdok7rAspfbQfQKvf1/wnD58tHii4TlH6DExC5V FNTVhfE+h7hA2uFC3i0pt5L0qcjXxQ4= X-Google-Smtp-Source: APBJJlGF70SDbq3ZEJP5RArk6dyLbOvKEAdAx8uwY/2QO3SFNPIUp6BScgv9kfKmHPsrEaR+kwKTcLne1bo= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a81:ac15:0:b0:55d:d5b1:c2bd with SMTP id k21-20020a81ac15000000b0055dd5b1c2bdmr9808ywh.8.1689970770756; Fri, 21 Jul 2023 13:19:30 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 21 Jul 2023 13:18:53 -0700 In-Reply-To: <20230721201859.2307736-1-seanjc@google.com> Mime-Version: 1.0 References: <20230721201859.2307736-1-seanjc@google.com> X-Mailer: git-send-email 2.41.0.487.g6d72f3e995-goog Message-ID: <20230721201859.2307736-14-seanjc@google.com> Subject: [PATCH v4 13/19] x86/virt: KVM: Open code cpu_has_svm() into kvm_is_svm_supported() From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, Sean Christopherson , Paolo Bonzini Cc: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, Andrew Cooper , Kai Huang , Chao Gao Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Fold the guts of cpu_has_svm() into kvm_is_svm_supported(), its sole remaining user. No functional change intended. Reviewed-by: Kai Huang Signed-off-by: Sean Christopherson --- arch/x86/include/asm/virtext.h | 28 ---------------------------- arch/x86/kvm/svm/svm.c | 11 ++++++++--- 2 files changed, 8 insertions(+), 31 deletions(-) diff --git a/arch/x86/include/asm/virtext.h b/arch/x86/include/asm/virtext.h index be50c414efe4..632575e257d8 100644 --- a/arch/x86/include/asm/virtext.h +++ b/arch/x86/include/asm/virtext.h @@ -22,35 +22,7 @@ /* * SVM functions: */ - -/** Check if the CPU has SVM support - * - * You can use the 'msg' arg to get a message describing the problem, - * if the function returns zero. Simply pass NULL if you are not interested - * on the messages; gcc should take care of not generating code for - * the messages on this case. - */ -static inline int cpu_has_svm(const char **msg) -{ - if (boot_cpu_data.x86_vendor !=3D X86_VENDOR_AMD && - boot_cpu_data.x86_vendor !=3D X86_VENDOR_HYGON) { - if (msg) - *msg =3D "not amd or hygon"; - return 0; - } - - if (!boot_cpu_has(X86_FEATURE_SVM)) { - if (msg) - *msg =3D "svm not available"; - return 0; - } - return 1; -} - - /** Disable SVM on the current CPU - * - * You should call this only if cpu_has_svm() returned true. */ static inline void cpu_svm_disable(void) { diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index 1ae9c2c7eacb..ff6c769aafb2 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c @@ -521,11 +521,16 @@ static void svm_init_osvw(struct kvm_vcpu *vcpu) static bool kvm_is_svm_supported(void) { int cpu =3D raw_smp_processor_id(); - const char *msg; u64 vm_cr; =20 - if (!cpu_has_svm(&msg)) { - pr_err("SVM not supported by CPU %d, %s\n", cpu, msg); + if (boot_cpu_data.x86_vendor !=3D X86_VENDOR_AMD && + boot_cpu_data.x86_vendor !=3D X86_VENDOR_HYGON) { + pr_err("CPU %d isn't AMD or Hygon\n", cpu); + return false; + } + + if (!boot_cpu_has(X86_FEATURE_SVM)) { + pr_err("SVM not supported by CPU %d\n", cpu); return false; } =20 --=20 2.41.0.487.g6d72f3e995-goog From nobody Thu Sep 18 20:23:28 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 84744C0015E for ; Fri, 21 Jul 2023 20:20:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231336AbjGUUUe (ORCPT ); Fri, 21 Jul 2023 16:20:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39200 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231439AbjGUUUF (ORCPT ); Fri, 21 Jul 2023 16:20:05 -0400 Received: from mail-yb1-xb49.google.com (mail-yb1-xb49.google.com [IPv6:2607:f8b0:4864:20::b49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1F5664216 for ; Fri, 21 Jul 2023 13:19:33 -0700 (PDT) Received: by mail-yb1-xb49.google.com with SMTP id 3f1490d57ef6-c5a479bc2d4so2186020276.1 for ; Fri, 21 Jul 2023 13:19:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1689970772; x=1690575572; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=uhoBWFPCBOgYgQs9Wbxf7anLd8UMxgDqXbWpH+bX2KY=; b=BqJBwjEk+/zhFp4v5DrxQNReQVSoNlh/Bn0hUbsO8ZiEO/etbAIosjlClY7b6ifBBy wi3HDJfJE+jUUceINpJHN6wSTWbqUmyfzZi1k/JQKHiIJGwA7j6u5reIne4DWb5D1UeB p7pcAuNqF4vndeC9wIz/8uWHcddezeQhI8tep358HE0tpXUoAcue+Ghl6YEpRZEzbO/U J3+qH1Zc8yvKpVjIjnIx/75G7nRlmV863pOmIuyo9J3xyzA50nbAtEa0Myyb9v3ipxpH kDIyKNilgiU1a4xzVdhXo9++dIWuAlATjktwg3RONaxNWlAzAQafqJVzTHbU6vE4FJ1e d34g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689970772; x=1690575572; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=uhoBWFPCBOgYgQs9Wbxf7anLd8UMxgDqXbWpH+bX2KY=; b=ioPM0KVZdVzr750WNG+NuaMelMGwhJd6klo0H6CLm7AtcHPPs49o1WIXUuD5ax9uiH anZML967yc8GhY297JQpXwuEBVZZezLGyQOkEb8BaCBtG5cLF9CVw2OapMyXe7eOz5XL 6L1+HNeMYuhot2dt8Gp+Y7IDQZxMpSz75qmaGVC5tk4piVE9P/5eIgVDx3dBVmupoWsX qsZqHS67nExZfNZXEwRZJcCtq1iBJTFK4kspH2F6Mo+VBggVvwgM9agWAQeW1oz+62KU DJLBmryrZsjxQ69N9SEYuslGFR8A+q4Kagvcl3vUvMFuwGx5pqoZAUTnt8OqpyFJhQSB Z1Bw== X-Gm-Message-State: ABy/qLbSm38lnt9zS73T52AvEvPT8+sa/m2sKB83tjTuJtuxTxXvKLDZ RkyYRe3ENsbFRo+bQT+RVSMKmoXjiLg= X-Google-Smtp-Source: APBJJlG9ebou3TaLXiHu0JkY9v2NmMh1lndfe7foTeWkhO+fnKnQ3AxYQbow6FKkByZbw6dWuECajNEM6v4= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a25:40cf:0:b0:cc7:b850:7f2 with SMTP id n198-20020a2540cf000000b00cc7b85007f2mr19598yba.5.1689970772756; Fri, 21 Jul 2023 13:19:32 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 21 Jul 2023 13:18:54 -0700 In-Reply-To: <20230721201859.2307736-1-seanjc@google.com> Mime-Version: 1.0 References: <20230721201859.2307736-1-seanjc@google.com> X-Mailer: git-send-email 2.41.0.487.g6d72f3e995-goog Message-ID: <20230721201859.2307736-15-seanjc@google.com> Subject: [PATCH v4 14/19] KVM: SVM: Check that the current CPU supports SVM in kvm_is_svm_supported() From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, Sean Christopherson , Paolo Bonzini Cc: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, Andrew Cooper , Kai Huang , Chao Gao Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Check "this" CPU instead of the boot CPU when querying SVM support so that the per-CPU checks done during hardware enabling actually function as intended, i.e. will detect issues where SVM isn't support on all CPUs. Disable migration for the use from svm_init() mostly so that the standard accessors for the per-CPU data can be used without getting yelled at by CONFIG_DEBUG_PREEMPT=3Dy sanity checks. Preventing the "disabled by BIOS" error message from reporting the wrong CPU is largely a bonus, as ensuring a stable CPU during module load is a non-goal for KVM. Link: https://lore.kernel.org/all/ZAdxNgv0M6P63odE@google.com Cc: Kai Huang Cc: Chao Gao Reviewed-by: Kai Huang Signed-off-by: Sean Christopherson --- arch/x86/kvm/svm/svm.c | 25 +++++++++++++++++++------ 1 file changed, 19 insertions(+), 6 deletions(-) diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index ff6c769aafb2..9e449167e71b 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c @@ -518,18 +518,20 @@ static void svm_init_osvw(struct kvm_vcpu *vcpu) vcpu->arch.osvw.status |=3D 1; } =20 -static bool kvm_is_svm_supported(void) +static bool __kvm_is_svm_supported(void) { - int cpu =3D raw_smp_processor_id(); + int cpu =3D smp_processor_id(); + struct cpuinfo_x86 *c =3D &cpu_data(cpu); + u64 vm_cr; =20 - if (boot_cpu_data.x86_vendor !=3D X86_VENDOR_AMD && - boot_cpu_data.x86_vendor !=3D X86_VENDOR_HYGON) { + if (c->x86_vendor !=3D X86_VENDOR_AMD && + c->x86_vendor !=3D X86_VENDOR_HYGON) { pr_err("CPU %d isn't AMD or Hygon\n", cpu); return false; } =20 - if (!boot_cpu_has(X86_FEATURE_SVM)) { + if (!cpu_has(c, X86_FEATURE_SVM)) { pr_err("SVM not supported by CPU %d\n", cpu); return false; } @@ -548,9 +550,20 @@ static bool kvm_is_svm_supported(void) return true; } =20 +static bool kvm_is_svm_supported(void) +{ + bool supported; + + migrate_disable(); + supported =3D __kvm_is_svm_supported(); + migrate_enable(); + + return supported; +} + static int svm_check_processor_compat(void) { - if (!kvm_is_svm_supported()) + if (!__kvm_is_svm_supported()) return -EIO; =20 return 0; --=20 2.41.0.487.g6d72f3e995-goog From nobody Thu Sep 18 20:23:28 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AD61BC001DE for ; Fri, 21 Jul 2023 20:20:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231325AbjGUUUa (ORCPT ); Fri, 21 Jul 2023 16:20:30 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39180 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230460AbjGUUUE (ORCPT ); Fri, 21 Jul 2023 16:20:04 -0400 Received: from mail-yw1-x1149.google.com (mail-yw1-x1149.google.com [IPv6:2607:f8b0:4864:20::1149]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2B680422D for ; Fri, 21 Jul 2023 13:19:35 -0700 (PDT) Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-5734d919156so26637607b3.3 for ; Fri, 21 Jul 2023 13:19:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1689970774; x=1690575574; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=wPlbfa+N8zgfbkJ5TFjNvfC430Np593GRATA+kV4uv8=; b=YMYRUHzUDCVMuYn52ynlOKvBnRXOyiLr1dfgtoseNBOu4bDGsTucDMvLpEpL1MqmUe 4GSO5jlD2I9GxnBXQQRXoK5LwMCPEGlozlc4vlePKaqf3bhgsDxPjj+cBfWd7xupVa2/ i0NHbQtd24teDqosxIhfXN30cvUCTA5BSKtrqljErAnCSidWLnVSreSjcp9t3cgs8/8f j1Q2DQWnzLCp1MpXEbmnFdAh1ipvd0bngkKlEdgjYWvm70IfbIeCjdTuzIFN9rQlcRMX LZJ/W3WzKqojxqlMu8WlcWnJ0ILoNtfwLgJUzJKLYFQ7bJ52u6OwVjmUKa7/uN93JeuR y32A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689970774; x=1690575574; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=wPlbfa+N8zgfbkJ5TFjNvfC430Np593GRATA+kV4uv8=; b=eQYFBcWsRet42VEw/ul8L+f8mxIANPU9eEG3NTrERtSixg4Ztg9gjrNVoegI1ibwuB TxQl/TbXi45cVRK95qEnwhgGcBcCPFI3x2fIMaimQbB95QGTk6PifbGiV4/E7TLmxDDw KeUzR1A2ewJQzBGXnFG6Y9uVrgkPFdThwdu0k9l4IRIV5Q9tLVWYYmsMU6D8byzI9uCx TzXOVblKklRAwq08T9wopTY/6z8cVhSFBx24jyPpYIrNWZGrb2gYzR83ZYjCs/y+PTis jvKmMXCL+6KkUXwNmG4KDHCvaD3uQZYwEkTx8FVpmiR4fqJ4BGVeuPIm9BrIu7v86ufa yd6g== X-Gm-Message-State: ABy/qLZx6X6EORU5PV4tOwdeM2O93jEMdUPTylL0PiJ8jrFS3cO3UVdG zAXpEG4SM1fX2HO30BpDYrf48ESvIyU= X-Google-Smtp-Source: APBJJlFEHHeJp5RT7zszaIw4ZA9cDrjfXBLHHV5QoHMtGzvyQ4EWMvYocU2KpSc+oerwCYSdXu+UWVkF0+Q= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a81:ad55:0:b0:583:6f04:4267 with SMTP id l21-20020a81ad55000000b005836f044267mr10407ywk.8.1689970774670; Fri, 21 Jul 2023 13:19:34 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 21 Jul 2023 13:18:55 -0700 In-Reply-To: <20230721201859.2307736-1-seanjc@google.com> Mime-Version: 1.0 References: <20230721201859.2307736-1-seanjc@google.com> X-Mailer: git-send-email 2.41.0.487.g6d72f3e995-goog Message-ID: <20230721201859.2307736-16-seanjc@google.com> Subject: [PATCH v4 15/19] KVM: VMX: Ensure CPU is stable when probing basic VMX support From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, Sean Christopherson , Paolo Bonzini Cc: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, Andrew Cooper , Kai Huang , Chao Gao Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Disable migration when probing VMX support during module load to ensure the CPU is stable, mostly to match similar SVM logic, where allowing migration effective requires deliberately writing buggy code. As a bonus, KVM won't report the wrong CPU to userspace if VMX is unsupported, but in practice that is a very, very minor bonus as the only way that reporting the wrong CPU would actually matter is if hardware is broken or if the system is misconfigured, i.e. if KVM gets migrated from a CPU that _does_ support VMX to a CPU that does _not_ support VMX. Reviewed-by: Kai Huang Signed-off-by: Sean Christopherson --- arch/x86/kvm/vmx/vmx.c | 17 ++++++++++++++--- 1 file changed, 14 insertions(+), 3 deletions(-) diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 6f4fcd82fa6e..0e1f3856a9be 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -2737,9 +2737,9 @@ static int setup_vmcs_config(struct vmcs_config *vmcs= _conf, return 0; } =20 -static bool kvm_is_vmx_supported(void) +static bool __kvm_is_vmx_supported(void) { - int cpu =3D raw_smp_processor_id(); + int cpu =3D smp_processor_id(); =20 if (!(cpuid_ecx(1) & feature_bit(VMX))) { pr_err("VMX not supported by CPU %d\n", cpu); @@ -2755,13 +2755,24 @@ static bool kvm_is_vmx_supported(void) return true; } =20 +static bool kvm_is_vmx_supported(void) +{ + bool supported; + + migrate_disable(); + supported =3D __kvm_is_vmx_supported(); + migrate_enable(); + + return supported; +} + static int vmx_check_processor_compat(void) { int cpu =3D raw_smp_processor_id(); struct vmcs_config vmcs_conf; struct vmx_capability vmx_cap; =20 - if (!kvm_is_vmx_supported()) + if (!__kvm_is_vmx_supported()) return -EIO; =20 if (setup_vmcs_config(&vmcs_conf, &vmx_cap) < 0) { --=20 2.41.0.487.g6d72f3e995-goog From nobody Thu Sep 18 20:23:28 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3E4DBC001DE for ; Fri, 21 Jul 2023 20:20:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231530AbjGUUUx (ORCPT ); Fri, 21 Jul 2023 16:20:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38780 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231311AbjGUUUT (ORCPT ); Fri, 21 Jul 2023 16:20:19 -0400 Received: from mail-pl1-x64a.google.com (mail-pl1-x64a.google.com [IPv6:2607:f8b0:4864:20::64a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 85CB94493 for ; Fri, 21 Jul 2023 13:19:40 -0700 (PDT) Received: by mail-pl1-x64a.google.com with SMTP id d9443c01a7336-1b8af49a5d2so20437515ad.2 for ; Fri, 21 Jul 2023 13:19:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1689970777; x=1690575577; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=7MVEPPMPG5BIE/ZORskfpxdkerEVsprNO1/ccabxK1U=; b=dsfCRhtXu+8WBW50E8oV+3uhZ2IRmNPTr+fYzCXaE7ye0usyPQ3FcgUx3z2bDgBp+b 2JtfRRWLip6QDC4cSBq7Ef5ocutDHXkXkY6mDSEOjLvzRlaA2XjN+g38YAUYj932UeSm FH2O1fYjeHuQfQFMZ+X68cDDdQ0UDLkoOdRf/CzfI13WPYzklLaH5a295isOpx4XsjFI ACUsOr8ONPZJWnXokIUrjZxk++/XpQnD0doONOc5k8lzC3jKOYe3fnHmlqne9aCmE0mt ZCSzWJ2xDQKusbcs0tM4RFyc9d8iON18lHvW/YwToWhPC8ogAzcrqzslH4pNfPJ19Nnf /vJA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689970777; x=1690575577; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=7MVEPPMPG5BIE/ZORskfpxdkerEVsprNO1/ccabxK1U=; b=Fg4WQhq4KLb9UMrqO+LDnfO26vwZM/barmO0XamyuLrWbbk2j+H7PNLlHEyt+iy4ny p6s6ewQd8cLzy+9SbZimIMBQDsFGu/sgJEoObSkRHjF0pJG+Nfva+VfPfgAUnZr67pwh MKQeo86zbaEG1e8VG/rLGywS8pSR+7w260gd18HJJbaUqW65WsBe2T8c0eU88NPSAXZb RtKwPUeGXhI+CSfuOXeLX9FnHmnFwcDghcEhFFJy8M2oEkP5Idp6n4tfg8aupkOgm1fX lXcDBdVS+YgUXs3fqny3ABQoIsnSbnPVdQO+y1PQGTL0OYVBZJWen81em7buVbdaEPVc 0j4A== X-Gm-Message-State: ABy/qLbKrhlOSlXQMku8AZe2ht7SYlDvSTU81ljvu56lJtx9hwiPy85g VoTsdejxKlRjjy9sqfiNkYhIKnzIjDs= X-Google-Smtp-Source: APBJJlH7v1jWQJgdvmTHmNodVC5MgWoiQ35LLIzu27y8MUq20QZMDNCkMIZSTIPdq94UkFGXcfYhULYma4g= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a17:903:22c9:b0:1b8:7f21:6d3 with SMTP id y9-20020a17090322c900b001b87f2106d3mr12669plg.6.1689970776409; Fri, 21 Jul 2023 13:19:36 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 21 Jul 2023 13:18:56 -0700 In-Reply-To: <20230721201859.2307736-1-seanjc@google.com> Mime-Version: 1.0 References: <20230721201859.2307736-1-seanjc@google.com> X-Mailer: git-send-email 2.41.0.487.g6d72f3e995-goog Message-ID: <20230721201859.2307736-17-seanjc@google.com> Subject: [PATCH v4 16/19] x86/virt: KVM: Move "disable SVM" helper into KVM SVM From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, Sean Christopherson , Paolo Bonzini Cc: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, Andrew Cooper , Kai Huang , Chao Gao Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Move cpu_svm_disable() into KVM proper now that all hardware virtualization management is routed through KVM. Remove the now-empty virtext.h. No functional change intended. Reviewed-by: Kai Huang Signed-off-by: Sean Christopherson --- arch/x86/include/asm/virtext.h | 50 ---------------------------------- arch/x86/kvm/svm/svm.c | 29 +++++++++++++++++--- 2 files changed, 25 insertions(+), 54 deletions(-) delete mode 100644 arch/x86/include/asm/virtext.h diff --git a/arch/x86/include/asm/virtext.h b/arch/x86/include/asm/virtext.h deleted file mode 100644 index 632575e257d8..000000000000 --- a/arch/x86/include/asm/virtext.h +++ /dev/null @@ -1,50 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0-only */ -/* CPU virtualization extensions handling - * - * This should carry the code for handling CPU virtualization extensions - * that needs to live in the kernel core. - * - * Author: Eduardo Habkost - * - * Copyright (C) 2008, Red Hat Inc. - * - * Contains code from KVM, Copyright (C) 2006 Qumranet, Inc. - */ -#ifndef _ASM_X86_VIRTEX_H -#define _ASM_X86_VIRTEX_H - -#include - -#include -#include -#include - -/* - * SVM functions: - */ -/** Disable SVM on the current CPU - */ -static inline void cpu_svm_disable(void) -{ - uint64_t efer; - - wrmsrl(MSR_VM_HSAVE_PA, 0); - rdmsrl(MSR_EFER, efer); - if (efer & EFER_SVME) { - /* - * Force GIF=3D1 prior to disabling SVM to ensure INIT and NMI - * aren't blocked, e.g. if a fatal error occurred between CLGI - * and STGI. Note, STGI may #UD if SVM is disabled from NMI - * context between reading EFER and executing STGI. In that - * case, GIF must already be set, otherwise the NMI would have - * been blocked, so just eat the fault. - */ - asm_volatile_goto("1: stgi\n\t" - _ASM_EXTABLE(1b, %l[fault]) - ::: "memory" : fault); -fault: - wrmsrl(MSR_EFER, efer & ~EFER_SVME); - } -} - -#endif /* _ASM_X86_VIRTEX_H */ diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index 9e449167e71b..47f9c7156609 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c @@ -42,8 +42,6 @@ #include #include =20 -#include - #include =20 #include "trace.h" @@ -582,9 +580,32 @@ void __svm_write_tsc_multiplier(u64 multiplier) preempt_enable(); } =20 +static inline void kvm_cpu_svm_disable(void) +{ + uint64_t efer; + + wrmsrl(MSR_VM_HSAVE_PA, 0); + rdmsrl(MSR_EFER, efer); + if (efer & EFER_SVME) { + /* + * Force GIF=3D1 prior to disabling SVM to ensure INIT and NMI + * aren't blocked, e.g. if a fatal error occurred between CLGI + * and STGI. Note, STGI may #UD if SVM is disabled from NMI + * context between reading EFER and executing STGI. In that + * case, GIF must already be set, otherwise the NMI would have + * been blocked, so just eat the fault. + */ + asm_volatile_goto("1: stgi\n\t" + _ASM_EXTABLE(1b, %l[fault]) + ::: "memory" : fault); +fault: + wrmsrl(MSR_EFER, efer & ~EFER_SVME); + } +} + static void svm_emergency_disable(void) { - cpu_svm_disable(); + kvm_cpu_svm_disable(); } =20 static void svm_hardware_disable(void) @@ -593,7 +614,7 @@ static void svm_hardware_disable(void) if (tsc_scaling) __svm_write_tsc_multiplier(SVM_TSC_RATIO_DEFAULT); =20 - cpu_svm_disable(); + kvm_cpu_svm_disable(); =20 amd_pmu_disable_virt(); } --=20 2.41.0.487.g6d72f3e995-goog From nobody Thu Sep 18 20:23:28 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 03243C41513 for ; Fri, 21 Jul 2023 20:21:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231458AbjGUUVC (ORCPT ); Fri, 21 Jul 2023 16:21:02 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38926 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231503AbjGUUUf (ORCPT ); Fri, 21 Jul 2023 16:20:35 -0400 Received: from mail-yb1-xb4a.google.com (mail-yb1-xb4a.google.com [IPv6:2607:f8b0:4864:20::b4a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 488923A8C for ; Fri, 21 Jul 2023 13:19:50 -0700 (PDT) Received: by mail-yb1-xb4a.google.com with SMTP id 3f1490d57ef6-c8f360a07a2so2266280276.2 for ; Fri, 21 Jul 2023 13:19:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1689970779; x=1690575579; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=0ufOgL/ovXKIGQEYukNshYDCW2QOagoVOtmN2psRo9c=; b=b12Ttu/+eAKMYYr+ZtWZDhO9f8tIZzhiWDUyfkYKkQAGZ8PdoM7SUfhXHfFjda9SM8 wHTbZcaIxbvpo6ZC55UJc/d6aRulVjhV0CUNJrul/2S2FZZweyArAU2EVJUBrcBBXquN vU05kJwjb1NAJHt3dCz+9FG0HhYvLHlgxT/IyC/seKzCxR/UFSkK+WuX3JN2T1Zjexz/ cn2JU6UHxz3FopptseYCiqsx2AYAfIKAmTFUMY10/d1txCxH5yYIetLq3/s5ohB7VhvC xsH/zGKExBxyiHcmYi130rGplxqvQDFOSlqz2e7jwYp5AIvNTdYLOF8Ndw+tBLCt7PL8 HW5A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689970779; x=1690575579; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=0ufOgL/ovXKIGQEYukNshYDCW2QOagoVOtmN2psRo9c=; b=jqmezKvGRiaAfz+LvDoeIAf4E9uHBtMnT/dIotQcBOBA7EZvI0Uuo+T3fdZ9eVApkF 3q60f9tKOK6MbhTC4nwmFFEPx9d1zSAuiYOBEcUTB1fndOLUYluLifeCRqFuVjbIhkyD JL7nnr4eYtBHkcADK1r9Kay4lN0h3IZ14KFjmVJStCbpZ6W6AtQANHn7ONTK8/kV9A26 zWaDCvxKRfidb7CT8LgjTN8FludMiIY4RMMWa14TORgzruUEeCEKbiJPaT1JlX0NDZRj rTW1QOWulVqqd+zj+LTBwBLLg172g5jqHoTisbLLfqD1uLlrIMYt1n533Qr3MRMzGOwK +/Rw== X-Gm-Message-State: ABy/qLZZpcZCKK5bf+nqujVJkS03UwAsReJl/64z+lwMqBtRnlKeY+S/ 0fCjKpYfJNaWaFnNSLhgJ7ICpU+yDeM= X-Google-Smtp-Source: APBJJlEMRg8khkFQs+v6K4Haj1LsdDQNVMJ0EH61NH5YR90us2W3yxUe/jzK4/gqbgOaQJvbK9pl9eS5wa8= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a25:81cf:0:b0:c5d:dac2:cb2a with SMTP id n15-20020a2581cf000000b00c5ddac2cb2amr18048ybm.13.1689970779472; Fri, 21 Jul 2023 13:19:39 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 21 Jul 2023 13:18:57 -0700 In-Reply-To: <20230721201859.2307736-1-seanjc@google.com> Mime-Version: 1.0 References: <20230721201859.2307736-1-seanjc@google.com> X-Mailer: git-send-email 2.41.0.487.g6d72f3e995-goog Message-ID: <20230721201859.2307736-18-seanjc@google.com> Subject: [PATCH v4 17/19] KVM: x86: Force kvm_rebooting=true during emergency reboot/crash From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, Sean Christopherson , Paolo Bonzini Cc: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, Andrew Cooper , Kai Huang , Chao Gao Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Set kvm_rebooting when virtualization is disabled in an emergency so that KVM eats faults on virtualization instructions even if kvm_reboot() isn't reached. Reviewed-by: Kai Huang Signed-off-by: Sean Christopherson --- arch/x86/kvm/svm/svm.c | 2 ++ arch/x86/kvm/vmx/vmx.c | 2 ++ 2 files changed, 4 insertions(+) diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index 47f9c7156609..8d1b3c801629 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c @@ -605,6 +605,8 @@ static inline void kvm_cpu_svm_disable(void) =20 static void svm_emergency_disable(void) { + kvm_rebooting =3D true; + kvm_cpu_svm_disable(); } =20 diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 0e1f3856a9be..5d21931842a5 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -771,6 +771,8 @@ static void vmx_emergency_disable(void) int cpu =3D raw_smp_processor_id(); struct loaded_vmcs *v; =20 + kvm_rebooting =3D true; + list_for_each_entry(v, &per_cpu(loaded_vmcss_on_cpu, cpu), loaded_vmcss_on_cpu_link) vmcs_clear(v->vmcs); --=20 2.41.0.487.g6d72f3e995-goog From nobody Thu Sep 18 20:23:28 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 626F5C0015E for ; Fri, 21 Jul 2023 20:21:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231466AbjGUUVG (ORCPT ); Fri, 21 Jul 2023 16:21:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38750 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231229AbjGUUUq (ORCPT ); Fri, 21 Jul 2023 16:20:46 -0400 Received: from mail-yw1-x1149.google.com (mail-yw1-x1149.google.com [IPv6:2607:f8b0:4864:20::1149]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BD780359F for ; Fri, 21 Jul 2023 13:19:55 -0700 (PDT) Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-5839bad0ba7so11013217b3.0 for ; Fri, 21 Jul 2023 13:19:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1689970781; x=1690575581; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=vAOLLgTu+0QNT9Om4NpxWB91aAPEeSvRzwHwJVAa2Xs=; b=imhgBKzxrXtouuf5PfcKL7zBn0uz33+q5k8Z0nIhm6BNxLjvMukg2vArkkk5NgTFDM NCaOxaOau0vLfoyQRsInuw+KAa7AslkSTBn6yul+5eiPK0IjOxtTfZI+5BUlLIVKvaaf zctolS0OU9qe/eA0SfoB4J52oxFAqZ+fh+FLtJaIaIPFFG750jsFlZ2Mc9quoQp2Unyu 4inyMQQxkm2n98I2kfK3YCgsPjNs4UlB9VWVghYk87tTe4jdkygy4sS2+Mqd1axuaOKZ Y80wBoNxrdfW0VnTAvlYpjVOt8tpfiwmBahTWl8/tfevz/dYYviEbtSr/R7buC1VS27v nuqg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689970781; x=1690575581; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=vAOLLgTu+0QNT9Om4NpxWB91aAPEeSvRzwHwJVAa2Xs=; b=DauQHLsbRiqO7ytJPcQtbJG8RaVZN73hfCe6Pj8jaohdtAVgsS+ma1FCpK8KAndWSc SWa/CHpGmWOGyfunjcGLHC6fmGW3OPnFoeLI4f7NqAnBH02ZtqPYrendrrexfgZ7MVu3 WHdkYMtcJGpCGTXkgwE3O9lBx1dqfGQt/q6zve5RH65R++1TzH98wUdE1RTA6W08YPMF muVhRUgo1htmQrvk4bqKC30B+QbfV+PLEeW+zuFdwx5dTJznIo/vCiTHwlohPTXTFos3 S6G648iV1bxBC2CFVsXDgmbcX31yWU+mHYfIMCPx1C6PZ6VcUvTjR1z0wzJ+J0fnHADq wNeg== X-Gm-Message-State: ABy/qLZdBhwI1JNWVciQn8xZBklDUQaLNN56i7+/GmVKJ6ZRGynPhqZB MgbrefUPfvc6iBInBhJDz4ifXcvDx1o= X-Google-Smtp-Source: APBJJlFg1G379oAP55UGHQ0zEE/xeGlSUtowvWP1iIaOgGSS++UyKOVvNLNpz7tVyRlxXJ14xmaMnRFherI= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a25:81cf:0:b0:c5d:dac2:cb2a with SMTP id n15-20020a2581cf000000b00c5ddac2cb2amr18050ybm.13.1689970781205; Fri, 21 Jul 2023 13:19:41 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 21 Jul 2023 13:18:58 -0700 In-Reply-To: <20230721201859.2307736-1-seanjc@google.com> Mime-Version: 1.0 References: <20230721201859.2307736-1-seanjc@google.com> X-Mailer: git-send-email 2.41.0.487.g6d72f3e995-goog Message-ID: <20230721201859.2307736-19-seanjc@google.com> Subject: [PATCH v4 18/19] KVM: SVM: Use "standard" stgi() helper when disabling SVM From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, Sean Christopherson , Paolo Bonzini Cc: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, Andrew Cooper , Kai Huang , Chao Gao Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Now that kvm_rebooting is guaranteed to be true prior to disabling SVM in an emergency, use the existing stgi() helper instead of open coding STGI. In effect, eat faults on STGI if and only if kvm_rebooting=3D=3Dtrue. Signed-off-by: Sean Christopherson --- arch/x86/kvm/svm/svm.c | 13 +++---------- 1 file changed, 3 insertions(+), 10 deletions(-) diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index 8d1b3c801629..4785d780cce3 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c @@ -588,17 +588,10 @@ static inline void kvm_cpu_svm_disable(void) rdmsrl(MSR_EFER, efer); if (efer & EFER_SVME) { /* - * Force GIF=3D1 prior to disabling SVM to ensure INIT and NMI - * aren't blocked, e.g. if a fatal error occurred between CLGI - * and STGI. Note, STGI may #UD if SVM is disabled from NMI - * context between reading EFER and executing STGI. In that - * case, GIF must already be set, otherwise the NMI would have - * been blocked, so just eat the fault. + * Force GIF=3D1 prior to disabling SVM, e.g. to ensure INIT and + * NMI aren't blocked. */ - asm_volatile_goto("1: stgi\n\t" - _ASM_EXTABLE(1b, %l[fault]) - ::: "memory" : fault); -fault: + stgi(); wrmsrl(MSR_EFER, efer & ~EFER_SVME); } } --=20 2.41.0.487.g6d72f3e995-goog From nobody Thu Sep 18 20:23:28 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BB00EEB64DD for ; Fri, 21 Jul 2023 20:21:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231598AbjGUUVV (ORCPT ); Fri, 21 Jul 2023 16:21:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38862 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231311AbjGUUUy (ORCPT ); Fri, 21 Jul 2023 16:20:54 -0400 Received: from mail-yb1-xb49.google.com (mail-yb1-xb49.google.com [IPv6:2607:f8b0:4864:20::b49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F04883AA2 for ; Fri, 21 Jul 2023 13:20:02 -0700 (PDT) Received: by mail-yb1-xb49.google.com with SMTP id 3f1490d57ef6-c8f360a07a2so2266429276.2 for ; Fri, 21 Jul 2023 13:20:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1689970783; x=1690575583; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=f1LNi9Sq5NIBXtzGJVGKttDHRo4VG7MGlit5nvyUtF0=; b=lupqznqfPScgKgAlsl68k2HwQOWYVzqFz7+e39gz+UVj6wvvC+ma4BNLkDyLgMbDCr sXcaIXufKIRqTjEo6kaASseH4LuovJPFo1FpnMySbHlymLZC8toxxIjvN2Aiw4ju6EAG VaD/mWdhEO8ur/DDeO9tEnMiMm8VItIJH8OP+EP+LpWzQ/NrR5L8c8s2LMhTcLypPS2U oBg+8IvkdXfhKuyTvHG3Xef0QbwUQ6ZzSvJM7Mo/H4jFrr/vFd5FgUJyl+qeE3dTtsH5 MY6DHfpkgF/82Qh+aTdBRtINfnVpzFGxerBDIxZVXvcHDxqf5FZ0G6Srcd0y+VCH+a4h +9CA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689970783; x=1690575583; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=f1LNi9Sq5NIBXtzGJVGKttDHRo4VG7MGlit5nvyUtF0=; b=UE9jJl4FmdRBt8lVAb8J0GqHbC9FtiPMgkofkW2dh9ugrklT4sy1td/iXtfna/RJT6 GjIIlvu9TU/QzVZjrCtQV0WWwWsJoW7OjVbg64Uo7lNcafxofTJSaV0iJOkiWCsKXyhA ADcT2SYT6+W5mMeExzrQaeFzsPU9aB5cBZNbUReCOuQy7hk78nQuLe5Bh1zm/R6nZse5 9uZKmtqkXjtDI4Su8ShA6TvfBm3cUjhllE5bFq4ksGoXWvC+/sucbPauPpYaO6erEq/s ladM1OwD4Q1SJck+Pc8pU8p6WCQPHvnF/rY4oqd15+oCeE/vtHSqruVikiHbLxfUzjLI BR2w== X-Gm-Message-State: ABy/qLYiejwXCdiKng84QDxgOLtaZUQFWEEVF3pGVngPUNnd8ijDShwK f4PlfqIVhNQARc8ZvT4EYh4LpbWwP4k= X-Google-Smtp-Source: APBJJlGqCihblJf1jaycZ9unQQ6pi+gVUILg04BI7aSdWb1kqkXeAsB0DrslgKJ6+cdV8k8Utd5kIZ2tJk4= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a5b:bc2:0:b0:ce9:64b3:80dc with SMTP id c2-20020a5b0bc2000000b00ce964b380dcmr18891ybr.1.1689970783817; Fri, 21 Jul 2023 13:19:43 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 21 Jul 2023 13:18:59 -0700 In-Reply-To: <20230721201859.2307736-1-seanjc@google.com> Mime-Version: 1.0 References: <20230721201859.2307736-1-seanjc@google.com> X-Mailer: git-send-email 2.41.0.487.g6d72f3e995-goog Message-ID: <20230721201859.2307736-20-seanjc@google.com> Subject: [PATCH v4 19/19] KVM: VMX: Skip VMCLEAR logic during emergency reboots if CR4.VMXE=0 From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, Sean Christopherson , Paolo Bonzini Cc: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, Andrew Cooper , Kai Huang , Chao Gao Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Bail from vmx_emergency_disable() without processing the list of loaded VMCSes if CR4.VMXE=3D0, i.e. if the CPU can't be post-VMXON. It should be impossible for the list to have entries if VMX is already disabled, and even if that invariant doesn't hold, VMCLEAR will #UD anyways, i.e. processing the list is pointless even if it somehow isn't empty. Assuming no existing KVM bugs, this should be a glorified nop. The primary motivation for the change is to avoid having code that looks like it does VMCLEAR, but then skips VMXON, which is nonsensical. Suggested-by: Kai Huang Signed-off-by: Sean Christopherson Reviewed-by: Kai Huang --- arch/x86/kvm/vmx/vmx.c | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 5d21931842a5..0ef5ede9cb7c 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -773,12 +773,20 @@ static void vmx_emergency_disable(void) =20 kvm_rebooting =3D true; =20 + /* + * Note, CR4.VMXE can be _cleared_ in NMI context, but it can only be + * set in task context. If this races with VMX is disabled by an NMI, + * VMCLEAR and VMXOFF may #UD, but KVM will eat those faults due to + * kvm_rebooting set. + */ + if (!(__read_cr4() & X86_CR4_VMXE)) + return; + list_for_each_entry(v, &per_cpu(loaded_vmcss_on_cpu, cpu), loaded_vmcss_on_cpu_link) vmcs_clear(v->vmcs); =20 - if (__read_cr4() & X86_CR4_VMXE) - kvm_cpu_vmxoff(); + kvm_cpu_vmxoff(); } =20 static void __loaded_vmcs_clear(void *arg) --=20 2.41.0.487.g6d72f3e995-goog