From nobody Mon Jun 15 07:34:28 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 033B8B67E for ; Wed, 8 Apr 2026 18:22:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775672541; cv=none; b=fO0br7jMuhW1mOCUQd66p73e7AvsJedCa8f1m1Kq2+AhnYoXYiUH9gNfSCf24kSXrUsyV7nVt8SanTnp+YzrtiQHRyNcYgIua+bbT5Z1TuyroZt91FJWcEChgEvWsZOEzCOcjK+pp8zXsgcO+bz3E4kKjujiGYIICOpiZYQM5Do= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775672541; c=relaxed/simple; bh=+B6uJp30XVb4QtEZB1/s2I2efJW9ubVPLc9UdlwT1eY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=HVs+K4fbHhVoFZPEmapAOPmcozPvi1Kl3zp/aK1BJMHr9u1Ejd02XYn4HjpnISY0WOD9s3ysRXgUFnlnrsDurR7v7Bz+CDi6tQ/es3X0W/cHHA5ImlRyE2zAvNostH+zd9iu4Nv4qwZKoW5zOwXhS6o2+0HvyTYvYScAgz0kzjg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=Hf5J4YN8; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Hf5J4YN8" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1775672539; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Xleidnph+NUshh5pvUls0SU0uJehr12XDnRVHezXhPQ=; b=Hf5J4YN8aIBbbUtMKcg5qz++naFFqD77z2Q4YGlLgPXQr6auHO4ky/tsOTs3Upc5aEBE4G r33CbCrrjK2L/OeVB88blbvpO4rlaRC6CLyecUwzZ2Zj5xNrZem0DdhbmBYsfrUrgszps0 IihUmn6wXEM2XJGcieLRxmVIzjwTaBY= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-674-afIGGmNnN8u446X6GB901A-1; Wed, 08 Apr 2026 14:22:17 -0400 X-MC-Unique: afIGGmNnN8u446X6GB901A-1 X-Mimecast-MFC-AGG-ID: afIGGmNnN8u446X6GB901A_1775672536 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id D9C121956058; Wed, 8 Apr 2026 18:22:15 +0000 (UTC) Received: from virtlab1023.lab.eng.rdu2.redhat.lab.eng.rdu2.redhat.com (virtlab1023.lab.eng.rdu2.redhat.com [10.8.1.187]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 4F74E19560A6; Wed, 8 Apr 2026 18:22:15 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: seanjc@google.com, chang.seok.bae@intel.com Subject: [PATCH 1/7] KVM: VMX: remove regs argument of __vmx_vcpu_run Date: Wed, 8 Apr 2026 14:22:07 -0400 Message-ID: <20260408182213.522713-2-pbonzini@redhat.com> In-Reply-To: <20260408182213.522713-1-pbonzini@redhat.com> References: <20260408182213.522713-1-pbonzini@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 Content-Type: text/plain; charset="utf-8" Registers are reachable through vcpu_vmx, no need to pass a separate pointer to the regs[] array. No functional change intended. Signed-off-by: Paolo Bonzini --- arch/x86/kvm/kvm-asm-offsets.c | 1 + arch/x86/kvm/vmx/vmenter.S | 58 +++++++++++++++------------------- arch/x86/kvm/vmx/vmx.c | 3 +- arch/x86/kvm/vmx/vmx.h | 3 +- 4 files changed, 28 insertions(+), 37 deletions(-) diff --git a/arch/x86/kvm/kvm-asm-offsets.c b/arch/x86/kvm/kvm-asm-offsets.c index 24a710d37323..36ac61724dd7 100644 --- a/arch/x86/kvm/kvm-asm-offsets.c +++ b/arch/x86/kvm/kvm-asm-offsets.c @@ -24,6 +24,7 @@ static void __used common(void) =20 if (IS_ENABLED(CONFIG_KVM_INTEL)) { BLANK(); + OFFSET(VMX_vcpu_arch_regs, vcpu_vmx, vcpu.arch.regs); OFFSET(VMX_spec_ctrl, vcpu_vmx, spec_ctrl); } } diff --git a/arch/x86/kvm/vmx/vmenter.S b/arch/x86/kvm/vmx/vmenter.S index 4426d34811fc..3530c5b7b729 100644 --- a/arch/x86/kvm/vmx/vmenter.S +++ b/arch/x86/kvm/vmx/vmenter.S @@ -11,24 +11,24 @@ =20 #define WORD_SIZE (BITS_PER_LONG / 8) =20 -#define VCPU_RAX __VCPU_REGS_RAX * WORD_SIZE -#define VCPU_RCX __VCPU_REGS_RCX * WORD_SIZE -#define VCPU_RDX __VCPU_REGS_RDX * WORD_SIZE -#define VCPU_RBX __VCPU_REGS_RBX * WORD_SIZE +#define VCPU_RAX (VMX_vcpu_arch_regs + __VCPU_REGS_RAX * WORD_SIZE) +#define VCPU_RCX (VMX_vcpu_arch_regs + __VCPU_REGS_RCX * WORD_SIZE) +#define VCPU_RDX (VMX_vcpu_arch_regs + __VCPU_REGS_RDX * WORD_SIZE) +#define VCPU_RBX (VMX_vcpu_arch_regs + __VCPU_REGS_RBX * WORD_SIZE) /* Intentionally omit RSP as it's context switched by hardware */ -#define VCPU_RBP __VCPU_REGS_RBP * WORD_SIZE -#define VCPU_RSI __VCPU_REGS_RSI * WORD_SIZE -#define VCPU_RDI __VCPU_REGS_RDI * WORD_SIZE +#define VCPU_RBP (VMX_vcpu_arch_regs + __VCPU_REGS_RBP * WORD_SIZE) +#define VCPU_RSI (VMX_vcpu_arch_regs + __VCPU_REGS_RSI * WORD_SIZE) +#define VCPU_RDI (VMX_vcpu_arch_regs + __VCPU_REGS_RDI * WORD_SIZE) =20 #ifdef CONFIG_X86_64 -#define VCPU_R8 __VCPU_REGS_R8 * WORD_SIZE -#define VCPU_R9 __VCPU_REGS_R9 * WORD_SIZE -#define VCPU_R10 __VCPU_REGS_R10 * WORD_SIZE -#define VCPU_R11 __VCPU_REGS_R11 * WORD_SIZE -#define VCPU_R12 __VCPU_REGS_R12 * WORD_SIZE -#define VCPU_R13 __VCPU_REGS_R13 * WORD_SIZE -#define VCPU_R14 __VCPU_REGS_R14 * WORD_SIZE -#define VCPU_R15 __VCPU_REGS_R15 * WORD_SIZE +#define VCPU_R8 (VMX_vcpu_arch_regs + __VCPU_REGS_R8 * WORD_SIZE) +#define VCPU_R9 (VMX_vcpu_arch_regs + __VCPU_REGS_R9 * WORD_SIZE) +#define VCPU_R10 (VMX_vcpu_arch_regs + __VCPU_REGS_R10 * WORD_SIZE) +#define VCPU_R11 (VMX_vcpu_arch_regs + __VCPU_REGS_R11 * WORD_SIZE) +#define VCPU_R12 (VMX_vcpu_arch_regs + __VCPU_REGS_R12 * WORD_SIZE) +#define VCPU_R13 (VMX_vcpu_arch_regs + __VCPU_REGS_R13 * WORD_SIZE) +#define VCPU_R14 (VMX_vcpu_arch_regs + __VCPU_REGS_R14 * WORD_SIZE) +#define VCPU_R15 (VMX_vcpu_arch_regs + __VCPU_REGS_R15 * WORD_SIZE) #endif =20 .macro VMX_DO_EVENT_IRQOFF call_insn call_target @@ -68,7 +68,6 @@ /** * __vmx_vcpu_run - Run a vCPU via a transition to VMX guest mode * @vmx: struct vcpu_vmx * - * @regs: unsigned long * (to guest registers) * @flags: VMX_RUN_VMRESUME: use VMRESUME instead of VMLAUNCH * VMX_RUN_SAVE_SPEC_CTRL: save guest SPEC_CTRL into vmx->spec_ctrl * VMX_RUN_CLEAR_CPU_BUFFERS_FOR_MMIO: vCPU can access host MMIO @@ -94,12 +93,6 @@ SYM_FUNC_START(__vmx_vcpu_run) push %_ASM_ARG1 =20 /* Save @flags (used for VMLAUNCH vs. VMRESUME and mitigations). */ - push %_ASM_ARG3 - - /* - * Save @regs, _ASM_ARG2 may be modified by vmx_update_host_rsp() and - * @regs is needed after VM-Exit to save the guest's register values. - */ push %_ASM_ARG2 =20 lea (%_ASM_SP), %_ASM_ARG2 @@ -107,6 +100,9 @@ SYM_FUNC_START(__vmx_vcpu_run) =20 ALTERNATIVE "jmp .Lspec_ctrl_done", "", X86_FEATURE_MSR_SPEC_CTRL =20 + /* Reload @vmx, _ASM_ARG1 may be modified by vmx_update_host_rsp(). */ + mov WORD_SIZE(%_ASM_SP), %_ASM_DI + /* * SPEC_CTRL handling: if the guest's SPEC_CTRL value differs from the * host's, write the MSR. @@ -115,7 +111,6 @@ SYM_FUNC_START(__vmx_vcpu_run) * there must not be any returns or indirect branches between this code * and vmentry. */ - mov 2*WORD_SIZE(%_ASM_SP), %_ASM_DI #ifdef CONFIG_X86_64 mov VMX_spec_ctrl(%rdi), %rdx cmp PER_CPU_VAR(x86_spec_ctrl_current), %rdx @@ -142,8 +137,8 @@ SYM_FUNC_START(__vmx_vcpu_run) * an LFENCE to stop speculation from skipping the wrmsr. */ =20 - /* Load @regs to RAX. */ - mov (%_ASM_SP), %_ASM_AX + /* Load @vmx to RAX. */ + mov WORD_SIZE(%_ASM_SP), %_ASM_AX =20 /* Load guest registers. Don't clobber flags. */ mov VCPU_RCX(%_ASM_AX), %_ASM_CX @@ -162,7 +157,7 @@ SYM_FUNC_START(__vmx_vcpu_run) mov VCPU_R14(%_ASM_AX), %r14 mov VCPU_R15(%_ASM_AX), %r15 #endif - /* Load guest RAX. This kills the @regs pointer! */ + /* Load guest RAX. This kills the @vmx pointer! */ mov VCPU_RAX(%_ASM_AX), %_ASM_AX =20 /* @@ -172,7 +167,7 @@ SYM_FUNC_START(__vmx_vcpu_run) * do VERW. Else, do nothing (no mitigations needed/enabled). */ ALTERNATIVE_2 "", \ - __stringify(testl $VMX_RUN_CLEAR_CPU_BUFFERS_FOR_MMIO, WORD_SIZE(%= _ASM_SP); \ + __stringify(testl $VMX_RUN_CLEAR_CPU_BUFFERS_FOR_MMIO, (%_ASM_SP);= \ jz .Lskip_mmio_verw; \ VERW; \ .Lskip_mmio_verw:), \ @@ -180,7 +175,7 @@ SYM_FUNC_START(__vmx_vcpu_run) __stringify(VERW), X86_FEATURE_CLEAR_CPU_BUF_VM =20 /* Check @flags to see if VMLAUNCH or VMRESUME is needed. */ - testl $VMX_RUN_VMRESUME, WORD_SIZE(%_ASM_SP) + testl $VMX_RUN_VMRESUME, (%_ASM_SP) jz .Lvmlaunch =20 /* @@ -215,8 +210,8 @@ SYM_INNER_LABEL_ALIGN(vmx_vmexit, SYM_L_GLOBAL) /* Temporarily save guest's RAX. */ push %_ASM_AX =20 - /* Reload @regs to RAX. */ - mov WORD_SIZE(%_ASM_SP), %_ASM_AX + /* Reload @vmx to RAX. */ + mov 2*WORD_SIZE(%_ASM_SP), %_ASM_AX =20 /* Save all guest registers, including RAX from the stack */ pop VCPU_RAX(%_ASM_AX) @@ -241,9 +236,6 @@ SYM_INNER_LABEL_ALIGN(vmx_vmexit, SYM_L_GLOBAL) xor %ebx, %ebx =20 .Lclear_regs: - /* Discard @regs. The register is irrelevant, it just can't be RBX. */ - pop %_ASM_AX - /* * Clear all general purpose registers except RSP and RBX to prevent * speculative use of the guest's values, even those that are reloaded diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 8b24e682535b..054542ff0d02 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -7577,8 +7577,7 @@ static noinstr void vmx_vcpu_enter_exit(struct kvm_vc= pu *vcpu, if (vcpu->arch.cr2 !=3D native_read_cr2()) native_write_cr2(vcpu->arch.cr2); =20 - vmx->fail =3D __vmx_vcpu_run(vmx, (unsigned long *)&vcpu->arch.regs, - flags); + vmx->fail =3D __vmx_vcpu_run(vmx, flags); =20 vcpu->arch.cr2 =3D native_read_cr2(); vcpu->arch.regs_avail &=3D ~VMX_REGS_LAZY_LOAD_SET; diff --git a/arch/x86/kvm/vmx/vmx.h b/arch/x86/kvm/vmx/vmx.h index 70bfe81dea54..f963cb2982c9 100644 --- a/arch/x86/kvm/vmx/vmx.h +++ b/arch/x86/kvm/vmx/vmx.h @@ -373,8 +373,7 @@ void pt_update_intercept_for_msr(struct kvm_vcpu *vcpu); void vmx_update_host_rsp(struct vcpu_vmx *vmx, unsigned long host_rsp); void vmx_spec_ctrl_restore_host(struct vcpu_vmx *vmx, unsigned int flags); unsigned int __vmx_vcpu_run_flags(struct vcpu_vmx *vmx); -bool __vmx_vcpu_run(struct vcpu_vmx *vmx, unsigned long *regs, - unsigned int flags); +bool __vmx_vcpu_run(struct vcpu_vmx *vmx, unsigned int flags); void vmx_ept_load_pdptrs(struct kvm_vcpu *vcpu); =20 void vmx_set_intercept_for_msr(struct kvm_vcpu *vcpu, u32 msr, int type, b= ool set); --=20 2.52.0 From nobody Mon Jun 15 07:34:28 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 90A6D3D6694 for ; Wed, 8 Apr 2026 18:22:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775672543; cv=none; b=WvJukUqovGzL3tUBWeF4gKeARac/lVgxhUR9+7oLeNuvLVh81GRC9zJLvwab5kdTWFcGPeZP7RJg1msM+Y6QLDskdUt6e7W6SYQMqNmf0m6ILI4LQwBXqRvkgbEF4IX0Ogf0uAgCDL8CYWnRSiFTA3borKsQCp6huZgU9hUcvLA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775672543; c=relaxed/simple; bh=hyoP9LFQFVrPfFKNVAkgouFUkAuOnZpxm9Qf6CYdpGw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=kXXwmd57fh1tzRKP/jbBM3OKv3USAdmP9sEkhWLpsB+UFee4q2JlYPH38FLeShLQHKJbxU88jch7QL/cVBG1/W7xw4WrvG12c55jDE3A8muvbKX10u+4kVWwb/u2U+WoiUdn5zvLIyL6UvVvWANuusgo/ADEoBrVbjbxzoyF0Oo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=HD4Pw5RG; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="HD4Pw5RG" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1775672541; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=takCC9Js/aI1hvcqXzZQpTpg64EmmXa70IK5LR4cmiE=; b=HD4Pw5RGeHwmt5qSjHRfHdbGpuTTxzTBRr3amuHg7TJdVZhUVoNElhXd8lCo5ctpvO/8Un zlURUPViz2UHPCFPAf3ibAajqfv0yTtNsDs4U69oc7P2W/LECvb93xvCgRdFz6xRzGmnaL NAcTNp34fHbPlCQxrSIsFdBSR/Nta1A= Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-627-tqXIm0lsPCixkhOwYz2D7Q-1; Wed, 08 Apr 2026 14:22:18 -0400 X-MC-Unique: tqXIm0lsPCixkhOwYz2D7Q-1 X-Mimecast-MFC-AGG-ID: tqXIm0lsPCixkhOwYz2D7Q_1775672537 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id CEB5D180047F; Wed, 8 Apr 2026 18:22:16 +0000 (UTC) Received: from virtlab1023.lab.eng.rdu2.redhat.lab.eng.rdu2.redhat.com (virtlab1023.lab.eng.rdu2.redhat.com [10.8.1.187]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 0C59719560A6; Wed, 8 Apr 2026 18:22:15 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: seanjc@google.com, chang.seok.bae@intel.com Subject: [PATCH 2/7] KVM: VMX: more cleanups to __vmx_vcpu_run Date: Wed, 8 Apr 2026 14:22:08 -0400 Message-ID: <20260408182213.522713-3-pbonzini@redhat.com> In-Reply-To: <20260408182213.522713-1-pbonzini@redhat.com> References: <20260408182213.522713-1-pbonzini@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 Content-Type: text/plain; charset="utf-8" Slightly improve register allocation, loading vmx only once before vmlaunch/vmresume. This also makes the code slightly more similar to the one for AMD processors, in that both keep the pointer to struct vcpu_vmx or vcpu_svm in %rdi. The code for restoring the guest value of SPEC_CTRL is also the same for Intel and AMD. Signed-off-by: Paolo Bonzini --- arch/x86/kvm/vmx/vmenter.S | 83 ++++++++++++++++++-------------------- 1 file changed, 40 insertions(+), 43 deletions(-) diff --git a/arch/x86/kvm/vmx/vmenter.S b/arch/x86/kvm/vmx/vmenter.S index 3530c5b7b729..babafd7dfa5b 100644 --- a/arch/x86/kvm/vmx/vmenter.S +++ b/arch/x86/kvm/vmx/vmenter.S @@ -98,11 +98,11 @@ SYM_FUNC_START(__vmx_vcpu_run) lea (%_ASM_SP), %_ASM_ARG2 call vmx_update_host_rsp =20 - ALTERNATIVE "jmp .Lspec_ctrl_done", "", X86_FEATURE_MSR_SPEC_CTRL - /* Reload @vmx, _ASM_ARG1 may be modified by vmx_update_host_rsp(). */ mov WORD_SIZE(%_ASM_SP), %_ASM_DI =20 + ALTERNATIVE "jmp .Lspec_ctrl_done", "", X86_FEATURE_MSR_SPEC_CTRL + /* * SPEC_CTRL handling: if the guest's SPEC_CTRL value differs from the * host's, write the MSR. @@ -122,9 +122,9 @@ SYM_FUNC_START(__vmx_vcpu_run) mov PER_CPU_VAR(x86_spec_ctrl_current), %ecx xor %eax, %ecx mov VMX_spec_ctrl + 4(%edi), %edx - mov PER_CPU_VAR(x86_spec_ctrl_current + 4), %edi - xor %edx, %edi - or %edi, %ecx + mov PER_CPU_VAR(x86_spec_ctrl_current + 4), %esi + xor %edx, %esi + or %esi, %ecx je .Lspec_ctrl_done #endif mov $MSR_IA32_SPEC_CTRL, %ecx @@ -137,28 +137,25 @@ SYM_FUNC_START(__vmx_vcpu_run) * an LFENCE to stop speculation from skipping the wrmsr. */ =20 - /* Load @vmx to RAX. */ - mov WORD_SIZE(%_ASM_SP), %_ASM_AX - /* Load guest registers. Don't clobber flags. */ - mov VCPU_RCX(%_ASM_AX), %_ASM_CX - mov VCPU_RDX(%_ASM_AX), %_ASM_DX - mov VCPU_RBX(%_ASM_AX), %_ASM_BX - mov VCPU_RBP(%_ASM_AX), %_ASM_BP - mov VCPU_RSI(%_ASM_AX), %_ASM_SI - mov VCPU_RDI(%_ASM_AX), %_ASM_DI + mov VCPU_RAX(%_ASM_DI), %_ASM_AX + mov VCPU_RCX(%_ASM_DI), %_ASM_CX + mov VCPU_RDX(%_ASM_DI), %_ASM_DX + mov VCPU_RBX(%_ASM_DI), %_ASM_BX + mov VCPU_RBP(%_ASM_DI), %_ASM_BP + mov VCPU_RSI(%_ASM_DI), %_ASM_SI #ifdef CONFIG_X86_64 - mov VCPU_R8 (%_ASM_AX), %r8 - mov VCPU_R9 (%_ASM_AX), %r9 - mov VCPU_R10(%_ASM_AX), %r10 - mov VCPU_R11(%_ASM_AX), %r11 - mov VCPU_R12(%_ASM_AX), %r12 - mov VCPU_R13(%_ASM_AX), %r13 - mov VCPU_R14(%_ASM_AX), %r14 - mov VCPU_R15(%_ASM_AX), %r15 + mov VCPU_R8 (%_ASM_DI), %r8 + mov VCPU_R9 (%_ASM_DI), %r9 + mov VCPU_R10(%_ASM_DI), %r10 + mov VCPU_R11(%_ASM_DI), %r11 + mov VCPU_R12(%_ASM_DI), %r12 + mov VCPU_R13(%_ASM_DI), %r13 + mov VCPU_R14(%_ASM_DI), %r14 + mov VCPU_R15(%_ASM_DI), %r15 #endif - /* Load guest RAX. This kills the @vmx pointer! */ - mov VCPU_RAX(%_ASM_AX), %_ASM_AX + /* Load guest RDI. This kills the @vmx pointer! */ + mov VCPU_RDI(%_ASM_DI), %_ASM_DI =20 /* * Note, ALTERNATIVE_2 works in reverse order. If CLEAR_CPU_BUF_VM is @@ -207,29 +204,29 @@ SYM_INNER_LABEL_ALIGN(vmx_vmexit, SYM_L_GLOBAL) UNWIND_HINT_RESTORE ENDBR =20 - /* Temporarily save guest's RAX. */ - push %_ASM_AX + /* Temporarily save guest's RDI. */ + push %_ASM_DI =20 /* Reload @vmx to RAX. */ - mov 2*WORD_SIZE(%_ASM_SP), %_ASM_AX + mov 2*WORD_SIZE(%_ASM_SP), %_ASM_DI =20 - /* Save all guest registers, including RAX from the stack */ - pop VCPU_RAX(%_ASM_AX) - mov %_ASM_CX, VCPU_RCX(%_ASM_AX) - mov %_ASM_DX, VCPU_RDX(%_ASM_AX) - mov %_ASM_BX, VCPU_RBX(%_ASM_AX) - mov %_ASM_BP, VCPU_RBP(%_ASM_AX) - mov %_ASM_SI, VCPU_RSI(%_ASM_AX) - mov %_ASM_DI, VCPU_RDI(%_ASM_AX) + /* Save all guest registers, including RDI from the stack */ + mov %_ASM_AX, VCPU_RAX(%_ASM_DI) + mov %_ASM_CX, VCPU_RCX(%_ASM_DI) + mov %_ASM_DX, VCPU_RDX(%_ASM_DI) + mov %_ASM_BX, VCPU_RBX(%_ASM_DI) + mov %_ASM_BP, VCPU_RBP(%_ASM_DI) + mov %_ASM_SI, VCPU_RSI(%_ASM_DI) + pop VCPU_RDI(%_ASM_DI) #ifdef CONFIG_X86_64 - mov %r8, VCPU_R8 (%_ASM_AX) - mov %r9, VCPU_R9 (%_ASM_AX) - mov %r10, VCPU_R10(%_ASM_AX) - mov %r11, VCPU_R11(%_ASM_AX) - mov %r12, VCPU_R12(%_ASM_AX) - mov %r13, VCPU_R13(%_ASM_AX) - mov %r14, VCPU_R14(%_ASM_AX) - mov %r15, VCPU_R15(%_ASM_AX) + mov %r8, VCPU_R8 (%_ASM_DI) + mov %r9, VCPU_R9 (%_ASM_DI) + mov %r10, VCPU_R10(%_ASM_DI) + mov %r11, VCPU_R11(%_ASM_DI) + mov %r12, VCPU_R12(%_ASM_DI) + mov %r13, VCPU_R13(%_ASM_DI) + mov %r14, VCPU_R14(%_ASM_DI) + mov %r15, VCPU_R15(%_ASM_DI) #endif =20 /* Clear return value to indicate VM-Exit (as opposed to VM-Fail). */ --=20 2.52.0 From nobody Mon Jun 15 07:34:28 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 924D43D9DB6 for ; Wed, 8 Apr 2026 18:22:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775672546; cv=none; b=DrLWbrczsqJOAyG3wvWUTDGAbtYsFrU8XE1GtCtmpN7C0J5jbsMl9b2ou+HDnZzliHKJVXnVQ+q5SMZSRikKbi+6si1O0lZTRml3EVIPmbIEQVDPQHTIpg76+3TuDFv04OW6+Ou4Epa+xiYsTuPS8Oc9Y2sCP7m/mk+9XVO9ysU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775672546; c=relaxed/simple; bh=SMWaakQdG8v8qnJYS8hgA/X3zjMLR3elSRIZKhk76d0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=RB+Q9eFpiQ4Z0+gKjNtIdQ3zUNuSPM3mMzRZLM7wL9hwraa9BMmvZXORFzk5N9nHdtpsZScC4b1VrjP+ClDTY3hIxUYS8gYaKAfqscr45yotJuCdCr5qU6cQtoQT0GWDH96uPj/tUcQC/oGBt0CRfjsqmkPGFM5d6eM2Wkr7hZg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=IXQmw2iS; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="IXQmw2iS" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1775672544; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=XpD7XUIH/aJ98uDRklM9NUpHLaSPhnqbG4dNKJV4vrg=; b=IXQmw2iS1Yns/WZVbryTcByfl8P+3e+EZxN2ObdUBy8XYe3GJSgNayD+JUkSnUzIARIRcp kNg5T9Ca8Tz3kR795UPEj/4fF+/kFSB0YYpxiry+5YL71wgl7VGYqTOtkflFZbWjkmzN5y NUHHxNPUiK6ycJ9zcw0mBcBBEFLPyhs= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-192-7hNTWQXcMziEp-qCKzMdKg-1; Wed, 08 Apr 2026 14:22:22 -0400 X-MC-Unique: 7hNTWQXcMziEp-qCKzMdKg-1 X-Mimecast-MFC-AGG-ID: 7hNTWQXcMziEp-qCKzMdKg_1775672537 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 916D31955F35; Wed, 8 Apr 2026 18:22:17 +0000 (UTC) Received: from virtlab1023.lab.eng.rdu2.redhat.lab.eng.rdu2.redhat.com (virtlab1023.lab.eng.rdu2.redhat.com [10.8.1.187]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 011941955F2B; Wed, 8 Apr 2026 18:22:16 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: seanjc@google.com, chang.seok.bae@intel.com Subject: [PATCH 3/7] KVM: SVM: prepare for making SPEC_CTRL switch common with VMX Date: Wed, 8 Apr 2026 14:22:09 -0400 Message-ID: <20260408182213.522713-4-pbonzini@redhat.com> In-Reply-To: <20260408182213.522713-1-pbonzini@redhat.com> References: <20260408182213.522713-1-pbonzini@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 Content-Type: text/plain; charset="utf-8" Remove the local labels and restrict RESTORE_*_BODY to just the restore code itself. Since the alternatives are different between VMX and SVM, having labels in the per-vendor file and jumps in another would be too confusing. Signed-off-by: Paolo Bonzini --- arch/x86/kvm/svm/vmenter.S | 54 +++++++++++++++++++++----------------- 1 file changed, 30 insertions(+), 24 deletions(-) diff --git a/arch/x86/kvm/svm/vmenter.S b/arch/x86/kvm/svm/vmenter.S index 3392bcadfb89..f4ea862652d8 100644 --- a/arch/x86/kvm/svm/vmenter.S +++ b/arch/x86/kvm/svm/vmenter.S @@ -39,10 +39,8 @@ ALTERNATIVE_2 "", \ "jmp 800f", X86_FEATURE_MSR_SPEC_CTRL, \ "", X86_FEATURE_V_SPEC_CTRL -801: .endm -.macro RESTORE_GUEST_SPEC_CTRL_BODY -800: +.macro RESTORE_GUEST_SPEC_CTRL_BODY guest_spec_ctrl:req, label:req /* * SPEC_CTRL handling: if the guest's SPEC_CTRL value differs from the * host's, write the MSR. This is kept out-of-line so that the common @@ -53,24 +51,23 @@ * and vmentry. */ #ifdef CONFIG_X86_64 - mov SVM_spec_ctrl(%rdi), %rdx + mov \guest_spec_ctrl, %rdx cmp PER_CPU_VAR(x86_spec_ctrl_current), %rdx - je 801b + je \label movl %edx, %eax shr $32, %rdx #else - mov SVM_spec_ctrl(%edi), %eax + mov \guest_spec_ctrl, %eax mov PER_CPU_VAR(x86_spec_ctrl_current), %ecx xor %eax, %ecx - mov SVM_spec_ctrl + 4(%edi), %edx + mov 4 + \guest_spec_ctrl, %edx mov PER_CPU_VAR(x86_spec_ctrl_current + 4), %esi xor %edx, %esi or %esi, %ecx - je 801b + je \label #endif mov $MSR_IA32_SPEC_CTRL, %ecx wrmsr - jmp 801b .endm =20 .macro RESTORE_HOST_SPEC_CTRL @@ -78,10 +75,8 @@ ALTERNATIVE_2 "", \ "jmp 900f", X86_FEATURE_MSR_SPEC_CTRL, \ "", X86_FEATURE_V_SPEC_CTRL -901: .endm -.macro RESTORE_HOST_SPEC_CTRL_BODY spec_ctrl_intercepted:req -900: +.macro RESTORE_HOST_SPEC_CTRL_BODY guest_spec_ctrl:req, spec_ctrl_intercep= ted:req, label:req /* Same for after vmexit. */ mov $MSR_IA32_SPEC_CTRL, %ecx =20 @@ -92,28 +87,27 @@ cmpb $0, \spec_ctrl_intercepted jnz 998f rdmsr - movl %eax, SVM_spec_ctrl(%_ASM_DI) - movl %edx, SVM_spec_ctrl + 4(%_ASM_DI) + movl %eax, \guest_spec_ctrl + movl %edx, 4 + \guest_spec_ctrl 998: /* Now restore the host value of the MSR if different from the guest's. = */ #ifdef CONFIG_X86_64 mov PER_CPU_VAR(x86_spec_ctrl_current), %rdx - cmp SVM_spec_ctrl(%rdi), %rdx - je 901b + cmp \guest_spec_ctrl, %rdx + je \label movl %edx, %eax shr $32, %rdx #else mov PER_CPU_VAR(x86_spec_ctrl_current), %eax - mov SVM_spec_ctrl(%edi), %esi + mov \guest_spec_ctrl, %esi xor %eax, %esi mov PER_CPU_VAR(x86_spec_ctrl_current + 4), %edx - mov SVM_spec_ctrl + 4(%edi), %edi + mov 4 + \guest_spec_ctrl, %edi xor %edx, %edi or %edi, %esi - je 901b + je \label #endif wrmsr - jmp 901b .endm =20 #define SVM_CLEAR_CPU_BUFFERS \ @@ -162,6 +156,7 @@ SYM_FUNC_START(__svm_vcpu_run) =20 /* Clobbers RAX, RCX, RDX (and ESI on 32-bit), consumes RDI (@svm). */ RESTORE_GUEST_SPEC_CTRL +801: =20 /* * Use a single vmcb (vmcb01 because it's always valid) for @@ -242,6 +237,7 @@ SYM_FUNC_START(__svm_vcpu_run) * and RSP (pointer to @spec_ctrl_intercepted). */ RESTORE_HOST_SPEC_CTRL +901: =20 /* * Mitigate RETBleed for AMD/Hygon Zen uarch. RET should be @@ -295,8 +291,12 @@ SYM_FUNC_START(__svm_vcpu_run) pop %_ASM_BP RET =20 - RESTORE_GUEST_SPEC_CTRL_BODY - RESTORE_HOST_SPEC_CTRL_BODY (%_ASM_SP) +800: + RESTORE_GUEST_SPEC_CTRL_BODY SVM_spec_ctrl(%_ASM_DI), 801b + jmp 801b +900: + RESTORE_HOST_SPEC_CTRL_BODY SVM_spec_ctrl(%_ASM_DI), (%_ASM_SP), 901b + jmp 901b =20 10: cmpb $0, _ASM_RIP(kvm_rebooting) jne 2b @@ -362,6 +362,7 @@ SYM_FUNC_START(__svm_sev_es_vcpu_run) =20 /* Clobbers RAX, RCX, and RDX (@hostsa), consumes RDI (@svm). */ RESTORE_GUEST_SPEC_CTRL +801: =20 /* Get svm->current_vmcb->pa into RAX. */ mov SVM_current_vmcb(%rdi), %rax @@ -378,6 +379,7 @@ SYM_FUNC_START(__svm_sev_es_vcpu_run) =20 /* Clobbers RAX, RCX, RDX, consumes RDI (@svm) and RSI (@spec_ctrl_interc= epted). */ RESTORE_HOST_SPEC_CTRL +901: =20 /* * Mitigate RETBleed for AMD/Hygon Zen uarch. RET should be @@ -391,8 +393,12 @@ SYM_FUNC_START(__svm_sev_es_vcpu_run) FRAME_END RET =20 - RESTORE_GUEST_SPEC_CTRL_BODY - RESTORE_HOST_SPEC_CTRL_BODY %sil +800: + RESTORE_GUEST_SPEC_CTRL_BODY SVM_spec_ctrl(%_ASM_DI), 801b + jmp 801b +900: + RESTORE_HOST_SPEC_CTRL_BODY SVM_spec_ctrl(%_ASM_DI), %sil, 901b + jmp 901b =20 3: cmpb $0, kvm_rebooting(%rip) jne 2b --=20 2.52.0 From nobody Mon Jun 15 07:34:28 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A23343D9037 for ; Wed, 8 Apr 2026 18:22:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775672554; cv=none; b=d+J4iiSMSKfaGaHTkDW2uectRgbDjVcwdYrtaAVNB4wyTbIfSm7orz3tB4TgG5AlS/nOtrM17yl47yrGlDyPq4IFyNInscHKKHkj1YsDilXWn4Noo4Uu4qoYR2gnFnl6CsB1YeoEQiGNr4WkSuBcVItbifWd8JqJqoW9y7jrg5Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775672554; c=relaxed/simple; bh=XAXsoS89VcrRSkgM4uTrvlppPDRWHMLFH/06oqTfBF0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=LgburQ8RkcVAFMTBAC5saFNnq6uu41yC7iZquBNuhMbblIIzgok2cP7Akfm1ifpXz/MEgh02g6NF5F9YMLHjv02E38LJM1e8cUsYrnCuYuSUjHjmvH5JBmFOATKJortPt2gnvQCX/RKNvlN1JnNrDWKmhEfmvJiYkVq3TnJmNWI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=Te2bfQmv; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Te2bfQmv" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1775672551; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/JsFVBcKynKpBm3OxqTNh9hV/BVX0i5eBbcHwpUj5DE=; b=Te2bfQmvdwFjbAr7gIf9MVuTU8vOdpLf3pZF23RIpJ21RhSeU8fEzwYPsR0wi2F2Fo25Tg 8NjvGiNKnyLYV9OtKWW65EMrVT2yoGTZNGbRMmw/jsrbvClKaY2kHUUZ1JzPMxTJFYeRfe 7pWarVP9u00FKgZiL9JPc1oMRT0VY0k= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-66-Nj265khYN8KmPMXXItxuxA-1; Wed, 08 Apr 2026 14:22:22 -0400 X-MC-Unique: Nj265khYN8KmPMXXItxuxA-1 X-Mimecast-MFC-AGG-ID: Nj265khYN8KmPMXXItxuxA_1775672538 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 5C3D81956096; Wed, 8 Apr 2026 18:22:18 +0000 (UTC) Received: from virtlab1023.lab.eng.rdu2.redhat.lab.eng.rdu2.redhat.com (virtlab1023.lab.eng.rdu2.redhat.com [10.8.1.187]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id AF0361955F2B; Wed, 8 Apr 2026 18:22:17 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: seanjc@google.com, chang.seok.bae@intel.com Subject: [PATCH 4/7] KVM: SVM: adopt the same VMX_RUN_* flags as VMX Date: Wed, 8 Apr 2026 14:22:10 -0400 Message-ID: <20260408182213.522713-5-pbonzini@redhat.com> In-Reply-To: <20260408182213.522713-1-pbonzini@redhat.com> References: <20260408182213.522713-1-pbonzini@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 Content-Type: text/plain; charset="utf-8" Rename VMX_RUN_* to KVM_ENTER_* (to not confuse with KVM_RUN_* that already exists) and replace SVM's spec_ctrl_intercepted with that same bitmask. Signed-off-by: Paolo Bonzini --- arch/x86/kvm/svm/svm.c | 14 +++++++++----- arch/x86/kvm/svm/vmenter.S | 21 +++++++++++---------- arch/x86/kvm/vmenter.h | 9 +++++++++ arch/x86/kvm/vmx/run_flags.h | 9 --------- arch/x86/kvm/vmx/vmenter.S | 12 ++++++------ arch/x86/kvm/vmx/vmx.c | 13 +++++++------ arch/x86/kvm/vmx/vmx.h | 3 +-- 7 files changed, 43 insertions(+), 38 deletions(-) create mode 100644 arch/x86/kvm/vmenter.h delete mode 100644 arch/x86/kvm/vmx/run_flags.h diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index e6477affac9a..19808eee43de 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c @@ -49,6 +49,7 @@ =20 #include "trace.h" =20 +#include "vmenter.h" #include "svm.h" #include "svm_ops.h" =20 @@ -4253,7 +4254,7 @@ static fastpath_t svm_exit_handlers_fastpath(struct k= vm_vcpu *vcpu) return EXIT_FASTPATH_NONE; } =20 -static noinstr void svm_vcpu_enter_exit(struct kvm_vcpu *vcpu, bool spec_c= trl_intercepted) +static noinstr void svm_vcpu_enter_exit(struct kvm_vcpu *vcpu, unsigned en= ter_flags) { struct svm_cpu_data *sd =3D per_cpu_ptr(&svm_data, vcpu->cpu); struct vcpu_svm *svm =3D to_svm(vcpu); @@ -4275,10 +4276,10 @@ static noinstr void svm_vcpu_enter_exit(struct kvm_= vcpu *vcpu, bool spec_ctrl_in amd_clear_divider(); =20 if (sev_es_guest(vcpu->kvm)) - __svm_sev_es_vcpu_run(svm, spec_ctrl_intercepted, + __svm_sev_es_vcpu_run(svm, enter_flags, sev_es_host_save_area(sd)); else - __svm_vcpu_run(svm, spec_ctrl_intercepted); + __svm_vcpu_run(svm, enter_flags); =20 raw_local_irq_disable(); =20 @@ -4289,7 +4290,10 @@ static __no_kcsan fastpath_t svm_vcpu_run(struct kvm= _vcpu *vcpu, u64 run_flags) { bool force_immediate_exit =3D run_flags & KVM_RUN_FORCE_IMMEDIATE_EXIT; struct vcpu_svm *svm =3D to_svm(vcpu); - bool spec_ctrl_intercepted =3D msr_write_intercepted(vcpu, MSR_IA32_SPEC_= CTRL); + unsigned enter_flags =3D 0; + + if (msr_write_intercepted(vcpu, MSR_IA32_SPEC_CTRL)) + enter_flags |=3D KVM_ENTER_SAVE_SPEC_CTRL; =20 trace_kvm_entry(vcpu, force_immediate_exit); =20 @@ -4370,7 +4374,7 @@ static __no_kcsan fastpath_t svm_vcpu_run(struct kvm_= vcpu *vcpu, u64 run_flags) if (!static_cpu_has(X86_FEATURE_V_SPEC_CTRL)) x86_spec_ctrl_set_guest(svm->virt_spec_ctrl); =20 - svm_vcpu_enter_exit(vcpu, spec_ctrl_intercepted); + svm_vcpu_enter_exit(vcpu, enter_flags); =20 if (!static_cpu_has(X86_FEATURE_V_SPEC_CTRL)) x86_spec_ctrl_restore_host(svm->virt_spec_ctrl); diff --git a/arch/x86/kvm/svm/vmenter.S b/arch/x86/kvm/svm/vmenter.S index f4ea862652d8..545dc55582bf 100644 --- a/arch/x86/kvm/svm/vmenter.S +++ b/arch/x86/kvm/svm/vmenter.S @@ -7,6 +7,7 @@ #include #include #include "kvm-asm-offsets.h" +#include "vmenter.h" =20 #define WORD_SIZE (BITS_PER_LONG / 8) =20 @@ -76,7 +77,7 @@ "jmp 900f", X86_FEATURE_MSR_SPEC_CTRL, \ "", X86_FEATURE_V_SPEC_CTRL .endm -.macro RESTORE_HOST_SPEC_CTRL_BODY guest_spec_ctrl:req, spec_ctrl_intercep= ted:req, label:req +.macro RESTORE_HOST_SPEC_CTRL_BODY guest_spec_ctrl:req, enter_flags:req, l= abel:req /* Same for after vmexit. */ mov $MSR_IA32_SPEC_CTRL, %ecx =20 @@ -84,7 +85,7 @@ * Load the value that the guest had written into MSR_IA32_SPEC_CTRL, * if it was not intercepted during guest execution. */ - cmpb $0, \spec_ctrl_intercepted + testl $KVM_ENTER_SAVE_SPEC_CTRL, \enter_flags jnz 998f rdmsr movl %eax, \guest_spec_ctrl @@ -115,8 +116,8 @@ =20 /** * __svm_vcpu_run - Run a vCPU via a transition to SVM guest mode - * @svm: struct vcpu_svm * - * @spec_ctrl_intercepted: bool + * @svm: struct vcpu_svm * + * @enter_flags: u32 */ SYM_FUNC_START(__svm_vcpu_run) push %_ASM_BP @@ -274,7 +275,7 @@ SYM_FUNC_START(__svm_vcpu_run) xor %r15d, %r15d #endif =20 - /* "Pop" @spec_ctrl_intercepted. */ + /* "Pop" @enter_flags. */ pop %_ASM_BX =20 pop %_ASM_BX @@ -335,8 +336,8 @@ SYM_FUNC_END(__svm_vcpu_run) =20 /** * __svm_sev_es_vcpu_run - Run a SEV-ES vCPU via a transition to SVM guest= mode - * @svm: struct vcpu_svm * - * @spec_ctrl_intercepted: bool + * @svm: struct vcpu_svm * + * @enter_flags: u32 */ SYM_FUNC_START(__svm_sev_es_vcpu_run) FRAME_BEGIN @@ -355,7 +356,7 @@ SYM_FUNC_START(__svm_sev_es_vcpu_run) =20 /* * Save volatile registers that hold arguments that are needed after - * #VMEXIT (RDI=3D@svm and RSI=3D@spec_ctrl_intercepted). + * #VMEXIT (RDI=3D@svm and RSI=3D@enter_flags). */ mov %rdi, SEV_ES_RDI (%rdx) mov %rsi, SEV_ES_RSI (%rdx) @@ -377,7 +378,7 @@ SYM_FUNC_START(__svm_sev_es_vcpu_run) /* IMPORTANT: Stuff the RSB immediately after VM-Exit, before RET! */ FILL_RETURN_BUFFER %rax, RSB_CLEAR_LOOPS, X86_FEATURE_RSB_VMEXIT =20 - /* Clobbers RAX, RCX, RDX, consumes RDI (@svm) and RSI (@spec_ctrl_interc= epted). */ + /* Clobbers RAX, RCX, RDX, consumes RDI (@svm) and RSI (@enter_flags). */ RESTORE_HOST_SPEC_CTRL 901: =20 @@ -397,7 +398,7 @@ SYM_FUNC_START(__svm_sev_es_vcpu_run) RESTORE_GUEST_SPEC_CTRL_BODY SVM_spec_ctrl(%_ASM_DI), 801b jmp 801b 900: - RESTORE_HOST_SPEC_CTRL_BODY SVM_spec_ctrl(%_ASM_DI), %sil, 901b + RESTORE_HOST_SPEC_CTRL_BODY SVM_spec_ctrl(%_ASM_DI), %esi, 901b jmp 901b =20 3: cmpb $0, kvm_rebooting(%rip) diff --git a/arch/x86/kvm/vmenter.h b/arch/x86/kvm/vmenter.h new file mode 100644 index 000000000000..29cdae650069 --- /dev/null +++ b/arch/x86/kvm/vmenter.h @@ -0,0 +1,9 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef __KVM_X86_VMENTER_H +#define __KVM_X86_VMENTER_H + +#define KVM_ENTER_VMRESUME BIT(0) +#define KVM_ENTER_SAVE_SPEC_CTRL BIT(1) +#define KVM_ENTER_CLEAR_CPU_BUFFERS_FOR_MMIO BIT(2) + +#endif /* __KVM_X86_VMENTER_H */ diff --git a/arch/x86/kvm/vmx/run_flags.h b/arch/x86/kvm/vmx/run_flags.h deleted file mode 100644 index 6a87a12135fb..000000000000 --- a/arch/x86/kvm/vmx/run_flags.h +++ /dev/null @@ -1,9 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0 */ -#ifndef __KVM_X86_VMX_RUN_FLAGS_H -#define __KVM_X86_VMX_RUN_FLAGS_H - -#define VMX_RUN_VMRESUME BIT(0) -#define VMX_RUN_SAVE_SPEC_CTRL BIT(1) -#define VMX_RUN_CLEAR_CPU_BUFFERS_FOR_MMIO BIT(2) - -#endif /* __KVM_X86_VMX_RUN_FLAGS_H */ diff --git a/arch/x86/kvm/vmx/vmenter.S b/arch/x86/kvm/vmx/vmenter.S index babafd7dfa5b..bed5a753166b 100644 --- a/arch/x86/kvm/vmx/vmenter.S +++ b/arch/x86/kvm/vmx/vmenter.S @@ -7,7 +7,7 @@ #include #include #include "kvm-asm-offsets.h" -#include "run_flags.h" +#include "vmenter.h" =20 #define WORD_SIZE (BITS_PER_LONG / 8) =20 @@ -68,9 +68,9 @@ /** * __vmx_vcpu_run - Run a vCPU via a transition to VMX guest mode * @vmx: struct vcpu_vmx * - * @flags: VMX_RUN_VMRESUME: use VMRESUME instead of VMLAUNCH - * VMX_RUN_SAVE_SPEC_CTRL: save guest SPEC_CTRL into vmx->spec_ctrl - * VMX_RUN_CLEAR_CPU_BUFFERS_FOR_MMIO: vCPU can access host MMIO + * @flags: KVM_ENTER_RUN_VMRESUME: use VMRESUME instead of VMLAUNCH + * KVM_ENTER_RUN_SAVE_SPEC_CTRL: save guest SPEC_CTRL into vmx->spec_ctrl + * KVM_ENTER_RUN_CLEAR_CPU_BUFFERS_FOR_MMIO: vCPU can access host MMIO * * Returns: * 0 on VM-Exit, 1 on VM-Fail @@ -164,7 +164,7 @@ SYM_FUNC_START(__vmx_vcpu_run) * do VERW. Else, do nothing (no mitigations needed/enabled). */ ALTERNATIVE_2 "", \ - __stringify(testl $VMX_RUN_CLEAR_CPU_BUFFERS_FOR_MMIO, (%_ASM_SP);= \ + __stringify(testl $KVM_ENTER_CLEAR_CPU_BUFFERS_FOR_MMIO, (%_ASM_SP= ); \ jz .Lskip_mmio_verw; \ VERW; \ .Lskip_mmio_verw:), \ @@ -172,7 +172,7 @@ SYM_FUNC_START(__vmx_vcpu_run) __stringify(VERW), X86_FEATURE_CLEAR_CPU_BUF_VM =20 /* Check @flags to see if VMLAUNCH or VMRESUME is needed. */ - testl $VMX_RUN_VMRESUME, (%_ASM_SP) + testl $KVM_ENTER_VMRESUME, (%_ASM_SP) jz .Lvmlaunch =20 /* diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 054542ff0d02..ef0de3cc0bc2 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -72,6 +72,7 @@ #include "x86_ops.h" #include "smm.h" #include "vmx_onhyperv.h" +#include "vmenter.h" #include "posted_intr.h" =20 #include "mmu/spte.h" @@ -1000,12 +1001,12 @@ static bool msr_write_intercepted(struct vcpu_vmx *= vmx, u32 msr) return vmx_test_msr_bitmap_write(vmx->loaded_vmcs->msr_bitmap, msr); } =20 -unsigned int __vmx_vcpu_run_flags(struct vcpu_vmx *vmx) +unsigned int __vmx_vcpu_enter_flags(struct vcpu_vmx *vmx) { unsigned int flags =3D 0; =20 if (vmx->loaded_vmcs->launched) - flags |=3D VMX_RUN_VMRESUME; + flags |=3D KVM_ENTER_VMRESUME; =20 /* * If writes to the SPEC_CTRL MSR aren't intercepted, the guest is free @@ -1013,11 +1014,11 @@ unsigned int __vmx_vcpu_run_flags(struct vcpu_vmx *= vmx) * it after vmexit and store it in vmx->spec_ctrl. */ if (!msr_write_intercepted(vmx, MSR_IA32_SPEC_CTRL)) - flags |=3D VMX_RUN_SAVE_SPEC_CTRL; + flags |=3D KVM_ENTER_SAVE_SPEC_CTRL; =20 if (cpu_feature_enabled(X86_FEATURE_CLEAR_CPU_BUF_VM_MMIO) && kvm_vcpu_can_access_host_mmio(&vmx->vcpu)) - flags |=3D VMX_RUN_CLEAR_CPU_BUFFERS_FOR_MMIO; + flags |=3D KVM_ENTER_CLEAR_CPU_BUFFERS_FOR_MMIO; =20 return flags; } @@ -7504,7 +7505,7 @@ void noinstr vmx_spec_ctrl_restore_host(struct vcpu_v= mx *vmx, if (!cpu_feature_enabled(X86_FEATURE_MSR_SPEC_CTRL)) return; =20 - if (flags & VMX_RUN_SAVE_SPEC_CTRL) + if (flags & KVM_ENTER_SAVE_SPEC_CTRL) vmx->spec_ctrl =3D native_rdmsrq(MSR_IA32_SPEC_CTRL); =20 /* @@ -7695,7 +7696,7 @@ fastpath_t vmx_vcpu_run(struct kvm_vcpu *vcpu, u64 ru= n_flags) kvm_wait_lapic_expire(vcpu); =20 /* The actual VMENTER/EXIT is in the .noinstr.text section. */ - vmx_vcpu_enter_exit(vcpu, __vmx_vcpu_run_flags(vmx)); + vmx_vcpu_enter_exit(vcpu, __vmx_vcpu_enter_flags(vmx)); =20 /* All fields are clean at this point */ if (kvm_is_using_evmcs()) { diff --git a/arch/x86/kvm/vmx/vmx.h b/arch/x86/kvm/vmx/vmx.h index f963cb2982c9..9cb5a205f19d 100644 --- a/arch/x86/kvm/vmx/vmx.h +++ b/arch/x86/kvm/vmx/vmx.h @@ -15,7 +15,6 @@ #include "vmcs.h" #include "vmx_ops.h" #include "../cpuid.h" -#include "run_flags.h" #include "../mmu.h" #include "common.h" =20 @@ -372,7 +371,7 @@ struct vmx_uret_msr *vmx_find_uret_msr(struct vcpu_vmx = *vmx, u32 msr); void pt_update_intercept_for_msr(struct kvm_vcpu *vcpu); void vmx_update_host_rsp(struct vcpu_vmx *vmx, unsigned long host_rsp); void vmx_spec_ctrl_restore_host(struct vcpu_vmx *vmx, unsigned int flags); -unsigned int __vmx_vcpu_run_flags(struct vcpu_vmx *vmx); +unsigned int __vmx_vcpu_enter_flags(struct vcpu_vmx *vmx); bool __vmx_vcpu_run(struct vcpu_vmx *vmx, unsigned int flags); void vmx_ept_load_pdptrs(struct kvm_vcpu *vcpu); =20 --=20 2.52.0 From nobody Mon Jun 15 07:34:28 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 613D93D6689 for ; Wed, 8 Apr 2026 18:22:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775672547; cv=none; b=T2UeakCOuGVpJgWZL7XfobaUhthAADu8Uv4iBQHP7YbETJcpba5+v1I8hGepFyp7wwLE8h+5xNz3A17YDtpwyFIWB6bhJad1x2uZmXaST1lR+bljuAzYojJhQpmfc4cpaUwFeIOleOm9VP8rYSqYr+EX/7H9ljiOoa7X+lvczfI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775672547; c=relaxed/simple; bh=nUqrBWeXfG9HLCddqvAosufgQMY5yXV/dnUFQRz3KUQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=QldPc930dyz0dRnwPcsdIeKQN9tc7eyRYf4QX/nw7LoORLPzJkpIGyReUqjTXfPJn45iPQGl+Vx3ryzeR9mNeJ/n61qF5C1IKzg391rofzg5VvrDPdKQ+7hHOg7YxxKc0A9P5dpzmYLgT9925WOXTItC1T4FfGexHzLdoZMDEGw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=e7JyATWs; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="e7JyATWs" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1775672545; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=utRJDEDUt40XYoA1dmLdZdoz1BpbTCbac6tFrHskUNg=; b=e7JyATWslb+SdpXR3Sczz4AVrPJB3H5lU/t+B/p7lCFGunkH5n3Dp2huDC9EhnObcpqi9W H94TSd5p6XmMZH3aX/Q5qEJyCmGAtp7pXgNc1ZJ8E0UDzUYiCL/UX94K2vGaLHeC/mqN6n tJ4j2YWxq5UlGcRLxx2io5hi3GZwbNU= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-260-1_wbkni1PXm0sS_WpeI5bw-1; Wed, 08 Apr 2026 14:22:21 -0400 X-MC-Unique: 1_wbkni1PXm0sS_WpeI5bw-1 X-Mimecast-MFC-AGG-ID: 1_wbkni1PXm0sS_WpeI5bw_1775672539 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id F04EA1955DE3; Wed, 8 Apr 2026 18:22:18 +0000 (UTC) Received: from virtlab1023.lab.eng.rdu2.redhat.lab.eng.rdu2.redhat.com (virtlab1023.lab.eng.rdu2.redhat.com [10.8.1.187]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 697231955D6C; Wed, 8 Apr 2026 18:22:18 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: seanjc@google.com, chang.seok.bae@intel.com Subject: [PATCH 5/7] KVM: SVM: extract RESTORE_*_SPEC_CTRL_BODY out of svm/vmenter.S Date: Wed, 8 Apr 2026 14:22:11 -0400 Message-ID: <20260408182213.522713-6-pbonzini@redhat.com> In-Reply-To: <20260408182213.522713-1-pbonzini@redhat.com> References: <20260408182213.522713-1-pbonzini@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 Content-Type: text/plain; charset="utf-8" Pure code movement, in preparation for using these macros for VMX as well. Signed-off-by: Paolo Bonzini --- arch/x86/kvm/svm/vmenter.S | 62 ---------------------------------- arch/x86/kvm/vmenter.h | 68 +++++++++++++++++++++++++++++++++++++- 2 files changed, 67 insertions(+), 63 deletions(-) diff --git a/arch/x86/kvm/svm/vmenter.S b/arch/x86/kvm/svm/vmenter.S index 545dc55582bf..67a05fe9827e 100644 --- a/arch/x86/kvm/svm/vmenter.S +++ b/arch/x86/kvm/svm/vmenter.S @@ -41,35 +41,6 @@ "jmp 800f", X86_FEATURE_MSR_SPEC_CTRL, \ "", X86_FEATURE_V_SPEC_CTRL .endm -.macro RESTORE_GUEST_SPEC_CTRL_BODY guest_spec_ctrl:req, label:req - /* - * SPEC_CTRL handling: if the guest's SPEC_CTRL value differs from the - * host's, write the MSR. This is kept out-of-line so that the common - * case does not have to jump. - * - * IMPORTANT: To avoid RSB underflow attacks and any other nastiness, - * there must not be any returns or indirect branches between this code - * and vmentry. - */ -#ifdef CONFIG_X86_64 - mov \guest_spec_ctrl, %rdx - cmp PER_CPU_VAR(x86_spec_ctrl_current), %rdx - je \label - movl %edx, %eax - shr $32, %rdx -#else - mov \guest_spec_ctrl, %eax - mov PER_CPU_VAR(x86_spec_ctrl_current), %ecx - xor %eax, %ecx - mov 4 + \guest_spec_ctrl, %edx - mov PER_CPU_VAR(x86_spec_ctrl_current + 4), %esi - xor %edx, %esi - or %esi, %ecx - je \label -#endif - mov $MSR_IA32_SPEC_CTRL, %ecx - wrmsr -.endm =20 .macro RESTORE_HOST_SPEC_CTRL /* No need to do anything if SPEC_CTRL is unset or V_SPEC_CTRL is set */ @@ -77,39 +48,6 @@ "jmp 900f", X86_FEATURE_MSR_SPEC_CTRL, \ "", X86_FEATURE_V_SPEC_CTRL .endm -.macro RESTORE_HOST_SPEC_CTRL_BODY guest_spec_ctrl:req, enter_flags:req, l= abel:req - /* Same for after vmexit. */ - mov $MSR_IA32_SPEC_CTRL, %ecx - - /* - * Load the value that the guest had written into MSR_IA32_SPEC_CTRL, - * if it was not intercepted during guest execution. - */ - testl $KVM_ENTER_SAVE_SPEC_CTRL, \enter_flags - jnz 998f - rdmsr - movl %eax, \guest_spec_ctrl - movl %edx, 4 + \guest_spec_ctrl -998: - /* Now restore the host value of the MSR if different from the guest's. = */ -#ifdef CONFIG_X86_64 - mov PER_CPU_VAR(x86_spec_ctrl_current), %rdx - cmp \guest_spec_ctrl, %rdx - je \label - movl %edx, %eax - shr $32, %rdx -#else - mov PER_CPU_VAR(x86_spec_ctrl_current), %eax - mov \guest_spec_ctrl, %esi - xor %eax, %esi - mov PER_CPU_VAR(x86_spec_ctrl_current + 4), %edx - mov 4 + \guest_spec_ctrl, %edi - xor %edx, %edi - or %edi, %esi - je \label -#endif - wrmsr -.endm =20 #define SVM_CLEAR_CPU_BUFFERS \ ALTERNATIVE "", __CLEAR_CPU_BUFFERS, X86_FEATURE_CLEAR_CPU_BUF_VM diff --git a/arch/x86/kvm/vmenter.h b/arch/x86/kvm/vmenter.h index 29cdae650069..e746e1328d3f 100644 --- a/arch/x86/kvm/vmenter.h +++ b/arch/x86/kvm/vmenter.h @@ -6,4 +6,70 @@ #define KVM_ENTER_SAVE_SPEC_CTRL BIT(1) #define KVM_ENTER_CLEAR_CPU_BUFFERS_FOR_MMIO BIT(2) =20 -#endif /* __KVM_X86_VMENTER_H */ +#ifdef __ASSEMBLER__ +.macro RESTORE_GUEST_SPEC_CTRL_BODY guest_spec_ctrl:req, label:req + /* + * SPEC_CTRL handling: if the guest's SPEC_CTRL value differs from the + * host's, write the MSR. This is kept out-of-line so that the common + * case does not have to jump. + * + * IMPORTANT: To avoid RSB underflow attacks and any other nastiness, + * there must not be any returns or indirect branches between this code + * and vmentry. + */ +#ifdef CONFIG_X86_64 + mov \guest_spec_ctrl, %rdx + cmp PER_CPU_VAR(x86_spec_ctrl_current), %rdx + je \label + movl %edx, %eax + shr $32, %rdx +#else + mov \guest_spec_ctrl, %eax + mov PER_CPU_VAR(x86_spec_ctrl_current), %ecx + xor %eax, %ecx + mov 4 + \guest_spec_ctrl, %edx + mov PER_CPU_VAR(x86_spec_ctrl_current + 4), %esi + xor %edx, %esi + or %esi, %ecx + je \label +#endif + mov $MSR_IA32_SPEC_CTRL, %ecx + wrmsr +.endm + +.macro RESTORE_HOST_SPEC_CTRL_BODY guest_spec_ctrl:req, enter_flags:req, l= abel:req + /* Same for after vmexit. */ + mov $MSR_IA32_SPEC_CTRL, %ecx + + /* + * Load the value that the guest had written into MSR_IA32_SPEC_CTRL, + * if it was not intercepted during guest execution. + */ + testl $KVM_ENTER_SAVE_SPEC_CTRL, \enter_flags + jnz 998f + rdmsr + movl %eax, \guest_spec_ctrl + movl %edx, 4 + \guest_spec_ctrl +998: + /* Now restore the host value of the MSR if different from the guest's. = */ +#ifdef CONFIG_X86_64 + mov PER_CPU_VAR(x86_spec_ctrl_current), %rdx + cmp \guest_spec_ctrl, %rdx + je \label + movl %edx, %eax + shr $32, %rdx +#else + mov PER_CPU_VAR(x86_spec_ctrl_current), %eax + mov \guest_spec_ctrl, %esi + xor %eax, %esi + mov PER_CPU_VAR(x86_spec_ctrl_current + 4), %edx + mov 4 + \guest_spec_ctrl, %edi + xor %edx, %edi + or %edi, %esi + je \label +#endif + wrmsr +.endm + +#endif /* __ASSEMBLER__ */ +#endif /* __KVM_X86_ENTER_FLAGS_H */ --=20 2.52.0 From nobody Mon Jun 15 07:34:28 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3F3053D9DD3 for ; Wed, 8 Apr 2026 18:22:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775672548; cv=none; b=AP5QKTRpMcjKOzckza/H0fwX95FCvHCXRpSsMUz7UGcRW3b1bV1+Us91BdpdunD6RMjJv35Y4nPDAyCuPn+DFZaaeE/+SGFuHeO24Iz7ZvGv80VtwDmXTo6iUdljYzju41PzveaLCVtJK+Ph93u5wSspNMAIdcxOC8QHAl2UCoE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775672548; c=relaxed/simple; bh=sxVvvF7VVW9sDO+Z1b5JFHKhHYt0RWjuEL+Xbt386Z4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=jnk3E1GsV912WIIWC7PLD5eJmujO0fVx4dra9BCN4B7HO4bKl4WtzgJ3lz8YjRwwwSJKZcQd+WRo1WW+YzinbwpSahqSxEUWto6X4rdz6jhh/WQQK9FZ5soczyONQwU20U1XCCxn5O92wDY8VpkSFV2eR/PnX4qEhszNRDSaXZ0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=X7UIX0Iw; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="X7UIX0Iw" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1775672546; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MNO0/3n9B+Qzk4y+UjdgKHduanoKZMiB6XbuLsFUjGE=; b=X7UIX0IwhgaQ0KT5ZvA7+2ISeEIXHIwzoWw4gioZPTuPiPUINmYyl2LWOmu5erGG9Y2ud0 1QLvbBl9gGYJkPi616SEgUGV4IxlrwuGyGHL3hQTcpwkiKxAOu1cSb/Ah+TCTem3yh/vUd MuC1No71EbPjarbh+9vor7oX2uC6dYk= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-153-c1w--ndnNoKa66J4vo2v3w-1; Wed, 08 Apr 2026 14:22:23 -0400 X-MC-Unique: c1w--ndnNoKa66J4vo2v3w-1 X-Mimecast-MFC-AGG-ID: c1w--ndnNoKa66J4vo2v3w_1775672539 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id B497319560B0; Wed, 8 Apr 2026 18:22:19 +0000 (UTC) Received: from virtlab1023.lab.eng.rdu2.redhat.lab.eng.rdu2.redhat.com (virtlab1023.lab.eng.rdu2.redhat.com [10.8.1.187]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 234F819560A6; Wed, 8 Apr 2026 18:22:19 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: seanjc@google.com, chang.seok.bae@intel.com Subject: [PATCH 6/7] KVM: VMX: switch to RESTORE_GUEST_SPEC_CTRL_BODY Date: Wed, 8 Apr 2026 14:22:12 -0400 Message-ID: <20260408182213.522713-7-pbonzini@redhat.com> In-Reply-To: <20260408182213.522713-1-pbonzini@redhat.com> References: <20260408182213.522713-1-pbonzini@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 Content-Type: text/plain; charset="utf-8" This has exactly the same expansion, so there is no change. Signed-off-by: Paolo Bonzini --- arch/x86/kvm/vmx/vmenter.S | 31 ++++--------------------------- 1 file changed, 4 insertions(+), 27 deletions(-) diff --git a/arch/x86/kvm/vmx/vmenter.S b/arch/x86/kvm/vmx/vmenter.S index bed5a753166b..f3240a3ff8f5 100644 --- a/arch/x86/kvm/vmx/vmenter.S +++ b/arch/x86/kvm/vmx/vmenter.S @@ -101,35 +101,12 @@ SYM_FUNC_START(__vmx_vcpu_run) /* Reload @vmx, _ASM_ARG1 may be modified by vmx_update_host_rsp(). */ mov WORD_SIZE(%_ASM_SP), %_ASM_DI =20 - ALTERNATIVE "jmp .Lspec_ctrl_done", "", X86_FEATURE_MSR_SPEC_CTRL - /* - * SPEC_CTRL handling: if the guest's SPEC_CTRL value differs from the - * host's, write the MSR. - * - * IMPORTANT: To avoid RSB underflow attacks and any other nastiness, - * there must not be any returns or indirect branches between this code - * and vmentry. + * Unlike AMD there's no V_SPEC_CTRL here, so do not leave the body + * out of line. Clobbers RAX, RCX, RDX, RSI. */ -#ifdef CONFIG_X86_64 - mov VMX_spec_ctrl(%rdi), %rdx - cmp PER_CPU_VAR(x86_spec_ctrl_current), %rdx - je .Lspec_ctrl_done - movl %edx, %eax - shr $32, %rdx -#else - mov VMX_spec_ctrl(%edi), %eax - mov PER_CPU_VAR(x86_spec_ctrl_current), %ecx - xor %eax, %ecx - mov VMX_spec_ctrl + 4(%edi), %edx - mov PER_CPU_VAR(x86_spec_ctrl_current + 4), %esi - xor %edx, %esi - or %esi, %ecx - je .Lspec_ctrl_done -#endif - mov $MSR_IA32_SPEC_CTRL, %ecx - wrmsr - + ALTERNATIVE "jmp .Lspec_ctrl_done", "", X86_FEATURE_MSR_SPEC_CTRL + RESTORE_GUEST_SPEC_CTRL_BODY VMX_spec_ctrl(%_ASM_DI), .Lspec_ctrl_done .Lspec_ctrl_done: =20 /* --=20 2.52.0 From nobody Mon Jun 15 07:34:28 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A910E3D9DBE for ; Wed, 8 Apr 2026 18:22:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775672548; cv=none; b=jyu7yToEfXlTdh3va3B4CU2F8VG/mMZdteCXWXEwvk0sQzp4j0+bTfme+SSybvm7CYRIFd33hjEkacwuyxB4H2fFwW64rzEV4Os7P9/Zw5L4GUJBukTcUNngucs9RiJzdXHklj9KpV0QWIhdtmFscgAWQ794/Yc1ab/lKYjQ7uo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775672548; c=relaxed/simple; bh=VjZ55oAqC42wXWGnGBHq1RQ1LIgCSsaaM7DWAn/XvmI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=hhQaA4YTdaVwUyvzteVdHm689077es3y///FNWh/EnPe66bDdBKEQ0qdMrZRbFtLkh4ViwWLUd2hZgYGRwwMDm6757ed2qL1+MPcleE9zMY5SVy8lphiVswkY2W2Stf4vYX26bODLsTptQOGJ1U3f5Zq2gkcimXXwCAlKsa1mpw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=anTBxOuB; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="anTBxOuB" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1775672545; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=YBs0Xwd63Gm3Mbj4g9Q5otR64g/nA9BtUBYKmO4Lh2s=; b=anTBxOuBBA0b70aLVLiyWplDcIS9tqZNq3QM6bUNY3mWUiOEq1Br7xoyxwv1Uv6depRObh A6ybfpN62U8XElssiCzDO6JP0qo5/bEcoWWSUZLmXWk5dpkfbB+5bCRi/RspHOKDAUZgq7 i8GL40ZVrG+93Lba7fQvvl2h65neER8= Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-135-5P5W8TevOgGn84fv3KjWqw-1; Wed, 08 Apr 2026 14:22:22 -0400 X-MC-Unique: 5P5W8TevOgGn84fv3KjWqw-1 X-Mimecast-MFC-AGG-ID: 5P5W8TevOgGn84fv3KjWqw_1775672540 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 6AF9D1956053; Wed, 8 Apr 2026 18:22:20 +0000 (UTC) Received: from virtlab1023.lab.eng.rdu2.redhat.lab.eng.rdu2.redhat.com (virtlab1023.lab.eng.rdu2.redhat.com [10.8.1.187]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id D82751955D6B; Wed, 8 Apr 2026 18:22:19 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: seanjc@google.com, chang.seok.bae@intel.com Subject: [PATCH 7/7] KVM: VMX: replace vmx_spec_ctrl_restore_host with RESTORE_HOST_SPEC_CTRL_BODY Date: Wed, 8 Apr 2026 14:22:13 -0400 Message-ID: <20260408182213.522713-8-pbonzini@redhat.com> In-Reply-To: <20260408182213.522713-1-pbonzini@redhat.com> References: <20260408182213.522713-1-pbonzini@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 Content-Type: text/plain; charset="utf-8" Reuse the same assembly as SVM, just with alternatives instead of cpu_feature_enabled(X86_FEATURE_KERNEL_IBRS). Due to the dearth of registers in 32-bit x86, save/restore %eax across the restoring sequence, keeping it simple; the main alternative would be to leave vmx and flags on the stack, avoiding the need to read flags from %ebx. It's not really measurable difference. Signed-off-by: Paolo Bonzini --- arch/x86/kvm/vmenter.h | 4 ++-- arch/x86/kvm/vmx/vmenter.S | 25 +++++++++++++++---------- arch/x86/kvm/vmx/vmx.c | 25 ------------------------- arch/x86/kvm/vmx/vmx.h | 1 - 4 files changed, 17 insertions(+), 38 deletions(-) diff --git a/arch/x86/kvm/vmenter.h b/arch/x86/kvm/vmenter.h index e746e1328d3f..d822dafc75f7 100644 --- a/arch/x86/kvm/vmenter.h +++ b/arch/x86/kvm/vmenter.h @@ -55,7 +55,7 @@ #ifdef CONFIG_X86_64 mov PER_CPU_VAR(x86_spec_ctrl_current), %rdx cmp \guest_spec_ctrl, %rdx - je \label + ALTERNATIVE "", __stringify(je \label), X86_FEATURE_KERNEL_IBRS movl %edx, %eax shr $32, %rdx #else @@ -66,7 +66,7 @@ mov 4 + \guest_spec_ctrl, %edi xor %edx, %edi or %edi, %esi - je \label + ALTERNATIVE "", __stringify(je \label), X86_FEATURE_KERNEL_IBRS #endif wrmsr .endm diff --git a/arch/x86/kvm/vmx/vmenter.S b/arch/x86/kvm/vmx/vmenter.S index f3240a3ff8f5..c0574d68ff45 100644 --- a/arch/x86/kvm/vmx/vmenter.S +++ b/arch/x86/kvm/vmx/vmenter.S @@ -105,9 +105,9 @@ SYM_FUNC_START(__vmx_vcpu_run) * Unlike AMD there's no V_SPEC_CTRL here, so do not leave the body * out of line. Clobbers RAX, RCX, RDX, RSI. */ - ALTERNATIVE "jmp .Lspec_ctrl_done", "", X86_FEATURE_MSR_SPEC_CTRL - RESTORE_GUEST_SPEC_CTRL_BODY VMX_spec_ctrl(%_ASM_DI), .Lspec_ctrl_done -.Lspec_ctrl_done: + ALTERNATIVE "jmp .Lspec_ctrl_guest_done", "", X86_FEATURE_MSR_SPEC_CTRL + RESTORE_GUEST_SPEC_CTRL_BODY VMX_spec_ctrl(%_ASM_DI), .Lspec_ctrl_guest_d= one +.Lspec_ctrl_guest_done: =20 /* * Since vmentry is serializing on affected CPUs, there's no need for @@ -252,16 +252,21 @@ SYM_INNER_LABEL_ALIGN(vmx_vmexit, SYM_L_GLOBAL) FILL_RETURN_BUFFER %_ASM_CX, RSB_CLEAR_LOOPS, X86_FEATURE_RSB_VMEXIT,\ X86_FEATURE_RSB_VMEXIT_LITE =20 - pop %_ASM_ARG2 /* @flags */ - pop %_ASM_ARG1 /* @vmx */ - - call vmx_spec_ctrl_restore_host - - CLEAR_BRANCH_HISTORY_VMEXIT - /* Put return value in AX */ mov %_ASM_BX, %_ASM_AX =20 + pop %_ASM_BX /* @flags */ + pop %_ASM_DI /* @vmx */ + + /* Clobbers RAX, RCX, RDX, RSI. */ + ALTERNATIVE "jmp .Lspec_ctrl_host_done", "", X86_FEATURE_MSR_SPEC_CTRL + push %_ASM_AX + RESTORE_HOST_SPEC_CTRL_BODY VMX_spec_ctrl(%_ASM_DI), %ebx, .Lspec_ctrl_ho= st_done + pop %_ASM_AX +.Lspec_ctrl_host_done: + + CLEAR_BRANCH_HISTORY_VMEXIT + pop %_ASM_BX #ifdef CONFIG_X86_64 pop %r12 diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index ef0de3cc0bc2..84a5441142fc 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -7497,31 +7497,6 @@ void noinstr vmx_update_host_rsp(struct vcpu_vmx *vm= x, unsigned long host_rsp) } } =20 -void noinstr vmx_spec_ctrl_restore_host(struct vcpu_vmx *vmx, - unsigned int flags) -{ - u64 hostval =3D this_cpu_read(x86_spec_ctrl_current); - - if (!cpu_feature_enabled(X86_FEATURE_MSR_SPEC_CTRL)) - return; - - if (flags & KVM_ENTER_SAVE_SPEC_CTRL) - vmx->spec_ctrl =3D native_rdmsrq(MSR_IA32_SPEC_CTRL); - - /* - * If the guest/host SPEC_CTRL values differ, restore the host value. - * - * For legacy IBRS, the IBRS bit always needs to be written after - * transitioning from a less privileged predictor mode, regardless of - * whether the guest/host values differ. - */ - if (cpu_feature_enabled(X86_FEATURE_KERNEL_IBRS) || - vmx->spec_ctrl !=3D hostval) - native_wrmsrq(MSR_IA32_SPEC_CTRL, hostval); - - barrier_nospec(); -} - static fastpath_t vmx_exit_handlers_fastpath(struct kvm_vcpu *vcpu, bool force_immediate_exit) { diff --git a/arch/x86/kvm/vmx/vmx.h b/arch/x86/kvm/vmx/vmx.h index 9cb5a205f19d..fec85c739fad 100644 --- a/arch/x86/kvm/vmx/vmx.h +++ b/arch/x86/kvm/vmx/vmx.h @@ -370,7 +370,6 @@ void vmx_set_virtual_apic_mode(struct kvm_vcpu *vcpu); struct vmx_uret_msr *vmx_find_uret_msr(struct vcpu_vmx *vmx, u32 msr); void pt_update_intercept_for_msr(struct kvm_vcpu *vcpu); void vmx_update_host_rsp(struct vcpu_vmx *vmx, unsigned long host_rsp); -void vmx_spec_ctrl_restore_host(struct vcpu_vmx *vmx, unsigned int flags); unsigned int __vmx_vcpu_enter_flags(struct vcpu_vmx *vmx); bool __vmx_vcpu_run(struct vcpu_vmx *vmx, unsigned int flags); void vmx_ept_load_pdptrs(struct kvm_vcpu *vcpu); --=20 2.52.0