From nobody Mon Feb 9 12:12:42 2026 Received: from out-182.mta0.migadu.com (out-182.mta0.migadu.com [91.218.175.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 81220200B85 for ; Fri, 21 Feb 2025 16:34:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=91.218.175.182 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740155651; cv=none; b=SGPN41Uqyp+lQvVHqAd1yo8f7DkbiG/VOv6/DSniKWQRvHLCg7PsRxsuwkUrPEj6Y8C1aHxyzKx7vM0Wk9rBWQqCHDDu7v6TDTndUtpSM+SyKKXY+LTiOGNFYHdW3ur5yNQcc10rUOhwGtIvUIOZcESa0japQeEJIh9lF4COH0E= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740155651; c=relaxed/simple; bh=LiLn0i9Td7SClEMbhcgHHlr9u1P2brpGX9PYN01puA4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=BiVFvXvRZJ8hoHtpo0zDMs/tq5aOwo9TOCZ3cpRGiwhEZTYoPXPLf3qk0SssIsMr9HpeoiYhX0ZC5IxCP6X7q5n6s3Dr+apmoxAT7PGKIRHmOjek3FWTBPS3v5G4d311Oz11+UIZrgexAEIccOEwTzjRv76P5Ar/fr20BVVpLEg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=FMvuTohp; arc=none smtp.client-ip=91.218.175.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="FMvuTohp" X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1740155647; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=nQc/HabnpnH39zEM1pbLYGL57sbgP/YZ4LZE9qk3CJM=; b=FMvuTohpaxe+k1t6UCOb4B9HgZInrJ5KvveooB2AZc9ii1bmM4KjYZxmWuevwfEEpDfCA9 OHZYKqICITLZOULcpgUxviQKDBONTyiRko6mrCNxZ29WCWjTmpo/g1LENmUoCm+r2ugZ4u Yk5DjawysYP1rBnlAxZb1BGQBnextKE= From: Yosry Ahmed To: x86@kernel.org, Sean Christopherson Cc: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Paolo Bonzini , Jim Mattson , "Kaplan, David" , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Yosry Ahmed Subject: [PATCH 1/3] x86/cpufeatures: Define X86_FEATURE_AMD_IBRS_SAME_MODE Date: Fri, 21 Feb 2025 16:33:50 +0000 Message-ID: <20250221163352.3818347-2-yosry.ahmed@linux.dev> In-Reply-To: <20250221163352.3818347-1-yosry.ahmed@linux.dev> References: <20250221163352.3818347-1-yosry.ahmed@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Migadu-Flow: FLOW_OUT Content-Type: text/plain; charset="utf-8" Per the APM [1]: Some processors, identified by CPUID Fn8000_0008_EBX[IbrsSameMode] (bit 19) =3D 1, provide additional speculation limits. For these processors, when IBRS is set, indirect branch predictions are not influenced by any prior indirect branches, regardless of mode (CPL and guest/host) and regardless of whether the prior indirect branches occurred before or after the setting of IBRS. This is referred to as Same Mode IBRS. Define this feature bit, which will be used by KVM to determine if an IBPB is required on nested VM-exits in SVM. [1] AMD64 Architecture Programmer's Manual Pub. 40332, Rev 4.08 - April 2024, Volume 2, 3.2.9 Speculation Control MSRs Signed-off-by: Yosry Ahmed --- arch/x86/include/asm/cpufeatures.h | 1 + tools/arch/x86/include/asm/cpufeatures.h | 1 + 2 files changed, 2 insertions(+) diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpuf= eatures.h index 645aa360628da..46af88357ac89 100644 --- a/arch/x86/include/asm/cpufeatures.h +++ b/arch/x86/include/asm/cpufeatures.h @@ -344,6 +344,7 @@ #define X86_FEATURE_AMD_IBRS (13*32+14) /* Indirect Branch Restricted Spe= culation */ #define X86_FEATURE_AMD_STIBP (13*32+15) /* Single Thread Indirect Branch= Predictors */ #define X86_FEATURE_AMD_STIBP_ALWAYS_ON (13*32+17) /* Single Thread Indire= ct Branch Predictors always-on preferred */ +#define X86_FEATURE_AMD_IBRS_SAME_MODE (13*32+19) /* Indirect Branch Restr= icted Speculation same mode protection*/ #define X86_FEATURE_AMD_PPIN (13*32+23) /* "amd_ppin" Protected Processor= Inventory Number */ #define X86_FEATURE_AMD_SSBD (13*32+24) /* Speculative Store Bypass Disab= le */ #define X86_FEATURE_VIRT_SSBD (13*32+25) /* "virt_ssbd" Virtualized Specu= lative Store Bypass Disable */ diff --git a/tools/arch/x86/include/asm/cpufeatures.h b/tools/arch/x86/incl= ude/asm/cpufeatures.h index 17b6590748c00..c99d626357c1e 100644 --- a/tools/arch/x86/include/asm/cpufeatures.h +++ b/tools/arch/x86/include/asm/cpufeatures.h @@ -344,6 +344,7 @@ #define X86_FEATURE_AMD_IBRS (13*32+14) /* Indirect Branch Restricted Spe= culation */ #define X86_FEATURE_AMD_STIBP (13*32+15) /* Single Thread Indirect Branch= Predictors */ #define X86_FEATURE_AMD_STIBP_ALWAYS_ON (13*32+17) /* Single Thread Indire= ct Branch Predictors always-on preferred */ +#define X86_FEATURE_AMD_IBRS_SAME_MODE (13*32+19) /* Indirect Branch Restr= icted Speculation same mode protection*/ #define X86_FEATURE_AMD_PPIN (13*32+23) /* "amd_ppin" Protected Processor= Inventory Number */ #define X86_FEATURE_AMD_SSBD (13*32+24) /* Speculative Store Bypass Disab= le */ #define X86_FEATURE_VIRT_SSBD (13*32+25) /* "virt_ssbd" Virtualized Specu= lative Store Bypass Disable */ --=20 2.48.1.601.g30ceb7b040-goog From nobody Mon Feb 9 12:12:42 2026 Received: from out-188.mta0.migadu.com (out-188.mta0.migadu.com [91.218.175.188]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5FE6E2101A0 for ; Fri, 21 Feb 2025 16:34:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=91.218.175.188 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740155654; cv=none; b=qNdf0pO+5FDvlLBAsuBviriZKDCQf9R4k8AcwfJtTMUzEDJqiAqE4tQrFJer8ZDg8uygPH6bJz24BCyn1it5CEMs+bLU1ydQBBJUfqOhlFjdzGxdbXqluXhKmNMnbWGVifTtk5dHrYL9f+Gd8bxHr72XWgDPH8CTqLT0TGc4G0g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740155654; c=relaxed/simple; bh=YXp7VZBdS3IyMkWci8sv6ipPQLiGOAkk2P8v6sqOrS8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=V++Sjz5305Ak0IjwrMl+XW1F+dRlCiFocVUIwObW+9H3dnN2Wp+v5ZOu3vQfy2MooC/KDJ1qAx82lmbSNlOtf0/BXleO5JYrSzm7eN5gWwUGQgWqkRghm4pKHcw+xJjzgu/Rcm+MvSxDdngbop8EsI14wYMWVKB6vUlwjmH6A8E= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=k/jmITgm; arc=none smtp.client-ip=91.218.175.188 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="k/jmITgm" X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1740155650; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=kJ4r4UkNHVRp0o1mave86XFAreLgoybFLmy+ftaGobA=; b=k/jmITgmEj30FvHYDBUb5/TklLz/pGUrveHxVGS2lN6Z+2AlvJUDKMs3YDAJl9fgPkfjCM GDc2jvzwKMiFOT4fbr6dm7rgaztqF3H4HnsOFW4uLmvIYe61rEuvUSHh2uFwbuKKb1p8MI 8qOYjO05xoqHB5vvfp3dGsn4fvK5uaU= From: Yosry Ahmed To: x86@kernel.org, Sean Christopherson Cc: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Paolo Bonzini , Jim Mattson , "Kaplan, David" , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Yosry Ahmed Subject: [PATCH 2/3] KVM: x86: Propagate AMD's IbrsSameMode to the guest Date: Fri, 21 Feb 2025 16:33:51 +0000 Message-ID: <20250221163352.3818347-3-yosry.ahmed@linux.dev> In-Reply-To: <20250221163352.3818347-1-yosry.ahmed@linux.dev> References: <20250221163352.3818347-1-yosry.ahmed@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Migadu-Flow: FLOW_OUT Content-Type: text/plain; charset="utf-8" If IBRS provides same mode (kernel/user or host/guest) protection on the host, then by definition it also provides same mode protection in the guest. In fact, all different modes from the guest's perspective are the same mode from the host's perspective anyway. Propagate IbrsSameMode to the guests. Signed-off-by: Yosry Ahmed --- arch/x86/kvm/cpuid.c | 1 + 1 file changed, 1 insertion(+) diff --git a/arch/x86/kvm/cpuid.c b/arch/x86/kvm/cpuid.c index edef30359c198..05d7bbfbb8885 100644 --- a/arch/x86/kvm/cpuid.c +++ b/arch/x86/kvm/cpuid.c @@ -1103,6 +1103,7 @@ void kvm_set_cpu_caps(void) F(AMD_SSB_NO), F(AMD_STIBP), F(AMD_STIBP_ALWAYS_ON), + F(AMD_IBRS_SAME_MODE), F(AMD_PSFD), F(AMD_IBPB_RET), ); --=20 2.48.1.601.g30ceb7b040-goog From nobody Mon Feb 9 12:12:42 2026 Received: from out-172.mta0.migadu.com (out-172.mta0.migadu.com [91.218.175.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 50D5621129A for ; Fri, 21 Feb 2025 16:34:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=91.218.175.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740155657; cv=none; b=Nmv8Nu/YXaQWG0Kw1dilXNJYQ2o81REgABG6eDO1YiKhJI3a6KprSPzxoxWQdbyeGb8d2KycqkoTk+nYrFfNkK8+YVePaHhEG3eNAr2kVj5tRuYNMqQH+cvveMuSb95zglTDcmtBJngW1V8LsLPSh8oITTCsmagLUQrr2wUd3ck= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740155657; c=relaxed/simple; bh=3/HKHruvaE4GIGU5SloCVdMtixlMfnzkYTXdJPXjiIg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=BPZnNL669vjMDkFUDTbQusIQ/K/LsQ10UXaTe9g1TAFxBi20/3b3XDMBqQioqARisTTcFSbrswNrC1JZCB5nNUv/LWLMDjoSrAal32QYVaA5RcD8Xon7IFsy60wLlwAhHbOYMYACvZ/b0foAIHVK5VZFsJ3lOBBWt2HA2psobA8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=cnyBWkd+; arc=none smtp.client-ip=91.218.175.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="cnyBWkd+" X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1740155653; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=CpOlmsaxwBv8z/TyEJM+sPdDJ8NinYorBdTwh3llrPI=; b=cnyBWkd+RQDE5nc/sQQO+335uBwxpzqlx53qVy8jVDPl26qTLeZVms7/1hr5au5+CNfQMN du2P48WUigiLhedBhJ+IMdgdJWXnv5jUAEpR7KouRl9P+JubQHKwXfuWROLzRdE2074NgU MyVv7i1KPUY41oexgaVL4i9Up9cThNg= From: Yosry Ahmed To: x86@kernel.org, Sean Christopherson Cc: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Paolo Bonzini , Jim Mattson , "Kaplan, David" , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Yosry Ahmed Subject: [PATCH 3/3] KVM: x86: Generalize IBRS virtualization on emulated VM-exit Date: Fri, 21 Feb 2025 16:33:52 +0000 Message-ID: <20250221163352.3818347-4-yosry.ahmed@linux.dev> In-Reply-To: <20250221163352.3818347-1-yosry.ahmed@linux.dev> References: <20250221163352.3818347-1-yosry.ahmed@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Migadu-Flow: FLOW_OUT Content-Type: text/plain; charset="utf-8" Commit 2e7eab81425a ("KVM: VMX: Execute IBPB on emulated VM-exit when guest has IBRS") added an IBPB in the emulated VM-exit path on Intel to properly virtualize IBRS by providing separate predictor modes for L1 and L2. AMD requires similar handling, except when IbrsSameMode is enumerated by the host CPU (which is the case on most/all AMD CPUs). With IbrsSameMode, hardware IBRS is sufficient and no extra handling is needed from KVM. Generalize the handling in nested_vmx_vmexit() by moving it into a generic function, add the AMD handling, and use it in nested_svm_vmexit() too. The main reason for using a generic function is to have a single place to park the huge comment about virtualizing IBRS. Signed-off-by: Yosry Ahmed Reviewed-by: Jim Mattson --- arch/x86/kvm/svm/nested.c | 2 ++ arch/x86/kvm/vmx/nested.c | 11 +---------- arch/x86/kvm/x86.h | 18 ++++++++++++++++++ 3 files changed, 21 insertions(+), 10 deletions(-) diff --git a/arch/x86/kvm/svm/nested.c b/arch/x86/kvm/svm/nested.c index d77b094d9a4d6..61b73ff30807e 100644 --- a/arch/x86/kvm/svm/nested.c +++ b/arch/x86/kvm/svm/nested.c @@ -1041,6 +1041,8 @@ int nested_svm_vmexit(struct vcpu_svm *svm) =20 nested_svm_copy_common_state(svm->nested.vmcb02.ptr, svm->vmcb01.ptr); =20 + kvm_nested_vmexit_handle_spec_ctrl(vcpu); + svm_switch_vmcb(svm, &svm->vmcb01); =20 /* diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c index 8a7af02d466e9..453d52a6e836a 100644 --- a/arch/x86/kvm/vmx/nested.c +++ b/arch/x86/kvm/vmx/nested.c @@ -5018,16 +5018,7 @@ void nested_vmx_vmexit(struct kvm_vcpu *vcpu, u32 vm= _exit_reason, =20 vmx_switch_vmcs(vcpu, &vmx->vmcs01); =20 - /* - * If IBRS is advertised to the vCPU, KVM must flush the indirect - * branch predictors when transitioning from L2 to L1, as L1 expects - * hardware (KVM in this case) to provide separate predictor modes. - * Bare metal isolates VMX root (host) from VMX non-root (guest), but - * doesn't isolate different VMCSs, i.e. in this case, doesn't provide - * separate modes for L2 vs L1. - */ - if (guest_cpu_cap_has(vcpu, X86_FEATURE_SPEC_CTRL)) - indirect_branch_prediction_barrier(); + kvm_nested_vmexit_handle_spec_ctrl(vcpu); =20 /* Update any VMCS fields that might have changed while L2 ran */ vmcs_write32(VM_EXIT_MSR_LOAD_COUNT, vmx->msr_autoload.host.nr); diff --git a/arch/x86/kvm/x86.h b/arch/x86/kvm/x86.h index 7a87c5fc57f1b..008c8d381c253 100644 --- a/arch/x86/kvm/x86.h +++ b/arch/x86/kvm/x86.h @@ -116,6 +116,24 @@ static inline void kvm_leave_nested(struct kvm_vcpu *v= cpu) kvm_x86_ops.nested_ops->leave_nested(vcpu); } =20 +/* + * If IBRS is advertised to the vCPU, KVM must flush the indirect branch + * predictors when transitioning from L2 to L1, as L1 expects hardware (KV= M in + * this case) to provide separate predictor modes. Bare metal isolates th= e host + * from the guest, but doesn't isolate different guests from one another (= in + * this case L1 and L2). The exception is if bare metal supports same mode= IBRS, + * which offers protection within the same mode, and hence protects L1 fro= m L2. + */ +static inline void kvm_nested_vmexit_handle_spec_ctrl(struct kvm_vcpu *vcp= u) +{ + if (cpu_feature_enabled(X86_FEATURE_AMD_IBRS_SAME_MODE)) + return; + + if (guest_cpu_cap_has(vcpu, X86_FEATURE_SPEC_CTRL) || + guest_cpu_cap_has(vcpu, X86_FEATURE_AMD_IBRS)) + indirect_branch_prediction_barrier(); +} + static inline bool kvm_vcpu_has_run(struct kvm_vcpu *vcpu) { return vcpu->arch.last_vmentry_cpu !=3D -1; --=20 2.48.1.601.g30ceb7b040-goog