From nobody Sat Feb 7 19:45:54 2026 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DA78A226CFD for ; Wed, 28 Jan 2026 01:34:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769564079; cv=none; b=mx7ndf0Qt1afPvmQKhFWJWIu+uZBW8ThWjMsha9MMAf6icCHkE4Zv7VCcQwVd3iYFZGl3G+vfTYQBehzjS9kR7A20SxW3/7eUU/2wzv2dpCSH95p7V1k8r13smey09a7UL2/lEL+8pJoLNqr8LpKoVAdS4/0EJMr4RW5Q0kRenk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769564079; c=relaxed/simple; bh=80qQLVQ5jJzLtpnLzVFm9M9H2UbAiPmAYz+2rzEoNUo=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=qcfrU3tEMmX0Qg6EfDuvpWra/peC1I4pvcSmgaSvdAIquL7oQCbKhbB80k28EXxw/V0pzpxv6KLYDZZVVFAaIh4hSn8h9RDScDMAeE0v17QXiNL81uLzhVL1CC+rlg54O5LSJN3GyNW6Utx8UX0R+oCvLm3EJmqHCq78kaZ/v2c= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=UAcSfLNG; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="UAcSfLNG" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-34ea5074935so5233360a91.0 for ; Tue, 27 Jan 2026 17:34:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1769564077; x=1770168877; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=BUVUKuSQV+U+8gsuswu6Brp9Cg3f+EwR6/8UYrViixw=; b=UAcSfLNGiwZhcqxv0RkRsZvSz/t3tvroGAMaBprXVOWTk0qhO0qnAhRy47ygpf6k3m HpzAJdD5kLyN9Gr/Wovwg/6j8p4qUhZjcDRkRzPj+J3G/0WJGg99anneF6lVx5i8wKZp pLF9BEEAi8LqO7wsN7f4HHij0nnc33MA+SdeQU7vuUZPOhykIv6zRaEWp0UPZV8eiteG /XHQYIwrdwis/QQDJop3WI3bwx/vO8BW6ikjH08cekyPJ5uWjfGrRS6wJFLn4In1zOc/ oXoHB+U/VYp2bsUjQG0+QVs5iJmN43ZaNa/EFs+KDHPc/Ir+O1xngDkrvM15QKvbmdZL r3cA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1769564077; x=1770168877; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=BUVUKuSQV+U+8gsuswu6Brp9Cg3f+EwR6/8UYrViixw=; b=KMCpzaqUUYeZzkiIj2D3TNDZDlVEsEgjzBhwmbQd3lzu2VgieTvBx4b+/IgwU3vjid 10qybwUmdcksLB+3wKZTpKC3K1Ej5B9MwyH0dYA5d6s4AwSBpPpJV34EMx2/mjoecs4K mD099sLQqao9XGn5Ao7qYhBpb/v2hCZiE/QZEnP7+ZU5x56d0inlbfUXOvJJzWMk1esk u8GzL19M8oOPXaRBTYqwn7ZwoW8vteFchLaoWrOcp1c3EcOGnehdOwoxJjUzSB8pmK8b HLuZQcfSqv+ZIkqVUqNev1Fw+7jAuuh1YjsI9W271TLq3diFxWqJWTE44f/y+9TIBW7Y Zq4g== X-Forwarded-Encrypted: i=1; AJvYcCW9oLg+K/4rS6NfrygDHYnfV2MEuzpn8lK3ALjJDEznUNyaglV3vdoUVN9BOyCa6AtZzsWf93aDnYES9MU=@vger.kernel.org X-Gm-Message-State: AOJu0YxVNImwhyI7a/LZoDbOFQP/33Lna9KcNWeDExDIMssVmfOuOv7r nq7XceIcQj/fBO9dZ6K6JNMf4JGFl9SHfA9SWWZux8RdjaGn29aNp4n+taE38AA6jz0Ee/AlMqA qkq1FLA== X-Received: from pja13.prod.google.com ([2002:a17:90b:548d:b0:352:ba50:2819]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:270e:b0:34a:b459:bd10 with SMTP id 98e67ed59e1d1-353fed7104bmr3202540a91.24.1769564077205; Tue, 27 Jan 2026 17:34:37 -0800 (PST) Reply-To: Sean Christopherson Date: Tue, 27 Jan 2026 17:34:31 -0800 In-Reply-To: <20260128013432.3250805-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260128013432.3250805-1-seanjc@google.com> X-Mailer: git-send-email 2.52.0.457.g6b5491de43-goog Message-ID: <20260128013432.3250805-2-seanjc@google.com> Subject: [PATCH 1/2] KVM: x86: Defer IBPBs for vCPU and nested transitions until core run loop From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Yosry Ahmed , Jim Mattson , David Kaplan Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When emitting an Indirect Branch Prediction Barrier to isolate different guest security domains (different vCPUs or L1 vs. L2 in the same vCPU), defer the IBPB until VM-Enter is imminent to avoid redundant and/or unnecessary IBPBs. E.g. if a vCPU is loaded on a CPU without ever doing VM-Enter, then _KVM_ isn't responsible for doing an IBPB as KVM's job is purely to mitigate guests<=3D>guest attacks; guest=3D>host attacks are cove= red by IBRS. Cc: stable@vger.kernel.org Cc: Yosry Ahmed Cc: Jim Mattson Cc: David Kaplan Signed-off-by: Sean Christopherson --- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/x86.c | 7 ++++++- arch/x86/kvm/x86.h | 2 +- 3 files changed, 8 insertions(+), 2 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_hos= t.h index e441f270f354..76bbc80a2d1d 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -826,6 +826,7 @@ struct kvm_vcpu_arch { u64 smbase; u64 smi_count; bool at_instruction_boundary; + bool need_ibpb; bool tpr_access_reporting; bool xfd_no_write_intercept; u64 microcode_version; diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 8acfdfc583a1..e5ae655702b4 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -5187,7 +5187,7 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cp= u) * is handled on the nested VM-Exit path. */ if (static_branch_likely(&switch_vcpu_ibpb)) - indirect_branch_prediction_barrier(); + vcpu->arch.need_ibpb =3D true; per_cpu(last_vcpu, cpu) =3D vcpu; } =20 @@ -11315,6 +11315,11 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu) kvm_make_request(KVM_REQ_EVENT, vcpu); } =20 + if (unlikely(vcpu->arch.need_ibpb)) { + indirect_branch_prediction_barrier(); + vcpu->arch.need_ibpb =3D false; + } + fpregs_assert_state_consistent(); if (test_thread_flag(TIF_NEED_FPU_LOAD)) switch_fpu_return(); diff --git a/arch/x86/kvm/x86.h b/arch/x86/kvm/x86.h index 70e81f008030..6708142d051d 100644 --- a/arch/x86/kvm/x86.h +++ b/arch/x86/kvm/x86.h @@ -169,7 +169,7 @@ static inline void kvm_nested_vmexit_handle_ibrs(struct= kvm_vcpu *vcpu) =20 if (guest_cpu_cap_has(vcpu, X86_FEATURE_SPEC_CTRL) || guest_cpu_cap_has(vcpu, X86_FEATURE_AMD_IBRS)) - indirect_branch_prediction_barrier(); + vcpu->arch.need_ibpb =3D true; } =20 /* --=20 2.52.0.457.g6b5491de43-goog From nobody Sat Feb 7 19:45:54 2026 Received: from mail-pg1-f202.google.com (mail-pg1-f202.google.com [209.85.215.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9532622FDEA for ; Wed, 28 Jan 2026 01:34:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769564081; cv=none; b=T29Q5dNO0KrauR9FS9SwSTycabOKzVilArkHZ7pU49rbTq1tWg+sI68oyetSYFMAgs2nhWQND2qq9X6gw8txrEipUXBXW0EJknssNxr1GqVHH1Fe5YkQjJYgQiUT8LlSCfH591Eew6yDzu1VLt4nejDulAZXyOWgxDjor1k9Epo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769564081; c=relaxed/simple; bh=R13kez/KFVrfJYieeBKxhtp8/CpHkd4vcp7L1n+2ofo=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Fo33myXbsjfOuqfp3qILcFXrp+F+bMvOsgFR9Q8Z+uUtlPgNU8tSNRNeER3EP8TPLCvinHBCIzYuIQcWgrluaHQYk8FklLTEwTrXoV0JE7fsg2no5C+7fcJDjSZXScZ3CFTBTpRq7fsk6c4vpbX+OBkbv6iE6yHy952zaAFmH10= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=Kn1lalxB; arc=none smtp.client-ip=209.85.215.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="Kn1lalxB" Received: by mail-pg1-f202.google.com with SMTP id 41be03b00d2f7-c52d37d346fso278559a12.0 for ; Tue, 27 Jan 2026 17:34:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1769564079; x=1770168879; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=1VryWUi83hnS06/n+x/xjvRCvWKYQoklYztPIe61o0Y=; b=Kn1lalxBIeHpxC5M89NL6c9SRuRtCQnLaCPxwShwDwuhAaeosvogloLT9pd7DskWlk znTckx24+CT3bDLWZH50bBThpSI8h0uCYQK9M+R8nSYQRo4MZtdr1jkvPoNqn8FXMepu lNjrpS1btbxlKnelDW0TuE3Vt4mXDKnuQwBsId84p/uvWa0xfsNv86wskoWkSs4o9GJ+ ti2BqcV9EcrS8YagHN5UJsxDijl700i1pmo4OGg5yXljgDqXwJYzf1riZsrWFs0kKoOl Jvj5SWYIiKqpxunIeqaSHq9dO0nFUpmfCaFuvd1aYBaoH/mcx6c1JChpu59w9TLxALTh PlXQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1769564079; x=1770168879; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=1VryWUi83hnS06/n+x/xjvRCvWKYQoklYztPIe61o0Y=; b=m6MLkasV/HVYeAS+9LoRMLzhUVRkeef19exarC+KkJqWXeMLZkEJ7L4G8rT6RhblKm rRPvGwNaZ4DZA9yUJ7/Nq9JRZujd/LY6iXXZuS17Y+CC009n+f66+iToqsv4qJa7rnt1 kPe52wbyVOYWucJSE0BcEJFBT6BKEQby72p+EMYsbd2tsUZFu9mW8QNAGFUIYdYZutmC /+4Oa68XpTCEDQTmRR3ESReTM/3twlbjXTVCuLbiq10+HZ/BuW7DoKy9jSo273ROmNqt 7nC9jWWfxqOfDEpezTueKahbVuXjy0Dkxilaicu2Ucvr3Beeo0JgTZ+44Iz4fa85lPTl NMkQ== X-Forwarded-Encrypted: i=1; AJvYcCXY/qIym4H9m13mv7UG3dHljt/XXtHhAQdehaxr0cFtQ4ko1gPur/GiPnPvDzTTW+RQEP0GFC46Z8yjKZM=@vger.kernel.org X-Gm-Message-State: AOJu0YyyodbfKDRN08AjfvrJo9K9HtUgaErBBjFh9JyBX3lq5KWSAR6g va4KNauz/q0ldRHhZZbCdH50m/MdM8bbp/PQy0jgZna3Fpql6zvc27wxPBQfr+Ysopqsm8LSzXK a36leLA== X-Received: from pgct8.prod.google.com ([2002:a05:6a02:5288:b0:c61:277a:16af]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a21:497:b0:38e:6774:382d with SMTP id adf61e73a8af0-38ec5cf9c32mr3498880637.8.1769564078792; Tue, 27 Jan 2026 17:34:38 -0800 (PST) Reply-To: Sean Christopherson Date: Tue, 27 Jan 2026 17:34:32 -0800 In-Reply-To: <20260128013432.3250805-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260128013432.3250805-1-seanjc@google.com> X-Mailer: git-send-email 2.52.0.457.g6b5491de43-goog Message-ID: <20260128013432.3250805-3-seanjc@google.com> Subject: [PATCH 2/2] KVM: x86: Emit IBPB on pCPU migration if IBPB is advertised to guest From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Yosry Ahmed , Jim Mattson , David Kaplan Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Emit an Indirect Branch Prediction Barrier if a vCPU is migrated to a different pCPU and IBPB support is advertised to the guest, to ensure any IBPBs performed by the guest are effective across pCPUs. Ideally, KVM would only emit IBPB if the guest performed an IBPB since the vCPU last ran on the "new" pCPU, but pCPU migration is a relatively rare/slow path, and so the cost of tracking which pCPUs a vCPUs has run on, let alone intercepting PRED_CMD writes, outweighs the potential benefits of avoiding IBPBs on pCPU migration. E.g. if a single vCPU is bouncing between pCPUs A and B, and the guest is doing IBPBs on context switches to mitigate cross-task attacks, then the following scenario can occur and needs to be mitigated by KVM: 1. vCPU starts on pCPU A. It runs a userspace task (task #1) which installs various branch predictions into pCPU A's BTB. 2. The vCPU is migrated to pCPU B. 3. The guest switches to userspace task #2 and emits an IBPB, on pCPU B. 4. The vCPU is migrated back to pCPU A. Userspace task (task #2) in the guest now consumes the potentially dangerous branch predictions installed in step 1 from task #1. Reported-by: David Kaplan Cc: stable@vger.kernel.org Signed-off-by: Sean Christopherson --- arch/x86/kvm/x86.c | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index e5ae655702b4..9d1641c2d83c 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -5201,6 +5201,19 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int c= pu) kvm_make_request(KVM_REQ_CLOCK_UPDATE, vcpu); } =20 + /* + * If the vCPU is migrated to a different pCPU than the one on which + * the vCPU last ran, and IBPB is advertised to the vCPU, then flush + * indirect branch predictors before the next VM-Enter to ensure the + * vCPU doesn't consume prediction information from a previous run on + * the "new" pCPU. + */ + if (unlikely(vcpu->arch.last_vmentry_cpu !=3D cpu && + vcpu->arch.last_vmentry_cpu >=3D 0) && + (guest_cpu_cap_has(vcpu, X86_FEATURE_SPEC_CTRL) || + guest_cpu_cap_has(vcpu, X86_FEATURE_AMD_IBPB))) + vcpu->arch.need_ibpb =3D true; + if (unlikely(vcpu->cpu !=3D cpu) || kvm_check_tsc_unstable()) { s64 tsc_delta =3D !vcpu->arch.last_host_tsc ? 0 : rdtsc() - vcpu->arch.last_host_tsc; --=20 2.52.0.457.g6b5491de43-goog