From nobody Sun Oct 5 10:47:10 2025 Received: from mail-pf1-f202.google.com (mail-pf1-f202.google.com [209.85.210.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B4A2C295DBE for ; Tue, 5 Aug 2025 19:05:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754420733; cv=none; b=tmZI+vhHYiHeaXyE5IBt3hwxze32Qu1wPZ6XT5yDDvB+lkG1C9ZMDZLDYemPTfs4lrRRFn5mo3ZVLYIe48TKR+oOa3zyWavua/7WlO/BZHAZoCBpGBL4h82ragUWQPEEUEgzKV3lZBrQrb7lchLsCvfmrUw/Pb2cqLmyal3QOl4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754420733; c=relaxed/simple; bh=egmX3I+xWiBArhvmUxzBeMAeu0B5Tb33ykONyOMBO3k=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=YssufpmUMLIcA/uY7JXb7JbemP29AyQzhsQV5GpQu71KC0ixxEASfOb5ALuKNZd3T2tzleng1BmR7n43SFvlUwg9B/7BkttRgR2dsXoiCJXOn28M6pz/F7BMbiJOIKUP85H6hFSDZrtkPF3pA/5MQBZ/sz44WFk9iotUrR2Ou+s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=MYkhmdjL; arc=none smtp.client-ip=209.85.210.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="MYkhmdjL" Received: by mail-pf1-f202.google.com with SMTP id d2e1a72fcca58-76bb326ad0bso5299939b3a.0 for ; Tue, 05 Aug 2025 12:05:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1754420731; x=1755025531; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=A1yA3Tox8y0Vt47rQoMx2qI7LaxORtPHP2V794GePOY=; b=MYkhmdjLxjMs1C4gvRLAzni4Um74dQt4qSC0sSl6pfvdK/JtXB4SF9ThcuO8h05rp/ 9EV2ZdOHNPsJjEgw0nmStFuLt9wjWGHyDxZrdje3wE/IAI6lUwmwN2SuaG6Ei8brhzra YAUn8tsNVUQDdN39Mihuxk1goBLkTtRvd0ihXmewmEgd6vGl2Rkl5uxKJukJOkO95WUP sma1sXwJBU6JmG7Aps+cw7h3trLP2NXBRddUk3hT5nLOSaHMNGq+uzh6o+c7+4cfUvSL kJsvYC4VzDsnuf8TvhLswudpZP55r9uxA4ijicT2v1Gjvc4J+WAnKFLnb0yEjq6aXpgf YE2g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754420731; x=1755025531; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=A1yA3Tox8y0Vt47rQoMx2qI7LaxORtPHP2V794GePOY=; b=TF+NmSgNFyqXuyaNqfXgy+d/8RqyeFFKvuoY23wDHrJFNHAJLL+IC5Eq9DzEqV/99+ hC1T/ew+QpkKXk1p9CbVMyMxHOfIhLoWL7Ml81nqrm7NjyefJKxsonQnrSV1LikeZqiQ BOELkE0RhbdTPm9AsI2C1efkuQqHk3RwnXcpEhk7fRDi02hXhJdrCThezNin6dDIlkr8 P4MhJlsbIrF8Vxqw/zryCKhdGfdavy9CGW3vknPjewJ6IfaR5Dkh4J2VPQYM81e27wkL pBvTms1tsfSlYoDi10PCyOZePXJDww+agUUCzCQjAwPg6mPXHIFvcYXcDOrX/pzWzcMU YR8g== X-Forwarded-Encrypted: i=1; AJvYcCX7qikNfjfT8BEpUg0NFAUSZz+Z3krV7PebBfDdyuC69fo/2xQx2umkoSqssERcoeVQaVsMSPTMyZQMYac=@vger.kernel.org X-Gm-Message-State: AOJu0YwWy1LJlZ/sM4M1kI1ATFc3GIsPYG8WMkHzefkdXqZPrV17yoT/ 2EiaEaHf5o6KrSJ0CALIMWHvIo5mH4dBXbV3YNP0c7R94dniC8zUWOYcLF9ud0S5v3Ak2KYLgPt tjRr41A== X-Google-Smtp-Source: AGHT+IHadUAEVDdXXxm0Sgn+N9jgrencXx34zcy0U/IA8j77dHu13CAUEGcxSCQj8cplRDe4BHgnu5bo6FM= X-Received: from pgct18.prod.google.com ([2002:a05:6a02:5292:b0:b42:f1d:a69b]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a20:2444:b0:240:168b:31b with SMTP id adf61e73a8af0-240313ba206mr270310637.16.1754420730960; Tue, 05 Aug 2025 12:05:30 -0700 (PDT) Reply-To: Sean Christopherson Date: Tue, 5 Aug 2025 12:05:09 -0700 In-Reply-To: <20250805190526.1453366-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250805190526.1453366-1-seanjc@google.com> X-Mailer: git-send-email 2.50.1.565.gc32cd1483b-goog Message-ID: <20250805190526.1453366-2-seanjc@google.com> Subject: [PATCH 01/18] KVM: SVM: Skip fastpath emulation on VM-Exit if next RIP isn't valid From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Xin Li , Dapeng Mi , Sandipan Das Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Skip the WRMSR and HLT fastpaths in SVM's VM-Exit handler if the next RIP isn't valid, e.g. because KVM is running with nrips=3Dfalse. SVM must decode and emulate to skip the instruction if the CPU doesn't provide the next RIP, and getting the instruction bytes to decode requires reading guest memory. Reading guest memory through the emulator can fault, i.e. can sleep, which is disallowed since the fastpath handlers run with IRQs disabled. BUG: sleeping function called from invalid context at ./include/linux/uacc= ess.h:106 in_atomic(): 1, irqs_disabled(): 1, non_block: 0, pid: 32611, name: qemu preempt_count: 1, expected: 0 INFO: lockdep is turned off. irq event stamp: 30580 hardirqs last enabled at (30579): [] vcpu_run+0x1787/0x= 1db0 [kvm] hardirqs last disabled at (30580): [] __schedule+0x1e2/0= xed0 softirqs last enabled at (30570): [] fpu_swap_kvm_fpsta= te+0x44/0x210 softirqs last disabled at (30568): [] fpu_swap_kvm_fpsta= te+0x44/0x210 CPU: 298 UID: 0 PID: 32611 Comm: qemu Tainted: G U 6.16.0= -smp--e6c618b51cfe-sleep #782 NONE Tainted: [U]=3DUSER Hardware name: Google Astoria-Turin/astoria, BIOS 0.20241223.2-0 01/17/2025 Call Trace: dump_stack_lvl+0x7d/0xb0 __might_resched+0x271/0x290 __might_fault+0x28/0x80 kvm_vcpu_read_guest_page+0x8d/0xc0 [kvm] kvm_fetch_guest_virt+0x92/0xc0 [kvm] __do_insn_fetch_bytes+0xf3/0x1e0 [kvm] x86_decode_insn+0xd1/0x1010 [kvm] x86_emulate_instruction+0x105/0x810 [kvm] __svm_skip_emulated_instruction+0xc4/0x140 [kvm_amd] handle_fastpath_invd+0xc4/0x1a0 [kvm] vcpu_run+0x11a1/0x1db0 [kvm] kvm_arch_vcpu_ioctl_run+0x5cc/0x730 [kvm] kvm_vcpu_ioctl+0x578/0x6a0 [kvm] __se_sys_ioctl+0x6d/0xb0 do_syscall_64+0x8a/0x2c0 entry_SYSCALL_64_after_hwframe+0x4b/0x53 RIP: 0033:0x7f479d57a94b Note, this is essentially a reapply of commit 5c30e8101e8d ("KVM: SVM: Skip WRMSR fastpath on VM-Exit if next RIP isn't valid"), but with different justification (KVM now grabs SRCU when skipping the instruction for other reasons). Fixes: b439eb8ab578 ("Revert "KVM: SVM: Skip WRMSR fastpath on VM-Exit if n= ext RIP isn't valid"") Cc: stable@vger.kernel.org Signed-off-by: Sean Christopherson --- arch/x86/kvm/svm/svm.c | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index d9931c6c4bc6..829d9d46718d 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c @@ -4181,13 +4181,21 @@ static int svm_vcpu_pre_run(struct kvm_vcpu *vcpu) static fastpath_t svm_exit_handlers_fastpath(struct kvm_vcpu *vcpu) { struct vcpu_svm *svm =3D to_svm(vcpu); + struct vmcb_control_area *control =3D &svm->vmcb->control; + + /* + * Next RIP must be provided as IRQs are disabled, and accessing guest + * memory to decode the instruction might fault, i.e. might sleep. + */ + if (!nrips || !control->next_rip) + return EXIT_FASTPATH_NONE; =20 if (is_guest_mode(vcpu)) return EXIT_FASTPATH_NONE; =20 - switch (svm->vmcb->control.exit_code) { + switch (control->exit_code) { case SVM_EXIT_MSR: - if (!svm->vmcb->control.exit_info_1) + if (!control->exit_info_1) break; return handle_fastpath_set_msr_irqoff(vcpu); case SVM_EXIT_HLT: --=20 2.50.1.565.gc32cd1483b-goog From nobody Sun Oct 5 10:47:10 2025 Received: from mail-pl1-f201.google.com (mail-pl1-f201.google.com [209.85.214.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8CC5829AB1C for ; Tue, 5 Aug 2025 19:05:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754420735; cv=none; b=EcUctIZf7eLtBRZW1ZiwLFDFaCE6pdbObYXZnMoKae/aWEmH8e90XKjaXe+uGkG/h2hE4yZ+47TTCiLs3F3Ctbbyk28JT60i3PCQtkqkQs4czfD7whCCz1/VHqgYuN1O0uqa38ByNZd6oYexMhQyLfNq2fqd0DCC+ubWvrz76oE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754420735; c=relaxed/simple; bh=HbVZXzciHuWe7qp7KdZ5PNobwj23hgNSTftSTqFbOGU=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Ummm6oDRMP06UQF+legpo2hPL31SkwdqXKqePWDSLVv/8YHhAFc4JNkRv2sjXgf9N3riErJVENnY6tZTHjeEXppofWC1cFg41uo0pAua5sqDRgvOG8Wb7pnrT8xxL4sfQvJVZMMOPQ4puXpAbzaTn8TRyzRj/7EEx8EchRHoa0U= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=ibcI1tvj; arc=none smtp.client-ip=209.85.214.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="ibcI1tvj" Received: by mail-pl1-f201.google.com with SMTP id d9443c01a7336-2403e4c82ddso45229455ad.1 for ; Tue, 05 Aug 2025 12:05:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1754420733; x=1755025533; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=kwv2uoNl6IM1Rm8d9mEpWGTse8EQAo0NkYkscIAw3C4=; b=ibcI1tvjNbqUSANSvbxo28wHlhJnfNiOzbaEHt+4P6izhIQCVKQjBXRpJI1fAzx/9m zvyb3JxieOrk1IBSIY2FqFsUjdBG7AqfxyU/AWK0Ffxs62phJngae1V8jvOeyOXXqLPU /W5/WE6d9jUItdVoc7eOaQ2+kFq3aFhRA/3jx7QfemrlYwn4LMAxDCMen9Qzlm+/IO/f D4MK8VVmac5msYAdA9+4vrs6UQ+57wtfePzpDsH1L8o1BX38f4+/qeulSaObOyze9C+J qYYROdOIT/z3edGVfsBpgocGyDceqsjSYQlXPpzrmqxST6z2JcFQ3HWtLxKFHWoecEgN +ECw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754420733; x=1755025533; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=kwv2uoNl6IM1Rm8d9mEpWGTse8EQAo0NkYkscIAw3C4=; b=dYyiBf2CRGWdxHW7MihjrjCbPU9bt18eguTy/WaOsxOplvTARFA8Pm7lyeKkG6Ecqd 73ODhzPsW0f2U/y9rH0RLvEmNhfNdZAC6z034BQ/Y8osl3jMS/uQqQSqy1szkjOKQaAc E92DwHBXYuLBXbgIh/htss5c3LD3h953UXpyhPh3HPKWaueTELp8itCrSTxOc5UJPE9G F9AE6wzDt/7PtIfcqmf9Bl79qqfkFC5KLh9XDeWRblbeuNpFMCTuggKYc1NgVw5EoEuU cDmjuTvwpTNj+7D2/ZBQdIAarCN8dpedHmoc26d97NnezXiDfvX8xZ0IuaUVqdypjoHD KiDw== X-Forwarded-Encrypted: i=1; AJvYcCWHTMPk9YTgXuU2QFISLIQpZzA+LB8PYV8ncda3rVqzfX+qjY3JTdANIgVpj6i/IaZ6P4DJtj7FdeOfRhw=@vger.kernel.org X-Gm-Message-State: AOJu0Yx17jCtLH3k2ITOqn1n6QBbAIUHcHawQFM2AZaSQA1XLu399rZC 5hgofSS7TjbqCLIyL0GFw1kRPIiRfuOPXwDIWc9Ch15h7FfnZTqOA4VjWioPeesklduyj0RRzmt 8ofnwVg== X-Google-Smtp-Source: AGHT+IECGQbfrhzAcpL9CeFtEuwum+PP5UpXCFSrhMP8ea/+VjUuaLEHCKh0Fq5BBGoQCFW8zYtB28FTAYc= X-Received: from pjbnd10.prod.google.com ([2002:a17:90b:4cca:b0:311:f699:df0a]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:903:2284:b0:240:5c0e:758b with SMTP id d9443c01a7336-2429f671811mr2057625ad.50.1754420732956; Tue, 05 Aug 2025 12:05:32 -0700 (PDT) Reply-To: Sean Christopherson Date: Tue, 5 Aug 2025 12:05:10 -0700 In-Reply-To: <20250805190526.1453366-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250805190526.1453366-1-seanjc@google.com> X-Mailer: git-send-email 2.50.1.565.gc32cd1483b-goog Message-ID: <20250805190526.1453366-3-seanjc@google.com> Subject: [PATCH 02/18] KVM: x86: Add kvm_icr_to_lapic_irq() helper to allow for fastpath IPIs From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Xin Li , Dapeng Mi , Sandipan Das Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Extract the code for converting an ICR message into a kvm_lapic_irq structure into a local helper so that a fast-only IPI path can share the conversion logic. No functional change intended. Signed-off-by: Sean Christopherson --- arch/x86/kvm/lapic.c | 30 ++++++++++++++++++------------ 1 file changed, 18 insertions(+), 12 deletions(-) diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c index 8172c2042dd6..9f9846980625 100644 --- a/arch/x86/kvm/lapic.c +++ b/arch/x86/kvm/lapic.c @@ -1481,24 +1481,30 @@ void kvm_apic_set_eoi_accelerated(struct kvm_vcpu *= vcpu, int vector) } EXPORT_SYMBOL_GPL(kvm_apic_set_eoi_accelerated); =20 -void kvm_apic_send_ipi(struct kvm_lapic *apic, u32 icr_low, u32 icr_high) +static void kvm_icr_to_lapic_irq(struct kvm_lapic *apic, u32 icr_low, + u32 icr_high, struct kvm_lapic_irq *irq) { - struct kvm_lapic_irq irq; - /* KVM has no delay and should always clear the BUSY/PENDING flag. */ WARN_ON_ONCE(icr_low & APIC_ICR_BUSY); =20 - irq.vector =3D icr_low & APIC_VECTOR_MASK; - irq.delivery_mode =3D icr_low & APIC_MODE_MASK; - irq.dest_mode =3D icr_low & APIC_DEST_MASK; - irq.level =3D (icr_low & APIC_INT_ASSERT) !=3D 0; - irq.trig_mode =3D icr_low & APIC_INT_LEVELTRIG; - irq.shorthand =3D icr_low & APIC_SHORT_MASK; - irq.msi_redir_hint =3D false; + irq->vector =3D icr_low & APIC_VECTOR_MASK; + irq->delivery_mode =3D icr_low & APIC_MODE_MASK; + irq->dest_mode =3D icr_low & APIC_DEST_MASK; + irq->level =3D (icr_low & APIC_INT_ASSERT) !=3D 0; + irq->trig_mode =3D icr_low & APIC_INT_LEVELTRIG; + irq->shorthand =3D icr_low & APIC_SHORT_MASK; + irq->msi_redir_hint =3D false; if (apic_x2apic_mode(apic)) - irq.dest_id =3D icr_high; + irq->dest_id =3D icr_high; else - irq.dest_id =3D GET_XAPIC_DEST_FIELD(icr_high); + irq->dest_id =3D GET_XAPIC_DEST_FIELD(icr_high); +} + +void kvm_apic_send_ipi(struct kvm_lapic *apic, u32 icr_low, u32 icr_high) +{ + struct kvm_lapic_irq irq; + + kvm_icr_to_lapic_irq(apic, icr_low, icr_high, &irq); =20 trace_kvm_apic_ipi(icr_low, irq.dest_id); =20 --=20 2.50.1.565.gc32cd1483b-goog From nobody Sun Oct 5 10:47:10 2025 Received: from mail-pg1-f201.google.com (mail-pg1-f201.google.com [209.85.215.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 31F9E29B20D for ; Tue, 5 Aug 2025 19:05:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754420736; cv=none; b=kdeo7Ww+al3i4d79hQDveiRpHnKkt7Ty72oQFcGGGS8v8l3D8GM6umHVxAlYzKJrn5ayUdC77d8jUNkMdq75VBe5wy6AqUeh9I04U4Z4PK7vxpvdE270Hf5w3Emi2g7blG8g3YlatOM43l08JYmqE++DhkINNeJcJAiXo97Q0SI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754420736; c=relaxed/simple; bh=wn8FH/CbQefJkydk31Od56LqbL8DTaoAuQuLtn3ojvs=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=sy0f0HSlIjwaco8HxwFxCiWFDCxT63eN8AH4Xc/NRNgNs/TZN2Pn+NV7eJfAOZdldHpXmFo6RB7RhkZgmilzlsusSZs0dQ/en0W0TqtXDmjTcQe4xk+7VUYFyrTj3RXMOmK/6FOZqcIJ2G0HKu+eISBzf4v034xNe2UHqUtIdpg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=yfSaQQch; arc=none smtp.client-ip=209.85.215.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="yfSaQQch" Received: by mail-pg1-f201.google.com with SMTP id 41be03b00d2f7-b423dcff27aso7005588a12.0 for ; Tue, 05 Aug 2025 12:05:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1754420734; x=1755025534; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=bUQ5kLCEv2X5mdMO2xveh6XBRkOp4xrjHvyUwtUsZso=; b=yfSaQQchUtUrGj76Fv5/KILsDdOCQ77CiiakfzJWL64HAIT4868x9K5kr+anWrDNP5 ocUK1vuXUf+i9JKqUtrYcoOK5LCI4yYcayi0V6w9H84NnPZSRKdZNrKKMWlN9byFrRCK q+o0LLjYWEw7dcueMOdm39muhJBmK96/aY3L0dLBFiRfMiLepdAeJ8MqKKh/TF/HRmdp 5sBAkKsjuFeGqkqAeV6vrEwxTKNZ6VrOBfRlAGYzUUmNxHFE2WQib/SuGVLgrXBzkjGF ZSmXS2/iRC/8QcRTFXaU9OTSEOUBCOm67sPxOH2N2YNrgu7xYidTwDpySy7DL+12Pc2w EVdA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754420734; x=1755025534; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=bUQ5kLCEv2X5mdMO2xveh6XBRkOp4xrjHvyUwtUsZso=; b=EHpbVZhkNWhWuTH/yBwxVf5rm6WpJLrpyUeYbtPBweySijMA2nhqRy6Zt6Do7eW2IH Y+rjjkP27IGtfxu4C39y68sD60sO4KuGWNZ9dbpN9YQ2yQOE5I9wArmvATYulxzod21M BzdW/birVPtISfZtfkMzzleR4awI5Zmy2g29aVuKGOkhR2SUw+LzJ8aYP4aX38HP1Eyw 615eJxOmMOvIQekMq08R4H3V8MTkPCgcnN3MZpTVKiCLGcldGssk179CxfzfnQKo6zpL WS4rM8BbR4IZKSgujE+6ymuRxGy3uZ8Uuuiy3O86VhxNccl7VuW8efVpU1GA0MmD4aej X4Nw== X-Forwarded-Encrypted: i=1; AJvYcCWOwWFVnfC46hqU/pbdDyf50zmONrViInbd8AMRgSAhpqHTcCihPDC/R2XlyRtoieWL3O9KGwbEHtQ8u00=@vger.kernel.org X-Gm-Message-State: AOJu0YxT1K/woG9NBOcWGB5knP1/ADxErUOUowGKTBD+bs/aS7Z5em9C L3RYmRsEhRCpcGyXnaYfHkP5HRia1RXajz4jzqgPhdqCqUbtejbYpIt2QjYgC7cD7pYjnQr9/2Z ATrQzPg== X-Google-Smtp-Source: AGHT+IE+OsfSdhSCn/Gs6PlSpEaDYKfPnlddZ+qWQCd+uo32vxz0OpikkULyxj2ptofDKrqY1oT+77lEksA= X-Received: from plbkf15.prod.google.com ([2002:a17:903:5cf:b0:234:9673:1d13]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:902:d50e:b0:240:5c38:7555 with SMTP id d9443c01a7336-2429f1ddb60mr2746245ad.5.1754420734532; Tue, 05 Aug 2025 12:05:34 -0700 (PDT) Reply-To: Sean Christopherson Date: Tue, 5 Aug 2025 12:05:11 -0700 In-Reply-To: <20250805190526.1453366-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250805190526.1453366-1-seanjc@google.com> X-Mailer: git-send-email 2.50.1.565.gc32cd1483b-goog Message-ID: <20250805190526.1453366-4-seanjc@google.com> Subject: [PATCH 03/18] KVM: x86: Only allow "fast" IPIs in fastpath WRMSR(X2APIC_ICR) handler From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Xin Li , Dapeng Mi , Sandipan Das Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Explicitly restrict fastpath ICR writes to IPIs that are "fast", i.e. can be delivered without having to walk all vCPUs, and that target at most 16 vCPUs. Artificially restricting ICR writes to physical mode guarantees at most one vCPU will receive in IPI (because x2APIC IDs are read-only), but that delivery might not be "fast". E.g. even if the vCPU exists, KVM might have to iterate over 4096 vCPUs to find the right one. Limiting delivery to fast IPIs aligns the WRMSR fastpath with kvm_arch_set_irq_inatomic() (which also runs with IRQs disabled), and will allow dropping the semi-arbitrary restrictions on delivery mode and type. Signed-off-by: Sean Christopherson --- arch/x86/kvm/lapic.c | 27 +++++++++++++++++++++++++-- arch/x86/kvm/lapic.h | 2 +- arch/x86/kvm/x86.c | 2 +- 3 files changed, 27 insertions(+), 4 deletions(-) diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c index 9f9846980625..bd3232dd7a63 100644 --- a/arch/x86/kvm/lapic.c +++ b/arch/x86/kvm/lapic.c @@ -2439,7 +2439,7 @@ EXPORT_SYMBOL_GPL(kvm_lapic_set_eoi); =20 #define X2APIC_ICR_RESERVED_BITS (GENMASK_ULL(31, 20) | GENMASK_ULL(17, 16= ) | BIT(13)) =20 -int kvm_x2apic_icr_write(struct kvm_lapic *apic, u64 data) +static int __kvm_x2apic_icr_write(struct kvm_lapic *apic, u64 data, bool f= ast) { if (data & X2APIC_ICR_RESERVED_BITS) return 1; @@ -2454,7 +2454,20 @@ int kvm_x2apic_icr_write(struct kvm_lapic *apic, u64= data) */ data &=3D ~APIC_ICR_BUSY; =20 - kvm_apic_send_ipi(apic, (u32)data, (u32)(data >> 32)); + if (fast) { + struct kvm_lapic_irq irq; + int ignored; + + kvm_icr_to_lapic_irq(apic, (u32)data, (u32)(data >> 32), &irq); + + if (!kvm_irq_delivery_to_apic_fast(apic->vcpu->kvm, apic, &irq, + &ignored, NULL)) + return -EWOULDBLOCK; + + trace_kvm_apic_ipi((u32)data, irq.dest_id); + } else { + kvm_apic_send_ipi(apic, (u32)data, (u32)(data >> 32)); + } if (kvm_x86_ops.x2apic_icr_is_split) { kvm_lapic_set_reg(apic, APIC_ICR, data); kvm_lapic_set_reg(apic, APIC_ICR2, data >> 32); @@ -2465,6 +2478,16 @@ int kvm_x2apic_icr_write(struct kvm_lapic *apic, u64= data) return 0; } =20 +static int kvm_x2apic_icr_write(struct kvm_lapic *apic, u64 data) +{ + return __kvm_x2apic_icr_write(apic, data, false); +} + +int kvm_x2apic_icr_write_fast(struct kvm_lapic *apic, u64 data) +{ + return __kvm_x2apic_icr_write(apic, data, true); +} + static u64 kvm_x2apic_icr_read(struct kvm_lapic *apic) { if (kvm_x86_ops.x2apic_icr_is_split) diff --git a/arch/x86/kvm/lapic.h b/arch/x86/kvm/lapic.h index 72de14527698..1b2d408816aa 100644 --- a/arch/x86/kvm/lapic.h +++ b/arch/x86/kvm/lapic.h @@ -137,7 +137,7 @@ int kvm_lapic_set_vapic_addr(struct kvm_vcpu *vcpu, gpa= _t vapic_addr); void kvm_lapic_sync_from_vapic(struct kvm_vcpu *vcpu); void kvm_lapic_sync_to_vapic(struct kvm_vcpu *vcpu); =20 -int kvm_x2apic_icr_write(struct kvm_lapic *apic, u64 data); +int kvm_x2apic_icr_write_fast(struct kvm_lapic *apic, u64 data); int kvm_x2apic_msr_write(struct kvm_vcpu *vcpu, u32 msr, u64 data); int kvm_x2apic_msr_read(struct kvm_vcpu *vcpu, u32 msr, u64 *data); =20 diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index a1c49bc681c4..8c8b7d7902a0 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -2149,7 +2149,7 @@ static int handle_fastpath_set_x2apic_icr_irqoff(stru= ct kvm_vcpu *vcpu, u64 data ((data & APIC_DEST_MASK) =3D=3D APIC_DEST_PHYSICAL) && ((data & APIC_MODE_MASK) =3D=3D APIC_DM_FIXED) && ((u32)(data >> 32) !=3D X2APIC_BROADCAST)) - return kvm_x2apic_icr_write(vcpu->arch.apic, data); + return kvm_x2apic_icr_write_fast(vcpu->arch.apic, data); =20 return 1; } --=20 2.50.1.565.gc32cd1483b-goog From nobody Sun Oct 5 10:47:10 2025 Received: from mail-pg1-f201.google.com (mail-pg1-f201.google.com [209.85.215.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DEEEB29B78E for ; Tue, 5 Aug 2025 19:05:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754420738; cv=none; b=frk6dudICeUWauRIeP4rtu6YBhRg8ylgD9cZ8zkftMdP7IEmwzArCtoWRzhUMEKEurJs4efOCi2ZS7nItqxh+W5cyCQU7nJaEeEvQ5BHdQ61jklUclmPJ5vDdnASuV5tEi8E7rv0lLo0nX6ZFwBw3u9eQDFyJGriFyLgf9hdRcg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754420738; c=relaxed/simple; bh=EeTLHsTvTDj8Ey13p+W1lWhVKS1l/trEcm4QlAWCCfk=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=EAKLqZTV2tMellBSzn/uQaUeTLlYli50FkrEjRUSGmDp9e1/G3ypO5jKGDkMOOO+h2aeYyIG9Qovpi8SvL2b1aYgC0Xw1E2sU+kPHptWiMzk2DwdJdLgTPgOSk1A9UU1jqJXz9GNSBd1Z7WxVNuZRTSrqChuEqQ2uix/sSnvf3c= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=vifgQNSl; arc=none smtp.client-ip=209.85.215.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="vifgQNSl" Received: by mail-pg1-f201.google.com with SMTP id 41be03b00d2f7-b2c00e965d0so4556220a12.2 for ; Tue, 05 Aug 2025 12:05:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1754420736; x=1755025536; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=ZFfqB1AL/AlReFpKct61YHUAJLOWiS9uPENYpzvXzio=; b=vifgQNSlzMOGOuStBxRmuNl0aZUJ10qkAGHDWeaM5yx+IoglD1zhkMxuvWxfMRlDH1 MUgb5A0yipp5sWDVicH+EwxKz7KjPxcgXPTuXu/V6qw1xaL1hbjFDdZcgqCdbhaBTDXw shcsj89AKlWuUnJfmu2eJCONQWNSE5pQco8OZYt7a/VR3v8YFK6sixM2go3XO7jXzuoa aN6qDdgRelX/8aIHCZjvNbLSya114K7jYk0GYu8VefdpGXsiiOkUNhFCmPYf9yGc/EbK pADQA4r4eK3qevwGF+LuW+s5ZkRNSspdwy0fxFvJHEGxuDgcTwNfxcCimDtNmqQcDudW yozw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754420736; x=1755025536; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=ZFfqB1AL/AlReFpKct61YHUAJLOWiS9uPENYpzvXzio=; b=XDIG5idfFqdGXDOn/ECBuMFzrhVxWjVjvYQD685H2KTUsvVNOfkO18uJ3ORcG19MV/ LO1WFOVoqYBMi59l6FgB2aX2f4Js60vRLLJgaENQRbVK4VWVME+p+bd9Qp3EDhlIoi03 1V/BpygNp2og+XGgQ+zD06IikV7Q/Ot2gZq9ZQiMlc3/R9ytid0tI8Mt2+xnnsPZtz1d F6cGvd5WEIP0QCfskwuV4Yh7gP67Bbyp3xN1q/uGns1qAAdcWQ1zW1b+XR/C2Mkr6+lu ZSU+cPdeXVGc+Udt+q6XT3+fpsWriIS0ZbPviSvur9mr1sL9/uybY70o5gFR/M1j+hBX 5CPA== X-Forwarded-Encrypted: i=1; AJvYcCUAnXsv6njBLZvmPW0lR+/YDjXzU0+b/aqvY8uRsbEtGGOe4ofp8VRjKd6neaxmfaO6AFdNqP0LDfrHqL0=@vger.kernel.org X-Gm-Message-State: AOJu0YzVD680VLlLUYcPcx5rs5bMIGeJM3dxKQK1nlSa5FX0jKfmQbJK Y7sHmoCWGfZB1cLvhaQOJ9HPldBtArkO2Ml7vbvFKopkg2mt6wqyAusOXHloF3VfZvoa+X8BXr/ J0OsyzQ== X-Google-Smtp-Source: AGHT+IHBWzHfL+L7GB3q7VFL96dyhmem4ajYx6SknmEz91pFfowhZsGDKVn++VKvzcmUoICFnKZXrj3SyWA= X-Received: from pge13.prod.google.com ([2002:a05:6a02:2d0d:b0:b42:3711:3de7]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a20:6a20:b0:240:1d4f:720b with SMTP id adf61e73a8af0-240314553a2mr238202637.23.1754420736287; Tue, 05 Aug 2025 12:05:36 -0700 (PDT) Reply-To: Sean Christopherson Date: Tue, 5 Aug 2025 12:05:12 -0700 In-Reply-To: <20250805190526.1453366-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250805190526.1453366-1-seanjc@google.com> X-Mailer: git-send-email 2.50.1.565.gc32cd1483b-goog Message-ID: <20250805190526.1453366-5-seanjc@google.com> Subject: [PATCH 04/18] KVM: x86: Drop semi-arbitrary restrictions on IPI type in fastpath From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Xin Li , Dapeng Mi , Sandipan Das Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Drop the restrictions on fastpath IPIs only working for fixed IRQs with a physical destination now that the fastpath is explicitly limited to "fast" delivery. Limiting delivery to a single physical APIC ID guarantees only one vCPU will receive the event, but that isn't necessary "fast", e.g. if the targeted vCPU is the last of 4096 vCPUs. And logical destination mode or shorthand (to self) can also be fast, e.g. if only a few vCPUs are being targeted. Lastly, there's nothing inherently slow about delivering an NMI, INIT, SIPI, SMI, etc., i.e. there's no reason to artificially limit fastpath delivery to fixed vector IRQs. Signed-off-by: Sean Christopherson --- arch/x86/kvm/x86.c | 8 +------- 1 file changed, 1 insertion(+), 7 deletions(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 8c8b7d7902a0..ea117c4b20c8 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -2145,13 +2145,7 @@ static int handle_fastpath_set_x2apic_icr_irqoff(str= uct kvm_vcpu *vcpu, u64 data if (!lapic_in_kernel(vcpu) || !apic_x2apic_mode(vcpu->arch.apic)) return 1; =20 - if (((data & APIC_SHORT_MASK) =3D=3D APIC_DEST_NOSHORT) && - ((data & APIC_DEST_MASK) =3D=3D APIC_DEST_PHYSICAL) && - ((data & APIC_MODE_MASK) =3D=3D APIC_DM_FIXED) && - ((u32)(data >> 32) !=3D X2APIC_BROADCAST)) - return kvm_x2apic_icr_write_fast(vcpu->arch.apic, data); - - return 1; + return kvm_x2apic_icr_write_fast(vcpu->arch.apic, data); } =20 static int handle_fastpath_set_tscdeadline(struct kvm_vcpu *vcpu, u64 data) --=20 2.50.1.565.gc32cd1483b-goog From nobody Sun Oct 5 10:47:10 2025 Received: from mail-pg1-f202.google.com (mail-pg1-f202.google.com [209.85.215.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D6A0C298242 for ; Tue, 5 Aug 2025 19:05:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754420740; cv=none; b=eVtD0W1ntwKhJm3eGao8eQA3qRhRJVmfcb9XP93CTcD4LsuqzTD8JGjWHw19iiX/webgKRiopuUW0+0C81YPw1QOyeDxVrEv7sUSAmjnMbctwHWYyWylYZQfL/EfUTJ3XiJpRS0raYEXQ79R+kOLm5Z1E+0wspQYAQRMVRpNpKM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754420740; c=relaxed/simple; bh=xpuAfe6osI2bpYjxTC/iM3WTy5qqB334ATvjgCjikJg=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=qIIAk3PcLIz5fjZUbSsKH/LMP86Rf809BPmyEpDpWiPm5nhyfgtKP3RctH1KhVmte0ESAJCoAH6HrlmoRz2CL6vKUU/VlG3m9Qtpeje2FWyR0vXhgUG2yn65fRXtBr49CPzO0vefyMGvWuXUNRSNVqNYnLBx+9rkHzA3AQO0L1A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=Qj7kBf2O; arc=none smtp.client-ip=209.85.215.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="Qj7kBf2O" Received: by mail-pg1-f202.google.com with SMTP id 41be03b00d2f7-b3f71e509d6so8150518a12.0 for ; Tue, 05 Aug 2025 12:05:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1754420738; x=1755025538; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=k3BpaAX0seqYxAkXBaAjUiDEESfLKvS8cQgIlkV/n2U=; b=Qj7kBf2OxWGaiN5qyvI6Oqr93NJy7YT6glTmeg/MGRU8aLbYt8lffZyYeSpZXk5sI3 0WH0+Iuip0/fnNu8wEeVjGVXFTno5qZt643+UM7WMpMbiHSN84MnZrf5GGConsQC99FX DFH87CCeIlz2+WjTW8ckxzHR6ihONTnzAgZdO+lBTHy2e8Idj3bxL1Dem8H3ltabV3Qq +pBAIDSe7jLSfiZilgffBce8F+XkFEVfzC8MTo6Nj5qZwlCvqM5UeUvFGtgafA/ABF9U 0N+xY5Wh2HddoIu/2U0QGSoQZuJUNKcNNZxGdcH2lqsbkNMEfRYy0vBw6KVRPnU4smQB AlGQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754420738; x=1755025538; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=k3BpaAX0seqYxAkXBaAjUiDEESfLKvS8cQgIlkV/n2U=; b=wNxTzpYNHuxu9U3A6HnBZolMY5aQvklZCJvlpG/2IrDMn7O1u7bJZR+djCkIzI3ivY JtWxVINYaM2EU19O8LeLUKoRy7ZmFYpq9sSQWw6mZUmBF5I35S7TSmDEq/YCVf5kfk1q Zj6VthgziXiWCo9eCc2Vjf47eZ5rB20OnoW44pEpqwIbsI/25p06cYw6itPxpnzo8778 wwPipMgUscp1ibZmX3m0jxHgVaqhmNAHo9N0+qXmdx4Nuq/u42AohqRYqou0+qCL98X/ Nc/hc3OLUWgjDbXIJqlcoeIqgff99oKt/gs1POdSTc/8xxkDbLoTM2677llwjB5x5bTS sxnA== X-Forwarded-Encrypted: i=1; AJvYcCV30cYD+JF5TVjkcTRTfyy8lNmvWx8jrUgLItUCW8tE3DxhQQMpof1RcLNTT3TfJOMPhWNzeh+R6Uqb2mo=@vger.kernel.org X-Gm-Message-State: AOJu0YxQHfP80Kp5o7IHb28mXEamAeK4d6CcDH2eVZjfDs/ewZt6NR3o PuaDNfDF/vGHWuxOPH5/UZqvKWLOq03+lm5OqGB2HUZJKzfndnw5tF2cg0wV4yp69iJhnd4xxGJ rwEpjbw== X-Google-Smtp-Source: AGHT+IE1TxB7aC6rWf+ncWBha/qBhkqCB4f7eWGEN/MljCoe7AxrV10Ivh+oq2kbcx6+avQsLJSNKHHyjOc= X-Received: from pgci4.prod.google.com ([2002:a63:6d04:0:b0:b31:dbad:8412]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a20:3942:b0:23f:f99d:462b with SMTP id adf61e73a8af0-24031455024mr228734637.41.1754420738118; Tue, 05 Aug 2025 12:05:38 -0700 (PDT) Reply-To: Sean Christopherson Date: Tue, 5 Aug 2025 12:05:13 -0700 In-Reply-To: <20250805190526.1453366-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250805190526.1453366-1-seanjc@google.com> X-Mailer: git-send-email 2.50.1.565.gc32cd1483b-goog Message-ID: <20250805190526.1453366-6-seanjc@google.com> Subject: [PATCH 05/18] KVM: x86: Unconditionally handle MSR_IA32_TSC_DEADLINE in fastpath exits From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Xin Li , Dapeng Mi , Sandipan Das Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Drop the fastpath VM-Exit requirement that KVM can use the hypervisor timer to emulate the APIC timer in TSC deadline mode. I.e. unconditionally handle MSR_IA32_TSC_DEADLINE WRMSRs in the fastpath. Restricting the fastpath to *maybe* using the VMX preemption timer is ineffective and unnecessary. If the requested deadline can't be programmed into the VMX preemption timer, KVM will fall back to hrtimers, i.e. the restriction is ineffective as far as preventing any kind of worst case scenario. But guarding against a worst case scenario is completely unnecessary as the "slow" path, start_sw_tscdeadline() =3D> hrtimer_start(), explicitly disables IRQs. In fact, the worst case scenario is when KVM thinks it can use the VMX preemption timer, as KVM will eat the overhead of calling into vmx_set_hv_timer() and falling back to hrtimers. Opportunistically limit kvm_can_use_hv_timer() to lapic.c as the fastpath code was the only external user. Stating the obvious, this allows handling MSR_IA32_TSC_DEADLINE writes in the fastpath on AMD CPUs. Signed-off-by: Sean Christopherson --- arch/x86/kvm/lapic.c | 2 +- arch/x86/kvm/lapic.h | 1 - arch/x86/kvm/x86.c | 3 --- 3 files changed, 1 insertion(+), 5 deletions(-) diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c index bd3232dd7a63..e19545b8cc98 100644 --- a/arch/x86/kvm/lapic.c +++ b/arch/x86/kvm/lapic.c @@ -130,7 +130,7 @@ static bool kvm_can_post_timer_interrupt(struct kvm_vcp= u *vcpu) (kvm_mwait_in_guest(vcpu->kvm) || kvm_hlt_in_guest(vcpu->kvm)); } =20 -bool kvm_can_use_hv_timer(struct kvm_vcpu *vcpu) +static bool kvm_can_use_hv_timer(struct kvm_vcpu *vcpu) { return kvm_x86_ops.set_hv_timer && !(kvm_mwait_in_guest(vcpu->kvm) || diff --git a/arch/x86/kvm/lapic.h b/arch/x86/kvm/lapic.h index 1b2d408816aa..8b00e29741de 100644 --- a/arch/x86/kvm/lapic.h +++ b/arch/x86/kvm/lapic.h @@ -249,7 +249,6 @@ void kvm_lapic_switch_to_hv_timer(struct kvm_vcpu *vcpu= ); void kvm_lapic_expired_hv_timer(struct kvm_vcpu *vcpu); bool kvm_lapic_hv_timer_in_use(struct kvm_vcpu *vcpu); void kvm_lapic_restart_hv_timer(struct kvm_vcpu *vcpu); -bool kvm_can_use_hv_timer(struct kvm_vcpu *vcpu); =20 static inline enum lapic_mode kvm_apic_mode(u64 apic_base) { diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index ea117c4b20c8..63ca9185d133 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -2150,9 +2150,6 @@ static int handle_fastpath_set_x2apic_icr_irqoff(stru= ct kvm_vcpu *vcpu, u64 data =20 static int handle_fastpath_set_tscdeadline(struct kvm_vcpu *vcpu, u64 data) { - if (!kvm_can_use_hv_timer(vcpu)) - return 1; - kvm_set_lapic_tscdeadline_msr(vcpu, data); return 0; } --=20 2.50.1.565.gc32cd1483b-goog From nobody Sun Oct 5 10:47:10 2025 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8BA6129DB65 for ; Tue, 5 Aug 2025 19:05:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754420743; cv=none; b=GXV+JJySZiDvd4TP5sFWdZ9bmmutwRvfBscZj4qsOw454uC0nK0G8e/zNfI3fiSy7y+++RFVNpqS55kxjXRIDlp5sd3oK05p3DAGJHtbaWj3MEHnudX/OgSHkYTwdw+n2dIvSVM1zz2Z37FBN4Q0RvNfN1Taik96+jtgeNUeM6I= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754420743; c=relaxed/simple; bh=J3s/+HhO0F/23Anh4HAPKQwoROmPWYLeSlo9ij8W49k=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=nEYWMu7fXIFxk0qKOE/uL7jP0U6nNKRnTpjcNfDf7jyPwyQCXi3riorxznLdVwCMvZ5kMnGv5ihdqP+tOt6znn8HfMlbMtzO8Fv4fuhLyggcj9VhQ8YXE28fERdZjtx1/jUaJqvtX+XRCoTCKwn6Mvy9r5jvRWJi338mVFl/qas= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=bopbRLrb; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="bopbRLrb" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-31332dc2b59so6201115a91.0 for ; Tue, 05 Aug 2025 12:05:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1754420740; x=1755025540; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=YozS3xn+VACq0+w8nwexO84fpxAPR1pw7uiWTT6tQo0=; b=bopbRLrbATJsDSF+5lYLguL8c8CmxJt/VQ/HEmlpupSzTlWWeNkU9kgoeXuekeQwfO S22at6wVdzLfonQlBb+gGP1N9nBnxlSMn246rNaBq+sazuP6nbrZapRoAH2g7N8gZdVC XUWoW3oZaa3Utn1lwGpARjrM31+rxrhbFYtg9dcKUUP888lzLKi3Ztk030aGdO8wiB3b 4ERFyLJ2o8PYK2zHmv4cvPlVWy91cWtUk7AlKkwJN3TDbuVCLEKGfFfaVW8289pyILVk gRYkSC9DyXkAv57ZyOKPkLmNJOzzB5ePPfT85oVNQpwE2pbpB0D532CBj7dV9nUp3431 2LOg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754420740; x=1755025540; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=YozS3xn+VACq0+w8nwexO84fpxAPR1pw7uiWTT6tQo0=; b=RhuibAn1kErhQ7qTdJ3dNLpkbLKUPrg0QrrBk6tecvADkLoDhRqd/tZ+t9+amo+4Oq p6g1tWIJU4DfR9VEgYJRdAInznROuDvM88BNcWEoArrLr6zAJD2UY+Ts9wf2677tMOH3 VYIRuxA2nv9SAFluR0/bOOPeF7MtTBVO6eE+Gfl28lxTRfNLGB0w7HrF83DXE33G9QVW 59P3LO+dB9Fq8P+hcC4/sW+JEAtdrN/kQ9ib73NM4Wo7VVRG/vyyGi+MYj2hI8jaHhqs n4ZCABAckdS19ja8kemCcUetRr2NaICtd+RAAMPe1A/0dT5o+2xuRp/Swgc+3e4vrD6A z2mQ== X-Forwarded-Encrypted: i=1; AJvYcCVvMt3l6wPaDhdjABIB8bUEAdQhuL5F5+i+Oxa+T3BaM/cHko5A4Uf0pZ/8h9eCDRMx7kqHeBLfcvyvXZ8=@vger.kernel.org X-Gm-Message-State: AOJu0YzaB93OA2AkyqDrWdyudYn864Ey/7zbFiZUbRkSN5M+aiwc2gmC YjyOGC5NBiiSqnxjFFPiLUVke3v4xb40y4/GptQ0lbvQwpBzwRKrh3LpIwUvdtZGx2cT84cNCvz 0Ujuamg== X-Google-Smtp-Source: AGHT+IHQCIBBLF4UPzRODvXLUW1O4/zaAQuZxCWjMgyqDmJ4I9AAEjiuFfAHOHrKIvXYXsSDfb0p1J6tmDo= X-Received: from pjro3.prod.google.com ([2002:a17:90a:b883:b0:320:e3b2:68de]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:17c2:b0:321:59e7:c5c2 with SMTP id 98e67ed59e1d1-32159e7c6d8mr3836032a91.9.1754420739953; Tue, 05 Aug 2025 12:05:39 -0700 (PDT) Reply-To: Sean Christopherson Date: Tue, 5 Aug 2025 12:05:14 -0700 In-Reply-To: <20250805190526.1453366-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250805190526.1453366-1-seanjc@google.com> X-Mailer: git-send-email 2.50.1.565.gc32cd1483b-goog Message-ID: <20250805190526.1453366-7-seanjc@google.com> Subject: [PATCH 06/18] KVM: x86: Acquire SRCU in WRMSR fastpath iff instruction needs to be skipped From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Xin Li , Dapeng Mi , Sandipan Das Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Acquire SRCU in the WRMSR fastpath if and only if an instruction needs to be skipped, i.e. only if the fastpath succeeds. The reasoning in commit 3f2739bd1e0b ("KVM: x86: Acquire SRCU read lock when handling fastpath MSR writes") about "avoid having to play whack-a-mole" seems sound, but in hindsight unconditionally acquiring SRCU does more harm than good. While acquiring/releasing SRCU isn't slow per se, the things that are _protected_ by kvm->srcu are generally safe to access only in the "slow" VM-Exit path. E.g. accessing memslots in generic helpers is never safe, because accessing guest memory with IRQs disabled is unless unsafe (except when kvm_vcpu_read_guest_atomic() is used, but that API should never be used in emulation helpers). In other words, playing whack-a-mole is actually desirable in this case, because every access to an asset protected by kvm->srcu warrants further scrutiny. Signed-off-by: Sean Christopherson --- arch/x86/kvm/x86.c | 21 ++++++++------------- 1 file changed, 8 insertions(+), 13 deletions(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 63ca9185d133..69c668f4d2b6 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -2158,10 +2158,8 @@ fastpath_t handle_fastpath_set_msr_irqoff(struct kvm= _vcpu *vcpu) { u32 msr =3D kvm_rcx_read(vcpu); u64 data; - fastpath_t ret; bool handled; - - kvm_vcpu_srcu_read_lock(vcpu); + int r; =20 switch (msr) { case APIC_BASE_MSR + (APIC_ICR >> 4): @@ -2177,19 +2175,16 @@ fastpath_t handle_fastpath_set_msr_irqoff(struct kv= m_vcpu *vcpu) break; } =20 - if (handled) { - if (!kvm_skip_emulated_instruction(vcpu)) - ret =3D EXIT_FASTPATH_EXIT_USERSPACE; - else - ret =3D EXIT_FASTPATH_REENTER_GUEST; - trace_kvm_msr_write(msr, data); - } else { - ret =3D EXIT_FASTPATH_NONE; - } + if (!handled) + return EXIT_FASTPATH_NONE; =20 + kvm_vcpu_srcu_read_lock(vcpu); + r =3D kvm_skip_emulated_instruction(vcpu); kvm_vcpu_srcu_read_unlock(vcpu); =20 - return ret; + trace_kvm_msr_write(msr, data); + + return r ? EXIT_FASTPATH_REENTER_GUEST : EXIT_FASTPATH_EXIT_USERSPACE; } EXPORT_SYMBOL_GPL(handle_fastpath_set_msr_irqoff); =20 --=20 2.50.1.565.gc32cd1483b-goog From nobody Sun Oct 5 10:47:10 2025 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6327729A32D for ; Tue, 5 Aug 2025 19:05:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754420743; cv=none; b=KwrQ3FNECeRl1K/ScGfPlgDqVtNO0tRxa9N7Df6OUK7tcf4wTfEl/HNShAo0J62QFQbZ5gUv7ShxtKznCWi5DUGJMJokhP0O6Pa9Cd2P5NcP/m6PRrmB7HEEGR7cvoOdWz3RRn+1l9mk1CJU93CWizz1KtDMmURLa8opOi8+HuU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754420743; c=relaxed/simple; bh=FjHicGFKJBMj6eqQRJA4zeVHrXkQNDh3WomaZ2JWU6w=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=gLFXHnXjCk1GL41LrAfX6fWRqEVPRwiJHZmQUHZD6k5Pf4mD0qTe82F0Kj2eARFiHQyCKx2v+Ql1uqP3Nh6Jf2cl89o5CnxjG1VgMBnNDv7hJGGyQFtR5fhnkdNcEHeh5DQNKJniSnzIQkI+LISsIRqY9aeaEYfLvYcpYnlmiQY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=XM8k8oCP; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="XM8k8oCP" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-31eac278794so6046148a91.3 for ; Tue, 05 Aug 2025 12:05:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1754420742; x=1755025542; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=sD1yq2ng+3s/RvhfKS4rEdaP5yHsiyn1WT57oqE0wt8=; b=XM8k8oCPxx282wuw+6dbhLacCNonDw2E2pjSZ3pyz8/khplEZIRjLuyHRUmVqgkqRZ x7a7iV69Ct7tkoBpbg2GDL9zsrvxkr+LrXZgKhahWf6ugn2NKhmdOYlHle1wP56/iKNP 0E1ilzdfu5XK7M5snBMpxSMefp1xFpCHmzOVcCziES6nKXKmx3xorI/ju8Re9pJnf0ay +y7+Mwpg90eU1wSpGoJl8gAsnT1BNhf8QjBD9qf3TLACvnG2oeGPcCTlgVisoExf561p sCttityu0MYtAcXiGj/ZNDNGzpKwB/hCJZiss5u7hVpWIdWNnhXGBaWKLa7CCa839u6O KsNw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754420742; x=1755025542; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=sD1yq2ng+3s/RvhfKS4rEdaP5yHsiyn1WT57oqE0wt8=; b=ocfyv4vDS7jqZ5epWR3t/eBDfaUBwDWh0nEKu8tK9TlCCGwRGbtweW8YxKOxK/BDQs RdYYmH/SJRer2EQIcuc5Bz5Ctc6XqR8u3e2EGgUJ+VsHCUDk7h7oxpnvTLjSt9/UE0j8 opl5sBczggLlZ+uplaXUvkqGZseG5AO1mNjONY30RiRZdoDM6LGLQBwDBLfD4bKoDmHC 6pPVrrdViyOyJL/jgc2DWKOvLhMSATD6l4uzYZ2XM2LKlcI4UeLsYD3wa03DJughr9F4 oAqcPeJa4ZHY8jTcyo0Tlh+0W1d6pqyp/DjDdQw21f3Ks0vCqBwYqYYVf1bPmTsKKUZw /C5Q== X-Forwarded-Encrypted: i=1; AJvYcCWmFbURUtI3vMBiJ1TguaGHVo/+pr9pmvmWNryGgrKAGcZ9xiT/FpSgHPo8hLJR7aQVR4qChTZ/1wnQdkc=@vger.kernel.org X-Gm-Message-State: AOJu0Yzd+9+hzxQxA7sV5mSK4WLYkAp9tEff/fgRqMX8ieDF2LHB+he8 Wj8TBHjsDykxMAYRTWtnl+SHiLaUiDogZkoKv5r+vpUHUOSzQGgJt+Uxl80jidZoGjGSphUtLv2 38tLTFw== X-Google-Smtp-Source: AGHT+IFJUuShphCDKjpbJ4bBJ29lWr9wj20brYqPy6fH3C0e3fDw9eD2bFKuavKmEfXY9vwq/YetKdwwOJQ= X-Received: from pjrz21.prod.google.com ([2002:a17:90a:bd95:b0:312:fb53:41c0]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:3e84:b0:321:1df6:97d3 with SMTP id 98e67ed59e1d1-3211df698c4mr15042572a91.4.1754420741580; Tue, 05 Aug 2025 12:05:41 -0700 (PDT) Reply-To: Sean Christopherson Date: Tue, 5 Aug 2025 12:05:15 -0700 In-Reply-To: <20250805190526.1453366-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250805190526.1453366-1-seanjc@google.com> X-Mailer: git-send-email 2.50.1.565.gc32cd1483b-goog Message-ID: <20250805190526.1453366-8-seanjc@google.com> Subject: [PATCH 07/18] KVM: x86: Unconditionally grab data from EDX:EAX in WRMSR fastpath From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Xin Li , Dapeng Mi , Sandipan Das Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Always grab EDX:EAX in the WRMSR fastpath to deduplicate and simplify the case statements, and to prepare for handling immediate variants of WRMSRNS in the fastpath (the data register is explicitly provided in that case). There's no harm in reading the registers, as their values are always available, i.e. don't require VMREADs (or similarly slow operations). No real functional change intended. Cc: Xin Li Signed-off-by: Sean Christopherson --- arch/x86/kvm/x86.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 69c668f4d2b6..e6c221f9b92e 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -2156,18 +2156,16 @@ static int handle_fastpath_set_tscdeadline(struct k= vm_vcpu *vcpu, u64 data) =20 fastpath_t handle_fastpath_set_msr_irqoff(struct kvm_vcpu *vcpu) { + u64 data =3D kvm_read_edx_eax(vcpu); u32 msr =3D kvm_rcx_read(vcpu); - u64 data; bool handled; int r; =20 switch (msr) { case APIC_BASE_MSR + (APIC_ICR >> 4): - data =3D kvm_read_edx_eax(vcpu); handled =3D !handle_fastpath_set_x2apic_icr_irqoff(vcpu, data); break; case MSR_IA32_TSC_DEADLINE: - data =3D kvm_read_edx_eax(vcpu); handled =3D !handle_fastpath_set_tscdeadline(vcpu, data); break; default: --=20 2.50.1.565.gc32cd1483b-goog From nobody Sun Oct 5 10:47:10 2025 Received: from mail-pg1-f202.google.com (mail-pg1-f202.google.com [209.85.215.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3367929A9ED for ; Tue, 5 Aug 2025 19:05:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754420745; cv=none; b=b+h/sKPTMJyMO7fnhv9DCw7S4pfr83wPpJwp2hP1DfBaoVpq8DhXgyFBoRN1oIqS/YNxMTcv97pFMa0ehQ8zq1NL6wpJhcfh1VlneZyRcyeZLv34AvxBoUl18dPfcBn03Liaun6zx7fMJCU0vF+gxfSWPgTM4LOE1MmEPp7iOOY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754420745; c=relaxed/simple; bh=ATOJWurILJ+pXvAy7vgZb2du/uRW5OngKfJEtQ9MjgE=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=s46v3VOBvz3to7K06Z2XxR/Vu/9RKDmIn95YCSu9wKO4k5ZDY3cGL+QMsdx98bjJNsUdVfAUAP0sKSOHJQ4BKdv6jcSnZiUWMrwexK91skETVM0T33UjRxZSrXsE3fbSwoGqe2ljul6h4Sa4Jpag4+APx17G9V0k3uF1DWCD8no= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=A5qUfrX9; arc=none smtp.client-ip=209.85.215.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="A5qUfrX9" Received: by mail-pg1-f202.google.com with SMTP id 41be03b00d2f7-b3f38d3cabeso4054915a12.3 for ; Tue, 05 Aug 2025 12:05:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1754420743; x=1755025543; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=fXcZGtwr18gSNfbGq/ApRRZ2cGpCSejOkegwkd2RB7g=; b=A5qUfrX9p8W5M2MMDCKietvNz9pH4MYXTCrKsz3zFzXI2DeY3E0wnplRn7A3LM8VP8 XYF7Wr/HBKbgYr6LOaSarsjoP8BkbjIK9ZplauyILn4ZGmZrP0lcdnaE7fVnqkL3EkG+ PC1yc8x6EX3fYKkKQwSIQGTsHAJ22oy8+cGJHEWxTm05Hn/F+J8NhA7+Hi9nHCMWq+IR e+/5/hSC0KVXlq5U0Wkajan1p9vU+HI5okyPrV6PYzhqyIT08XHhnkBE947jidGTCDV5 iILU6z4VVEDvgoophv7olhAK1gkzNlziC7hJpmawAbmjVubFjF7aICe0ZiptQIZONAVN QEgA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754420743; x=1755025543; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=fXcZGtwr18gSNfbGq/ApRRZ2cGpCSejOkegwkd2RB7g=; b=OsVZp4nw0IALhfan7KZHXqPBbWJSfJM196GlZoZ0kT/uZ0AJzKuY/xLPWrdm7PZrPd i+4g9zpFmfLgcUBoBXihlQMxbI8XE8doYohhwaVK7jri0o9zzng7n24NSfRFpbsnUew2 93FgYyD8UOXxVMyDh0ARmF8BcycK13P+3RGcUbpXXkqYI+1QAS3dcHqrLVNAdNADSChs +3uPskbUk95lF30H3xvSwuz7HR1On7BjC9YqLKSr8/SILRbpLROvsPEA8koIzTDpuKfU ry8JIm0KIcAuP+6BfhAIdbWylnT+t4WLp0lHCFCNo2vtSyxQlXirk51VsKiiiQX7ZAbK FHOg== X-Forwarded-Encrypted: i=1; AJvYcCVM9ZOEVi31P9X6z1Ww/pLVaAAOxa+9IdlZSnZSrtlMrIMaTwoDkCFqYFx8FVRvHrRMHrarivrqWUpY844=@vger.kernel.org X-Gm-Message-State: AOJu0Yx7Cg+D6uHmK/KQxSTycSztUg9AscVXxgomr9zBwZC9T7zwXeWQ MUUGWTZPgSbb+leNjzt0cX1tvq5xQto+aS/2RS9R9mTW49yimhk5QYUcA8uOXjw19N1U9OGxlAf Gqd7jHg== X-Google-Smtp-Source: AGHT+IFzn/biQe1Qxm6XvedDBvzagTRf5b39R3idgrE1MR4qbj1/mKxMhU5L7MOTzCIbn6eqqm8/r77blPw= X-Received: from pjbli2.prod.google.com ([2002:a17:90b:48c2:b0:2ff:6132:8710]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:1346:b0:31f:9114:ead8 with SMTP id 98e67ed59e1d1-321161dd744mr21743989a91.6.1754420743452; Tue, 05 Aug 2025 12:05:43 -0700 (PDT) Reply-To: Sean Christopherson Date: Tue, 5 Aug 2025 12:05:16 -0700 In-Reply-To: <20250805190526.1453366-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250805190526.1453366-1-seanjc@google.com> X-Mailer: git-send-email 2.50.1.565.gc32cd1483b-goog Message-ID: <20250805190526.1453366-9-seanjc@google.com> Subject: [PATCH 08/18] KVM: x86: Fold WRMSR fastpath helpers into the main handler From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Xin Li , Dapeng Mi , Sandipan Das Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Fold the per-MSR WRMSR fastpath helpers into the main handler now that the IPI path in particular is relatively tiny. In addition to eliminating a decent amount of boilerplate, this removes the ugly -errno/1/0 =3D> bool conversion (which is "necessitated" by kvm_x2apic_icr_write_fast()). Opportunistically drop the comment about IPIs, as the purpose of the fastpath is hopefully self-evident, and _if_ it needs more documentation, the documentation (and rules!) should be placed in a more central location. No functional change intended. Signed-off-by: Sean Christopherson --- arch/x86/kvm/x86.c | 34 +++++----------------------------- 1 file changed, 5 insertions(+), 29 deletions(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index e6c221f9b92e..a4441f036929 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -2133,48 +2133,24 @@ static inline bool kvm_vcpu_exit_request(struct kvm= _vcpu *vcpu) kvm_request_pending(vcpu) || xfer_to_guest_mode_work_pending(); } =20 -/* - * The fast path for frequent and performance sensitive wrmsr emulation, - * i.e. the sending of IPI, sending IPI early in the VM-Exit flow reduces - * the latency of virtual IPI by avoiding the expensive bits of transition= ing - * from guest to host, e.g. reacquiring KVM's SRCU lock. In contrast to the - * other cases which must be called after interrupts are enabled on the ho= st. - */ -static int handle_fastpath_set_x2apic_icr_irqoff(struct kvm_vcpu *vcpu, u6= 4 data) -{ - if (!lapic_in_kernel(vcpu) || !apic_x2apic_mode(vcpu->arch.apic)) - return 1; - - return kvm_x2apic_icr_write_fast(vcpu->arch.apic, data); -} - -static int handle_fastpath_set_tscdeadline(struct kvm_vcpu *vcpu, u64 data) -{ - kvm_set_lapic_tscdeadline_msr(vcpu, data); - return 0; -} - fastpath_t handle_fastpath_set_msr_irqoff(struct kvm_vcpu *vcpu) { u64 data =3D kvm_read_edx_eax(vcpu); u32 msr =3D kvm_rcx_read(vcpu); - bool handled; int r; =20 switch (msr) { case APIC_BASE_MSR + (APIC_ICR >> 4): - handled =3D !handle_fastpath_set_x2apic_icr_irqoff(vcpu, data); + if (!lapic_in_kernel(vcpu) || !apic_x2apic_mode(vcpu->arch.apic) || + kvm_x2apic_icr_write_fast(vcpu->arch.apic, data)) + return EXIT_FASTPATH_NONE; break; case MSR_IA32_TSC_DEADLINE: - handled =3D !handle_fastpath_set_tscdeadline(vcpu, data); + kvm_set_lapic_tscdeadline_msr(vcpu, data); break; default: - handled =3D false; - break; - } - - if (!handled) return EXIT_FASTPATH_NONE; + } =20 kvm_vcpu_srcu_read_lock(vcpu); r =3D kvm_skip_emulated_instruction(vcpu); --=20 2.50.1.565.gc32cd1483b-goog From nobody Sun Oct 5 10:47:10 2025 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 083B32BD580 for ; Tue, 5 Aug 2025 19:05:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754420747; cv=none; b=RGNpPfR4YvviaeSnhlDPpPQYS523GeXji2HUekhhVg4Rdn/R7JZq8DESVSJoKXILRb/ojkde/C3nQtZs2qe9zlPTLbr8h7Tm2ioWcCsFBwPWNi6TofAonvqyJq6xt/0chdPfFYml5rLApcm/1qAd3aXvEe9VCwJ5suOVD0cfUZU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754420747; c=relaxed/simple; bh=jIspsSukkN0GcXbnX6kROdAooWTTjWRdQLcWRVvZSm8=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=okSP3QrhThBlwSzQBOAJIjtxhcoCOZexJq/6pAqmLzNtmZCXefGZIxi6nXTC0f16KEj7FnXrdXokzFB3LIJsYOYuh60K+nVLYy/oEu4i4DrNX0D7RaUdY+bdz7x1s2IClyrcxla4rNoJgR+13rGU0p4LvgSARKuAK6idmAVnoS8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=1FZsnIT3; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="1FZsnIT3" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-31effad1358so124803a91.1 for ; Tue, 05 Aug 2025 12:05:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1754420745; x=1755025545; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=MHm9cX+2a6wEZOmXR7zbZQOpP1vpd9jsxwALDJhOINI=; b=1FZsnIT3yJ8q2jprrQl17a+wlTnvPRnswOiXvhW6M46or3ohP4Ar597wqNeGE17r+y acwBKMQkytbLVK2PmjG0fHvNjj+G1DOwXvhKMvIGC+minWXMNf6E7VuwDBIH8/NBUf9T 5KQT7nakWSB1m8ln+ZXLQ+giFTKv0t/kKJc4RanJOeuluZU/euq7pU9LRIHq4MOqpErw KXRGlXwBfqAmsjOF57Jgb7YatO11jsj+cq6201PrWl4iztrG5Wf9SNH8tXpO8x29DBfD anVGYRzvPz0Ypq/o/pJ6nX0qihKYr4ax6xQAU48sx+YRD14E08sg5q7uYRz3XYkdT/dY 34Yg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754420745; x=1755025545; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=MHm9cX+2a6wEZOmXR7zbZQOpP1vpd9jsxwALDJhOINI=; b=SZxx23cG/dCkmyay5mQthAe3ImBCAm8Pk9onq8IEmwNxyy5Pvss0K7FbRc5iHL9A13 FY61Gg3RNPA2ou14FWijkFJZz9t1kYPzcQW7PaNkmkdaY+Z9ERLfCgLU+S7I69XaC7EI kmqqLQfdPTjImH/wTfNPGIluAUsnGc3iT1Q5x/3Ip1bqE4Dh2ddVzRhej2yW9/MZILL9 IsSPc+YJ59SlGI/cDQzrIoJtBxaLrTtycYuzF/mNd/kD9pZa9gq/2UPBwxKskGCRcCCE dtg9taH0GoIXFgTaCVtMvknuvkKWF9EMaHp1g85M8FGVXtclgBQUcZWrKlX042FO4ait EdJQ== X-Forwarded-Encrypted: i=1; AJvYcCXX9FgFYQadbDS8m0Wx4CPNGIlS7aiC+idBeejzWZNDG81wYtiOZ3PKCu/VWs7Y7HN55qVs6Ykx2oXv7gk=@vger.kernel.org X-Gm-Message-State: AOJu0YwWkG6di9rkbOg0WVTXEGhJ6VslFPovL3Aff/fvGFYdEkwz/5hN skJGzbBd2hwcmfwT/UCdfbKUD9AA9XDOUqSOwubw143TSUacKx1fttpK0ZHr660nDnHIzvPE2Er dm9qt6w== X-Google-Smtp-Source: AGHT+IGDasRXwFiMQ86DEaBEHpeUlL+2maKDx6zxVfD8zjsFQ8EmHiEcKjpKDebLzmI3pok4pfMV0qtxJc0= X-Received: from pjbns14.prod.google.com ([2002:a17:90b:250e:b0:31e:a865:8b32]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:3e8c:b0:31e:f30f:6d3b with SMTP id 98e67ed59e1d1-3216684b05emr89332a91.2.1754420745302; Tue, 05 Aug 2025 12:05:45 -0700 (PDT) Reply-To: Sean Christopherson Date: Tue, 5 Aug 2025 12:05:17 -0700 In-Reply-To: <20250805190526.1453366-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250805190526.1453366-1-seanjc@google.com> X-Mailer: git-send-email 2.50.1.565.gc32cd1483b-goog Message-ID: <20250805190526.1453366-10-seanjc@google.com> Subject: [PATCH 09/18] KVM: x86/pmu: Move kvm_init_pmu_capability() to pmu.c From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Xin Li , Dapeng Mi , Sandipan Das Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Move kvm_init_pmu_capability() to pmu.c so that future changes can access variables that have no business being visible outside of pmu.c. kvm_init_pmu_capability() is called once per module load, there's is zero reason it needs to be inlined. No functional change intended. Cc: Dapeng Mi Cc: Sandipan Das Signed-off-by: Sean Christopherson Reviewed-by: Dapeng Mi --- arch/x86/kvm/pmu.c | 47 ++++++++++++++++++++++++++++++++++++++++++++++ arch/x86/kvm/pmu.h | 47 +--------------------------------------------- 2 files changed, 48 insertions(+), 46 deletions(-) diff --git a/arch/x86/kvm/pmu.c b/arch/x86/kvm/pmu.c index 75e9cfc689f8..eb17d90916ea 100644 --- a/arch/x86/kvm/pmu.c +++ b/arch/x86/kvm/pmu.c @@ -96,6 +96,53 @@ void kvm_pmu_ops_update(const struct kvm_pmu_ops *pmu_op= s) #undef __KVM_X86_PMU_OP } =20 +void kvm_init_pmu_capability(const struct kvm_pmu_ops *pmu_ops) +{ + bool is_intel =3D boot_cpu_data.x86_vendor =3D=3D X86_VENDOR_INTEL; + int min_nr_gp_ctrs =3D pmu_ops->MIN_NR_GP_COUNTERS; + + /* + * Hybrid PMUs don't play nice with virtualization without careful + * configuration by userspace, and KVM's APIs for reporting supported + * vPMU features do not account for hybrid PMUs. Disable vPMU support + * for hybrid PMUs until KVM gains a way to let userspace opt-in. + */ + if (cpu_feature_enabled(X86_FEATURE_HYBRID_CPU)) + enable_pmu =3D false; + + if (enable_pmu) { + perf_get_x86_pmu_capability(&kvm_pmu_cap); + + /* + * WARN if perf did NOT disable hardware PMU if the number of + * architecturally required GP counters aren't present, i.e. if + * there are a non-zero number of counters, but fewer than what + * is architecturally required. + */ + if (!kvm_pmu_cap.num_counters_gp || + WARN_ON_ONCE(kvm_pmu_cap.num_counters_gp < min_nr_gp_ctrs)) + enable_pmu =3D false; + else if (is_intel && !kvm_pmu_cap.version) + enable_pmu =3D false; + } + + if (!enable_pmu) { + memset(&kvm_pmu_cap, 0, sizeof(kvm_pmu_cap)); + return; + } + + kvm_pmu_cap.version =3D min(kvm_pmu_cap.version, 2); + kvm_pmu_cap.num_counters_gp =3D min(kvm_pmu_cap.num_counters_gp, + pmu_ops->MAX_NR_GP_COUNTERS); + kvm_pmu_cap.num_counters_fixed =3D min(kvm_pmu_cap.num_counters_fixed, + KVM_MAX_NR_FIXED_COUNTERS); + + kvm_pmu_eventsel.INSTRUCTIONS_RETIRED =3D + perf_get_hw_event_config(PERF_COUNT_HW_INSTRUCTIONS); + kvm_pmu_eventsel.BRANCH_INSTRUCTIONS_RETIRED =3D + perf_get_hw_event_config(PERF_COUNT_HW_BRANCH_INSTRUCTIONS); +} + static inline void __kvm_perf_overflow(struct kvm_pmc *pmc, bool in_pmi) { struct kvm_pmu *pmu =3D pmc_to_pmu(pmc); diff --git a/arch/x86/kvm/pmu.h b/arch/x86/kvm/pmu.h index ad89d0bd6005..13477066eb40 100644 --- a/arch/x86/kvm/pmu.h +++ b/arch/x86/kvm/pmu.h @@ -180,52 +180,7 @@ static inline bool pmc_speculative_in_use(struct kvm_p= mc *pmc) extern struct x86_pmu_capability kvm_pmu_cap; extern struct kvm_pmu_emulated_event_selectors kvm_pmu_eventsel; =20 -static inline void kvm_init_pmu_capability(const struct kvm_pmu_ops *pmu_o= ps) -{ - bool is_intel =3D boot_cpu_data.x86_vendor =3D=3D X86_VENDOR_INTEL; - int min_nr_gp_ctrs =3D pmu_ops->MIN_NR_GP_COUNTERS; - - /* - * Hybrid PMUs don't play nice with virtualization without careful - * configuration by userspace, and KVM's APIs for reporting supported - * vPMU features do not account for hybrid PMUs. Disable vPMU support - * for hybrid PMUs until KVM gains a way to let userspace opt-in. - */ - if (cpu_feature_enabled(X86_FEATURE_HYBRID_CPU)) - enable_pmu =3D false; - - if (enable_pmu) { - perf_get_x86_pmu_capability(&kvm_pmu_cap); - - /* - * WARN if perf did NOT disable hardware PMU if the number of - * architecturally required GP counters aren't present, i.e. if - * there are a non-zero number of counters, but fewer than what - * is architecturally required. - */ - if (!kvm_pmu_cap.num_counters_gp || - WARN_ON_ONCE(kvm_pmu_cap.num_counters_gp < min_nr_gp_ctrs)) - enable_pmu =3D false; - else if (is_intel && !kvm_pmu_cap.version) - enable_pmu =3D false; - } - - if (!enable_pmu) { - memset(&kvm_pmu_cap, 0, sizeof(kvm_pmu_cap)); - return; - } - - kvm_pmu_cap.version =3D min(kvm_pmu_cap.version, 2); - kvm_pmu_cap.num_counters_gp =3D min(kvm_pmu_cap.num_counters_gp, - pmu_ops->MAX_NR_GP_COUNTERS); - kvm_pmu_cap.num_counters_fixed =3D min(kvm_pmu_cap.num_counters_fixed, - KVM_MAX_NR_FIXED_COUNTERS); - - kvm_pmu_eventsel.INSTRUCTIONS_RETIRED =3D - perf_get_hw_event_config(PERF_COUNT_HW_INSTRUCTIONS); - kvm_pmu_eventsel.BRANCH_INSTRUCTIONS_RETIRED =3D - perf_get_hw_event_config(PERF_COUNT_HW_BRANCH_INSTRUCTIONS); -} +void kvm_init_pmu_capability(const struct kvm_pmu_ops *pmu_ops); =20 static inline void kvm_pmu_request_counter_reprogram(struct kvm_pmc *pmc) { --=20 2.50.1.565.gc32cd1483b-goog From nobody Sun Oct 5 10:47:10 2025 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CB4552BDC01 for ; Tue, 5 Aug 2025 19:05:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754420749; cv=none; b=B4ftU/Y+bSDXi9MVVL0kg5X6ijz60KAnHBu3kpCwjCmWCJaSV6BmYydtDalOGXWzbysuQin6lL9NY90+CujT1Q/GfpDaP9VsmetpawdWrki4p9OYyijFxyFu1OSEo6JOv5mItD52EFtm1tmWAzOJQPOt2aj4oayFD+Y+veXncJM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754420749; c=relaxed/simple; bh=fQdHBwN4VEt+pvYyDup9bjJdFXgdWHQSPp8w7ovATOI=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=DC+93h0cfyKyaITHqKNeRSvygmgz0Cx8woi+24OWZqfXPNUDubGkCUPCfYwN6Z2SxttDxCyVQEip2QD8ApOqFIrcJ0b4huhIAPZPQAZM8TRcFPF39QnjvoFBkzy8B1fDz6kLUkO7MsuGvGUr4SvDFDnWiEUaIvfBQxDbNbYB4ps= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=3wcsUuup; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="3wcsUuup" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-315af08594fso6200181a91.2 for ; Tue, 05 Aug 2025 12:05:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1754420747; x=1755025547; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=+L4JanujMoX8MoOBANSSIqbWu9pkFvKE1OPkqsOcNNY=; b=3wcsUuup94S3siCWcSAJRg62bzsdrFmuvyu3NGhxOaE9YhkizxrHAmfc5LRYjujQgx nO7ImQgitEIJ03aBajd+GCRGdbB533TnVEMmwcIDEIMI4IYmWjf2pnVmHS3rU6Ivl1cA LeOEW6HgiZWjwxHIgjb2O3B4uUEYKDhKSKxcvIQICHmQuGr0kzqi9ZZwdkJbn9ABP9EF OR1IDpwEIem4/EiYWIpUFGI2y0sCS0FdFEdN9pGvG08k1spIh/vXejNpK1Xjx1eCISlV elgZZvk56lId4BSDnV5q+Qc+g/b3V57yh7/yrMkyopj8gmg5OTjsiIok81BWDWppVvkt djkA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754420747; x=1755025547; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=+L4JanujMoX8MoOBANSSIqbWu9pkFvKE1OPkqsOcNNY=; b=CD/UziC2rbEbdUX4rI2mP7nXTS1lwz8YXkGC2zoGX9/p9Z4DEZ9NMAf91FU9vDSr3H BwfdcaON4fxgJg1Q9JSHLh+aVLs+68u9mErwazyla4y6a3hJn9q5wtO+HThVgPIw5DgO VgJdeXq4DHn0oGzcbN6ZxuTLY/NK8th5vAFSBLScBeXw+Sn94tx0DUsbR6LE7NKJPF/f Gmj5/4T5EVfUvmYXqIYLoPQBXRhR5AJK4wu/G4xsT47obkmIUyL8ERKa0xAv+j9IQS16 qEO5XM8cCXMi+mCc+aYNxs2CDQxwbszpxURpQoMiiTSN+gnW6A1BzBMsnp/pHmGg9fXZ 4Ldg== X-Forwarded-Encrypted: i=1; AJvYcCUevYrf8X5XrkM15MQPId5kDoeqAE6sRG08DixsCMr8eUQMgbZKIMDualgOYpezrZiFE8kOTMCyVQFN/co=@vger.kernel.org X-Gm-Message-State: AOJu0Yz7FDiepmYGU5RPUQNzGJ5BmGmQPyxdpD6/VtyeUYw/ps6WBzOx 9ROxstytwIok6w3UnD115/XVxP3+M+h1mIT3p1zj500fNxWOiW0Nl6bBvYcF2mrNEf1ePZC3/U9 6G3l63g== X-Google-Smtp-Source: AGHT+IFskUMxmRr7Bj7HjcQz9vGA7uoUEXtiOjCozXPo6XWYD6hnHLqJur4fscHNQZMzgjvF5+H0EbT8TBg= X-Received: from pjbnd10.prod.google.com ([2002:a17:90b:4cca:b0:311:f699:df0a]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90a:dfcc:b0:312:f650:c795 with SMTP id 98e67ed59e1d1-321162bb980mr18647574a91.21.1754420747170; Tue, 05 Aug 2025 12:05:47 -0700 (PDT) Reply-To: Sean Christopherson Date: Tue, 5 Aug 2025 12:05:18 -0700 In-Reply-To: <20250805190526.1453366-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250805190526.1453366-1-seanjc@google.com> X-Mailer: git-send-email 2.50.1.565.gc32cd1483b-goog Message-ID: <20250805190526.1453366-11-seanjc@google.com> Subject: [PATCH 10/18] KVM: x86/pmu: Add wrappers for counting emulated instructions/branches From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Xin Li , Dapeng Mi , Sandipan Das Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add wrappers for triggering instruction retired and branch retired PMU events in anticipation of reworking the internal mechanisms to track which PMCs need to be evaluated, e.g. to avoid having to walk and check every PMC. Opportunistically bury "struct kvm_pmu_emulated_event_selectors" in pmu.c. No functional change intended. Signed-off-by: Sean Christopherson --- arch/x86/kvm/pmu.c | 22 ++++++++++++++++++---- arch/x86/kvm/pmu.h | 9 ++------- arch/x86/kvm/vmx/nested.c | 2 +- arch/x86/kvm/x86.c | 6 +++--- 4 files changed, 24 insertions(+), 15 deletions(-) diff --git a/arch/x86/kvm/pmu.c b/arch/x86/kvm/pmu.c index eb17d90916ea..e1911b366c43 100644 --- a/arch/x86/kvm/pmu.c +++ b/arch/x86/kvm/pmu.c @@ -29,8 +29,11 @@ struct x86_pmu_capability __read_mostly kvm_pmu_cap; EXPORT_SYMBOL_GPL(kvm_pmu_cap); =20 -struct kvm_pmu_emulated_event_selectors __read_mostly kvm_pmu_eventsel; -EXPORT_SYMBOL_GPL(kvm_pmu_eventsel); +struct kvm_pmu_emulated_event_selectors { + u64 INSTRUCTIONS_RETIRED; + u64 BRANCH_INSTRUCTIONS_RETIRED; +}; +static struct kvm_pmu_emulated_event_selectors __read_mostly kvm_pmu_event= sel; =20 /* Precise Distribution of Instructions Retired (PDIR) */ static const struct x86_cpu_id vmx_pebs_pdir_cpu[] =3D { @@ -907,7 +910,7 @@ static inline bool cpl_is_matched(struct kvm_pmc *pmc) select_user; } =20 -void kvm_pmu_trigger_event(struct kvm_vcpu *vcpu, u64 eventsel) +static void kvm_pmu_trigger_event(struct kvm_vcpu *vcpu, u64 eventsel) { DECLARE_BITMAP(bitmap, X86_PMC_IDX_MAX); struct kvm_pmu *pmu =3D vcpu_to_pmu(vcpu); @@ -944,7 +947,18 @@ void kvm_pmu_trigger_event(struct kvm_vcpu *vcpu, u64 = eventsel) kvm_pmu_incr_counter(pmc); } } -EXPORT_SYMBOL_GPL(kvm_pmu_trigger_event); + +void kvm_pmu_instruction_retired(struct kvm_vcpu *vcpu) +{ + kvm_pmu_trigger_event(vcpu, kvm_pmu_eventsel.INSTRUCTIONS_RETIRED); +} +EXPORT_SYMBOL_GPL(kvm_pmu_instruction_retired); + +void kvm_pmu_branch_retired(struct kvm_vcpu *vcpu) +{ + kvm_pmu_trigger_event(vcpu, kvm_pmu_eventsel.BRANCH_INSTRUCTIONS_RETIRED); +} +EXPORT_SYMBOL_GPL(kvm_pmu_branch_retired); =20 static bool is_masked_filter_valid(const struct kvm_x86_pmu_event_filter *= filter) { diff --git a/arch/x86/kvm/pmu.h b/arch/x86/kvm/pmu.h index 13477066eb40..740af816af37 100644 --- a/arch/x86/kvm/pmu.h +++ b/arch/x86/kvm/pmu.h @@ -23,11 +23,6 @@ =20 #define KVM_FIXED_PMC_BASE_IDX INTEL_PMC_IDX_FIXED =20 -struct kvm_pmu_emulated_event_selectors { - u64 INSTRUCTIONS_RETIRED; - u64 BRANCH_INSTRUCTIONS_RETIRED; -}; - struct kvm_pmu_ops { struct kvm_pmc *(*rdpmc_ecx_to_pmc)(struct kvm_vcpu *vcpu, unsigned int idx, u64 *mask); @@ -178,7 +173,6 @@ static inline bool pmc_speculative_in_use(struct kvm_pm= c *pmc) } =20 extern struct x86_pmu_capability kvm_pmu_cap; -extern struct kvm_pmu_emulated_event_selectors kvm_pmu_eventsel; =20 void kvm_init_pmu_capability(const struct kvm_pmu_ops *pmu_ops); =20 @@ -227,7 +221,8 @@ void kvm_pmu_init(struct kvm_vcpu *vcpu); void kvm_pmu_cleanup(struct kvm_vcpu *vcpu); void kvm_pmu_destroy(struct kvm_vcpu *vcpu); int kvm_vm_ioctl_set_pmu_event_filter(struct kvm *kvm, void __user *argp); -void kvm_pmu_trigger_event(struct kvm_vcpu *vcpu, u64 eventsel); +void kvm_pmu_instruction_retired(struct kvm_vcpu *vcpu); +void kvm_pmu_branch_retired(struct kvm_vcpu *vcpu); =20 bool is_vmware_backdoor_pmc(u32 pmc_idx); =20 diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c index b8ea1969113d..db2fd4eedc90 100644 --- a/arch/x86/kvm/vmx/nested.c +++ b/arch/x86/kvm/vmx/nested.c @@ -3690,7 +3690,7 @@ static int nested_vmx_run(struct kvm_vcpu *vcpu, bool= launch) return 1; } =20 - kvm_pmu_trigger_event(vcpu, kvm_pmu_eventsel.BRANCH_INSTRUCTIONS_RETIRED); + kvm_pmu_branch_retired(vcpu); =20 if (CC(evmptrld_status =3D=3D EVMPTRLD_VMFAIL)) return nested_vmx_failInvalid(vcpu); diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index a4441f036929..f2b2eaaec6f8 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -8824,7 +8824,7 @@ int kvm_skip_emulated_instruction(struct kvm_vcpu *vc= pu) if (unlikely(!r)) return 0; =20 - kvm_pmu_trigger_event(vcpu, kvm_pmu_eventsel.INSTRUCTIONS_RETIRED); + kvm_pmu_instruction_retired(vcpu); =20 /* * rflags is the old, "raw" value of the flags. The new value has @@ -9158,9 +9158,9 @@ int x86_emulate_instruction(struct kvm_vcpu *vcpu, gp= a_t cr2_or_gpa, */ if (!ctxt->have_exception || exception_type(ctxt->exception.vector) =3D=3D EXCPT_TRAP) { - kvm_pmu_trigger_event(vcpu, kvm_pmu_eventsel.INSTRUCTIONS_RETIRED); + kvm_pmu_instruction_retired(vcpu); if (ctxt->is_branch) - kvm_pmu_trigger_event(vcpu, kvm_pmu_eventsel.BRANCH_INSTRUCTIONS_RETIR= ED); + kvm_pmu_branch_retired(vcpu); kvm_rip_write(vcpu, ctxt->eip); if (r && (ctxt->tf || (vcpu->guest_debug & KVM_GUESTDBG_SINGLESTEP))) r =3D kvm_vcpu_do_singlestep(vcpu); --=20 2.50.1.565.gc32cd1483b-goog From nobody Sun Oct 5 10:47:10 2025 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 893312BDC3B for ; Tue, 5 Aug 2025 19:05:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754420751; cv=none; b=P5kz07lYGzNhmYDeBg5k3zWF8uJA7hE76K/7XkOcwlY93OfIrlWePp6v5pmRhVWDetLELBX5aZPcRueKtREDX8h6gt7rggirIfNrSV1V7Z3kiLlCeiGmeri2b93DmwkMubSsHEBBaTm11ABA7xvuRifoYzk+hiBn+i0m0pVG6L0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754420751; c=relaxed/simple; bh=0v4/4KHczFm86J8EpOGmKd1AC5YsaffazCEHV5VEVkw=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=HYTe5h1/R7D6KXVRiGcn5fOQP2gom/5Hk3kyArbUkq6rwejQlNJWZdQhzUROYiNcNjDP8qq5LvfakdstAhOW6AMuZVeRD97w0T5t7MkFvTFqkwd1g7H53SVpN5MoQFBSoaW7NCzCniA/GLaar4P6XiT8ncyQTDqNEtEZ2ya/NLM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=SrsmURyf; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="SrsmURyf" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-315af08594fso6200193a91.2 for ; Tue, 05 Aug 2025 12:05:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1754420749; x=1755025549; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=SMTNldksocyQikuDTQSTc2Wa+EvxEezz/3PYk/VDlp0=; b=SrsmURyfixCGZazaZxoxYydfyjMYIOhduTyUouqpsgK9VywERjLizkFZt2xTgH8T98 kNQINdClh6UVVKCyCJzvHcqe61T1AGTDIB9nkdI83/WgCcNbB+9+Zd8qF1gx22RfSruI PZvrcbKq+BxOeyuD3LJUcpBaQUIWAh4cf1TkR4GLJC+BfV2xvM9CvV7w4kmgAVJHpT0h Iq6LjmcxN9PxBsaBJu9BYPRpVfnRFoqPDfnkcySaM1EJhwE+JB9LkBIj/EJQNmA1yWhI hQvvrqkx1XP3xa/R9Bh/UQDZ06CIfeHZthnfePnOoe56C/YMhz2iosK8MIJT6Aa5GQeW zxZw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754420749; x=1755025549; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=SMTNldksocyQikuDTQSTc2Wa+EvxEezz/3PYk/VDlp0=; b=u0k/vDx3cpJK90qSY0xJpv8OpOPkxVvVr9Q6SjA5YahJtWzvLvi260zpVxo297f6WI vlbo+Cr7C91Jc5bPJt8kIPtVGszgeRc9qi3otnpuElM45mEcVdPzGH2xD11kFcqNC7D8 nUYYBAE7XaKuetZPNvmr8C8ocHPEjTdw8GOcM9JxVNvg56+vfkU+ghECReH1R7z8i1Hw BPt2z8KfqYcPOc0dhV4ZpzFgMBAl+Y+QIdFKw/QTuuePZr1gDWJRR4wAH8RDIgd7gqfn cj7BEFHl5H2nPFXcCSogEsiz3vVpdIjjQyMMM0ybWZh2TUBi4/n9daBApLHeb+AfGDNu LNlA== X-Forwarded-Encrypted: i=1; AJvYcCVZHXgzD+n7OPaIIVUbP5NBGCCyQPDscgU+Gb905R35drJ7AtDSp0rOwJF2eQCIT54Eqw3qMjYZyFCcMDE=@vger.kernel.org X-Gm-Message-State: AOJu0YyAOnhvJLolDDrhe8UTyWTkngu3kVxS/PUX15sGxNr+D53R0MW2 W4IRAtMTTUOMrNPRgp+RGzhB1tJxuKisZLSu1i2s8+R+HzwgWl5/su48Vs1URvi6qbHD5VvEBnt KxF+VAw== X-Google-Smtp-Source: AGHT+IEkU2DLpTTTF/CafhOsbR+sMu2xr0HobiD/z4Iv5/o9d7+Aigp4lVvqAMuxRXCbFsZ5KjBy8v86iD4= X-Received: from pjbsx8.prod.google.com ([2002:a17:90b:2cc8:b0:312:187d:382d]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:2e8d:b0:31f:336a:f0db with SMTP id 98e67ed59e1d1-3211620ad0fmr21125942a91.10.1754420748685; Tue, 05 Aug 2025 12:05:48 -0700 (PDT) Reply-To: Sean Christopherson Date: Tue, 5 Aug 2025 12:05:19 -0700 In-Reply-To: <20250805190526.1453366-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250805190526.1453366-1-seanjc@google.com> X-Mailer: git-send-email 2.50.1.565.gc32cd1483b-goog Message-ID: <20250805190526.1453366-12-seanjc@google.com> Subject: [PATCH 11/18] KVM: x86/pmu: Calculate set of to-be-emulated PMCs at time of WRMSRs From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Xin Li , Dapeng Mi , Sandipan Das Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Calculate and track PMCs that are counting instructions/branches retired when the PMC's event selector (or fixed counter control) is modified instead evaluating the event selector on-demand. Immediately recalc a PMC's configuration on writes to avoid false negatives/positives when KVM skips an emulated WRMSR, which is guaranteed to occur before the main run loop processes KVM_REQ_PMU. Out of an abundance of caution, and because it's relatively cheap, recalc reprogrammed PMCs in kvm_pmu_handle_event() as well. Recalculating in response to KVM_REQ_PMU _should_ be unnecessary, but for now be paranoid to avoid introducing easily-avoidable bugs in edge cases. The code can be removed in the future if necessary, e.g. in the unlikely event that the overhead of recalculating to-be-emulated PMCs is noticeable. Note! Deliberately don't check the PMU event filters, as doing so could result in KVM consuming stale information. Tracking which PMCs are counting branches/instructions will allow grabbing SRCU in the fastpath VM-Exit handlers if and only if a PMC event might be triggered (to consult the event filters), and will also allow the upcoming mediated PMU to do the right thing with respect to counting instructions (the mediated PMU won't be able to update PMCs in the VM-Exit fastpath). Signed-off-by: Sean Christopherson --- arch/x86/include/asm/kvm_host.h | 3 ++ arch/x86/kvm/pmu.c | 75 ++++++++++++++++++++++++--------- arch/x86/kvm/pmu.h | 4 ++ 3 files changed, 61 insertions(+), 21 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_hos= t.h index f19a76d3ca0e..d7680612ba1e 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -579,6 +579,9 @@ struct kvm_pmu { DECLARE_BITMAP(all_valid_pmc_idx, X86_PMC_IDX_MAX); DECLARE_BITMAP(pmc_in_use, X86_PMC_IDX_MAX); =20 + DECLARE_BITMAP(pmc_counting_instructions, X86_PMC_IDX_MAX); + DECLARE_BITMAP(pmc_counting_branches, X86_PMC_IDX_MAX); + u64 ds_area; u64 pebs_enable; u64 pebs_enable_rsvd; diff --git a/arch/x86/kvm/pmu.c b/arch/x86/kvm/pmu.c index e1911b366c43..b0f0275a2c2e 100644 --- a/arch/x86/kvm/pmu.c +++ b/arch/x86/kvm/pmu.c @@ -542,6 +542,47 @@ static int reprogram_counter(struct kvm_pmc *pmc) eventsel & ARCH_PERFMON_EVENTSEL_INT); } =20 +static bool pmc_is_event_match(struct kvm_pmc *pmc, u64 eventsel) +{ + /* + * Ignore checks for edge detect (all events currently emulated by KVM + * are always rising edges), pin control (unsupported by modern CPUs), + * and counter mask and its invert flag (KVM doesn't emulate multiple + * events in a single clock cycle). + * + * Note, the uppermost nibble of AMD's mask overlaps Intel's IN_TX (bit + * 32) and IN_TXCP (bit 33), as well as two reserved bits (bits 35:34). + * Checking the "in HLE/RTM transaction" flags is correct as the vCPU + * can't be in a transaction if KVM is emulating an instruction. + * + * Checking the reserved bits might be wrong if they are defined in the + * future, but so could ignoring them, so do the simple thing for now. + */ + return !((pmc->eventsel ^ eventsel) & AMD64_RAW_EVENT_MASK_NB); +} + +void kvm_pmu_recalc_pmc_emulation(struct kvm_pmu *pmu, struct kvm_pmc *pmc) +{ + bitmap_clear(pmu->pmc_counting_instructions, pmc->idx, 1); + bitmap_clear(pmu->pmc_counting_branches, pmc->idx, 1); + + /* + * Do NOT consult the PMU event filters, as the filters must be checked + * at the time of emulation to ensure KVM uses fresh information, e.g. + * omitting a PMC from a bitmap could result in a missed event if the + * filter is changed to allow counting the event. + */ + if (!pmc_speculative_in_use(pmc)) + return; + + if (pmc_is_event_match(pmc, kvm_pmu_eventsel.INSTRUCTIONS_RETIRED)) + bitmap_set(pmu->pmc_counting_instructions, pmc->idx, 1); + + if (pmc_is_event_match(pmc, kvm_pmu_eventsel.BRANCH_INSTRUCTIONS_RETIRED)) + bitmap_set(pmu->pmc_counting_branches, pmc->idx, 1); +} +EXPORT_SYMBOL_GPL(kvm_pmu_recalc_pmc_emulation); + void kvm_pmu_handle_event(struct kvm_vcpu *vcpu) { DECLARE_BITMAP(bitmap, X86_PMC_IDX_MAX); @@ -577,6 +618,9 @@ void kvm_pmu_handle_event(struct kvm_vcpu *vcpu) */ if (unlikely(pmu->need_cleanup)) kvm_pmu_cleanup(vcpu); + + kvm_for_each_pmc(pmu, pmc, bit, bitmap) + kvm_pmu_recalc_pmc_emulation(pmu, pmc); } =20 int kvm_pmu_check_rdpmc_early(struct kvm_vcpu *vcpu, unsigned int idx) @@ -910,7 +954,8 @@ static inline bool cpl_is_matched(struct kvm_pmc *pmc) select_user; } =20 -static void kvm_pmu_trigger_event(struct kvm_vcpu *vcpu, u64 eventsel) +static void kvm_pmu_trigger_event(struct kvm_vcpu *vcpu, + const unsigned long *event_pmcs) { DECLARE_BITMAP(bitmap, X86_PMC_IDX_MAX); struct kvm_pmu *pmu =3D vcpu_to_pmu(vcpu); @@ -919,29 +964,17 @@ static void kvm_pmu_trigger_event(struct kvm_vcpu *vc= pu, u64 eventsel) =20 BUILD_BUG_ON(sizeof(pmu->global_ctrl) * BITS_PER_BYTE !=3D X86_PMC_IDX_MA= X); =20 + if (bitmap_empty(event_pmcs, X86_PMC_IDX_MAX)) + return; + if (!kvm_pmu_has_perf_global_ctrl(pmu)) - bitmap_copy(bitmap, pmu->all_valid_pmc_idx, X86_PMC_IDX_MAX); - else if (!bitmap_and(bitmap, pmu->all_valid_pmc_idx, + bitmap_copy(bitmap, event_pmcs, X86_PMC_IDX_MAX); + else if (!bitmap_and(bitmap, event_pmcs, (unsigned long *)&pmu->global_ctrl, X86_PMC_IDX_MAX)) return; =20 kvm_for_each_pmc(pmu, pmc, i, bitmap) { - /* - * Ignore checks for edge detect (all events currently emulated - * but KVM are always rising edges), pin control (unsupported - * by modern CPUs), and counter mask and its invert flag (KVM - * doesn't emulate multiple events in a single clock cycle). - * - * Note, the uppermost nibble of AMD's mask overlaps Intel's - * IN_TX (bit 32) and IN_TXCP (bit 33), as well as two reserved - * bits (bits 35:34). Checking the "in HLE/RTM transaction" - * flags is correct as the vCPU can't be in a transaction if - * KVM is emulating an instruction. Checking the reserved bits - * might be wrong if they are defined in the future, but so - * could ignoring them, so do the simple thing for now. - */ - if (((pmc->eventsel ^ eventsel) & AMD64_RAW_EVENT_MASK_NB) || - !pmc_event_is_allowed(pmc) || !cpl_is_matched(pmc)) + if (!pmc_event_is_allowed(pmc) || !cpl_is_matched(pmc)) continue; =20 kvm_pmu_incr_counter(pmc); @@ -950,13 +983,13 @@ static void kvm_pmu_trigger_event(struct kvm_vcpu *vc= pu, u64 eventsel) =20 void kvm_pmu_instruction_retired(struct kvm_vcpu *vcpu) { - kvm_pmu_trigger_event(vcpu, kvm_pmu_eventsel.INSTRUCTIONS_RETIRED); + kvm_pmu_trigger_event(vcpu, vcpu_to_pmu(vcpu)->pmc_counting_instructions); } EXPORT_SYMBOL_GPL(kvm_pmu_instruction_retired); =20 void kvm_pmu_branch_retired(struct kvm_vcpu *vcpu) { - kvm_pmu_trigger_event(vcpu, kvm_pmu_eventsel.BRANCH_INSTRUCTIONS_RETIRED); + kvm_pmu_trigger_event(vcpu, vcpu_to_pmu(vcpu)->pmc_counting_branches); } EXPORT_SYMBOL_GPL(kvm_pmu_branch_retired); =20 diff --git a/arch/x86/kvm/pmu.h b/arch/x86/kvm/pmu.h index 740af816af37..cb93a936a177 100644 --- a/arch/x86/kvm/pmu.h +++ b/arch/x86/kvm/pmu.h @@ -176,8 +176,12 @@ extern struct x86_pmu_capability kvm_pmu_cap; =20 void kvm_init_pmu_capability(const struct kvm_pmu_ops *pmu_ops); =20 +void kvm_pmu_recalc_pmc_emulation(struct kvm_pmu *pmu, struct kvm_pmc *pmc= ); + static inline void kvm_pmu_request_counter_reprogram(struct kvm_pmc *pmc) { + kvm_pmu_recalc_pmc_emulation(pmc_to_pmu(pmc), pmc); + set_bit(pmc->idx, pmc_to_pmu(pmc)->reprogram_pmi); kvm_make_request(KVM_REQ_PMU, pmc->vcpu); } --=20 2.50.1.565.gc32cd1483b-goog From nobody Sun Oct 5 10:47:10 2025 Received: from mail-pg1-f201.google.com (mail-pg1-f201.google.com [209.85.215.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4546F2BE633 for ; Tue, 5 Aug 2025 19:05:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754420752; cv=none; b=EgmmKS6uW0bSbFZawF7vJNGZ0b6Xzthcty67UeLRHZeMfA03ft0o/CUp6Ni+TRu7hG26z/MphxgzsacnLbDm8O44NkXjvzIyD6TsE6ynNLO2TmzYgDC1yCwYQqX2gRqg3yxi7Dlb71NvXuctk9WHZm62qclSl+bfxx05FCsIpOY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754420752; c=relaxed/simple; bh=pnz5GPHbUL1uVz2QV1RJMlpLJa1Rveb6gZH923+QTvs=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=UoI8F+2KZan/Og12V5eW+z2VVuePEBWfyIU4vEEV0Xiwpl24z//xZvrxnRvoLuSRvn4CPBVPejzMbGbhX/H/huKhvBQtO69MsMV8QDFod3qulMF+7nqXMSuYmUDeUd3EpO72ljXp5kRkf/eSlxPndjERdAkFxp6HM4i7rUd4058= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=ey9SPOyK; arc=none smtp.client-ip=209.85.215.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="ey9SPOyK" Received: by mail-pg1-f201.google.com with SMTP id 41be03b00d2f7-b4227538a47so4044239a12.1 for ; Tue, 05 Aug 2025 12:05:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1754420751; x=1755025551; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=9grQ6DM7PkSOUdH/ZZWeVtWjpj5Tz/lOK2rUIx4UhFM=; b=ey9SPOyK0DebtnjLsryGCz1TvdIw5UIwB5fE6d99dVCB5RiXqblBkEgM7qe9C8TbJl nHHlloN8CsS49565roKY7E5EjCQFBpN7OWM80AMcezJj562cckopijXfKuTKOHu9muG7 Ih+Rs7+86qSLpMiJeEnMf4V1aqpjb/EY2Nnw0FCgZ0MvYMdW4NqiSqrAh3Kyy5CUWhFR hKMrStfgoGr6lSGrcCDQyvEZmh6QaW4j9PZyyJfPSY36xBLspglO6zy3CJYYif2yXdzP yNAYEcRive4GQqPHoEIb37q7vCrycKhLmoKIrG5auVOBaf+y9geO3NXzlDU/CrDQWtSQ DpOg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754420751; x=1755025551; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=9grQ6DM7PkSOUdH/ZZWeVtWjpj5Tz/lOK2rUIx4UhFM=; b=M/W16THb5ljgsVKO0bw5ZtuiSYQz4lVaIX+iTktVFAAQOaajYkhvZckBU+VNAQnhoF W1rOwglY90+j+NQ0g7uAhG+fwYS+rmZ0D820HTpQRNrL2vpqlDPFR0QNmTx0hFmGfrKE hwbL6wqL3APtDbeECRmxROfwDQaWtESHi7kpwHFxtWLfFhl6van8wmsr0J8c6l5rt47R 4AxnTmYVkhSGOPFRUKV112dPJ26gnE4M2CYeHBwTj49GTi01rdss7BDl4yHZWaatjNvA bvbh/HWe7aLvJXen0ZMq+b/g9aZgV2iPqAAOBBDrTfC7e3MuQByrGMLAmNFcyv615K+h UyWw== X-Forwarded-Encrypted: i=1; AJvYcCVRlpJ53mvki9RyNe1YPG6r7Uql4in1z4P9S2hF1awueL3v6qXjdXuJtnHZ1ISeDR6Aj5iN8Ss7BQ4+i7k=@vger.kernel.org X-Gm-Message-State: AOJu0Yx/LOl6y6SEeD7hvdSj8TK0lEXh3rpzwFeCy1Ocfdk+p9aMQBJY Vb/9BRml3M0GPNKRrv/61dTm3F3CaoDdGI1u8Kn4BG7/pQCkI0ghzDuoYF9ao/qzwequjZcTs8G wfL5how== X-Google-Smtp-Source: AGHT+IHWOk3hCecdCHsM4icWj0tVZamc2jPv94x5aLLpqfrfCuf/TvnX0Djxp+K6yNBHPlMk/y1/Xi9Nx8A= X-Received: from pjyt9.prod.google.com ([2002:a17:90a:e509:b0:31c:2fe4:33b7]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:1c92:b0:321:265a:e0b6 with SMTP id 98e67ed59e1d1-321265ae50emr14628340a91.20.1754420750687; Tue, 05 Aug 2025 12:05:50 -0700 (PDT) Reply-To: Sean Christopherson Date: Tue, 5 Aug 2025 12:05:20 -0700 In-Reply-To: <20250805190526.1453366-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250805190526.1453366-1-seanjc@google.com> X-Mailer: git-send-email 2.50.1.565.gc32cd1483b-goog Message-ID: <20250805190526.1453366-13-seanjc@google.com> Subject: [PATCH 12/18] KVM: x86/pmu: Rename pmc_speculative_in_use() to pmc_is_locally_enabled() From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Xin Li , Dapeng Mi , Sandipan Das Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Rename pmc_speculative_in_use() to pmc_is_locally_enabled() to better capture what it actually tracks, and to show its relationship to pmc_is_globally_enabled(). While neither AMD nor Intel refer to event selectors or the fixed counter control MSR as "local", it's the obvious name to pair with "global". As for "speculative", there's absolutely nothing speculative about the checks. E.g. for PMUs without PERF_GLOBAL_CTRL, from the guest's perspective, the counters are "in use" without any qualifications. No functional change intended. Signed-off-by: Sean Christopherson --- arch/x86/kvm/pmu.c | 6 +++--- arch/x86/kvm/pmu.h | 2 +- arch/x86/kvm/vmx/pmu_intel.c | 2 +- 3 files changed, 5 insertions(+), 5 deletions(-) diff --git a/arch/x86/kvm/pmu.c b/arch/x86/kvm/pmu.c index b0f0275a2c2e..e73c2a44028b 100644 --- a/arch/x86/kvm/pmu.c +++ b/arch/x86/kvm/pmu.c @@ -493,7 +493,7 @@ static bool check_pmu_event_filter(struct kvm_pmc *pmc) =20 static bool pmc_event_is_allowed(struct kvm_pmc *pmc) { - return pmc_is_globally_enabled(pmc) && pmc_speculative_in_use(pmc) && + return pmc_is_globally_enabled(pmc) && pmc_is_locally_enabled(pmc) && check_pmu_event_filter(pmc); } =20 @@ -572,7 +572,7 @@ void kvm_pmu_recalc_pmc_emulation(struct kvm_pmu *pmu, = struct kvm_pmc *pmc) * omitting a PMC from a bitmap could result in a missed event if the * filter is changed to allow counting the event. */ - if (!pmc_speculative_in_use(pmc)) + if (!pmc_is_locally_enabled(pmc)) return; =20 if (pmc_is_event_match(pmc, kvm_pmu_eventsel.INSTRUCTIONS_RETIRED)) @@ -907,7 +907,7 @@ void kvm_pmu_cleanup(struct kvm_vcpu *vcpu) pmu->pmc_in_use, X86_PMC_IDX_MAX); =20 kvm_for_each_pmc(pmu, pmc, i, bitmask) { - if (pmc->perf_event && !pmc_speculative_in_use(pmc)) + if (pmc->perf_event && !pmc_is_locally_enabled(pmc)) pmc_stop_counter(pmc); } =20 diff --git a/arch/x86/kvm/pmu.h b/arch/x86/kvm/pmu.h index cb93a936a177..08ae644db00e 100644 --- a/arch/x86/kvm/pmu.h +++ b/arch/x86/kvm/pmu.h @@ -160,7 +160,7 @@ static inline struct kvm_pmc *get_fixed_pmc(struct kvm_= pmu *pmu, u32 msr) return NULL; } =20 -static inline bool pmc_speculative_in_use(struct kvm_pmc *pmc) +static inline bool pmc_is_locally_enabled(struct kvm_pmc *pmc) { struct kvm_pmu *pmu =3D pmc_to_pmu(pmc); =20 diff --git a/arch/x86/kvm/vmx/pmu_intel.c b/arch/x86/kvm/vmx/pmu_intel.c index 0b173602821b..07baff96300f 100644 --- a/arch/x86/kvm/vmx/pmu_intel.c +++ b/arch/x86/kvm/vmx/pmu_intel.c @@ -762,7 +762,7 @@ void intel_pmu_cross_mapped_check(struct kvm_pmu *pmu) int bit, hw_idx; =20 kvm_for_each_pmc(pmu, pmc, bit, (unsigned long *)&pmu->global_ctrl) { - if (!pmc_speculative_in_use(pmc) || + if (!pmc_is_locally_enabled(pmc) || !pmc_is_globally_enabled(pmc) || !pmc->perf_event) continue; =20 --=20 2.50.1.565.gc32cd1483b-goog From nobody Sun Oct 5 10:47:10 2025 Received: from mail-pg1-f201.google.com (mail-pg1-f201.google.com [209.85.215.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1B7B42BE7A5 for ; Tue, 5 Aug 2025 19:05:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754420754; cv=none; b=Lm23k36VkOQ2zAsE+K5FM7DVjMCZ2LrIdSgDW2gKfcxbaRJMbREB/1ucxN4ptLDOlya1cEbpZcKcERsDbQBeZmX4EP/5Zld2oQtb+c4nwmCroY0U/YcXcJcN2LJehBM8TxSb9tRDiYEtAqWvX5pWIhxyhxjs4bkSg1YLcwOD6zM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754420754; c=relaxed/simple; bh=Kypk+/CIeqoZ/r5aB2dTj7/V8NMqnRxOmJHlZq0EdRE=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Hwfe8xmSewb7XFIckTcY5ozG4bpR6aoD43+P49uQYH3m+X/HdlTOl0zuHjAcs/k+1pP+hNNDYUmqX8A2xatH9HCbqdF2XDaoNvjtTHwGFwNVTPg2X+K3xnw+Zob4ya4VAveHdi+N8kSpVnbNOPFGAe3r7HWgoki0gR9WjkGFTGk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=BW51GNMW; arc=none smtp.client-ip=209.85.215.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="BW51GNMW" Received: by mail-pg1-f201.google.com with SMTP id 41be03b00d2f7-b38fc4d8dbaso5741144a12.2 for ; Tue, 05 Aug 2025 12:05:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1754420752; x=1755025552; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=fDzHgBJ4HosuuKeQovaY/hyA5EBdxNO5TIzhzBsDnxo=; b=BW51GNMWVH2YnQaW3EAEKBoEmp2xSsSuFdC1JyAmya3G5dvRygJX2mHw7svqqzWbgE 6HV3QWdYCrlSq6I1vSmG2o7xnknLcj1YkTqTr/cNXB3V8B+OyYlI77MPqJZPf41Ka9wa xof8napwLg5sCt7w8nbQJvhjK+pd6oi8sxksS0V8r3PHoy/ME8c5iLnC1dCofLFEjhK0 cD4NqM3A/2ZZzJoU3G9nE9AHAxIOGAuhjH4CGzAtIjDb7nvYq0PAKY9YTTTha5Q9TQ7H naVoZ4E3EAsvJilbDjInYGadF6E4XdB0QckCaaXA8fsJSYejV+MkBUKFjbkNjgP0GMPA vVZw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754420752; x=1755025552; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=fDzHgBJ4HosuuKeQovaY/hyA5EBdxNO5TIzhzBsDnxo=; b=uB0VmnUn1vZydHiH9Owyh+SBhLrjU53VEJcpFfxRLkQCrQgHG9qgTz1qo1zYyO5/hA DvRJAB/b97jNM0A+S6bTKnOcfNbKPQqVxJUaNoxHkproy9sRPaPoAnsn+exM2bz04BCA 6oodpR4KWfLJl1a+W+mB3Lhjw2lQSa98f7OqPGO2O7CTcBjKxJlZYIJ3mcc+skWeWvHI HwRtkhjmYq0VEUMdADi2Z064aLoT8Ii/humbJcd8T8tbaRiQu3HMmpO99OprdkmXfuh5 JJPIPZGYgylH/KmwKYRbClD4FAr63Zk3nVa3R7sMUCq4C7rDeLQBy38GueSvN5y95yfd rQ+A== X-Forwarded-Encrypted: i=1; AJvYcCVQHXFfL+bnih50spqv/VwH2ovasPXNxxxBzl5uEEWb06/1PWSAPTc+634hvMiaryGYKVYiK1v3spccgaY=@vger.kernel.org X-Gm-Message-State: AOJu0YxHy3BdsoHINr7RbDhhUv8FF0j1u6EspBxaDtC5E+SRNDidUa/q ynPkfnz0cGJw4h0uDR5oSf5dlkymPpU/d15tte36DMeKfFA5unhbFnfDjxMrlKyVzLwzegCDDiA 70hC0Lg== X-Google-Smtp-Source: AGHT+IFMZ78t6jlZ4Hg4XAsyFyrTTPw2ebUp6YO0LiIwWC4Id16cUZUsvZRcvyi7Iqfg/ZZ9YEb5MYML3L8= X-Received: from pjbpa2.prod.google.com ([2002:a17:90b:2642:b0:311:7d77:229f]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:1f85:b0:31f:3cfd:d334 with SMTP id 98e67ed59e1d1-321161f06d3mr19905789a91.4.1754420752515; Tue, 05 Aug 2025 12:05:52 -0700 (PDT) Reply-To: Sean Christopherson Date: Tue, 5 Aug 2025 12:05:21 -0700 In-Reply-To: <20250805190526.1453366-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250805190526.1453366-1-seanjc@google.com> X-Mailer: git-send-email 2.50.1.565.gc32cd1483b-goog Message-ID: <20250805190526.1453366-14-seanjc@google.com> Subject: [PATCH 13/18] KVM: x86/pmu: Open code pmc_event_is_allowed() in its callers From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Xin Li , Dapeng Mi , Sandipan Das Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Open code pmc_event_is_allowed() in its callers, as kvm_pmu_trigger_event() only needs to check the event filter (both global and local enables are consulted outside of the loop). No functional change intended. Signed-off-by: Sean Christopherson --- arch/x86/kvm/pmu.c | 12 ++++-------- 1 file changed, 4 insertions(+), 8 deletions(-) diff --git a/arch/x86/kvm/pmu.c b/arch/x86/kvm/pmu.c index e73c2a44028b..a495ab5d0556 100644 --- a/arch/x86/kvm/pmu.c +++ b/arch/x86/kvm/pmu.c @@ -491,12 +491,6 @@ static bool check_pmu_event_filter(struct kvm_pmc *pmc) return is_fixed_event_allowed(filter, pmc->idx); } =20 -static bool pmc_event_is_allowed(struct kvm_pmc *pmc) -{ - return pmc_is_globally_enabled(pmc) && pmc_is_locally_enabled(pmc) && - check_pmu_event_filter(pmc); -} - static int reprogram_counter(struct kvm_pmc *pmc) { struct kvm_pmu *pmu =3D pmc_to_pmu(pmc); @@ -507,7 +501,8 @@ static int reprogram_counter(struct kvm_pmc *pmc) =20 emulate_overflow =3D pmc_pause_counter(pmc); =20 - if (!pmc_event_is_allowed(pmc)) + if (!pmc_is_globally_enabled(pmc) || !pmc_is_locally_enabled(pmc) || + !check_pmu_event_filter(pmc)) return 0; =20 if (emulate_overflow) @@ -974,7 +969,8 @@ static void kvm_pmu_trigger_event(struct kvm_vcpu *vcpu, return; =20 kvm_for_each_pmc(pmu, pmc, i, bitmap) { - if (!pmc_event_is_allowed(pmc) || !cpl_is_matched(pmc)) + if (!pmc_is_globally_enabled(pmc) || !pmc_is_locally_enabled(pmc) || + !check_pmu_event_filter(pmc) || !cpl_is_matched(pmc)) continue; =20 kvm_pmu_incr_counter(pmc); --=20 2.50.1.565.gc32cd1483b-goog From nobody Sun Oct 5 10:47:10 2025 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D29692BEC22 for ; Tue, 5 Aug 2025 19:05:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754420756; cv=none; b=Ru3dilp3zpp7R4l6YGZk7HrMyMmPfRBBtcOZMizkVqmP7stnLhxLvB5JxsI8HoqC7TT4fexBWhCmwyiIOi6qi4B4OyalABry90o9TJ9r+CHD+M3lxCSPcwfSXtz8r/+mMwgMGTGYQPKpNIzm4VBhAweMtGJ/DaCMIWwc6NenBOc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754420756; c=relaxed/simple; bh=Uf3LhARRUHUx/5ijEPMCZbQc7umhxBRmBlCRrbgyJB8=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=JyzeCAjjJXBSd9TOBw9QyHlUopwC0jY/EUE9zgNXuHMbdCMNYO0rHFRF3UxXQGl2OUHfNU4wsmospn1AJlDb1PzFKJ0ealVZuxXW+1GMMLxlRO+QTbvgFYkHSfBTcU/z9w9/le++8tJ+ODhkC1/53Ozp2At83dPo/uKzmq0I96I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=ltajJh7k; arc=none smtp.client-ip=209.85.214.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="ltajJh7k" Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-23fc5b1c983so1827465ad.0 for ; Tue, 05 Aug 2025 12:05:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1754420754; x=1755025554; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=m7pIQtdgO1Ly4runtD1BlGYWjUan7Zr3ARLYIITYVLU=; b=ltajJh7ka2OJ7IlnELzoncqvpzb1jLBEG//8fpeI4FYeeS6Fl3FkPLzSR9bDckywuv Lu83HzthbEMspV8eJMwyR8OU+R6DtExqQ2sGbD8q1KsMHDg9HeSXaetEeeH1UOG/hzi9 Mixrt31O7ObMqt9rHwO3R2pTjQwYqd2GPjXeNtlEJJt1KMsW+S/Uxe15pOzG2i9uFDwy uNclZibmi+0ie03rvSDSJT2miYFHUjlDIskIRcFMijbPAImeZHH5+NmBj/uB9I9eaQO/ LdTcxkYEXZvigOqb6BZzUUvyEQFUr8wHDVeAJw7fSe5q1i/LUNrpBfQ2+NPIMv2+rdcJ DrDQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754420754; x=1755025554; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=m7pIQtdgO1Ly4runtD1BlGYWjUan7Zr3ARLYIITYVLU=; b=gQIPmNtyoCvVjeIQpTok5hxlM2BO0J0iP52+CKXiDJJvVzJbLPkiykEehA8q0fiu41 g0b5Wr8Dsym/AQNKs06GJvztRKJvH9VMd2T4Uz+zkWEddZQspqvnB8ss6OdjxBXq2c65 r0YN08LeTh+l2lWvjwaSDXpUC/euAH/x48fCADApt7whslmOYpAYeI6REcHOh0hMeZsy E0g2qsNQJbLIspJ1BHhZfB9nE5LPSikiEmmw0CasZnOSPTE7Y6bmXLuxGhSKEinEn3QL zrYnz3EbGl1O1MCwnkDS3jpuFKpzlf8fCkYCxjfH5hIM1Y9vr6E6ikfGXPMJoUBMhfom 02QA== X-Forwarded-Encrypted: i=1; AJvYcCWjbd9UQg5imAKrM6I1kZMnK4HgC5De3QY41Yaz4z1Zny6R7kLPZb2QGaioebVkA7lOGH5TUz4f197XEeQ=@vger.kernel.org X-Gm-Message-State: AOJu0YwslREzrZ1TpB1/0tAy4MYxcakVTq0z8eJUP2D0PDIQs65DvyW9 b1FP55P1tjyAd4vVtT1bkXN49Uo9ZdcSJq3QW1Q6sAuuJGkdzJeItWlUy5HJTg+Dbb6fYa0OYzP 8C3ygjA== X-Google-Smtp-Source: AGHT+IG3+BUJeywCnhbOTBO0gZnJzDqaPsYALbJj+kP15NYqy9TsM5+IMTgOn1X2lnbVEgPLhVTAv8iMyVE= X-Received: from plblf6.prod.google.com ([2002:a17:902:fb46:b0:240:1be2:19ee]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:902:ef4e:b0:240:86b2:ae9c with SMTP id d9443c01a7336-2429f959d4dmr859115ad.14.1754420754232; Tue, 05 Aug 2025 12:05:54 -0700 (PDT) Reply-To: Sean Christopherson Date: Tue, 5 Aug 2025 12:05:22 -0700 In-Reply-To: <20250805190526.1453366-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250805190526.1453366-1-seanjc@google.com> X-Mailer: git-send-email 2.50.1.565.gc32cd1483b-goog Message-ID: <20250805190526.1453366-15-seanjc@google.com> Subject: [PATCH 14/18] KVM: x86/pmu: Drop redundant check on PMC being globally enabled for emulation From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Xin Li , Dapeng Mi , Sandipan Das Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When triggering PMC events in response to emulation, drop the redundant checks on a PMC being globally and locally enabled, as the passed in bitmap contains only PMCs that are locally enabled (and counting the right event), and the local copy of the bitmap has already been masked with global_ctrl. No true functional change intended. Signed-off-by: Sean Christopherson --- arch/x86/kvm/pmu.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/kvm/pmu.c b/arch/x86/kvm/pmu.c index a495ab5d0556..bdcd9c6f0ec0 100644 --- a/arch/x86/kvm/pmu.c +++ b/arch/x86/kvm/pmu.c @@ -969,7 +969,7 @@ static void kvm_pmu_trigger_event(struct kvm_vcpu *vcpu, return; =20 kvm_for_each_pmc(pmu, pmc, i, bitmap) { - if (!pmc_is_globally_enabled(pmc) || !pmc_is_locally_enabled(pmc) || + if (!pmc_is_locally_enabled(pmc) || !check_pmu_event_filter(pmc) || !cpl_is_matched(pmc)) continue; =20 --=20 2.50.1.565.gc32cd1483b-goog From nobody Sun Oct 5 10:47:10 2025 Received: from mail-pg1-f202.google.com (mail-pg1-f202.google.com [209.85.215.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C63932BF00E for ; Tue, 5 Aug 2025 19:05:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754420759; cv=none; b=piIShSHXI+YaedU3oZIbAqcVMPw7dLvUPBkz8xuji74DdVeeeKhUSlrjQsIWY69oiUzLXJ9/M0oXi+HKJD8t/JdrzpTq9o8EcU90yEi2bCMptsnETwMy4dpHyaORYLrYUtG3N4QM6KkmHnFiMomxjxhQmYIYpAUvJTPWL2p+YD0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754420759; c=relaxed/simple; bh=/sPXmM0Zs1fLp6muGCf+tjOXJkyB32siq6QuACEEbu4=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=AH7v+6YXAZsTN5nLLTng3F2liLRRqY483U4g/eY3PgCxg/hYpxRQXVLrDQ0/LVl9oYhRmMQdw1iZXRMqlMx+7eUbn+dQ64F3S47cA8hUCRM5zuyLyrAwZesm6kaBKQdIWMVY0CtX1dZNFlV64Fv8kpfSqHgqy7814UozL5gaeTA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=3Do8gQd6; arc=none smtp.client-ip=209.85.215.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="3Do8gQd6" Received: by mail-pg1-f202.google.com with SMTP id 41be03b00d2f7-b4225ab1829so3350779a12.0 for ; Tue, 05 Aug 2025 12:05:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1754420757; x=1755025557; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=oCGEKAkbeIoGipEXNV8T0RUXqCHxNcix8LcZk/J0Axs=; b=3Do8gQd6Zgufyho4I057T0TokdKhRR4BSHkUQNflkztGtIWnCwyPowTcQjJvbIS1lO vRwOUze9Lmc6ECoi6TW0iaJUdJUp6oscjVyCsu1v/Tx+7VsNkAck/U0LCo4cWrjYORap bsf3MzCp+BtlPYnSt0xvOnJZiOHrtqXOAOoIgkHdrE/hkwdhRXT8/Yl58SyxQZlfzKXh 3ywmuOkJw/xzIiNoxNOnYNuLOTRPZPRfCyj3BGITlQPcgNFYxpX22MhbA/681QpEdQfI 31MqCq1759KB4YXfLElR13Lm6HLJAkAHTWn1VjKMk3G3q7lB5A66629iyQKI98NDb38m vUHw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754420757; x=1755025557; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=oCGEKAkbeIoGipEXNV8T0RUXqCHxNcix8LcZk/J0Axs=; b=wsXIrcbLqAqGshSXSAZ58JpN7kDlTRVYF5WFMASiW2l+UUg8TNzHUKY2KoULSSVHj+ rqYgKYSUl3oAK8VoXjydcjSwsB3anMN33C9EJf5sxDSx9a7tHYVkyn8AIbW3rHTISVFv aZ6zH0KXCNAUQNT1ZQU2z/hFXrBlbsnVdt1vVGP2akg1Woz4YNeuX+kQZyq/PfaidP4c MpQf0N4nWMEkH3hl9Rut5gBrWnCGqWWgwWiaMkKDNb/zneuoJ+FdM5DcqBtyfkUkATPj AHO9SlyeI/koBxXm+SB5WPzYjONhMZfk7olq9kPQZDU8WkiTt+KS0vHYyDJRkTqhi/hs ilLA== X-Forwarded-Encrypted: i=1; AJvYcCV77SELPuJtfl/MSTm+X80h1v0+JXFYueZO1EBBGbt/6Wpa8HwSdzPPTdQiLAirQjVNbA0/e0x5rVDn174=@vger.kernel.org X-Gm-Message-State: AOJu0YxC5cMSwOb9h6uSn8CeLh7J7p63Q07/mueSZ8uUt+/yVViNifO3 qcoAAiKF05KMqWZe9vMpMnC+o0N+je0wp//HEF89Y98rGsvGmcvN0fdThMzf+rjTdKH5jGdxNWw fJFJaLg== X-Google-Smtp-Source: AGHT+IElV3TzNFEvi73astSTm6isgDxLFlXWng4KtkNOcbY0eY9l7+S3hsyx3YXtIUtoz742t5l8ACYhINA= X-Received: from pgah7.prod.google.com ([2002:a05:6a02:4e87:b0:b2c:3d70:9c1]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a20:5481:b0:240:a06:7837 with SMTP id adf61e73a8af0-2403146a502mr275717637.17.1754420757037; Tue, 05 Aug 2025 12:05:57 -0700 (PDT) Reply-To: Sean Christopherson Date: Tue, 5 Aug 2025 12:05:23 -0700 In-Reply-To: <20250805190526.1453366-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250805190526.1453366-1-seanjc@google.com> X-Mailer: git-send-email 2.50.1.565.gc32cd1483b-goog Message-ID: <20250805190526.1453366-16-seanjc@google.com> Subject: [PATCH 15/18] KVM: x86/pmu: Drop redundant check on PMC being locally enabled for emulation From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Xin Li , Dapeng Mi , Sandipan Das Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Drop the check on a PMC being locally enabled when triggering emulated events, as the bitmap of passed-in PMCs only contains locally enabled PMCs. Signed-off-by: Sean Christopherson --- arch/x86/kvm/pmu.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/arch/x86/kvm/pmu.c b/arch/x86/kvm/pmu.c index bdcd9c6f0ec0..422af7734846 100644 --- a/arch/x86/kvm/pmu.c +++ b/arch/x86/kvm/pmu.c @@ -969,8 +969,7 @@ static void kvm_pmu_trigger_event(struct kvm_vcpu *vcpu, return; =20 kvm_for_each_pmc(pmu, pmc, i, bitmap) { - if (!pmc_is_locally_enabled(pmc) || - !check_pmu_event_filter(pmc) || !cpl_is_matched(pmc)) + if (!check_pmu_event_filter(pmc) || !cpl_is_matched(pmc)) continue; =20 kvm_pmu_incr_counter(pmc); --=20 2.50.1.565.gc32cd1483b-goog From nobody Sun Oct 5 10:47:10 2025 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 587E02BF3F3 for ; Tue, 5 Aug 2025 19:05:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754420760; cv=none; b=BrciVHrcqonY6bJnGRWMUQvRVXoPhwjw95xdP+CYiqzpT98Dtt9+oHPCTtnAL9eL08VwbE6CmOi4J2creexrekBIwSKSEkWqzeVtoRO6jREhSMXvlSpwjgznG/SDP6dIXgFP4DzdmN3h8tqDv9BsZ0euYXsBSPpNJ1Sb5D8VKe8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754420760; c=relaxed/simple; bh=psR4I+ufoU1UhwaQI/3V99Tsfg4a+RYPNxW/6rK3XCY=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=CPrisVB41sHi2xIbGnzErKg8uWtHxfgMZpRExMBdG+D7jNxIz5C11ugnTkg6kVdVY+35XLQWe5W2CKjCCtQU0ZF+t7M3xjwb0/qJrXPZzD+SYL9Yp+44QtZpLLWj05w3rzU5PX64h5G3yX0197HXCt/wAGNV21D3Cq7yHRxJZqY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=BHvibVWH; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="BHvibVWH" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-31eec17b5acso5935150a91.2 for ; Tue, 05 Aug 2025 12:05:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1754420759; x=1755025559; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=fLHs8RygRrQALNjclLr7Le+Q1ExkVvDCVfxeia7/R9A=; b=BHvibVWH2qGtT8e/EATDkj92rZiu/diVQdVrdpQ/JZ42ePuvIpEfLGm7F4WEOscr9o DSWdVUOoT67XuIA5q6hzq4P4XfTBzzfQl4nelbgRvEG+fjBWaHbwzvlBkVGjf+BGGKjx QuuCJHC93Hl7uwb68O0tymVLda0NSRdY4FY9F7BHqQ8Kkpjgydn0W0AvH0p1I9Iu6rKG yVsg9r6Nk8TRsVcvyOG0let7NhDdD5RN1FXGMqabQ5WzifPD0H5zRzoycQOYZCU1smyp B7OU5/oOAnZkQM1SQDp+ehSxNgIBuq6+XmKt8B6MLy3vor38CSy3hOl1qERvb8p6wPUK UBqA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754420759; x=1755025559; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=fLHs8RygRrQALNjclLr7Le+Q1ExkVvDCVfxeia7/R9A=; b=Y9nSH+WvUalPdTkl0ppR00ec8LE8kF5eWrBwg08LcYY3lE0y31M36wpdzICdqhWT0a G1OyBwm43Uk00J174zbFg4Eca7JbxxEX8zrQZceFnOMC+tsDHNzzVIPNq9tJns+QIk/N 3kQTdbWrrqfbpdiUdeexCzqZDWsgdjtuzAkpxB5UyVUhBZERK5jQZGFGKWhru3K4/KaW NtJYT147xI2RkghHiD+8GskMx7V8jUC58TGwWLnWXBxe2GcnMuCIHVOlzqxTYGoppI7r H+0yUvQp3BE43DXXR1VZnv8I8Nl89jjZMSw7z2THY40txjetIBM/oSMTxNdBud3ewvIn k/+w== X-Forwarded-Encrypted: i=1; AJvYcCUTNaUczRmvubAY/z9iD51wUEw+XdTrz+fS6OroT5HpU6697htbz+i3EzJE1Nt8IuDLwKUhTl2/ytdY3zw=@vger.kernel.org X-Gm-Message-State: AOJu0Yw5X9qEsxBukFRkoPkMVqIdmTTvLPStc0dl4PPjFbg2Me2XqJCB NOAi/YUBV/12umHOjt4qeoloPaVGlcvzWd5FfFKL6gSzL3SLHWSdJxVe1aoguTRF/kL81Ep3p3B Yate+TQ== X-Google-Smtp-Source: AGHT+IHhRzbfPkZZCi4sJ+HYHlO8wPFG7lfHkjOAeauEAzcP+gDuNgftd87bIo4kHYo6mkitvsEho/wi4LQ= X-Received: from pjxx5.prod.google.com ([2002:a17:90b:58c5:b0:31f:b2f:aeed]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:1b4f:b0:31f:30a6:56ff with SMTP id 98e67ed59e1d1-321162a204fmr20394494a91.19.1754420758726; Tue, 05 Aug 2025 12:05:58 -0700 (PDT) Reply-To: Sean Christopherson Date: Tue, 5 Aug 2025 12:05:24 -0700 In-Reply-To: <20250805190526.1453366-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250805190526.1453366-1-seanjc@google.com> X-Mailer: git-send-email 2.50.1.565.gc32cd1483b-goog Message-ID: <20250805190526.1453366-17-seanjc@google.com> Subject: [PATCH 16/18] KVM: x86/pmu: Rename check_pmu_event_filter() to pmc_is_event_allowed() From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Xin Li , Dapeng Mi , Sandipan Das Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Rename check_pmu_event_filter() to make its polarity more obvious, and to connect the dots to is_gp_event_allowed() and is_fixed_event_allowed(). No functional change intended. Signed-off-by: Sean Christopherson --- arch/x86/kvm/pmu.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/arch/x86/kvm/pmu.c b/arch/x86/kvm/pmu.c index 422af7734846..e75671b6e88c 100644 --- a/arch/x86/kvm/pmu.c +++ b/arch/x86/kvm/pmu.c @@ -476,7 +476,7 @@ static bool is_fixed_event_allowed(struct kvm_x86_pmu_e= vent_filter *filter, return true; } =20 -static bool check_pmu_event_filter(struct kvm_pmc *pmc) +static bool pmc_is_event_allowed(struct kvm_pmc *pmc) { struct kvm_x86_pmu_event_filter *filter; struct kvm *kvm =3D pmc->vcpu->kvm; @@ -502,7 +502,7 @@ static int reprogram_counter(struct kvm_pmc *pmc) emulate_overflow =3D pmc_pause_counter(pmc); =20 if (!pmc_is_globally_enabled(pmc) || !pmc_is_locally_enabled(pmc) || - !check_pmu_event_filter(pmc)) + !pmc_is_event_allowed(pmc)) return 0; =20 if (emulate_overflow) @@ -969,7 +969,7 @@ static void kvm_pmu_trigger_event(struct kvm_vcpu *vcpu, return; =20 kvm_for_each_pmc(pmu, pmc, i, bitmap) { - if (!check_pmu_event_filter(pmc) || !cpl_is_matched(pmc)) + if (!pmc_is_event_allowed(pmc) || !cpl_is_matched(pmc)) continue; =20 kvm_pmu_incr_counter(pmc); --=20 2.50.1.565.gc32cd1483b-goog From nobody Sun Oct 5 10:47:10 2025 Received: from mail-pg1-f202.google.com (mail-pg1-f202.google.com [209.85.215.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2F3B12C08BF for ; Tue, 5 Aug 2025 19:06:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754420762; cv=none; b=FV7pepXNYmsUAzp4i3ZdZOzSZ4xgyeMDX8pEW+TD3pdv9bmQnndMyDM9//OIMMhGzJldTTJJJFp4AvcoU53zkOChlO4fhh4RvjhPPBQtqjG4LmYgZXVb3s42t+d86btFRA7x5rqBcor3hm23qZ0nVwFMdbeaWjlnHHAwjTg8g0U= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754420762; c=relaxed/simple; bh=zATxkyxAcSPZiTxDepTukQWJ8dyUc3y2yARhHF2lPNo=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Lxy4F/w+/3F6bBC0cUzPaV07GMdqzxd+AcEkTcD7vPbsq0wKwQDqoYMf3i+w7y41v+s8Hg4aVzbLAgrSICJr7UYvZMxc39pVYSTJBRDwZiwERZSs13yya4Yk9uBwEYDrBpWbd4cBrpeVG06LmOqLfucObStCjyvVkzizlcJZ0dE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=VpnuU6x7; arc=none smtp.client-ip=209.85.215.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="VpnuU6x7" Received: by mail-pg1-f202.google.com with SMTP id 41be03b00d2f7-b42249503c4so141494a12.0 for ; Tue, 05 Aug 2025 12:06:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1754420760; x=1755025560; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=rAVrGWcmedyfCvbeUeNYdLWZXibA/eVVDu2b3ZtfBaQ=; b=VpnuU6x7oAbvKQP03oy+uT1Rs2ZLS9gMTtf9f8mCuuyLgzNF9LYGz9Z6fYj59xoxHi Q+IExm9N8nxQA2tAiB00E4A6/Z0KAf9ntbOH3A1qDizz60t53PCv4mI3ApxfIsweoRLK kt++a8JWNVZQMeaeriMXw0izO0TpLtIBDTtcPvUrvIQtR2AiaxVALWcYdvJ8w/RZq3GO FH5aAOORD4PnK0PgwXO6qIR5BuF+N5KxNC8+46rXHYLYSHPoQQWtCNpl5du5QRHR4w2Y 3ZloOqSHOcvz8Z0WD3JSlARlvUInFs0zG5f5LeQ92klXwGkiOATBgA+czQU4ZYPE8eI3 54+A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754420760; x=1755025560; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=rAVrGWcmedyfCvbeUeNYdLWZXibA/eVVDu2b3ZtfBaQ=; b=RTKzDlcd52f8JlbHgPFqQIsk9L+ylVZxNmwwXpnvFUsnSZhRKkoLc4cXmLfis6Ewv7 dwrmEbpy06NNE4LmqJ2LazbNE01NbceRASXO3P8KkiS0eaXlBhd8pwwizb709fp0Obdh uIqnWhm3De043Amtg/N7k6fxMcGq2ET3ps7KBfK5T2LtpLAy3Csn3QKIYSXkxXVbwTDH vN4GTl5t3lwrvbpOOs+4x4qS3xGQoi3kaL54KxsMVwqqYWfHXPFCU90bdTXTSoGpgzUa eYuh6lsuPTptOsYsSSzvvrGD5sS6qaugWWgHX9jZg9XCStvr415iRMGAXYdU9nqf4E1F sucg== X-Forwarded-Encrypted: i=1; AJvYcCVcXEbaEHc9KCAVSWxlnJD7Xk/Wn7P7w49U8iegRz4WT112FtZy9u03RJXC6UnMzJU8yVg9sEhq7rEXXXo=@vger.kernel.org X-Gm-Message-State: AOJu0YySflzU+BBjvMcNWJYUkt1sxeRtlMB+oC0QkyIC+IDGX4BadFCC 6i0oK9c+tyHXYh/TQ4Uvzz0ua+420m4XKNMTsX0BRpx55kWxUnzPbfEpU09WT0PIa3urZQqMyrC 1m00JGw== X-Google-Smtp-Source: AGHT+IHN8xHb/VjKiH3ppGdR8gJCBUsLr/MHH+tOxG5LkRUI154GMU25wC6+IauyXC0h6zXwo/6t8nUNWuw= X-Received: from pjsk12.prod.google.com ([2002:a17:90a:62cc:b0:31f:37f:d381]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:568d:b0:31f:ecf:36f with SMTP id 98e67ed59e1d1-3216684b48dmr113350a91.1.1754420760530; Tue, 05 Aug 2025 12:06:00 -0700 (PDT) Reply-To: Sean Christopherson Date: Tue, 5 Aug 2025 12:05:25 -0700 In-Reply-To: <20250805190526.1453366-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250805190526.1453366-1-seanjc@google.com> X-Mailer: git-send-email 2.50.1.565.gc32cd1483b-goog Message-ID: <20250805190526.1453366-18-seanjc@google.com> Subject: [PATCH 17/18] KVM: x86: Push acquisition of SRCU in fastpath into kvm_pmu_trigger_event() From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Xin Li , Dapeng Mi , Sandipan Das Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Acquire SRCU in the VM-Exit fastpath if and only if KVM needs to check the PMU event filter, to further trim the amount of code that is executed with SRCU protection in the fastpath. Counter-intuitively, holding SRCU can do more harm than good due to masking potential bugs, and introducing a new SRCU-protected asset to code reachable via kvm_skip_emulated_instruction() would be quite notable, i.e. definitely worth auditing. E.g. the primary user of kvm->srcu is KVM's memslots, accessing memslots all but guarantees guest memory may be accessed, accessing guest memory can fault, and page faults might sleep, which isn't allowed while IRQs are disabled. Not acquiring SRCU means the (hypothetical) illegal sleep would be flagged when running with PROVE_RCU=3Dy, even if DEBUG_ATOMIC_SLEEP=3Dn. Note, performance is NOT a motivating factor, as SRCU lock/unlock only adds ~15 cycles of latency to fastpath VM-Exits. I.e. overhead isn't a concern _if_ SRCU protection needs to be extended beyond PMU events, e.g. to honor userspace MSR filters. Signed-off-by: Sean Christopherson --- arch/x86/kvm/pmu.c | 4 +++- arch/x86/kvm/x86.c | 18 +++++------------- 2 files changed, 8 insertions(+), 14 deletions(-) diff --git a/arch/x86/kvm/pmu.c b/arch/x86/kvm/pmu.c index e75671b6e88c..3206412a35a1 100644 --- a/arch/x86/kvm/pmu.c +++ b/arch/x86/kvm/pmu.c @@ -955,7 +955,7 @@ static void kvm_pmu_trigger_event(struct kvm_vcpu *vcpu, DECLARE_BITMAP(bitmap, X86_PMC_IDX_MAX); struct kvm_pmu *pmu =3D vcpu_to_pmu(vcpu); struct kvm_pmc *pmc; - int i; + int i, idx; =20 BUILD_BUG_ON(sizeof(pmu->global_ctrl) * BITS_PER_BYTE !=3D X86_PMC_IDX_MA= X); =20 @@ -968,12 +968,14 @@ static void kvm_pmu_trigger_event(struct kvm_vcpu *vc= pu, (unsigned long *)&pmu->global_ctrl, X86_PMC_IDX_MAX)) return; =20 + idx =3D srcu_read_lock(&vcpu->kvm->srcu); kvm_for_each_pmc(pmu, pmc, i, bitmap) { if (!pmc_is_event_allowed(pmc) || !cpl_is_matched(pmc)) continue; =20 kvm_pmu_incr_counter(pmc); } + srcu_read_unlock(&vcpu->kvm->srcu, idx); } =20 void kvm_pmu_instruction_retired(struct kvm_vcpu *vcpu) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index f2b2eaaec6f8..a56f83b40a55 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -2137,7 +2137,6 @@ fastpath_t handle_fastpath_set_msr_irqoff(struct kvm_= vcpu *vcpu) { u64 data =3D kvm_read_edx_eax(vcpu); u32 msr =3D kvm_rcx_read(vcpu); - int r; =20 switch (msr) { case APIC_BASE_MSR + (APIC_ICR >> 4): @@ -2152,13 +2151,12 @@ fastpath_t handle_fastpath_set_msr_irqoff(struct kv= m_vcpu *vcpu) return EXIT_FASTPATH_NONE; } =20 - kvm_vcpu_srcu_read_lock(vcpu); - r =3D kvm_skip_emulated_instruction(vcpu); - kvm_vcpu_srcu_read_unlock(vcpu); - trace_kvm_msr_write(msr, data); =20 - return r ? EXIT_FASTPATH_REENTER_GUEST : EXIT_FASTPATH_EXIT_USERSPACE; + if (!kvm_skip_emulated_instruction(vcpu)) + return EXIT_FASTPATH_EXIT_USERSPACE; + + return EXIT_FASTPATH_REENTER_GUEST; } EXPORT_SYMBOL_GPL(handle_fastpath_set_msr_irqoff); =20 @@ -11251,13 +11249,7 @@ EXPORT_SYMBOL_GPL(kvm_emulate_halt); =20 fastpath_t handle_fastpath_hlt(struct kvm_vcpu *vcpu) { - int ret; - - kvm_vcpu_srcu_read_lock(vcpu); - ret =3D kvm_emulate_halt(vcpu); - kvm_vcpu_srcu_read_unlock(vcpu); - - if (!ret) + if (!kvm_emulate_halt(vcpu)) return EXIT_FASTPATH_EXIT_USERSPACE; =20 if (kvm_vcpu_running(vcpu)) --=20 2.50.1.565.gc32cd1483b-goog From nobody Sun Oct 5 10:47:10 2025 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0C8DC2C1596 for ; Tue, 5 Aug 2025 19:06:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754420764; cv=none; b=N5sTcvpS8k9HnaOtQwnLcclY8oCfXzvPRMarDD0vE6IL62SFzK2ybUyf00KgAfFuiQCLbFjS9W7IQXGnnQUUUyNr1PpltASBc3ih+lL/yfEoYdJ2MMm/gEjfkwZWh//3oiPhWQdqmFvhwWMoW/bUKBcFUSByYbNKulzWS9egFk4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754420764; c=relaxed/simple; bh=6eCmSKhIpszLUkxsSJHASWeflETYxy1FzP1/dimvvmQ=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=e1J/OfwKsc1C/nmk1OXJeTQ9ZkRrn8W6rDOn4qD0V25qeO/B2po0QEZoS0RLPkC25QOVCpKTuu3joeWBU2H+WLVr1c1Ak1LBr+yQ5oWFh0A6b3gtzct6/bQpE8Vu6nfBzRrR8bL8/laMQIri8oXqbuLEJiU+ne6XZbMXMA+nYUc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=QFxEOMC1; arc=none smtp.client-ip=209.85.214.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="QFxEOMC1" Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-23fd8c99dbfso47145795ad.2 for ; Tue, 05 Aug 2025 12:06:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1754420762; x=1755025562; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=jYPOGKxz/YMagjLqc/SXFdXBFj+5MGEvuF5f/vLTJD0=; b=QFxEOMC1jWlIZ7oAvYjs9TX2WLj4RaTOngtBRRXK1YJbbbvwahutSqPS3Kll1HXCa7 11s59NiOKf9bsdsUDpzzUE1/NvWCD+RGu+/mU6Vo0kL2EJKvDIsJAahdPIajDQ2iHAV6 UtLtUt5S8+ScV/4O1Bf2X+0lxnR0ulmPvxcJWnRr3qWWsvRBlexaRGkDE54jgCKaiEdO vV7mQy38ltvQ4aDuRHUYqVlfjpu7QYoieo6T6Ui4ZBD6mdUd8ZynnJy+UqcC2etxd4E1 l1yVEOZS8avkclYc8hdoYFNdXkF3PUwnK7dqUOJznYv/E/tGOzw9LzFgaQdT+LnQU1pN 7P9Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754420762; x=1755025562; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=jYPOGKxz/YMagjLqc/SXFdXBFj+5MGEvuF5f/vLTJD0=; b=klxMNkLTcmKw09V7LNGz4yL1xmqThGxgtHErA+S/IAEIKlKJtkw2MKcZii1Chs1chN DUh9IbWzoW9K3UwiO4z4u5IAe/Kltt9+wFG9I6688JuJDk8cPtnNFhXhIkaC0soToKnq 3uxkQZyLzUj5MKem6TKpNy9c0cTIKS878IMhRlyPJ6KLnJwqkOszz6qwrOfgj9QBHqy+ 7tvrDLpoOOooneFY1pLI1SUnbNf7xbHPqu+WCrsU0HvzGqrKnrK4EFJrdGWayo0GFMaZ bOcJ1xLQesTOy9faxZ0BKPo7MmqhxT+JYFcsywrRT47Deo7mJeFryURYP8RCHU/eCyNv xzSw== X-Forwarded-Encrypted: i=1; AJvYcCWwkStOrOBiljiPzZr5Qi81oZLQrtZuHvzWkicn3IHt9i85VOWKOsOJB5dnQi05NbL6nzuu18SXN6FQBY0=@vger.kernel.org X-Gm-Message-State: AOJu0YyUzc9BRPzrz2dL4WbsQwcNzkXqyOqQOw6c41j4iSpzNxDFbFPO ciXRCsDincgJ1UvBS5cWqgxT1ZieauPRX/kwQNNR9w2h8mMcozGeS+y/4X6xHZHfSO0RLhmz7Ji 24jheBA== X-Google-Smtp-Source: AGHT+IHbdxTlG7hgheXP+mYT9oIu4lCPyl255ao0Ex9Ybo3KzljkvgO/MKcbZ0k0rRO+Jid7Q1iCdyPOclQ= X-Received: from plcx21.prod.google.com ([2002:a17:903:d5:b0:240:8fa2:15c3]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:903:1b30:b0:240:11be:4dd3 with SMTP id d9443c01a7336-2429f521ac4mr2366575ad.3.1754420762334; Tue, 05 Aug 2025 12:06:02 -0700 (PDT) Reply-To: Sean Christopherson Date: Tue, 5 Aug 2025 12:05:26 -0700 In-Reply-To: <20250805190526.1453366-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250805190526.1453366-1-seanjc@google.com> X-Mailer: git-send-email 2.50.1.565.gc32cd1483b-goog Message-ID: <20250805190526.1453366-19-seanjc@google.com> Subject: [PATCH 18/18] KVM: x86: Add a fastpath handler for INVD From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Xin Li , Dapeng Mi , Sandipan Das Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add a fastpath handler for INVD so that the common fastpath logic can be trivially tested on both Intel and AMD. Under KVM, INVD is always: (a) intercepted, (b) available to the guest, and (c) emulated as a nop, with no side effects. Combined with INVD not having any inputs or outputs, i.e. no register constraints, INVD is the perfect instruction for exercising KVM's fastpath as it can be inserted into practically any guest-side code stream. Signed-off-by: Sean Christopherson --- arch/x86/kvm/svm/svm.c | 2 ++ arch/x86/kvm/vmx/vmx.c | 2 ++ arch/x86/kvm/x86.c | 9 +++++++++ arch/x86/kvm/x86.h | 1 + 4 files changed, 14 insertions(+) diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index 829d9d46718d..f7e1e665a826 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c @@ -4200,6 +4200,8 @@ static fastpath_t svm_exit_handlers_fastpath(struct k= vm_vcpu *vcpu) return handle_fastpath_set_msr_irqoff(vcpu); case SVM_EXIT_HLT: return handle_fastpath_hlt(vcpu); + case SVM_EXIT_INVD: + return handle_fastpath_invd(vcpu); default: break; } diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index aa157fe5b7b3..95765db52992 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -7175,6 +7175,8 @@ static fastpath_t vmx_exit_handlers_fastpath(struct k= vm_vcpu *vcpu, return handle_fastpath_preemption_timer(vcpu, force_immediate_exit); case EXIT_REASON_HLT: return handle_fastpath_hlt(vcpu); + case EXIT_REASON_INVD: + return handle_fastpath_invd(vcpu); default: return EXIT_FASTPATH_NONE; } diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index a56f83b40a55..5af2c5aed0f2 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -2086,6 +2086,15 @@ int kvm_emulate_invd(struct kvm_vcpu *vcpu) } EXPORT_SYMBOL_GPL(kvm_emulate_invd); =20 +fastpath_t handle_fastpath_invd(struct kvm_vcpu *vcpu) +{ + if (!kvm_emulate_invd(vcpu)) + return EXIT_FASTPATH_EXIT_USERSPACE; + + return EXIT_FASTPATH_REENTER_GUEST; +} +EXPORT_SYMBOL_GPL(handle_fastpath_invd); + int kvm_handle_invalid_op(struct kvm_vcpu *vcpu) { kvm_queue_exception(vcpu, UD_VECTOR); diff --git a/arch/x86/kvm/x86.h b/arch/x86/kvm/x86.h index bcfd9b719ada..46220b04cdf2 100644 --- a/arch/x86/kvm/x86.h +++ b/arch/x86/kvm/x86.h @@ -439,6 +439,7 @@ int x86_emulate_instruction(struct kvm_vcpu *vcpu, gpa_= t cr2_or_gpa, int emulation_type, void *insn, int insn_len); fastpath_t handle_fastpath_set_msr_irqoff(struct kvm_vcpu *vcpu); fastpath_t handle_fastpath_hlt(struct kvm_vcpu *vcpu); +fastpath_t handle_fastpath_invd(struct kvm_vcpu *vcpu); =20 extern struct kvm_caps kvm_caps; extern struct kvm_host_values kvm_host; --=20 2.50.1.565.gc32cd1483b-goog