From nobody Fri Apr 10 00:55:58 2026 Received: from fra-out-006.esa.eu-central-1.outbound.mail-perimeter.amazon.com (fra-out-006.esa.eu-central-1.outbound.mail-perimeter.amazon.com [18.197.217.180]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C8975372B28; Wed, 4 Mar 2026 16:50:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=18.197.217.180 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772643058; cv=none; b=qZjAJY8A6Ltnwzp2mhVIgMw/fQNkDC82HTYExqU66Wf5GcQwQMcX+B/pKo+BOzCd/8lUeD+y7W8gjdJd+DQVhossLJFUOPTfFgeO0z5v3nb7dku+4tqcyZ+NR9bfjC7AUIz9D+ObUJZ9hl1ar9mDL7HotcLjAFjoy0Cy1a8m2qM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772643058; c=relaxed/simple; bh=TSJEzLisChRwIKbS4tNTrQ3lRrAttafPT50LExE7m2I=; h=From:To:CC:Subject:Date:Message-ID:MIME-Version:Content-Type; b=ui5FUAksPBvxoz6Oi/LTs07FXhRQsj0nIx9KAnyp0ZtPE7GK1HfumKBVOgLisdK3cCPTaCcCES79Y0qr3r4BFN1A8aH0ME/W04EjcMyci80xw95nVEMHbrAo/6AAofMFVIy6LIAe26hWedeXmz+wizNQVKmyfDopEfM3bLDqZ8w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amazon.de; spf=pass smtp.mailfrom=amazon.de; dkim=pass (2048-bit key) header.d=amazon.de header.i=@amazon.de header.b=SUcVizHl; arc=none smtp.client-ip=18.197.217.180 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amazon.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=amazon.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=amazon.de header.i=@amazon.de header.b="SUcVizHl" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.de; i=@amazon.de; q=dns/txt; s=amazoncorp2; t=1772643056; x=1804179056; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=TSJEzLisChRwIKbS4tNTrQ3lRrAttafPT50LExE7m2I=; b=SUcVizHlFfCUKYUexf36/Y9Wvn9AW6ZLEZWU283DZqwuwSQkilw4In0u 0FJtQabKVndOE0RFtP/9TyON1306Qx7yoFpkzIxV6GGeDeXv/yh5Baxly vA0f5eMwaoxT0joUXk0xRxQmTCYDKCmNLC9T6Ky4bYfuPJv9kPk4z0ZuH EoDyotncUyh/jp3Z6NGVjuHIEebGln5QShhU2uyUqmNdPEWLvPRUyNmOW 3XcewUgw5AIciNCmct913UiqKle80suh5nItJhpGeYrQaSWZhqZoVp99I yBTijLP8FWl3fHylGPEzs0ie+h4OsOnC2c0ptxby2X01tQcU/m4UNRMJP Q==; X-CSE-ConnectionGUID: xHqRYF9pR42yiWvNAbl2Vg== X-CSE-MsgGUID: 08s3AjMGRtuBrNR4iOuN8A== X-IronPort-AV: E=Sophos;i="6.21,324,1763424000"; d="scan'208";a="10326686" Received: from ip-10-6-3-216.eu-central-1.compute.internal (HELO smtpout.naws.eu-central-1.prod.farcaster.email.amazon.dev) ([10.6.3.216]) by internal-fra-out-006.esa.eu-central-1.outbound.mail-perimeter.amazon.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Mar 2026 16:50:52 +0000 Received: from EX19MTAEUC002.ant.amazon.com [54.240.197.228:27252] by smtpin.naws.eu-central-1.prod.farcaster.email.amazon.dev [10.0.35.199:2525] with esmtp (Farcaster) id 2fc45bde-6d52-4055-b37a-7f01721fad01; Wed, 4 Mar 2026 16:50:52 +0000 (UTC) X-Farcaster-Flow-ID: 2fc45bde-6d52-4055-b37a-7f01721fad01 Received: from EX19D006EUC001.ant.amazon.com (10.252.51.203) by EX19MTAEUC002.ant.amazon.com (10.252.51.245) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA) id 15.2.2562.37; Wed, 4 Mar 2026 16:50:51 +0000 Received: from dev-dsk-dssauerw-1b-2c5f429c.eu-west-1.amazon.com (10.13.238.31) by EX19D006EUC001.ant.amazon.com (10.252.51.203) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA) id 15.2.2562.37; Wed, 4 Mar 2026 16:50:49 +0000 From: David Sauerwein To: Sean Christopherson , Paolo Bonzini CC: , , David Woodhouse , , David Sauerwein , =?UTF-8?q?Jan=20H=2E=20Sch=C3=B6nherr?= Subject: [PATCH] x86: kvm: Initialize static calls before SMP boot Date: Wed, 4 Mar 2026 16:50:12 +0000 Message-ID: <20260304165012.13660-1-dssauerw@amazon.de> X-Mailer: git-send-email 2.47.3 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: EX19D039UWB001.ant.amazon.com (10.13.138.119) To EX19D006EUC001.ant.amazon.com (10.252.51.203) Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Updating static calls is expensive on wide SMP systems because all online CPUs need to act in a coordinated manner for code patching to work as expected. Static are initialized only after SMP boot, where the code patching overhead is noticeable. Pre-initialize the majority of these static calls before SMP boot to get rid of that overhead. The pre-initialization most likely already sets the correct value. To still handle any potential differences, make the static call initialization re-entrant so that the differences are corrected during post-SMP-boot initialization. Values that are already set correctly are skipped. This patch was tested on a 6th Gen Intel Xeon Platinum 8375C CPU with 128 SMT cores. When comparing before and after we see that 85% less time is spent in kvm_ops_update (29.5ms -> 4.2ms). Co-developed-by: Jan H. Sch=C3=B6nherr Signed-off-by: Jan H. Sch=C3=B6nherr Signed-off-by: David Sauerwein --- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/svm/svm.c | 11 +++++++++++ arch/x86/kvm/vmx/vmx.c | 18 ++++++++++++++++++ arch/x86/kvm/x86.c | 19 ++++++++++++++++++- kernel/events/core.c | 6 +++++- 5 files changed, 53 insertions(+), 2 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_hos= t.h index 48598d017d6f..18072d7bed36 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -2000,6 +2000,7 @@ extern struct kvm_x86_ops kvm_x86_ops; #define KVM_X86_OP_OPTIONAL_RET0 KVM_X86_OP #include =20 +int kvm_x86_vendor_init_early(struct kvm_x86_init_ops *ops); int kvm_x86_vendor_init(struct kvm_x86_init_ops *ops); void kvm_x86_vendor_exit(void); =20 diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index 9d29b2e7e855..de7aa4d1f0e6 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c @@ -5452,6 +5452,17 @@ static void __svm_exit(void) kvm_x86_vendor_exit(); } =20 +#ifndef MODULE +static int __init svm_ops_early_init(void) +{ + if (!kvm_is_svm_supported()) + return -EOPNOTSUPP; + + return kvm_x86_vendor_init_early(&svm_init_ops); +} +early_initcall(svm_ops_early_init); +#endif + static int __init svm_init(void) { int r; diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 91b6f2f3edc2..569545854e16 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -8617,6 +8617,8 @@ __init int vmx_hardware_setup(void) vt_x86_ops.set_hv_timer =3D NULL; vt_x86_ops.cancel_hv_timer =3D NULL; } + WARN_ON(enable_preemption_timer && vt_x86_ops.set_hv_timer =3D=3D NULL); + WARN_ON(enable_preemption_timer && vt_x86_ops.cancel_hv_timer =3D=3D NULL= ); =20 kvm_caps.supported_mce_cap |=3D MCG_LMCE_P; kvm_caps.supported_mce_cap |=3D MCG_CMCI_P; @@ -8698,6 +8700,22 @@ void vmx_exit(void) kvm_x86_vendor_exit(); } =20 +#ifndef MODULE +static int __init vmx_ops_early_init(void) +{ + if (!kvm_is_vmx_supported()) + return -EOPNOTSUPP; + + if (!enable_preemption_timer) { + vt_x86_ops.set_hv_timer =3D NULL; + vt_x86_ops.cancel_hv_timer =3D NULL; + } + + return kvm_x86_vendor_init_early(&vt_init_ops); +} +early_initcall(vmx_ops_early_init); +#endif + int __init vmx_init(void) { int r, cpu; diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index c9c2aa6f4705..dbd00a26538f 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -9971,6 +9971,22 @@ static void kvm_x86_check_cpu_compat(void *ret) *(int *)ret =3D kvm_x86_check_processor_compatibility(); } =20 +int kvm_x86_vendor_init_early(struct kvm_x86_init_ops *ops) +{ + guard(mutex)(&vendor_module_lock); + + if (kvm_x86_ops.enable_virtualization_cpu) { + pr_err("already loaded vendor module '%s'\n", kvm_x86_ops.name); + return -EEXIST; + } + + kvm_ops_update(ops); + kvm_register_perf_callbacks(ops->handle_intel_pt_intr); + + return 0; +} +EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_x86_vendor_init_early); + int kvm_x86_vendor_init(struct kvm_x86_init_ops *ops) { u64 host_pat; @@ -9978,7 +9994,8 @@ int kvm_x86_vendor_init(struct kvm_x86_init_ops *ops) =20 guard(mutex)(&vendor_module_lock); =20 - if (kvm_x86_ops.enable_virtualization_cpu) { + if (kvm_x86_ops.enable_virtualization_cpu && + kvm_x86_ops.enable_virtualization_cpu !=3D ops->runtime_ops->enable_v= irtualization_cpu) { pr_err("already loaded vendor module '%s'\n", kvm_x86_ops.name); return -EEXIST; } diff --git a/kernel/events/core.c b/kernel/events/core.c index 2c35acc2722b..731975791895 100644 --- a/kernel/events/core.c +++ b/kernel/events/core.c @@ -7378,7 +7378,8 @@ DEFINE_STATIC_CALL_RET0(__perf_guest_handle_intel_pt_= intr, *perf_guest_cbs->hand =20 void perf_register_guest_info_callbacks(struct perf_guest_info_callbacks *= cbs) { - if (WARN_ON_ONCE(rcu_access_pointer(perf_guest_cbs))) + if (WARN_ON_ONCE(rcu_access_pointer(perf_guest_cbs) && + rcu_access_pointer(perf_guest_cbs) !=3D cbs)) return; =20 rcu_assign_pointer(perf_guest_cbs, cbs); @@ -7389,6 +7390,9 @@ void perf_register_guest_info_callbacks(struct perf_g= uest_info_callbacks *cbs) if (cbs->handle_intel_pt_intr) static_call_update(__perf_guest_handle_intel_pt_intr, cbs->handle_intel_pt_intr); + else + static_call_update(__perf_guest_handle_intel_pt_intr, + (void *)__static_call_return0); } EXPORT_SYMBOL_GPL(perf_register_guest_info_callbacks); =20 --=20 2.47.3 Amazon Web Services Development Center Germany GmbH Tamara-Danz-Str. 13 10243 Berlin Geschaeftsfuehrung: Christof Hellmis, Andreas Stieger Eingetragen am Amtsgericht Charlottenburg unter HRB 257764 B Sitz: Berlin Ust-ID: DE 365 538 597