From nobody Mon Aug 24 19:52:35 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=reject dis=none) header.from=google.com ARC-Seal: i=1; a=rsa-sha256; t=1782934591; cv=none; d=zohomail.com; s=zohoarc; b=M2FvPRNNNXAdL0d1VWF1RkQ+t19vFEQi+erCAEk2teOYUXgn6H5m9sssSg/wDF+EFT7M7IrPHPbL4BK39wF8upBJZuOeQjb1yEBb9GwOIPhfysKs5OTZMaM+EpcvtI7QIUrzwZGQ2M4J5LsMArM9wzfGZg/r7s1+CR26K/0w1Qw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1782934591; h=Content-Type:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Reply-To:Reply-To:References:Sender:Subject:Subject:To:To:Message-Id; bh=YK2zVmNLEJoBEawF7PlPSBbbkhRpzdy0apLvLWVt+lo=; b=UWZDaX10wsHzd8FOWRDLNXyNAIOE34Iwn94SlX4Vq8Grj5ukuEA6hHBZ4K91l/4zQD61UOtY3pdRdhB6gigQVWKsiiu6XMaTg7BDhdAlZhisssRsG919KaQgG7T7mI548ssmVoLaiY3BJAeSBn/vq5fYRkHoZQYaolH7g+iGhyA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1782934591496816.6843045205176; Wed, 1 Jul 2026 12:36:31 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1350740.1608250 (Exim 4.92) (envelope-from ) id 1wf0jB-00056V-Lu; Wed, 01 Jul 2026 19:36:09 +0000 Received: by outflank-mailman (output) from mailman id 1350740.1608250; Wed, 01 Jul 2026 19:36:09 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1wf0jB-00056J-HW; Wed, 01 Jul 2026 19:36:09 +0000 Received: by outflank-mailman (input) for mailman id 1350740; Wed, 01 Jul 2026 19:36:08 +0000 Received: from mx.expurgate.net ([194.145.224.10]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from <3b2tFagYKCbQmYUhdWaiiafY.WigrYh-XYpYffcmnm.rYhjlidYWn.ila@flex--seanjc.bounces.google.com>) id 1wf0jA-000535-D5 for xen-devel@lists.xenproject.org; Wed, 01 Jul 2026 19:36:08 +0000 Received: from mx.expurgate.net (helo=localhost) by mx.expurgate.net with esmtp id 1wf0j9-00Bwbb-QG for xen-devel@lists.xenproject.org; Wed, 01 Jul 2026 21:36:07 +0200 Received: from [10.42.69.5] (helo=localhost) by localhost with ESMTP (eXpurgate MTA 0.9.1) (envelope-from <3b2tFagYKCbQmYUhdWaiiafY.WigrYh-XYpYffcmnm.rYhjlidYWn.ila@flex--seanjc.bounces.google.com>) id 6a456c14-5cb7-0a2a0a5109dd-0a2a4505baa4-32 for ; Wed, 01 Jul 2026 21:36:07 +0200 Received: from [209.85.216.74] (helo=mail-pj1-f74.google.com) by tlsNG-c201ff.mxtls.expurgate.net with ESMTPS (eXpurgate 4.57.1) (envelope-from <3b2tFagYKCbQmYUhdWaiiafY.WigrYh-XYpYffcmnm.rYhjlidYWn.ila@flex--seanjc.bounces.google.com>) id 6a456b70-3cb2-0a2a45050019-d155d84ae8e7-3 for ; Wed, 01 Jul 2026 21:33:05 +0200 Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-37fccad2b01so1710983a91.2 for ; Wed, 01 Jul 2026 12:33:04 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" Authentication-Results: eu.smtp.expurgate.cloud; dkim=pass header.s=20251104 header.d=google.com header.i="@google.com" header.h="Cc:To:From:Subject:Message-ID:References:Mime-Version:In-Reply-To:Date:Reply-To" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1782934383; x=1783539183; darn=lists.xenproject.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=YK2zVmNLEJoBEawF7PlPSBbbkhRpzdy0apLvLWVt+lo=; b=GfPMSMmuPk3zh8O5e84EIZ6P/vKNB7Eyzg2O/pzTKiZJ5+ulRzUqGZa/OLYHl7XdB5 EKZ7abIj+8dtJJ2qh98qtNXdClw+ERNQLjGUi7NIeqikuiYmznNKa0WkypvEbWKYhq++ r2a9s75lRvNwb7YwVHyvlq44LpvZ7GKKKdHCsr49c21xXvJm16E6wTt9Zh9ogft4ty4I ItJov+7e1/dYoO1A+M3nbuz5ONeWGQt/aQscE1TRPDlZnKBOdO5SnWmCQtTmGLLsgxeJ ozpyJCFJLQ99ybhoiGWHRpxH6sckAF5WHfViZfxFcsxmtTZNBYWMUwWBetNA0zBjSBqY x54Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1782934383; x=1783539183; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=YK2zVmNLEJoBEawF7PlPSBbbkhRpzdy0apLvLWVt+lo=; b=WSAcXYIeSdEVYhKY+uGGhocrUuqskQIvJn07RYO/mdBIFyEJmDezVGHx72l/g+k+/c FDHE+nTSShGSkh4Chh9Dn+OSYBPeoLoypoctrDe1baZQrmqiZz/WkFvc0Jz0thWr//oI pZjHpT8KbWW6VQNFTOros/vRsURHEksg3AGYBdUEhBZ4F8QUZbp/u14M0d1GI9fA0Drd OMHtFwdLcH9YElQtTxDZGLR/41PaiX0iCC/qEBmACwZNY8ulcRbJc7Qz4OVRDHieIQCV MATnlQh5hlABEtZONO+LHQlSrrjIs+/4DDfZOFUWo9xMdp1LmIHfOq+i9mizpcVfES48 Bh+g== X-Forwarded-Encrypted: i=1; AFNElJ9HhiwmbRhvuJUlzwLs8YKy2CJPOcDVSWMOcaIhTVBjGo3qYB98CVA/VwZdl8horqq25gwPfhv9AZ4=@lists.xenproject.org X-Gm-Message-State: AOJu0YyeSCV+jVNKfLKoJqTOfSCQHcX552R3klRFWmTBt5zh7msYKhgb kkLwbrPq+la3pNSZNZWJRjmfGh+49J0lFdnP994tHKsIrgO1ah8+SaEetcavxImj8fMtO9z8/Ou SN0Y3Qg== X-Received: from pgwa6.prod.google.com ([2002:a65:6546:0:b0:c99:cbb4:1dcb]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a20:a11b:b0:3b2:924c:567d with SMTP id adf61e73a8af0-3bff42bedd3mr2415279637.46.1782934383031; Wed, 01 Jul 2026 12:33:03 -0700 (PDT) Reply-To: Sean Christopherson Date: Wed, 1 Jul 2026 12:31:50 -0700 In-Reply-To: <20260701193212.749551-1-seanjc@google.com> Mime-Version: 1.0 References: <20260701193212.749551-1-seanjc@google.com> X-Mailer: git-send-email 2.55.0.rc0.799.gd6f94ed593-goog Message-ID: <20260701193212.749551-30-seanjc@google.com> Subject: [PATCH v5 29/51] x86/kvm: Don't disable kvmclock on BSP in syscore_suspend() From: Sean Christopherson To: Jonathan Corbet , Paolo Bonzini , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, Kiryl Shutsemau , Rick Edgecombe , Sean Christopherson , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Long Li , Ajay Kaher , Alexey Makhalov , Jan Kiszka , Andy Lutomirski , Peter Zijlstra , Juergen Gross , Daniel Lezcano , John Stultz Cc: Shuah Khan , "H. Peter Anvin" , Vitaly Kuznetsov , Broadcom internal kernel review list , Boris Ostrovsky , Stephen Boyd , linux-doc@vger.kernel.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, linux-hyperv@vger.kernel.org, virtualization@lists.linux.dev, xen-devel@lists.xenproject.org, Tom Lendacky , Nikunj A Dadhania , David Woodhouse , David Woodhouse , Michael Kelley , Thomas Gleixner X-purgate-ID: tlsNG-c201ff/1782934385-169102B8-DC817DCC/13/0 X-purgate-type: clean X-purgate-size: 5417 X-ZohoMail-DKIM: pass (identity @google.com) X-ZM-MESSAGEID: 1782934593461158500 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Don't disable kvmclock on the BSP during syscore_suspend(), as the BSP's clock is NOT restored during syscore_resume(), but is instead restored earlier via the sched_clock restore callback. If suspend is aborted, e.g. due to a late wakeup, the BSP will run without its clock enabled, which "works" only because KVM-the-hypervisor is kind enough to not clobber the shared memory when the clock is disabled. But over time, the BSP's view of time will drift from APs. Plumb in an "action" to KVM-as-a-guest and kvmclock code in preparation for additional cleanups to kvmclock's suspend/resume logic. Fixes: c02027b5742b ("x86/kvm: Disable kvmclock on all CPUs on shutdown") Cc: stable@vger.kernel.org Reviewed-by: David Woodhouse Signed-off-by: Sean Christopherson --- arch/x86/include/asm/kvm_para.h | 8 +++++++- arch/x86/kernel/kvm.c | 15 ++++++++------- arch/x86/kernel/kvmclock.c | 31 +++++++++++++++++++++++++------ 3 files changed, 40 insertions(+), 14 deletions(-) diff --git a/arch/x86/include/asm/kvm_para.h b/arch/x86/include/asm/kvm_par= a.h index 4a49fc286b4c..08686ff19caa 100644 --- a/arch/x86/include/asm/kvm_para.h +++ b/arch/x86/include/asm/kvm_para.h @@ -118,8 +118,14 @@ static inline long kvm_sev_hypercall3(unsigned int nr,= unsigned long p1, } =20 #ifdef CONFIG_KVM_GUEST +enum kvm_guest_cpu_action { + KVM_GUEST_BSP_SUSPEND, + KVM_GUEST_AP_OFFLINE, + KVM_GUEST_SHUTDOWN, +}; + void kvmclock_init(bool prefer_tsc); -void kvmclock_disable(void); +void kvmclock_cpu_action(enum kvm_guest_cpu_action action); bool kvm_para_available(void); unsigned int kvm_arch_para_features(void); unsigned int kvm_arch_para_hints(void); diff --git a/arch/x86/kernel/kvm.c b/arch/x86/kernel/kvm.c index 6c7011ff7bd1..604b52f233aa 100644 --- a/arch/x86/kernel/kvm.c +++ b/arch/x86/kernel/kvm.c @@ -460,7 +460,7 @@ static void __init sev_map_percpu_data(void) } } =20 -static void kvm_guest_cpu_offline(bool shutdown) +static void kvm_guest_cpu_offline(enum kvm_guest_cpu_action action) { kvm_disable_steal_time(); if (kvm_para_has_feature(KVM_FEATURE_PV_EOI)) @@ -468,9 +468,10 @@ static void kvm_guest_cpu_offline(bool shutdown) if (kvm_para_has_feature(KVM_FEATURE_MIGRATION_CONTROL)) wrmsrq(MSR_KVM_MIGRATION_CONTROL, 0); kvm_pv_disable_apf(); - if (!shutdown) + if (action !=3D KVM_GUEST_SHUTDOWN) apf_task_wake_all(); - kvmclock_disable(); + + kvmclock_cpu_action(action); } =20 static int kvm_cpu_online(unsigned int cpu) @@ -726,7 +727,7 @@ static int kvm_cpu_down_prepare(unsigned int cpu) unsigned long flags; =20 local_irq_save(flags); - kvm_guest_cpu_offline(false); + kvm_guest_cpu_offline(KVM_GUEST_AP_OFFLINE); local_irq_restore(flags); return 0; } @@ -737,7 +738,7 @@ static int kvm_suspend(void *data) { u64 val =3D 0; =20 - kvm_guest_cpu_offline(false); + kvm_guest_cpu_offline(KVM_GUEST_BSP_SUSPEND); =20 #ifdef CONFIG_ARCH_CPUIDLE_HALTPOLL if (kvm_para_has_feature(KVM_FEATURE_POLL_CONTROL)) @@ -768,7 +769,7 @@ static struct syscore kvm_syscore =3D { =20 static void kvm_pv_guest_cpu_reboot(void *unused) { - kvm_guest_cpu_offline(true); + kvm_guest_cpu_offline(KVM_GUEST_SHUTDOWN); } =20 static int kvm_pv_reboot_notify(struct notifier_block *nb, @@ -792,7 +793,7 @@ static struct notifier_block kvm_pv_reboot_nb =3D { #ifdef CONFIG_CRASH_DUMP static void kvm_crash_shutdown(struct pt_regs *regs) { - kvm_guest_cpu_offline(true); + kvm_guest_cpu_offline(KVM_GUEST_SHUTDOWN); native_machine_crash_shutdown(regs); } #endif diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c index b0c871ba8232..a3ec298d56d7 100644 --- a/arch/x86/kernel/kvmclock.c +++ b/arch/x86/kernel/kvmclock.c @@ -199,8 +199,22 @@ static void kvm_register_clock(char *txt) pr_debug("kvm-clock: cpu %d, msr %llx, %s", smp_processor_id(), pa, txt); } =20 +static void kvmclock_disable(void) +{ + if (msr_kvm_system_time) + native_write_msr(msr_kvm_system_time, 0); +} + static void kvm_save_sched_clock_state(void) { + /* + * Stop host writes to kvmclock immediately prior to suspend/hibernate. + * If the system is hibernating, then kvmclock will likely reside at a + * different physical address when the system awakens, and host writes + * to the old address prior to reconfiguring kvmclock would clobber + * random memory. + */ + kvmclock_disable(); } =20 static void kvm_restore_sched_clock_state(void) @@ -208,6 +222,17 @@ static void kvm_restore_sched_clock_state(void) kvm_register_clock("primary cpu clock, resume"); } =20 +void kvmclock_cpu_action(enum kvm_guest_cpu_action action) +{ + /* + * Don't disable kvmclock on the BSP during suspend. If kvmclock is + * being used for sched_clock, then it needs to be kept alive until the + * last minute, and restored as quickly as possible after resume. + */ + if (action !=3D KVM_GUEST_BSP_SUSPEND) + kvmclock_disable(); +} + #ifdef CONFIG_SMP static void kvm_setup_secondary_clock(void) { @@ -215,12 +240,6 @@ static void kvm_setup_secondary_clock(void) } #endif =20 -void kvmclock_disable(void) -{ - if (msr_kvm_system_time) - native_write_msr(msr_kvm_system_time, 0); -} - static void __init kvmclock_init_mem(void) { unsigned long ncpus; --=20 2.55.0.rc0.799.gd6f94ed593-goog