From nobody Sun Feb 8 11:45:18 2026 Received: from pdx-out-001.esa.us-west-2.outbound.mail-perimeter.amazon.com (pdx-out-001.esa.us-west-2.outbound.mail-perimeter.amazon.com [44.245.243.92]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 98B4E39C636; Thu, 15 Jan 2026 14:15:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=44.245.243.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1768486528; cv=none; b=JhOFHsylzRLy/YMWIhioUhUwcqsdOMjGRXqOyYSo7Q9pSCowTnfWl+RP91KceJpyOQ5CUBkMcxzCTww9qVTpic0K1GS7Fv8U8UW+JvEzVyAQssu1vP03fRgfGoL4pSl6R1pVNzXUNS6/S2Lyf/PtX3nUUuruvP4oG2OD9URNGhk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1768486528; c=relaxed/simple; bh=TwCKSmjzMIbtWUecU+6xCW1eizZSP6AWdIVZfNzXWqc=; h=From:To:CC:Subject:Date:Message-ID:MIME-Version:Content-Type; b=AFNc8ZT15y6HzaKWwaATaSXq0poxm455iDVc5U4qu/2GqfL9elc61Rz524gQD+KXp4duB+ejql7X+G5R25bV2aq1Z+nV3Pb2T/r6Ny5PZEI0Zrc/OKE1DnYDOS6GBnp6UY/Mx2uMrHs1pbN2Y2MZ7mEXoTjNus9xe6HV7f/9V2w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amazon.com; spf=pass smtp.mailfrom=amazon.de; dkim=pass (2048-bit key) header.d=amazon.com header.i=@amazon.com header.b=AUbtnOVc; arc=none smtp.client-ip=44.245.243.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amazon.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=amazon.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=amazon.com header.i=@amazon.com header.b="AUbtnOVc" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazoncorp2; t=1768486526; x=1800022526; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=TwCKSmjzMIbtWUecU+6xCW1eizZSP6AWdIVZfNzXWqc=; b=AUbtnOVcPU8Nyi3c1Tc524NuaPl41PxgHIEp/eWrPuw63eTodoggX6cc 3hGnjsLQ2x0wbBYEWdxirHCTUcsw4tccFo9snXcgxE0zmMYmdRfBlQyDa B7z3b5MZ6I9UFa1gcgyCePW7U4yp+N2Sl6FapElvYI0vY1zyA3lL1FufA 1aDLQ+mJlsV9iEeMPTyNPgEnlZXMH5CInTLw7cX8xZ6jeXeY2dR+bk/ci j2f93aD0bDUhetnE8aqG2TcfzgelGLO1cuDz7xc2XwiMD3y0PVmyUsiU4 ra2/AwwrvCLyh+PNIFUm6SRsz1C9tp37UH9xj5/43P9lKIc5Ymx+KhGYK g==; X-CSE-ConnectionGUID: QkUcPMgXTZCMt4avVCRudw== X-CSE-MsgGUID: 0vJGAvcnSg+yGy0tQITfJQ== X-IronPort-AV: E=Sophos;i="6.21,228,1763424000"; d="scan'208";a="10459992" Received: from ip-10-5-0-115.us-west-2.compute.internal (HELO smtpout.naws.us-west-2.prod.farcaster.email.amazon.dev) ([10.5.0.115]) by internal-pdx-out-001.esa.us-west-2.outbound.mail-perimeter.amazon.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 15 Jan 2026 14:15:24 +0000 Received: from EX19MTAUWA001.ant.amazon.com [205.251.233.236:3945] by smtpin.naws.us-west-2.prod.farcaster.email.amazon.dev [10.0.37.11:2525] with esmtp (Farcaster) id e82e2b81-7724-4f9f-bc8c-7d262e21790e; Thu, 15 Jan 2026 14:15:24 +0000 (UTC) X-Farcaster-Flow-ID: e82e2b81-7724-4f9f-bc8c-7d262e21790e Received: from EX19D020UWC004.ant.amazon.com (10.13.138.149) by EX19MTAUWA001.ant.amazon.com (10.250.64.204) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA) id 15.2.2562.35; Thu, 15 Jan 2026 14:15:23 +0000 Received: from ip-10-253-83-51.amazon.com (172.19.99.218) by EX19D020UWC004.ant.amazon.com (10.13.138.149) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA) id 15.2.2562.35; Thu, 15 Jan 2026 14:15:21 +0000 From: Alexander Graf To: CC: , , , "Paolo Bonzini" , Sean Christopherson , Vitaly Kuznetsov , , , , Michael Kelley , John Starks Subject: [PATCH] kvm: hyper-v: Delay firing of expired stimers Date: Thu, 15 Jan 2026 14:15:20 +0000 Message-ID: <20260115141520.24176-1-graf@amazon.com> X-Mailer: git-send-email 2.47.1 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: EX19D039UWB003.ant.amazon.com (10.13.138.93) To EX19D020UWC004.ant.amazon.com (10.13.138.149) Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable During Windows Server 2025 hibernation, I have seen Windows' calculation of interrupt target time get skewed over the hypervisor view of the same. This can cause Windows to emit timer events in the past for events that do not fire yet according to the real time source. This then leads to interrupt storms in the guest which slow down execution to a point where watchdogs trigger. Those manifest as bugchecks 0x9f and 0xa0 during hibernation, typically in the resume path. To work around this problem, we can delay timers that get created with a target time in the past by a tiny bit (10=C2=B5s) to give the guest CPU time to process real work and make forward progress, hopefully recovering its interrupt logic in the process. While this small delay can marginally reduce accuracy of guest timers, 10=C2=B5s are within the noise of VM entry/exit overhead (~1-2 =C2=B5s) so I do not expect to see real world imp= act. To still provide some level of visibility when this happens, add a trace point that clearly shows the discrepancy between the target time and the current time. Signed-off-by: Alexander Graf --- arch/x86/kvm/hyperv.c | 22 ++++++++++++++++++---- arch/x86/kvm/trace.h | 26 ++++++++++++++++++++++++++ 2 files changed, 44 insertions(+), 4 deletions(-) diff --git a/arch/x86/kvm/hyperv.c b/arch/x86/kvm/hyperv.c index 72b19a88a776..c41061acbcbc 100644 --- a/arch/x86/kvm/hyperv.c +++ b/arch/x86/kvm/hyperv.c @@ -666,13 +666,27 @@ static int stimer_start(struct kvm_vcpu_hv_stimer *st= imer) stimer->exp_time =3D stimer->count; if (time_now >=3D stimer->count) { /* - * Expire timer according to Hypervisor Top-Level Functional - * specification v4(15.3.1): + * Hypervisor Top-Level Functional specification v4(15.3.1): * "If a one shot is enabled and the specified count is in * the past, it will expire immediately." + * + * However, there are cases during hibernation when Windows's + * interrupt count calculation can go out of sync with KVM's + * view of it, causing Windows to emit timer events in the past + * for events that do not fire yet according to the real time + * source. This then leads to interrupt storms in the guest + * which slow down execution to a point where watchdogs trigger. + * + * Instead of taking TLFS literally on what "immediately" means, + * give the guest at least 10=C2=B5s to process work. While this can + * marginally reduce accuracy of guest timers, 10=C2=B5s are within + * the noise of VM entry/exit overhead (~1-2 =C2=B5s). */ - stimer_mark_pending(stimer, false); - return 0; + trace_kvm_hv_stimer_start_expired( + hv_stimer_to_vcpu(stimer)->vcpu_id, + stimer->index, + time_now, stimer->count); + stimer->count =3D time_now + 100; } =20 trace_kvm_hv_stimer_start_one_shot(hv_stimer_to_vcpu(stimer)->vcpu_id, diff --git a/arch/x86/kvm/trace.h b/arch/x86/kvm/trace.h index 57d79fd31df0..f9e69c4d9e9b 100644 --- a/arch/x86/kvm/trace.h +++ b/arch/x86/kvm/trace.h @@ -1401,6 +1401,32 @@ TRACE_EVENT(kvm_hv_stimer_start_one_shot, __entry->count) ); =20 +/* + * Tracepoint for stimer_start(one-shot timer already expired). + */ +TRACE_EVENT(kvm_hv_stimer_start_expired, + TP_PROTO(int vcpu_id, int timer_index, u64 time_now, u64 count), + TP_ARGS(vcpu_id, timer_index, time_now, count), + + TP_STRUCT__entry( + __field(int, vcpu_id) + __field(int, timer_index) + __field(u64, time_now) + __field(u64, count) + ), + + TP_fast_assign( + __entry->vcpu_id =3D vcpu_id; + __entry->timer_index =3D timer_index; + __entry->time_now =3D time_now; + __entry->count =3D count; + ), + + TP_printk("vcpu_id %d timer %d time_now %llu count %llu (expired)", + __entry->vcpu_id, __entry->timer_index, __entry->time_now, + __entry->count) +); + /* * Tracepoint for stimer_timer_callback. */ --=20 2.47.1 Amazon Web Services Development Center Germany GmbH Tamara-Danz-Str. 13 10243 Berlin Geschaeftsfuehrung: Christof Hellmis, Andreas Stieger Eingetragen am Amtsgericht Charlottenburg unter HRB 257764 B Sitz: Berlin Ust-ID: DE 365 538 597