From nobody Thu Nov 5 07:29:37 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=xenproject.org ARC-Seal: i=1; a=rsa-sha256; t=1793366691; cv=none; d=zohomail.com; s=zohoarc; b=FM5p90JvQfOahe0iWtosayP9bCqMCA+cYjSFVHQ0Cvexsmf65AuEYt5Ij7y0noeqEMUrtkkxA9aHpYdtuKK9wU1P0RbpVS4B0J26YGhmBv7T1TTGZmSkcvaKuaUJnEUu5IrFqnOMmnq1QwTmH5gwQHE7wH0ASs+uMxzEEFGYvv4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1793366691; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=2Lf9FJlpDBMjDqifcNltXmBL+3wrGoy3egUW4LZ+zNE=; b=L4/ra6zflUMHCtBR6PUZdQ3MoXxVzT6sTh6pGi/skbuSC9cY2B0vKBNmS0T1WU1+AhtQeOkelg9x4Pgw1kE5r8JNQmLOCEiejNqcJsxwQItdIgSUulMTX1imnDMGM0nhFWhX1eeDsW2sdx0TmwLGG4FvtZt9DmnsC6o3gJRnSwY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1793366691692661.0245008458833; Fri, 30 Oct 2026 06:24:51 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1465658.1676128 (Exim 4.92) (envelope-from ) id 1xMmb1-0007kH-GY; Fri, 30 Oct 2026 13:24:39 +0000 Received: by outflank-mailman (output) from mailman id 1465658.1676128; Fri, 30 Oct 2026 13:24:39 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xMmb1-0007kA-Ct; Fri, 30 Oct 2026 13:24:39 +0000 Received: by outflank-mailman (input) for mailman id 1465658; Fri, 30 Oct 2026 13:24:38 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xMmb0-0007jf-20 for xen-devel@lists.xenproject.org; Fri, 30 Oct 2026 13:24:38 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.98.2) (envelope-from ) id 1xMmaz-00000004Gqa-0JTr; Fri, 30 Oct 2026 13:24:36 +0000 Received: from 224.pool85-54-217.dynamic.orange.es ([85.54.217.224] helo=localhost) by xenbits.xenproject.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.98.2) (envelope-from ) id 1xMmay-00000007k31-1wAl; Fri, 30 Oct 2026 13:24:36 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xenproject.org; s=20200302mail; h=Content-Transfer-Encoding:Content-Type: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From; bh=2Lf9FJlpDBMjDqifcNltXmBL+3wrGoy3egUW4LZ+zNE=; b=ncp6VXGx5uOWwe8lKCUBf7dmO2 C5AD8oDaWf79fY6ffM8p98TZ0wOAFiIfdB/e9oDHTRqT6P694TEFRhOsqfbQqkFJ1Sk+ZRyxtSfZG zqmCv2JPNOaQyOfQ6Dfe6gEnxpjJ5liegmn4++YUASXLdegH8NOrcwZBA9dGqLJoBZos=; From: Roger Pau Monne To: xen-devel@lists.xenproject.org Cc: Roger Pau Monne , Jan Beulich , Andrew Cooper , Teddy Astie Subject: [PATCH 1/2] x86/cpu: keep cached TSC_AUX state coherent with hardware Date: Fri, 30 Oct 2026 14:24:18 +0100 Message-ID: <20261030132419.84264-2-roger@xenproject.org> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20261030132419.84264-1-roger@xenproject.org> References: <20261030132419.84264-1-roger@xenproject.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @xenproject.org) X-ZM-MESSAGEID: 1793366694321154100 On CPU bringup don't assume that TSC_AUX is 0, write 0 to it to ensure the per-CPU cached state matches the hardware. Also propagate the cached value on resume from suspend, in case the register value has been reset during suspension. Fixes: cc0e45db2779 ("x86/pv: Avoid leaking other guests' MSR_TSC_AUX value= s into PV context") Signed-off-by: Roger Pau Monn=C3=A9 Reviewed-by: Teddy Astie --- xen/arch/x86/acpi/power.c | 3 +++ xen/arch/x86/cpu/common.c | 5 +++++ 2 files changed, 8 insertions(+) diff --git a/xen/arch/x86/acpi/power.c b/xen/arch/x86/acpi/power.c index 3452650a61d5..464846e73727 100644 --- a/xen/arch/x86/acpi/power.c +++ b/xen/arch/x86/acpi/power.c @@ -279,6 +279,9 @@ static int enter_state(u32 state) ci->last_spec_ctrl =3D default_xen_spec_ctrl; } =20 + if ( cpu_has_msr_tsc_aux ) + wrmsrns(MSR_TSC_AUX, this_cpu(tsc_aux)); + update_mcu_opt_ctrl(); update_pb_opt_ctrl(); =20 diff --git a/xen/arch/x86/cpu/common.c b/xen/arch/x86/cpu/common.c index 43785d19cc73..5beae0e87f49 100644 --- a/xen/arch/x86/cpu/common.c +++ b/xen/arch/x86/cpu/common.c @@ -926,6 +926,11 @@ void cpu_init(void) =20 /* Enable NMIs. Our loader (e.g. Tboot) may have left them disabled. */ enable_nmis(); + + if (cpu_has_msr_tsc_aux) { + wrmsrns(MSR_TSC_AUX, 0); + this_cpu(tsc_aux) =3D 0; + } } =20 void cpu_uninit(unsigned int cpu) --=20 2.55.0 From nobody Thu Nov 5 07:29:37 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=xenproject.org ARC-Seal: i=1; a=rsa-sha256; t=1793366698; cv=none; d=zohomail.com; s=zohoarc; b=MduxXgOofBUA4pCfH8sLm0sr06qBJgpEK3/uYaeZNVtvqinGg8TXw+utOszvAXpHcvjE3YKtkSTF3hsZKsYCFy2Ogs2xG9cUKSyxnRsqT2Pgrq6xJxfkI06Oa1NtKtvB9trFVKyf7PYpyEVvteTtokYC1G+kfekYwU/DdQUtCIU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1793366698; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=fv865zl5JzNvZdc+vUmKul0mo11ea8d5CxR8YPF0vJw=; b=FZL2wGyvYxViTRot4fxojgwDN761kWoyGZqtHdz7EpA4vg9EN0BZLogym+zJ87Jg5ZqMhFYUINHfXUSyrE9Q4iocyRromnGHSTy6iLe+CDtuy42lLwwXgTJ0BgI1l4jzUOn4K6RP3joceFYcriC03SKyvq8Pu9AglpjQ8RKAcPg= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 179336669881458.78385125775969; Fri, 30 Oct 2026 06:24:58 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1465659.1676134 (Exim 4.92) (envelope-from ) id 1xMmb1-0007mj-OS; Fri, 30 Oct 2026 13:24:39 +0000 Received: by outflank-mailman (output) from mailman id 1465659.1676134; Fri, 30 Oct 2026 13:24:39 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xMmb1-0007mX-JJ; Fri, 30 Oct 2026 13:24:39 +0000 Received: by outflank-mailman (input) for mailman id 1465659; Fri, 30 Oct 2026 13:24:39 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1xMmb1-0007jy-42 for xen-devel@lists.xenproject.org; Fri, 30 Oct 2026 13:24:39 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.98.2) (envelope-from ) id 1xMmb1-00000004Gqi-0249; Fri, 30 Oct 2026 13:24:38 +0000 Received: from 224.pool85-54-217.dynamic.orange.es ([85.54.217.224] helo=localhost) by xenbits.xenproject.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.98.2) (envelope-from ) id 1xMmb0-00000007k4Z-27Yf; Fri, 30 Oct 2026 13:24:38 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xenproject.org; s=20200302mail; h=Content-Transfer-Encoding:Content-Type: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From; bh=fv865zl5JzNvZdc+vUmKul0mo11ea8d5CxR8YPF0vJw=; b=kKciUV1IgKBlLSTfpR19gwsPcB X428zkaf64a7toiZDT+UuSp6x0jaDrEkLmn0oO103AbDc++owXVp16pjF/xJKnUM0HO3wzuVLvHfl L2zypsFEsXy7IlJrlc0+I5vZw3zG8m9G/WSAHW6FPHKakE443Up+geD0mTerMNHZwjQs=; From: Roger Pau Monne To: xen-devel@lists.xenproject.org Cc: Roger Pau Monne , Jan Beulich , Andrew Cooper , Teddy Astie Subject: [PATCH 2/2] x86/hvm: sync APs TSC with the BSP at creation only Date: Fri, 30 Oct 2026 14:24:19 +0100 Message-ID: <20261030132419.84264-3-roger@xenproject.org> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20261030132419.84264-1-roger@xenproject.org> References: <20261030132419.84264-1-roger@xenproject.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @xenproject.org) X-ZM-MESSAGEID: 1793366700611158500 The current logic syncs the APs' TSC with the BSP both at reset and when setting the vCPU state. This doesn't match hardware behavior, where APs are only in sync at platform start. Move the TSC sync so it's done in hvm_vcpu_initialise(), that's called a single time at vCPU creation. This matches the hardware behavior. INIT-SIPI and VCPUOP_initialise no longer re-sync a vCPU's TSC to vCPU 0's. On migration the TSC offset for offline vCPUs is not transferred (hvm_save_cpu_ctxt() skips offline vCPUs). To avoid an unexpected skew synchronize offline vCPUs with the first online vCPU, taking into account the TSC_ADJUST value. The clearing of TSC_ADJUST is also made consistent with hardware behavior: clear it on S3 resume, but not across an INIT-SIPI or VCPUOP_initialise. Additionally, on S3 resume start all vCPUs from a shared TSC reading, so they are in sync. Fixes: c31f8336e64e ("[HVM] Initialise AP TSC to same as BP TSC.") Fixes: 6e0336351ed9 ("x86: Implement TSC adjust feature for HVM guest") Signed-off-by: Roger Pau Monn=C3=A9 --- xen/arch/x86/hvm/domain.c | 5 ----- xen/arch/x86/hvm/hvm.c | 19 +++++++++++-------- xen/arch/x86/hvm/save.c | 34 ++++++++++++++++++++++++++++++++-- xen/arch/x86/time.c | 5 ++--- 4 files changed, 45 insertions(+), 18 deletions(-) diff --git a/xen/arch/x86/hvm/domain.c b/xen/arch/x86/hvm/domain.c index 978d8b222256..71f49ec7ce9f 100644 --- a/xen/arch/x86/hvm/domain.c +++ b/xen/arch/x86/hvm/domain.c @@ -312,11 +312,6 @@ int arch_set_info_hvm_guest(struct vcpu *v, const stru= ct vcpu_hvm_context *ctx) hvm_set_segment_register(v, x86_seg_es, &es); hvm_set_segment_register(v, x86_seg_tss, &tr); =20 - /* Sync AP's TSC with BSP's. */ - v->arch.hvm.cache_tsc_offset =3D - d->vcpu[0]->arch.hvm.cache_tsc_offset; - hvm_set_tsc_offset(v, v->arch.hvm.cache_tsc_offset); - paging_update_paging_modes(v); =20 hvmemul_cancel(v); diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c index ef9c7cd05777..81e90cc8fc8e 100644 --- a/xen/arch/x86/hvm/hvm.c +++ b/xen/arch/x86/hvm/hvm.c @@ -1667,6 +1667,12 @@ int hvm_vcpu_initialise(struct vcpu *v) /* Init guest TSC to start from zero. */ hvm_set_guest_tsc(v, 0); } + else + { + /* Start in step with vCPU 0, as all CPUs do at power-on. */ + v->arch.hvm.cache_tsc_offset =3D d->vcpu[0]->arch.hvm.cache_tsc_of= fset; + hvm_set_tsc_offset(v, v->arch.hvm.cache_tsc_offset); + } =20 return 0; =20 @@ -4021,13 +4027,6 @@ void hvm_vcpu_reset_state(struct vcpu *v, uint16_t c= s, uint16_t ip) hvm_set_segment_register(v, x86_seg_gdt, ®); hvm_set_segment_register(v, x86_seg_idt, ®); =20 - /* Sync AP's TSC with BSP's. */ - v->arch.hvm.cache_tsc_offset =3D - v->domain->vcpu[0]->arch.hvm.cache_tsc_offset; - hvm_set_tsc_offset(v, v->arch.hvm.cache_tsc_offset); - - v->arch.hvm.msr_tsc_adjust =3D 0; - paging_update_paging_modes(v); =20 v->arch.flags |=3D TF_kernel_mode; @@ -4079,9 +4078,13 @@ static void hvm_s3_resume(struct domain *d) if ( test_and_clear_bool(d->arch.hvm.is_s3_suspended) ) { struct vcpu *v; + uint64_t at_tsc =3D rdtsc(); =20 for_each_vcpu( d, v ) - hvm_set_guest_tsc(v, 0); + { + v->arch.hvm.msr_tsc_adjust =3D 0; + hvm_set_guest_tsc_fixed(v, 0, at_tsc); + } domain_unpause(d); } } diff --git a/xen/arch/x86/hvm/save.c b/xen/arch/x86/hvm/save.c index 8ab64057064e..65ebfd384d69 100644 --- a/xen/arch/x86/hvm/save.c +++ b/xen/arch/x86/hvm/save.c @@ -289,6 +289,7 @@ int hvm_load(struct domain *d, bool real, hvm_domain_co= ntext_t *h) { const struct hvm_save_header *hdr; struct hvm_save_descriptor *desc; + struct vcpu *v; int rc; =20 if ( d->is_dying ) @@ -302,8 +303,6 @@ int hvm_load(struct domain *d, bool real, hvm_domain_co= ntext_t *h) rc =3D arch_hvm_check(d, hdr); if ( real ) { - struct vcpu *v; - ASSERT(!rc); =20 /* @@ -341,6 +340,37 @@ int hvm_load(struct domain *d, bool real, hvm_domain_c= ontext_t *h) /* Reset cursor for hvm_load(, true, ). */ if ( !real ) h->cur =3D 0; + else + { + const struct vcpu *base =3D NULL; + + /* + * Offline vCPUs have no CPU record. Adjust the TSC offset + * based on the first online vCPU, taking into account + * TSC_ADJUST which is saved regardless of whether the vCP= U is + * offline. + */ + for_each_vcpu(d, v) + if ( !test_bit(_VPF_down, &v->pause_flags) ) + { + base =3D v; + break; + } + + if ( !base ) + return 0; + + for_each_vcpu(d, v) + if ( test_bit(_VPF_down, &v->pause_flags) ) + { + v->arch.hvm.cache_tsc_offset =3D + base->arch.hvm.cache_tsc_offset - + base->arch.hvm.msr_tsc_adjust + + v->arch.hvm.msr_tsc_adjust; + hvm_set_tsc_offset(v, v->arch.hvm.cache_tsc_offset= ); + } + } + return 0; } =20 diff --git a/xen/arch/x86/time.c b/xen/arch/x86/time.c index f37fddc61e47..79c7123c0a5b 100644 --- a/xen/arch/x86/time.c +++ b/xen/arch/x86/time.c @@ -3087,9 +3087,8 @@ int tsc_set_info(struct domain *d, * set_tsc_offset() is called from hvm_vcpu_initialise() before * tsc_set_info(). New vtsc mode may require recomputing TSC * offset. - * We only need to do this for BSP during initial boot. APs wi= ll - * call set_tsc_offset() later from hvm_vcpu_reset_state() and= they - * will sync their TSC to BSP's sync_tsc. + * We only need to do this for BSP during initial boot. APs c= opy + * vCPU 0's offset when created in hvm_vcpu_initialise(). */ d->arch.hvm.sync_tsc =3D rdtsc(); hvm_set_tsc_offset(d->vcpu[0], --=20 2.55.0