From nobody Sun May 5 04:06:19 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=none (zoho.com: 192.237.175.120 is neither permitted nor denied by domain of lists.xenproject.org) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org ARC-Seal: i=1; a=rsa-sha256; t=1567448900; cv=none; d=zoho.com; s=zohoarc; b=QYiAOVXtyuUFb0XjNDIlBIZ7I4etHki/+0/HfeqYEErB/vqnPXPW/x4/lHhIqUiHk387FqX5er7Kh6Nuc3pQ62m/FTIsFzSAxPNKPveBdhHo0Tmg4Xq2mZiF7S0+n3/ESE36P/EQuqyAd3XKl24Ygaq0di7PfiasHxzvbBg0kE4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1567448900; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=efPO7d/u0zRpcUiVftvHbxRy2X7Bv0J5s9/ZmM5LhiQ=; b=khQZH8NEe/ldxpBhfrqM9lD9lP9SEK93d7J4PAGsPhQb91+YUPsCqyGAWBJsL9BK0+IpheDDGL9oi/I73WV38VI6Za2jeYZJZHc2baRgLzg/LnhzNnF5H3z2WbC/wG225leJmIdfc5a8rvmOTpVZAFu5HVvWa+L2mlhQ3i/kNWE= ARC-Authentication-Results: i=1; mx.zoho.com; dkim=fail; spf=none (zoho.com: 192.237.175.120 is neither permitted nor denied by domain of lists.xenproject.org) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1567448900472323.55856691170857; Mon, 2 Sep 2019 11:28:20 -0700 (PDT) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1i4r2p-000105-3F; Mon, 02 Sep 2019 18:27:15 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1i4r2n-0000zu-D2 for xen-devel@lists.xenproject.org; Mon, 02 Sep 2019 18:27:13 +0000 Received: from esa2.hc3370-68.iphmx.com (unknown [216.71.145.153]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 47d4c844-cdaf-11e9-b95f-bc764e2007e4; Mon, 02 Sep 2019 18:27:12 +0000 (UTC) X-Inumbo-ID: 47d4c844-cdaf-11e9-b95f-bc764e2007e4 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1567448833; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Iw8lpmvWW06kgHF+Bh2AWLGQjfELVrhguAYDaK3ZqrU=; b=VkI/uJhk2wAFej8RYOXBxXdQIEOFNEQIJQRvrIs8FX7Xc21W8PuIyRLj qIl+voL5P3Qiwsbyi6ADRk1qXndfedJ5tDvvETZdehS7WS708GVfrxK5W rPnQVQpwh3HUpb25GO13eW8f51YgNzHJG7s3x4Wm3sWv5NRo41ofirOL3 8=; Authentication-Results: esa2.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none; spf=None smtp.pra=edvin.torok@citrix.com; spf=Pass smtp.mailfrom=edvin.torok@citrix.com; spf=None smtp.helo=postmaster@mail.citrix.com Received-SPF: none (zoho.com: 192.237.175.120 is neither permitted nor denied by domain of lists.xenproject.org) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Received-SPF: None (esa2.hc3370-68.iphmx.com: no sender authenticity information available from domain of edvin.torok@citrix.com) identity=pra; client-ip=162.221.158.21; receiver=esa2.hc3370-68.iphmx.com; envelope-from="edvin.torok@citrix.com"; x-sender="edvin.torok@citrix.com"; x-conformance=sidf_compatible Received-SPF: Pass (esa2.hc3370-68.iphmx.com: domain of edvin.torok@citrix.com designates 162.221.158.21 as permitted sender) identity=mailfrom; client-ip=162.221.158.21; receiver=esa2.hc3370-68.iphmx.com; envelope-from="edvin.torok@citrix.com"; x-sender="edvin.torok@citrix.com"; x-conformance=sidf_compatible; x-record-type="v=spf1"; x-record-text="v=spf1 ip4:209.167.231.154 ip4:178.63.86.133 ip4:195.66.111.40/30 ip4:85.115.9.32/28 ip4:199.102.83.4 ip4:192.28.146.160 ip4:192.28.146.107 ip4:216.52.6.88 ip4:216.52.6.188 ip4:162.221.158.21 ip4:162.221.156.83 ~all" Received-SPF: None (esa2.hc3370-68.iphmx.com: no sender authenticity information available from domain of postmaster@mail.citrix.com) identity=helo; client-ip=162.221.158.21; receiver=esa2.hc3370-68.iphmx.com; envelope-from="edvin.torok@citrix.com"; x-sender="postmaster@mail.citrix.com"; x-conformance=sidf_compatible IronPort-SDR: i9aMzh2f13U2vixr+H1V7v6iLZZvEKYkSkqco4q/va9Cl8pqTmBdFAYF4ocVJNZND2ZenZdzTL fP+nR9Q5EHYReyNrd42xPRwjoVNioJw2e1jc0JWASA6PBTx39EAKKH+7H9pDWN4H6U11jB1oVZ 51UpjhIg39sF/WeGd+mlt/I/x9bitkMHMSFpAavCa8N8lyagFDYW7SyPRGtVaJpBC6+LFcXBp+ fDSWCRFPju3E8ANnBHDBxwNToQ/OvhIBX6gf7q96FL0xbF3Dx9wm3ZZzmJhkcIP1MlhsyznhBU cpw= X-SBRS: 2.7 X-MesageID: 5025228 X-Ironport-Server: esa2.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-IronPort-AV: E=Sophos;i="5.64,460,1559534400"; d="scan'208";a="5025228" From: =?UTF-8?q?Edwin=20T=C3=B6r=C3=B6k?= To: Date: Mon, 2 Sep 2019 19:27:07 +0100 Message-ID: X-Mailer: git-send-email 2.20.1 In-Reply-To: References: MIME-Version: 1.0 Subject: [Xen-devel] [PATCH 1/1] x86/arch: VM resume: avoid RDTSC emulation due to host clock drift X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Andrew Cooper , Wei Liu , =?UTF-8?q?Edwin=20T=C3=B6r=C3=B6k?= , Jan Beulich , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) On a Intel(R) Xeon(R) CPU E5-2697 v3 @ 2.60GHz the host frequency drifts: ``` (XEN) [ 6.607693] Detected 2600.004 MHz processor. (XEN) [ 2674.213081] dom1(hvm): mode=3D0,ofs=3D0xfffee6f70b7faa48,khz=3D260= 0018,inc=3D3 (XEN) [ 2674.213087] dom2(hvm): mode=3D0,ofs=3D0xfffee6fd499835c0,khz=3D260= 0018,inc=3D2 ``` The 2 domains were suspended prior to rebooting the host and applying a xen/microcode patch. After the reboot the frequency of the host was deemed = to be slightly different, and therefore switching on RDTSC emulation for the L= inux HVM guest, even though the difference was only 5 ppm. This CPU doesn't supp= ort TSC scaling. Therefore we should either measure the standard deviation of our calibration and have a range of acceptable frequencies as "same", or have a static tolerance value. The platform timer's clock frequency accuracy is: * IA-PC HPET Specification 1.0a sections 2.2 and 2.4.1: 500 ppm or better * ACPI PM timer, and PIT timer do not have defined accuracies * Intel 300 Series datasheet section 25.6: 24 MHz crystal 100 ppm or better * NTP FAQ section 3.3 Clock Quality: 11 ppm drift due to temperature * section 2.2.2 claims that PIT/ACPI PM timer share the same crystal as HPE= T and thus 500 ppm as an upper bound, "the real drift is usually smaller than 30p= pm" For simplicity and determinism opted for a static tolerance value of 100 ppm here, such that the any error would be well within the error you would get = with HPET/Linux's calibration. NTP can cope with a drift < 500 ppm. Most importantly this should stop Xen from claiming that the clock frequenc= y on the same host is different across reboots. Specifications do not currently mandate an accuracy higher than 100 ppm, therefore OSes should already be a= ble to cope with such drift on real hardware. Any improvements in accuracy from future specifications/motherboards wouldn't be applicable, because they wou= ld also come with newer CPUs that support TSC scaling. If the CPU does support TSC scaling Xen will of course still attempt to mat= ch the exact frequency value it thinks the guest had when it was suspended. See below for `if ( hvm_tsc_scaling_supported && !d->arch.vtsc )` (not visi= ble in patch context). llabs() doesn't appear to be available when building xen, hence the 2 compa= risons. After this patch when suspending a VM, and rebooting the host I get this ou= tput: ``` (XEN) [ 6.614703] Detected 2600.010 MHz processor. (XEN) [ 138.924342] TSC marked as reliable, warp =3D 0 (count=3D2) (XEN) [ 138.924346] dom1(hvm): mode=3D0,ofs=3D0xfffed01901016d18,khz=3D260= 0012,inc=3D2 ``` Signed-off-by: Edwin T=C3=B6r=C3=B6k --- xen/arch/x86/time.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/xen/arch/x86/time.c b/xen/arch/x86/time.c index 9a6ea8ffcb..a0b99f5fff 100644 --- a/xen/arch/x86/time.c +++ b/xen/arch/x86/time.c @@ -2171,6 +2171,12 @@ void tsc_get_info(struct domain *d, uint32_t *tsc_mo= de, *elapsed_nsec =3D 0; } =20 +static inline int frequency_same_with_tolerance(int64_t khz1, int64_t khz2) +{ + int64_t ppm =3D (khz2 - khz1) * 1000000 / khz2; + return -100 < ppm && ppm < 100; +} + /* * This may be called as many as three times for a domain, once when the * hypervisor creates the domain, once when the toolstack creates the @@ -2207,7 +2213,7 @@ int tsc_set_info(struct domain *d, * d->arch.tsc_khz =3D=3D cpu_khz. Thus no need to check incarnati= on. */ if ( tsc_mode =3D=3D TSC_MODE_DEFAULT && host_tsc_is_safe() && - (d->arch.tsc_khz =3D=3D cpu_khz || + (frequency_same_with_tolerance(d->arch.tsc_khz, cpu_khz) || (is_hvm_domain(d) && hvm_get_tsc_scaling_ratio(d->arch.tsc_khz))) ) { --=20 2.20.1 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel