From nobody Fri Apr 26 04:07:40 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=quarantine dis=none) header.from=suse.com ARC-Seal: i=1; a=rsa-sha256; t=1617270863; cv=none; d=zohomail.com; s=zohoarc; b=h7JcvrpNkefVcwyMNgNoLSQwK2+qRppb1qRIyfAEyfJi4HnnhxVt/H4haP8v2yqcytDtiWlwAO8GEII++KdDkhP9oiMW1DUFixu7fj1neVblToeQum5r8dDW3+39/JvHua6TsVmHrGTbqaP1M/dpM6xiW4tPRDgnfne56uWF1Ts= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1617270863; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=SRJupnLGMfszPfCGs6C2rPqjAEwXFCBQV3qorCClVzA=; b=LUpxgadgfolQJRzJioLwx2xFi6780lETqQGptF/gj1DeKGLubifsNu2n1u98vMzGWDNePtyIdpFksRN2SqaiwibVa71FNSJb29hJ9kyNnGFKY/ZymZPu3pH89iHrLgavHZGAENyUX+1LtKE8umUJmVMrhfAWQK8LCPY2jJx22/Q= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=quarantine dis=none) header.from= Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1617270863805456.5463690763087; Thu, 1 Apr 2021 02:54:23 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.104261.199206 (Exim 4.92) (envelope-from ) id 1lRu1g-0006Om-Qz; Thu, 01 Apr 2021 09:54:08 +0000 Received: by outflank-mailman (output) from mailman id 104261.199206; Thu, 01 Apr 2021 09:54:08 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lRu1g-0006Of-Nl; Thu, 01 Apr 2021 09:54:08 +0000 Received: by outflank-mailman (input) for mailman id 104261; Thu, 01 Apr 2021 09:54:07 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lRu1f-0006OW-C9 for xen-devel@lists.xenproject.org; Thu, 01 Apr 2021 09:54:07 +0000 Received: from mx2.suse.de (unknown [195.135.220.15]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id ad03d872-721a-4051-970f-b25f37750982; Thu, 01 Apr 2021 09:54:06 +0000 (UTC) Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id A7811AEA6; Thu, 1 Apr 2021 09:54:05 +0000 (UTC) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: ad03d872-721a-4051-970f-b25f37750982 X-Virus-Scanned: by amavisd-new at test-mx.suse.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1617270845; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=SRJupnLGMfszPfCGs6C2rPqjAEwXFCBQV3qorCClVzA=; b=KCAosS6xkauPgkpAGntfN1NQ3BkhEk3qijzYbYAZnEbC7f6X0J7Ha8TA8wzqnBu3ZlsSDI XPrD3il53j+BooEb/LRJjyocRLj/IWdCkcrRRX3pxADC3B/dQYoQLega5hbh/TzrZOrJJ4 chJ5IgF8SfqgmfG34j4jFu9H7XkVVOM= Subject: [PATCH v4 1/3] x86/time: latch to-be-written TSC value early in rendezvous loop From: Jan Beulich To: "xen-devel@lists.xenproject.org" Cc: Andrew Cooper , Wei Liu , =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= References: Message-ID: <35bbad56-d0f0-a37d-674c-e635eaf9c94c@suse.com> Date: Thu, 1 Apr 2021 11:54:05 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.9.0 MIME-Version: 1.0 In-Reply-To: Content-Language: en-US Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @suse.com) Content-Type: text/plain; charset="utf-8" To reduce latency on time_calibration_tsc_rendezvous()'s last loop iteration, read the value to be written on the last iteration at the end of the loop body (i.e. in particular at the end of the second to last iteration). On my single-socket 18-core Skylake system this reduces the average loop exit time on CPU0 (from the TSC write on the last iteration to until after the main loop) from around 32k cycles to around 29k (albeit the values measured on separate runs vary quite significantly). Signed-off-by: Jan Beulich Reviewed-by: Roger Pau Monn=C3=A9 --- v4: Different approach. v3: New. --- Of course it would also be nice to avoid the pretty likely branch misprediction on the last iteration. But with the static prediction hints having been rather short-lived in the architecture, I don't see any good means to do so. --- a/xen/arch/x86/time.c +++ b/xen/arch/x86/time.c @@ -1683,7 +1683,7 @@ static void time_calibration_tsc_rendezv int i; struct calibration_rendezvous *r =3D _r; unsigned int total_cpus =3D cpumask_weight(&r->cpu_calibration_map); - uint64_t tsc =3D 0; + uint64_t tsc =3D 0, master_tsc =3D 0; =20 /* Loop to get rid of cache effects on TSC skew. */ for ( i =3D 4; i >=3D 0; i-- ) @@ -1708,7 +1708,7 @@ static void time_calibration_tsc_rendezv atomic_inc(&r->semaphore); =20 if ( i =3D=3D 0 ) - write_tsc(r->master_tsc_stamp); + write_tsc(master_tsc); =20 while ( atomic_read(&r->semaphore) !=3D (2*total_cpus - 1) ) cpu_relax(); @@ -1730,7 +1730,7 @@ static void time_calibration_tsc_rendezv } =20 if ( i =3D=3D 0 ) - write_tsc(r->master_tsc_stamp); + write_tsc(master_tsc); =20 atomic_inc(&r->semaphore); while ( atomic_read(&r->semaphore) > total_cpus ) @@ -1739,9 +1739,17 @@ static void time_calibration_tsc_rendezv =20 /* Just in case a read above ended up reading zero. */ tsc +=3D !tsc; + + /* + * To reduce latency of the TSC write on the last iteration, + * fetch the value to be written into a local variable. To avoid + * introducing yet another conditional branch (which the CPU may + * have difficulty predicting well) do this on all iterations. + */ + master_tsc =3D r->master_tsc_stamp; } =20 - time_calibration_rendezvous_tail(r, tsc, r->master_tsc_stamp); + time_calibration_rendezvous_tail(r, tsc, master_tsc); } =20 /* Ordinary rendezvous function which does not modify TSC values. */ From nobody Fri Apr 26 04:07:40 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=quarantine dis=none) header.from=suse.com ARC-Seal: i=1; a=rsa-sha256; t=1617270884; cv=none; d=zohomail.com; s=zohoarc; b=UvIJ9oAkcSnRdqMPIKoSyJWNQLZXUCyCHsVxvJH/JSALSfHtpZSdGssOJiUYgLdD2OSjzZy99sNS2jSlug7AUD80Td7pU46JUA8VQQ8m9DR7BcIh5OeTf0/vHkTTqBseQohZbFHi9Q+A9S2vrMIq7fy5MmdGckeWWiEHfWOKxs0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1617270884; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=QdS547I0MHs8GHpeGea/kzNdcUxE2McJtcRJ1hPnylM=; b=TUmtZ552LCpiluOusZdOhQ2hZ7muugztsxKVoTBd8gioHY5dbmy2RSQOgOLbci8iMwJ578cWDXiKeCYSzsXmOlC8fkUzdri3CPMiC8esMtu0P1qSlG5m9yENPzw3mZffxcETOmcwWWH8jOwP+whtgrtT1TfQe9BpwsHRGOQhYCY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=quarantine dis=none) header.from= Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1617270884760748.9043936580016; Thu, 1 Apr 2021 02:54:44 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.104263.199218 (Exim 4.92) (envelope-from ) id 1lRu23-0006Vh-3B; Thu, 01 Apr 2021 09:54:31 +0000 Received: by outflank-mailman (output) from mailman id 104263.199218; Thu, 01 Apr 2021 09:54:31 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lRu23-0006Va-05; Thu, 01 Apr 2021 09:54:31 +0000 Received: by outflank-mailman (input) for mailman id 104263; Thu, 01 Apr 2021 09:54:29 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lRu21-0006VT-IZ for xen-devel@lists.xenproject.org; Thu, 01 Apr 2021 09:54:29 +0000 Received: from mx2.suse.de (unknown [195.135.220.15]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 162e1716-1d1c-432e-86aa-435eef0c2cd3; Thu, 01 Apr 2021 09:54:28 +0000 (UTC) Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id 262E0B032; Thu, 1 Apr 2021 09:54:28 +0000 (UTC) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 162e1716-1d1c-432e-86aa-435eef0c2cd3 X-Virus-Scanned: by amavisd-new at test-mx.suse.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1617270868; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=QdS547I0MHs8GHpeGea/kzNdcUxE2McJtcRJ1hPnylM=; b=Yw2aNBfYvGgfmOIZZAydVv7NE5hING06Tbr1dHmF+WBYBdMmxtdW4AzLiCWY3VRhuyLZzc AZcIoARXZo0GVFdu7M7vh+nGvIdi9FjUUtTruyszBKAdP8hw88mvwJ6fS9zcFqMXoJgSoe Uj3Isdgek2k9ntlCRjq6CB31hRlAAnI= Subject: [PATCH v4 2/3] x86/time: yield to hyperthreads after updating TSC during rendezvous From: Jan Beulich To: "xen-devel@lists.xenproject.org" Cc: Andrew Cooper , Wei Liu , =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= References: Message-ID: <81da85eb-2e8e-9b76-2fb3-2beddc33e9af@suse.com> Date: Thu, 1 Apr 2021 11:54:27 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.9.0 MIME-Version: 1.0 In-Reply-To: Content-Language: en-US Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @suse.com) Content-Type: text/plain; charset="utf-8" Since we'd like the updates to be done as synchronously as possible, make an attempt at yielding immediately after the TSC write. Signed-off-by: Jan Beulich Reviewed-by: Roger Pau Monn=C3=A9 --- v4: New. --- a/xen/arch/x86/time.c +++ b/xen/arch/x86/time.c @@ -1708,7 +1708,14 @@ static void time_calibration_tsc_rendezv atomic_inc(&r->semaphore); =20 if ( i =3D=3D 0 ) + { write_tsc(master_tsc); + /* + * Try to give our hyperthread(s), if any, a chance to do + * the same as instantly as possible. + */ + cpu_relax(); + } =20 while ( atomic_read(&r->semaphore) !=3D (2*total_cpus - 1) ) cpu_relax(); @@ -1730,7 +1737,14 @@ static void time_calibration_tsc_rendezv } =20 if ( i =3D=3D 0 ) + { write_tsc(master_tsc); + /* + * Try to give our hyperthread(s), if any, a chance to do + * the same as instantly as possible. + */ + cpu_relax(); + } =20 atomic_inc(&r->semaphore); while ( atomic_read(&r->semaphore) > total_cpus ) From nobody Fri Apr 26 04:07:40 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=quarantine dis=none) header.from=suse.com ARC-Seal: i=1; a=rsa-sha256; t=1617270928; cv=none; d=zohomail.com; s=zohoarc; b=ilYz+6ucFxC5jB1ph/bcklx0Gv8N5ob06vzbVfziS4huhNFxtfeHyFOB8CC1dn6zo7lTkP+PGMB+MhYwaVv+cNUOReqq6+IdeWiAP7i1swGcg2miHOHreBHXetMIBNIx44YupkQfFMXJru9KqTA7wlERBNvriVElv/UY19KOyNI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1617270928; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=/4bBVDd6Hj6a8b8XlnvfpgQxmTCGoW9DQonxX1lum1U=; b=b8qlcpHvDj/r4YZSdPRN6UbKJSh1mOCkcGSrc33486GCmqoPaB9rsthRDU5U66ipD/QcMErBMcmsSU/IxItL3Aw26xQ3v9YdqFpHUs2wJwaXp6NwCJl0N1LUFrHsm/Xwtnnjq22wnMqk3vezi72Ke1Xu+5uPeh6RtTe65pAlArk= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=quarantine dis=none) header.from= Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1617270928367506.46814388792393; Thu, 1 Apr 2021 02:55:28 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.104267.199229 (Exim 4.92) (envelope-from ) id 1lRu2k-0006df-Cj; Thu, 01 Apr 2021 09:55:14 +0000 Received: by outflank-mailman (output) from mailman id 104267.199229; Thu, 01 Apr 2021 09:55:14 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lRu2k-0006dY-9h; Thu, 01 Apr 2021 09:55:14 +0000 Received: by outflank-mailman (input) for mailman id 104267; Thu, 01 Apr 2021 09:55:12 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lRu2i-0006dG-Gy for xen-devel@lists.xenproject.org; Thu, 01 Apr 2021 09:55:12 +0000 Received: from mx2.suse.de (unknown [195.135.220.15]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 4a3658ca-cfbc-451f-9886-16fc96a54119; Thu, 01 Apr 2021 09:55:11 +0000 (UTC) Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id 04D25B032; Thu, 1 Apr 2021 09:55:11 +0000 (UTC) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 4a3658ca-cfbc-451f-9886-16fc96a54119 X-Virus-Scanned: by amavisd-new at test-mx.suse.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1617270911; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/4bBVDd6Hj6a8b8XlnvfpgQxmTCGoW9DQonxX1lum1U=; b=tMyzVyeb/17TZQ3RGutErR3hQvhLIo/di2rDhns5u9ADhQOQwn82UKhNTJ7TvTs6QzDZs0 HkEPXkeflxHNeOkL6+gRVOGiOLffJAxyv6z119bgPAg8CZNZTSZi/A2Fv1UJBDTFkKdWXP 73AhBd+cg2PvrtSrjiwQC+Qb0u/wEwM= Subject: [PATCH v4 3/3] x86/time: avoid reading the platform timer in rendezvous functions From: Jan Beulich To: "xen-devel@lists.xenproject.org" Cc: Andrew Cooper , Wei Liu , =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= References: Message-ID: Date: Thu, 1 Apr 2021 11:55:10 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.9.0 MIME-Version: 1.0 In-Reply-To: Content-Language: en-US Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @suse.com) Content-Type: text/plain; charset="utf-8" Reading the platform timer isn't cheap, so we'd better avoid it when the resulting value is of no interest to anyone. The consumer of master_stime, obtained by time_calibration_{std,tsc}_rendezvous() and propagated through this_cpu(cpu_calibration), is local_time_calibration(). With CONSTANT_TSC the latter function uses an early exit path, which doesn't explicitly use the field. While this_cpu(cpu_calibration) (including the master_stime field) gets propagated to this_cpu(cpu_time).stamp on that path, both structures' fields get consumed only by the !CONSTANT_TSC logic of the function. Signed-off-by: Jan Beulich Acked-by: Roger Pau Monn=C3=A9 --- v4: New. --- I realize there's some risk associated with potential new uses of the field down the road. What would people think about compiling time.c a 2nd time into a dummy object file, with a conditional enabled to force assuming CONSTANT_TSC, and with that conditional used to suppress presence of the field as well as all audited used of it (i.e. in particular that large part of local_time_calibration())? Unexpected new users of the field would then cause build time errors. --- a/xen/arch/x86/time.c +++ b/xen/arch/x86/time.c @@ -52,6 +52,7 @@ unsigned long pit0_ticks; struct cpu_time_stamp { u64 local_tsc; s_time_t local_stime; + /* Next field unconditionally valid only when !CONSTANT_TSC. */ s_time_t master_stime; }; =20 @@ -1702,7 +1703,7 @@ static void time_calibration_tsc_rendezv * iteration. */ r->master_tsc_stamp =3D r->max_tsc_stamp; - else if ( i =3D=3D 0 ) + else if ( !boot_cpu_has(X86_FEATURE_CONSTANT_TSC) && i =3D=3D = 0 ) r->master_stime =3D read_platform_stime(NULL); =20 atomic_inc(&r->semaphore); @@ -1776,8 +1777,11 @@ static void time_calibration_std_rendezv { while ( atomic_read(&r->semaphore) !=3D (total_cpus - 1) ) cpu_relax(); - r->master_stime =3D read_platform_stime(NULL); - smp_wmb(); /* write r->master_stime /then/ signal */ + if ( !boot_cpu_has(X86_FEATURE_CONSTANT_TSC) ) + { + r->master_stime =3D read_platform_stime(NULL); + smp_wmb(); /* write r->master_stime /then/ signal */ + } atomic_inc(&r->semaphore); } else