From nobody Sun Apr 28 20:40:05 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=none (zohomail.com: 192.237.175.120 is neither permitted nor denied by domain of lists.xenproject.org) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail(p=none dis=none) header.from=citrix.com ARC-Seal: i=1; a=rsa-sha256; t=1576277338; cv=none; d=zohomail.com; s=zohoarc; b=nFGR8pKY3e1Ss85vrgDQ2xi0G1v2fEBtstF47bSpHyH6eRTNExUwvF93N7RTuZHkLIwHLI224sfea0iBsFDRngYByIIEtNHNd3W1hCys5YJQxmLNPOr/Zlw/4ZjbVVc6mXnnRb0LU8w3z6Fr0sUnq8y9nv97qUKD+xbnhzZZS5g= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1576277338; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=2nQexbGmq+EurkooDt/8Jvp8S/HcQ59i/GDIfdeBSz8=; b=c8SKGVhQ0zXMswh/M4kAYoFXDlSnTJuERsU2brXZ9CeI1/CsqYOUMzH9yrtSjlhZFNuGXMy+ud6smLLr9qeq7KcBFh9Ag63eRmLsMCcKOJRSJNLTNYPqfhz/zPzlx+pb+zASklPd+xsfdw+0OwByygaRh9OiHcMBcLfu3p3+I38= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=none (zohomail.com: 192.237.175.120 is neither permitted nor denied by domain of lists.xenproject.org) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 15762773385621013.8357696665418; Fri, 13 Dec 2019 14:48:58 -0800 (PST) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1iftjX-0003Ao-8b; Fri, 13 Dec 2019 22:48:27 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1iftjV-0003AP-6H for xen-devel@lists.xenproject.org; Fri, 13 Dec 2019 22:48:25 +0000 Received: from esa4.hc3370-68.iphmx.com (unknown [216.71.155.144]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id a6a9a308-1dfa-11ea-88e7-bc764e2007e4; Fri, 13 Dec 2019 22:48:16 +0000 (UTC) X-Inumbo-ID: a6a9a308-1dfa-11ea-88e7-bc764e2007e4 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1576277296; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version; bh=COc48C8hnO3YxD5NXPqosfwfgM1CqBkFABkSOGZli4I=; b=Fd3GB1+ww40msNx2Yjju11m4GoFaImlCnH+q4O7oomZ5WRPwYn0hHpOh rL5S09BpFQrzg2PXmH/7WsSdzjHDTxgodtIj/Nmm0GMEPATAGqpf48T+2 WneCrfbtnUbQgJJoR0ipglzkt7JTGDQ6UyuewfsB7s2TfEskJ9kJs2P9S 8=; Authentication-Results: esa4.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none; spf=None smtp.pra=igor.druzhinin@citrix.com; spf=Pass smtp.mailfrom=igor.druzhinin@citrix.com; spf=None smtp.helo=postmaster@mail.citrix.com Received-SPF: none (zohomail.com: 192.237.175.120 is neither permitted nor denied by domain of lists.xenproject.org) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Received-SPF: None (esa4.hc3370-68.iphmx.com: no sender authenticity information available from domain of igor.druzhinin@citrix.com) identity=pra; client-ip=162.221.158.21; receiver=esa4.hc3370-68.iphmx.com; envelope-from="igor.druzhinin@citrix.com"; x-sender="igor.druzhinin@citrix.com"; x-conformance=sidf_compatible Received-SPF: Pass (esa4.hc3370-68.iphmx.com: domain of igor.druzhinin@citrix.com designates 162.221.158.21 as permitted sender) identity=mailfrom; client-ip=162.221.158.21; receiver=esa4.hc3370-68.iphmx.com; envelope-from="igor.druzhinin@citrix.com"; x-sender="igor.druzhinin@citrix.com"; x-conformance=sidf_compatible; x-record-type="v=spf1"; x-record-text="v=spf1 ip4:209.167.231.154 ip4:178.63.86.133 ip4:195.66.111.40/30 ip4:85.115.9.32/28 ip4:199.102.83.4 ip4:192.28.146.160 ip4:192.28.146.107 ip4:216.52.6.88 ip4:216.52.6.188 ip4:162.221.158.21 ip4:162.221.156.83 ip4:168.245.78.127 ~all" Received-SPF: None (esa4.hc3370-68.iphmx.com: no sender authenticity information available from domain of postmaster@mail.citrix.com) identity=helo; client-ip=162.221.158.21; receiver=esa4.hc3370-68.iphmx.com; envelope-from="igor.druzhinin@citrix.com"; x-sender="postmaster@mail.citrix.com"; x-conformance=sidf_compatible IronPort-SDR: XRj+dJjBurxpzvqeSaxi80P7h0YwzABaAWh6qJ/gdm1SWVzw80isNX8p25AOchabm/O9rAJ275 5ccQUBqazR0RFl9RNc8DA39PBmYdiGImp0H91MYAE9E1ceaJ1vOSssn9UwrlGcnr5y1QETDBI6 +d15BABrEoZ8+IRbPHYg14ww3E/vtbrU2WufyLbI05EmZ5WRHqRgO6tZGvZT+0VTKko8vN9XGP eTm071Am0VkR2PwRciLQVa3DnsolIiKvBJeqz2g+IjBBN2tfRehyBfeFktymakrmrezqMBQtin nn0= X-SBRS: 2.7 X-MesageID: 10241745 X-Ironport-Server: esa4.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-IronPort-AV: E=Sophos;i="5.69,311,1571716800"; d="scan'208";a="10241745" From: Igor Druzhinin To: Date: Fri, 13 Dec 2019 22:48:01 +0000 Message-ID: <1576277282-6590-2-git-send-email-igor.druzhinin@citrix.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1576277282-6590-1-git-send-email-igor.druzhinin@citrix.com> References: <1576277282-6590-1-git-send-email-igor.druzhinin@citrix.com> MIME-Version: 1.0 Subject: [Xen-devel] [PATCH 1/2] x86/time: drop vtsc_{kern, user}count debug counters X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: andrew.cooper3@citrix.com, Igor Druzhinin , wl@xen.org, jbeulich@suse.com, roger.pau@citrix.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) They either need to be transformed to atomics to work correctly (currently they left unprotected for HVM domains) or dropped entirely as taking a per-domain spinlock is too expensive for high-vCPU count domains even for debug build given this lock is taken too often. Choose the latter as they are not extremely important anyway. Signed-off-by: Igor Druzhinin Acked-by: Andrew Cooper Reviewed-by: Roger Pau Monn=C3=A9 --- xen/arch/x86/hvm/hvm.c | 32 ++------------------------------ xen/arch/x86/time.c | 12 ------------ xen/include/asm-x86/domain.h | 4 ---- 3 files changed, 2 insertions(+), 46 deletions(-) diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c index 47573f7..614ed60 100644 --- a/xen/arch/x86/hvm/hvm.c +++ b/xen/arch/x86/hvm/hvm.c @@ -3405,37 +3405,9 @@ int hvm_vmexit_cpuid(struct cpu_user_regs *regs, uns= igned int inst_len) return hvm_monitor_cpuid(inst_len, leaf, subleaf); } =20 -static uint64_t _hvm_rdtsc_intercept(void) -{ - struct vcpu *curr =3D current; -#if !defined(NDEBUG) || defined(CONFIG_PERF_COUNTERS) - struct domain *currd =3D curr->domain; - - if ( currd->arch.vtsc ) - switch ( hvm_guest_x86_mode(curr) ) - { - case 8: - case 4: - case 2: - if ( unlikely(hvm_get_cpl(curr)) ) - { - case 1: - currd->arch.vtsc_usercount++; - break; - } - /* fall through */ - case 0: - currd->arch.vtsc_kerncount++; - break; - } -#endif - - return hvm_get_guest_tsc(curr); -} - void hvm_rdtsc_intercept(struct cpu_user_regs *regs) { - msr_split(regs, _hvm_rdtsc_intercept()); + msr_split(regs, hvm_get_guest_tsc(current)); =20 HVMTRACE_2D(RDTSC, regs->eax, regs->edx); } @@ -3464,7 +3436,7 @@ int hvm_msr_read_intercept(unsigned int msr, uint64_t= *msr_content) break; =20 case MSR_IA32_TSC: - *msr_content =3D _hvm_rdtsc_intercept(); + *msr_content =3D hvm_get_guest_tsc(v); break; =20 case MSR_IA32_TSC_ADJUST: diff --git a/xen/arch/x86/time.c b/xen/arch/x86/time.c index 27a3a10..216169a 100644 --- a/xen/arch/x86/time.c +++ b/xen/arch/x86/time.c @@ -2135,13 +2135,6 @@ uint64_t pv_soft_rdtsc(const struct vcpu *v, const s= truct cpu_user_regs *regs) =20 spin_lock(&d->arch.vtsc_lock); =20 -#if !defined(NDEBUG) || defined(CONFIG_PERF_COUNTERS) - if ( guest_kernel_mode(v, regs) ) - d->arch.vtsc_kerncount++; - else - d->arch.vtsc_usercount++; -#endif - if ( (int64_t)(now - d->arch.vtsc_last) > 0 ) d->arch.vtsc_last =3D now; else @@ -2318,11 +2311,6 @@ static void dump_softtsc(unsigned char key) printk(",khz=3D%"PRIu32, d->arch.tsc_khz); if ( d->arch.incarnation ) printk(",inc=3D%"PRIu32, d->arch.incarnation); -#if !defined(NDEBUG) || defined(CONFIG_PERF_COUNTERS) - if ( d->arch.vtsc_kerncount | d->arch.vtsc_usercount ) - printk(",vtsc count: %"PRIu64" kernel,%"PRIu64" user", - d->arch.vtsc_kerncount, d->arch.vtsc_usercount); -#endif printk("\n"); domcnt++; } diff --git a/xen/include/asm-x86/domain.h b/xen/include/asm-x86/domain.h index 212303f..3780287 100644 --- a/xen/include/asm-x86/domain.h +++ b/xen/include/asm-x86/domain.h @@ -374,10 +374,6 @@ struct arch_domain hardware TSC scaling cases */ uint32_t incarnation; /* incremented every restore or live migrate (possibly other cases in the future */ -#if !defined(NDEBUG) || defined(CONFIG_PERF_COUNTERS) - uint64_t vtsc_kerncount; - uint64_t vtsc_usercount; -#endif =20 /* Pseudophysical e820 map (XENMEM_memory_map). */ spinlock_t e820_lock; --=20 2.7.4 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel From nobody Sun Apr 28 20:40:05 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=none (zohomail.com: 192.237.175.120 is neither permitted nor denied by domain of lists.xenproject.org) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail(p=none dis=none) header.from=citrix.com ARC-Seal: i=1; a=rsa-sha256; t=1576277336; cv=none; d=zohomail.com; s=zohoarc; b=SbTcWGOdKfXN2vyhDdaE1unoz7VfUnZM2xaQqOJeZVBL/ij2Ly2Ag6LM5VXbMKxW47e5hT/NaESqTWphzmLknUKMtKRWMHuvatLnifzSqB+YloQgNE6Y0u7C7bYYb/He3WknGksO4GFee3WIAj8dr7mLK37nCO2xiOAzOzf9oGI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1576277336; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=pUkSFhAGNRoSD+q12fzEHLrGjCY3e8OfF0U2yA/C328=; b=VLkd53VGd1JONo9gUqeyxN+fE5FXqWVgfeyI39+aLa6LSL9hy+a2Xg4cbxHqGi5NgDLBpaqqxPQvdaxIMnesq7B4V97jly/F/mt94wfa8kiqdWATSRsPOFveRoafQgnBeCe4xCselt27Bbr/2hrwWRb+8apYnZDOw6FGn1l6WvY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=none (zohomail.com: 192.237.175.120 is neither permitted nor denied by domain of lists.xenproject.org) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1576277336887112.91441629374117; Fri, 13 Dec 2019 14:48:56 -0800 (PST) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1iftjO-00039P-Vm; Fri, 13 Dec 2019 22:48:18 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1iftjN-00039G-Qn for xen-devel@lists.xenproject.org; Fri, 13 Dec 2019 22:48:17 +0000 Received: from esa4.hc3370-68.iphmx.com (unknown [216.71.155.144]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id a6b375c2-1dfa-11ea-9024-12813bfff9fa; Fri, 13 Dec 2019 22:48:16 +0000 (UTC) X-Inumbo-ID: a6b375c2-1dfa-11ea-9024-12813bfff9fa DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1576277296; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version; bh=GL+f2nCbKc2Xgo48QJHzrfoquRwqoJDWkZDJ+YnYajk=; b=LrXUQW1WB5tvYki2eyJ/3GYxYZGjz1m4jiDOopf+0iicZnNhR/cyrVue nqLKIJo1i0dUkBx3UiIpW5Np+XOVdZRJDMda07Pm0zf0cKSg0OZDSRi3A fnBwA5g/U9N2FNfzJrBt6Fsy/ChkMMdimoL5O8c9zm18+NvTAGzfs7WoM g=; Authentication-Results: esa4.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none; spf=None smtp.pra=igor.druzhinin@citrix.com; spf=Pass smtp.mailfrom=igor.druzhinin@citrix.com; spf=None smtp.helo=postmaster@mail.citrix.com Received-SPF: none (zohomail.com: 192.237.175.120 is neither permitted nor denied by domain of lists.xenproject.org) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Received-SPF: None (esa4.hc3370-68.iphmx.com: no sender authenticity information available from domain of igor.druzhinin@citrix.com) identity=pra; client-ip=162.221.158.21; receiver=esa4.hc3370-68.iphmx.com; envelope-from="igor.druzhinin@citrix.com"; x-sender="igor.druzhinin@citrix.com"; x-conformance=sidf_compatible Received-SPF: Pass (esa4.hc3370-68.iphmx.com: domain of igor.druzhinin@citrix.com designates 162.221.158.21 as permitted sender) identity=mailfrom; client-ip=162.221.158.21; receiver=esa4.hc3370-68.iphmx.com; envelope-from="igor.druzhinin@citrix.com"; x-sender="igor.druzhinin@citrix.com"; x-conformance=sidf_compatible; x-record-type="v=spf1"; x-record-text="v=spf1 ip4:209.167.231.154 ip4:178.63.86.133 ip4:195.66.111.40/30 ip4:85.115.9.32/28 ip4:199.102.83.4 ip4:192.28.146.160 ip4:192.28.146.107 ip4:216.52.6.88 ip4:216.52.6.188 ip4:162.221.158.21 ip4:162.221.156.83 ip4:168.245.78.127 ~all" Received-SPF: None (esa4.hc3370-68.iphmx.com: no sender authenticity information available from domain of postmaster@mail.citrix.com) identity=helo; client-ip=162.221.158.21; receiver=esa4.hc3370-68.iphmx.com; envelope-from="igor.druzhinin@citrix.com"; x-sender="postmaster@mail.citrix.com"; x-conformance=sidf_compatible IronPort-SDR: QjRDI5Jd0cVmArNUc9d43OJIrdgbTE0PVdXl4UWfG/YveUAH8lFGuDwXfF2oYkEHA8iKQj/eFb 1R3TyBzqKpZfAp3JiA8GDOIrrV73wbiM2/KmqIvP0+JZBwpG19dLhBsbMDzMocFwQAe83lUbQt RQzL1+a4SHecs6vxQaI4jyr8VI7KDSrcQrmjLtR1uOkGlDHU8jQC2mhFHfY553hcq+tvcQkFOQ eKfo3yi6CS3Yx3R14BHQp7Ad9yB7ot8H+hyv9PBFlAechTE0AAcaGbz18cn2kk9DRQUT0enGWa qB4= X-SBRS: 2.7 X-MesageID: 10241746 X-Ironport-Server: esa4.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-IronPort-AV: E=Sophos;i="5.69,311,1571716800"; d="scan'208";a="10241746" From: Igor Druzhinin To: Date: Fri, 13 Dec 2019 22:48:02 +0000 Message-ID: <1576277282-6590-3-git-send-email-igor.druzhinin@citrix.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1576277282-6590-1-git-send-email-igor.druzhinin@citrix.com> References: <1576277282-6590-1-git-send-email-igor.druzhinin@citrix.com> MIME-Version: 1.0 Subject: [Xen-devel] [PATCH 2/2] x86/time: update vtsc_last with cmpxchg and drop vtsc_lock X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: andrew.cooper3@citrix.com, Igor Druzhinin , wl@xen.org, jbeulich@suse.com, roger.pau@citrix.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Now that vtsc_last is the only entity protected by vtsc_lock we can simply update it using a single atomic operation and drop the spinlock entirely. This is extremely important for the case of running nested (e.g. shim instance with lots of vCPUs assigned) since if preemption happens somewhere inside the critical section that would immediately mean that other vCPU stop progressing (and probably being preempted as well) waiting for the spinlock to be freed. This fixes constant shim guest boot lockups with ~32 vCPUs if there is vCPU overcommit present (which increases the likelihood of preemption). Signed-off-by: Igor Druzhinin --- xen/arch/x86/domain.c | 1 - xen/arch/x86/time.c | 16 ++++++---------- xen/include/asm-x86/domain.h | 1 - 3 files changed, 6 insertions(+), 12 deletions(-) diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c index bed19fc..94531be 100644 --- a/xen/arch/x86/domain.c +++ b/xen/arch/x86/domain.c @@ -539,7 +539,6 @@ int arch_domain_create(struct domain *d, INIT_PAGE_LIST_HEAD(&d->arch.relmem_list); =20 spin_lock_init(&d->arch.e820_lock); - spin_lock_init(&d->arch.vtsc_lock); =20 /* Minimal initialisation for the idle domain. */ if ( unlikely(is_idle_domain(d)) ) diff --git a/xen/arch/x86/time.c b/xen/arch/x86/time.c index 216169a..202446f 100644 --- a/xen/arch/x86/time.c +++ b/xen/arch/x86/time.c @@ -2130,19 +2130,15 @@ u64 gtsc_to_gtime(struct domain *d, u64 tsc) =20 uint64_t pv_soft_rdtsc(const struct vcpu *v, const struct cpu_user_regs *r= egs) { - s_time_t now =3D get_s_time(); + s_time_t old, new, now =3D get_s_time(); struct domain *d =3D v->domain; =20 - spin_lock(&d->arch.vtsc_lock); - - if ( (int64_t)(now - d->arch.vtsc_last) > 0 ) - d->arch.vtsc_last =3D now; - else - now =3D ++d->arch.vtsc_last; - - spin_unlock(&d->arch.vtsc_lock); + do { + old =3D d->arch.vtsc_last; + new =3D (int64_t)(now - d->arch.vtsc_last) > 0 ? now : old + 1; + } while ( cmpxchg(&d->arch.vtsc_last, old, new) !=3D old ); =20 - return gtime_to_gtsc(d, now); + return gtime_to_gtsc(d, new); } =20 bool clocksource_is_tsc(void) diff --git a/xen/include/asm-x86/domain.h b/xen/include/asm-x86/domain.h index 3780287..e4da373 100644 --- a/xen/include/asm-x86/domain.h +++ b/xen/include/asm-x86/domain.h @@ -364,7 +364,6 @@ struct arch_domain int tsc_mode; /* see include/asm-x86/time.h */ bool_t vtsc; /* tsc is emulated (may change after migrate)= */ s_time_t vtsc_last; /* previous TSC value (guarantee monotonicity= ) */ - spinlock_t vtsc_lock; uint64_t vtsc_offset; /* adjustment for save/restore/migrate */ uint32_t tsc_khz; /* cached guest khz for certain emulated or hardware TSC scaling cases */ --=20 2.7.4 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel