From nobody Fri May 3 09:36:20 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; arc=pass (i=1dmarc=pass fromdomain=epam.com); dmarc=pass(p=quarantine dis=none) header.from=epam.com ARC-Seal: i=2; a=rsa-sha256; t=1591921404; cv=pass; d=zohomail.com; s=zohoarc; b=dWL4ooW27+rreL3pBKPxZ83XkPA1dra6xBjkkVWCPi47zrxYL0ljg9UPt/1OcsfYUeGVZOZOqp7vNYenUDZp8N6T2DbAzWpldFYO8uteTfHAgqiEyg/AiVd60A7uyCoD5ClWGS9U596Oon44zFj01SDeR6kW75ryvY6f+78NeIA= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1591921404; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=8n6iL6VdFsc8wy50qekTiQl7oB+xdogsCttW9HtWe3c=; b=ZYVyxHYnp1ZuAhOfgVe/IkJCymI4xeVlZh38FxjRO0ygsMmpEySQd93CBPFSm+sd60G2whpS/+u2angkz6BRE2a09Yg+yTWX4UprLr5l5kkaqTdKgAe7U8Wbzbm/nCj1GziPHODPt6RrB47TH83XHbQ8n5SslrAVQplzayrNirY= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; arc=pass (i=1dmarc=pass fromdomain=epam.com); dmarc=pass header.from= (p=quarantine dis=none) header.from= Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 15919214043901011.2771422073855; Thu, 11 Jun 2020 17:23:24 -0700 (PDT) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jjXTA-0007NN-DT; Fri, 12 Jun 2020 00:22:52 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jjXT9-0007MF-76 for xen-devel@lists.xenproject.org; Fri, 12 Jun 2020 00:22:51 +0000 Received: from EUR04-VI1-obe.outbound.protection.outlook.com (unknown [40.107.8.49]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id d38d35c0-ac42-11ea-b594-12813bfff9fa; Fri, 12 Jun 2020 00:22:41 +0000 (UTC) Received: from VI1PR0302MB3407.eurprd03.prod.outlook.com (2603:10a6:803:23::18) by VI1PR0302MB2654.eurprd03.prod.outlook.com (2603:10a6:800:e0::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3088.22; Fri, 12 Jun 2020 00:22:36 +0000 Received: from VI1PR0302MB3407.eurprd03.prod.outlook.com ([fe80::a01e:c0b1:7174:4c75]) by VI1PR0302MB3407.eurprd03.prod.outlook.com ([fe80::a01e:c0b1:7174:4c75%5]) with mapi id 15.20.3088.018; Fri, 12 Jun 2020 00:22:36 +0000 X-Inumbo-ID: d38d35c0-ac42-11ea-b594-12813bfff9fa ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=mcP4sA+hEQVJF1W6Y9DzG8r+tQez69RJY1B1fyJTUs1IR9R2OfAYVgzYUypwYlA0WXhY3ki533i7dVFvQ90dXMaI80uXA4iqxcffQjqyIx3SXgifMdK3g8XuZYY52ysdd7dqNYLzEebBIJFdjO5YBAyOQxG7ClFDs542XaglFfB4GA5Q7hE2r8VNgB8thitHmGBWgIyifPKt3HviXB9WbJwxvhwuS3j4vM+74eXlR1xTlxUEX1lnxFbeXz9OOrqysrA3FrvgVTRxBVSuqLaKi6Xmga8++eXhGI1Xw9rg3IlFFpMwcP/P1KUHTLDdU9hwrrBBLxhoRFZAeL7kr7gFnQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=8n6iL6VdFsc8wy50qekTiQl7oB+xdogsCttW9HtWe3c=; b=hRYEygeO19t/RydII5dQd+S0i0PuJpDBo14j8oy6PcG7Du4EgmTEiaa4TiW6iv0JIF5A5stYPsrhVFiA1cl0XBIkm9Ji9+7YvHzmAUBURu9d1uWgqOPuY/U6/Rl9wLPSUhwNT7OBiExEcvm0/DJNG5a1aKNWvnNPjP24U3iaGXt2Hs74Hrnv5teGDW9CWve4S5I3YC/SWqx3hZUXU3oSZr6+C174kPkdMN1qHPsSPDImeiw+f5FTHhCxuxMd1vgQigNg0WLeI2SBHIvCZN/aIWKZO4VEshx5AtKKxwCmTc5TyZ82vrCjBGX1XoK23AxZ24prkaqODbICHKQJWvWL5Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com; dkim=pass header.d=epam.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=8n6iL6VdFsc8wy50qekTiQl7oB+xdogsCttW9HtWe3c=; b=xjEHcBSjLKdQGJ45QJ4OUNfVtLN+yE1e1FP+/fZadJc0msL9J+csATfp++5vWmFyfpuEXiK8jfGOxx1/Iki9lNDw9ox7owlOosy3HJLrCyoi5uDRTvEMwYVb823RZpewYSXHU1m5UBBmZpEOnuFNPlNQPOUWNR0TmVuDHS2QrIVMvgScIRTpsw8/tZjk+5Q2HdDK/rd1lXlQk0ljb58ant3aXTtXxsHtBjyiv1pAtaMGsTZoyKo5Mhg9twgBKJkpkbHOn8nXzy/iUcBRkMlaR7sHM3lk7kYhxCKtNf0R05hEthxf+u+GmKM3F6HjS9RBEVpwN2tc5UEI9sTp1gOw+w== From: Volodymyr Babchuk To: "xen-devel@lists.xenproject.org" Subject: [RFC PATCH v1 1/6] sched: track time spent in IRQ handler Thread-Topic: [RFC PATCH v1 1/6] sched: track time spent in IRQ handler Thread-Index: AQHWQE+STyWApsQdk0CG54zgwiPygA== Date: Fri, 12 Jun 2020 00:22:36 +0000 Message-ID: <20200612002205.174295-2-volodymyr_babchuk@epam.com> References: <20200612002205.174295-1-volodymyr_babchuk@epam.com> In-Reply-To: <20200612002205.174295-1-volodymyr_babchuk@epam.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: lists.xenproject.org; dkim=none (message not signed) header.d=none;lists.xenproject.org; dmarc=none action=none header.from=epam.com; x-originating-ip: [176.36.48.175] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 9d142841-7f32-43ca-1742-08d80e66b585 x-ms-traffictypediagnostic: VI1PR0302MB2654: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:6430; x-forefront-prvs: 0432A04947 x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: A3DHQlteC/uRdv11c01jNEar4XsYAKwmOfpvHOPK2ntTM4kd6C8WD67kIQATvoBx9L2LWqa7SIc3PxwJIYftMuSja+IskTrPzBtrL610WK1UyK3+pZTYyAJ6SbOAEI+kfptfw2u4PG+XLDMIRZJMMDQfXEkuPa3Vcsipw4ptdvBhXlMYPIwSZJ7P3AyEzvreHxWajKwGYEVIkx7sFTnZTIG22x1ZQcH6ZobiWvNteBlr9yiTOIox6aHYAVwN2MDQGS+pz/hZbNTEiZ0X3hfh5rUoOstFT3YHClw3Mf2AO3rYOKqalIylQErnKCdbrflyTPcaUMUxILTgGKMa8M3gSDE1kqvhBdczvTjVgMndQ4oXJAg7CE2qlC78UcIT9MjH x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:VI1PR0302MB3407.eurprd03.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(376002)(396003)(366004)(39860400002)(136003)(346002)(66446008)(64756008)(478600001)(66946007)(66556008)(76116006)(66476007)(26005)(91956017)(36756003)(8676002)(86362001)(55236004)(186003)(8936002)(4326008)(6506007)(7416002)(54906003)(6512007)(2906002)(6486002)(316002)(5660300002)(83380400001)(6916009)(71200400001)(1076003)(2616005)(21314003); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: moyzJTyMdelEEhrkI8khnrHuQvBY3ENfYyyw2lctvcOpCNPjPehzM0cstOKmAyOycpxvDut0/jJjQvx51DOh5PMFOcrJI4JLb3J5p1dxogGKHfScfDkA5fXANDIKa9bEsJqhIBtY8nq3aw0Cx4PLjPLKTRVvA++3UeCcs6j8FhMlAdJQH+TvDVF57sm1s4XTsUZoB2NgO6Z0OIFTZXppsxcVbNg2dRNhbrL+7I0zBCaYJadSN2Tp50ChC/78LCNJShrUwVjmTMDK8rxLcpu7rZydfF4i/wKX6Mc+zQong7MykOO81TI5XIPxEOUFtaWSgJmpFwZ4jWkkUwrUCwioqPR9dAKuEuWBsEktpo9FL4NRM+DMOOM1XnIdvNb0Nic8ZXaSkhTl4xAdIiZ6ZEJo61JGRK2hTagacS7MeaqmeTk/85qjBkzW0yYLwdMX5C0K5m4YlL3clH1DhcNr+mefjgexuIqSdqErB8iBd9QfNrA= Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: epam.com X-MS-Exchange-CrossTenant-Network-Message-Id: 9d142841-7f32-43ca-1742-08d80e66b585 X-MS-Exchange-CrossTenant-originalarrivaltime: 12 Jun 2020 00:22:36.3359 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: BYPRm6ZMprkAWgcLBms6TtzX9LlRvESjFYyChLURXmpabqyq5woLdP1h2tGSVp2ih7mSTYD3VjEyl3V2GfveVg/nzkwHoRu9YNUGInmdDJE= X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0302MB2654 X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Stefano Stabellini , Julien Grall , Wei Liu , Andrew Cooper , Ian Jackson , George Dunlap , Dario Faggioli , Jan Beulich , Volodymyr Babchuk , =?iso-8859-1?Q?Roger_Pau_Monn=E9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-ZohoMail-DKIM: pass (identity @epam.com) Content-Type: text/plain; charset="utf-8" Add code that saves time spent in IRQ handler, so later we can make adjustments to schedule unit run time. This and following changes are called upon to provide fair scheduling. Problem is that any running vCPU can be interrupted by to handle IRQ which is bound to some other vCPU. Thus, current vCPU can be charged for a time, it actually didn't used. TODO: move vcpu_{begin|end}_irq_handler() calls to entry.S for even more fair time tracking. Signed-off-by: Volodymyr Babchuk --- xen/arch/arm/irq.c | 2 ++ xen/arch/x86/irq.c | 2 ++ xen/common/sched/core.c | 29 +++++++++++++++++++++++++++++ xen/include/xen/sched.h | 13 +++++++++++++ 4 files changed, 46 insertions(+) diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c index 3877657a52..51b517c0cd 100644 --- a/xen/arch/arm/irq.c +++ b/xen/arch/arm/irq.c @@ -201,6 +201,7 @@ void do_IRQ(struct cpu_user_regs *regs, unsigned int ir= q, int is_fiq) struct irq_desc *desc =3D irq_to_desc(irq); struct irqaction *action; =20 + vcpu_begin_irq_handler(); perfc_incr(irqs); =20 ASSERT(irq >=3D 16); /* SGIs do not come down this path */ @@ -267,6 +268,7 @@ out: out_no_end: spin_unlock(&desc->lock); irq_exit(); + vcpu_end_irq_handler(); } =20 void release_irq(unsigned int irq, const void *dev_id) diff --git a/xen/arch/x86/irq.c b/xen/arch/x86/irq.c index a69937c840..3ef4221b64 100644 --- a/xen/arch/x86/irq.c +++ b/xen/arch/x86/irq.c @@ -1895,6 +1895,7 @@ void do_IRQ(struct cpu_user_regs *regs) int irq =3D this_cpu(vector_irq)[vector]; struct cpu_user_regs *old_regs =3D set_irq_regs(regs); =20 + vcpu_begin_irq_handler(); perfc_incr(irqs); this_cpu(irq_count)++; irq_enter(); @@ -2024,6 +2025,7 @@ void do_IRQ(struct cpu_user_regs *regs) out_no_unlock: irq_exit(); set_irq_regs(old_regs); + vcpu_end_irq_handler(); } =20 static inline bool is_free_pirq(const struct domain *d, diff --git a/xen/common/sched/core.c b/xen/common/sched/core.c index cb49a8bc02..8f642ada05 100644 --- a/xen/common/sched/core.c +++ b/xen/common/sched/core.c @@ -916,6 +916,35 @@ void vcpu_unblock(struct vcpu *v) vcpu_wake(v); } =20 +void vcpu_begin_irq_handler(void) +{ + if (is_idle_vcpu(current)) + return; + + /* XXX: Looks like ASSERT_INTERRUPTS_DISABLED() is available only for = x86 */ + if ( current->irq_nesting++ ) + return; + + current->irq_entry_time =3D NOW(); +} + +void vcpu_end_irq_handler(void) +{ + int delta; + + if (is_idle_vcpu(current)) + return; + + ASSERT(current->irq_nesting); + + if ( --current->irq_nesting ) + return; + + /* We assume that irq handling time will not overflow int */ + delta =3D NOW() - current->irq_entry_time; + atomic_add(delta, ¤t->sched_unit->irq_time); +} + /* * Do the actual movement of an unit from old to new CPU. Locks for *both* * CPUs needs to have been taken already when calling this! diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h index ac53519d7f..ceed53364b 100644 --- a/xen/include/xen/sched.h +++ b/xen/include/xen/sched.h @@ -237,6 +237,9 @@ struct vcpu evtchn_port_t virq_to_evtchn[NR_VIRQS]; spinlock_t virq_lock; =20 + /* Fair scheduling state */ + uint64_t irq_entry_time; + unsigned int irq_nesting; /* Tasklet for continue_hypercall_on_cpu(). */ struct tasklet continue_hypercall_tasklet; =20 @@ -276,6 +279,9 @@ struct sched_unit { /* Vcpu state summary. */ unsigned int runstate_cnt[4]; =20 + /* Fair scheduling correction value */ + atomic_t irq_time; + /* Bitmask of CPUs on which this VCPU may run. */ cpumask_var_t cpu_hard_affinity; /* Used to save affinity during temporary pinning. */ @@ -690,6 +696,13 @@ long vcpu_yield(void); void vcpu_sleep_nosync(struct vcpu *v); void vcpu_sleep_sync(struct vcpu *v); =20 +/* + * Report IRQ handling time to scheduler. As IRQs can be nested, + * next two functions are re-enterable. + */ +void vcpu_begin_irq_handler(void); +void vcpu_end_irq_handler(void); + /* * Force synchronisation of given VCPU's state. If it is currently desched= uled, * this call will ensure that all its state is committed to memory and that --=20 2.27.0 From nobody Fri May 3 09:36:20 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; arc=pass (i=1dmarc=pass fromdomain=epam.com); dmarc=pass(p=quarantine dis=none) header.from=epam.com ARC-Seal: i=2; a=rsa-sha256; t=1591921404; cv=pass; d=zohomail.com; s=zohoarc; b=gDCzbFtmS/3+1WKtF6iNzzbYQZsoi4NEglgqRVGVf5rbExyHCLtXQQwTqn8cbvskduRWkgp6Pupo3ahUnwa8CEGc7VPwu4y2VflFySJ3Zi9Me9iB6T5362nqVcJ14KfR8DaQmdYNgClJH7YLhG/K5CwLdNzRxcRQWMEVIyiHz3M= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1591921404; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=5kLqXND8iIZTqE5y3gEpFrrEeZngnuQJdxwT84/FVNk=; b=LN55hVA+rK6M1H60t9Uvbklz+JBjE99W0xEe6Ns/sJ4gcvPgnUUQ7V48hVMvSsndyWNJguiX8xe9m+zZ8b/1aPEgtbCBNEKSsbCVJG0uMM7xSlRf7mBXsmj9FIR2GurNCWyEM6vRTPxI50oykT2RJn36kQ4wWnwr+SBpxgh279M= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; arc=pass (i=1dmarc=pass fromdomain=epam.com); dmarc=pass header.from= (p=quarantine dis=none) header.from= Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1591921404742456.1415602739213; Thu, 11 Jun 2020 17:23:24 -0700 (PDT) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jjXT5-0007Mu-4m; Fri, 12 Jun 2020 00:22:47 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jjXT4-0007MF-6t for xen-devel@lists.xenproject.org; Fri, 12 Jun 2020 00:22:46 +0000 Received: from EUR04-VI1-obe.outbound.protection.outlook.com (unknown [40.107.8.89]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id d401da06-ac42-11ea-b594-12813bfff9fa; Fri, 12 Jun 2020 00:22:41 +0000 (UTC) Received: from VI1PR0302MB3407.eurprd03.prod.outlook.com (2603:10a6:803:23::18) by VI1PR0302MB2654.eurprd03.prod.outlook.com (2603:10a6:800:e0::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3088.22; Fri, 12 Jun 2020 00:22:37 +0000 Received: from VI1PR0302MB3407.eurprd03.prod.outlook.com ([fe80::a01e:c0b1:7174:4c75]) by VI1PR0302MB3407.eurprd03.prod.outlook.com ([fe80::a01e:c0b1:7174:4c75%5]) with mapi id 15.20.3088.018; Fri, 12 Jun 2020 00:22:37 +0000 X-Inumbo-ID: d401da06-ac42-11ea-b594-12813bfff9fa ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=jo9NyPPpKEn+J8Voeiv3iGJVohkKdj4IWKAB+DnRIkFLM0GTps00q98t8AbRURO1mbF77azVaagcAoklvK1sSy5VuGl4X8qO7nju8QHFRqpxMLgLymcIZGLmCntv9oU5aj94EXaNniT296MpT5eFK0T1A8xVkwKjtRvqhMBgJWdaZwmP0HgMh+6iePT1e/tdim0ESe3D3bu4qIRdx/7xEzM+YL516PDMds3mI/fuX4PyXi8fdjqKdnldzTsWIlAPLbGX4pPAu4meZYTTF3hobGtOlXjYVak7puwtRGLiLHBF6nRcr4hMZNYxRmNCR361efJkqqVJmQ4ZuJcz/ZLn+w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=5kLqXND8iIZTqE5y3gEpFrrEeZngnuQJdxwT84/FVNk=; b=hYtBRad60UC73911tFjnOdEa+2150Fg3x9OMkfBzc3IV7D0WcGNSYraIHbYQCnMLW04Cnt+2kx62ibjurpFL2v9KtuZnDkCHkruS1mQ6E0KZAmAbH/Gr5neKRlpiO7YqYd1JI2wX5kzuUMQVnEd9o/8xOLCtCao6CqcxsKpGwOowNlu2vHlwcgUZkj6UfjlAWTu4mitAFPp5Y3YgE9dTVBh+Z4GEi4PyYRYkR7Z4uC+NOKEWLaKnPoEEN2IZZsJZhZKSxTxEMAEv8d9WvzR7GvQYKH9f+z0N92cuO2dLp2NNLWmy9zyuNToZFa9wCe4FHU3dlOPnCfT8nHkX4RUS2Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com; dkim=pass header.d=epam.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=5kLqXND8iIZTqE5y3gEpFrrEeZngnuQJdxwT84/FVNk=; b=3aroGyhRoiUXnrA8zoEiBjScWWCWvDFx7r+MhI1yPCFfzTgZ4zKc1bSEEaW5nZMYwLvFx9vCvz7TXBDokwai/xwQBPB0zp+I5ZI2lOsF2VcJrRqvTqUty5+yNe8hr06tFHOuhzjxS5v26m7bhQiTCWML6A2JF8x0PfCWLTJt5/bGKivAixFzj0Mx5VYMKHkGU7bblgZYcRgwbLrHus2h0vdbl3xVcAf0W6a4B8z4b6mfRvSuq72b1AUcS2e1O0hktYaTl1sAdRbkVBrhmzsL1TrjQEgSnOovbQe72h7uSZ/9Ss2PkDYvYzj0WyKFmwgoL5PP6I/lMfmP66JRTRk3wA== From: Volodymyr Babchuk To: "xen-devel@lists.xenproject.org" Subject: [RFC PATCH v1 2/6] sched: track time spent in hypervisor tasks Thread-Topic: [RFC PATCH v1 2/6] sched: track time spent in hypervisor tasks Thread-Index: AQHWQE+SUtF98x/3akmRxwBff1s+mw== Date: Fri, 12 Jun 2020 00:22:36 +0000 Message-ID: <20200612002205.174295-3-volodymyr_babchuk@epam.com> References: <20200612002205.174295-1-volodymyr_babchuk@epam.com> In-Reply-To: <20200612002205.174295-1-volodymyr_babchuk@epam.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: lists.xenproject.org; dkim=none (message not signed) header.d=none;lists.xenproject.org; dmarc=none action=none header.from=epam.com; x-originating-ip: [176.36.48.175] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 6defc929-7bcc-49ef-5dcb-08d80e66b5b4 x-ms-traffictypediagnostic: VI1PR0302MB2654: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:9508; x-forefront-prvs: 0432A04947 x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: x1+xAcMfAXSvFiQX8tqHfR67YXzSp0qQY+JrI8vrE1QDS8ZmLsi+zvyOY4QuEqWslt3TbnoI74whZN7vZsoM3SCQWnKauFoIs6q5VB1Ob2Z1DyXhb/Ivi2nPR+9pEiOADj1FUULp/g1XZEgAgkd8ME9f8ZGeYEIgYjmF9wcGqyTqtDeOghHA4RuOyh73HW4j/4rVNRe/swfpAUhyrIsaBHvSBp4OBGDt4hvNsOmCU7ItIQYMu+ly6MUezan6JAgvp8APxk+isDAlfd5dw4PkMIJ/o6KB4h+xFbRr8psuvr49sFL3uG8MWY1jol/3ocdXb952rWTYdBOFyLMnZkJT6Q== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:VI1PR0302MB3407.eurprd03.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(376002)(396003)(366004)(39860400002)(136003)(346002)(66446008)(64756008)(478600001)(66946007)(66556008)(76116006)(66476007)(26005)(91956017)(36756003)(8676002)(86362001)(55236004)(186003)(8936002)(4326008)(6506007)(54906003)(6512007)(2906002)(6486002)(316002)(5660300002)(83380400001)(6916009)(71200400001)(1076003)(2616005); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: fazK2nhL2+zYI8zNGHh+oRHoCK01Nb1SRwRu6hAp1gbhcvKwv6DvzmKuuEY29l70yE/jwh6j4NmuomG8mpwX0oH+5fMWMj5PKyi/K7pawWmOmpiE0173QUkPuYAyWrK4+FoG8pxTCu2tSrIFqDn5h75WcZyHdHT+BmHklivHKvQkNMKWwEJVQ4249MPGoqZYjPEBXolnOAi6f8t5feCUctNiamkV8sqRaTTX9VQPCAwyAjtxeRmmTAIWGrYXP3+dIRFLEsgpYMSzll4WTcBm7eeUfHhbSc0M5pzk7fNUAiunCnCItaTqNjvp4O8qJnvwcarbd3e8ol4ZqUmpSg7PkGL8duw+6sZZZCp8mKeLV9GTosbXQqwZ0Obsvwp63RS9/8hYVArm3hcq4qAgLNqFnXyrFYiLJOV5wzPVrj5YHArf8OZ/C6/E+yN6Iv+OOW1ngBiqAKGE0DSHXWAsfp8mkwdSNYu47illWy6PxrWxUZ0= Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: epam.com X-MS-Exchange-CrossTenant-Network-Message-Id: 6defc929-7bcc-49ef-5dcb-08d80e66b5b4 X-MS-Exchange-CrossTenant-originalarrivaltime: 12 Jun 2020 00:22:36.6637 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: oF7wcJr5jLlKnmkhhLRDCZY73ubGoERNa5aD1tl/ppUX00kRfiFW8opx13jBOm8zZBmGlDIIzIZQD6vv31vkLy0JxW8us1g56LZZa037mS8= X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0302MB2654 X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Stefano Stabellini , Julien Grall , Wei Liu , Andrew Cooper , Ian Jackson , George Dunlap , Dario Faggioli , Jan Beulich , Volodymyr Babchuk Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-ZohoMail-DKIM: pass (identity @epam.com) Content-Type: text/plain; charset="utf-8" In most cases hypervisor code performs guest-related jobs. Tasks like hypercall handling or MMIO access emulation are done for calling vCPU so it is okay to charge time spent in hypervisor to the current vCPU. But, there are also tasks that are not originated from guests. This includes things like TLB flushing or running tasklets. We don't want to track time spent in this tasks to a total scheduling unit run time. So we need to track time spent in such housekeeping tasks separately. Those hypervisor tasks are run in do_softirq() function, so we'll install our hooks there. TODO: This change is not tested on ARM, and probably we'll get a failing assertion there. This is because ARM code exits from schedule() and have chance to get to end of do_softirq(). Signed-off-by: Volodymyr Babchuk --- xen/common/sched/core.c | 32 ++++++++++++++++++++++++++++++++ xen/common/softirq.c | 2 ++ xen/include/xen/sched.h | 16 +++++++++++++++- 3 files changed, 49 insertions(+), 1 deletion(-) diff --git a/xen/common/sched/core.c b/xen/common/sched/core.c index 8f642ada05..d597811fef 100644 --- a/xen/common/sched/core.c +++ b/xen/common/sched/core.c @@ -945,6 +945,37 @@ void vcpu_end_irq_handler(void) atomic_add(delta, ¤t->sched_unit->irq_time); } =20 +void vcpu_begin_hyp_task(struct vcpu *v) +{ + if ( is_idle_vcpu(v) ) + return; + + ASSERT(!v->in_hyp_task); + + v->hyp_entry_time =3D NOW(); +#ifndef NDEBUG + v->in_hyp_task =3D true; +#endif +} + +void vcpu_end_hyp_task(struct vcpu *v) +{ + int delta; + + if ( is_idle_vcpu(v) ) + return; + + ASSERT(v->in_hyp_task); + + /* We assume that hypervisor task time will not overflow int */ + delta =3D NOW() - v->hyp_entry_time; + atomic_add(delta, &v->sched_unit->hyp_time); + +#ifndef NDEBUG + v->in_hyp_task =3D false; +#endif +} + /* * Do the actual movement of an unit from old to new CPU. Locks for *both* * CPUs needs to have been taken already when calling this! @@ -2615,6 +2646,7 @@ static void schedule(void) =20 SCHED_STAT_CRANK(sched_run); =20 + vcpu_end_hyp_task(current); rcu_read_lock(&sched_res_rculock); =20 lock =3D pcpu_schedule_lock_irq(cpu); diff --git a/xen/common/softirq.c b/xen/common/softirq.c index 063e93cbe3..03a29384d1 100644 --- a/xen/common/softirq.c +++ b/xen/common/softirq.c @@ -71,7 +71,9 @@ void process_pending_softirqs(void) void do_softirq(void) { ASSERT_NOT_IN_ATOMIC(); + vcpu_begin_hyp_task(current); __do_softirq(0); + vcpu_end_hyp_task(current); } =20 void open_softirq(int nr, softirq_handler handler) diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h index ceed53364b..51dc7c4551 100644 --- a/xen/include/xen/sched.h +++ b/xen/include/xen/sched.h @@ -239,7 +239,12 @@ struct vcpu =20 /* Fair scheduling state */ uint64_t irq_entry_time; + uint64_t hyp_entry_time; unsigned int irq_nesting; +#ifndef NDEBUG + bool in_hyp_task; +#endif + /* Tasklet for continue_hypercall_on_cpu(). */ struct tasklet continue_hypercall_tasklet; =20 @@ -279,8 +284,9 @@ struct sched_unit { /* Vcpu state summary. */ unsigned int runstate_cnt[4]; =20 - /* Fair scheduling correction value */ + /* Fair scheduling correction values */ atomic_t irq_time; + atomic_t hyp_time; =20 /* Bitmask of CPUs on which this VCPU may run. */ cpumask_var_t cpu_hard_affinity; @@ -703,6 +709,14 @@ void vcpu_sleep_sync(struct vcpu *v); void vcpu_begin_irq_handler(void); void vcpu_end_irq_handler(void); =20 +/* + * Report to scheduler when we are doing housekeeping tasks on the + * current vcpu. This is called during do_softirq() but can be called + * anywhere else. + */ +void vcpu_begin_hyp_task(struct vcpu *v); +void vcpu_end_hyp_task(struct vcpu *v); + /* * Force synchronisation of given VCPU's state. If it is currently desched= uled, * this call will ensure that all its state is committed to memory and that --=20 2.27.0 From nobody Fri May 3 09:36:20 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; arc=pass (i=1dmarc=pass fromdomain=epam.com); dmarc=pass(p=quarantine dis=none) header.from=epam.com ARC-Seal: i=2; a=rsa-sha256; t=1591921404; cv=pass; d=zohomail.com; s=zohoarc; b=i1hlN93+6hEwlsulzaDg3RMKfQ81icB+/4KMvojqNzx267Jckk+CMRS5BTQLBfrujKoVF/SPfvDV23vXnt7UDMMc16HGrgfRhnxxasuMbvGFDsS/EjPM3/I8fFRwKKOa1OOIYxkZpjQavTBcJTEhzhIBsMDyGFA2QSCMeRr+MgU= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1591921404; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=W/q3vwR74SO4t+n7E/mQzwfg4LX6JiGo9tkrL9EGFJ4=; b=J6ytykQxxzOPZLako3i9WiLysYsrhuQM0ut3hUVPf2jXXAs7dXVFwyhaM5rpOAXIgos+fpTJk/WNE6JyT8/ASAM7t8YsLLusI5XEd777el7PRE6BvznKkM6hLGIuiLJNwFYT9mXMpFrVEMQlpGLoVeYwQqYEmINGRu/Uy6hz7qk= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; arc=pass (i=1dmarc=pass fromdomain=epam.com); dmarc=pass header.from= (p=quarantine dis=none) header.from= Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1591921403934880.964849459785; Thu, 11 Jun 2020 17:23:23 -0700 (PDT) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jjXTF-0007ON-ML; Fri, 12 Jun 2020 00:22:57 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jjXTE-0007MF-7D for xen-devel@lists.xenproject.org; Fri, 12 Jun 2020 00:22:56 +0000 Received: from EUR04-VI1-obe.outbound.protection.outlook.com (unknown [40.107.8.89]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id d401da07-ac42-11ea-b594-12813bfff9fa; Fri, 12 Jun 2020 00:22:42 +0000 (UTC) Received: from VI1PR0302MB3407.eurprd03.prod.outlook.com (2603:10a6:803:23::18) by VI1PR0302MB2654.eurprd03.prod.outlook.com (2603:10a6:800:e0::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3088.22; Fri, 12 Jun 2020 00:22:37 +0000 Received: from VI1PR0302MB3407.eurprd03.prod.outlook.com ([fe80::a01e:c0b1:7174:4c75]) by VI1PR0302MB3407.eurprd03.prod.outlook.com ([fe80::a01e:c0b1:7174:4c75%5]) with mapi id 15.20.3088.018; Fri, 12 Jun 2020 00:22:37 +0000 X-Inumbo-ID: d401da07-ac42-11ea-b594-12813bfff9fa ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=RpWMGfCCZSJVSokHiAz8NWVDcBgkjz0dIICBn7VgaTItnZcYdpCeYii+awqDg+i5lDsMNCTIxtULcbjNNVsheQ1n6+besrDP3IkrJu4pbYGEWmA6MCxn9Citd4Uq1nGv5TfnhD+YlshYKtJeOP2rk/quiNo/7uHaXAtz4xLQ7SKFJSCnRJT89y9LdO1XGx8hqUiJg65PEQCb0JhBI/KuB9GBLStcZErKQH5HgRMJxyplJHAxnfuvyQYejWtGpx70XGMM8SjKWhLwzeVTkYBxyEFy6ys1Pe2ZNokHr2yzEQb9205vcqPc8rI7j2n3H/p5LABA+GKm66o/aF9W/za0eQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=W/q3vwR74SO4t+n7E/mQzwfg4LX6JiGo9tkrL9EGFJ4=; b=Fqy3ZLqxZ/pm1El9ahfddqktqa46HX0Ywo3QBPVIU94yllz/Y9YOiLIulPUdTRUzkBOP7kmV64dhz3slmYTQuG+6JASpXHG00hUzidQYhKZt/VglXVB/y2dgWPmlcKjyAYQtoUTZKrkO6SUpP1PNzeFXzlgbwpA2DcSqnopooHI3sbQgYZoYzQ6laaHqvKAMAGC+PyxHCFckXAnBHsSonS78CjkUf5LTy24ifzzByXr6gYwk9RjYvpF/bx47n6nEyVwC2VM4U2IqZDcQFFUPC4S6nP4mJPy8lIFRK9NW6jDCCJyYh7Ln9sXJsOUOMcbYE51Xv6/KE0gx4ai5dYmoxA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com; dkim=pass header.d=epam.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=W/q3vwR74SO4t+n7E/mQzwfg4LX6JiGo9tkrL9EGFJ4=; b=GaFBPZzhl2MwDnOsG9G+k9TNiApNWnl65zjLNOlneqtv6icQ16qMaHibdtxMhaWnTwyup7L5FVmtsBELux5/fiQz2aaPJK+0a16o8Fghw82ufZFaoRayTjREdwgZlCaMnD/OiPcx2vpsgFpyWnS+vWX+3rG+Svsj5/jhNiJKj60Ho1H7zFsjXZ86V2kC6AzKLkJ+/CnGcjzZuSTvvjMEY8cZGOoQg0YmHb0OlTwyrOH/mZxS9VIyM5rzXiMFAgpP8XZTxO76AepBychP5abbDhqeW638QGhxIy7LdFF2AYpO+4S9veGYV6TAWkCQUY89h5oDYgnlXxFCil8meKtpEA== From: Volodymyr Babchuk To: "xen-devel@lists.xenproject.org" Subject: [RFC PATCH v1 3/6] sched, credit2: improve scheduler fairness Thread-Topic: [RFC PATCH v1 3/6] sched, credit2: improve scheduler fairness Thread-Index: AQHWQE+TBfX1MPdkGkC6vJ7ur/uY5A== Date: Fri, 12 Jun 2020 00:22:36 +0000 Message-ID: <20200612002205.174295-4-volodymyr_babchuk@epam.com> References: <20200612002205.174295-1-volodymyr_babchuk@epam.com> In-Reply-To: <20200612002205.174295-1-volodymyr_babchuk@epam.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: lists.xenproject.org; dkim=none (message not signed) header.d=none;lists.xenproject.org; dmarc=none action=none header.from=epam.com; x-originating-ip: [176.36.48.175] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: de4db258-52dd-4a6d-14f8-08d80e66b5e6 x-ms-traffictypediagnostic: VI1PR0302MB2654: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:249; x-forefront-prvs: 0432A04947 x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: MST4SVLzE67xLf4DmB6pbDIntKiMPc29u+C5QgOqr/3L8YM2uwcxfj8HwSM1RrwWwmeybo+xty+KUU8vW4QBthBV8u6HRwTiEm5eDaKNzP58RpGIg9a5oxvLZJWs44G8NcTlMFo1GdEOT8KgzexWzxAfKi1v74UT9uQ/WNRDlIzVJsEL44xDXGQlR5xP1wH/JMpSTp6urY1qLew3HT23+Zsl4lhoPj1XqOAGtjXVyFKvaSOdfAb85i1pAAry9/X78FYqTJ1Ao53BCZAUJCkatRHPUA1K5+JvCnoIIoSJ5aAP5G/hqsiezNrrpq9hmAnu x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:VI1PR0302MB3407.eurprd03.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(376002)(396003)(366004)(39860400002)(136003)(346002)(66446008)(64756008)(478600001)(66946007)(66556008)(76116006)(66476007)(26005)(91956017)(36756003)(8676002)(86362001)(55236004)(186003)(8936002)(4326008)(6506007)(54906003)(6512007)(2906002)(6486002)(316002)(5660300002)(83380400001)(6916009)(71200400001)(1076003)(2616005); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: 8RLhe3RMRbjQT8lwxVw2mQ4vDohuoTAqS9JK4p2KTlYWKGYRZlfougZneij4TSNmVjZ/zb0ccbJjmVVcnYeqUqtLCxc8OKhI0GnOGzzm2qF0F5MOiI0uXErNIegCADFdTl7OW31RfaDt3N5ya46Gz2NGazabwC3TfU6b57/tlRczk7Xh1r/N8db76bCdRVseoWy94m9xnxG4dxCCwEWv4nABVWlwR9hTwJRWSYODMZkDjvL0+ks0Aiwa6R9sHvfRF2Dq2xiYGIahoToTjFca0lwhu1BUAHvYhigpmZ/9DK/NgrkgXPE9yestkqq/jPS5GXC9h+KWy4VrjTL3S1XUUnavzwQYYWJSshMZCkdyMuOmksfa6PVykG5v2y5u7+xnAZtzlDPIrFjwGAxDEBIpXe7Ui4az3jCc1kdlPV8+qXsT/LEXNjqGIgzRnGvZeuxjGiwhfG1btjuk17X902GLynVsi+tB3pmdJPtZQdTQUKs= Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: epam.com X-MS-Exchange-CrossTenant-Network-Message-Id: de4db258-52dd-4a6d-14f8-08d80e66b5e6 X-MS-Exchange-CrossTenant-originalarrivaltime: 12 Jun 2020 00:22:36.9255 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: GA0TCH7Ztw0IguOTKlSCcOSNU3bchUtqDPJEDwkZedpLtOh/BDZlWtFmD6uF3QJrq+Dw7I5O9/SDdZgIxeKg0t75I5qvP2A58UeyLYNPkg0= X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0302MB2654 X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Volodymyr Babchuk , George Dunlap , Dario Faggioli Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-ZohoMail-DKIM: pass (identity @epam.com) Content-Type: text/plain; charset="utf-8" Now we can make corrections for scheduling unit run time, based on data gathered in previous patches. This includes time spent in IRQ handlers and time spent for hypervisor housekeeping tasks. Those time spans needs to be deduced from a total run time. This patch adds sched_get_time_correction() function which returns time correction value. This value should be subtracted by a scheduler implementation from a total vCPU/shced_unit run time. TODO: Make the corresponding changes to all other schedulers. Signed-off-by: Volodymyr Babchuk --- xen/common/sched/core.c | 23 +++++++++++++++++++++++ xen/common/sched/credit2.c | 2 +- xen/common/sched/private.h | 10 ++++++++++ 3 files changed, 34 insertions(+), 1 deletion(-) diff --git a/xen/common/sched/core.c b/xen/common/sched/core.c index d597811fef..a7294ff5c3 100644 --- a/xen/common/sched/core.c +++ b/xen/common/sched/core.c @@ -974,6 +974,29 @@ void vcpu_end_hyp_task(struct vcpu *v) #ifndef NDEBUG v->in_hyp_task =3D false; #endif + +s_time_t sched_get_time_correction(struct sched_unit *u) +{ + unsigned long flags; + int irq, hyp; + + while ( true ) + { + irq =3D atomic_read(&u->irq_time); + if ( likely( irq =3D=3D atomic_cmpxchg(&u->irq_time, irq, 0)) ) + break; + } + + while ( true ) + { + hyp =3D atomic_read(&u->hyp_time); + if ( likely( hyp =3D=3D atomic_cmpxchg(&u->hyp_time, hyp, 0)) ) + break; + } + + return irq + hyp; +} + } =20 /* diff --git a/xen/common/sched/credit2.c b/xen/common/sched/credit2.c index 34f05c3e2a..7a0aca078b 100644 --- a/xen/common/sched/credit2.c +++ b/xen/common/sched/credit2.c @@ -1722,7 +1722,7 @@ void burn_credits(struct csched2_runqueue_data *rqd, return; } =20 - delta =3D now - svc->start_time; + delta =3D now - svc->start_time - sched_get_time_correction(svc->unit); =20 if ( unlikely(delta <=3D 0) ) { diff --git a/xen/common/sched/private.h b/xen/common/sched/private.h index b9a5b4c01c..3f4859ce23 100644 --- a/xen/common/sched/private.h +++ b/xen/common/sched/private.h @@ -604,4 +604,14 @@ void cpupool_put(struct cpupool *pool); int cpupool_add_domain(struct domain *d, int poolid); void cpupool_rm_domain(struct domain *d); =20 +/* + * Get amount of time spent doing non-guest related work on + * current scheduling unit. This includes time spent in soft IRQs + * and in hardware interrupt handlers. + * + * Call to this function resets the counters, so it is supposed to + * be called when scheduler calculates time used by the scheduling + * unit. + */ +s_time_t sched_get_time_correction(struct sched_unit *u); #endif /* __XEN_SCHED_IF_H__ */ --=20 2.27.0 From nobody Fri May 3 09:36:20 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; arc=pass (i=1dmarc=pass fromdomain=epam.com); dmarc=pass(p=quarantine dis=none) header.from=epam.com ARC-Seal: i=2; a=rsa-sha256; t=1591921406; cv=pass; d=zohomail.com; s=zohoarc; b=guwYNJex+vjcr2zduuC6FyFjj6BgjtwXtrY3VtIaq4zWHo0BahI0kc4UVnFICzeWoWoo4KijlALQ3ze1kmm02dWi7RaX03aQ/fCOmoAMjFImXtnZYyuwxjaQ+upXIGsVLOCW96X6hhY8OX2Pzaz1ehJ48MiVNecKqJXUXkdYYxQ= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1591921406; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=rPxSP0GQhjiILc2BkcY4EKDjsGfYhx91tgpTPBpzd0E=; b=aHtUuzElpjgfEEwZs0amnsElubpKgvNkgd45ZRDa9XI+BEcXJiUZPGNww15whuoFl75aZdSabfd9CmkSG6Pxdg+vkjlnkqsqS7/khfCAKWdUpxSZN8WqyMi4axq13G4eR4XWi9h9UQPM9m5lr2376TGxN/MX9xryMfeFh/3OsIc= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; arc=pass (i=1dmarc=pass fromdomain=epam.com); dmarc=pass header.from= (p=quarantine dis=none) header.from= Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1591921406023806.9080863910228; Thu, 11 Jun 2020 17:23:26 -0700 (PDT) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jjXTK-0007QC-3c; Fri, 12 Jun 2020 00:23:02 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jjXTJ-0007MF-7C for xen-devel@lists.xenproject.org; Fri, 12 Jun 2020 00:23:01 +0000 Received: from EUR04-VI1-obe.outbound.protection.outlook.com (unknown [40.107.8.49]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id d4c65386-ac42-11ea-b594-12813bfff9fa; Fri, 12 Jun 2020 00:22:42 +0000 (UTC) Received: from VI1PR0302MB3407.eurprd03.prod.outlook.com (2603:10a6:803:23::18) by VI1PR0302MB2654.eurprd03.prod.outlook.com (2603:10a6:800:e0::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3088.22; Fri, 12 Jun 2020 00:22:37 +0000 Received: from VI1PR0302MB3407.eurprd03.prod.outlook.com ([fe80::a01e:c0b1:7174:4c75]) by VI1PR0302MB3407.eurprd03.prod.outlook.com ([fe80::a01e:c0b1:7174:4c75%5]) with mapi id 15.20.3088.018; Fri, 12 Jun 2020 00:22:37 +0000 X-Inumbo-ID: d4c65386-ac42-11ea-b594-12813bfff9fa ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Vnqa+hG5vBpP5PV2b7uIX+OWR91yp8O1vutZIGwzUvcUs722S8ipAWzMIpFsJmotvH+VFtOPsUmoFf8Xqndg7H8DHVLphgcWtrBSKfk0w9KLiLC25TxfWJ+drA6xbTcFev06X7LY3OjYvxQz7ht3FdfbcU5W7DJP9Cj6sV6jlsUncLVCZ2W9WriY5ny65zoMW1rSxOZwaKWKUL7yAm2E5o7WvXq+Eh+f/eaSo0h/ZM/NFTQ1zjSYdMKh+VdW0mYunXOAsUJhgbesV5UYk/S1u9Pu9H2OJ1VoQ+XEz6yb/ghAIsqsYjreJm5qH3iR82LCFWYfz/jaLCBW5v8CcroKZw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=rPxSP0GQhjiILc2BkcY4EKDjsGfYhx91tgpTPBpzd0E=; b=f3HvGdIATHStrX5gc1XIHNo09QVKiguk+5YXsR0WR2do7f8MP2HrbpeoKTb59yZ8JFIFJUO423CvOGX9/k4wB7P9jR6bS4Y09tNw/5ZAr0bV4BGVWfJzSFj/zbGjNvjHqiclsZ+zju5USHPmpR0uzvHgqHAt7HYlTie/Fbg/JrnVq6iTz79y8rZI7rB/UqbR0Fdw5gHUbLXlcdYew+H5Sp3tOBJGwVwsMApAXwjYNsPYfwy8sCQym67y9qoBias/UOntIqaD7J8gTUMZ79w14ABHV4BF2z2hAZFJIM8U0pTdt1IbTZr0hk3Numd7uz8Mknv96SQtdY7bKqgaCFc+0w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com; dkim=pass header.d=epam.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=rPxSP0GQhjiILc2BkcY4EKDjsGfYhx91tgpTPBpzd0E=; b=CJZu8c2VGLi4KiNmikSF9I77XgmqSKXjcnOFNqsk+6NWA1oYdyULczjbrNgIX62/fx5VXiWuY6BYj3sOim7sf4g8do1JYvmUl1/cf0R39aU4GUnTHCTkhzhJ/rz/xgiCJshxBWzPYzHooWXPv70PQLBR64Pg+p/80omFWn6ooY++vMFXAS362wODCKTKd0fB0LO/dNWIjKAdJeDMJN3gF0Wx68qk08Yao1tByz6DNvPe3mysp2qgyb6ADLVvct8jyqFV+ZXpe6FAsX7m9iZ8m5HzD8Gzf35qgxLXJHddbRfphpVWv9Y8kTn0Sj2oycJSmIT7gOT4npdApJG0tEkrwA== From: Volodymyr Babchuk To: "xen-devel@lists.xenproject.org" Subject: [RFC PATCH v1 4/6] xentop: collect IRQ and HYP time statistics. Thread-Topic: [RFC PATCH v1 4/6] xentop: collect IRQ and HYP time statistics. Thread-Index: AQHWQE+Tib/gTK49mk6bUudd1VvXZg== Date: Fri, 12 Jun 2020 00:22:37 +0000 Message-ID: <20200612002205.174295-5-volodymyr_babchuk@epam.com> References: <20200612002205.174295-1-volodymyr_babchuk@epam.com> In-Reply-To: <20200612002205.174295-1-volodymyr_babchuk@epam.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: lists.xenproject.org; dkim=none (message not signed) header.d=none;lists.xenproject.org; dmarc=none action=none header.from=epam.com; x-originating-ip: [176.36.48.175] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 23a7788d-bd81-4360-1a38-08d80e66b618 x-ms-traffictypediagnostic: VI1PR0302MB2654: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:185; x-forefront-prvs: 0432A04947 x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: +WG4zCDKubqHVW8Wsa/b4Nt1wx9y59jcWALTg2ZV2P3zyhTI9J3GWnPGbDsCXruFJ50Sb9VsowieBaQdC7AKQEfGgyINj2aAUmpo3Pjs//BoB1UWKfmVOcDBOT7l/6gPhkyV/MKKkM6COFnuv/Xyj2rAwJJGn4yNA/P1vHsBr7q4UBj3zPLWJFAypWpQQWTrmiFxz1V1Ze2XiLeRIxZ8Zq+t57GNe7FphI7zB6mrV6D6470uLI81u5ArfEa3ZzAl4vSgMRvwTPHOAtGytJuKbjnq6ZP+UPtac1hzF1SPNvvi0Ew3gYfPZJcSbYiF8Hpe5TDZPO6ORkI3IDJoftW96A== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:VI1PR0302MB3407.eurprd03.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(376002)(396003)(366004)(39860400002)(136003)(346002)(66446008)(64756008)(478600001)(66946007)(66556008)(76116006)(66476007)(26005)(91956017)(36756003)(8676002)(86362001)(55236004)(186003)(8936002)(4326008)(6506007)(54906003)(6512007)(2906002)(6486002)(316002)(5660300002)(83380400001)(6916009)(71200400001)(1076003)(2616005); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: 0QYbbdXA4WK1wngTahjcoICVN1QSbFF3wx/kayfpEoemoIHHwaHVfVcDez3KBF/XxTOhCJFEtmK8N3IdQ5D0h+I24NIyV9BRMsnPD7QxhlYfOCdOvAuL9HZRCh8tvFM6PhVF5Y0iih2HLOD56GzMaUYzaWBDY6/q9WBlL7+xb5IORnGWb8PC8OV+tP1aZBGRTxsMVNoi2TUh8mnGG4zuzpnh5KvbTAFsFzbhXAyq/CPMbTkhYE4g47//gs2mjChYUurusFmXO0+pf4n/lEqVwRe36NEwgqbfKYvZzO4b3s8KuwHJUGcSeBWQS8yXalKrAC8V4jXoDZLJrxiEnJnUkWOaEAMzDlPzKjUuXQgqBRfpwgV4gg/9mA74CtvLWesmKv0P/hG9DUVtIBXKY27m5+F8hfK/A1us2bhKUTM6Una2yIBdgr/cYE6zCUYlerpOHXQy8fW+Cvr1j1A/+7Yjhn8T78ghHAUZt5P4siIZ2xk= Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: epam.com X-MS-Exchange-CrossTenant-Network-Message-Id: 23a7788d-bd81-4360-1a38-08d80e66b618 X-MS-Exchange-CrossTenant-originalarrivaltime: 12 Jun 2020 00:22:37.2004 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: oUl+MaUevMcx4U4o4FvHJXKlzBRK1w6bNGAYD9sp1vGyIupugNLr2PZ6qmJk6wLxf2bTn4MjtG6wDs+fg6i0ZQCf4QYOv4InTbK9VnQ5iKM= X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0302MB2654 X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Stefano Stabellini , Julien Grall , Wei Liu , Andrew Cooper , Ian Jackson , George Dunlap , Dario Faggioli , Jan Beulich , Volodymyr Babchuk Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-ZohoMail-DKIM: pass (identity @epam.com) Content-Type: text/plain; charset="utf-8" As scheduler code now collects time spent in IRQ handlers and in do_softirq(), we can present those values to userspace tools like xentop, so system administrator can see how system behaves. We are updating counters only in sched_get_time_correction() function to minimize number of taken spinlocks. As atomic_t is 32 bit wide, it is not enough to store time with nanosecond precision. So we need to use 64 bit variables and protect them with spinlock. Signed-off-by: Volodymyr Babchuk --- xen/common/sched/core.c | 17 +++++++++++++++++ xen/common/sysctl.c | 1 + xen/include/public/sysctl.h | 4 +++- xen/include/xen/sched.h | 2 ++ 4 files changed, 23 insertions(+), 1 deletion(-) diff --git a/xen/common/sched/core.c b/xen/common/sched/core.c index a7294ff5c3..ee6b1d9161 100644 --- a/xen/common/sched/core.c +++ b/xen/common/sched/core.c @@ -95,6 +95,10 @@ static struct scheduler __read_mostly ops; =20 static bool scheduler_active; =20 +static DEFINE_SPINLOCK(sched_stat_lock); +s_time_t sched_stat_irq_time; +s_time_t sched_stat_hyp_time; + static void sched_set_affinity( struct sched_unit *unit, const cpumask_t *hard, const cpumask_t *soft); =20 @@ -994,9 +998,22 @@ s_time_t sched_get_time_correction(struct sched_unit *= u) break; } =20 + spin_lock_irqsave(&sched_stat_lock, flags); + sched_stat_irq_time +=3D irq; + sched_stat_hyp_time +=3D hyp; + spin_unlock_irqrestore(&sched_stat_lock, flags); + return irq + hyp; } =20 +void sched_get_time_stats(uint64_t *irq_time, uint64_t *hyp_time) +{ + unsigned long flags; + + spin_lock_irqsave(&sched_stat_lock, flags); + *irq_time =3D sched_stat_irq_time; + *hyp_time =3D sched_stat_hyp_time; + spin_unlock_irqrestore(&sched_stat_lock, flags); } =20 /* diff --git a/xen/common/sysctl.c b/xen/common/sysctl.c index 1c6a817476..00683bc93f 100644 --- a/xen/common/sysctl.c +++ b/xen/common/sysctl.c @@ -270,6 +270,7 @@ long do_sysctl(XEN_GUEST_HANDLE_PARAM(xen_sysctl_t) u_s= ysctl) pi->scrub_pages =3D 0; pi->cpu_khz =3D cpu_khz; pi->max_mfn =3D get_upper_mfn_bound(); + sched_get_time_stats(&pi->irq_time, &pi->hyp_time); arch_do_physinfo(pi); if ( iommu_enabled ) { diff --git a/xen/include/public/sysctl.h b/xen/include/public/sysctl.h index 3a08c512e8..f320144d40 100644 --- a/xen/include/public/sysctl.h +++ b/xen/include/public/sysctl.h @@ -35,7 +35,7 @@ #include "domctl.h" #include "physdev.h" =20 -#define XEN_SYSCTL_INTERFACE_VERSION 0x00000013 +#define XEN_SYSCTL_INTERFACE_VERSION 0x00000014 =20 /* * Read console content from Xen buffer ring. @@ -118,6 +118,8 @@ struct xen_sysctl_physinfo { uint64_aligned_t scrub_pages; uint64_aligned_t outstanding_pages; uint64_aligned_t max_mfn; /* Largest possible MFN on this host */ + uint64_aligned_t irq_time; + uint64_aligned_t hyp_time; uint32_t hw_cap[8]; }; =20 diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h index 51dc7c4551..869d4efbd6 100644 --- a/xen/include/xen/sched.h +++ b/xen/include/xen/sched.h @@ -717,6 +717,8 @@ void vcpu_end_irq_handler(void); void vcpu_begin_hyp_task(struct vcpu *v); void vcpu_end_hyp_task(struct vcpu *v); =20 +void sched_get_time_stats(uint64_t *irq_time, uint64_t *hyp_time); + /* * Force synchronisation of given VCPU's state. If it is currently desched= uled, * this call will ensure that all its state is committed to memory and that --=20 2.27.0 From nobody Fri May 3 09:36:20 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; arc=pass (i=1dmarc=pass fromdomain=epam.com); dmarc=pass(p=quarantine dis=none) header.from=epam.com ARC-Seal: i=2; a=rsa-sha256; t=1591921409; cv=pass; d=zohomail.com; s=zohoarc; b=Ida34Gz/YSnjw7wVJMhwrNldn9Pc3suCY3MfSWImeY1zSESxKD3Voto4J0VF7zRHa6XbO0GnXRR7DAbMV362UXZ31vyDy0hn+i1kYeFm+JaS++POown+KCiN3QzdPwwfgIjrhT+UwFHMk1llNwIdu6FnSSNqPGoejj/scbyisDY= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1591921409; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=JFyM2tlxS7owCAmfGeZffNBhcSzmXAb+/nDqdHLXNkk=; b=LTv2Mr5su7neJo+wDjJKLKNa3iguppMf3GDMLpUvPHeB5ErM+pYhKZJWEteoMPAF2KNmv6URO6tBuyWh5yDhYeCrnoVKeyKU9MXqtODcNX/hDrkIeBKGCARKyIP0Vf2Su00bQkdKg2vn2gKfuCeojQDBuJI0G+WOOUM8eFD1TeM= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; arc=pass (i=1dmarc=pass fromdomain=epam.com); dmarc=pass header.from= (p=quarantine dis=none) header.from= Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1591921409271739.4636131499324; Thu, 11 Jun 2020 17:23:29 -0700 (PDT) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jjXTU-0007VO-LX; Fri, 12 Jun 2020 00:23:12 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jjXTT-0007MF-7V for xen-devel@lists.xenproject.org; Fri, 12 Jun 2020 00:23:11 +0000 Received: from EUR04-VI1-obe.outbound.protection.outlook.com (unknown [40.107.8.49]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id d4c65387-ac42-11ea-b594-12813bfff9fa; Fri, 12 Jun 2020 00:22:43 +0000 (UTC) Received: from VI1PR0302MB3407.eurprd03.prod.outlook.com (2603:10a6:803:23::18) by VI1PR0302MB2654.eurprd03.prod.outlook.com (2603:10a6:800:e0::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3088.22; Fri, 12 Jun 2020 00:22:38 +0000 Received: from VI1PR0302MB3407.eurprd03.prod.outlook.com ([fe80::a01e:c0b1:7174:4c75]) by VI1PR0302MB3407.eurprd03.prod.outlook.com ([fe80::a01e:c0b1:7174:4c75%5]) with mapi id 15.20.3088.018; Fri, 12 Jun 2020 00:22:38 +0000 X-Inumbo-ID: d4c65387-ac42-11ea-b594-12813bfff9fa ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=S1FRzIwqi8MaiPpQT0ukz+r83Mutf9qp8uoEL+e+gOb8FaYY5jZIsqJ1hyPwyNwn0mQCfz7AV8cCKNrg/tQ3lavNSYPbgn0g/yw0aG6Ww8CBY2Il6KU82KvDMwSemkPBqlWgLVOQotkjqNT7t2+m/nqwt0W5rCPwdv2JyEqeii4thZOW6D/iTJrQto2KSWCK4n06IzMNgFY+6NykQOeI54QWWU83EV6QqzFZAJGPM40N+lnHKofjrAwJrqRSoJrPSka/g/Von1fzZjuwIR6HKPMDmRaDCuZ50s34WbkDDlxps40x2TtLUvzBVwOxQOhVibXvrsS4H08LkzKD0Ydpfw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=JFyM2tlxS7owCAmfGeZffNBhcSzmXAb+/nDqdHLXNkk=; b=oaNEtKE5HJNls9cu54vmsDZRK7MKwPdSlk80tAUbUxgHF6wLbf+flJ98CICp34Web5aYB5C9WP49JLtiP8EFJmtrA5qvCyx2GhwSQRjYLhfUSvya1BrZVmFMvdJ/+nmcsdcn8DzRLm3O9rstH6xa2FmGaLwE5l/EQBCgUdqNSwxg7HRJuUkt6AxkbfOMHoPVIh6vnTbjEQw1Cmfg4FrifBKIIv3nBZqUf5fyRtIrR04NeENHhdCriL+gReDVOR7ghJRQl7oNeDK8kyw4S3NWaJf4wgskorzoHFz2jwT1jN4gjNfKqojt145PDjwgM4ramDfAsccFghSm64dmq48f4w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com; dkim=pass header.d=epam.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=JFyM2tlxS7owCAmfGeZffNBhcSzmXAb+/nDqdHLXNkk=; b=WzoUUqUw+unQHvd2hdNvqdrgF/zIn4qwUTzJ57vReEYJNnijL1FgGRQp29JYmD/MKcbS4x106pzbxQc+I6kN3rADJ73uP6LMns1PDKnBeNQA+lBc5HJuzdRdvD6M6KzrioCjTXAPXTQIvaT4E6cLffq0p+hZy6NDeawSA8JBg0CMcuP5dij8dBm0LlrJrwrgtwINi75bm0+nAfAFBTfhb8keo9fnmNuDZ33xYBCOVQddrmhWeGBu/i9TZcGRZlFwEgWK2dpMlb6OjeluoGFsGR2Y1DV6lQwZlWeCXJyZiH9YnDCSjWsDmYUYtv6079Ejul3Pga3CIphWa4yFmelbAw== From: Volodymyr Babchuk To: "xen-devel@lists.xenproject.org" Subject: [RFC PATCH v1 5/6] tools: xentop: show time spent in IRQ and HYP states. Thread-Topic: [RFC PATCH v1 5/6] tools: xentop: show time spent in IRQ and HYP states. Thread-Index: AQHWQE+TFzeBItaOmEyTKqcXTMXcTQ== Date: Fri, 12 Jun 2020 00:22:37 +0000 Message-ID: <20200612002205.174295-6-volodymyr_babchuk@epam.com> References: <20200612002205.174295-1-volodymyr_babchuk@epam.com> In-Reply-To: <20200612002205.174295-1-volodymyr_babchuk@epam.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: lists.xenproject.org; dkim=none (message not signed) header.d=none;lists.xenproject.org; dmarc=none action=none header.from=epam.com; x-originating-ip: [176.36.48.175] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 26c3444d-dc48-4175-4685-08d80e66b64a x-ms-traffictypediagnostic: VI1PR0302MB2654: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:1284; x-forefront-prvs: 0432A04947 x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 3dkx/YE7+OM4uMShWEMttf7KIAQGyTBPVYXYEeHruipcrBRvHQf9VqETR4nJBEaLvfkAvY2s7kzfc8qqQO5Bhv1J7h63hmeW3G5phOZQtLpnnYSffGyFgj9iWXcdsMyGk5Ulio6xVmVfvcXGgjFPbIi/EaQu1o/9SHwedL0DJ44sEfxP4YNiCjIH5ynU8RhPO9HEmsjCPfYDo5UXFtUkpQe/m+WK4mIlGlsyabk90lxsNgA9C0CQfivWaNvfbkgi1QpUiqyyicPKm1cKZ8vEJClhgwXx/6Q6k0DXFZbqoPAy4kJGB2OpvNne4sX5oaevqPt/FNmCIcCLQoSVArCwcw== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:VI1PR0302MB3407.eurprd03.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(376002)(396003)(366004)(39860400002)(136003)(346002)(66446008)(64756008)(478600001)(66946007)(66556008)(76116006)(66476007)(26005)(91956017)(36756003)(8676002)(86362001)(55236004)(186003)(8936002)(4326008)(6506007)(54906003)(6512007)(2906002)(6486002)(316002)(5660300002)(83380400001)(6916009)(71200400001)(1076003)(2616005); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: bi3L2Rd+Ypak3jw+nmFG5AZ45lMFpRn25AIGmlURXqVTKRIDpoP6nRhkIdXhMmGQnsfJhTQiS4vcH90gi7FBNng302CVEE9rkrPgGVhL0b/4d8nXOF5tor+YPsKaYNROyH5PslvM6parVWwFZwFUSgx/e7Fnu3ufkR7aUeMeNsMrfaiEluTpPW1YW/J0zO0HyBX2nNiJPvL8DoaqgyNalu5stGP7FrAF1Iq1hl0MLcoRP2venprQ/FkOo4q6d9yLBelV2zUeDa07Zb3dX8dSjMqtv9S5RwKZkVPXnD5IiN6fBCeCGrbaUylBZcT+4ky9TmtGNN9+aetnyVoeyxnS8osndmelFwbeYeq/dc1JSU5ufVrv3P2UiuiuMfnwLRR8RS92F6tojcju+/EF4yLc0h46/yUJsfHa7OMq0vwHeehaSSWU3PcKIqTr+e4eRzqcnVUG2SVi+yS3KBtrr5dk31Qz5f2kjh4/kHqsJoPrqB8= Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: epam.com X-MS-Exchange-CrossTenant-Network-Message-Id: 26c3444d-dc48-4175-4685-08d80e66b64a X-MS-Exchange-CrossTenant-originalarrivaltime: 12 Jun 2020 00:22:37.5722 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: kWMZzAL8vjfN53DwivgmgDStxodDGbD9NN3vgiL0qtApBd5BRL1V5okhZx7HL9H0BU4SO8MS/1XiL95tHACPk9He7KDZEjgst3kgkwausnU= X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0302MB2654 X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Ian Jackson , Volodymyr Babchuk , Wei Liu Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-ZohoMail-DKIM: pass (identity @epam.com) Content-Type: text/plain; charset="utf-8" xentop show the values in the header like this: IRQ Time 0.2s 0.0% HYP Time 1.3s 0.1% The first value is the total time spent in corresponding mode, the second value is the instant load percentage, similar to vCPU load value. "IRQ" corresponds to time spent in IRQ handler. "HYP" is the time used by hypervisor for own tasks. Signed-off-by: Volodymyr Babchuk --- tools/xenstat/libxenstat/src/xenstat.c | 12 +++++ tools/xenstat/libxenstat/src/xenstat.h | 6 +++ tools/xenstat/libxenstat/src/xenstat_priv.h | 2 + tools/xenstat/xentop/xentop.c | 54 ++++++++++++++++----- 4 files changed, 63 insertions(+), 11 deletions(-) diff --git a/tools/xenstat/libxenstat/src/xenstat.c b/tools/xenstat/libxens= tat/src/xenstat.c index 6f93d4e982..30c9d3d2cc 100644 --- a/tools/xenstat/libxenstat/src/xenstat.c +++ b/tools/xenstat/libxenstat/src/xenstat.c @@ -162,6 +162,8 @@ xenstat_node *xenstat_get_node(xenstat_handle * handle,= unsigned int flags) node->free_mem =3D ((unsigned long long)physinfo.free_pages) * handle->page_size; =20 + node->irq_time =3D physinfo.irq_time; + node->hyp_time =3D physinfo.hyp_time; node->freeable_mb =3D 0; /* malloc(0) is not portable, so allocate a single domain. This will * be resized below. */ @@ -332,6 +334,16 @@ unsigned long long xenstat_node_cpu_hz(xenstat_node * = node) return node->cpu_hz; } =20 +unsigned long long xenstat_node_irq_time(xenstat_node * node) +{ + return node->irq_time; +} + +unsigned long long xenstat_node_hyp_time(xenstat_node * node) +{ + return node->hyp_time; +} + /* Get the domain ID for this domain */ unsigned xenstat_domain_id(xenstat_domain * domain) { diff --git a/tools/xenstat/libxenstat/src/xenstat.h b/tools/xenstat/libxens= tat/src/xenstat.h index 76a660f321..8d2e561008 100644 --- a/tools/xenstat/libxenstat/src/xenstat.h +++ b/tools/xenstat/libxenstat/src/xenstat.h @@ -80,6 +80,12 @@ unsigned int xenstat_node_num_cpus(xenstat_node * node); /* Get information about the CPU speed */ unsigned long long xenstat_node_cpu_hz(xenstat_node * node); =20 +/* Get information about time spent in IRQ handlers */ +unsigned long long xenstat_node_irq_time(xenstat_node * node); + +/* Get information about time spent doing hypervisor work */ +unsigned long long xenstat_node_hyp_time(xenstat_node * node); + /* * Domain functions - extract information from a xenstat_domain */ diff --git a/tools/xenstat/libxenstat/src/xenstat_priv.h b/tools/xenstat/li= bxenstat/src/xenstat_priv.h index 4eb44a8ebb..d259765593 100644 --- a/tools/xenstat/libxenstat/src/xenstat_priv.h +++ b/tools/xenstat/libxenstat/src/xenstat_priv.h @@ -48,6 +48,8 @@ struct xenstat_node { unsigned long long tot_mem; unsigned long long free_mem; unsigned int num_domains; + unsigned long long irq_time; + unsigned long long hyp_time; xenstat_domain *domains; /* Array of length num_domains */ long freeable_mb; }; diff --git a/tools/xenstat/xentop/xentop.c b/tools/xenstat/xentop/xentop.c index ebed070c0f..aaeba81cd9 100644 --- a/tools/xenstat/xentop/xentop.c +++ b/tools/xenstat/xentop/xentop.c @@ -496,11 +496,25 @@ static void print_cpu(xenstat_domain *domain) print("%10llu", xenstat_domain_cpu_ns(domain)/1000000000); } =20 +/* Helper to calculate CPU load percentage */ +static double calc_time_pct(uint64_t cur_time_ns, uint64_t prev_time_ns) +{ + double us_elapsed; + + /* Calculate the time elapsed in microseconds */ + us_elapsed =3D ((curtime.tv_sec-oldtime.tv_sec)*1000000.0 + +(curtime.tv_usec - oldtime.tv_usec)); + + /* In the following, nanoseconds must be multiplied by 1000.0 to + * convert to microseconds, then divided by 100.0 to get a percentage, + * resulting in a multiplication by 10.0 */ + return ((cur_time_ns - prev_time_ns) / 10.0) / us_elapsed; +} + /* Computes the CPU percentage used for a specified domain */ static double get_cpu_pct(xenstat_domain *domain) { xenstat_domain *old_domain; - double us_elapsed; =20 /* Can't calculate CPU percentage without a previous sample. */ if(prev_node =3D=3D NULL) @@ -510,15 +524,8 @@ static double get_cpu_pct(xenstat_domain *domain) if(old_domain =3D=3D NULL) return 0.0; =20 - /* Calculate the time elapsed in microseconds */ - us_elapsed =3D ((curtime.tv_sec-oldtime.tv_sec)*1000000.0 - +(curtime.tv_usec - oldtime.tv_usec)); - - /* In the following, nanoseconds must be multiplied by 1000.0 to - * convert to microseconds, then divided by 100.0 to get a percentage, - * resulting in a multiplication by 10.0 */ - return ((xenstat_domain_cpu_ns(domain) - -xenstat_domain_cpu_ns(old_domain))/10.0)/us_elapsed; + return calc_time_pct(xenstat_domain_cpu_ns(domain), + xenstat_domain_cpu_ns(old_domain)); } =20 static int compare_cpu_pct(xenstat_domain *domain1, xenstat_domain *domain= 2) @@ -878,6 +885,23 @@ static void print_ssid(xenstat_domain *domain) print("%4u", xenstat_domain_ssid(domain)); } =20 +/* Computes the Xen time stats in percents */ +static void get_xen_time_stats(double *irq_pct, double *hyp_pct) +{ + /* Can't calculate CPU percentage without a previous sample. */ + if(prev_node =3D=3D NULL) + { + *irq_pct =3D 0.0; + *hyp_pct =3D 0.0; + return; + } + + *irq_pct =3D calc_time_pct(xenstat_node_irq_time(cur_node), + xenstat_node_irq_time(prev_node)); + *hyp_pct =3D calc_time_pct(xenstat_node_hyp_time(cur_node), + xenstat_node_hyp_time(prev_node)); +} + /* Resets default_width for fields with potentially large numbers */ void reset_field_widths(void) { @@ -943,6 +967,7 @@ void do_summary(void) crash =3D 0, dying =3D 0, shutdown =3D 0; unsigned i, num_domains =3D 0; unsigned long long used =3D 0; + double irq_pct, hyp_pct; xenstat_domain *domain; time_t curt; =20 @@ -975,9 +1000,16 @@ void do_summary(void) xenstat_node_tot_mem(cur_node)/1024, used/1024, xenstat_node_free_mem(cur_node)/1024); =20 - print("CPUs: %u @ %lluMHz\n", + print("CPUs: %u @ %lluMHz ", xenstat_node_num_cpus(cur_node), xenstat_node_cpu_hz(cur_node)/1000000); + + get_xen_time_stats(&irq_pct, &hyp_pct); + print("IRQ Time %.1fs %6.1f%% HYP Time %.1fs %6.1f%%\n", + xenstat_node_irq_time(cur_node) / 1000000000.0, + irq_pct, + xenstat_node_hyp_time(cur_node) / 1000000000.0, + hyp_pct); } =20 /* Display the top header for the domain table */ --=20 2.27.0 From nobody Fri May 3 09:36:20 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; arc=pass (i=1dmarc=pass fromdomain=epam.com); dmarc=pass(p=quarantine dis=none) header.from=epam.com ARC-Seal: i=2; a=rsa-sha256; t=1591921410; cv=pass; d=zohomail.com; s=zohoarc; b=cF5Gm9FWtBtS72H3/eBmVcv7vUf7btBI42tIUfyyBRVZ3e28EkiqCYIgWKVFzGkYk2p5cmrY8mZm6/EdqCPfJXgsMtP0OVRYrkCyMAfJRaa/8D73VvtFjZdr/agZO/3wnocsDfE9P8NcvfrHWgL4XYTprg42TEy/50EC9zDfJX8= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1591921410; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=OhQNqLAduYZF/1BUXnFYEz+JT7fMJefrT4kFPEI56d4=; b=dlu3ABL5lEA1WAGeE+nkQ6uoonBT2DwxyJLOGBTG18WFZBdzRtDv04QENi5KCS4wFDJgDZSRW7yKNoJtEzlClKVuqLP74UpJe6mDK/yo9CtmmCdani73xmw8xNkbnOubyvLE4qgJHdHyI823u1tt5hI05DRrB34vUCOCBaHSjs0= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; arc=pass (i=1dmarc=pass fromdomain=epam.com); dmarc=pass header.from= (p=quarantine dis=none) header.from= Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1591921409971865.4142877538301; Thu, 11 Jun 2020 17:23:29 -0700 (PDT) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jjXTP-0007TI-C5; Fri, 12 Jun 2020 00:23:07 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jjXTO-0007MF-7K for xen-devel@lists.xenproject.org; Fri, 12 Jun 2020 00:23:06 +0000 Received: from EUR04-VI1-obe.outbound.protection.outlook.com (unknown [40.107.8.89]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id d526ac36-ac42-11ea-b594-12813bfff9fa; Fri, 12 Jun 2020 00:22:43 +0000 (UTC) Received: from VI1PR0302MB3407.eurprd03.prod.outlook.com (2603:10a6:803:23::18) by VI1PR0302MB2654.eurprd03.prod.outlook.com (2603:10a6:800:e0::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3088.22; Fri, 12 Jun 2020 00:22:38 +0000 Received: from VI1PR0302MB3407.eurprd03.prod.outlook.com ([fe80::a01e:c0b1:7174:4c75]) by VI1PR0302MB3407.eurprd03.prod.outlook.com ([fe80::a01e:c0b1:7174:4c75%5]) with mapi id 15.20.3088.018; Fri, 12 Jun 2020 00:22:38 +0000 X-Inumbo-ID: d526ac36-ac42-11ea-b594-12813bfff9fa ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=TpU3T1x/ikoK34ceEilCbGfH4gsXlZpp6QNPIV3SE9eWByAb1I34vf17B/AhSFSSXA5OM0tBb+vReMd+zQh8tPPsugyXfHljQM7CIpm5RCBXMcRRyRSBTq+ret28CB8NdaJaNgu28ebTkBs3kNHSkQUlH2S5cKLfAK+AUZuiDpuibBwRGhADHf4gFmNcJKaNcNt5whe4Ah1L3CRWbEy5yvcoELkmFOt7MTiLyNd1qb/yoLyDG4D2TA3mZD0c1bx6z1rmm8upnoLKt2xQlcGk7WrgYaSFzfKjY5bkCziRA8lr6XrhuG4d0QXAh+I7MaTuxt7oNmsUXJUAfIOhP80nMQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=OhQNqLAduYZF/1BUXnFYEz+JT7fMJefrT4kFPEI56d4=; b=E07xBZTxSZ/4SLqRSjiFYywimdCXBo/5kXME1UiL1cBfF/ah3uwYwdCjQRJg4a8nk86va1bfG6SFE9W6DjRey7Np2EiZgzv84iKgfCziPpGFhZzxTufnBXU/5wb+Twd6FYaN5laYc7/ONF9e29PYRTawfdkiTGcJGEZVdS61kduD/a45aeV0PFL0uxb8K0fi6tzbLIlBExTocrIUbgqMjAALxBCQ24gEEO3bPay1C21EajCMkYznTL/QRTCdshkoaNytKYGbPL90pciEF3vtZuqtUf7iplmVcX0KO0CkAkUrzfwnkGuVZm6mv9z24R6lPvPznk5Uf3x3lXFphJ+03w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com; dkim=pass header.d=epam.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=OhQNqLAduYZF/1BUXnFYEz+JT7fMJefrT4kFPEI56d4=; b=TjkF86XsEXus/Da+pkq5ScryeKBHnJrCS0a+tTYn9iU139MnP6k68xDpJKjoNQR9YhiGSKEPUJzrvTvLsnHjVVdVp9jAKsCZKuYYMBSqQdDGJRVa3ZyYyW2jiRcKFrz4EQounuEpWSNK3ofelas07HyWcDz3dTHw5AhgtOr0SoLw3ER9kJY0w5TMWUxHV8fR9KK9SeDRB7i8znvLw2Eui6aXb5DxH/oHfVMrWIoqPm0UHsXynstBr9xWvvk/kp2LTgiFVwP5Vigvoz/5yMB5Js9LxUkGSr14s6iUH9Rkqo7tlm1OCFoxoffQYBnoJVvBlDyl3zEgdalX0xRm/M5QCg== From: Volodymyr Babchuk To: "xen-devel@lists.xenproject.org" Subject: [RFC PATCH v1 6/6] trace: add fair scheduling trace events Thread-Topic: [RFC PATCH v1 6/6] trace: add fair scheduling trace events Thread-Index: AQHWQE+TrxaSoGuic062xf5W0YJ0vg== Date: Fri, 12 Jun 2020 00:22:37 +0000 Message-ID: <20200612002205.174295-7-volodymyr_babchuk@epam.com> References: <20200612002205.174295-1-volodymyr_babchuk@epam.com> In-Reply-To: <20200612002205.174295-1-volodymyr_babchuk@epam.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: lists.xenproject.org; dkim=none (message not signed) header.d=none;lists.xenproject.org; dmarc=none action=none header.from=epam.com; x-originating-ip: [176.36.48.175] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 7643c403-423d-4a9a-72d3-08d80e66b67c x-ms-traffictypediagnostic: VI1PR0302MB2654: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:50; x-forefront-prvs: 0432A04947 x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: X2SG3eldEkEDFpLCJXcdAFMMjpk8lZHC+Za0YD37Wt+ErFwmTpX9pLIAKh/KsLfW0woEFoFlTFltG+deHmOGVhvr0LAuvzv0ILCxksln0Kt76E1auC4iJHoYewnK5+l8DaIka9r/rIyo1xgjBoHPsYf56mrPK5c2Eo9phAuGtYQYyakJGLlDIkmnD/6dJGV9bnH0m/PboLRmZa6sJgDjbVpLPy09Xe+7cZ8L+Yc9mKoeaLWfBV14Uv11dkrLlDMug1ZZmUKhTgWhyJp/fDSlnq2QoskJWZlW1yTwFdIxiCYMqXoCXE2Zr7iTH1s5IMOnuvmmyDQ1/vj49DRglUU0cslJbiEmLAV9MoeOAqMSE4o4WGGOg8+owic2u8fWJ3UU x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:VI1PR0302MB3407.eurprd03.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(376002)(396003)(366004)(39860400002)(136003)(346002)(66446008)(64756008)(478600001)(66946007)(66556008)(76116006)(66476007)(26005)(91956017)(36756003)(8676002)(86362001)(55236004)(186003)(8936002)(4326008)(6506007)(54906003)(6512007)(2906002)(6486002)(316002)(5660300002)(83380400001)(6916009)(71200400001)(1076003)(2616005)(21314003); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: X276hooWpkKRPXUz48Ksn0UX+Cch9MGZxn89XRWtxFloRBUQKzrKJKgCnj/+TXfvs2c2SoklIrCrDeZE6RZ763ngaHZRPCIa6xeKVFftM8SmnY5CwOGJ6jpLLmhLHId0zrWad4arUmSgurvZg+evmdF8XspqgYWyaLeJByPGD2wV1c2RsHk0g27m/6N56rmRq3NmIGT5g2UmEAEnFtInZVGS9RgjblugkoRFGoDxriAAr/t2BB7/oEG6gkvPL5qZ7b+UBDUSvO7c/cne00Ah248ZSeyj+3YAgfyICAlUwMpSLXUl++XYqfDHXLuapJ8HWtd4KOz9mujkjSRyBnaM0AvECTmF95KGBNO+b+pKK8LIquh2pRekVvjkD1lWn5b2KnK1zHo8Dv7DNtBAFnRYoo/IXMWERA6tZBgsh0M9kIYWL7cKkPeiMpfj33dwuTdC9HfFQxOP+W/kMZk5nrWWvhF6ESYynPpHsT0tCp3d0GY= Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: epam.com X-MS-Exchange-CrossTenant-Network-Message-Id: 7643c403-423d-4a9a-72d3-08d80e66b67c X-MS-Exchange-CrossTenant-originalarrivaltime: 12 Jun 2020 00:22:37.9310 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: urOv9Ezf93ttC7W+llqH8ci9+LtMuiO/8/StYjUvaPWgPcHbA7HVp2tsU+5CqLjNzh+FcPI91eYfTgwDUe8v1GLH5amJpkloQcTV3mIZIgc= X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0302MB2654 X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Stefano Stabellini , Julien Grall , Wei Liu , Andrew Cooper , Ian Jackson , George Dunlap , Dario Faggioli , Jan Beulich , Volodymyr Babchuk Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-ZohoMail-DKIM: pass (identity @epam.com) Content-Type: text/plain; charset="utf-8" We are tracing each IRQ or HYP mode change and the calculated time adjustment values. Signed-off-by: Volodymyr Babchuk --- tools/xentrace/xenalyze.c | 37 +++++++++++++++++++++++++++++++++++++ xen/common/sched/core.c | 9 +++++++++ xen/include/public/trace.h | 5 +++++ 3 files changed, 51 insertions(+) diff --git a/tools/xentrace/xenalyze.c b/tools/xentrace/xenalyze.c index b7f4e2bea8..bcde830f0e 100644 --- a/tools/xentrace/xenalyze.c +++ b/tools/xentrace/xenalyze.c @@ -7546,6 +7546,43 @@ void sched_process(struct pcpu_info *p) printf("\n"); } break; + case TRC_SCHED_IRQ_ENTRY: + case TRC_SCHED_IRQ_LEAVE: + if(opt.dump_all) + { + struct { + unsigned int nesting; + } *r =3D (typeof(r))ri->d; + + printf(" %s sched_irq_%s nesting =3D %u\n", ri->dump_heade= r, + ri->event =3D=3D TRC_SCHED_IRQ_ENTRY ? "entry":"lea= ve", + r->nesting); + } + break; + case TRC_SCHED_HYP_ENTRY: + case TRC_SCHED_HYP_LEAVE: + if(opt.dump_all) + { + struct { + unsigned int domid, vcpuid; + } *r =3D (typeof(r))ri->d; + + printf(" %s sched_hyp_%s d%uv%u\n", ri->dump_header, + ri->event =3D=3D TRC_SCHED_HYP_ENTRY ? "entry":"lea= ve", + r->domid, r->vcpuid); + } + break; + case TRC_SCHED_TIME_ADJ: + if(opt.dump_all) + { + struct { + unsigned int irq, hyp; + } *r =3D (typeof(r))ri->d; + + printf(" %s sched time adjust IRQ %uns HYP %uns Total %uns= \n", ri->dump_header, + r->irq, r->hyp, r->irq + r->hyp); + } + break; case TRC_SCHED_CTL: case TRC_SCHED_S_TIMER_FN: case TRC_SCHED_T_TIMER_FN: diff --git a/xen/common/sched/core.c b/xen/common/sched/core.c index ee6b1d9161..9e82a6a22b 100644 --- a/xen/common/sched/core.c +++ b/xen/common/sched/core.c @@ -925,6 +925,8 @@ void vcpu_begin_irq_handler(void) if (is_idle_vcpu(current)) return; =20 + TRACE_1D(TRC_SCHED_IRQ_ENTRY, current->irq_nesting); + /* XXX: Looks like ASSERT_INTERRUPTS_DISABLED() is available only for = x86 */ if ( current->irq_nesting++ ) return; @@ -941,6 +943,8 @@ void vcpu_end_irq_handler(void) =20 ASSERT(current->irq_nesting); =20 + TRACE_1D(TRC_SCHED_IRQ_LEAVE, current->irq_nesting - 1); + if ( --current->irq_nesting ) return; =20 @@ -960,6 +964,7 @@ void vcpu_begin_hyp_task(struct vcpu *v) #ifndef NDEBUG v->in_hyp_task =3D true; #endif + TRACE_2D(TRC_SCHED_HYP_ENTRY, v->domain->domain_id, v->vcpu_id); } =20 void vcpu_end_hyp_task(struct vcpu *v) @@ -978,6 +983,8 @@ void vcpu_end_hyp_task(struct vcpu *v) #ifndef NDEBUG v->in_hyp_task =3D false; #endif + TRACE_2D(TRC_SCHED_HYP_LEAVE, v->domain->domain_id, v->vcpu_id); +} =20 s_time_t sched_get_time_correction(struct sched_unit *u) { @@ -1003,6 +1010,8 @@ s_time_t sched_get_time_correction(struct sched_unit = *u) sched_stat_hyp_time +=3D hyp; spin_unlock_irqrestore(&sched_stat_lock, flags); =20 + TRACE_2D(TRC_SCHED_TIME_ADJ, irq, hyp); + return irq + hyp; } =20 diff --git a/xen/include/public/trace.h b/xen/include/public/trace.h index d5fa4aea8d..6161980095 100644 --- a/xen/include/public/trace.h +++ b/xen/include/public/trace.h @@ -117,6 +117,11 @@ #define TRC_SCHED_SWITCH_INFNEXT (TRC_SCHED_VERBOSE + 15) #define TRC_SCHED_SHUTDOWN_CODE (TRC_SCHED_VERBOSE + 16) #define TRC_SCHED_SWITCH_INFCONT (TRC_SCHED_VERBOSE + 17) +#define TRC_SCHED_IRQ_ENTRY (TRC_SCHED_VERBOSE + 18) +#define TRC_SCHED_IRQ_LEAVE (TRC_SCHED_VERBOSE + 19) +#define TRC_SCHED_HYP_ENTRY (TRC_SCHED_VERBOSE + 20) +#define TRC_SCHED_HYP_LEAVE (TRC_SCHED_VERBOSE + 21) +#define TRC_SCHED_TIME_ADJ (TRC_SCHED_VERBOSE + 22) =20 #define TRC_DOM0_DOM_ADD (TRC_DOM0_DOMOPS + 1) #define TRC_DOM0_DOM_REM (TRC_DOM0_DOMOPS + 2) --=20 2.27.0