From nobody Mon Feb 9 09:34:58 2026 Delivered-To: importer@patchew.org Received-SPF: none (zoho.com: 192.237.175.120 is neither permitted nor denied by domain of lists.xenproject.org) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; spf=none (zoho.com: 192.237.175.120 is neither permitted nor denied by domain of lists.xenproject.org) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org ARC-Seal: i=1; a=rsa-sha256; t=1557125905; cv=none; d=zoho.com; s=zohoarc; b=J1KdHRNRecukx3J54JsRIFaXLOSFbqp5PyLcudqQC6ZEOFiWbGkxz2W2hazUl38W6uAn7MntF0r8amHtg5BApJPBQl014f+b4o5PK5r6cbVJhC5zV2L1RAgcr5RS3KYhqzKQMnT1BY/y6W/aYH8te/dT3TDr60B/wvYCybejdCc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1557125905; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=aCNUI+P4NyYTjn06gAaHJI5xMmuzyqk8WeMfBQLoCY4=; b=g94QWX1sG80AZ0i7+QCup3stDizhpbCNg+JqRuNiW7PLRY6twahibj5lWV0y2TPkhKB+XKnkxoCwu4JC1UNEdVlF6TsOuG2QjGWiqdl9eI/8Lnqj/WrlOw958Haha742d6hPEEZOC+xlK+ngBRWucT2kujXPsxRuDDC+/HV7BPo= ARC-Authentication-Results: i=1; mx.zoho.com; spf=none (zoho.com: 192.237.175.120 is neither permitted nor denied by domain of lists.xenproject.org) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1557125905845544.1715647605505; Sun, 5 May 2019 23:58:25 -0700 (PDT) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hNXYj-0002Al-2z; Mon, 06 May 2019 06:57:09 +0000 Received: from us1-rack-dfw2.inumbo.com ([104.130.134.6]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hNXYf-00022t-ES for xen-devel@lists.xenproject.org; Mon, 06 May 2019 06:57:05 +0000 Received: from mx1.suse.de (unknown [195.135.220.15]) by us1-rack-dfw2.inumbo.com (Halon) with ESMTPS id 2508648d-6fcc-11e9-843c-bc764e045a96; Mon, 06 May 2019 06:57:00 +0000 (UTC) Received: from relay2.suse.de (unknown [195.135.220.254]) by mx1.suse.de (Postfix) with ESMTP id 9731FAF26; Mon, 6 May 2019 06:56:55 +0000 (UTC) X-Inumbo-ID: 2508648d-6fcc-11e9-843c-bc764e045a96 X-Virus-Scanned: by amavisd-new at test-mx.suse.de From: Juergen Gross To: xen-devel@lists.xenproject.org Date: Mon, 6 May 2019 08:56:24 +0200 Message-Id: <20190506065644.7415-26-jgross@suse.com> X-Mailer: git-send-email 2.16.4 In-Reply-To: <20190506065644.7415-1-jgross@suse.com> References: <20190506065644.7415-1-jgross@suse.com> Subject: [Xen-devel] [PATCH RFC V2 25/45] xen/sched: use sched_resource cpu instead smp_processor_id in schedulers X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Juergen Gross , Tim Deegan , Stefano Stabellini , Wei Liu , Konrad Rzeszutek Wilk , George Dunlap , Andrew Cooper , Ian Jackson , Robert VanVossen , Dario Faggioli , Julien Grall , Josh Whitehead , Meng Xu , Jan Beulich MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" Especially in the do_schedule() functions of the different schedulers using smp_processor_id() for the local cpu number is correct only if the sched_item is a single vcpu. As soon as larger sched_items are used most uses should be replaced by the cpu number of the local sched_resource instead. Add a helper to get that sched_resource cpu and modify the schedulers to use it in a correct way. Signed-off-by: Juergen Gross --- xen/common/sched_arinc653.c | 2 +- xen/common/sched_credit.c | 19 ++++++++-------- xen/common/sched_credit2.c | 53 +++++++++++++++++++++++------------------= ---- xen/common/sched_null.c | 17 ++++++++------- xen/common/sched_rt.c | 17 ++++++++------- xen/include/xen/sched-if.h | 5 +++++ 6 files changed, 61 insertions(+), 52 deletions(-) diff --git a/xen/common/sched_arinc653.c b/xen/common/sched_arinc653.c index 61f9ea6824..3919c0a3e9 100644 --- a/xen/common/sched_arinc653.c +++ b/xen/common/sched_arinc653.c @@ -513,7 +513,7 @@ a653sched_do_schedule( static unsigned int sched_index =3D 0; static s_time_t next_switch_time; a653sched_priv_t *sched_priv =3D SCHED_PRIV(ops); - const unsigned int cpu =3D smp_processor_id(); + const unsigned int cpu =3D sched_get_resource_cpu(smp_processor_id()); unsigned long flags; =20 spin_lock_irqsave(&sched_priv->lock, flags); diff --git a/xen/common/sched_credit.c b/xen/common/sched_credit.c index 9db5c3fc71..4734f52fc7 100644 --- a/xen/common/sched_credit.c +++ b/xen/common/sched_credit.c @@ -1839,8 +1839,9 @@ static struct task_slice csched_schedule( const struct scheduler *ops, s_time_t now, bool_t tasklet_work_schedul= ed) { - const int cpu =3D smp_processor_id(); - struct list_head * const runq =3D RUNQ(cpu); + const unsigned int cpu =3D smp_processor_id(); + const unsigned int sched_cpu =3D sched_get_resource_cpu(cpu); + struct list_head * const runq =3D RUNQ(sched_cpu); struct sched_item *item =3D current->sched_item; struct csched_item * const scurr =3D CSCHED_ITEM(item); struct csched_private *prv =3D CSCHED_PRIV(ops); @@ -1950,7 +1951,7 @@ csched_schedule( { BUG_ON( is_idle_item(item) || list_empty(runq) ); /* Current has blocked. Update the runnable counter for this cpu. = */ - dec_nr_runnable(cpu); + dec_nr_runnable(sched_cpu); } =20 snext =3D __runq_elem(runq->next); @@ -1960,7 +1961,7 @@ csched_schedule( if ( tasklet_work_scheduled ) { TRACE_0D(TRC_CSCHED_SCHED_TASKLET); - snext =3D CSCHED_ITEM(sched_idle_item(cpu)); + snext =3D CSCHED_ITEM(sched_idle_item(sched_cpu)); snext->pri =3D CSCHED_PRI_TS_BOOST; } =20 @@ -1980,7 +1981,7 @@ csched_schedule( if ( snext->pri > CSCHED_PRI_TS_OVER ) __runq_remove(snext); else - snext =3D csched_load_balance(prv, cpu, snext, &ret.migrated); + snext =3D csched_load_balance(prv, sched_cpu, snext, &ret.migrated= ); =20 /* * Update idlers mask if necessary. When we're idling, other CPUs @@ -1988,12 +1989,12 @@ csched_schedule( */ if ( !tasklet_work_scheduled && snext->pri =3D=3D CSCHED_PRI_IDLE ) { - if ( !cpumask_test_cpu(cpu, prv->idlers) ) - cpumask_set_cpu(cpu, prv->idlers); + if ( !cpumask_test_cpu(sched_cpu, prv->idlers) ) + cpumask_set_cpu(sched_cpu, prv->idlers); } - else if ( cpumask_test_cpu(cpu, prv->idlers) ) + else if ( cpumask_test_cpu(sched_cpu, prv->idlers) ) { - cpumask_clear_cpu(cpu, prv->idlers); + cpumask_clear_cpu(sched_cpu, prv->idlers); } =20 if ( !is_idle_item(snext->item) ) diff --git a/xen/common/sched_credit2.c b/xen/common/sched_credit2.c index 7918d46a23..d5cb8c0200 100644 --- a/xen/common/sched_credit2.c +++ b/xen/common/sched_credit2.c @@ -3447,7 +3447,8 @@ static struct task_slice csched2_schedule( const struct scheduler *ops, s_time_t now, bool tasklet_work_scheduled) { - const int cpu =3D smp_processor_id(); + const unsigned int cpu =3D smp_processor_id(); + const unsigned int sched_cpu =3D sched_get_resource_cpu(cpu); struct csched2_runqueue_data *rqd; struct sched_item *curritem =3D current->sched_item; struct csched2_item * const scurr =3D csched2_item(curritem); @@ -3459,22 +3460,22 @@ csched2_schedule( SCHED_STAT_CRANK(schedule); CSCHED2_ITEM_CHECK(curritem); =20 - BUG_ON(!cpumask_test_cpu(cpu, &csched2_priv(ops)->initialized)); + BUG_ON(!cpumask_test_cpu(sched_cpu, &csched2_priv(ops)->initialized)); =20 - rqd =3D c2rqd(ops, cpu); - BUG_ON(!cpumask_test_cpu(cpu, &rqd->active)); + rqd =3D c2rqd(ops, sched_cpu); + BUG_ON(!cpumask_test_cpu(sched_cpu, &rqd->active)); =20 - ASSERT(spin_is_locked(per_cpu(sched_res, cpu)->schedule_lock)); + ASSERT(spin_is_locked(per_cpu(sched_res, sched_cpu)->schedule_lock)); =20 BUG_ON(!is_idle_item(curritem) && scurr->rqd !=3D rqd); =20 /* Clear "tickled" bit now that we've been scheduled */ - tickled =3D cpumask_test_cpu(cpu, &rqd->tickled); + tickled =3D cpumask_test_cpu(sched_cpu, &rqd->tickled); if ( tickled ) { - __cpumask_clear_cpu(cpu, &rqd->tickled); + __cpumask_clear_cpu(sched_cpu, &rqd->tickled); cpumask_andnot(cpumask_scratch, &rqd->idle, &rqd->tickled); - smt_idle_mask_set(cpu, cpumask_scratch, &rqd->smt_idle); + smt_idle_mask_set(sched_cpu, cpumask_scratch, &rqd->smt_idle); } =20 if ( unlikely(tb_init_done) ) @@ -3484,10 +3485,10 @@ csched2_schedule( unsigned tasklet:8, idle:8, smt_idle:8, tickled:8; } d; d.cpu =3D cpu; - d.rq_id =3D c2r(cpu); + d.rq_id =3D c2r(sched_cpu); d.tasklet =3D tasklet_work_scheduled; d.idle =3D is_idle_item(curritem); - d.smt_idle =3D cpumask_test_cpu(cpu, &rqd->smt_idle); + d.smt_idle =3D cpumask_test_cpu(sched_cpu, &rqd->smt_idle); d.tickled =3D tickled; __trace_var(TRC_CSCHED2_SCHEDULE, 1, sizeof(d), @@ -3527,10 +3528,10 @@ csched2_schedule( { __clear_bit(__CSFLAG_item_yield, &scurr->flags); trace_var(TRC_CSCHED2_SCHED_TASKLET, 1, 0, NULL); - snext =3D csched2_item(sched_idle_item(cpu)); + snext =3D csched2_item(sched_idle_item(sched_cpu)); } else - snext =3D runq_candidate(rqd, scurr, cpu, now, &skipped_items); + snext =3D runq_candidate(rqd, scurr, sched_cpu, now, &skipped_item= s); =20 /* If switching from a non-idle runnable item, put it * back on the runqueue. */ @@ -3555,10 +3556,10 @@ csched2_schedule( } =20 /* Clear the idle mask if necessary */ - if ( cpumask_test_cpu(cpu, &rqd->idle) ) + if ( cpumask_test_cpu(sched_cpu, &rqd->idle) ) { - __cpumask_clear_cpu(cpu, &rqd->idle); - smt_idle_mask_clear(cpu, &rqd->smt_idle); + __cpumask_clear_cpu(sched_cpu, &rqd->idle); + smt_idle_mask_clear(sched_cpu, &rqd->smt_idle); } =20 /* @@ -3577,18 +3578,18 @@ csched2_schedule( */ if ( skipped_items =3D=3D 0 && snext->credit <=3D CSCHED2_CREDIT_R= ESET ) { - reset_credit(ops, cpu, now, snext); - balance_load(ops, cpu, now); + reset_credit(ops, sched_cpu, now, snext); + balance_load(ops, sched_cpu, now); } =20 snext->start_time =3D now; snext->tickled_cpu =3D -1; =20 /* Safe because lock for old processor is held */ - if ( sched_item_cpu(snext->item) !=3D cpu ) + if ( sched_item_cpu(snext->item) !=3D sched_cpu ) { snext->credit +=3D CSCHED2_MIGRATE_COMPENSATION; - sched_set_res(snext->item, per_cpu(sched_res, cpu)); + sched_set_res(snext->item, per_cpu(sched_res, sched_cpu)); SCHED_STAT_CRANK(migrated); ret.migrated =3D 1; } @@ -3601,17 +3602,17 @@ csched2_schedule( */ if ( tasklet_work_scheduled ) { - if ( cpumask_test_cpu(cpu, &rqd->idle) ) + if ( cpumask_test_cpu(sched_cpu, &rqd->idle) ) { - __cpumask_clear_cpu(cpu, &rqd->idle); - smt_idle_mask_clear(cpu, &rqd->smt_idle); + __cpumask_clear_cpu(sched_cpu, &rqd->idle); + smt_idle_mask_clear(sched_cpu, &rqd->smt_idle); } } - else if ( !cpumask_test_cpu(cpu, &rqd->idle) ) + else if ( !cpumask_test_cpu(sched_cpu, &rqd->idle) ) { - __cpumask_set_cpu(cpu, &rqd->idle); + __cpumask_set_cpu(sched_cpu, &rqd->idle); cpumask_andnot(cpumask_scratch, &rqd->idle, &rqd->tickled); - smt_idle_mask_set(cpu, cpumask_scratch, &rqd->smt_idle); + smt_idle_mask_set(sched_cpu, cpumask_scratch, &rqd->smt_idle); } /* Make sure avgload gets updated periodically even * if there's no activity */ @@ -3621,7 +3622,7 @@ csched2_schedule( /* * Return task to run next... */ - ret.time =3D csched2_runtime(ops, cpu, snext, now); + ret.time =3D csched2_runtime(ops, sched_cpu, snext, now); ret.task =3D snext->item; =20 CSCHED2_ITEM_CHECK(ret.task); diff --git a/xen/common/sched_null.c b/xen/common/sched_null.c index ceb026c8af..34ce7a05d3 100644 --- a/xen/common/sched_null.c +++ b/xen/common/sched_null.c @@ -709,6 +709,7 @@ static struct task_slice null_schedule(const struct sch= eduler *ops, { unsigned int bs; const unsigned int cpu =3D smp_processor_id(); + const unsigned int sched_cpu =3D sched_get_resource_cpu(cpu); struct null_private *prv =3D null_priv(ops); struct null_item *wvc; struct task_slice ret; @@ -724,14 +725,14 @@ static struct task_slice null_schedule(const struct s= cheduler *ops, } d; d.cpu =3D cpu; d.tasklet =3D tasklet_work_scheduled; - if ( per_cpu(npc, cpu).item =3D=3D NULL ) + if ( per_cpu(npc, sched_cpu).item =3D=3D NULL ) { d.item =3D d.dom =3D -1; } else { - d.item =3D per_cpu(npc, cpu).item->item_id; - d.dom =3D per_cpu(npc, cpu).item->domain->domain_id; + d.item =3D per_cpu(npc, sched_cpu).item->item_id; + d.dom =3D per_cpu(npc, sched_cpu).item->domain->domain_id; } __trace_var(TRC_SNULL_SCHEDULE, 1, sizeof(d), &d); } @@ -739,10 +740,10 @@ static struct task_slice null_schedule(const struct s= cheduler *ops, if ( tasklet_work_scheduled ) { trace_var(TRC_SNULL_TASKLET, 1, 0, NULL); - ret.task =3D sched_idle_item(cpu); + ret.task =3D sched_idle_item(sched_cpu); } else - ret.task =3D per_cpu(npc, cpu).item; + ret.task =3D per_cpu(npc, sched_cpu).item; ret.migrated =3D 0; ret.time =3D -1; =20 @@ -773,9 +774,9 @@ static struct task_slice null_schedule(const struct sch= eduler *ops, !has_soft_affinity(wvc->item) ) continue; =20 - if ( item_check_affinity(wvc->item, cpu, bs) ) + if ( item_check_affinity(wvc->item, sched_cpu, bs) ) { - item_assign(prv, wvc->item, cpu); + item_assign(prv, wvc->item, sched_cpu); list_del_init(&wvc->waitq_elem); ret.task =3D wvc->item; goto unlock; @@ -787,7 +788,7 @@ static struct task_slice null_schedule(const struct sch= eduler *ops, } =20 if ( unlikely(ret.task =3D=3D NULL || !item_runnable(ret.task)) ) - ret.task =3D sched_idle_item(cpu); + ret.task =3D sched_idle_item(sched_cpu); =20 NULL_ITEM_CHECK(ret.task); return ret; diff --git a/xen/common/sched_rt.c b/xen/common/sched_rt.c index 730aa292d4..2366e33beb 100644 --- a/xen/common/sched_rt.c +++ b/xen/common/sched_rt.c @@ -1065,7 +1065,8 @@ runq_pick(const struct scheduler *ops, const cpumask_= t *mask) static struct task_slice rt_schedule(const struct scheduler *ops, s_time_t now, bool_t tasklet_work= _scheduled) { - const int cpu =3D smp_processor_id(); + const unsigned int cpu =3D smp_processor_id(); + const unsigned int sched_cpu =3D sched_get_resource_cpu(cpu); struct rt_private *prv =3D rt_priv(ops); struct rt_item *const scurr =3D rt_item(current->sched_item); struct rt_item *snext =3D NULL; @@ -1079,7 +1080,7 @@ rt_schedule(const struct scheduler *ops, s_time_t now= , bool_t tasklet_work_sched } d; d.cpu =3D cpu; d.tasklet =3D tasklet_work_scheduled; - d.tickled =3D cpumask_test_cpu(cpu, &prv->tickled); + d.tickled =3D cpumask_test_cpu(sched_cpu, &prv->tickled); d.idle =3D is_idle_item(curritem); trace_var(TRC_RTDS_SCHEDULE, 1, sizeof(d), @@ -1087,7 +1088,7 @@ rt_schedule(const struct scheduler *ops, s_time_t now= , bool_t tasklet_work_sched } =20 /* clear ticked bit now that we've been scheduled */ - cpumask_clear_cpu(cpu, &prv->tickled); + cpumask_clear_cpu(sched_cpu, &prv->tickled); =20 /* burn_budget would return for IDLE ITEM */ burn_budget(ops, scurr, now); @@ -1095,13 +1096,13 @@ rt_schedule(const struct scheduler *ops, s_time_t n= ow, bool_t tasklet_work_sched if ( tasklet_work_scheduled ) { trace_var(TRC_RTDS_SCHED_TASKLET, 1, 0, NULL); - snext =3D rt_item(sched_idle_item(cpu)); + snext =3D rt_item(sched_idle_item(sched_cpu)); } else { - snext =3D runq_pick(ops, cpumask_of(cpu)); + snext =3D runq_pick(ops, cpumask_of(sched_cpu)); if ( snext =3D=3D NULL ) - snext =3D rt_item(sched_idle_item(cpu)); + snext =3D rt_item(sched_idle_item(sched_cpu)); =20 /* if scurr has higher priority and budget, still pick scurr */ if ( !is_idle_item(curritem) && @@ -1126,9 +1127,9 @@ rt_schedule(const struct scheduler *ops, s_time_t now= , bool_t tasklet_work_sched q_remove(snext); __set_bit(__RTDS_scheduled, &snext->flags); } - if ( sched_item_cpu(snext->item) !=3D cpu ) + if ( sched_item_cpu(snext->item) !=3D sched_cpu ) { - sched_set_res(snext->item, per_cpu(sched_res, cpu)); + sched_set_res(snext->item, per_cpu(sched_res, sched_cpu)); ret.migrated =3D 1; } ret.time =3D snext->cur_budget; /* invoke the scheduler next time = */ diff --git a/xen/include/xen/sched-if.h b/xen/include/xen/sched-if.h index 65e2c3bba8..2506538649 100644 --- a/xen/include/xen/sched-if.h +++ b/xen/include/xen/sched-if.h @@ -100,6 +100,11 @@ static inline struct sched_item *sched_idle_item(unsig= ned int cpu) return idle_vcpu[cpu]->sched_item; } =20 +static inline unsigned int sched_get_resource_cpu(unsigned int cpu) +{ + return per_cpu(sched_res, cpu)->processor; +} + /* * Scratch space, for avoiding having too many cpumask_t on the stack. * Within each scheduler, when using the scratch mask of one pCPU: --=20 2.16.4 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel