From nobody Tue Feb 10 02:49:06 2026 Delivered-To: importer@patchew.org Received-SPF: none (zoho.com: 192.237.175.120 is neither permitted nor denied by domain of lists.xenproject.org) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; spf=none (zoho.com: 192.237.175.120 is neither permitted nor denied by domain of lists.xenproject.org) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org ARC-Seal: i=1; a=rsa-sha256; t=1568451356; cv=none; d=zoho.com; s=zohoarc; b=jYY199HcNfwzZZJ7tMsyIvjaGwO/4sHvjhR35KFC89lnEvmnesECyxyTaNvGIrMvYGgVKJvDbKG8q0x+hlu3QraWmTYUmSRiVl7ErLW4e6c166B6R8xo0RrNzUKYY4C3BsLczYb/JGWBc/mzDztb55DPYdwpg6Hd7asodvw9YLk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1568451356; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=ugFooomD4oONSh0eHkh/scPLMP6+k+K66cHMwDqX9cE=; b=MnKn65b7nx8wBSkFnfXrWFud6+dBSfsrbAXWOBFyEj0aWf7lpyNFfOGQbqz6MlzIwTymBmN/G01epkCHwgLFhOAZFamj8MkSxQ9Fd+JNpRsFqCM0ZXnMFtgordf/WnCpi+bDLBfcKG4lo5dNqfmlluXUcuyKeEn3kLKlkST68TI= ARC-Authentication-Results: i=1; mx.zoho.com; spf=none (zoho.com: 192.237.175.120 is neither permitted nor denied by domain of lists.xenproject.org) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1568451356742958.1685712646552; Sat, 14 Sep 2019 01:55:56 -0700 (PDT) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1i93pr-0002Xa-UN; Sat, 14 Sep 2019 08:55:15 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1i93pp-0002Ty-Qv for xen-devel@lists.xenproject.org; Sat, 14 Sep 2019 08:55:13 +0000 Received: from mx1.suse.de (unknown [195.135.220.15]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 135a736c-d6cd-11e9-95c1-12813bfff9fa; Sat, 14 Sep 2019 08:53:09 +0000 (UTC) Received: from relay2.suse.de (unknown [195.135.220.254]) by mx1.suse.de (Postfix) with ESMTP id 02147B67E; Sat, 14 Sep 2019 08:53:09 +0000 (UTC) X-Inumbo-ID: 135a736c-d6cd-11e9-95c1-12813bfff9fa X-Virus-Scanned: by amavisd-new at test-mx.suse.de From: Juergen Gross To: xen-devel@lists.xenproject.org Date: Sat, 14 Sep 2019 10:52:49 +0200 Message-Id: <20190914085251.18816-46-jgross@suse.com> X-Mailer: git-send-email 2.16.4 In-Reply-To: <20190914085251.18816-1-jgross@suse.com> References: <20190914085251.18816-1-jgross@suse.com> Subject: [Xen-devel] [PATCH v3 45/47] xen/sched: support core scheduling for moving cpus to/from cpupools X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Juergen Gross , Tim Deegan , Stefano Stabellini , Wei Liu , Konrad Rzeszutek Wilk , George Dunlap , Andrew Cooper , Ian Jackson , Dario Faggioli , Julien Grall , Jan Beulich MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" With core scheduling active it is necessary to move multiple cpus at the same time to or from a cpupool in order to avoid split scheduling resources in between. Signed-off-by: Juergen Gross --- V1: new patch --- xen/common/cpupool.c | 100 +++++++++++++++++++++++++++++++++--------= ---- xen/common/schedule.c | 3 +- xen/include/xen/sched-if.h | 1 + 3 files changed, 76 insertions(+), 28 deletions(-) diff --git a/xen/common/cpupool.c b/xen/common/cpupool.c index ada3a7e824..7f173f5099 100644 --- a/xen/common/cpupool.c +++ b/xen/common/cpupool.c @@ -263,23 +263,30 @@ static int cpupool_assign_cpu_locked(struct cpupool *= c, unsigned int cpu) { int ret; struct domain *d; + const cpumask_t *cpus; + + cpus =3D sched_get_opt_cpumask(c->opt_granularity, cpu); =20 if ( (cpupool_moving_cpu =3D=3D cpu) && (c !=3D cpupool_cpu_moving) ) return -EADDRNOTAVAIL; - ret =3D schedule_cpu_add(cpu, c); + ret =3D schedule_cpu_add(cpumask_first(cpus), c); if ( ret ) return ret; =20 - cpumask_clear_cpu(cpu, &cpupool_free_cpus); + rcu_read_lock(&sched_res_rculock); + + cpumask_andnot(&cpupool_free_cpus, &cpupool_free_cpus, cpus); if (cpupool_moving_cpu =3D=3D cpu) { cpupool_moving_cpu =3D -1; cpupool_put(cpupool_cpu_moving); cpupool_cpu_moving =3D NULL; } - cpumask_set_cpu(cpu, c->cpu_valid); + cpumask_or(c->cpu_valid, c->cpu_valid, cpus); cpumask_and(c->res_valid, c->cpu_valid, sched_res_mask); =20 + rcu_read_unlock(&sched_res_rculock); + rcu_read_lock(&domlist_read_lock); for_each_domain_in_cpupool(d, c) { @@ -293,6 +300,7 @@ static int cpupool_assign_cpu_locked(struct cpupool *c,= unsigned int cpu) static int cpupool_unassign_cpu_finish(struct cpupool *c) { int cpu =3D cpupool_moving_cpu; + const cpumask_t *cpus; struct domain *d; int ret; =20 @@ -305,7 +313,10 @@ static int cpupool_unassign_cpu_finish(struct cpupool = *c) */ rcu_read_lock(&domlist_read_lock); ret =3D cpu_disable_scheduler(cpu); - cpumask_set_cpu(cpu, &cpupool_free_cpus); + + rcu_read_lock(&sched_res_rculock); + cpus =3D get_sched_res(cpu)->cpus; + cpumask_or(&cpupool_free_cpus, &cpupool_free_cpus, cpus); =20 /* * cpu_disable_scheduler() returning an error doesn't require resetting @@ -318,7 +329,7 @@ static int cpupool_unassign_cpu_finish(struct cpupool *= c) { ret =3D schedule_cpu_rm(cpu); if ( ret ) - cpumask_clear_cpu(cpu, &cpupool_free_cpus); + cpumask_andnot(&cpupool_free_cpus, &cpupool_free_cpus, cpus); else { cpupool_moving_cpu =3D -1; @@ -326,6 +337,7 @@ static int cpupool_unassign_cpu_finish(struct cpupool *= c) cpupool_cpu_moving =3D NULL; } } + rcu_read_unlock(&sched_res_rculock); =20 for_each_domain_in_cpupool(d, c) { @@ -340,6 +352,7 @@ static int cpupool_unassign_cpu_start(struct cpupool *c= , unsigned int cpu) { int ret; struct domain *d; + const cpumask_t *cpus; =20 spin_lock(&cpupool_lock); ret =3D -EADDRNOTAVAIL; @@ -348,7 +361,11 @@ static int cpupool_unassign_cpu_start(struct cpupool *= c, unsigned int cpu) goto out; =20 ret =3D 0; - if ( (c->n_dom > 0) && (cpumask_weight(c->cpu_valid) =3D=3D 1) && + rcu_read_lock(&sched_res_rculock); + cpus =3D get_sched_res(cpu)->cpus; + + if ( (c->n_dom > 0) && + (cpumask_weight(c->cpu_valid) =3D=3D cpumask_weight(cpus)) && (cpu !=3D cpupool_moving_cpu) ) { rcu_read_lock(&domlist_read_lock); @@ -370,9 +387,10 @@ static int cpupool_unassign_cpu_start(struct cpupool *= c, unsigned int cpu) cpupool_moving_cpu =3D cpu; atomic_inc(&c->refcnt); cpupool_cpu_moving =3D c; - cpumask_clear_cpu(cpu, c->cpu_valid); + cpumask_andnot(c->cpu_valid, c->cpu_valid, cpus); cpumask_and(c->res_valid, c->cpu_valid, sched_res_mask); =20 + rcu_read_unlock(&domlist_read_lock); out: spin_unlock(&cpupool_lock); =20 @@ -412,11 +430,13 @@ static int cpupool_unassign_cpu(struct cpupool *c, un= signed int cpu) { int work_cpu; int ret; + unsigned int master_cpu; =20 debugtrace_printk("cpupool_unassign_cpu(pool=3D%d,cpu=3D%d)\n", c->cpupool_id, cpu); =20 - ret =3D cpupool_unassign_cpu_start(c, cpu); + master_cpu =3D sched_get_resource_cpu(cpu); + ret =3D cpupool_unassign_cpu_start(c, master_cpu); if ( ret ) { debugtrace_printk("cpupool_unassign_cpu(pool=3D%d,cpu=3D%d) ret %d= \n", @@ -424,12 +444,12 @@ static int cpupool_unassign_cpu(struct cpupool *c, un= signed int cpu) return ret; } =20 - work_cpu =3D smp_processor_id(); - if ( work_cpu =3D=3D cpu ) + work_cpu =3D sched_get_resource_cpu(smp_processor_id()); + if ( work_cpu =3D=3D master_cpu ) { work_cpu =3D cpumask_first(cpupool0->cpu_valid); - if ( work_cpu =3D=3D cpu ) - work_cpu =3D cpumask_next(cpu, cpupool0->cpu_valid); + if ( work_cpu =3D=3D master_cpu ) + work_cpu =3D cpumask_last(cpupool0->cpu_valid); } return continue_hypercall_on_cpu(work_cpu, cpupool_unassign_cpu_helper= , c); } @@ -495,6 +515,7 @@ void cpupool_rm_domain(struct domain *d) static int cpupool_cpu_add(unsigned int cpu) { int ret =3D 0; + const cpumask_t *cpus; =20 spin_lock(&cpupool_lock); cpumask_clear_cpu(cpu, &cpupool_locked_cpus); @@ -508,7 +529,11 @@ static int cpupool_cpu_add(unsigned int cpu) */ rcu_read_lock(&sched_res_rculock); get_sched_res(cpu)->cpupool =3D NULL; - ret =3D cpupool_assign_cpu_locked(cpupool0, cpu); + + cpus =3D sched_get_opt_cpumask(cpupool0->opt_granularity, cpu); + if ( cpumask_subset(cpus, &cpupool_free_cpus) ) + ret =3D cpupool_assign_cpu_locked(cpupool0, cpu); + rcu_read_unlock(&sched_res_rculock); =20 spin_unlock(&cpupool_lock); @@ -543,27 +568,33 @@ static void cpupool_cpu_remove(unsigned int cpu) static int cpupool_cpu_remove_prologue(unsigned int cpu) { int ret =3D 0; + cpumask_t *cpus; + unsigned int master_cpu; =20 spin_lock(&cpupool_lock); =20 - if ( cpumask_test_cpu(cpu, &cpupool_locked_cpus) ) + rcu_read_lock(&sched_res_rculock); + cpus =3D get_sched_res(cpu)->cpus; + master_cpu =3D sched_get_resource_cpu(cpu); + if ( cpumask_intersects(cpus, &cpupool_locked_cpus) ) ret =3D -EBUSY; else cpumask_set_cpu(cpu, &cpupool_locked_cpus); + rcu_read_unlock(&sched_res_rculock); =20 spin_unlock(&cpupool_lock); =20 if ( ret ) return ret; =20 - if ( cpumask_test_cpu(cpu, cpupool0->cpu_valid) ) + if ( cpumask_test_cpu(master_cpu, cpupool0->cpu_valid) ) { /* Cpupool0 is populated only after all cpus are up. */ ASSERT(system_state =3D=3D SYS_STATE_active); =20 - ret =3D cpupool_unassign_cpu_start(cpupool0, cpu); + ret =3D cpupool_unassign_cpu_start(cpupool0, master_cpu); } - else if ( !cpumask_test_cpu(cpu, &cpupool_free_cpus) ) + else if ( !cpumask_test_cpu(master_cpu, &cpupool_free_cpus) ) ret =3D -ENODEV; =20 return ret; @@ -580,12 +611,13 @@ static void cpupool_cpu_remove_forced(unsigned int cp= u) { struct cpupool **c; int ret; + unsigned int master_cpu =3D sched_get_resource_cpu(cpu); =20 for_each_cpupool ( c ) { - if ( cpumask_test_cpu(cpu, (*c)->cpu_valid) ) + if ( cpumask_test_cpu(master_cpu, (*c)->cpu_valid) ) { - ret =3D cpupool_unassign_cpu_start(*c, cpu); + ret =3D cpupool_unassign_cpu_start(*c, master_cpu); BUG_ON(ret); ret =3D cpupool_unassign_cpu_finish(*c); BUG_ON(ret); @@ -653,29 +685,45 @@ int cpupool_do_sysctl(struct xen_sysctl_cpupool_op *o= p) case XEN_SYSCTL_CPUPOOL_OP_ADDCPU: { unsigned cpu; + const cpumask_t *cpus; =20 cpu =3D op->cpu; debugtrace_printk("cpupool_assign_cpu(pool=3D%d,cpu=3D%d)\n", op->cpupool_id, cpu); + spin_lock(&cpupool_lock); + + c =3D cpupool_find_by_id(op->cpupool_id); + ret =3D -ENOENT; + if ( c =3D=3D NULL ) + goto addcpu_out; if ( cpu =3D=3D XEN_SYSCTL_CPUPOOL_PAR_ANY ) - cpu =3D cpumask_first(&cpupool_free_cpus); + { + for_each_cpu ( cpu, &cpupool_free_cpus ) + { + cpus =3D sched_get_opt_cpumask(c->opt_granularity, cpu); + if ( cpumask_subset(cpus, &cpupool_free_cpus) ) + break; + } + ret =3D -ENODEV; + if ( cpu >=3D nr_cpu_ids ) + goto addcpu_out; + } ret =3D -EINVAL; if ( cpu >=3D nr_cpu_ids ) goto addcpu_out; ret =3D -ENODEV; - if ( !cpumask_test_cpu(cpu, &cpupool_free_cpus) || - cpumask_test_cpu(cpu, &cpupool_locked_cpus) ) - goto addcpu_out; - c =3D cpupool_find_by_id(op->cpupool_id); - ret =3D -ENOENT; - if ( c =3D=3D NULL ) + cpus =3D sched_get_opt_cpumask(c->opt_granularity, cpu); + if ( !cpumask_subset(cpus, &cpupool_free_cpus) || + cpumask_intersects(cpus, &cpupool_locked_cpus) ) goto addcpu_out; ret =3D cpupool_assign_cpu_locked(c, cpu); + addcpu_out: spin_unlock(&cpupool_lock); debugtrace_printk("cpupool_assign_cpu(pool=3D%d,cpu=3D%d) ret %d\n= ", op->cpupool_id, cpu, ret); + } break; =20 diff --git a/xen/common/schedule.c b/xen/common/schedule.c index 4d56cd907f..4b2ccb7ddc 100644 --- a/xen/common/schedule.c +++ b/xen/common/schedule.c @@ -2566,8 +2566,7 @@ static struct notifier_block cpu_schedule_nfb =3D { .notifier_call =3D cpu_schedule_callback }; =20 -static const cpumask_t *sched_get_opt_cpumask(enum sched_gran opt, - unsigned int cpu) +const cpumask_t *sched_get_opt_cpumask(enum sched_gran opt, unsigned int c= pu) { const cpumask_t *mask; =20 diff --git a/xen/include/xen/sched-if.h b/xen/include/xen/sched-if.h index 1770b9bca1..f1e8f44f62 100644 --- a/xen/include/xen/sched-if.h +++ b/xen/include/xen/sched-if.h @@ -625,5 +625,6 @@ affinity_balance_cpumask(const struct sched_unit *unit,= int step, } =20 void sched_rm_cpu(unsigned int cpu); +const cpumask_t *sched_get_opt_cpumask(enum sched_gran opt, unsigned int c= pu); =20 #endif /* __XEN_SCHED_IF_H__ */ --=20 2.16.4 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel