From nobody Mon Feb 9 22:24:28 2026 Delivered-To: importer@patchew.org Received-SPF: none (zoho.com: 192.237.175.120 is neither permitted nor denied by domain of lists.xenproject.org) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; spf=none (zoho.com: 192.237.175.120 is neither permitted nor denied by domain of lists.xenproject.org) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org ARC-Seal: i=1; a=rsa-sha256; t=1559039743; cv=none; d=zoho.com; s=zohoarc; b=Qm95muJlFY18jM6oVEbIsutK+c3CoIkMXnscBfGvwQIn6bCSF8E48Gdm8HankNMCFX5WQdo2qfQSOMeU/TPv+4C3G0+C2wvSZd5xi15GwglrI2te52ay4xZVCGZhO44wRWgO29tBIxEa7uGVjrCKCRFWnaN1HImyjaeQFplagHk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1559039743; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=7MW4Wxcmb4KpYVwZdFbHoBvgZ12Q3FGI0SyQfI/jO3Q=; b=A6zmeymU6brWdR8CELV0qo9wRrdSeo5VhRwEueB9zf5xru3KAmygMEG/w5m2hdRGc4pgAgfwW4ivi8OpAk6JT5RYdev/uwjTi+R3XiOMHf82BiwXp7W7DcuysPpe9NaBdiDEUW4P7SG8Vy+RmLxJmSdCwbnH+v+JDEGFFYGkAsg= ARC-Authentication-Results: i=1; mx.zoho.com; spf=none (zoho.com: 192.237.175.120 is neither permitted nor denied by domain of lists.xenproject.org) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1559039743278181.47100912924225; Tue, 28 May 2019 03:35:43 -0700 (PDT) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hVZRC-0006zX-CA; Tue, 28 May 2019 10:34:34 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hVZQH-00050d-Uc for xen-devel@lists.xenproject.org; Tue, 28 May 2019 10:33:37 +0000 Received: from mx1.suse.de (unknown [195.135.220.15]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 081754e6-8134-11e9-95b2-776e31728be0; Tue, 28 May 2019 10:33:29 +0000 (UTC) Received: from relay2.suse.de (unknown [195.135.220.254]) by mx1.suse.de (Postfix) with ESMTP id 35317B034; Tue, 28 May 2019 10:33:28 +0000 (UTC) X-Inumbo-ID: 081754e6-8134-11e9-95b2-776e31728be0 X-Virus-Scanned: by amavisd-new at test-mx.suse.de From: Juergen Gross To: xen-devel@lists.xenproject.org Date: Tue, 28 May 2019 12:32:58 +0200 Message-Id: <20190528103313.1343-46-jgross@suse.com> X-Mailer: git-send-email 2.16.4 In-Reply-To: <20190528103313.1343-1-jgross@suse.com> References: <20190528103313.1343-1-jgross@suse.com> Subject: [Xen-devel] [PATCH 45/60] xen/sched: make vcpu_wake() and vcpu_sleep() core scheduling aware X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Juergen Gross , Tim Deegan , Stefano Stabellini , Wei Liu , Konrad Rzeszutek Wilk , George Dunlap , Andrew Cooper , Ian Jackson , Dario Faggioli , Julien Grall , Jan Beulich MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" vcpu_wake() and vcpu_sleep() need to be made core scheduling aware: they might need to switch a single vcpu of an already scheduled unit between running and not running. Especially when vcpu_sleep() for a vcpu is being called by a vcpu of the same scheduling unit special care must be taken in order to avoid a deadlock: the vcpu to be put asleep must be forced through a context switch without doing so for the calling vcpu. For this purpose add a vcpu flag handled in sched_slave() and in sched_wait_rendezvous_in() allowing a vcpu of the currently running unit to switch state at a higher priority than a normal schedule event. Use the same mechanism when waking up a vcpu of a currently active unit. While at it make vcpu_sleep_nosync_locked() static as it is used in schedule.c only. Signed-off-by: Juergen Gross --- RFC V2: add vcpu_sleep() handling and force_context_switch flag --- xen/common/schedule.c | 144 ++++++++++++++++++++++++++++++++++++++++-= ---- xen/include/xen/sched-if.h | 9 ++- xen/include/xen/sched.h | 2 + 3 files changed, 136 insertions(+), 19 deletions(-) diff --git a/xen/common/schedule.c b/xen/common/schedule.c index e0103fdb1d..e189cc7c2b 100644 --- a/xen/common/schedule.c +++ b/xen/common/schedule.c @@ -79,21 +79,21 @@ extern const struct scheduler *__start_schedulers_array= [], *__end_schedulers_arr =20 static struct scheduler __read_mostly ops; =20 -static inline struct vcpu *sched_unit2vcpu_cpu(struct sched_unit *unit, - unsigned int cpu) +static inline struct vcpu *unit2vcpu_cpu(struct sched_unit *unit, + unsigned int cpu) { unsigned int idx =3D unit->unit_id + per_cpu(sched_res_idx, cpu); const struct domain *d =3D unit->domain; - struct vcpu *v; =20 - if ( idx < d->max_vcpus && d->vcpu[idx] ) - { - v =3D d->vcpu[idx]; - if ( v->new_state =3D=3D RUNSTATE_running ) - return v; - } + return (idx < d->max_vcpus && d->vcpu[idx]) ? d->vcpu[idx] : NULL; +} =20 - return idle_vcpu[cpu]; +static inline struct vcpu *sched_unit2vcpu_cpu(struct sched_unit *unit, + unsigned int cpu) +{ + struct vcpu *v =3D unit2vcpu_cpu(unit, cpu); + + return (v && v->new_state =3D=3D RUNSTATE_running) ? v : idle_vcpu[cpu= ]; } =20 static inline struct scheduler *dom_scheduler(const struct domain *d) @@ -656,8 +656,10 @@ void sched_destroy_domain(struct domain *d) } } =20 -void vcpu_sleep_nosync_locked(struct vcpu *v) +static void vcpu_sleep_nosync_locked(struct vcpu *v) { + struct sched_unit *unit =3D v->sched_unit; + ASSERT(spin_is_locked(get_sched_res(v->processor)->schedule_lock)); =20 if ( likely(!vcpu_runnable(v)) ) @@ -665,7 +667,14 @@ void vcpu_sleep_nosync_locked(struct vcpu *v) if ( v->runstate.state =3D=3D RUNSTATE_runnable ) vcpu_runstate_change(v, RUNSTATE_offline, NOW()); =20 - sched_sleep(vcpu_scheduler(v), v->sched_unit); + if ( likely(!unit_runnable(unit)) ) + sched_sleep(vcpu_scheduler(v), unit); + else if ( unit_running(unit) > 1 && v->is_running && + !v->force_context_switch ) + { + v->force_context_switch =3D true; + cpu_raise_softirq(v->processor, SCHED_SLAVE_SOFTIRQ); + } } } =20 @@ -697,16 +706,22 @@ void vcpu_wake(struct vcpu *v) { unsigned long flags; spinlock_t *lock; + struct sched_unit *unit =3D v->sched_unit; =20 TRACE_2D(TRC_SCHED_WAKE, v->domain->domain_id, v->vcpu_id); =20 - lock =3D unit_schedule_lock_irqsave(v->sched_unit, &flags); + lock =3D unit_schedule_lock_irqsave(unit, &flags); =20 if ( likely(vcpu_runnable(v)) ) { if ( v->runstate.state >=3D RUNSTATE_blocked ) vcpu_runstate_change(v, RUNSTATE_runnable, NOW()); - sched_wake(vcpu_scheduler(v), v->sched_unit); + sched_wake(vcpu_scheduler(v), unit); + if ( unit->is_running && !v->is_running && !v->force_context_switc= h ) + { + v->force_context_switch =3D true; + cpu_raise_softirq(v->processor, SCHED_SLAVE_SOFTIRQ); + } } else if ( !(v->pause_flags & VPF_blocked) ) { @@ -714,7 +729,7 @@ void vcpu_wake(struct vcpu *v) vcpu_runstate_change(v, RUNSTATE_offline, NOW()); } =20 - unit_schedule_unlock_irqrestore(lock, flags, v->sched_unit); + unit_schedule_unlock_irqrestore(lock, flags, unit); } =20 void vcpu_unblock(struct vcpu *v) @@ -1856,6 +1871,61 @@ static void sched_context_switch(struct vcpu *vprev,= struct vcpu *vnext, context_switch(vprev, vnext); } =20 +/* + * Force a context switch of a single vcpu of an unit. + * Might be called either if a vcpu of an already running unit is woken up + * or if a vcpu of a running unit is put asleep with other vcpus of the sa= me + * unit still running. + */ +static struct vcpu *sched_force_context_switch(struct vcpu *vprev, + struct vcpu *v, + int cpu, s_time_t now) +{ + v->force_context_switch =3D false; + + if ( vcpu_runnable(v) =3D=3D v->is_running ) + return NULL; + + if ( vcpu_runnable(v) ) + { + if ( is_idle_vcpu(vprev) ) + { + vcpu_runstate_change(vprev, RUNSTATE_runnable, now); + vprev->sched_unit =3D get_sched_res(cpu)->sched_unit_idle; + } + vcpu_runstate_change(v, RUNSTATE_running, now); + } + else + { + /* Make sure not to switch last vcpu of an unit away. */ + if ( unit_running(v->sched_unit) =3D=3D 1 ) + return NULL; + + vcpu_runstate_change(v, vcpu_runstate_blocked(v), now); + v =3D sched_unit2vcpu_cpu(vprev->sched_unit, cpu); + if ( v !=3D vprev ) + { + if ( is_idle_vcpu(vprev) ) + { + vcpu_runstate_change(vprev, RUNSTATE_runnable, now); + vprev->sched_unit =3D get_sched_res(cpu)->sched_unit_idle; + } + else + { + v->sched_unit =3D vprev->sched_unit; + vcpu_runstate_change(v, RUNSTATE_running, now); + } + } + } + + v->is_running =3D 1; + + /* Make sure not to loose another slave call. */ + raise_softirq(SCHED_SLAVE_SOFTIRQ); + + return v; +} + /* * Rendezvous before taking a scheduling decision. * Called with schedule lock held, so all accesses to the rendezvous count= er @@ -1871,6 +1941,7 @@ static struct sched_unit *sched_wait_rendezvous_in(st= ruct sched_unit *prev, s_time_t now) { struct sched_unit *next; + struct vcpu *v; =20 if ( !--prev->rendezvous_in_cnt ) { @@ -1879,8 +1950,28 @@ static struct sched_unit *sched_wait_rendezvous_in(s= truct sched_unit *prev, return next; } =20 + v =3D unit2vcpu_cpu(prev, cpu); while ( prev->rendezvous_in_cnt ) { + if ( v && v->force_context_switch ) + { + struct vcpu *vprev =3D current; + + v =3D sched_force_context_switch(vprev, v, cpu, now); + + if ( v ) + { + /* We'll come back another time, so adjust rendezvous_in_c= nt. */ + prev->rendezvous_in_cnt++; + + pcpu_schedule_unlock_irq(lock, cpu); + + sched_context_switch(vprev, v, false, now); + } + + v =3D unit2vcpu_cpu(prev, cpu); + } + pcpu_schedule_unlock_irq(lock, cpu); =20 /* Coming from idle might need to do tasklet work. */ @@ -1897,10 +1988,11 @@ static struct sched_unit *sched_wait_rendezvous_in(= struct sched_unit *prev, =20 static void sched_slave(void) { - struct vcpu *vprev =3D current; + struct vcpu *v, *vprev =3D current; struct sched_unit *prev =3D vprev->sched_unit, *next; s_time_t now; spinlock_t *lock; + bool do_softirq =3D false; int cpu =3D smp_processor_id(); =20 ASSERT_NOT_IN_ATOMIC(); @@ -1909,9 +2001,29 @@ static void sched_slave(void) =20 now =3D NOW(); =20 + v =3D unit2vcpu_cpu(prev, cpu); + if ( v && v->force_context_switch ) + { + v =3D sched_force_context_switch(vprev, v, cpu, now); + + if ( v ) + { + pcpu_schedule_unlock_irq(lock, cpu); + + sched_context_switch(vprev, v, false, now); + } + + do_softirq =3D true; + } + if ( !prev->rendezvous_in_cnt ) { pcpu_schedule_unlock_irq(lock, cpu); + + /* Check for failed forced context switch. */ + if ( do_softirq ) + raise_softirq(SCHEDULE_SOFTIRQ); + return; } =20 diff --git a/xen/include/xen/sched-if.h b/xen/include/xen/sched-if.h index a1aefa2a25..f5962cbcfb 100644 --- a/xen/include/xen/sched-if.h +++ b/xen/include/xen/sched-if.h @@ -88,6 +88,11 @@ static inline bool unit_runnable(const struct sched_unit= *unit) return false; } =20 +static inline int vcpu_runstate_blocked(struct vcpu *v) +{ + return (v->pause_flags & VPF_blocked) ? RUNSTATE_blocked : RUNSTATE_of= fline; +} + static inline bool unit_runnable_state(const struct sched_unit *unit) { struct vcpu *v; @@ -100,9 +105,7 @@ static inline bool unit_runnable_state(const struct sch= ed_unit *unit) { runnable =3D vcpu_runnable(v); =20 - v->new_state =3D runnable ? RUNSTATE_running - : (v->pause_flags & VPF_blocked) - ? RUNSTATE_blocked : RUNSTATE_offline; + v->new_state =3D runnable ? RUNSTATE_running : vcpu_runstate_block= ed(v); =20 if ( runnable ) ret =3D true; diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h index 0581c7d44f..b6496f57f6 100644 --- a/xen/include/xen/sched.h +++ b/xen/include/xen/sched.h @@ -186,6 +186,8 @@ struct vcpu bool is_running; /* VCPU should wake fast (do not deep sleep the CPU). */ bool is_urgent; + /* VCPU must context_switch without scheduling unit. */ + bool force_context_switch; =20 #ifdef VCPU_TRAP_LAST #define VCPU_TRAP_NONE 0 --=20 2.16.4 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel