From nobody Tue Nov 11 11:49:36 2025 Delivered-To: importer@patchew.org Received-SPF: none (zoho.com: 192.237.175.120 is neither permitted nor denied by domain of lists.xenproject.org) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; spf=none (zoho.com: 192.237.175.120 is neither permitted nor denied by domain of lists.xenproject.org) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org ARC-Seal: i=1; a=rsa-sha256; t=1564122428; cv=none; d=zoho.com; s=zohoarc; b=CkkV62m/PPc7sUQSe8WufJMk6FFg1IUE57Lm7wmdCLcDpZEgaMwNN/gGcjAYmsx4hTAPHLOyn/U0K36DCX7I7XmsBHIdtnbaMfiaBIkNA0emqpaB8Wa2WCx0lVJzRdRPMvH79AJF1GRyr5EWPHIju+OUJrI+MOJ6W28NRlm1/KI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1564122428; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=TNWkDdoLteBkzHPxsLdNRe2cMNUDNeKHXqupZ3JMqEI=; b=i0erBHC1V4hsfKJl8vZVY60/n1RBnRsV5CeB18JVMWGQ4zpqyLIVXdn3ohaHsukdUGOPlWx4J/36/nxjZ1C10TTUW1mO004pwpHaGSyqZDMKQYoyHImU2d+LhCE22SBi2U8ou2UvH2Ky5klhJ3hIzlZFgP7w+e/IRcIhnuc6PXM= ARC-Authentication-Results: i=1; mx.zoho.com; spf=none (zoho.com: 192.237.175.120 is neither permitted nor denied by domain of lists.xenproject.org) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1564122428020129.11027203932917; Thu, 25 Jul 2019 23:27:08 -0700 (PDT) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hqtg0-00039d-J5; Fri, 26 Jul 2019 06:26:00 +0000 Received: from us1-rack-dfw2.inumbo.com ([104.130.134.6]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hqtfz-00039P-J8 for xen-devel@lists.xenproject.org; Fri, 26 Jul 2019 06:25:59 +0000 Received: from mx1.suse.de (unknown [195.135.220.15]) by us1-rack-dfw2.inumbo.com (Halon) with ESMTPS id 3a8deccd-af6e-11e9-8980-bc764e045a96; Fri, 26 Jul 2019 06:25:58 +0000 (UTC) Received: from relay2.suse.de (unknown [195.135.220.254]) by mx1.suse.de (Postfix) with ESMTP id 3B627AD31; Fri, 26 Jul 2019 06:25:57 +0000 (UTC) X-Inumbo-ID: 3a8deccd-af6e-11e9-8980-bc764e045a96 X-Virus-Scanned: by amavisd-new at test-mx.suse.de From: Dario Faggioli To: xen-devel@lists.xenproject.org Date: Fri, 26 Jul 2019 08:25:56 +0200 Message-ID: <156412235656.2385.13861979113936528474.stgit@Palanthas> In-Reply-To: <156412188377.2385.12588508835559819141.stgit@Palanthas> References: <156412188377.2385.12588508835559819141.stgit@Palanthas> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 Subject: [Xen-devel] [PATCH v2 2/4] xen: sched: deal with vCPUs being or becoming online or offline X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: George Dunlap , Stefano Stabellini , Roger Pau Monne Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" If a vCPU is, or is going, offline we want it to be neither assigned to a pCPU, nor in the wait list, so: - if an offline vcpu is inserted (or migrated) it must not go on a pCPU, nor in the wait list; - if an offline vcpu is removed, we are sure that it is neither on a pCPU nor in the wait list already, so we should just bail, avoiding doing any further action; - if a vCPU goes offline we need to remove it either from its pCPU or from the wait list. Signed-off-by: Dario Faggioli Reviewed-by: George Dunlap --- Cc: George Dunlap Cc: Stefano Stabellini Cc: Roger Pau Monne --- Changes from v1: * improved wording in changelog and comments * this patch is the result of the merge of patches 2 and 3 from v1 --- xen/common/sched_null.c | 80 +++++++++++++++++++++++++++++++++++++++----= ---- 1 file changed, 66 insertions(+), 14 deletions(-) diff --git a/xen/common/sched_null.c b/xen/common/sched_null.c index c47c1b5aae..10e96f21dd 100644 --- a/xen/common/sched_null.c +++ b/xen/common/sched_null.c @@ -339,6 +339,8 @@ static unsigned int pick_cpu(struct null_private *prv, = struct vcpu *v) static void vcpu_assign(struct null_private *prv, struct vcpu *v, unsigned int cpu) { + ASSERT(is_vcpu_online(v)); + per_cpu(npc, cpu).vcpu =3D v; v->processor =3D cpu; cpumask_clear_cpu(cpu, &prv->cpus_free); @@ -358,7 +360,8 @@ static void vcpu_assign(struct null_private *prv, struc= t vcpu *v, } } =20 -static void vcpu_deassign(struct null_private *prv, struct vcpu *v) +/* Returns true if a cpu was tickled */ +static bool vcpu_deassign(struct null_private *prv, struct vcpu *v) { unsigned int bs; unsigned int cpu =3D v->processor; @@ -403,11 +406,13 @@ static void vcpu_deassign(struct null_private *prv, s= truct vcpu *v) vcpu_assign(prv, wvc->vcpu, cpu); cpu_raise_softirq(cpu, SCHEDULE_SOFTIRQ); spin_unlock(&prv->waitq_lock); - return; + return true; } } } spin_unlock(&prv->waitq_lock); + + return false; } =20 /* Change the scheduler of cpu to us (null). */ @@ -445,8 +450,14 @@ static void null_vcpu_insert(const struct scheduler *o= ps, struct vcpu *v) ASSERT(!is_idle_vcpu(v)); =20 lock =3D vcpu_schedule_lock_irq(v); - retry: =20 + if ( unlikely(!is_vcpu_online(v)) ) + { + vcpu_schedule_unlock_irq(lock, v); + return; + } + + retry: cpu =3D v->processor =3D pick_cpu(prv, v); =20 spin_unlock(lock); @@ -500,6 +511,14 @@ static void null_vcpu_remove(const struct scheduler *o= ps, struct vcpu *v) =20 lock =3D vcpu_schedule_lock_irq(v); =20 + /* If offline, the vcpu shouldn't be assigned, nor in the waitqueue */ + if ( unlikely(!is_vcpu_online(v)) ) + { + ASSERT(per_cpu(npc, v->processor).vcpu !=3D v); + ASSERT(list_empty(&nvc->waitq_elem)); + goto out; + } + /* If v is in waitqueue, just get it out of there and bail */ if ( unlikely(!list_empty(&nvc->waitq_elem)) ) { @@ -549,11 +568,33 @@ static void null_vcpu_wake(const struct scheduler *op= s, struct vcpu *v) =20 static void null_vcpu_sleep(const struct scheduler *ops, struct vcpu *v) { + struct null_private *prv =3D null_priv(ops); + unsigned int cpu =3D v->processor; + bool tickled =3D false; + ASSERT(!is_idle_vcpu(v)); =20 + /*=20 + * Check if the vcpu is in the process of being offlined. if yes, + * we need to remove it from either its pCPU or the waitqueue. + */ + if ( unlikely(!is_vcpu_online(v)) ) + { + struct null_vcpu *nvc =3D null_vcpu(v); + + if ( unlikely(!list_empty(&nvc->waitq_elem)) ) + { + spin_lock(&prv->waitq_lock); + list_del_init(&nvc->waitq_elem); + spin_unlock(&prv->waitq_lock); + } + else if ( per_cpu(npc, cpu).vcpu =3D=3D v ) + tickled =3D vcpu_deassign(prv, v); + } + /* If v is not assigned to a pCPU, or is not running, no need to bothe= r */ - if ( curr_on_cpu(v->processor) =3D=3D v ) - cpu_raise_softirq(v->processor, SCHEDULE_SOFTIRQ); + if ( likely(!tickled && curr_on_cpu(cpu) =3D=3D v) ) + cpu_raise_softirq(cpu, SCHEDULE_SOFTIRQ); =20 SCHED_STAT_CRANK(vcpu_sleep); } @@ -589,24 +630,34 @@ static void null_vcpu_migrate(const struct scheduler = *ops, struct vcpu *v, } =20 /* - * v is either assigned to a pCPU, or in the waitqueue. - * - * In the former case, the pCPU to which it was assigned would - * become free, and we, therefore, should check whether there is - * anyone in the waitqueue that can be assigned to it. - * - * In the latter, there is just nothing to do. + * If v is assigned to a pCPU, then such pCPU becomes free, and we + * should look in the waitqueue if anyone else can be assigned to it. */ - if ( likely(list_empty(&nvc->waitq_elem)) ) + if ( likely(per_cpu(npc, v->processor).vcpu =3D=3D v) ) { vcpu_deassign(prv, v); SCHED_STAT_CRANK(migrate_running); } - else + else if ( !list_empty(&nvc->waitq_elem) ) SCHED_STAT_CRANK(migrate_on_runq); =20 SCHED_STAT_CRANK(migrated); =20 + /* + * If a vcpu is (going) offline, we want it to be neither assigned + * to a pCPU, nor in the waitqueue. + * + * If it was on a cpu, we've removed it from there above. If it is + * in the waitqueue, we remove it from there now. And then we bail. + */ + if ( unlikely(!is_vcpu_online(v)) ) + { + spin_lock(&prv->waitq_lock); + list_del_init(&nvc->waitq_elem); + spin_unlock(&prv->waitq_lock); + goto out; + } + /* * Let's now consider new_cpu, which is where v is being sent. It can = be * either free, or have a vCPU already assigned to it. @@ -646,6 +697,7 @@ static void null_vcpu_migrate(const struct scheduler *o= ps, struct vcpu *v, * at least. In case of suspend, any temporary inconsistency caused * by this, will be fixed-up during resume. */ + out: v->processor =3D new_cpu; } =20 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel