From nobody Wed May 15 20:19:40 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org ARC-Seal: i=1; a=rsa-sha256; t=1590701404; cv=none; d=zohomail.com; s=zohoarc; b=azC6qDdIuIplWHwgZK9yNf5NHERjehZQxCZ2+bI1CVwwhzy5j3RWipAqKIAPZmv/hG00J5lRPLa2VKBJbLDznTP02mWBIxky/za2HUV4tQ05M7VCC04K4JLa3TcIJqzPaxjMjU/dl90riusxsZLeIrw6ugqoRMHl4wi9X2/q5qk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1590701404; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=JBpkxXxyifxfXBjwP1Ac2WLMkdMOyk1zAam+z/GHBmg=; b=IGKRX1iy6AJ9sSacb3x03/UuTPAA4CPXh/EUkEE7Bj75Ngu/PR+K74XUJVefc3VF7csjcIX6xpba0hk1Ub2Wjit9ypw1BIEIKBTMG/AvzAhomXGfXIZA+2FuUJRqYzV/fF3NtzVTNJL/HdT+VNB48OtcHPGgBGUGzXuPexT1lYM= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1590701403990762.7688441271902; Thu, 28 May 2020 14:30:03 -0700 (PDT) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jeQ5g-0000SL-Pk; Thu, 28 May 2020 21:29:28 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jeQ5f-0000SE-BZ for xen-devel@lists.xenproject.org; Thu, 28 May 2020 21:29:27 +0000 Received: from mail-wr1-f67.google.com (unknown [209.85.221.67]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 4e3be89e-a12a-11ea-8993-bc764e2007e4; Thu, 28 May 2020 21:29:26 +0000 (UTC) Received: by mail-wr1-f67.google.com with SMTP id x6so839989wrm.13 for ; Thu, 28 May 2020 14:29:26 -0700 (PDT) Received: from [192.168.0.36] (87.78.186.89.cust.ip.kpnqwest.it. [89.186.78.87]) by smtp.gmail.com with ESMTPSA id d2sm6729892wrs.95.2020.05.28.14.29.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 28 May 2020 14:29:25 -0700 (PDT) X-Inumbo-ID: 4e3be89e-a12a-11ea-8993-bc764e2007e4 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:from:to:cc:date:message-id:in-reply-to :references:user-agent:mime-version:content-transfer-encoding; bh=JBpkxXxyifxfXBjwP1Ac2WLMkdMOyk1zAam+z/GHBmg=; b=ISUONMeK9D/WCUOtqVufpoY5qKXiYNIIA8PSggey/W7A7gYNcpu1ghO6f33Dg1Wi6n 3hJ2bX/loxqIb1Cxx0OYr6MV2iG2vRdS6551jflbwQKSpRuKKxiRYhcMDPHcvDZgl+qE xmgf00kfEEcx+l+1MyJZq2v8OHbASRdAdAKP0uIZIpSxJ1zFHDF6L9h0D19phBeTF175 krje1XT/YlJuv6yg0Eaf4QPBg1MHDV6/XR6CTIRCsNv4ieUaY/pzXxH0FArYFMJL3Aib IgHw36Hul68uv780YmydvZw7LboqPhZF3NVSrH0Wt1q+zMWlDtTpmjgSOirAfGAgmUDR WyGQ== X-Gm-Message-State: AOAM531T1Uh5gENu4d26PMQFIkTXkU0SYenQ/WG/DqKbDwSH0kbdUqfw kA9Cc+SDv3VGi3Vml4L5k6sKVytE X-Google-Smtp-Source: ABdhPJxZDMoEaDChbbXxfLrdkKE5tLhILEdSe7ifgSNEpvEpXkayrtHxOHruayVsuicYAMU2jV2edQ== X-Received: by 2002:a5d:68c2:: with SMTP id p2mr5351046wrw.253.1590701365691; Thu, 28 May 2020 14:29:25 -0700 (PDT) Subject: [PATCH v2 1/7] xen: credit2: factor cpu to runqueue matching in a function From: Dario Faggioli To: xen-devel@lists.xenproject.org Date: Thu, 28 May 2020 23:29:24 +0200 Message-ID: <159070136424.12060.2223986236933194278.stgit@Palanthas> In-Reply-To: <159070133878.12060.13318432301910522647.stgit@Palanthas> References: <159070133878.12060.13318432301910522647.stgit@Palanthas> User-Agent: StGit/0.21 MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Juergen Gross , George Dunlap Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" Just move the big if() condition in an inline function. No functional change intended. Signed-off-by: Dario Faggioli Reviewed-by: Juergen Gross --- Cc: George Dunlap Cc: Juergen Gross --- xen/common/sched/credit2.c | 28 +++++++++++++++++----------- 1 file changed, 17 insertions(+), 11 deletions(-) diff --git a/xen/common/sched/credit2.c b/xen/common/sched/credit2.c index 34f05c3e2a..697c9f917d 100644 --- a/xen/common/sched/credit2.c +++ b/xen/common/sched/credit2.c @@ -838,6 +838,20 @@ static inline bool same_core(unsigned int cpua, unsign= ed int cpub) cpu_to_core(cpua) =3D=3D cpu_to_core(cpub); } =20 +static inline bool +cpu_runqueue_match(const struct csched2_runqueue_data *rqd, unsigned int c= pu) +{ + unsigned int peer_cpu =3D rqd->pick_bias; + + BUG_ON(cpu_to_socket(peer_cpu) =3D=3D XEN_INVALID_SOCKET_ID); + + /* OPT_RUNQUEUE_CPU will never find an existing runqueue. */ + return opt_runqueue =3D=3D OPT_RUNQUEUE_ALL || + (opt_runqueue =3D=3D OPT_RUNQUEUE_CORE && same_core(peer_cpu, c= pu)) || + (opt_runqueue =3D=3D OPT_RUNQUEUE_SOCKET && same_socket(peer_cp= u, cpu)) || + (opt_runqueue =3D=3D OPT_RUNQUEUE_NODE && same_node(peer_cpu, c= pu)); +} + static struct csched2_runqueue_data * cpu_add_to_runqueue(struct csched2_private *prv, unsigned int cpu) { @@ -855,21 +869,11 @@ cpu_add_to_runqueue(struct csched2_private *prv, unsi= gned int cpu) rqd_ins =3D &prv->rql; list_for_each_entry ( rqd, &prv->rql, rql ) { - unsigned int peer_cpu; - /* Remember first unused queue index. */ if ( !rqi_unused && rqd->id > rqi ) rqi_unused =3D true; =20 - peer_cpu =3D rqd->pick_bias; - BUG_ON(cpu_to_socket(cpu) =3D=3D XEN_INVALID_SOCKET_ID || - cpu_to_socket(peer_cpu) =3D=3D XEN_INVALID_SOCKET_ID); - - /* OPT_RUNQUEUE_CPU will never find an existing runqueue. */ - if ( opt_runqueue =3D=3D OPT_RUNQUEUE_ALL || - (opt_runqueue =3D=3D OPT_RUNQUEUE_CORE && same_core(peer_cpu,= cpu)) || - (opt_runqueue =3D=3D OPT_RUNQUEUE_SOCKET && same_socket(peer_= cpu, cpu)) || - (opt_runqueue =3D=3D OPT_RUNQUEUE_NODE && same_node(peer_cpu,= cpu)) ) + if ( cpu_runqueue_match(rqd, cpu) ) { rqd_valid =3D true; break; @@ -3744,6 +3748,8 @@ csched2_alloc_pdata(const struct scheduler *ops, int = cpu) struct csched2_pcpu *spc; struct csched2_runqueue_data *rqd; =20 + BUG_ON(cpu_to_socket(cpu) =3D=3D XEN_INVALID_SOCKET_ID); + spc =3D xzalloc(struct csched2_pcpu); if ( spc =3D=3D NULL ) return ERR_PTR(-ENOMEM); From nobody Wed May 15 20:19:40 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org ARC-Seal: i=1; a=rsa-sha256; t=1590701401; cv=none; d=zohomail.com; s=zohoarc; b=gdR+e1q8qRfuIpjVEUAls3w+eF6OxH1RYNPtRWkXWlAOkYKcil0iAxDChK0oCTIKMUnwnd/V/XSsPZKHTUm1sHegbdZijTkIeOvWf8TdMf38v8aVL1FsKaaxA8/ZQgVwJl0Skpry6i3PXhICIurM4yXO4AGYOAE9hc9zCMH+ttI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1590701401; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=RguJeW3M7hHVpj8Rg/UhcbGBJXMeePiaJDG/SqFx9qw=; b=coGe2Ftgt5X9cy7EcvzjIlqRNEEeRh+KjwAn3sQHDoq4ahwZaYsjRNFJfA1a9BaPwq6kEPRJW1cPzFGbuXAkj6NnHkprmlkoayfP3hoyQfhmpDdYhzpXiksgWKy8guECyEPYCMQ/NNMG77ODK6lctLNYUpHvFdI3HC7gZXVZNB4= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1590701401877256.2409428287624; Thu, 28 May 2020 14:30:01 -0700 (PDT) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jeQ5n-0000T9-6h; Thu, 28 May 2020 21:29:35 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jeQ5m-0000Sw-1L for xen-devel@lists.xenproject.org; Thu, 28 May 2020 21:29:34 +0000 Received: from mail-wm1-f67.google.com (unknown [209.85.128.67]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 5227e598-a12a-11ea-a83e-12813bfff9fa; Thu, 28 May 2020 21:29:33 +0000 (UTC) Received: by mail-wm1-f67.google.com with SMTP id r15so712207wmh.5 for ; Thu, 28 May 2020 14:29:33 -0700 (PDT) Received: from [192.168.0.36] (87.78.186.89.cust.ip.kpnqwest.it. [89.186.78.87]) by smtp.gmail.com with ESMTPSA id u13sm7390046wmm.6.2020.05.28.14.29.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 28 May 2020 14:29:31 -0700 (PDT) X-Inumbo-ID: 5227e598-a12a-11ea-a83e-12813bfff9fa X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:from:to:cc:date:message-id:in-reply-to :references:user-agent:mime-version:content-transfer-encoding; bh=RguJeW3M7hHVpj8Rg/UhcbGBJXMeePiaJDG/SqFx9qw=; b=Cp4uWq9AV541hp6APMQuO6c/0ap0HLV3X0gKNGOxpP6Wkt7IuD3/MWH9geeJYFkmgE 0sLDXTTtJQLbDC45YbuNDGCwR3GOga0Ah2pD2ej0DW5fEPkTMSU9ukTPMZxBPANkw5CE LSsMsavhRku5YFbKs7BilRKnE0TwMqJsfhDeOD7qesemR2t2038O4fq+cCa8mRrvgv93 wpwBDTArI9umvriIJqKb6W4JAoYgnR6tpjjm3VGhqOZxB0jFJqspjSe5IAegUJkAuXBt f7lCV+uAn4E13tcGUMfO+Qd2DumU8mdi9R9xw49+ZUVfwvTAHINs8L8IDIlIfZSHQ7e/ TYJQ== X-Gm-Message-State: AOAM531/7kB9YLmCqKFTExCO98DoUMufPnluAgbDhKZhDoMq2WL+A3yP XEEdFI86rut5L8r3i5s+3EI= X-Google-Smtp-Source: ABdhPJw0iKqTYcG09RFoWIAaYF3y8Wo0zb4gte2lvn5QnotHfh+oR6jtQRIiygSVayWiXsFmRbMpNQ== X-Received: by 2002:a7b:c005:: with SMTP id c5mr5162869wmb.22.1590701372297; Thu, 28 May 2020 14:29:32 -0700 (PDT) Subject: [PATCH v2 2/7] xen: credit2: factor runqueue initialization in its own function. From: Dario Faggioli To: xen-devel@lists.xenproject.org Date: Thu, 28 May 2020 23:29:30 +0200 Message-ID: <159070137084.12060.14661333224235870762.stgit@Palanthas> In-Reply-To: <159070133878.12060.13318432301910522647.stgit@Palanthas> References: <159070133878.12060.13318432301910522647.stgit@Palanthas> User-Agent: StGit/0.21 MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Juergen Gross , George Dunlap Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" As it will be useful in later changes. While there, fix the doc-comment. No functional change intended. Signed-off-by: Dario Faggioli Reviewed-by: Juergen Gross --- Cc: George Dunlap Cc: Juergen Gross --- Changes from v1: * new patch --- xen/common/sched/credit2.c | 35 +++++++++++++++++++++++------------ 1 file changed, 23 insertions(+), 12 deletions(-) diff --git a/xen/common/sched/credit2.c b/xen/common/sched/credit2.c index 697c9f917d..8a4f28b9f5 100644 --- a/xen/common/sched/credit2.c +++ b/xen/common/sched/credit2.c @@ -3766,21 +3766,16 @@ csched2_alloc_pdata(const struct scheduler *ops, in= t cpu) return spc; } =20 -/* Returns the ID of the runqueue the cpu is assigned to. */ -static struct csched2_runqueue_data * -init_pdata(struct csched2_private *prv, struct csched2_pcpu *spc, - unsigned int cpu) +/* + * Do what's necessary to add cpu to the rqd (including activating the + * runqueue, if this is the first CPU we put in it). + */ +static void +init_cpu_runqueue(struct csched2_private *prv, struct csched2_pcpu *spc, + unsigned int cpu, struct csched2_runqueue_data *rqd) { - struct csched2_runqueue_data *rqd; unsigned int rcpu; =20 - ASSERT(rw_is_write_locked(&prv->lock)); - ASSERT(!cpumask_test_cpu(cpu, &prv->initialized)); - /* CPU data needs to be allocated, but still uninitialized. */ - ASSERT(spc); - - rqd =3D spc->rqd; - ASSERT(rqd && !cpumask_test_cpu(cpu, &spc->rqd->active)); =20 printk(XENLOG_INFO "Adding cpu %d to runqueue %d\n", cpu, rqd->id); @@ -3816,6 +3811,22 @@ init_pdata(struct csched2_private *prv, struct csche= d2_pcpu *spc, =20 if ( rqd->nr_cpus =3D=3D 1 ) rqd->pick_bias =3D cpu; +} + +/* Returns a pointer to the runqueue the cpu is assigned to. */ +static struct csched2_runqueue_data * +init_pdata(struct csched2_private *prv, struct csched2_pcpu *spc, + unsigned int cpu) +{ + struct csched2_runqueue_data *rqd; + + ASSERT(rw_is_write_locked(&prv->lock)); + ASSERT(!cpumask_test_cpu(cpu, &prv->initialized)); + /* CPU data needs to be allocated, but still uninitialized. */ + ASSERT(spc); + + rqd =3D spc->rqd; + init_cpu_runqueue(prv, spc, cpu, rqd); =20 return rqd; } From nobody Wed May 15 20:19:40 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org ARC-Seal: i=1; a=rsa-sha256; t=1590701404; cv=none; d=zohomail.com; s=zohoarc; b=j3ICmdlXBLHmeBCCs5EVjDF3BG2wqoiZ7MkUWb77M8eJRlHg+cd6Brhb74lRb5+w+sUc0Mu4kfI7Mz2CXZoNiUhC+pfl3mSlB22rhL0kYxpK5s7jApQKuFBx/vKEsX20ylH9SNCBRECvrNv/EcJ8dbx+HU+rQJkjCgoD1BXF83s= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1590701404; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=wP+bknS/YQFF/cic7V0+7p/sNGGep29LYMTamuMpA6Y=; b=EYXwY8FTfCuKCVZNl/Gv4q8audXcNYt0vfAXuFJaDVdR/R/Oq2IukRJb/aaQ1tUhR43V1iunDHjb3kGRUqcAS+ZkzfXHMujFjvr3jkBCtTyaSy/YDX+mjBXyl/G3RBeG/5cGggzfzDCfSQAc8nrPo+b+C+YoC7G060tnmoUvkxs= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1590701403974897.6394259924002; Thu, 28 May 2020 14:30:03 -0700 (PDT) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jeQ5u-0000Un-Et; Thu, 28 May 2020 21:29:42 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jeQ5s-0000UQ-Ob for xen-devel@lists.xenproject.org; Thu, 28 May 2020 21:29:40 +0000 Received: from mail-wr1-f65.google.com (unknown [209.85.221.65]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 560df526-a12a-11ea-a83e-12813bfff9fa; Thu, 28 May 2020 21:29:39 +0000 (UTC) Received: by mail-wr1-f65.google.com with SMTP id l11so994528wru.0 for ; Thu, 28 May 2020 14:29:39 -0700 (PDT) Received: from [192.168.0.36] (87.78.186.89.cust.ip.kpnqwest.it. [89.186.78.87]) by smtp.gmail.com with ESMTPSA id f128sm7844905wme.1.2020.05.28.14.29.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 28 May 2020 14:29:38 -0700 (PDT) X-Inumbo-ID: 560df526-a12a-11ea-a83e-12813bfff9fa X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:from:to:cc:date:message-id:in-reply-to :references:user-agent:mime-version:content-transfer-encoding; bh=wP+bknS/YQFF/cic7V0+7p/sNGGep29LYMTamuMpA6Y=; b=GiXcJy/GYhrFuVn2NGUdbeREpOAD9M8S3lmGVAwWC30MuNPeG2epOw+3z3pQ92NAiq 9AuqQrnHUXO4b8cS6BzdS70R5ScN5Pox2wMCfd+OBxbz2oM7761vPGr35r3Dxb5dlzET folZ9RzY4kCyvIpIqnuUde1cauSse3FKRPvbPzzplrRr+rMbXtOAF7rSusg6llTPj0Nn hIBZZUc16dBY1et0LuHrjoMrLrt4uAhcMjLfsVhUSLAhx0J0T5C5wHbVeYwVBqMQdpqM lkd3Okf0WcQtzRFvU3G+IHr4PCkqwCy2CyC7RlBGT7Twa/pLRBgSVZz3QBk/GrP87sFK sKCA== X-Gm-Message-State: AOAM5319ij2fFo+I820nWKkCY4meFAIxLgssFVkAZxI0HM2qIHm6TolS bPDo54WckDQJHPq9ok6d58fSrQap X-Google-Smtp-Source: ABdhPJxGVTpK4BVv64hCPnZesAphm6BdyzNlNg637ASAcJ5f6KCUWPUlc2RuiWxUibxdp0d1o5JJcw== X-Received: by 2002:adf:e648:: with SMTP id b8mr5534153wrn.386.1590701378810; Thu, 28 May 2020 14:29:38 -0700 (PDT) Subject: [PATCH v2 3/7] xen: cpupool: add a back-pointer from a scheduler to its pool From: Dario Faggioli To: xen-devel@lists.xenproject.org Date: Thu, 28 May 2020 23:29:37 +0200 Message-ID: <159070137738.12060.10928971799525755388.stgit@Palanthas> In-Reply-To: <159070133878.12060.13318432301910522647.stgit@Palanthas> References: <159070133878.12060.13318432301910522647.stgit@Palanthas> User-Agent: StGit/0.21 MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Juergen Gross , George Dunlap Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" If we need to know within which pool a particular scheduler is working, we can do that by querying the cpupool pointer of any of the sched_resource-s (i.e., ~ any of the CPUs) assigned to the scheduler itself. Basically, we pick any sched_resource that we know uses that scheduler, and we check its *cpupool pointer. If we really know that the resource uses the scheduler, this is fine, as it also means the resource is inside the pool we are looking for. But, of course, we can do that for a pool/scheduler that has not any been given any sched_resource yet (or if we do not know whether or not it has any sched_resource). To overcome such limitation, add a back pointer from the scheduler, to its own pool. Signed-off-by: Dario Faggioli Reviewed-by: Juergen Gross --- Cc: Juergen Gross Cc: George Dunlap --- Changes from v1: * new patch --- xen/common/sched/cpupool.c | 1 + xen/common/sched/private.h | 1 + 2 files changed, 2 insertions(+) diff --git a/xen/common/sched/cpupool.c b/xen/common/sched/cpupool.c index 0664f7fa3d..7ea641ca26 100644 --- a/xen/common/sched/cpupool.c +++ b/xen/common/sched/cpupool.c @@ -287,6 +287,7 @@ static struct cpupool *cpupool_create( if ( c->sched =3D=3D NULL ) goto err; } + c->sched->cpupool =3D c; c->gran =3D opt_sched_granularity; =20 *q =3D c; diff --git a/xen/common/sched/private.h b/xen/common/sched/private.h index b9a5b4c01c..df50976eb2 100644 --- a/xen/common/sched/private.h +++ b/xen/common/sched/private.h @@ -275,6 +275,7 @@ struct scheduler { char *opt_name; /* option name for this scheduler */ unsigned int sched_id; /* ID for this scheduler */ void *sched_data; /* global data pointer */ + struct cpupool *cpupool;/* points to this scheduler's pool */ =20 int (*global_init) (void); =20 From nobody Wed May 15 20:19:40 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org ARC-Seal: i=1; a=rsa-sha256; t=1590701413; cv=none; d=zohomail.com; s=zohoarc; b=OY6konBGIs7CJLPG2vzDrpPQs9+OP6WLc02fX361LiPNa84xwM5SMiLWUpjxzyijNFK+cKVBlh4oafpVT3PVipESbWP7BMsP6EbEJo8f7amuD2ckM+y4j/qyt6fFG1gDW2D7+dH/6t0qGZz6whNExZDtn9dpx70AVWFqFG/h9lg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1590701413; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=3OHWyR/HwCR9rAgCd2sPFwONc/WZNxWgzepLW7mEl5s=; b=DuCFIo7KCmYMw+K9ngleYTgJijPSNvrj4cvcPsd1QrtO6VWuBSW0A0+DBDaIIHSan+EReWShw+7YomsB28CQs5z3WZcCtBbgajtLTzlEvU/JUtwKDV4fn+WYTjsK6IlX+KDYdtEV7CwmdKJjFU13fp7a/p71NktBNVaEI72vRrA= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1590701413321359.68437387280176; Thu, 28 May 2020 14:30:13 -0700 (PDT) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jeQ60-0000Wv-NR; Thu, 28 May 2020 21:29:48 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jeQ5z-0000Wd-KF for xen-devel@lists.xenproject.org; Thu, 28 May 2020 21:29:47 +0000 Received: from mail-wr1-f68.google.com (unknown [209.85.221.68]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 5a133d48-a12a-11ea-9dbe-bc764e2007e4; Thu, 28 May 2020 21:29:46 +0000 (UTC) Received: by mail-wr1-f68.google.com with SMTP id j16so908973wrb.7 for ; Thu, 28 May 2020 14:29:46 -0700 (PDT) Received: from [192.168.0.36] (87.78.186.89.cust.ip.kpnqwest.it. [89.186.78.87]) by smtp.gmail.com with ESMTPSA id t185sm7871999wmt.28.2020.05.28.14.29.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 28 May 2020 14:29:44 -0700 (PDT) X-Inumbo-ID: 5a133d48-a12a-11ea-9dbe-bc764e2007e4 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:from:to:cc:date:message-id:in-reply-to :references:user-agent:mime-version:content-transfer-encoding; bh=3OHWyR/HwCR9rAgCd2sPFwONc/WZNxWgzepLW7mEl5s=; b=O6ZQheLB0zc2hDGifGFP21rWCimraF4c6i5Q/171V7ADdEislZXv+y3ddg3/bSsR5J xv6EzoRkCIAqjwxjjNqxsnbmfCFmIg6bbPXL6HSvCRbr/gAe0nMUpd39oscDC57HSR2v 7Fc7P9etgurKf6C+JhMsoDY6lZ0NnKQoHw5At30PgWH0YO/OCwlvHiSzUV32IbXPkEYQ LKOjDCddNZpF3dblT3d9NHo4DYVg4qJjzgJERYmg1zbRFmqBO89Bi6baSJ9to2ad4ZOS ipbYA0hr3yfHH+x0HMBG+glGDgi1yVgjTWU2dUc/vmsuKkOmzUAbhO1RKlulAoeQSOff zr1A== X-Gm-Message-State: AOAM530o+25K35YLrLfA9Qbhn3n38BXHFFTiRmj4Yx+IJdi8qku5bNvS UeTtbtc5qi9cv5X4Q9tcjtbAo5vr X-Google-Smtp-Source: ABdhPJyDtEKhRCBAmDt+TXjHz1fweMI24AbQNOf2IVNAfXv58Q7YnhR6n8/q/qoSew4h9144P3iIxQ== X-Received: by 2002:adf:ee47:: with SMTP id w7mr5240607wro.171.1590701385533; Thu, 28 May 2020 14:29:45 -0700 (PDT) Subject: [PATCH v2 4/7] xen: credit2: limit the max number of CPUs in a runqueue From: Dario Faggioli To: xen-devel@lists.xenproject.org Date: Thu, 28 May 2020 23:29:44 +0200 Message-ID: <159070138395.12060.9523981885146042705.stgit@Palanthas> In-Reply-To: <159070133878.12060.13318432301910522647.stgit@Palanthas> References: <159070133878.12060.13318432301910522647.stgit@Palanthas> User-Agent: StGit/0.21 MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Juergen Gross , Andrew Cooper , George Dunlap , Jan Beulich Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" In Credit2 CPUs (can) share runqueues, depending on the topology. For instance, with per-socket runqueues (the default) all the CPUs that are part of the same socket share a runqueue. On platform with a huge number of CPUs per socket, that could be a problem. An example is AMD EPYC2 servers, where we can have up to 128 CPUs in a socket. It is of course possible to define other, still topology-based, runqueue arrangements (e.g., per-LLC, per-DIE, etc). But that may still result in runqueues with too many CPUs on other/future platforms. For instance, a system with 96 CPUs and 2 NUMA nodes will end up having 48 CPUs per runqueue. Not as bad, but still a lot! Therefore, let's set a limit to the max number of CPUs that can share a Credit2 runqueue. The actual value is configurable (at boot time), the default being 16. If, for instance, there are more than 16 CPUs in a socket, they'll be split among two (or more) runqueues. Note: with core scheduling enabled, this parameter sets the max number of *scheduling resources* that can share a runqueue. Therefore, with granularity set to core (and assumint 2 threads per core), we will have at most 16 cores per runqueue, which corresponds to 32 threads. But that is fine, considering how core scheduling works. Signed-off-by: Dario Faggioli Reviewed-by: Juergen Gross --- Cc: Andrew Cooper Cc: George Dunlap Cc: Jan Beulich Cc: Juergen Gross --- Changes from v1: - always try to add a CPU to the runqueue with the least CPUs already in it. This should guarantee a more even distribution of CPUs among runqueues, as requested during review; - rename the matching function from foo_smt_bar() to foo_siblings_bar(), which is more generic, and do the same to the per-arch wrappers; - deal with the case where the user is trying to set fewer CPUs per runqueue than there are siblings per core (by putting siblings in the same runq anyway, but logging a message), as requested during review; - use the per-cpupool value for the scheduling granularity, as requested during review; - add a comment about why we also count siblings that are currently outside of our cpupool, as suggested during review; - add a boot command line doc entry; - fix typos in comments; --- docs/misc/xen-command-line.pandoc | 14 ++++ xen/common/sched/credit2.c | 144 +++++++++++++++++++++++++++++++++= ++-- xen/include/asm-arm/cpufeature.h | 5 + xen/include/asm-x86/processor.h | 5 + 4 files changed, 162 insertions(+), 6 deletions(-) diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line= .pandoc index e16bb90184..1787f2c8fb 100644 --- a/docs/misc/xen-command-line.pandoc +++ b/docs/misc/xen-command-line.pandoc @@ -1840,6 +1840,20 @@ with read and write permissions. =20 Choose the default scheduler. =20 +### sched_credit2_max_cpus_runqueue +> `=3D ` + +> Default: `16` + +Defines how many CPUs will be put, at most, in each Credit2 runqueue. + +Runqueues are still arranged according to the host topology (and following +what indicated by the 'credit2_runqueue' parameter). But we also have a cap +to the number of CPUs that share each runqueues. + +A value that is a submultiple of the number of online CPUs is recommended, +as that would likely produce a perfectly balanced runqueue configuration. + ### sched_credit2_migrate_resist > `=3D ` =20 diff --git a/xen/common/sched/credit2.c b/xen/common/sched/credit2.c index 8a4f28b9f5..f4d3f8ae6b 100644 --- a/xen/common/sched/credit2.c +++ b/xen/common/sched/credit2.c @@ -25,6 +25,7 @@ #include #include #include +#include =20 #include "private.h" =20 @@ -471,6 +472,22 @@ static int __init parse_credit2_runqueue(const char *s) } custom_param("credit2_runqueue", parse_credit2_runqueue); =20 +/* + * How many CPUs will be put, at most, in each runqueue. + * + * Runqueues are still arranged according to the host topology (and accord= ing + * to the value of the 'credit2_runqueue' parameter). But we also have a c= ap + * to the number of CPUs that share runqueues. + * + * This should be considered an upper limit. In fact, we also try to balan= ce + * the number of CPUs in each runqueue. And, when doing that, it is possib= le + * that fewer CPUs than what this parameters mandates will actually be put + * in each runqueue. + */ +#define MAX_CPUS_RUNQ 16 +static unsigned int __read_mostly opt_max_cpus_runqueue =3D MAX_CPUS_RUNQ; +integer_param("sched_credit2_max_cpus_runqueue", opt_max_cpus_runqueue); + /* * Per-runqueue data */ @@ -852,18 +869,83 @@ cpu_runqueue_match(const struct csched2_runqueue_data= *rqd, unsigned int cpu) (opt_runqueue =3D=3D OPT_RUNQUEUE_NODE && same_node(peer_cpu, c= pu)); } =20 +/* + * Additional checks, to avoid separating siblings in different runqueues. + * This deals with both Intel's HTs and AMD's CUs. An arch that does not h= ave + * any similar concept will just have cpu_nr_siblings() always return 1, a= nd + * setup the cpu_sibling_mask-s acordingly (as currently does ARM), and th= ings + * will just work as well. + */ +static bool +cpu_runqueue_siblings_match(const struct csched2_runqueue_data *rqd, + unsigned int cpu, unsigned int max_cpus_runq) +{ + unsigned int nr_sibls =3D cpu_nr_siblings(cpu); + unsigned int rcpu, tot_sibls =3D 0; + + /* + * If we put the CPU in this runqueue, we must be sure that there will + * be enough room for accepting its sibling(s) as well. + */ + cpumask_clear(cpumask_scratch_cpu(cpu)); + for_each_cpu ( rcpu, &rqd->active ) + { + ASSERT(rcpu !=3D cpu); + if ( !cpumask_intersects(per_cpu(cpu_sibling_mask, rcpu), cpumask_= scratch_cpu(cpu)) ) + { + /* + * For each CPU already in the runqueue, account for it and for + * its sibling(s), independently from whether they are in the + * runqueue or not. Of course, we do this only once, for each = CPU + * that is already inside the runqueue and all its siblings! + * + * This way, even if there are CPUs in the runqueue with sibli= ngs + * in a different cpupools, we still count all of them here. + * The reason for this is that, if at some future point we will + * move those sibling CPUs to this cpupool, we want them to la= nd + * in this runqueue. Hence we must be sure to leave space for = them. + */ + cpumask_or(cpumask_scratch_cpu(cpu), cpumask_scratch_cpu(cpu), + per_cpu(cpu_sibling_mask, rcpu)); + tot_sibls +=3D cpu_nr_siblings(rcpu); + } + } + /* + * We know that neither the CPU, nor any of its sibling are here, + * or we wouldn't even have entered the function. + */ + ASSERT(!cpumask_intersects(cpumask_scratch_cpu(cpu), + per_cpu(cpu_sibling_mask, cpu))); + + /* Try adding CPU and its sibling(s) to the count and check... */ + return tot_sibls + nr_sibls <=3D max_cpus_runq; +} + static struct csched2_runqueue_data * -cpu_add_to_runqueue(struct csched2_private *prv, unsigned int cpu) +cpu_add_to_runqueue(const struct scheduler *ops, unsigned int cpu) { + struct csched2_private *prv =3D csched2_priv(ops); struct csched2_runqueue_data *rqd, *rqd_new; + struct csched2_runqueue_data *rqd_valid =3D NULL; struct list_head *rqd_ins; unsigned long flags; int rqi =3D 0; - bool rqi_unused =3D false, rqd_valid =3D false; + unsigned int min_rqs, max_cpus_runq; + bool rqi_unused =3D false; =20 /* Prealloc in case we need it - not allowed with interrupts off. */ rqd_new =3D xzalloc(struct csched2_runqueue_data); =20 + /* + * While respecting the limit of not having more than the max number of + * CPUs per runqueue, let's also try to "spread" the CPU, as evenly as + * possible, among the runqueues. For doing that, we need to know upfr= ont + * how many CPUs we have, so let's use the number of CPUs that are onl= ine + * for that. + */ + min_rqs =3D ((num_online_cpus() - 1) / opt_max_cpus_runqueue) + 1; + max_cpus_runq =3D num_online_cpus() / min_rqs; + write_lock_irqsave(&prv->lock, flags); =20 rqd_ins =3D &prv->rql; @@ -873,10 +955,59 @@ cpu_add_to_runqueue(struct csched2_private *prv, unsi= gned int cpu) if ( !rqi_unused && rqd->id > rqi ) rqi_unused =3D true; =20 + /* + * First of all, let's check whether, according to the system + * topology, this CPU belongs in this runqueue. + */ if ( cpu_runqueue_match(rqd, cpu) ) { - rqd_valid =3D true; - break; + /* + * If the CPU has any siblings, they are online and they are + * being added to this cpupool, always keep them together. Even + * if that means violating what the opt_max_cpus_runqueue param + * indicates. However, if this happens, chances are high that a + * too small value was used for the parameter, so warn the user + * about that. + * + * Note that we cannot check this once and for all, say, during + * scheduler initialization. In fact, at least in theory, the + * number of siblings a CPU has may not be the same for all the + * CPUs. + */ + if ( cpumask_intersects(&rqd->active, per_cpu(cpu_sibling_mask= , cpu)) ) + { + if ( cpumask_weight(&rqd->active) >=3D opt_max_cpus_runque= ue ) + { + printk("WARNING: %s: more than opt_max_cpus_runque= ue " + "in a runqueue (%u vs %u), due to topology = constraints.\n" + "Consider raising it!\n", + __func__, opt_max_cpus_runqueue, + cpumask_weight(&rqd->active)); + } + rqd_valid =3D rqd; + break; + } + + /* + * If we're using core (or socket) scheduling, no need to do a= ny + * further checking beyond the number of CPUs already in this + * runqueue respecting our upper bound. + * + * Otherwise, let's try to make sure that siblings stay in the + * same runqueue, pretty much under any cinrcumnstances. + */ + if ( rqd->refcnt < max_cpus_runq && (ops->cpupool->gran !=3D S= CHED_GRAN_cpu || + cpu_runqueue_siblings_match(rqd, cpu, max_cpus_runq)) ) + { + /* + * This runqueue is ok, but as we said, we also want an ev= en + * distribution of the CPUs. So, unless this is the very f= irst + * match, we go on, check all runqueues and actually add t= he + * CPU into the one that is less full. + */ + if ( !rqd_valid || rqd->refcnt < rqd_valid->refcnt ) + rqd_valid =3D rqd; + } } =20 if ( !rqi_unused ) @@ -900,6 +1031,8 @@ cpu_add_to_runqueue(struct csched2_private *prv, unsig= ned int cpu) rqd->pick_bias =3D cpu; rqd->id =3D rqi; } + else + rqd =3D rqd_valid; =20 rqd->refcnt++; =20 @@ -3744,7 +3877,6 @@ csched2_dump(const struct scheduler *ops) static void * csched2_alloc_pdata(const struct scheduler *ops, int cpu) { - struct csched2_private *prv =3D csched2_priv(ops); struct csched2_pcpu *spc; struct csched2_runqueue_data *rqd; =20 @@ -3754,7 +3886,7 @@ csched2_alloc_pdata(const struct scheduler *ops, int = cpu) if ( spc =3D=3D NULL ) return ERR_PTR(-ENOMEM); =20 - rqd =3D cpu_add_to_runqueue(prv, cpu); + rqd =3D cpu_add_to_runqueue(ops, cpu); if ( IS_ERR(rqd) ) { xfree(spc); diff --git a/xen/include/asm-arm/cpufeature.h b/xen/include/asm-arm/cpufeat= ure.h index 9af5666628..8fdf9685d7 100644 --- a/xen/include/asm-arm/cpufeature.h +++ b/xen/include/asm-arm/cpufeature.h @@ -64,6 +64,11 @@ static inline bool cpus_have_cap(unsigned int num) return test_bit(num, cpu_hwcaps); } =20 +static inline cpu_nr_siblings(unsigned int) +{ + return 1; +} + /* System capability check for constant cap */ #define cpus_have_const_cap(num) ({ \ register_t __ret; \ diff --git a/xen/include/asm-x86/processor.h b/xen/include/asm-x86/processo= r.h index 070691882b..73017c3f4b 100644 --- a/xen/include/asm-x86/processor.h +++ b/xen/include/asm-x86/processor.h @@ -174,6 +174,11 @@ extern void init_intel_cacheinfo(struct cpuinfo_x86 *c= ); =20 unsigned int apicid_to_socket(unsigned int); =20 +static inline int cpu_nr_siblings(unsigned int cpu) +{ + return cpu_data[cpu].x86_num_siblings; +} + /* * Generic CPUID function * clear %ecx since some cpus (Cyrix MII) do not set or clear %ecx From nobody Wed May 15 20:19:40 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org ARC-Seal: i=1; a=rsa-sha256; t=1590701416; cv=none; d=zohomail.com; s=zohoarc; b=TZ6Q9O5b0HMSpvaxtxSlp6VL93FOzuFuho34y8qfJDkIjO7Oi8hbc7UJBRZnusYLcr8exuPfmQKMsIyyZUJ7lqJDVE1u4bZqoFMs5KBHMaF3ZH1d1rPbl35agMQwzLtGORxM1/2LOuXRyphKt90SVFEXpyy7C47IRCY8KPxVfeg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1590701416; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=rQ7Tl9PBNxTBxCYn4Y9mVqk3mlMzPIyms5hZ/Bt2tmY=; b=fE/1ojnpmXW2IoL771varOl7qEEDOVYpG068gHrQTyz91PPrlxhXdek6peQYghVE0xLcSs1eN7QE2ViZWNQ6uSXgesDpfYI3KOESoUEpBJ5RQnWs3j6o7RnHC99xywPQGp0SwvWVcqYnBP2kozfm71HY90+uRFPexfOifRAJTTk= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 15907014163941009.6136492890777; Thu, 28 May 2020 14:30:16 -0700 (PDT) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jeQ67-0000ZI-15; Thu, 28 May 2020 21:29:55 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jeQ66-0000Z6-Cb for xen-devel@lists.xenproject.org; Thu, 28 May 2020 21:29:54 +0000 Received: from mail-wr1-f66.google.com (unknown [209.85.221.66]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 5dfc8d56-a12a-11ea-a83e-12813bfff9fa; Thu, 28 May 2020 21:29:53 +0000 (UTC) Received: by mail-wr1-f66.google.com with SMTP id y17so866668wrn.11 for ; Thu, 28 May 2020 14:29:53 -0700 (PDT) Received: from [192.168.0.36] (87.78.186.89.cust.ip.kpnqwest.it. [89.186.78.87]) by smtp.gmail.com with ESMTPSA id y207sm9318196wmd.7.2020.05.28.14.29.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 28 May 2020 14:29:51 -0700 (PDT) X-Inumbo-ID: 5dfc8d56-a12a-11ea-a83e-12813bfff9fa X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:from:to:cc:date:message-id:in-reply-to :references:user-agent:mime-version:content-transfer-encoding; bh=rQ7Tl9PBNxTBxCYn4Y9mVqk3mlMzPIyms5hZ/Bt2tmY=; b=XW2tBMNn0zNlfZv58/8SgZLlaHX5/d8REukdrsubBAeSTWiMX8u1mwBaJd/bUJDBnK ONXW6BJD1F59FfL49ynBEeT/yYCZnFhxoaU+hPfBqgmCV9AfieFljzfytoLz1SgpUA7M ZdYJBmxFMg+PGQKSY2gqjuRbbWxpi2xS/6LJ+P92MigyJu6BK6cC8Imx+5KBfHk6TbaV RX7Dcy7Sx6miCVHYDPvOSP4fSR9sJI1Nhd4YWRTugGq483Syb4HUoDfqggWhE76aRSvg TMB/LAlGI1cR9ezQtmks34jumw0N8sIkLgf64ZaO9Fgdb28wa3DXtE3sku1icgpEqlsb m7Cw== X-Gm-Message-State: AOAM5330f5NY8sR2I3i6Dii3XSJLv1n1EmUmmCEzxnJpsKJh0Scoptmz 8Dtb/edOd9gFRUBCHUZyQUM= X-Google-Smtp-Source: ABdhPJy6mBC6m3+RplBD4/x+yE9izwjhW1w9Ukki0KWqb83MAezO3fTKYvF3932his/v/tC6w+pq8w== X-Received: by 2002:adf:a3c9:: with SMTP id m9mr5312588wrb.405.1590701392123; Thu, 28 May 2020 14:29:52 -0700 (PDT) Subject: [PATCH v2 5/7] xen: credit2: compute cpus per-runqueue more dynamically. From: Dario Faggioli To: xen-devel@lists.xenproject.org Date: Thu, 28 May 2020 23:29:50 +0200 Message-ID: <159070139062.12060.9216996770730278147.stgit@Palanthas> In-Reply-To: <159070133878.12060.13318432301910522647.stgit@Palanthas> References: <159070133878.12060.13318432301910522647.stgit@Palanthas> User-Agent: StGit/0.21 MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Juergen Gross , George Dunlap Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" During boot, we use num_online_cpus() as an indication of how many CPUs will end up in cpupool 0. We then decide (basing also on the value of the boot time parameter opt_max_cpus_runqueue) the actual number of CPUs that we want in each runqueue, in such a way that the runqueue themselves are as balanced (in therms of how many CPUs they have) as much as possible. After boot, though, when for instance we are creating a cpupool, it would be more appropriate to use the number of CPUs of the pool, rather than the total number of online CPUs. Do exactly that, even if this means (since from Xen's perspective CPUs are added to pools one by one) we'll be computing a different maximum number of CPUs per runqueue at each time. In fact, we do it in preparation for the next change where, after having computed the new value, we will also re-balance the runqueues, by rebuilding them in such a way that the newly computed maximum is actually respected for all of them. Signed-off-by: Dario Faggioli --- Cc: George Dunlap Cc: Juergen Gross --- Changes from v1: * new patch --- xen/common/sched/credit2.c | 30 +++++++++++++++++++++++++----- 1 file changed, 25 insertions(+), 5 deletions(-) diff --git a/xen/common/sched/credit2.c b/xen/common/sched/credit2.c index f4d3f8ae6b..af6d374677 100644 --- a/xen/common/sched/credit2.c +++ b/xen/common/sched/credit2.c @@ -922,7 +922,8 @@ cpu_runqueue_siblings_match(const struct csched2_runque= ue_data *rqd, } =20 static struct csched2_runqueue_data * -cpu_add_to_runqueue(const struct scheduler *ops, unsigned int cpu) +cpu_add_to_runqueue(const struct scheduler *ops, unsigned int nr_cpus, + unsigned int cpu) { struct csched2_private *prv =3D csched2_priv(ops); struct csched2_runqueue_data *rqd, *rqd_new; @@ -943,8 +944,8 @@ cpu_add_to_runqueue(const struct scheduler *ops, unsign= ed int cpu) * how many CPUs we have, so let's use the number of CPUs that are onl= ine * for that. */ - min_rqs =3D ((num_online_cpus() - 1) / opt_max_cpus_runqueue) + 1; - max_cpus_runq =3D num_online_cpus() / min_rqs; + min_rqs =3D ((nr_cpus - 1) / opt_max_cpus_runqueue) + 1; + max_cpus_runq =3D nr_cpus / min_rqs; =20 write_lock_irqsave(&prv->lock, flags); =20 @@ -3781,8 +3782,10 @@ csched2_dump(const struct scheduler *ops) */ read_lock_irqsave(&prv->lock, flags); =20 - printk("Active queues: %d\n" + printk("Active CPUs: %u\n" + "Active queues: %u\n" "\tdefault-weight =3D %d\n", + cpumask_weight(&prv->initialized), prv->active_queues, CSCHED2_DEFAULT_WEIGHT); list_for_each_entry ( rqd, &prv->rql, rql ) @@ -3879,6 +3882,7 @@ csched2_alloc_pdata(const struct scheduler *ops, int = cpu) { struct csched2_pcpu *spc; struct csched2_runqueue_data *rqd; + unsigned int nr_cpus; =20 BUG_ON(cpu_to_socket(cpu) =3D=3D XEN_INVALID_SOCKET_ID); =20 @@ -3886,7 +3890,23 @@ csched2_alloc_pdata(const struct scheduler *ops, int= cpu) if ( spc =3D=3D NULL ) return ERR_PTR(-ENOMEM); =20 - rqd =3D cpu_add_to_runqueue(ops, cpu); + /* + * If the system is booting, we know that, at this point, num_online_c= pus() + * CPUs have been brought up, and will be added to the default cpupool= and + * hence to this scheduler. This is valuable information that we can u= se + * to build the runqueues in an already balanced state. + * + * On the other hand, when we are live, and e.g., are creating a new + * cpupool, or adding CPUs to an already existing one, we have no way= to + * know in advance, from here, how many CPUs it will have. Therefore, = in + * that case, we just use the current number of CPUs that the pool has, + * plus 1, because we are in the process of adding it, for the balanci= ng. + * This will likely provide suboptimal results, and we rely on dynamic + * runqueue rebalancing for fixing it up. + */ + nr_cpus =3D system_state < SYS_STATE_active ? num_online_cpus() : + cpumask_weight(&csched2_priv(ops)->initialized) + 1; + rqd =3D cpu_add_to_runqueue(ops, nr_cpus, cpu); if ( IS_ERR(rqd) ) { xfree(spc); From nobody Wed May 15 20:19:40 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org ARC-Seal: i=1; a=rsa-sha256; t=1590701424; cv=none; d=zohomail.com; s=zohoarc; b=lRvmhokWxjf6ZmUxitxRUJkI/y2ek6ErcrlVmbzn+WfzNM1zKNJ1V1NfKj6oIBhWu8F23iAi/dSPA82tJl/WclVGewtTND2slbKso1JyJX48Udg5LOEzFmuwO5Kk8JMG2ThAQpQP9j3p3d3qjLCgKi+O7CdGhrJLn+FeIR5m6is= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1590701424; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=fW30aG1W0jN7We9433N5nJ1Y6xjm4AOXMc+J0AYDRT0=; b=fYq+WNsMlUCJvdW/GjRK50RCTWoziUPC1kHC1QTBsOJ7wOAE/2GIKff0x42hzWYSyZmAOqoEtxq7lBvGHRe19wkBMORCX8OfhPmByeAHKPeJnymqa9CZJY8/QLY2ha1AkSxklsQrVVfjuGzAs/xVOwOa7xN9YJfNcQN6Z5Tv1vk= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1590701424061831.7205521691385; Thu, 28 May 2020 14:30:24 -0700 (PDT) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jeQ6E-0000kp-F6; Thu, 28 May 2020 21:30:02 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jeQ6C-0000c9-Nn for xen-devel@lists.xenproject.org; Thu, 28 May 2020 21:30:00 +0000 Received: from mail-wr1-f65.google.com (unknown [209.85.221.65]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 62131932-a12a-11ea-a83e-12813bfff9fa; Thu, 28 May 2020 21:29:59 +0000 (UTC) Received: by mail-wr1-f65.google.com with SMTP id y17so867126wrn.11 for ; Thu, 28 May 2020 14:29:59 -0700 (PDT) Received: from [192.168.0.36] (87.78.186.89.cust.ip.kpnqwest.it. [89.186.78.87]) by smtp.gmail.com with ESMTPSA id f128sm7845886wme.1.2020.05.28.14.29.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 28 May 2020 14:29:58 -0700 (PDT) X-Inumbo-ID: 62131932-a12a-11ea-a83e-12813bfff9fa X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:from:to:cc:date:message-id:in-reply-to :references:user-agent:mime-version:content-transfer-encoding; bh=fW30aG1W0jN7We9433N5nJ1Y6xjm4AOXMc+J0AYDRT0=; b=aoPYK8Y4idezJX48/tvAOnk18aUtbwgpwMpW4hadoxewRPhjwq99EclFNHu8AHAyht sS3mIoNxwC11a9NCorGXMGEnfTN2Gas/V4+xZUVkF+j7iMurBwGLzvEAQcytZSwhXoYS UrfasN4TsZZCfHjpycQGFSn8+mlb7sGPOUvLDWABndGRotgtsk4pEWJFTjO1v1yzhnnJ mOWpp/4Bp7VkSuGCPGp482HX2uTW7B0spszRDEw9WPI4bEhCiU5sTyT3sHnqQcu1OPpf 0VVM6X14AgOKSWW1Sdd+rK7Cj+cFAYBnDwPA5bxEziEQNLuKCOosfflM7+uYUTEPiRd/ svMA== X-Gm-Message-State: AOAM531SPO9iHSzbA5cFFk3dLleab6jRUEfDgmhX3X6tSWRvboRPlYFB m2HMrTNOHvt2g/YUNOOAWGE= X-Google-Smtp-Source: ABdhPJzwKIRpWKuB1+AU4ejvi/dOi5O1e15rqE5i9fKNhqi+asxrnpNPwSxZ+Hur5VKKNbo9+/1JVQ== X-Received: by 2002:adf:8b0c:: with SMTP id n12mr5861182wra.340.1590701398903; Thu, 28 May 2020 14:29:58 -0700 (PDT) Subject: [PATCH v2 6/7] cpupool: create an the 'cpupool sync' infrastructure From: Dario Faggioli To: xen-devel@lists.xenproject.org Date: Thu, 28 May 2020 23:29:57 +0200 Message-ID: <159070139727.12060.7434914618426479787.stgit@Palanthas> In-Reply-To: <159070133878.12060.13318432301910522647.stgit@Palanthas> References: <159070133878.12060.13318432301910522647.stgit@Palanthas> User-Agent: StGit/0.21 MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Juergen Gross , Stefano Stabellini , Julien Grall , Andrew Cooper , George Dunlap , Jan Beulich Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" In case we want to make some live changes to the configuration of (typically) the scheduler of a cpupool, we need things to be quiet in that pool. Not necessarily like with stop machine, but we at least need to make sure that no domains are neither running not sitting in the runqueues of the scheduler itself. In fact, we need exactly something like this mechanism, for changing "on the fly" which CPUs are assigned to which runqueue in a Credit2 cpupool (check the following changes). Therefore, instead than doing something specific for such a use case, let's implement a generic mechanism. Reason is, of course, that it may turn out to be useful for other purposes, in future. But even for this specific case, it is much easier and cleaner to just cede control to cpupool code, instead of trying to do everything inside the scheduler. Within the new cpupool_sync() function, we want to pause all domains of a pool, including potentially the one calling the function. Therefore, we defer the pausing, the actual work and also the unpausing to a tasklet. Suggested-by: Juergen Gross Signed-off-by: Dario Faggioli --- Cc: Juergen Gross Cc: George Dunlap Cc: Andrew Cooper Cc: Jan Beulich Cc: Julien Grall Cc: Stefano Stabellini --- Changes from v1: * new patch --- xen/common/sched/cpupool.c | 52 ++++++++++++++++++++++++++++++++++++++++= ++++ xen/common/sched/private.h | 6 +++++ xen/include/xen/sched.h | 1 + 3 files changed, 59 insertions(+) diff --git a/xen/common/sched/cpupool.c b/xen/common/sched/cpupool.c index 7ea641ca26..122c371c7a 100644 --- a/xen/common/sched/cpupool.c +++ b/xen/common/sched/cpupool.c @@ -234,6 +234,42 @@ void cpupool_put(struct cpupool *pool) free_cpupool_struct(pool); } =20 +void do_cpupool_sync(void *arg) +{ + struct cpupool *c =3D arg; + struct domain *d; + + + spin_lock(&cpupool_lock); + + /* + * With this second call (and this time to domain_pause()) we basically + * make sure that all the domains have actually stopped running. + */ + rcu_read_lock(&domlist_read_lock); + for_each_domain_in_cpupool(d, c) + domain_pause(d); + rcu_read_unlock(&domlist_read_lock); + + /* + * Let's invoke the function that the caller provided. We pass a refer= ence + * to our own scheduler as a parameter, with which it should easily re= ach + * anything it needs. + */ + c->sync_ctl.func(c->sched); + + /* We called pause twice, so we need to to the same with unpause. */ + rcu_read_lock(&domlist_read_lock); + for_each_domain_in_cpupool(d, c) + { + domain_unpause(d); + domain_unpause(d); + } + rcu_read_unlock(&domlist_read_lock); + + spin_unlock(&cpupool_lock); +} + /* * create a new cpupool with specified poolid and scheduler * returns pointer to new cpupool structure if okay, NULL else @@ -292,6 +328,8 @@ static struct cpupool *cpupool_create( =20 *q =3D c; =20 + tasklet_init(&c->sync_ctl.tasklet, do_cpupool_sync, c); + spin_unlock(&cpupool_lock); =20 debugtrace_printk("Created cpupool %d with scheduler %s (%s)\n", @@ -332,6 +370,7 @@ static int cpupool_destroy(struct cpupool *c) return -EBUSY; } *q =3D c->next; + tasklet_kill(&c->sync_ctl.tasklet); spin_unlock(&cpupool_lock); =20 cpupool_put(c); @@ -372,6 +411,19 @@ int cpupool_move_domain(struct domain *d, struct cpupo= ol *c) return ret; } =20 +void cpupool_sync(struct cpupool *c, void (*func)(void*)) +{ + struct domain *d; + + rcu_read_lock(&domlist_read_lock); + for_each_domain_in_cpupool(d, c) + domain_pause_nosync(d); + rcu_read_unlock(&domlist_read_lock); + + c->sync_ctl.func =3D func; + tasklet_schedule_on_cpu(&c->sync_ctl.tasklet, cpumask_first(c->cpu_val= id)); +} + /* * assign a specific cpu to a cpupool * cpupool_lock must be held diff --git a/xen/common/sched/private.h b/xen/common/sched/private.h index df50976eb2..4705c8b119 100644 --- a/xen/common/sched/private.h +++ b/xen/common/sched/private.h @@ -503,6 +503,11 @@ static inline void sched_unit_unpause(const struct sch= ed_unit *unit) #define REGISTER_SCHEDULER(x) static const struct scheduler *x##_entry \ __used_section(".data.schedulers") =3D &x; =20 +struct cpupool_sync_ctl { + struct tasklet tasklet; + void (*func)(void*); +}; + struct cpupool { int cpupool_id; @@ -514,6 +519,7 @@ struct cpupool struct scheduler *sched; atomic_t refcnt; enum sched_gran gran; + struct cpupool_sync_ctl sync_ctl; }; =20 static inline cpumask_t *cpupool_domain_master_cpumask(const struct domain= *d) diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h index ac53519d7f..e2a233c96c 100644 --- a/xen/include/xen/sched.h +++ b/xen/include/xen/sched.h @@ -1061,6 +1061,7 @@ extern enum cpufreq_controller { } cpufreq_controller; =20 int cpupool_move_domain(struct domain *d, struct cpupool *c); +void cpupool_sync(struct cpupool *c, void (*func)(void*)); int cpupool_do_sysctl(struct xen_sysctl_cpupool_op *op); int cpupool_get_id(const struct domain *d); const cpumask_t *cpupool_valid_cpus(const struct cpupool *pool); From nobody Wed May 15 20:19:40 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org ARC-Seal: i=1; a=rsa-sha256; t=1590701433; cv=none; d=zohomail.com; s=zohoarc; b=kkGGqFH5nx9Pu9MxvDGk+z3lSlfgxLqmAaUbzkB8csFgBofSnpI5ZM1AxjLDZagfi+nbbo4KTcZxhOIuZ3hDmsuXtbnbprScLkrNGl0N+yD9yBGwj6E+SyoiL3GGAJSBrTm6e2hC/INWE/HfaLNu0IqDe1silrS4NUgP29DfZF4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1590701433; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=yM9i/il81pQeM/q8E3fSLqcs+S42XWd8SHRMmUQBSRY=; b=nl3Wa39qT2M9JgD4H8QfTM/O4yRU3ollfW7CzopXlPydiGIM4/HgM9nnHpLTtpN7Cxr1ZldRUJzk/RVODPsZpf3EbtEjiijFEDLr3tRZO7JOMYT1OGcwy9A5wcDH09k9ec+v1rqzbv0Y5+sSdOIW/3FU4kGyEc911Li2/KlGnQU= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1590701433866349.4351687687954; Thu, 28 May 2020 14:30:33 -0700 (PDT) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jeQ6K-0001KY-RJ; Thu, 28 May 2020 21:30:08 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jeQ6J-0001GE-NN for xen-devel@lists.xenproject.org; Thu, 28 May 2020 21:30:07 +0000 Received: from mail-wr1-f67.google.com (unknown [209.85.221.67]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 661e1d24-a12a-11ea-a83e-12813bfff9fa; Thu, 28 May 2020 21:30:06 +0000 (UTC) Received: by mail-wr1-f67.google.com with SMTP id x13so942679wrv.4 for ; Thu, 28 May 2020 14:30:06 -0700 (PDT) Received: from [192.168.0.36] (87.78.186.89.cust.ip.kpnqwest.it. [89.186.78.87]) by smtp.gmail.com with ESMTPSA id l19sm7717544wmj.14.2020.05.28.14.30.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 28 May 2020 14:30:04 -0700 (PDT) X-Inumbo-ID: 661e1d24-a12a-11ea-a83e-12813bfff9fa X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:from:to:cc:date:message-id:in-reply-to :references:user-agent:mime-version:content-transfer-encoding; bh=yM9i/il81pQeM/q8E3fSLqcs+S42XWd8SHRMmUQBSRY=; b=lCwj9cddSm2qkutmRcrGljB4ZLt+V1qujCnsa56Pi+xWNUTCJGnTnW0O+IrayXXWz6 ZxEHnvuySMVJPftn7Q6sINk979BicwX3lM553TGOlW2+kQODjl3t4ap8teVHwTztujOK SQ9XyuI9BD9ge9DYt0FR8R7+SPYAW4ph9xXRUYwAwoRaXegG+vkRZ5mzwB57fac4GKR/ Ufnr8qQNXmXsTneSZeDoG4+WX7ixTFbLFpt79+kIC4jj5KQJpVP3YUsF2sTBSbCjrSPQ J5Ba6WVb08qqiSs8NwEVwD0YIlcs6MNaFZS4VXxZSf817m7OytKt5HJHhfbABUogGyM6 AWvA== X-Gm-Message-State: AOAM530GxwZTbDbIIButsZgwA7SfWZx7ZC7K1v6jo1m/ld7cC0n52XZX F063Z9TGWITRixycolCciVk= X-Google-Smtp-Source: ABdhPJzRpciuBc1EO5nViCEa20e7q+SNxn2KdzPCdCSiQCg93fz7vpi2Hvt1MwpVn8NPee/t3YxoGg== X-Received: by 2002:adf:9ccf:: with SMTP id h15mr5428672wre.275.1590701405541; Thu, 28 May 2020 14:30:05 -0700 (PDT) Subject: [PATCH v2 7/7] xen: credit2: rebalance the number of CPUs in the scheduler runqueues From: Dario Faggioli To: xen-devel@lists.xenproject.org Date: Thu, 28 May 2020 23:30:04 +0200 Message-ID: <159070140400.12060.4466204111704460801.stgit@Palanthas> In-Reply-To: <159070133878.12060.13318432301910522647.stgit@Palanthas> References: <159070133878.12060.13318432301910522647.stgit@Palanthas> User-Agent: StGit/0.21 MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Juergen Gross , George Dunlap Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" When adding and removing CPUs to/from a pool, we can end up in a situation where some runqueues have a lot of CPUs, while other have only a couple of them. Even if the scheduler (namely, the load balancer) should be capable of dealing with such a situation, it is something that is better avoided. We now have all the pieces in place to attempt an actual re-balancement of the Credit2 scheduler runqueues, so let's go for it. In short: - every time we add or remove a CPU, especially considering the topology implications (e.g., we may have removed the last HT from a queue, so now there's space there for two CPUs, etc), we try to rebalance; - rebalancing happens under the control of the cpupool_sync() mechanism. Basically, it happens from inside a tasklet, after having put the cpupool in a quiescent state; - the new runqueue configuration may end up being both different, but also identical to the current one. It would be good to have a way to check whether the result would be identical, and in which case skip the balancing, but there is no way to do that. Rebalancing, since it pauses all the domain of a pool, etc, is a time consuming operation. But it only happens when the cpupool configuration is changed, so it is considered acceptable. Signed-off-by: Dario Faggioli --- Cc: George Dunlap Cc: Juergen Gross --- Changes from v1: * new patch --- xen/common/sched/credit2.c | 208 ++++++++++++++++++++++++++++++++++++++++= ++++ 1 file changed, 207 insertions(+), 1 deletion(-) diff --git a/xen/common/sched/credit2.c b/xen/common/sched/credit2.c index af6d374677..72d1961b1b 100644 --- a/xen/common/sched/credit2.c +++ b/xen/common/sched/credit2.c @@ -3983,6 +3983,194 @@ init_pdata(struct csched2_private *prv, struct csch= ed2_pcpu *spc, return rqd; } =20 +/* + * Let's get the hard work of rebalancing runqueues done. + * + * This function is called from a tasklet, spawned by cpupool_sync(). + * We run in idle vcpu context and we can assume that all the vcpus of all + * the domains within this cpupool are stopped... So we are relatively free + * to manipulate the scheduler's runqueues. + */ +static void do_rebalance_runqueues(void *arg) +{ + const struct scheduler *ops =3D arg; + struct csched2_private *prv =3D csched2_priv(ops); + struct csched2_runqueue_data *rqd, *rqd_temp; + struct csched2_unit *csu, *csu_temp; + struct list_head rq_list, csu_list; + spinlock_t temp_lock; + unsigned long flags; + unsigned int cpu; + + INIT_LIST_HEAD(&rq_list); + INIT_LIST_HEAD(&csu_list); + spin_lock_init(&temp_lock); + + /* + * This is where we will temporarily re-route the locks of all the CPU= s, + * while we take them outside of their current runqueue, and before ad= ding + * them to their new ones. Let's just take it right away, so any sort = of + * scheduling activity in any of them will stop at it, and won't race = with + * us. + */ + spin_lock_irq(&temp_lock); + + /* + * Everyone is paused, so we don't have any unit in any runqueue. Stil= l, + * units are "assigned" each one to a runqueue, for debug dumps and for + * calculating and tracking the weights. Since the current runqueues a= re + * going away, we need to deassign everyone from its runqueue. We will + * put all of them back into one of the new runqueue, before the end. + */ + write_lock(&prv->lock); + list_for_each_entry_safe ( rqd, rqd_temp, &prv->rql, rql ) + { + spin_lock(&rqd->lock); + /* + * We're deassigning the units, but we don't want to loose track + * of them... Otherwise how do we do the re-assignment to the new + * runqueues? So, let's stash them in a list. + */ + list_for_each_entry_safe ( csu, csu_temp, &rqd->svc, rqd_elem ) + { + runq_deassign(csu->unit); + list_add(&csu->rqd_elem, &csu_list); + } + + /* + * Now we want to prepare for getting rid of the runqueues as well. + * Each CPU has a pointer to the scheduler lock, which in case of + * Credit2 is the runqueue lock of the runqueue where the CPU is. + * But again, runqueues are vanishing, so let's re-route all such + * locks to our safe temporary solution that we introduced above. + */ + for_each_cpu ( cpu, &rqd->active ) + get_sched_res(cpu)->schedule_lock =3D &temp_lock; + spin_unlock(&rqd->lock); + + /* + * And, finally, "dequeue the runqueues", one by one. Similarly to + * what we do with units, we need to park them in a temporary list. + * In this case, they are actually going away, but we need to do t= his + * because we can't free() them with IRQs disabled. + */ + prv->active_queues--; + list_del(&rqd->rql); + list_add(&rqd->rql, &rq_list); + } + ASSERT(prv->active_queues =3D=3D 0); + write_unlock(&prv->lock); + + spin_unlock_irq(&temp_lock); + + /* + * Since we have to drop the lock anyway (in order to be able to call + * cpu_add_to_runqueue() below), let's also get rid of the old runqueu= es, + * now that we can. + */ + list_for_each_entry_safe ( rqd, rqd_temp, &rq_list, rql ) + { + list_del(&rqd->rql); + xfree(rqd); + } + rqd =3D NULL; + + /* + * We've got no lock! Well, this is still fine as: + * - the CPUs may, for some reason, try to schedule, and manage to do = so, + * taking turns on our global temporary spinlock. But there should be + * nothing to schedule; + * - we are safe from more cpupool manipulations as cpupool_sync() owns + * the cpupool_lock. + */ + + /* + * Now, for each CPU, we have to put them back in a runqueue. Of cours= e, + * we have no runqueue any longer, so they'll be re-created. We basica= lly + * follow pretty much the exact same path of when we add a CPU to a po= ol. + */ + for_each_cpu ( cpu, &prv->initialized ) + { + struct csched2_pcpu *spc =3D csched2_pcpu(cpu); + + /* + * The new runqueues need to be allocated, and cpu_add_to_runqueue= () + * takes care of that. We are, however, in a very delicate state, = as + * we have destroyed all the previous runqueues. I.e., if an error + * (e.g., not enough memory) occurs here, there is no way we can + * go back to a sane previous state, so let's just crash. + * + * Note that, at this time, the number of CPUs we have in the + * "initialized" mask represents how many CPUs we have in this poo= l. + * So we can use it, for computing the balance, basically in the s= ame + * way as we use num_online_cpu() during boot time.=20 + */ + rqd =3D cpu_add_to_runqueue(ops, cpumask_weight(&prv->initialized)= , cpu); + if ( IS_ERR(rqd) ) + { + printk(XENLOG_ERR " Major problems while rebalancing the runqu= eues!\n"); + BUG(); + } + spc->rqd =3D rqd; + + spin_lock_irq(&temp_lock); + write_lock(&prv->lock); + + init_cpu_runqueue(prv, spc, cpu, rqd); + + /* + * Bring the scheduler lock back to where it belongs, given the new + * runqueue, for the various CPUs. Barrier is there because we want + * all the runqueue initialization steps that we have made to be + * visible, exactly as it was for everyone that takes the lock + * (see the comment in common/sched/core.c:schedule_cpu_add() ). + */ + smp_wmb(); + get_sched_res(cpu)->schedule_lock =3D &rqd->lock; + + write_unlock(&prv->lock); + spin_unlock_irq(&temp_lock); + } + + /* + * And, finally, everything should be in place again. We can finalize = the + * work by adding back the units in the runqueues' lists (picking them + * up from the temporary list we used). Note that it is not necessary = to + * call csched2_res_pick(), for deciding on which runqueue to put each + * of them. Thins is: + * - with the old runqueue, each entity was associated to a + * sched_resource / CPU; + * - they where assigned to the runqueue in which such CPU was; + * - all the CPUs that were there, with the old runqueues, are still + * here, although in different runqueues; + * - we can just let the units be associated with the runqueues where + * theirs CPU has gone. + * + * This means that, even though the load was balanced, with the previ= ous + * runqueues, it now most likely now will not be. But this is not a b= ig + * deal as the load balancer will make things even again, given a lit= tle + * time. + */ + list_for_each_entry_safe ( csu, csu_temp, &csu_list, rqd_elem ) + { + spinlock_t *lock; + + lock =3D unit_schedule_lock_irqsave(csu->unit, &flags); + list_del_init(&csu->rqd_elem); + runq_assign(csu->unit); + unit_schedule_unlock_irqrestore(lock, flags, csu->unit); + } +} + +static void rebalance_runqueues(const struct scheduler *ops) +{ + struct cpupool *c =3D ops->cpupool; + + ASSERT(c->sched =3D=3D ops); + + cpupool_sync(c, do_rebalance_runqueues); +} + /* Change the scheduler of cpu to us (Credit2). */ static spinlock_t * csched2_switch_sched(struct scheduler *new_ops, unsigned int cpu, @@ -4017,6 +4205,16 @@ csched2_switch_sched(struct scheduler *new_ops, unsi= gned int cpu, */ ASSERT(get_sched_res(cpu)->schedule_lock !=3D &rqd->lock); =20 + /* + * We have added a CPU to the pool. Unless we are booting (in which + * case cpu_add_to_runqueue() balances the CPUs by itself) or we are in + * the very special case of still having fewer CPUs than how many we + * can put in just one runqueue... We need to try to rebalance. + */ + if ( system_state >=3D SYS_STATE_active && + cpumask_weight(&prv->initialized) > opt_max_cpus_runqueue ) + rebalance_runqueues(new_ops); + write_unlock(&prv->lock); =20 return &rqd->lock; @@ -4105,7 +4303,15 @@ csched2_free_pdata(const struct scheduler *ops, void= *pcpu, int cpu) else rqd =3D NULL; =20 - write_unlock_irqrestore(&prv->lock, flags); + /* + * Similarly to what said in csched2_switch_sched(), since we have just + * removed a CPU, it's good to check whether we can rebalance the + * runqueues. + */ + if ( cpumask_weight(&prv->initialized) >=3D opt_max_cpus_runqueue ) + rebalance_runqueues(ops); + + write_unlock_irqrestore(&prv->lock, flags); =20 xfree(rqd); xfree(pcpu);