From nobody Fri Apr 10 23:24:11 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 41637C25B08 for ; Thu, 18 Aug 2022 03:45:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S242886AbiHRDpt (ORCPT ); Wed, 17 Aug 2022 23:45:49 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39554 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240595AbiHRDpp (ORCPT ); Wed, 17 Aug 2022 23:45:45 -0400 Received: from mail-pj1-x1033.google.com (mail-pj1-x1033.google.com [IPv6:2607:f8b0:4864:20::1033]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A69799411C for ; Wed, 17 Aug 2022 20:45:44 -0700 (PDT) Received: by mail-pj1-x1033.google.com with SMTP id x63-20020a17090a6c4500b001fabbf8debfso612748pjj.4 for ; Wed, 17 Aug 2022 20:45:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=pWnT33mPHdMvZcJ8SNdBSZ87EcCdDL5ffg44dY+n5ac=; b=3606vPhRV9ZndngZeLSAK9BjgLHmT/nKkpd2u9wNdZ+/5EDn/sF0kaDWHDRhYvIJ5j VPRt7OTjAo8MNE+Lz4PIayUQvwccLb7p+VlxUVEaADeEzojR3p1LcDPRHrFtFGkyqhXZ 4ulgiAiz6FVB8Hbbjw/OJPr0v5g8LWQ4ZXhZEuN5SvXeizMsSZXApwYxYOaldWmF515i 0AUZmYtMpPMEHM90hYK1ecv4jDBVG3H2qF4Pd9UnLrrYqsapV/L7Ee1Od0/krJgwC+7j hXWxvMAu8k8WKSquppsBP4BV12Gi//bKDsfWs+F6NJH9HL+U+pxTRjeYCanEzeahv3lf pWAA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=pWnT33mPHdMvZcJ8SNdBSZ87EcCdDL5ffg44dY+n5ac=; b=K4iujJ8IHD6bvBavuswd3nszvo5fix+/flG1YedxQg16PNFU5WeZz1podwgOdwG0JH M7Y7I7gjobHhMdjOHyIqmNsLwLwZkC1AIEqaL7NnMtXi69RKvybXRLLopcMlo/WtSNON EzR6mP3BJkysqmnMFUSd42LAVC03fMLPKGwqZCNO5aQR+CBVFhKjCfzSsMb+TXckRSOJ wQ0U9+WGRNaO9Xr7vCrIlCZmNtM09VA0rf2g0vNC4up1Q+iB2mUBNa82pyWpP1/9PK/A A0UWeDY8ECuij07mxNmSkZkX+j5wk7VNqVjt75DZhm6OWUMGU52/s5frfuOjTqFqQlgo yWtQ== X-Gm-Message-State: ACgBeo3kl+HlA9wgTYHphPm8x1YDpBTsWtu8W+tUtb2eBGlKGaCP/JpU cVF8mCAodkn3MC6MjYM3JZ0Jkw== X-Google-Smtp-Source: AA6agR6xaurWKzuiWpVoUfPL93HJH+0sBR27OlFy+r7toqZKCp+h0t6x3O17oCX28HdA7Opdf9xQFg== X-Received: by 2002:a17:903:22c7:b0:171:516d:d2a4 with SMTP id y7-20020a17090322c700b00171516dd2a4mr965888plg.98.1660794344160; Wed, 17 Aug 2022 20:45:44 -0700 (PDT) Received: from C02CV1DAMD6P.bytedance.net ([139.177.225.246]) by smtp.gmail.com with ESMTPSA id m11-20020a65564b000000b00419acadde52sm268700pgs.46.2022.08.17.20.45.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 Aug 2022 20:45:43 -0700 (PDT) From: Chengming Zhou To: vincent.guittot@linaro.org, dietmar.eggemann@arm.com, mingo@redhat.com, peterz@infradead.org, rostedt@goodmis.org, bsegall@google.com, vschneid@redhat.com Cc: linux-kernel@vger.kernel.org, tj@kernel.org, Chengming Zhou Subject: [PATCH v5 1/9] sched/fair: maintain task se depth in set_task_rq() Date: Thu, 18 Aug 2022 11:43:35 +0800 Message-Id: <20220818034343.87625-2-zhouchengming@bytedance.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220818034343.87625-1-zhouchengming@bytedance.com> References: <20220818034343.87625-1-zhouchengming@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Previously we only maintain task se depth in task_move_group_fair(), if a !fair task change task group, its se depth will not be updated, so commit eb7a59b2c888 ("sched/fair: Reset se-depth when task switched to F= AIR") fix the problem by updating se depth in switched_to_fair() too. Then commit daa59407b558 ("sched/fair: Unify switched_{from,to}_fair() and task_move_group_fair()") unified these two functions, moved se.depth setting to attach_task_cfs_rq(), which further into attach_entity_cfs_rq() with commit df217913e72e ("sched/fair: Factorize attach/detach entity"). This patch move task se depth maintenance from attach_entity_cfs_rq() to set_task_rq(), which will be called when CPU/cgroup change, so its depth will always be correct. This patch is preparation for the next patch. Signed-off-by: Chengming Zhou Reviewed-by: Dietmar Eggemann Reviewed-by: Vincent Guittot --- kernel/sched/fair.c | 8 -------- kernel/sched/sched.h | 1 + 2 files changed, 1 insertion(+), 8 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index a71d6686149b..c5ee08b187ec 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -11726,14 +11726,6 @@ static void attach_entity_cfs_rq(struct sched_enti= ty *se) { struct cfs_rq *cfs_rq =3D cfs_rq_of(se); =20 -#ifdef CONFIG_FAIR_GROUP_SCHED - /* - * Since the real-depth could have been changed (only FAIR - * class maintain depth value), reset depth properly. - */ - se->depth =3D se->parent ? se->parent->depth + 1 : 0; -#endif - /* Synchronize entity with its cfs_rq */ update_load_avg(cfs_rq, se, sched_feat(ATTACH_AGE_LOAD) ? 0 : SKIP_AGE_LO= AD); attach_entity_load_avg(cfs_rq, se); diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index ddcfc7837595..628ffa974123 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -1932,6 +1932,7 @@ static inline void set_task_rq(struct task_struct *p,= unsigned int cpu) set_task_rq_fair(&p->se, p->se.cfs_rq, tg->cfs_rq[cpu]); p->se.cfs_rq =3D tg->cfs_rq[cpu]; p->se.parent =3D tg->se[cpu]; + p->se.depth =3D tg->se[cpu] ? tg->se[cpu]->depth + 1 : 0; #endif =20 #ifdef CONFIG_RT_GROUP_SCHED --=20 2.37.2 From nobody Fri Apr 10 23:24:11 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 942A4C25B08 for ; Thu, 18 Aug 2022 03:45:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S242985AbiHRDpy (ORCPT ); Wed, 17 Aug 2022 23:45:54 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39610 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240595AbiHRDpu (ORCPT ); Wed, 17 Aug 2022 23:45:50 -0400 Received: from mail-pj1-x1032.google.com (mail-pj1-x1032.google.com [IPv6:2607:f8b0:4864:20::1032]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B9A969411C for ; Wed, 17 Aug 2022 20:45:48 -0700 (PDT) Received: by mail-pj1-x1032.google.com with SMTP id pm17so515090pjb.3 for ; Wed, 17 Aug 2022 20:45:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=cq9WBgnay6MlRUXX21E7/GvZMXn3RD4DJtZwNfGfMLY=; b=fTH5AQyb70fbfjzlPmrKwqpWI2SN6Od0VKvQz6oIZ7Fg+Y3Ts8vMIYQ4VzD+NctwzY wFtEdbFJXZdB21cu+PUhjUGJHE00nc06JqyfJO2QY8LH1iuTmTZNls53hj+s8uOWSe6J zShxU+lDP1bDlf5Yvmq02EBGYehI0YNkQIuVebBAeiJu0QGP5bz0JrDa3fNNDpCHHQ1T W4HKzDeZdjN5wGPk85t6j6ikaKoFCV6CAXmWaJhTaVGZeKZnz7yMYPEyQ6/hBPB3v7VC 1HP9eIqDDKH+soDgezn+mhlwNttM00itCfF6LfVeJzuIf8ag0FhYvFeom1cA3XOL7Thj skvw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=cq9WBgnay6MlRUXX21E7/GvZMXn3RD4DJtZwNfGfMLY=; b=N8Aw+AblhwzI/C0kby7mbEamVG+f2mdBAsXqFJrzdaGcdNyQfyTc89+LZZovQT9vEf 8GB3TjUyl5zab147rl66/g6RnHE5sbo04V3042PYs+ej9p7jTe+SSPKmBCHjGE0yVfTg 42uSMhAIQsOFWRWOJIiyXylihENVPnBMna3mxa9mfipyAVBW3mJB2sLUozm0M5Agtp6f jYOfOidZF9OTxSEa3I2E3fPM8ObCoYMAHF9xEnjFb0ypF6EqZopYDsEnyiGdeAg5PNa1 6emqj8xGgR9dLEJEyCFYrNRXZjksNA6kolcSqTf65MvF8NTs29mjubtwPJH02HJCGwez Vqpg== X-Gm-Message-State: ACgBeo38KANKPquAzYfTkY66Vg/S7vSFkTLKdxrMJv4a4XDsvtbkSSU6 hCD/t4RF4vJpJBSXqqUGgm3qKA== X-Google-Smtp-Source: AA6agR78ZlGtoG4klkFQJzLwLDgdqVBEL6qnJTx9iF7Txr5I02iW14OUuhDG4Ztfzmc0rZ/hI98XLA== X-Received: by 2002:a17:90b:1b45:b0:1f5:753d:82bc with SMTP id nv5-20020a17090b1b4500b001f5753d82bcmr6778457pjb.68.1660794348305; Wed, 17 Aug 2022 20:45:48 -0700 (PDT) Received: from C02CV1DAMD6P.bytedance.net ([139.177.225.246]) by smtp.gmail.com with ESMTPSA id m11-20020a65564b000000b00419acadde52sm268700pgs.46.2022.08.17.20.45.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 Aug 2022 20:45:48 -0700 (PDT) From: Chengming Zhou To: vincent.guittot@linaro.org, dietmar.eggemann@arm.com, mingo@redhat.com, peterz@infradead.org, rostedt@goodmis.org, bsegall@google.com, vschneid@redhat.com Cc: linux-kernel@vger.kernel.org, tj@kernel.org, Chengming Zhou Subject: [PATCH v5 2/9] sched/fair: remove redundant cpu_cgrp_subsys->fork() Date: Thu, 18 Aug 2022 11:43:36 +0800 Message-Id: <20220818034343.87625-3-zhouchengming@bytedance.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220818034343.87625-1-zhouchengming@bytedance.com> References: <20220818034343.87625-1-zhouchengming@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" We use cpu_cgrp_subsys->fork() to set task group for the new fair task in cgroup_post_fork(). Since commit b1e8206582f9 ("sched: Fix yet more sched_fork() races") has already set_task_rq() for the new fair task in sched_cgroup_fork(), so cpu_cgrp_subsys->fork() can be removed. cgroup_can_fork() --> pin parent's sched_task_group sched_cgroup_fork() __set_task_cpu() set_task_rq() cgroup_post_fork() ss->fork() :=3D cpu_cgroup_fork() sched_change_group(..., TASK_SET_GROUP) task_set_group_fair() set_task_rq() --> can be removed After this patch's change, task_change_group_fair() only need to care about task cgroup migration, make the code much simplier. Signed-off-by: Chengming Zhou Reviewed-by: Vincent Guittot Reviewed-by: Dietmar Eggemann --- kernel/sched/core.c | 27 ++++----------------------- kernel/sched/fair.c | 23 +---------------------- kernel/sched/sched.h | 5 +---- 3 files changed, 6 insertions(+), 49 deletions(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 863b5203e357..8e3f1c3f0b2c 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -481,8 +481,7 @@ sched_core_dequeue(struct rq *rq, struct task_struct *p= , int flags) { } * p->se.load, p->rt_priority, * p->dl.dl_{runtime, deadline, period, flags, bw, density} * - sched_setnuma(): p->numa_preferred_nid - * - sched_move_task()/ - * cpu_cgroup_fork(): p->sched_task_group + * - sched_move_task(): p->sched_task_group * - uclamp_update_active() p->uclamp* * * p->state <- TASK_*: @@ -10166,7 +10165,7 @@ void sched_release_group(struct task_group *tg) spin_unlock_irqrestore(&task_group_lock, flags); } =20 -static void sched_change_group(struct task_struct *tsk, int type) +static void sched_change_group(struct task_struct *tsk) { struct task_group *tg; =20 @@ -10182,7 +10181,7 @@ static void sched_change_group(struct task_struct *= tsk, int type) =20 #ifdef CONFIG_FAIR_GROUP_SCHED if (tsk->sched_class->task_change_group) - tsk->sched_class->task_change_group(tsk, type); + tsk->sched_class->task_change_group(tsk); else #endif set_task_rq(tsk, task_cpu(tsk)); @@ -10213,7 +10212,7 @@ void sched_move_task(struct task_struct *tsk) if (running) put_prev_task(rq, tsk); =20 - sched_change_group(tsk, TASK_MOVE_GROUP); + sched_change_group(tsk); =20 if (queued) enqueue_task(rq, tsk, queue_flags); @@ -10291,23 +10290,6 @@ static void cpu_cgroup_css_free(struct cgroup_subs= ys_state *css) sched_unregister_group(tg); } =20 -/* - * This is called before wake_up_new_task(), therefore we really only - * have to set its group bits, all the other stuff does not apply. - */ -static void cpu_cgroup_fork(struct task_struct *task) -{ - struct rq_flags rf; - struct rq *rq; - - rq =3D task_rq_lock(task, &rf); - - update_rq_clock(rq); - sched_change_group(task, TASK_SET_GROUP); - - task_rq_unlock(rq, task, &rf); -} - static int cpu_cgroup_can_attach(struct cgroup_taskset *tset) { struct task_struct *task; @@ -11173,7 +11155,6 @@ struct cgroup_subsys cpu_cgrp_subsys =3D { .css_released =3D cpu_cgroup_css_released, .css_free =3D cpu_cgroup_css_free, .css_extra_stat_show =3D cpu_extra_stat_show, - .fork =3D cpu_cgroup_fork, .can_attach =3D cpu_cgroup_can_attach, .attach =3D cpu_cgroup_attach, .legacy_cftypes =3D cpu_legacy_files, diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index c5ee08b187ec..4b95599aa951 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -11821,15 +11821,7 @@ void init_cfs_rq(struct cfs_rq *cfs_rq) } =20 #ifdef CONFIG_FAIR_GROUP_SCHED -static void task_set_group_fair(struct task_struct *p) -{ - struct sched_entity *se =3D &p->se; - - set_task_rq(p, task_cpu(p)); - se->depth =3D se->parent ? se->parent->depth + 1 : 0; -} - -static void task_move_group_fair(struct task_struct *p) +static void task_change_group_fair(struct task_struct *p) { detach_task_cfs_rq(p); set_task_rq(p, task_cpu(p)); @@ -11841,19 +11833,6 @@ static void task_move_group_fair(struct task_struc= t *p) attach_task_cfs_rq(p); } =20 -static void task_change_group_fair(struct task_struct *p, int type) -{ - switch (type) { - case TASK_SET_GROUP: - task_set_group_fair(p); - break; - - case TASK_MOVE_GROUP: - task_move_group_fair(p); - break; - } -} - void free_fair_sched_group(struct task_group *tg) { int i; diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 628ffa974123..2db7b0494c19 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -2195,11 +2195,8 @@ struct sched_class { =20 void (*update_curr)(struct rq *rq); =20 -#define TASK_SET_GROUP 0 -#define TASK_MOVE_GROUP 1 - #ifdef CONFIG_FAIR_GROUP_SCHED - void (*task_change_group)(struct task_struct *p, int type); + void (*task_change_group)(struct task_struct *p); #endif }; =20 --=20 2.37.2 From nobody Fri Apr 10 23:24:11 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 63BF4C25B08 for ; Thu, 18 Aug 2022 03:46:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243120AbiHRDp7 (ORCPT ); Wed, 17 Aug 2022 23:45:59 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39658 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242973AbiHRDpx (ORCPT ); Wed, 17 Aug 2022 23:45:53 -0400 Received: from mail-pf1-x42a.google.com (mail-pf1-x42a.google.com [IPv6:2607:f8b0:4864:20::42a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D413594135 for ; Wed, 17 Aug 2022 20:45:52 -0700 (PDT) Received: by mail-pf1-x42a.google.com with SMTP id z187so412094pfb.12 for ; Wed, 17 Aug 2022 20:45:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=T9NjXKCirMmjalxT2TDdOclIHIt09Z9HaaeNsJZTPXw=; b=qE6DbIRbQ4pGOYMM7dQuG0ute6j4MLfn+kSU2L8olPgM9KcEPDsVwiKmJVC6kpvv8v p7VIzbkR9pYOUX+2uBUm3xWU3TEpz7o8+e0m2GCj8mPK8n2Prq8c3YiJewStyWBy7o7C ZaC5siCeTyy5vJYtAGdZpvy9/TdKemDwFty59NocSszDMomsa7ssbYMhA91G1CiGWVgh a+qHGjAqNHGKAYXK/Yqs8lsl8QzzFRIxDt5uYHhG/lda7oqEAlQO/Y4b5KWZAGyIstKG 9Ub7hRiHV5hGFKzjwwOBqqPahV3b88sMNSeF1Q5mYL+y6p/wvO7smwUwFRMGJSh54fGy 2oxA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=T9NjXKCirMmjalxT2TDdOclIHIt09Z9HaaeNsJZTPXw=; b=H9aITPowGvGlmI1i0bhhqDyk8TgYIXamFxO7j2E1AkwS8GQ9euTP3oFaXzeYUbLlmg TBUTe8iNGZ+L62rkdK+Qmh1X/XgzJMfLGx1j1LT2eUT/6PUQAqEGgLPGFLIWSEawIb81 aF0+Y2p2asm1llZJQa7Bk96QDHD5VGCtOBBan8WEo5OFrp0iCIpbCrtR517w8dmPXd2T CHDyD0aXQlCQ3RRMs9rfuojJScYoOFYQHW3T8sxbGEtYQWxSKRydRIdn9KlJ3lrbL0qq Mm1JXZvy1LlVOFF6Ql5F6UTR6kQw0LoITpBlAC6JXiOCe2GCDrTMf2G5yE5JC6tFAy9u CrvQ== X-Gm-Message-State: ACgBeo3yFHTIjXhh3EDKD6WrUDoWhc5ilxvT2i4K4X8e6FhADkJbeOGe V+oGpbIDoYqLEQFMV0WkvaHS1g== X-Google-Smtp-Source: AA6agR7FBHeX1XxF0Tmwrm8aBmDcSaAbYpuXymjV0tYaGcbiRAhsQeq38UhXnmRh2LL4yL7M/ypE2A== X-Received: by 2002:a05:6a00:1a88:b0:52f:52df:ce57 with SMTP id e8-20020a056a001a8800b0052f52dfce57mr1140692pfv.71.1660794352409; Wed, 17 Aug 2022 20:45:52 -0700 (PDT) Received: from C02CV1DAMD6P.bytedance.net ([139.177.225.246]) by smtp.gmail.com with ESMTPSA id m11-20020a65564b000000b00419acadde52sm268700pgs.46.2022.08.17.20.45.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 Aug 2022 20:45:52 -0700 (PDT) From: Chengming Zhou To: vincent.guittot@linaro.org, dietmar.eggemann@arm.com, mingo@redhat.com, peterz@infradead.org, rostedt@goodmis.org, bsegall@google.com, vschneid@redhat.com Cc: linux-kernel@vger.kernel.org, tj@kernel.org, Chengming Zhou Subject: [PATCH v5 3/9] sched/fair: reset sched_avg last_update_time before set_task_rq() Date: Thu, 18 Aug 2022 11:43:37 +0800 Message-Id: <20220818034343.87625-4-zhouchengming@bytedance.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220818034343.87625-1-zhouchengming@bytedance.com> References: <20220818034343.87625-1-zhouchengming@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" set_task_rq() -> set_task_rq_fair() will try to synchronize the blocked task's sched_avg when migrate, which is not needed for already detached task. task_change_group_fair() will detached the task sched_avg from prev cfs_rq first, so reset sched_avg last_update_time before set_task_rq() to avoid th= at. Signed-off-by: Chengming Zhou Reviewed-by: Dietmar Eggemann Reviewed-by: Vincent Guittot --- kernel/sched/fair.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 4b95599aa951..5a704109472a 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -11824,12 +11824,12 @@ void init_cfs_rq(struct cfs_rq *cfs_rq) static void task_change_group_fair(struct task_struct *p) { detach_task_cfs_rq(p); - set_task_rq(p, task_cpu(p)); =20 #ifdef CONFIG_SMP /* Tell se's cfs_rq has been changed -- migrated */ p->se.avg.last_update_time =3D 0; #endif + set_task_rq(p, task_cpu(p)); attach_task_cfs_rq(p); } =20 --=20 2.37.2 From nobody Fri Apr 10 23:24:11 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9FF49C25B08 for ; Thu, 18 Aug 2022 03:46:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S242766AbiHRDqN (ORCPT ); Wed, 17 Aug 2022 23:46:13 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39742 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243123AbiHRDp7 (ORCPT ); Wed, 17 Aug 2022 23:45:59 -0400 Received: from mail-pl1-x629.google.com (mail-pl1-x629.google.com [IPv6:2607:f8b0:4864:20::629]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2E90D95ACD for ; Wed, 17 Aug 2022 20:45:57 -0700 (PDT) Received: by mail-pl1-x629.google.com with SMTP id y4so499387plb.2 for ; Wed, 17 Aug 2022 20:45:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=a8l9uZA41LqnoJJEYr1pG3p/xrUXauS4MtvHWBuecMk=; b=SURwyuWzyL9AH+VFSNZaoSuz4Zso+OE14ff/JQ8jVkDGN2ToQ3+PTdEPMVGe+GiJN1 W4kLNGrbmyfOob9PmPOOSIVUNq1LB3NwtLqeDBRtRzUHlM73dvEETPS2h92Mio0/MgkX 03JyP+UnvLIDAQGvAS94Q3lFSN0/36MH0q4Av/rKWwCUHkigMgjVbcu1RM7U6TxACt0N 58Bg5VbeSDv9aNQB/lo6/e349Ma5mBzQ3KOgZxN5ezkkeWYtn97GNon0w/jC9wJQwVN+ DhL97Pg6vyV+fXuyzL63bEh/Kw9s+zgyk8ASQjZBH5iRCq3kOYbrzJg+/7Iyr8+/7rVa tk/A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=a8l9uZA41LqnoJJEYr1pG3p/xrUXauS4MtvHWBuecMk=; b=y4VLir7zk6sfJD8J2xMd/0cBTrWoMkSEeAW3hZxhx2sNCNnEvOhDEEmr+4nKaG0hEs HaA/LaEtDk/9hwU5PQxN3BgU5h1yWa+J4SlVwD2MpKGuWGlFhtWSUoKFH542cTO8chAJ gMNxH6iRaZcsQbNiuPli+6tXMfEuHrsFlQV6cY5945Mwv9rQh4kHz7nMc0Z2A6QnUUKt +OrwnIVafZLC3oU2l/2WKD9U7cVJZIwSqC40Qb+6pHLOKGP2NWLfvVXVTSXGWuQv6v3j MVMwmcTMVbbzNb47DcSNZ/Dm/dGGnZ9mIqmt/jarkeRMDg7tJkuLi+l/H5A4DM+QUjiS UggA== X-Gm-Message-State: ACgBeo1TO4dwLbtCWP243R1OWrmLwT2FfGffRsUrJ04XhRs+/qfyU1jo hpGk6DApyZJCHWlHJb+PgsFo5A== X-Google-Smtp-Source: AA6agR49fjIZvotLFo/f6v+l5KJjPgJUGB0WLtj6X1i53kYYPfBwKf31tfoss6tYtIpoWM9aGKLZNA== X-Received: by 2002:a17:90b:3c01:b0:1fa:b364:878 with SMTP id pb1-20020a17090b3c0100b001fab3640878mr6142193pjb.225.1660794356508; Wed, 17 Aug 2022 20:45:56 -0700 (PDT) Received: from C02CV1DAMD6P.bytedance.net ([139.177.225.246]) by smtp.gmail.com with ESMTPSA id m11-20020a65564b000000b00419acadde52sm268700pgs.46.2022.08.17.20.45.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 Aug 2022 20:45:56 -0700 (PDT) From: Chengming Zhou To: vincent.guittot@linaro.org, dietmar.eggemann@arm.com, mingo@redhat.com, peterz@infradead.org, rostedt@goodmis.org, bsegall@google.com, vschneid@redhat.com Cc: linux-kernel@vger.kernel.org, tj@kernel.org, Chengming Zhou Subject: [PATCH v5 4/9] sched/fair: update comments in enqueue/dequeue_entity() Date: Thu, 18 Aug 2022 11:43:38 +0800 Message-Id: <20220818034343.87625-5-zhouchengming@bytedance.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220818034343.87625-1-zhouchengming@bytedance.com> References: <20220818034343.87625-1-zhouchengming@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" When reading the sched_avg related code, I found the comments in enqueue/dequeue_entity() are not updated with the current code. We don't add/subtract entity's runnable_avg from cfs_rq->runnable_avg during enqueue/dequeue_entity(), those are done only for attach/detach. This patch updates the comments to reflect the current code working. Signed-off-by: Chengming Zhou Acked-by: Vincent Guittot --- kernel/sched/fair.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 5a704109472a..372e5f4a49a3 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -4598,7 +4598,8 @@ enqueue_entity(struct cfs_rq *cfs_rq, struct sched_en= tity *se, int flags) /* * When enqueuing a sched_entity, we must: * - Update loads to have both entity and cfs_rq synced with now. - * - Add its load to cfs_rq->runnable_avg + * - For group_entity, update its runnable_weight to reflect the new + * h_nr_running of its group cfs_rq. * - For group_entity, update its weight to reflect the new share of * its group cfs_rq * - Add its new weight to cfs_rq->load.weight @@ -4683,7 +4684,8 @@ dequeue_entity(struct cfs_rq *cfs_rq, struct sched_en= tity *se, int flags) /* * When dequeuing a sched_entity, we must: * - Update loads to have both entity and cfs_rq synced with now. - * - Subtract its load from the cfs_rq->runnable_avg. + * - For group_entity, update its runnable_weight to reflect the new + * h_nr_running of its group cfs_rq. * - Subtract its previous weight from cfs_rq->load.weight. * - For group entity, update its weight to reflect the new share * of its group cfs_rq. --=20 2.37.2 From nobody Fri Apr 10 23:24:11 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CCA0FC25B08 for ; Thu, 18 Aug 2022 03:46:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S242933AbiHRDqR (ORCPT ); Wed, 17 Aug 2022 23:46:17 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39736 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242997AbiHRDqC (ORCPT ); Wed, 17 Aug 2022 23:46:02 -0400 Received: from mail-pj1-x102d.google.com (mail-pj1-x102d.google.com [IPv6:2607:f8b0:4864:20::102d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1180D94EDE for ; Wed, 17 Aug 2022 20:46:01 -0700 (PDT) Received: by mail-pj1-x102d.google.com with SMTP id t22so530050pjy.1 for ; Wed, 17 Aug 2022 20:46:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=3GAWkeAWlGlQB12EQDsOtb30kGIL9nEjgqc5w218gmE=; b=za0V7ADzYm9BXhGN+6SwnM61n3dN4Wss1/9zVYm9HoPsQBNMXSAgiTgOXdXAz3BrCL khbA/Ot+QHjjDe4ONfowzyxSXJ7MFzdWC9Dd+dsPfK7lm87g9DaUnwm4vNSWbXuaTQVg Hb5nb1ajCZ3E1rCJD3kOGkeD+RUCPBgTRsjvXWrSzRdNz5Jl7hLbxWoXTB1/tDELEKlc 3WIlC/HAYhvS1Olhmh8WRpoQT6JP7sPGywTTupTwh0gAmAQn6qT6yQpM0mOyzmXo8qaL lhGZKmZyx+h4aK6MXZOoV6JmKptAf8WG8x1hlFkK6SaZUYfe1WA63h13ZCphSFyfDUOl QN7g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=3GAWkeAWlGlQB12EQDsOtb30kGIL9nEjgqc5w218gmE=; b=B3bVj/LCABytg+1SruBSUc8MJYuG3JzottSTrLnbRPnUpOqWcDUWAG+LRwtbwhDavF Nx5K8w8RPxGH9FVJXxwZMzSBn3ryBfpQVAu7IDfYafmsGJ64HOdgBhBLDPtcFoA0AoDU KsC0vozsiyOJuFdna5XgQcsaA+SpzgyldccH3G+yQjcuNjvyGsPsuERQQ05EV3P64mJF IVozHn1jIb3I3iXOQSwVzfb/3apm/0o5G7Rjs+6URJZMdUOmoChuxCy5hh19kB2yWZC4 3+FOil3YWmf0uedrnswpwkezDAYJ9ke9KGhJnsOUOgqtatucdCrERstqV0pQn7hLFLkv rthQ== X-Gm-Message-State: ACgBeo0RKq8SuySMZdhoF97c4aZaETFO45gDR+93GYz++ooPiic8CNxh ZJChq1gssW0naQP5/TR8lU47TQ== X-Google-Smtp-Source: AA6agR4nkry76eQ+iNfANpZ7+NVoGi3Ahs+/B7yLTUeEXU+Rj5/wVxy6Nn+8syayU31IgQM5+Fy0+Q== X-Received: by 2002:a17:90b:1b05:b0:1f7:3ac7:f044 with SMTP id nu5-20020a17090b1b0500b001f73ac7f044mr1081379pjb.214.1660794360598; Wed, 17 Aug 2022 20:46:00 -0700 (PDT) Received: from C02CV1DAMD6P.bytedance.net ([139.177.225.246]) by smtp.gmail.com with ESMTPSA id m11-20020a65564b000000b00419acadde52sm268700pgs.46.2022.08.17.20.45.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 Aug 2022 20:46:00 -0700 (PDT) From: Chengming Zhou To: vincent.guittot@linaro.org, dietmar.eggemann@arm.com, mingo@redhat.com, peterz@infradead.org, rostedt@goodmis.org, bsegall@google.com, vschneid@redhat.com Cc: linux-kernel@vger.kernel.org, tj@kernel.org, Chengming Zhou Subject: [PATCH v5 5/9] sched/fair: combine detach into dequeue when migrating task Date: Thu, 18 Aug 2022 11:43:39 +0800 Message-Id: <20220818034343.87625-6-zhouchengming@bytedance.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220818034343.87625-1-zhouchengming@bytedance.com> References: <20220818034343.87625-1-zhouchengming@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" When we are migrating task out of the CPU, we can combine detach and propagation into dequeue_entity() to save the detach_entity_cfs_rq() in migrate_task_rq_fair(). This optimization is like combining DO_ATTACH in the enqueue_entity() when migrating task to the CPU. So we don't have to traverse the CFS tree extra time to do the detach_entity_cfs_rq() -> propagate_entity_cfs_rq(), which wouldn't be called anymore with this patch's change. detach_task() deactivate_task() dequeue_task_fair() for_each_sched_entity(se) dequeue_entity() update_load_avg() /* (1) */ detach_entity_load_avg() set_task_cpu() migrate_task_rq_fair() detach_entity_cfs_rq() /* (2) */ update_load_avg(); detach_entity_load_avg(); propagate_entity_cfs_rq(); for_each_sched_entity() update_load_avg() This patch save the detach_entity_cfs_rq() called in (2) by doing the detach_entity_load_avg() for a CPU migrating task inside (1) (the task being the first se in the loop) Signed-off-by: Chengming Zhou Reviewed-by: Vincent Guittot --- kernel/sched/fair.c | 28 ++++++++++++++++------------ 1 file changed, 16 insertions(+), 12 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 372e5f4a49a3..1eb3fb3d95c3 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -4167,6 +4167,7 @@ static void detach_entity_load_avg(struct cfs_rq *cfs= _rq, struct sched_entity *s #define UPDATE_TG 0x1 #define SKIP_AGE_LOAD 0x2 #define DO_ATTACH 0x4 +#define DO_DETACH 0x8 =20 /* Update task and its cfs_rq load average */ static inline void update_load_avg(struct cfs_rq *cfs_rq, struct sched_ent= ity *se, int flags) @@ -4196,6 +4197,13 @@ static inline void update_load_avg(struct cfs_rq *cf= s_rq, struct sched_entity *s attach_entity_load_avg(cfs_rq, se); update_tg_load_avg(cfs_rq); =20 + } else if (flags & DO_DETACH) { + /* + * DO_DETACH means we're here from dequeue_entity() + * and we are migrating task out of the CPU. + */ + detach_entity_load_avg(cfs_rq, se); + update_tg_load_avg(cfs_rq); } else if (decayed) { cfs_rq_util_change(cfs_rq, 0); =20 @@ -4456,6 +4464,7 @@ static inline bool cfs_rq_is_decayed(struct cfs_rq *c= fs_rq) #define UPDATE_TG 0x0 #define SKIP_AGE_LOAD 0x0 #define DO_ATTACH 0x0 +#define DO_DETACH 0x0 =20 static inline void update_load_avg(struct cfs_rq *cfs_rq, struct sched_ent= ity *se, int not_used1) { @@ -4676,6 +4685,11 @@ static __always_inline void return_cfs_rq_runtime(st= ruct cfs_rq *cfs_rq); static void dequeue_entity(struct cfs_rq *cfs_rq, struct sched_entity *se, int flags) { + int action =3D UPDATE_TG; + + if (entity_is_task(se) && task_on_rq_migrating(task_of(se))) + action |=3D DO_DETACH; + /* * Update run-time statistics of the 'current'. */ @@ -4690,7 +4704,7 @@ dequeue_entity(struct cfs_rq *cfs_rq, struct sched_en= tity *se, int flags) * - For group entity, update its weight to reflect the new share * of its group cfs_rq. */ - update_load_avg(cfs_rq, se, UPDATE_TG); + update_load_avg(cfs_rq, se, action); se_update_runnable(se); =20 update_stats_dequeue_fair(cfs_rq, se, flags); @@ -7242,8 +7256,6 @@ select_task_rq_fair(struct task_struct *p, int prev_c= pu, int wake_flags) return new_cpu; } =20 -static void detach_entity_cfs_rq(struct sched_entity *se); - /* * Called immediately before a task is migrated to a new CPU; task_cpu(p) = and * cfs_rq_of(p) references at time of call are still valid and identify the @@ -7265,15 +7277,7 @@ static void migrate_task_rq_fair(struct task_struct = *p, int new_cpu) se->vruntime -=3D u64_u32_load(cfs_rq->min_vruntime); } =20 - if (p->on_rq =3D=3D TASK_ON_RQ_MIGRATING) { - /* - * In case of TASK_ON_RQ_MIGRATING we in fact hold the 'old' - * rq->lock and can modify state directly. - */ - lockdep_assert_rq_held(task_rq(p)); - detach_entity_cfs_rq(se); - - } else { + if (!task_on_rq_migrating(p)) { remove_entity_load_avg(se); =20 /* --=20 2.37.2 From nobody Fri Apr 10 23:24:11 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 57879C25B08 for ; Thu, 18 Aug 2022 03:46:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243144AbiHRDqV (ORCPT ); Wed, 17 Aug 2022 23:46:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39828 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243124AbiHRDqF (ORCPT ); Wed, 17 Aug 2022 23:46:05 -0400 Received: from mail-pf1-x433.google.com (mail-pf1-x433.google.com [IPv6:2607:f8b0:4864:20::433]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2C6E39568A for ; Wed, 17 Aug 2022 20:46:05 -0700 (PDT) Received: by mail-pf1-x433.google.com with SMTP id a22so442784pfg.3 for ; Wed, 17 Aug 2022 20:46:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=FRKOdv3Syw9Tzig7kJCUcxunGx7WyPi3SKqsZ5Hh+4Q=; b=V87Pq/x7RLiRWbmQoXapUXLwX2aHbsl5PbZmlqq7R5HsQeaaYCkvUXR7oyFbYRy8Kt Z0e/ktl/zUA+3WmA3k25M0w00BMBPcRLC2V45bjZxY16cbstSYt7xfyLiiMvst40/p/Z sroebfYZxmqdOB/8pDDufM3okd6aFxL0gFvRR2H5dEOAURLBAadIPV+PntgkeLUNDTJ8 NkkLKLjvmetKN6T9OczP9e4DncGGhLQGW/i32c9pHmRJNRaKHZY4KvayhJhkh3WxDerm hBx4t/g7NOFyYNC2SEtxD0aaW7Lp2TsqjvJLZROaKcmeuZ+xzNUq8+KJoaiQY+QoRtBO lFhA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=FRKOdv3Syw9Tzig7kJCUcxunGx7WyPi3SKqsZ5Hh+4Q=; b=pzElRE5/Plh/EiOXP4rzuv6Wl9SjdU+Wo4dbXYsjdV85hO86je3ISImzo3skui7qWl 52PWEpLl1/IYDzYFFKIC4OFkXaejaJJHa7lDhjSZclQB2tsJVOKmht3NGdxW1k/+DNmn FwRlNJ79mP268tcaqlNOh3lCRNjj17sQUm+M6VRwmcja4liGauG1JzxgV5KjtF7oxUgL 8qxJEUU5tvPnQIBpi1g6N9lgk7NZdTq9/fbemwpX59Zud/FtSJFFsu9UXpqerc+baHv4 tpsh1QIVUhBHcP2K7Zdivfg51J3eHv7UsayVuCwFkjkgBfTkESgaBAdl95t4xlsRL2f5 FrNQ== X-Gm-Message-State: ACgBeo1JFSHbc3fkXzL3HxBIDq2RHKC2A8MDxFVW1nmKfW1rTLKXWwPg cUyt9acWlNrHmr/4BU/l1LrEdA== X-Google-Smtp-Source: AA6agR789RTZFWiTps6quGQP+dP1RBB+CEQ8OsIqCoTcDoX50n2+/ErOqb/pVZIk/1UMukCSjFA09w== X-Received: by 2002:a63:d841:0:b0:427:e7f2:bef with SMTP id k1-20020a63d841000000b00427e7f20befmr1030819pgj.386.1660794364697; Wed, 17 Aug 2022 20:46:04 -0700 (PDT) Received: from C02CV1DAMD6P.bytedance.net ([139.177.225.246]) by smtp.gmail.com with ESMTPSA id m11-20020a65564b000000b00419acadde52sm268700pgs.46.2022.08.17.20.46.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 Aug 2022 20:46:04 -0700 (PDT) From: Chengming Zhou To: vincent.guittot@linaro.org, dietmar.eggemann@arm.com, mingo@redhat.com, peterz@infradead.org, rostedt@goodmis.org, bsegall@google.com, vschneid@redhat.com Cc: linux-kernel@vger.kernel.org, tj@kernel.org, Chengming Zhou Subject: [PATCH v5 6/9] sched/fair: fix another detach on unattached task corner case Date: Thu, 18 Aug 2022 11:43:40 +0800 Message-Id: <20220818034343.87625-7-zhouchengming@bytedance.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220818034343.87625-1-zhouchengming@bytedance.com> References: <20220818034343.87625-1-zhouchengming@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" commit 7dc603c9028e ("sched/fair: Fix PELT integrity for new tasks") fixed two load tracking problems for new task, including detach on unattached new task problem. There still left another detach on unattached task problem for the task which has been woken up by try_to_wake_up() and waiting for actually being woken up by sched_ttwu_pending(). try_to_wake_up(p) cpu =3D select_task_rq(p) if (task_cpu(p) !=3D cpu) set_task_cpu(p, cpu) migrate_task_rq_fair() remove_entity_load_avg() --> unattached se->avg.last_update_time =3D 0; __set_task_cpu() ttwu_queue(p, cpu) ttwu_queue_wakelist() __ttwu_queue_wakelist() task_change_group_fair() detach_task_cfs_rq() detach_entity_cfs_rq() detach_entity_load_avg() --> detach on unattached task set_task_rq() attach_task_cfs_rq() attach_entity_cfs_rq() attach_entity_load_avg() The reason of this problem is similar, we should check in detach_entity_cfs= _rq() that se->avg.last_update_time !=3D 0, before do detach_entity_load_avg(). Signed-off-by: Chengming Zhou --- kernel/sched/fair.c | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 1eb3fb3d95c3..eba8a64f905a 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -11721,6 +11721,17 @@ static void detach_entity_cfs_rq(struct sched_enti= ty *se) { struct cfs_rq *cfs_rq =3D cfs_rq_of(se); =20 +#ifdef CONFIG_SMP + /* + * In case the task sched_avg hasn't been attached: + * - A forked task which hasn't been woken up by wake_up_new_task(). + * - A task which has been woken up by try_to_wake_up() but is + * waiting for actually being woken up by sched_ttwu_pending(). + */ + if (!se->avg.last_update_time) + return; +#endif + /* Catch up with the cfs_rq and remove our load when we leave */ update_load_avg(cfs_rq, se, 0); detach_entity_load_avg(cfs_rq, se); --=20 2.37.2 From nobody Fri Apr 10 23:24:11 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 48D64C32771 for ; Thu, 18 Aug 2022 03:46:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243150AbiHRDqY (ORCPT ); Wed, 17 Aug 2022 23:46:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39714 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243126AbiHRDqK (ORCPT ); Wed, 17 Aug 2022 23:46:10 -0400 Received: from mail-pf1-x429.google.com (mail-pf1-x429.google.com [IPv6:2607:f8b0:4864:20::429]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 88F6895680 for ; Wed, 17 Aug 2022 20:46:09 -0700 (PDT) Received: by mail-pf1-x429.google.com with SMTP id 81so418268pfz.10 for ; Wed, 17 Aug 2022 20:46:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=w96IsTb3GQtW46GghcS8avnysfYYRuHuTPsKLYARCUM=; b=Lbo0sm4b3VCe1e81KQcnj0Uo8jxOJvs0JV470Z7MXEVfLFKMrGFdXzPO3wtGUc1Vx2 McH+g/o4Tt0J36WdBLo8DKhUTxYyXD0zOxeutkn8tjXdTcpOAOy0RpLbThn/bq3BECcU VxjKDNQDmf8jxjNSkytt+ogUAeQjXFOKiR3/N77b2q57XTHF63vFfOcOc2hNzni0JzCb VU7fNDPNef766Zf9+1vBgfmsUsjghjrfwp7uAC7rAfRREK6gOu/jgjLGcLEVcy+0Zp/Z bO+waBUkH5gohM068ET0g1BkRYrdyY28Dd8Zrzte81IkRG1dXGRQSu2RuE6BN1bdo77S MvEw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=w96IsTb3GQtW46GghcS8avnysfYYRuHuTPsKLYARCUM=; b=c5xeKkx0XR9cu/ObnTexd5vqgx0Dl7ayDQbJ6mekQxGVnpWUsSMDaJN4ipixH/2lJK SDijyXxYUiqS2Cb45XjWYQ2t4rXZPGX0s1iKlGGUq6o4BAkZODjXXBBS+ArTdLT1LL2O MQ8lPrzhiuYDdUfcLPdcFklAua3vvH15A3phvSDn9NvjIDPpT2DScfzSD1PIkm/6MljK bnE8XUuRSUYHpaEPzcadDIJ5xbq47veHpKZN6rD5qU1Geqqk3KllwqR7iGHuunw8XL10 EZF9aWba9LrpoKakkqWhOc3zgxV8LF7DSDNMDHWi80zGCE2ZKahFF//t8cGEjKj5MQV3 tkTg== X-Gm-Message-State: ACgBeo2ml8eeDcg9V/CTKhrUdzpbxGWaIrZyvCZabL8u2eTUBOOwnESw O3m7jNUkn2uztj4owECmHJpyhg== X-Google-Smtp-Source: AA6agR6a7JJX+IywiQrBnUNjn49NfgqaXo2Uaudj+SXY+rDDEYMjstMBGpLSoxX4YsRpnauagG6law== X-Received: by 2002:a05:6a00:21c5:b0:52b:fc9c:295b with SMTP id t5-20020a056a0021c500b0052bfc9c295bmr1201176pfj.56.1660794368842; Wed, 17 Aug 2022 20:46:08 -0700 (PDT) Received: from C02CV1DAMD6P.bytedance.net ([139.177.225.246]) by smtp.gmail.com with ESMTPSA id m11-20020a65564b000000b00419acadde52sm268700pgs.46.2022.08.17.20.46.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 Aug 2022 20:46:08 -0700 (PDT) From: Chengming Zhou To: vincent.guittot@linaro.org, dietmar.eggemann@arm.com, mingo@redhat.com, peterz@infradead.org, rostedt@goodmis.org, bsegall@google.com, vschneid@redhat.com Cc: linux-kernel@vger.kernel.org, tj@kernel.org, Chengming Zhou Subject: [PATCH v5 7/9] sched/fair: allow changing cgroup of new forked task Date: Thu, 18 Aug 2022 11:43:41 +0800 Message-Id: <20220818034343.87625-8-zhouchengming@bytedance.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220818034343.87625-1-zhouchengming@bytedance.com> References: <20220818034343.87625-1-zhouchengming@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" commit 7dc603c9028e ("sched/fair: Fix PELT integrity for new tasks") introduce a TASK_NEW state and an unnessary limitation that would fail when changing cgroup of new forked task. Because at that time, we can't handle task_change_group_fair() for new forked fair task which hasn't been woken up by wake_up_new_task(), which will cause detach on an unattached task sched_avg problem. This patch delete this unnessary limitation by adding check before do detach or attach in task_change_group_fair(). So cpu_cgrp_subsys.can_attach() has nothing to do for fair tasks, only define it in #ifdef CONFIG_RT_GROUP_SCHED. Signed-off-by: Chengming Zhou --- include/linux/sched.h | 5 ++--- kernel/sched/core.c | 30 +++++++----------------------- kernel/sched/fair.c | 7 +++++++ 3 files changed, 16 insertions(+), 26 deletions(-) diff --git a/include/linux/sched.h b/include/linux/sched.h index e7b2f8a5c711..0b296e855dee 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -96,10 +96,9 @@ struct task_group; #define TASK_WAKEKILL 0x0100 #define TASK_WAKING 0x0200 #define TASK_NOLOAD 0x0400 -#define TASK_NEW 0x0800 /* RT specific auxilliary flag to mark RT lock waiters */ -#define TASK_RTLOCK_WAIT 0x1000 -#define TASK_STATE_MAX 0x2000 +#define TASK_RTLOCK_WAIT 0x0800 +#define TASK_STATE_MAX 0x1000 =20 /* Convenience macros for the sake of set_current_state: */ #define TASK_KILLABLE (TASK_WAKEKILL | TASK_UNINTERRUPTIBLE) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 8e3f1c3f0b2c..157f7461a08a 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -4550,11 +4550,11 @@ int sched_fork(unsigned long clone_flags, struct ta= sk_struct *p) { __sched_fork(clone_flags, p); /* - * We mark the process as NEW here. This guarantees that + * We mark the process as running here. This guarantees that * nobody will actually run it, and a signal or other external * event cannot wake it up and insert it on the runqueue either. */ - p->__state =3D TASK_NEW; + p->__state =3D TASK_RUNNING; =20 /* * Make sure we do not leak PI boosting priority to the child. @@ -4672,7 +4672,6 @@ void wake_up_new_task(struct task_struct *p) struct rq *rq; =20 raw_spin_lock_irqsave(&p->pi_lock, rf.flags); - WRITE_ONCE(p->__state, TASK_RUNNING); #ifdef CONFIG_SMP /* * Fork balancing, do it here and not earlier because: @@ -10290,36 +10289,19 @@ static void cpu_cgroup_css_free(struct cgroup_sub= sys_state *css) sched_unregister_group(tg); } =20 +#ifdef CONFIG_RT_GROUP_SCHED static int cpu_cgroup_can_attach(struct cgroup_taskset *tset) { struct task_struct *task; struct cgroup_subsys_state *css; - int ret =3D 0; =20 cgroup_taskset_for_each(task, css, tset) { -#ifdef CONFIG_RT_GROUP_SCHED if (!sched_rt_can_attach(css_tg(css), task)) return -EINVAL; -#endif - /* - * Serialize against wake_up_new_task() such that if it's - * running, we're sure to observe its full state. - */ - raw_spin_lock_irq(&task->pi_lock); - /* - * Avoid calling sched_move_task() before wake_up_new_task() - * has happened. This would lead to problems with PELT, due to - * move wanting to detach+attach while we're not attached yet. - */ - if (READ_ONCE(task->__state) =3D=3D TASK_NEW) - ret =3D -EINVAL; - raw_spin_unlock_irq(&task->pi_lock); - - if (ret) - break; } - return ret; + return 0; } +#endif =20 static void cpu_cgroup_attach(struct cgroup_taskset *tset) { @@ -11155,7 +11137,9 @@ struct cgroup_subsys cpu_cgrp_subsys =3D { .css_released =3D cpu_cgroup_css_released, .css_free =3D cpu_cgroup_css_free, .css_extra_stat_show =3D cpu_extra_stat_show, +#ifdef CONFIG_RT_GROUP_SCHED .can_attach =3D cpu_cgroup_can_attach, +#endif .attach =3D cpu_cgroup_attach, .legacy_cftypes =3D cpu_legacy_files, .dfl_cftypes =3D cpu_files, diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index eba8a64f905a..e0d34ecdabae 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -11840,6 +11840,13 @@ void init_cfs_rq(struct cfs_rq *cfs_rq) #ifdef CONFIG_FAIR_GROUP_SCHED static void task_change_group_fair(struct task_struct *p) { + /* + * We couldn't detach or attach a forked task which + * hasn't been woken up by wake_up_new_task(). + */ + if (!p->on_rq && !p->se.sum_exec_runtime) + return; + detach_task_cfs_rq(p); =20 #ifdef CONFIG_SMP --=20 2.37.2 From nobody Fri Apr 10 23:24:11 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5B662C25B08 for ; Thu, 18 Aug 2022 03:46:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243169AbiHRDqi (ORCPT ); Wed, 17 Aug 2022 23:46:38 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39966 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242948AbiHRDqO (ORCPT ); Wed, 17 Aug 2022 23:46:14 -0400 Received: from mail-pj1-x1033.google.com (mail-pj1-x1033.google.com [IPv6:2607:f8b0:4864:20::1033]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4A9069411D for ; Wed, 17 Aug 2022 20:46:13 -0700 (PDT) Received: by mail-pj1-x1033.google.com with SMTP id x63-20020a17090a6c4500b001fabbf8debfso613549pjj.4 for ; Wed, 17 Aug 2022 20:46:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=NxxucBO4QrVt96mM1hHKarI4sR940EM9ZXnYf+M0A98=; b=HkNkehnITQZloctQe8RnTDYTs+3zO+4QF0s1Eydaku5Nepp0g0LKBTkoR78aENSZzh STuAAKgf3EJYuv4q63nmg3ezGlCTptrHUwWrT3S+66oxcjDPSrW5GMQjml/YIM/hmSln mM8o3CEEy114kR9m003e/ZdL5VvDKSe94wjKKgwmgJ0lY+Ex2yLJrZ2BOzGgL2Lg6FfN S9kk6D2rHxZ+WHSFQ9Fb9ZYoZTnOHOGF+uBasXTjkC4fVHtzykVemDyREAqOvsJBqlud AtrZO7BArjObwaZLHQ0ETGfaq2Y8GTDQTZKzoPDBKsTj5acrx51dqlfSiuwZaW8OY99W B+Pg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=NxxucBO4QrVt96mM1hHKarI4sR940EM9ZXnYf+M0A98=; b=BZDM7NxoHZeqnC/4wibuE6Qqc3bsIEog3wKEe02pF0Jhs0TLe73g+LnguOmA8MNSbD AnmyVLtWHd5jMlGffg7/hqlzOPmwOEsJOp/r9Z1JLm79cgYDEh530ugLPY+Yk6c6R2Cq aYYqb5IN29xI34+Kj3LViLpgcgR5jeW8ei4LbYCpLnK8LGMWdjGAVuNFy5CGISKaJgGx zh85Kl14vA8W9pnZ12ACUGchYdVPvS3RiRxRYCMYj/WJD6ID0//DZbSZvLvDOQSzuIpI ZxGaxriJpavwUeJYZe3MbkFsTcqkUwJUEBIL5WqLdNH9r+6qf/guRChApOjLgXZE8QpJ +coQ== X-Gm-Message-State: ACgBeo1rnFVaHvTBNic3+JSN/y14BNSeDHZvWnSrg/Ho8m/UbDZ9lWbq TBY3i70VdDGtzNvKZoYaySKwog== X-Google-Smtp-Source: AA6agR7x7lXSj5gijX30ittT8hKx+Po6pLby8DjpClfykm5ZCN5EUCgbfiBMYeQ2o/iZvlB59ImmbQ== X-Received: by 2002:a17:902:b207:b0:16f:2349:6116 with SMTP id t7-20020a170902b20700b0016f23496116mr934512plr.80.1660794373030; Wed, 17 Aug 2022 20:46:13 -0700 (PDT) Received: from C02CV1DAMD6P.bytedance.net ([139.177.225.246]) by smtp.gmail.com with ESMTPSA id m11-20020a65564b000000b00419acadde52sm268700pgs.46.2022.08.17.20.46.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 Aug 2022 20:46:12 -0700 (PDT) From: Chengming Zhou To: vincent.guittot@linaro.org, dietmar.eggemann@arm.com, mingo@redhat.com, peterz@infradead.org, rostedt@goodmis.org, bsegall@google.com, vschneid@redhat.com Cc: linux-kernel@vger.kernel.org, tj@kernel.org, Chengming Zhou Subject: [PATCH v5 8/9] sched/fair: defer task sched_avg attach to enqueue_entity() Date: Thu, 18 Aug 2022 11:43:42 +0800 Message-Id: <20220818034343.87625-9-zhouchengming@bytedance.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220818034343.87625-1-zhouchengming@bytedance.com> References: <20220818034343.87625-1-zhouchengming@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" When wake_up_new_task(), we would use post_init_entity_util_avg() to init util_avg/runnable_avg based on cpu's util_avg at that time, then attach task sched_avg to cfs_rq. Since enqueue_entity() would always attach any unattached task entity, so we can defer this work to enqueue_entity(). post_init_entity_util_avg(p) attach_entity_cfs_rq() --> (1) activate_task(rq, p) enqueue_task() :=3D enqueue_task_fair() enqueue_entity() update_load_avg(cfs_rq, se, UPDATE_TG | DO_ATTACH) if (!se->avg.last_update_time && (flags & DO_ATTACH)) attach_entity_load_avg() --> (2) This patch defer attach from (1) to (2) Signed-off-by: Chengming Zhou --- kernel/sched/fair.c | 4 ---- 1 file changed, 4 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index e0d34ecdabae..aacf38a72714 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -799,8 +799,6 @@ void init_entity_runnable_average(struct sched_entity *= se) /* when this task enqueue'ed, it will contribute to its cfs_rq's load_avg= */ } =20 -static void attach_entity_cfs_rq(struct sched_entity *se); - /* * With new tasks being created, their initial util_avgs are extrapolated * based on the cfs_rq's current util_avg: @@ -863,8 +861,6 @@ void post_init_entity_util_avg(struct task_struct *p) se->avg.last_update_time =3D cfs_rq_clock_pelt(cfs_rq); return; } - - attach_entity_cfs_rq(se); } =20 #else /* !CONFIG_SMP */ --=20 2.37.2 From nobody Fri Apr 10 23:24:11 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E27C1C25B08 for ; Thu, 18 Aug 2022 03:46:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243163AbiHRDqp (ORCPT ); Wed, 17 Aug 2022 23:46:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40076 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243136AbiHRDqS (ORCPT ); Wed, 17 Aug 2022 23:46:18 -0400 Received: from mail-pl1-x62a.google.com (mail-pl1-x62a.google.com [IPv6:2607:f8b0:4864:20::62a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id ADC58956B5 for ; Wed, 17 Aug 2022 20:46:17 -0700 (PDT) Received: by mail-pl1-x62a.google.com with SMTP id m2so492431pls.4 for ; Wed, 17 Aug 2022 20:46:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=7cI2F3yXPonauMWsO3HcY8FVNQRGdZ4rG2wgR3FbNkc=; b=rmqR1dPNdfpW3nRVRnGjOLgNPZx5moCpN/Vj8TSCGNZu69GhYskWNPAVhe7fbzt1UM 00idvx2FZjjuMtDQp1P1z4NZLmiHOc3uxK7+PW0/mBCcAtyhdzGV7ChIsbZLXHIMG8WO ypCoNEkhHjJZLEPRrXhXCWVMkT2E3H8wziWvEDgpoTHIM/dAfhZPhPKUajuefu1VVzSm 4T9KFgGu0BfQTJK+eOGwWhLv/2KYcPIqkBIv+t+U+0RoPex4Fzh1mXLzY+bCsSVVLoCS tzx58VuLr4VOEmKrZAsU7o3lKssppqed4vBx6KX5GBYTE6dVg8T991CHXfq6TWun/hCz T+gQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=7cI2F3yXPonauMWsO3HcY8FVNQRGdZ4rG2wgR3FbNkc=; b=UWUS+Kqf1DPLAKh5RLPsyUJziiFUTWL2RB/wIfz/zVPwEJDddHfNEq+L/DlgpdxRFk GNkj7Lit0GFMymYv9S0ZgUc091OvKbA4SJq/UKdHtWedKJnk7NzdwX5KYPvAH26Gb8vl pM/vQ5QUANCjQrOw3oGHuYQ5BOpij2HvaRPzHer+P4YQ2G3I8r4VilDtDERPskeIuOu/ Fi0ZYFJKGYvUoQrw1Ea5+MugnQ/UrHg15kQzdNgQnPmxF2GAminsjvDjaZPT9YDowHH4 viebHcoQt5vTLHzygkXCCo3/LEL0bYNaGtpIGWqodsUTwH3SpNA2myIWd4iSnarue5r+ zkpw== X-Gm-Message-State: ACgBeo2PajE7R7IO94bDIQ4Zq0zoXc0wsYaqXLYXITDrTVn8othWZ+68 KconbeZ8vuj4wrkwr0zyej6/aw== X-Google-Smtp-Source: AA6agR5raRrJtTp7ad5RUuJ5jzzsYPF7j/ZkwcYV5JTEdY4R5PF3Xw0+5tHcRevk5WvWfkGbHrDZ5Q== X-Received: by 2002:a17:902:c94d:b0:16d:c10a:650e with SMTP id i13-20020a170902c94d00b0016dc10a650emr1174768pla.29.1660794377183; Wed, 17 Aug 2022 20:46:17 -0700 (PDT) Received: from C02CV1DAMD6P.bytedance.net ([139.177.225.246]) by smtp.gmail.com with ESMTPSA id m11-20020a65564b000000b00419acadde52sm268700pgs.46.2022.08.17.20.46.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 Aug 2022 20:46:16 -0700 (PDT) From: Chengming Zhou To: vincent.guittot@linaro.org, dietmar.eggemann@arm.com, mingo@redhat.com, peterz@infradead.org, rostedt@goodmis.org, bsegall@google.com, vschneid@redhat.com Cc: linux-kernel@vger.kernel.org, tj@kernel.org, Chengming Zhou Subject: [PATCH v5 9/9] sched/fair: don't init util/runnable_avg for !fair task Date: Thu, 18 Aug 2022 11:43:43 +0800 Message-Id: <20220818034343.87625-10-zhouchengming@bytedance.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220818034343.87625-1-zhouchengming@bytedance.com> References: <20220818034343.87625-1-zhouchengming@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" post_init_entity_util_avg() init task util_avg according to the cpu util_avg at the time of fork, which will decay when switched_to_fair() some time lat= er, we'd better to not set them at all in the case of !fair task. Suggested-by: Vincent Guittot Signed-off-by: Chengming Zhou --- kernel/sched/fair.c | 28 ++++++++++++++-------------- 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index aacf38a72714..235b59b9d75a 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -833,20 +833,6 @@ void post_init_entity_util_avg(struct task_struct *p) long cpu_scale =3D arch_scale_cpu_capacity(cpu_of(rq_of(cfs_rq))); long cap =3D (long)(cpu_scale - cfs_rq->avg.util_avg) / 2; =20 - if (cap > 0) { - if (cfs_rq->avg.util_avg !=3D 0) { - sa->util_avg =3D cfs_rq->avg.util_avg * se->load.weight; - sa->util_avg /=3D (cfs_rq->avg.load_avg + 1); - - if (sa->util_avg > cap) - sa->util_avg =3D cap; - } else { - sa->util_avg =3D cap; - } - } - - sa->runnable_avg =3D sa->util_avg; - if (p->sched_class !=3D &fair_sched_class) { /* * For !fair tasks do: @@ -861,6 +847,20 @@ void post_init_entity_util_avg(struct task_struct *p) se->avg.last_update_time =3D cfs_rq_clock_pelt(cfs_rq); return; } + + if (cap > 0) { + if (cfs_rq->avg.util_avg !=3D 0) { + sa->util_avg =3D cfs_rq->avg.util_avg * se->load.weight; + sa->util_avg /=3D (cfs_rq->avg.load_avg + 1); + + if (sa->util_avg > cap) + sa->util_avg =3D cap; + } else { + sa->util_avg =3D cap; + } + } + + sa->runnable_avg =3D sa->util_avg; } =20 #else /* !CONFIG_SMP */ --=20 2.37.2