From nobody Mon Sep 28 04:48:45 2026 Received: from mail-ed1-f48.google.com (mail-ed1-f48.google.com [209.85.208.48]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6E7CF3101A0 for ; Wed, 26 Aug 2026 10:18:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.48 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787739517; cv=none; b=KnXnO+rEthpO3zYr19fJpCccwK/3QwGh44jKj5QqQGE3s/X4mIvWEUWHuccifyS8YgD50SOXzhzL2j1i6rc9klVkEV/7lDSEstFFBReq77K/2eZw0LNTupcZpsGc8FA3CVXDODJfwE+vYs+RBa8PArOh8bown/R+XGfb8M/EbRQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787739517; c=relaxed/simple; bh=XDxe6faXA0YczdspahhCXbr6Sqpl1bIFT5Gk5YFdB3s=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=PkkXPKBnmFvS+6DD5RtS6yjPADdk+EDWmTihtkVU58lWCsxMAWZU+nU8rR0igiApnzoyyHPT3eumbAp0P5tduwxqQ/Cv+1mk7aw0lBBcVxyU6Sbn/Kh+OUgPKK0x72qQ+1RavCG16wH2XDSoaz82Ke46JxcSu+uNlPQTKdXcQAI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=oGww1aiq; arc=none smtp.client-ip=209.85.208.48 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="oGww1aiq" Received: by mail-ed1-f48.google.com with SMTP id 4fb4d7f45d1cf-6a5d8f8fe99so70449a12.2 for ; Wed, 26 Aug 2026 03:18:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1787739513; x=1788344313; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=0ZGM3/RxGQgvaE752b6JQ6zgAgS1wMO/OWHkmxZA/Lk=; b=oGww1aiqeb6DlfsFGkNmlKx8Ud1UZzDzeD8QjX4X8LxPnu2gqTQxzUnZoh7XonWLul ZeRdLaHKajiSsgYmpwZQCa3ZCckbvo/LV56LqhtKr+o01hxWKFkOWkgKqqaFWVCW6BKD T7p9sryi25ODCtpa3R/DkSB+lPO1TL24bHC0S6SV+6jzGJzE8qLypjyVacTCfQwC4oD8 R+0cq7hLaIMycngT67S9Hh0fjJwlE6MoAs75rdV/VNrPSfS4VavFnURAOQfE7vxBy1rY LKaEE7WSMuUox23HH8fkPBDEmBEUno1c1m75ALwrPPJDKkJemNqyJX08HToTo96pbO5k CqxA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1787739513; x=1788344313; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=0ZGM3/RxGQgvaE752b6JQ6zgAgS1wMO/OWHkmxZA/Lk=; b=Smgyo8n9hM+E4dprKiXNFTlLgQiUG08GJH8TPJABMEP4l1CPsNT5850WhN43zIuLqs /MSDqJg4XR0PFs8VVxzOoHUnDB3BPH2iJ1imsrk3/2+9PkR4ByQAB4g6wrAUfOfWUwTc CtXC0rq9WbZU0olsbLDvEBr8euxfYfTDysJJP8gX4iiy9MVzFlNr5lBOosgsM3/DTOUi oAuGCrXAG+29CY4AJzOQqlz3/9qCfW8evVuqYji20SDjhuJ2tUMd6vgsrm+7uTlT+C6O NAsdG9i2bd9IZwG9jW/a1FqqIzjyPYCAnwmYaCzHChxv+xzQIW0SZFqJ5r++Z8hBHUlf zq6Q== X-Gm-Message-State: AFuF++nEg+Z1W+vJZ7M9msBtjVhyvSJ15zCc/nsiL5jJaShbZfQdrAce swBrXjilgXcP99dCMA7VQGmlS/zLQnSsM37PxAJ8rX119f1K9p5/kajD X-Gm-Gg: AR+sD10SHux8duMWXgvDMI22pY8mZmDQ3+jkgtoghzjwBiZYJPSRdQlbtcR4SIoJ8sN AFZ/Iyj4MqJXFZZlpMOeGkIBnwjAIxuwetA+H3Q8oLHrUTc/90pAKrM89QUlVpH+OUdEv+eZ1g9 6kHoT+lAbh4AgDkWqLsDQURck/rcwNXkoDYrkieUEb5uQjVNrTEcihHgfmgWqBplkMtjHnVul76 Yj7cXhMkqMZJ8ksR+TWxekcGUl57CZoEAx0czZABRkQI2sKMZJxlkZgRiQgwSs99s6JN7G5na3Y MmZGFv8zaKKAtys37kqvhzk+EI1TOsJi7GrSUOMFwIkOwsmJMLJF4MGay+/lmaOvENAD/P1oXNx MEsyCXYhW45WIqOcrKAbk5Qv+uru5Hcn+rrCCczqGmwFgfIUMX73qeOMF7B0wgxsGdX1CC4F9xL TngV+jyHa7mWxRWxB2hiddXw5FmfuOGK0vFnFPwkPTkQl6oIRbgly84dE2eNmEtPx9fJTNz5AnN WzANYh9ZcY= X-Received: by 2002:a17:907:1b27:b0:c20:c5f4:e19f with SMTP id a640c23a62f3a-c250c37aff9mr379387766b.4.1787739512582; Wed, 26 Aug 2026 03:18:32 -0700 (PDT) Received: from lima-kdev.local ([85.100.66.184]) by smtp.gmail.com with ESMTPSA id 4fb4d7f45d1cf-6a5de8b641fsm3942205a12.3.2026.08.26.03.18.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 26 Aug 2026 03:18:32 -0700 (PDT) From: Kayra Cizmeci To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , K Prateek Nayak Cc: linux-kernel@vger.kernel.org, Kayra Cizmeci Subject: [PATCH v2 1/2] sched/fair: reuse the ENQUEUE_DELAYED calculation in enqueue_task_fair() Date: Wed, 26 Aug 2026 13:15:41 +0300 Message-ID: <0b1ef9d0122ae3037dac38d2549f13c9b063369a.1787737648.git.kayracizmeci@gmail.com> X-Mailer: git-send-email 2.53.0 In-Reply-To: References: <20260824125223.508178-1-kayracizmeci@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Currently flags & ENQUEUE_DELAYED checks spread throughout the enqueue_task_fair(), connect these checks to a bool 'delayed' that gets calculated at the start of the function. No functional change intended. Signed-off-by: Kayra Cizmeci --- kernel/sched/fair.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 6d881e530f89..b411384125ec 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -7985,7 +7985,7 @@ enqueue_task_fair(struct rq *rq, struct task_struct *= p, int flags) struct sched_entity *se =3D &p->se; struct cfs_rq *cfs_rq =3D &rq->cfs; unsigned long weight; - bool curr; + bool curr, delayed =3D (flags & ENQUEUE_DELAYED); =20 if (task_is_throttled(p) && enqueue_throttled_task(p)) return; @@ -7996,12 +7996,12 @@ enqueue_task_fair(struct rq *rq, struct task_struct= *p, int flags) * Let's add the task's estimated utilization to the cfs_rq's * estimated utilization, before we update schedutil. */ - if (!p->se.sched_delayed || (flags & ENQUEUE_DELAYED)) + if (!p->se.sched_delayed || delayed) util_est_enqueue(cfs_rq, p); =20 update_curr_eevdf(cfs_rq); =20 - if (flags & ENQUEUE_DELAYED) { + if (delayed) { requeue_delayed_entity(cfs_rq, se); return; } --=20 2.53.0 From nobody Mon Sep 28 04:48:45 2026 Received: from mail-ej1-f46.google.com (mail-ej1-f46.google.com [209.85.218.46]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6A4903BED06 for ; Wed, 26 Aug 2026 10:18:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.46 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787739520; cv=none; b=oYvpNRlV01wwFJDN8k9lAm/0iELtcZu+JQyBa1y2wcLu2B89wOIVN58iA1/ZUEgmYjzRPgnq0eL+p8RV1iGeIE3Vlt/T63+ERgFOoIRgvZfcV3Wsq/n7ecru/V47+iDY1G9+JRdth8pV4K2ZKSte6/j1Qpd2o0NpFjmyp7XGdf4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787739520; c=relaxed/simple; bh=lr/GdKoAuSX/7iX8XjDXDEqANeVWXeNJqyJuV7ByWIE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=uzEs3TqdEMVEtUnl7rICaO4mT78L5/hdDCKhQhLEuyciworfUCq9BKI0xQz+iEAL5kaK21oS+kjTR/sFiE4bXnFzxo07yleOeZM46szunWRNXAU9xJvpXcr6I+s+VoLie1cx/q8cJ+oTxKQUYTYywFTwGZJCg2XBsGlF3nt4qEg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=Q0qaGki2; arc=none smtp.client-ip=209.85.218.46 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Q0qaGki2" Received: by mail-ej1-f46.google.com with SMTP id a640c23a62f3a-c15b51ba80dso11143866b.2 for ; Wed, 26 Aug 2026 03:18:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1787739516; x=1788344316; darn=vger.kernel.org; h=content-transfer-encoding:content-type:mime-version:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to:content-type; bh=yw4V5PCOk6N28hts+pesxfocscQ4Cju14AaITwF55SE=; b=Q0qaGki2UuDAGRF8D0l7rY5fIg8dFLCC4IGffIgo30n6Op8E+TSTg0l6jl4eFnsV1C RwmYyV93l7DfqTJ2kAIVGMTu8hrvawzVYsgpgT0Sbsh39R7aCuvuM2UpDXZZr/H9sbD1 8UlVPkf3/9WDi1FneiPqaiJfUXTaJ9XbnqVm2P3RVwsuc5/0ZOGJIsGlx24zDTWZSikF hlN9KBkQS4+4ljQhsw+RjsG7EnsosZar2qIlNevjauZKl6w9UaQkGfXiYQHemmosAEmF 2yo5Xbg+3HEFMisOMWYBQP+XA0BMO2USu53+4LUCpRWSnYQTtf+hT0QIzJtmVeo3Kfzu dV/A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1787739516; x=1788344316; h=content-transfer-encoding:content-type:mime-version:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=yw4V5PCOk6N28hts+pesxfocscQ4Cju14AaITwF55SE=; b=s0TKHFrPTaJ4kovwo763lsN1upsVYCsXDMVLsuNiphr+jGTGAhGMHbbCLlP8bGSm4f FYlFXUbhpAAjCrfOvl4ZenilCF9bYNj7IhwxIynLpycr38m98Ez+KcUNMt7Wyw02TLSI qZcaREwrdNdlfDPqBLvm9dj95N+gGc/8/3uZYOpR6DYWNUO6OdFORyRRSClHWWGbYIfi NebohHxJT8XwoJFLpzOvaBN12uIA3A6ivSybrzcVgLcAYTJTBWtT5sQ2GIxlOCl57bt0 WXvCmdCRZexjMQuaup64PoYewh1tOKm84YQbAMsP3GNJeaC06QAa47XnvCQ3WmswmB8K IJ8Q== X-Gm-Message-State: AFuF++n4IVMRvHGMDvGYCrcQtJGs2AouuozcIn+NXTcyuo1WAugC+Ayk AQamuJWjxYVceZFi6zPrlJQU8HRZLbizYe5Iq2yoWYDVxpZvuvMvr3rZjQha6QA5 X-Gm-Gg: AR+sD12qPJKLyCmWxkFQcXcQXOT7XjsBLBrUVgzUI9IjHiVXjYvtexYqwmHM5lRl9zl LeAQmhimVyPvniI1h9OyVSp9zE+DmWPAKPl3JHGHJki5sxRTe6Wh5hOnt0DQJlLN51GDERTgyZw tToT5eGKBnglXC23LQZ0OtrP+M87bb/2aI3dALNBttMDJby0uwyHdNfEz3OH37LkB4ti/T/A4l9 cr52uedsNSwYFC/y3CRncWWXT4ITRgbV3K7rbiwc4Wt+s6ptudpdgSV4aOf1rdTfFHccA9rueIT uMKjIw7y6rDo2v9EtqTRdgKtgmUFsmhCUnZY8kPv6wHhKzSJLmKj9geKe7h7QyozPQxGLVctKBn Gio6cITcpadlxsTOVVUk28G0AB/NZLYJ43cMeHjU1UHJr6vxnhiYW/v7TgtwnEtyYy/YBXOI5Zl 1+p7+BFWuEYmuj6686IjfdaqjDoRPbYzW5mD2Vs0Lj4yiRq7U+87KnreJa1EVXwaTNj0fLXQFMq dmoQvNKHSY= X-Received: by 2002:a17:907:1c0f:b0:c12:6db4:4bf4 with SMTP id a640c23a62f3a-c250b9fdfa0mr389429366b.0.1787739516480; Wed, 26 Aug 2026 03:18:36 -0700 (PDT) Received: from lima-kdev.local ([85.100.66.184]) by smtp.gmail.com with ESMTPSA id 4fb4d7f45d1cf-6a5de8b641fsm3942205a12.3.2026.08.26.03.18.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 26 Aug 2026 03:18:36 -0700 (PDT) From: Kayra Cizmeci To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , K Prateek Nayak Cc: linux-kernel@vger.kernel.org, Kayra Cizmeci Subject: [PATCH v2 2/2] sched/fair: avoid recalculating curr status in place_entity() and requeue_delayed_entity() Date: Wed, 26 Aug 2026 13:15:42 +0300 Message-ID: <60fd4fc5d17b706766a428b63760d1f69e6d11bb.1787737648.git.kayracizmeci@gmail.com> X-Mailer: git-send-email 2.53.0 In-Reply-To: References: <20260824125223.508178-1-kayracizmeci@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable In enqueue_task_fair() a bool is calculated by cfs_rq->curr =3D=3D se. But this information gets recalculated on requeue_delayed_entity() and requeue_delayed_entity() only gets called in enqueue_task_fair(). And on place_entity() if se =3D=3D curr we call the avg_vruntime_weight() twice with the same input. place_entity() only gets called in enqueue_task_fair() and requeue_delayed_entity(). Use the information on enqueue_task_fair() in requeue_delayed_entity(). And place_entity() to avoid calling avg_vruntime_weight() twice. Signed-off-by: Kayra Cizmeci --- The patch needs curr to be invariant so I added the same=C2=A0 (cfs_rq->curr =3D=3D se) to places in requeue_delayed_entity() and and enqueue_task_fair(). Like after the place_entity() call in requeue_delayed_entity(), or like before if (curr) the old place of the calculation. And added WARN_ON_ONCE(curr !=3D new_calc_curr) or something like that I don't know how to say it normally. Then I booted these changes on x86 (Zen 3) called perf bench sched messaging with 200 groups and 5000 loops. And then I make sure if the requeue_delayed_entity() was really working with ftrace. The results were good but I left the computer on 2 more hours and then checked the results again. It was still good, but considering that I booted the kernel with busybox I don't think much happened on that 2 hour window. kernel/sched/fair.c | 36 ++++++++++++++++++++++-------------- 1 file changed, 22 insertions(+), 14 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index b411384125ec..f1b46fe26d12 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -6175,7 +6175,7 @@ void __setparam_fair(struct task_struct *p, const str= uct sched_attr *attr) } =20 static void -place_entity(struct cfs_rq *cfs_rq, struct sched_entity *se, int flags) +place_entity(struct cfs_rq *cfs_rq, struct sched_entity *se, int flags, bo= ol is_curr) { u64 vslice, vruntime =3D avg_vruntime(cfs_rq); unsigned int nr_queued =3D cfs_rq->h_nr_queued; @@ -6199,7 +6199,7 @@ place_entity(struct cfs_rq *cfs_rq, struct sched_enti= ty *se, int flags) */ if (sched_feat(PLACE_LAG) && nr_queued && se->vlag) { struct sched_entity *curr =3D cfs_rq->curr; - long load, weight; + long load, weight, curr_weight; =20 lag =3D se->vlag; =20 @@ -6256,10 +6256,17 @@ place_entity(struct cfs_rq *cfs_rq, struct sched_en= tity *se, int flags) * vl_i =3D (W + w_i)*vl'_i / W */ load =3D cfs_rq->sum_weight; - if (curr && curr->on_rq) - load +=3D avg_vruntime_weight(cfs_rq, curr->h_load.weight); + if (curr && (curr->on_rq || is_curr)) { + curr_weight =3D avg_vruntime_weight(cfs_rq, curr->h_load.weight); + if (curr->on_rq) + load +=3D curr_weight; + } + + if (is_curr) + weight =3D curr_weight; + else + weight =3D avg_vruntime_weight(cfs_rq, se->h_load.weight); =20 - weight =3D avg_vruntime_weight(cfs_rq, se->h_load.weight); lag *=3D load + weight; if (WARN_ON_ONCE(!load)) load =3D 1; @@ -7900,7 +7907,7 @@ static int choose_idle_cpu(int cpu, struct task_struc= t *p) } =20 static void -requeue_delayed_entity(struct cfs_rq *cfs_rq, struct sched_entity *se) +requeue_delayed_entity(struct cfs_rq *cfs_rq, struct sched_entity *se, boo= l curr) { /* * se->sched_delayed should imply: se->on_rq =3D=3D 1. @@ -7912,10 +7919,10 @@ requeue_delayed_entity(struct cfs_rq *cfs_rq, struc= t sched_entity *se) =20 if (update_entity_lag(cfs_rq, se)) { cfs_rq->h_nr_queued--; - if (se !=3D cfs_rq->curr) + if (!curr) __dequeue_entity(cfs_rq, se); - place_entity(cfs_rq, se, 0); - if (se !=3D cfs_rq->curr) + place_entity(cfs_rq, se, 0, curr); + if (!curr) __enqueue_entity(cfs_rq, se); cfs_rq->h_nr_queued++; } @@ -8000,9 +8007,10 @@ enqueue_task_fair(struct rq *rq, struct task_struct = *p, int flags) util_est_enqueue(cfs_rq, p); =20 update_curr_eevdf(cfs_rq); + curr =3D (cfs_rq->curr =3D=3D se); =20 if (delayed) { - requeue_delayed_entity(cfs_rq, se); + requeue_delayed_entity(cfs_rq, se, curr); return; } =20 @@ -8017,18 +8025,18 @@ enqueue_task_fair(struct rq *rq, struct task_struct= *p, int flags) /* * XXX comment on the curr thing */ - curr =3D (cfs_rq->curr =3D=3D se); + if (curr) - place_entity(cfs_rq, se, flags); + place_entity(cfs_rq, se, flags, curr); =20 if (se->on_rq && se->sched_delayed) - requeue_delayed_entity(cfs_rq, se); + requeue_delayed_entity(cfs_rq, se, curr); =20 weight =3D enqueue_hierarchy(p, flags); =20 if (!curr) { reweight_eevdf(cfs_rq, se, weight, false); - place_entity(cfs_rq, se, flags | ENQUEUE_QUEUED); + place_entity(cfs_rq, se, flags | ENQUEUE_QUEUED, curr); __enqueue_entity(cfs_rq, se); } =20 --=20 2.53.0