From nobody Thu Feb 12 06:28:14 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 09B93EE645B for ; Fri, 15 Sep 2023 12:49:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235114AbjIOMtx (ORCPT ); Fri, 15 Sep 2023 08:49:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55692 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235065AbjIOMtt (ORCPT ); Fri, 15 Sep 2023 08:49:49 -0400 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 526D4173A for ; Fri, 15 Sep 2023 05:49:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-Id:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:In-Reply-To; bh=+WwWDnoUiqbGQMNUOC/ZDSNUt5pEDK3ZmH2V16dcBZo=; b=qC5245E2acMXTSKfuWyWzMsP7O es34wu6noSN5Azve8F3y4O75lycVRk/lzMkNDk/NQb1HonvT6RqxD5J1jFLz9b2OIxOTavgG63XVj in4rdI+ZpVuqynqvBY1xOKguTvkC48sTC0khtwlXGh7CkPWZk7gWRY63PsbCxhiWYyQQeN0183Whj O1TkeWkA4IlBLs5+rnvq9MkEREeDGXOHn4uFd4A47xItoNX+Rg00zucEmE0iJEKzG9eG0aayOmkvN 09a7wr+iDTdGrzRbbRQK4iC0UeO9wffAUNrRR53noxDzkAS1WUDpBj3XxSxc8dv3oUIg9INUv/Ipq gRMKSIvQ==; Received: from j130084.upc-j.chello.nl ([24.132.130.84] helo=noisy.programming.kicks-ass.net) by casper.infradead.org with esmtpsa (Exim 4.94.2 #2 (Red Hat Linux)) id 1qh8Ft-009kyK-OT; Fri, 15 Sep 2023 12:49:05 +0000 Received: by noisy.programming.kicks-ass.net (Postfix, from userid 0) id 659A63005FA; Fri, 15 Sep 2023 14:49:05 +0200 (CEST) Message-Id: <20230915124822.847197830@noisy.programming.kicks-ass.net> User-Agent: quilt/0.65 Date: Fri, 15 Sep 2023 14:43:55 +0200 From: peterz@infradead.org To: mingo@kernel.org Cc: linux-kernel@vger.kernel.org, peterz@infradead.org, vincent.guittot@linaro.org, juri.lelli@redhat.com, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, bristot@redhat.com, corbet@lwn.net, qyousef@layalina.io, chris.hyser@oracle.com, patrick.bellasi@matbug.net, pjt@google.com, pavel@ucw.cz, qperret@google.com, tim.c.chen@linux.intel.com, joshdon@google.com, timj@gnu.org, kprateek.nayak@amd.com, yu.c.chen@intel.com, youssefesmat@chromium.org, joel@joelfernandes.org, efault@gmx.de, tglx@linutronix.de, daniel.m.jordan@oracle.com Subject: [PATCH 1/2] sched/eevdf: Also update slice on placement References: <20230915124354.416936110@noisy.programming.kicks-ass.net> MIME-Version: 1.0 Content-Disposition: inline; filename=peterz-sched-eevdf-slice-update.patch Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Tasks that never consume their full slice would not update their slice valu= e. This means that tasks that are spawned before the sysctl scaling keep their original (UP) slice length. Fixes: 147f3efaa241 ("sched/fair: Implement an EEVDF-like scheduling policy= ") Signed-off-by: Peter Zijlstra (Intel) --- kernel/sched/fair.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) Index: linux-2.6/kernel/sched/fair.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- linux-2.6.orig/kernel/sched/fair.c +++ linux-2.6/kernel/sched/fair.c @@ -4919,10 +4919,12 @@ static inline void update_misfit_status( static void place_entity(struct cfs_rq *cfs_rq, struct sched_entity *se, int flags) { - u64 vslice =3D calc_delta_fair(se->slice, se); - u64 vruntime =3D avg_vruntime(cfs_rq); + u64 vslice, vruntime =3D avg_vruntime(cfs_rq); s64 lag =3D 0; =20 + se->slice =3D sysctl_sched_base_slice; + vslice =3D calc_delta_fair(se->slice, se); + /* * Due to how V is constructed as the weighted average of entities, * adding tasks with positive lag, or removing tasks with negative lag From nobody Thu Feb 12 06:28:14 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AE21FEE6457 for ; Fri, 15 Sep 2023 12:49:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235163AbjIOMt7 (ORCPT ); Fri, 15 Sep 2023 08:49:59 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55716 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235106AbjIOMtv (ORCPT ); Fri, 15 Sep 2023 08:49:51 -0400 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F06861713 for ; Fri, 15 Sep 2023 05:49:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-Id:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:In-Reply-To; bh=vvyr4ysvwC+14BCIXh6MvqNIHme9hgsvbApmalFbkI0=; b=N8lNK7E8zBEG2Ps40TqsaN6Deo fR5P+iReWILYXPBmkceH+fy5ncSWpVFxz6r0p8PqU03z7fk/wPr1KnEqpjb0sDT7LihOoZLyZfREB f2+4tnghqhbHA3HnjRPmhcF1rIJYqsR/a75/ZgMIXKqOlQXBJcodFCo0LRH951t54Z2rMkZIQuWIu OkAhlSW0fuYYLfymlHK6BkieIkwTbJGtSLf62603/LcLM9iukvM+m0orSpeeFASFGO/NMzEdF56E7 C82LgyTkndjZvRGMfzUqT0dng+j4DBzak6GrsCsZp7WRYx53/gRsfMnvcP20d/dO0zFZ3hE8zCLFx AzBsPUoA==; Received: from j130084.upc-j.chello.nl ([24.132.130.84] helo=noisy.programming.kicks-ass.net) by casper.infradead.org with esmtpsa (Exim 4.94.2 #2 (Red Hat Linux)) id 1qh8Ft-009kyL-PT; Fri, 15 Sep 2023 12:49:05 +0000 Received: by noisy.programming.kicks-ass.net (Postfix, from userid 0) id 69EAB30067B; Fri, 15 Sep 2023 14:49:05 +0200 (CEST) Message-Id: <20230915124822.956946622@noisy.programming.kicks-ass.net> User-Agent: quilt/0.65 Date: Fri, 15 Sep 2023 14:43:56 +0200 From: peterz@infradead.org To: mingo@kernel.org Cc: linux-kernel@vger.kernel.org, peterz@infradead.org, vincent.guittot@linaro.org, juri.lelli@redhat.com, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, bristot@redhat.com, corbet@lwn.net, qyousef@layalina.io, chris.hyser@oracle.com, patrick.bellasi@matbug.net, pjt@google.com, pavel@ucw.cz, qperret@google.com, tim.c.chen@linux.intel.com, joshdon@google.com, timj@gnu.org, kprateek.nayak@amd.com, yu.c.chen@intel.com, youssefesmat@chromium.org, joel@joelfernandes.org, efault@gmx.de, tglx@linutronix.de, daniel.m.jordan@oracle.com Subject: [PATCH 2/2] sched/eevdf: Use sched_attr::sched_runtime to set request/slice suggestion References: <20230915124354.416936110@noisy.programming.kicks-ass.net> MIME-Version: 1.0 Content-Disposition: inline; filename=peterz-sched_attr-runtime.patch Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Allow applications to directly set a suggested request/slice length using sched_attr::sched_runtime. The implementation clamps the value to: 0.1[ms] <=3D slice <=3D 100[ms] which is 1/10 the size of HZ=3D1000 and 10 times the size of HZ=3D100. Applications should strive to use their periodic runtime at a high confidence interval (95%+) as the target slice. Using a smaller slice will introduce undue preemptions, while using a larger value will increase latency. For all the following examples assume a scheduling quantum of 8, and for consistency all examples have W=3D4: {A,B,C,D}(w=3D1,r=3D8): ABCD... +---+---+---+--- t=3D0, V=3D1.5 t=3D1, V=3D3.5 A |------< A |------< B |------< B |------< C |------< C |------< D |------< D |------< ---+*------+-------+--- ---+--*----+-------+--- t=3D2, V=3D5.5 t=3D3, V=3D7.5 A |------< A |------< B |------< B |------< C |------< C |------< D |------< D |------< ---+----*--+-------+--- ---+------*+-------+--- Note: 4 identical tasks in FIFO order ~~~ {A,B}(w=3D1,r=3D16) C(w=3D2,r=3D16) AACCBBCC... +---+---+---+--- t=3D0, V=3D1.25 t=3D2, V=3D5.25 A |--------------< A |--------------< B |--------------< B |--------------< C |------< C |------< ---+*------+-------+--- ---+----*--+-------+--- t=3D4, V=3D8.25 t=3D6, V=3D12.25 A |--------------< A |--------------< B |--------------< B |--------------< C |------< C |------< ---+-------*-------+--- ---+-------+---*---+--- Note: 1 heavy task -- because q=3D8, double r such that the deadline of the= w=3D2 task doesn't go below q. Note: observe the full schedule becomes: W*max(r_i/w_i) =3D 4*2q =3D 8q in = length. Note: the period of the heavy task is half the full period at: W*(r_i/w_i) =3D 4*(2q/2) =3D 4q ~~~ {A,C,D}(w=3D1,r=3D16) B(w=3D1,r=3D8): BAACCBDD... +---+---+---+--- t=3D0, V=3D1.5 t=3D1, V=3D3.5 A |--------------< A |---------------< B |------< B |------< C |--------------< C |--------------< D |--------------< D |--------------< ---+*------+-------+--- ---+--*----+-------+--- t=3D3, V=3D7.5 t=3D5, V=3D11.5 A |---------------< A |---------------< B |------< B |------< C |--------------< C |-------------= -< D |--------------< D |--------------< ---+------*+-------+--- ---+-------+--*----+--- t=3D6, V=3D13.5 A |---------------< B |------< C |--------------< D |--------------< ---+-------+----*--+--- Note: 1 short task -- again double r so that the deadline of the short task won't be below q. Made B short because its not the leftmost task, but= is eligible with the 0,1,2,3 spread. Note: like with the heavy task, the period of the short task observes: W*(r_i/w_i) =3D 4*(1q/1) =3D 4q ~~~ A(w=3D1,r=3D16) B(w=3D1,r=3D8) C(w=3D2,r=3D16) BCCAABCC... +---+---+---+--- t=3D0, V=3D1.25 t=3D1, V=3D3.25 A |--------------< A |--------------< B |------< B |------< C |------< C |------< ---+*------+-------+--- ---+--*----+-------+--- t=3D3, V=3D7.25 t=3D5, V=3D11.25 A |--------------< A |--------------< B |------< B |------< C |------< C |------< ---+------*+-------+--- ---+-------+--*----+--- t=3D6, V=3D13.25 A |--------------< B |------< C |------< ---+-------+----*--+--- Note: 1 heavy and 1 short task -- combine them all. Note: both the short and heavy task end up with a period of 4q ~~~ A(w=3D1,r=3D16) B(w=3D2,r=3D16) C(w=3D1,r=3D8) BBCAABBC... +---+---+---+--- t=3D0, V=3D1 t=3D2, V=3D5 A |--------------< A |--------------< B |------< B |------< C |------< C |------< ---+*------+-------+--- ---+----*--+-------+--- t=3D3, V=3D7 t=3D5, V=3D11 A |--------------< A |--------------< B |------< B |------< C |------< C |------< ---+------*+-------+--- ---+-------+--*----+--- t=3D7, V=3D15 A |--------------< B |------< C |------< ---+-------+------*+--- Note: as before but permuted ~~~ >From all this it can be deduced that, for the steady state: - the total period (P) of a schedule is: W*max(r_i/w_i) - the average period of a task is: W*(r_i/w_i) - each task obtains the fair share: w_i/W of each full period P Signed-off-by: Peter Zijlstra (Intel) --- include/linux/sched.h | 3 +++ kernel/sched/core.c | 33 ++++++++++++++++++++++++++------- kernel/sched/fair.c | 6 ++++-- 3 files changed, 33 insertions(+), 9 deletions(-) Index: linux-2.6/include/linux/sched.h =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- linux-2.6.orig/include/linux/sched.h +++ linux-2.6/include/linux/sched.h @@ -555,6 +555,9 @@ struct sched_entity { struct list_head group_node; unsigned int on_rq; =20 + unsigned int custom_slice : 1; + /* 31 bits hole */ + u64 exec_start; u64 sum_exec_runtime; u64 prev_sum_exec_runtime; Index: linux-2.6/kernel/sched/core.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- linux-2.6.orig/kernel/sched/core.c +++ linux-2.6/kernel/sched/core.c @@ -4501,7 +4501,6 @@ static void __sched_fork(unsigned long c p->se.nr_migrations =3D 0; p->se.vruntime =3D 0; p->se.vlag =3D 0; - p->se.slice =3D sysctl_sched_base_slice; INIT_LIST_HEAD(&p->se.group_node); =20 #ifdef CONFIG_FAIR_GROUP_SCHED @@ -4755,6 +4754,8 @@ int sched_fork(unsigned long clone_flags =20 p->prio =3D p->normal_prio =3D p->static_prio; set_load_weight(p, false); + p->se.custom_slice =3D 0; + p->se.slice =3D sysctl_sched_base_slice; =20 /* * We don't need the reset flag anymore after the fork. It has @@ -7552,10 +7553,20 @@ static void __setscheduler_params(struct =20 p->policy =3D policy; =20 - if (dl_policy(policy)) + if (dl_policy(policy)) { __setparam_dl(p, attr); - else if (fair_policy(policy)) + } else if (fair_policy(policy)) { p->static_prio =3D NICE_TO_PRIO(attr->sched_nice); + if (attr->sched_runtime) { + p->se.custom_slice =3D 1; + p->se.slice =3D clamp_t(u64, attr->sched_runtime, + NSEC_PER_MSEC/10, /* HZ=3D1000 * 10 */ + NSEC_PER_MSEC*100); /* HZ=3D100 / 10 */ + } else { + p->se.custom_slice =3D 0; + p->se.slice =3D sysctl_sched_base_slice; + } + } =20 /* * __sched_setscheduler() ensures attr->sched_priority =3D=3D 0 when @@ -7740,7 +7751,9 @@ recheck: * but store a possible modification of reset_on_fork. */ if (unlikely(policy =3D=3D p->policy)) { - if (fair_policy(policy) && attr->sched_nice !=3D task_nice(p)) + if (fair_policy(policy) && + (attr->sched_nice !=3D task_nice(p) || + (attr->sched_runtime && attr->sched_runtime !=3D p->se.slice))) goto change; if (rt_policy(policy) && attr->sched_priority !=3D p->rt_priority) goto change; @@ -7886,6 +7899,9 @@ static int _sched_setscheduler(struct ta .sched_nice =3D PRIO_TO_NICE(p->static_prio), }; =20 + if (p->se.custom_slice) + attr.sched_runtime =3D p->se.slice; + /* Fixup the legacy SCHED_RESET_ON_FORK hack. */ if ((policy !=3D SETPARAM_POLICY) && (policy & SCHED_RESET_ON_FORK)) { attr.sched_flags |=3D SCHED_FLAG_RESET_ON_FORK; @@ -8062,12 +8078,14 @@ err_size: =20 static void get_params(struct task_struct *p, struct sched_attr *attr) { - if (task_has_dl_policy(p)) + if (task_has_dl_policy(p)) { __getparam_dl(p, attr); - else if (task_has_rt_policy(p)) + } else if (task_has_rt_policy(p)) { attr->sched_priority =3D p->rt_priority; - else + } else { attr->sched_nice =3D task_nice(p); + attr->sched_runtime =3D p->se.slice; + } } =20 /** @@ -10086,6 +10104,7 @@ void __init sched_init(void) } =20 set_load_weight(&init_task, false); + init_task.se.slice =3D sysctl_sched_base_slice, =20 /* * The boot idle thread does lazy MMU switching as well: Index: linux-2.6/kernel/sched/fair.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- linux-2.6.orig/kernel/sched/fair.c +++ linux-2.6/kernel/sched/fair.c @@ -974,7 +974,8 @@ static void update_deadline(struct cfs_r * nice) while the request time r_i is determined by * sysctl_sched_base_slice. */ - se->slice =3D sysctl_sched_base_slice; + if (!se->custom_slice) + se->slice =3D sysctl_sched_base_slice; =20 /* * EEVDF: vd_i =3D ve_i + r_i / w_i @@ -4922,7 +4923,8 @@ place_entity(struct cfs_rq *cfs_rq, stru u64 vslice, vruntime =3D avg_vruntime(cfs_rq); s64 lag =3D 0; =20 - se->slice =3D sysctl_sched_base_slice; + if (!se->custom_slice) + se->slice =3D sysctl_sched_base_slice; vslice =3D calc_delta_fair(se->slice, se); =20 /*