From nobody Fri Sep 25 19:19:54 2026 Received: from m16.mail.163.com (m16.mail.163.com [220.197.31.3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 06F7F3C4551 for ; Wed, 9 Sep 2026 09:31:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=220.197.31.3 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788946318; cv=none; b=Mk15cwLL2umF5WwtsP2GgdQ6kD8l2AzdiBKEkb7Qb4XV+svzsH5GHS2RPoRy7mkpj6sCuNgL8JPJmeT5rXk2wkb2PdCCsCYdYu4dIKDj+XwxK6IfSGguRPPAgAq6W71PWwUmlPh6mDwpRqhHxJaQl6+xrBC1bQinh7sccBGCYaQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788946318; c=relaxed/simple; bh=IoWeGBdGvnDIYBYlTg5e5KCt5rnvAX3mhuJfEFpdbB4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=FuzYIL3tMN6tauwbLOOoK3CVnMZr6x4IzOi31yZCquywRcWmrPMWLT9Mu2XhVEbSwApNo9k4pYaM0xKK4bMyOZU7z2mb/cBzs3TIe3j28vu2rSxXwR6TWIFzvcgjUmtPwXtfUv6gcY1lqV8sAD/wPtNO0ahjXk0sqYiaCj6pBxw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=163.com; spf=pass smtp.mailfrom=163.com; dkim=pass (1024-bit key) header.d=163.com header.i=@163.com header.b=nfF+uV1j; arc=none smtp.client-ip=220.197.31.3 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=163.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=163.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=163.com header.i=@163.com header.b="nfF+uV1j" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=163.com; s=s110527; h=From:To:Subject:Date:Message-ID:MIME-Version; bh=rh FFwsHpcmge2L38izgNldpMrnUT4j5IDnTv3DlM0og=; b=nfF+uV1jVP+ic2jGGj 3ekJKQQ10mSS6cR1zQpFWcfwNOUqBHewG9p4rdZAVhdRDz1G60H7F2GtuitdaHF4 naoiJBinGNjefyRGOGh+pV0SlK100RFn59Pao2AzZ9AK1KIm7ylIhJW9I2S8euhP AWJkw8aiuTaiJ2yQGXPbmBLow= Received: from localhost (unknown []) by gzsmtp4 (Coremail) with SMTP id PygvCgAXECQQJ6Fq+iIwQw--.14950S2; Wed, 09 Sep 2026 17:29:53 +0800 (CST) From: Hui Su To: peterz@infradead.org, mingo@redhat.com, tim.c.chen@linux.intel.com, yu.c.chen@intel.com, kprateek.nayak@amd.com Cc: juri.lelli@redhat.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, vschneid@redhat.com, connoro@google.com, jstultz@google.com, arighi@nvidia.com, tj@kernel.org, void@manifault.com, changwoo@igalia.com, linux-kernel@vger.kernel.org, sched-ext@lists.linux.dev Subject: [PATCH v4 1/5] sched: Dispatch task ticks for donor and execution classes Date: Wed, 9 Sep 2026 18:28:57 +0900 Message-ID: <20260909092901.2989564-2-sh_def@163.com> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260909092901.2989564-1-sh_def@163.com> References: <20260909092901.2989564-1-sh_def@163.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: PygvCgAXECQQJ6Fq+iIwQw--.14950S2 X-Coremail-Antispam: 1Uf129KBjvJXoW3AFW3JFWrXFy8CF4rWryUKFg_yoWftry7pF WDXayDGa1DJa1jgryxArWkZa1fWwn3Xa4agFZ3tw4FkF1rKr1FqF1qqF42vr45JrW0vFy2 vrWj9rW7Kr18Gw7anT9S1TB71UUUUU7qnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDUYxBIdaVFxhVjvjDU0xZFpf9x07jaq2NUUUUU= X-CM-SenderInfo: xvkbvvri6rljoofrz/xtbC6hHQMGqhJxFlHwAA3c Content-Type: text/plain; charset="utf-8" Proxy execution can run a task from one scheduling class on behalf of a donor from another. Scheduler ticks therefore need to reach both the class which owns the scheduling context and the class which owns the execution context. Remove the task argument from sched_class::task_tick() and let each class select the state it owns through the runqueue. Add a common task_tick() dispatcher which calls the donor class first and, when proxy execution splits the classes, the execution class afterwards. Calling the donor first preserves the existing runtime-accounting order for execution-context consumers. Keep the existing class-specific tick behavior donor-gated in this patch, so the change only introduces the new interface and dispatch mechanism. Suggested-by: Peter Zijlstra Link: https://lore.kernel.org/r/20260908104407.GD687043@noisy.programming.k= icks-ass.net Signed-off-by: Hui Su --- kernel/sched/core.c | 22 +++++++++++++++++++--- kernel/sched/deadline.c | 10 +++++++--- kernel/sched/ext/ext.c | 10 ++++++++-- kernel/sched/fair.c | 25 +++++++++++++++---------- kernel/sched/idle.c | 8 ++++---- kernel/sched/rt.c | 13 +++++++++---- kernel/sched/sched.h | 2 +- kernel/sched/stop_task.c | 5 ++--- 8 files changed, 65 insertions(+), 30 deletions(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index b998ef6b87af..05e599665fdd 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -892,6 +892,22 @@ void update_rq_clock(struct rq *rq) update_rq_clock_task(rq, delta); } =20 +/* + * Run the scheduling-context class first so its runtime update precedes + * execution-context tick work. A different execution class runs second. + * Same-class proxy execution gets one callback; ownership-specific work + * can select rq->donor or rq->curr as appropriate. + */ +static inline void task_tick(struct rq *rq, int queued) +{ + const struct sched_class *curr_class =3D rq->curr->sched_class; + const struct sched_class *donor_class =3D rq->donor->sched_class; + + donor_class->task_tick(rq, queued); + if (sched_proxy_exec() && curr_class !=3D donor_class) + curr_class->task_tick(rq, queued); +} + #ifdef CONFIG_SCHED_HRTICK /* * Use HR-timers to deliver accurate preemption points. @@ -923,7 +939,7 @@ static enum hrtimer_restart hrtick(struct hrtimer *time= r) =20 rq_lock(rq, &rf); update_rq_clock(rq); - rq->donor->sched_class->task_tick(rq, rq->donor, 1); + task_tick(rq, 1); rq_unlock(rq, &rf); =20 return HRTIMER_NORESTART; @@ -5799,7 +5815,7 @@ void sched_tick(void) if (dynamic_preempt_lazy() && tif_test_bit(TIF_NEED_RESCHED_LAZY)) resched_curr(rq); =20 - donor->sched_class->task_tick(rq, donor, 0); + task_tick(rq, 0); if (sched_feat(LATENCY_WARN)) resched_latency =3D cpu_resched_latency(rq); calc_global_load_tick(rq); @@ -5895,7 +5911,7 @@ static void sched_tick_remote(struct work_struct *wor= k) u64 delta =3D rq_clock_task(rq) - curr->se.exec_start; WARN_ON_ONCE(delta > (u64)NSEC_PER_SEC * 30); } - curr->sched_class->task_tick(rq, curr, 0); + task_tick(rq, 0); =20 calc_load_nohz_remote(rq); } diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c index 0663c00c41c0..da7613acab18 100644 --- a/kernel/sched/deadline.c +++ b/kernel/sched/deadline.c @@ -2873,11 +2873,15 @@ static void put_prev_task_dl(struct rq *rq, struct = task_struct *p, struct task_s * * NOTE: This function can be called remotely by the tick offload that * goes along full dynticks. Therefore no local assumption can be made - * and everything must be accessed through the @rq and @curr passed in - * parameters. + * and all state must be accessed through @rq. */ -static void task_tick_dl(struct rq *rq, struct task_struct *p, int queued) +static void task_tick_dl(struct rq *rq, int queued) { + struct task_struct *p =3D rq->donor; + + if (p->sched_class !=3D &dl_sched_class) + return; + update_curr_dl(rq); =20 update_dl_rq_load_avg(rq_clock_pelt(rq), rq, 1); diff --git a/kernel/sched/ext/ext.c b/kernel/sched/ext/ext.c index 51de1d8b72a1..70815bef5296 100644 --- a/kernel/sched/ext/ext.c +++ b/kernel/sched/ext/ext.c @@ -3789,9 +3789,15 @@ void scx_tick(struct rq *rq) update_other_load_avgs(rq); } =20 -static void task_tick_scx(struct rq *rq, struct task_struct *curr, int que= ued) +static void task_tick_scx(struct rq *rq, int queued) { - struct scx_sched *sch =3D scx_task_sched(curr); + struct task_struct *curr =3D rq->donor; + struct scx_sched *sch; + + if (curr->sched_class !=3D &ext_sched_class) + return; + + sch =3D scx_task_sched(curr); =20 update_curr_scx(rq); =20 diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index ade1eceb39b8..6f1777799371 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -15057,12 +15057,17 @@ static inline void task_tick_core(struct rq *rq, = struct task_struct *curr) {} * * NOTE: This function can be called remotely by the tick offload that * goes along full dynticks. Therefore no local assumption can be made - * and everything must be accessed through the @rq and @curr passed in - * parameters. + * and all state must be accessed through @rq. */ -static void task_tick_fair(struct rq *rq, struct task_struct *curr, int qu= eued) +static void task_tick_fair(struct rq *rq, int queued) { - struct sched_entity *se =3D &curr->se; + struct task_struct *donor =3D rq->donor; + struct sched_entity *se; + + if (donor->sched_class !=3D &fair_sched_class) + return; + + se =3D &donor->se; =20 if (se->on_rq) { unsigned long weight =3D NICE_0_LOAD; @@ -15075,7 +15080,7 @@ static void task_tick_fair(struct rq *rq, struct ta= sk_struct *curr, int queued) weight =3D __calc_prop_weight(cfs_rq, se, weight); } =20 - se =3D &curr->se; + se =3D &donor->se; reweight_eevdf(cfs_rq, se, weight, se->on_rq); } =20 @@ -15083,14 +15088,14 @@ static void task_tick_fair(struct rq *rq, struct = task_struct *curr, int queued) return; =20 if (static_branch_unlikely(&sched_numa_balancing)) - task_tick_numa(rq, curr); + task_tick_numa(rq, donor); =20 - task_tick_cache(rq, curr); + task_tick_cache(rq, donor); =20 - update_misfit_status(curr, rq); - check_update_overutilized_status(task_rq(curr)); + update_misfit_status(donor, rq); + check_update_overutilized_status(task_rq(donor)); =20 - task_tick_core(rq, curr); + task_tick_core(rq, donor); } =20 /* diff --git a/kernel/sched/idle.c b/kernel/sched/idle.c index eb73b65ce6c4..c1e597b0912a 100644 --- a/kernel/sched/idle.c +++ b/kernel/sched/idle.c @@ -532,12 +532,12 @@ dequeue_task_idle(struct rq *rq, struct task_struct *= p, int flags) * * NOTE: This function can be called remotely by the tick offload that * goes along full dynticks. Therefore no local assumption can be made - * and everything must be accessed through the @rq and @curr passed in - * parameters. + * and all state must be accessed through @rq. */ -static void task_tick_idle(struct rq *rq, struct task_struct *curr, int qu= eued) +static void task_tick_idle(struct rq *rq, int queued) { - update_curr_idle(rq); + if (rq->donor->sched_class =3D=3D &idle_sched_class) + update_curr_idle(rq); } =20 static void switching_to_idle(struct rq *rq, struct task_struct *p) diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c index 85303add726d..dd058a6ca06b 100644 --- a/kernel/sched/rt.c +++ b/kernel/sched/rt.c @@ -2535,12 +2535,17 @@ static inline void watchdog(struct rq *rq, struct t= ask_struct *p) { } * * NOTE: This function can be called remotely by the tick offload that * goes along full dynticks. Therefore no local assumption can be made - * and everything must be accessed through the @rq and @curr passed in - * parameters. + * and all state must be accessed through @rq. */ -static void task_tick_rt(struct rq *rq, struct task_struct *p, int queued) +static void task_tick_rt(struct rq *rq, int queued) { - struct sched_rt_entity *rt_se =3D &p->rt; + struct task_struct *p =3D rq->donor; + struct sched_rt_entity *rt_se; + + if (p->sched_class !=3D &rt_sched_class) + return; + + rt_se =3D &p->rt; =20 update_curr_rt(rq); update_rt_rq_load_avg(rq_clock_pelt(rq), rq, 1); diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index e656c7059bf8..6a8deddc725b 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -2718,7 +2718,7 @@ struct sched_class { * sched_tick: rq->lock * sched_tick_remote: rq->lock */ - void (*task_tick)(struct rq *rq, struct task_struct *p, int queued); + void (*task_tick)(struct rq *rq, int queued); /* * sched_cgroup_fork: p->pi_lock */ diff --git a/kernel/sched/stop_task.c b/kernel/sched/stop_task.c index c909ca0d8c87..87b46fc74f81 100644 --- a/kernel/sched/stop_task.c +++ b/kernel/sched/stop_task.c @@ -68,10 +68,9 @@ static void put_prev_task_stop(struct rq *rq, struct tas= k_struct *prev, struct t * * NOTE: This function can be called remotely by the tick offload that * goes along full dynticks. Therefore no local assumption can be made - * and everything must be accessed through the @rq and @curr passed in - * parameters. + * and all state must be accessed through @rq. */ -static void task_tick_stop(struct rq *rq, struct task_struct *curr, int qu= eued) +static void task_tick_stop(struct rq *rq, int queued) { } =20 --=20 2.55.0 From nobody Fri Sep 25 19:19:54 2026 Received: from m16.mail.163.com (m16.mail.163.com [220.197.31.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 525CD3B3BE7 for ; Wed, 9 Sep 2026 09:31:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=220.197.31.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788946323; cv=none; b=J/uMbyT+y8g9fNNfUjYbUaxHQWmYeBKpHIRivQ9YBcpkbfcUYGXmeWbMNUPG+ezofbSpxal19F6v6+ixS/Rv+70MWvllGb4zaWLRbUJYSHSdHqAQZnDK+wfm6SO1tt/fMySmGVZspYG8gOkqgE4gTh7TpRIGqMnQ/rW/dTse6Y8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788946323; c=relaxed/simple; bh=aaqpDOS9Q9cZvmbCAbQLpx2aBdRlsOav7VgVnknEK0g=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=uYWvYb2Hi0doq9vIBg4TjBarYS21nKiyUKfPjQdcALN+Nw6AKM3zuJCGD1OAQMjWFTbd47yQVB4jPm9cu0mU4Ic6ZoiJQj0DwrOo4D7OMk1A9KDn5LkYlMGdz4SNSugoVjt1/qcJmNceeM4LeDCZ1YzWFmf7PsQXb04VUSlb78o= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=163.com; spf=pass smtp.mailfrom=163.com; dkim=pass (1024-bit key) header.d=163.com header.i=@163.com header.b=E9ORiZoi; arc=none smtp.client-ip=220.197.31.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=163.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=163.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=163.com header.i=@163.com header.b="E9ORiZoi" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=163.com; s=s110527; h=From:To:Subject:Date:Message-ID:MIME-Version; bh=j6 BQax1ALxulWvxdR+klhw1jBgkWS9cMRWy4AOIlkWk=; b=E9ORiZoivaGpH5n/np x29ZqGaaYzaqd+IDAmET7IQ7ZFV7WzS96iFwUQXweCC4GNIsrILirkxiYc5l23mk efUZVjaT3UBXFNw2OHq/PRTuHG75VU8tyuSuuuB7KsCJAFJ6b3fNwuyOEDvMuSRM ZrQjvcNDTCYoouIn8mJemuITY= Received: from localhost (unknown []) by gzga-smtp-mtada-g0-2 (Coremail) with SMTP id _____wD311ARJ6FqNvVuBg--.47931S2; Wed, 09 Sep 2026 17:29:54 +0800 (CST) From: Hui Su To: peterz@infradead.org, mingo@redhat.com, tim.c.chen@linux.intel.com, yu.c.chen@intel.com, kprateek.nayak@amd.com Cc: juri.lelli@redhat.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, vschneid@redhat.com, connoro@google.com, jstultz@google.com, arighi@nvidia.com, tj@kernel.org, void@manifault.com, changwoo@igalia.com, linux-kernel@vger.kernel.org, sched-ext@lists.linux.dev Subject: [PATCH v4 2/5] sched/numa: Drive NUMA task tick from execution context Date: Wed, 9 Sep 2026 18:28:58 +0900 Message-ID: <20260909092901.2989564-3-sh_def@163.com> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260909092901.2989564-1-sh_def@163.com> References: <20260909092901.2989564-1-sh_def@163.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: _____wD311ARJ6FqNvVuBg--.47931S2 X-Coremail-Antispam: 1Uf129KBjvJXoWxJw1kGr45GFykuF4kWry8Zrb_yoW5CF1fpF s0ka4aq3y7ta1YqF17ZrWDX3WfGwn3A347WFZrGFW8ur1UK3sY9rn3KFySqF4rArW0kFy2 vrWj9r17Cr1UKw7anT9S1TB71UUUUU7qnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDUYxBIdaVFxhVjvjDU0xZFpf9x07jaUDXUUUUU= X-CM-SenderInfo: xvkbvvri6rljoofrz/xtbCwhLQMGqhJxIp4AAA3H Content-Type: text/plain; charset="utf-8" Proxy execution separates the scheduling context in rq->donor from the execution context in rq->curr. The task tick dispatcher can therefore invoke task_tick_fair() for either side of a cross-class proxy relationship. task_tick_numa() operates on state associated with the task actually executing, including its mm and NUMA work state. Task-level execution runtime is likewise accounted to rq->curr, and task_tick_numa() uses that runtime to drive periodic NUMA scanning. Split task_tick_fair() into donor and execution-context sections and run the NUMA tick only when rq->curr belongs to the fair scheduling class. The donor class tick runs first, so execution runtime is accounted before the NUMA tick consumes it. Fixes: 7de9d4f94638 ("sched: Start blocked_on chain processing in find_prox= y_task()") Suggested-by: K Prateek Nayak Suggested-by: Tim Chen Signed-off-by: Hui Su --- kernel/sched/fair.c | 44 +++++++++++++++++++++++--------------------- 1 file changed, 23 insertions(+), 21 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 6f1777799371..a8c7a9a29ace 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -15061,41 +15061,43 @@ static inline void task_tick_core(struct rq *rq, = struct task_struct *curr) {} */ static void task_tick_fair(struct rq *rq, int queued) { - struct task_struct *donor =3D rq->donor; - struct sched_entity *se; + struct task_struct *curr =3D rq->curr, *donor =3D rq->donor; =20 - if (donor->sched_class !=3D &fair_sched_class) - return; + if (donor->sched_class =3D=3D &fair_sched_class) { + struct sched_entity *se =3D &donor->se; =20 - se =3D &donor->se; + if (se->on_rq) { + unsigned long weight =3D NICE_0_LOAD; + struct cfs_rq *cfs_rq; =20 - if (se->on_rq) { - unsigned long weight =3D NICE_0_LOAD; - struct cfs_rq *cfs_rq; + for_each_sched_entity(se) { + cfs_rq =3D cfs_rq_of(se); + entity_tick(cfs_rq, se, queued); =20 - for_each_sched_entity(se) { - cfs_rq =3D cfs_rq_of(se); - entity_tick(cfs_rq, se, queued); + weight =3D __calc_prop_weight(cfs_rq, se, weight); + } =20 - weight =3D __calc_prop_weight(cfs_rq, se, weight); + se =3D &donor->se; + reweight_eevdf(cfs_rq, se, weight, se->on_rq); } - - se =3D &donor->se; - reweight_eevdf(cfs_rq, se, weight, se->on_rq); } =20 if (queued) return; =20 - if (static_branch_unlikely(&sched_numa_balancing)) - task_tick_numa(rq, donor); + /* Update state owned by the execution context. */ + if (curr->sched_class =3D=3D &fair_sched_class && + static_branch_unlikely(&sched_numa_balancing)) + task_tick_numa(rq, curr); =20 - task_tick_cache(rq, donor); + if (donor->sched_class =3D=3D &fair_sched_class) { + task_tick_cache(rq, donor); =20 - update_misfit_status(donor, rq); - check_update_overutilized_status(task_rq(donor)); + update_misfit_status(donor, rq); + check_update_overutilized_status(task_rq(donor)); =20 - task_tick_core(rq, donor); + task_tick_core(rq, donor); + } } =20 /* --=20 2.55.0 From nobody Fri Sep 25 19:19:54 2026 Received: from m16.mail.163.com (m16.mail.163.com [220.197.31.3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2612518859B for ; Wed, 9 Sep 2026 09:32:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=220.197.31.3 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788946335; cv=none; b=YQKIbKvRRORVljjPIyJ+3KOu3ZRHfu6pa3tt6aMysAjyGTCHk0cgMRVpthmaoMjbQbO/B+OStlzDAkOlJn5OoihyMvESnBC+L4y7qmusUMiA/1QNRiJoQKXg0ldrUdcB6mfQ/wW6ptFv0SAxQHZJNKSRJqAaE/4AmjZpLMpwhPM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788946335; c=relaxed/simple; bh=dzp/cz6XtJEb7AshXqW047SmmM7FxkI0NQk8T/5SQ7Q=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=B4iwsJhyitB+4yrvOp0PK6WowSmKAd3oDwq6OYP5/eF6e/6MlhCUYCVsiLfg6/Lpw9kXfSkAlGH9GJPRIcBArA4FDU272fYhrkAqUS6BD+Jx9irX1RWH2yP7FMb8JpTwR3TfVKFdeSczY25o9uF3JWudiKxizD1LaQ61IIOBfc0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=163.com; spf=pass smtp.mailfrom=163.com; dkim=pass (1024-bit key) header.d=163.com header.i=@163.com header.b=gnGUSYSo; arc=none smtp.client-ip=220.197.31.3 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=163.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=163.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=163.com header.i=@163.com header.b="gnGUSYSo" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=163.com; s=s110527; h=From:To:Subject:Date:Message-ID:MIME-Version; bh=Ih 1JQwfZlCrnNtHvDKt/ccOFDKngz150nEn9NUT9v14=; b=gnGUSYSo7CNn1TZGQG wt2lcUvEiabg4mwlR6iH15zfFbjxwSghwmjtHEavdREXiSjQIlcdFxgceBgBRX3k lZv5zHHQgdlo6PJGdofAbvy/87mhrulu7jV1k8dh4yZJxDj4p5ANGGuyXNtfFHtS qmOgU+YgQqIhbI0VONVpYLsN0= Received: from localhost (unknown []) by gzsmtp5 (Coremail) with SMTP id QCgvCgDXfysSJ6FqovnTQg--.10264S2; Wed, 09 Sep 2026 17:29:55 +0800 (CST) From: Hui Su To: peterz@infradead.org, mingo@redhat.com, tim.c.chen@linux.intel.com, yu.c.chen@intel.com, kprateek.nayak@amd.com Cc: juri.lelli@redhat.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, vschneid@redhat.com, connoro@google.com, jstultz@google.com, arighi@nvidia.com, tj@kernel.org, void@manifault.com, changwoo@igalia.com, linux-kernel@vger.kernel.org, sched-ext@lists.linux.dev Subject: [PATCH v4 3/5] sched/cache: Drive cache task tick from execution context Date: Wed, 9 Sep 2026 18:28:59 +0900 Message-ID: <20260909092901.2989564-4-sh_def@163.com> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260909092901.2989564-1-sh_def@163.com> References: <20260909092901.2989564-1-sh_def@163.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: QCgvCgDXfysSJ6FqovnTQg--.10264S2 X-Coremail-Antispam: 1Uf129KBjvJXoW7uFy5ZrWxGrW7Jr1rWF48JFb_yoW8CrW3pr WDuF47Wr4rta15tFWxZan5XF1rW3s7A342gF4DGF4rur1rG34rKr18ta1agFWjyryFkF9F vryj9ry7Gr4jk3DanT9S1TB71UUUUU7qnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDUYxBIdaVFxhVjvjDU0xZFpf9x07jaUDXUUUUU= X-CM-SenderInfo: xvkbvvri6rljoofrz/xtbC6RPQMGqhJxObewAA3F Content-Type: text/plain; charset="utf-8" Cache-aware scheduling accounts CPU runtime to the mm of the task actually executing. update_se() passes rq->curr to account_mm_sched() for this purpose. With proxy execution, task_tick_cache() still runs for the scheduling context in rq->donor. When a fair task executes on behalf of an RT or deadline donor, its mm runtime advances but its cache scan epoch is not driven. This can cause account_mm_sched() to invalidate the mm's preferred LLC. Run task_tick_cache() from the fair execution-context section of task_tick_fair(), alongside NUMA tick handling, so cache work and runtime accounting refer to the same task and mm. Fixes: df0d98475954 ("sched/cache: Introduce infrastructure for cache-aware= load balancing") Suggested-by: Tim Chen Signed-off-by: Hui Su --- kernel/sched/fair.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index a8c7a9a29ace..43d558289856 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -15086,13 +15086,15 @@ static void task_tick_fair(struct rq *rq, int que= ued) return; =20 /* Update state owned by the execution context. */ - if (curr->sched_class =3D=3D &fair_sched_class && - static_branch_unlikely(&sched_numa_balancing)) - task_tick_numa(rq, curr); + if (curr->sched_class =3D=3D &fair_sched_class) { + if (static_branch_unlikely(&sched_numa_balancing)) + task_tick_numa(rq, curr); =20 - if (donor->sched_class =3D=3D &fair_sched_class) { - task_tick_cache(rq, donor); + task_tick_cache(rq, curr); + } =20 + /* Update state owned by the scheduling context. */ + if (donor->sched_class =3D=3D &fair_sched_class) { update_misfit_status(donor, rq); check_update_overutilized_status(task_rq(donor)); =20 --=20 2.55.0 From nobody Fri Sep 25 19:19:54 2026 Received: from m16.mail.163.com (m16.mail.163.com [117.135.210.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 949F23E49E4 for ; Wed, 9 Sep 2026 09:32:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=117.135.210.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788946339; cv=none; b=QwQg87atE+rS0Xf2vxBEgaKXaWsJE76oIZam6EPIXVtYMzFcVTsBAaxYAFIg7wRYwiDg43Un5J2JPwL103TfOH0hJJ3qoqcRLPNvMVSs3pztFviZLfbTysI3kH/s03gm6DO2zR50VjfClEGq5OzK5qPyOQi+5wdeVf1+ePiqqs4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788946339; c=relaxed/simple; bh=Dz4FveT8OUkLOszhSNpDynyKbpeqEK0RRW5tge5WD5Q=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Yb8hF737HgoIoxO/qCsosNjBFRRvXiuyvSbJd9soQ9YIM9gzNcAk9G4x0XlatVfCOr/Ici3tP1Rjm+GvgfrZUOEcUOM+v9VQWKcI3h+8hopucEwAVCf4qs3H6Ig0hlCqMinxxbQYYuIIY0b6jIjlZETsXigKlYXABKoxXX2S6sI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=163.com; spf=pass smtp.mailfrom=163.com; dkim=pass (1024-bit key) header.d=163.com header.i=@163.com header.b=o5WRQfdp; arc=none smtp.client-ip=117.135.210.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=163.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=163.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=163.com header.i=@163.com header.b="o5WRQfdp" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=163.com; s=s110527; h=From:To:Subject:Date:Message-ID:MIME-Version; bh=hu gNHo8qlNHoP/MVhiB2gdPt4TVaEftzdU5dpBV1j8s=; b=o5WRQfdpGRPCxkR6uZ l8kwU3jyHwHqa6kOX/3k8sHN4SHeV6jXQ4s5rL+I8Pu9KXRHjfei3rU6u6H73OCR Uht4bs97Mw7AM+1iDzosHToQWO9SGgjnCdXH39pqAayKr2acaJ2xrTgSy8Bs1RGH qWMrJFxB3+S2hJNMB6CqilUvE= Received: from localhost (unknown []) by gzsmtp3 (Coremail) with SMTP id PigvCgAXJfwUJ6FqRggZRw--.47966S2; Wed, 09 Sep 2026 17:29:56 +0800 (CST) From: Hui Su To: peterz@infradead.org, mingo@redhat.com, tim.c.chen@linux.intel.com, yu.c.chen@intel.com, kprateek.nayak@amd.com Cc: juri.lelli@redhat.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, vschneid@redhat.com, connoro@google.com, jstultz@google.com, arighi@nvidia.com, tj@kernel.org, void@manifault.com, changwoo@igalia.com, linux-kernel@vger.kernel.org, sched-ext@lists.linux.dev Subject: [PATCH v4 4/5] sched/rt: Fix RT watchdog accounting for proxy execution Date: Wed, 9 Sep 2026 18:29:00 +0900 Message-ID: <20260909092901.2989564-5-sh_def@163.com> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260909092901.2989564-1-sh_def@163.com> References: <20260909092901.2989564-1-sh_def@163.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: PigvCgAXJfwUJ6FqRggZRw--.47966S2 X-Coremail-Antispam: 1Uf129KBjvJXoWxJw1kGr43Wr1kWF47tFykuFg_yoWrtr18pF ZY9a45Jw1DKFWUKFyxAw4DW3WfGwn3Jay7WF4DJryrAFn8Kr1Fgr1vv34avFW5Cr1SvFya vF4jv3yxC34jyrJanT9S1TB71UUUUU7qnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDUYxBIdaVFxhVjvjDU0xZFpf9x07j3Z2-UUUUU= X-CM-SenderInfo: xvkbvvri6rljoofrz/xtbCwhXRMWqhJxUqGgAA3+ Content-Type: text/plain; charset="utf-8" Proxy execution separates the scheduling context in rq->donor from the execution context in rq->curr. When rq->donor belongs to the RT class, task_tick_rt() keeps RT scheduling-class state, load tracking, and RR time-slice management associated with rq->donor. The RT watchdog is different. It looks up RLIMIT_RTTIME through its task argument and updates that task's rt.timeout and posix_cputimers state. update_curr_rt(), however, accounts elapsed task runtime to rq->curr, and run_posix_cpu_timers() checks the execution task after the scheduler tick. Keep the RT scheduling work on rq->donor, but run watchdog() for rq->curr. The watchdog state then follows the task whose execution runtime advances. The callback can also be dispatched for an RT execution context whose donor belongs to another scheduling class. In that case, run only the watchdog for rq->curr, without applying RT donor accounting or RR time-slice management. Reset rt.timeout when a task blocks under proxy execution. A mutex-blocked task can remain on the runqueue and bypass ENQUEUE_WAKEUP, which normally resets the RT watchdog interval. Also reset it when a task without an RT policy is subsequently selected with neither the execution nor scheduling context in the RT class. Preserve the timeout across ordinary scheduler preemption and when an RT-policy task is temporarily PI-boosted into the deadline class. The donor/curr attribution was checked with both the in-kernel mutex reproducer and a userspace owner in a two-node QEMU guest. FIFO and RR donors produced donor-targeted watchdog traces on the baseline kernel and execution-owner-targeted traces with this change; both proxy runs completed successfully. A DL donor with an RT execution owner was also exercised; the execution owner's timeout advanced and SIGXCPU was delivered to it, while an RR execution owner's rt.time_slice remained unchanged in execution-only callbacks. A targeted rtmutex test set rt.timeout to 123 on a SCHED_FIFO owner. A DL waiter then PI-boosted it into the deadline class. Without the policy guard, scheduling the boosted owner reset the timeout to zero. With the guard, it remained 123 during the boost and after deboosting. A FAIR owner leaving an RT proxy interval still reset its timeout from 123 to zero. Fixes: 7de9d4f94638 ("sched: Start blocked_on chain processing in find_prox= y_task()") Signed-off-by: Hui Su --- kernel/sched/core.c | 18 ++++++++++++++++++ kernel/sched/rt.c | 12 ++++++++++-- 2 files changed, 28 insertions(+), 2 deletions(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 05e599665fdd..d8a785bec639 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -6768,6 +6768,14 @@ static bool try_to_block_task(struct rq *rq, struct = task_struct *p, return false; } =20 + /* + * Proxy execution can keep a mutex-blocked task on the runqueue, so it + * may not pass through ENQUEUE_WAKEUP, which normally resets the RT + * watchdog interval. + */ + if (sched_proxy_exec() && p->rt.timeout) + p->rt.timeout =3D 0; + p->is_blocked =3D 1; =20 /* @@ -7243,6 +7251,16 @@ static void __sched notrace __schedule(int sched_mod= e) rq_set_donor(rq, next); } =20 + /* + * End a previous RT proxy watchdog interval once neither context is + * in the RT class. Preserve the interval for an RT-policy task that is + * temporarily PI-boosted into the DL class. + */ + if (sched_proxy_exec() && !task_has_rt_policy(next) && + !rt_prio(next->prio) && + !rt_prio(rq->donor->prio) && next->rt.timeout) + next->rt.timeout =3D 0; + picked: clear_tsk_need_resched(prev); clear_preempt_need_resched(); diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c index dd058a6ca06b..0fdb8edb7528 100644 --- a/kernel/sched/rt.c +++ b/kernel/sched/rt.c @@ -2542,15 +2542,23 @@ static void task_tick_rt(struct rq *rq, int queued) struct task_struct *p =3D rq->donor; struct sched_rt_entity *rt_se; =20 - if (p->sched_class !=3D &rt_sched_class) + if (p->sched_class !=3D &rt_sched_class) { + /* + * The RT callback can also be dispatched for an RT execution + * context whose scheduling context belongs to another class. + * Keep the watchdog tied to the task whose runtime is advancing. + */ + if (rq->curr->sched_class =3D=3D &rt_sched_class) + watchdog(rq, rq->curr); return; + } =20 rt_se =3D &p->rt; =20 update_curr_rt(rq); update_rt_rq_load_avg(rq_clock_pelt(rq), rq, 1); =20 - watchdog(rq, p); + watchdog(rq, rq->curr); =20 /* * RR tasks need a special form of time-slice management. --=20 2.55.0 From nobody Fri Sep 25 19:19:54 2026 Received: from m16.mail.163.com (m16.mail.163.com [220.197.31.2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 872FD492E5C for ; Wed, 9 Sep 2026 09:31:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=220.197.31.2 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788946328; cv=none; b=q0xj6wPrhYMjtvNryoD4G6NN+Q4MdIi+ipz9cse086NsIPt8kNSRv/08VjzvoXpE4azD7dxOF/A5GNsTncHUWonRmEFVKrfMrh9hydfCzS3bwrIfTLXc6n4mnmmQrUI4170JK+weK5tFsSikvFxeMLXOrNeDKE9lT7c6tjk4saY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788946328; c=relaxed/simple; bh=sZtOrM6LqEaGZnj/DA+GuR8Qt+PbicPLN3ePrrNAQf0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=IXVgFqvu8al8jkQg5HESiZfUTk7D1HCs8TOwcg65TgwfXvGkLG8IhXcDXLes5HxEUKoELY7tK1u6W7JctUZePBM7gcf64B/JJSMAlLNnB5MhgyzDx/4uTM0p+/7WhB2pGykivntEAFFvqkxQZnPdGIaA876cL9MFNitqJxXEKMY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=163.com; spf=pass smtp.mailfrom=163.com; dkim=pass (1024-bit key) header.d=163.com header.i=@163.com header.b=jQPh8NY5; arc=none smtp.client-ip=220.197.31.2 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=163.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=163.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=163.com header.i=@163.com header.b="jQPh8NY5" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=163.com; s=s110527; h=From:To:Subject:Date:Message-ID:MIME-Version; bh=Lx IMRscRls6xcpSeU4En25ja11Xm3g7TFdEKV5Km54g=; b=jQPh8NY5VdHOiLo2J7 Lykdx/TZoJTv+EItjWsJa86951Z1nnayvz2egb0CcxNryoRodsiVImMvYeNrzIPZ diiMPMhj1yjl8S9HGiNzjHf6s03DXPmADqckBNpjcE2dB0XV9lyU0BtzoQNyDL2x qqJyKYWAysY0q/IINqwL/dcH8= Received: from localhost (unknown []) by gzga-smtp-mtada-g1-4 (Coremail) with SMTP id _____wDnr5sVJ6FqvPS5Bg--.63400S2; Wed, 09 Sep 2026 17:29:58 +0800 (CST) From: Hui Su To: peterz@infradead.org, mingo@redhat.com, tim.c.chen@linux.intel.com, yu.c.chen@intel.com, kprateek.nayak@amd.com Cc: juri.lelli@redhat.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, vschneid@redhat.com, connoro@google.com, jstultz@google.com, arighi@nvidia.com, tj@kernel.org, void@manifault.com, changwoo@igalia.com, linux-kernel@vger.kernel.org, sched-ext@lists.linux.dev Subject: [PATCH v4 5/5] sched/core: Fix donor slice accounting under proxy execution Date: Wed, 9 Sep 2026 18:29:01 +0900 Message-ID: <20260909092901.2989564-6-sh_def@163.com> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260909092901.2989564-1-sh_def@163.com> References: <20260909092901.2989564-1-sh_def@163.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: _____wDnr5sVJ6FqvPS5Bg--.63400S2 X-Coremail-Antispam: 1Uf129KBjvJXoWxXw1rAw43ZF47Xw15ZrW3trb_yoW7JrWfpF Z8ZF1UJw4kta4jgFWUArs5u343K3s3ta45ur4vyayrCr1rtw4FqF10qF1avF4Ykr92kFya vr4I9ry7Cw1Utr7anT9S1TB71UUUUU7qnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDUYxBIdaVFxhVjvjDU0xZFpf9x07j3Xo7UUUUU= X-CM-SenderInfo: xvkbvvri6rljoofrz/xtbCwRbRMWqhJxYW9AAA3v Content-Type: text/plain; charset="utf-8" Core scheduling uses __entity_slice_used() to decide whether the current scheduling context has consumed enough of its slice to let a force-idled SMT sibling run. The check is correctly made against rq->donor, since the slice belongs to the scheduling context. With proxy execution, however, task runtime is accounted to rq->curr. The donor's sum_exec_runtime therefore does not advance while another task executes on its behalf, causing se->sum_exec_runtime - se->prev_sum_exec_runtime to remain near zero and preventing the force-idle reschedule from triggering. Using rq->curr is not correct either, as that would compare the execution task's runtime against its own slice rather than the donor's slice. Track the donor's task-clock timestamp when it is selected and measure the elapsed service using se->exec_start. update_se() advances the donor's exec_start from rq_clock_task() even under proxy execution, while the accumulated task runtime itself is charged to rq->curr. Keeping the comparison in the task-clock domain also avoids depending on the entity's weight. A vruntime delta accumulated across different weights cannot reliably be compared against a slice converted using only the current weight. Only snapshot task entities, as task_tick_core() performs the consumed slice check on the donor task. In non-proxy testing, the task-clock predicate matched the existing sum_exec_runtime predicate across HZ=3D100/250/1000 and nice -10/0/+10. Under proxy execution, the donor's sum_exec_runtime delta remained zero while the task-clock delta advanced and triggered the force-idle reschedule. Fixes: aa4f74dfd42b ("sched: Fix runtime accounting w/ split exec & sched c= ontexts") Suggested-by: Tim Chen Signed-off-by: Hui Su --- include/linux/sched.h | 3 +++ kernel/sched/core.c | 3 +++ kernel/sched/fair.c | 15 +++++++++------ 3 files changed, 15 insertions(+), 6 deletions(-) diff --git a/include/linux/sched.h b/include/linux/sched.h index 8b3d47a325cc..c32d9931129f 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -590,6 +590,9 @@ struct sched_entity { u64 sum_exec_runtime; u64 prev_sum_exec_runtime; u64 vruntime; +#ifdef CONFIG_SCHED_CORE + u64 core_sched_start; +#endif /* Approximated virtual lag: */ s64 vlag; /* 'Protected' deadline, to give out minimum quantums: */ diff --git a/kernel/sched/core.c b/kernel/sched/core.c index d8a785bec639..87178ec08392 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -4602,6 +4602,9 @@ static void __sched_fork(u64 clone_flags, struct task= _struct *p) p->se.prev_sum_exec_runtime =3D 0; p->se.nr_migrations =3D 0; p->se.vruntime =3D 0; +#ifdef CONFIG_SCHED_CORE + p->se.core_sched_start =3D 0; +#endif p->se.vlag =3D 0; p->se.rel_deadline =3D 0; INIT_LIST_HEAD(&p->se.group_node); diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 43d558289856..0528dc4846f3 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -6502,6 +6502,10 @@ set_next_entity(struct cfs_rq *cfs_rq, struct sched_= entity *se) } =20 se->prev_sum_exec_runtime =3D se->sum_exec_runtime; +#ifdef CONFIG_SCHED_CORE + if (entity_is_task(se)) + se->core_sched_start =3D se->exec_start; +#endif } =20 static bool __dequeue_task(struct rq *rq, struct task_struct *p, int flags= ); @@ -14788,14 +14792,13 @@ static void rq_offline_fair(struct rq *rq) static inline bool __entity_slice_used(struct sched_entity *se, int min_nr_tasks) { - u64 rtime =3D se->sum_exec_runtime - se->prev_sum_exec_runtime; - u64 slice =3D se->slice; + u64 rtime =3D se->exec_start - se->core_sched_start; =20 - return (rtime * min_nr_tasks > slice); + return (rtime * min_nr_tasks > se->slice); } =20 #define MIN_NR_TASKS_DURING_FORCEIDLE 2 -static inline void task_tick_core(struct rq *rq, struct task_struct *curr) +static inline void task_tick_core(struct rq *rq, struct task_struct *donor) { if (!sched_core_enabled(rq)) return; @@ -14815,7 +14818,7 @@ static inline void task_tick_core(struct rq *rq, st= ruct task_struct *curr) * if we need to give up the CPU. */ if (rq->core->core_forceidle_count && rq->cfs.h_nr_queued =3D=3D 1 && - __entity_slice_used(&curr->se, MIN_NR_TASKS_DURING_FORCEIDLE)) + __entity_slice_used(&donor->se, MIN_NR_TASKS_DURING_FORCEIDLE)) resched_curr(rq); } =20 @@ -15049,7 +15052,7 @@ static int task_is_throttled_fair(struct task_struc= t *p, int cpu) return throttled_hierarchy(cfs_rq); } #else /* !CONFIG_SCHED_CORE: */ -static inline void task_tick_core(struct rq *rq, struct task_struct *curr)= {} +static inline void task_tick_core(struct rq *rq, struct task_struct *donor= ) {} #endif /* !CONFIG_SCHED_CORE */ =20 /* --=20 2.55.0