From nobody Fri Sep 25 12:05:44 2026 Received: from m16.mail.163.com (m16.mail.163.com [220.197.31.3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E49DB3C3F4A for ; Sun, 13 Sep 2026 06:48:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=220.197.31.3 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789282136; cv=none; b=LqaiIOt7Zxn2a8IHnYO6H6uzWNDBhbkI21dTb0z9xMqTQny9lVg0sWgrAx38+IDeMUgGEgs8s9KSQBB7Mt+rI0T+RwMOdPR/HZGesKxLbWLpNDt05edUTjWY1Y/IfCZWqVLeitvhXttb68iFzg5qRl03R9tqL+iNbI0OEwQ0zDw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789282136; c=relaxed/simple; bh=9mMiQOw8U0G19b+vr2PVHFWq5kT98PYpQA9pvwG21DA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=HJf7sJjVsaDFwB8tRQrcZvYRtP60f6SeCNoipnSywt6bhIurfJBhkq3hP2LY1DiKT4cvidcgztcsMFBq5+EgA8XiBChYgcZTFjbSk9AVpmFW4iyysoDrOvN5aCEFbHBjNfZJJKyhn5jqkjvP2kbH5VNietxqQh+KDViJo2wUrtw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=163.com; spf=pass smtp.mailfrom=163.com; dkim=pass (1024-bit key) header.d=163.com header.i=@163.com header.b=HruOhVy9; arc=none smtp.client-ip=220.197.31.3 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=163.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=163.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=163.com header.i=@163.com header.b="HruOhVy9" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=163.com; s=s110527; h=From:To:Subject:Date:Message-ID:MIME-Version; bh=mY k33ME0xBHs5d9qvvC1DwhqFT9RdVTw6xajF06W/BE=; b=HruOhVy9LG4klLTNcx shyz9Pk4Bc1qm2vMMWEns+yK7QLHQi8V2NPYHyHKXpoOSp1tRViUyPLvUaeq97sp LBFDvjMZE+fZJWTIrtCNOs9fBHr1ECxl3eNPKX4mgqdO6tPpdv3bmbnG3AEL/3cO jTx1zIye+N1XDk81mM5mg8xSI= Received: from localhost (unknown []) by gzsmtp2 (Coremail) with SMTP id PSgvCgDXKosCR6ZqNeXCPw--.41105S2; Sun, 13 Sep 2026 14:47:31 +0800 (CST) From: Hui Su To: peterz@infradead.org, soolaugust@gmail.com, arighi@nvidia.com Cc: mingo@redhat.com, kprateek.nayak@amd.com, juri.lelli@redhat.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, vschneid@redhat.com, connoro@google.com, jstultz@google.com, linux-kernel@vger.kernel.org, sched-ext@lists.linux.dev Subject: [PATCH v5 1/4] sched: Dispatch task ticks for donor and execution classes Date: Sun, 13 Sep 2026 15:47:19 +0900 Message-ID: <20260913064722.1534766-2-sh_def@163.com> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260913064722.1534766-1-sh_def@163.com> References: <20260913064722.1534766-1-sh_def@163.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: PSgvCgDXKosCR6ZqNeXCPw--.41105S2 X-Coremail-Antispam: 1Uf129KBjvJXoW3ur43Zr1ftFy8Jw1kKF4DArb_yoWDtFWDpF WDXasxGa1DJa1YgryxArWkZ3WfWwn3Xa4agFZ3tw4FkF15tr1FqF1qqF42vr45JrW0vFy2 vrWq93y7Kr18G3DanT9S1TB71UUUUU7qnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDUYxBIdaVFxhVjvjDU0xZFpf9x07UthL5UUUUU= X-CM-SenderInfo: xvkbvvri6rljoofrz/xtbC6gMWdWqmRwMqkQAA35 Content-Type: text/plain; charset="utf-8" Proxy execution can run a task from one scheduling class on behalf of a donor from another. Scheduler ticks therefore need to reach both the class which owns the scheduling context and the class which owns the execution context. Remove the task argument from sched_class::task_tick() and let each class select the state it owns through the runqueue. Add a common task_tick() dispatcher which calls the donor class first and, when proxy execution splits the classes, the execution class afterwards. Calling the donor first preserves the existing runtime-accounting order for execution-context consumers. Keep the existing class-specific tick behavior donor-gated in this patch, so the change only introduces the new interface and dispatch mechanism. In particular, keep task_tick_scx() explicitly gated on the donor class. If the callback is reached only because the execution context belongs to sched_ext while another class supplies the donor, it must not perform SCX scheduling-context work. This aligns the dispatcher with the ownership direction used by the pending sched_ext proxy-execution work. Full sched_ext/proxy-execution integration is not enabled by the current Kconfig; that accounting conversion remains outside this patch. Suggested-by: Peter Zijlstra Link: https://lore.kernel.org/r/20260908104407.GD687043@noisy.programming.k= icks-ass.net Link: https://lore.kernel.org/r/aqGa0J1_LM99oDkP@gpd4 Signed-off-by: Hui Su --- kernel/sched/core.c | 22 +++++++++++++++++++--- kernel/sched/deadline.c | 10 +++++++--- kernel/sched/ext/ext.c | 22 +++++++++++++++++----- kernel/sched/fair.c | 25 +++++++++++++++---------- kernel/sched/idle.c | 8 ++++---- kernel/sched/rt.c | 13 +++++++++---- kernel/sched/sched.h | 2 +- kernel/sched/stop_task.c | 5 ++--- 8 files changed, 74 insertions(+), 33 deletions(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index b998ef6b87af..05e599665fdd 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -892,6 +892,22 @@ void update_rq_clock(struct rq *rq) update_rq_clock_task(rq, delta); } =20 +/* + * Run the scheduling-context class first so its runtime update precedes + * execution-context tick work. A different execution class runs second. + * Same-class proxy execution gets one callback; ownership-specific work + * can select rq->donor or rq->curr as appropriate. + */ +static inline void task_tick(struct rq *rq, int queued) +{ + const struct sched_class *curr_class =3D rq->curr->sched_class; + const struct sched_class *donor_class =3D rq->donor->sched_class; + + donor_class->task_tick(rq, queued); + if (sched_proxy_exec() && curr_class !=3D donor_class) + curr_class->task_tick(rq, queued); +} + #ifdef CONFIG_SCHED_HRTICK /* * Use HR-timers to deliver accurate preemption points. @@ -923,7 +939,7 @@ static enum hrtimer_restart hrtick(struct hrtimer *time= r) =20 rq_lock(rq, &rf); update_rq_clock(rq); - rq->donor->sched_class->task_tick(rq, rq->donor, 1); + task_tick(rq, 1); rq_unlock(rq, &rf); =20 return HRTIMER_NORESTART; @@ -5799,7 +5815,7 @@ void sched_tick(void) if (dynamic_preempt_lazy() && tif_test_bit(TIF_NEED_RESCHED_LAZY)) resched_curr(rq); =20 - donor->sched_class->task_tick(rq, donor, 0); + task_tick(rq, 0); if (sched_feat(LATENCY_WARN)) resched_latency =3D cpu_resched_latency(rq); calc_global_load_tick(rq); @@ -5895,7 +5911,7 @@ static void sched_tick_remote(struct work_struct *wor= k) u64 delta =3D rq_clock_task(rq) - curr->se.exec_start; WARN_ON_ONCE(delta > (u64)NSEC_PER_SEC * 30); } - curr->sched_class->task_tick(rq, curr, 0); + task_tick(rq, 0); =20 calc_load_nohz_remote(rq); } diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c index 0663c00c41c0..da7613acab18 100644 --- a/kernel/sched/deadline.c +++ b/kernel/sched/deadline.c @@ -2873,11 +2873,15 @@ static void put_prev_task_dl(struct rq *rq, struct = task_struct *p, struct task_s * * NOTE: This function can be called remotely by the tick offload that * goes along full dynticks. Therefore no local assumption can be made - * and everything must be accessed through the @rq and @curr passed in - * parameters. + * and all state must be accessed through @rq. */ -static void task_tick_dl(struct rq *rq, struct task_struct *p, int queued) +static void task_tick_dl(struct rq *rq, int queued) { + struct task_struct *p =3D rq->donor; + + if (p->sched_class !=3D &dl_sched_class) + return; + update_curr_dl(rq); =20 update_dl_rq_load_avg(rq_clock_pelt(rq), rq, 1); diff --git a/kernel/sched/ext/ext.c b/kernel/sched/ext/ext.c index 51de1d8b72a1..8c389852be6e 100644 --- a/kernel/sched/ext/ext.c +++ b/kernel/sched/ext/ext.c @@ -3789,9 +3789,21 @@ void scx_tick(struct rq *rq) update_other_load_avgs(rq); } =20 -static void task_tick_scx(struct rq *rq, struct task_struct *curr, int que= ued) +static void task_tick_scx(struct rq *rq, int queued) { - struct scx_sched *sch =3D scx_task_sched(curr); + struct task_struct *donor =3D rq->donor; + struct scx_sched *sch; + + /* + * task_tick() may also invoke this callback as the execution class + * after dispatching the donor class. SCX scheduling state belongs to + * the scheduling context, so there is nothing to do here unless the + * donor itself belongs to sched_ext. + */ + if (donor->sched_class !=3D &ext_sched_class) + return; + + sch =3D scx_task_sched(donor); =20 update_curr_scx(rq); =20 @@ -3800,11 +3812,11 @@ static void task_tick_scx(struct rq *rq, struct tas= k_struct *curr, int queued) * management. */ if (scx_bypassing(sch, cpu_of(rq))) - scx_set_task_slice(curr, 0); + scx_set_task_slice(donor, 0); else if (SCX_HAS_OP(sch, tick)) - SCX_CALL_OP_TASK(sch, tick, rq, curr); + SCX_CALL_OP_TASK(sch, tick, rq, donor); =20 - if (!curr->scx.slice) + if (!donor->scx.slice) resched_curr(rq); } =20 diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index ade1eceb39b8..6f1777799371 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -15057,12 +15057,17 @@ static inline void task_tick_core(struct rq *rq, = struct task_struct *curr) {} * * NOTE: This function can be called remotely by the tick offload that * goes along full dynticks. Therefore no local assumption can be made - * and everything must be accessed through the @rq and @curr passed in - * parameters. + * and all state must be accessed through @rq. */ -static void task_tick_fair(struct rq *rq, struct task_struct *curr, int qu= eued) +static void task_tick_fair(struct rq *rq, int queued) { - struct sched_entity *se =3D &curr->se; + struct task_struct *donor =3D rq->donor; + struct sched_entity *se; + + if (donor->sched_class !=3D &fair_sched_class) + return; + + se =3D &donor->se; =20 if (se->on_rq) { unsigned long weight =3D NICE_0_LOAD; @@ -15075,7 +15080,7 @@ static void task_tick_fair(struct rq *rq, struct ta= sk_struct *curr, int queued) weight =3D __calc_prop_weight(cfs_rq, se, weight); } =20 - se =3D &curr->se; + se =3D &donor->se; reweight_eevdf(cfs_rq, se, weight, se->on_rq); } =20 @@ -15083,14 +15088,14 @@ static void task_tick_fair(struct rq *rq, struct = task_struct *curr, int queued) return; =20 if (static_branch_unlikely(&sched_numa_balancing)) - task_tick_numa(rq, curr); + task_tick_numa(rq, donor); =20 - task_tick_cache(rq, curr); + task_tick_cache(rq, donor); =20 - update_misfit_status(curr, rq); - check_update_overutilized_status(task_rq(curr)); + update_misfit_status(donor, rq); + check_update_overutilized_status(task_rq(donor)); =20 - task_tick_core(rq, curr); + task_tick_core(rq, donor); } =20 /* diff --git a/kernel/sched/idle.c b/kernel/sched/idle.c index eb73b65ce6c4..c1e597b0912a 100644 --- a/kernel/sched/idle.c +++ b/kernel/sched/idle.c @@ -532,12 +532,12 @@ dequeue_task_idle(struct rq *rq, struct task_struct *= p, int flags) * * NOTE: This function can be called remotely by the tick offload that * goes along full dynticks. Therefore no local assumption can be made - * and everything must be accessed through the @rq and @curr passed in - * parameters. + * and all state must be accessed through @rq. */ -static void task_tick_idle(struct rq *rq, struct task_struct *curr, int qu= eued) +static void task_tick_idle(struct rq *rq, int queued) { - update_curr_idle(rq); + if (rq->donor->sched_class =3D=3D &idle_sched_class) + update_curr_idle(rq); } =20 static void switching_to_idle(struct rq *rq, struct task_struct *p) diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c index 85303add726d..dd058a6ca06b 100644 --- a/kernel/sched/rt.c +++ b/kernel/sched/rt.c @@ -2535,12 +2535,17 @@ static inline void watchdog(struct rq *rq, struct t= ask_struct *p) { } * * NOTE: This function can be called remotely by the tick offload that * goes along full dynticks. Therefore no local assumption can be made - * and everything must be accessed through the @rq and @curr passed in - * parameters. + * and all state must be accessed through @rq. */ -static void task_tick_rt(struct rq *rq, struct task_struct *p, int queued) +static void task_tick_rt(struct rq *rq, int queued) { - struct sched_rt_entity *rt_se =3D &p->rt; + struct task_struct *p =3D rq->donor; + struct sched_rt_entity *rt_se; + + if (p->sched_class !=3D &rt_sched_class) + return; + + rt_se =3D &p->rt; =20 update_curr_rt(rq); update_rt_rq_load_avg(rq_clock_pelt(rq), rq, 1); diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index e656c7059bf8..6a8deddc725b 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -2718,7 +2718,7 @@ struct sched_class { * sched_tick: rq->lock * sched_tick_remote: rq->lock */ - void (*task_tick)(struct rq *rq, struct task_struct *p, int queued); + void (*task_tick)(struct rq *rq, int queued); /* * sched_cgroup_fork: p->pi_lock */ diff --git a/kernel/sched/stop_task.c b/kernel/sched/stop_task.c index c909ca0d8c87..87b46fc74f81 100644 --- a/kernel/sched/stop_task.c +++ b/kernel/sched/stop_task.c @@ -68,10 +68,9 @@ static void put_prev_task_stop(struct rq *rq, struct tas= k_struct *prev, struct t * * NOTE: This function can be called remotely by the tick offload that * goes along full dynticks. Therefore no local assumption can be made - * and everything must be accessed through the @rq and @curr passed in - * parameters. + * and all state must be accessed through @rq. */ -static void task_tick_stop(struct rq *rq, struct task_struct *curr, int qu= eued) +static void task_tick_stop(struct rq *rq, int queued) { } =20 --=20 2.55.0 From nobody Fri Sep 25 12:05:44 2026 Received: from m16.mail.163.com (m16.mail.163.com [117.135.210.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B23E8356750 for ; Sun, 13 Sep 2026 06:48:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=117.135.210.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789282132; cv=none; b=W7XRfcVEjhRUCUIbJhPXQKU7TTjqjC4iu+M3VZpgpxJNzpD7GSWXIJA6nTsq20xE6oI7kK7eN7lTpZPA6Sjr7BrBO5JEpgPZpnBPZzVojcmvgtQvS1ehV/KUV7LrESw/4uWMZMKzuVin/XG0KLRZPiT9ZqVpMnUwuRKgaC/aE9w= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789282132; c=relaxed/simple; bh=N5ToNp5LZjfiYCgANS1U7zesxQ9cbBbdJ9qJa4nhuoc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=m84MtLC8vAbKks1SvgBiRZvSISryOEOOuXLGtvxZzZinT3ea7nG9WzS89rX2ZG+3s3o6YyTNDgTaNGH/WbldjsqvViPBmsghSmopo9J641xmMv9KPIOVGghhMNBnQktE0hXUXYKpfo0Kw8/GNFFV21/Nupr3xSfM2OnknUzfvuI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=163.com; spf=pass smtp.mailfrom=163.com; dkim=pass (1024-bit key) header.d=163.com header.i=@163.com header.b=cq/H7W41; arc=none smtp.client-ip=117.135.210.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=163.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=163.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=163.com header.i=@163.com header.b="cq/H7W41" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=163.com; s=s110527; h=From:To:Subject:Date:Message-ID:MIME-Version; bh=z6 VU2sEzMZnU3iYO4KQfIiHIv73H1TxSzgtCCUe/gSg=; b=cq/H7W41JvviZ/6lZT j746NyVWszYL6zQ0lAoUh2GNg1Hfbq7tz84ym6udd0XJZEm1+Mgpmb3DCfF20SGn npQJtaqKFLfw3nfkK47I4LaZ0Fkv/0IQVvvDfRuiovos0cP34nLd3rH19GsrtVUg kMXot8clPn273EWAnwGoaGEdI= Received: from localhost (unknown []) by gzga-smtp-mtada-g0-3 (Coremail) with SMTP id _____wD3V0ADR6ZqhX2YAQ--.37172S2; Sun, 13 Sep 2026 14:47:32 +0800 (CST) From: Hui Su To: peterz@infradead.org, soolaugust@gmail.com, arighi@nvidia.com Cc: mingo@redhat.com, kprateek.nayak@amd.com, juri.lelli@redhat.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, vschneid@redhat.com, connoro@google.com, jstultz@google.com, linux-kernel@vger.kernel.org, sched-ext@lists.linux.dev Subject: [PATCH v5 2/4] sched/numa: Drive NUMA task tick from execution context Date: Sun, 13 Sep 2026 15:47:20 +0900 Message-ID: <20260913064722.1534766-3-sh_def@163.com> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260913064722.1534766-1-sh_def@163.com> References: <20260913064722.1534766-1-sh_def@163.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: _____wD3V0ADR6ZqhX2YAQ--.37172S2 X-Coremail-Antispam: 1Uf129KBjvJXoWxJw1kGryUXF48Gr18WrW5Awb_yoW5uFW7pF Z0ka4aqrZrKa1YqF17ZrWDX3WfGwn7A347WF4DGF48ur1Ut39Y9rn3KFWSqF4YyrW0kFy2 vrWj9r17Cr1UKw7anT9S1TB71UUUUU7qnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDUYxBIdaVFxhVjvjDU0xZFpf9x07UtrchUUUUU= X-CM-SenderInfo: xvkbvvri6rljoofrz/xtbC6QQXdmqmRwSX4wAA33 Content-Type: text/plain; charset="utf-8" Proxy execution separates the scheduling context in rq->donor from the execution context in rq->curr. The task tick dispatcher can therefore invoke task_tick_fair() for either side of a cross-class proxy relationship. task_tick_numa() operates on state associated with the task actually executing, including its mm and NUMA work state. Task-level execution runtime is likewise accounted to rq->curr, and task_tick_numa() uses that runtime to drive periodic NUMA scanning. Order task_tick_fair() as one donor-owned block followed by one execution-context block. Keep the cache tick in the donor block for this patch; the following patch moves it with the other execution-context work. Run the NUMA tick only when rq->curr belongs to the fair scheduling class. The donor class tick runs first, so execution runtime is accounted before the NUMA tick consumes it. Fixes: 7de9d4f94638 ("sched: Start blocked_on chain processing in find_prox= y_task()") Suggested-by: K Prateek Nayak Suggested-by: Tim Chen Suggested-by: Peter Zijlstra Link: https://lore.kernel.org/r/20260909110348.GZ4120091@noisy.programming.= kicks-ass.net Signed-off-by: Hui Su --- kernel/sched/fair.c | 46 +++++++++++++++++++++++---------------------- 1 file changed, 24 insertions(+), 22 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 6f1777799371..002456cf9f8d 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -15061,41 +15061,43 @@ static inline void task_tick_core(struct rq *rq, = struct task_struct *curr) {} */ static void task_tick_fair(struct rq *rq, int queued) { - struct task_struct *donor =3D rq->donor; - struct sched_entity *se; + struct task_struct *curr =3D rq->curr, *donor =3D rq->donor; =20 - if (donor->sched_class !=3D &fair_sched_class) - return; + if (donor->sched_class =3D=3D &fair_sched_class) { + struct sched_entity *se =3D &donor->se; =20 - se =3D &donor->se; + if (se->on_rq) { + unsigned long weight =3D NICE_0_LOAD; + struct cfs_rq *cfs_rq; =20 - if (se->on_rq) { - unsigned long weight =3D NICE_0_LOAD; - struct cfs_rq *cfs_rq; + for_each_sched_entity(se) { + cfs_rq =3D cfs_rq_of(se); + entity_tick(cfs_rq, se, queued); =20 - for_each_sched_entity(se) { - cfs_rq =3D cfs_rq_of(se); - entity_tick(cfs_rq, se, queued); + weight =3D __calc_prop_weight(cfs_rq, se, weight); + } =20 - weight =3D __calc_prop_weight(cfs_rq, se, weight); + se =3D &donor->se; + reweight_eevdf(cfs_rq, se, weight, se->on_rq); } =20 - se =3D &donor->se; - reweight_eevdf(cfs_rq, se, weight, se->on_rq); + if (!queued) { + task_tick_cache(rq, donor); + update_misfit_status(donor, rq); + check_update_overutilized_status(task_rq(donor)); + + task_tick_core(rq, donor); + } } =20 if (queued) return; =20 - if (static_branch_unlikely(&sched_numa_balancing)) - task_tick_numa(rq, donor); - - task_tick_cache(rq, donor); - - update_misfit_status(donor, rq); - check_update_overutilized_status(task_rq(donor)); + /* Update state owned by the execution context. */ + if (curr->sched_class =3D=3D &fair_sched_class && + static_branch_unlikely(&sched_numa_balancing)) + task_tick_numa(rq, curr); =20 - task_tick_core(rq, donor); } =20 /* --=20 2.55.0 From nobody Fri Sep 25 12:05:44 2026 Received: from m16.mail.163.com (m16.mail.163.com [117.135.210.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4DE443CA497 for ; Sun, 13 Sep 2026 06:48:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=117.135.210.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789282136; cv=none; b=JkLpwv7trPg5SWu9X6ZfGXBSRm1Tkvs8ncp+7GsMYEzAFgRFxGRRlC6yl/E+YwzSvvGF4n6hDKHcL57FfMl5uIjdEM2qT8EhsOhlx6odtGn85H9gICM9RcZGGmCYp9ITZfJA+g0jTAfxboX6RSLRdanoWHURv+0i0sjGTg1dhlw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789282136; c=relaxed/simple; bh=NutYZjBxzMe66jW2zqFRhhxwy5CY16mbZyEZ+NN/89A=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=BHSFDZ/K5Kquo/JKRqVXcS/YVERzH4690rxOdMIMha7Zri5Njzpe4nhWzPKKqijEDp/uHVL7yv1LTEyQCMHHI1o+5rbTBzqweFq1U9eE/2ODYEPKMn4U25coJw+1N/pssJUNA412IviclpeTB5B/IGDVoITupg9tDTsWsYjqVW8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=163.com; spf=pass smtp.mailfrom=163.com; dkim=pass (1024-bit key) header.d=163.com header.i=@163.com header.b=hCVbDCBa; arc=none smtp.client-ip=117.135.210.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=163.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=163.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=163.com header.i=@163.com header.b="hCVbDCBa" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=163.com; s=s110527; h=From:To:Subject:Date:Message-ID:MIME-Version; bh=GU +TuUcfrWu4SxBOyCiFLSOs3WkR+5LcdV+bX5c+4nM=; b=hCVbDCBa8XtpSb2uVt 0oQsDngYAWuOTDruf1z1IN7H+bOfMKePyE8ZuvY09t+0OXpLZA1Z4tfqXITK7WLT X2iwlVSGf1Zwekyh8wK/GtJVmRLYDd57oMoFMKb6dEhhuru6Ky5rbWtEu8Ij1t2b NfUvfSofipTVvt9u/bKPat3KI= Received: from localhost (unknown []) by gzsmtp5 (Coremail) with SMTP id QCgvCgBHJh4ER6ZqaYFfRQ--.32988S2; Sun, 13 Sep 2026 14:47:33 +0800 (CST) From: Hui Su To: peterz@infradead.org, soolaugust@gmail.com, arighi@nvidia.com Cc: mingo@redhat.com, kprateek.nayak@amd.com, juri.lelli@redhat.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, vschneid@redhat.com, connoro@google.com, jstultz@google.com, linux-kernel@vger.kernel.org, sched-ext@lists.linux.dev Subject: [PATCH v5 3/4] sched/cache: Drive cache task tick from execution context Date: Sun, 13 Sep 2026 15:47:21 +0900 Message-ID: <20260913064722.1534766-4-sh_def@163.com> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260913064722.1534766-1-sh_def@163.com> References: <20260913064722.1534766-1-sh_def@163.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: QCgvCgBHJh4ER6ZqaYFfRQ--.32988S2 X-Coremail-Antispam: 1Uf129KBjvJXoW7Ar4rZr4kAryDGr4rtr15XFb_yoW8ZFyUpr WqgF4UJrWktay5ZFy8Aa9YqF1fuwn3J342gFs8Gr4F9ry5G3WrKrnYq3W2vF10yr40yF12 vr1j9r9rGw4jk3DanT9S1TB71UUUUU7qnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDUYxBIdaVFxhVjvjDU0xZFpf9x07UtrchUUUUU= X-CM-SenderInfo: xvkbvvri6rljoofrz/xtbC6QUXdmqmRwWX8QAA3l Content-Type: text/plain; charset="utf-8" Proxy execution separates the scheduling context in rq->donor from the execution context in rq->curr. Keep the fair task tick organized as one donor-owned block followed by one execution-context block. Task cache accounting is performed for rq->curr by account_mm_sched(), and cache tick handling must operate on the same execution context. With proxy execution, keeping task_tick_cache() on the donor can leave the executing task's cache state and epoch stale. Move task_tick_cache() into the execution-context section alongside the NUMA tick. The final layout has one donor-owned block containing entity, misfit, overutilized and core-scheduling work, followed by one execution-context block containing NUMA and cache work. Preserve the normal queued/hrtick gating while attributing each operation to the right context. Fixes: df0d98475954 ("sched/cache: Introduce infrastructure for cache-aware= load balancing") Suggested-by: Tim Chen Signed-off-by: Hui Su --- kernel/sched/fair.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 002456cf9f8d..ed08b287016a 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -15082,7 +15082,6 @@ static void task_tick_fair(struct rq *rq, int queue= d) } =20 if (!queued) { - task_tick_cache(rq, donor); update_misfit_status(donor, rq); check_update_overutilized_status(task_rq(donor)); =20 @@ -15094,10 +15093,12 @@ static void task_tick_fair(struct rq *rq, int que= ued) return; =20 /* Update state owned by the execution context. */ - if (curr->sched_class =3D=3D &fair_sched_class && - static_branch_unlikely(&sched_numa_balancing)) - task_tick_numa(rq, curr); + if (curr->sched_class =3D=3D &fair_sched_class) { + if (static_branch_unlikely(&sched_numa_balancing)) + task_tick_numa(rq, curr); =20 + task_tick_cache(rq, curr); + } } =20 /* --=20 2.55.0 From nobody Fri Sep 25 12:05:44 2026 Received: from m16.mail.163.com (m16.mail.163.com [117.135.210.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7BD8829AB02 for ; Sun, 13 Sep 2026 06:48:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=117.135.210.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789282133; cv=none; b=VLOIhSANgzllXcV//i8P08Y5IXd0L6GC0pim1y+ul1jwc/s2mcIrR0mNRexhOEOZtdZteg2NWN4p8e7o+ueAWde3SAXcRoex+dNb4hF373wyaXdAOkbmVBq3AOu/jrsJ3bO0ghNVw9zvLoZrrCSCnKoxReqfJCWNTCGeJDWgZI8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789282133; c=relaxed/simple; bh=e51Lw+eXyTMRoF1WNvIMXex1i56BEBjlDeaWNEuNchA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=FpkkqFTowMx8Xj1ZvJks4dnQl9a3G241k0NA3vHifP1/8cAHeg/qWYXuvfBvwDCDA8eOkL/piZnS8AmMfQz2yNwATPjdYaW0x5kpW004OW1F7KpZ9TKD8MicemfyzqUQozrgeymUhGjn3N/zejLhMnDqxSXqObFTt5qxh5rfT0w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=163.com; spf=pass smtp.mailfrom=163.com; dkim=pass (1024-bit key) header.d=163.com header.i=@163.com header.b=R0FT9v+h; arc=none smtp.client-ip=117.135.210.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=163.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=163.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=163.com header.i=@163.com header.b="R0FT9v+h" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=163.com; s=s110527; h=From:To:Subject:Date:Message-ID:MIME-Version; bh=tM jkso+o5p+Voknae42nMgD3a3sKhBWIkLuJ9y3vptM=; b=R0FT9v+hA7ndNtLtO9 BEyltfDdpSXjywrUUSiQcJdX7xRG267bMG/IHqmATuhVSHiPG8i/u61JyaUcaYrr rtanmk8uvAbpWo7aRSlqHtCM/DURHOqmzKMSIQWOX3xCiaDkIUy37t6AuzjeOotK 6+8Du3ZX9qkEG403hfdQdJlZs= Received: from localhost (unknown []) by gzga-smtp-mtada-g1-4 (Coremail) with SMTP id _____wAXqWkGR6ZqwKmwAQ--.56670S2; Sun, 13 Sep 2026 14:47:34 +0800 (CST) From: Hui Su To: peterz@infradead.org, soolaugust@gmail.com, arighi@nvidia.com Cc: mingo@redhat.com, kprateek.nayak@amd.com, juri.lelli@redhat.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, vschneid@redhat.com, connoro@google.com, jstultz@google.com, linux-kernel@vger.kernel.org, sched-ext@lists.linux.dev Subject: [PATCH v5 4/4] sched/core: Fix donor slice accounting under proxy execution Date: Sun, 13 Sep 2026 15:47:22 +0900 Message-ID: <20260913064722.1534766-5-sh_def@163.com> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260913064722.1534766-1-sh_def@163.com> References: <20260913064722.1534766-1-sh_def@163.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: _____wAXqWkGR6ZqwKmwAQ--.56670S2 X-Coremail-Antispam: 1Uf129KBjvJXoW3Xr1ruFWkur1xJr17KF4Durg_yoWxZr47pF Z0va4Yqw4Dta4jgFWDArs5Wa43K3s3J345urs8tayFkr1rt3yYqF1Sqr1jvF45Cr1vkF12 yr4q9ry7C3W7trJanT9S1TB71UUUUU7qnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDUYxBIdaVFxhVjvjDU0xZFpf9x07UCsjbUUUUU= X-CM-SenderInfo: xvkbvvri6rljoofrz/xtbC6gcXdmqmRwcqvAAA3W Content-Type: text/plain; charset="utf-8" Core scheduling uses __entity_slice_used() to decide whether the current scheduling context has consumed enough of its slice to let a force-idled SMT sibling run. The check is correctly made against rq->donor, since the slice belongs to the scheduling context. With proxy execution, however, task runtime is accounted to rq->curr. The donor's sum_exec_runtime therefore does not advance while another task executes on its behalf, causing se->sum_exec_runtime - se->prev_sum_exec_runtime to remain near zero and preventing the force-idle reschedule from triggering. Using rq->curr is not correct either, as that would compare the execution task's runtime against its own slice rather than the donor's slice. Track the donor's task-clock timestamp when it is selected and measure the elapsed service using exec_start. update_se() advances the donor's exec_start from rq_clock_task() even under proxy execution, while the accumulated task runtime itself is charged to rq->curr. Keep the selection baseline in struct rq: there is only one active donor per runqueue. Storing it in every sched_entity needlessly increases the size of the hot sched_entity structure, especially on 32-bit builds. Proxy execution can switch rq->curr from one execution owner to another without changing rq->donor. The scheduler performs a synthetic put_prev_task()/set_next_task() pair for the unchanged donor in that case. Preserve the baseline for that reselect, or service accumulated before the owner handoff would be lost. Refresh it when selecting a different donor or through an explicit set-next reactivation. Keeping the comparison in the task-clock domain also avoids depending on the entity's weight. A vruntime delta accumulated across different weights cannot reliably be compared against a slice converted using only the current weight. Only snapshot task entities, as task_tick_core() performs the consumed slice check on the donor task. In non-proxy testing, the task-clock predicate matched the existing sum_exec_runtime predicate across HZ=3D100/250/1000 and nice -10/0/+10. Under proxy execution, the donor's sum_exec_runtime delta remained zero while the task-clock delta advanced and triggered the force-idle reschedule. A focused same-donor owner-handoff test split the donor's service across two execution owners. Each owner's individual service stayed below half of the donor slice, while their accumulated service exceeded the threshold and triggered the force-idle reschedule. Both phases kept the same core_sched_start baseline. Fixes: aa4f74dfd42b ("sched: Fix runtime accounting w/ split exec & sched c= ontexts") Suggested-by: Tim Chen Signed-off-by: Hui Su --- kernel/sched/fair.c | 27 ++++++++++++++++++--------- kernel/sched/sched.h | 2 ++ 2 files changed, 20 insertions(+), 9 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index ed08b287016a..9a7be87f1991 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -6469,7 +6469,7 @@ dequeue_entity(struct cfs_rq *cfs_rq, struct sched_en= tity *se, int flags) } =20 static void -set_next_entity(struct cfs_rq *cfs_rq, struct sched_entity *se) +set_next_entity(struct cfs_rq *cfs_rq, struct sched_entity *se, bool reset= _core_slice) { /* 'current' is not kept within the tree. */ if (se->on_rq) { @@ -6502,6 +6502,10 @@ set_next_entity(struct cfs_rq *cfs_rq, struct sched_= entity *se) } =20 se->prev_sum_exec_runtime =3D se->sum_exec_runtime; +#ifdef CONFIG_SCHED_CORE + if (reset_core_slice && entity_is_task(se)) + rq_of(cfs_rq)->core_sched_start =3D se->exec_start; +#endif } =20 static bool __dequeue_task(struct rq *rq, struct task_struct *p, int flags= ); @@ -14786,16 +14790,16 @@ static void rq_offline_fair(struct rq *rq) =20 #ifdef CONFIG_SCHED_CORE static inline bool -__entity_slice_used(struct sched_entity *se, int min_nr_tasks) +__entity_slice_used(struct rq *rq, struct sched_entity *se, int min_nr_tas= ks) { - u64 rtime =3D se->sum_exec_runtime - se->prev_sum_exec_runtime; - u64 slice =3D se->slice; + /* exec_start advances with donor service under proxy execution. */ + u64 rtime =3D se->exec_start - rq->core_sched_start; =20 - return (rtime * min_nr_tasks > slice); + return (rtime * min_nr_tasks > se->slice); } =20 #define MIN_NR_TASKS_DURING_FORCEIDLE 2 -static inline void task_tick_core(struct rq *rq, struct task_struct *curr) +static inline void task_tick_core(struct rq *rq, struct task_struct *donor) { if (!sched_core_enabled(rq)) return; @@ -14815,7 +14819,7 @@ static inline void task_tick_core(struct rq *rq, st= ruct task_struct *curr) * if we need to give up the CPU. */ if (rq->core->core_forceidle_count && rq->cfs.h_nr_queued =3D=3D 1 && - __entity_slice_used(&curr->se, MIN_NR_TASKS_DURING_FORCEIDLE)) + __entity_slice_used(rq, &donor->se, MIN_NR_TASKS_DURING_FORCEIDLE)) resched_curr(rq); } =20 @@ -15049,7 +15053,7 @@ static int task_is_throttled_fair(struct task_struc= t *p, int cpu) return throttled_hierarchy(cfs_rq); } #else /* !CONFIG_SCHED_CORE: */ -static inline void task_tick_core(struct rq *rq, struct task_struct *curr)= {} +static inline void task_tick_core(struct rq *rq, struct task_struct *donor= ) {} #endif /* !CONFIG_SCHED_CORE */ =20 /* @@ -15257,6 +15261,11 @@ static void set_next_task_fair(struct rq *rq, stru= ct task_struct *p, bool first) { struct sched_entity *se =3D &p->se; bool throttled =3D false; + /* + * Reset for a new donor or reactivation, but preserve a same-donor + * proxy reselect so service before an owner handoff is retained. + */ + bool reset_core_slice =3D !first || rq->donor !=3D p; struct cfs_rq *cfs_rq =3D &rq->cfs; unsigned long weight =3D NICE_0_LOAD; bool on_rq =3D se->on_rq; @@ -15271,7 +15280,7 @@ static void set_next_task_fair(struct rq *rq, struc= t task_struct *p, bool first) =20 if (!IS_ENABLED(CONFIG_FAIR_GROUP_SCHED) || !first || !cfs_rq->h_curr) - set_next_entity(cfs_rq, se); + set_next_entity(cfs_rq, se, reset_core_slice); =20 /* ensure bandwidth has been allocated on our new cfs_rq */ throttled |=3D account_cfs_rq_runtime(cfs_rq, 0); diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 6a8deddc725b..f2f1e3642831 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -1382,6 +1382,8 @@ struct rq { unsigned int core_forceidle_seq; unsigned int core_forceidle_occupation; u64 core_forceidle_start; + /* Task-clock baseline for the current core-scheduling donor slice. */ + u64 core_sched_start; unsigned int core_pick_in_flight; #endif /* CONFIG_SCHED_CORE */ =20 --=20 2.55.0