From nobody Sun Jun 14 15:54:15 2026 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1F69838C2AE; Fri, 3 Apr 2026 12:30:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775219414; cv=none; b=tOfjCqSJ+VYKCL5OiSe/byi8wUAQ23YuuWC2hgA1aPOc+QpwXzbE7B6uvJwL49Xu5+R+OCNtZdzo4f214rXF+zBGw0SeKbWn0EhPHXxTUIqYlv1OuS1LXZwLzL/cNLI9xef/Q6LFXyBPe4MOuWgnfaJP29Vi5mFvunmKX8llTx8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775219414; c=relaxed/simple; bh=j+1KZDC8aO3UWrqsgjIlUSdSPTXO4s7fKOpvl3XV/tM=; h=Date:From:To:Subject:Cc:In-Reply-To:References:MIME-Version: Message-ID:Content-Type; b=FAI4Isuu2xJIbpSZqXp4r/WG6oMvERpxfdp9lxz+aNbr9F6yDyYQQlsEjkE1FxG0nXj5BFl5LLkQxzB0uJX4vDTbYotSwThpUU1k74MRvozICDftF/wwfc3stK7z494ikFLl2k17WkFfv+r7vgCRO0XFN5UUN9fVhInBhMO2rTU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=uC9waasa; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=d8Q63gxN; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="uC9waasa"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="d8Q63gxN" Date: Fri, 03 Apr 2026 12:30:04 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1775219405; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qY/+WgMfIY5J4QOg0+NniM2lque3sNLCL4M7r3pq0xc=; b=uC9waasa/NqRH4wAoX7yogHqnrNBZc2jAyhe08Tcwg03PO3P7BLlByZ9xva4A819h33pf6 gQG4I8i27e4EwmAqw1KYR3ZNkr4n0pOUsETauxSkEj32kDF3n3854R19ndP4Gr8Mjqf3r+ YX3lsYwj6FLEiVTwtSCRXitSQb0qRh9dQQS/whTmOyHKUVdeZ323ikwBYvPZVHyfsJX23p vgVGmtDwKGAxxhXd1UojX/jC6oVnpfvrI3Tw7sgZvk7R31Mm7TlAMyF7Ft2Rj53IXsmtOl e3E9gL5DsUTjEosMjQTHO7wRgPr6hn4gMdWtfB4VmJdGktDRAUWzSbmnN7wNkA== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1775219405; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qY/+WgMfIY5J4QOg0+NniM2lque3sNLCL4M7r3pq0xc=; b=d8Q63gxNIoal9wboIuJ566EJl1GFWg78dgVw5VieR639I0UBaC3ZkLt07SPJWx/UCxaENw IISUBxsuHmQnENCQ== From: "tip-bot2 for Vincent Guittot" Sender: tip-bot2@linutronix.de Reply-to: linux-kernel@vger.kernel.org To: linux-tip-commits@vger.kernel.org Subject: [tip: sched/core] sched/fair: Prevent negative lag increase during delayed dequeue Cc: Vincent Guittot , "Peter Zijlstra (Intel)" , x86@kernel.org, linux-kernel@vger.kernel.org In-Reply-To: <20260331162352.551501-1-vincent.guittot@linaro.org> References: <20260331162352.551501-1-vincent.guittot@linaro.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-ID: <177521940443.1647592.2249539659928731910.tip-bot2@tip-bot2> Robot-ID: Robot-Unsubscribe: Contact to get blacklisted from these emails Precedence: bulk Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable The following commit has been merged into the sched/core branch of tip: Commit-ID: 059258b0d424510202b6f2796279dbdbf0c6a83d Gitweb: https://git.kernel.org/tip/059258b0d424510202b6f2796279dbdbf= 0c6a83d Author: Vincent Guittot AuthorDate: Tue, 31 Mar 2026 18:23:52 +02:00 Committer: Peter Zijlstra CommitterDate: Fri, 03 Apr 2026 14:23:41 +02:00 sched/fair: Prevent negative lag increase during delayed dequeue Delayed dequeue feature aims to reduce the negative lag of a dequeued task while sleeping but it can happens that newly enqueued tasks will move backward the avg vruntime and increase its negative lag. When the delayed dequeued task wakes up, it has more neg lag compared to being dequeued immediately or to other tasks that have been dequeued just before theses new enqueues. Ensure that the negative lag of a delayed dequeued task doesn't increase during its delayed dequeued phase while waiting for its neg lag to diseappear. Similarly, we remove any positive lag that the delayed dequeued task could have gain during thsi period. Short slice tasks are particularly impacted in overloaded system. Test on snapdragon rb5: hackbench -T -p -l 16000000 -g 2 1> /dev/null & cyclictest -t 1 -i 2777 -D 333 --policy=3Dfair --mlock -h 20000 -q The scheduling latency of cyclictest is: tip/sched/core tip/sched/core +this patch cyclictest slice (ms) (default)2.8 8 8 hackbench slice (ms) (default)2.8 20 20 Total Samples | 115632 119733 119806 Average (us) | 364 64(-82%) 61(- 5%) Median (P50) (us) | 60 56(- 7%) 56( 0%) 90th Percentile (us) | 1166 62(-95%) 62( 0%) 99th Percentile (us) | 4192 73(-98%) 72(- 1%) 99.9th Percentile (us) | 8528 2707(-68%) 1300(-52%) Maximum (us) | 17735 14273(-20%) 13525(- 5%) Signed-off-by: Vincent Guittot Signed-off-by: Peter Zijlstra (Intel) Link: https://patch.msgid.link/20260331162352.551501-1-vincent.guittot@lina= ro.org --- kernel/sched/fair.c | 56 ++++++++++++++++++++++++++------------------ 1 file changed, 34 insertions(+), 22 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 41293d5..597ce5b 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -840,11 +840,33 @@ static s64 entity_lag(struct cfs_rq *cfs_rq, struct s= ched_entity *se, u64 avrunt return clamp(vlag, -limit, limit); } =20 -static void update_entity_lag(struct cfs_rq *cfs_rq, struct sched_entity *= se) +/* + * Delayed dequeue aims to reduce the negative lag of a dequeued task. Whi= le + * updating the lag of an entity, check that negative lag didn't increase + * during the delayed dequeue period which would be unfair. + * Similarly, check that the entity didn't gain positive lag when DELAY_ZE= RO + * is set. + * + * Return true if the lag has been adjusted. + */ +static __always_inline +bool update_entity_lag(struct cfs_rq *cfs_rq, struct sched_entity *se) { + s64 vlag =3D entity_lag(cfs_rq, se, avg_vruntime(cfs_rq)); + bool ret; + WARN_ON_ONCE(!se->on_rq); =20 - se->vlag =3D entity_lag(cfs_rq, se, avg_vruntime(cfs_rq)); + if (se->sched_delayed) { + /* previous vlag < 0 otherwise se would not be delayed */ + vlag =3D max(vlag, se->vlag); + if (sched_feat(DELAY_ZERO)) + vlag =3D min(vlag, 0); + } + ret =3D (vlag =3D=3D se->vlag); + se->vlag =3D vlag; + + return ret; } =20 /* @@ -5564,13 +5586,6 @@ static void clear_delayed(struct sched_entity *se) } } =20 -static inline void finish_delayed_dequeue_entity(struct sched_entity *se) -{ - clear_delayed(se); - if (sched_feat(DELAY_ZERO) && se->vlag > 0) - se->vlag =3D 0; -} - static bool dequeue_entity(struct cfs_rq *cfs_rq, struct sched_entity *se, int flags) { @@ -5596,6 +5611,7 @@ dequeue_entity(struct cfs_rq *cfs_rq, struct sched_en= tity *se, int flags) if (sched_feat(DELAY_DEQUEUE) && delay && !entity_eligible(cfs_rq, se)) { update_load_avg(cfs_rq, se, 0); + update_entity_lag(cfs_rq, se); set_delayed(se); return false; } @@ -5635,7 +5651,7 @@ dequeue_entity(struct cfs_rq *cfs_rq, struct sched_en= tity *se, int flags) update_cfs_group(se); =20 if (flags & DEQUEUE_DELAYED) - finish_delayed_dequeue_entity(se); + clear_delayed(se); =20 if (cfs_rq->nr_queued =3D=3D 0) { update_idle_cfs_rq_clock_pelt(cfs_rq); @@ -7084,18 +7100,14 @@ requeue_delayed_entity(struct sched_entity *se) WARN_ON_ONCE(!se->sched_delayed); WARN_ON_ONCE(!se->on_rq); =20 - if (sched_feat(DELAY_ZERO)) { - update_entity_lag(cfs_rq, se); - if (se->vlag > 0) { - cfs_rq->nr_queued--; - if (se !=3D cfs_rq->curr) - __dequeue_entity(cfs_rq, se); - se->vlag =3D 0; - place_entity(cfs_rq, se, 0); - if (se !=3D cfs_rq->curr) - __enqueue_entity(cfs_rq, se); - cfs_rq->nr_queued++; - } + if (update_entity_lag(cfs_rq, se)) { + cfs_rq->nr_queued--; + if (se !=3D cfs_rq->curr) + __dequeue_entity(cfs_rq, se); + place_entity(cfs_rq, se, 0); + if (se !=3D cfs_rq->curr) + __enqueue_entity(cfs_rq, se); + cfs_rq->nr_queued++; } =20 update_load_avg(cfs_rq, se, 0);