From nobody Mon Jun 15 13:57:38 2026 Received: from mail-wm1-f48.google.com (mail-wm1-f48.google.com [209.85.128.48]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7054038736C for ; Wed, 22 Apr 2026 09:34:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.48 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776850449; cv=none; b=kZDih1Ybz8oZeG0zyF3WRIKblhk1U4SqfG4wUV1Kvl0jtK3ZCeua71xQTSm0Hmkw94mnrcsHJUk7Dj0QDmjVOeNHnKBkPizlERC9jl7Bon7Rcjl/K9MXG/GZc4jBszEo9eTooX3UpaPZBg8kxyp3Qv9T55btvw2sNW2ry4OB3lM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776850449; c=relaxed/simple; bh=kDtbpuido9izU9GEQOOkApQMdG0UqrjEqGK7AyjGPQU=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version; b=It2nRzKUAEzC0ecP71oAf4lq+CNI/Q8eXrecHjWJDP+2MdjsUUIVLTKhb5B0ZD2CEErh1Ekoj7fxKEvGzKC4Kmgo/ZBgMG+/Wk44XagKjZiOZNgGLfsCCJdx8VP5j0/xPv3AuAEDzkRKSJoE0pdGzHZ2rHQlqvaUjA5QbXi8bXY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=PjCAuX/w; arc=none smtp.client-ip=209.85.128.48 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="PjCAuX/w" Received: by mail-wm1-f48.google.com with SMTP id 5b1f17b1804b1-488a9033b2cso58759425e9.2 for ; Wed, 22 Apr 2026 02:34:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1776850444; x=1777455244; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=L/b1zQGsWQ32VVaOxYVizoAINLnRXBs1H/znp0bYMGE=; b=PjCAuX/wzb/1pzDZirBgnNmCk9Jb8Z0LVv0DzTmA0hRtdctu/3W7AvEVy5JpRQmDCW rVA0nP9Lsq1ONvTkS1+AsCXgGE0Hx7y8wMFdDBouTRZEGwom5rKFKx06wipaALt5exyP HqFtqD9UQvFEEqDqhOlwW7Ud/XSkreM6glvGZ6Dw5arlHr+0BpzK9SKubP9ac91BMbDG f9Vjo/ctSwpNYGyBDUA8oUtfkx3zdPHRf2nlq9eNP5KRFi/imBVAhoH7PVlOFUFyPgXU IB5pu5bGotudthGKYxUGnsTpfvDTH93wPTGTUYGa0dhnvP7TWADrl3khX5SP01jNhxMX asoQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1776850444; x=1777455244; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-gg:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=L/b1zQGsWQ32VVaOxYVizoAINLnRXBs1H/znp0bYMGE=; b=nExM9jwdRszo+WZRt0LlcyGaHjvATzYLTFCDWqqDw8LIzMp29dfOxW6V2w6bg1EuB3 1Q3bkE8vxWGBdL3pu7yMVnMIgxRcwyFOZ55V99XQ2mmOaFa59yn96wk2Z2qXkEUptFrt QIKDdzOEGBEPzNJzFIQFmkaB9akAA/pivePFFr38UkA+l209Ox+9e+jrK8L9kZ89C1xJ SgUBm5omN4cRBLDRp02y9ajhwPMV99VK/47YESki+LCN93r1Z7gxU8U3AGtdI1IX/Zlc bz1UV7VQf7FWjA1tITr0V3fG/k2j4/SUUlrYy8xYiSW7SsRF2p6asRdXH/rCE9ZMUcPv l1Zw== X-Forwarded-Encrypted: i=1; AFNElJ/QROqYS31OBOR5xV2ZgG5aE1EZmCp3D8lOUCZFYSmOWPQhReHIBeb5K+H2P3BLBW+z0dbwUynpH9uE5nI=@vger.kernel.org X-Gm-Message-State: AOJu0Yw5m/DtN9/rHK+2FD654kl01WTQGMaXN1YlSJee0GeW9+nKXGql Ik7eafeX9HH67Gab58GdEZvTZy2+dP8f0JdSLR4wURThAEnKJRhqI0bTQzH4CrFW7Lc= X-Gm-Gg: AeBDies5hjlxX85i+xw74RBZBuuPgmFZNjfS3ruBDfe8prDImGMhuFC7Sr4pgrvOmhP Y+ZFcGr4uGH4WClxHWHIvLRR7pI8fLk5rRz9/FJJPpGxT0hMUafj8XrohZgeLQFcer0ocuRIv1u Z3PMRCs5qM6YT3hVHXtKN/j6l31ffT87aVbEsnT34Te/F62LYIYWWuLJo5adgIsKhH/4/L/K5lE 1TeqF1xwlCpipBmdga6R2Sab4x6oC8rPctMZKShHfo6475KEUN2qrbSXx1n0gPCRouC2zPAKKU/ lNwmjUN6KJKK1cRqk+WYWbK0DNX7pDhJq9QBBiZlyYPQPnavowSEbv9mknmkeDnfsNXuGQvUtVC UISPNWnRcbMMozFcKZrGRPNWs6toYMws8BuUuNW8VU3aBipDB6Bp4bFp0aBfR9plN9XHb49IaUz hB58y/tuPeKhyWDSGoc5EfNCFxgvV//aW6jtKLaPSoow== X-Received: by 2002:a05:600c:c4a2:b0:488:c078:bfda with SMTP id 5b1f17b1804b1-488fb78eebdmr298179455e9.26.1776850444326; Wed, 22 Apr 2026 02:34:04 -0700 (PDT) Received: from vingu-cube.. ([2a01:e0a:f:6020:435:2a79:fa7c:acff]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-488fc0f8188sm459538285e9.2.2026.04.22.02.34.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 22 Apr 2026 02:34:03 -0700 (PDT) From: Vincent Guittot To: mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, vschneid@redhat.com, kprateek.nayak@amd.com, linux-kernel@vger.kernel.org, qyousef@layalina.io Cc: Vincent Guittot Subject: [PATCH v2] sched/fair: delayed dequeued entity bias wakeup preempt short Date: Wed, 22 Apr 2026 11:34:00 +0200 Message-ID: <20260422093400.319251-1-vincent.guittot@linaro.org> X-Mailer: git-send-email 2.43.0 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When checking if a shorter slice entity can preempt curr, we compare it with the next entity to be picked but delayed dequeue entities can screw the decision whereas they will be dequeued when picking next entity. Dequeue them while checking for preemption as they will be dequeued anyway when picking next entity. tip/sched/core tip/sched/core +this patch cyclictest slice (ms) (default)2.8 8 8 hackbench slice (ms) (default)2.8 20 20 Total Samples | 22559 22595 22683 Average (us) | 157 64( 59%) 59( 8%) Median (P50) (us) | 57 57( 0%) 58(- 2%) 90th Percentile (us) | 64 60( 6%) 60( 0%) 99th Percentile (us) | 2407 67( 97%) 67( 0%) 99.9th Percentile (us) | 3400 2288( 33%) 727( 68%) Maximum (us) | 5037 9252(-84%) 7461( 19%) Signed-off-by: Vincent Guittot --- v2 changes: - use pick_next_entity() instead of duplicating the code in wakeup_preempt_fair() kernel/sched/fair.c | 27 ++++++++++++++------------- 1 file changed, 14 insertions(+), 13 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index f179faf7a6a1..d99e56b6dcc9 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -1099,7 +1099,7 @@ static inline void cancel_protect_slice(struct sched_= entity *se) * * Which allows tree pruning through eligibility. */ -static struct sched_entity *__pick_eevdf(struct cfs_rq *cfs_rq, bool prote= ct) +static struct sched_entity *pick_eevdf(struct cfs_rq *cfs_rq, bool protect) { struct rb_node *node =3D cfs_rq->tasks_timeline.rb_root.rb_node; struct sched_entity *se =3D __pick_first_entity(cfs_rq); @@ -1170,11 +1170,6 @@ static struct sched_entity *__pick_eevdf(struct cfs_= rq *cfs_rq, bool protect) return best; } =20 -static struct sched_entity *pick_eevdf(struct cfs_rq *cfs_rq) -{ - return __pick_eevdf(cfs_rq, true); -} - struct sched_entity *__pick_last_entity(struct cfs_rq *cfs_rq) { struct rb_node *last =3D rb_last(&cfs_rq->tasks_timeline.rb_root); @@ -5749,11 +5744,11 @@ static int dequeue_entities(struct rq *rq, struct s= ched_entity *se, int flags); * 4) do not run the "skip" process, if something else is available */ static struct sched_entity * -pick_next_entity(struct rq *rq, struct cfs_rq *cfs_rq) +pick_next_entity(struct rq *rq, struct cfs_rq *cfs_rq, bool protect) { struct sched_entity *se; =20 - se =3D pick_eevdf(cfs_rq); + se =3D pick_eevdf(cfs_rq, protect); if (se->sched_delayed) { dequeue_entities(rq, se, DEQUEUE_SLEEP | DEQUEUE_DELAYED); /* @@ -9015,7 +9010,7 @@ static void wakeup_preempt_fair(struct rq *rq, struct= task_struct *p, int wake_f { enum preempt_wakeup_action preempt_action =3D PREEMPT_WAKEUP_PICK; struct task_struct *donor =3D rq->donor; - struct sched_entity *se =3D &donor->se, *pse =3D &p->se; + struct sched_entity *nse, *se =3D &donor->se, *pse =3D &p->se; struct cfs_rq *cfs_rq =3D task_cfs_rq(donor); int cse_is_idle, pse_is_idle; =20 @@ -9126,11 +9121,17 @@ static void wakeup_preempt_fair(struct rq *rq, stru= ct task_struct *p, int wake_f } =20 pick: + nse =3D pick_next_entity(rq, cfs_rq, preempt_action !=3D PREEMPT_WAKEUP_S= HORT); + /* If @p has become the most eligible task, force preemption */ + if (nse =3D=3D pse) + goto preempt; + /* - * If @p has become the most eligible task, force preemption. + * Because p is enqueued, nse being null can only mean that we + * dequeued a delayed task. */ - if (__pick_eevdf(cfs_rq, preempt_action !=3D PREEMPT_WAKEUP_SHORT) =3D=3D= pse) - goto preempt; + if (!nse) + goto pick; =20 if (sched_feat(RUN_TO_PARITY)) update_protect_slice(cfs_rq, se); @@ -9167,7 +9168,7 @@ static struct task_struct *pick_task_fair(struct rq *= rq, struct rq_flags *rf) =20 throttled |=3D check_cfs_rq_runtime(cfs_rq); =20 - se =3D pick_next_entity(rq, cfs_rq); + se =3D pick_next_entity(rq, cfs_rq, true); if (!se) goto again; cfs_rq =3D group_cfs_rq(se); --=20 2.43.0