From nobody Thu Sep 24 20:04:08 2026 Received: from mail-wm2-f13.google.com (mail-wm2-f13.google.com [74.125.225.141]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5EF9B4A5C29 for ; Mon, 21 Sep 2026 15:22:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.225.141 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790004174; cv=none; b=Zo6QZbt0+Dr98OYeDnukGCK8M80NqhJAGLnEf7EeJTdqsSn17DNocN92fGjhLcA5uAcLDh3CB6vhVM92hGa+0+/3/O3C1IdgtUmXUcG1S1wGeFLXzS/qB9OiNwbWBedkYZ9bQ6fJcW6tyX5jaDa4rFtKMbp3EkfegaStijRrdZk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790004174; c=relaxed/simple; bh=h+ZKIHAKHBXOHNFQ4rdKxT20jM6QS9cKS3j38JGgcKg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=pbBXnfnT8uz8Mg6YB9cAWzcbTCSxF5W7KoTGjzHrXLN6lPYEF1tm0MaU8WqcYapj204Ev2I9Dd6+pRHJth2fAFNTN2V2ajuozzUNThXuxgliRAIVPy9OSGp4ckPPkkLZoiFCayxWSi6FgiGkoNPEcEnFXvEPYQ2nf7KS+61GGhA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=fHoxc9CT; arc=none smtp.client-ip=74.125.225.141 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="fHoxc9CT" Received: by mail-wm2-f13.google.com with SMTP id 5b1f17b1804b1-49cd38e0f79so17611985e9.3 for ; Mon, 21 Sep 2026 08:22:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1790004170; x=1790608970; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=ivh65GK1bhKN65DM2APn3F7RhHRIzPyKv2hl9XfM3vk=; b=fHoxc9CTzXZCV041biT3g9R5+P2r5Wt0I85dIKS8B0Bs+uqiYv9MGyOG1G8qHW+shc zcRqaE27vHxcW9UgwG9JVln33v/dCx46DU/dB4WPwf6/E+w7flxmi6mJGyyNVLkkaA4m /OxyBTl4EVF0pvFrQz+2nUEika8swDZ+myw3N5cwBqrYLzumxbC8vl5n/p3UVNDN5wx9 umgxzomVchXP4Ts7BTC9B/pm7BQk6bmBjNirL0IwKR3kElQMwrcdftmb2mCXGaKmz4bL eswVrquQcB+yyUXU+meo639dNTlz/Xi7mUeNXJHnN7StsR1WimQn6qq/+3kPjjpvx5si vdWg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1790004170; x=1790608970; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=ivh65GK1bhKN65DM2APn3F7RhHRIzPyKv2hl9XfM3vk=; b=rMOmSzZ96jjxLWnB9ONCWngxiinPgwHXKb/WBCLVPck3tc4Ul9aYz1wh3TQmPNyYpM 74PpTMIZYT1uG5TIUAq/U+Yj5ykpy00bthwidptWEfb2G8l//kKp3iLY8fwGoeK8bANl 3BLc16llfgINj7aiIlALyPpaVd+lJs27Bc8C155+U4P79dXFUnrcQs03Frm1LKhPQiLR WUC3ZoQ3U5ISO/1wgegTa5KDVCHBPOCmWvqA5zvcpJb2CRl60j1kRSViJ3wNf/3IVJPL w0Mth6lpy8l7wPYHprwO+T7loQHNCMfxZiBHAoiAcuBxQmnBVbc0/emafJ1QsQFKYg60 vUmw== X-Forwarded-Encrypted: i=1; AKwUvBysbrmCvqJjrnVM93GNsoGrSc07udRlFYbqjkhp+DnPKMv00B9jgrVmOgzLmZm8N9JmI0yZTzjbDDppb2w=@vger.kernel.org X-Gm-Message-State: AFuF++kijgR4gGm64HrsaxLpKxy1MvXyhzm5CCA/tOQxxcj5jMbke/2v BTEID/nq5J10A8DDN9HbGlzALUdM5IHPktuTTz7JmGoX25rZBRdoIBrmhPmxzFTNsKo= X-Gm-Gg: AYBFou1aVgWcfzngg6Ol1r9l5l4hQkOuWG3nIN7ddFh2+HtrS0hVbzNKMfXYPA7MXjQ h3DOp+T3ypH+N9E0H1o7xHaEUGN0nFj4EN8v7+uaZ14Q+qJaEpUDkVXvCM93VC5I+fr7VS8YyBh 1yC/ixj2kXCrSjgb/j3PrjEpOtlRPHN/CXg4menI1FNH4B7LqUz0hL3iyr3nv/5l9tF+03zxlui HwVzk8P9ce8nTapXKhp44oIxwSdh32jF/PRISLwOeTU6n/tMhZhfKMw1iGGbg1bgh47aHjJ9Sse zixoN/AQCmZ/na93/VFo3wcx00E6rvrolxdanEvFm1ie9IjZYxPY5Z02Uq/+wR7yg4IZrWNKKna pbjKB5AcA00MnIus+bOeWA0xPT5T4OX6Q+AaLq4T1Xr6fgDWX8XFnTW0KKBq4liF950a392fIMW z3S9s+nS3El2ibExb6kTXG/Ov/zyBkZr09G61R2GSvGw1dh10k4H4kVvWng/sL6SI6O6y0knVy1 4LSHkzwoBTj/2Q= X-Received: by 2002:a05:600c:c490:b0:49d:2555:1a6d with SMTP id 5b1f17b1804b1-49fc573e8b8mr191625845e9.16.1790004170391; Mon, 21 Sep 2026 08:22:50 -0700 (PDT) Received: from vingu-cube.home ([2a01:e0a:11f4:3020:df5:e9a4:98b5:65d1]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-49fcd8bf159sm223745675e9.10.2026.09.21.08.22.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Sep 2026 08:22:49 -0700 (PDT) From: Vincent Guittot To: mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, vschneid@redhat.com, kprateek.nayak@amd.com, linux-kernel@vger.kernel.org Cc: qyousef@layalina.io, Vincent Guittot Subject: [PATCH 1/8] sched/eevdf: Ensure that vprot will never go above a min slice Date: Mon, 21 Sep 2026 17:22:31 +0200 Message-ID: <20260921152238.3804392-2-vincent.guittot@linaro.org> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260921152238.3804392-1-vincent.guittot@linaro.org> References: <20260921152238.3804392-1-vincent.guittot@linaro.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" ineligible_vruntime() can be after the duration of a min slice but we want to make sure that current will not run more than a min slice. Signed-off-by: Vincent Guittot --- kernel/sched/fair.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index a42360ca94c3..1c171cb7e4af 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -1127,10 +1127,9 @@ static inline void set_protect_slice(struct cfs_rq *= cfs_rq, struct sched_entity =20 /* If there are shorter slices than se's one */ if (slice !=3D se->slice) { + vprot =3D min_vruntime(vprot, se->vruntime + calc_delta_fair(slice, se)); if (sched_feat(PREEMPT_SHORT)) vprot =3D min_vruntime(vprot, ineligible_vruntime(cfs_rq)); - else - vprot =3D min_vruntime(vprot, se->vruntime + calc_delta_fair(slice, se)= ); } =20 se->vprot =3D vprot; --=20 2.53.0 From nobody Thu Sep 24 20:04:08 2026 Received: from mail-wm2-f13.google.com (mail-wm2-f13.google.com [74.125.225.141]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E9EF94A7C9E for ; Mon, 21 Sep 2026 15:22:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.225.141 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790004174; cv=none; b=HhsS7CUIvuNdTqM1JQY7MmgmlitNV5rppEa/3dw+iwuE6aQ/syWO2xcgMyQhu1DYJr3aLK2gpZRVaCGwvAsJOLTJUCDD5NUoqi/sKVlldGt3jwVJiKgO2K5JdYMcDAEsOgGn/AwSK8C/6yRAQzGeXiNTVEteUUUC8DMHajigtQA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790004174; c=relaxed/simple; bh=3BAhotiCwMRSV4UvLtnMYjKDNdNpMkVFmgoR/DpHOXY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ptyCOxe37YDqK3TUlE0MxWVoxgOxvdltafzuYBBa67MwYeTfXUKu9oxog7TH9GWQ44B7Om1eaLOmjqsBtPKMO0OFhKzhkIzA64Yrly2bJ3TTYR4BpYFkuzWMhejX21RmtJ6Mm39m+C8yxE1Dkp8YpVlxkPy1y8C//uNVYqBgVFM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=tUoV06oo; arc=none smtp.client-ip=74.125.225.141 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="tUoV06oo" Received: by mail-wm2-f13.google.com with SMTP id 5b1f17b1804b1-49b912e4b11so16665165e9.3 for ; Mon, 21 Sep 2026 08:22:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1790004171; x=1790608971; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=QzrjTGdBQf2LSry5gpHzNLdOxXZt5pOkV8UB4rqL8mY=; b=tUoV06ooqsvSURdprs0G33+vDsrNV534V+gT3lB079nqgi/A5QGzMvPmDJX8Oln3Iu CZ9rt7eH+60tcUiCvta4erseSF4SiXEPnPgtRvrADD/RF22luzJxnCQca5JDg3T5i9tX 7OglAwT9u3nwBPeu7FOrXxXT5D04P+X3xQMY98UJAO64CEEoQ/8Od4EiSQcLg6lGXI+A lz3ak3X3zywlZeF7oqX17AtyLIxE67lk6rRKrqdU94XbEeNpyfvRjZmULEz5UP5fah8E dbW3J8lFdfAoMQ8zVCKFbFbfyCkAOQY+NXXRZCrGimO3G9gte0EwdAMxbd3OyU1KuQH2 xOqg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1790004171; x=1790608971; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=QzrjTGdBQf2LSry5gpHzNLdOxXZt5pOkV8UB4rqL8mY=; b=nm9VJ6zxl7DjE9TaNlYsyDsmv0i73wh56lGGYjgizT+wcBn2d2nvCChfhzYAYn+L3+ GgOhTRnQB45vDBakw2AEMLBN8cglmIroitnJIziv1R5OAVRDPxKp9qmYsdMY/P5aRpM+ U+Y4PIN0V3Rwv3JIForWkFma3H9QZvD57EvUQcRr9HK1gwxLCDBCKnXdhYRYbMGUSnmK g8suBV3NJ0BIEaYAGBJv2nTjNQ1eiqJsWbEYEZgAx2ZFhb/v9cZ1QZSUPZiKXEbYXBIx n6av0+rxTary/rLDBxX8Nuu++mLIiYMD7hVJlm9fcZ02xmYu+WgBcA8Dbno8ocH5ficH lEEw== X-Forwarded-Encrypted: i=1; AKwUvBwTM0PiqOrsswLQgJBdivwyhGoep7PMtik2Rh38o3+Nb3Tk4prxKM17r8HpjmOq0cEFswcmVGnlspYScK8=@vger.kernel.org X-Gm-Message-State: AFuF++mKK+JPFP41TKxs798HGV8vmxbGkB5FnBlleo7jwEADp/qwdmHW k/aPujQnkhkBOEFD2L+xR74f4ypvRrOYuPhJVyejDDJgpz7C42r6PtRtReHFpv3YoRE= X-Gm-Gg: AYBFou1vtw3tKeSB84SRKaQ2btPYnuNY3+iFYkaGqqI6O6cGBOU/aPrttnZ6Q6kReSs XGguP85SfB+uuD6hojT809xuKj6PPYGVp1rMcvdBA52Qkpb2eH6ZsEfaokKpbeTDwh8GjYcqTnD EBl4xE4j3hNYDewQV2KwZU27d1nVqrCamHR7aF0/jojYkUh4+CO9nyQF9M+yWT1dppKf4QcN05E GdESYYWf90UIGRd06zNN1sRvbLVshnQCfjwy1FOFJdQRdJHKsqdVQbT37ZFlLtrGeu7GAHwQ5Z6 fYkxA0BeGBNVKJllhjvEPPJyekeVkuM6G/+aYqHkvGrbDzWw8Df3AVCdCkC3lINBUgcAuxZk/ct 854cxRQxc9P4ZjDRtjYm1v4HGUWehLg4Zn7dW4EbryUuEMSFb+O+aqd8mYMp1lwOYKOHzcsmVP9 MZ3B3wOenaH7ZpJdO17aknT0Xc1+RNwEqBTkmKf6zjC8/LDAwuyIIhlIbvgSDDlI6SAOTz71fhM 2SvP5WnTxP2SR/J X-Received: by 2002:a05:600c:6217:b0:49e:8418:389d with SMTP id 5b1f17b1804b1-49fc571423dmr142368645e9.9.1790004171230; Mon, 21 Sep 2026 08:22:51 -0700 (PDT) Received: from vingu-cube.home ([2a01:e0a:11f4:3020:df5:e9a4:98b5:65d1]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-49fcd8bf159sm223745675e9.10.2026.09.21.08.22.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Sep 2026 08:22:50 -0700 (PDT) From: Vincent Guittot To: mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, vschneid@redhat.com, kprateek.nayak@amd.com, linux-kernel@vger.kernel.org Cc: qyousef@layalina.io, Vincent Guittot Subject: [PATCH 2/8] sched/eevdf: Align update_protect_slice to set_protect_slice Date: Mon, 21 Sep 2026 17:22:32 +0200 Message-ID: <20260921152238.3804392-3-vincent.guittot@linaro.org> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260921152238.3804392-1-vincent.guittot@linaro.org> References: <20260921152238.3804392-1-vincent.guittot@linaro.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Apply the same logic as for set_protect_slice when updating it. Signed-off-by: Vincent Guittot --- kernel/sched/fair.c | 19 +++++++++++-------- 1 file changed, 11 insertions(+), 8 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 1c171cb7e4af..92382db1d297 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -820,12 +820,6 @@ static u64 ineligible_vruntime(struct cfs_rq *cfs_rq) if (curr && !curr->on_rq) curr =3D NULL; =20 - /* - * This is called from set_next_task_fair(.first=3Dtrue) / - * set_protect_slice() so curr had better be set and on_rq. - */ - WARN_ON_ONCE(!curr); - if (weight) { s64 runtime =3D cfs_rq->sum_w_vruntime; =20 @@ -1137,10 +1131,19 @@ static inline void set_protect_slice(struct cfs_rq = *cfs_rq, struct sched_entity =20 static inline void update_protect_slice(struct cfs_rq *cfs_rq, struct sche= d_entity *se) { - u64 slice =3D cfs_rq_min_slice(cfs_rq); u64 vruntime =3D min_vruntime(se->vruntime, avg_vruntime(cfs_rq)); + u64 slice =3D normalized_sysctl_sched_base_slice; + u64 vprot; + + if (sched_feat(RUN_TO_PARITY)) + slice =3D cfs_rq_min_slice(cfs_rq); + + vprot =3D min_vruntime(se->vprot, vruntime + calc_delta_fair(slice, se)); =20 - se->vprot =3D min_vruntime(se->vprot, vruntime + calc_delta_fair(slice, s= e)); + if (sched_feat(PREEMPT_SHORT) && slice !=3D se->slice) + vprot =3D min_vruntime(vprot, ineligible_vruntime(cfs_rq)); + + se->vprot =3D vprot; } =20 static inline bool protect_slice(struct sched_entity *se) --=20 2.53.0 From nobody Thu Sep 24 20:04:08 2026 Received: from mail-wm2-f12.google.com (mail-wm2-f12.google.com [74.125.225.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 226CC4AA020 for ; Mon, 21 Sep 2026 15:22:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.225.140 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790004175; cv=none; b=SM0n1aQ1pTpKqJcXkuy8LcfVGaxNq7PxMpjRi/wAVvrImZCo2R/SKBlCBXY+zUWwZkoHMD1vWap/UZ+UiSaymtgnhy7JAsDFXjZO7PXtqga8HqFGg2CZ3k1XTlzmEaXCCFkSffwZuWmbvtoEvoUUYzcjDbn/4pRLUaema6WeOwc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790004175; c=relaxed/simple; bh=YuamKqf3wWIm6d/wjsmPfZXrsEI4VwApWaNoKhz2ZC4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=caGl/tx6Kus6PkHdNFyXN0cODln4f1iTpDloqem7ILluHkFOW7AMZTQPjiD4SYz1P3hmCBfpMXxNqImj3i1qUOWckGQUYB5E2JBGhRx2l121uRTa09yqZH7pbgeoaNPjzP2TdnwtRkUc+p365KLOAKVWsuYCXFYukBbc76j0tt8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=lAe+xyeL; arc=none smtp.client-ip=74.125.225.140 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="lAe+xyeL" Received: by mail-wm2-f12.google.com with SMTP id 5b1f17b1804b1-49e6598dd44so16322605e9.1 for ; Mon, 21 Sep 2026 08:22:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1790004172; x=1790608972; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=Jq1r9pNI9TYrvOgY1YTb97iUOHNuCe4gvl0wKr4+QyU=; b=lAe+xyeLo/UAN2PQJAHyNiCkBWP3ZTefOuYLE4/zbBhfAuZDxRmOAyforUXafJRNOI FmqtwzFfn648YrmLIwKkO8I3T5HQlzWOVeQsPFL3tXhUURjRGskuHSoMJ3AdowGVGwjg gyFV7Rsp0MCH5dX0fvH3IZ7LlFLn4XQgSfjl1ggKniv7YTT86F8BEQEm0G+g+bTSX51I dB+DbQb67+1/bKd9XQGeovxPqZck3UHjfZsDLDT7yte7Bcv0t2yWnNs1K8uKzNSpGAhG H62e2FZfrk/2GBkNJnljwp/y1d7/nj64pUYS7Jlp4/I5Ahh5yY+KrwQjF9rZfU0CNJxc DO8Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1790004172; x=1790608972; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=Jq1r9pNI9TYrvOgY1YTb97iUOHNuCe4gvl0wKr4+QyU=; b=qGxYRGC+H6IFB4TWwCe/VZNLWTcdNhCrk5nxKM+GmjC2dQrcmSiQnNxInAoNCzoBj+ b45zG2x6PjGz/gnaaVQZwJ+GK0kfIhBOTrCmL7iUFW16CoKbssvxFP0aLvXwVUp68AOT BAXZLlXP+3JjnWCUOehf2EeMJqnv+POtTu1VHLdnbt4gXH2gLeH3J+TNDrnOOTGzbp9q B4YG67D//DXn0cF+AiEIOabZItNjBzS3nkH44CJ5F/nNN1sEAQ1I7k9c7OFCAFN+NzFK NEJwSmQrB70JslD/lSZIw+SOCFSSzkgNlPBv3wdfcH1AT/r9v1QvhfNlcOGicYHaJGFK Gr/A== X-Forwarded-Encrypted: i=1; AKwUvBzmsUXftHx0Kr72x6AjOGGdmiFFxY80CZLY1Sonfmv80pzTxC847tv6518w2VEN7ZM6INjFFOLmuhfOT6c=@vger.kernel.org X-Gm-Message-State: AFuF++loEKl80iWkk3GhiUwtJ0CCrKfozcjScnrxHnau+s0ioVwpfOSI 6NHasIN/ohCZoteD4NZ5swPHdZjUbkYnAgzhheu4bUU/gJ3oqvFAfIlpAj1C4WdHp4g= X-Gm-Gg: AYBFou2He1HktaVvv8DMrA+wlOJug9jsx2A2Jct5xa9FRImwVE5z/Ju7djVN/xq54BT XFmKFlXn7z7wMQNB9Gc9Itzl5F/YHYFzLqmcR89wdEWfSS572YY5saTZCKj9mm3Cd6bVkAf4/Dc ROPW0eM0lyfZLNWU3+tT2W22skPyMBgw/RQp2Vq5NjjBO0B6kEblThNjJxKNOFJ12IYPbl0LPNd dpHzMyUc1o3TzU3FNsc5MDIXIRcCgAdai3m7DkbHE4Tk5Tidtn81CTdR3SPtsfk5JKzH/BRwIWH ArYKekXUqYH4xtt10rk4J8+BxYFRR5xQt1uaF/Tc+t2sdiyZebPIlKNCarNu404l1w3NqJY2clI DKl73RxxaoHx9QsFRatSru3tBVNgjn1FcfAEG/W6CgEnxaQY8qoIsTTbWUGDxtSnJZmCCGFAxww zNhfYyvZqKogWPFDi1YN5hW/vK6OyNaJnTCbDyDjvijelvYPlzEGZtGtbJIq258c/TRb44uu5zZ tUsf2aogBgclyk= X-Received: by 2002:a05:600c:81c8:b0:49f:bb8a:b4ce with SMTP id 5b1f17b1804b1-49fc573dae5mr142907945e9.28.1790004172141; Mon, 21 Sep 2026 08:22:52 -0700 (PDT) Received: from vingu-cube.home ([2a01:e0a:11f4:3020:df5:e9a4:98b5:65d1]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-49fcd8bf159sm223745675e9.10.2026.09.21.08.22.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Sep 2026 08:22:51 -0700 (PDT) From: Vincent Guittot To: mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, vschneid@redhat.com, kprateek.nayak@amd.com, linux-kernel@vger.kernel.org Cc: qyousef@layalina.io, Vincent Guittot Subject: [PATCH 3/8] sched/eevdf: Handle more short slice waking cases Date: Mon, 21 Sep 2026 17:22:33 +0200 Message-ID: <20260921152238.3804392-4-vincent.guittot@linaro.org> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260921152238.3804392-1-vincent.guittot@linaro.org> References: <20260921152238.3804392-1-vincent.guittot@linaro.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Handle better the cases with several short slice tasks waking up simultaneously. Signed-off-by: Vincent Guittot --- kernel/sched/fair.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 92382db1d297..e6eb9a4c03be 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -9853,8 +9853,14 @@ static inline bool set_preempt_buddy(struct cfs_rq *= cfs_rq, struct sched_entity =20 static inline bool set_short_buddy(struct cfs_rq *cfs_rq, struct sched_ent= ity *pse) { - if (cfs_rq->next && cfs_rq->next->slice < pse->slice) - return false; + if (cfs_rq->next) { + if (cfs_rq->next->slice < pse->slice) + return false; + + if (cfs_rq->next->slice =3D=3D pse->slice && + entity_before(cfs_rq->next, pse)) + return false; + } =20 set_next_buddy(cfs_rq, pse); return true; --=20 2.53.0 From nobody Thu Sep 24 20:04:08 2026 Received: from mail-wm2-f12.google.com (mail-wm2-f12.google.com [74.125.225.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7E67E4B1468 for ; Mon, 21 Sep 2026 15:22:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.225.140 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790004177; cv=none; b=Htwh0V39cFKZQUeuGaSisYQncxpwNoTbLaZLRBujqc49Du0FQkFQ9rh4hVE44pxvUXXYSskHjRZ1o2K0xPa7jh2cR6Gz8ScT0OESGb/Eykrh4o0LPS+rKuuxhc3jQMDcmoCB2dhkoMGLKzj3u78TlQ9MPbPuf6TC4FX1kfZXj4A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790004177; c=relaxed/simple; bh=r/DXnnZh82n9G0B82/HvKKmY0wfkV/SCJC8LIPGqQ24=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=BJym7N31km4dGEsUxDq7Gb5796OjvXJPhZ9oCwp2vg3Jj8dLpTf8P6Fe4eSdb9BU2MFrqmKsDk1GLwKGTy4wW1yEzFAbvSbHeuq/y/wtrx4E1GVnz4SyN3yr/yJbh+SeDXE/CN3CgJuDKLSKOno7WMNvFugZO3uIkq2zm9koQj8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=T+Zm4pE6; arc=none smtp.client-ip=74.125.225.140 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="T+Zm4pE6" Received: by mail-wm2-f12.google.com with SMTP id 5b1f17b1804b1-49cd5462b69so16801195e9.1 for ; Mon, 21 Sep 2026 08:22:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1790004173; x=1790608973; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=95ptzx8elX4G0jOf5GWK4LtCTZK63m+CYC1QnfEGz0c=; b=T+Zm4pE6nAvONIzUfsAEgvILD/dSx663DFA2MwPZXaLQOxRUDGucNBVHyZVufyyze3 Os0T7Pf7md8aiLwBbfjxJU/WKjRsvpuO8EOL1B0AJejgz8EJR701SiVWYUfXxPeNaL0w 58D2t2KooXJBYiCW/AdOz5dNPcshqrxEp0U0PVLr7aK/BqPVf2i3yq+b1ZGuIOcGqmho gn1Dr/cfBfbQ4PYT1BGX9xGZcUSouIRpA0WrPgqgUa2cbfvNsH7GPCh2gH+7nfRGIc6J bO7SNSnK7vmXj7yBv3L6HVdYtRHjAux6u5szJFI3Ysqs96Ac427KkV+cjN31MhGH3Qa7 64og== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1790004173; x=1790608973; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=95ptzx8elX4G0jOf5GWK4LtCTZK63m+CYC1QnfEGz0c=; b=KBkFrYemp3Irbq0YosuGyvWZEFoaGJ7EyWy7RaznGDwxnGm0LOQowEHi+pRnzOzkzy xcEVg6TiLSh5b3i15ZMTzYeD74ZlgyjRuvJitJ8/1BXONCLm2CGyEBd5ZUUGbOIlnofQ XZ7PoUyvA4dEF8CR/t7+e0fYpEWxtxcAujnjKPvnZofz90WiS24KreY4Ndlz+e/dNNpG RCMusT0nYhkcOt38aWALKpozql12xH/vSX8ttm/5H33BxUr/c9lZlMHLe2XTJqdVUqF/ kqqiOr4+yeZzKqteekZPhyq0ME4arQ4mCfjinA2lNYyo/5E++l0RfCFTOgYGZZwgfBbb S1Ww== X-Forwarded-Encrypted: i=1; AKwUvByuI/cEayGam/qiwZCJ76kjgzmJa7xfyf4jmNFGQjQ4z7bV+srt0IEDONNmZYOx6gOXtsJy+5wWO2Cehjw=@vger.kernel.org X-Gm-Message-State: AFuF++mu6f4cE2WcHAFUd9TS3eFqXnSn/f63JeE/rh/Z6BIUzFVwOaEZ hfJiR3d5CiwUXpczeq578oI54nsZyw2fxdpDqCyEG//lI3pC4+3pl+fg0B9zXu8r7i0= X-Gm-Gg: AYBFou16CgSv0SrOUayPD77imaWH7bq83byhfWNlHrYu+8JCqKP73FMSMq8lWn288B9 rqJr5nRlCZy5Wujhjuby7nAxHGbmt8LWAvnjwDbav7yzyzoTN/LyjQH/HFOlVAclGL1FohRdjcj 5gzprWdX1XQcoJjlReHZ9GN9z7+xOgS8akSsEmApAwLGc28kcN0W33MkR0UaSuTgPpPmZuNiBce EmhvKpuyVo8a82m9YvRJKbCk0HjPZO7g+AEmyfgBnNDXI8YwLeO9XMhgOxPPBoPLJiGWkSQSRR2 weM8S7G5i9ismicqi5b7JUq/IbvVXEEqR9KhlJKEltYXSlDKM+n29CGPqL3yXAQGGgMZuGO7/v/ oP0YoxUwRVi2xfk9p5MXW/DoMcKrsr9Wju1xX5fFY/P7tptlhc5SqOHVb0Jv8GUrxPisY+R2s/n vzCa9bzVoMq19sDCuH5o4F6kCvTCu/kHAZhaX/rgIzu2t/FPPpytaFuoTi5tuFA2pLeMz0T7unl YK2LUulbu4tI/0= X-Received: by 2002:a05:600c:470d:b0:49f:ce78:356b with SMTP id 5b1f17b1804b1-49fce7836bdmr96536655e9.28.1790004173599; Mon, 21 Sep 2026 08:22:53 -0700 (PDT) Received: from vingu-cube.home ([2a01:e0a:11f4:3020:df5:e9a4:98b5:65d1]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-49fcd8bf159sm223745675e9.10.2026.09.21.08.22.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Sep 2026 08:22:52 -0700 (PDT) From: Vincent Guittot To: mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, vschneid@redhat.com, kprateek.nayak@amd.com, linux-kernel@vger.kernel.org Cc: qyousef@layalina.io, Vincent Guittot Subject: [PATCH 4/8] sched/eevdf: Decay positive lag of sleeping entities Date: Mon, 21 Sep 2026 17:22:34 +0200 Message-ID: <20260921152238.3804392-5-vincent.guittot@linaro.org> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260921152238.3804392-1-vincent.guittot@linaro.org> References: <20260921152238.3804392-1-vincent.guittot@linaro.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Similarly to delayed dequeue that enables an entity to decay its negative lag while sleeping, a task should not keep a positive lag forever. The sleep duration and the weight of the entity is used to decay the positive lag at wakeup. Signed-off-by: Vincent Guittot --- kernel/sched/fair.c | 33 +++++++++++++++++++++++++++++++-- 1 file changed, 31 insertions(+), 2 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index e6eb9a4c03be..4230954d10d0 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -896,6 +896,32 @@ bool update_entity_lag(struct cfs_rq *cfs_rq, struct s= ched_entity *se) return avruntime - vlag !=3D se->vruntime; } =20 +static __always_inline +void decay_entity_lag(struct cfs_rq *cfs_rq, struct sched_entity *se, int = flags) +{ + s64 vlag =3D se->vlag; + s64 delta_exec; + + WARN_ON_ONCE(se->on_rq); + + /* Negative lag implies delayed dequeue */ + if (vlag <=3D 0) + return; + + if (flags & ENQUEUE_MIGRATED) + return; + + /* Compute the sleep time */ + delta_exec =3D rq_clock_task(rq_of(cfs_rq)) - se->exec_start; + if (unlikely(delta_exec <=3D 0)) + return; + + vlag -=3D calc_delta_fair(delta_exec, se); + + /* vlag can't become negative while sleeping */ + se->vlag =3D max(0, vlag); +} + /* * Entity is eligible once it received less service than it ought to have, * eg. lag >=3D 0. @@ -7986,7 +8012,7 @@ static void enqueue_task_fair(struct rq *rq, struct task_struct *p, int flags) { int rq_h_nr_queued =3D rq->cfs.h_nr_queued; - int task_new =3D !(flags & ENQUEUE_WAKEUP); + int task_wake =3D flags & ENQUEUE_WAKEUP; struct sched_entity *se =3D &p->se; struct cfs_rq *cfs_rq =3D &rq->cfs; unsigned long weight; @@ -8019,6 +8045,9 @@ enqueue_task_fair(struct rq *rq, struct task_struct *= p, int flags) if (p->in_iowait) cpufreq_update_util(rq, SCHED_CPUFREQ_IOWAIT); =20 + if (task_wake) + decay_entity_lag(cfs_rq, se, flags); + /* * XXX comment on the curr thing */ @@ -8057,7 +8086,7 @@ enqueue_task_fair(struct rq *rq, struct task_struct *= p, int flags) * into account, but that is not straightforward to implement, * and the following generally works well enough in practice. */ - if (!task_new) + if (task_wake) check_update_overutilized_status(rq); =20 assert_list_leaf_cfs_rq(rq); --=20 2.53.0 From nobody Thu Sep 24 20:04:08 2026 Received: from mail-wm2-f12.google.com (mail-wm2-f12.google.com [74.125.225.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D93E14570FA for ; Mon, 21 Sep 2026 15:22:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.225.140 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790004178; cv=none; b=MAS73yUh9bebSpZe1q1FARrQuZzEzrnbMmNV/pkLYRwZB+dRwhQ3PTRAekfn4NTyohj4OTjuwlMf9Oiq+MxBNoNv1NcyMTAXPj6qXxBJmUIg+5g1Ehe2QHCN3qCrNRWF36t2N641U4o5Qj4/lmFZMQzqRXWYSKMJEBpXnXD5yQc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790004178; c=relaxed/simple; bh=/y1cxhTfCkNKLF5SlxaguLdy4vHWteMhMy0X//GfzgU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=TXUFDaj7qGZG/auQo3dTrF7ty1qwcAgw1Nix3ce9mP/epIIlOPHEooNvaj7iZOnZ8iJZMyDu/TQnOWuXnsFK1JDAoMhYoowCSRFNx/ioZS3kDy+aWxGQ1YuixGQQ68x/ODOp1YYwnVIDLZ5dIxWAI5Fq23n7+bXq/ZVUkiRAyZ8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=PTLKxRQf; arc=none smtp.client-ip=74.125.225.140 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="PTLKxRQf" Received: by mail-wm2-f12.google.com with SMTP id 5b1f17b1804b1-49d097b4939so16279775e9.0 for ; Mon, 21 Sep 2026 08:22:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1790004175; x=1790608975; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=GkwZJrXCTkJBGFdmK6ATkqCroiAul7HPoyKB4YWO+sY=; b=PTLKxRQfzkifKn2vXavywsLkaAiJN+tzi0ay/bwZGG4J46WIVyuLGBRmRZ4JM7CsUm jDpfhl6ovvD75zuPP/C9KjuVpSEX5oh62EHVJRCA63erp1l8x7EGiO3GBq3ZWblI7or2 vF5FnqOuDACAIeOLfQ52YLC9tnzb9qtYIDAz90PS8zsEa8BrTAny0wLUAzCEP1PYTw+u dT9wGhWAKdjsv2xIrhAqfpRqI3M9YVmXAm18U7P0Mz9UdgTzPsaRjRQEeXyvHcnpA0D8 7B3B6QiMe2qY2YTMCVsmJs4ZqtaMw6bfvX/mQBB5kIM4yZN6ZP7c6s27Etm8zlNtIqzV HQaQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1790004175; x=1790608975; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=GkwZJrXCTkJBGFdmK6ATkqCroiAul7HPoyKB4YWO+sY=; b=zJ02aHUoo7oxhq0OLInU6Vbxj5P/SsYsSFlpphKenu1mEv1i8ATmWMKtnEi9j8X98b ubYO06f1KBHwdT891Oyh32t9NCByokNV+Mp7y3WXy12TJznyXK4lboL3sTJOSqpF3Gx6 RNrp3O5bi8Xtj1vXY2z8SvT35R44q+UnzdMLcPx6VUJ7xDf3JqwXK3sd6BhMqm2/gXkA 8J4p5Szj06eFg1wCA7f4enxEHr71P9wmsfOWLebQZmeWnxcb0AjRtQwlIh6MHTFF10A/ 5F59D/DXesGrTitKDCvOuwEd255GNfNgAi2WZLLyygePUj/DBtN+vDCrpA7edXnKJ511 fWLQ== X-Forwarded-Encrypted: i=1; AKwUvBxKtMPALvwKtDuCu3aCCPIkQXQpZCajT0WMW13Ygv9eeE3Heehbj31xWHfUjEK7hhpfbDwaz7hL4/o035A=@vger.kernel.org X-Gm-Message-State: AFuF++kGBK9jE63ePg3IqX7AAfK0xHgsbFerZOomFZuCkLbLn8v2Dvut JFb+6eQp0l50xUbcSJpa+VWkUYt2HYNNCivyXYqr7hPzjEECuztfk0zp/rFAvurHNDA= X-Gm-Gg: AYBFou0CO1oleU/W4qxaYa7u14U+IlZSZlvjPJFPIx0TsUH6IG5DpIq3gISwCV9DHfh NPFOb6SnVaegQajD0SoIk9KPrNN7hKMb0xp3lGJVw9LZiLxf6vkD9wKUSxgJdHq5DplKkiZ8aD7 88F6GSVyAqNksdFAQjvRh3v3/WWarBneNRHUoIH2j1MRBGWeh+EdSgvg81mGQsYoYwBaaOSwLbS Xkl1eMsuRRcTT62wHMEyxGQzrEaquyCF5qNr1BH6ZpO6vz/X9904h0tzsPyvyZcA1zsEnmAFzNj +hbJoYKkJFjxBOdRwwuj4oRvwsDODBU4xIgJtL+OtlEv/V8Jb4X0dVx1U0mOGnlSTKgUB1ZbVnB VyR9P80BgSt/rr9xqR8y/avXw2gMHfMhJcgakuNcKagR/ROl6ikirFXjLO8UFGfWzLpLp2ip4Aa +xoGAcXi/5N7ErsvuUDQsVpvim5pUtyplm0xWmmZLTKBmu3565iXLMg/ODVzQfMDi2bvtBPfVct it13V6Zd4O7D9U= X-Received: by 2002:a05:600c:a14:b0:49d:7f5:4cdd with SMTP id 5b1f17b1804b1-49fc574be1dmr152970115e9.27.1790004175075; Mon, 21 Sep 2026 08:22:55 -0700 (PDT) Received: from vingu-cube.home ([2a01:e0a:11f4:3020:df5:e9a4:98b5:65d1]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-49fcd8bf159sm223745675e9.10.2026.09.21.08.22.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Sep 2026 08:22:53 -0700 (PDT) From: Vincent Guittot To: mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, vschneid@redhat.com, kprateek.nayak@amd.com, linux-kernel@vger.kernel.org Cc: qyousef@layalina.io, Vincent Guittot Subject: [PATCH 5/8] sched/eevdf: Reset lag when waking up on idle cpu Date: Mon, 21 Sep 2026 17:22:35 +0200 Message-ID: <20260921152238.3804392-6-vincent.guittot@linaro.org> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260921152238.3804392-1-vincent.guittot@linaro.org> References: <20260921152238.3804392-1-vincent.guittot@linaro.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When several tasks wake up simultaneously on an idle CPU, their final vlag will depend of the ordering as the first one will lose its lag but not the next ones. Reset the lag when the enqueue happens while no fair task has already been picked et set as the running task. As a typical example: CPU0 is idle TA with vlag 0ms and TB with vlag 5ms wake up on CPU0 simultaneously. Depending which grab the lock 1st the behavior will be different: If TA is enqueued 1st, TB will be enqueued with a positive lag and will be picked 1st. But if TB is enqueued 1st, it will loose its positive vlag and both TA and TB will have 0 vlag when fair will pick a task. Signed-off-by: Vincent Guittot --- kernel/sched/fair.c | 27 +++++++++++++++++---------- 1 file changed, 17 insertions(+), 10 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 4230954d10d0..739a3af60520 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -900,7 +900,7 @@ static __always_inline void decay_entity_lag(struct cfs_rq *cfs_rq, struct sched_entity *se, int = flags) { s64 vlag =3D se->vlag; - s64 delta_exec; + struct rq *rq; =20 WARN_ON_ONCE(se->on_rq); =20 @@ -908,18 +908,25 @@ void decay_entity_lag(struct cfs_rq *cfs_rq, struct s= ched_entity *se, int flags) if (vlag <=3D 0) return; =20 - if (flags & ENQUEUE_MIGRATED) - return; + rq =3D rq_of(cfs_rq); =20 - /* Compute the sleep time */ - delta_exec =3D rq_clock_task(rq_of(cfs_rq)) - se->exec_start; - if (unlikely(delta_exec <=3D 0)) - return; + if (rq->curr =3D=3D rq->idle) { + /* You can't claim any lag when waking on idle CPU */ + vlag =3D 0; + } else if (!(flags & ENQUEUE_MIGRATED)) { + u64 now =3D rq_clock_task(rq); + s64 delta_exec; =20 - vlag -=3D calc_delta_fair(delta_exec, se); + /* Compute the sleep time */ + delta_exec =3D now - se->exec_start; + if (unlikely(delta_exec <=3D 0)) + return; =20 - /* vlag can't become negative while sleeping */ - se->vlag =3D max(0, vlag); + vlag -=3D calc_delta_fair(delta_exec, se); + + /* vlag can't become neg while sleeping */ + se->vlag =3D max(0, vlag); + } } =20 /* --=20 2.53.0 From nobody Thu Sep 24 20:04:08 2026 Received: from mail-wm2-f13.google.com (mail-wm2-f13.google.com [74.125.225.141]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B82E34BB271 for ; Mon, 21 Sep 2026 15:22:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.225.141 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790004181; cv=none; b=tnsN4boqDH12AjV+AZAhcaAKz2BYZNxgfF4CJQ5GX5pf37kX+RgPZgV+TWiwkkoUmvcfe+LmViRyd0yD/uBTbEH44Qc6xq9YV1VS7oR0ruAGZKbYoYQfuf2YQ3qBXVUohJTvFvuowuPXim2+e8CatLqgGXAlm3wwFr0+w2hnDN0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790004181; c=relaxed/simple; bh=VmklHztae8noGlZDZPbgvZMWDFIRiHMVBZFA5OBshaQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=YPo25P3XmY5iOKGubuL84DOi6zZ/4HqK4/SsGp03TOwGKqpamWJMAhuIwnQIlHZKnhHa4Zq+eUD7+a4AGfz98e7NSE4xrLskQ116RxGN4th5B/NOnnfQt1OPqtIboSkpVOk+ooW+tGLgjFdA4G+xkwfGsdzRyUPX6gy+JLU5b5s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=il5nlsCN; arc=none smtp.client-ip=74.125.225.141 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="il5nlsCN" Received: by mail-wm2-f13.google.com with SMTP id 5b1f17b1804b1-49cd38e0e5dso44366785e9.2 for ; Mon, 21 Sep 2026 08:22:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1790004178; x=1790608978; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=cQPNgCwsYO9IyF2ZmLp32Dch9UnZOPItESi3x5Q27kM=; b=il5nlsCNhGTPtxCambClYiMct4qVfyj/Un6tLRrOIRQ9HOX4ODohCTyyWyX3tb+VNU WOE9D3dokrbXeqxHgT8yFcX+kY1HPhfnM3o8wLJEzTX8V0fz0Fu+MUUvCuXd5sTNxGrA WijqYMWwdt5/o+WnNBbe1lY+By6NlQ/6gOlv3B0X9NfD1QiL3r+Tva+uAZ4sEiyZQd4h FlZf468FLCWjiv3AVFcadzG7F4tvNYhgpyfGpSc9XvwZ9dNNZ5s8rnbGAHzxYiTAn5K8 BNeEv5COXAp0qIykL6s1dOAmnAgiLe4mENYRORcf1u5gp5gK4Cz+zoLqVih4uDxe8Pfk 5p1w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1790004178; x=1790608978; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=cQPNgCwsYO9IyF2ZmLp32Dch9UnZOPItESi3x5Q27kM=; b=kG6OViwiOIMxrGWt4f5tKw198ec9TcABLvQCgytN8KyPwzRH8xW1doavpFE/TCaB5n 6FixB8LWjrbgZwidjMqQjOzWH/mHz41c0XLsp7yQNZZ5Osm9CS9PppcRPdBBEgUWhRl+ c8WioG4DnC8FrjK7xLRpy2Xdp7Cx+07QDR4Jxx3vBalfqafoZp16HKAa5ogs2dYEjaKk ZmcvbkuzjeIdmsASyOmspJDvpPJ4K4fq1fVjRYPS+V6A8Lna1P4Wq2fyy0Vjf5kDJvq/ +CNS2m65Vce196CofRkMLiGhrMFo9N/pUyvPfYXRwPnby8fjUEMQrLnEbq5JXLtmF4Lw OIGQ== X-Forwarded-Encrypted: i=1; AKwUvBylBpFfDbHg5OEngjllwC9Ykpn/uAbLIiFI1R/DxbQQueTySv9Rv9+OZp8z9gxRrusoaKfST32k0GfIyPo=@vger.kernel.org X-Gm-Message-State: AFuF++kDiWiXtFxT8dYuQsWrZAman/lr6UiIFfJkAUwvp5rKQbHYBFnw 8GLQv76pCCuT+tB8e1dZhpWOn3ENcQlJz1HxVu/FX9zMXN9K0slFvvuEiDuoVv6Wyzg= X-Gm-Gg: AYBFou0bIh85GB0nzZS12IVDlu5dTftLroTrldLQYIgDhakI9lqDu18T5plKt/0W/cf u0wUENWC9cWeYgaz5Cd5xOQ8w7WqheeHSWgZZJJleN1esfYXadbIheo4lqcAwmZ9Cagnk9PMLmL iZ6s/6USM4HRgCPsYZa8Hwa6SXCrLtWvK1VlJ7wuv+9Wp0gK5XpZtR8OU0wHMMhn+PClYaFXwJ1 RFCVH0nE7IYdeomtbOZHjvpaxZrAqwBBfT07S1INerfhQi+M0vETJ6swGCkv37zMKkXi3UCEBPB IrK9OxovjuZZoAW/mtThKvMUgL/foK7CuouB4GKwcyw7U0P452GUWlevutlKEgFQRpWVN5nW8bF n7IW3P/XkMCo1fy7ij22C5JN2pvriorUJn/jHW8QSfmSGQ3HrwXhjsQ0popW3uafaZFBBmzxKMY /dt/mldUyHu9ZKVaSNQDtXnO4aewzvp/s2AnAAArGpVmVr9B1uPFdNQf0dLrGnTWiD7UrqJ0+Ug zRX9ZlzUtt73zg= X-Received: by 2002:a05:600c:4e14:b0:49f:bd3c:bc19 with SMTP id 5b1f17b1804b1-49fc572eff5mr154822575e9.20.1790004177762; Mon, 21 Sep 2026 08:22:57 -0700 (PDT) Received: from vingu-cube.home ([2a01:e0a:11f4:3020:df5:e9a4:98b5:65d1]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-49fcd8bf159sm223745675e9.10.2026.09.21.08.22.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Sep 2026 08:22:55 -0700 (PDT) From: Vincent Guittot To: mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, vschneid@redhat.com, kprateek.nayak@amd.com, linux-kernel@vger.kernel.org Cc: qyousef@layalina.io, Vincent Guittot Subject: [PATCH 6/8] sched/eevdf: Add per cpu cached min_slice Date: Mon, 21 Sep 2026 17:22:36 +0200 Message-ID: <20260921152238.3804392-7-vincent.guittot@linaro.org> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260921152238.3804392-1-vincent.guittot@linaro.org> References: <20260921152238.3804392-1-vincent.guittot@linaro.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" In order to use min_slice during cpu selection, update a cached value when needed after en/dequeing a new task. This cached value includes current task. Signed-off-by: Vincent Guittot --- kernel/sched/fair.c | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 739a3af60520..a01a88dde811 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -1089,6 +1089,21 @@ static inline bool min_vruntime_update(struct sched_= entity *se, bool exit) RB_DECLARE_CALLBACKS(static, min_vruntime_cb, struct sched_entity, run_node, min_vruntime, min_vruntime_update); =20 +static DEFINE_PER_CPU(u64, rq_min_slice) =3D INT_MAX; + +static u64 get_rq_min_slice(struct rq *rq) +{ + return READ_ONCE(per_cpu(rq_min_slice, cpu_of(rq))); +} + +static void __update_rq_min_slice(struct rq *rq) +{ + u64 min =3D cfs_rq_min_slice(&rq->cfs); + + if (min !=3D get_rq_min_slice(rq)) + WRITE_ONCE(per_cpu(rq_min_slice, cpu_of(rq)), min); +} + /* * Enqueue an entity into the rb-tree: */ @@ -8073,6 +8088,8 @@ enqueue_task_fair(struct rq *rq, struct task_struct *= p, int flags) __enqueue_entity(cfs_rq, se); } =20 + __update_rq_min_slice(rq); + if (!rq_h_nr_queued && rq->cfs.h_nr_queued) dl_server_start(&rq->fair_server); =20 @@ -8195,6 +8212,8 @@ static bool __dequeue_task(struct rq *rq, struct task= _struct *p, int flags) if (se !=3D cfs_rq->curr) __dequeue_entity(cfs_rq, se); =20 + __update_rq_min_slice(rq); + sub_nr_running(rq, 1); =20 /* balance early to pull high priority tasks */ --=20 2.53.0 From nobody Thu Sep 24 20:04:08 2026 Received: from mail-wm2-f12.google.com (mail-wm2-f12.google.com [74.125.225.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 92EA44BB27E for ; Mon, 21 Sep 2026 15:23:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.225.140 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790004182; cv=none; b=qbd8HRwGXOh9R7RJp6saunBub2cnuQfwY3boGopF4AhOx7mi0qWWxxsn8JbJpiEZPayrCgjxsxNzRvUiFvR4/IixOEEvVWKq0QmRtz5YsE6dFuJ27e+L9YbBSAhfDh68T9xOXZw5xUbmoD8dWm0Uf+fVWvp+g19DFymV/d7koFs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790004182; c=relaxed/simple; bh=UMlJjVG9gzwcasVi4RDqGJGopy523m6VNfnWtI/sYW4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=m+3845v6rnEJzJX0FIS5+X+JGHCgziwOxlHg9m7ywKuZ7cmz3q6WEJOyA18YoQuR6/bPtApel6T/6xACIBkdQkDvwEkHgcEY2LHui6iI2DxDzCojr8DvdPRjNXMj9d0sL0qJioEAcaGyjAGOdnuF/HzPQLzHIcV/qLtvkVQhhS8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=L5zp1BWm; arc=none smtp.client-ip=74.125.225.140 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="L5zp1BWm" Received: by mail-wm2-f12.google.com with SMTP id 5b1f17b1804b1-49d097b4939so16280395e9.0 for ; Mon, 21 Sep 2026 08:23:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1790004179; x=1790608979; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=fa7koBn9vHguCbWn4P865SMrYtBgXn6RCiTJC302GpY=; b=L5zp1BWmtoOxqIogUABT4G/HwdgIFWFpJ/W/AyfvMJ4cHdvtqluzyt4sBkOKsb5wPx D/b8y00m+eEDeeevhn51TUIbs3wWnNDeqsHitELT+FJSNAIQMv135xLzBKNpZAI2Nhaf QJK7tRRlvjiKIlpfSEw8kgC5+rzGsLqaevhWznbPpFf1jAlrUZFt0B8Eizcjf/jeDCTw o3s7kjmrU5HVI0AHInTjKWMP402c6Ce6dy/NBD5ob6kvCYsbYiE5MAbKk6087jTfA8C6 rXo79xVnzajh7IeRIbOOylIjN229BYzX0WM0Os4M/HRQd0aCi/WE0aUKr/+s89HUgFQM CAxQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1790004179; x=1790608979; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=fa7koBn9vHguCbWn4P865SMrYtBgXn6RCiTJC302GpY=; b=gtzDeeom1LUMyZYmJC83lBa+O2qdZIyOlTE+9HEr7AO8/FomifPdEVKjm9LjsJOsUz K6lWJNrchYPR6JOM7ygKSXAXJLYqbISHupvMF6m58kdJgDfRw+7sontvbxk0mCcnX9OX ZS6L9i2O4ACThDnkTXt9qjrPl8sGhnVysGwtc/dQkn/rh1KghekK46xEOh3wnd0id2S+ 4D2IXdxxUoXEq2Q0gDDJyUkqEembGKPM99zRZU9ANoSM/ZR4CUZRlbtgs6UlQzYANSjS rASk3CYu3qQICkrVHGbuPyv3mnKyw6BXO2drkU7X9rDuLkhbzJM9mgeFPETv6NTlxtRU XUxw== X-Forwarded-Encrypted: i=1; AKwUvBxWrXiYDYB/ObVAKxHgA9HAipzbbq+737TrVM6HpkCSntnAacFuDj4Jl4cVSt071+QeIPVUhmX/gl72TD8=@vger.kernel.org X-Gm-Message-State: AFuF++kLaie8qu4UznnWGTMagenqpH+FjaFOC/7+B1JFiU6jTP3AjRc2 6ooA/xe+Pui3OdgJDeaeIiMtpYAN9sVcCAXvfy4fTSIoda2GNd9lisYz5ofqzhe9/E8= X-Gm-Gg: AYBFou3vfvnemOfC49ZAw4lSqWiX2CUa1Q4yweqP9aMoiFSNinLs2ZXigYDFQBMuilQ emLAvtdk4KwQiyhMvRf1xEK9T6JXtFTgSUxsotYS+92FEb5ed6me5X+VV2Cvmu98mR0PbJl3PxN C9rb7WNUfh912hcaMIONxYqfpZaF20V9a98AP34No/jbfaTBtk00bt+zmNn6x4umMKI2+9qMJ6t BCtPXlIL8C/9dWDWRM2aWn4WNfq5ofvva/ENK4kzT48Y7VwfNiBJzh/H5gBgnqktF6Vr7ydWMNq O1hWbBkaHa8xGS+v3lgSmb9eGooU4hvQzZLZxjBMY0D5OXI6dbZKcOhn2I8zoIMdDld1rBfN6+w L+CoB+jJbavxqTHIIET5fsjbU42OeCtKjqrfi5AcMGIzztwmYDVljb7fqsxu3L1GuBKzuyGqn5W YAEcikbuGro73iuVyaNP8/vfPAPsMgcu8EtvB6qx5u9WUwkO2vv5eJm+EdJm2AU78rVAyNTQ9Nc odE2WCzCdgXsps= X-Received: by 2002:a05:600c:4f09:b0:49e:8418:3858 with SMTP id 5b1f17b1804b1-49fc56b84c3mr151695145e9.9.1790004178652; Mon, 21 Sep 2026 08:22:58 -0700 (PDT) Received: from vingu-cube.home ([2a01:e0a:11f4:3020:df5:e9a4:98b5:65d1]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-49fcd8bf159sm223745675e9.10.2026.09.21.08.22.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Sep 2026 08:22:58 -0700 (PDT) From: Vincent Guittot To: mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, vschneid@redhat.com, kprateek.nayak@amd.com, linux-kernel@vger.kernel.org Cc: qyousef@layalina.io, Vincent Guittot Subject: [PATCH 7/8] sched/eevdf: Compare min slice during wake_affine Date: Mon, 21 Sep 2026 17:22:37 +0200 Message-ID: <20260921152238.3804392-8-vincent.guittot@linaro.org> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260921152238.3804392-1-vincent.guittot@linaro.org> References: <20260921152238.3804392-1-vincent.guittot@linaro.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add a new level in wake affine where we check on which CPU the task would most probably run 1st between this and prev CPUs. Signed-off-by: Vincent Guittot --- kernel/sched/fair.c | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index a01a88dde811..452da94289fe 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -8402,6 +8402,9 @@ static int wake_wide(struct task_struct *p) * wake_affine_idle() - only considers 'now', it check if the waking CPU is * cache-affine and is (or will be) idle. * + * wake_affine_slice() - only considers 'now', it check if the waking CPU = can + * be preempted because using longer slice. + * * wake_affine_weight() - considers the weight to reflect the average * scheduling latency of the CPUs. This seems to work * for the overloaded case. @@ -8437,6 +8440,20 @@ wake_affine_idle(int this_cpu, int prev_cpu, int syn= c) return nr_cpumask_bits; } =20 +static int +wake_affine_slice(struct task_struct *p, int this_cpu, int prev_cpu) +{ + struct sched_entity *se =3D &p->se; + + if (se->slice < get_rq_min_slice(cpu_rq(prev_cpu))) + return prev_cpu; + + if (se->slice < get_rq_min_slice(cpu_rq(this_cpu))) + return this_cpu; + + return nr_cpumask_bits; +} + static int wake_affine_weight(struct sched_domain *sd, struct task_struct *p, int this_cpu, int prev_cpu, int sync) @@ -8488,6 +8505,9 @@ static int wake_affine(struct sched_domain *sd, struc= t task_struct *p, if (sched_feat(WA_IDLE)) target =3D wake_affine_idle(this_cpu, prev_cpu, sync); =20 + if (sched_feat(PREEMPT_SHORT) && target =3D=3D nr_cpumask_bits) + target =3D wake_affine_slice(p, this_cpu, prev_cpu); + if (sched_feat(WA_WEIGHT) && target =3D=3D nr_cpumask_bits) target =3D wake_affine_weight(sd, p, this_cpu, prev_cpu, sync); =20 --=20 2.53.0 From nobody Thu Sep 24 20:04:09 2026 Received: from mail-wm2-f12.google.com (mail-wm2-f12.google.com [74.125.225.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4E47F4BA1F4 for ; Mon, 21 Sep 2026 15:23:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.225.140 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790004183; cv=none; b=GCZjGtHKh45GhVIueGZdeKEw6kcL5xHTZm4rDWeoHLD6PNcBqPVu6FY/Nw9rzNMjG0pkTm6WfkR1GI97J0EwaR54WHWsgogz6znh+KUW1b3RVeMZiWv9nLQ/1pgZ7n7Qz/ZPk7tG2uH7imA4qSsyM1nMMAUnbTW76MKSzOuZonY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790004183; c=relaxed/simple; bh=C6F+Fmk3tTihtf08pXapvACKxjH2ct8OvBSqTK4plhU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=KZltmqrmMPRzFNhAP7zoqJibZzu9lh+F6F1FZ0/rk9jCiQrbK7T/tuUIWF+5QimE+eBYMlf3sSqUiWRV79L2UTOhVjqCH5EHc/JW+hf4k2RyT89zG9IiCUgHxFGpoGlvmeO4GVle7W4Nr87+6FRkG/tGmh+JcJ2RTVdBl2F5z5s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=onx5Omux; arc=none smtp.client-ip=74.125.225.140 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="onx5Omux" Received: by mail-wm2-f12.google.com with SMTP id 5b1f17b1804b1-49cd5462b69so16802155e9.1 for ; Mon, 21 Sep 2026 08:23:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1790004180; x=1790608980; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=/uhUsAanGewnh2bM3AvBeAQ7knN/46rOtFLZpMDn/lo=; b=onx5OmuxhPuINRBgcBZp5x6O/H09XEJXtVTPwu13q5HEREoJwrnCZn5BQw7Yb1TGUK ofIzLE3tCc9x5LOHVAEXcRxSK7++vM/BXmCssDLOEfBa2uqs6esBJIe0mcGeESq44cr/ KUz+Iuk1bPqZT0sDgLRGzZAvqiU/VHKLk2diHjjFlGqYX6ufoRNSH+nffsGNPZWt6snf 0zrxQJwJHWFv/oneLSfMC2Cs4pNlSViPzvTfaTESkb7+inysmMHcfK3pxvMkDK5EngLv GUbgLXcS4E77jP8aQwqINkwCe/KLB/bopSjA5YIQe/+3EZ+Ftzr9KKEwQngllVcD0I7D aV2w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1790004180; x=1790608980; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=/uhUsAanGewnh2bM3AvBeAQ7knN/46rOtFLZpMDn/lo=; b=aEPh1ePharA5VqKi5vjn7iRL+zsVBhpwyCE+JHWFS6nBAnmebV2D9q2jxEwfu268Rf ubUltcz2dpwlQLgGMd2iu3bq3clIhGxiZIr1YIHCnQo6Kj7r/68m0yXJ9baI/1J1OXGB J1T6PjY41PGH4j79rttLqCOTGepTTXRFLjdF8dWr5T0YhOOkubtP+NHjGtOA4QQ8es7c NMaSQSHiEf6DXGrBg974aEPboI8dULmPcvxkXLzfYLHKGpvLOb1Z4N5VN//jgO8CWC4E /yX6yHRsp4SnosI/hC2FK4K5MMxsqMJ6e1jSxLHvk6OBSybdgnr7+KTkUKzqAodY3Qri AAqw== X-Forwarded-Encrypted: i=1; AKwUvBxMf1c0uOAZTJQpIsE9MWNgLkx/jrpO0DSdxUKfvRKZ+2jmBU3TLAsMj9kCVg0F6XYUZoloOeX+tgRA2oM=@vger.kernel.org X-Gm-Message-State: AFuF++kQ4bJQV1HtpddQv0vWS899JE6/zDAtyIqmnXVo4NDx+kL4mrUF tB8JZkKWX7cSYXMER7wRT1Py9QpBtirJzJQgA9F9mmwv0OIdLFpKP3Hpcz89R5ldBqA= X-Gm-Gg: AYBFou2HDISIsYVh2HEuduE3Z7klFs4pgmi7ygqPICo2osY9NA9VpYV1hnZtXXT0Vci 9uaqGRIKLsv6LozYZ7CL6N970EL98MU+bqyWvoqr/F422qoh7TDrxLWO5LDnihn2woTlP+1URSb 0lUf5aj4mKeJBPBIgl0n/3p1xctZKsN9awu8V1uvht/j+kUWS6SvxzdCQ2jdtO5jvbfoDNoZPwL F/E2bZ+kgG3PTw8Zja5lGIJP8yDGfxps4QA9mcyf9iBTDFrslyLIC1v1NjjZX0QxRDlOL8nvb6N 5ALXZWVDpOYkHbOCqc3EKDrkRewhglGXs27EiD8aAvvDO+UQ0S7cc449ZxCGT1Becw9MwB24iX4 +mUwUdUAsTHyG5S7vvs/ygcSIECacTujGoCIWxdhupIiFYhvQq+nY2vrHy/BdpLSmUXq5MfF6jt TCRqfS7rKZBb/7nUa4RKJuovCuINxT/01uJa+uVfbmGYbWTvEjz7PuSqaeWoIRQvpSJq1ycnorV GM53WAXo7Hnr38= X-Received: by 2002:a05:600c:6305:b0:49f:c2ef:cfb9 with SMTP id 5b1f17b1804b1-49fc56719a8mr123284685e9.4.1790004179612; Mon, 21 Sep 2026 08:22:59 -0700 (PDT) Received: from vingu-cube.home ([2a01:e0a:11f4:3020:df5:e9a4:98b5:65d1]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-49fcd8bf159sm223745675e9.10.2026.09.21.08.22.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Sep 2026 08:22:59 -0700 (PDT) From: Vincent Guittot To: mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, vschneid@redhat.com, kprateek.nayak@amd.com, linux-kernel@vger.kernel.org Cc: qyousef@layalina.io, Vincent Guittot Subject: [PATCH 8/8] sched/eevdf: Add min slice check when selecting CPU Date: Mon, 21 Sep 2026 17:22:38 +0200 Message-ID: <20260921152238.3804392-9-vincent.guittot@linaro.org> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260921152238.3804392-1-vincent.guittot@linaro.org> References: <20260921152238.3804392-1-vincent.guittot@linaro.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add a new level for selecting CPU when select_task_rq_fair() fails to find an idle CPU. This last level will compare the slice to select a CPU where the task could run 1st. This helps a waking task to select a CPU where a longer slice runs instead of one where a task with the same or shorter slice already runs. Signed-off-by: Vincent Guittot --- kernel/sched/fair.c | 63 ++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 62 insertions(+), 1 deletion(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 452da94289fe..5c9add3e853c 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -9023,6 +9023,59 @@ static inline bool asym_fits_cpu(unsigned long util, return true; } =20 +static int select_slice_cpu(struct task_struct *p, struct sched_domain *sd= , int target) +{ + unsigned long task_util, util_min, util_max; + int cpu, nr =3D INT_MAX; + u64 slice =3D p->se.slice; + struct cpumask *cpus; + + if (sched_feat(SIS_UTIL) && sd->shared) { + /* + * Same nr_idle_scan hint as select_idle_cpu(), nr only limits + * the scan when not preferring an idle core. + */ + nr =3D READ_ONCE(sd->shared->nr_idle_scan) + 1; + /* overloaded domain is unlikely to have idle cpu/core */ + if (nr =3D=3D 1) + return -1; + } + + cpus =3D this_cpu_cpumask_var_ptr(select_rq_mask); + cpumask_and(cpus, sched_domain_span(sd), p->cpus_ptr); + + if (sched_asym_cpucap_active()) { + task_util =3D task_util_est(p); + util_min =3D uclamp_eff_value(p, UCLAMP_MIN); + util_max =3D uclamp_eff_value(p, UCLAMP_MAX); + } + + /* Those CPUs have been tested not being idle and fiting */ + for_each_cpu_wrap(cpu, cpus, target) { + /* + * Stop when the nr_idle_scan is exhausted (mirrors + * select_idle_cpu() logic). + */ + if (--nr <=3D 0) + return -1; + + if (slice >=3D get_rq_min_slice(cpu_rq(cpu))) + continue; + + if (sched_asym_cpucap_active()) { + int fits =3D util_fits_cpu(task_util, util_min, util_max, cpu); + + /* Perfect fit: capacity satisfies util + uclamp */ + if (fits > 0) + return cpu; + } else { + return cpu; + } + } + + return -1; +} + /* * Try and locate an idle core/thread in the LLC cache domain. */ @@ -9117,7 +9170,10 @@ static int select_idle_sibling(struct task_struct *p= , int prev, int target) */ if (sd) { i =3D select_idle_capacity(p, sd, target); - return ((unsigned)i < nr_cpumask_bits) ? i : target; + if ((unsigned int)i < nr_cpumask_bits) + return i; + + goto select_slice; } } =20 @@ -9150,6 +9206,11 @@ static int select_idle_sibling(struct task_struct *p= , int prev, int target) if ((unsigned int)recent_used_cpu < nr_cpumask_bits) return recent_used_cpu; =20 +select_slice: + i =3D select_slice_cpu(p, sd, target); + if ((unsigned int)i < nr_cpumask_bits) + return i; + return target; } =20 --=20 2.53.0