From nobody Tue Oct 7 15:58:55 2025 Received: from mail-wr1-f53.google.com (mail-wr1-f53.google.com [209.85.221.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D142E20296E for ; Tue, 8 Jul 2025 16:56:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.53 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1751993797; cv=none; b=ZiLzYUY/ScZURlSAV/cMxh/t0y756qyfJ7JAChF7NxaAhm2jd/Zoh3yBugD+N7/icYc4U2Go2gm4WW8qfkDSAWRyHE8AQYpquLWVpm1DQmqdqg+ZlWRE/6W2+C25V7uNIJNF1DoOw30CWDn4o1uR7OI0u+8WEkUYARq4e+kc5B0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1751993797; c=relaxed/simple; bh=AhbBIgHkLNVCNMm17JJi+tNpziBECUFPFNFEnQtPCi0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=kf57SAsZx8YLf0jUW9oo1okRMGNupVxS5utJ961jxH/kLbKG8XYQF3bk8Hlz7BnW9zsW11ePCN7DIJZ4ffUoYElsP2XxnPOcmopHAj2TPovmq1za9GldPxGItQiUsvflm1IacZDndYrHg6YThtDgXSEM+i5XgbbZfw9ZZsccXg8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=RmqkU9GJ; arc=none smtp.client-ip=209.85.221.53 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="RmqkU9GJ" Received: by mail-wr1-f53.google.com with SMTP id ffacd0b85a97d-3a51481a598so2628616f8f.3 for ; Tue, 08 Jul 2025 09:56:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1751993794; x=1752598594; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=4Hs2oYF3e3J4FN8zRTDauoaC2/4C3r7PLe1ATMwxGhY=; b=RmqkU9GJoeX8AjJbOHuI3JPL+kt4RkpakDzQMZmox+UPwNFohoB+bdATPfUeU1jmoa E8A1tmEm/58tism8trtIavlC8DVoynMs85NAmKQ7B62ujESYc9WBdt6bwPLDQYRySzaj KA6QqhR9DVdEJIELtqTJ0YTnvEq8/jGpgfjG+ZHSpOfHZH+KyfPHKOBPtH+DYyzezp5H P3gCeZosbSUts5vcZ8DzF152TVL20Ca9i+ULyrorGG0qDqSOlVUvoIviUQNprplBiDdT afPbQAnjoaKbU4n6/CzMDhRObJqdsige+AmjiyQ4wPppaLzqWR/o/o86EcKKwB5VOxvI gnqw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1751993794; x=1752598594; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=4Hs2oYF3e3J4FN8zRTDauoaC2/4C3r7PLe1ATMwxGhY=; b=aMFNb8fvEFEMEH7Clf1Ml0mUZZD9zjAvslXTfj+9UM12FCc0ub0A+jrTa6gAhlmoVe VqlT6cFnkvSW19G+iGfQQjBvOiiOvl5Wm/mHRbGJ+mYQ03JHncjJRnAUL7u8DdgLx5BD VKzDUWRsRfAuYknawAVjnUek3J0BnwAbDiuFZI6akTcaKmSMCArgeu6x26B6na43B6aY mvL9sHbc1C++D9M1uv1XjBr/Fl+tswnC0IGurBzdr7BVlnOaHRPK0Bsp2Q96hXuZaCTL e2Kp7jzud7bL6Z4Pby0Q2gl3MEb9c4I/PthOpEJHzb7UKOz1CGjUlrWulvPBKTpwx/TD EDqg== X-Forwarded-Encrypted: i=1; AJvYcCXkwh8f5fMQ6upBodPl5Ub4JJpv20TyDmu9RRPD1voXkwJwxZcFhZf74x6zqKiffkmOWF7Ak0x9Qob/Yxw=@vger.kernel.org X-Gm-Message-State: AOJu0YyGRai90RrXztwDpKJ9aAZsEA+s13pn0FzK4oi8n6P7PPJ+ndim /1bA/PRtSzM+p2yh3Timeb3291tfbtUj/8aNOWc6fUcLl8WyCPwQJBbDpoObnVqbg9c= X-Gm-Gg: ASbGncuCmrPpIQ4MJR5Z/4p7CAgowndHUIq7weDfNvLH08kLbV0C4F7ef4Dut4XuQ7M q9Bn3QuBJhCwjZKBHRMc8X6CvjZXTugbBLfIUIxV9h/bcLojS2H10yx6HBY+d+ZWotJJuMj5zO5 EqjIlDbVfQmhcRqniaf5noi0N8ZOBKW/fvo584ACz7EUHxs1lWPhJuqfxxL/baUNg0na75WCWBv uwVdJ5JQMUbrTPdw2p5tThUomkreS4xIAbtXsosO/Tfq7mGI+lfkImPiPc9bmbHDzEs/2Bz2LCp q0b23aqUmWr14fUn2s9azP2cy6voxad7uYP2GJllukRxfIrG1D3JsYrgCKNepULODk/Psxk= X-Google-Smtp-Source: AGHT+IGVR8h+iqDXFVtFVuUgLLbZaH5cTpE0x0yO/RQ3cy1MfYVihe/jtDd59Qwn59KARKeKcep5zA== X-Received: by 2002:a05:6000:4308:b0:3a6:e1bd:6102 with SMTP id ffacd0b85a97d-3b4970429fdmr12405783f8f.49.1751993794131; Tue, 08 Jul 2025 09:56:34 -0700 (PDT) Received: from vingu-cube.. ([2a01:e0a:f:6020:2a4a:c4af:2315:5f08]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-454cd3d7d51sm27295965e9.29.2025.07.08.09.56.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Jul 2025 09:56:33 -0700 (PDT) From: Vincent Guittot To: mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, vschneid@redhat.com, dhaval@gianis.ca, linux-kernel@vger.kernel.org Cc: Vincent Guittot Subject: [PATCH v3 1/6] sched/fair: Use protect_slice() instead of direct comparison Date: Tue, 8 Jul 2025 18:56:25 +0200 Message-ID: <20250708165630.1948751-2-vincent.guittot@linaro.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250708165630.1948751-1-vincent.guittot@linaro.org> References: <20250708165630.1948751-1-vincent.guittot@linaro.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Replace the test by the relevant protect_slice() function. Signed-off-by: Vincent Guittot Reviewed-by: Dhaval Giani (AMD) --- kernel/sched/fair.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 7e2963efe800..43712403ec98 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -1161,7 +1161,7 @@ static inline bool did_preempt_short(struct cfs_rq *c= fs_rq, struct sched_entity if (!sched_feat(PREEMPT_SHORT)) return false; =20 - if (curr->vlag =3D=3D curr->deadline) + if (protect_slice(curr)) return false; =20 return !entity_eligible(cfs_rq, curr); --=20 2.43.0 From nobody Tue Oct 7 15:58:55 2025 Received: from mail-wm1-f52.google.com (mail-wm1-f52.google.com [209.85.128.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C94D12E5439 for ; Tue, 8 Jul 2025 16:56:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.52 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1751993800; cv=none; b=kC5OzVoKHhOI8ZenihZVim8MevnB69PE1uz9jq9N6KWbkIV6yTGBX/S2qW68W9WXGKN/E0NUxtdqLiImnr0RHT6qET4i9I5OGO7GRz4mqp0MKrSCmOblEfoKvtcESfWOhzkNhmhQJ5mChL4HdAl0pG5UdY3g4XbJhnIHdycJw2k= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1751993800; c=relaxed/simple; bh=ydZRP6+llQn1XG0Wed/Oa5HzhIcOTAV7GgM8Xhyvv10=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=YtgupIR5aFe12aeqDZ1/AYzNB1GngNzMryWorrWB5ywM2dQ5qWAMbYW4JDxTfL9tpPyfbTLkkk08yPzcx5n24V63sMVFydqGFhhAuXSWL4O2s+67ngwewAfFnCuzcJ/UJsT1UkYlc66tudrkauIcIsRxo4FSMkZ1nT6n21ePBgo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=KGGbi5My; arc=none smtp.client-ip=209.85.128.52 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="KGGbi5My" Received: by mail-wm1-f52.google.com with SMTP id 5b1f17b1804b1-453066fad06so33051975e9.2 for ; Tue, 08 Jul 2025 09:56:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1751993797; x=1752598597; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=AkQp02knqPCKiukiiHCeRaV1SL9ZYAgdUSB+fSbfNIY=; b=KGGbi5My0ceUWYViUlK6m8kbO4tUbQxojY22lnE5EQYrrGy2fCuoD/TpaV67jfL2m+ AjsSWbKgLNmbOikil2AKmZleYqpWep+C9eEhGpk7bHFW2AEm333ucqEmh6Joiwxuu+L6 33EqcSfk6juOY3Mvv79TXzbPZj6CVVJL2lK8fi0xe+2iHjUbEt3xam6UqEhUj7uHAxtn PrsYy04v4F6rDHC+uuq3NchGU1LFPAEL//jkQlnjRw/2UxB6EUzLByHc2szqFZr8HvB6 X2TvSh0iAno1Hn6+ro63hwZeMhk1cl+ypyLhb6RoYoSpd+Olpqb9rbMjVzN3sPXnVUXL L8aw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1751993797; x=1752598597; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=AkQp02knqPCKiukiiHCeRaV1SL9ZYAgdUSB+fSbfNIY=; b=fufziMA/5NEmliiDMBhcAyOJ1JSRvat0iH+uvban+HSDRleOnVCHxEvwvNLVH8kOaj bAO5sKaRfxe/wsnkVFdWgzlLuEFRo0DoF1dtuUZ9t1Q3JhLqlrDfnh7bq1fsmZwQ0E/f eaVB3F/abYth3GcJDLmr63TyuL3XQ8OpY2y1e6UyBlF1p6LbLpoxt3YcW+5qzJTFWx4X tD8+tIyo50v0uyD4pigcr25T29nHi41PCSOSp+znZ7xV+T6akwrOHztdndOmTxiCug3h RjgW8lt3r+JUep13L+E6cbg/9Wn6Hbt00B1Tv/Bw+23fuKr+L0HGgeBX63mz/rdd2arW Zagg== X-Forwarded-Encrypted: i=1; AJvYcCV3cug+hQcVPMTki8+HoeRkAt/jqaOt5yvKgDRvL/gKNl3/WDoNeFsB1dNxtY+1evlpN7dLwnfQyelhph8=@vger.kernel.org X-Gm-Message-State: AOJu0YwjpdfqpjMw7QO1eFzrGhMCFYacNB/lUv4txEpAK9AWG23SM0S6 1wCY2/QP3ObMh5o5JNUvK3RTBzrlVsQgLlot4jOyNkZ7jEcFzLMldnYZuwn7WXL6aN0= X-Gm-Gg: ASbGncukhr6qeaabDJgQh64eiIM4MZIumYRgkRQpkK5WwA7Xma4iQ77E85IRJE5fUID ePaWe0F9KgqxP5xR8AVjVQspJG/uCXUWC8/DqnX0yfEjqWhYxBaMv53EfHNlHsOie7WR401CDns 9YJYMS6YS8khO2NHsiypLIpt+OJ/AP1jSSt2ScEM9ynkZNt65M4rcEMJvNKlxRxnsv8Kock2Q+g Rf2ZbyFU9Qoft89LABggnCNB7iERacK92Wge7YKXAEExt1RlhClr65ch2Zg5GetIz4zbS0pL+IC EyLbPncMuSCWAiLvqcYRQquiJ70MH0WK7LeUpTsdg1xZOWv0AQxiHz2E7Wcjl8XhSbnptWg= X-Google-Smtp-Source: AGHT+IElq5a9wfG2FKaD6LJBmCIfVXPss7CPTB2Oy9YVsJNsJezn70nwkPSvkFe5k+j5shzq9IiDZA== X-Received: by 2002:a05:600c:1c1b:b0:445:1984:2479 with SMTP id 5b1f17b1804b1-454d3882436mr2357545e9.5.1751993796936; Tue, 08 Jul 2025 09:56:36 -0700 (PDT) Received: from vingu-cube.. ([2a01:e0a:f:6020:2a4a:c4af:2315:5f08]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-454cd3d7d51sm27295965e9.29.2025.07.08.09.56.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Jul 2025 09:56:34 -0700 (PDT) From: Vincent Guittot To: mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, vschneid@redhat.com, dhaval@gianis.ca, linux-kernel@vger.kernel.org Cc: Vincent Guittot Subject: [PATCH v3 2/6] sched/fair: Fix NO_RUN_TO_PARITY case Date: Tue, 8 Jul 2025 18:56:26 +0200 Message-ID: <20250708165630.1948751-3-vincent.guittot@linaro.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250708165630.1948751-1-vincent.guittot@linaro.org> References: <20250708165630.1948751-1-vincent.guittot@linaro.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" EEVDF expects the scheduler to allocate a time quantum to the selected entity and then pick a new entity for next quantum. Although this notion of time quantum is not strictly doable in our case, we can ensure a minimum runtime for each task most of the time and pick a new entity after a minimum time has elapsed. Reuse the slice protection of run to parity to ensure such runtime quantum. Signed-off-by: Vincent Guittot --- include/linux/sched.h | 10 +++++++++- kernel/sched/fair.c | 30 +++++++++++++++++++----------- 2 files changed, 28 insertions(+), 12 deletions(-) diff --git a/include/linux/sched.h b/include/linux/sched.h index eec6b225e9d1..75579f2fb009 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -583,7 +583,15 @@ struct sched_entity { u64 sum_exec_runtime; u64 prev_sum_exec_runtime; u64 vruntime; - s64 vlag; + union { + /* + * When !@on_rq this field is vlag. + * When cfs_rq->curr =3D=3D se (which implies @on_rq) + * this field is vprot. See protect_slice(). + */ + s64 vlag; + u64 vprot; + }; u64 slice; =20 u64 nr_migrations; diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 43712403ec98..97cf99bb71d6 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -882,23 +882,34 @@ struct sched_entity *__pick_first_entity(struct cfs_r= q *cfs_rq) } =20 /* - * HACK, stash a copy of deadline at the point of pick in vlag, - * which isn't used until dequeue. + * Set the vruntime up to which an entity can run before looking + * for another entity to pick. + * In case of run to parity, we protect the entity up to its deadline. + * When run to parity is disabled, we give a minimum quantum to the running + * entity to ensure progress. */ static inline void set_protect_slice(struct sched_entity *se) { - se->vlag =3D se->deadline; + u64 quantum =3D se->slice; + + if (!sched_feat(RUN_TO_PARITY)) + quantum =3D min(quantum, normalized_sysctl_sched_base_slice); + + if (quantum !=3D se->slice) + se->vprot =3D min_vruntime(se->deadline, se->vruntime + calc_delta_fair(= quantum, se)); + else + se->vprot =3D se->deadline; } =20 static inline bool protect_slice(struct sched_entity *se) { - return se->vlag =3D=3D se->deadline; + return ((s64)(se->vprot - se->vruntime) > 0); } =20 static inline void cancel_protect_slice(struct sched_entity *se) { if (protect_slice(se)) - se->vlag =3D se->deadline + 1; + se->vprot =3D se->vruntime; } =20 /* @@ -937,7 +948,7 @@ static struct sched_entity *pick_eevdf(struct cfs_rq *c= fs_rq) if (curr && (!curr->on_rq || !entity_eligible(cfs_rq, curr))) curr =3D NULL; =20 - if (sched_feat(RUN_TO_PARITY) && curr && protect_slice(curr)) + if (curr && protect_slice(curr)) return curr; =20 /* Pick the leftmost entity if it's eligible */ @@ -1156,11 +1167,8 @@ static inline void update_curr_task(struct task_stru= ct *p, s64 delta_exec) cgroup_account_cputime(p, delta_exec); } =20 -static inline bool did_preempt_short(struct cfs_rq *cfs_rq, struct sched_e= ntity *curr) +static inline bool resched_next_quantum(struct cfs_rq *cfs_rq, struct sche= d_entity *curr) { - if (!sched_feat(PREEMPT_SHORT)) - return false; - if (protect_slice(curr)) return false; =20 @@ -1248,7 +1256,7 @@ static void update_curr(struct cfs_rq *cfs_rq) if (cfs_rq->nr_queued =3D=3D 1) return; =20 - if (resched || did_preempt_short(cfs_rq, curr)) { + if (resched || resched_next_quantum(cfs_rq, curr)) { resched_curr_lazy(rq); clear_buddies(cfs_rq, curr); } --=20 2.43.0 From nobody Tue Oct 7 15:58:55 2025 Received: from mail-wr1-f54.google.com (mail-wr1-f54.google.com [209.85.221.54]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7F7922E5B11 for ; Tue, 8 Jul 2025 16:56:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.54 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1751993801; cv=none; b=KCJGLFIiYY9uqfUKggE6G/kU33UtQ7Sp8FS3yf27B9qgoPbE8n6peTolUskZWgZHGStM7yM0NvndZdV/0Cv+HC+fcIyk+k+ne8KjLye9WzyZGRppail2fCa3sMlBu5h2ZkI/GhCvICmnEGej2sOKp8DNjI3n7EMmv9a06Y2ImoA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1751993801; c=relaxed/simple; bh=+hKCEhP4wYBwtiBfNHK4dWd4+CVQUfB5UoPPhvv3fqY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=P8RJhif9X8f5DyXWSLgRejjvKyRBnaVztipxOTWHYzrAm9fN5+Msbcibo1uzrSj8mhsUM7DbQLf5IvURl2DKGNkAHz31/a00Ct7/pnk4bvzrxPUkk3e33AaHVturq9Jp3DSNx0Y2fSiKsN3ABaVCyd5zP83k+CgvpmAHHtkp2RU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=nEJRKd5b; arc=none smtp.client-ip=209.85.221.54 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="nEJRKd5b" Received: by mail-wr1-f54.google.com with SMTP id ffacd0b85a97d-3a4f72cba73so66094f8f.1 for ; Tue, 08 Jul 2025 09:56:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1751993798; x=1752598598; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Qw0/sY1luj9u4dFIQwVq3jk5mAy8ce+dq4koRYOPL3I=; b=nEJRKd5b9060RWMf69cHOSzfZF6T5JrietJhhMsu2OMi3hSh1m9Z1F2XePAYmyNbab OFkDixBBGCf4V/Y2HGARIjeG+fAPAwKCuyytPLKUwFxllv7HShgBnPFzi8KBQmpZgmf2 9S92fUjlSOjNl0iM0na4H+hisgGNf03wuM0a1rieZ2uSTb+Xa4DYwg+ee74gSSYh2hWQ LXl7x25sT37Qi7+Q+fqIb8oPl93i1yk/CJx1O5PXH922OKwFRAlsjN6iBxDWt4Gt6FF4 L6G1YidgPwclkoJviEBY18URzAXb36Id1KYQ5wCx4yPSLZJLTMOrDO4jYotTgxcnDH7W Vlcg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1751993798; x=1752598598; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Qw0/sY1luj9u4dFIQwVq3jk5mAy8ce+dq4koRYOPL3I=; b=IC9S72GFTW1GCiUdT6ToICT0C7dY/tah8+aObPsxSJOmusjjBBFf7hndzZdyTJY7jX N8MAz2rOhSUGJRdIIDfrODkAHf8m2QPsv/hefIrTPxFLQBY8/shho6bnG8zUXWY8mnnv /vFtNcTAhECe2ELH99RGL1j8vStouI5thDQ/bAETaG7FAM/LRtCI6aL7MYoYP1pOqXSx dPXxAOtwTYCLvwZHASUq+byLsCC8NS2UgPQfFsw+ETy4VUVx+5m7Uy5HJU2YV21tRVt6 U9wNsROU8eiswJ2pxuHmMIwf53ZfGEJ0MSY1kP/XvgC37YPOoZQdw2oORZVMeJN+zpbj FDwg== X-Forwarded-Encrypted: i=1; AJvYcCX1yTauZlEbhw9ovdQhkBrfJJRAGV64sZAYeGxnXxHSCHdLAvjVOiZWFm2pR2HiT3nXICsGxcSZjvyPMIM=@vger.kernel.org X-Gm-Message-State: AOJu0YxU0hdoSzu2Nb5RqtS/DabUCsINn9sm8rUqW1HFc8xdAaS9uxC7 B9/mkLtlwiFrI+5xz25xPOQ7UHdF/gQNK3/AOsDDOz7eRCBNbyP/rFOEaRUWIhtiqNY= X-Gm-Gg: ASbGncunMmycJ6IyGLGQxP2oUtUwMlrvtGwvO2Uo5vsWLmJYU3JWvsYBpd5OCH7eP8o i8tSlkJGf1MMiP7MzaPzEvEipofTwKHuIEaPWuEa4e76ynbruOTb5pLtb3NSupbdebtUUwKPCiz 5VF8hZ71a53sCTltdqQsyDAd4BkGdV0oZPQMDTiIdXQcCXSrkxWBKhScjWumAGn98clZsvZU5sJ w14X3P4b3gPjtyzaH7FSMW3OILdihemJHV4VzeubtVjuHM0lsU+9k8qbXNkjCSMdHnr+RSlPOZA jFcAiGmuGggn41ANCOOsMyEzTYzsB5BGipPAJhZ7QLX91418GqKgVKepn1SqC5UZ1gs+ELI= X-Google-Smtp-Source: AGHT+IEhFOlmagskQM2r5RzgT7BHKcPGdBCsQxmqbsZNtHcOI+6pwTYyTcX41NSxWi6wI0OkFi9pAA== X-Received: by 2002:a05:6000:40e0:b0:3a4:cec5:b59c with SMTP id ffacd0b85a97d-3b5de034ac1mr3523393f8f.25.1751993797811; Tue, 08 Jul 2025 09:56:37 -0700 (PDT) Received: from vingu-cube.. ([2a01:e0a:f:6020:2a4a:c4af:2315:5f08]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-454cd3d7d51sm27295965e9.29.2025.07.08.09.56.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Jul 2025 09:56:37 -0700 (PDT) From: Vincent Guittot To: mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, vschneid@redhat.com, dhaval@gianis.ca, linux-kernel@vger.kernel.org Cc: Vincent Guittot Subject: [PATCH v3 3/6] sched/fair: Remove spurious shorter slice preemption Date: Tue, 8 Jul 2025 18:56:27 +0200 Message-ID: <20250708165630.1948751-4-vincent.guittot@linaro.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250708165630.1948751-1-vincent.guittot@linaro.org> References: <20250708165630.1948751-1-vincent.guittot@linaro.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Even if the waking task can preempt current, it might not be the one selected by pick_task_fair. Check that the waking task will be selected if we cancel the slice protection before doing so. Signed-off-by: Vincent Guittot --- kernel/sched/fair.c | 44 ++++++++++++++------------------------------ 1 file changed, 14 insertions(+), 30 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 97cf99bb71d6..7e82b357763a 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -931,7 +931,7 @@ static inline void cancel_protect_slice(struct sched_en= tity *se) * * Which allows tree pruning through eligibility. */ -static struct sched_entity *pick_eevdf(struct cfs_rq *cfs_rq) +static struct sched_entity *__pick_eevdf(struct cfs_rq *cfs_rq, bool prote= ct) { struct rb_node *node =3D cfs_rq->tasks_timeline.rb_root.rb_node; struct sched_entity *se =3D __pick_first_entity(cfs_rq); @@ -948,7 +948,7 @@ static struct sched_entity *pick_eevdf(struct cfs_rq *c= fs_rq) if (curr && (!curr->on_rq || !entity_eligible(cfs_rq, curr))) curr =3D NULL; =20 - if (curr && protect_slice(curr)) + if (curr && protect && protect_slice(curr)) return curr; =20 /* Pick the leftmost entity if it's eligible */ @@ -992,6 +992,11 @@ static struct sched_entity *pick_eevdf(struct cfs_rq *= cfs_rq) return best; } =20 +static struct sched_entity *pick_eevdf(struct cfs_rq *cfs_rq) +{ + return __pick_eevdf(cfs_rq, true); +} + struct sched_entity *__pick_last_entity(struct cfs_rq *cfs_rq) { struct rb_node *last =3D rb_last(&cfs_rq->tasks_timeline.rb_root); @@ -1175,27 +1180,6 @@ static inline bool resched_next_quantum(struct cfs_r= q *cfs_rq, struct sched_enti return !entity_eligible(cfs_rq, curr); } =20 -static inline bool do_preempt_short(struct cfs_rq *cfs_rq, - struct sched_entity *pse, struct sched_entity *se) -{ - if (!sched_feat(PREEMPT_SHORT)) - return false; - - if (pse->slice >=3D se->slice) - return false; - - if (!entity_eligible(cfs_rq, pse)) - return false; - - if (entity_before(pse, se)) - return true; - - if (!entity_eligible(cfs_rq, se)) - return true; - - return false; -} - /* * Used by other classes to account runtime. */ @@ -8666,6 +8650,7 @@ static void check_preempt_wakeup_fair(struct rq *rq, = struct task_struct *p, int struct sched_entity *se =3D &donor->se, *pse =3D &p->se; struct cfs_rq *cfs_rq =3D task_cfs_rq(donor); int cse_is_idle, pse_is_idle; + bool do_preempt_short =3D false; =20 if (unlikely(se =3D=3D pse)) return; @@ -8714,7 +8699,7 @@ static void check_preempt_wakeup_fair(struct rq *rq, = struct task_struct *p, int * When non-idle entity preempt an idle entity, * don't give idle entity slice protection. */ - cancel_protect_slice(se); + do_preempt_short =3D true; goto preempt; } =20 @@ -8732,22 +8717,21 @@ static void check_preempt_wakeup_fair(struct rq *rq= , struct task_struct *p, int /* * If @p has a shorter slice than current and @p is eligible, override * current's slice protection in order to allow preemption. - * - * Note that even if @p does not turn out to be the most eligible - * task at this moment, current's slice protection will be lost. */ - if (do_preempt_short(cfs_rq, pse, se)) - cancel_protect_slice(se); + do_preempt_short =3D sched_feat(PREEMPT_SHORT) && (pse->slice < se->slice= ); =20 /* * If @p has become the most eligible task, force preemption. */ - if (pick_eevdf(cfs_rq) =3D=3D pse) + if (__pick_eevdf(cfs_rq, !do_preempt_short) =3D=3D pse) goto preempt; =20 return; =20 preempt: + if (do_preempt_short) + cancel_protect_slice(se); + resched_curr_lazy(rq); } =20 --=20 2.43.0 From nobody Tue Oct 7 15:58:55 2025 Received: from mail-wm1-f50.google.com (mail-wm1-f50.google.com [209.85.128.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 037792E5B37 for ; Tue, 8 Jul 2025 16:56:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.50 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1751993803; cv=none; b=AaxQ1e7E4r/itWoQyEwZJl40zIOZwlCs6d2q7KCJDUh3evCEQChlvcbkY83s4s3UALP9c0p1XVMzbTeNvF6nEOcedrKrpY+pL4bozu+CcnWgAAoBhysIZXw82BjpySTZzukye7EJdtQ42qoSuWq8BSVJN2AIjIXBdHaHqDmg8cE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1751993803; c=relaxed/simple; bh=tGQfUzEblmv82RM1x1NgNBUIurEhO/o0ushNrrpnTBE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=LVLHlLRSSH+bpdpA4KYXbkdUS6xd8yMvTPUtoX8MFGpZQRy9IFHAtKNUFR1WGHzomUK/+15d1ren+IWH3TLG9/9UmECuYNrZG/YZWnePQy3pIGwf3mD9L9Iu18w65iaJu5n37RAEsaxx3aCcwWO3oLOHzIFBEA6JlsKsiZzzmWU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=DPbqLhaU; arc=none smtp.client-ip=209.85.128.50 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="DPbqLhaU" Received: by mail-wm1-f50.google.com with SMTP id 5b1f17b1804b1-451d3f72391so47552575e9.3 for ; Tue, 08 Jul 2025 09:56:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1751993800; x=1752598600; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=nv1g3oLhp9LClJpIDjWjq3gCYnWK358SMoHHVdyx1nk=; b=DPbqLhaUkTXnGJ4yL4pGHGvFEscRz6pN4P7Q93vuxCiLcxcfB037Jxu38r3q35v/op HGj6qnzPYA70o+8cV1i1iOLKMkr8iq5xaBuhlvX2AndyN8OGZ9stePBruhecJdvoQaEe QKUp9YWYiZUEVoh7lKNemE3E9DyRr+B81zono9PgLyqNg8WQQHyJ5Ndt7tKt8hEGwW2p 5AWaaZ4SjpHlTbBNReo36y3xRNo1EB/V+mqvsURXIv26Boh32GpAvuE280XgvySqK9Vg NMTCYGtUOLjBDA6YoEHbhKsv1Hdi6sWrCIGA/xOmhtcUMh0IYuONDHW1+1iJz0Ap1ZBK JvbA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1751993800; x=1752598600; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=nv1g3oLhp9LClJpIDjWjq3gCYnWK358SMoHHVdyx1nk=; b=RMyHMHBQ9+QhBE05vSXKfFgENzxGWCZ6H6RgLxinYKerThbQzPmgFTXs954dbFZcyB u2oTMAy7/84SwE0EE6f++yyORk3Pt/Gj7ZiSv48EkaeLsynzN9sVrKVbG0qS+DqZeFqz gQiDmebEUHagEdROxiZqiskyHqCsNxH1wterte+D7A0qeDnbPcWjgpzvgVaTKmQPchtD dKuCALgH9VnnMIW1x5sQeZ1x9zBQ7YiIEvE9nUom2fQBScV9bLPhW7awJOomGasaTvf7 Y2O1FxsLmrDJzeh9SltFf6JVGb1Q8yZeILylA/ubkQl/DP5N0geYYssqCFtGnnEC99NM +yCg== X-Forwarded-Encrypted: i=1; AJvYcCWoq4CrJ5CuYmYfgaKfBMZZbzf92LaoF6gbYSaCupjBZu5NhUffhVdPj5M7AZ04WcBB2nzVf6SyTsNAPQw=@vger.kernel.org X-Gm-Message-State: AOJu0YwJHX37RaGy3+E3qDMkRy1LuzTAep7NAfQsZV3WnGS3/FhUS6St +xCA/tTE/7L1U5Sp15DCOZsJ0ZrKSW93T+MnBOuYxAvIiFINsHWCt0ux8FIUbc09EIw= X-Gm-Gg: ASbGnctOvy+q9TWaDV74hI9Q9bh7WxAeH83s1NOSl8qXgpgyVYTbGeV+WPowW5fvAMY Io7W5xBzTKyB4kGsGg/mzu/ic8eMJUJ5XfVbg3giKBYcZNhUA9A819wjNUCrpVhWUzeoMX7VVAX gP/3o9xPnjvGRhPWRhHQIrU0JSbNuARmEwTdmbST4tcGJFy83Hsvd3RZ2veqCgC9ZLB9krkGPMt 9SV/TbYoAOlJ/WYtG32VtfFDgHRy2qNmvn2wP7ws7fnXlpGS16BSQjsXB4oDoO1tUN/Q2L989f9 dOJrkQPYlXSNylukYCXQX7POK8Iok7RKiDTrlKxyrzs9Lh1PtQSv1TZ2kXMm5Y29M4sWooI= X-Google-Smtp-Source: AGHT+IHOpyZU8Q+q2tLlijRQ0LZXkdlgUs+EvMoHKUMc8miI9KhZ/rO8YYQF6q9OCmCm3lQ3+BnUbg== X-Received: by 2002:a05:600c:8416:b0:450:d4a6:799d with SMTP id 5b1f17b1804b1-454b306b0c9mr151534335e9.7.1751993800277; Tue, 08 Jul 2025 09:56:40 -0700 (PDT) Received: from vingu-cube.. ([2a01:e0a:f:6020:2a4a:c4af:2315:5f08]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-454cd3d7d51sm27295965e9.29.2025.07.08.09.56.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Jul 2025 09:56:38 -0700 (PDT) From: Vincent Guittot To: mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, vschneid@redhat.com, dhaval@gianis.ca, linux-kernel@vger.kernel.org Cc: Vincent Guittot Subject: [PATCH v3 4/6] sched/fair: Limit run to parity to the min slice of enqueued entities Date: Tue, 8 Jul 2025 18:56:28 +0200 Message-ID: <20250708165630.1948751-5-vincent.guittot@linaro.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250708165630.1948751-1-vincent.guittot@linaro.org> References: <20250708165630.1948751-1-vincent.guittot@linaro.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Run to parity ensures that current will get a chance to run its full slice in one go but this can create large latency and/or lag for entities with shorter slice that have exhausted their previous slice and wait to run their next slice. Clamp the run to parity to the shortest slice of all enqueued entities. Signed-off-by: Vincent Guittot --- kernel/sched/fair.c | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 7e82b357763a..85238f2e026a 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -884,16 +884,20 @@ struct sched_entity *__pick_first_entity(struct cfs_r= q *cfs_rq) /* * Set the vruntime up to which an entity can run before looking * for another entity to pick. - * In case of run to parity, we protect the entity up to its deadline. + * In case of run to parity, we use the shortest slice of the enqueued + * entities to set the protected period. * When run to parity is disabled, we give a minimum quantum to the running * entity to ensure progress. */ static inline void set_protect_slice(struct sched_entity *se) { - u64 quantum =3D se->slice; + u64 quantum; =20 - if (!sched_feat(RUN_TO_PARITY)) - quantum =3D min(quantum, normalized_sysctl_sched_base_slice); + if (sched_feat(RUN_TO_PARITY)) + quantum =3D cfs_rq_min_slice(cfs_rq_of(se)); + else + quantum =3D normalized_sysctl_sched_base_slice; + quantum =3D min(quantum, se->slice); =20 if (quantum !=3D se->slice) se->vprot =3D min_vruntime(se->deadline, se->vruntime + calc_delta_fair(= quantum, se)); --=20 2.43.0 From nobody Tue Oct 7 15:58:55 2025 Received: from mail-wm1-f46.google.com (mail-wm1-f46.google.com [209.85.128.46]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C0AC22E612C for ; Tue, 8 Jul 2025 16:56:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.46 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1751993805; cv=none; b=FN2VKGMimuWn2wM4/+vnQDMV//DWhXamCmJxSp+ANFzs6iD1eyZmpie90aOmHm7JWrNkdNDpQUYuQo+PfmJO2xNeV4Y61Rjj+Wx3LaHMSss1t7jGI7tuE3/zOxTOe2mtSP+M9THzLNDPED9/aIztSEYW5fB4lLOzmBJqevszcv0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1751993805; c=relaxed/simple; bh=ZXF3oCG7ibV36hVD1H+76FIfM4kzG+s2hW4qeQQEF+k=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=tMhdXIaiXYlX8kKAPYXnyDRDGYd0HcExGUqjQy8TNxjOIVlTKLvJB95C+O6cGIi5tIiFQnt1nn7LRlr7lYjMvr7g+vULQw1zCD9UGzyAtCiAo/Kfgakgps2vMwAuBSd72d0ttuhMpkORb+L+Dgbtc48IZk3MzEjlZ2WBKUnJGfA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=ZA035RHZ; arc=none smtp.client-ip=209.85.128.46 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="ZA035RHZ" Received: by mail-wm1-f46.google.com with SMTP id 5b1f17b1804b1-451e2f0d9c2so823985e9.1 for ; Tue, 08 Jul 2025 09:56:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1751993802; x=1752598602; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=mgrc0RZh3NEys+vWGop1P04H8K+3Thfdpsx94SuLWJg=; b=ZA035RHZ/oet0m6V/gjiKODZq9I7p1jHJk3Pzdw/iSci+zSXgj7pNYyRSJ6y5YSnnI +5f0tOxn1Go/S0dE/l0ApLJBUmwAAz436AyOwMOFBKthehNz1ZVoMbvrsx1KOK3nGsZ6 yYewfpNLxtbs+uMZ09JRtIqemlVSzwqdYhAeJwAMlKpTJ0fW6VrOqVYG793jmiXj0Xyw NE8x4nz3rmwDtTr9cH03CoE7CjeAHHH/Dvol7gS+cz7tY3pfbpCqSBq0iYv24zsNDoKm 0PzYZ6nc1BqMaE4CpG834Otr2MT1lDX8e37wJylVh7/jrCJe0OqdjdV29+vmvExJ18R2 bulw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1751993802; x=1752598602; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=mgrc0RZh3NEys+vWGop1P04H8K+3Thfdpsx94SuLWJg=; b=i9N8CyuaJw4mI+f1SqWxCfAT4StdQyK8tNTVRr2TKVyYGF3Al7JcFXEhWa4/KFimA6 TE66N2TkFwPM4FxNKbmrgg9vlRkD+VIhnzClfU1HSGJdyoeGxnm3nRN1mtrd8Q770LEJ l19h3TE1zUc1fWhCej3KBVo9GOaluMEviV3HeOlYcC9yRCQCE96FrJ7X2qpFdXUNqBEu wQWw2DOwgegHX/7R9OB226+6Dq0c965RPZDU+ZkB1h9k3vyUOteJx507YZPsHXCO/Jz9 4B0M+t20akA8oiVs4WOMgHUemQbV2ebJBWx01yBWQwpNXkwQX9YkEb0qihleKC5PunQU 6GIQ== X-Forwarded-Encrypted: i=1; AJvYcCUQtWMBpQyRo3Vze+BUTw6jtqM4O+AXuK+nd3XOOCgAk52D+L8TkPfq8huuFfpKgE3A+SSQxHbCJTDPo8Y=@vger.kernel.org X-Gm-Message-State: AOJu0YxSK9B3SeT6Z7nzyWVwIBZC0QibkgMXI/c7wVrshn9f1IKpMBGz JomFbS1Qtq281/G7mOfXK0HUEOKiiOSgYezUOTdByGSIhG3U5DmnS6YkY2QJe+4QXNs= X-Gm-Gg: ASbGncueRgyZeIfQeZb8AGiCIehWtWxoeUHLAqcF9d0gDNLA8lJmG0Coj6T9ibfOyhd Uel3STttk7R8XBrKd9DctjnOwkOE7hnwbvre7meJltl6ePX1gkzIKUiNUsI55VC3hBRa8FZhB08 v8ztMyxV6FP1lcf0rYoEskS0wamdPaZ/9Bn/5j7RYBFae5WvItPVUiwsxF8zuTWyeuSQCZCBwQW DWMyJsMMDVQBzajH7QElf5GUZMmZDAEwBNtnCvB7oYxFF0jXRLyx8FuqhcGRo4Yis/qTKBt6wYm op30PM5X4ouNjIR1YLXf6zlClSaRakvCOoZ/RYO6Ot/InQm51sJgLe8neNGVOKJnV+lbGyE= X-Google-Smtp-Source: AGHT+IFevQFiPprYHsFZNqRJ6pLWR8/V8YHEPiMUzoDJWyI/jJwOp4nt8xCWiTzyX31ZVIoI2QyGAQ== X-Received: by 2002:a05:600c:548d:b0:453:ec2:c7b2 with SMTP id 5b1f17b1804b1-454cd85dd18mr33162365e9.7.1751993802014; Tue, 08 Jul 2025 09:56:42 -0700 (PDT) Received: from vingu-cube.. ([2a01:e0a:f:6020:2a4a:c4af:2315:5f08]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-454cd3d7d51sm27295965e9.29.2025.07.08.09.56.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Jul 2025 09:56:40 -0700 (PDT) From: Vincent Guittot To: mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, vschneid@redhat.com, dhaval@gianis.ca, linux-kernel@vger.kernel.org Cc: Vincent Guittot Subject: [PATCH v3 5/6] sched/fair: Fix entity's lag with run to parity Date: Tue, 8 Jul 2025 18:56:29 +0200 Message-ID: <20250708165630.1948751-6-vincent.guittot@linaro.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250708165630.1948751-1-vincent.guittot@linaro.org> References: <20250708165630.1948751-1-vincent.guittot@linaro.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When an entity is enqueued without preempting current, we must ensure that the slice protection is updated to take into account the slice duration of the newly enqueued task so that its lag will not exceed its slice (+ tick). Signed-off-by: Vincent Guittot --- kernel/sched/fair.c | 16 +++++++++++++--- 1 file changed, 13 insertions(+), 3 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 85238f2e026a..d815609526d5 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -889,12 +889,12 @@ struct sched_entity *__pick_first_entity(struct cfs_r= q *cfs_rq) * When run to parity is disabled, we give a minimum quantum to the running * entity to ensure progress. */ -static inline void set_protect_slice(struct sched_entity *se) +static inline void set_protect_slice(struct cfs_rq *cfs_rq, struct sched_e= ntity *se) { u64 quantum; =20 if (sched_feat(RUN_TO_PARITY)) - quantum =3D cfs_rq_min_slice(cfs_rq_of(se)); + quantum =3D cfs_rq_min_slice(cfs_rq); else quantum =3D normalized_sysctl_sched_base_slice; quantum =3D min(quantum, se->slice); @@ -905,6 +905,13 @@ static inline void set_protect_slice(struct sched_enti= ty *se) se->vprot =3D se->deadline; } =20 +static inline void update_protect_slice(struct cfs_rq *cfs_rq, struct sche= d_entity *se) +{ + u64 quantum =3D cfs_rq_min_slice(cfs_rq); + + se->vprot =3D min_vruntime(se->vprot, se->vruntime + calc_delta_fair(quan= tum, se)); +} + static inline bool protect_slice(struct sched_entity *se) { return ((s64)(se->vprot - se->vruntime) > 0); @@ -5468,7 +5475,7 @@ set_next_entity(struct cfs_rq *cfs_rq, struct sched_e= ntity *se) __dequeue_entity(cfs_rq, se); update_load_avg(cfs_rq, se, UPDATE_TG); =20 - set_protect_slice(se); + set_protect_slice(cfs_rq, se); } =20 update_stats_curr_start(cfs_rq, se); @@ -8730,6 +8737,9 @@ static void check_preempt_wakeup_fair(struct rq *rq, = struct task_struct *p, int if (__pick_eevdf(cfs_rq, !do_preempt_short) =3D=3D pse) goto preempt; =20 + if (sched_feat(RUN_TO_PARITY) && do_preempt_short) + update_protect_slice(cfs_rq, se); + return; =20 preempt: --=20 2.43.0 From nobody Tue Oct 7 15:58:55 2025 Received: from mail-wr1-f48.google.com (mail-wr1-f48.google.com [209.85.221.48]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9B50C2E62B8 for ; Tue, 8 Jul 2025 16:56:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.48 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1751993806; cv=none; b=s4Jp+9HE5+29UGnK39+62esmlh/j/bk19UX7hq2W8kABCRrr93iZt8hpgxfq2spdTO4ksIZu92k8Fjv57roO1norqTwKjDmx3dd3V8l5f9Sdk3ujG88rYw4AbQ7M30YNoTRMa1OxMRenbHUeM8YAyXxSQ5BvxQXKlbbBZn3FzOw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1751993806; c=relaxed/simple; bh=YDHzCZD1O4x6a9V2ciIeVZsYJg4YFiL4AoSP+V1JNEE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=X0qxtsZrBaSPyKyCldLJ6ZyWFDwDlpGKzOXmU83eiOqW3RD34ETL87eZrD6QSQjjWu1ub7/Zx2b045pFIKeeRxeNEnoJxAYkLhc3XtwwYzKE6eh4jOUN6XXqFnNbjwv1yLzQDoQXgBYw9O9QxVSvcTjPiZHEZwXzAYdcRQBMoIY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=Ezyrr3EB; arc=none smtp.client-ip=209.85.221.48 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="Ezyrr3EB" Received: by mail-wr1-f48.google.com with SMTP id ffacd0b85a97d-3a6d1369d4eso2755299f8f.2 for ; Tue, 08 Jul 2025 09:56:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1751993803; x=1752598603; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=zZl7Fer6pxxctYiak9nxWwOWdwOt4/BaPnKvvUOawZk=; b=Ezyrr3EBFE1UlkT2/wnYG9tpsOAJg+2eO5g4x49D7l60H5KSniHrkXsfPVdwwivVRY F2qSCTvSqkj52but6QYw+373lthDF0h6MQV+CZM0GIaP8V5TxLl9zS1aX65t8IN60SFA /ClQEi31VgKPlNutyBreUu2CyY6qBUHsTI3Uc+ewvqvWbEO2aBzc2g2Sil2NPzdtmTAF cI2V/SyJM6foFvVJUI6yKydMLYrVM2q1wFRmG9AqPH8wVzYJ2PfdxqYJTxnK2g34v+D5 obmKudHTUHgqZViW+11Rz+bRgSbKdZYDlTMJRmo18tkOKD7YxEFfAZFyLOun51dwERI7 pGDw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1751993803; x=1752598603; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=zZl7Fer6pxxctYiak9nxWwOWdwOt4/BaPnKvvUOawZk=; b=QY222J5XZ7zcqljXzryrBJY/prKzoQVPZ4osMJ5csRZIPBI9yawBoaKooL2YcfMIg+ pwfTzI2oeDqB/Hqz70q/8cAZU/4YCQu8qts/dqGXJbiYsIiqbOzYqRS3dzcHB9E+CuwM d631FEhjf2/StwHBcRtA1F3hcM5OXqRDTd8aYF1K116Ug/g0wj6qN+CO+cqXecfVTh09 13Llyc/i/Hj4RkLwQOf8U1DRoOhJImMFjONQAXOvOZCPYT4PMjiSyDyOX/wpnC2SI7Zm zOVtOMEJF1bMiyyEhVe0H35YC1PV51bBYb5rEKkW5JP7O8s7tvSrSAuxDUU6NoqruRGS mdLA== X-Forwarded-Encrypted: i=1; AJvYcCXH56Qaf+1khGTmsQeVnBNvTmmyi6aMBay4/aUyAPkfub3IPOHDmOHJxoDSI/yC3EQS0dFf5oNqph08BUs=@vger.kernel.org X-Gm-Message-State: AOJu0YxCoQIw2yx4Hrl5PCksLvGQm+FlrBguMVPGLcMfa6rT/cHz51Qr qSUAWsLoHw7Cg7GMWHkooGfWIpUhMogQ5SFCAF6FsiV/Gfy+/wDbcERfUsdHExI60gc= X-Gm-Gg: ASbGncsH75+rHr3UA6HdIq/+yMyp1SymHtoEYtdusyj5leqF0/tA0QsKZYGgCgUKh9a JXGjq7qRZU4jLdjrU1rFzf6JLsIdvSr4lYXgB8KCUNvCOedcrl41kS3dDU+ZEfPe8pCX8VMUUez fBrBDdMhQsgQlUn84hQx7/ynKm4LEnUll3vGl5rSOliftVHuaHEKbQpiUG4e1KuUDi//fl3Zi06 F34cC/NvpDr2kvVT6wc6/zNRQyrOwR79adWu3EAPw52P+0ZvazxKe1agIQ3sOtuhemDyt/6c3DP +h5YtkSM7Ft8Y9bNDxMGU39Canc1SOgE+neRuAu+8Dq4IiuUme8+MGh6xgNj1nRpYPZ7GDUNnvd gY9yKag== X-Google-Smtp-Source: AGHT+IG/A6kRJu2SjEN1uKoSXT92i0pOXWJrm/dgz/mCuQn8JiG5arI1A7md/gXc7lQ/lI1lm2ZAJg== X-Received: by 2002:a5d:5e88:0:b0:3a6:f2da:7fe5 with SMTP id ffacd0b85a97d-3b5ddea60d5mr3525069f8f.55.1751993803000; Tue, 08 Jul 2025 09:56:43 -0700 (PDT) Received: from vingu-cube.. ([2a01:e0a:f:6020:2a4a:c4af:2315:5f08]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-454cd3d7d51sm27295965e9.29.2025.07.08.09.56.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Jul 2025 09:56:42 -0700 (PDT) From: Vincent Guittot To: mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, vschneid@redhat.com, dhaval@gianis.ca, linux-kernel@vger.kernel.org Cc: Vincent Guittot Subject: [PATCH v3 6/6] sched/fair: Always trigger resched at the end of a protected period Date: Tue, 8 Jul 2025 18:56:30 +0200 Message-ID: <20250708165630.1948751-7-vincent.guittot@linaro.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250708165630.1948751-1-vincent.guittot@linaro.org> References: <20250708165630.1948751-1-vincent.guittot@linaro.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Always trigger a resched after a protected period even if the entity is still eligible. It can happen that an entity remains eligible at the end of the protected period but must let an entity with a shorter slice to run in order to keep its lag shorter than slice. This is particulalry true with run to parity which tries to maximize the lag. Signed-off-by: Vincent Guittot --- kernel/sched/fair.c | 10 +--------- 1 file changed, 1 insertion(+), 9 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index d815609526d5..fbe969adf287 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -1183,14 +1183,6 @@ static inline void update_curr_task(struct task_stru= ct *p, s64 delta_exec) cgroup_account_cputime(p, delta_exec); } =20 -static inline bool resched_next_quantum(struct cfs_rq *cfs_rq, struct sche= d_entity *curr) -{ - if (protect_slice(curr)) - return false; - - return !entity_eligible(cfs_rq, curr); -} - /* * Used by other classes to account runtime. */ @@ -1251,7 +1243,7 @@ static void update_curr(struct cfs_rq *cfs_rq) if (cfs_rq->nr_queued =3D=3D 1) return; =20 - if (resched || resched_next_quantum(cfs_rq, curr)) { + if (resched || !protect_slice(curr)) { resched_curr_lazy(rq); clear_buddies(cfs_rq, curr); } --=20 2.43.0