From nobody Mon Dec 1 22:05:59 2025 Received: from mail-wr1-f48.google.com (mail-wr1-f48.google.com [209.85.221.48]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C79173016E1 for ; Mon, 1 Dec 2025 09:13:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.48 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764580403; cv=none; b=HmYK/sdXMYa6kA+w9zSJYw4QzhdpCxO4fWbrRvgraGxAGloUzRTf1T1AD9FATjH1EMKvWElw/qT7v0dnDqJIrFo1EuboWRh4eRHdhdgtdWe5lhk1lcglBvjh/k2HCsgQh+TY4UXJhPENdqZb1PTzuR9Dxg/tdNekFXZGP1pdUmo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764580403; c=relaxed/simple; bh=qiH6JApkt155+leNQvk4txPd9WgvnIHl5mspK/fIUvA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=YCfmrSUoh1/F43Zpu/bXxLhTOq0jvsOeFQa+cot3XHpi0tIjxT+cIrgY3vLdCUMJBkHeTLd8BZ0KQy8ZLYxXQkGfv5OjUv4OyW1nezf2eRXPZBK2k/ukHTqrtS01tuSJV6Jo8BHyHR3yF3NpSols0cQ9GG3kmVJF0MJO2Ei2BGc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=daetojZz; arc=none smtp.client-ip=209.85.221.48 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="daetojZz" Received: by mail-wr1-f48.google.com with SMTP id ffacd0b85a97d-42b38693c4dso1358998f8f.3 for ; Mon, 01 Dec 2025 01:13:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1764580400; x=1765185200; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=HuNRomfv0vCIa2D0+6on2ZVGZKQRa6gA3SVt7VHzktk=; b=daetojZzNyPIMoYKGGQB0rWIaAodZuaTgcTUHNiwESwjVymHhns8kFGkyymsz290qV i3tdGGPGQqAkfxtlVEaqVDAwCpHOlRlirWhRtrL1FRpkdGBBNF50L4ANlBiggP6sYarn sctzfEB2+JUu9b+jzwBhC1aUiSml9Ju0CHF409trN+1+cTDpGkY70WwlzR0hscgkBU/5 BWjgvfhjSFSO8RgMGlAVPb7ARifWSuEvU5SIkoAv03oDgRgOG1GZvxSKtzbyJU2bWADT Stf3vsG9352evCADzdlT5KEeqfvhqyFT8nwRWtKN4duiAp6buu/vKbx88aEC0N6iNDJ+ baNA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764580400; x=1765185200; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=HuNRomfv0vCIa2D0+6on2ZVGZKQRa6gA3SVt7VHzktk=; b=Ms5juhN6X5fJ/34gJQrEWs+FznDFP3fyokpoWsEkb57c2GU0Y7zc4EEydTwgrwqviN rhUyjGcJoMZpHrOaBXxjcGRhVmrrkrQf5QX/Dd5YDvq0vjPc2KFCwgHW6CzRGVN+/yKc Afm2TKVclrYqfeWWD1sgxmKWILmM2OUN2LApGQoMlTbeNXqDgZoKDTt/TM7d0z+Gbkzo pd+gIN2/lsxnrEmyyFAWjvHO7GerCkwOs4PMENm6aD83YvHksRsmtqaDvN3zeCSu3uqx sXgVq0Oy9YcQ1OXXnlozyJ81tM8uma7XJGxNqZ3kQwBstU4ULQBU4c0TxTFJF/w3b7L/ FfnA== X-Forwarded-Encrypted: i=1; AJvYcCUkMv3FndrOwme83xWnXB8v8Wx7ZHbtiuQM2n0FCQs5PCpk+Txv/UwiHwWhOWZ/0e3c2ScOwdbKzRfQcB8=@vger.kernel.org X-Gm-Message-State: AOJu0YwmaKECRZjEPzy3SrQMJ8jRany3FE5n1So1y9lLGmOPsJwfmLps 8TXjRTM0ZIdfWqUYIBLR4IRwG7w7yR3XehD+hCxHSkrz5RwqpPwDuysRWE6LMzU0eKw= X-Gm-Gg: ASbGnct283f3VBAQsFYRiJY4YPlaOEkb4uAq3B6Be8Ev/pZ52qVI/EW0vPA3UspR4WU WGaTz1BR8vOk+zzB0AsSMWtK9b1irPkbURVFWKr8riXjYZ/u1TvPiEcPrMMAmR3KeHBon00MTdL 8P3JjRDJO3XcSNG6cU/zSY8iQIfW3t1J/MeBMgkCyWPwJa1AELByeqn0SeJvqzAr3MI1QjbyQK5 sxFgT2dKPqxyfVaojz9j+U/isXrIqslmOWSeDk9A3PFux+0ERFsgb0oC0gkIN4NAP8nGdZzBt6+ A/iAuTojSyY3ZdIU/aGW8DxcJgfrKJ2DiJAnKy2RhuuxdZ8f6F5T0pKNGGhPVneGOERdxvFmMEa rt9bXi+HKClkFb5JM1lKd7oOS4pygBeoa//HOSF5ShFUFmDeahTSXFiY/NVriarrg9IynT3qzN5 mdXMNA0+sLHSC4owwCJ0c= X-Google-Smtp-Source: AGHT+IFg9Xc0ON3PE4VBuS79Bj3CwMQXsVYJVY6JvZMubox5KOFxDBvIFISn4TLDaLNsymYPVz5zzw== X-Received: by 2002:a05:6000:400e:b0:429:ba48:4d8 with SMTP id ffacd0b85a97d-42e0f22ccc9mr26615223f8f.25.1764580400146; Mon, 01 Dec 2025 01:13:20 -0800 (PST) Received: from vingu-cube.. ([2a01:e0a:f:6020:77d1:da59:4ce1:741c]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-42e1caa767dsm25344110f8f.38.2025.12.01.01.13.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 01 Dec 2025 01:13:18 -0800 (PST) From: Vincent Guittot To: mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, vschneid@redhat.com, linux-kernel@vger.kernel.org, pierre.gondois@arm.com, kprateek.nayak@amd.com Cc: qyousef@layalina.io, hongyan.xia2@arm.com, christian.loehle@arm.com, luis.machado@arm.com, Vincent Guittot Subject: [RFC PATCH 6/6 v7] sched/fair: Add EAS and idle cpu push trigger Date: Mon, 1 Dec 2025 10:13:08 +0100 Message-ID: <20251201091308.761711-7-vincent.guittot@linaro.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251201091308.761711-1-vincent.guittot@linaro.org> References: <20251201091308.761711-1-vincent.guittot@linaro.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" EAS is based on wakeup events to efficiently place tasks on the system, but there are cases where a task doesn't have wakeup events anymore or at a far too low pace. For such cases, we check if it's worht pushing hte task on another CPUs instead of putting it back in the enqueued list. Wake up events remain the main way to migrate tasks but we now detect situation where a task is stuck on a CPU by checking that its utilization is larger than the max available compute capacity (max cpu capacity or uclamp max setting) When the system becomes overutilized and some CPUs are idle, we try to push tasks instead of waiting periodic load balance. Signed-off-by: Vincent Guittot --- kernel/sched/fair.c | 65 +++++++++++++++++++++++++++++++++++++++++ kernel/sched/topology.c | 3 ++ 2 files changed, 68 insertions(+) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 9af8d0a61856..e9e1d0c05805 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -6990,6 +6990,7 @@ enqueue_task_fair(struct rq *rq, struct task_struct *= p, int flags) } =20 static void fair_remove_pushable_task(struct rq *rq, struct task_struct *p= ); + /* * Basically dequeue_task_fair(), except it can deal with dequeue_entity() * failing half-way through and resume the dequeue later. @@ -8499,8 +8500,72 @@ static inline bool sched_push_task_enabled(void) return static_branch_unlikely(&sched_push_task); } =20 +static inline bool task_stuck_on_cpu(struct task_struct *p, int cpu) +{ + unsigned long max_capa, util; + + max_capa =3D min(get_actual_cpu_capacity(cpu), + uclamp_eff_value(p, UCLAMP_MAX)); + util =3D max(task_util_est(p), task_runnable(p)); + + /* + * Return true only if the task might not sleep/wakeup because of a low + * compute capacity. Tasks, which wake up regularly, will be handled by + * feec(). + */ + return (util > max_capa); +} + +static inline bool sched_energy_push_task(struct task_struct *p, struct rq= *rq) +{ + if (!sched_energy_enabled()) + return false; + + if (is_rd_overutilized(rq->rd)) + return false; + + if (task_stuck_on_cpu(p, cpu_of(rq))) + return true; + + if (!task_fits_cpu(p, cpu_of(rq))) + return true; + + return false; +} + +static inline bool sched_idle_push_task(struct task_struct *p, struct rq *= rq) +{ + if (rq->nr_running =3D=3D 1) + return false; + + if (!is_rd_overutilized(rq->rd)) + return false; + + /* If there are idle cpus in the llc then try to push the task on it */ + if (test_idle_cores(cpu_of(rq))) + return true; + + return false; +} + + static bool fair_push_task(struct rq *rq, struct task_struct *p) { + if (!task_on_rq_queued(p)) + return false; + + if (p->se.sched_delayed) + return false; + + if (p->nr_cpus_allowed =3D=3D 1) + return false; + + if (sched_energy_push_task(p, rq)) + return true; + + if (sched_idle_push_task(p, rq)) + return true; + return false; } =20 diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c index cf643a5ddedd..5edf7b117ed9 100644 --- a/kernel/sched/topology.c +++ b/kernel/sched/topology.c @@ -391,10 +391,13 @@ static void sched_energy_set(bool has_eas) if (sched_debug()) pr_info("%s: stopping EAS\n", __func__); static_branch_disable_cpuslocked(&sched_energy_present); + static_branch_dec_cpuslocked(&sched_push_task); + } else if (has_eas && !sched_energy_enabled()) { } else if (has_eas && !static_branch_unlikely(&sched_energy_present)) { if (sched_debug()) pr_info("%s: starting EAS\n", __func__); static_branch_enable_cpuslocked(&sched_energy_present); + static_branch_inc_cpuslocked(&sched_push_task); } } =20 --=20 2.43.0